2020-08-07T14:38:46.120 INFO:root:teuthology version: 1.0.0-9643473 2020-08-07T14:38:46.123 DEBUG:teuthology.run:Teuthology command: teuthology --lock --description rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/on msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/radosbench-high-concurrency} --name shuzhenyi-2020-08-07_08:55:00-rados:thrash-wip-shuzhenyi-testing-2020-08-06-1028-distro-basic-smithi --archive /home/teuthworker/archive/shuzhenyi-2020-08-07_08:55:00-rados:thrash-wip-shuzhenyi-testing-2020-08-06-1028-distro-basic-smithi/5306571 --block --owner scheduled_shuzhenyi@teuthology --verbose -- /tmp/teuthology-worker.kbdg_hs_.tmp 2020-08-07T14:38:46.244 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-08-07T14:38:46.395 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/shuzhenyi-2020-08-07_08:55:00-rados:thrash-wip-shuzhenyi-testing-2020-08-06-1028-distro-basic-smithi/5306571 branch: wip-shuzhenyi-testing-2020-08-06-1028 description: rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/on msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/radosbench-high-concurrency} email: null first_in_suite: false job_id: '5306571' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: shuzhenyi-2020-08-07_08:55:00-rados:thrash-wip-shuzhenyi-testing-2020-08-06-1028-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 30 os_type: rhel os_version: '8.1' overrides: admin_socket: branch: wip-shuzhenyi-testing-2020-08-06-1028 ceph: conf: client.0: debug ms: 1 debug objecter: 20 debug rados: 20 global: mon client directed command retry: 5 ms bind msgr1: false ms bind msgr2: true ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: osd ms inject internal delays: 0.002 ms inject socket failures: 2500 ms type: async osd_async_recovery_min_cost: 1 osd_max_pg_log_entries: 2 osd_min_pg_log_entries: 1 osd_object_clean_region_max_num_intervals: 1000 osd_pg_log_trim_min: 0 osd_pool_default_min_size: 2 osd_pool_default_size: 3 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug reject backfill probability: 0.3 osd debug verify cached snaps: true osd debug verify missing on start: true osd delete sleep: 1 osd failsafe full ratio: 0.95 osd max backfills: 3 osd max markdown count: 1000 osd memory target: 1610612736 osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 fs: xfs log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request sha1: e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: sha1: e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 min_in: 4 workunit: branch: wip-shuzhenyi-testing-2020-08-06-1028 sha1: e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d owner: scheduled_shuzhenyi@teuthology priority: 50 repo: git://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b sha1: e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d sleep_before_teardown: 0 suite: rados:thrash suite_branch: wip-shuzhenyi-testing-2020-08-06-1028 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_wip-shuzhenyi-testing-2020-08-06-1028/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d tasks: - install: null - ceph: conf: osd: debug monc: 20 - thrashosds: chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 timeout: 1200 - full_sequential: - radosbench: clients: - client.0 concurrency: 128 size: 8192 time: 90 - radosbench: clients: - client.0 concurrency: 128 size: 8192 time: 90 - radosbench: clients: - client.0 concurrency: 128 size: 8192 time: 90 - radosbench: clients: - client.0 concurrency: 128 size: 8192 time: 90 - radosbench: clients: - client.0 concurrency: 128 size: 8192 time: 90 - radosbench: clients: - client.0 concurrency: 128 size: 8192 time: 90 - radosbench: clients: - client.0 concurrency: 128 size: 8192 time: 90 - radosbench: clients: - client.0 concurrency: 128 size: 8192 time: 90 teuthology_branch: master timestamp: 2020-08-07_08:55:00 tube: smithi user: shuzhenyi verbose: true worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.1880 2020-08-07T14:38:46.396 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_wip-shuzhenyi-testing-2020-08-06-1028/qa; will attempt to use it 2020-08-07T14:38:46.397 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_wip-shuzhenyi-testing-2020-08-06-1028/qa/tasks 2020-08-07T14:38:46.397 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-08-07T14:38:46.406 INFO:teuthology.task.internal:Checking packages... 2020-08-07T14:38:46.482 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'basic' and ceph hash 'e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d' 2020-08-07T14:38:46.483 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-08-07T14:38:46.484 INFO:teuthology.packaging:ref: None 2020-08-07T14:38:46.484 INFO:teuthology.packaging:tag: None 2020-08-07T14:38:46.484 INFO:teuthology.packaging:branch: wip-shuzhenyi-testing-2020-08-06-1028 2020-08-07T14:38:46.485 INFO:teuthology.packaging:sha1: e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d 2020-08-07T14:38:46.485 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-shuzhenyi-testing-2020-08-06-1028 2020-08-07T14:38:46.704 INFO:teuthology.task.internal:Found packages for ceph version 16.0.0-3802.ge19d2f2a442 2020-08-07T14:38:46.705 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-08-07T14:38:46.727 INFO:teuthology.task.internal:no buildpackages task found 2020-08-07T14:38:46.727 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-08-07T14:38:46.741 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-08-07T14:38:46.750 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-08-07T14:38:46.893 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:38:56.923 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:39:06.951 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:39:16.980 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-08-07T14:39:27.015 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-08-07T14:39:37.035 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-08-07T14:39:47.076 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:39:57.095 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:40:07.116 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:40:17.146 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:40:27.169 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:40:37.190 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:40:47.244 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:40:57.269 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:41:07.292 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:41:17.315 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:41:27.341 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-08-07T14:41:37.356 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-08-07T14:41:47.384 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-08-07T14:41:57.413 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:42:07.440 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:42:17.458 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:42:27.483 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:42:37.512 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:42:47.535 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:42:57.556 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:43:07.581 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:43:17.597 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:43:27.621 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:43:37.663 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:43:47.691 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:43:57.719 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:44:07.739 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:44:17.756 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:44:27.774 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:44:37.793 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:44:47.827 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:44:57.850 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-08-07T14:45:07.870 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-08-07T14:45:17.889 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-08-07T14:45:27.908 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-08-07T14:45:37.929 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:45:47.952 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:45:57.994 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:46:08.017 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:46:18.040 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-08-07T14:46:28.088 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:46:38.114 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:46:48.131 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:46:58.158 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:47:08.204 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:47:18.226 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:47:28.259 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:47:38.286 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:47:48.310 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:47:58.328 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-08-07T14:48:08.360 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:48:18.396 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-08-07T14:48:28.419 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-08-07T14:48:38.447 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-08-07T14:48:48.459 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:48:58.524 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:49:08.551 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:49:18.586 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:49:28.617 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:49:38.644 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-08-07T14:49:48.665 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-08-07T14:49:58.684 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-08-07T14:50:08.698 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-08-07T14:50:18.720 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-08-07T14:50:28.739 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-08-07T14:50:38.761 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-08-07T14:50:48.793 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-08-07T14:50:58.835 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-08-07T14:51:08.862 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-08-07T14:51:18.888 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-08-07T14:51:28.911 DEBUG:teuthology.lock.ops:lock_many request: {'locked_by': 'scheduled_shuzhenyi@teuthology', 'count': 2, 'machine_type': 'smithi', 'description': '/home/teuthworker/archive/shuzhenyi-2020-08-07_08:55:00-rados:thrash-wip-shuzhenyi-testing-2020-08-06-1028-distro-basic-smithi/5306571'} 2020-08-07T14:51:28.940 DEBUG:teuthology.lock.ops:locked ubuntu@smithi074.front.sepia.ceph.com, ubuntu@smithi096.front.sepia.ceph.com 2020-08-07T14:51:29.022 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa8022977b8>, signals=[15]) 2020-08-07T14:51:29.282 INFO:teuthology.provision.fog.smithi074:Scheduling deploy of rhel 8.1 2020-08-07T14:51:29.286 INFO:teuthology.provision.fog.smithi096:Scheduling deploy of rhel 8.1 2020-08-07T14:51:29.637 INFO:teuthology.orchestra.console:Power off smithi074 2020-08-07T14:51:29.639 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi074.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-08-07T14:51:29.681 INFO:teuthology.orchestra.console:Power off smithi096 2020-08-07T14:51:29.683 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-08-07T14:51:29.698 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi074.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-08-07T14:51:29.716 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-08-07T14:51:29.836 INFO:teuthology.orchestra.console:Power off for smithi074 completed 2020-08-07T14:51:29.846 INFO:teuthology.orchestra.console:Power off for smithi096 completed 2020-08-07T14:51:29.938 INFO:teuthology.orchestra.console:Power on smithi074 2020-08-07T14:51:29.939 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi074.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-08-07T14:51:29.958 INFO:teuthology.orchestra.console:Power on smithi096 2020-08-07T14:51:29.960 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-08-07T14:51:29.981 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi074.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-08-07T14:51:30.001 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-08-07T14:51:34.022 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi074.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-08-07T14:51:34.035 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-08-07T14:51:34.238 INFO:teuthology.orchestra.console:Power on for smithi074 completed 2020-08-07T14:51:34.254 INFO:teuthology.orchestra.console:Power on for smithi096 completed 2020-08-07T14:51:34.341 INFO:teuthology.provision.fog.smithi074:Waiting for deploy to finish 2020-08-07T14:51:34.360 INFO:teuthology.provision.fog.smithi096:Waiting for deploy to finish 2020-08-07T14:54:05.674 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-07T14:54:05.708 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi074.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-07T14:54:14.776 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi074.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-07T14:54:14.778 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-07T14:54:23.871 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi074.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-07T14:54:23.881 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-07T14:54:32.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi074.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-07T14:54:33.009 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-07T14:54:42.044 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi074.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-07T14:54:42.105 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-07T14:54:51.160 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi074.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-07T14:54:51.186 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-07T14:55:00.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi074.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-07T14:55:00.246 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-07T14:55:09.304 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi074.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-07T14:55:09.331 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-07T14:55:41.059 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-08-07T14:55:41.063 INFO:teuthology.orchestra.run.smithi074:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-08-07T14:56:01.704 INFO:teuthology.provision.fog.smithi074:Node is ready 2020-08-07T14:56:01.706 INFO:teuthology.orchestra.run.smithi074:> hostname 2020-08-07T14:56:01.728 INFO:teuthology.orchestra.run.smithi074.stdout:smithi074.front.sepia.ceph.com 2020-08-07T14:56:01.728 INFO:teuthology.orchestra.run.smithi074:> grep smithi074.front.sepia.ceph.com /etc/hosts 2020-08-07T14:56:01.803 INFO:teuthology.orchestra.run.smithi074.stdout:172.21.15.74 smithi074.front.sepia.ceph.com smithi074 2020-08-07T14:56:01.803 INFO:teuthology.orchestra.run.smithi074:> sudo hostname smithi074 2020-08-07T14:56:01.966 INFO:teuthology.orchestra.run.smithi074:> sudo sed -i -e 's/smithi074.front.sepia.ceph.com/smithi074/g' /etc/hosts 2020-08-07T14:56:02.005 INFO:teuthology.orchestra.run.smithi074:> sudo sed -i -e 's/172.21.15.74/172.21.15.74/g' /etc/hosts 2020-08-07T14:56:02.077 INFO:teuthology.orchestra.run.smithi074:> sudo sed -i -e 's/smithi074.front.sepia.ceph.com/smithi074/g' /etc/hostname 2020-08-07T14:56:02.151 INFO:teuthology.orchestra.run.smithi074:> sudo hostname smithi074 2020-08-07T14:56:02.227 INFO:teuthology.provision.fog.smithi074:Deploy complete! 2020-08-07T14:56:02.228 DEBUG:teuthology.parallel:result is None 2020-08-07T14:56:15.345 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-07T14:56:15.714 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-08-07T14:56:15.715 INFO:teuthology.orchestra.run.smithi096:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-08-07T14:56:16.063 INFO:teuthology.provision.fog.smithi096:Node is ready 2020-08-07T14:56:16.063 INFO:teuthology.orchestra.run.smithi096:> hostname 2020-08-07T14:56:16.091 INFO:teuthology.orchestra.run.smithi096.stdout:smithi096.front.sepia.ceph.com 2020-08-07T14:56:16.091 INFO:teuthology.orchestra.run.smithi096:> grep smithi096.front.sepia.ceph.com /etc/hosts 2020-08-07T14:56:16.154 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96 smithi096.front.sepia.ceph.com smithi096 2020-08-07T14:56:16.155 INFO:teuthology.orchestra.run.smithi096:> sudo hostname smithi096 2020-08-07T14:56:16.295 INFO:teuthology.orchestra.run.smithi096:> sudo sed -i -e 's/smithi096.front.sepia.ceph.com/smithi096/g' /etc/hosts 2020-08-07T14:56:16.334 INFO:teuthology.orchestra.run.smithi096:> sudo sed -i -e 's/172.21.15.96/172.21.15.96/g' /etc/hosts 2020-08-07T14:56:16.408 INFO:teuthology.orchestra.run.smithi096:> sudo sed -i -e 's/smithi096.front.sepia.ceph.com/smithi096/g' /etc/hostname 2020-08-07T14:56:16.478 INFO:teuthology.orchestra.run.smithi096:> sudo hostname smithi096 2020-08-07T14:56:16.553 INFO:teuthology.provision.fog.smithi096:Deploy complete! 2020-08-07T14:56:16.554 DEBUG:teuthology.parallel:result is None 2020-08-07T14:56:16.748 INFO:teuthology.lock.ops:Checking smithi074.front.sepia.ceph.com 2020-08-07T14:56:16.750 INFO:teuthology.lock.ops:New key found. Updating... 2020-08-07T14:56:16.775 INFO:teuthology.lock.ops:Checking smithi096.front.sepia.ceph.com 2020-08-07T14:56:16.776 INFO:teuthology.lock.ops:New key found. Updating... 2020-08-07T14:56:16.820 INFO:teuthology.lock.ops:Updating [smithi074.front.sepia.ceph.com]: set os type and version on server 2020-08-07T14:56:16.821 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-08-07T14:56:16.835 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi074.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-07T14:56:16.978 INFO:teuthology.orchestra.run.smithi074:> true 2020-08-07T14:56:17.151 INFO:teuthology.orchestra.run.smithi074:> uname -m 2020-08-07T14:56:17.207 INFO:teuthology.orchestra.run.smithi074.stdout:x86_64 2020-08-07T14:56:17.208 INFO:teuthology.orchestra.run.smithi074:> cat /etc/os-release 2020-08-07T14:56:17.262 INFO:teuthology.orchestra.run.smithi074.stdout:NAME="Red Hat Enterprise Linux" 2020-08-07T14:56:17.263 INFO:teuthology.orchestra.run.smithi074.stdout:VERSION="8.1 (Ootpa)" 2020-08-07T14:56:17.263 INFO:teuthology.orchestra.run.smithi074.stdout:ID="rhel" 2020-08-07T14:56:17.263 INFO:teuthology.orchestra.run.smithi074.stdout:ID_LIKE="fedora" 2020-08-07T14:56:17.263 INFO:teuthology.orchestra.run.smithi074.stdout:VERSION_ID="8.1" 2020-08-07T14:56:17.264 INFO:teuthology.orchestra.run.smithi074.stdout:PLATFORM_ID="platform:el8" 2020-08-07T14:56:17.264 INFO:teuthology.orchestra.run.smithi074.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.1 (Ootpa)" 2020-08-07T14:56:17.264 INFO:teuthology.orchestra.run.smithi074.stdout:ANSI_COLOR="0;31" 2020-08-07T14:56:17.264 INFO:teuthology.orchestra.run.smithi074.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.1:GA" 2020-08-07T14:56:17.267 INFO:teuthology.orchestra.run.smithi074.stdout:HOME_URL="https://www.redhat.com/" 2020-08-07T14:56:17.267 INFO:teuthology.orchestra.run.smithi074.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2020-08-07T14:56:17.267 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T14:56:17.267 INFO:teuthology.orchestra.run.smithi074.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2020-08-07T14:56:17.267 INFO:teuthology.orchestra.run.smithi074.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.1 2020-08-07T14:56:17.268 INFO:teuthology.orchestra.run.smithi074.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2020-08-07T14:56:17.268 INFO:teuthology.orchestra.run.smithi074.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.1" 2020-08-07T14:56:17.269 INFO:teuthology.lock.ops:Updating smithi074.front.sepia.ceph.com on lock server 2020-08-07T14:56:17.292 INFO:teuthology.lock.ops:Updating [smithi096.front.sepia.ceph.com]: set os type and version on server 2020-08-07T14:56:17.292 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-08-07T14:56:17.293 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-07T14:56:17.369 INFO:teuthology.orchestra.run.smithi096:> true 2020-08-07T14:56:17.531 INFO:teuthology.orchestra.run.smithi096:> uname -m 2020-08-07T14:56:17.589 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2020-08-07T14:56:17.590 INFO:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2020-08-07T14:56:17.645 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="Red Hat Enterprise Linux" 2020-08-07T14:56:17.645 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="8.1 (Ootpa)" 2020-08-07T14:56:17.646 INFO:teuthology.orchestra.run.smithi096.stdout:ID="rhel" 2020-08-07T14:56:17.646 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE="fedora" 2020-08-07T14:56:17.646 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="8.1" 2020-08-07T14:56:17.646 INFO:teuthology.orchestra.run.smithi096.stdout:PLATFORM_ID="platform:el8" 2020-08-07T14:56:17.646 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.1 (Ootpa)" 2020-08-07T14:56:17.647 INFO:teuthology.orchestra.run.smithi096.stdout:ANSI_COLOR="0;31" 2020-08-07T14:56:17.647 INFO:teuthology.orchestra.run.smithi096.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.1:GA" 2020-08-07T14:56:17.647 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://www.redhat.com/" 2020-08-07T14:56:17.647 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2020-08-07T14:56:17.647 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T14:56:17.648 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2020-08-07T14:56:17.648 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.1 2020-08-07T14:56:17.648 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2020-08-07T14:56:17.648 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.1" 2020-08-07T14:56:17.650 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2020-08-07T14:56:17.673 INFO:teuthology.task.internal.lock_machines:2 smithi machines locked this try, 2/2 locked so far 2020-08-07T14:56:17.756 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi074.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDeEIcBdq9e99p4DguAmP1GKEvfUqLz0tkqIetjmuBti17VTcn35q803Ri5fdgeyBfDlfevb+lc5x1TeF3ixLNtfbVzJ7oVdXIDqX7SkyafDSq6vXodQQzlvJyI+bTAlh4od1VjtWLz+6rHxt8KDHVu1yY4/jEfzQI0e1BStfSmelISr67klfLdzYRXQkgNoITAj1lpZzb1KNZA9AvtW/JbG4a5wELNF3j4IWTZoXm8t8ANekDJ5b0HqfiF0fGh/sr0gY4dk5e7UjOHpoSzpCuFcyLDWBGHwQ+EdYgNOpN4vsY/inpmNUpdD5B9Cb1K5IrtAp8XZAPt1EUsm5EtX47t9ywuTwwyX4LVB/AkxtkOQFHSTfKFlIg91DFOuf6VdorJWIlOHgGko4s/hRGAMS9/k+flh3XBoXAxuI2Yt+/ZTwlqtnhnbtC20ppNkAnOx8/SZzkjrwW2qlw1SsGgNOLZvyvHXnMN9kb3pxIhthf8XJFGd511UpOstS2dWIeXo40= smithi096.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCf4CtmyPbc9S5DEGTKwGREVs6f9MEaeOEDqh1p7IZ2+XpZIh2Jd3Kr5yEgldr6R4q6AsIXp2gLu73yJY+3SquSke1DPOC8rUeEpRPU79omJ4u3txXOty3D718RzboivgcEcdSNMdDcHqgOK7YV7ucK9clbS+tzvri1/rLCe8Wa6BXVVNOTR+bkcM3SoJg99uAgRU3JiQr3eGDgcwYeC+nspMhs5EfB+ipUuTejadpgTzERQqVPQ/lWgUWJVj3w3y0nuqpiu60cQL4OVVzZkQrCTCK+o+ct+BgzaftBaRDZC0gOT64gjCHknnZyGdIXYPJNnFQW2CZQR/Tn0Llx071ZkM4pjT1oDWX/fUPYo6G4Sc39JHlhBWToO+iREe0igUjt6P2sleuVwqSMuZmMEwUIJfQ98eeMLJGZeT+a9huNYzu5CkgWg1kJIzXmp/QRjjV1LQWe8Y7bcAat1ezOBrOq5QYKt+ECw9THtHEdt7WxZEIj/K9faAYmHUj9oAmurCU= 2020-08-07T14:56:17.756 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-08-07T14:56:17.850 INFO:teuthology.run_tasks:Running task internal.save_config... 2020-08-07T14:56:17.855 INFO:teuthology.task.internal:Saving configuration 2020-08-07T14:56:17.924 INFO:teuthology.run_tasks:Running task internal.check_lock... 2020-08-07T14:56:17.941 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-08-07T14:56:18.018 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-08-07 14:51:28.924671', 'description': '/home/teuthworker/archive/shuzhenyi-2020-08-07_08:55:00-rados:thrash-wip-shuzhenyi-testing-2020-08-06-1028-distro-basic-smithi/5306571', 'locked_by': 'scheduled_shuzhenyi@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMIGLLrkNzTZ4TmIO9sUCAJ35LbvNbtErJYOU3QDnUZ7SdEPyTdD+QTdNuSxSMDhmr/8CgAHGWWRc7MJhg0c20g=', 'os_version': '8.1', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'rhel', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi074.front.sepia.ceph.com'} 2020-08-07T14:56:18.054 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-08-07 14:51:28.924691', 'description': '/home/teuthworker/archive/shuzhenyi-2020-08-07_08:55:00-rados:thrash-wip-shuzhenyi-testing-2020-08-06-1028-distro-basic-smithi/5306571', 'locked_by': 'scheduled_shuzhenyi@teuthology', 'up': True, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJTnitnxcBVSJd8aHuGuhrnZZKWzoCgdw49rS1RFLG7QKvfX3BOa4ErxKPKm0EzsnxQ2QFLaIUlI59IBHhOw2Vk=', 'os_version': '8.1', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'rhel', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi096.front.sepia.ceph.com'} 2020-08-07T14:56:18.054 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2020-08-07T14:56:18.064 INFO:teuthology.task.internal:roles: ubuntu@smithi074.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2020-08-07T14:56:18.065 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2020-08-07T14:56:18.066 INFO:teuthology.run_tasks:Running task console_log... 2020-08-07T14:56:18.430 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa8022978c8>, signals=[15]) 2020-08-07T14:56:18.432 INFO:teuthology.run_tasks:Running task internal.connect... 2020-08-07T14:56:18.474 INFO:teuthology.task.internal:Opening connections... 2020-08-07T14:56:18.474 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi074.front.sepia.ceph.com 2020-08-07T14:56:18.476 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi074.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-07T14:56:18.622 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi096.front.sepia.ceph.com 2020-08-07T14:56:18.623 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-07T14:56:18.733 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2020-08-07T14:56:18.747 INFO:teuthology.orchestra.run.smithi074:> uname -m 2020-08-07T14:56:18.764 INFO:teuthology.orchestra.run.smithi074.stdout:x86_64 2020-08-07T14:56:18.765 INFO:teuthology.orchestra.run.smithi074:> cat /etc/os-release 2020-08-07T14:56:18.819 INFO:teuthology.orchestra.run.smithi074.stdout:NAME="Red Hat Enterprise Linux" 2020-08-07T14:56:18.820 INFO:teuthology.orchestra.run.smithi074.stdout:VERSION="8.1 (Ootpa)" 2020-08-07T14:56:18.820 INFO:teuthology.orchestra.run.smithi074.stdout:ID="rhel" 2020-08-07T14:56:18.820 INFO:teuthology.orchestra.run.smithi074.stdout:ID_LIKE="fedora" 2020-08-07T14:56:18.820 INFO:teuthology.orchestra.run.smithi074.stdout:VERSION_ID="8.1" 2020-08-07T14:56:18.821 INFO:teuthology.orchestra.run.smithi074.stdout:PLATFORM_ID="platform:el8" 2020-08-07T14:56:18.821 INFO:teuthology.orchestra.run.smithi074.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.1 (Ootpa)" 2020-08-07T14:56:18.821 INFO:teuthology.orchestra.run.smithi074.stdout:ANSI_COLOR="0;31" 2020-08-07T14:56:18.826 INFO:teuthology.orchestra.run.smithi074.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.1:GA" 2020-08-07T14:56:18.826 INFO:teuthology.orchestra.run.smithi074.stdout:HOME_URL="https://www.redhat.com/" 2020-08-07T14:56:18.827 INFO:teuthology.orchestra.run.smithi074.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2020-08-07T14:56:18.827 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T14:56:18.827 INFO:teuthology.orchestra.run.smithi074.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2020-08-07T14:56:18.828 INFO:teuthology.orchestra.run.smithi074.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.1 2020-08-07T14:56:18.828 INFO:teuthology.orchestra.run.smithi074.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2020-08-07T14:56:18.829 INFO:teuthology.orchestra.run.smithi074.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.1" 2020-08-07T14:56:18.830 INFO:teuthology.lock.ops:Updating smithi074.front.sepia.ceph.com on lock server 2020-08-07T14:56:18.856 INFO:teuthology.orchestra.run.smithi096:> uname -m 2020-08-07T14:56:18.881 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2020-08-07T14:56:18.882 INFO:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2020-08-07T14:56:18.949 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="Red Hat Enterprise Linux" 2020-08-07T14:56:18.949 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="8.1 (Ootpa)" 2020-08-07T14:56:18.949 INFO:teuthology.orchestra.run.smithi096.stdout:ID="rhel" 2020-08-07T14:56:18.950 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE="fedora" 2020-08-07T14:56:18.950 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="8.1" 2020-08-07T14:56:18.967 INFO:teuthology.orchestra.run.smithi096.stdout:PLATFORM_ID="platform:el8" 2020-08-07T14:56:18.967 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.1 (Ootpa)" 2020-08-07T14:56:18.968 INFO:teuthology.orchestra.run.smithi096.stdout:ANSI_COLOR="0;31" 2020-08-07T14:56:18.968 INFO:teuthology.orchestra.run.smithi096.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.1:GA" 2020-08-07T14:56:18.980 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://www.redhat.com/" 2020-08-07T14:56:18.999 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2020-08-07T14:56:18.999 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T14:56:19.000 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2020-08-07T14:56:19.000 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.1 2020-08-07T14:56:19.000 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2020-08-07T14:56:19.000 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.1" 2020-08-07T14:56:19.001 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2020-08-07T14:56:19.030 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2020-08-07T14:56:19.085 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2020-08-07T14:56:19.191 INFO:teuthology.task.internal:Checking for old test directory... 2020-08-07T14:56:19.193 INFO:teuthology.orchestra.run.smithi074:> test '!' -e /home/ubuntu/cephtest 2020-08-07T14:56:19.204 INFO:teuthology.orchestra.run.smithi096:> test '!' -e /home/ubuntu/cephtest 2020-08-07T14:56:19.219 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2020-08-07T14:56:19.314 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2020-08-07T14:56:19.315 INFO:teuthology.orchestra.run.smithi074:> test -z $(ls -A /var/lib/ceph) 2020-08-07T14:56:19.334 INFO:teuthology.orchestra.run.smithi096:> test -z $(ls -A /var/lib/ceph) 2020-08-07T14:56:19.351 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2020-08-07T14:56:19.419 INFO:teuthology.run_tasks:Running task kernel... 2020-08-07T14:56:19.484 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}, 'mgr.y': {'kdb': True, 'sha1': 'distro'}, 'osd.0': {'kdb': True, 'sha1': 'distro'}, 'osd.1': {'kdb': True, 'sha1': 'distro'}, 'osd.2': {'kdb': True, 'sha1': 'distro'}, 'osd.3': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'node-exporter.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}, 'osd.4': {'kdb': True, 'sha1': 'distro'}, 'osd.5': {'kdb': True, 'sha1': 'distro'}, 'osd.6': {'kdb': True, 'sha1': 'distro'}, 'osd.7': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}, 'prometheus.a': {'kdb': True, 'sha1': 'distro'}, 'node-exporter.b': {'kdb': True, 'sha1': 'distro'}} 2020-08-07T14:56:19.485 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2020-08-07T14:56:19.485 INFO:teuthology.orchestra.run.smithi074:> uname -r 2020-08-07T14:56:19.510 INFO:teuthology.orchestra.run.smithi074.stdout:4.18.0-193.el8.x86_64 2020-08-07T14:56:19.511 INFO:teuthology.task.kernel:Running kernel on smithi074: 4.18.0-193.el8.x86_64 2020-08-07T14:56:19.511 INFO:teuthology.orchestra.run.smithi074:> sudo yum install -y kernel 2020-08-07T14:56:21.460 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T14:56:21.473 INFO:teuthology.orchestra.run.smithi074.stdout:Unable to read consumer identity 2020-08-07T14:56:21.588 INFO:teuthology.orchestra.run.smithi074.stdout:This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. 2020-08-07T14:56:23.034 INFO:teuthology.orchestra.run.smithi074.stdout:Copr repo for ceph-el8 owned by ktdreyer 1.2 kB/s | 257 B 00:00 2020-08-07T14:56:23.517 INFO:teuthology.orchestra.run.smithi074.stdout:Extra Packages for Enterprise Linux 18 MB/s | 7.5 MB 00:00 2020-08-07T14:56:24.987 INFO:teuthology.orchestra.run.smithi074.stdout:lab-extras 4.1 MB/s | 20 kB 00:00 2020-08-07T14:56:25.286 INFO:teuthology.orchestra.run.smithi074.stderr:Modular dependency problem: 2020-08-07T14:56:25.286 INFO:teuthology.orchestra.run.smithi074.stderr: 2020-08-07T14:56:25.286 INFO:teuthology.orchestra.run.smithi074.stderr: Problem: conflicting requests 2020-08-07T14:56:25.287 INFO:teuthology.orchestra.run.smithi074.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-08-07T14:56:25.471 INFO:teuthology.orchestra.run.smithi074.stdout:Package kernel-4.18.0-193.el8.x86_64 is already installed. 2020-08-07T14:56:25.471 INFO:teuthology.orchestra.run.smithi074.stdout:Package kernel-4.18.0-147.el8.x86_64 is already installed. 2020-08-07T14:56:25.547 INFO:teuthology.orchestra.run.smithi074.stdout:Dependencies resolved. 2020-08-07T14:56:25.575 INFO:teuthology.orchestra.run.smithi074.stdout:Nothing to do. 2020-08-07T14:56:25.575 INFO:teuthology.orchestra.run.smithi074.stdout:Complete! 2020-08-07T14:56:25.751 INFO:teuthology.orchestra.run.smithi074:> echo no | sudo yum reinstall kernel || true 2020-08-07T14:56:26.094 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T14:56:26.095 INFO:teuthology.orchestra.run.smithi074.stdout:Unable to read consumer identity 2020-08-07T14:56:26.130 INFO:teuthology.orchestra.run.smithi074.stdout:This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. 2020-08-07T14:56:26.311 INFO:teuthology.orchestra.run.smithi074.stdout:Last metadata expiration check: 0:00:02 ago on Fri 07 Aug 2020 02:56:24 PM UTC. 2020-08-07T14:56:26.326 INFO:teuthology.orchestra.run.smithi074.stderr:Modular dependency problem: 2020-08-07T14:56:26.327 INFO:teuthology.orchestra.run.smithi074.stderr: 2020-08-07T14:56:26.327 INFO:teuthology.orchestra.run.smithi074.stderr: Problem: conflicting requests 2020-08-07T14:56:26.328 INFO:teuthology.orchestra.run.smithi074.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-08-07T14:56:26.411 INFO:teuthology.orchestra.run.smithi074.stdout:Installed package kernel-4.18.0-193.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2020-08-07T14:56:26.412 INFO:teuthology.orchestra.run.smithi074.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-08-07T14:56:26.476 INFO:teuthology.orchestra.run.smithi074.stderr:Error: No packages marked for reinstall. 2020-08-07T14:56:26.576 INFO:teuthology.orchestra.run.smithi074:> sudo yum reinstall -y kernel || true 2020-08-07T14:56:26.912 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T14:56:26.913 INFO:teuthology.orchestra.run.smithi074.stdout:Unable to read consumer identity 2020-08-07T14:56:26.959 INFO:teuthology.orchestra.run.smithi074.stdout:This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. 2020-08-07T14:56:27.122 INFO:teuthology.orchestra.run.smithi074.stdout:Last metadata expiration check: 0:00:03 ago on Fri 07 Aug 2020 02:56:24 PM UTC. 2020-08-07T14:56:27.151 INFO:teuthology.orchestra.run.smithi074.stderr:Modular dependency problem: 2020-08-07T14:56:27.151 INFO:teuthology.orchestra.run.smithi074.stderr: 2020-08-07T14:56:27.151 INFO:teuthology.orchestra.run.smithi074.stderr: Problem: conflicting requests 2020-08-07T14:56:27.152 INFO:teuthology.orchestra.run.smithi074.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-08-07T14:56:27.239 INFO:teuthology.orchestra.run.smithi074.stdout:Installed package kernel-4.18.0-193.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2020-08-07T14:56:27.240 INFO:teuthology.orchestra.run.smithi074.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-08-07T14:56:27.294 INFO:teuthology.orchestra.run.smithi074.stderr:Error: No packages marked for reinstall. 2020-08-07T14:56:27.363 INFO:teuthology.orchestra.run.smithi074:> rpm -q kernel --last | head -n 1 2020-08-07T14:56:27.429 INFO:teuthology.orchestra.run.smithi074.stdout:kernel-4.18.0-193.el8.x86_64 Sun 10 May 2020 01:56:55 PM UTC 2020-08-07T14:56:27.430 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-193.el8.x86_64 2020-08-07T14:56:27.430 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2020-08-07T14:56:27.431 INFO:teuthology.orchestra.run.smithi096:> uname -r 2020-08-07T14:56:27.448 INFO:teuthology.orchestra.run.smithi096.stdout:4.18.0-193.el8.x86_64 2020-08-07T14:56:27.449 INFO:teuthology.task.kernel:Running kernel on smithi096: 4.18.0-193.el8.x86_64 2020-08-07T14:56:27.449 INFO:teuthology.orchestra.run.smithi096:> sudo yum install -y kernel 2020-08-07T14:56:29.605 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T14:56:29.643 INFO:teuthology.orchestra.run.smithi096.stdout:Unable to read consumer identity 2020-08-07T14:56:29.996 INFO:teuthology.orchestra.run.smithi096.stdout:This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. 2020-08-07T14:56:31.426 INFO:teuthology.orchestra.run.smithi096.stdout:Copr repo for ceph-el8 owned by ktdreyer 1.7 kB/s | 257 B 00:00 2020-08-07T14:56:31.911 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 17 MB/s | 7.5 MB 00:00 2020-08-07T14:56:33.401 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 3.9 MB/s | 20 kB 00:00 2020-08-07T14:56:33.732 INFO:teuthology.orchestra.run.smithi096.stderr:Modular dependency problem: 2020-08-07T14:56:33.733 INFO:teuthology.orchestra.run.smithi096.stderr: 2020-08-07T14:56:33.733 INFO:teuthology.orchestra.run.smithi096.stderr: Problem: conflicting requests 2020-08-07T14:56:33.733 INFO:teuthology.orchestra.run.smithi096.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-08-07T14:56:33.910 INFO:teuthology.orchestra.run.smithi096.stdout:Package kernel-4.18.0-193.el8.x86_64 is already installed. 2020-08-07T14:56:33.910 INFO:teuthology.orchestra.run.smithi096.stdout:Package kernel-4.18.0-147.el8.x86_64 is already installed. 2020-08-07T14:56:34.004 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2020-08-07T14:56:34.007 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2020-08-07T14:56:34.007 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2020-08-07T14:56:34.207 INFO:teuthology.orchestra.run.smithi096:> echo no | sudo yum reinstall kernel || true 2020-08-07T14:56:34.540 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T14:56:34.541 INFO:teuthology.orchestra.run.smithi096.stdout:Unable to read consumer identity 2020-08-07T14:56:34.624 INFO:teuthology.orchestra.run.smithi096.stdout:This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. 2020-08-07T14:56:34.786 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:00:01 ago on Fri 07 Aug 2020 02:56:33 PM UTC. 2020-08-07T14:56:34.820 INFO:teuthology.orchestra.run.smithi096.stderr:Modular dependency problem: 2020-08-07T14:56:34.821 INFO:teuthology.orchestra.run.smithi096.stderr: 2020-08-07T14:56:34.821 INFO:teuthology.orchestra.run.smithi096.stderr: Problem: conflicting requests 2020-08-07T14:56:34.822 INFO:teuthology.orchestra.run.smithi096.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-08-07T14:56:34.901 INFO:teuthology.orchestra.run.smithi096.stdout:Installed package kernel-4.18.0-193.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2020-08-07T14:56:34.901 INFO:teuthology.orchestra.run.smithi096.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-08-07T14:56:35.020 INFO:teuthology.orchestra.run.smithi096.stderr:Error: No packages marked for reinstall. 2020-08-07T14:56:35.103 INFO:teuthology.orchestra.run.smithi096:> sudo yum reinstall -y kernel || true 2020-08-07T14:56:35.429 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T14:56:35.430 INFO:teuthology.orchestra.run.smithi096.stdout:Unable to read consumer identity 2020-08-07T14:56:35.465 INFO:teuthology.orchestra.run.smithi096.stdout:This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. 2020-08-07T14:56:35.634 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:00:02 ago on Fri 07 Aug 2020 02:56:33 PM UTC. 2020-08-07T14:56:35.665 INFO:teuthology.orchestra.run.smithi096.stderr:Modular dependency problem: 2020-08-07T14:56:35.666 INFO:teuthology.orchestra.run.smithi096.stderr: 2020-08-07T14:56:35.666 INFO:teuthology.orchestra.run.smithi096.stderr: Problem: conflicting requests 2020-08-07T14:56:35.666 INFO:teuthology.orchestra.run.smithi096.stderr: - nothing provides module(perl:5.26) needed by module perl-YAML:1.24:8010020190322131811:a5949e2e-0.x86_64 2020-08-07T14:56:35.750 INFO:teuthology.orchestra.run.smithi096.stdout:Installed package kernel-4.18.0-193.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2020-08-07T14:56:35.751 INFO:teuthology.orchestra.run.smithi096.stdout:Installed package kernel-4.18.0-147.el8.x86_64 (from anaconda) not available. 2020-08-07T14:56:35.822 INFO:teuthology.orchestra.run.smithi096.stderr:Error: No packages marked for reinstall. 2020-08-07T14:56:35.923 INFO:teuthology.orchestra.run.smithi096:> rpm -q kernel --last | head -n 1 2020-08-07T14:56:35.985 INFO:teuthology.orchestra.run.smithi096.stdout:kernel-4.18.0-193.el8.x86_64 Sun 10 May 2020 01:56:55 PM UTC 2020-08-07T14:56:35.986 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-193.el8.x86_64 2020-08-07T14:56:35.998 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2020-08-07T14:56:35.999 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2020-08-07T14:56:36.000 INFO:teuthology.orchestra.run.smithi074:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-08-07T14:56:36.050 INFO:teuthology.orchestra.run.smithi074.stdout:ttyS1 2020-08-07T14:56:36.088 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2020-08-07T14:56:36.089 INFO:teuthology.orchestra.run.smithi096:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2020-08-07T14:56:36.159 INFO:teuthology.orchestra.run.smithi096.stdout:ttyS1 2020-08-07T14:56:36.184 INFO:teuthology.run_tasks:Running task internal.base... 2020-08-07T14:56:36.231 INFO:teuthology.task.internal:Creating test directory... 2020-08-07T14:56:36.232 INFO:teuthology.orchestra.run.smithi074:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-08-07T14:56:36.254 INFO:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-08-07T14:56:36.279 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2020-08-07T14:56:36.302 INFO:teuthology.run_tasks:Running task internal.archive... 2020-08-07T14:56:36.314 INFO:teuthology.task.internal:Creating archive directory... 2020-08-07T14:56:36.315 INFO:teuthology.orchestra.run.smithi074:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-08-07T14:56:36.326 INFO:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-08-07T14:56:36.371 INFO:teuthology.run_tasks:Running task internal.coredump... 2020-08-07T14:56:36.398 INFO:teuthology.task.internal:Enabling coredump saving... 2020-08-07T14:56:36.399 INFO:teuthology.orchestra.run.smithi074:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-08-07T14:56:36.415 INFO:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-08-07T14:56:36.464 INFO:teuthology.orchestra.run.smithi074.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-08-07T14:56:36.475 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-08-07T14:56:36.484 INFO:teuthology.orchestra.run.smithi074.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-08-07T14:56:36.487 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-08-07T14:56:36.489 INFO:teuthology.run_tasks:Running task internal.sudo... 2020-08-07T14:56:36.565 INFO:teuthology.task.internal:Configuring sudo... 2020-08-07T14:56:36.566 INFO:teuthology.orchestra.run.smithi074:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-08-07T14:56:36.593 INFO:teuthology.orchestra.run.smithi096:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-08-07T14:56:36.629 INFO:teuthology.run_tasks:Running task internal.syslog... 2020-08-07T14:56:36.652 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2020-08-07T14:56:36.653 INFO:teuthology.orchestra.run.smithi074:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-08-07T14:56:36.677 INFO:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-08-07T14:56:36.692 INFO:teuthology.orchestra.run.smithi074:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-08-07T14:56:36.771 INFO:teuthology.orchestra.run.smithi074:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2020-08-07T14:56:36.848 INFO:teuthology.orchestra.run.smithi074:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-08-07T14:56:36.926 INFO:teuthology.orchestra.run.smithi074:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2020-08-07T14:56:36.979 INFO:teuthology.orchestra.run.smithi074:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2020-08-07T14:56:37.088 INFO:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-08-07T14:56:37.129 INFO:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2020-08-07T14:56:37.188 INFO:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-08-07T14:56:37.270 INFO:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2020-08-07T14:56:37.318 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2020-08-07T14:56:37.416 INFO:teuthology.orchestra.run.smithi074:> sudo service rsyslog restart 2020-08-07T14:56:37.421 INFO:teuthology.orchestra.run.smithi096:> sudo service rsyslog restart 2020-08-07T14:56:37.489 INFO:teuthology.orchestra.run.smithi074.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2020-08-07T14:56:37.513 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2020-08-07T14:56:37.839 INFO:teuthology.run_tasks:Running task internal.timer... 2020-08-07T14:56:37.857 INFO:teuthology.task.internal:Starting timer... 2020-08-07T14:56:37.858 INFO:teuthology.run_tasks:Running task pcp... 2020-08-07T14:56:37.903 INFO:teuthology.run_tasks:Running task selinux... 2020-08-07T14:56:37.926 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2020-08-07T14:56:37.928 INFO:teuthology.orchestra.run.smithi074:> sudo service auditd rotate 2020-08-07T14:56:37.997 INFO:teuthology.orchestra.run.smithi074.stdout:Rotating logs: [ OK ] 2020-08-07T14:56:37.997 INFO:teuthology.orchestra.run.smithi096:> sudo service auditd rotate 2020-08-07T14:56:38.067 INFO:teuthology.orchestra.run.smithi096.stdout:Rotating logs: [ OK ] 2020-08-07T14:56:38.068 DEBUG:teuthology.task.selinux:Getting current SELinux state 2020-08-07T14:56:38.069 INFO:teuthology.orchestra.run.smithi074:> /usr/sbin/getenforce 2020-08-07T14:56:38.093 INFO:teuthology.orchestra.run.smithi074.stdout:Permissive 2020-08-07T14:56:38.096 INFO:teuthology.orchestra.run.smithi096:> /usr/sbin/getenforce 2020-08-07T14:56:38.117 INFO:teuthology.orchestra.run.smithi096.stdout:Permissive 2020-08-07T14:56:38.119 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi074.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi096.front.sepia.ceph.com': 'permissive'} 2020-08-07T14:56:38.119 INFO:teuthology.orchestra.run.smithi074:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|scontext=system_u:system_r:logrotate_t:s0\)' 2020-08-07T14:56:38.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-08-07T14:56:38.174 INFO:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|scontext=system_u:system_r:logrotate_t:s0\)' 2020-08-07T14:56:38.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-08-07T14:56:38.228 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2020-08-07T14:56:38.229 INFO:teuthology.orchestra.run.smithi074:> sudo /usr/sbin/setenforce permissive 2020-08-07T14:56:38.259 INFO:teuthology.orchestra.run.smithi096:> sudo /usr/sbin/setenforce permissive 2020-08-07T14:56:38.297 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2020-08-07T14:56:38.310 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated; assuming it is current 2020-08-07T14:56:38.311 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2020-08-07T14:56:38.344 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2020-08-07T14:56:38.347 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi074.front.sepia.ceph.com,smithi096.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2020-08-07T14:56:38.761 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2020-08-07T14:56:39.932 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2020-08-07T14:56:42.662 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2020-08-07T14:56:42.694 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:42 +0000 (0:00:00.080) 0:00:00.080 ********* 2020-08-07T14:56:42.710 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:42 +0000 (0:00:00.018) 0:00:00.098 ********* 2020-08-07T14:56:44.239 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi074.front.sepia.ceph.com] 2020-08-07T14:56:44.247 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] 2020-08-07T14:56:44.347 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:44 +0000 (0:00:01.644) 0:00:01.742 ********* 2020-08-07T14:56:44.383 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:44 +0000 (0:00:00.033) 0:00:01.776 ********* 2020-08-07T14:56:44.855 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-08-07T14:56:44.907 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:44 +0000 (0:00:00.525) 0:00:02.302 ********* 2020-08-07T14:56:44.920 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-08-07T14:56:44.962 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:44 +0000 (0:00:00.049) 0:00:02.351 ********* 2020-08-07T14:56:45.533 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** ok: [smithi074.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-08-07T14:56:45.573 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:45 +0000 (0:00:00.615) 0:00:02.966 ********* 2020-08-07T14:56:45.583 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-08-07T14:56:45.623 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:45 +0000 (0:00:00.051) 0:00:03.018 ********* 2020-08-07T14:56:45.637 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:56:45.679 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:45 +0000 (0:00:00.055) 0:00:03.073 ********* 2020-08-07T14:56:45.690 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:56:45.738 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:45 +0000 (0:00:00.059) 0:00:03.133 ********* 2020-08-07T14:56:46.212 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** ok: [smithi074.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-08-07T14:56:46.253 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:46 +0000 (0:00:00.507) 0:00:03.641 ********* 2020-08-07T14:56:46.258 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-08-07T14:56:46.305 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:46 +0000 (0:00:00.058) 0:00:03.699 ********* 2020-08-07T14:56:47.737 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2020-08-07T14:56:47.739 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1596812206.3500519-148406044665240/source", "state": "file", "uid": 0} changed: [smithi074.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1596812206.2647617-201972612783016/source", "state": "file", "uid": 0} 2020-08-07T14:56:47.899 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:47 +0000 (0:00:01.594) 0:00:05.294 ********* 2020-08-07T14:56:47.946 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:47 +0000 (0:00:00.046) 0:00:05.341 ********* 2020-08-07T14:56:48.550 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtC 2020-08-07T14:56:48.553 INFO:teuthology.task.ansible.out:toHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-08-07T14:56:48.583 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-08-07T14:56:48.585 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-08-07T14:56:48.708 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2020-08-07T14:56:48.721 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2020-08-07T14:56:48.883 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:48 +0000 (0:00:00.936) 0:00:06.278 ********* 2020-08-07T14:56:48.935 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:48 +0000 (0:00:00.052) 0:00:06.330 ********* 2020-08-07T14:56:49.683 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi074.front.sepia.ceph.com] 2020-08-07T14:56:49.765 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] 2020-08-07T14:56:49.833 INFO:teuthology.task.ansible.out:[WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. Friday 07 August 2020 14:56:49 +0000 (0:00:00.892) 0:00:07.222 ********* 2020-08-07T14:56:50.017 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:50 +0000 (0:00:00.188) 0:00:07.411 ********* 2020-08-07T14:56:50.672 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2020-08-07T14:56:50.674 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.193578", "end": "2020-08-07 14:56:50.642707", "rc": 0, "start": "2020-08-07 14:56:50.449129", "stderr": "", "stderr_lines": [], "stdout": "822ea1561785feff19f519c0701e8aab1dcab883", "stdout_lines": ["822ea1561785feff19f519c0701e8aab1dcab883"]} 2020-08-07T14:56:50.708 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:50 +0000 (0:00:00.692) 0:00:08.103 ********* 2020-08-07T14:56:50.721 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.267035", "end": "2020-08-07 14:56:50.660093", "rc": 0, "start": "2020-08-07 14:56:50.393058", "stderr": "", "stderr_lines": [], "stdout": "822ea1561785feff19f519c0701e8aab1dcab883", "stdout_lines": ["822ea1561785feff19f519c0701e8aab1dcab883"]} 2020-08-07T14:56:50.771 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:50 +0000 (0:00:00.062) 0:00:08.166 ********* 2020-08-07T14:56:50.949 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** changed: [smithi074.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.013282", "end": "2020-08-07 14:56:50.926026", "failed_when_result": false, "rc": 0, "start": "2020-08-07 14:56:50.912744", "stderr": "", "stderr_lines": [], "stdout": "f148133fb0c07a635fda93cc17e7dcc2a5d008b7", "stdout_lines": ["f148133fb0c07a635fda93cc17e7dcc2a5d008b7"]} 2020-08-07T14:56:50.989 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:50 +0000 (0:00:00.218) 0:00:08.385 ********* 2020-08-07T14:56:51.012 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.012861", "end": "2020-08-07 14:56:50.981848", "failed_when_result": false, "rc": 0, "start": "2020-08-07 14:56:50.968987", "stderr": "", "stderr_lines": [], "stdout": "f148133fb0c07a635fda93cc17e7dcc2a5d008b7", "stdout_lines": ["f148133fb0c07a635fda93cc17e7dcc2a5d008b7"]} 2020-08-07T14:56:51.059 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:51 +0000 (0:00:00.069) 0:00:08.455 ********* 2020-08-07T14:56:51.071 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-08-07T14:56:51.073 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:56:51.114 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:51 +0000 (0:00:00.054) 0:00:08.509 ********* 2020-08-07T14:56:51.128 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:56:51.165 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:51 +0000 (0:00:00.050) 0:00:08.559 ********* 2020-08-07T14:56:51.182 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:56:51.213 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:51 +0000 (0:00:00.048) 0:00:08.608 ********* 2020-08-07T14:56:51.224 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:56:51.266 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:51 +0000 (0:00:00.049) 0:00:08.658 ********* 2020-08-07T14:56:51.273 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:56:51.313 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:51 +0000 (0:00:00.049) 0:00:08.708 ********* 2020-08-07T14:56:51.323 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:56:51.370 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:51 +0000 (0:00:00.057) 0:00:08.765 ********* 2020-08-07T14:56:51.380 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2020-08-07T14:56:51.382 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:56:51.420 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:51 +0000 (0:00:00.050) 0:00:08.815 ********* 2020-08-07T14:56:51.430 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:56:51.469 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:51 +0000 (0:00:00.048) 0:00:08.864 ********* 2020-08-07T14:56:51.481 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2020-08-07T14:56:51.483 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:56:51.530 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:51 +0000 (0:00:00.060) 0:00:08.925 ********* 2020-08-07T14:56:51.540 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:56:51.586 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:51 +0000 (0:00:00.056) 0:00:08.981 ********* 2020-08-07T14:56:51.744 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2020-08-07T14:56:51.750 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-08-07T14:56:51.805 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:51 +0000 (0:00:00.218) 0:00:09.199 ********* 2020-08-07T14:56:51.838 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-08-07T14:56:51.878 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:56:51 +0000 (0:00:00.071) 0:00:09.271 ********* 2020-08-07T14:56:52.072 INFO:teuthology.task.ansible.out: 2020-08-07T14:56:52.074 INFO:teuthology.task.ansible.out:TASK [users : Create all admin users with sudo access.] ************************ ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2020-08-07T14:56:52.188 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2020-08-07T14:56:52.298 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2020-08-07T14:56:52.385 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2020-08-07T14:56:52.503 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2020-08-07T14:56:52.618 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2020-08-07T14:56:52.707 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2020-08-07T14:56:52.823 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2020-08-07T14:56:52.930 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2020-08-07T14:56:53.026 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2020-08-07T14:56:53.150 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2020-08-07T14:56:53.236 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2020-08-07T14:56:53.346 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2020-08-07T14:56:53.449 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2020-08-07T14:56:53.562 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2020-08-07T14:56:53.666 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2020-08-07T14:56:53.797 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2020-08-07T14:56:53.881 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2020-08-07T14:56:54.002 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2020-08-07T14:56:54.115 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2020-08-07T14:56:54.226 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2020-08-07T14:56:54.340 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2020-08-07T14:56:54.459 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2020-08-07T14:56:54.572 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2020-08-07T14:56:54.676 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2020-08-07T14:56:54.782 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2020-08-07T14:56:54.898 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2020-08-07T14:56:55.006 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2020-08-07T14:56:55.119 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2020-08-07T14:56:55.216 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2020-08-07T14:56:55.330 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2020-08-07T14:56:55.437 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2020-08-07T14:56:55.549 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2020-08-07T14:56:55.658 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2020-08-07T14:56:55.760 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2020-08-07T14:56:55.878 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2020-08-07T14:56:55.973 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2020-08-07T14:56:56.116 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2020-08-07T14:56:56.181 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2020-08-07T14:56:56.330 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2020-08-07T14:56:56.395 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2020-08-07T14:56:56.540 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2020-08-07T14:56:56.613 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1122} 2020-08-07T14:56:56.753 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1122} 2020-08-07T14:56:56.813 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1123} 2020-08-07T14:56:56.962 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1123} 2020-08-07T14:56:57.025 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1124} 2020-08-07T14:56:57.149 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1124} 2020-08-07T14:56:57.261 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1125} 2020-08-07T14:56:57.368 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1125} 2020-08-07T14:56:57.463 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1126} 2020-08-07T14:56:57.580 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1126} 2020-08-07T14:56:57.698 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1127} 2020-08-07T14:56:57.804 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1127} 2020-08-07T14:56:57.926 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1128} 2020-08-07T14:56:58.017 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1128} 2020-08-07T14:56:58.164 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1129} 2020-08-07T14:56:58.218 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1129} 2020-08-07T14:56:58.363 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1130} 2020-08-07T14:56:58.463 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1130} 2020-08-07T14:56:58.590 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1131} 2020-08-07T14:56:58.709 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1131} 2020-08-07T14:56:58.801 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1132} 2020-08-07T14:56:58.907 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1132} 2020-08-07T14:56:59.007 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1133} 2020-08-07T14:56:59.130 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1133} 2020-08-07T14:56:59.229 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1134} 2020-08-07T14:56:59.360 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1134} 2020-08-07T14:56:59.451 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1135} 2020-08-07T14:56:59.590 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1135} 2020-08-07T14:56:59.660 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1136} 2020-08-07T14:56:59.800 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1136} 2020-08-07T14:56:59.864 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1137} 2020-08-07T14:57:00.015 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1137} 2020-08-07T14:57:00.210 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1138} 2020-08-07T14:57:00.240 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1138} 2020-08-07T14:57:00.293 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1139} 2020-08-07T14:57:00.474 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1139} 2020-08-07T14:57:00.512 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2020-08-07T14:57:00.666 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2020-08-07T14:57:00.742 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1141} 2020-08-07T14:57:00.886 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1141} 2020-08-07T14:57:00.951 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1142} 2020-08-07T14:57:01.088 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1142} 2020-08-07T14:57:01.162 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1143} 2020-08-07T14:57:01.287 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1143} 2020-08-07T14:57:01.371 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1144} 2020-08-07T14:57:01.515 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1144} 2020-08-07T14:57:01.560 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1145} 2020-08-07T14:57:01.719 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1145} 2020-08-07T14:57:01.810 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1146} 2020-08-07T14:57:01.941 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1146} 2020-08-07T14:57:02.037 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1147} 2020-08-07T14:57:02.147 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1147} 2020-08-07T14:57:02.250 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2020-08-07T14:57:02.354 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2020-08-07T14:57:02.466 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1149} 2020-08-07T14:57:02.575 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1149} 2020-08-07T14:57:02.677 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1150} 2020-08-07T14:57:02.798 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1150} 2020-08-07T14:57:02.927 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1151} 2020-08-07T14:57:03.004 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1151} 2020-08-07T14:57:03.146 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1152} 2020-08-07T14:57:03.235 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1152} 2020-08-07T14:57:03.383 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1153} 2020-08-07T14:57:03.430 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1153} 2020-08-07T14:57:03.610 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1154} 2020-08-07T14:57:03.663 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1154} 2020-08-07T14:57:03.835 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1155} 2020-08-07T14:57:03.876 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1155} 2020-08-07T14:57:04.079 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1156} 2020-08-07T14:57:04.098 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1156} 2020-08-07T14:57:04.294 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1157} 2020-08-07T14:57:04.411 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1157} 2020-08-07T14:57:04.574 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1158} 2020-08-07T14:57:04.638 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1158} 2020-08-07T14:57:04.782 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1159} 2020-08-07T14:57:04.911 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1159} 2020-08-07T14:57:05.104 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1160} 2020-08-07T14:57:05.170 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1160} 2020-08-07T14:57:05.311 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1161} 2020-08-07T14:57:05.382 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1161} 2020-08-07T14:57:05.525 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1162} 2020-08-07T14:57:05.637 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1162} 2020-08-07T14:57:05.750 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1163} 2020-08-07T14:57:05.863 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1163} 2020-08-07T14:57:05.955 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1164} 2020-08-07T14:57:06.084 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1164} 2020-08-07T14:57:06.185 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2020-08-07T14:57:06.319 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1140} 2020-08-07T14:57:06.420 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1165} 2020-08-07T14:57:06.550 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1165} 2020-08-07T14:57:06.624 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1166} 2020-08-07T14:57:06.790 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1166} 2020-08-07T14:57:06.883 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1167} 2020-08-07T14:57:07.041 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1167} 2020-08-07T14:57:07.107 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1168} 2020-08-07T14:57:07.260 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1168} 2020-08-07T14:57:07.347 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1169} 2020-08-07T14:57:07.490 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1169} 2020-08-07T14:57:07.574 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1170} 2020-08-07T14:57:07.716 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1170} 2020-08-07T14:57:07.810 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1171} 2020-08-07T14:57:07.975 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1171} 2020-08-07T14:57:08.038 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1172} 2020-08-07T14:57:08.185 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1172} 2020-08-07T14:57:08.252 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1173} 2020-08-07T14:57:08.418 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1173} 2020-08-07T14:57:08.484 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1174} 2020-08-07T14:57:08.618 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1174} 2020-08-07T14:57:08.685 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1175} 2020-08-07T14:57:08.829 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1175} 2020-08-07T14:57:08.924 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1176} 2020-08-07T14:57:09.059 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1176} 2020-08-07T14:57:09.139 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1177} 2020-08-07T14:57:09.266 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1177} 2020-08-07T14:57:09.378 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1178} 2020-08-07T14:57:09.495 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1178} 2020-08-07T14:57:09.600 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1179} 2020-08-07T14:57:09.706 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1179} 2020-08-07T14:57:09.824 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1180} 2020-08-07T14:57:09.913 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1180} 2020-08-07T14:57:10.016 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1181} 2020-08-07T14:57:10.147 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1181} 2020-08-07T14:57:10.223 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1182} 2020-08-07T14:57:10.334 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1182} 2020-08-07T14:57:10.446 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1183} 2020-08-07T14:57:10.539 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1183} 2020-08-07T14:57:10.662 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1184} 2020-08-07T14:57:10.759 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1184} 2020-08-07T14:57:10.870 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1185} 2020-08-07T14:57:10.979 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1185} 2020-08-07T14:57:11.109 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1186} 2020-08-07T14:57:11.186 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1186} 2020-08-07T14:57:11.320 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1187} 2020-08-07T14:57:11.425 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1187} 2020-08-07T14:57:11.537 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1188} 2020-08-07T14:57:11.661 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1188} 2020-08-07T14:57:11.750 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1189} 2020-08-07T14:57:11.880 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1189} 2020-08-07T14:57:11.980 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1190} 2020-08-07T14:57:12.106 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1190} 2020-08-07T14:57:12.190 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1191} 2020-08-07T14:57:12.316 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1191} 2020-08-07T14:57:12.422 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1192} 2020-08-07T14:57:12.540 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1192} 2020-08-07T14:57:12.688 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1193} 2020-08-07T14:57:12.754 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1193} 2020-08-07T14:57:12.896 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1194} 2020-08-07T14:57:12.971 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1194} 2020-08-07T14:57:13.113 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1195} 2020-08-07T14:57:13.182 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1195} 2020-08-07T14:57:13.374 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1196} 2020-08-07T14:57:13.415 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1196} 2020-08-07T14:57:13.576 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1197} 2020-08-07T14:57:13.644 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1197} 2020-08-07T14:57:13.808 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1198} 2020-08-07T14:57:13.851 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1198} 2020-08-07T14:57:14.036 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1199} 2020-08-07T14:57:14.118 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1199} 2020-08-07T14:57:14.256 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1200} 2020-08-07T14:57:14.332 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1200} 2020-08-07T14:57:14.456 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1201} 2020-08-07T14:57:14.534 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1201} 2020-08-07T14:57:14.678 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2020-08-07T14:57:14.741 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1148} 2020-08-07T14:57:14.902 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1202} 2020-08-07T14:57:14.962 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1202} 2020-08-07T14:57:15.154 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1203} 2020-08-07T14:57:15.182 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1203} 2020-08-07T14:57:15.379 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1204} 2020-08-07T14:57:15.392 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1204} 2020-08-07T14:57:15.588 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1205} 2020-08-07T14:57:15.711 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1205} 2020-08-07T14:57:15.814 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1206} 2020-08-07T14:57:15.833 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1206} 2020-08-07T14:57:16.012 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1207} 2020-08-07T14:57:16.062 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1207} 2020-08-07T14:57:16.232 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1208} 2020-08-07T14:57:16.315 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1208} 2020-08-07T14:57:16.470 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1209} 2020-08-07T14:57:16.525 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1209} 2020-08-07T14:57:16.682 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1210} 2020-08-07T14:57:16.762 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1210} 2020-08-07T14:57:16.896 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1211} 2020-08-07T14:57:17.004 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1211} 2020-08-07T14:57:17.129 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1212} 2020-08-07T14:57:17.250 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1212} 2020-08-07T14:57:17.377 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1213} 2020-08-07T14:57:17.456 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1213} 2020-08-07T14:57:17.595 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1214} 2020-08-07T14:57:17.680 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1214} 2020-08-07T14:57:18.941 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10102, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2020-08-07T14:57:19.105 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10102, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2020-08-07T14:57:20.167 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "name": "prateek", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2020-08-07T14:57:20.342 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "name": "prateek", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2020-08-07T14:57:21.330 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "name": "jelopez", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10104} 2020-08-07T14:57:21.518 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "name": "jelopez", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10104} 2020-08-07T14:57:22.586 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10105, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "name": "dcain", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10105} 2020-08-07T14:57:22.731 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10105, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "name": "dcain", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10105} 2020-08-07T14:57:23.720 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10106, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "name": "sunny", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10106} 2020-08-07T14:57:23.870 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10106, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "name": "sunny", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10106} 2020-08-07T14:57:25.031 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10107, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10107} 2020-08-07T14:57:25.060 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10107, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10107} 2020-08-07T14:57:26.222 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10108, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "name": "aclewett", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10108} 2020-08-07T14:57:26.278 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10108, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "name": "aclewett", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10108} 2020-08-07T14:57:27.455 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10109, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "name": "cblum", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10109} 2020-08-07T14:57:27.467 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10109, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "name": "cblum", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10109} 2020-08-07T14:57:28.615 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10110, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10110} 2020-08-07T14:57:28.659 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10110, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10110} 2020-08-07T14:57:29.795 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10111, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10111} 2020-08-07T14:57:29.862 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10111, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10111} 2020-08-07T14:57:31.014 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10112, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10112} 2020-08-07T14:57:31.078 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10112, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10112} 2020-08-07T14:57:32.239 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10113, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10113} 2020-08-07T14:57:32.343 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10113, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10113} 2020-08-07T14:57:33.350 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10114, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "name": "kchang", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10114} 2020-08-07T14:57:33.478 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:57:33 +0000 (0:00:41.602) 0:00:50.874 ********* 2020-08-07T14:57:33.480 INFO:teuthology.task.ansible.out: 2020-08-07T14:57:33.534 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10114, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "name": "kchang", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10114} 2020-08-07T14:57:33.536 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-08-07T14:57:33.587 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:57:33 +0000 (0:00:00.107) 0:00:50.981 ********* 2020-08-07T14:57:33.623 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-08-07T14:57:33.670 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2020-08-07T14:57:33.672 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 2020-08-07T14:57:33.673 INFO:teuthology.task.ansible.out: 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b26 2020-08-07T14:57:33.675 INFO:teuthology.task.ansible.out:5c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivanci 2020-08-07T14:57:33.677 INFO:teuthology.task.ansible.out:ch", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0Wihhvk 2020-08-07T14:57:33.679 INFO:teuthology.task.ansible.out:KztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 2020-08-07T14:57:33.681 INFO:teuthology.task.ansible.out: HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfp 2020-08-07T14:57:33.683 INFO:teuthology.task.ansible.out:fKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1 2020-08-07T14:57:33.684 INFO:teuthology.task.ansible.out:GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73 2020-08-07T14:57:33.685 INFO:teuthology.task.ansible.out:dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localho 2020-08-07T14:57:33.687 INFO:teuthology.task.ansible.out:st eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn" 2020-08-07T14:57:33.701 INFO:teuthology.task.ansible.out:: "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"} 2020-08-07T14:57:33.705 INFO:teuthology.task.ansible.out:, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d 2020-08-07T14:57:33.706 INFO:teuthology.task.ansible.out:3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb 2020-08-07T14:57:33.713 INFO:teuthology.task.ansible.out:326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}]}, "changed": false} 2020-08-07T14:57:33.752 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:57:33 +0000 (0:00:00.165) 0:00:51.147 ********* 2020-08-07T14:57:33.793 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:57:33 +0000 (0:00:00.041) 0:00:51.188 ********* 2020-08-07T14:57:33.828 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-08-07T14:57:33.867 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:57:33 +0000 (0:00:00.073) 0:00:51.262 ********* 2020-08-07T14:57:33.954 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c 2020-08-07T14:57:33.956 INFO:teuthology.task.ansible.out:68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": " 2020-08-07T14:57:33.958 INFO:teuthology.task.ansible.out:ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 2020-08-07T14:57:33.960 INFO:teuthology.task.ansible.out:79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce66 2020-08-07T14:57:33.961 INFO:teuthology.task.ansible.out:8079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144 2020-08-07T14:57:33.964 INFO:teuthology.task.ansible.out:712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542 2020-08-07T14:57:33.965 INFO:teuthology.task.ansible.out:da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1 2020-08-07T14:57:33.972 INFO:teuthology.task.ansible.out:GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7c 2020-08-07T14:57:33.974 INFO:teuthology.task.ansible.out:ca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 6 2020-08-07T14:57:33.975 INFO:teuthology.task.ansible.out:4V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr 2020-08-07T14:57:33.977 INFO:teuthology.task.ansible.out:0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubl 2020-08-07T14:57:33.979 INFO:teuthology.task.ansible.out:i", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11 2020-08-07T14:57:33.980 INFO:teuthology.task.ansible.out:af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}]}, "changed": false} 2020-08-07T14:57:34.021 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:57:34 +0000 (0:00:00.154) 0:00:51.416 ********* 2020-08-07T14:57:35.021 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2020-08-07T14:57:35.022 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com -> localhost] => {"after": "822ea1561785feff19f519c0701e8aab1dcab883", "attempts": 1, "before": "822ea1561785feff19f519c0701e8aab1dcab883", "changed": false, "remote_url_changed": false} 2020-08-07T14:57:35.065 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:57:35 +0000 (0:00:01.043) 0:00:52.460 ********* 2020-08-07T14:57:35.157 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com -> localhost] => {"after": "822ea1561785feff19f519c0701e8aab1dcab883", "attempts": 1, "before": "822ea1561785feff19f519c0701e8aab1dcab883", "changed": false, "remote_url_changed": false} 2020-08-07T14:57:35.198 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:57:35 +0000 (0:00:00.126) 0:00:52.587 ********* 2020-08-07T14:57:35.470 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2020-08-07T14:57:35.570 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2020-08-07T14:57:35.781 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2020-08-07T14:57:35.873 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2020-08-07T14:57:36.081 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2020-08-07T14:57:36.196 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2020-08-07T14:57:36.392 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZ 2020-08-07T14:57:36.394 INFO:teuthology.task.ansible.out:tIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2020-08-07T14:57:36.478 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZ 2020-08-07T14:57:36.480 INFO:teuthology.task.ansible.out:tIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2020-08-07T14:57:36.689 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2020-08-07T14:57:36.835 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2020-08-07T14:57:37.011 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+ 2020-08-07T14:57:37.013 INFO:teuthology.task.ansible.out:o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2020-08-07T14:57:37.150 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+ 2020-08-07T14:57:37.162 INFO:teuthology.task.ansible.out:o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2020-08-07T14:57:37.340 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2020-08-07T14:57:37.439 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2020-08-07T14:57:37.646 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2020-08-07T14:57:37.764 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2020-08-07T14:57:37.939 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2020-08-07T14:57:37.951 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:57:38.063 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2020-08-07T14:57:38.073 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:57:38.251 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMor 2020-08-07T14:57:38.253 INFO:teuthology.task.ansible.out:bnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2020-08-07T14:57:38.354 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMor 2020-08-07T14:57:38.358 INFO:teuthology.task.ansible.out:bnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2020-08-07T14:57:38.537 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "a 2020-08-07T14:57:38.539 INFO:teuthology.task.ansible.out:emerson", "validate_certs": true} 2020-08-07T14:57:38.546 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:57:38.557 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:57:38.671 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "a 2020-08-07T14:57:38.673 INFO:teuthology.task.ansible.out:emerson", "validate_certs": true} 2020-08-07T14:57:38.676 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:57:38.699 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:57:38.835 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2020-08-07T14:57:38.985 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2020-08-07T14:57:39.150 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtS 2020-08-07T14:57:39.152 INFO:teuthology.task.ansible.out:oAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2020-08-07T14:57:39.317 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtS 2020-08-07T14:57:39.319 INFO:teuthology.task.ansible.out:oAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2020-08-07T14:57:39.449 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2020-08-07T14:57:39.603 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2020-08-07T14:57:39.742 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2020-08-07T14:57:39.909 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2020-08-07T14:57:40.060 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2020-08-07T14:57:40.195 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2020-08-07T14:57:40.347 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2020-08-07T14:57:40.512 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2020-08-07T14:57:40.661 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2020-08-07T14:57:40.826 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2020-08-07T14:57:40.963 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2020-08-07T14:57:41.096 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2020-08-07T14:57:41.302 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9 2020-08-07T14:57:41.304 INFO:teuthology.task.ansible.out:CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2020-08-07T14:57:41.424 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9 2020-08-07T14:57:41.426 INFO:teuthology.task.ansible.out:CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2020-08-07T14:57:41.585 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBY 2020-08-07T14:57:41.594 INFO:teuthology.task.ansible.out:B0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2020-08-07T14:57:41.757 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBY 2020-08-07T14:57:41.762 INFO:teuthology.task.ansible.out:B0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2020-08-07T14:57:41.889 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2020-08-07T14:57:42.079 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2020-08-07T14:57:42.218 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2020-08-07T14:57:42.400 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2020-08-07T14:57:42.539 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46 2020-08-07T14:57:42.541 INFO:teuthology.task.ansible.out:TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2020-08-07T14:57:42.708 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46 2020-08-07T14:57:42.710 INFO:teuthology.task.ansible.out:TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2020-08-07T14:57:42.870 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2020-08-07T14:57:43.062 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2020-08-07T14:57:43.179 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2020-08-07T14:57:43.338 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2020-08-07T14:57:43.471 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2020-08-07T14:57:43.650 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2020-08-07T14:57:43.796 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2020-08-07T14:57:43.968 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2020-08-07T14:57:44.086 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2020-08-07T14:57:44.270 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2020-08-07T14:57:44.393 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQv 2020-08-07T14:57:44.395 INFO:teuthology.task.ansible.out:UxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2020-08-07T14:57:44.563 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQv 2020-08-07T14:57:44.575 INFO:teuthology.task.ansible.out:UxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2020-08-07T14:57:44.698 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2020-08-07T14:57:44.853 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2020-08-07T14:57:44.985 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2020-08-07T14:57:45.150 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2020-08-07T14:57:45.299 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2020-08-07T14:57:45.454 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2020-08-07T14:57:45.598 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2020-08-07T14:57:45.778 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2020-08-07T14:57:45.893 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2020-08-07T14:57:46.067 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2020-08-07T14:57:46.213 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2020-08-07T14:57:46.389 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2020-08-07T14:57:46.510 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2020-08-07T14:57:46.735 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2020-08-07T14:57:46.839 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2020-08-07T14:57:47.033 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2020-08-07T14:57:47.151 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBUVNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2020-08-07T14:57:47.323 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBUVNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2020-08-07T14:57:47.444 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1 2020-08-07T14:57:47.451 INFO:teuthology.task.ansible.out:OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2020-08-07T14:57:47.631 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1 2020-08-07T14:57:47.633 INFO:teuthology.task.ansible.out:OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2020-08-07T14:57:47.738 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyK 2020-08-07T14:57:47.739 INFO:teuthology.task.ansible.out:z2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2020-08-07T14:57:47.919 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyK 2020-08-07T14:57:47.921 INFO:teuthology.task.ansible.out:z2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2020-08-07T14:57:48.042 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2020-08-07T14:57:48.210 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2020-08-07T14:57:48.360 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2020-08-07T14:57:48.514 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2020-08-07T14:57:48.664 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2020-08-07T14:57:48.811 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2020-08-07T14:57:48.965 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nu 2020-08-07T14:57:48.974 INFO:teuthology.task.ansible.out:ll, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2020-08-07T14:57:49.166 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nu 2020-08-07T14:57:49.174 INFO:teuthology.task.ansible.out:ll, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2020-08-07T14:57:49.279 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjE 2020-08-07T14:57:49.283 INFO:teuthology.task.ansible.out:fdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+d 2020-08-07T14:57:49.285 INFO:teuthology.task.ansible.out:TzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2020-08-07T14:57:49.470 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjE 2020-08-07T14:57:49.478 INFO:teuthology.task.ansible.out:fdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+d 2020-08-07T14:57:49.481 INFO:teuthology.task.ansible.out:TzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2020-08-07T14:57:49.562 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2020-08-07T14:57:49.778 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2020-08-07T14:57:49.896 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2020-08-07T14:57:50.092 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2020-08-07T14:57:50.205 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2020-08-07T14:57:50.390 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2020-08-07T14:57:50.515 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2020-08-07T14:57:50.728 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2020-08-07T14:57:50.834 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2020-08-07T14:57:51.074 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2020-08-07T14:57:51.136 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home 2020-08-07T14:57:51.138 INFO:teuthology.task.ansible.out:/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2020-08-07T14:57:51.382 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home 2020-08-07T14:57:51.390 INFO:teuthology.task.ansible.out:/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2020-08-07T14:57:51.482 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2020-08-07T14:57:51.690 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2020-08-07T14:57:51.766 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2020-08-07T14:57:52.001 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2020-08-07T14:57:52.092 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": " 2020-08-07T14:57:52.097 INFO:teuthology.task.ansible.out:gabrioux", "validate_certs": true} 2020-08-07T14:57:52.357 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": " 2020-08-07T14:57:52.362 INFO:teuthology.task.ansible.out:gabrioux", "validate_certs": true} 2020-08-07T14:57:52.434 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2020-08-07T14:57:52.654 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2020-08-07T14:57:52.737 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2020-08-07T14:57:52.962 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2020-08-07T14:57:53.067 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kk 2020-08-07T14:57:53.069 INFO:teuthology.task.ansible.out:ke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2020-08-07T14:57:53.271 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kk 2020-08-07T14:57:53.274 INFO:teuthology.task.ansible.out:ke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2020-08-07T14:57:53.372 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2020-08-07T14:57:53.585 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2020-08-07T14:57:53.679 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2020-08-07T14:57:53.881 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2020-08-07T14:57:53.988 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2020-08-07T14:57:54.172 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2020-08-07T14:57:54.283 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2020-08-07T14:57:54.485 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2020-08-07T14:57:54.611 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2020-08-07T14:57:54.781 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2020-08-07T14:57:54.894 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2020-08-07T14:57:55.077 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2020-08-07T14:57:55.227 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2020-08-07T14:57:55.389 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2020-08-07T14:57:55.521 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2020-08-07T14:57:55.684 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2020-08-07T14:57:55.839 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2020-08-07T14:57:56.004 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2020-08-07T14:57:56.153 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2020-08-07T14:57:56.299 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2020-08-07T14:57:56.470 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2020-08-07T14:57:56.594 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2020-08-07T14:57:56.792 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2020-08-07T14:57:56.885 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2020-08-07T14:57:57.076 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2020-08-07T14:57:57.198 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2020-08-07T14:57:57.406 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2020-08-07T14:57:57.515 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2020-08-07T14:57:57.731 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2020-08-07T14:57:57.818 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2020-08-07T14:57:58.034 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2020-08-07T14:57:58.140 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2020-08-07T14:57:58.322 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2020-08-07T14:57:58.442 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2020-08-07T14:57:58.627 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2020-08-07T14:57:58.739 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2020-08-07T14:57:58.911 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2020-08-07T14:57:59.029 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2020-08-07T14:57:59.196 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2020-08-07T14:57:59.309 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2020-08-07T14:57:59.492 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2020-08-07T14:57:59.609 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2020-08-07T14:57:59.788 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2020-08-07T14:57:59.883 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2020-08-07T14:58:00.077 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2020-08-07T14:58:00.177 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2020-08-07T14:58:00.416 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2020-08-07T14:58:00.480 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2020-08-07T14:58:00.688 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2020-08-07T14:58:00.803 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2020-08-07T14:58:00.985 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.locald 2020-08-07T14:58:00.988 INFO:teuthology.task.ansible.out:omain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2020-08-07T14:58:01.128 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.locald 2020-08-07T14:58:01.132 INFO:teuthology.task.ansible.out:omain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2020-08-07T14:58:01.279 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq 2020-08-07T14:58:01.291 INFO:teuthology.task.ansible.out:2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2020-08-07T14:58:01.420 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq 2020-08-07T14:58:01.422 INFO:teuthology.task.ansible.out:2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2020-08-07T14:58:01.588 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2020-08-07T14:58:01.702 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2020-08-07T14:58:01.890 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2020-08-07T14:58:02.003 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2020-08-07T14:58:02.169 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2020-08-07T14:58:02.315 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2020-08-07T14:58:02.504 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2020-08-07T14:58:02.623 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2020-08-07T14:58:02.803 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2020-08-07T14:58:02.935 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2020-08-07T14:58:03.095 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2020-08-07T14:58:03.242 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2020-08-07T14:58:03.419 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2020-08-07T14:58:03.532 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2020-08-07T14:58:03.695 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2020-08-07T14:58:03.822 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2020-08-07T14:58:04.021 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2020-08-07T14:58:04.124 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2020-08-07T14:58:04.332 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2020-08-07T14:58:04.406 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2020-08-07T14:58:04.606 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2020-08-07T14:58:04.695 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2020-08-07T14:58:04.902 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2020-08-07T14:58:05.005 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2020-08-07T14:58:05.183 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2020-08-07T14:58:05.334 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2020-08-07T14:58:05.339 INFO:teuthology.task.ansible.out: 2020-08-07T14:58:05.489 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2020-08-07T14:58:05.635 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2020-08-07T14:58:05.772 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", 2020-08-07T14:58:05.774 INFO:teuthology.task.ansible.out:"manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2020-08-07T14:58:05.920 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", 2020-08-07T14:58:05.922 INFO:teuthology.task.ansible.out:"manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2020-08-07T14:58:06.072 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2020-08-07T14:58:06.211 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2020-08-07T14:58:06.367 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2020-08-07T14:58:06.523 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2020-08-07T14:58:06.676 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2020-08-07T14:58:06.840 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2020-08-07T14:58:07.001 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2020-08-07T14:58:07.149 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2020-08-07T14:58:07.296 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2020-08-07T14:58:07.451 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2020-08-07T14:58:07.607 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2020-08-07T14:58:07.789 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2020-08-07T14:58:07.909 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2020-08-07T14:58:08.078 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2020-08-07T14:58:08.242 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2020-08-07T14:58:08.245 INFO:teuthology.task.ansible.out: 2020-08-07T14:58:08.440 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2020-08-07T14:58:08.554 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2020-08-07T14:58:08.739 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2020-08-07T14:58:08.838 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2020-08-07T14:58:09.030 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2020-08-07T14:58:09.163 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2020-08-07T14:58:09.341 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2020-08-07T14:58:09.456 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2020-08-07T14:58:09.670 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2020-08-07T14:58:09.810 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2020-08-07T14:58:09.955 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2020-08-07T14:58:10.100 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2020-08-07T14:58:10.256 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2020-08-07T14:58:10.408 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2020-08-07T14:58:10.556 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2020-08-07T14:58:10.695 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2020-08-07T14:58:10.854 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2020-08-07T14:58:10.986 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2020-08-07T14:58:11.135 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2020-08-07T14:58:11.294 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2020-08-07T14:58:11.447 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2020-08-07T14:58:11.602 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpb 2020-08-07T14:58:11.605 INFO:teuthology.task.ansible.out:cmjLd+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2020-08-07T14:58:11.735 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpb 2020-08-07T14:58:11.750 INFO:teuthology.task.ansible.out:cmjLd+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2020-08-07T14:58:11.896 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2020-08-07T14:58:12.046 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2020-08-07T14:58:12.203 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2020-08-07T14:58:12.337 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2020-08-07T14:58:12.500 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2020-08-07T14:58:12.643 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2020-08-07T14:58:12.802 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2020-08-07T14:58:12.950 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2020-08-07T14:58:13.099 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2020-08-07T14:58:13.259 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2020-08-07T14:58:13.412 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2020-08-07T14:58:13.565 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2020-08-07T14:58:13.725 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2020-08-07T14:58:13.851 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:13 +0000 (0:00:38.655) 0:01:31.242 ********* 2020-08-07T14:58:13.869 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2020-08-07T14:58:13.934 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:13.936 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:13.983 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:13 +0000 (0:00:00.134) 0:01:31.377 ********* 2020-08-07T14:58:14.006 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr 2020-08-07T14:58:14.008 INFO:teuthology.task.ansible.out:+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_ 2020-08-07T14:58:14.009 INFO:teuthology.task.ansible.out:loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:14.074 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:14.085 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:14.094 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:14.108 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:14.130 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:14.149 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:14.159 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratzas@laptop 65tzczvq6U9MaLuTtZUWJA 36cc4b5af314d8d2ae1cc974086b62fd4d9a722b1469f7cf00ba647241b834c9"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:14.166 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:14.298 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2020-08-07T14:58:14.314 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:14.468 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2020-08-07T14:58:14.486 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:14.506 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:14.616 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2020-08-07T14:58:14.792 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2020-08-07T14:58:14.907 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2020-08-07T14:58:14.927 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:14.938 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:14.970 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ub 2020-08-07T14:58:14.976 INFO:teuthology.task.ansible.out:untu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:14.995 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.009 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.014 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.027 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.029 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.047 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.059 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.070 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.081 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.092 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.110 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.123 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} ok: [smithi096.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b2 2020-08-07T14:58:15.134 INFO:teuthology.task.ansible.out:68bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.142 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.152 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.187 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.229 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.286 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vas 2020-08-07T14:58:15.294 INFO:teuthology.task.ansible.out:u", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01 2020-08-07T14:58:15.296 INFO:teuthology.task.ansible.out:e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.310 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ub 2020-08-07T14:58:15.312 INFO:teuthology.task.ansible.out:untu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.315 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.315 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.316 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96 2020-08-07T14:58:15.318 INFO:teuthology.task.ansible.out:a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.358 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.364 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': ' 2020-08-07T14:58:15.370 INFO:teuthology.task.ansible.out:gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.389 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.fr 2020-08-07T14:58:15.394 INFO:teuthology.task.ansible.out:ont.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen 2020-08-07T14:58:15.402 INFO:teuthology.task.ansible.out:@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooa 2020-08-07T14:58:15.404 INFO:teuthology.task.ansible.out:RzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovp 2020-08-07T14:58:15.410 INFO:teuthology.task.ansible.out:n': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.439 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.462 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.471 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.481 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.498 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.510 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.529 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.544 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 2020-08-07T14:58:15.546 INFO:teuthology.task.ansible.out:31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.589 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.600 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.612 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.617 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_lo 2020-08-07T14:58:15.619 INFO:teuthology.task.ansible.out:op_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditi 2020-08-07T14:58:15.620 INFO:teuthology.task.ansible.out:onal result was False"} 2020-08-07T14:58:15.630 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.640 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.677 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (it 2020-08-07T14:58:15.680 INFO:teuthology.task.ansible.out:em={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493d 2020-08-07T14:58:15.682 INFO:teuthology.task.ansible.out:a361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ans 2020-08-07T14:58:15.686 INFO:teuthology.task.ansible.out:ible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional re 2020-08-07T14:58:15.689 INFO:teuthology.task.ansible.out:sult was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.693 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.706 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.716 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.730 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.740 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.758 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.782 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'toabctl' 2020-08-07T14:58:15.786 INFO:teuthology.task.ansible.out:, 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.819 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23 2020-08-07T14:58:15.823 INFO:teuthology.task.ansible.out:F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c1641 2020-08-07T14:58:15.834 INFO:teuthology.task.ansible.out:5a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smith 2020-08-07T14:58:15.836 INFO:teuthology.task.ansible.out:i096.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-lap 2020-08-07T14:58:15.839 INFO:teuthology.task.ansible.out:top ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thin 2020-08-07T14:58:15.844 INFO:teuthology.task.ansible.out:kpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWo 2020-08-07T14:58:15.847 INFO:teuthology.task.ansible.out:JeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.851 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.863 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.876 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'prateek', 2020-08-07T14:58:15.878 INFO:teuthology.task.ansible.out: 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.886 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.900 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.910 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.912 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.914 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.927 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.929 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8 2020-08-07T14:58:15.937 INFO:teuthology.task.ansible.out:kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.942 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.954 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.955 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.957 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.971 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:15.983 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:16.010 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:16.046 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:16 +0000 (0:00:02.063) 0:01:33.441 ********* 2020-08-07T14:58:16.061 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:16.074 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:16.110 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:16.143 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:16 +0000 (0:00:00.092) 0:01:33.533 ********* 2020-08-07T14:58:16.153 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@hrk-thinkpad 6DWhbAZZDUo789PultFaJw f586e852c2dab5f56e3e12237dadd8f76f61d9136479329698e6f7dcd91f9034"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:16.155 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:16.201 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:16.205 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBo 2020-08-07T14:58:16.209 INFO:teuthology.task.ansible.out:x 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:16.237 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:16.250 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:16.265 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:16.283 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:16.298 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:16.352 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:16 +0000 (0:00:00.207) 0:01:33.741 ********* 2020-08-07T14:58:16.395 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:16.436 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:16 +0000 (0:00:00.089) 0:01:33.831 ********* 2020-08-07T14:58:16.458 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-08-07T14:58:16.460 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2020-08-07T14:58:16.616 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2020-08-07T14:58:16.695 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2020-08-07T14:58:16.822 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2020-08-07T14:58:16.913 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2020-08-07T14:58:17.030 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2020-08-07T14:58:17.113 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2020-08-07T14:58:17.233 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2020-08-07T14:58:17.331 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2020-08-07T14:58:17.448 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2020-08-07T14:58:17.552 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2020-08-07T14:58:17.684 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2020-08-07T14:58:17.766 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2020-08-07T14:58:17.892 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2020-08-07T14:58:17.998 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2020-08-07T14:58:18.123 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2020-08-07T14:58:18.223 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2020-08-07T14:58:18.339 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2020-08-07T14:58:18.413 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2020-08-07T14:58:18.554 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2020-08-07T14:58:18.623 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2020-08-07T14:58:18.755 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2020-08-07T14:58:18.824 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2020-08-07T14:58:18.964 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2020-08-07T14:58:19.071 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2020-08-07T14:58:19.192 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2020-08-07T14:58:19.282 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2020-08-07T14:58:19.394 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2020-08-07T14:58:19.523 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2020-08-07T14:58:19.635 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2020-08-07T14:58:19.754 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2020-08-07T14:58:19.830 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2020-08-07T14:58:19.994 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2020-08-07T14:58:20.066 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2020-08-07T14:58:20.223 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2020-08-07T14:58:20.296 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2020-08-07T14:58:20.423 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2020-08-07T14:58:20.512 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2020-08-07T14:58:20.646 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2020-08-07T14:58:20.720 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2020-08-07T14:58:20.848 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2020-08-07T14:58:20.960 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2020-08-07T14:58:21.049 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2020-08-07T14:58:21.186 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2020-08-07T14:58:21.272 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2020-08-07T14:58:21.401 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2020-08-07T14:58:21.467 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2020-08-07T14:58:21.606 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2020-08-07T14:58:21.667 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2020-08-07T14:58:21.818 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2020-08-07T14:58:21.879 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2020-08-07T14:58:22.021 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2020-08-07T14:58:22.100 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2020-08-07T14:58:22.223 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2020-08-07T14:58:22.301 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2020-08-07T14:58:22.426 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2020-08-07T14:58:22.491 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2020-08-07T14:58:22.630 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2020-08-07T14:58:22.706 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2020-08-07T14:58:22.850 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2020-08-07T14:58:22.906 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2020-08-07T14:58:23.063 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2020-08-07T14:58:23.105 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2020-08-07T14:58:23.256 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2020-08-07T14:58:23.347 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2020-08-07T14:58:23.461 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2020-08-07T14:58:23.546 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2020-08-07T14:58:23.655 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2020-08-07T14:58:23.763 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2020-08-07T14:58:23.873 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2020-08-07T14:58:23.997 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2020-08-07T14:58:24.105 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2020-08-07T14:58:24.254 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2020-08-07T14:58:24.351 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2020-08-07T14:58:24.471 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2020-08-07T14:58:24.563 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2020-08-07T14:58:24.703 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2020-08-07T14:58:24.770 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2020-08-07T14:58:24.912 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2020-08-07T14:58:24.988 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2020-08-07T14:58:25.138 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2020-08-07T14:58:25.188 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2020-08-07T14:58:25.329 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2020-08-07T14:58:25.407 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2020-08-07T14:58:25.478 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:25 +0000 (0:00:09.041) 0:01:42.873 ********* 2020-08-07T14:58:25.531 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2020-08-07T14:58:25.742 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2020-08-07T14:58:25.814 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:25 +0000 (0:00:00.336) 0:01:43.209 ********* 2020-08-07T14:58:26.137 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** changed: [smithi074.front.sepia.ceph.com] => {"changed": true, "checksum": "dfabc4e1b5ab2587587be9421b71f186491dffbb", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "89d6f07086900b48ad59882973396e6f", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1596812305.5104723-232838850766097/source", "state": "file", "uid": 0} 2020-08-07T14:58:26.189 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:26 +0000 (0:00:00.374) 0:01:43.583 ********* 2020-08-07T14:58:26.227 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-08-07T14:58:26.229 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => { "msg": "Host smithi074.front.sepia.ceph.com is running RedHat 8.1 (Ootpa)" } 2020-08-07T14:58:26.269 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:26 +0000 (0:00:00.080) 0:01:43.663 ********* 2020-08-07T14:58:26.336 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2020-08-07T14:58:26.338 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/redhat_8.yml) => {"ansible_facts": {"nrpe_selinux_packages": ["python3-libsemanage", "python3-policycoreutils"], "rhsm_repos": ["rhel-8-for-x86_64-baseos-rpms", "rhel-8-for-x86_64-appstream-rpms"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/redhat_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/redhat_8.yml"} 2020-08-07T14:58:26.376 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:26 +0000 (0:00:00.107) 0:01:43.771 ********* 2020-08-07T14:58:26.420 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-08-07T14:58:26.422 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:26.461 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:26 +0000 (0:00:00.084) 0:01:43.856 ********* 2020-08-07T14:58:26.472 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-08-07T14:58:26.474 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "checksum": "dfabc4e1b5ab2587587be9421b71f186491dffbb", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "89d6f07086900b48ad59882973396e6f", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1596812305.854913-128814439237098/source", "state": "file", "uid": 0} 2020-08-07T14:58:26.533 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:26 +0000 (0:00:00.072) 0:01:43.928 ********* 2020-08-07T14:58:26.576 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-08-07T14:58:26.578 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => { "msg": "Host smithi096.front.sepia.ceph.com is running RedHat 8.1 (Ootpa)" } 2020-08-07T14:58:26.622 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:26 +0000 (0:00:00.088) 0:01:44.017 ********* 2020-08-07T14:58:26.666 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2020-08-07T14:58:26.668 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/redhat_8.yml) => {"ansible_facts": {"nrpe_selinux_packages": ["python3-libsemanage", "python3-policycoreutils"], "rhsm_repos": ["rhel-8-for-x86_64-baseos-rpms", "rhel-8-for-x86_64-appstream-rpms"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/redhat_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/redhat_8.yml"} 2020-08-07T14:58:26.705 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:26 +0000 (0:00:00.083) 0:01:44.100 ********* 2020-08-07T14:58:26.748 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:26.812 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:26 +0000 (0:00:00.106) 0:01:44.207 ********* 2020-08-07T14:58:27.044 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-08-07T14:58:27.046 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.341781", "end": "2020-08-07 14:58:27.012622", "rc": 0, "start": "2020-08-07 14:58:26.670841", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-08-07T14:58:27.081 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:27 +0000 (0:00:00.269) 0:01:44.476 ********* 2020-08-07T14:58:27.138 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:27.219 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:27 +0000 (0:00:00.121) 0:01:44.597 ********* 2020-08-07T14:58:27.251 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:27.306 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:27 +0000 (0:00:00.099) 0:01:44.697 ********* 2020-08-07T14:58:27.349 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:27.387 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:27 +0000 (0:00:00.084) 0:01:44.782 ********* 2020-08-07T14:58:27.402 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.335881", "end": "2020-08-07 14:58:27.355031", "rc": 0, "start": "2020-08-07 14:58:27.019150", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-08-07T14:58:27.438 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:27 +0000 (0:00:00.050) 0:01:44.833 ********* 2020-08-07T14:58:27.449 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:27.497 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:27 +0000 (0:00:00.059) 0:01:44.893 ********* 2020-08-07T14:58:27.510 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:27.566 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:27 +0000 (0:00:00.068) 0:01:44.961 ********* 2020-08-07T14:58:27.581 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:27.637 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:27 +0000 (0:00:00.070) 0:01:45.032 ********* 2020-08-07T14:58:27.653 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2020-08-07T14:58:27.655 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:27.694 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:27 +0000 (0:00:00.056) 0:01:45.089 ********* 2020-08-07T14:58:27.707 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-08-07T14:58:27.709 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"ansible_facts": {"entitlements_path": "/etc/ansible/secrets/entitlements.yml"}, "changed": false} 2020-08-07T14:58:27.749 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:27 +0000 (0:00:00.055) 0:01:45.144 ********* 2020-08-07T14:58:27.759 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:27.797 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:27 +0000 (0:00:00.047) 0:01:45.192 ********* 2020-08-07T14:58:27.831 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2020-08-07T14:58:27.835 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} ok: [smithi074.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-08-07T14:58:27.871 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:27 +0000 (0:00:00.073) 0:01:45.265 ********* 2020-08-07T14:58:27.880 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2020-08-07T14:58:27.889 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:27.958 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:27 +0000 (0:00:00.087) 0:01:45.353 ********* 2020-08-07T14:58:27.968 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2020-08-07T14:58:27.970 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"ansible_facts": {"have_entitlements": true}, "changed": false} 2020-08-07T14:58:28.020 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:28 +0000 (0:00:00.061) 0:01:45.414 ********* 2020-08-07T14:58:28.031 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:28.085 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:28 +0000 (0:00:00.065) 0:01:45.480 ********* 2020-08-07T14:58:28.087 INFO:teuthology.task.ansible.out: 2020-08-07T14:58:28.146 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-08-07T14:58:28.149 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"entitlements_path": "/etc/ansible/secrets/entitlements.yml"}, "changed": false} 2020-08-07T14:58:28.185 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:28 +0000 (0:00:00.100) 0:01:45.580 ********* 2020-08-07T14:58:28.268 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** ok: [smithi096.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} ok: [smithi096.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-08-07T14:58:28.309 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:28 +0000 (0:00:00.123) 0:01:45.704 ********* 2020-08-07T14:58:28.366 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"have_entitlements": true}, "changed": false} 2020-08-07T14:58:28.400 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:28 +0000 (0:00:00.091) 0:01:45.795 ********* 2020-08-07T14:58:28.403 INFO:teuthology.task.ansible.out: 2020-08-07T14:58:29.371 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2020-08-07T14:58:29.374 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed /root/.ansible/tmp/ansible-moduletmp-1596812309.2144654-1mwqamq2/katello-ca-consumer-latest.noarchf1s1bx_0.rpm"]} 2020-08-07T14:58:29.428 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:29 +0000 (0:00:01.028) 0:01:46.823 ********* 2020-08-07T14:58:29.439 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed /root/.ansible/tmp/ansible-moduletmp-1596812309.2286198-kn1k7hrn/katello-ca-consumer-latest.noarchzuk7daos.rpm"]} 2020-08-07T14:58:29.481 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:29 +0000 (0:00:00.052) 0:01:46.876 ********* 2020-08-07T14:58:30.371 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** fatal: [smithi096.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} ...ignoring 2020-08-07T14:58:30.452 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:30 +0000 (0:00:00.970) 0:01:47.847 ********* 2020-08-07T14:58:30.464 INFO:teuthology.task.ansible.out:fatal: [smithi074.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} ...ignoring 2020-08-07T14:58:30.514 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:30 +0000 (0:00:00.059) 0:01:47.906 ********* 2020-08-07T14:58:30.530 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"rhsm_registered": false}, "changed": false} 2020-08-07T14:58:30.574 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:30 +0000 (0:00:00.062) 0:01:47.968 ********* 2020-08-07T14:58:30.596 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"ansible_facts": {"rhsm_registered": false}, "changed": false} 2020-08-07T14:58:30.641 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:30 +0000 (0:00:00.067) 0:01:48.036 ********* 2020-08-07T14:58:37.078 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** changed: [smithi074.front.sepia.ceph.com] => {"attempts": 1, "censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-08-07T14:58:37.145 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:37 +0000 (0:00:06.493) 0:01:54.529 ********* 2020-08-07T14:58:37.189 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** ok: [smithi074.front.sepia.ceph.com] => {"ansible_facts": {"rhsm_registered": true}, "changed": false} 2020-08-07T14:58:37.229 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:37 +0000 (0:00:00.092) 0:01:54.622 ********* 2020-08-07T14:58:37.386 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2020-08-07T14:58:37.389 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"attempts": 1, "censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-08-07T14:58:37.438 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:37 +0000 (0:00:00.208) 0:01:54.830 ********* 2020-08-07T14:58:37.506 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2020-08-07T14:58:37.508 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"rhsm_registered": true}, "changed": false} 2020-08-07T14:58:37.566 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:37 +0000 (0:00:00.129) 0:01:54.960 ********* 2020-08-07T14:58:38.081 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2020-08-07T14:58:38.097 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "cmd": "subscription-manager release --list | grep -E '[0-9]'", "delta": "0:00:00.621103", "end": "2020-08-07 14:58:38.049086", "failed_when_result": false, "rc": 0, "start": "2020-08-07 14:58:37.427983", "stderr": "", "stderr_lines": [], "stdout": "7.4\n7.5\n7.6\n7.7\n7.8\n7Server\n8\n8.0\n8.1\n8.2", "stdout_lines": ["7.4", "7.5", "7.6", "7.7", "7.8", "7Server", "8", "8.0", "8.1", "8.2"]} 2020-08-07T14:58:38.142 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:38 +0000 (0:00:00.573) 0:01:55.533 ********* 2020-08-07T14:58:38.196 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:38.242 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:38 +0000 (0:00:00.103) 0:01:55.637 ********* 2020-08-07T14:58:38.308 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:38.358 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:38 +0000 (0:00:00.104) 0:01:55.741 ********* 2020-08-07T14:58:38.413 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [smithi074.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-08-07T14:58:38.440 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:38 +0000 (0:00:00.094) 0:01:55.836 ********* 2020-08-07T14:58:38.454 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "cmd": "subscription-manager release --list | grep -E '[0-9]'", "delta": "0:00:00.624829", "end": "2020-08-07 14:58:38.396003", "failed_when_result": false, "rc": 0, "start": "2020-08-07 14:58:37.771174", "stderr": "", "stderr_lines": [], "stdout": "7.4\n7.5\n7.6\n7.7\n7.8\n7Server\n8\n8.0\n8.1\n8.2", "stdout_lines": ["7.4", "7.5", "7.6", "7.7", "7.8", "7Server", "8", "8.0", "8.1", "8.2"]} 2020-08-07T14:58:38.513 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:38 +0000 (0:00:00.067) 0:01:55.903 ********* 2020-08-07T14:58:38.520 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:38.577 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:38 +0000 (0:00:00.069) 0:01:55.973 ********* 2020-08-07T14:58:38.593 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:38.647 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:38 +0000 (0:00:00.061) 0:01:56.034 ********* 2020-08-07T14:58:38.698 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:38.746 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:38 +0000 (0:00:00.102) 0:01:56.136 ********* 2020-08-07T14:58:38.789 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [smithi096.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-08-07T14:58:38.830 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:38 +0000 (0:00:00.086) 0:01:56.223 ********* 2020-08-07T14:58:38.876 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:38.921 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:38 +0000 (0:00:00.092) 0:01:56.315 ********* 2020-08-07T14:58:48.807 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2020-08-07T14:58:48.811 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "cmd": "subscription-manager repos --list | grep -B4 'Enabled:.*1' | grep 'Repo ID:' | sed -e 's/Repo ID:\\s*\\(.*\\)/\\1/' | sort", "delta": "0:00:09.995596", "end": "2020-08-07 14:58:48.775497", "rc": 0, "start": "2020-08-07 14:58:38.779901", "stderr": "", "stderr_lines": [], "stdout": "rhel-8-for-x86_64-appstream-rpms\nrhel-8-for-x86_64-baseos-rpms", "stdout_lines": ["rhel-8-for-x86_64-appstream-rpms", "rhel-8-for-x86_64-baseos-rpms"]} 2020-08-07T14:58:48.846 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:48 +0000 (0:00:09.926) 0:02:06.241 ********* 2020-08-07T14:58:48.895 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ ok: [smithi074.front.sepia.ceph.com] => {"ansible_facts": {"repo_list": ["rhel-8-for-x86_64-appstream-rpms", "rhel-8-for-x86_64-baseos-rpms"]}, "changed": false} 2020-08-07T14:58:48.931 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:48 +0000 (0:00:00.084) 0:02:06.326 ********* 2020-08-07T14:58:48.990 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:49.040 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:49 +0000 (0:00:00.094) 0:02:06.421 ********* 2020-08-07T14:58:49.084 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** ok: [smithi074.front.sepia.ceph.com] => {"ansible_facts": {"replace_repos": false}, "changed": false} 2020-08-07T14:58:49.134 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:49 +0000 (0:00:00.094) 0:02:06.516 ********* 2020-08-07T14:58:49.181 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* ok: [smithi074.front.sepia.ceph.com] => {"ansible_facts": {"replace_repos": true}, "changed": false} 2020-08-07T14:58:49.218 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:49 +0000 (0:00:00.094) 0:02:06.610 ********* 2020-08-07T14:58:49.650 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "cmd": "subscription-manager repos --list | grep -B4 'Enabled:.*1' | grep 'Repo ID:' | sed -e 's/Repo ID:\\s*\\(.*\\)/\\1/' | sort", "delta": "0:00:10.481639", "end": "2020-08-07 14:58:49.619480", "rc": 0, "start": "2020-08-07 14:58:39.137841", "stderr": "", "stderr_lines": [], "stdout": "rhel-8-for-x86_64-appstream-rpms\nrhel-8-for-x86_64-baseos-rpms", "stdout_lines": ["rhel-8-for-x86_64-appstream-rpms", "rhel-8-for-x86_64-baseos-rpms"]} 2020-08-07T14:58:49.687 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:49 +0000 (0:00:00.472) 0:02:07.083 ********* 2020-08-07T14:58:49.751 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2020-08-07T14:58:49.753 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"repo_list": ["rhel-8-for-x86_64-appstream-rpms", "rhel-8-for-x86_64-baseos-rpms"]}, "changed": false} 2020-08-07T14:58:49.806 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:49 +0000 (0:00:00.109) 0:02:07.192 ********* 2020-08-07T14:58:49.866 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:58:49.914 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:49 +0000 (0:00:00.103) 0:02:07.296 ********* 2020-08-07T14:58:49.974 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2020-08-07T14:58:49.976 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"replace_repos": false}, "changed": false} 2020-08-07T14:58:50.023 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:50 +0000 (0:00:00.121) 0:02:07.418 ********* 2020-08-07T14:58:50.086 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"replace_repos": true}, "changed": false} 2020-08-07T14:58:50.145 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:50 +0000 (0:00:00.112) 0:02:07.531 ********* 2020-08-07T14:58:53.359 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** changed: [smithi074.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-08-07T14:58:53.398 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:53 +0000 (0:00:03.260) 0:02:10.791 ********* 2020-08-07T14:58:54.403 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-08-07T14:58:54.456 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:54 +0000 (0:00:01.058) 0:02:11.850 ********* 2020-08-07T14:58:57.243 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2020-08-07T14:58:57.266 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => {"changed": true, "cmd": ["subscription-manager", "repos", "--enable=rhel-8-for-x86_64-baseos-rpms", "--enable=rhel-8-for-x86_64-appstream-rpms"], "delta": "0:00:03.583550", "end": "2020-08-07 14:58:57.214327", "rc": 0, "start": "2020-08-07 14:58:53.630777", "stderr": "", "stderr_lines": [], "stdout": "Internal Server Error): Required lock is already taken by other running tasks.\nPlease inspect their state, fix their errors and resume them.\n\nRequired lock: update\nConflicts with tasks:\n https://satellite.front.sepia.ceph.com/foreman_tasks/tasks/749819f8 2692 451d 8438 ae1a2ccaa613\n\nRepository 'rhel-8-for-x86_64-baseos-rpms' is enabled for this system.\nRepository 'rhel-8-for-x86_64-appstream-rpms' is enabled for this system.", "stdout_lines": ["Internal Server Error): Required lock is already taken by other running tasks.", "Please inspect their state, fix their errors and resume them.", "", "Required lock: update", "Conflicts with tasks:", " https://satellite.front.sepia.ceph.com/foreman_tasks/tasks/749819f8 2692 451d 8438 ae1a2ccaa613", "", "Repository 'rhel-8-for-x86_64-baseos-rpms' is enabled for this system.", "Repository 'rhel-8-for-x86_64-appstream-rpms' is enabled for this system."]} 2020-08-07T14:58:57.302 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:57 +0000 (0:00:02.839) 0:02:14.690 ********* 2020-08-07T14:58:57.972 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "path": "/etc/yum.repos.d/cd.repo", "state": "absent"} 2020-08-07T14:58:58.025 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:58 +0000 (0:00:00.729) 0:02:15.420 ********* 2020-08-07T14:58:58.171 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2020-08-07T14:58:58.173 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "cmd": ["subscription-manager", "repos", "--enable=rhel-8-for-x86_64-baseos-rpms", "--enable=rhel-8-for-x86_64-appstream-rpms"], "delta": "0:00:03.455765", "end": "2020-08-07 14:58:58.139413", "rc": 0, "start": "2020-08-07 14:58:54.683648", "stderr": "", "stderr_lines": [], "stdout": "Internal Server Error): Required lock is already taken by other running tasks.\nPlease inspect their state, fix their errors and resume them.\n\nRequired lock: update\nConflicts with tasks:\n https://satellite.front.sepia.ceph.com/foreman_tasks/tasks/3325789a 9d17 4039 adf1 1718b41c07fb\n\nRepository 'rhel-8-for-x86_64-baseos-rpms' is enabled for this system.\nRepository 'rhel-8-for-x86_64-appstream-rpms' is enabled for this system.", "stdout_lines": ["Internal Server Error): Required lock is already taken by other running tasks.", "Please inspect their state, fix their errors and resume them.", "", "Required lock: update", "Conflicts with tasks:", " https://satellite.front.sepia.ceph.com/foreman_tasks/tasks/3325789a 9d17 4039 adf1 1718b41c07fb", "", "Repository 'rhel-8-for-x86_64-baseos-rpms' is enabled for this system.", "Repository 'rhel-8-for-x86_64-appstream-rpms' is enabled for this system."]} 2020-08-07T14:58:58.248 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:58 +0000 (0:00:00.223) 0:02:15.643 ********* 2020-08-07T14:58:58.292 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2020-08-07T14:58:58.295 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-08-07T14:58:58.360 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:58 +0000 (0:00:00.098) 0:02:15.741 ********* 2020-08-07T14:58:58.497 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2020-08-07T14:58:58.499 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "path": "/etc/yum.repos.d/cd.repo", "state": "absent"} 2020-08-07T14:58:58.543 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:58 +0000 (0:00:00.197) 0:02:15.938 ********* 2020-08-07T14:58:58.837 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** ok: [smithi096.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-08-07T14:58:58.884 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:58 +0000 (0:00:00.332) 0:02:16.270 ********* 2020-08-07T14:58:58.976 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** ok: [smithi074.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "state": "file", "uid": 0} 2020-08-07T14:58:59.488 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "state": "file", "uid": 0} 2020-08-07T14:58:59.570 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "state": "file", "uid": 0} 2020-08-07T14:58:59.637 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:58:59 +0000 (0:00:00.761) 0:02:17.032 ********* 2020-08-07T14:59:00.101 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "state": "file", "uid": 0} 2020-08-07T14:59:00.159 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:00 +0000 (0:00:00.508) 0:02:17.541 ********* 2020-08-07T14:59:00.250 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* ok: [smithi074.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "3c9ba7bacd1e30bed014cefd1fb4eab0f0f8fa10", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 484, "state": "file", "uid": 0} 2020-08-07T14:59:00.800 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "3c9ba7bacd1e30bed014cefd1fb4eab0f0f8fa10", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 484, "state": "file", "uid": 0} 2020-08-07T14:59:00.837 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "d542c3e2c30a0d3eb7d454775342c302c0ded15b", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 540, "state": "file", "uid": 0} 2020-08-07T14:59:00.908 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:00 +0000 (0:00:00.762) 0:02:18.303 ********* 2020-08-07T14:59:00.976 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-08-07T14:59:00.978 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:01.021 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:01 +0000 (0:00:00.112) 0:02:18.416 ********* 2020-08-07T14:59:01.075 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:01.132 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:01 +0000 (0:00:00.106) 0:02:18.523 ********* 2020-08-07T14:59:01.178 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:01.218 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:01 +0000 (0:00:00.085) 0:02:18.609 ********* 2020-08-07T14:59:01.266 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:01.305 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:01 +0000 (0:00:00.088) 0:02:18.698 ********* 2020-08-07T14:59:01.340 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-08-07T14:59:01.342 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:01.392 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:01 +0000 (0:00:00.086) 0:02:18.785 ********* 2020-08-07T14:59:01.422 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* ok: [smithi096.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "d542c3e2c30a0d3eb7d454775342c302c0ded15b", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 540, "state": "file", "uid": 0} 2020-08-07T14:59:01.494 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:01 +0000 (0:00:00.101) 0:02:18.886 ********* 2020-08-07T14:59:01.506 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:01.554 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:01 +0000 (0:00:00.060) 0:02:18.947 ********* 2020-08-07T14:59:01.575 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:01.674 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:01 +0000 (0:00:00.109) 0:02:19.056 ********* 2020-08-07T14:59:01.681 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:01.741 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:01 +0000 (0:00:00.079) 0:02:19.135 ********* 2020-08-07T14:59:01.756 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:01.816 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:01 +0000 (0:00:00.068) 0:02:19.204 ********* 2020-08-07T14:59:01.821 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:01.886 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:01 +0000 (0:00:00.071) 0:02:19.275 ********* 2020-08-07T14:59:01.905 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:01.958 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:01 +0000 (0:00:00.069) 0:02:19.344 ********* TASK [common : Refresh repos] ************************************************** 2020-08-07T14:59:01.969 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:02.030 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:02 +0000 (0:00:00.079) 0:02:19.424 ********* 2020-08-07T14:59:02.050 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:02.094 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:02 +0000 (0:00:00.056) 0:02:19.481 ********* 2020-08-07T14:59:02.166 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:02.207 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:02 +0000 (0:00:00.120) 0:02:19.601 ********* 2020-08-07T14:59:02.296 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:02.335 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:02 +0000 (0:00:00.126) 0:02:19.728 ********* 2020-08-07T14:59:02.407 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:02.438 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:02 +0000 (0:00:00.104) 0:02:19.833 ********* 2020-08-07T14:59:02.476 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:02.513 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:02 +0000 (0:00:00.074) 0:02:19.908 ********* 2020-08-07T14:59:02.533 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-08-07T14:59:02.536 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.265065", "end": "2020-08-07 14:59:02.500334", "rc": 0, "start": "2020-08-07 14:59:02.235269", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-08-07T14:59:02.589 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:02 +0000 (0:00:00.076) 0:02:19.984 ********* 2020-08-07T14:59:02.601 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:02.645 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:02 +0000 (0:00:00.056) 0:02:20.040 ********* 2020-08-07T14:59:03.521 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.624274", "end": "2020-08-07 14:59:03.500350", "rc": 0, "start": "2020-08-07 14:59:02.876076", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-08-07T14:59:03.566 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:03 +0000 (0:00:00.920) 0:02:20.961 ********* 2020-08-07T14:59:04.990 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: krb5-workstation"]} 2020-08-07T14:59:05.038 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:05 +0000 (0:00:01.464) 0:02:22.426 ********* 2020-08-07T14:59:05.090 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:05.133 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:05 +0000 (0:00:00.099) 0:02:22.526 ********* 2020-08-07T14:59:05.173 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:05.226 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:05 +0000 (0:00:00.091) 0:02:22.617 ********* 2020-08-07T14:59:05.272 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:05.310 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:05 +0000 (0:00:00.088) 0:02:22.705 ********* 2020-08-07T14:59:05.362 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:05.409 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:05 +0000 (0:00:00.094) 0:02:22.800 ********* 2020-08-07T14:59:05.935 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2020-08-07T14:59:05.981 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:05 +0000 (0:00:00.576) 0:02:23.377 ********* 2020-08-07T14:59:06.020 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: krb5-workstation"]} 2020-08-07T14:59:06.070 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:06 +0000 (0:00:00.082) 0:02:23.459 ********* 2020-08-07T14:59:06.100 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-08-07T14:59:06.104 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:06.144 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:06 +0000 (0:00:00.079) 0:02:23.538 ********* 2020-08-07T14:59:06.198 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:06.244 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:06 +0000 (0:00:00.095) 0:02:23.634 ********* 2020-08-07T14:59:06.282 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:06.328 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:06 +0000 (0:00:00.078) 0:02:23.713 ********* 2020-08-07T14:59:06.367 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-08-07T14:59:06.369 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:06.403 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:06 +0000 (0:00:00.083) 0:02:23.797 ********* 2020-08-07T14:59:06.958 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2020-08-07T14:59:07.044 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:07 +0000 (0:00:00.641) 0:02:24.438 ********* 2020-08-07T14:59:08.233 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-08-07T14:59:08.237 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: smartmontools"]} 2020-08-07T14:59:08.283 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:08 +0000 (0:00:01.239) 0:02:25.678 ********* 2020-08-07T14:59:08.940 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [smithi074.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2020-08-07T14:59:09.198 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-08-07T14:59:09.201 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: smartmontools"]} 2020-08-07T14:59:09.262 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:09 +0000 (0:00:00.978) 0:02:26.657 ********* 2020-08-07T14:59:09.530 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [smithi074.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2020-08-07T14:59:09.933 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2020-08-07T14:59:10.077 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2020-08-07T14:59:10.120 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:10 +0000 (0:00:00.859) 0:02:27.516 ********* 2020-08-07T14:59:10.347 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-08-07T14:59:10.385 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:10 +0000 (0:00:00.262) 0:02:27.778 ********* 2020-08-07T14:59:10.465 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [smithi096.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2020-08-07T14:59:10.944 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** ok: [smithi074.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2020-08-07T14:59:11.019 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [smithi096.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2020-08-07T14:59:11.069 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:11 +0000 (0:00:00.683) 0:02:28.461 ********* 2020-08-07T14:59:11.292 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-08-07T14:59:11.339 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:11 +0000 (0:00:00.266) 0:02:28.728 ********* 2020-08-07T14:59:11.509 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** ok: [smithi074.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2020-08-07T14:59:11.914 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2020-08-07T14:59:12.009 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2020-08-07T14:59:12.047 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:12 +0000 (0:00:00.714) 0:02:29.442 ********* 2020-08-07T14:59:12.102 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:12.139 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:12 +0000 (0:00:00.091) 0:02:29.534 ********* 2020-08-07T14:59:12.210 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** ok: [smithi074.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2020-08-07T14:59:12.246 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:12 +0000 (0:00:00.102) 0:02:29.637 ********* 2020-08-07T14:59:12.421 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** ok: [smithi096.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2020-08-07T14:59:12.479 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* changed: [smithi074.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-08-07T14:59:12.540 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:12 +0000 (0:00:00.285) 0:02:29.923 ********* 2020-08-07T14:59:12.618 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-08-07T14:59:12.620 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-08-07T14:59:12.632 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-08-07T14:59:12.642 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-08-07T14:59:12.644 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-08-07T14:59:12.681 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:12 +0000 (0:00:00.153) 0:02:30.076 ********* 2020-08-07T14:59:12.962 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** ok: [smithi096.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2020-08-07T14:59:13.005 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:12 +0000 (0:00:00.322) 0:02:30.398 ********* 2020-08-07T14:59:13.048 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:13.110 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:13 +0000 (0:00:00.105) 0:02:30.504 ********* 2020-08-07T14:59:13.167 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2020-08-07T14:59:13.206 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:13 +0000 (0:00:00.096) 0:02:30.600 ********* 2020-08-07T14:59:13.449 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-08-07T14:59:13.451 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2020-08-07T14:59:13.491 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:13 +0000 (0:00:00.285) 0:02:30.886 ********* 2020-08-07T14:59:13.565 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-08-07T14:59:13.567 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-08-07T14:59:13.577 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-08-07T14:59:13.599 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-08-07T14:59:13.601 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-08-07T14:59:13.651 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:13 +0000 (0:00:00.160) 0:02:31.046 ********* 2020-08-07T14:59:19.986 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-08-07T14:59:19.991 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: nagios-common", "Installed: nrpe", "Installed: nagios-plugins", "Installed: nagios-plugins-load", "Installed: nagios-plugins-2.3.3-4.el8.x86_64", "Installed: nagios-plugins-load-2.3.3-4.el8.x86_64", "Installed: nrpe-4.0.3-1.el8.x86_64", "Removed: nrpe-4.0.2-2.el8.x86_64", "Removed: nagios-plugins-2.3.3-1.el8.x86_64", "Removed: nagios-plugins-load-2.3.3-1.el8.x86_64"]} 2020-08-07T14:59:20.025 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:20 +0000 (0:00:06.374) 0:02:37.420 ********* 2020-08-07T14:59:20.069 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-08-07T14:59:20.087 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:20.129 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:20 +0000 (0:00:00.099) 0:02:37.520 ********* 2020-08-07T14:59:20.678 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2020-08-07T14:59:20.732 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:20 +0000 (0:00:00.602) 0:02:38.124 ********* 2020-08-07T14:59:21.275 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-08-07T14:59:21.277 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2020-08-07T14:59:21.345 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:21 +0000 (0:00:00.603) 0:02:38.727 ********* 2020-08-07T14:59:21.387 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ skipping: [smithi074.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "DAEMON_OPTS", "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "NRPE_OPTS", "skip_reason": "Conditional result was False"} 2020-08-07T14:59:21.430 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:21 +0000 (0:00:00.097) 0:02:38.825 ********* 2020-08-07T14:59:21.454 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: nagios-common", "Installed: nrpe", "Installed: nagios-plugins", "Installed: nagios-plugins-load", "Installed: nagios-plugins-2.3.3-4.el8.x86_64", "Installed: nagios-plugins-load-2.3.3-4.el8.x86_64", "Installed: nrpe-4.0.3-1.el8.x86_64", "Removed: nrpe-4.0.2-2.el8.x86_64", "Removed: nagios-plugins-2.3.3-1.el8.x86_64", "Removed: nagios-plugins-load-2.3.3-1.el8.x86_64"]} 2020-08-07T14:59:21.493 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:21 +0000 (0:00:00.063) 0:02:38.888 ********* 2020-08-07T14:59:21.555 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:21.609 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:21 +0000 (0:00:00.109) 0:02:38.998 ********* 2020-08-07T14:59:21.650 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* ok: [smithi074.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-08-07T14:59:21.726 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:21 +0000 (0:00:00.119) 0:02:39.117 ********* 2020-08-07T14:59:21.989 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** fatal: [smithi074.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2020-08-07T14:59:22.032 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:22 +0000 (0:00:00.309) 0:02:39.427 ********* 2020-08-07T14:59:22.099 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:22.146 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:22 +0000 (0:00:00.114) 0:02:39.541 ********* 2020-08-07T14:59:22.187 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2020-08-07T14:59:22.242 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:22 +0000 (0:00:00.096) 0:02:39.637 ********* 2020-08-07T14:59:22.792 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* changed: [smithi074.front.sepia.ceph.com] => {"changed": true, "checksum": "aa9150d4582f919d7a8504c01c8d86f05e3eeb5d", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "md5sum": "28732085640fd0d909d798c6d3e4021a", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1184, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1596812362.1812007-123937711888352/source", "state": "file", "uid": 0} TASK [common : Upload nagios check_mem script] ********************************* ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2020-08-07T14:59:22.827 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:22 +0000 (0:00:00.583) 0:02:40.220 ********* 2020-08-07T14:59:22.869 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:22 +0000 (0:00:00.041) 0:02:40.262 ********* 2020-08-07T14:59:22.923 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ skipping: [smithi096.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "DAEMON_OPTS", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "NRPE_OPTS", "skip_reason": "Conditional result was False"} 2020-08-07T14:59:22.972 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:22 +0000 (0:00:00.102) 0:02:40.364 ********* 2020-08-07T14:59:23.234 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* ok: [smithi096.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-08-07T14:59:23.282 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:23 +0000 (0:00:00.311) 0:02:40.676 ********* 2020-08-07T14:59:23.603 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-08-07T14:59:23.607 INFO:teuthology.task.ansible.out:fatal: [smithi096.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2020-08-07T14:59:23.654 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:23 +0000 (0:00:00.366) 0:02:41.042 ********* 2020-08-07T14:59:23.689 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T14:59:23.725 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:23 +0000 (0:00:00.077) 0:02:41.120 ********* 2020-08-07T14:59:23.865 INFO:teuthology.task.ansible.out: 2020-08-07T14:59:23.868 INFO:teuthology.task.ansible.out:TASK [common : Make sure nagios nrpe service is running.] ********************** ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2020-08-07 14:59:16 UTC", "ActiveEnterTimestampMonotonic": "235572680", "ActiveExitTimestamp": "Fri 2020-08-07 14:59:16 UTC", "ActiveExitTimestampMonotonic": "235549983", "ActiveState": "active", "After": "basic.target nss-lookup.target time-sync.target sysinit.target systemd-journald.socket network.target local-fs.target tmp.mount system.slice -.mount var-run.mount systemd-tmpfiles-setup.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2020-08-07 14:59:16 UTC", "AssertTimestampMonotonic": "235571963", "Before": "plymouth-quit.service xdm.service getty@tty1.service multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2020-08-07 14:59:16 UTC", "ConditionTimestampMonotonic": "235571963", "C 2020-08-07T14:59:23.870 INFO:teuthology.task.ansible.out:onfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "16844", "ExecMainStartTimestamp": "Fri 2020-08-07 14:59:16 UTC", "ExecMainStartTimestampMonotonic": "235572651", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "GID": "986", "Group": "nrpe", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Fri 2020-08-07 14:59:16 UTC", "InactiveEnterTimestampMonotonic": "235571346", "InactiveExitTimestamp": "Fri 2020-08-07 14:59:16 UTC", "InactiveExitTimestampMonotonic": "235572680", "InvocationID": "0c291c8fe8414bc08a8adb925e1ac565", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-g 2020-08-07T14:59:23.872 INFO:teuthology.task.ansible.out:roup", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127424", "LimitNPROCSoft": "127424", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127424", "LimitSIGPENDINGSoft": "127424", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "16844", "MemoryAccounting": "yes", "MemoryCurrent": "897024", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nrpe/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/var/tmp 2020-08-07T14:59:23.874 INFO:teuthology.task.ansible.out:/run/nrpe", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectory": "nrpe", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2020-08-07 14:59:16 UTC", "StateChangeTimestampMonotonic": "235572680", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "990", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2020-08-07 14:59:16 UTC", "WatchdogTimestampMonotonic": "235572679", "WatchdogUSec": "0"}} 2020-08-07T14:59:23.908 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:23 +0000 (0:00:00.183) 0:02:41.303 ********* 2020-08-07T14:59:24.158 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* changed: [smithi074.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.001591", "end": "2020-08-07 14:59:24.129402", "rc": 0, "start": "2020-08-07 14:59:24.127811", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2020-08-07T14:59:24.212 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:24 +0000 (0:00:00.303) 0:02:41.607 ********* 2020-08-07T14:59:24.362 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-08-07T14:59:24.364 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "checksum": "aa9150d4582f919d7a8504c01c8d86f05e3eeb5d", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "md5sum": "28732085640fd0d909d798c6d3e4021a", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1184, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1596812363.763529-193893720115711/source", "state": "file", "uid": 0} 2020-08-07T14:59:24.450 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:24 +0000 (0:00:00.227) 0:02:41.835 ********* 2020-08-07T14:59:24.770 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2020-08-07T14:59:24.771 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2020-08-07 14:59:18 UTC", "ActiveEnterTimestampMonotonic": "232816188", "ActiveExitTimestamp": "Fri 2020-08-07 14:59:18 UTC", "ActiveExitTimestampMonotonic": "232788691", "ActiveState": "active", "After": "systemd-tmpfiles-setup.service sysinit.target basic.target var-run.mount -.mount systemd-journald.socket system.slice tmp.mount local-fs.target time-sync.target network.target nss-lookup.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2020-08-07 14:59:18 UTC", "AssertTimestampMonotonic": "232815044", "Before": "plymouth-quit.service xdm.service getty@tty1.service shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2020-08-07 14:59:18 UTC", "ConditionTimestampMonotonic": "232815044", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "C 2020-08-07T14:59:24.773 INFO:teuthology.task.ansible.out:ontrolGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "16846", "ExecMainStartTimestamp": "Fri 2020-08-07 14:59:18 UTC", "ExecMainStartTimestampMonotonic": "232816152", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "GID": "986", "Group": "nrpe", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Fri 2020-08-07 14:59:18 UTC", "InactiveEnterTimestampMonotonic": "232814381", "InactiveExitTimestamp": "Fri 2020-08-07 14:59:18 UTC", "InactiveExitTimestampMonotonic": "232816188", "InvocationID": "74b671da6bcb4d20b260709af38cb7b8", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "Limi 2020-08-07T14:59:24.774 INFO:teuthology.task.ansible.out:tCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127425", "LimitNPROCSoft": "127425", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127425", "LimitSIGPENDINGSoft": "127425", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "16846", "MemoryAccounting": "yes", "MemoryCurrent": "892928", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nrpe/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/var/tmp /run/nrpe", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": " 2020-08-07T14:59:24.776 INFO:teuthology.task.ansible.out:no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectory": "nrpe", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2020-08-07 14:59:18 UTC", "StateChangeTimestampMonotonic": "232816188", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "990", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2020-08-07 14:59:18 UTC", "WatchdogTimestampMonotonic": "232816186", "WatchdogUSec": "0"}} 2020-08-07T14:59:24.822 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:24 +0000 (0:00:00.382) 0:02:42.218 ********* 2020-08-07T14:59:25.040 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-08-07T14:59:25.233 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.001605", "end": "2020-08-07 14:59:25.012631", "rc": 0, "start": "2020-08-07 14:59:25.011026", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} Friday 07 August 2020 14:59:25 +0000 (0:00:00.255) 0:02:42.473 ********* 2020-08-07T14:59:26.337 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: python3-libsemanage", "Installed: python3-policycoreutils"]} 2020-08-07T14:59:26.391 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:26 +0000 (0:00:01.310) 0:02:43.784 ********* 2020-08-07T14:59:27.251 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: python3-libsemanage", "Installed: python3-policycoreutils"]} 2020-08-07T14:59:27.303 INFO:teuthology.task.ansible.out:Friday 07 August 2020 14:59:27 +0000 (0:00:00.913) 0:02:44.698 ********* 2020-08-07T15:00:07.363 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-08-07T15:00:07.372 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: selinux-policy-targeted", "Installed: selinux-policy-3.14.3-41.el8_2.5.noarch", "Installed: selinux-policy-targeted-3.14.3-41.el8_2.5.noarch", "Removed: selinux-policy-3.14.3-41.el8_2.2.noarch", "Removed: selinux-policy-targeted-3.14.3-41.el8_2.2.noarch"]} 2020-08-07T15:00:07.422 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:07 +0000 (0:00:40.117) 0:03:24.815 ********* 2020-08-07T15:00:08.508 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2020-08-07T15:00:08.513 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2020-08-07T15:00:08.580 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:08 +0000 (0:00:01.154) 0:03:25.970 ********* 2020-08-07T15:00:08.630 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: selinux-policy-targeted", "Installed: selinux-policy-3.14.3-41.el8_2.5.noarch", "Installed: selinux-policy-targeted-3.14.3-41.el8_2.5.noarch", "Removed: selinux-policy-3.14.3-41.el8_2.2.noarch", "Removed: selinux-policy-targeted-3.14.3-41.el8_2.2.noarch"]} 2020-08-07T15:00:08.674 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:08 +0000 (0:00:00.099) 0:03:26.070 ********* 2020-08-07T15:00:09.174 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2020-08-07T15:00:09.205 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:09 +0000 (0:00:00.530) 0:03:26.600 ********* 2020-08-07T15:00:24.761 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** changed: [smithi074.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:15.950626", "end": "2020-08-07 15:00:24.728043", "failed_when_result": false, "rc": 0, "start": "2020-08-07 15:00:08.777417", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2020-08-07T15:00:24.810 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:24 +0000 (0:00:15.603) 0:03:42.204 ********* 2020-08-07T15:00:25.374 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2020-08-07T15:00:25.409 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:25 +0000 (0:00:00.597) 0:03:42.802 ********* 2020-08-07T15:00:25.546 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:16.094807", "end": "2020-08-07 15:00:25.504237", "failed_when_result": false, "rc": 0, "start": "2020-08-07 15:00:09.409430", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2020-08-07T15:00:25.585 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:25 +0000 (0:00:00.178) 0:03:42.980 ********* 2020-08-07T15:00:25.646 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** changed: [smithi074.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.009665", "end": "2020-08-07 15:00:25.624449", "rc": 0, "start": "2020-08-07 15:00:25.614784", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-08-07T15:00:25.698 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:25 +0000 (0:00:00.109) 0:03:43.089 ********* 2020-08-07T15:00:25.950 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** changed: [smithi074.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.009201", "end": "2020-08-07 15:00:25.907791", "rc": 0, "start": "2020-08-07 15:00:25.898590", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-08-07T15:00:25.996 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:25 +0000 (0:00:00.302) 0:03:43.392 ********* 2020-08-07T15:00:26.178 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2020-08-07T15:00:26.180 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2020-08-07T15:00:26.228 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:26 +0000 (0:00:00.231) 0:03:43.623 ********* 2020-08-07T15:00:26.481 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-08-07T15:00:26.484 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.023218", "end": "2020-08-07 15:00:26.462586", "rc": 0, "start": "2020-08-07 15:00:26.439368", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-08-07T15:00:26.550 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:26 +0000 (0:00:00.322) 0:03:43.945 ********* 2020-08-07T15:00:26.821 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.005128", "end": "2020-08-07 15:00:26.795886", "rc": 0, "start": "2020-08-07 15:00:26.790758", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-08-07T15:00:26.862 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:26 +0000 (0:00:00.311) 0:03:44.256 ********* 2020-08-07T15:00:42.301 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2020-08-07T15:00:42.304 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:16.081700", "end": "2020-08-07 15:00:42.268481", "rc": 0, "start": "2020-08-07 15:00:26.186781", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-08-07T15:00:42.342 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:42 +0000 (0:00:15.480) 0:03:59.737 ********* 2020-08-07T15:00:42.587 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2020-08-07T15:00:42.627 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:42 +0000 (0:00:00.284) 0:04:00.022 ********* 2020-08-07T15:00:42.670 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-08-07T15:00:42.672 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/dnf_systems.yml"], "changed": false} 2020-08-07T15:00:42.726 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:42 +0000 (0:00:00.098) 0:04:00.121 ********* 2020-08-07T15:00:42.769 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-08-07T15:00:42.786 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-08-07T15:00:42.808 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:42 +0000 (0:00:00.082) 0:04:00.204 ********* 2020-08-07T15:00:42.862 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-08-07T15:00:42.865 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/redhat_8.yml) => {"ansible_facts": {"common_yum_repos": {"lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "epel_packages": ["dbench"], "nfs_service": "nfs-server", "ntp_service_name": "chronyd", "packages": ["dnf-utils", "git-all", "sysstat", "libedit", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse-libs", "openssl", "libuuid", "attr", "ant", "lsof", "gettext", "bc", "xfsdump", "blktrace", "usbredir", "podman", "podman-docker", "libev-devel", "valgrind", "nfs-utils", "ncurses-devel", "gcc", "git", "python3-nose", "python3-virtualenv", "genisoimage", "qemu-img", "qemu-kvm-core", "qemu-kvm-block-rbd", "libacl-devel", "autoconf", "gdb", "iozone"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/redhat_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/redhat_8.yml"} 2020-08-07T15:00:42.925 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:42 +0000 (0:00:00.114) 0:04:00.319 ********* 2020-08-07T15:00:42.991 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-08-07T15:00:42.993 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-08-07T15:00:43.048 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:43 +0000 (0:00:00.123) 0:04:00.442 ********* 2020-08-07T15:00:43.224 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:16.096341", "end": "2020-08-07 15:00:43.195090", "rc": 0, "start": "2020-08-07 15:00:27.098749", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-08-07T15:00:43.314 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:43 +0000 (0:00:00.254) 0:04:00.697 ********* TASK [testnode : Ensure the sudo group exists.] ******************************** ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-08-07T15:00:43.360 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:43 +0000 (0:00:00.057) 0:04:00.755 ********* 2020-08-07T15:00:43.526 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2020-08-07T15:00:43.566 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:43 +0000 (0:00:00.203) 0:04:00.959 ********* 2020-08-07T15:00:43.587 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "name": "ubuntu", "state": "present", "system": false} 2020-08-07T15:00:43.638 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:43 +0000 (0:00:00.072) 0:04:01.031 ********* 2020-08-07T15:00:43.659 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/dnf_systems.yml"], "changed": false} 2020-08-07T15:00:43.705 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:43 +0000 (0:00:00.064) 0:04:01.096 ********* 2020-08-07T15:00:43.720 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:00:43.756 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:43 +0000 (0:00:00.055) 0:04:01.151 ********* 2020-08-07T15:00:43.770 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* ok: [smithi096.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-08-07T15:00:43.814 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:43 +0000 (0:00:00.054) 0:04:01.206 ********* 2020-08-07T15:00:43.823 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:00:43.867 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:43 +0000 (0:00:00.056) 0:04:01.262 ********* 2020-08-07T15:00:43.878 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-08-07T15:00:43.881 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/redhat_8.yml) => {"ansible_facts": {"common_yum_repos": {"lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "epel_packages": ["dbench"], "nfs_service": "nfs-server", "ntp_service_name": "chronyd", "packages": ["dnf-utils", "git-all", "sysstat", "libedit", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse-libs", "openssl", "libuuid", "attr", "ant", "lsof", "gettext", "bc", "xfsdump", "blktrace", "usbredir", "podman", "podman-docker", "libev-devel", "valgrind", "nfs-utils", "ncurses-devel", "gcc", "git", "python3-nose", "python3-virtualenv", "genisoimage", "qemu-img", "qemu-kvm-core", "qemu-kvm-block-rbd", "libacl-devel", "autoconf", "gdb", "iozone"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/redhat_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/redhat_8.yml"} 2020-08-07T15:00:43.950 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:43 +0000 (0:00:00.082) 0:04:01.345 ********* 2020-08-07T15:00:44.025 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ ok: [smithi096.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-08-07T15:00:44.070 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:44 +0000 (0:00:00.119) 0:04:01.464 ********* 2020-08-07T15:00:44.072 INFO:teuthology.task.ansible.out: 2020-08-07T15:00:44.148 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** ok: [smithi074.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2020-08-07T15:00:44.185 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:44 +0000 (0:00:00.115) 0:04:01.580 ********* 2020-08-07T15:00:44.315 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2020-08-07T15:00:44.354 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:44 +0000 (0:00:00.168) 0:04:01.749 ********* 2020-08-07T15:00:44.589 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "name": "ubuntu", "state": "present", "system": false} 2020-08-07T15:00:44.629 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:44 +0000 (0:00:00.275) 0:04:02.024 ********* 2020-08-07T15:00:44.694 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-08-07T15:00:44.702 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:00:44.753 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:44 +0000 (0:00:00.115) 0:04:02.139 ********* 2020-08-07T15:00:44.783 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-08-07T15:00:44.786 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:00:44.844 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:44 +0000 (0:00:00.099) 0:04:02.239 ********* 2020-08-07T15:00:44.875 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2020-08-07T15:00:44.916 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:44 +0000 (0:00:00.071) 0:04:02.311 ********* 2020-08-07T15:00:44.987 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* ok: [smithi074.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi074"}, "changed": false} 2020-08-07T15:00:45.053 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:45 +0000 (0:00:00.127) 0:04:02.438 ********* 2020-08-07T15:00:45.098 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-08-07T15:00:45.103 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2020-08-07T15:00:45.182 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:45 +0000 (0:00:00.138) 0:04:02.577 ********* 2020-08-07T15:00:45.897 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-08-07T15:00:45.899 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2020-08-07T15:00:45.955 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:45 +0000 (0:00:00.769) 0:04:03.346 ********* 2020-08-07T15:00:46.044 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi096"}, "changed": false} 2020-08-07T15:00:46.102 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:46 +0000 (0:00:00.150) 0:04:03.496 ********* 2020-08-07T15:00:46.427 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ ok: [smithi074.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi074", "ansible_hostname": "smithi074", "ansible_nodename": "smithi074"}, "changed": false, "name": "smithi074"} 2020-08-07T15:00:46.480 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:46 +0000 (0:00:00.379) 0:04:03.875 ********* 2020-08-07T15:00:46.739 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi096", "ansible_hostname": "smithi096", "ansible_nodename": "smithi096"}, "changed": false, "name": "smithi096"} 2020-08-07T15:00:46.831 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:46 +0000 (0:00:00.349) 0:04:04.225 ********* 2020-08-07T15:00:47.103 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "checksum": "a5e550b2aa74f6355e926d70d73a50ad9ab4b4be", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1043, "state": "file", "uid": 0} 2020-08-07T15:00:47.162 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:47 +0000 (0:00:00.328) 0:04:04.554 ********* 2020-08-07T15:00:47.431 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "a5e550b2aa74f6355e926d70d73a50ad9ab4b4be", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1043, "state": "file", "uid": 0} 2020-08-07T15:00:47.472 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:47 +0000 (0:00:00.312) 0:04:04.866 ********* 2020-08-07T15:00:47.787 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2020-08-07T15:00:47.789 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2020-08-07T15:00:47.844 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:47 +0000 (0:00:00.367) 0:04:05.233 ********* 2020-08-07T15:00:48.086 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2020-08-07T15:00:48.135 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:48 +0000 (0:00:00.296) 0:04:05.530 ********* 2020-08-07T15:00:48.546 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** changed: [smithi074.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-08-07T15:00:48.584 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:48 +0000 (0:00:00.448) 0:04:05.979 ********* 2020-08-07T15:00:48.869 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-08-07T15:00:48.905 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:48 +0000 (0:00:00.313) 0:04:06.293 ********* 2020-08-07T15:00:49.236 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-08-07T15:00:49.238 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2020-08-07T15:00:49.295 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:49 +0000 (0:00:00.395) 0:04:06.688 ********* 2020-08-07T15:00:49.501 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2020-08-07T15:00:49.561 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:49 +0000 (0:00:00.261) 0:04:06.950 ********* 2020-08-07T15:00:49.567 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-08-07T15:00:49.568 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2020-08-07T15:00:49.615 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:49 +0000 (0:00:00.052) 0:04:07.003 ********* 2020-08-07T15:00:49.786 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2020-08-07T15:00:49.839 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:49 +0000 (0:00:00.228) 0:04:07.232 ********* 2020-08-07T15:00:49.864 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "gid": 36, "name": "kvm", "state": "present", "system": false} 2020-08-07T15:00:49.914 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:49 +0000 (0:00:00.077) 0:04:07.309 ********* 2020-08-07T15:00:50.069 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "gid": 36, "name": "kvm", "state": "present", "system": false} 2020-08-07T15:00:50.145 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:50 +0000 (0:00:00.230) 0:04:07.540 ********* 2020-08-07T15:00:50.175 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-08-07T15:00:50.184 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1215, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-08-07T15:00:50.228 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:50 +0000 (0:00:00.082) 0:04:07.622 ********* 2020-08-07T15:00:50.419 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1215, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-08-07T15:00:50.466 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:50 +0000 (0:00:00.237) 0:04:07.860 ********* 2020-08-07T15:00:50.795 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2020-08-07T15:00:50.852 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:50 +0000 (0:00:00.385) 0:04:08.246 ********* 2020-08-07T15:00:51.116 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2020-08-07T15:00:51.122 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:51 +0000 (0:00:00.271) 0:04:08.517 ********* 2020-08-07T15:00:51.535 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2020-08-07T15:00:51.587 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:51 +0000 (0:00:00.464) 0:04:08.982 ********* 2020-08-07T15:00:51.747 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2020-08-07T15:00:51.796 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:51 +0000 (0:00:00.206) 0:04:09.188 ********* 2020-08-07T15:00:53.551 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-08-07T15:00:53.553 INFO:teuthology.task.ansible.out:[WARNING]: Consider using the yum, dnf or zypper module rather than running 'rpm'. If you need to use command because yum, dnf or zypper is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi074.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:01.725316", "end": "2020-08-07 15:00:53.522946", "rc": 0, "start": "2020-08-07 15:00:51.797630", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-08-07T15:00:53.594 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:53 +0000 (0:00:01.800) 0:04:10.989 ********* 2020-08-07T15:00:53.650 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-08-07T15:00:53.654 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:00:53.710 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:53 +0000 (0:00:00.102) 0:04:11.091 ********* 2020-08-07T15:00:53.850 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-08-07T15:00:53.852 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:01.813542", "end": "2020-08-07 15:00:53.819430", "rc": 0, "start": "2020-08-07 15:00:52.005888", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-08-07T15:00:53.891 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:53 +0000 (0:00:00.194) 0:04:11.286 ********* 2020-08-07T15:00:53.945 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-08-07T15:00:53.948 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:00:53.983 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:53 +0000 (0:00:00.092) 0:04:11.378 ********* 2020-08-07T15:00:54.281 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-08-07T15:00:54.283 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item={'key': 'lab-extras', 'value': {'name': 'lab-extras', 'baseurl': 'http://apt-mirror.front.sepia.ceph.com/lab-extras/8/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a0e2ac0628a264e14b96b1c57627a3aa344a6c52", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 203, "state": "file", "uid": 0} 2020-08-07T15:00:54.329 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:54 +0000 (0:00:00.346) 0:04:11.725 ********* 2020-08-07T15:00:54.376 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-08-07T15:00:54.448 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:54 +0000 (0:00:00.116) 0:04:11.841 ********* 2020-08-07T15:00:54.480 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-08-07T15:00:54.538 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:54 +0000 (0:00:00.090) 0:04:11.931 ********* 2020-08-07T15:00:54.586 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:00:54.624 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:54 +0000 (0:00:00.088) 0:04:12.019 ********* 2020-08-07T15:00:54.641 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* ok: [smithi096.front.sepia.ceph.com] => (item={'key': 'lab-extras', 'value': {'name': 'lab-extras', 'baseurl': 'http://apt-mirror.front.sepia.ceph.com/lab-extras/8/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a0e2ac0628a264e14b96b1c57627a3aa344a6c52", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 203, "state": "file", "uid": 0} 2020-08-07T15:00:54.711 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:54 +0000 (0:00:00.086) 0:04:12.106 ********* 2020-08-07T15:00:54.723 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-08-07T15:00:54.725 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-08-07T15:00:54.775 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:54 +0000 (0:00:00.064) 0:04:12.170 ********* 2020-08-07T15:00:54.790 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-08-07T15:00:54.850 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:54 +0000 (0:00:00.069) 0:04:12.239 ********* 2020-08-07T15:00:54.858 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:00:54.901 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:54 +0000 (0:00:00.057) 0:04:12.296 ********* 2020-08-07T15:00:54.915 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-08-07T15:00:54.968 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:54 +0000 (0:00:00.067) 0:04:12.363 ********* 2020-08-07T15:00:54.981 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-08-07T15:00:54.984 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:00:55.030 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:55 +0000 (0:00:00.061) 0:04:12.425 ********* 2020-08-07T15:00:55.042 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-08-07T15:00:55.044 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:00:55.084 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:55 +0000 (0:00:00.053) 0:04:12.479 ********* 2020-08-07T15:00:55.096 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:00:55.142 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:55 +0000 (0:00:00.056) 0:04:12.536 ********* 2020-08-07T15:00:55.193 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-08-07T15:00:55.199 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-08-07T15:00:55.239 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:55 +0000 (0:00:00.098) 0:04:12.634 ********* 2020-08-07T15:00:55.251 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:00:55.297 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:55 +0000 (0:00:00.055) 0:04:12.690 ********* 2020-08-07T15:00:55.307 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:00:55.345 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:55 +0000 (0:00:00.050) 0:04:12.740 ********* 2020-08-07T15:00:55.407 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:00:55.447 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:55 +0000 (0:00:00.101) 0:04:12.842 ********* 2020-08-07T15:00:55.491 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:00:55.547 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:55 +0000 (0:00:00.095) 0:04:12.937 ********* 2020-08-07T15:00:55.568 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* fatal: [smithi074.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.007166", "end": "2020-08-07 15:00:55.514222", "msg": "non-zero return code", "rc": 1, "start": "2020-08-07 15:00:55.507056", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2020-08-07T15:00:55.612 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:55 +0000 (0:00:00.069) 0:04:13.007 ********* 2020-08-07T15:00:55.626 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:00:55.672 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:55 +0000 (0:00:00.060) 0:04:13.068 ********* 2020-08-07T15:00:55.683 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-08-07T15:00:55.686 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:00:55.723 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:55 +0000 (0:00:00.049) 0:04:13.118 ********* 2020-08-07T15:00:55.913 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* fatal: [smithi096.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.007239", "end": "2020-08-07 15:00:55.886894", "msg": "non-zero return code", "rc": 1, "start": "2020-08-07 15:00:55.879655", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2020-08-07T15:00:55.972 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:55 +0000 (0:00:00.242) 0:04:13.361 ********* 2020-08-07T15:00:56.046 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:00:56.088 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:00:56 +0000 (0:00:00.121) 0:04:13.482 ********* 2020-08-07T15:01:01.181 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-08-07T15:01:01.184 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: qemu-kvm-block-rbd-15:2.12.0-99.module+el8.2.0+5827+8c39933c.x86_64", "Removed: librados2-1:12.2.7-9.el8.x86_64", "Removed: librbd1-1:12.2.7-9.el8.x86_64"]} 2020-08-07T15:01:01.221 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:01 +0000 (0:00:05.134) 0:04:18.617 ********* 2020-08-07T15:01:01.630 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: qemu-kvm-block-rbd-15:2.12.0-99.module+el8.2.0+5827+8c39933c.x86_64", "Removed: librados2-1:12.2.7-9.el8.x86_64", "Removed: librbd1-1:12.2.7-9.el8.x86_64"]} 2020-08-07T15:01:01.670 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:01 +0000 (0:00:00.442) 0:04:19.059 ********* 2020-08-07T15:01:06.346 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* changed: [smithi074.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: boost-program-options-1.66.0-7.el8.x86_64", "Removed: boost-random-1.66.0-7.el8.x86_64"]} 2020-08-07T15:01:06.402 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:06 +0000 (0:00:04.733) 0:04:23.792 ********* 2020-08-07T15:01:06.806 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: boost-program-options-1.66.0-7.el8.x86_64", "Removed: boost-random-1.66.0-7.el8.x86_64"]} 2020-08-07T15:01:06.868 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:06 +0000 (0:00:00.471) 0:04:24.264 ********* 2020-08-07T15:01:18.435 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* changed: [smithi074.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: dnf-utils", "Installed: git-all", "Installed: sysstat", "Installed: libedit", "Installed: boost-thread", "Installed: xfsprogs", "Installed: gdisk", "Installed: parted", "Installed: libgcrypt", "Installed: fuse-libs", "Installed: openssl", "Installed: libuuid", "Installed: attr", "Installed: ant", "Installed: lsof", "Installed: gettext", "Installed: bc", "Installed: xfsdump", "Installed: blktrace", "Installed: usbredir", "Installed: podman", "Installed: podman-docker", "Installed: libev-devel", "Installed: valgrind", "Installed: nfs-utils", "Installed: ncurses-devel", "Installed: gcc", "Installed: git", "Installed: python3-nose", "Installed: python3-virtualenv", "Installed: genisoimage", "Installed: qemu-img", "Installed: qemu-kvm-core", "Installed: qemu-kvm-block-rbd", "Installed: libacl-devel", "Installed: autoconf", "Installed: gdb", "Installed: iozone", "Installed: qemu-kvm-common-15:2.12.0-99.module+el8.2.0+6870+55b789b4.2.x86_64", "Installed: qemu-kvm-block-rbd-15:2.12.0-99.module+el8.2.0+6870+55b789b4.2.x86_64", "Installed: qemu-img-15:2.12.0-99.module+el8.2.0+6870+55b789b4.2.x86_64", "Installed: librados2-1:12.2.7-9.el8.x86_64", "Installed: librbd1-1:12.2.7-9.el8.x86_64", "Installed: podman-1.9.3-2.module+el8.2.1+6867+366c07d6.x86_64", "Installed: boost-program-options-1.66.0-7.el8.x86_64", "Installed: boost-random-1.66.0-7.el8.x86_64", "Installed: podman-docker-1.9.3-2.module+el8.2.1+6867+366c07d6.noarch", "Installed: qemu-kvm-core-15:2.12.0-99.module+el8.2.0+6870+55b789b4.2.x86_64", "Removed: qemu-img-15:2.12.0-99.module+el8.2.0+5827+8c39933c.x86_64", "Removed: qemu-kvm-common-15:2.12.0-99.module+el8.2.0+5827+8c39933c.x86_64", "Removed: qemu-kvm-core-15:2.12.0-99.module+el8.2.0+5827+8c39933c.x86_64", "Removed: podman-1.6.4-11.module+el8.2.0+6368+cf16aa14.x86_64", "Remov 2020-08-07T15:01:18.438 INFO:teuthology.task.ansible.out:ed: podman-docker-1.6.4-11.module+el8.2.0+6368+cf16aa14.noarch"]} 2020-08-07T15:01:18.493 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:18 +0000 (0:00:11.623) 0:04:35.887 ********* 2020-08-07T15:01:19.309 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: dnf-utils", "Installed: git-all", "Installed: sysstat", "Installed: libedit", "Installed: boost-thread", "Installed: xfsprogs", "Installed: gdisk", "Installed: parted", "Installed: libgcrypt", "Installed: fuse-libs", "Installed: openssl", "Installed: libuuid", "Installed: attr", "Installed: ant", "Installed: lsof", "Installed: gettext", "Installed: bc", "Installed: xfsdump", "Installed: blktrace", "Installed: usbredir", "Installed: podman", "Installed: podman-docker", "Installed: libev-devel", "Installed: valgrind", "Installed: nfs-utils", "Installed: ncurses-devel", "Installed: gcc", "Installed: git", "Installed: python3-nose", "Installed: python3-virtualenv", "Installed: genisoimage", "Installed: qemu-img", "Installed: qemu-kvm-core", "Installed: qemu-kvm-block-rbd", "Installed: libacl-devel", "Installed: autoconf", "Installed: gdb", "Installed: iozone", "Installed: qemu-kvm-common-15:2.12.0-99.module+el8.2.0+6870+55b789b4.2.x86_64", "Installed: qemu-kvm-block-rbd-15:2.12.0-99.module+el8.2.0+6870+55b789b4.2.x86_64", "Installed: qemu-img-15:2.12.0-99.module+el8.2.0+6870+55b789b4.2.x86_64", "Installed: librados2-1:12.2.7-9.el8.x86_64", "Installed: librbd1-1:12.2.7-9.el8.x86_64", "Installed: podman-1.9.3-2.module+el8.2.1+6867+366c07d6.x86_64", "Installed: boost-program-options-1.66.0-7.el8.x86_64", "Installed: boost-random-1.66.0-7.el8.x86_64", "Installed: podman-docker-1.9.3-2.module+el8.2.1+6867+366c07d6.noarch", "Installed: qemu-kvm-core-15:2.12.0-99.module+el8.2.0+6870+55b789b4.2.x86_64", "Removed: qemu-img-15:2.12.0-99.module+el8.2.0+5827+8c39933c.x86_64", "Removed: qemu-kvm-common-15:2.12.0-99.module+el8.2.0+5827+8c39933c.x86_64", "Removed: qemu-kvm-core-15:2.12.0-99.module+el8.2.0+5827+8c39933c.x86_64", "Removed: podman-1.6.4-11.module+el8.2.0+6368+cf16aa14.x86_64", "Removed: podman-docker-1.6.4-11.module+el8.2.0+6368+cf16aa14.noarch"]} 2020-08-07T15:01:19.351 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:19 +0000 (0:00:00.856) 0:04:36.744 ********* 2020-08-07T15:01:20.651 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: dbench"]} 2020-08-07T15:01:20.691 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:20 +0000 (0:00:01.342) 0:04:38.086 ********* 2020-08-07T15:01:21.611 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: dbench"]} 2020-08-07T15:01:21.654 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:21 +0000 (0:00:00.962) 0:04:39.048 ********* 2020-08-07T15:01:22.943 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-08-07T15:01:22.946 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2020-08-07T15:01:23.009 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:23 +0000 (0:00:01.355) 0:04:40.404 ********* 2020-08-07T15:01:23.051 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-08-07T15:01:23.053 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:23.091 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:23 +0000 (0:00:00.082) 0:04:40.486 ********* 2020-08-07T15:01:23.437 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-08-07T15:01:23.438 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "basic.target dbus.service polkit.service dbus.socket sysinit.target system.slice", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "network-pre.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "ebtables.service shutdown.target ipset.service ip6tables.service iptables.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTim 2020-08-07T15:01:23.440 INFO:teuthology.task.ansible.out:estampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127424", "LimitNPROCSoft": "127424", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127424", 2020-08-07T15:01:23.441 INFO:teuthology.task.ansible.out:"LimitSIGPENDINGSoft": "127424", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "yes", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "dbus.socket sysinit.target system.slice", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestampMonotonic": "0", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "SyslogFacility": "3 2020-08-07T15:01:23.443 INFO:teuthology.task.ansible.out:", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-08-07T15:01:23.485 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:23 +0000 (0:00:00.388) 0:04:40.874 ********* 2020-08-07T15:01:23.844 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2020-08-07T15:01:23.918 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:23 +0000 (0:00:00.438) 0:04:41.313 ********* 2020-08-07T15:01:23.930 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ fatal: [smithi074.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2020-08-07T15:01:23.973 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:23 +0000 (0:00:00.055) 0:04:41.368 ********* 2020-08-07T15:01:24.009 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-08-07T15:01:24.012 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:24.081 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:24 +0000 (0:00:00.107) 0:04:41.476 ********* 2020-08-07T15:01:24.458 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "polkit.service dbus.socket dbus.service sysinit.target system.slice basic.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target network-pre.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target ebtables.service ip6tables.service iptables.service ipset.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "DynamicUser": "no", "EnvironmentFiles": " 2020-08-07T15:01:24.460 INFO:teuthology.task.ansible.out:/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127425", "LimitNPROCSoft": "127425", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Li 2020-08-07T15:01:24.461 INFO:teuthology.task.ansible.out:mitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127425", "LimitSIGPENDINGSoft": "127425", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "yes", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "dbus.socket sysinit.target system.slice", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestampMonotonic": "0", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWh 2020-08-07T15:01:24.463 INFO:teuthology.task.ansible.out:enUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2020-08-07T15:01:24.498 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:24 +0000 (0:00:00.417) 0:04:41.893 ********* 2020-08-07T15:01:24.830 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ fatal: [smithi096.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2020-08-07T15:01:24.879 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:24 +0000 (0:00:00.372) 0:04:42.266 ********* 2020-08-07T15:01:25.032 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-08-07T15:01:25.034 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2020-08-07T15:01:25.078 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:25 +0000 (0:00:00.203) 0:04:42.469 ********* 2020-08-07T15:01:25.124 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:25.162 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:25 +0000 (0:00:00.085) 0:04:42.555 ********* 2020-08-07T15:01:25.189 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-08-07T15:01:25.190 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2020-08-07T15:01:25.239 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:25 +0000 (0:00:00.076) 0:04:42.631 ********* 2020-08-07T15:01:25.256 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:25.289 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:25 +0000 (0:00:00.053) 0:04:42.684 ********* 2020-08-07T15:01:25.303 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2020-08-07T15:01:25.538 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Friday 07 August 2020 15:01:25 +0000 (0:00:00.076) 0:04:42.761 ********* TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Friday 07 August 2020 15:01:25 +0000 (0:00:00.121) 0:04:42.882 ********* TASK [testnode : Enable abrt-auto-reporting] *********************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:25.571 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:25 +0000 (0:00:00.081) 0:04:42.964 ********* 2020-08-07T15:01:25.592 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2020-08-07T15:01:25.599 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:25.651 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:25 +0000 (0:00:00.082) 0:04:43.047 ********* 2020-08-07T15:01:25.663 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2020-08-07T15:01:25.667 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:25.721 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:25 +0000 (0:00:00.068) 0:04:43.115 ********* 2020-08-07T15:01:25.738 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:25.781 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:25 +0000 (0:00:00.060) 0:04:43.176 ********* 2020-08-07T15:01:25.794 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:25.846 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:25 +0000 (0:00:00.059) 0:04:43.236 ********* 2020-08-07T15:01:25.859 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:25.901 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:25 +0000 (0:00:00.058) 0:04:43.294 ********* 2020-08-07T15:01:25.916 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:25.960 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:25 +0000 (0:00:00.058) 0:04:43.353 ********* 2020-08-07T15:01:25.974 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-08-07T15:01:26.023 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:26 +0000 (0:00:00.060) 0:04:43.414 ********* 2020-08-07T15:01:26.032 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2020-08-07T15:01:26.038 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:26.090 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:26 +0000 (0:00:00.065) 0:04:43.480 ********* 2020-08-07T15:01:26.102 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:26.146 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:26 +0000 (0:00:00.057) 0:04:43.538 ********* 2020-08-07T15:01:26.155 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-08-07T15:01:26.206 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:26 +0000 (0:00:00.063) 0:04:43.602 ********* 2020-08-07T15:01:26.228 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:26.276 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:26 +0000 (0:00:00.068) 0:04:43.670 ********* 2020-08-07T15:01:26.289 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2020-08-07T15:01:26.291 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:26.334 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:26 +0000 (0:00:00.058) 0:04:43.729 ********* 2020-08-07T15:01:26.358 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-08-07T15:01:26.360 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-08-07T15:01:26.408 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:26 +0000 (0:00:00.074) 0:04:43.803 ********* 2020-08-07T15:01:26.430 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:26.477 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:26 +0000 (0:00:00.069) 0:04:43.873 ********* 2020-08-07T15:01:26.500 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-08-07T15:01:26.503 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:26.557 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:26 +0000 (0:00:00.077) 0:04:43.951 ********* 2020-08-07T15:01:26.581 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2020-08-07T15:01:26.583 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2020-08-07T15:01:26.646 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:26 +0000 (0:00:00.084) 0:04:44.035 ********* 2020-08-07T15:01:26.652 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-08-07T15:01:26.710 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:26 +0000 (0:00:00.070) 0:04:44.106 ********* 2020-08-07T15:01:26.722 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-08-07T15:01:26.724 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:26.771 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:26 +0000 (0:00:00.060) 0:04:44.166 ********* 2020-08-07T15:01:26.782 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:26.841 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:26 +0000 (0:00:00.069) 0:04:44.235 ********* 2020-08-07T15:01:26.854 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-08-07T15:01:26.902 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:26 +0000 (0:00:00.060) 0:04:44.296 ********* 2020-08-07T15:01:26.921 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:26.970 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:26 +0000 (0:00:00.066) 0:04:44.362 ********* 2020-08-07T15:01:26.977 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2020-08-07T15:01:26.984 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:27.022 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:27 +0000 (0:00:00.055) 0:04:44.418 ********* 2020-08-07T15:01:27.032 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2020-08-07T15:01:27.034 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:27.079 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:27 +0000 (0:00:00.055) 0:04:44.473 ********* 2020-08-07T15:01:27.092 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:27.131 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:27 +0000 (0:00:00.052) 0:04:44.526 ********* 2020-08-07T15:01:27.144 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-08-07T15:01:27.148 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:27.186 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:27 +0000 (0:00:00.055) 0:04:44.581 ********* 2020-08-07T15:01:27.199 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:27.242 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:27 +0000 (0:00:00.053) 0:04:44.634 ********* 2020-08-07T15:01:27.253 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-08-07T15:01:27.256 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:27.305 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:27 +0000 (0:00:00.065) 0:04:44.700 ********* 2020-08-07T15:01:27.320 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:27.358 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:27 +0000 (0:00:00.051) 0:04:44.751 ********* 2020-08-07T15:01:27.368 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:27.405 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:27 +0000 (0:00:00.049) 0:04:44.801 ********* 2020-08-07T15:01:27.418 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:27.460 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:27 +0000 (0:00:00.054) 0:04:44.855 ********* 2020-08-07T15:01:27.470 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-08-07T15:01:27.475 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:27.523 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:27 +0000 (0:00:00.063) 0:04:44.919 ********* 2020-08-07T15:01:27.526 INFO:teuthology.task.ansible.out: 2020-08-07T15:01:27.536 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-08-07T15:01:27.539 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:27.609 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:27 +0000 (0:00:00.085) 0:04:45.005 ********* 2020-08-07T15:01:27.634 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:27.679 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:27 +0000 (0:00:00.066) 0:04:45.071 ********* 2020-08-07T15:01:27.699 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:27.743 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:27 +0000 (0:00:00.065) 0:04:45.137 ********* 2020-08-07T15:01:27.752 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-08-07T15:01:27.754 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:27.796 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:27 +0000 (0:00:00.052) 0:04:45.189 ********* 2020-08-07T15:01:27.807 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-08-07T15:01:27.809 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:27.850 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:27 +0000 (0:00:00.056) 0:04:45.245 ********* 2020-08-07T15:01:27.875 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:27.918 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:27 +0000 (0:00:00.066) 0:04:45.312 ********* 2020-08-07T15:01:27.938 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:27.970 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:27 +0000 (0:00:00.050) 0:04:45.363 ********* 2020-08-07T15:01:27.979 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:28.023 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:28 +0000 (0:00:00.053) 0:04:45.416 ********* 2020-08-07T15:01:28.033 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:28.077 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:28 +0000 (0:00:00.050) 0:04:45.467 ********* 2020-08-07T15:01:28.087 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-08-07T15:01:28.089 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:28.131 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:28 +0000 (0:00:00.058) 0:04:45.525 ********* 2020-08-07T15:01:28.142 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2020-08-07T15:01:28.145 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:28.181 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:28 +0000 (0:00:00.050) 0:04:45.576 ********* 2020-08-07T15:01:28.194 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:28.237 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:28 +0000 (0:00:00.051) 0:04:45.628 ********* 2020-08-07T15:01:28.245 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:28.286 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:28 +0000 (0:00:00.050) 0:04:45.678 ********* 2020-08-07T15:01:28.299 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:28.339 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:28 +0000 (0:00:00.056) 0:04:45.734 ********* 2020-08-07T15:01:28.350 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:28.389 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:28 +0000 (0:00:00.050) 0:04:45.784 ********* 2020-08-07T15:01:28.400 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-08-07T15:01:28.402 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:28.441 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:28 +0000 (0:00:00.051) 0:04:45.836 ********* 2020-08-07T15:01:28.454 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-08-07T15:01:28.455 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:28.508 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:28 +0000 (0:00:00.067) 0:04:45.903 ********* 2020-08-07T15:01:28.522 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:28.574 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:28 +0000 (0:00:00.062) 0:04:45.966 ********* 2020-08-07T15:01:28.583 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:28.627 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:28 +0000 (0:00:00.054) 0:04:46.021 ********* 2020-08-07T15:01:28.637 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-08-07T15:01:28.641 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:28.685 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:28 +0000 (0:00:00.058) 0:04:46.080 ********* 2020-08-07T15:01:28.702 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-08-07T15:01:28.711 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:28.754 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:28 +0000 (0:00:00.065) 0:04:46.145 ********* 2020-08-07T15:01:28.770 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:28.818 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:28 +0000 (0:00:00.067) 0:04:46.212 ********* 2020-08-07T15:01:28.832 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:28.875 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:28 +0000 (0:00:00.057) 0:04:46.270 ********* 2020-08-07T15:01:28.889 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-08-07T15:01:28.891 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:28.942 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:28 +0000 (0:00:00.062) 0:04:46.333 ********* 2020-08-07T15:01:28.957 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:29.002 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:28 +0000 (0:00:00.062) 0:04:46.395 ********* 2020-08-07T15:01:29.011 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-08-07T15:01:29.014 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:29.056 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:29 +0000 (0:00:00.056) 0:04:46.451 ********* 2020-08-07T15:01:29.072 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:29.112 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:29 +0000 (0:00:00.055) 0:04:46.507 ********* 2020-08-07T15:01:29.129 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-08-07T15:01:29.131 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:29.170 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:29 +0000 (0:00:00.058) 0:04:46.565 ********* 2020-08-07T15:01:29.186 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:29.242 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:29 +0000 (0:00:00.070) 0:04:46.635 ********* 2020-08-07T15:01:29.261 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:29.342 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:29 +0000 (0:00:00.099) 0:04:46.735 ********* 2020-08-07T15:01:29.356 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:29.402 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:29 +0000 (0:00:00.056) 0:04:46.791 ********* 2020-08-07T15:01:29.407 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-08-07T15:01:29.414 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:29.482 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:29 +0000 (0:00:00.073) 0:04:46.865 ********* TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:29.554 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:29 +0000 (0:00:00.074) 0:04:46.939 ********* 2020-08-07T15:01:29.558 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:29.618 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:29 +0000 (0:00:00.073) 0:04:47.012 ********* 2020-08-07T15:01:29.652 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:29.696 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:29 +0000 (0:00:00.071) 0:04:47.083 ********* 2020-08-07T15:01:29.710 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:29.748 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:29 +0000 (0:00:00.060) 0:04:47.143 ********* 2020-08-07T15:01:29.763 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:29.809 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:29 +0000 (0:00:00.061) 0:04:47.205 ********* 2020-08-07T15:01:29.821 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-08-07T15:01:29.828 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:29.881 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:29 +0000 (0:00:00.069) 0:04:47.274 ********* 2020-08-07T15:01:29.894 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-08-07T15:01:29.898 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:29.950 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:29 +0000 (0:00:00.070) 0:04:47.344 ********* 2020-08-07T15:01:29.966 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:30.018 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:30 +0000 (0:00:00.067) 0:04:47.412 ********* 2020-08-07T15:01:30.030 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:30.078 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:30 +0000 (0:00:00.060) 0:04:47.473 ********* 2020-08-07T15:01:30.084 INFO:teuthology.task.ansible.out: 2020-08-07T15:01:30.099 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-08-07T15:01:30.101 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:30.168 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:30 +0000 (0:00:00.086) 0:04:47.560 ********* 2020-08-07T15:01:30.185 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:30.242 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:30 +0000 (0:00:00.066) 0:04:47.627 ********* 2020-08-07T15:01:30.244 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:30.303 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:30 +0000 (0:00:00.071) 0:04:47.698 ********* 2020-08-07T15:01:30.323 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:30.384 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:30 +0000 (0:00:00.078) 0:04:47.776 ********* 2020-08-07T15:01:30.408 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:30.454 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:30 +0000 (0:00:00.072) 0:04:47.849 ********* 2020-08-07T15:01:30.471 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:30.518 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:30 +0000 (0:00:00.054) 0:04:47.903 ********* 2020-08-07T15:01:30.522 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:30.572 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:30 +0000 (0:00:00.064) 0:04:47.968 ********* 2020-08-07T15:01:30.585 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:30.625 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:30 +0000 (0:00:00.052) 0:04:48.020 ********* 2020-08-07T15:01:30.647 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:30.694 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:30 +0000 (0:00:00.065) 0:04:48.086 ********* 2020-08-07T15:01:30.719 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:30.767 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:30 +0000 (0:00:00.075) 0:04:48.162 ********* 2020-08-07T15:01:30.784 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:30.824 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:30 +0000 (0:00:00.055) 0:04:48.218 ********* 2020-08-07T15:01:30.837 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:30.884 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:30 +0000 (0:00:00.059) 0:04:48.278 ********* 2020-08-07T15:01:30.898 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:30.949 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:30 +0000 (0:00:00.061) 0:04:48.340 ********* 2020-08-07T15:01:30.962 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-08-07T15:01:30.970 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:31.017 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:31 +0000 (0:00:00.071) 0:04:48.411 ********* 2020-08-07T15:01:31.034 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:31.116 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:31 +0000 (0:00:00.098) 0:04:48.510 ********* 2020-08-07T15:01:31.138 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-08-07T15:01:31.144 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:31.196 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:31 +0000 (0:00:00.080) 0:04:48.590 ********* 2020-08-07T15:01:31.211 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-08-07T15:01:31.213 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:31.266 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:31 +0000 (0:00:00.071) 0:04:48.661 ********* 2020-08-07T15:01:31.286 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-08-07T15:01:31.288 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:31.331 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:31 +0000 (0:00:00.064) 0:04:48.725 ********* 2020-08-07T15:01:31.344 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:31.388 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:31 +0000 (0:00:00.055) 0:04:48.781 ********* 2020-08-07T15:01:31.399 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-08-07T15:01:31.401 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:31.453 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:31 +0000 (0:00:00.066) 0:04:48.848 ********* 2020-08-07T15:01:31.465 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-08-07T15:01:31.466 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:31.514 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:31 +0000 (0:00:00.060) 0:04:48.909 ********* 2020-08-07T15:01:31.525 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:31.579 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:31 +0000 (0:00:00.061) 0:04:48.970 ********* 2020-08-07T15:01:31.586 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-08-07T15:01:31.588 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:31.628 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:31 +0000 (0:00:00.051) 0:04:49.022 ********* 2020-08-07T15:01:31.641 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:31.678 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:31 +0000 (0:00:00.051) 0:04:49.073 ********* 2020-08-07T15:01:31.689 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:31.737 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:31 +0000 (0:00:00.057) 0:04:49.131 ********* 2020-08-07T15:01:31.750 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:31.793 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:31 +0000 (0:00:00.057) 0:04:49.188 ********* 2020-08-07T15:01:31.807 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-08-07T15:01:31.809 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:31.849 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:31 +0000 (0:00:00.055) 0:04:49.243 ********* 2020-08-07T15:01:31.864 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:31.901 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:31 +0000 (0:00:00.053) 0:04:49.297 ********* 2020-08-07T15:01:31.914 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-08-07T15:01:31.914 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:31.954 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:31 +0000 (0:00:00.052) 0:04:49.349 ********* 2020-08-07T15:01:31.966 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2020-08-07T15:01:31.968 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:32.016 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:32 +0000 (0:00:00.055) 0:04:49.405 ********* 2020-08-07T15:01:32.021 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-08-07T15:01:32.023 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:32.077 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:32 +0000 (0:00:00.066) 0:04:49.471 ********* 2020-08-07T15:01:32.093 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:32.146 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:32 +0000 (0:00:00.066) 0:04:49.538 ********* 2020-08-07T15:01:32.156 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:32.209 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:32 +0000 (0:00:00.064) 0:04:49.602 ********* 2020-08-07T15:01:32.219 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-08-07T15:01:32.222 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:32.273 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:32 +0000 (0:00:00.064) 0:04:49.667 ********* 2020-08-07T15:01:32.286 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:32.337 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:32 +0000 (0:00:00.064) 0:04:49.732 ********* 2020-08-07T15:01:32.350 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:32.399 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:32 +0000 (0:00:00.062) 0:04:49.794 ********* 2020-08-07T15:01:32.417 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:32.459 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:32 +0000 (0:00:00.057) 0:04:49.852 ********* 2020-08-07T15:01:32.473 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:32.518 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:32 +0000 (0:00:00.061) 0:04:49.914 ********* 2020-08-07T15:01:32.529 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-08-07T15:01:32.535 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:32.588 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:32 +0000 (0:00:00.067) 0:04:49.981 ********* 2020-08-07T15:01:32.611 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:32.661 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:32 +0000 (0:00:00.074) 0:04:50.056 ********* 2020-08-07T15:01:32.671 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-08-07T15:01:32.673 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:32.724 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:32 +0000 (0:00:00.062) 0:04:50.118 ********* 2020-08-07T15:01:32.742 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:32.780 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:32 +0000 (0:00:00.057) 0:04:50.176 ********* 2020-08-07T15:01:32.794 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-08-07T15:01:32.836 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:32.868 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:32 +0000 (0:00:00.085) 0:04:50.261 ********* 2020-08-07T15:01:32.898 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:32.930 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:32 +0000 (0:00:00.053) 0:04:50.315 ********* 2020-08-07T15:01:32.938 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:32.977 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:32 +0000 (0:00:00.056) 0:04:50.372 ********* 2020-08-07T15:01:32.988 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:33.061 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:33 +0000 (0:00:00.081) 0:04:50.453 ********* 2020-08-07T15:01:33.082 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:33.116 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:33 +0000 (0:00:00.055) 0:04:50.508 ********* 2020-08-07T15:01:33.126 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:33.164 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:33 +0000 (0:00:00.051) 0:04:50.559 ********* 2020-08-07T15:01:33.176 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2020-08-07T15:01:33.178 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:33.224 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:33 +0000 (0:00:00.059) 0:04:50.619 ********* 2020-08-07T15:01:33.237 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-08-07T15:01:33.241 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:33.279 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:33 +0000 (0:00:00.055) 0:04:50.674 ********* 2020-08-07T15:01:33.295 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi074.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-08-07T15:01:33.330 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:33 +0000 (0:00:00.051) 0:04:50.725 ********* 2020-08-07T15:01:33.344 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:33.380 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:33 +0000 (0:00:00.050) 0:04:50.776 ********* 2020-08-07T15:01:33.402 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-08-07T15:01:33.404 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-08-07T15:01:33.419 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-08-07T15:01:33.421 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-08-07T15:01:33.458 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:33 +0000 (0:00:00.077) 0:04:50.853 ********* 2020-08-07T15:01:33.470 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi074.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-08-07T15:01:33.472 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-08-07T15:01:33.484 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-08-07T15:01:33.534 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:33 +0000 (0:00:00.073) 0:04:50.926 ********* 2020-08-07T15:01:33.547 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-08-07T15:01:33.560 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} skipping: [smithi096.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2020-08-07T15:01:33.575 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-08-07T15:01:33.634 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:33 +0000 (0:00:00.102) 0:04:51.029 ********* 2020-08-07T15:01:33.848 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** changed: [smithi074.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.013137", "end": "2020-08-07 15:01:33.785737", "rc": 0, "start": "2020-08-07 15:01:33.772600", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-08-07T15:01:33.899 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:33 +0000 (0:00:00.265) 0:04:51.294 ********* 2020-08-07T15:01:33.983 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:34.038 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:34 +0000 (0:00:00.138) 0:04:51.433 ********* 2020-08-07T15:01:34.057 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** changed: [smithi096.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.013102", "end": "2020-08-07 15:01:33.944481", "rc": 0, "start": "2020-08-07 15:01:33.931379", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-08-07T15:01:34.123 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:34 +0000 (0:00:00.082) 0:04:51.516 ********* 2020-08-07T15:01:34.147 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:34.278 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:34 +0000 (0:00:00.086) 0:04:51.602 ********* TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:34.298 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:34 +0000 (0:00:00.089) 0:04:51.692 ********* 2020-08-07T15:01:34.314 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:34.364 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:34 +0000 (0:00:00.066) 0:04:51.758 ********* 2020-08-07T15:01:34.379 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:34.438 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:34 +0000 (0:00:00.073) 0:04:51.831 ********* 2020-08-07T15:01:34.451 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:34.506 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:34 +0000 (0:00:00.064) 0:04:51.896 ********* 2020-08-07T15:01:34.532 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:34.616 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:34 +0000 (0:00:00.112) 0:04:52.009 ********* 2020-08-07T15:01:34.670 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:34.723 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:34 +0000 (0:00:00.107) 0:04:52.116 ********* 2020-08-07T15:01:36.724 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2020-08-07T15:01:36.731 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: gdisk", "Installed: device-mapper"]} 2020-08-07T15:01:36.773 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:36 +0000 (0:00:02.049) 0:04:54.166 ********* 2020-08-07T15:01:36.845 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ ok: [smithi074.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930490568704, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227170549, 'block_used': 12924559, 'inode_total': 61054976, 'inode_available': 60986774, 'inode_used': 68202, 'uuid': '0dcc33de-53c3-4ea8-baab-7ca3c59fe577'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227170549, "block_size": 4096, "block_total": 240095108, "block_used": 12924559, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60986774, "inode_total": 61054976, "inode_used": 68202, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930490568704, "size_total": 983429562368, "uuid": "0dcc33de-53c3-4ea8-baab-7ca3c59fe577"}} 2020-08-07T15:01:36.903 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:36 +0000 (0:00:00.132) 0:04:54.299 ********* 2020-08-07T15:01:36.990 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": ["Installed: gdisk", "Installed: device-mapper"]} 2020-08-07T15:01:37.028 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:37 +0000 (0:00:00.124) 0:04:54.423 ********* 2020-08-07T15:01:37.116 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ ok: [smithi096.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930490568704, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227170549, 'block_used': 12924559, 'inode_total': 61054976, 'inode_available': 60986774, 'inode_used': 68202, 'uuid': '0dcc33de-53c3-4ea8-baab-7ca3c59fe577'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227170549, "block_size": 4096, "block_total": 240095108, "block_used": 12924559, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60986774, "inode_total": 61054976, "inode_used": 68202, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930490568704, "size_total": 983429562368, "uuid": "0dcc33de-53c3-4ea8-baab-7ca3c59fe577"}} 2020-08-07T15:01:37.154 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:37 +0000 (0:00:00.125) 0:04:54.548 ********* 2020-08-07T15:01:37.171 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** changed: [smithi074.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.004909", "end": "2020-08-07 15:01:37.116772", "rc": 0, "start": "2020-08-07 15:01:37.111863", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-08-07T15:01:37.233 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:37 +0000 (0:00:00.075) 0:04:54.624 ********* 2020-08-07T15:01:37.309 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-08-07T15:01:37.311 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930490568704, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227170549, 'block_used': 12924559, 'inode_total': 61054976, 'inode_available': 60986774, 'inode_used': 68202, 'uuid': '0dcc33de-53c3-4ea8-baab-7ca3c59fe577'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227170549, "block_size": 4096, "block_total": 240095108, "block_used": 12924559, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60986774, "inode_total": 61054976, "inode_used": 68202, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930490568704, "size_total": 983429562368, "uuid": "0dcc33de-53c3-4ea8-baab-7ca3c59fe577"}, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:37.362 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:37 +0000 (0:00:00.130) 0:04:54.755 ********* 2020-08-07T15:01:37.408 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-08-07T15:01:37.413 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.005031", "end": "2020-08-07 15:01:37.369489", "rc": 0, "start": "2020-08-07 15:01:37.364458", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-08-07T15:01:37.450 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:37 +0000 (0:00:00.088) 0:04:54.843 ********* 2020-08-07T15:01:37.514 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:37 +0000 (0:00:00.060) 0:04:54.904 ********* 2020-08-07T15:01:37.580 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** skipping: [smithi096.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930490568704, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227170549, 'block_used': 12924559, 'inode_total': 61054976, 'inode_available': 60986774, 'inode_used': 68202, 'uuid': '0dcc33de-53c3-4ea8-baab-7ca3c59fe577'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227170549, "block_size": 4096, "block_total": 240095108, "block_used": 12924559, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60986774, "inode_total": 61054976, "inode_used": 68202, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930490568704, "size_total": 983429562368, "uuid": "0dcc33de-53c3-4ea8-baab-7ca3c59fe577"}, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:37.623 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:37 +0000 (0:00:00.111) 0:04:55.016 ********* 2020-08-07T15:01:37.672 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-08-07T15:01:37.730 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:37 +0000 (0:00:00.105) 0:04:55.122 ********* 2020-08-07T15:01:39.174 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi074.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-hQPbhH8bV8vDljFoKMBlUYXbiPoDh0LrJgA2rXcrFBR2OpgnIskqU2a4S7zOul8F'], 'uuids': ['98686a80-cb77-4eb3-9927-80357f771e24'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.413954", "end": "2020-08-07 15:01:39.116812", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-hQPbhH8bV8vDljFoKMBlUYXbiPoDh0LrJgA2rXcrFBR2OpgnIskqU2a4S7zOul8F"], "labels": [], "masters": [], "uuids": ["98686a80-cb77-4eb3-9927-80357f771e24"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-08-07 15:01:37.702858", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk u 2020-08-07T15:01:39.179 INFO:teuthology.task.ansible.out:sing fdisk or", "other utilities."]} 2020-08-07T15:01:39.299 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-SA5nt44e8AejLQNRZcmoAlKaCTd0oQ3m3hhnZJ0sRPq4YM32KoVT5rAl449qBBOa'], 'uuids': ['f7c1b61e-32c6-4bc8-bd75-ec0bbd23ad79'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.280686", "end": "2020-08-07 15:01:39.246428", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-SA5nt44e8AejLQNRZcmoAlKaCTd0oQ3m3hhnZJ0sRPq4YM32KoVT5rAl449qBBOa"], "labels": [], "masters": [], "uuids": ["f7c1b61e-32c6-4bc8-bd75-ec0bbd23ad79"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-08-07 15:01:37.965742", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-08-07T15:01:40.458 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-38HoTq-Rl5T-BCfn-svCe-jBPe-udvz-IskRVO', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400SB400BGN', 'nvme-nvme.8086-50484654363230343030534234303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.073081", "end": "2020-08-07 15:01:40.420640", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-38HoTq-Rl5T-BCfn-svCe-jBPe-udvz-IskRVO", "nvme-INTEL_SSDPEDMD400G4_PHFT620400SB400BGN", "nvme-nvme.8086-50484654363230343030534234303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-08-07 15:01:39.347559", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2020-08-07T15:01:40.466 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-08-07T15:01:40.616 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-SR5adZ-MJvD-H6YC-s1N2-FYts-WjuT-CTUKws', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400NH400BGN', 'nvme-nvme.8086-504846543632303430304e4834303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.058508", "end": "2020-08-07 15:01:40.576149", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-SR5adZ-MJvD-H6YC-s1N2-FYts-WjuT-CTUKws", "nvme-INTEL_SSDPEDMD400G4_PHFT620400NH400BGN", "nvme-nvme.8086-504846543632303430304e4834303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-08-07 15:01:39.517641", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2020-08-07T15:01:40.618 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-08-07T15:01:41.782 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-hQPbhH8bV8vDljFoKMBlUYXbiPoDh0LrL6xfUZxOUaUuvgnccQNXPKZ3mQjlm11g'], 'uuids': ['6c2fd4db-c8e4-42eb-814c-9c99eda84e74'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.073245", "end": "2020-08-07 15:01:41.744263", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-hQPbhH8bV8vDljFoKMBlUYXbiPoDh0LrL6xfUZxOUaUuvgnccQNXPKZ3mQjlm11g"], "labels": [], "masters": [], "uuids": ["6c2fd4db-c8e4-42eb-814c-9c99eda84e74"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-08-07 15:01:40.671018", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-08-07T15:01:41.888 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-SA5nt44e8AejLQNRZcmoAlKaCTd0oQ3mzA0vafKNSe6PZNJ1gNmLgtkG9s46Ffzj'], 'uuids': ['dd43e72e-94fb-4b4e-b29c-26e5f297a47d'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.044503", "end": "2020-08-07 15:01:41.837097", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-SA5nt44e8AejLQNRZcmoAlKaCTd0oQ3mzA0vafKNSe6PZNJ1gNmLgtkG9s46Ffzj"], "labels": [], "masters": [], "uuids": ["dd43e72e-94fb-4b4e-b29c-26e5f297a47d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-08-07 15:01:40.792594", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-08-07T15:01:43.048 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-hQPbhH8bV8vDljFoKMBlUYXbiPoDh0Lr07m1HaqEOGXo2Zt4GDmRanmjZ0G4tFHP'], 'uuids': ['da3677d2-5ebe-4b5e-be58-bebb38a9d74d'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.056998", "end": "2020-08-07 15:01:43.011218", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-hQPbhH8bV8vDljFoKMBlUYXbiPoDh0Lr07m1HaqEOGXo2Zt4GDmRanmjZ0G4tFHP"], "labels": [], "masters": [], "uuids": ["da3677d2-5ebe-4b5e-be58-bebb38a9d74d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-08-07 15:01:41.954220", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-08-07T15:01:43.166 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-SA5nt44e8AejLQNRZcmoAlKaCTd0oQ3mqjnrVc9pyPtTNxZdIRGA2OzoK018DhGs'], 'uuids': ['9805a10e-8c00-4db1-9d19-ef903726938d'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.061414", "end": "2020-08-07 15:01:43.127970", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-SA5nt44e8AejLQNRZcmoAlKaCTd0oQ3mqjnrVc9pyPtTNxZdIRGA2OzoK018DhGs"], "labels": [], "masters": [], "uuids": ["9805a10e-8c00-4db1-9d19-ef903726938d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-08-07 15:01:42.066556", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-08-07T15:01:44.326 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-hQPbhH8bV8vDljFoKMBlUYXbiPoDh0LrK26ujDQ7ygSGSRR0vBHv58COq5rq7zFD'], 'uuids': ['ec0a6c3f-61a5-4c53-838f-90947db7d04c'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.061183", "end": "2020-08-07 15:01:44.286179", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-hQPbhH8bV8vDljFoKMBlUYXbiPoDh0LrK26ujDQ7ygSGSRR0vBHv58COq5rq7zFD"], "labels": [], "masters": [], "uuids": ["ec0a6c3f-61a5-4c53-838f-90947db7d04c"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-08-07 15:01:43.224996", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-08-07T15:01:44.338 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QD01', 'wwn-0x5000c50091e03265'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W5QD01', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c50091e03265', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QD01-part1', 'wwn-0x5000c50091e03265-part1'], 'uuids': ['0dcc33de-53c3-4ea8-baab-7ca3c59fe577'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '0dcc33de-53c3-4ea8-baab-7ca3c59fe577', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QD01", "wwn-0x5000c50091e03265"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QD01-part1", "wwn-0x5000c50091e03265-part1"], "labels": [], "masters": [], "uuids": ["0dcc33de-53c3-4ea8-baab-7ca3c59fe577"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "0dcc33de-53c3-4ea8-baab-7ca3c59fe577"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QD01", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091e03265"}}, 2020-08-07T15:01:44.341 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-08-07T15:01:44.441 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-SA5nt44e8AejLQNRZcmoAlKaCTd0oQ3mE4pPtbKcs5ITe1Uctwv0JRo0nVugDtwT'], 'uuids': ['0967cf46-fa71-4d22-b870-7337d8c2d002'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.051680", "end": "2020-08-07 15:01:44.396341", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-SA5nt44e8AejLQNRZcmoAlKaCTd0oQ3mE4pPtbKcs5ITe1Uctwv0JRo0nVugDtwT"], "labels": [], "masters": [], "uuids": ["0967cf46-fa71-4d22-b870-7337d8c2d002"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-08-07 15:01:43.344661", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-08-07T15:01:44.445 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QGMP', 'wwn-0x5000c50091dfbebc'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W5QGMP', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c50091dfbebc', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QGMP-part1', 'wwn-0x5000c50091dfbebc-part1'], 'uuids': ['0dcc33de-53c3-4ea8-baab-7ca3c59fe577'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '0dcc33de-53c3-4ea8-baab-7ca3c59fe577', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QGMP", "wwn-0x5000c50091dfbebc"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QGMP-part1", "wwn-0x5000c50091dfbebc-part1"], "labels": [], "masters": [], "uuids": ["0dcc33de-53c3-4ea8-baab-7ca3c59fe577"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "0dcc33de-53c3-4ea8-baab-7ca3c59fe577"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QGMP", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091dfbebc"}}, 2020-08-07T15:01:44.450 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-08-07T15:01:45.602 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-hQPbhH8bV8vDljFoKMBlUYXbiPoDh0LrNH0AUqwiL0HhgXLpS1mulPtZaH1ik68a'], 'uuids': ['448beff7-57d6-4133-a25f-ab1136fc51ae'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.049674", "end": "2020-08-07 15:01:45.562828", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-hQPbhH8bV8vDljFoKMBlUYXbiPoDh0LrNH0AUqwiL0HhgXLpS1mulPtZaH1ik68a"], "labels": [], "masters": [], "uuids": ["448beff7-57d6-4133-a25f-ab1136fc51ae"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-08-07 15:01:44.513154", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-08-07T15:01:45.746 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:45 +0000 (0:00:08.015) 0:05:03.138 ********* 2020-08-07T15:01:45.774 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-SA5nt44e8AejLQNRZcmoAlKaCTd0oQ3mtmtHa06IOEiRByRwfEbBCPXOtOhq3y1Q'], 'uuids': ['a2a08521-f63e-4b06-b089-05690fdbe4b8'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.047565", "end": "2020-08-07 15:01:45.678764", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-SA5nt44e8AejLQNRZcmoAlKaCTd0oQ3mtmtHa06IOEiRByRwfEbBCPXOtOhq3y1Q"], "labels": [], "masters": [], "uuids": ["a2a08521-f63e-4b06-b089-05690fdbe4b8"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-08-07 15:01:44.631199", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-08-07T15:01:45.918 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:45 +0000 (0:00:00.171) 0:05:03.309 ********* 2020-08-07T15:01:46.194 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [smithi074.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-hQPbhH8bV8vDljFoKMBlUYXbiPoDh0LrJgA2rXcrFBR2OpgnIskqU2a4S7zOul8F'], 'uuids': ['98686a80-cb77-4eb3-9927-80357f771e24'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.141065", "end": "2020-08-07 15:01:46.156111", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-hQPbhH8bV8vDljFoKMBlUYXbiPoDh0LrJgA2rXcrFBR2OpgnIskqU2a4S7zOul8F"], "labels": [], "masters": [], "uuids": ["98686a80-cb77-4eb3-9927-80357f771e24"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-08-07 15:01:46.015046", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.138402 s, 833 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.138402 s, 833 MB/s"], "stdout": "", "stdout_lines": []} 2020-08-07T15:01:46.377 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-SA5nt44e8AejLQNRZcmoAlKaCTd0oQ3m3hhnZJ0sRPq4YM32KoVT5rAl449qBBOa'], 'uuids': ['f7c1b61e-32c6-4bc8-bd75-ec0bbd23ad79'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.139254", "end": "2020-08-07 15:01:46.324560", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-SA5nt44e8AejLQNRZcmoAlKaCTd0oQ3m3hhnZJ0sRPq4YM32KoVT5rAl449qBBOa"], "labels": [], "masters": [], "uuids": ["f7c1b61e-32c6-4bc8-bd75-ec0bbd23ad79"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-08-07 15:01:46.185306", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.136541 s, 845 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.136541 s, 845 MB/s"], "stdout": "", "stdout_lines": []} 2020-08-07T15:01:46.454 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-38HoTq-Rl5T-BCfn-svCe-jBPe-udvz-IskRVO', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400SB400BGN', 'nvme-nvme.8086-50484654363230343030534234303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.036087", "end": "2020-08-07 15:01:46.405702", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-38HoTq-Rl5T-BCfn-svCe-jBPe-udvz-IskRVO", "nvme-INTEL_SSDPEDMD400G4_PHFT620400SB400BGN", "nvme-nvme.8086-50484654363230343030534234303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-08-07 15:01:46.369615", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0335897 s, 3.4 GB/s", "std 2020-08-07T15:01:46.457 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0335897 s, 3.4 GB/s"], "stdout": "", "stdout_lines": []} 2020-08-07T15:01:46.633 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-SR5adZ-MJvD-H6YC-s1N2-FYts-WjuT-CTUKws', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400NH400BGN', 'nvme-nvme.8086-504846543632303430304e4834303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.036997", "end": "2020-08-07 15:01:46.597908", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-SR5adZ-MJvD-H6YC-s1N2-FYts-WjuT-CTUKws", "nvme-INTEL_SSDPEDMD400G4_PHFT620400NH400BGN", "nvme-nvme.8086-504846543632303430304e4834303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-08-07 15:01:46.560911", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0344972 s, 3.3 GB/s", "std 2020-08-07T15:01:46.635 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0344972 s, 3.3 GB/s"], "stdout": "", "stdout_lines": []} 2020-08-07T15:01:46.810 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-hQPbhH8bV8vDljFoKMBlUYXbiPoDh0LrL6xfUZxOUaUuvgnccQNXPKZ3mQjlm11g'], 'uuids': ['6c2fd4db-c8e4-42eb-814c-9c99eda84e74'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.142952", "end": "2020-08-07 15:01:46.780585", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-hQPbhH8bV8vDljFoKMBlUYXbiPoDh0LrL6xfUZxOUaUuvgnccQNXPKZ3mQjlm11g"], "labels": [], "masters": [], "uuids": ["6c2fd4db-c8e4-42eb-814c-9c99eda84e74"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-08-07 15:01:46.637633", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.140307 s, 822 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.140307 s, 822 MB/s"], "stdout": "", "stdout_lines": []} 2020-08-07T15:01:46.977 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-SA5nt44e8AejLQNRZcmoAlKaCTd0oQ3mzA0vafKNSe6PZNJ1gNmLgtkG9s46Ffzj'], 'uuids': ['dd43e72e-94fb-4b4e-b29c-26e5f297a47d'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.138491", "end": "2020-08-07 15:01:46.948701", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-SA5nt44e8AejLQNRZcmoAlKaCTd0oQ3mzA0vafKNSe6PZNJ1gNmLgtkG9s46Ffzj"], "labels": [], "masters": [], "uuids": ["dd43e72e-94fb-4b4e-b29c-26e5f297a47d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-08-07 15:01:46.810210", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.1361 s, 847 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.1361 s, 847 MB/s"], "stdout": "", "stdout_lines": []} 2020-08-07T15:01:47.217 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-hQPbhH8bV8vDljFoKMBlUYXbiPoDh0Lr07m1HaqEOGXo2Zt4GDmRanmjZ0G4tFHP'], 'uuids': ['da3677d2-5ebe-4b5e-be58-bebb38a9d74d'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.142295", "end": "2020-08-07 15:01:47.187204", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-hQPbhH8bV8vDljFoKMBlUYXbiPoDh0Lr07m1HaqEOGXo2Zt4GDmRanmjZ0G4tFHP"], "labels": [], "masters": [], "uuids": ["da3677d2-5ebe-4b5e-be58-bebb38a9d74d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-08-07 15:01:47.044909", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.139673 s, 826 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.139673 s, 826 MB/s"], "stdout": "", "stdout_lines": []} 2020-08-07T15:01:47.381 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-SA5nt44e8AejLQNRZcmoAlKaCTd0oQ3mqjnrVc9pyPtTNxZdIRGA2OzoK018DhGs'], 'uuids': ['9805a10e-8c00-4db1-9d19-ef903726938d'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.139262", "end": "2020-08-07 15:01:47.349493", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-SA5nt44e8AejLQNRZcmoAlKaCTd0oQ3mqjnrVc9pyPtTNxZdIRGA2OzoK018DhGs"], "labels": [], "masters": [], "uuids": ["9805a10e-8c00-4db1-9d19-ef903726938d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-08-07 15:01:47.210231", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.136747 s, 843 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.136747 s, 843 MB/s"], "stdout": "", "stdout_lines": []} 2020-08-07T15:01:47.582 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-hQPbhH8bV8vDljFoKMBlUYXbiPoDh0LrK26ujDQ7ygSGSRR0vBHv58COq5rq7zFD'], 'uuids': ['ec0a6c3f-61a5-4c53-838f-90947db7d04c'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.143339", "end": "2020-08-07 15:01:47.550116", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-hQPbhH8bV8vDljFoKMBlUYXbiPoDh0LrK26ujDQ7ygSGSRR0vBHv58COq5rq7zFD"], "labels": [], "masters": [], "uuids": ["ec0a6c3f-61a5-4c53-838f-90947db7d04c"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-08-07 15:01:47.406777", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.140727 s, 820 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.140727 s, 820 MB/s"], "stdout": "", "stdout_lines": []} 2020-08-07T15:01:47.594 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QD01', 'wwn-0x5000c50091e03265'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W5QD01', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c50091e03265', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QD01-part1', 'wwn-0x5000c50091e03265-part1'], 'uuids': ['0dcc33de-53c3-4ea8-baab-7ca3c59fe577'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '0dcc33de-53c3-4ea8-baab-7ca3c59fe577', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QD01", "wwn-0x5000c50091e03265"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QD01-part1", "wwn-0x5000c50091e03265-part1"], "labels": [], "masters": [], "uuids": ["0dcc33de-53c3-4ea8-baab-7ca3c59fe577"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "0dcc33de-53c3-4ea8-baab-7ca3c59fe577"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QD01", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091e03265"}}, 2020-08-07T15:01:47.602 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-08-07T15:01:47.754 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-SA5nt44e8AejLQNRZcmoAlKaCTd0oQ3mE4pPtbKcs5ITe1Uctwv0JRo0nVugDtwT'], 'uuids': ['0967cf46-fa71-4d22-b870-7337d8c2d002'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.142019", "end": "2020-08-07 15:01:47.696643", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-SA5nt44e8AejLQNRZcmoAlKaCTd0oQ3mE4pPtbKcs5ITe1Uctwv0JRo0nVugDtwT"], "labels": [], "masters": [], "uuids": ["0967cf46-fa71-4d22-b870-7337d8c2d002"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-08-07 15:01:47.554624", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.139466 s, 827 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.139466 s, 827 MB/s"], "stdout": "", "stdout_lines": []} skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QGMP', 'wwn-0x5000c50091dfbebc'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W5QGMP', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c50091dfbebc', 'partition 2020-08-07T15:01:47.758 INFO:teuthology.task.ansible.out:s': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QGMP-part1', 'wwn-0x5000c50091dfbebc-part1'], 'uuids': ['0dcc33de-53c3-4ea8-baab-7ca3c59fe577'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': '0dcc33de-53c3-4ea8-baab-7ca3c59fe577', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QGMP", "wwn-0x5000c50091dfbebc"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QGMP-part1", "wwn-0x5000c50091dfbebc-part1"], "labels": [], "masters": [], "uuids": ["0dcc33de-53c3-4ea8-baab-7ca3c59fe577"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "0dcc33de-53c3-4ea8-baab-7ca3c59fe577"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QGMP", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091dfbebc"}}, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:47.955 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-hQPbhH8bV8vDljFoKMBlUYXbiPoDh0LrNH0AUqwiL0HhgXLpS1mulPtZaH1ik68a'], 'uuids': ['448beff7-57d6-4133-a25f-ab1136fc51ae'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.140255", "end": "2020-08-07 15:01:47.906962", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-hQPbhH8bV8vDljFoKMBlUYXbiPoDh0LrNH0AUqwiL0HhgXLpS1mulPtZaH1ik68a"], "labels": [], "masters": [], "uuids": ["448beff7-57d6-4133-a25f-ab1136fc51ae"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-08-07 15:01:47.766707", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.13768 s, 838 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.13768 s, 838 MB/s"], "stdout": "", "stdout_lines": []} 2020-08-07T15:01:48.026 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:48 +0000 (0:00:02.099) 0:05:05.409 ********* 2020-08-07T15:01:48.136 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-SA5nt44e8AejLQNRZcmoAlKaCTd0oQ3mtmtHa06IOEiRByRwfEbBCPXOtOhq3y1Q'], 'uuids': ['a2a08521-f63e-4b06-b089-05690fdbe4b8'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.138917", "end": "2020-08-07 15:01:48.072665", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-SA5nt44e8AejLQNRZcmoAlKaCTd0oQ3mtmtHa06IOEiRByRwfEbBCPXOtOhq3y1Q"], "labels": [], "masters": [], "uuids": ["a2a08521-f63e-4b06-b089-05690fdbe4b8"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-08-07 15:01:47.933748", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.1364 s, 846 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.1364 s, 846 MB/s"], "stdout": "", "stdout_lines": []} 2020-08-07T15:01:48.186 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:48 +0000 (0:00:00.166) 0:05:05.575 ********* 2020-08-07T15:01:48.322 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** changed: [smithi074.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.059578", "end": "2020-08-07 15:01:48.278405", "rc": 0, "start": "2020-08-07 15:01:48.218827", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-08-07T15:01:48.384 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:48 +0000 (0:00:00.198) 0:05:05.776 ********* 2020-08-07T15:01:48.500 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.054096", "end": "2020-08-07 15:01:48.451120", "rc": 0, "start": "2020-08-07 15:01:48.397024", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-08-07T15:01:48.554 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:48 +0000 (0:00:00.167) 0:05:05.943 ********* 2020-08-07T15:01:48.647 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-08-07T15:01:48.653 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.042954", "end": "2020-08-07 15:01:48.616049", "rc": 0, "start": "2020-08-07 15:01:48.573095", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-08-07T15:01:48.687 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:48 +0000 (0:00:00.138) 0:05:06.082 ********* 2020-08-07T15:01:48.745 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-08-07T15:01:48.793 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:48 +0000 (0:00:00.106) 0:05:06.188 ********* 2020-08-07T15:01:48.820 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.042576", "end": "2020-08-07 15:01:48.776487", "rc": 0, "start": "2020-08-07 15:01:48.733911", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-08-07T15:01:48.862 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:48 +0000 (0:00:00.068) 0:05:06.257 ********* 2020-08-07T15:01:48.893 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:48.926 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:48 +0000 (0:00:00.064) 0:05:06.321 ********* 2020-08-07T15:01:48.936 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-08-07T15:01:49.008 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:49 +0000 (0:00:00.082) 0:05:06.403 ********* 2020-08-07T15:01:49.031 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:49.065 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:49 +0000 (0:00:00.056) 0:05:06.460 ********* 2020-08-07T15:01:49.083 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:49.128 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:49 +0000 (0:00:00.059) 0:05:06.520 ********* 2020-08-07T15:01:49.136 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:49.185 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:49 +0000 (0:00:00.060) 0:05:06.580 ********* 2020-08-07T15:01:49.196 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2020-08-07T15:01:49.199 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:49.243 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:49 +0000 (0:00:00.054) 0:05:06.635 ********* 2020-08-07T15:01:49.251 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-08-07T15:01:49.255 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:49.293 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:49 +0000 (0:00:00.052) 0:05:06.688 ********* 2020-08-07T15:01:49.303 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-08-07T15:01:49.306 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:49.359 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:49 +0000 (0:00:00.063) 0:05:06.752 ********* 2020-08-07T15:01:49.422 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2020-08-07T15:01:49.428 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:49.465 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:49 +0000 (0:00:00.107) 0:05:06.860 ********* 2020-08-07T15:01:49.527 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-08-07T15:01:49.530 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-08-07T15:01:49.582 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:49 +0000 (0:00:00.114) 0:05:06.975 ********* 2020-08-07T15:01:49.646 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ skipping: [smithi074.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930490568704, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227170549, 'block_used': 12924559, 'inode_total': 61054976, 'inode_available': 60986774, 'inode_used': 68202, 'uuid': '0dcc33de-53c3-4ea8-baab-7ca3c59fe577'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227170549, "block_size": 4096, "block_total": 240095108, "block_used": 12924559, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60986774, "inode_total": 61054976, "inode_used": 68202, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930490568704, "size_total": 983429562368, "uuid": "0dcc33de-53c3-4ea8-baab-7ca3c59fe577"}, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:49.680 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:49 +0000 (0:00:00.099) 0:05:07.075 ********* 2020-08-07T15:01:49.711 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ ok: [smithi096.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-08-07T15:01:49.755 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:49 +0000 (0:00:00.075) 0:05:07.150 ********* 2020-08-07T15:01:49.766 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:49.815 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:49 +0000 (0:00:00.059) 0:05:07.210 ********* 2020-08-07T15:01:49.830 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-08-07T15:01:49.832 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930490568704, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227170549, 'block_used': 12924559, 'inode_total': 61054976, 'inode_available': 60986774, 'inode_used': 68202, 'uuid': '0dcc33de-53c3-4ea8-baab-7ca3c59fe577'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227170549, "block_size": 4096, "block_total": 240095108, "block_used": 12924559, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60986774, "inode_total": 61054976, "inode_used": 68202, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930490568704, "size_total": 983429562368, "uuid": "0dcc33de-53c3-4ea8-baab-7ca3c59fe577"}, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:49.882 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:49 +0000 (0:00:00.061) 0:05:07.272 ********* 2020-08-07T15:01:49.889 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:49.930 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:49 +0000 (0:00:00.053) 0:05:07.325 ********* 2020-08-07T15:01:49.944 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-08-07T15:01:49.945 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:49.998 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:49 +0000 (0:00:00.060) 0:05:07.386 ********* 2020-08-07T15:01:50.004 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-08-07T15:01:50.008 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:50.046 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:50 +0000 (0:00:00.052) 0:05:07.438 ********* 2020-08-07T15:01:50.054 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-08-07T15:01:50.055 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:50.104 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:50 +0000 (0:00:00.058) 0:05:07.497 ********* 2020-08-07T15:01:50.119 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:50.163 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:50 +0000 (0:00:00.061) 0:05:07.558 ********* 2020-08-07T15:01:50.178 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:50.220 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:50 +0000 (0:00:00.057) 0:05:07.616 ********* 2020-08-07T15:01:50.230 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-08-07T15:01:50.231 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:50.284 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:50 +0000 (0:00:00.063) 0:05:07.679 ********* 2020-08-07T15:01:50.297 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-08-07T15:01:50.299 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:50.347 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:50 +0000 (0:00:00.061) 0:05:07.741 ********* 2020-08-07T15:01:50.362 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:50.414 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:50 +0000 (0:00:00.064) 0:05:07.806 ********* 2020-08-07T15:01:50.426 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:50.486 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:50 +0000 (0:00:00.071) 0:05:07.878 ********* 2020-08-07T15:01:50.563 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:50.618 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:50 +0000 (0:00:00.135) 0:05:08.013 ********* 2020-08-07T15:01:51.293 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-08-07T15:01:51.340 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:51 +0000 (0:00:00.722) 0:05:08.736 ********* 2020-08-07T15:01:51.357 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-08-07T15:01:51.438 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:51 +0000 (0:00:00.097) 0:05:08.833 ********* 2020-08-07T15:01:52.077 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-08-07T15:01:52.079 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-08-07T15:01:52.130 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-08-07T15:01:52.466 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-08-07T15:01:52.490 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-08-07T15:01:52.867 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-08-07T15:01:53.216 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-08-07T15:01:53.259 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-08-07T15:01:53.571 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-08-07T15:01:53.615 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:53 +0000 (0:00:02.176) 0:05:11.010 ********* 2020-08-07T15:01:53.645 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-08-07T15:01:53.701 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:53 +0000 (0:00:00.085) 0:05:11.096 ********* 2020-08-07T15:01:53.855 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2020-08-07T15:01:53.857 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-08-07T15:01:53.902 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:53 +0000 (0:00:00.195) 0:05:11.291 ********* 2020-08-07T15:01:53.934 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-08-07T15:01:53.981 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:53 +0000 (0:00:00.084) 0:05:11.376 ********* 2020-08-07T15:01:54.180 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-08-07T15:01:54.183 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-08-07T15:01:54.298 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-08-07T15:01:54.381 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-08-07T15:01:54.502 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-08-07T15:01:54.584 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-08-07T15:01:54.714 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-08-07T15:01:54.794 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-08-07T15:01:54.825 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:54.882 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:54 +0000 (0:00:00.894) 0:05:12.271 ********* 2020-08-07T15:01:54.932 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-08-07T15:01:54.960 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:55.010 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:55 +0000 (0:00:00.131) 0:05:12.402 ********* 2020-08-07T15:01:55.129 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-08-07T15:01:55.131 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-08-07T15:01:55.179 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:55 +0000 (0:00:00.170) 0:05:12.573 ********* 2020-08-07T15:01:55.230 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:55.283 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:55 +0000 (0:00:00.103) 0:05:12.676 ********* 2020-08-07T15:01:55.312 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2020-08-07T15:01:55.376 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:55 +0000 (0:00:00.092) 0:05:12.769 ********* 2020-08-07T15:01:55.440 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:55.478 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:55 +0000 (0:00:00.104) 0:05:12.873 ********* 2020-08-07T15:01:56.970 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* changed: [smithi096.front.sepia.ceph.com] => {"changed": true} changed: [smithi074.front.sepia.ceph.com] => {"changed": true} 2020-08-07T15:01:57.044 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:57 +0000 (0:00:01.565) 0:05:14.439 ********* 2020-08-07T15:01:57.137 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:57 +0000 (0:00:00.093) 0:05:14.532 ********* 2020-08-07T15:01:57.749 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-08-07T15:01:57.797 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:57 +0000 (0:00:00.660) 0:05:15.192 ********* 2020-08-07T15:01:57.811 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-08-07T15:01:57.868 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:57 +0000 (0:00:00.070) 0:05:15.263 ********* 2020-08-07T15:01:57.887 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-08-07T15:01:57.889 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:57.942 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:57 +0000 (0:00:00.066) 0:05:15.330 ********* 2020-08-07T15:01:57.945 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:57.984 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:57 +0000 (0:00:00.049) 0:05:15.379 ********* 2020-08-07T15:01:57.995 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-08-07T15:01:57.997 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:58.037 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:58 +0000 (0:00:00.053) 0:05:15.432 ********* 2020-08-07T15:01:58.049 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:58.090 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:58 +0000 (0:00:00.052) 0:05:15.485 ********* 2020-08-07T15:01:58.109 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:58.144 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:58 +0000 (0:00:00.054) 0:05:15.539 ********* 2020-08-07T15:01:58.167 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:58.209 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:58 +0000 (0:00:00.063) 0:05:15.603 ********* 2020-08-07T15:01:58.221 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2020-08-07T15:01:58.223 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:58.261 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:58 +0000 (0:00:00.051) 0:05:15.655 ********* 2020-08-07T15:01:58.272 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:58.319 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:58 +0000 (0:00:00.059) 0:05:15.714 ********* 2020-08-07T15:01:58.335 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:58.385 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:58 +0000 (0:00:00.065) 0:05:15.780 ********* 2020-08-07T15:01:58.398 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:58.439 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:58 +0000 (0:00:00.053) 0:05:15.834 ********* 2020-08-07T15:01:58.454 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:58.491 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:58 +0000 (0:00:00.051) 0:05:15.886 ********* 2020-08-07T15:01:58.502 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:58.543 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:58 +0000 (0:00:00.052) 0:05:15.939 ********* 2020-08-07T15:01:58.557 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2020-08-07T15:01:58.559 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:58.616 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:58 +0000 (0:00:00.065) 0:05:16.004 ********* 2020-08-07T15:01:58.622 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:58.660 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:58 +0000 (0:00:00.051) 0:05:16.056 ********* 2020-08-07T15:01:58.671 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-08-07T15:01:58.673 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:58.709 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:58 +0000 (0:00:00.048) 0:05:16.105 ********* 2020-08-07T15:01:58.721 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:58.775 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:58 +0000 (0:00:00.065) 0:05:16.170 ********* 2020-08-07T15:01:58.792 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:58.833 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:58 +0000 (0:00:00.056) 0:05:16.226 ********* 2020-08-07T15:01:58.842 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:58.881 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:58 +0000 (0:00:00.048) 0:05:16.275 ********* 2020-08-07T15:01:58.890 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-08-07T15:01:58.893 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:58.933 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:58 +0000 (0:00:00.052) 0:05:16.328 ********* 2020-08-07T15:01:58.944 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:58.994 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:58 +0000 (0:00:00.060) 0:05:16.389 ********* 2020-08-07T15:01:59.033 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:59.075 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:59 +0000 (0:00:00.081) 0:05:16.470 ********* 2020-08-07T15:01:59.092 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:01:59.137 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:59 +0000 (0:00:00.061) 0:05:16.532 ********* 2020-08-07T15:01:59.784 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-08-07T15:01:59.786 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "a7ecef00073feab34f868065d34e9dac55134241", "dest": "/etc/chrony.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/chrony.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 215, "state": "file", "uid": 0} 2020-08-07T15:01:59.828 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:59 +0000 (0:00:00.691) 0:05:17.223 ********* 2020-08-07T15:01:59.853 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "checksum": "a7ecef00073feab34f868065d34e9dac55134241", "dest": "/etc/chrony.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/chrony.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 215, "state": "file", "uid": 0} 2020-08-07T15:01:59.896 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:01:59 +0000 (0:00:00.066) 0:05:17.290 ********* 2020-08-07T15:02:00.197 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "chronyd", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2020-08-07 14:55:42 UTC", "ActiveEnterTimestampMonotonic": "16476354", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target tmp.mount system.slice sntp.service systemd-tmpfiles-setup.service ntpdate.service ntpd.service -.mount sysinit.target systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2020-08-07 14:55:40 UTC", "AssertTimestampMonotonic": "15216216", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2020-08-07 14:55:40 UTC", "ConditionTimestampMonotonic": "15216186", "ConfigurationDirectoryMode": "0755", "Conflicts": "ntpd.service shutdown.target systemd-timesyncd.service", "ControlGroup": "/system.slice/chronyd.servic 2020-08-07T15:02:00.199 INFO:teuthology.task.ansible.out:e", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "NTP client/server", "DevicePolicy": "auto", "Documentation": "man:chronyd(8) man:chrony.conf(5)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/chronyd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2445", "ExecMainStartTimestamp": "Fri 2020-08-07 14:55:41 UTC", "ExecMainStartTimestampMonotonic": "16366327", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/chronyd ; argv[]=/usr/sbin/chronyd $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartPost": "{ path=/usr/libexec/chrony-helper ; argv[]=/usr/libexec/chrony-helper update-daemon ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/chronyd.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "chronyd.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Fri 2020-08-07 14:55:40 UTC", "InactiveExitTimestampMonotonic": "15247502", "InvocationID": "3e2b369c024a4077ad9e3923cf7ca3f4", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infini 2020-08-07T15:02:00.201 INFO:teuthology.task.ansible.out:ty", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127425", "LimitNPROCSoft": "127425", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127425", "LimitSIGPENDINGSoft": "127425", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2445", "MemoryAccounting": "yes", "MemoryCurrent": "3043328", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "chronyd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PIDFile": "/run/chrony/chronyd.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "yes", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "full", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits" 2020-08-07T15:02:00.203 INFO:teuthology.task.ansible.out:: "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2020-08-07 14:55:42 UTC", "StateChangeTimestampMonotonic": "16476354", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2020-08-07 14:55:41 UTC", "WatchdogTimestampMonotonic": "16366337", "WatchdogUSec": "0"}} 2020-08-07T15:02:00.238 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:00 +0000 (0:00:00.343) 0:05:17.633 ********* 2020-08-07T15:02:00.268 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "chronyd", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2020-08-07 14:55:36 UTC", "ActiveEnterTimestampMonotonic": "15364828", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-tmpfiles-setup.service basic.target -.mount ntpd.service tmp.mount sntp.service system.slice ntpdate.service sysinit.target systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2020-08-07 14:55:36 UTC", "AssertTimestampMonotonic": "15015266", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2020-08-07 14:55:36 UTC", "ConditionTimestampMonotonic": "15015245", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service ntpd.service shutdown.target", "ControlGroup": "/system.slice/chronyd.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description" 2020-08-07T15:02:00.270 INFO:teuthology.task.ansible.out:: "NTP client/server", "DevicePolicy": "auto", "Documentation": "man:chronyd(8) man:chrony.conf(5)", "DynamicUser": "no", "EnvironmentFiles": "/etc/sysconfig/chronyd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "2444", "ExecMainStartTimestamp": "Fri 2020-08-07 14:55:36 UTC", "ExecMainStartTimestampMonotonic": "15169689", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/chronyd ; argv[]=/usr/sbin/chronyd $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartPost": "{ path=/usr/libexec/chrony-helper ; argv[]=/usr/libexec/chrony-helper update-daemon ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/chronyd.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "chronyd.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Fri 2020-08-07 14:55:36 UTC", "InactiveExitTimestampMonotonic": "15071953", "InvocationID": "d12403fde4c640379aff578c16919f7f", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "167772 2020-08-07T15:02:00.272 INFO:teuthology.task.ansible.out:16", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127424", "LimitNPROCSoft": "127424", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127424", "LimitSIGPENDINGSoft": "127424", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "2444", "MemoryAccounting": "yes", "MemoryCurrent": "2457600", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "chronyd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PIDFile": "/run/chrony/chronyd.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "yes", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "full", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardE 2020-08-07T15:02:00.275 INFO:teuthology.task.ansible.out:rror": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2020-08-07 14:55:36 UTC", "StateChangeTimestampMonotonic": "15364828", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2020-08-07 14:55:36 UTC", "WatchdogTimestampMonotonic": "15169705", "WatchdogUSec": "0"}} 2020-08-07T15:02:00.310 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:00 +0000 (0:00:00.068) 0:05:17.701 ********* 2020-08-07T15:02:00.323 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-08-07T15:02:00.365 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:00 +0000 (0:00:00.058) 0:05:17.760 ********* 2020-08-07T15:02:00.378 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-08-07T15:02:00.430 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:00 +0000 (0:00:00.065) 0:05:17.825 ********* 2020-08-07T15:02:00.442 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2020-08-07T15:02:00.488 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:00 +0000 (0:00:00.056) 0:05:17.881 ********* 2020-08-07T15:02:00.502 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2020-08-07T15:02:00.542 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:00 +0000 (0:00:00.051) 0:05:17.934 ********* 2020-08-07T15:02:00.558 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:00.589 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:00 +0000 (0:00:00.050) 0:05:17.984 ********* 2020-08-07T15:02:00.601 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:00.639 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:00 +0000 (0:00:00.050) 0:05:18.034 ********* 2020-08-07T15:02:00.658 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:00.706 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:00 +0000 (0:00:00.062) 0:05:18.097 ********* 2020-08-07T15:02:00.715 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:00.760 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:00 +0000 (0:00:00.056) 0:05:18.153 ********* 2020-08-07T15:02:00.771 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:00.810 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:00 +0000 (0:00:00.051) 0:05:18.205 ********* 2020-08-07T15:02:00.822 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:00.862 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:00 +0000 (0:00:00.052) 0:05:18.257 ********* 2020-08-07T15:02:01.031 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2020-08-07T15:02:01.033 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2020-08-07T15:02:01.091 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:01 +0000 (0:00:00.225) 0:05:18.483 ********* 2020-08-07T15:02:01.111 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2020-08-07T15:02:01.157 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:01 +0000 (0:00:00.068) 0:05:18.552 ********* 2020-08-07T15:02:01.652 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2020-08-07T15:02:01.655 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2020-08-07T15:02:01.707 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:01 +0000 (0:00:00.545) 0:05:19.098 ********* 2020-08-07T15:02:01.759 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2020-08-07T15:02:01.807 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:01 +0000 (0:00:00.102) 0:05:19.200 ********* 2020-08-07T15:02:03.261 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-08-07T15:02:03.264 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.\n", "stderr_lines": ["WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead."], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages\nRequirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages", "Requirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-08-07T15:02:03.298 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:03 +0000 (0:00:01.493) 0:05:20.693 ********* 2020-08-07T15:02:03.313 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.\n", "stderr_lines": ["WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead."], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages\nRequirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages", "Requirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.6/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-08-07T15:02:03.350 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:03 +0000 (0:00:00.051) 0:05:20.745 ********* 2020-08-07T15:02:03.361 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-08-07T15:02:03.363 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:03.400 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:03 +0000 (0:00:00.049) 0:05:20.795 ********* 2020-08-07T15:02:03.410 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:03.448 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:03 +0000 (0:00:00.048) 0:05:20.843 ********* 2020-08-07T15:02:03.467 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ ok: [smithi074.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-08-07T15:02:03.505 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:03 +0000 (0:00:00.057) 0:05:20.900 ********* 2020-08-07T15:02:03.517 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-08-07T15:02:03.558 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:03 +0000 (0:00:00.052) 0:05:20.953 ********* 2020-08-07T15:02:03.725 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-08-07T15:02:03.726 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2020-08-07T15:02:03.762 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:03 +0000 (0:00:00.203) 0:05:21.156 ********* 2020-08-07T15:02:03.787 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2020-08-07T15:02:03.828 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:03 +0000 (0:00:00.064) 0:05:21.221 ********* 2020-08-07T15:02:03.838 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-08-07T15:02:03.838 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:03.897 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:03 +0000 (0:00:00.071) 0:05:21.292 ********* 2020-08-07T15:02:03.909 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:03.950 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:03 +0000 (0:00:00.052) 0:05:21.345 ********* 2020-08-07T15:02:03.962 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2020-08-07T15:02:03.964 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-08-07T15:02:04.007 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:04 +0000 (0:00:00.055) 0:05:21.401 ********* 2020-08-07T15:02:04.018 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-08-07T15:02:04.056 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:04 +0000 (0:00:00.050) 0:05:21.452 ********* 2020-08-07T15:02:04.071 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:04.109 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:04 +0000 (0:00:00.052) 0:05:21.504 ********* 2020-08-07T15:02:04.120 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:04.175 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:04 +0000 (0:00:00.051) 0:05:21.556 ********* 2020-08-07T15:02:04.329 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-08-07T15:02:04.331 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-08-07T15:02:04.368 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:04 +0000 (0:00:00.207) 0:05:21.763 ********* 2020-08-07T15:02:04.393 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-08-07T15:02:04.445 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:04 +0000 (0:00:00.076) 0:05:21.840 ********* 2020-08-07T15:02:04.619 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* changed: [smithi074.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-08-07T15:02:04.683 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:04 +0000 (0:00:00.231) 0:05:22.072 ********* 2020-08-07T15:02:04.720 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-08-07T15:02:04.758 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:04 +0000 (0:00:00.079) 0:05:22.151 ********* 2020-08-07T15:02:04.770 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:04.830 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:04 +0000 (0:00:00.073) 0:05:22.225 ********* 2020-08-07T15:02:04.841 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:04.889 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:04 +0000 (0:00:00.059) 0:05:22.284 ********* 2020-08-07T15:02:04.908 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-08-07T15:02:04.911 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2020-08-07T15:02:04.926 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-08-07T15:02:04.967 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:04 +0000 (0:00:00.078) 0:05:22.362 ********* 2020-08-07T15:02:04.988 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2020-08-07T15:02:04.990 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2020-08-07T15:02:05.000 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-08-07T15:02:05.002 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} skipping: [smithi096.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2020-08-07T15:02:05.047 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:05 +0000 (0:00:00.080) 0:05:22.443 ********* 2020-08-07T15:02:05.071 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** skipping: [smithi074.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:05.074 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi074.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:05.075 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:05.119 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:05 +0000 (0:00:00.071) 0:05:22.514 ********* 2020-08-07T15:02:05.133 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:05.135 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:05.138 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:05.155 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:05.157 INFO:teuthology.task.ansible.out: 2020-08-07T15:02:05.168 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:05.170 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-08-07T15:02:05.170 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:05.219 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:05 +0000 (0:00:00.100) 0:05:22.615 ********* 2020-08-07T15:02:05.272 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:05 +0000 (0:00:00.052) 0:05:22.667 ********* 2020-08-07T15:02:05.290 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:05.343 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:05 +0000 (0:00:00.070) 0:05:22.738 ********* 2020-08-07T15:02:05.356 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-08-07T15:02:05.358 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:05.398 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:05 +0000 (0:00:00.055) 0:05:22.794 ********* 2020-08-07T15:02:05.411 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2020-08-07T15:02:05.413 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:05.466 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:05 +0000 (0:00:00.065) 0:05:22.859 ********* 2020-08-07T15:02:05.479 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-08-07T15:02:05.480 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:05.563 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:05 +0000 (0:00:00.096) 0:05:22.956 ********* 2020-08-07T15:02:05.574 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-08-07T15:02:05.574 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:05.627 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:05 +0000 (0:00:00.065) 0:05:23.021 ********* 2020-08-07T15:02:05.643 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:05.707 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:05 +0000 (0:00:00.077) 0:05:23.099 ********* 2020-08-07T15:02:05.716 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:05.787 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:05 +0000 (0:00:00.083) 0:05:23.182 ********* 2020-08-07T15:02:05.802 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:05.838 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:05 +0000 (0:00:00.051) 0:05:23.233 ********* 2020-08-07T15:02:05.848 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-08-07T15:02:05.850 INFO:teuthology.task.ansible.out:skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:05.898 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:05 +0000 (0:00:00.059) 0:05:23.293 ********* 2020-08-07T15:02:05.909 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-08-07T15:02:05.911 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:05.956 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:05 +0000 (0:00:00.053) 0:05:23.347 ********* 2020-08-07T15:02:05.968 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:06.010 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:06 +0000 (0:00:00.058) 0:05:23.406 ********* 2020-08-07T15:02:06.022 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-08-07T15:02:06.038 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:06.064 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:06 +0000 (0:00:00.053) 0:05:23.459 ********* 2020-08-07T15:02:06.075 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi074.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:06.121 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:06 +0000 (0:00:00.057) 0:05:23.517 ********* 2020-08-07T15:02:06.139 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-08-07T15:02:06.141 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:06.193 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:06 +0000 (0:00:00.071) 0:05:23.588 ********* 2020-08-07T15:02:06.227 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-08-07T15:02:06.229 INFO:teuthology.task.ansible.out:skipping: [smithi096.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-08-07T15:02:06.266 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:06 +0000 (0:00:00.072) 0:05:23.660 ********* 2020-08-07T15:02:06.377 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi074.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.004970", "end": "2020-08-07 15:02:06.337212", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-08-07 15:02:06.332242", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-08-07T15:02:06.527 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.004912", "end": "2020-08-07 15:02:06.480209", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-08-07 15:02:06.475297", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-08-07T15:02:06.584 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.003141", "end": "2020-08-07 15:02:06.544638", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-08-07 15:02:06.541497", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-08-07T15:02:06.748 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.003099", "end": "2020-08-07 15:02:06.704065", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-08-07 15:02:06.700966", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-08-07T15:02:06.786 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.002734", "end": "2020-08-07 15:02:06.744854", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-08-07 15:02:06.742120", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2020-08-07T15:02:06.935 INFO:teuthology.task.ansible.out:ok: [smithi096.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.002872", "end": "2020-08-07 15:02:06.904916", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-08-07 15:02:06.902044", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2020-08-07T15:02:06.994 INFO:teuthology.task.ansible.out:ok: [smithi074.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.003095", "end": "2020-08-07 15:02:06.937077", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-08-07 15:02:06.933982", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-08-07T15:02:07.047 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:07 +0000 (0:00:00.769) 0:05:24.430 ********* 2020-08-07T15:02:07.107 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-08-07T15:02:07.140 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:07 +0000 (0:00:00.104) 0:05:24.535 ********* 2020-08-07T15:02:07.168 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi096.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.003174", "end": "2020-08-07 15:02:07.107703", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-08-07 15:02:07.104529", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-08-07T15:02:07.226 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:07 +0000 (0:00:00.086) 0:05:24.621 ********* 2020-08-07T15:02:07.286 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-08-07T15:02:07.340 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:07 +0000 (0:00:00.099) 0:05:24.720 ********* 2020-08-07T15:02:07.387 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2020-08-07T15:02:07.390 INFO:teuthology.task.ansible.out:changed: [smithi074.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2020-08-07T15:02:07.442 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:07 +0000 (0:00:00.117) 0:05:24.838 ********* 2020-08-07T15:02:07.631 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2020-08-07T15:02:07.886 INFO:teuthology.task.ansible.out: RUNNING HANDLER [common : restart nagios-nrpe-server] ************************** changed: [smithi074.front.sepia.ceph.com] => {"changed": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2020-08-07 14:59:16 UTC", "ActiveEnterTimestampMonotonic": "235572680", "ActiveExitTimestamp": "Fri 2020-08-07 14:59:16 UTC", "ActiveExitTimestampMonotonic": "235549983", "ActiveState": "active", "After": "tmp.mount network.target systemd-journald.socket basic.target sysinit.target local-fs.target systemd-tmpfiles-setup.service nss-lookup.target system.slice -.mount var-run.mount time-sync.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2020-08-07 14:59:16 UTC", "AssertTimestampMonotonic": "235571963", "Before": "shutdown.target multi-user.target getty@tty1.service plymouth-quit.service xdm.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2020-08-07 14:59:16 UTC", "ConditionTimestampMonotonic": "235571963", "Configuratio 2020-08-07T15:02:07.889 INFO:teuthology.task.ansible.out:nDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "16844", "ExecMainStartTimestamp": "Fri 2020-08-07 14:59:16 UTC", "ExecMainStartTimestampMonotonic": "235572651", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "GID": "986", "Group": "nrpe", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Fri 2020-08-07 14:59:16 UTC", "InactiveEnterTimestampMonotonic": "235571346", "InactiveExitTimestamp": "Fri 2020-08-07 14:59:16 UTC", "InactiveExitTimestampMonotonic": "235572680", "InvocationID": "0c291c8fe8414bc08a8adb925e1ac565", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "Kil 2020-08-07T15:02:07.892 INFO:teuthology.task.ansible.out:lSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127424", "LimitNPROCSoft": "127424", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127424", "LimitSIGPENDINGSoft": "127424", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "16844", "MemoryAccounting": "yes", "MemoryCurrent": "1171456", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nrpe/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/var/tmp /run/nrpe" 2020-08-07T15:02:07.894 INFO:teuthology.task.ansible.out:, "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectory": "nrpe", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2020-08-07 14:59:16 UTC", "StateChangeTimestampMonotonic": "235572680", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "990", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2020-08-07 14:59:16 UTC", "WatchdogTimestampMonotonic": "235572679", "WatchdogUSec": "0"}} 2020-08-07T15:02:07.908 INFO:teuthology.task.ansible.out:Friday 07 August 2020 15:02:07 +0000 (0:00:00.465) 0:05:25.303 ********* 2020-08-07T15:02:08.355 INFO:teuthology.task.ansible.out:changed: [smithi096.front.sepia.ceph.com] => {"changed": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Fri 2020-08-07 14:59:18 UTC", "ActiveEnterTimestampMonotonic": "232816188", "ActiveExitTimestamp": "Fri 2020-08-07 14:59:18 UTC", "ActiveExitTimestampMonotonic": "232788691", "ActiveState": "active", "After": "basic.target var-run.mount systemd-journald.socket network.target system.slice systemd-tmpfiles-setup.service nss-lookup.target local-fs.target -.mount sysinit.target time-sync.target tmp.mount", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Fri 2020-08-07 14:59:18 UTC", "AssertTimestampMonotonic": "232815044", "Before": "xdm.service getty@tty1.service plymouth-quit.service shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Fri 2020-08-07 14:59:18 UTC", "ConditionTimestampMonotonic": "232815044", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": 2020-08-07T15:02:08.357 INFO:teuthology.task.ansible.out: "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "16846", "ExecMainStartTimestamp": "Fri 2020-08-07 14:59:18 UTC", "ExecMainStartTimestampMonotonic": "232816152", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "GID": "986", "Group": "nrpe", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Fri 2020-08-07 14:59:18 UTC", "InactiveEnterTimestampMonotonic": "232814381", "InactiveExitTimestamp": "Fri 2020-08-07 14:59:18 UTC", "InactiveExitTimestampMonotonic": "232816188", "InvocationID": "74b671da6bcb4d20b260709af38cb7b8", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infi 2020-08-07T15:02:08.360 INFO:teuthology.task.ansible.out:nity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "16777216", "LimitMEMLOCKSoft": "16777216", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "127425", "LimitNPROCSoft": "127425", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127425", "LimitSIGPENDINGSoft": "127425", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "16846", "MemoryAccounting": "yes", "MemoryCurrent": "892928", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nrpe/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount system.slice sysinit.target", "RequiresMountsFor": "/var/tmp /run/nrpe", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "Restric 2020-08-07T15:02:08.361 INFO:teuthology.task.ansible.out:tRealtime": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectory": "nrpe", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Fri 2020-08-07 14:59:18 UTC", "StateChangeTimestampMonotonic": "232816188", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "26213", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "990", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Fri 2020-08-07 14:59:18 UTC", "WatchdogTimestampMonotonic": "232816186", "WatchdogUSec": "0"}} 2020-08-07T15:02:08.489 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2020-08-07T15:02:08.502 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2020-08-07T15:02:08.507 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched PLAY RECAP ********************************************************************* smithi074.front.sepia.ceph.com : ok=128 changed=41 unreachable=0 failed=0 skipped=162 rescued=0 ignored=4 smithi096.front.sepia.ceph.com : ok=128 changed=41 unreachable=0 failed=0 skipped=162 rescued=0 ignored=4 Friday 07 August 2020 15:02:08 +0000 (0:00:00.597) 0:05:25.901 ********* =============================================================================== users : Create all admin users with sudo access. ----------------------- 41.60s common : nrpe - Ensure SELinux policy is up to date -------------------- 40.12s users : Update authorized_keys using the keys repo --------------------- 38.66s common : nrpe - Remove SELinux policy package -------------------------- 15.60s common : nrpe - Load SELinux policy package ---------------------------- 15.48s testnode : Install packages -------------------------------------------- 11.62s common : Get list of enabled RHSM repos --------------------------------- 9.93s users : Remove revoked users -------------------------------------------- 9.04s testnode : Zap all non-root disks --------------------------------------- 8.02s common : Register with subscription-manager. ---------------------------- 6.49s common : Install nrpe package and dependencies (RHEL/CentOS) ------------ 6.37s 2020-08-07T15:02:08.511 INFO:teuthology.task.ansible.out:testnode : Ensure ceph packages are not present. ------------------------ 5.13s testnode : Ensure ceph dependency packages are not present. ------------- 4.73s common : Disable all rhsm repos ----------------------------------------- 3.26s common : Enable necessary rhsm repos ------------------------------------ 2.84s testnode : Create logical volume(s) ------------------------------------- 2.18s testnode : Blow away lingering OSD data and FSIDs ----------------------- 2.10s users : Update 2020-08-07T15:02:08.514 INFO:teuthology.task.ansible.out:authorized_keys for each user with literal keys ---------- 2.06s testnode : Make sure rpm dependencies are installed --------------------- 2.05s testnode : Rebuild rpmdb ------------------------------------------------ 1.80s 2020-08-07T15:02:09.055 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi074.front.sepia.ceph.com'), Remote(name='ubuntu@smithi096.front.sepia.ceph.com')] 2020-08-07T15:02:09.057 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-08-07T15:02:09.065 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi074.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-07T15:02:09.157 INFO:teuthology.orchestra.run.smithi074:> true 2020-08-07T15:02:09.244 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-08-07T15:02:09.245 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-08-07T15:02:09.382 INFO:teuthology.orchestra.run.smithi096:> true 2020-08-07T15:02:09.471 INFO:teuthology.run_tasks:Running task clock... 2020-08-07T15:02:09.487 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2020-08-07T15:02:09.487 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-08-07T15:02:09.488 INFO:teuthology.orchestra.run.smithi074:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-08-07T15:02:09.531 INFO:teuthology.orchestra.run.smithi074.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2020-08-07T15:02:09.548 INFO:teuthology.orchestra.run.smithi074.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2020-08-07T15:02:09.597 INFO:teuthology.orchestra.run.smithi074.stderr:sudo: ntpd: command not found 2020-08-07T15:02:09.615 INFO:teuthology.orchestra.run.smithi074.stdout:506 Cannot talk to daemon 2020-08-07T15:02:09.629 INFO:teuthology.orchestra.run.smithi074.stderr:Failed to start ntp.service: Unit ntp.service not found. 2020-08-07T15:02:09.646 INFO:teuthology.orchestra.run.smithi074.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2020-08-07T15:02:09.708 INFO:teuthology.orchestra.run.smithi074.stderr:bash: ntpq: command not found 2020-08-07T15:02:09.710 INFO:teuthology.orchestra.run.smithi074.stdout:210 Number of sources = 4 2020-08-07T15:02:09.711 INFO:teuthology.orchestra.run.smithi074.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-08-07T15:02:09.711 INFO:teuthology.orchestra.run.smithi074.stdout:=============================================================================== 2020-08-07T15:02:09.711 INFO:teuthology.orchestra.run.smithi074.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-08-07T15:02:09.711 INFO:teuthology.orchestra.run.smithi074.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-08-07T15:02:09.712 INFO:teuthology.orchestra.run.smithi074.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-08-07T15:02:09.712 INFO:teuthology.orchestra.run.smithi074.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-08-07T15:02:09.712 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-08-07T15:02:09.713 INFO:teuthology.orchestra.run.smithi096:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-08-07T15:02:09.761 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2020-08-07T15:02:09.774 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2020-08-07T15:02:09.820 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: ntpd: command not found 2020-08-07T15:02:09.831 INFO:teuthology.orchestra.run.smithi096.stdout:506 Cannot talk to daemon 2020-08-07T15:02:09.849 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntp.service: Unit ntp.service not found. 2020-08-07T15:02:09.867 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2020-08-07T15:02:09.926 INFO:teuthology.orchestra.run.smithi096.stderr:bash: ntpq: command not found 2020-08-07T15:02:09.930 INFO:teuthology.orchestra.run.smithi096.stdout:210 Number of sources = 4 2020-08-07T15:02:09.930 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2020-08-07T15:02:09.930 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2020-08-07T15:02:09.930 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-08-07T15:02:09.931 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-08-07T15:02:09.931 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-08-07T15:02:09.931 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2020-08-07T15:02:09.931 INFO:teuthology.run_tasks:Running task install... 2020-08-07T15:02:09.956 DEBUG:teuthology.task.install:project ceph 2020-08-07T15:02:09.957 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': 'e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d'}} 2020-08-07T15:02:09.957 DEBUG:teuthology.task.install:config {'sha1': 'e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d'} 2020-08-07T15:02:09.957 INFO:teuthology.task.install:Using flavor: basic 2020-08-07T15:02:09.995 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2020-08-07T15:02:09.996 INFO:teuthology.task.install:extra packages: [] 2020-08-07T15:02:09.996 INFO:teuthology.orchestra.run.smithi074:> sudo lsb_release -is 2020-08-07T15:02:10.053 INFO:teuthology.orchestra.run.smithi074.stdout:RedHatEnterprise 2020-08-07T15:02:10.054 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2020-08-07T15:02:10.055 INFO:teuthology.orchestra.run.smithi096:> sudo lsb_release -is 2020-08-07T15:02:10.061 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'tag': None, 'sha1': 'e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d', 'debuginfo': None, 'flavor': 'basic', 'downgrade_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'exclude_packages': [], 'extras': None, 'wait_for_package': False, 'project': 'ceph', 'packages': {}, 'install_ceph_packages': True, 'repos_only': False} 2020-08-07T15:02:10.062 INFO:teuthology.orchestra.run.smithi074:> sudo lsb_release -is 2020-08-07T15:02:10.121 INFO:teuthology.orchestra.run.smithi096.stdout:RedHatEnterprise 2020-08-07T15:02:10.122 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2020-08-07T15:02:10.123 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'tag': None, 'sha1': 'e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d', 'debuginfo': None, 'flavor': 'basic', 'downgrade_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'exclude_packages': [], 'extras': None, 'wait_for_package': False, 'project': 'ceph', 'packages': {}, 'install_ceph_packages': True, 'repos_only': False} 2020-08-07T15:02:10.124 INFO:teuthology.orchestra.run.smithi096:> sudo lsb_release -is 2020-08-07T15:02:10.135 INFO:teuthology.orchestra.run.smithi074.stdout:RedHatEnterprise 2020-08-07T15:02:10.138 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2020-08-07T15:02:10.140 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d 2020-08-07T15:02:10.203 INFO:teuthology.orchestra.run.smithi096.stdout:RedHatEnterprise 2020-08-07T15:02:10.204 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2020-08-07T15:02:10.204 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d 2020-08-07T15:02:10.371 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-shuzhenyi-testing-2020-08-06-1028/e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d/centos/8/flavors/default/ 2020-08-07T15:02:10.372 INFO:teuthology.task.install.rpm:Package version is 16.0.0-3802.ge19d2f2a442 2020-08-07T15:02:10.397 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-shuzhenyi-testing-2020-08-06-1028/e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d/centos/8/flavors/default/ 2020-08-07T15:02:10.397 INFO:teuthology.task.install.rpm:Package version is 16.0.0-3802.ge19d2f2a442 2020-08-07T15:02:10.595 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-shuzhenyi-testing-2020-08-06-1028/e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-shuzhenyi-testing-2020-08-06-1028/e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-shuzhenyi-testing-2020-08-06-1028/e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2020-08-07T15:02:10.596 INFO:teuthology.orchestra.run.smithi074:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2020-08-07T15:02:10.613 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-shuzhenyi-testing-2020-08-06-1028/e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-shuzhenyi-testing-2020-08-06-1028/e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-shuzhenyi-testing-2020-08-06-1028/e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2020-08-07T15:02:10.615 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2020-08-07T15:02:10.693 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2020-08-07T15:02:10.694 INFO:teuthology.orchestra.run.smithi074:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d/;g' /etc/yum.repos.d/ceph.repo ; fi 2020-08-07T15:02:10.707 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2020-08-07T15:02:10.707 INFO:teuthology.orchestra.run.smithi096:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d/;g' /etc/yum.repos.d/ceph.repo ; fi 2020-08-07T15:02:10.779 INFO:teuthology.orchestra.run.smithi074:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2020-08-07T15:02:10.782 INFO:teuthology.orchestra.run.smithi096:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2020-08-07T15:02:10.876 INFO:teuthology.orchestra.run.smithi074:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2020-08-07T15:02:10.877 INFO:teuthology.orchestra.run.smithi096:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2020-08-07T15:02:10.939 INFO:teuthology.orchestra.run.smithi074.stdout:check_obsoletes = 1 2020-08-07T15:02:10.940 INFO:teuthology.orchestra.run.smithi074:> sudo yum clean all 2020-08-07T15:02:10.946 INFO:teuthology.orchestra.run.smithi096.stdout:check_obsoletes = 1 2020-08-07T15:02:10.947 INFO:teuthology.orchestra.run.smithi096:> sudo yum clean all 2020-08-07T15:02:11.315 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T15:02:11.320 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T15:02:11.624 INFO:teuthology.orchestra.run.smithi074.stdout:52 files removed 2020-08-07T15:02:11.669 INFO:teuthology.orchestra.run.smithi074:> sudo yum -y install ceph-radosgw 2020-08-07T15:02:11.670 INFO:teuthology.orchestra.run.smithi096.stdout:52 files removed 2020-08-07T15:02:11.692 INFO:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-radosgw 2020-08-07T15:02:11.998 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T15:02:12.020 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T15:02:12.661 INFO:teuthology.orchestra.run.smithi074.stdout:ceph packages for x86_64 275 kB/s | 71 kB 00:00 2020-08-07T15:02:12.686 INFO:teuthology.orchestra.run.smithi096.stdout:ceph packages for x86_64 279 kB/s | 71 kB 00:00 2020-08-07T15:02:12.906 INFO:teuthology.orchestra.run.smithi074.stdout:ceph noarch packages 69 kB/s | 15 kB 00:00 2020-08-07T15:02:12.921 INFO:teuthology.orchestra.run.smithi096.stdout:ceph noarch packages 69 kB/s | 15 kB 00:00 2020-08-07T15:02:13.145 INFO:teuthology.orchestra.run.smithi074.stdout:ceph source packages 6.6 kB/s | 1.4 kB 00:00 2020-08-07T15:02:13.157 INFO:teuthology.orchestra.run.smithi096.stdout:ceph source packages 6.7 kB/s | 1.4 kB 00:00 2020-08-07T15:02:13.326 INFO:teuthology.orchestra.run.smithi074.stdout:Copr repo for ceph-el8 owned by ktdreyer 1.5 kB/s | 257 B 00:00 2020-08-07T15:02:13.358 INFO:teuthology.orchestra.run.smithi096.stdout:Copr repo for ceph-el8 owned by ktdreyer 1.4 kB/s | 257 B 00:00 2020-08-07T15:02:13.729 INFO:teuthology.orchestra.run.smithi074.stdout:Extra Packages for Enterprise Linux 20 MB/s | 7.5 MB 00:00 2020-08-07T15:02:13.775 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 19 MB/s | 7.5 MB 00:00 2020-08-07T15:02:15.194 INFO:teuthology.orchestra.run.smithi074.stdout:lab-extras 4.6 MB/s | 20 kB 00:00 2020-08-07T15:02:15.261 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 4.0 MB/s | 20 kB 00:00 2020-08-07T15:02:15.438 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 97 MB/s | 19 MB 00:00 2020-08-07T15:02:15.506 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 96 MB/s | 19 MB 00:00 2020-08-07T15:02:18.318 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 109 MB/s | 20 MB 00:00 2020-08-07T15:02:18.494 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 106 MB/s | 20 MB 00:00 2020-08-07T15:02:23.029 INFO:teuthology.orchestra.run.smithi074.stdout:Dependencies resolved. 2020-08-07T15:02:23.041 INFO:teuthology.orchestra.run.smithi074.stdout:======================================================================================================= 2020-08-07T15:02:23.041 INFO:teuthology.orchestra.run.smithi074.stdout: Package Arch Version Repository Size 2020-08-07T15:02:23.041 INFO:teuthology.orchestra.run.smithi074.stdout:======================================================================================================= 2020-08-07T15:02:23.042 INFO:teuthology.orchestra.run.smithi074.stdout:Installing: 2020-08-07T15:02:23.042 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-radosgw x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 10 M 2020-08-07T15:02:23.042 INFO:teuthology.orchestra.run.smithi074.stdout:Upgrading: 2020-08-07T15:02:23.042 INFO:teuthology.orchestra.run.smithi074.stdout: librados2 x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 3.7 M 2020-08-07T15:02:23.043 INFO:teuthology.orchestra.run.smithi074.stdout: librbd1 x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 2.1 M 2020-08-07T15:02:23.043 INFO:teuthology.orchestra.run.smithi074.stdout:Installing dependencies: 2020-08-07T15:02:23.043 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-base x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 6.0 M 2020-08-07T15:02:23.043 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-common x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 22 M 2020-08-07T15:02:23.043 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-selinux x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 25 k 2020-08-07T15:02:23.044 INFO:teuthology.orchestra.run.smithi074.stdout: fmt x86_64 6.2.1-1.el8 epel 116 k 2020-08-07T15:02:23.045 INFO:teuthology.orchestra.run.smithi074.stdout: gperftools-libs x86_64 2.7-6.el8 epel 315 k 2020-08-07T15:02:23.045 INFO:teuthology.orchestra.run.smithi074.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2020-08-07T15:02:23.045 INFO:teuthology.orchestra.run.smithi074.stdout: libcephfs2 x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 536 k 2020-08-07T15:02:23.045 INFO:teuthology.orchestra.run.smithi074.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2020-08-07T15:02:23.045 INFO:teuthology.orchestra.run.smithi074.stdout: librabbitmq x86_64 0.9.0-1.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2020-08-07T15:02:23.046 INFO:teuthology.orchestra.run.smithi074.stdout: libradosstriper1 x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 485 k 2020-08-07T15:02:23.046 INFO:teuthology.orchestra.run.smithi074.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2020-08-07T15:02:23.046 INFO:teuthology.orchestra.run.smithi074.stdout: librgw2 x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 3.4 M 2020-08-07T15:02:23.046 INFO:teuthology.orchestra.run.smithi074.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2020-08-07T15:02:23.046 INFO:teuthology.orchestra.run.smithi074.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2020-08-07T15:02:23.047 INFO:teuthology.orchestra.run.smithi074.stdout: python3-ceph-argparse x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 40 k 2020-08-07T15:02:23.047 INFO:teuthology.orchestra.run.smithi074.stdout: python3-ceph-common x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 69 k 2020-08-07T15:02:23.047 INFO:teuthology.orchestra.run.smithi074.stdout: python3-cephfs x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 204 k 2020-08-07T15:02:23.047 INFO:teuthology.orchestra.run.smithi074.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2020-08-07T15:02:23.047 INFO:teuthology.orchestra.run.smithi074.stdout: python3-rados x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 364 k 2020-08-07T15:02:23.048 INFO:teuthology.orchestra.run.smithi074.stdout: python3-rbd x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 335 k 2020-08-07T15:02:23.048 INFO:teuthology.orchestra.run.smithi074.stdout: python3-rgw x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 113 k 2020-08-07T15:02:23.048 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:02:23.048 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction Summary 2020-08-07T15:02:23.048 INFO:teuthology.orchestra.run.smithi074.stdout:======================================================================================================= 2020-08-07T15:02:23.049 INFO:teuthology.orchestra.run.smithi074.stdout:Install 22 Packages 2020-08-07T15:02:23.049 INFO:teuthology.orchestra.run.smithi074.stdout:Upgrade 2 Packages 2020-08-07T15:02:23.049 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:02:23.049 INFO:teuthology.orchestra.run.smithi074.stdout:Total download size: 52 M 2020-08-07T15:02:23.049 INFO:teuthology.orchestra.run.smithi074.stdout:Downloading Packages: 2020-08-07T15:02:23.264 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2020-08-07T15:02:23.277 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================================= 2020-08-07T15:02:23.277 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2020-08-07T15:02:23.278 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================================= 2020-08-07T15:02:23.278 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2020-08-07T15:02:23.278 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 10 M 2020-08-07T15:02:23.278 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrading: 2020-08-07T15:02:23.278 INFO:teuthology.orchestra.run.smithi096.stdout: librados2 x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 3.7 M 2020-08-07T15:02:23.279 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1 x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 2.1 M 2020-08-07T15:02:23.279 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2020-08-07T15:02:23.279 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 6.0 M 2020-08-07T15:02:23.279 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 22 M 2020-08-07T15:02:23.279 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 25 k 2020-08-07T15:02:23.280 INFO:teuthology.orchestra.run.smithi096.stdout: fmt x86_64 6.2.1-1.el8 epel 116 k 2020-08-07T15:02:23.280 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs x86_64 2.7-6.el8 epel 315 k 2020-08-07T15:02:23.280 INFO:teuthology.orchestra.run.smithi096.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2020-08-07T15:02:23.280 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2 x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 536 k 2020-08-07T15:02:23.280 INFO:teuthology.orchestra.run.smithi096.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2020-08-07T15:02:23.281 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq x86_64 0.9.0-1.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2020-08-07T15:02:23.281 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1 x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 485 k 2020-08-07T15:02:23.281 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2020-08-07T15:02:23.281 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2 x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 3.4 M 2020-08-07T15:02:23.281 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2020-08-07T15:02:23.282 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2020-08-07T15:02:23.282 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 40 k 2020-08-07T15:02:23.282 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 69 k 2020-08-07T15:02:23.282 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 204 k 2020-08-07T15:02:23.282 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2020-08-07T15:02:23.283 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 364 k 2020-08-07T15:02:23.283 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 335 k 2020-08-07T15:02:23.283 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 113 k 2020-08-07T15:02:23.283 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:02:23.283 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2020-08-07T15:02:23.284 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================================= 2020-08-07T15:02:23.284 INFO:teuthology.orchestra.run.smithi096.stdout:Install 22 Packages 2020-08-07T15:02:23.284 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrade 2 Packages 2020-08-07T15:02:23.284 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:02:23.285 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 52 M 2020-08-07T15:02:23.285 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2020-08-07T15:02:23.870 INFO:teuthology.orchestra.run.smithi074.stdout:(1/24): ceph-base-16.0.0-3802.ge19d2f2a442.el8. 7.2 MB/s | 6.0 MB 00:00 2020-08-07T15:02:23.922 INFO:teuthology.orchestra.run.smithi074.stdout:(2/24): ceph-selinux-16.0.0-3802.ge19d2f2a442.e 481 kB/s | 25 kB 00:00 2020-08-07T15:02:23.995 INFO:teuthology.orchestra.run.smithi074.stdout:(3/24): libcephfs2-16.0.0-3802.ge19d2f2a442.el8 7.1 MB/s | 536 kB 00:00 2020-08-07T15:02:24.070 INFO:teuthology.orchestra.run.smithi074.stdout:(4/24): libradosstriper1-16.0.0-3802.ge19d2f2a4 6.3 MB/s | 485 kB 00:00 2020-08-07T15:02:24.374 INFO:teuthology.orchestra.run.smithi074.stdout:(5/24): ceph-radosgw-16.0.0-3802.ge19d2f2a442.e 7.9 MB/s | 10 MB 00:01 2020-08-07T15:02:24.419 INFO:teuthology.orchestra.run.smithi074.stdout:(6/24): python3-ceph-argparse-16.0.0-3802.ge19d 795 kB/s | 40 kB 00:00 2020-08-07T15:02:24.468 INFO:teuthology.orchestra.run.smithi074.stdout:(7/24): python3-ceph-common-16.0.0-3802.ge19d2f 1.3 MB/s | 69 kB 00:00 2020-08-07T15:02:24.489 INFO:teuthology.orchestra.run.smithi074.stdout:(8/24): librgw2-16.0.0-3802.ge19d2f2a442.el8.x8 8.1 MB/s | 3.4 MB 00:00 2020-08-07T15:02:24.511 INFO:teuthology.orchestra.run.smithi074.stdout:(9/24): python3-cephfs-16.0.0-3802.ge19d2f2a442 4.6 MB/s | 204 kB 00:00 2020-08-07T15:02:24.538 INFO:teuthology.orchestra.run.smithi074.stdout:(10/24): python3-rados-16.0.0-3802.ge19d2f2a442 7.2 MB/s | 364 kB 00:00 2020-08-07T15:02:24.572 INFO:teuthology.orchestra.run.smithi074.stdout:(11/24): python3-rbd-16.0.0-3802.ge19d2f2a442.e 6.0 MB/s | 335 kB 00:00 2020-08-07T15:02:24.584 INFO:teuthology.orchestra.run.smithi074.stdout:(12/24): python3-rgw-16.0.0-3802.ge19d2f2a442.e 2.5 MB/s | 113 kB 00:00 2020-08-07T15:02:24.646 INFO:teuthology.orchestra.run.smithi074.stdout:(13/24): fmt-6.2.1-1.el8.x86_64.rpm 1.5 MB/s | 116 kB 00:00 2020-08-07T15:02:24.674 INFO:teuthology.orchestra.run.smithi074.stdout:(14/24): gperftools-libs-2.7-6.el8.x86_64.rpm 3.3 MB/s | 315 kB 00:00 2020-08-07T15:02:24.675 INFO:teuthology.orchestra.run.smithi074.stdout:(15/24): leveldb-1.22-1.el8.x86_64.rpm 5.5 MB/s | 181 kB 00:00 2020-08-07T15:02:24.695 INFO:teuthology.orchestra.run.smithi074.stdout:(16/24): liboath-2.6.2-3.el8.x86_64.rpm 3.6 MB/s | 59 kB 00:00 2020-08-07T15:02:24.696 INFO:teuthology.orchestra.run.smithi074.stdout:(17/24): libunwind-1.3.1-3.el8.x86_64.rpm 4.6 MB/s | 75 kB 00:00 2020-08-07T15:02:24.731 INFO:teuthology.orchestra.run.smithi074.stdout:(18/24): python3-prettytable-0.7.2-14.el8.noarc 1.2 MB/s | 44 kB 00:00 2020-08-07T15:02:24.737 INFO:teuthology.orchestra.run.smithi074.stdout:(19/24): librdkafka-0.11.4-1.el8.x86_64.rpm 7.7 MB/s | 353 kB 00:00 2020-08-07T15:02:24.749 INFO:teuthology.orchestra.run.smithi074.stdout:(20/24): lttng-ust-2.8.1-11.el8.x86_64.rpm 12 MB/s | 259 kB 00:00 2020-08-07T15:02:24.768 INFO:teuthology.orchestra.run.smithi074.stdout:(21/24): librabbitmq-0.9.0-1.el8.x86_64.rpm 2.4 MB/s | 47 kB 00:00 2020-08-07T15:02:25.038 INFO:teuthology.orchestra.run.smithi074.stdout:(22/24): librbd1-16.0.0-3802.ge19d2f2a442.el8.x 7.7 MB/s | 2.1 MB 00:00 2020-08-07T15:02:25.230 INFO:teuthology.orchestra.run.smithi074.stdout:(23/24): librados2-16.0.0-3802.ge19d2f2a442.el8 7.8 MB/s | 3.7 MB 00:00 2020-08-07T15:02:25.360 INFO:teuthology.orchestra.run.smithi074.stdout:(24/24): ceph-common-16.0.0-3802.ge19d2f2a442.e 9.7 MB/s | 22 MB 00:02 2020-08-07T15:02:25.362 INFO:teuthology.orchestra.run.smithi074.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:02:25.363 INFO:teuthology.orchestra.run.smithi074.stdout:Total 22 MB/s | 52 MB 00:02 2020-08-07T15:02:25.367 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction check 2020-08-07T15:02:25.407 INFO:teuthology.orchestra.run.smithi096.stdout:(1/24): ceph-base-16.0.0-3802.ge19d2f2a442.el8. 2.8 MB/s | 6.0 MB 00:02 2020-08-07T15:02:25.409 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction check succeeded. 2020-08-07T15:02:25.409 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction test 2020-08-07T15:02:25.432 INFO:teuthology.orchestra.run.smithi096.stdout:(2/24): ceph-selinux-16.0.0-3802.ge19d2f2a442.e 976 kB/s | 25 kB 00:00 2020-08-07T15:02:25.483 INFO:teuthology.orchestra.run.smithi096.stdout:(3/24): libcephfs2-16.0.0-3802.ge19d2f2a442.el8 10 MB/s | 536 kB 00:00 2020-08-07T15:02:25.533 INFO:teuthology.orchestra.run.smithi096.stdout:(4/24): libradosstriper1-16.0.0-3802.ge19d2f2a4 9.5 MB/s | 485 kB 00:00 2020-08-07T15:02:25.736 INFO:teuthology.orchestra.run.smithi096.stdout:(5/24): librgw2-16.0.0-3802.ge19d2f2a442.el8.x8 17 MB/s | 3.4 MB 00:00 2020-08-07T15:02:25.758 INFO:teuthology.orchestra.run.smithi096.stdout:(6/24): python3-ceph-argparse-16.0.0-3802.ge19d 1.6 MB/s | 40 kB 00:00 2020-08-07T15:02:25.787 INFO:teuthology.orchestra.run.smithi096.stdout:(7/24): python3-ceph-common-16.0.0-3802.ge19d2f 2.7 MB/s | 69 kB 00:00 2020-08-07T15:02:25.815 INFO:teuthology.orchestra.run.smithi096.stdout:(8/24): python3-cephfs-16.0.0-3802.ge19d2f2a442 7.6 MB/s | 204 kB 00:00 2020-08-07T15:02:25.846 INFO:teuthology.orchestra.run.smithi096.stdout:(9/24): python3-rados-16.0.0-3802.ge19d2f2a442. 11 MB/s | 364 kB 00:00 2020-08-07T15:02:25.871 INFO:teuthology.orchestra.run.smithi096.stdout:(10/24): python3-rbd-16.0.0-3802.ge19d2f2a442.e 12 MB/s | 335 kB 00:00 2020-08-07T15:02:25.903 INFO:teuthology.orchestra.run.smithi096.stdout:(11/24): python3-rgw-16.0.0-3802.ge19d2f2a442.e 4.4 MB/s | 113 kB 00:00 2020-08-07T15:02:25.982 INFO:teuthology.orchestra.run.smithi096.stdout:(12/24): fmt-6.2.1-1.el8.x86_64.rpm 1.4 MB/s | 116 kB 00:00 2020-08-07T15:02:26.012 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction test succeeded. 2020-08-07T15:02:26.013 INFO:teuthology.orchestra.run.smithi096.stdout:(13/24): gperftools-libs-2.7-6.el8.x86_64.rpm 9.4 MB/s | 315 kB 00:00 2020-08-07T15:02:26.013 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction 2020-08-07T15:02:26.041 INFO:teuthology.orchestra.run.smithi096.stdout:(14/24): leveldb-1.22-1.el8.x86_64.rpm 10 MB/s | 181 kB 00:00 2020-08-07T15:02:26.055 INFO:teuthology.orchestra.run.smithi096.stdout:(15/24): liboath-2.6.2-3.el8.x86_64.rpm 3.6 MB/s | 59 kB 00:00 2020-08-07T15:02:26.067 INFO:teuthology.orchestra.run.smithi096.stdout:(16/24): libunwind-1.3.1-3.el8.x86_64.rpm 4.5 MB/s | 75 kB 00:00 2020-08-07T15:02:26.091 INFO:teuthology.orchestra.run.smithi096.stdout:(17/24): python3-prettytable-0.7.2-14.el8.noarc 1.4 MB/s | 44 kB 00:00 2020-08-07T15:02:26.113 INFO:teuthology.orchestra.run.smithi096.stdout:(18/24): librdkafka-0.11.4-1.el8.x86_64.rpm 15 MB/s | 353 kB 00:00 2020-08-07T15:02:26.134 INFO:teuthology.orchestra.run.smithi096.stdout:(19/24): lttng-ust-2.8.1-11.el8.x86_64.rpm 12 MB/s | 259 kB 00:00 2020-08-07T15:02:26.158 INFO:teuthology.orchestra.run.smithi096.stdout:(20/24): librabbitmq-0.9.0-1.el8.x86_64.rpm 2.3 MB/s | 47 kB 00:00 2020-08-07T15:02:26.370 INFO:teuthology.orchestra.run.smithi096.stdout:(21/24): librados2-16.0.0-3802.ge19d2f2a442.el8 18 MB/s | 3.7 MB 00:00 2020-08-07T15:02:26.492 INFO:teuthology.orchestra.run.smithi096.stdout:(22/24): librbd1-16.0.0-3802.ge19d2f2a442.el8.x 18 MB/s | 2.1 MB 00:00 2020-08-07T15:02:27.280 INFO:teuthology.orchestra.run.smithi096.stdout:(23/24): ceph-radosgw-16.0.0-3802.ge19d2f2a442. 2.6 MB/s | 10 MB 00:03 2020-08-07T15:02:27.554 INFO:teuthology.orchestra.run.smithi074.stdout: Preparing : 1/1 2020-08-07T15:02:27.784 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : fmt-6.2.1-1.el8.x86_64 1/26 2020-08-07T15:02:27.938 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/26 2020-08-07T15:02:28.341 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/26 2020-08-07T15:02:28.517 INFO:teuthology.orchestra.run.smithi074.stdout: Upgrading : librados2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 3/26 2020-08-07T15:02:28.597 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: librados2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 3/26 2020-08-07T15:02:28.749 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-rados-2:16.0.0-3802.ge19d2f2a442.el8.x86_6 4/26 2020-08-07T15:02:28.912 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/26 2020-08-07T15:02:29.103 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : libcephfs2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 6/26 2020-08-07T15:02:29.200 INFO:teuthology.orchestra.run.smithi096.stdout:(24/24): ceph-common-16.0.0-3802.ge19d2f2a442.e 3.8 MB/s | 22 MB 00:05 2020-08-07T15:02:29.202 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:02:29.204 INFO:teuthology.orchestra.run.smithi096.stdout:Total 8.7 MB/s | 52 MB 00:05 2020-08-07T15:02:29.209 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2020-08-07T15:02:29.258 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2020-08-07T15:02:29.258 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2020-08-07T15:02:29.365 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: libcephfs2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 6/26 2020-08-07T15:02:29.559 INFO:teuthology.orchestra.run.smithi074.stdout: Upgrading : librbd1-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 7/26 2020-08-07T15:02:29.629 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: librbd1-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 7/26 2020-08-07T15:02:29.656 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2020-08-07T15:02:29.659 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2020-08-07T15:02:29.818 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : librabbitmq-0.9.0-1.el8.x86_64 8/26 2020-08-07T15:02:29.976 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 9/26 2020-08-07T15:02:30.351 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 9/26 2020-08-07T15:02:30.463 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : librgw2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 10/26 2020-08-07T15:02:30.535 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: librgw2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 10/26 2020-08-07T15:02:30.731 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : leveldb-1.22-1.el8.x86_64 11/26 2020-08-07T15:02:30.843 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2020-08-07T15:02:30.863 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-ceph-argparse-2:16.0.0-3802.ge19d2f2a442.e 12/26 2020-08-07T15:02:31.015 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-cephfs-2:16.0.0-3802.ge19d2f2a442.el8.x86_ 13/26 2020-08-07T15:02:31.072 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : fmt-6.2.1-1.el8.x86_64 1/26 2020-08-07T15:02:31.214 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-rgw-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 14/26 2020-08-07T15:02:31.266 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/26 2020-08-07T15:02:31.391 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-rbd-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 15/26 2020-08-07T15:02:31.534 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : libradosstriper1-2:16.0.0-3802.ge19d2f2a442.el8.x8 16/26 2020-08-07T15:02:31.606 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: libradosstriper1-2:16.0.0-3802.ge19d2f2a442.el8.x8 16/26 2020-08-07T15:02:31.690 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/26 2020-08-07T15:02:31.754 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 17/26 2020-08-07T15:02:31.865 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librados2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 3/26 2020-08-07T15:02:31.932 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 18/26 2020-08-07T15:02:31.944 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 3/26 2020-08-07T15:02:32.112 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : gperftools-libs-2.7-6.el8.x86_64 19/26 2020-08-07T15:02:32.146 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rados-2:16.0.0-3802.ge19d2f2a442.el8.x86_6 4/26 2020-08-07T15:02:32.242 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-ceph-common-2:16.0.0-3802.ge19d2f2a442.el8 20/26 2020-08-07T15:02:32.347 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/26 2020-08-07T15:02:32.506 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 6/26 2020-08-07T15:02:32.774 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephfs2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 6/26 2020-08-07T15:02:32.974 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librbd1-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 7/26 2020-08-07T15:02:33.060 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 7/26 2020-08-07T15:02:33.224 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librabbitmq-0.9.0-1.el8.x86_64 8/26 2020-08-07T15:02:33.382 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 9/26 2020-08-07T15:02:33.735 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 9/26 2020-08-07T15:02:33.865 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librgw2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 10/26 2020-08-07T15:02:33.966 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librgw2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 10/26 2020-08-07T15:02:34.117 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : leveldb-1.22-1.el8.x86_64 11/26 2020-08-07T15:02:34.252 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-argparse-2:16.0.0-3802.ge19d2f2a442.e 12/26 2020-08-07T15:02:34.445 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cephfs-2:16.0.0-3802.ge19d2f2a442.el8.x86_ 13/26 2020-08-07T15:02:34.621 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rgw-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 14/26 2020-08-07T15:02:34.843 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rbd-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 15/26 2020-08-07T15:02:35.010 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libradosstriper1-2:16.0.0-3802.ge19d2f2a442.el8.x8 16/26 2020-08-07T15:02:35.062 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libradosstriper1-2:16.0.0-3802.ge19d2f2a442.el8.x8 16/26 2020-08-07T15:02:35.250 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 17/26 2020-08-07T15:02:35.460 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 18/26 2020-08-07T15:02:35.537 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: ceph-common-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 21/26 2020-08-07T15:02:35.682 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : gperftools-libs-2.7-6.el8.x86_64 19/26 2020-08-07T15:02:35.766 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : ceph-common-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 21/26 2020-08-07T15:02:35.843 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-common-2:16.0.0-3802.ge19d2f2a442.el8 20/26 2020-08-07T15:02:36.490 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: ceph-common-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 21/26 2020-08-07T15:02:36.754 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : ceph-base-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 22/26 2020-08-07T15:02:36.825 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: ceph-base-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 22/26 2020-08-07T15:02:36.925 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : ceph-selinux-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 23/26 2020-08-07T15:02:39.191 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 21/26 2020-08-07T15:02:39.414 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-common-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 21/26 2020-08-07T15:02:40.163 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 21/26 2020-08-07T15:02:40.462 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-base-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 22/26 2020-08-07T15:02:40.591 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-base-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 22/26 2020-08-07T15:02:40.713 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-selinux-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 23/26 2020-08-07T15:02:53.476 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: ceph-selinux-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 23/26 2020-08-07T15:02:53.478 INFO:teuthology.orchestra.run.smithi074.stdout:skipping the directory /sys 2020-08-07T15:02:53.478 INFO:teuthology.orchestra.run.smithi074.stdout:skipping the directory /proc 2020-08-07T15:02:53.478 INFO:teuthology.orchestra.run.smithi074.stdout:skipping the directory /dev 2020-08-07T15:02:53.478 INFO:teuthology.orchestra.run.smithi074.stdout:skipping the directory /run 2020-08-07T15:02:53.478 INFO:teuthology.orchestra.run.smithi074.stdout:skipping the directory /mnt 2020-08-07T15:02:53.479 INFO:teuthology.orchestra.run.smithi074.stdout:skipping the directory /var/tmp 2020-08-07T15:02:53.479 INFO:teuthology.orchestra.run.smithi074.stdout:skipping the directory /home 2020-08-07T15:02:53.479 INFO:teuthology.orchestra.run.smithi074.stdout:skipping the directory /tmp 2020-08-07T15:02:53.480 INFO:teuthology.orchestra.run.smithi074.stdout:skipping the directory /dev 2020-08-07T15:02:53.481 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:02:54.614 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : ceph-radosgw-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 24/26 2020-08-07T15:02:54.692 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: ceph-radosgw-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 24/26 2020-08-07T15:02:54.699 INFO:teuthology.orchestra.run.smithi074.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 25/26 2020-08-07T15:02:54.875 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 25/26 2020-08-07T15:02:54.875 INFO:teuthology.orchestra.run.smithi074.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 26/26 2020-08-07T15:02:55.555 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 26/26 2020-08-07T15:02:55.555 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : ceph-base-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/26 2020-08-07T15:02:55.555 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : ceph-common-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2/26 2020-08-07T15:02:55.555 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : ceph-radosgw-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 3/26 2020-08-07T15:02:55.556 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : ceph-selinux-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 4/26 2020-08-07T15:02:55.556 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : libcephfs2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 5/26 2020-08-07T15:02:55.556 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : libradosstriper1-2:16.0.0-3802.ge19d2f2a442.el8.x8 6/26 2020-08-07T15:02:55.556 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : librgw2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 7/26 2020-08-07T15:02:55.556 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-ceph-argparse-2:16.0.0-3802.ge19d2f2a442.e 8/26 2020-08-07T15:02:55.557 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-ceph-common-2:16.0.0-3802.ge19d2f2a442.el8 9/26 2020-08-07T15:02:55.557 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-cephfs-2:16.0.0-3802.ge19d2f2a442.el8.x86_ 10/26 2020-08-07T15:02:55.557 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-rados-2:16.0.0-3802.ge19d2f2a442.el8.x86_6 11/26 2020-08-07T15:02:55.557 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-rbd-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 12/26 2020-08-07T15:02:55.557 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-rgw-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 13/26 2020-08-07T15:02:55.558 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 14/26 2020-08-07T15:02:55.558 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : gperftools-libs-2.7-6.el8.x86_64 15/26 2020-08-07T15:02:55.558 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : leveldb-1.22-1.el8.x86_64 16/26 2020-08-07T15:02:55.558 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 17/26 2020-08-07T15:02:55.559 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 18/26 2020-08-07T15:02:55.559 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 19/26 2020-08-07T15:02:55.559 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 20/26 2020-08-07T15:02:55.559 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 21/26 2020-08-07T15:02:55.559 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : librabbitmq-0.9.0-1.el8.x86_64 22/26 2020-08-07T15:02:55.560 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : librados2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 23/26 2020-08-07T15:02:55.560 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 24/26 2020-08-07T15:02:55.560 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : librbd1-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 25/26 2020-08-07T15:02:56.353 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 26/26 2020-08-07T15:02:56.354 INFO:teuthology.orchestra.run.smithi074.stdout:Installed products updated. 2020-08-07T15:02:57.380 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-selinux-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 23/26 2020-08-07T15:02:57.381 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /sys 2020-08-07T15:02:57.381 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /proc 2020-08-07T15:02:57.381 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /dev 2020-08-07T15:02:57.381 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /run 2020-08-07T15:02:57.381 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /mnt 2020-08-07T15:02:57.382 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /var/tmp 2020-08-07T15:02:57.382 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /home 2020-08-07T15:02:57.382 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /tmp 2020-08-07T15:02:57.382 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /dev 2020-08-07T15:02:57.382 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:02:58.206 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:02:58.207 INFO:teuthology.orchestra.run.smithi074.stdout:Upgraded: 2020-08-07T15:02:58.207 INFO:teuthology.orchestra.run.smithi074.stdout: librados2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:02:58.207 INFO:teuthology.orchestra.run.smithi074.stdout: librbd1-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:02:58.207 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:02:58.208 INFO:teuthology.orchestra.run.smithi074.stdout:Installed: 2020-08-07T15:02:58.208 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-base-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:02:58.208 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-common-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:02:58.208 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-radosgw-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:02:58.208 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-selinux-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:02:58.209 INFO:teuthology.orchestra.run.smithi074.stdout: fmt-6.2.1-1.el8.x86_64 2020-08-07T15:02:58.209 INFO:teuthology.orchestra.run.smithi074.stdout: gperftools-libs-2.7-6.el8.x86_64 2020-08-07T15:02:58.209 INFO:teuthology.orchestra.run.smithi074.stdout: leveldb-1.22-1.el8.x86_64 2020-08-07T15:02:58.209 INFO:teuthology.orchestra.run.smithi074.stdout: libcephfs2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:02:58.217 INFO:teuthology.orchestra.run.smithi074.stdout: liboath-2.6.2-3.el8.x86_64 2020-08-07T15:02:58.217 INFO:teuthology.orchestra.run.smithi074.stdout: librabbitmq-0.9.0-1.el8.x86_64 2020-08-07T15:02:58.218 INFO:teuthology.orchestra.run.smithi074.stdout: libradosstriper1-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:02:58.218 INFO:teuthology.orchestra.run.smithi074.stdout: librdkafka-0.11.4-1.el8.x86_64 2020-08-07T15:02:58.218 INFO:teuthology.orchestra.run.smithi074.stdout: librgw2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:02:58.218 INFO:teuthology.orchestra.run.smithi074.stdout: libunwind-1.3.1-3.el8.x86_64 2020-08-07T15:02:58.218 INFO:teuthology.orchestra.run.smithi074.stdout: lttng-ust-2.8.1-11.el8.x86_64 2020-08-07T15:02:58.219 INFO:teuthology.orchestra.run.smithi074.stdout: python3-ceph-argparse-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:02:58.219 INFO:teuthology.orchestra.run.smithi074.stdout: python3-ceph-common-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:02:58.219 INFO:teuthology.orchestra.run.smithi074.stdout: python3-cephfs-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:02:58.219 INFO:teuthology.orchestra.run.smithi074.stdout: python3-prettytable-0.7.2-14.el8.noarch 2020-08-07T15:02:58.219 INFO:teuthology.orchestra.run.smithi074.stdout: python3-rados-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:02:58.220 INFO:teuthology.orchestra.run.smithi074.stdout: python3-rbd-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:02:58.220 INFO:teuthology.orchestra.run.smithi074.stdout: python3-rgw-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:02:58.220 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:02:58.220 INFO:teuthology.orchestra.run.smithi074.stdout:Complete! 2020-08-07T15:02:58.379 INFO:teuthology.orchestra.run.smithi074:> sudo yum -y install ceph-test 2020-08-07T15:02:58.551 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-radosgw-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 24/26 2020-08-07T15:02:58.612 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-radosgw-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 24/26 2020-08-07T15:02:58.612 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 25/26 2020-08-07T15:02:58.737 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T15:02:58.853 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 25/26 2020-08-07T15:02:58.854 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 26/26 2020-08-07T15:02:59.215 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 100 kB/s | 2.8 kB 00:00 2020-08-07T15:02:59.311 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 88 kB/s | 2.4 kB 00:00 2020-08-07T15:02:59.562 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 26/26 2020-08-07T15:02:59.563 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-base-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/26 2020-08-07T15:02:59.563 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-common-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2/26 2020-08-07T15:02:59.563 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-radosgw-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 3/26 2020-08-07T15:02:59.564 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-selinux-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 4/26 2020-08-07T15:02:59.564 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 5/26 2020-08-07T15:02:59.564 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libradosstriper1-2:16.0.0-3802.ge19d2f2a442.el8.x8 6/26 2020-08-07T15:02:59.564 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librgw2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 7/26 2020-08-07T15:02:59.565 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-argparse-2:16.0.0-3802.ge19d2f2a442.e 8/26 2020-08-07T15:02:59.565 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-common-2:16.0.0-3802.ge19d2f2a442.el8 9/26 2020-08-07T15:02:59.565 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cephfs-2:16.0.0-3802.ge19d2f2a442.el8.x86_ 10/26 2020-08-07T15:02:59.568 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rados-2:16.0.0-3802.ge19d2f2a442.el8.x86_6 11/26 2020-08-07T15:02:59.568 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rbd-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 12/26 2020-08-07T15:02:59.569 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rgw-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 13/26 2020-08-07T15:02:59.569 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 14/26 2020-08-07T15:02:59.569 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : gperftools-libs-2.7-6.el8.x86_64 15/26 2020-08-07T15:02:59.569 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : leveldb-1.22-1.el8.x86_64 16/26 2020-08-07T15:02:59.570 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 17/26 2020-08-07T15:02:59.570 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 18/26 2020-08-07T15:02:59.571 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 19/26 2020-08-07T15:02:59.571 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 20/26 2020-08-07T15:02:59.571 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 21/26 2020-08-07T15:02:59.572 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librabbitmq-0.9.0-1.el8.x86_64 22/26 2020-08-07T15:02:59.572 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 23/26 2020-08-07T15:02:59.572 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 24/26 2020-08-07T15:02:59.572 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 25/26 2020-08-07T15:03:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 26/26 2020-08-07T15:03:00.191 INFO:teuthology.orchestra.run.smithi096.stdout:Installed products updated. 2020-08-07T15:03:00.329 INFO:teuthology.orchestra.run.smithi074.stdout:Dependencies resolved. 2020-08-07T15:03:00.329 INFO:teuthology.orchestra.run.smithi074.stdout:============================================================================================ 2020-08-07T15:03:00.330 INFO:teuthology.orchestra.run.smithi074.stdout: Package Arch Version Repository Size 2020-08-07T15:03:00.330 INFO:teuthology.orchestra.run.smithi074.stdout:============================================================================================ 2020-08-07T15:03:00.330 INFO:teuthology.orchestra.run.smithi074.stdout:Installing: 2020-08-07T15:03:00.331 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-test x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 46 M 2020-08-07T15:03:00.332 INFO:teuthology.orchestra.run.smithi074.stdout:Installing dependencies: 2020-08-07T15:03:00.332 INFO:teuthology.orchestra.run.smithi074.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2020-08-07T15:03:00.332 INFO:teuthology.orchestra.run.smithi074.stdout: libxslt x86_64 1.1.32-4.el8 rhel-8-for-x86_64-baseos-rpms 249 k 2020-08-07T15:03:00.334 INFO:teuthology.orchestra.run.smithi074.stdout: oniguruma x86_64 6.8.2-1.el8 rhel-8-for-x86_64-appstream-rpms 188 k 2020-08-07T15:03:00.335 INFO:teuthology.orchestra.run.smithi074.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2020-08-07T15:03:00.336 INFO:teuthology.orchestra.run.smithi074.stdout: xmlstarlet x86_64 1.6.1-11.el8 epel 69 k 2020-08-07T15:03:00.336 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:03:00.336 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction Summary 2020-08-07T15:03:00.337 INFO:teuthology.orchestra.run.smithi074.stdout:============================================================================================ 2020-08-07T15:03:00.337 INFO:teuthology.orchestra.run.smithi074.stdout:Install 6 Packages 2020-08-07T15:03:00.338 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:03:00.338 INFO:teuthology.orchestra.run.smithi074.stdout:Total download size: 47 M 2020-08-07T15:03:00.338 INFO:teuthology.orchestra.run.smithi074.stdout:Installed size: 199 M 2020-08-07T15:03:00.339 INFO:teuthology.orchestra.run.smithi074.stdout:Downloading Packages: 2020-08-07T15:03:00.368 INFO:teuthology.orchestra.run.smithi074.stdout:(1/6): jq-1.5-12.el8.x86_64.rpm 5.2 MB/s | 161 kB 00:00 2020-08-07T15:03:00.377 INFO:teuthology.orchestra.run.smithi074.stdout:(2/6): oniguruma-6.8.2-1.el8.x86_64.rpm 9.2 MB/s | 188 kB 00:00 2020-08-07T15:03:00.400 INFO:teuthology.orchestra.run.smithi074.stdout:(3/6): socat-1.7.3.3-2.el8.x86_64.rpm 13 MB/s | 302 kB 00:00 2020-08-07T15:03:00.407 INFO:teuthology.orchestra.run.smithi074.stdout:(4/6): xmlstarlet-1.6.1-11.el8.x86_64.rpm 854 kB/s | 69 kB 00:00 2020-08-07T15:03:00.420 INFO:teuthology.orchestra.run.smithi074.stdout:(5/6): libxslt-1.1.32-4.el8.x86_64.rpm 12 MB/s | 249 kB 00:00 2020-08-07T15:03:02.111 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:03:02.112 INFO:teuthology.orchestra.run.smithi096.stdout:Upgraded: 2020-08-07T15:03:02.112 INFO:teuthology.orchestra.run.smithi096.stdout: librados2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:02.112 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:02.112 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:03:02.113 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2020-08-07T15:03:02.113 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:02.113 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:02.113 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:02.114 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:02.114 INFO:teuthology.orchestra.run.smithi096.stdout: fmt-6.2.1-1.el8.x86_64 2020-08-07T15:03:02.115 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs-2.7-6.el8.x86_64 2020-08-07T15:03:02.115 INFO:teuthology.orchestra.run.smithi096.stdout: leveldb-1.22-1.el8.x86_64 2020-08-07T15:03:02.115 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:02.115 INFO:teuthology.orchestra.run.smithi096.stdout: liboath-2.6.2-3.el8.x86_64 2020-08-07T15:03:02.115 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq-0.9.0-1.el8.x86_64 2020-08-07T15:03:02.115 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:02.116 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka-0.11.4-1.el8.x86_64 2020-08-07T15:03:02.116 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:02.116 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind-1.3.1-3.el8.x86_64 2020-08-07T15:03:02.116 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust-2.8.1-11.el8.x86_64 2020-08-07T15:03:02.116 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:02.117 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:02.117 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:02.118 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable-0.7.2-14.el8.noarch 2020-08-07T15:03:02.118 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:02.118 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:02.119 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:02.119 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:03:02.119 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2020-08-07T15:03:02.372 INFO:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-test 2020-08-07T15:03:02.718 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T15:03:02.974 INFO:teuthology.orchestra.run.smithi074.stdout:(6/6): ceph-test-16.0.0-3802.ge19d2f2a442.el8.x 17 MB/s | 46 MB 00:02 2020-08-07T15:03:02.974 INFO:teuthology.orchestra.run.smithi074.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:03:02.974 INFO:teuthology.orchestra.run.smithi074.stdout:Total 18 MB/s | 47 MB 00:02 2020-08-07T15:03:02.978 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction check 2020-08-07T15:03:03.001 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction check succeeded. 2020-08-07T15:03:03.002 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction test 2020-08-07T15:03:03.213 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 99 kB/s | 2.8 kB 00:00 2020-08-07T15:03:03.309 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 83 kB/s | 2.4 kB 00:00 2020-08-07T15:03:03.430 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction test succeeded. 2020-08-07T15:03:03.430 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction 2020-08-07T15:03:04.210 INFO:teuthology.orchestra.run.smithi074.stdout: Preparing : 1/1 2020-08-07T15:03:04.366 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2020-08-07T15:03:04.367 INFO:teuthology.orchestra.run.smithi096.stdout:============================================================================================ 2020-08-07T15:03:04.367 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2020-08-07T15:03:04.367 INFO:teuthology.orchestra.run.smithi096.stdout:============================================================================================ 2020-08-07T15:03:04.368 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2020-08-07T15:03:04.368 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 46 M 2020-08-07T15:03:04.368 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2020-08-07T15:03:04.368 INFO:teuthology.orchestra.run.smithi096.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2020-08-07T15:03:04.368 INFO:teuthology.orchestra.run.smithi096.stdout: libxslt x86_64 1.1.32-4.el8 rhel-8-for-x86_64-baseos-rpms 249 k 2020-08-07T15:03:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: oniguruma x86_64 6.8.2-1.el8 rhel-8-for-x86_64-appstream-rpms 188 k 2020-08-07T15:03:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2020-08-07T15:03:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet x86_64 1.6.1-11.el8 epel 69 k 2020-08-07T15:03:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:03:04.369 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2020-08-07T15:03:04.370 INFO:teuthology.orchestra.run.smithi096.stdout:============================================================================================ 2020-08-07T15:03:04.370 INFO:teuthology.orchestra.run.smithi096.stdout:Install 6 Packages 2020-08-07T15:03:04.370 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:03:04.370 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 47 M 2020-08-07T15:03:04.371 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 199 M 2020-08-07T15:03:04.371 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2020-08-07T15:03:04.390 INFO:teuthology.orchestra.run.smithi096.stdout:(1/6): jq-1.5-12.el8.x86_64.rpm 5.2 MB/s | 161 kB 00:00 2020-08-07T15:03:04.414 INFO:teuthology.orchestra.run.smithi096.stdout:(2/6): oniguruma-6.8.2-1.el8.x86_64.rpm 9.0 MB/s | 188 kB 00:00 2020-08-07T15:03:04.433 INFO:teuthology.orchestra.run.smithi096.stdout:(3/6): socat-1.7.3.3-2.el8.x86_64.rpm 13 MB/s | 302 kB 00:00 2020-08-07T15:03:04.440 INFO:teuthology.orchestra.run.smithi096.stdout:(4/6): xmlstarlet-1.6.1-11.el8.x86_64.rpm 859 kB/s | 69 kB 00:00 2020-08-07T15:03:04.456 INFO:teuthology.orchestra.run.smithi096.stdout:(5/6): libxslt-1.1.32-4.el8.x86_64.rpm 12 MB/s | 249 kB 00:00 2020-08-07T15:03:04.457 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : libxslt-1.1.32-4.el8.x86_64 1/6 2020-08-07T15:03:04.610 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : xmlstarlet-1.6.1-11.el8.x86_64 2/6 2020-08-07T15:03:04.801 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 3/6 2020-08-07T15:03:04.972 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : oniguruma-6.8.2-1.el8.x86_64 4/6 2020-08-07T15:03:05.036 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: oniguruma-6.8.2-1.el8.x86_64 4/6 2020-08-07T15:03:06.535 INFO:teuthology.orchestra.run.smithi096.stdout:(6/6): ceph-test-16.0.0-3802.ge19d2f2a442.el8.x 21 MB/s | 46 MB 00:02 2020-08-07T15:03:06.536 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:03:06.537 INFO:teuthology.orchestra.run.smithi096.stdout:Total 22 MB/s | 47 MB 00:02 2020-08-07T15:03:06.542 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2020-08-07T15:03:06.570 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2020-08-07T15:03:06.571 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2020-08-07T15:03:06.889 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2020-08-07T15:03:06.890 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2020-08-07T15:03:07.388 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2020-08-07T15:03:07.636 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libxslt-1.1.32-4.el8.x86_64 1/6 2020-08-07T15:03:07.816 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xmlstarlet-1.6.1-11.el8.x86_64 2/6 2020-08-07T15:03:08.008 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 3/6 2020-08-07T15:03:08.248 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : oniguruma-6.8.2-1.el8.x86_64 4/6 2020-08-07T15:03:08.325 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: oniguruma-6.8.2-1.el8.x86_64 4/6 2020-08-07T15:03:09.623 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : jq-1.5-12.el8.x86_64 5/6 2020-08-07T15:03:10.202 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : ceph-test-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 6/6 2020-08-07T15:03:10.419 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: ceph-test-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 6/6 2020-08-07T15:03:10.419 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : ceph-test-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/6 2020-08-07T15:03:10.420 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : xmlstarlet-1.6.1-11.el8.x86_64 2/6 2020-08-07T15:03:10.420 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : jq-1.5-12.el8.x86_64 3/6 2020-08-07T15:03:10.420 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : oniguruma-6.8.2-1.el8.x86_64 4/6 2020-08-07T15:03:10.421 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 5/6 2020-08-07T15:03:10.711 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : libxslt-1.1.32-4.el8.x86_64 6/6 2020-08-07T15:03:10.711 INFO:teuthology.orchestra.run.smithi074.stdout:Installed products updated. 2020-08-07T15:03:12.629 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:03:12.630 INFO:teuthology.orchestra.run.smithi074.stdout:Installed: 2020-08-07T15:03:12.630 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-test-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:12.631 INFO:teuthology.orchestra.run.smithi074.stdout: jq-1.5-12.el8.x86_64 2020-08-07T15:03:12.631 INFO:teuthology.orchestra.run.smithi074.stdout: libxslt-1.1.32-4.el8.x86_64 2020-08-07T15:03:12.631 INFO:teuthology.orchestra.run.smithi074.stdout: oniguruma-6.8.2-1.el8.x86_64 2020-08-07T15:03:12.632 INFO:teuthology.orchestra.run.smithi074.stdout: socat-1.7.3.3-2.el8.x86_64 2020-08-07T15:03:12.632 INFO:teuthology.orchestra.run.smithi074.stdout: xmlstarlet-1.6.1-11.el8.x86_64 2020-08-07T15:03:12.633 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:03:12.633 INFO:teuthology.orchestra.run.smithi074.stdout:Complete! 2020-08-07T15:03:12.781 INFO:teuthology.orchestra.run.smithi074:> sudo yum -y install ceph 2020-08-07T15:03:13.115 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T15:03:13.613 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 99 kB/s | 2.8 kB 00:00 2020-08-07T15:03:13.709 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 84 kB/s | 2.4 kB 00:00 2020-08-07T15:03:13.803 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : jq-1.5-12.el8.x86_64 5/6 2020-08-07T15:03:14.112 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-test-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 6/6 2020-08-07T15:03:14.310 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-test-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 6/6 2020-08-07T15:03:14.310 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-test-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/6 2020-08-07T15:03:14.311 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xmlstarlet-1.6.1-11.el8.x86_64 2/6 2020-08-07T15:03:14.311 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : jq-1.5-12.el8.x86_64 3/6 2020-08-07T15:03:14.311 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : oniguruma-6.8.2-1.el8.x86_64 4/6 2020-08-07T15:03:14.312 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 5/6 2020-08-07T15:03:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libxslt-1.1.32-4.el8.x86_64 6/6 2020-08-07T15:03:14.582 INFO:teuthology.orchestra.run.smithi096.stdout:Installed products updated. 2020-08-07T15:03:14.739 INFO:teuthology.orchestra.run.smithi074.stdout:Dependencies resolved. 2020-08-07T15:03:14.742 INFO:teuthology.orchestra.run.smithi074.stdout:========================================================================================================== 2020-08-07T15:03:14.743 INFO:teuthology.orchestra.run.smithi074.stdout: Package Arch Version Repository Size 2020-08-07T15:03:14.743 INFO:teuthology.orchestra.run.smithi074.stdout:========================================================================================================== 2020-08-07T15:03:14.744 INFO:teuthology.orchestra.run.smithi074.stdout:Installing: 2020-08-07T15:03:14.744 INFO:teuthology.orchestra.run.smithi074.stdout: ceph x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 6.5 k 2020-08-07T15:03:14.745 INFO:teuthology.orchestra.run.smithi074.stdout:Installing dependencies: 2020-08-07T15:03:14.745 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-mds x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 2.1 M 2020-08-07T15:03:14.745 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-mgr x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 1.5 M 2020-08-07T15:03:14.746 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-mgr-modules-core noarch 2:16.0.0-3802.ge19d2f2a442.el8 ceph-noarch 176 k 2020-08-07T15:03:14.747 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-mon x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 4.9 M 2020-08-07T15:03:14.747 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-osd x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 18 M 2020-08-07T15:03:14.748 INFO:teuthology.orchestra.run.smithi074.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2020-08-07T15:03:14.748 INFO:teuthology.orchestra.run.smithi074.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2020-08-07T15:03:14.749 INFO:teuthology.orchestra.run.smithi074.stdout: python3-cheroot noarch 8.2.1-1.el8 epel 145 k 2020-08-07T15:03:14.749 INFO:teuthology.orchestra.run.smithi074.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2020-08-07T15:03:14.749 INFO:teuthology.orchestra.run.smithi074.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2020-08-07T15:03:14.750 INFO:teuthology.orchestra.run.smithi074.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2020-08-07T15:03:14.750 INFO:teuthology.orchestra.run.smithi074.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2020-08-07T15:03:14.752 INFO:teuthology.orchestra.run.smithi074.stdout: python3-lxml x86_64 4.2.3-1.el8 rhel-8-for-x86_64-appstream-rpms 1.5 M 2020-08-07T15:03:14.754 INFO:teuthology.orchestra.run.smithi074.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2020-08-07T15:03:14.756 INFO:teuthology.orchestra.run.smithi074.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2020-08-07T15:03:14.756 INFO:teuthology.orchestra.run.smithi074.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2020-08-07T15:03:14.756 INFO:teuthology.orchestra.run.smithi074.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2020-08-07T15:03:14.758 INFO:teuthology.orchestra.run.smithi074.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2020-08-07T15:03:14.758 INFO:teuthology.orchestra.run.smithi074.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2020-08-07T15:03:14.759 INFO:teuthology.orchestra.run.smithi074.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2020-08-07T15:03:14.759 INFO:teuthology.orchestra.run.smithi074.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2020-08-07T15:03:14.760 INFO:teuthology.orchestra.run.smithi074.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2020-08-07T15:03:14.760 INFO:teuthology.orchestra.run.smithi074.stdout: python3-trustme noarch 0.5.2-4.el8 epel 26 k 2020-08-07T15:03:14.760 INFO:teuthology.orchestra.run.smithi074.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2020-08-07T15:03:14.761 INFO:teuthology.orchestra.run.smithi074.stdout: python3-webencodings noarch 0.5.1-6.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2020-08-07T15:03:14.761 INFO:teuthology.orchestra.run.smithi074.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2020-08-07T15:03:14.761 INFO:teuthology.orchestra.run.smithi074.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2020-08-07T15:03:14.762 INFO:teuthology.orchestra.run.smithi074.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2020-08-07T15:03:14.762 INFO:teuthology.orchestra.run.smithi074.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2020-08-07T15:03:14.763 INFO:teuthology.orchestra.run.smithi074.stdout:Installing weak dependencies: 2020-08-07T15:03:14.763 INFO:teuthology.orchestra.run.smithi074.stdout: python3-cssselect noarch 0.9.2-10.el8 epel 40 k 2020-08-07T15:03:14.763 INFO:teuthology.orchestra.run.smithi074.stdout: python3-html5lib noarch 1:0.999999999-6.el8 rhel-8-for-x86_64-appstream-rpms 214 k 2020-08-07T15:03:14.764 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:03:14.764 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction Summary 2020-08-07T15:03:14.764 INFO:teuthology.orchestra.run.smithi074.stdout:========================================================================================================== 2020-08-07T15:03:14.765 INFO:teuthology.orchestra.run.smithi074.stdout:Install 32 Packages 2020-08-07T15:03:14.765 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:03:14.766 INFO:teuthology.orchestra.run.smithi074.stdout:Total download size: 31 M 2020-08-07T15:03:14.767 INFO:teuthology.orchestra.run.smithi074.stdout:Installed size: 108 M 2020-08-07T15:03:14.767 INFO:teuthology.orchestra.run.smithi074.stdout:Downloading Packages: 2020-08-07T15:03:14.854 INFO:teuthology.orchestra.run.smithi074.stdout:(1/32): ceph-16.0.0-3802.ge19d2f2a442.el8.x86_6 62 kB/s | 6.5 kB 00:00 2020-08-07T15:03:15.116 INFO:teuthology.orchestra.run.smithi074.stdout:(2/32): ceph-mds-16.0.0-3802.ge19d2f2a442.el8.x 5.8 MB/s | 2.1 MB 00:00 2020-08-07T15:03:15.124 INFO:teuthology.orchestra.run.smithi074.stdout:(3/32): ceph-mgr-16.0.0-3802.ge19d2f2a442.el8.x 4.0 MB/s | 1.5 MB 00:00 2020-08-07T15:03:15.189 INFO:teuthology.orchestra.run.smithi074.stdout:(4/32): ceph-mgr-modules-core-16.0.0-3802.ge19d 2.7 MB/s | 176 kB 00:00 2020-08-07T15:03:15.277 INFO:teuthology.orchestra.run.smithi074.stdout:(5/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 577 kB/s | 44 kB 00:00 2020-08-07T15:03:15.376 INFO:teuthology.orchestra.run.smithi074.stdout:(6/32): ceph-mon-16.0.0-3802.ge19d2f2a442.el8.x 10 MB/s | 4.9 MB 00:00 2020-08-07T15:03:15.376 INFO:teuthology.orchestra.run.smithi074.stdout:(7/32): python3-beautifulsoup4-4.6.3-2.el8.1.no 2.9 MB/s | 185 kB 00:00 2020-08-07T15:03:15.376 INFO:teuthology.orchestra.run.smithi074.stdout:(8/32): python3-cherrypy-18.4.0-1.el8.noarch.rp 9.7 MB/s | 384 kB 00:00 2020-08-07T15:03:15.394 INFO:teuthology.orchestra.run.smithi074.stdout:(9/32): python3-cssselect-0.9.2-10.el8.noarch.r 1.7 MB/s | 40 kB 00:00 2020-08-07T15:03:15.414 INFO:teuthology.orchestra.run.smithi074.stdout:(10/32): python3-jaraco-6.2-6.el8.noarch.rpm 581 kB/s | 11 kB 00:00 2020-08-07T15:03:15.422 INFO:teuthology.orchestra.run.smithi074.stdout:(11/32): python3-cheroot-8.2.1-1.el8.noarch.rpm 1.5 MB/s | 145 kB 00:00 2020-08-07T15:03:15.430 INFO:teuthology.orchestra.run.smithi074.stdout:(12/32): python3-jaraco-functools-2.0-4.el8.noa 914 kB/s | 18 kB 00:00 2020-08-07T15:03:15.443 INFO:teuthology.orchestra.run.smithi074.stdout:(13/32): python3-logutils-0.3.5-11.el8.noarch.r 2.5 MB/s | 49 kB 00:00 2020-08-07T15:03:15.446 INFO:teuthology.orchestra.run.smithi074.stdout:(14/32): python3-more-itertools-7.2.0-3.el8.noa 3.0 MB/s | 59 kB 00:00 2020-08-07T15:03:15.466 INFO:teuthology.orchestra.run.smithi074.stdout:(15/32): python3-portend-2.6-1.el8.noarch.rpm 829 kB/s | 16 kB 00:00 2020-08-07T15:03:15.476 INFO:teuthology.orchestra.run.smithi074.stdout:(16/32): python3-pecan-1.3.2-9.el8.noarch.rpm 7.2 MB/s | 283 kB 00:00 2020-08-07T15:03:15.484 INFO:teuthology.orchestra.run.smithi074.stdout:(17/32): python3-simplegeneric-0.8.1-17.el8.noa 987 kB/s | 19 kB 00:00 2020-08-07T15:03:15.495 INFO:teuthology.orchestra.run.smithi074.stdout:(18/32): python3-singledispatch-3.4.0.3-18.el8. 1.2 MB/s | 24 kB 00:00 2020-08-07T15:03:15.503 INFO:teuthology.orchestra.run.smithi074.stdout:(19/32): python3-tempora-1.14.1-5.el8.noarch.rp 1.5 MB/s | 29 kB 00:00 2020-08-07T15:03:15.515 INFO:teuthology.orchestra.run.smithi074.stdout:(20/32): python3-trustme-0.5.2-4.el8.noarch.rpm 1.4 MB/s | 26 kB 00:00 2020-08-07T15:03:15.524 INFO:teuthology.orchestra.run.smithi074.stdout:(21/32): python3-waitress-1.2.1-2.el8.1.noarch. 11 MB/s | 241 kB 00:00 2020-08-07T15:03:15.536 INFO:teuthology.orchestra.run.smithi074.stdout:(22/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 12 MB/s | 251 kB 00:00 2020-08-07T15:03:15.544 INFO:teuthology.orchestra.run.smithi074.stdout:(23/32): python3-webtest-2.0.33-1.el8.noarch.rp 4.3 MB/s | 85 kB 00:00 2020-08-07T15:03:15.562 INFO:teuthology.orchestra.run.smithi074.stdout:(24/32): python3-zc-lockfile-2.0-2.el8.noarch.r 987 kB/s | 23 kB 00:00 2020-08-07T15:03:15.572 INFO:teuthology.orchestra.run.smithi074.stdout:(25/32): python3-webencodings-0.5.1-6.el8.noarc 926 kB/s | 27 kB 00:00 2020-08-07T15:03:15.601 INFO:teuthology.orchestra.run.smithi074.stdout:(26/32): python3-werkzeug-0.12.2-4.el8.noarch.r 15 MB/s | 457 kB 00:00 2020-08-07T15:03:15.606 INFO:teuthology.orchestra.run.smithi074.stdout:(27/32): python3-lxml-4.2.3-1.el8.x86_64.rpm 32 MB/s | 1.5 MB 00:00 2020-08-07T15:03:15.634 INFO:teuthology.orchestra.run.smithi074.stdout:(28/32): python3-mako-1.0.6-13.el8.noarch.rpm 4.8 MB/s | 157 kB 00:00 2020-08-07T15:03:15.642 INFO:teuthology.orchestra.run.smithi074.stdout:(29/32): python3-html5lib-0.999999999-6.el8.noa 5.9 MB/s | 214 kB 00:00 2020-08-07T15:03:15.659 INFO:teuthology.orchestra.run.smithi074.stdout:(30/32): python3-pytz-2017.2-9.el8.noarch.rpm 2.1 MB/s | 54 kB 00:00 2020-08-07T15:03:15.664 INFO:teuthology.orchestra.run.smithi074.stdout:(31/32): python3-markupsafe-0.23-19.el8.x86_64. 1.7 MB/s | 39 kB 00:00 2020-08-07T15:03:15.964 INFO:teuthology.orchestra.run.smithi074.stdout:(32/32): ceph-osd-16.0.0-3802.ge19d2f2a442.el8. 21 MB/s | 18 MB 00:00 2020-08-07T15:03:15.966 INFO:teuthology.orchestra.run.smithi074.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:03:15.967 INFO:teuthology.orchestra.run.smithi074.stdout:Total 25 MB/s | 31 MB 00:01 2020-08-07T15:03:15.976 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction check 2020-08-07T15:03:16.012 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction check succeeded. 2020-08-07T15:03:16.012 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction test 2020-08-07T15:03:16.325 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction test succeeded. 2020-08-07T15:03:16.328 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction 2020-08-07T15:03:16.702 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:03:16.703 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2020-08-07T15:03:16.703 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:16.703 INFO:teuthology.orchestra.run.smithi096.stdout: jq-1.5-12.el8.x86_64 2020-08-07T15:03:16.703 INFO:teuthology.orchestra.run.smithi096.stdout: libxslt-1.1.32-4.el8.x86_64 2020-08-07T15:03:16.703 INFO:teuthology.orchestra.run.smithi096.stdout: oniguruma-6.8.2-1.el8.x86_64 2020-08-07T15:03:16.704 INFO:teuthology.orchestra.run.smithi096.stdout: socat-1.7.3.3-2.el8.x86_64 2020-08-07T15:03:16.704 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet-1.6.1-11.el8.x86_64 2020-08-07T15:03:16.704 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:03:16.704 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2020-08-07T15:03:16.908 INFO:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph 2020-08-07T15:03:17.245 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T15:03:17.478 INFO:teuthology.orchestra.run.smithi074.stdout: Preparing : 1/1 2020-08-07T15:03:17.735 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2020-08-07T15:03:17.766 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 99 kB/s | 2.8 kB 00:00 2020-08-07T15:03:17.862 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 85 kB/s | 2.4 kB 00:00 2020-08-07T15:03:17.898 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2020-08-07T15:03:18.157 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 3/32 2020-08-07T15:03:18.377 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 4/32 2020-08-07T15:03:18.631 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 5/32 2020-08-07T15:03:18.789 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 6/32 2020-08-07T15:03:18.894 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2020-08-07T15:03:18.898 INFO:teuthology.orchestra.run.smithi096.stdout:========================================================================================================== 2020-08-07T15:03:18.898 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2020-08-07T15:03:18.899 INFO:teuthology.orchestra.run.smithi096.stdout:========================================================================================================== 2020-08-07T15:03:18.899 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2020-08-07T15:03:18.899 INFO:teuthology.orchestra.run.smithi096.stdout: ceph x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 6.5 k 2020-08-07T15:03:18.900 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2020-08-07T15:03:18.900 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 2.1 M 2020-08-07T15:03:18.900 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 1.5 M 2020-08-07T15:03:18.901 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core noarch 2:16.0.0-3802.ge19d2f2a442.el8 ceph-noarch 176 k 2020-08-07T15:03:18.901 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 4.9 M 2020-08-07T15:03:18.901 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 18 M 2020-08-07T15:03:18.902 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2020-08-07T15:03:18.902 INFO:teuthology.orchestra.run.smithi096.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2020-08-07T15:03:18.902 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot noarch 8.2.1-1.el8 epel 145 k 2020-08-07T15:03:18.903 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2020-08-07T15:03:18.903 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2020-08-07T15:03:18.903 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2020-08-07T15:03:18.904 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2020-08-07T15:03:18.904 INFO:teuthology.orchestra.run.smithi096.stdout: python3-lxml x86_64 4.2.3-1.el8 rhel-8-for-x86_64-appstream-rpms 1.5 M 2020-08-07T15:03:18.904 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2020-08-07T15:03:18.905 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2020-08-07T15:03:18.905 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2020-08-07T15:03:18.905 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2020-08-07T15:03:18.906 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2020-08-07T15:03:18.906 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2020-08-07T15:03:18.906 INFO:teuthology.orchestra.run.smithi096.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2020-08-07T15:03:18.907 INFO:teuthology.orchestra.run.smithi096.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2020-08-07T15:03:18.907 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2020-08-07T15:03:18.908 INFO:teuthology.orchestra.run.smithi096.stdout: python3-trustme noarch 0.5.2-4.el8 epel 26 k 2020-08-07T15:03:18.908 INFO:teuthology.orchestra.run.smithi096.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2020-08-07T15:03:18.908 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webencodings noarch 0.5.1-6.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2020-08-07T15:03:18.909 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2020-08-07T15:03:18.909 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2020-08-07T15:03:18.909 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2020-08-07T15:03:18.910 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2020-08-07T15:03:18.910 INFO:teuthology.orchestra.run.smithi096.stdout:Installing weak dependencies: 2020-08-07T15:03:18.910 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cssselect noarch 0.9.2-10.el8 epel 40 k 2020-08-07T15:03:18.911 INFO:teuthology.orchestra.run.smithi096.stdout: python3-html5lib noarch 1:0.999999999-6.el8 rhel-8-for-x86_64-appstream-rpms 214 k 2020-08-07T15:03:18.911 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:03:18.912 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2020-08-07T15:03:18.912 INFO:teuthology.orchestra.run.smithi096.stdout:========================================================================================================== 2020-08-07T15:03:18.912 INFO:teuthology.orchestra.run.smithi096.stdout:Install 32 Packages 2020-08-07T15:03:18.913 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:03:18.914 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 31 M 2020-08-07T15:03:18.915 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 108 M 2020-08-07T15:03:18.915 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2020-08-07T15:03:19.009 INFO:teuthology.orchestra.run.smithi096.stdout:(1/32): ceph-16.0.0-3802.ge19d2f2a442.el8.x86_6 61 kB/s | 6.5 kB 00:00 2020-08-07T15:03:19.088 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-webencodings-0.5.1-6.el8.noarch 7/32 2020-08-07T15:03:19.261 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-html5lib-1:0.999999999-6.el8.noarch 8/32 2020-08-07T15:03:19.492 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 9/32 2020-08-07T15:03:19.602 INFO:teuthology.orchestra.run.smithi096.stdout:(2/32): ceph-mgr-16.0.0-3802.ge19d2f2a442.el8.x 2.2 MB/s | 1.5 MB 00:00 2020-08-07T15:03:19.736 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 10/32 2020-08-07T15:03:19.830 INFO:teuthology.orchestra.run.smithi096.stdout:(3/32): ceph-mds-16.0.0-3802.ge19d2f2a442.el8.x 2.3 MB/s | 2.1 MB 00:00 2020-08-07T15:03:19.894 INFO:teuthology.orchestra.run.smithi096.stdout:(4/32): ceph-mgr-modules-core-16.0.0-3802.ge19d 2.6 MB/s | 176 kB 00:00 2020-08-07T15:03:19.918 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-trustme-0.5.2-4.el8.noarch 11/32 2020-08-07T15:03:19.969 INFO:teuthology.orchestra.run.smithi096.stdout:(5/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 579 kB/s | 44 kB 00:00 2020-08-07T15:03:20.026 INFO:teuthology.orchestra.run.smithi096.stdout:(6/32): python3-beautifulsoup4-4.6.3-2.el8.1.no 3.2 MB/s | 185 kB 00:00 2020-08-07T15:03:20.045 INFO:teuthology.orchestra.run.smithi096.stdout:(7/32): python3-cheroot-8.2.1-1.el8.noarch.rpm 7.0 MB/s | 145 kB 00:00 2020-08-07T15:03:20.083 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 12/32 2020-08-07T15:03:20.084 INFO:teuthology.orchestra.run.smithi096.stdout:(8/32): python3-cherrypy-18.4.0-1.el8.noarch.rp 9.7 MB/s | 384 kB 00:00 2020-08-07T15:03:20.103 INFO:teuthology.orchestra.run.smithi096.stdout:(9/32): python3-cssselect-0.9.2-10.el8.noarch.r 2.1 MB/s | 40 kB 00:00 2020-08-07T15:03:20.126 INFO:teuthology.orchestra.run.smithi096.stdout:(10/32): python3-jaraco-6.2-6.el8.noarch.rpm 589 kB/s | 11 kB 00:00 2020-08-07T15:03:20.142 INFO:teuthology.orchestra.run.smithi096.stdout:(11/32): python3-jaraco-functools-2.0-4.el8.noa 934 kB/s | 18 kB 00:00 2020-08-07T15:03:20.163 INFO:teuthology.orchestra.run.smithi096.stdout:(12/32): python3-logutils-0.3.5-11.el8.noarch.r 2.5 MB/s | 49 kB 00:00 2020-08-07T15:03:20.182 INFO:teuthology.orchestra.run.smithi096.stdout:(13/32): python3-more-itertools-7.2.0-3.el8.noa 2.9 MB/s | 59 kB 00:00 2020-08-07T15:03:20.201 INFO:teuthology.orchestra.run.smithi096.stdout:(14/32): python3-pecan-1.3.2-9.el8.noarch.rpm 13 MB/s | 283 kB 00:00 2020-08-07T15:03:20.221 INFO:teuthology.orchestra.run.smithi096.stdout:(15/32): python3-portend-2.6-1.el8.noarch.rpm 828 kB/s | 16 kB 00:00 2020-08-07T15:03:20.241 INFO:teuthology.orchestra.run.smithi096.stdout:(16/32): python3-simplegeneric-0.8.1-17.el8.noa 986 kB/s | 19 kB 00:00 2020-08-07T15:03:20.260 INFO:teuthology.orchestra.run.smithi096.stdout:(17/32): python3-singledispatch-3.4.0.3-18.el8. 1.2 MB/s | 24 kB 00:00 2020-08-07T15:03:20.279 INFO:teuthology.orchestra.run.smithi096.stdout:(18/32): python3-tempora-1.14.1-5.el8.noarch.rp 1.5 MB/s | 29 kB 00:00 2020-08-07T15:03:20.280 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 13/32 2020-08-07T15:03:20.298 INFO:teuthology.orchestra.run.smithi096.stdout:(19/32): python3-trustme-0.5.2-4.el8.noarch.rpm 1.3 MB/s | 26 kB 00:00 2020-08-07T15:03:20.327 INFO:teuthology.orchestra.run.smithi096.stdout:(20/32): python3-waitress-1.2.1-2.el8.1.noarch. 11 MB/s | 241 kB 00:00 2020-08-07T15:03:20.340 INFO:teuthology.orchestra.run.smithi096.stdout:(21/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 12 MB/s | 251 kB 00:00 2020-08-07T15:03:20.361 INFO:teuthology.orchestra.run.smithi096.stdout:(22/32): python3-webtest-2.0.33-1.el8.noarch.rp 4.3 MB/s | 85 kB 00:00 2020-08-07T15:03:20.379 INFO:teuthology.orchestra.run.smithi096.stdout:(23/32): python3-zc-lockfile-2.0-2.el8.noarch.r 1.2 MB/s | 23 kB 00:00 2020-08-07T15:03:20.409 INFO:teuthology.orchestra.run.smithi096.stdout:(24/32): python3-webencodings-0.5.1-6.el8.noarc 907 kB/s | 27 kB 00:00 2020-08-07T15:03:20.446 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 14/32 2020-08-07T15:03:20.447 INFO:teuthology.orchestra.run.smithi096.stdout:(25/32): python3-lxml-4.2.3-1.el8.x86_64.rpm 51 MB/s | 1.5 MB 00:00 2020-08-07T15:03:20.463 INFO:teuthology.orchestra.run.smithi096.stdout:(26/32): python3-werkzeug-0.12.2-4.el8.noarch.r 18 MB/s | 457 kB 00:00 2020-08-07T15:03:20.486 INFO:teuthology.orchestra.run.smithi096.stdout:(27/32): python3-mako-1.0.6-13.el8.noarch.rpm 6.8 MB/s | 157 kB 00:00 2020-08-07T15:03:20.507 INFO:teuthology.orchestra.run.smithi096.stdout:(28/32): python3-html5lib-0.999999999-6.el8.noa 10 MB/s | 214 kB 00:00 2020-08-07T15:03:20.529 INFO:teuthology.orchestra.run.smithi096.stdout:(29/32): python3-pytz-2017.2-9.el8.noarch.rpm 2.7 MB/s | 54 kB 00:00 2020-08-07T15:03:20.551 INFO:teuthology.orchestra.run.smithi096.stdout:(30/32): python3-markupsafe-0.23-19.el8.x86_64. 1.8 MB/s | 39 kB 00:00 2020-08-07T15:03:20.619 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 15/32 2020-08-07T15:03:20.820 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 16/32 2020-08-07T15:03:21.058 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-cheroot-8.2.1-1.el8.noarch 17/32 2020-08-07T15:03:21.219 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 18/32 2020-08-07T15:03:21.322 INFO:teuthology.orchestra.run.smithi096.stdout:(31/32): ceph-mon-16.0.0-3802.ge19d2f2a442.el8. 2.1 MB/s | 4.9 MB 00:02 2020-08-07T15:03:21.503 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-portend-2.6-1.el8.noarch 19/32 2020-08-07T15:03:21.734 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 20/32 2020-08-07T15:03:22.121 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-cssselect-0.9.2-10.el8.noarch 21/32 2020-08-07T15:03:22.390 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-lxml-4.2.3-1.el8.x86_64 22/32 2020-08-07T15:03:22.604 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 23/32 2020-08-07T15:03:22.908 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 24/32 2020-08-07T15:03:23.188 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 25/32 2020-08-07T15:03:23.216 INFO:teuthology.orchestra.run.smithi096.stdout:(32/32): ceph-osd-16.0.0-3802.ge19d2f2a442.el8. 4.9 MB/s | 18 MB 00:03 2020-08-07T15:03:23.216 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:03:23.217 INFO:teuthology.orchestra.run.smithi096.stdout:Total 7.2 MB/s | 31 MB 00:04 2020-08-07T15:03:23.223 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2020-08-07T15:03:23.282 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2020-08-07T15:03:23.283 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2020-08-07T15:03:23.490 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 26/32 2020-08-07T15:03:23.579 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2020-08-07T15:03:23.581 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2020-08-07T15:03:23.886 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : ceph-mgr-modules-core-2:16.0.0-3802.ge19d2f2a442.e 27/32 2020-08-07T15:03:24.093 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : ceph-mgr-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 28/32 2020-08-07T15:03:24.763 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2020-08-07T15:03:25.030 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2020-08-07T15:03:25.241 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2020-08-07T15:03:25.510 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 3/32 2020-08-07T15:03:25.620 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: ceph-mgr-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 28/32 2020-08-07T15:03:25.802 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : ceph-osd-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 29/32 2020-08-07T15:03:25.917 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 4/32 2020-08-07T15:03:26.227 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: ceph-osd-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 29/32 2020-08-07T15:03:26.350 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 5/32 2020-08-07T15:03:26.377 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : ceph-mon-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 30/32 2020-08-07T15:03:26.548 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 6/32 2020-08-07T15:03:26.579 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: ceph-mon-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 30/32 2020-08-07T15:03:26.735 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : ceph-mds-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 31/32 2020-08-07T15:03:26.754 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: ceph-mds-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 31/32 2020-08-07T15:03:26.838 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-webencodings-0.5.1-6.el8.noarch 7/32 2020-08-07T15:03:26.859 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : ceph-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 32/32 2020-08-07T15:03:27.019 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-html5lib-1:0.999999999-6.el8.noarch 8/32 2020-08-07T15:03:27.165 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: ceph-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 32/32 2020-08-07T15:03:27.182 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : ceph-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/32 2020-08-07T15:03:27.182 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : ceph-mds-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2/32 2020-08-07T15:03:27.182 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : ceph-mgr-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 3/32 2020-08-07T15:03:27.182 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : ceph-mon-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 4/32 2020-08-07T15:03:27.183 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : ceph-osd-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 5/32 2020-08-07T15:03:27.183 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : ceph-mgr-modules-core-2:16.0.0-3802.ge19d2f2a442.e 6/32 2020-08-07T15:03:27.183 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/32 2020-08-07T15:03:27.183 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/32 2020-08-07T15:03:27.183 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-cheroot-8.2.1-1.el8.noarch 9/32 2020-08-07T15:03:27.184 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/32 2020-08-07T15:03:27.184 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-cssselect-0.9.2-10.el8.noarch 11/32 2020-08-07T15:03:27.184 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 12/32 2020-08-07T15:03:27.184 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 13/32 2020-08-07T15:03:27.184 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 14/32 2020-08-07T15:03:27.185 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 15/32 2020-08-07T15:03:27.185 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 16/32 2020-08-07T15:03:27.185 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-portend-2.6-1.el8.noarch 17/32 2020-08-07T15:03:27.185 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 18/32 2020-08-07T15:03:27.185 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 19/32 2020-08-07T15:03:27.185 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 20/32 2020-08-07T15:03:27.186 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-trustme-0.5.2-4.el8.noarch 21/32 2020-08-07T15:03:27.186 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 22/32 2020-08-07T15:03:27.186 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 23/32 2020-08-07T15:03:27.186 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 24/32 2020-08-07T15:03:27.186 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 25/32 2020-08-07T15:03:27.187 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-webencodings-0.5.1-6.el8.noarch 26/32 2020-08-07T15:03:27.187 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-lxml-4.2.3-1.el8.x86_64 27/32 2020-08-07T15:03:27.187 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 28/32 2020-08-07T15:03:27.187 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 29/32 2020-08-07T15:03:27.188 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-html5lib-1:0.999999999-6.el8.noarch 30/32 2020-08-07T15:03:27.188 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 31/32 2020-08-07T15:03:27.259 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 9/32 2020-08-07T15:03:27.482 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 10/32 2020-08-07T15:03:27.694 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-trustme-0.5.2-4.el8.noarch 11/32 2020-08-07T15:03:27.904 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 12/32 2020-08-07T15:03:27.942 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 32/32 2020-08-07T15:03:27.943 INFO:teuthology.orchestra.run.smithi074.stdout:Installed products updated. 2020-08-07T15:03:28.122 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 13/32 2020-08-07T15:03:28.348 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 14/32 2020-08-07T15:03:28.574 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 15/32 2020-08-07T15:03:28.848 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 16/32 2020-08-07T15:03:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cheroot-8.2.1-1.el8.noarch 17/32 2020-08-07T15:03:29.343 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 18/32 2020-08-07T15:03:29.654 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-portend-2.6-1.el8.noarch 19/32 2020-08-07T15:03:29.890 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:03:29.890 INFO:teuthology.orchestra.run.smithi074.stdout:Installed: 2020-08-07T15:03:29.890 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:29.891 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-mds-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:29.891 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-mgr-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:29.891 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-mgr-modules-core-2:16.0.0-3802.ge19d2f2a442.el8.noarch 2020-08-07T15:03:29.891 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-mon-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:29.891 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-osd-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:29.892 INFO:teuthology.orchestra.run.smithi074.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2020-08-07T15:03:29.892 INFO:teuthology.orchestra.run.smithi074.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2020-08-07T15:03:29.892 INFO:teuthology.orchestra.run.smithi074.stdout: python3-cheroot-8.2.1-1.el8.noarch 2020-08-07T15:03:29.892 INFO:teuthology.orchestra.run.smithi074.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2020-08-07T15:03:29.892 INFO:teuthology.orchestra.run.smithi074.stdout: python3-cssselect-0.9.2-10.el8.noarch 2020-08-07T15:03:29.892 INFO:teuthology.orchestra.run.smithi074.stdout: python3-html5lib-1:0.999999999-6.el8.noarch 2020-08-07T15:03:29.893 INFO:teuthology.orchestra.run.smithi074.stdout: python3-jaraco-6.2-6.el8.noarch 2020-08-07T15:03:29.893 INFO:teuthology.orchestra.run.smithi074.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2020-08-07T15:03:29.893 INFO:teuthology.orchestra.run.smithi074.stdout: python3-logutils-0.3.5-11.el8.noarch 2020-08-07T15:03:29.893 INFO:teuthology.orchestra.run.smithi074.stdout: python3-lxml-4.2.3-1.el8.x86_64 2020-08-07T15:03:29.893 INFO:teuthology.orchestra.run.smithi074.stdout: python3-mako-1.0.6-13.el8.noarch 2020-08-07T15:03:29.894 INFO:teuthology.orchestra.run.smithi074.stdout: python3-markupsafe-0.23-19.el8.x86_64 2020-08-07T15:03:29.894 INFO:teuthology.orchestra.run.smithi074.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2020-08-07T15:03:29.894 INFO:teuthology.orchestra.run.smithi074.stdout: python3-pecan-1.3.2-9.el8.noarch 2020-08-07T15:03:29.894 INFO:teuthology.orchestra.run.smithi074.stdout: python3-portend-2.6-1.el8.noarch 2020-08-07T15:03:29.901 INFO:teuthology.orchestra.run.smithi074.stdout: python3-pytz-2017.2-9.el8.noarch 2020-08-07T15:03:29.901 INFO:teuthology.orchestra.run.smithi074.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2020-08-07T15:03:29.901 INFO:teuthology.orchestra.run.smithi074.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2020-08-07T15:03:29.901 INFO:teuthology.orchestra.run.smithi074.stdout: python3-tempora-1.14.1-5.el8.noarch 2020-08-07T15:03:29.902 INFO:teuthology.orchestra.run.smithi074.stdout: python3-trustme-0.5.2-4.el8.noarch 2020-08-07T15:03:29.902 INFO:teuthology.orchestra.run.smithi074.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2020-08-07T15:03:29.902 INFO:teuthology.orchestra.run.smithi074.stdout: python3-webencodings-0.5.1-6.el8.noarch 2020-08-07T15:03:29.902 INFO:teuthology.orchestra.run.smithi074.stdout: python3-webob-1.8.5-1.el8.1.noarch 2020-08-07T15:03:29.903 INFO:teuthology.orchestra.run.smithi074.stdout: python3-webtest-2.0.33-1.el8.noarch 2020-08-07T15:03:29.903 INFO:teuthology.orchestra.run.smithi074.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2020-08-07T15:03:29.903 INFO:teuthology.orchestra.run.smithi074.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2020-08-07T15:03:29.903 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:03:29.904 INFO:teuthology.orchestra.run.smithi074.stdout:Complete! 2020-08-07T15:03:29.916 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 20/32 2020-08-07T15:03:30.098 INFO:teuthology.orchestra.run.smithi074:> sudo yum -y install ceph-base 2020-08-07T15:03:30.281 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cssselect-0.9.2-10.el8.noarch 21/32 2020-08-07T15:03:30.450 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T15:03:30.610 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-lxml-4.2.3-1.el8.x86_64 22/32 2020-08-07T15:03:30.866 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 23/32 2020-08-07T15:03:30.970 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 99 kB/s | 2.8 kB 00:00 2020-08-07T15:03:31.061 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 80 kB/s | 2.4 kB 00:00 2020-08-07T15:03:31.211 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 24/32 2020-08-07T15:03:31.825 INFO:teuthology.orchestra.run.smithi074.stdout:Package ceph-base-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 is already installed. 2020-08-07T15:03:32.002 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 25/32 2020-08-07T15:03:32.073 INFO:teuthology.orchestra.run.smithi074.stdout:Dependencies resolved. 2020-08-07T15:03:32.073 INFO:teuthology.orchestra.run.smithi074.stdout:Nothing to do. 2020-08-07T15:03:32.073 INFO:teuthology.orchestra.run.smithi074.stdout:Complete! 2020-08-07T15:03:32.226 INFO:teuthology.orchestra.run.smithi074:> sudo yum -y install cephadm 2020-08-07T15:03:32.332 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 26/32 2020-08-07T15:03:32.569 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T15:03:32.729 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-modules-core-2:16.0.0-3802.ge19d2f2a442.e 27/32 2020-08-07T15:03:32.978 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 28/32 2020-08-07T15:03:33.075 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 97 kB/s | 2.8 kB 00:00 2020-08-07T15:03:33.170 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 84 kB/s | 2.4 kB 00:00 2020-08-07T15:03:34.168 INFO:teuthology.orchestra.run.smithi074.stdout:Dependencies resolved. 2020-08-07T15:03:34.170 INFO:teuthology.orchestra.run.smithi074.stdout:================================================================================ 2020-08-07T15:03:34.170 INFO:teuthology.orchestra.run.smithi074.stdout: Package Arch Version Repo Size 2020-08-07T15:03:34.171 INFO:teuthology.orchestra.run.smithi074.stdout:================================================================================ 2020-08-07T15:03:34.171 INFO:teuthology.orchestra.run.smithi074.stdout:Installing: 2020-08-07T15:03:34.171 INFO:teuthology.orchestra.run.smithi074.stdout: cephadm x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 48 k 2020-08-07T15:03:34.172 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:03:34.174 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction Summary 2020-08-07T15:03:34.174 INFO:teuthology.orchestra.run.smithi074.stdout:================================================================================ 2020-08-07T15:03:34.174 INFO:teuthology.orchestra.run.smithi074.stdout:Install 1 Package 2020-08-07T15:03:34.175 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:03:34.175 INFO:teuthology.orchestra.run.smithi074.stdout:Total download size: 48 k 2020-08-07T15:03:34.175 INFO:teuthology.orchestra.run.smithi074.stdout:Installed size: 178 k 2020-08-07T15:03:34.176 INFO:teuthology.orchestra.run.smithi074.stdout:Downloading Packages: 2020-08-07T15:03:34.327 INFO:teuthology.orchestra.run.smithi074.stdout:cephadm-16.0.0-3802.ge19d2f2a442.el8.x86_64.rpm 314 kB/s | 48 kB 00:00 2020-08-07T15:03:34.327 INFO:teuthology.orchestra.run.smithi074.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:03:34.327 INFO:teuthology.orchestra.run.smithi074.stdout:Total 311 kB/s | 48 kB 00:00 2020-08-07T15:03:34.328 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction check 2020-08-07T15:03:34.332 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction check succeeded. 2020-08-07T15:03:34.333 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction test 2020-08-07T15:03:34.334 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction test succeeded. 2020-08-07T15:03:34.338 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction 2020-08-07T15:03:34.379 INFO:teuthology.orchestra.run.smithi074.stdout: Preparing : 1/1 2020-08-07T15:03:34.487 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 28/32 2020-08-07T15:03:34.738 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-osd-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 29/32 2020-08-07T15:03:35.161 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-osd-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 29/32 2020-08-07T15:03:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mon-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 30/32 2020-08-07T15:03:35.596 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mon-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 30/32 2020-08-07T15:03:35.771 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: cephadm-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:03:35.829 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mds-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 31/32 2020-08-07T15:03:35.847 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mds-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 31/32 2020-08-07T15:03:35.950 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : cephadm-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:03:36.016 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 32/32 2020-08-07T15:03:36.107 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: cephadm-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:03:36.338 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 32/32 2020-08-07T15:03:36.339 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/32 2020-08-07T15:03:36.339 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mds-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2/32 2020-08-07T15:03:36.339 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 3/32 2020-08-07T15:03:36.340 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mon-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 4/32 2020-08-07T15:03:36.340 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-osd-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 5/32 2020-08-07T15:03:36.340 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-modules-core-2:16.0.0-3802.ge19d2f2a442.e 6/32 2020-08-07T15:03:36.341 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/32 2020-08-07T15:03:36.341 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/32 2020-08-07T15:03:36.341 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cheroot-8.2.1-1.el8.noarch 9/32 2020-08-07T15:03:36.358 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/32 2020-08-07T15:03:36.359 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cssselect-0.9.2-10.el8.noarch 11/32 2020-08-07T15:03:36.359 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 12/32 2020-08-07T15:03:36.360 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 13/32 2020-08-07T15:03:36.360 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 14/32 2020-08-07T15:03:36.362 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 15/32 2020-08-07T15:03:36.362 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 16/32 2020-08-07T15:03:36.362 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-portend-2.6-1.el8.noarch 17/32 2020-08-07T15:03:36.363 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 18/32 2020-08-07T15:03:36.363 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 19/32 2020-08-07T15:03:36.363 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 20/32 2020-08-07T15:03:36.364 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-trustme-0.5.2-4.el8.noarch 21/32 2020-08-07T15:03:36.364 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 22/32 2020-08-07T15:03:36.364 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 23/32 2020-08-07T15:03:36.365 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 24/32 2020-08-07T15:03:36.365 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 25/32 2020-08-07T15:03:36.365 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-webencodings-0.5.1-6.el8.noarch 26/32 2020-08-07T15:03:36.366 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-lxml-4.2.3-1.el8.x86_64 27/32 2020-08-07T15:03:36.366 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 28/32 2020-08-07T15:03:36.366 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 29/32 2020-08-07T15:03:36.367 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-html5lib-1:0.999999999-6.el8.noarch 30/32 2020-08-07T15:03:36.367 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 31/32 2020-08-07T15:03:36.368 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : cephadm-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:03:36.368 INFO:teuthology.orchestra.run.smithi074.stdout:Installed products updated. 2020-08-07T15:03:37.068 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 32/32 2020-08-07T15:03:37.075 INFO:teuthology.orchestra.run.smithi096.stdout:Installed products updated. 2020-08-07T15:03:38.898 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:03:38.898 INFO:teuthology.orchestra.run.smithi074.stdout:Installed: 2020-08-07T15:03:38.899 INFO:teuthology.orchestra.run.smithi074.stdout: cephadm-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:38.899 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:03:38.899 INFO:teuthology.orchestra.run.smithi074.stdout:Complete! 2020-08-07T15:03:39.050 INFO:teuthology.orchestra.run.smithi074:> sudo yum -y install ceph-immutable-object-cache 2020-08-07T15:03:39.093 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:03:39.093 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2020-08-07T15:03:39.094 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:39.094 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:39.095 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:39.095 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core-2:16.0.0-3802.ge19d2f2a442.el8.noarch 2020-08-07T15:03:39.095 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:39.096 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:39.096 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2020-08-07T15:03:39.096 INFO:teuthology.orchestra.run.smithi096.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2020-08-07T15:03:39.097 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot-8.2.1-1.el8.noarch 2020-08-07T15:03:39.097 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2020-08-07T15:03:39.097 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cssselect-0.9.2-10.el8.noarch 2020-08-07T15:03:39.098 INFO:teuthology.orchestra.run.smithi096.stdout: python3-html5lib-1:0.999999999-6.el8.noarch 2020-08-07T15:03:39.098 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-6.2-6.el8.noarch 2020-08-07T15:03:39.099 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2020-08-07T15:03:39.099 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils-0.3.5-11.el8.noarch 2020-08-07T15:03:39.099 INFO:teuthology.orchestra.run.smithi096.stdout: python3-lxml-4.2.3-1.el8.x86_64 2020-08-07T15:03:39.100 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako-1.0.6-13.el8.noarch 2020-08-07T15:03:39.100 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe-0.23-19.el8.x86_64 2020-08-07T15:03:39.100 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2020-08-07T15:03:39.101 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan-1.3.2-9.el8.noarch 2020-08-07T15:03:39.102 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend-2.6-1.el8.noarch 2020-08-07T15:03:39.102 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz-2017.2-9.el8.noarch 2020-08-07T15:03:39.102 INFO:teuthology.orchestra.run.smithi096.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2020-08-07T15:03:39.103 INFO:teuthology.orchestra.run.smithi096.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2020-08-07T15:03:39.103 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora-1.14.1-5.el8.noarch 2020-08-07T15:03:39.103 INFO:teuthology.orchestra.run.smithi096.stdout: python3-trustme-0.5.2-4.el8.noarch 2020-08-07T15:03:39.104 INFO:teuthology.orchestra.run.smithi096.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2020-08-07T15:03:39.104 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webencodings-0.5.1-6.el8.noarch 2020-08-07T15:03:39.104 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob-1.8.5-1.el8.1.noarch 2020-08-07T15:03:39.105 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webtest-2.0.33-1.el8.noarch 2020-08-07T15:03:39.105 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2020-08-07T15:03:39.105 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2020-08-07T15:03:39.106 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:03:39.106 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2020-08-07T15:03:39.301 INFO:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-base 2020-08-07T15:03:39.407 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T15:03:39.654 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T15:03:39.903 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 98 kB/s | 2.8 kB 00:00 2020-08-07T15:03:39.995 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 82 kB/s | 2.4 kB 00:00 2020-08-07T15:03:40.137 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 99 kB/s | 2.8 kB 00:00 2020-08-07T15:03:40.233 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 83 kB/s | 2.4 kB 00:00 2020-08-07T15:03:40.986 INFO:teuthology.orchestra.run.smithi074.stdout:Dependencies resolved. 2020-08-07T15:03:40.987 INFO:teuthology.orchestra.run.smithi074.stdout:================================================================================ 2020-08-07T15:03:40.987 INFO:teuthology.orchestra.run.smithi074.stdout: Package Arch Version Repo Size 2020-08-07T15:03:40.987 INFO:teuthology.orchestra.run.smithi074.stdout:================================================================================ 2020-08-07T15:03:40.988 INFO:teuthology.orchestra.run.smithi074.stdout:Installing: 2020-08-07T15:03:40.988 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-immutable-object-cache x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 202 k 2020-08-07T15:03:40.988 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:03:40.988 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction Summary 2020-08-07T15:03:40.988 INFO:teuthology.orchestra.run.smithi074.stdout:================================================================================ 2020-08-07T15:03:40.989 INFO:teuthology.orchestra.run.smithi074.stdout:Install 1 Package 2020-08-07T15:03:40.989 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:03:40.989 INFO:teuthology.orchestra.run.smithi074.stdout:Total download size: 202 k 2020-08-07T15:03:40.990 INFO:teuthology.orchestra.run.smithi074.stdout:Installed size: 689 k 2020-08-07T15:03:40.991 INFO:teuthology.orchestra.run.smithi074.stdout:Downloading Packages: 2020-08-07T15:03:41.019 INFO:teuthology.orchestra.run.smithi096.stdout:Package ceph-base-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 is already installed. 2020-08-07T15:03:41.195 INFO:teuthology.orchestra.run.smithi074.stdout:ceph-immutable-object-cache-16.0.0-3802.ge19d2f 1.0 MB/s | 202 kB 00:00 2020-08-07T15:03:41.195 INFO:teuthology.orchestra.run.smithi074.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:03:41.195 INFO:teuthology.orchestra.run.smithi074.stdout:Total 1.0 MB/s | 202 kB 00:00 2020-08-07T15:03:41.196 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction check 2020-08-07T15:03:41.202 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction check succeeded. 2020-08-07T15:03:41.203 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction test 2020-08-07T15:03:41.251 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction test succeeded. 2020-08-07T15:03:41.270 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2020-08-07T15:03:41.270 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2020-08-07T15:03:41.270 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2020-08-07T15:03:41.270 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction 2020-08-07T15:03:41.376 INFO:teuthology.orchestra.run.smithi074.stdout: Preparing : 1/1 2020-08-07T15:03:41.498 INFO:teuthology.orchestra.run.smithi096:> sudo yum -y install cephadm 2020-08-07T15:03:41.521 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : ceph-immutable-object-cache-2:16.0.0-3802.ge19d2f2a4 1/1 2020-08-07T15:03:41.839 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T15:03:41.845 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.0.0-3802.ge19d2f2a4 1/1 2020-08-07T15:03:42.075 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : ceph-immutable-object-cache-2:16.0.0-3802.ge19d2f2a4 1/1 2020-08-07T15:03:42.075 INFO:teuthology.orchestra.run.smithi074.stdout:Installed products updated. 2020-08-07T15:03:42.322 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 99 kB/s | 2.8 kB 00:00 2020-08-07T15:03:42.418 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 83 kB/s | 2.4 kB 00:00 2020-08-07T15:03:44.054 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:03:44.065 INFO:teuthology.orchestra.run.smithi074.stdout:Installed: 2020-08-07T15:03:44.066 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-immutable-object-cache-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:44.066 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:03:44.068 INFO:teuthology.orchestra.run.smithi074.stdout:Complete! 2020-08-07T15:03:44.203 INFO:teuthology.orchestra.run.smithi074:> sudo yum -y install ceph-mgr 2020-08-07T15:03:44.221 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2020-08-07T15:03:44.221 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2020-08-07T15:03:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repo Size 2020-08-07T15:03:44.221 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2020-08-07T15:03:44.222 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2020-08-07T15:03:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 48 k 2020-08-07T15:03:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:03:44.222 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2020-08-07T15:03:44.222 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2020-08-07T15:03:44.223 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2020-08-07T15:03:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:03:44.231 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 48 k 2020-08-07T15:03:44.231 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 178 k 2020-08-07T15:03:44.232 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2020-08-07T15:03:44.369 INFO:teuthology.orchestra.run.smithi096.stdout:cephadm-16.0.0-3802.ge19d2f2a442.el8.x86_64.rpm 318 kB/s | 48 kB 00:00 2020-08-07T15:03:44.370 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:03:44.371 INFO:teuthology.orchestra.run.smithi096.stdout:Total 316 kB/s | 48 kB 00:00 2020-08-07T15:03:44.371 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2020-08-07T15:03:44.377 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2020-08-07T15:03:44.377 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2020-08-07T15:03:44.378 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2020-08-07T15:03:44.381 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2020-08-07T15:03:44.428 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2020-08-07T15:03:44.550 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T15:03:45.046 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 89 kB/s | 2.8 kB 00:00 2020-08-07T15:03:45.141 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 80 kB/s | 2.4 kB 00:00 2020-08-07T15:03:45.843 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephadm-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:03:45.939 INFO:teuthology.orchestra.run.smithi074.stdout:Package ceph-mgr-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 is already installed. 2020-08-07T15:03:46.071 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephadm-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:03:46.163 INFO:teuthology.orchestra.run.smithi074.stdout:Dependencies resolved. 2020-08-07T15:03:46.163 INFO:teuthology.orchestra.run.smithi074.stdout:Nothing to do. 2020-08-07T15:03:46.164 INFO:teuthology.orchestra.run.smithi074.stdout:Complete! 2020-08-07T15:03:46.234 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephadm-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:03:46.316 INFO:teuthology.orchestra.run.smithi074:> sudo yum -y install ceph-mgr-dashboard 2020-08-07T15:03:46.451 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephadm-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:03:46.452 INFO:teuthology.orchestra.run.smithi096.stdout:Installed products updated. 2020-08-07T15:03:46.662 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T15:03:47.218 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 100 kB/s | 2.8 kB 00:00 2020-08-07T15:03:47.293 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 84 kB/s | 2.4 kB 00:00 2020-08-07T15:03:48.294 INFO:teuthology.orchestra.run.smithi074.stdout:Dependencies resolved. 2020-08-07T15:03:48.295 INFO:teuthology.orchestra.run.smithi074.stdout:====================================================================================================== 2020-08-07T15:03:48.295 INFO:teuthology.orchestra.run.smithi074.stdout: Package Arch Version Repository Size 2020-08-07T15:03:48.295 INFO:teuthology.orchestra.run.smithi074.stdout:====================================================================================================== 2020-08-07T15:03:48.296 INFO:teuthology.orchestra.run.smithi074.stdout:Installing: 2020-08-07T15:03:48.296 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-mgr-dashboard noarch 2:16.0.0-3802.ge19d2f2a442.el8 ceph-noarch 3.2 M 2020-08-07T15:03:48.296 INFO:teuthology.orchestra.run.smithi074.stdout:Installing dependencies: 2020-08-07T15:03:48.297 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-grafana-dashboards noarch 2:16.0.0-3802.ge19d2f2a442.el8 ceph-noarch 24 k 2020-08-07T15:03:48.297 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-prometheus-alerts noarch 2:16.0.0-3802.ge19d2f2a442.el8 ceph-noarch 8.8 k 2020-08-07T15:03:48.297 INFO:teuthology.orchestra.run.smithi074.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2020-08-07T15:03:48.298 INFO:teuthology.orchestra.run.smithi074.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2020-08-07T15:03:48.298 INFO:teuthology.orchestra.run.smithi074.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2020-08-07T15:03:48.298 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:03:48.299 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction Summary 2020-08-07T15:03:48.299 INFO:teuthology.orchestra.run.smithi074.stdout:====================================================================================================== 2020-08-07T15:03:48.304 INFO:teuthology.orchestra.run.smithi074.stdout:Install 6 Packages 2020-08-07T15:03:48.304 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:03:48.304 INFO:teuthology.orchestra.run.smithi074.stdout:Total download size: 3.5 M 2020-08-07T15:03:48.305 INFO:teuthology.orchestra.run.smithi074.stdout:Installed size: 14 M 2020-08-07T15:03:48.305 INFO:teuthology.orchestra.run.smithi074.stdout:Downloading Packages: 2020-08-07T15:03:48.410 INFO:teuthology.orchestra.run.smithi074.stdout:(1/6): ceph-prometheus-alerts-16.0.0-3802.ge19d 78 kB/s | 8.8 kB 00:00 2020-08-07T15:03:48.430 INFO:teuthology.orchestra.run.smithi074.stdout:(2/6): ceph-grafana-dashboards-16.0.0-3802.ge19 184 kB/s | 24 kB 00:00 2020-08-07T15:03:48.462 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:03:48.463 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2020-08-07T15:03:48.463 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:48.463 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:03:48.463 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2020-08-07T15:03:48.463 INFO:teuthology.orchestra.run.smithi074.stdout:(3/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 703 kB/s | 34 kB 00:00 2020-08-07T15:03:48.488 INFO:teuthology.orchestra.run.smithi074.stdout:(4/6): python3-jwt-1.6.1-2.el8.noarch.rpm 1.4 MB/s | 43 kB 00:00 2020-08-07T15:03:48.539 INFO:teuthology.orchestra.run.smithi074.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.7 MB/s | 196 kB 00:00 2020-08-07T15:03:48.655 INFO:teuthology.orchestra.run.smithi074.stdout:(6/6): ceph-mgr-dashboard-16.0.0-3802.ge19d2f2a 9.0 MB/s | 3.2 MB 00:00 2020-08-07T15:03:48.656 INFO:teuthology.orchestra.run.smithi074.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:03:48.657 INFO:teuthology.orchestra.run.smithi074.stdout:Total 9.8 MB/s | 3.5 MB 00:00 2020-08-07T15:03:48.658 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction check 2020-08-07T15:03:48.668 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction check succeeded. 2020-08-07T15:03:48.669 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction test 2020-08-07T15:03:48.687 INFO:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-immutable-object-cache 2020-08-07T15:03:48.782 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction test succeeded. 2020-08-07T15:03:48.785 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction 2020-08-07T15:03:49.040 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T15:03:49.105 INFO:teuthology.orchestra.run.smithi074.stdout: Preparing : 1/1 2020-08-07T15:03:49.312 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 1/6 2020-08-07T15:03:49.532 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 2/6 2020-08-07T15:03:49.755 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 98 kB/s | 2.8 kB 00:00 2020-08-07T15:03:49.755 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 81 kB/s | 2.4 kB 00:00 2020-08-07T15:03:49.756 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 3/6 2020-08-07T15:03:49.818 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : ceph-prometheus-alerts-2:16.0.0-3802.ge19d2f2a442.el 4/6 2020-08-07T15:03:50.782 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2020-08-07T15:03:50.782 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2020-08-07T15:03:50.783 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repo Size 2020-08-07T15:03:50.783 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2020-08-07T15:03:50.783 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2020-08-07T15:03:50.784 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 202 k 2020-08-07T15:03:50.786 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:03:50.786 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2020-08-07T15:03:50.786 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2020-08-07T15:03:50.787 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2020-08-07T15:03:50.787 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:03:50.787 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 202 k 2020-08-07T15:03:50.788 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 689 k 2020-08-07T15:03:50.788 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2020-08-07T15:03:50.836 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : ceph-grafana-dashboards-2:16.0.0-3802.ge19d2f2a442.e 5/6 2020-08-07T15:03:50.983 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-immutable-object-cache-16.0.0-3802.ge19d2f 1.0 MB/s | 202 kB 00:00 2020-08-07T15:03:50.984 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:03:50.984 INFO:teuthology.orchestra.run.smithi096.stdout:Total 1.0 MB/s | 202 kB 00:00 2020-08-07T15:03:50.984 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2020-08-07T15:03:50.996 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2020-08-07T15:03:50.997 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2020-08-07T15:03:51.045 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2020-08-07T15:03:51.047 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2020-08-07T15:03:51.134 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : ceph-mgr-dashboard-2:16.0.0-3802.ge19d2f2a442.el8.no 6/6 2020-08-07T15:03:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2020-08-07T15:03:51.364 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.0.0-3802.ge19d2f2a442.el8.no 6/6 2020-08-07T15:03:51.364 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : ceph-grafana-dashboards-2:16.0.0-3802.ge19d2f2a442.e 1/6 2020-08-07T15:03:51.364 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : ceph-mgr-dashboard-2:16.0.0-3802.ge19d2f2a442.el8.no 2/6 2020-08-07T15:03:51.364 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : ceph-prometheus-alerts-2:16.0.0-3802.ge19d2f2a442.el 3/6 2020-08-07T15:03:51.365 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/6 2020-08-07T15:03:51.365 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/6 2020-08-07T15:03:51.389 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-immutable-object-cache-2:16.0.0-3802.ge19d2f2a4 1/1 2020-08-07T15:03:51.621 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 6/6 2020-08-07T15:03:51.622 INFO:teuthology.orchestra.run.smithi074.stdout:Installed products updated. 2020-08-07T15:03:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.0.0-3802.ge19d2f2a4 1/1 2020-08-07T15:03:51.967 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-immutable-object-cache-2:16.0.0-3802.ge19d2f2a4 1/1 2020-08-07T15:03:51.967 INFO:teuthology.orchestra.run.smithi096.stdout:Installed products updated. 2020-08-07T15:03:53.535 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:03:53.535 INFO:teuthology.orchestra.run.smithi074.stdout:Installed: 2020-08-07T15:03:53.536 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-grafana-dashboards-2:16.0.0-3802.ge19d2f2a442.el8.noarch 2020-08-07T15:03:53.536 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-mgr-dashboard-2:16.0.0-3802.ge19d2f2a442.el8.noarch 2020-08-07T15:03:53.536 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-prometheus-alerts-2:16.0.0-3802.ge19d2f2a442.el8.noarch 2020-08-07T15:03:53.536 INFO:teuthology.orchestra.run.smithi074.stdout: python3-jwt-1.6.1-2.el8.noarch 2020-08-07T15:03:53.536 INFO:teuthology.orchestra.run.smithi074.stdout: python3-repoze-lru-0.7-6.el8.noarch 2020-08-07T15:03:53.537 INFO:teuthology.orchestra.run.smithi074.stdout: python3-routes-2.4.1-12.el8.noarch 2020-08-07T15:03:53.537 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:03:53.537 INFO:teuthology.orchestra.run.smithi074.stdout:Complete! 2020-08-07T15:03:53.681 INFO:teuthology.orchestra.run.smithi074:> sudo yum -y install ceph-mgr-diskprediction-local 2020-08-07T15:03:53.937 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:03:53.938 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2020-08-07T15:03:53.938 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:03:53.938 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:03:53.938 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2020-08-07T15:03:54.030 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T15:03:54.162 INFO:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-mgr 2020-08-07T15:03:54.502 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 94 kB/s | 2.8 kB 00:00 2020-08-07T15:03:54.508 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T15:03:54.598 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 84 kB/s | 2.4 kB 00:00 2020-08-07T15:03:54.993 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 99 kB/s | 2.8 kB 00:00 2020-08-07T15:03:55.080 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 83 kB/s | 2.4 kB 00:00 2020-08-07T15:03:55.580 INFO:teuthology.orchestra.run.smithi074.stdout:Dependencies resolved. 2020-08-07T15:03:55.581 INFO:teuthology.orchestra.run.smithi074.stdout:====================================================================================================================== 2020-08-07T15:03:55.581 INFO:teuthology.orchestra.run.smithi074.stdout: Package Arch Version Repository Size 2020-08-07T15:03:55.581 INFO:teuthology.orchestra.run.smithi074.stdout:====================================================================================================================== 2020-08-07T15:03:55.582 INFO:teuthology.orchestra.run.smithi074.stdout:Installing: 2020-08-07T15:03:55.582 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-mgr-diskprediction-local noarch 2:16.0.0-3802.ge19d2f2a442.el8 ceph-noarch 8.6 M 2020-08-07T15:03:55.582 INFO:teuthology.orchestra.run.smithi074.stdout:Installing dependencies: 2020-08-07T15:03:55.582 INFO:teuthology.orchestra.run.smithi074.stdout: libgfortran x86_64 8.3.1-5.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2020-08-07T15:03:55.583 INFO:teuthology.orchestra.run.smithi074.stdout: libquadmath x86_64 8.3.1-5.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2020-08-07T15:03:55.583 INFO:teuthology.orchestra.run.smithi074.stdout: openblas x86_64 0.3.3-5.el8 rhel-8-for-x86_64-appstream-rpms 4.3 M 2020-08-07T15:03:55.583 INFO:teuthology.orchestra.run.smithi074.stdout: openblas-threads x86_64 0.3.3-5.el8 rhel-8-for-x86_64-appstream-rpms 4.4 M 2020-08-07T15:03:55.583 INFO:teuthology.orchestra.run.smithi074.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2020-08-07T15:03:55.583 INFO:teuthology.orchestra.run.smithi074.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2020-08-07T15:03:55.583 INFO:teuthology.orchestra.run.smithi074.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2020-08-07T15:03:55.584 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:03:55.584 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction Summary 2020-08-07T15:03:55.584 INFO:teuthology.orchestra.run.smithi074.stdout:====================================================================================================================== 2020-08-07T15:03:55.584 INFO:teuthology.orchestra.run.smithi074.stdout:Install 8 Packages 2020-08-07T15:03:55.584 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:03:55.585 INFO:teuthology.orchestra.run.smithi074.stdout:Total download size: 37 M 2020-08-07T15:03:55.586 INFO:teuthology.orchestra.run.smithi074.stdout:Installed size: 196 M 2020-08-07T15:03:55.586 INFO:teuthology.orchestra.run.smithi074.stdout:Downloading Packages: 2020-08-07T15:03:55.632 INFO:teuthology.orchestra.run.smithi074.stdout:(1/8): python3-numpy-f2py-1.14.3-9.el8.x86_64.r 4.5 MB/s | 225 kB 00:00 2020-08-07T15:03:55.659 INFO:teuthology.orchestra.run.smithi074.stdout:(2/8): python3-numpy-1.14.3-9.el8.x86_64.rpm 49 MB/s | 3.7 MB 00:00 2020-08-07T15:03:55.713 INFO:teuthology.orchestra.run.smithi074.stdout:(3/8): openblas-threads-0.3.3-5.el8.x86_64.rpm 81 MB/s | 4.4 MB 00:00 2020-08-07T15:03:55.772 INFO:teuthology.orchestra.run.smithi074.stdout:(4/8): openblas-0.3.3-5.el8.x86_64.rpm 81 MB/s | 4.3 MB 00:00 2020-08-07T15:03:55.840 INFO:teuthology.orchestra.run.smithi074.stdout:(5/8): python3-scipy-1.0.0-20.module+el8.1.0+33 70 MB/s | 15 MB 00:00 2020-08-07T15:03:55.871 INFO:teuthology.orchestra.run.smithi074.stdout:(6/8): libquadmath-8.3.1-5.el8.x86_64.rpm 1.7 MB/s | 169 kB 00:00 2020-08-07T15:03:55.872 INFO:teuthology.orchestra.run.smithi074.stdout:(7/8): libgfortran-8.3.1-5.el8.x86_64.rpm 24 MB/s | 639 kB 00:00 2020-08-07T15:03:55.892 INFO:teuthology.orchestra.run.smithi096.stdout:Package ceph-mgr-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 is already installed. 2020-08-07T15:03:56.138 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2020-08-07T15:03:56.139 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2020-08-07T15:03:56.139 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2020-08-07T15:03:56.182 INFO:teuthology.orchestra.run.smithi074.stdout:(8/8): ceph-mgr-diskprediction-local-16.0.0-380 14 MB/s | 8.6 MB 00:00 2020-08-07T15:03:56.183 INFO:teuthology.orchestra.run.smithi074.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:03:56.183 INFO:teuthology.orchestra.run.smithi074.stdout:Total 61 MB/s | 37 MB 00:00 2020-08-07T15:03:56.192 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction check 2020-08-07T15:03:56.212 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction check succeeded. 2020-08-07T15:03:56.212 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction test 2020-08-07T15:03:56.365 INFO:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-mgr-dashboard 2020-08-07T15:03:56.502 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction test succeeded. 2020-08-07T15:03:56.518 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction 2020-08-07T15:03:56.708 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T15:03:57.079 INFO:teuthology.orchestra.run.smithi074.stdout: Preparing : 1/1 2020-08-07T15:03:57.241 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : libquadmath-8.3.1-5.el8.x86_64 1/8 2020-08-07T15:03:57.241 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 97 kB/s | 2.8 kB 00:00 2020-08-07T15:03:57.334 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 80 kB/s | 2.4 kB 00:00 2020-08-07T15:03:57.378 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: libquadmath-8.3.1-5.el8.x86_64 1/8 2020-08-07T15:03:57.582 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : libgfortran-8.3.1-5.el8.x86_64 2/8 2020-08-07T15:03:58.092 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: libgfortran-8.3.1-5.el8.x86_64 2/8 2020-08-07T15:03:58.206 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : openblas-threads-0.3.3-5.el8.x86_64 3/8 2020-08-07T15:03:58.365 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2020-08-07T15:03:58.366 INFO:teuthology.orchestra.run.smithi096.stdout:====================================================================================================== 2020-08-07T15:03:58.366 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2020-08-07T15:03:58.366 INFO:teuthology.orchestra.run.smithi096.stdout:====================================================================================================== 2020-08-07T15:03:58.366 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2020-08-07T15:03:58.366 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard noarch 2:16.0.0-3802.ge19d2f2a442.el8 ceph-noarch 3.2 M 2020-08-07T15:03:58.367 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2020-08-07T15:03:58.367 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards noarch 2:16.0.0-3802.ge19d2f2a442.el8 ceph-noarch 24 k 2020-08-07T15:03:58.367 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts noarch 2:16.0.0-3802.ge19d2f2a442.el8 ceph-noarch 8.8 k 2020-08-07T15:03:58.367 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2020-08-07T15:03:58.367 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2020-08-07T15:03:58.368 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2020-08-07T15:03:58.368 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:03:58.368 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2020-08-07T15:03:58.368 INFO:teuthology.orchestra.run.smithi096.stdout:====================================================================================================== 2020-08-07T15:03:58.368 INFO:teuthology.orchestra.run.smithi096.stdout:Install 6 Packages 2020-08-07T15:03:58.369 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:03:58.369 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 3.5 M 2020-08-07T15:03:58.369 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 14 M 2020-08-07T15:03:58.369 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2020-08-07T15:03:58.480 INFO:teuthology.orchestra.run.smithi096.stdout:(1/6): ceph-prometheus-alerts-16.0.0-3802.ge19d 78 kB/s | 8.8 kB 00:00 2020-08-07T15:03:58.494 INFO:teuthology.orchestra.run.smithi096.stdout:(2/6): ceph-grafana-dashboards-16.0.0-3802.ge19 188 kB/s | 24 kB 00:00 2020-08-07T15:03:58.529 INFO:teuthology.orchestra.run.smithi096.stdout:(3/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 705 kB/s | 34 kB 00:00 2020-08-07T15:03:58.562 INFO:teuthology.orchestra.run.smithi096.stdout:(4/6): python3-jwt-1.6.1-2.el8.noarch.rpm 1.4 MB/s | 43 kB 00:00 2020-08-07T15:03:58.589 INFO:teuthology.orchestra.run.smithi096.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 2.0 MB/s | 196 kB 00:00 2020-08-07T15:03:58.715 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: openblas-threads-0.3.3-5.el8.x86_64 3/8 2020-08-07T15:03:58.724 INFO:teuthology.orchestra.run.smithi096.stdout:(6/6): ceph-mgr-dashboard-16.0.0-3802.ge19d2f2a 9.1 MB/s | 3.2 MB 00:00 2020-08-07T15:03:58.724 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:03:58.725 INFO:teuthology.orchestra.run.smithi096.stdout:Total 9.9 MB/s | 3.5 MB 00:00 2020-08-07T15:03:58.736 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2020-08-07T15:03:58.736 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2020-08-07T15:03:58.736 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2020-08-07T15:03:58.847 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : openblas-0.3.3-5.el8.x86_64 4/8 2020-08-07T15:03:58.853 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2020-08-07T15:03:58.856 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2020-08-07T15:03:59.141 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2020-08-07T15:03:59.380 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 1/6 2020-08-07T15:03:59.854 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 2/6 2020-08-07T15:03:59.986 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: openblas-0.3.3-5.el8.x86_64 4/8 2020-08-07T15:04:00.239 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 3/6 2020-08-07T15:04:00.329 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 5/8 2020-08-07T15:04:00.388 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-prometheus-alerts-2:16.0.0-3802.ge19d2f2a442.el 4/6 2020-08-07T15:04:00.556 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2020-08-07T15:04:01.479 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-grafana-dashboards-2:16.0.0-3802.ge19d2f2a442.e 5/6 2020-08-07T15:04:01.747 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-dashboard-2:16.0.0-3802.ge19d2f2a442.el8.no 6/6 2020-08-07T15:04:01.941 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.0.0-3802.ge19d2f2a442.el8.no 6/6 2020-08-07T15:04:01.942 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-grafana-dashboards-2:16.0.0-3802.ge19d2f2a442.e 1/6 2020-08-07T15:04:01.942 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-dashboard-2:16.0.0-3802.ge19d2f2a442.el8.no 2/6 2020-08-07T15:04:01.951 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-prometheus-alerts-2:16.0.0-3802.ge19d2f2a442.el 3/6 2020-08-07T15:04:01.952 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/6 2020-08-07T15:04:01.953 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/6 2020-08-07T15:04:02.198 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 6/6 2020-08-07T15:04:02.198 INFO:teuthology.orchestra.run.smithi096.stdout:Installed products updated. 2020-08-07T15:04:02.541 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2020-08-07T15:04:03.815 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 7/8 2020-08-07T15:04:04.033 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : ceph-mgr-diskprediction-local-2:16.0.0-3802.ge19d2f2 8/8 2020-08-07T15:04:04.142 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:04.143 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2020-08-07T15:04:04.143 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards-2:16.0.0-3802.ge19d2f2a442.el8.noarch 2020-08-07T15:04:04.143 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard-2:16.0.0-3802.ge19d2f2a442.el8.noarch 2020-08-07T15:04:04.144 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts-2:16.0.0-3802.ge19d2f2a442.el8.noarch 2020-08-07T15:04:04.144 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jwt-1.6.1-2.el8.noarch 2020-08-07T15:04:04.144 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru-0.7-6.el8.noarch 2020-08-07T15:04:04.145 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes-2.4.1-12.el8.noarch 2020-08-07T15:04:04.145 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:04.145 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2020-08-07T15:04:04.259 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.0.0-3802.ge19d2f2 8/8 2020-08-07T15:04:04.259 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.0.0-3802.ge19d2f2 1/8 2020-08-07T15:04:04.260 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 2/8 2020-08-07T15:04:04.260 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 3/8 2020-08-07T15:04:04.260 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 4/8 2020-08-07T15:04:04.261 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : openblas-threads-0.3.3-5.el8.x86_64 5/8 2020-08-07T15:04:04.261 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : openblas-0.3.3-5.el8.x86_64 6/8 2020-08-07T15:04:04.261 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : libquadmath-8.3.1-5.el8.x86_64 7/8 2020-08-07T15:04:04.370 INFO:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-mgr-diskprediction-local 2020-08-07T15:04:04.556 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : libgfortran-8.3.1-5.el8.x86_64 8/8 2020-08-07T15:04:04.557 INFO:teuthology.orchestra.run.smithi074.stdout:Installed products updated. 2020-08-07T15:04:04.718 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T15:04:05.200 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 97 kB/s | 2.8 kB 00:00 2020-08-07T15:04:05.300 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 83 kB/s | 2.4 kB 00:00 2020-08-07T15:04:06.541 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2020-08-07T15:04:06.541 INFO:teuthology.orchestra.run.smithi096.stdout:====================================================================================================================== 2020-08-07T15:04:06.542 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2020-08-07T15:04:06.542 INFO:teuthology.orchestra.run.smithi096.stdout:====================================================================================================================== 2020-08-07T15:04:06.542 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2020-08-07T15:04:06.542 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local noarch 2:16.0.0-3802.ge19d2f2a442.el8 ceph-noarch 8.6 M 2020-08-07T15:04:06.566 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2020-08-07T15:04:06.566 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran x86_64 8.3.1-5.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2020-08-07T15:04:06.567 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath x86_64 8.3.1-5.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2020-08-07T15:04:06.567 INFO:teuthology.orchestra.run.smithi096.stdout: openblas x86_64 0.3.3-5.el8 rhel-8-for-x86_64-appstream-rpms 4.3 M 2020-08-07T15:04:06.567 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-threads x86_64 0.3.3-5.el8 rhel-8-for-x86_64-appstream-rpms 4.4 M 2020-08-07T15:04:06.567 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2020-08-07T15:04:06.567 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2020-08-07T15:04:06.568 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2020-08-07T15:04:06.568 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:06.568 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2020-08-07T15:04:06.568 INFO:teuthology.orchestra.run.smithi096.stdout:====================================================================================================================== 2020-08-07T15:04:06.568 INFO:teuthology.orchestra.run.smithi096.stdout:Install 8 Packages 2020-08-07T15:04:06.568 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:06.569 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 37 M 2020-08-07T15:04:06.569 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 196 M 2020-08-07T15:04:06.569 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2020-08-07T15:04:06.583 INFO:teuthology.orchestra.run.smithi096.stdout:(1/8): python3-numpy-f2py-1.14.3-9.el8.x86_64.r 4.5 MB/s | 225 kB 00:00 2020-08-07T15:04:06.602 INFO:teuthology.orchestra.run.smithi096.stdout:(2/8): python3-numpy-1.14.3-9.el8.x86_64.rpm 54 MB/s | 3.7 MB 00:00 2020-08-07T15:04:06.660 INFO:teuthology.orchestra.run.smithi096.stdout:(3/8): openblas-threads-0.3.3-5.el8.x86_64.rpm 82 MB/s | 4.4 MB 00:00 2020-08-07T15:04:06.718 INFO:teuthology.orchestra.run.smithi096.stdout:(4/8): openblas-0.3.3-5.el8.x86_64.rpm 80 MB/s | 4.3 MB 00:00 2020-08-07T15:04:06.755 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:04:06.756 INFO:teuthology.orchestra.run.smithi074.stdout:Installed: 2020-08-07T15:04:06.756 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-mgr-diskprediction-local-2:16.0.0-3802.ge19d2f2a442.el8.noarch 2020-08-07T15:04:06.756 INFO:teuthology.orchestra.run.smithi074.stdout: libgfortran-8.3.1-5.el8.x86_64 2020-08-07T15:04:06.756 INFO:teuthology.orchestra.run.smithi074.stdout: libquadmath-8.3.1-5.el8.x86_64 2020-08-07T15:04:06.756 INFO:teuthology.orchestra.run.smithi074.stdout: openblas-0.3.3-5.el8.x86_64 2020-08-07T15:04:06.757 INFO:teuthology.orchestra.run.smithi074.stdout: openblas-threads-0.3.3-5.el8.x86_64 2020-08-07T15:04:06.757 INFO:teuthology.orchestra.run.smithi074.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2020-08-07T15:04:06.757 INFO:teuthology.orchestra.run.smithi074.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2020-08-07T15:04:06.762 INFO:teuthology.orchestra.run.smithi074.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2020-08-07T15:04:06.763 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:04:06.763 INFO:teuthology.orchestra.run.smithi074.stdout:Complete! 2020-08-07T15:04:06.792 INFO:teuthology.orchestra.run.smithi096.stdout:(5/8): python3-scipy-1.0.0-20.module+el8.1.0+33 70 MB/s | 15 MB 00:00 2020-08-07T15:04:06.832 INFO:teuthology.orchestra.run.smithi096.stdout:(6/8): libquadmath-8.3.1-5.el8.x86_64.rpm 1.6 MB/s | 169 kB 00:00 2020-08-07T15:04:06.833 INFO:teuthology.orchestra.run.smithi096.stdout:(7/8): libgfortran-8.3.1-5.el8.x86_64.rpm 24 MB/s | 639 kB 00:00 2020-08-07T15:04:06.935 INFO:teuthology.orchestra.run.smithi074:> sudo yum -y install ceph-mgr-rook 2020-08-07T15:04:07.292 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T15:04:07.787 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 99 kB/s | 2.8 kB 00:00 2020-08-07T15:04:07.884 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 83 kB/s | 2.4 kB 00:00 2020-08-07T15:04:08.313 INFO:teuthology.orchestra.run.smithi096.stdout:(8/8): ceph-mgr-diskprediction-local-16.0.0-380 4.8 MB/s | 8.6 MB 00:01 2020-08-07T15:04:08.314 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:04:08.315 INFO:teuthology.orchestra.run.smithi096.stdout:Total 20 MB/s | 37 MB 00:01 2020-08-07T15:04:08.325 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2020-08-07T15:04:08.345 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2020-08-07T15:04:08.345 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2020-08-07T15:04:08.650 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2020-08-07T15:04:08.652 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2020-08-07T15:04:08.920 INFO:teuthology.orchestra.run.smithi074.stdout:Dependencies resolved. 2020-08-07T15:04:08.921 INFO:teuthology.orchestra.run.smithi074.stdout:=========================================================================================================== 2020-08-07T15:04:08.922 INFO:teuthology.orchestra.run.smithi074.stdout: Package Arch Version Repository Size 2020-08-07T15:04:08.922 INFO:teuthology.orchestra.run.smithi074.stdout:=========================================================================================================== 2020-08-07T15:04:08.922 INFO:teuthology.orchestra.run.smithi074.stdout:Installing: 2020-08-07T15:04:08.922 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-mgr-rook noarch 2:16.0.0-3802.ge19d2f2a442.el8 ceph-noarch 27 k 2020-08-07T15:04:08.923 INFO:teuthology.orchestra.run.smithi074.stdout:Installing dependencies: 2020-08-07T15:04:08.923 INFO:teuthology.orchestra.run.smithi074.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2020-08-07T15:04:08.923 INFO:teuthology.orchestra.run.smithi074.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2020-08-07T15:04:08.923 INFO:teuthology.orchestra.run.smithi074.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2020-08-07T15:04:08.923 INFO:teuthology.orchestra.run.smithi074.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2020-08-07T15:04:08.924 INFO:teuthology.orchestra.run.smithi074.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2020-08-07T15:04:08.924 INFO:teuthology.orchestra.run.smithi074.stdout: python3-kubernetes noarch 11.0.0-2.el8 epel 1.2 M 2020-08-07T15:04:08.924 INFO:teuthology.orchestra.run.smithi074.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2020-08-07T15:04:08.924 INFO:teuthology.orchestra.run.smithi074.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2020-08-07T15:04:08.924 INFO:teuthology.orchestra.run.smithi074.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2020-08-07T15:04:08.925 INFO:teuthology.orchestra.run.smithi074.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2020-08-07T15:04:08.925 INFO:teuthology.orchestra.run.smithi074.stdout: python3-rsa noarch 3.4.2-15.el8 epel 73 k 2020-08-07T15:04:08.925 INFO:teuthology.orchestra.run.smithi074.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2020-08-07T15:04:08.925 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:04:08.925 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction Summary 2020-08-07T15:04:08.936 INFO:teuthology.orchestra.run.smithi074.stdout:=========================================================================================================== 2020-08-07T15:04:08.937 INFO:teuthology.orchestra.run.smithi074.stdout:Install 13 Packages 2020-08-07T15:04:08.937 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:04:08.938 INFO:teuthology.orchestra.run.smithi074.stdout:Total download size: 1.9 M 2020-08-07T15:04:08.939 INFO:teuthology.orchestra.run.smithi074.stdout:Installed size: 24 M 2020-08-07T15:04:08.939 INFO:teuthology.orchestra.run.smithi074.stdout:Downloading Packages: 2020-08-07T15:04:08.983 INFO:teuthology.orchestra.run.smithi074.stdout:(1/13): python3-certifi-2018.10.15-7.el8.noarch 282 kB/s | 16 kB 00:00 2020-08-07T15:04:08.986 INFO:teuthology.orchestra.run.smithi074.stdout:(2/13): python3-cachetools-3.1.1-4.el8.noarch.r 566 kB/s | 33 kB 00:00 2020-08-07T15:04:09.032 INFO:teuthology.orchestra.run.smithi074.stdout:(3/13): python3-google-auth-1.1.1-10.el8.noarch 2.1 MB/s | 81 kB 00:00 2020-08-07T15:04:09.045 INFO:teuthology.orchestra.run.smithi074.stdout:(4/13): python3-rsa-3.4.2-15.el8.noarch.rpm 3.7 MB/s | 73 kB 00:00 2020-08-07T15:04:09.066 INFO:teuthology.orchestra.run.smithi074.stdout:(5/13): ceph-mgr-rook-16.0.0-3802.ge19d2f2a442. 212 kB/s | 27 kB 00:00 2020-08-07T15:04:09.066 INFO:teuthology.orchestra.run.smithi074.stdout:(6/13): python3-websocket-client-0.56.0-5.el8.n 2.6 MB/s | 61 kB 00:00 2020-08-07T15:04:09.094 INFO:teuthology.orchestra.run.smithi074.stdout:(7/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 661 kB/s | 27 kB 00:00 2020-08-07T15:04:09.101 INFO:teuthology.orchestra.run.smithi074.stdout:(8/13): python3-jsonpointer-1.10-11.el8.noarch. 534 kB/s | 20 kB 00:00 2020-08-07T15:04:09.123 INFO:teuthology.orchestra.run.smithi074.stdout:(9/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 4.9 MB/s | 126 kB 00:00 2020-08-07T15:04:09.134 INFO:teuthology.orchestra.run.smithi074.stdout:(10/13): python3-pyasn1-modules-0.3.7-6.el8.noa 4.2 MB/s | 110 kB 00:00 2020-08-07T15:04:09.143 INFO:teuthology.orchestra.run.smithi074.stdout:(11/13): python3-kubernetes-11.0.0-2.el8.noarch 7.5 MB/s | 1.2 MB 00:00 2020-08-07T15:04:09.143 INFO:teuthology.orchestra.run.smithi074.stdout:(12/13): python3-requests-oauthlib-1.0.0-1.el8. 2.0 MB/s | 43 kB 00:00 2020-08-07T15:04:09.162 INFO:teuthology.orchestra.run.smithi074.stdout:(13/13): python3-oauthlib-2.1.0-1.el8.noarch.rp 5.6 MB/s | 155 kB 00:00 2020-08-07T15:04:09.163 INFO:teuthology.orchestra.run.smithi074.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:04:09.163 INFO:teuthology.orchestra.run.smithi074.stdout:Total 8.3 MB/s | 1.9 MB 00:00 2020-08-07T15:04:09.163 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction check 2020-08-07T15:04:09.173 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction check succeeded. 2020-08-07T15:04:09.174 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction test 2020-08-07T15:04:09.220 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2020-08-07T15:04:09.241 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction test succeeded. 2020-08-07T15:04:09.241 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction 2020-08-07T15:04:09.398 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libquadmath-8.3.1-5.el8.x86_64 1/8 2020-08-07T15:04:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libquadmath-8.3.1-5.el8.x86_64 1/8 2020-08-07T15:04:09.740 INFO:teuthology.orchestra.run.smithi074.stdout: Preparing : 1/1 2020-08-07T15:04:09.806 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libgfortran-8.3.1-5.el8.x86_64 2/8 2020-08-07T15:04:09.971 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2020-08-07T15:04:10.314 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libgfortran-8.3.1-5.el8.x86_64 2/8 2020-08-07T15:04:10.434 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-rsa-3.4.2-15.el8.noarch 2/13 2020-08-07T15:04:10.486 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-threads-0.3.3-5.el8.x86_64 3/8 2020-08-07T15:04:10.682 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/13 2020-08-07T15:04:10.872 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 4/13 2020-08-07T15:04:10.992 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: openblas-threads-0.3.3-5.el8.x86_64 3/8 2020-08-07T15:04:11.043 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 5/13 2020-08-07T15:04:11.193 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-0.3.3-5.el8.x86_64 4/8 2020-08-07T15:04:11.246 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 6/13 2020-08-07T15:04:11.427 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 7/13 2020-08-07T15:04:11.715 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 8/13 2020-08-07T15:04:11.918 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: openblas-0.3.3-5.el8.x86_64 4/8 2020-08-07T15:04:11.943 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 9/13 2020-08-07T15:04:12.173 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 10/13 2020-08-07T15:04:12.298 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 5/8 2020-08-07T15:04:12.518 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2020-08-07T15:04:13.350 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 11/13 2020-08-07T15:04:13.657 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-kubernetes-11.0.0-2.el8.noarch 12/13 2020-08-07T15:04:13.814 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : ceph-mgr-rook-2:16.0.0-3802.ge19d2f2a442.el8.noarc 13/13 2020-08-07T15:04:14.035 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: ceph-mgr-rook-2:16.0.0-3802.ge19d2f2a442.el8.noarc 13/13 2020-08-07T15:04:14.035 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : ceph-mgr-rook-2:16.0.0-3802.ge19d2f2a442.el8.noarc 1/13 2020-08-07T15:04:14.035 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/13 2020-08-07T15:04:14.036 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/13 2020-08-07T15:04:14.036 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/13 2020-08-07T15:04:14.036 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-kubernetes-11.0.0-2.el8.noarch 5/13 2020-08-07T15:04:14.036 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-rsa-3.4.2-15.el8.noarch 6/13 2020-08-07T15:04:14.036 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/13 2020-08-07T15:04:14.037 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 8/13 2020-08-07T15:04:14.037 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 9/13 2020-08-07T15:04:14.048 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 10/13 2020-08-07T15:04:14.049 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 11/13 2020-08-07T15:04:14.049 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 12/13 2020-08-07T15:04:14.378 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 13/13 2020-08-07T15:04:14.378 INFO:teuthology.orchestra.run.smithi074.stdout:Installed products updated. 2020-08-07T15:04:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2020-08-07T15:04:15.894 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 7/8 2020-08-07T15:04:16.118 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-diskprediction-local-2:16.0.0-3802.ge19d2f2 8/8 2020-08-07T15:04:16.369 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.0.0-3802.ge19d2f2 8/8 2020-08-07T15:04:16.369 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.0.0-3802.ge19d2f2 1/8 2020-08-07T15:04:16.370 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 2/8 2020-08-07T15:04:16.370 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 3/8 2020-08-07T15:04:16.370 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 4/8 2020-08-07T15:04:16.370 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-threads-0.3.3-5.el8.x86_64 5/8 2020-08-07T15:04:16.371 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-0.3.3-5.el8.x86_64 6/8 2020-08-07T15:04:16.371 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libquadmath-8.3.1-5.el8.x86_64 7/8 2020-08-07T15:04:16.371 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:04:16.371 INFO:teuthology.orchestra.run.smithi074.stdout:Installed: 2020-08-07T15:04:16.371 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-mgr-rook-2:16.0.0-3802.ge19d2f2a442.el8.noarch 2020-08-07T15:04:16.372 INFO:teuthology.orchestra.run.smithi074.stdout: python3-cachetools-3.1.1-4.el8.noarch 2020-08-07T15:04:16.372 INFO:teuthology.orchestra.run.smithi074.stdout: python3-certifi-2018.10.15-7.el8.noarch 2020-08-07T15:04:16.372 INFO:teuthology.orchestra.run.smithi074.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2020-08-07T15:04:16.372 INFO:teuthology.orchestra.run.smithi074.stdout: python3-jsonpatch-1.21-2.el8.noarch 2020-08-07T15:04:16.372 INFO:teuthology.orchestra.run.smithi074.stdout: python3-jsonpointer-1.10-11.el8.noarch 2020-08-07T15:04:16.373 INFO:teuthology.orchestra.run.smithi074.stdout: python3-kubernetes-11.0.0-2.el8.noarch 2020-08-07T15:04:16.373 INFO:teuthology.orchestra.run.smithi074.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2020-08-07T15:04:16.373 INFO:teuthology.orchestra.run.smithi074.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2020-08-07T15:04:16.373 INFO:teuthology.orchestra.run.smithi074.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2020-08-07T15:04:16.373 INFO:teuthology.orchestra.run.smithi074.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2020-08-07T15:04:16.382 INFO:teuthology.orchestra.run.smithi074.stdout: python3-rsa-3.4.2-15.el8.noarch 2020-08-07T15:04:16.382 INFO:teuthology.orchestra.run.smithi074.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2020-08-07T15:04:16.382 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:04:16.383 INFO:teuthology.orchestra.run.smithi074.stdout:Complete! 2020-08-07T15:04:16.543 INFO:teuthology.orchestra.run.smithi074:> sudo yum -y install ceph-mgr-cephadm 2020-08-07T15:04:16.867 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libgfortran-8.3.1-5.el8.x86_64 8/8 2020-08-07T15:04:16.868 INFO:teuthology.orchestra.run.smithi096.stdout:Installed products updated. 2020-08-07T15:04:16.902 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T15:04:17.390 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 99 kB/s | 2.8 kB 00:00 2020-08-07T15:04:17.487 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 83 kB/s | 2.4 kB 00:00 2020-08-07T15:04:18.525 INFO:teuthology.orchestra.run.smithi074.stdout:Dependencies resolved. 2020-08-07T15:04:18.526 INFO:teuthology.orchestra.run.smithi074.stdout:================================================================================================== 2020-08-07T15:04:18.526 INFO:teuthology.orchestra.run.smithi074.stdout: Package Arch Version Repository Size 2020-08-07T15:04:18.527 INFO:teuthology.orchestra.run.smithi074.stdout:================================================================================================== 2020-08-07T15:04:18.527 INFO:teuthology.orchestra.run.smithi074.stdout:Installing: 2020-08-07T15:04:18.527 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-mgr-cephadm noarch 2:16.0.0-3802.ge19d2f2a442.el8 ceph-noarch 56 k 2020-08-07T15:04:18.527 INFO:teuthology.orchestra.run.smithi074.stdout:Installing dependencies: 2020-08-07T15:04:18.528 INFO:teuthology.orchestra.run.smithi074.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2020-08-07T15:04:18.528 INFO:teuthology.orchestra.run.smithi074.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2020-08-07T15:04:18.528 INFO:teuthology.orchestra.run.smithi074.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2020-08-07T15:04:18.528 INFO:teuthology.orchestra.run.smithi074.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2020-08-07T15:04:18.528 INFO:teuthology.orchestra.run.smithi074.stdout: python3-remoto noarch 1.1.4-4.el8 epel 48 k 2020-08-07T15:04:18.528 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:04:18.529 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction Summary 2020-08-07T15:04:18.529 INFO:teuthology.orchestra.run.smithi074.stdout:================================================================================================== 2020-08-07T15:04:18.529 INFO:teuthology.orchestra.run.smithi074.stdout:Install 6 Packages 2020-08-07T15:04:18.529 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:04:18.531 INFO:teuthology.orchestra.run.smithi074.stdout:Total download size: 5.8 M 2020-08-07T15:04:18.531 INFO:teuthology.orchestra.run.smithi074.stdout:Installed size: 24 M 2020-08-07T15:04:18.531 INFO:teuthology.orchestra.run.smithi074.stdout:Downloading Packages: 2020-08-07T15:04:18.576 INFO:teuthology.orchestra.run.smithi074.stdout:(1/6): python3-apipkg-1.5-6.el8.noarch.rpm 449 kB/s | 21 kB 00:00 2020-08-07T15:04:18.609 INFO:teuthology.orchestra.run.smithi074.stdout:(2/6): python3-remoto-1.1.4-4.el8.noarch.rpm 1.5 MB/s | 48 kB 00:00 2020-08-07T15:04:18.624 INFO:teuthology.orchestra.run.smithi074.stdout:(3/6): python3-execnet-1.7.1-1.el8.noarch.rpm 3.7 MB/s | 355 kB 00:00 2020-08-07T15:04:18.646 INFO:teuthology.orchestra.run.smithi074.stdout:(4/6): python3-jinja2-2.10.1-2.el8_0.noarch.rpm 14 MB/s | 538 kB 00:00 2020-08-07T15:04:18.688 INFO:teuthology.orchestra.run.smithi074.stdout:(5/6): python3-babel-2.5.1-5.el8.noarch.rpm 81 MB/s | 4.8 MB 00:00 2020-08-07T15:04:18.688 INFO:teuthology.orchestra.run.smithi074.stdout:(6/6): ceph-mgr-cephadm-16.0.0-3802.ge19d2f2a44 359 kB/s | 56 kB 00:00 2020-08-07T15:04:18.688 INFO:teuthology.orchestra.run.smithi074.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:04:18.689 INFO:teuthology.orchestra.run.smithi074.stdout:Total 37 MB/s | 5.8 MB 00:00 2020-08-07T15:04:18.701 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction check 2020-08-07T15:04:18.706 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction check succeeded. 2020-08-07T15:04:18.706 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction test 2020-08-07T15:04:18.785 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction test succeeded. 2020-08-07T15:04:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:18.786 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2020-08-07T15:04:18.787 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local-2:16.0.0-3802.ge19d2f2a442.el8.noarch 2020-08-07T15:04:18.787 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran-8.3.1-5.el8.x86_64 2020-08-07T15:04:18.787 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath-8.3.1-5.el8.x86_64 2020-08-07T15:04:18.787 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-0.3.3-5.el8.x86_64 2020-08-07T15:04:18.788 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-threads-0.3.3-5.el8.x86_64 2020-08-07T15:04:18.788 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2020-08-07T15:04:18.788 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2020-08-07T15:04:18.789 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2020-08-07T15:04:18.789 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:18.789 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2020-08-07T15:04:18.791 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction 2020-08-07T15:04:19.039 INFO:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-mgr-rook 2020-08-07T15:04:19.386 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T15:04:19.783 INFO:teuthology.orchestra.run.smithi074.stdout: Preparing : 1/1 2020-08-07T15:04:19.891 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 92 kB/s | 2.8 kB 00:00 2020-08-07T15:04:19.984 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 78 kB/s | 2.4 kB 00:00 2020-08-07T15:04:20.164 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 1/6 2020-08-07T15:04:20.544 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 2/6 2020-08-07T15:04:20.817 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 3/6 2020-08-07T15:04:21.018 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2020-08-07T15:04:21.020 INFO:teuthology.orchestra.run.smithi096.stdout:=========================================================================================================== 2020-08-07T15:04:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2020-08-07T15:04:21.021 INFO:teuthology.orchestra.run.smithi096.stdout:=========================================================================================================== 2020-08-07T15:04:21.021 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2020-08-07T15:04:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook noarch 2:16.0.0-3802.ge19d2f2a442.el8 ceph-noarch 27 k 2020-08-07T15:04:21.022 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2020-08-07T15:04:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2020-08-07T15:04:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2020-08-07T15:04:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2020-08-07T15:04:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2020-08-07T15:04:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2020-08-07T15:04:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes noarch 11.0.0-2.el8 epel 1.2 M 2020-08-07T15:04:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2020-08-07T15:04:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2020-08-07T15:04:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2020-08-07T15:04:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2020-08-07T15:04:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa noarch 3.4.2-15.el8 epel 73 k 2020-08-07T15:04:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2020-08-07T15:04:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:21.028 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2020-08-07T15:04:21.028 INFO:teuthology.orchestra.run.smithi096.stdout:=========================================================================================================== 2020-08-07T15:04:21.029 INFO:teuthology.orchestra.run.smithi096.stdout:Install 13 Packages 2020-08-07T15:04:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:21.032 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 1.9 M 2020-08-07T15:04:21.032 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 24 M 2020-08-07T15:04:21.032 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2020-08-07T15:04:21.070 INFO:teuthology.orchestra.run.smithi096.stdout:(1/13): python3-certifi-2018.10.15-7.el8.noarch 348 kB/s | 16 kB 00:00 2020-08-07T15:04:21.080 INFO:teuthology.orchestra.run.smithi096.stdout:(2/13): python3-cachetools-3.1.1-4.el8.noarch.r 581 kB/s | 33 kB 00:00 2020-08-07T15:04:21.082 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 4/6 2020-08-07T15:04:21.117 INFO:teuthology.orchestra.run.smithi096.stdout:(3/13): python3-google-auth-1.1.1-10.el8.noarch 1.7 MB/s | 81 kB 00:00 2020-08-07T15:04:21.133 INFO:teuthology.orchestra.run.smithi096.stdout:(4/13): python3-rsa-3.4.2-15.el8.noarch.rpm 4.4 MB/s | 73 kB 00:00 2020-08-07T15:04:21.149 INFO:teuthology.orchestra.run.smithi096.stdout:(5/13): python3-websocket-client-0.56.0-5.el8.n 3.8 MB/s | 61 kB 00:00 2020-08-07T15:04:21.151 INFO:teuthology.orchestra.run.smithi096.stdout:(6/13): ceph-mgr-rook-16.0.0-3802.ge19d2f2a442. 212 kB/s | 27 kB 00:00 2020-08-07T15:04:21.189 INFO:teuthology.orchestra.run.smithi096.stdout:(7/13): python3-jsonpointer-1.10-11.el8.noarch. 529 kB/s | 20 kB 00:00 2020-08-07T15:04:21.194 INFO:teuthology.orchestra.run.smithi096.stdout:(8/13): python3-kubernetes-11.0.0-2.el8.noarch. 10 MB/s | 1.2 MB 00:00 2020-08-07T15:04:21.202 INFO:teuthology.orchestra.run.smithi096.stdout:(9/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 526 kB/s | 27 kB 00:00 2020-08-07T15:04:21.227 INFO:teuthology.orchestra.run.smithi096.stdout:(10/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 3.4 MB/s | 126 kB 00:00 2020-08-07T15:04:21.231 INFO:teuthology.orchestra.run.smithi096.stdout:(11/13): python3-requests-oauthlib-1.0.0-1.el8. 1.3 MB/s | 43 kB 00:00 2020-08-07T15:04:21.238 INFO:teuthology.orchestra.run.smithi096.stdout:(12/13): python3-pyasn1-modules-0.3.7-6.el8.noa 2.8 MB/s | 110 kB 00:00 2020-08-07T15:04:21.250 INFO:teuthology.orchestra.run.smithi096.stdout:(13/13): python3-oauthlib-2.1.0-1.el8.noarch.rp 6.0 MB/s | 155 kB 00:00 2020-08-07T15:04:21.251 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:04:21.251 INFO:teuthology.orchestra.run.smithi096.stdout:Total 8.4 MB/s | 1.9 MB 00:00 2020-08-07T15:04:21.256 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2020-08-07T15:04:21.270 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2020-08-07T15:04:21.272 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2020-08-07T15:04:21.334 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2020-08-07T15:04:21.337 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2020-08-07T15:04:21.354 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : python3-remoto-1.1.4-4.el8.noarch 5/6 2020-08-07T15:04:21.486 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : ceph-mgr-cephadm-2:16.0.0-3802.ge19d2f2a442.el8.noar 6/6 2020-08-07T15:04:21.705 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.0.0-3802.ge19d2f2a442.el8.noar 6/6 2020-08-07T15:04:21.705 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : ceph-mgr-cephadm-2:16.0.0-3802.ge19d2f2a442.el8.noar 1/6 2020-08-07T15:04:21.706 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 2/6 2020-08-07T15:04:21.706 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 3/6 2020-08-07T15:04:21.706 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-remoto-1.1.4-4.el8.noarch 4/6 2020-08-07T15:04:21.706 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 5/6 2020-08-07T15:04:21.845 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2020-08-07T15:04:21.971 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 6/6 2020-08-07T15:04:21.972 INFO:teuthology.orchestra.run.smithi074.stdout:Installed products updated. 2020-08-07T15:04:22.077 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2020-08-07T15:04:22.354 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rsa-3.4.2-15.el8.noarch 2/13 2020-08-07T15:04:22.631 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/13 2020-08-07T15:04:22.880 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 4/13 2020-08-07T15:04:23.089 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 5/13 2020-08-07T15:04:23.299 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 6/13 2020-08-07T15:04:23.512 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 7/13 2020-08-07T15:04:23.770 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 8/13 2020-08-07T15:04:23.918 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:04:23.919 INFO:teuthology.orchestra.run.smithi074.stdout:Installed: 2020-08-07T15:04:23.919 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-mgr-cephadm-2:16.0.0-3802.ge19d2f2a442.el8.noarch 2020-08-07T15:04:23.919 INFO:teuthology.orchestra.run.smithi074.stdout: python3-apipkg-1.5-6.el8.noarch 2020-08-07T15:04:23.920 INFO:teuthology.orchestra.run.smithi074.stdout: python3-babel-2.5.1-5.el8.noarch 2020-08-07T15:04:23.920 INFO:teuthology.orchestra.run.smithi074.stdout: python3-execnet-1.7.1-1.el8.noarch 2020-08-07T15:04:23.920 INFO:teuthology.orchestra.run.smithi074.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2020-08-07T15:04:23.920 INFO:teuthology.orchestra.run.smithi074.stdout: python3-remoto-1.1.4-4.el8.noarch 2020-08-07T15:04:23.920 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:04:23.921 INFO:teuthology.orchestra.run.smithi074.stdout:Complete! 2020-08-07T15:04:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 9/13 2020-08-07T15:04:24.063 INFO:teuthology.orchestra.run.smithi074:> sudo yum -y install ceph-fuse 2020-08-07T15:04:24.261 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 10/13 2020-08-07T15:04:24.420 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T15:04:24.914 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 95 kB/s | 2.8 kB 00:00 2020-08-07T15:04:25.009 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 83 kB/s | 2.4 kB 00:00 2020-08-07T15:04:25.421 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 11/13 2020-08-07T15:04:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-kubernetes-11.0.0-2.el8.noarch 12/13 2020-08-07T15:04:25.909 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-rook-2:16.0.0-3802.ge19d2f2a442.el8.noarc 13/13 2020-08-07T15:04:26.010 INFO:teuthology.orchestra.run.smithi074.stdout:Dependencies resolved. 2020-08-07T15:04:26.011 INFO:teuthology.orchestra.run.smithi074.stdout:========================================================================================== 2020-08-07T15:04:26.011 INFO:teuthology.orchestra.run.smithi074.stdout: Package Arch Version Repository Size 2020-08-07T15:04:26.011 INFO:teuthology.orchestra.run.smithi074.stdout:========================================================================================== 2020-08-07T15:04:26.011 INFO:teuthology.orchestra.run.smithi074.stdout:Installing: 2020-08-07T15:04:26.011 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-fuse x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 604 k 2020-08-07T15:04:26.012 INFO:teuthology.orchestra.run.smithi074.stdout:Installing dependencies: 2020-08-07T15:04:26.012 INFO:teuthology.orchestra.run.smithi074.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2020-08-07T15:04:26.012 INFO:teuthology.orchestra.run.smithi074.stdout: fuse-common x86_64 3.2.1-12.el8 rhel-8-for-x86_64-baseos-rpms 21 k 2020-08-07T15:04:26.012 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:04:26.012 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction Summary 2020-08-07T15:04:26.013 INFO:teuthology.orchestra.run.smithi074.stdout:========================================================================================== 2020-08-07T15:04:26.013 INFO:teuthology.orchestra.run.smithi074.stdout:Install 3 Packages 2020-08-07T15:04:26.013 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:04:26.014 INFO:teuthology.orchestra.run.smithi074.stdout:Total download size: 708 k 2020-08-07T15:04:26.014 INFO:teuthology.orchestra.run.smithi074.stdout:Installed size: 2.2 M 2020-08-07T15:04:26.014 INFO:teuthology.orchestra.run.smithi074.stdout:Downloading Packages: 2020-08-07T15:04:26.056 INFO:teuthology.orchestra.run.smithi074.stdout:(1/3): fuse-2.9.7-12.el8.x86_64.rpm 1.8 MB/s | 83 kB 00:00 2020-08-07T15:04:26.059 INFO:teuthology.orchestra.run.smithi074.stdout:(2/3): fuse-common-3.2.1-12.el8.x86_64.rpm 449 kB/s | 21 kB 00:00 2020-08-07T15:04:26.133 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-rook-2:16.0.0-3802.ge19d2f2a442.el8.noarc 13/13 2020-08-07T15:04:26.134 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-rook-2:16.0.0-3802.ge19d2f2a442.el8.noarc 1/13 2020-08-07T15:04:26.134 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/13 2020-08-07T15:04:26.135 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/13 2020-08-07T15:04:26.135 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/13 2020-08-07T15:04:26.135 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-kubernetes-11.0.0-2.el8.noarch 5/13 2020-08-07T15:04:26.135 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rsa-3.4.2-15.el8.noarch 6/13 2020-08-07T15:04:26.136 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/13 2020-08-07T15:04:26.136 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 8/13 2020-08-07T15:04:26.136 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 9/13 2020-08-07T15:04:26.136 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 10/13 2020-08-07T15:04:26.137 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 11/13 2020-08-07T15:04:26.137 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 12/13 2020-08-07T15:04:26.239 INFO:teuthology.orchestra.run.smithi074.stdout:(3/3): ceph-fuse-16.0.0-3802.ge19d2f2a442.el8.x 2.6 MB/s | 604 kB 00:00 2020-08-07T15:04:26.240 INFO:teuthology.orchestra.run.smithi074.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:04:26.240 INFO:teuthology.orchestra.run.smithi074.stdout:Total 3.0 MB/s | 708 kB 00:00 2020-08-07T15:04:26.250 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction check 2020-08-07T15:04:26.266 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction check succeeded. 2020-08-07T15:04:26.266 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction test 2020-08-07T15:04:26.334 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction test succeeded. 2020-08-07T15:04:26.335 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction 2020-08-07T15:04:26.460 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 13/13 2020-08-07T15:04:26.461 INFO:teuthology.orchestra.run.smithi096.stdout:Installed products updated. 2020-08-07T15:04:26.562 INFO:teuthology.orchestra.run.smithi074.stdout: Preparing : 1/1 2020-08-07T15:04:26.563 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : fuse-common-3.2.1-12.el8.x86_64 1/3 2020-08-07T15:04:26.563 INFO:teuthology.orchestra.run.smithi074.stdout:warning: /etc/fuse.conf created as /etc/fuse.conf.rpmnew 2020-08-07T15:04:26.563 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:04:26.978 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : fuse-2.9.7-12.el8.x86_64 2/3 2020-08-07T15:04:27.196 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : ceph-fuse-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 3/3 2020-08-07T15:04:27.511 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: ceph-fuse-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 3/3 2020-08-07T15:04:27.512 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : ceph-fuse-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/3 2020-08-07T15:04:27.512 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 2/3 2020-08-07T15:04:27.794 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : fuse-common-3.2.1-12.el8.x86_64 3/3 2020-08-07T15:04:27.794 INFO:teuthology.orchestra.run.smithi074.stdout:Installed products updated. 2020-08-07T15:04:28.335 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:28.335 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2020-08-07T15:04:28.336 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook-2:16.0.0-3802.ge19d2f2a442.el8.noarch 2020-08-07T15:04:28.336 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools-3.1.1-4.el8.noarch 2020-08-07T15:04:28.336 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi-2018.10.15-7.el8.noarch 2020-08-07T15:04:28.337 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2020-08-07T15:04:28.337 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch-1.21-2.el8.noarch 2020-08-07T15:04:28.337 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer-1.10-11.el8.noarch 2020-08-07T15:04:28.346 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes-11.0.0-2.el8.noarch 2020-08-07T15:04:28.346 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2020-08-07T15:04:28.346 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2020-08-07T15:04:28.347 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2020-08-07T15:04:28.347 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2020-08-07T15:04:28.347 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa-3.4.2-15.el8.noarch 2020-08-07T15:04:28.348 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2020-08-07T15:04:28.348 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:28.354 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2020-08-07T15:04:28.587 INFO:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-mgr-cephadm 2020-08-07T15:04:28.942 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T15:04:29.451 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 96 kB/s | 2.8 kB 00:00 2020-08-07T15:04:29.550 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 81 kB/s | 2.4 kB 00:00 2020-08-07T15:04:29.861 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:04:29.862 INFO:teuthology.orchestra.run.smithi074.stdout:Installed: 2020-08-07T15:04:29.868 INFO:teuthology.orchestra.run.smithi074.stdout: ceph-fuse-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 fuse-2.9.7-12.el8.x86_64 2020-08-07T15:04:29.869 INFO:teuthology.orchestra.run.smithi074.stdout: fuse-common-3.2.1-12.el8.x86_64 2020-08-07T15:04:29.869 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:04:29.869 INFO:teuthology.orchestra.run.smithi074.stdout:Complete! 2020-08-07T15:04:30.026 INFO:teuthology.orchestra.run.smithi074:> sudo yum -y install librados-devel 2020-08-07T15:04:30.388 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T15:04:30.591 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2020-08-07T15:04:30.591 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================================== 2020-08-07T15:04:30.592 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2020-08-07T15:04:30.592 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================================== 2020-08-07T15:04:30.592 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2020-08-07T15:04:30.592 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm noarch 2:16.0.0-3802.ge19d2f2a442.el8 ceph-noarch 56 k 2020-08-07T15:04:30.592 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2020-08-07T15:04:30.593 INFO:teuthology.orchestra.run.smithi096.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2020-08-07T15:04:30.593 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2020-08-07T15:04:30.593 INFO:teuthology.orchestra.run.smithi096.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2020-08-07T15:04:30.593 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2020-08-07T15:04:30.594 INFO:teuthology.orchestra.run.smithi096.stdout: python3-remoto noarch 1.1.4-4.el8 epel 48 k 2020-08-07T15:04:30.594 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:30.594 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2020-08-07T15:04:30.594 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================================== 2020-08-07T15:04:30.594 INFO:teuthology.orchestra.run.smithi096.stdout:Install 6 Packages 2020-08-07T15:04:30.594 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:30.595 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 5.8 M 2020-08-07T15:04:30.598 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 24 M 2020-08-07T15:04:30.598 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2020-08-07T15:04:30.640 INFO:teuthology.orchestra.run.smithi096.stdout:(1/6): python3-apipkg-1.5-6.el8.noarch.rpm 452 kB/s | 21 kB 00:00 2020-08-07T15:04:30.673 INFO:teuthology.orchestra.run.smithi096.stdout:(2/6): python3-remoto-1.1.4-4.el8.noarch.rpm 1.5 MB/s | 48 kB 00:00 2020-08-07T15:04:30.709 INFO:teuthology.orchestra.run.smithi096.stdout:(3/6): python3-jinja2-2.10.1-2.el8_0.noarch.rpm 15 MB/s | 538 kB 00:00 2020-08-07T15:04:30.709 INFO:teuthology.orchestra.run.smithi096.stdout:(4/6): python3-execnet-1.7.1-1.el8.noarch.rpm 3.0 MB/s | 355 kB 00:00 2020-08-07T15:04:30.758 INFO:teuthology.orchestra.run.smithi096.stdout:(5/6): python3-babel-2.5.1-5.el8.noarch.rpm 97 MB/s | 4.8 MB 00:00 2020-08-07T15:04:30.759 INFO:teuthology.orchestra.run.smithi096.stdout:(6/6): ceph-mgr-cephadm-16.0.0-3802.ge19d2f2a44 342 kB/s | 56 kB 00:00 2020-08-07T15:04:30.759 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:04:30.759 INFO:teuthology.orchestra.run.smithi096.stdout:Total 35 MB/s | 5.8 MB 00:00 2020-08-07T15:04:30.761 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2020-08-07T15:04:30.770 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2020-08-07T15:04:30.771 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2020-08-07T15:04:30.859 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2020-08-07T15:04:30.862 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2020-08-07T15:04:30.982 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 99 kB/s | 2.8 kB 00:00 2020-08-07T15:04:31.078 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 80 kB/s | 2.4 kB 00:00 2020-08-07T15:04:31.896 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2020-08-07T15:04:32.073 INFO:teuthology.orchestra.run.smithi074.stdout:Dependencies resolved. 2020-08-07T15:04:32.075 INFO:teuthology.orchestra.run.smithi074.stdout:================================================================================ 2020-08-07T15:04:32.076 INFO:teuthology.orchestra.run.smithi074.stdout: Package Arch Version Repo Size 2020-08-07T15:04:32.076 INFO:teuthology.orchestra.run.smithi074.stdout:================================================================================ 2020-08-07T15:04:32.076 INFO:teuthology.orchestra.run.smithi074.stdout:Installing: 2020-08-07T15:04:32.076 INFO:teuthology.orchestra.run.smithi074.stdout: librados-devel x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 131 k 2020-08-07T15:04:32.076 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:04:32.077 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction Summary 2020-08-07T15:04:32.077 INFO:teuthology.orchestra.run.smithi074.stdout:================================================================================ 2020-08-07T15:04:32.077 INFO:teuthology.orchestra.run.smithi074.stdout:Install 1 Package 2020-08-07T15:04:32.079 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:04:32.079 INFO:teuthology.orchestra.run.smithi074.stdout:Total download size: 131 k 2020-08-07T15:04:32.079 INFO:teuthology.orchestra.run.smithi074.stdout:Installed size: 436 k 2020-08-07T15:04:32.080 INFO:teuthology.orchestra.run.smithi074.stdout:Downloading Packages: 2020-08-07T15:04:32.249 INFO:teuthology.orchestra.run.smithi074.stdout:librados-devel-16.0.0-3802.ge19d2f2a442.el8.x86 749 kB/s | 131 kB 00:00 2020-08-07T15:04:32.250 INFO:teuthology.orchestra.run.smithi074.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:04:32.251 INFO:teuthology.orchestra.run.smithi074.stdout:Total 744 kB/s | 131 kB 00:00 2020-08-07T15:04:32.253 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction check 2020-08-07T15:04:32.258 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction check succeeded. 2020-08-07T15:04:32.258 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction test 2020-08-07T15:04:32.303 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 1/6 2020-08-07T15:04:32.308 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction test succeeded. 2020-08-07T15:04:32.311 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction 2020-08-07T15:04:32.427 INFO:teuthology.orchestra.run.smithi074.stdout: Preparing : 1/1 2020-08-07T15:04:32.600 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : librados-devel-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:04:32.821 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: librados-devel-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:04:33.066 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : librados-devel-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:04:33.067 INFO:teuthology.orchestra.run.smithi074.stdout:Installed products updated. 2020-08-07T15:04:33.113 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 2/6 2020-08-07T15:04:33.408 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 3/6 2020-08-07T15:04:33.712 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 4/6 2020-08-07T15:04:34.015 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-remoto-1.1.4-4.el8.noarch 5/6 2020-08-07T15:04:34.246 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-cephadm-2:16.0.0-3802.ge19d2f2a442.el8.noar 6/6 2020-08-07T15:04:34.497 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.0.0-3802.ge19d2f2a442.el8.noar 6/6 2020-08-07T15:04:34.497 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-cephadm-2:16.0.0-3802.ge19d2f2a442.el8.noar 1/6 2020-08-07T15:04:34.497 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 2/6 2020-08-07T15:04:34.498 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 3/6 2020-08-07T15:04:34.498 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-remoto-1.1.4-4.el8.noarch 4/6 2020-08-07T15:04:34.498 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 5/6 2020-08-07T15:04:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 6/6 2020-08-07T15:04:34.795 INFO:teuthology.orchestra.run.smithi096.stdout:Installed products updated. 2020-08-07T15:04:34.999 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:04:35.000 INFO:teuthology.orchestra.run.smithi074.stdout:Installed: 2020-08-07T15:04:35.000 INFO:teuthology.orchestra.run.smithi074.stdout: librados-devel-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:04:35.001 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:04:35.001 INFO:teuthology.orchestra.run.smithi074.stdout:Complete! 2020-08-07T15:04:35.158 INFO:teuthology.orchestra.run.smithi074:> sudo yum -y install libcephfs2 2020-08-07T15:04:35.518 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T15:04:35.999 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 98 kB/s | 2.8 kB 00:00 2020-08-07T15:04:36.101 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 83 kB/s | 2.4 kB 00:00 2020-08-07T15:04:36.682 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:36.684 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2020-08-07T15:04:36.684 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm-2:16.0.0-3802.ge19d2f2a442.el8.noarch 2020-08-07T15:04:36.685 INFO:teuthology.orchestra.run.smithi096.stdout: python3-apipkg-1.5-6.el8.noarch 2020-08-07T15:04:36.685 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel-2.5.1-5.el8.noarch 2020-08-07T15:04:36.685 INFO:teuthology.orchestra.run.smithi096.stdout: python3-execnet-1.7.1-1.el8.noarch 2020-08-07T15:04:36.686 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2020-08-07T15:04:36.686 INFO:teuthology.orchestra.run.smithi096.stdout: python3-remoto-1.1.4-4.el8.noarch 2020-08-07T15:04:36.686 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:36.687 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2020-08-07T15:04:36.897 INFO:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-fuse 2020-08-07T15:04:37.251 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T15:04:37.607 INFO:teuthology.orchestra.run.smithi074.stdout:Package libcephfs2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 is already installed. 2020-08-07T15:04:37.761 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 94 kB/s | 2.8 kB 00:00 2020-08-07T15:04:37.844 INFO:teuthology.orchestra.run.smithi074.stdout:Dependencies resolved. 2020-08-07T15:04:37.844 INFO:teuthology.orchestra.run.smithi074.stdout:Nothing to do. 2020-08-07T15:04:37.845 INFO:teuthology.orchestra.run.smithi074.stdout:Complete! 2020-08-07T15:04:37.859 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 86 kB/s | 2.4 kB 00:00 2020-08-07T15:04:37.980 INFO:teuthology.orchestra.run.smithi074:> sudo yum -y install libcephfs-devel 2020-08-07T15:04:38.333 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T15:04:38.838 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 93 kB/s | 2.8 kB 00:00 2020-08-07T15:04:38.923 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2020-08-07T15:04:38.924 INFO:teuthology.orchestra.run.smithi096.stdout:========================================================================================== 2020-08-07T15:04:38.924 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2020-08-07T15:04:38.924 INFO:teuthology.orchestra.run.smithi096.stdout:========================================================================================== 2020-08-07T15:04:38.924 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2020-08-07T15:04:38.924 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 604 k 2020-08-07T15:04:38.925 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2020-08-07T15:04:38.925 INFO:teuthology.orchestra.run.smithi096.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2020-08-07T15:04:38.925 INFO:teuthology.orchestra.run.smithi096.stdout: fuse-common x86_64 3.2.1-12.el8 rhel-8-for-x86_64-baseos-rpms 21 k 2020-08-07T15:04:38.925 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:38.925 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2020-08-07T15:04:38.926 INFO:teuthology.orchestra.run.smithi096.stdout:========================================================================================== 2020-08-07T15:04:38.926 INFO:teuthology.orchestra.run.smithi096.stdout:Install 3 Packages 2020-08-07T15:04:38.926 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:38.926 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 708 k 2020-08-07T15:04:38.926 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 2.2 M 2020-08-07T15:04:38.927 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2020-08-07T15:04:38.942 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 83 kB/s | 2.4 kB 00:00 2020-08-07T15:04:38.966 INFO:teuthology.orchestra.run.smithi096.stdout:(1/3): fuse-2.9.7-12.el8.x86_64.rpm 2.0 MB/s | 83 kB 00:00 2020-08-07T15:04:38.970 INFO:teuthology.orchestra.run.smithi096.stdout:(2/3): fuse-common-3.2.1-12.el8.x86_64.rpm 473 kB/s | 21 kB 00:00 2020-08-07T15:04:39.152 INFO:teuthology.orchestra.run.smithi096.stdout:(3/3): ceph-fuse-16.0.0-3802.ge19d2f2a442.el8.x 2.6 MB/s | 604 kB 00:00 2020-08-07T15:04:39.153 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:04:39.153 INFO:teuthology.orchestra.run.smithi096.stdout:Total 3.0 MB/s | 708 kB 00:00 2020-08-07T15:04:39.153 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2020-08-07T15:04:39.175 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2020-08-07T15:04:39.176 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2020-08-07T15:04:39.242 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2020-08-07T15:04:39.244 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2020-08-07T15:04:39.477 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2020-08-07T15:04:39.477 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : fuse-common-3.2.1-12.el8.x86_64 1/3 2020-08-07T15:04:39.477 INFO:teuthology.orchestra.run.smithi096.stdout:warning: /etc/fuse.conf created as /etc/fuse.conf.rpmnew 2020-08-07T15:04:39.477 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:39.914 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : fuse-2.9.7-12.el8.x86_64 2/3 2020-08-07T15:04:39.944 INFO:teuthology.orchestra.run.smithi074.stdout:Dependencies resolved. 2020-08-07T15:04:39.944 INFO:teuthology.orchestra.run.smithi074.stdout:================================================================================ 2020-08-07T15:04:39.945 INFO:teuthology.orchestra.run.smithi074.stdout: Package Arch Version Repo Size 2020-08-07T15:04:39.945 INFO:teuthology.orchestra.run.smithi074.stdout:================================================================================ 2020-08-07T15:04:39.945 INFO:teuthology.orchestra.run.smithi074.stdout:Installing: 2020-08-07T15:04:39.945 INFO:teuthology.orchestra.run.smithi074.stdout: libcephfs-devel x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 23 k 2020-08-07T15:04:39.946 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:04:39.946 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction Summary 2020-08-07T15:04:39.946 INFO:teuthology.orchestra.run.smithi074.stdout:================================================================================ 2020-08-07T15:04:39.946 INFO:teuthology.orchestra.run.smithi074.stdout:Install 1 Package 2020-08-07T15:04:39.946 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:04:39.947 INFO:teuthology.orchestra.run.smithi074.stdout:Total download size: 23 k 2020-08-07T15:04:39.947 INFO:teuthology.orchestra.run.smithi074.stdout:Installed size: 84 k 2020-08-07T15:04:39.947 INFO:teuthology.orchestra.run.smithi074.stdout:Downloading Packages: 2020-08-07T15:04:40.076 INFO:teuthology.orchestra.run.smithi074.stdout:libcephfs-devel-16.0.0-3802.ge19d2f2a442.el8.x8 183 kB/s | 23 kB 00:00 2020-08-07T15:04:40.077 INFO:teuthology.orchestra.run.smithi074.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:04:40.077 INFO:teuthology.orchestra.run.smithi074.stdout:Total 181 kB/s | 23 kB 00:00 2020-08-07T15:04:40.077 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction check 2020-08-07T15:04:40.078 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction check succeeded. 2020-08-07T15:04:40.078 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction test 2020-08-07T15:04:40.082 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction test succeeded. 2020-08-07T15:04:40.083 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction 2020-08-07T15:04:40.130 INFO:teuthology.orchestra.run.smithi074.stdout: Preparing : 1/1 2020-08-07T15:04:40.158 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-fuse-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 3/3 2020-08-07T15:04:40.383 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : libcephfs-devel-2:16.0.0-3802.ge19d2f2a442.el8.x86_6 1/1 2020-08-07T15:04:40.487 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-fuse-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 3/3 2020-08-07T15:04:40.488 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-fuse-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/3 2020-08-07T15:04:40.488 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 2/3 2020-08-07T15:04:40.583 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: libcephfs-devel-2:16.0.0-3802.ge19d2f2a442.el8.x86_6 1/1 2020-08-07T15:04:40.740 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : fuse-common-3.2.1-12.el8.x86_64 3/3 2020-08-07T15:04:40.740 INFO:teuthology.orchestra.run.smithi096.stdout:Installed products updated. 2020-08-07T15:04:40.798 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : libcephfs-devel-2:16.0.0-3802.ge19d2f2a442.el8.x86_6 1/1 2020-08-07T15:04:40.799 INFO:teuthology.orchestra.run.smithi074.stdout:Installed products updated. 2020-08-07T15:04:42.776 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:04:42.777 INFO:teuthology.orchestra.run.smithi074.stdout:Installed: 2020-08-07T15:04:42.777 INFO:teuthology.orchestra.run.smithi074.stdout: libcephfs-devel-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:04:42.777 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:04:42.777 INFO:teuthology.orchestra.run.smithi074.stdout:Complete! 2020-08-07T15:04:42.817 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:42.818 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2020-08-07T15:04:42.818 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 fuse-2.9.7-12.el8.x86_64 2020-08-07T15:04:42.818 INFO:teuthology.orchestra.run.smithi096.stdout: fuse-common-3.2.1-12.el8.x86_64 2020-08-07T15:04:42.818 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:42.819 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2020-08-07T15:04:42.927 INFO:teuthology.orchestra.run.smithi074:> sudo yum -y install librados2 2020-08-07T15:04:43.032 INFO:teuthology.orchestra.run.smithi096:> sudo yum -y install librados-devel 2020-08-07T15:04:43.283 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T15:04:43.396 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T15:04:43.767 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 103 kB/s | 2.8 kB 00:00 2020-08-07T15:04:43.856 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 83 kB/s | 2.4 kB 00:00 2020-08-07T15:04:43.894 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 101 kB/s | 2.8 kB 00:00 2020-08-07T15:04:43.995 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 86 kB/s | 2.4 kB 00:00 2020-08-07T15:04:44.626 INFO:teuthology.orchestra.run.smithi074.stdout:Package librados2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 is already installed. 2020-08-07T15:04:44.870 INFO:teuthology.orchestra.run.smithi074.stdout:Dependencies resolved. 2020-08-07T15:04:44.870 INFO:teuthology.orchestra.run.smithi074.stdout:Nothing to do. 2020-08-07T15:04:44.871 INFO:teuthology.orchestra.run.smithi074.stdout:Complete! 2020-08-07T15:04:45.007 INFO:teuthology.orchestra.run.smithi074:> sudo yum -y install librbd1 2020-08-07T15:04:45.257 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2020-08-07T15:04:45.257 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2020-08-07T15:04:45.258 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repo Size 2020-08-07T15:04:45.258 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2020-08-07T15:04:45.258 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2020-08-07T15:04:45.258 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 131 k 2020-08-07T15:04:45.259 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:45.259 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2020-08-07T15:04:45.259 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2020-08-07T15:04:45.259 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2020-08-07T15:04:45.260 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:45.260 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 131 k 2020-08-07T15:04:45.260 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 436 k 2020-08-07T15:04:45.260 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2020-08-07T15:04:45.374 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T15:04:45.434 INFO:teuthology.orchestra.run.smithi096.stdout:librados-devel-16.0.0-3802.ge19d2f2a442.el8.x86 739 kB/s | 131 kB 00:00 2020-08-07T15:04:45.435 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:04:45.436 INFO:teuthology.orchestra.run.smithi096.stdout:Total 734 kB/s | 131 kB 00:00 2020-08-07T15:04:45.436 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2020-08-07T15:04:45.448 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2020-08-07T15:04:45.448 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2020-08-07T15:04:45.494 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2020-08-07T15:04:45.497 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2020-08-07T15:04:45.652 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2020-08-07T15:04:45.850 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librados-devel-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:04:45.865 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 96 kB/s | 2.8 kB 00:00 2020-08-07T15:04:45.966 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 79 kB/s | 2.4 kB 00:00 2020-08-07T15:04:46.086 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados-devel-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:04:46.304 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados-devel-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:04:46.304 INFO:teuthology.orchestra.run.smithi096.stdout:Installed products updated. 2020-08-07T15:04:46.733 INFO:teuthology.orchestra.run.smithi074.stdout:Package librbd1-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 is already installed. 2020-08-07T15:04:46.967 INFO:teuthology.orchestra.run.smithi074.stdout:Dependencies resolved. 2020-08-07T15:04:46.967 INFO:teuthology.orchestra.run.smithi074.stdout:Nothing to do. 2020-08-07T15:04:46.968 INFO:teuthology.orchestra.run.smithi074.stdout:Complete! 2020-08-07T15:04:47.107 INFO:teuthology.orchestra.run.smithi074:> sudo yum -y install python3-rados 2020-08-07T15:04:47.473 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T15:04:47.980 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 97 kB/s | 2.8 kB 00:00 2020-08-07T15:04:48.080 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 84 kB/s | 2.4 kB 00:00 2020-08-07T15:04:48.358 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:48.358 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2020-08-07T15:04:48.358 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:04:48.359 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:48.359 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2020-08-07T15:04:48.568 INFO:teuthology.orchestra.run.smithi096:> sudo yum -y install libcephfs2 2020-08-07T15:04:48.848 INFO:teuthology.orchestra.run.smithi074.stdout:Package python3-rados-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 is already installed. 2020-08-07T15:04:48.926 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T15:04:49.093 INFO:teuthology.orchestra.run.smithi074.stdout:Dependencies resolved. 2020-08-07T15:04:49.094 INFO:teuthology.orchestra.run.smithi074.stdout:Nothing to do. 2020-08-07T15:04:49.094 INFO:teuthology.orchestra.run.smithi074.stdout:Complete! 2020-08-07T15:04:49.231 INFO:teuthology.orchestra.run.smithi074:> sudo yum -y install python3-rgw 2020-08-07T15:04:49.414 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 103 kB/s | 2.8 kB 00:00 2020-08-07T15:04:49.510 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 88 kB/s | 2.4 kB 00:00 2020-08-07T15:04:49.586 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T15:04:50.087 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 98 kB/s | 2.8 kB 00:00 2020-08-07T15:04:50.190 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 79 kB/s | 2.4 kB 00:00 2020-08-07T15:04:50.535 INFO:teuthology.orchestra.run.smithi096.stdout:Package libcephfs2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 is already installed. 2020-08-07T15:04:50.783 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2020-08-07T15:04:50.783 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2020-08-07T15:04:50.783 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2020-08-07T15:04:50.956 INFO:teuthology.orchestra.run.smithi074.stdout:Package python3-rgw-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 is already installed. 2020-08-07T15:04:50.972 INFO:teuthology.orchestra.run.smithi096:> sudo yum -y install libcephfs-devel 2020-08-07T15:04:51.196 INFO:teuthology.orchestra.run.smithi074.stdout:Dependencies resolved. 2020-08-07T15:04:51.197 INFO:teuthology.orchestra.run.smithi074.stdout:Nothing to do. 2020-08-07T15:04:51.198 INFO:teuthology.orchestra.run.smithi074.stdout:Complete! 2020-08-07T15:04:51.330 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T15:04:51.337 INFO:teuthology.orchestra.run.smithi074:> sudo yum -y install python3-cephfs 2020-08-07T15:04:51.697 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T15:04:51.992 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 99 kB/s | 2.8 kB 00:00 2020-08-07T15:04:52.089 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 80 kB/s | 2.4 kB 00:00 2020-08-07T15:04:52.196 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 96 kB/s | 2.8 kB 00:00 2020-08-07T15:04:52.289 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 82 kB/s | 2.4 kB 00:00 2020-08-07T15:04:53.063 INFO:teuthology.orchestra.run.smithi074.stdout:Package python3-cephfs-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 is already installed. 2020-08-07T15:04:53.138 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2020-08-07T15:04:53.138 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2020-08-07T15:04:53.139 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repo Size 2020-08-07T15:04:53.139 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2020-08-07T15:04:53.139 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2020-08-07T15:04:53.139 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 23 k 2020-08-07T15:04:53.140 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:53.140 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2020-08-07T15:04:53.140 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2020-08-07T15:04:53.140 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2020-08-07T15:04:53.140 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:53.146 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 23 k 2020-08-07T15:04:53.146 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 84 k 2020-08-07T15:04:53.146 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2020-08-07T15:04:53.269 INFO:teuthology.orchestra.run.smithi096.stdout:libcephfs-devel-16.0.0-3802.ge19d2f2a442.el8.x8 182 kB/s | 23 kB 00:00 2020-08-07T15:04:53.269 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:04:53.269 INFO:teuthology.orchestra.run.smithi096.stdout:Total 180 kB/s | 23 kB 00:00 2020-08-07T15:04:53.270 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2020-08-07T15:04:53.272 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2020-08-07T15:04:53.272 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2020-08-07T15:04:53.274 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2020-08-07T15:04:53.277 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2020-08-07T15:04:53.304 INFO:teuthology.orchestra.run.smithi074.stdout:Dependencies resolved. 2020-08-07T15:04:53.314 INFO:teuthology.orchestra.run.smithi074.stdout:Nothing to do. 2020-08-07T15:04:53.315 INFO:teuthology.orchestra.run.smithi074.stdout:Complete! 2020-08-07T15:04:53.332 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2020-08-07T15:04:53.544 INFO:teuthology.orchestra.run.smithi074:> sudo yum -y install python3-rbd 2020-08-07T15:04:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs-devel-2:16.0.0-3802.ge19d2f2a442.el8.x86_6 1/1 2020-08-07T15:04:53.799 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephfs-devel-2:16.0.0-3802.ge19d2f2a442.el8.x86_6 1/1 2020-08-07T15:04:53.899 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T15:04:54.029 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs-devel-2:16.0.0-3802.ge19d2f2a442.el8.x86_6 1/1 2020-08-07T15:04:54.030 INFO:teuthology.orchestra.run.smithi096.stdout:Installed products updated. 2020-08-07T15:04:54.412 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 101 kB/s | 2.8 kB 00:00 2020-08-07T15:04:54.506 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 91 kB/s | 2.4 kB 00:00 2020-08-07T15:04:55.272 INFO:teuthology.orchestra.run.smithi074.stdout:Package python3-rbd-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 is already installed. 2020-08-07T15:04:55.518 INFO:teuthology.orchestra.run.smithi074.stdout:Dependencies resolved. 2020-08-07T15:04:55.519 INFO:teuthology.orchestra.run.smithi074.stdout:Nothing to do. 2020-08-07T15:04:55.519 INFO:teuthology.orchestra.run.smithi074.stdout:Complete! 2020-08-07T15:04:55.661 INFO:teuthology.orchestra.run.smithi074:> sudo yum -y install rbd-fuse 2020-08-07T15:04:55.970 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:55.971 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2020-08-07T15:04:55.971 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:04:55.971 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:04:55.971 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2020-08-07T15:04:56.017 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T15:04:56.158 INFO:teuthology.orchestra.run.smithi096:> sudo yum -y install librados2 2020-08-07T15:04:56.515 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 99 kB/s | 2.8 kB 00:00 2020-08-07T15:04:56.537 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T15:04:56.612 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 77 kB/s | 2.4 kB 00:00 2020-08-07T15:04:57.044 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 95 kB/s | 2.8 kB 00:00 2020-08-07T15:04:57.148 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 85 kB/s | 2.4 kB 00:00 2020-08-07T15:04:57.644 INFO:teuthology.orchestra.run.smithi074.stdout:Dependencies resolved. 2020-08-07T15:04:57.645 INFO:teuthology.orchestra.run.smithi074.stdout:================================================================================ 2020-08-07T15:04:57.645 INFO:teuthology.orchestra.run.smithi074.stdout: Package Arch Version Repo Size 2020-08-07T15:04:57.646 INFO:teuthology.orchestra.run.smithi074.stdout:================================================================================ 2020-08-07T15:04:57.646 INFO:teuthology.orchestra.run.smithi074.stdout:Installing: 2020-08-07T15:04:57.646 INFO:teuthology.orchestra.run.smithi074.stdout: rbd-fuse x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 87 k 2020-08-07T15:04:57.646 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:04:57.647 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction Summary 2020-08-07T15:04:57.647 INFO:teuthology.orchestra.run.smithi074.stdout:================================================================================ 2020-08-07T15:04:57.647 INFO:teuthology.orchestra.run.smithi074.stdout:Install 1 Package 2020-08-07T15:04:57.647 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:04:57.647 INFO:teuthology.orchestra.run.smithi074.stdout:Total download size: 87 k 2020-08-07T15:04:57.648 INFO:teuthology.orchestra.run.smithi074.stdout:Installed size: 296 k 2020-08-07T15:04:57.648 INFO:teuthology.orchestra.run.smithi074.stdout:Downloading Packages: 2020-08-07T15:04:57.798 INFO:teuthology.orchestra.run.smithi074.stdout:rbd-fuse-16.0.0-3802.ge19d2f2a442.el8.x86_64.rp 574 kB/s | 87 kB 00:00 2020-08-07T15:04:57.799 INFO:teuthology.orchestra.run.smithi074.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:04:57.799 INFO:teuthology.orchestra.run.smithi074.stdout:Total 570 kB/s | 87 kB 00:00 2020-08-07T15:04:57.799 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction check 2020-08-07T15:04:57.820 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction check succeeded. 2020-08-07T15:04:57.820 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction test 2020-08-07T15:04:57.863 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction test succeeded. 2020-08-07T15:04:57.864 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction 2020-08-07T15:04:57.911 INFO:teuthology.orchestra.run.smithi096.stdout:Package librados2-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 is already installed. 2020-08-07T15:04:57.966 INFO:teuthology.orchestra.run.smithi074.stdout: Preparing : 1/1 2020-08-07T15:04:58.117 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : rbd-fuse-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:04:58.155 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2020-08-07T15:04:58.156 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2020-08-07T15:04:58.156 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2020-08-07T15:04:58.344 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: rbd-fuse-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:04:58.378 INFO:teuthology.orchestra.run.smithi096:> sudo yum -y install librbd1 2020-08-07T15:04:58.580 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : rbd-fuse-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:04:58.580 INFO:teuthology.orchestra.run.smithi074.stdout:Installed products updated. 2020-08-07T15:04:58.734 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T15:04:59.238 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 103 kB/s | 2.8 kB 00:00 2020-08-07T15:04:59.358 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 85 kB/s | 2.4 kB 00:00 2020-08-07T15:05:00.138 INFO:teuthology.orchestra.run.smithi096.stdout:Package librbd1-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 is already installed. 2020-08-07T15:05:00.379 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2020-08-07T15:05:00.380 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2020-08-07T15:05:00.380 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2020-08-07T15:05:00.503 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:00.504 INFO:teuthology.orchestra.run.smithi074.stdout:Installed: 2020-08-07T15:05:00.504 INFO:teuthology.orchestra.run.smithi074.stdout: rbd-fuse-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:05:00.504 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:00.504 INFO:teuthology.orchestra.run.smithi074.stdout:Complete! 2020-08-07T15:05:00.547 INFO:teuthology.orchestra.run.smithi096:> sudo yum -y install python3-rados 2020-08-07T15:05:00.659 INFO:teuthology.orchestra.run.smithi074:> sudo yum -y install rbd-mirror 2020-08-07T15:05:00.900 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T15:05:01.022 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T15:05:01.391 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 101 kB/s | 2.8 kB 00:00 2020-08-07T15:05:01.489 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 86 kB/s | 2.4 kB 00:00 2020-08-07T15:05:01.521 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 100 kB/s | 2.8 kB 00:00 2020-08-07T15:05:01.617 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 83 kB/s | 2.4 kB 00:00 2020-08-07T15:05:02.293 INFO:teuthology.orchestra.run.smithi096.stdout:Package python3-rados-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 is already installed. 2020-08-07T15:05:02.538 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2020-08-07T15:05:02.539 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2020-08-07T15:05:02.539 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2020-08-07T15:05:02.667 INFO:teuthology.orchestra.run.smithi074.stdout:Dependencies resolved. 2020-08-07T15:05:02.668 INFO:teuthology.orchestra.run.smithi074.stdout:================================================================================ 2020-08-07T15:05:02.668 INFO:teuthology.orchestra.run.smithi074.stdout: Package Arch Version Repo Size 2020-08-07T15:05:02.668 INFO:teuthology.orchestra.run.smithi074.stdout:================================================================================ 2020-08-07T15:05:02.669 INFO:teuthology.orchestra.run.smithi074.stdout:Installing: 2020-08-07T15:05:02.669 INFO:teuthology.orchestra.run.smithi074.stdout: rbd-mirror x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 3.1 M 2020-08-07T15:05:02.669 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:02.669 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction Summary 2020-08-07T15:05:02.669 INFO:teuthology.orchestra.run.smithi074.stdout:================================================================================ 2020-08-07T15:05:02.671 INFO:teuthology.orchestra.run.smithi074.stdout:Install 1 Package 2020-08-07T15:05:02.671 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:02.671 INFO:teuthology.orchestra.run.smithi074.stdout:Total download size: 3.1 M 2020-08-07T15:05:02.672 INFO:teuthology.orchestra.run.smithi074.stdout:Installed size: 14 M 2020-08-07T15:05:02.672 INFO:teuthology.orchestra.run.smithi074.stdout:Downloading Packages: 2020-08-07T15:05:02.707 INFO:teuthology.orchestra.run.smithi096:> sudo yum -y install python3-rgw 2020-08-07T15:05:03.058 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T15:05:03.570 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 96 kB/s | 2.8 kB 00:00 2020-08-07T15:05:03.670 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 82 kB/s | 2.4 kB 00:00 2020-08-07T15:05:04.201 INFO:teuthology.orchestra.run.smithi074.stdout:rbd-mirror-16.0.0-3802.ge19d2f2a442.el8.x86_64. 2.0 MB/s | 3.1 MB 00:01 2020-08-07T15:05:04.202 INFO:teuthology.orchestra.run.smithi074.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:05:04.202 INFO:teuthology.orchestra.run.smithi074.stdout:Total 2.0 MB/s | 3.1 MB 00:01 2020-08-07T15:05:04.203 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction check 2020-08-07T15:05:04.216 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction check succeeded. 2020-08-07T15:05:04.217 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction test 2020-08-07T15:05:04.283 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction test succeeded. 2020-08-07T15:05:04.287 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction 2020-08-07T15:05:04.444 INFO:teuthology.orchestra.run.smithi096.stdout:Package python3-rgw-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 is already installed. 2020-08-07T15:05:04.688 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2020-08-07T15:05:04.689 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2020-08-07T15:05:04.689 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2020-08-07T15:05:04.690 INFO:teuthology.orchestra.run.smithi074.stdout: Preparing : 1/1 2020-08-07T15:05:04.849 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : rbd-mirror-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:05:04.885 INFO:teuthology.orchestra.run.smithi096:> sudo yum -y install python3-cephfs 2020-08-07T15:05:05.186 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: rbd-mirror-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:05:05.238 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T15:05:05.425 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : rbd-mirror-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:05:05.426 INFO:teuthology.orchestra.run.smithi074.stdout:Installed products updated. 2020-08-07T15:05:05.742 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 105 kB/s | 2.8 kB 00:00 2020-08-07T15:05:05.842 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 89 kB/s | 2.4 kB 00:00 2020-08-07T15:05:06.855 INFO:teuthology.orchestra.run.smithi096.stdout:Package python3-cephfs-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 is already installed. 2020-08-07T15:05:07.092 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2020-08-07T15:05:07.093 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2020-08-07T15:05:07.093 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2020-08-07T15:05:07.279 INFO:teuthology.orchestra.run.smithi096:> sudo yum -y install python3-rbd 2020-08-07T15:05:07.453 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:07.454 INFO:teuthology.orchestra.run.smithi074.stdout:Installed: 2020-08-07T15:05:07.454 INFO:teuthology.orchestra.run.smithi074.stdout: rbd-mirror-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:05:07.454 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:07.454 INFO:teuthology.orchestra.run.smithi074.stdout:Complete! 2020-08-07T15:05:07.610 INFO:teuthology.orchestra.run.smithi074:> sudo yum -y install rbd-nbd 2020-08-07T15:05:07.640 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T15:05:07.966 INFO:teuthology.orchestra.run.smithi074.stdout:Updating Subscription Management repositories. 2020-08-07T15:05:08.106 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 96 kB/s | 2.8 kB 00:00 2020-08-07T15:05:08.202 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 83 kB/s | 2.4 kB 00:00 2020-08-07T15:05:08.482 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 97 kB/s | 2.8 kB 00:00 2020-08-07T15:05:08.590 INFO:teuthology.orchestra.run.smithi074.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 84 kB/s | 2.4 kB 00:00 2020-08-07T15:05:08.978 INFO:teuthology.orchestra.run.smithi096.stdout:Package python3-rbd-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 is already installed. 2020-08-07T15:05:09.219 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2020-08-07T15:05:09.220 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2020-08-07T15:05:09.220 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2020-08-07T15:05:09.393 INFO:teuthology.orchestra.run.smithi096:> sudo yum -y install rbd-fuse 2020-08-07T15:05:09.700 INFO:teuthology.orchestra.run.smithi074.stdout:Dependencies resolved. 2020-08-07T15:05:09.701 INFO:teuthology.orchestra.run.smithi074.stdout:================================================================================ 2020-08-07T15:05:09.701 INFO:teuthology.orchestra.run.smithi074.stdout: Package Arch Version Repo Size 2020-08-07T15:05:09.701 INFO:teuthology.orchestra.run.smithi074.stdout:================================================================================ 2020-08-07T15:05:09.701 INFO:teuthology.orchestra.run.smithi074.stdout:Installing: 2020-08-07T15:05:09.702 INFO:teuthology.orchestra.run.smithi074.stdout: rbd-nbd x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 161 k 2020-08-07T15:05:09.702 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:09.702 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction Summary 2020-08-07T15:05:09.702 INFO:teuthology.orchestra.run.smithi074.stdout:================================================================================ 2020-08-07T15:05:09.703 INFO:teuthology.orchestra.run.smithi074.stdout:Install 1 Package 2020-08-07T15:05:09.703 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:09.703 INFO:teuthology.orchestra.run.smithi074.stdout:Total download size: 161 k 2020-08-07T15:05:09.703 INFO:teuthology.orchestra.run.smithi074.stdout:Installed size: 608 k 2020-08-07T15:05:09.703 INFO:teuthology.orchestra.run.smithi074.stdout:Downloading Packages: 2020-08-07T15:05:09.748 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T15:05:09.892 INFO:teuthology.orchestra.run.smithi074.stdout:rbd-nbd-16.0.0-3802.ge19d2f2a442.el8.x86_64.rpm 840 kB/s | 161 kB 00:00 2020-08-07T15:05:09.893 INFO:teuthology.orchestra.run.smithi074.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:05:09.893 INFO:teuthology.orchestra.run.smithi074.stdout:Total 834 kB/s | 161 kB 00:00 2020-08-07T15:05:09.894 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction check 2020-08-07T15:05:09.905 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction check succeeded. 2020-08-07T15:05:09.906 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction test 2020-08-07T15:05:09.957 INFO:teuthology.orchestra.run.smithi074.stdout:Transaction test succeeded. 2020-08-07T15:05:09.966 INFO:teuthology.orchestra.run.smithi074.stdout:Running transaction 2020-08-07T15:05:10.078 INFO:teuthology.orchestra.run.smithi074.stdout: Preparing : 1/1 2020-08-07T15:05:10.257 INFO:teuthology.orchestra.run.smithi074.stdout: Installing : rbd-nbd-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:05:10.269 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 99 kB/s | 2.8 kB 00:00 2020-08-07T15:05:10.378 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 77 kB/s | 2.4 kB 00:00 2020-08-07T15:05:10.467 INFO:teuthology.orchestra.run.smithi074.stdout: Running scriptlet: rbd-nbd-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:05:10.719 INFO:teuthology.orchestra.run.smithi074.stdout: Verifying : rbd-nbd-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:05:10.720 INFO:teuthology.orchestra.run.smithi074.stdout:Installed products updated. 2020-08-07T15:05:11.401 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2020-08-07T15:05:11.402 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2020-08-07T15:05:11.402 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repo Size 2020-08-07T15:05:11.403 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2020-08-07T15:05:11.403 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2020-08-07T15:05:11.403 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 87 k 2020-08-07T15:05:11.403 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:11.404 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2020-08-07T15:05:11.404 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2020-08-07T15:05:11.404 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2020-08-07T15:05:11.404 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:11.404 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 87 k 2020-08-07T15:05:11.405 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 296 k 2020-08-07T15:05:11.405 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2020-08-07T15:05:11.555 INFO:teuthology.orchestra.run.smithi096.stdout:rbd-fuse-16.0.0-3802.ge19d2f2a442.el8.x86_64.rp 579 kB/s | 87 kB 00:00 2020-08-07T15:05:11.555 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:05:11.555 INFO:teuthology.orchestra.run.smithi096.stdout:Total 574 kB/s | 87 kB 00:00 2020-08-07T15:05:11.555 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2020-08-07T15:05:11.581 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2020-08-07T15:05:11.582 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2020-08-07T15:05:11.618 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2020-08-07T15:05:11.647 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2020-08-07T15:05:11.726 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2020-08-07T15:05:12.017 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-fuse-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:05:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: rbd-fuse-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:05:12.562 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-fuse-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:05:12.563 INFO:teuthology.orchestra.run.smithi096.stdout:Installed products updated. 2020-08-07T15:05:12.690 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:12.691 INFO:teuthology.orchestra.run.smithi074.stdout:Installed: 2020-08-07T15:05:12.691 INFO:teuthology.orchestra.run.smithi074.stdout: rbd-nbd-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:05:12.691 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:12.691 INFO:teuthology.orchestra.run.smithi074.stdout:Complete! 2020-08-07T15:05:12.844 DEBUG:teuthology.parallel:result is None 2020-08-07T15:05:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:14.595 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2020-08-07T15:05:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:05:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:14.596 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2020-08-07T15:05:14.816 INFO:teuthology.orchestra.run.smithi096:> sudo yum -y install rbd-mirror 2020-08-07T15:05:15.177 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T15:05:15.659 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 97 kB/s | 2.8 kB 00:00 2020-08-07T15:05:15.754 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 84 kB/s | 2.4 kB 00:00 2020-08-07T15:05:16.803 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2020-08-07T15:05:16.803 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2020-08-07T15:05:16.804 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repo Size 2020-08-07T15:05:16.804 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2020-08-07T15:05:16.804 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2020-08-07T15:05:16.805 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 3.1 M 2020-08-07T15:05:16.805 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:16.805 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2020-08-07T15:05:16.808 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2020-08-07T15:05:16.809 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2020-08-07T15:05:16.809 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:16.810 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 3.1 M 2020-08-07T15:05:16.810 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 14 M 2020-08-07T15:05:16.810 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2020-08-07T15:05:17.393 INFO:teuthology.orchestra.run.smithi096.stdout:rbd-mirror-16.0.0-3802.ge19d2f2a442.el8.x86_64. 5.2 MB/s | 3.1 MB 00:00 2020-08-07T15:05:17.394 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:05:17.394 INFO:teuthology.orchestra.run.smithi096.stdout:Total 5.2 MB/s | 3.1 MB 00:00 2020-08-07T15:05:17.406 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2020-08-07T15:05:17.407 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2020-08-07T15:05:17.407 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2020-08-07T15:05:17.468 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2020-08-07T15:05:17.469 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2020-08-07T15:05:17.910 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2020-08-07T15:05:18.120 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-mirror-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:05:18.480 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: rbd-mirror-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:05:18.714 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-mirror-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:05:18.715 INFO:teuthology.orchestra.run.smithi096.stdout:Installed products updated. 2020-08-07T15:05:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:20.747 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2020-08-07T15:05:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:05:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:20.747 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2020-08-07T15:05:20.950 INFO:teuthology.orchestra.run.smithi096:> sudo yum -y install rbd-nbd 2020-08-07T15:05:21.309 INFO:teuthology.orchestra.run.smithi096.stdout:Updating Subscription Management repositories. 2020-08-07T15:05:21.875 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 91 kB/s | 2.8 kB 00:00 2020-08-07T15:05:22.014 INFO:teuthology.orchestra.run.smithi096.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 83 kB/s | 2.4 kB 00:00 2020-08-07T15:05:23.060 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2020-08-07T15:05:23.061 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2020-08-07T15:05:23.061 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repo Size 2020-08-07T15:05:23.061 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2020-08-07T15:05:23.061 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2020-08-07T15:05:23.062 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd x86_64 2:16.0.0-3802.ge19d2f2a442.el8 ceph 161 k 2020-08-07T15:05:23.062 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:23.062 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2020-08-07T15:05:23.062 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2020-08-07T15:05:23.062 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2020-08-07T15:05:23.063 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:23.063 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 161 k 2020-08-07T15:05:23.063 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 608 k 2020-08-07T15:05:23.063 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2020-08-07T15:05:23.229 INFO:teuthology.orchestra.run.smithi096.stdout:rbd-nbd-16.0.0-3802.ge19d2f2a442.el8.x86_64.rpm 914 kB/s | 161 kB 00:00 2020-08-07T15:05:23.230 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2020-08-07T15:05:23.230 INFO:teuthology.orchestra.run.smithi096.stdout:Total 907 kB/s | 161 kB 00:00 2020-08-07T15:05:23.230 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2020-08-07T15:05:23.241 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2020-08-07T15:05:23.241 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2020-08-07T15:05:23.294 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2020-08-07T15:05:23.297 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2020-08-07T15:05:23.413 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2020-08-07T15:05:23.635 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-nbd-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:05:23.853 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: rbd-nbd-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:05:24.107 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-nbd-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 1/1 2020-08-07T15:05:24.107 INFO:teuthology.orchestra.run.smithi096.stdout:Installed products updated. 2020-08-07T15:05:26.206 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:26.207 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2020-08-07T15:05:26.207 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd-2:16.0.0-3802.ge19d2f2a442.el8.x86_64 2020-08-07T15:05:26.207 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:26.207 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2020-08-07T15:05:26.380 DEBUG:teuthology.parallel:result is None 2020-08-07T15:05:26.381 INFO:teuthology.orchestra.run.smithi074:> sudo lsb_release -is 2020-08-07T15:05:26.451 INFO:teuthology.orchestra.run.smithi074.stdout:RedHatEnterprise 2020-08-07T15:05:26.452 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2020-08-07T15:05:26.453 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d 2020-08-07T15:05:26.683 INFO:teuthology.orchestra.run.smithi074:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2020-08-07T15:05:26.709 INFO:teuthology.orchestra.run.smithi074.stdout:16.0.0-3802.ge19d2f2a442.el8 2020-08-07T15:05:26.710 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-3802.ge19d2f2a442.el8 2020-08-07T15:05:26.710 INFO:teuthology.task.install:The correct ceph version 16.0.0-3802.ge19d2f2a442 is installed. 2020-08-07T15:05:26.711 INFO:teuthology.orchestra.run.smithi096:> sudo lsb_release -is 2020-08-07T15:05:26.750 INFO:teuthology.orchestra.run.smithi096.stdout:RedHatEnterprise 2020-08-07T15:05:26.753 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2020-08-07T15:05:26.754 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d 2020-08-07T15:05:26.949 INFO:teuthology.orchestra.run.smithi096:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2020-08-07T15:05:26.974 INFO:teuthology.orchestra.run.smithi096.stdout:16.0.0-3802.ge19d2f2a442.el8 2020-08-07T15:05:26.976 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-3802.ge19d2f2a442.el8 2020-08-07T15:05:26.977 INFO:teuthology.task.install:The correct ceph version 16.0.0-3802.ge19d2f2a442 is installed. 2020-08-07T15:05:26.977 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2020-08-07T15:05:26.978 INFO:teuthology.orchestra.run.smithi074:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2020-08-07T15:05:27.035 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2020-08-07T15:05:27.077 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2020-08-07T15:05:27.078 INFO:teuthology.orchestra.run.smithi074:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2020-08-07T15:05:27.110 INFO:teuthology.orchestra.run.smithi074:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-08-07T15:05:27.180 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2020-08-07T15:05:27.211 INFO:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-08-07T15:05:27.295 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2020-08-07T15:05:27.295 INFO:teuthology.orchestra.run.smithi074:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2020-08-07T15:05:27.383 INFO:teuthology.orchestra.run.smithi074:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-08-07T15:05:27.459 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2020-08-07T15:05:27.506 INFO:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-08-07T15:05:27.575 INFO:teuthology.run_tasks:Running task ceph... 2020-08-07T15:05:27.708 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2020-08-07T15:05:27.709 INFO:teuthology.orchestra.run.smithi074:> sudo chmod 777 /var/log/ceph 2020-08-07T15:05:27.738 INFO:teuthology.orchestra.run.smithi096:> sudo chmod 777 /var/log/ceph 2020-08-07T15:05:27.770 INFO:tasks.ceph:Disabling ceph logrotate... 2020-08-07T15:05:27.771 INFO:teuthology.orchestra.run.smithi074:> sudo rm -f -- /etc/logrotate.d/ceph 2020-08-07T15:05:27.809 INFO:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/logrotate.d/ceph 2020-08-07T15:05:27.843 INFO:tasks.ceph:Creating extra log directories... 2020-08-07T15:05:27.843 INFO:teuthology.orchestra.run.smithi074:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-08-07T15:05:27.885 INFO:teuthology.orchestra.run.smithi096:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-08-07T15:05:27.914 INFO:tasks.ceph:Creating ceph cluster ceph... 2020-08-07T15:05:27.914 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'mon client directed command retry': 5, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_async_recovery_min_cost': 1, 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2020-08-07T15:05:27.915 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/shuzhenyi-2020-08-07_08:55:00-rados:thrash-wip-shuzhenyi-testing-2020-08-06-1028-distro-basic-smithi/5306571', 'branch': 'wip-shuzhenyi-testing-2020-08-06-1028', 'description': 'rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} d-balancer/on msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/radosbench-high-concurrency}', 'email': None, 'first_in_suite': False, 'job_id': '5306571', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'shuzhenyi-2020-08-07_08:55:00-rados:thrash-wip-shuzhenyi-testing-2020-08-06-1028-distro-basic-smithi', 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 30}}], 'os_type': 'rhel', 'os_version': '8.1', 'overrides': {'admin_socket': {'branch': 'wip-shuzhenyi-testing-2020-08-06-1028'}, 'ceph': {'conf': {'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'mon client directed command retry': 5, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_async_recovery_min_cost': 1, 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'fs': 'xfs', 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request'], 'sha1': 'e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'sha1': 'e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5, 'min_in': 4}, 'workunit': {'branch': 'wip-shuzhenyi-testing-2020-08-06-1028', 'sha1': 'e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d'}}, 'owner': 'scheduled_shuzhenyi@teuthology', 'priority': 50, 'repo': 'git://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'sha1': 'e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d', 'sleep_before_teardown': 0, 'suite': 'rados:thrash', 'suite_branch': 'wip-shuzhenyi-testing-2020-08-06-1028', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_wip-shuzhenyi-testing-2020-08-06-1028/qa', 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph-ci.git', 'suite_sha1': 'e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d', 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (2, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'mon client directed command retry': 5, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_async_recovery_min_cost': 1, 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}}, 'fs': 'xfs', 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request'], 'sha1': 'e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d', 'cluster': 'ceph'}}, {'thrashosds': {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200}}, {'full_sequential': [{'radosbench': {'clients': ['client.0'], 'concurrency': 128, 'size': 8192, 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'concurrency': 128, 'size': 8192, 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'concurrency': 128, 'size': 8192, 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'concurrency': 128, 'size': 8192, 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'concurrency': 128, 'size': 8192, 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'concurrency': 128, 'size': 8192, 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'concurrency': 128, 'size': 8192, 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'concurrency': 128, 'size': 8192, 'time': 90}}]}], 'teuthology_branch': 'master', 'timestamp': '2020-08-07_08:55:00', 'tube': 'smithi', 'user': 'shuzhenyi', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.1880', 'targets': {'smithi074.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDeEIcBdq9e99p4DguAmP1GKEvfUqLz0tkqIetjmuBti17VTcn35q803Ri5fdgeyBfDlfevb+lc5x1TeF3ixLNtfbVzJ7oVdXIDqX7SkyafDSq6vXodQQzlvJyI+bTAlh4od1VjtWLz+6rHxt8KDHVu1yY4/jEfzQI0e1BStfSmelISr67klfLdzYRXQkgNoITAj1lpZzb1KNZA9AvtW/JbG4a5wELNF3j4IWTZoXm8t8ANekDJ5b0HqfiF0fGh/sr0gY4dk5e7UjOHpoSzpCuFcyLDWBGHwQ+EdYgNOpN4vsY/inpmNUpdD5B9Cb1K5IrtAp8XZAPt1EUsm5EtX47t9ywuTwwyX4LVB/AkxtkOQFHSTfKFlIg91DFOuf6VdorJWIlOHgGko4s/hRGAMS9/k+flh3XBoXAxuI2Yt+/ZTwlqtnhnbtC20ppNkAnOx8/SZzkjrwW2qlw1SsGgNOLZvyvHXnMN9kb3pxIhthf8XJFGd511UpOstS2dWIeXo40=', 'smithi096.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCf4CtmyPbc9S5DEGTKwGREVs6f9MEaeOEDqh1p7IZ2+XpZIh2Jd3Kr5yEgldr6R4q6AsIXp2gLu73yJY+3SquSke1DPOC8rUeEpRPU79omJ4u3txXOty3D718RzboivgcEcdSNMdDcHqgOK7YV7ucK9clbS+tzvri1/rLCe8Wa6BXVVNOTR+bkcM3SoJg99uAgRU3JiQr3eGDgcwYeC+nspMhs5EfB+ipUuTejadpgTzERQqVPQ/lWgUWJVj3w3y0nuqpiu60cQL4OVVzZkQrCTCK+o+ct+BgzaftBaRDZC0gOT64gjCHknnZyGdIXYPJNnFQW2CZQR/Tn0Llx071ZkM4pjT1oDWX/fUPYo6G4Sc39JHlhBWToO+iREe0igUjt6P2sleuVwqSMuZmMEwUIJfQ98eeMLJGZeT+a9huNYzu5CkgWg1kJIzXmp/QRjjV1LQWe8Y7bcAat1ezOBrOq5QYKt+ECw9THtHEdt7WxZEIj/K9faAYmHUj9oAmurCU='}} 2020-08-07T15:05:27.916 INFO:teuthology.orchestra.run.smithi074:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-08-07T15:05:27.962 INFO:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-08-07T15:05:27.977 INFO:teuthology.orchestra.run.smithi074:> sudo install -d -m0777 -- /var/run/ceph 2020-08-07T15:05:28.016 INFO:teuthology.orchestra.run.smithi096:> sudo install -d -m0777 -- /var/run/ceph 2020-08-07T15:05:28.133 DEBUG:teuthology.orchestra.remote:smithi074:/scratch_devs is 72B 2020-08-07T15:05:28.188 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-08-07T15:05:28.191 INFO:teuthology.orchestra.run.smithi074:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-08-07T15:05:28.207 INFO:teuthology.orchestra.run.smithi074.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-08-07T15:05:28.207 INFO:teuthology.orchestra.run.smithi074.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-08-07T15:05:28.207 INFO:teuthology.orchestra.run.smithi074.stdout:Device: 6h/6d Inode: 99181 Links: 1 2020-08-07T15:05:28.208 INFO:teuthology.orchestra.run.smithi074.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-08-07T15:05:28.208 INFO:teuthology.orchestra.run.smithi074.stdout:Context: system_u:object_r:device_t:s0 2020-08-07T15:05:28.208 INFO:teuthology.orchestra.run.smithi074.stdout:Access: 2020-08-07 15:05:05.004318560 +0000 2020-08-07T15:05:28.208 INFO:teuthology.orchestra.run.smithi074.stdout:Modify: 2020-08-07 15:01:52.039909265 +0000 2020-08-07T15:05:28.209 INFO:teuthology.orchestra.run.smithi074.stdout:Change: 2020-08-07 15:01:52.039909265 +0000 2020-08-07T15:05:28.209 INFO:teuthology.orchestra.run.smithi074.stdout: Birth: - 2020-08-07T15:05:28.236 INFO:teuthology.orchestra.run.smithi074.stderr:1+0 records in 2020-08-07T15:05:28.236 INFO:teuthology.orchestra.run.smithi074.stderr:1+0 records out 2020-08-07T15:05:28.237 INFO:teuthology.orchestra.run.smithi074.stderr:512 bytes copied, 0.000261021 s, 2.0 MB/s 2020-08-07T15:05:28.237 INFO:teuthology.orchestra.run.smithi074:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-08-07T15:05:28.292 INFO:teuthology.orchestra.run.smithi074.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-08-07T15:05:28.293 INFO:teuthology.orchestra.run.smithi074.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-08-07T15:05:28.293 INFO:teuthology.orchestra.run.smithi074.stdout:Device: 6h/6d Inode: 103677 Links: 1 2020-08-07T15:05:28.294 INFO:teuthology.orchestra.run.smithi074.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-08-07T15:05:28.294 INFO:teuthology.orchestra.run.smithi074.stdout:Context: system_u:object_r:device_t:s0 2020-08-07T15:05:28.295 INFO:teuthology.orchestra.run.smithi074.stdout:Access: 2020-08-07 15:05:05.004318560 +0000 2020-08-07T15:05:28.295 INFO:teuthology.orchestra.run.smithi074.stdout:Modify: 2020-08-07 15:01:52.451900043 +0000 2020-08-07T15:05:28.295 INFO:teuthology.orchestra.run.smithi074.stdout:Change: 2020-08-07 15:01:52.451900043 +0000 2020-08-07T15:05:28.295 INFO:teuthology.orchestra.run.smithi074.stdout: Birth: - 2020-08-07T15:05:28.309 INFO:teuthology.orchestra.run.smithi074.stderr:1+0 records in 2020-08-07T15:05:28.309 INFO:teuthology.orchestra.run.smithi074.stderr:1+0 records out 2020-08-07T15:05:28.309 INFO:teuthology.orchestra.run.smithi074.stderr:512 bytes copied, 0.000210264 s, 2.4 MB/s 2020-08-07T15:05:28.310 INFO:teuthology.orchestra.run.smithi074:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-08-07T15:05:28.368 INFO:teuthology.orchestra.run.smithi074.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-08-07T15:05:28.368 INFO:teuthology.orchestra.run.smithi074.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-08-07T15:05:28.369 INFO:teuthology.orchestra.run.smithi074.stdout:Device: 6h/6d Inode: 99236 Links: 1 2020-08-07T15:05:28.369 INFO:teuthology.orchestra.run.smithi074.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-08-07T15:05:28.369 INFO:teuthology.orchestra.run.smithi074.stdout:Context: system_u:object_r:device_t:s0 2020-08-07T15:05:28.369 INFO:teuthology.orchestra.run.smithi074.stdout:Access: 2020-08-07 15:05:05.004318560 +0000 2020-08-07T15:05:28.390 INFO:teuthology.orchestra.run.smithi074.stdout:Modify: 2020-08-07 15:01:52.814891918 +0000 2020-08-07T15:05:28.390 INFO:teuthology.orchestra.run.smithi074.stdout:Change: 2020-08-07 15:01:52.814891918 +0000 2020-08-07T15:05:28.390 INFO:teuthology.orchestra.run.smithi074.stdout: Birth: - 2020-08-07T15:05:28.392 INFO:teuthology.orchestra.run.smithi074.stderr:1+0 records in 2020-08-07T15:05:28.404 INFO:teuthology.orchestra.run.smithi074.stderr:1+0 records out 2020-08-07T15:05:28.404 INFO:teuthology.orchestra.run.smithi074.stderr:512 bytes copied, 0.000215849 s, 2.4 MB/s 2020-08-07T15:05:28.404 INFO:teuthology.orchestra.run.smithi074:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-08-07T15:05:28.463 INFO:teuthology.orchestra.run.smithi074.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-08-07T15:05:28.463 INFO:teuthology.orchestra.run.smithi074.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-08-07T15:05:28.463 INFO:teuthology.orchestra.run.smithi074.stdout:Device: 6h/6d Inode: 104701 Links: 1 2020-08-07T15:05:28.464 INFO:teuthology.orchestra.run.smithi074.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-08-07T15:05:28.464 INFO:teuthology.orchestra.run.smithi074.stdout:Context: system_u:object_r:device_t:s0 2020-08-07T15:05:28.464 INFO:teuthology.orchestra.run.smithi074.stdout:Access: 2020-08-07 15:05:05.004318560 +0000 2020-08-07T15:05:28.464 INFO:teuthology.orchestra.run.smithi074.stdout:Modify: 2020-08-07 15:01:53.218882875 +0000 2020-08-07T15:05:28.464 INFO:teuthology.orchestra.run.smithi074.stdout:Change: 2020-08-07 15:01:53.218882875 +0000 2020-08-07T15:05:28.464 INFO:teuthology.orchestra.run.smithi074.stdout: Birth: - 2020-08-07T15:05:28.486 INFO:teuthology.orchestra.run.smithi074.stderr:1+0 records in 2020-08-07T15:05:28.486 INFO:teuthology.orchestra.run.smithi074.stderr:1+0 records out 2020-08-07T15:05:28.487 INFO:teuthology.orchestra.run.smithi074.stderr:512 bytes copied, 0.000167548 s, 3.1 MB/s 2020-08-07T15:05:28.487 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-08-07T15:05:28.533 DEBUG:teuthology.orchestra.remote:smithi096:/scratch_devs is 72B 2020-08-07T15:05:28.606 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-08-07T15:05:28.606 INFO:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-08-07T15:05:28.648 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2020-08-07T15:05:28.648 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2020-08-07T15:05:28.648 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000223164 s, 2.3 MB/s 2020-08-07T15:05:28.648 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-08-07T15:05:28.649 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-08-07T15:05:28.649 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 6h/6d Inode: 99268 Links: 1 2020-08-07T15:05:28.649 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-08-07T15:05:28.649 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2020-08-07T15:05:28.650 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2020-08-07 15:05:18.301203426 +0000 2020-08-07T15:05:28.650 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2020-08-07 15:01:52.040774699 +0000 2020-08-07T15:05:28.650 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2020-08-07 15:01:52.040774699 +0000 2020-08-07T15:05:28.650 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2020-08-07T15:05:28.650 INFO:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-08-07T15:05:28.705 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-08-07T15:05:28.713 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-08-07T15:05:28.713 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 6h/6d Inode: 101800 Links: 1 2020-08-07T15:05:28.715 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-08-07T15:05:28.715 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2020-08-07T15:05:28.721 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2020-08-07 15:05:18.301203426 +0000 2020-08-07T15:05:28.721 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2020-08-07 15:01:52.420765694 +0000 2020-08-07T15:05:28.721 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2020-08-07 15:01:52.420765694 +0000 2020-08-07T15:05:28.722 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2020-08-07T15:05:28.734 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2020-08-07T15:05:28.734 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2020-08-07T15:05:28.734 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000193859 s, 2.6 MB/s 2020-08-07T15:05:28.735 INFO:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-08-07T15:05:28.806 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-08-07T15:05:28.806 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-08-07T15:05:28.807 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 6h/6d Inode: 98095 Links: 1 2020-08-07T15:05:28.807 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-08-07T15:05:28.807 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2020-08-07T15:05:28.807 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2020-08-07 15:05:18.301203426 +0000 2020-08-07T15:05:28.807 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2020-08-07 15:01:52.803756617 +0000 2020-08-07T15:05:28.808 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2020-08-07 15:01:52.803756617 +0000 2020-08-07T15:05:28.808 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2020-08-07T15:05:28.818 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2020-08-07T15:05:28.819 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2020-08-07T15:05:28.819 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000211981 s, 2.4 MB/s 2020-08-07T15:05:28.820 INFO:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-08-07T15:05:28.878 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-08-07T15:05:28.878 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-08-07T15:05:28.879 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 6h/6d Inode: 101225 Links: 1 2020-08-07T15:05:28.879 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-08-07T15:05:28.879 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2020-08-07T15:05:28.879 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2020-08-07 15:05:18.301203426 +0000 2020-08-07T15:05:28.879 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2020-08-07 15:01:53.179747706 +0000 2020-08-07T15:05:28.880 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2020-08-07 15:01:53.179747706 +0000 2020-08-07T15:05:28.880 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2020-08-07T15:05:28.890 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2020-08-07T15:05:28.891 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2020-08-07T15:05:28.891 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000187448 s, 2.7 MB/s 2020-08-07T15:05:28.891 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-08-07T15:05:28.892 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi074.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi096.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2020-08-07T15:05:28.892 INFO:tasks.ceph:Generating config... 2020-08-07T15:05:28.903 INFO:tasks.ceph:[osd] debug monc = 20 2020-08-07T15:05:28.903 INFO:tasks.ceph:[osd] bdev async discard = True 2020-08-07T15:05:28.904 INFO:tasks.ceph:[osd] bdev enable discard = True 2020-08-07T15:05:28.904 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2020-08-07T15:05:28.904 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2020-08-07T15:05:28.905 INFO:tasks.ceph:[osd] debug bluefs = 20 2020-08-07T15:05:28.905 INFO:tasks.ceph:[osd] debug bluestore = 20 2020-08-07T15:05:28.905 INFO:tasks.ceph:[osd] debug filestore = 20 2020-08-07T15:05:28.905 INFO:tasks.ceph:[osd] debug journal = 20 2020-08-07T15:05:28.906 INFO:tasks.ceph:[osd] debug ms = 20 2020-08-07T15:05:28.906 INFO:tasks.ceph:[osd] debug osd = 25 2020-08-07T15:05:28.906 INFO:tasks.ceph:[osd] debug rocksdb = 10 2020-08-07T15:05:28.907 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2020-08-07T15:05:28.907 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2020-08-07T15:05:28.907 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2020-08-07T15:05:28.907 INFO:tasks.ceph:[osd] osd debug reject backfill probability = 0.3 2020-08-07T15:05:28.908 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2020-08-07T15:05:28.908 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2020-08-07T15:05:28.908 INFO:tasks.ceph:[osd] osd delete sleep = 1 2020-08-07T15:05:28.909 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2020-08-07T15:05:28.909 INFO:tasks.ceph:[osd] osd max backfills = 3 2020-08-07T15:05:28.909 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2020-08-07T15:05:28.923 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2020-08-07T15:05:28.924 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2020-08-07T15:05:28.924 INFO:tasks.ceph:[osd] osd op queue = debug_random 2020-08-07T15:05:28.930 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2020-08-07T15:05:28.930 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2020-08-07T15:05:28.930 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2020-08-07T15:05:28.931 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2020-08-07T15:05:28.931 INFO:tasks.ceph:[osd] osd snap trim sleep = 2 2020-08-07T15:05:28.931 INFO:tasks.ceph:[client.0] debug ms = 1 2020-08-07T15:05:28.932 INFO:tasks.ceph:[client.0] debug objecter = 20 2020-08-07T15:05:28.932 INFO:tasks.ceph:[client.0] debug rados = 20 2020-08-07T15:05:28.932 INFO:tasks.ceph:[global] mon client directed command retry = 5 2020-08-07T15:05:28.932 INFO:tasks.ceph:[global] ms bind msgr1 = False 2020-08-07T15:05:28.933 INFO:tasks.ceph:[global] ms bind msgr2 = True 2020-08-07T15:05:28.933 INFO:tasks.ceph:[global] ms inject delay max = 1 2020-08-07T15:05:28.933 INFO:tasks.ceph:[global] ms inject delay probability = 0.005 2020-08-07T15:05:28.942 INFO:tasks.ceph:[global] ms inject delay type = osd 2020-08-07T15:05:28.943 INFO:tasks.ceph:[global] ms inject internal delays = 0.002 2020-08-07T15:05:28.943 INFO:tasks.ceph:[global] ms inject socket failures = 2500 2020-08-07T15:05:28.943 INFO:tasks.ceph:[global] ms type = async 2020-08-07T15:05:28.943 INFO:tasks.ceph:[global] osd_async_recovery_min_cost = 1 2020-08-07T15:05:28.944 INFO:tasks.ceph:[global] osd_max_pg_log_entries = 2 2020-08-07T15:05:28.944 INFO:tasks.ceph:[global] osd_min_pg_log_entries = 1 2020-08-07T15:05:28.944 INFO:tasks.ceph:[global] osd_object_clean_region_max_num_intervals = 1000 2020-08-07T15:05:28.945 INFO:tasks.ceph:[global] osd_pg_log_trim_min = 0 2020-08-07T15:05:28.945 INFO:tasks.ceph:[global] osd_pool_default_min_size = 2 2020-08-07T15:05:28.945 INFO:tasks.ceph:[global] osd_pool_default_size = 3 2020-08-07T15:05:28.945 INFO:tasks.ceph:[mgr] debug mgr = 20 2020-08-07T15:05:28.963 INFO:tasks.ceph:[mgr] debug ms = 1 2020-08-07T15:05:28.964 INFO:tasks.ceph:[mon] debug mon = 20 2020-08-07T15:05:28.964 INFO:tasks.ceph:[mon] debug ms = 1 2020-08-07T15:05:28.964 INFO:tasks.ceph:[mon] debug paxos = 20 2020-08-07T15:05:28.964 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2020-08-07T15:05:28.965 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2020-08-07T15:05:28.965 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2020-08-07T15:05:28.965 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2020-08-07T15:05:28.966 INFO:tasks.ceph:[mon] paxos service trim min = 10 2020-08-07T15:05:28.966 INFO:tasks.ceph:Setting up mon.a... 2020-08-07T15:05:28.967 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2020-08-07T15:05:29.042 INFO:teuthology.orchestra.run.smithi074.stdout:creating /etc/ceph/ceph.keyring 2020-08-07T15:05:29.043 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2020-08-07T15:05:29.091 INFO:teuthology.orchestra.run.smithi074:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-08-07T15:05:29.136 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.74'), ('mon.c', '[v2:172.21.15.74:3301,v1:172.21.15.74:6790]'), ('mon.b', '172.21.15.96')] 2020-08-07T15:05:29.137 INFO:teuthology.orchestra.run.smithi074:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.15.74 --addv c '[v2:172.21.15.74:3301,v1:172.21.15.74:6790]' --add b 172.21.15.96 --print /home/ubuntu/cephtest/ceph.monmap 2020-08-07T15:05:29.216 INFO:teuthology.orchestra.run.smithi074.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2020-08-07T15:05:29.217 INFO:teuthology.orchestra.run.smithi074.stdout:monmaptool: generated fsid a01d3e82-8d24-4288-9e42-576ab3ff4957 2020-08-07T15:05:29.217 INFO:teuthology.orchestra.run.smithi074.stdout:epoch 0 2020-08-07T15:05:29.217 INFO:teuthology.orchestra.run.smithi074.stdout:fsid a01d3e82-8d24-4288-9e42-576ab3ff4957 2020-08-07T15:05:29.218 INFO:teuthology.orchestra.run.smithi074.stdout:last_changed 2020-08-07T15:05:29.214088+0000 2020-08-07T15:05:29.218 INFO:teuthology.orchestra.run.smithi074.stdout:created 2020-08-07T15:05:29.214088+0000 2020-08-07T15:05:29.218 INFO:teuthology.orchestra.run.smithi074.stdout:min_mon_release 0 (unknown) 2020-08-07T15:05:29.219 INFO:teuthology.orchestra.run.smithi074.stdout:0: [v2:172.21.15.74:3300/0,v1:172.21.15.74:6789/0] mon.a 2020-08-07T15:05:29.219 INFO:teuthology.orchestra.run.smithi074.stdout:1: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2020-08-07T15:05:29.219 INFO:teuthology.orchestra.run.smithi074.stdout:2: [v2:172.21.15.74:3301/0,v1:172.21.15.74:6790/0] mon.c 2020-08-07T15:05:29.220 INFO:teuthology.orchestra.run.smithi074.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2020-08-07T15:05:29.220 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID a01d3e82-8d24-4288-9e42-576ab3ff4957... 2020-08-07T15:05:29.226 INFO:teuthology.orchestra.run.smithi074:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-08-07T15:05:29.259 INFO:teuthology.orchestra.run.smithi096:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-08-07T15:05:29.310 INFO:teuthology.orchestra.run.smithi074.stdout:[global] 2020-08-07T15:05:29.310 INFO:teuthology.orchestra.run.smithi074.stdout: chdir = "" 2020-08-07T15:05:29.311 INFO:teuthology.orchestra.run.smithi074.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-08-07T15:05:29.311 INFO:teuthology.orchestra.run.smithi074.stdout: auth supported = cephx 2020-08-07T15:05:29.311 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:29.311 INFO:teuthology.orchestra.run.smithi074.stdout: filestore xattr use omap = true 2020-08-07T15:05:29.311 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:29.312 INFO:teuthology.orchestra.run.smithi074.stdout: mon clock drift allowed = 1.000 2020-08-07T15:05:29.312 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:29.312 INFO:teuthology.orchestra.run.smithi074.stdout: osd crush chooseleaf type = 0 2020-08-07T15:05:29.312 INFO:teuthology.orchestra.run.smithi074.stdout: auth debug = true 2020-08-07T15:05:29.312 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:29.313 INFO:teuthology.orchestra.run.smithi074.stdout: ms die on old message = true 2020-08-07T15:05:29.313 INFO:teuthology.orchestra.run.smithi074.stdout: ms die on bug = true 2020-08-07T15:05:29.313 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:29.313 INFO:teuthology.orchestra.run.smithi074.stdout: mon max pg per osd = 10000 # >= luminous 2020-08-07T15:05:29.313 INFO:teuthology.orchestra.run.smithi074.stdout: mon pg warn max object skew = 0 2020-08-07T15:05:29.323 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:29.324 INFO:teuthology.orchestra.run.smithi074.stdout: # disable pg_autoscaler by default for new pools 2020-08-07T15:05:29.324 INFO:teuthology.orchestra.run.smithi074.stdout: osd_pool_default_pg_autoscale_mode = off 2020-08-07T15:05:29.324 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:29.324 INFO:teuthology.orchestra.run.smithi074.stdout: osd pool default size = 2 2020-08-07T15:05:29.324 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:29.325 INFO:teuthology.orchestra.run.smithi074.stdout: mon osd allow primary affinity = true 2020-08-07T15:05:29.325 INFO:teuthology.orchestra.run.smithi074.stdout: mon osd allow pg remap = true 2020-08-07T15:05:29.325 INFO:teuthology.orchestra.run.smithi074.stdout: mon warn on legacy crush tunables = false 2020-08-07T15:05:29.325 INFO:teuthology.orchestra.run.smithi074.stdout: mon warn on crush straw calc version zero = false 2020-08-07T15:05:29.325 INFO:teuthology.orchestra.run.smithi074.stdout: mon warn on no sortbitwise = false 2020-08-07T15:05:29.326 INFO:teuthology.orchestra.run.smithi074.stdout: mon warn on osd down out interval zero = false 2020-08-07T15:05:29.326 INFO:teuthology.orchestra.run.smithi074.stdout: mon warn on too few osds = false 2020-08-07T15:05:29.326 INFO:teuthology.orchestra.run.smithi074.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-08-07T15:05:29.326 INFO:teuthology.orchestra.run.smithi074.stdout: mon_warn_on_pool_no_redundancy = false 2020-08-07T15:05:29.326 INFO:teuthology.orchestra.run.smithi074.stdout: mon_allow_pool_size_one = true 2020-08-07T15:05:29.327 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:29.334 INFO:teuthology.orchestra.run.smithi074.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-08-07T15:05:29.334 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:29.334 INFO:teuthology.orchestra.run.smithi074.stdout: osd default data pool replay window = 5 2020-08-07T15:05:29.334 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:29.335 INFO:teuthology.orchestra.run.smithi074.stdout: mon allow pool delete = true 2020-08-07T15:05:29.335 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:29.335 INFO:teuthology.orchestra.run.smithi074.stdout: mon cluster log file level = debug 2020-08-07T15:05:29.346 INFO:teuthology.orchestra.run.smithi074.stdout: debug asserts on shutdown = true 2020-08-07T15:05:29.346 INFO:teuthology.orchestra.run.smithi074.stdout: mon host = "172.21.15.74,[v2:172.21.15.74:3301,v1:172.21.15.74:6790],172.21.15.96" 2020-08-07T15:05:29.346 INFO:teuthology.orchestra.run.smithi074.stdout: mon client directed command retry = 5 2020-08-07T15:05:29.346 INFO:teuthology.orchestra.run.smithi074.stdout: ms bind msgr1 = False 2020-08-07T15:05:29.347 INFO:teuthology.orchestra.run.smithi074.stdout: ms bind msgr2 = True 2020-08-07T15:05:29.347 INFO:teuthology.orchestra.run.smithi074.stdout: ms inject delay max = 1 2020-08-07T15:05:29.347 INFO:teuthology.orchestra.run.smithi074.stdout: ms inject delay probability = 0.005 2020-08-07T15:05:29.347 INFO:teuthology.orchestra.run.smithi074.stdout: ms inject delay type = osd 2020-08-07T15:05:29.347 INFO:teuthology.orchestra.run.smithi074.stdout: ms inject internal delays = 0.002 2020-08-07T15:05:29.348 INFO:teuthology.orchestra.run.smithi074.stdout: ms inject socket failures = 2500 2020-08-07T15:05:29.348 INFO:teuthology.orchestra.run.smithi074.stdout: ms type = async 2020-08-07T15:05:29.348 INFO:teuthology.orchestra.run.smithi074.stdout: osd_async_recovery_min_cost = 1 2020-08-07T15:05:29.348 INFO:teuthology.orchestra.run.smithi074.stdout: osd_max_pg_log_entries = 2 2020-08-07T15:05:29.348 INFO:teuthology.orchestra.run.smithi074.stdout: osd_min_pg_log_entries = 1 2020-08-07T15:05:29.349 INFO:teuthology.orchestra.run.smithi074.stdout: osd_object_clean_region_max_num_intervals = 1000 2020-08-07T15:05:29.349 INFO:teuthology.orchestra.run.smithi074.stdout: osd_pg_log_trim_min = 0 2020-08-07T15:05:29.350 INFO:teuthology.orchestra.run.smithi074.stdout: osd_pool_default_min_size = 2 2020-08-07T15:05:29.350 INFO:teuthology.orchestra.run.smithi074.stdout: osd_pool_default_size = 3 2020-08-07T15:05:29.350 INFO:teuthology.orchestra.run.smithi074.stdout: fsid = a01d3e82-8d24-4288-9e42-576ab3ff4957 2020-08-07T15:05:29.350 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:29.350 INFO:teuthology.orchestra.run.smithi074.stdout:[osd] 2020-08-07T15:05:29.350 INFO:teuthology.orchestra.run.smithi074.stdout: osd journal size = 100 2020-08-07T15:05:29.351 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:29.351 INFO:teuthology.orchestra.run.smithi074.stdout: osd scrub load threshold = 5.0 2020-08-07T15:05:29.351 INFO:teuthology.orchestra.run.smithi074.stdout: osd scrub max interval = 120 2020-08-07T15:05:29.351 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:29.351 INFO:teuthology.orchestra.run.smithi074.stdout: osd recover clone overlap = true 2020-08-07T15:05:29.352 INFO:teuthology.orchestra.run.smithi074.stdout: osd recovery max chunk = 1048576 2020-08-07T15:05:29.352 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:29.352 INFO:teuthology.orchestra.run.smithi074.stdout: osd debug shutdown = true 2020-08-07T15:05:29.352 INFO:teuthology.orchestra.run.smithi074.stdout: osd debug op order = true 2020-08-07T15:05:29.352 INFO:teuthology.orchestra.run.smithi074.stdout: osd debug verify stray on activate = true 2020-08-07T15:05:29.353 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:29.353 INFO:teuthology.orchestra.run.smithi074.stdout: osd open classes on start = true 2020-08-07T15:05:29.353 INFO:teuthology.orchestra.run.smithi074.stdout: osd debug pg log writeout = true 2020-08-07T15:05:29.353 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:29.353 INFO:teuthology.orchestra.run.smithi074.stdout: osd deep scrub update digest min age = 30 2020-08-07T15:05:29.360 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:29.374 INFO:teuthology.orchestra.run.smithi074.stdout: osd map max advance = 10 2020-08-07T15:05:29.374 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:29.374 INFO:teuthology.orchestra.run.smithi074.stdout: journal zero on create = true 2020-08-07T15:05:29.375 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:29.375 INFO:teuthology.orchestra.run.smithi074.stdout: filestore ondisk finisher threads = 3 2020-08-07T15:05:29.375 INFO:teuthology.orchestra.run.smithi074.stdout: filestore apply finisher threads = 3 2020-08-07T15:05:29.375 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:29.376 INFO:teuthology.orchestra.run.smithi074.stdout: bdev debug aio = true 2020-08-07T15:05:29.376 INFO:teuthology.orchestra.run.smithi074.stdout: osd debug misdirected ops = true 2020-08-07T15:05:29.376 INFO:teuthology.orchestra.run.smithi074.stdout: debug monc = 20 2020-08-07T15:05:29.376 INFO:teuthology.orchestra.run.smithi074.stdout: bdev async discard = True 2020-08-07T15:05:29.376 INFO:teuthology.orchestra.run.smithi074.stdout: bdev enable discard = True 2020-08-07T15:05:29.377 INFO:teuthology.orchestra.run.smithi074.stdout: bluestore block size = 96636764160 2020-08-07T15:05:29.377 INFO:teuthology.orchestra.run.smithi074.stdout: bluestore fsck on mount = True 2020-08-07T15:05:29.377 INFO:teuthology.orchestra.run.smithi074.stdout: debug bluefs = 20 2020-08-07T15:05:29.377 INFO:teuthology.orchestra.run.smithi074.stdout: debug bluestore = 20 2020-08-07T15:05:29.377 INFO:teuthology.orchestra.run.smithi074.stdout: debug filestore = 20 2020-08-07T15:05:29.388 INFO:teuthology.orchestra.run.smithi074.stdout: debug journal = 20 2020-08-07T15:05:29.389 INFO:teuthology.orchestra.run.smithi074.stdout: debug ms = 20 2020-08-07T15:05:29.389 INFO:teuthology.orchestra.run.smithi074.stdout: debug osd = 25 2020-08-07T15:05:29.389 INFO:teuthology.orchestra.run.smithi074.stdout: debug rocksdb = 10 2020-08-07T15:05:29.389 INFO:teuthology.orchestra.run.smithi074.stdout: mon osd backfillfull_ratio = 0.85 2020-08-07T15:05:29.389 INFO:teuthology.orchestra.run.smithi074.stdout: mon osd full ratio = 0.9 2020-08-07T15:05:29.390 INFO:teuthology.orchestra.run.smithi074.stdout: mon osd nearfull ratio = 0.8 2020-08-07T15:05:29.390 INFO:teuthology.orchestra.run.smithi074.stdout: osd debug reject backfill probability = 0.3 2020-08-07T15:05:29.390 INFO:teuthology.orchestra.run.smithi074.stdout: osd debug verify cached snaps = True 2020-08-07T15:05:29.390 INFO:teuthology.orchestra.run.smithi074.stdout: osd debug verify missing on start = True 2020-08-07T15:05:29.390 INFO:teuthology.orchestra.run.smithi074.stdout: osd delete sleep = 1 2020-08-07T15:05:29.391 INFO:teuthology.orchestra.run.smithi074.stdout: osd failsafe full ratio = 0.95 2020-08-07T15:05:29.391 INFO:teuthology.orchestra.run.smithi074.stdout: osd max backfills = 3 2020-08-07T15:05:29.391 INFO:teuthology.orchestra.run.smithi074.stdout: osd max markdown count = 1000 2020-08-07T15:05:29.391 INFO:teuthology.orchestra.run.smithi074.stdout: osd memory target = 1610612736 2020-08-07T15:05:29.391 INFO:teuthology.orchestra.run.smithi074.stdout: osd objectstore = bluestore 2020-08-07T15:05:29.392 INFO:teuthology.orchestra.run.smithi074.stdout: osd op queue = debug_random 2020-08-07T15:05:29.392 INFO:teuthology.orchestra.run.smithi074.stdout: osd op queue cut off = debug_random 2020-08-07T15:05:29.392 INFO:teuthology.orchestra.run.smithi074.stdout: osd scrub min interval = 60 2020-08-07T15:05:29.392 INFO:teuthology.orchestra.run.smithi074.stdout: osd shutdown pgref assert = True 2020-08-07T15:05:29.392 INFO:teuthology.orchestra.run.smithi074.stdout: osd snap trim sleep = 2 2020-08-07T15:05:29.393 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:29.393 INFO:teuthology.orchestra.run.smithi074.stdout:[mgr] 2020-08-07T15:05:29.393 INFO:teuthology.orchestra.run.smithi074.stdout: debug ms = 1 2020-08-07T15:05:29.393 INFO:teuthology.orchestra.run.smithi074.stdout: debug mgr = 20 2020-08-07T15:05:29.393 INFO:teuthology.orchestra.run.smithi074.stdout: debug mon = 20 2020-08-07T15:05:29.394 INFO:teuthology.orchestra.run.smithi074.stdout: debug auth = 20 2020-08-07T15:05:29.394 INFO:teuthology.orchestra.run.smithi074.stdout: mon reweight min pgs per osd = 4 2020-08-07T15:05:29.394 INFO:teuthology.orchestra.run.smithi074.stdout: mon reweight min bytes per osd = 10 2020-08-07T15:05:29.394 INFO:teuthology.orchestra.run.smithi074.stdout: mgr/telemetry/nag = false 2020-08-07T15:05:29.394 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:29.394 INFO:teuthology.orchestra.run.smithi074.stdout:[mon] 2020-08-07T15:05:29.395 INFO:teuthology.orchestra.run.smithi074.stdout: debug ms = 1 2020-08-07T15:05:29.395 INFO:teuthology.orchestra.run.smithi074.stdout: debug mon = 20 2020-08-07T15:05:29.395 INFO:teuthology.orchestra.run.smithi074.stdout: debug paxos = 20 2020-08-07T15:05:29.396 INFO:teuthology.orchestra.run.smithi074.stdout: debug auth = 20 2020-08-07T15:05:29.396 INFO:teuthology.orchestra.run.smithi074.stdout: mon data avail warn = 5 2020-08-07T15:05:29.396 INFO:teuthology.orchestra.run.smithi074.stdout: mon mgr mkfs grace = 240 2020-08-07T15:05:29.396 INFO:teuthology.orchestra.run.smithi074.stdout: mon reweight min pgs per osd = 4 2020-08-07T15:05:29.396 INFO:teuthology.orchestra.run.smithi074.stdout: mon osd reporter subtree level = osd 2020-08-07T15:05:29.397 INFO:teuthology.orchestra.run.smithi074.stdout: mon osd prime pg temp = true 2020-08-07T15:05:29.397 INFO:teuthology.orchestra.run.smithi074.stdout: mon reweight min bytes per osd = 10 2020-08-07T15:05:29.397 INFO:teuthology.orchestra.run.smithi074.stdout: mon min osdmap epochs = 50 2020-08-07T15:05:29.398 INFO:teuthology.orchestra.run.smithi074.stdout: mon osdmap full prune interval = 2 2020-08-07T15:05:29.398 INFO:teuthology.orchestra.run.smithi074.stdout: mon osdmap full prune min = 15 2020-08-07T15:05:29.398 INFO:teuthology.orchestra.run.smithi074.stdout: mon osdmap full prune txsize = 2 2020-08-07T15:05:29.398 INFO:teuthology.orchestra.run.smithi074.stdout: paxos service trim min = 10 2020-08-07T15:05:29.398 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:05:29.399 INFO:teuthology.orchestra.run.smithi074.stdout:[client] 2020-08-07T15:05:29.399 INFO:teuthology.orchestra.run.smithi074.stdout: rgw cache enabled = true 2020-08-07T15:05:29.399 INFO:teuthology.orchestra.run.smithi074.stdout: rgw enable ops log = true 2020-08-07T15:05:29.406 INFO:teuthology.orchestra.run.smithi074.stdout: rgw enable usage log = true 2020-08-07T15:05:29.407 INFO:teuthology.orchestra.run.smithi074.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-08-07T15:05:29.407 INFO:teuthology.orchestra.run.smithi074.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-08-07T15:05:29.407 INFO:teuthology.orchestra.run.smithi074.stdout:[mon.a] 2020-08-07T15:05:29.412 INFO:teuthology.orchestra.run.smithi074.stdout:[mon.c] 2020-08-07T15:05:29.412 INFO:teuthology.orchestra.run.smithi074.stdout:[mon.b] 2020-08-07T15:05:29.412 INFO:teuthology.orchestra.run.smithi074.stdout:[client.0] 2020-08-07T15:05:29.412 INFO:teuthology.orchestra.run.smithi074.stdout: debug ms = 1 2020-08-07T15:05:29.412 INFO:teuthology.orchestra.run.smithi074.stdout: debug objecter = 20 2020-08-07T15:05:29.413 INFO:teuthology.orchestra.run.smithi074.stdout: debug rados = 20 2020-08-07T15:05:29.414 INFO:teuthology.orchestra.run.smithi096.stdout:[global] 2020-08-07T15:05:29.414 INFO:teuthology.orchestra.run.smithi096.stdout: chdir = "" 2020-08-07T15:05:29.415 INFO:teuthology.orchestra.run.smithi096.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-08-07T15:05:29.415 INFO:teuthology.orchestra.run.smithi096.stdout: auth supported = cephx 2020-08-07T15:05:29.415 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:29.416 INFO:teuthology.orchestra.run.smithi096.stdout: filestore xattr use omap = true 2020-08-07T15:05:29.416 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:29.416 INFO:teuthology.orchestra.run.smithi096.stdout: mon clock drift allowed = 1.000 2020-08-07T15:05:29.416 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:29.416 INFO:teuthology.orchestra.run.smithi096.stdout: osd crush chooseleaf type = 0 2020-08-07T15:05:29.417 INFO:teuthology.orchestra.run.smithi096.stdout: auth debug = true 2020-08-07T15:05:29.417 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:29.417 INFO:teuthology.orchestra.run.smithi096.stdout: ms die on old message = true 2020-08-07T15:05:29.417 INFO:teuthology.orchestra.run.smithi096.stdout: ms die on bug = true 2020-08-07T15:05:29.417 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:29.418 INFO:teuthology.orchestra.run.smithi096.stdout: mon max pg per osd = 10000 # >= luminous 2020-08-07T15:05:29.418 INFO:teuthology.orchestra.run.smithi096.stdout: mon pg warn max object skew = 0 2020-08-07T15:05:29.418 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:29.418 INFO:teuthology.orchestra.run.smithi096.stdout: # disable pg_autoscaler by default for new pools 2020-08-07T15:05:29.419 INFO:teuthology.orchestra.run.smithi096.stdout: osd_pool_default_pg_autoscale_mode = off 2020-08-07T15:05:29.419 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:29.419 INFO:teuthology.orchestra.run.smithi096.stdout: osd pool default size = 2 2020-08-07T15:05:29.419 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:29.419 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd allow primary affinity = true 2020-08-07T15:05:29.419 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd allow pg remap = true 2020-08-07T15:05:29.420 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on legacy crush tunables = false 2020-08-07T15:05:29.420 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on crush straw calc version zero = false 2020-08-07T15:05:29.420 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on no sortbitwise = false 2020-08-07T15:05:29.420 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on osd down out interval zero = false 2020-08-07T15:05:29.420 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on too few osds = false 2020-08-07T15:05:29.421 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-08-07T15:05:29.421 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_pool_no_redundancy = false 2020-08-07T15:05:29.421 INFO:teuthology.orchestra.run.smithi096.stdout: mon_allow_pool_size_one = true 2020-08-07T15:05:29.421 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:29.421 INFO:teuthology.orchestra.run.smithi096.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-08-07T15:05:29.422 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:29.422 INFO:teuthology.orchestra.run.smithi096.stdout: osd default data pool replay window = 5 2020-08-07T15:05:29.422 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:29.422 INFO:teuthology.orchestra.run.smithi096.stdout: mon allow pool delete = true 2020-08-07T15:05:29.422 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:29.423 INFO:teuthology.orchestra.run.smithi096.stdout: mon cluster log file level = debug 2020-08-07T15:05:29.423 INFO:teuthology.orchestra.run.smithi096.stdout: debug asserts on shutdown = true 2020-08-07T15:05:29.423 INFO:teuthology.orchestra.run.smithi096.stdout: mon host = "172.21.15.74,[v2:172.21.15.74:3301,v1:172.21.15.74:6790],172.21.15.96" 2020-08-07T15:05:29.423 INFO:teuthology.orchestra.run.smithi096.stdout: mon client directed command retry = 5 2020-08-07T15:05:29.423 INFO:teuthology.orchestra.run.smithi096.stdout: ms bind msgr1 = False 2020-08-07T15:05:29.424 INFO:teuthology.orchestra.run.smithi096.stdout: ms bind msgr2 = True 2020-08-07T15:05:29.424 INFO:teuthology.orchestra.run.smithi096.stdout: ms inject delay max = 1 2020-08-07T15:05:29.424 INFO:teuthology.orchestra.run.smithi096.stdout: ms inject delay probability = 0.005 2020-08-07T15:05:29.424 INFO:teuthology.orchestra.run.smithi096.stdout: ms inject delay type = osd 2020-08-07T15:05:29.425 INFO:teuthology.orchestra.run.smithi096.stdout: ms inject internal delays = 0.002 2020-08-07T15:05:29.425 INFO:teuthology.orchestra.run.smithi096.stdout: ms inject socket failures = 2500 2020-08-07T15:05:29.425 INFO:teuthology.orchestra.run.smithi096.stdout: ms type = async 2020-08-07T15:05:29.425 INFO:teuthology.orchestra.run.smithi096.stdout: osd_async_recovery_min_cost = 1 2020-08-07T15:05:29.425 INFO:teuthology.orchestra.run.smithi096.stdout: osd_max_pg_log_entries = 2 2020-08-07T15:05:29.425 INFO:teuthology.orchestra.run.smithi096.stdout: osd_min_pg_log_entries = 1 2020-08-07T15:05:29.426 INFO:teuthology.orchestra.run.smithi096.stdout: osd_object_clean_region_max_num_intervals = 1000 2020-08-07T15:05:29.426 INFO:teuthology.orchestra.run.smithi096.stdout: osd_pg_log_trim_min = 0 2020-08-07T15:05:29.426 INFO:teuthology.orchestra.run.smithi096.stdout: osd_pool_default_min_size = 2 2020-08-07T15:05:29.426 INFO:teuthology.orchestra.run.smithi096.stdout: osd_pool_default_size = 3 2020-08-07T15:05:29.426 INFO:teuthology.orchestra.run.smithi096.stdout: fsid = a01d3e82-8d24-4288-9e42-576ab3ff4957 2020-08-07T15:05:29.427 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:29.427 INFO:teuthology.orchestra.run.smithi096.stdout:[osd] 2020-08-07T15:05:29.427 INFO:teuthology.orchestra.run.smithi096.stdout: osd journal size = 100 2020-08-07T15:05:29.428 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:29.428 INFO:teuthology.orchestra.run.smithi096.stdout: osd scrub load threshold = 5.0 2020-08-07T15:05:29.428 INFO:teuthology.orchestra.run.smithi096.stdout: osd scrub max interval = 120 2020-08-07T15:05:29.428 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:29.428 INFO:teuthology.orchestra.run.smithi096.stdout: osd recover clone overlap = true 2020-08-07T15:05:29.429 INFO:teuthology.orchestra.run.smithi096.stdout: osd recovery max chunk = 1048576 2020-08-07T15:05:29.429 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:29.429 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug shutdown = true 2020-08-07T15:05:29.429 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug op order = true 2020-08-07T15:05:29.429 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug verify stray on activate = true 2020-08-07T15:05:29.430 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:29.430 INFO:teuthology.orchestra.run.smithi096.stdout: osd open classes on start = true 2020-08-07T15:05:29.430 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug pg log writeout = true 2020-08-07T15:05:29.430 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:29.430 INFO:teuthology.orchestra.run.smithi096.stdout: osd deep scrub update digest min age = 30 2020-08-07T15:05:29.430 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:29.431 INFO:teuthology.orchestra.run.smithi096.stdout: osd map max advance = 10 2020-08-07T15:05:29.431 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:29.431 INFO:teuthology.orchestra.run.smithi096.stdout: journal zero on create = true 2020-08-07T15:05:29.431 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:29.431 INFO:teuthology.orchestra.run.smithi096.stdout: filestore ondisk finisher threads = 3 2020-08-07T15:05:29.432 INFO:teuthology.orchestra.run.smithi096.stdout: filestore apply finisher threads = 3 2020-08-07T15:05:29.432 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:29.432 INFO:teuthology.orchestra.run.smithi096.stdout: bdev debug aio = true 2020-08-07T15:05:29.432 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug misdirected ops = true 2020-08-07T15:05:29.432 INFO:teuthology.orchestra.run.smithi096.stdout: debug monc = 20 2020-08-07T15:05:29.433 INFO:teuthology.orchestra.run.smithi096.stdout: bdev async discard = True 2020-08-07T15:05:29.433 INFO:teuthology.orchestra.run.smithi096.stdout: bdev enable discard = True 2020-08-07T15:05:29.433 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore block size = 96636764160 2020-08-07T15:05:29.433 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore fsck on mount = True 2020-08-07T15:05:29.433 INFO:teuthology.orchestra.run.smithi096.stdout: debug bluefs = 20 2020-08-07T15:05:29.433 INFO:teuthology.orchestra.run.smithi096.stdout: debug bluestore = 20 2020-08-07T15:05:29.434 INFO:teuthology.orchestra.run.smithi096.stdout: debug filestore = 20 2020-08-07T15:05:29.434 INFO:teuthology.orchestra.run.smithi096.stdout: debug journal = 20 2020-08-07T15:05:29.434 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 20 2020-08-07T15:05:29.434 INFO:teuthology.orchestra.run.smithi096.stdout: debug osd = 25 2020-08-07T15:05:29.434 INFO:teuthology.orchestra.run.smithi096.stdout: debug rocksdb = 10 2020-08-07T15:05:29.435 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd backfillfull_ratio = 0.85 2020-08-07T15:05:29.435 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd full ratio = 0.9 2020-08-07T15:05:29.435 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd nearfull ratio = 0.8 2020-08-07T15:05:29.435 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug reject backfill probability = 0.3 2020-08-07T15:05:29.435 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug verify cached snaps = True 2020-08-07T15:05:29.436 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug verify missing on start = True 2020-08-07T15:05:29.436 INFO:teuthology.orchestra.run.smithi096.stdout: osd delete sleep = 1 2020-08-07T15:05:29.436 INFO:teuthology.orchestra.run.smithi096.stdout: osd failsafe full ratio = 0.95 2020-08-07T15:05:29.436 INFO:teuthology.orchestra.run.smithi096.stdout: osd max backfills = 3 2020-08-07T15:05:29.436 INFO:teuthology.orchestra.run.smithi096.stdout: osd max markdown count = 1000 2020-08-07T15:05:29.437 INFO:teuthology.orchestra.run.smithi096.stdout: osd memory target = 1610612736 2020-08-07T15:05:29.437 INFO:teuthology.orchestra.run.smithi096.stdout: osd objectstore = bluestore 2020-08-07T15:05:29.437 INFO:teuthology.orchestra.run.smithi096.stdout: osd op queue = debug_random 2020-08-07T15:05:29.437 INFO:teuthology.orchestra.run.smithi096.stdout: osd op queue cut off = debug_random 2020-08-07T15:05:29.437 INFO:teuthology.orchestra.run.smithi096.stdout: osd scrub min interval = 60 2020-08-07T15:05:29.437 INFO:teuthology.orchestra.run.smithi096.stdout: osd shutdown pgref assert = True 2020-08-07T15:05:29.438 INFO:teuthology.orchestra.run.smithi096.stdout: osd snap trim sleep = 2 2020-08-07T15:05:29.438 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:29.438 INFO:teuthology.orchestra.run.smithi096.stdout:[mgr] 2020-08-07T15:05:29.438 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2020-08-07T15:05:29.439 INFO:teuthology.orchestra.run.smithi096.stdout: debug mgr = 20 2020-08-07T15:05:29.439 INFO:teuthology.orchestra.run.smithi096.stdout: debug mon = 20 2020-08-07T15:05:29.439 INFO:teuthology.orchestra.run.smithi096.stdout: debug auth = 20 2020-08-07T15:05:29.439 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min pgs per osd = 4 2020-08-07T15:05:29.439 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min bytes per osd = 10 2020-08-07T15:05:29.439 INFO:teuthology.orchestra.run.smithi096.stdout: mgr/telemetry/nag = false 2020-08-07T15:05:29.440 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:29.440 INFO:teuthology.orchestra.run.smithi096.stdout:[mon] 2020-08-07T15:05:29.440 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2020-08-07T15:05:29.441 INFO:teuthology.orchestra.run.smithi096.stdout: debug mon = 20 2020-08-07T15:05:29.441 INFO:teuthology.orchestra.run.smithi096.stdout: debug paxos = 20 2020-08-07T15:05:29.442 INFO:teuthology.orchestra.run.smithi096.stdout: debug auth = 20 2020-08-07T15:05:29.442 INFO:teuthology.orchestra.run.smithi096.stdout: mon data avail warn = 5 2020-08-07T15:05:29.442 INFO:teuthology.orchestra.run.smithi096.stdout: mon mgr mkfs grace = 240 2020-08-07T15:05:29.442 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min pgs per osd = 4 2020-08-07T15:05:29.442 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd reporter subtree level = osd 2020-08-07T15:05:29.443 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd prime pg temp = true 2020-08-07T15:05:29.443 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min bytes per osd = 10 2020-08-07T15:05:29.443 INFO:teuthology.orchestra.run.smithi096.stdout: mon min osdmap epochs = 50 2020-08-07T15:05:29.443 INFO:teuthology.orchestra.run.smithi096.stdout: mon osdmap full prune interval = 2 2020-08-07T15:05:29.443 INFO:teuthology.orchestra.run.smithi096.stdout: mon osdmap full prune min = 15 2020-08-07T15:05:29.443 INFO:teuthology.orchestra.run.smithi096.stdout: mon osdmap full prune txsize = 2 2020-08-07T15:05:29.444 INFO:teuthology.orchestra.run.smithi096.stdout: paxos service trim min = 10 2020-08-07T15:05:29.444 INFO:teuthology.orchestra.run.smithi096.stdout: 2020-08-07T15:05:29.444 INFO:teuthology.orchestra.run.smithi096.stdout:[client] 2020-08-07T15:05:29.444 INFO:teuthology.orchestra.run.smithi096.stdout: rgw cache enabled = true 2020-08-07T15:05:29.444 INFO:teuthology.orchestra.run.smithi096.stdout: rgw enable ops log = true 2020-08-07T15:05:29.445 INFO:teuthology.orchestra.run.smithi096.stdout: rgw enable usage log = true 2020-08-07T15:05:29.445 INFO:teuthology.orchestra.run.smithi096.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-08-07T15:05:29.445 INFO:teuthology.orchestra.run.smithi096.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-08-07T15:05:29.445 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.a] 2020-08-07T15:05:29.445 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.c] 2020-08-07T15:05:29.446 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.b] 2020-08-07T15:05:29.446 INFO:teuthology.orchestra.run.smithi096.stdout:[client.0] 2020-08-07T15:05:29.446 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2020-08-07T15:05:29.446 INFO:teuthology.orchestra.run.smithi096.stdout: debug objecter = 20 2020-08-07T15:05:29.446 INFO:teuthology.orchestra.run.smithi096.stdout: debug rados = 20 2020-08-07T15:05:29.447 INFO:tasks.ceph:Creating admin key on mon.a... 2020-08-07T15:05:29.447 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2020-08-07T15:05:29.546 INFO:tasks.ceph:Copying monmap to all nodes... 2020-08-07T15:05:29.572 DEBUG:teuthology.orchestra.remote:smithi074:/etc/ceph/ceph.keyring is 206B 2020-08-07T15:05:29.700 DEBUG:teuthology.orchestra.remote:smithi074:/home/ubuntu/cephtest/ceph.monmap is 379B 2020-08-07T15:05:29.731 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi074.front.sepia.ceph.com 2020-08-07T15:05:29.732 INFO:teuthology.orchestra.run.smithi074:> sudo sh -c 'cat > /etc/ceph/ceph.keyring' && sudo chmod 0644 /etc/ceph/ceph.keyring 2020-08-07T15:05:29.790 INFO:teuthology.orchestra.run.smithi074:> cat > /home/ubuntu/cephtest/ceph.monmap 2020-08-07T15:05:29.880 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi096.front.sepia.ceph.com 2020-08-07T15:05:29.880 INFO:teuthology.orchestra.run.smithi096:> sudo sh -c 'cat > /etc/ceph/ceph.keyring' && sudo chmod 0644 /etc/ceph/ceph.keyring 2020-08-07T15:05:29.952 INFO:teuthology.orchestra.run.smithi096:> cat > /home/ubuntu/cephtest/ceph.monmap 2020-08-07T15:05:30.063 INFO:tasks.ceph:Setting up mon nodes... 2020-08-07T15:05:30.064 INFO:tasks.ceph:Setting up mgr nodes... 2020-08-07T15:05:30.064 INFO:teuthology.orchestra.run.smithi074:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2020-08-07T15:05:30.118 INFO:teuthology.orchestra.run.smithi074.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2020-08-07T15:05:30.120 INFO:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2020-08-07T15:05:30.202 INFO:teuthology.orchestra.run.smithi096.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2020-08-07T15:05:30.203 INFO:tasks.ceph:Setting up mds nodes... 2020-08-07T15:05:30.203 INFO:tasks.ceph_client:Setting up client nodes... 2020-08-07T15:05:30.204 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2020-08-07T15:05:30.262 INFO:teuthology.orchestra.run.smithi074.stdout:creating /etc/ceph/ceph.client.0.keyring 2020-08-07T15:05:30.289 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2020-08-07T15:05:30.343 INFO:teuthology.orchestra.run.smithi096.stdout:creating /etc/ceph/ceph.client.1.keyring 2020-08-07T15:05:30.365 INFO:tasks.ceph:Running mkfs on osd nodes... 2020-08-07T15:05:30.366 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi074.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi096.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2020-08-07T15:05:30.366 INFO:teuthology.orchestra.run.smithi074:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2020-08-07T15:05:30.395 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-08-07T15:05:30.395 INFO:tasks.ceph:role: osd.0 2020-08-07T15:05:30.395 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi074.front.sepia.ceph.com 2020-08-07T15:05:30.396 INFO:teuthology.orchestra.run.smithi074:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-08-07T15:05:30.938 INFO:teuthology.orchestra.run.smithi074.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-08-07T15:05:30.939 INFO:teuthology.orchestra.run.smithi074.stdout: = sectsz=512 attr=2, projid32bit=1 2020-08-07T15:05:30.939 INFO:teuthology.orchestra.run.smithi074.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-08-07T15:05:30.939 INFO:teuthology.orchestra.run.smithi074.stdout: = reflink=1 2020-08-07T15:05:30.942 INFO:teuthology.orchestra.run.smithi074.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-08-07T15:05:30.942 INFO:teuthology.orchestra.run.smithi074.stdout: = sunit=0 swidth=0 blks 2020-08-07T15:05:30.942 INFO:teuthology.orchestra.run.smithi074.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-08-07T15:05:30.958 INFO:teuthology.orchestra.run.smithi074.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-08-07T15:05:30.959 INFO:teuthology.orchestra.run.smithi074.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-08-07T15:05:30.959 INFO:teuthology.orchestra.run.smithi074.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-08-07T15:05:30.960 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi074.front.sepia.ceph.com -o noatime 2020-08-07T15:05:30.960 INFO:teuthology.orchestra.run.smithi074:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2020-08-07T15:05:31.078 INFO:teuthology.orchestra.run.smithi074:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2020-08-07T15:05:31.111 INFO:teuthology.orchestra.run.smithi074:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2020-08-07T15:05:31.184 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-08-07T15:05:31.184 INFO:tasks.ceph:role: osd.1 2020-08-07T15:05:31.185 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi074.front.sepia.ceph.com 2020-08-07T15:05:31.185 INFO:teuthology.orchestra.run.smithi074:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-08-07T15:05:31.700 INFO:teuthology.orchestra.run.smithi074.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-08-07T15:05:31.700 INFO:teuthology.orchestra.run.smithi074.stdout: = sectsz=512 attr=2, projid32bit=1 2020-08-07T15:05:31.700 INFO:teuthology.orchestra.run.smithi074.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-08-07T15:05:31.701 INFO:teuthology.orchestra.run.smithi074.stdout: = reflink=1 2020-08-07T15:05:31.701 INFO:teuthology.orchestra.run.smithi074.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-08-07T15:05:31.701 INFO:teuthology.orchestra.run.smithi074.stdout: = sunit=0 swidth=0 blks 2020-08-07T15:05:31.701 INFO:teuthology.orchestra.run.smithi074.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-08-07T15:05:31.701 INFO:teuthology.orchestra.run.smithi074.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-08-07T15:05:31.714 INFO:teuthology.orchestra.run.smithi074.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-08-07T15:05:31.714 INFO:teuthology.orchestra.run.smithi074.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-08-07T15:05:31.716 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi074.front.sepia.ceph.com -o noatime 2020-08-07T15:05:31.716 INFO:teuthology.orchestra.run.smithi074:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2020-08-07T15:05:31.759 INFO:teuthology.orchestra.run.smithi074:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2020-08-07T15:05:31.839 INFO:teuthology.orchestra.run.smithi074:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2020-08-07T15:05:31.917 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-08-07T15:05:31.918 INFO:tasks.ceph:role: osd.2 2020-08-07T15:05:31.918 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi074.front.sepia.ceph.com 2020-08-07T15:05:31.918 INFO:teuthology.orchestra.run.smithi074:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-08-07T15:05:32.466 INFO:teuthology.orchestra.run.smithi074.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-08-07T15:05:32.467 INFO:teuthology.orchestra.run.smithi074.stdout: = sectsz=512 attr=2, projid32bit=1 2020-08-07T15:05:32.467 INFO:teuthology.orchestra.run.smithi074.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-08-07T15:05:32.467 INFO:teuthology.orchestra.run.smithi074.stdout: = reflink=1 2020-08-07T15:05:32.468 INFO:teuthology.orchestra.run.smithi074.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-08-07T15:05:32.468 INFO:teuthology.orchestra.run.smithi074.stdout: = sunit=0 swidth=0 blks 2020-08-07T15:05:32.468 INFO:teuthology.orchestra.run.smithi074.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-08-07T15:05:32.468 INFO:teuthology.orchestra.run.smithi074.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-08-07T15:05:32.468 INFO:teuthology.orchestra.run.smithi074.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-08-07T15:05:32.469 INFO:teuthology.orchestra.run.smithi074.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-08-07T15:05:32.469 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi074.front.sepia.ceph.com -o noatime 2020-08-07T15:05:32.469 INFO:teuthology.orchestra.run.smithi074:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2020-08-07T15:05:32.545 INFO:teuthology.orchestra.run.smithi074:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2020-08-07T15:05:32.579 INFO:teuthology.orchestra.run.smithi074:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2020-08-07T15:05:32.647 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-08-07T15:05:32.648 INFO:tasks.ceph:role: osd.3 2020-08-07T15:05:32.649 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi074.front.sepia.ceph.com 2020-08-07T15:05:32.650 INFO:teuthology.orchestra.run.smithi074:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-08-07T15:05:33.205 INFO:teuthology.orchestra.run.smithi074.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-08-07T15:05:33.206 INFO:teuthology.orchestra.run.smithi074.stdout: = sectsz=512 attr=2, projid32bit=1 2020-08-07T15:05:33.206 INFO:teuthology.orchestra.run.smithi074.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-08-07T15:05:33.206 INFO:teuthology.orchestra.run.smithi074.stdout: = reflink=1 2020-08-07T15:05:33.206 INFO:teuthology.orchestra.run.smithi074.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-08-07T15:05:33.206 INFO:teuthology.orchestra.run.smithi074.stdout: = sunit=0 swidth=0 blks 2020-08-07T15:05:33.212 INFO:teuthology.orchestra.run.smithi074.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-08-07T15:05:33.212 INFO:teuthology.orchestra.run.smithi074.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-08-07T15:05:33.213 INFO:teuthology.orchestra.run.smithi074.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-08-07T15:05:33.215 INFO:teuthology.orchestra.run.smithi074.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-08-07T15:05:33.215 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi074.front.sepia.ceph.com -o noatime 2020-08-07T15:05:33.216 INFO:teuthology.orchestra.run.smithi074:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2020-08-07T15:05:33.331 INFO:teuthology.orchestra.run.smithi074:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2020-08-07T15:05:33.362 INFO:teuthology.orchestra.run.smithi074:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-08-07T15:05:33.448 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:05:33.446+0000 7fa58e6d7ec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2020-08-07T15:05:33.451 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:05:33.450+0000 7fa58e6d7ec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2020-08-07T15:05:33.454 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:05:33.450+0000 7fa58e6d7ec0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-08-07T15:05:33.455 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:05:33.450+0000 7fa58e6d7ec0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-08-07T15:05:33.455 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:05:33.450+0000 7fa58e6d7ec0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-08-07T15:05:33.456 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:05:33.454+0000 7fa58e6d7ec0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2020-08-07T15:05:35.813 INFO:teuthology.orchestra.run.smithi074:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2020-08-07T15:05:35.878 INFO:teuthology.orchestra.run.smithi074:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-08-07T15:05:35.979 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:05:35.974+0000 7f9c408baec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2020-08-07T15:05:35.979 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:05:35.974+0000 7f9c408baec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2020-08-07T15:05:35.979 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:05:35.974+0000 7f9c408baec0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-08-07T15:05:35.980 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:05:35.974+0000 7f9c408baec0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-08-07T15:05:35.980 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:05:35.974+0000 7f9c408baec0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-08-07T15:05:35.980 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:05:35.974+0000 7f9c408baec0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2020-08-07T15:05:38.323 INFO:teuthology.orchestra.run.smithi074:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2020-08-07T15:05:38.374 INFO:teuthology.orchestra.run.smithi074:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-08-07T15:05:38.472 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:05:38.471+0000 7fdbeb19fec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2020-08-07T15:05:38.472 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:05:38.471+0000 7fdbeb19fec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2020-08-07T15:05:38.473 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:05:38.471+0000 7fdbeb19fec0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-08-07T15:05:38.473 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:05:38.471+0000 7fdbeb19fec0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-08-07T15:05:38.473 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:05:38.471+0000 7fdbeb19fec0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-08-07T15:05:38.474 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:05:38.472+0000 7fdbeb19fec0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2020-08-07T15:05:40.815 INFO:teuthology.orchestra.run.smithi074:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2020-08-07T15:05:40.894 INFO:teuthology.orchestra.run.smithi074:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-08-07T15:05:40.990 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:05:40.988+0000 7ffa4ab21ec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2020-08-07T15:05:40.990 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:05:40.988+0000 7ffa4ab21ec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2020-08-07T15:05:40.990 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:05:40.988+0000 7ffa4ab21ec0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-08-07T15:05:40.991 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:05:40.988+0000 7ffa4ab21ec0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-08-07T15:05:40.991 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:05:40.988+0000 7ffa4ab21ec0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-08-07T15:05:40.992 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:05:40.989+0000 7ffa4ab21ec0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2020-08-07T15:05:43.332 INFO:teuthology.orchestra.run.smithi074:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2020-08-07T15:05:43.371 INFO:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2020-08-07T15:05:43.405 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-08-07T15:05:43.406 INFO:tasks.ceph:role: osd.4 2020-08-07T15:05:43.407 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi096.front.sepia.ceph.com 2020-08-07T15:05:43.407 INFO:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-08-07T15:05:43.938 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-08-07T15:05:43.939 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2020-08-07T15:05:43.966 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-08-07T15:05:43.966 INFO:teuthology.orchestra.run.smithi096.stdout: = reflink=1 2020-08-07T15:05:43.966 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-08-07T15:05:43.966 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2020-08-07T15:05:43.967 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-08-07T15:05:43.967 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-08-07T15:05:43.967 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-08-07T15:05:43.967 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-08-07T15:05:43.968 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2020-08-07T15:05:43.983 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2020-08-07T15:05:44.126 INFO:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2020-08-07T15:05:44.162 INFO:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2020-08-07T15:05:44.230 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-08-07T15:05:44.231 INFO:tasks.ceph:role: osd.5 2020-08-07T15:05:44.231 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi096.front.sepia.ceph.com 2020-08-07T15:05:44.231 INFO:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-08-07T15:05:44.738 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-08-07T15:05:44.738 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2020-08-07T15:05:44.739 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-08-07T15:05:44.739 INFO:teuthology.orchestra.run.smithi096.stdout: = reflink=1 2020-08-07T15:05:44.739 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-08-07T15:05:44.739 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2020-08-07T15:05:44.740 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-08-07T15:05:44.740 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-08-07T15:05:44.740 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-08-07T15:05:44.740 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-08-07T15:05:44.741 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2020-08-07T15:05:44.742 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2020-08-07T15:05:44.807 INFO:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2020-08-07T15:05:44.841 INFO:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2020-08-07T15:05:44.913 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-08-07T15:05:44.913 INFO:tasks.ceph:role: osd.6 2020-08-07T15:05:44.914 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi096.front.sepia.ceph.com 2020-08-07T15:05:44.914 INFO:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-08-07T15:05:45.454 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-08-07T15:05:45.455 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2020-08-07T15:05:45.455 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-08-07T15:05:45.455 INFO:teuthology.orchestra.run.smithi096.stdout: = reflink=1 2020-08-07T15:05:45.456 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-08-07T15:05:45.456 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2020-08-07T15:05:45.456 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-08-07T15:05:45.456 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-08-07T15:05:45.456 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-08-07T15:05:45.457 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-08-07T15:05:45.457 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2020-08-07T15:05:45.457 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2020-08-07T15:05:45.534 INFO:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2020-08-07T15:05:45.568 INFO:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2020-08-07T15:05:45.636 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-08-07T15:05:45.636 INFO:tasks.ceph:role: osd.7 2020-08-07T15:05:45.636 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi096.front.sepia.ceph.com 2020-08-07T15:05:45.637 INFO:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-08-07T15:05:46.191 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-08-07T15:05:46.191 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2020-08-07T15:05:46.191 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2020-08-07T15:05:46.192 INFO:teuthology.orchestra.run.smithi096.stdout: = reflink=1 2020-08-07T15:05:46.192 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-08-07T15:05:46.192 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2020-08-07T15:05:46.192 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2020-08-07T15:05:46.192 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-08-07T15:05:46.193 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-08-07T15:05:46.193 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-08-07T15:05:46.196 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2020-08-07T15:05:46.197 INFO:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2020-08-07T15:05:46.311 INFO:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2020-08-07T15:05:46.345 INFO:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-08-07T15:05:46.436 INFO:teuthology.orchestra.run.smithi096.stderr:2020-08-07T15:05:46.431+0000 7f3aee585ec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2020-08-07T15:05:46.438 INFO:teuthology.orchestra.run.smithi096.stderr:2020-08-07T15:05:46.434+0000 7f3aee585ec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2020-08-07T15:05:46.438 INFO:teuthology.orchestra.run.smithi096.stderr:2020-08-07T15:05:46.434+0000 7f3aee585ec0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-08-07T15:05:46.439 INFO:teuthology.orchestra.run.smithi096.stderr:2020-08-07T15:05:46.434+0000 7f3aee585ec0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-08-07T15:05:46.439 INFO:teuthology.orchestra.run.smithi096.stderr:2020-08-07T15:05:46.434+0000 7f3aee585ec0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-08-07T15:05:46.445 INFO:teuthology.orchestra.run.smithi096.stderr:2020-08-07T15:05:46.441+0000 7f3aee585ec0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2020-08-07T15:05:48.797 INFO:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2020-08-07T15:05:48.871 INFO:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-08-07T15:05:48.968 INFO:teuthology.orchestra.run.smithi096.stderr:2020-08-07T15:05:48.960+0000 7fc53895cec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2020-08-07T15:05:48.969 INFO:teuthology.orchestra.run.smithi096.stderr:2020-08-07T15:05:48.960+0000 7fc53895cec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2020-08-07T15:05:48.969 INFO:teuthology.orchestra.run.smithi096.stderr:2020-08-07T15:05:48.960+0000 7fc53895cec0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-08-07T15:05:48.970 INFO:teuthology.orchestra.run.smithi096.stderr:2020-08-07T15:05:48.960+0000 7fc53895cec0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-08-07T15:05:48.970 INFO:teuthology.orchestra.run.smithi096.stderr:2020-08-07T15:05:48.960+0000 7fc53895cec0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-08-07T15:05:48.971 INFO:teuthology.orchestra.run.smithi096.stderr:2020-08-07T15:05:48.961+0000 7fc53895cec0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2020-08-07T15:05:51.312 INFO:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2020-08-07T15:05:51.356 INFO:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-08-07T15:05:51.440 INFO:teuthology.orchestra.run.smithi096.stderr:2020-08-07T15:05:51.438+0000 7fc7b09dfec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2020-08-07T15:05:51.441 INFO:teuthology.orchestra.run.smithi096.stderr:2020-08-07T15:05:51.438+0000 7fc7b09dfec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2020-08-07T15:05:51.441 INFO:teuthology.orchestra.run.smithi096.stderr:2020-08-07T15:05:51.438+0000 7fc7b09dfec0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-08-07T15:05:51.442 INFO:teuthology.orchestra.run.smithi096.stderr:2020-08-07T15:05:51.438+0000 7fc7b09dfec0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-08-07T15:05:51.442 INFO:teuthology.orchestra.run.smithi096.stderr:2020-08-07T15:05:51.438+0000 7fc7b09dfec0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-08-07T15:05:51.443 INFO:teuthology.orchestra.run.smithi096.stderr:2020-08-07T15:05:51.439+0000 7fc7b09dfec0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2020-08-07T15:05:53.784 INFO:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2020-08-07T15:05:53.816 INFO:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-08-07T15:05:53.910 INFO:teuthology.orchestra.run.smithi096.stderr:2020-08-07T15:05:53.903+0000 7f62dd1c7ec0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2020-08-07T15:05:53.910 INFO:teuthology.orchestra.run.smithi096.stderr:2020-08-07T15:05:53.903+0000 7f62dd1c7ec0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2020-08-07T15:05:53.910 INFO:teuthology.orchestra.run.smithi096.stderr:2020-08-07T15:05:53.903+0000 7f62dd1c7ec0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-08-07T15:05:53.911 INFO:teuthology.orchestra.run.smithi096.stderr:2020-08-07T15:05:53.904+0000 7f62dd1c7ec0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-08-07T15:05:53.911 INFO:teuthology.orchestra.run.smithi096.stderr:2020-08-07T15:05:53.904+0000 7f62dd1c7ec0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-08-07T15:05:53.911 INFO:teuthology.orchestra.run.smithi096.stderr:2020-08-07T15:05:53.904+0000 7f62dd1c7ec0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2020-08-07T15:05:56.253 INFO:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2020-08-07T15:05:56.287 INFO:tasks.ceph:Reading keys from all nodes... 2020-08-07T15:05:56.288 INFO:teuthology.orchestra.run.smithi074:> mktemp 2020-08-07T15:05:56.308 INFO:teuthology.orchestra.run.smithi074.stdout:/tmp/tmp.Q5DsRQevT0 2020-08-07T15:05:56.309 INFO:teuthology.orchestra.run.smithi074:> sudo cp /var/lib/ceph/mgr/ceph-y/keyring /tmp/tmp.Q5DsRQevT0 2020-08-07T15:05:56.382 INFO:teuthology.orchestra.run.smithi074:> sudo chmod 0666 /tmp/tmp.Q5DsRQevT0 2020-08-07T15:05:56.522 DEBUG:teuthology.orchestra.remote:smithi074:/tmp/tmp.Q5DsRQevT0 is 56B 2020-08-07T15:05:56.581 INFO:teuthology.orchestra.run.smithi074:> rm -fr /tmp/tmp.Q5DsRQevT0 2020-08-07T15:05:56.603 INFO:teuthology.orchestra.run.smithi074:> mktemp 2020-08-07T15:05:56.660 INFO:teuthology.orchestra.run.smithi074.stdout:/tmp/tmp.QL48odwDrJ 2020-08-07T15:05:56.661 INFO:teuthology.orchestra.run.smithi074:> sudo cp /var/lib/ceph/osd/ceph-0/keyring /tmp/tmp.QL48odwDrJ 2020-08-07T15:05:56.732 INFO:teuthology.orchestra.run.smithi074:> sudo chmod 0666 /tmp/tmp.QL48odwDrJ 2020-08-07T15:05:56.890 DEBUG:teuthology.orchestra.remote:smithi074:/tmp/tmp.QL48odwDrJ is 56B 2020-08-07T15:05:56.969 INFO:teuthology.orchestra.run.smithi074:> rm -fr /tmp/tmp.QL48odwDrJ 2020-08-07T15:05:56.985 INFO:teuthology.orchestra.run.smithi074:> mktemp 2020-08-07T15:05:57.040 INFO:teuthology.orchestra.run.smithi074.stdout:/tmp/tmp.QWk4QtMIFD 2020-08-07T15:05:57.041 INFO:teuthology.orchestra.run.smithi074:> sudo cp /var/lib/ceph/osd/ceph-1/keyring /tmp/tmp.QWk4QtMIFD 2020-08-07T15:05:57.112 INFO:teuthology.orchestra.run.smithi074:> sudo chmod 0666 /tmp/tmp.QWk4QtMIFD 2020-08-07T15:05:57.282 DEBUG:teuthology.orchestra.remote:smithi074:/tmp/tmp.QWk4QtMIFD is 56B 2020-08-07T15:05:57.346 INFO:teuthology.orchestra.run.smithi074:> rm -fr /tmp/tmp.QWk4QtMIFD 2020-08-07T15:05:57.374 INFO:teuthology.orchestra.run.smithi074:> mktemp 2020-08-07T15:05:57.402 INFO:teuthology.orchestra.run.smithi074.stdout:/tmp/tmp.HJkg68bapx 2020-08-07T15:05:57.403 INFO:teuthology.orchestra.run.smithi074:> sudo cp /var/lib/ceph/osd/ceph-2/keyring /tmp/tmp.HJkg68bapx 2020-08-07T15:05:57.484 INFO:teuthology.orchestra.run.smithi074:> sudo chmod 0666 /tmp/tmp.HJkg68bapx 2020-08-07T15:05:57.660 DEBUG:teuthology.orchestra.remote:smithi074:/tmp/tmp.HJkg68bapx is 56B 2020-08-07T15:05:57.786 INFO:teuthology.orchestra.run.smithi074:> rm -fr /tmp/tmp.HJkg68bapx 2020-08-07T15:05:57.811 INFO:teuthology.orchestra.run.smithi074:> mktemp 2020-08-07T15:05:57.882 INFO:teuthology.orchestra.run.smithi074.stdout:/tmp/tmp.kA9LV9UiDE 2020-08-07T15:05:57.883 INFO:teuthology.orchestra.run.smithi074:> sudo cp /var/lib/ceph/osd/ceph-3/keyring /tmp/tmp.kA9LV9UiDE 2020-08-07T15:05:57.959 INFO:teuthology.orchestra.run.smithi074:> sudo chmod 0666 /tmp/tmp.kA9LV9UiDE 2020-08-07T15:05:58.115 DEBUG:teuthology.orchestra.remote:smithi074:/tmp/tmp.kA9LV9UiDE is 56B 2020-08-07T15:05:58.147 INFO:teuthology.orchestra.run.smithi074:> rm -fr /tmp/tmp.kA9LV9UiDE 2020-08-07T15:05:58.165 INFO:teuthology.orchestra.run.smithi096:> mktemp 2020-08-07T15:05:58.220 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.zmtuD9PoJj 2020-08-07T15:05:58.221 INFO:teuthology.orchestra.run.smithi096:> sudo cp /var/lib/ceph/mgr/ceph-x/keyring /tmp/tmp.zmtuD9PoJj 2020-08-07T15:05:58.294 INFO:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.zmtuD9PoJj 2020-08-07T15:05:58.461 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.zmtuD9PoJj is 56B 2020-08-07T15:05:58.525 INFO:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.zmtuD9PoJj 2020-08-07T15:05:58.556 INFO:teuthology.orchestra.run.smithi096:> mktemp 2020-08-07T15:05:58.620 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.Fr54EG6ifm 2020-08-07T15:05:58.621 INFO:teuthology.orchestra.run.smithi096:> sudo cp /var/lib/ceph/osd/ceph-4/keyring /tmp/tmp.Fr54EG6ifm 2020-08-07T15:05:58.698 INFO:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.Fr54EG6ifm 2020-08-07T15:05:58.829 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.Fr54EG6ifm is 56B 2020-08-07T15:05:58.851 INFO:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.Fr54EG6ifm 2020-08-07T15:05:58.866 INFO:teuthology.orchestra.run.smithi096:> mktemp 2020-08-07T15:05:58.921 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.SWIfHTlDdG 2020-08-07T15:05:58.922 INFO:teuthology.orchestra.run.smithi096:> sudo cp /var/lib/ceph/osd/ceph-5/keyring /tmp/tmp.SWIfHTlDdG 2020-08-07T15:05:58.990 INFO:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.SWIfHTlDdG 2020-08-07T15:05:59.126 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.SWIfHTlDdG is 56B 2020-08-07T15:05:59.204 INFO:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.SWIfHTlDdG 2020-08-07T15:05:59.220 INFO:teuthology.orchestra.run.smithi096:> mktemp 2020-08-07T15:05:59.279 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.ELK02wMiSB 2020-08-07T15:05:59.280 INFO:teuthology.orchestra.run.smithi096:> sudo cp /var/lib/ceph/osd/ceph-6/keyring /tmp/tmp.ELK02wMiSB 2020-08-07T15:05:59.369 INFO:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.ELK02wMiSB 2020-08-07T15:05:59.509 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.ELK02wMiSB is 56B 2020-08-07T15:05:59.602 INFO:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.ELK02wMiSB 2020-08-07T15:05:59.629 INFO:teuthology.orchestra.run.smithi096:> mktemp 2020-08-07T15:05:59.682 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.PKXbsaXaCa 2020-08-07T15:05:59.683 INFO:teuthology.orchestra.run.smithi096:> sudo cp /var/lib/ceph/osd/ceph-7/keyring /tmp/tmp.PKXbsaXaCa 2020-08-07T15:05:59.759 INFO:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.PKXbsaXaCa 2020-08-07T15:05:59.887 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.PKXbsaXaCa is 56B 2020-08-07T15:05:59.982 INFO:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.PKXbsaXaCa 2020-08-07T15:06:00.054 DEBUG:teuthology.orchestra.remote:smithi074:/etc/ceph/ceph.client.0.keyring is 59B 2020-08-07T15:06:00.129 DEBUG:teuthology.orchestra.remote:smithi096:/etc/ceph/ceph.client.1.keyring is 59B 2020-08-07T15:06:00.196 INFO:tasks.ceph:Adding keys to all mons... 2020-08-07T15:06:00.197 INFO:teuthology.orchestra.run.smithi074:> sudo tee -a /etc/ceph/ceph.keyring 2020-08-07T15:06:00.215 INFO:teuthology.orchestra.run.smithi096:> sudo tee -a /etc/ceph/ceph.keyring 2020-08-07T15:06:00.259 INFO:teuthology.orchestra.run.smithi074.stdout:[mgr.y] 2020-08-07T15:06:00.260 INFO:teuthology.orchestra.run.smithi074.stdout: key = AQC6bS1fXO0IBxAAvbMBDNyuFzGz4YlOlIPpsQ== 2020-08-07T15:06:00.260 INFO:teuthology.orchestra.run.smithi074.stdout:[osd.0] 2020-08-07T15:06:00.261 INFO:teuthology.orchestra.run.smithi074.stdout: key = AQC9bS1fG6+5GhAAbWVlEaVMYHwDcDIotaVUeg== 2020-08-07T15:06:00.261 INFO:teuthology.orchestra.run.smithi074.stdout:[osd.1] 2020-08-07T15:06:00.261 INFO:teuthology.orchestra.run.smithi074.stdout: key = AQC/bS1fV4MZOhAAw5V10BaqESxAh8ruJLOyfg== 2020-08-07T15:06:00.262 INFO:teuthology.orchestra.run.smithi074.stdout:[osd.2] 2020-08-07T15:06:00.262 INFO:teuthology.orchestra.run.smithi074.stdout: key = AQDCbS1f6rYeHBAAzBmeE1LMCB93CkR88JCRgQ== 2020-08-07T15:06:00.262 INFO:teuthology.orchestra.run.smithi074.stdout:[osd.3] 2020-08-07T15:06:00.263 INFO:teuthology.orchestra.run.smithi074.stdout: key = AQDEbS1fw17yOhAAu0/O/6g92XhEFAKXSlwbtA== 2020-08-07T15:06:00.263 INFO:teuthology.orchestra.run.smithi074.stdout:[mgr.x] 2020-08-07T15:06:00.282 INFO:teuthology.orchestra.run.smithi074.stdout: key = AQC6bS1fGKHaCxAAobz9ougTfbLNniprfW3bwQ== 2020-08-07T15:06:00.282 INFO:teuthology.orchestra.run.smithi074.stdout:[osd.4] 2020-08-07T15:06:00.283 INFO:teuthology.orchestra.run.smithi074.stdout: key = AQDKbS1fsRjEGRAAZIoaEoGhUCj/M3HOc7zV8g== 2020-08-07T15:06:00.283 INFO:teuthology.orchestra.run.smithi074.stdout:[osd.5] 2020-08-07T15:06:00.283 INFO:teuthology.orchestra.run.smithi074.stdout: key = AQDMbS1f5mFPORAAjGu2k0tpNbHlNuu/M/41IA== 2020-08-07T15:06:00.283 INFO:teuthology.orchestra.run.smithi074.stdout:[osd.6] 2020-08-07T15:06:00.283 INFO:teuthology.orchestra.run.smithi074.stdout: key = AQDPbS1ftqYzGhAAukZSnPxzcH8SAiyNxnkcgA== 2020-08-07T15:06:00.284 INFO:teuthology.orchestra.run.smithi074.stdout:[osd.7] 2020-08-07T15:06:00.284 INFO:teuthology.orchestra.run.smithi074.stdout: key = AQDRbS1fsuHtNRAAGJAYXdvZqdTQPx0A5T8I3w== 2020-08-07T15:06:00.284 INFO:teuthology.orchestra.run.smithi074.stdout:[client.0] 2020-08-07T15:06:00.284 INFO:teuthology.orchestra.run.smithi074.stdout: key = AQC6bS1fcMCPDxAAz8ygh5CjaooLyRPQ140zLQ== 2020-08-07T15:06:00.285 INFO:teuthology.orchestra.run.smithi074.stdout:[client.1] 2020-08-07T15:06:00.285 INFO:teuthology.orchestra.run.smithi074.stdout: key = AQC6bS1fEE9yFBAAhfbW8dk+EuW4MpSu91VhYg== 2020-08-07T15:06:00.286 INFO:teuthology.orchestra.run.smithi096.stdout:[mgr.y] 2020-08-07T15:06:00.286 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQC6bS1fXO0IBxAAvbMBDNyuFzGz4YlOlIPpsQ== 2020-08-07T15:06:00.286 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.0] 2020-08-07T15:06:00.287 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQC9bS1fG6+5GhAAbWVlEaVMYHwDcDIotaVUeg== 2020-08-07T15:06:00.287 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.1] 2020-08-07T15:06:00.287 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQC/bS1fV4MZOhAAw5V10BaqESxAh8ruJLOyfg== 2020-08-07T15:06:00.287 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.2] 2020-08-07T15:06:00.287 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQDCbS1f6rYeHBAAzBmeE1LMCB93CkR88JCRgQ== 2020-08-07T15:06:00.288 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.3] 2020-08-07T15:06:00.288 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQDEbS1fw17yOhAAu0/O/6g92XhEFAKXSlwbtA== 2020-08-07T15:06:00.288 INFO:teuthology.orchestra.run.smithi096.stdout:[mgr.x] 2020-08-07T15:06:00.288 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQC6bS1fGKHaCxAAobz9ougTfbLNniprfW3bwQ== 2020-08-07T15:06:00.288 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.4] 2020-08-07T15:06:00.320 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQDKbS1fsRjEGRAAZIoaEoGhUCj/M3HOc7zV8g== 2020-08-07T15:06:00.321 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.5] 2020-08-07T15:06:00.321 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQDMbS1f5mFPORAAjGu2k0tpNbHlNuu/M/41IA== 2020-08-07T15:06:00.322 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.6] 2020-08-07T15:06:00.322 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQDPbS1ftqYzGhAAukZSnPxzcH8SAiyNxnkcgA== 2020-08-07T15:06:00.322 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.7] 2020-08-07T15:06:00.339 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQDRbS1fsuHtNRAAGJAYXdvZqdTQPx0A5T8I3w== 2020-08-07T15:06:00.339 INFO:teuthology.orchestra.run.smithi096.stdout:[client.0] 2020-08-07T15:06:00.340 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQC6bS1fcMCPDxAAz8ygh5CjaooLyRPQ140zLQ== 2020-08-07T15:06:00.340 INFO:teuthology.orchestra.run.smithi096.stdout:[client.1] 2020-08-07T15:06:00.340 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQC6bS1fEE9yFBAAhfbW8dk+EuW4MpSu91VhYg== 2020-08-07T15:06:00.342 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-08-07T15:06:00.384 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-08-07T15:06:00.451 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-08-07T15:06:00.462 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-08-07T15:06:00.511 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-08-07T15:06:00.520 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-08-07T15:06:00.600 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-08-07T15:06:00.618 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-08-07T15:06:00.691 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-08-07T15:06:00.723 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-08-07T15:06:00.826 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-08-07T15:06:00.831 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-08-07T15:06:00.919 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-08-07T15:06:00.934 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-08-07T15:06:01.015 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-08-07T15:06:01.027 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-08-07T15:06:01.077 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-08-07T15:06:01.080 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-08-07T15:06:01.135 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-08-07T15:06:01.155 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-08-07T15:06:01.227 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-08-07T15:06:01.242 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-08-07T15:06:01.297 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-08-07T15:06:01.320 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-08-07T15:06:01.376 INFO:tasks.ceph:Running mkfs on mon nodes... 2020-08-07T15:06:01.377 INFO:teuthology.orchestra.run.smithi074:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2020-08-07T15:06:01.430 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-08-07T15:06:01.528 INFO:teuthology.orchestra.run.smithi074:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2020-08-07T15:06:01.566 INFO:teuthology.orchestra.run.smithi074:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2020-08-07T15:06:01.638 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-08-07T15:06:01.734 INFO:teuthology.orchestra.run.smithi074:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2020-08-07T15:06:01.773 INFO:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2020-08-07T15:06:01.803 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-08-07T15:06:01.903 INFO:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2020-08-07T15:06:01.940 INFO:teuthology.orchestra.run.smithi074:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-08-07T15:06:01.950 INFO:teuthology.orchestra.run.smithi096:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-08-07T15:06:01.999 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2020-08-07T15:06:02.000 INFO:tasks.ceph.mon.a:Restarting daemon 2020-08-07T15:06:02.002 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2020-08-07T15:06:02.028 INFO:tasks.ceph.mon.a:Started 2020-08-07T15:06:02.029 INFO:tasks.ceph.mon.c:Restarting daemon 2020-08-07T15:06:02.029 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2020-08-07T15:06:02.036 INFO:tasks.ceph.mon.c:Started 2020-08-07T15:06:02.037 INFO:tasks.ceph.mon.b:Restarting daemon 2020-08-07T15:06:02.037 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2020-08-07T15:06:02.046 INFO:tasks.ceph.mon.b:Started 2020-08-07T15:06:02.047 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2020-08-07T15:06:02.055 INFO:tasks.ceph.mgr.y:Restarting daemon 2020-08-07T15:06:02.056 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2020-08-07T15:06:02.068 INFO:tasks.ceph.mgr.y:Started 2020-08-07T15:06:02.068 INFO:tasks.ceph.mgr.x:Restarting daemon 2020-08-07T15:06:02.069 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2020-08-07T15:06:02.071 INFO:tasks.ceph.mgr.x:Started 2020-08-07T15:06:02.072 INFO:tasks.ceph:Setting crush tunables to default 2020-08-07T15:06:02.072 INFO:teuthology.orchestra.run.smithi074:> sudo ceph --cluster ceph osd crush tunables default 2020-08-07T15:06:07.494 INFO:teuthology.orchestra.run.smithi074.stderr:adjusted tunables profile to default 2020-08-07T15:06:07.507 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2020-08-07T15:06:07.508 INFO:teuthology.orchestra.run.smithi074:> mktemp 2020-08-07T15:06:07.527 INFO:teuthology.orchestra.run.smithi074.stdout:/tmp/tmp.lnGhAVVDcj 2020-08-07T15:06:07.528 INFO:teuthology.orchestra.run.smithi074:> sudo cp /var/lib/ceph/osd/ceph-0/fsid /tmp/tmp.lnGhAVVDcj 2020-08-07T15:06:07.599 INFO:teuthology.orchestra.run.smithi074:> sudo chmod 0666 /tmp/tmp.lnGhAVVDcj 2020-08-07T15:06:07.754 DEBUG:teuthology.orchestra.remote:smithi074:/tmp/tmp.lnGhAVVDcj is 37B 2020-08-07T15:06:07.829 INFO:teuthology.orchestra.run.smithi074:> rm -fr /tmp/tmp.lnGhAVVDcj 2020-08-07T15:06:07.855 INFO:teuthology.orchestra.run.smithi074:> mktemp 2020-08-07T15:06:07.911 INFO:teuthology.orchestra.run.smithi074.stdout:/tmp/tmp.68t7AoGz0d 2020-08-07T15:06:07.911 INFO:teuthology.orchestra.run.smithi074:> sudo cp /var/lib/ceph/osd/ceph-1/fsid /tmp/tmp.68t7AoGz0d 2020-08-07T15:06:07.984 INFO:teuthology.orchestra.run.smithi074:> sudo chmod 0666 /tmp/tmp.68t7AoGz0d 2020-08-07T15:06:08.138 DEBUG:teuthology.orchestra.remote:smithi074:/tmp/tmp.68t7AoGz0d is 37B 2020-08-07T15:06:08.226 INFO:teuthology.orchestra.run.smithi074:> rm -fr /tmp/tmp.68t7AoGz0d 2020-08-07T15:06:08.265 INFO:teuthology.orchestra.run.smithi074:> mktemp 2020-08-07T15:06:08.312 INFO:teuthology.orchestra.run.smithi074.stdout:/tmp/tmp.Dl2KiY0ouw 2020-08-07T15:06:08.313 INFO:teuthology.orchestra.run.smithi074:> sudo cp /var/lib/ceph/osd/ceph-2/fsid /tmp/tmp.Dl2KiY0ouw 2020-08-07T15:06:08.382 INFO:teuthology.orchestra.run.smithi074:> sudo chmod 0666 /tmp/tmp.Dl2KiY0ouw 2020-08-07T15:06:08.533 DEBUG:teuthology.orchestra.remote:smithi074:/tmp/tmp.Dl2KiY0ouw is 37B 2020-08-07T15:06:08.614 INFO:teuthology.orchestra.run.smithi074:> rm -fr /tmp/tmp.Dl2KiY0ouw 2020-08-07T15:06:08.635 INFO:teuthology.orchestra.run.smithi074:> mktemp 2020-08-07T15:06:08.688 INFO:teuthology.orchestra.run.smithi074.stdout:/tmp/tmp.3JDETzYFCh 2020-08-07T15:06:08.689 INFO:teuthology.orchestra.run.smithi074:> sudo cp /var/lib/ceph/osd/ceph-3/fsid /tmp/tmp.3JDETzYFCh 2020-08-07T15:06:08.757 INFO:teuthology.orchestra.run.smithi074:> sudo chmod 0666 /tmp/tmp.3JDETzYFCh 2020-08-07T15:06:08.894 DEBUG:teuthology.orchestra.remote:smithi074:/tmp/tmp.3JDETzYFCh is 37B 2020-08-07T15:06:08.956 INFO:teuthology.orchestra.run.smithi074:> rm -fr /tmp/tmp.3JDETzYFCh 2020-08-07T15:06:08.985 INFO:teuthology.orchestra.run.smithi096:> mktemp 2020-08-07T15:06:09.009 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.qMuty8rIqW 2020-08-07T15:06:09.010 INFO:teuthology.orchestra.run.smithi096:> sudo cp /var/lib/ceph/osd/ceph-4/fsid /tmp/tmp.qMuty8rIqW 2020-08-07T15:06:09.078 INFO:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.qMuty8rIqW 2020-08-07T15:06:09.215 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.qMuty8rIqW is 37B 2020-08-07T15:06:09.309 INFO:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.qMuty8rIqW 2020-08-07T15:06:09.334 INFO:teuthology.orchestra.run.smithi096:> mktemp 2020-08-07T15:06:09.383 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.Nuyk1VsuvS 2020-08-07T15:06:09.396 INFO:teuthology.orchestra.run.smithi096:> sudo cp /var/lib/ceph/osd/ceph-5/fsid /tmp/tmp.Nuyk1VsuvS 2020-08-07T15:06:09.453 INFO:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.Nuyk1VsuvS 2020-08-07T15:06:09.598 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.Nuyk1VsuvS is 37B 2020-08-07T15:06:09.695 INFO:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.Nuyk1VsuvS 2020-08-07T15:06:09.736 INFO:teuthology.orchestra.run.smithi096:> mktemp 2020-08-07T15:06:09.795 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.f3dlF3BDZf 2020-08-07T15:06:09.796 INFO:teuthology.orchestra.run.smithi096:> sudo cp /var/lib/ceph/osd/ceph-6/fsid /tmp/tmp.f3dlF3BDZf 2020-08-07T15:06:09.874 INFO:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.f3dlF3BDZf 2020-08-07T15:06:10.048 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.f3dlF3BDZf is 37B 2020-08-07T15:06:10.120 INFO:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.f3dlF3BDZf 2020-08-07T15:06:10.188 INFO:teuthology.orchestra.run.smithi096:> mktemp 2020-08-07T15:06:10.247 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.05arYrsOa8 2020-08-07T15:06:10.247 INFO:teuthology.orchestra.run.smithi096:> sudo cp /var/lib/ceph/osd/ceph-7/fsid /tmp/tmp.05arYrsOa8 2020-08-07T15:06:10.339 INFO:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.05arYrsOa8 2020-08-07T15:06:10.498 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.05arYrsOa8 is 37B 2020-08-07T15:06:10.591 INFO:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.05arYrsOa8 2020-08-07T15:06:10.649 INFO:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd new fc1ebb23-3ef4-4fc0-b395-382e385885a3 0 2020-08-07T15:06:10.882 INFO:teuthology.orchestra.run.smithi096.stdout:0 2020-08-07T15:06:10.896 INFO:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd new 1fbe530a-63ce-4dc8-82ab-18793d6ebc0b 1 2020-08-07T15:06:11.108 INFO:teuthology.orchestra.run.smithi096.stdout:1 2020-08-07T15:06:11.113 INFO:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd new d6fe4338-3e11-4f73-9cbb-e1b046ce286a 2 2020-08-07T15:06:11.316 INFO:teuthology.orchestra.run.smithi096.stdout:2 2020-08-07T15:06:11.338 INFO:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd new bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9 3 2020-08-07T15:06:11.546 INFO:teuthology.orchestra.run.smithi096.stdout:3 2020-08-07T15:06:11.563 INFO:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd new 01aadc13-29c5-4ddd-9bb9-de24b2b20412 4 2020-08-07T15:06:11.878 INFO:teuthology.orchestra.run.smithi096.stdout:4 2020-08-07T15:06:11.889 INFO:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd new f5556a82-73b4-47b1-8e29-1d2fe031d8fd 5 2020-08-07T15:06:12.178 INFO:teuthology.orchestra.run.smithi096.stdout:5 2020-08-07T15:06:12.192 INFO:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd new 9b98b337-e1cc-43cd-9e9b-f21ae3f20899 6 2020-08-07T15:06:12.490 INFO:teuthology.orchestra.run.smithi096.stdout:6 2020-08-07T15:06:12.503 INFO:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd new 777662d8-bd34-48e8-83e1-24ce9e61f14b 7 2020-08-07T15:06:12.791 INFO:teuthology.orchestra.run.smithi096.stdout:7 2020-08-07T15:06:12.815 INFO:tasks.ceph.osd.0:Restarting daemon 2020-08-07T15:06:12.815 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2020-08-07T15:06:12.822 INFO:tasks.ceph.osd.0:Started 2020-08-07T15:06:12.822 INFO:tasks.ceph.osd.1:Restarting daemon 2020-08-07T15:06:12.822 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2020-08-07T15:06:12.829 INFO:tasks.ceph.osd.1:Started 2020-08-07T15:06:12.829 INFO:tasks.ceph.osd.2:Restarting daemon 2020-08-07T15:06:12.830 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2020-08-07T15:06:12.838 INFO:tasks.ceph.osd.2:Started 2020-08-07T15:06:12.838 INFO:tasks.ceph.osd.3:Restarting daemon 2020-08-07T15:06:12.839 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2020-08-07T15:06:12.862 INFO:tasks.ceph.osd.3:Started 2020-08-07T15:06:12.863 INFO:tasks.ceph.osd.4:Restarting daemon 2020-08-07T15:06:12.863 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2020-08-07T15:06:12.882 INFO:tasks.ceph.osd.4:Started 2020-08-07T15:06:12.883 INFO:tasks.ceph.osd.5:Restarting daemon 2020-08-07T15:06:12.883 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2020-08-07T15:06:12.889 INFO:tasks.ceph.osd.5:Started 2020-08-07T15:06:12.890 INFO:tasks.ceph.osd.6:Restarting daemon 2020-08-07T15:06:12.890 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2020-08-07T15:06:12.895 INFO:tasks.ceph.osd.6:Started 2020-08-07T15:06:12.895 INFO:tasks.ceph.osd.7:Restarting daemon 2020-08-07T15:06:12.896 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2020-08-07T15:06:12.911 INFO:tasks.ceph.osd.7:Started 2020-08-07T15:06:12.911 INFO:tasks.ceph:Waiting for OSDs to come up 2020-08-07T15:06:13.444 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:13.441+0000 7ff492b1bec0 -1 Falling back to public interface 2020-08-07T15:06:13.444 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:13.442+0000 7f693ddcbec0 -1 Falling back to public interface 2020-08-07T15:06:13.448 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:06:13.446+0000 7f23dfe7fec0 -1 Falling back to public interface 2020-08-07T15:06:13.452 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:06:13.450+0000 7f322a338ec0 -1 Falling back to public interface 2020-08-07T15:06:13.485 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:13.483+0000 7fcd8dccaec0 -1 Falling back to public interface 2020-08-07T15:06:13.498 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:13.497+0000 7fd448599ec0 -1 Falling back to public interface 2020-08-07T15:06:13.510 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:13.501+0000 7fa8c1a7bec0 -1 Falling back to public interface 2020-08-07T15:06:13.511 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:13.509+0000 7f4e8fd50ec0 -1 Falling back to public interface 2020-08-07T15:06:13.726 INFO:teuthology.orchestra.run.smithi074:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-08-07T15:06:14.024 INFO:teuthology.misc.health.smithi074.stdout: 2020-08-07T15:06:14.032 INFO:teuthology.misc.health.smithi074.stdout:{"epoch":10,"fsid":"a01d3e82-8d24-4288-9e42-576ab3ff4957","created":"2020-08-07T15:06:07.342912+0000","modified":"2020-08-07T15:06:12.787353+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-08-07T15:06:10.648876+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"4","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"fc1ebb23-3ef4-4fc0-b395-382e385885a3","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1fbe530a-63ce-4dc8-82ab-18793d6ebc0b","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d6fe4338-3e11-4f73-9cbb-e1b046ce286a","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"01aadc13-29c5-4ddd-9bb9-de24b2b20412","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f5556a82-73b4-47b1-8e29-1d2fe031d8fd","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9b98b337-e1cc-43cd-9e9b-f21ae3f20899","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"777662d8-bd34-48e8-83e1-24ce9e61f14b","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-08-07T15:06:14.033 DEBUG:teuthology.misc:0 of 8 OSDs are up 2020-08-07T15:06:14.283 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:14.281+0000 7ff492b1bec0 -1 osd.0 0 log_to_monitors {default=true} 2020-08-07T15:06:14.340 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:14.337+0000 7fa8c1a7bec0 -1 osd.6 0 log_to_monitors {default=true} 2020-08-07T15:06:14.519 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:14.515+0000 7f693ddcbec0 -1 osd.2 0 log_to_monitors {default=true} 2020-08-07T15:06:14.538 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:06:14.533+0000 7f23dfe7fec0 -1 osd.1 0 log_to_monitors {default=true} 2020-08-07T15:06:14.554 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:06:14.539+0000 7f322a338ec0 -1 osd.3 0 log_to_monitors {default=true} 2020-08-07T15:06:14.555 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:14.553+0000 7fcd8dccaec0 -1 osd.4 0 log_to_monitors {default=true} 2020-08-07T15:06:14.582 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:14.568+0000 7fd448599ec0 -1 osd.5 0 log_to_monitors {default=true} 2020-08-07T15:06:14.595 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:14.590+0000 7f4e8fd50ec0 -1 osd.7 0 log_to_monitors {default=true} 2020-08-07T15:06:15.686 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:15.682+0000 7f6935ddc700 -1 osd.2 0 waiting for initial osdmap 2020-08-07T15:06:15.687 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:06:15.683+0000 7f3220b46700 -1 osd.3 0 waiting for initial osdmap 2020-08-07T15:06:15.687 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:15.681+0000 7fd43eda7700 -1 osd.5 0 waiting for initial osdmap 2020-08-07T15:06:15.687 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:15.682+0000 7f4e8655e700 -1 osd.7 0 waiting for initial osdmap 2020-08-07T15:06:15.688 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:15.683+0000 7fcd844d8700 -1 osd.4 0 waiting for initial osdmap 2020-08-07T15:06:15.694 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:15.689+0000 7ff489329700 -1 osd.0 0 waiting for initial osdmap 2020-08-07T15:06:15.701 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:15.699+0000 7ff489329700 -1 osd.0 0 waiting for initial osdmap 2020-08-07T15:06:16.320 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:06:16.312+0000 7f23d668d700 -1 osd.1 0 waiting for initial osdmap 2020-08-07T15:06:16.689 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:16.682+0000 7fa8b9a8c700 -1 osd.6 0 waiting for initial osdmap 2020-08-07T15:06:20.867 INFO:teuthology.orchestra.run.smithi074:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-08-07T15:06:21.162 INFO:teuthology.misc.health.smithi074.stdout: 2020-08-07T15:06:21.163 INFO:teuthology.misc.health.smithi074.stdout:{"epoch":16,"fsid":"a01d3e82-8d24-4288-9e42-576ab3ff4957","created":"2020-08-07T15:06:07.342912+0000","modified":"2020-08-07T15:06:19.690723+0000","last_up_change":"2020-08-07T15:06:17.674292+0000","last_in_change":"2020-08-07T15:06:17.674292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-08-07T15:06:10.648876+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"fc1ebb23-3ef4-4fc0-b395-382e385885a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6801","nonce":31577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6802","nonce":31577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6804","nonce":31577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6803","nonce":31577}]},"public_addr":"172.21.15.74:6801/31577","cluster_addr":"172.21.15.74:6802/31577","heartbeat_back_addr":"172.21.15.74:6804/31577","heartbeat_front_addr":"172.21.15.74:6803/31577","state":["exists","up"]},{"osd":1,"uuid":"1fbe530a-63ce-4dc8-82ab-18793d6ebc0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6809","nonce":31578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6810","nonce":31578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6812","nonce":31578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6811","nonce":31578}]},"public_addr":"172.21.15.74:6809/31578","cluster_addr":"172.21.15.74:6810/31578","heartbeat_back_addr":"172.21.15.74:6812/31578","heartbeat_front_addr":"172.21.15.74:6811/31578","state":["exists","up"]},{"osd":2,"uuid":"d6fe4338-3e11-4f73-9cbb-e1b046ce286a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6805","nonce":31576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6806","nonce":31576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6808","nonce":31576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6807","nonce":31576}]},"public_addr":"172.21.15.74:6805/31576","cluster_addr":"172.21.15.74:6806/31576","heartbeat_back_addr":"172.21.15.74:6808/31576","heartbeat_front_addr":"172.21.15.74:6807/31576","state":["exists","up"]},{"osd":3,"uuid":"bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6813","nonce":31579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6814","nonce":31579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6816","nonce":31579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6815","nonce":31579}]},"public_addr":"172.21.15.74:6813/31579","cluster_addr":"172.21.15.74:6814/31579","heartbeat_back_addr":"172.21.15.74:6816/31579","heartbeat_front_addr":"172.21.15.74:6815/31579","state":["exists","up"]},{"osd":4,"uuid":"01aadc13-29c5-4ddd-9bb9-de24b2b20412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":31545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6801","nonce":31545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6803","nonce":31545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":31545}]},"public_addr":"172.21.15.96:6800/31545","cluster_addr":"172.21.15.96:6801/31545","heartbeat_back_addr":"172.21.15.96:6803/31545","heartbeat_front_addr":"172.21.15.96:6802/31545","state":["exists","up"]},{"osd":5,"uuid":"f5556a82-73b4-47b1-8e29-1d2fe031d8fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":31547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6805","nonce":31547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6807","nonce":31547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":31547}]},"public_addr":"172.21.15.96:6804/31547","cluster_addr":"172.21.15.96:6805/31547","heartbeat_back_addr":"172.21.15.96:6807/31547","heartbeat_front_addr":"172.21.15.96:6806/31547","state":["exists","up"]},{"osd":6,"uuid":"9b98b337-e1cc-43cd-9e9b-f21ae3f20899","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":31546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":31546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":31546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":31546}]},"public_addr":"172.21.15.96:6808/31546","cluster_addr":"172.21.15.96:6809/31546","heartbeat_back_addr":"172.21.15.96:6811/31546","heartbeat_front_addr":"172.21.15.96:6810/31546","state":["exists","up"]},{"osd":7,"uuid":"777662d8-bd34-48e8-83e1-24ce9e61f14b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":31570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":31570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":31570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":31570}]},"public_addr":"172.21.15.96:6812/31570","cluster_addr":"172.21.15.96:6813/31570","heartbeat_back_addr":"172.21.15.96:6815/31570","heartbeat_front_addr":"172.21.15.96:6814/31570","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.298063+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.577450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.542874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.493720+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.534281+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.601875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.340718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.549109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-08-07T15:06:21.174 DEBUG:teuthology.misc:8 of 8 OSDs are up 2020-08-07T15:06:21.175 INFO:tasks.ceph:Creating RBD pool 2020-08-07T15:06:21.175 INFO:teuthology.orchestra.run.smithi074:> sudo ceph --cluster ceph osd pool create rbd 8 2020-08-07T15:06:22.075 INFO:teuthology.orchestra.run.smithi074.stderr:pool 'rbd' created 2020-08-07T15:06:22.091 INFO:teuthology.orchestra.run.smithi074:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2020-08-07T15:06:23.078 INFO:teuthology.orchestra.run.smithi074.stderr:enabled application 'rbd' on pool 'rbd' 2020-08-07T15:06:23.084 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2020-08-07T15:06:23.085 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-08-07T15:06:23.097 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2020-08-07T15:06:23.372 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:06:23.372 INFO:teuthology.orchestra.run.smithi074.stdout:{"epoch":18,"fsid":"a01d3e82-8d24-4288-9e42-576ab3ff4957","created":"2020-08-07T15:06:07.342912+0000","modified":"2020-08-07T15:06:23.063560+0000","last_up_change":"2020-08-07T15:06:17.674292+0000","last_in_change":"2020-08-07T15:06:17.674292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-08-07T15:06:10.648876+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-08-07T15:06:21.474648+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"fc1ebb23-3ef4-4fc0-b395-382e385885a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6801","nonce":31577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6802","nonce":31577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6804","nonce":31577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6803","nonce":31577}]},"public_addr":"172.21.15.74:6801/31577","cluster_addr":"172.21.15.74:6802/31577","heartbeat_back_addr":"172.21.15.74:6804/31577","heartbeat_front_addr":"172.21.15.74:6803/31577","state":["exists","up"]},{"osd":1,"uuid":"1fbe530a-63ce-4dc8-82ab-18793d6ebc0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6809","nonce":31578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6810","nonce":31578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6812","nonce":31578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6811","nonce":31578}]},"public_addr":"172.21.15.74:6809/31578","cluster_addr":"172.21.15.74:6810/31578","heartbeat_back_addr":"172.21.15.74:6812/31578","heartbeat_front_addr":"172.21.15.74:6811/31578","state":["exists","up"]},{"osd":2,"uuid":"d6fe4338-3e11-4f73-9cbb-e1b046ce286a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6805","nonce":31576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6806","nonce":31576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6808","nonce":31576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6807","nonce":31576}]},"public_addr":"172.21.15.74:6805/31576","cluster_addr":"172.21.15.74:6806/31576","heartbeat_back_addr":"172.21.15.74:6808/31576","heartbeat_front_addr":"172.21.15.74:6807/31576","state":["exists","up"]},{"osd":3,"uuid":"bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6813","nonce":31579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6814","nonce":31579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6816","nonce":31579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6815","nonce":31579}]},"public_addr":"172.21.15.74:6813/31579","cluster_addr":"172.21.15.74:6814/31579","heartbeat_back_addr":"172.21.15.74:6816/31579","heartbeat_front_addr":"172.21.15.74:6815/31579","state":["exists","up"]},{"osd":4,"uuid":"01aadc13-29c5-4ddd-9bb9-de24b2b20412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":31545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6801","nonce":31545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6803","nonce":31545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":31545}]},"public_addr":"172.21.15.96:6800/31545","cluster_addr":"172.21.15.96:6801/31545","heartbeat_back_addr":"172.21.15.96:6803/31545","heartbeat_front_addr":"172.21.15.96:6802/31545","state":["exists","up"]},{"osd":5,"uuid":"f5556a82-73b4-47b1-8e29-1d2fe031d8fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":31547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6805","nonce":31547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6807","nonce":31547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":31547}]},"public_addr":"172.21.15.96:6804/31547","cluster_addr":"172.21.15.96:6805/31547","heartbeat_back_addr":"172.21.15.96:6807/31547","heartbeat_front_addr":"172.21.15.96:6806/31547","state":["exists","up"]},{"osd":6,"uuid":"9b98b337-e1cc-43cd-9e9b-f21ae3f20899","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":31546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":31546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":31546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":31546}]},"public_addr":"172.21.15.96:6808/31546","cluster_addr":"172.21.15.96:6809/31546","heartbeat_back_addr":"172.21.15.96:6811/31546","heartbeat_front_addr":"172.21.15.96:6810/31546","state":["exists","up"]},{"osd":7,"uuid":"777662d8-bd34-48e8-83e1-24ce9e61f14b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":31570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":31570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":31570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":31570}]},"public_addr":"172.21.15.96:6812/31570","cluster_addr":"172.21.15.96:6813/31570","heartbeat_back_addr":"172.21.15.96:6815/31570","heartbeat_front_addr":"172.21.15.96:6814/31570","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.298063+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.577450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.542874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.493720+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.534281+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.601875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.340718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.549109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-08-07T15:06:23.384 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-08-07T15:06:10.648876+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-08-07T15:06:21.474648+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-08-07T15:06:23.386 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get device_health_metrics pg_num 2020-08-07T15:06:23.659 INFO:teuthology.orchestra.run.smithi074.stdout:pg_num: 1 2020-08-07T15:06:23.670 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool get rbd pg_num 2020-08-07T15:06:23.949 INFO:teuthology.orchestra.run.smithi074.stdout:pg_num: 8 2020-08-07T15:06:23.961 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2020-08-07T15:06:23.961 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2020-08-07T15:06:23.962 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early mgr dump --format=json 2020-08-07T15:06:24.289 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:06:24.303 INFO:teuthology.orchestra.run.smithi074.stdout:{"epoch":4,"active_gid":4106,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6800","nonce":30982}]},"active_addr":"172.21.15.74:6800/30982","active_change":"2020-08-07T15:06:10.607956+0000","active_mgr_features":4540138292837744639,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540138292837744639,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","osd_support","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","osd_support","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.74:0","nonce":4228972323}]},{"addrvec":[{"type":"v2","addr":"172.21.15.74:0","nonce":1685009765}]},{"addrvec":[{"type":"v2","addr":"172.21.15.74:0","nonce":1216705024}]}]}} 2020-08-07T15:06:24.309 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2020-08-07T15:06:24.309 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2020-08-07T15:06:24.316 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-08-07T15:06:24.593 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:06:24.594 INFO:teuthology.orchestra.run.smithi074.stdout:{"epoch":19,"fsid":"a01d3e82-8d24-4288-9e42-576ab3ff4957","created":"2020-08-07T15:06:07.342912+0000","modified":"2020-08-07T15:06:24.069155+0000","last_up_change":"2020-08-07T15:06:17.674292+0000","last_in_change":"2020-08-07T15:06:17.674292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-08-07T15:06:10.648876+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-08-07T15:06:21.474648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"fc1ebb23-3ef4-4fc0-b395-382e385885a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6801","nonce":31577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6802","nonce":31577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6804","nonce":31577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6803","nonce":31577}]},"public_addr":"172.21.15.74:6801/31577","cluster_addr":"172.21.15.74:6802/31577","heartbeat_back_addr":"172.21.15.74:6804/31577","heartbeat_front_addr":"172.21.15.74:6803/31577","state":["exists","up"]},{"osd":1,"uuid":"1fbe530a-63ce-4dc8-82ab-18793d6ebc0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6809","nonce":31578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6810","nonce":31578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6812","nonce":31578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6811","nonce":31578}]},"public_addr":"172.21.15.74:6809/31578","cluster_addr":"172.21.15.74:6810/31578","heartbeat_back_addr":"172.21.15.74:6812/31578","heartbeat_front_addr":"172.21.15.74:6811/31578","state":["exists","up"]},{"osd":2,"uuid":"d6fe4338-3e11-4f73-9cbb-e1b046ce286a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6805","nonce":31576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6806","nonce":31576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6808","nonce":31576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6807","nonce":31576}]},"public_addr":"172.21.15.74:6805/31576","cluster_addr":"172.21.15.74:6806/31576","heartbeat_back_addr":"172.21.15.74:6808/31576","heartbeat_front_addr":"172.21.15.74:6807/31576","state":["exists","up"]},{"osd":3,"uuid":"bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6813","nonce":31579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6814","nonce":31579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6816","nonce":31579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6815","nonce":31579}]},"public_addr":"172.21.15.74:6813/31579","cluster_addr":"172.21.15.74:6814/31579","heartbeat_back_addr":"172.21.15.74:6816/31579","heartbeat_front_addr":"172.21.15.74:6815/31579","state":["exists","up"]},{"osd":4,"uuid":"01aadc13-29c5-4ddd-9bb9-de24b2b20412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":31545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6801","nonce":31545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6803","nonce":31545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":31545}]},"public_addr":"172.21.15.96:6800/31545","cluster_addr":"172.21.15.96:6801/31545","heartbeat_back_addr":"172.21.15.96:6803/31545","heartbeat_front_addr":"172.21.15.96:6802/31545","state":["exists","up"]},{"osd":5,"uuid":"f5556a82-73b4-47b1-8e29-1d2fe031d8fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":31547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6805","nonce":31547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6807","nonce":31547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":31547}]},"public_addr":"172.21.15.96:6804/31547","cluster_addr":"172.21.15.96:6805/31547","heartbeat_back_addr":"172.21.15.96:6807/31547","heartbeat_front_addr":"172.21.15.96:6806/31547","state":["exists","up"]},{"osd":6,"uuid":"9b98b337-e1cc-43cd-9e9b-f21ae3f20899","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":31546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":31546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":31546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":31546}]},"public_addr":"172.21.15.96:6808/31546","cluster_addr":"172.21.15.96:6809/31546","heartbeat_back_addr":"172.21.15.96:6811/31546","heartbeat_front_addr":"172.21.15.96:6810/31546","state":["exists","up"]},{"osd":7,"uuid":"777662d8-bd34-48e8-83e1-24ce9e61f14b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":31570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":31570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":31570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":31570}]},"public_addr":"172.21.15.96:6812/31570","cluster_addr":"172.21.15.96:6813/31570","heartbeat_back_addr":"172.21.15.96:6815/31570","heartbeat_front_addr":"172.21.15.96:6814/31570","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.298063+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.577450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.542874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.493720+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.534281+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.601875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.340718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.549109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-08-07T15:06:24.605 INFO:tasks.ceph.ceph_manager.ceph:all up! 2020-08-07T15:06:24.605 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-08-07T15:06:24.876 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:06:24.877 INFO:teuthology.orchestra.run.smithi074.stdout:{"epoch":19,"fsid":"a01d3e82-8d24-4288-9e42-576ab3ff4957","created":"2020-08-07T15:06:07.342912+0000","modified":"2020-08-07T15:06:24.069155+0000","last_up_change":"2020-08-07T15:06:17.674292+0000","last_in_change":"2020-08-07T15:06:17.674292+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-08-07T15:06:10.648876+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-08-07T15:06:21.474648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"fc1ebb23-3ef4-4fc0-b395-382e385885a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6801","nonce":31577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6802","nonce":31577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6804","nonce":31577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6803","nonce":31577}]},"public_addr":"172.21.15.74:6801/31577","cluster_addr":"172.21.15.74:6802/31577","heartbeat_back_addr":"172.21.15.74:6804/31577","heartbeat_front_addr":"172.21.15.74:6803/31577","state":["exists","up"]},{"osd":1,"uuid":"1fbe530a-63ce-4dc8-82ab-18793d6ebc0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6809","nonce":31578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6810","nonce":31578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6812","nonce":31578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6811","nonce":31578}]},"public_addr":"172.21.15.74:6809/31578","cluster_addr":"172.21.15.74:6810/31578","heartbeat_back_addr":"172.21.15.74:6812/31578","heartbeat_front_addr":"172.21.15.74:6811/31578","state":["exists","up"]},{"osd":2,"uuid":"d6fe4338-3e11-4f73-9cbb-e1b046ce286a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6805","nonce":31576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6806","nonce":31576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6808","nonce":31576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6807","nonce":31576}]},"public_addr":"172.21.15.74:6805/31576","cluster_addr":"172.21.15.74:6806/31576","heartbeat_back_addr":"172.21.15.74:6808/31576","heartbeat_front_addr":"172.21.15.74:6807/31576","state":["exists","up"]},{"osd":3,"uuid":"bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6813","nonce":31579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6814","nonce":31579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6816","nonce":31579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6815","nonce":31579}]},"public_addr":"172.21.15.74:6813/31579","cluster_addr":"172.21.15.74:6814/31579","heartbeat_back_addr":"172.21.15.74:6816/31579","heartbeat_front_addr":"172.21.15.74:6815/31579","state":["exists","up"]},{"osd":4,"uuid":"01aadc13-29c5-4ddd-9bb9-de24b2b20412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":31545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6801","nonce":31545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6803","nonce":31545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":31545}]},"public_addr":"172.21.15.96:6800/31545","cluster_addr":"172.21.15.96:6801/31545","heartbeat_back_addr":"172.21.15.96:6803/31545","heartbeat_front_addr":"172.21.15.96:6802/31545","state":["exists","up"]},{"osd":5,"uuid":"f5556a82-73b4-47b1-8e29-1d2fe031d8fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":31547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6805","nonce":31547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6807","nonce":31547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":31547}]},"public_addr":"172.21.15.96:6804/31547","cluster_addr":"172.21.15.96:6805/31547","heartbeat_back_addr":"172.21.15.96:6807/31547","heartbeat_front_addr":"172.21.15.96:6806/31547","state":["exists","up"]},{"osd":6,"uuid":"9b98b337-e1cc-43cd-9e9b-f21ae3f20899","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":31546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":31546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":31546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":31546}]},"public_addr":"172.21.15.96:6808/31546","cluster_addr":"172.21.15.96:6809/31546","heartbeat_back_addr":"172.21.15.96:6811/31546","heartbeat_front_addr":"172.21.15.96:6810/31546","state":["exists","up"]},{"osd":7,"uuid":"777662d8-bd34-48e8-83e1-24ce9e61f14b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":31570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":31570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":31570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":31570}]},"public_addr":"172.21.15.96:6812/31570","cluster_addr":"172.21.15.96:6813/31570","heartbeat_back_addr":"172.21.15.96:6815/31570","heartbeat_front_addr":"172.21.15.96:6814/31570","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.298063+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.577450+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.542874+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.493720+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.534281+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.601875+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.340718+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:06:15.549109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-08-07T15:06:24.887 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.0 flush_pg_stats 2020-08-07T15:06:25.207 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:06:25.200+0000 7f354cc53700 0 --2- 172.21.15.74:0/1686044262 >> [v2:172.21.15.74:3301/0,v1:172.21.15.74:6790/0] conn(0x7f35480a1db0 0x7f354809fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2020-08-07T15:06:25.245 INFO:teuthology.orchestra.run.smithi074.stdout:55834574852 2020-08-07T15:06:25.246 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.1 flush_pg_stats 2020-08-07T15:06:25.426 INFO:teuthology.orchestra.run.smithi074.stdout:55834574851 2020-08-07T15:06:25.427 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.2 flush_pg_stats 2020-08-07T15:06:25.615 INFO:teuthology.orchestra.run.smithi074.stdout:55834574851 2020-08-07T15:06:25.616 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.3 flush_pg_stats 2020-08-07T15:06:25.783 INFO:teuthology.orchestra.run.smithi074.stdout:55834574851 2020-08-07T15:06:25.783 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.4 flush_pg_stats 2020-08-07T15:06:25.939 INFO:teuthology.orchestra.run.smithi074.stdout:55834574851 2020-08-07T15:06:25.939 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.5 flush_pg_stats 2020-08-07T15:06:26.105 INFO:teuthology.orchestra.run.smithi074.stdout:55834574851 2020-08-07T15:06:26.106 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.6 flush_pg_stats 2020-08-07T15:06:26.260 INFO:teuthology.orchestra.run.smithi074.stdout:60129542147 2020-08-07T15:06:26.261 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early tell osd.7 flush_pg_stats 2020-08-07T15:06:26.423 INFO:teuthology.orchestra.run.smithi074.stdout:55834574852 2020-08-07T15:06:26.423 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.0 2020-08-07T15:06:26.695 INFO:teuthology.orchestra.run.smithi074.stdout:55834574851 2020-08-07T15:06:26.735 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.0 2020-08-07T15:06:27.752 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.0 2020-08-07T15:06:28.035 INFO:teuthology.orchestra.run.smithi074.stdout:55834574852 2020-08-07T15:06:28.050 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.0 2020-08-07T15:06:28.051 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.1 2020-08-07T15:06:28.324 INFO:teuthology.orchestra.run.smithi074.stdout:55834574851 2020-08-07T15:06:28.346 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.1 2020-08-07T15:06:28.346 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.2 2020-08-07T15:06:28.626 INFO:teuthology.orchestra.run.smithi074.stdout:55834574851 2020-08-07T15:06:28.637 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.2 2020-08-07T15:06:28.638 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.3 2020-08-07T15:06:28.911 INFO:teuthology.orchestra.run.smithi074.stdout:55834574851 2020-08-07T15:06:28.922 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.3 2020-08-07T15:06:28.923 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.4 2020-08-07T15:06:29.194 INFO:teuthology.orchestra.run.smithi074.stdout:55834574852 2020-08-07T15:06:29.206 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.4 2020-08-07T15:06:29.207 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.5 2020-08-07T15:06:29.493 INFO:teuthology.orchestra.run.smithi074.stdout:55834574852 2020-08-07T15:06:29.500 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.5 2020-08-07T15:06:29.501 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.6 2020-08-07T15:06:29.778 INFO:teuthology.orchestra.run.smithi074.stdout:60129542148 2020-08-07T15:06:29.784 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542148 for osd.6 2020-08-07T15:06:29.786 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd last-stat-seq osd.7 2020-08-07T15:06:30.054 INFO:teuthology.orchestra.run.smithi074.stdout:55834574853 2020-08-07T15:06:30.070 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.7 2020-08-07T15:06:30.070 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2020-08-07T15:06:30.071 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:06:30.340 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:06:30.341 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:06:30.350 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2020-08-07T15:06:28.613745+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8389932,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746584788,"statfs":{"total":773094113280,"available":764502822912,"internally_reserved":8589934592,"allocated":1290240,"data_stored":473368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.547831"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.074419+0000","last_change":"2020-08-07T15:06:23.073702+0000","last_active":"2020-08-07T15:06:23.074419+0000","last_peered":"2020-08-07T15:06:23.074419+0000","last_clean":"2020-08-07T15:06:23.074419+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:06:23.074419+0000","last_undegraded":"2020-08-07T15:06:23.074419+0000","last_fullsized":"2020-08-07T15:06:23.074419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.074727+0000","last_change":"2020-08-07T15:06:23.074076+0000","last_active":"2020-08-07T15:06:23.074727+0000","last_peered":"2020-08-07T15:06:23.074727+0000","last_clean":"2020-08-07T15:06:23.074727+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:06:23.074727+0000","last_undegraded":"2020-08-07T15:06:23.074727+0000","last_fullsized":"2020-08-07T15:06:23.074727+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.226496+0000","last_change":"2020-08-07T15:06:23.225886+0000","last_active":"2020-08-07T15:06:23.226496+0000","last_peered":"2020-08-07T15:06:23.226496+0000","last_clean":"2020-08-07T15:06:23.226496+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:06:23.226496+0000","last_undegraded":"2020-08-07T15:06:23.226496+0000","last_fullsized":"2020-08-07T15:06:23.226496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.224609+0000","last_change":"2020-08-07T15:06:23.224115+0000","last_active":"2020-08-07T15:06:23.224609+0000","last_peered":"2020-08-07T15:06:23.224609+0000","last_clean":"2020-08-07T15:06:23.224609+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:06:23.224609+0000","last_undegraded":"2020-08-07T15:06:23.224609+0000","last_fullsized":"2020-08-07T15:06:23.224609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"17","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.069375+0000","last_change":"2020-08-07T15:06:19.198215+0000","last_active":"2020-08-07T15:06:23.069375+0000","last_peered":"2020-08-07T15:06:23.069375+0000","last_clean":"2020-08-07T15:06:23.069375+0000","last_became_active":"2020-08-07T15:06:19.197819+0000","last_became_peered":"2020-08-07T15:06:19.197819+0000","last_unstale":"2020-08-07T15:06:23.069375+0000","last_undegraded":"2020-08-07T15:06:23.069375+0000","last_fullsized":"2020-08-07T15:06:23.069375+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:06:10.878833+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.076769+0000","last_change":"2020-08-07T15:06:23.075958+0000","last_active":"2020-08-07T15:06:23.076769+0000","last_peered":"2020-08-07T15:06:23.076769+0000","last_clean":"2020-08-07T15:06:23.076769+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:06:23.076769+0000","last_undegraded":"2020-08-07T15:06:23.076769+0000","last_fullsized":"2020-08-07T15:06:23.076769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.224396+0000","last_change":"2020-08-07T15:06:23.223924+0000","last_active":"2020-08-07T15:06:23.224396+0000","last_peered":"2020-08-07T15:06:23.224396+0000","last_clean":"2020-08-07T15:06:23.224396+0000","last_became_active":"2020-08-07T15:06:23.223489+0000","last_became_peered":"2020-08-07T15:06:23.223489+0000","last_unstale":"2020-08-07T15:06:23.224396+0000","last_undegraded":"2020-08-07T15:06:23.224396+0000","last_fullsized":"2020-08-07T15:06:23.224396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.075437+0000","last_change":"2020-08-07T15:06:23.074962+0000","last_active":"2020-08-07T15:06:23.075437+0000","last_peered":"2020-08-07T15:06:23.075437+0000","last_clean":"2020-08-07T15:06:23.075437+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:06:23.075437+0000","last_undegraded":"2020-08-07T15:06:23.075437+0000","last_fullsized":"2020-08-07T15:06:23.075437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.224960+0000","last_change":"2020-08-07T15:06:23.224263+0000","last_active":"2020-08-07T15:06:23.224960+0000","last_peered":"2020-08-07T15:06:23.224960+0000","last_clean":"2020-08-07T15:06:23.224960+0000","last_became_active":"2020-08-07T15:06:23.223840+0000","last_became_peered":"2020-08-07T15:06:23.223840+0000","last_unstale":"2020-08-07T15:06:23.224960+0000","last_undegraded":"2020-08-07T15:06:23.224960+0000","last_fullsized":"2020-08-07T15:06:23.224960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048744,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323096,"statfs":{"total":96636764160,"available":95562850304,"internally_reserved":1073741824,"allocated":163840,"data_stored":59626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]}]},{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048744,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323096,"statfs":{"total":96636764160,"available":95562850304,"internally_reserved":1073741824,"allocated":163840,"data_stored":59626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048740,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323100,"statfs":{"total":96636764160,"available":95562854400,"internally_reserved":1073741824,"allocated":159744,"data_stored":58798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048740,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323100,"statfs":{"total":96636764160,"available":95562854400,"internally_reserved":1073741824,"allocated":159744,"data_stored":59048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.272}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3080000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048736,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323104,"statfs":{"total":96636764160,"available":95562858496,"internally_reserved":1073741824,"allocated":155648,"data_stored":58220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048740,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323100,"statfs":{"total":96636764160,"available":95562854400,"internally_reserved":1073741824,"allocated":159744,"data_stored":58798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4770000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7490000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048744,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323096,"statfs":{"total":96636764160,"available":95562850304,"internally_reserved":1073741824,"allocated":163840,"data_stored":59626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4490000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4650000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048744,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323096,"statfs":{"total":96636764160,"available":95562850304,"internally_reserved":1073741824,"allocated":163840,"data_stored":59626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.242}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]}]}]}} 2020-08-07T15:06:30.352 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:06:30.695 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:06:30.696 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:06:30.706 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2020-08-07T15:06:30.614080+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8389932,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746584788,"statfs":{"total":773094113280,"available":764502822912,"internally_reserved":8589934592,"allocated":1290240,"data_stored":473368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001265"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.074419+0000","last_change":"2020-08-07T15:06:23.073702+0000","last_active":"2020-08-07T15:06:23.074419+0000","last_peered":"2020-08-07T15:06:23.074419+0000","last_clean":"2020-08-07T15:06:23.074419+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:06:23.074419+0000","last_undegraded":"2020-08-07T15:06:23.074419+0000","last_fullsized":"2020-08-07T15:06:23.074419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.074727+0000","last_change":"2020-08-07T15:06:23.074076+0000","last_active":"2020-08-07T15:06:23.074727+0000","last_peered":"2020-08-07T15:06:23.074727+0000","last_clean":"2020-08-07T15:06:23.074727+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:06:23.074727+0000","last_undegraded":"2020-08-07T15:06:23.074727+0000","last_fullsized":"2020-08-07T15:06:23.074727+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.226496+0000","last_change":"2020-08-07T15:06:23.225886+0000","last_active":"2020-08-07T15:06:23.226496+0000","last_peered":"2020-08-07T15:06:23.226496+0000","last_clean":"2020-08-07T15:06:23.226496+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:06:23.226496+0000","last_undegraded":"2020-08-07T15:06:23.226496+0000","last_fullsized":"2020-08-07T15:06:23.226496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.224609+0000","last_change":"2020-08-07T15:06:23.224115+0000","last_active":"2020-08-07T15:06:23.224609+0000","last_peered":"2020-08-07T15:06:23.224609+0000","last_clean":"2020-08-07T15:06:23.224609+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:06:23.224609+0000","last_undegraded":"2020-08-07T15:06:23.224609+0000","last_fullsized":"2020-08-07T15:06:23.224609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"17","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.069375+0000","last_change":"2020-08-07T15:06:19.198215+0000","last_active":"2020-08-07T15:06:23.069375+0000","last_peered":"2020-08-07T15:06:23.069375+0000","last_clean":"2020-08-07T15:06:23.069375+0000","last_became_active":"2020-08-07T15:06:19.197819+0000","last_became_peered":"2020-08-07T15:06:19.197819+0000","last_unstale":"2020-08-07T15:06:23.069375+0000","last_undegraded":"2020-08-07T15:06:23.069375+0000","last_fullsized":"2020-08-07T15:06:23.069375+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:06:10.878833+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.076769+0000","last_change":"2020-08-07T15:06:23.075958+0000","last_active":"2020-08-07T15:06:23.076769+0000","last_peered":"2020-08-07T15:06:23.076769+0000","last_clean":"2020-08-07T15:06:23.076769+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:06:23.076769+0000","last_undegraded":"2020-08-07T15:06:23.076769+0000","last_fullsized":"2020-08-07T15:06:23.076769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.224396+0000","last_change":"2020-08-07T15:06:23.223924+0000","last_active":"2020-08-07T15:06:23.224396+0000","last_peered":"2020-08-07T15:06:23.224396+0000","last_clean":"2020-08-07T15:06:23.224396+0000","last_became_active":"2020-08-07T15:06:23.223489+0000","last_became_peered":"2020-08-07T15:06:23.223489+0000","last_unstale":"2020-08-07T15:06:23.224396+0000","last_undegraded":"2020-08-07T15:06:23.224396+0000","last_fullsized":"2020-08-07T15:06:23.224396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.075437+0000","last_change":"2020-08-07T15:06:23.074962+0000","last_active":"2020-08-07T15:06:23.075437+0000","last_peered":"2020-08-07T15:06:23.075437+0000","last_clean":"2020-08-07T15:06:23.075437+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:06:23.075437+0000","last_undegraded":"2020-08-07T15:06:23.075437+0000","last_fullsized":"2020-08-07T15:06:23.075437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.224960+0000","last_change":"2020-08-07T15:06:23.224263+0000","last_active":"2020-08-07T15:06:23.224960+0000","last_peered":"2020-08-07T15:06:23.224960+0000","last_clean":"2020-08-07T15:06:23.224960+0000","last_became_active":"2020-08-07T15:06:23.223840+0000","last_became_peered":"2020-08-07T15:06:23.223840+0000","last_unstale":"2020-08-07T15:06:23.224960+0000","last_undegraded":"2020-08-07T15:06:23.224960+0000","last_fullsized":"2020-08-07T15:06:23.224960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048744,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323096,"statfs":{"total":96636764160,"available":95562850304,"internally_reserved":1073741824,"allocated":163840,"data_stored":59626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]}]},{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048744,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323096,"statfs":{"total":96636764160,"available":95562850304,"internally_reserved":1073741824,"allocated":163840,"data_stored":59626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048740,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323100,"statfs":{"total":96636764160,"available":95562854400,"internally_reserved":1073741824,"allocated":159744,"data_stored":58798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048740,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323100,"statfs":{"total":96636764160,"available":95562854400,"internally_reserved":1073741824,"allocated":159744,"data_stored":59048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.272}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3080000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048736,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323104,"statfs":{"total":96636764160,"available":95562858496,"internally_reserved":1073741824,"allocated":155648,"data_stored":58220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048740,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323100,"statfs":{"total":96636764160,"available":95562854400,"internally_reserved":1073741824,"allocated":159744,"data_stored":58798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4770000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7490000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048744,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323096,"statfs":{"total":96636764160,"available":95562850304,"internally_reserved":1073741824,"allocated":163840,"data_stored":59626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4490000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4650000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048744,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323096,"statfs":{"total":96636764160,"available":95562850304,"internally_reserved":1073741824,"allocated":163840,"data_stored":59626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.242}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]}]}]}} 2020-08-07T15:06:30.709 INFO:tasks.ceph.ceph_manager.ceph:clean! 2020-08-07T15:06:30.709 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2020-08-07T15:06:30.710 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2020-08-07T15:06:30.710 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early health --format=json 2020-08-07T15:06:31.039 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:06:31.051 INFO:teuthology.orchestra.run.smithi074.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2020-08-07T15:06:31.051 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2020-08-07T15:06:31.052 INFO:teuthology.run_tasks:Running task thrashosds... 2020-08-07T15:06:31.074 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2020-08-07T15:06:31.075 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-shuzhenyi-testing-2020-08-06-1028'}, 'ceph': {'conf': {'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'mon client directed command retry': 5, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'osd', 'ms inject internal delays': 0.002, 'ms inject socket failures': 2500, 'ms type': 'async', 'osd_async_recovery_min_cost': 1, 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_object_clean_region_max_num_intervals': 1000, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'fs': 'xfs', 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request'], 'sha1': 'e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'sha1': 'e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5, 'min_in': 4}, 'workunit': {'branch': 'wip-shuzhenyi-testing-2020-08-06-1028', 'sha1': 'e19d2f2a44224dee0a6f38ef5aaeaafb12bdb41d'}} 2020-08-07T15:06:31.076 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5, 'min_in': 4} 2020-08-07T15:06:31.076 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early config set mgr mgr_debug_aggressive_pg_num_changes true 2020-08-07T15:06:31.373 INFO:tasks.thrashosds:Beginning thrashosds... 2020-08-07T15:06:31.375 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2020-08-07T15:06:31.375 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:06:31.644 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:06:31.650 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:06:31.655 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2020-08-07T15:06:30.614080+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8389932,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746584788,"statfs":{"total":773094113280,"available":764502822912,"internally_reserved":8589934592,"allocated":1290240,"data_stored":473368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001265"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.074419+0000","last_change":"2020-08-07T15:06:23.073702+0000","last_active":"2020-08-07T15:06:23.074419+0000","last_peered":"2020-08-07T15:06:23.074419+0000","last_clean":"2020-08-07T15:06:23.074419+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:06:23.074419+0000","last_undegraded":"2020-08-07T15:06:23.074419+0000","last_fullsized":"2020-08-07T15:06:23.074419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.074727+0000","last_change":"2020-08-07T15:06:23.074076+0000","last_active":"2020-08-07T15:06:23.074727+0000","last_peered":"2020-08-07T15:06:23.074727+0000","last_clean":"2020-08-07T15:06:23.074727+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:06:23.074727+0000","last_undegraded":"2020-08-07T15:06:23.074727+0000","last_fullsized":"2020-08-07T15:06:23.074727+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.226496+0000","last_change":"2020-08-07T15:06:23.225886+0000","last_active":"2020-08-07T15:06:23.226496+0000","last_peered":"2020-08-07T15:06:23.226496+0000","last_clean":"2020-08-07T15:06:23.226496+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:06:23.226496+0000","last_undegraded":"2020-08-07T15:06:23.226496+0000","last_fullsized":"2020-08-07T15:06:23.226496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.224609+0000","last_change":"2020-08-07T15:06:23.224115+0000","last_active":"2020-08-07T15:06:23.224609+0000","last_peered":"2020-08-07T15:06:23.224609+0000","last_clean":"2020-08-07T15:06:23.224609+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:06:23.224609+0000","last_undegraded":"2020-08-07T15:06:23.224609+0000","last_fullsized":"2020-08-07T15:06:23.224609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"17","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.069375+0000","last_change":"2020-08-07T15:06:19.198215+0000","last_active":"2020-08-07T15:06:23.069375+0000","last_peered":"2020-08-07T15:06:23.069375+0000","last_clean":"2020-08-07T15:06:23.069375+0000","last_became_active":"2020-08-07T15:06:19.197819+0000","last_became_peered":"2020-08-07T15:06:19.197819+0000","last_unstale":"2020-08-07T15:06:23.069375+0000","last_undegraded":"2020-08-07T15:06:23.069375+0000","last_fullsized":"2020-08-07T15:06:23.069375+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:06:10.878833+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.076769+0000","last_change":"2020-08-07T15:06:23.075958+0000","last_active":"2020-08-07T15:06:23.076769+0000","last_peered":"2020-08-07T15:06:23.076769+0000","last_clean":"2020-08-07T15:06:23.076769+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:06:23.076769+0000","last_undegraded":"2020-08-07T15:06:23.076769+0000","last_fullsized":"2020-08-07T15:06:23.076769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.224396+0000","last_change":"2020-08-07T15:06:23.223924+0000","last_active":"2020-08-07T15:06:23.224396+0000","last_peered":"2020-08-07T15:06:23.224396+0000","last_clean":"2020-08-07T15:06:23.224396+0000","last_became_active":"2020-08-07T15:06:23.223489+0000","last_became_peered":"2020-08-07T15:06:23.223489+0000","last_unstale":"2020-08-07T15:06:23.224396+0000","last_undegraded":"2020-08-07T15:06:23.224396+0000","last_fullsized":"2020-08-07T15:06:23.224396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.075437+0000","last_change":"2020-08-07T15:06:23.074962+0000","last_active":"2020-08-07T15:06:23.075437+0000","last_peered":"2020-08-07T15:06:23.075437+0000","last_clean":"2020-08-07T15:06:23.075437+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:06:23.075437+0000","last_undegraded":"2020-08-07T15:06:23.075437+0000","last_fullsized":"2020-08-07T15:06:23.075437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.224960+0000","last_change":"2020-08-07T15:06:23.224263+0000","last_active":"2020-08-07T15:06:23.224960+0000","last_peered":"2020-08-07T15:06:23.224960+0000","last_clean":"2020-08-07T15:06:23.224960+0000","last_became_active":"2020-08-07T15:06:23.223840+0000","last_became_peered":"2020-08-07T15:06:23.223840+0000","last_unstale":"2020-08-07T15:06:23.224960+0000","last_undegraded":"2020-08-07T15:06:23.224960+0000","last_fullsized":"2020-08-07T15:06:23.224960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048744,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323096,"statfs":{"total":96636764160,"available":95562850304,"internally_reserved":1073741824,"allocated":163840,"data_stored":59626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]}]},{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048744,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323096,"statfs":{"total":96636764160,"available":95562850304,"internally_reserved":1073741824,"allocated":163840,"data_stored":59626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048740,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323100,"statfs":{"total":96636764160,"available":95562854400,"internally_reserved":1073741824,"allocated":159744,"data_stored":58798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048740,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323100,"statfs":{"total":96636764160,"available":95562854400,"internally_reserved":1073741824,"allocated":159744,"data_stored":59048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.272}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3080000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048736,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323104,"statfs":{"total":96636764160,"available":95562858496,"internally_reserved":1073741824,"allocated":155648,"data_stored":58220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048740,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323100,"statfs":{"total":96636764160,"available":95562854400,"internally_reserved":1073741824,"allocated":159744,"data_stored":58798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4770000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7490000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048744,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323096,"statfs":{"total":96636764160,"available":95562850304,"internally_reserved":1073741824,"allocated":163840,"data_stored":59626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4490000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4650000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048744,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323096,"statfs":{"total":96636764160,"available":95562850304,"internally_reserved":1073741824,"allocated":163840,"data_stored":59626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.242}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]}]}]}} 2020-08-07T15:06:31.657 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:06:31.934 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:06:31.936 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:06:31.945 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2020-08-07T15:06:30.614080+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8389932,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746584788,"statfs":{"total":773094113280,"available":764502822912,"internally_reserved":8589934592,"allocated":1290240,"data_stored":473368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001265"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.074419+0000","last_change":"2020-08-07T15:06:23.073702+0000","last_active":"2020-08-07T15:06:23.074419+0000","last_peered":"2020-08-07T15:06:23.074419+0000","last_clean":"2020-08-07T15:06:23.074419+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:06:23.074419+0000","last_undegraded":"2020-08-07T15:06:23.074419+0000","last_fullsized":"2020-08-07T15:06:23.074419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.074727+0000","last_change":"2020-08-07T15:06:23.074076+0000","last_active":"2020-08-07T15:06:23.074727+0000","last_peered":"2020-08-07T15:06:23.074727+0000","last_clean":"2020-08-07T15:06:23.074727+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:06:23.074727+0000","last_undegraded":"2020-08-07T15:06:23.074727+0000","last_fullsized":"2020-08-07T15:06:23.074727+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.226496+0000","last_change":"2020-08-07T15:06:23.225886+0000","last_active":"2020-08-07T15:06:23.226496+0000","last_peered":"2020-08-07T15:06:23.226496+0000","last_clean":"2020-08-07T15:06:23.226496+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:06:23.226496+0000","last_undegraded":"2020-08-07T15:06:23.226496+0000","last_fullsized":"2020-08-07T15:06:23.226496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.224609+0000","last_change":"2020-08-07T15:06:23.224115+0000","last_active":"2020-08-07T15:06:23.224609+0000","last_peered":"2020-08-07T15:06:23.224609+0000","last_clean":"2020-08-07T15:06:23.224609+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:06:23.224609+0000","last_undegraded":"2020-08-07T15:06:23.224609+0000","last_fullsized":"2020-08-07T15:06:23.224609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"17","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.069375+0000","last_change":"2020-08-07T15:06:19.198215+0000","last_active":"2020-08-07T15:06:23.069375+0000","last_peered":"2020-08-07T15:06:23.069375+0000","last_clean":"2020-08-07T15:06:23.069375+0000","last_became_active":"2020-08-07T15:06:19.197819+0000","last_became_peered":"2020-08-07T15:06:19.197819+0000","last_unstale":"2020-08-07T15:06:23.069375+0000","last_undegraded":"2020-08-07T15:06:23.069375+0000","last_fullsized":"2020-08-07T15:06:23.069375+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:06:10.878833+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.076769+0000","last_change":"2020-08-07T15:06:23.075958+0000","last_active":"2020-08-07T15:06:23.076769+0000","last_peered":"2020-08-07T15:06:23.076769+0000","last_clean":"2020-08-07T15:06:23.076769+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:06:23.076769+0000","last_undegraded":"2020-08-07T15:06:23.076769+0000","last_fullsized":"2020-08-07T15:06:23.076769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.224396+0000","last_change":"2020-08-07T15:06:23.223924+0000","last_active":"2020-08-07T15:06:23.224396+0000","last_peered":"2020-08-07T15:06:23.224396+0000","last_clean":"2020-08-07T15:06:23.224396+0000","last_became_active":"2020-08-07T15:06:23.223489+0000","last_became_peered":"2020-08-07T15:06:23.223489+0000","last_unstale":"2020-08-07T15:06:23.224396+0000","last_undegraded":"2020-08-07T15:06:23.224396+0000","last_fullsized":"2020-08-07T15:06:23.224396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.075437+0000","last_change":"2020-08-07T15:06:23.074962+0000","last_active":"2020-08-07T15:06:23.075437+0000","last_peered":"2020-08-07T15:06:23.075437+0000","last_clean":"2020-08-07T15:06:23.075437+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:06:23.075437+0000","last_undegraded":"2020-08-07T15:06:23.075437+0000","last_fullsized":"2020-08-07T15:06:23.075437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2020-08-07T15:06:23.224960+0000","last_change":"2020-08-07T15:06:23.224263+0000","last_active":"2020-08-07T15:06:23.224960+0000","last_peered":"2020-08-07T15:06:23.224960+0000","last_clean":"2020-08-07T15:06:23.224960+0000","last_became_active":"2020-08-07T15:06:23.223840+0000","last_became_peered":"2020-08-07T15:06:23.223840+0000","last_unstale":"2020-08-07T15:06:23.224960+0000","last_undegraded":"2020-08-07T15:06:23.224960+0000","last_fullsized":"2020-08-07T15:06:23.224960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048744,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323096,"statfs":{"total":96636764160,"available":95562850304,"internally_reserved":1073741824,"allocated":163840,"data_stored":59626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]}]},{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048744,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323096,"statfs":{"total":96636764160,"available":95562850304,"internally_reserved":1073741824,"allocated":163840,"data_stored":59626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048740,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323100,"statfs":{"total":96636764160,"available":95562854400,"internally_reserved":1073741824,"allocated":159744,"data_stored":58798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048740,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323100,"statfs":{"total":96636764160,"available":95562854400,"internally_reserved":1073741824,"allocated":159744,"data_stored":59048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.272}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3080000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048736,"kb_used_data":152,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323104,"statfs":{"total":96636764160,"available":95562858496,"internally_reserved":1073741824,"allocated":155648,"data_stored":58220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048740,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323100,"statfs":{"total":96636764160,"available":95562854400,"internally_reserved":1073741824,"allocated":159744,"data_stored":58798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4770000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7490000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048744,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323096,"statfs":{"total":96636764160,"available":95562850304,"internally_reserved":1073741824,"allocated":163840,"data_stored":59626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4490000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4650000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048744,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323096,"statfs":{"total":96636764160,"available":95562850304,"internally_reserved":1073741824,"allocated":163840,"data_stored":59626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.242}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]}]}]}} 2020-08-07T15:06:31.947 INFO:tasks.ceph.ceph_manager.ceph:clean! 2020-08-07T15:06:31.948 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump 2020-08-07T15:06:32.217 INFO:teuthology.orchestra.run.smithi074.stdout:epoch 19 2020-08-07T15:06:32.218 INFO:teuthology.orchestra.run.smithi074.stdout:fsid a01d3e82-8d24-4288-9e42-576ab3ff4957 2020-08-07T15:06:32.218 INFO:teuthology.orchestra.run.smithi074.stdout:created 2020-08-07T15:06:07.342912+0000 2020-08-07T15:06:32.218 INFO:teuthology.orchestra.run.smithi074.stdout:modified 2020-08-07T15:06:24.069155+0000 2020-08-07T15:06:32.218 INFO:teuthology.orchestra.run.smithi074.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2020-08-07T15:06:32.219 INFO:teuthology.orchestra.run.smithi074.stdout:crush_version 5 2020-08-07T15:06:32.219 INFO:teuthology.orchestra.run.smithi074.stdout:full_ratio 0.95 2020-08-07T15:06:32.219 INFO:teuthology.orchestra.run.smithi074.stdout:backfillfull_ratio 0.9 2020-08-07T15:06:32.219 INFO:teuthology.orchestra.run.smithi074.stdout:nearfull_ratio 0.85 2020-08-07T15:06:32.219 INFO:teuthology.orchestra.run.smithi074.stdout:require_min_compat_client luminous 2020-08-07T15:06:32.219 INFO:teuthology.orchestra.run.smithi074.stdout:min_compat_client jewel 2020-08-07T15:06:32.220 INFO:teuthology.orchestra.run.smithi074.stdout:require_osd_release pacific 2020-08-07T15:06:32.220 INFO:teuthology.orchestra.run.smithi074.stdout:pool 1 'device_health_metrics' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 16 flags hashpspool stripe_width 0 pg_num_min 1 application mgr_devicehealth 2020-08-07T15:06:32.220 INFO:teuthology.orchestra.run.smithi074.stdout:pool 2 'rbd' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 19 flags hashpspool stripe_width 0 application rbd 2020-08-07T15:06:32.220 INFO:teuthology.orchestra.run.smithi074.stdout:max_osd 8 2020-08-07T15:06:32.220 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.74:6801/31577 v2:172.21.15.74:6802/31577 exists,up fc1ebb23-3ef4-4fc0-b395-382e385885a3 2020-08-07T15:06:32.231 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) v2:172.21.15.74:6809/31578 v2:172.21.15.74:6810/31578 exists,up 1fbe530a-63ce-4dc8-82ab-18793d6ebc0b 2020-08-07T15:06:32.231 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) v2:172.21.15.74:6805/31576 v2:172.21.15.74:6806/31576 exists,up d6fe4338-3e11-4f73-9cbb-e1b046ce286a 2020-08-07T15:06:32.231 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.74:6813/31579 v2:172.21.15.74:6814/31579 exists,up bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9 2020-08-07T15:06:32.231 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.96:6800/31545 v2:172.21.15.96:6801/31545 exists,up 01aadc13-29c5-4ddd-9bb9-de24b2b20412 2020-08-07T15:06:32.232 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) v2:172.21.15.96:6804/31547 v2:172.21.15.96:6805/31547 exists,up f5556a82-73b4-47b1-8e29-1d2fe031d8fd 2020-08-07T15:06:32.232 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6 up in weight 1 up_from 14 up_thru 17 down_at 0 last_clean_interval [0,0) v2:172.21.15.96:6808/31546 v2:172.21.15.96:6809/31546 exists,up 9b98b337-e1cc-43cd-9e9b-f21ae3f20899 2020-08-07T15:06:32.232 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) v2:172.21.15.96:6812/31570 v2:172.21.15.96:6813/31570 exists,up 777662d8-bd34-48e8-83e1-24ce9e61f14b 2020-08-07T15:06:32.234 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.74:6801/31577 v2:172.21.15.74:6802/31577 exists,up fc1ebb23-3ef4-4fc0-b395-382e385885a3', 'osd.1 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) v2:172.21.15.74:6809/31578 v2:172.21.15.74:6810/31578 exists,up 1fbe530a-63ce-4dc8-82ab-18793d6ebc0b', 'osd.2 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) v2:172.21.15.74:6805/31576 v2:172.21.15.74:6806/31576 exists,up d6fe4338-3e11-4f73-9cbb-e1b046ce286a', 'osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.74:6813/31579 v2:172.21.15.74:6814/31579 exists,up bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9', 'osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.96:6800/31545 v2:172.21.15.96:6801/31545 exists,up 01aadc13-29c5-4ddd-9bb9-de24b2b20412', 'osd.5 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) v2:172.21.15.96:6804/31547 v2:172.21.15.96:6805/31547 exists,up f5556a82-73b4-47b1-8e29-1d2fe031d8fd', 'osd.6 up in weight 1 up_from 14 up_thru 17 down_at 0 last_clean_interval [0,0) v2:172.21.15.96:6808/31546 v2:172.21.15.96:6809/31546 exists,up 9b98b337-e1cc-43cd-9e9b-f21ae3f20899', 'osd.7 up in weight 1 up_from 13 up_thru 17 down_at 0 last_clean_interval [0,0) v2:172.21.15.96:6812/31570 v2:172.21.15.96:6813/31570 exists,up 777662d8-bd34-48e8-83e1-24ce9e61f14b'] 2020-08-07T15:06:32.235 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2020-08-07T15:06:32.347 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:06:32.347 INFO:teuthology.orchestra.run.smithi074.stdout: "name": "mon.a", 2020-08-07T15:06:32.348 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster": "ceph", 2020-08-07T15:06:32.348 INFO:teuthology.orchestra.run.smithi074.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2020-08-07T15:06:32.348 INFO:teuthology.orchestra.run.smithi074.stdout: "admin_socket_mode": "", 2020-08-07T15:06:32.348 INFO:teuthology.orchestra.run.smithi074.stdout: "auth_client_required": "cephx, none", 2020-08-07T15:06:32.348 INFO:teuthology.orchestra.run.smithi074.stdout: "auth_cluster_required": "cephx", 2020-08-07T15:06:32.349 INFO:teuthology.orchestra.run.smithi074.stdout: "auth_debug": "true", 2020-08-07T15:06:32.349 INFO:teuthology.orchestra.run.smithi074.stdout: "auth_mon_ticket_ttl": "43200.000000", 2020-08-07T15:06:32.349 INFO:teuthology.orchestra.run.smithi074.stdout: "auth_service_required": "cephx", 2020-08-07T15:06:32.349 INFO:teuthology.orchestra.run.smithi074.stdout: "auth_service_ticket_ttl": "3600.000000", 2020-08-07T15:06:32.349 INFO:teuthology.orchestra.run.smithi074.stdout: "auth_supported": "cephx", 2020-08-07T15:06:32.351 INFO:teuthology.orchestra.run.smithi074.stdout: "bdev_aio": "true", 2020-08-07T15:06:32.351 INFO:teuthology.orchestra.run.smithi074.stdout: "bdev_aio_max_queue_depth": "1024", 2020-08-07T15:06:32.352 INFO:teuthology.orchestra.run.smithi074.stdout: "bdev_aio_poll_ms": "250", 2020-08-07T15:06:32.352 INFO:teuthology.orchestra.run.smithi074.stdout: "bdev_aio_reap_max": "16", 2020-08-07T15:06:32.352 INFO:teuthology.orchestra.run.smithi074.stdout: "bdev_async_discard": "false", 2020-08-07T15:06:32.352 INFO:teuthology.orchestra.run.smithi074.stdout: "bdev_block_size": "4096", 2020-08-07T15:06:32.352 INFO:teuthology.orchestra.run.smithi074.stdout: "bdev_debug_aio": "false", 2020-08-07T15:06:32.353 INFO:teuthology.orchestra.run.smithi074.stdout: "bdev_debug_aio_log_age": "5.000000", 2020-08-07T15:06:32.353 INFO:teuthology.orchestra.run.smithi074.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2020-08-07T15:06:32.353 INFO:teuthology.orchestra.run.smithi074.stdout: "bdev_debug_inflight_ios": "false", 2020-08-07T15:06:32.353 INFO:teuthology.orchestra.run.smithi074.stdout: "bdev_enable_discard": "false", 2020-08-07T15:06:32.354 INFO:teuthology.orchestra.run.smithi074.stdout: "bdev_inject_crash": "0", 2020-08-07T15:06:32.354 INFO:teuthology.orchestra.run.smithi074.stdout: "bdev_inject_crash_flush_delay": "2", 2020-08-07T15:06:32.354 INFO:teuthology.orchestra.run.smithi074.stdout: "bdev_nvme_retry_count": "-1", 2020-08-07T15:06:32.354 INFO:teuthology.orchestra.run.smithi074.stdout: "bdev_nvme_unbind_from_kernel": "false", 2020-08-07T15:06:32.354 INFO:teuthology.orchestra.run.smithi074.stdout: "bluefs_alloc_size": "1048576", 2020-08-07T15:06:32.355 INFO:teuthology.orchestra.run.smithi074.stdout: "bluefs_allocator": "hybrid", 2020-08-07T15:06:32.355 INFO:teuthology.orchestra.run.smithi074.stdout: "bluefs_buffered_io": "false", 2020-08-07T15:06:32.355 INFO:teuthology.orchestra.run.smithi074.stdout: "bluefs_compact_log_sync": "false", 2020-08-07T15:06:32.355 INFO:teuthology.orchestra.run.smithi074.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2020-08-07T15:06:32.359 INFO:teuthology.orchestra.run.smithi074.stdout: "bluefs_log_compact_min_size": "16777216", 2020-08-07T15:06:32.359 INFO:teuthology.orchestra.run.smithi074.stdout: "bluefs_log_replay_check_allocations": "true", 2020-08-07T15:06:32.359 INFO:teuthology.orchestra.run.smithi074.stdout: "bluefs_max_log_runway": "4194304", 2020-08-07T15:06:32.359 INFO:teuthology.orchestra.run.smithi074.stdout: "bluefs_max_prefetch": "1048576", 2020-08-07T15:06:32.360 INFO:teuthology.orchestra.run.smithi074.stdout: "bluefs_min_flush_size": "524288", 2020-08-07T15:06:32.360 INFO:teuthology.orchestra.run.smithi074.stdout: "bluefs_min_log_runway": "1048576", 2020-08-07T15:06:32.360 INFO:teuthology.orchestra.run.smithi074.stdout: "bluefs_shared_alloc_size": "65536", 2020-08-07T15:06:32.360 INFO:teuthology.orchestra.run.smithi074.stdout: "bluefs_sync_write": "false", 2020-08-07T15:06:32.360 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2020-08-07T15:06:32.360 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2020-08-07T15:06:32.361 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2020-08-07T15:06:32.361 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_allocator": "hybrid", 2020-08-07T15:06:32.369 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2020-08-07T15:06:32.369 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2020-08-07T15:06:32.369 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2020-08-07T15:06:32.370 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_bitmapallocator_span_size": "1024", 2020-08-07T15:06:32.370 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_blobid_prealloc": "10240", 2020-08-07T15:06:32.370 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_block_create": "true", 2020-08-07T15:06:32.370 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_block_db_create": "false", 2020-08-07T15:06:32.370 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_block_db_path": "", 2020-08-07T15:06:32.371 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_block_db_size": "0", 2020-08-07T15:06:32.371 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_block_path": "", 2020-08-07T15:06:32.371 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_block_preallocate_file": "false", 2020-08-07T15:06:32.371 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_block_size": "107374182400", 2020-08-07T15:06:32.371 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_block_wal_create": "false", 2020-08-07T15:06:32.372 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_block_wal_path": "", 2020-08-07T15:06:32.372 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_block_wal_size": "100663296", 2020-08-07T15:06:32.372 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_bluefs": "true", 2020-08-07T15:06:32.372 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2020-08-07T15:06:32.372 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_bluefs_balance_interval": "1.000000", 2020-08-07T15:06:32.373 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_bluefs_db_compatibility": "true", 2020-08-07T15:06:32.373 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_bluefs_env_mirror": "false", 2020-08-07T15:06:32.373 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_bluefs_gift_ratio": "0.020000", 2020-08-07T15:06:32.373 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_bluefs_max_free": "10737418240", 2020-08-07T15:06:32.373 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_bluefs_max_ratio": "0.900000", 2020-08-07T15:06:32.374 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_bluefs_min": "1073741824", 2020-08-07T15:06:32.374 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_bluefs_min_free": "1073741824", 2020-08-07T15:06:32.374 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_bluefs_min_ratio": "0.020000", 2020-08-07T15:06:32.374 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_bluefs_reclaim_ratio": "0.200000", 2020-08-07T15:06:32.374 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_cache_autotune": "true", 2020-08-07T15:06:32.374 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_cache_autotune_interval": "5.000000", 2020-08-07T15:06:32.375 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_cache_kv_ratio": "0.400000", 2020-08-07T15:06:32.375 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_cache_meta_ratio": "0.400000", 2020-08-07T15:06:32.375 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_cache_size": "0", 2020-08-07T15:06:32.375 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_cache_size_hdd": "1073741824", 2020-08-07T15:06:32.375 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_cache_size_ssd": "3221225472", 2020-08-07T15:06:32.376 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_cache_trim_interval": "0.050000", 2020-08-07T15:06:32.376 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_cache_trim_max_skip_pinned": "64", 2020-08-07T15:06:32.376 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_cache_type": "2q", 2020-08-07T15:06:32.376 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_clone_cow": "true", 2020-08-07T15:06:32.376 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_compression_algorithm": "snappy", 2020-08-07T15:06:32.377 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_compression_max_blob_size": "0", 2020-08-07T15:06:32.377 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2020-08-07T15:06:32.377 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2020-08-07T15:06:32.377 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_compression_min_blob_size": "0", 2020-08-07T15:06:32.377 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2020-08-07T15:06:32.377 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_compression_min_blob_size_ssd": "8192", 2020-08-07T15:06:32.378 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_compression_mode": "none", 2020-08-07T15:06:32.378 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_compression_required_ratio": "0.875000", 2020-08-07T15:06:32.378 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_csum_type": "crc32c", 2020-08-07T15:06:32.378 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_debug_enforce_settings": "default", 2020-08-07T15:06:32.378 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_debug_freelist": "false", 2020-08-07T15:06:32.379 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_debug_fsck_abort": "false", 2020-08-07T15:06:32.379 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_debug_inject_bug21040": "false", 2020-08-07T15:06:32.379 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2020-08-07T15:06:32.379 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_debug_inject_read_err": "false", 2020-08-07T15:06:32.379 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_debug_misc": "false", 2020-08-07T15:06:32.380 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_debug_no_reuse_blocks": "false", 2020-08-07T15:06:32.380 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_debug_omit_block_device_write": "false", 2020-08-07T15:06:32.380 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_debug_omit_kv_commit": "false", 2020-08-07T15:06:32.380 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2020-08-07T15:06:32.380 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_debug_prefill": "0.000000", 2020-08-07T15:06:32.380 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_debug_prefragment_max": "1048576", 2020-08-07T15:06:32.381 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_debug_random_read_err": "0.000000", 2020-08-07T15:06:32.381 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2020-08-07T15:06:32.381 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_debug_small_allocations": "0", 2020-08-07T15:06:32.381 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2020-08-07T15:06:32.381 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_default_buffered_read": "true", 2020-08-07T15:06:32.382 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_default_buffered_write": "false", 2020-08-07T15:06:32.382 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_deferred_batch_ops": "0", 2020-08-07T15:06:32.382 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2020-08-07T15:06:32.382 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2020-08-07T15:06:32.382 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2020-08-07T15:06:32.383 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_extent_map_shard_max_size": "1200", 2020-08-07T15:06:32.383 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_extent_map_shard_min_size": "150", 2020-08-07T15:06:32.383 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_extent_map_shard_target_size": "500", 2020-08-07T15:06:32.383 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2020-08-07T15:06:32.383 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_freelist_blocks_per_key": "128", 2020-08-07T15:06:32.383 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2020-08-07T15:06:32.384 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2020-08-07T15:06:32.384 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_fsck_on_mkfs": "true", 2020-08-07T15:06:32.384 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2020-08-07T15:06:32.384 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_fsck_on_mount": "false", 2020-08-07T15:06:32.384 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_fsck_on_mount_deep": "false", 2020-08-07T15:06:32.385 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_fsck_on_umount": "false", 2020-08-07T15:06:32.385 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_fsck_on_umount_deep": "false", 2020-08-07T15:06:32.385 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_fsck_quick_fix_on_mount": "true", 2020-08-07T15:06:32.385 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_fsck_quick_fix_threads": "2", 2020-08-07T15:06:32.385 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2020-08-07T15:06:32.386 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_gc_enable_blob_threshold": "0", 2020-08-07T15:06:32.386 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_gc_enable_total_threshold": "0", 2020-08-07T15:06:32.386 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2020-08-07T15:06:32.386 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_ignore_data_csum": "false", 2020-08-07T15:06:32.386 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_ioring": "false", 2020-08-07T15:06:32.386 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_kvbackend": "rocksdb", 2020-08-07T15:06:32.387 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_log_collection_list_age": "60.000000", 2020-08-07T15:06:32.387 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2020-08-07T15:06:32.387 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_log_op_age": "5.000000", 2020-08-07T15:06:32.387 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_max_alloc_size": "0", 2020-08-07T15:06:32.387 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_max_blob_size": "0", 2020-08-07T15:06:32.388 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_max_blob_size_hdd": "65536", 2020-08-07T15:06:32.388 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_max_blob_size_ssd": "65536", 2020-08-07T15:06:32.388 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_max_defer_interval": "3.000000", 2020-08-07T15:06:32.389 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_max_deferred_txc": "32", 2020-08-07T15:06:32.389 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_min_alloc_size": "0", 2020-08-07T15:06:32.389 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_min_alloc_size_hdd": "4096", 2020-08-07T15:06:32.389 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_min_alloc_size_ssd": "4096", 2020-08-07T15:06:32.389 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_nid_prealloc": "1024", 2020-08-07T15:06:32.390 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_prefer_deferred_size": "0", 2020-08-07T15:06:32.390 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2020-08-07T15:06:32.390 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2020-08-07T15:06:32.390 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_retry_disk_reads": "3", 2020-08-07T15:06:32.390 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_rocksdb_cf": "true", 2020-08-07T15:06:32.390 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_rocksdb_cfs": "m(3) O(3,0-13) L", 2020-08-07T15:06:32.391 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824", 2020-08-07T15:06:32.391 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_spdk_coremask": "0x1", 2020-08-07T15:06:32.391 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_spdk_io_sleep": "5", 2020-08-07T15:06:32.391 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_spdk_max_io_completion": "0", 2020-08-07T15:06:32.391 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_spdk_mem": "512", 2020-08-07T15:06:32.392 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_sync_submit_transaction": "false", 2020-08-07T15:06:32.392 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_throttle_bytes": "67108864", 2020-08-07T15:06:32.392 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_throttle_cost_per_io": "0", 2020-08-07T15:06:32.392 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2020-08-07T15:06:32.392 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2020-08-07T15:06:32.393 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2020-08-07T15:06:32.393 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_throttle_trace_rate": "0.000000", 2020-08-07T15:06:32.393 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_tracing": "false", 2020-08-07T15:06:32.393 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_volume_selection_policy": "rocksdb_original", 2020-08-07T15:06:32.393 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_volume_selection_reserved": "0", 2020-08-07T15:06:32.393 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2020-08-07T15:06:32.394 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2020-08-07T15:06:32.394 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_warn_on_legacy_statfs": "true", 2020-08-07T15:06:32.394 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2020-08-07T15:06:32.394 INFO:teuthology.orchestra.run.smithi074.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2020-08-07T15:06:32.394 INFO:teuthology.orchestra.run.smithi074.stdout: "cephadm_path": "/usr/sbin/cephadm", 2020-08-07T15:06:32.395 INFO:teuthology.orchestra.run.smithi074.stdout: "cephx_cluster_require_signatures": "false", 2020-08-07T15:06:32.395 INFO:teuthology.orchestra.run.smithi074.stdout: "cephx_cluster_require_version": "1", 2020-08-07T15:06:32.395 INFO:teuthology.orchestra.run.smithi074.stdout: "cephx_require_signatures": "false", 2020-08-07T15:06:32.395 INFO:teuthology.orchestra.run.smithi074.stdout: "cephx_require_version": "1", 2020-08-07T15:06:32.395 INFO:teuthology.orchestra.run.smithi074.stdout: "cephx_service_require_signatures": "false", 2020-08-07T15:06:32.396 INFO:teuthology.orchestra.run.smithi074.stdout: "cephx_service_require_version": "1", 2020-08-07T15:06:32.396 INFO:teuthology.orchestra.run.smithi074.stdout: "cephx_sign_messages": "true", 2020-08-07T15:06:32.396 INFO:teuthology.orchestra.run.smithi074.stdout: "chdir": "", 2020-08-07T15:06:32.396 INFO:teuthology.orchestra.run.smithi074.stdout: "client_acl_type": "", 2020-08-07T15:06:32.396 INFO:teuthology.orchestra.run.smithi074.stdout: "client_asio_thread_count": "2", 2020-08-07T15:06:32.397 INFO:teuthology.orchestra.run.smithi074.stdout: "client_cache_mid": "0.750000", 2020-08-07T15:06:32.397 INFO:teuthology.orchestra.run.smithi074.stdout: "client_cache_size": "16384", 2020-08-07T15:06:32.397 INFO:teuthology.orchestra.run.smithi074.stdout: "client_caps_release_delay": "5", 2020-08-07T15:06:32.397 INFO:teuthology.orchestra.run.smithi074.stdout: "client_check_pool_perm": "true", 2020-08-07T15:06:32.397 INFO:teuthology.orchestra.run.smithi074.stdout: "client_debug_force_sync_read": "false", 2020-08-07T15:06:32.397 INFO:teuthology.orchestra.run.smithi074.stdout: "client_debug_getattr_caps": "false", 2020-08-07T15:06:32.398 INFO:teuthology.orchestra.run.smithi074.stdout: "client_debug_inject_tick_delay": "0", 2020-08-07T15:06:32.398 INFO:teuthology.orchestra.run.smithi074.stdout: "client_die_on_failed_dentry_invalidate": "true", 2020-08-07T15:06:32.398 INFO:teuthology.orchestra.run.smithi074.stdout: "client_die_on_failed_remount": "false", 2020-08-07T15:06:32.399 INFO:teuthology.orchestra.run.smithi074.stdout: "client_dirsize_rbytes": "true", 2020-08-07T15:06:32.399 INFO:teuthology.orchestra.run.smithi074.stdout: "client_force_lazyio": "false", 2020-08-07T15:06:32.399 INFO:teuthology.orchestra.run.smithi074.stdout: "client_fs": "", 2020-08-07T15:06:32.399 INFO:teuthology.orchestra.run.smithi074.stdout: "client_inject_fixed_oldest_tid": "false", 2020-08-07T15:06:32.400 INFO:teuthology.orchestra.run.smithi074.stdout: "client_inject_release_failure": "false", 2020-08-07T15:06:32.400 INFO:teuthology.orchestra.run.smithi074.stdout: "client_max_inline_size": "4096", 2020-08-07T15:06:32.400 INFO:teuthology.orchestra.run.smithi074.stdout: "client_mds_namespace": "", 2020-08-07T15:06:32.400 INFO:teuthology.orchestra.run.smithi074.stdout: "client_metadata": "", 2020-08-07T15:06:32.400 INFO:teuthology.orchestra.run.smithi074.stdout: "client_mount_gid": "-1", 2020-08-07T15:06:32.401 INFO:teuthology.orchestra.run.smithi074.stdout: "client_mount_timeout": "300.000000", 2020-08-07T15:06:32.401 INFO:teuthology.orchestra.run.smithi074.stdout: "client_mount_uid": "-1", 2020-08-07T15:06:32.401 INFO:teuthology.orchestra.run.smithi074.stdout: "client_mountpoint": "/", 2020-08-07T15:06:32.403 INFO:teuthology.orchestra.run.smithi074.stdout: "client_notify_timeout": "10", 2020-08-07T15:06:32.403 INFO:teuthology.orchestra.run.smithi074.stdout: "client_oc": "true", 2020-08-07T15:06:32.403 INFO:teuthology.orchestra.run.smithi074.stdout: "client_oc_max_dirty": "104857600", 2020-08-07T15:06:32.404 INFO:teuthology.orchestra.run.smithi074.stdout: "client_oc_max_dirty_age": "5.000000", 2020-08-07T15:06:32.404 INFO:teuthology.orchestra.run.smithi074.stdout: "client_oc_max_objects": "1000", 2020-08-07T15:06:32.404 INFO:teuthology.orchestra.run.smithi074.stdout: "client_oc_size": "209715200", 2020-08-07T15:06:32.404 INFO:teuthology.orchestra.run.smithi074.stdout: "client_oc_target_dirty": "8388608", 2020-08-07T15:06:32.404 INFO:teuthology.orchestra.run.smithi074.stdout: "client_permissions": "true", 2020-08-07T15:06:32.405 INFO:teuthology.orchestra.run.smithi074.stdout: "client_quota_df": "true", 2020-08-07T15:06:32.405 INFO:teuthology.orchestra.run.smithi074.stdout: "client_readahead_max_bytes": "0", 2020-08-07T15:06:32.405 INFO:teuthology.orchestra.run.smithi074.stdout: "client_readahead_max_periods": "4", 2020-08-07T15:06:32.405 INFO:teuthology.orchestra.run.smithi074.stdout: "client_readahead_min": "131072", 2020-08-07T15:06:32.405 INFO:teuthology.orchestra.run.smithi074.stdout: "client_reconnect_stale": "false", 2020-08-07T15:06:32.406 INFO:teuthology.orchestra.run.smithi074.stdout: "client_shutdown_timeout": "30", 2020-08-07T15:06:32.406 INFO:teuthology.orchestra.run.smithi074.stdout: "client_snapdir": ".snap", 2020-08-07T15:06:32.418 INFO:teuthology.orchestra.run.smithi074.stdout: "client_tick_interval": "1.000000", 2020-08-07T15:06:32.418 INFO:teuthology.orchestra.run.smithi074.stdout: "client_trace": "", 2020-08-07T15:06:32.418 INFO:teuthology.orchestra.run.smithi074.stdout: "client_try_dentry_invalidate": "false", 2020-08-07T15:06:32.418 INFO:teuthology.orchestra.run.smithi074.stdout: "client_use_faked_inos": "false", 2020-08-07T15:06:32.424 INFO:teuthology.orchestra.run.smithi074.stdout: "client_use_random_mds": "false", 2020-08-07T15:06:32.424 INFO:teuthology.orchestra.run.smithi074.stdout: "clog_to_graylog": "false", 2020-08-07T15:06:32.424 INFO:teuthology.orchestra.run.smithi074.stdout: "clog_to_graylog_host": "127.0.0.1", 2020-08-07T15:06:32.424 INFO:teuthology.orchestra.run.smithi074.stdout: "clog_to_graylog_port": "12201", 2020-08-07T15:06:32.424 INFO:teuthology.orchestra.run.smithi074.stdout: "clog_to_monitors": "default=true", 2020-08-07T15:06:32.425 INFO:teuthology.orchestra.run.smithi074.stdout: "clog_to_syslog": "false", 2020-08-07T15:06:32.425 INFO:teuthology.orchestra.run.smithi074.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2020-08-07T15:06:32.425 INFO:teuthology.orchestra.run.smithi074.stdout: "clog_to_syslog_level": "info", 2020-08-07T15:06:32.425 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "-", 2020-08-07T15:06:32.426 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_network": "", 2020-08-07T15:06:32.426 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_network_interface": "", 2020-08-07T15:06:32.426 INFO:teuthology.orchestra.run.smithi074.stdout: "compressor_zlib_isal": "false", 2020-08-07T15:06:32.426 INFO:teuthology.orchestra.run.smithi074.stdout: "compressor_zlib_level": "5", 2020-08-07T15:06:32.426 INFO:teuthology.orchestra.run.smithi074.stdout: "compressor_zlib_winsize": "-15", 2020-08-07T15:06:32.427 INFO:teuthology.orchestra.run.smithi074.stdout: "compressor_zstd_level": "1", 2020-08-07T15:06:32.427 INFO:teuthology.orchestra.run.smithi074.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2020-08-07T15:06:32.427 INFO:teuthology.orchestra.run.smithi074.stdout: "crash_dir": "/var/lib/ceph/crash", 2020-08-07T15:06:32.428 INFO:teuthology.orchestra.run.smithi074.stdout: "crimson_osd_obc_lru_size": "10", 2020-08-07T15:06:32.428 INFO:teuthology.orchestra.run.smithi074.stdout: "crimson_osd_scheduler_concurrency": "0", 2020-08-07T15:06:32.428 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_location": "", 2020-08-07T15:06:32.428 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_location_hook": "", 2020-08-07T15:06:32.429 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_location_hook_timeout": "10", 2020-08-07T15:06:32.429 INFO:teuthology.orchestra.run.smithi074.stdout: "daemonize": "false", 2020-08-07T15:06:32.429 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_allow_any_pool_priority": "false", 2020-08-07T15:06:32.429 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_asok": "1/5", 2020-08-07T15:06:32.429 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_asok_assert_abort": "false", 2020-08-07T15:06:32.430 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_asserts_on_shutdown": "true", 2020-08-07T15:06:32.430 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_auth": "20/20", 2020-08-07T15:06:32.430 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_bdev": "1/3", 2020-08-07T15:06:32.440 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_bluefs": "1/5", 2020-08-07T15:06:32.441 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_bluestore": "1/5", 2020-08-07T15:06:32.441 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_buffer": "0/1", 2020-08-07T15:06:32.441 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_civetweb": "1/10", 2020-08-07T15:06:32.441 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_client": "0/5", 2020-08-07T15:06:32.441 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_compressor": "1/5", 2020-08-07T15:06:32.442 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_context": "0/1", 2020-08-07T15:06:32.442 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_crush": "1/1", 2020-08-07T15:06:32.442 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_crypto": "1/5", 2020-08-07T15:06:32.442 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_deliberately_leak_memory": "false", 2020-08-07T15:06:32.442 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_disable_randomized_ping": "false", 2020-08-07T15:06:32.442 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_dpdk": "1/5", 2020-08-07T15:06:32.443 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_eventtrace": "1/5", 2020-08-07T15:06:32.443 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_filer": "0/1", 2020-08-07T15:06:32.443 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_filestore": "1/3", 2020-08-07T15:06:32.443 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_finisher": "1/1", 2020-08-07T15:06:32.443 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_fuse": "1/5", 2020-08-07T15:06:32.444 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_heartbeat_testing_span": "0", 2020-08-07T15:06:32.444 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_heartbeatmap": "1/5", 2020-08-07T15:06:32.444 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_immutable_obj_cache": "0/5", 2020-08-07T15:06:32.444 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_javaclient": "1/5", 2020-08-07T15:06:32.444 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_journal": "1/3", 2020-08-07T15:06:32.445 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_journaler": "0/5", 2020-08-07T15:06:32.445 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_kstore": "1/5", 2020-08-07T15:06:32.445 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_leveldb": "4/5", 2020-08-07T15:06:32.445 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_lockdep": "0/1", 2020-08-07T15:06:32.445 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_mds": "1/5", 2020-08-07T15:06:32.445 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_mds_balancer": "1/5", 2020-08-07T15:06:32.446 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_mds_locker": "1/5", 2020-08-07T15:06:32.446 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_mds_log": "1/5", 2020-08-07T15:06:32.446 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_mds_log_expire": "1/5", 2020-08-07T15:06:32.446 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_mds_migrator": "1/5", 2020-08-07T15:06:32.446 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_memdb": "4/5", 2020-08-07T15:06:32.447 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_mgr": "1/5", 2020-08-07T15:06:32.447 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_mgrc": "1/5", 2020-08-07T15:06:32.447 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_mon": "20/20", 2020-08-07T15:06:32.447 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_monc": "0/10", 2020-08-07T15:06:32.447 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_ms": "1/1", 2020-08-07T15:06:32.448 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_none": "0/5", 2020-08-07T15:06:32.448 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_objclass": "0/5", 2020-08-07T15:06:32.448 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_objectcacher": "0/5", 2020-08-07T15:06:32.448 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_objecter": "0/1", 2020-08-07T15:06:32.448 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_optracker": "0/5", 2020-08-07T15:06:32.448 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_osd": "1/5", 2020-08-07T15:06:32.449 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_paxos": "20/20", 2020-08-07T15:06:32.449 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_perfcounter": "1/5", 2020-08-07T15:06:32.449 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_prioritycache": "1/5", 2020-08-07T15:06:32.449 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_rados": "0/5", 2020-08-07T15:06:32.449 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_rbd": "0/5", 2020-08-07T15:06:32.450 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_rbd_mirror": "0/5", 2020-08-07T15:06:32.451 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_rbd_replay": "0/5", 2020-08-07T15:06:32.451 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_rbd_rwl": "0/5", 2020-08-07T15:06:32.451 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_refs": "0/0", 2020-08-07T15:06:32.451 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_reserver": "1/1", 2020-08-07T15:06:32.452 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_rgw": "1/5", 2020-08-07T15:06:32.452 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_rgw_sync": "1/5", 2020-08-07T15:06:32.452 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_rocksdb": "4/5", 2020-08-07T15:06:32.452 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_striper": "0/1", 2020-08-07T15:06:32.452 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_test": "0/5", 2020-08-07T15:06:32.453 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_throttle": "1/1", 2020-08-07T15:06:32.453 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_timer": "0/1", 2020-08-07T15:06:32.453 INFO:teuthology.orchestra.run.smithi074.stdout: "debug_tp": "0/5", 2020-08-07T15:06:32.453 INFO:teuthology.orchestra.run.smithi074.stdout: "device_failure_prediction_mode": "none", 2020-08-07T15:06:32.453 INFO:teuthology.orchestra.run.smithi074.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2020-08-07T15:06:32.453 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2020-08-07T15:06:32.454 INFO:teuthology.orchestra.run.smithi074.stdout: "err_to_graylog": "false", 2020-08-07T15:06:32.454 INFO:teuthology.orchestra.run.smithi074.stdout: "err_to_stderr": "true", 2020-08-07T15:06:32.454 INFO:teuthology.orchestra.run.smithi074.stdout: "err_to_syslog": "false", 2020-08-07T15:06:32.454 INFO:teuthology.orchestra.run.smithi074.stdout: "event_tracing": "false", 2020-08-07T15:06:32.454 INFO:teuthology.orchestra.run.smithi074.stdout: "fake_statfs_for_testing": "0", 2020-08-07T15:06:32.455 INFO:teuthology.orchestra.run.smithi074.stdout: "fatal_signal_handlers": "true", 2020-08-07T15:06:32.455 INFO:teuthology.orchestra.run.smithi074.stdout: "filer_max_purge_ops": "10", 2020-08-07T15:06:32.455 INFO:teuthology.orchestra.run.smithi074.stdout: "filer_max_truncate_ops": "128", 2020-08-07T15:06:32.455 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_apply_finisher_threads": "1", 2020-08-07T15:06:32.455 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_blackhole": "false", 2020-08-07T15:06:32.456 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_btrfs_clone_range": "true", 2020-08-07T15:06:32.456 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_btrfs_snap": "true", 2020-08-07T15:06:32.456 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_caller_concurrency": "10", 2020-08-07T15:06:32.456 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_collect_device_partition_information": "true", 2020-08-07T15:06:32.456 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_commit_timeout": "600.000000", 2020-08-07T15:06:32.457 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_debug_inject_read_err": "false", 2020-08-07T15:06:32.457 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_debug_omap_check": "false", 2020-08-07T15:06:32.457 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_debug_random_read_err": "0.000000", 2020-08-07T15:06:32.457 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_debug_verify_split": "false", 2020-08-07T15:06:32.457 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_dump_file": "", 2020-08-07T15:06:32.458 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2020-08-07T15:06:32.458 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_expected_throughput_ops": "200.000000", 2020-08-07T15:06:32.458 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_fadvise": "true", 2020-08-07T15:06:32.459 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_fail_eio": "true", 2020-08-07T15:06:32.459 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_fd_cache_shards": "16", 2020-08-07T15:06:32.459 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_fd_cache_size": "128", 2020-08-07T15:06:32.459 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_fiemap": "false", 2020-08-07T15:06:32.459 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_fiemap_threshold": "4096", 2020-08-07T15:06:32.459 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_fsync_flushes_journal_data": "false", 2020-08-07T15:06:32.460 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_index_retry_probability": "0.000000", 2020-08-07T15:06:32.460 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_inject_stall": "0", 2020-08-07T15:06:32.460 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_journal_parallel": "false", 2020-08-07T15:06:32.460 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_journal_trailing": "false", 2020-08-07T15:06:32.460 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_journal_writeahead": "false", 2020-08-07T15:06:32.461 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_kill_at": "0", 2020-08-07T15:06:32.461 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_max_alloc_hint_size": "1048576", 2020-08-07T15:06:32.462 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_max_inline_xattr_size": "0", 2020-08-07T15:06:32.462 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2020-08-07T15:06:32.462 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_max_inline_xattr_size_other": "512", 2020-08-07T15:06:32.462 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2020-08-07T15:06:32.462 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_max_inline_xattrs": "0", 2020-08-07T15:06:32.463 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2020-08-07T15:06:32.463 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_max_inline_xattrs_other": "2", 2020-08-07T15:06:32.463 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_max_inline_xattrs_xfs": "10", 2020-08-07T15:06:32.463 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_max_sync_interval": "5.000000", 2020-08-07T15:06:32.463 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_max_xattr_value_size": "0", 2020-08-07T15:06:32.464 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2020-08-07T15:06:32.464 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_max_xattr_value_size_other": "1024", 2020-08-07T15:06:32.464 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2020-08-07T15:06:32.470 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_merge_threshold": "-10", 2020-08-07T15:06:32.471 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_min_sync_interval": "0.010000", 2020-08-07T15:06:32.471 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_odsync_write": "false", 2020-08-07T15:06:32.471 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_omap_backend": "rocksdb", 2020-08-07T15:06:32.471 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_omap_backend_path": "", 2020-08-07T15:06:32.471 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_omap_header_cache_size": "1024", 2020-08-07T15:06:32.472 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_ondisk_finisher_threads": "1", 2020-08-07T15:06:32.472 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_op_thread_suicide_timeout": "180", 2020-08-07T15:06:32.472 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_op_thread_timeout": "60", 2020-08-07T15:06:32.472 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_op_threads": "2", 2020-08-07T15:06:32.472 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_punch_hole": "false", 2020-08-07T15:06:32.473 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2020-08-07T15:06:32.473 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2020-08-07T15:06:32.473 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2020-08-07T15:06:32.473 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_queue_high_threshhold": "0.900000", 2020-08-07T15:06:32.473 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_queue_low_threshhold": "0.300000", 2020-08-07T15:06:32.474 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_queue_max_bytes": "104857600", 2020-08-07T15:06:32.474 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2020-08-07T15:06:32.474 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2020-08-07T15:06:32.474 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2020-08-07T15:06:32.475 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_queue_max_ops": "50", 2020-08-07T15:06:32.475 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2020-08-07T15:06:32.475 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_seek_data_hole": "false", 2020-08-07T15:06:32.475 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_sloppy_crc": "false", 2020-08-07T15:06:32.476 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_sloppy_crc_block_size": "65536", 2020-08-07T15:06:32.476 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_splice": "false", 2020-08-07T15:06:32.476 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_split_multiple": "2", 2020-08-07T15:06:32.476 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_split_rand_factor": "20", 2020-08-07T15:06:32.476 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_update_to": "1000", 2020-08-07T15:06:32.477 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2020-08-07T15:06:32.477 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2020-08-07T15:06:32.477 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2020-08-07T15:06:32.477 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2020-08-07T15:06:32.477 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2020-08-07T15:06:32.478 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2020-08-07T15:06:32.478 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_wbthrottle_enable": "true", 2020-08-07T15:06:32.478 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2020-08-07T15:06:32.478 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2020-08-07T15:06:32.478 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2020-08-07T15:06:32.479 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2020-08-07T15:06:32.479 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2020-08-07T15:06:32.479 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2020-08-07T15:06:32.479 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_xfs_extsize": "false", 2020-08-07T15:06:32.479 INFO:teuthology.orchestra.run.smithi074.stdout: "filestore_zfs_snap": "false", 2020-08-07T15:06:32.479 INFO:teuthology.orchestra.run.smithi074.stdout: "fio_dir": "/tmp/fio", 2020-08-07T15:06:32.480 INFO:teuthology.orchestra.run.smithi074.stdout: "fsid": "a01d3e82-8d24-4288-9e42-576ab3ff4957", 2020-08-07T15:06:32.480 INFO:teuthology.orchestra.run.smithi074.stdout: "fuse_allow_other": "true", 2020-08-07T15:06:32.480 INFO:teuthology.orchestra.run.smithi074.stdout: "fuse_atomic_o_trunc": "true", 2020-08-07T15:06:32.480 INFO:teuthology.orchestra.run.smithi074.stdout: "fuse_big_writes": "true", 2020-08-07T15:06:32.480 INFO:teuthology.orchestra.run.smithi074.stdout: "fuse_debug": "false", 2020-08-07T15:06:32.481 INFO:teuthology.orchestra.run.smithi074.stdout: "fuse_default_permissions": "false", 2020-08-07T15:06:32.481 INFO:teuthology.orchestra.run.smithi074.stdout: "fuse_disable_pagecache": "false", 2020-08-07T15:06:32.481 INFO:teuthology.orchestra.run.smithi074.stdout: "fuse_max_write": "0", 2020-08-07T15:06:32.481 INFO:teuthology.orchestra.run.smithi074.stdout: "fuse_multithreaded": "true", 2020-08-07T15:06:32.481 INFO:teuthology.orchestra.run.smithi074.stdout: "fuse_require_active_mds": "true", 2020-08-07T15:06:32.482 INFO:teuthology.orchestra.run.smithi074.stdout: "fuse_set_user_groups": "true", 2020-08-07T15:06:32.482 INFO:teuthology.orchestra.run.smithi074.stdout: "fuse_splice_move": "true", 2020-08-07T15:06:32.482 INFO:teuthology.orchestra.run.smithi074.stdout: "fuse_splice_read": "true", 2020-08-07T15:06:32.482 INFO:teuthology.orchestra.run.smithi074.stdout: "fuse_splice_write": "true", 2020-08-07T15:06:32.482 INFO:teuthology.orchestra.run.smithi074.stdout: "fuse_syncfs_on_mksnap": "true", 2020-08-07T15:06:32.482 INFO:teuthology.orchestra.run.smithi074.stdout: "fuse_use_invalidate_cb": "true", 2020-08-07T15:06:32.483 INFO:teuthology.orchestra.run.smithi074.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2020-08-07T15:06:32.483 INFO:teuthology.orchestra.run.smithi074.stdout: "gss_target_name": "ceph", 2020-08-07T15:06:32.483 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_file": "", 2020-08-07T15:06:32.483 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_inject_failure": "0", 2020-08-07T15:06:32.483 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_interval": "5", 2020-08-07T15:06:32.484 INFO:teuthology.orchestra.run.smithi074.stdout: "host": "", 2020-08-07T15:06:32.484 INFO:teuthology.orchestra.run.smithi074.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2020-08-07T15:06:32.484 INFO:teuthology.orchestra.run.smithi074.stdout: "immutable_object_cache_max_inflight_ops": "128", 2020-08-07T15:06:32.484 INFO:teuthology.orchestra.run.smithi074.stdout: "immutable_object_cache_max_size": "1073741824", 2020-08-07T15:06:32.484 INFO:teuthology.orchestra.run.smithi074.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2020-08-07T15:06:32.485 INFO:teuthology.orchestra.run.smithi074.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2020-08-07T15:06:32.485 INFO:teuthology.orchestra.run.smithi074.stdout: "immutable_object_cache_watermark": "0.100000", 2020-08-07T15:06:32.485 INFO:teuthology.orchestra.run.smithi074.stdout: "inject_early_sigterm": "false", 2020-08-07T15:06:32.485 INFO:teuthology.orchestra.run.smithi074.stdout: "journal_aio": "true", 2020-08-07T15:06:32.485 INFO:teuthology.orchestra.run.smithi074.stdout: "journal_align_min_size": "65536", 2020-08-07T15:06:32.486 INFO:teuthology.orchestra.run.smithi074.stdout: "journal_block_align": "true", 2020-08-07T15:06:32.486 INFO:teuthology.orchestra.run.smithi074.stdout: "journal_block_size": "4096", 2020-08-07T15:06:32.486 INFO:teuthology.orchestra.run.smithi074.stdout: "journal_dio": "true", 2020-08-07T15:06:32.486 INFO:teuthology.orchestra.run.smithi074.stdout: "journal_discard": "false", 2020-08-07T15:06:32.486 INFO:teuthology.orchestra.run.smithi074.stdout: "journal_force_aio": "false", 2020-08-07T15:06:32.486 INFO:teuthology.orchestra.run.smithi074.stdout: "journal_ignore_corruption": "false", 2020-08-07T15:06:32.487 INFO:teuthology.orchestra.run.smithi074.stdout: "journal_max_write_bytes": "10485760", 2020-08-07T15:06:32.487 INFO:teuthology.orchestra.run.smithi074.stdout: "journal_max_write_entries": "100", 2020-08-07T15:06:32.487 INFO:teuthology.orchestra.run.smithi074.stdout: "journal_replay_from": "0", 2020-08-07T15:06:32.487 INFO:teuthology.orchestra.run.smithi074.stdout: "journal_throttle_high_multiple": "0.000000", 2020-08-07T15:06:32.487 INFO:teuthology.orchestra.run.smithi074.stdout: "journal_throttle_high_threshhold": "0.900000", 2020-08-07T15:06:32.488 INFO:teuthology.orchestra.run.smithi074.stdout: "journal_throttle_low_threshhold": "0.600000", 2020-08-07T15:06:32.488 INFO:teuthology.orchestra.run.smithi074.stdout: "journal_throttle_max_multiple": "0.000000", 2020-08-07T15:06:32.488 INFO:teuthology.orchestra.run.smithi074.stdout: "journal_write_header_frequency": "0", 2020-08-07T15:06:32.488 INFO:teuthology.orchestra.run.smithi074.stdout: "journal_zero_on_create": "false", 2020-08-07T15:06:32.488 INFO:teuthology.orchestra.run.smithi074.stdout: "journaler_prefetch_periods": "10", 2020-08-07T15:06:32.489 INFO:teuthology.orchestra.run.smithi074.stdout: "journaler_prezero_periods": "5", 2020-08-07T15:06:32.489 INFO:teuthology.orchestra.run.smithi074.stdout: "journaler_write_head_interval": "15", 2020-08-07T15:06:32.489 INFO:teuthology.orchestra.run.smithi074.stdout: "key": "", 2020-08-07T15:06:32.489 INFO:teuthology.orchestra.run.smithi074.stdout: "keyfile": "", 2020-08-07T15:06:32.489 INFO:teuthology.orchestra.run.smithi074.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2020-08-07T15:06:32.490 INFO:teuthology.orchestra.run.smithi074.stdout: "kstore_backend": "rocksdb", 2020-08-07T15:06:32.490 INFO:teuthology.orchestra.run.smithi074.stdout: "kstore_default_stripe_size": "65536", 2020-08-07T15:06:32.490 INFO:teuthology.orchestra.run.smithi074.stdout: "kstore_fsck_on_mount": "false", 2020-08-07T15:06:32.490 INFO:teuthology.orchestra.run.smithi074.stdout: "kstore_fsck_on_mount_deep": "true", 2020-08-07T15:06:32.490 INFO:teuthology.orchestra.run.smithi074.stdout: "kstore_max_bytes": "67108864", 2020-08-07T15:06:32.490 INFO:teuthology.orchestra.run.smithi074.stdout: "kstore_max_ops": "512", 2020-08-07T15:06:32.491 INFO:teuthology.orchestra.run.smithi074.stdout: "kstore_nid_prealloc": "1024", 2020-08-07T15:06:32.491 INFO:teuthology.orchestra.run.smithi074.stdout: "kstore_onode_map_size": "1024", 2020-08-07T15:06:32.491 INFO:teuthology.orchestra.run.smithi074.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2020-08-07T15:06:32.491 INFO:teuthology.orchestra.run.smithi074.stdout: "kstore_sync_submit_transaction": "false", 2020-08-07T15:06:32.492 INFO:teuthology.orchestra.run.smithi074.stdout: "kstore_sync_transaction": "false", 2020-08-07T15:06:32.493 INFO:teuthology.orchestra.run.smithi074.stdout: "leveldb_block_size": "65536", 2020-08-07T15:06:32.493 INFO:teuthology.orchestra.run.smithi074.stdout: "leveldb_bloom_size": "0", 2020-08-07T15:06:32.493 INFO:teuthology.orchestra.run.smithi074.stdout: "leveldb_cache_size": "536870912", 2020-08-07T15:06:32.493 INFO:teuthology.orchestra.run.smithi074.stdout: "leveldb_compact_on_mount": "false", 2020-08-07T15:06:32.494 INFO:teuthology.orchestra.run.smithi074.stdout: "leveldb_compression": "false", 2020-08-07T15:06:32.494 INFO:teuthology.orchestra.run.smithi074.stdout: "leveldb_log": "", 2020-08-07T15:06:32.494 INFO:teuthology.orchestra.run.smithi074.stdout: "leveldb_log_to_ceph_log": "true", 2020-08-07T15:06:32.494 INFO:teuthology.orchestra.run.smithi074.stdout: "leveldb_max_open_files": "0", 2020-08-07T15:06:32.494 INFO:teuthology.orchestra.run.smithi074.stdout: "leveldb_paranoid": "false", 2020-08-07T15:06:32.495 INFO:teuthology.orchestra.run.smithi074.stdout: "leveldb_write_buffer_size": "33554432", 2020-08-07T15:06:32.495 INFO:teuthology.orchestra.run.smithi074.stdout: "librados_thread_count": "2", 2020-08-07T15:06:32.495 INFO:teuthology.orchestra.run.smithi074.stdout: "lockdep": "false", 2020-08-07T15:06:32.495 INFO:teuthology.orchestra.run.smithi074.stdout: "lockdep_force_backtrace": "false", 2020-08-07T15:06:32.495 INFO:teuthology.orchestra.run.smithi074.stdout: "log_coarse_timestamps": "true", 2020-08-07T15:06:32.496 INFO:teuthology.orchestra.run.smithi074.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2020-08-07T15:06:32.496 INFO:teuthology.orchestra.run.smithi074.stdout: "log_flush_on_exit": "false", 2020-08-07T15:06:32.496 INFO:teuthology.orchestra.run.smithi074.stdout: "log_graylog_host": "127.0.0.1", 2020-08-07T15:06:32.496 INFO:teuthology.orchestra.run.smithi074.stdout: "log_graylog_port": "12201", 2020-08-07T15:06:32.496 INFO:teuthology.orchestra.run.smithi074.stdout: "log_max_new": "1000", 2020-08-07T15:06:32.496 INFO:teuthology.orchestra.run.smithi074.stdout: "log_max_recent": "10000", 2020-08-07T15:06:32.497 INFO:teuthology.orchestra.run.smithi074.stdout: "log_stderr_prefix": "", 2020-08-07T15:06:32.497 INFO:teuthology.orchestra.run.smithi074.stdout: "log_stop_at_utilization": "0.970000", 2020-08-07T15:06:32.497 INFO:teuthology.orchestra.run.smithi074.stdout: "log_to_file": "true", 2020-08-07T15:06:32.497 INFO:teuthology.orchestra.run.smithi074.stdout: "log_to_graylog": "false", 2020-08-07T15:06:32.497 INFO:teuthology.orchestra.run.smithi074.stdout: "log_to_stderr": "false", 2020-08-07T15:06:32.498 INFO:teuthology.orchestra.run.smithi074.stdout: "log_to_syslog": "false", 2020-08-07T15:06:32.498 INFO:teuthology.orchestra.run.smithi074.stdout: "max_rotating_auth_attempts": "10", 2020-08-07T15:06:32.498 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_action_on_write_error": "1", 2020-08-07T15:06:32.499 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_asio_thread_count": "2", 2020-08-07T15:06:32.499 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_bal_export_pin": "true", 2020-08-07T15:06:32.499 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_bal_fragment_dirs": "true", 2020-08-07T15:06:32.499 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2020-08-07T15:06:32.499 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_bal_fragment_interval": "5", 2020-08-07T15:06:32.500 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_bal_fragment_size_max": "100000", 2020-08-07T15:06:32.500 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_bal_idle_threshold": "0.000000", 2020-08-07T15:06:32.500 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_bal_interval": "10", 2020-08-07T15:06:32.500 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_bal_max": "-1", 2020-08-07T15:06:32.500 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_bal_max_until": "-1", 2020-08-07T15:06:32.500 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_bal_merge_size": "50", 2020-08-07T15:06:32.501 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_bal_midchunk": "0.300000", 2020-08-07T15:06:32.501 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_bal_min_rebalance": "0.100000", 2020-08-07T15:06:32.501 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_bal_min_start": "0.200000", 2020-08-07T15:06:32.501 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_bal_minchunk": "0.001000", 2020-08-07T15:06:32.501 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_bal_mode": "0", 2020-08-07T15:06:32.502 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_bal_need_max": "1.200000", 2020-08-07T15:06:32.502 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_bal_need_min": "0.800000", 2020-08-07T15:06:32.502 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_bal_replicate_threshold": "8000.000000", 2020-08-07T15:06:32.503 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_bal_sample_interval": "3.000000", 2020-08-07T15:06:32.503 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_bal_split_bits": "3", 2020-08-07T15:06:32.503 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_bal_split_rd": "25000.000000", 2020-08-07T15:06:32.503 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_bal_split_size": "10000", 2020-08-07T15:06:32.503 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_bal_split_wr": "10000.000000", 2020-08-07T15:06:32.504 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_bal_target_decay": "10.000000", 2020-08-07T15:06:32.504 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2020-08-07T15:06:32.504 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_beacon_grace": "15.000000", 2020-08-07T15:06:32.504 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_beacon_interval": "4.000000", 2020-08-07T15:06:32.504 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_cache_memory_limit": "4294967296", 2020-08-07T15:06:32.505 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_cache_mid": "0.700000", 2020-08-07T15:06:32.505 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_cache_release_free_interval": "10", 2020-08-07T15:06:32.505 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_cache_reservation": "0.050000", 2020-08-07T15:06:32.506 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_cache_trim_decay_rate": "1.000000", 2020-08-07T15:06:32.506 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_cache_trim_interval": "1", 2020-08-07T15:06:32.506 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_cache_trim_threshold": "65536", 2020-08-07T15:06:32.506 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2020-08-07T15:06:32.506 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_client_delegate_inos_pct": "50", 2020-08-07T15:06:32.507 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_client_prealloc_inos": "1000", 2020-08-07T15:06:32.507 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2020-08-07T15:06:32.507 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_damage_table_max_entries": "10000", 2020-08-07T15:06:32.507 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2020-08-07T15:06:32.509 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_debug_auth_pins": "false", 2020-08-07T15:06:32.509 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_debug_frag": "false", 2020-08-07T15:06:32.509 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_debug_scatterstat": "false", 2020-08-07T15:06:32.509 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_debug_subtrees": "false", 2020-08-07T15:06:32.509 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_decay_halflife": "5.000000", 2020-08-07T15:06:32.510 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_default_dir_hash": "2", 2020-08-07T15:06:32.510 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_defer_session_stale": "true", 2020-08-07T15:06:32.510 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_deny_all_reconnect": "false", 2020-08-07T15:06:32.510 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_dir_keys_per_op": "16384", 2020-08-07T15:06:32.510 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_dir_max_commit_size": "10", 2020-08-07T15:06:32.510 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_dirstat_min_interval": "1.000000", 2020-08-07T15:06:32.511 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_dump_cache_after_rejoin": "false", 2020-08-07T15:06:32.511 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_dump_cache_on_map": "false", 2020-08-07T15:06:32.511 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_dump_cache_threshold_file": "0", 2020-08-07T15:06:32.512 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2020-08-07T15:06:32.512 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_early_reply": "true", 2020-08-07T15:06:32.512 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_enable_op_tracker": "true", 2020-08-07T15:06:32.512 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_enforce_unique_name": "true", 2020-08-07T15:06:32.512 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_export_ephemeral_distributed": "true", 2020-08-07T15:06:32.513 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_export_ephemeral_random": "true", 2020-08-07T15:06:32.513 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_export_ephemeral_random_max": "0.010000", 2020-08-07T15:06:32.513 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_forward_all_requests_to_auth": "false", 2020-08-07T15:06:32.513 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_freeze_tree_timeout": "30.000000", 2020-08-07T15:06:32.513 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2020-08-07T15:06:32.513 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_health_cache_threshold": "1.500000", 2020-08-07T15:06:32.514 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_health_summarize_threshold": "10", 2020-08-07T15:06:32.514 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_heartbeat_grace": "15.000000", 2020-08-07T15:06:32.514 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_inject_migrator_session_race": "false", 2020-08-07T15:06:32.514 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2020-08-07T15:06:32.514 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_join_fs": "", 2020-08-07T15:06:32.515 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_journal_format": "1", 2020-08-07T15:06:32.515 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_kill_create_at": "0", 2020-08-07T15:06:32.515 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_kill_export_at": "0", 2020-08-07T15:06:32.515 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_kill_import_at": "0", 2020-08-07T15:06:32.515 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_kill_journal_at": "0", 2020-08-07T15:06:32.516 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_kill_journal_expire_at": "0", 2020-08-07T15:06:32.516 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_kill_journal_replay_at": "0", 2020-08-07T15:06:32.517 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_kill_link_at": "0", 2020-08-07T15:06:32.517 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_kill_mdstable_at": "0", 2020-08-07T15:06:32.518 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_kill_openc_at": "0", 2020-08-07T15:06:32.518 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_kill_rename_at": "0", 2020-08-07T15:06:32.518 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_log_events_per_segment": "1024", 2020-08-07T15:06:32.518 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_log_max_events": "-1", 2020-08-07T15:06:32.518 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_log_max_segments": "128", 2020-08-07T15:06:32.519 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_log_pause": "false", 2020-08-07T15:06:32.519 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_log_segment_size": "0", 2020-08-07T15:06:32.519 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_log_skip_corrupt_events": "false", 2020-08-07T15:06:32.519 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_log_warn_factor": "2.000000", 2020-08-07T15:06:32.519 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_max_caps_per_client": "1048576", 2020-08-07T15:06:32.520 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_max_completed_flushes": "100000", 2020-08-07T15:06:32.520 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_max_completed_requests": "100000", 2020-08-07T15:06:32.520 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_max_export_size": "20971520", 2020-08-07T15:06:32.520 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_max_file_recover": "32", 2020-08-07T15:06:32.520 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_max_purge_files": "64", 2020-08-07T15:06:32.521 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_max_purge_ops": "8192", 2020-08-07T15:06:32.521 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2020-08-07T15:06:32.521 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_max_retries_on_remount_failure": "5", 2020-08-07T15:06:32.522 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_max_scrub_ops_in_progress": "5", 2020-08-07T15:06:32.522 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_max_snaps_per_dir": "100", 2020-08-07T15:06:32.523 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_max_xattr_pairs_size": "65536", 2020-08-07T15:06:32.523 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_metrics_update_interval": "2", 2020-08-07T15:06:32.523 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_min_caps_per_client": "100", 2020-08-07T15:06:32.523 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_mon_shutdown_timeout": "5.000000", 2020-08-07T15:06:32.523 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_numa_node": "-1", 2020-08-07T15:06:32.524 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_op_complaint_time": "30.000000", 2020-08-07T15:06:32.524 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_op_history_duration": "600", 2020-08-07T15:06:32.524 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_op_history_size": "20", 2020-08-07T15:06:32.524 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_op_log_threshold": "5", 2020-08-07T15:06:32.524 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_ping_grace": "15", 2020-08-07T15:06:32.524 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_ping_interval": "5", 2020-08-07T15:06:32.525 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2020-08-07T15:06:32.525 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_recall_global_max_decay_threshold": "65536", 2020-08-07T15:06:32.525 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_recall_max_caps": "5000", 2020-08-07T15:06:32.525 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_recall_max_decay_rate": "2.500000", 2020-08-07T15:06:32.525 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_recall_max_decay_threshold": "16384", 2020-08-07T15:06:32.526 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_recall_warning_decay_rate": "60.000000", 2020-08-07T15:06:32.526 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_recall_warning_threshold": "32768", 2020-08-07T15:06:32.526 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_reconnect_timeout": "45.000000", 2020-08-07T15:06:32.526 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_replay_interval": "1.000000", 2020-08-07T15:06:32.526 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_replay_unsafe_with_closed_session": "false", 2020-08-07T15:06:32.527 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_request_load_average_decay_rate": "60.000000", 2020-08-07T15:06:32.527 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_root_ino_gid": "0", 2020-08-07T15:06:32.527 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_root_ino_uid": "0", 2020-08-07T15:06:32.527 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_scatter_nudge_interval": "5.000000", 2020-08-07T15:06:32.527 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_session_blacklist_on_evict": "true", 2020-08-07T15:06:32.528 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_session_blacklist_on_timeout": "true", 2020-08-07T15:06:32.528 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2020-08-07T15:06:32.528 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_session_cache_liveness_magnitude": "10", 2020-08-07T15:06:32.528 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_sessionmap_keys_per_op": "1024", 2020-08-07T15:06:32.528 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_shutdown_check": "0", 2020-08-07T15:06:32.529 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_skip_ino": "0", 2020-08-07T15:06:32.529 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_snap_max_uid": "4294967294", 2020-08-07T15:06:32.529 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_snap_min_uid": "0", 2020-08-07T15:06:32.529 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_snap_rstat": "false", 2020-08-07T15:06:32.529 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_task_status_update_interval": "2.000000", 2020-08-07T15:06:32.529 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_thrash_exports": "0", 2020-08-07T15:06:32.530 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_thrash_fragments": "0", 2020-08-07T15:06:32.530 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_tick_interval": "5.000000", 2020-08-07T15:06:32.530 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_verify_backtrace": "1", 2020-08-07T15:06:32.530 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_verify_scatter": "false", 2020-08-07T15:06:32.530 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_wipe_ino_prealloc": "false", 2020-08-07T15:06:32.531 INFO:teuthology.orchestra.run.smithi074.stdout: "mds_wipe_sessions": "false", 2020-08-07T15:06:32.531 INFO:teuthology.orchestra.run.smithi074.stdout: "mempool_debug": "false", 2020-08-07T15:06:32.531 INFO:teuthology.orchestra.run.smithi074.stdout: "memstore_debug_omit_block_device_write": "false", 2020-08-07T15:06:32.531 INFO:teuthology.orchestra.run.smithi074.stdout: "memstore_device_bytes": "1073741824", 2020-08-07T15:06:32.531 INFO:teuthology.orchestra.run.smithi074.stdout: "memstore_page_set": "false", 2020-08-07T15:06:32.532 INFO:teuthology.orchestra.run.smithi074.stdout: "memstore_page_size": "65536", 2020-08-07T15:06:32.532 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_client_bytes": "134217728", 2020-08-07T15:06:32.532 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_client_messages": "512", 2020-08-07T15:06:32.532 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2020-08-07T15:06:32.532 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_connect_retry_interval": "1.000000", 2020-08-07T15:06:32.532 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2020-08-07T15:06:32.533 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2020-08-07T15:06:32.533 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_disabled_modules": "", 2020-08-07T15:06:32.533 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_initial_modules": "restful iostat", 2020-08-07T15:06:32.533 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_mds_bytes": "134217728", 2020-08-07T15:06:32.533 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_mds_messages": "128", 2020-08-07T15:06:32.534 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2020-08-07T15:06:32.534 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_mon_bytes": "134217728", 2020-08-07T15:06:32.534 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_mon_messages": "128", 2020-08-07T15:06:32.534 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_osd_bytes": "536870912", 2020-08-07T15:06:32.534 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_osd_messages": "8192", 2020-08-07T15:06:32.535 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_service_beacon_grace": "60.000000", 2020-08-07T15:06:32.535 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_stats_period": "5", 2020-08-07T15:06:32.535 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_stats_threshold": "5", 2020-08-07T15:06:32.535 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_tick_period": "2", 2020-08-07T15:06:32.535 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_accept_timeout_factor": "2.000000", 2020-08-07T15:06:32.536 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_allow_pool_delete": "true", 2020-08-07T15:06:32.536 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_allow_pool_size_one": "true", 2020-08-07T15:06:32.536 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2020-08-07T15:06:32.536 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2020-08-07T15:06:32.536 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_client_bytes": "104857600", 2020-08-07T15:06:32.536 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_client_directed_command_retry": "5", 2020-08-07T15:06:32.537 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_client_hunt_interval": "3.000000", 2020-08-07T15:06:32.537 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2020-08-07T15:06:32.537 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2020-08-07T15:06:32.537 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2020-08-07T15:06:32.537 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_client_hunt_parallel": "3", 2020-08-07T15:06:32.538 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_client_log_interval": "1.000000", 2020-08-07T15:06:32.538 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_client_max_log_entries_per_message": "1000", 2020-08-07T15:06:32.538 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_client_ping_interval": "10.000000", 2020-08-07T15:06:32.538 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_client_ping_timeout": "30.000000", 2020-08-07T15:06:32.538 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_clock_drift_allowed": "1.000000", 2020-08-07T15:06:32.539 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2020-08-07T15:06:32.539 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2020-08-07T15:06:32.539 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_cluster_log_file_level": "debug", 2020-08-07T15:06:32.539 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_cluster_log_to_file": "true", 2020-08-07T15:06:32.539 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_cluster_log_to_graylog": "false", 2020-08-07T15:06:32.539 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2020-08-07T15:06:32.540 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_cluster_log_to_graylog_port": "12201", 2020-08-07T15:06:32.540 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_cluster_log_to_stderr": "false", 2020-08-07T15:06:32.540 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_cluster_log_to_syslog": "default=false", 2020-08-07T15:06:32.540 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2020-08-07T15:06:32.540 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_cluster_log_to_syslog_level": "info", 2020-08-07T15:06:32.541 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_compact_on_bootstrap": "false", 2020-08-07T15:06:32.541 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_compact_on_start": "false", 2020-08-07T15:06:32.541 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_compact_on_trim": "true", 2020-08-07T15:06:32.541 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_config_key_max_entry_size": "65536", 2020-08-07T15:06:32.541 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_cpu_threads": "4", 2020-08-07T15:06:32.542 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_crush_min_required_version": "hammer", 2020-08-07T15:06:32.542 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_daemon_bytes": "419430400", 2020-08-07T15:06:32.542 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2020-08-07T15:06:32.542 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_data_avail_crit": "5", 2020-08-07T15:06:32.542 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_data_avail_warn": "5", 2020-08-07T15:06:32.542 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_data_size_warn": "16106127360", 2020-08-07T15:06:32.543 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_debug_block_osdmap_trim": "false", 2020-08-07T15:06:32.543 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_debug_deprecated_as_obsolete": "false", 2020-08-07T15:06:32.543 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_debug_dump_json": "false", 2020-08-07T15:06:32.543 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2020-08-07T15:06:32.543 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_debug_dump_transactions": "false", 2020-08-07T15:06:32.544 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_debug_extra_checks": "false", 2020-08-07T15:06:32.544 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_debug_no_initial_persistent_features": "false", 2020-08-07T15:06:32.544 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2020-08-07T15:06:32.544 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_debug_no_require_octopus": "false", 2020-08-07T15:06:32.544 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_debug_no_require_pacific": "false", 2020-08-07T15:06:32.545 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2020-08-07T15:06:32.545 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_delta_reset_interval": "10.000000", 2020-08-07T15:06:32.545 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_dns_srv_name": "ceph-mon", 2020-08-07T15:06:32.545 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_election_timeout": "5.000000", 2020-08-07T15:06:32.545 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_enable_op_tracker": "true", 2020-08-07T15:06:32.546 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_fake_pool_delete": "false", 2020-08-07T15:06:32.546 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_force_quorum_join": "false", 2020-08-07T15:06:32.546 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_globalid_prealloc": "10000", 2020-08-07T15:06:32.546 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_health_log_update_period": "5", 2020-08-07T15:06:32.547 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_health_max_detail": "50", 2020-08-07T15:06:32.547 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_health_to_clog": "true", 2020-08-07T15:06:32.547 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_health_to_clog_interval": "3600", 2020-08-07T15:06:32.547 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2020-08-07T15:06:32.547 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_host": "172.21.15.74,[v2:172.21.15.74:3301,v1:172.21.15.74:6790],172.21.15.96", 2020-08-07T15:06:32.547 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_initial_members": "", 2020-08-07T15:06:32.548 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2020-08-07T15:06:32.548 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2020-08-07T15:06:32.548 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_inject_transaction_delay_max": "10.000000", 2020-08-07T15:06:32.548 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2020-08-07T15:06:32.548 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_keyvaluedb": "rocksdb", 2020-08-07T15:06:32.549 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_lease": "5.000000", 2020-08-07T15:06:32.549 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2020-08-07T15:06:32.549 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_lease_renew_interval_factor": "0.600000", 2020-08-07T15:06:32.549 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_log_max_summary": "50", 2020-08-07T15:06:32.549 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_max_log_entries_per_event": "4096", 2020-08-07T15:06:32.550 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_max_log_epochs": "500", 2020-08-07T15:06:32.550 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_max_mdsmap_epochs": "500", 2020-08-07T15:06:32.550 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_max_mgrmap_epochs": "500", 2020-08-07T15:06:32.550 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_max_osd": "10000", 2020-08-07T15:06:32.550 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_max_pg_per_osd": "10000", 2020-08-07T15:06:32.551 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_max_pool_pg_num": "65536", 2020-08-07T15:06:32.551 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_max_snap_prune_per_epoch": "100", 2020-08-07T15:06:32.551 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_mds_blacklist_interval": "86400.000000", 2020-08-07T15:06:32.551 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_mds_force_trim_to": "0", 2020-08-07T15:06:32.551 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_mds_skip_sanity": "false", 2020-08-07T15:06:32.551 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_memory_autotune": "true", 2020-08-07T15:06:32.552 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_memory_target": "2147483648", 2020-08-07T15:06:32.552 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_mgr_beacon_grace": "30", 2020-08-07T15:06:32.552 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_mgr_blacklist_interval": "86400.000000", 2020-08-07T15:06:32.552 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_mgr_digest_period": "5", 2020-08-07T15:06:32.552 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_mgr_inactive_grace": "60", 2020-08-07T15:06:32.553 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_mgr_mkfs_grace": "240", 2020-08-07T15:06:32.553 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2020-08-07T15:06:32.553 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_min_osdmap_epochs": "50", 2020-08-07T15:06:32.553 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_op_complaint_time": "30", 2020-08-07T15:06:32.553 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_op_history_duration": "600", 2020-08-07T15:06:32.554 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_op_history_size": "20", 2020-08-07T15:06:32.554 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_op_history_slow_op_size": "20", 2020-08-07T15:06:32.554 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_op_history_slow_op_threshold": "10", 2020-08-07T15:06:32.554 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_op_log_threshold": "5", 2020-08-07T15:06:32.554 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_adjust_down_out_interval": "true", 2020-08-07T15:06:32.554 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2020-08-07T15:06:32.555 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2020-08-07T15:06:32.555 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_auto_mark_in": "false", 2020-08-07T15:06:32.555 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_auto_mark_new_in": "true", 2020-08-07T15:06:32.555 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2020-08-07T15:06:32.555 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_blacklist_default_expire": "3600.000000", 2020-08-07T15:06:32.556 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_cache_size": "500", 2020-08-07T15:06:32.556 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_cache_size_min": "134217728", 2020-08-07T15:06:32.556 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_crush_smoke_test": "true", 2020-08-07T15:06:32.556 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_destroyed_out_interval": "600", 2020-08-07T15:06:32.556 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_down_out_interval": "600", 2020-08-07T15:06:32.557 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_down_out_subtree_limit": "rack", 2020-08-07T15:06:32.557 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2020-08-07T15:06:32.557 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_force_trim_to": "0", 2020-08-07T15:06:32.557 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_full_ratio": "0.950000", 2020-08-07T15:06:32.557 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2020-08-07T15:06:32.557 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_laggy_halflife": "3600", 2020-08-07T15:06:32.558 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_laggy_max_interval": "300", 2020-08-07T15:06:32.558 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_laggy_weight": "0.300000", 2020-08-07T15:06:32.558 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2020-08-07T15:06:32.559 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_max_creating_pgs": "1024", 2020-08-07T15:06:32.559 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_max_initial_pgs": "1024", 2020-08-07T15:06:32.560 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_min_down_reporters": "2", 2020-08-07T15:06:32.560 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_min_in_ratio": "0.750000", 2020-08-07T15:06:32.560 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_min_up_ratio": "0.300000", 2020-08-07T15:06:32.560 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_nearfull_ratio": "0.850000", 2020-08-07T15:06:32.560 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_prime_pg_temp": "true", 2020-08-07T15:06:32.561 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2020-08-07T15:06:32.561 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2020-08-07T15:06:32.561 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_report_timeout": "900", 2020-08-07T15:06:32.561 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_reporter_subtree_level": "osd", 2020-08-07T15:06:32.561 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2020-08-07T15:06:32.562 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_warn_num_repaired": "10", 2020-08-07T15:06:32.562 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osd_warn_op_age": "32.000000", 2020-08-07T15:06:32.562 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osdmap_full_prune_enabled": "true", 2020-08-07T15:06:32.562 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osdmap_full_prune_interval": "2", 2020-08-07T15:06:32.562 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osdmap_full_prune_min": "15", 2020-08-07T15:06:32.562 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_osdmap_full_prune_txsize": "2", 2020-08-07T15:06:32.563 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2020-08-07T15:06:32.563 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_pg_stuck_threshold": "60", 2020-08-07T15:06:32.563 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2020-08-07T15:06:32.563 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_pg_warn_min_objects": "10000", 2020-08-07T15:06:32.563 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_pg_warn_min_per_osd": "0", 2020-08-07T15:06:32.564 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_pg_warn_min_pool_objects": "1000", 2020-08-07T15:06:32.564 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_pool_quota_crit_threshold": "0", 2020-08-07T15:06:32.564 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_pool_quota_warn_threshold": "0", 2020-08-07T15:06:32.564 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_probe_timeout": "2.000000", 2020-08-07T15:06:32.564 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_reweight_max_change": "0.050000", 2020-08-07T15:06:32.565 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_reweight_max_osds": "4", 2020-08-07T15:06:32.565 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_reweight_min_bytes_per_osd": "10", 2020-08-07T15:06:32.565 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_reweight_min_pgs_per_osd": "4", 2020-08-07T15:06:32.565 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2020-08-07T15:06:32.565 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2020-08-07T15:06:32.565 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2020-08-07T15:06:32.566 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_scrub_interval": "86400", 2020-08-07T15:06:32.566 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_scrub_max_keys": "100", 2020-08-07T15:06:32.566 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_scrub_timeout": "300", 2020-08-07T15:06:32.566 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_session_timeout": "300", 2020-08-07T15:06:32.566 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_smart_report_timeout": "5", 2020-08-07T15:06:32.567 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_stat_smooth_intervals": "6", 2020-08-07T15:06:32.567 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_subscribe_interval": "86400.000000", 2020-08-07T15:06:32.567 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_sync_debug": "false", 2020-08-07T15:06:32.567 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_sync_max_payload_keys": "2000", 2020-08-07T15:06:32.567 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_sync_max_payload_size": "1048576", 2020-08-07T15:06:32.568 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_sync_provider_kill_at": "0", 2020-08-07T15:06:32.569 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_sync_requester_kill_at": "0", 2020-08-07T15:06:32.569 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_sync_timeout": "60.000000", 2020-08-07T15:06:32.569 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_target_pg_per_osd": "100", 2020-08-07T15:06:32.569 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_tick_interval": "5", 2020-08-07T15:06:32.570 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_timecheck_interval": "300.000000", 2020-08-07T15:06:32.570 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_timecheck_skew_interval": "30.000000", 2020-08-07T15:06:32.570 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2020-08-07T15:06:32.570 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2020-08-07T15:06:32.571 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2020-08-07T15:06:32.571 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_warn_on_misplaced": "false", 2020-08-07T15:06:32.571 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2020-08-07T15:06:32.571 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2020-08-07T15:06:32.571 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_warn_on_pool_no_app": "true", 2020-08-07T15:06:32.572 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_warn_on_pool_no_redundancy": "false", 2020-08-07T15:06:32.572 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2020-08-07T15:06:32.572 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2020-08-07T15:06:32.572 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2020-08-07T15:06:32.572 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_warn_on_too_few_osds": "false", 2020-08-07T15:06:32.573 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2020-08-07T15:06:32.573 INFO:teuthology.orchestra.run.smithi074.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2020-08-07T15:06:32.573 INFO:teuthology.orchestra.run.smithi074.stdout: "monmap": "", 2020-08-07T15:06:32.573 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_async_max_op_threads": "5", 2020-08-07T15:06:32.573 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_async_op_threads": "3", 2020-08-07T15:06:32.574 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_async_rdma_buffer_size": "131072", 2020-08-07T15:06:32.574 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_async_rdma_cm": "false", 2020-08-07T15:06:32.574 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_async_rdma_device_name": "", 2020-08-07T15:06:32.574 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_async_rdma_dscp": "96", 2020-08-07T15:06:32.574 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_async_rdma_enable_hugepage": "false", 2020-08-07T15:06:32.575 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_async_rdma_gid_idx": "0", 2020-08-07T15:06:32.575 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_async_rdma_local_gid": "", 2020-08-07T15:06:32.575 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_async_rdma_polling_us": "1000", 2020-08-07T15:06:32.575 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_async_rdma_port_num": "1", 2020-08-07T15:06:32.575 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_async_rdma_receive_buffers": "32768", 2020-08-07T15:06:32.576 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_async_rdma_receive_queue_len": "4096", 2020-08-07T15:06:32.576 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_async_rdma_roce_ver": "1", 2020-08-07T15:06:32.576 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_async_rdma_send_buffers": "1024", 2020-08-07T15:06:32.576 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_async_rdma_sl": "3", 2020-08-07T15:06:32.577 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_async_rdma_support_srq": "true", 2020-08-07T15:06:32.577 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_async_rdma_type": "ib", 2020-08-07T15:06:32.577 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_bind_before_connect": "false", 2020-08-07T15:06:32.577 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_bind_ipv4": "true", 2020-08-07T15:06:32.577 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_bind_ipv6": "false", 2020-08-07T15:06:32.578 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_bind_msgr1": "false", 2020-08-07T15:06:32.578 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_bind_msgr2": "true", 2020-08-07T15:06:32.578 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_bind_port_max": "7300", 2020-08-07T15:06:32.578 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_bind_port_min": "6800", 2020-08-07T15:06:32.578 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_bind_prefer_ipv4": "false", 2020-08-07T15:06:32.579 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_bind_retry_count": "3", 2020-08-07T15:06:32.579 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_bind_retry_delay": "5", 2020-08-07T15:06:32.579 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_blackhole_client": "false", 2020-08-07T15:06:32.579 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_blackhole_mds": "false", 2020-08-07T15:06:32.579 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_blackhole_mgr": "false", 2020-08-07T15:06:32.580 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_blackhole_mon": "false", 2020-08-07T15:06:32.580 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_blackhole_osd": "false", 2020-08-07T15:06:32.580 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_client_mode": "crc secure", 2020-08-07T15:06:32.580 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_cluster_mode": "crc secure", 2020-08-07T15:06:32.580 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_cluster_type": "", 2020-08-07T15:06:32.580 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_connection_idle_timeout": "900", 2020-08-07T15:06:32.581 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_connection_ready_timeout": "10", 2020-08-07T15:06:32.581 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_crc_data": "true", 2020-08-07T15:06:32.581 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_crc_header": "true", 2020-08-07T15:06:32.581 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_die_on_bad_msg": "false", 2020-08-07T15:06:32.581 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_die_on_bug": "true", 2020-08-07T15:06:32.582 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_die_on_old_message": "true", 2020-08-07T15:06:32.582 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_die_on_skipped_message": "false", 2020-08-07T15:06:32.582 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_die_on_unhandled_msg": "false", 2020-08-07T15:06:32.582 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_dispatch_throttle_bytes": "104857600", 2020-08-07T15:06:32.582 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_dpdk_coremask": "0xF", 2020-08-07T15:06:32.582 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_dpdk_debug_allow_loopback": "false", 2020-08-07T15:06:32.583 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2020-08-07T15:06:32.583 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_dpdk_host_ipv4_addr": "", 2020-08-07T15:06:32.583 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_dpdk_hugepages": "", 2020-08-07T15:06:32.583 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_dpdk_hw_flow_control": "true", 2020-08-07T15:06:32.583 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2020-08-07T15:06:32.584 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_dpdk_lro": "true", 2020-08-07T15:06:32.584 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_dpdk_memory_channel": "4", 2020-08-07T15:06:32.584 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2020-08-07T15:06:32.584 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_dpdk_pmd": "", 2020-08-07T15:06:32.584 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_dpdk_port_id": "0", 2020-08-07T15:06:32.585 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2020-08-07T15:06:32.585 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_dump_corrupt_message_level": "1", 2020-08-07T15:06:32.585 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_dump_on_send": "false", 2020-08-07T15:06:32.585 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_initial_backoff": "0.200000", 2020-08-07T15:06:32.585 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_inject_delay_max": "1.000000", 2020-08-07T15:06:32.586 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_inject_delay_msg_type": "", 2020-08-07T15:06:32.586 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_inject_delay_probability": "0.005000", 2020-08-07T15:06:32.586 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_inject_delay_type": "osd", 2020-08-07T15:06:32.586 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_inject_internal_delays": "0.002000", 2020-08-07T15:06:32.586 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_inject_socket_failures": "2500", 2020-08-07T15:06:32.586 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_learn_addr_from_peer": "true", 2020-08-07T15:06:32.587 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_max_accept_failures": "4", 2020-08-07T15:06:32.587 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_max_backoff": "15.000000", 2020-08-07T15:06:32.587 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_mon_client_mode": "secure crc", 2020-08-07T15:06:32.587 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_mon_cluster_mode": "secure crc", 2020-08-07T15:06:32.587 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_mon_service_mode": "secure crc", 2020-08-07T15:06:32.588 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2020-08-07T15:06:32.588 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_pq_min_cost": "65536", 2020-08-07T15:06:32.588 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_public_type": "", 2020-08-07T15:06:32.588 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_service_mode": "crc secure", 2020-08-07T15:06:32.588 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_tcp_listen_backlog": "512", 2020-08-07T15:06:32.588 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_tcp_nodelay": "true", 2020-08-07T15:06:32.589 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_tcp_prefetch_max_size": "4096", 2020-08-07T15:06:32.589 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_tcp_rcvbuf": "0", 2020-08-07T15:06:32.589 INFO:teuthology.orchestra.run.smithi074.stdout: "ms_type": "async", 2020-08-07T15:06:32.589 INFO:teuthology.orchestra.run.smithi074.stdout: "no_config_file": "false", 2020-08-07T15:06:32.589 INFO:teuthology.orchestra.run.smithi074.stdout: "objecter_completion_locks_per_session": "32", 2020-08-07T15:06:32.590 INFO:teuthology.orchestra.run.smithi074.stdout: "objecter_debug_inject_relock_delay": "false", 2020-08-07T15:06:32.590 INFO:teuthology.orchestra.run.smithi074.stdout: "objecter_inflight_op_bytes": "104857600", 2020-08-07T15:06:32.590 INFO:teuthology.orchestra.run.smithi074.stdout: "objecter_inflight_ops": "1024", 2020-08-07T15:06:32.590 INFO:teuthology.orchestra.run.smithi074.stdout: "objecter_inject_no_watch_ping": "false", 2020-08-07T15:06:32.590 INFO:teuthology.orchestra.run.smithi074.stdout: "objecter_retry_writes_after_first_reply": "false", 2020-08-07T15:06:32.591 INFO:teuthology.orchestra.run.smithi074.stdout: "objecter_tick_interval": "5.000000", 2020-08-07T15:06:32.591 INFO:teuthology.orchestra.run.smithi074.stdout: "objecter_timeout": "10.000000", 2020-08-07T15:06:32.591 INFO:teuthology.orchestra.run.smithi074.stdout: "objectstore_blackhole": "false", 2020-08-07T15:06:32.591 INFO:teuthology.orchestra.run.smithi074.stdout: "openssl_engine_opts": "", 2020-08-07T15:06:32.591 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_agent_delay_time": "5.000000", 2020-08-07T15:06:32.591 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_agent_hist_halflife": "1000", 2020-08-07T15:06:32.592 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_agent_max_low_ops": "2", 2020-08-07T15:06:32.592 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_agent_max_ops": "4", 2020-08-07T15:06:32.592 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_agent_min_evict_effort": "0.100000", 2020-08-07T15:06:32.592 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_agent_quantize_effort": "0.100000", 2020-08-07T15:06:32.592 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_agent_slop": "0.020000", 2020-08-07T15:06:32.593 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_allow_recovery_below_min_size": "true", 2020-08-07T15:06:32.593 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_asio_thread_count": "2", 2020-08-07T15:06:32.593 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_async_recovery_min_cost": "1", 2020-08-07T15:06:32.593 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_auto_mark_unfound_lost": "false", 2020-08-07T15:06:32.593 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_backfill_retry_interval": "30.000000", 2020-08-07T15:06:32.593 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_backfill_scan_max": "512", 2020-08-07T15:06:32.594 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_backfill_scan_min": "64", 2020-08-07T15:06:32.594 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_backoff_on_degraded": "false", 2020-08-07T15:06:32.594 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_backoff_on_peering": "false", 2020-08-07T15:06:32.594 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_backoff_on_unfound": "true", 2020-08-07T15:06:32.594 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_beacon_report_interval": "300", 2020-08-07T15:06:32.595 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_bench_duration": "30", 2020-08-07T15:06:32.595 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_bench_large_size_max_throughput": "104857600", 2020-08-07T15:06:32.595 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_bench_max_block_size": "67108864", 2020-08-07T15:06:32.595 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_bench_small_size_max_iops": "100", 2020-08-07T15:06:32.595 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_blkin_trace_all": "false", 2020-08-07T15:06:32.596 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2020-08-07T15:06:32.596 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2020-08-07T15:06:32.596 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_check_for_log_corruption": "false", 2020-08-07T15:06:32.596 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_check_max_object_name_len_on_startup": "true", 2020-08-07T15:06:32.596 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue", 2020-08-07T15:06:32.597 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2020-08-07T15:06:32.597 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue", 2020-08-07T15:06:32.597 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_class_update_on_start": "true", 2020-08-07T15:06:32.597 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_client_message_cap": "0", 2020-08-07T15:06:32.597 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_client_message_size_cap": "524288000", 2020-08-07T15:06:32.598 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_client_op_priority": "63", 2020-08-07T15:06:32.598 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_client_watch_timeout": "30", 2020-08-07T15:06:32.598 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_command_max_records": "256", 2020-08-07T15:06:32.598 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_command_thread_suicide_timeout": "900", 2020-08-07T15:06:32.599 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_command_thread_timeout": "600", 2020-08-07T15:06:32.599 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_compact_on_start": "false", 2020-08-07T15:06:32.599 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_copyfrom_max_chunk": "8388608", 2020-08-07T15:06:32.600 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_crush_chooseleaf_type": "0", 2020-08-07T15:06:32.600 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_crush_initial_weight": "-1.000000", 2020-08-07T15:06:32.600 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_crush_update_on_start": "true", 2020-08-07T15:06:32.600 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_crush_update_weight_set": "true", 2020-08-07T15:06:32.600 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2020-08-07T15:06:32.601 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2020-08-07T15:06:32.601 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2020-08-07T15:06:32.601 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_debug_drop_ping_duration": "0", 2020-08-07T15:06:32.601 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_debug_drop_ping_probability": "0.000000", 2020-08-07T15:06:32.601 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_debug_feed_pullee": "-1", 2020-08-07T15:06:32.602 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_debug_inject_copyfrom_error": "false", 2020-08-07T15:06:32.602 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2020-08-07T15:06:32.602 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2020-08-07T15:06:32.602 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_debug_misdirected_ops": "false", 2020-08-07T15:06:32.602 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_debug_no_acting_change": "false", 2020-08-07T15:06:32.603 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_debug_no_purge_strays": "false", 2020-08-07T15:06:32.603 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_debug_op_order": "false", 2020-08-07T15:06:32.603 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_debug_pg_log_writeout": "false", 2020-08-07T15:06:32.603 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_debug_pretend_recovery_active": "false", 2020-08-07T15:06:32.603 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_debug_random_push_read_error": "0.000000", 2020-08-07T15:06:32.603 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2020-08-07T15:06:32.604 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_debug_shutdown": "false", 2020-08-07T15:06:32.604 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2020-08-07T15:06:32.604 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2020-08-07T15:06:32.604 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_debug_verify_cached_snaps": "false", 2020-08-07T15:06:32.604 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_debug_verify_missing_on_start": "false", 2020-08-07T15:06:32.605 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_debug_verify_snaps": "false", 2020-08-07T15:06:32.605 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_debug_verify_stray_on_activate": "false", 2020-08-07T15:06:32.605 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_deep_scrub_interval": "604800.000000", 2020-08-07T15:06:32.605 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_deep_scrub_keys": "1024", 2020-08-07T15:06:32.605 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2020-08-07T15:06:32.606 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2020-08-07T15:06:32.606 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2020-08-07T15:06:32.606 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_deep_scrub_stride": "524288", 2020-08-07T15:06:32.606 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2020-08-07T15:06:32.606 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_default_data_pool_replay_window": "5", 2020-08-07T15:06:32.607 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_default_notify_timeout": "30", 2020-08-07T15:06:32.607 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_delete_sleep": "0.000000", 2020-08-07T15:06:32.607 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_delete_sleep_hdd": "5.000000", 2020-08-07T15:06:32.607 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_delete_sleep_hybrid": "2.000000", 2020-08-07T15:06:32.607 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_delete_sleep_ssd": "0.000000", 2020-08-07T15:06:32.608 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_discard_disconnected_ops": "true", 2020-08-07T15:06:32.608 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_enable_op_tracker": "true", 2020-08-07T15:06:32.608 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_erasure_code_plugins": "jerasure lrc isa", 2020-08-07T15:06:32.608 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_failsafe_full_ratio": "0.970000", 2020-08-07T15:06:32.608 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_fast_fail_on_connection_refused": "true", 2020-08-07T15:06:32.608 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_fast_info": "true", 2020-08-07T15:06:32.609 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_fast_shutdown": "true", 2020-08-07T15:06:32.609 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_find_best_info_ignore_history_les": "false", 2020-08-07T15:06:32.609 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_force_auth_primary_missing_objects": "100", 2020-08-07T15:06:32.609 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2020-08-07T15:06:32.609 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_function_tracing": "false", 2020-08-07T15:06:32.610 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_heartbeat_grace": "20", 2020-08-07T15:06:32.610 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_heartbeat_interval": "6", 2020-08-07T15:06:32.610 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2020-08-07T15:06:32.610 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_heartbeat_min_peers": "10", 2020-08-07T15:06:32.610 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_heartbeat_min_size": "2000", 2020-08-07T15:06:32.610 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_heartbeat_stale": "600", 2020-08-07T15:06:32.611 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2020-08-07T15:06:32.611 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_hit_set_max_size": "100000", 2020-08-07T15:06:32.611 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_hit_set_min_size": "1000", 2020-08-07T15:06:32.611 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_hit_set_namespace": ".ceph-internal", 2020-08-07T15:06:32.611 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_ignore_stale_divergent_priors": "false", 2020-08-07T15:06:32.612 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2020-08-07T15:06:32.612 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_inject_failure_on_pg_removal": "false", 2020-08-07T15:06:32.612 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2020-08-07T15:06:32.612 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_journal_flush_on_shutdown": "true", 2020-08-07T15:06:32.612 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_journal_size": "5120", 2020-08-07T15:06:32.613 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_kill_backfill_at": "0", 2020-08-07T15:06:32.613 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_loop_before_reset_tphandle": "64", 2020-08-07T15:06:32.613 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_map_cache_size": "50", 2020-08-07T15:06:32.613 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_map_dedup": "true", 2020-08-07T15:06:32.613 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_map_message_max": "40", 2020-08-07T15:06:32.614 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_map_message_max_bytes": "10485760", 2020-08-07T15:06:32.614 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_map_share_max_epochs": "40", 2020-08-07T15:06:32.614 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_max_attr_name_len": "100", 2020-08-07T15:06:32.614 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_max_attr_size": "0", 2020-08-07T15:06:32.614 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_max_backfills": "1", 2020-08-07T15:06:32.614 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_max_markdown_count": "5", 2020-08-07T15:06:32.615 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_max_markdown_period": "600", 2020-08-07T15:06:32.615 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_max_object_name_len": "2048", 2020-08-07T15:06:32.615 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_max_object_namespace_len": "256", 2020-08-07T15:06:32.615 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_max_object_size": "134217728", 2020-08-07T15:06:32.615 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2020-08-07T15:06:32.616 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_max_omap_entries_per_request": "1024", 2020-08-07T15:06:32.616 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_max_pg_blocked_by": "16", 2020-08-07T15:06:32.616 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_max_pg_log_entries": "2", 2020-08-07T15:06:32.616 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2020-08-07T15:06:32.616 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_max_pgls": "1024", 2020-08-07T15:06:32.617 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_max_push_cost": "8388608", 2020-08-07T15:06:32.617 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_max_push_objects": "10", 2020-08-07T15:06:32.617 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_max_scrubs": "1", 2020-08-07T15:06:32.617 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2020-08-07T15:06:32.618 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_max_trimming_pgs": "2", 2020-08-07T15:06:32.618 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_max_write_op_reply_len": "32", 2020-08-07T15:06:32.618 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_max_write_size": "90", 2020-08-07T15:06:32.618 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2020-08-07T15:06:32.619 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_mclock_scheduler_background_best_effort_lim": "999999", 2020-08-07T15:06:32.619 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_mclock_scheduler_background_best_effort_res": "1", 2020-08-07T15:06:32.619 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2020-08-07T15:06:32.619 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_mclock_scheduler_background_recovery_lim": "999999", 2020-08-07T15:06:32.619 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_mclock_scheduler_background_recovery_res": "1", 2020-08-07T15:06:32.620 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2020-08-07T15:06:32.620 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_mclock_scheduler_client_lim": "999999", 2020-08-07T15:06:32.620 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_mclock_scheduler_client_res": "1", 2020-08-07T15:06:32.620 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_mclock_scheduler_client_wgt": "1", 2020-08-07T15:06:32.620 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_memory_base": "805306368", 2020-08-07T15:06:32.620 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_memory_cache_min": "134217728", 2020-08-07T15:06:32.621 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_memory_cache_resize_interval": "1.000000", 2020-08-07T15:06:32.621 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_memory_expected_fragmentation": "0.150000", 2020-08-07T15:06:32.621 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_memory_target": "4294967296", 2020-08-07T15:06:32.621 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2020-08-07T15:06:32.621 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_min_pg_log_entries": "1", 2020-08-07T15:06:32.622 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_min_recovery_priority": "0", 2020-08-07T15:06:32.622 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_mon_ack_timeout": "30.000000", 2020-08-07T15:06:32.622 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_mon_heartbeat_interval": "30", 2020-08-07T15:06:32.622 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2020-08-07T15:06:32.622 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_mon_report_interval": "5", 2020-08-07T15:06:32.623 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_mon_report_max_in_flight": "2", 2020-08-07T15:06:32.623 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_mon_shutdown_timeout": "5.000000", 2020-08-07T15:06:32.623 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_num_cache_shards": "32", 2020-08-07T15:06:32.623 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_num_op_tracker_shard": "32", 2020-08-07T15:06:32.623 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_numa_auto_affinity": "true", 2020-08-07T15:06:32.624 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_numa_node": "-1", 2020-08-07T15:06:32.624 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_numa_prefer_iface": "true", 2020-08-07T15:06:32.624 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_object_clean_region_max_num_intervals": "1000", 2020-08-07T15:06:32.624 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_objecter_finishers": "1", 2020-08-07T15:06:32.624 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_objectstore": "bluestore", 2020-08-07T15:06:32.624 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_objectstore_fuse": "false", 2020-08-07T15:06:32.625 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_objectstore_tracing": "false", 2020-08-07T15:06:32.625 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_op_complaint_time": "30.000000", 2020-08-07T15:06:32.625 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_op_history_duration": "600", 2020-08-07T15:06:32.625 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_op_history_size": "20", 2020-08-07T15:06:32.625 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_op_history_slow_op_size": "20", 2020-08-07T15:06:32.626 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2020-08-07T15:06:32.626 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_op_log_threshold": "5", 2020-08-07T15:06:32.626 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_op_num_shards": "0", 2020-08-07T15:06:32.626 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_op_num_shards_hdd": "5", 2020-08-07T15:06:32.627 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_op_num_shards_ssd": "8", 2020-08-07T15:06:32.627 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_op_num_threads_per_shard": "0", 2020-08-07T15:06:32.627 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2020-08-07T15:06:32.627 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2020-08-07T15:06:32.627 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2020-08-07T15:06:32.627 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_op_pq_min_cost": "65536", 2020-08-07T15:06:32.628 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_op_queue": "wpq", 2020-08-07T15:06:32.628 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_op_queue_cut_off": "high", 2020-08-07T15:06:32.628 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_op_thread_suicide_timeout": "150", 2020-08-07T15:06:32.628 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_op_thread_timeout": "15", 2020-08-07T15:06:32.628 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_open_classes_on_start": "true", 2020-08-07T15:06:32.629 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_os_flags": "0", 2020-08-07T15:06:32.629 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_peering_op_priority": "255", 2020-08-07T15:06:32.629 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pg_delete_cost": "1048576", 2020-08-07T15:06:32.629 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pg_delete_priority": "5", 2020-08-07T15:06:32.629 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2020-08-07T15:06:32.630 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2020-08-07T15:06:32.630 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pg_log_dups_tracked": "3000", 2020-08-07T15:06:32.630 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pg_log_trim_max": "10000", 2020-08-07T15:06:32.630 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pg_log_trim_min": "0", 2020-08-07T15:06:32.630 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2020-08-07T15:06:32.630 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pg_object_context_cache_count": "64", 2020-08-07T15:06:32.631 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pg_stat_report_interval_max": "500", 2020-08-07T15:06:32.631 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2020-08-07T15:06:32.631 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pool_default_cache_min_evict_age": "0", 2020-08-07T15:06:32.631 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pool_default_cache_min_flush_age": "0", 2020-08-07T15:06:32.631 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2020-08-07T15:06:32.632 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2020-08-07T15:06:32.632 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2020-08-07T15:06:32.632 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pool_default_crush_rule": "-1", 2020-08-07T15:06:32.632 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pool_default_ec_fast_read": "false", 2020-08-07T15:06:32.632 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd", 2020-08-07T15:06:32.633 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pool_default_flag_hashpspool": "true", 2020-08-07T15:06:32.633 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pool_default_flag_nodelete": "false", 2020-08-07T15:06:32.633 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pool_default_flag_nopgchange": "false", 2020-08-07T15:06:32.634 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pool_default_flag_nosizechange": "false", 2020-08-07T15:06:32.634 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pool_default_flags": "0", 2020-08-07T15:06:32.635 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2020-08-07T15:06:32.635 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pool_default_min_size": "2", 2020-08-07T15:06:32.635 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2020-08-07T15:06:32.635 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pool_default_pg_num": "32", 2020-08-07T15:06:32.636 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pool_default_pgp_num": "0", 2020-08-07T15:06:32.636 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2020-08-07T15:06:32.636 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pool_default_size": "3", 2020-08-07T15:06:32.636 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pool_default_type": "replicated", 2020-08-07T15:06:32.637 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2020-08-07T15:06:32.637 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_pool_use_gmt_hitset": "true", 2020-08-07T15:06:32.637 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_push_per_object_cost": "1000", 2020-08-07T15:06:32.637 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_read_ec_check_for_errors": "false", 2020-08-07T15:06:32.638 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_recover_clone_overlap": "true", 2020-08-07T15:06:32.638 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_recover_clone_overlap_limit": "10", 2020-08-07T15:06:32.638 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_recovery_cost": "20971520", 2020-08-07T15:06:32.639 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_recovery_delay_start": "0.000000", 2020-08-07T15:06:32.639 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_recovery_max_active": "0", 2020-08-07T15:06:32.639 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_recovery_max_active_hdd": "3", 2020-08-07T15:06:32.639 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_recovery_max_active_ssd": "10", 2020-08-07T15:06:32.640 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_recovery_max_chunk": "8388608", 2020-08-07T15:06:32.648 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2020-08-07T15:06:32.649 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_recovery_max_single_start": "1", 2020-08-07T15:06:32.649 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_recovery_op_priority": "3", 2020-08-07T15:06:32.649 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_recovery_op_warn_multiple": "16", 2020-08-07T15:06:32.649 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_recovery_priority": "5", 2020-08-07T15:06:32.650 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_recovery_retry_interval": "30.000000", 2020-08-07T15:06:32.650 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_recovery_sleep": "0.000000", 2020-08-07T15:06:32.650 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_recovery_sleep_hdd": "0.100000", 2020-08-07T15:06:32.650 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2020-08-07T15:06:32.650 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_recovery_sleep_ssd": "0.000000", 2020-08-07T15:06:32.651 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_repair_during_recovery": "false", 2020-08-07T15:06:32.651 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_requested_scrub_priority": "120", 2020-08-07T15:06:32.651 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_rollback_to_cluster_snap": "", 2020-08-07T15:06:32.651 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_scrub_auto_repair": "false", 2020-08-07T15:06:32.652 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_scrub_auto_repair_num_errors": "5", 2020-08-07T15:06:32.652 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_scrub_backoff_ratio": "0.660000", 2020-08-07T15:06:32.652 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_scrub_begin_hour": "0", 2020-08-07T15:06:32.652 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_scrub_begin_week_day": "0", 2020-08-07T15:06:32.653 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_scrub_chunk_max": "25", 2020-08-07T15:06:32.653 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_scrub_chunk_min": "5", 2020-08-07T15:06:32.653 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_scrub_cost": "52428800", 2020-08-07T15:06:32.653 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_scrub_during_recovery": "false", 2020-08-07T15:06:32.654 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_scrub_end_hour": "24", 2020-08-07T15:06:32.654 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_scrub_end_week_day": "7", 2020-08-07T15:06:32.655 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_scrub_extended_sleep": "0.000000", 2020-08-07T15:06:32.655 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2020-08-07T15:06:32.655 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_scrub_invalid_stats": "true", 2020-08-07T15:06:32.655 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_scrub_load_threshold": "0.500000", 2020-08-07T15:06:32.656 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_scrub_max_interval": "604800.000000", 2020-08-07T15:06:32.656 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_scrub_max_preemptions": "5", 2020-08-07T15:06:32.656 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_scrub_min_interval": "86400.000000", 2020-08-07T15:06:32.656 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_scrub_priority": "5", 2020-08-07T15:06:32.656 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_scrub_sleep": "0.000000", 2020-08-07T15:06:32.657 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_shutdown_pgref_assert": "false", 2020-08-07T15:06:32.657 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_skip_data_digest": "false", 2020-08-07T15:06:32.657 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_smart_report_timeout": "5", 2020-08-07T15:06:32.657 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_snap_trim_cost": "1048576", 2020-08-07T15:06:32.658 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_snap_trim_priority": "5", 2020-08-07T15:06:32.658 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_snap_trim_sleep": "0.000000", 2020-08-07T15:06:32.658 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2020-08-07T15:06:32.658 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2020-08-07T15:06:32.659 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2020-08-07T15:06:32.659 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_stats_ack_timeout_decay": "0.900000", 2020-08-07T15:06:32.659 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_stats_ack_timeout_factor": "2.000000", 2020-08-07T15:06:32.659 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2020-08-07T15:06:32.659 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_target_transaction_size": "30", 2020-08-07T15:06:32.660 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_tier_default_cache_hit_set_count": "4", 2020-08-07T15:06:32.660 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2020-08-07T15:06:32.660 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2020-08-07T15:06:32.660 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2020-08-07T15:06:32.661 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2020-08-07T15:06:32.662 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2020-08-07T15:06:32.662 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2020-08-07T15:06:32.662 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_tier_default_cache_mode": "writeback", 2020-08-07T15:06:32.663 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2020-08-07T15:06:32.663 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_tier_promote_max_objects_sec": "25", 2020-08-07T15:06:32.663 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_tracing": "false", 2020-08-07T15:06:32.663 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_use_stale_snap": "false", 2020-08-07T15:06:32.664 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2020-08-07T15:06:32.664 INFO:teuthology.orchestra.run.smithi074.stdout: "osdc_blkin_trace_all": "false", 2020-08-07T15:06:32.664 INFO:teuthology.orchestra.run.smithi074.stdout: "paxos_kill_at": "0", 2020-08-07T15:06:32.664 INFO:teuthology.orchestra.run.smithi074.stdout: "paxos_max_join_drift": "10", 2020-08-07T15:06:32.664 INFO:teuthology.orchestra.run.smithi074.stdout: "paxos_min": "500", 2020-08-07T15:06:32.665 INFO:teuthology.orchestra.run.smithi074.stdout: "paxos_min_wait": "0.050000", 2020-08-07T15:06:32.665 INFO:teuthology.orchestra.run.smithi074.stdout: "paxos_propose_interval": "1.000000", 2020-08-07T15:06:32.665 INFO:teuthology.orchestra.run.smithi074.stdout: "paxos_service_trim_max": "500", 2020-08-07T15:06:32.666 INFO:teuthology.orchestra.run.smithi074.stdout: "paxos_service_trim_min": "10", 2020-08-07T15:06:32.666 INFO:teuthology.orchestra.run.smithi074.stdout: "paxos_stash_full_interval": "25", 2020-08-07T15:06:32.666 INFO:teuthology.orchestra.run.smithi074.stdout: "paxos_trim_max": "500", 2020-08-07T15:06:32.666 INFO:teuthology.orchestra.run.smithi074.stdout: "paxos_trim_min": "250", 2020-08-07T15:06:32.667 INFO:teuthology.orchestra.run.smithi074.stdout: "perf": "true", 2020-08-07T15:06:32.667 INFO:teuthology.orchestra.run.smithi074.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2020-08-07T15:06:32.667 INFO:teuthology.orchestra.run.smithi074.stdout: "plugin_crypto_accelerator": "crypto_isal", 2020-08-07T15:06:32.667 INFO:teuthology.orchestra.run.smithi074.stdout: "plugin_dir": "/usr/lib64/ceph", 2020-08-07T15:06:32.667 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "-", 2020-08-07T15:06:32.668 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrv": "", 2020-08-07T15:06:32.668 INFO:teuthology.orchestra.run.smithi074.stdout: "public_bind_addr": "-", 2020-08-07T15:06:32.668 INFO:teuthology.orchestra.run.smithi074.stdout: "public_network": "", 2020-08-07T15:06:32.668 INFO:teuthology.orchestra.run.smithi074.stdout: "public_network_interface": "", 2020-08-07T15:06:32.669 INFO:teuthology.orchestra.run.smithi074.stdout: "qat_compressor_enabled": "false", 2020-08-07T15:06:32.669 INFO:teuthology.orchestra.run.smithi074.stdout: "rados_mon_op_timeout": "0.000000", 2020-08-07T15:06:32.670 INFO:teuthology.orchestra.run.smithi074.stdout: "rados_osd_op_timeout": "0.000000", 2020-08-07T15:06:32.671 INFO:teuthology.orchestra.run.smithi074.stdout: "rados_tracing": "false", 2020-08-07T15:06:32.671 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_atime_update_interval": "60", 2020-08-07T15:06:32.671 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2020-08-07T15:06:32.671 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_balance_parent_reads": "false", 2020-08-07T15:06:32.672 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_balance_snap_reads": "false", 2020-08-07T15:06:32.672 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_blacklist_expire_seconds": "0", 2020-08-07T15:06:32.672 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_blacklist_on_break_lock": "true", 2020-08-07T15:06:32.672 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_blkin_trace_all": "false", 2020-08-07T15:06:32.672 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_cache": "true", 2020-08-07T15:06:32.673 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_cache_block_writes_upfront": "false", 2020-08-07T15:06:32.673 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_cache_max_dirty": "25165824", 2020-08-07T15:06:32.673 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_cache_max_dirty_age": "1.000000", 2020-08-07T15:06:32.673 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_cache_max_dirty_object": "0", 2020-08-07T15:06:32.673 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_cache_policy": "writearound", 2020-08-07T15:06:32.674 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_cache_size": "33554432", 2020-08-07T15:06:32.674 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_cache_target_dirty": "16777216", 2020-08-07T15:06:32.674 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_cache_writethrough_until_flush": "true", 2020-08-07T15:06:32.674 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_clone_copy_on_read": "false", 2020-08-07T15:06:32.674 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_compression_hint": "none", 2020-08-07T15:06:32.675 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_concurrent_management_ops": "10", 2020-08-07T15:06:32.675 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_default_clone_format": "auto", 2020-08-07T15:06:32.675 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_default_data_pool": "", 2020-08-07T15:06:32.675 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_default_features": "61", 2020-08-07T15:06:32.675 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_default_format": "2", 2020-08-07T15:06:32.676 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_default_map_options": "", 2020-08-07T15:06:32.676 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_default_order": "22", 2020-08-07T15:06:32.676 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_default_pool": "rbd", 2020-08-07T15:06:32.677 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2020-08-07T15:06:32.677 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_default_stripe_count": "0", 2020-08-07T15:06:32.677 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_default_stripe_unit": "0", 2020-08-07T15:06:32.677 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_disable_zero_copy_writes": "true", 2020-08-07T15:06:32.678 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_discard_granularity_bytes": "65536", 2020-08-07T15:06:32.678 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_discard_on_zeroed_write_same": "true", 2020-08-07T15:06:32.678 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_enable_alloc_hint": "true", 2020-08-07T15:06:32.678 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2020-08-07T15:06:32.678 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_io_scheduler": "simple", 2020-08-07T15:06:32.679 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2020-08-07T15:06:32.679 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_journal_commit_age": "5.000000", 2020-08-07T15:06:32.679 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2020-08-07T15:06:32.679 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_journal_max_payload_bytes": "16384", 2020-08-07T15:06:32.679 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_journal_object_flush_age": "0.000000", 2020-08-07T15:06:32.679 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_journal_object_flush_bytes": "1048576", 2020-08-07T15:06:32.680 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_journal_object_flush_interval": "0", 2020-08-07T15:06:32.680 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2020-08-07T15:06:32.680 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2020-08-07T15:06:32.680 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_journal_order": "24", 2020-08-07T15:06:32.680 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_journal_pool": "", 2020-08-07T15:06:32.681 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_journal_splay_width": "4", 2020-08-07T15:06:32.681 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_localize_parent_reads": "false", 2020-08-07T15:06:32.681 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_localize_snap_reads": "false", 2020-08-07T15:06:32.681 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2020-08-07T15:06:32.682 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2020-08-07T15:06:32.682 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2020-08-07T15:06:32.682 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2020-08-07T15:06:32.682 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2020-08-07T15:06:32.683 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2020-08-07T15:06:32.683 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirror_image_policy_type": "simple", 2020-08-07T15:06:32.683 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2020-08-07T15:06:32.683 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirror_image_state_check_interval": "30", 2020-08-07T15:06:32.683 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2020-08-07T15:06:32.684 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2020-08-07T15:06:32.684 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2020-08-07T15:06:32.684 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2020-08-07T15:06:32.685 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2020-08-07T15:06:32.685 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirror_memory_autotune": "true", 2020-08-07T15:06:32.685 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirror_memory_base": "805306368", 2020-08-07T15:06:32.685 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2020-08-07T15:06:32.685 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirror_memory_cache_min": "134217728", 2020-08-07T15:06:32.686 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2020-08-07T15:06:32.686 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2020-08-07T15:06:32.686 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirror_memory_target": "4294967296", 2020-08-07T15:06:32.686 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirror_perf_stats_prio": "5", 2020-08-07T15:06:32.687 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2020-08-07T15:06:32.687 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2020-08-07T15:06:32.687 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirroring_delete_delay": "0", 2020-08-07T15:06:32.687 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirroring_max_mirroring_snapshots": "3", 2020-08-07T15:06:32.687 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirroring_replay_delay": "0", 2020-08-07T15:06:32.688 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2020-08-07T15:06:32.688 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2020-08-07T15:06:32.688 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_move_to_trash_on_remove": "false", 2020-08-07T15:06:32.688 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2020-08-07T15:06:32.689 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_mtime_update_interval": "60", 2020-08-07T15:06:32.689 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_non_blocking_aio": "true", 2020-08-07T15:06:32.689 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_op_thread_timeout": "60", 2020-08-07T15:06:32.689 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_op_threads": "1", 2020-08-07T15:06:32.690 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_parent_cache_enabled": "false", 2020-08-07T15:06:32.690 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_plugins": "", 2020-08-07T15:06:32.690 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_qos_bps_burst": "0", 2020-08-07T15:06:32.690 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_qos_bps_burst_seconds": "1", 2020-08-07T15:06:32.690 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_qos_bps_limit": "0", 2020-08-07T15:06:32.691 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_qos_iops_burst": "0", 2020-08-07T15:06:32.691 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_qos_iops_burst_seconds": "1", 2020-08-07T15:06:32.691 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_qos_iops_limit": "0", 2020-08-07T15:06:32.691 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_qos_read_bps_burst": "0", 2020-08-07T15:06:32.692 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2020-08-07T15:06:32.692 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_qos_read_bps_limit": "0", 2020-08-07T15:06:32.692 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_qos_read_iops_burst": "0", 2020-08-07T15:06:32.692 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2020-08-07T15:06:32.693 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_qos_read_iops_limit": "0", 2020-08-07T15:06:32.693 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_qos_schedule_tick_min": "50", 2020-08-07T15:06:32.693 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_qos_write_bps_burst": "0", 2020-08-07T15:06:32.693 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2020-08-07T15:06:32.694 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_qos_write_bps_limit": "0", 2020-08-07T15:06:32.694 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_qos_write_iops_burst": "0", 2020-08-07T15:06:32.694 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2020-08-07T15:06:32.694 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_qos_write_iops_limit": "0", 2020-08-07T15:06:32.694 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_quiesce_notification_attempts": "10", 2020-08-07T15:06:32.695 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_read_from_replica_policy": "default", 2020-08-07T15:06:32.695 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2020-08-07T15:06:32.695 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_readahead_max_bytes": "524288", 2020-08-07T15:06:32.695 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_readahead_trigger_requests": "10", 2020-08-07T15:06:32.696 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_request_timed_out_seconds": "30", 2020-08-07T15:06:32.696 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_rwl_enabled": "false", 2020-08-07T15:06:32.696 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_rwl_log_periodic_stats": "false", 2020-08-07T15:06:32.696 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_rwl_path": "/tmp", 2020-08-07T15:06:32.696 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_rwl_size": "1073741824", 2020-08-07T15:06:32.697 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_skip_partial_discard": "true", 2020-08-07T15:06:32.697 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2020-08-07T15:06:32.697 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_tracing": "false", 2020-08-07T15:06:32.697 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_validate_names": "true", 2020-08-07T15:06:32.697 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd_validate_pool": "true", 2020-08-07T15:06:32.698 INFO:teuthology.orchestra.run.smithi074.stdout: "restapi_base_url": "", 2020-08-07T15:06:32.698 INFO:teuthology.orchestra.run.smithi074.stdout: "restapi_log_level": "", 2020-08-07T15:06:32.698 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_acl_grants_max_num": "100", 2020-08-07T15:06:32.698 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_admin_entry": "admin", 2020-08-07T15:06:32.698 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_barbican_url": "", 2020-08-07T15:06:32.699 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2020-08-07T15:06:32.699 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_bucket_default_quota_max_size": "-1", 2020-08-07T15:06:32.699 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_bucket_index_max_aio": "128", 2020-08-07T15:06:32.699 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_bucket_quota_cache_size": "10000", 2020-08-07T15:06:32.700 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_bucket_quota_soft_threshold": "0.950000", 2020-08-07T15:06:32.700 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_bucket_quota_ttl": "600", 2020-08-07T15:06:32.700 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_cache_enabled": "true", 2020-08-07T15:06:32.700 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_cache_expiry_interval": "900", 2020-08-07T15:06:32.701 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_cache_lru_size": "10000", 2020-08-07T15:06:32.701 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_content_length_compat": "false", 2020-08-07T15:06:32.701 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_copy_obj_progress": "true", 2020-08-07T15:06:32.701 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2020-08-07T15:06:32.702 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_cors_rules_max_num": "100", 2020-08-07T15:06:32.702 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_cross_domain_policy": "", 2020-08-07T15:06:32.702 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_crypt_default_encryption_key": "", 2020-08-07T15:06:32.702 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_crypt_require_ssl": "true", 2020-08-07T15:06:32.703 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2020-08-07T15:06:32.703 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2020-08-07T15:06:32.703 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_crypt_suppress_logs": "true", 2020-08-07T15:06:32.703 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_crypt_vault_addr": "", 2020-08-07T15:06:32.703 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_crypt_vault_auth": "token", 2020-08-07T15:06:32.704 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_crypt_vault_namespace": "", 2020-08-07T15:06:32.704 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_crypt_vault_prefix": "", 2020-08-07T15:06:32.704 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_crypt_vault_secret_engine": "transit", 2020-08-07T15:06:32.704 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_crypt_vault_token_file": "", 2020-08-07T15:06:32.704 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_curl_low_speed_limit": "1024", 2020-08-07T15:06:32.705 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_curl_low_speed_time": "300", 2020-08-07T15:06:32.705 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_curl_wait_timeout_ms": "1000", 2020-08-07T15:06:32.705 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2020-08-07T15:06:32.705 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_data_log_changes_size": "1000", 2020-08-07T15:06:32.705 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_data_log_num_shards": "128", 2020-08-07T15:06:32.706 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_data_log_obj_prefix": "data_log", 2020-08-07T15:06:32.706 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_data_log_window": "30", 2020-08-07T15:06:32.706 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_data_notify_interval_msec": "200", 2020-08-07T15:06:32.706 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_default_realm_info_oid": "default.realm", 2020-08-07T15:06:32.706 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_default_region_info_oid": "default.region", 2020-08-07T15:06:32.707 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_default_zone_info_oid": "default.zone", 2020-08-07T15:06:32.707 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2020-08-07T15:06:32.707 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_defer_to_bucket_acls": "", 2020-08-07T15:06:32.707 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_delete_multi_obj_max_num": "1000", 2020-08-07T15:06:32.707 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_dmclock_admin_lim": "0.000000", 2020-08-07T15:06:32.708 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_dmclock_admin_res": "100.000000", 2020-08-07T15:06:32.708 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2020-08-07T15:06:32.708 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_dmclock_auth_lim": "0.000000", 2020-08-07T15:06:32.708 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_dmclock_auth_res": "200.000000", 2020-08-07T15:06:32.708 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2020-08-07T15:06:32.709 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_dmclock_data_lim": "0.000000", 2020-08-07T15:06:32.709 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_dmclock_data_res": "500.000000", 2020-08-07T15:06:32.709 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_dmclock_data_wgt": "500.000000", 2020-08-07T15:06:32.709 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2020-08-07T15:06:32.709 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_dmclock_metadata_res": "500.000000", 2020-08-07T15:06:32.710 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2020-08-07T15:06:32.710 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_dns_name": "", 2020-08-07T15:06:32.710 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_dns_s3website_name": "", 2020-08-07T15:06:32.710 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_dynamic_resharding": "true", 2020-08-07T15:06:32.710 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, pubsub", 2020-08-07T15:06:32.711 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_enable_gc_threads": "true", 2020-08-07T15:06:32.711 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_enable_lc_threads": "true", 2020-08-07T15:06:32.711 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_enable_ops_log": "false", 2020-08-07T15:06:32.711 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_enable_quota_threads": "true", 2020-08-07T15:06:32.711 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_enable_static_website": "false", 2020-08-07T15:06:32.712 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_enable_usage_log": "false", 2020-08-07T15:06:32.712 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_enforce_swift_acls": "true", 2020-08-07T15:06:32.712 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_exit_timeout_secs": "120", 2020-08-07T15:06:32.712 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_expose_bucket": "false", 2020-08-07T15:06:32.712 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_extended_http_attrs": "", 2020-08-07T15:06:32.713 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_fcgi_socket_backlog": "1024", 2020-08-07T15:06:32.713 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2020-08-07T15:06:32.713 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_frontends": "beast port=7480", 2020-08-07T15:06:32.713 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_gc_max_concurrent_io": "10", 2020-08-07T15:06:32.713 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_gc_max_deferred": "50", 2020-08-07T15:06:32.714 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2020-08-07T15:06:32.714 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_gc_max_objs": "32", 2020-08-07T15:06:32.714 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_gc_max_queue_size": "134213632", 2020-08-07T15:06:32.714 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_gc_max_trim_chunk": "16", 2020-08-07T15:06:32.714 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_gc_obj_min_wait": "7200", 2020-08-07T15:06:32.715 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_gc_processor_max_time": "3600", 2020-08-07T15:06:32.715 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_gc_processor_period": "3600", 2020-08-07T15:06:32.715 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_get_obj_max_req_size": "4194304", 2020-08-07T15:06:32.715 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_get_obj_window_size": "16777216", 2020-08-07T15:06:32.715 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_healthcheck_disabling_path": "", 2020-08-07T15:06:32.716 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_host": "", 2020-08-07T15:06:32.716 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_ignore_get_invalid_range": "false", 2020-08-07T15:06:32.716 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_init_timeout": "300", 2020-08-07T15:06:32.716 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2020-08-07T15:06:32.716 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_keystone_accepted_admin_roles": "", 2020-08-07T15:06:32.717 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2020-08-07T15:06:32.717 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_keystone_admin_domain": "", 2020-08-07T15:06:32.718 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_keystone_admin_password": "", 2020-08-07T15:06:32.718 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_keystone_admin_password_path": "", 2020-08-07T15:06:32.718 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_keystone_admin_project": "", 2020-08-07T15:06:32.718 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_keystone_admin_tenant": "", 2020-08-07T15:06:32.719 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_keystone_admin_token": "", 2020-08-07T15:06:32.719 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_keystone_admin_token_path": "", 2020-08-07T15:06:32.719 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_keystone_admin_user": "", 2020-08-07T15:06:32.721 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_keystone_api_version": "2", 2020-08-07T15:06:32.722 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_keystone_barbican_domain": "", 2020-08-07T15:06:32.722 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_keystone_barbican_password": "", 2020-08-07T15:06:32.722 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_keystone_barbican_project": "", 2020-08-07T15:06:32.722 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_keystone_barbican_tenant": "", 2020-08-07T15:06:32.723 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_keystone_barbican_user": "", 2020-08-07T15:06:32.723 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_keystone_implicit_tenants": "false", 2020-08-07T15:06:32.723 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_keystone_token_cache_size": "10000", 2020-08-07T15:06:32.723 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_keystone_url": "", 2020-08-07T15:06:32.723 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_keystone_verify_ssl": "true", 2020-08-07T15:06:32.723 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_lc_debug_interval": "-1", 2020-08-07T15:06:32.724 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_lc_lock_max_time": "90", 2020-08-07T15:06:32.724 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_lc_max_objs": "32", 2020-08-07T15:06:32.724 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_lc_max_rules": "1000", 2020-08-07T15:06:32.724 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_lc_max_worker": "3", 2020-08-07T15:06:32.724 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_lc_max_wp_worker": "3", 2020-08-07T15:06:32.725 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_lc_thread_delay": "0", 2020-08-07T15:06:32.725 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2020-08-07T15:06:32.725 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_ldap_dnattr": "uid", 2020-08-07T15:06:32.725 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2020-08-07T15:06:32.725 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_ldap_searchfilter": "", 2020-08-07T15:06:32.726 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2020-08-07T15:06:32.726 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_ldap_uri": "ldaps://", 2020-08-07T15:06:32.726 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2020-08-07T15:06:32.726 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_list_bucket_min_readahead": "1000", 2020-08-07T15:06:32.726 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_list_buckets_max_chunk": "1000", 2020-08-07T15:06:32.727 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_log_http_headers": "", 2020-08-07T15:06:32.727 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_log_nonexistent_bucket": "false", 2020-08-07T15:06:32.727 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2020-08-07T15:06:32.727 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_log_object_name_utc": "false", 2020-08-07T15:06:32.727 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_max_attr_name_len": "0", 2020-08-07T15:06:32.727 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_max_attr_size": "0", 2020-08-07T15:06:32.728 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_max_attrs_num_in_req": "0", 2020-08-07T15:06:32.728 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_max_chunk_size": "4194304", 2020-08-07T15:06:32.728 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_max_concurrent_requests": "1024", 2020-08-07T15:06:32.728 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_max_dynamic_shards": "1999", 2020-08-07T15:06:32.728 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_max_listing_results": "1000", 2020-08-07T15:06:32.729 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_max_notify_retries": "3", 2020-08-07T15:06:32.729 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_max_objs_per_shard": "100000", 2020-08-07T15:06:32.729 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_max_put_param_size": "1048576", 2020-08-07T15:06:32.729 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_max_put_size": "5368709120", 2020-08-07T15:06:32.729 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_max_slo_entries": "1000", 2020-08-07T15:06:32.730 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_md_log_max_shards": "64", 2020-08-07T15:06:32.730 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_md_notify_interval_msec": "200", 2020-08-07T15:06:32.730 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_mime_types_file": "/etc/mime.types", 2020-08-07T15:06:32.730 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_mp_lock_max_time": "600", 2020-08-07T15:06:32.730 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_multipart_min_part_size": "5242880", 2020-08-07T15:06:32.731 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_multipart_part_upload_limit": "10000", 2020-08-07T15:06:32.731 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_nfs_fhcache_partitions": "3", 2020-08-07T15:06:32.731 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_nfs_fhcache_size": "2017", 2020-08-07T15:06:32.731 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2020-08-07T15:06:32.731 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_nfs_lru_lanes": "5", 2020-08-07T15:06:32.732 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_nfs_max_gc": "300", 2020-08-07T15:06:32.732 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_nfs_namespace_expire_secs": "300", 2020-08-07T15:06:32.732 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_nfs_run_gc_threads": "false", 2020-08-07T15:06:32.732 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_nfs_run_lc_threads": "false", 2020-08-07T15:06:32.732 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_nfs_run_quota_threads": "false", 2020-08-07T15:06:32.732 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_nfs_run_sync_thread": "false", 2020-08-07T15:06:32.733 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_nfs_s3_fast_attrs": "false", 2020-08-07T15:06:32.733 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_nfs_write_completion_interval_s": "10", 2020-08-07T15:06:32.733 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_num_async_rados_threads": "32", 2020-08-07T15:06:32.733 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_num_control_oids": "8", 2020-08-07T15:06:32.733 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_num_rados_handles": "1", 2020-08-07T15:06:32.734 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_numa_node": "-1", 2020-08-07T15:06:32.734 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_obj_stripe_size": "4194304", 2020-08-07T15:06:32.734 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_obj_tombstone_cache_size": "1000", 2020-08-07T15:06:32.734 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_objexp_chunk_size": "100", 2020-08-07T15:06:32.734 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_objexp_gc_interval": "600", 2020-08-07T15:06:32.734 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_objexp_hints_num_shards": "127", 2020-08-07T15:06:32.735 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_olh_pending_timeout_sec": "3600", 2020-08-07T15:06:32.735 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_op_thread_suicide_timeout": "0", 2020-08-07T15:06:32.735 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_op_thread_timeout": "600", 2020-08-07T15:06:32.735 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_op_tracing": "false", 2020-08-07T15:06:32.735 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_opa_token": "", 2020-08-07T15:06:32.736 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_opa_url": "", 2020-08-07T15:06:32.736 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_opa_verify_ssl": "true", 2020-08-07T15:06:32.736 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_ops_log_data_backlog": "5242880", 2020-08-07T15:06:32.736 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_ops_log_rados": "true", 2020-08-07T15:06:32.736 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_ops_log_socket_path": "", 2020-08-07T15:06:32.737 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_override_bucket_index_max_shards": "0", 2020-08-07T15:06:32.737 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2020-08-07T15:06:32.737 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_period_push_interval": "2.000000", 2020-08-07T15:06:32.737 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_period_push_interval_max": "30.000000", 2020-08-07T15:06:32.737 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_period_root_pool": ".rgw.root", 2020-08-07T15:06:32.737 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_port": "", 2020-08-07T15:06:32.738 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_print_continue": "true", 2020-08-07T15:06:32.738 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_print_prohibited_content_length": "false", 2020-08-07T15:06:32.738 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_put_obj_max_window_size": "67108864", 2020-08-07T15:06:32.738 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_put_obj_min_window_size": "16777216", 2020-08-07T15:06:32.738 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2020-08-07T15:06:32.739 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_rados_pool_pg_num_min": "8", 2020-08-07T15:06:32.739 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_rados_pool_recovery_priority": "5", 2020-08-07T15:06:32.739 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_rados_tracing": "false", 2020-08-07T15:06:32.739 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_realm": "", 2020-08-07T15:06:32.739 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_realm_root_pool": ".rgw.root", 2020-08-07T15:06:32.740 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_region": "", 2020-08-07T15:06:32.740 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_region_root_pool": ".rgw.root", 2020-08-07T15:06:32.740 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_relaxed_region_enforcement": "false", 2020-08-07T15:06:32.740 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_relaxed_s3_bucket_names": "false", 2020-08-07T15:06:32.740 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2020-08-07T15:06:32.740 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_request_uri": "", 2020-08-07T15:06:32.741 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_reshard_batch_size": "64", 2020-08-07T15:06:32.741 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_reshard_bucket_lock_duration": "360", 2020-08-07T15:06:32.741 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_reshard_max_aio": "128", 2020-08-07T15:06:32.741 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_reshard_num_logs": "16", 2020-08-07T15:06:32.741 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_reshard_thread_interval": "600", 2020-08-07T15:06:32.742 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_resolve_cname": "false", 2020-08-07T15:06:32.742 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_rest_getusage_op_compat": "false", 2020-08-07T15:06:32.742 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_run_sync_thread": "true", 2020-08-07T15:06:32.742 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_s3_auth_order": "sts, external, local", 2020-08-07T15:06:32.742 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_s3_auth_use_keystone": "false", 2020-08-07T15:06:32.743 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_s3_auth_use_ldap": "false", 2020-08-07T15:06:32.743 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_s3_auth_use_rados": "true", 2020-08-07T15:06:32.743 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_s3_auth_use_sts": "false", 2020-08-07T15:06:32.744 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_s3_success_create_obj_status": "0", 2020-08-07T15:06:32.744 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_safe_max_objects_per_shard": "102400", 2020-08-07T15:06:32.744 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_scheduler_type": "throttler", 2020-08-07T15:06:32.744 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_script_uri": "", 2020-08-07T15:06:32.744 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_service_provider_name": "", 2020-08-07T15:06:32.745 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_shard_warning_threshold": "90.000000", 2020-08-07T15:06:32.745 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_socket_path": "", 2020-08-07T15:06:32.745 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_sts_client_id": "", 2020-08-07T15:06:32.745 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_sts_client_secret": "", 2020-08-07T15:06:32.745 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_sts_entry": "sts", 2020-08-07T15:06:32.746 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_sts_key": "sts", 2020-08-07T15:06:32.746 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_sts_max_session_duration": "43200", 2020-08-07T15:06:32.746 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_sts_token_introspection_url": "", 2020-08-07T15:06:32.746 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_swift_account_in_url": "false", 2020-08-07T15:06:32.746 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_swift_auth_entry": "auth", 2020-08-07T15:06:32.747 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_swift_auth_url": "", 2020-08-07T15:06:32.747 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_swift_custom_header": "", 2020-08-07T15:06:32.747 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_swift_enforce_content_length": "false", 2020-08-07T15:06:32.747 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_swift_need_stats": "true", 2020-08-07T15:06:32.747 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_swift_tenant_name": "", 2020-08-07T15:06:32.747 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_swift_token_expiration": "86400", 2020-08-07T15:06:32.748 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_swift_url": "", 2020-08-07T15:06:32.748 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_swift_url_prefix": "swift", 2020-08-07T15:06:32.748 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_swift_versioning_enabled": "false", 2020-08-07T15:06:32.748 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2020-08-07T15:06:32.748 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_sync_lease_period": "120", 2020-08-07T15:06:32.749 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2020-08-07T15:06:32.749 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_sync_log_trim_interval": "1200", 2020-08-07T15:06:32.749 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_sync_log_trim_max_buckets": "16", 2020-08-07T15:06:32.749 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2020-08-07T15:06:32.749 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2020-08-07T15:06:32.750 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_sync_obj_etag_verify": "false", 2020-08-07T15:06:32.750 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_sync_trace_history_size": "4096", 2020-08-07T15:06:32.750 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_sync_trace_per_node_log_size": "32", 2020-08-07T15:06:32.750 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2020-08-07T15:06:32.750 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_thread_pool_size": "512", 2020-08-07T15:06:32.750 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_torrent_comment": "", 2020-08-07T15:06:32.751 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_torrent_createby": "", 2020-08-07T15:06:32.751 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_torrent_encoding": "", 2020-08-07T15:06:32.751 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_torrent_flag": "false", 2020-08-07T15:06:32.751 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_torrent_origin": "", 2020-08-07T15:06:32.751 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_torrent_sha_unit": "524288", 2020-08-07T15:06:32.752 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_torrent_tracker": "", 2020-08-07T15:06:32.752 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_trust_forwarded_https": "false", 2020-08-07T15:06:32.752 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_usage_log_flush_threshold": "1024", 2020-08-07T15:06:32.752 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_usage_log_tick_interval": "30", 2020-08-07T15:06:32.752 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_usage_max_shards": "32", 2020-08-07T15:06:32.753 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_usage_max_user_shards": "1", 2020-08-07T15:06:32.753 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_use_opa_authz": "false", 2020-08-07T15:06:32.753 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_user_default_quota_max_objects": "-1", 2020-08-07T15:06:32.753 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_user_default_quota_max_size": "-1", 2020-08-07T15:06:32.753 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_user_max_buckets": "1000", 2020-08-07T15:06:32.754 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2020-08-07T15:06:32.754 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_user_quota_sync_idle_users": "false", 2020-08-07T15:06:32.754 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_user_quota_sync_interval": "86400", 2020-08-07T15:06:32.754 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_user_quota_sync_wait_time": "86400", 2020-08-07T15:06:32.755 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_user_unique_email": "true", 2020-08-07T15:06:32.755 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_verify_ssl": "true", 2020-08-07T15:06:32.755 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_website_routing_rules_max_num": "50", 2020-08-07T15:06:32.755 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_zone": "", 2020-08-07T15:06:32.755 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_zone_root_pool": ".rgw.root", 2020-08-07T15:06:32.756 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_zonegroup": "", 2020-08-07T15:06:32.756 INFO:teuthology.orchestra.run.smithi074.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2020-08-07T15:06:32.756 INFO:teuthology.orchestra.run.smithi074.stdout: "rocksdb_block_size": "4096", 2020-08-07T15:06:32.756 INFO:teuthology.orchestra.run.smithi074.stdout: "rocksdb_bloom_bits_per_key": "20", 2020-08-07T15:06:32.756 INFO:teuthology.orchestra.run.smithi074.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2020-08-07T15:06:32.756 INFO:teuthology.orchestra.run.smithi074.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "true", 2020-08-07T15:06:32.757 INFO:teuthology.orchestra.run.smithi074.stdout: "rocksdb_cache_row_ratio": "0.000000", 2020-08-07T15:06:32.757 INFO:teuthology.orchestra.run.smithi074.stdout: "rocksdb_cache_shard_bits": "4", 2020-08-07T15:06:32.757 INFO:teuthology.orchestra.run.smithi074.stdout: "rocksdb_cache_size": "536870912", 2020-08-07T15:06:32.757 INFO:teuthology.orchestra.run.smithi074.stdout: "rocksdb_cache_type": "binned_lru", 2020-08-07T15:06:32.758 INFO:teuthology.orchestra.run.smithi074.stdout: "rocksdb_collect_compaction_stats": "false", 2020-08-07T15:06:32.758 INFO:teuthology.orchestra.run.smithi074.stdout: "rocksdb_collect_extended_stats": "false", 2020-08-07T15:06:32.759 INFO:teuthology.orchestra.run.smithi074.stdout: "rocksdb_collect_memory_stats": "false", 2020-08-07T15:06:32.760 INFO:teuthology.orchestra.run.smithi074.stdout: "rocksdb_delete_range_threshold": "1048576", 2020-08-07T15:06:32.760 INFO:teuthology.orchestra.run.smithi074.stdout: "rocksdb_index_type": "binary_search", 2020-08-07T15:06:32.760 INFO:teuthology.orchestra.run.smithi074.stdout: "rocksdb_log_to_ceph_log": "true", 2020-08-07T15:06:32.760 INFO:teuthology.orchestra.run.smithi074.stdout: "rocksdb_metadata_block_size": "4096", 2020-08-07T15:06:32.760 INFO:teuthology.orchestra.run.smithi074.stdout: "rocksdb_partition_filters": "false", 2020-08-07T15:06:32.761 INFO:teuthology.orchestra.run.smithi074.stdout: "rocksdb_perf": "false", 2020-08-07T15:06:32.761 INFO:teuthology.orchestra.run.smithi074.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2020-08-07T15:06:32.761 INFO:teuthology.orchestra.run.smithi074.stdout: "rotating_keys_bootstrap_timeout": "30", 2020-08-07T15:06:32.761 INFO:teuthology.orchestra.run.smithi074.stdout: "rotating_keys_renewal_timeout": "10", 2020-08-07T15:06:32.761 INFO:teuthology.orchestra.run.smithi074.stdout: "run_dir": "/var/run/ceph", 2020-08-07T15:06:32.762 INFO:teuthology.orchestra.run.smithi074.stdout: "setgroup": "", 2020-08-07T15:06:32.762 INFO:teuthology.orchestra.run.smithi074.stdout: "setuser": "", 2020-08-07T15:06:32.762 INFO:teuthology.orchestra.run.smithi074.stdout: "setuser_match_path": "", 2020-08-07T15:06:32.762 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_misplaced_ratio": "0.050000", 2020-08-07T15:06:32.763 INFO:teuthology.orchestra.run.smithi074.stdout: "thp": "false", 2020-08-07T15:06:32.763 INFO:teuthology.orchestra.run.smithi074.stdout: "threadpool_default_timeout": "60", 2020-08-07T15:06:32.763 INFO:teuthology.orchestra.run.smithi074.stdout: "threadpool_empty_queue_max_wait": "2", 2020-08-07T15:06:32.763 INFO:teuthology.orchestra.run.smithi074.stdout: "throttler_perf_counter": "true" 2020-08-07T15:06:32.763 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:06:32.766 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2020-08-07T15:06:32.897 INFO:teuthology.orchestra.run.smithi074.stdout:mon.a: {} 2020-08-07T15:06:32.914 INFO:teuthology.orchestra.run.smithi074.stdout:mon.b: {} 2020-08-07T15:06:32.933 INFO:teuthology.orchestra.run.smithi074.stdout:mon.c: {} 2020-08-07T15:06:32.934 INFO:teuthology.orchestra.run.smithi074.stderr:mon.a: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2020-08-07T15:06:32.934 INFO:teuthology.orchestra.run.smithi074.stderr:mon.b: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2020-08-07T15:06:32.934 INFO:teuthology.orchestra.run.smithi074.stderr:mon.c: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2020-08-07T15:06:32.940 INFO:teuthology.orchestra.run.smithi074:> type ceph-objectstore-tool 2020-08-07T15:06:32.955 INFO:teuthology.orchestra.run.smithi074.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2020-08-07T15:06:32.955 INFO:teuthology.orchestra.run.smithi096:> type ceph-objectstore-tool 2020-08-07T15:06:32.972 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2020-08-07T15:06:32.973 INFO:teuthology.run_tasks:Running task full_sequential... 2020-08-07T15:06:32.994 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2020-08-07T15:06:32.997 INFO:tasks.radosbench:Beginning radosbench... 2020-08-07T15:06:32.997 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2020-08-07T15:06:33.008 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create unique_pool_0 16 2020-08-07T15:06:33.009 INFO:tasks.thrashosds.thrasher:starting do_thrash 2020-08-07T15:06:33.010 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:06:33.010 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:06:33.010 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.719617 2020-08-07T15:06:33.011 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd primary-affinity 1 0.7196170506262938 2020-08-07T15:06:33.012 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2020-08-07T15:06:33.013 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2020-08-07T15:06:33.013 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:06:33.013 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2020-08-07T15:06:33.014 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:06:33.014 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2020-08-07T15:06:33.015 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:06:33.016 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:33.013+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:33.115 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:33.114+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:33.153 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:06:33.153 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:06:33.153 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:06:33.154 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:06:33.163 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:06:33.194 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:06:33.216 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:33.215+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:33.225 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:06:33.264 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:06:33.312 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:06:33.319 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:33.317+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:33.354 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:06:33.382 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:33.382 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:33.393 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:06:33.404 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:06:33.438 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:06:33.439 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:06:33.434+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:06:33.477 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:06:33.477 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:06:33.478 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:06:33.479 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:06:33.479 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:06:33.480 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:06:33.480 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:06:33.480 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:06:33.481 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:06:33.526 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:06:33.521+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:06:33.527 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:33.527 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:33.536 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:06:33.630 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:33.626+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:33.697 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:33.698 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:33.703 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:06:33.736 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:33.728+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:33.824 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:33.825 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:33.835 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:06:33.837 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:33.836+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:06:33.940 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:06:33.938+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:06:33.952 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:33.952 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:33.963 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:06:34.042 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:34.040+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:34.121 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:34.122 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:34.127 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:06:34.153 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:34.146+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:06:34.166 INFO:teuthology.orchestra.run.smithi074.stderr:set osd.1 primary-affinity to 0.719617 (8471602) 2020-08-07T15:06:34.167 INFO:teuthology.orchestra.run.smithi074.stderr:pool 'unique_pool_0' created 2020-08-07T15:06:34.167 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:06:34.179 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2020-08-07T15:06:34.263 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:34.264 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:34.264 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:34.257+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:34.294 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:06:34.358 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:34.348+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:34.452 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:34.450+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:34.457 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:34.457 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:34.473 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:06:34.560 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:34.558+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:34.600 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:34.600 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:34.607 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:06:34.663 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:06:34.662+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:06:34.765 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:34.763+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:34.776 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:34.776 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:34.784 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:06:34.874 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:34.870+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:34.903 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:34.903 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:34.919 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:06:34.972 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:34.970+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:35.055 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:35.056 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:35.057 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:06:35.111 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:35.103+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:35.162 INFO:teuthology.orchestra.run.smithi074.stderr:enabled application 'rados' on pool 'unique_pool_0' 2020-08-07T15:06:35.179 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:35.177+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:06:35.235 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:35.236 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:35.245 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:06:35.288 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:35.285+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:35.383 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:06:35.380+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:06:35.530 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:06:35.533 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:35.533 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:35.534 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:06:35.536 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:35.535+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:35.631 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:35.630+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:35.667 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:35.668 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:35.683 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:06:35.687 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:06:35.714 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:06:35.735 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:06:35.734+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:06:35.736 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:06:35.762 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:06:35.786 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:06:35.808 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:06:35.839 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:06:35.839 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:35.837+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:35.850 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:06:35.850 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:06:35.851 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:06:35.851 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:06:35.851 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:06:35.862 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:06:35.863 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:06:35.863 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:06:35.863 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:06:35.863 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:06:35.863 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:06:35.864 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:06:35.864 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:06:35.864 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:06:35.864 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:06:35.945 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:06:35.940+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:06:36.020 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:06:36.021 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:06:36.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:06:36.022 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [] 2020-08-07T15:06:36.022 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:06:36.044 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:36.043+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:36.046 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:06:36.147 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:36.145+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:06:36.187 INFO:teuthology.orchestra.run.smithi074:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 8192 --object-size 65536 -p unique_pool_0 bench 90 write' 2020-08-07T15:06:36.188 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:06:36.197 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2020-08-07T15:06:36.198 INFO:teuthology.orchestra.run:waiting for 3000 2020-08-07T15:06:36.210 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:06:36.211 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:06:36.211 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2020-08-07T15:06:36.211 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:06:36.220 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:06:36.241 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:hints = 1 2020-08-07T15:06:36.241 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Maintaining 16 concurrent writes of 8192 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2020-08-07T15:06:36.242 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Object prefix: benchmark_data_smithi074_35447 2020-08-07T15:06:36.242 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-08-07T15:06:36.242 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 0 0 0 0 0 0 - 0 2020-08-07T15:06:36.254 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:06:36.252+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:06:36.358 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:36.353+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:06:36.410 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:06:36.410 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:06:36.411 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:06:36.411 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:06:36.411 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:06:36.418 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:06:36.460 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:36.458+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:36.558 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:36.557+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:36.597 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:06:36.598 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:06:36.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:06:36.598 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:06:36.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.598 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:13 3.0 e22/20 3:017cf19c:::benchmark_data_smithi074_35447_object1:head v 22'5, mlcod=0'0)", 2020-08-07T15:06:36.599 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:36.250316+0000", 2020-08-07T15:06:36.599 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.34419814500000001, 2020-08-07T15:06:36.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083617650000000002, 2020-08-07T15:06:36.600 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:36.600 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:36.600 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:36.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:36.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.250316+0000", 2020-08-07T15:06:36.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:36.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:36.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.250316+0000", 2020-08-07T15:06:36.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4708e-05 2020-08-07T15:06:36.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:36.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.250351+0000", 2020-08-07T15:06:36.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9059999999999997e-06 2020-08-07T15:06:36.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:36.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.250361+0000", 2020-08-07T15:06:36.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2020-08-07T15:06:36.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:36.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.250406+0000", 2020-08-07T15:06:36.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7049999999999999e-06 2020-08-07T15:06:36.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:36.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.250413+0000", 2020-08-07T15:06:36.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6908e-05 2020-08-07T15:06:36.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:36.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.250450+0000", 2020-08-07T15:06:36.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1165000000000001e-05 2020-08-07T15:06:36.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:36.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.250471+0000", 2020-08-07T15:06:36.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081499599999999995 2020-08-07T15:06:36.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:36.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.258621+0000", 2020-08-07T15:06:36.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6338000000000002e-05 2020-08-07T15:06:36.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:36.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.258678+0000", 2020-08-07T15:06:36.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083617650000000002 2020-08-07T15:06:36.616 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.616 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:36.617 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.618 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:14 3.0 e22/20 3:017cf19c:::benchmark_data_smithi074_35447_object1:head v 22'6, mlcod=0'0)", 2020-08-07T15:06:36.618 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:36.250580+0000", 2020-08-07T15:06:36.619 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.34393407500000001, 2020-08-07T15:06:36.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081388940000000007, 2020-08-07T15:06:36.620 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:36.620 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:36.620 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:36.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:36.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.250580+0000", 2020-08-07T15:06:36.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:36.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:36.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.250580+0000", 2020-08-07T15:06:36.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2155000000000004e-05 2020-08-07T15:06:36.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:36.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.250642+0000", 2020-08-07T15:06:36.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1489999999999999e-06 2020-08-07T15:06:36.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:36.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.250648+0000", 2020-08-07T15:06:36.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3724999999999998e-05 2020-08-07T15:06:36.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:36.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.250702+0000", 2020-08-07T15:06:36.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9939999999999997e-06 2020-08-07T15:06:36.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:36.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.250708+0000", 2020-08-07T15:06:36.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4888999999999997e-05 2020-08-07T15:06:36.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:36.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.250753+0000", 2020-08-07T15:06:36.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3298e-05 2020-08-07T15:06:36.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:36.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.250766+0000", 2020-08-07T15:06:36.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0079156169999999998 2020-08-07T15:06:36.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:36.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.258682+0000", 2020-08-07T15:06:36.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7067000000000003e-05 2020-08-07T15:06:36.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:36.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.258719+0000", 2020-08-07T15:06:36.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081388940000000007 2020-08-07T15:06:36.646 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.646 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:36.646 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.647 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:15 3.0 e22/20 3:017cf19c:::benchmark_data_smithi074_35447_object1:head v 22'7, mlcod=0'0)", 2020-08-07T15:06:36.647 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:36.251268+0000", 2020-08-07T15:06:36.647 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.34324626600000002, 2020-08-07T15:06:36.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0074892650000000002, 2020-08-07T15:06:36.648 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:36.648 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:36.648 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:36.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:36.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.251268+0000", 2020-08-07T15:06:36.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:36.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:36.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.251268+0000", 2020-08-07T15:06:36.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2771000000000002e-05 2020-08-07T15:06:36.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:36.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.251301+0000", 2020-08-07T15:06:36.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6330000000000002e-06 2020-08-07T15:06:36.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:36.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.251304+0000", 2020-08-07T15:06:36.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6520000000000003e-05 2020-08-07T15:06:36.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:36.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.251341+0000", 2020-08-07T15:06:36.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2499999999999998e-06 2020-08-07T15:06:36.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:36.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.251344+0000", 2020-08-07T15:06:36.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.311e-05 2020-08-07T15:06:36.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:36.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.251377+0000", 2020-08-07T15:06:36.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5007999999999999e-05 2020-08-07T15:06:36.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:36.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.251392+0000", 2020-08-07T15:06:36.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0073281930000000002 2020-08-07T15:06:36.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:36.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.258720+0000", 2020-08-07T15:06:36.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6779999999999997e-05 2020-08-07T15:06:36.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:36.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.258757+0000", 2020-08-07T15:06:36.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0074892650000000002 2020-08-07T15:06:36.659 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.659 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:36.659 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.659 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:16 3.0 e22/20 3:017cf19c:::benchmark_data_smithi074_35447_object1:head v 22'8, mlcod=0'0)", 2020-08-07T15:06:36.660 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:36.251839+0000", 2020-08-07T15:06:36.660 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.34267514300000002, 2020-08-07T15:06:36.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0069537109999999996, 2020-08-07T15:06:36.660 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:36.660 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:36.661 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:36.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:36.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.251839+0000", 2020-08-07T15:06:36.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:36.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:36.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.251839+0000", 2020-08-07T15:06:36.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5770000000000002e-05 2020-08-07T15:06:36.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:36.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.251905+0000", 2020-08-07T15:06:36.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0365e-05 2020-08-07T15:06:36.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:36.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.251915+0000", 2020-08-07T15:06:36.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8082999999999999e-05 2020-08-07T15:06:36.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:36.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.251973+0000", 2020-08-07T15:06:36.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.831e-06 2020-08-07T15:06:36.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:36.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.251978+0000", 2020-08-07T15:06:36.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0272999999999999e-05 2020-08-07T15:06:36.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:36.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.252018+0000", 2020-08-07T15:06:36.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3557000000000002e-05 2020-08-07T15:06:36.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:36.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.252042+0000", 2020-08-07T15:06:36.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0067162580000000001 2020-08-07T15:06:36.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:36.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.258758+0000", 2020-08-07T15:06:36.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4573999999999998e-05 2020-08-07T15:06:36.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:36.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.258793+0000", 2020-08-07T15:06:36.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0069537109999999996 2020-08-07T15:06:36.670 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.670 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:36.671 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.671 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:25 3.a 3:540180c5:::benchmark_data_smithi074_35447_object3:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e22)", 2020-08-07T15:06:36.671 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:36.262986+0000", 2020-08-07T15:06:36.672 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.33152762600000002, 2020-08-07T15:06:36.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010369672, 2020-08-07T15:06:36.672 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:36.672 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:36.672 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:36.673 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:36.673 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:36.673 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25 2020-08-07T15:06:36.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.673 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:36.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:36.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.262986+0000", 2020-08-07T15:06:36.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:36.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:36.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.262986+0000", 2020-08-07T15:06:36.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9410000000000001e-05 2020-08-07T15:06:36.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:36.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263016+0000", 2020-08-07T15:06:36.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6979999999999997e-06 2020-08-07T15:06:36.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:36.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263021+0000", 2020-08-07T15:06:36.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1200000000000008e-06 2020-08-07T15:06:36.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:36.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263030+0000", 2020-08-07T15:06:36.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9279999999999998e-06 2020-08-07T15:06:36.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:36.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263036+0000", 2020-08-07T15:06:36.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2992999999999999e-05 2020-08-07T15:06:36.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:36.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263089+0000", 2020-08-07T15:06:36.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101099 2020-08-07T15:06:36.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:36.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263190+0000", 2020-08-07T15:06:36.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104953 2020-08-07T15:06:36.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:36.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263295+0000", 2020-08-07T15:06:36.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0098617029999999994 2020-08-07T15:06:36.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:36.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.273156+0000", 2020-08-07T15:06:36.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7770000000000002e-06 2020-08-07T15:06:36.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:36.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.273164+0000", 2020-08-07T15:06:36.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3733000000000006e-05 2020-08-07T15:06:36.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:36.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.273248+0000", 2020-08-07T15:06:36.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0800000000000004e-07 2020-08-07T15:06:36.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:36.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.273249+0000", 2020-08-07T15:06:36.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0865000000000001e-05 2020-08-07T15:06:36.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:36.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.273310+0000", 2020-08-07T15:06:36.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6585000000000002e-05 2020-08-07T15:06:36.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:36.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.273356+0000", 2020-08-07T15:06:36.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010369672 2020-08-07T15:06:36.688 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.689 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:36.689 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.689 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:26 3.a 3:540180c5:::benchmark_data_smithi074_35447_object3:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e22)", 2020-08-07T15:06:36.690 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:36.263066+0000", 2020-08-07T15:06:36.690 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.33144839799999998, 2020-08-07T15:06:36.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018026161999999998, 2020-08-07T15:06:36.690 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:36.691 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:36.691 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:36.691 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:36.691 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:36.691 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 26 2020-08-07T15:06:36.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.692 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:36.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:36.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263066+0000", 2020-08-07T15:06:36.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:36.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:36.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263066+0000", 2020-08-07T15:06:36.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0995999999999998e-05 2020-08-07T15:06:36.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:36.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263107+0000", 2020-08-07T15:06:36.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5729999999999999e-06 2020-08-07T15:06:36.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:36.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263110+0000", 2020-08-07T15:06:36.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2779999999999999e-06 2020-08-07T15:06:36.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:36.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263117+0000", 2020-08-07T15:06:36.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8779999999999998e-06 2020-08-07T15:06:36.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:36.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263119+0000", 2020-08-07T15:06:36.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00055661000000000003 2020-08-07T15:06:36.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:36.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263676+0000", 2020-08-07T15:06:36.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109066 2020-08-07T15:06:36.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:36.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263785+0000", 2020-08-07T15:06:36.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000172972 2020-08-07T15:06:36.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:36.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263958+0000", 2020-08-07T15:06:36.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0092430010000000007 2020-08-07T15:06:36.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:36.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.273201+0000", 2020-08-07T15:06:36.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021172500000000001 2020-08-07T15:06:36.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:36.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.273413+0000", 2020-08-07T15:06:36.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075797939999999999 2020-08-07T15:06:36.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:36.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.280993+0000", 2020-08-07T15:06:36.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1620000000000001e-06 2020-08-07T15:06:36.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:36.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.280994+0000", 2020-08-07T15:06:36.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2166000000000001e-05 2020-08-07T15:06:36.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:36.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281056+0000", 2020-08-07T15:06:36.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5941e-05 2020-08-07T15:06:36.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:36.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281092+0000", 2020-08-07T15:06:36.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018026161999999998 2020-08-07T15:06:36.706 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.706 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:36.706 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.707 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:27 3.a 3:540180c5:::benchmark_data_smithi074_35447_object3:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e22)", 2020-08-07T15:06:36.707 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:36.263146+0000", 2020-08-07T15:06:36.707 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.33136829299999998, 2020-08-07T15:06:36.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018148299, 2020-08-07T15:06:36.708 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:36.708 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:36.708 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:36.708 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:36.709 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:36.709 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 27 2020-08-07T15:06:36.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.709 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:36.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:36.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263146+0000", 2020-08-07T15:06:36.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:36.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:36.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263146+0000", 2020-08-07T15:06:36.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7124999999999999e-05 2020-08-07T15:06:36.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:36.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263173+0000", 2020-08-07T15:06:36.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8449999999999999e-06 2020-08-07T15:06:36.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:36.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263176+0000", 2020-08-07T15:06:36.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5529999999999998e-06 2020-08-07T15:06:36.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:36.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263182+0000", 2020-08-07T15:06:36.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9809999999999999e-06 2020-08-07T15:06:36.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:36.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263185+0000", 2020-08-07T15:06:36.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00233767 2020-08-07T15:06:36.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:36.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.265523+0000", 2020-08-07T15:06:36.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8627000000000004e-05 2020-08-07T15:06:36.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:36.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.265612+0000", 2020-08-07T15:06:36.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000119163 2020-08-07T15:06:36.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:36.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.265731+0000", 2020-08-07T15:06:36.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0074918650000000003 2020-08-07T15:06:36.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:36.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.273223+0000", 2020-08-07T15:06:36.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0079237089999999993 2020-08-07T15:06:36.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:36.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281146+0000", 2020-08-07T15:06:36.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2766999999999998e-05 2020-08-07T15:06:36.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:36.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281199+0000", 2020-08-07T15:06:36.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3900000000000005e-07 2020-08-07T15:06:36.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:36.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281200+0000", 2020-08-07T15:06:36.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0120999999999997e-05 2020-08-07T15:06:36.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:36.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281250+0000", 2020-08-07T15:06:36.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4333999999999999e-05 2020-08-07T15:06:36.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:36.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281294+0000", 2020-08-07T15:06:36.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018148299 2020-08-07T15:06:36.723 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.723 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:36.723 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.724 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:28 3.a 3:540180c5:::benchmark_data_smithi074_35447_object3:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e22)", 2020-08-07T15:06:36.724 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:36.263204+0000", 2020-08-07T15:06:36.724 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.33131032799999999, 2020-08-07T15:06:36.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018335269000000001, 2020-08-07T15:06:36.725 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:36.725 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:36.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:36.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:36.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:36.726 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 28 2020-08-07T15:06:36.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.726 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:36.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:36.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263204+0000", 2020-08-07T15:06:36.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:36.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:36.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263204+0000", 2020-08-07T15:06:36.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7064999999999999e-05 2020-08-07T15:06:36.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:36.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263231+0000", 2020-08-07T15:06:36.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8329999999999998e-06 2020-08-07T15:06:36.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:36.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263234+0000", 2020-08-07T15:06:36.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0690000000000001e-06 2020-08-07T15:06:36.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:36.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263239+0000", 2020-08-07T15:06:36.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.17e-06 2020-08-07T15:06:36.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:36.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263241+0000", 2020-08-07T15:06:36.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006265329 2020-08-07T15:06:36.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:36.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.269506+0000", 2020-08-07T15:06:36.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9980000000000002e-05 2020-08-07T15:06:36.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:36.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.269606+0000", 2020-08-07T15:06:36.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017308699999999999 2020-08-07T15:06:36.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:36.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.269779+0000", 2020-08-07T15:06:36.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011409163 2020-08-07T15:06:36.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:36.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281188+0000", 2020-08-07T15:06:36.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000119527 2020-08-07T15:06:36.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:36.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281308+0000", 2020-08-07T15:06:36.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000115097 2020-08-07T15:06:36.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:36.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281423+0000", 2020-08-07T15:06:36.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1200000000000002e-07 2020-08-07T15:06:36.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:36.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281424+0000", 2020-08-07T15:06:36.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7111000000000002e-05 2020-08-07T15:06:36.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:36.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281491+0000", 2020-08-07T15:06:36.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8126e-05 2020-08-07T15:06:36.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:36.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281539+0000", 2020-08-07T15:06:36.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018335269000000001 2020-08-07T15:06:36.751 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.752 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:36.752 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.752 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:29 3.a 3:540180c5:::benchmark_data_smithi074_35447_object3:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e22)", 2020-08-07T15:06:36.753 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:36.263258+0000", 2020-08-07T15:06:36.753 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.33125633300000001, 2020-08-07T15:06:36.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019645268, 2020-08-07T15:06:36.753 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:36.753 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:36.754 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:36.754 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:36.754 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:36.754 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 29 2020-08-07T15:06:36.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.755 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:36.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:36.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263258+0000", 2020-08-07T15:06:36.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:36.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:36.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263258+0000", 2020-08-07T15:06:36.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6665999999999998e-05 2020-08-07T15:06:36.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:36.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263284+0000", 2020-08-07T15:06:36.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.514e-06 2020-08-07T15:06:36.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:36.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263287+0000", 2020-08-07T15:06:36.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5809999999999996e-06 2020-08-07T15:06:36.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:36.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263293+0000", 2020-08-07T15:06:36.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7389999999999999e-06 2020-08-07T15:06:36.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:36.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263295+0000", 2020-08-07T15:06:36.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010160035 2020-08-07T15:06:36.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:36.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.273455+0000", 2020-08-07T15:06:36.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010453700000000001 2020-08-07T15:06:36.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:36.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.273560+0000", 2020-08-07T15:06:36.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019277899999999999 2020-08-07T15:06:36.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:36.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.273753+0000", 2020-08-07T15:06:36.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075956119999999998 2020-08-07T15:06:36.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:36.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281348+0000", 2020-08-07T15:06:36.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001465168 2020-08-07T15:06:36.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:36.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.282813+0000", 2020-08-07T15:06:36.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1749999999999999e-06 2020-08-07T15:06:36.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:36.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.282821+0000", 2020-08-07T15:06:36.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6899999999999998e-07 2020-08-07T15:06:36.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:36.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.282821+0000", 2020-08-07T15:06:36.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8722e-05 2020-08-07T15:06:36.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:36.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.282860+0000", 2020-08-07T15:06:36.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3371000000000002e-05 2020-08-07T15:06:36.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:36.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.282903+0000", 2020-08-07T15:06:36.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019645268 2020-08-07T15:06:36.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.769 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:36.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.770 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:30 3.a 3:540180c5:::benchmark_data_smithi074_35447_object3:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e22)", 2020-08-07T15:06:36.770 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:36.263312+0000", 2020-08-07T15:06:36.770 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.33120207699999998, 2020-08-07T15:06:36.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.025534471999999999, 2020-08-07T15:06:36.770 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:36.771 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:36.771 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:36.771 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:36.771 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:36.771 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30 2020-08-07T15:06:36.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.772 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:36.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:36.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263312+0000", 2020-08-07T15:06:36.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:36.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:36.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263312+0000", 2020-08-07T15:06:36.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7704000000000001e-05 2020-08-07T15:06:36.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:36.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263340+0000", 2020-08-07T15:06:36.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.525e-06 2020-08-07T15:06:36.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:36.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263342+0000", 2020-08-07T15:06:36.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2329999999999997e-06 2020-08-07T15:06:36.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:36.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263347+0000", 2020-08-07T15:06:36.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1289999999999999e-06 2020-08-07T15:06:36.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:36.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263350+0000", 2020-08-07T15:06:36.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013770683000000001 2020-08-07T15:06:36.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:36.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.277120+0000", 2020-08-07T15:06:36.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8572999999999995e-05 2020-08-07T15:06:36.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:36.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.277199+0000", 2020-08-07T15:06:36.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016149299999999999 2020-08-07T15:06:36.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:36.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.277360+0000", 2020-08-07T15:06:36.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0077457510000000004 2020-08-07T15:06:36.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:36.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.285106+0000", 2020-08-07T15:06:36.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9800000000000003e-07 2020-08-07T15:06:36.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:36.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.285107+0000", 2020-08-07T15:06:36.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4962e-05 2020-08-07T15:06:36.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:36.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.285162+0000", 2020-08-07T15:06:36.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00084000800000000005 2020-08-07T15:06:36.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:36.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.286002+0000", 2020-08-07T15:06:36.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027792630000000001 2020-08-07T15:06:36.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:36.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.288781+0000", 2020-08-07T15:06:36.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2799999999999995e-07 2020-08-07T15:06:36.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:36.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.288782+0000", 2020-08-07T15:06:36.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4622000000000006e-05 2020-08-07T15:06:36.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:36.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.288846+0000", 2020-08-07T15:06:36.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.025534471999999999 2020-08-07T15:06:36.801 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.802 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:36.802 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.802 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:31 3.a 3:540180c5:::benchmark_data_smithi074_35447_object3:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e22)", 2020-08-07T15:06:36.803 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:36.263368+0000", 2020-08-07T15:06:36.803 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.33114642700000002, 2020-08-07T15:06:36.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.028083049999999998, 2020-08-07T15:06:36.803 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:36.803 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:36.804 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:36.804 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:36.804 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:36.804 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31 2020-08-07T15:06:36.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.805 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:36.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:36.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263368+0000", 2020-08-07T15:06:36.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:36.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:36.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263368+0000", 2020-08-07T15:06:36.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0440999999999999e-05 2020-08-07T15:06:36.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:36.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263398+0000", 2020-08-07T15:06:36.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9280000000000002e-06 2020-08-07T15:06:36.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:36.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263401+0000", 2020-08-07T15:06:36.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4750000000000001e-06 2020-08-07T15:06:36.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:36.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263407+0000", 2020-08-07T15:06:36.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0369999999999999e-06 2020-08-07T15:06:36.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:36.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263410+0000", 2020-08-07T15:06:36.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018160326000000001 2020-08-07T15:06:36.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:36.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281570+0000", 2020-08-07T15:06:36.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9647999999999995e-05 2020-08-07T15:06:36.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:36.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281640+0000", 2020-08-07T15:06:36.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000170545 2020-08-07T15:06:36.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:36.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281810+0000", 2020-08-07T15:06:36.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0087142439999999995 2020-08-07T15:06:36.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:36.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.290524+0000", 2020-08-07T15:06:36.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.066e-06 2020-08-07T15:06:36.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:36.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.290525+0000", 2020-08-07T15:06:36.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2020-08-07T15:06:36.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:36.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.290589+0000", 2020-08-07T15:06:36.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00070321399999999997 2020-08-07T15:06:36.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:36.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.291293+0000", 2020-08-07T15:06:36.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3114000000000004e-05 2020-08-07T15:06:36.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:36.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.291376+0000", 2020-08-07T15:06:36.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9500000000000002e-07 2020-08-07T15:06:36.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:36.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.291376+0000", 2020-08-07T15:06:36.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4363999999999995e-05 2020-08-07T15:06:36.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:36.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.291451+0000", 2020-08-07T15:06:36.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.028083049999999998 2020-08-07T15:06:36.820 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.820 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:36.820 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.821 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:32 3.a 3:540180c5:::benchmark_data_smithi074_35447_object3:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e22)", 2020-08-07T15:06:36.821 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:36.263428+0000", 2020-08-07T15:06:36.821 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.331086516, 2020-08-07T15:06:36.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031358047, 2020-08-07T15:06:36.821 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:36.822 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:36.822 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:36.822 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:36.822 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:36.822 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 32 2020-08-07T15:06:36.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.823 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:36.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:36.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263428+0000", 2020-08-07T15:06:36.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:36.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:36.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263428+0000", 2020-08-07T15:06:36.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6202000000000001e-05 2020-08-07T15:06:36.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:36.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263454+0000", 2020-08-07T15:06:36.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.447e-06 2020-08-07T15:06:36.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:36.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263456+0000", 2020-08-07T15:06:36.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0730000000000001e-06 2020-08-07T15:06:36.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:36.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263462+0000", 2020-08-07T15:06:36.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.12e-06 2020-08-07T15:06:36.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:36.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.263464+0000", 2020-08-07T15:06:36.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01874756 2020-08-07T15:06:36.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:36.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.282212+0000", 2020-08-07T15:06:36.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8226999999999994e-05 2020-08-07T15:06:36.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:36.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.282300+0000", 2020-08-07T15:06:36.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016032899999999999 2020-08-07T15:06:36.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:36.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.282461+0000", 2020-08-07T15:06:36.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081363540000000002 2020-08-07T15:06:36.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:36.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.290597+0000", 2020-08-07T15:06:36.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6599999999999996e-07 2020-08-07T15:06:36.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:36.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.290598+0000", 2020-08-07T15:06:36.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7995999999999998e-05 2020-08-07T15:06:36.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:36.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.290656+0000", 2020-08-07T15:06:36.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068378099999999995 2020-08-07T15:06:36.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:36.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.291339+0000", 2020-08-07T15:06:36.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033677720000000002 2020-08-07T15:06:36.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:36.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.294707+0000", 2020-08-07T15:06:36.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4e-07 2020-08-07T15:06:36.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:36.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.294708+0000", 2020-08-07T15:06:36.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7579999999999999e-05 2020-08-07T15:06:36.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:36.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.294786+0000", 2020-08-07T15:06:36.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031358047 2020-08-07T15:06:36.838 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.839 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:36.840 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.841 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:27 3.a e22/20 ondisk, result = 0)", 2020-08-07T15:06:36.841 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:36.273620+0000", 2020-08-07T15:06:36.841 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.32089404999999999, 2020-08-07T15:06:36.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075303740000000003, 2020-08-07T15:06:36.852 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:36.852 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:36.852 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:36.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:36.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.273620+0000", 2020-08-07T15:06:36.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:36.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:36.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.273620+0000", 2020-08-07T15:06:36.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8734000000000001e-05 2020-08-07T15:06:36.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:36.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.273639+0000", 2020-08-07T15:06:36.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8469999999999998e-06 2020-08-07T15:06:36.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:36.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.273643+0000", 2020-08-07T15:06:36.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0045e-05 2020-08-07T15:06:36.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:36.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.273683+0000", 2020-08-07T15:06:36.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3099999999999997e-06 2020-08-07T15:06:36.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:36.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.273689+0000", 2020-08-07T15:06:36.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0074393879999999999 2020-08-07T15:06:36.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:36.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281128+0000", 2020-08-07T15:06:36.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3599999999999996e-06 2020-08-07T15:06:36.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:36.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281137+0000", 2020-08-07T15:06:36.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3689999999999999e-05 2020-08-07T15:06:36.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:36.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281150+0000", 2020-08-07T15:06:36.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075303740000000003 2020-08-07T15:06:36.876 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.876 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:36.876 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.877 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:28 3.a e22/20 ondisk, result = 0)", 2020-08-07T15:06:36.877 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:36.274400+0000", 2020-08-07T15:06:36.877 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.32011377200000002, 2020-08-07T15:06:36.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0069084819999999996, 2020-08-07T15:06:36.877 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:36.888 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:36.889 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:36.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:36.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.274400+0000", 2020-08-07T15:06:36.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:36.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:36.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.274400+0000", 2020-08-07T15:06:36.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7244999999999998e-05 2020-08-07T15:06:36.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:36.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.274418+0000", 2020-08-07T15:06:36.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5959999999999997e-06 2020-08-07T15:06:36.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:36.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.274422+0000", 2020-08-07T15:06:36.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0179e-05 2020-08-07T15:06:36.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:36.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.274452+0000", 2020-08-07T15:06:36.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.693e-06 2020-08-07T15:06:36.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:36.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.274456+0000", 2020-08-07T15:06:36.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0067145429999999999 2020-08-07T15:06:36.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:36.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281171+0000", 2020-08-07T15:06:36.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7170000000000003e-06 2020-08-07T15:06:36.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:36.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281180+0000", 2020-08-07T15:06:36.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012850900000000001 2020-08-07T15:06:36.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:36.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281309+0000", 2020-08-07T15:06:36.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0069084819999999996 2020-08-07T15:06:36.918 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.918 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:36.919 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.920 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:42 3.0 e22/20 3:0c1b7d92:::benchmark_data_smithi074_35447_object5:head v 22'10, mlcod=22'7)", 2020-08-07T15:06:36.921 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:36.277042+0000", 2020-08-07T15:06:36.921 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.317472425, 2020-08-07T15:06:36.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0080480299999999994, 2020-08-07T15:06:36.922 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:36.922 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:36.923 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:36.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:36.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.277042+0000", 2020-08-07T15:06:36.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:36.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:36.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.277042+0000", 2020-08-07T15:06:36.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2108000000000003e-05 2020-08-07T15:06:36.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:36.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.277094+0000", 2020-08-07T15:06:36.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3389999999999998e-06 2020-08-07T15:06:36.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:36.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.277100+0000", 2020-08-07T15:06:36.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3661000000000001e-05 2020-08-07T15:06:36.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:36.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.277144+0000", 2020-08-07T15:06:36.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5969999999999997e-06 2020-08-07T15:06:36.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:36.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.277150+0000", 2020-08-07T15:06:36.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1427999999999997e-05 2020-08-07T15:06:36.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:36.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.277192+0000", 2020-08-07T15:06:36.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3181e-05 2020-08-07T15:06:36.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:36.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.277215+0000", 2020-08-07T15:06:36.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0078176800000000005 2020-08-07T15:06:36.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:36.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.285033+0000", 2020-08-07T15:06:36.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7036000000000001e-05 2020-08-07T15:06:36.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:36.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.285090+0000", 2020-08-07T15:06:36.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0080480299999999994 2020-08-07T15:06:36.955 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.955 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:36.955 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.956 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:43 3.0 e22/20 3:0c1b7d92:::benchmark_data_smithi074_35447_object5:head v 22'11, mlcod=22'7)", 2020-08-07T15:06:36.956 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:36.277671+0000", 2020-08-07T15:06:36.956 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.31684319799999999, 2020-08-07T15:06:36.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012822765, 2020-08-07T15:06:36.957 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:36.957 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:36.957 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:36.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:36.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.277671+0000", 2020-08-07T15:06:36.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:36.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:36.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.277671+0000", 2020-08-07T15:06:36.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4128999999999998e-05 2020-08-07T15:06:36.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:36.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.277705+0000", 2020-08-07T15:06:36.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4490000000000001e-06 2020-08-07T15:06:36.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:36.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.277709+0000", 2020-08-07T15:06:36.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1399e-05 2020-08-07T15:06:36.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:36.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.277761+0000", 2020-08-07T15:06:36.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9189999999999998e-06 2020-08-07T15:06:36.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:36.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.277766+0000", 2020-08-07T15:06:36.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032231999999999998 2020-08-07T15:06:36.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:36.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.280989+0000", 2020-08-07T15:06:36.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8219999999999998e-05 2020-08-07T15:06:36.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:36.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281007+0000", 2020-08-07T15:06:36.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0094308350000000003 2020-08-07T15:06:36.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:36.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.290438+0000", 2020-08-07T15:06:36.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5614e-05 2020-08-07T15:06:36.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:36.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.290494+0000", 2020-08-07T15:06:36.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012822765 2020-08-07T15:06:36.970 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.970 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:36.970 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.971 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:44 3.0 e22/20 3:0c1b7d92:::benchmark_data_smithi074_35447_object5:head v 22'12, mlcod=22'7)", 2020-08-07T15:06:36.971 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:36.280686+0000", 2020-08-07T15:06:36.972 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.31382776600000001, 2020-08-07T15:06:36.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0098487399999999999, 2020-08-07T15:06:36.972 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:36.973 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:36.973 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:36.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:36.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.280686+0000", 2020-08-07T15:06:36.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:36.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:36.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.280686+0000", 2020-08-07T15:06:36.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2017999999999997e-05 2020-08-07T15:06:36.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:36.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.280718+0000", 2020-08-07T15:06:36.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2760000000000002e-06 2020-08-07T15:06:36.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:36.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.280723+0000", 2020-08-07T15:06:36.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8162999999999999e-05 2020-08-07T15:06:36.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:36.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.280771+0000", 2020-08-07T15:06:36.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.721e-06 2020-08-07T15:06:36.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:36.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.280775+0000", 2020-08-07T15:06:36.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049085399999999999 2020-08-07T15:06:36.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:36.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281265+0000", 2020-08-07T15:06:36.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1773e-05 2020-08-07T15:06:36.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:36.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281287+0000", 2020-08-07T15:06:36.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0092089319999999995 2020-08-07T15:06:36.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:36.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.290496+0000", 2020-08-07T15:06:36.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9002999999999999e-05 2020-08-07T15:06:36.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:36.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.290535+0000", 2020-08-07T15:06:36.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0098487399999999999 2020-08-07T15:06:36.986 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.986 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:36.986 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:36.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.987 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:45 3.0 e22/20 3:0c1b7d92:::benchmark_data_smithi074_35447_object5:head v 22'13, mlcod=22'7)", 2020-08-07T15:06:36.987 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:36.281722+0000", 2020-08-07T15:06:36.987 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.31279245100000003, 2020-08-07T15:06:36.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0088499919999999992, 2020-08-07T15:06:36.988 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:36.988 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:36.988 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:36.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:36.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281722+0000", 2020-08-07T15:06:36.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:36.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:36.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281722+0000", 2020-08-07T15:06:36.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8886000000000001e-05 2020-08-07T15:06:36.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:36.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281751+0000", 2020-08-07T15:06:36.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7699999999999999e-06 2020-08-07T15:06:36.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:36.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281754+0000", 2020-08-07T15:06:36.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5852000000000002e-05 2020-08-07T15:06:36.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:36.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281790+0000", 2020-08-07T15:06:36.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1480000000000002e-06 2020-08-07T15:06:36.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:36.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:36.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:36.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281793+0000", 2020-08-07T15:06:37.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8300000000000003e-05 2020-08-07T15:06:37.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:37.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281832+0000", 2020-08-07T15:06:37.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4189000000000002e-05 2020-08-07T15:06:37.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:37.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.281856+0000", 2020-08-07T15:06:37.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086809230000000001 2020-08-07T15:06:37.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:37.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.290537+0000", 2020-08-07T15:06:37.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4924e-05 2020-08-07T15:06:37.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:37.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.290572+0000", 2020-08-07T15:06:37.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0088499919999999992 2020-08-07T15:06:37.008 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.008 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:37.008 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.009 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:46 3.0 e22/20 3:0c1b7d92:::benchmark_data_smithi074_35447_object5:head v 22'14, mlcod=22'7)", 2020-08-07T15:06:37.009 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:36.282348+0000", 2020-08-07T15:06:37.009 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.312166002, 2020-08-07T15:06:37.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0082606620000000002, 2020-08-07T15:06:37.010 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:37.010 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:37.010 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:37.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:37.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.282348+0000", 2020-08-07T15:06:37.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:37.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:37.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.282348+0000", 2020-08-07T15:06:37.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1388000000000003e-05 2020-08-07T15:06:37.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:37.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.282399+0000", 2020-08-07T15:06:37.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5359999999999998e-06 2020-08-07T15:06:37.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:37.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.282406+0000", 2020-08-07T15:06:37.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6699999999999995e-05 2020-08-07T15:06:37.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:37.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.282473+0000", 2020-08-07T15:06:37.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3520000000000003e-06 2020-08-07T15:06:37.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:37.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.282479+0000", 2020-08-07T15:06:37.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9227000000000002e-05 2020-08-07T15:06:37.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:37.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.282518+0000", 2020-08-07T15:06:37.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2081000000000001e-05 2020-08-07T15:06:37.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:37.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.282540+0000", 2020-08-07T15:06:37.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0080334120000000002 2020-08-07T15:06:37.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:37.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.290574+0000", 2020-08-07T15:06:37.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4965999999999997e-05 2020-08-07T15:06:37.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:37.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.290609+0000", 2020-08-07T15:06:37.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0082606620000000002 2020-08-07T15:06:37.020 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.020 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:37.020 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.021 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:47 3.0 e22/20 3:0c1b7d92:::benchmark_data_smithi074_35447_object5:head v 22'15, mlcod=22'8)", 2020-08-07T15:06:37.021 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:36.282925+0000", 2020-08-07T15:06:37.021 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.31158887000000002, 2020-08-07T15:06:37.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0077205049999999999, 2020-08-07T15:06:37.021 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:37.021 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:37.022 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:37.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:37.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.282925+0000", 2020-08-07T15:06:37.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:37.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:37.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.282925+0000", 2020-08-07T15:06:37.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1056999999999995e-05 2020-08-07T15:06:37.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:37.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.282986+0000", 2020-08-07T15:06:37.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3999999999999997e-06 2020-08-07T15:06:37.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:37.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.282993+0000", 2020-08-07T15:06:37.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0779999999999997e-05 2020-08-07T15:06:37.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:37.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.283053+0000", 2020-08-07T15:06:37.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9550000000000003e-06 2020-08-07T15:06:37.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:37.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.283060+0000", 2020-08-07T15:06:37.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7286999999999997e-05 2020-08-07T15:06:37.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:37.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.283098+0000", 2020-08-07T15:06:37.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2162e-05 2020-08-07T15:06:37.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:37.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.283110+0000", 2020-08-07T15:06:37.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075016390000000001 2020-08-07T15:06:37.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:37.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.290612+0000", 2020-08-07T15:06:37.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4224999999999998e-05 2020-08-07T15:06:37.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:37.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.290646+0000", 2020-08-07T15:06:37.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0077205049999999999 2020-08-07T15:06:37.031 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.031 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:37.031 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.032 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.032 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:37.032 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:06:37.034 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:06:37.034 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:37.032+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:37.134 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:37.133+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:37.173 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:06:37.238 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:06:37.237+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:06:37.240 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 1 16 1501 1485 11.6009 11.6016 0.0189244 0.0107096 2020-08-07T15:06:37.251 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:06:37.251 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:06:37.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.252 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:1512 3.d 3.f9b1ff7d (undecoded) ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:37.252 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.244263+0000", 2020-08-07T15:06:37.253 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0057139629999999999, 2020-08-07T15:06:37.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057304080000000002, 2020-08-07T15:06:37.258 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:37.259 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:06:37.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:37.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:37.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:37.259 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1512 2020-08-07T15:06:37.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.260 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:37.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:37.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.244263+0000", 2020-08-07T15:06:37.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:37.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:37.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.244263+0000", 2020-08-07T15:06:37.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0096000000000003e-05 2020-08-07T15:06:37.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:37.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.244313+0000", 2020-08-07T15:06:37.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9710000000000001e-06 2020-08-07T15:06:37.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:37.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.244319+0000", 2020-08-07T15:06:37.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0917999999999999e-05 2020-08-07T15:06:37.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:37.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.244330+0000", 2020-08-07T15:06:37.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.702e-06 2020-08-07T15:06:37.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:37.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.244334+0000", 2020-08-07T15:06:37.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1687000000000007e-05 2020-08-07T15:06:37.282 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.282 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:37.282 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.283 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:1505 3.d e23/20)", 2020-08-07T15:06:37.284 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.245486+0000", 2020-08-07T15:06:37.284 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0044907819999999996, 2020-08-07T15:06:37.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004543577, 2020-08-07T15:06:37.284 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:37.284 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:06:37.285 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:37.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:37.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.245486+0000", 2020-08-07T15:06:37.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:37.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:37.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.245486+0000", 2020-08-07T15:06:37.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0673000000000003e-05 2020-08-07T15:06:37.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:37.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.245517+0000", 2020-08-07T15:06:37.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.835e-06 2020-08-07T15:06:37.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:37.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.245522+0000", 2020-08-07T15:06:37.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4830999999999997e-05 2020-08-07T15:06:37.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:37.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.245577+0000", 2020-08-07T15:06:37.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5299999999999999e-06 2020-08-07T15:06:37.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:37.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.245585+0000", 2020-08-07T15:06:37.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8869000000000005e-05 2020-08-07T15:06:37.293 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.293 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:37.293 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.294 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:1505 3.d e23/20)", 2020-08-07T15:06:37.294 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.246191+0000", 2020-08-07T15:06:37.294 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0037861940000000001, 2020-08-07T15:06:37.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038654679999999999, 2020-08-07T15:06:37.295 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:37.295 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:06:37.295 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:37.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:37.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.246191+0000", 2020-08-07T15:06:37.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:37.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:37.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.246191+0000", 2020-08-07T15:06:37.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2319e-05 2020-08-07T15:06:37.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:37.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.246223+0000", 2020-08-07T15:06:37.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7869999999999998e-06 2020-08-07T15:06:37.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:37.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.246229+0000", 2020-08-07T15:06:37.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9640000000000001e-05 2020-08-07T15:06:37.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:37.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.246258+0000", 2020-08-07T15:06:37.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2020-08-07T15:06:37.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:37.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.246264+0000", 2020-08-07T15:06:37.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3348000000000003e-05 2020-08-07T15:06:37.303 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.303 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:37.303 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.304 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:1448 3.f 3:f4bbdfc5:::benchmark_data_smithi074_35447_object180:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:37.304 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.209656+0000", 2020-08-07T15:06:37.305 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.040320882000000002, 2020-08-07T15:06:37.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040440070000000002, 2020-08-07T15:06:37.306 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:37.307 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:37.307 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:37.307 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:37.307 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:37.308 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1448 2020-08-07T15:06:37.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.308 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:37.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:37.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.209656+0000", 2020-08-07T15:06:37.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:37.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:37.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.209656+0000", 2020-08-07T15:06:37.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0454999999999997e-05 2020-08-07T15:06:37.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:37.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.209706+0000", 2020-08-07T15:06:37.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.178e-06 2020-08-07T15:06:37.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:37.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.209713+0000", 2020-08-07T15:06:37.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1361e-05 2020-08-07T15:06:37.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:37.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.209724+0000", 2020-08-07T15:06:37.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7010000000000001e-06 2020-08-07T15:06:37.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:37.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.209730+0000", 2020-08-07T15:06:37.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013466500000000001 2020-08-07T15:06:37.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:37.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.209864+0000", 2020-08-07T15:06:37.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6986999999999998e-05 2020-08-07T15:06:37.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:37.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.209961+0000", 2020-08-07T15:06:37.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018270600000000001 2020-08-07T15:06:37.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:37.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.210144+0000", 2020-08-07T15:06:37.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035264739999999999 2020-08-07T15:06:37.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:37.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.213670+0000", 2020-08-07T15:06:37.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0869999999999999e-06 2020-08-07T15:06:37.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:37.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.213671+0000", 2020-08-07T15:06:37.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6928999999999996e-05 2020-08-07T15:06:37.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:37.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.213738+0000", 2020-08-07T15:06:37.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040825430000000001 2020-08-07T15:06:37.322 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.323 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:37.323 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.324 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:1505 3.d 3:beff8d9f:::benchmark_data_smithi074_35447_object188:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:37.324 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.241818+0000", 2020-08-07T15:06:37.324 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0081585770000000002, 2020-08-07T15:06:37.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083636330000000005, 2020-08-07T15:06:37.324 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:37.325 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:37.325 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:37.325 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:37.326 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:37.326 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1505 2020-08-07T15:06:37.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.326 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:37.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:37.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.241818+0000", 2020-08-07T15:06:37.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:37.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:37.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.241818+0000", 2020-08-07T15:06:37.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0833000000000001e-05 2020-08-07T15:06:37.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:37.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.241849+0000", 2020-08-07T15:06:37.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4750000000000004e-06 2020-08-07T15:06:37.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:37.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.241853+0000", 2020-08-07T15:06:37.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4100000000000008e-06 2020-08-07T15:06:37.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:37.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.241862+0000", 2020-08-07T15:06:37.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3259999999999997e-06 2020-08-07T15:06:37.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:37.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.241866+0000", 2020-08-07T15:06:37.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7859999999999999e-05 2020-08-07T15:06:37.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:37.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.241914+0000", 2020-08-07T15:06:37.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017463999999999999 2020-08-07T15:06:37.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:37.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.242089+0000", 2020-08-07T15:06:37.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000179173 2020-08-07T15:06:37.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:06:37.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.242268+0000", 2020-08-07T15:06:37.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005625555 2020-08-07T15:06:37.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:37.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.247893+0000", 2020-08-07T15:06:37.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1209999999999999e-06 2020-08-07T15:06:37.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:37.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.247895+0000", 2020-08-07T15:06:37.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8819000000000006e-05 2020-08-07T15:06:37.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:37.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.247963+0000", 2020-08-07T15:06:37.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061452119999999997 2020-08-07T15:06:37.339 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.339 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:37.339 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.340 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:1506 3.d 3:beff8d9f:::benchmark_data_smithi074_35447_object188:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:37.340 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.241892+0000", 2020-08-07T15:06:37.340 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0080848689999999997, 2020-08-07T15:06:37.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083448759999999993, 2020-08-07T15:06:37.341 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:37.341 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:06:37.341 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:37.341 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:37.342 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:37.351 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1506 2020-08-07T15:06:37.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.352 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:37.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:37.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.241892+0000", 2020-08-07T15:06:37.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:37.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:37.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.241892+0000", 2020-08-07T15:06:37.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3324999999999997e-05 2020-08-07T15:06:37.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:37.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.241925+0000", 2020-08-07T15:06:37.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6299999999999997e-06 2020-08-07T15:06:37.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:37.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.241930+0000", 2020-08-07T15:06:37.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0280000000000006e-06 2020-08-07T15:06:37.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:37.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.241939+0000", 2020-08-07T15:06:37.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0799999999999999e-06 2020-08-07T15:06:37.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:37.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.241943+0000", 2020-08-07T15:06:37.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011074360000000001 2020-08-07T15:06:37.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:37.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.243050+0000", 2020-08-07T15:06:37.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016356000000000001 2020-08-07T15:06:37.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:37.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.243214+0000", 2020-08-07T15:06:37.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00035524900000000003 2020-08-07T15:06:37.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:06:37.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.243569+0000", 2020-08-07T15:06:37.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001677308 2020-08-07T15:06:37.387 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.390 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:37.391 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.392 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:1507 3.d 3:beff8d9f:::benchmark_data_smithi074_35447_object188:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:37.394 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.241972+0000", 2020-08-07T15:06:37.395 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0080048229999999995, 2020-08-07T15:06:37.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083276050000000001, 2020-08-07T15:06:37.396 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:37.396 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:06:37.396 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:37.396 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:37.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:37.397 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1507 2020-08-07T15:06:37.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.397 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:37.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:37.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.241972+0000", 2020-08-07T15:06:37.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:37.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:37.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.241972+0000", 2020-08-07T15:06:37.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1974999999999997e-05 2020-08-07T15:06:37.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:37.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.242014+0000", 2020-08-07T15:06:37.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.163e-06 2020-08-07T15:06:37.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:37.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.242017+0000", 2020-08-07T15:06:37.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7440000000000004e-06 2020-08-07T15:06:37.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:37.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.242023+0000", 2020-08-07T15:06:37.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7939999999999998e-06 2020-08-07T15:06:37.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:37.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.242026+0000", 2020-08-07T15:06:37.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020548279999999999 2020-08-07T15:06:37.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:37.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.244080+0000", 2020-08-07T15:06:37.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000119479 2020-08-07T15:06:37.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:37.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.244200+0000", 2020-08-07T15:06:37.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018259299999999999 2020-08-07T15:06:37.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:06:37.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.244383+0000", 2020-08-07T15:06:37.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024105760000000002 2020-08-07T15:06:37.407 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.407 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:37.407 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.408 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:1508 3.d 3:beff8d9f:::benchmark_data_smithi074_35447_object188:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:37.408 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.242044+0000", 2020-08-07T15:06:37.408 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.007932438, 2020-08-07T15:06:37.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083159610000000002, 2020-08-07T15:06:37.408 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:37.409 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:06:37.409 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:37.409 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:37.409 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:37.410 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1508 2020-08-07T15:06:37.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.411 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:37.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:37.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.242044+0000", 2020-08-07T15:06:37.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:37.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:37.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.242044+0000", 2020-08-07T15:06:37.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.459e-05 2020-08-07T15:06:37.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:37.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.242069+0000", 2020-08-07T15:06:37.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4490000000000002e-06 2020-08-07T15:06:37.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:37.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.242076+0000", 2020-08-07T15:06:37.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6529999999999999e-06 2020-08-07T15:06:37.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:37.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.242082+0000", 2020-08-07T15:06:37.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2790000000000001e-06 2020-08-07T15:06:37.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:37.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.242084+0000", 2020-08-07T15:06:37.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036804469999999999 2020-08-07T15:06:37.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:37.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.245765+0000", 2020-08-07T15:06:37.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7021999999999994e-05 2020-08-07T15:06:37.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:37.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.245862+0000", 2020-08-07T15:06:37.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017482399999999999 2020-08-07T15:06:37.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:06:37.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.246037+0000", 2020-08-07T15:06:37.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039922639999999997 2020-08-07T15:06:37.420 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.421 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:37.421 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.421 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:1509 3.d 3:beff8d9f:::benchmark_data_smithi074_35447_object188:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:37.422 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.243322+0000", 2020-08-07T15:06:37.423 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0066549519999999996, 2020-08-07T15:06:37.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070846850000000003, 2020-08-07T15:06:37.423 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:37.423 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:06:37.423 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:37.424 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:37.424 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:37.424 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1509 2020-08-07T15:06:37.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.426 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:37.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:37.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.243322+0000", 2020-08-07T15:06:37.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:37.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:37.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.243322+0000", 2020-08-07T15:06:37.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9194999999999999e-05 2020-08-07T15:06:37.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:37.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.243371+0000", 2020-08-07T15:06:37.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.392e-06 2020-08-07T15:06:37.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:37.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.243377+0000", 2020-08-07T15:06:37.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2223e-05 2020-08-07T15:06:37.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:37.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.243390+0000", 2020-08-07T15:06:37.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9290000000000003e-06 2020-08-07T15:06:37.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:37.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.243396+0000", 2020-08-07T15:06:37.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045965099999999998 2020-08-07T15:06:37.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:37.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.247992+0000", 2020-08-07T15:06:37.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012170199999999999 2020-08-07T15:06:37.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:37.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.248114+0000", 2020-08-07T15:06:37.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016939 2020-08-07T15:06:37.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:06:37.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.248283+0000", 2020-08-07T15:06:37.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049613410000000002 2020-08-07T15:06:37.435 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.436 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:37.436 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.437 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:1510 3.d 3.f9b1ff7d (undecoded) ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:37.437 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.244041+0000", 2020-08-07T15:06:37.437 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0059353410000000002, 2020-08-07T15:06:37.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064152089999999998, 2020-08-07T15:06:37.438 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:37.438 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:06:37.438 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:37.438 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:37.438 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:37.439 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1510 2020-08-07T15:06:37.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.439 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:37.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:37.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.244041+0000", 2020-08-07T15:06:37.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:37.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:37.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.244041+0000", 2020-08-07T15:06:37.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8053000000000001e-05 2020-08-07T15:06:37.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:37.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.244099+0000", 2020-08-07T15:06:37.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2809999999999997e-06 2020-08-07T15:06:37.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:37.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.244106+0000", 2020-08-07T15:06:37.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2098e-05 2020-08-07T15:06:37.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:37.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.244118+0000", 2020-08-07T15:06:37.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7869999999999998e-06 2020-08-07T15:06:37.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:37.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.244124+0000", 2020-08-07T15:06:37.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2218999999999993e-05 2020-08-07T15:06:37.447 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.447 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:37.447 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.448 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:1511 3.d 3.f9b1ff7d (undecoded) ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:37.448 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.244155+0000", 2020-08-07T15:06:37.448 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0058215510000000003, 2020-08-07T15:06:37.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006367958, 2020-08-07T15:06:37.449 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:37.449 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:06:37.449 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:37.449 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:37.449 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:37.450 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1511 2020-08-07T15:06:37.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.450 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:37.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:37.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.244155+0000", 2020-08-07T15:06:37.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:37.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:37.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.244155+0000", 2020-08-07T15:06:37.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1430000000000001e-05 2020-08-07T15:06:37.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:37.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.244207+0000", 2020-08-07T15:06:37.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6400000000000002e-06 2020-08-07T15:06:37.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:37.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.244212+0000", 2020-08-07T15:06:37.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1727e-05 2020-08-07T15:06:37.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:37.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.244224+0000", 2020-08-07T15:06:37.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9420000000000003e-06 2020-08-07T15:06:37.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:37.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.244229+0000", 2020-08-07T15:06:37.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3739000000000007e-05 2020-08-07T15:06:37.458 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.459 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:37.459 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.459 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.459 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:06:37.459 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 11 2020-08-07T15:06:37.460 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:06:37.461 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:06:37.474 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:37.470+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:06:37.567 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:37.564+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:37.671 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:37.669+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:37.686 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:06:37.687 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:06:37.687 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:06:37.687 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:06:37.687 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:06:37.700 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:06:37.782 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:37.779+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:37.867 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:37.865+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:37.879 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:06:37.900 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:06:37.900 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:06:37.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:06:37.901 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:06:37.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.901 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:1216 3.f 3:f9861620:::benchmark_data_smithi074_35447_object151:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e22)", 2020-08-07T15:06:37.901 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.086932+0000", 2020-08-07T15:06:37.901 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.80643742399999996, 2020-08-07T15:06:37.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.082726305, 2020-08-07T15:06:37.902 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:37.902 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:37.902 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:37.903 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:37.903 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:37.903 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1216 2020-08-07T15:06:37.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.904 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:37.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:37.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.086932+0000", 2020-08-07T15:06:37.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:37.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:37.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.086932+0000", 2020-08-07T15:06:37.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7265999999999998e-05 2020-08-07T15:06:37.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:37.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.086980+0000", 2020-08-07T15:06:37.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1399999999999999e-06 2020-08-07T15:06:37.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:37.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.086985+0000", 2020-08-07T15:06:37.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0672999999999999e-05 2020-08-07T15:06:37.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:37.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.086995+0000", 2020-08-07T15:06:37.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6099999999999999e-06 2020-08-07T15:06:37.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:37.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.087000+0000", 2020-08-07T15:06:37.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031776840000000001 2020-08-07T15:06:37.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:37.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.090178+0000", 2020-08-07T15:06:37.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6989000000000003e-05 2020-08-07T15:06:37.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:37.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.090275+0000", 2020-08-07T15:06:37.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017160499999999999 2020-08-07T15:06:37.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:37.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.090446+0000", 2020-08-07T15:06:37.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057417680000000004 2020-08-07T15:06:37.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:37.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.096188+0000", 2020-08-07T15:06:37.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010404870000000001 2020-08-07T15:06:37.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:37.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.097229+0000", 2020-08-07T15:06:37.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.072317084000000004 2020-08-07T15:06:37.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:37.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.169546+0000", 2020-08-07T15:06:37.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.066e-06 2020-08-07T15:06:37.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:37.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.169547+0000", 2020-08-07T15:06:37.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3497999999999994e-05 2020-08-07T15:06:37.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:37.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.169610+0000", 2020-08-07T15:06:37.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8435e-05 2020-08-07T15:06:37.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:37.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.169659+0000", 2020-08-07T15:06:37.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.082726305 2020-08-07T15:06:37.922 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.922 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:37.922 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.923 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:1448 3.f 3:f4bbdfc5:::benchmark_data_smithi074_35447_object180:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:37.923 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.209656+0000", 2020-08-07T15:06:37.923 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.68371392499999994, 2020-08-07T15:06:37.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056182748999999997, 2020-08-07T15:06:37.923 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:37.924 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:37.924 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:37.924 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:37.924 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:37.924 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1448 2020-08-07T15:06:37.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.925 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:37.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:37.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.209656+0000", 2020-08-07T15:06:37.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:37.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:37.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.209656+0000", 2020-08-07T15:06:37.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0454999999999997e-05 2020-08-07T15:06:37.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:37.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.209706+0000", 2020-08-07T15:06:37.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.178e-06 2020-08-07T15:06:37.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:37.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.209713+0000", 2020-08-07T15:06:37.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1361e-05 2020-08-07T15:06:37.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:37.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.209724+0000", 2020-08-07T15:06:37.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7010000000000001e-06 2020-08-07T15:06:37.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:37.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.209730+0000", 2020-08-07T15:06:37.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013466500000000001 2020-08-07T15:06:37.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:37.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.209864+0000", 2020-08-07T15:06:37.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6986999999999998e-05 2020-08-07T15:06:37.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:37.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.209961+0000", 2020-08-07T15:06:37.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018270600000000001 2020-08-07T15:06:37.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:37.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.210144+0000", 2020-08-07T15:06:37.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035264739999999999 2020-08-07T15:06:37.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:37.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.213670+0000", 2020-08-07T15:06:37.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0869999999999999e-06 2020-08-07T15:06:37.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:37.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.213671+0000", 2020-08-07T15:06:37.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6928999999999996e-05 2020-08-07T15:06:37.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:37.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.213738+0000", 2020-08-07T15:06:37.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050888173000000002 2020-08-07T15:06:37.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:37.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.264627+0000", 2020-08-07T15:06:37.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011542340000000001 2020-08-07T15:06:37.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:37.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.265781+0000", 2020-08-07T15:06:37.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5300000000000003e-07 2020-08-07T15:06:37.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:37.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.265782+0000", 2020-08-07T15:06:37.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7046000000000002e-05 2020-08-07T15:06:37.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:37.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.265839+0000", 2020-08-07T15:06:37.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056182748999999997 2020-08-07T15:06:37.943 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.944 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:37.944 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.944 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:1448 3.f e23/20 ondisk, result = 0)", 2020-08-07T15:06:37.945 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.212696+0000", 2020-08-07T15:06:37.945 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.68067387099999999, 2020-08-07T15:06:37.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.053147563000000002, 2020-08-07T15:06:37.945 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:37.946 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:37.946 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:37.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:37.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.212696+0000", 2020-08-07T15:06:37.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:37.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:37.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.212696+0000", 2020-08-07T15:06:37.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3853999999999998e-05 2020-08-07T15:06:37.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:37.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.212730+0000", 2020-08-07T15:06:37.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1789999999999996e-06 2020-08-07T15:06:37.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:37.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.212736+0000", 2020-08-07T15:06:37.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.052984667999999999 2020-08-07T15:06:37.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:37.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.265721+0000", 2020-08-07T15:06:37.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1189999999999999e-06 2020-08-07T15:06:37.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:37.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.265725+0000", 2020-08-07T15:06:37.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5886999999999997e-05 2020-08-07T15:06:37.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:37.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.265761+0000", 2020-08-07T15:06:37.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0203e-05 2020-08-07T15:06:37.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:37.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.265771+0000", 2020-08-07T15:06:37.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2652999999999997e-05 2020-08-07T15:06:37.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:37.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.265843+0000", 2020-08-07T15:06:37.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.053147563000000002 2020-08-07T15:06:37.969 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.969 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:37.969 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.970 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:1552 3.f 3:f26d5620:::benchmark_data_smithi074_35447_object193:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:37.970 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.266724+0000", 2020-08-07T15:06:37.970 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.626645805, 2020-08-07T15:06:37.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056675062999999998, 2020-08-07T15:06:37.970 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:37.971 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:37.971 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:37.971 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:37.971 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:37.971 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1552 2020-08-07T15:06:37.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.972 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:37.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:37.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.266724+0000", 2020-08-07T15:06:37.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:37.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:37.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.266724+0000", 2020-08-07T15:06:37.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4388000000000003e-05 2020-08-07T15:06:37.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:37.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.266768+0000", 2020-08-07T15:06:37.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7409999999999998e-06 2020-08-07T15:06:37.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:37.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.266774+0000", 2020-08-07T15:06:37.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0463e-05 2020-08-07T15:06:37.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:37.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.266785+0000", 2020-08-07T15:06:37.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3359999999999997e-06 2020-08-07T15:06:37.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:37.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.266789+0000", 2020-08-07T15:06:37.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028321969999999998 2020-08-07T15:06:37.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:37.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.269621+0000", 2020-08-07T15:06:37.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3462999999999996e-05 2020-08-07T15:06:37.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:37.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.269715+0000", 2020-08-07T15:06:37.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017893299999999999 2020-08-07T15:06:37.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:37.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.269894+0000", 2020-08-07T15:06:37.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025520859999999999 2020-08-07T15:06:37.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:37.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.272446+0000", 2020-08-07T15:06:37.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5000000000000002e-07 2020-08-07T15:06:37.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:37.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.272446+0000", 2020-08-07T15:06:37.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7522999999999997e-05 2020-08-07T15:06:37.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:37.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.272514+0000", 2020-08-07T15:06:37.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019605619000000001 2020-08-07T15:06:37.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:37.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.292119+0000", 2020-08-07T15:06:37.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031230462 2020-08-07T15:06:37.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:37.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.323350+0000", 2020-08-07T15:06:37.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2399999999999997e-07 2020-08-07T15:06:37.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:37.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.323351+0000", 2020-08-07T15:06:37.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8477999999999999e-05 2020-08-07T15:06:37.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:37.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.323399+0000", 2020-08-07T15:06:37.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056675062999999998 2020-08-07T15:06:37.995 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.995 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:37.996 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:37.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:37.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:37.997 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:1846 3.e 3:7b7119ba:::benchmark_data_smithi074_35447_object230:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:37.998 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.412892+0000", 2020-08-07T15:06:37.999 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.48047749699999998, 2020-08-07T15:06:37.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057229926, 2020-08-07T15:06:37.999 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:38.000 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:38.000 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:38.000 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:38.001 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:38.001 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1846 2020-08-07T15:06:38.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.002 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:38.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:38.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.412892+0000", 2020-08-07T15:06:38.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:38.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:38.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.412892+0000", 2020-08-07T15:06:38.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6558000000000003e-05 2020-08-07T15:06:38.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:38.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.412949+0000", 2020-08-07T15:06:38.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.066e-06 2020-08-07T15:06:38.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:38.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.412955+0000", 2020-08-07T15:06:38.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2332e-05 2020-08-07T15:06:38.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:38.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.412967+0000", 2020-08-07T15:06:38.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7200000000000003e-06 2020-08-07T15:06:38.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:38.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.412973+0000", 2020-08-07T15:06:38.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017713819999999999 2020-08-07T15:06:38.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:38.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.414744+0000", 2020-08-07T15:06:38.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3847999999999997e-05 2020-08-07T15:06:38.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:38.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.414798+0000", 2020-08-07T15:06:38.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000159755 2020-08-07T15:06:38.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:38.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.414958+0000", 2020-08-07T15:06:38.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040739909999999999 2020-08-07T15:06:38.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:38.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.419032+0000", 2020-08-07T15:06:38.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1399999999999997e-07 2020-08-07T15:06:38.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:38.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.419032+0000", 2020-08-07T15:06:38.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8164000000000001e-05 2020-08-07T15:06:38.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:38.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.419091+0000", 2020-08-07T15:06:38.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036758229999999999 2020-08-07T15:06:38.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:38.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.422766+0000", 2020-08-07T15:06:38.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047313753 2020-08-07T15:06:38.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:38.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.470080+0000", 2020-08-07T15:06:38.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2079999999999999e-06 2020-08-07T15:06:38.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:38.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.470081+0000", 2020-08-07T15:06:38.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0812000000000002e-05 2020-08-07T15:06:38.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:38.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.470122+0000", 2020-08-07T15:06:38.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057229926 2020-08-07T15:06:38.032 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.032 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:38.032 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.033 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:1847 3.e 3:7b7119ba:::benchmark_data_smithi074_35447_object230:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:38.033 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.413195+0000", 2020-08-07T15:06:38.033 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.48017528700000001, 2020-08-07T15:06:38.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057200619000000001, 2020-08-07T15:06:38.033 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:38.034 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:38.034 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:38.034 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:38.034 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:38.034 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1847 2020-08-07T15:06:38.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.035 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:38.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:38.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.413195+0000", 2020-08-07T15:06:38.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:38.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:38.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.413195+0000", 2020-08-07T15:06:38.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3659000000000002e-05 2020-08-07T15:06:38.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:38.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.413248+0000", 2020-08-07T15:06:38.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5089999999999999e-06 2020-08-07T15:06:38.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:38.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.413255+0000", 2020-08-07T15:06:38.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2982e-05 2020-08-07T15:06:38.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:38.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.413268+0000", 2020-08-07T15:06:38.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.392e-06 2020-08-07T15:06:38.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:38.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.413274+0000", 2020-08-07T15:06:38.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023784930000000002 2020-08-07T15:06:38.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:38.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.415653+0000", 2020-08-07T15:06:38.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00094896800000000001 2020-08-07T15:06:38.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:38.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.416602+0000", 2020-08-07T15:06:38.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021029299999999999 2020-08-07T15:06:38.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:38.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.416812+0000", 2020-08-07T15:06:38.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023909159999999999 2020-08-07T15:06:38.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:38.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.419203+0000", 2020-08-07T15:06:38.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1100000000000001e-07 2020-08-07T15:06:38.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:38.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.419203+0000", 2020-08-07T15:06:38.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4474000000000001e-05 2020-08-07T15:06:38.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:38.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.419258+0000", 2020-08-07T15:06:38.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035497319999999999 2020-08-07T15:06:38.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:38.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.422807+0000", 2020-08-07T15:06:38.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047548451999999998 2020-08-07T15:06:38.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:38.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.470356+0000", 2020-08-07T15:06:38.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.68e-07 2020-08-07T15:06:38.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:38.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.470356+0000", 2020-08-07T15:06:38.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8670000000000001e-05 2020-08-07T15:06:38.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:38.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.470395+0000", 2020-08-07T15:06:38.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057200619000000001 2020-08-07T15:06:38.054 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.054 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:38.055 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.056 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:1848 3.e 3:7b7119ba:::benchmark_data_smithi074_35447_object230:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:38.056 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.413310+0000", 2020-08-07T15:06:38.056 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.48005973600000001, 2020-08-07T15:06:38.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057241856000000001, 2020-08-07T15:06:38.057 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:38.057 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:38.057 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:38.058 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:38.058 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:38.058 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1848 2020-08-07T15:06:38.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.059 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:38.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:38.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.413310+0000", 2020-08-07T15:06:38.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:38.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:38.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.413310+0000", 2020-08-07T15:06:38.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9487999999999998e-05 2020-08-07T15:06:38.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:38.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.413360+0000", 2020-08-07T15:06:38.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3900000000000001e-06 2020-08-07T15:06:38.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:38.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.413365+0000", 2020-08-07T15:06:38.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1276e-05 2020-08-07T15:06:38.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:38.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.413376+0000", 2020-08-07T15:06:38.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1059999999999999e-06 2020-08-07T15:06:38.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:38.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.413381+0000", 2020-08-07T15:06:38.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040345809999999998 2020-08-07T15:06:38.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:38.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.417416+0000", 2020-08-07T15:06:38.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014445699999999999 2020-08-07T15:06:38.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:38.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.417560+0000", 2020-08-07T15:06:38.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015338299999999999 2020-08-07T15:06:38.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:38.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.417714+0000", 2020-08-07T15:06:38.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051834020000000001 2020-08-07T15:06:38.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:38.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.422897+0000", 2020-08-07T15:06:38.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025060700000000002 2020-08-07T15:06:38.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:38.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.423148+0000", 2020-08-07T15:06:38.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047290475999999998 2020-08-07T15:06:38.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:38.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.470438+0000", 2020-08-07T15:06:38.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8599999999999998e-07 2020-08-07T15:06:38.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:38.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.470439+0000", 2020-08-07T15:06:38.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7319999999999999e-05 2020-08-07T15:06:38.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:38.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.470506+0000", 2020-08-07T15:06:38.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5683999999999998e-05 2020-08-07T15:06:38.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:38.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.470552+0000", 2020-08-07T15:06:38.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057241856000000001 2020-08-07T15:06:38.081 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.081 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:38.081 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.082 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:1992 3.d 3:be5e18c3:::benchmark_data_smithi074_35447_object248:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:38.082 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.485984+0000", 2020-08-07T15:06:38.082 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.40738561699999998, 2020-08-07T15:06:38.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.088926784999999994, 2020-08-07T15:06:38.083 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:38.083 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:38.083 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:38.083 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:38.083 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:38.084 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1992 2020-08-07T15:06:38.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.084 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:38.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:38.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.485984+0000", 2020-08-07T15:06:38.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:38.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:38.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.485984+0000", 2020-08-07T15:06:38.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7330999999999998e-05 2020-08-07T15:06:38.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:38.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.486032+0000", 2020-08-07T15:06:38.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.163e-06 2020-08-07T15:06:38.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:38.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.486035+0000", 2020-08-07T15:06:38.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1079999999999998e-06 2020-08-07T15:06:38.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:38.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.486041+0000", 2020-08-07T15:06:38.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6709999999999999e-06 2020-08-07T15:06:38.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:38.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.486043+0000", 2020-08-07T15:06:38.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0047515910000000003 2020-08-07T15:06:38.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:38.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.490795+0000", 2020-08-07T15:06:38.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5330999999999999e-05 2020-08-07T15:06:38.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:38.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.490890+0000", 2020-08-07T15:06:38.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016098299999999999 2020-08-07T15:06:38.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:06:38.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.491051+0000", 2020-08-07T15:06:38.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025878960000000001 2020-08-07T15:06:38.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:38.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.493639+0000", 2020-08-07T15:06:38.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9800000000000003e-07 2020-08-07T15:06:38.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:38.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.493640+0000", 2020-08-07T15:06:38.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3831999999999995e-05 2020-08-07T15:06:38.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:38.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.493704+0000", 2020-08-07T15:06:38.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021665360000000002 2020-08-07T15:06:38.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:38.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.495870+0000", 2020-08-07T15:06:38.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.078974454999999999 2020-08-07T15:06:38.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:38.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.574845+0000", 2020-08-07T15:06:38.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8100000000000001e-07 2020-08-07T15:06:38.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:38.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.574846+0000", 2020-08-07T15:06:38.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5408999999999997e-05 2020-08-07T15:06:38.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:38.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.574911+0000", 2020-08-07T15:06:38.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.088926784999999994 2020-08-07T15:06:38.101 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.101 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:38.101 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.102 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:1992 3.d e23/20 ondisk, result = 0)", 2020-08-07T15:06:38.102 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.492954+0000", 2020-08-07T15:06:38.102 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.40041564600000001, 2020-08-07T15:06:38.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.081961185000000006, 2020-08-07T15:06:38.102 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:38.102 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:38.103 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:38.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:38.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.492954+0000", 2020-08-07T15:06:38.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:38.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:38.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.492954+0000", 2020-08-07T15:06:38.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7756000000000002e-05 2020-08-07T15:06:38.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:38.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.493002+0000", 2020-08-07T15:06:38.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7259999999999998e-06 2020-08-07T15:06:38.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:38.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.493005+0000", 2020-08-07T15:06:38.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.081463000999999993 2020-08-07T15:06:38.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:38.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.574468+0000", 2020-08-07T15:06:38.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4799999999999998e-06 2020-08-07T15:06:38.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:38.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.574474+0000", 2020-08-07T15:06:38.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00035203300000000002 2020-08-07T15:06:38.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:38.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.574826+0000", 2020-08-07T15:06:38.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8079999999999999e-06 2020-08-07T15:06:38.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:38.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.574836+0000", 2020-08-07T15:06:38.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9381000000000005e-05 2020-08-07T15:06:38.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:38.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.574915+0000", 2020-08-07T15:06:38.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.081961185000000006 2020-08-07T15:06:38.111 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.112 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:38.112 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.112 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:2040 3.f 3:f51b48fb:::benchmark_data_smithi074_35447_object254:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:38.112 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.510787+0000", 2020-08-07T15:06:38.113 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.38258260700000002, 2020-08-07T15:06:38.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.10126199600000001, 2020-08-07T15:06:38.113 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:38.113 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:38.113 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:38.114 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:38.114 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:38.114 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2040 2020-08-07T15:06:38.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.114 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:38.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:38.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.510787+0000", 2020-08-07T15:06:38.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:38.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:38.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.510787+0000", 2020-08-07T15:06:38.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2152e-05 2020-08-07T15:06:38.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:38.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.510819+0000", 2020-08-07T15:06:38.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0780000000000003e-06 2020-08-07T15:06:38.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:38.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.510824+0000", 2020-08-07T15:06:38.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4208e-05 2020-08-07T15:06:38.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:38.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.510839+0000", 2020-08-07T15:06:38.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9540000000000003e-06 2020-08-07T15:06:38.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:38.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.510844+0000", 2020-08-07T15:06:38.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001202037 2020-08-07T15:06:38.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:38.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.512046+0000", 2020-08-07T15:06:38.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010483499999999999 2020-08-07T15:06:38.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:38.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.512150+0000", 2020-08-07T15:06:38.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016058399999999999 2020-08-07T15:06:38.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:38.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.512311+0000", 2020-08-07T15:06:38.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001516597 2020-08-07T15:06:38.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:38.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.513828+0000", 2020-08-07T15:06:38.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0240000000000001e-06 2020-08-07T15:06:38.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:38.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.513829+0000", 2020-08-07T15:06:38.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7694999999999994e-05 2020-08-07T15:06:38.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:38.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.513906+0000", 2020-08-07T15:06:38.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023810749999999999 2020-08-07T15:06:38.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:38.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.516287+0000", 2020-08-07T15:06:38.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.095691538000000007 2020-08-07T15:06:38.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:38.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.611979+0000", 2020-08-07T15:06:38.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.85e-07 2020-08-07T15:06:38.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:38.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.611980+0000", 2020-08-07T15:06:38.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9434e-05 2020-08-07T15:06:38.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:38.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.612049+0000", 2020-08-07T15:06:38.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.10126199600000001 2020-08-07T15:06:38.130 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.130 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:38.131 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.131 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:2040 3.f e23/20 ondisk, result = 0)", 2020-08-07T15:06:38.131 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.520273+0000", 2020-08-07T15:06:38.132 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.37309679699999998, 2020-08-07T15:06:38.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091780795999999998, 2020-08-07T15:06:38.132 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:38.132 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:38.133 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:38.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:38.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.520273+0000", 2020-08-07T15:06:38.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:38.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:38.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.520273+0000", 2020-08-07T15:06:38.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2904000000000002e-05 2020-08-07T15:06:38.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:38.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.520306+0000", 2020-08-07T15:06:38.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.511e-06 2020-08-07T15:06:38.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:38.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.520312+0000", 2020-08-07T15:06:38.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090207990000000002 2020-08-07T15:06:38.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:38.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.610520+0000", 2020-08-07T15:06:38.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9000000000000003e-06 2020-08-07T15:06:38.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:38.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.610526+0000", 2020-08-07T15:06:38.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014350369999999999 2020-08-07T15:06:38.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:38.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.611961+0000", 2020-08-07T15:06:38.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6719999999999999e-06 2020-08-07T15:06:38.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:38.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.611971+0000", 2020-08-07T15:06:38.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2781999999999994e-05 2020-08-07T15:06:38.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:38.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.612054+0000", 2020-08-07T15:06:38.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091780795999999998 2020-08-07T15:06:38.141 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.141 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:38.142 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.142 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:2160 3.d 3:b2aad63b:::benchmark_data_smithi074_35447_object269:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:38.142 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.575581+0000", 2020-08-07T15:06:38.142 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.31778872800000002, 2020-08-07T15:06:38.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14826365699999999, 2020-08-07T15:06:38.143 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:38.143 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:38.143 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:38.143 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:38.144 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:38.144 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2160 2020-08-07T15:06:38.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.144 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:38.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:38.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.575581+0000", 2020-08-07T15:06:38.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:38.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:38.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.575581+0000", 2020-08-07T15:06:38.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0838999999999999e-05 2020-08-07T15:06:38.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:38.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.575632+0000", 2020-08-07T15:06:38.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2009999999999996e-06 2020-08-07T15:06:38.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:38.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.575638+0000", 2020-08-07T15:06:38.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1881e-05 2020-08-07T15:06:38.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:38.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.575650+0000", 2020-08-07T15:06:38.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0290000000000002e-06 2020-08-07T15:06:38.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:38.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.575656+0000", 2020-08-07T15:06:38.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000258022 2020-08-07T15:06:38.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:38.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.575914+0000", 2020-08-07T15:06:38.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8507999999999999e-05 2020-08-07T15:06:38.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:38.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.576013+0000", 2020-08-07T15:06:38.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021717399999999999 2020-08-07T15:06:38.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:06:38.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.576230+0000", 2020-08-07T15:06:38.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028471500000000001 2020-08-07T15:06:38.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:38.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.579077+0000", 2020-08-07T15:06:38.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.023e-06 2020-08-07T15:06:38.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:38.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.579078+0000", 2020-08-07T15:06:38.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3954000000000005e-05 2020-08-07T15:06:38.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:38.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.579142+0000", 2020-08-07T15:06:38.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001332099 2020-08-07T15:06:38.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:38.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.580474+0000", 2020-08-07T15:06:38.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14332280999999999 2020-08-07T15:06:38.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:38.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.723797+0000", 2020-08-07T15:06:38.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9400000000000005e-07 2020-08-07T15:06:38.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:38.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.723797+0000", 2020-08-07T15:06:38.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7273e-05 2020-08-07T15:06:38.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:38.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.723845+0000", 2020-08-07T15:06:38.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14826365699999999 2020-08-07T15:06:38.177 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.177 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:38.177 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.178 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:2160 3.d e23/20 ondisk, result = 0)", 2020-08-07T15:06:38.179 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.582550+0000", 2020-08-07T15:06:38.179 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.31081965099999997, 2020-08-07T15:06:38.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14130019399999999, 2020-08-07T15:06:38.180 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:38.180 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:38.180 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:38.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:38.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.582550+0000", 2020-08-07T15:06:38.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:38.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:38.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.582550+0000", 2020-08-07T15:06:38.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8008999999999999e-05 2020-08-07T15:06:38.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:38.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.582578+0000", 2020-08-07T15:06:38.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6099999999999999e-06 2020-08-07T15:06:38.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:38.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.582583+0000", 2020-08-07T15:06:38.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14116865000000001 2020-08-07T15:06:38.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.723751+0000", 2020-08-07T15:06:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2740000000000001e-06 2020-08-07T15:06:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.723756+0000", 2020-08-07T15:06:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0642000000000002e-05 2020-08-07T15:06:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.723786+0000", 2020-08-07T15:06:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1699999999999996e-06 2020-08-07T15:06:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.723792+0000", 2020-08-07T15:06:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8838999999999997e-05 2020-08-07T15:06:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.723850+0000", 2020-08-07T15:06:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14130019399999999 2020-08-07T15:06:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:2232 3.e 3:7db3449e:::benchmark_data_smithi074_35447_object278:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.606053+0000", 2020-08-07T15:06:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.287316299, 2020-08-07T15:06:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.069955568999999995, 2020-08-07T15:06:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2232 2020-08-07T15:06:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.606053+0000", 2020-08-07T15:06:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.606053+0000", 2020-08-07T15:06:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3424999999999998e-05 2020-08-07T15:06:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.606107+0000", 2020-08-07T15:06:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2020-08-07T15:06:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.606112+0000", 2020-08-07T15:06:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4242e-05 2020-08-07T15:06:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.606127+0000", 2020-08-07T15:06:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.101e-06 2020-08-07T15:06:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.606132+0000", 2020-08-07T15:06:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054716499999999998 2020-08-07T15:06:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:38.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.611603+0000", 2020-08-07T15:06:38.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9007999999999999e-05 2020-08-07T15:06:38.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:38.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.611692+0000", 2020-08-07T15:06:38.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016920300000000001 2020-08-07T15:06:38.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:38.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.611862+0000", 2020-08-07T15:06:38.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037142659999999999 2020-08-07T15:06:38.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:38.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.615576+0000", 2020-08-07T15:06:38.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9599999999999998e-07 2020-08-07T15:06:38.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:38.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.615577+0000", 2020-08-07T15:06:38.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7203000000000001e-05 2020-08-07T15:06:38.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.615634+0000", 2020-08-07T15:06:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00063766599999999995 2020-08-07T15:06:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.616272+0000", 2020-08-07T15:06:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.059665535999999998 2020-08-07T15:06:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.675937+0000", 2020-08-07T15:06:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1900000000000001e-07 2020-08-07T15:06:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.675938+0000", 2020-08-07T15:06:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1055000000000003e-05 2020-08-07T15:06:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.676009+0000", 2020-08-07T15:06:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.069955568999999995 2020-08-07T15:06:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:2239 3.f 3:f4e7d174:::benchmark_data_smithi074_35447_object279:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.609418+0000", 2020-08-07T15:06:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.28395226400000001, 2020-08-07T15:06:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064290665999999996, 2020-08-07T15:06:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2239 2020-08-07T15:06:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.609418+0000", 2020-08-07T15:06:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.609418+0000", 2020-08-07T15:06:38.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0725e-05 2020-08-07T15:06:38.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:38.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.609468+0000", 2020-08-07T15:06:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4920000000000001e-06 2020-08-07T15:06:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.609474+0000", 2020-08-07T15:06:38.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1052e-05 2020-08-07T15:06:38.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:38.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.609485+0000", 2020-08-07T15:06:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6199999999999998e-06 2020-08-07T15:06:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.609489+0000", 2020-08-07T15:06:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054942480000000002 2020-08-07T15:06:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.614984+0000", 2020-08-07T15:06:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5758e-05 2020-08-07T15:06:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.615079+0000", 2020-08-07T15:06:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000160512 2020-08-07T15:06:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.615240+0000", 2020-08-07T15:06:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0047578179999999996 2020-08-07T15:06:38.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:38.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.619998+0000", 2020-08-07T15:06:38.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051544709999999999 2020-08-07T15:06:38.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.625152+0000", 2020-08-07T15:06:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048438923000000002 2020-08-07T15:06:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.673591+0000", 2020-08-07T15:06:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1200000000000001e-06 2020-08-07T15:06:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.673592+0000", 2020-08-07T15:06:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5143999999999998e-05 2020-08-07T15:06:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.673657+0000", 2020-08-07T15:06:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0782999999999998e-05 2020-08-07T15:06:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:38.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.673708+0000", 2020-08-07T15:06:38.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064290665999999996 2020-08-07T15:06:38.229 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.229 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:2240 3.f 3:f4e7d174:::benchmark_data_smithi074_35447_object279:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.609524+0000", 2020-08-07T15:06:38.231 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.28384536999999999, 2020-08-07T15:06:38.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091060390000000005, 2020-08-07T15:06:38.231 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:38.232 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:38.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:38.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:38.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:38.232 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2240 2020-08-07T15:06:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.609524+0000", 2020-08-07T15:06:38.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:38.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:38.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.609524+0000", 2020-08-07T15:06:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9509999999999999e-05 2020-08-07T15:06:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.609574+0000", 2020-08-07T15:06:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7389999999999996e-06 2020-08-07T15:06:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.609580+0000", 2020-08-07T15:06:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0701e-05 2020-08-07T15:06:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.609590+0000", 2020-08-07T15:06:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2560000000000004e-06 2020-08-07T15:06:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.609595+0000", 2020-08-07T15:06:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006234369 2020-08-07T15:06:38.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:38.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.615829+0000", 2020-08-07T15:06:38.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5988000000000003e-05 2020-08-07T15:06:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.615925+0000", 2020-08-07T15:06:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017731999999999999 2020-08-07T15:06:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:38.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.616102+0000", 2020-08-07T15:06:38.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0090567029999999993 2020-08-07T15:06:38.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:38.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.625159+0000", 2020-08-07T15:06:38.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0299999999999998e-07 2020-08-07T15:06:38.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:38.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.625160+0000", 2020-08-07T15:06:38.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0486999999999999e-05 2020-08-07T15:06:38.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:38.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.625220+0000", 2020-08-07T15:06:38.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048535425 2020-08-07T15:06:38.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:38.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.673756+0000", 2020-08-07T15:06:38.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.026772826999999999 2020-08-07T15:06:38.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:38.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.700528+0000", 2020-08-07T15:06:38.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7999999999999995e-07 2020-08-07T15:06:38.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:38.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.700529+0000", 2020-08-07T15:06:38.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5682e-05 2020-08-07T15:06:38.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:38.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.700585+0000", 2020-08-07T15:06:38.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091060390000000005 2020-08-07T15:06:38.248 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.248 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:38.249 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.249 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:2232 3.e e23/20 ondisk, result = 0)", 2020-08-07T15:06:38.249 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.617010+0000", 2020-08-07T15:06:38.249 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.276359515, 2020-08-07T15:06:38.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.059002865000000002, 2020-08-07T15:06:38.250 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:38.250 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:38.250 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:38.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:38.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.617010+0000", 2020-08-07T15:06:38.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:38.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:38.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.617010+0000", 2020-08-07T15:06:38.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6024000000000001e-05 2020-08-07T15:06:38.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:38.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.617026+0000", 2020-08-07T15:06:38.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9400000000000001e-06 2020-08-07T15:06:38.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:38.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.617031+0000", 2020-08-07T15:06:38.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056657206000000002 2020-08-07T15:06:38.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:38.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.673688+0000", 2020-08-07T15:06:38.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.25e-06 2020-08-07T15:06:38.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:38.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.673693+0000", 2020-08-07T15:06:38.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002227311 2020-08-07T15:06:38.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.675920+0000", 2020-08-07T15:06:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6059999999999997e-06 2020-08-07T15:06:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.675928+0000", 2020-08-07T15:06:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5527999999999998e-05 2020-08-07T15:06:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:38.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.676013+0000", 2020-08-07T15:06:38.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.059002865000000002 2020-08-07T15:06:38.259 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.260 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:38.260 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.260 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:2240 3.f e23/20 ondisk, result = 0)", 2020-08-07T15:06:38.260 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.619849+0000", 2020-08-07T15:06:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.27352035499999999, 2020-08-07T15:06:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.080742013000000001, 2020-08-07T15:06:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.619849+0000", 2020-08-07T15:06:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.619849+0000", 2020-08-07T15:06:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4949999999999999e-05 2020-08-07T15:06:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.619864+0000", 2020-08-07T15:06:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6150000000000001e-06 2020-08-07T15:06:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.619868+0000", 2020-08-07T15:06:38.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.079282021999999994 2020-08-07T15:06:38.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.699150+0000", 2020-08-07T15:06:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.417e-06 2020-08-07T15:06:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.699155+0000", 2020-08-07T15:06:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013617410000000001 2020-08-07T15:06:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.700517+0000", 2020-08-07T15:06:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8950000000000003e-06 2020-08-07T15:06:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.700523+0000", 2020-08-07T15:06:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8373000000000004e-05 2020-08-07T15:06:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.700591+0000", 2020-08-07T15:06:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.080742013000000001 2020-08-07T15:06:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:2448 3.d 3:b7852c08:::benchmark_data_smithi074_35447_object305:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.723999+0000", 2020-08-07T15:06:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.169371045, 2020-08-07T15:06:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15153707999999999, 2020-08-07T15:06:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2448 2020-08-07T15:06:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.723999+0000", 2020-08-07T15:06:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.723999+0000", 2020-08-07T15:06:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9113999999999998e-05 2020-08-07T15:06:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.724048+0000", 2020-08-07T15:06:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3959999999999997e-06 2020-08-07T15:06:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.724053+0000", 2020-08-07T15:06:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0475999999999999e-05 2020-08-07T15:06:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.724064+0000", 2020-08-07T15:06:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6739999999999996e-06 2020-08-07T15:06:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.724068+0000", 2020-08-07T15:06:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00053356300000000003 2020-08-07T15:06:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.724602+0000", 2020-08-07T15:06:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9407000000000006e-05 2020-08-07T15:06:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.724701+0000", 2020-08-07T15:06:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016530599999999999 2020-08-07T15:06:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:06:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.724867+0000", 2020-08-07T15:06:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021475690000000002 2020-08-07T15:06:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.727014+0000", 2020-08-07T15:06:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0900000000000002e-07 2020-08-07T15:06:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.727015+0000", 2020-08-07T15:06:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4678000000000001e-05 2020-08-07T15:06:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.727069+0000", 2020-08-07T15:06:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048504289999999999 2020-08-07T15:06:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.731920+0000", 2020-08-07T15:06:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14354978099999999 2020-08-07T15:06:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.875470+0000", 2020-08-07T15:06:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1200000000000002e-07 2020-08-07T15:06:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.875470+0000", 2020-08-07T15:06:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5566000000000002e-05 2020-08-07T15:06:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.875536+0000", 2020-08-07T15:06:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15153707999999999 2020-08-07T15:06:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:2448 3.d e23/20 ondisk, result = 0)", 2020-08-07T15:06:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.727706+0000", 2020-08-07T15:06:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.16566330500000001, 2020-08-07T15:06:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14783311299999999, 2020-08-07T15:06:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.727706+0000", 2020-08-07T15:06:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.727706+0000", 2020-08-07T15:06:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0843000000000003e-05 2020-08-07T15:06:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.727737+0000", 2020-08-07T15:06:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.578e-06 2020-08-07T15:06:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:38.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.727742+0000", 2020-08-07T15:06:38.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14669671400000001 2020-08-07T15:06:38.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.874439+0000", 2020-08-07T15:06:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0610000000000001e-06 2020-08-07T15:06:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.874445+0000", 2020-08-07T15:06:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010084549999999999 2020-08-07T15:06:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.875453+0000", 2020-08-07T15:06:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9880000000000004e-06 2020-08-07T15:06:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.875461+0000", 2020-08-07T15:06:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8473999999999994e-05 2020-08-07T15:06:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.875540+0000", 2020-08-07T15:06:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14783311299999999 2020-08-07T15:06:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:38.326 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:06:38.330 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:38.327+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:06:38.331 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:06:38.331 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:06:38.332 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:06:38.332 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:06:38.333 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:06:38.333 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:06:38.333 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:06:38.334 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 2 16 3339 3323 12.9796 14.3594 0.00596466 0.00959299 2020-08-07T15:06:38.335 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:06:38.351 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:06:38.352 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:06:38.352 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:06:38.352 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:06:38.352 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:06:38.353 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:06:38.353 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:06:38.353 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:06:38.353 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:06:38.430 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:38.429+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:06:38.529 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:38.530 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:38.532 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:06:38.531+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:06:38.545 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:06:38.634 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:38.632+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:38.741 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:38.739+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:06:38.767 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:38.767 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:38.782 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:06:38.848 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:38.842+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:38.944 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:38.943+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:38.956 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:38.956 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:38.971 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:06:39.048 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:39.043+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:39.147 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:39.145+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:39.159 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:39.160 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:39.190 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:06:39.191 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:06:39.191 INFO:tasks.thrashosds.thrasher:inject_pause on 6 2020-08-07T15:06:39.192 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2020-08-07T15:06:39.192 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2020-08-07T15:06:39.193 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2020-08-07T15:06:39.202 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:06:39.204 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:06:39.242 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 3 16 5053 5037 13.1163 13.3906 0.0353153 0.00945383 2020-08-07T15:06:39.247 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:39.246+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:06:39.349 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:39.347+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:39.369 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:06:39.369 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2020-08-07T15:06:39.370 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:06:39.415 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:39.416 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:39.425 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:06:39.450 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:39.448+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:39.550 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:39.549+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:39.621 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:39.622 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:39.636 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:06:39.651 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:39.650+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:06:39.753 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:39.752+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:39.854 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:39.852+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:39.859 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:39.859 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:39.874 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:06:39.955 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:39.954+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:06:40.056 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:40.055+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:40.092 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:40.094 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:40.104 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:06:40.161 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:40.159+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:06:40.184 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:06:40.241 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 4 16 6722 6706 13.0968 13.0391 0.0123295 0.00951042 2020-08-07T15:06:40.263 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:06:40.262+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:06:40.284 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:40.284 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:40.303 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:06:40.362 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:40.361+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:40.365 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:06:40.465 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:40.463+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:40.523 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:40.523 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:40.537 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:06:40.578 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:40.577+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:40.605 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:06:40.646 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:06:40.667 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:40.665+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:40.691 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:06:40.725 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:06:40.767 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:06:40.768 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:06:40.768 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:06:40.768 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:06:40.769 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:06:40.769 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:06:40.770 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:40.766+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:40.780 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:06:40.801 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:06:40.840 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:06:40.872 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:40.870+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:40.881 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:06:40.899 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:06:40.900 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:06:40.900 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:06:40.900 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:06:40.901 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:06:40.901 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:06:40.901 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:06:40.901 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:06:40.971 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:06:40.971 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:06:40.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:06:40.972 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:06:40.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:40.973 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:544 3.b 3:ddefb1a6:::benchmark_data_smithi074_35447_object67:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e22)", 2020-08-07T15:06:40.973 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:36.729434+0000", 2020-08-07T15:06:40.973 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4.23722288, 2020-08-07T15:06:40.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14661148299999999, 2020-08-07T15:06:40.974 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:40.974 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:40.974 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:40.974 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:40.975 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:40.975 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 544 2020-08-07T15:06:40.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:40.975 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:40.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:40.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:40.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.729434+0000", 2020-08-07T15:06:40.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:40.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:40.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:40.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:40.978 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.729434+0000", 2020-08-07T15:06:40.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7930999999999999e-05 2020-08-07T15:06:40.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:40.979 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:40.979 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:40.979 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.729482+0000", 2020-08-07T15:06:40.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8390000000000003e-06 2020-08-07T15:06:40.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:40.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:40.981 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:40.981 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.729488+0000", 2020-08-07T15:06:40.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.17e-05 2020-08-07T15:06:40.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:40.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:40.982 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:40.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.729500+0000", 2020-08-07T15:06:40.982 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2580000000000003e-06 2020-08-07T15:06:40.982 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:40.983 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:40.983 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:40.983 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.729505+0000", 2020-08-07T15:06:40.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.008922948 2020-08-07T15:06:40.983 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:40.983 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:40.984 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:40.984 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.738428+0000", 2020-08-07T15:06:40.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8058000000000002e-05 2020-08-07T15:06:40.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:40.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:40.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:40.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.738526+0000", 2020-08-07T15:06:40.985 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017383500000000001 2020-08-07T15:06:40.985 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:40.985 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:40.986 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:40.986 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.738700+0000", 2020-08-07T15:06:40.986 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012493000000000001 2020-08-07T15:06:40.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:40.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:40.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:40.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.739949+0000", 2020-08-07T15:06:40.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9299999999999997e-07 2020-08-07T15:06:40.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:40.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:40.990 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:40.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.739950+0000", 2020-08-07T15:06:40.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4191999999999998e-05 2020-08-07T15:06:40.990 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:40.990 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:40.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:40.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.740014+0000", 2020-08-07T15:06:40.991 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.029007440999999998 2020-08-07T15:06:40.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:40.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:40.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:40.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.769021+0000", 2020-08-07T15:06:40.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.106953786 2020-08-07T15:06:40.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:40.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:40.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:40.993 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.875975+0000", 2020-08-07T15:06:40.993 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9400000000000005e-07 2020-08-07T15:06:40.993 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:40.993 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:40.994 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:40.994 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.875976+0000", 2020-08-07T15:06:40.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9708000000000004e-05 2020-08-07T15:06:40.994 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:40.994 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:40.995 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:40.995 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.876046+0000", 2020-08-07T15:06:40.995 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14661148299999999 2020-08-07T15:06:40.995 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:40.995 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:40.996 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:40.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:40.996 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:40.996 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:1024 3.b 3:dab1244e:::benchmark_data_smithi074_35447_object127:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e22)", 2020-08-07T15:06:40.996 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:36.984082+0000", 2020-08-07T15:06:40.997 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 3.9825754409999998, 2020-08-07T15:06:40.997 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.099414719999999998, 2020-08-07T15:06:40.997 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:40.997 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:40.999 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:41.000 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:41.000 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:41.000 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 1024 2020-08-07T15:06:41.000 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.001 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:41.001 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.001 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:41.001 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.984082+0000", 2020-08-07T15:06:41.001 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:41.002 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.002 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.002 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:41.002 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.984082+0000", 2020-08-07T15:06:41.002 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2406e-05 2020-08-07T15:06:41.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:41.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.984124+0000", 2020-08-07T15:06:41.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2230000000000001e-06 2020-08-07T15:06:41.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:41.004 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.984128+0000", 2020-08-07T15:06:41.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.268e-06 2020-08-07T15:06:41.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:41.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.984138+0000", 2020-08-07T15:06:41.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.033e-06 2020-08-07T15:06:41.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:41.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.984141+0000", 2020-08-07T15:06:41.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0050588109999999999 2020-08-07T15:06:41.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:41.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.989199+0000", 2020-08-07T15:06:41.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7435000000000006e-05 2020-08-07T15:06:41.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:41.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.989287+0000", 2020-08-07T15:06:41.008 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014799 2020-08-07T15:06:41.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.009 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:41.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.989435+0000", 2020-08-07T15:06:41.010 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0048735590000000004 2020-08-07T15:06:41.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.010 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:41.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.994308+0000", 2020-08-07T15:06:41.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2000000000000001e-07 2020-08-07T15:06:41.012 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.012 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:41.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.994309+0000", 2020-08-07T15:06:41.013 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4069000000000002e-05 2020-08-07T15:06:41.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.014 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:41.014 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.994343+0000", 2020-08-07T15:06:41.014 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.031261111000000001 2020-08-07T15:06:41.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.025604+0000", 2020-08-07T15:06:41.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.057825354000000002 2020-08-07T15:06:41.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.083429+0000", 2020-08-07T15:06:41.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2999999999999999e-07 2020-08-07T15:06:41.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:41.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.083430+0000", 2020-08-07T15:06:41.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6310999999999996e-05 2020-08-07T15:06:41.030 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.030 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.030 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:41.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.083496+0000", 2020-08-07T15:06:41.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.099414719999999998 2020-08-07T15:06:41.031 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.031 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:41.032 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.032 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:1208 3.b 3:da696a66:::benchmark_data_smithi074_35447_object150:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e22)", 2020-08-07T15:06:41.033 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:37.084692+0000", 2020-08-07T15:06:41.033 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 3.8819649759999999, 2020-08-07T15:06:41.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.088029038000000004, 2020-08-07T15:06:41.033 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:41.033 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:41.034 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:41.034 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:41.034 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:41.034 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 1208 2020-08-07T15:06:41.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.035 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:41.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:41.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.084692+0000", 2020-08-07T15:06:41.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:41.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:41.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.084692+0000", 2020-08-07T15:06:41.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2908999999999998e-05 2020-08-07T15:06:41.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.037 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:41.038 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.084745+0000", 2020-08-07T15:06:41.038 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5590000000000002e-06 2020-08-07T15:06:41.038 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.038 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.039 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:41.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.084752+0000", 2020-08-07T15:06:41.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2954e-05 2020-08-07T15:06:41.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:41.040 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.084765+0000", 2020-08-07T15:06:41.040 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.48e-06 2020-08-07T15:06:41.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.041 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:41.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.084770+0000", 2020-08-07T15:06:41.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3019000000000003e-05 2020-08-07T15:06:41.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.042 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:41.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.084813+0000", 2020-08-07T15:06:41.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5507000000000006e-05 2020-08-07T15:06:41.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:41.043 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.084909+0000", 2020-08-07T15:06:41.045 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016816499999999999 2020-08-07T15:06:41.046 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.046 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.046 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:41.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.085077+0000", 2020-08-07T15:06:41.046 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034814709999999999 2020-08-07T15:06:41.047 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.047 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:41.047 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.088558+0000", 2020-08-07T15:06:41.047 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8400000000000003e-07 2020-08-07T15:06:41.048 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.048 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.048 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:41.048 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.088559+0000", 2020-08-07T15:06:41.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4061999999999998e-05 2020-08-07T15:06:41.049 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.049 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.049 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:41.049 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.088623+0000", 2020-08-07T15:06:41.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.033887682000000002 2020-08-07T15:06:41.050 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.050 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.050 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.050 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.122511+0000", 2020-08-07T15:06:41.051 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.050131579000000002 2020-08-07T15:06:41.051 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.051 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.051 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.172642+0000", 2020-08-07T15:06:41.052 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.16e-07 2020-08-07T15:06:41.053 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.053 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:41.054 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.172643+0000", 2020-08-07T15:06:41.054 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7850999999999997e-05 2020-08-07T15:06:41.054 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.055 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:41.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.172721+0000", 2020-08-07T15:06:41.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.088029038000000004 2020-08-07T15:06:41.056 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.056 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:41.057 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.059 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:1800 3.b 3:d14541be:::benchmark_data_smithi074_35447_object224:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:41.059 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:37.393810+0000", 2020-08-07T15:06:41.059 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 3.5728472729999998, 2020-08-07T15:06:41.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.091536357999999998, 2020-08-07T15:06:41.060 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:41.060 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:41.060 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:41.060 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:41.061 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:41.061 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 1800 2020-08-07T15:06:41.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.061 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:41.062 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.062 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:41.062 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.393810+0000", 2020-08-07T15:06:41.062 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:41.062 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.063 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:41.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.393810+0000", 2020-08-07T15:06:41.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7985000000000001e-05 2020-08-07T15:06:41.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.064 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:41.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.393838+0000", 2020-08-07T15:06:41.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1690000000000001e-06 2020-08-07T15:06:41.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.065 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:41.065 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.393841+0000", 2020-08-07T15:06:41.065 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5169999999999999e-06 2020-08-07T15:06:41.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.066 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.066 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:41.067 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.393846+0000", 2020-08-07T15:06:41.067 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6120000000000001e-06 2020-08-07T15:06:41.067 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.067 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.067 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:41.069 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.393849+0000", 2020-08-07T15:06:41.069 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001020499 2020-08-07T15:06:41.070 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.070 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.070 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:41.070 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.394870+0000", 2020-08-07T15:06:41.070 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2178e-05 2020-08-07T15:06:41.071 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.071 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.071 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:41.071 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.394932+0000", 2020-08-07T15:06:41.071 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011501399999999999 2020-08-07T15:06:41.072 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.072 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.072 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:41.072 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.395047+0000", 2020-08-07T15:06:41.072 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012871740000000001 2020-08-07T15:06:41.073 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.073 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.074 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:41.074 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.396334+0000", 2020-08-07T15:06:41.074 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.75e-07 2020-08-07T15:06:41.074 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.075 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.075 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:41.075 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.396335+0000", 2020-08-07T15:06:41.075 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5294000000000003e-05 2020-08-07T15:06:41.075 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.076 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.078 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:41.078 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.396390+0000", 2020-08-07T15:06:41.078 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.057008741000000002 2020-08-07T15:06:41.079 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.079 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.079 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.079 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.453399+0000", 2020-08-07T15:06:41.079 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.031871785999999999 2020-08-07T15:06:41.081 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.081 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.081 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.082 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.485270+0000", 2020-08-07T15:06:41.082 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7899999999999997e-07 2020-08-07T15:06:41.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.083 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.083 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:41.083 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.485271+0000", 2020-08-07T15:06:41.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4935e-05 2020-08-07T15:06:41.084 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.084 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:41.084 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.485346+0000", 2020-08-07T15:06:41.084 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.091536357999999998 2020-08-07T15:06:41.085 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.085 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:41.085 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.086 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:1800 3.b e23/20 ondisk, result = 0)", 2020-08-07T15:06:41.086 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:37.399583+0000", 2020-08-07T15:06:41.086 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 3.5670740809999999, 2020-08-07T15:06:41.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.085771950999999999, 2020-08-07T15:06:41.086 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:41.087 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:06:41.087 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:41.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:41.087 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.399583+0000", 2020-08-07T15:06:41.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:41.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:41.088 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.399583+0000", 2020-08-07T15:06:41.089 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5597999999999999e-05 2020-08-07T15:06:41.089 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.089 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.089 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:41.089 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.399599+0000", 2020-08-07T15:06:41.092 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.762e-06 2020-08-07T15:06:41.092 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.092 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.092 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:41.092 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.399601+0000", 2020-08-07T15:06:41.093 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.085610008000000001 2020-08-07T15:06:41.093 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.093 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.093 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:41.093 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.485211+0000", 2020-08-07T15:06:41.094 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.524e-06 2020-08-07T15:06:41.094 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.094 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.094 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:41.094 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.485217+0000", 2020-08-07T15:06:41.095 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3155999999999999e-05 2020-08-07T15:06:41.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.095 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.095 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:41.095 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.485250+0000", 2020-08-07T15:06:41.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9840000000000007e-06 2020-08-07T15:06:41.096 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.096 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.096 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:41.096 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.485259+0000", 2020-08-07T15:06:41.097 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5919000000000001e-05 2020-08-07T15:06:41.097 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.097 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.097 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:41.097 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.485355+0000", 2020-08-07T15:06:41.098 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.085771950999999999 2020-08-07T15:06:41.098 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.098 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:41.098 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.098 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.099 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.099 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:2600 3.b 3:d6a1a0d3:::benchmark_data_smithi074_35447_object324:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:41.099 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:37.800755+0000", 2020-08-07T15:06:41.099 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 3.1659024979999999, 2020-08-07T15:06:41.099 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.215895015, 2020-08-07T15:06:41.100 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:41.100 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:41.100 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:41.100 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:41.102 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:41.102 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 2600 2020-08-07T15:06:41.102 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.103 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:41.103 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.103 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:41.103 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.800755+0000", 2020-08-07T15:06:41.103 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:41.104 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.104 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.104 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:41.104 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.800755+0000", 2020-08-07T15:06:41.104 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7878999999999999e-05 2020-08-07T15:06:41.105 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.105 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.105 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:41.105 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.800782+0000", 2020-08-07T15:06:41.105 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0520000000000003e-06 2020-08-07T15:06:41.106 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.106 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.106 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:41.106 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.800787+0000", 2020-08-07T15:06:41.106 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0587999999999999e-05 2020-08-07T15:06:41.107 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.107 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.107 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:41.107 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.800797+0000", 2020-08-07T15:06:41.107 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4970000000000001e-06 2020-08-07T15:06:41.108 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.108 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.108 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:41.108 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.800803+0000", 2020-08-07T15:06:41.108 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0059548989999999996 2020-08-07T15:06:41.109 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.109 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.109 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:41.109 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.806757+0000", 2020-08-07T15:06:41.109 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000111375 2020-08-07T15:06:41.110 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.110 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.110 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:41.110 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.806869+0000", 2020-08-07T15:06:41.110 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000159345 2020-08-07T15:06:41.111 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.111 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.111 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:41.111 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.807028+0000", 2020-08-07T15:06:41.111 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002275329 2020-08-07T15:06:41.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.112 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:41.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.809304+0000", 2020-08-07T15:06:41.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2900000000000002e-07 2020-08-07T15:06:41.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.113 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:41.113 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.809304+0000", 2020-08-07T15:06:41.113 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8708000000000002e-05 2020-08-07T15:06:41.113 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.114 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:41.114 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.809353+0000", 2020-08-07T15:06:41.114 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.026379923 2020-08-07T15:06:41.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.115 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.115 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.115 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.835733+0000", 2020-08-07T15:06:41.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18084716200000001 2020-08-07T15:06:41.115 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.116 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.116 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.116 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.016580+0000", 2020-08-07T15:06:41.116 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9299999999999997e-07 2020-08-07T15:06:41.116 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.117 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:41.117 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.016581+0000", 2020-08-07T15:06:41.117 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8535999999999996e-05 2020-08-07T15:06:41.117 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.118 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.118 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:41.118 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.016650+0000", 2020-08-07T15:06:41.118 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.215895015 2020-08-07T15:06:41.118 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.118 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:41.119 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.119 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.119 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.119 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:2637 3.7 3:efb0df7a:::benchmark_data_smithi074_35447_object329:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:41.119 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:37.830953+0000", 2020-08-07T15:06:41.120 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 3.1357042490000002, 2020-08-07T15:06:41.120 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.092704192000000005, 2020-08-07T15:06:41.120 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:41.120 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:41.121 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:41.121 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:41.121 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:41.121 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 2637 2020-08-07T15:06:41.121 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.122 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:41.122 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.122 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:41.122 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.830953+0000", 2020-08-07T15:06:41.122 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:41.123 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.123 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.123 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:41.123 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.830953+0000", 2020-08-07T15:06:41.123 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8674999999999999e-05 2020-08-07T15:06:41.125 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.125 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.126 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:41.126 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.830981+0000", 2020-08-07T15:06:41.126 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1089999999999998e-06 2020-08-07T15:06:41.126 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.126 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.127 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:41.127 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.830985+0000", 2020-08-07T15:06:41.127 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2600000000000002e-06 2020-08-07T15:06:41.127 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.127 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.128 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:41.128 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.830991+0000", 2020-08-07T15:06:41.128 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7300000000000001e-06 2020-08-07T15:06:41.128 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.128 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.129 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:41.129 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.830994+0000", 2020-08-07T15:06:41.129 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0022845589999999998 2020-08-07T15:06:41.129 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.129 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.130 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:41.130 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.833278+0000", 2020-08-07T15:06:41.130 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.666e-05 2020-08-07T15:06:41.130 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.131 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.131 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:41.131 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.833365+0000", 2020-08-07T15:06:41.131 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000180874 2020-08-07T15:06:41.131 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.131 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.132 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:06:41.132 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.833546+0000", 2020-08-07T15:06:41.132 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033029460000000002 2020-08-07T15:06:41.132 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.132 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.133 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:41.133 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.836849+0000", 2020-08-07T15:06:41.133 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8000000000000004e-07 2020-08-07T15:06:41.135 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.136 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.136 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:41.136 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.836850+0000", 2020-08-07T15:06:41.137 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8195000000000002e-05 2020-08-07T15:06:41.137 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.137 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.138 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:41.138 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.836898+0000", 2020-08-07T15:06:41.138 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017705799999999999 2020-08-07T15:06:41.138 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.139 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.139 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.139 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.838668+0000", 2020-08-07T15:06:41.139 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.084915710000000005 2020-08-07T15:06:41.140 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.140 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.140 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.141 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.923584+0000", 2020-08-07T15:06:41.141 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3699999999999999e-07 2020-08-07T15:06:41.141 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.142 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.142 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:41.142 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.923585+0000", 2020-08-07T15:06:41.142 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2176999999999997e-05 2020-08-07T15:06:41.142 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.143 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.143 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:41.143 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.923657+0000", 2020-08-07T15:06:41.143 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.092704192000000005 2020-08-07T15:06:41.143 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.144 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:41.144 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.146 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.146 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.146 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:2638 3.7 3:efb0df7a:::benchmark_data_smithi074_35447_object329:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:41.146 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:37.831805+0000", 2020-08-07T15:06:41.147 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 3.1348524919999998, 2020-08-07T15:06:41.147 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.092084481999999995, 2020-08-07T15:06:41.147 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:41.147 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:41.148 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:41.148 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:41.148 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:41.148 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 2638 2020-08-07T15:06:41.148 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.149 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:41.149 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.149 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:41.149 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.831805+0000", 2020-08-07T15:06:41.149 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:41.150 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.150 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.150 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:41.150 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.831805+0000", 2020-08-07T15:06:41.151 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6946000000000001e-05 2020-08-07T15:06:41.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.151 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:41.151 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.831852+0000", 2020-08-07T15:06:41.151 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3790000000000003e-06 2020-08-07T15:06:41.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:41.152 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.831858+0000", 2020-08-07T15:06:41.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2719e-05 2020-08-07T15:06:41.153 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.153 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.153 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:41.153 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.831871+0000", 2020-08-07T15:06:41.153 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1890000000000004e-06 2020-08-07T15:06:41.154 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.154 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.154 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:41.155 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.831877+0000", 2020-08-07T15:06:41.155 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0020517930000000001 2020-08-07T15:06:41.155 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.155 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.155 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:41.155 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.833929+0000", 2020-08-07T15:06:41.156 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010235599999999999 2020-08-07T15:06:41.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.156 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:41.156 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.834031+0000", 2020-08-07T15:06:41.157 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018244499999999999 2020-08-07T15:06:41.157 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.157 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.157 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:06:41.157 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.834213+0000", 2020-08-07T15:06:41.158 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003091618 2020-08-07T15:06:41.158 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.158 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.158 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:41.158 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.837305+0000", 2020-08-07T15:06:41.159 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8999999999999995e-07 2020-08-07T15:06:41.159 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.159 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.159 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:41.159 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.837306+0000", 2020-08-07T15:06:41.160 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5615000000000002e-05 2020-08-07T15:06:41.160 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.160 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.160 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:41.160 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.837361+0000", 2020-08-07T15:06:41.160 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0014429930000000001 2020-08-07T15:06:41.161 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.161 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.161 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.161 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.838804+0000", 2020-08-07T15:06:41.161 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.085029281999999998 2020-08-07T15:06:41.162 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.162 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.162 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.162 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.923834+0000", 2020-08-07T15:06:41.162 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5700000000000002e-07 2020-08-07T15:06:41.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.163 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.163 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:41.163 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.923834+0000", 2020-08-07T15:06:41.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4799999999999997e-05 2020-08-07T15:06:41.164 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:41.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.923889+0000", 2020-08-07T15:06:41.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.092084481999999995 2020-08-07T15:06:41.165 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.165 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:41.165 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.166 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.166 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.166 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:2639 3.7 3:efb0df7a:::benchmark_data_smithi074_35447_object329:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:41.166 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:37.831913+0000", 2020-08-07T15:06:41.166 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 3.134743764, 2020-08-07T15:06:41.167 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.092101769, 2020-08-07T15:06:41.167 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:41.167 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:41.167 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:41.167 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:41.168 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:41.168 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 2639 2020-08-07T15:06:41.168 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.168 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:41.168 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.169 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:41.169 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.831913+0000", 2020-08-07T15:06:41.169 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:41.169 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.169 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.170 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:41.170 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.831913+0000", 2020-08-07T15:06:41.170 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6539000000000003e-05 2020-08-07T15:06:41.170 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.170 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.171 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:41.171 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.831960+0000", 2020-08-07T15:06:41.171 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5979999999999996e-06 2020-08-07T15:06:41.171 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.171 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.172 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:41.172 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.831965+0000", 2020-08-07T15:06:41.172 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2515e-05 2020-08-07T15:06:41.172 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.172 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.172 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:41.173 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.831978+0000", 2020-08-07T15:06:41.173 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1760000000000001e-06 2020-08-07T15:06:41.173 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.173 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.173 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:41.174 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.831983+0000", 2020-08-07T15:06:41.174 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0025534730000000001 2020-08-07T15:06:41.174 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.174 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.174 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:41.175 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.834537+0000", 2020-08-07T15:06:41.175 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000105548 2020-08-07T15:06:41.175 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.175 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.175 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:41.176 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.834642+0000", 2020-08-07T15:06:41.176 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015467899999999999 2020-08-07T15:06:41.177 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.177 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.177 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:06:41.177 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.834797+0000", 2020-08-07T15:06:41.178 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0025747180000000001 2020-08-07T15:06:41.178 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.179 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.179 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:41.179 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.837372+0000", 2020-08-07T15:06:41.179 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7999999999999995e-07 2020-08-07T15:06:41.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.180 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.180 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:41.180 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.837372+0000", 2020-08-07T15:06:41.180 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4648999999999999e-05 2020-08-07T15:06:41.181 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.181 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.181 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:41.181 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.837417+0000", 2020-08-07T15:06:41.181 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001425529 2020-08-07T15:06:41.182 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.182 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.838842+0000", 2020-08-07T15:06:41.182 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.085104157999999999 2020-08-07T15:06:41.183 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.183 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.183 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.183 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.923946+0000", 2020-08-07T15:06:41.183 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1399999999999997e-07 2020-08-07T15:06:41.184 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.184 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.184 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:41.184 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.923947+0000", 2020-08-07T15:06:41.184 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7992999999999998e-05 2020-08-07T15:06:41.184 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.185 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.185 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:41.185 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.924015+0000", 2020-08-07T15:06:41.185 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.092101769 2020-08-07T15:06:41.185 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.186 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:41.186 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.186 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.186 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.186 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:2640 3.7 3:efb0df7a:::benchmark_data_smithi074_35447_object329:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:41.187 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:37.832021+0000", 2020-08-07T15:06:41.187 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 3.1346361360000001, 2020-08-07T15:06:41.187 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.092079343999999994, 2020-08-07T15:06:41.187 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:41.187 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:41.188 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:41.188 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:41.188 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:41.188 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 2640 2020-08-07T15:06:41.188 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.189 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:41.189 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.189 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:41.189 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.832021+0000", 2020-08-07T15:06:41.189 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:41.189 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.190 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.190 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:41.190 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.832021+0000", 2020-08-07T15:06:41.190 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8513000000000001e-05 2020-08-07T15:06:41.190 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.191 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.191 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:41.191 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.832069+0000", 2020-08-07T15:06:41.191 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4689999999999996e-06 2020-08-07T15:06:41.191 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.192 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.192 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:41.192 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.832075+0000", 2020-08-07T15:06:41.192 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3039999999999999e-06 2020-08-07T15:06:41.192 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.192 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.193 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:41.193 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.832082+0000", 2020-08-07T15:06:41.193 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3730000000000001e-06 2020-08-07T15:06:41.193 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.193 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.194 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:41.194 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.832086+0000", 2020-08-07T15:06:41.194 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003102105 2020-08-07T15:06:41.194 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.194 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.195 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:41.195 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.835188+0000", 2020-08-07T15:06:41.195 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5891999999999996e-05 2020-08-07T15:06:41.195 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.195 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.196 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:41.196 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.835284+0000", 2020-08-07T15:06:41.196 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015661900000000001 2020-08-07T15:06:41.196 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.196 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.197 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:06:41.197 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.835440+0000", 2020-08-07T15:06:41.197 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019824970000000002 2020-08-07T15:06:41.197 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.197 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.197 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:41.198 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.837423+0000", 2020-08-07T15:06:41.198 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4999999999999998e-07 2020-08-07T15:06:41.198 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.198 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.198 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:41.199 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.837423+0000", 2020-08-07T15:06:41.199 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5011000000000001e-05 2020-08-07T15:06:41.199 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.199 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.199 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:41.200 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.837458+0000", 2020-08-07T15:06:41.200 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.041953071000000002 2020-08-07T15:06:41.200 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.200 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.200 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.201 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.879411+0000", 2020-08-07T15:06:41.201 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.044651001000000003 2020-08-07T15:06:41.201 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.201 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.201 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.201 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.924062+0000", 2020-08-07T15:06:41.202 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9599999999999999e-07 2020-08-07T15:06:41.202 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.202 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.202 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:41.202 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.924063+0000", 2020-08-07T15:06:41.203 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7543000000000003e-05 2020-08-07T15:06:41.203 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.203 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.203 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:41.203 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.924100+0000", 2020-08-07T15:06:41.204 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.092079343999999994 2020-08-07T15:06:41.204 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.204 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:41.204 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.204 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.205 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.205 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:2637 3.7 e23/20 ondisk, result = 0)", 2020-08-07T15:06:41.205 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:37.837528+0000", 2020-08-07T15:06:41.205 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 3.1291287130000001, 2020-08-07T15:06:41.205 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.086141176, 2020-08-07T15:06:41.205 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:41.206 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:06:41.206 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:41.206 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.206 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:41.206 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.837528+0000", 2020-08-07T15:06:41.207 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:41.207 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.207 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.207 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:41.207 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.837528+0000", 2020-08-07T15:06:41.208 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1758000000000003e-05 2020-08-07T15:06:41.208 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.208 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.208 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:41.208 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.837560+0000", 2020-08-07T15:06:41.209 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8320000000000002e-06 2020-08-07T15:06:41.209 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.210 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.210 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:41.210 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.837566+0000", 2020-08-07T15:06:41.210 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.085271244999999996 2020-08-07T15:06:41.210 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.211 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.211 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:41.211 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.922837+0000", 2020-08-07T15:06:41.211 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5679999999999999e-06 2020-08-07T15:06:41.211 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.212 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.212 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:41.212 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.922843+0000", 2020-08-07T15:06:41.212 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00072349199999999995 2020-08-07T15:06:41.212 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.212 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.213 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:41.213 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.923566+0000", 2020-08-07T15:06:41.213 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6130000000000004e-06 2020-08-07T15:06:41.213 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.213 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.214 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:41.214 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.923575+0000", 2020-08-07T15:06:41.214 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4667999999999996e-05 2020-08-07T15:06:41.214 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.214 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.215 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:41.215 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.923670+0000", 2020-08-07T15:06:41.215 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.086141176 2020-08-07T15:06:41.215 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.215 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:41.216 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.216 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.216 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.216 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:2638 3.7 e23/20 ondisk, result = 0)", 2020-08-07T15:06:41.216 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:37.837596+0000", 2020-08-07T15:06:41.216 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 3.1290612210000002, 2020-08-07T15:06:41.217 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.086300399999999999, 2020-08-07T15:06:41.217 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:41.217 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:06:41.217 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:41.217 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.218 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:41.218 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.837596+0000", 2020-08-07T15:06:41.218 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:41.218 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.218 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.219 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:41.219 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.837596+0000", 2020-08-07T15:06:41.219 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0145e-05 2020-08-07T15:06:41.219 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.220 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.220 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:41.221 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.837626+0000", 2020-08-07T15:06:41.222 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9930000000000003e-06 2020-08-07T15:06:41.222 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.222 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.222 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:41.223 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.837631+0000", 2020-08-07T15:06:41.223 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.085223959000000002 2020-08-07T15:06:41.223 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.223 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.223 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:41.224 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.922855+0000", 2020-08-07T15:06:41.224 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3189999999999999e-06 2020-08-07T15:06:41.224 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.224 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.224 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:41.225 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.922858+0000", 2020-08-07T15:06:41.225 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00095973499999999997 2020-08-07T15:06:41.225 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.226 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.226 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:41.226 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.923818+0000", 2020-08-07T15:06:41.226 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5529999999999993e-06 2020-08-07T15:06:41.227 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.227 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.227 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:41.227 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.923827+0000", 2020-08-07T15:06:41.227 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9696000000000005e-05 2020-08-07T15:06:41.228 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.228 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.228 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:41.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.923896+0000", 2020-08-07T15:06:41.228 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.086300399999999999 2020-08-07T15:06:41.229 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.229 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:41.229 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.229 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.229 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.230 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:2944 3.b 3:d7542c3f:::benchmark_data_smithi074_35447_object367:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:41.230 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:38.017796+0000", 2020-08-07T15:06:41.230 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 2.948861532, 2020-08-07T15:06:41.230 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.102218534, 2020-08-07T15:06:41.230 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:41.231 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:41.231 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:41.231 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:41.231 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:41.232 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 2944 2020-08-07T15:06:41.232 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.232 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:41.233 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.233 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:41.233 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.017796+0000", 2020-08-07T15:06:41.233 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:41.233 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.234 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.234 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:41.234 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.017796+0000", 2020-08-07T15:06:41.234 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4678999999999997e-05 2020-08-07T15:06:41.234 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.235 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.235 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:41.235 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.017840+0000", 2020-08-07T15:06:41.235 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.535e-06 2020-08-07T15:06:41.235 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.236 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.236 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:41.236 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.017844+0000", 2020-08-07T15:06:41.236 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4690000000000002e-06 2020-08-07T15:06:41.236 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.237 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.237 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:41.237 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.017850+0000", 2020-08-07T15:06:41.237 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.287e-06 2020-08-07T15:06:41.237 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.238 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.238 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:41.238 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.017854+0000", 2020-08-07T15:06:41.238 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0014247280000000001 2020-08-07T15:06:41.238 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.239 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.239 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:41.239 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.019278+0000", 2020-08-07T15:06:41.239 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6663999999999996e-05 2020-08-07T15:06:41.239 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.240 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.240 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:41.240 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.019355+0000", 2020-08-07T15:06:41.240 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000101704 2020-08-07T15:06:41.240 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.240 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.241 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:41.241 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.019457+0000", 2020-08-07T15:06:41.241 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0016716050000000001 2020-08-07T15:06:41.241 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.241 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.242 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:41.242 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.021128+0000", 2020-08-07T15:06:41.243 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9399999999999993e-07 2020-08-07T15:06:41.243 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.243 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.243 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:41.243 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.021129+0000", 2020-08-07T15:06:41.244 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1882000000000001e-05 2020-08-07T15:06:41.244 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.244 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.244 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:41.244 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.021201+0000", 2020-08-07T15:06:41.245 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0081014329999999999 2020-08-07T15:06:41.245 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.245 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.245 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.245 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.029303+0000", 2020-08-07T15:06:41.246 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.090647648999999997 2020-08-07T15:06:41.246 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.246 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.246 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.246 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.119950+0000", 2020-08-07T15:06:41.247 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2099999999999995e-07 2020-08-07T15:06:41.247 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.247 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.247 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:41.247 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.119951+0000", 2020-08-07T15:06:41.248 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2984000000000005e-05 2020-08-07T15:06:41.248 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.248 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.248 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:41.248 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.120014+0000", 2020-08-07T15:06:41.248 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.102218534 2020-08-07T15:06:41.249 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.249 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:41.249 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.249 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.249 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.250 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:7398 3.8 e24/20 ondisk, result = 0)", 2020-08-07T15:06:41.250 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.654005+0000", 2020-08-07T15:06:41.250 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.31265194000000002, 2020-08-07T15:06:41.250 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.095649112999999994, 2020-08-07T15:06:41.250 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:41.251 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:06:41.251 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:41.251 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.251 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:41.251 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.654005+0000", 2020-08-07T15:06:41.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:41.252 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.252 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.253 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:41.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.654005+0000", 2020-08-07T15:06:41.253 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8331e-05 2020-08-07T15:06:41.253 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.254 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.254 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:41.255 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.654033+0000", 2020-08-07T15:06:41.255 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8590000000000004e-06 2020-08-07T15:06:41.255 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.256 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.256 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:41.256 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.654038+0000", 2020-08-07T15:06:41.256 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.094344800000000006 2020-08-07T15:06:41.256 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.257 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.257 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:41.257 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.748383+0000", 2020-08-07T15:06:41.257 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9009999999999999e-06 2020-08-07T15:06:41.257 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.257 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.258 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:41.258 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.748389+0000", 2020-08-07T15:06:41.258 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001168346 2020-08-07T15:06:41.258 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.258 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.259 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:41.259 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.749557+0000", 2020-08-07T15:06:41.259 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9310000000000008e-06 2020-08-07T15:06:41.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.259 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.260 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:41.260 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.749565+0000", 2020-08-07T15:06:41.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8944999999999996e-05 2020-08-07T15:06:41.260 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.260 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.261 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:41.261 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.749654+0000", 2020-08-07T15:06:41.261 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.095649112999999994 2020-08-07T15:06:41.261 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.261 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:41.262 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.262 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.262 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.262 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:7576 3.b 3:d854a547:::benchmark_data_smithi074_35447_object946:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:41.262 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.736458+0000", 2020-08-07T15:06:41.263 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.23019909099999999, 2020-08-07T15:06:41.263 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19357406199999999, 2020-08-07T15:06:41.263 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:41.264 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:41.264 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:41.264 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:41.264 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:41.264 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7576 2020-08-07T15:06:41.265 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.265 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:41.265 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.265 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:41.265 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.736458+0000", 2020-08-07T15:06:41.266 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:41.266 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.266 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.266 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:41.266 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.736458+0000", 2020-08-07T15:06:41.267 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2197999999999997e-05 2020-08-07T15:06:41.267 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.267 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.267 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:41.267 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.736510+0000", 2020-08-07T15:06:41.268 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2579999999999995e-06 2020-08-07T15:06:41.268 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.268 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.268 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:41.268 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.736518+0000", 2020-08-07T15:06:41.269 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3842999999999999e-05 2020-08-07T15:06:41.269 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.269 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.269 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:41.269 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.736532+0000", 2020-08-07T15:06:41.269 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0379999999999996e-06 2020-08-07T15:06:41.270 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.270 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.270 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:41.270 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.736538+0000", 2020-08-07T15:06:41.270 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044985529999999998 2020-08-07T15:06:41.271 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.271 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:41.271 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.741037+0000", 2020-08-07T15:06:41.271 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6891000000000005e-05 2020-08-07T15:06:41.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:41.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.741134+0000", 2020-08-07T15:06:41.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000198555 2020-08-07T15:06:41.273 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.273 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:41.273 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.741332+0000", 2020-08-07T15:06:41.274 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028259190000000001 2020-08-07T15:06:41.274 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.274 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.274 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:41.275 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.744158+0000", 2020-08-07T15:06:41.275 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1100000000000001e-07 2020-08-07T15:06:41.275 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.275 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.275 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:41.276 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.744159+0000", 2020-08-07T15:06:41.276 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1880999999999998e-05 2020-08-07T15:06:41.276 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.276 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.276 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:41.277 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.744241+0000", 2020-08-07T15:06:41.277 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7608000000000002e-05 2020-08-07T15:06:41.277 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.277 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.277 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.278 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.744288+0000", 2020-08-07T15:06:41.278 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18566353199999999 2020-08-07T15:06:41.278 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.278 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.278 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.279 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.929952+0000", 2020-08-07T15:06:41.279 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.02e-06 2020-08-07T15:06:41.279 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.279 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.279 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:41.280 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.929953+0000", 2020-08-07T15:06:41.280 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9355000000000002e-05 2020-08-07T15:06:41.280 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.280 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.280 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:41.280 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.930032+0000", 2020-08-07T15:06:41.281 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19357406199999999 2020-08-07T15:06:41.281 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.281 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:41.281 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.281 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.282 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.282 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:7598 3.8 3:11c6b9be:::benchmark_data_smithi074_35447_object949:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:41.282 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.747877+0000", 2020-08-07T15:06:41.282 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.21877988900000001, 2020-08-07T15:06:41.282 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.095763631000000002, 2020-08-07T15:06:41.283 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:41.283 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:41.283 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:41.283 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:41.283 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:41.284 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7598 2020-08-07T15:06:41.284 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.284 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:41.285 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.285 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:41.285 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.747877+0000", 2020-08-07T15:06:41.285 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:41.285 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.286 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.286 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:41.286 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.747877+0000", 2020-08-07T15:06:41.286 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7117000000000002e-05 2020-08-07T15:06:41.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.287 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:41.287 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.747934+0000", 2020-08-07T15:06:41.288 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.579e-06 2020-08-07T15:06:41.288 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.288 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.288 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:41.288 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.747941+0000", 2020-08-07T15:06:41.289 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1763000000000001e-05 2020-08-07T15:06:41.289 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.289 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.289 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:41.290 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.747953+0000", 2020-08-07T15:06:41.290 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5550000000000002e-06 2020-08-07T15:06:41.290 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.290 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.290 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:41.291 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.747958+0000", 2020-08-07T15:06:41.291 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000936807 2020-08-07T15:06:41.291 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.291 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.291 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:41.292 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.748895+0000", 2020-08-07T15:06:41.292 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010639199999999999 2020-08-07T15:06:41.292 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.292 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.292 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:41.293 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.749001+0000", 2020-08-07T15:06:41.293 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018759899999999999 2020-08-07T15:06:41.293 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.293 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.294 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:06:41.294 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.749189+0000", 2020-08-07T15:06:41.294 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0036272230000000002 2020-08-07T15:06:41.295 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.295 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.295 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:41.295 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.752816+0000", 2020-08-07T15:06:41.296 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4799999999999998e-07 2020-08-07T15:06:41.296 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.296 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.296 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:41.296 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.752817+0000", 2020-08-07T15:06:41.297 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3554999999999998e-05 2020-08-07T15:06:41.297 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.297 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.297 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:41.297 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.752870+0000", 2020-08-07T15:06:41.298 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0027157180000000002 2020-08-07T15:06:41.298 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.298 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.298 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.298 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.755586+0000", 2020-08-07T15:06:41.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.087977048000000002 2020-08-07T15:06:41.299 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.299 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.299 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.299 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.843563+0000", 2020-08-07T15:06:41.300 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.313e-06 2020-08-07T15:06:41.300 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.300 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.300 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:41.300 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.843564+0000", 2020-08-07T15:06:41.301 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6414000000000004e-05 2020-08-07T15:06:41.301 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.301 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.301 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:41.301 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.843641+0000", 2020-08-07T15:06:41.302 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.095763631000000002 2020-08-07T15:06:41.302 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.302 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:41.302 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.302 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.303 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.303 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:7599 3.8 3:11c6b9be:::benchmark_data_smithi074_35447_object949:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:41.303 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.747995+0000", 2020-08-07T15:06:41.303 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.21866208600000001, 2020-08-07T15:06:41.303 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.096785070000000001, 2020-08-07T15:06:41.304 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:41.304 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:41.304 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:41.304 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:41.304 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:41.305 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7599 2020-08-07T15:06:41.305 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.306 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:41.306 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.306 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:41.306 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.747995+0000", 2020-08-07T15:06:41.306 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:41.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.307 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.307 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:41.307 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.747995+0000", 2020-08-07T15:06:41.307 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2102999999999998e-05 2020-08-07T15:06:41.308 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.308 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:41.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.748057+0000", 2020-08-07T15:06:41.308 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7030000000000003e-06 2020-08-07T15:06:41.308 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.309 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.309 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:41.309 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.748063+0000", 2020-08-07T15:06:41.309 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6877999999999999e-05 2020-08-07T15:06:41.309 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.310 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.310 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:41.310 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.748090+0000", 2020-08-07T15:06:41.310 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0540000000000001e-05 2020-08-07T15:06:41.310 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.311 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.311 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:41.311 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.748100+0000", 2020-08-07T15:06:41.311 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019153600000000001 2020-08-07T15:06:41.311 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.312 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.312 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:41.312 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.750016+0000", 2020-08-07T15:06:41.312 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010135499999999999 2020-08-07T15:06:41.312 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.313 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.313 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:41.313 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.750117+0000", 2020-08-07T15:06:41.313 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019746799999999999 2020-08-07T15:06:41.313 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.314 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.314 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:06:41.314 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.750314+0000", 2020-08-07T15:06:41.314 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0025636159999999999 2020-08-07T15:06:41.314 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.314 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.315 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:41.315 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.752878+0000", 2020-08-07T15:06:41.315 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9599999999999999e-07 2020-08-07T15:06:41.316 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.316 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.316 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:41.316 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.752879+0000", 2020-08-07T15:06:41.317 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.982e-05 2020-08-07T15:06:41.317 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.317 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.317 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:41.317 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.752938+0000", 2020-08-07T15:06:41.318 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0027479129999999998 2020-08-07T15:06:41.318 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.318 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.319 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.319 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.755686+0000", 2020-08-07T15:06:41.319 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.089012944999999996 2020-08-07T15:06:41.319 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.320 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.320 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.320 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.844699+0000", 2020-08-07T15:06:41.320 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0800000000000003e-07 2020-08-07T15:06:41.320 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.321 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.321 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:41.321 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.844700+0000", 2020-08-07T15:06:41.321 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9864999999999995e-05 2020-08-07T15:06:41.321 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.322 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.322 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:41.322 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.844780+0000", 2020-08-07T15:06:41.322 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.096785070000000001 2020-08-07T15:06:41.322 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.322 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:41.323 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.323 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.323 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.323 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:7600 3.8 3:11c6b9be:::benchmark_data_smithi074_35447_object949:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:41.323 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.748143+0000", 2020-08-07T15:06:41.324 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.21851378199999999, 2020-08-07T15:06:41.324 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.102592691, 2020-08-07T15:06:41.324 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:41.324 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:41.324 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:41.325 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:41.325 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:41.325 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7600 2020-08-07T15:06:41.325 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.326 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:41.326 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.327 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:41.327 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.748143+0000", 2020-08-07T15:06:41.327 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:41.327 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.328 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.328 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:41.328 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.748143+0000", 2020-08-07T15:06:41.328 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3086e-05 2020-08-07T15:06:41.328 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.328 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.329 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:41.329 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.748196+0000", 2020-08-07T15:06:41.329 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.477e-06 2020-08-07T15:06:41.329 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.329 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.330 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:41.330 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.748203+0000", 2020-08-07T15:06:41.330 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2943e-05 2020-08-07T15:06:41.330 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.330 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.331 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:41.331 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.748216+0000", 2020-08-07T15:06:41.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4269999999999999e-06 2020-08-07T15:06:41.331 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.331 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.332 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:41.332 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.748221+0000", 2020-08-07T15:06:41.332 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002591627 2020-08-07T15:06:41.332 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.332 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.333 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:41.333 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.750813+0000", 2020-08-07T15:06:41.333 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000104891 2020-08-07T15:06:41.333 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.333 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.334 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:41.334 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.750918+0000", 2020-08-07T15:06:41.334 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018150299999999999 2020-08-07T15:06:41.334 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.334 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.335 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:06:41.335 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.751099+0000", 2020-08-07T15:06:41.335 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001847125 2020-08-07T15:06:41.335 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.336 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:41.336 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.752946+0000", 2020-08-07T15:06:41.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000005e-07 2020-08-07T15:06:41.336 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.337 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.337 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:41.337 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.752947+0000", 2020-08-07T15:06:41.337 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8338999999999998e-05 2020-08-07T15:06:41.338 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.338 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.338 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:41.338 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.753005+0000", 2020-08-07T15:06:41.341 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028934939999999999 2020-08-07T15:06:41.341 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.342 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.343 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.755899+0000", 2020-08-07T15:06:41.343 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.094765409999999994 2020-08-07T15:06:41.343 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.343 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.343 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.344 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.850664+0000", 2020-08-07T15:06:41.344 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3600000000000003e-07 2020-08-07T15:06:41.344 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.344 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.344 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:41.345 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.850665+0000", 2020-08-07T15:06:41.345 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1131999999999996e-05 2020-08-07T15:06:41.345 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.345 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.345 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:41.346 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.850736+0000", 2020-08-07T15:06:41.346 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.102592691 2020-08-07T15:06:41.346 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.346 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:41.346 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.347 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.347 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:7598 3.8 e24/20 ondisk, result = 0)", 2020-08-07T15:06:41.347 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.753135+0000", 2020-08-07T15:06:41.347 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.213521602, 2020-08-07T15:06:41.348 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.090511443999999996, 2020-08-07T15:06:41.348 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:41.348 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:06:41.348 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:41.348 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.348 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:41.349 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.753135+0000", 2020-08-07T15:06:41.349 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:41.349 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.349 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.349 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:41.350 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.753135+0000", 2020-08-07T15:06:41.350 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3821e-05 2020-08-07T15:06:41.350 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.350 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.350 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:41.351 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.753169+0000", 2020-08-07T15:06:41.351 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5139999999999998e-06 2020-08-07T15:06:41.351 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.351 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.351 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:41.352 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.753176+0000", 2020-08-07T15:06:41.352 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.089899006000000004 2020-08-07T15:06:41.352 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.353 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.353 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:41.353 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.843075+0000", 2020-08-07T15:06:41.353 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.229e-06 2020-08-07T15:06:41.354 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.354 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.354 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:41.354 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.843081+0000", 2020-08-07T15:06:41.354 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00046134099999999999 2020-08-07T15:06:41.355 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.355 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.355 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:41.355 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.843542+0000", 2020-08-07T15:06:41.355 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3149999999999998e-06 2020-08-07T15:06:41.356 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.356 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.356 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:41.356 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.843552+0000", 2020-08-07T15:06:41.356 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5217999999999995e-05 2020-08-07T15:06:41.357 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.357 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.357 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:41.357 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.843647+0000", 2020-08-07T15:06:41.357 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.090511443999999996 2020-08-07T15:06:41.358 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.358 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:41.358 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.358 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.358 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.358 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:7696 3.7 3:ee5cfea7:::benchmark_data_smithi074_35447_object961:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:41.359 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.806620+0000", 2020-08-07T15:06:41.359 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.160037544, 2020-08-07T15:06:41.359 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.12604768499999999, 2020-08-07T15:06:41.359 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:41.359 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:41.360 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:41.360 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:41.360 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:41.360 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7696 2020-08-07T15:06:41.360 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.361 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:41.361 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.361 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:41.361 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.806620+0000", 2020-08-07T15:06:41.361 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:41.362 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.362 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.362 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:41.363 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.806620+0000", 2020-08-07T15:06:41.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0121000000000003e-05 2020-08-07T15:06:41.363 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:41.364 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.806680+0000", 2020-08-07T15:06:41.364 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.024e-06 2020-08-07T15:06:41.364 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.365 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.365 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:41.365 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.806687+0000", 2020-08-07T15:06:41.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2645e-05 2020-08-07T15:06:41.365 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.366 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:41.366 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.806699+0000", 2020-08-07T15:06:41.366 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6299999999999993e-06 2020-08-07T15:06:41.366 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.367 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.367 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:41.367 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.806709+0000", 2020-08-07T15:06:41.367 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6279000000000004e-05 2020-08-07T15:06:41.367 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.368 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.368 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:41.368 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.806805+0000", 2020-08-07T15:06:41.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010478499999999999 2020-08-07T15:06:41.368 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.369 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:41.369 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.806910+0000", 2020-08-07T15:06:41.369 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018237799999999999 2020-08-07T15:06:41.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.370 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.370 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:06:41.370 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.807092+0000", 2020-08-07T15:06:41.370 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0037732249999999998 2020-08-07T15:06:41.370 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.371 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.371 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:41.371 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.810866+0000", 2020-08-07T15:06:41.371 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7499999999999999e-07 2020-08-07T15:06:41.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.371 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.372 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:41.372 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.810866+0000", 2020-08-07T15:06:41.372 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7073000000000001e-05 2020-08-07T15:06:41.372 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.373 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:41.373 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.810923+0000", 2020-08-07T15:06:41.373 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.029912563999999999 2020-08-07T15:06:41.374 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.374 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.375 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.375 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.840836+0000", 2020-08-07T15:06:41.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.091729725999999998 2020-08-07T15:06:41.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.376 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:41.376 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.932566+0000", 2020-08-07T15:06:41.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3900000000000003e-07 2020-08-07T15:06:41.377 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:41.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.932567+0000", 2020-08-07T15:06:41.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000100521 2020-08-07T15:06:41.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:41.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:41.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:41.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.932667+0000", 2020-08-07T15:06:41.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.12604768499999999 2020-08-07T15:06:41.379 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.379 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:41.379 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.379 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:41.380 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:41.380 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:06:41.382 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:06:41.382 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 5 16 8310 8294 12.9582 12.4062 0.113391 0.00959681 2020-08-07T15:06:41.383 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:41.380+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:41.484 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:41.484+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:41.551 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:06:41.552 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:06:41.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:41.552 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:8794 3.b e24/20 3:d72f0dc8:::benchmark_data_smithi074_35447_object1099:head v 24'522, mlcod=24'519)", 2020-08-07T15:06:41.552 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.550673+0000", 2020-08-07T15:06:41.552 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00029794700000000001, 2020-08-07T15:06:41.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031360300000000003, 2020-08-07T15:06:41.553 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:41.553 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:41.553 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:41.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:41.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:41.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.550673+0000", 2020-08-07T15:06:41.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:41.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:41.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:41.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:41.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.550673+0000", 2020-08-07T15:06:41.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1891999999999997e-05 2020-08-07T15:06:41.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:41.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:41.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:41.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.550725+0000", 2020-08-07T15:06:41.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7510000000000008e-06 2020-08-07T15:06:41.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:41.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:41.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:41.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.550733+0000", 2020-08-07T15:06:41.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5275000000000003e-05 2020-08-07T15:06:41.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:41.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:41.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:41.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.550778+0000", 2020-08-07T15:06:41.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.693e-06 2020-08-07T15:06:41.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:41.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:41.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:41.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.550782+0000", 2020-08-07T15:06:41.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0809999999999997e-05 2020-08-07T15:06:41.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:41.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:41.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:41.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.550822+0000", 2020-08-07T15:06:41.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5466999999999999e-05 2020-08-07T15:06:41.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:41.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:41.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:41.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.550848+0000", 2020-08-07T15:06:41.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000174888 2020-08-07T15:06:41.561 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:41.561 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:41.561 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:41.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:41.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:41.562 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:8760 3.9 3:9f02d263:::benchmark_data_smithi074_35447_object1094:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:41.562 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.531357+0000", 2020-08-07T15:06:41.562 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.019613766000000001, 2020-08-07T15:06:41.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019694613999999999, 2020-08-07T15:06:41.563 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:41.563 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:41.563 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:41.563 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:41.563 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:41.564 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 8760 2020-08-07T15:06:41.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:41.564 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:41.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:41.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:41.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.531357+0000", 2020-08-07T15:06:41.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:41.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:41.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:41.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:41.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.531357+0000", 2020-08-07T15:06:41.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9942999999999999e-05 2020-08-07T15:06:41.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:41.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:41.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:41.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.531407+0000", 2020-08-07T15:06:41.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8319999999999996e-06 2020-08-07T15:06:41.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:41.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:41.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:41.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.531412+0000", 2020-08-07T15:06:41.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0128e-05 2020-08-07T15:06:41.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:41.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:41.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:41.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.531422+0000", 2020-08-07T15:06:41.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2370000000000003e-06 2020-08-07T15:06:41.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:41.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:41.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:41.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.531426+0000", 2020-08-07T15:06:41.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034586119999999998 2020-08-07T15:06:41.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:41.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:41.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:41.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.534885+0000", 2020-08-07T15:06:41.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6256999999999997e-05 2020-08-07T15:06:41.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:41.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:41.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:41.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.534981+0000", 2020-08-07T15:06:41.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000171373 2020-08-07T15:06:41.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:41.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:41.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:06:41.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.535153+0000", 2020-08-07T15:06:41.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018094879999999999 2020-08-07T15:06:41.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:41.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:41.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:41.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.536962+0000", 2020-08-07T15:06:41.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9400000000000005e-07 2020-08-07T15:06:41.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:41.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:41.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:41.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.536963+0000", 2020-08-07T15:06:41.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7188999999999997e-05 2020-08-07T15:06:41.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:41.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:41.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:41.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.537030+0000", 2020-08-07T15:06:41.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005672753 2020-08-07T15:06:41.576 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:41.577 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:41.577 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:41.577 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:41.577 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:06:41.577 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 2 2020-08-07T15:06:41.578 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:06:41.579 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:06:41.625 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:41.624+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:41.688 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:41.686+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:41.752 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:06:41.752 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:06:41.752 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:06:41.752 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:06:41.753 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:06:41.781 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:06:41.800 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:41.799+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:41.903 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:41.902+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:41.998 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:06:41.998 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:06:41.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:06:41.999 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:06:42.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.004 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:544 3.b e22/20 3:ddefb1a6:::benchmark_data_smithi074_35447_object67:head v 22'48, mlcod=22'42)", 2020-08-07T15:06:42.004 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:36.739068+0000", 2020-08-07T15:06:42.004 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.2546701540000003, 2020-08-07T15:06:42.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13652066299999999, 2020-08-07T15:06:42.005 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.005 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:42.005 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.739068+0000", 2020-08-07T15:06:42.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.739068+0000", 2020-08-07T15:06:42.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8701999999999994e-05 2020-08-07T15:06:42.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.739137+0000", 2020-08-07T15:06:42.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3130000000000004e-06 2020-08-07T15:06:42.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.739143+0000", 2020-08-07T15:06:42.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12951985099999999 2020-08-07T15:06:42.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.868663+0000", 2020-08-07T15:06:42.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8010000000000001e-06 2020-08-07T15:06:42.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.868667+0000", 2020-08-07T15:06:42.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5367000000000002e-05 2020-08-07T15:06:42.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:42.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.868702+0000", 2020-08-07T15:06:42.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6690000000000001e-05 2020-08-07T15:06:42.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:42.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.868719+0000", 2020-08-07T15:06:42.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0068076669999999999 2020-08-07T15:06:42.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:42.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.875526+0000", 2020-08-07T15:06:42.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2272000000000003e-05 2020-08-07T15:06:42.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:42.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.875589+0000", 2020-08-07T15:06:42.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13652066299999999 2020-08-07T15:06:42.060 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.060 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.060 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.061 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:1024 3.b e22/20 3:dab1244e:::benchmark_data_smithi074_35447_object127:head v 22'96, mlcod=22'87)", 2020-08-07T15:06:42.061 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:36.989664+0000", 2020-08-07T15:06:42.062 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.0040745839999996, 2020-08-07T15:06:42.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.092481192000000004, 2020-08-07T15:06:42.062 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.063 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:42.064 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.989664+0000", 2020-08-07T15:06:42.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.989664+0000", 2020-08-07T15:06:42.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1632000000000003e-05 2020-08-07T15:06:42.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.989715+0000", 2020-08-07T15:06:42.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5421999999999999e-05 2020-08-07T15:06:42.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:36.989731+0000", 2020-08-07T15:06:42.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090445908000000005 2020-08-07T15:06:42.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.080177+0000", 2020-08-07T15:06:42.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1340000000000003e-06 2020-08-07T15:06:42.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.080182+0000", 2020-08-07T15:06:42.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7679000000000003e-05 2020-08-07T15:06:42.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:42.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.080219+0000", 2020-08-07T15:06:42.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2036e-05 2020-08-07T15:06:42.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:42.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.080241+0000", 2020-08-07T15:06:42.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018566240000000001 2020-08-07T15:06:42.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:42.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.082098+0000", 2020-08-07T15:06:42.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6757e-05 2020-08-07T15:06:42.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:42.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.082145+0000", 2020-08-07T15:06:42.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.092481192000000004 2020-08-07T15:06:42.083 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.084 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.084 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.084 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:1208 3.b e22/20 3:da696a66:::benchmark_data_smithi074_35447_object150:head v 22'104, mlcod=22'95)", 2020-08-07T15:06:42.085 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.085335+0000", 2020-08-07T15:06:42.085 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.908403217, 2020-08-07T15:06:42.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.086797795999999997, 2020-08-07T15:06:42.086 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.086 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:42.086 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.085335+0000", 2020-08-07T15:06:42.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.085335+0000", 2020-08-07T15:06:42.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9539e-05 2020-08-07T15:06:42.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.085364+0000", 2020-08-07T15:06:42.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7299999999999999e-06 2020-08-07T15:06:42.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.085368+0000", 2020-08-07T15:06:42.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.083361272 2020-08-07T15:06:42.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.168729+0000", 2020-08-07T15:06:42.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4070000000000001e-06 2020-08-07T15:06:42.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.168735+0000", 2020-08-07T15:06:42.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2438999999999998e-05 2020-08-07T15:06:42.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:42.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.168777+0000", 2020-08-07T15:06:42.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2082e-05 2020-08-07T15:06:42.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:42.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.168799+0000", 2020-08-07T15:06:42.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032803649999999999 2020-08-07T15:06:42.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:42.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.172080+0000", 2020-08-07T15:06:42.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2961999999999999e-05 2020-08-07T15:06:42.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:42.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.172133+0000", 2020-08-07T15:06:42.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.086797795999999997 2020-08-07T15:06:42.097 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.097 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.097 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.098 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:1392 3.b e23/20 3:d2605baa:::benchmark_data_smithi074_35447_object173:head v 23'112, mlcod=22'103)", 2020-08-07T15:06:42.098 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.174536+0000", 2020-08-07T15:06:42.098 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8192019190000002, 2020-08-07T15:06:42.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.061286912999999998, 2020-08-07T15:06:42.098 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.099 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:42.099 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.174536+0000", 2020-08-07T15:06:42.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.174536+0000", 2020-08-07T15:06:42.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4691999999999998e-05 2020-08-07T15:06:42.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.174581+0000", 2020-08-07T15:06:42.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9710000000000001e-06 2020-08-07T15:06:42.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.174587+0000", 2020-08-07T15:06:42.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.059339600999999999 2020-08-07T15:06:42.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.233926+0000", 2020-08-07T15:06:42.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0659999999999998e-06 2020-08-07T15:06:42.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.233934+0000", 2020-08-07T15:06:42.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9688999999999998e-05 2020-08-07T15:06:42.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:42.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.233983+0000", 2020-08-07T15:06:42.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9505e-05 2020-08-07T15:06:42.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:42.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.234013+0000", 2020-08-07T15:06:42.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001775509 2020-08-07T15:06:42.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:42.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.235788+0000", 2020-08-07T15:06:42.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999998e-05 2020-08-07T15:06:42.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:42.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.235823+0000", 2020-08-07T15:06:42.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.061286912999999998 2020-08-07T15:06:42.109 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.110 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.110 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.111 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:1912 3.3 3:c59aafef:::benchmark_data_smithi074_35447_object238:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:42.111 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.446603+0000", 2020-08-07T15:06:42.111 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.5471349679999999, 2020-08-07T15:06:42.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.074237148000000003, 2020-08-07T15:06:42.116 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.116 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:42.116 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:42.116 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:42.118 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:42.118 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1912 2020-08-07T15:06:42.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.119 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.446603+0000", 2020-08-07T15:06:42.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.446603+0000", 2020-08-07T15:06:42.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7880999999999998e-05 2020-08-07T15:06:42.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.446651+0000", 2020-08-07T15:06:42.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1619999999999999e-06 2020-08-07T15:06:42.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.446656+0000", 2020-08-07T15:06:42.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0563e-05 2020-08-07T15:06:42.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.446667+0000", 2020-08-07T15:06:42.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2209999999999999e-06 2020-08-07T15:06:42.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.446671+0000", 2020-08-07T15:06:42.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039131189999999996 2020-08-07T15:06:42.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:42.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.450584+0000", 2020-08-07T15:06:42.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3996000000000002e-05 2020-08-07T15:06:42.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:42.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.450648+0000", 2020-08-07T15:06:42.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000188646 2020-08-07T15:06:42.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,6", 2020-08-07T15:06:42.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.450837+0000", 2020-08-07T15:06:42.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001377599 2020-08-07T15:06:42.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:42.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.452214+0000", 2020-08-07T15:06:42.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3300000000000003e-07 2020-08-07T15:06:42.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:42.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.452215+0000", 2020-08-07T15:06:42.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2660000000000002e-05 2020-08-07T15:06:42.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:42.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.452247+0000", 2020-08-07T15:06:42.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0079174229999999998 2020-08-07T15:06:42.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:42.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.460165+0000", 2020-08-07T15:06:42.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.060615417999999997 2020-08-07T15:06:42.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:42.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.520780+0000", 2020-08-07T15:06:42.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2559999999999999e-06 2020-08-07T15:06:42.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:42.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.520782+0000", 2020-08-07T15:06:42.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8770999999999997e-05 2020-08-07T15:06:42.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:42.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.520840+0000", 2020-08-07T15:06:42.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.074237148000000003 2020-08-07T15:06:42.141 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.142 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.142 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.144 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:2343 3.3 3:ceba440c:::benchmark_data_smithi074_35447_object292:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:42.144 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.664937+0000", 2020-08-07T15:06:42.145 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.3288008759999999, 2020-08-07T15:06:42.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.073195416999999999, 2020-08-07T15:06:42.145 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.146 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:42.146 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:42.146 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:42.146 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:42.147 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2343 2020-08-07T15:06:42.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.147 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.664937+0000", 2020-08-07T15:06:42.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.664937+0000", 2020-08-07T15:06:42.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5846e-05 2020-08-07T15:06:42.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.664993+0000", 2020-08-07T15:06:42.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2589999999999997e-06 2020-08-07T15:06:42.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.664999+0000", 2020-08-07T15:06:42.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1824000000000001e-05 2020-08-07T15:06:42.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.665011+0000", 2020-08-07T15:06:42.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3266e-05 2020-08-07T15:06:42.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.665024+0000", 2020-08-07T15:06:42.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013429749999999999 2020-08-07T15:06:42.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:42.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.666367+0000", 2020-08-07T15:06:42.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2276999999999998e-05 2020-08-07T15:06:42.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:42.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.666460+0000", 2020-08-07T15:06:42.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000161037 2020-08-07T15:06:42.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,6", 2020-08-07T15:06:42.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.666621+0000", 2020-08-07T15:06:42.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016089139999999999 2020-08-07T15:06:42.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:42.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.668230+0000", 2020-08-07T15:06:42.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4099999999999999e-07 2020-08-07T15:06:42.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:42.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.668230+0000", 2020-08-07T15:06:42.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0176000000000002e-05 2020-08-07T15:06:42.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:42.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.668280+0000", 2020-08-07T15:06:42.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016096452000000001 2020-08-07T15:06:42.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:42.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.684377+0000", 2020-08-07T15:06:42.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.053675874999999998 2020-08-07T15:06:42.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:42.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.738053+0000", 2020-08-07T15:06:42.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.488e-06 2020-08-07T15:06:42.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:42.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.738054+0000", 2020-08-07T15:06:42.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8486999999999996e-05 2020-08-07T15:06:42.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:42.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.738133+0000", 2020-08-07T15:06:42.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.073195416999999999 2020-08-07T15:06:42.166 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.166 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.166 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.167 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:2344 3.3 3:ceba440c:::benchmark_data_smithi074_35447_object292:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:42.167 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.665045+0000", 2020-08-07T15:06:42.167 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.3286928409999996, 2020-08-07T15:06:42.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.096725685000000006, 2020-08-07T15:06:42.168 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.168 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:42.168 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:42.168 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:42.169 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:42.169 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2344 2020-08-07T15:06:42.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.170 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.665045+0000", 2020-08-07T15:06:42.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.665045+0000", 2020-08-07T15:06:42.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3442999999999998e-05 2020-08-07T15:06:42.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.665079+0000", 2020-08-07T15:06:42.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.045e-06 2020-08-07T15:06:42.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.665084+0000", 2020-08-07T15:06:42.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0100000000000001e-06 2020-08-07T15:06:42.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.665090+0000", 2020-08-07T15:06:42.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1030000000000002e-06 2020-08-07T15:06:42.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.665093+0000", 2020-08-07T15:06:42.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018961049999999999 2020-08-07T15:06:42.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:42.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.666989+0000", 2020-08-07T15:06:42.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3561999999999996e-05 2020-08-07T15:06:42.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:42.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.667083+0000", 2020-08-07T15:06:42.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000160836 2020-08-07T15:06:42.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,6", 2020-08-07T15:06:42.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.667243+0000", 2020-08-07T15:06:42.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018500540000000001 2020-08-07T15:06:42.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:42.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.669093+0000", 2020-08-07T15:06:42.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.122e-06 2020-08-07T15:06:42.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:42.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.669095+0000", 2020-08-07T15:06:42.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4412000000000006e-05 2020-08-07T15:06:42.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:42.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.669159+0000", 2020-08-07T15:06:42.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018183589 2020-08-07T15:06:42.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:42.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.687343+0000", 2020-08-07T15:06:42.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.074364071000000004 2020-08-07T15:06:42.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:42.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.761707+0000", 2020-08-07T15:06:42.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0729999999999999e-06 2020-08-07T15:06:42.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:42.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.761708+0000", 2020-08-07T15:06:42.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3260000000000001e-05 2020-08-07T15:06:42.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:42.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.761771+0000", 2020-08-07T15:06:42.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.096725685000000006 2020-08-07T15:06:42.188 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.188 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.189 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.189 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:2343 3.3 e23/20 ondisk, result = 0)", 2020-08-07T15:06:42.189 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.668625+0000", 2020-08-07T15:06:42.190 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.3251128960000003, 2020-08-07T15:06:42.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.069514279999999998, 2020-08-07T15:06:42.190 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.190 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:42.190 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.668625+0000", 2020-08-07T15:06:42.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.668625+0000", 2020-08-07T15:06:42.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1260000000000002e-05 2020-08-07T15:06:42.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.668656+0000", 2020-08-07T15:06:42.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8989999999999998e-06 2020-08-07T15:06:42.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.668662+0000", 2020-08-07T15:06:42.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.069288237000000003 2020-08-07T15:06:42.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.737951+0000", 2020-08-07T15:06:42.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2349999999999999e-06 2020-08-07T15:06:42.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.737959+0000", 2020-08-07T15:06:42.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4000999999999998e-05 2020-08-07T15:06:42.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:42.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.738033+0000", 2020-08-07T15:06:42.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0740999999999999e-05 2020-08-07T15:06:42.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:42.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.738044+0000", 2020-08-07T15:06:42.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5907000000000002e-05 2020-08-07T15:06:42.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:42.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.738140+0000", 2020-08-07T15:06:42.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.069514279999999998 2020-08-07T15:06:42.200 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.200 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.200 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.201 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:2600 3.b e23/20 3:d6a1a0d3:::benchmark_data_smithi074_35447_object324:head v 23'208, mlcod=23'201)", 2020-08-07T15:06:42.201 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.807190+0000", 2020-08-07T15:06:42.201 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.186548342, 2020-08-07T15:06:42.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20835313, 2020-08-07T15:06:42.202 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.202 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:42.202 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.807190+0000", 2020-08-07T15:06:42.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.807190+0000", 2020-08-07T15:06:42.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.938e-05 2020-08-07T15:06:42.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.807219+0000", 2020-08-07T15:06:42.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2780000000000004e-06 2020-08-07T15:06:42.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.807223+0000", 2020-08-07T15:06:42.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20663868699999999 2020-08-07T15:06:42.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.013862+0000", 2020-08-07T15:06:42.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.659e-06 2020-08-07T15:06:42.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.013866+0000", 2020-08-07T15:06:42.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7111999999999999e-05 2020-08-07T15:06:42.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:42.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.013913+0000", 2020-08-07T15:06:42.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5225999999999999e-05 2020-08-07T15:06:42.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:42.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.013938+0000", 2020-08-07T15:06:42.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015643429999999999 2020-08-07T15:06:42.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:42.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.015502+0000", 2020-08-07T15:06:42.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0445000000000003e-05 2020-08-07T15:06:42.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:42.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.015543+0000", 2020-08-07T15:06:42.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20835313 2020-08-07T15:06:42.213 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.213 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.214 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.214 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:2944 3.b e23/20 3:d7542c3f:::benchmark_data_smithi074_35447_object367:head v 23'216, mlcod=23'208)", 2020-08-07T15:06:42.214 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:38.019610+0000", 2020-08-07T15:06:42.215 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.9741283630000002, 2020-08-07T15:06:42.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.098428219999999997, 2020-08-07T15:06:42.215 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.215 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:42.215 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.019610+0000", 2020-08-07T15:06:42.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.019610+0000", 2020-08-07T15:06:42.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1959999999999999e-05 2020-08-07T15:06:42.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.019642+0000", 2020-08-07T15:06:42.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6379999999999998e-06 2020-08-07T15:06:42.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.019648+0000", 2020-08-07T15:06:42.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.097582494000000006 2020-08-07T15:06:42.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.117231+0000", 2020-08-07T15:06:42.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.215e-06 2020-08-07T15:06:42.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.117236+0000", 2020-08-07T15:06:42.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2833000000000001e-05 2020-08-07T15:06:42.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:42.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.117289+0000", 2020-08-07T15:06:42.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2370000000000001e-05 2020-08-07T15:06:42.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:42.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.117311+0000", 2020-08-07T15:06:42.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068160400000000004 2020-08-07T15:06:42.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:42.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.117993+0000", 2020-08-07T15:06:42.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5105999999999998e-05 2020-08-07T15:06:42.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:42.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.118038+0000", 2020-08-07T15:06:42.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.098428219999999997 2020-08-07T15:06:42.227 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.227 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.227 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.228 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:2968 3.f e23/20 3:f6aabc25:::benchmark_data_smithi074_35447_object370:head v 23'240, mlcod=23'231)", 2020-08-07T15:06:42.228 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:38.030627+0000", 2020-08-07T15:06:42.228 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.9631110380000001, 2020-08-07T15:06:42.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.076142292, 2020-08-07T15:06:42.229 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.229 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:42.229 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.030627+0000", 2020-08-07T15:06:42.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.030627+0000", 2020-08-07T15:06:42.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9036000000000003e-05 2020-08-07T15:06:42.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.030676+0000", 2020-08-07T15:06:42.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5299999999999999e-06 2020-08-07T15:06:42.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.030684+0000", 2020-08-07T15:06:42.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.070285136999999998 2020-08-07T15:06:42.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.100969+0000", 2020-08-07T15:06:42.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5537e-05 2020-08-07T15:06:42.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.100984+0000", 2020-08-07T15:06:42.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1665000000000002e-05 2020-08-07T15:06:42.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:42.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.101026+0000", 2020-08-07T15:06:42.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7548e-05 2020-08-07T15:06:42.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:42.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.101064+0000", 2020-08-07T15:06:42.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056594540000000004 2020-08-07T15:06:42.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:42.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.106723+0000", 2020-08-07T15:06:42.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6384999999999997e-05 2020-08-07T15:06:42.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:42.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.106769+0000", 2020-08-07T15:06:42.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.076142292 2020-08-07T15:06:42.250 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.250 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.250 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.251 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:3120 3.e e23/20 3:7937251e:::benchmark_data_smithi074_35447_object389:head v 23'232, mlcod=23'223)", 2020-08-07T15:06:42.251 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:38.114162+0000", 2020-08-07T15:06:42.251 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.8795760970000002, 2020-08-07T15:06:42.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063076737999999993, 2020-08-07T15:06:42.252 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.252 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:42.252 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.114162+0000", 2020-08-07T15:06:42.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.114162+0000", 2020-08-07T15:06:42.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4970999999999999e-05 2020-08-07T15:06:42.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.114217+0000", 2020-08-07T15:06:42.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.19e-06 2020-08-07T15:06:42.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.114223+0000", 2020-08-07T15:06:42.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062312412999999997 2020-08-07T15:06:42.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.176536+0000", 2020-08-07T15:06:42.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0690000000000004e-06 2020-08-07T15:06:42.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.176543+0000", 2020-08-07T15:06:42.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0448999999999998e-05 2020-08-07T15:06:42.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:42.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.176583+0000", 2020-08-07T15:06:42.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4091e-05 2020-08-07T15:06:42.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:42.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.176607+0000", 2020-08-07T15:06:42.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00059302499999999995 2020-08-07T15:06:42.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:42.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.177200+0000", 2020-08-07T15:06:42.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8529999999999999e-05 2020-08-07T15:06:42.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:42.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.177239+0000", 2020-08-07T15:06:42.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063076737999999993 2020-08-07T15:06:42.264 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.264 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.264 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.265 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:3136 3.b e23/20 3:d4088078:::benchmark_data_smithi074_35447_object391:head v 23'224, mlcod=23'215)", 2020-08-07T15:06:42.265 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:38.124710+0000", 2020-08-07T15:06:42.265 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.8690277559999999, 2020-08-07T15:06:42.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.080409670000000003, 2020-08-07T15:06:42.266 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.266 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:42.266 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.124710+0000", 2020-08-07T15:06:42.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.124710+0000", 2020-08-07T15:06:42.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9439000000000001e-05 2020-08-07T15:06:42.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.124740+0000", 2020-08-07T15:06:42.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2139999999999998e-06 2020-08-07T15:06:42.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.124744+0000", 2020-08-07T15:06:42.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.079412008000000006 2020-08-07T15:06:42.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.204156+0000", 2020-08-07T15:06:42.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4099999999999999e-06 2020-08-07T15:06:42.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.204161+0000", 2020-08-07T15:06:42.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6763e-05 2020-08-07T15:06:42.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:42.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.204198+0000", 2020-08-07T15:06:42.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7235e-05 2020-08-07T15:06:42.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:42.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.204215+0000", 2020-08-07T15:06:42.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00084113000000000005 2020-08-07T15:06:42.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:42.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.205057+0000", 2020-08-07T15:06:42.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3471000000000003e-05 2020-08-07T15:06:42.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:42.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.205120+0000", 2020-08-07T15:06:42.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.080409670000000003 2020-08-07T15:06:42.278 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.278 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.278 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.279 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:7576 3.b e24/20 3:d854a547:::benchmark_data_smithi074_35447_object946:head v 24'456, mlcod=24'448)", 2020-08-07T15:06:42.279 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:40.741593+0000", 2020-08-07T15:06:42.279 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.252145297, 2020-08-07T15:06:42.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18773600700000001, 2020-08-07T15:06:42.279 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.280 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:42.280 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.741593+0000", 2020-08-07T15:06:42.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.741593+0000", 2020-08-07T15:06:42.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1712000000000002e-05 2020-08-07T15:06:42.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.741645+0000", 2020-08-07T15:06:42.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2389999999999999e-06 2020-08-07T15:06:42.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.741651+0000", 2020-08-07T15:06:42.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18616382400000001 2020-08-07T15:06:42.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.927815+0000", 2020-08-07T15:06:42.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7359999999999999e-06 2020-08-07T15:06:42.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.927818+0000", 2020-08-07T15:06:42.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3000000000000002e-05 2020-08-07T15:06:42.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:42.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.927861+0000", 2020-08-07T15:06:42.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4202e-05 2020-08-07T15:06:42.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:42.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.927886+0000", 2020-08-07T15:06:42.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014097070000000001 2020-08-07T15:06:42.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:42.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.929295+0000", 2020-08-07T15:06:42.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3587000000000002e-05 2020-08-07T15:06:42.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:42.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.929329+0000", 2020-08-07T15:06:42.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18773600700000001 2020-08-07T15:06:42.290 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.290 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.290 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.291 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:7848 3.9 3:905b1512:::benchmark_data_smithi074_35447_object980:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:42.291 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:40.895061+0000", 2020-08-07T15:06:42.292 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.098676738, 2020-08-07T15:06:42.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.187604353, 2020-08-07T15:06:42.292 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.292 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:42.292 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:42.293 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:42.293 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:42.293 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7848 2020-08-07T15:06:42.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.293 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.895061+0000", 2020-08-07T15:06:42.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.895061+0000", 2020-08-07T15:06:42.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7772999999999997e-05 2020-08-07T15:06:42.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.895119+0000", 2020-08-07T15:06:42.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3860000000000004e-06 2020-08-07T15:06:42.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.895126+0000", 2020-08-07T15:06:42.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1783000000000001e-05 2020-08-07T15:06:42.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.895137+0000", 2020-08-07T15:06:42.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6629999999999998e-06 2020-08-07T15:06:42.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.895143+0000", 2020-08-07T15:06:42.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024182299999999999 2020-08-07T15:06:42.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:42.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.897561+0000", 2020-08-07T15:06:42.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00098884499999999992 2020-08-07T15:06:42.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:42.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.898550+0000", 2020-08-07T15:06:42.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015507699999999999 2020-08-07T15:06:42.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:06:42.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.898705+0000", 2020-08-07T15:06:42.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021473949999999999 2020-08-07T15:06:42.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:42.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.900853+0000", 2020-08-07T15:06:42.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001259788 2020-08-07T15:06:42.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:42.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.902112+0000", 2020-08-07T15:06:42.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.180465141 2020-08-07T15:06:42.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:42.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.082577+0000", 2020-08-07T15:06:42.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9599999999999998e-07 2020-08-07T15:06:42.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:42.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.082578+0000", 2020-08-07T15:06:42.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9520000000000001e-05 2020-08-07T15:06:42.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:42.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.082618+0000", 2020-08-07T15:06:42.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7956e-05 2020-08-07T15:06:42.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:42.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.082666+0000", 2020-08-07T15:06:42.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.187604353 2020-08-07T15:06:42.309 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.309 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.309 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.310 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:8112 3.9 3:9b275d48:::benchmark_data_smithi074_35447_object1013:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:42.310 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.083981+0000", 2020-08-07T15:06:42.310 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.90975728, 2020-08-07T15:06:42.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31911657999999998, 2020-08-07T15:06:42.311 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.311 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:42.311 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:42.311 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:42.311 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:42.311 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 8112 2020-08-07T15:06:42.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.312 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.083981+0000", 2020-08-07T15:06:42.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.083981+0000", 2020-08-07T15:06:42.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4051999999999999e-05 2020-08-07T15:06:42.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084015+0000", 2020-08-07T15:06:42.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7290000000000002e-06 2020-08-07T15:06:42.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084019+0000", 2020-08-07T15:06:42.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2280000000000001e-06 2020-08-07T15:06:42.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084026+0000", 2020-08-07T15:06:42.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7780000000000001e-06 2020-08-07T15:06:42.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084030+0000", 2020-08-07T15:06:42.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00077372300000000003 2020-08-07T15:06:42.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:42.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084803+0000", 2020-08-07T15:06:42.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1502000000000003e-05 2020-08-07T15:06:42.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:42.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084855+0000", 2020-08-07T15:06:42.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5110000000000002e-05 2020-08-07T15:06:42.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:06:42.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084950+0000", 2020-08-07T15:06:42.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012531370000000001 2020-08-07T15:06:42.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:42.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.086203+0000", 2020-08-07T15:06:42.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4799999999999998e-07 2020-08-07T15:06:42.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:42.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.086204+0000", 2020-08-07T15:06:42.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.905e-05 2020-08-07T15:06:42.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:42.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.086253+0000", 2020-08-07T15:06:42.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00062940999999999995 2020-08-07T15:06:42.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:42.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.086882+0000", 2020-08-07T15:06:42.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31612944199999998 2020-08-07T15:06:42.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:42.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.403012+0000", 2020-08-07T15:06:42.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9099999999999991e-07 2020-08-07T15:06:42.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:42.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.403013+0000", 2020-08-07T15:06:42.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.488e-05 2020-08-07T15:06:42.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:42.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.403097+0000", 2020-08-07T15:06:42.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31911657999999998 2020-08-07T15:06:42.329 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.329 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.330 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.330 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:8351 3.1 e24/20 3:82ead401:::benchmark_data_smithi074_35447_object1043:head v 24'583, mlcod=24'575)", 2020-08-07T15:06:42.330 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.268081+0000", 2020-08-07T15:06:42.331 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.72565689499999997, 2020-08-07T15:06:42.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063025766999999996, 2020-08-07T15:06:42.331 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.331 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:42.332 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.268081+0000", 2020-08-07T15:06:42.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.268081+0000", 2020-08-07T15:06:42.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0880999999999998e-05 2020-08-07T15:06:42.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.268112+0000", 2020-08-07T15:06:42.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9709999999999998e-06 2020-08-07T15:06:42.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.268116+0000", 2020-08-07T15:06:42.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058810059999999997 2020-08-07T15:06:42.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.326926+0000", 2020-08-07T15:06:42.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5289999999999999e-06 2020-08-07T15:06:42.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.326932+0000", 2020-08-07T15:06:42.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9787999999999999e-05 2020-08-07T15:06:42.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:42.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.326971+0000", 2020-08-07T15:06:42.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6696000000000001e-05 2020-08-07T15:06:42.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:42.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.326988+0000", 2020-08-07T15:06:42.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040743350000000001 2020-08-07T15:06:42.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:42.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.331062+0000", 2020-08-07T15:06:42.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4506999999999999e-05 2020-08-07T15:06:42.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:42.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.331107+0000", 2020-08-07T15:06:42.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063025766999999996 2020-08-07T15:06:42.342 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.342 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.343 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.343 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:8352 3.1 e24/20 3:82ead401:::benchmark_data_smithi074_35447_object1043:head v 24'584, mlcod=24'576)", 2020-08-07T15:06:42.343 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.268509+0000", 2020-08-07T15:06:42.343 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.72522871600000005, 2020-08-07T15:06:42.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065221127000000004, 2020-08-07T15:06:42.344 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.344 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:42.344 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.268509+0000", 2020-08-07T15:06:42.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.268509+0000", 2020-08-07T15:06:42.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2293999999999998e-05 2020-08-07T15:06:42.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.268562+0000", 2020-08-07T15:06:42.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.421e-06 2020-08-07T15:06:42.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.268568+0000", 2020-08-07T15:06:42.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058794632999999999 2020-08-07T15:06:42.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.327363+0000", 2020-08-07T15:06:42.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.999e-06 2020-08-07T15:06:42.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.327366+0000", 2020-08-07T15:06:42.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00063740900000000002 2020-08-07T15:06:42.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:42.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.328003+0000", 2020-08-07T15:06:42.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.298e-05 2020-08-07T15:06:42.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:42.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.328026+0000", 2020-08-07T15:06:42.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005654226 2020-08-07T15:06:42.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:42.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.333680+0000", 2020-08-07T15:06:42.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0164999999999998e-05 2020-08-07T15:06:42.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:42.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.333731+0000", 2020-08-07T15:06:42.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065221127000000004 2020-08-07T15:06:42.354 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.355 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.355 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.355 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:8600 3.9 3:9bae7392:::benchmark_data_smithi074_35447_object1074:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:42.356 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.426771+0000", 2020-08-07T15:06:42.356 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.56696695699999999, 2020-08-07T15:06:42.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.105394234, 2020-08-07T15:06:42.356 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.356 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:42.356 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:42.357 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:42.357 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:42.357 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 8600 2020-08-07T15:06:42.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.357 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.426771+0000", 2020-08-07T15:06:42.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.426771+0000", 2020-08-07T15:06:42.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5098e-05 2020-08-07T15:06:42.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.426826+0000", 2020-08-07T15:06:42.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4060000000000002e-06 2020-08-07T15:06:42.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.426833+0000", 2020-08-07T15:06:42.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8304999999999998e-05 2020-08-07T15:06:42.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.426851+0000", 2020-08-07T15:06:42.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5010000000000001e-06 2020-08-07T15:06:42.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.426857+0000", 2020-08-07T15:06:42.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016480780000000001 2020-08-07T15:06:42.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:42.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.428506+0000", 2020-08-07T15:06:42.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8722999999999997e-05 2020-08-07T15:06:42.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:42.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.428594+0000", 2020-08-07T15:06:42.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016715499999999999 2020-08-07T15:06:42.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:06:42.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.428761+0000", 2020-08-07T15:06:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013903780000000001 2020-08-07T15:06:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.430152+0000", 2020-08-07T15:06:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0049999999999999e-06 2020-08-07T15:06:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.430153+0000", 2020-08-07T15:06:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000295471 2020-08-07T15:06:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.430448+0000", 2020-08-07T15:06:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062501017000000006 2020-08-07T15:06:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.492949+0000", 2020-08-07T15:06:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039169753000000002 2020-08-07T15:06:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.532119+0000", 2020-08-07T15:06:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.074e-06 2020-08-07T15:06:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.532120+0000", 2020-08-07T15:06:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5269999999999999e-05 2020-08-07T15:06:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.532165+0000", 2020-08-07T15:06:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.105394234 2020-08-07T15:06:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:8760 3.9 3:9f02d263:::benchmark_data_smithi074_35447_object1094:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.531357+0000", 2020-08-07T15:06:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.46238089599999999, 2020-08-07T15:06:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19863291199999999, 2020-08-07T15:06:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 8760 2020-08-07T15:06:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.531357+0000", 2020-08-07T15:06:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.531357+0000", 2020-08-07T15:06:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9942999999999999e-05 2020-08-07T15:06:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.531407+0000", 2020-08-07T15:06:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8319999999999996e-06 2020-08-07T15:06:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.531412+0000", 2020-08-07T15:06:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0128e-05 2020-08-07T15:06:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.531422+0000", 2020-08-07T15:06:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2370000000000003e-06 2020-08-07T15:06:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.531426+0000", 2020-08-07T15:06:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034586119999999998 2020-08-07T15:06:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.534885+0000", 2020-08-07T15:06:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6256999999999997e-05 2020-08-07T15:06:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.534981+0000", 2020-08-07T15:06:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000171373 2020-08-07T15:06:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:06:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.535153+0000", 2020-08-07T15:06:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018094879999999999 2020-08-07T15:06:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.536962+0000", 2020-08-07T15:06:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9400000000000005e-07 2020-08-07T15:06:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.536963+0000", 2020-08-07T15:06:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7188999999999997e-05 2020-08-07T15:06:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.537030+0000", 2020-08-07T15:06:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034211371999999997 2020-08-07T15:06:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.571241+0000", 2020-08-07T15:06:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15868652799999999 2020-08-07T15:06:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.729928+0000", 2020-08-07T15:06:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0139999999999999e-06 2020-08-07T15:06:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.729929+0000", 2020-08-07T15:06:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1245000000000001e-05 2020-08-07T15:06:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.729990+0000", 2020-08-07T15:06:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19863291199999999 2020-08-07T15:06:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.396 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:06:42.397 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:06:42.398 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 6 16 9989 9973 12.9843 13.1172 0.00842889 0.00961598 2020-08-07T15:06:42.399 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:06:42.396+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:06:42.399 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:06:42.499 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:42.497+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:06:42.602 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:06:42.598+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:06:42.612 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:06:42.612 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:06:42.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.613 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:10615 3.2 e24/20 3:4fc42c81:::benchmark_data_smithi074_35447_object1326:head v 24'471, mlcod=24'464)", 2020-08-07T15:06:42.613 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.606909+0000", 2020-08-07T15:06:42.614 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0031829710000000002, 2020-08-07T15:06:42.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032017970000000001, 2020-08-07T15:06:42.614 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.615 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:42.615 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.606909+0000", 2020-08-07T15:06:42.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.606909+0000", 2020-08-07T15:06:42.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2750000000000001e-05 2020-08-07T15:06:42.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.606962+0000", 2020-08-07T15:06:42.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5740000000000003e-06 2020-08-07T15:06:42.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.606968+0000", 2020-08-07T15:06:42.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013088999999999999 2020-08-07T15:06:42.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.607099+0000", 2020-08-07T15:06:42.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999999e-06 2020-08-07T15:06:42.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.607102+0000", 2020-08-07T15:06:42.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6748000000000001e-05 2020-08-07T15:06:42.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:42.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.607139+0000", 2020-08-07T15:06:42.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8870000000000001e-05 2020-08-07T15:06:42.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:42.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.607158+0000", 2020-08-07T15:06:42.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024833199999999998 2020-08-07T15:06:42.628 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.628 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.629 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.629 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:10616 3.2 e24/20 3:4fc42c81:::benchmark_data_smithi074_35447_object1326:head v 24'472, mlcod=24'466)", 2020-08-07T15:06:42.629 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.608118+0000", 2020-08-07T15:06:42.630 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0019738809999999998, 2020-08-07T15:06:42.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020672759999999998, 2020-08-07T15:06:42.630 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.630 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:42.630 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.608118+0000", 2020-08-07T15:06:42.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.608118+0000", 2020-08-07T15:06:42.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7042999999999997e-05 2020-08-07T15:06:42.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.608166+0000", 2020-08-07T15:06:42.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1869999999999999e-06 2020-08-07T15:06:42.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.608170+0000", 2020-08-07T15:06:42.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2527000000000002e-05 2020-08-07T15:06:42.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.608202+0000", 2020-08-07T15:06:42.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2629999999999997e-06 2020-08-07T15:06:42.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.608207+0000", 2020-08-07T15:06:42.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8742000000000003e-05 2020-08-07T15:06:42.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:42.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.608245+0000", 2020-08-07T15:06:42.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4187999999999999e-05 2020-08-07T15:06:42.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:42.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.608269+0000", 2020-08-07T15:06:42.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015095000000000001 2020-08-07T15:06:42.643 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.644 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.644 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.645 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:10625 3.5 3:a4e9edb0:::benchmark_data_smithi074_35447_object1328:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:42.645 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.608693+0000", 2020-08-07T15:06:42.663 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.001398896, 2020-08-07T15:06:42.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015829850000000001, 2020-08-07T15:06:42.664 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.665 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:06:42.665 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:42.665 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:42.666 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:42.666 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 10625 2020-08-07T15:06:42.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.666 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.608693+0000", 2020-08-07T15:06:42.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.608693+0000", 2020-08-07T15:06:42.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1280999999999999e-05 2020-08-07T15:06:42.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.608745+0000", 2020-08-07T15:06:42.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9020000000000004e-06 2020-08-07T15:06:42.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.608751+0000", 2020-08-07T15:06:42.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4562999999999999e-05 2020-08-07T15:06:42.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.608765+0000", 2020-08-07T15:06:42.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5169999999999996e-06 2020-08-07T15:06:42.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.608772+0000", 2020-08-07T15:06:42.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3377000000000002e-05 2020-08-07T15:06:42.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:42.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.608805+0000", 2020-08-07T15:06:42.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012924799999999999 2020-08-07T15:06:42.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:42.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.608934+0000", 2020-08-07T15:06:42.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011928 2020-08-07T15:06:42.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:06:42.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.609054+0000", 2020-08-07T15:06:42.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00036016800000000001 2020-08-07T15:06:42.679 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.680 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.680 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.681 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:10626 3.5 3:a4e9edb0:::benchmark_data_smithi074_35447_object1328:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:42.681 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.609061+0000", 2020-08-07T15:06:42.682 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0010316399999999999, 2020-08-07T15:06:42.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013158410000000001, 2020-08-07T15:06:42.682 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.683 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:42.683 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:42.683 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:42.684 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:42.684 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 10626 2020-08-07T15:06:42.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.685 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.609061+0000", 2020-08-07T15:06:42.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.609061+0000", 2020-08-07T15:06:42.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7567000000000001e-05 2020-08-07T15:06:42.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.609098+0000", 2020-08-07T15:06:42.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7419999999999999e-06 2020-08-07T15:06:42.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.609102+0000", 2020-08-07T15:06:42.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3300000000000001e-06 2020-08-07T15:06:42.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.609109+0000", 2020-08-07T15:06:42.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3730000000000003e-06 2020-08-07T15:06:42.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.609114+0000", 2020-08-07T15:06:42.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00113462 2020-08-07T15:06:42.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:42.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.610248+0000", 2020-08-07T15:06:42.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6256999999999997e-05 2020-08-07T15:06:42.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:42.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.610345+0000", 2020-08-07T15:06:42.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012838890000000001 2020-08-07T15:06:42.697 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.698 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.698 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.699 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:10625 3.5 e24/20)", 2020-08-07T15:06:42.699 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.610065+0000", 2020-08-07T15:06:42.699 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 2.7453999999999998e-05, 2020-08-07T15:06:42.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039651999999999998, 2020-08-07T15:06:42.699 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.700 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:06:42.700 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.610065+0000", 2020-08-07T15:06:42.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.610065+0000", 2020-08-07T15:06:42.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0167999999999999e-05 2020-08-07T15:06:42.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.610105+0000", 2020-08-07T15:06:42.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9750000000000004e-06 2020-08-07T15:06:42.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.610111+0000", 2020-08-07T15:06:42.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4126000000000003e-05 2020-08-07T15:06:42.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.610165+0000", 2020-08-07T15:06:42.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7529999999999998e-06 2020-08-07T15:06:42.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.610171+0000", 2020-08-07T15:06:42.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106022 2020-08-07T15:06:42.706 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.706 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.706 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.707 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:10607 3.5 e24/20)", 2020-08-07T15:06:42.707 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.607840+0000", 2020-08-07T15:06:42.707 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0022528399999999999, 2020-08-07T15:06:42.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002677174, 2020-08-07T15:06:42.708 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.708 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:06:42.708 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.607840+0000", 2020-08-07T15:06:42.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.607840+0000", 2020-08-07T15:06:42.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2379000000000003e-05 2020-08-07T15:06:42.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.607872+0000", 2020-08-07T15:06:42.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1199999999999999e-06 2020-08-07T15:06:42.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.607878+0000", 2020-08-07T15:06:42.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024282219999999998 2020-08-07T15:06:42.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.610306+0000", 2020-08-07T15:06:42.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2140000000000004e-06 2020-08-07T15:06:42.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.610311+0000", 2020-08-07T15:06:42.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024719350000000002 2020-08-07T15:06:42.714 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.714 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.714 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.715 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:10607 3.5 3:aeace627:::benchmark_data_smithi074_35447_object1325:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:42.717 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.599752+0000", 2020-08-07T15:06:42.717 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.010340185999999999, 2020-08-07T15:06:42.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010830375, 2020-08-07T15:06:42.717 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.718 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:06:42.718 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:42.718 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:42.718 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:42.718 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 10607 2020-08-07T15:06:42.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.719 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.599752+0000", 2020-08-07T15:06:42.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.599752+0000", 2020-08-07T15:06:42.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1662000000000001e-05 2020-08-07T15:06:42.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.599804+0000", 2020-08-07T15:06:42.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4049999999999997e-06 2020-08-07T15:06:42.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.599810+0000", 2020-08-07T15:06:42.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2152e-05 2020-08-07T15:06:42.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.599822+0000", 2020-08-07T15:06:42.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3040000000000004e-06 2020-08-07T15:06:42.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.599828+0000", 2020-08-07T15:06:42.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042944109999999997 2020-08-07T15:06:42.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:42.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.604122+0000", 2020-08-07T15:06:42.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0761999999999996e-05 2020-08-07T15:06:42.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:42.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.604213+0000", 2020-08-07T15:06:42.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018540200000000001 2020-08-07T15:06:42.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:06:42.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.604398+0000", 2020-08-07T15:06:42.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034052750000000001 2020-08-07T15:06:42.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:42.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.607804+0000", 2020-08-07T15:06:42.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00054665699999999998 2020-08-07T15:06:42.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:42.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.608350+0000", 2020-08-07T15:06:42.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0085980299999999996 2020-08-07T15:06:42.729 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.730 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.730 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:10608 3.5 3:aeace627:::benchmark_data_smithi074_35447_object1325:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:42.732 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.599863+0000", 2020-08-07T15:06:42.732 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.010229518999999999, 2020-08-07T15:06:42.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010836609000000001, 2020-08-07T15:06:42.732 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:42.733 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:06:42.733 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:42.733 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:42.733 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:42.733 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 10608 2020-08-07T15:06:42.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.734 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:42.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:42.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.599863+0000", 2020-08-07T15:06:42.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:42.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:42.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.599863+0000", 2020-08-07T15:06:42.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7299999999999999e-05 2020-08-07T15:06:42.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:42.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.599900+0000", 2020-08-07T15:06:42.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.25e-06 2020-08-07T15:06:42.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:42.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.599904+0000", 2020-08-07T15:06:42.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4400000000000002e-06 2020-08-07T15:06:42.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:42.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.599911+0000", 2020-08-07T15:06:42.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9189999999999999e-06 2020-08-07T15:06:42.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:42.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.599914+0000", 2020-08-07T15:06:42.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052143490000000001 2020-08-07T15:06:42.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:42.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.605128+0000", 2020-08-07T15:06:42.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105122 2020-08-07T15:06:42.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:42.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.605233+0000", 2020-08-07T15:06:42.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011178700000000001 2020-08-07T15:06:42.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:06:42.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.605345+0000", 2020-08-07T15:06:42.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025008579999999999 2020-08-07T15:06:42.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:42.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:42.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:42.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.607846+0000", 2020-08-07T15:06:42.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0079830249999999995 2020-08-07T15:06:42.747 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.748 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:42.748 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.748 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:42.748 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:06:42.748 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 8 2020-08-07T15:06:42.749 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:06:42.751 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:42.749+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:42.751 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:06:42.854 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:42.850+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:42.905 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:06:42.954 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:42.952+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:42.954 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:06:42.955 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:06:42.955 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:06:42.955 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:06:42.955 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:06:42.967 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:06:43.055 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:43.054+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:43.103 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:06:43.130 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:06:43.168 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:06:43.169 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:06:43.167+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:06:43.187 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:06:43.188 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:06:43.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:06:43.188 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:06:43.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.189 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:1328 3.5 3:a1f5a068:::benchmark_data_smithi074_35447_object165:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e22)", 2020-08-07T15:06:43.189 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.143595+0000", 2020-08-07T15:06:43.190 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.0388373179999997, 2020-08-07T15:06:43.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.083907251000000002, 2020-08-07T15:06:43.190 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:43.191 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:43.195 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:43.195 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:43.196 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:43.196 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1328 2020-08-07T15:06:43.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.196 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:43.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:43.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.143595+0000", 2020-08-07T15:06:43.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:43.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:43.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.143595+0000", 2020-08-07T15:06:43.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5600000000000003e-05 2020-08-07T15:06:43.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:43.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.143651+0000", 2020-08-07T15:06:43.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2689999999999999e-06 2020-08-07T15:06:43.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:43.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.143656+0000", 2020-08-07T15:06:43.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2170999999999999e-05 2020-08-07T15:06:43.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:43.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.143668+0000", 2020-08-07T15:06:43.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2439999999999998e-06 2020-08-07T15:06:43.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:43.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.143675+0000", 2020-08-07T15:06:43.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026876159999999999 2020-08-07T15:06:43.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:43.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.146362+0000", 2020-08-07T15:06:43.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010663299999999999 2020-08-07T15:06:43.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:43.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.146469+0000", 2020-08-07T15:06:43.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017848000000000001 2020-08-07T15:06:43.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:06:43.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.146647+0000", 2020-08-07T15:06:43.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014717160000000001 2020-08-07T15:06:43.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:43.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.148119+0000", 2020-08-07T15:06:43.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.82e-07 2020-08-07T15:06:43.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:43.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.148120+0000", 2020-08-07T15:06:43.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9073999999999997e-05 2020-08-07T15:06:43.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:43.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.148179+0000", 2020-08-07T15:06:43.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075196599999999996 2020-08-07T15:06:43.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:43.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.148931+0000", 2020-08-07T15:06:43.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.078501594999999993 2020-08-07T15:06:43.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:43.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.227432+0000", 2020-08-07T15:06:43.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7499999999999999e-07 2020-08-07T15:06:43.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:43.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.227433+0000", 2020-08-07T15:06:43.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9529999999999993e-05 2020-08-07T15:06:43.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:43.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.227503+0000", 2020-08-07T15:06:43.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.083907251000000002 2020-08-07T15:06:43.223 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.224 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:43.224 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.224 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:1328 3.5 e22/20 ondisk, result = 0)", 2020-08-07T15:06:43.225 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.148614+0000", 2020-08-07T15:06:43.225 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.033819018, 2020-08-07T15:06:43.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.079063096999999999, 2020-08-07T15:06:43.225 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:43.225 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:43.226 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:43.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:43.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.148614+0000", 2020-08-07T15:06:43.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:43.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:43.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.148614+0000", 2020-08-07T15:06:43.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4468000000000002e-05 2020-08-07T15:06:43.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:43.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.148658+0000", 2020-08-07T15:06:43.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9479999999999996e-06 2020-08-07T15:06:43.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:43.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.148664+0000", 2020-08-07T15:06:43.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.077586575000000005 2020-08-07T15:06:43.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:43.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.226251+0000", 2020-08-07T15:06:43.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.188e-05 2020-08-07T15:06:43.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:43.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.226263+0000", 2020-08-07T15:06:43.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001149452 2020-08-07T15:06:43.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:43.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.227412+0000", 2020-08-07T15:06:43.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8920000000000006e-06 2020-08-07T15:06:43.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:43.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.227421+0000", 2020-08-07T15:06:43.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025588199999999997 2020-08-07T15:06:43.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:43.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.227677+0000", 2020-08-07T15:06:43.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.079063096999999999 2020-08-07T15:06:43.235 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.235 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:43.235 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.236 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:1592 3.5 3:ab93003a:::benchmark_data_smithi074_35447_object198:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:43.236 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.292205+0000", 2020-08-07T15:06:43.236 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.890228145, 2020-08-07T15:06:43.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13617961100000001, 2020-08-07T15:06:43.236 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:43.237 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:43.237 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:43.237 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:43.237 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:43.237 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1592 2020-08-07T15:06:43.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.238 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:43.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:43.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.292205+0000", 2020-08-07T15:06:43.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:43.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:43.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.292205+0000", 2020-08-07T15:06:43.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7688999999999997e-05 2020-08-07T15:06:43.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:43.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.292252+0000", 2020-08-07T15:06:43.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0139999999999998e-06 2020-08-07T15:06:43.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:43.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.292257+0000", 2020-08-07T15:06:43.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0682e-05 2020-08-07T15:06:43.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:43.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.292268+0000", 2020-08-07T15:06:43.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1590000000000003e-06 2020-08-07T15:06:43.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:43.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.292272+0000", 2020-08-07T15:06:43.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030296559999999999 2020-08-07T15:06:43.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:43.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.295302+0000", 2020-08-07T15:06:43.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9184000000000004e-05 2020-08-07T15:06:43.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:43.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.295401+0000", 2020-08-07T15:06:43.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111563 2020-08-07T15:06:43.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:06:43.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.295513+0000", 2020-08-07T15:06:43.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030877980000000001 2020-08-07T15:06:43.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:43.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.298600+0000", 2020-08-07T15:06:43.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8800000000000002e-07 2020-08-07T15:06:43.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:43.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.298601+0000", 2020-08-07T15:06:43.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7547999999999999e-05 2020-08-07T15:06:43.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:43.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.298658+0000", 2020-08-07T15:06:43.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038224410000000002 2020-08-07T15:06:43.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:43.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.302481+0000", 2020-08-07T15:06:43.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.125813647 2020-08-07T15:06:43.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:43.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.428295+0000", 2020-08-07T15:06:43.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.048e-06 2020-08-07T15:06:43.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:43.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.428296+0000", 2020-08-07T15:06:43.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8593999999999999e-05 2020-08-07T15:06:43.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:43.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.428384+0000", 2020-08-07T15:06:43.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13617961100000001 2020-08-07T15:06:43.255 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.255 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:43.255 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.256 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:3152 3.5 3:afe03b3a:::benchmark_data_smithi074_35447_object393:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:43.256 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:38.127120+0000", 2020-08-07T15:06:43.256 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.0553129319999996, 2020-08-07T15:06:43.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.088781400999999996, 2020-08-07T15:06:43.256 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:43.257 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:43.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:43.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:43.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:43.257 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3152 2020-08-07T15:06:43.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.258 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:43.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:43.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.127120+0000", 2020-08-07T15:06:43.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:43.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:43.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.127120+0000", 2020-08-07T15:06:43.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0979000000000001e-05 2020-08-07T15:06:43.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:43.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.127171+0000", 2020-08-07T15:06:43.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2931999999999999e-05 2020-08-07T15:06:43.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:43.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.127184+0000", 2020-08-07T15:06:43.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1583000000000001e-05 2020-08-07T15:06:43.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:43.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.127195+0000", 2020-08-07T15:06:43.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.434e-06 2020-08-07T15:06:43.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:43.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.127201+0000", 2020-08-07T15:06:43.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059899319999999999 2020-08-07T15:06:43.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:43.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.133191+0000", 2020-08-07T15:06:43.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101814 2020-08-07T15:06:43.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:43.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.133292+0000", 2020-08-07T15:06:43.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017860700000000001 2020-08-07T15:06:43.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:06:43.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.133471+0000", 2020-08-07T15:06:43.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001283782 2020-08-07T15:06:43.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:43.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.134755+0000", 2020-08-07T15:06:43.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2099999999999998e-07 2020-08-07T15:06:43.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:43.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.134755+0000", 2020-08-07T15:06:43.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3996000000000003e-05 2020-08-07T15:06:43.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:43.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.134809+0000", 2020-08-07T15:06:43.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058894539999999997 2020-08-07T15:06:43.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:43.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.140699+0000", 2020-08-07T15:06:43.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.075139474999999997 2020-08-07T15:06:43.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:43.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.215838+0000", 2020-08-07T15:06:43.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.004e-06 2020-08-07T15:06:43.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:43.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.215839+0000", 2020-08-07T15:06:43.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2088000000000006e-05 2020-08-07T15:06:43.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:43.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.215901+0000", 2020-08-07T15:06:43.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.088781400999999996 2020-08-07T15:06:43.275 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.275 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:43.276 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.277 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:3152 3.5 e23/20 ondisk, result = 0)", 2020-08-07T15:06:43.277 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:38.139623+0000", 2020-08-07T15:06:43.277 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.0428096949999999, 2020-08-07T15:06:43.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.076284063999999999, 2020-08-07T15:06:43.278 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:43.278 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:43.278 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:43.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:43.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.139623+0000", 2020-08-07T15:06:43.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:43.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:43.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.139623+0000", 2020-08-07T15:06:43.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2765999999999998e-05 2020-08-07T15:06:43.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:43.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.139656+0000", 2020-08-07T15:06:43.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.852e-06 2020-08-07T15:06:43.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:43.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.139662+0000", 2020-08-07T15:06:43.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.075145223999999997 2020-08-07T15:06:43.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:43.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.214807+0000", 2020-08-07T15:06:43.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3460000000000004e-06 2020-08-07T15:06:43.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:43.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.214811+0000", 2020-08-07T15:06:43.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001008285 2020-08-07T15:06:43.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:43.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.215819+0000", 2020-08-07T15:06:43.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2620000000000004e-06 2020-08-07T15:06:43.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:43.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.215829+0000", 2020-08-07T15:06:43.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8329000000000002e-05 2020-08-07T15:06:43.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:43.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.215907+0000", 2020-08-07T15:06:43.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.076284063999999999 2020-08-07T15:06:43.288 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.288 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:43.288 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.289 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:7552 3.5 e24/20 ondisk, result = 0)", 2020-08-07T15:06:43.289 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:40.726272+0000", 2020-08-07T15:06:43.290 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 2.4561604950000002, 2020-08-07T15:06:43.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091565685999999993, 2020-08-07T15:06:43.290 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:43.290 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:43.290 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:43.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:43.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.726272+0000", 2020-08-07T15:06:43.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:43.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:43.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.726272+0000", 2020-08-07T15:06:43.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3808e-05 2020-08-07T15:06:43.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:43.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.726296+0000", 2020-08-07T15:06:43.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7009999999999998e-06 2020-08-07T15:06:43.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:43.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.726300+0000", 2020-08-07T15:06:43.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091430682999999999 2020-08-07T15:06:43.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:43.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.817730+0000", 2020-08-07T15:06:43.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2890000000000002e-06 2020-08-07T15:06:43.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:43.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.817734+0000", 2020-08-07T15:06:43.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5105999999999999e-05 2020-08-07T15:06:43.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:43.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.817769+0000", 2020-08-07T15:06:43.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0180000000000007e-06 2020-08-07T15:06:43.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:43.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.817778+0000", 2020-08-07T15:06:43.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0081000000000003e-05 2020-08-07T15:06:43.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:43.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.817838+0000", 2020-08-07T15:06:43.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091565685999999993 2020-08-07T15:06:43.304 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.304 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:43.304 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.305 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:8160 3.4 e24/20 3:22d65f3a:::benchmark_data_smithi074_35447_object1019:head v 24'488, mlcod=24'484)", 2020-08-07T15:06:43.305 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.127249+0000", 2020-08-07T15:06:43.305 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 2.0551839260000002, 2020-08-07T15:06:43.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.111736107, 2020-08-07T15:06:43.306 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:43.306 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:43.306 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:43.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:43.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.127249+0000", 2020-08-07T15:06:43.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:43.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:43.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.127249+0000", 2020-08-07T15:06:43.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8744000000000001e-05 2020-08-07T15:06:43.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:43.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.127278+0000", 2020-08-07T15:06:43.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8530000000000002e-06 2020-08-07T15:06:43.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:43.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.127281+0000", 2020-08-07T15:06:43.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11100056899999999 2020-08-07T15:06:43.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:43.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.238282+0000", 2020-08-07T15:06:43.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4380000000000003e-06 2020-08-07T15:06:43.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:43.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.238287+0000", 2020-08-07T15:06:43.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1980999999999997e-05 2020-08-07T15:06:43.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:43.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.238329+0000", 2020-08-07T15:06:43.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9125000000000001e-05 2020-08-07T15:06:43.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:43.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.238349+0000", 2020-08-07T15:06:43.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00057688100000000001 2020-08-07T15:06:43.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:43.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.238925+0000", 2020-08-07T15:06:43.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9515999999999997e-05 2020-08-07T15:06:43.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:43.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.238985+0000", 2020-08-07T15:06:43.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.111736107 2020-08-07T15:06:43.317 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.317 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:43.317 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.318 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:8400 3.5 3:a221d962:::benchmark_data_smithi074_35447_object1049:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:43.318 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.296991+0000", 2020-08-07T15:06:43.318 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.8854412519999999, 2020-08-07T15:06:43.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.068921284999999999, 2020-08-07T15:06:43.319 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:43.319 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:43.319 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:43.319 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:43.319 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:43.320 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 8400 2020-08-07T15:06:43.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.320 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:43.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:43.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.296991+0000", 2020-08-07T15:06:43.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:43.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:43.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.296991+0000", 2020-08-07T15:06:43.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9699000000000001e-05 2020-08-07T15:06:43.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:43.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.297021+0000", 2020-08-07T15:06:43.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4105e-05 2020-08-07T15:06:43.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:43.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.297035+0000", 2020-08-07T15:06:43.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0770000000000008e-06 2020-08-07T15:06:43.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:43.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.297043+0000", 2020-08-07T15:06:43.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9779999999999999e-06 2020-08-07T15:06:43.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:43.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.297047+0000", 2020-08-07T15:06:43.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000930582 2020-08-07T15:06:43.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:43.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.297978+0000", 2020-08-07T15:06:43.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5179999999999997e-05 2020-08-07T15:06:43.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:43.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.298033+0000", 2020-08-07T15:06:43.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8086999999999998e-05 2020-08-07T15:06:43.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:06:43.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.298131+0000", 2020-08-07T15:06:43.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003380882 2020-08-07T15:06:43.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:43.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.301512+0000", 2020-08-07T15:06:43.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6400000000000003e-07 2020-08-07T15:06:43.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:43.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.301513+0000", 2020-08-07T15:06:43.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0618000000000002e-05 2020-08-07T15:06:43.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:43.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.301563+0000", 2020-08-07T15:06:43.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011305038999999999 2020-08-07T15:06:43.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:43.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.312868+0000", 2020-08-07T15:06:43.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05297843 2020-08-07T15:06:43.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:43.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.365847+0000", 2020-08-07T15:06:43.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.64e-07 2020-08-07T15:06:43.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:43.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.365848+0000", 2020-08-07T15:06:43.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5179999999999996e-05 2020-08-07T15:06:43.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:43.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.365913+0000", 2020-08-07T15:06:43.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.068921284999999999 2020-08-07T15:06:43.337 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.337 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:43.337 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.338 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:8688 3.c 3:36190623:::benchmark_data_smithi074_35447_object1085:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:43.338 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.490068+0000", 2020-08-07T15:06:43.338 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.692365149, 2020-08-07T15:06:43.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067523304000000006, 2020-08-07T15:06:43.339 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:43.339 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:43.339 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:43.339 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:43.339 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:43.340 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 8688 2020-08-07T15:06:43.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.340 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:43.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:43.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.490068+0000", 2020-08-07T15:06:43.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:43.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:43.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.490068+0000", 2020-08-07T15:06:43.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3492999999999998e-05 2020-08-07T15:06:43.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:43.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.490121+0000", 2020-08-07T15:06:43.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5156e-05 2020-08-07T15:06:43.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:43.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.490136+0000", 2020-08-07T15:06:43.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0845e-05 2020-08-07T15:06:43.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:43.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.490147+0000", 2020-08-07T15:06:43.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4209999999999997e-06 2020-08-07T15:06:43.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:43.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.490151+0000", 2020-08-07T15:06:43.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035676499999999999 2020-08-07T15:06:43.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:43.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.493719+0000", 2020-08-07T15:06:43.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0690000000000001e-05 2020-08-07T15:06:43.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:43.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.493810+0000", 2020-08-07T15:06:43.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015637299999999999 2020-08-07T15:06:43.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:06:43.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.493966+0000", 2020-08-07T15:06:43.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001083751 2020-08-07T15:06:43.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:43.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.495050+0000", 2020-08-07T15:06:43.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0499999999999999e-07 2020-08-07T15:06:43.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:43.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.495050+0000", 2020-08-07T15:06:43.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0635e-05 2020-08-07T15:06:43.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:43.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.495091+0000", 2020-08-07T15:06:43.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014004499999999999 2020-08-07T15:06:43.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:43.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.495231+0000", 2020-08-07T15:06:43.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062318637000000003 2020-08-07T15:06:43.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:43.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.557550+0000", 2020-08-07T15:06:43.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7199999999999997e-07 2020-08-07T15:06:43.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:43.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.557551+0000", 2020-08-07T15:06:43.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330999999999997e-05 2020-08-07T15:06:43.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:43.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.557591+0000", 2020-08-07T15:06:43.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067523304000000006 2020-08-07T15:06:43.357 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.357 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:43.357 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.358 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:8848 3.c 3:3cdda30e:::benchmark_data_smithi074_35447_object1105:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:43.358 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.577090+0000", 2020-08-07T15:06:43.358 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.6053428750000001, 2020-08-07T15:06:43.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089659833999999994, 2020-08-07T15:06:43.359 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:43.359 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:43.359 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:43.359 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:43.360 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:43.360 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 8848 2020-08-07T15:06:43.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.360 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:43.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:43.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.577090+0000", 2020-08-07T15:06:43.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:43.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:43.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.577090+0000", 2020-08-07T15:06:43.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0127999999999998e-05 2020-08-07T15:06:43.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:43.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.577140+0000", 2020-08-07T15:06:43.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6550000000000001e-06 2020-08-07T15:06:43.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:43.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.577146+0000", 2020-08-07T15:06:43.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1037e-05 2020-08-07T15:06:43.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:43.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.577157+0000", 2020-08-07T15:06:43.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8470000000000003e-06 2020-08-07T15:06:43.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:43.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.577162+0000", 2020-08-07T15:06:43.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034233039999999998 2020-08-07T15:06:43.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:43.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.580585+0000", 2020-08-07T15:06:43.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6587000000000002e-05 2020-08-07T15:06:43.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:43.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.580681+0000", 2020-08-07T15:06:43.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000168515 2020-08-07T15:06:43.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:06:43.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.580850+0000", 2020-08-07T15:06:43.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013840949999999999 2020-08-07T15:06:43.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:43.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.582234+0000", 2020-08-07T15:06:43.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029454600000000002 2020-08-07T15:06:43.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:43.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.582529+0000", 2020-08-07T15:06:43.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.084145440000000002 2020-08-07T15:06:43.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:43.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.666674+0000", 2020-08-07T15:06:43.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2399999999999997e-07 2020-08-07T15:06:43.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:43.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.666675+0000", 2020-08-07T15:06:43.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8688e-05 2020-08-07T15:06:43.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:43.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.666714+0000", 2020-08-07T15:06:43.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6168000000000003e-05 2020-08-07T15:06:43.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:43.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.666750+0000", 2020-08-07T15:06:43.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089659833999999994 2020-08-07T15:06:43.382 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.382 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:43.382 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.383 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:8848 3.c e24/20 ondisk, result = 0)", 2020-08-07T15:06:43.383 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.582681+0000", 2020-08-07T15:06:43.383 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.599751972, 2020-08-07T15:06:43.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.084073614000000005, 2020-08-07T15:06:43.384 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:43.384 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:43.384 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:43.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:43.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.582681+0000", 2020-08-07T15:06:43.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:43.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:43.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.582681+0000", 2020-08-07T15:06:43.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5597999999999999e-05 2020-08-07T15:06:43.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:43.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.582696+0000", 2020-08-07T15:06:43.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3659999999999999e-06 2020-08-07T15:06:43.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:43.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.582699+0000", 2020-08-07T15:06:43.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.082853424999999994 2020-08-07T15:06:43.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:43.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.665552+0000", 2020-08-07T15:06:43.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6099999999999997e-06 2020-08-07T15:06:43.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:43.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.665558+0000", 2020-08-07T15:06:43.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010992039999999999 2020-08-07T15:06:43.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:43.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.666657+0000", 2020-08-07T15:06:43.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3769999999999996e-06 2020-08-07T15:06:43.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:43.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.666665+0000", 2020-08-07T15:06:43.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9034000000000002e-05 2020-08-07T15:06:43.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:43.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.666754+0000", 2020-08-07T15:06:43.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.084073614000000005 2020-08-07T15:06:43.394 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.395 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:43.395 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.395 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:9000 3.c 3:3de33f60:::benchmark_data_smithi074_35447_object1124:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:43.396 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.667897+0000", 2020-08-07T15:06:43.396 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.5145356249999999, 2020-08-07T15:06:43.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.098984628000000005, 2020-08-07T15:06:43.396 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:43.396 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:43.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:43.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:43.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:43.397 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 9000 2020-08-07T15:06:43.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.398 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:43.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:43.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.667897+0000", 2020-08-07T15:06:43.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:43.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:43.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.667897+0000", 2020-08-07T15:06:43.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8347000000000003e-05 2020-08-07T15:06:43.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:43.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.667945+0000", 2020-08-07T15:06:43.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.587e-06 2020-08-07T15:06:43.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:43.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.667951+0000", 2020-08-07T15:06:43.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0832000000000001e-05 2020-08-07T15:06:43.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:43.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.667962+0000", 2020-08-07T15:06:43.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5730000000000001e-06 2020-08-07T15:06:43.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:43.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.667966+0000", 2020-08-07T15:06:43.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013138620000000001 2020-08-07T15:06:43.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:43.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.669280+0000", 2020-08-07T15:06:43.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2025000000000002e-05 2020-08-07T15:06:43.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:43.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.669362+0000", 2020-08-07T15:06:43.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4760999999999996e-05 2020-08-07T15:06:43.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:06:43.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.669457+0000", 2020-08-07T15:06:43.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052464249999999999 2020-08-07T15:06:43.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:43.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.674704+0000", 2020-08-07T15:06:43.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.27e-07 2020-08-07T15:06:43.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:43.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.674704+0000", 2020-08-07T15:06:43.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4468000000000002e-05 2020-08-07T15:06:43.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:43.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.674748+0000", 2020-08-07T15:06:43.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0097141350000000005 2020-08-07T15:06:43.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:43.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.684463+0000", 2020-08-07T15:06:43.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.082350192000000003 2020-08-07T15:06:43.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:43.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.766813+0000", 2020-08-07T15:06:43.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9500000000000001e-07 2020-08-07T15:06:43.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:43.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.766814+0000", 2020-08-07T15:06:43.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8099e-05 2020-08-07T15:06:43.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:43.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.766882+0000", 2020-08-07T15:06:43.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.098984628000000005 2020-08-07T15:06:43.415 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.415 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:43.415 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.416 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:9000 3.c e24/20 ondisk, result = 0)", 2020-08-07T15:06:43.416 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.674798+0000", 2020-08-07T15:06:43.416 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.5076345120000001, 2020-08-07T15:06:43.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.092089158000000004, 2020-08-07T15:06:43.416 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:43.417 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:43.417 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:43.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:43.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.674798+0000", 2020-08-07T15:06:43.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:43.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:43.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.674798+0000", 2020-08-07T15:06:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1404e-05 2020-08-07T15:06:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.674830+0000", 2020-08-07T15:06:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9519999999999999e-06 2020-08-07T15:06:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.674836+0000", 2020-08-07T15:06:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091707999999999998 2020-08-07T15:06:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:43.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.766544+0000", 2020-08-07T15:06:43.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9200000000000001e-06 2020-08-07T15:06:43.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:43.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.766549+0000", 2020-08-07T15:06:43.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024535300000000002 2020-08-07T15:06:43.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:43.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.766795+0000", 2020-08-07T15:06:43.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6400000000000003e-06 2020-08-07T15:06:43.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:43.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.766803+0000", 2020-08-07T15:06:43.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3888999999999996e-05 2020-08-07T15:06:43.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:43.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.766887+0000", 2020-08-07T15:06:43.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.092089158000000004 2020-08-07T15:06:43.426 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.426 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:43.427 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.427 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:9472 3.5 3:a4757067:::benchmark_data_smithi074_35447_object1183:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:43.427 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.956935+0000", 2020-08-07T15:06:43.428 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.2254975530000001, 2020-08-07T15:06:43.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.113662296, 2020-08-07T15:06:43.428 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:43.428 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:43.429 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:43.429 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:43.429 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:43.429 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 9472 2020-08-07T15:06:43.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.429 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:43.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:43.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.956935+0000", 2020-08-07T15:06:43.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:43.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:43.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.956935+0000", 2020-08-07T15:06:43.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8940999999999999e-05 2020-08-07T15:06:43.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:43.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.956974+0000", 2020-08-07T15:06:43.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.754e-06 2020-08-07T15:06:43.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:43.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.956978+0000", 2020-08-07T15:06:43.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6989999999999997e-06 2020-08-07T15:06:43.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:43.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.956985+0000", 2020-08-07T15:06:43.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4539999999999999e-06 2020-08-07T15:06:43.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:43.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.956988+0000", 2020-08-07T15:06:43.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081913590000000005 2020-08-07T15:06:43.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:43.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.965179+0000", 2020-08-07T15:06:43.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9920000000000006e-05 2020-08-07T15:06:43.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:43.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.965269+0000", 2020-08-07T15:06:43.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017601800000000001 2020-08-07T15:06:43.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:06:43.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.965445+0000", 2020-08-07T15:06:43.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017736519999999999 2020-08-07T15:06:43.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:43.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.967219+0000", 2020-08-07T15:06:43.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6000000000000004e-07 2020-08-07T15:06:43.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:43.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.967220+0000", 2020-08-07T15:06:43.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2242999999999993e-05 2020-08-07T15:06:43.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:43.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.967282+0000", 2020-08-07T15:06:43.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013706836999999999 2020-08-07T15:06:43.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:43.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.980989+0000", 2020-08-07T15:06:43.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089526219000000004 2020-08-07T15:06:43.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:43.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.070515+0000", 2020-08-07T15:06:43.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.052e-06 2020-08-07T15:06:43.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:43.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.070516+0000", 2020-08-07T15:06:43.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1588000000000006e-05 2020-08-07T15:06:43.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:43.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.070597+0000", 2020-08-07T15:06:43.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.113662296 2020-08-07T15:06:43.446 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.447 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:43.447 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.447 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:9472 3.5 e24/20 ondisk, result = 0)", 2020-08-07T15:06:43.448 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.968785+0000", 2020-08-07T15:06:43.448 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.213647683, 2020-08-07T15:06:43.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.101821116, 2020-08-07T15:06:43.448 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:43.448 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:43.448 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:43.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:43.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.968785+0000", 2020-08-07T15:06:43.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:43.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:43.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.968785+0000", 2020-08-07T15:06:43.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0179999999999999e-05 2020-08-07T15:06:43.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:43.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.968815+0000", 2020-08-07T15:06:43.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6710000000000002e-06 2020-08-07T15:06:43.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:43.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.968822+0000", 2020-08-07T15:06:43.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.101247675 2020-08-07T15:06:43.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:43.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.070070+0000", 2020-08-07T15:06:43.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1050000000000001e-06 2020-08-07T15:06:43.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:43.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.070079+0000", 2020-08-07T15:06:43.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000418632 2020-08-07T15:06:43.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:43.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.070497+0000", 2020-08-07T15:06:43.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0100000000000001e-06 2020-08-07T15:06:43.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:43.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.070506+0000", 2020-08-07T15:06:43.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9842999999999999e-05 2020-08-07T15:06:43.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:43.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.070606+0000", 2020-08-07T15:06:43.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.101821116 2020-08-07T15:06:43.458 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.458 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:43.458 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.459 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:9552 3.2 e24/20 3:43e5b7cb:::benchmark_data_smithi074_35447_object1193:head v 24'416, mlcod=24'409)", 2020-08-07T15:06:43.459 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.009664+0000", 2020-08-07T15:06:43.459 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.1727686450000001, 2020-08-07T15:06:43.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065861243, 2020-08-07T15:06:43.460 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:43.460 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:43.460 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:43.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:43.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.009664+0000", 2020-08-07T15:06:43.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:43.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:43.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.009664+0000", 2020-08-07T15:06:43.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3906999999999997e-05 2020-08-07T15:06:43.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:43.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.009718+0000", 2020-08-07T15:06:43.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2380000000000002e-06 2020-08-07T15:06:43.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:43.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.009724+0000", 2020-08-07T15:06:43.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.059973494000000002 2020-08-07T15:06:43.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:43.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.069698+0000", 2020-08-07T15:06:43.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0400000000000003e-06 2020-08-07T15:06:43.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:43.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.069702+0000", 2020-08-07T15:06:43.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2738999999999999e-05 2020-08-07T15:06:43.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:43.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.069734+0000", 2020-08-07T15:06:43.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7240000000000001e-05 2020-08-07T15:06:43.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:43.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.069752+0000", 2020-08-07T15:06:43.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057185700000000001 2020-08-07T15:06:43.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:43.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.075470+0000", 2020-08-07T15:06:43.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5015000000000001e-05 2020-08-07T15:06:43.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:43.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.075525+0000", 2020-08-07T15:06:43.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065861243 2020-08-07T15:06:43.483 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.483 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:43.484 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.484 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:9664 3.5 3:aa9f4869:::benchmark_data_smithi074_35447_object1207:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:43.484 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.075874+0000", 2020-08-07T15:06:43.485 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.10655857, 2020-08-07T15:06:43.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.094313890999999997, 2020-08-07T15:06:43.485 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:43.485 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:43.485 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:43.486 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:43.486 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:43.486 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 9664 2020-08-07T15:06:43.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.486 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:43.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:43.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.075874+0000", 2020-08-07T15:06:43.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:43.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:43.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.075874+0000", 2020-08-07T15:06:43.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5795999999999998e-05 2020-08-07T15:06:43.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:43.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.075900+0000", 2020-08-07T15:06:43.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9900000000000002e-06 2020-08-07T15:06:43.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:43.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.075903+0000", 2020-08-07T15:06:43.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8100000000000003e-06 2020-08-07T15:06:43.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:43.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.075909+0000", 2020-08-07T15:06:43.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8119999999999999e-06 2020-08-07T15:06:43.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:43.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.075912+0000", 2020-08-07T15:06:43.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00083684999999999999 2020-08-07T15:06:43.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:43.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.076748+0000", 2020-08-07T15:06:43.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2955999999999998e-05 2020-08-07T15:06:43.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:43.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.076811+0000", 2020-08-07T15:06:43.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117762 2020-08-07T15:06:43.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:06:43.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.076929+0000", 2020-08-07T15:06:43.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001747427 2020-08-07T15:06:43.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:43.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.078677+0000", 2020-08-07T15:06:43.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7599999999999997e-07 2020-08-07T15:06:43.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:43.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.078677+0000", 2020-08-07T15:06:43.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5717999999999998e-05 2020-08-07T15:06:43.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:43.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.078723+0000", 2020-08-07T15:06:43.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016839355 2020-08-07T15:06:43.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:43.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.095562+0000", 2020-08-07T15:06:43.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.074583632999999996 2020-08-07T15:06:43.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:43.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.170146+0000", 2020-08-07T15:06:43.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.305e-06 2020-08-07T15:06:43.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:43.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.170147+0000", 2020-08-07T15:06:43.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1001000000000003e-05 2020-08-07T15:06:43.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:43.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.170188+0000", 2020-08-07T15:06:43.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.094313890999999997 2020-08-07T15:06:43.503 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.503 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:43.503 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.505 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:9664 3.5 e24/20 ondisk, result = 0)", 2020-08-07T15:06:43.505 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.080080+0000", 2020-08-07T15:06:43.505 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.102352258, 2020-08-07T15:06:43.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090112431000000007, 2020-08-07T15:06:43.505 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:43.506 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:43.506 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:43.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:43.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.080080+0000", 2020-08-07T15:06:43.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:43.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:43.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.080080+0000", 2020-08-07T15:06:43.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7782999999999999e-05 2020-08-07T15:06:43.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:43.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.080098+0000", 2020-08-07T15:06:43.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4230000000000001e-06 2020-08-07T15:06:43.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:43.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.080102+0000", 2020-08-07T15:06:43.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089993337000000007 2020-08-07T15:06:43.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:43.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.170095+0000", 2020-08-07T15:06:43.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0260000000000004e-06 2020-08-07T15:06:43.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:43.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.170101+0000", 2020-08-07T15:06:43.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1990999999999999e-05 2020-08-07T15:06:43.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:43.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.170133+0000", 2020-08-07T15:06:43.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0400000000000004e-06 2020-08-07T15:06:43.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:43.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.170140+0000", 2020-08-07T15:06:43.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2831000000000003e-05 2020-08-07T15:06:43.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:43.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.170193+0000", 2020-08-07T15:06:43.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090112431000000007 2020-08-07T15:06:43.517 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.517 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:43.517 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.518 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:10184 3.4 e24/20 3:235800a2:::benchmark_data_smithi074_35447_object1272:head v 24'600, mlcod=24'592)", 2020-08-07T15:06:43.518 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.363210+0000", 2020-08-07T15:06:43.518 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.81922262199999996, 2020-08-07T15:06:43.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.097209515999999996, 2020-08-07T15:06:43.519 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:43.519 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:43.519 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:43.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:43.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.363210+0000", 2020-08-07T15:06:43.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:43.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:43.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.363210+0000", 2020-08-07T15:06:43.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7963e-05 2020-08-07T15:06:43.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:43.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.363238+0000", 2020-08-07T15:06:43.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5999999999999998e-06 2020-08-07T15:06:43.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:43.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.363242+0000", 2020-08-07T15:06:43.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.095293311000000006 2020-08-07T15:06:43.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:43.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.458535+0000", 2020-08-07T15:06:43.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6279999999999999e-06 2020-08-07T15:06:43.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:43.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.458542+0000", 2020-08-07T15:06:43.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1872000000000002e-05 2020-08-07T15:06:43.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:43.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.458583+0000", 2020-08-07T15:06:43.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9436000000000002e-05 2020-08-07T15:06:43.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:43.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.458603+0000", 2020-08-07T15:06:43.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017727750000000001 2020-08-07T15:06:43.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:43.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.460376+0000", 2020-08-07T15:06:43.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3930999999999997e-05 2020-08-07T15:06:43.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:43.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.460420+0000", 2020-08-07T15:06:43.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.097209515999999996 2020-08-07T15:06:43.529 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.530 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:43.530 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.530 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:10632 3.5 3:a4e9edb0:::benchmark_data_smithi074_35447_object1328:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:43.531 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.615589+0000", 2020-08-07T15:06:43.531 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.56684335500000005, 2020-08-07T15:06:43.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.070700895, 2020-08-07T15:06:43.531 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:43.531 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:43.532 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:43.532 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:43.532 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:43.532 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 10632 2020-08-07T15:06:43.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.533 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:43.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:43.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.615589+0000", 2020-08-07T15:06:43.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:43.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:43.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.615589+0000", 2020-08-07T15:06:43.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1409999999999997e-05 2020-08-07T15:06:43.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:43.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.615641+0000", 2020-08-07T15:06:43.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8690000000000001e-06 2020-08-07T15:06:43.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:43.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.615647+0000", 2020-08-07T15:06:43.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1653e-05 2020-08-07T15:06:43.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:43.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.615658+0000", 2020-08-07T15:06:43.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.468e-06 2020-08-07T15:06:43.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:43.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.615664+0000", 2020-08-07T15:06:43.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00083561900000000001 2020-08-07T15:06:43.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:43.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.616499+0000", 2020-08-07T15:06:43.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00078214500000000004 2020-08-07T15:06:43.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:43.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.617282+0000", 2020-08-07T15:06:43.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027713100000000001 2020-08-07T15:06:43.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:06:43.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.617559+0000", 2020-08-07T15:06:43.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00108606 2020-08-07T15:06:43.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:43.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.618645+0000", 2020-08-07T15:06:43.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3e-07 2020-08-07T15:06:43.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:43.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.618645+0000", 2020-08-07T15:06:43.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023093699999999999 2020-08-07T15:06:43.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:43.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.618876+0000", 2020-08-07T15:06:43.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021121209999999998 2020-08-07T15:06:43.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:43.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.620989+0000", 2020-08-07T15:06:43.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065233094000000005 2020-08-07T15:06:43.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:43.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.686222+0000", 2020-08-07T15:06:43.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3600000000000002e-07 2020-08-07T15:06:43.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:43.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.686223+0000", 2020-08-07T15:06:43.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7722e-05 2020-08-07T15:06:43.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:43.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:43.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:43.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.686290+0000", 2020-08-07T15:06:43.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.070700895 2020-08-07T15:06:43.553 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.553 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:43.553 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.554 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:43.554 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:43.554 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:06:43.557 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:06:43.558 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:06:43.558 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:06:43.558 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:06:43.559 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:06:43.559 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:06:43.559 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 7 16 11541 11525 12.8614 12.125 0.00902059 0.00969697 2020-08-07T15:06:43.560 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:06:43.560 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:06:43.560 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:06:43.560 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:06:43.560 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:06:43.561 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:06:43.561 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:06:43.561 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:06:43.561 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:06:43.554+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:06:43.562 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:06:43.657 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:43.656+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:43.741 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:43.742 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:43.756 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:06:43.767 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:43.765+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:43.868 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:43.866+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:43.929 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:43.929 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:43.939 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:06:43.970 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:43.970+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:44.072 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:44.071+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:06:44.137 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:44.137 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:44.148 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:06:44.190 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:44.188+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:44.241 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 8 16 13160 13144 12.8347 12.6484 0.00618876 0.0097158 2020-08-07T15:06:44.301 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:44.299+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:44.362 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:44.363 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:44.386 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:06:44.387 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:06:44.388 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 0.000000 2020-08-07T15:06:44.389 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd primary-affinity 0 0 2020-08-07T15:06:44.394 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:06:44.441 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:06:44.440+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:06:44.508 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:44.502+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:44.568 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:44.568 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:44.582 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:06:44.608 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:44.606+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:44.712 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:44.707+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:44.742 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:44.743 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:44.754 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:06:44.815 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:44.811+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:44.925 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:44.931 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:44.932 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:44.924+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:44.934 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:06:45.027 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:45.026+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:45.131 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:45.130+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:45.139 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:45.141 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:45.151 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:06:45.193 INFO:teuthology.orchestra.run.smithi074.stderr:set osd.0 primary-affinity to 0 (802) 2020-08-07T15:06:45.236 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:45.230+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:45.242 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 9 16 14650 14634 12.7019 11.6406 0.00319287 0.00979236 2020-08-07T15:06:45.335 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:45.333+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:45.341 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:45.342 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:45.347 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:06:45.445 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:45.440+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:45.510 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:45.511 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:45.527 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:06:45.564 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:06:45.565 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:06:45.575 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:45.567+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:45.652 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:45.651+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:45.698 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:45.698 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:45.716 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:06:45.729 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:06:45.753 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:45.752+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:45.836 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:45.837 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:45.846 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:06:45.854 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:45.854+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:45.952 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:06:45.955 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:45.955+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:45.966 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:45.967 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:45.984 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:06:45.985 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:06:46.002 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:06:46.029 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:06:46.065 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:06:46.057+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:06:46.072 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:06:46.097 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:06:46.140 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:06:46.141 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:06:46.141 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:06:46.141 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:06:46.143 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:06:46.143 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:06:46.144 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:06:46.144 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:06:46.144 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:06:46.145 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:06:46.146 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:06:46.146 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:06:46.146 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:06:46.147 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:06:46.151 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:06:46.193 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:46.191+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:46.201 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:06:46.243 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 10 15 14792 14777 11.5435 1.11719 0.0162589 0.0108187 2020-08-07T15:06:46.300 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:46.299+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:46.366 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:06:46.367 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:06:46.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:06:46.368 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:06:46.371 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.372 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:1480 3.5 e23/20 3:a1af8afe:::benchmark_data_smithi074_35447_object184:head v 23'80, mlcod=22'63)", 2020-08-07T15:06:46.372 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:37.232119+0000", 2020-08-07T15:06:46.372 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 9.1304287310000003, 2020-08-07T15:06:46.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.058138836999999999, 2020-08-07T15:06:46.378 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:46.378 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:46.379 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:46.383 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.384 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:46.390 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.232119+0000", 2020-08-07T15:06:46.391 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:46.392 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.392 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.393 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:46.393 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.232119+0000", 2020-08-07T15:06:46.393 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4518999999999998e-05 2020-08-07T15:06:46.394 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.394 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.394 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:46.395 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.232174+0000", 2020-08-07T15:06:46.395 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1989999999999997e-06 2020-08-07T15:06:46.395 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.396 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.396 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:46.396 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.232179+0000", 2020-08-07T15:06:46.397 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.052779509000000002 2020-08-07T15:06:46.397 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.398 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.398 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:46.399 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.284959+0000", 2020-08-07T15:06:46.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2750000000000003e-06 2020-08-07T15:06:46.400 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.401 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.401 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:46.402 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.284964+0000", 2020-08-07T15:06:46.403 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9722999999999999e-05 2020-08-07T15:06:46.404 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.404 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.404 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:46.405 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.285004+0000", 2020-08-07T15:06:46.406 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3047000000000001e-05 2020-08-07T15:06:46.406 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.407 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.407 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:46.408 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.285027+0000", 2020-08-07T15:06:46.408 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0051910070000000001 2020-08-07T15:06:46.411 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.412 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.412 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:46.413 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.290218+0000", 2020-08-07T15:06:46.413 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0558e-05 2020-08-07T15:06:46.414 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.414 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.415 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:46.415 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.290258+0000", 2020-08-07T15:06:46.415 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.058138836999999999 2020-08-07T15:06:46.416 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.423 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:46.423 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.424 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.424 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:1592 3.5 e23/20 3:ab93003a:::benchmark_data_smithi074_35447_object198:head v 23'88, mlcod=23'81)", 2020-08-07T15:06:46.425 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:37.295641+0000", 2020-08-07T15:06:46.425 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 9.0669071500000005, 2020-08-07T15:06:46.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.127764135, 2020-08-07T15:06:46.437 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:46.437 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:46.438 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:46.438 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.439 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:46.439 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.295641+0000", 2020-08-07T15:06:46.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:46.440 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.440 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:46.441 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.295641+0000", 2020-08-07T15:06:46.441 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4150000000000003e-05 2020-08-07T15:06:46.442 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.442 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.442 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:46.443 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.295685+0000", 2020-08-07T15:06:46.443 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5971e-05 2020-08-07T15:06:46.443 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.444 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:46.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.295701+0000", 2020-08-07T15:06:46.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.12675505100000001 2020-08-07T15:06:46.445 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.445 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.446 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:46.446 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.422456+0000", 2020-08-07T15:06:46.447 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7000000000000002e-06 2020-08-07T15:06:46.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:46.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.422460+0000", 2020-08-07T15:06:46.449 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3812e-05 2020-08-07T15:06:46.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.450 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:46.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.422504+0000", 2020-08-07T15:06:46.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3331000000000001e-05 2020-08-07T15:06:46.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:46.453 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.422527+0000", 2020-08-07T15:06:46.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00082727300000000001 2020-08-07T15:06:46.454 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.454 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.454 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:46.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.423354+0000", 2020-08-07T15:06:46.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0847000000000003e-05 2020-08-07T15:06:46.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:46.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.423405+0000", 2020-08-07T15:06:46.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.127764135 2020-08-07T15:06:46.457 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.458 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:46.459 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.459 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.460 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:7848 3.9 e24/20 3:905b1512:::benchmark_data_smithi074_35447_object980:head v 24'408, mlcod=24'399)", 2020-08-07T15:06:46.460 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.898821+0000", 2020-08-07T15:06:46.460 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.4637270239999998, 2020-08-07T15:06:46.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.182786849, 2020-08-07T15:06:46.461 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:46.461 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:46.462 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:46.462 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.462 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:46.463 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.898821+0000", 2020-08-07T15:06:46.463 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:46.463 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.464 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.464 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:46.464 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.898821+0000", 2020-08-07T15:06:46.465 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5016000000000002e-05 2020-08-07T15:06:46.465 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.466 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.466 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:46.466 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.898886+0000", 2020-08-07T15:06:46.467 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2740000000000007e-06 2020-08-07T15:06:46.467 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.467 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.468 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:46.469 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.898894+0000", 2020-08-07T15:06:46.469 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.181521612 2020-08-07T15:06:46.469 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.470 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.470 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:46.470 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.080416+0000", 2020-08-07T15:06:46.471 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8980000000000004e-06 2020-08-07T15:06:46.472 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.472 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.472 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:46.473 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.080421+0000", 2020-08-07T15:06:46.473 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6742000000000001e-05 2020-08-07T15:06:46.473 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.474 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.474 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:46.475 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.080468+0000", 2020-08-07T15:06:46.475 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3512000000000001e-05 2020-08-07T15:06:46.476 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.476 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.476 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:46.477 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.080491+0000", 2020-08-07T15:06:46.477 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010625750000000001 2020-08-07T15:06:46.478 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.478 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.479 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:46.479 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.081554+0000", 2020-08-07T15:06:46.479 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4219999999999999e-05 2020-08-07T15:06:46.480 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.480 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:46.481 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.081608+0000", 2020-08-07T15:06:46.481 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.182786849 2020-08-07T15:06:46.482 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.483 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:46.483 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.485 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.486 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.486 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:8088 3.4 3:2323337c:::benchmark_data_smithi074_35447_object1010:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:46.487 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.063382+0000", 2020-08-07T15:06:46.487 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.2991659599999998, 2020-08-07T15:06:46.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.057967306000000003, 2020-08-07T15:06:46.488 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:46.488 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:46.489 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:46.489 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:46.492 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:46.492 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 8088 2020-08-07T15:06:46.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.493 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:46.493 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:46.494 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.063382+0000", 2020-08-07T15:06:46.494 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:46.494 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.494 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:46.495 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.063382+0000", 2020-08-07T15:06:46.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0263000000000003e-05 2020-08-07T15:06:46.495 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.495 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.495 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:46.496 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.063433+0000", 2020-08-07T15:06:46.496 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.516e-06 2020-08-07T15:06:46.496 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.497 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.497 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:46.497 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.063439+0000", 2020-08-07T15:06:46.498 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2894000000000001e-05 2020-08-07T15:06:46.498 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.498 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.499 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:46.499 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.063452+0000", 2020-08-07T15:06:46.500 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3850000000000001e-06 2020-08-07T15:06:46.500 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.500 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.501 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:46.502 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.063457+0000", 2020-08-07T15:06:46.503 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0020091779999999999 2020-08-07T15:06:46.503 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.503 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.504 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:46.504 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.065466+0000", 2020-08-07T15:06:46.505 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5630000000000006e-05 2020-08-07T15:06:46.505 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.505 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.506 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:46.506 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.065542+0000", 2020-08-07T15:06:46.506 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016716699999999999 2020-08-07T15:06:46.507 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.507 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.507 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:06:46.508 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.065709+0000", 2020-08-07T15:06:46.508 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00079925299999999999 2020-08-07T15:06:46.508 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.509 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.509 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:46.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.066509+0000", 2020-08-07T15:06:46.510 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6300000000000005e-07 2020-08-07T15:06:46.510 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:46.511 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.066509+0000", 2020-08-07T15:06:46.511 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6437999999999997e-05 2020-08-07T15:06:46.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.512 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.512 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:46.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.066566+0000", 2020-08-07T15:06:46.513 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00066017799999999998 2020-08-07T15:06:46.513 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.514 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:46.514 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.067226+0000", 2020-08-07T15:06:46.515 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.054055594999999998 2020-08-07T15:06:46.515 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.515 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.516 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:46.516 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.121281+0000", 2020-08-07T15:06:46.516 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8700000000000005e-07 2020-08-07T15:06:46.517 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.517 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.518 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:46.518 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.121282+0000", 2020-08-07T15:06:46.519 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7558999999999995e-05 2020-08-07T15:06:46.519 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.519 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:46.520 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.121350+0000", 2020-08-07T15:06:46.520 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.057967306000000003 2020-08-07T15:06:46.521 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.521 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:46.521 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.522 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.522 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.523 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:8112 3.9 e24/20 3:9b275d48:::benchmark_data_smithi074_35447_object1013:head v 24'416, mlcod=24'408)", 2020-08-07T15:06:46.523 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.085042+0000", 2020-08-07T15:06:46.523 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.277506571, 2020-08-07T15:06:46.533 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31443637000000002, 2020-08-07T15:06:46.533 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:46.544 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:46.544 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:46.545 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:46.546 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085042+0000", 2020-08-07T15:06:46.546 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:46.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.547 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:46.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085042+0000", 2020-08-07T15:06:46.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4534000000000003e-05 2020-08-07T15:06:46.548 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.549 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:46.549 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085096+0000", 2020-08-07T15:06:46.549 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5789999999999998e-06 2020-08-07T15:06:46.552 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.553 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:46.553 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085104+0000", 2020-08-07T15:06:46.553 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31316244700000001 2020-08-07T15:06:46.554 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.554 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.554 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:46.555 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398266+0000", 2020-08-07T15:06:46.555 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3019999999999996e-06 2020-08-07T15:06:46.555 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:46.556 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398270+0000", 2020-08-07T15:06:46.557 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3467000000000002e-05 2020-08-07T15:06:46.557 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.557 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.559 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:46.559 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398314+0000", 2020-08-07T15:06:46.559 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8184000000000001e-05 2020-08-07T15:06:46.559 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.559 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.560 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:46.560 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398332+0000", 2020-08-07T15:06:46.560 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010908059999999999 2020-08-07T15:06:46.560 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.560 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.561 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:46.561 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.399423+0000", 2020-08-07T15:06:46.561 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5050999999999999e-05 2020-08-07T15:06:46.561 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.561 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.562 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:46.562 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.399478+0000", 2020-08-07T15:06:46.562 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31443637000000002 2020-08-07T15:06:46.562 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.562 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:46.562 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.563 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.563 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.563 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:8160 3.4 3:22d65f3a:::benchmark_data_smithi074_35447_object1019:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:46.563 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.126579+0000", 2020-08-07T15:06:46.563 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.2359693490000003, 2020-08-07T15:06:46.564 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11270446200000001, 2020-08-07T15:06:46.564 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:46.564 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:46.564 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:46.564 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:46.565 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:46.565 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 8160 2020-08-07T15:06:46.565 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.565 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:46.565 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.566 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:46.566 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.126579+0000", 2020-08-07T15:06:46.566 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:46.566 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.566 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.567 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:46.567 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.126579+0000", 2020-08-07T15:06:46.567 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6927000000000002e-05 2020-08-07T15:06:46.567 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.567 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.568 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:46.568 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.126606+0000", 2020-08-07T15:06:46.568 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6330000000000002e-06 2020-08-07T15:06:46.568 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.568 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.569 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:46.569 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.126609+0000", 2020-08-07T15:06:46.569 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0690000000000004e-06 2020-08-07T15:06:46.569 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.569 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.569 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:46.570 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.126616+0000", 2020-08-07T15:06:46.570 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.489e-06 2020-08-07T15:06:46.570 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.570 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.571 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:46.571 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.126620+0000", 2020-08-07T15:06:46.571 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013894299999999999 2020-08-07T15:06:46.571 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.571 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.571 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:46.572 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.126759+0000", 2020-08-07T15:06:46.572 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000103078 2020-08-07T15:06:46.572 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.572 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.572 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:46.573 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.126862+0000", 2020-08-07T15:06:46.573 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000199781 2020-08-07T15:06:46.573 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.573 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.573 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:06:46.575 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.127062+0000", 2020-08-07T15:06:46.575 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004376446 2020-08-07T15:06:46.575 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.575 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:46.576 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.131438+0000", 2020-08-07T15:06:46.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9100000000000002e-07 2020-08-07T15:06:46.576 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.576 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.576 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:46.577 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.131439+0000", 2020-08-07T15:06:46.577 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5309999999999998e-05 2020-08-07T15:06:46.577 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.577 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.577 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:46.578 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.131484+0000", 2020-08-07T15:06:46.578 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020192300000000001 2020-08-07T15:06:46.578 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.578 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.578 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:46.579 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.131686+0000", 2020-08-07T15:06:46.579 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.107534138 2020-08-07T15:06:46.579 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.579 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.579 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:46.580 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.239220+0000", 2020-08-07T15:06:46.580 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.136e-06 2020-08-07T15:06:46.580 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.580 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.580 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:46.581 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.239222+0000", 2020-08-07T15:06:46.581 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1698000000000005e-05 2020-08-07T15:06:46.581 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.581 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.581 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:46.581 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.239283+0000", 2020-08-07T15:06:46.582 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11270446200000001 2020-08-07T15:06:46.582 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.582 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:46.582 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.582 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.583 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.583 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:8400 3.5 e24/20 3:a221d962:::benchmark_data_smithi074_35447_object1049:head v 24'400, mlcod=24'392)", 2020-08-07T15:06:46.583 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.298345+0000", 2020-08-07T15:06:46.583 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.0642033020000001, 2020-08-07T15:06:46.583 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.067263969000000007, 2020-08-07T15:06:46.584 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:46.584 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:46.584 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:46.584 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.584 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:46.585 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.298345+0000", 2020-08-07T15:06:46.585 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:46.585 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.585 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.585 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:46.586 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.298345+0000", 2020-08-07T15:06:46.586 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7000999999999999e-05 2020-08-07T15:06:46.586 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.586 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.587 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:46.587 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.298392+0000", 2020-08-07T15:06:46.587 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6180000000000003e-06 2020-08-07T15:06:46.587 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.587 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.588 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:46.588 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.298398+0000", 2020-08-07T15:06:46.588 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.065513196999999995 2020-08-07T15:06:46.588 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.589 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:46.589 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.363911+0000", 2020-08-07T15:06:46.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.34e-06 2020-08-07T15:06:46.589 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.589 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:46.590 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.363918+0000", 2020-08-07T15:06:46.590 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3618000000000001e-05 2020-08-07T15:06:46.590 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.590 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.590 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:46.591 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.363962+0000", 2020-08-07T15:06:46.591 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4422000000000001e-05 2020-08-07T15:06:46.591 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.591 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.591 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:46.592 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.363986+0000", 2020-08-07T15:06:46.592 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015807449999999999 2020-08-07T15:06:46.592 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.592 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.592 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:46.593 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.365567+0000", 2020-08-07T15:06:46.593 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2027999999999998e-05 2020-08-07T15:06:46.593 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.593 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.593 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:46.594 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.365609+0000", 2020-08-07T15:06:46.595 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.067263969000000007 2020-08-07T15:06:46.595 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.595 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:46.595 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.596 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.596 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.596 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:8504 3.4 3:2ea54ce5:::benchmark_data_smithi074_35447_object1062:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:46.596 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.367006+0000", 2020-08-07T15:06:46.596 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4.9955421869999999, 2020-08-07T15:06:46.597 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.056726692000000002, 2020-08-07T15:06:46.597 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:46.597 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:46.597 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:46.597 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:46.597 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:46.598 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 8504 2020-08-07T15:06:46.598 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.598 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:46.598 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.599 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:46.599 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.367006+0000", 2020-08-07T15:06:46.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:46.599 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.599 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.599 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:46.600 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.367006+0000", 2020-08-07T15:06:46.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6777000000000001e-05 2020-08-07T15:06:46.600 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.600 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.600 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:46.601 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.367033+0000", 2020-08-07T15:06:46.601 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7690000000000001e-06 2020-08-07T15:06:46.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.601 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:46.602 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.367036+0000", 2020-08-07T15:06:46.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4820000000000002e-06 2020-08-07T15:06:46.602 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.602 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.602 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:46.603 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.367041+0000", 2020-08-07T15:06:46.603 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1179999999999999e-06 2020-08-07T15:06:46.603 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.603 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:46.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.367043+0000", 2020-08-07T15:06:46.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0036577250000000001 2020-08-07T15:06:46.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:46.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.370701+0000", 2020-08-07T15:06:46.605 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3728999999999998e-05 2020-08-07T15:06:46.605 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.605 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.605 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:46.605 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.370775+0000", 2020-08-07T15:06:46.606 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000149548 2020-08-07T15:06:46.606 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.606 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.606 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:06:46.606 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.370924+0000", 2020-08-07T15:06:46.607 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00088482000000000003 2020-08-07T15:06:46.607 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.607 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.608 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:46.608 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.371809+0000", 2020-08-07T15:06:46.608 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7700000000000004e-07 2020-08-07T15:06:46.608 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.608 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.609 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:46.609 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.371810+0000", 2020-08-07T15:06:46.609 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8349999999999997e-05 2020-08-07T15:06:46.609 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.609 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.610 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:46.610 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.371848+0000", 2020-08-07T15:06:46.610 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00089187399999999997 2020-08-07T15:06:46.610 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.611 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.611 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:46.611 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.372740+0000", 2020-08-07T15:06:46.611 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.050945941000000002 2020-08-07T15:06:46.611 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.612 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.612 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:46.612 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.423686+0000", 2020-08-07T15:06:46.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8699999999999995e-07 2020-08-07T15:06:46.612 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.613 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:46.613 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.423686+0000", 2020-08-07T15:06:46.613 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6295000000000003e-05 2020-08-07T15:06:46.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.614 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.614 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:46.614 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.423733+0000", 2020-08-07T15:06:46.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.056726692000000002 2020-08-07T15:06:46.614 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.615 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:46.615 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.615 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.615 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.615 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:8600 3.9 e24/20 3:9bae7392:::benchmark_data_smithi074_35447_object1074:head v 24'440, mlcod=24'431)", 2020-08-07T15:06:46.616 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.429273+0000", 2020-08-07T15:06:46.616 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4.9332747790000004, 2020-08-07T15:06:46.616 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.10165708, 2020-08-07T15:06:46.616 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:46.616 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:46.617 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:46.617 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.617 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:46.617 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.429273+0000", 2020-08-07T15:06:46.617 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:46.618 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.618 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.618 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:46.618 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.429273+0000", 2020-08-07T15:06:46.618 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7905999999999998e-05 2020-08-07T15:06:46.620 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.620 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.620 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:46.621 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.429321+0000", 2020-08-07T15:06:46.621 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1460000000000001e-06 2020-08-07T15:06:46.621 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.621 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.621 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:46.622 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.429327+0000", 2020-08-07T15:06:46.622 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.098736862999999994 2020-08-07T15:06:46.622 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.622 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.622 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:46.623 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.528064+0000", 2020-08-07T15:06:46.623 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8349999999999997e-06 2020-08-07T15:06:46.623 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.623 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.623 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:46.624 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.528068+0000", 2020-08-07T15:06:46.624 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.392e-05 2020-08-07T15:06:46.624 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.624 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.624 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:46.624 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.528112+0000", 2020-08-07T15:06:46.625 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3583000000000001e-05 2020-08-07T15:06:46.625 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.625 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.625 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:46.625 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.528136+0000", 2020-08-07T15:06:46.626 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0027354720000000001 2020-08-07T15:06:46.626 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.626 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.626 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:46.626 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.530871+0000", 2020-08-07T15:06:46.627 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9354999999999997e-05 2020-08-07T15:06:46.627 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.627 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:46.627 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.530931+0000", 2020-08-07T15:06:46.628 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.10165708 2020-08-07T15:06:46.628 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.629 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:46.629 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.629 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.629 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.631 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:8760 3.9 e24/20 3:9f02d263:::benchmark_data_smithi074_35447_object1094:head v 24'448, mlcod=24'441)", 2020-08-07T15:06:46.631 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.535293+0000", 2020-08-07T15:06:46.631 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4.8272553790000003, 2020-08-07T15:06:46.631 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.194025799, 2020-08-07T15:06:46.631 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:46.632 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:46.632 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:46.632 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.632 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:46.632 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.535293+0000", 2020-08-07T15:06:46.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:46.633 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.633 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.633 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:46.633 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.535293+0000", 2020-08-07T15:06:46.633 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1352e-05 2020-08-07T15:06:46.634 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.634 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.634 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:46.634 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.535334+0000", 2020-08-07T15:06:46.634 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6660000000000002e-06 2020-08-07T15:06:46.635 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.635 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.635 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:46.635 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.535341+0000", 2020-08-07T15:06:46.641 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19233371499999999 2020-08-07T15:06:46.642 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.642 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.643 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:46.643 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.727675+0000", 2020-08-07T15:06:46.644 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1819999999999997e-06 2020-08-07T15:06:46.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.644 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:46.645 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.727680+0000", 2020-08-07T15:06:46.647 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0288999999999999e-05 2020-08-07T15:06:46.647 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.648 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.648 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:46.649 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.727730+0000", 2020-08-07T15:06:46.649 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0340000000000001e-05 2020-08-07T15:06:46.649 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.650 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.650 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:46.651 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.727760+0000", 2020-08-07T15:06:46.651 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015059489999999999 2020-08-07T15:06:46.652 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.652 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.652 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:46.653 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.729266+0000", 2020-08-07T15:06:46.653 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2305999999999997e-05 2020-08-07T15:06:46.654 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.654 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.655 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:46.655 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.729319+0000", 2020-08-07T15:06:46.656 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.194025799 2020-08-07T15:06:46.656 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.656 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:46.657 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.657 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.658 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.658 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:9880 3.4 3:27c71187:::benchmark_data_smithi074_35447_object1234:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:46.658 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.189485+0000", 2020-08-07T15:06:46.659 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4.1730627250000003, 2020-08-07T15:06:46.659 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.058145912000000001, 2020-08-07T15:06:46.659 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:46.660 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:46.660 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:46.661 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:46.661 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:46.661 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 9880 2020-08-07T15:06:46.662 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.662 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:46.662 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.663 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:46.663 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.189485+0000", 2020-08-07T15:06:46.663 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:46.664 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.664 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:46.665 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.189485+0000", 2020-08-07T15:06:46.665 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013917300000000001 2020-08-07T15:06:46.665 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.666 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.666 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:46.666 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.190877+0000", 2020-08-07T15:06:46.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.217e-05 2020-08-07T15:06:46.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:46.668 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.190889+0000", 2020-08-07T15:06:46.668 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3243e-05 2020-08-07T15:06:46.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:46.670 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.190903+0000", 2020-08-07T15:06:46.670 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.541e-06 2020-08-07T15:06:46.671 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.671 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.671 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:46.672 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.190908+0000", 2020-08-07T15:06:46.672 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00065531200000000002 2020-08-07T15:06:46.672 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.672 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.672 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:46.673 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.191563+0000", 2020-08-07T15:06:46.673 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2817000000000003e-05 2020-08-07T15:06:46.673 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.673 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.673 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:46.675 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.191656+0000", 2020-08-07T15:06:46.675 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013346699999999999 2020-08-07T15:06:46.675 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.676 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.676 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:06:46.676 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.191790+0000", 2020-08-07T15:06:46.676 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00092389700000000002 2020-08-07T15:06:46.676 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:46.677 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.192714+0000", 2020-08-07T15:06:46.677 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2899999999999999e-07 2020-08-07T15:06:46.677 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.678 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.678 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:46.678 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.192714+0000", 2020-08-07T15:06:46.678 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4608999999999998e-05 2020-08-07T15:06:46.678 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.679 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:46.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.192769+0000", 2020-08-07T15:06:46.679 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042368520000000002 2020-08-07T15:06:46.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.680 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:46.680 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.197006+0000", 2020-08-07T15:06:46.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.050586719000000002 2020-08-07T15:06:46.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:46.681 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.247592+0000", 2020-08-07T15:06:46.681 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3880000000000001e-06 2020-08-07T15:06:46.681 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.681 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.684 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:46.684 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.247594+0000", 2020-08-07T15:06:46.684 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7737999999999997e-05 2020-08-07T15:06:46.684 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.684 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.685 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:46.685 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.247631+0000", 2020-08-07T15:06:46.685 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.058145912000000001 2020-08-07T15:06:46.685 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.685 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:46.686 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.686 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.686 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.686 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:10184 3.4 3:235800a2:::benchmark_data_smithi074_35447_object1272:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:46.687 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.362426+0000", 2020-08-07T15:06:46.687 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4.0001225229999999, 2020-08-07T15:06:46.687 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.098475605999999993, 2020-08-07T15:06:46.687 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:46.688 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:46.688 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:46.688 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:46.688 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:46.688 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 10184 2020-08-07T15:06:46.689 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.689 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:46.689 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.689 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:46.689 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.362426+0000", 2020-08-07T15:06:46.690 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:46.690 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.690 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.691 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:46.691 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.362426+0000", 2020-08-07T15:06:46.691 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8531e-05 2020-08-07T15:06:46.691 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.691 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.692 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:46.692 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.362474+0000", 2020-08-07T15:06:46.692 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0399999999999998e-06 2020-08-07T15:06:46.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.693 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.693 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:46.693 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.362480+0000", 2020-08-07T15:06:46.693 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2958e-05 2020-08-07T15:06:46.694 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.694 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.694 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:46.694 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.362493+0000", 2020-08-07T15:06:46.694 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2859999999999996e-06 2020-08-07T15:06:46.694 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.695 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:46.695 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.362499+0000", 2020-08-07T15:06:46.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00027227000000000002 2020-08-07T15:06:46.695 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.696 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:46.696 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.362772+0000", 2020-08-07T15:06:46.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3516000000000004e-05 2020-08-07T15:06:46.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:46.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.362865+0000", 2020-08-07T15:06:46.697 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016789800000000001 2020-08-07T15:06:46.697 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.698 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.698 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:06:46.698 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.363033+0000", 2020-08-07T15:06:46.698 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0083392569999999992 2020-08-07T15:06:46.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.699 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.699 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:46.699 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.371372+0000", 2020-08-07T15:06:46.699 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3099999999999996e-07 2020-08-07T15:06:46.699 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.699 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.700 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:46.700 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.371373+0000", 2020-08-07T15:06:46.700 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8289999999999998e-05 2020-08-07T15:06:46.700 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.701 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:46.701 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.371442+0000", 2020-08-07T15:06:46.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038546909999999999 2020-08-07T15:06:46.701 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.702 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:46.702 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.375296+0000", 2020-08-07T15:06:46.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.085539451000000002 2020-08-07T15:06:46.702 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.702 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.703 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:46.703 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.460836+0000", 2020-08-07T15:06:46.703 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.136e-06 2020-08-07T15:06:46.703 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.703 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:46.704 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.460837+0000", 2020-08-07T15:06:46.704 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4350999999999994e-05 2020-08-07T15:06:46.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.705 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:46.705 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.460901+0000", 2020-08-07T15:06:46.705 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.098475605999999993 2020-08-07T15:06:46.705 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.706 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:46.706 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.706 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.706 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.706 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:10264 3.0 e24/20 3:09474b05:::benchmark_data_smithi074_35447_object1282:head v 24'712, mlcod=24'702)", 2020-08-07T15:06:46.707 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.407266+0000", 2020-08-07T15:06:46.707 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 3.9552820510000002, 2020-08-07T15:06:46.707 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.057301527999999997, 2020-08-07T15:06:46.707 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:46.707 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:46.708 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:46.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.708 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:46.708 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.407266+0000", 2020-08-07T15:06:46.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:46.709 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.709 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.709 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:46.709 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.407266+0000", 2020-08-07T15:06:46.709 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8767999999999999e-05 2020-08-07T15:06:46.709 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.710 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.710 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:46.710 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.407305+0000", 2020-08-07T15:06:46.710 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9179999999999997e-06 2020-08-07T15:06:46.710 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.712 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:46.712 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.407312+0000", 2020-08-07T15:06:46.713 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.055686894000000001 2020-08-07T15:06:46.713 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.713 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.713 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:46.714 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.462999+0000", 2020-08-07T15:06:46.714 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0719999999999999e-06 2020-08-07T15:06:46.714 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.714 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.715 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:46.715 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.463004+0000", 2020-08-07T15:06:46.715 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6109999999999997e-05 2020-08-07T15:06:46.715 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.715 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.716 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:46.716 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.463050+0000", 2020-08-07T15:06:46.716 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4124000000000001e-05 2020-08-07T15:06:46.716 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.716 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.717 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:46.717 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.463074+0000", 2020-08-07T15:06:46.717 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0014475130000000001 2020-08-07T15:06:46.717 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.717 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.718 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:46.718 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.464522+0000", 2020-08-07T15:06:46.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6128999999999998e-05 2020-08-07T15:06:46.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.718 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:46.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.464568+0000", 2020-08-07T15:06:46.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.057301527999999997 2020-08-07T15:06:46.719 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.719 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:46.720 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.720 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:10511 3.4 3:26a24ed0:::benchmark_data_smithi074_35447_object1313:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:46.720 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.543983+0000", 2020-08-07T15:06:46.720 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 3.8185649920000002, 2020-08-07T15:06:46.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11060945799999999, 2020-08-07T15:06:46.721 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:46.721 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:46.721 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:46.722 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:46.722 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:46.722 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 10511 2020-08-07T15:06:46.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.723 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:46.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.723 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:46.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.543983+0000", 2020-08-07T15:06:46.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:46.724 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.724 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.724 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:46.724 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.543983+0000", 2020-08-07T15:06:46.724 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0179000000000003e-05 2020-08-07T15:06:46.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.725 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:46.725 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.544023+0000", 2020-08-07T15:06:46.725 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.681e-06 2020-08-07T15:06:46.726 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.726 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.726 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:46.726 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.544027+0000", 2020-08-07T15:06:46.726 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1189999999999999e-06 2020-08-07T15:06:46.727 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.727 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.727 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:46.727 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.544034+0000", 2020-08-07T15:06:46.727 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5080000000000001e-06 2020-08-07T15:06:46.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:46.728 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.544038+0000", 2020-08-07T15:06:46.728 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019932209999999999 2020-08-07T15:06:46.729 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.729 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.729 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:46.730 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.546031+0000", 2020-08-07T15:06:46.730 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4738e-05 2020-08-07T15:06:46.730 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.730 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.730 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:46.731 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.546126+0000", 2020-08-07T15:06:46.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024350199999999999 2020-08-07T15:06:46.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.731 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:06:46.732 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.546369+0000", 2020-08-07T15:06:46.732 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001585159 2020-08-07T15:06:46.732 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.732 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:46.733 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.547954+0000", 2020-08-07T15:06:46.733 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.11e-07 2020-08-07T15:06:46.733 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.733 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:46.735 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.547955+0000", 2020-08-07T15:06:46.735 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7725999999999998e-05 2020-08-07T15:06:46.735 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.735 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.736 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:46.736 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.547982+0000", 2020-08-07T15:06:46.736 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.033040505999999997 2020-08-07T15:06:46.737 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.737 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.737 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:46.737 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.581023+0000", 2020-08-07T15:06:46.737 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.073497703999999997 2020-08-07T15:06:46.738 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.738 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.738 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:46.738 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.654520+0000", 2020-08-07T15:06:46.738 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3300000000000001e-07 2020-08-07T15:06:46.751 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.752 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:46.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.654521+0000", 2020-08-07T15:06:46.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1470999999999994e-05 2020-08-07T15:06:46.763 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.764 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:46.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.654593+0000", 2020-08-07T15:06:46.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11060945799999999 2020-08-07T15:06:46.765 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.765 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:46.766 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.766 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.768 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:10512 3.4 3:26a24ed0:::benchmark_data_smithi074_35447_object1313:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:46.768 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.544057+0000", 2020-08-07T15:06:46.769 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 3.8184914640000001, 2020-08-07T15:06:46.770 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.110753139, 2020-08-07T15:06:46.770 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:46.771 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:46.771 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:46.772 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:46.772 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:46.772 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 10512 2020-08-07T15:06:46.773 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.773 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:46.773 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.774 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:46.774 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.544057+0000", 2020-08-07T15:06:46.774 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:46.775 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.775 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:46.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.544057+0000", 2020-08-07T15:06:46.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.552e-05 2020-08-07T15:06:46.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:46.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.544082+0000", 2020-08-07T15:06:46.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9380000000000001e-06 2020-08-07T15:06:46.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.779 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:46.779 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.544085+0000", 2020-08-07T15:06:46.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2870000000000004e-06 2020-08-07T15:06:46.780 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.781 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:46.781 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.544090+0000", 2020-08-07T15:06:46.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2630000000000002e-06 2020-08-07T15:06:46.782 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.782 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:46.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.544093+0000", 2020-08-07T15:06:46.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0025992150000000002 2020-08-07T15:06:46.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:46.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.546692+0000", 2020-08-07T15:06:46.786 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1206999999999996e-05 2020-08-07T15:06:46.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.787 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:46.787 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.546783+0000", 2020-08-07T15:06:46.788 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015901000000000001 2020-08-07T15:06:46.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.789 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.789 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:06:46.789 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.546942+0000", 2020-08-07T15:06:46.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001413601 2020-08-07T15:06:46.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:46.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.548356+0000", 2020-08-07T15:06:46.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0049999999999999e-06 2020-08-07T15:06:46.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.793 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.793 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:46.793 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.548357+0000", 2020-08-07T15:06:46.794 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8139e-05 2020-08-07T15:06:46.794 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.795 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.795 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:46.795 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.548415+0000", 2020-08-07T15:06:46.796 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.041151559999999997 2020-08-07T15:06:46.796 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.797 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:46.797 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.589566+0000", 2020-08-07T15:06:46.797 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.065168221999999998 2020-08-07T15:06:46.798 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.798 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:46.800 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.654735+0000", 2020-08-07T15:06:46.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3300000000000001e-07 2020-08-07T15:06:46.801 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.802 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.802 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:46.802 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.654735+0000", 2020-08-07T15:06:46.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4438999999999997e-05 2020-08-07T15:06:46.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:46.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.654810+0000", 2020-08-07T15:06:46.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.110753139 2020-08-07T15:06:46.813 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.813 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:46.813 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.814 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:10511 3.4 e24/20 ondisk, result = 0)", 2020-08-07T15:06:46.814 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.567711+0000", 2020-08-07T15:06:46.814 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 3.794837308, 2020-08-07T15:06:46.814 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.086970120999999997, 2020-08-07T15:06:46.814 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:46.815 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:06:46.815 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:46.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.815 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:46.815 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.567711+0000", 2020-08-07T15:06:46.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:46.816 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:46.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.567711+0000", 2020-08-07T15:06:46.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1423000000000001e-05 2020-08-07T15:06:46.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.817 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:46.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.567742+0000", 2020-08-07T15:06:46.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9669999999999998e-06 2020-08-07T15:06:46.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:46.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.567748+0000", 2020-08-07T15:06:46.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.086538514999999996 2020-08-07T15:06:46.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:46.828 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.654287+0000", 2020-08-07T15:06:46.828 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2530000000000004e-06 2020-08-07T15:06:46.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.829 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:46.833 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.654292+0000", 2020-08-07T15:06:46.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000210089 2020-08-07T15:06:46.833 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:46.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.654502+0000", 2020-08-07T15:06:46.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9910000000000008e-06 2020-08-07T15:06:46.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.838 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:46.838 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.654511+0000", 2020-08-07T15:06:46.838 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016988300000000001 2020-08-07T15:06:46.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.839 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:46.840 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.654681+0000", 2020-08-07T15:06:46.840 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.086970120999999997 2020-08-07T15:06:46.840 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.841 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:46.841 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.849 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.849 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.849 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:10512 3.4 e24/20 ondisk, result = 0)", 2020-08-07T15:06:46.850 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.567778+0000", 2020-08-07T15:06:46.850 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 3.7947702310000002, 2020-08-07T15:06:46.850 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.087039287000000007, 2020-08-07T15:06:46.851 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:46.851 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:06:46.851 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:46.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:46.852 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.567778+0000", 2020-08-07T15:06:46.853 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:46.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.862 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:46.862 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.567778+0000", 2020-08-07T15:06:46.863 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9141000000000001e-05 2020-08-07T15:06:46.868 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.869 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.869 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:46.870 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.567807+0000", 2020-08-07T15:06:46.870 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8919999999999999e-06 2020-08-07T15:06:46.870 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.871 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:46.871 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.567812+0000", 2020-08-07T15:06:46.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.086664707999999993 2020-08-07T15:06:46.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.872 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.873 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:46.873 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.654477+0000", 2020-08-07T15:06:46.880 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8720000000000004e-06 2020-08-07T15:06:46.881 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.881 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.881 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:46.882 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.654481+0000", 2020-08-07T15:06:46.882 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00023685500000000001 2020-08-07T15:06:46.882 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.882 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.882 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:46.883 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.654717+0000", 2020-08-07T15:06:46.883 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8890000000000008e-06 2020-08-07T15:06:46.883 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.883 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.884 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:46.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.654726+0000", 2020-08-07T15:06:46.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0929999999999998e-05 2020-08-07T15:06:46.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:46.885 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.654817+0000", 2020-08-07T15:06:46.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.087039287000000007 2020-08-07T15:06:46.886 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.891 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:46.891 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.892 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.892 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.892 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:10766 3.4 3:23da6170:::benchmark_data_smithi074_35447_object1345:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:46.892 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.692730+0000", 2020-08-07T15:06:46.893 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 3.6698186989999999, 2020-08-07T15:06:46.893 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.077379715000000002, 2020-08-07T15:06:46.893 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:46.893 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:46.893 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:46.894 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:46.894 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:46.894 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 10766 2020-08-07T15:06:46.894 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.894 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:46.895 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.895 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:46.895 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.692730+0000", 2020-08-07T15:06:46.895 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:46.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.896 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:46.896 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.692730+0000", 2020-08-07T15:06:46.896 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3331000000000002e-05 2020-08-07T15:06:46.897 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.897 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.897 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:46.897 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.692773+0000", 2020-08-07T15:06:46.897 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.981e-06 2020-08-07T15:06:46.903 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.904 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:46.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.692779+0000", 2020-08-07T15:06:46.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0492999999999999e-05 2020-08-07T15:06:46.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.906 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:46.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.692789+0000", 2020-08-07T15:06:46.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6890000000000003e-06 2020-08-07T15:06:46.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:46.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.692794+0000", 2020-08-07T15:06:46.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017495329999999999 2020-08-07T15:06:46.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:46.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.694544+0000", 2020-08-07T15:06:46.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010938999999999999 2020-08-07T15:06:46.926 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:46.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.694653+0000", 2020-08-07T15:06:46.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020541400000000001 2020-08-07T15:06:46.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:06:46.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.694858+0000", 2020-08-07T15:06:46.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045422470000000001 2020-08-07T15:06:46.934 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:46.936 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.699401+0000", 2020-08-07T15:06:46.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4499999999999995e-07 2020-08-07T15:06:46.937 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:46.938 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.699402+0000", 2020-08-07T15:06:46.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3e-05 2020-08-07T15:06:46.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.955 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.956 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:46.956 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.699465+0000", 2020-08-07T15:06:46.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044844000000000004 2020-08-07T15:06:46.957 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.957 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:46.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.703949+0000", 2020-08-07T15:06:46.958 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.066093846999999997 2020-08-07T15:06:46.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:46.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.770043+0000", 2020-08-07T15:06:46.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4759999999999999e-06 2020-08-07T15:06:46.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:46.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.770044+0000", 2020-08-07T15:06:46.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4968999999999994e-05 2020-08-07T15:06:46.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.962 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:46.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.770109+0000", 2020-08-07T15:06:46.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.077379715000000002 2020-08-07T15:06:46.962 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.962 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:46.963 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:46.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.968 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:10767 3.4 3:23da6170:::benchmark_data_smithi074_35447_object1345:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:46.968 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.692828+0000", 2020-08-07T15:06:46.969 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 3.6697202600000001, 2020-08-07T15:06:46.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.078276883000000005, 2020-08-07T15:06:46.971 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:46.971 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:46.971 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:46.971 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:46.971 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:46.972 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 10767 2020-08-07T15:06:46.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.972 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:46.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:46.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.692828+0000", 2020-08-07T15:06:46.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:46.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:46.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.692828+0000", 2020-08-07T15:06:46.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6833999999999999e-05 2020-08-07T15:06:46.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:46.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.692875+0000", 2020-08-07T15:06:46.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5459999999999999e-06 2020-08-07T15:06:46.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:46.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:46.994 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.692880+0000", 2020-08-07T15:06:46.998 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0429e-05 2020-08-07T15:06:46.999 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:46.999 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.000 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:47.000 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.692891+0000", 2020-08-07T15:06:47.000 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.882e-06 2020-08-07T15:06:47.001 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.001 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:47.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.692896+0000", 2020-08-07T15:06:47.010 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034010569999999999 2020-08-07T15:06:47.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.014 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.015 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:47.015 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.696297+0000", 2020-08-07T15:06:47.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7219000000000005e-05 2020-08-07T15:06:47.016 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:47.017 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.696394+0000", 2020-08-07T15:06:47.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017463400000000001 2020-08-07T15:06:47.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:06:47.033 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.696569+0000", 2020-08-07T15:06:47.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0029035409999999999 2020-08-07T15:06:47.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:47.034 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.699472+0000", 2020-08-07T15:06:47.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1600000000000001e-07 2020-08-07T15:06:47.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:47.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.699473+0000", 2020-08-07T15:06:47.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2442999999999999e-05 2020-08-07T15:06:47.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:47.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.699525+0000", 2020-08-07T15:06:47.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0048498839999999996 2020-08-07T15:06:47.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.037 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:47.037 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.704375+0000", 2020-08-07T15:06:47.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.066658295000000006 2020-08-07T15:06:47.038 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.038 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:47.038 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.771033+0000", 2020-08-07T15:06:47.038 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5900000000000005e-07 2020-08-07T15:06:47.039 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.039 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.039 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:47.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.771034+0000", 2020-08-07T15:06:47.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0643999999999997e-05 2020-08-07T15:06:47.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.049 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:47.050 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.771105+0000", 2020-08-07T15:06:47.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.078276883000000005 2020-08-07T15:06:47.050 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:47.050 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:47.050 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:47.054 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.054 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.055 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:10768 3.4 3:23da6170:::benchmark_data_smithi074_35447_object1345:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:47.055 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.694482+0000", 2020-08-07T15:06:47.056 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 3.6680666849999999, 2020-08-07T15:06:47.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.077384296000000005, 2020-08-07T15:06:47.056 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:47.056 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:47.056 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:47.057 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:47.057 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:47.057 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 10768 2020-08-07T15:06:47.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.057 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:47.058 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.061 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:47.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.694482+0000", 2020-08-07T15:06:47.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:47.062 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.062 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.062 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:47.062 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.694482+0000", 2020-08-07T15:06:47.062 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9116000000000003e-05 2020-08-07T15:06:47.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.063 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:47.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.694521+0000", 2020-08-07T15:06:47.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8089999999999998e-06 2020-08-07T15:06:47.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.064 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:47.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.694526+0000", 2020-08-07T15:06:47.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5016e-05 2020-08-07T15:06:47.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.065 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:47.065 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.694541+0000", 2020-08-07T15:06:47.065 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.37e-06 2020-08-07T15:06:47.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.070 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.070 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:47.070 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.694548+0000", 2020-08-07T15:06:47.070 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002420171 2020-08-07T15:06:47.070 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.071 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.078 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:47.078 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.696968+0000", 2020-08-07T15:06:47.078 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010175699999999999 2020-08-07T15:06:47.079 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.079 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.079 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:47.079 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.697070+0000", 2020-08-07T15:06:47.079 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012413699999999999 2020-08-07T15:06:47.080 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.080 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.080 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:06:47.080 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.697194+0000", 2020-08-07T15:06:47.081 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0025497879999999999 2020-08-07T15:06:47.081 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.081 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.081 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:47.082 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.699744+0000", 2020-08-07T15:06:47.082 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5600000000000005e-07 2020-08-07T15:06:47.082 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.082 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:47.083 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.699744+0000", 2020-08-07T15:06:47.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.469e-05 2020-08-07T15:06:47.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.083 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:47.084 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.699799+0000", 2020-08-07T15:06:47.084 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0046199090000000002 2020-08-07T15:06:47.084 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.084 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.085 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:47.085 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.704419+0000", 2020-08-07T15:06:47.085 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.067382986000000006 2020-08-07T15:06:47.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.086 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:47.086 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.771802+0000", 2020-08-07T15:06:47.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0299999999999999e-07 2020-08-07T15:06:47.086 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.086 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:47.087 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.771803+0000", 2020-08-07T15:06:47.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3287999999999994e-05 2020-08-07T15:06:47.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:47.088 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.771866+0000", 2020-08-07T15:06:47.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.077384296000000005 2020-08-07T15:06:47.088 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:47.088 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:47.089 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:47.089 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:47.089 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:47.089 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:06:47.091 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:06:47.090+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:06:47.091 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:06:47.192 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:47.190+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:47.246 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 11 16 16429 16413 11.6559 12.7812 0.00499119 0.0107139 2020-08-07T15:06:47.299 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:06:47.298+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:06:47.314 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:06:47.314 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:06:47.315 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.315 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:16416 3.b 3:d957dec9:::benchmark_data_smithi074_35447_object2051:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:06:47.315 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.233668+0000", 2020-08-07T15:06:47.315 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.073066967999999996, 2020-08-07T15:06:47.316 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.073092502000000004, 2020-08-07T15:06:47.316 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:47.316 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:47.316 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:47.316 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:47.317 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:47.317 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 16416 2020-08-07T15:06:47.317 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.317 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:47.317 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.318 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:47.318 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.233668+0000", 2020-08-07T15:06:47.318 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:47.318 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.318 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.319 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:47.319 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.233668+0000", 2020-08-07T15:06:47.319 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0918000000000003e-05 2020-08-07T15:06:47.319 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.319 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.327 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:47.327 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.233719+0000", 2020-08-07T15:06:47.327 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.981e-06 2020-08-07T15:06:47.328 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.328 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.328 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:47.328 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.233725+0000", 2020-08-07T15:06:47.328 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3123e-05 2020-08-07T15:06:47.329 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.329 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.329 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:47.329 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.233738+0000", 2020-08-07T15:06:47.329 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3300000000000004e-06 2020-08-07T15:06:47.330 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.330 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.330 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:47.330 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.233745+0000", 2020-08-07T15:06:47.330 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00064557899999999997 2020-08-07T15:06:47.330 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.331 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.331 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:47.331 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.234390+0000", 2020-08-07T15:06:47.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010767799999999999 2020-08-07T15:06:47.331 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.332 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.332 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:47.332 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.234498+0000", 2020-08-07T15:06:47.332 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020364800000000001 2020-08-07T15:06:47.332 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.333 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.333 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:47.333 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.234702+0000", 2020-08-07T15:06:47.333 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024297490000000001 2020-08-07T15:06:47.342 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.342 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.342 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:47.342 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.237131+0000", 2020-08-07T15:06:47.343 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5199999999999996e-07 2020-08-07T15:06:47.343 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.343 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.343 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:47.343 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.237132+0000", 2020-08-07T15:06:47.344 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4466999999999999e-05 2020-08-07T15:06:47.344 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.344 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.344 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:47.344 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.237186+0000", 2020-08-07T15:06:47.345 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003518125 2020-08-07T15:06:47.345 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:47.345 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:47.345 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:47.345 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:47.346 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:06:47.346 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 1 2020-08-07T15:06:47.346 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:06:47.348 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:06:47.402 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:47.401+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:06:47.504 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:47.503+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:47.544 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:06:47.545 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:06:47.545 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:06:47.545 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:06:47.545 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:06:47.577 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:06:47.609 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:47.604+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:47.707 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:47.706+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:47.760 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:06:47.760 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:06:47.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:06:47.761 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:06:47.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.762 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:544 3.b 3:ddefb1a6:::benchmark_data_smithi074_35447_object67:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e22)", 2020-08-07T15:06:47.762 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:36.729434+0000", 2020-08-07T15:06:47.762 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.026406594999999, 2020-08-07T15:06:47.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14661148299999999, 2020-08-07T15:06:47.763 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:47.763 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:47.764 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:47.764 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:47.764 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:47.764 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 544 2020-08-07T15:06:47.764 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.765 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:47.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.765 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:47.766 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.729434+0000", 2020-08-07T15:06:47.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:47.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:47.767 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.729434+0000", 2020-08-07T15:06:47.768 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7930999999999999e-05 2020-08-07T15:06:47.768 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.768 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.768 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:47.768 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.729482+0000", 2020-08-07T15:06:47.769 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8390000000000003e-06 2020-08-07T15:06:47.769 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.769 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.769 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:47.769 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.729488+0000", 2020-08-07T15:06:47.770 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.17e-05 2020-08-07T15:06:47.770 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.770 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.770 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:47.771 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.729500+0000", 2020-08-07T15:06:47.771 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2580000000000003e-06 2020-08-07T15:06:47.771 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.771 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.771 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:47.771 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.729505+0000", 2020-08-07T15:06:47.772 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.008922948 2020-08-07T15:06:47.772 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.772 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.772 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:47.772 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.738428+0000", 2020-08-07T15:06:47.773 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8058000000000002e-05 2020-08-07T15:06:47.773 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.773 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.773 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:47.773 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.738526+0000", 2020-08-07T15:06:47.774 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017383500000000001 2020-08-07T15:06:47.774 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.774 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.774 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:47.774 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.738700+0000", 2020-08-07T15:06:47.775 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012493000000000001 2020-08-07T15:06:47.775 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.775 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:47.775 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.739949+0000", 2020-08-07T15:06:47.775 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9299999999999997e-07 2020-08-07T15:06:47.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:47.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.739950+0000", 2020-08-07T15:06:47.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4191999999999998e-05 2020-08-07T15:06:47.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:47.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.740014+0000", 2020-08-07T15:06:47.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.029007440999999998 2020-08-07T15:06:47.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.779 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:47.779 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.769021+0000", 2020-08-07T15:06:47.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.106953786 2020-08-07T15:06:47.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.780 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:47.780 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.875975+0000", 2020-08-07T15:06:47.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9400000000000005e-07 2020-08-07T15:06:47.780 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.781 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:47.781 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.875976+0000", 2020-08-07T15:06:47.781 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9708000000000004e-05 2020-08-07T15:06:47.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.782 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.782 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:47.782 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:36.876046+0000", 2020-08-07T15:06:47.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14661148299999999 2020-08-07T15:06:47.782 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:47.782 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:47.783 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:47.783 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.783 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:2600 3.b 3:d6a1a0d3:::benchmark_data_smithi074_35447_object324:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:47.783 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:37.800755+0000", 2020-08-07T15:06:47.784 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 9.9550862129999995, 2020-08-07T15:06:47.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.215895015, 2020-08-07T15:06:47.784 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:47.784 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:47.784 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:47.785 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:47.785 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:47.785 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 2600 2020-08-07T15:06:47.785 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.785 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:47.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.786 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:47.786 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.800755+0000", 2020-08-07T15:06:47.786 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:47.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:47.787 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.800755+0000", 2020-08-07T15:06:47.788 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7878999999999999e-05 2020-08-07T15:06:47.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.788 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:47.788 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.800782+0000", 2020-08-07T15:06:47.789 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0520000000000003e-06 2020-08-07T15:06:47.789 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.789 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.789 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:47.789 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.800787+0000", 2020-08-07T15:06:47.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0587999999999999e-05 2020-08-07T15:06:47.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.790 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:47.790 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.800797+0000", 2020-08-07T15:06:47.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4970000000000001e-06 2020-08-07T15:06:47.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:47.791 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.800803+0000", 2020-08-07T15:06:47.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0059548989999999996 2020-08-07T15:06:47.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.792 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.792 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:47.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.806757+0000", 2020-08-07T15:06:47.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000111375 2020-08-07T15:06:47.793 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.793 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.793 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:47.793 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.806869+0000", 2020-08-07T15:06:47.793 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000159345 2020-08-07T15:06:47.794 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.794 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.794 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:47.794 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.807028+0000", 2020-08-07T15:06:47.794 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002275329 2020-08-07T15:06:47.795 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.795 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.795 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:47.795 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.809304+0000", 2020-08-07T15:06:47.795 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2900000000000002e-07 2020-08-07T15:06:47.796 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.796 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:47.796 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.809304+0000", 2020-08-07T15:06:47.796 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8708000000000002e-05 2020-08-07T15:06:47.797 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.797 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.797 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:47.797 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.809353+0000", 2020-08-07T15:06:47.797 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.026379923 2020-08-07T15:06:47.798 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.798 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:47.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.835733+0000", 2020-08-07T15:06:47.799 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18084716200000001 2020-08-07T15:06:47.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:47.800 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.016580+0000", 2020-08-07T15:06:47.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9299999999999997e-07 2020-08-07T15:06:47.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:47.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.016581+0000", 2020-08-07T15:06:47.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8535999999999996e-05 2020-08-07T15:06:47.801 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.801 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.801 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:47.802 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.016650+0000", 2020-08-07T15:06:47.802 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.215895015 2020-08-07T15:06:47.802 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:47.802 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:47.802 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:47.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.806 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:2944 3.b 3:d7542c3f:::benchmark_data_smithi074_35447_object367:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:47.806 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:38.017796+0000", 2020-08-07T15:06:47.807 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 9.7380452470000005, 2020-08-07T15:06:47.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.102218534, 2020-08-07T15:06:47.807 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:47.807 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:47.807 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:47.808 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:47.808 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:47.808 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 2944 2020-08-07T15:06:47.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.808 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:47.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.809 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:47.809 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.017796+0000", 2020-08-07T15:06:47.809 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:47.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.810 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.810 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:47.810 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.017796+0000", 2020-08-07T15:06:47.810 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4678999999999997e-05 2020-08-07T15:06:47.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.811 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:47.811 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.017840+0000", 2020-08-07T15:06:47.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.535e-06 2020-08-07T15:06:47.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:47.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.017844+0000", 2020-08-07T15:06:47.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4690000000000002e-06 2020-08-07T15:06:47.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.813 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:47.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.017850+0000", 2020-08-07T15:06:47.813 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.287e-06 2020-08-07T15:06:47.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.814 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.814 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:47.814 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.017854+0000", 2020-08-07T15:06:47.814 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0014247280000000001 2020-08-07T15:06:47.814 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.815 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:47.815 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.019278+0000", 2020-08-07T15:06:47.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6663999999999996e-05 2020-08-07T15:06:47.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:47.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.019355+0000", 2020-08-07T15:06:47.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000101704 2020-08-07T15:06:47.816 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.817 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:47.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.019457+0000", 2020-08-07T15:06:47.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0016716050000000001 2020-08-07T15:06:47.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.818 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:47.818 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.021128+0000", 2020-08-07T15:06:47.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9399999999999993e-07 2020-08-07T15:06:47.818 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.818 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:47.819 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.021129+0000", 2020-08-07T15:06:47.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1882000000000001e-05 2020-08-07T15:06:47.819 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:47.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.021201+0000", 2020-08-07T15:06:47.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0081014329999999999 2020-08-07T15:06:47.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:47.821 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.029303+0000", 2020-08-07T15:06:47.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.090647648999999997 2020-08-07T15:06:47.821 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:47.822 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.119950+0000", 2020-08-07T15:06:47.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2099999999999995e-07 2020-08-07T15:06:47.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.822 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.822 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:47.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.119951+0000", 2020-08-07T15:06:47.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2984000000000005e-05 2020-08-07T15:06:47.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:47.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.120014+0000", 2020-08-07T15:06:47.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.102218534 2020-08-07T15:06:47.825 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:47.825 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:47.825 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:47.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.826 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:7576 3.b 3:d854a547:::benchmark_data_smithi074_35447_object946:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:47.826 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.736458+0000", 2020-08-07T15:06:47.826 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 7.0193828060000003, 2020-08-07T15:06:47.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19357406199999999, 2020-08-07T15:06:47.827 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:47.827 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:47.827 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:47.827 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:47.828 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:47.828 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7576 2020-08-07T15:06:47.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.828 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:47.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.829 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:47.829 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.736458+0000", 2020-08-07T15:06:47.829 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:47.829 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.829 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.830 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:47.830 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.736458+0000", 2020-08-07T15:06:47.830 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2197999999999997e-05 2020-08-07T15:06:47.830 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.830 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.830 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:47.831 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.736510+0000", 2020-08-07T15:06:47.831 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2579999999999995e-06 2020-08-07T15:06:47.831 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.831 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.831 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:47.832 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.736518+0000", 2020-08-07T15:06:47.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3842999999999999e-05 2020-08-07T15:06:47.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.832 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:47.833 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.736532+0000", 2020-08-07T15:06:47.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0379999999999996e-06 2020-08-07T15:06:47.833 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:47.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.736538+0000", 2020-08-07T15:06:47.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044985529999999998 2020-08-07T15:06:47.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:47.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.741037+0000", 2020-08-07T15:06:47.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6891000000000005e-05 2020-08-07T15:06:47.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:47.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.741134+0000", 2020-08-07T15:06:47.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000198555 2020-08-07T15:06:47.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.836 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:47.836 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.741332+0000", 2020-08-07T15:06:47.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028259190000000001 2020-08-07T15:06:47.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:47.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.744158+0000", 2020-08-07T15:06:47.838 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1100000000000001e-07 2020-08-07T15:06:47.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.838 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:47.838 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.744159+0000", 2020-08-07T15:06:47.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1880999999999998e-05 2020-08-07T15:06:47.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.839 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:47.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.744241+0000", 2020-08-07T15:06:47.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7608000000000002e-05 2020-08-07T15:06:47.840 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.840 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.840 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:47.840 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.744288+0000", 2020-08-07T15:06:47.840 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18566353199999999 2020-08-07T15:06:47.841 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:47.841 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.929952+0000", 2020-08-07T15:06:47.841 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.02e-06 2020-08-07T15:06:47.842 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.842 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.842 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:47.842 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.929953+0000", 2020-08-07T15:06:47.842 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9355000000000002e-05 2020-08-07T15:06:47.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.843 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:47.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.930032+0000", 2020-08-07T15:06:47.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19357406199999999 2020-08-07T15:06:47.844 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:47.844 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:47.844 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:47.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.844 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:7600 3.8 3:11c6b9be:::benchmark_data_smithi074_35447_object949:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:47.845 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.748143+0000", 2020-08-07T15:06:47.845 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 7.0076974969999997, 2020-08-07T15:06:47.845 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.102592691, 2020-08-07T15:06:47.845 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:47.845 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:47.846 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:47.846 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:47.847 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:47.847 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7600 2020-08-07T15:06:47.847 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.848 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:47.848 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.848 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:47.848 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.748143+0000", 2020-08-07T15:06:47.848 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:47.849 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.849 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.849 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:47.849 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.748143+0000", 2020-08-07T15:06:47.850 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3086e-05 2020-08-07T15:06:47.850 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.850 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:47.850 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.748196+0000", 2020-08-07T15:06:47.851 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.477e-06 2020-08-07T15:06:47.851 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.851 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:47.852 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.748203+0000", 2020-08-07T15:06:47.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2943e-05 2020-08-07T15:06:47.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:47.853 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.748216+0000", 2020-08-07T15:06:47.853 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4269999999999999e-06 2020-08-07T15:06:47.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.853 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:47.854 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.748221+0000", 2020-08-07T15:06:47.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002591627 2020-08-07T15:06:47.854 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.854 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.855 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:47.855 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.750813+0000", 2020-08-07T15:06:47.855 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000104891 2020-08-07T15:06:47.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.856 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:47.856 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.750918+0000", 2020-08-07T15:06:47.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018150299999999999 2020-08-07T15:06:47.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.856 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:06:47.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.751099+0000", 2020-08-07T15:06:47.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001847125 2020-08-07T15:06:47.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.858 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:47.858 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.752946+0000", 2020-08-07T15:06:47.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000005e-07 2020-08-07T15:06:47.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.859 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.859 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:47.859 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.752947+0000", 2020-08-07T15:06:47.859 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8338999999999998e-05 2020-08-07T15:06:47.860 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.860 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:47.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.753005+0000", 2020-08-07T15:06:47.860 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028934939999999999 2020-08-07T15:06:47.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.861 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:47.861 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.755899+0000", 2020-08-07T15:06:47.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.094765409999999994 2020-08-07T15:06:47.862 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.862 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.862 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:47.862 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.850664+0000", 2020-08-07T15:06:47.862 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3600000000000003e-07 2020-08-07T15:06:47.862 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:47.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.850665+0000", 2020-08-07T15:06:47.863 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1131999999999996e-05 2020-08-07T15:06:47.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:47.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.850736+0000", 2020-08-07T15:06:47.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.102592691 2020-08-07T15:06:47.864 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:47.865 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:47.865 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:47.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.865 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:7696 3.7 3:ee5cfea7:::benchmark_data_smithi074_35447_object961:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:47.866 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.806620+0000", 2020-08-07T15:06:47.866 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.9492212589999998, 2020-08-07T15:06:47.866 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.12604768499999999, 2020-08-07T15:06:47.866 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:47.866 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:47.867 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:47.867 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:47.868 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:47.868 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7696 2020-08-07T15:06:47.869 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.869 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:47.870 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.870 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:47.870 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.806620+0000", 2020-08-07T15:06:47.871 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:47.871 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.871 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:47.871 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.806620+0000", 2020-08-07T15:06:47.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0121000000000003e-05 2020-08-07T15:06:47.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.872 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.872 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:47.872 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.806680+0000", 2020-08-07T15:06:47.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.024e-06 2020-08-07T15:06:47.873 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.873 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:47.873 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.806687+0000", 2020-08-07T15:06:47.874 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2645e-05 2020-08-07T15:06:47.874 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:47.875 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.806699+0000", 2020-08-07T15:06:47.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6299999999999993e-06 2020-08-07T15:06:47.876 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.877 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:47.877 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.806709+0000", 2020-08-07T15:06:47.877 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6279000000000004e-05 2020-08-07T15:06:47.878 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.879 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:47.879 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.806805+0000", 2020-08-07T15:06:47.880 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010478499999999999 2020-08-07T15:06:47.880 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.881 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:47.881 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.806910+0000", 2020-08-07T15:06:47.881 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018237799999999999 2020-08-07T15:06:47.882 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.882 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.882 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:06:47.883 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.807092+0000", 2020-08-07T15:06:47.883 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0037732249999999998 2020-08-07T15:06:47.883 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.884 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.884 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:47.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.810866+0000", 2020-08-07T15:06:47.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7499999999999999e-07 2020-08-07T15:06:47.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.886 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:47.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.810866+0000", 2020-08-07T15:06:47.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7073000000000001e-05 2020-08-07T15:06:47.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:47.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.810923+0000", 2020-08-07T15:06:47.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.029912563999999999 2020-08-07T15:06:47.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:47.890 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.840836+0000", 2020-08-07T15:06:47.890 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.091729725999999998 2020-08-07T15:06:47.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.891 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.891 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:47.891 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.932566+0000", 2020-08-07T15:06:47.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3900000000000003e-07 2020-08-07T15:06:47.892 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.893 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:47.893 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.932567+0000", 2020-08-07T15:06:47.894 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000100521 2020-08-07T15:06:47.894 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.894 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.895 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:47.895 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.932667+0000", 2020-08-07T15:06:47.895 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.12604768499999999 2020-08-07T15:06:47.896 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:47.896 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:47.896 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:47.897 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.897 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.897 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:7862 3.8 3:19e3d6c3:::benchmark_data_smithi074_35447_object982:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:47.898 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.904034+0000", 2020-08-07T15:06:47.898 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.851807194, 2020-08-07T15:06:47.899 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14287051000000001, 2020-08-07T15:06:47.899 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:47.899 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:47.899 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:47.900 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:47.900 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:47.900 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7862 2020-08-07T15:06:47.901 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.901 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:47.902 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.902 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:47.902 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.904034+0000", 2020-08-07T15:06:47.903 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:47.903 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.904 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:47.904 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.904034+0000", 2020-08-07T15:06:47.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1113999999999999e-05 2020-08-07T15:06:47.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:47.906 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.904085+0000", 2020-08-07T15:06:47.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1970000000000001e-06 2020-08-07T15:06:47.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:47.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.904091+0000", 2020-08-07T15:06:47.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2551000000000001e-05 2020-08-07T15:06:47.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:47.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.904103+0000", 2020-08-07T15:06:47.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2619999999999998e-06 2020-08-07T15:06:47.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:47.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.904109+0000", 2020-08-07T15:06:47.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00055555700000000003 2020-08-07T15:06:47.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:47.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.904664+0000", 2020-08-07T15:06:47.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8480999999999997e-05 2020-08-07T15:06:47.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.918 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:47.918 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.904733+0000", 2020-08-07T15:06:47.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000184766 2020-08-07T15:06:47.919 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:06:47.920 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.904918+0000", 2020-08-07T15:06:47.920 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0029007989999999999 2020-08-07T15:06:47.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:47.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.907818+0000", 2020-08-07T15:06:47.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3099999999999998e-07 2020-08-07T15:06:47.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:47.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.907819+0000", 2020-08-07T15:06:47.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4267000000000001e-05 2020-08-07T15:06:47.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:47.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.907873+0000", 2020-08-07T15:06:47.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012119469999999999 2020-08-07T15:06:47.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:47.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.909085+0000", 2020-08-07T15:06:47.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13774118299999999 2020-08-07T15:06:47.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:47.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.046826+0000", 2020-08-07T15:06:47.931 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3009999999999999e-06 2020-08-07T15:06:47.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.932 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:47.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.046827+0000", 2020-08-07T15:06:47.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6654000000000002e-05 2020-08-07T15:06:47.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.934 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.934 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:47.934 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.046904+0000", 2020-08-07T15:06:47.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14287051000000001 2020-08-07T15:06:47.935 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:47.935 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:47.936 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:47.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.937 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:7863 3.8 3:19e3d6c3:::benchmark_data_smithi074_35447_object982:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:47.937 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.904144+0000", 2020-08-07T15:06:47.937 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.8516967559999999, 2020-08-07T15:06:47.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14606724500000001, 2020-08-07T15:06:47.938 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:47.938 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:47.939 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:47.939 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:47.939 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:47.940 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7863 2020-08-07T15:06:47.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.940 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:47.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:47.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.904144+0000", 2020-08-07T15:06:47.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:47.942 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:47.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.904144+0000", 2020-08-07T15:06:47.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9619e-05 2020-08-07T15:06:47.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:47.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.904194+0000", 2020-08-07T15:06:47.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6949999999999997e-06 2020-08-07T15:06:47.946 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.946 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.946 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:47.947 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.904199+0000", 2020-08-07T15:06:47.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0501e-05 2020-08-07T15:06:47.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.948 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:47.948 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.904210+0000", 2020-08-07T15:06:47.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4649999999999996e-06 2020-08-07T15:06:47.949 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.950 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:47.950 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.904214+0000", 2020-08-07T15:06:47.950 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010671789999999999 2020-08-07T15:06:47.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:47.952 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.905281+0000", 2020-08-07T15:06:47.952 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010363 2020-08-07T15:06:47.952 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.953 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.953 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:47.953 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.905385+0000", 2020-08-07T15:06:47.954 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017612300000000001 2020-08-07T15:06:47.954 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.954 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.955 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:06:47.955 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.905561+0000", 2020-08-07T15:06:47.955 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002321432 2020-08-07T15:06:47.956 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.956 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.956 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:47.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.907883+0000", 2020-08-07T15:06:47.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9800000000000004e-07 2020-08-07T15:06:47.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.975 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:47.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.907883+0000", 2020-08-07T15:06:47.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4762000000000002e-05 2020-08-07T15:06:47.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.978 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:47.978 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.907938+0000", 2020-08-07T15:06:47.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001182925 2020-08-07T15:06:47.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:47.980 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.909121+0000", 2020-08-07T15:06:47.980 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.141045639 2020-08-07T15:06:47.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.981 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:47.981 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.050166+0000", 2020-08-07T15:06:47.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0900000000000001e-07 2020-08-07T15:06:47.982 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.982 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:47.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.050167+0000", 2020-08-07T15:06:47.982 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4067999999999999e-05 2020-08-07T15:06:47.983 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.983 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.983 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:47.983 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.050211+0000", 2020-08-07T15:06:47.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14606724500000001 2020-08-07T15:06:47.984 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:47.984 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:47.984 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:47.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.985 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:7864 3.8 3:19e3d6c3:::benchmark_data_smithi074_35447_object982:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:47.985 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.904248+0000", 2020-08-07T15:06:47.985 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.8515928869999998, 2020-08-07T15:06:47.985 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.146147058, 2020-08-07T15:06:47.986 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:47.987 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:47.987 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:47.987 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:47.987 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:47.987 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7864 2020-08-07T15:06:47.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.988 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:47.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:47.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.904248+0000", 2020-08-07T15:06:47.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:47.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:47.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.904248+0000", 2020-08-07T15:06:47.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0362999999999999e-05 2020-08-07T15:06:47.990 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.990 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.990 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:47.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.904298+0000", 2020-08-07T15:06:47.991 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0239999999999997e-06 2020-08-07T15:06:47.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:47.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.904303+0000", 2020-08-07T15:06:47.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0304e-05 2020-08-07T15:06:47.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:47.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.904314+0000", 2020-08-07T15:06:47.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4769999999999997e-06 2020-08-07T15:06:47.995 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.995 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.995 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:47.995 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.904318+0000", 2020-08-07T15:06:47.995 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001610642 2020-08-07T15:06:47.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.996 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.996 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:47.996 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.905929+0000", 2020-08-07T15:06:47.996 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9888000000000004e-05 2020-08-07T15:06:47.997 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.997 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.997 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:47.997 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.906019+0000", 2020-08-07T15:06:47.997 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000167787 2020-08-07T15:06:47.998 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.998 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.998 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:06:47.998 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.906186+0000", 2020-08-07T15:06:47.998 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017590329999999999 2020-08-07T15:06:47.998 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:47.999 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:47.999 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:47.999 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.907945+0000", 2020-08-07T15:06:48.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8799999999999998e-07 2020-08-07T15:06:48.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:48.004 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.907946+0000", 2020-08-07T15:06:48.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3075000000000003e-05 2020-08-07T15:06:48.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:48.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.907999+0000", 2020-08-07T15:06:48.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011599010000000001 2020-08-07T15:06:48.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:48.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.909159+0000", 2020-08-07T15:06:48.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14118561600000001 2020-08-07T15:06:48.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:48.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.050344+0000", 2020-08-07T15:06:48.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8500000000000002e-07 2020-08-07T15:06:48.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:48.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.050345+0000", 2020-08-07T15:06:48.008 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0074999999999997e-05 2020-08-07T15:06:48.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:48.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.050395+0000", 2020-08-07T15:06:48.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.146147058 2020-08-07T15:06:48.009 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:48.009 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:48.009 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:48.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.010 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:7862 3.8 e24/20 ondisk, result = 0)", 2020-08-07T15:06:48.010 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.912530+0000", 2020-08-07T15:06:48.010 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.8433105760000004, 2020-08-07T15:06:48.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13437780199999999, 2020-08-07T15:06:48.011 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:48.011 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:06:48.011 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:48.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:48.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.912530+0000", 2020-08-07T15:06:48.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:48.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.013 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:48.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.912530+0000", 2020-08-07T15:06:48.014 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4199000000000002e-05 2020-08-07T15:06:48.014 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.014 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:48.014 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.912564+0000", 2020-08-07T15:06:48.015 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1489999999999999e-06 2020-08-07T15:06:48.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.015 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.015 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:48.016 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.912571+0000", 2020-08-07T15:06:48.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13417991300000001 2020-08-07T15:06:48.016 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.016 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:48.016 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.046750+0000", 2020-08-07T15:06:48.017 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6579999999999999e-06 2020-08-07T15:06:48.017 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:48.017 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.046756+0000", 2020-08-07T15:06:48.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0067000000000001e-05 2020-08-07T15:06:48.018 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.018 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.018 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:48.018 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.046796+0000", 2020-08-07T15:06:48.019 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8780999999999999e-05 2020-08-07T15:06:48.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.019 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.019 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:48.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.046815+0000", 2020-08-07T15:06:48.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3035000000000006e-05 2020-08-07T15:06:48.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.020 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.020 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:48.020 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.046908+0000", 2020-08-07T15:06:48.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13437780199999999 2020-08-07T15:06:48.021 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:48.021 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:48.021 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:48.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.022 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:10880 3.8 3:1f8353e6:::benchmark_data_smithi074_35447_object1359:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:48.022 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.768861+0000", 2020-08-07T15:06:48.022 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4.986979754, 2020-08-07T15:06:48.022 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.113536312, 2020-08-07T15:06:48.022 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:48.023 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:48.023 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:48.024 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:48.024 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:48.024 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 10880 2020-08-07T15:06:48.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.024 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:48.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:48.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.768861+0000", 2020-08-07T15:06:48.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:48.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:48.026 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.768861+0000", 2020-08-07T15:06:48.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2180000000000004e-05 2020-08-07T15:06:48.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:48.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.768923+0000", 2020-08-07T15:06:48.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6489999999999996e-06 2020-08-07T15:06:48.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.030 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.030 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:48.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.768929+0000", 2020-08-07T15:06:48.030 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1892000000000001e-05 2020-08-07T15:06:48.030 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:48.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.768941+0000", 2020-08-07T15:06:48.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1911e-05 2020-08-07T15:06:48.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:48.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.768953+0000", 2020-08-07T15:06:48.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024148970000000001 2020-08-07T15:06:48.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:48.033 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.771368+0000", 2020-08-07T15:06:48.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000103866 2020-08-07T15:06:48.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.034 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.034 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:48.034 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.771471+0000", 2020-08-07T15:06:48.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00023276699999999999 2020-08-07T15:06:48.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:06:48.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.771704+0000", 2020-08-07T15:06:48.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001533696 2020-08-07T15:06:48.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:48.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.773238+0000", 2020-08-07T15:06:48.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5400000000000002e-07 2020-08-07T15:06:48.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.037 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:48.037 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.773238+0000", 2020-08-07T15:06:48.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3484e-05 2020-08-07T15:06:48.038 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.038 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:48.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.773272+0000", 2020-08-07T15:06:48.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.024435985 2020-08-07T15:06:48.039 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:48.040 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.797708+0000", 2020-08-07T15:06:48.040 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.084614228 2020-08-07T15:06:48.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.041 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:48.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.882322+0000", 2020-08-07T15:06:48.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6199999999999997e-07 2020-08-07T15:06:48.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.042 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:48.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.882322+0000", 2020-08-07T15:06:48.042 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4840999999999997e-05 2020-08-07T15:06:48.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:48.043 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.882397+0000", 2020-08-07T15:06:48.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.113536312 2020-08-07T15:06:48.043 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:48.044 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:48.044 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:48.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.044 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:11272 3.b 3:dd379a19:::benchmark_data_smithi074_35447_object1408:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:48.045 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:43.063400+0000", 2020-08-07T15:06:48.045 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4.6924404959999997, 2020-08-07T15:06:48.045 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.103479802, 2020-08-07T15:06:48.045 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:48.045 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:48.046 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:48.046 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:48.046 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:48.046 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 11272 2020-08-07T15:06:48.046 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.046 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:48.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.047 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:48.047 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:43.063400+0000", 2020-08-07T15:06:48.047 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:48.047 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.048 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.048 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:48.048 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:43.063400+0000", 2020-08-07T15:06:48.048 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4234999999999998e-05 2020-08-07T15:06:48.048 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.049 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.049 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:48.049 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:43.063455+0000", 2020-08-07T15:06:48.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7910000000000001e-06 2020-08-07T15:06:48.049 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.050 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.050 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:48.050 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:43.063460+0000", 2020-08-07T15:06:48.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2254e-05 2020-08-07T15:06:48.050 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.051 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:48.051 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:43.063473+0000", 2020-08-07T15:06:48.051 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5899999999999998e-06 2020-08-07T15:06:48.051 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.052 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.052 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:48.052 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:43.063478+0000", 2020-08-07T15:06:48.052 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00085836799999999998 2020-08-07T15:06:48.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.053 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:48.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:43.064337+0000", 2020-08-07T15:06:48.053 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4504000000000002e-05 2020-08-07T15:06:48.053 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.054 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:48.054 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:43.064431+0000", 2020-08-07T15:06:48.054 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00021003999999999999 2020-08-07T15:06:48.055 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.055 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:48.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:43.064641+0000", 2020-08-07T15:06:48.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001141967 2020-08-07T15:06:48.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:48.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:43.065783+0000", 2020-08-07T15:06:48.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3099999999999996e-07 2020-08-07T15:06:48.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:48.057 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:43.065784+0000", 2020-08-07T15:06:48.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3328999999999997e-05 2020-08-07T15:06:48.058 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.058 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.058 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:48.058 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:43.065837+0000", 2020-08-07T15:06:48.058 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043873599999999999 2020-08-07T15:06:48.059 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.059 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.059 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:48.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:43.070225+0000", 2020-08-07T15:06:48.059 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.096582638999999998 2020-08-07T15:06:48.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:48.060 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:43.166807+0000", 2020-08-07T15:06:48.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.389e-06 2020-08-07T15:06:48.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.061 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:48.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:43.166809+0000", 2020-08-07T15:06:48.062 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1404999999999998e-05 2020-08-07T15:06:48.062 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.062 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.062 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:48.062 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:43.166880+0000", 2020-08-07T15:06:48.062 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.103479802 2020-08-07T15:06:48.063 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:48.063 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:48.063 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:48.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.064 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:15512 3.7 3:e072e908:::benchmark_data_smithi074_35447_object1938:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e27)", 2020-08-07T15:06:48.064 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:46.629945+0000", 2020-08-07T15:06:48.064 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 1.125895761, 2020-08-07T15:06:48.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11894345200000001, 2020-08-07T15:06:48.064 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:48.065 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:48.065 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:48.065 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:48.065 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:48.065 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 15512 2020-08-07T15:06:48.066 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.066 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:48.066 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.066 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:48.066 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.629945+0000", 2020-08-07T15:06:48.067 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:48.067 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.067 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.067 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:48.068 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.629945+0000", 2020-08-07T15:06:48.068 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5476000000000001e-05 2020-08-07T15:06:48.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.068 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.069 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:48.069 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.629970+0000", 2020-08-07T15:06:48.069 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2179999999999999e-06 2020-08-07T15:06:48.069 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.069 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.070 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:48.070 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.629974+0000", 2020-08-07T15:06:48.070 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6450000000000002e-06 2020-08-07T15:06:48.070 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.070 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.070 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:48.071 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.629979+0000", 2020-08-07T15:06:48.071 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3120000000000001e-06 2020-08-07T15:06:48.071 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.071 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.071 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:48.072 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.629982+0000", 2020-08-07T15:06:48.072 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074144909999999996 2020-08-07T15:06:48.072 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.072 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.072 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:48.073 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.637396+0000", 2020-08-07T15:06:48.073 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010406100000000001 2020-08-07T15:06:48.073 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.073 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.073 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:48.074 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.637500+0000", 2020-08-07T15:06:48.074 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000175592 2020-08-07T15:06:48.074 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.074 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.074 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:06:48.075 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.637676+0000", 2020-08-07T15:06:48.075 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0057334700000000001 2020-08-07T15:06:48.075 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.075 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.075 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:48.075 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.643409+0000", 2020-08-07T15:06:48.076 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4100999999999998e-05 2020-08-07T15:06:48.076 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.076 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.076 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:48.076 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.643433+0000", 2020-08-07T15:06:48.077 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2138999999999996e-05 2020-08-07T15:06:48.077 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.077 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.077 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:48.077 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.643496+0000", 2020-08-07T15:06:48.078 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0021843639999999998 2020-08-07T15:06:48.078 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.078 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.078 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:48.078 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.645680+0000", 2020-08-07T15:06:48.079 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.10311376799999999 2020-08-07T15:06:48.079 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.079 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.079 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:48.079 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.748794+0000", 2020-08-07T15:06:48.079 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5799999999999998e-07 2020-08-07T15:06:48.080 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.080 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.080 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:48.080 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.748795+0000", 2020-08-07T15:06:48.080 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3956999999999995e-05 2020-08-07T15:06:48.081 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.081 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.081 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:48.081 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.748888+0000", 2020-08-07T15:06:48.081 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11894345200000001 2020-08-07T15:06:48.082 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:48.082 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:48.082 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:48.082 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.083 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:16120 3.b 3:d6e301ab:::benchmark_data_smithi074_35447_object2014:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:06:48.083 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.036311+0000", 2020-08-07T15:06:48.083 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.71952994100000001, 2020-08-07T15:06:48.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.119002674, 2020-08-07T15:06:48.083 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:48.083 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:48.084 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:48.084 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:48.084 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:48.084 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 16120 2020-08-07T15:06:48.084 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.085 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:48.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.085 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:48.085 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.036311+0000", 2020-08-07T15:06:48.085 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:48.086 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.086 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.086 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:48.086 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.036311+0000", 2020-08-07T15:06:48.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4678000000000001e-05 2020-08-07T15:06:48.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:48.087 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.036366+0000", 2020-08-07T15:06:48.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0549999999999999e-06 2020-08-07T15:06:48.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:48.088 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.036371+0000", 2020-08-07T15:06:48.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5187e-05 2020-08-07T15:06:48.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.089 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.089 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:48.089 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.036386+0000", 2020-08-07T15:06:48.089 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2689999999999999e-06 2020-08-07T15:06:48.089 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.090 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.090 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:48.090 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.036391+0000", 2020-08-07T15:06:48.090 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001415988 2020-08-07T15:06:48.090 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.090 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.091 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:48.091 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.037807+0000", 2020-08-07T15:06:48.091 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6477999999999993e-05 2020-08-07T15:06:48.091 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.091 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.092 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:48.092 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.037903+0000", 2020-08-07T15:06:48.092 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017799299999999999 2020-08-07T15:06:48.092 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.092 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.093 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:48.093 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.038081+0000", 2020-08-07T15:06:48.093 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013909300000000001 2020-08-07T15:06:48.093 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.093 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.094 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:48.094 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.039472+0000", 2020-08-07T15:06:48.094 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1799999999999995e-07 2020-08-07T15:06:48.094 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.094 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.094 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:48.095 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.039473+0000", 2020-08-07T15:06:48.095 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5359000000000003e-05 2020-08-07T15:06:48.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.095 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.095 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:48.096 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.039528+0000", 2020-08-07T15:06:48.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00042998199999999998 2020-08-07T15:06:48.096 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.096 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.096 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:48.097 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.039958+0000", 2020-08-07T15:06:48.097 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11531282499999999 2020-08-07T15:06:48.097 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.097 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.097 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:48.098 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.155271+0000", 2020-08-07T15:06:48.098 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5099999999999999e-06 2020-08-07T15:06:48.098 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.098 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.098 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:48.098 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.155273+0000", 2020-08-07T15:06:48.099 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0802e-05 2020-08-07T15:06:48.099 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.099 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.099 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:48.099 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.155314+0000", 2020-08-07T15:06:48.100 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.119002674 2020-08-07T15:06:48.100 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:48.100 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:48.100 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:48.100 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.101 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.101 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:16120 3.b e27/20 ondisk, result = 0)", 2020-08-07T15:06:48.101 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.041093+0000", 2020-08-07T15:06:48.101 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.71474796799999996, 2020-08-07T15:06:48.101 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.114226276, 2020-08-07T15:06:48.101 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:48.102 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:06:48.102 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:48.102 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.102 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:48.102 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.041093+0000", 2020-08-07T15:06:48.103 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:48.103 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.103 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.103 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:48.103 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.041093+0000", 2020-08-07T15:06:48.104 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0258000000000002e-05 2020-08-07T15:06:48.104 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.104 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.104 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:48.104 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.041113+0000", 2020-08-07T15:06:48.105 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8489999999999999e-06 2020-08-07T15:06:48.105 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.105 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.105 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:48.105 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.041117+0000", 2020-08-07T15:06:48.105 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11265335899999999 2020-08-07T15:06:48.106 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.106 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.106 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:48.106 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.153770+0000", 2020-08-07T15:06:48.106 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6990000000000002e-06 2020-08-07T15:06:48.107 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.107 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.107 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:48.107 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.153775+0000", 2020-08-07T15:06:48.107 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001484179 2020-08-07T15:06:48.108 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.108 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.108 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:48.108 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.155259+0000", 2020-08-07T15:06:48.108 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.925e-06 2020-08-07T15:06:48.108 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.109 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.109 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:48.109 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.155265+0000", 2020-08-07T15:06:48.109 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4007e-05 2020-08-07T15:06:48.110 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.110 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.110 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:48.111 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.155319+0000", 2020-08-07T15:06:48.111 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.114226276 2020-08-07T15:06:48.111 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:48.111 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:48.111 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:48.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.112 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:16416 3.b 3:d957dec9:::benchmark_data_smithi074_35447_object2051:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:06:48.112 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.233668+0000", 2020-08-07T15:06:48.112 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.52217242200000002, 2020-08-07T15:06:48.113 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.15147770799999999, 2020-08-07T15:06:48.113 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:48.113 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:48.113 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:48.113 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:48.113 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:48.114 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 16416 2020-08-07T15:06:48.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.114 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:48.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.114 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:48.115 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.233668+0000", 2020-08-07T15:06:48.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:48.115 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.115 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.115 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:48.116 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.233668+0000", 2020-08-07T15:06:48.116 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0918000000000003e-05 2020-08-07T15:06:48.116 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.116 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.116 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:48.116 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.233719+0000", 2020-08-07T15:06:48.117 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.981e-06 2020-08-07T15:06:48.117 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.117 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:48.117 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.233725+0000", 2020-08-07T15:06:48.118 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3123e-05 2020-08-07T15:06:48.118 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.118 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.118 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:48.118 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.233738+0000", 2020-08-07T15:06:48.119 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3300000000000004e-06 2020-08-07T15:06:48.119 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.119 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.119 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:48.119 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.233745+0000", 2020-08-07T15:06:48.120 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00064557899999999997 2020-08-07T15:06:48.120 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.120 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.120 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:48.120 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.234390+0000", 2020-08-07T15:06:48.120 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010767799999999999 2020-08-07T15:06:48.121 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.121 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.121 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:48.121 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.234498+0000", 2020-08-07T15:06:48.121 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020364800000000001 2020-08-07T15:06:48.122 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.122 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.122 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:48.122 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.234702+0000", 2020-08-07T15:06:48.122 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024297490000000001 2020-08-07T15:06:48.123 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.123 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.123 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:48.123 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.237131+0000", 2020-08-07T15:06:48.123 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5199999999999996e-07 2020-08-07T15:06:48.123 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.124 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.124 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:48.124 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.237132+0000", 2020-08-07T15:06:48.124 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4466999999999999e-05 2020-08-07T15:06:48.124 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.125 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.125 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:48.125 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.237186+0000", 2020-08-07T15:06:48.125 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14628802399999999 2020-08-07T15:06:48.125 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.126 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.126 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:48.126 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.383475+0000", 2020-08-07T15:06:48.126 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015972930000000001 2020-08-07T15:06:48.126 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.126 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.127 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:48.127 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.385072+0000", 2020-08-07T15:06:48.127 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8400000000000004e-07 2020-08-07T15:06:48.127 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.127 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.128 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:48.128 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.385072+0000", 2020-08-07T15:06:48.128 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3682000000000004e-05 2020-08-07T15:06:48.128 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.128 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.129 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:48.129 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.385146+0000", 2020-08-07T15:06:48.129 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.15147770799999999 2020-08-07T15:06:48.129 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:48.129 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:48.129 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:48.130 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.130 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.130 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:16416 3.b e27/20 ondisk, result = 0)", 2020-08-07T15:06:48.130 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.237170+0000", 2020-08-07T15:06:48.130 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.518671035, 2020-08-07T15:06:48.131 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14798144699999999, 2020-08-07T15:06:48.131 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:48.131 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:06:48.131 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:48.131 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.132 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:48.132 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.237170+0000", 2020-08-07T15:06:48.132 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:48.132 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.132 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.132 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:48.133 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.237170+0000", 2020-08-07T15:06:48.133 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1019000000000002e-05 2020-08-07T15:06:48.134 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.134 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.134 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:48.134 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.237201+0000", 2020-08-07T15:06:48.134 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6600000000000001e-06 2020-08-07T15:06:48.135 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.135 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.135 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:48.135 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.237204+0000", 2020-08-07T15:06:48.135 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14736898400000001 2020-08-07T15:06:48.135 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.136 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.136 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:48.136 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.384573+0000", 2020-08-07T15:06:48.136 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0209999999999999e-06 2020-08-07T15:06:48.136 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.137 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.137 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:48.137 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.384578+0000", 2020-08-07T15:06:48.138 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00047671399999999999 2020-08-07T15:06:48.138 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.138 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.139 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:48.139 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.385055+0000", 2020-08-07T15:06:48.139 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3410000000000003e-06 2020-08-07T15:06:48.140 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.140 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.140 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:48.140 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.385063+0000", 2020-08-07T15:06:48.141 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7707999999999994e-05 2020-08-07T15:06:48.141 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.141 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.142 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:48.142 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.385151+0000", 2020-08-07T15:06:48.142 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14798144699999999 2020-08-07T15:06:48.143 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:48.143 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:48.143 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:48.144 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.144 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.144 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:16664 3.8 3:1ae54aa8:::benchmark_data_smithi074_35447_object2082:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:06:48.145 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.401669+0000", 2020-08-07T15:06:48.145 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.35417205800000001, 2020-08-07T15:06:48.145 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.169055539, 2020-08-07T15:06:48.146 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:48.146 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:48.146 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:48.147 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:48.147 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:48.147 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 16664 2020-08-07T15:06:48.148 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.148 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:48.148 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.149 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:48.149 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.401669+0000", 2020-08-07T15:06:48.149 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:48.150 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.150 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.150 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:48.151 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.401669+0000", 2020-08-07T15:06:48.151 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5195000000000003e-05 2020-08-07T15:06:48.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:48.152 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.401724+0000", 2020-08-07T15:06:48.153 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1500000000000004e-06 2020-08-07T15:06:48.153 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.153 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.154 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:48.154 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.401730+0000", 2020-08-07T15:06:48.154 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7010000000000001e-05 2020-08-07T15:06:48.155 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.155 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.155 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:48.156 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.401747+0000", 2020-08-07T15:06:48.156 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8669999999999999e-06 2020-08-07T15:06:48.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.157 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.157 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:48.157 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.401753+0000", 2020-08-07T15:06:48.158 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00039438299999999999 2020-08-07T15:06:48.158 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.158 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.159 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:48.159 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.402147+0000", 2020-08-07T15:06:48.160 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1470000000000006e-05 2020-08-07T15:06:48.160 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.160 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.161 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:48.161 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.402209+0000", 2020-08-07T15:06:48.161 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000126897 2020-08-07T15:06:48.162 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.162 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.163 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:06:48.163 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.402336+0000", 2020-08-07T15:06:48.163 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0016153070000000001 2020-08-07T15:06:48.164 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:48.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.403951+0000", 2020-08-07T15:06:48.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7800000000000002e-07 2020-08-07T15:06:48.166 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.166 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.166 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:48.167 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.403951+0000", 2020-08-07T15:06:48.167 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00051122899999999998 2020-08-07T15:06:48.338 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.338 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.339 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:48.339 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.404463+0000", 2020-08-07T15:06:48.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003395412 2020-08-07T15:06:48.340 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.341 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:48.342 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.407858+0000", 2020-08-07T15:06:48.342 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.162811399 2020-08-07T15:06:48.342 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.343 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.343 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:48.344 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.570670+0000", 2020-08-07T15:06:48.344 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7400000000000002e-07 2020-08-07T15:06:48.345 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.345 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.346 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:48.346 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.570670+0000", 2020-08-07T15:06:48.346 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3968000000000002e-05 2020-08-07T15:06:48.347 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.348 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:48.348 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.570724+0000", 2020-08-07T15:06:48.349 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.169055539 2020-08-07T15:06:48.349 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:48.362 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:48.362 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:48.363 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.363 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.364 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:16664 3.8 e27/20 ondisk, result = 0)", 2020-08-07T15:06:48.364 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.404204+0000", 2020-08-07T15:06:48.364 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.35163662699999998, 2020-08-07T15:06:48.364 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16652569, 2020-08-07T15:06:48.365 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:48.365 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:06:48.366 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:48.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.367 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:48.367 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.404204+0000", 2020-08-07T15:06:48.367 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:48.368 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.368 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.369 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:48.370 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.404204+0000", 2020-08-07T15:06:48.370 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2997999999999999e-05 2020-08-07T15:06:48.370 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.371 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.371 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:48.371 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.404227+0000", 2020-08-07T15:06:48.371 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9450000000000003e-06 2020-08-07T15:06:48.372 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.372 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:48.373 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.404231+0000", 2020-08-07T15:06:48.373 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.166379266 2020-08-07T15:06:48.373 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.373 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.373 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:48.374 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.570610+0000", 2020-08-07T15:06:48.374 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1250000000000001e-06 2020-08-07T15:06:48.374 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.374 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.374 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:48.375 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.570615+0000", 2020-08-07T15:06:48.375 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2184999999999998e-05 2020-08-07T15:06:48.375 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.375 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:48.376 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.570648+0000", 2020-08-07T15:06:48.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.4339999999999992e-06 2020-08-07T15:06:48.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.376 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:48.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.570656+0000", 2020-08-07T15:06:48.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3737000000000002e-05 2020-08-07T15:06:48.377 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:48.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.570730+0000", 2020-08-07T15:06:48.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16652569 2020-08-07T15:06:48.387 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:48.388 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:48.388 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:48.388 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.388 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.388 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:16696 3.7 3:e04999bb:::benchmark_data_smithi074_35447_object2086:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:06:48.389 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.431131+0000", 2020-08-07T15:06:48.389 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.32470934899999998, 2020-08-07T15:06:48.390 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13184515499999999, 2020-08-07T15:06:48.390 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:48.390 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:48.390 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:48.390 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:48.391 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:48.391 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 16696 2020-08-07T15:06:48.391 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.391 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:48.391 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.392 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:48.392 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.431131+0000", 2020-08-07T15:06:48.392 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:48.392 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.392 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.393 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:48.393 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.431131+0000", 2020-08-07T15:06:48.393 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0340999999999998e-05 2020-08-07T15:06:48.393 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.393 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.394 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:48.394 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.431182+0000", 2020-08-07T15:06:48.394 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4980000000000003e-06 2020-08-07T15:06:48.394 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.394 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.395 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:48.395 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.431188+0000", 2020-08-07T15:06:48.395 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2058999999999999e-05 2020-08-07T15:06:48.395 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.396 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.396 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:48.396 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.431200+0000", 2020-08-07T15:06:48.396 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0023999999999999e-05 2020-08-07T15:06:48.396 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.397 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.397 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:48.397 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.431210+0000", 2020-08-07T15:06:48.397 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011306980000000001 2020-08-07T15:06:48.397 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.398 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.398 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:48.398 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.432341+0000", 2020-08-07T15:06:48.398 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9942e-05 2020-08-07T15:06:48.399 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.399 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:48.399 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.432441+0000", 2020-08-07T15:06:48.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016754000000000001 2020-08-07T15:06:48.400 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.400 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:06:48.401 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.432609+0000", 2020-08-07T15:06:48.401 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033759710000000002 2020-08-07T15:06:48.401 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.401 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.402 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:48.402 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.435985+0000", 2020-08-07T15:06:48.402 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.032e-06 2020-08-07T15:06:48.403 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.403 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.403 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:48.403 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.435986+0000", 2020-08-07T15:06:48.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9296999999999997e-05 2020-08-07T15:06:48.404 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.404 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.404 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:48.405 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.436055+0000", 2020-08-07T15:06:48.405 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.031683652 2020-08-07T15:06:48.405 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.406 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:48.406 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.467738+0000", 2020-08-07T15:06:48.406 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.095162991000000002 2020-08-07T15:06:48.406 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.407 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.407 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:48.407 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.562901+0000", 2020-08-07T15:06:48.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2219999999999999e-06 2020-08-07T15:06:48.408 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.408 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.408 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:48.408 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.562903+0000", 2020-08-07T15:06:48.409 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3887999999999995e-05 2020-08-07T15:06:48.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:48.409 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:48.410 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:48.410 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.562977+0000", 2020-08-07T15:06:48.410 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13184515499999999 2020-08-07T15:06:48.410 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:48.411 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:48.411 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:48.411 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:48.412 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:48.412 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:06:48.413 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:06:48.414 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:06:48.416 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 12 15 17987 17972 11.6995 12.1797 0.00757598 0.0106619 2020-08-07T15:06:48.416 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:48.412+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:06:48.417 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:06:48.515 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:48.514+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:48.568 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:06:48.569 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:06:48.569 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2020-08-07T15:06:48.569 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:06:48.586 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:06:48.632 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:48.631+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:06:48.712 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:06:48.717 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:06:48.716+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:06:48.738 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:06:48.738 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:06:48.739 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:06:48.739 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:06:48.739 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:06:48.749 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:06:48.758 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:06:48.786 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:06:48.813 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:06:48.845 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:06:48.853 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:48.851+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:48.882 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:06:48.919 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:48.918+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:48.921 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:06:48.966 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:48.966 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:48.967 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:06:48.968 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:06:48.968 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:06:48.968 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:06:48.969 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:06:48.969 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:06:48.969 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:06:48.979 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:06:48.980 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:06:48.981 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:06:49.025 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:06:49.022+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:06:49.122 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:49.119+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:49.208 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:49.209 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:49.212 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:06:49.213 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:06:49.222 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:06:49.221+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:06:49.241 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 13 16 19362 19346 11.6252 10.7344 0.00775014 0.0107359 2020-08-07T15:06:49.325 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:49.324+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:49.432 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:49.430+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:49.437 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:49.437 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:49.451 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:06:49.533 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:06:49.532+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:06:49.639 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:49.634+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:49.639 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:49.640 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:49.651 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:06:49.744 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:49.739+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:49.839 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:49.839 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:49.840 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:49.838+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:49.845 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:06:49.950 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:49.946+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:50.035 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:50.036 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:50.044 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:06:50.049 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:50.048+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:50.151 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:50.150+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:50.210 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:06:50.210 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:06:50.211 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump -f json-pretty 2020-08-07T15:06:50.244 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 14 16 20895 20879 11.6502 11.9766 0.0849962 0.0107199 2020-08-07T15:06:50.252 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:06:50.251+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:06:50.255 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:50.256 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:50.267 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:06:50.358 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:50.358+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:50.457 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:50.454+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:06:50.495 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:50.496 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:50.501 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:06:50.606 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:50.600+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:50.635 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:06:50.636 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:06:50.636 INFO:teuthology.orchestra.run.smithi074.stdout: "epoch": 28, 2020-08-07T15:06:50.636 INFO:teuthology.orchestra.run.smithi074.stdout: "fsid": "a01d3e82-8d24-4288-9e42-576ab3ff4957", 2020-08-07T15:06:50.637 INFO:teuthology.orchestra.run.smithi074.stdout: "created": "2020-08-07T15:06:07.342912+0000", 2020-08-07T15:06:50.637 INFO:teuthology.orchestra.run.smithi074.stdout: "modified": "2020-08-07T15:06:49.199135+0000", 2020-08-07T15:06:50.637 INFO:teuthology.orchestra.run.smithi074.stdout: "last_up_change": "2020-08-07T15:06:17.674292+0000", 2020-08-07T15:06:50.646 INFO:teuthology.orchestra.run.smithi074.stdout: "last_in_change": "2020-08-07T15:06:17.674292+0000", 2020-08-07T15:06:50.646 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-08-07T15:06:50.647 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_num": 5806080, 2020-08-07T15:06:50.654 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_set": [ 2020-08-07T15:06:50.654 INFO:teuthology.orchestra.run.smithi074.stdout: "nodeep-scrub", 2020-08-07T15:06:50.655 INFO:teuthology.orchestra.run.smithi074.stdout: "noscrub", 2020-08-07T15:06:50.667 INFO:teuthology.orchestra.run.smithi074.stdout: "pglog_hardlimit", 2020-08-07T15:06:50.670 INFO:teuthology.orchestra.run.smithi074.stdout: "purged_snapdirs", 2020-08-07T15:06:50.670 INFO:teuthology.orchestra.run.smithi074.stdout: "recovery_deletes", 2020-08-07T15:06:50.671 INFO:teuthology.orchestra.run.smithi074.stdout: "sortbitwise" 2020-08-07T15:06:50.671 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:06:50.671 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_version": 5, 2020-08-07T15:06:50.672 INFO:teuthology.orchestra.run.smithi074.stdout: "full_ratio": 0.94999998807907104, 2020-08-07T15:06:50.672 INFO:teuthology.orchestra.run.smithi074.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-08-07T15:06:50.674 INFO:teuthology.orchestra.run.smithi074.stdout: "nearfull_ratio": 0.85000002384185791, 2020-08-07T15:06:50.674 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_snapshot": "", 2020-08-07T15:06:50.703 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_max": 3, 2020-08-07T15:06:50.703 INFO:teuthology.orchestra.run.smithi074.stdout: "max_osd": 8, 2020-08-07T15:06:50.704 INFO:teuthology.orchestra.run.smithi074.stdout: "require_min_compat_client": "luminous", 2020-08-07T15:06:50.704 INFO:teuthology.orchestra.run.smithi074.stdout: "min_compat_client": "jewel", 2020-08-07T15:06:50.705 INFO:teuthology.orchestra.run.smithi074.stdout: "require_osd_release": "pacific", 2020-08-07T15:06:50.705 INFO:teuthology.orchestra.run.smithi074.stdout: "pools": [ 2020-08-07T15:06:50.705 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:50.719 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 1, 2020-08-07T15:06:50.720 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "device_health_metrics", 2020-08-07T15:06:50.720 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:06:10.648876+0000", 2020-08-07T15:06:50.720 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:06:50.721 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:06:50.721 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:06:50.721 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:06:50.722 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:06:50.722 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:06:50.723 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:06:50.723 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:06:50.723 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 1, 2020-08-07T15:06:50.724 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 1, 2020-08-07T15:06:50.724 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 1, 2020-08-07T15:06:50.724 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 1, 2020-08-07T15:06:50.725 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 1, 2020-08-07T15:06:50.725 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:06:50.725 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:06:50.725 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:06:50.739 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:06:50.739 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:06:50.739 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:06:50.740 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:06:50.740 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:50.742 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "16", 2020-08-07T15:06:50.742 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:06:50.744 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:06:50.744 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "0", 2020-08-07T15:06:50.745 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:06:50.745 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:06:50.745 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:06:50.746 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:06:50.746 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:06:50.747 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:06:50.747 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:06:50.747 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:06:50.748 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:06:50.761 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:06:50.761 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:06:50.762 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:06:50.822 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:06:50.823 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:06:50.823 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:06:50.824 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:06:50.824 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:06:50.824 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:06:50.825 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:06:50.825 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:06:50.825 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:06:50.874 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:06:50.875 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:06:50.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:50.875 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:06:50.876 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:06:50.876 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:06:50.877 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:06:50.877 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:06:50.877 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:06:50.915 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:06:50.916 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:06:50.916 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:06:50.916 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:06:50.916 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:06:50.917 INFO:teuthology.orchestra.run.smithi074.stdout: "options": { 2020-08-07T15:06:50.917 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_min": 1 2020-08-07T15:06:50.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:50.922 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:06:50.923 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_devicehealth": {} 2020-08-07T15:06:50.923 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:50.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:50.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:50.924 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 2, 2020-08-07T15:06:50.924 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "rbd", 2020-08-07T15:06:50.924 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:06:21.474648+0000", 2020-08-07T15:06:50.925 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:06:50.925 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:06:50.951 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:06:50.952 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:06:50.952 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:06:50.952 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:06:50.953 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:06:50.953 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:06:50.953 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 8, 2020-08-07T15:06:50.963 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 8, 2020-08-07T15:06:50.964 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 8, 2020-08-07T15:06:50.964 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 8, 2020-08-07T15:06:50.964 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 8, 2020-08-07T15:06:50.965 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:06:50.965 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:06:50.966 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:06:50.966 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:06:50.966 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:06:50.967 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:06:50.967 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:06:50.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:50.968 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "19", 2020-08-07T15:06:51.004 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:06:51.004 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:06:51.004 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "0", 2020-08-07T15:06:51.005 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:06:51.015 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:06:51.016 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:06:51.025 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:06:51.026 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:06:51.026 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:06:51.026 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:06:51.026 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:06:51.027 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:06:51.027 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:06:51.027 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:06:51.027 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:06:51.027 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:06:51.028 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:06:51.028 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:06:51.028 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:06:51.028 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:06:51.028 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:06:51.029 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:06:51.029 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:06:51.029 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:06:51.029 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:06:51.029 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:06:51.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.053 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:06:51.053 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:06:51.053 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:06:51.053 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:06:51.054 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:06:51.054 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:06:51.054 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:06:51.054 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:06:51.054 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:06:51.054 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:06:51.055 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:06:51.055 INFO:teuthology.orchestra.run.smithi074.stdout: "options": {}, 2020-08-07T15:06:51.055 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:06:51.055 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd": {} 2020-08-07T15:06:51.063 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.063 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 3, 2020-08-07T15:06:51.064 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "unique_pool_0", 2020-08-07T15:06:51.064 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:06:33.421788+0000", 2020-08-07T15:06:51.064 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:06:51.064 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:06:51.064 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:06:51.065 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:06:51.065 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:06:51.065 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:06:51.065 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:06:51.065 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:06:51.082 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 16, 2020-08-07T15:06:51.082 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 16, 2020-08-07T15:06:51.082 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 16, 2020-08-07T15:06:51.083 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 16, 2020-08-07T15:06:51.083 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 16, 2020-08-07T15:06:51.085 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:06:51.085 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:06:51.085 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:06:51.094 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:06:51.094 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:06:51.094 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:06:51.095 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:06:51.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.095 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "22", 2020-08-07T15:06:51.095 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:06:51.095 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:06:51.096 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "0", 2020-08-07T15:06:51.096 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:06:51.096 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:06:51.096 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:06:51.096 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:06:51.097 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:06:51.097 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:06:51.097 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:06:51.097 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:06:51.097 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:06:51.098 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:06:51.098 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:06:51.098 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:06:51.098 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:06:51.098 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:06:51.099 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:06:51.099 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:06:51.099 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:06:51.099 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:06:51.099 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:06:51.099 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:06:51.100 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:06:51.100 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:06:51.100 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:06:51.100 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.100 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:06:51.101 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:06:51.101 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:06:51.101 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:06:51.101 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:06:51.101 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:06:51.145 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:06:51.145 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:06:51.146 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:06:51.147 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:06:51.147 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:06:51.147 INFO:teuthology.orchestra.run.smithi074.stdout: "options": {}, 2020-08-07T15:06:51.148 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:06:51.148 INFO:teuthology.orchestra.run.smithi074.stdout: "rados": {} 2020-08-07T15:06:51.148 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.149 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.149 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:06:51.149 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:06:51.150 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.151 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 0, 2020-08-07T15:06:51.151 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "fc1ebb23-3ef4-4fc0-b395-382e385885a3", 2020-08-07T15:06:51.151 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:06:51.152 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:06:51.162 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:06:51.162 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0, 2020-08-07T15:06:51.163 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:06:51.163 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:06:51.164 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:06:51.164 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 20, 2020-08-07T15:06:51.164 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:06:51.171 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:06:51.171 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:06:51.171 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.172 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.172 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.172 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6801", 2020-08-07T15:06:51.172 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:06:51.180 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.180 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.181 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:06:51.181 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.181 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.192 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.192 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6802", 2020-08-07T15:06:51.192 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:06:51.193 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.193 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.193 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.193 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:06:51.200 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.200 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.215 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.215 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6804", 2020-08-07T15:06:51.215 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:06:51.216 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.216 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.216 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.216 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:06:51.217 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.217 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.217 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.217 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6803", 2020-08-07T15:06:51.217 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:06:51.218 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.218 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.218 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.218 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6801/31577", 2020-08-07T15:06:51.218 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6802/31577", 2020-08-07T15:06:51.226 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6804/31577", 2020-08-07T15:06:51.226 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6803/31577", 2020-08-07T15:06:51.227 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:06:51.227 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:06:51.227 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:06:51.228 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.228 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.228 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.228 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 1, 2020-08-07T15:06:51.228 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "1fbe530a-63ce-4dc8-82ab-18793d6ebc0b", 2020-08-07T15:06:51.229 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:06:51.229 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:06:51.229 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:06:51.229 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0.7196044921875, 2020-08-07T15:06:51.242 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:06:51.243 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:06:51.243 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:06:51.243 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 26, 2020-08-07T15:06:51.244 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:06:51.244 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:06:51.244 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:06:51.245 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.245 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.245 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.246 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6809", 2020-08-07T15:06:51.246 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:06:51.246 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.246 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.246 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.247 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:06:51.247 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.247 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.247 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.247 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6810", 2020-08-07T15:06:51.248 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:06:51.248 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.248 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.248 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.248 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:06:51.249 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.249 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.249 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.249 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6812", 2020-08-07T15:06:51.249 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:06:51.266 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.266 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.266 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.266 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:06:51.267 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.268 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.268 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.268 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6811", 2020-08-07T15:06:51.275 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:06:51.275 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.275 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.276 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.276 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6809/31578", 2020-08-07T15:06:51.276 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6810/31578", 2020-08-07T15:06:51.276 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6812/31578", 2020-08-07T15:06:51.276 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6811/31578", 2020-08-07T15:06:51.277 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:06:51.277 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:06:51.277 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:06:51.277 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.277 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.278 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.278 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 2, 2020-08-07T15:06:51.278 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "d6fe4338-3e11-4f73-9cbb-e1b046ce286a", 2020-08-07T15:06:51.278 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:06:51.278 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:06:51.279 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:06:51.279 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:06:51.279 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:06:51.279 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:06:51.279 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:06:51.279 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 20, 2020-08-07T15:06:51.280 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:06:51.280 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:06:51.280 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:06:51.280 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.280 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.281 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.281 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6805", 2020-08-07T15:06:51.281 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:06:51.282 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.283 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.283 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.283 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:06:51.284 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.284 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.284 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.285 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6806", 2020-08-07T15:06:51.291 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:06:51.291 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.291 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.291 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.292 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:06:51.292 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.292 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.292 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.292 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6808", 2020-08-07T15:06:51.293 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:06:51.293 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.293 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.293 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.293 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:06:51.294 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.294 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.294 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.294 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6807", 2020-08-07T15:06:51.294 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:06:51.295 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.295 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.295 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.295 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6805/31576", 2020-08-07T15:06:51.295 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6806/31576", 2020-08-07T15:06:51.308 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6808/31576", 2020-08-07T15:06:51.308 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6807/31576", 2020-08-07T15:06:51.308 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:06:51.308 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:06:51.309 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:06:51.309 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.309 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.309 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.309 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 3, 2020-08-07T15:06:51.310 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9", 2020-08-07T15:06:51.310 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:06:51.310 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:06:51.310 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:06:51.310 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:06:51.310 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:06:51.311 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:06:51.311 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:06:51.311 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 20, 2020-08-07T15:06:51.311 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:06:51.311 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:06:51.312 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:06:51.312 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.312 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.312 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.312 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6813", 2020-08-07T15:06:51.313 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:06:51.313 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.313 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.313 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.313 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:06:51.314 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.314 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.314 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.314 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6814", 2020-08-07T15:06:51.314 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:06:51.314 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.315 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.315 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.315 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:06:51.315 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.315 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.316 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.316 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6816", 2020-08-07T15:06:51.316 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:06:51.316 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.316 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.317 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.317 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:06:51.317 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.317 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.317 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.329 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6815", 2020-08-07T15:06:51.330 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:06:51.330 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.330 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.331 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.332 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6813/31579", 2020-08-07T15:06:51.340 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6814/31579", 2020-08-07T15:06:51.340 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6816/31579", 2020-08-07T15:06:51.341 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6815/31579", 2020-08-07T15:06:51.341 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:06:51.342 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:06:51.342 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:06:51.343 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.343 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.344 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.344 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 4, 2020-08-07T15:06:51.344 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "01aadc13-29c5-4ddd-9bb9-de24b2b20412", 2020-08-07T15:06:51.344 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:06:51.344 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:06:51.345 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:06:51.345 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:06:51.345 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:06:51.345 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:06:51.345 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:06:51.346 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 26, 2020-08-07T15:06:51.346 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:06:51.346 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:06:51.346 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:06:51.346 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.347 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.347 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6800", 2020-08-07T15:06:51.347 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:06:51.347 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.347 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.348 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.348 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:06:51.348 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.348 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.348 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.349 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6801", 2020-08-07T15:06:51.349 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:06:51.349 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.349 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.367 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.367 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:06:51.367 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.367 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.367 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.368 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6803", 2020-08-07T15:06:51.368 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:06:51.368 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.368 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.368 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.369 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:06:51.369 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.369 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.369 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6802", 2020-08-07T15:06:51.382 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:06:51.382 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.385 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.385 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.386 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6800/31545", 2020-08-07T15:06:51.387 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6801/31545", 2020-08-07T15:06:51.387 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6803/31545", 2020-08-07T15:06:51.387 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6802/31545", 2020-08-07T15:06:51.387 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:06:51.388 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:06:51.388 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:06:51.388 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.389 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.389 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.389 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 5, 2020-08-07T15:06:51.389 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "f5556a82-73b4-47b1-8e29-1d2fe031d8fd", 2020-08-07T15:06:51.390 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:06:51.391 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:06:51.391 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:06:51.391 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:06:51.392 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:06:51.392 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:06:51.392 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:06:51.392 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 20, 2020-08-07T15:06:51.393 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:06:51.393 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:06:51.393 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:06:51.393 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.406 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.406 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.407 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6804", 2020-08-07T15:06:51.407 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:06:51.407 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.408 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.410 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.410 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:06:51.411 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.411 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.411 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.411 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6805", 2020-08-07T15:06:51.411 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:06:51.412 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.412 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.412 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.412 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:06:51.412 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.439 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.440 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.447 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6807", 2020-08-07T15:06:51.447 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:06:51.448 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.449 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.449 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:06:51.450 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.450 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.451 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6806", 2020-08-07T15:06:51.451 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:06:51.451 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.452 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.482 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.490 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6804/31547", 2020-08-07T15:06:51.491 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6805/31547", 2020-08-07T15:06:51.491 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6807/31547", 2020-08-07T15:06:51.491 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6806/31547", 2020-08-07T15:06:51.492 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:06:51.492 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:06:51.492 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:06:51.492 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.493 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.493 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 6, 2020-08-07T15:06:51.493 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "9b98b337-e1cc-43cd-9e9b-f21ae3f20899", 2020-08-07T15:06:51.493 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:06:51.506 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:06:51.507 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:06:51.507 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:06:51.507 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:06:51.507 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:06:51.507 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 14, 2020-08-07T15:06:51.508 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 20, 2020-08-07T15:06:51.508 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:06:51.508 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:06:51.508 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:06:51.508 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.509 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.509 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.509 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6808", 2020-08-07T15:06:51.509 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:06:51.509 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.510 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.510 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.510 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:06:51.510 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.511 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.511 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6809", 2020-08-07T15:06:51.511 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:06:51.511 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.512 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.512 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:06:51.512 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.512 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.513 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.513 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6811", 2020-08-07T15:06:51.513 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:06:51.550 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.558 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.571 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.572 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:06:51.572 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.572 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.573 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.573 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6810", 2020-08-07T15:06:51.591 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:06:51.591 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.591 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.601 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6808/31546", 2020-08-07T15:06:51.602 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6809/31546", 2020-08-07T15:06:51.613 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6811/31546", 2020-08-07T15:06:51.614 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6810/31546", 2020-08-07T15:06:51.614 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:06:51.614 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:06:51.614 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:06:51.615 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.615 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.615 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.615 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 7, 2020-08-07T15:06:51.615 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "777662d8-bd34-48e8-83e1-24ce9e61f14b", 2020-08-07T15:06:51.616 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:06:51.616 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:06:51.616 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:06:51.616 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:06:51.617 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:06:51.617 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:06:51.638 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:06:51.639 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 26, 2020-08-07T15:06:51.639 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:06:51.639 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:06:51.640 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:06:51.640 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.640 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.640 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.640 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6812", 2020-08-07T15:06:51.641 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:06:51.641 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.641 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.641 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.641 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:06:51.642 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.642 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.642 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.642 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6813", 2020-08-07T15:06:51.643 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:06:51.643 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.643 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.643 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:06:51.643 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.644 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.644 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.644 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6815", 2020-08-07T15:06:51.644 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:06:51.644 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.645 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.645 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:06:51.645 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:06:51.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.646 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:06:51.646 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6814", 2020-08-07T15:06:51.646 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:06:51.646 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.647 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.647 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.647 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6812/31570", 2020-08-07T15:06:51.647 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6813/31570", 2020-08-07T15:06:51.647 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6815/31570", 2020-08-07T15:06:51.648 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6814/31570", 2020-08-07T15:06:51.648 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:06:51.648 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:06:51.648 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:06:51.648 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:51.649 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.649 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:06:51.649 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_xinfo": [ 2020-08-07T15:06:51.649 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.649 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 0, 2020-08-07T15:06:51.650 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:06:51.650 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:06:51.650 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:06:51.651 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:06:51.651 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:06:51.651 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:06:15.298063+0000", 2020-08-07T15:06:51.651 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:06:51.651 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.651 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.652 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 1, 2020-08-07T15:06:51.652 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:06:51.652 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:06:51.652 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:06:51.652 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:06:51.653 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:06:51.653 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:06:15.577450+0000", 2020-08-07T15:06:51.653 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:06:51.653 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.653 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.654 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 2, 2020-08-07T15:06:51.654 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:06:51.654 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:06:51.654 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:06:51.654 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:06:51.654 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:06:51.655 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:06:15.542874+0000", 2020-08-07T15:06:51.655 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:06:51.655 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.656 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.656 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 3, 2020-08-07T15:06:51.656 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:06:51.657 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:06:51.657 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:06:51.657 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:06:51.658 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:06:51.658 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:06:15.493720+0000", 2020-08-07T15:06:51.658 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:06:51.659 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.659 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.659 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 4, 2020-08-07T15:06:51.660 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:06:51.660 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:06:51.660 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:06:51.660 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:06:51.663 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:06:51.664 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:06:15.534281+0000", 2020-08-07T15:06:51.664 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:06:51.665 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.665 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.665 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 5, 2020-08-07T15:06:51.666 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:06:51.666 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:06:51.666 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:06:51.667 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:06:51.691 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:06:51.692 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:06:15.601875+0000", 2020-08-07T15:06:51.695 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:06:51.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.698 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.699 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 6, 2020-08-07T15:06:51.699 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:06:51.699 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:06:51.699 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:06:51.699 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:06:51.700 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:06:51.700 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:06:15.340718+0000", 2020-08-07T15:06:51.700 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:06:51.700 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:51.701 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 7, 2020-08-07T15:06:51.701 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:06:51.701 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:06:51.701 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:06:51.701 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:06:51.713 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:06:51.714 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:06:15.549109+0000", 2020-08-07T15:06:51.716 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:06:51.716 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.716 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:06:51.716 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_upmap": [], 2020-08-07T15:06:51.721 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_upmap_items": [], 2020-08-07T15:06:51.722 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_temp": [], 2020-08-07T15:06:51.722 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_temp": [], 2020-08-07T15:06:51.722 INFO:teuthology.orchestra.run.smithi074.stdout: "blacklist": {}, 2020-08-07T15:06:51.722 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profiles": { 2020-08-07T15:06:51.722 INFO:teuthology.orchestra.run.smithi074.stdout: "default": { 2020-08-07T15:06:51.723 INFO:teuthology.orchestra.run.smithi074.stdout: "crush-failure-domain": "osd", 2020-08-07T15:06:51.723 INFO:teuthology.orchestra.run.smithi074.stdout: "k": "2", 2020-08-07T15:06:51.723 INFO:teuthology.orchestra.run.smithi074.stdout: "m": "1", 2020-08-07T15:06:51.723 INFO:teuthology.orchestra.run.smithi074.stdout: "plugin": "jerasure", 2020-08-07T15:06:51.723 INFO:teuthology.orchestra.run.smithi074.stdout: "ruleset-failure-domain": "osd", 2020-08-07T15:06:51.724 INFO:teuthology.orchestra.run.smithi074.stdout: "technique": "reed_sol_van" 2020-08-07T15:06:51.724 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:51.724 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:51.724 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps_queue": [], 2020-08-07T15:06:51.772 INFO:teuthology.orchestra.run.smithi074.stdout: "new_removed_snaps": [], 2020-08-07T15:06:51.772 INFO:teuthology.orchestra.run.smithi074.stdout: "new_purged_snaps": [], 2020-08-07T15:06:51.773 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_node_flags": {}, 2020-08-07T15:06:51.773 INFO:teuthology.orchestra.run.smithi074.stdout: "device_class_flags": {} 2020-08-07T15:06:51.773 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:06:51.775 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:06:51.776 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:06:51.797 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:51.797 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:51.798 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:06:51.799 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:51.775+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:51.810 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 15 16 22627 22611 11.7756 13.5312 0.00331902 0.0106043 2020-08-07T15:06:51.812 INFO:tasks.thrashosds.thrasher:j is {'epoch': 28, 'fsid': 'a01d3e82-8d24-4288-9e42-576ab3ff4957', 'created': '2020-08-07T15:06:07.342912+0000', 'modified': '2020-08-07T15:06:49.199135+0000', 'last_up_change': '2020-08-07T15:06:17.674292+0000', 'last_in_change': '2020-08-07T15:06:17.674292+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-08-07T15:06:10.648876+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-08-07T15:06:21.474648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2020-08-07T15:06:33.421788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': 'fc1ebb23-3ef4-4fc0-b395-382e385885a3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6801', 'nonce': 31577}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6802', 'nonce': 31577}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6804', 'nonce': 31577}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6803', 'nonce': 31577}]}, 'public_addr': '172.21.15.74:6801/31577', 'cluster_addr': '172.21.15.74:6802/31577', 'heartbeat_back_addr': '172.21.15.74:6804/31577', 'heartbeat_front_addr': '172.21.15.74:6803/31577', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '1fbe530a-63ce-4dc8-82ab-18793d6ebc0b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.7196044921875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6809', 'nonce': 31578}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6810', 'nonce': 31578}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6812', 'nonce': 31578}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6811', 'nonce': 31578}]}, 'public_addr': '172.21.15.74:6809/31578', 'cluster_addr': '172.21.15.74:6810/31578', 'heartbeat_back_addr': '172.21.15.74:6812/31578', 'heartbeat_front_addr': '172.21.15.74:6811/31578', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd6fe4338-3e11-4f73-9cbb-e1b046ce286a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6805', 'nonce': 31576}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6806', 'nonce': 31576}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6808', 'nonce': 31576}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6807', 'nonce': 31576}]}, 'public_addr': '172.21.15.74:6805/31576', 'cluster_addr': '172.21.15.74:6806/31576', 'heartbeat_back_addr': '172.21.15.74:6808/31576', 'heartbeat_front_addr': '172.21.15.74:6807/31576', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6813', 'nonce': 31579}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6814', 'nonce': 31579}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6816', 'nonce': 31579}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6815', 'nonce': 31579}]}, 'public_addr': '172.21.15.74:6813/31579', 'cluster_addr': '172.21.15.74:6814/31579', 'heartbeat_back_addr': '172.21.15.74:6816/31579', 'heartbeat_front_addr': '172.21.15.74:6815/31579', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '01aadc13-29c5-4ddd-9bb9-de24b2b20412', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6800', 'nonce': 31545}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6801', 'nonce': 31545}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6803', 'nonce': 31545}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6802', 'nonce': 31545}]}, 'public_addr': '172.21.15.96:6800/31545', 'cluster_addr': '172.21.15.96:6801/31545', 'heartbeat_back_addr': '172.21.15.96:6803/31545', 'heartbeat_front_addr': '172.21.15.96:6802/31545', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'f5556a82-73b4-47b1-8e29-1d2fe031d8fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6804', 'nonce': 31547}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6805', 'nonce': 31547}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6807', 'nonce': 31547}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6806', 'nonce': 31547}]}, 'public_addr': '172.21.15.96:6804/31547', 'cluster_addr': '172.21.15.96:6805/31547', 'heartbeat_back_addr': '172.21.15.96:6807/31547', 'heartbeat_front_addr': '172.21.15.96:6806/31547', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9b98b337-e1cc-43cd-9e9b-f21ae3f20899', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6808', 'nonce': 31546}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6809', 'nonce': 31546}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6811', 'nonce': 31546}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6810', 'nonce': 31546}]}, 'public_addr': '172.21.15.96:6808/31546', 'cluster_addr': '172.21.15.96:6809/31546', 'heartbeat_back_addr': '172.21.15.96:6811/31546', 'heartbeat_front_addr': '172.21.15.96:6810/31546', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '777662d8-bd34-48e8-83e1-24ce9e61f14b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6812', 'nonce': 31570}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6813', 'nonce': 31570}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6815', 'nonce': 31570}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6814', 'nonce': 31570}]}, 'public_addr': '172.21.15.96:6812/31570', 'cluster_addr': '172.21.15.96:6813/31570', 'heartbeat_back_addr': '172.21.15.96:6815/31570', 'heartbeat_front_addr': '172.21.15.96:6814/31570', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:06:15.298063+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:06:15.577450+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:06:15.542874+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:06:15.493720+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:06:15.534281+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:06:15.601875+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:06:15.340718+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:06:15.549109+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blacklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}} 2020-08-07T15:06:51.813 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:06:51.883 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:51.880+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:51.980 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:06:51.978+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:06:51.994 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:51.995 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:52.008 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:06:52.066 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:06:52.086 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:52.082+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:52.106 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:06:52.155 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:06:52.190 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:52.186+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:52.197 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:06:52.234 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:06:52.234 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:06:52.235 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:06:52.235 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:06:52.240 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:06:52.241 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 16 16 24095 24079 11.7563 11.4688 0.00820218 0.0106124 2020-08-07T15:06:52.261 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:06:52.298 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:52.291+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:52.312 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:06:52.313 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:06:52.314 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:06:52.330 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":46,"stamp":"2020-08-07T15:06:50.617640+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":129064960,"num_objects":1972,"num_object_clones":0,"num_object_copies":5916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15757,"num_write_kb":126042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":75,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8761972,"kb_used_data":373300,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746212748,"statfs":{"total":773094113280,"available":764121853952,"internally_reserved":8589934592,"allocated":382259200,"data_stored":380967248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,1,8,0,1,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":54026240,"num_objects":825,"num_object_clones":0,"num_object_copies":2475,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6595,"num_write_kb":52760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001094"},"pg_stats":[{"pgid":"3.f","version":"27'1144","reported_seq":"1158","reported_epoch":"27","state":"active+clean","last_fresh":"2020-08-07T15:06:46.594686+0000","last_change":"2020-08-07T15:06:35.163498+0000","last_active":"2020-08-07T15:06:46.594686+0000","last_peered":"2020-08-07T15:06:46.594686+0000","last_clean":"2020-08-07T15:06:46.594686+0000","last_became_active":"2020-08-07T15:06:35.162764+0000","last_became_peered":"2020-08-07T15:06:35.162764+0000","last_unstale":"2020-08-07T15:06:46.594686+0000","last_undegraded":"2020-08-07T15:06:46.594686+0000","last_fullsized":"2020-08-07T15:06:46.594686+0000","mapping_epoch":20,"log_start":"27'1136","ondisk_log_start":"27'1136","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":9371648,"num_objects":143,"num_object_clones":0,"num_object_copies":429,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1144,"num_write_kb":9152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"27'1032","reported_seq":"1046","reported_epoch":"27","state":"active+clean","last_fresh":"2020-08-07T15:06:46.673651+0000","last_change":"2020-08-07T15:06:35.169137+0000","last_active":"2020-08-07T15:06:46.673651+0000","last_peered":"2020-08-07T15:06:46.673651+0000","last_clean":"2020-08-07T15:06:46.673651+0000","last_became_active":"2020-08-07T15:06:35.168689+0000","last_became_peered":"2020-08-07T15:06:35.168689+0000","last_unstale":"2020-08-07T15:06:46.673651+0000","last_undegraded":"2020-08-07T15:06:46.673651+0000","last_fullsized":"2020-08-07T15:06:46.673651+0000","mapping_epoch":20,"log_start":"27'1024","ondisk_log_start":"27'1024","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8454144,"num_objects":129,"num_object_clones":0,"num_object_copies":387,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1032,"num_write_kb":8256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"27'944","reported_seq":"958","reported_epoch":"27","state":"active+clean","last_fresh":"2020-08-07T15:06:46.672546+0000","last_change":"2020-08-07T15:06:35.165586+0000","last_active":"2020-08-07T15:06:46.672546+0000","last_peered":"2020-08-07T15:06:46.672546+0000","last_clean":"2020-08-07T15:06:46.672546+0000","last_became_active":"2020-08-07T15:06:35.165144+0000","last_became_peered":"2020-08-07T15:06:35.165144+0000","last_unstale":"2020-08-07T15:06:46.672546+0000","last_undegraded":"2020-08-07T15:06:46.672546+0000","last_fullsized":"2020-08-07T15:06:46.672546+0000","mapping_epoch":20,"log_start":"27'938","ondisk_log_start":"27'938","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7733248,"num_objects":118,"num_object_clones":0,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":944,"num_write_kb":7552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"27'1104","reported_seq":"1119","reported_epoch":"27","state":"active+clean","last_fresh":"2020-08-07T15:06:46.694427+0000","last_change":"2020-08-07T15:06:35.167480+0000","last_active":"2020-08-07T15:06:46.694427+0000","last_peered":"2020-08-07T15:06:46.694427+0000","last_clean":"2020-08-07T15:06:46.694427+0000","last_became_active":"2020-08-07T15:06:35.167101+0000","last_became_peered":"2020-08-07T15:06:35.167101+0000","last_unstale":"2020-08-07T15:06:46.694427+0000","last_undegraded":"2020-08-07T15:06:46.694427+0000","last_fullsized":"2020-08-07T15:06:46.694427+0000","mapping_epoch":20,"log_start":"27'1100","ondisk_log_start":"27'1100","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":9043968,"num_objects":138,"num_object_clones":0,"num_object_copies":414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1104,"num_write_kb":8832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.b","version":"27'1008","reported_seq":"1023","reported_epoch":"27","state":"active+clean","last_fresh":"2020-08-07T15:06:46.603720+0000","last_change":"2020-08-07T15:06:35.170504+0000","last_active":"2020-08-07T15:06:46.603720+0000","last_peered":"2020-08-07T15:06:46.603720+0000","last_clean":"2020-08-07T15:06:46.603720+0000","last_became_active":"2020-08-07T15:06:35.169532+0000","last_became_peered":"2020-08-07T15:06:35.169532+0000","last_unstale":"2020-08-07T15:06:46.603720+0000","last_undegraded":"2020-08-07T15:06:46.603720+0000","last_fullsized":"2020-08-07T15:06:46.603720+0000","mapping_epoch":20,"log_start":"27'1000","ondisk_log_start":"27'1000","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8257536,"num_objects":126,"num_object_clones":0,"num_object_copies":378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1008,"num_write_kb":8064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"27'1240","reported_seq":"1254","reported_epoch":"27","state":"active+clean","last_fresh":"2020-08-07T15:06:47.697877+0000","last_change":"2020-08-07T15:06:35.170105+0000","last_active":"2020-08-07T15:06:47.697877+0000","last_peered":"2020-08-07T15:06:47.697877+0000","last_clean":"2020-08-07T15:06:47.697877+0000","last_became_active":"2020-08-07T15:06:35.169494+0000","last_became_peered":"2020-08-07T15:06:35.169494+0000","last_unstale":"2020-08-07T15:06:47.697877+0000","last_undegraded":"2020-08-07T15:06:47.697877+0000","last_fullsized":"2020-08-07T15:06:47.697877+0000","mapping_epoch":20,"log_start":"27'1234","ondisk_log_start":"27'1234","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":10158080,"num_objects":155,"num_object_clones":0,"num_object_copies":465,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1240,"num_write_kb":9920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"27'856","reported_seq":"871","reported_epoch":"27","state":"active+clean","last_fresh":"2020-08-07T15:06:46.685511+0000","last_change":"2020-08-07T15:06:35.163109+0000","last_active":"2020-08-07T15:06:46.685511+0000","last_peered":"2020-08-07T15:06:46.685511+0000","last_clean":"2020-08-07T15:06:46.685511+0000","last_became_active":"2020-08-07T15:06:35.162721+0000","last_became_peered":"2020-08-07T15:06:35.162721+0000","last_unstale":"2020-08-07T15:06:46.685511+0000","last_undegraded":"2020-08-07T15:06:46.685511+0000","last_fullsized":"2020-08-07T15:06:46.685511+0000","mapping_epoch":20,"log_start":"27'848","ondisk_log_start":"27'848","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7012352,"num_objects":107,"num_object_clones":0,"num_object_copies":321,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":856,"num_write_kb":6848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'904","reported_seq":"919","reported_epoch":"27","state":"active+clean","last_fresh":"2020-08-07T15:06:46.592395+0000","last_change":"2020-08-07T15:06:35.173504+0000","last_active":"2020-08-07T15:06:46.592395+0000","last_peered":"2020-08-07T15:06:46.592395+0000","last_clean":"2020-08-07T15:06:46.592395+0000","last_became_active":"2020-08-07T15:06:35.172437+0000","last_became_peered":"2020-08-07T15:06:35.172437+0000","last_unstale":"2020-08-07T15:06:46.592395+0000","last_undegraded":"2020-08-07T15:06:46.592395+0000","last_fullsized":"2020-08-07T15:06:46.592395+0000","mapping_epoch":20,"log_start":"27'896","ondisk_log_start":"27'896","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7405568,"num_objects":113,"num_object_clones":0,"num_object_copies":339,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":904,"num_write_kb":7232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"15","reported_epoch":"27","state":"active+clean","last_fresh":"2020-08-07T15:06:46.199001+0000","last_change":"2020-08-07T15:06:46.199001+0000","last_active":"2020-08-07T15:06:46.199001+0000","last_peered":"2020-08-07T15:06:46.199001+0000","last_clean":"2020-08-07T15:06:46.199001+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:06:46.199001+0000","last_undegraded":"2020-08-07T15:06:46.199001+0000","last_fullsized":"2020-08-07T15:06:46.199001+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.5","version":"27'891","reported_seq":"906","reported_epoch":"27","state":"active+clean","last_fresh":"2020-08-07T15:06:46.693146+0000","last_change":"2020-08-07T15:06:35.169307+0000","last_active":"2020-08-07T15:06:46.693146+0000","last_peered":"2020-08-07T15:06:46.693146+0000","last_clean":"2020-08-07T15:06:46.693146+0000","last_became_active":"2020-08-07T15:06:35.168887+0000","last_became_peered":"2020-08-07T15:06:35.168887+0000","last_unstale":"2020-08-07T15:06:46.693146+0000","last_undegraded":"2020-08-07T15:06:46.693146+0000","last_fullsized":"2020-08-07T15:06:46.693146+0000","mapping_epoch":20,"log_start":"27'882","ondisk_log_start":"27'882","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7299072,"num_objects":112,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":891,"num_write_kb":7128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"18","reported_epoch":"26","state":"active+clean","last_fresh":"2020-08-07T15:06:45.189337+0000","last_change":"2020-08-07T15:06:23.074962+0000","last_active":"2020-08-07T15:06:45.189337+0000","last_peered":"2020-08-07T15:06:45.189337+0000","last_clean":"2020-08-07T15:06:45.189337+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:06:45.189337+0000","last_undegraded":"2020-08-07T15:06:45.189337+0000","last_fullsized":"2020-08-07T15:06:45.189337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"27'928","reported_seq":"943","reported_epoch":"27","state":"active+clean","last_fresh":"2020-08-07T15:06:46.653397+0000","last_change":"2020-08-07T15:06:35.164593+0000","last_active":"2020-08-07T15:06:46.653397+0000","last_peered":"2020-08-07T15:06:46.653397+0000","last_clean":"2020-08-07T15:06:46.653397+0000","last_became_active":"2020-08-07T15:06:35.164108+0000","last_became_peered":"2020-08-07T15:06:35.164108+0000","last_unstale":"2020-08-07T15:06:46.653397+0000","last_undegraded":"2020-08-07T15:06:46.653397+0000","last_fullsized":"2020-08-07T15:06:46.653397+0000","mapping_epoch":20,"log_start":"27'920","ondisk_log_start":"27'920","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":928,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"18","reported_epoch":"26","state":"active+clean","last_fresh":"2020-08-07T15:06:45.206282+0000","last_change":"2020-08-07T15:06:23.223924+0000","last_active":"2020-08-07T15:06:45.206282+0000","last_peered":"2020-08-07T15:06:45.206282+0000","last_clean":"2020-08-07T15:06:45.206282+0000","last_became_active":"2020-08-07T15:06:23.223489+0000","last_became_peered":"2020-08-07T15:06:23.223489+0000","last_unstale":"2020-08-07T15:06:45.206282+0000","last_undegraded":"2020-08-07T15:06:45.206282+0000","last_fullsized":"2020-08-07T15:06:45.206282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"27'1096","reported_seq":"1108","reported_epoch":"27","state":"active+clean","last_fresh":"2020-08-07T15:06:46.704712+0000","last_change":"2020-08-07T15:06:35.170553+0000","last_active":"2020-08-07T15:06:46.704712+0000","last_peered":"2020-08-07T15:06:46.704712+0000","last_clean":"2020-08-07T15:06:46.704712+0000","last_became_active":"2020-08-07T15:06:35.170037+0000","last_became_peered":"2020-08-07T15:06:35.170037+0000","last_unstale":"2020-08-07T15:06:46.704712+0000","last_undegraded":"2020-08-07T15:06:46.704712+0000","last_fullsized":"2020-08-07T15:06:46.704712+0000","mapping_epoch":20,"log_start":"27'1089","ondisk_log_start":"27'1089","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8978432,"num_objects":137,"num_object_clones":0,"num_object_copies":411,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1096,"num_write_kb":8768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"18","reported_epoch":"26","state":"active+clean","last_fresh":"2020-08-07T15:06:45.189308+0000","last_change":"2020-08-07T15:06:23.075958+0000","last_active":"2020-08-07T15:06:45.189308+0000","last_peered":"2020-08-07T15:06:45.189308+0000","last_clean":"2020-08-07T15:06:45.189308+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:06:45.189308+0000","last_undegraded":"2020-08-07T15:06:45.189308+0000","last_fullsized":"2020-08-07T15:06:45.189308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"25","reported_epoch":"26","state":"active+clean","last_fresh":"2020-08-07T15:06:45.196634+0000","last_change":"2020-08-07T15:06:19.198215+0000","last_active":"2020-08-07T15:06:45.196634+0000","last_peered":"2020-08-07T15:06:45.196634+0000","last_clean":"2020-08-07T15:06:45.196634+0000","last_became_active":"2020-08-07T15:06:19.197819+0000","last_became_peered":"2020-08-07T15:06:19.197819+0000","last_unstale":"2020-08-07T15:06:45.196634+0000","last_undegraded":"2020-08-07T15:06:45.196634+0000","last_fullsized":"2020-08-07T15:06:45.196634+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:06:10.878833+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"27'792","reported_seq":"807","reported_epoch":"27","state":"active+clean","last_fresh":"2020-08-07T15:06:46.645128+0000","last_change":"2020-08-07T15:06:35.169888+0000","last_active":"2020-08-07T15:06:46.645128+0000","last_peered":"2020-08-07T15:06:46.645128+0000","last_clean":"2020-08-07T15:06:46.645128+0000","last_became_active":"2020-08-07T15:06:35.169385+0000","last_became_peered":"2020-08-07T15:06:35.169385+0000","last_unstale":"2020-08-07T15:06:46.645128+0000","last_undegraded":"2020-08-07T15:06:46.645128+0000","last_fullsized":"2020-08-07T15:06:46.645128+0000","mapping_epoch":20,"log_start":"27'787","ondisk_log_start":"27'787","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6488064,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":6336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"18","reported_epoch":"26","state":"active+clean","last_fresh":"2020-08-07T15:06:45.196654+0000","last_change":"2020-08-07T15:06:23.224115+0000","last_active":"2020-08-07T15:06:45.196654+0000","last_peered":"2020-08-07T15:06:45.196654+0000","last_clean":"2020-08-07T15:06:45.196654+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:06:45.196654+0000","last_undegraded":"2020-08-07T15:06:45.196654+0000","last_fullsized":"2020-08-07T15:06:45.196654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"27'968","reported_seq":"984","reported_epoch":"27","state":"active+clean","last_fresh":"2020-08-07T15:06:46.666176+0000","last_change":"2020-08-07T15:06:46.214618+0000","last_active":"2020-08-07T15:06:46.666176+0000","last_peered":"2020-08-07T15:06:46.666176+0000","last_clean":"2020-08-07T15:06:46.666176+0000","last_became_active":"2020-08-07T15:06:46.202192+0000","last_became_peered":"2020-08-07T15:06:46.202192+0000","last_unstale":"2020-08-07T15:06:46.666176+0000","last_undegraded":"2020-08-07T15:06:46.666176+0000","last_fullsized":"2020-08-07T15:06:46.666176+0000","mapping_epoch":26,"log_start":"27'964","ondisk_log_start":"27'964","created":20,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7929856,"num_objects":121,"num_object_clones":0,"num_object_copies":363,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":968,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3],"acting":[4,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"18","reported_epoch":"26","state":"active+clean","last_fresh":"2020-08-07T15:06:45.196747+0000","last_change":"2020-08-07T15:06:23.225886+0000","last_active":"2020-08-07T15:06:45.196747+0000","last_peered":"2020-08-07T15:06:45.196747+0000","last_clean":"2020-08-07T15:06:45.196747+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:06:45.196747+0000","last_undegraded":"2020-08-07T15:06:45.196747+0000","last_fullsized":"2020-08-07T15:06:45.196747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"27'912","reported_seq":"927","reported_epoch":"27","state":"active+clean","last_fresh":"2020-08-07T15:06:46.609002+0000","last_change":"2020-08-07T15:06:35.169626+0000","last_active":"2020-08-07T15:06:46.609002+0000","last_peered":"2020-08-07T15:06:46.609002+0000","last_clean":"2020-08-07T15:06:46.609002+0000","last_became_active":"2020-08-07T15:06:35.169127+0000","last_became_peered":"2020-08-07T15:06:35.169127+0000","last_unstale":"2020-08-07T15:06:46.609002+0000","last_undegraded":"2020-08-07T15:06:46.609002+0000","last_fullsized":"2020-08-07T15:06:46.609002+0000","mapping_epoch":20,"log_start":"27'904","ondisk_log_start":"27'904","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7471104,"num_objects":114,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":912,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"27'912","reported_seq":"926","reported_epoch":"27","state":"active+clean","last_fresh":"2020-08-07T15:06:46.640323+0000","last_change":"2020-08-07T15:06:35.172780+0000","last_active":"2020-08-07T15:06:46.640323+0000","last_peered":"2020-08-07T15:06:46.640323+0000","last_clean":"2020-08-07T15:06:46.640323+0000","last_became_active":"2020-08-07T15:06:35.172360+0000","last_became_peered":"2020-08-07T15:06:35.172360+0000","last_unstale":"2020-08-07T15:06:46.640323+0000","last_undegraded":"2020-08-07T15:06:46.640323+0000","last_fullsized":"2020-08-07T15:06:46.640323+0000","mapping_epoch":20,"log_start":"27'905","ondisk_log_start":"27'905","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7471104,"num_objects":114,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":912,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"18","reported_epoch":"26","state":"active+clean","last_fresh":"2020-08-07T15:06:45.189631+0000","last_change":"2020-08-07T15:06:23.074076+0000","last_active":"2020-08-07T15:06:45.189631+0000","last_peered":"2020-08-07T15:06:45.189631+0000","last_clean":"2020-08-07T15:06:45.189631+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:06:45.189631+0000","last_undegraded":"2020-08-07T15:06:45.189631+0000","last_fullsized":"2020-08-07T15:06:45.189631+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"18","reported_epoch":"26","state":"active+clean","last_fresh":"2020-08-07T15:06:45.199766+0000","last_change":"2020-08-07T15:06:23.073702+0000","last_active":"2020-08-07T15:06:45.199766+0000","last_peered":"2020-08-07T15:06:45.199766+0000","last_clean":"2020-08-07T15:06:45.199766+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:06:45.199766+0000","last_undegraded":"2020-08-07T15:06:45.199766+0000","last_fullsized":"2020-08-07T15:06:45.199766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"27'1024","reported_seq":"1033","reported_epoch":"27","state":"active+clean","last_fresh":"2020-08-07T15:06:46.658509+0000","last_change":"2020-08-07T15:06:46.199635+0000","last_active":"2020-08-07T15:06:46.658509+0000","last_peered":"2020-08-07T15:06:46.658509+0000","last_clean":"2020-08-07T15:06:46.658509+0000","last_became_active":"2020-08-07T15:06:46.199201+0000","last_became_peered":"2020-08-07T15:06:46.199201+0000","last_unstale":"2020-08-07T15:06:46.658509+0000","last_undegraded":"2020-08-07T15:06:46.658509+0000","last_fullsized":"2020-08-07T15:06:46.658509+0000","mapping_epoch":26,"log_start":"27'1015","ondisk_log_start":"27'1015","created":20,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8388608,"num_objects":128,"num_object_clones":0,"num_object_copies":384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1024,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":129064960,"num_objects":1970,"num_object_clones":0,"num_object_copies":5910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15755,"num_write_kb":126040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":389251072,"data_stored":389251072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1101056,"kb_used_data":52472,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93270784,"statfs":{"total":96636764160,"available":95509282816,"internally_reserved":1073741824,"allocated":53731328,"data_stored":53567729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7149999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9790000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.649}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8050000000000002}]}]},{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1094328,"kb_used_data":45744,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93277512,"statfs":{"total":96636764160,"available":95516172288,"internally_reserved":1073741824,"allocated":46841856,"data_stored":46678257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9950000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2629999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8240000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.052}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0939999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1096636,"kb_used_data":48052,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93275204,"statfs":{"total":96636764160,"available":95513808896,"internally_reserved":1073741824,"allocated":49205248,"data_stored":49044917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1093756,"kb_used_data":45172,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93278084,"statfs":{"total":96636764160,"available":95516758016,"internally_reserved":1073741824,"allocated":46256128,"data_stored":46096047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1078464,"kb_used_data":29880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93293376,"statfs":{"total":96636764160,"available":95532417024,"internally_reserved":1073741824,"allocated":30597120,"data_stored":30440307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.143}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1100748,"kb_used_data":52164,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93271092,"statfs":{"total":96636764160,"available":95509598208,"internally_reserved":1073741824,"allocated":53415936,"data_stored":53255605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.403}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.423}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.454}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1111392,"kb_used_data":62808,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93260448,"statfs":{"total":96636764160,"available":95498698752,"internally_reserved":1073741824,"allocated":64315392,"data_stored":64151793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1085592,"kb_used_data":37008,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93286248,"statfs":{"total":96636764160,"available":95525117952,"internally_reserved":1073741824,"allocated":37896192,"data_stored":37732593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,8],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.522}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2869999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.556}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":277.995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9019999999999999}]}]}]}} 2020-08-07T15:06:52.334 INFO:tasks.thrashosds.thrasher:Setting 3.7 to [5, 3, 2, 4, 0, 7] 2020-08-07T15:06:52.334 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.7', '5', '3', '2', '4', '0', '7'] 2020-08-07T15:06:52.334 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pg-upmap-items 3.7 5 3 2 4 0 7 2020-08-07T15:06:52.360 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:06:52.398 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:06:52.398 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:52.396+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:52.399 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:06:52.399 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:06:52.399 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:06:52.400 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:06:52.400 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:06:52.400 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:06:52.400 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:06:52.400 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:06:52.437 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:06:52.437 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:06:52.437 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:06:52.438 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:06:52.438 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:06:52.449 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:06:52.547 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:06:52.531+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:06:52.614 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:06:52.604+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:06:52.672 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:06:52.672 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:06:52.673 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:06:52.673 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:06:52.673 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.673 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:7615 3.6 3:68351b60:::benchmark_data_smithi074_35447_object951:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:52.673 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.755785+0000", 2020-08-07T15:06:52.684 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.912731334, 2020-08-07T15:06:52.684 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11113547, 2020-08-07T15:06:52.684 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:52.684 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:52.684 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:52.685 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:52.685 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:52.685 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7615 2020-08-07T15:06:52.685 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.685 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:52.686 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.686 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:52.686 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.755785+0000", 2020-08-07T15:06:52.686 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:52.686 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.687 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.687 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:52.687 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.755785+0000", 2020-08-07T15:06:52.687 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1455000000000001e-05 2020-08-07T15:06:52.687 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.688 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.688 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:52.688 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.755837+0000", 2020-08-07T15:06:52.688 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0115999999999999e-05 2020-08-07T15:06:52.688 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.688 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.689 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:52.689 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.755847+0000", 2020-08-07T15:06:52.689 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1518000000000001e-05 2020-08-07T15:06:52.689 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.690 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.700 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:52.700 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.755858+0000", 2020-08-07T15:06:52.700 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7980000000000002e-06 2020-08-07T15:06:52.700 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.701 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:52.701 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.755864+0000", 2020-08-07T15:06:52.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001938709 2020-08-07T15:06:52.701 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.702 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.702 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:52.702 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.757803+0000", 2020-08-07T15:06:52.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2016999999999997e-05 2020-08-07T15:06:52.706 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.706 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.706 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:52.718 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.757895+0000", 2020-08-07T15:06:52.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015044 2020-08-07T15:06:52.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:52.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.758045+0000", 2020-08-07T15:06:52.726 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001953177 2020-08-07T15:06:52.726 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.726 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.726 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:52.727 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.759999+0000", 2020-08-07T15:06:52.727 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8100000000000003e-07 2020-08-07T15:06:52.730 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.730 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.731 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:52.731 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.759999+0000", 2020-08-07T15:06:52.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2781000000000002e-05 2020-08-07T15:06:52.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:52.732 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.760052+0000", 2020-08-07T15:06:52.732 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011239300000000001 2020-08-07T15:06:52.732 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.732 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.733 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:52.733 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.760164+0000", 2020-08-07T15:06:52.733 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.106679305 2020-08-07T15:06:52.733 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.734 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:52.734 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.866844+0000", 2020-08-07T15:06:52.734 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.071e-06 2020-08-07T15:06:52.734 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.735 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.735 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:52.735 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.866845+0000", 2020-08-07T15:06:52.735 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6209000000000002e-05 2020-08-07T15:06:52.746 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.747 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.747 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:52.747 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.866921+0000", 2020-08-07T15:06:52.747 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11113547 2020-08-07T15:06:52.748 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:52.748 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:52.748 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:52.748 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.749 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.749 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:7616 3.6 3:68351b60:::benchmark_data_smithi074_35447_object951:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:52.749 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.756114+0000", 2020-08-07T15:06:52.749 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.912402984, 2020-08-07T15:06:52.749 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11758613499999999, 2020-08-07T15:06:52.750 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:52.750 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:52.751 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:52.751 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:52.751 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:52.751 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7616 2020-08-07T15:06:52.751 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.752 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:52.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.752 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:52.752 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.756114+0000", 2020-08-07T15:06:52.752 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:52.753 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.753 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.753 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:52.753 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.756114+0000", 2020-08-07T15:06:52.753 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8724999999999999e-05 2020-08-07T15:06:52.754 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.754 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.754 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:52.754 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.756162+0000", 2020-08-07T15:06:52.755 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0189999999999997e-06 2020-08-07T15:06:52.755 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.755 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:52.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.756167+0000", 2020-08-07T15:06:52.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1259e-05 2020-08-07T15:06:52.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:52.756 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.756179+0000", 2020-08-07T15:06:52.757 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9170000000000003e-06 2020-08-07T15:06:52.757 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.757 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:52.757 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.756185+0000", 2020-08-07T15:06:52.758 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023972080000000001 2020-08-07T15:06:52.758 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.758 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.759 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:52.759 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.758582+0000", 2020-08-07T15:06:52.759 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9760999999999999e-05 2020-08-07T15:06:52.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.760 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:52.760 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.758642+0000", 2020-08-07T15:06:52.760 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016533800000000001 2020-08-07T15:06:52.760 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.760 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.761 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:52.761 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.758807+0000", 2020-08-07T15:06:52.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015475 2020-08-07T15:06:52.761 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.762 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:52.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.760354+0000", 2020-08-07T15:06:52.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5899999999999996e-07 2020-08-07T15:06:52.763 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.763 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:52.763 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.760355+0000", 2020-08-07T15:06:52.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4606e-05 2020-08-07T15:06:52.764 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.764 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:52.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.760390+0000", 2020-08-07T15:06:52.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001638406 2020-08-07T15:06:52.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.765 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:52.765 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.762028+0000", 2020-08-07T15:06:52.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.111598564 2020-08-07T15:06:52.766 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.766 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:52.766 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.873627+0000", 2020-08-07T15:06:52.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0399999999999996e-07 2020-08-07T15:06:52.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:52.767 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.873627+0000", 2020-08-07T15:06:52.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2569000000000003e-05 2020-08-07T15:06:52.768 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.768 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.768 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:52.768 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.873700+0000", 2020-08-07T15:06:52.768 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11758613499999999 2020-08-07T15:06:52.768 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:52.769 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:52.769 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:52.769 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.769 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.770 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:7615 3.6 e24/20 ondisk, result = 0)", 2020-08-07T15:06:52.770 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.762099+0000", 2020-08-07T15:06:52.770 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.9064178, 2020-08-07T15:06:52.770 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.104828924, 2020-08-07T15:06:52.771 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:52.771 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:06:52.771 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:52.771 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.772 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:52.772 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.762099+0000", 2020-08-07T15:06:52.772 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:52.772 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.772 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.773 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:52.773 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.762099+0000", 2020-08-07T15:06:52.773 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7991999999999999e-05 2020-08-07T15:06:52.773 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.773 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.774 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:52.774 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.762127+0000", 2020-08-07T15:06:52.774 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3179999999999997e-06 2020-08-07T15:06:52.774 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.774 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.774 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:52.775 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.762132+0000", 2020-08-07T15:06:52.775 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.10382048200000001 2020-08-07T15:06:52.775 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:52.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.865953+0000", 2020-08-07T15:06:52.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1599999999999997e-06 2020-08-07T15:06:52.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:52.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.865958+0000", 2020-08-07T15:06:52.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00086268499999999997 2020-08-07T15:06:52.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:52.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.866820+0000", 2020-08-07T15:06:52.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3933e-05 2020-08-07T15:06:52.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:52.779 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.866834+0000", 2020-08-07T15:06:52.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3354000000000001e-05 2020-08-07T15:06:52.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.779 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:52.780 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.866928+0000", 2020-08-07T15:06:52.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.104828924 2020-08-07T15:06:52.780 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:52.780 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:52.780 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:52.780 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.781 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:7696 3.7 e24/20 3:ee5cfea7:::benchmark_data_smithi074_35447_object961:head v 24'504, mlcod=24'496)", 2020-08-07T15:06:52.781 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.807555+0000", 2020-08-07T15:06:52.781 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.860962043000001, 2020-08-07T15:06:52.781 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.124762208, 2020-08-07T15:06:52.782 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:52.782 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:52.782 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:52.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:52.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.807555+0000", 2020-08-07T15:06:52.783 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:52.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:52.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.807555+0000", 2020-08-07T15:06:52.785 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7098000000000003e-05 2020-08-07T15:06:52.785 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:52.786 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.807592+0000", 2020-08-07T15:06:52.786 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5120000000000002e-06 2020-08-07T15:06:52.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.786 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:52.786 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.807596+0000", 2020-08-07T15:06:52.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.12021193600000001 2020-08-07T15:06:52.787 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.787 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:52.787 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.927808+0000", 2020-08-07T15:06:52.788 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3749999999999999e-06 2020-08-07T15:06:52.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.788 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:52.788 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.927815+0000", 2020-08-07T15:06:52.789 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0791000000000002e-05 2020-08-07T15:06:52.789 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.789 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.789 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:52.789 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.927865+0000", 2020-08-07T15:06:52.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7851000000000002e-05 2020-08-07T15:06:52.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.790 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:52.790 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.927893+0000", 2020-08-07T15:06:52.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043673590000000003 2020-08-07T15:06:52.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:52.791 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.932261+0000", 2020-08-07T15:06:52.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6286000000000003e-05 2020-08-07T15:06:52.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.792 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.792 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:52.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.932317+0000", 2020-08-07T15:06:52.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.124762208 2020-08-07T15:06:52.793 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:52.793 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:52.793 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:52.793 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.793 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.794 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:7800 3.6 3:6a121f3d:::benchmark_data_smithi074_35447_object974:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:52.794 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.866632+0000", 2020-08-07T15:06:52.794 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.801885018, 2020-08-07T15:06:52.794 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.099086428000000004, 2020-08-07T15:06:52.795 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:52.795 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:52.795 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:52.795 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:52.795 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:52.796 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7800 2020-08-07T15:06:52.796 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.796 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:52.797 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.797 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:52.797 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.866632+0000", 2020-08-07T15:06:52.798 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:52.798 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.798 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:52.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.866632+0000", 2020-08-07T15:06:52.799 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3369999999999999e-05 2020-08-07T15:06:52.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:52.800 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.866685+0000", 2020-08-07T15:06:52.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3399999999999997e-06 2020-08-07T15:06:52.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.801 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.801 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:52.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.866690+0000", 2020-08-07T15:06:52.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1031e-05 2020-08-07T15:06:52.801 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.802 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.802 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:52.802 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.866701+0000", 2020-08-07T15:06:52.802 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0270000000000003e-06 2020-08-07T15:06:52.802 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:52.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.866706+0000", 2020-08-07T15:06:52.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039773150000000004 2020-08-07T15:06:52.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:52.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.870684+0000", 2020-08-07T15:06:52.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5081000000000007e-05 2020-08-07T15:06:52.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:52.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.870779+0000", 2020-08-07T15:06:52.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017596800000000001 2020-08-07T15:06:52.806 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:52.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.870955+0000", 2020-08-07T15:06:52.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0030135420000000001 2020-08-07T15:06:52.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:52.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.873968+0000", 2020-08-07T15:06:52.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2900000000000004e-07 2020-08-07T15:06:52.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.809 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:52.809 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.873969+0000", 2020-08-07T15:06:52.809 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2441000000000002e-05 2020-08-07T15:06:52.810 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.810 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.810 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:52.810 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.874021+0000", 2020-08-07T15:06:52.810 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.029723065999999999 2020-08-07T15:06:52.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.811 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:52.811 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.903744+0000", 2020-08-07T15:06:52.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.061911028999999999 2020-08-07T15:06:52.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:52.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.965655+0000", 2020-08-07T15:06:52.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8199999999999998e-07 2020-08-07T15:06:52.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.813 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:52.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.965656+0000", 2020-08-07T15:06:52.813 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1807e-05 2020-08-07T15:06:52.814 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.814 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.814 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:52.814 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.965718+0000", 2020-08-07T15:06:52.814 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.099086428000000004 2020-08-07T15:06:52.814 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:52.815 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:52.815 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:52.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.815 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:9240 3.6 3:61cd9749:::benchmark_data_smithi074_35447_object1154:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:52.816 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.815172+0000", 2020-08-07T15:06:52.816 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 10.853345134, 2020-08-07T15:06:52.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.10781420899999999, 2020-08-07T15:06:52.816 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:52.816 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:52.817 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:52.817 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:52.817 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:52.817 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 9240 2020-08-07T15:06:52.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.818 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:52.818 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:52.819 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.815172+0000", 2020-08-07T15:06:52.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:52.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:52.821 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.815172+0000", 2020-08-07T15:06:52.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0300000000000003e-05 2020-08-07T15:06:52.821 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.822 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.822 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:52.822 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.815222+0000", 2020-08-07T15:06:52.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.437e-05 2020-08-07T15:06:52.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.822 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:52.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.815236+0000", 2020-08-07T15:06:52.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1197e-05 2020-08-07T15:06:52.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:52.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.815247+0000", 2020-08-07T15:06:52.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5329999999999996e-06 2020-08-07T15:06:52.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:52.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.815252+0000", 2020-08-07T15:06:52.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0055860279999999998 2020-08-07T15:06:52.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:52.826 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.820838+0000", 2020-08-07T15:06:52.826 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0519999999999995e-05 2020-08-07T15:06:52.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:52.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.820918+0000", 2020-08-07T15:06:52.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00025618800000000002 2020-08-07T15:06:52.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.827 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:52.828 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.821175+0000", 2020-08-07T15:06:52.828 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001010934 2020-08-07T15:06:52.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:52.829 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.822186+0000", 2020-08-07T15:06:52.829 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1500000000000004e-07 2020-08-07T15:06:52.829 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.829 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.829 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:52.830 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.822186+0000", 2020-08-07T15:06:52.830 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8641999999999998e-05 2020-08-07T15:06:52.830 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.830 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.830 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:52.830 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.822255+0000", 2020-08-07T15:06:52.831 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001099884 2020-08-07T15:06:52.831 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.831 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.831 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:52.831 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.823355+0000", 2020-08-07T15:06:52.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.099570468999999995 2020-08-07T15:06:52.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:52.833 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922925+0000", 2020-08-07T15:06:52.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0610000000000001e-06 2020-08-07T15:06:52.833 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:52.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922926+0000", 2020-08-07T15:06:52.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9468000000000001e-05 2020-08-07T15:06:52.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:52.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922986+0000", 2020-08-07T15:06:52.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.10781420899999999 2020-08-07T15:06:52.835 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:52.835 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:52.835 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:52.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.836 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:9416 3.6 3:6d5d7913:::benchmark_data_smithi074_35447_object1176:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:52.836 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.922485+0000", 2020-08-07T15:06:52.836 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 10.746031519000001, 2020-08-07T15:06:52.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16975183499999999, 2020-08-07T15:06:52.837 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:52.837 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:52.837 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:52.837 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:52.838 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:52.838 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 9416 2020-08-07T15:06:52.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.838 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:52.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.839 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:52.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922485+0000", 2020-08-07T15:06:52.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:52.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.839 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:52.840 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922485+0000", 2020-08-07T15:06:52.840 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6072999999999998e-05 2020-08-07T15:06:52.840 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.840 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.840 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:52.841 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922521+0000", 2020-08-07T15:06:52.841 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.241e-06 2020-08-07T15:06:52.841 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:52.842 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922527+0000", 2020-08-07T15:06:52.842 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2075e-05 2020-08-07T15:06:52.842 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.842 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.842 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:52.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922540+0000", 2020-08-07T15:06:52.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0589999999999999e-06 2020-08-07T15:06:52.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.843 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:52.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922546+0000", 2020-08-07T15:06:52.844 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00063213199999999998 2020-08-07T15:06:52.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.844 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:52.844 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.923178+0000", 2020-08-07T15:06:52.845 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9913e-05 2020-08-07T15:06:52.845 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:52.845 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.923238+0000", 2020-08-07T15:06:52.846 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010683099999999999 2020-08-07T15:06:52.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.846 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.846 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:52.846 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.923344+0000", 2020-08-07T15:06:52.846 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004139663 2020-08-07T15:06:52.847 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.847 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.847 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:52.847 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.927484+0000", 2020-08-07T15:06:52.847 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001063149 2020-08-07T15:06:52.848 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.848 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.848 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:52.848 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.928547+0000", 2020-08-07T15:06:52.848 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16358086099999999 2020-08-07T15:06:52.849 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.849 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.849 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:52.850 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092128+0000", 2020-08-07T15:06:52.850 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3699999999999999e-07 2020-08-07T15:06:52.850 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.858 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:52.858 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092129+0000", 2020-08-07T15:06:52.859 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3859e-05 2020-08-07T15:06:52.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.859 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.859 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:52.859 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092193+0000", 2020-08-07T15:06:52.860 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4141999999999999e-05 2020-08-07T15:06:52.860 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.860 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:52.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092237+0000", 2020-08-07T15:06:52.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16975183499999999 2020-08-07T15:06:52.861 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:52.861 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:52.866 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:52.866 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.866 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.867 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:9416 3.6 e24/20 ondisk, result = 0)", 2020-08-07T15:06:52.867 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.926716+0000", 2020-08-07T15:06:52.867 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 10.741801126, 2020-08-07T15:06:52.867 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16552591899999999, 2020-08-07T15:06:52.867 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:52.868 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:06:52.868 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:52.868 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.868 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:52.868 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.926716+0000", 2020-08-07T15:06:52.869 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:52.869 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.869 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.869 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:52.869 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.926716+0000", 2020-08-07T15:06:52.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9971000000000002e-05 2020-08-07T15:06:52.879 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.879 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:52.880 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.926745+0000", 2020-08-07T15:06:52.880 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.51e-06 2020-08-07T15:06:52.880 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.880 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:52.881 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.926750+0000", 2020-08-07T15:06:52.881 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.165288254 2020-08-07T15:06:52.881 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.881 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.881 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:52.882 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092038+0000", 2020-08-07T15:06:52.882 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1109999999999999e-06 2020-08-07T15:06:52.882 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.882 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.882 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:52.883 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092043+0000", 2020-08-07T15:06:52.893 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7359000000000003e-05 2020-08-07T15:06:52.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.894 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:52.894 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092111+0000", 2020-08-07T15:06:52.894 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6889999999999993e-06 2020-08-07T15:06:52.894 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.894 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.895 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:52.895 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092119+0000", 2020-08-07T15:06:52.898 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000122025 2020-08-07T15:06:52.898 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.898 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.898 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:52.899 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092241+0000", 2020-08-07T15:06:52.899 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16552591899999999 2020-08-07T15:06:52.899 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:52.899 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:52.899 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:52.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.905 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:9687 3.6 3:667d8bb7:::benchmark_data_smithi074_35447_object1210:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:52.905 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.094514+0000", 2020-08-07T15:06:52.905 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 10.574002509, 2020-08-07T15:06:52.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.141938179, 2020-08-07T15:06:52.905 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:52.906 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:52.906 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:52.906 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:52.906 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:52.908 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 9687 2020-08-07T15:06:52.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.908 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:52.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:52.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094514+0000", 2020-08-07T15:06:52.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:52.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.910 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:52.910 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094514+0000", 2020-08-07T15:06:52.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2519000000000003e-05 2020-08-07T15:06:52.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:52.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094567+0000", 2020-08-07T15:06:52.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2570000000000004e-06 2020-08-07T15:06:52.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:52.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094573+0000", 2020-08-07T15:06:52.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1446e-05 2020-08-07T15:06:52.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:52.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094584+0000", 2020-08-07T15:06:52.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5439999999999998e-06 2020-08-07T15:06:52.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:52.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094590+0000", 2020-08-07T15:06:52.914 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00081871299999999999 2020-08-07T15:06:52.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:52.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.095409+0000", 2020-08-07T15:06:52.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2918999999999996e-05 2020-08-07T15:06:52.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:52.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.095502+0000", 2020-08-07T15:06:52.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017535200000000001 2020-08-07T15:06:52.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:52.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.095677+0000", 2020-08-07T15:06:52.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026867729999999999 2020-08-07T15:06:52.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.918 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:52.918 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.098364+0000", 2020-08-07T15:06:52.918 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8400000000000002e-07 2020-08-07T15:06:52.918 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.918 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:52.919 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.098365+0000", 2020-08-07T15:06:52.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4826999999999994e-05 2020-08-07T15:06:52.919 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.919 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:52.920 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.098429+0000", 2020-08-07T15:06:52.920 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001860022 2020-08-07T15:06:52.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:52.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.100289+0000", 2020-08-07T15:06:52.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.136088662 2020-08-07T15:06:52.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:52.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.236378+0000", 2020-08-07T15:06:52.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0559999999999999e-06 2020-08-07T15:06:52.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.922 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:52.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.236379+0000", 2020-08-07T15:06:52.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3104999999999999e-05 2020-08-07T15:06:52.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:52.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.236452+0000", 2020-08-07T15:06:52.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.141938179 2020-08-07T15:06:52.924 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:52.924 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:52.924 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:52.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.925 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:9688 3.6 3:667d8bb7:::benchmark_data_smithi074_35447_object1210:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:52.925 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.094625+0000", 2020-08-07T15:06:52.925 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 10.573891988, 2020-08-07T15:06:52.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14221277900000001, 2020-08-07T15:06:52.925 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:52.926 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:52.926 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:52.926 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:52.926 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:52.926 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 9688 2020-08-07T15:06:52.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.927 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:52.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:52.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094625+0000", 2020-08-07T15:06:52.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:52.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:52.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094625+0000", 2020-08-07T15:06:52.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4469000000000003e-05 2020-08-07T15:06:52.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:52.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094679+0000", 2020-08-07T15:06:52.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.201e-05 2020-08-07T15:06:52.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:52.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094691+0000", 2020-08-07T15:06:52.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0873000000000001e-05 2020-08-07T15:06:52.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:52.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094702+0000", 2020-08-07T15:06:52.932 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9599999999999999e-06 2020-08-07T15:06:52.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.932 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.932 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:52.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094707+0000", 2020-08-07T15:06:52.932 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017070119999999999 2020-08-07T15:06:52.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.942 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:52.942 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.096414+0000", 2020-08-07T15:06:52.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3101000000000002e-05 2020-08-07T15:06:52.942 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:52.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.096507+0000", 2020-08-07T15:06:52.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017744399999999999 2020-08-07T15:06:52.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.944 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:52.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.096685+0000", 2020-08-07T15:06:52.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017538250000000001 2020-08-07T15:06:52.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:52.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.098438+0000", 2020-08-07T15:06:52.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.82e-07 2020-08-07T15:06:52.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.946 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.946 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:52.946 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.098439+0000", 2020-08-07T15:06:52.946 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3258999999999999e-05 2020-08-07T15:06:52.946 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.947 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.947 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:52.947 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.098492+0000", 2020-08-07T15:06:52.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001822908 2020-08-07T15:06:52.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.948 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:52.948 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.100315+0000", 2020-08-07T15:06:52.948 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13645548299999999 2020-08-07T15:06:52.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.949 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:52.949 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.236771+0000", 2020-08-07T15:06:52.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7999999999999996e-07 2020-08-07T15:06:52.949 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.950 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:52.950 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.236771+0000", 2020-08-07T15:06:52.950 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6372999999999996e-05 2020-08-07T15:06:52.950 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.950 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:52.951 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.236837+0000", 2020-08-07T15:06:52.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14221277900000001 2020-08-07T15:06:52.951 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:52.951 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:52.952 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:52.952 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.952 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.952 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:9687 3.6 e24/20 ondisk, result = 0)", 2020-08-07T15:06:52.952 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.098678+0000", 2020-08-07T15:06:52.952 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 10.569838367999999, 2020-08-07T15:06:52.953 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13805130800000001, 2020-08-07T15:06:52.953 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:52.953 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:06:52.953 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:52.953 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.954 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:52.954 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.098678+0000", 2020-08-07T15:06:52.954 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:52.954 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.954 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.955 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:52.955 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.098678+0000", 2020-08-07T15:06:52.955 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1142000000000001e-05 2020-08-07T15:06:52.955 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.955 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.956 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:52.956 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.098709+0000", 2020-08-07T15:06:52.956 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1229999999999999e-06 2020-08-07T15:06:52.956 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.956 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.956 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:52.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.098715+0000", 2020-08-07T15:06:52.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13550553000000001 2020-08-07T15:06:52.957 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.957 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:52.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.234220+0000", 2020-08-07T15:06:52.958 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9459999999999997e-06 2020-08-07T15:06:52.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:52.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.234225+0000", 2020-08-07T15:06:52.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002135797 2020-08-07T15:06:52.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:52.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.236361+0000", 2020-08-07T15:06:52.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6309999999999992e-06 2020-08-07T15:06:52.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:52.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.236369+0000", 2020-08-07T15:06:52.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00036013899999999999 2020-08-07T15:06:52.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:52.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.236730+0000", 2020-08-07T15:06:52.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13805130800000001 2020-08-07T15:06:52.962 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:52.962 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:52.962 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:52.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.963 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:10240 3.6 3:665d71ff:::benchmark_data_smithi074_35447_object1279:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:52.963 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:06:52.963 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 10.275297689, 2020-08-07T15:06:52.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25191569699999999, 2020-08-07T15:06:52.964 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:52.964 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:52.964 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:52.964 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:52.965 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:52.965 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 10240 2020-08-07T15:06:52.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.965 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:52.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:52.966 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:06:52.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:52.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:52.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:06:52.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5463000000000002e-05 2020-08-07T15:06:52.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:52.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393264+0000", 2020-08-07T15:06:52.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9219999999999999e-06 2020-08-07T15:06:52.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:52.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393268+0000", 2020-08-07T15:06:52.969 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6149999999999997e-06 2020-08-07T15:06:52.969 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.969 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.969 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:52.970 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393277+0000", 2020-08-07T15:06:52.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2039999999999996e-06 2020-08-07T15:06:52.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.970 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:52.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393282+0000", 2020-08-07T15:06:52.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001225243 2020-08-07T15:06:52.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:52.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394507+0000", 2020-08-07T15:06:52.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7108999999999995e-05 2020-08-07T15:06:52.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:52.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394605+0000", 2020-08-07T15:06:52.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017490800000000001 2020-08-07T15:06:52.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:52.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394779+0000", 2020-08-07T15:06:52.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032608699999999999 2020-08-07T15:06:52.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:52.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.398040+0000", 2020-08-07T15:06:52.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013015259999999999 2020-08-07T15:06:52.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.975 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:52.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.399342+0000", 2020-08-07T15:06:52.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24566893400000001 2020-08-07T15:06:52.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:52.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645011+0000", 2020-08-07T15:06:52.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.217e-06 2020-08-07T15:06:52.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:52.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645012+0000", 2020-08-07T15:06:52.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0803000000000006e-05 2020-08-07T15:06:52.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.978 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:52.978 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645083+0000", 2020-08-07T15:06:52.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1882999999999998e-05 2020-08-07T15:06:52.979 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.979 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.979 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:52.979 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645135+0000", 2020-08-07T15:06:52.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25191569699999999 2020-08-07T15:06:52.980 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:52.980 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:52.980 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:52.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.980 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:15648 3.6 e27/26 3:679b4910:::benchmark_data_smithi074_35447_object1955:head v 27'1024, mlcod=27'1014)", 2020-08-07T15:06:52.981 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:46.705699+0000", 2020-08-07T15:06:52.981 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.9628181419999997, 2020-08-07T15:06:52.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11892449300000001, 2020-08-07T15:06:52.981 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:52.981 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:52.982 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:52.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.982 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:52.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.705699+0000", 2020-08-07T15:06:52.982 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:52.983 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.983 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.983 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:52.983 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.705699+0000", 2020-08-07T15:06:52.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0484e-05 2020-08-07T15:06:52.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.984 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:52.984 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.705749+0000", 2020-08-07T15:06:52.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4710000000000004e-06 2020-08-07T15:06:52.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.985 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:52.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.705755+0000", 2020-08-07T15:06:52.985 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11719257499999999 2020-08-07T15:06:52.985 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.986 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:52.986 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.822948+0000", 2020-08-07T15:06:52.986 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4450000000000001e-06 2020-08-07T15:06:52.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.987 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:52.987 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.822951+0000", 2020-08-07T15:06:52.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8692999999999997e-05 2020-08-07T15:06:52.987 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:52.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.822990+0000", 2020-08-07T15:06:52.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2031999999999998e-05 2020-08-07T15:06:52.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:52.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.823012+0000", 2020-08-07T15:06:52.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015700689999999999 2020-08-07T15:06:52.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.990 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:52.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.824582+0000", 2020-08-07T15:06:52.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0723999999999998e-05 2020-08-07T15:06:52.990 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.990 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:52.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.824623+0000", 2020-08-07T15:06:52.991 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11892449300000001 2020-08-07T15:06:52.991 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:52.991 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:52.991 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:52.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.992 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:15992 3.f e27/20 3:fe86d281:::benchmark_data_smithi074_35447_object1998:head v 27'1168, mlcod=27'1161)", 2020-08-07T15:06:52.992 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:46.932773+0000", 2020-08-07T15:06:52.992 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.7357431749999996, 2020-08-07T15:06:52.993 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11411433, 2020-08-07T15:06:52.993 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:52.993 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:52.993 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:52.993 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.994 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:52.994 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.932773+0000", 2020-08-07T15:06:52.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:52.994 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.994 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.994 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:52.995 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.932773+0000", 2020-08-07T15:06:52.995 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0827e-05 2020-08-07T15:06:52.995 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.995 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.995 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:52.996 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.932824+0000", 2020-08-07T15:06:52.996 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1770000000000003e-06 2020-08-07T15:06:52.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.996 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.996 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:52.997 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.932830+0000", 2020-08-07T15:06:52.997 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.112613254 2020-08-07T15:06:52.997 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.997 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.997 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:52.997 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.045444+0000", 2020-08-07T15:06:52.998 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6519999999999999e-06 2020-08-07T15:06:52.998 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.998 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.998 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:52.998 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.045447+0000", 2020-08-07T15:06:52.999 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3791999999999997e-05 2020-08-07T15:06:52.999 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:52.999 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:52.999 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:52.999 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.045491+0000", 2020-08-07T15:06:53.000 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3762999999999999e-05 2020-08-07T15:06:53.000 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.000 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.000 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:53.000 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.045515+0000", 2020-08-07T15:06:53.000 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001326214 2020-08-07T15:06:53.001 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.001 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.001 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:53.001 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.046841+0000", 2020-08-07T15:06:53.001 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6650999999999998e-05 2020-08-07T15:06:53.002 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.002 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.002 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:53.002 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.046888+0000", 2020-08-07T15:06:53.002 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11411433 2020-08-07T15:06:53.003 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.003 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:53.003 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.003 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:16144 3.f e27/20 3:ff74134a:::benchmark_data_smithi074_35447_object2017:head v 27'1176, mlcod=27'1170)", 2020-08-07T15:06:53.004 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.055958+0000", 2020-08-07T15:06:53.004 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.6125585850000004, 2020-08-07T15:06:53.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13659865900000001, 2020-08-07T15:06:53.004 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:53.004 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:53.005 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:53.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:53.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.055958+0000", 2020-08-07T15:06:53.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:53.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:53.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.055958+0000", 2020-08-07T15:06:53.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1301000000000002e-05 2020-08-07T15:06:53.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:53.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.056009+0000", 2020-08-07T15:06:53.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2360000000000001e-06 2020-08-07T15:06:53.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:53.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.056016+0000", 2020-08-07T15:06:53.008 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13378537900000001 2020-08-07T15:06:53.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.009 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.009 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:53.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.189801+0000", 2020-08-07T15:06:53.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5190000000000003e-06 2020-08-07T15:06:53.009 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.010 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:53.010 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.189808+0000", 2020-08-07T15:06:53.010 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0105999999999999e-05 2020-08-07T15:06:53.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.011 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:53.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.189849+0000", 2020-08-07T15:06:53.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5304e-05 2020-08-07T15:06:53.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.012 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:53.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.189874+0000", 2020-08-07T15:06:53.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026358559999999998 2020-08-07T15:06:53.012 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.013 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:53.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.192510+0000", 2020-08-07T15:06:53.013 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6958e-05 2020-08-07T15:06:53.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.014 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:53.014 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.192557+0000", 2020-08-07T15:06:53.014 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13659865900000001 2020-08-07T15:06:53.014 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.014 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:53.015 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.015 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.015 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:16640 3.6 e27/26 3:63b550b6:::benchmark_data_smithi074_35447_object2079:head v 27'1104, mlcod=27'1096)", 2020-08-07T15:06:53.015 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.390454+0000", 2020-08-07T15:06:53.016 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.2780622060000004, 2020-08-07T15:06:53.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.144390454, 2020-08-07T15:06:53.016 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:53.016 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:53.016 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:53.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:53.017 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.390454+0000", 2020-08-07T15:06:53.017 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:53.017 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.018 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.018 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:53.018 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.390454+0000", 2020-08-07T15:06:53.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0198999999999998e-05 2020-08-07T15:06:53.018 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.018 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.019 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:53.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.390505+0000", 2020-08-07T15:06:53.019 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7270000000000001e-06 2020-08-07T15:06:53.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.019 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.020 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:53.020 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.390511+0000", 2020-08-07T15:06:53.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14174463700000001 2020-08-07T15:06:53.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.020 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:53.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.532256+0000", 2020-08-07T15:06:53.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0260000000000002e-06 2020-08-07T15:06:53.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:53.022 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.532263+0000", 2020-08-07T15:06:53.022 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6328000000000001e-05 2020-08-07T15:06:53.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:53.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.532309+0000", 2020-08-07T15:06:53.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4360000000000001e-05 2020-08-07T15:06:53.023 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.023 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:53.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.532334+0000", 2020-08-07T15:06:53.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024563200000000001 2020-08-07T15:06:53.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:53.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.534790+0000", 2020-08-07T15:06:53.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4857e-05 2020-08-07T15:06:53.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:53.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.534845+0000", 2020-08-07T15:06:53.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.144390454 2020-08-07T15:06:53.026 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.026 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:53.026 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.027 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:18264 3.b e27/20 3:d79881d2:::benchmark_data_smithi074_35447_object2282:head v 27'1224, mlcod=27'1213)", 2020-08-07T15:06:53.027 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:48.412254+0000", 2020-08-07T15:06:53.027 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4.2562623589999999, 2020-08-07T15:06:53.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.129284388, 2020-08-07T15:06:53.028 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:53.028 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:53.028 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:53.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:53.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.412254+0000", 2020-08-07T15:06:53.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:53.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:53.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.412254+0000", 2020-08-07T15:06:53.030 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011058800000000001 2020-08-07T15:06:53.030 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.030 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.030 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:53.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.412365+0000", 2020-08-07T15:06:53.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.268e-06 2020-08-07T15:06:53.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:53.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.412371+0000", 2020-08-07T15:06:53.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.127203449 2020-08-07T15:06:53.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:53.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.539575+0000", 2020-08-07T15:06:53.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.702e-06 2020-08-07T15:06:53.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:53.033 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.539579+0000", 2020-08-07T15:06:53.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3665000000000002e-05 2020-08-07T15:06:53.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.034 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.034 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:53.034 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.539633+0000", 2020-08-07T15:06:53.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6284e-05 2020-08-07T15:06:53.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:53.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.539669+0000", 2020-08-07T15:06:53.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0018059580000000001 2020-08-07T15:06:53.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:53.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.541475+0000", 2020-08-07T15:06:53.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3473999999999996e-05 2020-08-07T15:06:53.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.037 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:53.037 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.541539+0000", 2020-08-07T15:06:53.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.129284388 2020-08-07T15:06:53.037 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.038 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:53.038 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.038 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.038 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.038 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:24040 3.f e28/20 3:fef349f0:::benchmark_data_smithi074_35447_object3004:head v 28'1776, mlcod=28'1769)", 2020-08-07T15:06:53.039 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.178634+0000", 2020-08-07T15:06:53.039 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.48988245800000002, 2020-08-07T15:06:53.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13518461400000001, 2020-08-07T15:06:53.039 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:53.039 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:53.039 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:53.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:53.040 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.178634+0000", 2020-08-07T15:06:53.040 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:53.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.041 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:53.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.178634+0000", 2020-08-07T15:06:53.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000134857 2020-08-07T15:06:53.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.042 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:53.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.178769+0000", 2020-08-07T15:06:53.042 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1680000000000001e-06 2020-08-07T15:06:53.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.042 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:53.043 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.178775+0000", 2020-08-07T15:06:53.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13391025300000001 2020-08-07T15:06:53.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.044 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:53.044 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.312685+0000", 2020-08-07T15:06:53.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2349999999999999e-06 2020-08-07T15:06:53.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:53.045 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.312694+0000", 2020-08-07T15:06:53.045 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6122000000000002e-05 2020-08-07T15:06:53.045 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.045 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:53.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.312750+0000", 2020-08-07T15:06:53.046 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3825000000000002e-05 2020-08-07T15:06:53.046 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.046 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.046 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:53.047 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.312784+0000", 2020-08-07T15:06:53.047 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00097391999999999997 2020-08-07T15:06:53.047 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.047 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:53.048 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.313758+0000", 2020-08-07T15:06:53.048 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1234000000000004e-05 2020-08-07T15:06:53.048 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.048 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.048 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:53.049 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.313819+0000", 2020-08-07T15:06:53.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13518461400000001 2020-08-07T15:06:53.049 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.049 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:53.049 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.049 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.050 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.050 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:24047 3.1 e28/26 3:87d059ba:::benchmark_data_smithi074_35447_object3005:head v 28'1439, mlcod=28'1429)", 2020-08-07T15:06:53.050 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.187365+0000", 2020-08-07T15:06:53.050 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.48115131100000003, 2020-08-07T15:06:53.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14839102400000001, 2020-08-07T15:06:53.051 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:53.051 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:53.051 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:53.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.051 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:53.052 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.187365+0000", 2020-08-07T15:06:53.052 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:53.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.052 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.052 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:53.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.187365+0000", 2020-08-07T15:06:53.053 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4627999999999999e-05 2020-08-07T15:06:53.053 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.053 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:53.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.187420+0000", 2020-08-07T15:06:53.054 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5010000000000001e-06 2020-08-07T15:06:53.054 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.054 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.054 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:53.054 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.187426+0000", 2020-08-07T15:06:53.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14725048399999999 2020-08-07T15:06:53.055 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.055 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:53.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.334677+0000", 2020-08-07T15:06:53.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.066e-06 2020-08-07T15:06:53.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:53.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.334683+0000", 2020-08-07T15:06:53.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4073000000000003e-05 2020-08-07T15:06:53.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:53.057 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.334727+0000", 2020-08-07T15:06:53.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2813999999999999e-05 2020-08-07T15:06:53.058 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.058 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.058 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:53.058 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.334750+0000", 2020-08-07T15:06:53.058 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00097284100000000001 2020-08-07T15:06:53.059 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.059 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.059 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:53.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.335723+0000", 2020-08-07T15:06:53.059 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3617e-05 2020-08-07T15:06:53.059 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:53.060 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.335756+0000", 2020-08-07T15:06:53.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14839102400000001 2020-08-07T15:06:53.060 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.061 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:53.061 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.061 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:24048 3.1 e28/26 3:87d059ba:::benchmark_data_smithi074_35447_object3005:head v 28'1440, mlcod=28'1429)", 2020-08-07T15:06:53.062 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.187973+0000", 2020-08-07T15:06:53.062 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.48054318200000001, 2020-08-07T15:06:53.062 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16042247300000001, 2020-08-07T15:06:53.062 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:53.062 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:53.062 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:53.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.063 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:53.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.187973+0000", 2020-08-07T15:06:53.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:53.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.064 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:53.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.187973+0000", 2020-08-07T15:06:53.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0841999999999998e-05 2020-08-07T15:06:53.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.065 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:53.065 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.188034+0000", 2020-08-07T15:06:53.065 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6880000000000002e-06 2020-08-07T15:06:53.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.066 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.067 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:53.067 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.188041+0000", 2020-08-07T15:06:53.067 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.15963588500000001 2020-08-07T15:06:53.067 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.067 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.068 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:53.068 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.347677+0000", 2020-08-07T15:06:53.068 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2729999999999999e-06 2020-08-07T15:06:53.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.068 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.069 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:53.069 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.347683+0000", 2020-08-07T15:06:53.069 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5182999999999998e-05 2020-08-07T15:06:53.069 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.069 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.070 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:53.070 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.347718+0000", 2020-08-07T15:06:53.070 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7371999999999999e-05 2020-08-07T15:06:53.070 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.070 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.070 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:53.071 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.347736+0000", 2020-08-07T15:06:53.071 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00060796199999999998 2020-08-07T15:06:53.071 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.071 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.071 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:53.072 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.348344+0000", 2020-08-07T15:06:53.072 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2268000000000002e-05 2020-08-07T15:06:53.072 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.072 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.072 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:53.073 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.348396+0000", 2020-08-07T15:06:53.073 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16042247300000001 2020-08-07T15:06:53.073 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.073 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:53.073 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.074 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.074 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:53.074 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:06:53.080 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:53.080+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:53.081 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:06:53.187 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:06:53.180+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:06:53.239 INFO:teuthology.orchestra.run.smithi074.stderr:set 3.7 pg_upmap_items mapping to [5->3,2->4,0->7] 2020-08-07T15:06:53.240 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:06:53.242 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 17 16 25664 25648 11.7858 12.2578 0.00653119 0.0105911 2020-08-07T15:06:53.295 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:53.285+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:53.301 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:06:53.301 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:06:53.302 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.302 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:25744 3.6 3:6833d28b:::benchmark_data_smithi074_35447_object3217:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:06:53.302 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:53.287175+0000", 2020-08-07T15:06:53.302 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.013316095999999999, 2020-08-07T15:06:53.303 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.013336905, 2020-08-07T15:06:53.304 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:53.304 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:06:53.306 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:53.306 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:53.306 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:53.306 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 25744 2020-08-07T15:06:53.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.307 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:53.307 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:53.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.287175+0000", 2020-08-07T15:06:53.308 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:53.308 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.309 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.309 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:53.310 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.287175+0000", 2020-08-07T15:06:53.310 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8717999999999997e-05 2020-08-07T15:06:53.310 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.310 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.310 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:53.311 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.287224+0000", 2020-08-07T15:06:53.311 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4280000000000001e-06 2020-08-07T15:06:53.311 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.311 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.311 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:53.312 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.287230+0000", 2020-08-07T15:06:53.312 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1090999999999999e-05 2020-08-07T15:06:53.312 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.312 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.312 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:53.313 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.287241+0000", 2020-08-07T15:06:53.313 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.913e-06 2020-08-07T15:06:53.313 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.313 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.313 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:53.314 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.287247+0000", 2020-08-07T15:06:53.314 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033180929999999998 2020-08-07T15:06:53.314 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.314 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.314 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:53.315 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.290565+0000", 2020-08-07T15:06:53.316 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000107397 2020-08-07T15:06:53.316 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.316 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.316 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:53.317 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.290673+0000", 2020-08-07T15:06:53.317 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000157532 2020-08-07T15:06:53.317 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.318 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.318 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:53.318 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.290830+0000", 2020-08-07T15:06:53.318 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00102592 2020-08-07T15:06:53.319 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.319 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.319 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:53.319 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.291856+0000", 2020-08-07T15:06:53.319 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032842430000000001 2020-08-07T15:06:53.320 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.320 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.320 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:53.320 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.295140+0000", 2020-08-07T15:06:53.320 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0079653350000000005 2020-08-07T15:06:53.321 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.321 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:53.321 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.321 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.321 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:06:53.322 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 1 2020-08-07T15:06:53.322 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:06:53.323 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:06:53.410 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:06:53.402+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:06:53.503 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:53.499+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:06:53.550 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:06:53.551 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:06:53.551 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:06:53.551 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:06:53.551 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:06:53.566 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:06:53.649 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:53.647+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:53.702 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:53.700+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:53.758 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:06:53.758 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:06:53.759 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:06:53.759 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:06:53.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.759 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:7424 3.0 3:0f05fc62:::benchmark_data_smithi074_35447_object927:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:53.759 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.655503+0000", 2020-08-07T15:06:53.760 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 13.092167119000001, 2020-08-07T15:06:53.760 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.161346657, 2020-08-07T15:06:53.760 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:53.760 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:53.760 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:53.761 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:53.761 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:53.761 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7424 2020-08-07T15:06:53.761 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.761 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:53.762 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.762 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:53.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.655503+0000", 2020-08-07T15:06:53.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:53.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.765 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:53.766 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.655503+0000", 2020-08-07T15:06:53.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2389999999999998e-05 2020-08-07T15:06:53.766 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.766 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:53.767 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.655556+0000", 2020-08-07T15:06:53.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4210000000000003e-06 2020-08-07T15:06:53.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:53.768 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.655561+0000", 2020-08-07T15:06:53.768 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2663999999999999e-05 2020-08-07T15:06:53.768 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.768 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.768 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:53.769 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.655574+0000", 2020-08-07T15:06:53.769 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7609999999999998e-06 2020-08-07T15:06:53.769 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.770 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.770 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:53.770 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.655579+0000", 2020-08-07T15:06:53.770 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0060481349999999996 2020-08-07T15:06:53.770 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.772 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.772 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:53.772 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.661627+0000", 2020-08-07T15:06:53.772 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011472 2020-08-07T15:06:53.772 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.773 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.774 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:53.774 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.661742+0000", 2020-08-07T15:06:53.774 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000205551 2020-08-07T15:06:53.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:06:53.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.661947+0000", 2020-08-07T15:06:53.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013687879999999999 2020-08-07T15:06:53.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:53.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.663316+0000", 2020-08-07T15:06:53.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1500000000000004e-07 2020-08-07T15:06:53.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.779 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:53.779 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.663317+0000", 2020-08-07T15:06:53.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9407999999999995e-05 2020-08-07T15:06:53.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.780 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:53.780 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.663406+0000", 2020-08-07T15:06:53.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00094817999999999996 2020-08-07T15:06:53.780 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.781 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:53.781 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.664354+0000", 2020-08-07T15:06:53.781 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.15240937399999999 2020-08-07T15:06:53.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.782 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.782 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:53.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.816764+0000", 2020-08-07T15:06:53.783 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0299999999999998e-07 2020-08-07T15:06:53.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:53.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.816764+0000", 2020-08-07T15:06:53.785 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5747000000000004e-05 2020-08-07T15:06:53.785 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:53.787 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.816850+0000", 2020-08-07T15:06:53.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.161346657 2020-08-07T15:06:53.787 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.788 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:53.788 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.788 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:7424 3.0 e24/20 ondisk, result = 0)", 2020-08-07T15:06:53.789 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.664869+0000", 2020-08-07T15:06:53.789 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 13.082801247000001, 2020-08-07T15:06:53.789 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.15201024399999999, 2020-08-07T15:06:53.789 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:53.789 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:06:53.790 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:53.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.790 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:53.790 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.664869+0000", 2020-08-07T15:06:53.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:53.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:53.791 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.664869+0000", 2020-08-07T15:06:53.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3584000000000002e-05 2020-08-07T15:06:53.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.792 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.792 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:53.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.664903+0000", 2020-08-07T15:06:53.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4979999999999997e-06 2020-08-07T15:06:53.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.793 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.793 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:53.793 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.664908+0000", 2020-08-07T15:06:53.793 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.150144792 2020-08-07T15:06:53.793 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.793 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.794 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:53.794 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.815053+0000", 2020-08-07T15:06:53.794 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4269999999999999e-06 2020-08-07T15:06:53.794 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.794 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.795 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:53.795 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.815059+0000", 2020-08-07T15:06:53.795 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0016879169999999999 2020-08-07T15:06:53.795 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.795 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.796 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:53.796 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.816747+0000", 2020-08-07T15:06:53.797 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9200000000000004e-06 2020-08-07T15:06:53.798 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.798 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.798 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:53.798 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.816754+0000", 2020-08-07T15:06:53.798 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012510599999999999 2020-08-07T15:06:53.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:53.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.816880+0000", 2020-08-07T15:06:53.799 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.15201024399999999 2020-08-07T15:06:53.799 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.800 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:53.800 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.800 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:7863 3.8 e24/20 3:19e3d6c3:::benchmark_data_smithi074_35447_object982:head v 24'479, mlcod=24'470)", 2020-08-07T15:06:53.801 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.906441+0000", 2020-08-07T15:06:53.801 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 12.841229768, 2020-08-07T15:06:53.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14016840999999999, 2020-08-07T15:06:53.801 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:53.801 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:53.802 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:53.802 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.802 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:53.802 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.906441+0000", 2020-08-07T15:06:53.802 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:53.802 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:53.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.906441+0000", 2020-08-07T15:06:53.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9838e-05 2020-08-07T15:06:53.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:53.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.906491+0000", 2020-08-07T15:06:53.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2230000000000004e-06 2020-08-07T15:06:53.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:53.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.906497+0000", 2020-08-07T15:06:53.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13934068199999999 2020-08-07T15:06:53.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:53.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.045838+0000", 2020-08-07T15:06:53.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4530000000000001e-06 2020-08-07T15:06:53.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.808 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:53.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.045843+0000", 2020-08-07T15:06:53.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5800000000000002e-05 2020-08-07T15:06:53.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.810 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.810 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:53.810 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.045889+0000", 2020-08-07T15:06:53.810 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3988000000000001e-05 2020-08-07T15:06:53.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.811 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:53.811 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.045913+0000", 2020-08-07T15:06:53.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00064511500000000003 2020-08-07T15:06:53.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:53.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.046558+0000", 2020-08-07T15:06:53.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1310999999999997e-05 2020-08-07T15:06:53.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.813 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:53.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.046609+0000", 2020-08-07T15:06:53.813 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14016840999999999 2020-08-07T15:06:53.813 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.814 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:53.814 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.814 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.814 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.814 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:7864 3.8 e24/20 3:19e3d6c3:::benchmark_data_smithi074_35447_object982:head v 24'480, mlcod=24'470)", 2020-08-07T15:06:53.815 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.906526+0000", 2020-08-07T15:06:53.815 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 12.841144587, 2020-08-07T15:06:53.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.142735994, 2020-08-07T15:06:53.815 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:53.815 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:53.816 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:53.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:53.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.906526+0000", 2020-08-07T15:06:53.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:53.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.817 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:53.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.906526+0000", 2020-08-07T15:06:53.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7259000000000003e-05 2020-08-07T15:06:53.818 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.818 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.818 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:53.818 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.906573+0000", 2020-08-07T15:06:53.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5500000000000002e-06 2020-08-07T15:06:53.818 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:53.819 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.906579+0000", 2020-08-07T15:06:53.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14077097 2020-08-07T15:06:53.819 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:53.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.047350+0000", 2020-08-07T15:06:53.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.596e-06 2020-08-07T15:06:53.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:53.821 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.047353+0000", 2020-08-07T15:06:53.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9909000000000001e-05 2020-08-07T15:06:53.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:53.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.047383+0000", 2020-08-07T15:06:53.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4171e-05 2020-08-07T15:06:53.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:53.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.047397+0000", 2020-08-07T15:06:53.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001824621 2020-08-07T15:06:53.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:53.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.049222+0000", 2020-08-07T15:06:53.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9918e-05 2020-08-07T15:06:53.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:53.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.049262+0000", 2020-08-07T15:06:53.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.142735994 2020-08-07T15:06:53.834 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.835 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:53.835 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.835 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:8368 3.a e24/20 3:5e6bacde:::benchmark_data_smithi074_35447_object1045:head v 24'672, mlcod=24'665)", 2020-08-07T15:06:53.836 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.280770+0000", 2020-08-07T15:06:53.836 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 12.466900427000001, 2020-08-07T15:06:53.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13593296299999999, 2020-08-07T15:06:53.836 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:53.836 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:53.837 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:53.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:53.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.280770+0000", 2020-08-07T15:06:53.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:53.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.838 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:53.838 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.280770+0000", 2020-08-07T15:06:53.838 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0998999999999999e-05 2020-08-07T15:06:53.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.839 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:53.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.280801+0000", 2020-08-07T15:06:53.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1540000000000004e-06 2020-08-07T15:06:53.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.840 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.840 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:53.840 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.280805+0000", 2020-08-07T15:06:53.840 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13030248799999999 2020-08-07T15:06:53.840 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:53.841 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.411108+0000", 2020-08-07T15:06:53.841 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1769999999999997e-06 2020-08-07T15:06:53.841 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.842 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.842 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:53.842 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.411113+0000", 2020-08-07T15:06:53.842 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3248999999999999e-05 2020-08-07T15:06:53.842 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.843 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:53.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.411156+0000", 2020-08-07T15:06:53.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6160999999999999e-05 2020-08-07T15:06:53.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.844 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:53.844 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.411182+0000", 2020-08-07T15:06:53.845 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054520660000000002 2020-08-07T15:06:53.845 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:53.846 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.416634+0000", 2020-08-07T15:06:53.846 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8669000000000003e-05 2020-08-07T15:06:53.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.846 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.846 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:53.847 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.416703+0000", 2020-08-07T15:06:53.847 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13593296299999999 2020-08-07T15:06:53.847 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.847 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:53.847 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.847 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.848 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.848 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:9352 3.e e24/20 3:7113579b:::benchmark_data_smithi074_35447_object1168:head v 24'656, mlcod=24'648)", 2020-08-07T15:06:53.848 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.889020+0000", 2020-08-07T15:06:53.848 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.858650112999999, 2020-08-07T15:06:53.848 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24471712600000001, 2020-08-07T15:06:53.849 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:53.849 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:53.849 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:53.849 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.849 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:53.850 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.889020+0000", 2020-08-07T15:06:53.850 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:53.850 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.850 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:53.850 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.889020+0000", 2020-08-07T15:06:53.851 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7401999999999998e-05 2020-08-07T15:06:53.851 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.851 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.851 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:53.851 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.889068+0000", 2020-08-07T15:06:53.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.669e-06 2020-08-07T15:06:53.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:53.852 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.889075+0000", 2020-08-07T15:06:53.853 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23823541500000001 2020-08-07T15:06:53.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.853 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:53.853 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.127310+0000", 2020-08-07T15:06:53.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.895e-06 2020-08-07T15:06:53.854 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.854 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.854 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:53.854 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.127314+0000", 2020-08-07T15:06:53.855 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3778000000000001e-05 2020-08-07T15:06:53.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.855 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:53.855 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.127348+0000", 2020-08-07T15:06:53.855 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6517000000000001e-05 2020-08-07T15:06:53.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.856 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.856 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:53.856 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.127364+0000", 2020-08-07T15:06:53.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063235399999999999 2020-08-07T15:06:53.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:53.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.133688+0000", 2020-08-07T15:06:53.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9910000000000002e-05 2020-08-07T15:06:53.858 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.858 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:53.858 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.133738+0000", 2020-08-07T15:06:53.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24471712600000001 2020-08-07T15:06:53.858 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.859 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:53.859 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.859 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.859 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:9560 3.0 3:06f2f7cb:::benchmark_data_smithi074_35447_object1194:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:53.860 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.009069+0000", 2020-08-07T15:06:53.860 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.738601312, 2020-08-07T15:06:53.860 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.142485851, 2020-08-07T15:06:53.860 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:53.860 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:53.861 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:53.861 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:53.861 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:53.861 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 9560 2020-08-07T15:06:53.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.862 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:53.862 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.862 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:53.862 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.009069+0000", 2020-08-07T15:06:53.862 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:53.862 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:53.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.009069+0000", 2020-08-07T15:06:53.863 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9017999999999997e-05 2020-08-07T15:06:53.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:53.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.009118+0000", 2020-08-07T15:06:53.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0939999999999998e-06 2020-08-07T15:06:53.864 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.865 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:53.865 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.009123+0000", 2020-08-07T15:06:53.865 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0564000000000001e-05 2020-08-07T15:06:53.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.866 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:53.866 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.009134+0000", 2020-08-07T15:06:53.866 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5190000000000003e-06 2020-08-07T15:06:53.866 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.866 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.867 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:53.867 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.009138+0000", 2020-08-07T15:06:53.867 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045353879999999996 2020-08-07T15:06:53.867 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.867 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.867 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:53.868 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.013674+0000", 2020-08-07T15:06:53.868 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1734e-05 2020-08-07T15:06:53.868 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.868 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.868 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:53.869 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.013766+0000", 2020-08-07T15:06:53.869 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00026788499999999999 2020-08-07T15:06:53.869 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.869 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.869 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:06:53.870 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.014033+0000", 2020-08-07T15:06:53.870 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023794749999999998 2020-08-07T15:06:53.870 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.870 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.870 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:53.870 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.016413+0000", 2020-08-07T15:06:53.871 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2799999999999996e-07 2020-08-07T15:06:53.871 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.871 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:53.871 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.016414+0000", 2020-08-07T15:06:53.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5718999999999999e-05 2020-08-07T15:06:53.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.872 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.872 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:53.872 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.016479+0000", 2020-08-07T15:06:53.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0048736250000000004 2020-08-07T15:06:53.873 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.873 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:53.873 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.021353+0000", 2020-08-07T15:06:53.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.130146965 2020-08-07T15:06:53.874 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.874 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.874 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:53.874 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.151500+0000", 2020-08-07T15:06:53.874 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9999999999999996e-07 2020-08-07T15:06:53.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:53.875 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.151501+0000", 2020-08-07T15:06:53.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4336999999999998e-05 2020-08-07T15:06:53.876 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.876 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:53.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.151555+0000", 2020-08-07T15:06:53.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.142485851 2020-08-07T15:06:53.876 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.877 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:53.877 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.877 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.877 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.877 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:9688 3.6 e24/20 3:667d8bb7:::benchmark_data_smithi074_35447_object1210:head v 24'696, mlcod=24'689)", 2020-08-07T15:06:53.878 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.097125+0000", 2020-08-07T15:06:53.878 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.650545394, 2020-08-07T15:06:53.878 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13688761899999999, 2020-08-07T15:06:53.878 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:53.878 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:53.879 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:53.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.879 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:53.879 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.097125+0000", 2020-08-07T15:06:53.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:53.880 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.880 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:53.880 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.097125+0000", 2020-08-07T15:06:53.880 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6968e-05 2020-08-07T15:06:53.880 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.881 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.881 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:53.881 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.097152+0000", 2020-08-07T15:06:53.881 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9469999999999996e-06 2020-08-07T15:06:53.881 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.882 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.882 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:53.882 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.097156+0000", 2020-08-07T15:06:53.882 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13112575900000001 2020-08-07T15:06:53.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.884 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.884 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:53.885 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.228282+0000", 2020-08-07T15:06:53.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5199999999999997e-06 2020-08-07T15:06:53.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:53.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.228287+0000", 2020-08-07T15:06:53.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3969e-05 2020-08-07T15:06:53.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.886 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.886 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:53.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.228321+0000", 2020-08-07T15:06:53.887 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8095000000000001e-05 2020-08-07T15:06:53.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:53.887 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.228349+0000", 2020-08-07T15:06:53.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0056111959999999997 2020-08-07T15:06:53.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:53.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.233961+0000", 2020-08-07T15:06:53.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2164999999999999e-05 2020-08-07T15:06:53.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:53.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.234013+0000", 2020-08-07T15:06:53.890 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13688761899999999 2020-08-07T15:06:53.890 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.890 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:53.890 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.891 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:10240 3.6 e24/20 3:665d71ff:::benchmark_data_smithi074_35447_object1279:head v 24'728, mlcod=24'718)", 2020-08-07T15:06:53.891 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.394891+0000", 2020-08-07T15:06:53.891 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.352779755, 2020-08-07T15:06:53.891 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24904923400000001, 2020-08-07T15:06:53.891 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:53.892 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:53.892 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:53.892 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.892 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:53.892 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394891+0000", 2020-08-07T15:06:53.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:53.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.893 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:53.893 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394891+0000", 2020-08-07T15:06:53.893 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9815000000000003e-05 2020-08-07T15:06:53.894 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.894 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.894 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:53.894 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394941+0000", 2020-08-07T15:06:53.894 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5100000000000004e-06 2020-08-07T15:06:53.895 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.895 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.895 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:53.895 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394947+0000", 2020-08-07T15:06:53.895 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24715435899999999 2020-08-07T15:06:53.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.896 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:53.896 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.642101+0000", 2020-08-07T15:06:53.896 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7069999999999992e-06 2020-08-07T15:06:53.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.897 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.897 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:53.897 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.642109+0000", 2020-08-07T15:06:53.897 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9301000000000002e-05 2020-08-07T15:06:53.897 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.898 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.898 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:53.898 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.642148+0000", 2020-08-07T15:06:53.898 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7227999999999998e-05 2020-08-07T15:06:53.898 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.899 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.899 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:53.899 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.642166+0000", 2020-08-07T15:06:53.899 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017236300000000001 2020-08-07T15:06:53.899 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.900 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:53.900 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.643889+0000", 2020-08-07T15:06:53.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0683999999999998e-05 2020-08-07T15:06:53.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.901 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:53.901 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.643940+0000", 2020-08-07T15:06:53.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24904923400000001 2020-08-07T15:06:53.901 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.901 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:53.902 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.902 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.902 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.902 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:10535 3.0 3:0d011956:::benchmark_data_smithi074_35447_object1316:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:53.902 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.559858+0000", 2020-08-07T15:06:53.903 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.187812983000001, 2020-08-07T15:06:53.903 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14335071999999999, 2020-08-07T15:06:53.903 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:53.903 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:53.903 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:53.904 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:53.904 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:53.904 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 10535 2020-08-07T15:06:53.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.904 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:53.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:53.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.559858+0000", 2020-08-07T15:06:53.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:53.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.906 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:53.906 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.559858+0000", 2020-08-07T15:06:53.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5068999999999999e-05 2020-08-07T15:06:53.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.906 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:53.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.559893+0000", 2020-08-07T15:06:53.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9170000000000003e-06 2020-08-07T15:06:53.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:53.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.559899+0000", 2020-08-07T15:06:53.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1372999999999999e-05 2020-08-07T15:06:53.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.908 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:53.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.559910+0000", 2020-08-07T15:06:53.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.216e-05 2020-08-07T15:06:53.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:53.910 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.559922+0000", 2020-08-07T15:06:53.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033972569999999999 2020-08-07T15:06:53.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.910 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:53.910 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.563319+0000", 2020-08-07T15:06:53.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8225000000000003e-05 2020-08-07T15:06:53.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:53.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.563408+0000", 2020-08-07T15:06:53.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000150827 2020-08-07T15:06:53.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:06:53.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.563558+0000", 2020-08-07T15:06:53.914 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0020454929999999998 2020-08-07T15:06:53.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:53.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.565604+0000", 2020-08-07T15:06:53.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6600000000000002e-07 2020-08-07T15:06:53.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:53.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.565604+0000", 2020-08-07T15:06:53.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7257999999999999e-05 2020-08-07T15:06:53.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:53.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.565632+0000", 2020-08-07T15:06:53.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.081359644999999994 2020-08-07T15:06:53.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:53.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.646991+0000", 2020-08-07T15:06:53.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.056145957000000003 2020-08-07T15:06:53.918 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.918 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:53.918 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.703137+0000", 2020-08-07T15:06:53.918 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.671e-06 2020-08-07T15:06:53.919 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.919 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:53.919 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.703139+0000", 2020-08-07T15:06:53.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9502e-05 2020-08-07T15:06:53.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:53.920 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.703208+0000", 2020-08-07T15:06:53.920 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14335071999999999 2020-08-07T15:06:53.920 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.921 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:53.921 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.921 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:10536 3.0 3:0d011956:::benchmark_data_smithi074_35447_object1316:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:53.922 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.559957+0000", 2020-08-07T15:06:53.922 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.187713616, 2020-08-07T15:06:53.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.15294944999999999, 2020-08-07T15:06:53.922 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:53.922 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:53.923 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:53.923 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:53.923 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:53.923 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 10536 2020-08-07T15:06:53.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.924 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:53.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:53.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.559957+0000", 2020-08-07T15:06:53.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:53.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:53.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.559957+0000", 2020-08-07T15:06:53.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1481999999999999e-05 2020-08-07T15:06:53.925 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.926 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:53.926 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.560008+0000", 2020-08-07T15:06:53.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6189999999999999e-06 2020-08-07T15:06:53.926 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:53.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.560014+0000", 2020-08-07T15:06:53.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2164000000000001e-05 2020-08-07T15:06:53.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:53.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.560026+0000", 2020-08-07T15:06:53.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9100000000000004e-06 2020-08-07T15:06:53.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:53.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.560031+0000", 2020-08-07T15:06:53.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039962110000000004 2020-08-07T15:06:53.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:53.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.564027+0000", 2020-08-07T15:06:53.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000104509 2020-08-07T15:06:53.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:53.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.564132+0000", 2020-08-07T15:06:53.932 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019285099999999999 2020-08-07T15:06:53.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.933 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.933 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:06:53.933 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.564325+0000", 2020-08-07T15:06:53.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039900780000000002 2020-08-07T15:06:53.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.934 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.934 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:53.934 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.568315+0000", 2020-08-07T15:06:53.934 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.071e-06 2020-08-07T15:06:53.934 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:53.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.568316+0000", 2020-08-07T15:06:53.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5197999999999994e-05 2020-08-07T15:06:53.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:53.936 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.568391+0000", 2020-08-07T15:06:53.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.078854879000000003 2020-08-07T15:06:53.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:53.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.647246+0000", 2020-08-07T15:06:53.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.065591031999999994 2020-08-07T15:06:53.937 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.938 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:53.938 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.712837+0000", 2020-08-07T15:06:53.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.279e-06 2020-08-07T15:06:53.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:53.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.712838+0000", 2020-08-07T15:06:53.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8167e-05 2020-08-07T15:06:53.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.940 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:53.940 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.712906+0000", 2020-08-07T15:06:53.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.15294944999999999 2020-08-07T15:06:53.940 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.940 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:53.940 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.941 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:10535 3.0 e24/20 ondisk, result = 0)", 2020-08-07T15:06:53.941 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.567168+0000", 2020-08-07T15:06:53.941 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.180502667000001, 2020-08-07T15:06:53.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13604614300000001, 2020-08-07T15:06:53.942 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:53.942 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:06:53.942 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:53.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:53.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.567168+0000", 2020-08-07T15:06:53.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:53.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.944 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:53.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.567168+0000", 2020-08-07T15:06:53.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1197e-05 2020-08-07T15:06:53.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:53.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.567199+0000", 2020-08-07T15:06:53.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9739999999999999e-06 2020-08-07T15:06:53.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.946 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.946 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:53.946 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.567205+0000", 2020-08-07T15:06:53.946 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13586675400000001 2020-08-07T15:06:53.946 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.947 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.947 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:53.947 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.703072+0000", 2020-08-07T15:06:53.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8700000000000002e-06 2020-08-07T15:06:53.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.948 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:53.948 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.703076+0000", 2020-08-07T15:06:53.948 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2326000000000001e-05 2020-08-07T15:06:53.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.949 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:53.949 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.703118+0000", 2020-08-07T15:06:53.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7059999999999999e-06 2020-08-07T15:06:53.949 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.950 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:53.950 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.703128+0000", 2020-08-07T15:06:53.950 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6316000000000005e-05 2020-08-07T15:06:53.950 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.950 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:53.951 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.703214+0000", 2020-08-07T15:06:53.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13604614300000001 2020-08-07T15:06:53.951 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.951 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:53.951 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.952 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.952 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.952 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:15576 3.e e27/20 3:727724a5:::benchmark_data_smithi074_35447_object1946:head v 27'1032, mlcod=27'1023)", 2020-08-07T15:06:53.952 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:06:53.952 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 7.0781879029999999, 2020-08-07T15:06:53.953 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.46091973000000003, 2020-08-07T15:06:53.953 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:53.953 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:53.953 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:53.953 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.954 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:53.954 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:06:53.954 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:53.954 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.954 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.954 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:53.955 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:06:53.955 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9592000000000001e-05 2020-08-07T15:06:53.955 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.955 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.955 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:53.956 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669542+0000", 2020-08-07T15:06:53.956 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.113e-06 2020-08-07T15:06:53.956 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.956 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.956 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:53.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669547+0000", 2020-08-07T15:06:53.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45151617799999999 2020-08-07T15:06:53.957 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.957 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:53.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121064+0000", 2020-08-07T15:06:53.958 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.377e-06 2020-08-07T15:06:53.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:53.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121067+0000", 2020-08-07T15:06:53.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9195e-05 2020-08-07T15:06:53.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:53.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121106+0000", 2020-08-07T15:06:53.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7509000000000001e-05 2020-08-07T15:06:53.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:53.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121124+0000", 2020-08-07T15:06:53.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0091783100000000003 2020-08-07T15:06:53.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:53.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.130302+0000", 2020-08-07T15:06:53.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000100456 2020-08-07T15:06:53.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.962 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:53.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.130402+0000", 2020-08-07T15:06:53.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.46091973000000003 2020-08-07T15:06:53.963 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.963 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:53.963 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.963 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:16440 3.4 e27/20 3:21d0104c:::benchmark_data_smithi074_35447_object2054:head v 27'976, mlcod=27'967)", 2020-08-07T15:06:53.964 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.251240+0000", 2020-08-07T15:06:53.964 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.4964306260000004, 2020-08-07T15:06:53.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14202326100000001, 2020-08-07T15:06:53.964 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:53.964 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:53.965 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:53.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:53.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.251240+0000", 2020-08-07T15:06:53.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:53.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:53.966 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.251240+0000", 2020-08-07T15:06:53.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5843999999999998e-05 2020-08-07T15:06:53.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:53.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.251276+0000", 2020-08-07T15:06:53.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8119999999999998e-06 2020-08-07T15:06:53.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:53.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.251281+0000", 2020-08-07T15:06:53.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14048959599999999 2020-08-07T15:06:53.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.969 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.969 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:53.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.391770+0000", 2020-08-07T15:06:53.969 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9490000000000004e-06 2020-08-07T15:06:53.969 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.970 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:53.970 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.391775+0000", 2020-08-07T15:06:53.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0737999999999999e-05 2020-08-07T15:06:53.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:53.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.391846+0000", 2020-08-07T15:06:53.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5124e-05 2020-08-07T15:06:53.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:53.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.391861+0000", 2020-08-07T15:06:53.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013707319999999999 2020-08-07T15:06:53.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:53.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.393232+0000", 2020-08-07T15:06:53.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1466e-05 2020-08-07T15:06:53.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:53.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.393263+0000", 2020-08-07T15:06:53.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14202326100000001 2020-08-07T15:06:53.974 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.974 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:53.974 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.975 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:16639 3.6 3:63b550b6:::benchmark_data_smithi074_35447_object2079:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:06:53.975 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.386489+0000", 2020-08-07T15:06:53.975 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.3611814139999998, 2020-08-07T15:06:53.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.15254161899999999, 2020-08-07T15:06:53.976 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:53.976 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:53.976 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:53.976 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:53.977 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:53.977 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 16639 2020-08-07T15:06:53.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.977 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:53.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:53.978 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.386489+0000", 2020-08-07T15:06:53.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:53.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.978 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:53.979 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.386489+0000", 2020-08-07T15:06:53.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7886999999999997e-05 2020-08-07T15:06:53.979 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.979 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.979 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:53.980 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.386537+0000", 2020-08-07T15:06:53.980 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2020000000000003e-06 2020-08-07T15:06:53.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:53.980 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.386542+0000", 2020-08-07T15:06:53.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2301e-05 2020-08-07T15:06:53.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.981 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:53.981 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.386555+0000", 2020-08-07T15:06:53.982 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4279999999999998e-06 2020-08-07T15:06:53.982 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.982 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:53.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.386559+0000", 2020-08-07T15:06:53.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023435449999999998 2020-08-07T15:06:53.983 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.983 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.983 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:53.983 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.388903+0000", 2020-08-07T15:06:53.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8611999999999998e-05 2020-08-07T15:06:53.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.984 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:53.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.388991+0000", 2020-08-07T15:06:53.985 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000110257 2020-08-07T15:06:53.985 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.985 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:53.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.389101+0000", 2020-08-07T15:06:53.986 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001796478 2020-08-07T15:06:53.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.986 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:53.986 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.390898+0000", 2020-08-07T15:06:53.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1900000000000002e-07 2020-08-07T15:06:53.987 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.987 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:53.987 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.390898+0000", 2020-08-07T15:06:53.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3504999999999996e-05 2020-08-07T15:06:53.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:53.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.390962+0000", 2020-08-07T15:06:53.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002068814 2020-08-07T15:06:53.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:53.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.393031+0000", 2020-08-07T15:06:53.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14592200999999999 2020-08-07T15:06:53.990 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.990 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.990 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:53.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.538953+0000", 2020-08-07T15:06:53.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1299999999999999e-07 2020-08-07T15:06:53.990 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:53.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.538954+0000", 2020-08-07T15:06:53.991 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7148000000000001e-05 2020-08-07T15:06:53.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:53.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.539031+0000", 2020-08-07T15:06:53.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.15254161899999999 2020-08-07T15:06:53.992 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.993 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:53.993 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:53.993 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.993 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.993 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:16640 3.6 3:63b550b6:::benchmark_data_smithi074_35447_object2079:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:06:53.994 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.386594+0000", 2020-08-07T15:06:53.994 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.3610763170000002, 2020-08-07T15:06:53.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.164350463, 2020-08-07T15:06:53.994 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:53.994 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:53.994 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:53.995 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:53.995 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:53.995 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 16640 2020-08-07T15:06:53.995 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.995 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:53.996 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.996 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:53.996 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.386594+0000", 2020-08-07T15:06:53.996 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:53.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.997 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.997 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:53.997 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.386594+0000", 2020-08-07T15:06:53.997 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6712999999999998e-05 2020-08-07T15:06:53.997 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.998 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.998 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:53.998 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.386641+0000", 2020-08-07T15:06:53.998 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3289999999999999e-06 2020-08-07T15:06:53.998 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.999 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:53.999 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:53.999 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.386647+0000", 2020-08-07T15:06:53.999 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5189e-05 2020-08-07T15:06:53.999 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:53.999 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.000 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:54.000 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.386662+0000", 2020-08-07T15:06:54.000 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5159999999999997e-06 2020-08-07T15:06:54.000 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.000 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.001 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:54.001 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.386670+0000", 2020-08-07T15:06:54.001 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034020579999999999 2020-08-07T15:06:54.001 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.001 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.001 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:54.002 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.390072+0000", 2020-08-07T15:06:54.002 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4134999999999994e-05 2020-08-07T15:06:54.002 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.002 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.002 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:54.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.390146+0000", 2020-08-07T15:06:54.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015176400000000001 2020-08-07T15:06:54.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:54.004 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.390298+0000", 2020-08-07T15:06:54.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00307179 2020-08-07T15:06:54.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:54.004 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.393370+0000", 2020-08-07T15:06:54.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8599999999999996e-07 2020-08-07T15:06:54.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:54.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.393371+0000", 2020-08-07T15:06:54.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0308e-05 2020-08-07T15:06:54.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:54.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.393421+0000", 2020-08-07T15:06:54.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14211117600000001 2020-08-07T15:06:54.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:54.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.535532+0000", 2020-08-07T15:06:54.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.01534336 2020-08-07T15:06:54.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:54.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.550876+0000", 2020-08-07T15:06:54.008 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3149999999999999e-06 2020-08-07T15:06:54.009 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.009 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.009 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:54.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.550877+0000", 2020-08-07T15:06:54.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7824000000000007e-05 2020-08-07T15:06:54.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.010 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:54.010 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.550945+0000", 2020-08-07T15:06:54.010 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.164350463 2020-08-07T15:06:54.010 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:54.011 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:54.011 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:54.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.011 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:16639 3.6 e27/26 ondisk, result = 0)", 2020-08-07T15:06:54.012 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.394185+0000", 2020-08-07T15:06:54.012 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.3534853450000002, 2020-08-07T15:06:54.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14485044399999999, 2020-08-07T15:06:54.012 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:54.012 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:06:54.013 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:54.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.013 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:54.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.394185+0000", 2020-08-07T15:06:54.014 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:54.014 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.014 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:54.014 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.394185+0000", 2020-08-07T15:06:54.015 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5434000000000002e-05 2020-08-07T15:06:54.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.015 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.015 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:54.015 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.394201+0000", 2020-08-07T15:06:54.015 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6350000000000002e-06 2020-08-07T15:06:54.016 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.016 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:54.016 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.394203+0000", 2020-08-07T15:06:54.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.142895313 2020-08-07T15:06:54.017 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:54.017 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.537099+0000", 2020-08-07T15:06:54.017 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9379999999999997e-06 2020-08-07T15:06:54.018 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.018 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.018 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:54.018 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.537105+0000", 2020-08-07T15:06:54.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0018291939999999999 2020-08-07T15:06:54.018 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.019 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.019 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:54.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.538934+0000", 2020-08-07T15:06:54.019 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5310000000000007e-06 2020-08-07T15:06:54.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.020 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.020 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:54.020 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.538943+0000", 2020-08-07T15:06:54.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2398999999999994e-05 2020-08-07T15:06:54.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:54.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.539036+0000", 2020-08-07T15:06:54.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14485044399999999 2020-08-07T15:06:54.021 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:54.021 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:54.022 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:54.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.022 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:24264 3.8 e28/20 3:1d3fdbda:::benchmark_data_smithi074_35447_object3032:head v 28'1520, mlcod=28'1515)", 2020-08-07T15:06:54.022 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:06:54.023 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 1.364557866, 2020-08-07T15:06:54.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41856254199999998, 2020-08-07T15:06:54.023 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:54.023 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:54.023 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:54.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:54.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:06:54.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:54.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:54.026 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:06:54.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1348999999999998e-05 2020-08-07T15:06:54.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:54.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383174+0000", 2020-08-07T15:06:54.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6579999999999996e-06 2020-08-07T15:06:54.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:54.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383181+0000", 2020-08-07T15:06:54.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41194352699999998 2020-08-07T15:06:54.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:54.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.795124+0000", 2020-08-07T15:06:54.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5729999999999999e-06 2020-08-07T15:06:54.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.030 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:54.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.795128+0000", 2020-08-07T15:06:54.030 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002170384 2020-08-07T15:06:54.030 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.030 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.030 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:54.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.797298+0000", 2020-08-07T15:06:54.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5412e-05 2020-08-07T15:06:54.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:54.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.797324+0000", 2020-08-07T15:06:54.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042943360000000002 2020-08-07T15:06:54.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:54.033 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.801618+0000", 2020-08-07T15:06:54.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7302999999999997e-05 2020-08-07T15:06:54.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:54.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.801675+0000", 2020-08-07T15:06:54.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41856254199999998 2020-08-07T15:06:54.036 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:54.037 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:54.037 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:54.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.037 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:24824 3.6 3:63691f2a:::benchmark_data_smithi074_35447_object3102:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e28)", 2020-08-07T15:06:54.038 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.732819+0000", 2020-08-07T15:06:54.038 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 1.014852037, 2020-08-07T15:06:54.038 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.152218515, 2020-08-07T15:06:54.038 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:54.039 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:54.039 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:54.039 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:54.039 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:54.039 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 24824 2020-08-07T15:06:54.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.040 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:54.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:54.040 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.732819+0000", 2020-08-07T15:06:54.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:54.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.041 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:54.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.732819+0000", 2020-08-07T15:06:54.042 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7262000000000002e-05 2020-08-07T15:06:54.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.042 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:54.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.732876+0000", 2020-08-07T15:06:54.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1390000000000003e-06 2020-08-07T15:06:54.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:54.043 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.732881+0000", 2020-08-07T15:06:54.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0428000000000001e-05 2020-08-07T15:06:54.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.044 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:54.044 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.732891+0000", 2020-08-07T15:06:54.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.673e-06 2020-08-07T15:06:54.045 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.045 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:54.045 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.732896+0000", 2020-08-07T15:06:54.045 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001717679 2020-08-07T15:06:54.046 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.046 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.046 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:54.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.734614+0000", 2020-08-07T15:06:54.046 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6438000000000002e-05 2020-08-07T15:06:54.047 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.047 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:54.047 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.734690+0000", 2020-08-07T15:06:54.047 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000118557 2020-08-07T15:06:54.048 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.048 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.048 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:54.048 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.734809+0000", 2020-08-07T15:06:54.048 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015055190000000001 2020-08-07T15:06:54.048 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.049 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.049 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:54.049 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.736314+0000", 2020-08-07T15:06:54.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2600000000000002e-07 2020-08-07T15:06:54.049 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.050 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.050 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:54.050 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.736315+0000", 2020-08-07T15:06:54.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7496e-05 2020-08-07T15:06:54.050 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.051 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:54.051 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.736372+0000", 2020-08-07T15:06:54.051 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0087590770000000005 2020-08-07T15:06:54.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.053 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:54.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.745131+0000", 2020-08-07T15:06:54.053 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13983647099999999 2020-08-07T15:06:54.054 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.054 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.054 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:54.054 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.884968+0000", 2020-08-07T15:06:54.054 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4000000000000001e-07 2020-08-07T15:06:54.054 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.055 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:54.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.884969+0000", 2020-08-07T15:06:54.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8510000000000006e-05 2020-08-07T15:06:54.055 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:54.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.885037+0000", 2020-08-07T15:06:54.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.152218515 2020-08-07T15:06:54.056 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:54.057 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:54.057 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:54.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.058 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:24992 3.0 3:008a8fe9:::benchmark_data_smithi074_35447_object3123:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e28)", 2020-08-07T15:06:54.058 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.846767+0000", 2020-08-07T15:06:54.058 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.90090333199999995, 2020-08-07T15:06:54.058 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21218208499999999, 2020-08-07T15:06:54.058 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:54.059 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:54.059 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:54.059 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:54.059 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:54.059 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 24992 2020-08-07T15:06:54.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.060 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:54.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:54.060 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.846767+0000", 2020-08-07T15:06:54.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:54.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.061 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:54.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.846767+0000", 2020-08-07T15:06:54.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0043000000000001e-05 2020-08-07T15:06:54.062 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.062 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.062 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:54.062 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.846827+0000", 2020-08-07T15:06:54.062 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1109999999999999e-06 2020-08-07T15:06:54.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.063 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:54.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.846832+0000", 2020-08-07T15:06:54.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.095e-05 2020-08-07T15:06:54.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.064 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:54.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.846843+0000", 2020-08-07T15:06:54.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3189999999999996e-06 2020-08-07T15:06:54.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.065 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:54.065 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.846848+0000", 2020-08-07T15:06:54.065 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0035319069999999999 2020-08-07T15:06:54.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.066 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.066 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:54.066 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.850380+0000", 2020-08-07T15:06:54.066 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000100331 2020-08-07T15:06:54.066 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.067 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.067 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:54.067 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.850480+0000", 2020-08-07T15:06:54.067 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000122416 2020-08-07T15:06:54.067 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.067 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.068 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:06:54.068 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.850602+0000", 2020-08-07T15:06:54.068 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002556928 2020-08-07T15:06:54.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.068 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.069 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:54.069 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.853159+0000", 2020-08-07T15:06:54.069 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5199999999999995e-07 2020-08-07T15:06:54.069 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.069 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.070 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:54.070 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.853160+0000", 2020-08-07T15:06:54.070 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1212999999999998e-05 2020-08-07T15:06:54.070 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.070 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.071 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:54.071 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.853221+0000", 2020-08-07T15:06:54.071 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017586749999999999 2020-08-07T15:06:54.071 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.071 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.071 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:54.072 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.854980+0000", 2020-08-07T15:06:54.072 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.203895095 2020-08-07T15:06:54.072 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.072 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.072 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:54.073 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.058875+0000", 2020-08-07T15:06:54.073 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6759999999999999e-06 2020-08-07T15:06:54.073 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.074 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.074 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:54.074 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.058877+0000", 2020-08-07T15:06:54.074 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2569000000000003e-05 2020-08-07T15:06:54.075 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.075 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.075 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:54.075 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.058949+0000", 2020-08-07T15:06:54.075 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21218208499999999 2020-08-07T15:06:54.076 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:54.076 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:54.076 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:54.076 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:54.076 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:54.077 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:06:54.077 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:06:54.078 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:06:53.933+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:06:54.081 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:06:54.078+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:06:54.182 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:54.178+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:54.240 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:06:54.240 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:06:54.241 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.241 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:27361 3.5 e29/20 3:a5a6bb63:::benchmark_data_smithi074_35447_object3420:head v 29'1553, mlcod=29'1543)", 2020-08-07T15:06:54.241 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:54.235840+0000", 2020-08-07T15:06:54.241 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0037888349999999999, 2020-08-07T15:06:54.241 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038135170000000002, 2020-08-07T15:06:54.242 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:54.242 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:06:54.242 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:54.242 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.242 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:54.243 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.235840+0000", 2020-08-07T15:06:54.243 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:54.243 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.243 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.243 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:54.244 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.235840+0000", 2020-08-07T15:06:54.244 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1711999999999997e-05 2020-08-07T15:06:54.244 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.244 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.244 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:54.245 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.235872+0000", 2020-08-07T15:06:54.245 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2259999999999996e-06 2020-08-07T15:06:54.245 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.245 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.246 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:54.246 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.235877+0000", 2020-08-07T15:06:54.246 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001620409 2020-08-07T15:06:54.246 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.246 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.247 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:54.247 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.237497+0000", 2020-08-07T15:06:54.247 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9310000000000001e-06 2020-08-07T15:06:54.247 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.247 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.248 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:54.248 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.237501+0000", 2020-08-07T15:06:54.248 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4777000000000003e-05 2020-08-07T15:06:54.248 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.248 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.249 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:54.249 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.237536+0000", 2020-08-07T15:06:54.249 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7051e-05 2020-08-07T15:06:54.249 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.250 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.250 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:54.250 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.237553+0000", 2020-08-07T15:06:54.250 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001713106 2020-08-07T15:06:54.251 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:54.251 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:54.251 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:54.251 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.251 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.252 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:27362 3.5 e29/20 3:a5a6bb63:::benchmark_data_smithi074_35447_object3420:head v 29'1554, mlcod=29'1543)", 2020-08-07T15:06:54.252 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:54.237445+0000", 2020-08-07T15:06:54.252 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0021835270000000002, 2020-08-07T15:06:54.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0022620180000000002, 2020-08-07T15:06:54.252 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:54.253 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:06:54.253 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:54.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.253 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:54.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.237445+0000", 2020-08-07T15:06:54.254 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:54.254 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.254 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.254 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:54.254 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.237445+0000", 2020-08-07T15:06:54.255 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0932999999999997e-05 2020-08-07T15:06:54.255 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.255 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.255 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:54.255 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.237476+0000", 2020-08-07T15:06:54.256 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1370000000000004e-06 2020-08-07T15:06:54.256 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.256 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.256 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:54.257 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.237480+0000", 2020-08-07T15:06:54.257 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00047001399999999999 2020-08-07T15:06:54.257 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.257 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.257 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:54.258 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.237950+0000", 2020-08-07T15:06:54.258 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0879999999999999e-06 2020-08-07T15:06:54.258 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.259 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.259 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:54.260 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.237953+0000", 2020-08-07T15:06:54.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5239e-05 2020-08-07T15:06:54.260 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.261 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.261 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:54.261 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.237979+0000", 2020-08-07T15:06:54.261 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6745999999999999e-05 2020-08-07T15:06:54.261 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:54.262 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:54.263 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:54.263 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.237995+0000", 2020-08-07T15:06:54.263 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00055015700000000001 2020-08-07T15:06:54.263 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:54.264 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:54.264 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:54.264 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:54.264 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:06:54.264 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 2 2020-08-07T15:06:54.265 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:06:54.266 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 18 16 27369 27353 11.8709 13.3203 0.0021798 0.0105139 2020-08-07T15:06:54.267 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:06:54.281 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:54.279+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:54.381 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:54.379+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:54.418 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:06:54.493 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:06:54.494 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:06:54.494 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:06:54.494 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:06:54.494 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:06:54.497 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:06:54.492+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:06:54.512 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:06:54.599 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:06:54.597+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:06:54.648 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:06:54.700 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:06:54.701 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:54.699+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:54.738 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:06:54.739 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:54.739 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:54.754 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:06:54.772 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:06:54.806 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:54.801+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:54.815 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:06:54.844 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:06:54.880 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:06:54.904 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:54.902+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:54.923 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:06:54.924 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:06:54.924 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:06:54.924 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:06:54.924 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:06:54.925 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:06:54.925 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:06:54.925 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:06:54.925 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:06:54.956 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:54.957 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:54.969 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:06:55.006 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:55.004+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:55.106 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:55.105+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:55.208 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:55.209 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:55.210 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:06:55.206+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:06:55.221 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:06:55.242 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 19 16 29013 28997 11.9221 12.8438 0.00768874 0.0104766 2020-08-07T15:06:55.256 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:06:55.336 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:06:55.330+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:06:55.416 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:55.412+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:55.434 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:55.435 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:55.448 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:06:55.515 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:55.513+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:55.611 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:55.612 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:55.616 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:06:55.614+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:06:55.623 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:06:55.719 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:06:55.717+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:06:55.792 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:55.792 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:55.805 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:06:55.835 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:55.833+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:55.966 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:55.963+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:06:55.984 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:55.985 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:56.003 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:06:56.108 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:56.093+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:56.173 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:56.169+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:56.231 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:56.231 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:56.243 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:2020-08-07T15:06:56.241673+0000 min lat: 0.00101515 max lat: 0.998834 avg lat: 0.0104282 2020-08-07T15:06:56.243 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-08-07T15:06:56.244 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 20 16 30663 30647 11.9705 12.8906 0.00469549 0.0104282 2020-08-07T15:06:56.244 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:06:56.250 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:06:56.274 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:56.271+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:56.374 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:56.372+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:56.446 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:56.447 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:56.455 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:06:56.510 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:56.506+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:56.582 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:06:56.580+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:06:56.628 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:56.628 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:56.639 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:06:56.684 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:56.682+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:56.786 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:56.784+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:56.855 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:56.856 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:56.881 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:06:56.895 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:56.886+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:56.939 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:06:57.011 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:56.993+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:57.086 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:57.086 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:57.090 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:57.089+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:06:57.098 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:06:57.151 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:06:57.184 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:06:57.191 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:06:57.189+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:06:57.221 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:06:57.242 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 21 16 32162 32146 11.9581 11.7109 0.0105728 0.0104479 2020-08-07T15:06:57.259 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:06:57.291 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:06:57.290+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:06:57.301 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:06:57.315 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:57.316 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:57.326 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:06:57.336 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:06:57.373 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:06:57.394 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:57.391+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:57.407 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:06:57.408 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:06:57.408 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:06:57.409 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:06:57.409 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:06:57.409 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:06:57.409 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:06:57.409 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:06:57.409 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:06:57.493 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:57.491+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:57.504 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:06:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:06:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:32586 3.9 e30/20 3:9d61b11a:::benchmark_data_smithi074_35447_object4073:head v 30'1866, mlcod=30'1863)", 2020-08-07T15:06:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:57.501769+0000", 2020-08-07T15:06:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.001468277, 2020-08-07T15:06:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001494207, 2020-08-07T15:06:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.501769+0000", 2020-08-07T15:06:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:57.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.501769+0000", 2020-08-07T15:06:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0521e-05 2020-08-07T15:06:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.501819+0000", 2020-08-07T15:06:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5719999999999999e-06 2020-08-07T15:06:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:57.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.501826+0000", 2020-08-07T15:06:57.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013094699999999999 2020-08-07T15:06:57.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.501957+0000", 2020-08-07T15:06:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1639999999999998e-06 2020-08-07T15:06:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.501963+0000", 2020-08-07T15:06:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8958999999999997e-05 2020-08-07T15:06:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.502002+0000", 2020-08-07T15:06:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1912e-05 2020-08-07T15:06:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.502024+0000", 2020-08-07T15:06:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025507500000000001 2020-08-07T15:06:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:57.539 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:57.540 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:57.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.540 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:32551 3.d 3:bd4ef7be:::benchmark_data_smithi074_35447_object4068:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-08-07T15:06:57.541 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:57.476386+0000", 2020-08-07T15:06:57.541 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.026850631, 2020-08-07T15:06:57.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.026961490000000001, 2020-08-07T15:06:57.541 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:57.541 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:57.542 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:57.542 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:57.542 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:57.542 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 32551 2020-08-07T15:06:57.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.543 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:57.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:57.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.476386+0000", 2020-08-07T15:06:57.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:57.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:57.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.476386+0000", 2020-08-07T15:06:57.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3106000000000003e-05 2020-08-07T15:06:57.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:57.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.476439+0000", 2020-08-07T15:06:57.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.19e-06 2020-08-07T15:06:57.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:57.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.476446+0000", 2020-08-07T15:06:57.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0188999999999998e-05 2020-08-07T15:06:57.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:57.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.476486+0000", 2020-08-07T15:06:57.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6799999999999995e-05 2020-08-07T15:06:57.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:57.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.476583+0000", 2020-08-07T15:06:57.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021280180000000002 2020-08-07T15:06:57.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:57.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.478711+0000", 2020-08-07T15:06:57.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010547 2020-08-07T15:06:57.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:57.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.478816+0000", 2020-08-07T15:06:57.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000181514 2020-08-07T15:06:57.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:06:57.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.478998+0000", 2020-08-07T15:06:57.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025062750000000001 2020-08-07T15:06:57.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:57.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.481504+0000", 2020-08-07T15:06:57.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2099999999999997e-07 2020-08-07T15:06:57.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:57.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.481504+0000", 2020-08-07T15:06:57.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0290999999999995e-05 2020-08-07T15:06:57.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:57.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.481575+0000", 2020-08-07T15:06:57.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014648234 2020-08-07T15:06:57.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:57.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.496223+0000", 2020-08-07T15:06:57.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019836607999999999 2020-08-07T15:06:57.568 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:57.569 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:57.569 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:57.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.569 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:32584 3.7 e30/20 3:e6cece44:::benchmark_data_smithi074_35447_object4072:head v 30'1976, mlcod=30'1967)", 2020-08-07T15:06:57.570 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:57.502011+0000", 2020-08-07T15:06:57.570 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0012255790000000001, 2020-08-07T15:06:57.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001461777, 2020-08-07T15:06:57.570 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:57.570 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:57.571 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:57.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:57.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.502011+0000", 2020-08-07T15:06:57.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:57.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:57.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.502011+0000", 2020-08-07T15:06:57.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6978000000000003e-05 2020-08-07T15:06:57.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:57.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.502058+0000", 2020-08-07T15:06:57.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0950000000000001e-06 2020-08-07T15:06:57.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:57.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.502064+0000", 2020-08-07T15:06:57.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3878999999999998e-05 2020-08-07T15:06:57.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:57.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.502108+0000", 2020-08-07T15:06:57.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4850000000000003e-06 2020-08-07T15:06:57.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:57.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.502113+0000", 2020-08-07T15:06:57.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000181836 2020-08-07T15:06:57.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:57.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.502295+0000", 2020-08-07T15:06:57.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3705e-05 2020-08-07T15:06:57.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:57.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.502308+0000", 2020-08-07T15:06:57.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029697799999999999 2020-08-07T15:06:57.592 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:57.593 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:57.593 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:57.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.594 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:32552 3.d 3:bd4ef7be:::benchmark_data_smithi074_35447_object4068:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-08-07T15:06:57.594 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:57.476620+0000", 2020-08-07T15:06:57.594 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.026617318000000001, 2020-08-07T15:06:57.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.026928411999999999, 2020-08-07T15:06:57.594 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:57.595 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:57.595 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:57.595 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:57.595 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:57.596 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 32552 2020-08-07T15:06:57.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.596 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:57.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:57.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.476620+0000", 2020-08-07T15:06:57.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:57.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:57.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.476620+0000", 2020-08-07T15:06:57.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9217e-05 2020-08-07T15:06:57.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:57.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.476669+0000", 2020-08-07T15:06:57.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7239999999999997e-06 2020-08-07T15:06:57.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:57.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.476675+0000", 2020-08-07T15:06:57.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0786e-05 2020-08-07T15:06:57.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:57.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.476685+0000", 2020-08-07T15:06:57.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.899e-06 2020-08-07T15:06:57.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:57.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.476690+0000", 2020-08-07T15:06:57.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035002200000000001 2020-08-07T15:06:57.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:57.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.480191+0000", 2020-08-07T15:06:57.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3560999999999994e-05 2020-08-07T15:06:57.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:57.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.480284+0000", 2020-08-07T15:06:57.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018455699999999999 2020-08-07T15:06:57.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:06:57.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.480469+0000", 2020-08-07T15:06:57.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018403790000000001 2020-08-07T15:06:57.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:57.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.482309+0000", 2020-08-07T15:06:57.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7300000000000005e-07 2020-08-07T15:06:57.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:57.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.482310+0000", 2020-08-07T15:06:57.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5977999999999997e-05 2020-08-07T15:06:57.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:57.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.482376+0000", 2020-08-07T15:06:57.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013881572999999999 2020-08-07T15:06:57.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:57.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.496257+0000", 2020-08-07T15:06:57.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019637667000000001 2020-08-07T15:06:57.621 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:57.621 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:57.621 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:57.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.623 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:32568 3.f 3:fc89e945:::benchmark_data_smithi074_35447_object4070:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-08-07T15:06:57.623 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:57.486387+0000", 2020-08-07T15:06:57.623 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.016850163000000001, 2020-08-07T15:06:57.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017276379000000001, 2020-08-07T15:06:57.623 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:57.624 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:57.624 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:57.624 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:57.624 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:57.624 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 32568 2020-08-07T15:06:57.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.625 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:57.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:57.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.486387+0000", 2020-08-07T15:06:57.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:57.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:57.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.486387+0000", 2020-08-07T15:06:57.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7744000000000001e-05 2020-08-07T15:06:57.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:57.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.486415+0000", 2020-08-07T15:06:57.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3550000000000001e-06 2020-08-07T15:06:57.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:57.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.486418+0000", 2020-08-07T15:06:57.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6699999999999997e-06 2020-08-07T15:06:57.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:57.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.486425+0000", 2020-08-07T15:06:57.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2349999999999999e-06 2020-08-07T15:06:57.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:57.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.486428+0000", 2020-08-07T15:06:57.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011083226999999999 2020-08-07T15:06:57.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:57.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.497511+0000", 2020-08-07T15:06:57.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8552000000000001e-05 2020-08-07T15:06:57.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:57.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.497610+0000", 2020-08-07T15:06:57.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019249499999999999 2020-08-07T15:06:57.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:57.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.497802+0000", 2020-08-07T15:06:57.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021783290000000001 2020-08-07T15:06:57.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:57.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.499980+0000", 2020-08-07T15:06:57.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9699999999999996e-07 2020-08-07T15:06:57.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:57.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.499981+0000", 2020-08-07T15:06:57.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2079e-05 2020-08-07T15:06:57.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:57.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.500043+0000", 2020-08-07T15:06:57.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001202236 2020-08-07T15:06:57.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:57.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.501245+0000", 2020-08-07T15:06:57.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014858418999999999 2020-08-07T15:06:57.639 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:57.639 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:57.639 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:57.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.640 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:32587 3.9 e30/20 3:9d61b11a:::benchmark_data_smithi074_35447_object4073:head v 30'1867, mlcod=30'1863)", 2020-08-07T15:06:57.640 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:57.502546+0000", 2020-08-07T15:06:57.640 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00069099099999999996, 2020-08-07T15:06:57.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012272679999999999, 2020-08-07T15:06:57.641 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:57.641 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:57.641 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:57.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:57.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.502546+0000", 2020-08-07T15:06:57.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:57.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:57.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.502546+0000", 2020-08-07T15:06:57.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024554500000000001 2020-08-07T15:06:57.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:57.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.502792+0000", 2020-08-07T15:06:57.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4579999999999998e-06 2020-08-07T15:06:57.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:57.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.502798+0000", 2020-08-07T15:06:57.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9394000000000005e-05 2020-08-07T15:06:57.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:57.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.502887+0000", 2020-08-07T15:06:57.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4720000000000003e-06 2020-08-07T15:06:57.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:57.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.502893+0000", 2020-08-07T15:06:57.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7329e-05 2020-08-07T15:06:57.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:57.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.502940+0000", 2020-08-07T15:06:57.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1753999999999999e-05 2020-08-07T15:06:57.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:57.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:57.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:57.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.502962+0000", 2020-08-07T15:06:57.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00041595199999999998 2020-08-07T15:06:57.649 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:57.650 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:57.650 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:57.650 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:57.650 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:06:57.650 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 6 2020-08-07T15:06:57.650 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:06:57.652 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:06:57.653 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:57.651+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:57.755 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:57.751+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:57.858 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:57.856+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:57.867 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:06:57.868 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:06:57.868 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:06:57.868 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:06:57.868 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:06:57.881 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:06:57.959 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:06:57.957+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:06:58.057 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:06:58.058 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:06:58.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:06:58.058 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:06:58.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.059 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:2400 3.e 3:70ffa62d:::benchmark_data_smithi074_35447_object299:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:06:58.059 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.694617+0000", 2020-08-07T15:06:58.059 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.359160116999998, 2020-08-07T15:06:58.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24075575099999999, 2020-08-07T15:06:58.059 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:58.060 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:58.060 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:58.060 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:58.060 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:58.060 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2400 2020-08-07T15:06:58.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.061 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:58.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:58.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.694617+0000", 2020-08-07T15:06:58.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:58.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:58.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.694617+0000", 2020-08-07T15:06:58.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5681999999999999e-05 2020-08-07T15:06:58.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:58.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.694643+0000", 2020-08-07T15:06:58.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0079999999999998e-06 2020-08-07T15:06:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.694646+0000", 2020-08-07T15:06:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.015e-06 2020-08-07T15:06:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:58.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.694652+0000", 2020-08-07T15:06:58.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7130000000000001e-06 2020-08-07T15:06:58.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:58.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.694655+0000", 2020-08-07T15:06:58.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031117319999999999 2020-08-07T15:06:58.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:58.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.697766+0000", 2020-08-07T15:06:58.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015382800000000001 2020-08-07T15:06:58.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:58.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.697920+0000", 2020-08-07T15:06:58.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00044993600000000001 2020-08-07T15:06:58.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:58.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.698370+0000", 2020-08-07T15:06:58.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027536739999999998 2020-08-07T15:06:58.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:58.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.701124+0000", 2020-08-07T15:06:58.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999999e-07 2020-08-07T15:06:58.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:58.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.701124+0000", 2020-08-07T15:06:58.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7494000000000001e-05 2020-08-07T15:06:58.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:58.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.701152+0000", 2020-08-07T15:06:58.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086935569999999993 2020-08-07T15:06:58.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.709845+0000", 2020-08-07T15:06:58.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.225456669 2020-08-07T15:06:58.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.935302+0000", 2020-08-07T15:06:58.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.043e-06 2020-08-07T15:06:58.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:58.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.935303+0000", 2020-08-07T15:06:58.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9963999999999996e-05 2020-08-07T15:06:58.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:58.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.935373+0000", 2020-08-07T15:06:58.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24075575099999999 2020-08-07T15:06:58.089 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.089 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:58.089 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.090 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:2400 3.e e23/20 ondisk, result = 0)", 2020-08-07T15:06:58.090 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.702171+0000", 2020-08-07T15:06:58.090 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.351605834000001, 2020-08-07T15:06:58.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23320996599999999, 2020-08-07T15:06:58.090 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:58.091 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:58.091 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:58.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:58.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.702171+0000", 2020-08-07T15:06:58.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:58.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:58.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.702171+0000", 2020-08-07T15:06:58.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0556999999999999e-05 2020-08-07T15:06:58.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:58.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.702202+0000", 2020-08-07T15:06:58.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9409999999999998e-06 2020-08-07T15:06:58.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:58.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.702207+0000", 2020-08-07T15:06:58.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23266323 2020-08-07T15:06:58.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:58.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.934870+0000", 2020-08-07T15:06:58.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2899999999999998e-06 2020-08-07T15:06:58.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:58.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.934873+0000", 2020-08-07T15:06:58.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00041063500000000001 2020-08-07T15:06:58.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:58.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.935284+0000", 2020-08-07T15:06:58.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8419999999999994e-06 2020-08-07T15:06:58.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:58.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.935293+0000", 2020-08-07T15:06:58.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8471e-05 2020-08-07T15:06:58.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:58.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.935381+0000", 2020-08-07T15:06:58.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23320996599999999 2020-08-07T15:06:58.100 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.100 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:58.100 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.101 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:9352 3.e 3:7113579b:::benchmark_data_smithi074_35447_object1168:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:58.101 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.885925+0000", 2020-08-07T15:06:58.101 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.167852162999999, 2020-08-07T15:06:58.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24955413800000001, 2020-08-07T15:06:58.101 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:58.102 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:58.102 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:58.102 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:58.102 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:58.102 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 9352 2020-08-07T15:06:58.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.103 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:58.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:58.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.885925+0000", 2020-08-07T15:06:58.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:58.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:58.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.885925+0000", 2020-08-07T15:06:58.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4308999999999998e-05 2020-08-07T15:06:58.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:58.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.885979+0000", 2020-08-07T15:06:58.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0440000000000001e-06 2020-08-07T15:06:58.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:58.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.885985+0000", 2020-08-07T15:06:58.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9462000000000001e-05 2020-08-07T15:06:58.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:58.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.886005+0000", 2020-08-07T15:06:58.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2530000000000001e-06 2020-08-07T15:06:58.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:58.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.886011+0000", 2020-08-07T15:06:58.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025236080000000001 2020-08-07T15:06:58.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:58.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.888535+0000", 2020-08-07T15:06:58.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112871 2020-08-07T15:06:58.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:58.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.888648+0000", 2020-08-07T15:06:58.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020764399999999999 2020-08-07T15:06:58.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:58.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.888855+0000", 2020-08-07T15:06:58.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048523799999999999 2020-08-07T15:06:58.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.893708+0000", 2020-08-07T15:06:58.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030190300000000001 2020-08-07T15:06:58.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:58.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.894010+0000", 2020-08-07T15:06:58.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24135353300000001 2020-08-07T15:06:58.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.135363+0000", 2020-08-07T15:06:58.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9899999999999999e-07 2020-08-07T15:06:58.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:58.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.135364+0000", 2020-08-07T15:06:58.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7601999999999994e-05 2020-08-07T15:06:58.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:58.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.135431+0000", 2020-08-07T15:06:58.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7729999999999999e-05 2020-08-07T15:06:58.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:58.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.135479+0000", 2020-08-07T15:06:58.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24955413800000001 2020-08-07T15:06:58.117 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.118 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:58.118 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.118 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:9976 3.e 3:7fb2e2e5:::benchmark_data_smithi074_35447_object1246:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:58.119 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:06:58.119 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.816579276000001, 2020-08-07T15:06:58.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28420182999999999, 2020-08-07T15:06:58.119 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:58.119 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:58.119 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:58.120 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:58.120 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:58.120 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 9976 2020-08-07T15:06:58.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.120 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:58.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:58.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:06:58.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:58.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:58.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:06:58.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7113e-05 2020-08-07T15:06:58.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:58.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237255+0000", 2020-08-07T15:06:58.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5869999999999998e-06 2020-08-07T15:06:58.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:58.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237262+0000", 2020-08-07T15:06:58.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4467999999999999e-05 2020-08-07T15:06:58.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:58.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237276+0000", 2020-08-07T15:06:58.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0749999999999997e-06 2020-08-07T15:06:58.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:58.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237281+0000", 2020-08-07T15:06:58.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013795109999999999 2020-08-07T15:06:58.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:58.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238661+0000", 2020-08-07T15:06:58.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100309 2020-08-07T15:06:58.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:58.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238761+0000", 2020-08-07T15:06:58.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000141784 2020-08-07T15:06:58.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:58.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238903+0000", 2020-08-07T15:06:58.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00070626599999999999 2020-08-07T15:06:58.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:58.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239609+0000", 2020-08-07T15:06:58.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3499999999999996e-07 2020-08-07T15:06:58.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:58.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239610+0000", 2020-08-07T15:06:58.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0846000000000006e-05 2020-08-07T15:06:58.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:58.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239680+0000", 2020-08-07T15:06:58.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022037080000000001 2020-08-07T15:06:58.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.261718+0000", 2020-08-07T15:06:58.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25961784500000001 2020-08-07T15:06:58.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521335+0000", 2020-08-07T15:06:58.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2839999999999999e-06 2020-08-07T15:06:58.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:58.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521337+0000", 2020-08-07T15:06:58.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3126999999999994e-05 2020-08-07T15:06:58.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:58.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521400+0000", 2020-08-07T15:06:58.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28420182999999999 2020-08-07T15:06:58.139 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.139 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:58.139 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.140 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:9976 3.e e24/20 ondisk, result = 0)", 2020-08-07T15:06:58.140 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.269400+0000", 2020-08-07T15:06:58.140 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.784377052, 2020-08-07T15:06:58.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25200699900000001, 2020-08-07T15:06:58.141 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:58.141 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:58.141 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:58.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.269400+0000", 2020-08-07T15:06:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.269400+0000", 2020-08-07T15:06:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1279999999999999e-05 2020-08-07T15:06:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.269431+0000", 2020-08-07T15:06:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.071e-06 2020-08-07T15:06:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.269438+0000", 2020-08-07T15:06:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24671454900000001 2020-08-07T15:06:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.516152+0000", 2020-08-07T15:06:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4140000000000002e-06 2020-08-07T15:06:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.516157+0000", 2020-08-07T15:06:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051603170000000002 2020-08-07T15:06:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521318+0000", 2020-08-07T15:06:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7479999999999999e-06 2020-08-07T15:06:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521327+0000", 2020-08-07T15:06:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0619999999999997e-05 2020-08-07T15:06:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:58.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521407+0000", 2020-08-07T15:06:58.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25200699900000001 2020-08-07T15:06:58.152 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.152 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:58.154 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.155 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15096 3.d 3:b3ff3030:::benchmark_data_smithi074_35447_object1886:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:06:58.155 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.412912+0000", 2020-08-07T15:06:58.155 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.640865441000001, 2020-08-07T15:06:58.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24065245299999999, 2020-08-07T15:06:58.156 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:58.156 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:58.156 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:58.156 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:58.156 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:58.157 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15096 2020-08-07T15:06:58.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.157 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:58.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:58.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.412912+0000", 2020-08-07T15:06:58.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:58.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:58.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.412912+0000", 2020-08-07T15:06:58.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2509999999999999e-05 2020-08-07T15:06:58.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:58.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.412944+0000", 2020-08-07T15:06:58.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3500000000000001e-06 2020-08-07T15:06:58.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:58.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.412948+0000", 2020-08-07T15:06:58.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1580000000000002e-06 2020-08-07T15:06:58.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:58.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.412954+0000", 2020-08-07T15:06:58.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1219999999999999e-06 2020-08-07T15:06:58.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:58.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.412957+0000", 2020-08-07T15:06:58.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034271409999999999 2020-08-07T15:06:58.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:58.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.416384+0000", 2020-08-07T15:06:58.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3467999999999995e-05 2020-08-07T15:06:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.416458+0000", 2020-08-07T15:06:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015322299999999999 2020-08-07T15:06:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:06:58.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.416611+0000", 2020-08-07T15:06:58.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005322027 2020-08-07T15:06:58.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:58.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.421933+0000", 2020-08-07T15:06:58.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.31e-06 2020-08-07T15:06:58.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:58.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.421934+0000", 2020-08-07T15:06:58.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5050999999999996e-05 2020-08-07T15:06:58.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:58.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.422019+0000", 2020-08-07T15:06:58.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028529359999999999 2020-08-07T15:06:58.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.424872+0000", 2020-08-07T15:06:58.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22862891099999999 2020-08-07T15:06:58.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.653501+0000", 2020-08-07T15:06:58.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2020-08-07T15:06:58.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:58.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.653502+0000", 2020-08-07T15:06:58.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2333000000000001e-05 2020-08-07T15:06:58.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:58.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.653564+0000", 2020-08-07T15:06:58.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24065245299999999 2020-08-07T15:06:58.172 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.173 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:58.173 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.173 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:15096 3.d e27/20 ondisk, result = 0)", 2020-08-07T15:06:58.174 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.422011+0000", 2020-08-07T15:06:58.174 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.631766504, 2020-08-07T15:06:58.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23155854300000001, 2020-08-07T15:06:58.174 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:58.175 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:58.175 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:58.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:58.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.422011+0000", 2020-08-07T15:06:58.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:58.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:58.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.422011+0000", 2020-08-07T15:06:58.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0474e-05 2020-08-07T15:06:58.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:58.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.422041+0000", 2020-08-07T15:06:58.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9069999999999998e-06 2020-08-07T15:06:58.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:58.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.422046+0000", 2020-08-07T15:06:58.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.231316472 2020-08-07T15:06:58.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:58.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.653363+0000", 2020-08-07T15:06:58.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2989999999999996e-06 2020-08-07T15:06:58.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:58.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.653368+0000", 2020-08-07T15:06:58.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116648 2020-08-07T15:06:58.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:58.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.653485+0000", 2020-08-07T15:06:58.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0010000000000001e-06 2020-08-07T15:06:58.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:58.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.653493+0000", 2020-08-07T15:06:58.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6742000000000005e-05 2020-08-07T15:06:58.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:58.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.653569+0000", 2020-08-07T15:06:58.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23155854300000001 2020-08-07T15:06:58.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.183 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:58.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.184 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15432 3.f 3:f247d22a:::benchmark_data_smithi074_35447_object1928:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:06:58.184 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.588833+0000", 2020-08-07T15:06:58.184 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.4649441, 2020-08-07T15:06:58.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22531295500000001, 2020-08-07T15:06:58.185 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:58.185 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:58.185 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:58.187 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:58.188 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:58.188 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15432 2020-08-07T15:06:58.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.188 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:58.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:58.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.588833+0000", 2020-08-07T15:06:58.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:58.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:58.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.588833+0000", 2020-08-07T15:06:58.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9542999999999997e-05 2020-08-07T15:06:58.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:58.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.588873+0000", 2020-08-07T15:06:58.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8249999999999998e-06 2020-08-07T15:06:58.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:58.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.588876+0000", 2020-08-07T15:06:58.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5490000000000006e-06 2020-08-07T15:06:58.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:58.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.588885+0000", 2020-08-07T15:06:58.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1389999999999997e-06 2020-08-07T15:06:58.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:58.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.588889+0000", 2020-08-07T15:06:58.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001004575 2020-08-07T15:06:58.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:58.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.589894+0000", 2020-08-07T15:06:58.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6419e-05 2020-08-07T15:06:58.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:58.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.589990+0000", 2020-08-07T15:06:58.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020380600000000001 2020-08-07T15:06:58.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:58.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.590194+0000", 2020-08-07T15:06:58.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00092655899999999998 2020-08-07T15:06:58.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:58.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.591121+0000", 2020-08-07T15:06:58.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5099999999999999e-07 2020-08-07T15:06:58.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:58.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.591121+0000", 2020-08-07T15:06:58.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0671000000000003e-05 2020-08-07T15:06:58.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:58.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.591172+0000", 2020-08-07T15:06:58.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033348620000000001 2020-08-07T15:06:58.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.594507+0000", 2020-08-07T15:06:58.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21957627299999999 2020-08-07T15:06:58.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.814083+0000", 2020-08-07T15:06:58.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7599999999999996e-07 2020-08-07T15:06:58.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:58.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.814084+0000", 2020-08-07T15:06:58.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2206999999999996e-05 2020-08-07T15:06:58.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:58.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.814146+0000", 2020-08-07T15:06:58.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22531295500000001 2020-08-07T15:06:58.213 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.213 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:58.213 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.214 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:15432 3.f e27/20 ondisk, result = 0)", 2020-08-07T15:06:58.214 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.594538+0000", 2020-08-07T15:06:58.214 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.459238792000001, 2020-08-07T15:06:58.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21961304500000001, 2020-08-07T15:06:58.214 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:58.215 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:58.215 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:58.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:58.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.594538+0000", 2020-08-07T15:06:58.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:58.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:58.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.594538+0000", 2020-08-07T15:06:58.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.525e-05 2020-08-07T15:06:58.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:58.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.594554+0000", 2020-08-07T15:06:58.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9620000000000004e-06 2020-08-07T15:06:58.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:58.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.594558+0000", 2020-08-07T15:06:58.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21915430599999999 2020-08-07T15:06:58.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:58.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.813712+0000", 2020-08-07T15:06:58.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5459999999999999e-06 2020-08-07T15:06:58.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:58.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.813717+0000", 2020-08-07T15:06:58.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034428300000000002 2020-08-07T15:06:58.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:58.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.814062+0000", 2020-08-07T15:06:58.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1030000000000002e-06 2020-08-07T15:06:58.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:58.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.814070+0000", 2020-08-07T15:06:58.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1594999999999994e-05 2020-08-07T15:06:58.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:58.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.814151+0000", 2020-08-07T15:06:58.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21961304500000001 2020-08-07T15:06:58.226 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.226 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:58.226 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.227 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15576 3.e 3:727724a5:::benchmark_data_smithi074_35447_object1946:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:06:58.227 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:06:58.228 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.387142313, 2020-08-07T15:06:58.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999, 2020-08-07T15:06:58.228 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:58.228 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:58.228 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:58.229 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:58.229 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:58.229 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15576 2020-08-07T15:06:58.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.229 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:58.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:58.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:06:58.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:58.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:58.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:06:58.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7156e-05 2020-08-07T15:06:58.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:58.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666682+0000", 2020-08-07T15:06:58.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0209999999999999e-06 2020-08-07T15:06:58.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:58.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666687+0000", 2020-08-07T15:06:58.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0730999999999999e-05 2020-08-07T15:06:58.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:58.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666698+0000", 2020-08-07T15:06:58.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2020-08-07T15:06:58.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:58.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666702+0000", 2020-08-07T15:06:58.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023978459999999999 2020-08-07T15:06:58.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:58.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669100+0000", 2020-08-07T15:06:58.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9664000000000001e-05 2020-08-07T15:06:58.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:58.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669190+0000", 2020-08-07T15:06:58.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015957100000000001 2020-08-07T15:06:58.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:58.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669350+0000", 2020-08-07T15:06:58.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016470149999999999 2020-08-07T15:06:58.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:58.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:06:58.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1299999999999999e-07 2020-08-07T15:06:58.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:58.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:06:58.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5269000000000002e-05 2020-08-07T15:06:58.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:58.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.671022+0000", 2020-08-07T15:06:58.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031414400000000001 2020-08-07T15:06:58.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.674163+0000", 2020-08-07T15:06:58.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45641417400000001 2020-08-07T15:06:58.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:06:58.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0399999999999995e-07 2020-08-07T15:06:58.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:58.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:06:58.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3343999999999997e-05 2020-08-07T15:06:58.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:58.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130622+0000", 2020-08-07T15:06:58.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999 2020-08-07T15:06:58.246 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.247 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:58.247 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.247 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15743 3.d 3:b5661143:::benchmark_data_smithi074_35447_object1967:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:06:58.247 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.768880+0000", 2020-08-07T15:06:58.248 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.284896902, 2020-08-07T15:06:58.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24181233499999999, 2020-08-07T15:06:58.248 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:58.248 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:58.248 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:58.249 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:58.249 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:58.249 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15743 2020-08-07T15:06:58.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.249 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:58.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:58.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.768880+0000", 2020-08-07T15:06:58.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:58.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:58.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.768880+0000", 2020-08-07T15:06:58.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8946000000000002e-05 2020-08-07T15:06:58.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:58.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.768929+0000", 2020-08-07T15:06:58.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9020000000000004e-06 2020-08-07T15:06:58.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:58.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.768935+0000", 2020-08-07T15:06:58.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2283000000000001e-05 2020-08-07T15:06:58.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:58.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.768947+0000", 2020-08-07T15:06:58.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.186e-06 2020-08-07T15:06:58.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:58.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.768953+0000", 2020-08-07T15:06:58.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0068896529999999999 2020-08-07T15:06:58.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:58.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.775842+0000", 2020-08-07T15:06:58.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0270000000000001e-05 2020-08-07T15:06:58.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:58.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.775933+0000", 2020-08-07T15:06:58.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017233799999999999 2020-08-07T15:06:58.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:06:58.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.776105+0000", 2020-08-07T15:06:58.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00117592 2020-08-07T15:06:58.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:58.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777281+0000", 2020-08-07T15:06:58.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6499999999999998e-07 2020-08-07T15:06:58.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:58.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777282+0000", 2020-08-07T15:06:58.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3357999999999999e-05 2020-08-07T15:06:58.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:58.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777345+0000", 2020-08-07T15:06:58.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030411169999999999 2020-08-07T15:06:58.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.780386+0000", 2020-08-07T15:06:58.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.230239216 2020-08-07T15:06:58.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.010625+0000", 2020-08-07T15:06:58.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5600000000000005e-07 2020-08-07T15:06:58.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:58.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.010626+0000", 2020-08-07T15:06:58.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6625000000000007e-05 2020-08-07T15:06:58.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:58.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.010693+0000", 2020-08-07T15:06:58.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24181233499999999 2020-08-07T15:06:58.266 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.267 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:58.267 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.267 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15744 3.d 3:b5661143:::benchmark_data_smithi074_35447_object1967:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:06:58.268 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.769136+0000", 2020-08-07T15:06:58.268 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.284641093999999, 2020-08-07T15:06:58.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24180031099999999, 2020-08-07T15:06:58.268 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:58.268 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:58.269 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:58.269 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:58.269 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:58.269 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15744 2020-08-07T15:06:58.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.269 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:58.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:58.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.769136+0000", 2020-08-07T15:06:58.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:58.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:58.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.769136+0000", 2020-08-07T15:06:58.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5503999999999997e-05 2020-08-07T15:06:58.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:58.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.769182+0000", 2020-08-07T15:06:58.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.664e-06 2020-08-07T15:06:58.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:58.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.769185+0000", 2020-08-07T15:06:58.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7200000000000006e-06 2020-08-07T15:06:58.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:58.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.769193+0000", 2020-08-07T15:06:58.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8260000000000003e-06 2020-08-07T15:06:58.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:58.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.769197+0000", 2020-08-07T15:06:58.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0074215849999999996 2020-08-07T15:06:58.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:58.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.776618+0000", 2020-08-07T15:06:58.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010513700000000001 2020-08-07T15:06:58.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:58.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.776724+0000", 2020-08-07T15:06:58.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016319800000000001 2020-08-07T15:06:58.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:06:58.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.776887+0000", 2020-08-07T15:06:58.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00090081899999999997 2020-08-07T15:06:58.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:58.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777788+0000", 2020-08-07T15:06:58.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5000000000000001e-07 2020-08-07T15:06:58.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:58.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777788+0000", 2020-08-07T15:06:58.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5197000000000006e-05 2020-08-07T15:06:58.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:58.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777854+0000", 2020-08-07T15:06:58.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22872283800000001 2020-08-07T15:06:58.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.006576+0000", 2020-08-07T15:06:58.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004298367 2020-08-07T15:06:58.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.010875+0000", 2020-08-07T15:06:58.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.08e-07 2020-08-07T15:06:58.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:58.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.010875+0000", 2020-08-07T15:06:58.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1198000000000006e-05 2020-08-07T15:06:58.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:58.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.010936+0000", 2020-08-07T15:06:58.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24180031099999999 2020-08-07T15:06:58.286 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.287 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:58.287 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.287 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:15744 3.d e27/20 ondisk, result = 0)", 2020-08-07T15:06:58.288 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.780273+0000", 2020-08-07T15:06:58.288 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.273504437, 2020-08-07T15:06:58.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22630540900000001, 2020-08-07T15:06:58.288 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:58.289 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:58.289 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:58.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:58.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.780273+0000", 2020-08-07T15:06:58.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:58.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:58.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.780273+0000", 2020-08-07T15:06:58.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5190000000000001e-05 2020-08-07T15:06:58.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:58.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.780298+0000", 2020-08-07T15:06:58.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1680000000000003e-06 2020-08-07T15:06:58.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:58.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.780303+0000", 2020-08-07T15:06:58.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22622377299999999 2020-08-07T15:06:58.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:58.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.006527+0000", 2020-08-07T15:06:58.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9520000000000002e-06 2020-08-07T15:06:58.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:58.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.006532+0000", 2020-08-07T15:06:58.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8175999999999999e-05 2020-08-07T15:06:58.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:58.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.006560+0000", 2020-08-07T15:06:58.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2040000000000003e-06 2020-08-07T15:06:58.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:58.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.006569+0000", 2020-08-07T15:06:58.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9460000000000004e-06 2020-08-07T15:06:58.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:58.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.006578+0000", 2020-08-07T15:06:58.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22630540900000001 2020-08-07T15:06:58.298 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.298 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:58.298 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.299 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:15743 3.d e27/20 ondisk, result = 0)", 2020-08-07T15:06:58.299 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.781802+0000", 2020-08-07T15:06:58.299 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.271975048, 2020-08-07T15:06:58.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22903343900000001, 2020-08-07T15:06:58.300 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:58.300 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:06:58.300 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:58.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:58.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.781802+0000", 2020-08-07T15:06:58.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:58.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:58.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.781802+0000", 2020-08-07T15:06:58.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7727000000000002e-05 2020-08-07T15:06:58.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:58.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.781820+0000", 2020-08-07T15:06:58.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6880000000000001e-06 2020-08-07T15:06:58.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:58.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.781824+0000", 2020-08-07T15:06:58.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22869176799999999 2020-08-07T15:06:58.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:58.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.010515+0000", 2020-08-07T15:06:58.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1220000000000003e-06 2020-08-07T15:06:58.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:58.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.010520+0000", 2020-08-07T15:06:58.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0396000000000007e-05 2020-08-07T15:06:58.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:58.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.010611+0000", 2020-08-07T15:06:58.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5660000000000003e-06 2020-08-07T15:06:58.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:58.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.010617+0000", 2020-08-07T15:06:58.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021817200000000001 2020-08-07T15:06:58.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:58.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.010836+0000", 2020-08-07T15:06:58.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22903343900000001 2020-08-07T15:06:58.309 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.309 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:58.310 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.310 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:16518 3.e 3:7f0d79c2:::benchmark_data_smithi074_35447_object2064:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e27)", 2020-08-07T15:06:58.310 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:47.307446+0000", 2020-08-07T15:06:58.311 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.746330873, 2020-08-07T15:06:58.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21888107000000001, 2020-08-07T15:06:58.311 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:58.311 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:58.311 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:58.311 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:58.312 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:58.312 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 16518 2020-08-07T15:06:58.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.312 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:58.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:58.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.307446+0000", 2020-08-07T15:06:58.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:58.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:58.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.307446+0000", 2020-08-07T15:06:58.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0618000000000002e-05 2020-08-07T15:06:58.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:58.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.307497+0000", 2020-08-07T15:06:58.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6590000000000004e-06 2020-08-07T15:06:58.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:58.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.307503+0000", 2020-08-07T15:06:58.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1430000000000001e-05 2020-08-07T15:06:58.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:58.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.307514+0000", 2020-08-07T15:06:58.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5730000000000001e-06 2020-08-07T15:06:58.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:58.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.307519+0000", 2020-08-07T15:06:58.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1866000000000001e-05 2020-08-07T15:06:58.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:58.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.307570+0000", 2020-08-07T15:06:58.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7078999999999998e-05 2020-08-07T15:06:58.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:58.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.307658+0000", 2020-08-07T15:06:58.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000177431 2020-08-07T15:06:58.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:58.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.307835+0000", 2020-08-07T15:06:58.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001066502 2020-08-07T15:06:58.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:58.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.308902+0000", 2020-08-07T15:06:58.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0200000000000001e-07 2020-08-07T15:06:58.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:58.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.308902+0000", 2020-08-07T15:06:58.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7828000000000003e-05 2020-08-07T15:06:58.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:58.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.308960+0000", 2020-08-07T15:06:58.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018485559999999999 2020-08-07T15:06:58.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.310809+0000", 2020-08-07T15:06:58.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.215444408 2020-08-07T15:06:58.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.526253+0000", 2020-08-07T15:06:58.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2500000000000001e-06 2020-08-07T15:06:58.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:58.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.526254+0000", 2020-08-07T15:06:58.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3168000000000001e-05 2020-08-07T15:06:58.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:58.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.526327+0000", 2020-08-07T15:06:58.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21888107000000001 2020-08-07T15:06:58.330 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.330 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:58.330 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.331 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:16519 3.e 3:7f0d79c2:::benchmark_data_smithi074_35447_object2064:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e27)", 2020-08-07T15:06:58.331 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:47.307555+0000", 2020-08-07T15:06:58.331 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.746222051, 2020-08-07T15:06:58.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21888080700000001, 2020-08-07T15:06:58.331 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:58.332 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:58.332 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:58.332 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:58.332 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:58.332 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 16519 2020-08-07T15:06:58.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.333 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:58.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:58.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.307555+0000", 2020-08-07T15:06:58.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:58.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:58.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.307555+0000", 2020-08-07T15:06:58.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9276000000000001e-05 2020-08-07T15:06:58.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:58.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.307604+0000", 2020-08-07T15:06:58.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1220000000000003e-06 2020-08-07T15:06:58.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:58.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.307610+0000", 2020-08-07T15:06:58.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030283799999999999 2020-08-07T15:06:58.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:58.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.307912+0000", 2020-08-07T15:06:58.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2419999999999997e-06 2020-08-07T15:06:58.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:58.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.307919+0000", 2020-08-07T15:06:58.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032277299999999999 2020-08-07T15:06:58.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:58.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.308241+0000", 2020-08-07T15:06:58.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5723000000000006e-05 2020-08-07T15:06:58.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:58.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.308327+0000", 2020-08-07T15:06:58.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000186424 2020-08-07T15:06:58.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:58.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.308514+0000", 2020-08-07T15:06:58.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001281101 2020-08-07T15:06:58.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:58.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.309795+0000", 2020-08-07T15:06:58.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3600000000000002e-07 2020-08-07T15:06:58.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:58.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.309796+0000", 2020-08-07T15:06:58.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2737999999999994e-05 2020-08-07T15:06:58.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:58.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.309868+0000", 2020-08-07T15:06:58.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013347529999999999 2020-08-07T15:06:58.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.311203+0000", 2020-08-07T15:06:58.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21516143200000001 2020-08-07T15:06:58.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.526365+0000", 2020-08-07T15:06:58.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8899999999999998e-07 2020-08-07T15:06:58.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:58.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.526365+0000", 2020-08-07T15:06:58.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0560000000000002e-05 2020-08-07T15:06:58.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:58.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.526436+0000", 2020-08-07T15:06:58.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21888080700000001 2020-08-07T15:06:58.360 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.360 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:58.361 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.361 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:16520 3.e 3:7f0d79c2:::benchmark_data_smithi074_35447_object2064:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:06:58.361 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:47.308351+0000", 2020-08-07T15:06:58.362 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.745426063, 2020-08-07T15:06:58.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21818120199999999, 2020-08-07T15:06:58.362 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:58.362 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:58.362 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:58.363 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:58.363 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:58.363 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 16520 2020-08-07T15:06:58.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.364 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:58.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:58.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.308351+0000", 2020-08-07T15:06:58.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:58.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:58.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.308351+0000", 2020-08-07T15:06:58.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2292e-05 2020-08-07T15:06:58.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:58.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.308403+0000", 2020-08-07T15:06:58.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7599999999999999e-06 2020-08-07T15:06:58.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:58.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.308409+0000", 2020-08-07T15:06:58.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1535999999999999e-05 2020-08-07T15:06:58.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:58.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.308421+0000", 2020-08-07T15:06:58.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3630000000000002e-06 2020-08-07T15:06:58.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:58.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.308426+0000", 2020-08-07T15:06:58.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00055786599999999996 2020-08-07T15:06:58.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:58.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.308984+0000", 2020-08-07T15:06:58.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8918000000000006e-05 2020-08-07T15:06:58.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:58.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.309063+0000", 2020-08-07T15:06:58.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014475999999999999 2020-08-07T15:06:58.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:58.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.309208+0000", 2020-08-07T15:06:58.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010691015 2020-08-07T15:06:58.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:58.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.319899+0000", 2020-08-07T15:06:58.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.105e-06 2020-08-07T15:06:58.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:58.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.319900+0000", 2020-08-07T15:06:58.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9193000000000001e-05 2020-08-07T15:06:58.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:58.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.319959+0000", 2020-08-07T15:06:58.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.113250298 2020-08-07T15:06:58.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.433209+0000", 2020-08-07T15:06:58.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.093268289000000004 2020-08-07T15:06:58.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.526478+0000", 2020-08-07T15:06:58.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5899999999999996e-07 2020-08-07T15:06:58.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:58.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.526478+0000", 2020-08-07T15:06:58.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4147999999999997e-05 2020-08-07T15:06:58.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:58.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.526532+0000", 2020-08-07T15:06:58.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21818120199999999 2020-08-07T15:06:58.382 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.382 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:58.382 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.383 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:16531 3.f 3:fe7e2e24:::benchmark_data_smithi074_35447_object2066:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:06:58.383 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:47.310352+0000", 2020-08-07T15:06:58.383 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.743425144, 2020-08-07T15:06:58.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21882831599999999, 2020-08-07T15:06:58.383 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:58.383 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:58.384 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:58.384 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:58.384 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:58.386 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 16531 2020-08-07T15:06:58.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.386 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:58.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:58.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.310352+0000", 2020-08-07T15:06:58.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:58.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:58.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.310352+0000", 2020-08-07T15:06:58.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8576999999999999e-05 2020-08-07T15:06:58.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:58.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.310401+0000", 2020-08-07T15:06:58.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.87e-06 2020-08-07T15:06:58.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:58.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.310406+0000", 2020-08-07T15:06:58.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7220000000000002e-06 2020-08-07T15:06:58.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:58.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.310412+0000", 2020-08-07T15:06:58.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0319999999999999e-06 2020-08-07T15:06:58.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:58.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.310415+0000", 2020-08-07T15:06:58.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011664589999999999 2020-08-07T15:06:58.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:58.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.311582+0000", 2020-08-07T15:06:58.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4077000000000001e-05 2020-08-07T15:06:58.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:58.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.311676+0000", 2020-08-07T15:06:58.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016982999999999999 2020-08-07T15:06:58.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:58.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.311846+0000", 2020-08-07T15:06:58.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010766339999999999 2020-08-07T15:06:58.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:58.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.322612+0000", 2020-08-07T15:06:58.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2900000000000003e-07 2020-08-07T15:06:58.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:58.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.322613+0000", 2020-08-07T15:06:58.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1699000000000001e-05 2020-08-07T15:06:58.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:58.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.322664+0000", 2020-08-07T15:06:58.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2978000000000003e-05 2020-08-07T15:06:58.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.322757+0000", 2020-08-07T15:06:58.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20636275600000001 2020-08-07T15:06:58.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.529120+0000", 2020-08-07T15:06:58.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7700000000000004e-07 2020-08-07T15:06:58.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:58.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.529121+0000", 2020-08-07T15:06:58.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9570000000000001e-05 2020-08-07T15:06:58.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:58.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.529180+0000", 2020-08-07T15:06:58.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21882831599999999 2020-08-07T15:06:58.416 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.416 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:58.416 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.417 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:16532 3.f 3:fe7e2e24:::benchmark_data_smithi074_35447_object2066:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:06:58.417 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:47.310646+0000", 2020-08-07T15:06:58.417 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.743131672000001, 2020-08-07T15:06:58.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.220340023, 2020-08-07T15:06:58.417 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:58.418 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:58.418 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:58.418 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:58.418 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:58.418 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 16532 2020-08-07T15:06:58.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.419 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:58.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:58.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.310646+0000", 2020-08-07T15:06:58.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:58.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:58.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.310646+0000", 2020-08-07T15:06:58.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6500999999999999e-05 2020-08-07T15:06:58.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:58.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.310672+0000", 2020-08-07T15:06:58.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5360000000000001e-06 2020-08-07T15:06:58.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:58.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.310676+0000", 2020-08-07T15:06:58.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7850000000000003e-06 2020-08-07T15:06:58.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:58.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.310682+0000", 2020-08-07T15:06:58.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3170000000000008e-06 2020-08-07T15:06:58.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:58.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.310692+0000", 2020-08-07T15:06:58.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035828100000000001 2020-08-07T15:06:58.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:58.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.314274+0000", 2020-08-07T15:06:58.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110301 2020-08-07T15:06:58.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:58.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.314385+0000", 2020-08-07T15:06:58.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015081400000000001 2020-08-07T15:06:58.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:58.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.314536+0000", 2020-08-07T15:06:58.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081383930000000007 2020-08-07T15:06:58.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:58.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.322674+0000", 2020-08-07T15:06:58.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.75e-07 2020-08-07T15:06:58.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:58.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.322675+0000", 2020-08-07T15:06:58.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0794000000000002e-05 2020-08-07T15:06:58.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:58.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.322725+0000", 2020-08-07T15:06:58.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013822199999999999 2020-08-07T15:06:58.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.322864+0000", 2020-08-07T15:06:58.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20805211000000001 2020-08-07T15:06:58.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.530916+0000", 2020-08-07T15:06:58.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1600000000000001e-07 2020-08-07T15:06:58.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:58.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.530916+0000", 2020-08-07T15:06:58.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9148999999999998e-05 2020-08-07T15:06:58.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:58.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.530986+0000", 2020-08-07T15:06:58.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.220340023 2020-08-07T15:06:58.435 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.435 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:58.435 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.436 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:17824 3.d 3:bd4e0935:::benchmark_data_smithi074_35447_object2227:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:06:58.436 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.140708+0000", 2020-08-07T15:06:58.436 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.9130693900000004, 2020-08-07T15:06:58.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25688457100000001, 2020-08-07T15:06:58.437 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:06:58.437 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:58.437 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:06:58.437 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:06:58.437 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:58.438 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 17824 2020-08-07T15:06:58.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.438 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:06:58.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:06:58.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140708+0000", 2020-08-07T15:06:58.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:06:58.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:06:58.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140708+0000", 2020-08-07T15:06:58.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0522000000000002e-05 2020-08-07T15:06:58.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:06:58.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140758+0000", 2020-08-07T15:06:58.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1680000000000001e-06 2020-08-07T15:06:58.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:06:58.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140765+0000", 2020-08-07T15:06:58.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5791999999999998e-05 2020-08-07T15:06:58.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:06:58.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140810+0000", 2020-08-07T15:06:58.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2389999999999996e-06 2020-08-07T15:06:58.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:06:58.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140818+0000", 2020-08-07T15:06:58.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001721544 2020-08-07T15:06:58.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:06:58.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.142539+0000", 2020-08-07T15:06:58.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020908100000000001 2020-08-07T15:06:58.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:06:58.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.142748+0000", 2020-08-07T15:06:58.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019000000000000001 2020-08-07T15:06:58.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:06:58.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.142938+0000", 2020-08-07T15:06:58.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001381279 2020-08-07T15:06:58.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.144319+0000", 2020-08-07T15:06:58.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010975570000000001 2020-08-07T15:06:58.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:06:58.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.145417+0000", 2020-08-07T15:06:58.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.252064128 2020-08-07T15:06:58.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:58.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397481+0000", 2020-08-07T15:06:58.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003e-06 2020-08-07T15:06:58.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:06:58.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397482+0000", 2020-08-07T15:06:58.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3081000000000001e-05 2020-08-07T15:06:58.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:06:58.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397545+0000", 2020-08-07T15:06:58.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7176999999999999e-05 2020-08-07T15:06:58.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:06:58.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:06:58.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:06:58.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397592+0000", 2020-08-07T15:06:58.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25688457100000001 2020-08-07T15:06:58.453 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.453 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:58.453 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.454 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:06:58.454 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:06:58.454 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:06:58.455 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:06:58.455 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:06:58.455 INFO:tasks.thrashosds.thrasher:inject_pause on 5 2020-08-07T15:06:58.456 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2020-08-07T15:06:58.456 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2020-08-07T15:06:58.456 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2020-08-07T15:06:58.457 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:06:58.459 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 22 16 33884 33868 12.026 13.4531 0.00282634 0.0103826 2020-08-07T15:06:58.460 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:58.454+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:58.461 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:06:58.556 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:06:58.555+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:06:58.638 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:06:58.638 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:06:58.638 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:58.638 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:34435 3.1 e30/26 3:8692a28d:::benchmark_data_smithi074_35447_object4304:head v 30'1995, mlcod=30'1990)", 2020-08-07T15:06:58.639 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:58.636642+0000", 2020-08-07T15:06:58.639 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.00062923799999999995, 2020-08-07T15:06:58.639 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00065819000000000001, 2020-08-07T15:06:58.639 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:58.639 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:06:58.640 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:58.640 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:58.640 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:58.640 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.636642+0000", 2020-08-07T15:06:58.640 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:58.641 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:58.641 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:58.641 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:58.641 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.636642+0000", 2020-08-07T15:06:58.641 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2963000000000001e-05 2020-08-07T15:06:58.642 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:58.642 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:58.642 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:58.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.636695+0000", 2020-08-07T15:06:58.642 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3430000000000001e-06 2020-08-07T15:06:58.642 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:58.643 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:58.643 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:58.643 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.636701+0000", 2020-08-07T15:06:58.643 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6767000000000006e-05 2020-08-07T15:06:58.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:58.644 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:58.644 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:58.644 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.636768+0000", 2020-08-07T15:06:58.644 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.63e-06 2020-08-07T15:06:58.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:58.644 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:58.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:58.645 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.636772+0000", 2020-08-07T15:06:58.645 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7404999999999999e-05 2020-08-07T15:06:58.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:58.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:58.646 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:58.646 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.636799+0000", 2020-08-07T15:06:58.646 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9975000000000001e-05 2020-08-07T15:06:58.646 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:58.646 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:58.647 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:58.647 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.636819+0000", 2020-08-07T15:06:58.647 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000177083 2020-08-07T15:06:58.647 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:58.647 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:58.647 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:58.648 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:58.648 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:06:58.648 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 1 2020-08-07T15:06:58.648 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:06:58.652 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:06:58.669 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:06:58.671 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2020-08-07T15:06:58.671 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:06:58.700 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:06:58.695+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:06:58.758 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:58.756+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:06:58.843 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:06:58.843 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:06:58.843 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:06:58.844 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:06:58.844 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:06:58.856 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:06:58.898 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:58.896+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:58.961 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:58.959+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:59.062 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:59.060+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:59.090 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:06:59.090 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:06:59.090 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:06:59.091 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:06:59.091 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.091 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:7696 3.7 e24/20 3:ee5cfea7:::benchmark_data_smithi074_35447_object961:head v 24'504, mlcod=24'496)", 2020-08-07T15:06:59.091 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.807555+0000", 2020-08-07T15:06:59.091 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 18.267329196999999, 2020-08-07T15:06:59.092 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.124762208, 2020-08-07T15:06:59.092 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:59.092 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:59.092 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:59.092 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.093 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:59.093 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.807555+0000", 2020-08-07T15:06:59.093 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:59.093 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.093 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.094 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:59.094 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.807555+0000", 2020-08-07T15:06:59.094 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7098000000000003e-05 2020-08-07T15:06:59.094 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.094 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.095 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:59.095 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.807592+0000", 2020-08-07T15:06:59.095 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5120000000000002e-06 2020-08-07T15:06:59.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.095 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.096 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:59.096 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.807596+0000", 2020-08-07T15:06:59.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.12021193600000001 2020-08-07T15:06:59.096 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.096 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.096 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:59.097 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.927808+0000", 2020-08-07T15:06:59.097 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3749999999999999e-06 2020-08-07T15:06:59.097 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.097 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.097 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:59.098 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.927815+0000", 2020-08-07T15:06:59.098 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0791000000000002e-05 2020-08-07T15:06:59.098 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.098 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.098 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:59.098 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.927865+0000", 2020-08-07T15:06:59.099 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7851000000000002e-05 2020-08-07T15:06:59.099 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.099 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.099 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:59.099 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.927893+0000", 2020-08-07T15:06:59.100 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043673590000000003 2020-08-07T15:06:59.100 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.100 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.100 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:59.100 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.932261+0000", 2020-08-07T15:06:59.101 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6286000000000003e-05 2020-08-07T15:06:59.101 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.101 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.101 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:59.101 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.932317+0000", 2020-08-07T15:06:59.102 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.124762208 2020-08-07T15:06:59.102 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.102 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:59.102 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.102 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.103 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.103 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:9416 3.6 3:6d5d7913:::benchmark_data_smithi074_35447_object1176:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:59.103 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.922485+0000", 2020-08-07T15:06:59.103 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 17.152398673, 2020-08-07T15:06:59.103 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16975183499999999, 2020-08-07T15:06:59.104 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:59.104 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:59.104 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:59.104 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:59.104 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:59.105 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 9416 2020-08-07T15:06:59.105 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.105 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:59.105 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.105 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:59.105 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922485+0000", 2020-08-07T15:06:59.106 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:59.106 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.106 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.106 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:59.106 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922485+0000", 2020-08-07T15:06:59.107 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6072999999999998e-05 2020-08-07T15:06:59.107 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.107 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.107 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:59.107 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922521+0000", 2020-08-07T15:06:59.108 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.241e-06 2020-08-07T15:06:59.108 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.108 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.108 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:59.108 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922527+0000", 2020-08-07T15:06:59.108 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2075e-05 2020-08-07T15:06:59.109 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.109 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.109 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:59.109 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922540+0000", 2020-08-07T15:06:59.109 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0589999999999999e-06 2020-08-07T15:06:59.110 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.110 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.110 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:59.110 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922546+0000", 2020-08-07T15:06:59.110 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00063213199999999998 2020-08-07T15:06:59.111 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.111 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.111 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:59.111 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.923178+0000", 2020-08-07T15:06:59.111 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9913e-05 2020-08-07T15:06:59.111 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.112 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:59.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.923238+0000", 2020-08-07T15:06:59.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010683099999999999 2020-08-07T15:06:59.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.113 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:59.113 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.923344+0000", 2020-08-07T15:06:59.113 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004139663 2020-08-07T15:06:59.113 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.114 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:59.114 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.927484+0000", 2020-08-07T15:06:59.114 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001063149 2020-08-07T15:06:59.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.115 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:59.115 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.928547+0000", 2020-08-07T15:06:59.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16358086099999999 2020-08-07T15:06:59.115 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.115 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.116 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:59.116 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092128+0000", 2020-08-07T15:06:59.116 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3699999999999999e-07 2020-08-07T15:06:59.116 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.116 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:59.117 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092129+0000", 2020-08-07T15:06:59.117 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3859e-05 2020-08-07T15:06:59.117 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.117 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.118 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:59.118 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092193+0000", 2020-08-07T15:06:59.118 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4141999999999999e-05 2020-08-07T15:06:59.118 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.118 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.118 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:59.119 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092237+0000", 2020-08-07T15:06:59.119 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16975183499999999 2020-08-07T15:06:59.119 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.119 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:59.119 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.120 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.120 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.120 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:9416 3.6 e24/20 ondisk, result = 0)", 2020-08-07T15:06:59.120 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.926716+0000", 2020-08-07T15:06:59.120 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 17.14816828, 2020-08-07T15:06:59.121 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16552591899999999, 2020-08-07T15:06:59.121 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:59.121 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:06:59.121 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:59.121 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.122 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:59.122 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.926716+0000", 2020-08-07T15:06:59.122 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:59.122 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.122 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.123 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:59.123 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.926716+0000", 2020-08-07T15:06:59.123 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9971000000000002e-05 2020-08-07T15:06:59.123 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.123 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.123 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:59.124 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.926745+0000", 2020-08-07T15:06:59.124 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.51e-06 2020-08-07T15:06:59.124 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.124 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.124 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:59.125 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.926750+0000", 2020-08-07T15:06:59.125 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.165288254 2020-08-07T15:06:59.125 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.125 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.125 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:59.126 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092038+0000", 2020-08-07T15:06:59.126 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1109999999999999e-06 2020-08-07T15:06:59.126 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.126 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.126 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:59.127 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092043+0000", 2020-08-07T15:06:59.127 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7359000000000003e-05 2020-08-07T15:06:59.127 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.127 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.127 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:59.128 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092111+0000", 2020-08-07T15:06:59.128 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6889999999999993e-06 2020-08-07T15:06:59.128 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.128 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.128 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:59.128 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092119+0000", 2020-08-07T15:06:59.129 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000122025 2020-08-07T15:06:59.129 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.129 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.129 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:59.129 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092241+0000", 2020-08-07T15:06:59.130 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16552591899999999 2020-08-07T15:06:59.130 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.130 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:59.130 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.130 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.131 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.131 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:9687 3.6 3:667d8bb7:::benchmark_data_smithi074_35447_object1210:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:59.131 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.094514+0000", 2020-08-07T15:06:59.131 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.980369663000001, 2020-08-07T15:06:59.131 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.141938179, 2020-08-07T15:06:59.132 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:59.132 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:59.132 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:59.132 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:59.132 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:59.132 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 9687 2020-08-07T15:06:59.133 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.133 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:59.133 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.133 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:59.133 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094514+0000", 2020-08-07T15:06:59.134 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:59.134 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.134 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.134 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:59.134 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094514+0000", 2020-08-07T15:06:59.135 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2519000000000003e-05 2020-08-07T15:06:59.135 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.135 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.135 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:59.135 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094567+0000", 2020-08-07T15:06:59.136 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2570000000000004e-06 2020-08-07T15:06:59.136 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.136 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.136 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:59.136 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094573+0000", 2020-08-07T15:06:59.136 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1446e-05 2020-08-07T15:06:59.137 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.137 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.137 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:59.137 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094584+0000", 2020-08-07T15:06:59.137 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5439999999999998e-06 2020-08-07T15:06:59.138 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.138 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.138 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:59.138 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094590+0000", 2020-08-07T15:06:59.138 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00081871299999999999 2020-08-07T15:06:59.139 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.139 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.139 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:59.139 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.095409+0000", 2020-08-07T15:06:59.139 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2918999999999996e-05 2020-08-07T15:06:59.140 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.140 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.140 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:59.140 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.095502+0000", 2020-08-07T15:06:59.140 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017535200000000001 2020-08-07T15:06:59.140 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.141 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.141 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:59.141 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.095677+0000", 2020-08-07T15:06:59.141 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026867729999999999 2020-08-07T15:06:59.141 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.142 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.142 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:59.142 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.098364+0000", 2020-08-07T15:06:59.142 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8400000000000002e-07 2020-08-07T15:06:59.142 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.143 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.143 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:59.143 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.098365+0000", 2020-08-07T15:06:59.143 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4826999999999994e-05 2020-08-07T15:06:59.143 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.143 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.144 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:59.144 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.098429+0000", 2020-08-07T15:06:59.144 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001860022 2020-08-07T15:06:59.144 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.144 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.145 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:59.145 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.100289+0000", 2020-08-07T15:06:59.145 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.136088662 2020-08-07T15:06:59.145 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.145 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.145 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:59.146 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.236378+0000", 2020-08-07T15:06:59.146 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0559999999999999e-06 2020-08-07T15:06:59.146 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.146 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.146 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:59.147 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.236379+0000", 2020-08-07T15:06:59.147 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3104999999999999e-05 2020-08-07T15:06:59.147 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.147 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.147 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:59.148 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.236452+0000", 2020-08-07T15:06:59.148 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.141938179 2020-08-07T15:06:59.148 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.148 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:59.148 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.148 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.149 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.149 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:9688 3.6 3:667d8bb7:::benchmark_data_smithi074_35447_object1210:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:59.149 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.094625+0000", 2020-08-07T15:06:59.149 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.980259142000001, 2020-08-07T15:06:59.149 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14221277900000001, 2020-08-07T15:06:59.150 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:59.150 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:59.150 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:59.150 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:59.150 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:59.151 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 9688 2020-08-07T15:06:59.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.151 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:59.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.151 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:59.151 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094625+0000", 2020-08-07T15:06:59.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:59.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:59.152 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094625+0000", 2020-08-07T15:06:59.153 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4469000000000003e-05 2020-08-07T15:06:59.153 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.153 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.153 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:59.153 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094679+0000", 2020-08-07T15:06:59.154 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.201e-05 2020-08-07T15:06:59.154 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.154 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.154 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:59.154 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094691+0000", 2020-08-07T15:06:59.154 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0873000000000001e-05 2020-08-07T15:06:59.155 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.155 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.155 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:59.155 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094702+0000", 2020-08-07T15:06:59.155 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9599999999999999e-06 2020-08-07T15:06:59.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.156 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:59.156 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094707+0000", 2020-08-07T15:06:59.156 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017070119999999999 2020-08-07T15:06:59.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.157 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.157 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:59.157 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.096414+0000", 2020-08-07T15:06:59.157 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3101000000000002e-05 2020-08-07T15:06:59.157 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.158 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.158 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:59.158 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.096507+0000", 2020-08-07T15:06:59.158 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017744399999999999 2020-08-07T15:06:59.158 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.159 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.159 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:59.159 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.096685+0000", 2020-08-07T15:06:59.159 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017538250000000001 2020-08-07T15:06:59.159 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.159 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.160 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:59.160 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.098438+0000", 2020-08-07T15:06:59.160 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.82e-07 2020-08-07T15:06:59.160 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.160 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.161 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:59.161 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.098439+0000", 2020-08-07T15:06:59.161 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3258999999999999e-05 2020-08-07T15:06:59.161 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.161 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.162 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:59.162 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.098492+0000", 2020-08-07T15:06:59.162 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001822908 2020-08-07T15:06:59.162 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.162 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.162 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:59.163 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.100315+0000", 2020-08-07T15:06:59.163 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13645548299999999 2020-08-07T15:06:59.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.163 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.163 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:59.164 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.236771+0000", 2020-08-07T15:06:59.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7999999999999996e-07 2020-08-07T15:06:59.164 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:59.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.236771+0000", 2020-08-07T15:06:59.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6372999999999996e-05 2020-08-07T15:06:59.165 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.165 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.165 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:59.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.236837+0000", 2020-08-07T15:06:59.166 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14221277900000001 2020-08-07T15:06:59.166 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.166 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:59.166 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.166 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.167 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.167 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:9687 3.6 e24/20 ondisk, result = 0)", 2020-08-07T15:06:59.167 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.098678+0000", 2020-08-07T15:06:59.167 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.976205522000001, 2020-08-07T15:06:59.167 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13805130800000001, 2020-08-07T15:06:59.168 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:59.168 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:06:59.168 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:59.168 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.168 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:59.168 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.098678+0000", 2020-08-07T15:06:59.169 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:59.169 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.169 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.169 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:59.169 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.098678+0000", 2020-08-07T15:06:59.170 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1142000000000001e-05 2020-08-07T15:06:59.170 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.170 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.170 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:59.170 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.098709+0000", 2020-08-07T15:06:59.171 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1229999999999999e-06 2020-08-07T15:06:59.171 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.171 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.171 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:59.171 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.098715+0000", 2020-08-07T15:06:59.171 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13550553000000001 2020-08-07T15:06:59.172 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.172 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.172 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:59.172 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.234220+0000", 2020-08-07T15:06:59.172 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9459999999999997e-06 2020-08-07T15:06:59.173 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.173 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.173 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:59.173 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.234225+0000", 2020-08-07T15:06:59.173 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002135797 2020-08-07T15:06:59.174 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.174 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.174 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:59.174 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.236361+0000", 2020-08-07T15:06:59.174 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6309999999999992e-06 2020-08-07T15:06:59.174 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.175 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.175 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:59.175 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.236369+0000", 2020-08-07T15:06:59.175 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00036013899999999999 2020-08-07T15:06:59.175 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.176 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.176 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:59.176 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.236730+0000", 2020-08-07T15:06:59.176 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13805130800000001 2020-08-07T15:06:59.176 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.177 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:59.177 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.177 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.177 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.177 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:10240 3.6 3:665d71ff:::benchmark_data_smithi074_35447_object1279:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:06:59.177 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:06:59.178 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.681664843, 2020-08-07T15:06:59.178 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25191569699999999, 2020-08-07T15:06:59.178 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:59.178 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:59.178 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:59.179 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:59.179 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:59.179 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 10240 2020-08-07T15:06:59.179 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.179 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:59.180 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.180 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:59.180 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:06:59.180 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:59.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.180 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.181 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:59.181 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:06:59.181 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5463000000000002e-05 2020-08-07T15:06:59.181 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.181 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:59.182 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393264+0000", 2020-08-07T15:06:59.182 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9219999999999999e-06 2020-08-07T15:06:59.182 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.183 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:59.183 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393268+0000", 2020-08-07T15:06:59.183 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6149999999999997e-06 2020-08-07T15:06:59.183 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.183 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.183 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:59.184 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393277+0000", 2020-08-07T15:06:59.184 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2039999999999996e-06 2020-08-07T15:06:59.184 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.184 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.184 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:59.185 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393282+0000", 2020-08-07T15:06:59.185 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001225243 2020-08-07T15:06:59.185 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.185 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.185 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:59.185 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394507+0000", 2020-08-07T15:06:59.186 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7108999999999995e-05 2020-08-07T15:06:59.186 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.186 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.186 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:59.186 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394605+0000", 2020-08-07T15:06:59.187 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017490800000000001 2020-08-07T15:06:59.187 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.187 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.187 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:06:59.187 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394779+0000", 2020-08-07T15:06:59.188 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032608699999999999 2020-08-07T15:06:59.188 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.188 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.188 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:59.188 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.398040+0000", 2020-08-07T15:06:59.189 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013015259999999999 2020-08-07T15:06:59.189 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.189 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.189 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:59.189 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.399342+0000", 2020-08-07T15:06:59.190 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24566893400000001 2020-08-07T15:06:59.190 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.190 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.190 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:59.190 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645011+0000", 2020-08-07T15:06:59.190 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.217e-06 2020-08-07T15:06:59.191 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.191 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.191 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:06:59.191 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645012+0000", 2020-08-07T15:06:59.191 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0803000000000006e-05 2020-08-07T15:06:59.192 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.192 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.192 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:59.192 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645083+0000", 2020-08-07T15:06:59.192 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1882999999999998e-05 2020-08-07T15:06:59.192 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.193 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.193 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:59.193 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645135+0000", 2020-08-07T15:06:59.193 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25191569699999999 2020-08-07T15:06:59.193 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.194 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:59.194 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.194 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.194 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.194 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:16144 3.f e27/20 3:ff74134a:::benchmark_data_smithi074_35447_object2017:head v 27'1176, mlcod=27'1170)", 2020-08-07T15:06:59.195 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.055958+0000", 2020-08-07T15:06:59.195 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 12.018925739, 2020-08-07T15:06:59.195 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13659865900000001, 2020-08-07T15:06:59.195 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:59.195 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:59.196 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:59.196 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.196 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:59.196 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.055958+0000", 2020-08-07T15:06:59.196 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:59.196 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.197 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.197 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:59.197 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.055958+0000", 2020-08-07T15:06:59.197 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1301000000000002e-05 2020-08-07T15:06:59.197 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.198 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.198 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:59.198 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.056009+0000", 2020-08-07T15:06:59.198 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2360000000000001e-06 2020-08-07T15:06:59.198 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.199 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.199 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:59.199 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.056016+0000", 2020-08-07T15:06:59.199 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13378537900000001 2020-08-07T15:06:59.199 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.200 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.200 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:59.200 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.189801+0000", 2020-08-07T15:06:59.200 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5190000000000003e-06 2020-08-07T15:06:59.200 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.201 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.201 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:59.201 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.189808+0000", 2020-08-07T15:06:59.201 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0105999999999999e-05 2020-08-07T15:06:59.201 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.201 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.202 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:59.202 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.189849+0000", 2020-08-07T15:06:59.202 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5304e-05 2020-08-07T15:06:59.202 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.202 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.203 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:59.203 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.189874+0000", 2020-08-07T15:06:59.203 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026358559999999998 2020-08-07T15:06:59.203 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.203 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.204 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:59.204 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.192510+0000", 2020-08-07T15:06:59.204 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6958e-05 2020-08-07T15:06:59.204 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.204 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.204 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:59.205 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.192557+0000", 2020-08-07T15:06:59.205 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13659865900000001 2020-08-07T15:06:59.205 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.205 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:59.205 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.206 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.206 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.206 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:16640 3.6 e27/26 3:63b550b6:::benchmark_data_smithi074_35447_object2079:head v 27'1104, mlcod=27'1096)", 2020-08-07T15:06:59.206 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.390454+0000", 2020-08-07T15:06:59.206 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.684429359999999, 2020-08-07T15:06:59.207 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.144390454, 2020-08-07T15:06:59.207 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:59.212 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:59.212 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:59.212 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.212 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:59.212 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.390454+0000", 2020-08-07T15:06:59.213 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:59.213 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.213 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.213 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:59.213 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.390454+0000", 2020-08-07T15:06:59.214 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0198999999999998e-05 2020-08-07T15:06:59.214 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.214 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.214 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:59.214 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.390505+0000", 2020-08-07T15:06:59.215 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7270000000000001e-06 2020-08-07T15:06:59.215 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.215 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.215 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:59.215 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.390511+0000", 2020-08-07T15:06:59.215 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14174463700000001 2020-08-07T15:06:59.216 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.216 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.216 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:59.216 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.532256+0000", 2020-08-07T15:06:59.216 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0260000000000002e-06 2020-08-07T15:06:59.217 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.217 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.217 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:59.217 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.532263+0000", 2020-08-07T15:06:59.217 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6328000000000001e-05 2020-08-07T15:06:59.218 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.218 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.218 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:59.218 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.532309+0000", 2020-08-07T15:06:59.218 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4360000000000001e-05 2020-08-07T15:06:59.218 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.219 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.219 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:59.219 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.532334+0000", 2020-08-07T15:06:59.219 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024563200000000001 2020-08-07T15:06:59.219 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.220 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.220 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:59.220 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.534790+0000", 2020-08-07T15:06:59.220 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4857e-05 2020-08-07T15:06:59.220 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.221 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.221 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:59.221 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.534845+0000", 2020-08-07T15:06:59.221 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.144390454 2020-08-07T15:06:59.221 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.221 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:59.222 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.222 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.222 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.222 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:18264 3.b e27/20 3:d79881d2:::benchmark_data_smithi074_35447_object2282:head v 27'1224, mlcod=27'1213)", 2020-08-07T15:06:59.222 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:48.412254+0000", 2020-08-07T15:06:59.223 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 10.662629513000001, 2020-08-07T15:06:59.223 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.129284388, 2020-08-07T15:06:59.223 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:59.223 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:59.223 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:59.224 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.224 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:59.224 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.412254+0000", 2020-08-07T15:06:59.224 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:59.224 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.224 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.225 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:59.225 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.412254+0000", 2020-08-07T15:06:59.225 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011058800000000001 2020-08-07T15:06:59.225 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.225 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.226 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:59.226 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.412365+0000", 2020-08-07T15:06:59.226 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.268e-06 2020-08-07T15:06:59.226 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.226 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.226 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:59.227 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.412371+0000", 2020-08-07T15:06:59.227 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.127203449 2020-08-07T15:06:59.227 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.227 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.227 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:59.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.539575+0000", 2020-08-07T15:06:59.228 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.702e-06 2020-08-07T15:06:59.228 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.228 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.228 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:59.229 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.539579+0000", 2020-08-07T15:06:59.229 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3665000000000002e-05 2020-08-07T15:06:59.229 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.229 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.229 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:59.229 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.539633+0000", 2020-08-07T15:06:59.230 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6284e-05 2020-08-07T15:06:59.230 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.230 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.230 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:59.230 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.539669+0000", 2020-08-07T15:06:59.231 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0018059580000000001 2020-08-07T15:06:59.231 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.231 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.231 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:59.231 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.541475+0000", 2020-08-07T15:06:59.232 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3473999999999996e-05 2020-08-07T15:06:59.232 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.232 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.232 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:59.232 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.541539+0000", 2020-08-07T15:06:59.232 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.129284388 2020-08-07T15:06:59.233 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.233 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:59.233 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.233 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.233 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.234 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:24040 3.f e28/20 3:fef349f0:::benchmark_data_smithi074_35447_object3004:head v 28'1776, mlcod=28'1769)", 2020-08-07T15:06:59.234 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.178634+0000", 2020-08-07T15:06:59.234 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.8962496120000001, 2020-08-07T15:06:59.234 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13518461400000001, 2020-08-07T15:06:59.234 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:59.234 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:59.235 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:59.235 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.235 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:59.235 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.178634+0000", 2020-08-07T15:06:59.235 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:59.236 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.236 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.236 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:59.236 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.178634+0000", 2020-08-07T15:06:59.236 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000134857 2020-08-07T15:06:59.237 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.237 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.237 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:59.237 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.178769+0000", 2020-08-07T15:06:59.237 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1680000000000001e-06 2020-08-07T15:06:59.237 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.238 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.238 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:59.238 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.178775+0000", 2020-08-07T15:06:59.238 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13391025300000001 2020-08-07T15:06:59.238 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.239 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.239 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:59.239 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.312685+0000", 2020-08-07T15:06:59.239 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2349999999999999e-06 2020-08-07T15:06:59.239 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.240 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.240 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:59.240 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.312694+0000", 2020-08-07T15:06:59.240 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6122000000000002e-05 2020-08-07T15:06:59.240 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.241 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.241 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:59.241 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.312750+0000", 2020-08-07T15:06:59.241 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3825000000000002e-05 2020-08-07T15:06:59.241 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.242 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.242 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:59.242 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.312784+0000", 2020-08-07T15:06:59.242 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00097391999999999997 2020-08-07T15:06:59.242 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.242 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.243 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:59.243 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.313758+0000", 2020-08-07T15:06:59.243 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1234000000000004e-05 2020-08-07T15:06:59.243 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.243 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.244 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:59.244 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.313819+0000", 2020-08-07T15:06:59.244 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13518461400000001 2020-08-07T15:06:59.244 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.244 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:59.245 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.245 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.245 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.245 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:24047 3.1 e28/26 3:87d059ba:::benchmark_data_smithi074_35447_object3005:head v 28'1439, mlcod=28'1429)", 2020-08-07T15:06:59.245 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.187365+0000", 2020-08-07T15:06:59.245 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.8875184650000003, 2020-08-07T15:06:59.246 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14839102400000001, 2020-08-07T15:06:59.246 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:59.246 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:59.246 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:59.246 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.247 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:59.247 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.187365+0000", 2020-08-07T15:06:59.247 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:59.247 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.247 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.248 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:59.248 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.187365+0000", 2020-08-07T15:06:59.248 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4627999999999999e-05 2020-08-07T15:06:59.248 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.248 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.249 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:59.249 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.187420+0000", 2020-08-07T15:06:59.249 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5010000000000001e-06 2020-08-07T15:06:59.249 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.249 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.249 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:59.250 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.187426+0000", 2020-08-07T15:06:59.250 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14725048399999999 2020-08-07T15:06:59.250 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.250 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.250 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:59.251 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.334677+0000", 2020-08-07T15:06:59.251 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.066e-06 2020-08-07T15:06:59.251 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.251 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.251 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:59.252 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.334683+0000", 2020-08-07T15:06:59.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4073000000000003e-05 2020-08-07T15:06:59.252 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.252 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.252 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:59.252 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.334727+0000", 2020-08-07T15:06:59.253 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2813999999999999e-05 2020-08-07T15:06:59.253 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.253 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:59.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.334750+0000", 2020-08-07T15:06:59.254 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00097284100000000001 2020-08-07T15:06:59.254 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.254 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.254 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:59.254 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.335723+0000", 2020-08-07T15:06:59.255 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3617e-05 2020-08-07T15:06:59.255 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.255 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.255 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:59.255 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.335756+0000", 2020-08-07T15:06:59.255 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14839102400000001 2020-08-07T15:06:59.256 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.256 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:59.256 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.256 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.256 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.257 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:24048 3.1 e28/26 3:87d059ba:::benchmark_data_smithi074_35447_object3005:head v 28'1440, mlcod=28'1429)", 2020-08-07T15:06:59.257 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.187973+0000", 2020-08-07T15:06:59.257 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.8869103359999997, 2020-08-07T15:06:59.257 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16042247300000001, 2020-08-07T15:06:59.257 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:59.258 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:59.258 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:59.258 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.258 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:59.258 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.187973+0000", 2020-08-07T15:06:59.258 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:59.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.259 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.259 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:59.259 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.187973+0000", 2020-08-07T15:06:59.259 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0841999999999998e-05 2020-08-07T15:06:59.260 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.260 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.260 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:59.260 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.188034+0000", 2020-08-07T15:06:59.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6880000000000002e-06 2020-08-07T15:06:59.261 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.261 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.261 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:59.261 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.188041+0000", 2020-08-07T15:06:59.261 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.15963588500000001 2020-08-07T15:06:59.261 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.262 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.262 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:59.262 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.347677+0000", 2020-08-07T15:06:59.262 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2729999999999999e-06 2020-08-07T15:06:59.262 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.263 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.263 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:59.263 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.347683+0000", 2020-08-07T15:06:59.263 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5182999999999998e-05 2020-08-07T15:06:59.263 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.264 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.264 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:59.264 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.347718+0000", 2020-08-07T15:06:59.264 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7371999999999999e-05 2020-08-07T15:06:59.264 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.264 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.265 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:59.265 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.347736+0000", 2020-08-07T15:06:59.265 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00060796199999999998 2020-08-07T15:06:59.265 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.265 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.266 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:59.266 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.348344+0000", 2020-08-07T15:06:59.267 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2268000000000002e-05 2020-08-07T15:06:59.267 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.267 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.268 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:59.268 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.348396+0000", 2020-08-07T15:06:59.268 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16042247300000001 2020-08-07T15:06:59.268 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.268 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:59.268 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.269 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.269 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.269 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:25400 3.3 e28/20 3:ce1f26a6:::benchmark_data_smithi074_35447_object3174:head v 28'1568, mlcod=28'1563)", 2020-08-07T15:06:59.269 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:53.083601+0000", 2020-08-07T15:06:59.269 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.9912829890000001, 2020-08-07T15:06:59.270 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13685635900000001, 2020-08-07T15:06:59.270 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:59.270 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:59.270 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:59.270 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:59.271 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.083601+0000", 2020-08-07T15:06:59.271 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:59.271 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.271 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:59.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.083601+0000", 2020-08-07T15:06:59.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5063999999999999e-05 2020-08-07T15:06:59.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:59.273 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.083666+0000", 2020-08-07T15:06:59.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7630000000000003e-06 2020-08-07T15:06:59.274 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.274 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.274 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:59.274 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.083673+0000", 2020-08-07T15:06:59.274 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.134047693 2020-08-07T15:06:59.275 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.275 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.275 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:59.275 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.217720+0000", 2020-08-07T15:06:59.275 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2350000000000001e-06 2020-08-07T15:06:59.276 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.276 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.276 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:59.276 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.217725+0000", 2020-08-07T15:06:59.276 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0209000000000001e-05 2020-08-07T15:06:59.276 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.277 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.277 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:59.277 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.217765+0000", 2020-08-07T15:06:59.277 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.838e-05 2020-08-07T15:06:59.277 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.278 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.278 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:59.278 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.217783+0000", 2020-08-07T15:06:59.278 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026151080000000001 2020-08-07T15:06:59.278 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.279 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.279 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:59.279 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.220398+0000", 2020-08-07T15:06:59.279 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8906999999999997e-05 2020-08-07T15:06:59.279 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.280 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.280 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:59.280 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.220457+0000", 2020-08-07T15:06:59.280 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13685635900000001 2020-08-07T15:06:59.280 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.280 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:59.281 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.281 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.281 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.281 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:25768 3.6 e29/26 3:6f81a4c8:::benchmark_data_smithi074_35447_object3220:head v 29'1712, mlcod=29'1704)", 2020-08-07T15:06:59.281 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:53.308914+0000", 2020-08-07T15:06:59.282 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.7659696660000002, 2020-08-07T15:06:59.282 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.12118696499999999, 2020-08-07T15:06:59.282 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:59.282 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:59.282 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:59.283 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.283 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:59.283 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.308914+0000", 2020-08-07T15:06:59.283 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:59.283 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.284 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.284 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:59.284 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.308914+0000", 2020-08-07T15:06:59.284 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7206999999999998e-05 2020-08-07T15:06:59.284 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.284 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.285 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:59.285 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.308951+0000", 2020-08-07T15:06:59.285 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4190000000000004e-06 2020-08-07T15:06:59.285 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.285 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.286 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:59.286 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.308956+0000", 2020-08-07T15:06:59.286 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.12046752199999999 2020-08-07T15:06:59.286 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.286 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.287 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:59.287 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.429423+0000", 2020-08-07T15:06:59.287 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5039999999999999e-06 2020-08-07T15:06:59.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.287 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:59.288 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.429430+0000", 2020-08-07T15:06:59.288 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3760000000000001e-05 2020-08-07T15:06:59.288 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.288 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.288 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:59.289 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.429474+0000", 2020-08-07T15:06:59.289 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6044000000000001e-05 2020-08-07T15:06:59.289 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.289 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.289 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:59.289 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.429490+0000", 2020-08-07T15:06:59.290 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00056027599999999996 2020-08-07T15:06:59.290 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.290 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.290 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:59.290 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.430050+0000", 2020-08-07T15:06:59.291 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1233000000000002e-05 2020-08-07T15:06:59.291 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.291 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.291 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:59.291 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.430101+0000", 2020-08-07T15:06:59.292 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.12118696499999999 2020-08-07T15:06:59.292 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.292 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:59.292 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.292 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.292 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.293 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:27048 3.1 e29/26 3:88501d52:::benchmark_data_smithi074_35447_object3380:head v 29'1608, mlcod=29'1602)", 2020-08-07T15:06:59.293 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:54.045944+0000", 2020-08-07T15:06:59.293 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.0289401060000003, 2020-08-07T15:06:59.293 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.15902358699999999, 2020-08-07T15:06:59.293 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:59.294 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:59.294 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:59.294 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.294 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:59.294 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.045944+0000", 2020-08-07T15:06:59.295 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:59.295 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.295 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.295 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:59.295 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.045944+0000", 2020-08-07T15:06:59.295 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1610999999999998e-05 2020-08-07T15:06:59.296 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.296 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.296 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:59.296 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.045995+0000", 2020-08-07T15:06:59.296 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5180000000000001e-06 2020-08-07T15:06:59.297 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.297 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.297 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:59.297 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.046002+0000", 2020-08-07T15:06:59.297 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.156437364 2020-08-07T15:06:59.298 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.298 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.298 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:59.298 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.202439+0000", 2020-08-07T15:06:59.298 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0700000000000001e-06 2020-08-07T15:06:59.298 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.299 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.299 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:59.299 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.202446+0000", 2020-08-07T15:06:59.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5476000000000003e-05 2020-08-07T15:06:59.299 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.300 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.300 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:59.300 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.202492+0000", 2020-08-07T15:06:59.300 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9876999999999999e-05 2020-08-07T15:06:59.300 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.300 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.301 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:59.301 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.202522+0000", 2020-08-07T15:06:59.301 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023908919999999999 2020-08-07T15:06:59.301 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.301 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.302 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:59.302 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.204913+0000", 2020-08-07T15:06:59.302 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4778999999999998e-05 2020-08-07T15:06:59.302 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.302 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.303 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:59.303 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.204967+0000", 2020-08-07T15:06:59.303 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.15902358699999999 2020-08-07T15:06:59.303 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.303 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:59.303 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.304 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.304 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.304 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:27464 3.3 e29/20 3:c9f8435d:::benchmark_data_smithi074_35447_object3432:head v 29'1680, mlcod=29'1671)", 2020-08-07T15:06:59.304 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:54.366582+0000", 2020-08-07T15:06:59.304 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4.7083015939999999, 2020-08-07T15:06:59.305 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14665884600000001, 2020-08-07T15:06:59.305 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:59.305 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:59.305 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:59.305 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.306 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:59.306 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.366582+0000", 2020-08-07T15:06:59.306 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:59.306 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.306 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.306 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:59.307 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.366582+0000", 2020-08-07T15:06:59.307 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6334e-05 2020-08-07T15:06:59.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.307 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.307 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:59.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.366639+0000", 2020-08-07T15:06:59.308 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.153e-05 2020-08-07T15:06:59.308 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.308 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:59.309 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.366650+0000", 2020-08-07T15:06:59.309 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14520557000000001 2020-08-07T15:06:59.309 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.309 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.309 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:59.309 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.511856+0000", 2020-08-07T15:06:59.310 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4920000000000001e-06 2020-08-07T15:06:59.310 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.310 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.310 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:59.310 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.511861+0000", 2020-08-07T15:06:59.311 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0255e-05 2020-08-07T15:06:59.311 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.311 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.311 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:59.311 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.511901+0000", 2020-08-07T15:06:59.311 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3002000000000001e-05 2020-08-07T15:06:59.312 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.312 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.312 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:59.312 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.511924+0000", 2020-08-07T15:06:59.312 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001263712 2020-08-07T15:06:59.313 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.313 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.313 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:59.313 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.513188+0000", 2020-08-07T15:06:59.313 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2951000000000002e-05 2020-08-07T15:06:59.314 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.314 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.314 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:59.314 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.513241+0000", 2020-08-07T15:06:59.314 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14665884600000001 2020-08-07T15:06:59.314 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.315 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:59.315 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.315 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.315 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.315 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:31760 3.6 e30/26 3:6672be19:::benchmark_data_smithi074_35447_object3969:head v 30'2104, mlcod=30'2094)", 2020-08-07T15:06:59.316 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:56.999757+0000", 2020-08-07T15:06:59.316 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 2.0751269219999999, 2020-08-07T15:06:59.316 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17823482099999999, 2020-08-07T15:06:59.316 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:59.316 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:59.317 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:59.317 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.317 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:59.317 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:56.999757+0000", 2020-08-07T15:06:59.317 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:59.317 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.318 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.318 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:59.318 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:56.999757+0000", 2020-08-07T15:06:59.318 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2955999999999999e-05 2020-08-07T15:06:59.318 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.319 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.319 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:59.319 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:56.999810+0000", 2020-08-07T15:06:59.319 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5509999999999996e-06 2020-08-07T15:06:59.319 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.320 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.320 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:59.320 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:56.999816+0000", 2020-08-07T15:06:59.320 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.177361504 2020-08-07T15:06:59.320 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.320 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.321 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:59.321 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177178+0000", 2020-08-07T15:06:59.321 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6979999999999997e-06 2020-08-07T15:06:59.321 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.321 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.322 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:59.322 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177183+0000", 2020-08-07T15:06:59.322 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1974999999999997e-05 2020-08-07T15:06:59.322 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.322 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.322 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:59.323 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177225+0000", 2020-08-07T15:06:59.323 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4612000000000001e-05 2020-08-07T15:06:59.323 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.323 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.323 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:59.324 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177249+0000", 2020-08-07T15:06:59.324 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00069063000000000002 2020-08-07T15:06:59.324 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.324 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.324 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:59.325 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177940+0000", 2020-08-07T15:06:59.325 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1894999999999997e-05 2020-08-07T15:06:59.325 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.325 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.325 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:59.325 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177992+0000", 2020-08-07T15:06:59.326 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17823482099999999 2020-08-07T15:06:59.326 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.326 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:59.326 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.326 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.327 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.327 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:32416 3.6 e30/26 3:67a7a68c:::benchmark_data_smithi074_35447_object4051:head v 30'2160, mlcod=30'2152)", 2020-08-07T15:06:59.327 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:57.403189+0000", 2020-08-07T15:06:59.327 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 1.671695038, 2020-08-07T15:06:59.327 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.12971449099999999, 2020-08-07T15:06:59.328 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:59.328 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:59.328 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:59.328 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.328 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:59.328 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.403189+0000", 2020-08-07T15:06:59.329 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:59.329 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.329 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.329 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:59.329 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.403189+0000", 2020-08-07T15:06:59.330 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9764e-05 2020-08-07T15:06:59.330 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.330 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.330 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:59.330 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.403239+0000", 2020-08-07T15:06:59.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.198e-06 2020-08-07T15:06:59.331 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.331 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.331 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:59.331 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.403244+0000", 2020-08-07T15:06:59.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.12494756799999999 2020-08-07T15:06:59.332 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.332 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.332 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:59.332 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.528191+0000", 2020-08-07T15:06:59.332 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1449999999999996e-06 2020-08-07T15:06:59.333 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.333 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.333 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:59.333 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.528197+0000", 2020-08-07T15:06:59.333 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7052000000000003e-05 2020-08-07T15:06:59.333 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.334 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.334 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:59.334 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.528244+0000", 2020-08-07T15:06:59.334 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5486999999999998e-05 2020-08-07T15:06:59.334 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.335 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:59.335 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.528270+0000", 2020-08-07T15:06:59.335 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045769699999999996 2020-08-07T15:06:59.335 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.336 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.336 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:59.336 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.532847+0000", 2020-08-07T15:06:59.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6307000000000001e-05 2020-08-07T15:06:59.336 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.336 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.337 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:59.337 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.532903+0000", 2020-08-07T15:06:59.337 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.12971449099999999 2020-08-07T15:06:59.337 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.337 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:59.338 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.338 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.338 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.338 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:33976 3.7 e30/20 3:e3749638:::benchmark_data_smithi074_35447_object4246:head v 30'2064, mlcod=30'2056)", 2020-08-07T15:06:59.338 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:58.308014+0000", 2020-08-07T15:06:59.338 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.76686946899999997, 2020-08-07T15:06:59.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.12144026500000001, 2020-08-07T15:06:59.339 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:59.339 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:59.339 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:59.339 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.340 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:59.340 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.308014+0000", 2020-08-07T15:06:59.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:59.340 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.341 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:59.341 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.308014+0000", 2020-08-07T15:06:59.341 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9530999999999997e-05 2020-08-07T15:06:59.341 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.341 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:59.342 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.308064+0000", 2020-08-07T15:06:59.342 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1789999999999996e-06 2020-08-07T15:06:59.342 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.342 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.342 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:59.343 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.308070+0000", 2020-08-07T15:06:59.343 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.116358663 2020-08-07T15:06:59.343 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.343 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.343 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:59.343 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.424429+0000", 2020-08-07T15:06:59.344 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6229999999999996e-06 2020-08-07T15:06:59.344 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.344 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.344 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:59.344 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.424433+0000", 2020-08-07T15:06:59.345 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3936000000000001e-05 2020-08-07T15:06:59.345 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.345 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.345 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:59.345 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.424477+0000", 2020-08-07T15:06:59.346 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3952e-05 2020-08-07T15:06:59.346 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.346 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.346 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:59.346 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.424501+0000", 2020-08-07T15:06:59.346 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0049074019999999999 2020-08-07T15:06:59.347 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.347 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:59.347 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.429409+0000", 2020-08-07T15:06:59.347 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5979000000000001e-05 2020-08-07T15:06:59.348 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.348 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.348 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:06:59.348 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.429455+0000", 2020-08-07T15:06:59.348 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.12144026500000001 2020-08-07T15:06:59.348 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.349 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:59.349 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.349 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.349 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:59.349 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:06:59.350 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:06:59.352 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 23 16 35498 35482 12.0513 12.6094 0.00781545 0.0103645 2020-08-07T15:06:59.352 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:06:59.353 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:06:59.350+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:06:59.423 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:06:59.453 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:59.452+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:59.554 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:59.552+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:59.567 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:06:59.568 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:06:59.568 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.568 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:35896 3.6 3:63150e25:::benchmark_data_smithi074_35447_object4486:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-08-07T15:06:59.568 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:59.463120+0000", 2020-08-07T15:06:59.568 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.103884454, 2020-08-07T15:06:59.569 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.103910291, 2020-08-07T15:06:59.569 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:06:59.569 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:06:59.569 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:06:59.569 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:06:59.570 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:06:59.570 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 35896 2020-08-07T15:06:59.570 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.570 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:06:59.570 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.571 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:06:59.571 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.463120+0000", 2020-08-07T15:06:59.571 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:06:59.571 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.571 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.572 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:06:59.572 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.463120+0000", 2020-08-07T15:06:59.572 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1536999999999998e-05 2020-08-07T15:06:59.572 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.572 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.573 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:06:59.573 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.463172+0000", 2020-08-07T15:06:59.573 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8229999999999997e-06 2020-08-07T15:06:59.573 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.573 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.573 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:06:59.574 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.463178+0000", 2020-08-07T15:06:59.574 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5097e-05 2020-08-07T15:06:59.574 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.574 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.574 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:06:59.575 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.463193+0000", 2020-08-07T15:06:59.575 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1809999999999998e-06 2020-08-07T15:06:59.575 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.575 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:06:59.576 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.463200+0000", 2020-08-07T15:06:59.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0014763090000000001 2020-08-07T15:06:59.576 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.576 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.576 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:06:59.577 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.464676+0000", 2020-08-07T15:06:59.577 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6814000000000001e-05 2020-08-07T15:06:59.577 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.577 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.578 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:06:59.578 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.464743+0000", 2020-08-07T15:06:59.578 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012269099999999999 2020-08-07T15:06:59.578 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.578 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.579 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:06:59.579 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.464865+0000", 2020-08-07T15:06:59.579 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069254319999999996 2020-08-07T15:06:59.579 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.579 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.579 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:06:59.580 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.471791+0000", 2020-08-07T15:06:59.580 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0899999999999999e-06 2020-08-07T15:06:59.580 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.580 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.580 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:06:59.581 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.471792+0000", 2020-08-07T15:06:59.581 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1272999999999993e-05 2020-08-07T15:06:59.581 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.581 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.581 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:06:59.582 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.471863+0000", 2020-08-07T15:06:59.582 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.014790243 2020-08-07T15:06:59.582 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:06:59.583 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:06:59.583 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:06:59.583 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.486654+0000", 2020-08-07T15:06:59.583 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.023533490000000001 2020-08-07T15:06:59.583 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.584 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:06:59.584 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.584 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:06:59.590 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:06:59.590 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 1 2020-08-07T15:06:59.590 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:06:59.592 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:06:59.642 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:06:59.658 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:06:59.654+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:06:59.686 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:06:59.717 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:06:59.758 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:59.756+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:06:59.761 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:06:59.802 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:06:59.815 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:06:59.816 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:06:59.823 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:06:59.845 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:06:59.860 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:06:59.858+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:06:59.879 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:06:59.922 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:06:59.924 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:06:59.925 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:06:59.925 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:06:59.925 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:06:59.926 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:06:59.926 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:06:59.926 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:06:59.926 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:06:59.963 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:06:59.962+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:00.014 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:00.014 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:00.032 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:07:00.076 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:00.074+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:00.164 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:00.162+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:00.203 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:00.203 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:00.215 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:07:00.248 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 24 16 37221 37205 12.11 13.4609 0.00711856 0.0103176 2020-08-07T15:07:00.268 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:00.265+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:00.376 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:00.375+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:00.425 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:00.425 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:00.435 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:07:00.486 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:00.484+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:00.583 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:00.581+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:00.618 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:00.619 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:00.627 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:07:00.684 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:00.683+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:00.785 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:00.784+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:00.856 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:00.856 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:00.867 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:07:00.932 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:00.912+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:00.989 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:00.988+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:01.054 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:01.054 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:01.064 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:07:01.106 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:01.104+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:01.193 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:01.190+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:01.242 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 25 15 38900 38885 12.1506 13.125 0.010658 0.0102652 2020-08-07T15:07:01.271 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:01.272 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:01.273 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:07:01.321 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:01.318+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:01.354 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:07:01.408 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:01.405+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:01.430 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:01.431 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:01.445 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:07:01.532 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:01.530+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:01.608 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:01.606+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:01.662 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:01.662 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:01.665 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:07:01.710 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:01.707+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:01.810 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:01.808+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:01.883 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:01.883 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:01.895 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:07:01.912 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:01.910+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:01.963 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:07:02.015 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:02.012+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:02.066 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:02.066 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:02.091 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:07:02.115 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:02.113+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:02.172 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:07:02.207 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:07:02.221 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:02.214+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:02.242 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 26 16 40428 40412 12.142 11.9297 0.00856265 0.0102868 2020-08-07T15:07:02.243 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:07:02.269 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:02.269 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:02.294 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:07:02.295 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:07:02.302 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:07:02.320 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:02.318+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:02.326 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:07:02.366 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:07:02.405 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:07:02.429 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:02.419+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:02.446 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:07:02.447 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:07:02.447 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:07:02.448 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:07:02.448 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:07:02.448 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:07:02.448 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:07:02.449 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:07:02.449 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:07:02.515 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:02.518 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:07:02.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:07:02.519 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:02.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.519 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:1592 3.5 3:ab93003a:::benchmark_data_smithi074_35447_object198:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:07:02.519 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.292205+0000", 2020-08-07T15:07:02.520 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.215739155000001, 2020-08-07T15:07:02.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13617961100000001, 2020-08-07T15:07:02.520 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:02.520 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:02.520 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:02.521 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:02.521 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:02.521 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1592 2020-08-07T15:07:02.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.521 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:02.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:02.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.292205+0000", 2020-08-07T15:07:02.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:02.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:02.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.292205+0000", 2020-08-07T15:07:02.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7688999999999997e-05 2020-08-07T15:07:02.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:02.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.292252+0000", 2020-08-07T15:07:02.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0139999999999998e-06 2020-08-07T15:07:02.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:02.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.292257+0000", 2020-08-07T15:07:02.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0682e-05 2020-08-07T15:07:02.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:02.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.292268+0000", 2020-08-07T15:07:02.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1590000000000003e-06 2020-08-07T15:07:02.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:02.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.292272+0000", 2020-08-07T15:07:02.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030296559999999999 2020-08-07T15:07:02.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:02.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.295302+0000", 2020-08-07T15:07:02.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9184000000000004e-05 2020-08-07T15:07:02.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:02.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.295401+0000", 2020-08-07T15:07:02.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111563 2020-08-07T15:07:02.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:07:02.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.295513+0000", 2020-08-07T15:07:02.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030877980000000001 2020-08-07T15:07:02.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:02.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.298600+0000", 2020-08-07T15:07:02.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8800000000000002e-07 2020-08-07T15:07:02.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:02.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.298601+0000", 2020-08-07T15:07:02.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7547999999999999e-05 2020-08-07T15:07:02.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:02.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.298658+0000", 2020-08-07T15:07:02.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038224410000000002 2020-08-07T15:07:02.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.302481+0000", 2020-08-07T15:07:02.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.125813647 2020-08-07T15:07:02.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.428295+0000", 2020-08-07T15:07:02.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.048e-06 2020-08-07T15:07:02.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:02.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.428296+0000", 2020-08-07T15:07:02.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8593999999999999e-05 2020-08-07T15:07:02.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:02.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.428384+0000", 2020-08-07T15:07:02.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13617961100000001 2020-08-07T15:07:02.555 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.555 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:02.556 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.556 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:15744 3.d e27/20 3:b5661143:::benchmark_data_smithi074_35447_object1967:head v 27'984, mlcod=27'977)", 2020-08-07T15:07:02.557 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.777088+0000", 2020-08-07T15:07:02.557 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.730856153, 2020-08-07T15:07:02.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23236351199999999, 2020-08-07T15:07:02.557 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:02.557 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:02.558 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:02.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:02.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777088+0000", 2020-08-07T15:07:02.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:02.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:02.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777088+0000", 2020-08-07T15:07:02.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9495e-05 2020-08-07T15:07:02.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:02.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777137+0000", 2020-08-07T15:07:02.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3049999999999998e-06 2020-08-07T15:07:02.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:02.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777143+0000", 2020-08-07T15:07:02.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23074995500000001 2020-08-07T15:07:02.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:02.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.007893+0000", 2020-08-07T15:07:02.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4390000000000003e-06 2020-08-07T15:07:02.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:02.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.007901+0000", 2020-08-07T15:07:02.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0071000000000003e-05 2020-08-07T15:07:02.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:02.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.007941+0000", 2020-08-07T15:07:02.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2722e-05 2020-08-07T15:07:02.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:02.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.007964+0000", 2020-08-07T15:07:02.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014374870000000001 2020-08-07T15:07:02.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:02.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.009401+0000", 2020-08-07T15:07:02.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0037999999999998e-05 2020-08-07T15:07:02.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:02.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.009451+0000", 2020-08-07T15:07:02.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23236351199999999 2020-08-07T15:07:02.571 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.571 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:02.572 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.572 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:16248 3.d e27/20 3:b7542ecb:::benchmark_data_smithi074_35447_object2030:head v 27'1000, mlcod=27'991)", 2020-08-07T15:07:02.572 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:47.122503+0000", 2020-08-07T15:07:02.573 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.385440609, 2020-08-07T15:07:02.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12336765500000001, 2020-08-07T15:07:02.573 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:02.573 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:02.573 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:02.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:02.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.122503+0000", 2020-08-07T15:07:02.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:02.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:02.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.122503+0000", 2020-08-07T15:07:02.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6597e-05 2020-08-07T15:07:02.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:02.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.122570+0000", 2020-08-07T15:07:02.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1250000000000004e-06 2020-08-07T15:07:02.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:02.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.122577+0000", 2020-08-07T15:07:02.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11575635300000001 2020-08-07T15:07:02.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:02.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.238333+0000", 2020-08-07T15:07:02.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5031999999999999e-05 2020-08-07T15:07:02.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:02.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.238348+0000", 2020-08-07T15:07:02.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001322658 2020-08-07T15:07:02.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:02.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.239671+0000", 2020-08-07T15:07:02.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0985999999999999e-05 2020-08-07T15:07:02.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:02.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.239692+0000", 2020-08-07T15:07:02.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061335269999999997 2020-08-07T15:07:02.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:02.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.245825+0000", 2020-08-07T15:07:02.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5377000000000003e-05 2020-08-07T15:07:02.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:02.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.245871+0000", 2020-08-07T15:07:02.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12336765500000001 2020-08-07T15:07:02.584 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.584 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:02.584 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.585 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:17856 3.5 3:a97d5eb0:::benchmark_data_smithi074_35447_object2231:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e27)", 2020-08-07T15:07:02.585 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.164730+0000", 2020-08-07T15:07:02.585 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.343213243999999, 2020-08-07T15:07:02.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.123557982, 2020-08-07T15:07:02.586 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:02.586 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:02.586 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:02.586 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:02.586 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:02.587 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 17856 2020-08-07T15:07:02.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.587 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:02.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:02.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.164730+0000", 2020-08-07T15:07:02.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:02.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:02.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.164730+0000", 2020-08-07T15:07:02.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5515000000000001e-05 2020-08-07T15:07:02.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:02.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.164766+0000", 2020-08-07T15:07:02.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0249999999999996e-06 2020-08-07T15:07:02.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:02.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.164770+0000", 2020-08-07T15:07:02.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5120000000000002e-06 2020-08-07T15:07:02.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:02.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.164778+0000", 2020-08-07T15:07:02.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9330000000000003e-06 2020-08-07T15:07:02.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:02.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.164781+0000", 2020-08-07T15:07:02.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001643278 2020-08-07T15:07:02.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:02.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.166425+0000", 2020-08-07T15:07:02.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107273 2020-08-07T15:07:02.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:02.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.166532+0000", 2020-08-07T15:07:02.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000114817 2020-08-07T15:07:02.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:07:02.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.166647+0000", 2020-08-07T15:07:02.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00562541 2020-08-07T15:07:02.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.172272+0000", 2020-08-07T15:07:02.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039347770000000004 2020-08-07T15:07:02.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:02.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.176207+0000", 2020-08-07T15:07:02.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.111947885 2020-08-07T15:07:02.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.288155+0000", 2020-08-07T15:07:02.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6599999999999994e-07 2020-08-07T15:07:02.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:02.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.288156+0000", 2020-08-07T15:07:02.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2797000000000001e-05 2020-08-07T15:07:02.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:02.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.288229+0000", 2020-08-07T15:07:02.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9793999999999997e-05 2020-08-07T15:07:02.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:02.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.288288+0000", 2020-08-07T15:07:02.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.123557982 2020-08-07T15:07:02.605 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.605 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:02.605 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.606 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:18064 3.c 3:345b6856:::benchmark_data_smithi074_35447_object2257:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:02.606 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.284794+0000", 2020-08-07T15:07:02.606 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.223149845, 2020-08-07T15:07:02.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.117991606, 2020-08-07T15:07:02.607 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:02.607 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:02.607 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:02.607 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:02.607 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:02.607 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 18064 2020-08-07T15:07:02.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.608 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:02.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:02.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.284794+0000", 2020-08-07T15:07:02.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:02.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:02.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.284794+0000", 2020-08-07T15:07:02.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6316999999999998e-05 2020-08-07T15:07:02.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:02.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.284830+0000", 2020-08-07T15:07:02.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1409999999999998e-06 2020-08-07T15:07:02.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:02.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.284834+0000", 2020-08-07T15:07:02.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6520000000000006e-06 2020-08-07T15:07:02.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:02.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.284842+0000", 2020-08-07T15:07:02.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1960000000000001e-06 2020-08-07T15:07:02.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:02.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.284846+0000", 2020-08-07T15:07:02.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2447999999999994e-05 2020-08-07T15:07:02.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:02.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.284929+0000", 2020-08-07T15:07:02.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5260999999999995e-05 2020-08-07T15:07:02.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:02.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.285024+0000", 2020-08-07T15:07:02.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018773999999999999 2020-08-07T15:07:02.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:07:02.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.285212+0000", 2020-08-07T15:07:02.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00096594200000000002 2020-08-07T15:07:02.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:02.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.286178+0000", 2020-08-07T15:07:02.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3200000000000005e-07 2020-08-07T15:07:02.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:02.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.286178+0000", 2020-08-07T15:07:02.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9715000000000002e-05 2020-08-07T15:07:02.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:02.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.286218+0000", 2020-08-07T15:07:02.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027229700000000001 2020-08-07T15:07:02.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.286490+0000", 2020-08-07T15:07:02.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116257684 2020-08-07T15:07:02.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.402748+0000", 2020-08-07T15:07:02.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4499999999999997e-07 2020-08-07T15:07:02.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:02.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.402749+0000", 2020-08-07T15:07:02.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6936e-05 2020-08-07T15:07:02.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:02.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.402785+0000", 2020-08-07T15:07:02.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.117991606 2020-08-07T15:07:02.631 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.631 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:02.632 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.632 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:24392 3.d e28/20 3:bb13957b:::benchmark_data_smithi074_35447_object3048:head v 28'1616, mlcod=28'1606)", 2020-08-07T15:07:02.632 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:52.463391+0000", 2020-08-07T15:07:02.633 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.044552387, 2020-08-07T15:07:02.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.162237187, 2020-08-07T15:07:02.633 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:02.633 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:02.633 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:02.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:02.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.463391+0000", 2020-08-07T15:07:02.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:02.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:02.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.463391+0000", 2020-08-07T15:07:02.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0294999999999999e-05 2020-08-07T15:07:02.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:02.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.463442+0000", 2020-08-07T15:07:02.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1149999999999999e-06 2020-08-07T15:07:02.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:02.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.463448+0000", 2020-08-07T15:07:02.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16129982900000001 2020-08-07T15:07:02.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:02.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.624748+0000", 2020-08-07T15:07:02.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6420000000000007e-06 2020-08-07T15:07:02.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:02.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.624755+0000", 2020-08-07T15:07:02.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8579000000000003e-05 2020-08-07T15:07:02.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:02.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.624814+0000", 2020-08-07T15:07:02.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5729999999999999e-05 2020-08-07T15:07:02.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:02.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.624840+0000", 2020-08-07T15:07:02.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00073357399999999997 2020-08-07T15:07:02.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:02.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.625573+0000", 2020-08-07T15:07:02.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5423000000000001e-05 2020-08-07T15:07:02.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:02.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.625629+0000", 2020-08-07T15:07:02.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.162237187 2020-08-07T15:07:02.647 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.648 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:02.648 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.648 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:24640 3.d e28/20 3:b3ce8a98:::benchmark_data_smithi074_35447_object3079:head v 28'1624, mlcod=28'1615)", 2020-08-07T15:07:02.649 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:52.630934+0000", 2020-08-07T15:07:02.649 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.8770092520000006, 2020-08-07T15:07:02.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14322729300000001, 2020-08-07T15:07:02.650 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:02.651 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:02.651 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:02.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:02.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.630934+0000", 2020-08-07T15:07:02.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:02.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:02.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.630934+0000", 2020-08-07T15:07:02.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1687000000000002e-05 2020-08-07T15:07:02.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:02.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.630986+0000", 2020-08-07T15:07:02.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3010000000000003e-06 2020-08-07T15:07:02.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:02.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.630992+0000", 2020-08-07T15:07:02.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14074687499999999 2020-08-07T15:07:02.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:02.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.771739+0000", 2020-08-07T15:07:02.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2229999999999998e-06 2020-08-07T15:07:02.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:02.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.771745+0000", 2020-08-07T15:07:02.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6329000000000003e-05 2020-08-07T15:07:02.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:02.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.771791+0000", 2020-08-07T15:07:02.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5248e-05 2020-08-07T15:07:02.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:02.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.771816+0000", 2020-08-07T15:07:02.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002296111 2020-08-07T15:07:02.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:02.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.774112+0000", 2020-08-07T15:07:02.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9518999999999998e-05 2020-08-07T15:07:02.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:02.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.774162+0000", 2020-08-07T15:07:02.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14322729300000001 2020-08-07T15:07:02.667 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.667 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:02.667 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.668 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:27313 3.5 3:a11d2067:::benchmark_data_smithi074_35447_object3414:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:02.668 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:54.203862+0000", 2020-08-07T15:07:02.669 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.3040821230000006, 2020-08-07T15:07:02.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.117610218, 2020-08-07T15:07:02.669 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:02.669 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:02.669 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:02.670 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:02.670 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:02.670 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 27313 2020-08-07T15:07:02.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.670 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:02.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:02.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.203862+0000", 2020-08-07T15:07:02.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:02.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:02.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.203862+0000", 2020-08-07T15:07:02.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0980999999999999e-05 2020-08-07T15:07:02.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:02.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.203913+0000", 2020-08-07T15:07:02.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9639999999999997e-06 2020-08-07T15:07:02.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:02.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.203920+0000", 2020-08-07T15:07:02.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6892999999999999e-05 2020-08-07T15:07:02.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:02.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.203936+0000", 2020-08-07T15:07:02.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1260000000000004e-06 2020-08-07T15:07:02.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:02.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.203946+0000", 2020-08-07T15:07:02.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6901999999999998e-05 2020-08-07T15:07:02.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:02.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204002+0000", 2020-08-07T15:07:02.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000219242 2020-08-07T15:07:02.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:02.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204222+0000", 2020-08-07T15:07:02.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021698099999999999 2020-08-07T15:07:02.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:07:02.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204439+0000", 2020-08-07T15:07:02.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036695790000000001 2020-08-07T15:07:02.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:02.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208108+0000", 2020-08-07T15:07:02.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5199999999999995e-07 2020-08-07T15:07:02.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:02.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208109+0000", 2020-08-07T15:07:02.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7765999999999997e-05 2020-08-07T15:07:02.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:02.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208147+0000", 2020-08-07T15:07:02.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015280599999999999 2020-08-07T15:07:02.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.209675+0000", 2020-08-07T15:07:02.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.111717699 2020-08-07T15:07:02.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.321393+0000", 2020-08-07T15:07:02.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2399999999999997e-07 2020-08-07T15:07:02.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:02.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.321393+0000", 2020-08-07T15:07:02.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8349000000000005e-05 2020-08-07T15:07:02.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:02.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.321472+0000", 2020-08-07T15:07:02.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.117610218 2020-08-07T15:07:02.694 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.694 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:02.694 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.695 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:27314 3.5 3:a11d2067:::benchmark_data_smithi074_35447_object3414:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:02.695 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:54.203986+0000", 2020-08-07T15:07:02.695 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.3039576830000001, 2020-08-07T15:07:02.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12129693699999999, 2020-08-07T15:07:02.696 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:02.696 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:02.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:02.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:02.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:02.697 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 27314 2020-08-07T15:07:02.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.698 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:02.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:02.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.203986+0000", 2020-08-07T15:07:02.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:02.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:02.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.203986+0000", 2020-08-07T15:07:02.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9731999999999999e-05 2020-08-07T15:07:02.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:02.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204016+0000", 2020-08-07T15:07:02.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7430000000000002e-06 2020-08-07T15:07:02.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:02.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204021+0000", 2020-08-07T15:07:02.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2346e-05 2020-08-07T15:07:02.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:02.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204033+0000", 2020-08-07T15:07:02.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7299999999999996e-06 2020-08-07T15:07:02.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:02.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204038+0000", 2020-08-07T15:07:02.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00090682499999999999 2020-08-07T15:07:02.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:02.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204944+0000", 2020-08-07T15:07:02.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010785099999999999 2020-08-07T15:07:02.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:02.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.205052+0000", 2020-08-07T15:07:02.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021478099999999999 2020-08-07T15:07:02.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:07:02.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.205267+0000", 2020-08-07T15:07:02.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028852370000000001 2020-08-07T15:07:02.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:02.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208152+0000", 2020-08-07T15:07:02.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5899999999999998e-07 2020-08-07T15:07:02.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:02.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208153+0000", 2020-08-07T15:07:02.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8082999999999999e-05 2020-08-07T15:07:02.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:02.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208181+0000", 2020-08-07T15:07:02.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001815416 2020-08-07T15:07:02.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.209996+0000", 2020-08-07T15:07:02.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115217685 2020-08-07T15:07:02.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.325214+0000", 2020-08-07T15:07:02.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2200000000000004e-07 2020-08-07T15:07:02.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:02.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.325214+0000", 2020-08-07T15:07:02.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8726999999999994e-05 2020-08-07T15:07:02.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:02.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.325283+0000", 2020-08-07T15:07:02.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12129693699999999 2020-08-07T15:07:02.714 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.715 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:02.715 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.716 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:27315 3.5 3:a11d2067:::benchmark_data_smithi074_35447_object3414:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:02.716 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:54.204176+0000", 2020-08-07T15:07:02.716 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.3037681600000006, 2020-08-07T15:07:02.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.123218044, 2020-08-07T15:07:02.716 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:02.716 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:02.717 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:02.717 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:02.718 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:02.718 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 27315 2020-08-07T15:07:02.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.718 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:02.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:02.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204176+0000", 2020-08-07T15:07:02.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:02.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:02.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204176+0000", 2020-08-07T15:07:02.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6219e-05 2020-08-07T15:07:02.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:02.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204212+0000", 2020-08-07T15:07:02.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5939999999999998e-06 2020-08-07T15:07:02.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:02.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204215+0000", 2020-08-07T15:07:02.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7169999999999997e-06 2020-08-07T15:07:02.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:02.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204224+0000", 2020-08-07T15:07:02.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3499999999999999e-06 2020-08-07T15:07:02.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:02.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204228+0000", 2020-08-07T15:07:02.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001455488 2020-08-07T15:07:02.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:02.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.205684+0000", 2020-08-07T15:07:02.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7450999999999999e-05 2020-08-07T15:07:02.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:02.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.205781+0000", 2020-08-07T15:07:02.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018575199999999999 2020-08-07T15:07:02.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:07:02.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.205967+0000", 2020-08-07T15:07:02.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022180199999999998 2020-08-07T15:07:02.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:02.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208185+0000", 2020-08-07T15:07:02.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.04e-07 2020-08-07T15:07:02.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:02.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208185+0000", 2020-08-07T15:07:02.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4456000000000003e-05 2020-08-07T15:07:02.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:02.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208230+0000", 2020-08-07T15:07:02.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001801031 2020-08-07T15:07:02.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.210031+0000", 2020-08-07T15:07:02.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.117308515 2020-08-07T15:07:02.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.327339+0000", 2020-08-07T15:07:02.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2399999999999997e-07 2020-08-07T15:07:02.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:02.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.327340+0000", 2020-08-07T15:07:02.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3423e-05 2020-08-07T15:07:02.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:02.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.327394+0000", 2020-08-07T15:07:02.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.123218044 2020-08-07T15:07:02.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.734 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:02.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.735 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:27316 3.5 3:a11d2067:::benchmark_data_smithi074_35447_object3414:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:02.735 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:54.204248+0000", 2020-08-07T15:07:02.735 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.3036954680000008, 2020-08-07T15:07:02.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12325235499999999, 2020-08-07T15:07:02.736 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:02.736 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:02.736 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:02.736 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:02.736 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:02.736 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 27316 2020-08-07T15:07:02.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.737 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:02.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:02.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204248+0000", 2020-08-07T15:07:02.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:02.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:02.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204248+0000", 2020-08-07T15:07:02.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1449999999999999e-05 2020-08-07T15:07:02.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:02.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204280+0000", 2020-08-07T15:07:02.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0970000000000002e-06 2020-08-07T15:07:02.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:02.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204283+0000", 2020-08-07T15:07:02.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1720000000000004e-06 2020-08-07T15:07:02.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:02.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204289+0000", 2020-08-07T15:07:02.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8219999999999998e-06 2020-08-07T15:07:02.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:02.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204292+0000", 2020-08-07T15:07:02.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020809700000000001 2020-08-07T15:07:02.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:02.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.206373+0000", 2020-08-07T15:07:02.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6865000000000002e-05 2020-08-07T15:07:02.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:02.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.206470+0000", 2020-08-07T15:07:02.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000170631 2020-08-07T15:07:02.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:07:02.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.206640+0000", 2020-08-07T15:07:02.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029298710000000001 2020-08-07T15:07:02.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.209570+0000", 2020-08-07T15:07:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9999999999999996e-07 2020-08-07T15:07:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.209571+0000", 2020-08-07T15:07:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2813999999999999e-05 2020-08-07T15:07:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.209634+0000", 2020-08-07T15:07:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00042981699999999998 2020-08-07T15:07:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.210064+0000", 2020-08-07T15:07:02.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.117374487 2020-08-07T15:07:02.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.327438+0000", 2020-08-07T15:07:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2799999999999996e-07 2020-08-07T15:07:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.327439+0000", 2020-08-07T15:07:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1931e-05 2020-08-07T15:07:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:02.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.327501+0000", 2020-08-07T15:07:02.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12325235499999999 2020-08-07T15:07:02.757 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.758 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:02.758 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.759 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:27317 3.5 3:a11d2067:::benchmark_data_smithi074_35447_object3414:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:02.759 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:54.205688+0000", 2020-08-07T15:07:02.759 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.3022555780000005, 2020-08-07T15:07:02.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.121962605, 2020-08-07T15:07:02.759 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:02.760 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:02.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:02.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:02.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:02.760 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 27317 2020-08-07T15:07:02.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.761 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:02.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:02.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.205688+0000", 2020-08-07T15:07:02.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:02.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:02.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.205688+0000", 2020-08-07T15:07:02.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8769e-05 2020-08-07T15:07:02.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:02.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.205737+0000", 2020-08-07T15:07:02.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1199999999999999e-06 2020-08-07T15:07:02.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:02.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.205743+0000", 2020-08-07T15:07:02.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1260000000000001e-05 2020-08-07T15:07:02.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:02.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.205754+0000", 2020-08-07T15:07:02.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7930000000000003e-06 2020-08-07T15:07:02.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:02.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.205760+0000", 2020-08-07T15:07:02.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013008449999999999 2020-08-07T15:07:02.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:02.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.207061+0000", 2020-08-07T15:07:02.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100398 2020-08-07T15:07:02.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:02.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.207161+0000", 2020-08-07T15:07:02.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016082499999999999 2020-08-07T15:07:02.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:07:02.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.207322+0000", 2020-08-07T15:07:02.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042353 2020-08-07T15:07:02.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.211557+0000", 2020-08-07T15:07:02.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034108000000000001 2020-08-07T15:07:02.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:02.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.211899+0000", 2020-08-07T15:07:02.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115642549 2020-08-07T15:07:02.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.327541+0000", 2020-08-07T15:07:02.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6899999999999998e-07 2020-08-07T15:07:02.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.327542+0000", 2020-08-07T15:07:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4193e-05 2020-08-07T15:07:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.327606+0000", 2020-08-07T15:07:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5003999999999998e-05 2020-08-07T15:07:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.327651+0000", 2020-08-07T15:07:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.121962605 2020-08-07T15:07:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.778 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:27318 3.5 3:a11d2067:::benchmark_data_smithi074_35447_object3414:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:02.778 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:54.206599+0000", 2020-08-07T15:07:02.778 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.3013445390000005, 2020-08-07T15:07:02.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12202629600000001, 2020-08-07T15:07:02.779 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:02.779 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:02.779 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:02.779 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:02.780 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:02.780 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 27318 2020-08-07T15:07:02.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.780 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:02.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:02.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.206599+0000", 2020-08-07T15:07:02.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:02.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:02.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.206599+0000", 2020-08-07T15:07:02.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2527e-05 2020-08-07T15:07:02.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:02.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.206652+0000", 2020-08-07T15:07:02.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2030000000000003e-06 2020-08-07T15:07:02.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:02.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.206659+0000", 2020-08-07T15:07:02.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3346000000000001e-05 2020-08-07T15:07:02.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:02.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.206672+0000", 2020-08-07T15:07:02.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1090000000000001e-05 2020-08-07T15:07:02.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:02.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.206683+0000", 2020-08-07T15:07:02.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00098956499999999998 2020-08-07T15:07:02.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:02.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.207673+0000", 2020-08-07T15:07:02.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9324999999999999e-05 2020-08-07T15:07:02.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:02.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.207732+0000", 2020-08-07T15:07:02.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010767700000000001 2020-08-07T15:07:02.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:07:02.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.207840+0000", 2020-08-07T15:07:02.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040669340000000003 2020-08-07T15:07:02.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:02.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.211907+0000", 2020-08-07T15:07:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5700000000000002e-07 2020-08-07T15:07:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.211908+0000", 2020-08-07T15:07:02.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5735000000000007e-05 2020-08-07T15:07:02.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:02.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.211973+0000", 2020-08-07T15:07:02.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00094504699999999997 2020-08-07T15:07:02.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.212918+0000", 2020-08-07T15:07:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115640482 2020-08-07T15:07:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.328559+0000", 2020-08-07T15:07:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2200000000000004e-07 2020-08-07T15:07:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.328559+0000", 2020-08-07T15:07:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5986000000000001e-05 2020-08-07T15:07:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:02.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.328625+0000", 2020-08-07T15:07:02.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12202629600000001 2020-08-07T15:07:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:27319 3.5 3:a11d2067:::benchmark_data_smithi074_35447_object3414:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:54.208019+0000", 2020-08-07T15:07:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.2999252230000007, 2020-08-07T15:07:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12071333099999999, 2020-08-07T15:07:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 27319 2020-08-07T15:07:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208019+0000", 2020-08-07T15:07:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208019+0000", 2020-08-07T15:07:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3840999999999997e-05 2020-08-07T15:07:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208052+0000", 2020-08-07T15:07:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5669999999999996e-06 2020-08-07T15:07:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208057+0000", 2020-08-07T15:07:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9900000000000003e-06 2020-08-07T15:07:02.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:02.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208066+0000", 2020-08-07T15:07:02.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0784999999999999e-05 2020-08-07T15:07:02.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:02.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208077+0000", 2020-08-07T15:07:02.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000200865 2020-08-07T15:07:02.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:02.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208278+0000", 2020-08-07T15:07:02.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9265000000000006e-05 2020-08-07T15:07:02.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:02.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208377+0000", 2020-08-07T15:07:02.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000184148 2020-08-07T15:07:02.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:07:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208561+0000", 2020-08-07T15:07:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003421491 2020-08-07T15:07:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.211982+0000", 2020-08-07T15:07:02.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7800000000000001e-07 2020-08-07T15:07:02.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:02.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.211983+0000", 2020-08-07T15:07:02.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7167999999999999e-05 2020-08-07T15:07:02.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:02.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.212040+0000", 2020-08-07T15:07:02.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030651553000000002 2020-08-07T15:07:02.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.242692+0000", 2020-08-07T15:07:02.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.085977397999999997 2020-08-07T15:07:02.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.328669+0000", 2020-08-07T15:07:02.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1200000000000003e-07 2020-08-07T15:07:02.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:02.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.328670+0000", 2020-08-07T15:07:02.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2169999999999996e-05 2020-08-07T15:07:02.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:02.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.328732+0000", 2020-08-07T15:07:02.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12071333099999999 2020-08-07T15:07:02.844 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.844 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:02.844 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.845 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:27320 3.5 3:a11d2067:::benchmark_data_smithi074_35447_object3414:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:02.845 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:54.208099+0000", 2020-08-07T15:07:02.847 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.299844277, 2020-08-07T15:07:02.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.120753469, 2020-08-07T15:07:02.847 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:02.847 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:02.848 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:02.848 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:02.848 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:02.849 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 27320 2020-08-07T15:07:02.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.849 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:02.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:02.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208099+0000", 2020-08-07T15:07:02.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:02.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:02.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208099+0000", 2020-08-07T15:07:02.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8172999999999999e-05 2020-08-07T15:07:02.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:02.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208128+0000", 2020-08-07T15:07:02.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0410000000000002e-06 2020-08-07T15:07:02.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:02.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208131+0000", 2020-08-07T15:07:02.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1860000000000003e-06 2020-08-07T15:07:02.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:02.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208138+0000", 2020-08-07T15:07:02.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9529999999999999e-06 2020-08-07T15:07:02.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:02.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208141+0000", 2020-08-07T15:07:02.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00082086400000000003 2020-08-07T15:07:02.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:02.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208962+0000", 2020-08-07T15:07:02.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2806000000000006e-05 2020-08-07T15:07:02.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:02.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.209054+0000", 2020-08-07T15:07:02.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000167596 2020-08-07T15:07:02.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:07:02.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.209222+0000", 2020-08-07T15:07:02.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028282979999999999 2020-08-07T15:07:02.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:02.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.212050+0000", 2020-08-07T15:07:02.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9100000000000004e-07 2020-08-07T15:07:02.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:02.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.212051+0000", 2020-08-07T15:07:02.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4016999999999994e-05 2020-08-07T15:07:02.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:02.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.212115+0000", 2020-08-07T15:07:02.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030628244999999998 2020-08-07T15:07:02.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.242743+0000", 2020-08-07T15:07:02.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.086037783000000007 2020-08-07T15:07:02.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.328781+0000", 2020-08-07T15:07:02.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8100000000000003e-07 2020-08-07T15:07:02.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:02.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.328782+0000", 2020-08-07T15:07:02.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1334999999999994e-05 2020-08-07T15:07:02.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:02.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.328853+0000", 2020-08-07T15:07:02.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.120753469 2020-08-07T15:07:02.865 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.865 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:02.865 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.866 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:27407 3.5 3:ab4f36a5:::benchmark_data_smithi074_35447_object3425:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:02.866 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:54.329694+0000", 2020-08-07T15:07:02.866 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.1782496360000003, 2020-08-07T15:07:02.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12782907299999999, 2020-08-07T15:07:02.866 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:02.867 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:02.867 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:02.867 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:02.867 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:02.867 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 27407 2020-08-07T15:07:02.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.868 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:02.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:02.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.329694+0000", 2020-08-07T15:07:02.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:02.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:02.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.329694+0000", 2020-08-07T15:07:02.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0769999999999997e-05 2020-08-07T15:07:02.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:02.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.329745+0000", 2020-08-07T15:07:02.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3279999999999996e-06 2020-08-07T15:07:02.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:02.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.329750+0000", 2020-08-07T15:07:02.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.112e-05 2020-08-07T15:07:02.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:02.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.329761+0000", 2020-08-07T15:07:02.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7890000000000002e-06 2020-08-07T15:07:02.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:02.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.329766+0000", 2020-08-07T15:07:02.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042646089999999999 2020-08-07T15:07:02.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:02.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.334031+0000", 2020-08-07T15:07:02.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5155999999999995e-05 2020-08-07T15:07:02.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:02.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.334126+0000", 2020-08-07T15:07:02.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033146100000000002 2020-08-07T15:07:02.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:07:02.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.334457+0000", 2020-08-07T15:07:02.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029586479999999999 2020-08-07T15:07:02.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:02.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.337416+0000", 2020-08-07T15:07:02.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9100000000000004e-07 2020-08-07T15:07:02.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:02.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.337416+0000", 2020-08-07T15:07:02.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1066999999999997e-05 2020-08-07T15:07:02.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:02.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.337468+0000", 2020-08-07T15:07:02.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033366189999999999 2020-08-07T15:07:02.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.340804+0000", 2020-08-07T15:07:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116648549 2020-08-07T15:07:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.457453+0000", 2020-08-07T15:07:02.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1850000000000001e-06 2020-08-07T15:07:02.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:02.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.457454+0000", 2020-08-07T15:07:02.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9281000000000003e-05 2020-08-07T15:07:02.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:02.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.457523+0000", 2020-08-07T15:07:02.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12782907299999999 2020-08-07T15:07:02.889 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.889 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:02.889 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.891 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:27408 3.5 3:ab4f36a5:::benchmark_data_smithi074_35447_object3425:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:02.891 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:54.329798+0000", 2020-08-07T15:07:02.891 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.1781453390000003, 2020-08-07T15:07:02.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.127800106, 2020-08-07T15:07:02.892 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:02.892 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:02.892 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:02.892 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:02.893 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:02.893 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 27408 2020-08-07T15:07:02.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.893 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:02.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.329798+0000", 2020-08-07T15:07:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.329798+0000", 2020-08-07T15:07:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5600999999999998e-05 2020-08-07T15:07:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:02.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.329854+0000", 2020-08-07T15:07:02.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3260000000000003e-06 2020-08-07T15:07:02.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:02.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.329859+0000", 2020-08-07T15:07:02.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0332e-05 2020-08-07T15:07:02.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:02.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.329870+0000", 2020-08-07T15:07:02.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4900000000000002e-06 2020-08-07T15:07:02.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:02.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.329874+0000", 2020-08-07T15:07:02.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059293100000000001 2020-08-07T15:07:02.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:02.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.335803+0000", 2020-08-07T15:07:02.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0387e-05 2020-08-07T15:07:02.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:02.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.335894+0000", 2020-08-07T15:07:02.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019818900000000001 2020-08-07T15:07:02.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:07:02.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.336092+0000", 2020-08-07T15:07:02.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001380448 2020-08-07T15:07:02.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:02.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.337472+0000", 2020-08-07T15:07:02.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.77e-07 2020-08-07T15:07:02.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:02.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.337473+0000", 2020-08-07T15:07:02.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8547999999999998e-05 2020-08-07T15:07:02.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:02.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.337501+0000", 2020-08-07T15:07:02.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033325920000000001 2020-08-07T15:07:02.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.340834+0000", 2020-08-07T15:07:02.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116723147 2020-08-07T15:07:02.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.457557+0000", 2020-08-07T15:07:02.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9899999999999999e-07 2020-08-07T15:07:02.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:02.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.457558+0000", 2020-08-07T15:07:02.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0460000000000002e-05 2020-08-07T15:07:02.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:02.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.457598+0000", 2020-08-07T15:07:02.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.127800106 2020-08-07T15:07:02.912 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.913 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:02.913 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.913 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:32096 3.d e30/20 3:b7dba9f8:::benchmark_data_smithi074_35447_object4011:head v 30'2152, mlcod=30'2143)", 2020-08-07T15:07:02.914 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:57.198961+0000", 2020-08-07T15:07:02.914 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.3089830769999997, 2020-08-07T15:07:02.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18294507300000001, 2020-08-07T15:07:02.914 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:02.914 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:02.915 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:02.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:02.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.198961+0000", 2020-08-07T15:07:02.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:02.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:02.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.198961+0000", 2020-08-07T15:07:02.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0689999999999997e-05 2020-08-07T15:07:02.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:02.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.199011+0000", 2020-08-07T15:07:02.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7429999999999996e-06 2020-08-07T15:07:02.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:02.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.199018+0000", 2020-08-07T15:07:02.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18124368699999999 2020-08-07T15:07:02.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:02.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.380262+0000", 2020-08-07T15:07:02.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2109999999999997e-06 2020-08-07T15:07:02.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:02.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.380267+0000", 2020-08-07T15:07:02.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8856000000000002e-05 2020-08-07T15:07:02.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:02.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.380306+0000", 2020-08-07T15:07:02.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3572999999999999e-05 2020-08-07T15:07:02.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:02.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.380329+0000", 2020-08-07T15:07:02.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001541749 2020-08-07T15:07:02.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:02.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.381871+0000", 2020-08-07T15:07:02.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4564000000000003e-05 2020-08-07T15:07:02.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:02.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.381906+0000", 2020-08-07T15:07:02.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18294507300000001 2020-08-07T15:07:02.925 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.926 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:02.926 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.927 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:33216 3.d e30/20 3:b86184aa:::benchmark_data_smithi074_35447_object4151:head v 30'2232, mlcod=30'2222)", 2020-08-07T15:07:02.927 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:57.864608+0000", 2020-08-07T15:07:02.927 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.643335972, 2020-08-07T15:07:02.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15879491600000001, 2020-08-07T15:07:02.927 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:02.927 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:02.928 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:02.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:02.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.864608+0000", 2020-08-07T15:07:02.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:02.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:02.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.864608+0000", 2020-08-07T15:07:02.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7546000000000002e-05 2020-08-07T15:07:02.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:02.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.864655+0000", 2020-08-07T15:07:02.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2410000000000003e-06 2020-08-07T15:07:02.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.864661+0000", 2020-08-07T15:07:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.157425711 2020-08-07T15:07:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.022086+0000", 2020-08-07T15:07:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4450000000000004e-06 2020-08-07T15:07:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:02.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.022092+0000", 2020-08-07T15:07:02.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3872000000000003e-05 2020-08-07T15:07:02.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:02.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.022136+0000", 2020-08-07T15:07:02.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3927999999999999e-05 2020-08-07T15:07:02.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:02.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.022159+0000", 2020-08-07T15:07:02.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011814760000000001 2020-08-07T15:07:02.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:02.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023341+0000", 2020-08-07T15:07:02.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1697000000000002e-05 2020-08-07T15:07:02.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:02.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023403+0000", 2020-08-07T15:07:02.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15879491600000001 2020-08-07T15:07:02.938 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.938 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:02.938 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.939 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:33488 3.c 3:3d1929c7:::benchmark_data_smithi074_35447_object4185:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-08-07T15:07:02.939 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:58.021233+0000", 2020-08-07T15:07:02.939 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.4867108829999998, 2020-08-07T15:07:02.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.2237055, 2020-08-07T15:07:02.940 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:02.940 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:02.940 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:02.940 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:02.941 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:02.941 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 33488 2020-08-07T15:07:02.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.942 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:02.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:02.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.021233+0000", 2020-08-07T15:07:02.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:02.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:02.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.021233+0000", 2020-08-07T15:07:02.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7682000000000002e-05 2020-08-07T15:07:02.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:02.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.021281+0000", 2020-08-07T15:07:02.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.955e-06 2020-08-07T15:07:02.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:02.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.021285+0000", 2020-08-07T15:07:02.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2132e-05 2020-08-07T15:07:02.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:02.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.021298+0000", 2020-08-07T15:07:02.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1390000000000003e-06 2020-08-07T15:07:02.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:02.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.021303+0000", 2020-08-07T15:07:02.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011846789999999999 2020-08-07T15:07:02.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:02.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.022487+0000", 2020-08-07T15:07:02.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3844000000000002e-05 2020-08-07T15:07:02.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:02.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.022541+0000", 2020-08-07T15:07:02.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013274700000000001 2020-08-07T15:07:02.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:07:02.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.022674+0000", 2020-08-07T15:07:02.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00161185 2020-08-07T15:07:02.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:02.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.024286+0000", 2020-08-07T15:07:02.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8800000000000003e-07 2020-08-07T15:07:02.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:02.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.024286+0000", 2020-08-07T15:07:02.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8810999999999997e-05 2020-08-07T15:07:02.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:02.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.024335+0000", 2020-08-07T15:07:02.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025354919999999999 2020-08-07T15:07:02.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.026871+0000", 2020-08-07T15:07:02.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21800207099999999 2020-08-07T15:07:02.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:02.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.244873+0000", 2020-08-07T15:07:02.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2279999999999999e-06 2020-08-07T15:07:02.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:02.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.244874+0000", 2020-08-07T15:07:02.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4381999999999994e-05 2020-08-07T15:07:02.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:02.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:02.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:02.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.244938+0000", 2020-08-07T15:07:02.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.2237055 2020-08-07T15:07:02.961 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.961 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:02.961 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.961 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:02.962 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:02.962 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:02.963 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:02.962+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:02.964 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:07:03.064 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:03.063+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:03.164 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:03.211 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:07:03.212 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2020-08-07T15:07:03.212 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:03.213 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:03.165+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:03.214 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:07:03.244 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 27 16 42111 42095 12.1793 13.1484 0.00372705 0.0102535 2020-08-07T15:07:03.283 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:03.273+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:03.380 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:03.376+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:03.449 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:03.450 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:07:03.450 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:07:03.450 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:07:03.450 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:03.462 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:07:03.508 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:03.502+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:03.569 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:03.568+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:03.671 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:03.669+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:03.679 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:03.679 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:07:03.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:07:03.681 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:03.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.682 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:8128 3.a 3:507f42e1:::benchmark_data_smithi074_35447_object1015:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:03.682 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.093056+0000", 2020-08-07T15:07:03.683 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.577914829000001, 2020-08-07T15:07:03.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.112960949, 2020-08-07T15:07:03.683 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:03.684 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:03.684 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:03.684 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:03.685 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:03.685 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 8128 2020-08-07T15:07:03.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.685 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:03.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:03.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.093056+0000", 2020-08-07T15:07:03.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:03.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:03.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.093056+0000", 2020-08-07T15:07:03.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0308e-05 2020-08-07T15:07:03.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:03.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.093107+0000", 2020-08-07T15:07:03.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2020-08-07T15:07:03.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:03.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.093113+0000", 2020-08-07T15:07:03.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6810999999999999e-05 2020-08-07T15:07:03.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:03.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.093130+0000", 2020-08-07T15:07:03.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1509999999999998e-06 2020-08-07T15:07:03.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:03.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.093137+0000", 2020-08-07T15:07:03.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010854790000000001 2020-08-07T15:07:03.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:03.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.094223+0000", 2020-08-07T15:07:03.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2392999999999997e-05 2020-08-07T15:07:03.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:03.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.094285+0000", 2020-08-07T15:07:03.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113342 2020-08-07T15:07:03.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:03.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.094398+0000", 2020-08-07T15:07:03.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033380229999999999 2020-08-07T15:07:03.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:03.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.097736+0000", 2020-08-07T15:07:03.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028459299999999998 2020-08-07T15:07:03.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:03.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.098021+0000", 2020-08-07T15:07:03.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.107933987 2020-08-07T15:07:03.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:03.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.205955+0000", 2020-08-07T15:07:03.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1500000000000004e-07 2020-08-07T15:07:03.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:03.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.205956+0000", 2020-08-07T15:07:03.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5666e-05 2020-08-07T15:07:03.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:03.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.205991+0000", 2020-08-07T15:07:03.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6173000000000002e-05 2020-08-07T15:07:03.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:03.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.206017+0000", 2020-08-07T15:07:03.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.112960949 2020-08-07T15:07:03.710 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.711 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:03.711 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.712 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:8367 3.a 3:5e6bacde:::benchmark_data_smithi074_35447_object1045:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:03.712 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.276619+0000", 2020-08-07T15:07:03.712 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.394352390000002, 2020-08-07T15:07:03.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.141821844, 2020-08-07T15:07:03.723 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:03.723 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:03.723 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:03.724 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:03.724 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:03.724 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 8367 2020-08-07T15:07:03.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.725 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:03.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:03.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276619+0000", 2020-08-07T15:07:03.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:03.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:03.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276619+0000", 2020-08-07T15:07:03.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4978e-05 2020-08-07T15:07:03.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:03.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276644+0000", 2020-08-07T15:07:03.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3009999999999998e-06 2020-08-07T15:07:03.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:03.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276647+0000", 2020-08-07T15:07:03.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0319999999999998e-06 2020-08-07T15:07:03.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:03.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276654+0000", 2020-08-07T15:07:03.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7199999999999998e-06 2020-08-07T15:07:03.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:03.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276657+0000", 2020-08-07T15:07:03.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023605660000000001 2020-08-07T15:07:03.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:03.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.279017+0000", 2020-08-07T15:07:03.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0431000000000004e-05 2020-08-07T15:07:03.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:03.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.279088+0000", 2020-08-07T15:07:03.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016790300000000001 2020-08-07T15:07:03.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:03.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.279256+0000", 2020-08-07T15:07:03.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031354550000000001 2020-08-07T15:07:03.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:03.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.282391+0000", 2020-08-07T15:07:03.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9900000000000001e-07 2020-08-07T15:07:03.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:03.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.282392+0000", 2020-08-07T15:07:03.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5717999999999999e-05 2020-08-07T15:07:03.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:03.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.282427+0000", 2020-08-07T15:07:03.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00053778099999999998 2020-08-07T15:07:03.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:03.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.282965+0000", 2020-08-07T15:07:03.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13540729400000001 2020-08-07T15:07:03.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:03.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418372+0000", 2020-08-07T15:07:03.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9600000000000008e-07 2020-08-07T15:07:03.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:03.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418373+0000", 2020-08-07T15:07:03.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7269999999999998e-05 2020-08-07T15:07:03.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:03.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418441+0000", 2020-08-07T15:07:03.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.141821844 2020-08-07T15:07:03.748 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.748 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:03.749 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.750 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:8368 3.a 3:5e6bacde:::benchmark_data_smithi074_35447_object1045:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:03.750 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.276795+0000", 2020-08-07T15:07:03.750 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.394176664, 2020-08-07T15:07:03.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14181318200000001, 2020-08-07T15:07:03.750 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:03.751 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:03.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:03.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:03.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:03.752 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 8368 2020-08-07T15:07:03.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.752 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:03.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:03.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276795+0000", 2020-08-07T15:07:03.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:03.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:03.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276795+0000", 2020-08-07T15:07:03.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8793e-05 2020-08-07T15:07:03.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:03.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276823+0000", 2020-08-07T15:07:03.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3890000000000001e-06 2020-08-07T15:07:03.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:03.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276827+0000", 2020-08-07T15:07:03.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2590000000000003e-06 2020-08-07T15:07:03.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:03.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276834+0000", 2020-08-07T15:07:03.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7889999999999999e-06 2020-08-07T15:07:03.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:03.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276837+0000", 2020-08-07T15:07:03.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030534199999999998 2020-08-07T15:07:03.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:03.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.279890+0000", 2020-08-07T15:07:03.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3449000000000006e-05 2020-08-07T15:07:03.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:03.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.279984+0000", 2020-08-07T15:07:03.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020054799999999999 2020-08-07T15:07:03.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:03.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.280184+0000", 2020-08-07T15:07:03.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002684353 2020-08-07T15:07:03.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:03.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.282869+0000", 2020-08-07T15:07:03.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7899999999999998e-07 2020-08-07T15:07:03.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:03.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.282869+0000", 2020-08-07T15:07:03.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5290000000000001e-05 2020-08-07T15:07:03.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:03.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.282924+0000", 2020-08-07T15:07:03.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031297999999999997 2020-08-07T15:07:03.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:03.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.283237+0000", 2020-08-07T15:07:03.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.135332799 2020-08-07T15:07:03.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:03.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418570+0000", 2020-08-07T15:07:03.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4600000000000004e-07 2020-08-07T15:07:03.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:03.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418571+0000", 2020-08-07T15:07:03.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6888000000000003e-05 2020-08-07T15:07:03.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:03.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418608+0000", 2020-08-07T15:07:03.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14181318200000001 2020-08-07T15:07:03.774 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.774 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:03.774 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.775 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:8367 3.a e24/20 ondisk, result = 0)", 2020-08-07T15:07:03.775 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.281490+0000", 2020-08-07T15:07:03.776 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.389481248999999, 2020-08-07T15:07:03.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13703254000000001, 2020-08-07T15:07:03.776 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:03.776 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:03.777 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:03.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:03.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.281490+0000", 2020-08-07T15:07:03.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:03.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:03.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.281490+0000", 2020-08-07T15:07:03.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5865000000000001e-05 2020-08-07T15:07:03.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:03.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.281506+0000", 2020-08-07T15:07:03.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5269999999999999e-06 2020-08-07T15:07:03.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:03.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.281509+0000", 2020-08-07T15:07:03.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13566299600000001 2020-08-07T15:07:03.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:03.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.417172+0000", 2020-08-07T15:07:03.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7019999999999998e-06 2020-08-07T15:07:03.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:03.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.417178+0000", 2020-08-07T15:07:03.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001176947 2020-08-07T15:07:03.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:03.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418355+0000", 2020-08-07T15:07:03.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6240000000000008e-06 2020-08-07T15:07:03.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:03.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418364+0000", 2020-08-07T15:07:03.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015887900000000001 2020-08-07T15:07:03.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:03.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418523+0000", 2020-08-07T15:07:03.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13703254000000001 2020-08-07T15:07:03.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.790 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:03.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.791 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:10480 3.d e24/20 3:b5817408:::benchmark_data_smithi074_35447_object1309:head v 24'624, mlcod=24'618)", 2020-08-07T15:07:03.791 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.530172+0000", 2020-08-07T15:07:03.792 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.140799125000001, 2020-08-07T15:07:03.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12942057800000001, 2020-08-07T15:07:03.792 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:03.792 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:03.793 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:03.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:03.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.530172+0000", 2020-08-07T15:07:03.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:03.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:03.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.530172+0000", 2020-08-07T15:07:03.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2287999999999999e-05 2020-08-07T15:07:03.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:03.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.530214+0000", 2020-08-07T15:07:03.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1789999999999996e-06 2020-08-07T15:07:03.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:03.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.530221+0000", 2020-08-07T15:07:03.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.124635727 2020-08-07T15:07:03.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:03.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.654856+0000", 2020-08-07T15:07:03.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6859999999999999e-06 2020-08-07T15:07:03.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:03.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.654860+0000", 2020-08-07T15:07:03.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5784999999999997e-05 2020-08-07T15:07:03.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:03.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.654896+0000", 2020-08-07T15:07:03.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1804e-05 2020-08-07T15:07:03.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:03.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.654918+0000", 2020-08-07T15:07:03.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004623032 2020-08-07T15:07:03.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:03.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.659541+0000", 2020-08-07T15:07:03.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2077000000000003e-05 2020-08-07T15:07:03.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:03.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.659593+0000", 2020-08-07T15:07:03.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12942057800000001 2020-08-07T15:07:03.807 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.807 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:03.807 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.808 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:10536 3.0 e24/20 3:0d011956:::benchmark_data_smithi074_35447_object1316:head v 24'744, mlcod=24'735)", 2020-08-07T15:07:03.809 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.564533+0000", 2020-08-07T15:07:03.809 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.106438742000002, 2020-08-07T15:07:03.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14762919499999999, 2020-08-07T15:07:03.810 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:03.810 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:03.810 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:03.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:03.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.564533+0000", 2020-08-07T15:07:03.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:03.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:03.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.564533+0000", 2020-08-07T15:07:03.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9255000000000002e-05 2020-08-07T15:07:03.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:03.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.564582+0000", 2020-08-07T15:07:03.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1309999999999997e-06 2020-08-07T15:07:03.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:03.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.564587+0000", 2020-08-07T15:07:03.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14241859400000001 2020-08-07T15:07:03.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:03.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.707006+0000", 2020-08-07T15:07:03.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2190000000000003e-06 2020-08-07T15:07:03.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:03.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.707011+0000", 2020-08-07T15:07:03.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7673999999999999e-05 2020-08-07T15:07:03.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:03.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.707048+0000", 2020-08-07T15:07:03.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7660999999999999e-05 2020-08-07T15:07:03.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:03.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.707066+0000", 2020-08-07T15:07:03.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050458969999999997 2020-08-07T15:07:03.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:03.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.712112+0000", 2020-08-07T15:07:03.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9764e-05 2020-08-07T15:07:03.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:03.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.712162+0000", 2020-08-07T15:07:03.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14762919499999999 2020-08-07T15:07:03.825 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.825 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:03.825 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.826 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:10952 3.d e24/20 3:b0c27f34:::benchmark_data_smithi074_35447_object1368:head v 24'656, mlcod=24'650)", 2020-08-07T15:07:03.827 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.818028+0000", 2020-08-07T15:07:03.827 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.852943031999999, 2020-08-07T15:07:03.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12391846199999999, 2020-08-07T15:07:03.827 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:03.828 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:03.828 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:03.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:03.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.818028+0000", 2020-08-07T15:07:03.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:03.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:03.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.818028+0000", 2020-08-07T15:07:03.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3337000000000001e-05 2020-08-07T15:07:03.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:03.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.818082+0000", 2020-08-07T15:07:03.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3339999999999998e-06 2020-08-07T15:07:03.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:03.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.818088+0000", 2020-08-07T15:07:03.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12297433100000001 2020-08-07T15:07:03.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:03.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.941062+0000", 2020-08-07T15:07:03.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1030000000000001e-06 2020-08-07T15:07:03.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:03.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.941067+0000", 2020-08-07T15:07:03.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0878999999999999e-05 2020-08-07T15:07:03.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:03.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.941108+0000", 2020-08-07T15:07:03.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3108999999999998e-05 2020-08-07T15:07:03.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:03.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.941131+0000", 2020-08-07T15:07:03.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00076055800000000005 2020-08-07T15:07:03.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:03.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.941892+0000", 2020-08-07T15:07:03.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4811000000000001e-05 2020-08-07T15:07:03.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:03.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.941947+0000", 2020-08-07T15:07:03.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12391846199999999 2020-08-07T15:07:03.841 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.842 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:03.842 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.843 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:11192 3.d e24/20 3:bb990748:::benchmark_data_smithi074_35447_object1398:head v 24'680, mlcod=24'671)", 2020-08-07T15:07:03.843 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:43.006852+0000", 2020-08-07T15:07:03.843 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.664119566, 2020-08-07T15:07:03.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15926784399999999, 2020-08-07T15:07:03.844 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:03.844 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:03.844 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:03.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:03.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.006852+0000", 2020-08-07T15:07:03.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:03.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:03.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.006852+0000", 2020-08-07T15:07:03.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0237e-05 2020-08-07T15:07:03.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:03.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.006902+0000", 2020-08-07T15:07:03.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2750000000000001e-06 2020-08-07T15:07:03.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:03.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.006908+0000", 2020-08-07T15:07:03.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15839128599999999 2020-08-07T15:07:03.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:03.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.165300+0000", 2020-08-07T15:07:03.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4450000000000004e-06 2020-08-07T15:07:03.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:03.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.165305+0000", 2020-08-07T15:07:03.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3010999999999999e-05 2020-08-07T15:07:03.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:03.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.165348+0000", 2020-08-07T15:07:03.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3289e-05 2020-08-07T15:07:03.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:03.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.165371+0000", 2020-08-07T15:07:03.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00069550299999999998 2020-08-07T15:07:03.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:03.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.166067+0000", 2020-08-07T15:07:03.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2797999999999998e-05 2020-08-07T15:07:03.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:03.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.166120+0000", 2020-08-07T15:07:03.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15926784399999999 2020-08-07T15:07:03.859 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.859 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:03.859 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.861 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:11222 3.a 3:54e517d0:::benchmark_data_smithi074_35447_object1402:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:03.861 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:43.020766+0000", 2020-08-07T15:07:03.861 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.650205507999999, 2020-08-07T15:07:03.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.110197616, 2020-08-07T15:07:03.862 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:03.862 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:03.862 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:03.863 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:03.863 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:03.863 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11222 2020-08-07T15:07:03.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.864 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:03.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:03.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.020766+0000", 2020-08-07T15:07:03.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:03.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:03.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.020766+0000", 2020-08-07T15:07:03.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8727999999999998e-05 2020-08-07T15:07:03.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:03.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.020815+0000", 2020-08-07T15:07:03.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4589999999999997e-06 2020-08-07T15:07:03.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:03.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.020820+0000", 2020-08-07T15:07:03.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0361e-05 2020-08-07T15:07:03.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:03.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.020830+0000", 2020-08-07T15:07:03.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.023e-06 2020-08-07T15:07:03.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:03.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.020835+0000", 2020-08-07T15:07:03.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029246659999999998 2020-08-07T15:07:03.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:03.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.023760+0000", 2020-08-07T15:07:03.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104319 2020-08-07T15:07:03.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:03.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.023864+0000", 2020-08-07T15:07:03.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017478599999999999 2020-08-07T15:07:03.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:03.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.024039+0000", 2020-08-07T15:07:03.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023881549999999999 2020-08-07T15:07:03.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:03.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.026427+0000", 2020-08-07T15:07:03.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0900000000000002e-07 2020-08-07T15:07:03.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:03.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.026428+0000", 2020-08-07T15:07:03.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0584999999999997e-05 2020-08-07T15:07:03.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:03.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.026488+0000", 2020-08-07T15:07:03.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010846299999999999 2020-08-07T15:07:03.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:03.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.026597+0000", 2020-08-07T15:07:03.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.10430205100000001 2020-08-07T15:07:03.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:03.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.130899+0000", 2020-08-07T15:07:03.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8700000000000004e-07 2020-08-07T15:07:03.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:03.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.130900+0000", 2020-08-07T15:07:03.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3623999999999999e-05 2020-08-07T15:07:03.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:03.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.130963+0000", 2020-08-07T15:07:03.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.110197616 2020-08-07T15:07:03.886 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.886 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:03.887 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.887 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:11223 3.a 3:54e517d0:::benchmark_data_smithi074_35447_object1402:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:03.888 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:43.021174+0000", 2020-08-07T15:07:03.888 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.649797774, 2020-08-07T15:07:03.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.109894164, 2020-08-07T15:07:03.888 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:03.889 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:03.889 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:03.889 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:03.890 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:03.890 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11223 2020-08-07T15:07:03.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.891 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:03.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:03.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.021174+0000", 2020-08-07T15:07:03.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:03.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:03.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.021174+0000", 2020-08-07T15:07:03.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8959000000000003e-05 2020-08-07T15:07:03.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:03.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.021222+0000", 2020-08-07T15:07:03.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6489999999999996e-06 2020-08-07T15:07:03.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:03.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.021228+0000", 2020-08-07T15:07:03.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1636e-05 2020-08-07T15:07:03.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:03.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.021240+0000", 2020-08-07T15:07:03.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000001e-06 2020-08-07T15:07:03.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:03.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.021245+0000", 2020-08-07T15:07:03.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036685450000000001 2020-08-07T15:07:03.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:03.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.024913+0000", 2020-08-07T15:07:03.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107681 2020-08-07T15:07:03.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:03.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.025021+0000", 2020-08-07T15:07:03.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017933200000000001 2020-08-07T15:07:03.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:03.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.025200+0000", 2020-08-07T15:07:03.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013008480000000001 2020-08-07T15:07:03.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:03.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.026501+0000", 2020-08-07T15:07:03.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9100000000000003e-07 2020-08-07T15:07:03.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:03.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.026502+0000", 2020-08-07T15:07:03.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7587999999999998e-05 2020-08-07T15:07:03.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:03.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.026559+0000", 2020-08-07T15:07:03.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0090730819999999997 2020-08-07T15:07:03.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:03.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.035632+0000", 2020-08-07T15:07:03.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.095375096000000006 2020-08-07T15:07:03.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:03.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.131007+0000", 2020-08-07T15:07:03.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3099999999999998e-07 2020-08-07T15:07:03.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:03.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.131008+0000", 2020-08-07T15:07:03.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9655999999999999e-05 2020-08-07T15:07:03.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:03.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.131068+0000", 2020-08-07T15:07:03.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.109894164 2020-08-07T15:07:03.919 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.920 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:03.920 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.921 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:17016 3.d e27/20 3:b0cf8e92:::benchmark_data_smithi074_35447_object2126:head v 27'1088, mlcod=27'1080)", 2020-08-07T15:07:03.922 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:47.662899+0000", 2020-08-07T15:07:03.922 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.008071861000001, 2020-08-07T15:07:03.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13459602200000001, 2020-08-07T15:07:03.923 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:03.923 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:03.923 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:03.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:03.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.662899+0000", 2020-08-07T15:07:03.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:03.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:03.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.662899+0000", 2020-08-07T15:07:03.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1554000000000002e-05 2020-08-07T15:07:03.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:03.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.662941+0000", 2020-08-07T15:07:03.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1989999999999999e-06 2020-08-07T15:07:03.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:03.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.662945+0000", 2020-08-07T15:07:03.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13278101 2020-08-07T15:07:03.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:03.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.795726+0000", 2020-08-07T15:07:03.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1690000000000002e-06 2020-08-07T15:07:03.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:03.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.795730+0000", 2020-08-07T15:07:03.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8744e-05 2020-08-07T15:07:03.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:03.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.795769+0000", 2020-08-07T15:07:03.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5993999999999999e-05 2020-08-07T15:07:03.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:03.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.795795+0000", 2020-08-07T15:07:03.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016482350000000001 2020-08-07T15:07:03.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:03.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.797443+0000", 2020-08-07T15:07:03.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2117000000000003e-05 2020-08-07T15:07:03.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:03.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.797495+0000", 2020-08-07T15:07:03.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13459602200000001 2020-08-07T15:07:03.941 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.941 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:03.941 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.942 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:17824 3.d e27/20 3:bd4e0935:::benchmark_data_smithi074_35447_object2227:head v 27'1168, mlcod=27'1163)", 2020-08-07T15:07:03.942 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:07:03.942 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.527531049, 2020-08-07T15:07:03.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25249224199999998, 2020-08-07T15:07:03.943 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:03.943 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:03.943 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:03.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:03.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:07:03.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:03.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:03.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:07:03.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9348000000000003e-05 2020-08-07T15:07:03.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:03.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143490+0000", 2020-08-07T15:07:03.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7800000000000003e-06 2020-08-07T15:07:03.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:03.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143496+0000", 2020-08-07T15:07:03.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25095783399999999 2020-08-07T15:07:03.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:03.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394454+0000", 2020-08-07T15:07:03.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6550000000000001e-06 2020-08-07T15:07:03.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:03.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394460+0000", 2020-08-07T15:07:03.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8814999999999999e-05 2020-08-07T15:07:03.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:03.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394509+0000", 2020-08-07T15:07:03.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3482999999999999e-05 2020-08-07T15:07:03.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:03.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394532+0000", 2020-08-07T15:07:03.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013496199999999999 2020-08-07T15:07:03.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:03.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.395882+0000", 2020-08-07T15:07:03.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0707000000000001e-05 2020-08-07T15:07:03.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:03.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.395932+0000", 2020-08-07T15:07:03.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25249224199999998 2020-08-07T15:07:03.960 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.960 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:03.961 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.961 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:18248 3.d e27/20 3:bfdc8e53:::benchmark_data_smithi074_35447_object2280:head v 27'1176, mlcod=27'1169)", 2020-08-07T15:07:03.961 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.399823+0000", 2020-08-07T15:07:03.962 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.271147993, 2020-08-07T15:07:03.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.129150182, 2020-08-07T15:07:03.962 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:03.962 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:03.962 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:03.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:03.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.399823+0000", 2020-08-07T15:07:03.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:03.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:03.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.399823+0000", 2020-08-07T15:07:03.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1798000000000003e-05 2020-08-07T15:07:03.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:03.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.399855+0000", 2020-08-07T15:07:03.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3220000000000003e-06 2020-08-07T15:07:03.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:03.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.399859+0000", 2020-08-07T15:07:03.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12722576199999999 2020-08-07T15:07:03.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:03.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.527085+0000", 2020-08-07T15:07:03.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0540000000000002e-06 2020-08-07T15:07:03.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:03.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.527090+0000", 2020-08-07T15:07:03.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9208999999999997e-05 2020-08-07T15:07:03.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:03.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.527129+0000", 2020-08-07T15:07:03.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2308000000000001e-05 2020-08-07T15:07:03.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:03.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.527152+0000", 2020-08-07T15:07:03.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001769673 2020-08-07T15:07:03.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:03.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.528921+0000", 2020-08-07T15:07:03.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2055999999999998e-05 2020-08-07T15:07:03.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:03.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.528973+0000", 2020-08-07T15:07:03.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.129150182 2020-08-07T15:07:03.973 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.973 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:03.973 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.974 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:18478 3.a 3:524412cd:::benchmark_data_smithi074_35447_object2309:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:03.974 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.592837+0000", 2020-08-07T15:07:03.974 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.07813397, 2020-08-07T15:07:03.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21005600699999999, 2020-08-07T15:07:03.975 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:03.975 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:03.975 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:03.975 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:03.976 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:03.976 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 18478 2020-08-07T15:07:03.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.976 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:03.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:03.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592837+0000", 2020-08-07T15:07:03.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:03.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:03.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592837+0000", 2020-08-07T15:07:03.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6718999999999998e-05 2020-08-07T15:07:03.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:03.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592884+0000", 2020-08-07T15:07:03.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2749999999999997e-06 2020-08-07T15:07:03.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:03.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592888+0000", 2020-08-07T15:07:03.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0831e-05 2020-08-07T15:07:03.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:03.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592899+0000", 2020-08-07T15:07:03.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9440000000000002e-06 2020-08-07T15:07:03.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:03.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592905+0000", 2020-08-07T15:07:03.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00088115000000000003 2020-08-07T15:07:03.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:03.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.593786+0000", 2020-08-07T15:07:03.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1789000000000002e-05 2020-08-07T15:07:03.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:03.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.593838+0000", 2020-08-07T15:07:03.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011097799999999999 2020-08-07T15:07:03.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:03.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.593949+0000", 2020-08-07T15:07:03.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00088680400000000004 2020-08-07T15:07:03.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:03.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.594836+0000", 2020-08-07T15:07:03.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1500000000000005e-07 2020-08-07T15:07:03.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:03.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.594836+0000", 2020-08-07T15:07:03.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9746000000000002e-05 2020-08-07T15:07:03.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:03.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.594876+0000", 2020-08-07T15:07:03.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045589070000000001 2020-08-07T15:07:03.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:03.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.599435+0000", 2020-08-07T15:07:03.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.203361244 2020-08-07T15:07:03.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:03.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802796+0000", 2020-08-07T15:07:03.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.178e-06 2020-08-07T15:07:03.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:03.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802797+0000", 2020-08-07T15:07:03.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5927000000000005e-05 2020-08-07T15:07:03.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:03.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802893+0000", 2020-08-07T15:07:03.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21005600699999999 2020-08-07T15:07:03.995 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.998 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:03.998 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:03.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:03.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:03.999 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:18478 3.a e27/20 ondisk, result = 0)", 2020-08-07T15:07:03.999 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.595603+0000", 2020-08-07T15:07:03.999 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.075368072, 2020-08-07T15:07:03.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.207367254, 2020-08-07T15:07:04.000 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:04.000 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:04.000 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:04.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:04.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.595603+0000", 2020-08-07T15:07:04.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:04.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:04.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.595603+0000", 2020-08-07T15:07:04.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8887999999999999e-05 2020-08-07T15:07:04.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:04.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.595622+0000", 2020-08-07T15:07:04.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8469999999999998e-06 2020-08-07T15:07:04.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:04.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.595626+0000", 2020-08-07T15:07:04.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20636312700000001 2020-08-07T15:07:04.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:04.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.801989+0000", 2020-08-07T15:07:04.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0889999999999999e-06 2020-08-07T15:07:04.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:04.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.801994+0000", 2020-08-07T15:07:04.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00078913099999999999 2020-08-07T15:07:04.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:04.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802783+0000", 2020-08-07T15:07:04.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8919999999999997e-06 2020-08-07T15:07:04.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:04.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802789+0000", 2020-08-07T15:07:04.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018128 2020-08-07T15:07:04.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:04.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802970+0000", 2020-08-07T15:07:04.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.207367254 2020-08-07T15:07:04.010 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:04.010 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:04.010 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:04.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.011 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:24784 3.a 3:532870eb:::benchmark_data_smithi074_35447_object3097:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e28)", 2020-08-07T15:07:04.011 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:52.713946+0000", 2020-08-07T15:07:04.011 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.957025741000001, 2020-08-07T15:07:04.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.119704095, 2020-08-07T15:07:04.012 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:04.012 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:04.012 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:04.013 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:04.013 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:04.013 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 24784 2020-08-07T15:07:04.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.014 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:04.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:04.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.713946+0000", 2020-08-07T15:07:04.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:04.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:04.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.713946+0000", 2020-08-07T15:07:04.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0874000000000001e-05 2020-08-07T15:07:04.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:04.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.713996+0000", 2020-08-07T15:07:04.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7780000000000001e-06 2020-08-07T15:07:04.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:04.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.714000+0000", 2020-08-07T15:07:04.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.003e-06 2020-08-07T15:07:04.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:04.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.714009+0000", 2020-08-07T15:07:04.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1170000000000003e-06 2020-08-07T15:07:04.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:04.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.714014+0000", 2020-08-07T15:07:04.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1779999999999997e-05 2020-08-07T15:07:04.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:04.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.714046+0000", 2020-08-07T15:07:04.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5666000000000005e-05 2020-08-07T15:07:04.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:04.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.714112+0000", 2020-08-07T15:07:04.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010418 2020-08-07T15:07:04.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:04.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.714216+0000", 2020-08-07T15:07:04.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001787921 2020-08-07T15:07:04.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:04.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.716004+0000", 2020-08-07T15:07:04.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2020-08-07T15:07:04.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:04.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.716004+0000", 2020-08-07T15:07:04.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3488e-05 2020-08-07T15:07:04.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:04.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.716058+0000", 2020-08-07T15:07:04.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.076624567000000005 2020-08-07T15:07:04.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:04.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.792682+0000", 2020-08-07T15:07:04.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040927922999999998 2020-08-07T15:07:04.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:04.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.833610+0000", 2020-08-07T15:07:04.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.102e-06 2020-08-07T15:07:04.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:04.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.833612+0000", 2020-08-07T15:07:04.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8139000000000002e-05 2020-08-07T15:07:04.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:04.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.833650+0000", 2020-08-07T15:07:04.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.119704095 2020-08-07T15:07:04.032 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:04.032 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:04.032 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:04.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.033 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:24784 3.a e28/20 ondisk, result = 0)", 2020-08-07T15:07:04.034 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:52.721435+0000", 2020-08-07T15:07:04.034 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.949536502000001, 2020-08-07T15:07:04.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.112219875, 2020-08-07T15:07:04.034 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:04.034 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:04.035 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:04.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:04.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.721435+0000", 2020-08-07T15:07:04.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:04.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:04.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.721435+0000", 2020-08-07T15:07:04.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2879999999999997e-05 2020-08-07T15:07:04.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:04.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.721468+0000", 2020-08-07T15:07:04.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0310000000000004e-06 2020-08-07T15:07:04.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:04.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.721474+0000", 2020-08-07T15:07:04.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.112077835 2020-08-07T15:07:04.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:04.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.833552+0000", 2020-08-07T15:07:04.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2020-08-07T15:07:04.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:04.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.833557+0000", 2020-08-07T15:07:04.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0399999999999999e-05 2020-08-07T15:07:04.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:04.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.833597+0000", 2020-08-07T15:07:04.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3549999999999999e-06 2020-08-07T15:07:04.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:04.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.833605+0000", 2020-08-07T15:07:04.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0135e-05 2020-08-07T15:07:04.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:04.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.833655+0000", 2020-08-07T15:07:04.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.112219875 2020-08-07T15:07:04.044 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:04.045 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:04.045 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:04.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.045 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:33488 3.c e30/20 3:3d1929c7:::benchmark_data_smithi074_35447_object4185:head v 30'2120, mlcod=30'2110)", 2020-08-07T15:07:04.046 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:58.023101+0000", 2020-08-07T15:07:04.046 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.6478702810000003, 2020-08-07T15:07:04.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22118623300000001, 2020-08-07T15:07:04.046 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:04.046 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:04.047 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:04.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:04.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023101+0000", 2020-08-07T15:07:04.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:04.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:04.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023101+0000", 2020-08-07T15:07:04.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8794000000000001e-05 2020-08-07T15:07:04.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:04.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023150+0000", 2020-08-07T15:07:04.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.545e-06 2020-08-07T15:07:04.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:04.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023156+0000", 2020-08-07T15:07:04.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.218926757 2020-08-07T15:07:04.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:04.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242083+0000", 2020-08-07T15:07:04.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1200000000000004e-06 2020-08-07T15:07:04.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:04.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242087+0000", 2020-08-07T15:07:04.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9561999999999998e-05 2020-08-07T15:07:04.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:04.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242127+0000", 2020-08-07T15:07:04.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3655999999999998e-05 2020-08-07T15:07:04.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:04.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242150+0000", 2020-08-07T15:07:04.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002089273 2020-08-07T15:07:04.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:04.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.244240+0000", 2020-08-07T15:07:04.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7525999999999999e-05 2020-08-07T15:07:04.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:04.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.244287+0000", 2020-08-07T15:07:04.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22118623300000001 2020-08-07T15:07:04.060 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:04.061 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:04.061 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:04.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.062 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:35856 3.a 3:5b5556e8:::benchmark_data_smithi074_35447_object4481:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-08-07T15:07:04.063 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:59.440060+0000", 2020-08-07T15:07:04.063 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.2309112249999998, 2020-08-07T15:07:04.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16587832199999999, 2020-08-07T15:07:04.064 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:04.064 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:04.065 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:04.065 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:04.065 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:04.065 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 35856 2020-08-07T15:07:04.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.066 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:04.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:04.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440060+0000", 2020-08-07T15:07:04.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:04.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:04.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440060+0000", 2020-08-07T15:07:04.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018524399999999999 2020-08-07T15:07:04.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:04.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440245+0000", 2020-08-07T15:07:04.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9519999999999999e-06 2020-08-07T15:07:04.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:04.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440251+0000", 2020-08-07T15:07:04.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1705999999999999e-05 2020-08-07T15:07:04.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:04.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440263+0000", 2020-08-07T15:07:04.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2820000000000004e-06 2020-08-07T15:07:04.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:04.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440268+0000", 2020-08-07T15:07:04.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023611449999999998 2020-08-07T15:07:04.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:04.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.442629+0000", 2020-08-07T15:07:04.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010644799999999999 2020-08-07T15:07:04.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:04.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.442736+0000", 2020-08-07T15:07:04.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017566000000000001 2020-08-07T15:07:04.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:04.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.442912+0000", 2020-08-07T15:07:04.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026960399999999998 2020-08-07T15:07:04.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:04.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.445608+0000", 2020-08-07T15:07:04.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0809999999999999e-06 2020-08-07T15:07:04.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:04.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.445609+0000", 2020-08-07T15:07:04.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0878999999999997e-05 2020-08-07T15:07:04.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:04.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.445680+0000", 2020-08-07T15:07:04.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00065560499999999997 2020-08-07T15:07:04.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:04.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.446335+0000", 2020-08-07T15:07:04.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.159534921 2020-08-07T15:07:04.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:04.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.605870+0000", 2020-08-07T15:07:04.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.187e-06 2020-08-07T15:07:04.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:04.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.605871+0000", 2020-08-07T15:07:04.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7172e-05 2020-08-07T15:07:04.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:04.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.605938+0000", 2020-08-07T15:07:04.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16587832199999999 2020-08-07T15:07:04.089 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:04.089 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:04.090 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:04.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.091 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:42104 3.d e32/20 3:bcc35439:::benchmark_data_smithi074_35447_object5262:head v 32'2864, mlcod=32'2858)", 2020-08-07T15:07:04.091 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:07:04.091 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.42794575200000001, 2020-08-07T15:07:04.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28296418899999998, 2020-08-07T15:07:04.092 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:04.092 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:04.092 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:04.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:04.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:07:04.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:04.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:04.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:07:04.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5550000000000002e-05 2020-08-07T15:07:04.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:04.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243081+0000", 2020-08-07T15:07:04.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8139999999999997e-06 2020-08-07T15:07:04.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:04.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243087+0000", 2020-08-07T15:07:04.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27968446800000002 2020-08-07T15:07:04.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:04.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522771+0000", 2020-08-07T15:07:04.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9459999999999997e-06 2020-08-07T15:07:04.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:04.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522776+0000", 2020-08-07T15:07:04.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0778000000000006e-05 2020-08-07T15:07:04.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:04.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522847+0000", 2020-08-07T15:07:04.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0622e-05 2020-08-07T15:07:04.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:04.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522868+0000", 2020-08-07T15:07:04.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030627580000000001 2020-08-07T15:07:04.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:04.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.525930+0000", 2020-08-07T15:07:04.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9252999999999997e-05 2020-08-07T15:07:04.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:04.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.525990+0000", 2020-08-07T15:07:04.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28296418899999998 2020-08-07T15:07:04.108 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:04.108 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:04.109 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:04.109 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:04.109 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:04.109 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:04.118 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:07:04.118 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:07:04.119 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump -f json-pretty 2020-08-07T15:07:04.121 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:07:04.122 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:04.119+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:04.222 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:04.220+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:04.243 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 28 16 43766 43750 12.206 12.9297 0.00457512 0.0102352 2020-08-07T15:07:04.338 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:07:04.339 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:04.340 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:04.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.340 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:43840 3.f 3:f2412c3c:::benchmark_data_smithi074_35447_object5479:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:07:04.340 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:04.279655+0000", 2020-08-07T15:07:04.341 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.053807768999999998, 2020-08-07T15:07:04.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.053829763000000003, 2020-08-07T15:07:04.341 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:04.341 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:04.341 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:04.344 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:04.344 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:04.344 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43840 2020-08-07T15:07:04.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.345 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:04.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:04.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.279655+0000", 2020-08-07T15:07:04.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:04.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:04.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.279655+0000", 2020-08-07T15:07:04.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0949000000000003e-05 2020-08-07T15:07:04.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:04.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.279706+0000", 2020-08-07T15:07:04.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8490000000000002e-06 2020-08-07T15:07:04.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:04.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.279712+0000", 2020-08-07T15:07:04.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2354000000000001e-05 2020-08-07T15:07:04.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:04.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.279724+0000", 2020-08-07T15:07:04.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.626e-06 2020-08-07T15:07:04.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:04.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.279730+0000", 2020-08-07T15:07:04.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0076186420000000001 2020-08-07T15:07:04.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:04.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.287349+0000", 2020-08-07T15:07:04.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3504999999999994e-05 2020-08-07T15:07:04.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:04.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.287432+0000", 2020-08-07T15:07:04.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109518 2020-08-07T15:07:04.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:04.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.287542+0000", 2020-08-07T15:07:04.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055581700000000003 2020-08-07T15:07:04.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:04.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.293100+0000", 2020-08-07T15:07:04.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1999999999999999e-07 2020-08-07T15:07:04.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:04.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.293101+0000", 2020-08-07T15:07:04.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9522000000000003e-05 2020-08-07T15:07:04.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:04.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.293170+0000", 2020-08-07T15:07:04.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039355689999999999 2020-08-07T15:07:04.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:04.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.297106+0000", 2020-08-07T15:07:04.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017450324 2020-08-07T15:07:04.360 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:04.360 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:04.360 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:04.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:43864 3.e 3:7fea21ab:::benchmark_data_smithi074_35447_object5482:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:07:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:04.299390+0000", 2020-08-07T15:07:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.034073112000000003, 2020-08-07T15:07:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034180653999999998, 2020-08-07T15:07:04.362 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:04.362 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:04.362 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:04.362 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:04.363 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:04.363 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 43864 2020-08-07T15:07:04.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.363 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:04.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.299390+0000", 2020-08-07T15:07:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:04.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.299390+0000", 2020-08-07T15:07:04.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8192000000000002e-05 2020-08-07T15:07:04.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:04.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.299438+0000", 2020-08-07T15:07:04.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0429999999999998e-06 2020-08-07T15:07:04.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:04.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.299443+0000", 2020-08-07T15:07:04.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0193999999999999e-05 2020-08-07T15:07:04.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.299453+0000", 2020-08-07T15:07:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0830000000000003e-06 2020-08-07T15:07:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:04.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.299459+0000", 2020-08-07T15:07:04.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022679800000000002 2020-08-07T15:07:04.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:04.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.301726+0000", 2020-08-07T15:07:04.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4858999999999998e-05 2020-08-07T15:07:04.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:04.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.301781+0000", 2020-08-07T15:07:04.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121928 2020-08-07T15:07:04.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:04.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.301903+0000", 2020-08-07T15:07:04.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021006029999999999 2020-08-07T15:07:04.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:04.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.304004+0000", 2020-08-07T15:07:04.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0699999999999997e-07 2020-08-07T15:07:04.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:04.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.304004+0000", 2020-08-07T15:07:04.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2941999999999997e-05 2020-08-07T15:07:04.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:04.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.304037+0000", 2020-08-07T15:07:04.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.028940059000000001 2020-08-07T15:07:04.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:04.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.332977+0000", 2020-08-07T15:07:04.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033587390000000002 2020-08-07T15:07:04.379 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:04.379 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:04.379 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:04.379 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:04.380 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:07:04.380 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 2 2020-08-07T15:07:04.380 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:04.381 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:04.338+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:04.382 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:07:04.441 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:04.439+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:04.458 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:07:04.547 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:04.545+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:04.556 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:07:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:07:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:07:04.557 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:04.566 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:07:04.592 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:07:04.592 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:04.593 INFO:teuthology.orchestra.run.smithi074.stdout: "epoch": 32, 2020-08-07T15:07:04.593 INFO:teuthology.orchestra.run.smithi074.stdout: "fsid": "a01d3e82-8d24-4288-9e42-576ab3ff4957", 2020-08-07T15:07:04.593 INFO:teuthology.orchestra.run.smithi074.stdout: "created": "2020-08-07T15:06:07.342912+0000", 2020-08-07T15:07:04.593 INFO:teuthology.orchestra.run.smithi074.stdout: "modified": "2020-08-07T15:07:02.272856+0000", 2020-08-07T15:07:04.594 INFO:teuthology.orchestra.run.smithi074.stdout: "last_up_change": "2020-08-07T15:06:17.674292+0000", 2020-08-07T15:07:04.594 INFO:teuthology.orchestra.run.smithi074.stdout: "last_in_change": "2020-08-07T15:06:17.674292+0000", 2020-08-07T15:07:04.594 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-08-07T15:07:04.594 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_num": 5806080, 2020-08-07T15:07:04.595 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_set": [ 2020-08-07T15:07:04.595 INFO:teuthology.orchestra.run.smithi074.stdout: "nodeep-scrub", 2020-08-07T15:07:04.595 INFO:teuthology.orchestra.run.smithi074.stdout: "noscrub", 2020-08-07T15:07:04.595 INFO:teuthology.orchestra.run.smithi074.stdout: "pglog_hardlimit", 2020-08-07T15:07:04.595 INFO:teuthology.orchestra.run.smithi074.stdout: "purged_snapdirs", 2020-08-07T15:07:04.595 INFO:teuthology.orchestra.run.smithi074.stdout: "recovery_deletes", 2020-08-07T15:07:04.596 INFO:teuthology.orchestra.run.smithi074.stdout: "sortbitwise" 2020-08-07T15:07:04.596 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:07:04.596 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_version": 5, 2020-08-07T15:07:04.596 INFO:teuthology.orchestra.run.smithi074.stdout: "full_ratio": 0.94999998807907104, 2020-08-07T15:07:04.596 INFO:teuthology.orchestra.run.smithi074.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-08-07T15:07:04.597 INFO:teuthology.orchestra.run.smithi074.stdout: "nearfull_ratio": 0.85000002384185791, 2020-08-07T15:07:04.597 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_snapshot": "", 2020-08-07T15:07:04.597 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_max": 3, 2020-08-07T15:07:04.597 INFO:teuthology.orchestra.run.smithi074.stdout: "max_osd": 8, 2020-08-07T15:07:04.599 INFO:teuthology.orchestra.run.smithi074.stdout: "require_min_compat_client": "luminous", 2020-08-07T15:07:04.599 INFO:teuthology.orchestra.run.smithi074.stdout: "min_compat_client": "luminous", 2020-08-07T15:07:04.599 INFO:teuthology.orchestra.run.smithi074.stdout: "require_osd_release": "pacific", 2020-08-07T15:07:04.599 INFO:teuthology.orchestra.run.smithi074.stdout: "pools": [ 2020-08-07T15:07:04.600 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.600 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 1, 2020-08-07T15:07:04.600 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "device_health_metrics", 2020-08-07T15:07:04.600 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:06:10.648876+0000", 2020-08-07T15:07:04.601 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:07:04.601 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:07:04.601 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:07:04.601 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:07:04.601 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:07:04.602 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:07:04.602 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:07:04.602 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:07:04.602 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 1, 2020-08-07T15:07:04.602 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 1, 2020-08-07T15:07:04.603 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 1, 2020-08-07T15:07:04.603 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 1, 2020-08-07T15:07:04.603 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 1, 2020-08-07T15:07:04.603 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:07:04.603 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:07:04.604 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:07:04.604 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:07:04.604 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:07:04.604 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:07:04.604 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:07:04.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.605 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "16", 2020-08-07T15:07:04.605 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:07:04.605 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:07:04.605 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "0", 2020-08-07T15:07:04.605 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:07:04.606 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:07:04.606 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:07:04.606 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:07:04.606 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:07:04.607 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:07:04.607 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:07:04.607 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:07:04.607 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:07:04.607 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:07:04.608 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:07:04.608 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:07:04.608 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:07:04.608 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:07:04.608 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:07:04.609 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:07:04.609 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:07:04.609 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:07:04.609 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:07:04.609 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:07:04.610 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:07:04.610 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:07:04.610 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:07:04.610 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.610 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:07:04.611 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:07:04.611 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:07:04.611 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:07:04.611 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:07:04.611 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:07:04.612 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:07:04.612 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:07:04.612 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:07:04.612 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:07:04.615 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:07:04.615 INFO:teuthology.orchestra.run.smithi074.stdout: "options": { 2020-08-07T15:07:04.615 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_min": 1 2020-08-07T15:07:04.616 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.616 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:07:04.617 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_devicehealth": {} 2020-08-07T15:07:04.617 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.617 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.617 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.617 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 2, 2020-08-07T15:07:04.618 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "rbd", 2020-08-07T15:07:04.618 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:06:21.474648+0000", 2020-08-07T15:07:04.618 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:07:04.618 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:07:04.618 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:07:04.619 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:07:04.619 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:07:04.621 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:07:04.621 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:07:04.621 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:07:04.621 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 8, 2020-08-07T15:07:04.621 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 8, 2020-08-07T15:07:04.622 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 8, 2020-08-07T15:07:04.622 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 8, 2020-08-07T15:07:04.622 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 8, 2020-08-07T15:07:04.622 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:07:04.622 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:07:04.623 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:07:04.623 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:07:04.623 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:07:04.623 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:07:04.623 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:07:04.623 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.624 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "19", 2020-08-07T15:07:04.624 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:07:04.624 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:07:04.626 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "0", 2020-08-07T15:07:04.626 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:07:04.627 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:07:04.627 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:07:04.628 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:07:04.628 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:07:04.628 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:07:04.629 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:07:04.629 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:07:04.629 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:07:04.630 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:07:04.630 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:07:04.630 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:07:04.630 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:07:04.630 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:07:04.632 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:07:04.632 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:07:04.632 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:07:04.632 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:07:04.633 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:07:04.633 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:07:04.633 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:07:04.633 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:07:04.633 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:07:04.634 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.634 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:07:04.634 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:07:04.634 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:07:04.634 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:07:04.635 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:07:04.635 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:07:04.635 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:07:04.635 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:07:04.635 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:07:04.636 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:07:04.636 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:07:04.636 INFO:teuthology.orchestra.run.smithi074.stdout: "options": {}, 2020-08-07T15:07:04.636 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:07:04.636 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd": {} 2020-08-07T15:07:04.636 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.637 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.637 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.637 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 3, 2020-08-07T15:07:04.637 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "unique_pool_0", 2020-08-07T15:07:04.638 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:06:33.421788+0000", 2020-08-07T15:07:04.643 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:07:04.643 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:07:04.643 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:07:04.643 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:07:04.644 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:07:04.644 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:07:04.644 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:07:04.645 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:07:04.645 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 16, 2020-08-07T15:07:04.645 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 16, 2020-08-07T15:07:04.646 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 16, 2020-08-07T15:07:04.646 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 16, 2020-08-07T15:07:04.646 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 16, 2020-08-07T15:07:04.646 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:07:04.646 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:07:04.647 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:07:04.647 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:07:04.647 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:07:04.647 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:07:04.647 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:07:04.648 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.648 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "22", 2020-08-07T15:07:04.648 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:07:04.648 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:07:04.648 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "0", 2020-08-07T15:07:04.650 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:07:04.650 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:07:04.650 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:07:04.651 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:07:04.651 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:07:04.651 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:07:04.651 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:07:04.651 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:07:04.652 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:07:04.652 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:07:04.652 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:07:04.652 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:07:04.652 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:07:04.653 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:07:04.653 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:07:04.653 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:07:04.653 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:07:04.653 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:07:04.654 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:07:04.654 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:07:04.654 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:07:04.654 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:07:04.655 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:07:04.655 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.655 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:07:04.655 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:07:04.655 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:07:04.656 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:07:04.656 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:07:04.656 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:07:04.656 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:07:04.656 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:07:04.657 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:07:04.657 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:07:04.657 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:07:04.657 INFO:teuthology.orchestra.run.smithi074.stdout: "options": {}, 2020-08-07T15:07:04.657 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:07:04.658 INFO:teuthology.orchestra.run.smithi074.stdout: "rados": {} 2020-08-07T15:07:04.658 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.658 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.658 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:07:04.658 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:07:04.659 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.659 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 0, 2020-08-07T15:07:04.659 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "fc1ebb23-3ef4-4fc0-b395-382e385885a3", 2020-08-07T15:07:04.659 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:07:04.660 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:07:04.660 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:07:04.661 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0, 2020-08-07T15:07:04.661 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:07:04.661 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:07:04.662 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:07:04.662 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 20, 2020-08-07T15:07:04.662 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:07:04.662 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:07:04.662 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:07:04.663 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.663 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.663 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.664 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6801", 2020-08-07T15:07:04.664 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:07:04.664 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.664 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.664 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.665 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:07:04.665 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.665 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.665 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.665 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6802", 2020-08-07T15:07:04.666 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:07:04.666 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.666 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.666 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.666 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:07:04.667 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.667 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.667 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6804", 2020-08-07T15:07:04.668 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:07:04.669 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.669 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.669 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:07:04.670 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.670 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.670 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.670 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6803", 2020-08-07T15:07:04.671 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:07:04.671 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.671 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.671 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.671 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6801/31577", 2020-08-07T15:07:04.672 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6802/31577", 2020-08-07T15:07:04.672 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6804/31577", 2020-08-07T15:07:04.672 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6803/31577", 2020-08-07T15:07:04.672 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:07:04.672 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:07:04.673 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:07:04.673 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.673 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.673 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.673 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 1, 2020-08-07T15:07:04.674 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "1fbe530a-63ce-4dc8-82ab-18793d6ebc0b", 2020-08-07T15:07:04.674 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:07:04.674 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:07:04.675 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:07:04.675 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0.7196044921875, 2020-08-07T15:07:04.675 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:07:04.675 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:07:04.676 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:07:04.676 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 26, 2020-08-07T15:07:04.676 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:07:04.676 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:07:04.676 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:07:04.677 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.677 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.677 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6809", 2020-08-07T15:07:04.677 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:07:04.678 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.678 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.679 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:07:04.679 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.679 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.680 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6810", 2020-08-07T15:07:04.680 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:07:04.680 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.680 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.681 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:07:04.681 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.681 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.681 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.682 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6812", 2020-08-07T15:07:04.682 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:07:04.682 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.682 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.683 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.683 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:07:04.683 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.683 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.684 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.684 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6811", 2020-08-07T15:07:04.684 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:07:04.684 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.684 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.684 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.685 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6809/31578", 2020-08-07T15:07:04.685 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6810/31578", 2020-08-07T15:07:04.685 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6812/31578", 2020-08-07T15:07:04.685 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6811/31578", 2020-08-07T15:07:04.686 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:07:04.686 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:07:04.686 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:07:04.686 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.687 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.687 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.687 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 2, 2020-08-07T15:07:04.687 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "d6fe4338-3e11-4f73-9cbb-e1b046ce286a", 2020-08-07T15:07:04.687 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:07:04.687 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:07:04.688 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:07:04.689 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:07:04.689 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:07:04.689 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:07:04.689 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:07:04.690 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 20, 2020-08-07T15:07:04.690 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:07:04.690 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:07:04.690 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:07:04.690 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.691 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.691 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.691 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6805", 2020-08-07T15:07:04.692 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:07:04.692 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.692 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.692 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:07:04.693 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.693 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.693 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.693 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6806", 2020-08-07T15:07:04.693 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:07:04.694 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.694 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.694 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.694 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:07:04.694 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.695 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.695 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6808", 2020-08-07T15:07:04.695 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:07:04.696 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.696 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.696 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:07:04.696 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.697 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.697 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6807", 2020-08-07T15:07:04.697 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:07:04.697 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.697 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.698 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6805/31576", 2020-08-07T15:07:04.698 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6806/31576", 2020-08-07T15:07:04.698 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6808/31576", 2020-08-07T15:07:04.698 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6807/31576", 2020-08-07T15:07:04.699 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:07:04.699 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:07:04.699 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:07:04.699 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.699 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.700 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 3, 2020-08-07T15:07:04.700 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9", 2020-08-07T15:07:04.700 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:07:04.700 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:07:04.701 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:07:04.701 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:07:04.701 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:07:04.701 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:07:04.701 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:07:04.701 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 20, 2020-08-07T15:07:04.702 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:07:04.702 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:07:04.702 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:07:04.702 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.702 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.703 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.703 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6813", 2020-08-07T15:07:04.703 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:07:04.703 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.703 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.704 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:07:04.704 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.704 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.704 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6814", 2020-08-07T15:07:04.705 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:07:04.705 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.705 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.705 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.705 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:07:04.706 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.706 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.706 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.706 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6816", 2020-08-07T15:07:04.706 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:07:04.707 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.707 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.707 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.707 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:07:04.707 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.708 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.708 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6815", 2020-08-07T15:07:04.708 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:07:04.708 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.708 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.709 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.709 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6813/31579", 2020-08-07T15:07:04.709 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6814/31579", 2020-08-07T15:07:04.709 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6816/31579", 2020-08-07T15:07:04.709 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6815/31579", 2020-08-07T15:07:04.710 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:07:04.710 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:07:04.710 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:07:04.710 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.711 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.711 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 4, 2020-08-07T15:07:04.711 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "01aadc13-29c5-4ddd-9bb9-de24b2b20412", 2020-08-07T15:07:04.711 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:07:04.712 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:07:04.712 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:07:04.712 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:07:04.712 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:07:04.712 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:07:04.713 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:07:04.713 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 26, 2020-08-07T15:07:04.713 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:07:04.713 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:07:04.713 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:07:04.714 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.714 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.714 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.714 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6800", 2020-08-07T15:07:04.714 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:07:04.714 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.715 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.715 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.715 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:07:04.715 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.715 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.716 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.716 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6801", 2020-08-07T15:07:04.716 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:07:04.716 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.716 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.716 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.717 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:07:04.717 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.717 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.717 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.717 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6803", 2020-08-07T15:07:04.718 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:07:04.718 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.718 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.718 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:07:04.719 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.719 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.719 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6802", 2020-08-07T15:07:04.719 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:07:04.720 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.720 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.720 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6800/31545", 2020-08-07T15:07:04.720 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6801/31545", 2020-08-07T15:07:04.720 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6803/31545", 2020-08-07T15:07:04.721 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6802/31545", 2020-08-07T15:07:04.721 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:07:04.721 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:07:04.721 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:07:04.721 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.730 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 5, 2020-08-07T15:07:04.730 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "f5556a82-73b4-47b1-8e29-1d2fe031d8fd", 2020-08-07T15:07:04.731 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:07:04.731 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:07:04.731 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:07:04.731 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:07:04.731 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:07:04.732 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:07:04.732 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:07:04.732 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 20, 2020-08-07T15:07:04.732 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:07:04.732 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:07:04.733 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:07:04.733 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.733 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.733 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6804", 2020-08-07T15:07:04.734 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:07:04.734 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.734 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.734 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.734 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:07:04.735 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.735 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.735 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.735 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6805", 2020-08-07T15:07:04.735 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:07:04.736 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.736 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.736 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:07:04.736 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.736 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.737 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.737 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6807", 2020-08-07T15:07:04.737 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:07:04.737 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.737 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.738 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.738 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:07:04.738 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.738 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.739 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.739 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6806", 2020-08-07T15:07:04.739 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:07:04.739 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.739 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.740 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.740 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6804/31547", 2020-08-07T15:07:04.740 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6805/31547", 2020-08-07T15:07:04.740 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6807/31547", 2020-08-07T15:07:04.740 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6806/31547", 2020-08-07T15:07:04.741 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:07:04.741 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:07:04.741 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:07:04.741 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.741 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.742 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 6, 2020-08-07T15:07:04.742 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "9b98b337-e1cc-43cd-9e9b-f21ae3f20899", 2020-08-07T15:07:04.742 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:07:04.742 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:07:04.742 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:07:04.743 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:07:04.743 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:07:04.743 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:07:04.743 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 14, 2020-08-07T15:07:04.743 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 20, 2020-08-07T15:07:04.744 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:07:04.744 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:07:04.744 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:07:04.744 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.744 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.745 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6808", 2020-08-07T15:07:04.745 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:07:04.745 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.745 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.745 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.746 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:07:04.746 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.746 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.746 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6809", 2020-08-07T15:07:04.747 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:07:04.747 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.747 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.747 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.747 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:07:04.747 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.748 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.748 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.748 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6811", 2020-08-07T15:07:04.748 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:07:04.748 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.749 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.749 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.749 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:07:04.749 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.749 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.750 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.750 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6810", 2020-08-07T15:07:04.750 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:07:04.750 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.750 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.750 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.751 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6808/31546", 2020-08-07T15:07:04.751 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6809/31546", 2020-08-07T15:07:04.751 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6811/31546", 2020-08-07T15:07:04.751 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6810/31546", 2020-08-07T15:07:04.751 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:07:04.752 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:07:04.752 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:07:04.752 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.752 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.753 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 7, 2020-08-07T15:07:04.753 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "777662d8-bd34-48e8-83e1-24ce9e61f14b", 2020-08-07T15:07:04.753 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:07:04.753 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:07:04.753 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:07:04.753 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:07:04.754 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:07:04.754 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:07:04.754 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:07:04.754 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 26, 2020-08-07T15:07:04.754 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:07:04.755 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:07:04.755 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:07:04.755 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.755 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.755 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.756 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6812", 2020-08-07T15:07:04.756 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:07:04.756 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.756 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.756 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:07:04.757 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.757 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.757 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6813", 2020-08-07T15:07:04.757 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:07:04.758 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.758 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.758 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.758 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:07:04.758 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.759 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.759 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6815", 2020-08-07T15:07:04.759 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:07:04.759 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.759 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.760 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.760 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:07:04.760 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:07:04.760 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.760 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:07:04.761 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6814", 2020-08-07T15:07:04.761 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:07:04.761 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.761 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.761 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.762 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6812/31570", 2020-08-07T15:07:04.762 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6813/31570", 2020-08-07T15:07:04.762 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6815/31570", 2020-08-07T15:07:04.762 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6814/31570", 2020-08-07T15:07:04.762 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:07:04.762 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:07:04.763 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:07:04.763 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.763 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.763 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:07:04.763 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_xinfo": [ 2020-08-07T15:07:04.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.764 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 0, 2020-08-07T15:07:04.764 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:07:04.764 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:07:04.764 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:07:04.765 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:07:04.765 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:07:04.765 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:06:15.298063+0000", 2020-08-07T15:07:04.765 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:07:04.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.766 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 1, 2020-08-07T15:07:04.766 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:07:04.766 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:07:04.766 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:07:04.766 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:07:04.767 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:07:04.767 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:06:15.577450+0000", 2020-08-07T15:07:04.767 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:07:04.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.768 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 2, 2020-08-07T15:07:04.768 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:07:04.768 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:07:04.768 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:07:04.768 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:07:04.769 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:07:04.769 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:06:15.542874+0000", 2020-08-07T15:07:04.769 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:07:04.769 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.770 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.770 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 3, 2020-08-07T15:07:04.770 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:07:04.770 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:07:04.770 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:07:04.771 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:07:04.771 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:07:04.771 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:06:15.493720+0000", 2020-08-07T15:07:04.771 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:07:04.771 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.771 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.772 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 4, 2020-08-07T15:07:04.772 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:07:04.772 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:07:04.772 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:07:04.772 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:07:04.773 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:07:04.773 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:06:15.534281+0000", 2020-08-07T15:07:04.773 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:07:04.773 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.773 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.774 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 5, 2020-08-07T15:07:04.774 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:07:04.774 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:07:04.774 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:07:04.774 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:07:04.774 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:07:04.775 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:06:15.601875+0000", 2020-08-07T15:07:04.775 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:07:04.775 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.775 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 6, 2020-08-07T15:07:04.776 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:07:04.776 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:07:04.776 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:07:04.776 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:07:04.776 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:07:04.777 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:06:15.340718+0000", 2020-08-07T15:07:04.777 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:07:04.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.777 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 7, 2020-08-07T15:07:04.778 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:07:04.778 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:07:04.778 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:07:04.778 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:07:04.779 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:07:04.779 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:06:15.549109+0000", 2020-08-07T15:07:04.779 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:07:04.779 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.779 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:07:04.779 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_upmap": [], 2020-08-07T15:07:04.780 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_upmap_items": [ 2020-08-07T15:07:04.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.780 INFO:teuthology.orchestra.run.smithi074.stdout: "pgid": "3.7", 2020-08-07T15:07:04.780 INFO:teuthology.orchestra.run.smithi074.stdout: "mappings": [ 2020-08-07T15:07:04.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:04.781 INFO:teuthology.orchestra.run.smithi074.stdout: "from": 0, 2020-08-07T15:07:04.781 INFO:teuthology.orchestra.run.smithi074.stdout: "to": 7 2020-08-07T15:07:04.781 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.781 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:04.781 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.782 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:07:04.782 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_temp": [], 2020-08-07T15:07:04.782 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_temp": [], 2020-08-07T15:07:04.782 INFO:teuthology.orchestra.run.smithi074.stdout: "blacklist": {}, 2020-08-07T15:07:04.782 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profiles": { 2020-08-07T15:07:04.783 INFO:teuthology.orchestra.run.smithi074.stdout: "default": { 2020-08-07T15:07:04.783 INFO:teuthology.orchestra.run.smithi074.stdout: "crush-failure-domain": "osd", 2020-08-07T15:07:04.783 INFO:teuthology.orchestra.run.smithi074.stdout: "k": "2", 2020-08-07T15:07:04.783 INFO:teuthology.orchestra.run.smithi074.stdout: "m": "1", 2020-08-07T15:07:04.784 INFO:teuthology.orchestra.run.smithi074.stdout: "plugin": "jerasure", 2020-08-07T15:07:04.784 INFO:teuthology.orchestra.run.smithi074.stdout: "ruleset-failure-domain": "osd", 2020-08-07T15:07:04.784 INFO:teuthology.orchestra.run.smithi074.stdout: "technique": "reed_sol_van" 2020-08-07T15:07:04.784 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:04.785 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:04.785 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps_queue": [], 2020-08-07T15:07:04.785 INFO:teuthology.orchestra.run.smithi074.stdout: "new_removed_snaps": [], 2020-08-07T15:07:04.785 INFO:teuthology.orchestra.run.smithi074.stdout: "new_purged_snaps": [], 2020-08-07T15:07:04.785 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_node_flags": {}, 2020-08-07T15:07:04.785 INFO:teuthology.orchestra.run.smithi074.stdout: "device_class_flags": {} 2020-08-07T15:07:04.786 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:04.788 INFO:tasks.thrashosds.thrasher:j is {'epoch': 32, 'fsid': 'a01d3e82-8d24-4288-9e42-576ab3ff4957', 'created': '2020-08-07T15:06:07.342912+0000', 'modified': '2020-08-07T15:07:02.272856+0000', 'last_up_change': '2020-08-07T15:06:17.674292+0000', 'last_in_change': '2020-08-07T15:06:17.674292+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-08-07T15:06:10.648876+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-08-07T15:06:21.474648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2020-08-07T15:06:33.421788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': 'fc1ebb23-3ef4-4fc0-b395-382e385885a3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6801', 'nonce': 31577}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6802', 'nonce': 31577}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6804', 'nonce': 31577}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6803', 'nonce': 31577}]}, 'public_addr': '172.21.15.74:6801/31577', 'cluster_addr': '172.21.15.74:6802/31577', 'heartbeat_back_addr': '172.21.15.74:6804/31577', 'heartbeat_front_addr': '172.21.15.74:6803/31577', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '1fbe530a-63ce-4dc8-82ab-18793d6ebc0b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.7196044921875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6809', 'nonce': 31578}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6810', 'nonce': 31578}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6812', 'nonce': 31578}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6811', 'nonce': 31578}]}, 'public_addr': '172.21.15.74:6809/31578', 'cluster_addr': '172.21.15.74:6810/31578', 'heartbeat_back_addr': '172.21.15.74:6812/31578', 'heartbeat_front_addr': '172.21.15.74:6811/31578', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd6fe4338-3e11-4f73-9cbb-e1b046ce286a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6805', 'nonce': 31576}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6806', 'nonce': 31576}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6808', 'nonce': 31576}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6807', 'nonce': 31576}]}, 'public_addr': '172.21.15.74:6805/31576', 'cluster_addr': '172.21.15.74:6806/31576', 'heartbeat_back_addr': '172.21.15.74:6808/31576', 'heartbeat_front_addr': '172.21.15.74:6807/31576', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6813', 'nonce': 31579}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6814', 'nonce': 31579}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6816', 'nonce': 31579}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6815', 'nonce': 31579}]}, 'public_addr': '172.21.15.74:6813/31579', 'cluster_addr': '172.21.15.74:6814/31579', 'heartbeat_back_addr': '172.21.15.74:6816/31579', 'heartbeat_front_addr': '172.21.15.74:6815/31579', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '01aadc13-29c5-4ddd-9bb9-de24b2b20412', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6800', 'nonce': 31545}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6801', 'nonce': 31545}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6803', 'nonce': 31545}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6802', 'nonce': 31545}]}, 'public_addr': '172.21.15.96:6800/31545', 'cluster_addr': '172.21.15.96:6801/31545', 'heartbeat_back_addr': '172.21.15.96:6803/31545', 'heartbeat_front_addr': '172.21.15.96:6802/31545', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'f5556a82-73b4-47b1-8e29-1d2fe031d8fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6804', 'nonce': 31547}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6805', 'nonce': 31547}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6807', 'nonce': 31547}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6806', 'nonce': 31547}]}, 'public_addr': '172.21.15.96:6804/31547', 'cluster_addr': '172.21.15.96:6805/31547', 'heartbeat_back_addr': '172.21.15.96:6807/31547', 'heartbeat_front_addr': '172.21.15.96:6806/31547', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9b98b337-e1cc-43cd-9e9b-f21ae3f20899', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6808', 'nonce': 31546}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6809', 'nonce': 31546}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6811', 'nonce': 31546}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6810', 'nonce': 31546}]}, 'public_addr': '172.21.15.96:6808/31546', 'cluster_addr': '172.21.15.96:6809/31546', 'heartbeat_back_addr': '172.21.15.96:6811/31546', 'heartbeat_front_addr': '172.21.15.96:6810/31546', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '777662d8-bd34-48e8-83e1-24ce9e61f14b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 26, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6812', 'nonce': 31570}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6813', 'nonce': 31570}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6815', 'nonce': 31570}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6814', 'nonce': 31570}]}, 'public_addr': '172.21.15.96:6812/31570', 'cluster_addr': '172.21.15.96:6813/31570', 'heartbeat_back_addr': '172.21.15.96:6815/31570', 'heartbeat_front_addr': '172.21.15.96:6814/31570', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:06:15.298063+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:06:15.577450+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:06:15.542874+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:06:15.493720+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:06:15.534281+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:06:15.601875+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:06:15.340718+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:06:15.549109+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.7', 'mappings': [{'from': 0, 'to': 7}]}], 'pg_temp': [], 'primary_temp': [], 'blacklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}} 2020-08-07T15:07:04.789 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:07:04.790 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:07:04.790 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:07:04.791 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:04.785+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:04.799 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:07:04.841 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:07:04.893 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:07:04.893 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:04.892+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:04.966 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:04.966 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:07:04.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:07:04.967 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:04.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.967 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:2400 3.e 3:70ffa62d:::benchmark_data_smithi074_35447_object299:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:07:04.967 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.694617+0000", 2020-08-07T15:07:04.967 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.266510054000001, 2020-08-07T15:07:04.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24075575099999999, 2020-08-07T15:07:04.968 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:04.968 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:04.968 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:04.968 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:04.969 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:04.969 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2400 2020-08-07T15:07:04.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.969 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:04.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:04.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.694617+0000", 2020-08-07T15:07:04.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:04.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:04.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.694617+0000", 2020-08-07T15:07:04.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5681999999999999e-05 2020-08-07T15:07:04.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:04.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.694643+0000", 2020-08-07T15:07:04.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0079999999999998e-06 2020-08-07T15:07:04.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:04.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.694646+0000", 2020-08-07T15:07:04.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.015e-06 2020-08-07T15:07:04.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:04.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.694652+0000", 2020-08-07T15:07:04.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7130000000000001e-06 2020-08-07T15:07:04.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:04.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.694655+0000", 2020-08-07T15:07:04.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031117319999999999 2020-08-07T15:07:04.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:04.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.697766+0000", 2020-08-07T15:07:04.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015382800000000001 2020-08-07T15:07:04.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:04.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.697920+0000", 2020-08-07T15:07:04.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00044993600000000001 2020-08-07T15:07:04.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:04.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.698370+0000", 2020-08-07T15:07:04.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027536739999999998 2020-08-07T15:07:04.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:04.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.701124+0000", 2020-08-07T15:07:04.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999999e-07 2020-08-07T15:07:04.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:04.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.701124+0000", 2020-08-07T15:07:04.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7494000000000001e-05 2020-08-07T15:07:04.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:04.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.701152+0000", 2020-08-07T15:07:04.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086935569999999993 2020-08-07T15:07:04.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:04.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.709845+0000", 2020-08-07T15:07:04.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.225456669 2020-08-07T15:07:04.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:04.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.935302+0000", 2020-08-07T15:07:04.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.043e-06 2020-08-07T15:07:04.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:04.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.935303+0000", 2020-08-07T15:07:04.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9963999999999996e-05 2020-08-07T15:07:04.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:04.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.935373+0000", 2020-08-07T15:07:04.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24075575099999999 2020-08-07T15:07:04.986 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:04.986 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:04.986 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:04.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.987 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:2400 3.e e23/20 ondisk, result = 0)", 2020-08-07T15:07:04.987 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.702171+0000", 2020-08-07T15:07:04.987 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.258955771, 2020-08-07T15:07:04.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23320996599999999, 2020-08-07T15:07:04.988 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:04.988 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:04.988 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:04.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:04.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.702171+0000", 2020-08-07T15:07:04.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:04.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:04.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.702171+0000", 2020-08-07T15:07:04.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0556999999999999e-05 2020-08-07T15:07:04.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:04.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.702202+0000", 2020-08-07T15:07:04.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9409999999999998e-06 2020-08-07T15:07:04.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:04.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.702207+0000", 2020-08-07T15:07:04.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23266323 2020-08-07T15:07:04.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:04.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.934870+0000", 2020-08-07T15:07:04.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2899999999999998e-06 2020-08-07T15:07:04.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:04.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.934873+0000", 2020-08-07T15:07:04.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00041063500000000001 2020-08-07T15:07:04.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:04.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.935284+0000", 2020-08-07T15:07:04.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8419999999999994e-06 2020-08-07T15:07:04.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:04.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.935293+0000", 2020-08-07T15:07:04.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8471e-05 2020-08-07T15:07:04.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:04.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.935381+0000", 2020-08-07T15:07:04.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23320996599999999 2020-08-07T15:07:04.997 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:04.997 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:04.997 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:04.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:04.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:04.998 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:9352 3.e 3:7113579b:::benchmark_data_smithi074_35447_object1168:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:04.998 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.885925+0000", 2020-08-07T15:07:04.998 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.075202099999998, 2020-08-07T15:07:04.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24955413800000001, 2020-08-07T15:07:04.999 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:04.999 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:04.999 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:04.999 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:05.000 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:05.000 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 9352 2020-08-07T15:07:05.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.000 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:05.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:05.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.885925+0000", 2020-08-07T15:07:05.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:05.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:05.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.885925+0000", 2020-08-07T15:07:05.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4308999999999998e-05 2020-08-07T15:07:05.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:05.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.885979+0000", 2020-08-07T15:07:05.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0440000000000001e-06 2020-08-07T15:07:05.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:05.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.885985+0000", 2020-08-07T15:07:05.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9462000000000001e-05 2020-08-07T15:07:05.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:05.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.886005+0000", 2020-08-07T15:07:05.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2530000000000001e-06 2020-08-07T15:07:05.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:05.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.886011+0000", 2020-08-07T15:07:05.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025236080000000001 2020-08-07T15:07:05.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:05.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.888535+0000", 2020-08-07T15:07:05.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112871 2020-08-07T15:07:05.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:05.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.888648+0000", 2020-08-07T15:07:05.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020764399999999999 2020-08-07T15:07:05.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:05.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.888855+0000", 2020-08-07T15:07:05.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048523799999999999 2020-08-07T15:07:05.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.893708+0000", 2020-08-07T15:07:05.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030190300000000001 2020-08-07T15:07:05.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:05.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.894010+0000", 2020-08-07T15:07:05.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24135353300000001 2020-08-07T15:07:05.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.135363+0000", 2020-08-07T15:07:05.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9899999999999999e-07 2020-08-07T15:07:05.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:05.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.135364+0000", 2020-08-07T15:07:05.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7601999999999994e-05 2020-08-07T15:07:05.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:05.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.135431+0000", 2020-08-07T15:07:05.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7729999999999999e-05 2020-08-07T15:07:05.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:05.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.135479+0000", 2020-08-07T15:07:05.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24955413800000001 2020-08-07T15:07:05.015 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.015 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:05.015 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.015 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:9976 3.e 3:7fb2e2e5:::benchmark_data_smithi074_35447_object1246:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:05.016 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:07:05.016 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.723929213000002, 2020-08-07T15:07:05.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28420182999999999, 2020-08-07T15:07:05.016 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:05.016 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:05.017 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:05.017 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:05.017 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:05.017 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 9976 2020-08-07T15:07:05.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.018 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:05.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:05.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:07:05.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:05.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:05.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:07:05.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7113e-05 2020-08-07T15:07:05.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:05.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237255+0000", 2020-08-07T15:07:05.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5869999999999998e-06 2020-08-07T15:07:05.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:05.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237262+0000", 2020-08-07T15:07:05.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4467999999999999e-05 2020-08-07T15:07:05.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:05.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237276+0000", 2020-08-07T15:07:05.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0749999999999997e-06 2020-08-07T15:07:05.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:05.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237281+0000", 2020-08-07T15:07:05.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013795109999999999 2020-08-07T15:07:05.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:05.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238661+0000", 2020-08-07T15:07:05.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100309 2020-08-07T15:07:05.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:05.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238761+0000", 2020-08-07T15:07:05.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000141784 2020-08-07T15:07:05.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:05.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238903+0000", 2020-08-07T15:07:05.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00070626599999999999 2020-08-07T15:07:05.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:05.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239609+0000", 2020-08-07T15:07:05.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3499999999999996e-07 2020-08-07T15:07:05.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:05.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239610+0000", 2020-08-07T15:07:05.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0846000000000006e-05 2020-08-07T15:07:05.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:05.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239680+0000", 2020-08-07T15:07:05.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022037080000000001 2020-08-07T15:07:05.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.261718+0000", 2020-08-07T15:07:05.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25961784500000001 2020-08-07T15:07:05.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521335+0000", 2020-08-07T15:07:05.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2839999999999999e-06 2020-08-07T15:07:05.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:05.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521337+0000", 2020-08-07T15:07:05.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3126999999999994e-05 2020-08-07T15:07:05.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:05.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521400+0000", 2020-08-07T15:07:05.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28420182999999999 2020-08-07T15:07:05.033 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.033 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:05.033 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.034 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:9976 3.e e24/20 ondisk, result = 0)", 2020-08-07T15:07:05.034 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.269400+0000", 2020-08-07T15:07:05.035 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.691726988999999, 2020-08-07T15:07:05.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25200699900000001, 2020-08-07T15:07:05.035 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:05.035 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:05.035 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:05.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:05.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.269400+0000", 2020-08-07T15:07:05.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:05.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:05.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.269400+0000", 2020-08-07T15:07:05.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1279999999999999e-05 2020-08-07T15:07:05.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:05.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.269431+0000", 2020-08-07T15:07:05.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.071e-06 2020-08-07T15:07:05.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:05.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.269438+0000", 2020-08-07T15:07:05.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24671454900000001 2020-08-07T15:07:05.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:05.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.516152+0000", 2020-08-07T15:07:05.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4140000000000002e-06 2020-08-07T15:07:05.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:05.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.516157+0000", 2020-08-07T15:07:05.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051603170000000002 2020-08-07T15:07:05.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:05.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521318+0000", 2020-08-07T15:07:05.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7479999999999999e-06 2020-08-07T15:07:05.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:05.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521327+0000", 2020-08-07T15:07:05.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0619999999999997e-05 2020-08-07T15:07:05.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:05.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521407+0000", 2020-08-07T15:07:05.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25200699900000001 2020-08-07T15:07:05.044 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.044 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:05.045 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.046 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15096 3.d 3:b3ff3030:::benchmark_data_smithi074_35447_object1886:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:05.046 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.412912+0000", 2020-08-07T15:07:05.046 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.548215377999998, 2020-08-07T15:07:05.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24065245299999999, 2020-08-07T15:07:05.046 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:05.046 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15096 2020-08-07T15:07:05.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.048 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:05.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:05.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.412912+0000", 2020-08-07T15:07:05.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:05.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:05.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.412912+0000", 2020-08-07T15:07:05.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2509999999999999e-05 2020-08-07T15:07:05.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:05.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.412944+0000", 2020-08-07T15:07:05.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3500000000000001e-06 2020-08-07T15:07:05.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:05.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.412948+0000", 2020-08-07T15:07:05.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1580000000000002e-06 2020-08-07T15:07:05.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:05.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.412954+0000", 2020-08-07T15:07:05.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1219999999999999e-06 2020-08-07T15:07:05.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:05.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.412957+0000", 2020-08-07T15:07:05.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034271409999999999 2020-08-07T15:07:05.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:05.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.416384+0000", 2020-08-07T15:07:05.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3467999999999995e-05 2020-08-07T15:07:05.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:05.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.416458+0000", 2020-08-07T15:07:05.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015322299999999999 2020-08-07T15:07:05.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:05.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.416611+0000", 2020-08-07T15:07:05.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005322027 2020-08-07T15:07:05.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:05.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.421933+0000", 2020-08-07T15:07:05.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.31e-06 2020-08-07T15:07:05.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:05.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.421934+0000", 2020-08-07T15:07:05.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5050999999999996e-05 2020-08-07T15:07:05.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:05.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.422019+0000", 2020-08-07T15:07:05.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028529359999999999 2020-08-07T15:07:05.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.424872+0000", 2020-08-07T15:07:05.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22862891099999999 2020-08-07T15:07:05.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.653501+0000", 2020-08-07T15:07:05.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2020-08-07T15:07:05.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:05.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.653502+0000", 2020-08-07T15:07:05.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2333000000000001e-05 2020-08-07T15:07:05.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:05.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.653564+0000", 2020-08-07T15:07:05.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24065245299999999 2020-08-07T15:07:05.063 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.063 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:05.064 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.064 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15576 3.e 3:727724a5:::benchmark_data_smithi074_35447_object1946:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:05.064 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:07:05.065 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.294492250000001, 2020-08-07T15:07:05.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999, 2020-08-07T15:07:05.065 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:05.065 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:05.065 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:05.065 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:05.066 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:05.066 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15576 2020-08-07T15:07:05.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.066 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:05.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:07:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:05.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:07:05.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7156e-05 2020-08-07T15:07:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666682+0000", 2020-08-07T15:07:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0209999999999999e-06 2020-08-07T15:07:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666687+0000", 2020-08-07T15:07:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0730999999999999e-05 2020-08-07T15:07:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666698+0000", 2020-08-07T15:07:05.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2020-08-07T15:07:05.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:05.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666702+0000", 2020-08-07T15:07:05.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023978459999999999 2020-08-07T15:07:05.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:05.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669100+0000", 2020-08-07T15:07:05.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9664000000000001e-05 2020-08-07T15:07:05.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:05.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669190+0000", 2020-08-07T15:07:05.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015957100000000001 2020-08-07T15:07:05.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:05.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669350+0000", 2020-08-07T15:07:05.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016470149999999999 2020-08-07T15:07:05.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:05.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:07:05.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1299999999999999e-07 2020-08-07T15:07:05.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:05.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:07:05.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5269000000000002e-05 2020-08-07T15:07:05.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:05.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.671022+0000", 2020-08-07T15:07:05.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031414400000000001 2020-08-07T15:07:05.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.674163+0000", 2020-08-07T15:07:05.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45641417400000001 2020-08-07T15:07:05.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:07:05.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0399999999999995e-07 2020-08-07T15:07:05.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:05.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:07:05.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3343999999999997e-05 2020-08-07T15:07:05.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:05.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130622+0000", 2020-08-07T15:07:05.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999 2020-08-07T15:07:05.082 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.083 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:05.083 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.083 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15743 3.d 3:b5661143:::benchmark_data_smithi074_35447_object1967:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:05.084 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.768880+0000", 2020-08-07T15:07:05.084 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.192246838999999, 2020-08-07T15:07:05.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24181233499999999, 2020-08-07T15:07:05.084 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:05.084 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:05.085 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:05.085 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:05.085 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:05.085 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15743 2020-08-07T15:07:05.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.086 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:05.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:05.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.768880+0000", 2020-08-07T15:07:05.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:05.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:05.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.768880+0000", 2020-08-07T15:07:05.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8946000000000002e-05 2020-08-07T15:07:05.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:05.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.768929+0000", 2020-08-07T15:07:05.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9020000000000004e-06 2020-08-07T15:07:05.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:05.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.768935+0000", 2020-08-07T15:07:05.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2283000000000001e-05 2020-08-07T15:07:05.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:05.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.768947+0000", 2020-08-07T15:07:05.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.186e-06 2020-08-07T15:07:05.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:05.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.768953+0000", 2020-08-07T15:07:05.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0068896529999999999 2020-08-07T15:07:05.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:05.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.775842+0000", 2020-08-07T15:07:05.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0270000000000001e-05 2020-08-07T15:07:05.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:05.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.775933+0000", 2020-08-07T15:07:05.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017233799999999999 2020-08-07T15:07:05.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:05.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.776105+0000", 2020-08-07T15:07:05.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00117592 2020-08-07T15:07:05.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:05.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777281+0000", 2020-08-07T15:07:05.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6499999999999998e-07 2020-08-07T15:07:05.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:05.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777282+0000", 2020-08-07T15:07:05.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3357999999999999e-05 2020-08-07T15:07:05.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:05.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777345+0000", 2020-08-07T15:07:05.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030411169999999999 2020-08-07T15:07:05.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.780386+0000", 2020-08-07T15:07:05.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.230239216 2020-08-07T15:07:05.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.010625+0000", 2020-08-07T15:07:05.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5600000000000005e-07 2020-08-07T15:07:05.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:05.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.010626+0000", 2020-08-07T15:07:05.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6625000000000007e-05 2020-08-07T15:07:05.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:05.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.010693+0000", 2020-08-07T15:07:05.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24181233499999999 2020-08-07T15:07:05.101 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.101 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:05.101 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.102 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15744 3.d 3:b5661143:::benchmark_data_smithi074_35447_object1967:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:05.102 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.769136+0000", 2020-08-07T15:07:05.102 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.191991031000001, 2020-08-07T15:07:05.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24180031099999999, 2020-08-07T15:07:05.102 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:05.103 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:05.103 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:05.103 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:05.103 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:05.103 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15744 2020-08-07T15:07:05.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.104 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:05.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:05.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.769136+0000", 2020-08-07T15:07:05.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:05.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:05.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.769136+0000", 2020-08-07T15:07:05.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5503999999999997e-05 2020-08-07T15:07:05.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:05.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.769182+0000", 2020-08-07T15:07:05.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.664e-06 2020-08-07T15:07:05.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:05.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.769185+0000", 2020-08-07T15:07:05.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7200000000000006e-06 2020-08-07T15:07:05.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:05.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.769193+0000", 2020-08-07T15:07:05.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8260000000000003e-06 2020-08-07T15:07:05.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:05.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.769197+0000", 2020-08-07T15:07:05.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0074215849999999996 2020-08-07T15:07:05.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:05.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.776618+0000", 2020-08-07T15:07:05.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010513700000000001 2020-08-07T15:07:05.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:05.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.776724+0000", 2020-08-07T15:07:05.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016319800000000001 2020-08-07T15:07:05.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:05.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.776887+0000", 2020-08-07T15:07:05.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00090081899999999997 2020-08-07T15:07:05.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:05.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777788+0000", 2020-08-07T15:07:05.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5000000000000001e-07 2020-08-07T15:07:05.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:05.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777788+0000", 2020-08-07T15:07:05.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5197000000000006e-05 2020-08-07T15:07:05.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:05.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777854+0000", 2020-08-07T15:07:05.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22872283800000001 2020-08-07T15:07:05.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.006576+0000", 2020-08-07T15:07:05.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004298367 2020-08-07T15:07:05.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.010875+0000", 2020-08-07T15:07:05.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.08e-07 2020-08-07T15:07:05.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:05.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.010875+0000", 2020-08-07T15:07:05.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1198000000000006e-05 2020-08-07T15:07:05.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:05.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.010936+0000", 2020-08-07T15:07:05.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24180031099999999 2020-08-07T15:07:05.120 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.120 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:05.120 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.121 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:17824 3.d 3:bd4e0935:::benchmark_data_smithi074_35447_object2227:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:05.121 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.140708+0000", 2020-08-07T15:07:05.121 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.820419327, 2020-08-07T15:07:05.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25688457100000001, 2020-08-07T15:07:05.122 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:05.122 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:05.122 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:05.122 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:05.122 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:05.123 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 17824 2020-08-07T15:07:05.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.123 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:05.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:05.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140708+0000", 2020-08-07T15:07:05.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:05.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:05.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140708+0000", 2020-08-07T15:07:05.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0522000000000002e-05 2020-08-07T15:07:05.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:05.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140758+0000", 2020-08-07T15:07:05.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1680000000000001e-06 2020-08-07T15:07:05.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:05.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140765+0000", 2020-08-07T15:07:05.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5791999999999998e-05 2020-08-07T15:07:05.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:05.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140810+0000", 2020-08-07T15:07:05.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2389999999999996e-06 2020-08-07T15:07:05.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:05.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140818+0000", 2020-08-07T15:07:05.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001721544 2020-08-07T15:07:05.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:05.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.142539+0000", 2020-08-07T15:07:05.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020908100000000001 2020-08-07T15:07:05.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:05.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.142748+0000", 2020-08-07T15:07:05.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019000000000000001 2020-08-07T15:07:05.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:05.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.142938+0000", 2020-08-07T15:07:05.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001381279 2020-08-07T15:07:05.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.144319+0000", 2020-08-07T15:07:05.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010975570000000001 2020-08-07T15:07:05.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:05.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.145417+0000", 2020-08-07T15:07:05.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.252064128 2020-08-07T15:07:05.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397481+0000", 2020-08-07T15:07:05.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003e-06 2020-08-07T15:07:05.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:05.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397482+0000", 2020-08-07T15:07:05.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3081000000000001e-05 2020-08-07T15:07:05.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:05.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397545+0000", 2020-08-07T15:07:05.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7176999999999999e-05 2020-08-07T15:07:05.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:05.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397592+0000", 2020-08-07T15:07:05.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25688457100000001 2020-08-07T15:07:05.146 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.147 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:05.147 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.147 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:35576 3.f 3:fca2de08:::benchmark_data_smithi074_35447_object4446:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-08-07T15:07:05.148 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:59.287747+0000", 2020-08-07T15:07:05.148 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.6733805239999997, 2020-08-07T15:07:05.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25861668500000001, 2020-08-07T15:07:05.148 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:05.148 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:05.149 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:05.149 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:05.149 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:05.149 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 35576 2020-08-07T15:07:05.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.150 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:05.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:05.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287747+0000", 2020-08-07T15:07:05.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:05.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:05.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287747+0000", 2020-08-07T15:07:05.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0089000000000001e-05 2020-08-07T15:07:05.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:05.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287797+0000", 2020-08-07T15:07:05.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4219999999999997e-06 2020-08-07T15:07:05.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:05.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287803+0000", 2020-08-07T15:07:05.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4132e-05 2020-08-07T15:07:05.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:05.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287817+0000", 2020-08-07T15:07:05.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1600000000000003e-06 2020-08-07T15:07:05.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:05.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287823+0000", 2020-08-07T15:07:05.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019418230000000001 2020-08-07T15:07:05.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:05.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.289765+0000", 2020-08-07T15:07:05.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110597 2020-08-07T15:07:05.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:05.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.289876+0000", 2020-08-07T15:07:05.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017674800000000001 2020-08-07T15:07:05.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:05.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.290053+0000", 2020-08-07T15:07:05.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052995730000000001 2020-08-07T15:07:05.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:05.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295352+0000", 2020-08-07T15:07:05.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9999999999999997e-07 2020-08-07T15:07:05.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:05.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295353+0000", 2020-08-07T15:07:05.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6803000000000004e-05 2020-08-07T15:07:05.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:05.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295420+0000", 2020-08-07T15:07:05.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015611539999999999 2020-08-07T15:07:05.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.296981+0000", 2020-08-07T15:07:05.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.249314866 2020-08-07T15:07:05.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546296+0000", 2020-08-07T15:07:05.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7300000000000005e-07 2020-08-07T15:07:05.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:05.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546296+0000", 2020-08-07T15:07:05.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6845000000000001e-05 2020-08-07T15:07:05.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:05.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546363+0000", 2020-08-07T15:07:05.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25861668500000001 2020-08-07T15:07:05.165 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.165 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:05.166 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.166 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:35576 3.f e31/20 ondisk, result = 0)", 2020-08-07T15:07:05.166 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:59.295024+0000", 2020-08-07T15:07:05.167 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.6661033510000003, 2020-08-07T15:07:05.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25134726499999999, 2020-08-07T15:07:05.167 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:05.167 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:05.167 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:05.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:05.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295024+0000", 2020-08-07T15:07:05.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:05.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:05.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295024+0000", 2020-08-07T15:07:05.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7844999999999999e-05 2020-08-07T15:07:05.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:05.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295052+0000", 2020-08-07T15:07:05.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7740000000000004e-06 2020-08-07T15:07:05.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:05.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295056+0000", 2020-08-07T15:07:05.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25017799099999999 2020-08-07T15:07:05.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:05.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.545234+0000", 2020-08-07T15:07:05.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5140000000000003e-06 2020-08-07T15:07:05.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:05.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.545239+0000", 2020-08-07T15:07:05.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010386989999999999 2020-08-07T15:07:05.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:05.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546278+0000", 2020-08-07T15:07:05.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0480000000000004e-06 2020-08-07T15:07:05.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:05.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546287+0000", 2020-08-07T15:07:05.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4394000000000005e-05 2020-08-07T15:07:05.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:05.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546371+0000", 2020-08-07T15:07:05.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25134726499999999 2020-08-07T15:07:05.178 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.178 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:05.178 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.179 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:36096 3.f 3:ffeda9b1:::benchmark_data_smithi074_35447_object4511:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-08-07T15:07:05.179 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:59.572029+0000", 2020-08-07T15:07:05.179 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.3890980839999996, 2020-08-07T15:07:05.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24717645599999999, 2020-08-07T15:07:05.179 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:05.180 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:05.180 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:05.180 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:05.180 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:05.181 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 36096 2020-08-07T15:07:05.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.182 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:05.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:05.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.572029+0000", 2020-08-07T15:07:05.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:05.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:05.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.572029+0000", 2020-08-07T15:07:05.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1946e-05 2020-08-07T15:07:05.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:05.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.572081+0000", 2020-08-07T15:07:05.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5559999999999999e-06 2020-08-07T15:07:05.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:05.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.572085+0000", 2020-08-07T15:07:05.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3620000000000002e-06 2020-08-07T15:07:05.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:05.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.572091+0000", 2020-08-07T15:07:05.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1020000000000001e-06 2020-08-07T15:07:05.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:05.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.572094+0000", 2020-08-07T15:07:05.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037454749999999998 2020-08-07T15:07:05.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:05.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.575840+0000", 2020-08-07T15:07:05.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1586e-05 2020-08-07T15:07:05.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:05.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.575931+0000", 2020-08-07T15:07:05.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014823700000000001 2020-08-07T15:07:05.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:05.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.576079+0000", 2020-08-07T15:07:05.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005540016 2020-08-07T15:07:05.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:05.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.581619+0000", 2020-08-07T15:07:05.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4499999999999995e-07 2020-08-07T15:07:05.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:05.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.581620+0000", 2020-08-07T15:07:05.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3874999999999994e-05 2020-08-07T15:07:05.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:05.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.581684+0000", 2020-08-07T15:07:05.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084986090000000007 2020-08-07T15:07:05.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.590183+0000", 2020-08-07T15:07:05.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22895041599999999 2020-08-07T15:07:05.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.819133+0000", 2020-08-07T15:07:05.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.44e-07 2020-08-07T15:07:05.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:05.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.819134+0000", 2020-08-07T15:07:05.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1687000000000007e-05 2020-08-07T15:07:05.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:05.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.819206+0000", 2020-08-07T15:07:05.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24717645599999999 2020-08-07T15:07:05.198 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.198 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:05.198 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.199 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:41431 3.d 3:b5c8a160:::benchmark_data_smithi074_35447_object5178:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:07:05.199 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.791939+0000", 2020-08-07T15:07:05.199 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 2.1691877210000001, 2020-08-07T15:07:05.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28436496, 2020-08-07T15:07:05.200 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:05.200 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:05.200 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:05.200 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:05.200 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:05.200 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 41431 2020-08-07T15:07:05.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.201 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:05.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:05.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791939+0000", 2020-08-07T15:07:05.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:05.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:05.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791939+0000", 2020-08-07T15:07:05.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6304999999999998e-05 2020-08-07T15:07:05.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:05.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791986+0000", 2020-08-07T15:07:05.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6339999999999998e-06 2020-08-07T15:07:05.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:05.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791991+0000", 2020-08-07T15:07:05.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.026e-05 2020-08-07T15:07:05.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:05.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792002+0000", 2020-08-07T15:07:05.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6240000000000001e-06 2020-08-07T15:07:05.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:05.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792006+0000", 2020-08-07T15:07:05.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015600220000000001 2020-08-07T15:07:05.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:05.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.793566+0000", 2020-08-07T15:07:05.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3628999999999994e-05 2020-08-07T15:07:05.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:05.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.793660+0000", 2020-08-07T15:07:05.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000168197 2020-08-07T15:07:05.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:05.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.793828+0000", 2020-08-07T15:07:05.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001273251 2020-08-07T15:07:05.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:05.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795101+0000", 2020-08-07T15:07:05.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5899999999999998e-07 2020-08-07T15:07:05.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:05.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795102+0000", 2020-08-07T15:07:05.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4904000000000001e-05 2020-08-07T15:07:05.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:05.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795127+0000", 2020-08-07T15:07:05.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00670237 2020-08-07T15:07:05.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801829+0000", 2020-08-07T15:07:05.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27442024999999998 2020-08-07T15:07:05.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076249+0000", 2020-08-07T15:07:05.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999997e-07 2020-08-07T15:07:05.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:05.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076250+0000", 2020-08-07T15:07:05.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4369000000000001e-05 2020-08-07T15:07:05.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:05.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076304+0000", 2020-08-07T15:07:05.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28436496 2020-08-07T15:07:05.225 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.225 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:05.225 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.226 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:41432 3.d 3:b5c8a160:::benchmark_data_smithi074_35447_object5178:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:07:05.226 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:07:05.226 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 2.1688370250000002, 2020-08-07T15:07:05.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.37720701699999998, 2020-08-07T15:07:05.227 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:05.227 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:05.227 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:05.227 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:05.227 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:05.228 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 41432 2020-08-07T15:07:05.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.228 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:05.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:05.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:07:05.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:05.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:05.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:07:05.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4787999999999999e-05 2020-08-07T15:07:05.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:05.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792335+0000", 2020-08-07T15:07:05.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5999999999999997e-06 2020-08-07T15:07:05.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:05.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792341+0000", 2020-08-07T15:07:05.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0998e-05 2020-08-07T15:07:05.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:05.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792352+0000", 2020-08-07T15:07:05.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2989999999999996e-06 2020-08-07T15:07:05.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:05.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792357+0000", 2020-08-07T15:07:05.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001852752 2020-08-07T15:07:05.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:05.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794210+0000", 2020-08-07T15:07:05.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5771000000000002e-05 2020-08-07T15:07:05.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:05.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794305+0000", 2020-08-07T15:07:05.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012776399999999999 2020-08-07T15:07:05.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:05.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794433+0000", 2020-08-07T15:07:05.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00069673599999999999 2020-08-07T15:07:05.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:05.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795130+0000", 2020-08-07T15:07:05.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1500000000000001e-07 2020-08-07T15:07:05.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:05.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795130+0000", 2020-08-07T15:07:05.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7319e-05 2020-08-07T15:07:05.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:05.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795167+0000", 2020-08-07T15:07:05.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27910026700000001 2020-08-07T15:07:05.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074268+0000", 2020-08-07T15:07:05.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.095150660999999997 2020-08-07T15:07:05.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169418+0000", 2020-08-07T15:07:05.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.423e-06 2020-08-07T15:07:05.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:05.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169420+0000", 2020-08-07T15:07:05.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7423999999999996e-05 2020-08-07T15:07:05.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:05.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169497+0000", 2020-08-07T15:07:05.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.37720701699999998 2020-08-07T15:07:05.274 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.275 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:05.275 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.275 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:41431 3.d e32/20 ondisk, result = 0)", 2020-08-07T15:07:05.276 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.796229+0000", 2020-08-07T15:07:05.276 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 2.164898123, 2020-08-07T15:07:05.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28007903299999998, 2020-08-07T15:07:05.276 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:05.276 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:05.277 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:05.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:05.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.796229+0000", 2020-08-07T15:07:05.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:05.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:05.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.796229+0000", 2020-08-07T15:07:05.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3272000000000003e-05 2020-08-07T15:07:05.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:05.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.796262+0000", 2020-08-07T15:07:05.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9579999999999998e-06 2020-08-07T15:07:05.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:05.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.796267+0000", 2020-08-07T15:07:05.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27993393599999999 2020-08-07T15:07:05.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:05.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076201+0000", 2020-08-07T15:07:05.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8709999999999999e-06 2020-08-07T15:07:05.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:05.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076205+0000", 2020-08-07T15:07:05.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9128e-05 2020-08-07T15:07:05.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:05.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076234+0000", 2020-08-07T15:07:05.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9800000000000001e-06 2020-08-07T15:07:05.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:05.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076241+0000", 2020-08-07T15:07:05.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6888000000000001e-05 2020-08-07T15:07:05.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:05.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076308+0000", 2020-08-07T15:07:05.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28007903299999998 2020-08-07T15:07:05.285 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.285 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:05.286 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.286 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:41432 3.d e32/20 ondisk, result = 0)", 2020-08-07T15:07:05.287 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.801630+0000", 2020-08-07T15:07:05.287 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 2.1594972220000002, 2020-08-07T15:07:05.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27264033999999998, 2020-08-07T15:07:05.287 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:05.288 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:05.288 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:05.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:05.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801630+0000", 2020-08-07T15:07:05.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:05.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:05.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801630+0000", 2020-08-07T15:07:05.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6257e-05 2020-08-07T15:07:05.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:05.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801646+0000", 2020-08-07T15:07:05.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5450000000000002e-06 2020-08-07T15:07:05.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:05.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801649+0000", 2020-08-07T15:07:05.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27254806999999998 2020-08-07T15:07:05.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:05.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074197+0000", 2020-08-07T15:07:05.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4079999999999997e-06 2020-08-07T15:07:05.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:05.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074202+0000", 2020-08-07T15:07:05.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5222000000000002e-05 2020-08-07T15:07:05.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:05.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074247+0000", 2020-08-07T15:07:05.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9459999999999993e-06 2020-08-07T15:07:05.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:05.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074257+0000", 2020-08-07T15:07:05.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2892e-05 2020-08-07T15:07:05.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:05.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074270+0000", 2020-08-07T15:07:05.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27264033999999998 2020-08-07T15:07:05.299 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.299 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:05.299 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.299 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:42104 3.d 3:bcc35439:::benchmark_data_smithi074_35447_object5262:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:07:05.300 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:07:05.300 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.7222048539999999, 2020-08-07T15:07:05.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28812091400000001, 2020-08-07T15:07:05.300 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:05.300 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:05.301 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:05.301 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:05.301 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:05.301 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42104 2020-08-07T15:07:05.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.302 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:05.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:05.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:07:05.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:05.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:05.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:07:05.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9734000000000002e-05 2020-08-07T15:07:05.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:05.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238972+0000", 2020-08-07T15:07:05.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6910000000000002e-06 2020-08-07T15:07:05.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:05.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238978+0000", 2020-08-07T15:07:05.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1864e-05 2020-08-07T15:07:05.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:05.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238990+0000", 2020-08-07T15:07:05.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.737e-06 2020-08-07T15:07:05.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:05.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238996+0000", 2020-08-07T15:07:05.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00283943 2020-08-07T15:07:05.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:05.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.241836+0000", 2020-08-07T15:07:05.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4712999999999999e-05 2020-08-07T15:07:05.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:05.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.241930+0000", 2020-08-07T15:07:05.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015328499999999999 2020-08-07T15:07:05.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:05.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.242084+0000", 2020-08-07T15:07:05.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016104400000000001 2020-08-07T15:07:05.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:05.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243694+0000", 2020-08-07T15:07:05.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1399999999999997e-07 2020-08-07T15:07:05.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:05.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243695+0000", 2020-08-07T15:07:05.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7800999999999998e-05 2020-08-07T15:07:05.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:05.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243753+0000", 2020-08-07T15:07:05.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090405661999999998 2020-08-07T15:07:05.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.334158+0000", 2020-08-07T15:07:05.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19283929299999999 2020-08-07T15:07:05.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.526997+0000", 2020-08-07T15:07:05.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.012e-06 2020-08-07T15:07:05.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:05.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.526998+0000", 2020-08-07T15:07:05.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4737999999999998e-05 2020-08-07T15:07:05.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:05.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.527043+0000", 2020-08-07T15:07:05.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28812091400000001 2020-08-07T15:07:05.318 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.318 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:05.318 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.319 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:44120 3.e 3:7e6f7bb9:::benchmark_data_smithi074_35447_object5514:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:07:05.319 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:04.450106+0000", 2020-08-07T15:07:05.319 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.51102132, 2020-08-07T15:07:05.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26702573699999999, 2020-08-07T15:07:05.319 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:05.320 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:05.320 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:05.320 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:05.320 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:05.320 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 44120 2020-08-07T15:07:05.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.321 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:05.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:05.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450106+0000", 2020-08-07T15:07:05.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:05.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:05.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450106+0000", 2020-08-07T15:07:05.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9926000000000003e-05 2020-08-07T15:07:05.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:05.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450156+0000", 2020-08-07T15:07:05.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4079999999999997e-06 2020-08-07T15:07:05.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:05.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450161+0000", 2020-08-07T15:07:05.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0494e-05 2020-08-07T15:07:05.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:05.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450172+0000", 2020-08-07T15:07:05.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3950000000000003e-06 2020-08-07T15:07:05.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:05.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450176+0000", 2020-08-07T15:07:05.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033213689999999998 2020-08-07T15:07:05.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:05.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.453497+0000", 2020-08-07T15:07:05.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102414 2020-08-07T15:07:05.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:05.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.453600+0000", 2020-08-07T15:07:05.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019776 2020-08-07T15:07:05.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:05.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.453798+0000", 2020-08-07T15:07:05.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001075456 2020-08-07T15:07:05.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:05.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.454873+0000", 2020-08-07T15:07:05.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7999999999999995e-07 2020-08-07T15:07:05.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:05.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.454874+0000", 2020-08-07T15:07:05.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3912000000000002e-05 2020-08-07T15:07:05.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:05.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.454928+0000", 2020-08-07T15:07:05.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058537550000000004 2020-08-07T15:07:05.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.460781+0000", 2020-08-07T15:07:05.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25630591899999999 2020-08-07T15:07:05.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.717087+0000", 2020-08-07T15:07:05.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.54e-07 2020-08-07T15:07:05.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:05.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.717088+0000", 2020-08-07T15:07:05.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3594999999999998e-05 2020-08-07T15:07:05.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:05.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.717132+0000", 2020-08-07T15:07:05.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26702573699999999 2020-08-07T15:07:05.337 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.338 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:05.338 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.338 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:44320 3.d 3:bd9891b7:::benchmark_data_smithi074_35447_object5539:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:07:05.339 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:04.579800+0000", 2020-08-07T15:07:05.340 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.38132765099999999, 2020-08-07T15:07:05.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.240159343, 2020-08-07T15:07:05.340 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:05.340 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:05.341 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:05.341 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:05.341 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:05.341 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 44320 2020-08-07T15:07:05.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.342 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:05.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:05.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.579800+0000", 2020-08-07T15:07:05.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:05.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:05.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.579800+0000", 2020-08-07T15:07:05.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8714999999999997e-05 2020-08-07T15:07:05.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:05.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.579848+0000", 2020-08-07T15:07:05.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0189999999999997e-06 2020-08-07T15:07:05.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:05.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.579853+0000", 2020-08-07T15:07:05.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0781000000000001e-05 2020-08-07T15:07:05.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:05.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.579864+0000", 2020-08-07T15:07:05.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.605e-06 2020-08-07T15:07:05.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:05.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.579869+0000", 2020-08-07T15:07:05.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037775339999999999 2020-08-07T15:07:05.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:05.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.583646+0000", 2020-08-07T15:07:05.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5248999999999996e-05 2020-08-07T15:07:05.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:05.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.583741+0000", 2020-08-07T15:07:05.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016458799999999999 2020-08-07T15:07:05.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:05.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.583906+0000", 2020-08-07T15:07:05.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015792289999999999 2020-08-07T15:07:05.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:05.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.585485+0000", 2020-08-07T15:07:05.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7500000000000001e-07 2020-08-07T15:07:05.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:05.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.585485+0000", 2020-08-07T15:07:05.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1119999999999999e-05 2020-08-07T15:07:05.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:05.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.585527+0000", 2020-08-07T15:07:05.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011384629999999999 2020-08-07T15:07:05.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.586665+0000", 2020-08-07T15:07:05.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23321710800000001 2020-08-07T15:07:05.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:05.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.819882+0000", 2020-08-07T15:07:05.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1210000000000001e-06 2020-08-07T15:07:05.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:05.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.819884+0000", 2020-08-07T15:07:05.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4536000000000006e-05 2020-08-07T15:07:05.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:05.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:05.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:05.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.819959+0000", 2020-08-07T15:07:05.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.240159343 2020-08-07T15:07:05.367 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.367 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:05.367 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.367 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:05.368 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:05.368 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:05.376 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:07:05.376 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:07:05.377 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:07:05.377 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:07:05.377 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:07:05.378 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:07:05.378 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:07:05.378 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:07:05.379 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:07:05.379 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:07:05.379 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:07:05.380 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:07:05.380 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2020-08-07T15:07:04.620043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":324861952,"num_objects":4960,"num_object_clones":0,"num_object_copies":14880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39658,"num_write_kb":317250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":75,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9334940,"kb_used_data":946268,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":745639780,"statfs":{"total":773094113280,"available":763535134720,"internally_reserved":8589934592,"allocated":968978432,"data_stored":967445296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,4,5,1,0,0,0,1,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":132644864,"num_objects":2024,"num_object_clones":0,"num_object_copies":6072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16192,"num_write_kb":129536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001357"},"pg_stats":[{"pgid":"3.f","version":"31'2683","reported_seq":"2697","reported_epoch":"31","state":"active+clean","last_fresh":"2020-08-07T15:07:01.696993+0000","last_change":"2020-08-07T15:06:35.163498+0000","last_active":"2020-08-07T15:07:01.696993+0000","last_peered":"2020-08-07T15:07:01.696993+0000","last_clean":"2020-08-07T15:07:01.696993+0000","last_became_active":"2020-08-07T15:06:35.162764+0000","last_became_peered":"2020-08-07T15:06:35.162764+0000","last_unstale":"2020-08-07T15:07:01.696993+0000","last_undegraded":"2020-08-07T15:07:01.696993+0000","last_fullsized":"2020-08-07T15:07:01.696993+0000","mapping_epoch":20,"log_start":"31'2675","ondisk_log_start":"31'2675","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21979136,"num_objects":336,"num_object_clones":0,"num_object_copies":1008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2683,"num_write_kb":21464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"31'2472","reported_seq":"2490","reported_epoch":"31","state":"active+clean","last_fresh":"2020-08-07T15:07:01.643113+0000","last_change":"2020-08-07T15:06:35.169137+0000","last_active":"2020-08-07T15:07:01.643113+0000","last_peered":"2020-08-07T15:07:01.643113+0000","last_clean":"2020-08-07T15:07:01.643113+0000","last_became_active":"2020-08-07T15:06:35.168689+0000","last_became_peered":"2020-08-07T15:06:35.168689+0000","last_unstale":"2020-08-07T15:07:01.643113+0000","last_undegraded":"2020-08-07T15:07:01.643113+0000","last_fullsized":"2020-08-07T15:07:01.643113+0000","mapping_epoch":20,"log_start":"31'2464","ondisk_log_start":"31'2464","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20250624,"num_objects":309,"num_object_clones":0,"num_object_copies":927,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2472,"num_write_kb":19776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"31'2680","reported_seq":"2699","reported_epoch":"31","state":"active+clean","last_fresh":"2020-08-07T15:07:01.632007+0000","last_change":"2020-08-07T15:06:35.165586+0000","last_active":"2020-08-07T15:07:01.632007+0000","last_peered":"2020-08-07T15:07:01.632007+0000","last_clean":"2020-08-07T15:07:01.632007+0000","last_became_active":"2020-08-07T15:06:35.165144+0000","last_became_peered":"2020-08-07T15:06:35.165144+0000","last_unstale":"2020-08-07T15:07:01.632007+0000","last_undegraded":"2020-08-07T15:07:01.632007+0000","last_fullsized":"2020-08-07T15:07:01.632007+0000","mapping_epoch":20,"log_start":"31'2672","ondisk_log_start":"31'2672","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21954560,"num_objects":335,"num_object_clones":0,"num_object_copies":1005,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2680,"num_write_kb":21440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"31'2448","reported_seq":"2467","reported_epoch":"31","state":"active+clean","last_fresh":"2020-08-07T15:07:01.661353+0000","last_change":"2020-08-07T15:06:35.167480+0000","last_active":"2020-08-07T15:07:01.661353+0000","last_peered":"2020-08-07T15:07:01.661353+0000","last_clean":"2020-08-07T15:07:01.661353+0000","last_became_active":"2020-08-07T15:06:35.167101+0000","last_became_peered":"2020-08-07T15:06:35.167101+0000","last_unstale":"2020-08-07T15:07:01.661353+0000","last_undegraded":"2020-08-07T15:07:01.661353+0000","last_fullsized":"2020-08-07T15:07:01.661353+0000","mapping_epoch":20,"log_start":"31'2439","ondisk_log_start":"31'2439","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20054016,"num_objects":306,"num_object_clones":0,"num_object_copies":918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2448,"num_write_kb":19584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.b","version":"31'2432","reported_seq":"2450","reported_epoch":"31","state":"active+clean","last_fresh":"2020-08-07T15:07:01.453520+0000","last_change":"2020-08-07T15:06:35.170504+0000","last_active":"2020-08-07T15:07:01.453520+0000","last_peered":"2020-08-07T15:07:01.453520+0000","last_clean":"2020-08-07T15:07:01.453520+0000","last_became_active":"2020-08-07T15:06:35.169532+0000","last_became_peered":"2020-08-07T15:06:35.169532+0000","last_unstale":"2020-08-07T15:07:01.453520+0000","last_undegraded":"2020-08-07T15:07:01.453520+0000","last_fullsized":"2020-08-07T15:07:01.453520+0000","mapping_epoch":20,"log_start":"31'2425","ondisk_log_start":"31'2425","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19922944,"num_objects":304,"num_object_clones":0,"num_object_copies":912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2432,"num_write_kb":19456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"32'2912","reported_seq":"2932","reported_epoch":"32","state":"active+clean","last_fresh":"2020-08-07T15:07:02.667597+0000","last_change":"2020-08-07T15:06:35.170105+0000","last_active":"2020-08-07T15:07:02.667597+0000","last_peered":"2020-08-07T15:07:02.667597+0000","last_clean":"2020-08-07T15:07:02.667597+0000","last_became_active":"2020-08-07T15:06:35.169494+0000","last_became_peered":"2020-08-07T15:06:35.169494+0000","last_unstale":"2020-08-07T15:07:02.667597+0000","last_undegraded":"2020-08-07T15:07:02.667597+0000","last_fullsized":"2020-08-07T15:07:02.667597+0000","mapping_epoch":20,"log_start":"32'2905","ondisk_log_start":"32'2905","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23855104,"num_objects":364,"num_object_clones":0,"num_object_copies":1092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2912,"num_write_kb":23296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"31'2408","reported_seq":"2425","reported_epoch":"31","state":"active+clean","last_fresh":"2020-08-07T15:07:01.671374+0000","last_change":"2020-08-07T15:06:35.163109+0000","last_active":"2020-08-07T15:07:01.671374+0000","last_peered":"2020-08-07T15:07:01.671374+0000","last_clean":"2020-08-07T15:07:01.671374+0000","last_became_active":"2020-08-07T15:06:35.162721+0000","last_became_peered":"2020-08-07T15:06:35.162721+0000","last_unstale":"2020-08-07T15:07:01.671374+0000","last_undegraded":"2020-08-07T15:07:01.671374+0000","last_fullsized":"2020-08-07T15:07:01.671374+0000","mapping_epoch":20,"log_start":"31'2401","ondisk_log_start":"31'2401","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19726336,"num_objects":301,"num_object_clones":0,"num_object_copies":903,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2408,"num_write_kb":19264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"31'2536","reported_seq":"2555","reported_epoch":"31","state":"active+clean","last_fresh":"2020-08-07T15:07:01.603798+0000","last_change":"2020-08-07T15:06:35.173504+0000","last_active":"2020-08-07T15:07:01.603798+0000","last_peered":"2020-08-07T15:07:01.603798+0000","last_clean":"2020-08-07T15:07:01.603798+0000","last_became_active":"2020-08-07T15:06:35.172437+0000","last_became_peered":"2020-08-07T15:06:35.172437+0000","last_unstale":"2020-08-07T15:07:01.603798+0000","last_undegraded":"2020-08-07T15:07:01.603798+0000","last_fullsized":"2020-08-07T15:07:01.603798+0000","mapping_epoch":20,"log_start":"31'2527","ondisk_log_start":"31'2527","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20774912,"num_objects":317,"num_object_clones":0,"num_object_copies":951,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2536,"num_write_kb":20288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"19","reported_epoch":"30","state":"active+clean","last_fresh":"2020-08-07T15:06:56.247942+0000","last_change":"2020-08-07T15:06:46.199001+0000","last_active":"2020-08-07T15:06:56.247942+0000","last_peered":"2020-08-07T15:06:56.247942+0000","last_clean":"2020-08-07T15:06:56.247942+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:06:56.247942+0000","last_undegraded":"2020-08-07T15:06:56.247942+0000","last_fullsized":"2020-08-07T15:06:56.247942+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.5","version":"31'2286","reported_seq":"2301","reported_epoch":"31","state":"active+clean","last_fresh":"2020-08-07T15:07:01.694378+0000","last_change":"2020-08-07T15:06:35.169307+0000","last_active":"2020-08-07T15:07:01.694378+0000","last_peered":"2020-08-07T15:07:01.694378+0000","last_clean":"2020-08-07T15:07:01.694378+0000","last_became_active":"2020-08-07T15:06:35.168887+0000","last_became_peered":"2020-08-07T15:06:35.168887+0000","last_unstale":"2020-08-07T15:07:01.694378+0000","last_undegraded":"2020-08-07T15:07:01.694378+0000","last_fullsized":"2020-08-07T15:07:01.694378+0000","mapping_epoch":20,"log_start":"31'2280","ondisk_log_start":"31'2280","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18726912,"num_objects":286,"num_object_clones":0,"num_object_copies":858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2286,"num_write_kb":18288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"22","reported_epoch":"30","state":"active+clean","last_fresh":"2020-08-07T15:06:56.238793+0000","last_change":"2020-08-07T15:06:23.074962+0000","last_active":"2020-08-07T15:06:56.238793+0000","last_peered":"2020-08-07T15:06:56.238793+0000","last_clean":"2020-08-07T15:06:56.238793+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:06:56.238793+0000","last_undegraded":"2020-08-07T15:06:56.238793+0000","last_fullsized":"2020-08-07T15:06:56.238793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"31'2376","reported_seq":"2394","reported_epoch":"31","state":"active+clean","last_fresh":"2020-08-07T15:07:01.547501+0000","last_change":"2020-08-07T15:06:35.164593+0000","last_active":"2020-08-07T15:07:01.547501+0000","last_peered":"2020-08-07T15:07:01.547501+0000","last_clean":"2020-08-07T15:07:01.547501+0000","last_became_active":"2020-08-07T15:06:35.164108+0000","last_became_peered":"2020-08-07T15:06:35.164108+0000","last_unstale":"2020-08-07T15:07:01.547501+0000","last_undegraded":"2020-08-07T15:07:01.547501+0000","last_fullsized":"2020-08-07T15:07:01.547501+0000","mapping_epoch":20,"log_start":"31'2370","ondisk_log_start":"31'2370","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19464192,"num_objects":297,"num_object_clones":0,"num_object_copies":891,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2376,"num_write_kb":19008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"22","reported_epoch":"30","state":"active+clean","last_fresh":"2020-08-07T15:06:56.248462+0000","last_change":"2020-08-07T15:06:23.223924+0000","last_active":"2020-08-07T15:06:56.248462+0000","last_peered":"2020-08-07T15:06:56.248462+0000","last_clean":"2020-08-07T15:06:56.248462+0000","last_became_active":"2020-08-07T15:06:23.223489+0000","last_became_peered":"2020-08-07T15:06:23.223489+0000","last_unstale":"2020-08-07T15:06:56.248462+0000","last_undegraded":"2020-08-07T15:06:56.248462+0000","last_fullsized":"2020-08-07T15:06:56.248462+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"31'2583","reported_seq":"2596","reported_epoch":"31","state":"active+clean","last_fresh":"2020-08-07T15:07:01.683218+0000","last_change":"2020-08-07T15:06:35.170553+0000","last_active":"2020-08-07T15:07:01.683218+0000","last_peered":"2020-08-07T15:07:01.683218+0000","last_clean":"2020-08-07T15:07:01.683218+0000","last_became_active":"2020-08-07T15:06:35.170037+0000","last_became_peered":"2020-08-07T15:06:35.170037+0000","last_unstale":"2020-08-07T15:07:01.683218+0000","last_undegraded":"2020-08-07T15:07:01.683218+0000","last_fullsized":"2020-08-07T15:07:01.683218+0000","mapping_epoch":20,"log_start":"31'2576","ondisk_log_start":"31'2576","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21159936,"num_objects":323,"num_object_clones":0,"num_object_copies":969,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2583,"num_write_kb":20664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"22","reported_epoch":"30","state":"active+clean","last_fresh":"2020-08-07T15:06:56.238796+0000","last_change":"2020-08-07T15:06:23.075958+0000","last_active":"2020-08-07T15:06:56.238796+0000","last_peered":"2020-08-07T15:06:56.238796+0000","last_clean":"2020-08-07T15:06:56.238796+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:06:56.238796+0000","last_undegraded":"2020-08-07T15:06:56.238796+0000","last_fullsized":"2020-08-07T15:06:56.238796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"29","reported_epoch":"30","state":"active+clean","last_fresh":"2020-08-07T15:06:56.247743+0000","last_change":"2020-08-07T15:06:19.198215+0000","last_active":"2020-08-07T15:06:56.247743+0000","last_peered":"2020-08-07T15:06:56.247743+0000","last_clean":"2020-08-07T15:06:56.247743+0000","last_became_active":"2020-08-07T15:06:19.197819+0000","last_became_peered":"2020-08-07T15:06:19.197819+0000","last_unstale":"2020-08-07T15:06:56.247743+0000","last_undegraded":"2020-08-07T15:06:56.247743+0000","last_fullsized":"2020-08-07T15:06:56.247743+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:06:10.878833+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"31'2184","reported_seq":"2203","reported_epoch":"31","state":"active+clean","last_fresh":"2020-08-07T15:07:01.680036+0000","last_change":"2020-08-07T15:06:35.169888+0000","last_active":"2020-08-07T15:07:01.680036+0000","last_peered":"2020-08-07T15:07:01.680036+0000","last_clean":"2020-08-07T15:07:01.680036+0000","last_became_active":"2020-08-07T15:06:35.169385+0000","last_became_peered":"2020-08-07T15:06:35.169385+0000","last_unstale":"2020-08-07T15:07:01.680036+0000","last_undegraded":"2020-08-07T15:07:01.680036+0000","last_fullsized":"2020-08-07T15:07:01.680036+0000","mapping_epoch":20,"log_start":"31'2178","ondisk_log_start":"31'2178","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":17891328,"num_objects":273,"num_object_clones":0,"num_object_copies":819,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2184,"num_write_kb":17472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"22","reported_epoch":"30","state":"active+clean","last_fresh":"2020-08-07T15:06:56.247922+0000","last_change":"2020-08-07T15:06:23.224115+0000","last_active":"2020-08-07T15:06:56.247922+0000","last_peered":"2020-08-07T15:06:56.247922+0000","last_clean":"2020-08-07T15:06:56.247922+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:06:56.247922+0000","last_undegraded":"2020-08-07T15:06:56.247922+0000","last_fullsized":"2020-08-07T15:06:56.247922+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"31'2384","reported_seq":"2403","reported_epoch":"31","state":"active+clean","last_fresh":"2020-08-07T15:07:01.463609+0000","last_change":"2020-08-07T15:06:46.214618+0000","last_active":"2020-08-07T15:07:01.463609+0000","last_peered":"2020-08-07T15:07:01.463609+0000","last_clean":"2020-08-07T15:07:01.463609+0000","last_became_active":"2020-08-07T15:06:46.202192+0000","last_became_peered":"2020-08-07T15:06:46.202192+0000","last_unstale":"2020-08-07T15:07:01.463609+0000","last_undegraded":"2020-08-07T15:07:01.463609+0000","last_fullsized":"2020-08-07T15:07:01.463609+0000","mapping_epoch":26,"log_start":"31'2376","ondisk_log_start":"31'2376","created":20,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19529728,"num_objects":298,"num_object_clones":0,"num_object_copies":894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2384,"num_write_kb":19072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3],"acting":[4,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"22","reported_epoch":"30","state":"active+clean","last_fresh":"2020-08-07T15:06:56.248084+0000","last_change":"2020-08-07T15:06:23.225886+0000","last_active":"2020-08-07T15:06:56.248084+0000","last_peered":"2020-08-07T15:06:56.248084+0000","last_clean":"2020-08-07T15:06:56.248084+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:06:56.248084+0000","last_undegraded":"2020-08-07T15:06:56.248084+0000","last_fullsized":"2020-08-07T15:06:56.248084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"31'2328","reported_seq":"2347","reported_epoch":"31","state":"active+clean","last_fresh":"2020-08-07T15:07:01.619812+0000","last_change":"2020-08-07T15:06:35.169626+0000","last_active":"2020-08-07T15:07:01.619812+0000","last_peered":"2020-08-07T15:07:01.619812+0000","last_clean":"2020-08-07T15:07:01.619812+0000","last_became_active":"2020-08-07T15:06:35.169127+0000","last_became_peered":"2020-08-07T15:06:35.169127+0000","last_unstale":"2020-08-07T15:07:01.619812+0000","last_undegraded":"2020-08-07T15:07:01.619812+0000","last_fullsized":"2020-08-07T15:07:01.619812+0000","mapping_epoch":20,"log_start":"31'2321","ondisk_log_start":"31'2321","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19070976,"num_objects":291,"num_object_clones":0,"num_object_copies":873,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2328,"num_write_kb":18624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"31'2336","reported_seq":"2355","reported_epoch":"31","state":"active+clean","last_fresh":"2020-08-07T15:07:01.572800+0000","last_change":"2020-08-07T15:06:35.172780+0000","last_active":"2020-08-07T15:07:01.572800+0000","last_peered":"2020-08-07T15:07:01.572800+0000","last_clean":"2020-08-07T15:07:01.572800+0000","last_became_active":"2020-08-07T15:06:35.172360+0000","last_became_peered":"2020-08-07T15:06:35.172360+0000","last_unstale":"2020-08-07T15:07:01.572800+0000","last_undegraded":"2020-08-07T15:07:01.572800+0000","last_fullsized":"2020-08-07T15:07:01.572800+0000","mapping_epoch":20,"log_start":"31'2329","ondisk_log_start":"31'2329","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19136512,"num_objects":292,"num_object_clones":0,"num_object_copies":876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2336,"num_write_kb":18688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"22","reported_epoch":"30","state":"active+clean","last_fresh":"2020-08-07T15:06:56.285090+0000","last_change":"2020-08-07T15:06:23.074076+0000","last_active":"2020-08-07T15:06:56.285090+0000","last_peered":"2020-08-07T15:06:56.285090+0000","last_clean":"2020-08-07T15:06:56.285090+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:06:56.285090+0000","last_undegraded":"2020-08-07T15:06:56.285090+0000","last_fullsized":"2020-08-07T15:06:56.285090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"23","reported_epoch":"31","state":"active+clean","last_fresh":"2020-08-07T15:06:59.293263+0000","last_change":"2020-08-07T15:06:23.073702+0000","last_active":"2020-08-07T15:06:59.293263+0000","last_peered":"2020-08-07T15:06:59.293263+0000","last_clean":"2020-08-07T15:06:59.293263+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:06:59.293263+0000","last_undegraded":"2020-08-07T15:06:59.293263+0000","last_fullsized":"2020-08-07T15:06:59.293263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"31'2608","reported_seq":"2629","reported_epoch":"31","state":"active+clean","last_fresh":"2020-08-07T15:07:01.671761+0000","last_change":"2020-08-07T15:06:46.199635+0000","last_active":"2020-08-07T15:07:01.671761+0000","last_peered":"2020-08-07T15:07:01.671761+0000","last_clean":"2020-08-07T15:07:01.671761+0000","last_became_active":"2020-08-07T15:06:46.199201+0000","last_became_peered":"2020-08-07T15:06:46.199201+0000","last_unstale":"2020-08-07T15:07:01.671761+0000","last_undegraded":"2020-08-07T15:07:01.671761+0000","last_fullsized":"2020-08-07T15:07:01.671761+0000","mapping_epoch":26,"log_start":"31'2603","ondisk_log_start":"31'2603","created":20,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21364736,"num_objects":326,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2608,"num_write_kb":20864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":324861952,"num_objects":4958,"num_object_clones":0,"num_object_copies":14874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39656,"num_write_kb":317248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":977174528,"data_stored":977174528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542155,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1169336,"kb_used_data":120752,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93202504,"statfs":{"total":96636764160,"available":95439364096,"internally_reserved":1073741824,"allocated":123650048,"data_stored":123456301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2360000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.371}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6779999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2679999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7290000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1389999999999998}]}]},{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1168720,"kb_used_data":120136,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93203120,"statfs":{"total":96636764160,"available":95439994880,"internally_reserved":1073741824,"allocated":123019264,"data_stored":122825517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2110000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1171004,"kb_used_data":122420,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93200836,"statfs":{"total":96636764160,"available":95437656064,"internally_reserved":1073741824,"allocated":125358080,"data_stored":125167601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.202}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.403}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2410000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.327}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3730000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1166812,"kb_used_data":118228,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93205028,"statfs":{"total":96636764160,"available":95441948672,"internally_reserved":1073741824,"allocated":121065472,"data_stored":120875243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3380000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1125608,"kb_used_data":77024,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93246232,"statfs":{"total":96636764160,"available":95484141568,"internally_reserved":1073741824,"allocated":78872576,"data_stored":78685615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3759999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4019999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2280000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.25}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.28}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1180980,"kb_used_data":132396,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93190860,"statfs":{"total":96636764160,"available":95427440640,"internally_reserved":1073741824,"allocated":135573504,"data_stored":135383025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,5,1,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3380000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.228}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.2030000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1209192,"kb_used_data":160608,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93162648,"statfs":{"total":96636764160,"available":95398551552,"internally_reserved":1073741824,"allocated":164462592,"data_stored":164268845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4910000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1143288,"kb_used_data":94704,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93228552,"statfs":{"total":96636764160,"available":95466037248,"internally_reserved":1073741824,"allocated":96976896,"data_stored":96783149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]}]}]}} 2020-08-07T15:07:05.382 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:07:05.382 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 29 16 45372 45356 12.2177 12.5469 0.00357089 0.0102254 2020-08-07T15:07:05.384 INFO:tasks.thrashosds.thrasher:Setting 3.d to [3, 4, 6] 2020-08-07T15:07:05.385 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.d', '3', '4', '6'] 2020-08-07T15:07:05.385 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pg-upmap 3.d 3 4 6 2020-08-07T15:07:05.388 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:07:05.388 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:05.367+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:05.389 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:07:05.471 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:05.470+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:05.574 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:05.572+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:05.610 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:05.610 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:05.624 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:07:05.708 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:05.706+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:05.776 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:05.774+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:05.846 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:05.847 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:05.860 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:07:05.877 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:05.876+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:05.978 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:05.976+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:06.064 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:06.064 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:06.070 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:07:06.088 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:06.081+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:06.183 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:06.181+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:06.248 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 30 16 47003 46987 12.2352 12.7422 0.00539088 0.0102052 2020-08-07T15:07:06.248 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:06.249 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:06.257 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:07:06.296 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:06.289+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:06.337 INFO:teuthology.orchestra.run.smithi074.stderr:set 3.d pg_upmap mapping to [3,4,6] 2020-08-07T15:07:06.394 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:06.389+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:06.424 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:06.424 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:06.439 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:07:06.488 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:06.486+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:06.586 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:06.586 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:06.588 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:07:06.595 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:06.592+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:06.701 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:06.691+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:06.738 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:06.738 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:06.739 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:07:06.794 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:06.792+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:06.895 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:06.893+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:06.895 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:06.895 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:06.915 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:07:07.009 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:07.001+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:07.083 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:07.084 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:07.084 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:07:07.100 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:07.097+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:07.198 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:07.197+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:07.243 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 31 16 47136 47120 11.874 1.03906 0.869044 0.0102655 2020-08-07T15:07:07.245 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:07.245 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:07.254 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:07:07.300 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:07.299+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:07.376 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:07.376 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:07.380 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:07:07.382 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:07:07.431 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:07:07.433 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:07.431+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:07.541 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:07.532+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:07.580 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:07.581 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:07.600 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:07:07.601 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:07:07.627 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:07:07.634 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:07.633+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:07.647 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:07:07.670 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:07:07.694 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:07:07.719 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:07:07.736 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:07.735+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:07.737 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:07.737 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:07:07.737 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2020-08-07T15:07:07.738 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:07.744 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:07:07.757 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:07:07.837 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:07.836+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:07.920 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:07.920 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:07:07.921 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:07:07.921 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:07:07.921 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:07.929 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:07:07.939 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:07.938+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:07.970 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:07:07.978 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:07:07.979 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:07:07.979 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:07:07.979 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:07:07.979 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:07:07.980 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:07:07.980 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:07:07.980 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:07:08.044 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:08.043+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:08.057 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:08.057 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:07:08.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:07:08.057 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:08.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.058 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:2600 3.b e23/20 3:d6a1a0d3:::benchmark_data_smithi074_35447_object324:head v 23'208, mlcod=23'201)", 2020-08-07T15:07:08.058 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.807190+0000", 2020-08-07T15:07:08.058 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 30.246703429, 2020-08-07T15:07:08.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20835313, 2020-08-07T15:07:08.059 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:08.059 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:08.059 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:08.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:08.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.807190+0000", 2020-08-07T15:07:08.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:08.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:08.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.807190+0000", 2020-08-07T15:07:08.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.938e-05 2020-08-07T15:07:08.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:08.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.807219+0000", 2020-08-07T15:07:08.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2780000000000004e-06 2020-08-07T15:07:08.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.807223+0000", 2020-08-07T15:07:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20663868699999999 2020-08-07T15:07:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:08.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.013862+0000", 2020-08-07T15:07:08.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.659e-06 2020-08-07T15:07:08.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:08.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.013866+0000", 2020-08-07T15:07:08.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7111999999999999e-05 2020-08-07T15:07:08.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:08.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.013913+0000", 2020-08-07T15:07:08.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5225999999999999e-05 2020-08-07T15:07:08.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:08.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.013938+0000", 2020-08-07T15:07:08.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015643429999999999 2020-08-07T15:07:08.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:08.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.015502+0000", 2020-08-07T15:07:08.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0445000000000003e-05 2020-08-07T15:07:08.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:08.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.015543+0000", 2020-08-07T15:07:08.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20835313 2020-08-07T15:07:08.070 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.071 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:08.071 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.073 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:7576 3.b e24/20 3:d854a547:::benchmark_data_smithi074_35447_object946:head v 24'456, mlcod=24'448)", 2020-08-07T15:07:08.073 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:40.741593+0000", 2020-08-07T15:07:08.073 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.312300384, 2020-08-07T15:07:08.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18773600700000001, 2020-08-07T15:07:08.073 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:08.074 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:08.074 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:08.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:08.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.741593+0000", 2020-08-07T15:07:08.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:08.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:08.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.741593+0000", 2020-08-07T15:07:08.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1712000000000002e-05 2020-08-07T15:07:08.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:08.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.741645+0000", 2020-08-07T15:07:08.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2389999999999999e-06 2020-08-07T15:07:08.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:08.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.741651+0000", 2020-08-07T15:07:08.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18616382400000001 2020-08-07T15:07:08.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:08.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.927815+0000", 2020-08-07T15:07:08.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7359999999999999e-06 2020-08-07T15:07:08.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:08.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.927818+0000", 2020-08-07T15:07:08.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3000000000000002e-05 2020-08-07T15:07:08.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:08.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.927861+0000", 2020-08-07T15:07:08.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4202e-05 2020-08-07T15:07:08.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:08.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.927886+0000", 2020-08-07T15:07:08.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014097070000000001 2020-08-07T15:07:08.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:08.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.929295+0000", 2020-08-07T15:07:08.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3587000000000002e-05 2020-08-07T15:07:08.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:08.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.929329+0000", 2020-08-07T15:07:08.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18773600700000001 2020-08-07T15:07:08.084 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.085 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:08.085 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.086 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:7848 3.9 3:905b1512:::benchmark_data_smithi074_35447_object980:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:08.086 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:40.895061+0000", 2020-08-07T15:07:08.086 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.158831825, 2020-08-07T15:07:08.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.187604353, 2020-08-07T15:07:08.087 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:08.087 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:08.087 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:08.087 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:08.088 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:08.088 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7848 2020-08-07T15:07:08.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.088 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:08.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:08.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.895061+0000", 2020-08-07T15:07:08.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:08.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:08.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.895061+0000", 2020-08-07T15:07:08.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7772999999999997e-05 2020-08-07T15:07:08.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:08.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.895119+0000", 2020-08-07T15:07:08.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3860000000000004e-06 2020-08-07T15:07:08.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:08.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.895126+0000", 2020-08-07T15:07:08.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1783000000000001e-05 2020-08-07T15:07:08.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:08.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.895137+0000", 2020-08-07T15:07:08.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6629999999999998e-06 2020-08-07T15:07:08.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:08.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.895143+0000", 2020-08-07T15:07:08.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024182299999999999 2020-08-07T15:07:08.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:08.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.897561+0000", 2020-08-07T15:07:08.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00098884499999999992 2020-08-07T15:07:08.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:08.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.898550+0000", 2020-08-07T15:07:08.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015507699999999999 2020-08-07T15:07:08.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:08.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.898705+0000", 2020-08-07T15:07:08.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021473949999999999 2020-08-07T15:07:08.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:08.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.900853+0000", 2020-08-07T15:07:08.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001259788 2020-08-07T15:07:08.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.902112+0000", 2020-08-07T15:07:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.180465141 2020-08-07T15:07:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.082577+0000", 2020-08-07T15:07:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9599999999999998e-07 2020-08-07T15:07:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.082578+0000", 2020-08-07T15:07:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9520000000000001e-05 2020-08-07T15:07:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.082618+0000", 2020-08-07T15:07:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7956e-05 2020-08-07T15:07:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.082666+0000", 2020-08-07T15:07:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.187604353 2020-08-07T15:07:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:8112 3.9 3:9b275d48:::benchmark_data_smithi074_35447_object1013:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.083981+0000", 2020-08-07T15:07:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.969912366999999, 2020-08-07T15:07:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31911657999999998, 2020-08-07T15:07:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 8112 2020-08-07T15:07:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.083981+0000", 2020-08-07T15:07:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.083981+0000", 2020-08-07T15:07:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4051999999999999e-05 2020-08-07T15:07:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084015+0000", 2020-08-07T15:07:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7290000000000002e-06 2020-08-07T15:07:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:08.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084019+0000", 2020-08-07T15:07:08.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2280000000000001e-06 2020-08-07T15:07:08.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:08.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084026+0000", 2020-08-07T15:07:08.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7780000000000001e-06 2020-08-07T15:07:08.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:08.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084030+0000", 2020-08-07T15:07:08.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00077372300000000003 2020-08-07T15:07:08.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:08.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084803+0000", 2020-08-07T15:07:08.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1502000000000003e-05 2020-08-07T15:07:08.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:08.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084855+0000", 2020-08-07T15:07:08.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5110000000000002e-05 2020-08-07T15:07:08.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:08.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084950+0000", 2020-08-07T15:07:08.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012531370000000001 2020-08-07T15:07:08.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:08.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.086203+0000", 2020-08-07T15:07:08.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4799999999999998e-07 2020-08-07T15:07:08.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:08.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.086204+0000", 2020-08-07T15:07:08.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.905e-05 2020-08-07T15:07:08.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:08.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.086253+0000", 2020-08-07T15:07:08.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00062940999999999995 2020-08-07T15:07:08.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:08.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.086882+0000", 2020-08-07T15:07:08.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31612944199999998 2020-08-07T15:07:08.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:08.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.403012+0000", 2020-08-07T15:07:08.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9099999999999991e-07 2020-08-07T15:07:08.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:08.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.403013+0000", 2020-08-07T15:07:08.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.488e-05 2020-08-07T15:07:08.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:08.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.403097+0000", 2020-08-07T15:07:08.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31911657999999998 2020-08-07T15:07:08.123 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.124 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:08.124 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.124 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:8760 3.9 3:9f02d263:::benchmark_data_smithi074_35447_object1094:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:08.125 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.531357+0000", 2020-08-07T15:07:08.125 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.522535983000001, 2020-08-07T15:07:08.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19863291199999999, 2020-08-07T15:07:08.125 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:08.125 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:08.126 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:08.126 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:08.126 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:08.126 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 8760 2020-08-07T15:07:08.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.126 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:08.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:08.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.531357+0000", 2020-08-07T15:07:08.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:08.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:08.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.531357+0000", 2020-08-07T15:07:08.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9942999999999999e-05 2020-08-07T15:07:08.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:08.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.531407+0000", 2020-08-07T15:07:08.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8319999999999996e-06 2020-08-07T15:07:08.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:08.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.531412+0000", 2020-08-07T15:07:08.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0128e-05 2020-08-07T15:07:08.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:08.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.531422+0000", 2020-08-07T15:07:08.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2370000000000003e-06 2020-08-07T15:07:08.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:08.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.531426+0000", 2020-08-07T15:07:08.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034586119999999998 2020-08-07T15:07:08.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:08.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.534885+0000", 2020-08-07T15:07:08.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6256999999999997e-05 2020-08-07T15:07:08.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:08.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.534981+0000", 2020-08-07T15:07:08.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000171373 2020-08-07T15:07:08.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:08.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.535153+0000", 2020-08-07T15:07:08.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018094879999999999 2020-08-07T15:07:08.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:08.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.536962+0000", 2020-08-07T15:07:08.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9400000000000005e-07 2020-08-07T15:07:08.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:08.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.536963+0000", 2020-08-07T15:07:08.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7188999999999997e-05 2020-08-07T15:07:08.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:08.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.537030+0000", 2020-08-07T15:07:08.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034211371999999997 2020-08-07T15:07:08.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:08.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.571241+0000", 2020-08-07T15:07:08.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15868652799999999 2020-08-07T15:07:08.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:08.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.729928+0000", 2020-08-07T15:07:08.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0139999999999999e-06 2020-08-07T15:07:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.729929+0000", 2020-08-07T15:07:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1245000000000001e-05 2020-08-07T15:07:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:08.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.729990+0000", 2020-08-07T15:07:08.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19863291199999999 2020-08-07T15:07:08.141 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.142 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:08.142 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.142 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:16223 3.9 3:9c72cb85:::benchmark_data_smithi074_35447_object2027:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:08.143 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:47.106937+0000", 2020-08-07T15:07:08.143 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.946955761000002, 2020-08-07T15:07:08.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.48645846999999998, 2020-08-07T15:07:08.143 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:08.143 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:08.144 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:08.144 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:08.144 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:08.144 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 16223 2020-08-07T15:07:08.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.144 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:08.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:08.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.106937+0000", 2020-08-07T15:07:08.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:08.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:08.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.106937+0000", 2020-08-07T15:07:08.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8390999999999998e-05 2020-08-07T15:07:08.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:08.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.106986+0000", 2020-08-07T15:07:08.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0120000000000002e-06 2020-08-07T15:07:08.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:08.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.106992+0000", 2020-08-07T15:07:08.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3148e-05 2020-08-07T15:07:08.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:08.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.107005+0000", 2020-08-07T15:07:08.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5719999999999999e-06 2020-08-07T15:07:08.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:08.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.107012+0000", 2020-08-07T15:07:08.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043495110000000003 2020-08-07T15:07:08.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:08.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.111361+0000", 2020-08-07T15:07:08.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0105999999999997e-05 2020-08-07T15:07:08.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:08.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.111421+0000", 2020-08-07T15:07:08.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012232399999999999 2020-08-07T15:07:08.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:08.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.111544+0000", 2020-08-07T15:07:08.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003099367 2020-08-07T15:07:08.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:08.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.114643+0000", 2020-08-07T15:07:08.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3099999999999997e-07 2020-08-07T15:07:08.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:08.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.114644+0000", 2020-08-07T15:07:08.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7627000000000003e-05 2020-08-07T15:07:08.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:08.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.114691+0000", 2020-08-07T15:07:08.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005765905 2020-08-07T15:07:08.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:08.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.120457+0000", 2020-08-07T15:07:08.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.47285765200000002 2020-08-07T15:07:08.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:08.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.593315+0000", 2020-08-07T15:07:08.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1969999999999999e-06 2020-08-07T15:07:08.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:08.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.593316+0000", 2020-08-07T15:07:08.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0026999999999998e-05 2020-08-07T15:07:08.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:08.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.593396+0000", 2020-08-07T15:07:08.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.48645846999999998 2020-08-07T15:07:08.161 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.161 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:08.161 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.162 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:16224 3.9 3:9c72cb85:::benchmark_data_smithi074_35447_object2027:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:08.162 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:47.108015+0000", 2020-08-07T15:07:08.162 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.945878023999999, 2020-08-07T15:07:08.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.512944332, 2020-08-07T15:07:08.162 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:08.163 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:08.163 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:08.163 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:08.163 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:08.163 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 16224 2020-08-07T15:07:08.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.164 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:08.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:08.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108015+0000", 2020-08-07T15:07:08.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:08.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:08.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108015+0000", 2020-08-07T15:07:08.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7508e-05 2020-08-07T15:07:08.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:08.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108063+0000", 2020-08-07T15:07:08.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.485e-06 2020-08-07T15:07:08.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:08.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108068+0000", 2020-08-07T15:07:08.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3387000000000001e-05 2020-08-07T15:07:08.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:08.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108082+0000", 2020-08-07T15:07:08.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1129999999999997e-06 2020-08-07T15:07:08.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:08.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108088+0000", 2020-08-07T15:07:08.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056051920000000002 2020-08-07T15:07:08.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:08.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.113693+0000", 2020-08-07T15:07:08.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9973000000000001e-05 2020-08-07T15:07:08.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:08.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.113783+0000", 2020-08-07T15:07:08.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001649 2020-08-07T15:07:08.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:08.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.113948+0000", 2020-08-07T15:07:08.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001023007 2020-08-07T15:07:08.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:08.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.114971+0000", 2020-08-07T15:07:08.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2600000000000001e-07 2020-08-07T15:07:08.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:08.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.114972+0000", 2020-08-07T15:07:08.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9228999999999998e-05 2020-08-07T15:07:08.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:08.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.115031+0000", 2020-08-07T15:07:08.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0063345550000000004 2020-08-07T15:07:08.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:08.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.121365+0000", 2020-08-07T15:07:08.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.49952866099999998 2020-08-07T15:07:08.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:08.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.620894+0000", 2020-08-07T15:07:08.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9299999999999996e-07 2020-08-07T15:07:08.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:08.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.620895+0000", 2020-08-07T15:07:08.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4503000000000002e-05 2020-08-07T15:07:08.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:08.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.620960+0000", 2020-08-07T15:07:08.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.512944332 2020-08-07T15:07:08.182 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.182 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:08.182 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.183 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:23896 3.a e28/20 3:55e62b6c:::benchmark_data_smithi074_35447_object2986:head v 28'1648, mlcod=28'1639)", 2020-08-07T15:07:08.183 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:52.053957+0000", 2020-08-07T15:07:08.183 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.999935749, 2020-08-07T15:07:08.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26202785699999998, 2020-08-07T15:07:08.184 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:08.184 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:08.185 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:08.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:08.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.053957+0000", 2020-08-07T15:07:08.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:08.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:08.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.053957+0000", 2020-08-07T15:07:08.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4489e-05 2020-08-07T15:07:08.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:08.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.054012+0000", 2020-08-07T15:07:08.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.314e-06 2020-08-07T15:07:08.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:08.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.054018+0000", 2020-08-07T15:07:08.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26134933700000001 2020-08-07T15:07:08.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:08.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.315368+0000", 2020-08-07T15:07:08.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.783e-06 2020-08-07T15:07:08.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:08.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.315371+0000", 2020-08-07T15:07:08.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0726000000000003e-05 2020-08-07T15:07:08.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:08.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.315412+0000", 2020-08-07T15:07:08.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7337999999999999e-05 2020-08-07T15:07:08.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:08.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.315429+0000", 2020-08-07T15:07:08.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051664600000000005 2020-08-07T15:07:08.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:08.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.315946+0000", 2020-08-07T15:07:08.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9224000000000003e-05 2020-08-07T15:07:08.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:08.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.315985+0000", 2020-08-07T15:07:08.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26202785699999998 2020-08-07T15:07:08.196 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.196 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:08.196 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.197 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:27048 3.1 3:88501d52:::benchmark_data_smithi074_35447_object3380:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:08.197 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:54.042197+0000", 2020-08-07T15:07:08.197 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.011695843, 2020-08-07T15:07:08.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.163294305, 2020-08-07T15:07:08.198 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:08.198 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:08.198 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:08.198 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:08.199 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:08.199 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 27048 2020-08-07T15:07:08.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.199 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:08.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:08.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.042197+0000", 2020-08-07T15:07:08.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:08.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:08.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.042197+0000", 2020-08-07T15:07:08.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2139999999999999e-05 2020-08-07T15:07:08.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:08.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.042250+0000", 2020-08-07T15:07:08.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4500000000000001e-06 2020-08-07T15:07:08.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:08.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.042256+0000", 2020-08-07T15:07:08.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2001000000000001e-05 2020-08-07T15:07:08.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:08.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.042268+0000", 2020-08-07T15:07:08.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0630000000000002e-06 2020-08-07T15:07:08.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:08.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.042274+0000", 2020-08-07T15:07:08.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003240162 2020-08-07T15:07:08.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:08.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.045514+0000", 2020-08-07T15:07:08.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9018999999999996e-05 2020-08-07T15:07:08.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:08.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.045603+0000", 2020-08-07T15:07:08.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018029600000000001 2020-08-07T15:07:08.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,3", 2020-08-07T15:07:08.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.045784+0000", 2020-08-07T15:07:08.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014937170000000001 2020-08-07T15:07:08.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:08.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.047277+0000", 2020-08-07T15:07:08.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030336109999999999 2020-08-07T15:07:08.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:08.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.050311+0000", 2020-08-07T15:07:08.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.155054622 2020-08-07T15:07:08.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:08.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.205365+0000", 2020-08-07T15:07:08.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1999999999999999e-07 2020-08-07T15:07:08.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:08.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.205366+0000", 2020-08-07T15:07:08.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3563e-05 2020-08-07T15:07:08.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:08.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.205440+0000", 2020-08-07T15:07:08.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1941000000000003e-05 2020-08-07T15:07:08.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:08.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.205492+0000", 2020-08-07T15:07:08.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.163294305 2020-08-07T15:07:08.224 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.224 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:08.224 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.225 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:27256 3.6 e29/26 3:69466024:::benchmark_data_smithi074_35447_object3406:head v 29'1832, mlcod=29'1824)", 2020-08-07T15:07:08.225 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:54.173701+0000", 2020-08-07T15:07:08.225 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.880191786999999, 2020-08-07T15:07:08.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.17083079200000001, 2020-08-07T15:07:08.225 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:08.226 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:08.226 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:08.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:08.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.173701+0000", 2020-08-07T15:07:08.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:08.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:08.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.173701+0000", 2020-08-07T15:07:08.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5343000000000001e-05 2020-08-07T15:07:08.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:08.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.173727+0000", 2020-08-07T15:07:08.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.847e-06 2020-08-07T15:07:08.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:08.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.173730+0000", 2020-08-07T15:07:08.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.17015693800000001 2020-08-07T15:07:08.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:08.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.343887+0000", 2020-08-07T15:07:08.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1989999999999997e-06 2020-08-07T15:07:08.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:08.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.343892+0000", 2020-08-07T15:07:08.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4974e-05 2020-08-07T15:07:08.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:08.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.343937+0000", 2020-08-07T15:07:08.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4836000000000001e-05 2020-08-07T15:07:08.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:08.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.343962+0000", 2020-08-07T15:07:08.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00054361600000000002 2020-08-07T15:07:08.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:08.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.344505+0000", 2020-08-07T15:07:08.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7039e-05 2020-08-07T15:07:08.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:08.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.344532+0000", 2020-08-07T15:07:08.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.17083079200000001 2020-08-07T15:07:08.241 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.242 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:08.242 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.243 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:33728 3.1 3:8fce2265:::benchmark_data_smithi074_35447_object4215:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-08-07T15:07:08.243 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:58.156056+0000", 2020-08-07T15:07:08.243 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.8978376229999991, 2020-08-07T15:07:08.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26861316299999999, 2020-08-07T15:07:08.245 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:08.246 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:08.246 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:08.246 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:08.246 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:08.246 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 33728 2020-08-07T15:07:08.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.247 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:08.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:08.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156056+0000", 2020-08-07T15:07:08.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:08.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:08.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156056+0000", 2020-08-07T15:07:08.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6795999999999998e-05 2020-08-07T15:07:08.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:08.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156092+0000", 2020-08-07T15:07:08.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4690000000000002e-06 2020-08-07T15:07:08.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:08.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156096+0000", 2020-08-07T15:07:08.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5270000000000004e-06 2020-08-07T15:07:08.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:08.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156102+0000", 2020-08-07T15:07:08.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3129999999999999e-06 2020-08-07T15:07:08.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:08.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156106+0000", 2020-08-07T15:07:08.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00337032 2020-08-07T15:07:08.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:08.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.159476+0000", 2020-08-07T15:07:08.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104874 2020-08-07T15:07:08.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:08.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.159581+0000", 2020-08-07T15:07:08.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016822300000000001 2020-08-07T15:07:08.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,3", 2020-08-07T15:07:08.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.159749+0000", 2020-08-07T15:07:08.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020156620000000001 2020-08-07T15:07:08.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:08.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.161765+0000", 2020-08-07T15:07:08.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1699999999999997e-07 2020-08-07T15:07:08.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:08.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.161765+0000", 2020-08-07T15:07:08.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6606e-05 2020-08-07T15:07:08.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:08.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.161822+0000", 2020-08-07T15:07:08.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0060344459999999997 2020-08-07T15:07:08.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:08.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.167857+0000", 2020-08-07T15:07:08.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25676037099999999 2020-08-07T15:07:08.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:08.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424617+0000", 2020-08-07T15:07:08.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3370000000000001e-06 2020-08-07T15:07:08.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:08.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424618+0000", 2020-08-07T15:07:08.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0501999999999999e-05 2020-08-07T15:07:08.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:08.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424669+0000", 2020-08-07T15:07:08.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26861316299999999 2020-08-07T15:07:08.272 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.273 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:08.274 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.274 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:33728 3.1 e30/26 ondisk, result = 0)", 2020-08-07T15:07:08.274 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:58.167159+0000", 2020-08-07T15:07:08.275 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.8867345909999997, 2020-08-07T15:07:08.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25751397999999998, 2020-08-07T15:07:08.275 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:08.275 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:08.275 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:08.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:08.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.167159+0000", 2020-08-07T15:07:08.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:08.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:08.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.167159+0000", 2020-08-07T15:07:08.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3473999999999998e-05 2020-08-07T15:07:08.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:08.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.167192+0000", 2020-08-07T15:07:08.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1480000000000003e-06 2020-08-07T15:07:08.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:08.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.167198+0000", 2020-08-07T15:07:08.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25735533700000002 2020-08-07T15:07:08.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:08.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424554+0000", 2020-08-07T15:07:08.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2959999999999998e-06 2020-08-07T15:07:08.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:08.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424559+0000", 2020-08-07T15:07:08.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0877000000000002e-05 2020-08-07T15:07:08.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:08.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424600+0000", 2020-08-07T15:07:08.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5580000000000001e-06 2020-08-07T15:07:08.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:08.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424608+0000", 2020-08-07T15:07:08.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4289999999999996e-05 2020-08-07T15:07:08.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:08.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424673+0000", 2020-08-07T15:07:08.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25751397999999998 2020-08-07T15:07:08.288 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.288 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:08.288 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.289 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:34624 3.9 3:946cdb13:::benchmark_data_smithi074_35447_object4327:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-08-07T15:07:08.289 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:58.744745+0000", 2020-08-07T15:07:08.289 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.3091485039999995, 2020-08-07T15:07:08.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28897188600000001, 2020-08-07T15:07:08.290 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:08.290 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:08.290 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:08.290 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:08.290 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:08.291 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 34624 2020-08-07T15:07:08.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.291 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:08.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:08.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744745+0000", 2020-08-07T15:07:08.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:08.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:08.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744745+0000", 2020-08-07T15:07:08.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1117e-05 2020-08-07T15:07:08.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:08.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744776+0000", 2020-08-07T15:07:08.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2140000000000001e-06 2020-08-07T15:07:08.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:08.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744782+0000", 2020-08-07T15:07:08.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1756999999999999e-05 2020-08-07T15:07:08.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:08.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744794+0000", 2020-08-07T15:07:08.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.674e-06 2020-08-07T15:07:08.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:08.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744800+0000", 2020-08-07T15:07:08.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012818110000000001 2020-08-07T15:07:08.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:08.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.746082+0000", 2020-08-07T15:07:08.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2163999999999997e-05 2020-08-07T15:07:08.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:08.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.746134+0000", 2020-08-07T15:07:08.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107853 2020-08-07T15:07:08.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:08.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.746242+0000", 2020-08-07T15:07:08.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00119732 2020-08-07T15:07:08.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:08.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.747440+0000", 2020-08-07T15:07:08.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5799999999999998e-07 2020-08-07T15:07:08.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:08.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.747440+0000", 2020-08-07T15:07:08.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0883000000000001e-05 2020-08-07T15:07:08.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:08.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.747481+0000", 2020-08-07T15:07:08.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000272355 2020-08-07T15:07:08.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:08.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.747754+0000", 2020-08-07T15:07:08.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28589292999999999 2020-08-07T15:07:08.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:08.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.033647+0000", 2020-08-07T15:07:08.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6600000000000005e-07 2020-08-07T15:07:08.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:08.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.033647+0000", 2020-08-07T15:07:08.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9183999999999994e-05 2020-08-07T15:07:08.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:08.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.033717+0000", 2020-08-07T15:07:08.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28897188600000001 2020-08-07T15:07:08.310 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.310 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:08.310 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.311 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:35280 3.6 e30/26 3:63cda715:::benchmark_data_smithi074_35447_object4409:head v 30'2424, mlcod=30'2420)", 2020-08-07T15:07:08.311 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:59.114249+0000", 2020-08-07T15:07:08.312 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.9396444810000002, 2020-08-07T15:07:08.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.208357021, 2020-08-07T15:07:08.312 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:08.312 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:08.312 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:08.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:08.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.114249+0000", 2020-08-07T15:07:08.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:08.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:08.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.114249+0000", 2020-08-07T15:07:08.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5002000000000001e-05 2020-08-07T15:07:08.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:08.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.114284+0000", 2020-08-07T15:07:08.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9580000000000001e-06 2020-08-07T15:07:08.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:08.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.114288+0000", 2020-08-07T15:07:08.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20552400700000001 2020-08-07T15:07:08.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:08.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.319812+0000", 2020-08-07T15:07:08.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2610000000000002e-06 2020-08-07T15:07:08.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:08.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.319815+0000", 2020-08-07T15:07:08.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7545000000000005e-05 2020-08-07T15:07:08.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:08.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.319883+0000", 2020-08-07T15:07:08.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4219e-05 2020-08-07T15:07:08.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:08.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.319907+0000", 2020-08-07T15:07:08.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026555049999999998 2020-08-07T15:07:08.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:08.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.322562+0000", 2020-08-07T15:07:08.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3523999999999999e-05 2020-08-07T15:07:08.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:08.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.322606+0000", 2020-08-07T15:07:08.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.208357021 2020-08-07T15:07:08.324 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.325 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:08.325 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.325 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:35880 3.9 3:905cfca2:::benchmark_data_smithi074_35447_object4484:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-08-07T15:07:08.326 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:59.449666+0000", 2020-08-07T15:07:08.326 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.6042275670000006, 2020-08-07T15:07:08.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18151558000000001, 2020-08-07T15:07:08.326 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:08.326 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:08.327 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:08.327 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:08.327 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:08.327 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 35880 2020-08-07T15:07:08.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.328 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:08.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:08.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.449666+0000", 2020-08-07T15:07:08.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:08.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:08.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.449666+0000", 2020-08-07T15:07:08.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6263000000000001e-05 2020-08-07T15:07:08.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:08.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.449712+0000", 2020-08-07T15:07:08.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2740000000000004e-06 2020-08-07T15:07:08.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:08.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.449718+0000", 2020-08-07T15:07:08.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1015999999999999e-05 2020-08-07T15:07:08.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:08.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.449729+0000", 2020-08-07T15:07:08.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.403e-06 2020-08-07T15:07:08.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:08.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.449734+0000", 2020-08-07T15:07:08.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045277370000000004 2020-08-07T15:07:08.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:08.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.454261+0000", 2020-08-07T15:07:08.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8332999999999998e-05 2020-08-07T15:07:08.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:08.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.454330+0000", 2020-08-07T15:07:08.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7396000000000001e-05 2020-08-07T15:07:08.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:08.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.454427+0000", 2020-08-07T15:07:08.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028244379999999999 2020-08-07T15:07:08.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:08.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.457252+0000", 2020-08-07T15:07:08.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1299999999999999e-07 2020-08-07T15:07:08.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:08.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.457252+0000", 2020-08-07T15:07:08.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4706000000000001e-05 2020-08-07T15:07:08.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:08.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.457307+0000", 2020-08-07T15:07:08.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.071726532999999995 2020-08-07T15:07:08.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:08.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.529034+0000", 2020-08-07T15:07:08.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.102109669 2020-08-07T15:07:08.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:08.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.631143+0000", 2020-08-07T15:07:08.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9100000000000003e-07 2020-08-07T15:07:08.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:08.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.631144+0000", 2020-08-07T15:07:08.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7208e-05 2020-08-07T15:07:08.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:08.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.631181+0000", 2020-08-07T15:07:08.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18151558000000001 2020-08-07T15:07:08.346 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.346 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:08.346 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.347 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:36064 3.1 3:889c2e29:::benchmark_data_smithi074_35447_object4507:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-08-07T15:07:08.347 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:59.553842+0000", 2020-08-07T15:07:08.347 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.5000515459999999, 2020-08-07T15:07:08.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.17637318900000001, 2020-08-07T15:07:08.347 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:08.347 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:08.348 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:08.348 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:08.348 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:08.348 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 36064 2020-08-07T15:07:08.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.349 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:08.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:08.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.553842+0000", 2020-08-07T15:07:08.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:08.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:08.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.553842+0000", 2020-08-07T15:07:08.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7801999999999999e-05 2020-08-07T15:07:08.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:08.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.553869+0000", 2020-08-07T15:07:08.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.126e-06 2020-08-07T15:07:08.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:08.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.553874+0000", 2020-08-07T15:07:08.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7500000000000003e-06 2020-08-07T15:07:08.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:08.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.553881+0000", 2020-08-07T15:07:08.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.233e-06 2020-08-07T15:07:08.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:08.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.553886+0000", 2020-08-07T15:07:08.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004192266 2020-08-07T15:07:08.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:08.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.558078+0000", 2020-08-07T15:07:08.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2524000000000001e-05 2020-08-07T15:07:08.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:08.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.558130+0000", 2020-08-07T15:07:08.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000114851 2020-08-07T15:07:08.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,3", 2020-08-07T15:07:08.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.558245+0000", 2020-08-07T15:07:08.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045586250000000002 2020-08-07T15:07:08.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:08.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.562804+0000", 2020-08-07T15:07:08.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3300000000000003e-07 2020-08-07T15:07:08.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:08.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.562804+0000", 2020-08-07T15:07:08.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4450999999999999e-05 2020-08-07T15:07:08.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:08.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.562839+0000", 2020-08-07T15:07:08.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027703770000000001 2020-08-07T15:07:08.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:08.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.565609+0000", 2020-08-07T15:07:08.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16453051599999999 2020-08-07T15:07:08.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:08.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.730140+0000", 2020-08-07T15:07:08.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9740000000000001e-06 2020-08-07T15:07:08.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:08.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.730142+0000", 2020-08-07T15:07:08.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3261000000000002e-05 2020-08-07T15:07:08.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:08.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.730215+0000", 2020-08-07T15:07:08.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.17637318900000001 2020-08-07T15:07:08.366 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.366 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:08.366 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.367 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:36064 3.1 e31/26 ondisk, result = 0)", 2020-08-07T15:07:08.367 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:59.566407+0000", 2020-08-07T15:07:08.367 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.4874864930000005, 2020-08-07T15:07:08.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.163813984, 2020-08-07T15:07:08.368 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:08.368 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:08.369 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:08.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:08.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.566407+0000", 2020-08-07T15:07:08.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:08.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:08.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.566407+0000", 2020-08-07T15:07:08.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1433000000000002e-05 2020-08-07T15:07:08.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:08.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.566438+0000", 2020-08-07T15:07:08.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9060000000000001e-06 2020-08-07T15:07:08.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:08.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.566443+0000", 2020-08-07T15:07:08.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16361609999999999 2020-08-07T15:07:08.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:08.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.730059+0000", 2020-08-07T15:07:08.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8779999999999998e-06 2020-08-07T15:07:08.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:08.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.730067+0000", 2020-08-07T15:07:08.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2531000000000002e-05 2020-08-07T15:07:08.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:08.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.730120+0000", 2020-08-07T15:07:08.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.062e-05 2020-08-07T15:07:08.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:08.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.730130+0000", 2020-08-07T15:07:08.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0515999999999999e-05 2020-08-07T15:07:08.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:08.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.730221+0000", 2020-08-07T15:07:08.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.163813984 2020-08-07T15:07:08.380 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.380 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:08.381 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.382 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:41328 3.6 e32/26 3:64d2f40a:::benchmark_data_smithi074_35447_object5165:head v 32'2696, mlcod=32'2688)", 2020-08-07T15:07:08.382 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.726750+0000", 2020-08-07T15:07:08.382 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.3271427219999996, 2020-08-07T15:07:08.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.17405157299999999, 2020-08-07T15:07:08.382 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:08.383 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:08.383 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:08.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:08.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.726750+0000", 2020-08-07T15:07:08.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:08.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:08.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.726750+0000", 2020-08-07T15:07:08.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2639000000000002e-05 2020-08-07T15:07:08.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:08.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.726803+0000", 2020-08-07T15:07:08.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6499999999999999e-06 2020-08-07T15:07:08.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:08.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.726810+0000", 2020-08-07T15:07:08.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.17339010099999999 2020-08-07T15:07:08.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:08.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.900200+0000", 2020-08-07T15:07:08.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3139999999999997e-06 2020-08-07T15:07:08.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:08.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.900204+0000", 2020-08-07T15:07:08.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9347000000000001e-05 2020-08-07T15:07:08.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:08.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.900244+0000", 2020-08-07T15:07:08.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9408000000000001e-05 2020-08-07T15:07:08.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:08.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.900263+0000", 2020-08-07T15:07:08.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049846099999999996 2020-08-07T15:07:08.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:08.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.900761+0000", 2020-08-07T15:07:08.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0652999999999999e-05 2020-08-07T15:07:08.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:08.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.900802+0000", 2020-08-07T15:07:08.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.17405157299999999 2020-08-07T15:07:08.394 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.394 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:08.394 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.395 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:42712 3.9 3:916738ca:::benchmark_data_smithi074_35447_object5338:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:07:08.395 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.591327+0000", 2020-08-07T15:07:08.395 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.4625666879999999, 2020-08-07T15:07:08.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19416059499999999, 2020-08-07T15:07:08.396 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:08.396 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:08.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:08.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:08.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:08.397 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42712 2020-08-07T15:07:08.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.398 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:08.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:08.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.591327+0000", 2020-08-07T15:07:08.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:08.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:08.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.591327+0000", 2020-08-07T15:07:08.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2016e-05 2020-08-07T15:07:08.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:08.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.591359+0000", 2020-08-07T15:07:08.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9710000000000003e-06 2020-08-07T15:07:08.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:08.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.591364+0000", 2020-08-07T15:07:08.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0318e-05 2020-08-07T15:07:08.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:08.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.591374+0000", 2020-08-07T15:07:08.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4660000000000001e-06 2020-08-07T15:07:08.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:08.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.591378+0000", 2020-08-07T15:07:08.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0069575119999999999 2020-08-07T15:07:08.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:08.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.598336+0000", 2020-08-07T15:07:08.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0378999999999996e-05 2020-08-07T15:07:08.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:08.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.598426+0000", 2020-08-07T15:07:08.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014588399999999999 2020-08-07T15:07:08.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:08.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.598572+0000", 2020-08-07T15:07:08.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018089930000000001 2020-08-07T15:07:08.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:08.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.600381+0000", 2020-08-07T15:07:08.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000728956 2020-08-07T15:07:08.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:08.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.601110+0000", 2020-08-07T15:07:08.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18427601699999999 2020-08-07T15:07:08.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:08.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.785386+0000", 2020-08-07T15:07:08.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1200000000000001e-07 2020-08-07T15:07:08.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:08.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.785387+0000", 2020-08-07T15:07:08.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7848000000000005e-05 2020-08-07T15:07:08.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:08.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.785455+0000", 2020-08-07T15:07:08.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2323000000000002e-05 2020-08-07T15:07:08.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:08.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.785487+0000", 2020-08-07T15:07:08.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19416059499999999 2020-08-07T15:07:08.426 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.426 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:08.427 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.427 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:42712 3.9 e32/20 ondisk, result = 0)", 2020-08-07T15:07:08.428 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.601557+0000", 2020-08-07T15:07:08.428 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.4523358569999996, 2020-08-07T15:07:08.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18393375300000001, 2020-08-07T15:07:08.428 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:08.428 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:08.429 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:08.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:08.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.601557+0000", 2020-08-07T15:07:08.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:08.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:08.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.601557+0000", 2020-08-07T15:07:08.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0848e-05 2020-08-07T15:07:08.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:08.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.601588+0000", 2020-08-07T15:07:08.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.169e-06 2020-08-07T15:07:08.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:08.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.601593+0000", 2020-08-07T15:07:08.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18372855499999999 2020-08-07T15:07:08.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:08.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.785322+0000", 2020-08-07T15:07:08.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6400000000000002e-06 2020-08-07T15:07:08.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:08.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.785328+0000", 2020-08-07T15:07:08.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8943999999999998e-05 2020-08-07T15:07:08.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:08.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.785367+0000", 2020-08-07T15:07:08.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9600000000000006e-06 2020-08-07T15:07:08.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:08.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.785375+0000", 2020-08-07T15:07:08.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000115637 2020-08-07T15:07:08.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:08.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.785491+0000", 2020-08-07T15:07:08.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18393375300000001 2020-08-07T15:07:08.442 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.442 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:08.442 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.443 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.443 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:08.443 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:08.445 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:07:08.446 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 32 16 47136 47120 11.5029 0 - 0.0102655 2020-08-07T15:07:08.446 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:08.443+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:08.447 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:07:08.546 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:08.545+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:08.648 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:08.646+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:08.661 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:08.662 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:08.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.662 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:47758 3.2 e36/20 3:4806842e:::benchmark_data_smithi074_35447_object5969:head v 36'2806, mlcod=36'2799)", 2020-08-07T15:07:08.662 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:08.649950+0000", 2020-08-07T15:07:08.663 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0011077070000000001, 2020-08-07T15:07:08.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011332580000000001, 2020-08-07T15:07:08.663 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:08.663 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:08.663 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:08.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:08.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.649950+0000", 2020-08-07T15:07:08.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:08.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:08.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.649950+0000", 2020-08-07T15:07:08.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9897e-05 2020-08-07T15:07:08.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:08.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.650000+0000", 2020-08-07T15:07:08.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1929999999999998e-06 2020-08-07T15:07:08.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:08.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.650006+0000", 2020-08-07T15:07:08.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0008e-05 2020-08-07T15:07:08.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:08.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.650056+0000", 2020-08-07T15:07:08.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3210000000000004e-06 2020-08-07T15:07:08.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:08.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.650062+0000", 2020-08-07T15:07:08.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6659000000000002e-05 2020-08-07T15:07:08.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:08.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.650108+0000", 2020-08-07T15:07:08.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3628000000000001e-05 2020-08-07T15:07:08.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:08.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.650132+0000", 2020-08-07T15:07:08.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018170600000000001 2020-08-07T15:07:08.672 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.673 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:08.673 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.673 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:47751 3.c e36/20)", 2020-08-07T15:07:08.674 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:08.650818+0000", 2020-08-07T15:07:08.674 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.000239664, 2020-08-07T15:07:08.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031627899999999999, 2020-08-07T15:07:08.674 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:08.674 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:07:08.674 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:08.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:08.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.650818+0000", 2020-08-07T15:07:08.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:08.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:08.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.650818+0000", 2020-08-07T15:07:08.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1149000000000003e-05 2020-08-07T15:07:08.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:08.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.650850+0000", 2020-08-07T15:07:08.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9699999999999996e-06 2020-08-07T15:07:08.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:08.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.650856+0000", 2020-08-07T15:07:08.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000118338 2020-08-07T15:07:08.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:08.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.650974+0000", 2020-08-07T15:07:08.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8600000000000001e-06 2020-08-07T15:07:08.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:08.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.650979+0000", 2020-08-07T15:07:08.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016031699999999999 2020-08-07T15:07:08.680 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.680 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:08.680 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.681 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:47751 3.c 3:34ceaf6a:::benchmark_data_smithi074_35447_object5968:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e36)", 2020-08-07T15:07:08.681 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:08.645312+0000", 2020-08-07T15:07:08.681 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0057460649999999999, 2020-08-07T15:07:08.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058561220000000001, 2020-08-07T15:07:08.682 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:08.682 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:08.682 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:08.682 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:08.683 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:08.683 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 47751 2020-08-07T15:07:08.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.683 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:08.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:08.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.645312+0000", 2020-08-07T15:07:08.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:08.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:08.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.645312+0000", 2020-08-07T15:07:08.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2119e-05 2020-08-07T15:07:08.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:08.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.645364+0000", 2020-08-07T15:07:08.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.784e-06 2020-08-07T15:07:08.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:08.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.645370+0000", 2020-08-07T15:07:08.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.148e-05 2020-08-07T15:07:08.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:08.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.645381+0000", 2020-08-07T15:07:08.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7400000000000004e-06 2020-08-07T15:07:08.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:08.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.645386+0000", 2020-08-07T15:07:08.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001201187 2020-08-07T15:07:08.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:08.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.646587+0000", 2020-08-07T15:07:08.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4110999999999994e-05 2020-08-07T15:07:08.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:08.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.646681+0000", 2020-08-07T15:07:08.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026238699999999999 2020-08-07T15:07:08.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:07:08.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.646944+0000", 2020-08-07T15:07:08.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018331669999999999 2020-08-07T15:07:08.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:08.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.648777+0000", 2020-08-07T15:07:08.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6300000000000005e-07 2020-08-07T15:07:08.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:08.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.648778+0000", 2020-08-07T15:07:08.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2274999999999997e-05 2020-08-07T15:07:08.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:08.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.648830+0000", 2020-08-07T15:07:08.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035178129999999998 2020-08-07T15:07:08.698 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.699 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:08.699 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.699 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:47752 3.c 3:34ceaf6a:::benchmark_data_smithi074_35447_object5968:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e36)", 2020-08-07T15:07:08.699 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:08.645423+0000", 2020-08-07T15:07:08.700 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0056355959999999997, 2020-08-07T15:07:08.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058292170000000003, 2020-08-07T15:07:08.700 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:08.700 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:08.700 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:08.701 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:08.701 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:08.701 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 47752 2020-08-07T15:07:08.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.701 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:08.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:08.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.645423+0000", 2020-08-07T15:07:08.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:08.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:08.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.645423+0000", 2020-08-07T15:07:08.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1817000000000002e-05 2020-08-07T15:07:08.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:08.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.645474+0000", 2020-08-07T15:07:08.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7910000000000001e-06 2020-08-07T15:07:08.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:08.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.645480+0000", 2020-08-07T15:07:08.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2021999999999999e-05 2020-08-07T15:07:08.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:08.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.645492+0000", 2020-08-07T15:07:08.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.814e-06 2020-08-07T15:07:08.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:08.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.645497+0000", 2020-08-07T15:07:08.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018677710000000001 2020-08-07T15:07:08.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:08.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.647365+0000", 2020-08-07T15:07:08.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1303000000000003e-05 2020-08-07T15:07:08.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:08.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.647456+0000", 2020-08-07T15:07:08.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016373599999999999 2020-08-07T15:07:08.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:07:08.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.647620+0000", 2020-08-07T15:07:08.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012184769999999999 2020-08-07T15:07:08.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:08.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.648838+0000", 2020-08-07T15:07:08.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3099999999999998e-07 2020-08-07T15:07:08.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:08.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.648839+0000", 2020-08-07T15:07:08.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2067000000000001e-05 2020-08-07T15:07:08.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:08.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:08.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:08.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.648891+0000", 2020-08-07T15:07:08.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034682290000000002 2020-08-07T15:07:08.713 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.713 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:08.714 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.714 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:08.714 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:07:08.714 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 4 2020-08-07T15:07:08.714 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:08.716 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:07:08.752 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:08.752+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:08.854 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:08.852+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:08.895 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:08.895 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:07:08.896 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:07:08.896 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:07:08.896 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:08.905 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:07:08.958 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:08.958+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:09.065 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:09.059+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:09.065 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:09.065 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:07:09.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:07:09.066 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:09.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.066 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:1592 3.5 3:ab93003a:::benchmark_data_smithi074_35447_object198:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:07:09.066 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.292205+0000", 2020-08-07T15:07:09.067 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.766313050000001, 2020-08-07T15:07:09.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13617961100000001, 2020-08-07T15:07:09.067 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:09.067 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:09.067 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:09.067 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:09.068 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:09.068 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1592 2020-08-07T15:07:09.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.068 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:09.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:09.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.292205+0000", 2020-08-07T15:07:09.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:09.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:09.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.292205+0000", 2020-08-07T15:07:09.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7688999999999997e-05 2020-08-07T15:07:09.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:09.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.292252+0000", 2020-08-07T15:07:09.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0139999999999998e-06 2020-08-07T15:07:09.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:09.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.292257+0000", 2020-08-07T15:07:09.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0682e-05 2020-08-07T15:07:09.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:09.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.292268+0000", 2020-08-07T15:07:09.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1590000000000003e-06 2020-08-07T15:07:09.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:09.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.292272+0000", 2020-08-07T15:07:09.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030296559999999999 2020-08-07T15:07:09.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:09.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.295302+0000", 2020-08-07T15:07:09.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9184000000000004e-05 2020-08-07T15:07:09.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:09.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.295401+0000", 2020-08-07T15:07:09.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111563 2020-08-07T15:07:09.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:07:09.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.295513+0000", 2020-08-07T15:07:09.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030877980000000001 2020-08-07T15:07:09.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:09.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.298600+0000", 2020-08-07T15:07:09.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8800000000000002e-07 2020-08-07T15:07:09.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:09.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.298601+0000", 2020-08-07T15:07:09.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7547999999999999e-05 2020-08-07T15:07:09.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:09.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.298658+0000", 2020-08-07T15:07:09.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038224410000000002 2020-08-07T15:07:09.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:09.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.302481+0000", 2020-08-07T15:07:09.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.125813647 2020-08-07T15:07:09.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:09.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.428295+0000", 2020-08-07T15:07:09.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.048e-06 2020-08-07T15:07:09.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:09.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.428296+0000", 2020-08-07T15:07:09.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8593999999999999e-05 2020-08-07T15:07:09.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:09.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.428384+0000", 2020-08-07T15:07:09.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13617961100000001 2020-08-07T15:07:09.091 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.091 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:09.092 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.092 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:15744 3.d e27/20 3:b5661143:::benchmark_data_smithi074_35447_object1967:head v 27'984, mlcod=27'977)", 2020-08-07T15:07:09.092 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.777088+0000", 2020-08-07T15:07:09.093 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.281430048000001, 2020-08-07T15:07:09.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23236351199999999, 2020-08-07T15:07:09.093 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:09.093 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:09.093 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:09.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:09.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777088+0000", 2020-08-07T15:07:09.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:09.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:09.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777088+0000", 2020-08-07T15:07:09.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9495e-05 2020-08-07T15:07:09.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:09.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777137+0000", 2020-08-07T15:07:09.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3049999999999998e-06 2020-08-07T15:07:09.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:09.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777143+0000", 2020-08-07T15:07:09.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23074995500000001 2020-08-07T15:07:09.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:09.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.007893+0000", 2020-08-07T15:07:09.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4390000000000003e-06 2020-08-07T15:07:09.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:09.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.007901+0000", 2020-08-07T15:07:09.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0071000000000003e-05 2020-08-07T15:07:09.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:09.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.007941+0000", 2020-08-07T15:07:09.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2722e-05 2020-08-07T15:07:09.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:09.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.007964+0000", 2020-08-07T15:07:09.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014374870000000001 2020-08-07T15:07:09.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:09.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.009401+0000", 2020-08-07T15:07:09.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0037999999999998e-05 2020-08-07T15:07:09.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:09.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.009451+0000", 2020-08-07T15:07:09.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23236351199999999 2020-08-07T15:07:09.105 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.105 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:09.105 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.106 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:16248 3.d e27/20 3:b7542ecb:::benchmark_data_smithi074_35447_object2030:head v 27'1000, mlcod=27'991)", 2020-08-07T15:07:09.106 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:47.122503+0000", 2020-08-07T15:07:09.106 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.936014503999999, 2020-08-07T15:07:09.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12336765500000001, 2020-08-07T15:07:09.107 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:09.107 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:09.107 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:09.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:09.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.122503+0000", 2020-08-07T15:07:09.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:09.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:09.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.122503+0000", 2020-08-07T15:07:09.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6597e-05 2020-08-07T15:07:09.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:09.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.122570+0000", 2020-08-07T15:07:09.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1250000000000004e-06 2020-08-07T15:07:09.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:09.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.122577+0000", 2020-08-07T15:07:09.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11575635300000001 2020-08-07T15:07:09.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:09.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.238333+0000", 2020-08-07T15:07:09.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5031999999999999e-05 2020-08-07T15:07:09.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:09.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.238348+0000", 2020-08-07T15:07:09.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001322658 2020-08-07T15:07:09.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:09.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.239671+0000", 2020-08-07T15:07:09.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0985999999999999e-05 2020-08-07T15:07:09.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:09.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.239692+0000", 2020-08-07T15:07:09.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061335269999999997 2020-08-07T15:07:09.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:09.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.245825+0000", 2020-08-07T15:07:09.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5377000000000003e-05 2020-08-07T15:07:09.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:09.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.245871+0000", 2020-08-07T15:07:09.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12336765500000001 2020-08-07T15:07:09.117 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.117 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:09.117 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.118 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:17856 3.5 3:a97d5eb0:::benchmark_data_smithi074_35447_object2231:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e27)", 2020-08-07T15:07:09.118 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.164730+0000", 2020-08-07T15:07:09.118 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.893787139000001, 2020-08-07T15:07:09.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.123557982, 2020-08-07T15:07:09.119 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:09.119 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:09.119 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:09.119 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:09.119 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:09.120 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 17856 2020-08-07T15:07:09.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.120 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:09.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:09.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.164730+0000", 2020-08-07T15:07:09.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:09.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:09.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.164730+0000", 2020-08-07T15:07:09.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5515000000000001e-05 2020-08-07T15:07:09.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:09.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.164766+0000", 2020-08-07T15:07:09.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0249999999999996e-06 2020-08-07T15:07:09.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:09.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.164770+0000", 2020-08-07T15:07:09.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5120000000000002e-06 2020-08-07T15:07:09.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:09.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.164778+0000", 2020-08-07T15:07:09.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9330000000000003e-06 2020-08-07T15:07:09.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:09.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.164781+0000", 2020-08-07T15:07:09.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001643278 2020-08-07T15:07:09.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:09.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.166425+0000", 2020-08-07T15:07:09.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107273 2020-08-07T15:07:09.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:09.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.166532+0000", 2020-08-07T15:07:09.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000114817 2020-08-07T15:07:09.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:07:09.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.166647+0000", 2020-08-07T15:07:09.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00562541 2020-08-07T15:07:09.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:09.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.172272+0000", 2020-08-07T15:07:09.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039347770000000004 2020-08-07T15:07:09.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:09.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.176207+0000", 2020-08-07T15:07:09.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.111947885 2020-08-07T15:07:09.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:09.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.288155+0000", 2020-08-07T15:07:09.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6599999999999994e-07 2020-08-07T15:07:09.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:09.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.288156+0000", 2020-08-07T15:07:09.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2797000000000001e-05 2020-08-07T15:07:09.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:09.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.288229+0000", 2020-08-07T15:07:09.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9793999999999997e-05 2020-08-07T15:07:09.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:09.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.288288+0000", 2020-08-07T15:07:09.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.123557982 2020-08-07T15:07:09.139 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.139 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:09.139 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.140 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:24392 3.d e28/20 3:bb13957b:::benchmark_data_smithi074_35447_object3048:head v 28'1616, mlcod=28'1606)", 2020-08-07T15:07:09.140 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:52.463391+0000", 2020-08-07T15:07:09.140 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.595126281999999, 2020-08-07T15:07:09.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.162237187, 2020-08-07T15:07:09.141 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:09.141 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:09.141 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:09.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:09.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.463391+0000", 2020-08-07T15:07:09.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:09.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:09.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.463391+0000", 2020-08-07T15:07:09.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0294999999999999e-05 2020-08-07T15:07:09.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:09.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.463442+0000", 2020-08-07T15:07:09.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1149999999999999e-06 2020-08-07T15:07:09.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:09.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.463448+0000", 2020-08-07T15:07:09.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16129982900000001 2020-08-07T15:07:09.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:09.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.624748+0000", 2020-08-07T15:07:09.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6420000000000007e-06 2020-08-07T15:07:09.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:09.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.624755+0000", 2020-08-07T15:07:09.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8579000000000003e-05 2020-08-07T15:07:09.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:09.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.624814+0000", 2020-08-07T15:07:09.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5729999999999999e-05 2020-08-07T15:07:09.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:09.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.624840+0000", 2020-08-07T15:07:09.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00073357399999999997 2020-08-07T15:07:09.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:09.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.625573+0000", 2020-08-07T15:07:09.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5423000000000001e-05 2020-08-07T15:07:09.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:09.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.625629+0000", 2020-08-07T15:07:09.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.162237187 2020-08-07T15:07:09.151 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.151 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:09.151 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.152 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:24640 3.d e28/20 3:b3ce8a98:::benchmark_data_smithi074_35447_object3079:head v 28'1624, mlcod=28'1615)", 2020-08-07T15:07:09.152 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:52.630934+0000", 2020-08-07T15:07:09.152 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.427583147, 2020-08-07T15:07:09.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14322729300000001, 2020-08-07T15:07:09.153 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:09.153 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:09.153 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:09.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:09.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.630934+0000", 2020-08-07T15:07:09.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:09.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:09.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.630934+0000", 2020-08-07T15:07:09.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1687000000000002e-05 2020-08-07T15:07:09.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:09.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.630986+0000", 2020-08-07T15:07:09.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3010000000000003e-06 2020-08-07T15:07:09.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:09.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.630992+0000", 2020-08-07T15:07:09.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14074687499999999 2020-08-07T15:07:09.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:09.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.771739+0000", 2020-08-07T15:07:09.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2229999999999998e-06 2020-08-07T15:07:09.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:09.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.771745+0000", 2020-08-07T15:07:09.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6329000000000003e-05 2020-08-07T15:07:09.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:09.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.771791+0000", 2020-08-07T15:07:09.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5248e-05 2020-08-07T15:07:09.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:09.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.771816+0000", 2020-08-07T15:07:09.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002296111 2020-08-07T15:07:09.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:09.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.774112+0000", 2020-08-07T15:07:09.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9518999999999998e-05 2020-08-07T15:07:09.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:09.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.774162+0000", 2020-08-07T15:07:09.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14322729300000001 2020-08-07T15:07:09.164 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.165 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:09.165 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.166 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:27314 3.5 3:a11d2067:::benchmark_data_smithi074_35447_object3414:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:09.166 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:54.203986+0000", 2020-08-07T15:07:09.167 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.854531578, 2020-08-07T15:07:09.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12129693699999999, 2020-08-07T15:07:09.168 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:09.168 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:09.168 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:09.169 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:09.169 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:09.169 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 27314 2020-08-07T15:07:09.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.173 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:09.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:09.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.203986+0000", 2020-08-07T15:07:09.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:09.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:09.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.203986+0000", 2020-08-07T15:07:09.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9731999999999999e-05 2020-08-07T15:07:09.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:09.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204016+0000", 2020-08-07T15:07:09.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7430000000000002e-06 2020-08-07T15:07:09.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:09.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204021+0000", 2020-08-07T15:07:09.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2346e-05 2020-08-07T15:07:09.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:09.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204033+0000", 2020-08-07T15:07:09.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7299999999999996e-06 2020-08-07T15:07:09.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:09.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204038+0000", 2020-08-07T15:07:09.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00090682499999999999 2020-08-07T15:07:09.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:09.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204944+0000", 2020-08-07T15:07:09.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010785099999999999 2020-08-07T15:07:09.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:09.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.205052+0000", 2020-08-07T15:07:09.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021478099999999999 2020-08-07T15:07:09.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:07:09.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.205267+0000", 2020-08-07T15:07:09.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028852370000000001 2020-08-07T15:07:09.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:09.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208152+0000", 2020-08-07T15:07:09.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5899999999999998e-07 2020-08-07T15:07:09.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:09.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208153+0000", 2020-08-07T15:07:09.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8082999999999999e-05 2020-08-07T15:07:09.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:09.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208181+0000", 2020-08-07T15:07:09.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001815416 2020-08-07T15:07:09.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:09.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.209996+0000", 2020-08-07T15:07:09.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115217685 2020-08-07T15:07:09.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:09.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.325214+0000", 2020-08-07T15:07:09.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2200000000000004e-07 2020-08-07T15:07:09.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:09.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.325214+0000", 2020-08-07T15:07:09.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8726999999999994e-05 2020-08-07T15:07:09.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:09.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.325283+0000", 2020-08-07T15:07:09.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12129693699999999 2020-08-07T15:07:09.196 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.196 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:09.196 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.197 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:27315 3.5 3:a11d2067:::benchmark_data_smithi074_35447_object3414:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:09.197 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:54.204176+0000", 2020-08-07T15:07:09.198 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.854342055, 2020-08-07T15:07:09.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.123218044, 2020-08-07T15:07:09.200 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:09.200 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:09.200 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:09.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:09.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:09.202 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 27315 2020-08-07T15:07:09.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.203 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:09.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:09.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204176+0000", 2020-08-07T15:07:09.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:09.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:09.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204176+0000", 2020-08-07T15:07:09.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6219e-05 2020-08-07T15:07:09.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:09.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204212+0000", 2020-08-07T15:07:09.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5939999999999998e-06 2020-08-07T15:07:09.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:09.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204215+0000", 2020-08-07T15:07:09.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7169999999999997e-06 2020-08-07T15:07:09.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:09.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204224+0000", 2020-08-07T15:07:09.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3499999999999999e-06 2020-08-07T15:07:09.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:09.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204228+0000", 2020-08-07T15:07:09.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001455488 2020-08-07T15:07:09.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:09.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.205684+0000", 2020-08-07T15:07:09.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7450999999999999e-05 2020-08-07T15:07:09.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:09.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.205781+0000", 2020-08-07T15:07:09.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018575199999999999 2020-08-07T15:07:09.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:07:09.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.205967+0000", 2020-08-07T15:07:09.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022180199999999998 2020-08-07T15:07:09.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:09.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208185+0000", 2020-08-07T15:07:09.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.04e-07 2020-08-07T15:07:09.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:09.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208185+0000", 2020-08-07T15:07:09.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4456000000000003e-05 2020-08-07T15:07:09.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:09.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208230+0000", 2020-08-07T15:07:09.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001801031 2020-08-07T15:07:09.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:09.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.210031+0000", 2020-08-07T15:07:09.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.117308515 2020-08-07T15:07:09.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:09.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.327339+0000", 2020-08-07T15:07:09.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2399999999999997e-07 2020-08-07T15:07:09.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:09.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.327340+0000", 2020-08-07T15:07:09.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3423e-05 2020-08-07T15:07:09.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:09.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.327394+0000", 2020-08-07T15:07:09.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.123218044 2020-08-07T15:07:09.230 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.230 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:09.230 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.231 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:27316 3.5 3:a11d2067:::benchmark_data_smithi074_35447_object3414:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:09.231 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:54.204248+0000", 2020-08-07T15:07:09.231 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.854269363, 2020-08-07T15:07:09.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12325235499999999, 2020-08-07T15:07:09.232 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:09.232 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:09.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:09.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:09.233 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:09.233 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 27316 2020-08-07T15:07:09.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.233 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:09.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:09.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204248+0000", 2020-08-07T15:07:09.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:09.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:09.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204248+0000", 2020-08-07T15:07:09.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1449999999999999e-05 2020-08-07T15:07:09.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:09.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204280+0000", 2020-08-07T15:07:09.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0970000000000002e-06 2020-08-07T15:07:09.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:09.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204283+0000", 2020-08-07T15:07:09.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1720000000000004e-06 2020-08-07T15:07:09.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:09.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204289+0000", 2020-08-07T15:07:09.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8219999999999998e-06 2020-08-07T15:07:09.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:09.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.204292+0000", 2020-08-07T15:07:09.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020809700000000001 2020-08-07T15:07:09.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:09.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.206373+0000", 2020-08-07T15:07:09.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6865000000000002e-05 2020-08-07T15:07:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.206470+0000", 2020-08-07T15:07:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000170631 2020-08-07T15:07:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:07:09.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.206640+0000", 2020-08-07T15:07:09.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029298710000000001 2020-08-07T15:07:09.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:09.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.209570+0000", 2020-08-07T15:07:09.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9999999999999996e-07 2020-08-07T15:07:09.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:09.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.209571+0000", 2020-08-07T15:07:09.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2813999999999999e-05 2020-08-07T15:07:09.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:09.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.209634+0000", 2020-08-07T15:07:09.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00042981699999999998 2020-08-07T15:07:09.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:09.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.210064+0000", 2020-08-07T15:07:09.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.117374487 2020-08-07T15:07:09.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:09.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.327438+0000", 2020-08-07T15:07:09.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2799999999999996e-07 2020-08-07T15:07:09.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:09.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.327439+0000", 2020-08-07T15:07:09.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1931e-05 2020-08-07T15:07:09.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:09.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.327501+0000", 2020-08-07T15:07:09.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12325235499999999 2020-08-07T15:07:09.266 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.266 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:09.266 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.267 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:27317 3.5 3:a11d2067:::benchmark_data_smithi074_35447_object3414:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:09.267 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:54.205688+0000", 2020-08-07T15:07:09.267 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.852829473, 2020-08-07T15:07:09.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.121962605, 2020-08-07T15:07:09.268 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:09.268 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:09.268 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:09.268 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:09.268 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:09.269 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 27317 2020-08-07T15:07:09.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.269 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:09.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:09.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.205688+0000", 2020-08-07T15:07:09.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:09.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:09.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.205688+0000", 2020-08-07T15:07:09.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8769e-05 2020-08-07T15:07:09.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:09.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.205737+0000", 2020-08-07T15:07:09.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1199999999999999e-06 2020-08-07T15:07:09.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:09.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.205743+0000", 2020-08-07T15:07:09.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1260000000000001e-05 2020-08-07T15:07:09.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:09.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.205754+0000", 2020-08-07T15:07:09.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7930000000000003e-06 2020-08-07T15:07:09.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:09.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.205760+0000", 2020-08-07T15:07:09.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013008449999999999 2020-08-07T15:07:09.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:09.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.207061+0000", 2020-08-07T15:07:09.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100398 2020-08-07T15:07:09.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:09.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.207161+0000", 2020-08-07T15:07:09.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016082499999999999 2020-08-07T15:07:09.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:07:09.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.207322+0000", 2020-08-07T15:07:09.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042353 2020-08-07T15:07:09.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:09.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.211557+0000", 2020-08-07T15:07:09.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034108000000000001 2020-08-07T15:07:09.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:09.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.211899+0000", 2020-08-07T15:07:09.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115642549 2020-08-07T15:07:09.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:09.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.327541+0000", 2020-08-07T15:07:09.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6899999999999998e-07 2020-08-07T15:07:09.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:09.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.327542+0000", 2020-08-07T15:07:09.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4193e-05 2020-08-07T15:07:09.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:09.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.327606+0000", 2020-08-07T15:07:09.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5003999999999998e-05 2020-08-07T15:07:09.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:09.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.327651+0000", 2020-08-07T15:07:09.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.121962605 2020-08-07T15:07:09.287 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.287 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:09.287 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.288 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:27318 3.5 3:a11d2067:::benchmark_data_smithi074_35447_object3414:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:09.288 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:54.206599+0000", 2020-08-07T15:07:09.288 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.851918434, 2020-08-07T15:07:09.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12202629600000001, 2020-08-07T15:07:09.289 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:09.289 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:09.289 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:09.289 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:09.290 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:09.290 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 27318 2020-08-07T15:07:09.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.290 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:09.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:09.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.206599+0000", 2020-08-07T15:07:09.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:09.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:09.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.206599+0000", 2020-08-07T15:07:09.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2527e-05 2020-08-07T15:07:09.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:09.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.206652+0000", 2020-08-07T15:07:09.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2030000000000003e-06 2020-08-07T15:07:09.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:09.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.206659+0000", 2020-08-07T15:07:09.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3346000000000001e-05 2020-08-07T15:07:09.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:09.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.206672+0000", 2020-08-07T15:07:09.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1090000000000001e-05 2020-08-07T15:07:09.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:09.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.206683+0000", 2020-08-07T15:07:09.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00098956499999999998 2020-08-07T15:07:09.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:09.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.207673+0000", 2020-08-07T15:07:09.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9324999999999999e-05 2020-08-07T15:07:09.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:09.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.207732+0000", 2020-08-07T15:07:09.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010767700000000001 2020-08-07T15:07:09.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:07:09.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.207840+0000", 2020-08-07T15:07:09.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040669340000000003 2020-08-07T15:07:09.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:09.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.211907+0000", 2020-08-07T15:07:09.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5700000000000002e-07 2020-08-07T15:07:09.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:09.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.211908+0000", 2020-08-07T15:07:09.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5735000000000007e-05 2020-08-07T15:07:09.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:09.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.211973+0000", 2020-08-07T15:07:09.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00094504699999999997 2020-08-07T15:07:09.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:09.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.212918+0000", 2020-08-07T15:07:09.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115640482 2020-08-07T15:07:09.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:09.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.328559+0000", 2020-08-07T15:07:09.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2200000000000004e-07 2020-08-07T15:07:09.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:09.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.328559+0000", 2020-08-07T15:07:09.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5986000000000001e-05 2020-08-07T15:07:09.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:09.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.328625+0000", 2020-08-07T15:07:09.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12202629600000001 2020-08-07T15:07:09.306 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.307 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:09.307 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.307 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:27320 3.5 3:a11d2067:::benchmark_data_smithi074_35447_object3414:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:09.308 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:54.208099+0000", 2020-08-07T15:07:09.308 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.850418171999999, 2020-08-07T15:07:09.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.120753469, 2020-08-07T15:07:09.308 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:09.308 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:09.308 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:09.309 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:09.309 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:09.309 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 27320 2020-08-07T15:07:09.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.309 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:09.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:09.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208099+0000", 2020-08-07T15:07:09.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:09.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:09.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208099+0000", 2020-08-07T15:07:09.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8172999999999999e-05 2020-08-07T15:07:09.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:09.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208128+0000", 2020-08-07T15:07:09.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0410000000000002e-06 2020-08-07T15:07:09.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:09.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208131+0000", 2020-08-07T15:07:09.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1860000000000003e-06 2020-08-07T15:07:09.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:09.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208138+0000", 2020-08-07T15:07:09.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9529999999999999e-06 2020-08-07T15:07:09.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:09.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208141+0000", 2020-08-07T15:07:09.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00082086400000000003 2020-08-07T15:07:09.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:09.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.208962+0000", 2020-08-07T15:07:09.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2806000000000006e-05 2020-08-07T15:07:09.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:09.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.209054+0000", 2020-08-07T15:07:09.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000167596 2020-08-07T15:07:09.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:07:09.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.209222+0000", 2020-08-07T15:07:09.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028282979999999999 2020-08-07T15:07:09.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:09.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.212050+0000", 2020-08-07T15:07:09.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9100000000000004e-07 2020-08-07T15:07:09.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:09.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.212051+0000", 2020-08-07T15:07:09.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4016999999999994e-05 2020-08-07T15:07:09.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:09.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.212115+0000", 2020-08-07T15:07:09.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030628244999999998 2020-08-07T15:07:09.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:09.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.242743+0000", 2020-08-07T15:07:09.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.086037783000000007 2020-08-07T15:07:09.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:09.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.328781+0000", 2020-08-07T15:07:09.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8100000000000003e-07 2020-08-07T15:07:09.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:09.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.328782+0000", 2020-08-07T15:07:09.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1334999999999994e-05 2020-08-07T15:07:09.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:09.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.328853+0000", 2020-08-07T15:07:09.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.120753469 2020-08-07T15:07:09.326 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.326 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:09.327 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.327 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:27407 3.5 3:ab4f36a5:::benchmark_data_smithi074_35447_object3425:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:09.327 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:54.329694+0000", 2020-08-07T15:07:09.328 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.728823531, 2020-08-07T15:07:09.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12782907299999999, 2020-08-07T15:07:09.328 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:09.328 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:09.328 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:09.329 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:09.329 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:09.330 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 27407 2020-08-07T15:07:09.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.330 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:09.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:09.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.329694+0000", 2020-08-07T15:07:09.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:09.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:09.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.329694+0000", 2020-08-07T15:07:09.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0769999999999997e-05 2020-08-07T15:07:09.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:09.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.329745+0000", 2020-08-07T15:07:09.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3279999999999996e-06 2020-08-07T15:07:09.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:09.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.329750+0000", 2020-08-07T15:07:09.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.112e-05 2020-08-07T15:07:09.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:09.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.329761+0000", 2020-08-07T15:07:09.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7890000000000002e-06 2020-08-07T15:07:09.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:09.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.329766+0000", 2020-08-07T15:07:09.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042646089999999999 2020-08-07T15:07:09.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:09.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.334031+0000", 2020-08-07T15:07:09.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5155999999999995e-05 2020-08-07T15:07:09.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:09.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.334126+0000", 2020-08-07T15:07:09.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033146100000000002 2020-08-07T15:07:09.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:07:09.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.334457+0000", 2020-08-07T15:07:09.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029586479999999999 2020-08-07T15:07:09.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:09.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.337416+0000", 2020-08-07T15:07:09.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9100000000000004e-07 2020-08-07T15:07:09.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:09.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.337416+0000", 2020-08-07T15:07:09.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1066999999999997e-05 2020-08-07T15:07:09.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:09.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.337468+0000", 2020-08-07T15:07:09.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033366189999999999 2020-08-07T15:07:09.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:09.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.340804+0000", 2020-08-07T15:07:09.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116648549 2020-08-07T15:07:09.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:09.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.457453+0000", 2020-08-07T15:07:09.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1850000000000001e-06 2020-08-07T15:07:09.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:09.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.457454+0000", 2020-08-07T15:07:09.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9281000000000003e-05 2020-08-07T15:07:09.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:09.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.457523+0000", 2020-08-07T15:07:09.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12782907299999999 2020-08-07T15:07:09.346 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.346 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:09.346 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.347 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:27408 3.5 3:ab4f36a5:::benchmark_data_smithi074_35447_object3425:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:09.347 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:54.329798+0000", 2020-08-07T15:07:09.347 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.728719234, 2020-08-07T15:07:09.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.127800106, 2020-08-07T15:07:09.348 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:09.348 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:09.348 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:09.348 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:09.349 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:09.349 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 27408 2020-08-07T15:07:09.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.349 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:09.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:09.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.329798+0000", 2020-08-07T15:07:09.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:09.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:09.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.329798+0000", 2020-08-07T15:07:09.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5600999999999998e-05 2020-08-07T15:07:09.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:09.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.329854+0000", 2020-08-07T15:07:09.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3260000000000003e-06 2020-08-07T15:07:09.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:09.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.329859+0000", 2020-08-07T15:07:09.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0332e-05 2020-08-07T15:07:09.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:09.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.329870+0000", 2020-08-07T15:07:09.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4900000000000002e-06 2020-08-07T15:07:09.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:09.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.329874+0000", 2020-08-07T15:07:09.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059293100000000001 2020-08-07T15:07:09.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:09.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.335803+0000", 2020-08-07T15:07:09.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0387e-05 2020-08-07T15:07:09.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:09.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.335894+0000", 2020-08-07T15:07:09.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019818900000000001 2020-08-07T15:07:09.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:07:09.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.336092+0000", 2020-08-07T15:07:09.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001380448 2020-08-07T15:07:09.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:09.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.337472+0000", 2020-08-07T15:07:09.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.77e-07 2020-08-07T15:07:09.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:09.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.337473+0000", 2020-08-07T15:07:09.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8547999999999998e-05 2020-08-07T15:07:09.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:09.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.337501+0000", 2020-08-07T15:07:09.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033325920000000001 2020-08-07T15:07:09.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:09.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.340834+0000", 2020-08-07T15:07:09.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116723147 2020-08-07T15:07:09.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:09.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.457557+0000", 2020-08-07T15:07:09.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9899999999999999e-07 2020-08-07T15:07:09.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:09.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.457558+0000", 2020-08-07T15:07:09.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0460000000000002e-05 2020-08-07T15:07:09.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:09.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:54.457598+0000", 2020-08-07T15:07:09.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.127800106 2020-08-07T15:07:09.365 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.365 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:09.365 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.365 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:32096 3.d e30/20 3:b7dba9f8:::benchmark_data_smithi074_35447_object4011:head v 30'2152, mlcod=30'2143)", 2020-08-07T15:07:09.366 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:57.198961+0000", 2020-08-07T15:07:09.366 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.859556972, 2020-08-07T15:07:09.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18294507300000001, 2020-08-07T15:07:09.366 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:09.366 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:09.367 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:09.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:09.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.198961+0000", 2020-08-07T15:07:09.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:09.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:09.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.198961+0000", 2020-08-07T15:07:09.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0689999999999997e-05 2020-08-07T15:07:09.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:09.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.199011+0000", 2020-08-07T15:07:09.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7429999999999996e-06 2020-08-07T15:07:09.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:09.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.199018+0000", 2020-08-07T15:07:09.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18124368699999999 2020-08-07T15:07:09.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:09.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.380262+0000", 2020-08-07T15:07:09.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2109999999999997e-06 2020-08-07T15:07:09.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:09.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.380267+0000", 2020-08-07T15:07:09.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8856000000000002e-05 2020-08-07T15:07:09.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:09.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.380306+0000", 2020-08-07T15:07:09.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3572999999999999e-05 2020-08-07T15:07:09.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:09.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.380329+0000", 2020-08-07T15:07:09.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001541749 2020-08-07T15:07:09.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:09.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.381871+0000", 2020-08-07T15:07:09.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4564000000000003e-05 2020-08-07T15:07:09.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:09.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.381906+0000", 2020-08-07T15:07:09.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18294507300000001 2020-08-07T15:07:09.377 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.377 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:09.377 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.379 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:33216 3.d e30/20 3:b86184aa:::benchmark_data_smithi074_35447_object4151:head v 30'2232, mlcod=30'2222)", 2020-08-07T15:07:09.379 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:57.864608+0000", 2020-08-07T15:07:09.379 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.193909867, 2020-08-07T15:07:09.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15879491600000001, 2020-08-07T15:07:09.379 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:09.380 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:09.380 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:09.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:09.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.864608+0000", 2020-08-07T15:07:09.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:09.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:09.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.864608+0000", 2020-08-07T15:07:09.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7546000000000002e-05 2020-08-07T15:07:09.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:09.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.864655+0000", 2020-08-07T15:07:09.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2410000000000003e-06 2020-08-07T15:07:09.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:09.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:57.864661+0000", 2020-08-07T15:07:09.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.157425711 2020-08-07T15:07:09.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:09.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.022086+0000", 2020-08-07T15:07:09.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4450000000000004e-06 2020-08-07T15:07:09.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:09.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.022092+0000", 2020-08-07T15:07:09.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3872000000000003e-05 2020-08-07T15:07:09.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:09.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.022136+0000", 2020-08-07T15:07:09.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3927999999999999e-05 2020-08-07T15:07:09.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:09.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.022159+0000", 2020-08-07T15:07:09.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011814760000000001 2020-08-07T15:07:09.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:09.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023341+0000", 2020-08-07T15:07:09.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1697000000000002e-05 2020-08-07T15:07:09.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:09.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023403+0000", 2020-08-07T15:07:09.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15879491600000001 2020-08-07T15:07:09.390 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.390 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:09.390 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.391 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:33488 3.c 3:3d1929c7:::benchmark_data_smithi074_35447_object4185:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-08-07T15:07:09.391 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:58.021233+0000", 2020-08-07T15:07:09.391 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.037284778, 2020-08-07T15:07:09.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.2237055, 2020-08-07T15:07:09.391 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:09.391 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:09.392 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:09.392 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:09.392 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:09.392 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 33488 2020-08-07T15:07:09.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.393 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:09.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:09.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.021233+0000", 2020-08-07T15:07:09.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:09.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:09.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.021233+0000", 2020-08-07T15:07:09.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7682000000000002e-05 2020-08-07T15:07:09.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:09.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.021281+0000", 2020-08-07T15:07:09.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.955e-06 2020-08-07T15:07:09.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:09.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.021285+0000", 2020-08-07T15:07:09.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2132e-05 2020-08-07T15:07:09.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:09.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.021298+0000", 2020-08-07T15:07:09.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1390000000000003e-06 2020-08-07T15:07:09.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:09.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.021303+0000", 2020-08-07T15:07:09.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011846789999999999 2020-08-07T15:07:09.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:09.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.022487+0000", 2020-08-07T15:07:09.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3844000000000002e-05 2020-08-07T15:07:09.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:09.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.022541+0000", 2020-08-07T15:07:09.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013274700000000001 2020-08-07T15:07:09.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:07:09.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.022674+0000", 2020-08-07T15:07:09.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00161185 2020-08-07T15:07:09.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:09.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.024286+0000", 2020-08-07T15:07:09.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8800000000000003e-07 2020-08-07T15:07:09.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:09.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.024286+0000", 2020-08-07T15:07:09.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8810999999999997e-05 2020-08-07T15:07:09.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:09.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.024335+0000", 2020-08-07T15:07:09.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025354919999999999 2020-08-07T15:07:09.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:09.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.026871+0000", 2020-08-07T15:07:09.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21800207099999999 2020-08-07T15:07:09.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:09.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.244873+0000", 2020-08-07T15:07:09.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2279999999999999e-06 2020-08-07T15:07:09.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:09.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.244874+0000", 2020-08-07T15:07:09.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4381999999999994e-05 2020-08-07T15:07:09.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:09.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.244938+0000", 2020-08-07T15:07:09.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.2237055 2020-08-07T15:07:09.408 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.408 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:09.409 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.409 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:41432 3.d e32/20 3:b5c8a160:::benchmark_data_smithi074_35447_object5178:head v 32'2848, mlcod=32'2838)", 2020-08-07T15:07:09.409 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.794575+0000", 2020-08-07T15:07:09.410 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.2639426460000003, 2020-08-07T15:07:09.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36243746199999999, 2020-08-07T15:07:09.410 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:09.410 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:09.410 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:09.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:09.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794575+0000", 2020-08-07T15:07:09.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:09.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:09.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794575+0000", 2020-08-07T15:07:09.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0696999999999999e-05 2020-08-07T15:07:09.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:09.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794626+0000", 2020-08-07T15:07:09.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4189999999999999e-06 2020-08-07T15:07:09.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:09.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794632+0000", 2020-08-07T15:07:09.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36145730399999998 2020-08-07T15:07:09.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:09.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.156089+0000", 2020-08-07T15:07:09.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2329999999999997e-06 2020-08-07T15:07:09.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:09.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.156095+0000", 2020-08-07T15:07:09.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0054999999999999e-05 2020-08-07T15:07:09.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:09.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.156125+0000", 2020-08-07T15:07:09.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5369000000000001e-05 2020-08-07T15:07:09.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:09.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.156140+0000", 2020-08-07T15:07:09.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00076810300000000001 2020-08-07T15:07:09.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:09.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.156908+0000", 2020-08-07T15:07:09.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104282 2020-08-07T15:07:09.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:09.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.157012+0000", 2020-08-07T15:07:09.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36243746199999999 2020-08-07T15:07:09.424 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.424 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:09.425 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.425 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:44456 3.5 3:a961686e:::benchmark_data_smithi074_35447_object5556:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:07:09.425 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:04.682156+0000", 2020-08-07T15:07:09.426 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.3763615700000003, 2020-08-07T15:07:09.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15661472800000001, 2020-08-07T15:07:09.426 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:09.426 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:09.426 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:09.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:09.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:09.427 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 44456 2020-08-07T15:07:09.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.427 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:09.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:09.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.682156+0000", 2020-08-07T15:07:09.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:09.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:09.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.682156+0000", 2020-08-07T15:07:09.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9931e-05 2020-08-07T15:07:09.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:09.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.682206+0000", 2020-08-07T15:07:09.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.409e-06 2020-08-07T15:07:09.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:09.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.682212+0000", 2020-08-07T15:07:09.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2774e-05 2020-08-07T15:07:09.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:09.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.682225+0000", 2020-08-07T15:07:09.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.516e-06 2020-08-07T15:07:09.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:09.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.682232+0000", 2020-08-07T15:07:09.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8317000000000003e-05 2020-08-07T15:07:09.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:09.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.682300+0000", 2020-08-07T15:07:09.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6048000000000003e-05 2020-08-07T15:07:09.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:09.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.682356+0000", 2020-08-07T15:07:09.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9207000000000001e-05 2020-08-07T15:07:09.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:07:09.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.682455+0000", 2020-08-07T15:07:09.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010423019999999999 2020-08-07T15:07:09.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:09.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.683498+0000", 2020-08-07T15:07:09.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0599999999999999e-07 2020-08-07T15:07:09.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:09.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.683498+0000", 2020-08-07T15:07:09.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4138000000000002e-05 2020-08-07T15:07:09.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:09.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.683563+0000", 2020-08-07T15:07:09.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043642413999999997 2020-08-07T15:07:09.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:09.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.727205+0000", 2020-08-07T15:07:09.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11149076300000001 2020-08-07T15:07:09.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:09.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.838696+0000", 2020-08-07T15:07:09.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1739999999999999e-06 2020-08-07T15:07:09.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:09.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.838697+0000", 2020-08-07T15:07:09.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3929000000000003e-05 2020-08-07T15:07:09.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:09.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.838771+0000", 2020-08-07T15:07:09.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15661472800000001 2020-08-07T15:07:09.452 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.452 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:09.452 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.453 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:44456 3.5 e32/20 ondisk, result = 0)", 2020-08-07T15:07:09.453 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:04.684736+0000", 2020-08-07T15:07:09.453 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.3737814359999998, 2020-08-07T15:07:09.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15404030499999999, 2020-08-07T15:07:09.454 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:09.454 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:09.454 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:09.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:09.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.684736+0000", 2020-08-07T15:07:09.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:09.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:09.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.684736+0000", 2020-08-07T15:07:09.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5227e-05 2020-08-07T15:07:09.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:09.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.684751+0000", 2020-08-07T15:07:09.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.655e-06 2020-08-07T15:07:09.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:09.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.684754+0000", 2020-08-07T15:07:09.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15385695299999999 2020-08-07T15:07:09.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:09.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.838611+0000", 2020-08-07T15:07:09.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4720000000000003e-06 2020-08-07T15:07:09.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:09.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.838616+0000", 2020-08-07T15:07:09.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7778000000000001e-05 2020-08-07T15:07:09.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:09.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.838674+0000", 2020-08-07T15:07:09.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3480000000000004e-06 2020-08-07T15:07:09.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:09.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.838683+0000", 2020-08-07T15:07:09.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3871999999999998e-05 2020-08-07T15:07:09.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:09.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:09.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:09.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.838776+0000", 2020-08-07T15:07:09.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15404030499999999 2020-08-07T15:07:09.462 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.463 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:09.463 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.463 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:09.463 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:09.463 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:09.465 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 33 16 48550 48534 11.4891 5.52344 0.00417141 0.0108744 2020-08-07T15:07:09.465 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:07:09.466 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:09.464+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:09.567 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:09.565+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:09.670 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:09.666+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:09.674 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:09.674 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:07:09.674 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2020-08-07T15:07:09.675 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:09.689 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:07:09.768 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:09.767+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:09.872 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:09.868+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:09.906 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:07:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:07:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:07:09.911 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:09.912 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:07:09.984 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:07:09.996 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:09.995+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:10.074 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:10.071+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:10.117 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:10.117 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:07:10.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:07:10.118 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:10.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.118 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:8367 3.a 3:5e6bacde:::benchmark_data_smithi074_35447_object1045:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:10.118 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.276619+0000", 2020-08-07T15:07:10.119 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.837865873999998, 2020-08-07T15:07:10.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.141821844, 2020-08-07T15:07:10.119 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:10.119 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:10.119 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:10.120 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:10.120 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:10.120 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 8367 2020-08-07T15:07:10.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.120 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:10.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:10.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276619+0000", 2020-08-07T15:07:10.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:10.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:10.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276619+0000", 2020-08-07T15:07:10.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4978e-05 2020-08-07T15:07:10.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:10.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276644+0000", 2020-08-07T15:07:10.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3009999999999998e-06 2020-08-07T15:07:10.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:10.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276647+0000", 2020-08-07T15:07:10.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0319999999999998e-06 2020-08-07T15:07:10.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:10.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276654+0000", 2020-08-07T15:07:10.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7199999999999998e-06 2020-08-07T15:07:10.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:10.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276657+0000", 2020-08-07T15:07:10.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023605660000000001 2020-08-07T15:07:10.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:10.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.279017+0000", 2020-08-07T15:07:10.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0431000000000004e-05 2020-08-07T15:07:10.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:10.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.279088+0000", 2020-08-07T15:07:10.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016790300000000001 2020-08-07T15:07:10.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:10.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.279256+0000", 2020-08-07T15:07:10.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031354550000000001 2020-08-07T15:07:10.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:10.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.282391+0000", 2020-08-07T15:07:10.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9900000000000001e-07 2020-08-07T15:07:10.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:10.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.282392+0000", 2020-08-07T15:07:10.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5717999999999999e-05 2020-08-07T15:07:10.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:10.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.282427+0000", 2020-08-07T15:07:10.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00053778099999999998 2020-08-07T15:07:10.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:10.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.282965+0000", 2020-08-07T15:07:10.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13540729400000001 2020-08-07T15:07:10.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:10.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418372+0000", 2020-08-07T15:07:10.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9600000000000008e-07 2020-08-07T15:07:10.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:10.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418373+0000", 2020-08-07T15:07:10.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7269999999999998e-05 2020-08-07T15:07:10.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:10.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418441+0000", 2020-08-07T15:07:10.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.141821844 2020-08-07T15:07:10.142 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.142 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:10.142 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.143 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:8368 3.a 3:5e6bacde:::benchmark_data_smithi074_35447_object1045:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:10.143 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.276795+0000", 2020-08-07T15:07:10.143 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.837690148, 2020-08-07T15:07:10.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14181318200000001, 2020-08-07T15:07:10.144 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:10.144 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:10.149 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:10.150 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:10.150 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:10.150 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 8368 2020-08-07T15:07:10.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.150 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:10.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:10.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276795+0000", 2020-08-07T15:07:10.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:10.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:10.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276795+0000", 2020-08-07T15:07:10.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8793e-05 2020-08-07T15:07:10.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:10.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276823+0000", 2020-08-07T15:07:10.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3890000000000001e-06 2020-08-07T15:07:10.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:10.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276827+0000", 2020-08-07T15:07:10.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2590000000000003e-06 2020-08-07T15:07:10.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:10.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276834+0000", 2020-08-07T15:07:10.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7889999999999999e-06 2020-08-07T15:07:10.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:10.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276837+0000", 2020-08-07T15:07:10.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030534199999999998 2020-08-07T15:07:10.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:10.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.279890+0000", 2020-08-07T15:07:10.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3449000000000006e-05 2020-08-07T15:07:10.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:10.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.279984+0000", 2020-08-07T15:07:10.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020054799999999999 2020-08-07T15:07:10.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:10.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.280184+0000", 2020-08-07T15:07:10.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002684353 2020-08-07T15:07:10.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:10.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.282869+0000", 2020-08-07T15:07:10.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7899999999999998e-07 2020-08-07T15:07:10.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:10.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.282869+0000", 2020-08-07T15:07:10.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5290000000000001e-05 2020-08-07T15:07:10.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:10.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.282924+0000", 2020-08-07T15:07:10.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031297999999999997 2020-08-07T15:07:10.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:10.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.283237+0000", 2020-08-07T15:07:10.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.135332799 2020-08-07T15:07:10.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:10.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418570+0000", 2020-08-07T15:07:10.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4600000000000004e-07 2020-08-07T15:07:10.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:10.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418571+0000", 2020-08-07T15:07:10.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6888000000000003e-05 2020-08-07T15:07:10.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:10.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418608+0000", 2020-08-07T15:07:10.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14181318200000001 2020-08-07T15:07:10.167 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.167 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:10.167 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.168 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:8367 3.a e24/20 ondisk, result = 0)", 2020-08-07T15:07:10.168 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.281490+0000", 2020-08-07T15:07:10.168 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.832994733, 2020-08-07T15:07:10.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13703254000000001, 2020-08-07T15:07:10.168 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:10.168 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:10.169 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:10.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:10.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.281490+0000", 2020-08-07T15:07:10.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:10.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:10.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.281490+0000", 2020-08-07T15:07:10.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5865000000000001e-05 2020-08-07T15:07:10.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:10.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.281506+0000", 2020-08-07T15:07:10.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5269999999999999e-06 2020-08-07T15:07:10.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:10.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.281509+0000", 2020-08-07T15:07:10.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13566299600000001 2020-08-07T15:07:10.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:10.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.417172+0000", 2020-08-07T15:07:10.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7019999999999998e-06 2020-08-07T15:07:10.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:10.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.417178+0000", 2020-08-07T15:07:10.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001176947 2020-08-07T15:07:10.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:10.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418355+0000", 2020-08-07T15:07:10.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6240000000000008e-06 2020-08-07T15:07:10.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:10.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418364+0000", 2020-08-07T15:07:10.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015887900000000001 2020-08-07T15:07:10.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:10.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418523+0000", 2020-08-07T15:07:10.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13703254000000001 2020-08-07T15:07:10.177 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.178 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:10.178 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.178 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:10536 3.0 e24/20 3:0d011956:::benchmark_data_smithi074_35447_object1316:head v 24'744, mlcod=24'735)", 2020-08-07T15:07:10.179 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.564533+0000", 2020-08-07T15:07:10.179 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.549952225999998, 2020-08-07T15:07:10.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14762919499999999, 2020-08-07T15:07:10.179 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:10.180 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:10.180 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:10.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:10.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.564533+0000", 2020-08-07T15:07:10.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:10.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:10.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.564533+0000", 2020-08-07T15:07:10.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9255000000000002e-05 2020-08-07T15:07:10.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:10.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.564582+0000", 2020-08-07T15:07:10.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1309999999999997e-06 2020-08-07T15:07:10.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:10.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.564587+0000", 2020-08-07T15:07:10.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14241859400000001 2020-08-07T15:07:10.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:10.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.707006+0000", 2020-08-07T15:07:10.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2190000000000003e-06 2020-08-07T15:07:10.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:10.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.707011+0000", 2020-08-07T15:07:10.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7673999999999999e-05 2020-08-07T15:07:10.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:10.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.707048+0000", 2020-08-07T15:07:10.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7660999999999999e-05 2020-08-07T15:07:10.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:10.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.707066+0000", 2020-08-07T15:07:10.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050458969999999997 2020-08-07T15:07:10.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:10.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.712112+0000", 2020-08-07T15:07:10.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9764e-05 2020-08-07T15:07:10.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:10.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.712162+0000", 2020-08-07T15:07:10.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14762919499999999 2020-08-07T15:07:10.189 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.189 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:10.190 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.190 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:11192 3.d e24/20 3:bb990748:::benchmark_data_smithi074_35447_object1398:head v 24'680, mlcod=24'671)", 2020-08-07T15:07:10.190 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:43.006852+0000", 2020-08-07T15:07:10.191 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.10763305, 2020-08-07T15:07:10.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15926784399999999, 2020-08-07T15:07:10.191 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:10.191 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:10.191 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:10.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:10.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.006852+0000", 2020-08-07T15:07:10.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:10.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:10.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.006852+0000", 2020-08-07T15:07:10.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0237e-05 2020-08-07T15:07:10.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:10.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.006902+0000", 2020-08-07T15:07:10.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2750000000000001e-06 2020-08-07T15:07:10.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:10.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.006908+0000", 2020-08-07T15:07:10.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15839128599999999 2020-08-07T15:07:10.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:10.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.165300+0000", 2020-08-07T15:07:10.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4450000000000004e-06 2020-08-07T15:07:10.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:10.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.165305+0000", 2020-08-07T15:07:10.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3010999999999999e-05 2020-08-07T15:07:10.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:10.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.165348+0000", 2020-08-07T15:07:10.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3289e-05 2020-08-07T15:07:10.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.165371+0000", 2020-08-07T15:07:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00069550299999999998 2020-08-07T15:07:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.166067+0000", 2020-08-07T15:07:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2797999999999998e-05 2020-08-07T15:07:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.166120+0000", 2020-08-07T15:07:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15926784399999999 2020-08-07T15:07:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:17016 3.d e27/20 3:b0cf8e92:::benchmark_data_smithi074_35447_object2126:head v 27'1088, mlcod=27'1080)", 2020-08-07T15:07:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:47.662899+0000", 2020-08-07T15:07:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.451585345000002, 2020-08-07T15:07:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13459602200000001, 2020-08-07T15:07:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.662899+0000", 2020-08-07T15:07:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.662899+0000", 2020-08-07T15:07:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1554000000000002e-05 2020-08-07T15:07:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:10.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.662941+0000", 2020-08-07T15:07:10.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1989999999999999e-06 2020-08-07T15:07:10.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:10.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.662945+0000", 2020-08-07T15:07:10.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13278101 2020-08-07T15:07:10.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:10.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.795726+0000", 2020-08-07T15:07:10.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1690000000000002e-06 2020-08-07T15:07:10.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:10.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.795730+0000", 2020-08-07T15:07:10.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8744e-05 2020-08-07T15:07:10.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:10.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.795769+0000", 2020-08-07T15:07:10.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5993999999999999e-05 2020-08-07T15:07:10.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:10.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.795795+0000", 2020-08-07T15:07:10.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016482350000000001 2020-08-07T15:07:10.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:10.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.797443+0000", 2020-08-07T15:07:10.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2117000000000003e-05 2020-08-07T15:07:10.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:10.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.797495+0000", 2020-08-07T15:07:10.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13459602200000001 2020-08-07T15:07:10.230 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.230 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:10.230 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.231 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:17824 3.d e27/20 3:bd4e0935:::benchmark_data_smithi074_35447_object2227:head v 27'1168, mlcod=27'1163)", 2020-08-07T15:07:10.231 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:07:10.231 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.971044533000001, 2020-08-07T15:07:10.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25249224199999998, 2020-08-07T15:07:10.232 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:10.232 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:10.232 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:10.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:10.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:07:10.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:10.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:10.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:07:10.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9348000000000003e-05 2020-08-07T15:07:10.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:10.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143490+0000", 2020-08-07T15:07:10.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7800000000000003e-06 2020-08-07T15:07:10.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:10.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143496+0000", 2020-08-07T15:07:10.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25095783399999999 2020-08-07T15:07:10.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:10.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394454+0000", 2020-08-07T15:07:10.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6550000000000001e-06 2020-08-07T15:07:10.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:10.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394460+0000", 2020-08-07T15:07:10.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8814999999999999e-05 2020-08-07T15:07:10.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:10.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394509+0000", 2020-08-07T15:07:10.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3482999999999999e-05 2020-08-07T15:07:10.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:10.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394532+0000", 2020-08-07T15:07:10.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013496199999999999 2020-08-07T15:07:10.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:10.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.395882+0000", 2020-08-07T15:07:10.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0707000000000001e-05 2020-08-07T15:07:10.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:10.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.395932+0000", 2020-08-07T15:07:10.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25249224199999998 2020-08-07T15:07:10.243 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.244 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:10.244 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.245 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:18478 3.a 3:524412cd:::benchmark_data_smithi074_35447_object2309:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:10.245 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.592837+0000", 2020-08-07T15:07:10.246 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.521647454, 2020-08-07T15:07:10.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21005600699999999, 2020-08-07T15:07:10.246 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:10.246 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:10.246 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:10.247 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:10.247 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:10.247 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 18478 2020-08-07T15:07:10.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.247 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:10.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:10.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592837+0000", 2020-08-07T15:07:10.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:10.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:10.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592837+0000", 2020-08-07T15:07:10.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6718999999999998e-05 2020-08-07T15:07:10.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:10.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592884+0000", 2020-08-07T15:07:10.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2749999999999997e-06 2020-08-07T15:07:10.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:10.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592888+0000", 2020-08-07T15:07:10.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0831e-05 2020-08-07T15:07:10.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:10.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592899+0000", 2020-08-07T15:07:10.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9440000000000002e-06 2020-08-07T15:07:10.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:10.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592905+0000", 2020-08-07T15:07:10.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00088115000000000003 2020-08-07T15:07:10.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:10.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.593786+0000", 2020-08-07T15:07:10.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1789000000000002e-05 2020-08-07T15:07:10.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:10.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.593838+0000", 2020-08-07T15:07:10.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011097799999999999 2020-08-07T15:07:10.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:10.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.593949+0000", 2020-08-07T15:07:10.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00088680400000000004 2020-08-07T15:07:10.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:10.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.594836+0000", 2020-08-07T15:07:10.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1500000000000005e-07 2020-08-07T15:07:10.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.594836+0000", 2020-08-07T15:07:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9746000000000002e-05 2020-08-07T15:07:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.594876+0000", 2020-08-07T15:07:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045589070000000001 2020-08-07T15:07:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.599435+0000", 2020-08-07T15:07:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.203361244 2020-08-07T15:07:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:10.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802796+0000", 2020-08-07T15:07:10.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.178e-06 2020-08-07T15:07:10.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:10.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802797+0000", 2020-08-07T15:07:10.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5927000000000005e-05 2020-08-07T15:07:10.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:10.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802893+0000", 2020-08-07T15:07:10.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21005600699999999 2020-08-07T15:07:10.271 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.271 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:10.271 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.272 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:18478 3.a e27/20 ondisk, result = 0)", 2020-08-07T15:07:10.272 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.595603+0000", 2020-08-07T15:07:10.272 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.518881556, 2020-08-07T15:07:10.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.207367254, 2020-08-07T15:07:10.272 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:10.272 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:10.273 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:10.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:10.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.595603+0000", 2020-08-07T15:07:10.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:10.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:10.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.595603+0000", 2020-08-07T15:07:10.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8887999999999999e-05 2020-08-07T15:07:10.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:10.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.595622+0000", 2020-08-07T15:07:10.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8469999999999998e-06 2020-08-07T15:07:10.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:10.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.595626+0000", 2020-08-07T15:07:10.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20636312700000001 2020-08-07T15:07:10.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:10.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.801989+0000", 2020-08-07T15:07:10.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0889999999999999e-06 2020-08-07T15:07:10.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:10.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.801994+0000", 2020-08-07T15:07:10.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00078913099999999999 2020-08-07T15:07:10.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:10.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802783+0000", 2020-08-07T15:07:10.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8919999999999997e-06 2020-08-07T15:07:10.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:10.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802789+0000", 2020-08-07T15:07:10.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018128 2020-08-07T15:07:10.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:10.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802970+0000", 2020-08-07T15:07:10.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.207367254 2020-08-07T15:07:10.281 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.282 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:10.282 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.282 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:33488 3.c e30/20 3:3d1929c7:::benchmark_data_smithi074_35447_object4185:head v 30'2120, mlcod=30'2110)", 2020-08-07T15:07:10.283 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:58.023101+0000", 2020-08-07T15:07:10.283 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 12.091383765, 2020-08-07T15:07:10.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22118623300000001, 2020-08-07T15:07:10.283 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:10.283 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:10.283 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:10.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:10.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023101+0000", 2020-08-07T15:07:10.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:10.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:10.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023101+0000", 2020-08-07T15:07:10.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8794000000000001e-05 2020-08-07T15:07:10.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:10.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023150+0000", 2020-08-07T15:07:10.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.545e-06 2020-08-07T15:07:10.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:10.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023156+0000", 2020-08-07T15:07:10.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.218926757 2020-08-07T15:07:10.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:10.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242083+0000", 2020-08-07T15:07:10.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1200000000000004e-06 2020-08-07T15:07:10.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:10.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242087+0000", 2020-08-07T15:07:10.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9561999999999998e-05 2020-08-07T15:07:10.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:10.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242127+0000", 2020-08-07T15:07:10.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3655999999999998e-05 2020-08-07T15:07:10.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:10.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242150+0000", 2020-08-07T15:07:10.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002089273 2020-08-07T15:07:10.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:10.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.244240+0000", 2020-08-07T15:07:10.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7525999999999999e-05 2020-08-07T15:07:10.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:10.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.244287+0000", 2020-08-07T15:07:10.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22118623300000001 2020-08-07T15:07:10.293 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.293 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:10.293 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.294 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:35856 3.a 3:5b5556e8:::benchmark_data_smithi074_35447_object4481:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-08-07T15:07:10.294 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:59.440060+0000", 2020-08-07T15:07:10.294 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.674424709, 2020-08-07T15:07:10.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16587832199999999, 2020-08-07T15:07:10.295 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:10.295 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:10.295 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:10.295 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:10.295 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:10.296 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 35856 2020-08-07T15:07:10.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.296 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:10.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:10.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440060+0000", 2020-08-07T15:07:10.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:10.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:10.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440060+0000", 2020-08-07T15:07:10.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018524399999999999 2020-08-07T15:07:10.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:10.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440245+0000", 2020-08-07T15:07:10.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9519999999999999e-06 2020-08-07T15:07:10.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:10.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440251+0000", 2020-08-07T15:07:10.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1705999999999999e-05 2020-08-07T15:07:10.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:10.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440263+0000", 2020-08-07T15:07:10.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2820000000000004e-06 2020-08-07T15:07:10.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:10.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440268+0000", 2020-08-07T15:07:10.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023611449999999998 2020-08-07T15:07:10.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:10.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.442629+0000", 2020-08-07T15:07:10.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010644799999999999 2020-08-07T15:07:10.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:10.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.442736+0000", 2020-08-07T15:07:10.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017566000000000001 2020-08-07T15:07:10.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:10.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.442912+0000", 2020-08-07T15:07:10.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026960399999999998 2020-08-07T15:07:10.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:10.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.445608+0000", 2020-08-07T15:07:10.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0809999999999999e-06 2020-08-07T15:07:10.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:10.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.445609+0000", 2020-08-07T15:07:10.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0878999999999997e-05 2020-08-07T15:07:10.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:10.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.445680+0000", 2020-08-07T15:07:10.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00065560499999999997 2020-08-07T15:07:10.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:10.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.446335+0000", 2020-08-07T15:07:10.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.159534921 2020-08-07T15:07:10.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:10.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.605870+0000", 2020-08-07T15:07:10.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.187e-06 2020-08-07T15:07:10.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:10.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.605871+0000", 2020-08-07T15:07:10.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7172e-05 2020-08-07T15:07:10.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:10.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.605938+0000", 2020-08-07T15:07:10.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16587832199999999 2020-08-07T15:07:10.311 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.312 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:10.312 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.312 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:42104 3.d e32/20 3:bcc35439:::benchmark_data_smithi074_35447_object5262:head v 32'2864, mlcod=32'2858)", 2020-08-07T15:07:10.313 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:07:10.313 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.8714592359999997, 2020-08-07T15:07:10.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28296418899999998, 2020-08-07T15:07:10.313 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:10.313 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:10.314 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:10.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:10.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:07:10.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:10.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:10.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:07:10.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5550000000000002e-05 2020-08-07T15:07:10.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:10.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243081+0000", 2020-08-07T15:07:10.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8139999999999997e-06 2020-08-07T15:07:10.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:10.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243087+0000", 2020-08-07T15:07:10.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27968446800000002 2020-08-07T15:07:10.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:10.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522771+0000", 2020-08-07T15:07:10.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9459999999999997e-06 2020-08-07T15:07:10.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:10.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522776+0000", 2020-08-07T15:07:10.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0778000000000006e-05 2020-08-07T15:07:10.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:10.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522847+0000", 2020-08-07T15:07:10.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0622e-05 2020-08-07T15:07:10.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:10.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522868+0000", 2020-08-07T15:07:10.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030627580000000001 2020-08-07T15:07:10.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:10.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.525930+0000", 2020-08-07T15:07:10.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9252999999999997e-05 2020-08-07T15:07:10.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:10.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.525990+0000", 2020-08-07T15:07:10.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28296418899999998 2020-08-07T15:07:10.323 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.323 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:10.324 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.329 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:42704 3.2 e32/20 3:435b82af:::benchmark_data_smithi074_35447_object5337:head v 32'2440, mlcod=32'2430)", 2020-08-07T15:07:10.329 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.590578+0000", 2020-08-07T15:07:10.329 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.5239067500000001, 2020-08-07T15:07:10.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.156827727, 2020-08-07T15:07:10.330 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:10.330 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:10.330 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:10.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:10.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.590578+0000", 2020-08-07T15:07:10.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:10.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:10.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.590578+0000", 2020-08-07T15:07:10.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0890000000000002e-05 2020-08-07T15:07:10.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:10.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.590629+0000", 2020-08-07T15:07:10.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9349999999999999e-06 2020-08-07T15:07:10.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:10.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.590635+0000", 2020-08-07T15:07:10.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15515162199999999 2020-08-07T15:07:10.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:10.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.745786+0000", 2020-08-07T15:07:10.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6650000000000001e-06 2020-08-07T15:07:10.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:10.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.745790+0000", 2020-08-07T15:07:10.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1458999999999997e-05 2020-08-07T15:07:10.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:10.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.745832+0000", 2020-08-07T15:07:10.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7506000000000001e-05 2020-08-07T15:07:10.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:10.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.745849+0000", 2020-08-07T15:07:10.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001453008 2020-08-07T15:07:10.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:10.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.747302+0000", 2020-08-07T15:07:10.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010364199999999999 2020-08-07T15:07:10.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:10.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.747406+0000", 2020-08-07T15:07:10.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.156827727 2020-08-07T15:07:10.345 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.346 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:10.346 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.346 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:42944 3.2 e32/20 3:45b0b8d5:::benchmark_data_smithi074_35447_object5367:head v 32'2464, mlcod=32'2452)", 2020-08-07T15:07:10.347 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.763245+0000", 2020-08-07T15:07:10.347 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.3512402689999998, 2020-08-07T15:07:10.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.154831946, 2020-08-07T15:07:10.347 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:10.347 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:10.347 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:10.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:10.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.763245+0000", 2020-08-07T15:07:10.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:10.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:10.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.763245+0000", 2020-08-07T15:07:10.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1844e-05 2020-08-07T15:07:10.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:10.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.763296+0000", 2020-08-07T15:07:10.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4489999999999996e-06 2020-08-07T15:07:10.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:10.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.763303+0000", 2020-08-07T15:07:10.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.150368538 2020-08-07T15:07:10.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:10.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.913671+0000", 2020-08-07T15:07:10.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8499999999999998e-06 2020-08-07T15:07:10.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:10.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.913674+0000", 2020-08-07T15:07:10.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002168096 2020-08-07T15:07:10.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:10.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.915842+0000", 2020-08-07T15:07:10.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1338000000000001e-05 2020-08-07T15:07:10.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:10.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.915864+0000", 2020-08-07T15:07:10.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021568550000000001 2020-08-07T15:07:10.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:10.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.918020+0000", 2020-08-07T15:07:10.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5976000000000001e-05 2020-08-07T15:07:10.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:10.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.918076+0000", 2020-08-07T15:07:10.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.154831946 2020-08-07T15:07:10.366 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.367 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:10.367 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.367 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:44744 3.2 e32/20 3:4dba534b:::benchmark_data_smithi074_35447_object5592:head v 32'2608, mlcod=32'2602)", 2020-08-07T15:07:10.368 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:04.868608+0000", 2020-08-07T15:07:10.368 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.245877042, 2020-08-07T15:07:10.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16039398499999999, 2020-08-07T15:07:10.368 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:10.368 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:10.369 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:10.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:10.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.868608+0000", 2020-08-07T15:07:10.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:10.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:10.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.868608+0000", 2020-08-07T15:07:10.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1788e-05 2020-08-07T15:07:10.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:10.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.868660+0000", 2020-08-07T15:07:10.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2920000000000001e-06 2020-08-07T15:07:10.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:10.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.868666+0000", 2020-08-07T15:07:10.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15842231200000001 2020-08-07T15:07:10.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:10.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:05.027088+0000", 2020-08-07T15:07:10.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5329999999999996e-06 2020-08-07T15:07:10.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:10.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:05.027093+0000", 2020-08-07T15:07:10.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0534999999999998e-05 2020-08-07T15:07:10.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:10.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:05.027123+0000", 2020-08-07T15:07:10.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6172999999999999e-05 2020-08-07T15:07:10.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:10.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:05.027139+0000", 2020-08-07T15:07:10.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001801813 2020-08-07T15:07:10.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:10.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:05.028941+0000", 2020-08-07T15:07:10.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0538999999999998e-05 2020-08-07T15:07:10.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:10.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:05.029002+0000", 2020-08-07T15:07:10.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16039398499999999 2020-08-07T15:07:10.379 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.380 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:10.380 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.380 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:47792 3.a 3:5815b9e8:::benchmark_data_smithi074_35447_object5973:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e36)", 2020-08-07T15:07:10.381 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:08.669034+0000", 2020-08-07T15:07:10.381 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.4454510110000001, 2020-08-07T15:07:10.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14798557800000001, 2020-08-07T15:07:10.381 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:10.381 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:10.383 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:10.383 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:10.383 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:10.384 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 47792 2020-08-07T15:07:10.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.384 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:10.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:10.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.669034+0000", 2020-08-07T15:07:10.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:10.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:10.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.669034+0000", 2020-08-07T15:07:10.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1081000000000003e-05 2020-08-07T15:07:10.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:10.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.669065+0000", 2020-08-07T15:07:10.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3069999999999996e-06 2020-08-07T15:07:10.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:10.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.669069+0000", 2020-08-07T15:07:10.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2977e-05 2020-08-07T15:07:10.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:10.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.669082+0000", 2020-08-07T15:07:10.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0940000000000001e-06 2020-08-07T15:07:10.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:10.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.669086+0000", 2020-08-07T15:07:10.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030010919999999999 2020-08-07T15:07:10.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:10.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.672087+0000", 2020-08-07T15:07:10.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5995999999999996e-05 2020-08-07T15:07:10.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:10.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.672153+0000", 2020-08-07T15:07:10.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001739 2020-08-07T15:07:10.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:10.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.672327+0000", 2020-08-07T15:07:10.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001122867 2020-08-07T15:07:10.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:10.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.673450+0000", 2020-08-07T15:07:10.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9999999999999996e-07 2020-08-07T15:07:10.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:10.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.673451+0000", 2020-08-07T15:07:10.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3608999999999994e-05 2020-08-07T15:07:10.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:10.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.673514+0000", 2020-08-07T15:07:10.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003311151 2020-08-07T15:07:10.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:10.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.676826+0000", 2020-08-07T15:07:10.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14012928099999999 2020-08-07T15:07:10.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:10.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.816955+0000", 2020-08-07T15:07:10.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3650000000000001e-06 2020-08-07T15:07:10.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:10.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.816956+0000", 2020-08-07T15:07:10.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3058000000000005e-05 2020-08-07T15:07:10.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:10.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.817019+0000", 2020-08-07T15:07:10.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14798557800000001 2020-08-07T15:07:10.401 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.401 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:10.401 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.402 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:47792 3.a e36/20 ondisk, result = 0)", 2020-08-07T15:07:10.402 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:08.680032+0000", 2020-08-07T15:07:10.402 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.4344532130000001, 2020-08-07T15:07:10.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13699692299999999, 2020-08-07T15:07:10.403 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:10.403 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:10.403 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:10.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:10.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.680032+0000", 2020-08-07T15:07:10.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:10.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:10.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.680032+0000", 2020-08-07T15:07:10.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0349e-05 2020-08-07T15:07:10.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:10.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.680062+0000", 2020-08-07T15:07:10.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2020-08-07T15:07:10.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:10.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.680067+0000", 2020-08-07T15:07:10.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.136824324 2020-08-07T15:07:10.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:10.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.816891+0000", 2020-08-07T15:07:10.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.186e-06 2020-08-07T15:07:10.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:10.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.816896+0000", 2020-08-07T15:07:10.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0296000000000002e-05 2020-08-07T15:07:10.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:10.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.816937+0000", 2020-08-07T15:07:10.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2629999999999992e-06 2020-08-07T15:07:10.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:10.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.816946+0000", 2020-08-07T15:07:10.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2458000000000002e-05 2020-08-07T15:07:10.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:10.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.817028+0000", 2020-08-07T15:07:10.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13699692299999999 2020-08-07T15:07:10.413 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.413 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:10.413 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.414 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:48144 3.0 e36/20 3:0cec1d53:::benchmark_data_smithi074_35447_object6017:head v 36'3128, mlcod=36'3120)", 2020-08-07T15:07:10.414 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:08.897123+0000", 2020-08-07T15:07:10.414 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.2173615259999999, 2020-08-07T15:07:10.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25221023399999998, 2020-08-07T15:07:10.415 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:10.415 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:10.415 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:10.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:10.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897123+0000", 2020-08-07T15:07:10.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:10.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:10.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897123+0000", 2020-08-07T15:07:10.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0655999999999997e-05 2020-08-07T15:07:10.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:10.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897174+0000", 2020-08-07T15:07:10.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6769999999999998e-06 2020-08-07T15:07:10.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:10.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897181+0000", 2020-08-07T15:07:10.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25090537899999998 2020-08-07T15:07:10.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:10.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148086+0000", 2020-08-07T15:07:10.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5520000000000004e-06 2020-08-07T15:07:10.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:10.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148092+0000", 2020-08-07T15:07:10.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6391999999999999e-05 2020-08-07T15:07:10.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:10.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148138+0000", 2020-08-07T15:07:10.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8124999999999999e-05 2020-08-07T15:07:10.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:10.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148166+0000", 2020-08-07T15:07:10.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011152009999999999 2020-08-07T15:07:10.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:10.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.149281+0000", 2020-08-07T15:07:10.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2252e-05 2020-08-07T15:07:10.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:10.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.149333+0000", 2020-08-07T15:07:10.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25221023399999998 2020-08-07T15:07:10.427 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.427 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:10.427 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.428 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:48776 3.2 e36/20 3:47029331:::benchmark_data_smithi074_35447_object6096:head v 36'2888, mlcod=36'2881)", 2020-08-07T15:07:10.428 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:07:10.428 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.70996471100000003, 2020-08-07T15:07:10.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27006165399999998, 2020-08-07T15:07:10.429 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:10.430 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:10.431 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:10.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:10.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:07:10.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:10.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:10.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:07:10.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8454000000000001e-05 2020-08-07T15:07:10.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:10.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404569+0000", 2020-08-07T15:07:10.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2349999999999996e-06 2020-08-07T15:07:10.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:10.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404575+0000", 2020-08-07T15:07:10.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26921690999999998 2020-08-07T15:07:10.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:10.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673792+0000", 2020-08-07T15:07:10.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3330000000000002e-06 2020-08-07T15:07:10.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:10.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673798+0000", 2020-08-07T15:07:10.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1013000000000001e-05 2020-08-07T15:07:10.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:10.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673849+0000", 2020-08-07T15:07:10.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7044999999999999e-05 2020-08-07T15:07:10.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:10.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673876+0000", 2020-08-07T15:07:10.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00065173100000000001 2020-08-07T15:07:10.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:10.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.674528+0000", 2020-08-07T15:07:10.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3933e-05 2020-08-07T15:07:10.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:10.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.674582+0000", 2020-08-07T15:07:10.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27006165399999998 2020-08-07T15:07:10.440 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.440 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:10.443 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.443 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:49208 3.2 e36/20 3:4d72b982:::benchmark_data_smithi074_35447_object6150:head v 36'2904, mlcod=36'2897)", 2020-08-07T15:07:10.443 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:09.692013+0000", 2020-08-07T15:07:10.444 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.42247168299999999, 2020-08-07T15:07:10.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.234386975, 2020-08-07T15:07:10.444 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:10.444 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:10.444 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:10.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:10.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.692013+0000", 2020-08-07T15:07:10.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:10.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:10.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.692013+0000", 2020-08-07T15:07:10.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0998000000000002e-05 2020-08-07T15:07:10.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:10.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.692064+0000", 2020-08-07T15:07:10.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0210000000000001e-05 2020-08-07T15:07:10.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:10.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.692074+0000", 2020-08-07T15:07:10.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.233053807 2020-08-07T15:07:10.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:10.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.925128+0000", 2020-08-07T15:07:10.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4879999999999998e-06 2020-08-07T15:07:10.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:10.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.925134+0000", 2020-08-07T15:07:10.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5154999999999997e-05 2020-08-07T15:07:10.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:10.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.925179+0000", 2020-08-07T15:07:10.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2710000000000001e-05 2020-08-07T15:07:10.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:10.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.925201+0000", 2020-08-07T15:07:10.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001143725 2020-08-07T15:07:10.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:10.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.926345+0000", 2020-08-07T15:07:10.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4882000000000001e-05 2020-08-07T15:07:10.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:10.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:10.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:10.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.926400+0000", 2020-08-07T15:07:10.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.234386975 2020-08-07T15:07:10.466 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.466 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:10.467 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.467 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:10.467 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:10.467 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:10.468 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:07:10.471 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:07:10.471 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:07:10.471 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:07:10.471 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:07:10.472 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:07:10.472 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:07:10.472 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:07:10.472 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:07:10.472 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 34 16 49854 49838 11.4508 10.1875 0.065666 0.0108676 2020-08-07T15:07:10.473 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:07:10.473 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:07:10.473 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:07:10.473 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:07:10.474 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:07:10.474 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:07:10.474 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:07:10.474 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:07:10.475 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:10.468+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:10.475 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:07:10.570 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:10.569+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:10.674 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:10.670+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:10.692 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:10.692 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:10.705 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:07:10.772 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:10.771+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:10.873 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:10.873+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:10.908 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:10.908 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:10.919 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:07:10.975 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:10.973+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:11.082 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:11.076+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:11.105 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:11.105 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:11.118 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:07:11.177 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:11.177+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:11.243 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 35 16 51314 51298 11.4495 11.4062 0.00504546 0.0109121 2020-08-07T15:07:11.282 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:11.277+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:11.294 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:11.294 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:11.302 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:07:11.351 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:07:11.351 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:07:11.353 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_0 2020-08-07T15:07:11.362 INFO:tasks.ceph.ceph_manager.ceph:contract_pool unique_pool_0 by 10 min 8 2020-08-07T15:07:11.363 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:07:11.364 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:07:11.419 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:11.418+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:11.486 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:11.483+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:11.538 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:11.539 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:11.552 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:07:11.585 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:11.584+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:11.694 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:11.692+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:11.780 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:11.781 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:11.792 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:07:11.796 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:07:11.797 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:11.794+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:11.799 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:07:11.813 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2020-08-07T15:07:11.356832+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":382664704,"num_objects":5841,"num_object_clones":0,"num_object_copies":17523,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46714,"num_write_kb":373698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":75,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9520364,"kb_used_data":1131692,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":745454356,"statfs":{"total":773094113280,"available":763345260544,"internally_reserved":8589934592,"allocated":1158852608,"data_stored":1157154054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,8,9,9,10,1,0,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":57802752,"num_objects":881,"num_object_clones":0,"num_object_copies":2643,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7056,"num_write_kb":56448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.033598"},"pg_stats":[{"pgid":"3.f","version":"33'3184","reported_seq":"3205","reported_epoch":"33","state":"active+clean","last_fresh":"2020-08-07T15:07:06.314602+0000","last_change":"2020-08-07T15:06:35.163498+0000","last_active":"2020-08-07T15:07:06.314602+0000","last_peered":"2020-08-07T15:07:06.314602+0000","last_clean":"2020-08-07T15:07:06.314602+0000","last_became_active":"2020-08-07T15:06:35.162764+0000","last_became_peered":"2020-08-07T15:06:35.162764+0000","last_unstale":"2020-08-07T15:07:06.314602+0000","last_undegraded":"2020-08-07T15:07:06.314602+0000","last_fullsized":"2020-08-07T15:07:06.314602+0000","mapping_epoch":20,"log_start":"33'3177","ondisk_log_start":"33'3177","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":26083328,"num_objects":398,"num_object_clones":0,"num_object_copies":1194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3184,"num_write_kb":25472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"33'2928","reported_seq":"2950","reported_epoch":"34","state":"active+clean","last_fresh":"2020-08-07T15:07:06.327364+0000","last_change":"2020-08-07T15:06:35.169137+0000","last_active":"2020-08-07T15:07:06.327364+0000","last_peered":"2020-08-07T15:07:06.327364+0000","last_clean":"2020-08-07T15:07:06.327364+0000","last_became_active":"2020-08-07T15:06:35.168689+0000","last_became_peered":"2020-08-07T15:06:35.168689+0000","last_unstale":"2020-08-07T15:07:06.327364+0000","last_undegraded":"2020-08-07T15:07:06.327364+0000","last_fullsized":"2020-08-07T15:07:06.327364+0000","mapping_epoch":20,"log_start":"33'2920","ondisk_log_start":"33'2920","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23986176,"num_objects":366,"num_object_clones":0,"num_object_copies":1098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2928,"num_write_kb":23424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"31'2680","reported_seq":"2699","reported_epoch":"31","state":"active+clean","last_fresh":"2020-08-07T15:07:01.632007+0000","last_change":"2020-08-07T15:06:35.165586+0000","last_active":"2020-08-07T15:07:01.632007+0000","last_peered":"2020-08-07T15:07:01.632007+0000","last_clean":"2020-08-07T15:07:01.632007+0000","last_became_active":"2020-08-07T15:06:35.165144+0000","last_became_peered":"2020-08-07T15:06:35.165144+0000","last_unstale":"2020-08-07T15:07:01.632007+0000","last_undegraded":"2020-08-07T15:07:01.632007+0000","last_fullsized":"2020-08-07T15:07:01.632007+0000","mapping_epoch":20,"log_start":"31'2672","ondisk_log_start":"31'2672","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21954560,"num_objects":335,"num_object_clones":0,"num_object_copies":1005,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2680,"num_write_kb":21440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"33'2960","reported_seq":"2981","reported_epoch":"34","state":"active+clean","last_fresh":"2020-08-07T15:07:06.590093+0000","last_change":"2020-08-07T15:06:35.167480+0000","last_active":"2020-08-07T15:07:06.590093+0000","last_peered":"2020-08-07T15:07:06.590093+0000","last_clean":"2020-08-07T15:07:06.590093+0000","last_became_active":"2020-08-07T15:06:35.167101+0000","last_became_peered":"2020-08-07T15:06:35.167101+0000","last_unstale":"2020-08-07T15:07:06.590093+0000","last_undegraded":"2020-08-07T15:07:06.590093+0000","last_fullsized":"2020-08-07T15:07:06.590093+0000","mapping_epoch":20,"log_start":"33'2951","ondisk_log_start":"33'2951","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24248320,"num_objects":370,"num_object_clones":0,"num_object_copies":1110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2960,"num_write_kb":23680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.b","version":"34'2896","reported_seq":"2917","reported_epoch":"34","state":"active+clean","last_fresh":"2020-08-07T15:07:06.354611+0000","last_change":"2020-08-07T15:06:35.170504+0000","last_active":"2020-08-07T15:07:06.354611+0000","last_peered":"2020-08-07T15:07:06.354611+0000","last_clean":"2020-08-07T15:07:06.354611+0000","last_became_active":"2020-08-07T15:06:35.169532+0000","last_became_peered":"2020-08-07T15:06:35.169532+0000","last_unstale":"2020-08-07T15:07:06.354611+0000","last_undegraded":"2020-08-07T15:07:06.354611+0000","last_fullsized":"2020-08-07T15:07:06.354611+0000","mapping_epoch":20,"log_start":"34'2893","ondisk_log_start":"34'2893","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23724032,"num_objects":362,"num_object_clones":0,"num_object_copies":1086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2896,"num_write_kb":23168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"34'3304","reported_seq":"3327","reported_epoch":"34","state":"active+clean","last_fresh":"2020-08-07T15:07:06.834196+0000","last_change":"2020-08-07T15:06:35.170105+0000","last_active":"2020-08-07T15:07:06.834196+0000","last_peered":"2020-08-07T15:07:06.834196+0000","last_clean":"2020-08-07T15:07:06.834196+0000","last_became_active":"2020-08-07T15:06:35.169494+0000","last_became_peered":"2020-08-07T15:06:35.169494+0000","last_unstale":"2020-08-07T15:07:06.834196+0000","last_undegraded":"2020-08-07T15:07:06.834196+0000","last_fullsized":"2020-08-07T15:07:06.834196+0000","mapping_epoch":20,"log_start":"34'3300","ondisk_log_start":"34'3300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27066368,"num_objects":413,"num_object_clones":0,"num_object_copies":1239,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3304,"num_write_kb":26432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"33'2792","reported_seq":"2814","reported_epoch":"33","state":"active+clean","last_fresh":"2020-08-07T15:07:06.246198+0000","last_change":"2020-08-07T15:06:35.163109+0000","last_active":"2020-08-07T15:07:06.246198+0000","last_peered":"2020-08-07T15:07:06.246198+0000","last_clean":"2020-08-07T15:07:06.246198+0000","last_became_active":"2020-08-07T15:06:35.162721+0000","last_became_peered":"2020-08-07T15:06:35.162721+0000","last_unstale":"2020-08-07T15:07:06.246198+0000","last_undegraded":"2020-08-07T15:07:06.246198+0000","last_fullsized":"2020-08-07T15:07:06.246198+0000","mapping_epoch":20,"log_start":"33'2785","ondisk_log_start":"33'2785","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22872064,"num_objects":349,"num_object_clones":0,"num_object_copies":1047,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2792,"num_write_kb":22336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"33'2968","reported_seq":"2990","reported_epoch":"33","state":"active+clean","last_fresh":"2020-08-07T15:07:05.938972+0000","last_change":"2020-08-07T15:06:35.173504+0000","last_active":"2020-08-07T15:07:05.938972+0000","last_peered":"2020-08-07T15:07:05.938972+0000","last_clean":"2020-08-07T15:07:05.938972+0000","last_became_active":"2020-08-07T15:06:35.172437+0000","last_became_peered":"2020-08-07T15:06:35.172437+0000","last_unstale":"2020-08-07T15:07:05.938972+0000","last_undegraded":"2020-08-07T15:07:05.938972+0000","last_fullsized":"2020-08-07T15:07:05.938972+0000","mapping_epoch":20,"log_start":"33'2960","ondisk_log_start":"33'2960","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24313856,"num_objects":371,"num_object_clones":0,"num_object_copies":1113,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2968,"num_write_kb":23744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"22","reported_epoch":"33","state":"active+clean","last_fresh":"2020-08-07T15:07:05.314879+0000","last_change":"2020-08-07T15:06:46.199001+0000","last_active":"2020-08-07T15:07:05.314879+0000","last_peered":"2020-08-07T15:07:05.314879+0000","last_clean":"2020-08-07T15:07:05.314879+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:07:05.314879+0000","last_undegraded":"2020-08-07T15:07:05.314879+0000","last_fullsized":"2020-08-07T15:07:05.314879+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.5","version":"33'2736","reported_seq":"2758","reported_epoch":"33","state":"active+clean","last_fresh":"2020-08-07T15:07:06.076433+0000","last_change":"2020-08-07T15:06:35.169307+0000","last_active":"2020-08-07T15:07:06.076433+0000","last_peered":"2020-08-07T15:07:06.076433+0000","last_clean":"2020-08-07T15:07:06.076433+0000","last_became_active":"2020-08-07T15:06:35.168887+0000","last_became_peered":"2020-08-07T15:06:35.168887+0000","last_unstale":"2020-08-07T15:07:06.076433+0000","last_undegraded":"2020-08-07T15:07:06.076433+0000","last_fullsized":"2020-08-07T15:07:06.076433+0000","mapping_epoch":20,"log_start":"33'2728","ondisk_log_start":"33'2728","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22413312,"num_objects":342,"num_object_clones":0,"num_object_copies":1026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2736,"num_write_kb":21888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"25","reported_epoch":"33","state":"active+clean","last_fresh":"2020-08-07T15:07:05.339373+0000","last_change":"2020-08-07T15:06:23.074962+0000","last_active":"2020-08-07T15:07:05.339373+0000","last_peered":"2020-08-07T15:07:05.339373+0000","last_clean":"2020-08-07T15:07:05.339373+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:07:05.339373+0000","last_undegraded":"2020-08-07T15:07:05.339373+0000","last_fullsized":"2020-08-07T15:07:05.339373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"33'2816","reported_seq":"2838","reported_epoch":"34","state":"active+clean","last_fresh":"2020-08-07T15:07:06.551061+0000","last_change":"2020-08-07T15:06:35.164593+0000","last_active":"2020-08-07T15:07:06.551061+0000","last_peered":"2020-08-07T15:07:06.551061+0000","last_clean":"2020-08-07T15:07:06.551061+0000","last_became_active":"2020-08-07T15:06:35.164108+0000","last_became_peered":"2020-08-07T15:06:35.164108+0000","last_unstale":"2020-08-07T15:07:06.551061+0000","last_undegraded":"2020-08-07T15:07:06.551061+0000","last_fullsized":"2020-08-07T15:07:06.551061+0000","mapping_epoch":20,"log_start":"33'2808","ondisk_log_start":"33'2808","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23068672,"num_objects":352,"num_object_clones":0,"num_object_copies":1056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2816,"num_write_kb":22528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"25","reported_epoch":"33","state":"active+clean","last_fresh":"2020-08-07T15:07:05.322306+0000","last_change":"2020-08-07T15:06:23.223924+0000","last_active":"2020-08-07T15:07:05.322306+0000","last_peered":"2020-08-07T15:07:05.322306+0000","last_clean":"2020-08-07T15:07:05.322306+0000","last_became_active":"2020-08-07T15:06:23.223489+0000","last_became_peered":"2020-08-07T15:06:23.223489+0000","last_unstale":"2020-08-07T15:07:05.322306+0000","last_undegraded":"2020-08-07T15:07:05.322306+0000","last_fullsized":"2020-08-07T15:07:05.322306+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"34'3056","reported_seq":"3076","reported_epoch":"34","state":"active+clean","last_fresh":"2020-08-07T15:07:06.344625+0000","last_change":"2020-08-07T15:06:35.170553+0000","last_active":"2020-08-07T15:07:06.344625+0000","last_peered":"2020-08-07T15:07:06.344625+0000","last_clean":"2020-08-07T15:07:06.344625+0000","last_became_active":"2020-08-07T15:06:35.170037+0000","last_became_peered":"2020-08-07T15:06:35.170037+0000","last_unstale":"2020-08-07T15:07:06.344625+0000","last_undegraded":"2020-08-07T15:07:06.344625+0000","last_fullsized":"2020-08-07T15:07:06.344625+0000","mapping_epoch":20,"log_start":"33'3048","ondisk_log_start":"33'3048","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25034752,"num_objects":382,"num_object_clones":0,"num_object_copies":1146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3056,"num_write_kb":24448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"25","reported_epoch":"33","state":"active+clean","last_fresh":"2020-08-07T15:07:05.339292+0000","last_change":"2020-08-07T15:06:23.075958+0000","last_active":"2020-08-07T15:07:05.339292+0000","last_peered":"2020-08-07T15:07:05.339292+0000","last_clean":"2020-08-07T15:07:05.339292+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:07:05.339292+0000","last_undegraded":"2020-08-07T15:07:05.339292+0000","last_fullsized":"2020-08-07T15:07:05.339292+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"32","reported_epoch":"33","state":"active+clean","last_fresh":"2020-08-07T15:07:05.314649+0000","last_change":"2020-08-07T15:06:19.198215+0000","last_active":"2020-08-07T15:07:05.314649+0000","last_peered":"2020-08-07T15:07:05.314649+0000","last_clean":"2020-08-07T15:07:05.314649+0000","last_became_active":"2020-08-07T15:06:19.197819+0000","last_became_peered":"2020-08-07T15:06:19.197819+0000","last_unstale":"2020-08-07T15:07:05.314649+0000","last_undegraded":"2020-08-07T15:07:05.314649+0000","last_fullsized":"2020-08-07T15:07:05.314649+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:06:10.878833+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"33'2768","reported_seq":"2790","reported_epoch":"33","state":"active+clean","last_fresh":"2020-08-07T15:07:06.327337+0000","last_change":"2020-08-07T15:06:35.169888+0000","last_active":"2020-08-07T15:07:06.327337+0000","last_peered":"2020-08-07T15:07:06.327337+0000","last_clean":"2020-08-07T15:07:06.327337+0000","last_became_active":"2020-08-07T15:06:35.169385+0000","last_became_peered":"2020-08-07T15:06:35.169385+0000","last_unstale":"2020-08-07T15:07:06.327337+0000","last_undegraded":"2020-08-07T15:07:06.327337+0000","last_fullsized":"2020-08-07T15:07:06.327337+0000","mapping_epoch":20,"log_start":"33'2762","ondisk_log_start":"33'2762","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22675456,"num_objects":346,"num_object_clones":0,"num_object_copies":1038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2768,"num_write_kb":22144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"25","reported_epoch":"33","state":"active+clean","last_fresh":"2020-08-07T15:07:05.314707+0000","last_change":"2020-08-07T15:06:23.224115+0000","last_active":"2020-08-07T15:07:05.314707+0000","last_peered":"2020-08-07T15:07:05.314707+0000","last_clean":"2020-08-07T15:07:05.314707+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:07:05.314707+0000","last_undegraded":"2020-08-07T15:07:05.314707+0000","last_fullsized":"2020-08-07T15:07:05.314707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"33'2776","reported_seq":"2799","reported_epoch":"33","state":"active+clean","last_fresh":"2020-08-07T15:07:06.181859+0000","last_change":"2020-08-07T15:06:46.214618+0000","last_active":"2020-08-07T15:07:06.181859+0000","last_peered":"2020-08-07T15:07:06.181859+0000","last_clean":"2020-08-07T15:07:06.181859+0000","last_became_active":"2020-08-07T15:06:46.202192+0000","last_became_peered":"2020-08-07T15:06:46.202192+0000","last_unstale":"2020-08-07T15:07:06.181859+0000","last_undegraded":"2020-08-07T15:07:06.181859+0000","last_fullsized":"2020-08-07T15:07:06.181859+0000","mapping_epoch":26,"log_start":"33'2769","ondisk_log_start":"33'2769","created":20,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22740992,"num_objects":347,"num_object_clones":0,"num_object_copies":1041,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2776,"num_write_kb":22208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3],"acting":[4,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"25","reported_epoch":"33","state":"active+clean","last_fresh":"2020-08-07T15:07:05.314845+0000","last_change":"2020-08-07T15:06:23.225886+0000","last_active":"2020-08-07T15:07:05.314845+0000","last_peered":"2020-08-07T15:07:05.314845+0000","last_clean":"2020-08-07T15:07:05.314845+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:07:05.314845+0000","last_undegraded":"2020-08-07T15:07:05.314845+0000","last_fullsized":"2020-08-07T15:07:05.314845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"33'2944","reported_seq":"2966","reported_epoch":"33","state":"active+clean","last_fresh":"2020-08-07T15:07:06.330883+0000","last_change":"2020-08-07T15:06:35.169626+0000","last_active":"2020-08-07T15:07:06.330883+0000","last_peered":"2020-08-07T15:07:06.330883+0000","last_clean":"2020-08-07T15:07:06.330883+0000","last_became_active":"2020-08-07T15:06:35.169127+0000","last_became_peered":"2020-08-07T15:06:35.169127+0000","last_unstale":"2020-08-07T15:07:06.330883+0000","last_undegraded":"2020-08-07T15:07:06.330883+0000","last_fullsized":"2020-08-07T15:07:06.330883+0000","mapping_epoch":20,"log_start":"33'2938","ondisk_log_start":"33'2938","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24117248,"num_objects":368,"num_object_clones":0,"num_object_copies":1104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2944,"num_write_kb":23552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"33'2840","reported_seq":"2862","reported_epoch":"34","state":"active+clean","last_fresh":"2020-08-07T15:07:06.330253+0000","last_change":"2020-08-07T15:06:35.172780+0000","last_active":"2020-08-07T15:07:06.330253+0000","last_peered":"2020-08-07T15:07:06.330253+0000","last_clean":"2020-08-07T15:07:06.330253+0000","last_became_active":"2020-08-07T15:06:35.172360+0000","last_became_peered":"2020-08-07T15:06:35.172360+0000","last_unstale":"2020-08-07T15:07:06.330253+0000","last_undegraded":"2020-08-07T15:07:06.330253+0000","last_fullsized":"2020-08-07T15:07:06.330253+0000","mapping_epoch":20,"log_start":"33'2833","ondisk_log_start":"33'2833","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23265280,"num_objects":355,"num_object_clones":0,"num_object_copies":1065,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2840,"num_write_kb":22720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"25","reported_epoch":"33","state":"active+clean","last_fresh":"2020-08-07T15:07:05.306626+0000","last_change":"2020-08-07T15:06:23.074076+0000","last_active":"2020-08-07T15:07:05.306626+0000","last_peered":"2020-08-07T15:07:05.306626+0000","last_clean":"2020-08-07T15:07:05.306626+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:07:05.306626+0000","last_undegraded":"2020-08-07T15:07:05.306626+0000","last_fullsized":"2020-08-07T15:07:05.306626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"26","reported_epoch":"34","state":"active+clean","last_fresh":"2020-08-07T15:07:06.326516+0000","last_change":"2020-08-07T15:06:23.073702+0000","last_active":"2020-08-07T15:07:06.326516+0000","last_peered":"2020-08-07T15:07:06.326516+0000","last_clean":"2020-08-07T15:07:06.326516+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:07:06.326516+0000","last_undegraded":"2020-08-07T15:07:06.326516+0000","last_fullsized":"2020-08-07T15:07:06.326516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"34'3064","reported_seq":"3088","reported_epoch":"34","state":"active+clean","last_fresh":"2020-08-07T15:07:06.343898+0000","last_change":"2020-08-07T15:06:46.199635+0000","last_active":"2020-08-07T15:07:06.343898+0000","last_peered":"2020-08-07T15:07:06.343898+0000","last_clean":"2020-08-07T15:07:06.343898+0000","last_became_active":"2020-08-07T15:06:46.199201+0000","last_became_peered":"2020-08-07T15:06:46.199201+0000","last_unstale":"2020-08-07T15:07:06.343898+0000","last_undegraded":"2020-08-07T15:07:06.343898+0000","last_fullsized":"2020-08-07T15:07:06.343898+0000","mapping_epoch":26,"log_start":"34'3061","ondisk_log_start":"34'3061","created":20,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25100288,"num_objects":383,"num_object_clones":0,"num_object_copies":1149,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3064,"num_write_kb":24512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":382664704,"num_objects":5839,"num_object_clones":0,"num_object_copies":17517,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46712,"num_write_kb":373696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1158012928,"data_stored":1158012928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542156,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1192852,"kb_used_data":144268,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93178988,"statfs":{"total":96636764160,"available":95415283712,"internally_reserved":1073741824,"allocated":147730432,"data_stored":147514140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]}]},{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1191324,"kb_used_data":142740,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93180516,"statfs":{"total":96636764160,"available":95416848384,"internally_reserved":1073741824,"allocated":146165760,"data_stored":145949468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,0,0,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.333}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.405}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1195032,"kb_used_data":146448,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93176808,"statfs":{"total":96636764160,"available":95413051392,"internally_reserved":1073741824,"allocated":149962752,"data_stored":149749728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6020000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8680000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6359999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1189512,"kb_used_data":140928,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93182328,"statfs":{"total":96636764160,"available":95418703872,"internally_reserved":1073741824,"allocated":144310272,"data_stored":144097498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.47}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4930000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1139456,"kb_used_data":90872,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93232384,"statfs":{"total":96636764160,"available":95469961216,"internally_reserved":1073741824,"allocated":93052928,"data_stored":92850893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,5,5,8,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.411}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1320000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.163}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2959999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1208472,"kb_used_data":159888,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93163368,"statfs":{"total":96636764160,"available":95399288832,"internally_reserved":1073741824,"allocated":163725312,"data_stored":163512288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,2,4,1,7,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6779999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5470000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.645}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.1639999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.758}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.714}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2869999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1239004,"kb_used_data":190420,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93132836,"statfs":{"total":96636764160,"available":95368024064,"internally_reserved":1073741824,"allocated":194990080,"data_stored":194773788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4910000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1164712,"kb_used_data":116128,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93207128,"statfs":{"total":96636764160,"available":95444099072,"internally_reserved":1073741824,"allocated":118915072,"data_stored":118706251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1759999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":530.30799999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]}]}]}} 2020-08-07T15:07:11.816 INFO:tasks.ceph.ceph_manager.ceph:would drop below min_pgs, proj 6, currently 16 2020-08-07T15:07:11.901 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:11.896+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:11.962 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:11.962 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:11.973 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:07:12.017 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:12.016+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:12.105 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:12.104+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:12.138 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:12.139 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:12.158 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:07:12.243 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:12.242+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:12.243 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 36 16 52789 52773 11.4515 11.5234 0.00564989 0.0109048 2020-08-07T15:07:12.307 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:12.306+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:12.369 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:12.370 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:12.382 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:07:12.425 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:12.423+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:12.476 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:07:12.518 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:12.516+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:12.619 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:12.617+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:12.619 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:12.620 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:12.632 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:07:12.719 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:12.718+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:12.744 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:07:12.782 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:07:12.816 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:07:12.816 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:12.816 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:07:12.816 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:07:12.817 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:07:12.817 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:12.821 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:12.820+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:12.827 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:07:12.852 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:07:12.902 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:07:12.927 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:12.923+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:12.927 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:07:12.964 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:07:13.006 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:07:13.007 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:07:13.007 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:07:13.007 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:07:13.007 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:07:13.007 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:07:13.008 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:07:13.008 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:07:13.008 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:07:13.032 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:13.032 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:07:13.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:07:13.033 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:07:13.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.033 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:1592 3.5 e23/20 3:ab93003a:::benchmark_data_smithi074_35447_object198:head v 23'88, mlcod=23'81)", 2020-08-07T15:07:13.033 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:37.295641+0000", 2020-08-07T15:07:13.034 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 35.733381780999999, 2020-08-07T15:07:13.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.127764135, 2020-08-07T15:07:13.034 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:13.034 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:13.034 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:13.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:13.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.295641+0000", 2020-08-07T15:07:13.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:13.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:13.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.295641+0000", 2020-08-07T15:07:13.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4150000000000003e-05 2020-08-07T15:07:13.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.037 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:13.037 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.295685+0000", 2020-08-07T15:07:13.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5971e-05 2020-08-07T15:07:13.038 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.038 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:13.038 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.295701+0000", 2020-08-07T15:07:13.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.12675505100000001 2020-08-07T15:07:13.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:13.040 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.422456+0000", 2020-08-07T15:07:13.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7000000000000002e-06 2020-08-07T15:07:13.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.041 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:13.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.422460+0000", 2020-08-07T15:07:13.042 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3812e-05 2020-08-07T15:07:13.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.042 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:13.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.422504+0000", 2020-08-07T15:07:13.042 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3331000000000001e-05 2020-08-07T15:07:13.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:13.043 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.422527+0000", 2020-08-07T15:07:13.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00082727300000000001 2020-08-07T15:07:13.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.044 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:13.044 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.423354+0000", 2020-08-07T15:07:13.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0847000000000003e-05 2020-08-07T15:07:13.045 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.045 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:13.045 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.423405+0000", 2020-08-07T15:07:13.045 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.127764135 2020-08-07T15:07:13.046 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.046 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:13.046 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.046 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.046 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.046 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:7848 3.9 e24/20 3:905b1512:::benchmark_data_smithi074_35447_object980:head v 24'408, mlcod=24'399)", 2020-08-07T15:07:13.047 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.898821+0000", 2020-08-07T15:07:13.047 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 32.130201655, 2020-08-07T15:07:13.047 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.182786849, 2020-08-07T15:07:13.047 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:13.047 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:13.048 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:13.048 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.048 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:13.048 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.898821+0000", 2020-08-07T15:07:13.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:13.049 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.049 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.049 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:13.049 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.898821+0000", 2020-08-07T15:07:13.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5016000000000002e-05 2020-08-07T15:07:13.050 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.050 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.050 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:13.050 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.898886+0000", 2020-08-07T15:07:13.051 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2740000000000007e-06 2020-08-07T15:07:13.051 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.051 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:13.051 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.898894+0000", 2020-08-07T15:07:13.052 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.181521612 2020-08-07T15:07:13.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.052 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.052 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:13.052 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.080416+0000", 2020-08-07T15:07:13.052 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8980000000000004e-06 2020-08-07T15:07:13.053 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.053 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:13.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.080421+0000", 2020-08-07T15:07:13.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6742000000000001e-05 2020-08-07T15:07:13.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.064 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:13.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.080468+0000", 2020-08-07T15:07:13.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3512000000000001e-05 2020-08-07T15:07:13.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.065 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:13.065 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.080491+0000", 2020-08-07T15:07:13.065 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010625750000000001 2020-08-07T15:07:13.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.066 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.066 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:13.066 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.081554+0000", 2020-08-07T15:07:13.066 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4219999999999999e-05 2020-08-07T15:07:13.066 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.066 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.067 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:13.067 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.081608+0000", 2020-08-07T15:07:13.067 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.182786849 2020-08-07T15:07:13.067 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.067 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:13.068 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.068 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.068 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:8112 3.9 e24/20 3:9b275d48:::benchmark_data_smithi074_35447_object1013:head v 24'416, mlcod=24'408)", 2020-08-07T15:07:13.068 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.085042+0000", 2020-08-07T15:07:13.069 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 31.943981202, 2020-08-07T15:07:13.069 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31443637000000002, 2020-08-07T15:07:13.069 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:13.069 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:13.069 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:13.070 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.070 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:13.070 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085042+0000", 2020-08-07T15:07:13.070 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:13.070 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.070 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.071 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:13.071 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085042+0000", 2020-08-07T15:07:13.071 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4534000000000003e-05 2020-08-07T15:07:13.071 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.071 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.072 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:13.072 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085096+0000", 2020-08-07T15:07:13.072 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5789999999999998e-06 2020-08-07T15:07:13.072 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.072 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.073 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:13.073 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085104+0000", 2020-08-07T15:07:13.073 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31316244700000001 2020-08-07T15:07:13.073 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.073 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.074 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:13.074 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398266+0000", 2020-08-07T15:07:13.074 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3019999999999996e-06 2020-08-07T15:07:13.074 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.074 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.075 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:13.075 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398270+0000", 2020-08-07T15:07:13.075 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3467000000000002e-05 2020-08-07T15:07:13.075 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.075 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.076 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:13.076 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398314+0000", 2020-08-07T15:07:13.076 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8184000000000001e-05 2020-08-07T15:07:13.076 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.076 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.077 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:13.077 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398332+0000", 2020-08-07T15:07:13.077 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010908059999999999 2020-08-07T15:07:13.077 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.077 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.078 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:13.078 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.399423+0000", 2020-08-07T15:07:13.078 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5050999999999999e-05 2020-08-07T15:07:13.078 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.078 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.079 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:13.079 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.399478+0000", 2020-08-07T15:07:13.079 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31443637000000002 2020-08-07T15:07:13.079 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.079 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:13.080 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.080 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.080 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.080 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:8760 3.9 e24/20 3:9f02d263:::benchmark_data_smithi074_35447_object1094:head v 24'448, mlcod=24'441)", 2020-08-07T15:07:13.080 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.535293+0000", 2020-08-07T15:07:13.081 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 31.49373001, 2020-08-07T15:07:13.081 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.194025799, 2020-08-07T15:07:13.081 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:13.081 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:13.081 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:13.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.082 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:13.086 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.535293+0000", 2020-08-07T15:07:13.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:13.086 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:13.087 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.535293+0000", 2020-08-07T15:07:13.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1352e-05 2020-08-07T15:07:13.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:13.088 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.535334+0000", 2020-08-07T15:07:13.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6660000000000002e-06 2020-08-07T15:07:13.089 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.089 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.089 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:13.089 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.535341+0000", 2020-08-07T15:07:13.089 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19233371499999999 2020-08-07T15:07:13.090 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.090 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.091 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:13.091 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.727675+0000", 2020-08-07T15:07:13.091 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1819999999999997e-06 2020-08-07T15:07:13.091 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.092 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.092 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:13.092 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.727680+0000", 2020-08-07T15:07:13.092 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0288999999999999e-05 2020-08-07T15:07:13.092 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.092 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.093 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:13.093 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.727730+0000", 2020-08-07T15:07:13.093 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0340000000000001e-05 2020-08-07T15:07:13.093 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.093 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.094 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:13.094 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.727760+0000", 2020-08-07T15:07:13.094 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015059489999999999 2020-08-07T15:07:13.094 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.094 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.095 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:13.095 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.729266+0000", 2020-08-07T15:07:13.095 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2305999999999997e-05 2020-08-07T15:07:13.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.095 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.096 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:13.096 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.729319+0000", 2020-08-07T15:07:13.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.194025799 2020-08-07T15:07:13.096 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.096 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:13.097 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.097 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.097 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.097 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:16223 3.9 e27/20 3:9c72cb85:::benchmark_data_smithi074_35447_object2027:head v 27'887, mlcod=27'881)", 2020-08-07T15:07:13.097 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.111666+0000", 2020-08-07T15:07:13.098 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 25.917356349999999, 2020-08-07T15:07:13.098 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.48138790999999997, 2020-08-07T15:07:13.098 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:13.098 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:13.098 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:13.099 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.099 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:13.099 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111666+0000", 2020-08-07T15:07:13.100 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:13.101 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.101 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.101 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:13.102 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111666+0000", 2020-08-07T15:07:13.102 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3168999999999999e-05 2020-08-07T15:07:13.102 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.103 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.103 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:13.103 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111720+0000", 2020-08-07T15:07:13.104 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6309999999999997e-06 2020-08-07T15:07:13.104 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.104 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.105 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:13.105 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111726+0000", 2020-08-07T15:07:13.105 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.47995092 2020-08-07T15:07:13.106 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.106 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.106 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:13.107 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591677+0000", 2020-08-07T15:07:13.107 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9879999999999998e-06 2020-08-07T15:07:13.107 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.108 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.108 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:13.108 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591684+0000", 2020-08-07T15:07:13.109 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5562000000000002e-05 2020-08-07T15:07:13.109 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.109 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.110 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:13.110 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591730+0000", 2020-08-07T15:07:13.119 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4731000000000001e-05 2020-08-07T15:07:13.119 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.120 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.120 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:13.120 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591754+0000", 2020-08-07T15:07:13.121 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012412739999999999 2020-08-07T15:07:13.121 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.121 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.122 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:13.122 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.592996+0000", 2020-08-07T15:07:13.122 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8634999999999997e-05 2020-08-07T15:07:13.123 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.123 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.123 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:13.124 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.593054+0000", 2020-08-07T15:07:13.124 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.48138790999999997 2020-08-07T15:07:13.124 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.125 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:13.125 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.125 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.126 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.126 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:16224 3.9 e27/20 3:9c72cb85:::benchmark_data_smithi074_35447_object2027:head v 27'888, mlcod=27'881)", 2020-08-07T15:07:13.126 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.114073+0000", 2020-08-07T15:07:13.127 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 25.914949622999998, 2020-08-07T15:07:13.127 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.50478892600000003, 2020-08-07T15:07:13.127 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:13.128 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:13.128 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:13.128 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.128 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:13.129 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114073+0000", 2020-08-07T15:07:13.129 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:13.130 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.130 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.130 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:13.131 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114073+0000", 2020-08-07T15:07:13.131 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3088999999999999e-05 2020-08-07T15:07:13.131 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.131 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.131 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:13.132 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114126+0000", 2020-08-07T15:07:13.132 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1629999999999999e-06 2020-08-07T15:07:13.132 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.132 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.132 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:13.133 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114133+0000", 2020-08-07T15:07:13.133 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.50357425199999994 2020-08-07T15:07:13.133 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.133 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.133 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:13.134 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617708+0000", 2020-08-07T15:07:13.134 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9160000000000004e-06 2020-08-07T15:07:13.134 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.134 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.134 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:13.135 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617715+0000", 2020-08-07T15:07:13.135 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8770999999999998e-05 2020-08-07T15:07:13.135 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.135 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.135 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:13.136 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617763+0000", 2020-08-07T15:07:13.136 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.544e-05 2020-08-07T15:07:13.136 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.136 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.136 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:13.136 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617789+0000", 2020-08-07T15:07:13.137 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010210359999999999 2020-08-07T15:07:13.137 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.137 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.137 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:13.137 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.618810+0000", 2020-08-07T15:07:13.138 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2259000000000002e-05 2020-08-07T15:07:13.138 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.138 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.138 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:13.138 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.618862+0000", 2020-08-07T15:07:13.139 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.50478892600000003 2020-08-07T15:07:13.139 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.139 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:13.139 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.140 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.140 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.141 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:16440 3.4 3:21d0104c:::benchmark_data_smithi074_35447_object2054:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:13.141 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.248910+0000", 2020-08-07T15:07:13.141 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 25.780113092000001, 2020-08-07T15:07:13.141 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.145337039, 2020-08-07T15:07:13.142 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:13.142 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:13.142 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:13.142 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:13.143 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:13.143 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 16440 2020-08-07T15:07:13.143 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.143 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:13.143 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.144 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:13.144 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.248910+0000", 2020-08-07T15:07:13.144 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:13.144 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.144 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.145 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:13.145 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.248910+0000", 2020-08-07T15:07:13.145 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1525e-05 2020-08-07T15:07:13.145 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.146 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.146 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:13.147 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.248951+0000", 2020-08-07T15:07:13.147 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0839999999999997e-06 2020-08-07T15:07:13.147 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.148 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.148 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:13.148 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.248957+0000", 2020-08-07T15:07:13.148 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.4610000000000008e-06 2020-08-07T15:07:13.149 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.149 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.149 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:13.149 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.248966+0000", 2020-08-07T15:07:13.150 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1290000000000004e-06 2020-08-07T15:07:13.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.151 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:13.151 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.248971+0000", 2020-08-07T15:07:13.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019651790000000001 2020-08-07T15:07:13.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:13.152 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.250936+0000", 2020-08-07T15:07:13.153 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5055000000000006e-05 2020-08-07T15:07:13.153 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.153 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.153 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:13.153 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.251011+0000", 2020-08-07T15:07:13.154 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000120529 2020-08-07T15:07:13.154 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.154 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.154 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:07:13.154 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.251132+0000", 2020-08-07T15:07:13.155 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001578404 2020-08-07T15:07:13.155 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.155 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.155 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:13.155 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.252710+0000", 2020-08-07T15:07:13.156 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3300000000000002e-07 2020-08-07T15:07:13.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.156 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:13.157 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.252711+0000", 2020-08-07T15:07:13.157 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8016000000000002e-05 2020-08-07T15:07:13.157 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.157 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.157 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:13.158 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.252769+0000", 2020-08-07T15:07:13.158 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028140320000000002 2020-08-07T15:07:13.158 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.158 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.158 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:13.159 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.255583+0000", 2020-08-07T15:07:13.159 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13859926 2020-08-07T15:07:13.159 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.159 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.159 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:13.160 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.394182+0000", 2020-08-07T15:07:13.160 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5209999999999999e-06 2020-08-07T15:07:13.160 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.160 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.160 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:13.160 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.394184+0000", 2020-08-07T15:07:13.161 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3211000000000002e-05 2020-08-07T15:07:13.161 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.161 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.162 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:13.162 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.394247+0000", 2020-08-07T15:07:13.162 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.145337039 2020-08-07T15:07:13.163 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.163 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:13.163 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.164 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:17184 3.4 3:2060c75c:::benchmark_data_smithi074_35447_object2147:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:13.164 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.749909+0000", 2020-08-07T15:07:13.165 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 25.279113651999999, 2020-08-07T15:07:13.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.116047953, 2020-08-07T15:07:13.165 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:13.165 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:13.166 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:13.166 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:13.166 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:13.167 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 17184 2020-08-07T15:07:13.167 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.167 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:13.168 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.168 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:13.168 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.749909+0000", 2020-08-07T15:07:13.168 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:13.169 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.169 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.169 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:13.169 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.749909+0000", 2020-08-07T15:07:13.169 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1341999999999997e-05 2020-08-07T15:07:13.170 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.170 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.170 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:13.170 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.749961+0000", 2020-08-07T15:07:13.170 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5790000000000003e-06 2020-08-07T15:07:13.171 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.171 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.171 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:13.171 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.749966+0000", 2020-08-07T15:07:13.171 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1814000000000001e-05 2020-08-07T15:07:13.172 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.172 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.172 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:13.172 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.749978+0000", 2020-08-07T15:07:13.173 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3499999999999996e-06 2020-08-07T15:07:13.173 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.173 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.173 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:13.173 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.749983+0000", 2020-08-07T15:07:13.174 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044351529999999998 2020-08-07T15:07:13.174 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.174 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.174 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:13.174 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.754418+0000", 2020-08-07T15:07:13.175 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5979000000000001e-05 2020-08-07T15:07:13.175 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.175 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.175 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:13.175 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.754474+0000", 2020-08-07T15:07:13.176 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0001349 2020-08-07T15:07:13.176 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.176 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.177 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:07:13.177 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.754609+0000", 2020-08-07T15:07:13.177 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028583409999999999 2020-08-07T15:07:13.177 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.177 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.178 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:13.178 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.757468+0000", 2020-08-07T15:07:13.178 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001398921 2020-08-07T15:07:13.178 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.178 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.178 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:13.179 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.758867+0000", 2020-08-07T15:07:13.179 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.10696436500000001 2020-08-07T15:07:13.179 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.179 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.179 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:13.180 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.865831+0000", 2020-08-07T15:07:13.180 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0189999999999999e-06 2020-08-07T15:07:13.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.180 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.181 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:13.181 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.865832+0000", 2020-08-07T15:07:13.181 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6778000000000002e-05 2020-08-07T15:07:13.181 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:13.182 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.865909+0000", 2020-08-07T15:07:13.183 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8412000000000003e-05 2020-08-07T15:07:13.183 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.183 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.183 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:13.183 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.865957+0000", 2020-08-07T15:07:13.184 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.116047953 2020-08-07T15:07:13.184 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.184 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:13.184 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.184 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.185 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.185 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:17856 3.5 e27/20 3:a97d5eb0:::benchmark_data_smithi074_35447_object2231:head v 27'1048, mlcod=27'1043)", 2020-08-07T15:07:13.185 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:48.166903+0000", 2020-08-07T15:07:13.185 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 24.862119984, 2020-08-07T15:07:13.185 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.121028751, 2020-08-07T15:07:13.186 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:13.186 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:13.186 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:13.186 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.186 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:13.187 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.166903+0000", 2020-08-07T15:07:13.187 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:13.187 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.187 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.188 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:13.188 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.166903+0000", 2020-08-07T15:07:13.188 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8375999999999999e-05 2020-08-07T15:07:13.188 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.189 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.189 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:13.189 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.166951+0000", 2020-08-07T15:07:13.189 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.083e-06 2020-08-07T15:07:13.189 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.190 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.190 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:13.190 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.166957+0000", 2020-08-07T15:07:13.191 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.120316715 2020-08-07T15:07:13.191 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.191 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.191 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:13.192 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.287274+0000", 2020-08-07T15:07:13.192 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2049999999999998e-06 2020-08-07T15:07:13.192 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.193 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.193 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:13.193 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.287277+0000", 2020-08-07T15:07:13.193 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0238999999999999e-05 2020-08-07T15:07:13.194 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.194 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.194 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:13.194 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.287317+0000", 2020-08-07T15:07:13.195 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1844e-05 2020-08-07T15:07:13.195 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.195 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.195 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:13.195 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.287339+0000", 2020-08-07T15:07:13.196 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00054682099999999996 2020-08-07T15:07:13.196 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.196 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.196 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:13.196 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.287886+0000", 2020-08-07T15:07:13.197 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5467999999999999e-05 2020-08-07T15:07:13.197 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.197 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.197 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:13.197 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.287932+0000", 2020-08-07T15:07:13.198 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.121028751 2020-08-07T15:07:13.198 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.198 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:13.198 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.199 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.199 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.199 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:24992 3.0 e28/20 3:008a8fe9:::benchmark_data_smithi074_35447_object3123:head v 28'1704, mlcod=28'1694)", 2020-08-07T15:07:13.199 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.852063+0000", 2020-08-07T15:07:13.199 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 20.176959488000001, 2020-08-07T15:07:13.200 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20540171199999999, 2020-08-07T15:07:13.200 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:13.200 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:13.200 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:13.200 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.201 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:13.201 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.852063+0000", 2020-08-07T15:07:13.201 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:13.201 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.201 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.201 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:13.202 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.852063+0000", 2020-08-07T15:07:13.202 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1409999999999997e-05 2020-08-07T15:07:13.202 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.202 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.202 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:13.203 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.852115+0000", 2020-08-07T15:07:13.203 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6100000000000002e-06 2020-08-07T15:07:13.203 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.203 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.203 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:13.204 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.852121+0000", 2020-08-07T15:07:13.204 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20455925999999999 2020-08-07T15:07:13.204 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.205 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.205 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:13.205 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.056681+0000", 2020-08-07T15:07:13.205 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8739999999999995e-06 2020-08-07T15:07:13.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.206 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.207 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:13.207 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.056689+0000", 2020-08-07T15:07:13.207 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2753999999999998e-05 2020-08-07T15:07:13.207 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.208 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.208 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:13.208 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.056731+0000", 2020-08-07T15:07:13.208 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9171e-05 2020-08-07T15:07:13.208 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.209 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.209 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:13.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.056750+0000", 2020-08-07T15:07:13.209 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00065542300000000003 2020-08-07T15:07:13.209 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.210 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.210 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:13.210 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.057406+0000", 2020-08-07T15:07:13.210 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9209999999999997e-05 2020-08-07T15:07:13.210 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.211 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.211 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:13.211 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.057465+0000", 2020-08-07T15:07:13.211 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20540171199999999 2020-08-07T15:07:13.211 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.212 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:13.212 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.212 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.212 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.212 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:27424 3.4 3:279cfe9c:::benchmark_data_smithi074_35447_object3427:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:13.213 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:54.338304+0000", 2020-08-07T15:07:13.213 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 18.690718660000002, 2020-08-07T15:07:13.213 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.12276002699999999, 2020-08-07T15:07:13.213 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:13.213 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:13.214 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:13.214 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:13.214 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:13.214 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 27424 2020-08-07T15:07:13.215 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.215 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:13.215 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.216 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:13.216 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.338304+0000", 2020-08-07T15:07:13.216 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:13.217 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.217 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.217 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:13.218 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.338304+0000", 2020-08-07T15:07:13.218 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9563e-05 2020-08-07T15:07:13.218 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.218 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.219 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:13.219 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.338354+0000", 2020-08-07T15:07:13.219 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6999999999999996e-06 2020-08-07T15:07:13.220 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.220 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.220 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:13.221 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.338359+0000", 2020-08-07T15:07:13.221 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1476e-05 2020-08-07T15:07:13.221 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.222 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.222 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:13.222 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.338371+0000", 2020-08-07T15:07:13.223 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7350000000000001e-06 2020-08-07T15:07:13.223 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.223 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.224 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:13.224 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.338377+0000", 2020-08-07T15:07:13.224 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0016087460000000001 2020-08-07T15:07:13.225 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.225 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.225 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:13.225 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.339985+0000", 2020-08-07T15:07:13.226 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0984999999999997e-05 2020-08-07T15:07:13.226 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.226 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.227 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:13.227 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.340076+0000", 2020-08-07T15:07:13.227 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017277500000000001 2020-08-07T15:07:13.228 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.228 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.228 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:07:13.229 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.340249+0000", 2020-08-07T15:07:13.229 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0025273549999999998 2020-08-07T15:07:13.229 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.230 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.230 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:13.230 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.342777+0000", 2020-08-07T15:07:13.231 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.187e-06 2020-08-07T15:07:13.231 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.231 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.232 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:13.232 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.342778+0000", 2020-08-07T15:07:13.232 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3108999999999995e-05 2020-08-07T15:07:13.233 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.233 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.234 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:13.234 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.342841+0000", 2020-08-07T15:07:13.234 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9672999999999994e-05 2020-08-07T15:07:13.235 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.235 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.235 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:13.236 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.342920+0000", 2020-08-07T15:07:13.236 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.1180755 2020-08-07T15:07:13.236 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.237 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.238 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:13.238 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.460996+0000", 2020-08-07T15:07:13.238 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.569999999999999e-07 2020-08-07T15:07:13.239 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.239 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.239 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:13.240 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.460997+0000", 2020-08-07T15:07:13.240 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7266000000000003e-05 2020-08-07T15:07:13.240 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.241 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.241 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:13.242 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.461064+0000", 2020-08-07T15:07:13.242 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.12276002699999999 2020-08-07T15:07:13.242 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.243 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:13.243 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.244 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.244 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.244 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:27912 3.4 3:2bcd4c15:::benchmark_data_smithi074_35447_object3488:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:13.245 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:54.598926+0000", 2020-08-07T15:07:13.245 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 18.430096692999999, 2020-08-07T15:07:13.245 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13448369399999999, 2020-08-07T15:07:13.246 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:13.246 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:13.246 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:13.247 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:13.247 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:13.247 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 27912 2020-08-07T15:07:13.248 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.248 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:13.248 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.249 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:13.249 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.598926+0000", 2020-08-07T15:07:13.249 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:13.250 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.250 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.250 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:13.251 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.598926+0000", 2020-08-07T15:07:13.251 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5176000000000003e-05 2020-08-07T15:07:13.251 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.252 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.252 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:13.252 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.598971+0000", 2020-08-07T15:07:13.253 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1919999999999998e-06 2020-08-07T15:07:13.254 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.254 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.254 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:13.254 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.598976+0000", 2020-08-07T15:07:13.255 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.105e-06 2020-08-07T15:07:13.255 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.255 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.255 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:13.255 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.598982+0000", 2020-08-07T15:07:13.256 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7650000000000002e-06 2020-08-07T15:07:13.256 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.256 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.256 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:13.256 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.598984+0000", 2020-08-07T15:07:13.256 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003429353 2020-08-07T15:07:13.257 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.257 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.257 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:13.257 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.602414+0000", 2020-08-07T15:07:13.257 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000102043 2020-08-07T15:07:13.258 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.258 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.258 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:13.258 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.602516+0000", 2020-08-07T15:07:13.258 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000175262 2020-08-07T15:07:13.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.259 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.259 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:07:13.259 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.602691+0000", 2020-08-07T15:07:13.259 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001460565 2020-08-07T15:07:13.260 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.260 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.260 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:13.260 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.604152+0000", 2020-08-07T15:07:13.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3499999999999996e-07 2020-08-07T15:07:13.261 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.261 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.261 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:13.261 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.604152+0000", 2020-08-07T15:07:13.261 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0321000000000001e-05 2020-08-07T15:07:13.262 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.262 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.262 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:13.262 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.604212+0000", 2020-08-07T15:07:13.262 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.026292044000000001 2020-08-07T15:07:13.263 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.263 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.263 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:13.263 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.630505+0000", 2020-08-07T15:07:13.264 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.102838783 2020-08-07T15:07:13.264 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.264 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.264 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:13.264 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.733343+0000", 2020-08-07T15:07:13.265 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.372e-06 2020-08-07T15:07:13.265 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.265 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.265 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:13.265 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.733345+0000", 2020-08-07T15:07:13.266 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5178000000000005e-05 2020-08-07T15:07:13.266 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.266 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.266 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:13.266 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.733410+0000", 2020-08-07T15:07:13.267 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13448369399999999 2020-08-07T15:07:13.267 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.267 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:13.267 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.267 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.268 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.268 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:34000 3.4 3:22a8b4e3:::benchmark_data_smithi074_35447_object4249:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-08-07T15:07:13.268 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:58.316850+0000", 2020-08-07T15:07:13.268 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 14.712172782, 2020-08-07T15:07:13.268 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19523959799999999, 2020-08-07T15:07:13.268 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:13.269 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:13.269 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:13.269 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:13.269 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:13.269 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 34000 2020-08-07T15:07:13.270 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.270 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:13.270 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.270 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:13.270 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316850+0000", 2020-08-07T15:07:13.271 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:13.271 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.271 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:13.271 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316850+0000", 2020-08-07T15:07:13.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7274999999999997e-05 2020-08-07T15:07:13.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:13.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316897+0000", 2020-08-07T15:07:13.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2249999999999999e-06 2020-08-07T15:07:13.275 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.276 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.277 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:13.277 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316903+0000", 2020-08-07T15:07:13.277 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0665e-05 2020-08-07T15:07:13.278 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.278 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.278 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:13.278 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316913+0000", 2020-08-07T15:07:13.278 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3089999999999997e-06 2020-08-07T15:07:13.279 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.279 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.279 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:13.279 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316918+0000", 2020-08-07T15:07:13.279 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041515279999999998 2020-08-07T15:07:13.280 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.280 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.280 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:13.280 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.321069+0000", 2020-08-07T15:07:13.280 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6084999999999999e-05 2020-08-07T15:07:13.281 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.281 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.281 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:13.281 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.321165+0000", 2020-08-07T15:07:13.281 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000185076 2020-08-07T15:07:13.282 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.282 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.282 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:07:13.282 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.321350+0000", 2020-08-07T15:07:13.282 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00079810799999999996 2020-08-07T15:07:13.282 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.283 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.283 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:13.283 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.322148+0000", 2020-08-07T15:07:13.283 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9299999999999997e-07 2020-08-07T15:07:13.283 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.284 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.284 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:13.284 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.322149+0000", 2020-08-07T15:07:13.284 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1654000000000003e-05 2020-08-07T15:07:13.284 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.286 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.286 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:13.286 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.322211+0000", 2020-08-07T15:07:13.287 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044003000000000002 2020-08-07T15:07:13.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.287 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:13.288 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.326611+0000", 2020-08-07T15:07:13.288 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18541316599999999 2020-08-07T15:07:13.288 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.288 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.288 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:13.289 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.512024+0000", 2020-08-07T15:07:13.289 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5290000000000001e-06 2020-08-07T15:07:13.289 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.289 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.289 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:13.290 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.512026+0000", 2020-08-07T15:07:13.290 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3985000000000005e-05 2020-08-07T15:07:13.290 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.290 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.290 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:13.291 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.512090+0000", 2020-08-07T15:07:13.291 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19523959799999999 2020-08-07T15:07:13.291 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.291 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:13.291 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.292 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.292 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.292 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:34000 3.4 e30/20 ondisk, result = 0)", 2020-08-07T15:07:13.292 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:58.324486+0000", 2020-08-07T15:07:13.292 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 14.704536854000001, 2020-08-07T15:07:13.293 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.187608989, 2020-08-07T15:07:13.293 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:13.293 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:13.293 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:13.293 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.294 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:13.294 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.324486+0000", 2020-08-07T15:07:13.294 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:13.294 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.294 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.295 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:13.295 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.324486+0000", 2020-08-07T15:07:13.295 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5866999999999999e-05 2020-08-07T15:07:13.295 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.295 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.296 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:13.296 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.324502+0000", 2020-08-07T15:07:13.296 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4119999999999999e-06 2020-08-07T15:07:13.296 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.296 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.296 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:13.297 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.324504+0000", 2020-08-07T15:07:13.297 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18677271500000001 2020-08-07T15:07:13.298 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.298 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.298 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:13.298 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.511277+0000", 2020-08-07T15:07:13.298 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5489999999999997e-06 2020-08-07T15:07:13.299 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.299 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.299 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:13.299 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.511283+0000", 2020-08-07T15:07:13.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00072544799999999996 2020-08-07T15:07:13.300 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.300 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.300 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:13.300 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.512008+0000", 2020-08-07T15:07:13.300 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7689999999999996e-06 2020-08-07T15:07:13.301 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.301 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.301 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:13.301 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.512016+0000", 2020-08-07T15:07:13.301 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9228999999999996e-05 2020-08-07T15:07:13.302 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.302 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.302 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:13.302 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.512095+0000", 2020-08-07T15:07:13.302 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.187608989 2020-08-07T15:07:13.303 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.303 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:13.303 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.303 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.303 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.303 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:34024 3.0 e30/20 3:0de6648d:::benchmark_data_smithi074_35447_object4252:head v 30'2224, mlcod=30'2219)", 2020-08-07T15:07:13.304 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:58.351656+0000", 2020-08-07T15:07:13.304 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 14.677366461, 2020-08-07T15:07:13.304 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.128381149, 2020-08-07T15:07:13.304 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:13.304 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:13.305 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:13.305 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.305 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:13.305 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.351656+0000", 2020-08-07T15:07:13.305 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:13.306 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.306 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.306 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:13.306 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.351656+0000", 2020-08-07T15:07:13.306 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.033e-05 2020-08-07T15:07:13.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.307 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.307 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:13.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.351717+0000", 2020-08-07T15:07:13.308 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5980000000000002e-06 2020-08-07T15:07:13.308 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.308 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.309 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:13.309 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.351723+0000", 2020-08-07T15:07:13.309 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.125838113 2020-08-07T15:07:13.309 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.309 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.310 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:13.310 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.477561+0000", 2020-08-07T15:07:13.310 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7879999999999995e-06 2020-08-07T15:07:13.310 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.310 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.311 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:13.311 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.477570+0000", 2020-08-07T15:07:13.311 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8072000000000003e-05 2020-08-07T15:07:13.311 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.311 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.311 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:13.312 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.477618+0000", 2020-08-07T15:07:13.312 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9921999999999999e-05 2020-08-07T15:07:13.312 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.312 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.312 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:13.313 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.477648+0000", 2020-08-07T15:07:13.313 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023362069999999999 2020-08-07T15:07:13.313 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.313 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.313 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:13.323 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.479984+0000", 2020-08-07T15:07:13.324 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3118999999999997e-05 2020-08-07T15:07:13.324 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.324 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.324 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:13.324 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.480038+0000", 2020-08-07T15:07:13.325 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.128381149 2020-08-07T15:07:13.325 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.325 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:13.325 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.325 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.326 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.326 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:34231 3.4 3:290ffd32:::benchmark_data_smithi074_35447_object4278:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-08-07T15:07:13.326 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:58.507180+0000", 2020-08-07T15:07:13.326 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 14.521842893000001, 2020-08-07T15:07:13.326 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.114670624, 2020-08-07T15:07:13.327 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:13.327 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:13.327 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:13.327 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:13.327 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:13.328 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 34231 2020-08-07T15:07:13.328 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.328 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:13.328 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.329 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:13.329 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.507180+0000", 2020-08-07T15:07:13.329 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:13.329 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.329 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.330 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:13.330 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.507180+0000", 2020-08-07T15:07:13.330 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7768000000000001e-05 2020-08-07T15:07:13.330 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.330 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.331 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:13.331 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.507228+0000", 2020-08-07T15:07:13.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3020000000000002e-06 2020-08-07T15:07:13.331 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.331 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.332 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:13.332 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.507233+0000", 2020-08-07T15:07:13.332 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0558e-05 2020-08-07T15:07:13.332 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.332 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.332 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:13.333 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.507244+0000", 2020-08-07T15:07:13.333 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9890000000000003e-06 2020-08-07T15:07:13.334 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.334 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.334 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:13.334 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.507248+0000", 2020-08-07T15:07:13.334 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041391140000000002 2020-08-07T15:07:13.335 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.335 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:13.335 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.511387+0000", 2020-08-07T15:07:13.335 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5251000000000001e-05 2020-08-07T15:07:13.335 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.336 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.336 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:13.336 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.511472+0000", 2020-08-07T15:07:13.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016629200000000001 2020-08-07T15:07:13.336 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.337 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.337 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:07:13.337 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.511638+0000", 2020-08-07T15:07:13.337 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0016648800000000001 2020-08-07T15:07:13.337 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.338 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.338 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:13.338 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.513303+0000", 2020-08-07T15:07:13.338 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.39e-07 2020-08-07T15:07:13.338 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.339 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.339 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:13.339 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.513304+0000", 2020-08-07T15:07:13.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2635e-05 2020-08-07T15:07:13.339 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.340 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:13.340 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.513356+0000", 2020-08-07T15:07:13.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0021098670000000001 2020-08-07T15:07:13.340 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.341 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:13.341 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.515466+0000", 2020-08-07T15:07:13.341 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.10631813 2020-08-07T15:07:13.341 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.342 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:13.342 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.621784+0000", 2020-08-07T15:07:13.342 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7899999999999997e-07 2020-08-07T15:07:13.342 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.342 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.343 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:13.343 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.621785+0000", 2020-08-07T15:07:13.343 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5519999999999996e-05 2020-08-07T15:07:13.343 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.344 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.344 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:13.344 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.621851+0000", 2020-08-07T15:07:13.344 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.114670624 2020-08-07T15:07:13.345 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.345 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:13.345 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.345 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.345 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.346 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:34624 3.9 e30/20 3:946cdb13:::benchmark_data_smithi074_35447_object4327:head v 30'2016, mlcod=30'2009)", 2020-08-07T15:07:13.346 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:58.746375+0000", 2020-08-07T15:07:13.346 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 14.282647979, 2020-08-07T15:07:13.346 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28564679799999998, 2020-08-07T15:07:13.346 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:13.347 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:13.347 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:13.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.347 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:13.347 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746375+0000", 2020-08-07T15:07:13.348 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:13.348 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.348 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.348 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:13.348 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746375+0000", 2020-08-07T15:07:13.349 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3674000000000001e-05 2020-08-07T15:07:13.349 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.349 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.350 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:13.350 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746429+0000", 2020-08-07T15:07:13.350 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9349999999999997e-06 2020-08-07T15:07:13.350 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.350 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.351 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:13.351 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746435+0000", 2020-08-07T15:07:13.351 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28165817700000001 2020-08-07T15:07:13.351 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.351 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.352 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:13.352 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028094+0000", 2020-08-07T15:07:13.352 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.512e-06 2020-08-07T15:07:13.352 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.352 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.352 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:13.353 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028097+0000", 2020-08-07T15:07:13.353 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7904999999999997e-05 2020-08-07T15:07:13.353 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.353 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.354 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:13.354 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028135+0000", 2020-08-07T15:07:13.355 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6243e-05 2020-08-07T15:07:13.355 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.355 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.355 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:13.355 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028151+0000", 2020-08-07T15:07:13.356 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003818372 2020-08-07T15:07:13.356 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.356 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.356 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:13.356 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.031970+0000", 2020-08-07T15:07:13.357 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.198e-05 2020-08-07T15:07:13.357 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.357 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.357 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:13.357 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.032022+0000", 2020-08-07T15:07:13.358 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28564679799999998 2020-08-07T15:07:13.358 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.358 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:13.358 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.358 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.359 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.359 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:35880 3.9 e31/20 3:905cfca2:::benchmark_data_smithi074_35447_object4484:head v 31'2128, mlcod=31'2120)", 2020-08-07T15:07:13.359 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:59.454559+0000", 2020-08-07T15:07:13.359 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 13.574464168, 2020-08-07T15:07:13.359 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17558973899999999, 2020-08-07T15:07:13.360 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:13.360 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:13.360 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:13.360 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.360 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:13.360 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.454559+0000", 2020-08-07T15:07:13.361 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:13.361 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.361 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.361 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:13.361 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.454559+0000", 2020-08-07T15:07:13.362 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5383000000000002e-05 2020-08-07T15:07:13.362 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.362 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.362 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:13.362 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.454614+0000", 2020-08-07T15:07:13.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5780000000000004e-06 2020-08-07T15:07:13.363 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.363 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.363 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:13.363 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.454621+0000", 2020-08-07T15:07:13.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17497574099999999 2020-08-07T15:07:13.364 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:13.365 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.629596+0000", 2020-08-07T15:07:13.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.561e-06 2020-08-07T15:07:13.365 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.365 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.366 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:13.366 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.629601+0000", 2020-08-07T15:07:13.366 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8960000000000001e-05 2020-08-07T15:07:13.366 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.366 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:13.367 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.629630+0000", 2020-08-07T15:07:13.367 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7209000000000001e-05 2020-08-07T15:07:13.367 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.367 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.367 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:13.368 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.629647+0000", 2020-08-07T15:07:13.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00046137800000000003 2020-08-07T15:07:13.368 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.368 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.368 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:13.369 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.630108+0000", 2020-08-07T15:07:13.369 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9928999999999997e-05 2020-08-07T15:07:13.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.369 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:13.370 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.630148+0000", 2020-08-07T15:07:13.370 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17558973899999999 2020-08-07T15:07:13.370 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.370 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:13.370 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.370 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.371 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.371 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:48216 3.4 3:29f3c7f3:::benchmark_data_smithi074_35447_object6026:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e36)", 2020-08-07T15:07:13.371 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:08.937978+0000", 2020-08-07T15:07:13.372 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4.0910447469999998, 2020-08-07T15:07:13.372 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17762572800000001, 2020-08-07T15:07:13.372 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:13.372 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:13.373 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:13.373 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:13.373 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:13.374 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48216 2020-08-07T15:07:13.374 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.374 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:13.374 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.374 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:13.374 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.937978+0000", 2020-08-07T15:07:13.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:13.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:13.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.937978+0000", 2020-08-07T15:07:13.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9499999999999997e-05 2020-08-07T15:07:13.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:13.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.938028+0000", 2020-08-07T15:07:13.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3929999999999999e-06 2020-08-07T15:07:13.379 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:13.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.938033+0000", 2020-08-07T15:07:13.380 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0659000000000001e-05 2020-08-07T15:07:13.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:13.380 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.938044+0000", 2020-08-07T15:07:13.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9569999999999999e-06 2020-08-07T15:07:13.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:13.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.938050+0000", 2020-08-07T15:07:13.382 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00063126600000000001 2020-08-07T15:07:13.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.382 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.382 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:13.382 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.938681+0000", 2020-08-07T15:07:13.383 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5762999999999998e-05 2020-08-07T15:07:13.383 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.383 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.383 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:13.383 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.938777+0000", 2020-08-07T15:07:13.383 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015617199999999999 2020-08-07T15:07:13.384 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.384 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.384 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:07:13.384 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.938933+0000", 2020-08-07T15:07:13.384 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0018355559999999999 2020-08-07T15:07:13.385 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.385 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.385 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:13.385 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.940768+0000", 2020-08-07T15:07:13.385 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5700000000000001e-07 2020-08-07T15:07:13.386 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.386 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.386 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:13.387 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.940769+0000", 2020-08-07T15:07:13.387 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0513000000000002e-05 2020-08-07T15:07:13.387 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.387 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.387 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:13.388 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.940830+0000", 2020-08-07T15:07:13.388 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0029664779999999998 2020-08-07T15:07:13.388 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.388 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.388 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:13.389 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.943796+0000", 2020-08-07T15:07:13.389 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17176112800000001 2020-08-07T15:07:13.389 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.389 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.389 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:13.390 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.115557+0000", 2020-08-07T15:07:13.390 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0899999999999999e-06 2020-08-07T15:07:13.390 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.390 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.390 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:13.391 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.115558+0000", 2020-08-07T15:07:13.391 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5395999999999997e-05 2020-08-07T15:07:13.391 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.391 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.391 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:13.391 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.115604+0000", 2020-08-07T15:07:13.392 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17762572800000001 2020-08-07T15:07:13.392 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.392 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:13.392 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.392 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.393 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.393 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:53680 3.5 e37/20 3:a4c15fd7:::benchmark_data_smithi074_35447_object6709:head v 37'3120, mlcod=37'3111)", 2020-08-07T15:07:13.393 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:12.774724+0000", 2020-08-07T15:07:13.393 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.25429922799999999, 2020-08-07T15:07:13.393 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13222435299999999, 2020-08-07T15:07:13.394 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:13.394 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:13.394 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:13.394 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.394 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:13.394 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:12.774724+0000", 2020-08-07T15:07:13.395 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:13.395 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.395 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.395 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:13.396 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:12.774724+0000", 2020-08-07T15:07:13.396 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1218000000000003e-05 2020-08-07T15:07:13.396 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.396 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.397 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:13.397 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:12.774775+0000", 2020-08-07T15:07:13.397 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2790000000000004e-06 2020-08-07T15:07:13.398 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.398 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.398 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:13.398 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:12.774781+0000", 2020-08-07T15:07:13.398 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13128326600000001 2020-08-07T15:07:13.399 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.399 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:13.399 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:12.906064+0000", 2020-08-07T15:07:13.399 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:07:13.400 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.400 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:13.400 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:12.906069+0000", 2020-08-07T15:07:13.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9119000000000002e-05 2020-08-07T15:07:13.400 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.401 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.401 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:13.401 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:12.906118+0000", 2020-08-07T15:07:13.401 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4093000000000001e-05 2020-08-07T15:07:13.401 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.402 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.402 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:13.402 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:12.906143+0000", 2020-08-07T15:07:13.402 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00076309200000000005 2020-08-07T15:07:13.402 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.403 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.403 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:13.403 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:12.906906+0000", 2020-08-07T15:07:13.403 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2351999999999997e-05 2020-08-07T15:07:13.403 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.404 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.404 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:13.404 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:12.906948+0000", 2020-08-07T15:07:13.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13222435299999999 2020-08-07T15:07:13.404 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.404 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:13.405 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.405 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.405 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:13.405 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:13.406 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:13.031+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:13.406 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:07:13.409 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:07:13.409 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 37 16 54417 54401 11.4857 12.7188 0.00671873 0.0108751 2020-08-07T15:07:13.410 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:13.406+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:13.508 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:13.507+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:13.609 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:13.608+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:13.622 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:13.622 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:07:13.622 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.623 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:54727 3.7 3:ee999259:::benchmark_data_smithi074_35447_object6840:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e37)", 2020-08-07T15:07:13.623 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:13.448978+0000", 2020-08-07T15:07:13.623 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.172292538, 2020-08-07T15:07:13.623 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17232666799999999, 2020-08-07T15:07:13.623 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:13.624 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:13.624 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:13.624 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:13.624 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:13.624 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 54727 2020-08-07T15:07:13.624 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.625 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:13.625 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.625 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:13.625 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.448978+0000", 2020-08-07T15:07:13.625 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:13.626 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.626 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.626 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:13.626 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.448978+0000", 2020-08-07T15:07:13.626 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7193e-05 2020-08-07T15:07:13.627 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.627 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:13.627 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.449026+0000", 2020-08-07T15:07:13.628 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4360000000000001e-06 2020-08-07T15:07:13.628 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.628 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.628 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:13.628 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.449031+0000", 2020-08-07T15:07:13.629 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0132000000000001e-05 2020-08-07T15:07:13.629 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.629 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.629 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:13.629 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.449041+0000", 2020-08-07T15:07:13.630 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7700000000000001e-06 2020-08-07T15:07:13.630 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.630 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.630 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:13.630 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.449046+0000", 2020-08-07T15:07:13.631 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0016099560000000001 2020-08-07T15:07:13.631 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.631 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.631 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:13.631 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.450656+0000", 2020-08-07T15:07:13.631 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011117500000000001 2020-08-07T15:07:13.632 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.632 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.632 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:13.632 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.450767+0000", 2020-08-07T15:07:13.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019104500000000001 2020-08-07T15:07:13.633 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.633 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.633 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:07:13.633 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.450958+0000", 2020-08-07T15:07:13.633 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032997370000000001 2020-08-07T15:07:13.634 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.634 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.634 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:13.634 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.454258+0000", 2020-08-07T15:07:13.634 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9999999999999996e-07 2020-08-07T15:07:13.634 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.635 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.635 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:13.635 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.454259+0000", 2020-08-07T15:07:13.635 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2459000000000006e-05 2020-08-07T15:07:13.635 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.636 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.636 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:13.636 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.454321+0000", 2020-08-07T15:07:13.636 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002932175 2020-08-07T15:07:13.636 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.637 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.637 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:13.637 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.457253+0000", 2020-08-07T15:07:13.637 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0082749780000000005 2020-08-07T15:07:13.637 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.637 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:13.638 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.638 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.638 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.638 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:54728 3.7 3:ee999259:::benchmark_data_smithi074_35447_object6840:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e37)", 2020-08-07T15:07:13.638 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:13.449079+0000", 2020-08-07T15:07:13.639 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.172191547, 2020-08-07T15:07:13.639 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17235039499999999, 2020-08-07T15:07:13.639 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:13.639 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:13.640 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:13.640 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:13.640 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:13.640 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 54728 2020-08-07T15:07:13.640 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.641 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:13.641 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.641 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:13.641 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.449079+0000", 2020-08-07T15:07:13.642 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:13.642 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.642 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.642 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:13.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.449079+0000", 2020-08-07T15:07:13.643 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9458000000000001e-05 2020-08-07T15:07:13.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.643 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.643 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:13.643 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.449119+0000", 2020-08-07T15:07:13.644 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8189999999999997e-06 2020-08-07T15:07:13.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.644 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.644 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:13.644 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.449125+0000", 2020-08-07T15:07:13.644 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4169999999999997e-06 2020-08-07T15:07:13.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:13.645 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.449131+0000", 2020-08-07T15:07:13.645 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6819999999999999e-06 2020-08-07T15:07:13.646 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.646 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.646 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:13.646 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.449134+0000", 2020-08-07T15:07:13.646 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002251542 2020-08-07T15:07:13.647 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.647 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.647 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:13.647 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.451385+0000", 2020-08-07T15:07:13.648 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010285600000000001 2020-08-07T15:07:13.648 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.648 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.648 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:13.648 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.451488+0000", 2020-08-07T15:07:13.649 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018447200000000001 2020-08-07T15:07:13.649 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.649 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.649 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:07:13.650 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.451673+0000", 2020-08-07T15:07:13.650 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026569620000000001 2020-08-07T15:07:13.650 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.650 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.650 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:13.651 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.454330+0000", 2020-08-07T15:07:13.651 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4000000000000001e-07 2020-08-07T15:07:13.651 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.651 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.651 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:13.652 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.454330+0000", 2020-08-07T15:07:13.652 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5572999999999998e-05 2020-08-07T15:07:13.652 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.652 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.652 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:13.653 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.454386+0000", 2020-08-07T15:07:13.653 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028997229999999999 2020-08-07T15:07:13.653 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.653 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.653 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:13.653 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.457285+0000", 2020-08-07T15:07:13.654 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0082061440000000003 2020-08-07T15:07:13.654 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.654 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:13.654 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.654 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.655 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.655 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:54864 3.8 3:1d7e7518:::benchmark_data_smithi074_35447_object6857:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e37)", 2020-08-07T15:07:13.655 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:13.544920+0000", 2020-08-07T15:07:13.655 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.076350982999999997, 2020-08-07T15:07:13.655 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.076626345999999998, 2020-08-07T15:07:13.656 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:13.656 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:13.656 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:13.656 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:13.656 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:13.656 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 54864 2020-08-07T15:07:13.657 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.657 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:13.657 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.657 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:13.657 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.544920+0000", 2020-08-07T15:07:13.658 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:13.658 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.658 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.658 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:13.658 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.544920+0000", 2020-08-07T15:07:13.659 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8347000000000003e-05 2020-08-07T15:07:13.659 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.659 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.659 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:13.659 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.544968+0000", 2020-08-07T15:07:13.659 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6099999999999997e-06 2020-08-07T15:07:13.660 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.660 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.660 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:13.660 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.544974+0000", 2020-08-07T15:07:13.660 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.092e-05 2020-08-07T15:07:13.661 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.661 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.661 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:13.661 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.544985+0000", 2020-08-07T15:07:13.661 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7600000000000002e-06 2020-08-07T15:07:13.662 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.662 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.662 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:13.662 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.544990+0000", 2020-08-07T15:07:13.662 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0029461399999999999 2020-08-07T15:07:13.663 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.663 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.663 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:13.663 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.547936+0000", 2020-08-07T15:07:13.663 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8350999999999994e-05 2020-08-07T15:07:13.664 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.664 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:13.664 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.548034+0000", 2020-08-07T15:07:13.664 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018204500000000001 2020-08-07T15:07:13.664 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.665 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.665 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:07:13.665 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.548216+0000", 2020-08-07T15:07:13.665 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00104597 2020-08-07T15:07:13.665 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.666 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.666 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:13.666 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.549262+0000", 2020-08-07T15:07:13.666 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.08e-07 2020-08-07T15:07:13.666 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.667 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:13.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.549262+0000", 2020-08-07T15:07:13.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3529000000000002e-05 2020-08-07T15:07:13.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:13.668 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.549316+0000", 2020-08-07T15:07:13.668 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0021348650000000001 2020-08-07T15:07:13.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:13.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:13.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:13.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.551451+0000", 2020-08-07T15:07:13.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0065309449999999998 2020-08-07T15:07:13.669 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.670 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:13.670 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.670 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:13.670 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:07:13.670 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 3 2020-08-07T15:07:13.670 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:13.672 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:07:13.713 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:13.712+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:13.811 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:13.810+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:13.894 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:13.894 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:07:13.894 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:07:13.895 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:07:13.895 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:13.907 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:07:13.914 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:13.912+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:14.015 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:14.013+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:14.116 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:14.115+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:14.136 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:14.136 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:07:14.136 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:07:14.137 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:07:14.138 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.139 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:2600 3.b 3:d6a1a0d3:::benchmark_data_smithi074_35447_object324:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:07:14.139 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:37.800755+0000", 2020-08-07T15:07:14.139 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 36.331157736000002, 2020-08-07T15:07:14.140 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.215895015, 2020-08-07T15:07:14.140 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:14.140 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:14.140 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:14.141 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:14.141 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:14.141 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 2600 2020-08-07T15:07:14.141 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.141 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:14.142 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.142 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:14.142 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.800755+0000", 2020-08-07T15:07:14.142 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:14.142 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.146 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.147 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:14.147 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.800755+0000", 2020-08-07T15:07:14.148 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7878999999999999e-05 2020-08-07T15:07:14.148 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.148 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.148 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:14.148 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.800782+0000", 2020-08-07T15:07:14.149 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0520000000000003e-06 2020-08-07T15:07:14.149 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.149 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.149 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:14.149 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.800787+0000", 2020-08-07T15:07:14.150 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0587999999999999e-05 2020-08-07T15:07:14.150 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.151 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:14.151 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.800797+0000", 2020-08-07T15:07:14.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4970000000000001e-06 2020-08-07T15:07:14.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:14.153 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.800803+0000", 2020-08-07T15:07:14.153 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0059548989999999996 2020-08-07T15:07:14.153 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.153 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.153 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:14.153 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.806757+0000", 2020-08-07T15:07:14.154 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000111375 2020-08-07T15:07:14.154 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.154 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.154 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:14.155 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.806869+0000", 2020-08-07T15:07:14.155 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000159345 2020-08-07T15:07:14.155 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.157 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:14.157 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.807028+0000", 2020-08-07T15:07:14.157 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002275329 2020-08-07T15:07:14.157 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.157 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.158 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:14.158 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.809304+0000", 2020-08-07T15:07:14.158 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2900000000000002e-07 2020-08-07T15:07:14.158 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.158 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.158 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:14.159 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.809304+0000", 2020-08-07T15:07:14.159 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8708000000000002e-05 2020-08-07T15:07:14.159 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.159 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.159 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:14.162 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.809353+0000", 2020-08-07T15:07:14.162 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.026379923 2020-08-07T15:07:14.162 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.163 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.163 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.163 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.835733+0000", 2020-08-07T15:07:14.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18084716200000001 2020-08-07T15:07:14.165 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.166 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.166 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.166 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.016580+0000", 2020-08-07T15:07:14.166 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9299999999999997e-07 2020-08-07T15:07:14.166 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.167 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.167 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:14.167 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.016581+0000", 2020-08-07T15:07:14.183 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8535999999999996e-05 2020-08-07T15:07:14.184 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.184 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.184 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:14.184 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.016650+0000", 2020-08-07T15:07:14.185 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.215895015 2020-08-07T15:07:14.185 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.185 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:14.185 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.185 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.186 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.186 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:7576 3.b 3:d854a547:::benchmark_data_smithi074_35447_object946:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:14.186 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.736458+0000", 2020-08-07T15:07:14.186 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 33.395454329000003, 2020-08-07T15:07:14.186 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19357406199999999, 2020-08-07T15:07:14.187 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:14.187 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:14.187 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:14.187 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:14.187 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:14.194 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7576 2020-08-07T15:07:14.194 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.194 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:14.195 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.195 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:14.195 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.736458+0000", 2020-08-07T15:07:14.195 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:14.195 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.196 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.196 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:14.196 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.736458+0000", 2020-08-07T15:07:14.196 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2197999999999997e-05 2020-08-07T15:07:14.196 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.197 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.197 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:14.197 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.736510+0000", 2020-08-07T15:07:14.197 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2579999999999995e-06 2020-08-07T15:07:14.197 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.197 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.198 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:14.198 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.736518+0000", 2020-08-07T15:07:14.198 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3842999999999999e-05 2020-08-07T15:07:14.198 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.198 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.199 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:14.199 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.736532+0000", 2020-08-07T15:07:14.199 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0379999999999996e-06 2020-08-07T15:07:14.199 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.199 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.200 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:14.200 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.736538+0000", 2020-08-07T15:07:14.200 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044985529999999998 2020-08-07T15:07:14.200 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.201 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.201 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:14.201 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.741037+0000", 2020-08-07T15:07:14.201 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6891000000000005e-05 2020-08-07T15:07:14.201 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.202 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.202 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:14.202 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.741134+0000", 2020-08-07T15:07:14.202 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000198555 2020-08-07T15:07:14.202 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.203 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.203 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:14.203 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.741332+0000", 2020-08-07T15:07:14.203 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028259190000000001 2020-08-07T15:07:14.203 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.203 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.204 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:14.204 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.744158+0000", 2020-08-07T15:07:14.204 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1100000000000001e-07 2020-08-07T15:07:14.204 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.204 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.205 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:14.205 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.744159+0000", 2020-08-07T15:07:14.205 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1880999999999998e-05 2020-08-07T15:07:14.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.205 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.206 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:14.206 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.744241+0000", 2020-08-07T15:07:14.206 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7608000000000002e-05 2020-08-07T15:07:14.206 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.206 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.206 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.207 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.744288+0000", 2020-08-07T15:07:14.207 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18566353199999999 2020-08-07T15:07:14.207 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.207 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.207 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.208 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.929952+0000", 2020-08-07T15:07:14.208 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.02e-06 2020-08-07T15:07:14.208 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.208 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.208 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:14.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.929953+0000", 2020-08-07T15:07:14.209 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9355000000000002e-05 2020-08-07T15:07:14.209 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.209 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.209 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:14.210 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.930032+0000", 2020-08-07T15:07:14.210 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19357406199999999 2020-08-07T15:07:14.210 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.210 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:14.210 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.210 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.211 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.211 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:18166 3.8 3:15dfcb22:::benchmark_data_smithi074_35447_object2270:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:14.211 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:48.342194+0000", 2020-08-07T15:07:14.211 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 25.789718283999999, 2020-08-07T15:07:14.211 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25725213099999999, 2020-08-07T15:07:14.212 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:14.212 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:14.212 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:14.212 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:14.212 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:14.213 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18166 2020-08-07T15:07:14.213 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.213 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:14.213 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.213 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:14.214 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.342194+0000", 2020-08-07T15:07:14.214 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:14.214 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.214 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.214 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:14.214 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.342194+0000", 2020-08-07T15:07:14.215 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6729e-05 2020-08-07T15:07:14.215 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.215 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.215 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:14.215 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.342231+0000", 2020-08-07T15:07:14.216 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7229999999999998e-06 2020-08-07T15:07:14.216 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.216 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.216 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:14.216 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.342237+0000", 2020-08-07T15:07:14.217 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0136000000000002e-05 2020-08-07T15:07:14.217 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.217 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.217 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:14.217 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.342258+0000", 2020-08-07T15:07:14.217 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9989999999999997e-06 2020-08-07T15:07:14.218 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.218 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.227 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:14.227 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.342264+0000", 2020-08-07T15:07:14.227 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012111499999999999 2020-08-07T15:07:14.227 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.228 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.228 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:14.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.342385+0000", 2020-08-07T15:07:14.228 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000103878 2020-08-07T15:07:14.228 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.228 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.229 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:14.229 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.342489+0000", 2020-08-07T15:07:14.229 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00022590199999999999 2020-08-07T15:07:14.229 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.229 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.230 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:07:14.230 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.342715+0000", 2020-08-07T15:07:14.230 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003627019 2020-08-07T15:07:14.230 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.231 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.231 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:14.231 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.346342+0000", 2020-08-07T15:07:14.231 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.158e-06 2020-08-07T15:07:14.231 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.231 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.232 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:14.232 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.346343+0000", 2020-08-07T15:07:14.232 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7310000000000004e-05 2020-08-07T15:07:14.232 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.232 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.233 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:14.233 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.346410+0000", 2020-08-07T15:07:14.233 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001407782 2020-08-07T15:07:14.233 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.233 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.234 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.234 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.347818+0000", 2020-08-07T15:07:14.234 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.251560536 2020-08-07T15:07:14.234 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.234 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.235 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.235 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.599378+0000", 2020-08-07T15:07:14.235 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2920000000000001e-06 2020-08-07T15:07:14.235 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.235 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.236 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:14.236 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.599380+0000", 2020-08-07T15:07:14.236 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6551999999999996e-05 2020-08-07T15:07:14.236 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.236 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.236 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:14.237 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.599446+0000", 2020-08-07T15:07:14.237 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25725213099999999 2020-08-07T15:07:14.237 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.237 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:14.237 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.238 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.238 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.238 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:18167 3.8 3:15dfcb22:::benchmark_data_smithi074_35447_object2270:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:14.238 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:48.342989+0000", 2020-08-07T15:07:14.238 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 25.788922833000001, 2020-08-07T15:07:14.239 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25656989899999999, 2020-08-07T15:07:14.239 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:14.239 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:14.239 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:14.239 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:14.240 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:14.240 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18167 2020-08-07T15:07:14.240 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.240 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:14.240 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.240 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:14.241 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.342989+0000", 2020-08-07T15:07:14.241 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:14.241 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.241 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.242 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:14.242 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.342989+0000", 2020-08-07T15:07:14.242 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7729999999999999e-05 2020-08-07T15:07:14.244 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.244 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.244 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:14.245 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.343037+0000", 2020-08-07T15:07:14.245 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9149999999999999e-06 2020-08-07T15:07:14.245 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.245 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.245 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:14.246 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.343044+0000", 2020-08-07T15:07:14.246 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9283999999999996e-05 2020-08-07T15:07:14.246 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.246 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.247 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:14.247 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.343113+0000", 2020-08-07T15:07:14.248 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3370000000000006e-06 2020-08-07T15:07:14.248 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.248 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.248 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:14.248 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.343123+0000", 2020-08-07T15:07:14.249 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0014698319999999999 2020-08-07T15:07:14.249 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.249 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.249 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:14.249 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.344593+0000", 2020-08-07T15:07:14.250 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000110857 2020-08-07T15:07:14.250 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.250 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.252 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:14.252 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.344703+0000", 2020-08-07T15:07:14.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017357700000000001 2020-08-07T15:07:14.253 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.253 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:07:14.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.344877+0000", 2020-08-07T15:07:14.253 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015451519999999999 2020-08-07T15:07:14.254 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.254 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.254 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:14.254 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.346422+0000", 2020-08-07T15:07:14.254 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5000000000000003e-07 2020-08-07T15:07:14.255 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.255 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.255 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:14.255 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.346423+0000", 2020-08-07T15:07:14.255 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0982e-05 2020-08-07T15:07:14.256 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.256 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.257 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:14.257 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.346484+0000", 2020-08-07T15:07:14.257 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0046474940000000003 2020-08-07T15:07:14.258 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.258 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.258 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.258 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.351131+0000", 2020-08-07T15:07:14.258 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24836395 2020-08-07T15:07:14.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.259 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.259 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.259 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.599495+0000", 2020-08-07T15:07:14.259 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7899999999999999e-07 2020-08-07T15:07:14.260 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.260 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.260 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:14.260 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.599496+0000", 2020-08-07T15:07:14.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3759999999999999e-05 2020-08-07T15:07:14.261 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.261 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.261 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:14.261 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.599559+0000", 2020-08-07T15:07:14.261 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25656989899999999 2020-08-07T15:07:14.262 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.262 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:14.262 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.262 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.262 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.262 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:18168 3.8 3:15dfcb22:::benchmark_data_smithi074_35447_object2270:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:14.263 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:48.343981+0000", 2020-08-07T15:07:14.263 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 25.787931566000001, 2020-08-07T15:07:14.263 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25567693400000002, 2020-08-07T15:07:14.263 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:14.263 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:14.264 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:14.264 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:14.264 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:14.264 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18168 2020-08-07T15:07:14.264 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.266 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:14.266 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.266 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:14.266 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.343981+0000", 2020-08-07T15:07:14.267 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:14.267 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.267 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.268 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:14.268 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.343981+0000", 2020-08-07T15:07:14.268 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2184e-05 2020-08-07T15:07:14.268 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.268 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.269 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:14.269 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.344033+0000", 2020-08-07T15:07:14.269 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3769999999999998e-06 2020-08-07T15:07:14.269 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.269 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.270 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:14.270 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.344037+0000", 2020-08-07T15:07:14.270 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5410000000000006e-06 2020-08-07T15:07:14.270 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.270 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:14.271 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.344047+0000", 2020-08-07T15:07:14.271 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8470000000000003e-06 2020-08-07T15:07:14.271 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.271 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:14.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.344052+0000", 2020-08-07T15:07:14.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011857199999999999 2020-08-07T15:07:14.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:14.273 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.345237+0000", 2020-08-07T15:07:14.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0568e-05 2020-08-07T15:07:14.273 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.273 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:14.274 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.345298+0000", 2020-08-07T15:07:14.274 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012783200000000001 2020-08-07T15:07:14.274 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.274 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.274 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:07:14.275 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.345426+0000", 2020-08-07T15:07:14.275 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001066858 2020-08-07T15:07:14.275 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.275 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.275 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:14.276 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.346493+0000", 2020-08-07T15:07:14.276 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.75e-07 2020-08-07T15:07:14.276 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.276 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.276 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:14.277 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.346493+0000", 2020-08-07T15:07:14.277 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013222400000000001 2020-08-07T15:07:14.277 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.277 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.277 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:14.277 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.346625+0000", 2020-08-07T15:07:14.278 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.073894926999999999 2020-08-07T15:07:14.278 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.278 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.278 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.278 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.420520+0000", 2020-08-07T15:07:14.279 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.179097231 2020-08-07T15:07:14.279 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.279 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.279 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.279 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.599618+0000", 2020-08-07T15:07:14.280 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9299999999999998e-07 2020-08-07T15:07:14.280 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.280 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.280 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:14.280 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.599618+0000", 2020-08-07T15:07:14.281 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9456999999999998e-05 2020-08-07T15:07:14.281 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.281 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.281 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:14.281 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.599658+0000", 2020-08-07T15:07:14.281 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25567693400000002 2020-08-07T15:07:14.282 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.282 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:14.282 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.282 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.282 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.283 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:24264 3.8 3:1d3fdbda:::benchmark_data_smithi074_35447_object3032:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e28)", 2020-08-07T15:07:14.283 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.377939+0000", 2020-08-07T15:07:14.283 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 21.753973708, 2020-08-07T15:07:14.283 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.425246179, 2020-08-07T15:07:14.283 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:14.284 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:14.284 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:14.284 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:14.284 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:14.284 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 24264 2020-08-07T15:07:14.285 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.285 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:14.285 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.285 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:14.285 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.377939+0000", 2020-08-07T15:07:14.286 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:14.286 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.286 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.286 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:14.286 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.377939+0000", 2020-08-07T15:07:14.287 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8847999999999997e-05 2020-08-07T15:07:14.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.287 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:14.287 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.377987+0000", 2020-08-07T15:07:14.287 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7389999999999996e-06 2020-08-07T15:07:14.288 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.288 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.288 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:14.288 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.377993+0000", 2020-08-07T15:07:14.288 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3533000000000001e-05 2020-08-07T15:07:14.289 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.289 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.289 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:14.289 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.378007+0000", 2020-08-07T15:07:14.289 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7910000000000001e-06 2020-08-07T15:07:14.290 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.290 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.290 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:14.290 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.378013+0000", 2020-08-07T15:07:14.290 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0046691550000000004 2020-08-07T15:07:14.291 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.291 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.291 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:14.291 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.382682+0000", 2020-08-07T15:07:14.291 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7563000000000003e-05 2020-08-07T15:07:14.292 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.292 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.292 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:14.292 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.382759+0000", 2020-08-07T15:07:14.292 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016565800000000001 2020-08-07T15:07:14.292 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.293 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.293 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:07:14.293 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.382925+0000", 2020-08-07T15:07:14.293 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026446130000000001 2020-08-07T15:07:14.293 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.294 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.294 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.294 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.385569+0000", 2020-08-07T15:07:14.294 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00070027000000000002 2020-08-07T15:07:14.294 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.294 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.295 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:14.295 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.386270+0000", 2020-08-07T15:07:14.295 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41678485799999998 2020-08-07T15:07:14.295 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.295 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.296 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.296 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.803055+0000", 2020-08-07T15:07:14.296 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3900000000000003e-07 2020-08-07T15:07:14.296 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.296 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.297 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:14.297 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.803056+0000", 2020-08-07T15:07:14.297 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0357000000000003e-05 2020-08-07T15:07:14.297 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.297 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.298 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:14.298 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.803136+0000", 2020-08-07T15:07:14.298 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8854999999999999e-05 2020-08-07T15:07:14.298 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.298 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.298 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:14.299 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.803185+0000", 2020-08-07T15:07:14.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.425246179 2020-08-07T15:07:14.299 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.299 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:14.299 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.300 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.300 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.300 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:27156 3.2 3:4fef2231:::benchmark_data_smithi074_35447_object3394:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:14.300 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:54.104778+0000", 2020-08-07T15:07:14.300 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 20.027134454999999, 2020-08-07T15:07:14.301 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21340421700000001, 2020-08-07T15:07:14.301 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:14.301 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:14.301 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:14.301 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:14.301 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:14.302 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 27156 2020-08-07T15:07:14.302 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.302 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:14.302 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.302 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:14.303 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.104778+0000", 2020-08-07T15:07:14.303 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:14.303 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.303 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.303 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:14.304 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.104778+0000", 2020-08-07T15:07:14.304 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1062e-05 2020-08-07T15:07:14.304 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.304 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.304 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:14.304 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.104829+0000", 2020-08-07T15:07:14.305 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6799999999999998e-06 2020-08-07T15:07:14.305 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.305 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.305 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:14.305 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.104835+0000", 2020-08-07T15:07:14.306 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.201e-05 2020-08-07T15:07:14.306 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.306 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.306 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:14.306 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.104847+0000", 2020-08-07T15:07:14.307 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7790000000000001e-06 2020-08-07T15:07:14.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.307 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.307 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:14.307 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.104852+0000", 2020-08-07T15:07:14.307 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00028754399999999998 2020-08-07T15:07:14.308 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.308 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:14.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105140+0000", 2020-08-07T15:07:14.308 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000116425 2020-08-07T15:07:14.309 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.309 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.309 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:14.309 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105256+0000", 2020-08-07T15:07:14.309 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000190729 2020-08-07T15:07:14.310 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.310 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.310 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:14.310 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105447+0000", 2020-08-07T15:07:14.310 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0022271819999999999 2020-08-07T15:07:14.311 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.311 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.311 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:14.311 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.107674+0000", 2020-08-07T15:07:14.311 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.285e-06 2020-08-07T15:07:14.311 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.312 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.312 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:14.312 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.107676+0000", 2020-08-07T15:07:14.313 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7856999999999998e-05 2020-08-07T15:07:14.313 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.313 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.313 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:14.313 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.107743+0000", 2020-08-07T15:07:14.314 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039706129999999996 2020-08-07T15:07:14.314 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.314 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.314 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.314 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.111714+0000", 2020-08-07T15:07:14.314 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20642095499999999 2020-08-07T15:07:14.315 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.315 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.315 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.315 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.318135+0000", 2020-08-07T15:07:14.315 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0100000000000001e-06 2020-08-07T15:07:14.316 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.316 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.316 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:14.316 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.318136+0000", 2020-08-07T15:07:14.316 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6085999999999999e-05 2020-08-07T15:07:14.317 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.317 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.317 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:14.317 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.318182+0000", 2020-08-07T15:07:14.317 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21340421700000001 2020-08-07T15:07:14.317 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.318 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:14.318 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.318 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.318 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.318 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:27157 3.2 3:4fef2231:::benchmark_data_smithi074_35447_object3394:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:14.319 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:54.105085+0000", 2020-08-07T15:07:14.319 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 20.026827227999998, 2020-08-07T15:07:14.319 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.213177318, 2020-08-07T15:07:14.319 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:14.319 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:14.320 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:14.320 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:14.320 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:14.320 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 27157 2020-08-07T15:07:14.320 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.320 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:14.321 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.321 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:14.321 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105085+0000", 2020-08-07T15:07:14.321 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:14.321 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.322 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.322 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:14.322 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105085+0000", 2020-08-07T15:07:14.322 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4802000000000001e-05 2020-08-07T15:07:14.322 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.323 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.323 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:14.323 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105140+0000", 2020-08-07T15:07:14.323 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.998e-06 2020-08-07T15:07:14.323 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.323 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.324 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:14.324 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105146+0000", 2020-08-07T15:07:14.324 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2187e-05 2020-08-07T15:07:14.324 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.324 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.325 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:14.325 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105158+0000", 2020-08-07T15:07:14.325 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9739999999999999e-06 2020-08-07T15:07:14.325 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.325 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.326 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:14.326 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105164+0000", 2020-08-07T15:07:14.326 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000735339 2020-08-07T15:07:14.326 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.326 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.326 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:14.327 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105899+0000", 2020-08-07T15:07:14.327 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0789999999999999e-05 2020-08-07T15:07:14.327 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.327 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.327 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:14.328 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105960+0000", 2020-08-07T15:07:14.328 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000114586 2020-08-07T15:07:14.328 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.328 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.328 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:14.328 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.106075+0000", 2020-08-07T15:07:14.329 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043816419999999998 2020-08-07T15:07:14.329 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.329 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.329 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:14.329 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.110456+0000", 2020-08-07T15:07:14.330 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5799999999999998e-07 2020-08-07T15:07:14.330 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.330 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.330 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:14.330 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.110457+0000", 2020-08-07T15:07:14.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7607e-05 2020-08-07T15:07:14.331 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.331 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.331 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:14.331 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.110505+0000", 2020-08-07T15:07:14.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012407989999999999 2020-08-07T15:07:14.332 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.332 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.332 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.332 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.111746+0000", 2020-08-07T15:07:14.332 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20647121099999999 2020-08-07T15:07:14.333 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.333 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.333 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.333 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.318217+0000", 2020-08-07T15:07:14.333 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7599999999999996e-07 2020-08-07T15:07:14.334 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.334 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.334 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:14.334 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.318218+0000", 2020-08-07T15:07:14.334 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4749000000000001e-05 2020-08-07T15:07:14.335 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.335 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:14.335 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.318262+0000", 2020-08-07T15:07:14.335 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.213177318 2020-08-07T15:07:14.336 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.336 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:14.336 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.336 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.336 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.336 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:27158 3.2 3:4fef2231:::benchmark_data_smithi074_35447_object3394:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:14.337 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:54.105200+0000", 2020-08-07T15:07:14.346 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 20.026712027999999, 2020-08-07T15:07:14.346 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21352283999999999, 2020-08-07T15:07:14.346 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:14.346 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:14.346 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:14.347 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:14.347 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:14.347 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 27158 2020-08-07T15:07:14.347 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.347 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:14.348 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.348 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:14.348 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105200+0000", 2020-08-07T15:07:14.348 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:14.348 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.349 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.349 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:14.349 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105200+0000", 2020-08-07T15:07:14.349 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8102999999999997e-05 2020-08-07T15:07:14.350 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.350 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.350 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:14.350 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105248+0000", 2020-08-07T15:07:14.350 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6820000000000002e-06 2020-08-07T15:07:14.351 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.351 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.351 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:14.351 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105253+0000", 2020-08-07T15:07:14.351 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3109999999999995e-06 2020-08-07T15:07:14.352 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.352 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.352 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:14.352 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105262+0000", 2020-08-07T15:07:14.352 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8519999999999997e-06 2020-08-07T15:07:14.353 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.353 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.353 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:14.353 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105266+0000", 2020-08-07T15:07:14.353 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001222133 2020-08-07T15:07:14.353 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.354 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.354 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:14.354 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.106488+0000", 2020-08-07T15:07:14.354 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4035999999999995e-05 2020-08-07T15:07:14.354 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.355 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.355 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:14.355 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.106552+0000", 2020-08-07T15:07:14.355 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000117271 2020-08-07T15:07:14.355 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.356 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.356 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:14.356 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.106670+0000", 2020-08-07T15:07:14.356 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038427420000000001 2020-08-07T15:07:14.356 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.357 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.357 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:14.357 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.110512+0000", 2020-08-07T15:07:14.357 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3300000000000002e-07 2020-08-07T15:07:14.358 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.358 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.358 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:14.358 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.110513+0000", 2020-08-07T15:07:14.358 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4688999999999998e-05 2020-08-07T15:07:14.359 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.359 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.359 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:14.359 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.110568+0000", 2020-08-07T15:07:14.359 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001217187 2020-08-07T15:07:14.360 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.360 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.360 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.360 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.111785+0000", 2020-08-07T15:07:14.360 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20686242099999999 2020-08-07T15:07:14.361 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.361 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.361 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.361 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.318647+0000", 2020-08-07T15:07:14.361 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4199999999999995e-07 2020-08-07T15:07:14.362 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.362 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.362 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:14.362 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.318648+0000", 2020-08-07T15:07:14.362 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5238e-05 2020-08-07T15:07:14.362 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.363 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.363 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:14.363 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.318723+0000", 2020-08-07T15:07:14.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21352283999999999 2020-08-07T15:07:14.363 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.364 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:14.364 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.364 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.364 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:27159 3.2 3:4fef2231:::benchmark_data_smithi074_35447_object3394:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:14.365 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:54.106350+0000", 2020-08-07T15:07:14.365 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 20.025562112999999, 2020-08-07T15:07:14.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21249198999999999, 2020-08-07T15:07:14.365 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:14.366 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:14.367 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:14.367 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:14.368 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:14.369 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 27159 2020-08-07T15:07:14.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.369 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:14.370 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.370 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:14.370 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.106350+0000", 2020-08-07T15:07:14.370 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:14.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.371 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.371 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:14.371 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.106350+0000", 2020-08-07T15:07:14.371 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3527000000000003e-05 2020-08-07T15:07:14.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.372 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:14.372 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.106414+0000", 2020-08-07T15:07:14.372 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6329999999999999e-06 2020-08-07T15:07:14.372 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.373 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.373 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:14.373 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.106420+0000", 2020-08-07T15:07:14.373 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3470000000000001e-05 2020-08-07T15:07:14.373 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.374 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.374 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:14.374 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.106444+0000", 2020-08-07T15:07:14.374 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3289999999999999e-06 2020-08-07T15:07:14.374 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.375 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:14.375 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.106450+0000", 2020-08-07T15:07:14.375 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00053049499999999997 2020-08-07T15:07:14.375 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.376 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:14.376 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.106981+0000", 2020-08-07T15:07:14.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8999000000000002e-05 2020-08-07T15:07:14.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:14.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.107040+0000", 2020-08-07T15:07:14.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000105346 2020-08-07T15:07:14.377 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:14.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.107145+0000", 2020-08-07T15:07:14.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034308810000000002 2020-08-07T15:07:14.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:14.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.110576+0000", 2020-08-07T15:07:14.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3e-07 2020-08-07T15:07:14.379 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:14.380 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.110577+0000", 2020-08-07T15:07:14.380 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9908000000000003e-05 2020-08-07T15:07:14.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:14.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.110636+0000", 2020-08-07T15:07:14.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011884739999999999 2020-08-07T15:07:14.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.382 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.111825+0000", 2020-08-07T15:07:14.382 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.206949144 2020-08-07T15:07:14.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.382 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.382 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.383 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.318774+0000", 2020-08-07T15:07:14.383 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8700000000000005e-07 2020-08-07T15:07:14.383 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.383 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.383 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:14.383 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.318775+0000", 2020-08-07T15:07:14.384 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7366999999999994e-05 2020-08-07T15:07:14.384 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.384 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.384 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:14.384 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.318842+0000", 2020-08-07T15:07:14.385 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21249198999999999 2020-08-07T15:07:14.385 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.385 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:14.385 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.385 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.386 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.386 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:27160 3.2 3:4fef2231:::benchmark_data_smithi074_35447_object3394:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:14.386 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:54.107950+0000", 2020-08-07T15:07:14.386 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 20.023962525999998, 2020-08-07T15:07:14.386 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21115057500000001, 2020-08-07T15:07:14.387 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:14.387 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:14.387 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:14.387 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:14.387 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:14.387 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 27160 2020-08-07T15:07:14.388 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.388 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:14.388 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.388 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:14.388 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.107950+0000", 2020-08-07T15:07:14.389 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:14.389 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.389 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.389 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:14.389 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.107950+0000", 2020-08-07T15:07:14.390 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5924000000000002e-05 2020-08-07T15:07:14.390 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.390 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.390 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:14.390 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.108006+0000", 2020-08-07T15:07:14.391 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3840000000000002e-06 2020-08-07T15:07:14.391 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.391 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.391 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:14.392 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.108012+0000", 2020-08-07T15:07:14.392 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2167999999999999e-05 2020-08-07T15:07:14.392 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.392 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.392 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:14.393 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.108024+0000", 2020-08-07T15:07:14.393 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6289999999999998e-06 2020-08-07T15:07:14.393 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.393 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.393 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:14.394 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.108030+0000", 2020-08-07T15:07:14.394 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015480100000000001 2020-08-07T15:07:14.394 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.394 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.394 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:14.395 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.108185+0000", 2020-08-07T15:07:14.395 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8844999999999994e-05 2020-08-07T15:07:14.395 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.395 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.395 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:14.395 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.108274+0000", 2020-08-07T15:07:14.396 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017925999999999999 2020-08-07T15:07:14.396 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.396 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.396 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:14.396 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.108453+0000", 2020-08-07T15:07:14.397 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0036588900000000001 2020-08-07T15:07:14.397 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.397 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.397 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.397 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.112112+0000", 2020-08-07T15:07:14.398 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000642386 2020-08-07T15:07:14.398 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.398 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.398 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:14.398 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.112754+0000", 2020-08-07T15:07:14.399 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20621706400000001 2020-08-07T15:07:14.399 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.399 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.399 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.318971+0000", 2020-08-07T15:07:14.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5000000000000002e-07 2020-08-07T15:07:14.400 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.400 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:14.400 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.318972+0000", 2020-08-07T15:07:14.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7302e-05 2020-08-07T15:07:14.401 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.401 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.401 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:14.401 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.319049+0000", 2020-08-07T15:07:14.401 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1171999999999997e-05 2020-08-07T15:07:14.402 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.402 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.402 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:14.402 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.319100+0000", 2020-08-07T15:07:14.402 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21115057500000001 2020-08-07T15:07:14.403 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.403 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:14.403 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.403 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.403 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.404 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:27296 3.b 3:de3f8919:::benchmark_data_smithi074_35447_object3411:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:14.404 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:54.195758+0000", 2020-08-07T15:07:14.404 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 19.936153912999998, 2020-08-07T15:07:14.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28811428300000003, 2020-08-07T15:07:14.404 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:14.404 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:14.405 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:14.405 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:14.405 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:14.405 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 27296 2020-08-07T15:07:14.405 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.406 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:14.406 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.406 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:14.406 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.195758+0000", 2020-08-07T15:07:14.406 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:14.407 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.407 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.407 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:14.407 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.195758+0000", 2020-08-07T15:07:14.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9490999999999997e-05 2020-08-07T15:07:14.407 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.408 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.408 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:14.408 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.195818+0000", 2020-08-07T15:07:14.408 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7470000000000002e-06 2020-08-07T15:07:14.408 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.409 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.409 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:14.409 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.195824+0000", 2020-08-07T15:07:14.409 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.167e-05 2020-08-07T15:07:14.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.410 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.410 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:14.410 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.195835+0000", 2020-08-07T15:07:14.410 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3129999999999998e-06 2020-08-07T15:07:14.410 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.410 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.411 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:14.411 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.195841+0000", 2020-08-07T15:07:14.411 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0061607880000000004 2020-08-07T15:07:14.411 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.411 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.412 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:14.412 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.202001+0000", 2020-08-07T15:07:14.412 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6767000000000005e-05 2020-08-07T15:07:14.412 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.412 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.413 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:14.413 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.202088+0000", 2020-08-07T15:07:14.413 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000180087 2020-08-07T15:07:14.413 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.413 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:14.414 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.202268+0000", 2020-08-07T15:07:14.414 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0055746279999999999 2020-08-07T15:07:14.414 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.414 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.414 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:14.415 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.207843+0000", 2020-08-07T15:07:14.415 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4099999999999998e-07 2020-08-07T15:07:14.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.415 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:14.416 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.207844+0000", 2020-08-07T15:07:14.416 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9927999999999999e-05 2020-08-07T15:07:14.416 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.416 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.416 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:14.416 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.207914+0000", 2020-08-07T15:07:14.417 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.019254090000000001 2020-08-07T15:07:14.417 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.417 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.417 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.417 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.227168+0000", 2020-08-07T15:07:14.418 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.256637636 2020-08-07T15:07:14.418 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.418 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.418 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.418 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.483805+0000", 2020-08-07T15:07:14.419 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7700000000000003e-07 2020-08-07T15:07:14.419 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.419 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.419 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:14.419 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.483806+0000", 2020-08-07T15:07:14.420 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6519999999999993e-05 2020-08-07T15:07:14.420 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.420 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.420 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:14.420 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.483873+0000", 2020-08-07T15:07:14.420 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28811428300000003 2020-08-07T15:07:14.421 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.421 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:14.421 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.421 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.422 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:27296 3.b e29/20 ondisk, result = 0)", 2020-08-07T15:07:14.422 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:54.207466+0000", 2020-08-07T15:07:14.422 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 19.924446607, 2020-08-07T15:07:14.422 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27641121299999999, 2020-08-07T15:07:14.422 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:14.423 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:14.423 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:14.423 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.423 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:14.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.207466+0000", 2020-08-07T15:07:14.423 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:14.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.424 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.424 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:14.424 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.207466+0000", 2020-08-07T15:07:14.424 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3191999999999997e-05 2020-08-07T15:07:14.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.425 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:14.425 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.207499+0000", 2020-08-07T15:07:14.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1140000000000002e-06 2020-08-07T15:07:14.426 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.426 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.426 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:14.426 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.207505+0000", 2020-08-07T15:07:14.426 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.275903079 2020-08-07T15:07:14.427 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.427 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:14.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.483408+0000", 2020-08-07T15:07:14.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6809999999999997e-06 2020-08-07T15:07:14.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:14.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.483413+0000", 2020-08-07T15:07:14.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00037547699999999998 2020-08-07T15:07:14.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.429 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:14.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.483788+0000", 2020-08-07T15:07:14.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5450000000000003e-06 2020-08-07T15:07:14.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.430 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.430 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:14.430 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.483797+0000", 2020-08-07T15:07:14.430 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0124999999999996e-05 2020-08-07T15:07:14.430 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.431 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:14.431 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.483877+0000", 2020-08-07T15:07:14.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27641121299999999 2020-08-07T15:07:14.431 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.431 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:14.432 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.432 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.432 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.432 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:47336 3.8 3:1652c683:::benchmark_data_smithi074_35447_object5916:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e36)", 2020-08-07T15:07:14.432 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:08.384949+0000", 2020-08-07T15:07:14.433 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.7469635610000003, 2020-08-07T15:07:14.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.232023073, 2020-08-07T15:07:14.433 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:14.433 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:14.433 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:14.434 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:14.434 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:14.434 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 47336 2020-08-07T15:07:14.434 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.434 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:14.435 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.435 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:14.435 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.384949+0000", 2020-08-07T15:07:14.435 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:14.435 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.435 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.436 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:14.436 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.384949+0000", 2020-08-07T15:07:14.436 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.545e-05 2020-08-07T15:07:14.436 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.436 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.437 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:14.437 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.384994+0000", 2020-08-07T15:07:14.437 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3240000000000002e-06 2020-08-07T15:07:14.437 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.437 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.437 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:14.438 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.385000+0000", 2020-08-07T15:07:14.438 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0817e-05 2020-08-07T15:07:14.438 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.438 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.438 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:14.439 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.385010+0000", 2020-08-07T15:07:14.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.011e-06 2020-08-07T15:07:14.439 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.439 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.439 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:14.440 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.385015+0000", 2020-08-07T15:07:14.440 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0029727579999999998 2020-08-07T15:07:14.440 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.440 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:14.440 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.387988+0000", 2020-08-07T15:07:14.441 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8638000000000003e-05 2020-08-07T15:07:14.441 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.441 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.441 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:14.441 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.388047+0000", 2020-08-07T15:07:14.442 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000142495 2020-08-07T15:07:14.442 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.442 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.442 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:07:14.442 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.388189+0000", 2020-08-07T15:07:14.443 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00210209 2020-08-07T15:07:14.443 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.443 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.443 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.443 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.390291+0000", 2020-08-07T15:07:14.443 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00089285899999999997 2020-08-07T15:07:14.444 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.444 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:14.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.391184+0000", 2020-08-07T15:07:14.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.225717942 2020-08-07T15:07:14.445 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.445 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.445 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.445 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.616902+0000", 2020-08-07T15:07:14.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7000000000000004e-07 2020-08-07T15:07:14.446 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.446 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.446 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:14.446 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.616903+0000", 2020-08-07T15:07:14.446 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0258000000000001e-05 2020-08-07T15:07:14.446 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.447 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:14.447 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.616933+0000", 2020-08-07T15:07:14.447 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8661000000000001e-05 2020-08-07T15:07:14.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:14.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.616972+0000", 2020-08-07T15:07:14.448 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.232023073 2020-08-07T15:07:14.448 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.448 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:14.449 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.449 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.449 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:48776 3.2 3:47029331:::benchmark_data_smithi074_35447_object6096:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e36)", 2020-08-07T15:07:14.450 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:09.403100+0000", 2020-08-07T15:07:14.450 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4.728812381, 2020-08-07T15:07:14.450 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27318728799999997, 2020-08-07T15:07:14.450 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:14.451 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:14.451 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:14.451 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:14.451 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:14.451 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48776 2020-08-07T15:07:14.451 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.452 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:14.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:14.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.403100+0000", 2020-08-07T15:07:14.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:14.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.453 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:14.453 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.403100+0000", 2020-08-07T15:07:14.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9872e-05 2020-08-07T15:07:14.454 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.454 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.454 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:14.454 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.403150+0000", 2020-08-07T15:07:14.454 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4469999999999997e-06 2020-08-07T15:07:14.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:14.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.403155+0000", 2020-08-07T15:07:14.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7580000000000004e-06 2020-08-07T15:07:14.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:14.456 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.403165+0000", 2020-08-07T15:07:14.456 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5980000000000001e-06 2020-08-07T15:07:14.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:14.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.403169+0000", 2020-08-07T15:07:14.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000890933 2020-08-07T15:07:14.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.458 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.458 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:14.458 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.404060+0000", 2020-08-07T15:07:14.458 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000102286 2020-08-07T15:07:14.458 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.458 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:14.459 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.404162+0000", 2020-08-07T15:07:14.459 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018597700000000001 2020-08-07T15:07:14.459 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.460 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:14.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.404348+0000", 2020-08-07T15:07:14.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010765029999999999 2020-08-07T15:07:14.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.460 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.461 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:14.461 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.405424+0000", 2020-08-07T15:07:14.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9600000000000001e-07 2020-08-07T15:07:14.461 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.461 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.461 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:14.462 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.405424+0000", 2020-08-07T15:07:14.462 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7682e-05 2020-08-07T15:07:14.462 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.462 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.462 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:14.463 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.405452+0000", 2020-08-07T15:07:14.463 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045217419999999996 2020-08-07T15:07:14.463 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.463 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.463 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.464 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.409974+0000", 2020-08-07T15:07:14.464 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26623468500000003 2020-08-07T15:07:14.464 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.464 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.464 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.465 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.676209+0000", 2020-08-07T15:07:14.465 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5099999999999998e-07 2020-08-07T15:07:14.465 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.465 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.466 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:14.466 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.676210+0000", 2020-08-07T15:07:14.466 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7657999999999994e-05 2020-08-07T15:07:14.466 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.466 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.467 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:14.467 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.676287+0000", 2020-08-07T15:07:14.467 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27318728799999997 2020-08-07T15:07:14.467 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.467 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:14.468 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.468 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.468 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.468 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:49208 3.2 3:4d72b982:::benchmark_data_smithi074_35447_object6150:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e36)", 2020-08-07T15:07:14.468 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:09.690130+0000", 2020-08-07T15:07:14.468 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4.441781894, 2020-08-07T15:07:14.469 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.237499918, 2020-08-07T15:07:14.469 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:14.469 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:14.469 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:14.469 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:14.470 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:14.470 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 49208 2020-08-07T15:07:14.470 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.470 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:14.470 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.471 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:14.471 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.690130+0000", 2020-08-07T15:07:14.471 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:14.471 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.471 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.471 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:14.472 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.690130+0000", 2020-08-07T15:07:14.472 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3445000000000003e-05 2020-08-07T15:07:14.472 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.472 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.472 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:14.473 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.690164+0000", 2020-08-07T15:07:14.473 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1869999999999999e-06 2020-08-07T15:07:14.473 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.473 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.473 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:14.474 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.690168+0000", 2020-08-07T15:07:14.474 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2430000000000005e-06 2020-08-07T15:07:14.474 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.474 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.474 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:14.474 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.690176+0000", 2020-08-07T15:07:14.475 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9579999999999998e-06 2020-08-07T15:07:14.475 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.475 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.475 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:14.475 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.690181+0000", 2020-08-07T15:07:14.476 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010947610000000001 2020-08-07T15:07:14.476 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.476 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.476 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:14.476 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.691276+0000", 2020-08-07T15:07:14.476 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000104084 2020-08-07T15:07:14.477 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.477 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.477 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:14.477 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.691380+0000", 2020-08-07T15:07:14.477 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020709799999999999 2020-08-07T15:07:14.478 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.478 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.478 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:14.478 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.691587+0000", 2020-08-07T15:07:14.478 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024913349999999999 2020-08-07T15:07:14.479 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.479 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.479 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.479 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.694079+0000", 2020-08-07T15:07:14.479 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0031218000000000001 2020-08-07T15:07:14.480 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.480 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:14.480 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.697200+0000", 2020-08-07T15:07:14.480 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23029393400000001 2020-08-07T15:07:14.480 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.481 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.481 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.481 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.927494+0000", 2020-08-07T15:07:14.481 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5799999999999998e-07 2020-08-07T15:07:14.481 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.482 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.482 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:14.482 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.927495+0000", 2020-08-07T15:07:14.482 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2087000000000004e-05 2020-08-07T15:07:14.482 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.482 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.483 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:14.483 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.927557+0000", 2020-08-07T15:07:14.483 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3127999999999995e-05 2020-08-07T15:07:14.483 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.483 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.484 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:14.484 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.927630+0000", 2020-08-07T15:07:14.484 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.237499918 2020-08-07T15:07:14.484 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.484 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:14.485 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.485 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.485 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.485 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:54280 3.b 3:d5ab750f:::benchmark_data_smithi074_35447_object6784:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e37)", 2020-08-07T15:07:14.485 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:13.151346+0000", 2020-08-07T15:07:14.485 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.98056593400000003, 2020-08-07T15:07:14.486 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20161517800000001, 2020-08-07T15:07:14.486 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:14.486 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:14.486 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:14.486 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:14.487 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:14.487 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 54280 2020-08-07T15:07:14.487 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.487 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:14.487 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:14.488 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.151346+0000", 2020-08-07T15:07:14.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:14.488 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:14.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.151346+0000", 2020-08-07T15:07:14.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0086999999999997e-05 2020-08-07T15:07:14.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.489 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.489 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:14.490 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.151396+0000", 2020-08-07T15:07:14.490 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1429999999999997e-06 2020-08-07T15:07:14.490 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.490 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.490 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:14.491 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.151402+0000", 2020-08-07T15:07:14.491 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1110999999999999e-05 2020-08-07T15:07:14.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.491 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.492 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:14.492 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.151413+0000", 2020-08-07T15:07:14.492 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3279999999999999e-06 2020-08-07T15:07:14.492 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.492 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:14.493 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.151417+0000", 2020-08-07T15:07:14.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032678080000000001 2020-08-07T15:07:14.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.493 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.494 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:14.494 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.154685+0000", 2020-08-07T15:07:14.494 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010518099999999999 2020-08-07T15:07:14.494 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.494 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:14.495 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.154790+0000", 2020-08-07T15:07:14.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000180824 2020-08-07T15:07:14.495 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.495 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.495 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:14.496 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.154971+0000", 2020-08-07T15:07:14.496 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0046537669999999996 2020-08-07T15:07:14.496 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.496 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.496 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:14.497 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.159625+0000", 2020-08-07T15:07:14.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5799999999999999e-07 2020-08-07T15:07:14.497 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.497 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.497 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:14.497 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.159625+0000", 2020-08-07T15:07:14.498 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00033638 2020-08-07T15:07:14.498 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.498 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.498 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:14.498 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.159962+0000", 2020-08-07T15:07:14.499 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000241081 2020-08-07T15:07:14.499 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.499 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.499 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.500 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.160203+0000", 2020-08-07T15:07:14.500 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19268943899999999 2020-08-07T15:07:14.500 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.500 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.501 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.501 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.352892+0000", 2020-08-07T15:07:14.501 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1710000000000001e-06 2020-08-07T15:07:14.501 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.501 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.502 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:14.502 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.352893+0000", 2020-08-07T15:07:14.502 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8300000000000007e-05 2020-08-07T15:07:14.502 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.502 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.503 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:14.503 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.352962+0000", 2020-08-07T15:07:14.503 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20161517800000001 2020-08-07T15:07:14.503 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.504 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:14.504 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.504 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.504 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.504 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:54727 3.7 3:ee999259:::benchmark_data_smithi074_35447_object6840:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e37)", 2020-08-07T15:07:14.504 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:13.448978+0000", 2020-08-07T15:07:14.505 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.68293395199999996, 2020-08-07T15:07:14.505 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.192205392, 2020-08-07T15:07:14.505 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:14.505 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:14.506 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:14.506 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:14.506 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:14.506 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 54727 2020-08-07T15:07:14.506 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.507 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:14.507 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.507 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:14.507 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.448978+0000", 2020-08-07T15:07:14.507 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:14.508 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.508 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.508 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:14.508 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.448978+0000", 2020-08-07T15:07:14.508 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7193e-05 2020-08-07T15:07:14.509 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.509 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.509 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:14.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.449026+0000", 2020-08-07T15:07:14.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4360000000000001e-06 2020-08-07T15:07:14.510 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.510 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:14.510 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.449031+0000", 2020-08-07T15:07:14.511 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0132000000000001e-05 2020-08-07T15:07:14.511 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:14.511 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.449041+0000", 2020-08-07T15:07:14.512 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7700000000000001e-06 2020-08-07T15:07:14.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.512 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.512 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:14.512 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.449046+0000", 2020-08-07T15:07:14.512 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0016099560000000001 2020-08-07T15:07:14.513 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.513 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.513 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:14.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.450656+0000", 2020-08-07T15:07:14.513 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011117500000000001 2020-08-07T15:07:14.514 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.514 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:14.514 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.450767+0000", 2020-08-07T15:07:14.514 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019104500000000001 2020-08-07T15:07:14.515 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.515 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.515 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:07:14.515 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.450958+0000", 2020-08-07T15:07:14.515 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032997370000000001 2020-08-07T15:07:14.515 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.516 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.516 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:14.516 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.454258+0000", 2020-08-07T15:07:14.516 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9999999999999996e-07 2020-08-07T15:07:14.516 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.517 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.517 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:14.517 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.454259+0000", 2020-08-07T15:07:14.517 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2459000000000006e-05 2020-08-07T15:07:14.517 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.518 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:14.518 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.454321+0000", 2020-08-07T15:07:14.518 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002932175 2020-08-07T15:07:14.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.519 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.519 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.457253+0000", 2020-08-07T15:07:14.519 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18386295999999999 2020-08-07T15:07:14.519 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.519 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.520 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.641116+0000", 2020-08-07T15:07:14.520 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.051e-06 2020-08-07T15:07:14.520 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.520 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:14.521 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.641117+0000", 2020-08-07T15:07:14.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6402999999999994e-05 2020-08-07T15:07:14.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.521 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.521 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:14.522 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.641184+0000", 2020-08-07T15:07:14.522 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.192205392 2020-08-07T15:07:14.522 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.522 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:14.522 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.523 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.523 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.523 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:54728 3.7 3:ee999259:::benchmark_data_smithi074_35447_object6840:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e37)", 2020-08-07T15:07:14.523 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:13.449079+0000", 2020-08-07T15:07:14.523 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.68283296100000002, 2020-08-07T15:07:14.523 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.192179771, 2020-08-07T15:07:14.524 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:14.524 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:14.524 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:14.524 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:14.524 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:14.525 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 54728 2020-08-07T15:07:14.525 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.525 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:14.525 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.525 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:14.526 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.449079+0000", 2020-08-07T15:07:14.526 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:14.526 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.526 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.526 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:14.526 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.449079+0000", 2020-08-07T15:07:14.527 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9458000000000001e-05 2020-08-07T15:07:14.527 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.527 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.527 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:14.527 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.449119+0000", 2020-08-07T15:07:14.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8189999999999997e-06 2020-08-07T15:07:14.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.528 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:14.528 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.449125+0000", 2020-08-07T15:07:14.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4169999999999997e-06 2020-08-07T15:07:14.529 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.529 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.529 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:14.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.449131+0000", 2020-08-07T15:07:14.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6819999999999999e-06 2020-08-07T15:07:14.530 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.530 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.530 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:14.530 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.449134+0000", 2020-08-07T15:07:14.530 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002251542 2020-08-07T15:07:14.531 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.531 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.531 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:14.531 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.451385+0000", 2020-08-07T15:07:14.531 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010285600000000001 2020-08-07T15:07:14.532 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.532 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.532 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:14.532 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.451488+0000", 2020-08-07T15:07:14.532 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018447200000000001 2020-08-07T15:07:14.532 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.533 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.533 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:07:14.533 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.451673+0000", 2020-08-07T15:07:14.533 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026569620000000001 2020-08-07T15:07:14.533 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.534 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.534 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:14.534 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.454330+0000", 2020-08-07T15:07:14.534 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4000000000000001e-07 2020-08-07T15:07:14.534 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:14.535 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.454330+0000", 2020-08-07T15:07:14.535 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5572999999999998e-05 2020-08-07T15:07:14.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:14.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.454386+0000", 2020-08-07T15:07:14.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028997229999999999 2020-08-07T15:07:14.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.536 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.537 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.457285+0000", 2020-08-07T15:07:14.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.183932806 2020-08-07T15:07:14.537 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.537 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.537 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.538 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.641218+0000", 2020-08-07T15:07:14.538 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8100000000000003e-07 2020-08-07T15:07:14.538 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.538 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.538 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:14.539 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.641219+0000", 2020-08-07T15:07:14.539 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0240000000000001e-05 2020-08-07T15:07:14.539 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.539 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.539 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:14.540 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.641259+0000", 2020-08-07T15:07:14.540 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.192179771 2020-08-07T15:07:14.540 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.540 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:14.540 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.540 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.541 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:54727 3.7 e37/20 ondisk, result = 0)", 2020-08-07T15:07:14.541 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:13.453471+0000", 2020-08-07T15:07:14.541 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.678440928, 2020-08-07T15:07:14.541 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18771787100000001, 2020-08-07T15:07:14.542 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:14.542 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:14.542 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:14.550 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.550 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:14.551 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.453471+0000", 2020-08-07T15:07:14.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:14.551 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.552 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:14.552 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.453471+0000", 2020-08-07T15:07:14.552 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2326000000000002e-05 2020-08-07T15:07:14.552 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.553 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:14.553 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.453504+0000", 2020-08-07T15:07:14.553 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6460000000000001e-06 2020-08-07T15:07:14.553 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.553 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.554 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:14.554 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.453508+0000", 2020-08-07T15:07:14.554 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.186921526 2020-08-07T15:07:14.554 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.554 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.555 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:14.555 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.640430+0000", 2020-08-07T15:07:14.555 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2619999999999998e-06 2020-08-07T15:07:14.555 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.555 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.555 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:14.556 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.640435+0000", 2020-08-07T15:07:14.556 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00066332700000000001 2020-08-07T15:07:14.556 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:14.557 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.641098+0000", 2020-08-07T15:07:14.557 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7460000000000006e-06 2020-08-07T15:07:14.557 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.557 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.557 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:14.558 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.641107+0000", 2020-08-07T15:07:14.558 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2038000000000003e-05 2020-08-07T15:07:14.558 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:14.558 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:14.558 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:14.558 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.641189+0000", 2020-08-07T15:07:14.559 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18771787100000001 2020-08-07T15:07:14.559 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.559 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:14.559 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.559 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:14.560 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:14.560 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:14.562 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 38 16 55977 55961 11.5042 12.1875 0.00526745 0.0108612 2020-08-07T15:07:14.562 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:07:14.563 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:14.560+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:14.563 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:07:14.662 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:14.661+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:14.760 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:14.761 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:14.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.761 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:56680 3.1 3:83552155:::benchmark_data_smithi074_35447_object7084:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e38)", 2020-08-07T15:07:14.761 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:14.647521+0000", 2020-08-07T15:07:14.763 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.097770836999999999, 2020-08-07T15:07:14.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.097811185999999994, 2020-08-07T15:07:14.764 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:14.764 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:14.764 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:14.764 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:14.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:14.765 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 56680 2020-08-07T15:07:14.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.765 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:14.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:14.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.647521+0000", 2020-08-07T15:07:14.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:14.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:14.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.647521+0000", 2020-08-07T15:07:14.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8937999999999998e-05 2020-08-07T15:07:14.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:14.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.647570+0000", 2020-08-07T15:07:14.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2249999999999999e-06 2020-08-07T15:07:14.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:14.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.647575+0000", 2020-08-07T15:07:14.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0263e-05 2020-08-07T15:07:14.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:14.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.647586+0000", 2020-08-07T15:07:14.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7369999999999997e-06 2020-08-07T15:07:14.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:14.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.647590+0000", 2020-08-07T15:07:14.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042509369999999998 2020-08-07T15:07:14.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:14.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.651841+0000", 2020-08-07T15:07:14.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9320000000000001e-05 2020-08-07T15:07:14.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:14.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.651901+0000", 2020-08-07T15:07:14.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013379700000000001 2020-08-07T15:07:14.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,3", 2020-08-07T15:07:14.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.652035+0000", 2020-08-07T15:07:14.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003129565 2020-08-07T15:07:14.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:14.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.655164+0000", 2020-08-07T15:07:14.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3600000000000002e-07 2020-08-07T15:07:14.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:14.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.655165+0000", 2020-08-07T15:07:14.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4185999999999997e-05 2020-08-07T15:07:14.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:14.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.655239+0000", 2020-08-07T15:07:14.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050231190000000004 2020-08-07T15:07:14.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.660262+0000", 2020-08-07T15:07:14.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012740922999999999 2020-08-07T15:07:14.778 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:14.778 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:14.779 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:14.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.779 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:56842 3.e e38/20 3:72d23b69:::benchmark_data_smithi074_35447_object7105:head v 38'3690, mlcod=38'3687)", 2020-08-07T15:07:14.779 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:14.744075+0000", 2020-08-07T15:07:14.780 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0012174530000000001, 2020-08-07T15:07:14.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001384936, 2020-08-07T15:07:14.780 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:14.780 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:14.780 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:14.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:14.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.744075+0000", 2020-08-07T15:07:14.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:14.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:14.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.744075+0000", 2020-08-07T15:07:14.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1097000000000002e-05 2020-08-07T15:07:14.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:14.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.744126+0000", 2020-08-07T15:07:14.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6000000000000003e-06 2020-08-07T15:07:14.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:14.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.744132+0000", 2020-08-07T15:07:14.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00090173100000000002 2020-08-07T15:07:14.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:14.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.745034+0000", 2020-08-07T15:07:14.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.237e-06 2020-08-07T15:07:14.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:14.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.745039+0000", 2020-08-07T15:07:14.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012826100000000001 2020-08-07T15:07:14.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:14.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.745168+0000", 2020-08-07T15:07:14.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0293999999999999e-05 2020-08-07T15:07:14.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:14.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.745188+0000", 2020-08-07T15:07:14.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011132200000000001 2020-08-07T15:07:14.789 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:14.789 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:14.789 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:14.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.790 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:56592 3.9 3:9c8f6dc2:::benchmark_data_smithi074_35447_object7073:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e38)", 2020-08-07T15:07:14.790 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:14.595886+0000", 2020-08-07T15:07:14.790 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.14940629699999999, 2020-08-07T15:07:14.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14965404299999999, 2020-08-07T15:07:14.791 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:14.791 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:14.791 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:14.791 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:14.791 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:14.792 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 56592 2020-08-07T15:07:14.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.792 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:14.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:14.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.595886+0000", 2020-08-07T15:07:14.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:14.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:14.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.595886+0000", 2020-08-07T15:07:14.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7808e-05 2020-08-07T15:07:14.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:14.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.595934+0000", 2020-08-07T15:07:14.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1539999999999999e-06 2020-08-07T15:07:14.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:14.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.595940+0000", 2020-08-07T15:07:14.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2046e-05 2020-08-07T15:07:14.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:14.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.595952+0000", 2020-08-07T15:07:14.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.587e-06 2020-08-07T15:07:14.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:14.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.595957+0000", 2020-08-07T15:07:14.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056391690000000003 2020-08-07T15:07:14.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:14.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.601597+0000", 2020-08-07T15:07:14.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3449000000000006e-05 2020-08-07T15:07:14.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:14.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.601690+0000", 2020-08-07T15:07:14.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018503500000000001 2020-08-07T15:07:14.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:14.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.601875+0000", 2020-08-07T15:07:14.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055386790000000003 2020-08-07T15:07:14.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:14.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.607414+0000", 2020-08-07T15:07:14.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9699999999999995e-07 2020-08-07T15:07:14.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:14.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.607415+0000", 2020-08-07T15:07:14.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9597000000000001e-05 2020-08-07T15:07:14.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:14.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.607454+0000", 2020-08-07T15:07:14.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023330849999999999 2020-08-07T15:07:14.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:14.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:14.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:14.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.609787+0000", 2020-08-07T15:07:14.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013901406 2020-08-07T15:07:14.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:14.805 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:14.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:14.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:14.805 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:07:14.806 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 3 2020-08-07T15:07:14.806 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:14.806 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:07:14.808 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:14.807+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:14.920 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:14.919+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:14.968 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:07:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:07:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:07:14.969 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:14.980 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:07:15.021 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:07:15.023 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:15.022+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:15.130 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:15.123+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:15.200 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:07:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:07:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:2600 3.b e23/20 3:d6a1a0d3:::benchmark_data_smithi074_35447_object324:head v 23'208, mlcod=23'201)", 2020-08-07T15:07:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.807190+0000", 2020-08-07T15:07:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.388240883999998, 2020-08-07T15:07:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20835313, 2020-08-07T15:07:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.807190+0000", 2020-08-07T15:07:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.807190+0000", 2020-08-07T15:07:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.938e-05 2020-08-07T15:07:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.807219+0000", 2020-08-07T15:07:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2780000000000004e-06 2020-08-07T15:07:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.807223+0000", 2020-08-07T15:07:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20663868699999999 2020-08-07T15:07:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.013862+0000", 2020-08-07T15:07:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.659e-06 2020-08-07T15:07:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.013866+0000", 2020-08-07T15:07:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7111999999999999e-05 2020-08-07T15:07:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:15.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.013913+0000", 2020-08-07T15:07:15.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5225999999999999e-05 2020-08-07T15:07:15.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:15.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.013938+0000", 2020-08-07T15:07:15.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015643429999999999 2020-08-07T15:07:15.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:15.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.015502+0000", 2020-08-07T15:07:15.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0445000000000003e-05 2020-08-07T15:07:15.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:15.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:38.015543+0000", 2020-08-07T15:07:15.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20835313 2020-08-07T15:07:15.221 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.222 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:15.222 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.223 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:7576 3.b e24/20 3:d854a547:::benchmark_data_smithi074_35447_object946:head v 24'456, mlcod=24'448)", 2020-08-07T15:07:15.223 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:40.741593+0000", 2020-08-07T15:07:15.224 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.453837839000002, 2020-08-07T15:07:15.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18773600700000001, 2020-08-07T15:07:15.226 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:15.226 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:15.226 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:15.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:15.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.741593+0000", 2020-08-07T15:07:15.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:15.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:15.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.741593+0000", 2020-08-07T15:07:15.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1712000000000002e-05 2020-08-07T15:07:15.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:15.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.741645+0000", 2020-08-07T15:07:15.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2389999999999999e-06 2020-08-07T15:07:15.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:15.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.741651+0000", 2020-08-07T15:07:15.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18616382400000001 2020-08-07T15:07:15.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:15.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.927815+0000", 2020-08-07T15:07:15.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7359999999999999e-06 2020-08-07T15:07:15.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:15.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.927818+0000", 2020-08-07T15:07:15.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3000000000000002e-05 2020-08-07T15:07:15.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:15.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.927861+0000", 2020-08-07T15:07:15.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4202e-05 2020-08-07T15:07:15.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:15.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.927886+0000", 2020-08-07T15:07:15.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014097070000000001 2020-08-07T15:07:15.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:15.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.929295+0000", 2020-08-07T15:07:15.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3587000000000002e-05 2020-08-07T15:07:15.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:15.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.929329+0000", 2020-08-07T15:07:15.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18773600700000001 2020-08-07T15:07:15.247 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.247 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:15.248 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.249 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:7848 3.9 3:905b1512:::benchmark_data_smithi074_35447_object980:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:15.249 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:40.895061+0000", 2020-08-07T15:07:15.250 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.300369279999998, 2020-08-07T15:07:15.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.187604353, 2020-08-07T15:07:15.250 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:15.251 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:15.251 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:15.251 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:15.252 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:15.253 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7848 2020-08-07T15:07:15.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.253 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:15.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:15.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.895061+0000", 2020-08-07T15:07:15.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:15.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:15.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.895061+0000", 2020-08-07T15:07:15.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7772999999999997e-05 2020-08-07T15:07:15.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:15.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.895119+0000", 2020-08-07T15:07:15.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3860000000000004e-06 2020-08-07T15:07:15.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:15.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.895126+0000", 2020-08-07T15:07:15.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1783000000000001e-05 2020-08-07T15:07:15.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:15.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.895137+0000", 2020-08-07T15:07:15.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6629999999999998e-06 2020-08-07T15:07:15.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:15.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.895143+0000", 2020-08-07T15:07:15.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024182299999999999 2020-08-07T15:07:15.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:15.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.897561+0000", 2020-08-07T15:07:15.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00098884499999999992 2020-08-07T15:07:15.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:15.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.898550+0000", 2020-08-07T15:07:15.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015507699999999999 2020-08-07T15:07:15.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:15.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.898705+0000", 2020-08-07T15:07:15.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021473949999999999 2020-08-07T15:07:15.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:15.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.900853+0000", 2020-08-07T15:07:15.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001259788 2020-08-07T15:07:15.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:15.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:40.902112+0000", 2020-08-07T15:07:15.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.180465141 2020-08-07T15:07:15.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:15.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.082577+0000", 2020-08-07T15:07:15.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9599999999999998e-07 2020-08-07T15:07:15.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:15.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.082578+0000", 2020-08-07T15:07:15.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9520000000000001e-05 2020-08-07T15:07:15.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:15.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.082618+0000", 2020-08-07T15:07:15.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7956e-05 2020-08-07T15:07:15.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:15.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.082666+0000", 2020-08-07T15:07:15.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.187604353 2020-08-07T15:07:15.274 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.274 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:15.274 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.275 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:8112 3.9 3:9b275d48:::benchmark_data_smithi074_35447_object1013:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:15.275 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.083981+0000", 2020-08-07T15:07:15.275 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.111449821999997, 2020-08-07T15:07:15.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31911657999999998, 2020-08-07T15:07:15.276 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:15.276 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:15.276 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:15.276 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:15.277 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:15.277 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 8112 2020-08-07T15:07:15.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.277 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:15.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:15.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.083981+0000", 2020-08-07T15:07:15.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:15.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:15.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.083981+0000", 2020-08-07T15:07:15.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4051999999999999e-05 2020-08-07T15:07:15.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:15.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084015+0000", 2020-08-07T15:07:15.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7290000000000002e-06 2020-08-07T15:07:15.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:15.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084019+0000", 2020-08-07T15:07:15.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2280000000000001e-06 2020-08-07T15:07:15.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:15.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084026+0000", 2020-08-07T15:07:15.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7780000000000001e-06 2020-08-07T15:07:15.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:15.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084030+0000", 2020-08-07T15:07:15.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00077372300000000003 2020-08-07T15:07:15.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:15.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084803+0000", 2020-08-07T15:07:15.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1502000000000003e-05 2020-08-07T15:07:15.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:15.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084855+0000", 2020-08-07T15:07:15.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5110000000000002e-05 2020-08-07T15:07:15.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:15.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084950+0000", 2020-08-07T15:07:15.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012531370000000001 2020-08-07T15:07:15.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:15.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.086203+0000", 2020-08-07T15:07:15.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4799999999999998e-07 2020-08-07T15:07:15.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:15.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.086204+0000", 2020-08-07T15:07:15.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.905e-05 2020-08-07T15:07:15.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:15.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.086253+0000", 2020-08-07T15:07:15.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00062940999999999995 2020-08-07T15:07:15.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:15.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.086882+0000", 2020-08-07T15:07:15.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31612944199999998 2020-08-07T15:07:15.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:15.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.403012+0000", 2020-08-07T15:07:15.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9099999999999991e-07 2020-08-07T15:07:15.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:15.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.403013+0000", 2020-08-07T15:07:15.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.488e-05 2020-08-07T15:07:15.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:15.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.403097+0000", 2020-08-07T15:07:15.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31911657999999998 2020-08-07T15:07:15.296 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.296 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:15.296 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.297 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:8760 3.9 3:9f02d263:::benchmark_data_smithi074_35447_object1094:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:15.297 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.531357+0000", 2020-08-07T15:07:15.297 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.664073438000003, 2020-08-07T15:07:15.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19863291199999999, 2020-08-07T15:07:15.298 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:15.298 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:15.298 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:15.299 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:15.299 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:15.299 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 8760 2020-08-07T15:07:15.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.299 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:15.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:15.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.531357+0000", 2020-08-07T15:07:15.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:15.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:15.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.531357+0000", 2020-08-07T15:07:15.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9942999999999999e-05 2020-08-07T15:07:15.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:15.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.531407+0000", 2020-08-07T15:07:15.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8319999999999996e-06 2020-08-07T15:07:15.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:15.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.531412+0000", 2020-08-07T15:07:15.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0128e-05 2020-08-07T15:07:15.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:15.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.531422+0000", 2020-08-07T15:07:15.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2370000000000003e-06 2020-08-07T15:07:15.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:15.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.531426+0000", 2020-08-07T15:07:15.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034586119999999998 2020-08-07T15:07:15.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:15.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.534885+0000", 2020-08-07T15:07:15.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6256999999999997e-05 2020-08-07T15:07:15.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:15.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.534981+0000", 2020-08-07T15:07:15.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000171373 2020-08-07T15:07:15.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:15.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.535153+0000", 2020-08-07T15:07:15.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018094879999999999 2020-08-07T15:07:15.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:15.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.536962+0000", 2020-08-07T15:07:15.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9400000000000005e-07 2020-08-07T15:07:15.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:15.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.536963+0000", 2020-08-07T15:07:15.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7188999999999997e-05 2020-08-07T15:07:15.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:15.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.537030+0000", 2020-08-07T15:07:15.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034211371999999997 2020-08-07T15:07:15.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:15.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.571241+0000", 2020-08-07T15:07:15.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15868652799999999 2020-08-07T15:07:15.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:15.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.729928+0000", 2020-08-07T15:07:15.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0139999999999999e-06 2020-08-07T15:07:15.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:15.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.729929+0000", 2020-08-07T15:07:15.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1245000000000001e-05 2020-08-07T15:07:15.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:15.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.729990+0000", 2020-08-07T15:07:15.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19863291199999999 2020-08-07T15:07:15.317 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.317 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:15.317 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.318 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:16223 3.9 3:9c72cb85:::benchmark_data_smithi074_35447_object2027:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:15.318 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:47.106937+0000", 2020-08-07T15:07:15.318 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.088493216, 2020-08-07T15:07:15.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.48645846999999998, 2020-08-07T15:07:15.319 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:15.319 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:15.319 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:15.319 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:15.319 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:15.320 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 16223 2020-08-07T15:07:15.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.320 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:15.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:15.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.106937+0000", 2020-08-07T15:07:15.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:15.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:15.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.106937+0000", 2020-08-07T15:07:15.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8390999999999998e-05 2020-08-07T15:07:15.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:15.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.106986+0000", 2020-08-07T15:07:15.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0120000000000002e-06 2020-08-07T15:07:15.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:15.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.106992+0000", 2020-08-07T15:07:15.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3148e-05 2020-08-07T15:07:15.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:15.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.107005+0000", 2020-08-07T15:07:15.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5719999999999999e-06 2020-08-07T15:07:15.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:15.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.107012+0000", 2020-08-07T15:07:15.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043495110000000003 2020-08-07T15:07:15.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:15.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.111361+0000", 2020-08-07T15:07:15.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0105999999999997e-05 2020-08-07T15:07:15.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:15.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.111421+0000", 2020-08-07T15:07:15.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012232399999999999 2020-08-07T15:07:15.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:15.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.111544+0000", 2020-08-07T15:07:15.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003099367 2020-08-07T15:07:15.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:15.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.114643+0000", 2020-08-07T15:07:15.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3099999999999997e-07 2020-08-07T15:07:15.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:15.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.114644+0000", 2020-08-07T15:07:15.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7627000000000003e-05 2020-08-07T15:07:15.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:15.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.114691+0000", 2020-08-07T15:07:15.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005765905 2020-08-07T15:07:15.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:15.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.120457+0000", 2020-08-07T15:07:15.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.47285765200000002 2020-08-07T15:07:15.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:15.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.593315+0000", 2020-08-07T15:07:15.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1969999999999999e-06 2020-08-07T15:07:15.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:15.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.593316+0000", 2020-08-07T15:07:15.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0026999999999998e-05 2020-08-07T15:07:15.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:15.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.593396+0000", 2020-08-07T15:07:15.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.48645846999999998 2020-08-07T15:07:15.336 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.336 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:15.336 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.337 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:16224 3.9 3:9c72cb85:::benchmark_data_smithi074_35447_object2027:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:15.337 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:47.108015+0000", 2020-08-07T15:07:15.337 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.087415479000001, 2020-08-07T15:07:15.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.512944332, 2020-08-07T15:07:15.337 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:15.338 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:15.338 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:15.338 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:15.338 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:15.338 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 16224 2020-08-07T15:07:15.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.339 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:15.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:15.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108015+0000", 2020-08-07T15:07:15.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:15.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:15.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108015+0000", 2020-08-07T15:07:15.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7508e-05 2020-08-07T15:07:15.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:15.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108063+0000", 2020-08-07T15:07:15.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.485e-06 2020-08-07T15:07:15.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:15.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108068+0000", 2020-08-07T15:07:15.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3387000000000001e-05 2020-08-07T15:07:15.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:15.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108082+0000", 2020-08-07T15:07:15.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1129999999999997e-06 2020-08-07T15:07:15.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:15.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108088+0000", 2020-08-07T15:07:15.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056051920000000002 2020-08-07T15:07:15.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:15.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.113693+0000", 2020-08-07T15:07:15.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9973000000000001e-05 2020-08-07T15:07:15.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:15.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.113783+0000", 2020-08-07T15:07:15.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001649 2020-08-07T15:07:15.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:15.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.113948+0000", 2020-08-07T15:07:15.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001023007 2020-08-07T15:07:15.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:15.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.114971+0000", 2020-08-07T15:07:15.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2600000000000001e-07 2020-08-07T15:07:15.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:15.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.114972+0000", 2020-08-07T15:07:15.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9228999999999998e-05 2020-08-07T15:07:15.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:15.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.115031+0000", 2020-08-07T15:07:15.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0063345550000000004 2020-08-07T15:07:15.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:15.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.121365+0000", 2020-08-07T15:07:15.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.49952866099999998 2020-08-07T15:07:15.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:15.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.620894+0000", 2020-08-07T15:07:15.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9299999999999996e-07 2020-08-07T15:07:15.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:15.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.620895+0000", 2020-08-07T15:07:15.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4503000000000002e-05 2020-08-07T15:07:15.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:15.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.620960+0000", 2020-08-07T15:07:15.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.512944332 2020-08-07T15:07:15.355 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.382 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:15.382 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.388 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:23896 3.a e28/20 3:55e62b6c:::benchmark_data_smithi074_35447_object2986:head v 28'1648, mlcod=28'1639)", 2020-08-07T15:07:15.388 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:52.053957+0000", 2020-08-07T15:07:15.388 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.141473204, 2020-08-07T15:07:15.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26202785699999998, 2020-08-07T15:07:15.389 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:15.389 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:15.389 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:15.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:15.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.053957+0000", 2020-08-07T15:07:15.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:15.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:15.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.053957+0000", 2020-08-07T15:07:15.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4489e-05 2020-08-07T15:07:15.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:15.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.054012+0000", 2020-08-07T15:07:15.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.314e-06 2020-08-07T15:07:15.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:15.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.054018+0000", 2020-08-07T15:07:15.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26134933700000001 2020-08-07T15:07:15.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:15.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.315368+0000", 2020-08-07T15:07:15.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.783e-06 2020-08-07T15:07:15.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:15.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.315371+0000", 2020-08-07T15:07:15.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0726000000000003e-05 2020-08-07T15:07:15.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:15.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.315412+0000", 2020-08-07T15:07:15.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7337999999999999e-05 2020-08-07T15:07:15.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:15.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.315429+0000", 2020-08-07T15:07:15.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051664600000000005 2020-08-07T15:07:15.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:15.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.315946+0000", 2020-08-07T15:07:15.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9224000000000003e-05 2020-08-07T15:07:15.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:15.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.315985+0000", 2020-08-07T15:07:15.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26202785699999998 2020-08-07T15:07:15.400 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.400 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:15.400 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.401 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:33728 3.1 3:8fce2265:::benchmark_data_smithi074_35447_object4215:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-08-07T15:07:15.401 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:58.156056+0000", 2020-08-07T15:07:15.401 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.039375077999999, 2020-08-07T15:07:15.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26861316299999999, 2020-08-07T15:07:15.402 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:15.402 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:15.402 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:15.402 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:15.403 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:15.403 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 33728 2020-08-07T15:07:15.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.403 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:15.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:15.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156056+0000", 2020-08-07T15:07:15.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:15.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:15.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156056+0000", 2020-08-07T15:07:15.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6795999999999998e-05 2020-08-07T15:07:15.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:15.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156092+0000", 2020-08-07T15:07:15.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4690000000000002e-06 2020-08-07T15:07:15.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:15.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156096+0000", 2020-08-07T15:07:15.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5270000000000004e-06 2020-08-07T15:07:15.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:15.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156102+0000", 2020-08-07T15:07:15.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3129999999999999e-06 2020-08-07T15:07:15.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:15.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156106+0000", 2020-08-07T15:07:15.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00337032 2020-08-07T15:07:15.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:15.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.159476+0000", 2020-08-07T15:07:15.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104874 2020-08-07T15:07:15.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:15.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.159581+0000", 2020-08-07T15:07:15.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016822300000000001 2020-08-07T15:07:15.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,3", 2020-08-07T15:07:15.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.159749+0000", 2020-08-07T15:07:15.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020156620000000001 2020-08-07T15:07:15.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:15.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.161765+0000", 2020-08-07T15:07:15.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1699999999999997e-07 2020-08-07T15:07:15.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:15.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.161765+0000", 2020-08-07T15:07:15.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6606e-05 2020-08-07T15:07:15.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:15.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.161822+0000", 2020-08-07T15:07:15.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0060344459999999997 2020-08-07T15:07:15.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:15.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.167857+0000", 2020-08-07T15:07:15.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25676037099999999 2020-08-07T15:07:15.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:15.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424617+0000", 2020-08-07T15:07:15.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3370000000000001e-06 2020-08-07T15:07:15.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:15.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424618+0000", 2020-08-07T15:07:15.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0501999999999999e-05 2020-08-07T15:07:15.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:15.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424669+0000", 2020-08-07T15:07:15.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26861316299999999 2020-08-07T15:07:15.418 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.419 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:15.419 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.420 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:33728 3.1 e30/26 ondisk, result = 0)", 2020-08-07T15:07:15.420 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:58.167159+0000", 2020-08-07T15:07:15.420 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.028272046000001, 2020-08-07T15:07:15.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25751397999999998, 2020-08-07T15:07:15.420 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:15.420 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:15.421 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:15.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:15.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.167159+0000", 2020-08-07T15:07:15.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:15.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:15.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.167159+0000", 2020-08-07T15:07:15.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3473999999999998e-05 2020-08-07T15:07:15.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:15.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.167192+0000", 2020-08-07T15:07:15.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1480000000000003e-06 2020-08-07T15:07:15.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:15.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.167198+0000", 2020-08-07T15:07:15.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25735533700000002 2020-08-07T15:07:15.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:15.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424554+0000", 2020-08-07T15:07:15.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2959999999999998e-06 2020-08-07T15:07:15.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:15.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424559+0000", 2020-08-07T15:07:15.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0877000000000002e-05 2020-08-07T15:07:15.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:15.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424600+0000", 2020-08-07T15:07:15.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5580000000000001e-06 2020-08-07T15:07:15.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:15.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424608+0000", 2020-08-07T15:07:15.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4289999999999996e-05 2020-08-07T15:07:15.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:15.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424673+0000", 2020-08-07T15:07:15.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25751397999999998 2020-08-07T15:07:15.429 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.429 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:15.430 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.430 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:34624 3.9 3:946cdb13:::benchmark_data_smithi074_35447_object4327:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-08-07T15:07:15.431 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:58.744745+0000", 2020-08-07T15:07:15.431 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.450685959000001, 2020-08-07T15:07:15.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28897188600000001, 2020-08-07T15:07:15.431 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:15.431 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:15.432 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:15.432 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:15.432 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:15.432 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 34624 2020-08-07T15:07:15.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.433 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:15.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:15.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744745+0000", 2020-08-07T15:07:15.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:15.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:15.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744745+0000", 2020-08-07T15:07:15.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1117e-05 2020-08-07T15:07:15.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:15.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744776+0000", 2020-08-07T15:07:15.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2140000000000001e-06 2020-08-07T15:07:15.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:15.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744782+0000", 2020-08-07T15:07:15.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1756999999999999e-05 2020-08-07T15:07:15.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:15.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744794+0000", 2020-08-07T15:07:15.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.674e-06 2020-08-07T15:07:15.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:15.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744800+0000", 2020-08-07T15:07:15.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012818110000000001 2020-08-07T15:07:15.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:15.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.746082+0000", 2020-08-07T15:07:15.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2163999999999997e-05 2020-08-07T15:07:15.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:15.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.746134+0000", 2020-08-07T15:07:15.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107853 2020-08-07T15:07:15.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:15.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.746242+0000", 2020-08-07T15:07:15.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00119732 2020-08-07T15:07:15.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:15.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.747440+0000", 2020-08-07T15:07:15.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5799999999999998e-07 2020-08-07T15:07:15.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:15.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.747440+0000", 2020-08-07T15:07:15.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0883000000000001e-05 2020-08-07T15:07:15.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:15.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.747481+0000", 2020-08-07T15:07:15.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000272355 2020-08-07T15:07:15.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:15.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.747754+0000", 2020-08-07T15:07:15.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28589292999999999 2020-08-07T15:07:15.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:15.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.033647+0000", 2020-08-07T15:07:15.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6600000000000005e-07 2020-08-07T15:07:15.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:15.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.033647+0000", 2020-08-07T15:07:15.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9183999999999994e-05 2020-08-07T15:07:15.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:15.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.033717+0000", 2020-08-07T15:07:15.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28897188600000001 2020-08-07T15:07:15.449 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.450 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:15.450 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.450 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:35280 3.6 e30/26 3:63cda715:::benchmark_data_smithi074_35447_object4409:head v 30'2424, mlcod=30'2420)", 2020-08-07T15:07:15.451 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:59.114249+0000", 2020-08-07T15:07:15.451 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.081181936, 2020-08-07T15:07:15.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.208357021, 2020-08-07T15:07:15.451 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:15.451 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:15.452 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:15.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:15.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.114249+0000", 2020-08-07T15:07:15.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:15.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:15.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.114249+0000", 2020-08-07T15:07:15.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5002000000000001e-05 2020-08-07T15:07:15.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:15.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.114284+0000", 2020-08-07T15:07:15.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9580000000000001e-06 2020-08-07T15:07:15.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:15.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.114288+0000", 2020-08-07T15:07:15.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20552400700000001 2020-08-07T15:07:15.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:15.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.319812+0000", 2020-08-07T15:07:15.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2610000000000002e-06 2020-08-07T15:07:15.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:15.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.319815+0000", 2020-08-07T15:07:15.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7545000000000005e-05 2020-08-07T15:07:15.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:15.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.319883+0000", 2020-08-07T15:07:15.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4219e-05 2020-08-07T15:07:15.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:15.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.319907+0000", 2020-08-07T15:07:15.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026555049999999998 2020-08-07T15:07:15.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:15.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.322562+0000", 2020-08-07T15:07:15.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3523999999999999e-05 2020-08-07T15:07:15.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:15.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.322606+0000", 2020-08-07T15:07:15.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.208357021 2020-08-07T15:07:15.465 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.465 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:15.466 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.467 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:42712 3.9 3:916738ca:::benchmark_data_smithi074_35447_object5338:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:07:15.467 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.591327+0000", 2020-08-07T15:07:15.467 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.604104143000001, 2020-08-07T15:07:15.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19416059499999999, 2020-08-07T15:07:15.469 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:15.469 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:15.470 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:15.470 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:15.470 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:15.471 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42712 2020-08-07T15:07:15.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.472 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:15.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:15.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.591327+0000", 2020-08-07T15:07:15.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:15.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:15.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.591327+0000", 2020-08-07T15:07:15.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2016e-05 2020-08-07T15:07:15.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:15.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.591359+0000", 2020-08-07T15:07:15.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9710000000000003e-06 2020-08-07T15:07:15.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:15.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.591364+0000", 2020-08-07T15:07:15.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0318e-05 2020-08-07T15:07:15.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:15.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.591374+0000", 2020-08-07T15:07:15.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4660000000000001e-06 2020-08-07T15:07:15.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:15.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.591378+0000", 2020-08-07T15:07:15.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0069575119999999999 2020-08-07T15:07:15.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:15.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.598336+0000", 2020-08-07T15:07:15.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0378999999999996e-05 2020-08-07T15:07:15.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:15.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.598426+0000", 2020-08-07T15:07:15.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014588399999999999 2020-08-07T15:07:15.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:15.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.598572+0000", 2020-08-07T15:07:15.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018089930000000001 2020-08-07T15:07:15.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:15.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.600381+0000", 2020-08-07T15:07:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000728956 2020-08-07T15:07:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.601110+0000", 2020-08-07T15:07:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18427601699999999 2020-08-07T15:07:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:15.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.785386+0000", 2020-08-07T15:07:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1200000000000001e-07 2020-08-07T15:07:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:15.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.785387+0000", 2020-08-07T15:07:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7848000000000005e-05 2020-08-07T15:07:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.785455+0000", 2020-08-07T15:07:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2323000000000002e-05 2020-08-07T15:07:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.785487+0000", 2020-08-07T15:07:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19416059499999999 2020-08-07T15:07:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:54199 3.3 3:cc40637e:::benchmark_data_smithi074_35447_object6774:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e37)", 2020-08-07T15:07:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:13.096518+0000", 2020-08-07T15:07:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 2.0989126769999999, 2020-08-07T15:07:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.242907547, 2020-08-07T15:07:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 54199 2020-08-07T15:07:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.096518+0000", 2020-08-07T15:07:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.096518+0000", 2020-08-07T15:07:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9442e-05 2020-08-07T15:07:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.096547+0000", 2020-08-07T15:07:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9689999999999996e-06 2020-08-07T15:07:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.096551+0000", 2020-08-07T15:07:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7849999999999997e-06 2020-08-07T15:07:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.096557+0000", 2020-08-07T15:07:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.475e-06 2020-08-07T15:07:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.096560+0000", 2020-08-07T15:07:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001230412 2020-08-07T15:07:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.097790+0000", 2020-08-07T15:07:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1904999999999995e-05 2020-08-07T15:07:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.097882+0000", 2020-08-07T15:07:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000137162 2020-08-07T15:07:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,6", 2020-08-07T15:07:15.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.098019+0000", 2020-08-07T15:07:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051773610000000001 2020-08-07T15:07:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:15.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.103197+0000", 2020-08-07T15:07:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8100000000000001e-07 2020-08-07T15:07:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.103197+0000", 2020-08-07T15:07:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4860000000000001e-05 2020-08-07T15:07:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.103242+0000", 2020-08-07T15:07:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14002068500000001 2020-08-07T15:07:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.243263+0000", 2020-08-07T15:07:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.096123423 2020-08-07T15:07:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.339386+0000", 2020-08-07T15:07:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9500000000000001e-07 2020-08-07T15:07:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.339387+0000", 2020-08-07T15:07:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8291999999999999e-05 2020-08-07T15:07:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.339426+0000", 2020-08-07T15:07:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.242907547 2020-08-07T15:07:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:54200 3.3 3:cc40637e:::benchmark_data_smithi074_35447_object6774:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e37)", 2020-08-07T15:07:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:13.097118+0000", 2020-08-07T15:07:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 2.098312623, 2020-08-07T15:07:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24303496599999999, 2020-08-07T15:07:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 54200 2020-08-07T15:07:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:15.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.097118+0000", 2020-08-07T15:07:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.097118+0000", 2020-08-07T15:07:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6094999999999997e-05 2020-08-07T15:07:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.097174+0000", 2020-08-07T15:07:15.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0040000000000002e-06 2020-08-07T15:07:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.097181+0000", 2020-08-07T15:07:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1990999999999999e-05 2020-08-07T15:07:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.097193+0000", 2020-08-07T15:07:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3580000000000002e-06 2020-08-07T15:07:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.097198+0000", 2020-08-07T15:07:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001099661 2020-08-07T15:07:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.098298+0000", 2020-08-07T15:07:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3878999999999997e-05 2020-08-07T15:07:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.098352+0000", 2020-08-07T15:07:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104274 2020-08-07T15:07:15.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,6", 2020-08-07T15:07:15.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.098456+0000", 2020-08-07T15:07:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0047920360000000004 2020-08-07T15:07:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:15.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.103248+0000", 2020-08-07T15:07:15.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4700000000000002e-07 2020-08-07T15:07:15.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:15.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.103249+0000", 2020-08-07T15:07:15.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8408e-05 2020-08-07T15:07:15.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:15.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.103277+0000", 2020-08-07T15:07:15.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14003026499999999 2020-08-07T15:07:15.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:15.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.243307+0000", 2020-08-07T15:07:15.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.096787082999999996 2020-08-07T15:07:15.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:15.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.340095+0000", 2020-08-07T15:07:15.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7999999999999995e-07 2020-08-07T15:07:15.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:15.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.340095+0000", 2020-08-07T15:07:15.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7785000000000003e-05 2020-08-07T15:07:15.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:15.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.340153+0000", 2020-08-07T15:07:15.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24303496599999999 2020-08-07T15:07:15.574 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.575 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:15.575 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.575 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:54199 3.3 e37/20 ondisk, result = 0)", 2020-08-07T15:07:15.575 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:13.099517+0000", 2020-08-07T15:07:15.576 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 2.095913334, 2020-08-07T15:07:15.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23991633600000001, 2020-08-07T15:07:15.576 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:15.577 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:15.577 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:15.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:15.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.099517+0000", 2020-08-07T15:07:15.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:15.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:15.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.099517+0000", 2020-08-07T15:07:15.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9927999999999999e-05 2020-08-07T15:07:15.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:15.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.099547+0000", 2020-08-07T15:07:15.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4090000000000002e-06 2020-08-07T15:07:15.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:15.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.099553+0000", 2020-08-07T15:07:15.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23978838499999999 2020-08-07T15:07:15.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:15.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.339341+0000", 2020-08-07T15:07:15.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.489e-06 2020-08-07T15:07:15.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:15.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.339345+0000", 2020-08-07T15:07:15.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0859999999999999e-05 2020-08-07T15:07:15.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:15.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.339375+0000", 2020-08-07T15:07:15.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5999999999999997e-06 2020-08-07T15:07:15.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:15.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.339381+0000", 2020-08-07T15:07:15.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2664999999999998e-05 2020-08-07T15:07:15.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:15.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.339434+0000", 2020-08-07T15:07:15.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23991633600000001 2020-08-07T15:07:15.589 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.590 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:15.590 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.590 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:54280 3.b e37/20 3:d5ab750f:::benchmark_data_smithi074_35447_object6784:head v 37'3312, mlcod=37'3305)", 2020-08-07T15:07:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:13.155226+0000", 2020-08-07T15:07:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 2.0402043129999998, 2020-08-07T15:07:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18888478, 2020-08-07T15:07:15.591 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:15.593 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.155226+0000", 2020-08-07T15:07:15.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.155226+0000", 2020-08-07T15:07:15.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5278999999999996e-05 2020-08-07T15:07:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.155292+0000", 2020-08-07T15:07:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5490000000000003e-06 2020-08-07T15:07:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.155298+0000", 2020-08-07T15:07:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18519654799999999 2020-08-07T15:07:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.340495+0000", 2020-08-07T15:07:15.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9939999999999997e-06 2020-08-07T15:07:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.340501+0000", 2020-08-07T15:07:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2743999999999994e-05 2020-08-07T15:07:15.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.340563+0000", 2020-08-07T15:07:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0432999999999999e-05 2020-08-07T15:07:15.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.340584+0000", 2020-08-07T15:07:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034756629999999999 2020-08-07T15:07:15.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.344060+0000", 2020-08-07T15:07:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1570000000000003e-05 2020-08-07T15:07:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.344111+0000", 2020-08-07T15:07:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18888478 2020-08-07T15:07:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:56592 3.9 3:9c8f6dc2:::benchmark_data_smithi074_35447_object7073:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e38)", 2020-08-07T15:07:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:14.595886+0000", 2020-08-07T15:07:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.599544839, 2020-08-07T15:07:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18857047900000001, 2020-08-07T15:07:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 56592 2020-08-07T15:07:15.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:15.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.595886+0000", 2020-08-07T15:07:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:15.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.595886+0000", 2020-08-07T15:07:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7808e-05 2020-08-07T15:07:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.595934+0000", 2020-08-07T15:07:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1539999999999999e-06 2020-08-07T15:07:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.595940+0000", 2020-08-07T15:07:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2046e-05 2020-08-07T15:07:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.595952+0000", 2020-08-07T15:07:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.587e-06 2020-08-07T15:07:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.595957+0000", 2020-08-07T15:07:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056391690000000003 2020-08-07T15:07:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:15.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.601597+0000", 2020-08-07T15:07:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3449000000000006e-05 2020-08-07T15:07:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:15.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.601690+0000", 2020-08-07T15:07:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018503500000000001 2020-08-07T15:07:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:15.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.601875+0000", 2020-08-07T15:07:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055386790000000003 2020-08-07T15:07:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:15.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.607414+0000", 2020-08-07T15:07:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9699999999999995e-07 2020-08-07T15:07:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.607415+0000", 2020-08-07T15:07:15.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9597000000000001e-05 2020-08-07T15:07:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.607454+0000", 2020-08-07T15:07:15.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023330849999999999 2020-08-07T15:07:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.609787+0000", 2020-08-07T15:07:15.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.17460308399999999 2020-08-07T15:07:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.784390+0000", 2020-08-07T15:07:15.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6299999999999993e-07 2020-08-07T15:07:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.784391+0000", 2020-08-07T15:07:15.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5025999999999997e-05 2020-08-07T15:07:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.784456+0000", 2020-08-07T15:07:15.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18857047900000001 2020-08-07T15:07:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:57000 3.d e38/35 3:b21fbb58:::benchmark_data_smithi074_35447_object7124:head v 38'3672, mlcod=38'3663)", 2020-08-07T15:07:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:14.856319+0000", 2020-08-07T15:07:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.33911127200000002, 2020-08-07T15:07:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.194414113, 2020-08-07T15:07:15.625 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.856319+0000", 2020-08-07T15:07:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.856319+0000", 2020-08-07T15:07:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7562999999999999e-05 2020-08-07T15:07:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.856357+0000", 2020-08-07T15:07:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2270000000000004e-06 2020-08-07T15:07:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.856361+0000", 2020-08-07T15:07:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19051695699999999 2020-08-07T15:07:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:15.046878+0000", 2020-08-07T15:07:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7759999999999997e-06 2020-08-07T15:07:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:15.046883+0000", 2020-08-07T15:07:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8800000000000001e-05 2020-08-07T15:07:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:15.046922+0000", 2020-08-07T15:07:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3723e-05 2020-08-07T15:07:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:15.046945+0000", 2020-08-07T15:07:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037350830000000002 2020-08-07T15:07:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:15.050681+0000", 2020-08-07T15:07:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2984e-05 2020-08-07T15:07:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:15.050734+0000", 2020-08-07T15:07:15.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.194414113 2020-08-07T15:07:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:57192 3.6 e38/26 3:6245fd63:::benchmark_data_smithi074_35447_object7148:head v 38'3800, mlcod=38'3791)", 2020-08-07T15:07:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:14.975152+0000", 2020-08-07T15:07:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.22027843, 2020-08-07T15:07:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20700897900000001, 2020-08-07T15:07:15.637 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:15.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.975152+0000", 2020-08-07T15:07:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.975152+0000", 2020-08-07T15:07:15.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1106999999999999e-05 2020-08-07T15:07:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.975183+0000", 2020-08-07T15:07:15.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.036e-06 2020-08-07T15:07:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.975187+0000", 2020-08-07T15:07:15.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.206250717 2020-08-07T15:07:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:15.181438+0000", 2020-08-07T15:07:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2640000000000002e-06 2020-08-07T15:07:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:15.181445+0000", 2020-08-07T15:07:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5849000000000002e-05 2020-08-07T15:07:15.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:15.181481+0000", 2020-08-07T15:07:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4384e-05 2020-08-07T15:07:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:15.181496+0000", 2020-08-07T15:07:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061321800000000001 2020-08-07T15:07:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:15.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:15.182109+0000", 2020-08-07T15:07:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2404000000000002e-05 2020-08-07T15:07:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:15.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:15.182161+0000", 2020-08-07T15:07:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20700897900000001 2020-08-07T15:07:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:15.648 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:15.649 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:15.651 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:07:15.652 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:07:15.652 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:07:15.652 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:07:15.652 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:07:15.652 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:07:15.653 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:07:15.653 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:07:15.653 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 39 16 57587 57571 11.5317 12.5781 0.00157939 0.0108309 2020-08-07T15:07:15.653 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:07:15.654 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:07:15.654 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:07:15.654 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:07:15.654 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:07:15.654 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:07:15.654 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:07:15.655 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:07:15.655 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:07:15.656 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:15.649+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:15.752 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:15.750+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:15.855 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:15.850+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:15.859 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:15.859 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:15.871 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:07:15.952 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:15.952+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:16.054 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:16.052+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:16.094 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:16.095 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:16.107 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:07:16.192 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:16.190+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:16.244 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:2020-08-07T15:07:16.243359+0000 min lat: 0.000913402 max lat: 1.95066 avg lat: 0.0108023 2020-08-07T15:07:16.244 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-08-07T15:07:16.245 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 40 16 59207 59191 11.5598 12.6562 0.00346488 0.0108023 2020-08-07T15:07:16.255 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:16.254+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:16.313 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:16.314 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:16.327 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:07:16.373 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:16.366+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:16.460 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:16.458+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:16.552 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:16.553 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:16.560 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:16.559+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:16.563 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:07:16.564 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:07:16.668 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:16.667+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:16.763 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:16.762+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:16.776 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:16.776 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:16.786 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:07:16.818 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:07:16.818 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:07:16.819 INFO:tasks.thrashosds.thrasher:fixing pg num pool device_health_metrics 2020-08-07T15:07:16.819 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:07:16.865 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:16.863+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:16.965 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:16.964+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:16.990 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:16.991 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:17.005 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:07:17.066 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:17.066+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:17.167 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:17.166+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:17.195 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:07:17.196 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:07:17.208 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2020-08-07T15:07:16.623229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":426696704,"num_objects":6513,"num_object_clones":0,"num_object_copies":19539,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":846,"num_objects_unfound":0,"num_objects_dirty":6513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52089,"num_write_kb":416698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9634424,"kb_used_data":1245752,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":745340296,"statfs":{"total":773094113280,"available":763228463104,"internally_reserved":8589934592,"allocated":1275650048,"data_stored":1273765111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,5,4,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":44032000,"num_objects":672,"num_object_clones":0,"num_object_copies":2016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":846,"num_objects_unfound":0,"num_objects_dirty":672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5375,"num_write_kb":43000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001161"},"pg_stats":[{"pgid":"3.f","version":"37'3552","reported_seq":"3575","reported_epoch":"37","state":"active+clean","last_fresh":"2020-08-07T15:07:11.527725+0000","last_change":"2020-08-07T15:06:35.163498+0000","last_active":"2020-08-07T15:07:11.527725+0000","last_peered":"2020-08-07T15:07:11.527725+0000","last_clean":"2020-08-07T15:07:11.527725+0000","last_became_active":"2020-08-07T15:06:35.162764+0000","last_became_peered":"2020-08-07T15:06:35.162764+0000","last_unstale":"2020-08-07T15:07:11.527725+0000","last_undegraded":"2020-08-07T15:07:11.527725+0000","last_fullsized":"2020-08-07T15:07:11.527725+0000","mapping_epoch":20,"log_start":"37'3545","ondisk_log_start":"37'3545","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":29097984,"num_objects":444,"num_object_clones":0,"num_object_copies":1332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3552,"num_write_kb":28416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"37'3296","reported_seq":"3320","reported_epoch":"37","state":"active+clean","last_fresh":"2020-08-07T15:07:11.620960+0000","last_change":"2020-08-07T15:06:35.169137+0000","last_active":"2020-08-07T15:07:11.620960+0000","last_peered":"2020-08-07T15:07:11.620960+0000","last_clean":"2020-08-07T15:07:11.620960+0000","last_became_active":"2020-08-07T15:06:35.168689+0000","last_became_peered":"2020-08-07T15:06:35.168689+0000","last_unstale":"2020-08-07T15:07:11.620960+0000","last_undegraded":"2020-08-07T15:07:11.620960+0000","last_fullsized":"2020-08-07T15:07:11.620960+0000","mapping_epoch":20,"log_start":"37'3287","ondisk_log_start":"37'3287","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27000832,"num_objects":412,"num_object_clones":0,"num_object_copies":1236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3296,"num_write_kb":26368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"37'3384","reported_seq":"3413","reported_epoch":"37","state":"active+remapped+backfill_toofull","last_fresh":"2020-08-07T15:07:11.512556+0000","last_change":"2020-08-07T15:07:08.296959+0000","last_active":"2020-08-07T15:07:11.512556+0000","last_peered":"2020-08-07T15:07:11.512556+0000","last_clean":"2020-08-07T15:07:06.228184+0000","last_became_active":"2020-08-07T15:07:08.269740+0000","last_became_peered":"2020-08-07T15:07:08.269740+0000","last_unstale":"2020-08-07T15:07:11.512556+0000","last_undegraded":"2020-08-07T15:07:11.512556+0000","last_fullsized":"2020-08-07T15:07:11.512556+0000","mapping_epoch":35,"log_start":"37'3376","ondisk_log_start":"37'3376","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27721728,"num_objects":423,"num_object_clones":0,"num_object_copies":1269,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":846,"num_objects_unfound":0,"num_objects_dirty":423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3384,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[5,6,7],"avail_no_missing":["5","6","7"],"object_location_counts":[{"shards":"5,6,7","objects":423}],"blocked_by":[],"up_primary":3,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"37'3336","reported_seq":"3360","reported_epoch":"37","state":"active+clean","last_fresh":"2020-08-07T15:07:11.628602+0000","last_change":"2020-08-07T15:06:35.167480+0000","last_active":"2020-08-07T15:07:11.628602+0000","last_peered":"2020-08-07T15:07:11.628602+0000","last_clean":"2020-08-07T15:07:11.628602+0000","last_became_active":"2020-08-07T15:06:35.167101+0000","last_became_peered":"2020-08-07T15:06:35.167101+0000","last_unstale":"2020-08-07T15:07:11.628602+0000","last_undegraded":"2020-08-07T15:07:11.628602+0000","last_fullsized":"2020-08-07T15:07:11.628602+0000","mapping_epoch":20,"log_start":"37'3329","ondisk_log_start":"37'3329","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27328512,"num_objects":417,"num_object_clones":0,"num_object_copies":1251,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3336,"num_write_kb":26688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.b","version":"37'3120","reported_seq":"3141","reported_epoch":"37","state":"active+clean","last_fresh":"2020-08-07T15:07:11.703841+0000","last_change":"2020-08-07T15:06:35.170504+0000","last_active":"2020-08-07T15:07:11.703841+0000","last_peered":"2020-08-07T15:07:11.703841+0000","last_clean":"2020-08-07T15:07:11.703841+0000","last_became_active":"2020-08-07T15:06:35.169532+0000","last_became_peered":"2020-08-07T15:06:35.169532+0000","last_unstale":"2020-08-07T15:07:11.703841+0000","last_undegraded":"2020-08-07T15:07:11.703841+0000","last_fullsized":"2020-08-07T15:07:11.703841+0000","mapping_epoch":20,"log_start":"37'3111","ondisk_log_start":"37'3111","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25559040,"num_objects":390,"num_object_clones":0,"num_object_copies":1170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3120,"num_write_kb":24960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"37'3704","reported_seq":"3727","reported_epoch":"37","state":"active+clean","last_fresh":"2020-08-07T15:07:12.658128+0000","last_change":"2020-08-07T15:06:35.170105+0000","last_active":"2020-08-07T15:07:12.658128+0000","last_peered":"2020-08-07T15:07:12.658128+0000","last_clean":"2020-08-07T15:07:12.658128+0000","last_became_active":"2020-08-07T15:06:35.169494+0000","last_became_peered":"2020-08-07T15:06:35.169494+0000","last_unstale":"2020-08-07T15:07:12.658128+0000","last_undegraded":"2020-08-07T15:07:12.658128+0000","last_fullsized":"2020-08-07T15:07:12.658128+0000","mapping_epoch":20,"log_start":"37'3696","ondisk_log_start":"37'3696","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":30343168,"num_objects":463,"num_object_clones":0,"num_object_copies":1389,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3704,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"37'2992","reported_seq":"3017","reported_epoch":"37","state":"active+clean","last_fresh":"2020-08-07T15:07:11.477871+0000","last_change":"2020-08-07T15:06:35.163109+0000","last_active":"2020-08-07T15:07:11.477871+0000","last_peered":"2020-08-07T15:07:11.477871+0000","last_clean":"2020-08-07T15:07:11.477871+0000","last_became_active":"2020-08-07T15:06:35.162721+0000","last_became_peered":"2020-08-07T15:06:35.162721+0000","last_unstale":"2020-08-07T15:07:11.477871+0000","last_undegraded":"2020-08-07T15:07:11.477871+0000","last_fullsized":"2020-08-07T15:07:11.477871+0000","mapping_epoch":20,"log_start":"37'2986","ondisk_log_start":"37'2986","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24510464,"num_objects":374,"num_object_clones":0,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2992,"num_write_kb":23936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"37'3288","reported_seq":"3313","reported_epoch":"37","state":"active+clean","last_fresh":"2020-08-07T15:07:11.703376+0000","last_change":"2020-08-07T15:06:35.173504+0000","last_active":"2020-08-07T15:07:11.703376+0000","last_peered":"2020-08-07T15:07:11.703376+0000","last_clean":"2020-08-07T15:07:11.703376+0000","last_became_active":"2020-08-07T15:06:35.172437+0000","last_became_peered":"2020-08-07T15:06:35.172437+0000","last_unstale":"2020-08-07T15:07:11.703376+0000","last_undegraded":"2020-08-07T15:07:11.703376+0000","last_fullsized":"2020-08-07T15:07:11.703376+0000","mapping_epoch":20,"log_start":"37'3281","ondisk_log_start":"37'3281","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":26935296,"num_objects":411,"num_object_clones":0,"num_object_copies":1233,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3288,"num_write_kb":26304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"25","reported_epoch":"36","state":"active+clean","last_fresh":"2020-08-07T15:07:08.140437+0000","last_change":"2020-08-07T15:06:46.199001+0000","last_active":"2020-08-07T15:07:08.140437+0000","last_peered":"2020-08-07T15:07:08.140437+0000","last_clean":"2020-08-07T15:07:08.140437+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:07:08.140437+0000","last_undegraded":"2020-08-07T15:07:08.140437+0000","last_fullsized":"2020-08-07T15:07:08.140437+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.5","version":"37'3024","reported_seq":"3047","reported_epoch":"37","state":"active+clean","last_fresh":"2020-08-07T15:07:11.682693+0000","last_change":"2020-08-07T15:06:35.169307+0000","last_active":"2020-08-07T15:07:11.682693+0000","last_peered":"2020-08-07T15:07:11.682693+0000","last_clean":"2020-08-07T15:07:11.682693+0000","last_became_active":"2020-08-07T15:06:35.168887+0000","last_became_peered":"2020-08-07T15:06:35.168887+0000","last_unstale":"2020-08-07T15:07:11.682693+0000","last_undegraded":"2020-08-07T15:07:11.682693+0000","last_fullsized":"2020-08-07T15:07:11.682693+0000","mapping_epoch":20,"log_start":"37'3003","ondisk_log_start":"37'3003","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24772608,"num_objects":378,"num_object_clones":0,"num_object_copies":1134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3024,"num_write_kb":24192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"28","reported_epoch":"36","state":"active+clean","last_fresh":"2020-08-07T15:07:08.141204+0000","last_change":"2020-08-07T15:06:23.074962+0000","last_active":"2020-08-07T15:07:08.141204+0000","last_peered":"2020-08-07T15:07:08.141204+0000","last_clean":"2020-08-07T15:07:08.141204+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:07:08.141204+0000","last_undegraded":"2020-08-07T15:07:08.141204+0000","last_fullsized":"2020-08-07T15:07:08.141204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"37'3096","reported_seq":"3121","reported_epoch":"37","state":"active+clean","last_fresh":"2020-08-07T15:07:11.588092+0000","last_change":"2020-08-07T15:06:35.164593+0000","last_active":"2020-08-07T15:07:11.588092+0000","last_peered":"2020-08-07T15:07:11.588092+0000","last_clean":"2020-08-07T15:07:11.588092+0000","last_became_active":"2020-08-07T15:06:35.164108+0000","last_became_peered":"2020-08-07T15:06:35.164108+0000","last_unstale":"2020-08-07T15:07:11.588092+0000","last_undegraded":"2020-08-07T15:07:11.588092+0000","last_fullsized":"2020-08-07T15:07:11.588092+0000","mapping_epoch":20,"log_start":"37'3089","ondisk_log_start":"37'3089","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25362432,"num_objects":387,"num_object_clones":0,"num_object_copies":1161,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3096,"num_write_kb":24768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"28","reported_epoch":"36","state":"active+clean","last_fresh":"2020-08-07T15:07:08.254563+0000","last_change":"2020-08-07T15:06:23.223924+0000","last_active":"2020-08-07T15:07:08.254563+0000","last_peered":"2020-08-07T15:07:08.254563+0000","last_clean":"2020-08-07T15:07:08.254563+0000","last_became_active":"2020-08-07T15:06:23.223489+0000","last_became_peered":"2020-08-07T15:06:23.223489+0000","last_unstale":"2020-08-07T15:07:08.254563+0000","last_undegraded":"2020-08-07T15:07:08.254563+0000","last_fullsized":"2020-08-07T15:07:08.254563+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"37'3288","reported_seq":"3313","reported_epoch":"37","state":"active+clean","last_fresh":"2020-08-07T15:07:11.642182+0000","last_change":"2020-08-07T15:06:35.170553+0000","last_active":"2020-08-07T15:07:11.642182+0000","last_peered":"2020-08-07T15:07:11.642182+0000","last_clean":"2020-08-07T15:07:11.642182+0000","last_became_active":"2020-08-07T15:06:35.170037+0000","last_became_peered":"2020-08-07T15:06:35.170037+0000","last_unstale":"2020-08-07T15:07:11.642182+0000","last_undegraded":"2020-08-07T15:07:11.642182+0000","last_fullsized":"2020-08-07T15:07:11.642182+0000","mapping_epoch":20,"log_start":"37'3281","ondisk_log_start":"37'3281","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":26935296,"num_objects":411,"num_object_clones":0,"num_object_copies":1233,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3288,"num_write_kb":26304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"28","reported_epoch":"36","state":"active+clean","last_fresh":"2020-08-07T15:07:08.141219+0000","last_change":"2020-08-07T15:06:23.075958+0000","last_active":"2020-08-07T15:07:08.141219+0000","last_peered":"2020-08-07T15:07:08.141219+0000","last_clean":"2020-08-07T15:07:08.141219+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:07:08.141219+0000","last_undegraded":"2020-08-07T15:07:08.141219+0000","last_fullsized":"2020-08-07T15:07:08.141219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"35","reported_epoch":"36","state":"active+clean","last_fresh":"2020-08-07T15:07:08.140288+0000","last_change":"2020-08-07T15:06:19.198215+0000","last_active":"2020-08-07T15:07:08.140288+0000","last_peered":"2020-08-07T15:07:08.140288+0000","last_clean":"2020-08-07T15:07:08.140288+0000","last_became_active":"2020-08-07T15:06:19.197819+0000","last_became_peered":"2020-08-07T15:06:19.197819+0000","last_unstale":"2020-08-07T15:07:08.140288+0000","last_undegraded":"2020-08-07T15:07:08.140288+0000","last_fullsized":"2020-08-07T15:07:08.140288+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:06:10.878833+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"37'3072","reported_seq":"3096","reported_epoch":"37","state":"active+clean","last_fresh":"2020-08-07T15:07:11.693553+0000","last_change":"2020-08-07T15:06:35.169888+0000","last_active":"2020-08-07T15:07:11.693553+0000","last_peered":"2020-08-07T15:07:11.693553+0000","last_clean":"2020-08-07T15:07:11.693553+0000","last_became_active":"2020-08-07T15:06:35.169385+0000","last_became_peered":"2020-08-07T15:06:35.169385+0000","last_unstale":"2020-08-07T15:07:11.693553+0000","last_undegraded":"2020-08-07T15:07:11.693553+0000","last_fullsized":"2020-08-07T15:07:11.693553+0000","mapping_epoch":20,"log_start":"37'3065","ondisk_log_start":"37'3065","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165824,"num_objects":384,"num_object_clones":0,"num_object_copies":1152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3072,"num_write_kb":24576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"28","reported_epoch":"36","state":"active+clean","last_fresh":"2020-08-07T15:07:08.140343+0000","last_change":"2020-08-07T15:06:23.224115+0000","last_active":"2020-08-07T15:07:08.140343+0000","last_peered":"2020-08-07T15:07:08.140343+0000","last_clean":"2020-08-07T15:07:08.140343+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:07:08.140343+0000","last_undegraded":"2020-08-07T15:07:08.140343+0000","last_fullsized":"2020-08-07T15:07:08.140343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"37'3120","reported_seq":"3146","reported_epoch":"37","state":"active+clean","last_fresh":"2020-08-07T15:07:11.652863+0000","last_change":"2020-08-07T15:06:46.214618+0000","last_active":"2020-08-07T15:07:11.652863+0000","last_peered":"2020-08-07T15:07:11.652863+0000","last_clean":"2020-08-07T15:07:11.652863+0000","last_became_active":"2020-08-07T15:06:46.202192+0000","last_became_peered":"2020-08-07T15:06:46.202192+0000","last_unstale":"2020-08-07T15:07:11.652863+0000","last_undegraded":"2020-08-07T15:07:11.652863+0000","last_fullsized":"2020-08-07T15:07:11.652863+0000","mapping_epoch":26,"log_start":"37'3113","ondisk_log_start":"37'3113","created":20,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25559040,"num_objects":390,"num_object_clones":0,"num_object_copies":1170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3120,"num_write_kb":24960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3],"acting":[4,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"28","reported_epoch":"36","state":"active+clean","last_fresh":"2020-08-07T15:07:08.140387+0000","last_change":"2020-08-07T15:06:23.225886+0000","last_active":"2020-08-07T15:07:08.140387+0000","last_peered":"2020-08-07T15:07:08.140387+0000","last_clean":"2020-08-07T15:07:08.140387+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:07:08.140387+0000","last_undegraded":"2020-08-07T15:07:08.140387+0000","last_fullsized":"2020-08-07T15:07:08.140387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"37'3264","reported_seq":"3289","reported_epoch":"37","state":"active+clean","last_fresh":"2020-08-07T15:07:11.650375+0000","last_change":"2020-08-07T15:06:35.169626+0000","last_active":"2020-08-07T15:07:11.650375+0000","last_peered":"2020-08-07T15:07:11.650375+0000","last_clean":"2020-08-07T15:07:11.650375+0000","last_became_active":"2020-08-07T15:06:35.169127+0000","last_became_peered":"2020-08-07T15:06:35.169127+0000","last_unstale":"2020-08-07T15:07:11.650375+0000","last_undegraded":"2020-08-07T15:07:11.650375+0000","last_fullsized":"2020-08-07T15:07:11.650375+0000","mapping_epoch":20,"log_start":"37'3254","ondisk_log_start":"37'3254","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":26738688,"num_objects":408,"num_object_clones":0,"num_object_copies":1224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3264,"num_write_kb":26112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"37'3168","reported_seq":"3193","reported_epoch":"37","state":"active+clean","last_fresh":"2020-08-07T15:07:11.431793+0000","last_change":"2020-08-07T15:06:35.172780+0000","last_active":"2020-08-07T15:07:11.431793+0000","last_peered":"2020-08-07T15:07:11.431793+0000","last_clean":"2020-08-07T15:07:11.431793+0000","last_became_active":"2020-08-07T15:06:35.172360+0000","last_became_peered":"2020-08-07T15:06:35.172360+0000","last_unstale":"2020-08-07T15:07:11.431793+0000","last_undegraded":"2020-08-07T15:07:11.431793+0000","last_fullsized":"2020-08-07T15:07:11.431793+0000","mapping_epoch":20,"log_start":"37'3161","ondisk_log_start":"37'3161","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25952256,"num_objects":396,"num_object_clones":0,"num_object_copies":1188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3168,"num_write_kb":25344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"28","reported_epoch":"36","state":"active+clean","last_fresh":"2020-08-07T15:07:08.255062+0000","last_change":"2020-08-07T15:06:23.074076+0000","last_active":"2020-08-07T15:07:08.255062+0000","last_peered":"2020-08-07T15:07:08.255062+0000","last_clean":"2020-08-07T15:07:08.255062+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:07:08.255062+0000","last_undegraded":"2020-08-07T15:07:08.255062+0000","last_fullsized":"2020-08-07T15:07:08.255062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"28","reported_epoch":"36","state":"active+clean","last_fresh":"2020-08-07T15:07:08.139636+0000","last_change":"2020-08-07T15:06:23.073702+0000","last_active":"2020-08-07T15:07:08.139636+0000","last_peered":"2020-08-07T15:07:08.139636+0000","last_clean":"2020-08-07T15:07:08.139636+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:07:08.139636+0000","last_undegraded":"2020-08-07T15:07:08.139636+0000","last_fullsized":"2020-08-07T15:07:08.139636+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"37'3383","reported_seq":"3404","reported_epoch":"37","state":"active+clean","last_fresh":"2020-08-07T15:07:11.704391+0000","last_change":"2020-08-07T15:06:46.199635+0000","last_active":"2020-08-07T15:07:11.704391+0000","last_peered":"2020-08-07T15:07:11.704391+0000","last_clean":"2020-08-07T15:07:11.704391+0000","last_became_active":"2020-08-07T15:06:46.199201+0000","last_became_peered":"2020-08-07T15:06:46.199201+0000","last_unstale":"2020-08-07T15:07:11.704391+0000","last_undegraded":"2020-08-07T15:07:11.704391+0000","last_fullsized":"2020-08-07T15:07:11.704391+0000","mapping_epoch":26,"log_start":"37'3376","ondisk_log_start":"37'3376","created":20,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27713536,"num_objects":423,"num_object_clones":0,"num_object_copies":1269,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3383,"num_write_kb":27064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":426696704,"num_objects":6511,"num_object_clones":0,"num_object_copies":19533,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":846,"num_objects_unfound":0,"num_objects_dirty":6511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52087,"num_write_kb":416696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1283571712,"data_stored":1283571712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542157,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1207480,"kb_used_data":158896,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93164360,"statfs":{"total":96636764160,"available":95400304640,"internally_reserved":1073741824,"allocated":162709504,"data_stored":162470848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.51}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5880000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.538}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4099999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1205776,"kb_used_data":157192,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93166064,"statfs":{"total":96636764160,"available":95402049536,"internally_reserved":1073741824,"allocated":160964608,"data_stored":160725952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,4,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2679999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8640000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8939999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.137}]}]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1209396,"kb_used_data":160812,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93162444,"statfs":{"total":96636764160,"available":95398342656,"internally_reserved":1073741824,"allocated":164671488,"data_stored":164436100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4419999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8460000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.234}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2810000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1203708,"kb_used_data":155124,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93168132,"statfs":{"total":96636764160,"available":95404167168,"internally_reserved":1073741824,"allocated":158846976,"data_stored":158611838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0209999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":124.738}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1149488,"kb_used_data":100904,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93222352,"statfs":{"total":96636764160,"available":95459688448,"internally_reserved":1073741824,"allocated":103325696,"data_stored":103093826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6439999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.496}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7190000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.581}]}]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1225276,"kb_used_data":176692,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93146564,"statfs":{"total":96636764160,"available":95382081536,"internally_reserved":1073741824,"allocated":180932608,"data_stored":180697220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2709999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3740000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1257824,"kb_used_data":209240,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93114016,"statfs":{"total":96636764160,"available":95348752384,"internally_reserved":1073741824,"allocated":214261760,"data_stored":214023104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0470000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.923}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1549999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8889999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.173}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3540000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1175476,"kb_used_data":126892,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93196364,"statfs":{"total":96636764160,"available":95433076736,"internally_reserved":1073741824,"allocated":129937408,"data_stored":129706223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7649999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7130000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.825}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]}]}]}} 2020-08-07T15:07:17.210 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set device_health_metrics pgp_num 1 2020-08-07T15:07:17.221 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:17.222 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:17.233 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:07:17.243 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 41 16 60416 60400 11.5082 9.44531 0.00223354 0.0108554 2020-08-07T15:07:17.275 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:17.273+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:17.379 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:17.371+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:17.476 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:17.477 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:17.478 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:07:17.482 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:07:17.482 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:17.480+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:17.576 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:17.574+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:17.656 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:07:17.666 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:17.667 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:17.677 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:07:17.703 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:17.702+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:17.777 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:17.776+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:17.865 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:07:17.878 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:17.877+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:17.880 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:17.880 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:07:17.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.881 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:61403 3.7 e40/20 3:e3971a21:::benchmark_data_smithi074_35447_object7675:head v 40'3691, mlcod=40'3687)", 2020-08-07T15:07:17.881 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:17.873539+0000", 2020-08-07T15:07:17.881 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0060826150000000004, 2020-08-07T15:07:17.881 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0061108229999999996, 2020-08-07T15:07:17.881 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:17.882 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:17.882 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:17.882 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.882 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:17.882 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.873539+0000", 2020-08-07T15:07:17.883 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:17.883 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:17.883 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.883 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:17.883 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.873539+0000", 2020-08-07T15:07:17.883 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3990000000000002e-05 2020-08-07T15:07:17.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:17.884 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.884 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:17.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.873603+0000", 2020-08-07T15:07:17.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0269999999999995e-06 2020-08-07T15:07:17.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:17.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:17.885 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.873611+0000", 2020-08-07T15:07:17.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00085086799999999996 2020-08-07T15:07:17.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:17.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:17.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.874462+0000", 2020-08-07T15:07:17.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5889999999999999e-06 2020-08-07T15:07:17.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:17.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.890 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:17.890 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.874469+0000", 2020-08-07T15:07:17.890 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000133017 2020-08-07T15:07:17.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:17.891 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.891 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:17.891 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.874602+0000", 2020-08-07T15:07:17.891 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1367000000000001e-05 2020-08-07T15:07:17.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:17.892 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.892 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:17.892 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.874623+0000", 2020-08-07T15:07:17.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001083858 2020-08-07T15:07:17.892 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:17.893 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:17.893 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:17.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:17.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.893 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:61404 3.7 e40/20 3:e3971a21:::benchmark_data_smithi074_35447_object7675:head v 40'3692, mlcod=40'3687)", 2020-08-07T15:07:17.894 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:17.874378+0000", 2020-08-07T15:07:17.894 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0052434250000000003, 2020-08-07T15:07:17.894 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0053270219999999998, 2020-08-07T15:07:17.894 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:17.894 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:17.895 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:17.895 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.895 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:17.895 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.874378+0000", 2020-08-07T15:07:17.895 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:17.895 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:17.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.896 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:17.896 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.874378+0000", 2020-08-07T15:07:17.897 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0547999999999998e-05 2020-08-07T15:07:17.897 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:17.897 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.897 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:17.897 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.874429+0000", 2020-08-07T15:07:17.898 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4749999999999998e-06 2020-08-07T15:07:17.898 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:17.898 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.898 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:17.898 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.874435+0000", 2020-08-07T15:07:17.899 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00074983199999999997 2020-08-07T15:07:17.899 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:17.899 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.899 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:17.899 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.875185+0000", 2020-08-07T15:07:17.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.763e-06 2020-08-07T15:07:17.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:17.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.900 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:17.900 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.875190+0000", 2020-08-07T15:07:17.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8388999999999999e-05 2020-08-07T15:07:17.901 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:17.901 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.901 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:17.901 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.875258+0000", 2020-08-07T15:07:17.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4698999999999998e-05 2020-08-07T15:07:17.902 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:17.902 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.902 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:17.902 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.875283+0000", 2020-08-07T15:07:17.902 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00090470600000000002 2020-08-07T15:07:17.903 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:17.903 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:17.903 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:17.903 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:17.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.903 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:61405 3.7 e40/20 3:e3971a21:::benchmark_data_smithi074_35447_object7675:head v 40'3693, mlcod=40'3687)", 2020-08-07T15:07:17.904 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:17.875139+0000", 2020-08-07T15:07:17.904 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0044825780000000001, 2020-08-07T15:07:17.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0046009049999999998, 2020-08-07T15:07:17.904 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:17.904 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:17.905 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:17.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:17.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.875139+0000", 2020-08-07T15:07:17.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:17.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:17.906 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.906 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:17.906 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.875139+0000", 2020-08-07T15:07:17.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6936000000000001e-05 2020-08-07T15:07:17.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:17.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:17.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.875166+0000", 2020-08-07T15:07:17.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.143e-06 2020-08-07T15:07:17.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:17.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.908 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:17.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.875170+0000", 2020-08-07T15:07:17.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023095339999999998 2020-08-07T15:07:17.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:17.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:17.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.877480+0000", 2020-08-07T15:07:17.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7449999999999995e-06 2020-08-07T15:07:17.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:17.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.910 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:17.910 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.877488+0000", 2020-08-07T15:07:17.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0840000000000002e-05 2020-08-07T15:07:17.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:17.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:17.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.877528+0000", 2020-08-07T15:07:17.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4843e-05 2020-08-07T15:07:17.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:17.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:17.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:17.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.877543+0000", 2020-08-07T15:07:17.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002404041 2020-08-07T15:07:17.913 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:17.913 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:17.913 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:17.913 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:17.913 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:07:17.914 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 3 2020-08-07T15:07:17.914 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:17.915 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:07:17.916 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:07:17.929 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:07:17.971 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:07:17.978 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:17.977+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:18.004 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:07:18.047 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:07:18.080 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:18.079+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:18.084 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:07:18.106 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:18.108 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:07:18.108 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:07:18.108 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:07:18.108 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:18.125 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:07:18.125 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:07:18.126 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:07:18.126 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:07:18.126 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:07:18.126 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:07:18.126 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:07:18.127 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:07:18.127 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:07:18.127 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:07:18.182 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:18.181+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:18.243 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 42 16 61988 61972 11.5266 12.2812 0.00327511 0.0108402 2020-08-07T15:07:18.284 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:18.282+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:18.351 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:18.351 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:07:18.352 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:07:18.352 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:07:18.352 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.352 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:9416 3.6 3:6d5d7913:::benchmark_data_smithi074_35447_object1176:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:18.352 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.922485+0000", 2020-08-07T15:07:18.353 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 36.425894290999999, 2020-08-07T15:07:18.353 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16975183499999999, 2020-08-07T15:07:18.353 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:18.353 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:18.353 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:18.354 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:18.354 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:18.354 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 9416 2020-08-07T15:07:18.354 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.354 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:18.355 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.355 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:18.358 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922485+0000", 2020-08-07T15:07:18.359 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:18.359 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.359 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.359 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:18.359 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922485+0000", 2020-08-07T15:07:18.360 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6072999999999998e-05 2020-08-07T15:07:18.360 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.360 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.360 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:18.360 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922521+0000", 2020-08-07T15:07:18.361 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.241e-06 2020-08-07T15:07:18.361 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.362 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.363 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:18.363 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922527+0000", 2020-08-07T15:07:18.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2075e-05 2020-08-07T15:07:18.363 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:18.364 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922540+0000", 2020-08-07T15:07:18.364 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0589999999999999e-06 2020-08-07T15:07:18.364 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.365 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.365 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:18.365 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922546+0000", 2020-08-07T15:07:18.366 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00063213199999999998 2020-08-07T15:07:18.366 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.367 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:18.369 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.923178+0000", 2020-08-07T15:07:18.369 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9913e-05 2020-08-07T15:07:18.370 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.370 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.370 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:18.370 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.923238+0000", 2020-08-07T15:07:18.370 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010683099999999999 2020-08-07T15:07:18.372 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.373 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.373 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:18.373 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.923344+0000", 2020-08-07T15:07:18.373 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004139663 2020-08-07T15:07:18.374 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.374 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.374 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:18.374 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.927484+0000", 2020-08-07T15:07:18.374 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001063149 2020-08-07T15:07:18.375 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.375 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:18.375 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.928547+0000", 2020-08-07T15:07:18.375 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16358086099999999 2020-08-07T15:07:18.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:18.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092128+0000", 2020-08-07T15:07:18.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3699999999999999e-07 2020-08-07T15:07:18.377 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:18.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092129+0000", 2020-08-07T15:07:18.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3859e-05 2020-08-07T15:07:18.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:18.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092193+0000", 2020-08-07T15:07:18.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4141999999999999e-05 2020-08-07T15:07:18.379 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:18.380 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092237+0000", 2020-08-07T15:07:18.380 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16975183499999999 2020-08-07T15:07:18.380 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.380 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:18.381 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.381 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:9416 3.6 e24/20 ondisk, result = 0)", 2020-08-07T15:07:18.381 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.926716+0000", 2020-08-07T15:07:18.382 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 36.421663897999998, 2020-08-07T15:07:18.382 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16552591899999999, 2020-08-07T15:07:18.382 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:18.382 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:18.382 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:18.383 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.383 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:18.383 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.926716+0000", 2020-08-07T15:07:18.383 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:18.383 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.384 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.384 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:18.384 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.926716+0000", 2020-08-07T15:07:18.384 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9971000000000002e-05 2020-08-07T15:07:18.384 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.384 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.385 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:18.385 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.926745+0000", 2020-08-07T15:07:18.385 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.51e-06 2020-08-07T15:07:18.385 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.385 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.386 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:18.386 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.926750+0000", 2020-08-07T15:07:18.386 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.165288254 2020-08-07T15:07:18.386 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.386 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.387 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:18.387 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092038+0000", 2020-08-07T15:07:18.387 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1109999999999999e-06 2020-08-07T15:07:18.387 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.387 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.387 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:18.388 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092043+0000", 2020-08-07T15:07:18.388 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7359000000000003e-05 2020-08-07T15:07:18.388 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.389 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.389 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:18.389 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092111+0000", 2020-08-07T15:07:18.389 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6889999999999993e-06 2020-08-07T15:07:18.389 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.390 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.390 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:18.390 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092119+0000", 2020-08-07T15:07:18.390 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000122025 2020-08-07T15:07:18.390 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.391 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.391 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:18.391 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092241+0000", 2020-08-07T15:07:18.391 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16552591899999999 2020-08-07T15:07:18.391 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.392 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:18.392 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.392 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.392 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.392 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:9688 3.6 3:667d8bb7:::benchmark_data_smithi074_35447_object1210:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:18.393 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.094625+0000", 2020-08-07T15:07:18.393 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 36.25375476, 2020-08-07T15:07:18.393 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14221277900000001, 2020-08-07T15:07:18.393 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:18.393 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:18.394 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:18.394 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:18.394 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:18.394 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 9688 2020-08-07T15:07:18.394 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.395 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:18.395 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.395 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:18.395 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094625+0000", 2020-08-07T15:07:18.395 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:18.395 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.396 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.396 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:18.396 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094625+0000", 2020-08-07T15:07:18.396 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4469000000000003e-05 2020-08-07T15:07:18.396 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.397 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.397 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:18.397 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094679+0000", 2020-08-07T15:07:18.397 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.201e-05 2020-08-07T15:07:18.397 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.398 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.398 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:18.398 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094691+0000", 2020-08-07T15:07:18.398 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0873000000000001e-05 2020-08-07T15:07:18.398 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.398 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.399 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:18.399 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094702+0000", 2020-08-07T15:07:18.399 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9599999999999999e-06 2020-08-07T15:07:18.399 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.400 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:18.400 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.094707+0000", 2020-08-07T15:07:18.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017070119999999999 2020-08-07T15:07:18.400 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.401 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:18.401 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.096414+0000", 2020-08-07T15:07:18.401 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3101000000000002e-05 2020-08-07T15:07:18.401 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.401 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.402 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:18.402 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.096507+0000", 2020-08-07T15:07:18.402 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017744399999999999 2020-08-07T15:07:18.402 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.402 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.402 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:18.403 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.096685+0000", 2020-08-07T15:07:18.403 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017538250000000001 2020-08-07T15:07:18.403 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.403 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.403 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:18.404 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.098438+0000", 2020-08-07T15:07:18.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.82e-07 2020-08-07T15:07:18.404 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.404 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.404 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:18.405 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.098439+0000", 2020-08-07T15:07:18.405 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3258999999999999e-05 2020-08-07T15:07:18.405 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.405 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:18.405 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.098492+0000", 2020-08-07T15:07:18.406 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001822908 2020-08-07T15:07:18.406 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.406 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.406 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:18.406 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.100315+0000", 2020-08-07T15:07:18.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13645548299999999 2020-08-07T15:07:18.407 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.407 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.407 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:18.407 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.236771+0000", 2020-08-07T15:07:18.408 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7999999999999996e-07 2020-08-07T15:07:18.408 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.408 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.408 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:18.408 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.236771+0000", 2020-08-07T15:07:18.409 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6372999999999996e-05 2020-08-07T15:07:18.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.409 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.409 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:18.409 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.236837+0000", 2020-08-07T15:07:18.410 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14221277900000001 2020-08-07T15:07:18.410 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.410 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:18.410 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.410 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.411 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.411 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:10240 3.6 3:665d71ff:::benchmark_data_smithi074_35447_object1279:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:18.411 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:07:18.411 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 35.955160460999998, 2020-08-07T15:07:18.411 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25191569699999999, 2020-08-07T15:07:18.411 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:18.412 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:18.412 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:18.412 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:18.412 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:18.412 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 10240 2020-08-07T15:07:18.413 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.413 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:18.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.413 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:18.413 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:07:18.414 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:18.414 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.414 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.414 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:18.414 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:07:18.414 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5463000000000002e-05 2020-08-07T15:07:18.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.415 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:18.415 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393264+0000", 2020-08-07T15:07:18.415 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9219999999999999e-06 2020-08-07T15:07:18.416 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.416 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.416 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:18.416 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393268+0000", 2020-08-07T15:07:18.416 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6149999999999997e-06 2020-08-07T15:07:18.417 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.417 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.417 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:18.417 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393277+0000", 2020-08-07T15:07:18.417 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2039999999999996e-06 2020-08-07T15:07:18.417 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.418 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.418 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:18.418 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393282+0000", 2020-08-07T15:07:18.418 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001225243 2020-08-07T15:07:18.418 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.419 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.419 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:18.419 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394507+0000", 2020-08-07T15:07:18.419 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7108999999999995e-05 2020-08-07T15:07:18.419 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.420 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.420 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:18.421 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394605+0000", 2020-08-07T15:07:18.422 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017490800000000001 2020-08-07T15:07:18.422 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.422 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:18.422 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394779+0000", 2020-08-07T15:07:18.423 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032608699999999999 2020-08-07T15:07:18.423 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.423 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.423 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:18.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.398040+0000", 2020-08-07T15:07:18.424 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013015259999999999 2020-08-07T15:07:18.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.424 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.424 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:18.424 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.399342+0000", 2020-08-07T15:07:18.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24566893400000001 2020-08-07T15:07:18.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.425 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:18.425 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645011+0000", 2020-08-07T15:07:18.426 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.217e-06 2020-08-07T15:07:18.435 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.435 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.435 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:18.436 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645012+0000", 2020-08-07T15:07:18.436 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0803000000000006e-05 2020-08-07T15:07:18.436 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.436 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.436 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:18.437 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645083+0000", 2020-08-07T15:07:18.437 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1882999999999998e-05 2020-08-07T15:07:18.437 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.437 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.437 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:18.438 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645135+0000", 2020-08-07T15:07:18.438 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25191569699999999 2020-08-07T15:07:18.438 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.438 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:18.438 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.438 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.439 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.439 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:16640 3.6 e27/26 3:63b550b6:::benchmark_data_smithi074_35447_object2079:head v 27'1104, mlcod=27'1096)", 2020-08-07T15:07:18.439 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.390454+0000", 2020-08-07T15:07:18.439 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 30.957924978000001, 2020-08-07T15:07:18.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.144390454, 2020-08-07T15:07:18.440 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:18.440 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:18.440 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:18.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.440 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:18.441 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.390454+0000", 2020-08-07T15:07:18.441 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:18.441 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.441 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.441 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:18.442 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.390454+0000", 2020-08-07T15:07:18.442 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0198999999999998e-05 2020-08-07T15:07:18.442 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.442 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.442 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:18.442 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.390505+0000", 2020-08-07T15:07:18.443 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7270000000000001e-06 2020-08-07T15:07:18.443 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.443 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.443 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:18.443 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.390511+0000", 2020-08-07T15:07:18.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14174463700000001 2020-08-07T15:07:18.444 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.444 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:18.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.532256+0000", 2020-08-07T15:07:18.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0260000000000002e-06 2020-08-07T15:07:18.445 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.445 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.445 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:18.445 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.532263+0000", 2020-08-07T15:07:18.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6328000000000001e-05 2020-08-07T15:07:18.446 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.446 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.446 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:18.446 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.532309+0000", 2020-08-07T15:07:18.446 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4360000000000001e-05 2020-08-07T15:07:18.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.447 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:18.447 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.532334+0000", 2020-08-07T15:07:18.447 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024563200000000001 2020-08-07T15:07:18.448 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:18.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.534790+0000", 2020-08-07T15:07:18.448 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4857e-05 2020-08-07T15:07:18.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.449 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.449 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:18.449 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.534845+0000", 2020-08-07T15:07:18.449 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.144390454 2020-08-07T15:07:18.450 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.450 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:18.450 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.451 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:24047 3.1 e28/26 3:87d059ba:::benchmark_data_smithi074_35447_object3005:head v 28'1439, mlcod=28'1429)", 2020-08-07T15:07:18.451 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.187365+0000", 2020-08-07T15:07:18.451 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 26.161014083000001, 2020-08-07T15:07:18.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14839102400000001, 2020-08-07T15:07:18.451 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:18.451 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:18.452 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:18.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:18.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.187365+0000", 2020-08-07T15:07:18.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:18.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.453 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:18.453 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.187365+0000", 2020-08-07T15:07:18.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4627999999999999e-05 2020-08-07T15:07:18.454 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.454 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.454 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:18.454 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.187420+0000", 2020-08-07T15:07:18.454 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5010000000000001e-06 2020-08-07T15:07:18.454 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:18.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.187426+0000", 2020-08-07T15:07:18.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14725048399999999 2020-08-07T15:07:18.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:18.456 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.334677+0000", 2020-08-07T15:07:18.456 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.066e-06 2020-08-07T15:07:18.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:18.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.334683+0000", 2020-08-07T15:07:18.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4073000000000003e-05 2020-08-07T15:07:18.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.458 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:18.458 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.334727+0000", 2020-08-07T15:07:18.458 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2813999999999999e-05 2020-08-07T15:07:18.458 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.458 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:18.459 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.334750+0000", 2020-08-07T15:07:18.459 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00097284100000000001 2020-08-07T15:07:18.459 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:18.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.335723+0000", 2020-08-07T15:07:18.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3617e-05 2020-08-07T15:07:18.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.460 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.460 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:18.461 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.335756+0000", 2020-08-07T15:07:18.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14839102400000001 2020-08-07T15:07:18.461 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.461 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:18.461 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.462 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.462 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.462 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:24048 3.1 e28/26 3:87d059ba:::benchmark_data_smithi074_35447_object3005:head v 28'1440, mlcod=28'1429)", 2020-08-07T15:07:18.462 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.187973+0000", 2020-08-07T15:07:18.462 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 26.160405954000002, 2020-08-07T15:07:18.462 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16042247300000001, 2020-08-07T15:07:18.463 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:18.463 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:18.463 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:18.463 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.463 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:18.464 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.187973+0000", 2020-08-07T15:07:18.464 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:18.464 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.464 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.464 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:18.465 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.187973+0000", 2020-08-07T15:07:18.465 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0841999999999998e-05 2020-08-07T15:07:18.465 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.465 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.465 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:18.466 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.188034+0000", 2020-08-07T15:07:18.466 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6880000000000002e-06 2020-08-07T15:07:18.466 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.466 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.466 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:18.467 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.188041+0000", 2020-08-07T15:07:18.467 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.15963588500000001 2020-08-07T15:07:18.467 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.467 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.467 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:18.467 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.347677+0000", 2020-08-07T15:07:18.468 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2729999999999999e-06 2020-08-07T15:07:18.468 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.468 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.468 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:18.469 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.347683+0000", 2020-08-07T15:07:18.469 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5182999999999998e-05 2020-08-07T15:07:18.469 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.469 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.469 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:18.469 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.347718+0000", 2020-08-07T15:07:18.470 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7371999999999999e-05 2020-08-07T15:07:18.470 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.470 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.470 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:18.470 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.347736+0000", 2020-08-07T15:07:18.471 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00060796199999999998 2020-08-07T15:07:18.471 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.471 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.471 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:18.471 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.348344+0000", 2020-08-07T15:07:18.472 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2268000000000002e-05 2020-08-07T15:07:18.472 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.472 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.472 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:18.472 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.348396+0000", 2020-08-07T15:07:18.472 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16042247300000001 2020-08-07T15:07:18.473 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.473 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:18.473 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.473 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.473 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.474 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:27048 3.1 e29/26 3:88501d52:::benchmark_data_smithi074_35447_object3380:head v 29'1608, mlcod=29'1602)", 2020-08-07T15:07:18.474 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:54.045944+0000", 2020-08-07T15:07:18.474 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 24.302435723999999, 2020-08-07T15:07:18.474 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.15902358699999999, 2020-08-07T15:07:18.474 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:18.475 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:18.475 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:18.475 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.475 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:18.475 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.045944+0000", 2020-08-07T15:07:18.476 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:18.476 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.476 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.476 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:18.476 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.045944+0000", 2020-08-07T15:07:18.477 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1610999999999998e-05 2020-08-07T15:07:18.477 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.477 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.477 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:18.477 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.045995+0000", 2020-08-07T15:07:18.477 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5180000000000001e-06 2020-08-07T15:07:18.478 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.478 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.478 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:18.478 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.046002+0000", 2020-08-07T15:07:18.478 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.156437364 2020-08-07T15:07:18.479 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.479 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.479 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:18.479 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.202439+0000", 2020-08-07T15:07:18.479 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0700000000000001e-06 2020-08-07T15:07:18.479 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.481 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.481 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:18.481 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.202446+0000", 2020-08-07T15:07:18.482 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5476000000000003e-05 2020-08-07T15:07:18.482 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.482 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.482 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:18.482 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.202492+0000", 2020-08-07T15:07:18.483 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9876999999999999e-05 2020-08-07T15:07:18.483 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.483 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.483 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:18.483 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.202522+0000", 2020-08-07T15:07:18.484 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023908919999999999 2020-08-07T15:07:18.484 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.484 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.484 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:18.484 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.204913+0000", 2020-08-07T15:07:18.485 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4778999999999998e-05 2020-08-07T15:07:18.485 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.485 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.485 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:18.485 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.204967+0000", 2020-08-07T15:07:18.486 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.15902358699999999 2020-08-07T15:07:18.486 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.486 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:18.486 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.487 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.487 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.487 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:27464 3.3 e29/20 3:c9f8435d:::benchmark_data_smithi074_35447_object3432:head v 29'1680, mlcod=29'1671)", 2020-08-07T15:07:18.487 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:54.366582+0000", 2020-08-07T15:07:18.487 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 23.981797212, 2020-08-07T15:07:18.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14665884600000001, 2020-08-07T15:07:18.488 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:18.488 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:18.488 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:18.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.489 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:18.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.366582+0000", 2020-08-07T15:07:18.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:18.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.489 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.489 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:18.490 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.366582+0000", 2020-08-07T15:07:18.490 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6334e-05 2020-08-07T15:07:18.490 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.490 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.490 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:18.491 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.366639+0000", 2020-08-07T15:07:18.491 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.153e-05 2020-08-07T15:07:18.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.491 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.491 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:18.492 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.366650+0000", 2020-08-07T15:07:18.492 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14520557000000001 2020-08-07T15:07:18.492 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.492 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.492 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:18.492 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.511856+0000", 2020-08-07T15:07:18.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4920000000000001e-06 2020-08-07T15:07:18.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.493 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:18.493 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.511861+0000", 2020-08-07T15:07:18.494 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0255e-05 2020-08-07T15:07:18.494 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.494 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:18.494 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.511901+0000", 2020-08-07T15:07:18.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3002000000000001e-05 2020-08-07T15:07:18.495 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.495 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.495 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:18.495 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.511924+0000", 2020-08-07T15:07:18.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001263712 2020-08-07T15:07:18.496 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.496 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.496 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:18.496 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.513188+0000", 2020-08-07T15:07:18.496 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2951000000000002e-05 2020-08-07T15:07:18.497 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.497 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.497 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:18.497 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.513241+0000", 2020-08-07T15:07:18.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14665884600000001 2020-08-07T15:07:18.498 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.498 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:18.498 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.498 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.498 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.498 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:31760 3.6 e30/26 3:6672be19:::benchmark_data_smithi074_35447_object3969:head v 30'2104, mlcod=30'2094)", 2020-08-07T15:07:18.499 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:56.999757+0000", 2020-08-07T15:07:18.499 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 21.348622540000001, 2020-08-07T15:07:18.499 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17823482099999999, 2020-08-07T15:07:18.499 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:18.499 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:18.500 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:18.500 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.500 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:18.500 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:56.999757+0000", 2020-08-07T15:07:18.500 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:18.501 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.501 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.501 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:18.501 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:56.999757+0000", 2020-08-07T15:07:18.501 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2955999999999999e-05 2020-08-07T15:07:18.501 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.502 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.502 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:18.502 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:56.999810+0000", 2020-08-07T15:07:18.502 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5509999999999996e-06 2020-08-07T15:07:18.502 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.503 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.503 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:18.503 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:56.999816+0000", 2020-08-07T15:07:18.503 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.177361504 2020-08-07T15:07:18.503 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.503 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.504 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:18.504 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177178+0000", 2020-08-07T15:07:18.504 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6979999999999997e-06 2020-08-07T15:07:18.504 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.504 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.505 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:18.505 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177183+0000", 2020-08-07T15:07:18.505 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1974999999999997e-05 2020-08-07T15:07:18.505 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.505 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.506 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:18.506 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177225+0000", 2020-08-07T15:07:18.506 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4612000000000001e-05 2020-08-07T15:07:18.506 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.506 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.506 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:18.507 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177249+0000", 2020-08-07T15:07:18.507 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00069063000000000002 2020-08-07T15:07:18.507 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.507 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.507 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:18.508 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177940+0000", 2020-08-07T15:07:18.508 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1894999999999997e-05 2020-08-07T15:07:18.508 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.508 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.508 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:18.508 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177992+0000", 2020-08-07T15:07:18.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17823482099999999 2020-08-07T15:07:18.509 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.509 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:18.509 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.509 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.510 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:40104 3.3 e31/20 3:c90cf9d0:::benchmark_data_smithi074_35447_object5012:head v 31'2416, mlcod=31'2407)", 2020-08-07T15:07:18.510 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:02.057882+0000", 2020-08-07T15:07:18.510 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.290497528, 2020-08-07T15:07:18.510 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.148032995, 2020-08-07T15:07:18.511 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:18.511 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:18.511 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:18.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:18.512 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:02.057882+0000", 2020-08-07T15:07:18.512 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:18.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.512 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.512 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:18.512 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:02.057882+0000", 2020-08-07T15:07:18.513 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6452999999999999e-05 2020-08-07T15:07:18.513 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.513 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.513 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:18.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:02.057908+0000", 2020-08-07T15:07:18.514 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9009999999999996e-06 2020-08-07T15:07:18.514 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.514 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:18.514 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:02.057912+0000", 2020-08-07T15:07:18.514 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14478486600000001 2020-08-07T15:07:18.515 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.515 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.515 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:18.515 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:02.202697+0000", 2020-08-07T15:07:18.515 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8339999999999998e-06 2020-08-07T15:07:18.516 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.516 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.516 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:18.516 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:02.202702+0000", 2020-08-07T15:07:18.517 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1689e-05 2020-08-07T15:07:18.517 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.517 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.517 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:18.517 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:02.202744+0000", 2020-08-07T15:07:18.518 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0477999999999999e-05 2020-08-07T15:07:18.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.518 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:18.518 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:02.202774+0000", 2020-08-07T15:07:18.518 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0030995060000000001 2020-08-07T15:07:18.519 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.519 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.519 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:18.519 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:02.205874+0000", 2020-08-07T15:07:18.519 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1267999999999999e-05 2020-08-07T15:07:18.520 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.520 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:18.520 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:02.205915+0000", 2020-08-07T15:07:18.520 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.148032995 2020-08-07T15:07:18.520 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.521 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:18.521 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.521 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.521 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:43335 3.6 e32/26 3:65a57ed3:::benchmark_data_smithi074_35447_object5416:head v 32'2831, mlcod=32'2822)", 2020-08-07T15:07:18.522 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:03.998850+0000", 2020-08-07T15:07:18.522 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 14.349529390000001, 2020-08-07T15:07:18.522 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.221384421, 2020-08-07T15:07:18.522 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:18.522 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:18.523 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:18.523 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.523 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:18.523 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998850+0000", 2020-08-07T15:07:18.523 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:18.524 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.524 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.524 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:18.524 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998850+0000", 2020-08-07T15:07:18.525 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4788000000000004e-05 2020-08-07T15:07:18.525 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.525 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.525 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:18.525 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998915+0000", 2020-08-07T15:07:18.526 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7909999999999999e-06 2020-08-07T15:07:18.526 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.526 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.526 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:18.526 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998922+0000", 2020-08-07T15:07:18.527 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22050499700000001 2020-08-07T15:07:18.527 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.527 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.527 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:18.527 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219427+0000", 2020-08-07T15:07:18.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0950000000000001e-06 2020-08-07T15:07:18.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.528 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:18.528 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219433+0000", 2020-08-07T15:07:18.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6342999999999997e-05 2020-08-07T15:07:18.529 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.529 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.529 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:18.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219509+0000", 2020-08-07T15:07:18.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4173999999999999e-05 2020-08-07T15:07:18.530 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.530 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.530 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:18.530 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219533+0000", 2020-08-07T15:07:18.530 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00064214899999999995 2020-08-07T15:07:18.530 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.531 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.531 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:18.531 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.220175+0000", 2020-08-07T15:07:18.531 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9083999999999999e-05 2020-08-07T15:07:18.531 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.532 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.532 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:18.532 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.220234+0000", 2020-08-07T15:07:18.532 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.221384421 2020-08-07T15:07:18.532 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.533 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:18.533 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.533 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.533 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.534 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:43336 3.6 e32/26 3:65a57ed3:::benchmark_data_smithi074_35447_object5416:head v 32'2832, mlcod=32'2822)", 2020-08-07T15:07:18.534 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:03.999509+0000", 2020-08-07T15:07:18.534 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 14.348870061, 2020-08-07T15:07:18.534 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22632864799999999, 2020-08-07T15:07:18.534 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:18.535 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:18.535 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:18.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:18.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999509+0000", 2020-08-07T15:07:18.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:18.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.536 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:18.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999509+0000", 2020-08-07T15:07:18.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0107e-05 2020-08-07T15:07:18.537 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.537 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.537 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:18.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999559+0000", 2020-08-07T15:07:18.538 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0889999999999996e-06 2020-08-07T15:07:18.538 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.538 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.538 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:18.539 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999566+0000", 2020-08-07T15:07:18.539 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22332094599999999 2020-08-07T15:07:18.539 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.539 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.540 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:18.541 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222887+0000", 2020-08-07T15:07:18.541 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5990000000000001e-06 2020-08-07T15:07:18.541 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.541 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:18.542 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222892+0000", 2020-08-07T15:07:18.542 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2342999999999998e-05 2020-08-07T15:07:18.542 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.542 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.542 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:18.543 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222924+0000", 2020-08-07T15:07:18.543 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9108999999999999e-05 2020-08-07T15:07:18.543 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.543 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.543 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:18.544 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222944+0000", 2020-08-07T15:07:18.544 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028409289999999999 2020-08-07T15:07:18.544 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.544 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.544 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:18.544 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.225784+0000", 2020-08-07T15:07:18.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3526000000000002e-05 2020-08-07T15:07:18.545 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.545 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:18.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.225838+0000", 2020-08-07T15:07:18.546 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22632864799999999 2020-08-07T15:07:18.546 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.546 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:18.546 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.547 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:43784 3.6 e32/26 3:6da53356:::benchmark_data_smithi074_35447_object5472:head v 32'2848, mlcod=32'2839)", 2020-08-07T15:07:18.547 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:04.253355+0000", 2020-08-07T15:07:18.547 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 14.095024489, 2020-08-07T15:07:18.547 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21456700500000001, 2020-08-07T15:07:18.548 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:18.548 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:18.548 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:18.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.549 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:18.549 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253355+0000", 2020-08-07T15:07:18.549 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:18.549 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.549 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.549 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:18.550 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253355+0000", 2020-08-07T15:07:18.550 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6168000000000002e-05 2020-08-07T15:07:18.550 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.550 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.550 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:18.551 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253401+0000", 2020-08-07T15:07:18.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3179999999999997e-06 2020-08-07T15:07:18.551 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.551 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.551 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:18.552 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253406+0000", 2020-08-07T15:07:18.552 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21194338600000001 2020-08-07T15:07:18.552 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.552 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:18.552 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465350+0000", 2020-08-07T15:07:18.553 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3759999999999999e-06 2020-08-07T15:07:18.553 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.553 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.553 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:18.553 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465355+0000", 2020-08-07T15:07:18.554 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8250999999999997e-05 2020-08-07T15:07:18.554 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.554 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.554 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:18.554 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465393+0000", 2020-08-07T15:07:18.555 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0296999999999999e-05 2020-08-07T15:07:18.555 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.555 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.555 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:18.555 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465414+0000", 2020-08-07T15:07:18.555 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024515209999999999 2020-08-07T15:07:18.556 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:18.556 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.467865+0000", 2020-08-07T15:07:18.556 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6687999999999997e-05 2020-08-07T15:07:18.557 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.557 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.557 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:18.557 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.467922+0000", 2020-08-07T15:07:18.557 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21456700500000001 2020-08-07T15:07:18.557 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.558 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:18.558 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.558 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.558 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.558 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:44152 3.6 e32/26 3:6c90cb4b:::benchmark_data_smithi074_35447_object5518:head v 32'2856, mlcod=32'2851)", 2020-08-07T15:07:18.559 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:04.473943+0000", 2020-08-07T15:07:18.559 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 13.874436666999999, 2020-08-07T15:07:18.559 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18875260999999999, 2020-08-07T15:07:18.559 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:18.560 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:18.560 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:18.560 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.560 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:18.560 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.473943+0000", 2020-08-07T15:07:18.561 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:18.561 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.561 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.561 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:18.561 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.473943+0000", 2020-08-07T15:07:18.562 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8267999999999999e-05 2020-08-07T15:07:18.562 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.562 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.562 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:18.562 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.473991+0000", 2020-08-07T15:07:18.563 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9689999999999999e-06 2020-08-07T15:07:18.563 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.563 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.563 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:18.563 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.473997+0000", 2020-08-07T15:07:18.563 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18142124000000001 2020-08-07T15:07:18.564 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.564 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:18.564 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.655418+0000", 2020-08-07T15:07:18.564 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7439999999999998e-06 2020-08-07T15:07:18.565 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.565 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.565 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:18.565 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.655423+0000", 2020-08-07T15:07:18.565 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0052999999999997e-05 2020-08-07T15:07:18.566 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.566 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.566 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:18.566 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.655473+0000", 2020-08-07T15:07:18.566 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3739000000000001e-05 2020-08-07T15:07:18.566 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.567 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.567 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:18.567 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.655497+0000", 2020-08-07T15:07:18.567 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0071399030000000004 2020-08-07T15:07:18.567 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.568 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.568 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:18.568 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.662637+0000", 2020-08-07T15:07:18.568 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8693999999999997e-05 2020-08-07T15:07:18.568 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.569 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.569 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:18.569 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.662695+0000", 2020-08-07T15:07:18.569 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18875260999999999 2020-08-07T15:07:18.569 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.570 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:18.570 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.570 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.570 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.570 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:54200 3.3 e37/20 3:cc40637e:::benchmark_data_smithi074_35447_object6774:head v 37'3248, mlcod=37'3236)", 2020-08-07T15:07:18.570 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:07:18.571 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.2498172409999997, 2020-08-07T15:07:18.571 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24065872499999999, 2020-08-07T15:07:18.571 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:18.571 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:18.571 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:18.572 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.572 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:18.572 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:07:18.572 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:18.572 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.573 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.573 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:18.573 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:07:18.573 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1671000000000002e-05 2020-08-07T15:07:18.573 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.574 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.574 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:18.574 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098594+0000", 2020-08-07T15:07:18.574 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0890000000000002e-06 2020-08-07T15:07:18.574 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.575 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:18.575 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098598+0000", 2020-08-07T15:07:18.575 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23875592300000001 2020-08-07T15:07:18.575 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.576 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.576 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:18.576 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337354+0000", 2020-08-07T15:07:18.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0420000000000002e-06 2020-08-07T15:07:18.576 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.577 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.577 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:18.577 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337359+0000", 2020-08-07T15:07:18.577 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0586000000000001e-05 2020-08-07T15:07:18.577 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.578 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.578 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:18.578 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337399+0000", 2020-08-07T15:07:18.578 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9415999999999998e-05 2020-08-07T15:07:18.578 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.579 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.579 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:18.579 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337419+0000", 2020-08-07T15:07:18.579 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017568390000000001 2020-08-07T15:07:18.579 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.580 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.580 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:18.580 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.339176+0000", 2020-08-07T15:07:18.580 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5158999999999999e-05 2020-08-07T15:07:18.580 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.580 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.581 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:18.581 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.339221+0000", 2020-08-07T15:07:18.581 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24065872499999999 2020-08-07T15:07:18.581 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.581 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:18.582 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.582 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.582 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.582 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:54656 3.1 e37/26 3:8c83b9d6:::benchmark_data_smithi074_35447_object6831:head v 37'3264, mlcod=37'3257)", 2020-08-07T15:07:18.582 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:13.403567+0000", 2020-08-07T15:07:18.583 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4.9448122239999996, 2020-08-07T15:07:18.583 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.159582795, 2020-08-07T15:07:18.583 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:18.583 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:18.583 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:18.583 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.584 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:18.584 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.403567+0000", 2020-08-07T15:07:18.584 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:18.584 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.584 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.585 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:18.585 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.403567+0000", 2020-08-07T15:07:18.585 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5383999999999997e-05 2020-08-07T15:07:18.585 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.585 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.586 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:18.586 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.403623+0000", 2020-08-07T15:07:18.586 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1099999999999997e-06 2020-08-07T15:07:18.586 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.586 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.586 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:18.587 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.403630+0000", 2020-08-07T15:07:18.587 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.157788494 2020-08-07T15:07:18.587 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.587 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.587 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:18.588 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.561418+0000", 2020-08-07T15:07:18.588 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0039999999999999e-06 2020-08-07T15:07:18.588 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.588 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:18.588 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.561423+0000", 2020-08-07T15:07:18.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2250000000000003e-05 2020-08-07T15:07:18.589 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.589 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:18.589 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.561475+0000", 2020-08-07T15:07:18.590 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7313e-05 2020-08-07T15:07:18.590 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.590 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.590 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:18.590 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.561513+0000", 2020-08-07T15:07:18.591 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001570702 2020-08-07T15:07:18.591 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.591 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.591 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:18.591 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.563083+0000", 2020-08-07T15:07:18.591 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6538000000000006e-05 2020-08-07T15:07:18.592 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.592 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.592 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:18.592 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.563150+0000", 2020-08-07T15:07:18.592 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.159582795 2020-08-07T15:07:18.593 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.593 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:18.593 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.593 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.593 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.594 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:54728 3.7 e37/20 3:ee999259:::benchmark_data_smithi074_35447_object6840:head v 37'3328, mlcod=37'3319)", 2020-08-07T15:07:18.594 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:13.452195+0000", 2020-08-07T15:07:18.594 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4.8961844599999997, 2020-08-07T15:07:18.594 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.187979807, 2020-08-07T15:07:18.594 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:18.594 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:18.595 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:18.595 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.595 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:18.595 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.452195+0000", 2020-08-07T15:07:18.596 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:18.596 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.596 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.596 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:18.596 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.452195+0000", 2020-08-07T15:07:18.597 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9023000000000001e-05 2020-08-07T15:07:18.597 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.597 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.597 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:18.597 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.452244+0000", 2020-08-07T15:07:18.598 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.733e-06 2020-08-07T15:07:18.598 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.598 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.598 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:18.598 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.452250+0000", 2020-08-07T15:07:18.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18453694300000001 2020-08-07T15:07:18.599 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.599 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.599 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:18.599 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.636787+0000", 2020-08-07T15:07:18.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5679999999999999e-06 2020-08-07T15:07:18.600 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.600 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.600 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:18.600 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.636792+0000", 2020-08-07T15:07:18.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0119999999999999e-05 2020-08-07T15:07:18.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.601 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:18.601 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.636872+0000", 2020-08-07T15:07:18.601 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6356999999999999e-05 2020-08-07T15:07:18.602 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.602 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.602 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:18.602 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.636899+0000", 2020-08-07T15:07:18.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003233737 2020-08-07T15:07:18.602 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.603 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:18.603 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.640132+0000", 2020-08-07T15:07:18.603 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2326000000000001e-05 2020-08-07T15:07:18.603 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:18.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.640175+0000", 2020-08-07T15:07:18.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.187979807 2020-08-07T15:07:18.604 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.605 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:18.605 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.605 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.605 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.605 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:55983 3.b e37/20 3:de9e9f00:::benchmark_data_smithi074_35447_object6997:head v 37'3399, mlcod=37'3392)", 2020-08-07T15:07:18.605 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:14.253268+0000", 2020-08-07T15:07:18.606 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4.0951111320000004, 2020-08-07T15:07:18.606 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.155558523, 2020-08-07T15:07:18.606 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:18.606 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:18.606 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:18.607 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.607 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:18.607 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.253268+0000", 2020-08-07T15:07:18.607 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:18.607 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.608 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.608 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:18.608 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.253268+0000", 2020-08-07T15:07:18.608 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5989999999999999e-05 2020-08-07T15:07:18.608 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.608 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.609 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:18.609 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.253304+0000", 2020-08-07T15:07:18.609 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3369999999999996e-06 2020-08-07T15:07:18.609 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.609 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.610 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:18.610 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.253311+0000", 2020-08-07T15:07:18.610 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.1535821 2020-08-07T15:07:18.610 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.610 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.611 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:18.611 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.406893+0000", 2020-08-07T15:07:18.611 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3800000000000002e-06 2020-08-07T15:07:18.611 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.611 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.611 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:18.612 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.406898+0000", 2020-08-07T15:07:18.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013867799999999999 2020-08-07T15:07:18.612 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.612 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.612 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:18.613 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.407037+0000", 2020-08-07T15:07:18.613 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6845000000000001e-05 2020-08-07T15:07:18.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.614 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:18.614 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.407064+0000", 2020-08-07T15:07:18.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017123259999999999 2020-08-07T15:07:18.614 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.614 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.615 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:18.615 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.408776+0000", 2020-08-07T15:07:18.615 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0866999999999999e-05 2020-08-07T15:07:18.615 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.615 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.615 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:18.616 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.408827+0000", 2020-08-07T15:07:18.616 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.155558523 2020-08-07T15:07:18.616 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.616 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:18.616 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.617 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.617 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.617 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:55984 3.b e37/20 3:de9e9f00:::benchmark_data_smithi074_35447_object6997:head v 37'3400, mlcod=37'3392)", 2020-08-07T15:07:18.617 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:14.254797+0000", 2020-08-07T15:07:18.617 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4.0935822450000003, 2020-08-07T15:07:18.618 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.15407087799999999, 2020-08-07T15:07:18.618 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:18.618 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:18.618 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:18.618 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.619 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:18.619 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.254797+0000", 2020-08-07T15:07:18.619 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:18.619 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.619 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.620 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:18.620 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.254797+0000", 2020-08-07T15:07:18.620 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2657000000000003e-05 2020-08-07T15:07:18.620 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.620 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.620 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:18.621 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.254830+0000", 2020-08-07T15:07:18.621 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4739999999999994e-06 2020-08-07T15:07:18.621 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.621 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.621 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:18.622 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.254839+0000", 2020-08-07T15:07:18.622 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.152073969 2020-08-07T15:07:18.622 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.622 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.622 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:18.623 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.406913+0000", 2020-08-07T15:07:18.623 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4399999999999998e-06 2020-08-07T15:07:18.623 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.623 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.623 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:18.624 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.406918+0000", 2020-08-07T15:07:18.624 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00061322999999999998 2020-08-07T15:07:18.624 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.624 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.624 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:18.625 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.407531+0000", 2020-08-07T15:07:18.625 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1022999999999998e-05 2020-08-07T15:07:18.625 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.625 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.626 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:18.626 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.407552+0000", 2020-08-07T15:07:18.626 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001277535 2020-08-07T15:07:18.626 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.626 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.627 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:18.627 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.408830+0000", 2020-08-07T15:07:18.627 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8550000000000002e-05 2020-08-07T15:07:18.627 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.627 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:18.628 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.408868+0000", 2020-08-07T15:07:18.628 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.15407087799999999 2020-08-07T15:07:18.628 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.628 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:18.628 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.629 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.629 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:18.629 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:18.642 INFO:teuthology.orchestra.run.smithi074.stderr:set pool 1 pgp_num to 1 2020-08-07T15:07:18.642 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:18.629+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:18.643 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:07:18.738 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:18.732+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:18.834 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:18.833+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:18.850 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:18.850 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:07:18.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.851 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:62950 3.a e41/20 3:5322bacb:::benchmark_data_smithi074_35447_object7868:head v 41'4214, mlcod=41'4203)", 2020-08-07T15:07:18.851 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:18.844039+0000", 2020-08-07T15:07:18.851 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0045427870000000004, 2020-08-07T15:07:18.851 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045614310000000003, 2020-08-07T15:07:18.851 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:18.852 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:18.852 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:18.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:18.853 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.844039+0000", 2020-08-07T15:07:18.853 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:18.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.853 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:18.854 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.844039+0000", 2020-08-07T15:07:18.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0995000000000002e-05 2020-08-07T15:07:18.854 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.854 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.856 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:18.856 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.844090+0000", 2020-08-07T15:07:18.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4679999999999997e-06 2020-08-07T15:07:18.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:18.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.844097+0000", 2020-08-07T15:07:18.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001282645 2020-08-07T15:07:18.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.858 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:18.859 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.845379+0000", 2020-08-07T15:07:18.859 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3390000000000004e-06 2020-08-07T15:07:18.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.859 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:18.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.845387+0000", 2020-08-07T15:07:18.860 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5345000000000001e-05 2020-08-07T15:07:18.860 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.860 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.861 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:18.861 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.845422+0000", 2020-08-07T15:07:18.863 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4969e-05 2020-08-07T15:07:18.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:18.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.845437+0000", 2020-08-07T15:07:18.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001397761 2020-08-07T15:07:18.864 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.864 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:18.865 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.865 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:62951 3.a e41/20 3:5322bacb:::benchmark_data_smithi074_35447_object7868:head v 41'4215, mlcod=41'4206)", 2020-08-07T15:07:18.865 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:18.845284+0000", 2020-08-07T15:07:18.866 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0032983499999999998, 2020-08-07T15:07:18.866 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033573399999999999, 2020-08-07T15:07:18.866 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:18.867 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:18.867 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:18.867 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.867 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:18.867 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.845284+0000", 2020-08-07T15:07:18.868 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:18.868 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.868 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.868 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:18.869 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.845284+0000", 2020-08-07T15:07:18.869 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5651e-05 2020-08-07T15:07:18.869 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.869 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.870 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:18.870 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.845339+0000", 2020-08-07T15:07:18.870 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.686e-06 2020-08-07T15:07:18.870 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.870 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.871 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:18.871 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.845346+0000", 2020-08-07T15:07:18.871 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00077088199999999997 2020-08-07T15:07:18.871 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.872 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:18.872 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.846117+0000", 2020-08-07T15:07:18.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4129999999999997e-06 2020-08-07T15:07:18.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.872 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.872 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:18.873 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.846122+0000", 2020-08-07T15:07:18.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6038999999999998e-05 2020-08-07T15:07:18.873 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.873 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:18.874 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.846158+0000", 2020-08-07T15:07:18.874 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8029e-05 2020-08-07T15:07:18.874 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:18.874 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:18.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:18.875 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.846176+0000", 2020-08-07T15:07:18.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00089269999999999996 2020-08-07T15:07:18.875 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.875 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:18.876 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.876 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:18.876 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:07:18.876 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 2 2020-08-07T15:07:18.876 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:18.878 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:07:18.934 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:18.933+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:19.034 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:19.034 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:07:19.034 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:07:19.035 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:07:19.035 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:19.037 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:19.036+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:19.055 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:07:19.138 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:19.137+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:19.225 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:19.225 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:07:19.225 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:07:19.226 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:07:19.226 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.226 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:9352 3.e e24/20 3:7113579b:::benchmark_data_smithi074_35447_object1168:head v 24'656, mlcod=24'648)", 2020-08-07T15:07:19.226 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.889020+0000", 2020-08-07T15:07:19.227 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 37.332148142000001, 2020-08-07T15:07:19.227 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24471712600000001, 2020-08-07T15:07:19.227 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:19.227 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:19.228 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:19.228 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.228 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:19.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.889020+0000", 2020-08-07T15:07:19.229 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:19.229 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.229 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.232 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:19.233 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.889020+0000", 2020-08-07T15:07:19.233 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7401999999999998e-05 2020-08-07T15:07:19.233 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.234 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.234 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:19.234 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.889068+0000", 2020-08-07T15:07:19.234 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.669e-06 2020-08-07T15:07:19.235 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.235 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.235 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:19.235 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.889075+0000", 2020-08-07T15:07:19.236 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23823541500000001 2020-08-07T15:07:19.236 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.236 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.236 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:19.237 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.127310+0000", 2020-08-07T15:07:19.237 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.895e-06 2020-08-07T15:07:19.237 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.237 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.238 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:19.238 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.127314+0000", 2020-08-07T15:07:19.238 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3778000000000001e-05 2020-08-07T15:07:19.238 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.239 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.239 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:19.239 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.127348+0000", 2020-08-07T15:07:19.239 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6517000000000001e-05 2020-08-07T15:07:19.240 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.240 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.240 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:19.240 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.127364+0000", 2020-08-07T15:07:19.241 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063235399999999999 2020-08-07T15:07:19.241 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.241 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.256 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:19.257 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.133688+0000", 2020-08-07T15:07:19.257 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9910000000000002e-05 2020-08-07T15:07:19.257 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.257 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.258 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:19.258 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.133738+0000", 2020-08-07T15:07:19.258 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24471712600000001 2020-08-07T15:07:19.258 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.259 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:19.259 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.259 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.260 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:10240 3.6 e24/20 3:665d71ff:::benchmark_data_smithi074_35447_object1279:head v 24'728, mlcod=24'718)", 2020-08-07T15:07:19.260 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.394891+0000", 2020-08-07T15:07:19.260 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 36.826277783999998, 2020-08-07T15:07:19.261 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24904923400000001, 2020-08-07T15:07:19.261 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:19.261 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:19.262 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:19.262 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.262 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:19.262 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394891+0000", 2020-08-07T15:07:19.263 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:19.263 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.263 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.263 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:19.264 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394891+0000", 2020-08-07T15:07:19.264 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9815000000000003e-05 2020-08-07T15:07:19.264 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.265 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.267 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:19.267 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394941+0000", 2020-08-07T15:07:19.267 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5100000000000004e-06 2020-08-07T15:07:19.267 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.268 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.268 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:19.268 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394947+0000", 2020-08-07T15:07:19.268 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24715435899999999 2020-08-07T15:07:19.269 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.269 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.269 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:19.269 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.642101+0000", 2020-08-07T15:07:19.270 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7069999999999992e-06 2020-08-07T15:07:19.270 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.270 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.270 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:19.271 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.642109+0000", 2020-08-07T15:07:19.271 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9301000000000002e-05 2020-08-07T15:07:19.271 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:19.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.642148+0000", 2020-08-07T15:07:19.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7227999999999998e-05 2020-08-07T15:07:19.273 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.273 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:19.273 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.642166+0000", 2020-08-07T15:07:19.274 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017236300000000001 2020-08-07T15:07:19.274 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.274 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.274 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:19.274 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.643889+0000", 2020-08-07T15:07:19.275 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0683999999999998e-05 2020-08-07T15:07:19.275 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.277 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.277 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:19.278 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.643940+0000", 2020-08-07T15:07:19.279 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24904923400000001 2020-08-07T15:07:19.280 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.280 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:19.280 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.280 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.281 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.281 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:15576 3.e e27/20 3:727724a5:::benchmark_data_smithi074_35447_object1946:head v 27'1032, mlcod=27'1023)", 2020-08-07T15:07:19.281 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:07:19.281 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 32.551685931999998, 2020-08-07T15:07:19.281 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.46091973000000003, 2020-08-07T15:07:19.282 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:19.282 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:19.282 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:19.282 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.282 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:19.283 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:07:19.283 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:19.283 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.283 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.283 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:19.284 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:07:19.284 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9592000000000001e-05 2020-08-07T15:07:19.284 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.284 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.285 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:19.285 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669542+0000", 2020-08-07T15:07:19.286 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.113e-06 2020-08-07T15:07:19.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.288 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:19.288 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669547+0000", 2020-08-07T15:07:19.288 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45151617799999999 2020-08-07T15:07:19.289 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.289 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.289 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:19.290 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121064+0000", 2020-08-07T15:07:19.290 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.377e-06 2020-08-07T15:07:19.290 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.291 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.291 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:19.291 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121067+0000", 2020-08-07T15:07:19.291 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9195e-05 2020-08-07T15:07:19.292 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.292 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.292 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:19.292 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121106+0000", 2020-08-07T15:07:19.293 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7509000000000001e-05 2020-08-07T15:07:19.293 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.293 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.293 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:19.294 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121124+0000", 2020-08-07T15:07:19.294 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0091783100000000003 2020-08-07T15:07:19.294 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.294 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.294 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:19.295 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.130302+0000", 2020-08-07T15:07:19.295 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000100456 2020-08-07T15:07:19.295 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.295 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.296 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:19.296 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.130402+0000", 2020-08-07T15:07:19.296 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.46091973000000003 2020-08-07T15:07:19.296 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.297 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:19.297 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.297 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.297 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.298 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:24264 3.8 e28/20 3:1d3fdbda:::benchmark_data_smithi074_35447_object3032:head v 28'1520, mlcod=28'1515)", 2020-08-07T15:07:19.298 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:07:19.298 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 26.838055895, 2020-08-07T15:07:19.298 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41856254199999998, 2020-08-07T15:07:19.299 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:19.299 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:19.299 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:19.300 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.301 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:19.302 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:07:19.302 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:19.302 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.302 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.303 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:19.304 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:07:19.304 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1348999999999998e-05 2020-08-07T15:07:19.305 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.305 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.305 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:19.305 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383174+0000", 2020-08-07T15:07:19.306 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6579999999999996e-06 2020-08-07T15:07:19.306 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.306 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.306 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:19.307 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383181+0000", 2020-08-07T15:07:19.307 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41194352699999998 2020-08-07T15:07:19.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.308 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:19.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.795124+0000", 2020-08-07T15:07:19.308 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5729999999999999e-06 2020-08-07T15:07:19.309 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.309 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.309 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:19.310 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.795128+0000", 2020-08-07T15:07:19.310 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002170384 2020-08-07T15:07:19.310 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.311 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.311 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:19.311 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.797298+0000", 2020-08-07T15:07:19.311 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5412e-05 2020-08-07T15:07:19.311 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.312 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.312 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:19.312 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.797324+0000", 2020-08-07T15:07:19.312 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042943360000000002 2020-08-07T15:07:19.313 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.313 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.313 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:19.313 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.801618+0000", 2020-08-07T15:07:19.314 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7302999999999997e-05 2020-08-07T15:07:19.314 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.314 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.315 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:19.315 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.801675+0000", 2020-08-07T15:07:19.315 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41856254199999998 2020-08-07T15:07:19.315 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.316 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:19.316 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.316 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.316 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.317 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:24992 3.0 3:008a8fe9:::benchmark_data_smithi074_35447_object3123:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e28)", 2020-08-07T15:07:19.317 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.846767+0000", 2020-08-07T15:07:19.317 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 26.374401361, 2020-08-07T15:07:19.317 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21218208499999999, 2020-08-07T15:07:19.318 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:19.318 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:19.318 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:19.318 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:19.319 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:19.319 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 24992 2020-08-07T15:07:19.320 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.320 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:19.320 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.320 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:19.320 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.846767+0000", 2020-08-07T15:07:19.321 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:19.321 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.321 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.321 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:19.322 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.846767+0000", 2020-08-07T15:07:19.322 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0043000000000001e-05 2020-08-07T15:07:19.322 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.322 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.323 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:19.325 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.846827+0000", 2020-08-07T15:07:19.325 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1109999999999999e-06 2020-08-07T15:07:19.325 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.325 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.326 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:19.326 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.846832+0000", 2020-08-07T15:07:19.326 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.095e-05 2020-08-07T15:07:19.326 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.329 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.329 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:19.329 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.846843+0000", 2020-08-07T15:07:19.330 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3189999999999996e-06 2020-08-07T15:07:19.330 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.330 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.330 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:19.331 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.846848+0000", 2020-08-07T15:07:19.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0035319069999999999 2020-08-07T15:07:19.331 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.331 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.332 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:19.332 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.850380+0000", 2020-08-07T15:07:19.332 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000100331 2020-08-07T15:07:19.332 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.332 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.333 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:19.333 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.850480+0000", 2020-08-07T15:07:19.334 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000122416 2020-08-07T15:07:19.334 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.334 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.335 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:07:19.336 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.850602+0000", 2020-08-07T15:07:19.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002556928 2020-08-07T15:07:19.336 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.336 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.337 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:19.337 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.853159+0000", 2020-08-07T15:07:19.337 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5199999999999995e-07 2020-08-07T15:07:19.338 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.338 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.338 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:19.339 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.853160+0000", 2020-08-07T15:07:19.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1212999999999998e-05 2020-08-07T15:07:19.339 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.339 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.340 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:19.340 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.853221+0000", 2020-08-07T15:07:19.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017586749999999999 2020-08-07T15:07:19.341 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.341 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:19.341 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.854980+0000", 2020-08-07T15:07:19.342 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.203895095 2020-08-07T15:07:19.342 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.342 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.342 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:19.343 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.058875+0000", 2020-08-07T15:07:19.343 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6759999999999999e-06 2020-08-07T15:07:19.343 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.343 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.344 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:19.344 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.058877+0000", 2020-08-07T15:07:19.344 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2569000000000003e-05 2020-08-07T15:07:19.345 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.345 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.345 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:19.345 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.058949+0000", 2020-08-07T15:07:19.346 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21218208499999999 2020-08-07T15:07:19.346 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.346 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:19.346 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.347 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.347 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:35280 3.6 3:63cda715:::benchmark_data_smithi074_35447_object4409:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-08-07T15:07:19.348 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:59.106156+0000", 2020-08-07T15:07:19.348 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 20.115012607000001, 2020-08-07T15:07:19.349 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.218044613, 2020-08-07T15:07:19.349 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:19.350 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:19.350 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:19.350 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:19.351 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:19.351 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 35280 2020-08-07T15:07:19.352 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.352 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:19.355 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.356 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:19.356 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.106156+0000", 2020-08-07T15:07:19.356 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:19.356 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.356 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.357 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:19.357 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.106156+0000", 2020-08-07T15:07:19.357 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5969000000000002e-05 2020-08-07T15:07:19.357 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.357 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.358 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:19.358 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.106182+0000", 2020-08-07T15:07:19.358 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9440000000000001e-06 2020-08-07T15:07:19.358 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.358 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.359 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:19.359 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.106185+0000", 2020-08-07T15:07:19.359 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7619999999999998e-06 2020-08-07T15:07:19.359 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.359 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.360 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:19.360 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.106192+0000", 2020-08-07T15:07:19.360 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4090000000000001e-06 2020-08-07T15:07:19.360 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.360 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.361 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:19.361 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.106194+0000", 2020-08-07T15:07:19.361 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0076024070000000003 2020-08-07T15:07:19.361 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.361 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.362 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:19.362 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.113796+0000", 2020-08-07T15:07:19.362 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6061000000000001e-05 2020-08-07T15:07:19.362 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.362 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.362 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:19.363 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.113893+0000", 2020-08-07T15:07:19.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019322600000000001 2020-08-07T15:07:19.363 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.363 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.363 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:19.364 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.114086+0000", 2020-08-07T15:07:19.364 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00074211299999999997 2020-08-07T15:07:19.364 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:19.365 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.114828+0000", 2020-08-07T15:07:19.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0440000000000001e-06 2020-08-07T15:07:19.365 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.365 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.365 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:19.366 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.114829+0000", 2020-08-07T15:07:19.366 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5028000000000001e-05 2020-08-07T15:07:19.366 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.366 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:19.366 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.114894+0000", 2020-08-07T15:07:19.367 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000291852 2020-08-07T15:07:19.367 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.367 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.367 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:19.367 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.115186+0000", 2020-08-07T15:07:19.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20895322599999999 2020-08-07T15:07:19.368 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.368 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.368 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:19.368 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.324139+0000", 2020-08-07T15:07:19.369 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4700000000000001e-07 2020-08-07T15:07:19.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.369 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:19.369 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.324140+0000", 2020-08-07T15:07:19.369 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0825000000000001e-05 2020-08-07T15:07:19.370 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.370 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.370 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:19.370 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.324201+0000", 2020-08-07T15:07:19.370 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.218044613 2020-08-07T15:07:19.371 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.371 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:19.371 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.371 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.372 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:43335 3.6 3:65a57ed3:::benchmark_data_smithi074_35447_object5416:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:07:19.372 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:03.997487+0000", 2020-08-07T15:07:19.372 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 15.223681716, 2020-08-07T15:07:19.372 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22588636300000001, 2020-08-07T15:07:19.372 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:19.373 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:19.373 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:19.373 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:19.373 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:19.373 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 43335 2020-08-07T15:07:19.374 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.374 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:19.374 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.374 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:19.374 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.997487+0000", 2020-08-07T15:07:19.374 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:19.375 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.375 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:19.375 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.997487+0000", 2020-08-07T15:07:19.375 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9805000000000002e-05 2020-08-07T15:07:19.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.376 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:19.376 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.997537+0000", 2020-08-07T15:07:19.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4040000000000003e-06 2020-08-07T15:07:19.377 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:19.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.997542+0000", 2020-08-07T15:07:19.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.061e-05 2020-08-07T15:07:19.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:19.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.997553+0000", 2020-08-07T15:07:19.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4039999999999997e-06 2020-08-07T15:07:19.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:19.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.997557+0000", 2020-08-07T15:07:19.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00085322200000000005 2020-08-07T15:07:19.379 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:19.380 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998410+0000", 2020-08-07T15:07:19.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9388999999999994e-05 2020-08-07T15:07:19.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:19.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998500+0000", 2020-08-07T15:07:19.382 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00022212999999999999 2020-08-07T15:07:19.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.382 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.382 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:19.382 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998722+0000", 2020-08-07T15:07:19.383 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0036367819999999999 2020-08-07T15:07:19.383 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.383 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.383 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:19.383 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.002359+0000", 2020-08-07T15:07:19.384 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2700000000000001e-07 2020-08-07T15:07:19.384 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.384 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.384 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:19.384 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.002359+0000", 2020-08-07T15:07:19.384 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5210000000000002e-05 2020-08-07T15:07:19.385 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.385 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.385 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:19.385 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.002414+0000", 2020-08-07T15:07:19.385 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0073127519999999996 2020-08-07T15:07:19.386 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.386 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.386 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:19.386 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.009727+0000", 2020-08-07T15:07:19.386 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21357918200000001 2020-08-07T15:07:19.387 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.387 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.387 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:19.387 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.223306+0000", 2020-08-07T15:07:19.387 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.08e-06 2020-08-07T15:07:19.388 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.388 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.388 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:19.388 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.223307+0000", 2020-08-07T15:07:19.388 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6166000000000003e-05 2020-08-07T15:07:19.389 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.389 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.389 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:19.389 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.223373+0000", 2020-08-07T15:07:19.389 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22588636300000001 2020-08-07T15:07:19.390 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.390 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:19.390 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.390 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.390 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.390 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:43336 3.6 3:65a57ed3:::benchmark_data_smithi074_35447_object5416:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:07:19.391 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:03.997813+0000", 2020-08-07T15:07:19.391 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 15.223355268000001, 2020-08-07T15:07:19.391 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22842448100000001, 2020-08-07T15:07:19.391 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:19.391 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:19.392 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:19.392 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:19.392 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:19.392 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 43336 2020-08-07T15:07:19.392 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.393 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:19.393 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.393 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:19.393 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.997813+0000", 2020-08-07T15:07:19.393 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:19.394 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.394 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.394 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:19.394 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.997813+0000", 2020-08-07T15:07:19.394 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3659000000000002e-05 2020-08-07T15:07:19.395 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.395 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.395 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:19.396 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.997867+0000", 2020-08-07T15:07:19.396 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9199999999999997e-06 2020-08-07T15:07:19.396 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.396 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.396 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:19.397 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.997871+0000", 2020-08-07T15:07:19.397 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6510000000000001e-06 2020-08-07T15:07:19.397 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.397 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.397 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:19.398 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.997879+0000", 2020-08-07T15:07:19.398 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3329999999999996e-06 2020-08-07T15:07:19.398 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.398 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.398 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:19.399 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.997884+0000", 2020-08-07T15:07:19.399 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012298560000000001 2020-08-07T15:07:19.399 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.399 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:19.400 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999114+0000", 2020-08-07T15:07:19.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4678000000000004e-05 2020-08-07T15:07:19.400 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.400 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:19.400 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999208+0000", 2020-08-07T15:07:19.401 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016877099999999999 2020-08-07T15:07:19.401 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.401 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.401 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:19.401 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999377+0000", 2020-08-07T15:07:19.402 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0048515140000000003 2020-08-07T15:07:19.402 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.402 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.402 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:19.402 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.004229+0000", 2020-08-07T15:07:19.403 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1100000000000005e-07 2020-08-07T15:07:19.403 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.403 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.403 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:19.403 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.004230+0000", 2020-08-07T15:07:19.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0355000000000002e-05 2020-08-07T15:07:19.404 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.404 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.404 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:19.404 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.004270+0000", 2020-08-07T15:07:19.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054997459999999998 2020-08-07T15:07:19.405 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.405 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:19.405 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.009770+0000", 2020-08-07T15:07:19.405 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21640456999999999 2020-08-07T15:07:19.406 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.406 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.406 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:19.406 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.226174+0000", 2020-08-07T15:07:19.406 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2200000000000003e-07 2020-08-07T15:07:19.407 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.407 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.407 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:19.407 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.226175+0000", 2020-08-07T15:07:19.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2895e-05 2020-08-07T15:07:19.407 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.408 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.408 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:19.408 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.226238+0000", 2020-08-07T15:07:19.408 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22842448100000001 2020-08-07T15:07:19.408 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.409 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:19.409 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.409 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.409 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:43784 3.6 3:6da53356:::benchmark_data_smithi074_35447_object5472:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:07:19.410 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:04.251360+0000", 2020-08-07T15:07:19.410 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 14.969808207, 2020-08-07T15:07:19.410 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.218968882, 2020-08-07T15:07:19.410 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:19.410 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:19.411 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:19.411 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:19.411 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:19.411 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 43784 2020-08-07T15:07:19.411 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.411 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:19.412 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.412 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:19.412 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.251360+0000", 2020-08-07T15:07:19.412 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:19.413 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.413 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:19.413 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.251360+0000", 2020-08-07T15:07:19.414 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9440000000000001e-05 2020-08-07T15:07:19.414 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.414 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.414 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:19.414 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.251410+0000", 2020-08-07T15:07:19.415 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1830000000000002e-06 2020-08-07T15:07:19.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.415 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:19.415 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.251415+0000", 2020-08-07T15:07:19.415 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0155e-05 2020-08-07T15:07:19.416 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.416 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.416 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:19.416 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.251425+0000", 2020-08-07T15:07:19.416 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3189999999999996e-06 2020-08-07T15:07:19.417 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.417 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.417 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:19.417 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.251429+0000", 2020-08-07T15:07:19.417 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001066481 2020-08-07T15:07:19.418 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.418 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.418 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:19.418 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.252496+0000", 2020-08-07T15:07:19.418 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000102051 2020-08-07T15:07:19.419 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.419 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.419 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:19.419 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.252598+0000", 2020-08-07T15:07:19.419 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000170042 2020-08-07T15:07:19.419 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.420 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.420 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:19.420 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.252768+0000", 2020-08-07T15:07:19.420 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015645100000000001 2020-08-07T15:07:19.420 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.421 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.421 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:19.421 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.254333+0000", 2020-08-07T15:07:19.421 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9200000000000002e-07 2020-08-07T15:07:19.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.422 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:19.422 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.254333+0000", 2020-08-07T15:07:19.422 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0572999999999998e-05 2020-08-07T15:07:19.422 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.423 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:19.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.254383+0000", 2020-08-07T15:07:19.423 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001776716 2020-08-07T15:07:19.423 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.423 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.424 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:19.424 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.256160+0000", 2020-08-07T15:07:19.424 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21410052900000001 2020-08-07T15:07:19.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.424 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:19.425 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.470261+0000", 2020-08-07T15:07:19.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1069999999999999e-06 2020-08-07T15:07:19.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.425 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.426 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:19.426 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.470262+0000", 2020-08-07T15:07:19.426 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7484000000000006e-05 2020-08-07T15:07:19.426 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.426 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:19.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.470329+0000", 2020-08-07T15:07:19.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.218968882 2020-08-07T15:07:19.427 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.427 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:19.427 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.428 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:44120 3.e e32/20 3:7e6f7bb9:::benchmark_data_smithi074_35447_object5514:head v 32'2752, mlcod=32'2742)", 2020-08-07T15:07:19.428 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:04.453943+0000", 2020-08-07T15:07:19.428 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 14.767225669, 2020-08-07T15:07:19.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26204982500000001, 2020-08-07T15:07:19.429 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:19.429 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:19.429 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:19.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.430 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:19.430 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.453943+0000", 2020-08-07T15:07:19.430 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:19.430 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.430 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.431 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:19.431 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.453943+0000", 2020-08-07T15:07:19.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2278000000000003e-05 2020-08-07T15:07:19.431 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.431 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:19.432 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.453995+0000", 2020-08-07T15:07:19.432 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7050000000000002e-06 2020-08-07T15:07:19.432 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.432 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.432 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:19.433 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.454002+0000", 2020-08-07T15:07:19.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26083199299999998 2020-08-07T15:07:19.433 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.433 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.433 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:19.434 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.714834+0000", 2020-08-07T15:07:19.434 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3139999999999997e-06 2020-08-07T15:07:19.434 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.434 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.434 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:19.435 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.714838+0000", 2020-08-07T15:07:19.435 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8711000000000002e-05 2020-08-07T15:07:19.435 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.435 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.435 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:19.436 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.714887+0000", 2020-08-07T15:07:19.436 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7895e-05 2020-08-07T15:07:19.436 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.437 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.437 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:19.437 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.714915+0000", 2020-08-07T15:07:19.437 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010364 2020-08-07T15:07:19.437 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.438 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.438 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:19.438 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.715951+0000", 2020-08-07T15:07:19.438 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1529000000000002e-05 2020-08-07T15:07:19.438 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.438 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.439 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:19.439 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.715993+0000", 2020-08-07T15:07:19.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26204982500000001 2020-08-07T15:07:19.439 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.439 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:19.440 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.440 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.440 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:44152 3.6 3:6c90cb4b:::benchmark_data_smithi074_35447_object5518:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:07:19.440 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:04.469251+0000", 2020-08-07T15:07:19.441 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 14.751918002, 2020-08-07T15:07:19.441 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19376047599999999, 2020-08-07T15:07:19.441 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:19.441 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:19.441 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:19.442 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:19.442 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:19.442 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 44152 2020-08-07T15:07:19.442 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.442 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:19.442 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.443 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:19.443 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.469251+0000", 2020-08-07T15:07:19.443 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:19.443 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.443 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.444 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:19.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.469251+0000", 2020-08-07T15:07:19.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0423000000000002e-05 2020-08-07T15:07:19.444 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.445 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:19.445 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.469301+0000", 2020-08-07T15:07:19.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4989999999999999e-06 2020-08-07T15:07:19.445 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.445 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.446 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:19.446 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.469308+0000", 2020-08-07T15:07:19.446 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7082999999999999e-05 2020-08-07T15:07:19.446 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.446 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.446 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:19.447 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.469325+0000", 2020-08-07T15:07:19.447 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1630000000000001e-06 2020-08-07T15:07:19.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.447 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:19.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.469331+0000", 2020-08-07T15:07:19.448 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0040459349999999996 2020-08-07T15:07:19.448 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:19.449 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.473377+0000", 2020-08-07T15:07:19.449 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6240000000000001e-05 2020-08-07T15:07:19.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.449 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.449 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:19.450 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.473463+0000", 2020-08-07T15:07:19.450 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000168961 2020-08-07T15:07:19.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.450 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:19.450 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.473632+0000", 2020-08-07T15:07:19.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044376570000000002 2020-08-07T15:07:19.451 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.451 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.451 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:19.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.478070+0000", 2020-08-07T15:07:19.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00042399000000000001 2020-08-07T15:07:19.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:19.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.478494+0000", 2020-08-07T15:07:19.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.184384882 2020-08-07T15:07:19.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.453 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:19.453 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.662878+0000", 2020-08-07T15:07:19.454 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.066e-06 2020-08-07T15:07:19.454 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.454 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.454 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:19.454 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.662879+0000", 2020-08-07T15:07:19.454 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7883999999999997e-05 2020-08-07T15:07:19.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:19.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.662937+0000", 2020-08-07T15:07:19.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3693000000000001e-05 2020-08-07T15:07:19.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:19.456 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.663011+0000", 2020-08-07T15:07:19.456 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19376047599999999 2020-08-07T15:07:19.457 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.457 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:19.457 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.467 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.467 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:47336 3.8 e36/20 3:1652c683:::benchmark_data_smithi074_35447_object5916:head v 36'2992, mlcod=36'2984)", 2020-08-07T15:07:19.467 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:08.389243+0000", 2020-08-07T15:07:19.468 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 10.831925648, 2020-08-07T15:07:19.468 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22662983, 2020-08-07T15:07:19.468 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:19.468 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:19.468 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:19.469 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.469 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:19.469 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.389243+0000", 2020-08-07T15:07:19.469 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:19.469 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.470 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.470 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:19.470 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.389243+0000", 2020-08-07T15:07:19.470 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0009e-05 2020-08-07T15:07:19.470 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.471 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.471 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:19.471 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.389273+0000", 2020-08-07T15:07:19.471 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1219999999999997e-06 2020-08-07T15:07:19.471 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.472 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.472 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:19.472 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.389277+0000", 2020-08-07T15:07:19.472 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.225964585 2020-08-07T15:07:19.472 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.472 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.473 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:19.473 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.615242+0000", 2020-08-07T15:07:19.473 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3490000000000003e-06 2020-08-07T15:07:19.473 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.474 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.475 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:19.475 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.615249+0000", 2020-08-07T15:07:19.475 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5593000000000001e-05 2020-08-07T15:07:19.475 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.475 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.476 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:19.476 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.615305+0000", 2020-08-07T15:07:19.476 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4199e-05 2020-08-07T15:07:19.476 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.476 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.477 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:19.477 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.615329+0000", 2020-08-07T15:07:19.477 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00050547700000000005 2020-08-07T15:07:19.477 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.477 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.478 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:19.478 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.615834+0000", 2020-08-07T15:07:19.478 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8495999999999999e-05 2020-08-07T15:07:19.478 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.478 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.479 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:19.479 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.615873+0000", 2020-08-07T15:07:19.479 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22662983 2020-08-07T15:07:19.479 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.479 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:19.479 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.480 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.480 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:48144 3.0 3:0cec1d53:::benchmark_data_smithi074_35447_object6017:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e36)", 2020-08-07T15:07:19.480 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:08.892002+0000", 2020-08-07T15:07:19.480 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 10.329166149000001, 2020-08-07T15:07:19.481 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.257647242, 2020-08-07T15:07:19.481 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:19.481 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:19.481 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:19.481 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:19.482 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:19.482 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48144 2020-08-07T15:07:19.482 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.482 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:19.482 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.483 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:19.483 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.892002+0000", 2020-08-07T15:07:19.483 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:19.483 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.483 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.484 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:19.484 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.892002+0000", 2020-08-07T15:07:19.484 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5709999999999999e-05 2020-08-07T15:07:19.484 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.484 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.485 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:19.485 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.892028+0000", 2020-08-07T15:07:19.485 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9110000000000002e-06 2020-08-07T15:07:19.485 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.485 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.485 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:19.486 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.892031+0000", 2020-08-07T15:07:19.486 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3320000000000003e-06 2020-08-07T15:07:19.486 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.486 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.486 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:19.487 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.892038+0000", 2020-08-07T15:07:19.487 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.356e-06 2020-08-07T15:07:19.487 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.487 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.487 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:19.488 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.892041+0000", 2020-08-07T15:07:19.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045711179999999999 2020-08-07T15:07:19.488 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:19.488 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.896612+0000", 2020-08-07T15:07:19.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012007599999999999 2020-08-07T15:07:19.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.489 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.489 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:19.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.896732+0000", 2020-08-07T15:07:19.490 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014412499999999999 2020-08-07T15:07:19.490 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.490 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.490 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:07:19.490 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.896876+0000", 2020-08-07T15:07:19.491 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054785700000000003 2020-08-07T15:07:19.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.491 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.491 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:19.491 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.902355+0000", 2020-08-07T15:07:19.492 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1990000000000001e-06 2020-08-07T15:07:19.492 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.492 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.492 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:19.492 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.902356+0000", 2020-08-07T15:07:19.492 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4178999999999997e-05 2020-08-07T15:07:19.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.493 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:19.493 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.902420+0000", 2020-08-07T15:07:19.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.011879413 2020-08-07T15:07:19.494 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.494 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:19.494 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.914299+0000", 2020-08-07T15:07:19.494 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23528454700000001 2020-08-07T15:07:19.495 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.495 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.495 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:19.495 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.149584+0000", 2020-08-07T15:07:19.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1379999999999999e-06 2020-08-07T15:07:19.496 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.496 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.496 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:19.496 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.149585+0000", 2020-08-07T15:07:19.496 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4567999999999996e-05 2020-08-07T15:07:19.496 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.497 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.497 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:19.497 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.149650+0000", 2020-08-07T15:07:19.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.257647242 2020-08-07T15:07:19.497 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.498 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:19.498 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.498 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.498 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.498 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:48543 3.6 3:69105db2:::benchmark_data_smithi074_35447_object6067:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e36)", 2020-08-07T15:07:19.499 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:09.238441+0000", 2020-08-07T15:07:19.499 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 9.9827278600000007, 2020-08-07T15:07:19.499 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19472503899999999, 2020-08-07T15:07:19.499 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:19.499 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:19.500 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:19.500 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:19.500 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:19.500 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48543 2020-08-07T15:07:19.500 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.500 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:19.501 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.501 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:19.501 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.238441+0000", 2020-08-07T15:07:19.501 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:19.501 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.502 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.502 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:19.502 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.238441+0000", 2020-08-07T15:07:19.502 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2636000000000002e-05 2020-08-07T15:07:19.502 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.503 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.503 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:19.503 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.238493+0000", 2020-08-07T15:07:19.503 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.853e-06 2020-08-07T15:07:19.503 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.503 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.504 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:19.504 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.238498+0000", 2020-08-07T15:07:19.504 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0251000000000001e-05 2020-08-07T15:07:19.504 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.504 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.505 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:19.505 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.238508+0000", 2020-08-07T15:07:19.505 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3519999999999997e-06 2020-08-07T15:07:19.505 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.505 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.506 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:19.506 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.238514+0000", 2020-08-07T15:07:19.506 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001585033 2020-08-07T15:07:19.506 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.506 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.507 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:19.507 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.240099+0000", 2020-08-07T15:07:19.507 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00046183200000000002 2020-08-07T15:07:19.507 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.507 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.507 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:19.508 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.240561+0000", 2020-08-07T15:07:19.508 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012428 2020-08-07T15:07:19.508 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.508 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.508 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:19.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.240685+0000", 2020-08-07T15:07:19.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001339858 2020-08-07T15:07:19.509 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.509 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.509 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:19.510 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.242025+0000", 2020-08-07T15:07:19.510 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0031697930000000002 2020-08-07T15:07:19.510 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.510 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:19.510 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.245195+0000", 2020-08-07T15:07:19.511 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.187867386 2020-08-07T15:07:19.511 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:19.511 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.433062+0000", 2020-08-07T15:07:19.512 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0020000000000001e-06 2020-08-07T15:07:19.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.512 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.512 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:19.512 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.433063+0000", 2020-08-07T15:07:19.513 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5699999999999999e-05 2020-08-07T15:07:19.513 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.513 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.513 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:19.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.433119+0000", 2020-08-07T15:07:19.514 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7063e-05 2020-08-07T15:07:19.514 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.514 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:19.514 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.433166+0000", 2020-08-07T15:07:19.514 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19472503899999999 2020-08-07T15:07:19.515 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.515 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:19.515 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.515 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.515 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.516 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:48544 3.6 3:69105db2:::benchmark_data_smithi074_35447_object6067:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e36)", 2020-08-07T15:07:19.516 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:09.238546+0000", 2020-08-07T15:07:19.516 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 9.9826229709999996, 2020-08-07T15:07:19.516 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19636568900000001, 2020-08-07T15:07:19.516 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:19.517 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:19.517 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:19.517 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:19.517 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:19.517 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48544 2020-08-07T15:07:19.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.518 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:19.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.518 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:19.518 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.238546+0000", 2020-08-07T15:07:19.518 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:19.519 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.519 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.519 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:19.519 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.238546+0000", 2020-08-07T15:07:19.519 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6196000000000002e-05 2020-08-07T15:07:19.520 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.520 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:19.520 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.238602+0000", 2020-08-07T15:07:19.520 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3499999999999996e-06 2020-08-07T15:07:19.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.521 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.521 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:19.521 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.238607+0000", 2020-08-07T15:07:19.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0675e-05 2020-08-07T15:07:19.522 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.522 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.522 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:19.522 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.238618+0000", 2020-08-07T15:07:19.522 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5249999999999999e-06 2020-08-07T15:07:19.523 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.523 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.523 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:19.523 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.238622+0000", 2020-08-07T15:07:19.523 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026960259999999998 2020-08-07T15:07:19.523 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.524 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.524 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:19.524 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.241318+0000", 2020-08-07T15:07:19.524 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.4481000000000007e-05 2020-08-07T15:07:19.524 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.525 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.525 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:19.525 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.241403+0000", 2020-08-07T15:07:19.525 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014875000000000001 2020-08-07T15:07:19.525 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.526 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.526 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:19.526 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.241552+0000", 2020-08-07T15:07:19.526 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001261896 2020-08-07T15:07:19.526 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.527 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.527 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:19.527 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.242814+0000", 2020-08-07T15:07:19.527 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0051506360000000001 2020-08-07T15:07:19.527 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.528 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:19.528 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.247964+0000", 2020-08-07T15:07:19.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18684694700000001 2020-08-07T15:07:19.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.529 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:19.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.434811+0000", 2020-08-07T15:07:19.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6499999999999998e-07 2020-08-07T15:07:19.529 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.529 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.530 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:19.530 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.434812+0000", 2020-08-07T15:07:19.530 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2774999999999994e-05 2020-08-07T15:07:19.530 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.530 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.531 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:19.531 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.434875+0000", 2020-08-07T15:07:19.531 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6667e-05 2020-08-07T15:07:19.531 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.531 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.532 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:19.532 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.434911+0000", 2020-08-07T15:07:19.532 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19636568900000001 2020-08-07T15:07:19.532 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.532 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:19.532 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.533 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.533 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.533 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:48543 3.6 e36/26 ondisk, result = 0)", 2020-08-07T15:07:19.533 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:09.248155+0000", 2020-08-07T15:07:19.533 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 9.9730136379999994, 2020-08-07T15:07:19.534 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.185015337, 2020-08-07T15:07:19.534 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:19.534 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:19.534 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:19.534 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:19.535 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.248155+0000", 2020-08-07T15:07:19.535 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:19.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:19.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.248155+0000", 2020-08-07T15:07:19.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8473000000000001e-05 2020-08-07T15:07:19.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.536 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:19.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.248173+0000", 2020-08-07T15:07:19.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.286e-06 2020-08-07T15:07:19.537 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.537 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.538 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:19.538 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.248177+0000", 2020-08-07T15:07:19.538 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18478586299999999 2020-08-07T15:07:19.539 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.539 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.539 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:19.541 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.432963+0000", 2020-08-07T15:07:19.541 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1110000000000002e-06 2020-08-07T15:07:19.541 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.543 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:19.543 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.432970+0000", 2020-08-07T15:07:19.543 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1811000000000007e-05 2020-08-07T15:07:19.543 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.543 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.544 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:19.544 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.433051+0000", 2020-08-07T15:07:19.544 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.203e-06 2020-08-07T15:07:19.544 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.544 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:19.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.433057+0000", 2020-08-07T15:07:19.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011359 2020-08-07T15:07:19.545 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.545 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.546 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:19.546 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.433170+0000", 2020-08-07T15:07:19.546 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.185015337 2020-08-07T15:07:19.546 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.546 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:19.547 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.547 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.547 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:48544 3.6 e36/26 ondisk, result = 0)", 2020-08-07T15:07:19.547 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:09.248192+0000", 2020-08-07T15:07:19.547 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 9.9729761759999995, 2020-08-07T15:07:19.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18672297099999999, 2020-08-07T15:07:19.548 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:19.548 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:19.548 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:19.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.549 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:19.549 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.248192+0000", 2020-08-07T15:07:19.549 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:19.549 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.549 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.550 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:19.550 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.248192+0000", 2020-08-07T15:07:19.550 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7833e-05 2020-08-07T15:07:19.550 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.550 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.551 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:19.551 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.248210+0000", 2020-08-07T15:07:19.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.463e-06 2020-08-07T15:07:19.551 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.551 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.551 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:19.552 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.248213+0000", 2020-08-07T15:07:19.552 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18654061999999999 2020-08-07T15:07:19.552 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.552 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:19.553 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.434753+0000", 2020-08-07T15:07:19.553 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.536e-06 2020-08-07T15:07:19.553 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.553 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.553 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:19.554 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.434759+0000", 2020-08-07T15:07:19.554 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4131000000000002e-05 2020-08-07T15:07:19.554 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.554 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.555 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:19.555 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.434793+0000", 2020-08-07T15:07:19.555 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0590000000000003e-06 2020-08-07T15:07:19.555 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.555 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:19.556 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.434801+0000", 2020-08-07T15:07:19.556 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000114329 2020-08-07T15:07:19.556 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:19.557 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.434915+0000", 2020-08-07T15:07:19.557 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18672297099999999 2020-08-07T15:07:19.557 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.557 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:19.557 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.558 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.558 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.558 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:57048 3.e e38/20 3:7bf33003:::benchmark_data_smithi074_35447_object7130:head v 38'3712, mlcod=38'3705)", 2020-08-07T15:07:19.559 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:14.890141+0000", 2020-08-07T15:07:19.559 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4.3310273559999999, 2020-08-07T15:07:19.559 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18246897100000001, 2020-08-07T15:07:19.559 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:19.559 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:19.560 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:19.560 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.560 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:19.560 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.890141+0000", 2020-08-07T15:07:19.560 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:19.561 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.561 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.561 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:19.561 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.890141+0000", 2020-08-07T15:07:19.561 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4042999999999997e-05 2020-08-07T15:07:19.562 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.562 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.562 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:19.562 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.890205+0000", 2020-08-07T15:07:19.562 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6760000000000001e-06 2020-08-07T15:07:19.563 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.563 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.563 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:19.563 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.890212+0000", 2020-08-07T15:07:19.563 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.180730156 2020-08-07T15:07:19.563 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.564 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:19.564 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:15.070942+0000", 2020-08-07T15:07:19.564 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4750000000000001e-06 2020-08-07T15:07:19.564 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.565 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.565 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:19.565 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:15.070948+0000", 2020-08-07T15:07:19.565 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5519999999999999e-05 2020-08-07T15:07:19.565 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.566 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.567 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:19.567 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:15.070983+0000", 2020-08-07T15:07:19.567 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5560999999999999e-05 2020-08-07T15:07:19.567 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.567 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.568 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:19.568 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:15.070999+0000", 2020-08-07T15:07:19.568 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015684410000000001 2020-08-07T15:07:19.568 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.568 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.568 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:19.569 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:15.072567+0000", 2020-08-07T15:07:19.569 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3099000000000002e-05 2020-08-07T15:07:19.569 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.569 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.569 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:19.571 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:15.072610+0000", 2020-08-07T15:07:19.571 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18246897100000001 2020-08-07T15:07:19.571 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.572 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:19.572 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.572 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.572 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.572 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:57192 3.6 3:6245fd63:::benchmark_data_smithi074_35447_object7148:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e38)", 2020-08-07T15:07:19.573 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:14.970997+0000", 2020-08-07T15:07:19.573 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4.2501715569999998, 2020-08-07T15:07:19.573 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21165962799999999, 2020-08-07T15:07:19.573 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:19.573 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:19.574 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:19.574 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:19.574 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:19.574 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 57192 2020-08-07T15:07:19.574 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.575 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:19.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.575 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:19.575 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.970997+0000", 2020-08-07T15:07:19.575 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:19.576 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.576 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.576 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:19.576 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.970997+0000", 2020-08-07T15:07:19.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1773000000000002e-05 2020-08-07T15:07:19.576 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.577 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.577 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:19.577 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.971029+0000", 2020-08-07T15:07:19.577 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1859999999999997e-06 2020-08-07T15:07:19.577 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.578 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.578 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:19.578 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.971035+0000", 2020-08-07T15:07:19.578 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9565e-05 2020-08-07T15:07:19.578 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.579 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.579 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:19.579 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.971055+0000", 2020-08-07T15:07:19.579 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5090000000000001e-06 2020-08-07T15:07:19.579 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.579 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.580 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:19.580 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.971060+0000", 2020-08-07T15:07:19.580 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0035880170000000002 2020-08-07T15:07:19.580 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.580 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.581 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:19.581 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.974648+0000", 2020-08-07T15:07:19.581 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6166000000000001e-05 2020-08-07T15:07:19.581 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.581 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.582 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:19.582 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.974744+0000", 2020-08-07T15:07:19.582 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018205000000000001 2020-08-07T15:07:19.582 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.582 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.583 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:19.583 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.974926+0000", 2020-08-07T15:07:19.583 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0030785220000000002 2020-08-07T15:07:19.583 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.583 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.583 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:19.584 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.978005+0000", 2020-08-07T15:07:19.584 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2213999999999999e-05 2020-08-07T15:07:19.584 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.584 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.584 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:19.585 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.978047+0000", 2020-08-07T15:07:19.585 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9099999999999999e-05 2020-08-07T15:07:19.585 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.585 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.585 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:19.586 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.978116+0000", 2020-08-07T15:07:19.586 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00093295599999999998 2020-08-07T15:07:19.586 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.586 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.586 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:19.586 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:14.979049+0000", 2020-08-07T15:07:19.587 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20352266399999999 2020-08-07T15:07:19.587 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.587 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.587 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:19.587 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:15.182572+0000", 2020-08-07T15:07:19.588 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0359999999999999e-06 2020-08-07T15:07:19.588 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.588 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:19.588 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:15.182573+0000", 2020-08-07T15:07:19.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3869999999999995e-05 2020-08-07T15:07:19.589 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.589 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:19.589 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:15.182657+0000", 2020-08-07T15:07:19.590 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21165962799999999 2020-08-07T15:07:19.590 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.590 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:19.590 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.590 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.590 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.591 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:62544 3.6 3:65ad46f8:::benchmark_data_smithi074_35447_object7817:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e41)", 2020-08-07T15:07:19.591 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:18.599653+0000", 2020-08-07T15:07:19.591 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.62151509699999996, 2020-08-07T15:07:19.591 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18619424200000001, 2020-08-07T15:07:19.591 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:19.592 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:19.592 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:19.592 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:19.592 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:19.592 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 62544 2020-08-07T15:07:19.593 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.593 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:19.593 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.593 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:19.593 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.599653+0000", 2020-08-07T15:07:19.593 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:19.594 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.594 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.594 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:19.594 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.599653+0000", 2020-08-07T15:07:19.594 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1217999999999997e-05 2020-08-07T15:07:19.595 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.595 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.595 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:19.595 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.599695+0000", 2020-08-07T15:07:19.595 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3829999999999997e-06 2020-08-07T15:07:19.596 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.596 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.596 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:19.596 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.599701+0000", 2020-08-07T15:07:19.596 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2748999999999999e-05 2020-08-07T15:07:19.597 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.597 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.597 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:19.597 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.599714+0000", 2020-08-07T15:07:19.597 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0100000000000001e-06 2020-08-07T15:07:19.597 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.598 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.598 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:19.598 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.599720+0000", 2020-08-07T15:07:19.598 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0029246200000000002 2020-08-07T15:07:19.598 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.599 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.599 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:19.599 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.602644+0000", 2020-08-07T15:07:19.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8367000000000002e-05 2020-08-07T15:07:19.599 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.600 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.600 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:19.600 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.602743+0000", 2020-08-07T15:07:19.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5746000000000001e-05 2020-08-07T15:07:19.600 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.600 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.601 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:19.601 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.602839+0000", 2020-08-07T15:07:19.601 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003522187 2020-08-07T15:07:19.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.602 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:19.602 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.606361+0000", 2020-08-07T15:07:19.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6300000000000005e-07 2020-08-07T15:07:19.602 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.602 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.603 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:19.603 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.606361+0000", 2020-08-07T15:07:19.603 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9863000000000005e-05 2020-08-07T15:07:19.603 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.603 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:19.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.606431+0000", 2020-08-07T15:07:19.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00079531000000000003 2020-08-07T15:07:19.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:19.605 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.607227+0000", 2020-08-07T15:07:19.605 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17857304500000001 2020-08-07T15:07:19.605 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.605 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.605 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:19.606 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.785800+0000", 2020-08-07T15:07:19.606 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3400000000000001e-06 2020-08-07T15:07:19.606 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.606 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.606 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:19.606 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.785801+0000", 2020-08-07T15:07:19.607 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6841000000000001e-05 2020-08-07T15:07:19.607 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.607 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.607 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:19.607 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.785848+0000", 2020-08-07T15:07:19.608 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18619424200000001 2020-08-07T15:07:19.608 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.608 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:19.608 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.608 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.609 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:19.609 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:19.609 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:07:19.611 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 43 16 63296 63280 11.4961 10.2188 0.00634597 0.0108681 2020-08-07T15:07:19.611 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:19.609+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:19.612 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:07:19.728 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:19.710+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:19.816 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:19.814+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:19.817 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:19.817 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:07:19.818 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.818 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:64150 3.4 3:28fabc42:::benchmark_data_smithi074_35447_object8018:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e41)", 2020-08-07T15:07:19.818 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:19.773780+0000", 2020-08-07T15:07:19.818 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.042016335000000002, 2020-08-07T15:07:19.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.042050663000000002, 2020-08-07T15:07:19.819 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:19.819 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:19.819 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:19.819 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:19.819 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:19.820 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 64150 2020-08-07T15:07:19.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.820 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:19.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:19.821 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.773780+0000", 2020-08-07T15:07:19.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:19.821 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.842 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:19.842 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.773780+0000", 2020-08-07T15:07:19.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7709e-05 2020-08-07T15:07:19.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.844 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:19.844 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.773828+0000", 2020-08-07T15:07:19.844 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2510000000000002e-06 2020-08-07T15:07:19.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:19.845 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.773833+0000", 2020-08-07T15:07:19.845 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0579e-05 2020-08-07T15:07:19.845 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.846 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:19.846 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.773843+0000", 2020-08-07T15:07:19.846 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2749999999999997e-06 2020-08-07T15:07:19.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.847 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.847 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:19.847 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.773848+0000", 2020-08-07T15:07:19.847 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028712439999999998 2020-08-07T15:07:19.847 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.847 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.848 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:19.848 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.776719+0000", 2020-08-07T15:07:19.848 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0430000000000002e-05 2020-08-07T15:07:19.848 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.848 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.849 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:19.849 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.776779+0000", 2020-08-07T15:07:19.849 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019802700000000001 2020-08-07T15:07:19.849 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.849 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.850 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:07:19.850 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.776977+0000", 2020-08-07T15:07:19.850 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023463300000000002 2020-08-07T15:07:19.850 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.851 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:19.851 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.779324+0000", 2020-08-07T15:07:19.851 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6599999999999996e-07 2020-08-07T15:07:19.851 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.851 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.851 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:19.852 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.779324+0000", 2020-08-07T15:07:19.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8250999999999997e-05 2020-08-07T15:07:19.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:19.853 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.779363+0000", 2020-08-07T15:07:19.853 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005137356 2020-08-07T15:07:19.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.853 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:19.854 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.784500+0000", 2020-08-07T15:07:19.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.010720118000000001 2020-08-07T15:07:19.854 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.854 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:19.854 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.855 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:64151 3.4 3:28fabc42:::benchmark_data_smithi074_35447_object8018:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e41)", 2020-08-07T15:07:19.855 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:19.773971+0000", 2020-08-07T15:07:19.855 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.041825190999999998, 2020-08-07T15:07:19.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.041982945000000001, 2020-08-07T15:07:19.856 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:19.856 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:19.856 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:19.856 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:19.857 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:19.857 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 64151 2020-08-07T15:07:19.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.857 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:19.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:19.858 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.773971+0000", 2020-08-07T15:07:19.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:19.858 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.859 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:19.859 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.773971+0000", 2020-08-07T15:07:19.859 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7890999999999999e-05 2020-08-07T15:07:19.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.860 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:19.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.774019+0000", 2020-08-07T15:07:19.860 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2510000000000002e-06 2020-08-07T15:07:19.860 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.861 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:19.861 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.774024+0000", 2020-08-07T15:07:19.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0604e-05 2020-08-07T15:07:19.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.862 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.879 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:19.879 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.774035+0000", 2020-08-07T15:07:19.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0489999999999997e-06 2020-08-07T15:07:19.880 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.880 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:19.880 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.774039+0000", 2020-08-07T15:07:19.881 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044931169999999996 2020-08-07T15:07:19.881 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.881 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.882 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:19.882 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.778532+0000", 2020-08-07T15:07:19.882 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8253999999999998e-05 2020-08-07T15:07:19.882 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.883 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.883 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:19.883 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.778630+0000", 2020-08-07T15:07:19.883 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018510099999999999 2020-08-07T15:07:19.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.884 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:07:19.885 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.778815+0000", 2020-08-07T15:07:19.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0048752709999999996 2020-08-07T15:07:19.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.886 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.886 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:19.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.783691+0000", 2020-08-07T15:07:19.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.78e-07 2020-08-07T15:07:19.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.886 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:19.887 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.783692+0000", 2020-08-07T15:07:19.887 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9308999999999997e-05 2020-08-07T15:07:19.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:19.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.783761+0000", 2020-08-07T15:07:19.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00088781999999999999 2020-08-07T15:07:19.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:19.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.784649+0000", 2020-08-07T15:07:19.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.010677644999999999 2020-08-07T15:07:19.889 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.889 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:19.890 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.891 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.892 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:64152 3.4 3:28fabc42:::benchmark_data_smithi074_35447_object8018:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e41)", 2020-08-07T15:07:19.892 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:19.774073+0000", 2020-08-07T15:07:19.892 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.041722836999999999, 2020-08-07T15:07:19.893 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.041992351999999997, 2020-08-07T15:07:19.893 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:19.893 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:19.893 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:19.895 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:19.895 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:19.895 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 64152 2020-08-07T15:07:19.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.896 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:19.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.897 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:19.897 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.774073+0000", 2020-08-07T15:07:19.897 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:19.897 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.898 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.899 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:19.899 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.774073+0000", 2020-08-07T15:07:19.899 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1276000000000001e-05 2020-08-07T15:07:19.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.900 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:19.900 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.774135+0000", 2020-08-07T15:07:19.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8459999999999996e-06 2020-08-07T15:07:19.901 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.901 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.902 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:19.902 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.774140+0000", 2020-08-07T15:07:19.902 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0451999999999999e-05 2020-08-07T15:07:19.902 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.903 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:19.904 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.774151+0000", 2020-08-07T15:07:19.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0879999999999997e-06 2020-08-07T15:07:19.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.906 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:19.906 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.774155+0000", 2020-08-07T15:07:19.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0052363779999999999 2020-08-07T15:07:19.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:19.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.779391+0000", 2020-08-07T15:07:19.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3251000000000006e-05 2020-08-07T15:07:19.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:19.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.779485+0000", 2020-08-07T15:07:19.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019867600000000001 2020-08-07T15:07:19.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.910 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:07:19.910 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.779683+0000", 2020-08-07T15:07:19.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0040862750000000003 2020-08-07T15:07:19.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:19.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.783770+0000", 2020-08-07T15:07:19.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9200000000000001e-07 2020-08-07T15:07:19.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:19.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.783770+0000", 2020-08-07T15:07:19.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5278999999999997e-05 2020-08-07T15:07:19.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:19.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:19.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:19.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.783825+0000", 2020-08-07T15:07:19.914 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0097520130000000003 2020-08-07T15:07:19.915 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.915 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:19.915 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.915 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:19.915 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:07:19.916 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 3 2020-08-07T15:07:19.916 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:19.917 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:19.916+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:19.918 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:07:20.019 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:20.018+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:20.120 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:20.119+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:20.132 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:20.132 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:07:20.132 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:07:20.133 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:07:20.133 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:20.138 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:07:20.142 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:07:20.226 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:20.225+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:20.244 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 44 16 64704 64688 11.4848 11 0.134409 0.0108551 2020-08-07T15:07:20.293 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:07:20.310 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:20.311 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:07:20.311 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:07:20.311 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:07:20.311 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.312 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:1592 3.5 e23/20 3:ab93003a:::benchmark_data_smithi074_35447_object198:head v 23'88, mlcod=23'81)", 2020-08-07T15:07:20.312 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:37.295641+0000", 2020-08-07T15:07:20.312 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 43.009757964999999, 2020-08-07T15:07:20.312 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.127764135, 2020-08-07T15:07:20.312 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:20.313 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:20.313 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:20.313 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.313 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:20.314 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.295641+0000", 2020-08-07T15:07:20.314 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:20.314 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.314 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.314 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:20.315 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.295641+0000", 2020-08-07T15:07:20.315 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4150000000000003e-05 2020-08-07T15:07:20.315 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.315 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.315 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:20.316 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.295685+0000", 2020-08-07T15:07:20.316 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5971e-05 2020-08-07T15:07:20.316 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.316 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.316 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:20.317 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.295701+0000", 2020-08-07T15:07:20.317 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.12675505100000001 2020-08-07T15:07:20.317 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.317 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.317 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:20.318 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.422456+0000", 2020-08-07T15:07:20.318 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7000000000000002e-06 2020-08-07T15:07:20.318 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.318 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.318 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:20.319 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.422460+0000", 2020-08-07T15:07:20.319 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3812e-05 2020-08-07T15:07:20.319 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.319 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.319 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:20.320 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.422504+0000", 2020-08-07T15:07:20.320 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3331000000000001e-05 2020-08-07T15:07:20.320 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.320 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.320 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:20.321 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.422527+0000", 2020-08-07T15:07:20.321 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00082727300000000001 2020-08-07T15:07:20.321 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.321 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.321 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:20.322 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.423354+0000", 2020-08-07T15:07:20.322 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0847000000000003e-05 2020-08-07T15:07:20.322 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.322 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.322 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:20.323 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.423405+0000", 2020-08-07T15:07:20.323 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.127764135 2020-08-07T15:07:20.323 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.323 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:20.323 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.323 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.324 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.324 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:7848 3.9 e24/20 3:905b1512:::benchmark_data_smithi074_35447_object980:head v 24'408, mlcod=24'399)", 2020-08-07T15:07:20.324 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.898821+0000", 2020-08-07T15:07:20.324 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 39.406577839000001, 2020-08-07T15:07:20.324 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.182786849, 2020-08-07T15:07:20.325 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:20.325 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:20.325 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:20.325 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.325 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:20.326 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.898821+0000", 2020-08-07T15:07:20.326 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:20.326 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.326 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.326 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:20.327 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.898821+0000", 2020-08-07T15:07:20.327 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5016000000000002e-05 2020-08-07T15:07:20.327 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.327 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.327 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:20.328 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.898886+0000", 2020-08-07T15:07:20.328 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2740000000000007e-06 2020-08-07T15:07:20.328 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.328 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.328 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:20.328 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.898894+0000", 2020-08-07T15:07:20.329 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.181521612 2020-08-07T15:07:20.329 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.329 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.330 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:20.331 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.080416+0000", 2020-08-07T15:07:20.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8980000000000004e-06 2020-08-07T15:07:20.331 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.331 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.332 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:20.332 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.080421+0000", 2020-08-07T15:07:20.332 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6742000000000001e-05 2020-08-07T15:07:20.332 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.333 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.333 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:20.333 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.080468+0000", 2020-08-07T15:07:20.334 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3512000000000001e-05 2020-08-07T15:07:20.334 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.334 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.334 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:20.334 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.080491+0000", 2020-08-07T15:07:20.335 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010625750000000001 2020-08-07T15:07:20.335 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.335 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:20.335 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.081554+0000", 2020-08-07T15:07:20.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4219999999999999e-05 2020-08-07T15:07:20.336 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.336 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.336 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:20.336 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.081608+0000", 2020-08-07T15:07:20.337 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.182786849 2020-08-07T15:07:20.337 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.337 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:20.337 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.337 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.338 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.338 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:8112 3.9 e24/20 3:9b275d48:::benchmark_data_smithi074_35447_object1013:head v 24'416, mlcod=24'408)", 2020-08-07T15:07:20.338 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.085042+0000", 2020-08-07T15:07:20.338 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 39.220357386000003, 2020-08-07T15:07:20.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31443637000000002, 2020-08-07T15:07:20.339 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:20.339 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:20.339 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:20.339 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.340 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:20.340 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085042+0000", 2020-08-07T15:07:20.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:20.340 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.341 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:20.341 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085042+0000", 2020-08-07T15:07:20.341 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4534000000000003e-05 2020-08-07T15:07:20.341 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.342 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:20.342 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085096+0000", 2020-08-07T15:07:20.342 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5789999999999998e-06 2020-08-07T15:07:20.342 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.342 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.343 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:20.343 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085104+0000", 2020-08-07T15:07:20.343 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31316244700000001 2020-08-07T15:07:20.343 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.343 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.344 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:20.344 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398266+0000", 2020-08-07T15:07:20.344 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3019999999999996e-06 2020-08-07T15:07:20.344 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.344 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.345 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:20.345 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398270+0000", 2020-08-07T15:07:20.345 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3467000000000002e-05 2020-08-07T15:07:20.345 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.346 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.346 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:20.346 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398314+0000", 2020-08-07T15:07:20.347 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8184000000000001e-05 2020-08-07T15:07:20.347 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.347 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:20.348 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398332+0000", 2020-08-07T15:07:20.349 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010908059999999999 2020-08-07T15:07:20.349 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.349 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.350 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:20.350 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.399423+0000", 2020-08-07T15:07:20.350 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5050999999999999e-05 2020-08-07T15:07:20.350 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.350 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.351 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:20.351 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.399478+0000", 2020-08-07T15:07:20.351 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31443637000000002 2020-08-07T15:07:20.351 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.351 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:20.352 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.352 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.352 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.352 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:8760 3.9 e24/20 3:9f02d263:::benchmark_data_smithi074_35447_object1094:head v 24'448, mlcod=24'441)", 2020-08-07T15:07:20.352 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.535293+0000", 2020-08-07T15:07:20.353 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 38.770106194, 2020-08-07T15:07:20.353 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.194025799, 2020-08-07T15:07:20.353 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:20.353 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:20.354 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:20.354 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.355 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:20.355 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.535293+0000", 2020-08-07T15:07:20.355 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:20.355 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.355 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.356 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:20.356 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.535293+0000", 2020-08-07T15:07:20.356 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1352e-05 2020-08-07T15:07:20.356 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.356 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.357 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:20.357 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.535334+0000", 2020-08-07T15:07:20.357 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6660000000000002e-06 2020-08-07T15:07:20.357 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.358 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.358 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:20.358 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.535341+0000", 2020-08-07T15:07:20.358 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19233371499999999 2020-08-07T15:07:20.359 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.359 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.359 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:20.359 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.727675+0000", 2020-08-07T15:07:20.359 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1819999999999997e-06 2020-08-07T15:07:20.360 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.360 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.360 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:20.360 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.727680+0000", 2020-08-07T15:07:20.361 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0288999999999999e-05 2020-08-07T15:07:20.361 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.361 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.361 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:20.362 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.727730+0000", 2020-08-07T15:07:20.362 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0340000000000001e-05 2020-08-07T15:07:20.362 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.362 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.363 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:20.363 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.727760+0000", 2020-08-07T15:07:20.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015059489999999999 2020-08-07T15:07:20.363 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:20.364 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.729266+0000", 2020-08-07T15:07:20.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2305999999999997e-05 2020-08-07T15:07:20.365 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.365 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.366 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:20.366 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.729319+0000", 2020-08-07T15:07:20.366 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.194025799 2020-08-07T15:07:20.366 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.366 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:20.367 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.367 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.367 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.367 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:16223 3.9 e27/20 3:9c72cb85:::benchmark_data_smithi074_35447_object2027:head v 27'887, mlcod=27'881)", 2020-08-07T15:07:20.367 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.111666+0000", 2020-08-07T15:07:20.368 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 33.193732533999999, 2020-08-07T15:07:20.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.48138790999999997, 2020-08-07T15:07:20.368 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:20.368 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:20.368 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:20.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.369 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:20.369 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111666+0000", 2020-08-07T15:07:20.369 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:20.370 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.370 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.370 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:20.370 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111666+0000", 2020-08-07T15:07:20.370 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3168999999999999e-05 2020-08-07T15:07:20.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.371 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.371 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:20.371 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111720+0000", 2020-08-07T15:07:20.371 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6309999999999997e-06 2020-08-07T15:07:20.372 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.372 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:20.372 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111726+0000", 2020-08-07T15:07:20.372 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.47995092 2020-08-07T15:07:20.373 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.373 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.373 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:20.373 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591677+0000", 2020-08-07T15:07:20.373 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9879999999999998e-06 2020-08-07T15:07:20.374 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.374 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.374 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:20.374 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591684+0000", 2020-08-07T15:07:20.374 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5562000000000002e-05 2020-08-07T15:07:20.374 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.375 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:20.376 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591730+0000", 2020-08-07T15:07:20.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4731000000000001e-05 2020-08-07T15:07:20.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:20.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591754+0000", 2020-08-07T15:07:20.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012412739999999999 2020-08-07T15:07:20.377 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:20.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.592996+0000", 2020-08-07T15:07:20.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8634999999999997e-05 2020-08-07T15:07:20.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:20.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.593054+0000", 2020-08-07T15:07:20.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.48138790999999997 2020-08-07T15:07:20.379 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.380 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:20.380 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.380 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:16224 3.9 e27/20 3:9c72cb85:::benchmark_data_smithi074_35447_object2027:head v 27'888, mlcod=27'881)", 2020-08-07T15:07:20.381 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.114073+0000", 2020-08-07T15:07:20.381 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 33.191325806999998, 2020-08-07T15:07:20.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.50478892600000003, 2020-08-07T15:07:20.381 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:20.381 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:20.381 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:20.382 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.382 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:20.382 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114073+0000", 2020-08-07T15:07:20.382 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:20.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.383 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.383 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:20.383 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114073+0000", 2020-08-07T15:07:20.383 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3088999999999999e-05 2020-08-07T15:07:20.383 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.384 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.384 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:20.384 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114126+0000", 2020-08-07T15:07:20.384 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1629999999999999e-06 2020-08-07T15:07:20.384 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.384 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.385 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:20.385 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114133+0000", 2020-08-07T15:07:20.385 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.50357425199999994 2020-08-07T15:07:20.385 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.385 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.386 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:20.386 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617708+0000", 2020-08-07T15:07:20.387 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9160000000000004e-06 2020-08-07T15:07:20.387 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.387 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.387 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:20.387 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617715+0000", 2020-08-07T15:07:20.388 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8770999999999998e-05 2020-08-07T15:07:20.388 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.388 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.388 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:20.390 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617763+0000", 2020-08-07T15:07:20.390 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.544e-05 2020-08-07T15:07:20.390 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.390 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.391 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:20.391 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617789+0000", 2020-08-07T15:07:20.391 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010210359999999999 2020-08-07T15:07:20.391 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.391 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.392 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:20.392 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.618810+0000", 2020-08-07T15:07:20.392 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2259000000000002e-05 2020-08-07T15:07:20.392 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.404 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.404 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:20.405 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.618862+0000", 2020-08-07T15:07:20.405 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.50478892600000003 2020-08-07T15:07:20.405 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.405 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:20.406 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.406 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.406 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.407 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:16440 3.4 3:21d0104c:::benchmark_data_smithi074_35447_object2054:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:20.407 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.248910+0000", 2020-08-07T15:07:20.407 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 33.056489276000001, 2020-08-07T15:07:20.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.145337039, 2020-08-07T15:07:20.408 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:20.408 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:20.408 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:20.408 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:20.408 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:20.409 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 16440 2020-08-07T15:07:20.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.409 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:20.409 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.409 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:20.410 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.248910+0000", 2020-08-07T15:07:20.410 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:20.410 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.410 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.410 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:20.411 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.248910+0000", 2020-08-07T15:07:20.411 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1525e-05 2020-08-07T15:07:20.411 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.411 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.411 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:20.412 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.248951+0000", 2020-08-07T15:07:20.412 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0839999999999997e-06 2020-08-07T15:07:20.412 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.412 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.412 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:20.412 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.248957+0000", 2020-08-07T15:07:20.413 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.4610000000000008e-06 2020-08-07T15:07:20.413 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.413 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:20.413 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.248966+0000", 2020-08-07T15:07:20.414 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1290000000000004e-06 2020-08-07T15:07:20.414 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.414 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.414 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:20.414 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.248971+0000", 2020-08-07T15:07:20.415 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019651790000000001 2020-08-07T15:07:20.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.415 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:20.415 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.250936+0000", 2020-08-07T15:07:20.416 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5055000000000006e-05 2020-08-07T15:07:20.416 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.416 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.416 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:20.416 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.251011+0000", 2020-08-07T15:07:20.416 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000120529 2020-08-07T15:07:20.417 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.417 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.418 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:07:20.418 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.251132+0000", 2020-08-07T15:07:20.418 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001578404 2020-08-07T15:07:20.418 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.418 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.419 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:20.419 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.252710+0000", 2020-08-07T15:07:20.419 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3300000000000002e-07 2020-08-07T15:07:20.419 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.419 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.420 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:20.420 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.252711+0000", 2020-08-07T15:07:20.420 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8016000000000002e-05 2020-08-07T15:07:20.420 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.420 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.420 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:20.421 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.252769+0000", 2020-08-07T15:07:20.421 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028140320000000002 2020-08-07T15:07:20.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.421 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.421 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:20.422 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.255583+0000", 2020-08-07T15:07:20.422 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13859926 2020-08-07T15:07:20.422 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.422 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:20.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.394182+0000", 2020-08-07T15:07:20.423 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5209999999999999e-06 2020-08-07T15:07:20.423 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.423 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.423 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:20.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.394184+0000", 2020-08-07T15:07:20.424 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3211000000000002e-05 2020-08-07T15:07:20.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.424 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.424 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:20.424 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.394247+0000", 2020-08-07T15:07:20.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.145337039 2020-08-07T15:07:20.425 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.425 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:20.425 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.426 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.426 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.426 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:24992 3.0 e28/20 3:008a8fe9:::benchmark_data_smithi074_35447_object3123:head v 28'1704, mlcod=28'1694)", 2020-08-07T15:07:20.426 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.852063+0000", 2020-08-07T15:07:20.426 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 27.453335672000001, 2020-08-07T15:07:20.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20540171199999999, 2020-08-07T15:07:20.427 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:20.427 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:20.427 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:20.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:20.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.852063+0000", 2020-08-07T15:07:20.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:20.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.429 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:20.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.852063+0000", 2020-08-07T15:07:20.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1409999999999997e-05 2020-08-07T15:07:20.430 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.430 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.430 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:20.430 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.852115+0000", 2020-08-07T15:07:20.430 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6100000000000002e-06 2020-08-07T15:07:20.431 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.431 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:20.431 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.852121+0000", 2020-08-07T15:07:20.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20455925999999999 2020-08-07T15:07:20.431 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.432 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.432 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:20.432 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.056681+0000", 2020-08-07T15:07:20.432 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8739999999999995e-06 2020-08-07T15:07:20.432 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.433 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.433 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:20.433 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.056689+0000", 2020-08-07T15:07:20.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2753999999999998e-05 2020-08-07T15:07:20.433 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.434 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.434 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:20.434 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.056731+0000", 2020-08-07T15:07:20.434 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9171e-05 2020-08-07T15:07:20.434 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.434 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.435 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:20.435 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.056750+0000", 2020-08-07T15:07:20.435 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00065542300000000003 2020-08-07T15:07:20.435 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.435 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.437 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:20.437 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.057406+0000", 2020-08-07T15:07:20.437 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9209999999999997e-05 2020-08-07T15:07:20.438 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.438 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.438 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:20.439 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.057465+0000", 2020-08-07T15:07:20.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20540171199999999 2020-08-07T15:07:20.439 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.439 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:20.439 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.440 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.440 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:27424 3.4 3:279cfe9c:::benchmark_data_smithi074_35447_object3427:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:20.440 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:54.338304+0000", 2020-08-07T15:07:20.440 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 25.967094843999998, 2020-08-07T15:07:20.441 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.12276002699999999, 2020-08-07T15:07:20.441 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:20.441 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:20.441 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:20.441 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:20.442 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:20.442 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 27424 2020-08-07T15:07:20.442 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.442 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:20.442 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.443 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:20.443 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.338304+0000", 2020-08-07T15:07:20.443 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:20.443 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.443 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.443 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:20.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.338304+0000", 2020-08-07T15:07:20.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9563e-05 2020-08-07T15:07:20.444 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.444 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:20.445 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.338354+0000", 2020-08-07T15:07:20.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6999999999999996e-06 2020-08-07T15:07:20.445 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.445 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.445 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:20.445 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.338359+0000", 2020-08-07T15:07:20.446 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1476e-05 2020-08-07T15:07:20.446 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.446 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.446 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:20.446 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.338371+0000", 2020-08-07T15:07:20.447 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7350000000000001e-06 2020-08-07T15:07:20.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.447 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:20.447 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.338377+0000", 2020-08-07T15:07:20.448 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0016087460000000001 2020-08-07T15:07:20.448 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:20.450 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.339985+0000", 2020-08-07T15:07:20.450 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0984999999999997e-05 2020-08-07T15:07:20.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.451 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.451 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:20.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.340076+0000", 2020-08-07T15:07:20.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017277500000000001 2020-08-07T15:07:20.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:07:20.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.340249+0000", 2020-08-07T15:07:20.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0025273549999999998 2020-08-07T15:07:20.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.453 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:20.453 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.342777+0000", 2020-08-07T15:07:20.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.187e-06 2020-08-07T15:07:20.454 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.454 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.454 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:20.454 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.342778+0000", 2020-08-07T15:07:20.454 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3108999999999995e-05 2020-08-07T15:07:20.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:20.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.342841+0000", 2020-08-07T15:07:20.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9672999999999994e-05 2020-08-07T15:07:20.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:20.456 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.342920+0000", 2020-08-07T15:07:20.456 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.1180755 2020-08-07T15:07:20.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:20.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.460996+0000", 2020-08-07T15:07:20.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.569999999999999e-07 2020-08-07T15:07:20.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.458 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.458 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:20.458 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.460997+0000", 2020-08-07T15:07:20.458 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7266000000000003e-05 2020-08-07T15:07:20.458 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:20.459 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.461064+0000", 2020-08-07T15:07:20.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.12276002699999999 2020-08-07T15:07:20.460 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.460 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:20.460 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.461 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.461 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:27912 3.4 3:2bcd4c15:::benchmark_data_smithi074_35447_object3488:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:20.461 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:54.598926+0000", 2020-08-07T15:07:20.461 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 25.706472876999999, 2020-08-07T15:07:20.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13448369399999999, 2020-08-07T15:07:20.462 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:20.462 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:20.462 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:20.462 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:20.462 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:20.463 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 27912 2020-08-07T15:07:20.463 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.463 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:20.463 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.463 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:20.463 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.598926+0000", 2020-08-07T15:07:20.464 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:20.464 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.464 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.464 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:20.464 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.598926+0000", 2020-08-07T15:07:20.465 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5176000000000003e-05 2020-08-07T15:07:20.465 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.465 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.465 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:20.465 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.598971+0000", 2020-08-07T15:07:20.466 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1919999999999998e-06 2020-08-07T15:07:20.467 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.467 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.467 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:20.467 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.598976+0000", 2020-08-07T15:07:20.467 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.105e-06 2020-08-07T15:07:20.468 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.468 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.468 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:20.468 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.598982+0000", 2020-08-07T15:07:20.468 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7650000000000002e-06 2020-08-07T15:07:20.469 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.469 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.469 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:20.470 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.598984+0000", 2020-08-07T15:07:20.470 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003429353 2020-08-07T15:07:20.470 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.470 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.470 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:20.471 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.602414+0000", 2020-08-07T15:07:20.471 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000102043 2020-08-07T15:07:20.471 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.471 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.471 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:20.472 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.602516+0000", 2020-08-07T15:07:20.472 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000175262 2020-08-07T15:07:20.472 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.472 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.472 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:07:20.473 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.602691+0000", 2020-08-07T15:07:20.473 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001460565 2020-08-07T15:07:20.473 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.473 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.473 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:20.474 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.604152+0000", 2020-08-07T15:07:20.484 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3499999999999996e-07 2020-08-07T15:07:20.485 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.485 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.485 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:20.485 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.604152+0000", 2020-08-07T15:07:20.485 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0321000000000001e-05 2020-08-07T15:07:20.486 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.486 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.486 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:20.486 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.604212+0000", 2020-08-07T15:07:20.487 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.026292044000000001 2020-08-07T15:07:20.487 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.487 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.487 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:20.487 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.630505+0000", 2020-08-07T15:07:20.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.102838783 2020-08-07T15:07:20.488 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:20.488 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.733343+0000", 2020-08-07T15:07:20.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.372e-06 2020-08-07T15:07:20.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.489 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.489 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:20.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.733345+0000", 2020-08-07T15:07:20.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5178000000000005e-05 2020-08-07T15:07:20.490 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.490 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.490 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:20.490 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.733410+0000", 2020-08-07T15:07:20.490 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13448369399999999 2020-08-07T15:07:20.491 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.491 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:20.491 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.495 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.495 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:34000 3.4 3:22a8b4e3:::benchmark_data_smithi074_35447_object4249:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-08-07T15:07:20.496 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:58.316850+0000", 2020-08-07T15:07:20.496 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 21.988548966, 2020-08-07T15:07:20.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19523959799999999, 2020-08-07T15:07:20.498 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:20.498 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:20.498 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:20.498 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:20.498 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:20.499 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 34000 2020-08-07T15:07:20.499 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.499 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:20.499 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.499 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:20.500 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316850+0000", 2020-08-07T15:07:20.500 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:20.500 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.500 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.500 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:20.500 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316850+0000", 2020-08-07T15:07:20.501 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7274999999999997e-05 2020-08-07T15:07:20.501 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.501 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.501 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:20.501 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316897+0000", 2020-08-07T15:07:20.502 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2249999999999999e-06 2020-08-07T15:07:20.502 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.502 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.502 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:20.502 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316903+0000", 2020-08-07T15:07:20.503 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0665e-05 2020-08-07T15:07:20.503 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.503 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.503 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:20.503 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316913+0000", 2020-08-07T15:07:20.503 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3089999999999997e-06 2020-08-07T15:07:20.504 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.504 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.505 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:20.505 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316918+0000", 2020-08-07T15:07:20.505 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041515279999999998 2020-08-07T15:07:20.505 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.505 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.506 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:20.506 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.321069+0000", 2020-08-07T15:07:20.506 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6084999999999999e-05 2020-08-07T15:07:20.506 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.506 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.507 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:20.507 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.321165+0000", 2020-08-07T15:07:20.507 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000185076 2020-08-07T15:07:20.507 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.507 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.507 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:07:20.508 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.321350+0000", 2020-08-07T15:07:20.508 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00079810799999999996 2020-08-07T15:07:20.508 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.508 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.508 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:20.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.322148+0000", 2020-08-07T15:07:20.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9299999999999997e-07 2020-08-07T15:07:20.509 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.509 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.509 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:20.510 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.322149+0000", 2020-08-07T15:07:20.510 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1654000000000003e-05 2020-08-07T15:07:20.510 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.510 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:20.511 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.322211+0000", 2020-08-07T15:07:20.511 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044003000000000002 2020-08-07T15:07:20.511 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:20.511 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.326611+0000", 2020-08-07T15:07:20.512 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18541316599999999 2020-08-07T15:07:20.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.512 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.512 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:20.512 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.512024+0000", 2020-08-07T15:07:20.513 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5290000000000001e-06 2020-08-07T15:07:20.513 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.513 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.513 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:20.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.512026+0000", 2020-08-07T15:07:20.513 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3985000000000005e-05 2020-08-07T15:07:20.514 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.514 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:20.514 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.512090+0000", 2020-08-07T15:07:20.515 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19523959799999999 2020-08-07T15:07:20.515 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.516 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:20.516 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.516 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.516 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.516 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:34000 3.4 e30/20 ondisk, result = 0)", 2020-08-07T15:07:20.517 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:58.324486+0000", 2020-08-07T15:07:20.517 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 21.980913038000001, 2020-08-07T15:07:20.517 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.187608989, 2020-08-07T15:07:20.517 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:20.517 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:20.518 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:20.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.518 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:20.518 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.324486+0000", 2020-08-07T15:07:20.518 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:20.519 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.519 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.519 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:20.519 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.324486+0000", 2020-08-07T15:07:20.519 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5866999999999999e-05 2020-08-07T15:07:20.520 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.520 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:20.520 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.324502+0000", 2020-08-07T15:07:20.520 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4119999999999999e-06 2020-08-07T15:07:20.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.521 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.521 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:20.521 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.324504+0000", 2020-08-07T15:07:20.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18677271500000001 2020-08-07T15:07:20.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.522 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.522 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:20.522 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.511277+0000", 2020-08-07T15:07:20.522 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5489999999999997e-06 2020-08-07T15:07:20.522 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.523 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.523 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:20.523 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.511283+0000", 2020-08-07T15:07:20.523 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00072544799999999996 2020-08-07T15:07:20.523 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.524 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.524 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:20.524 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.512008+0000", 2020-08-07T15:07:20.524 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7689999999999996e-06 2020-08-07T15:07:20.524 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.524 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.525 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:20.525 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.512016+0000", 2020-08-07T15:07:20.525 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9228999999999996e-05 2020-08-07T15:07:20.525 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.525 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.526 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:20.526 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.512095+0000", 2020-08-07T15:07:20.526 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.187608989 2020-08-07T15:07:20.526 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.526 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:20.527 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.527 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.527 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.527 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:34024 3.0 e30/20 3:0de6648d:::benchmark_data_smithi074_35447_object4252:head v 30'2224, mlcod=30'2219)", 2020-08-07T15:07:20.527 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:58.351656+0000", 2020-08-07T15:07:20.528 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 21.953742644999998, 2020-08-07T15:07:20.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.128381149, 2020-08-07T15:07:20.528 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:20.528 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:20.528 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:20.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.529 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:20.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.351656+0000", 2020-08-07T15:07:20.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:20.529 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.529 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.530 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:20.530 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.351656+0000", 2020-08-07T15:07:20.530 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.033e-05 2020-08-07T15:07:20.530 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.530 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.531 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:20.531 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.351717+0000", 2020-08-07T15:07:20.531 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5980000000000002e-06 2020-08-07T15:07:20.531 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.531 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.531 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:20.532 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.351723+0000", 2020-08-07T15:07:20.532 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.125838113 2020-08-07T15:07:20.532 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.532 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.532 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:20.533 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.477561+0000", 2020-08-07T15:07:20.533 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7879999999999995e-06 2020-08-07T15:07:20.533 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.533 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.533 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:20.534 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.477570+0000", 2020-08-07T15:07:20.534 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8072000000000003e-05 2020-08-07T15:07:20.534 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.534 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.534 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:20.534 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.477618+0000", 2020-08-07T15:07:20.535 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9921999999999999e-05 2020-08-07T15:07:20.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:20.535 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.477648+0000", 2020-08-07T15:07:20.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023362069999999999 2020-08-07T15:07:20.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.536 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:20.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.479984+0000", 2020-08-07T15:07:20.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3118999999999997e-05 2020-08-07T15:07:20.537 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.537 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.537 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:20.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.480038+0000", 2020-08-07T15:07:20.538 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.128381149 2020-08-07T15:07:20.538 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.538 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:20.538 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.539 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.539 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.539 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:34624 3.9 e30/20 3:946cdb13:::benchmark_data_smithi074_35447_object4327:head v 30'2016, mlcod=30'2009)", 2020-08-07T15:07:20.539 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:58.746375+0000", 2020-08-07T15:07:20.539 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 21.559024163, 2020-08-07T15:07:20.540 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28564679799999998, 2020-08-07T15:07:20.540 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:20.540 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:20.540 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:20.540 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.540 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:20.541 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746375+0000", 2020-08-07T15:07:20.541 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:20.541 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.541 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:20.542 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746375+0000", 2020-08-07T15:07:20.542 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3674000000000001e-05 2020-08-07T15:07:20.542 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.542 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.542 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:20.543 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746429+0000", 2020-08-07T15:07:20.543 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9349999999999997e-06 2020-08-07T15:07:20.543 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.543 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.543 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:20.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746435+0000", 2020-08-07T15:07:20.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28165817700000001 2020-08-07T15:07:20.545 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.546 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.546 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:20.546 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028094+0000", 2020-08-07T15:07:20.546 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.512e-06 2020-08-07T15:07:20.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.547 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:20.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028097+0000", 2020-08-07T15:07:20.547 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7904999999999997e-05 2020-08-07T15:07:20.547 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.548 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:20.548 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028135+0000", 2020-08-07T15:07:20.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6243e-05 2020-08-07T15:07:20.548 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.549 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:20.549 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028151+0000", 2020-08-07T15:07:20.549 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003818372 2020-08-07T15:07:20.549 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.549 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.550 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:20.550 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.031970+0000", 2020-08-07T15:07:20.550 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.198e-05 2020-08-07T15:07:20.550 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.550 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.551 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:20.551 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.032022+0000", 2020-08-07T15:07:20.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28564679799999998 2020-08-07T15:07:20.551 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.551 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:20.551 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.552 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.552 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:35880 3.9 e31/20 3:905cfca2:::benchmark_data_smithi074_35447_object4484:head v 31'2128, mlcod=31'2120)", 2020-08-07T15:07:20.552 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:59.454559+0000", 2020-08-07T15:07:20.552 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 20.850840351999999, 2020-08-07T15:07:20.553 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17558973899999999, 2020-08-07T15:07:20.553 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:20.553 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:20.553 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:20.553 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.554 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:20.554 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.454559+0000", 2020-08-07T15:07:20.554 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:20.554 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.554 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.555 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:20.556 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.454559+0000", 2020-08-07T15:07:20.556 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5383000000000002e-05 2020-08-07T15:07:20.556 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:20.557 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.454614+0000", 2020-08-07T15:07:20.557 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5780000000000004e-06 2020-08-07T15:07:20.557 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.558 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.558 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:20.558 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.454621+0000", 2020-08-07T15:07:20.558 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17497574099999999 2020-08-07T15:07:20.558 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.558 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.559 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:20.559 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.629596+0000", 2020-08-07T15:07:20.559 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.561e-06 2020-08-07T15:07:20.559 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.560 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.560 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:20.560 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.629601+0000", 2020-08-07T15:07:20.560 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8960000000000001e-05 2020-08-07T15:07:20.560 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.561 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.561 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:20.561 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.629630+0000", 2020-08-07T15:07:20.561 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7209000000000001e-05 2020-08-07T15:07:20.561 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.562 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.562 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:20.562 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.629647+0000", 2020-08-07T15:07:20.562 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00046137800000000003 2020-08-07T15:07:20.562 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.563 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.563 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:20.563 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.630108+0000", 2020-08-07T15:07:20.563 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9928999999999997e-05 2020-08-07T15:07:20.563 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.564 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:20.564 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.630148+0000", 2020-08-07T15:07:20.564 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17558973899999999 2020-08-07T15:07:20.564 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.564 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:20.565 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.565 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.565 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.565 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:48216 3.4 3:29f3c7f3:::benchmark_data_smithi074_35447_object6026:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e36)", 2020-08-07T15:07:20.565 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:08.937978+0000", 2020-08-07T15:07:20.566 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.367420931, 2020-08-07T15:07:20.566 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17762572800000001, 2020-08-07T15:07:20.566 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:20.566 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:20.566 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:20.567 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:20.567 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:20.567 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48216 2020-08-07T15:07:20.567 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.567 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:20.567 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.568 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:20.568 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.937978+0000", 2020-08-07T15:07:20.568 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:20.568 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.568 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.569 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:20.569 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.937978+0000", 2020-08-07T15:07:20.569 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9499999999999997e-05 2020-08-07T15:07:20.569 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.569 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.570 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:20.570 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.938028+0000", 2020-08-07T15:07:20.570 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3929999999999999e-06 2020-08-07T15:07:20.570 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.570 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.570 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:20.571 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.938033+0000", 2020-08-07T15:07:20.571 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0659000000000001e-05 2020-08-07T15:07:20.571 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.571 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.571 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:20.572 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.938044+0000", 2020-08-07T15:07:20.572 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9569999999999999e-06 2020-08-07T15:07:20.572 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.572 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.572 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:20.572 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.938050+0000", 2020-08-07T15:07:20.573 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00063126600000000001 2020-08-07T15:07:20.573 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.573 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.573 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:20.573 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.938681+0000", 2020-08-07T15:07:20.574 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5762999999999998e-05 2020-08-07T15:07:20.574 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.574 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.574 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:20.574 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.938777+0000", 2020-08-07T15:07:20.575 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015617199999999999 2020-08-07T15:07:20.575 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.575 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:07:20.575 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.938933+0000", 2020-08-07T15:07:20.575 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0018355559999999999 2020-08-07T15:07:20.576 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.576 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.576 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:20.576 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.940768+0000", 2020-08-07T15:07:20.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5700000000000001e-07 2020-08-07T15:07:20.577 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.577 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.577 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:20.577 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.940769+0000", 2020-08-07T15:07:20.577 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0513000000000002e-05 2020-08-07T15:07:20.578 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.578 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.578 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:20.578 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.940830+0000", 2020-08-07T15:07:20.578 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0029664779999999998 2020-08-07T15:07:20.579 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.579 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.579 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:20.579 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.943796+0000", 2020-08-07T15:07:20.579 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17176112800000001 2020-08-07T15:07:20.579 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.580 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.580 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:20.580 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.115557+0000", 2020-08-07T15:07:20.580 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0899999999999999e-06 2020-08-07T15:07:20.580 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.581 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.581 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:20.581 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.115558+0000", 2020-08-07T15:07:20.581 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5395999999999997e-05 2020-08-07T15:07:20.581 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.582 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.582 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:20.582 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.115604+0000", 2020-08-07T15:07:20.582 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17762572800000001 2020-08-07T15:07:20.582 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.583 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:20.583 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.583 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.584 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.585 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:53680 3.5 e37/20 3:a4c15fd7:::benchmark_data_smithi074_35447_object6709:head v 37'3120, mlcod=37'3111)", 2020-08-07T15:07:20.585 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:12.774724+0000", 2020-08-07T15:07:20.585 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 7.5306754119999999, 2020-08-07T15:07:20.585 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13222435299999999, 2020-08-07T15:07:20.586 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:20.586 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:20.586 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:20.586 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.586 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:20.587 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:12.774724+0000", 2020-08-07T15:07:20.587 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:20.587 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.587 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.587 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:20.587 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:12.774724+0000", 2020-08-07T15:07:20.588 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1218000000000003e-05 2020-08-07T15:07:20.588 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.588 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:20.588 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:12.774775+0000", 2020-08-07T15:07:20.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2790000000000004e-06 2020-08-07T15:07:20.589 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.589 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:20.589 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:12.774781+0000", 2020-08-07T15:07:20.590 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13128326600000001 2020-08-07T15:07:20.590 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.590 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.590 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:20.590 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:12.906064+0000", 2020-08-07T15:07:20.590 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:07:20.591 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.591 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.591 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:20.591 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:12.906069+0000", 2020-08-07T15:07:20.591 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9119000000000002e-05 2020-08-07T15:07:20.592 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.592 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.592 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:20.592 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:12.906118+0000", 2020-08-07T15:07:20.592 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4093000000000001e-05 2020-08-07T15:07:20.593 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.593 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.593 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:20.593 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:12.906143+0000", 2020-08-07T15:07:20.593 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00076309200000000005 2020-08-07T15:07:20.594 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.594 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.594 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:20.594 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:12.906906+0000", 2020-08-07T15:07:20.594 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2351999999999997e-05 2020-08-07T15:07:20.595 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.595 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.595 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:20.595 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:12.906948+0000", 2020-08-07T15:07:20.595 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13222435299999999 2020-08-07T15:07:20.596 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.596 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:20.596 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.596 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.596 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.596 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:61432 3.9 e40/20 3:97c7d025:::benchmark_data_smithi074_35447_object7678:head v 40'3592, mlcod=40'3585)", 2020-08-07T15:07:20.597 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:17.903150+0000", 2020-08-07T15:07:20.597 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 2.4022491819999998, 2020-08-07T15:07:20.597 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.15557183699999999, 2020-08-07T15:07:20.597 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:20.597 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:20.598 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:20.598 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.598 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:20.598 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.903150+0000", 2020-08-07T15:07:20.598 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:20.599 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.599 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.599 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:20.599 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.903150+0000", 2020-08-07T15:07:20.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9295999999999999e-05 2020-08-07T15:07:20.600 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.600 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.600 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:20.600 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.903179+0000", 2020-08-07T15:07:20.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3789999999999999e-06 2020-08-07T15:07:20.600 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.601 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:20.601 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:17.903184+0000", 2020-08-07T15:07:20.601 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.15427392600000001 2020-08-07T15:07:20.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.602 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.602 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:20.602 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.057457+0000", 2020-08-07T15:07:20.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.596e-06 2020-08-07T15:07:20.602 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.603 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:20.603 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.057464+0000", 2020-08-07T15:07:20.603 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1990000000000002e-05 2020-08-07T15:07:20.603 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:20.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.057516+0000", 2020-08-07T15:07:20.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6619999999999998e-05 2020-08-07T15:07:20.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.605 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:20.605 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.057553+0000", 2020-08-07T15:07:20.605 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011104559999999999 2020-08-07T15:07:20.605 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.605 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.606 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:20.606 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.058663+0000", 2020-08-07T15:07:20.606 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8573999999999999e-05 2020-08-07T15:07:20.606 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.606 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.606 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:20.607 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.058722+0000", 2020-08-07T15:07:20.607 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.15557183699999999 2020-08-07T15:07:20.607 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.607 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:20.607 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.608 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.608 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.608 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:63016 3.4 3:218829e8:::benchmark_data_smithi074_35447_object7876:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e41)", 2020-08-07T15:07:20.608 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:18.893754+0000", 2020-08-07T15:07:20.608 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 1.4116446549999999, 2020-08-07T15:07:20.609 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18301139899999999, 2020-08-07T15:07:20.609 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:20.609 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:20.609 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:20.609 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:20.609 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:20.610 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 63016 2020-08-07T15:07:20.610 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.610 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:20.610 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.610 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:20.611 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.893754+0000", 2020-08-07T15:07:20.611 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:20.611 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.611 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.611 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:20.611 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.893754+0000", 2020-08-07T15:07:20.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6196000000000001e-05 2020-08-07T15:07:20.612 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.612 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.612 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:20.612 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.893781+0000", 2020-08-07T15:07:20.613 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3400000000000002e-06 2020-08-07T15:07:20.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.613 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:20.613 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.893784+0000", 2020-08-07T15:07:20.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7850000000000003e-06 2020-08-07T15:07:20.614 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.614 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.614 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:20.614 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.893791+0000", 2020-08-07T15:07:20.615 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3210000000000001e-06 2020-08-07T15:07:20.615 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.615 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.615 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:20.615 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.893794+0000", 2020-08-07T15:07:20.615 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00053106699999999998 2020-08-07T15:07:20.616 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.616 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.616 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:20.616 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.894325+0000", 2020-08-07T15:07:20.616 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1616000000000001e-05 2020-08-07T15:07:20.617 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.617 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.617 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:20.617 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.894387+0000", 2020-08-07T15:07:20.617 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000104379 2020-08-07T15:07:20.618 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.618 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.618 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:07:20.618 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.894491+0000", 2020-08-07T15:07:20.618 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0018720900000000001 2020-08-07T15:07:20.618 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.619 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.619 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:20.619 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.896363+0000", 2020-08-07T15:07:20.619 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2999999999999999e-07 2020-08-07T15:07:20.619 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.620 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.620 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:20.620 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.896363+0000", 2020-08-07T15:07:20.620 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7368e-05 2020-08-07T15:07:20.620 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.621 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.621 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:20.621 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.896391+0000", 2020-08-07T15:07:20.621 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004266527 2020-08-07T15:07:20.621 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.622 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.622 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:20.622 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.900657+0000", 2020-08-07T15:07:20.622 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.176062467 2020-08-07T15:07:20.622 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.622 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.623 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:20.623 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.076720+0000", 2020-08-07T15:07:20.623 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.147e-06 2020-08-07T15:07:20.623 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.623 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.624 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:20.624 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.076721+0000", 2020-08-07T15:07:20.624 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4866e-05 2020-08-07T15:07:20.624 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.624 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.624 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:20.625 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.076766+0000", 2020-08-07T15:07:20.625 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18301139899999999 2020-08-07T15:07:20.625 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.625 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:20.625 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.626 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.626 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.626 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:63016 3.4 e41/20 ondisk, result = 0)", 2020-08-07T15:07:20.626 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:18.896305+0000", 2020-08-07T15:07:20.626 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 1.4090945189999999, 2020-08-07T15:07:20.627 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18046616200000001, 2020-08-07T15:07:20.627 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:20.627 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:20.627 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:20.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.627 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:20.628 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.896305+0000", 2020-08-07T15:07:20.628 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:20.628 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.628 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.628 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:20.629 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.896305+0000", 2020-08-07T15:07:20.629 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9275999999999999e-05 2020-08-07T15:07:20.629 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.629 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.629 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:20.630 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.896334+0000", 2020-08-07T15:07:20.630 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8609999999999997e-06 2020-08-07T15:07:20.630 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.630 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.630 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:20.630 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.896339+0000", 2020-08-07T15:07:20.631 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18032953400000001 2020-08-07T15:07:20.631 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.631 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.631 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:20.631 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.076668+0000", 2020-08-07T15:07:20.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.053e-06 2020-08-07T15:07:20.632 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.632 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.632 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:20.632 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.076672+0000", 2020-08-07T15:07:20.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3062999999999998e-05 2020-08-07T15:07:20.633 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.633 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.633 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:20.633 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.076705+0000", 2020-08-07T15:07:20.633 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6040000000000004e-06 2020-08-07T15:07:20.634 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.634 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.634 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:20.634 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.076713+0000", 2020-08-07T15:07:20.634 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7771e-05 2020-08-07T15:07:20.635 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:20.635 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:20.635 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:20.635 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.076771+0000", 2020-08-07T15:07:20.635 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18046616200000001 2020-08-07T15:07:20.635 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.636 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:20.636 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.636 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:20.636 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:20.636 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:20.640 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:07:20.640 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:07:20.640 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:07:20.641 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:07:20.641 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:07:20.641 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:07:20.641 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:07:20.641 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:07:20.642 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:07:20.642 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:07:20.642 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:07:20.642 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:07:20.643 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:07:20.643 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:07:20.643 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:07:20.643 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:07:20.643 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:07:20.644 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:20.637+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:20.739 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:20.737+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:20.840 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:20.839+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:20.851 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:20.852 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:20.864 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:07:20.941 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:20.940+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:21.049 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:21.046+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:21.093 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:21.094 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:21.110 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:07:21.161 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:21.160+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:21.244 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 45 16 66293 66277 11.5055 12.4141 0.00783517 0.01086 2020-08-07T15:07:21.269 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:21.267+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:21.336 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:21.336 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:21.348 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:07:21.394 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:21.392+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:21.464 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:21.463+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:21.541 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:21.541 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:21.553 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:07:21.594 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:21.592+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:21.670 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:21.669+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:21.766 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:21.767 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:21.771 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:21.770+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:21.780 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:07:21.875 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:21.873+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:21.974 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:21.974+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:22.009 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:22.009 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:22.015 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:07:22.088 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:22.086+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:22.189 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:22.188+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:22.199 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:22.199 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:22.211 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:07:22.245 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 46 16 67906 67890 11.5293 12.6016 0.0022519 0.010837 2020-08-07T15:07:22.290 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:22.288+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:22.388 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:22.389 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:22.391 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:22.390+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:22.401 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:07:22.491 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:22.491+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:22.585 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:22.586 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:22.596 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:07:22.630 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:22.591+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:22.645 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:07:22.646 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:07:22.697 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:22.696+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:22.798 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:22.796+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:22.813 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:22.814 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:22.824 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:07:22.879 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:07:22.898 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:22.897+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:22.915 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:07:22.954 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:07:22.991 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:07:22.999 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:22.998+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:23.018 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:23.018 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:23.025 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:07:23.033 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:07:23.071 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:07:23.100 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:23.098+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:23.101 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:07:23.140 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:07:23.151 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:07:23.357 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:07:23.358 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:07:23.358 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:07:23.358 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:07:23.370 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:07:23.370 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:07:23.370 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:07:23.373 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:23.373 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:07:23.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:07:23.374 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:23.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.374 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:8367 3.a 3:5e6bacde:::benchmark_data_smithi074_35447_object1045:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:23.375 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.276619+0000", 2020-08-07T15:07:23.375 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.938186494, 2020-08-07T15:07:23.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.141821844, 2020-08-07T15:07:23.375 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:23.375 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:23.375 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:23.376 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:23.376 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:23.376 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 8367 2020-08-07T15:07:23.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.376 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:23.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:23.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276619+0000", 2020-08-07T15:07:23.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:23.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:23.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276619+0000", 2020-08-07T15:07:23.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4978e-05 2020-08-07T15:07:23.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:23.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276644+0000", 2020-08-07T15:07:23.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3009999999999998e-06 2020-08-07T15:07:23.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:23.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276647+0000", 2020-08-07T15:07:23.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0319999999999998e-06 2020-08-07T15:07:23.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:23.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276654+0000", 2020-08-07T15:07:23.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7199999999999998e-06 2020-08-07T15:07:23.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:23.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276657+0000", 2020-08-07T15:07:23.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023605660000000001 2020-08-07T15:07:23.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:23.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.279017+0000", 2020-08-07T15:07:23.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0431000000000004e-05 2020-08-07T15:07:23.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:23.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.279088+0000", 2020-08-07T15:07:23.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016790300000000001 2020-08-07T15:07:23.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:23.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.279256+0000", 2020-08-07T15:07:23.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031354550000000001 2020-08-07T15:07:23.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:23.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.282391+0000", 2020-08-07T15:07:23.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9900000000000001e-07 2020-08-07T15:07:23.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:23.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.282392+0000", 2020-08-07T15:07:23.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5717999999999999e-05 2020-08-07T15:07:23.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:23.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.282427+0000", 2020-08-07T15:07:23.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00053778099999999998 2020-08-07T15:07:23.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:23.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.282965+0000", 2020-08-07T15:07:23.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13540729400000001 2020-08-07T15:07:23.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:23.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418372+0000", 2020-08-07T15:07:23.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9600000000000008e-07 2020-08-07T15:07:23.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:23.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418373+0000", 2020-08-07T15:07:23.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7269999999999998e-05 2020-08-07T15:07:23.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:23.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418441+0000", 2020-08-07T15:07:23.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.141821844 2020-08-07T15:07:23.393 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.393 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:23.393 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.394 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:8368 3.a 3:5e6bacde:::benchmark_data_smithi074_35447_object1045:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:23.394 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.276795+0000", 2020-08-07T15:07:23.394 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.938010767999998, 2020-08-07T15:07:23.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14181318200000001, 2020-08-07T15:07:23.395 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:23.395 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:23.395 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:23.395 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:23.395 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:23.396 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 8368 2020-08-07T15:07:23.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.396 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:23.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:23.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276795+0000", 2020-08-07T15:07:23.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:23.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:23.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276795+0000", 2020-08-07T15:07:23.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8793e-05 2020-08-07T15:07:23.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:23.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276823+0000", 2020-08-07T15:07:23.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3890000000000001e-06 2020-08-07T15:07:23.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:23.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276827+0000", 2020-08-07T15:07:23.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2590000000000003e-06 2020-08-07T15:07:23.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:23.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276834+0000", 2020-08-07T15:07:23.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7889999999999999e-06 2020-08-07T15:07:23.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:23.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.276837+0000", 2020-08-07T15:07:23.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030534199999999998 2020-08-07T15:07:23.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:23.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.279890+0000", 2020-08-07T15:07:23.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3449000000000006e-05 2020-08-07T15:07:23.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:23.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.279984+0000", 2020-08-07T15:07:23.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020054799999999999 2020-08-07T15:07:23.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:23.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.280184+0000", 2020-08-07T15:07:23.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002684353 2020-08-07T15:07:23.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:23.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.282869+0000", 2020-08-07T15:07:23.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7899999999999998e-07 2020-08-07T15:07:23.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:23.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.282869+0000", 2020-08-07T15:07:23.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5290000000000001e-05 2020-08-07T15:07:23.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:23.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.282924+0000", 2020-08-07T15:07:23.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031297999999999997 2020-08-07T15:07:23.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:23.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.283237+0000", 2020-08-07T15:07:23.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.135332799 2020-08-07T15:07:23.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:23.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418570+0000", 2020-08-07T15:07:23.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4600000000000004e-07 2020-08-07T15:07:23.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:23.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418571+0000", 2020-08-07T15:07:23.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6888000000000003e-05 2020-08-07T15:07:23.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:23.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418608+0000", 2020-08-07T15:07:23.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14181318200000001 2020-08-07T15:07:23.424 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.424 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:23.424 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.425 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:8367 3.a e24/20 ondisk, result = 0)", 2020-08-07T15:07:23.425 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.281490+0000", 2020-08-07T15:07:23.425 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.933315352999998, 2020-08-07T15:07:23.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13703254000000001, 2020-08-07T15:07:23.426 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:23.426 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:23.426 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:23.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:23.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.281490+0000", 2020-08-07T15:07:23.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:23.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:23.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.281490+0000", 2020-08-07T15:07:23.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5865000000000001e-05 2020-08-07T15:07:23.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:23.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.281506+0000", 2020-08-07T15:07:23.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5269999999999999e-06 2020-08-07T15:07:23.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:23.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.281509+0000", 2020-08-07T15:07:23.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13566299600000001 2020-08-07T15:07:23.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:23.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.417172+0000", 2020-08-07T15:07:23.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7019999999999998e-06 2020-08-07T15:07:23.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:23.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.417178+0000", 2020-08-07T15:07:23.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001176947 2020-08-07T15:07:23.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:23.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418355+0000", 2020-08-07T15:07:23.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6240000000000008e-06 2020-08-07T15:07:23.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:23.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418364+0000", 2020-08-07T15:07:23.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015887900000000001 2020-08-07T15:07:23.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:23.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.418523+0000", 2020-08-07T15:07:23.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13703254000000001 2020-08-07T15:07:23.444 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.444 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:23.444 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.445 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:10536 3.0 e24/20 3:0d011956:::benchmark_data_smithi074_35447_object1316:head v 24'744, mlcod=24'735)", 2020-08-07T15:07:23.445 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.564533+0000", 2020-08-07T15:07:23.445 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.650272846, 2020-08-07T15:07:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14762919499999999, 2020-08-07T15:07:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.564533+0000", 2020-08-07T15:07:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.564533+0000", 2020-08-07T15:07:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9255000000000002e-05 2020-08-07T15:07:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.564582+0000", 2020-08-07T15:07:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1309999999999997e-06 2020-08-07T15:07:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.564587+0000", 2020-08-07T15:07:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14241859400000001 2020-08-07T15:07:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.707006+0000", 2020-08-07T15:07:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2190000000000003e-06 2020-08-07T15:07:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.707011+0000", 2020-08-07T15:07:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7673999999999999e-05 2020-08-07T15:07:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:23.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.707048+0000", 2020-08-07T15:07:23.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7660999999999999e-05 2020-08-07T15:07:23.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:23.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.707066+0000", 2020-08-07T15:07:23.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050458969999999997 2020-08-07T15:07:23.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:23.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.712112+0000", 2020-08-07T15:07:23.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9764e-05 2020-08-07T15:07:23.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:23.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.712162+0000", 2020-08-07T15:07:23.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14762919499999999 2020-08-07T15:07:23.456 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.456 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:23.457 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.457 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:11192 3.d e24/20 3:bb990748:::benchmark_data_smithi074_35447_object1398:head v 24'680, mlcod=24'671)", 2020-08-07T15:07:23.457 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:43.006852+0000", 2020-08-07T15:07:23.458 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.207953670000002, 2020-08-07T15:07:23.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15926784399999999, 2020-08-07T15:07:23.458 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:23.458 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:23.458 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:23.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:23.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.006852+0000", 2020-08-07T15:07:23.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:23.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:23.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.006852+0000", 2020-08-07T15:07:23.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0237e-05 2020-08-07T15:07:23.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:23.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.006902+0000", 2020-08-07T15:07:23.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2750000000000001e-06 2020-08-07T15:07:23.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:23.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.006908+0000", 2020-08-07T15:07:23.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15839128599999999 2020-08-07T15:07:23.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:23.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.165300+0000", 2020-08-07T15:07:23.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4450000000000004e-06 2020-08-07T15:07:23.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:23.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.165305+0000", 2020-08-07T15:07:23.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3010999999999999e-05 2020-08-07T15:07:23.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:23.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.165348+0000", 2020-08-07T15:07:23.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3289e-05 2020-08-07T15:07:23.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:23.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.165371+0000", 2020-08-07T15:07:23.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00069550299999999998 2020-08-07T15:07:23.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:23.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.166067+0000", 2020-08-07T15:07:23.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2797999999999998e-05 2020-08-07T15:07:23.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:23.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.166120+0000", 2020-08-07T15:07:23.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15926784399999999 2020-08-07T15:07:23.468 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.469 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:23.469 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.469 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:17824 3.d e27/20 3:bd4e0935:::benchmark_data_smithi074_35447_object2227:head v 27'1168, mlcod=27'1163)", 2020-08-07T15:07:23.470 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:07:23.470 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.071365153000002, 2020-08-07T15:07:23.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25249224199999998, 2020-08-07T15:07:23.470 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:23.470 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:23.471 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:23.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:23.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:07:23.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:23.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:23.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:07:23.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9348000000000003e-05 2020-08-07T15:07:23.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:23.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143490+0000", 2020-08-07T15:07:23.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7800000000000003e-06 2020-08-07T15:07:23.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:23.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143496+0000", 2020-08-07T15:07:23.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25095783399999999 2020-08-07T15:07:23.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:23.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394454+0000", 2020-08-07T15:07:23.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6550000000000001e-06 2020-08-07T15:07:23.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:23.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394460+0000", 2020-08-07T15:07:23.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8814999999999999e-05 2020-08-07T15:07:23.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:23.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394509+0000", 2020-08-07T15:07:23.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3482999999999999e-05 2020-08-07T15:07:23.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:23.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394532+0000", 2020-08-07T15:07:23.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013496199999999999 2020-08-07T15:07:23.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:23.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.395882+0000", 2020-08-07T15:07:23.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0707000000000001e-05 2020-08-07T15:07:23.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:23.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.395932+0000", 2020-08-07T15:07:23.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25249224199999998 2020-08-07T15:07:23.481 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.481 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:23.481 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.482 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:18478 3.a 3:524412cd:::benchmark_data_smithi074_35447_object2309:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:23.482 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.592837+0000", 2020-08-07T15:07:23.482 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.621968074000002, 2020-08-07T15:07:23.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21005600699999999, 2020-08-07T15:07:23.482 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:23.483 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:23.483 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:23.483 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:23.483 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:23.483 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 18478 2020-08-07T15:07:23.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.484 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:23.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:23.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592837+0000", 2020-08-07T15:07:23.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:23.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:23.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592837+0000", 2020-08-07T15:07:23.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6718999999999998e-05 2020-08-07T15:07:23.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:23.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592884+0000", 2020-08-07T15:07:23.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2749999999999997e-06 2020-08-07T15:07:23.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:23.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592888+0000", 2020-08-07T15:07:23.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0831e-05 2020-08-07T15:07:23.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:23.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592899+0000", 2020-08-07T15:07:23.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9440000000000002e-06 2020-08-07T15:07:23.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:23.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592905+0000", 2020-08-07T15:07:23.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00088115000000000003 2020-08-07T15:07:23.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:23.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.593786+0000", 2020-08-07T15:07:23.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1789000000000002e-05 2020-08-07T15:07:23.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:23.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.593838+0000", 2020-08-07T15:07:23.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011097799999999999 2020-08-07T15:07:23.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:23.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.593949+0000", 2020-08-07T15:07:23.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00088680400000000004 2020-08-07T15:07:23.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:23.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.594836+0000", 2020-08-07T15:07:23.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1500000000000005e-07 2020-08-07T15:07:23.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:23.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.594836+0000", 2020-08-07T15:07:23.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9746000000000002e-05 2020-08-07T15:07:23.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:23.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.594876+0000", 2020-08-07T15:07:23.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045589070000000001 2020-08-07T15:07:23.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:23.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.599435+0000", 2020-08-07T15:07:23.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.203361244 2020-08-07T15:07:23.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:23.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802796+0000", 2020-08-07T15:07:23.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.178e-06 2020-08-07T15:07:23.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:23.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802797+0000", 2020-08-07T15:07:23.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5927000000000005e-05 2020-08-07T15:07:23.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:23.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802893+0000", 2020-08-07T15:07:23.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21005600699999999 2020-08-07T15:07:23.502 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.502 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:23.503 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.503 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:18478 3.a e27/20 ondisk, result = 0)", 2020-08-07T15:07:23.503 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.595603+0000", 2020-08-07T15:07:23.504 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.619202176000002, 2020-08-07T15:07:23.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.207367254, 2020-08-07T15:07:23.504 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:23.504 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:23.504 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:23.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:23.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.595603+0000", 2020-08-07T15:07:23.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:23.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:23.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.595603+0000", 2020-08-07T15:07:23.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8887999999999999e-05 2020-08-07T15:07:23.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:23.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.595622+0000", 2020-08-07T15:07:23.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8469999999999998e-06 2020-08-07T15:07:23.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:23.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.595626+0000", 2020-08-07T15:07:23.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20636312700000001 2020-08-07T15:07:23.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:23.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.801989+0000", 2020-08-07T15:07:23.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0889999999999999e-06 2020-08-07T15:07:23.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:23.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.801994+0000", 2020-08-07T15:07:23.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00078913099999999999 2020-08-07T15:07:23.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:23.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802783+0000", 2020-08-07T15:07:23.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8919999999999997e-06 2020-08-07T15:07:23.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:23.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802789+0000", 2020-08-07T15:07:23.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018128 2020-08-07T15:07:23.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:23.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802970+0000", 2020-08-07T15:07:23.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.207367254 2020-08-07T15:07:23.515 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.515 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:23.515 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.516 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:33488 3.c e30/20 3:3d1929c7:::benchmark_data_smithi074_35447_object4185:head v 30'2120, mlcod=30'2110)", 2020-08-07T15:07:23.516 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:58.023101+0000", 2020-08-07T15:07:23.516 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.191704385000001, 2020-08-07T15:07:23.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22118623300000001, 2020-08-07T15:07:23.516 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:23.517 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:23.517 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:23.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:23.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023101+0000", 2020-08-07T15:07:23.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:23.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:23.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023101+0000", 2020-08-07T15:07:23.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8794000000000001e-05 2020-08-07T15:07:23.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:23.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023150+0000", 2020-08-07T15:07:23.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.545e-06 2020-08-07T15:07:23.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:23.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023156+0000", 2020-08-07T15:07:23.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.218926757 2020-08-07T15:07:23.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:23.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242083+0000", 2020-08-07T15:07:23.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1200000000000004e-06 2020-08-07T15:07:23.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:23.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242087+0000", 2020-08-07T15:07:23.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9561999999999998e-05 2020-08-07T15:07:23.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:23.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242127+0000", 2020-08-07T15:07:23.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3655999999999998e-05 2020-08-07T15:07:23.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:23.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242150+0000", 2020-08-07T15:07:23.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002089273 2020-08-07T15:07:23.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:23.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.244240+0000", 2020-08-07T15:07:23.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7525999999999999e-05 2020-08-07T15:07:23.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:23.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.244287+0000", 2020-08-07T15:07:23.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22118623300000001 2020-08-07T15:07:23.527 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.527 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:23.527 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.528 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:35856 3.a 3:5b5556e8:::benchmark_data_smithi074_35447_object4481:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-08-07T15:07:23.528 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:59.440060+0000", 2020-08-07T15:07:23.528 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.774745329000002, 2020-08-07T15:07:23.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16587832199999999, 2020-08-07T15:07:23.529 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:23.529 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:23.529 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:23.529 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:23.529 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:23.530 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 35856 2020-08-07T15:07:23.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.533 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:23.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:23.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440060+0000", 2020-08-07T15:07:23.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:23.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:23.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440060+0000", 2020-08-07T15:07:23.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018524399999999999 2020-08-07T15:07:23.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:23.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440245+0000", 2020-08-07T15:07:23.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9519999999999999e-06 2020-08-07T15:07:23.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:23.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440251+0000", 2020-08-07T15:07:23.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1705999999999999e-05 2020-08-07T15:07:23.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:23.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440263+0000", 2020-08-07T15:07:23.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2820000000000004e-06 2020-08-07T15:07:23.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:23.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440268+0000", 2020-08-07T15:07:23.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023611449999999998 2020-08-07T15:07:23.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.442629+0000", 2020-08-07T15:07:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010644799999999999 2020-08-07T15:07:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:23.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.442736+0000", 2020-08-07T15:07:23.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017566000000000001 2020-08-07T15:07:23.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.442912+0000", 2020-08-07T15:07:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026960399999999998 2020-08-07T15:07:23.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:23.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.445608+0000", 2020-08-07T15:07:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0809999999999999e-06 2020-08-07T15:07:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.445609+0000", 2020-08-07T15:07:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0878999999999997e-05 2020-08-07T15:07:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.445680+0000", 2020-08-07T15:07:23.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00065560499999999997 2020-08-07T15:07:23.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:23.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.446335+0000", 2020-08-07T15:07:23.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.159534921 2020-08-07T15:07:23.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:23.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.605870+0000", 2020-08-07T15:07:23.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.187e-06 2020-08-07T15:07:23.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:23.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.605871+0000", 2020-08-07T15:07:23.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7172e-05 2020-08-07T15:07:23.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:23.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.605938+0000", 2020-08-07T15:07:23.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16587832199999999 2020-08-07T15:07:23.566 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.566 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:23.566 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.567 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:42104 3.d e32/20 3:bcc35439:::benchmark_data_smithi074_35447_object5262:head v 32'2864, mlcod=32'2858)", 2020-08-07T15:07:23.567 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:07:23.567 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.971779856000001, 2020-08-07T15:07:23.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28296418899999998, 2020-08-07T15:07:23.568 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:23.568 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:23.568 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:23.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:23.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:07:23.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:23.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:23.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:07:23.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5550000000000002e-05 2020-08-07T15:07:23.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:23.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243081+0000", 2020-08-07T15:07:23.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8139999999999997e-06 2020-08-07T15:07:23.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:23.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243087+0000", 2020-08-07T15:07:23.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27968446800000002 2020-08-07T15:07:23.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:23.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522771+0000", 2020-08-07T15:07:23.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9459999999999997e-06 2020-08-07T15:07:23.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:23.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522776+0000", 2020-08-07T15:07:23.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0778000000000006e-05 2020-08-07T15:07:23.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:23.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522847+0000", 2020-08-07T15:07:23.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0622e-05 2020-08-07T15:07:23.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:23.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522868+0000", 2020-08-07T15:07:23.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030627580000000001 2020-08-07T15:07:23.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:23.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.525930+0000", 2020-08-07T15:07:23.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9252999999999997e-05 2020-08-07T15:07:23.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:23.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.525990+0000", 2020-08-07T15:07:23.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28296418899999998 2020-08-07T15:07:23.580 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.580 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:23.580 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.581 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:42704 3.2 e32/20 3:435b82af:::benchmark_data_smithi074_35447_object5337:head v 32'2440, mlcod=32'2430)", 2020-08-07T15:07:23.581 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.590578+0000", 2020-08-07T15:07:23.581 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.62422737, 2020-08-07T15:07:23.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.156827727, 2020-08-07T15:07:23.582 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:23.582 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:23.582 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:23.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:23.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.590578+0000", 2020-08-07T15:07:23.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:23.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:23.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.590578+0000", 2020-08-07T15:07:23.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0890000000000002e-05 2020-08-07T15:07:23.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:23.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.590629+0000", 2020-08-07T15:07:23.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9349999999999999e-06 2020-08-07T15:07:23.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:23.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.590635+0000", 2020-08-07T15:07:23.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15515162199999999 2020-08-07T15:07:23.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:23.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.745786+0000", 2020-08-07T15:07:23.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6650000000000001e-06 2020-08-07T15:07:23.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:23.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.745790+0000", 2020-08-07T15:07:23.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1458999999999997e-05 2020-08-07T15:07:23.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:23.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.745832+0000", 2020-08-07T15:07:23.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7506000000000001e-05 2020-08-07T15:07:23.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:23.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.745849+0000", 2020-08-07T15:07:23.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001453008 2020-08-07T15:07:23.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:23.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.747302+0000", 2020-08-07T15:07:23.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010364199999999999 2020-08-07T15:07:23.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:23.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.747406+0000", 2020-08-07T15:07:23.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.156827727 2020-08-07T15:07:23.596 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.596 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:23.597 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.597 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:42944 3.2 e32/20 3:45b0b8d5:::benchmark_data_smithi074_35447_object5367:head v 32'2464, mlcod=32'2452)", 2020-08-07T15:07:23.597 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.763245+0000", 2020-08-07T15:07:23.599 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.451560889, 2020-08-07T15:07:23.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.154831946, 2020-08-07T15:07:23.599 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:23.599 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:23.599 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:23.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:23.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.763245+0000", 2020-08-07T15:07:23.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:23.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:23.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.763245+0000", 2020-08-07T15:07:23.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1844e-05 2020-08-07T15:07:23.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:23.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.763296+0000", 2020-08-07T15:07:23.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4489999999999996e-06 2020-08-07T15:07:23.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:23.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.763303+0000", 2020-08-07T15:07:23.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.150368538 2020-08-07T15:07:23.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:23.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.913671+0000", 2020-08-07T15:07:23.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8499999999999998e-06 2020-08-07T15:07:23.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:23.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.913674+0000", 2020-08-07T15:07:23.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002168096 2020-08-07T15:07:23.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:23.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.915842+0000", 2020-08-07T15:07:23.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1338000000000001e-05 2020-08-07T15:07:23.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:23.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.915864+0000", 2020-08-07T15:07:23.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021568550000000001 2020-08-07T15:07:23.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:23.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.918020+0000", 2020-08-07T15:07:23.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5976000000000001e-05 2020-08-07T15:07:23.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:23.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.918076+0000", 2020-08-07T15:07:23.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.154831946 2020-08-07T15:07:23.610 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.610 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:23.611 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.611 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:44744 3.2 e32/20 3:4dba534b:::benchmark_data_smithi074_35447_object5592:head v 32'2608, mlcod=32'2602)", 2020-08-07T15:07:23.611 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:04.868608+0000", 2020-08-07T15:07:23.612 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.346197662000002, 2020-08-07T15:07:23.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16039398499999999, 2020-08-07T15:07:23.612 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:23.612 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:23.612 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:23.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:23.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.868608+0000", 2020-08-07T15:07:23.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:23.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:23.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.868608+0000", 2020-08-07T15:07:23.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1788e-05 2020-08-07T15:07:23.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:23.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.868660+0000", 2020-08-07T15:07:23.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2920000000000001e-06 2020-08-07T15:07:23.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:23.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.868666+0000", 2020-08-07T15:07:23.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15842231200000001 2020-08-07T15:07:23.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:23.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:05.027088+0000", 2020-08-07T15:07:23.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5329999999999996e-06 2020-08-07T15:07:23.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:23.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:05.027093+0000", 2020-08-07T15:07:23.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0534999999999998e-05 2020-08-07T15:07:23.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:23.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:05.027123+0000", 2020-08-07T15:07:23.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6172999999999999e-05 2020-08-07T15:07:23.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:23.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:05.027139+0000", 2020-08-07T15:07:23.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001801813 2020-08-07T15:07:23.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:23.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:05.028941+0000", 2020-08-07T15:07:23.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0538999999999998e-05 2020-08-07T15:07:23.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:23.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:05.029002+0000", 2020-08-07T15:07:23.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16039398499999999 2020-08-07T15:07:23.622 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.622 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:23.622 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.623 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:47792 3.a 3:5815b9e8:::benchmark_data_smithi074_35447_object5973:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e36)", 2020-08-07T15:07:23.623 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:08.669034+0000", 2020-08-07T15:07:23.623 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.545771630999999, 2020-08-07T15:07:23.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14798557800000001, 2020-08-07T15:07:23.624 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:23.624 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:23.624 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:23.624 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:23.624 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:23.625 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 47792 2020-08-07T15:07:23.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.625 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:23.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:23.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.669034+0000", 2020-08-07T15:07:23.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:23.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:23.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.669034+0000", 2020-08-07T15:07:23.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1081000000000003e-05 2020-08-07T15:07:23.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:23.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.669065+0000", 2020-08-07T15:07:23.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3069999999999996e-06 2020-08-07T15:07:23.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:23.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.669069+0000", 2020-08-07T15:07:23.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2977e-05 2020-08-07T15:07:23.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:23.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.669082+0000", 2020-08-07T15:07:23.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0940000000000001e-06 2020-08-07T15:07:23.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:23.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.669086+0000", 2020-08-07T15:07:23.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030010919999999999 2020-08-07T15:07:23.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:23.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.672087+0000", 2020-08-07T15:07:23.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5995999999999996e-05 2020-08-07T15:07:23.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:23.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.672153+0000", 2020-08-07T15:07:23.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001739 2020-08-07T15:07:23.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:23.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.672327+0000", 2020-08-07T15:07:23.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001122867 2020-08-07T15:07:23.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:23.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.673450+0000", 2020-08-07T15:07:23.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9999999999999996e-07 2020-08-07T15:07:23.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:23.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.673451+0000", 2020-08-07T15:07:23.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3608999999999994e-05 2020-08-07T15:07:23.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:23.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.673514+0000", 2020-08-07T15:07:23.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003311151 2020-08-07T15:07:23.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:23.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.676826+0000", 2020-08-07T15:07:23.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14012928099999999 2020-08-07T15:07:23.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:23.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.816955+0000", 2020-08-07T15:07:23.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3650000000000001e-06 2020-08-07T15:07:23.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:23.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.816956+0000", 2020-08-07T15:07:23.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3058000000000005e-05 2020-08-07T15:07:23.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:23.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.817019+0000", 2020-08-07T15:07:23.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14798557800000001 2020-08-07T15:07:23.641 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.641 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:23.641 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.642 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:48144 3.0 e36/20 3:0cec1d53:::benchmark_data_smithi074_35447_object6017:head v 36'3128, mlcod=36'3120)", 2020-08-07T15:07:23.642 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:08.897123+0000", 2020-08-07T15:07:23.642 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.317682145999999, 2020-08-07T15:07:23.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25221023399999998, 2020-08-07T15:07:23.643 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:23.643 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:23.643 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:23.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:23.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897123+0000", 2020-08-07T15:07:23.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:23.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:23.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897123+0000", 2020-08-07T15:07:23.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0655999999999997e-05 2020-08-07T15:07:23.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:23.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897174+0000", 2020-08-07T15:07:23.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6769999999999998e-06 2020-08-07T15:07:23.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:23.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897181+0000", 2020-08-07T15:07:23.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25090537899999998 2020-08-07T15:07:23.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:23.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148086+0000", 2020-08-07T15:07:23.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5520000000000004e-06 2020-08-07T15:07:23.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:23.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148092+0000", 2020-08-07T15:07:23.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6391999999999999e-05 2020-08-07T15:07:23.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:23.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148138+0000", 2020-08-07T15:07:23.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8124999999999999e-05 2020-08-07T15:07:23.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:23.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148166+0000", 2020-08-07T15:07:23.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011152009999999999 2020-08-07T15:07:23.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:23.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.149281+0000", 2020-08-07T15:07:23.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2252e-05 2020-08-07T15:07:23.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:23.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.149333+0000", 2020-08-07T15:07:23.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25221023399999998 2020-08-07T15:07:23.655 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.655 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:23.655 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.655 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:48776 3.2 e36/20 3:47029331:::benchmark_data_smithi074_35447_object6096:head v 36'2888, mlcod=36'2881)", 2020-08-07T15:07:23.656 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:07:23.656 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.810285330999999, 2020-08-07T15:07:23.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27006165399999998, 2020-08-07T15:07:23.657 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:23.657 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:23.657 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:23.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:23.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:07:23.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:23.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:23.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:07:23.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8454000000000001e-05 2020-08-07T15:07:23.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:23.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404569+0000", 2020-08-07T15:07:23.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2349999999999996e-06 2020-08-07T15:07:23.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:23.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404575+0000", 2020-08-07T15:07:23.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26921690999999998 2020-08-07T15:07:23.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:23.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673792+0000", 2020-08-07T15:07:23.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3330000000000002e-06 2020-08-07T15:07:23.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:23.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673798+0000", 2020-08-07T15:07:23.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1013000000000001e-05 2020-08-07T15:07:23.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:23.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673849+0000", 2020-08-07T15:07:23.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7044999999999999e-05 2020-08-07T15:07:23.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:23.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673876+0000", 2020-08-07T15:07:23.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00065173100000000001 2020-08-07T15:07:23.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:23.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.674528+0000", 2020-08-07T15:07:23.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3933e-05 2020-08-07T15:07:23.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:23.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.674582+0000", 2020-08-07T15:07:23.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27006165399999998 2020-08-07T15:07:23.671 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.671 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:23.671 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.672 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:49208 3.2 e36/20 3:4d72b982:::benchmark_data_smithi074_35447_object6150:head v 36'2904, mlcod=36'2897)", 2020-08-07T15:07:23.672 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:09.692013+0000", 2020-08-07T15:07:23.672 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.522792302999999, 2020-08-07T15:07:23.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.234386975, 2020-08-07T15:07:23.673 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:23.673 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:23.673 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:23.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:23.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.692013+0000", 2020-08-07T15:07:23.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:23.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:23.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.692013+0000", 2020-08-07T15:07:23.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0998000000000002e-05 2020-08-07T15:07:23.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:23.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.692064+0000", 2020-08-07T15:07:23.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0210000000000001e-05 2020-08-07T15:07:23.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:23.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.692074+0000", 2020-08-07T15:07:23.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.233053807 2020-08-07T15:07:23.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:23.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.925128+0000", 2020-08-07T15:07:23.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4879999999999998e-06 2020-08-07T15:07:23.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:23.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.925134+0000", 2020-08-07T15:07:23.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5154999999999997e-05 2020-08-07T15:07:23.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:23.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.925179+0000", 2020-08-07T15:07:23.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2710000000000001e-05 2020-08-07T15:07:23.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:23.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.925201+0000", 2020-08-07T15:07:23.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001143725 2020-08-07T15:07:23.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:23.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.926345+0000", 2020-08-07T15:07:23.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4882000000000001e-05 2020-08-07T15:07:23.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:23.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.926400+0000", 2020-08-07T15:07:23.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.234386975 2020-08-07T15:07:23.687 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.687 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:23.687 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.688 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:55192 3.2 e37/20 3:4a96099f:::benchmark_data_smithi074_35447_object6898:head v 37'3336, mlcod=37'3326)", 2020-08-07T15:07:23.688 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:13.771422+0000", 2020-08-07T15:07:23.688 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4433836339999999, 2020-08-07T15:07:23.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16800543000000001, 2020-08-07T15:07:23.688 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:23.688 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:23.689 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:23.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:23.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.771422+0000", 2020-08-07T15:07:23.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:23.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:23.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.771422+0000", 2020-08-07T15:07:23.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9376000000000001e-05 2020-08-07T15:07:23.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:23.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.771451+0000", 2020-08-07T15:07:23.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8789999999999996e-06 2020-08-07T15:07:23.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:23.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.771455+0000", 2020-08-07T15:07:23.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16555510500000001 2020-08-07T15:07:23.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:23.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.937010+0000", 2020-08-07T15:07:23.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3770000000000004e-06 2020-08-07T15:07:23.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:23.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.937015+0000", 2020-08-07T15:07:23.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6736999999999997e-05 2020-08-07T15:07:23.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:23.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.937052+0000", 2020-08-07T15:07:23.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3654000000000001e-05 2020-08-07T15:07:23.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:23.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.937076+0000", 2020-08-07T15:07:23.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022779929999999999 2020-08-07T15:07:23.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:23.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.939354+0000", 2020-08-07T15:07:23.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3308999999999999e-05 2020-08-07T15:07:23.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:23.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.939427+0000", 2020-08-07T15:07:23.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16800543000000001 2020-08-07T15:07:23.700 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.700 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:23.700 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.701 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:55912 3.a 3:5ef33505:::benchmark_data_smithi074_35447_object6988:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e37)", 2020-08-07T15:07:23.701 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:14.211261+0000", 2020-08-07T15:07:23.701 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.0035447410000007, 2020-08-07T15:07:23.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14275030799999999, 2020-08-07T15:07:23.701 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:23.702 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:23.702 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:23.702 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:23.702 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:23.702 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 55912 2020-08-07T15:07:23.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.704 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:23.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:23.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.211261+0000", 2020-08-07T15:07:23.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:23.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:23.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.211261+0000", 2020-08-07T15:07:23.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0318999999999997e-05 2020-08-07T15:07:23.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:23.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.211311+0000", 2020-08-07T15:07:23.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9120000000000003e-06 2020-08-07T15:07:23.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:23.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.211317+0000", 2020-08-07T15:07:23.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2317000000000001e-05 2020-08-07T15:07:23.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:23.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.211329+0000", 2020-08-07T15:07:23.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8860000000000001e-06 2020-08-07T15:07:23.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:23.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.211335+0000", 2020-08-07T15:07:23.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043890420000000001 2020-08-07T15:07:23.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:23.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.215724+0000", 2020-08-07T15:07:23.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1188999999999997e-05 2020-08-07T15:07:23.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:23.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.215815+0000", 2020-08-07T15:07:23.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016276499999999999 2020-08-07T15:07:23.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:23.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.215978+0000", 2020-08-07T15:07:23.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001234413 2020-08-07T15:07:23.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:23.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.217212+0000", 2020-08-07T15:07:23.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5000000000000003e-07 2020-08-07T15:07:23.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:23.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.217213+0000", 2020-08-07T15:07:23.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4478999999999998e-05 2020-08-07T15:07:23.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:23.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.217268+0000", 2020-08-07T15:07:23.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015618590999999999 2020-08-07T15:07:23.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:23.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.232886+0000", 2020-08-07T15:07:23.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.12102110000000001 2020-08-07T15:07:23.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:23.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.353907+0000", 2020-08-07T15:07:23.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4099999999999997e-07 2020-08-07T15:07:23.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:23.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.353908+0000", 2020-08-07T15:07:23.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102904 2020-08-07T15:07:23.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:23.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.354011+0000", 2020-08-07T15:07:23.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14275030799999999 2020-08-07T15:07:23.722 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.722 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:23.722 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.722 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.723 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:23.723 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:23.723 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:07:23.727 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:07:23.728 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 47 16 69409 69393 11.5338 11.7422 0.00351629 0.0108337 2020-08-07T15:07:23.729 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:07:23.729 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:07:23.729 INFO:tasks.thrashosds.thrasher:Growing pool device_health_metrics 2020-08-07T15:07:23.730 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:07:23.731 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:23.379+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:23.731 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:23.728+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:23.830 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:23.829+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:23.903 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:23.903 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:23.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.903 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:70489 3.f 3:f05d7fc5:::benchmark_data_smithi074_35447_object8811:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e43)", 2020-08-07T15:07:23.904 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:23.895408+0000", 2020-08-07T15:07:23.904 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0057202939999999999, 2020-08-07T15:07:23.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057520949999999996, 2020-08-07T15:07:23.904 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:23.904 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:23.905 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:23.905 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:23.905 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:23.905 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 70489 2020-08-07T15:07:23.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.906 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:23.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:23.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.895408+0000", 2020-08-07T15:07:23.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:23.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:23.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.895408+0000", 2020-08-07T15:07:23.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0680000000000003e-05 2020-08-07T15:07:23.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:23.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.895459+0000", 2020-08-07T15:07:23.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2919999999999996e-06 2020-08-07T15:07:23.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:23.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.895467+0000", 2020-08-07T15:07:23.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4895999999999999e-05 2020-08-07T15:07:23.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:23.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.895482+0000", 2020-08-07T15:07:23.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5420000000000002e-06 2020-08-07T15:07:23.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:23.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.895488+0000", 2020-08-07T15:07:23.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8430000000000002e-05 2020-08-07T15:07:23.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.895537+0000", 2020-08-07T15:07:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019247200000000001 2020-08-07T15:07:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.895729+0000", 2020-08-07T15:07:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000223785 2020-08-07T15:07:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.895953+0000", 2020-08-07T15:07:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002495096 2020-08-07T15:07:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.898448+0000", 2020-08-07T15:07:23.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.33e-07 2020-08-07T15:07:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.898449+0000", 2020-08-07T15:07:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2087000000000004e-05 2020-08-07T15:07:23.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:23.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.898511+0000", 2020-08-07T15:07:23.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031032130000000001 2020-08-07T15:07:23.939 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.939 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:23.939 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.941 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:70490 3.f 3:f05d7fc5:::benchmark_data_smithi074_35447_object8811:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e43)", 2020-08-07T15:07:23.941 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:23.895529+0000", 2020-08-07T15:07:23.941 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0055992849999999999, 2020-08-07T15:07:23.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057493459999999998, 2020-08-07T15:07:23.942 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:23.942 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:07:23.942 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:23.942 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:23.942 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:23.943 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 70490 2020-08-07T15:07:23.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.943 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:23.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:23.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.895529+0000", 2020-08-07T15:07:23.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:23.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:23.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.895529+0000", 2020-08-07T15:07:23.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1338000000000002e-05 2020-08-07T15:07:23.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:23.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.895580+0000", 2020-08-07T15:07:23.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6829999999999996e-06 2020-08-07T15:07:23.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:23.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.895586+0000", 2020-08-07T15:07:23.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2716e-05 2020-08-07T15:07:23.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:23.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.895599+0000", 2020-08-07T15:07:23.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8420000000000004e-06 2020-08-07T15:07:23.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:23.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.895604+0000", 2020-08-07T15:07:23.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00097084099999999996 2020-08-07T15:07:23.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:23.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.896574+0000", 2020-08-07T15:07:23.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6264999999999997e-05 2020-08-07T15:07:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.896631+0000", 2020-08-07T15:07:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013137700000000001 2020-08-07T15:07:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.896762+0000", 2020-08-07T15:07:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001233062 2020-08-07T15:07:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:70491 3.f 3:f05d7fc5:::benchmark_data_smithi074_35447_object8811:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e43)", 2020-08-07T15:07:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:23.896165+0000", 2020-08-07T15:07:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0049636910000000001, 2020-08-07T15:07:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051708270000000002, 2020-08-07T15:07:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:07:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:23.970 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 70491 2020-08-07T15:07:23.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.970 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:23.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:23.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.896165+0000", 2020-08-07T15:07:23.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:23.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.896165+0000", 2020-08-07T15:07:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0028000000000002e-05 2020-08-07T15:07:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.896225+0000", 2020-08-07T15:07:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6800000000000004e-06 2020-08-07T15:07:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.896231+0000", 2020-08-07T15:07:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379e-05 2020-08-07T15:07:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.896245+0000", 2020-08-07T15:07:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3629999999999999e-06 2020-08-07T15:07:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.896251+0000", 2020-08-07T15:07:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091549000000000003 2020-08-07T15:07:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.897167+0000", 2020-08-07T15:07:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5045999999999999e-05 2020-08-07T15:07:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.897252+0000", 2020-08-07T15:07:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017994699999999999 2020-08-07T15:07:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.897432+0000", 2020-08-07T15:07:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001266933 2020-08-07T15:07:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:70492 3.f 3:f05d7fc5:::benchmark_data_smithi074_35447_object8811:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e43)", 2020-08-07T15:07:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:23.896288+0000", 2020-08-07T15:07:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0048399009999999998, 2020-08-07T15:07:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051109279999999998, 2020-08-07T15:07:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:07:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 70492 2020-08-07T15:07:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.896288+0000", 2020-08-07T15:07:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.896288+0000", 2020-08-07T15:07:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1105999999999994e-05 2020-08-07T15:07:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.896350+0000", 2020-08-07T15:07:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3600000000000001e-06 2020-08-07T15:07:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.896356+0000", 2020-08-07T15:07:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1233000000000001e-05 2020-08-07T15:07:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.896367+0000", 2020-08-07T15:07:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.938e-06 2020-08-07T15:07:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:24.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.896372+0000", 2020-08-07T15:07:24.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014587440000000001 2020-08-07T15:07:24.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.897831+0000", 2020-08-07T15:07:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9614e-05 2020-08-07T15:07:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.897920+0000", 2020-08-07T15:07:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017370299999999999 2020-08-07T15:07:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.898094+0000", 2020-08-07T15:07:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018056979999999999 2020-08-07T15:07:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:70493 3.f 3:f05d7fc5:::benchmark_data_smithi074_35447_object8811:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e43)", 2020-08-07T15:07:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:23.896407+0000", 2020-08-07T15:07:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0047217750000000001, 2020-08-07T15:07:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050571089999999997, 2020-08-07T15:07:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:24.014 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:07:24.014 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:24.014 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:24.014 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 70493 2020-08-07T15:07:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.896407+0000", 2020-08-07T15:07:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.896407+0000", 2020-08-07T15:07:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0572000000000003e-05 2020-08-07T15:07:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.896457+0000", 2020-08-07T15:07:24.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5650000000000001e-06 2020-08-07T15:07:24.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:24.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.896463+0000", 2020-08-07T15:07:24.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1613999999999999e-05 2020-08-07T15:07:24.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:24.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.896474+0000", 2020-08-07T15:07:24.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7910000000000004e-06 2020-08-07T15:07:24.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:24.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.896479+0000", 2020-08-07T15:07:24.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002080167 2020-08-07T15:07:24.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:24.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.898559+0000", 2020-08-07T15:07:24.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6817999999999994e-05 2020-08-07T15:07:24.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:24.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.898646+0000", 2020-08-07T15:07:24.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016087299999999999 2020-08-07T15:07:24.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:24.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.898807+0000", 2020-08-07T15:07:24.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024004 2020-08-07T15:07:24.024 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.024 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:24.024 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.025 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:70489 3.f e43/20)", 2020-08-07T15:07:24.025 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:23.898523+0000", 2020-08-07T15:07:24.025 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0026054199999999998, 2020-08-07T15:07:24.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002980141, 2020-08-07T15:07:24.025 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:24.026 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:07:24.026 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:24.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:24.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.898523+0000", 2020-08-07T15:07:24.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:24.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:24.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.898523+0000", 2020-08-07T15:07:24.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1275000000000001e-05 2020-08-07T15:07:24.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:24.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.898554+0000", 2020-08-07T15:07:24.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.852e-06 2020-08-07T15:07:24.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:24.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.898560+0000", 2020-08-07T15:07:24.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2566999999999998e-05 2020-08-07T15:07:24.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:24.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.898633+0000", 2020-08-07T15:07:24.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4319999999999998e-06 2020-08-07T15:07:24.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:24.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.898638+0000", 2020-08-07T15:07:24.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000115126 2020-08-07T15:07:24.032 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.032 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:24.033 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.033 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:70494 3.f 3.a3feba0f (undecoded) ondisk+write+known_if_redirected e43)", 2020-08-07T15:07:24.033 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:23.898872+0000", 2020-08-07T15:07:24.034 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0022567210000000002, 2020-08-07T15:07:24.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002657801, 2020-08-07T15:07:24.034 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:24.034 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:07:24.034 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:24.035 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:24.035 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:24.035 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 70494 2020-08-07T15:07:24.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.035 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:24.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:24.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.898872+0000", 2020-08-07T15:07:24.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:24.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:24.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.898872+0000", 2020-08-07T15:07:24.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3508000000000003e-05 2020-08-07T15:07:24.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:24.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.898925+0000", 2020-08-07T15:07:24.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8120000000000002e-06 2020-08-07T15:07:24.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:24.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.898932+0000", 2020-08-07T15:07:24.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3237e-05 2020-08-07T15:07:24.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:24.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.898945+0000", 2020-08-07T15:07:24.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8700000000000003e-06 2020-08-07T15:07:24.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:24.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.898952+0000", 2020-08-07T15:07:24.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0426999999999994e-05 2020-08-07T15:07:24.041 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.042 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:24.042 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.042 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:70489 3.f e43/20)", 2020-08-07T15:07:24.043 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:23.900266+0000", 2020-08-07T15:07:24.043 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00086182700000000001, 2020-08-07T15:07:24.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00129367, 2020-08-07T15:07:24.043 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:24.043 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:07:24.044 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:24.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:24.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900266+0000", 2020-08-07T15:07:24.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:24.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:24.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900266+0000", 2020-08-07T15:07:24.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1108000000000001e-05 2020-08-07T15:07:24.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:24.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900298+0000", 2020-08-07T15:07:24.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0299999999999999e-06 2020-08-07T15:07:24.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:24.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900304+0000", 2020-08-07T15:07:24.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0028999999999998e-05 2020-08-07T15:07:24.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:24.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900354+0000", 2020-08-07T15:07:24.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4480000000000002e-06 2020-08-07T15:07:24.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:24.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900359+0000", 2020-08-07T15:07:24.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2614999999999993e-05 2020-08-07T15:07:24.049 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.049 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:24.050 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.050 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:70490 3.f e43/20)", 2020-08-07T15:07:24.050 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:23.900298+0000", 2020-08-07T15:07:24.051 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00082986099999999999, 2020-08-07T15:07:24.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001286805, 2020-08-07T15:07:24.051 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:24.052 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:07:24.053 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:24.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:24.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900298+0000", 2020-08-07T15:07:24.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:24.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:24.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900298+0000", 2020-08-07T15:07:24.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2066000000000001e-05 2020-08-07T15:07:24.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:24.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900331+0000", 2020-08-07T15:07:24.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.074e-06 2020-08-07T15:07:24.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:24.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900336+0000", 2020-08-07T15:07:24.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2765000000000001e-05 2020-08-07T15:07:24.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:24.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900378+0000", 2020-08-07T15:07:24.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2399999999999998e-06 2020-08-07T15:07:24.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:24.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900384+0000", 2020-08-07T15:07:24.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5144999999999999e-05 2020-08-07T15:07:24.059 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.059 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:24.059 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.061 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:70490 3.f e43/20)", 2020-08-07T15:07:24.061 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:23.900446+0000", 2020-08-07T15:07:24.061 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00068272199999999997, 2020-08-07T15:07:24.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00116479, 2020-08-07T15:07:24.062 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:24.062 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:07:24.062 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:24.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:24.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900446+0000", 2020-08-07T15:07:24.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:24.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:24.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900446+0000", 2020-08-07T15:07:24.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7127e-05 2020-08-07T15:07:24.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:24.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900473+0000", 2020-08-07T15:07:24.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2119999999999999e-06 2020-08-07T15:07:24.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:24.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900476+0000", 2020-08-07T15:07:24.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1240000000000003e-05 2020-08-07T15:07:24.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:24.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900537+0000", 2020-08-07T15:07:24.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.994e-06 2020-08-07T15:07:24.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:24.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900540+0000", 2020-08-07T15:07:24.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4573000000000004e-05 2020-08-07T15:07:24.070 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.070 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:24.070 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.071 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:70491 3.f e43/20)", 2020-08-07T15:07:24.071 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:23.900491+0000", 2020-08-07T15:07:24.072 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00063687300000000004, 2020-08-07T15:07:24.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001167134, 2020-08-07T15:07:24.072 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:24.072 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:07:24.072 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:24.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:24.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900491+0000", 2020-08-07T15:07:24.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:24.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:24.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900491+0000", 2020-08-07T15:07:24.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4983999999999999e-05 2020-08-07T15:07:24.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:24.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900506+0000", 2020-08-07T15:07:24.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3520000000000001e-06 2020-08-07T15:07:24.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:24.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900509+0000", 2020-08-07T15:07:24.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7761e-05 2020-08-07T15:07:24.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:24.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900547+0000", 2020-08-07T15:07:24.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7710000000000001e-06 2020-08-07T15:07:24.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:24.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900548+0000", 2020-08-07T15:07:24.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6867999999999998e-05 2020-08-07T15:07:24.081 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.081 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:24.081 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.082 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:70491 3.f e43/20)", 2020-08-07T15:07:24.082 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:23.900629+0000", 2020-08-07T15:07:24.082 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00049961199999999997, 2020-08-07T15:07:24.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010818959999999999, 2020-08-07T15:07:24.082 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:24.083 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:07:24.083 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:24.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:24.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900629+0000", 2020-08-07T15:07:24.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:24.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:24.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900629+0000", 2020-08-07T15:07:24.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2249999999999998e-05 2020-08-07T15:07:24.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:24.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900661+0000", 2020-08-07T15:07:24.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.011e-06 2020-08-07T15:07:24.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:24.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900666+0000", 2020-08-07T15:07:24.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018529700000000001 2020-08-07T15:07:24.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:24.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900851+0000", 2020-08-07T15:07:24.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2410000000000003e-06 2020-08-07T15:07:24.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:24.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900857+0000", 2020-08-07T15:07:24.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022779899999999999 2020-08-07T15:07:24.090 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.090 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:24.090 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.101 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:70492 3.f e43/20)", 2020-08-07T15:07:24.101 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:23.900698+0000", 2020-08-07T15:07:24.101 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00042986199999999998, 2020-08-07T15:07:24.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010679179999999999, 2020-08-07T15:07:24.101 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:24.102 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:07:24.102 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:24.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:24.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900698+0000", 2020-08-07T15:07:24.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:24.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:24.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900698+0000", 2020-08-07T15:07:24.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3073e-05 2020-08-07T15:07:24.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:24.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900732+0000", 2020-08-07T15:07:24.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7480000000000001e-06 2020-08-07T15:07:24.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:24.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900736+0000", 2020-08-07T15:07:24.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013147699999999999 2020-08-07T15:07:24.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:24.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900868+0000", 2020-08-07T15:07:24.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1980000000000001e-06 2020-08-07T15:07:24.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:24.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900871+0000", 2020-08-07T15:07:24.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017249600000000001 2020-08-07T15:07:24.116 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.116 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:24.116 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.117 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:70493 3.f e43/20)", 2020-08-07T15:07:24.117 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:23.900767+0000", 2020-08-07T15:07:24.117 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00036108100000000002, 2020-08-07T15:07:24.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010506560000000001, 2020-08-07T15:07:24.117 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:24.118 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:07:24.118 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:24.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:24.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900767+0000", 2020-08-07T15:07:24.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:24.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:24.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900767+0000", 2020-08-07T15:07:24.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0108999999999999e-05 2020-08-07T15:07:24.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:24.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900797+0000", 2020-08-07T15:07:24.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.668e-06 2020-08-07T15:07:24.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:24.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900802+0000", 2020-08-07T15:07:24.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7837999999999996e-05 2020-08-07T15:07:24.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:24.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900880+0000", 2020-08-07T15:07:24.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.813e-06 2020-08-07T15:07:24.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:24.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.900883+0000", 2020-08-07T15:07:24.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011542799999999999 2020-08-07T15:07:24.142 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.142 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:24.142 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.143 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:70492 3.f e43/20)", 2020-08-07T15:07:24.143 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:23.901025+0000", 2020-08-07T15:07:24.143 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00010306000000000001, 2020-08-07T15:07:24.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00084395200000000003, 2020-08-07T15:07:24.144 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:24.144 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:07:24.151 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:24.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:24.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.901025+0000", 2020-08-07T15:07:24.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:24.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:24.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.901025+0000", 2020-08-07T15:07:24.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2450999999999998e-05 2020-08-07T15:07:24.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:24.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.901058+0000", 2020-08-07T15:07:24.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9889999999999997e-06 2020-08-07T15:07:24.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:24.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.901064+0000", 2020-08-07T15:07:24.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6334e-05 2020-08-07T15:07:24.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:24.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.901120+0000", 2020-08-07T15:07:24.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4380000000000003e-06 2020-08-07T15:07:24.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:24.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.901125+0000", 2020-08-07T15:07:24.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100212 2020-08-07T15:07:24.176 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.176 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:24.176 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.177 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:70493 3.f e43/20)", 2020-08-07T15:07:24.177 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:23.901196+0000", 2020-08-07T15:07:24.177 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4294967295.9999323, 2020-08-07T15:07:24.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00072645700000000001, 2020-08-07T15:07:24.178 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:24.178 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:07:24.178 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:24.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:24.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.901196+0000", 2020-08-07T15:07:24.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:24.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:24.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.901196+0000", 2020-08-07T15:07:24.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0769000000000003e-05 2020-08-07T15:07:24.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:24.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.901227+0000", 2020-08-07T15:07:24.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8730000000000004e-06 2020-08-07T15:07:24.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:24.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.901232+0000", 2020-08-07T15:07:24.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8992000000000001e-05 2020-08-07T15:07:24.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:24.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.901281+0000", 2020-08-07T15:07:24.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7570000000000004e-06 2020-08-07T15:07:24.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:24.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.901286+0000", 2020-08-07T15:07:24.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0390999999999996e-05 2020-08-07T15:07:24.184 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.184 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:24.184 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.184 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.184 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:07:24.185 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 16 2020-08-07T15:07:24.185 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:24.188 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:07:24.189 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2020-08-07T15:07:23.150005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":549912576,"num_objects":8393,"num_object_clones":0,"num_object_copies":25179,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1074,"num_objects_unfound":0,"num_objects_dirty":8393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67130,"num_write_kb":537026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124,"ondisk_log_size":124,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9992252,"kb_used_data":1603572,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":744982468,"statfs":{"total":773094113280,"available":762862047232,"internally_reserved":8589934592,"allocated":1642057728,"data_stored":1639872616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,3,2,5,11,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":62496768,"num_objects":953,"num_object_clones":0,"num_object_copies":2859,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":118,"num_objects_unfound":0,"num_objects_dirty":953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7629,"num_write_kb":61032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.762511"},"pg_stats":[{"pgid":"3.f","version":"42'4576","reported_seq":"4606","reported_epoch":"42","state":"active+clean","last_fresh":"2020-08-07T15:07:21.676202+0000","last_change":"2020-08-07T15:06:35.163498+0000","last_active":"2020-08-07T15:07:21.676202+0000","last_peered":"2020-08-07T15:07:21.676202+0000","last_clean":"2020-08-07T15:07:21.676202+0000","last_became_active":"2020-08-07T15:06:35.162764+0000","last_became_peered":"2020-08-07T15:06:35.162764+0000","last_unstale":"2020-08-07T15:07:21.676202+0000","last_undegraded":"2020-08-07T15:07:21.676202+0000","last_fullsized":"2020-08-07T15:07:21.676202+0000","mapping_epoch":20,"log_start":"42'4569","ondisk_log_start":"42'4569","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":37486592,"num_objects":572,"num_object_clones":0,"num_object_copies":1716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4576,"num_write_kb":36608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"42'4336","reported_seq":"4366","reported_epoch":"42","state":"active+clean","last_fresh":"2020-08-07T15:07:21.674098+0000","last_change":"2020-08-07T15:06:35.169137+0000","last_active":"2020-08-07T15:07:21.674098+0000","last_peered":"2020-08-07T15:07:21.674098+0000","last_clean":"2020-08-07T15:07:21.674098+0000","last_became_active":"2020-08-07T15:06:35.168689+0000","last_became_peered":"2020-08-07T15:06:35.168689+0000","last_unstale":"2020-08-07T15:07:21.674098+0000","last_undegraded":"2020-08-07T15:07:21.674098+0000","last_fullsized":"2020-08-07T15:07:21.674098+0000","mapping_epoch":20,"log_start":"42'4329","ondisk_log_start":"42'4329","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":35520512,"num_objects":542,"num_object_clones":0,"num_object_copies":1626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4336,"num_write_kb":34688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"42'4296","reported_seq":"4330","reported_epoch":"42","state":"active+remapped+backfill_toofull","last_fresh":"2020-08-07T15:07:21.692999+0000","last_change":"2020-08-07T15:07:08.296959+0000","last_active":"2020-08-07T15:07:21.692999+0000","last_peered":"2020-08-07T15:07:21.692999+0000","last_clean":"2020-08-07T15:07:06.228184+0000","last_became_active":"2020-08-07T15:07:08.269740+0000","last_became_peered":"2020-08-07T15:07:08.269740+0000","last_unstale":"2020-08-07T15:07:21.692999+0000","last_undegraded":"2020-08-07T15:07:21.692999+0000","last_fullsized":"2020-08-07T15:07:21.692999+0000","mapping_epoch":35,"log_start":"42'4287","ondisk_log_start":"42'4287","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":35192832,"num_objects":537,"num_object_clones":0,"num_object_copies":1611,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1074,"num_objects_unfound":0,"num_objects_dirty":537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4296,"num_write_kb":34368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[5,6,7],"avail_no_missing":["5","6","7"],"object_location_counts":[{"shards":"5,6,7","objects":537}],"blocked_by":[],"up_primary":3,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"42'4272","reported_seq":"4302","reported_epoch":"42","state":"active+clean","last_fresh":"2020-08-07T15:07:21.660115+0000","last_change":"2020-08-07T15:06:35.167480+0000","last_active":"2020-08-07T15:07:21.660115+0000","last_peered":"2020-08-07T15:07:21.660115+0000","last_clean":"2020-08-07T15:07:21.660115+0000","last_became_active":"2020-08-07T15:06:35.167101+0000","last_became_peered":"2020-08-07T15:06:35.167101+0000","last_unstale":"2020-08-07T15:07:21.660115+0000","last_undegraded":"2020-08-07T15:07:21.660115+0000","last_fullsized":"2020-08-07T15:07:21.660115+0000","mapping_epoch":20,"log_start":"42'4264","ondisk_log_start":"42'4264","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34996224,"num_objects":534,"num_object_clones":0,"num_object_copies":1602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4272,"num_write_kb":34176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.b","version":"42'4000","reported_seq":"4029","reported_epoch":"42","state":"active+clean","last_fresh":"2020-08-07T15:07:21.650858+0000","last_change":"2020-08-07T15:06:35.170504+0000","last_active":"2020-08-07T15:07:21.650858+0000","last_peered":"2020-08-07T15:07:21.650858+0000","last_clean":"2020-08-07T15:07:21.650858+0000","last_became_active":"2020-08-07T15:06:35.169532+0000","last_became_peered":"2020-08-07T15:06:35.169532+0000","last_unstale":"2020-08-07T15:07:21.650858+0000","last_undegraded":"2020-08-07T15:07:21.650858+0000","last_fullsized":"2020-08-07T15:07:21.650858+0000","mapping_epoch":20,"log_start":"42'3990","ondisk_log_start":"42'3990","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":32768000,"num_objects":500,"num_object_clones":0,"num_object_copies":1500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4000,"num_write_kb":32000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"42'4504","reported_seq":"4534","reported_epoch":"42","state":"active+clean","last_fresh":"2020-08-07T15:07:22.519188+0000","last_change":"2020-08-07T15:06:35.170105+0000","last_active":"2020-08-07T15:07:22.519188+0000","last_peered":"2020-08-07T15:07:22.519188+0000","last_clean":"2020-08-07T15:07:22.519188+0000","last_became_active":"2020-08-07T15:06:35.169494+0000","last_became_peered":"2020-08-07T15:06:35.169494+0000","last_unstale":"2020-08-07T15:07:22.519188+0000","last_undegraded":"2020-08-07T15:07:22.519188+0000","last_fullsized":"2020-08-07T15:07:22.519188+0000","mapping_epoch":20,"log_start":"42'4496","ondisk_log_start":"42'4496","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36896768,"num_objects":563,"num_object_clones":0,"num_object_copies":1689,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4504,"num_write_kb":36032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"42'3872","reported_seq":"3902","reported_epoch":"42","state":"active+clean","last_fresh":"2020-08-07T15:07:21.463098+0000","last_change":"2020-08-07T15:06:35.163109+0000","last_active":"2020-08-07T15:07:21.463098+0000","last_peered":"2020-08-07T15:07:21.463098+0000","last_clean":"2020-08-07T15:07:21.463098+0000","last_became_active":"2020-08-07T15:06:35.162721+0000","last_became_peered":"2020-08-07T15:06:35.162721+0000","last_unstale":"2020-08-07T15:07:21.463098+0000","last_undegraded":"2020-08-07T15:07:21.463098+0000","last_fullsized":"2020-08-07T15:07:21.463098+0000","mapping_epoch":20,"log_start":"42'3865","ondisk_log_start":"42'3865","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":31719424,"num_objects":484,"num_object_clones":0,"num_object_copies":1452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3872,"num_write_kb":30976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"42'4176","reported_seq":"4206","reported_epoch":"42","state":"active+clean","last_fresh":"2020-08-07T15:07:21.633478+0000","last_change":"2020-08-07T15:06:35.173504+0000","last_active":"2020-08-07T15:07:21.633478+0000","last_peered":"2020-08-07T15:07:21.633478+0000","last_clean":"2020-08-07T15:07:21.633478+0000","last_became_active":"2020-08-07T15:06:35.172437+0000","last_became_peered":"2020-08-07T15:06:35.172437+0000","last_unstale":"2020-08-07T15:07:21.633478+0000","last_undegraded":"2020-08-07T15:07:21.633478+0000","last_fullsized":"2020-08-07T15:07:21.633478+0000","mapping_epoch":20,"log_start":"42'4169","ondisk_log_start":"42'4169","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34209792,"num_objects":522,"num_object_clones":0,"num_object_copies":1566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4176,"num_write_kb":33408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"30","reported_epoch":"41","state":"active+clean","last_fresh":"2020-08-07T15:07:18.414237+0000","last_change":"2020-08-07T15:06:46.199001+0000","last_active":"2020-08-07T15:07:18.414237+0000","last_peered":"2020-08-07T15:07:18.414237+0000","last_clean":"2020-08-07T15:07:18.414237+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:07:18.414237+0000","last_undegraded":"2020-08-07T15:07:18.414237+0000","last_fullsized":"2020-08-07T15:07:18.414237+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.5","version":"42'4032","reported_seq":"4062","reported_epoch":"42","state":"active+clean","last_fresh":"2020-08-07T15:07:21.679128+0000","last_change":"2020-08-07T15:06:35.169307+0000","last_active":"2020-08-07T15:07:21.679128+0000","last_peered":"2020-08-07T15:07:21.679128+0000","last_clean":"2020-08-07T15:07:21.679128+0000","last_became_active":"2020-08-07T15:06:35.168887+0000","last_became_peered":"2020-08-07T15:06:35.168887+0000","last_unstale":"2020-08-07T15:07:21.679128+0000","last_undegraded":"2020-08-07T15:07:21.679128+0000","last_fullsized":"2020-08-07T15:07:21.679128+0000","mapping_epoch":20,"log_start":"42'4024","ondisk_log_start":"42'4024","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33030144,"num_objects":504,"num_object_clones":0,"num_object_copies":1512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4032,"num_write_kb":32256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"33","reported_epoch":"41","state":"active+clean","last_fresh":"2020-08-07T15:07:18.411308+0000","last_change":"2020-08-07T15:06:23.074962+0000","last_active":"2020-08-07T15:07:18.411308+0000","last_peered":"2020-08-07T15:07:18.411308+0000","last_clean":"2020-08-07T15:07:18.411308+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:07:18.411308+0000","last_undegraded":"2020-08-07T15:07:18.411308+0000","last_fullsized":"2020-08-07T15:07:18.411308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"42'4048","reported_seq":"4078","reported_epoch":"42","state":"active+clean","last_fresh":"2020-08-07T15:07:21.537802+0000","last_change":"2020-08-07T15:06:35.164593+0000","last_active":"2020-08-07T15:07:21.537802+0000","last_peered":"2020-08-07T15:07:21.537802+0000","last_clean":"2020-08-07T15:07:21.537802+0000","last_became_active":"2020-08-07T15:06:35.164108+0000","last_became_peered":"2020-08-07T15:06:35.164108+0000","last_unstale":"2020-08-07T15:07:21.537802+0000","last_undegraded":"2020-08-07T15:07:21.537802+0000","last_fullsized":"2020-08-07T15:07:21.537802+0000","mapping_epoch":20,"log_start":"42'4042","ondisk_log_start":"42'4042","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33161216,"num_objects":506,"num_object_clones":0,"num_object_copies":1518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4048,"num_write_kb":32384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"33","reported_epoch":"41","state":"active+clean","last_fresh":"2020-08-07T15:07:18.458023+0000","last_change":"2020-08-07T15:06:23.223924+0000","last_active":"2020-08-07T15:07:18.458023+0000","last_peered":"2020-08-07T15:07:18.458023+0000","last_clean":"2020-08-07T15:07:18.458023+0000","last_became_active":"2020-08-07T15:06:23.223489+0000","last_became_peered":"2020-08-07T15:06:23.223489+0000","last_unstale":"2020-08-07T15:07:18.458023+0000","last_undegraded":"2020-08-07T15:07:18.458023+0000","last_fullsized":"2020-08-07T15:07:18.458023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"42'4248","reported_seq":"4278","reported_epoch":"42","state":"active+clean","last_fresh":"2020-08-07T15:07:21.668710+0000","last_change":"2020-08-07T15:06:35.170553+0000","last_active":"2020-08-07T15:07:21.668710+0000","last_peered":"2020-08-07T15:07:21.668710+0000","last_clean":"2020-08-07T15:07:21.668710+0000","last_became_active":"2020-08-07T15:06:35.170037+0000","last_became_peered":"2020-08-07T15:06:35.170037+0000","last_unstale":"2020-08-07T15:07:21.668710+0000","last_undegraded":"2020-08-07T15:07:21.668710+0000","last_fullsized":"2020-08-07T15:07:21.668710+0000","mapping_epoch":20,"log_start":"42'4240","ondisk_log_start":"42'4240","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34799616,"num_objects":531,"num_object_clones":0,"num_object_copies":1593,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4248,"num_write_kb":33984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"33","reported_epoch":"41","state":"active+clean","last_fresh":"2020-08-07T15:07:18.411302+0000","last_change":"2020-08-07T15:06:23.075958+0000","last_active":"2020-08-07T15:07:18.411302+0000","last_peered":"2020-08-07T15:07:18.411302+0000","last_clean":"2020-08-07T15:07:18.411302+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:07:18.411302+0000","last_undegraded":"2020-08-07T15:07:18.411302+0000","last_fullsized":"2020-08-07T15:07:18.411302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"43","reported_epoch":"41","state":"active+clean","last_fresh":"2020-08-07T15:07:18.518133+0000","last_change":"2020-08-07T15:07:18.518133+0000","last_active":"2020-08-07T15:07:18.518133+0000","last_peered":"2020-08-07T15:07:18.518133+0000","last_clean":"2020-08-07T15:07:18.518133+0000","last_became_active":"2020-08-07T15:06:19.197819+0000","last_became_peered":"2020-08-07T15:06:19.197819+0000","last_unstale":"2020-08-07T15:07:18.518133+0000","last_undegraded":"2020-08-07T15:07:18.518133+0000","last_fullsized":"2020-08-07T15:07:18.518133+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:18.518049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:18.518049+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"42'4072","reported_seq":"4100","reported_epoch":"42","state":"active+clean","last_fresh":"2020-08-07T15:07:21.689613+0000","last_change":"2020-08-07T15:06:35.169888+0000","last_active":"2020-08-07T15:07:21.689613+0000","last_peered":"2020-08-07T15:07:21.689613+0000","last_clean":"2020-08-07T15:07:21.689613+0000","last_became_active":"2020-08-07T15:06:35.169385+0000","last_became_peered":"2020-08-07T15:06:35.169385+0000","last_unstale":"2020-08-07T15:07:21.689613+0000","last_undegraded":"2020-08-07T15:07:21.689613+0000","last_fullsized":"2020-08-07T15:07:21.689613+0000","mapping_epoch":20,"log_start":"42'4065","ondisk_log_start":"42'4065","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33357824,"num_objects":509,"num_object_clones":0,"num_object_copies":1527,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4072,"num_write_kb":32576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"33","reported_epoch":"41","state":"active+clean","last_fresh":"2020-08-07T15:07:18.414330+0000","last_change":"2020-08-07T15:06:23.224115+0000","last_active":"2020-08-07T15:07:18.414330+0000","last_peered":"2020-08-07T15:07:18.414330+0000","last_clean":"2020-08-07T15:07:18.414330+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:07:18.414330+0000","last_undegraded":"2020-08-07T15:07:18.414330+0000","last_fullsized":"2020-08-07T15:07:18.414330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"42'3920","reported_seq":"3950","reported_epoch":"42","state":"active+clean","last_fresh":"2020-08-07T15:07:21.638478+0000","last_change":"2020-08-07T15:06:46.214618+0000","last_active":"2020-08-07T15:07:21.638478+0000","last_peered":"2020-08-07T15:07:21.638478+0000","last_clean":"2020-08-07T15:07:21.638478+0000","last_became_active":"2020-08-07T15:06:46.202192+0000","last_became_peered":"2020-08-07T15:06:46.202192+0000","last_unstale":"2020-08-07T15:07:21.638478+0000","last_undegraded":"2020-08-07T15:07:21.638478+0000","last_fullsized":"2020-08-07T15:07:21.638478+0000","mapping_epoch":26,"log_start":"42'3912","ondisk_log_start":"42'3912","created":20,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":32112640,"num_objects":490,"num_object_clones":0,"num_object_copies":1470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3920,"num_write_kb":31360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3],"acting":[4,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"33","reported_epoch":"41","state":"active+clean","last_fresh":"2020-08-07T15:07:18.414391+0000","last_change":"2020-08-07T15:06:23.225886+0000","last_active":"2020-08-07T15:07:18.414391+0000","last_peered":"2020-08-07T15:07:18.414391+0000","last_clean":"2020-08-07T15:07:18.414391+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:07:18.414391+0000","last_undegraded":"2020-08-07T15:07:18.414391+0000","last_fullsized":"2020-08-07T15:07:18.414391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"42'4240","reported_seq":"4269","reported_epoch":"42","state":"active+clean","last_fresh":"2020-08-07T15:07:21.697765+0000","last_change":"2020-08-07T15:06:35.169626+0000","last_active":"2020-08-07T15:07:21.697765+0000","last_peered":"2020-08-07T15:07:21.697765+0000","last_clean":"2020-08-07T15:07:21.697765+0000","last_became_active":"2020-08-07T15:06:35.169127+0000","last_became_peered":"2020-08-07T15:06:35.169127+0000","last_unstale":"2020-08-07T15:07:21.697765+0000","last_undegraded":"2020-08-07T15:07:21.697765+0000","last_fullsized":"2020-08-07T15:07:21.697765+0000","mapping_epoch":20,"log_start":"42'4232","ondisk_log_start":"42'4232","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34734080,"num_objects":530,"num_object_clones":0,"num_object_copies":1590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4240,"num_write_kb":33920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"42'4088","reported_seq":"4118","reported_epoch":"42","state":"active+clean","last_fresh":"2020-08-07T15:07:21.636133+0000","last_change":"2020-08-07T15:06:35.172780+0000","last_active":"2020-08-07T15:07:21.636133+0000","last_peered":"2020-08-07T15:07:21.636133+0000","last_clean":"2020-08-07T15:07:21.636133+0000","last_became_active":"2020-08-07T15:06:35.172360+0000","last_became_peered":"2020-08-07T15:06:35.172360+0000","last_unstale":"2020-08-07T15:07:21.636133+0000","last_undegraded":"2020-08-07T15:07:21.636133+0000","last_fullsized":"2020-08-07T15:07:21.636133+0000","mapping_epoch":20,"log_start":"42'4080","ondisk_log_start":"42'4080","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33488896,"num_objects":511,"num_object_clones":0,"num_object_copies":1533,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4088,"num_write_kb":32704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"33","reported_epoch":"41","state":"active+clean","last_fresh":"2020-08-07T15:07:18.442090+0000","last_change":"2020-08-07T15:06:23.074076+0000","last_active":"2020-08-07T15:07:18.442090+0000","last_peered":"2020-08-07T15:07:18.442090+0000","last_clean":"2020-08-07T15:07:18.442090+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:07:18.442090+0000","last_undegraded":"2020-08-07T15:07:18.442090+0000","last_fullsized":"2020-08-07T15:07:18.442090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"33","reported_epoch":"41","state":"active+clean","last_fresh":"2020-08-07T15:07:18.406538+0000","last_change":"2020-08-07T15:06:23.073702+0000","last_active":"2020-08-07T15:07:18.406538+0000","last_peered":"2020-08-07T15:07:18.406538+0000","last_clean":"2020-08-07T15:07:18.406538+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:07:18.406538+0000","last_undegraded":"2020-08-07T15:07:18.406538+0000","last_fullsized":"2020-08-07T15:07:18.406538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"42'4448","reported_seq":"4479","reported_epoch":"42","state":"active+clean","last_fresh":"2020-08-07T15:07:21.685177+0000","last_change":"2020-08-07T15:06:46.199635+0000","last_active":"2020-08-07T15:07:21.685177+0000","last_peered":"2020-08-07T15:07:21.685177+0000","last_clean":"2020-08-07T15:07:21.685177+0000","last_became_active":"2020-08-07T15:06:46.199201+0000","last_became_peered":"2020-08-07T15:06:46.199201+0000","last_unstale":"2020-08-07T15:07:21.685177+0000","last_undegraded":"2020-08-07T15:07:21.685177+0000","last_fullsized":"2020-08-07T15:07:21.685177+0000","mapping_epoch":26,"log_start":"42'4442","ondisk_log_start":"42'4442","created":20,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36438016,"num_objects":556,"num_object_clones":0,"num_object_copies":1668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4448,"num_write_kb":35584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":549912576,"num_objects":8391,"num_object_clones":0,"num_object_copies":25173,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1074,"num_objects_unfound":0,"num_objects_dirty":8391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67128,"num_write_kb":537024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1652965376,"data_stored":1652965376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122,"ondisk_log_size":122,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1252204,"kb_used_data":203620,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93119636,"statfs":{"total":96636764160,"available":95354507264,"internally_reserved":1073741824,"allocated":208506880,"data_stored":208231636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":1.083}]},{"osd":1,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":5.7519999999999998,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":1.351},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":0.96899999999999997}]},{"osd":2,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":1.0529999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":5.5019999999999998,"5min":5.5019999999999998,"15min":5.5019999999999998},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":2.0720000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.9790000000000001,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.63100000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":37.506999999999998,"5min":37.506999999999998,"15min":37.506999999999998},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":831.88099999999997,"5min":831.88099999999997,"15min":831.88099999999997},"last":0.875},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":1.1040000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":3.1389999999999998,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":3.048,"5min":3.048,"15min":3.048},"last":1.3149999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1250940,"kb_used_data":202356,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93120900,"statfs":{"total":96636764160,"available":95355801600,"internally_reserved":1073741824,"allocated":207212544,"data_stored":206937300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":3.4020000000000001,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.383,"5min":2.383,"15min":2.383},"last":0.56899999999999995}]},{"osd":1,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":1.387,"5min":1.387,"15min":1.387},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.5939999999999999,"5min":3.5939999999999999,"15min":3.5939999999999999},"last":0.75600000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.272,"15min":1.272},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":4.0529999999999999,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":0.58399999999999996}]},{"osd":3,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":3.3580000000000001,"5min":3.3580000000000001,"15min":3.3580000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":3.004,"5min":3.004,"15min":3.004},"last":0.66400000000000003}]},{"osd":4,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.663,"15min":1.663},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":5.2729999999999997,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"min":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":5.2990000000000004,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":0.85799999999999998}]},{"osd":5,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.766,"5min":1.766,"15min":1.766},"min":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":5.327,"5min":5.327,"15min":5.327},"last":0.99099999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":3.3809999999999998,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":0.82899999999999996},{"interface":"front","average":{"1min":18.326000000000001,"5min":18.326000000000001,"15min":18.326000000000001},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":266.26999999999998,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.145}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1255592,"kb_used_data":207000,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93116248,"statfs":{"total":96636764160,"available":95351037952,"internally_reserved":1073741824,"allocated":211968000,"data_stored":211696024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":34.186,"5min":34.186,"15min":34.186},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":497.04000000000002,"5min":497.04000000000002,"15min":497.04000000000002},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":4.1980000000000004,"5min":4.1980000000000004,"15min":4.1980000000000004},"last":1.25}]},{"osd":2,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":1.278},{"interface":"front","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":4.0919999999999996,"5min":4.0919999999999996,"15min":4.0919999999999996},"last":1.206}]},{"osd":3,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":2.4590000000000001},{"interface":"front","average":{"1min":1.454,"5min":1.454,"15min":1.454},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.9889999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":2.484}]},{"osd":4,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":1.294},{"interface":"front","average":{"1min":1.2,"5min":1.2,"15min":1.2},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.401}]},{"osd":5,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":13.007999999999999,"5min":13.007999999999999,"15min":13.007999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":213.63800000000001,"5min":213.63800000000001,"15min":213.63800000000001},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":1.5669999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.694}]},{"osd":7,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.615,"5min":1.615,"15min":1.615},"min":{"1min":1.046,"5min":1.046,"15min":1.046},"max":{"1min":3.2810000000000001,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":1.738},{"interface":"front","average":{"1min":1.48,"5min":1.48,"15min":1.48},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.8759999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.4570000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1248136,"kb_used_data":199552,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93123704,"statfs":{"total":96636764160,"available":95358672896,"internally_reserved":1073741824,"allocated":204341248,"data_stored":204069522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3],"upper_bound":4},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":2.4500000000000002},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.2440000000000002,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":0.77400000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.1200000000000001},{"interface":"front","average":{"1min":32.740000000000002,"5min":32.740000000000002,"15min":32.740000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":606.18499999999995,"5min":606.18499999999995,"15min":606.18499999999995},"last":1.0049999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.3340000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.6949999999999998,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":2.4060000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.0780000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":2.206},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.98199999999999998}]},{"osd":6,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":8.6140000000000008,"5min":8.6140000000000008,"15min":8.6140000000000008},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":124.738,"5min":124.738,"15min":124.738},"last":1.6639999999999999},{"interface":"front","average":{"1min":18.652999999999999,"5min":18.652999999999999,"15min":18.652999999999999},"min":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"max":{"1min":294.94,"5min":294.94,"15min":294.94},"last":1.26}]},{"osd":7,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":6.468,"5min":6.468,"15min":6.468},"last":1.7150000000000001},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.883,"5min":2.883,"15min":2.883},"last":2.0640000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1176692,"kb_used_data":128108,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93195148,"statfs":{"total":96636764160,"available":95431831552,"internally_reserved":1073741824,"allocated":131182592,"data_stored":130914134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":3.2240000000000002,"5min":3.2240000000000002,"15min":3.2240000000000002},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.55500000000000005}]},{"osd":1,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":3.1320000000000001,"5min":3.1320000000000001,"15min":3.1320000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":3.3069999999999999,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.80300000000000005}]},{"osd":3,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.3759999999999999,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.556,"5min":1.556,"15min":1.556},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":8.2210000000000001,"5min":8.2210000000000001,"15min":8.2210000000000001},"last":1.3520000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":19.783000000000001},{"interface":"front","average":{"1min":1.643,"5min":1.643,"15min":1.643},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":3.2749999999999999,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":0.84899999999999998}]},{"osd":5,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.101},{"interface":"front","average":{"1min":25.524999999999999,"5min":25.524999999999999,"15min":25.524999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":415.40699999999998,"5min":415.40699999999998,"15min":415.40699999999998},"last":1.325}]},{"osd":6,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.94499999999999995}]},{"osd":7,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.742,"5min":1.742,"15min":1.742},"min":{"1min":1.093,"5min":1.093,"15min":1.093},"max":{"1min":4.056,"5min":4.056,"15min":4.056},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":4.4539999999999997,"5min":4.4539999999999997,"15min":4.4539999999999997},"last":1.204}]}]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1277048,"kb_used_data":228464,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93094792,"statfs":{"total":96636764160,"available":95329067008,"internally_reserved":1073741824,"allocated":233947136,"data_stored":233675160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,1,2,7,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":2.8220000000000001,"5min":2.8220000000000001,"15min":2.8220000000000001},"last":1.7210000000000001},{"interface":"front","average":{"1min":1.637,"5min":1.637,"15min":1.637},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":13.067,"5min":13.067,"15min":13.067},"last":1.0029999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.899,"5min":1.899,"15min":1.899},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":6.6829999999999998,"5min":6.6829999999999998,"15min":6.6829999999999998},"last":1.681},{"interface":"front","average":{"1min":2.012,"5min":2.012,"15min":2.012},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":11.936,"5min":11.936,"15min":11.936},"last":2.2690000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.589,"5min":3.589,"15min":3.589},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":5.5890000000000004,"5min":5.5890000000000004,"15min":5.5890000000000004},"last":1.397}]},{"osd":4,"last update":"Fri Aug 7 15:07:19 2020","interfaces":[{"interface":"back","average":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":4.556,"5min":4.556,"15min":4.556},"last":2.0539999999999998},{"interface":"front","average":{"1min":32.100000000000001,"5min":32.100000000000001,"15min":32.100000000000001},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":677.88599999999997,"5min":677.88599999999997,"15min":677.88599999999997},"last":677.88599999999997}]},{"osd":5,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":3.6600000000000001,"5min":3.6600000000000001,"15min":3.6600000000000001},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"min":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":5.4329999999999998,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":1.6970000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":3.621,"5min":3.621,"15min":3.621},"last":1.6559999999999999},{"interface":"front","average":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":5.6189999999999998,"5min":5.6189999999999998,"15min":5.6189999999999998},"last":1.4430000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":5.734,"5min":5.734,"15min":5.734},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":4.8540000000000001,"5min":4.8540000000000001,"15min":4.8540000000000001},"last":4.8540000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1317052,"kb_used_data":268468,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93054788,"statfs":{"total":96636764160,"available":95288102912,"internally_reserved":1073741824,"allocated":274911232,"data_stored":274635988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.181,"15min":1.181},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":4.9480000000000004,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.56799999999999995}]},{"osd":1,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":4.9829999999999997,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.377,"5min":1.377,"15min":1.377},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":5.0430000000000001,"5min":5.0430000000000001,"15min":5.0430000000000001},"last":1.095}]},{"osd":2,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.452,"15min":1.452},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":4.415,"5min":4.415,"15min":4.415},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":4.4340000000000002,"5min":4.4340000000000002,"15min":4.4340000000000002},"last":1.462}]},{"osd":3,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.508,"15min":1.508},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":5.0099999999999998,"5min":5.0099999999999998,"15min":5.0099999999999998},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":3.2029999999999998,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":0.77400000000000002}]},{"osd":5,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":5.157,"5min":5.157,"15min":5.157},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.627,"5min":1.627,"15min":1.627},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":5.0979999999999999,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":1.1879999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.79,"5min":1.79,"15min":1.79},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":5.1310000000000002,"5min":5.1310000000000002,"15min":5.1310000000000002},"last":1.1499999999999999},{"interface":"front","average":{"1min":19.109000000000002,"5min":19.109000000000002,"15min":19.109000000000002},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":389.14600000000002,"5min":389.14600000000002,"15min":389.14600000000002},"last":1.0049999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":3.3540000000000001,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":252.798},{"interface":"front","average":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":4.4569999999999999,"5min":4.4569999999999999,"15min":4.4569999999999999},"last":221.93799999999999}]}]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1214588,"kb_used_data":166004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93157252,"statfs":{"total":96636764160,"available":95393026048,"internally_reserved":1073741824,"allocated":169988096,"data_stored":169712852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3819999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4259999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.829}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1140000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]}]}]}} 2020-08-07T15:07:24.191 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:07:24.194 INFO:tasks.ceph.ceph_manager.ceph:increase pool size by 10 2020-08-07T15:07:24.194 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set device_health_metrics pg_num 11 2020-08-07T15:07:24.196 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:24.185+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:24.197 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:07:24.245 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 48 16 71045 71029 11.5598 12.7812 0.00167019 0.010806 2020-08-07T15:07:24.289 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:24.287+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:24.395 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:24.387+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:24.410 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:07:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:07:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:07:24.411 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:24.424 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:07:24.490 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:24.489+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:24.591 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:24.589+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:24.635 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:24.635 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:07:24.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:07:24.635 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:24.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.636 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:2400 3.e 3:70ffa62d:::benchmark_data_smithi074_35447_object299:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:07:24.636 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:37.694617+0000", 2020-08-07T15:07:24.636 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.937049281, 2020-08-07T15:07:24.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24075575099999999, 2020-08-07T15:07:24.637 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:24.637 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:24.637 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:24.637 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:24.637 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:24.638 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2400 2020-08-07T15:07:24.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.638 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:24.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:24.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.694617+0000", 2020-08-07T15:07:24.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:24.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:24.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.694617+0000", 2020-08-07T15:07:24.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5681999999999999e-05 2020-08-07T15:07:24.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:24.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.694643+0000", 2020-08-07T15:07:24.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0079999999999998e-06 2020-08-07T15:07:24.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:24.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.694646+0000", 2020-08-07T15:07:24.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.015e-06 2020-08-07T15:07:24.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:24.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.694652+0000", 2020-08-07T15:07:24.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7130000000000001e-06 2020-08-07T15:07:24.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:24.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.694655+0000", 2020-08-07T15:07:24.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031117319999999999 2020-08-07T15:07:24.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:24.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.697766+0000", 2020-08-07T15:07:24.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015382800000000001 2020-08-07T15:07:24.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:24.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.697920+0000", 2020-08-07T15:07:24.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00044993600000000001 2020-08-07T15:07:24.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:24.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.698370+0000", 2020-08-07T15:07:24.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027536739999999998 2020-08-07T15:07:24.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:24.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.701124+0000", 2020-08-07T15:07:24.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999999e-07 2020-08-07T15:07:24.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:24.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.701124+0000", 2020-08-07T15:07:24.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7494000000000001e-05 2020-08-07T15:07:24.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:24.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.701152+0000", 2020-08-07T15:07:24.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086935569999999993 2020-08-07T15:07:24.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:24.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.709845+0000", 2020-08-07T15:07:24.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.225456669 2020-08-07T15:07:24.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:24.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.935302+0000", 2020-08-07T15:07:24.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.043e-06 2020-08-07T15:07:24.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:24.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.935303+0000", 2020-08-07T15:07:24.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9963999999999996e-05 2020-08-07T15:07:24.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:24.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:37.935373+0000", 2020-08-07T15:07:24.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24075575099999999 2020-08-07T15:07:24.656 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.656 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:24.657 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.657 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:9352 3.e 3:7113579b:::benchmark_data_smithi074_35447_object1168:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:24.657 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.885925+0000", 2020-08-07T15:07:24.658 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.745741326999998, 2020-08-07T15:07:24.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24955413800000001, 2020-08-07T15:07:24.658 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:24.658 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:24.658 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:24.659 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:24.659 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:24.659 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 9352 2020-08-07T15:07:24.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.659 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:24.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:24.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.885925+0000", 2020-08-07T15:07:24.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:24.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:24.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.885925+0000", 2020-08-07T15:07:24.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4308999999999998e-05 2020-08-07T15:07:24.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:24.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.885979+0000", 2020-08-07T15:07:24.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0440000000000001e-06 2020-08-07T15:07:24.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:24.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.885985+0000", 2020-08-07T15:07:24.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9462000000000001e-05 2020-08-07T15:07:24.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:24.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.886005+0000", 2020-08-07T15:07:24.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2530000000000001e-06 2020-08-07T15:07:24.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:24.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.886011+0000", 2020-08-07T15:07:24.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025236080000000001 2020-08-07T15:07:24.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:24.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.888535+0000", 2020-08-07T15:07:24.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112871 2020-08-07T15:07:24.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:24.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.888648+0000", 2020-08-07T15:07:24.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020764399999999999 2020-08-07T15:07:24.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:24.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.888855+0000", 2020-08-07T15:07:24.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048523799999999999 2020-08-07T15:07:24.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:24.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.893708+0000", 2020-08-07T15:07:24.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030190300000000001 2020-08-07T15:07:24.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:24.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.894010+0000", 2020-08-07T15:07:24.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24135353300000001 2020-08-07T15:07:24.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:24.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.135363+0000", 2020-08-07T15:07:24.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9899999999999999e-07 2020-08-07T15:07:24.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:24.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.135364+0000", 2020-08-07T15:07:24.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7601999999999994e-05 2020-08-07T15:07:24.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:24.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.135431+0000", 2020-08-07T15:07:24.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7729999999999999e-05 2020-08-07T15:07:24.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:24.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.135479+0000", 2020-08-07T15:07:24.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24955413800000001 2020-08-07T15:07:24.675 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.675 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:24.675 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.676 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:9976 3.e 3:7fb2e2e5:::benchmark_data_smithi074_35447_object1246:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:24.676 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:07:24.676 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.394468439999997, 2020-08-07T15:07:24.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28420182999999999, 2020-08-07T15:07:24.676 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:24.677 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:24.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:24.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:24.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:24.677 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 9976 2020-08-07T15:07:24.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.678 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:24.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:24.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:07:24.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:24.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:24.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:07:24.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7113e-05 2020-08-07T15:07:24.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:24.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237255+0000", 2020-08-07T15:07:24.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5869999999999998e-06 2020-08-07T15:07:24.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:24.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237262+0000", 2020-08-07T15:07:24.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4467999999999999e-05 2020-08-07T15:07:24.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:24.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237276+0000", 2020-08-07T15:07:24.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0749999999999997e-06 2020-08-07T15:07:24.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:24.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237281+0000", 2020-08-07T15:07:24.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013795109999999999 2020-08-07T15:07:24.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:24.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238661+0000", 2020-08-07T15:07:24.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100309 2020-08-07T15:07:24.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:24.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238761+0000", 2020-08-07T15:07:24.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000141784 2020-08-07T15:07:24.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:24.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238903+0000", 2020-08-07T15:07:24.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00070626599999999999 2020-08-07T15:07:24.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:24.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239609+0000", 2020-08-07T15:07:24.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3499999999999996e-07 2020-08-07T15:07:24.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:24.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239610+0000", 2020-08-07T15:07:24.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0846000000000006e-05 2020-08-07T15:07:24.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:24.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239680+0000", 2020-08-07T15:07:24.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022037080000000001 2020-08-07T15:07:24.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:24.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.261718+0000", 2020-08-07T15:07:24.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25961784500000001 2020-08-07T15:07:24.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:24.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521335+0000", 2020-08-07T15:07:24.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2839999999999999e-06 2020-08-07T15:07:24.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:24.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521337+0000", 2020-08-07T15:07:24.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3126999999999994e-05 2020-08-07T15:07:24.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:24.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521400+0000", 2020-08-07T15:07:24.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28420182999999999 2020-08-07T15:07:24.695 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.696 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:24.696 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.696 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:9976 3.e e24/20 ondisk, result = 0)", 2020-08-07T15:07:24.697 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.269400+0000", 2020-08-07T15:07:24.697 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.362266216000002, 2020-08-07T15:07:24.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25200699900000001, 2020-08-07T15:07:24.704 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:24.704 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:24.704 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:24.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:24.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.269400+0000", 2020-08-07T15:07:24.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:24.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:24.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.269400+0000", 2020-08-07T15:07:24.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1279999999999999e-05 2020-08-07T15:07:24.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:24.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.269431+0000", 2020-08-07T15:07:24.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.071e-06 2020-08-07T15:07:24.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:24.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.269438+0000", 2020-08-07T15:07:24.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24671454900000001 2020-08-07T15:07:24.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:24.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.516152+0000", 2020-08-07T15:07:24.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4140000000000002e-06 2020-08-07T15:07:24.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:24.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.516157+0000", 2020-08-07T15:07:24.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051603170000000002 2020-08-07T15:07:24.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:24.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521318+0000", 2020-08-07T15:07:24.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7479999999999999e-06 2020-08-07T15:07:24.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:24.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521327+0000", 2020-08-07T15:07:24.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0619999999999997e-05 2020-08-07T15:07:24.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:24.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521407+0000", 2020-08-07T15:07:24.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25200699900000001 2020-08-07T15:07:24.716 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.716 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:24.716 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.717 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15576 3.e 3:727724a5:::benchmark_data_smithi074_35447_object1946:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:24.717 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:07:24.717 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.965031476999997, 2020-08-07T15:07:24.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999, 2020-08-07T15:07:24.718 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:24.718 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:24.718 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:24.718 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:24.718 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:24.719 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15576 2020-08-07T15:07:24.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.719 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:24.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:24.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:07:24.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:24.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:24.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:07:24.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7156e-05 2020-08-07T15:07:24.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:24.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666682+0000", 2020-08-07T15:07:24.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0209999999999999e-06 2020-08-07T15:07:24.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:24.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666687+0000", 2020-08-07T15:07:24.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0730999999999999e-05 2020-08-07T15:07:24.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:24.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666698+0000", 2020-08-07T15:07:24.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2020-08-07T15:07:24.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:24.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666702+0000", 2020-08-07T15:07:24.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023978459999999999 2020-08-07T15:07:24.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:24.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669100+0000", 2020-08-07T15:07:24.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9664000000000001e-05 2020-08-07T15:07:24.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:24.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669190+0000", 2020-08-07T15:07:24.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015957100000000001 2020-08-07T15:07:24.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:24.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669350+0000", 2020-08-07T15:07:24.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016470149999999999 2020-08-07T15:07:24.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:24.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:07:24.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1299999999999999e-07 2020-08-07T15:07:24.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:24.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:07:24.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5269000000000002e-05 2020-08-07T15:07:24.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:24.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.671022+0000", 2020-08-07T15:07:24.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031414400000000001 2020-08-07T15:07:24.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:24.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.674163+0000", 2020-08-07T15:07:24.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45641417400000001 2020-08-07T15:07:24.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:24.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:07:24.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0399999999999995e-07 2020-08-07T15:07:24.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:24.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:07:24.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3343999999999997e-05 2020-08-07T15:07:24.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:24.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130622+0000", 2020-08-07T15:07:24.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999 2020-08-07T15:07:24.768 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.768 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:24.768 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.769 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15743 3.d 3:b5661143:::benchmark_data_smithi074_35447_object1967:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:24.769 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.768880+0000", 2020-08-07T15:07:24.769 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.862786065999998, 2020-08-07T15:07:24.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24181233499999999, 2020-08-07T15:07:24.770 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:24.770 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:24.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:24.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:24.781 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:24.781 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15743 2020-08-07T15:07:24.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.782 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:24.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:24.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.768880+0000", 2020-08-07T15:07:24.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:24.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:24.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.768880+0000", 2020-08-07T15:07:24.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8946000000000002e-05 2020-08-07T15:07:24.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:24.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.768929+0000", 2020-08-07T15:07:24.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9020000000000004e-06 2020-08-07T15:07:24.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:24.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.768935+0000", 2020-08-07T15:07:24.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2283000000000001e-05 2020-08-07T15:07:24.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:24.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.768947+0000", 2020-08-07T15:07:24.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.186e-06 2020-08-07T15:07:24.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:24.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.768953+0000", 2020-08-07T15:07:24.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0068896529999999999 2020-08-07T15:07:24.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:24.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.775842+0000", 2020-08-07T15:07:24.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0270000000000001e-05 2020-08-07T15:07:24.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:24.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.775933+0000", 2020-08-07T15:07:24.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017233799999999999 2020-08-07T15:07:24.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:24.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.776105+0000", 2020-08-07T15:07:24.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00117592 2020-08-07T15:07:24.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:24.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777281+0000", 2020-08-07T15:07:24.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6499999999999998e-07 2020-08-07T15:07:24.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:24.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777282+0000", 2020-08-07T15:07:24.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3357999999999999e-05 2020-08-07T15:07:24.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:24.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777345+0000", 2020-08-07T15:07:24.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030411169999999999 2020-08-07T15:07:24.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:24.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.780386+0000", 2020-08-07T15:07:24.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.230239216 2020-08-07T15:07:24.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:24.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.010625+0000", 2020-08-07T15:07:24.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5600000000000005e-07 2020-08-07T15:07:24.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:24.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.010626+0000", 2020-08-07T15:07:24.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6625000000000007e-05 2020-08-07T15:07:24.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:24.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.010693+0000", 2020-08-07T15:07:24.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24181233499999999 2020-08-07T15:07:24.803 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.803 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:24.803 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.804 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15744 3.d 3:b5661143:::benchmark_data_smithi074_35447_object1967:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:24.804 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.769136+0000", 2020-08-07T15:07:24.805 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.862530258, 2020-08-07T15:07:24.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24180031099999999, 2020-08-07T15:07:24.805 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:24.806 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:24.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:24.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:24.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:24.806 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15744 2020-08-07T15:07:24.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.807 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:24.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:24.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.769136+0000", 2020-08-07T15:07:24.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:24.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:24.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.769136+0000", 2020-08-07T15:07:24.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5503999999999997e-05 2020-08-07T15:07:24.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:24.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.769182+0000", 2020-08-07T15:07:24.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.664e-06 2020-08-07T15:07:24.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:24.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.769185+0000", 2020-08-07T15:07:24.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7200000000000006e-06 2020-08-07T15:07:24.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:24.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.769193+0000", 2020-08-07T15:07:24.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8260000000000003e-06 2020-08-07T15:07:24.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:24.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.769197+0000", 2020-08-07T15:07:24.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0074215849999999996 2020-08-07T15:07:24.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:24.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.776618+0000", 2020-08-07T15:07:24.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010513700000000001 2020-08-07T15:07:24.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:24.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.776724+0000", 2020-08-07T15:07:24.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016319800000000001 2020-08-07T15:07:24.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:24.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.776887+0000", 2020-08-07T15:07:24.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00090081899999999997 2020-08-07T15:07:24.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:24.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777788+0000", 2020-08-07T15:07:24.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5000000000000001e-07 2020-08-07T15:07:24.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:24.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777788+0000", 2020-08-07T15:07:24.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5197000000000006e-05 2020-08-07T15:07:24.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:24.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777854+0000", 2020-08-07T15:07:24.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22872283800000001 2020-08-07T15:07:24.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:24.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.006576+0000", 2020-08-07T15:07:24.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004298367 2020-08-07T15:07:24.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:24.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.010875+0000", 2020-08-07T15:07:24.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.08e-07 2020-08-07T15:07:24.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:24.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.010875+0000", 2020-08-07T15:07:24.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1198000000000006e-05 2020-08-07T15:07:24.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:24.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.010936+0000", 2020-08-07T15:07:24.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24180031099999999 2020-08-07T15:07:24.859 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.859 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:24.860 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.860 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:17824 3.d 3:bd4e0935:::benchmark_data_smithi074_35447_object2227:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:24.860 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.140708+0000", 2020-08-07T15:07:24.861 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.490958554000002, 2020-08-07T15:07:24.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25688457100000001, 2020-08-07T15:07:24.861 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:24.861 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:24.861 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:24.862 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:24.862 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:24.862 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 17824 2020-08-07T15:07:24.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.862 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:24.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:24.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140708+0000", 2020-08-07T15:07:24.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:24.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:24.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140708+0000", 2020-08-07T15:07:24.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0522000000000002e-05 2020-08-07T15:07:24.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:24.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140758+0000", 2020-08-07T15:07:24.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1680000000000001e-06 2020-08-07T15:07:24.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:24.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140765+0000", 2020-08-07T15:07:24.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5791999999999998e-05 2020-08-07T15:07:24.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:24.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140810+0000", 2020-08-07T15:07:24.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2389999999999996e-06 2020-08-07T15:07:24.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:24.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140818+0000", 2020-08-07T15:07:24.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001721544 2020-08-07T15:07:24.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:24.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.142539+0000", 2020-08-07T15:07:24.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020908100000000001 2020-08-07T15:07:24.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:24.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.142748+0000", 2020-08-07T15:07:24.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019000000000000001 2020-08-07T15:07:24.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:24.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.142938+0000", 2020-08-07T15:07:24.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001381279 2020-08-07T15:07:24.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:24.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.144319+0000", 2020-08-07T15:07:24.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010975570000000001 2020-08-07T15:07:24.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:24.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.145417+0000", 2020-08-07T15:07:24.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.252064128 2020-08-07T15:07:24.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:24.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397481+0000", 2020-08-07T15:07:24.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003e-06 2020-08-07T15:07:24.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:24.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397482+0000", 2020-08-07T15:07:24.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3081000000000001e-05 2020-08-07T15:07:24.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:24.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397545+0000", 2020-08-07T15:07:24.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7176999999999999e-05 2020-08-07T15:07:24.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:24.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397592+0000", 2020-08-07T15:07:24.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25688457100000001 2020-08-07T15:07:24.892 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.892 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:24.892 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.893 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:35576 3.f 3:fca2de08:::benchmark_data_smithi074_35447_object4446:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-08-07T15:07:24.893 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:59.287747+0000", 2020-08-07T15:07:24.894 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.343919751000001, 2020-08-07T15:07:24.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25861668500000001, 2020-08-07T15:07:24.894 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:24.895 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:24.895 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:24.895 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:24.895 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:24.895 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 35576 2020-08-07T15:07:24.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.896 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:24.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:24.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287747+0000", 2020-08-07T15:07:24.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:24.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:24.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287747+0000", 2020-08-07T15:07:24.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0089000000000001e-05 2020-08-07T15:07:24.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:24.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287797+0000", 2020-08-07T15:07:24.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4219999999999997e-06 2020-08-07T15:07:24.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:24.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287803+0000", 2020-08-07T15:07:24.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4132e-05 2020-08-07T15:07:24.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:24.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287817+0000", 2020-08-07T15:07:24.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1600000000000003e-06 2020-08-07T15:07:24.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:24.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287823+0000", 2020-08-07T15:07:24.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019418230000000001 2020-08-07T15:07:24.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:24.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.289765+0000", 2020-08-07T15:07:24.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110597 2020-08-07T15:07:24.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:24.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.289876+0000", 2020-08-07T15:07:24.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017674800000000001 2020-08-07T15:07:24.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:24.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.290053+0000", 2020-08-07T15:07:24.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052995730000000001 2020-08-07T15:07:24.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:24.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295352+0000", 2020-08-07T15:07:24.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9999999999999997e-07 2020-08-07T15:07:24.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:24.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295353+0000", 2020-08-07T15:07:24.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6803000000000004e-05 2020-08-07T15:07:24.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:24.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295420+0000", 2020-08-07T15:07:24.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015611539999999999 2020-08-07T15:07:24.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:24.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.296981+0000", 2020-08-07T15:07:24.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.249314866 2020-08-07T15:07:24.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:24.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546296+0000", 2020-08-07T15:07:24.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7300000000000005e-07 2020-08-07T15:07:24.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:24.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546296+0000", 2020-08-07T15:07:24.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6845000000000001e-05 2020-08-07T15:07:24.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:24.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546363+0000", 2020-08-07T15:07:24.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25861668500000001 2020-08-07T15:07:24.914 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.914 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:24.917 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.918 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:35576 3.f e31/20 ondisk, result = 0)", 2020-08-07T15:07:24.918 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:59.295024+0000", 2020-08-07T15:07:24.918 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.336642577999999, 2020-08-07T15:07:24.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25134726499999999, 2020-08-07T15:07:24.918 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:24.919 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:24.919 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:24.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:24.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295024+0000", 2020-08-07T15:07:24.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:24.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:24.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295024+0000", 2020-08-07T15:07:24.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7844999999999999e-05 2020-08-07T15:07:24.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:24.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295052+0000", 2020-08-07T15:07:24.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7740000000000004e-06 2020-08-07T15:07:24.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:24.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295056+0000", 2020-08-07T15:07:24.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25017799099999999 2020-08-07T15:07:24.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:24.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.545234+0000", 2020-08-07T15:07:24.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5140000000000003e-06 2020-08-07T15:07:24.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:24.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.545239+0000", 2020-08-07T15:07:24.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010386989999999999 2020-08-07T15:07:24.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:24.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546278+0000", 2020-08-07T15:07:24.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0480000000000004e-06 2020-08-07T15:07:24.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:24.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546287+0000", 2020-08-07T15:07:24.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4394000000000005e-05 2020-08-07T15:07:24.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:24.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546371+0000", 2020-08-07T15:07:24.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25134726499999999 2020-08-07T15:07:24.930 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.930 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:24.931 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.931 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:36096 3.f 3:ffeda9b1:::benchmark_data_smithi074_35447_object4511:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-08-07T15:07:24.931 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:59.572029+0000", 2020-08-07T15:07:24.932 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.059637310999999, 2020-08-07T15:07:24.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24717645599999999, 2020-08-07T15:07:24.932 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:24.932 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:24.932 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:24.933 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:24.933 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:24.933 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 36096 2020-08-07T15:07:24.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.934 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:24.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:24.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.572029+0000", 2020-08-07T15:07:24.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:24.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:24.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.572029+0000", 2020-08-07T15:07:24.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1946e-05 2020-08-07T15:07:24.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:24.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.572081+0000", 2020-08-07T15:07:24.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5559999999999999e-06 2020-08-07T15:07:24.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:24.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.572085+0000", 2020-08-07T15:07:24.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3620000000000002e-06 2020-08-07T15:07:24.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:24.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.572091+0000", 2020-08-07T15:07:24.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1020000000000001e-06 2020-08-07T15:07:24.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:24.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.572094+0000", 2020-08-07T15:07:24.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037454749999999998 2020-08-07T15:07:24.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:24.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.575840+0000", 2020-08-07T15:07:24.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1586e-05 2020-08-07T15:07:24.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:24.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.575931+0000", 2020-08-07T15:07:24.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014823700000000001 2020-08-07T15:07:24.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:24.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.576079+0000", 2020-08-07T15:07:24.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005540016 2020-08-07T15:07:24.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:24.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.581619+0000", 2020-08-07T15:07:24.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4499999999999995e-07 2020-08-07T15:07:24.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:24.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.581620+0000", 2020-08-07T15:07:24.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3874999999999994e-05 2020-08-07T15:07:24.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:24.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.581684+0000", 2020-08-07T15:07:24.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084986090000000007 2020-08-07T15:07:24.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:24.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.590183+0000", 2020-08-07T15:07:24.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22895041599999999 2020-08-07T15:07:24.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:24.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.819133+0000", 2020-08-07T15:07:24.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.44e-07 2020-08-07T15:07:24.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:24.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.819134+0000", 2020-08-07T15:07:24.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1687000000000007e-05 2020-08-07T15:07:24.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:24.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.819206+0000", 2020-08-07T15:07:24.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24717645599999999 2020-08-07T15:07:24.962 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.962 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:24.962 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:24.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.963 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:41431 3.d 3:b5c8a160:::benchmark_data_smithi074_35447_object5178:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:07:24.963 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.791939+0000", 2020-08-07T15:07:24.963 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.839726947999999, 2020-08-07T15:07:24.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28436496, 2020-08-07T15:07:24.963 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:24.964 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:24.964 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:24.964 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:24.964 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:24.964 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 41431 2020-08-07T15:07:24.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.965 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:24.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:24.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791939+0000", 2020-08-07T15:07:24.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:24.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:24.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791939+0000", 2020-08-07T15:07:24.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6304999999999998e-05 2020-08-07T15:07:24.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:24.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791986+0000", 2020-08-07T15:07:24.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6339999999999998e-06 2020-08-07T15:07:24.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:24.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791991+0000", 2020-08-07T15:07:24.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.026e-05 2020-08-07T15:07:24.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:24.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792002+0000", 2020-08-07T15:07:24.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6240000000000001e-06 2020-08-07T15:07:24.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:24.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792006+0000", 2020-08-07T15:07:24.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015600220000000001 2020-08-07T15:07:24.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:24.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.793566+0000", 2020-08-07T15:07:24.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3628999999999994e-05 2020-08-07T15:07:24.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:24.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.793660+0000", 2020-08-07T15:07:24.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000168197 2020-08-07T15:07:24.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:24.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.793828+0000", 2020-08-07T15:07:24.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001273251 2020-08-07T15:07:24.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:24.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795101+0000", 2020-08-07T15:07:24.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5899999999999998e-07 2020-08-07T15:07:24.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:24.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795102+0000", 2020-08-07T15:07:24.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4904000000000001e-05 2020-08-07T15:07:24.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:24.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795127+0000", 2020-08-07T15:07:24.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00670237 2020-08-07T15:07:24.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:24.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801829+0000", 2020-08-07T15:07:24.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27442024999999998 2020-08-07T15:07:24.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:24.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:24.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:24.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076249+0000", 2020-08-07T15:07:24.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999997e-07 2020-08-07T15:07:24.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:25.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076250+0000", 2020-08-07T15:07:25.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4369000000000001e-05 2020-08-07T15:07:25.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:25.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076304+0000", 2020-08-07T15:07:25.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28436496 2020-08-07T15:07:25.001 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:25.002 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:25.003 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:25.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.003 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:41432 3.d 3:b5c8a160:::benchmark_data_smithi074_35447_object5178:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:07:25.004 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:07:25.004 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.839376252000001, 2020-08-07T15:07:25.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.37720701699999998, 2020-08-07T15:07:25.005 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:25.005 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:25.005 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:25.006 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:25.006 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:25.008 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 41432 2020-08-07T15:07:25.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.010 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:25.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:25.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:07:25.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:25.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:25.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:07:25.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4787999999999999e-05 2020-08-07T15:07:25.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:25.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792335+0000", 2020-08-07T15:07:25.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5999999999999997e-06 2020-08-07T15:07:25.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:25.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792341+0000", 2020-08-07T15:07:25.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0998e-05 2020-08-07T15:07:25.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:25.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792352+0000", 2020-08-07T15:07:25.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2989999999999996e-06 2020-08-07T15:07:25.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:25.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792357+0000", 2020-08-07T15:07:25.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001852752 2020-08-07T15:07:25.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:25.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794210+0000", 2020-08-07T15:07:25.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5771000000000002e-05 2020-08-07T15:07:25.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:25.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794305+0000", 2020-08-07T15:07:25.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012776399999999999 2020-08-07T15:07:25.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:25.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794433+0000", 2020-08-07T15:07:25.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00069673599999999999 2020-08-07T15:07:25.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:25.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795130+0000", 2020-08-07T15:07:25.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1500000000000001e-07 2020-08-07T15:07:25.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:25.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795130+0000", 2020-08-07T15:07:25.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7319e-05 2020-08-07T15:07:25.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:25.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795167+0000", 2020-08-07T15:07:25.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27910026700000001 2020-08-07T15:07:25.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:25.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074268+0000", 2020-08-07T15:07:25.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.095150660999999997 2020-08-07T15:07:25.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:25.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169418+0000", 2020-08-07T15:07:25.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.423e-06 2020-08-07T15:07:25.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:25.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169420+0000", 2020-08-07T15:07:25.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7423999999999996e-05 2020-08-07T15:07:25.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:25.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169497+0000", 2020-08-07T15:07:25.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.37720701699999998 2020-08-07T15:07:25.032 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:25.033 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:25.033 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:25.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.035 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:41431 3.d e32/20 ondisk, result = 0)", 2020-08-07T15:07:25.035 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.796229+0000", 2020-08-07T15:07:25.035 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.835437349999999, 2020-08-07T15:07:25.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28007903299999998, 2020-08-07T15:07:25.036 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:25.036 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:25.036 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:25.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:25.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.796229+0000", 2020-08-07T15:07:25.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:25.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:25.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.796229+0000", 2020-08-07T15:07:25.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3272000000000003e-05 2020-08-07T15:07:25.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:25.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.796262+0000", 2020-08-07T15:07:25.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9579999999999998e-06 2020-08-07T15:07:25.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:25.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.796267+0000", 2020-08-07T15:07:25.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27993393599999999 2020-08-07T15:07:25.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:25.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076201+0000", 2020-08-07T15:07:25.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8709999999999999e-06 2020-08-07T15:07:25.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:25.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076205+0000", 2020-08-07T15:07:25.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9128e-05 2020-08-07T15:07:25.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:25.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076234+0000", 2020-08-07T15:07:25.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9800000000000001e-06 2020-08-07T15:07:25.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:25.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076241+0000", 2020-08-07T15:07:25.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6888000000000001e-05 2020-08-07T15:07:25.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:25.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076308+0000", 2020-08-07T15:07:25.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28007903299999998 2020-08-07T15:07:25.045 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:25.046 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:25.046 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:25.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.047 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:41432 3.d e32/20 ondisk, result = 0)", 2020-08-07T15:07:25.047 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.801630+0000", 2020-08-07T15:07:25.047 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.830036449000001, 2020-08-07T15:07:25.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27264033999999998, 2020-08-07T15:07:25.048 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:25.048 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:25.048 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:25.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:25.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801630+0000", 2020-08-07T15:07:25.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:25.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:25.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801630+0000", 2020-08-07T15:07:25.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6257e-05 2020-08-07T15:07:25.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:25.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801646+0000", 2020-08-07T15:07:25.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5450000000000002e-06 2020-08-07T15:07:25.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:25.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801649+0000", 2020-08-07T15:07:25.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27254806999999998 2020-08-07T15:07:25.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:25.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074197+0000", 2020-08-07T15:07:25.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4079999999999997e-06 2020-08-07T15:07:25.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:25.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074202+0000", 2020-08-07T15:07:25.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5222000000000002e-05 2020-08-07T15:07:25.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:25.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074247+0000", 2020-08-07T15:07:25.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9459999999999993e-06 2020-08-07T15:07:25.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:25.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074257+0000", 2020-08-07T15:07:25.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2892e-05 2020-08-07T15:07:25.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:25.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074270+0000", 2020-08-07T15:07:25.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27264033999999998 2020-08-07T15:07:25.057 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:25.058 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:25.058 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:25.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.058 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:42104 3.d 3:bcc35439:::benchmark_data_smithi074_35447_object5262:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:07:25.059 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:07:25.059 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.392744081, 2020-08-07T15:07:25.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28812091400000001, 2020-08-07T15:07:25.059 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:25.061 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:25.061 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:25.061 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:25.061 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:25.062 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42104 2020-08-07T15:07:25.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.062 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:25.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:25.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:07:25.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:25.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:25.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:07:25.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9734000000000002e-05 2020-08-07T15:07:25.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:25.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238972+0000", 2020-08-07T15:07:25.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6910000000000002e-06 2020-08-07T15:07:25.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:25.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238978+0000", 2020-08-07T15:07:25.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1864e-05 2020-08-07T15:07:25.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:25.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238990+0000", 2020-08-07T15:07:25.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.737e-06 2020-08-07T15:07:25.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:25.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238996+0000", 2020-08-07T15:07:25.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00283943 2020-08-07T15:07:25.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:25.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.241836+0000", 2020-08-07T15:07:25.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4712999999999999e-05 2020-08-07T15:07:25.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:25.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.241930+0000", 2020-08-07T15:07:25.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015328499999999999 2020-08-07T15:07:25.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:25.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.242084+0000", 2020-08-07T15:07:25.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016104400000000001 2020-08-07T15:07:25.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:25.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243694+0000", 2020-08-07T15:07:25.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1399999999999997e-07 2020-08-07T15:07:25.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:25.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243695+0000", 2020-08-07T15:07:25.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7800999999999998e-05 2020-08-07T15:07:25.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:25.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243753+0000", 2020-08-07T15:07:25.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090405661999999998 2020-08-07T15:07:25.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:25.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.334158+0000", 2020-08-07T15:07:25.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19283929299999999 2020-08-07T15:07:25.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:25.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.526997+0000", 2020-08-07T15:07:25.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.012e-06 2020-08-07T15:07:25.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:25.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.526998+0000", 2020-08-07T15:07:25.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4737999999999998e-05 2020-08-07T15:07:25.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:25.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.527043+0000", 2020-08-07T15:07:25.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28812091400000001 2020-08-07T15:07:25.099 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:25.100 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:25.101 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:25.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.114 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:44120 3.e 3:7e6f7bb9:::benchmark_data_smithi074_35447_object5514:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:07:25.114 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:04.450106+0000", 2020-08-07T15:07:25.116 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.181560547, 2020-08-07T15:07:25.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26702573699999999, 2020-08-07T15:07:25.116 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:25.116 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:25.117 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:25.117 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:25.117 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:25.117 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 44120 2020-08-07T15:07:25.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.120 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:25.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:25.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450106+0000", 2020-08-07T15:07:25.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:25.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:25.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450106+0000", 2020-08-07T15:07:25.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9926000000000003e-05 2020-08-07T15:07:25.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:25.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450156+0000", 2020-08-07T15:07:25.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4079999999999997e-06 2020-08-07T15:07:25.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:25.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450161+0000", 2020-08-07T15:07:25.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0494e-05 2020-08-07T15:07:25.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:25.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450172+0000", 2020-08-07T15:07:25.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3950000000000003e-06 2020-08-07T15:07:25.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:25.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450176+0000", 2020-08-07T15:07:25.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033213689999999998 2020-08-07T15:07:25.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:25.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.453497+0000", 2020-08-07T15:07:25.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102414 2020-08-07T15:07:25.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:25.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.453600+0000", 2020-08-07T15:07:25.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019776 2020-08-07T15:07:25.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:25.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.453798+0000", 2020-08-07T15:07:25.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001075456 2020-08-07T15:07:25.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:25.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.454873+0000", 2020-08-07T15:07:25.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7999999999999995e-07 2020-08-07T15:07:25.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:25.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.454874+0000", 2020-08-07T15:07:25.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3912000000000002e-05 2020-08-07T15:07:25.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:25.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.454928+0000", 2020-08-07T15:07:25.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058537550000000004 2020-08-07T15:07:25.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:25.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.460781+0000", 2020-08-07T15:07:25.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25630591899999999 2020-08-07T15:07:25.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:25.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.717087+0000", 2020-08-07T15:07:25.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.54e-07 2020-08-07T15:07:25.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:25.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.717088+0000", 2020-08-07T15:07:25.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3594999999999998e-05 2020-08-07T15:07:25.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:25.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.717132+0000", 2020-08-07T15:07:25.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26702573699999999 2020-08-07T15:07:25.170 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:25.181 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:25.181 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:25.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.182 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:49800 3.e 3:7cb4a069:::benchmark_data_smithi074_35447_object6224:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e36)", 2020-08-07T15:07:25.182 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:10.076186+0000", 2020-08-07T15:07:25.183 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.555480809000001, 2020-08-07T15:07:25.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25479637100000002, 2020-08-07T15:07:25.183 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:25.183 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:25.183 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:25.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:25.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:25.184 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 49800 2020-08-07T15:07:25.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.184 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:25.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:25.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076186+0000", 2020-08-07T15:07:25.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:25.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:25.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076186+0000", 2020-08-07T15:07:25.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3579000000000003e-05 2020-08-07T15:07:25.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:25.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076239+0000", 2020-08-07T15:07:25.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0499999999999997e-06 2020-08-07T15:07:25.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:25.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076245+0000", 2020-08-07T15:07:25.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1756999999999999e-05 2020-08-07T15:07:25.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:25.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076257+0000", 2020-08-07T15:07:25.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8220000000000003e-06 2020-08-07T15:07:25.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:25.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076263+0000", 2020-08-07T15:07:25.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9265999999999999e-05 2020-08-07T15:07:25.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:25.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076312+0000", 2020-08-07T15:07:25.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2280000000000004e-05 2020-08-07T15:07:25.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:25.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076404+0000", 2020-08-07T15:07:25.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015721499999999999 2020-08-07T15:07:25.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:25.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076562+0000", 2020-08-07T15:07:25.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041832179999999998 2020-08-07T15:07:25.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:25.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.080745+0000", 2020-08-07T15:07:25.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.091e-06 2020-08-07T15:07:25.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:25.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.080746+0000", 2020-08-07T15:07:25.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3623999999999999e-05 2020-08-07T15:07:25.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:25.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.080809+0000", 2020-08-07T15:07:25.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11945386299999999 2020-08-07T15:07:25.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:25.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.200263+0000", 2020-08-07T15:07:25.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.130675596 2020-08-07T15:07:25.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:25.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.330939+0000", 2020-08-07T15:07:25.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.508e-06 2020-08-07T15:07:25.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:25.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.330940+0000", 2020-08-07T15:07:25.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1501999999999997e-05 2020-08-07T15:07:25.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:25.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.330982+0000", 2020-08-07T15:07:25.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25479637100000002 2020-08-07T15:07:25.235 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:25.236 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:25.236 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:25.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.236 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:69535 3.e 3:7dba033a:::benchmark_data_smithi074_35447_object8691:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e43)", 2020-08-07T15:07:25.237 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:23.311281+0000", 2020-08-07T15:07:25.237 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.3203857370000001, 2020-08-07T15:07:25.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24762457700000001, 2020-08-07T15:07:25.237 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:25.238 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:25.238 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:25.238 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:25.238 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:25.238 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 69535 2020-08-07T15:07:25.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.252 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:25.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:25.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.311281+0000", 2020-08-07T15:07:25.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:25.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:25.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.311281+0000", 2020-08-07T15:07:25.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1261999999999998e-05 2020-08-07T15:07:25.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:25.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.311332+0000", 2020-08-07T15:07:25.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.198e-06 2020-08-07T15:07:25.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:25.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.311337+0000", 2020-08-07T15:07:25.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0242e-05 2020-08-07T15:07:25.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:25.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.311347+0000", 2020-08-07T15:07:25.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2220000000000004e-06 2020-08-07T15:07:25.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:25.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.311352+0000", 2020-08-07T15:07:25.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044450189999999997 2020-08-07T15:07:25.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:25.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.315797+0000", 2020-08-07T15:07:25.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9657999999999997e-05 2020-08-07T15:07:25.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:25.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.315856+0000", 2020-08-07T15:07:25.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010547700000000001 2020-08-07T15:07:25.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:25.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.315962+0000", 2020-08-07T15:07:25.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050302630000000001 2020-08-07T15:07:25.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:25.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.320992+0000", 2020-08-07T15:07:25.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.105e-06 2020-08-07T15:07:25.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:25.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.320993+0000", 2020-08-07T15:07:25.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9343000000000003e-05 2020-08-07T15:07:25.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:25.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.321062+0000", 2020-08-07T15:07:25.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001560566 2020-08-07T15:07:25.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:25.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.322623+0000", 2020-08-07T15:07:25.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23620960299999999 2020-08-07T15:07:25.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:25.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.558833+0000", 2020-08-07T15:07:25.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5499999999999996e-07 2020-08-07T15:07:25.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:25.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.558834+0000", 2020-08-07T15:07:25.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1663999999999997e-05 2020-08-07T15:07:25.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:25.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.558905+0000", 2020-08-07T15:07:25.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24762457700000001 2020-08-07T15:07:25.328 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:25.328 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:25.329 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:25.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.329 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:69536 3.e 3:7dba033a:::benchmark_data_smithi074_35447_object8691:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e43)", 2020-08-07T15:07:25.330 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:23.311385+0000", 2020-08-07T15:07:25.330 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.320281115, 2020-08-07T15:07:25.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24764810000000001, 2020-08-07T15:07:25.331 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:25.332 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:25.332 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:25.332 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:25.332 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:25.332 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 69536 2020-08-07T15:07:25.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.338 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:25.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:25.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.311385+0000", 2020-08-07T15:07:25.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:25.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:25.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.311385+0000", 2020-08-07T15:07:25.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.761e-05 2020-08-07T15:07:25.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:25.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.311433+0000", 2020-08-07T15:07:25.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0599999999999998e-06 2020-08-07T15:07:25.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:25.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.311438+0000", 2020-08-07T15:07:25.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0456e-05 2020-08-07T15:07:25.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:25.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.311448+0000", 2020-08-07T15:07:25.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2020-08-07T15:07:25.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:25.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.311453+0000", 2020-08-07T15:07:25.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049773370000000001 2020-08-07T15:07:25.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:25.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.316430+0000", 2020-08-07T15:07:25.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000108507 2020-08-07T15:07:25.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:25.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.316538+0000", 2020-08-07T15:07:25.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000166706 2020-08-07T15:07:25.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:25.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.316705+0000", 2020-08-07T15:07:25.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004371425 2020-08-07T15:07:25.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:25.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.321076+0000", 2020-08-07T15:07:25.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.92e-07 2020-08-07T15:07:25.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:25.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.321077+0000", 2020-08-07T15:07:25.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6020999999999999e-05 2020-08-07T15:07:25.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:25.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.321123+0000", 2020-08-07T15:07:25.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017699650000000001 2020-08-07T15:07:25.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:25.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.322893+0000", 2020-08-07T15:07:25.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23607164999999999 2020-08-07T15:07:25.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:25.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.558965+0000", 2020-08-07T15:07:25.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6199999999999997e-07 2020-08-07T15:07:25.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:25.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.558966+0000", 2020-08-07T15:07:25.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7787000000000007e-05 2020-08-07T15:07:25.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:25.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:25.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:25.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.559033+0000", 2020-08-07T15:07:25.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24764810000000001 2020-08-07T15:07:25.376 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:25.376 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:25.376 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:25.377 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:25.377 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:25.377 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:25.389 INFO:teuthology.orchestra.run.smithi074.stderr:set pool 1 pg_num to 11 2020-08-07T15:07:25.391 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 49 16 72617 72601 11.5745 12.2812 0.0113324 0.0107947 2020-08-07T15:07:25.392 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:25.377+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:25.394 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:07:25.559 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:25.479+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:25.559 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:25.560 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:07:25.560 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:07:25.560 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:25.573 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:07:25.588 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:25.587+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:25.690 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:25.687+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:25.729 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:07:25.730 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:07:25.761 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:25.761 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:07:25.761 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:07:25.761 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:07:25.762 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:25.781 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:07:25.822 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:25.821+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:25.897 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:25.894+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:25.978 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:07:25.999 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:25.998+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:26.020 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:07:26.036 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:26.036 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:26.058 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:07:26.067 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:07:26.106 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:07:26.107 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:26.105+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:26.142 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:07:26.180 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:07:26.217 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:26.215+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:26.225 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:07:26.244 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 50 16 74147 74131 11.582 11.9531 0.00739935 0.0107861 2020-08-07T15:07:26.248 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:26.248 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:26.260 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:07:26.260 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:07:26.262 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:07:26.262 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:07:26.262 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:07:26.263 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:07:26.263 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:07:26.263 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:07:26.263 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:07:26.263 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:07:26.316 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:26.315+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:26.421 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:26.420+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:26.428 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:26.428 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:26.438 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:07:26.527 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:26.521+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:26.626 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:26.624+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:26.651 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:26.652 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:26.663 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:07:26.727 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:26.726+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:26.838 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:26.829+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:26.843 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:26.844 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:26.859 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:07:26.932 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:26.931+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:27.024 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:27.025 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:27.036 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:07:27.037 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:27.033+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:27.138 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:27.136+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:27.187 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:07:27.241 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:27.240+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:27.244 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 51 16 75680 75664 11.5897 11.9766 0.0120744 0.0107805 2020-08-07T15:07:27.250 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:27.251 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:27.261 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:07:27.346 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:27.341+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:27.447 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:27.445+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:27.462 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:27.463 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:27.474 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:07:27.551 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:27.547+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:27.641 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:27.642 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:27.649 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:27.648+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:27.652 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:07:27.761 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:27.756+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:27.858 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:27.857+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:27.865 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:27.865 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:27.878 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:07:27.959 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:27.958+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:28.058 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:28.058 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:28.061 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:28.060+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:28.071 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:07:28.162 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:28.161+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:28.247 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 52 16 77264 77248 11.6048 12.375 0.0112798 0.010761 2020-08-07T15:07:28.266 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:28.263+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:28.275 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:07:28.281 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:28.281 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:28.292 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:07:28.367 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:28.366+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:28.468 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:28.466+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:28.474 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:07:28.504 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:07:28.506 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:28.506 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:28.519 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:07:28.537 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:07:28.604 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:28.603+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:28.672 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:28.671+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:28.742 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:28.742 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:28.752 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:07:28.776 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:28.773+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:28.783 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:07:28.818 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:07:28.850 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:07:28.883 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:28.876+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:28.886 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:07:28.921 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:07:28.922 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:07:28.922 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:07:28.923 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:07:28.923 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:07:28.923 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:07:28.923 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:07:28.924 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:07:28.924 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:07:28.968 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:28.969 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:07:28.969 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:07:28.969 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:07:28.969 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:28.977 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:28.976+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:28.983 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:07:29.089 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:29.080+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:29.200 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:29.200 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:07:29.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:07:29.201 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:29.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.202 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:15744 3.d e27/20 3:b5661143:::benchmark_data_smithi074_35447_object1967:head v 27'984, mlcod=27'977)", 2020-08-07T15:07:29.202 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.777088+0000", 2020-08-07T15:07:29.203 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.403790088999997, 2020-08-07T15:07:29.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23236351199999999, 2020-08-07T15:07:29.203 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:29.204 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:29.205 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:29.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:29.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777088+0000", 2020-08-07T15:07:29.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:29.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:29.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777088+0000", 2020-08-07T15:07:29.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9495e-05 2020-08-07T15:07:29.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:29.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777137+0000", 2020-08-07T15:07:29.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3049999999999998e-06 2020-08-07T15:07:29.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:29.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777143+0000", 2020-08-07T15:07:29.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23074995500000001 2020-08-07T15:07:29.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:29.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.007893+0000", 2020-08-07T15:07:29.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4390000000000003e-06 2020-08-07T15:07:29.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:29.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.007901+0000", 2020-08-07T15:07:29.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0071000000000003e-05 2020-08-07T15:07:29.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:29.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.007941+0000", 2020-08-07T15:07:29.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2722e-05 2020-08-07T15:07:29.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:29.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.007964+0000", 2020-08-07T15:07:29.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014374870000000001 2020-08-07T15:07:29.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:29.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.009401+0000", 2020-08-07T15:07:29.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0037999999999998e-05 2020-08-07T15:07:29.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:29.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.009451+0000", 2020-08-07T15:07:29.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23236351199999999 2020-08-07T15:07:29.230 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.230 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:29.230 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.231 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:33488 3.c 3:3d1929c7:::benchmark_data_smithi074_35447_object4185:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-08-07T15:07:29.231 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:58.021233+0000", 2020-08-07T15:07:29.231 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.159644819, 2020-08-07T15:07:29.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.2237055, 2020-08-07T15:07:29.232 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:29.232 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:29.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:29.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:29.233 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:29.233 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 33488 2020-08-07T15:07:29.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.233 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:29.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:29.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.021233+0000", 2020-08-07T15:07:29.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:29.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.021233+0000", 2020-08-07T15:07:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7682000000000002e-05 2020-08-07T15:07:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.021281+0000", 2020-08-07T15:07:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.955e-06 2020-08-07T15:07:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.021285+0000", 2020-08-07T15:07:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2132e-05 2020-08-07T15:07:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.021298+0000", 2020-08-07T15:07:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1390000000000003e-06 2020-08-07T15:07:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.021303+0000", 2020-08-07T15:07:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011846789999999999 2020-08-07T15:07:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.022487+0000", 2020-08-07T15:07:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3844000000000002e-05 2020-08-07T15:07:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.022541+0000", 2020-08-07T15:07:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013274700000000001 2020-08-07T15:07:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:07:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.022674+0000", 2020-08-07T15:07:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00161185 2020-08-07T15:07:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.024286+0000", 2020-08-07T15:07:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8800000000000003e-07 2020-08-07T15:07:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.024286+0000", 2020-08-07T15:07:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8810999999999997e-05 2020-08-07T15:07:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.024335+0000", 2020-08-07T15:07:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025354919999999999 2020-08-07T15:07:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.026871+0000", 2020-08-07T15:07:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21800207099999999 2020-08-07T15:07:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.244873+0000", 2020-08-07T15:07:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2279999999999999e-06 2020-08-07T15:07:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.244874+0000", 2020-08-07T15:07:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4381999999999994e-05 2020-08-07T15:07:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.244938+0000", 2020-08-07T15:07:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.2237055 2020-08-07T15:07:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:41432 3.d e32/20 3:b5c8a160:::benchmark_data_smithi074_35447_object5178:head v 32'2848, mlcod=32'2838)", 2020-08-07T15:07:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.794575+0000", 2020-08-07T15:07:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.386302687000001, 2020-08-07T15:07:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36243746199999999, 2020-08-07T15:07:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794575+0000", 2020-08-07T15:07:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:29.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794575+0000", 2020-08-07T15:07:29.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0696999999999999e-05 2020-08-07T15:07:29.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:29.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794626+0000", 2020-08-07T15:07:29.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4189999999999999e-06 2020-08-07T15:07:29.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:29.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794632+0000", 2020-08-07T15:07:29.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36145730399999998 2020-08-07T15:07:29.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:29.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.156089+0000", 2020-08-07T15:07:29.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2329999999999997e-06 2020-08-07T15:07:29.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:29.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.156095+0000", 2020-08-07T15:07:29.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0054999999999999e-05 2020-08-07T15:07:29.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:29.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.156125+0000", 2020-08-07T15:07:29.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5369000000000001e-05 2020-08-07T15:07:29.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:29.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.156140+0000", 2020-08-07T15:07:29.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00076810300000000001 2020-08-07T15:07:29.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:29.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.156908+0000", 2020-08-07T15:07:29.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104282 2020-08-07T15:07:29.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:29.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.157012+0000", 2020-08-07T15:07:29.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36243746199999999 2020-08-07T15:07:29.329 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.329 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:29.329 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.331 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:54216 3.d 3:b7bdea69:::benchmark_data_smithi074_35447_object6776:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e37)", 2020-08-07T15:07:29.331 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:13.107384+0000", 2020-08-07T15:07:29.331 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.073493569, 2020-08-07T15:07:29.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19652251200000001, 2020-08-07T15:07:29.332 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:29.333 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:29.334 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:29.334 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:29.334 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:29.335 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 54216 2020-08-07T15:07:29.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.336 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:29.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:29.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.107384+0000", 2020-08-07T15:07:29.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:29.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:29.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.107384+0000", 2020-08-07T15:07:29.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6458999999999999e-05 2020-08-07T15:07:29.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:29.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.107411+0000", 2020-08-07T15:07:29.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6940000000000001e-06 2020-08-07T15:07:29.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:29.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.107414+0000", 2020-08-07T15:07:29.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8300000000000002e-06 2020-08-07T15:07:29.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:29.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.107423+0000", 2020-08-07T15:07:29.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4789999999999998e-06 2020-08-07T15:07:29.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:29.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.107428+0000", 2020-08-07T15:07:29.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0069090669999999996 2020-08-07T15:07:29.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:29.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.114337+0000", 2020-08-07T15:07:29.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8757999999999994e-05 2020-08-07T15:07:29.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:29.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.114415+0000", 2020-08-07T15:07:29.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121562 2020-08-07T15:07:29.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,4,6,7", 2020-08-07T15:07:29.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.114537+0000", 2020-08-07T15:07:29.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031680240000000002 2020-08-07T15:07:29.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:29.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.117705+0000", 2020-08-07T15:07:29.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00041821300000000002 2020-08-07T15:07:29.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.118123+0000", 2020-08-07T15:07:29.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011701118 2020-08-07T15:07:29.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.129824+0000", 2020-08-07T15:07:29.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4e-07 2020-08-07T15:07:29.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:29.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.129825+0000", 2020-08-07T15:07:29.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0556999999999995e-05 2020-08-07T15:07:29.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:29.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.129906+0000", 2020-08-07T15:07:29.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016020874000000001 2020-08-07T15:07:29.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.145927+0000", 2020-08-07T15:07:29.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.157813175 2020-08-07T15:07:29.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.303740+0000", 2020-08-07T15:07:29.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3799999999999996e-07 2020-08-07T15:07:29.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:29.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.303741+0000", 2020-08-07T15:07:29.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016602399999999999 2020-08-07T15:07:29.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:29.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.303907+0000", 2020-08-07T15:07:29.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19652251200000001 2020-08-07T15:07:29.377 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.377 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:29.377 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.378 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:56999 3.d 3:b21fbb58:::benchmark_data_smithi074_35447_object7124:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e38)", 2020-08-07T15:07:29.378 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:14.851515+0000", 2020-08-07T15:07:29.378 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.329362743000001, 2020-08-07T15:07:29.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19836387699999999, 2020-08-07T15:07:29.378 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:29.379 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:29.379 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:29.379 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:29.379 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:29.379 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 56999 2020-08-07T15:07:29.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.380 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:29.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:29.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.851515+0000", 2020-08-07T15:07:29.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:29.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:29.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.851515+0000", 2020-08-07T15:07:29.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0532999999999999e-05 2020-08-07T15:07:29.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:29.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.851565+0000", 2020-08-07T15:07:29.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5230000000000001e-06 2020-08-07T15:07:29.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:29.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.851572+0000", 2020-08-07T15:07:29.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3008e-05 2020-08-07T15:07:29.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:29.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.851585+0000", 2020-08-07T15:07:29.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9830000000000001e-06 2020-08-07T15:07:29.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:29.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.851591+0000", 2020-08-07T15:07:29.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00186714 2020-08-07T15:07:29.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:29.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.853458+0000", 2020-08-07T15:07:29.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000114256 2020-08-07T15:07:29.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:29.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.853572+0000", 2020-08-07T15:07:29.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021445700000000001 2020-08-07T15:07:29.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,4,6,7", 2020-08-07T15:07:29.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.853787+0000", 2020-08-07T15:07:29.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030110039999999999 2020-08-07T15:07:29.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:29.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.856798+0000", 2020-08-07T15:07:29.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5496999999999998e-05 2020-08-07T15:07:29.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.856883+0000", 2020-08-07T15:07:29.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014580280000000001 2020-08-07T15:07:29.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.858341+0000", 2020-08-07T15:07:29.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1799999999999995e-07 2020-08-07T15:07:29.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:29.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.858342+0000", 2020-08-07T15:07:29.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3446e-05 2020-08-07T15:07:29.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:29.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.858425+0000", 2020-08-07T15:07:29.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00090571300000000005 2020-08-07T15:07:29.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.859331+0000", 2020-08-07T15:07:29.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19041388300000001 2020-08-07T15:07:29.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:15.049745+0000", 2020-08-07T15:07:29.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3200000000000004e-07 2020-08-07T15:07:29.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:29.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:15.049746+0000", 2020-08-07T15:07:29.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000133156 2020-08-07T15:07:29.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:29.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:15.049879+0000", 2020-08-07T15:07:29.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19836387699999999 2020-08-07T15:07:29.401 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.401 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:29.401 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.403 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:57000 3.d 3:b21fbb58:::benchmark_data_smithi074_35447_object7124:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e38)", 2020-08-07T15:07:29.403 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:14.851626+0000", 2020-08-07T15:07:29.403 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.329251624999999, 2020-08-07T15:07:29.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20018433099999999, 2020-08-07T15:07:29.404 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:29.404 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:29.404 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:29.404 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:29.405 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:29.405 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57000 2020-08-07T15:07:29.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.405 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:29.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:29.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.851626+0000", 2020-08-07T15:07:29.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:29.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:29.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.851626+0000", 2020-08-07T15:07:29.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1008999999999999e-05 2020-08-07T15:07:29.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:29.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.851677+0000", 2020-08-07T15:07:29.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4629999999999998e-06 2020-08-07T15:07:29.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:29.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.851684+0000", 2020-08-07T15:07:29.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0695e-05 2020-08-07T15:07:29.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:29.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.851694+0000", 2020-08-07T15:07:29.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1900000000000003e-06 2020-08-07T15:07:29.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:29.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.851699+0000", 2020-08-07T15:07:29.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041108589999999997 2020-08-07T15:07:29.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:29.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.855810+0000", 2020-08-07T15:07:29.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112015 2020-08-07T15:07:29.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:29.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.855922+0000", 2020-08-07T15:07:29.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023106600000000001 2020-08-07T15:07:29.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,4,6,7", 2020-08-07T15:07:29.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.856153+0000", 2020-08-07T15:07:29.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002612156 2020-08-07T15:07:29.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.858765+0000", 2020-08-07T15:07:29.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4025999999999999e-05 2020-08-07T15:07:29.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:29.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.858780+0000", 2020-08-07T15:07:29.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00059822300000000005 2020-08-07T15:07:29.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.859378+0000", 2020-08-07T15:07:29.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.15e-07 2020-08-07T15:07:29.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:29.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.859378+0000", 2020-08-07T15:07:29.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053999999999994e-05 2020-08-07T15:07:29.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:29.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.859442+0000", 2020-08-07T15:07:29.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19220594799999999 2020-08-07T15:07:29.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:15.051648+0000", 2020-08-07T15:07:29.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4619e-05 2020-08-07T15:07:29.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:15.051703+0000", 2020-08-07T15:07:29.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1299999999999999e-07 2020-08-07T15:07:29.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:29.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:15.051703+0000", 2020-08-07T15:07:29.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010688000000000001 2020-08-07T15:07:29.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:29.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:15.051810+0000", 2020-08-07T15:07:29.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20018433099999999 2020-08-07T15:07:29.433 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.444 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:29.444 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.445 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:56999 3.d e38/35 ondisk, result = 0)", 2020-08-07T15:07:29.445 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:14.861229+0000", 2020-08-07T15:07:29.445 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.319648807, 2020-08-07T15:07:29.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.188654444, 2020-08-07T15:07:29.446 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:29.446 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:29.446 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:29.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:29.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.861229+0000", 2020-08-07T15:07:29.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:29.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:29.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.861229+0000", 2020-08-07T15:07:29.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8736999999999999e-05 2020-08-07T15:07:29.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:29.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.861258+0000", 2020-08-07T15:07:29.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5790000000000003e-06 2020-08-07T15:07:29.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:29.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:14.861263+0000", 2020-08-07T15:07:29.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.187934189 2020-08-07T15:07:29.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:29.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:15.049197+0000", 2020-08-07T15:07:29.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6349999999999999e-06 2020-08-07T15:07:29.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:29.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:15.049201+0000", 2020-08-07T15:07:29.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00053452399999999998 2020-08-07T15:07:29.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:29.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:15.049736+0000", 2020-08-07T15:07:29.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5410000000000002e-06 2020-08-07T15:07:29.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:29.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:15.049740+0000", 2020-08-07T15:07:29.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000143239 2020-08-07T15:07:29.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:29.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:15.049883+0000", 2020-08-07T15:07:29.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.188654444 2020-08-07T15:07:29.480 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.480 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:29.480 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.481 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:62248 3.d 3:bc63bb1c:::benchmark_data_smithi074_35447_object7780:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e41)", 2020-08-07T15:07:29.481 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:18.423100+0000", 2020-08-07T15:07:29.481 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.757777964000001, 2020-08-07T15:07:29.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.236870208, 2020-08-07T15:07:29.481 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:29.498 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:29.498 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:29.499 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:29.499 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:29.499 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 62248 2020-08-07T15:07:29.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.499 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:29.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:29.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.423100+0000", 2020-08-07T15:07:29.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:29.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:29.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.423100+0000", 2020-08-07T15:07:29.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0126000000000001e-05 2020-08-07T15:07:29.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:29.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.423150+0000", 2020-08-07T15:07:29.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8900000000000004e-06 2020-08-07T15:07:29.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:29.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.423156+0000", 2020-08-07T15:07:29.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1775e-05 2020-08-07T15:07:29.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:29.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.423167+0000", 2020-08-07T15:07:29.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0509999999999999e-06 2020-08-07T15:07:29.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:29.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.423175+0000", 2020-08-07T15:07:29.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0047868310000000001 2020-08-07T15:07:29.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:29.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.427961+0000", 2020-08-07T15:07:29.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012365499999999999 2020-08-07T15:07:29.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:29.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.428085+0000", 2020-08-07T15:07:29.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022334000000000001 2020-08-07T15:07:29.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,4,6,7", 2020-08-07T15:07:29.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.428308+0000", 2020-08-07T15:07:29.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011845499999999999 2020-08-07T15:07:29.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:29.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.429493+0000", 2020-08-07T15:07:29.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.020861682999999999 2020-08-07T15:07:29.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.450355+0000", 2020-08-07T15:07:29.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13080476599999999 2020-08-07T15:07:29.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.581159+0000", 2020-08-07T15:07:29.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.319e-06 2020-08-07T15:07:29.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:29.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.581161+0000", 2020-08-07T15:07:29.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4919000000000005e-05 2020-08-07T15:07:29.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:29.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.581246+0000", 2020-08-07T15:07:29.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.076846029999999996 2020-08-07T15:07:29.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.658092+0000", 2020-08-07T15:07:29.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017504 2020-08-07T15:07:29.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.659842+0000", 2020-08-07T15:07:29.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9500000000000003e-07 2020-08-07T15:07:29.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:29.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.659843+0000", 2020-08-07T15:07:29.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012737800000000001 2020-08-07T15:07:29.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:29.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.659970+0000", 2020-08-07T15:07:29.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.236870208 2020-08-07T15:07:29.528 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.528 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:29.528 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.529 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:62248 3.d e41/35 ondisk, result = 0)", 2020-08-07T15:07:29.529 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:18.430063+0000", 2020-08-07T15:07:29.529 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.750814956999999, 2020-08-07T15:07:29.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22803188599999999, 2020-08-07T15:07:29.529 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:29.530 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:29.530 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:29.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:29.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.430063+0000", 2020-08-07T15:07:29.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:29.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:29.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.430063+0000", 2020-08-07T15:07:29.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7802000000000002e-05 2020-08-07T15:07:29.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:29.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.430101+0000", 2020-08-07T15:07:29.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2480000000000001e-06 2020-08-07T15:07:29.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:29.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.430107+0000", 2020-08-07T15:07:29.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.226864811 2020-08-07T15:07:29.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:29.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.656972+0000", 2020-08-07T15:07:29.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6069999999999999e-06 2020-08-07T15:07:29.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:29.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.656975+0000", 2020-08-07T15:07:29.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001086587 2020-08-07T15:07:29.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:29.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.658062+0000", 2020-08-07T15:07:29.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7053000000000001e-05 2020-08-07T15:07:29.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:29.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.658079+0000", 2020-08-07T15:07:29.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5778e-05 2020-08-07T15:07:29.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:29.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.658095+0000", 2020-08-07T15:07:29.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22803188599999999 2020-08-07T15:07:29.539 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.539 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:29.539 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.540 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:62248 3.d e41/35 ondisk, result = 0)", 2020-08-07T15:07:29.540 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:18.430203+0000", 2020-08-07T15:07:29.540 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.750674352000001, 2020-08-07T15:07:29.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.229772009, 2020-08-07T15:07:29.540 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:29.541 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:29.541 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:29.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:29.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.430203+0000", 2020-08-07T15:07:29.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:29.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:29.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.430203+0000", 2020-08-07T15:07:29.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9278e-05 2020-08-07T15:07:29.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:29.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.430233+0000", 2020-08-07T15:07:29.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1120000000000004e-06 2020-08-07T15:07:29.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:29.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.430238+0000", 2020-08-07T15:07:29.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22955323399999999 2020-08-07T15:07:29.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:29.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.659791+0000", 2020-08-07T15:07:29.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.891e-06 2020-08-07T15:07:29.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:29.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.659797+0000", 2020-08-07T15:07:29.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3538000000000003e-05 2020-08-07T15:07:29.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:29.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.659830+0000", 2020-08-07T15:07:29.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3770000000000001e-06 2020-08-07T15:07:29.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:29.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.659837+0000", 2020-08-07T15:07:29.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000138579 2020-08-07T15:07:29.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:29.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.659975+0000", 2020-08-07T15:07:29.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.229772009 2020-08-07T15:07:29.550 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.550 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:29.550 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.551 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:64544 3.5 3:a49c5714:::benchmark_data_smithi074_35447_object8067:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e41)", 2020-08-07T15:07:29.551 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:20.050676+0000", 2020-08-07T15:07:29.551 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.1302013219999996, 2020-08-07T15:07:29.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30069369800000001, 2020-08-07T15:07:29.552 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:29.552 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:29.552 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:29.552 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:29.552 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:29.553 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 64544 2020-08-07T15:07:29.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.553 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:29.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:29.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.050676+0000", 2020-08-07T15:07:29.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:29.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:29.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.050676+0000", 2020-08-07T15:07:29.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7633000000000003e-05 2020-08-07T15:07:29.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:29.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.050724+0000", 2020-08-07T15:07:29.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.967e-06 2020-08-07T15:07:29.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:29.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.050729+0000", 2020-08-07T15:07:29.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1659999999999997e-06 2020-08-07T15:07:29.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:29.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.050736+0000", 2020-08-07T15:07:29.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6909999999999999e-06 2020-08-07T15:07:29.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:29.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.050740+0000", 2020-08-07T15:07:29.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052275669999999998 2020-08-07T15:07:29.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:29.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.055967+0000", 2020-08-07T15:07:29.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5745000000000002e-05 2020-08-07T15:07:29.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:29.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.056023+0000", 2020-08-07T15:07:29.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012673099999999999 2020-08-07T15:07:29.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:07:29.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.056150+0000", 2020-08-07T15:07:29.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016914740000000001 2020-08-07T15:07:29.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:29.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.057841+0000", 2020-08-07T15:07:29.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7100000000000001e-07 2020-08-07T15:07:29.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:29.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.057842+0000", 2020-08-07T15:07:29.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3665999999999998e-05 2020-08-07T15:07:29.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:29.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.057886+0000", 2020-08-07T15:07:29.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018919662 2020-08-07T15:07:29.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.076805+0000", 2020-08-07T15:07:29.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27452132699999998 2020-08-07T15:07:29.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.351327+0000", 2020-08-07T15:07:29.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0219999999999999e-06 2020-08-07T15:07:29.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:29.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.351328+0000", 2020-08-07T15:07:29.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2276e-05 2020-08-07T15:07:29.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:29.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.351370+0000", 2020-08-07T15:07:29.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30069369800000001 2020-08-07T15:07:29.570 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.570 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:29.570 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.571 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:64669 3.c 3:3eb1e7f5:::benchmark_data_smithi074_35447_object8083:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e41)", 2020-08-07T15:07:29.571 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:20.131378+0000", 2020-08-07T15:07:29.571 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.0494992879999998, 2020-08-07T15:07:29.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.187653446, 2020-08-07T15:07:29.572 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:29.572 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:29.572 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:29.572 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:29.573 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:29.573 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 64669 2020-08-07T15:07:29.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.573 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:29.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:29.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.131378+0000", 2020-08-07T15:07:29.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:29.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:29.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.131378+0000", 2020-08-07T15:07:29.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5621000000000001e-05 2020-08-07T15:07:29.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:29.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.131404+0000", 2020-08-07T15:07:29.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5560000000000002e-06 2020-08-07T15:07:29.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:29.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.131407+0000", 2020-08-07T15:07:29.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3310000000000003e-06 2020-08-07T15:07:29.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:29.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.131412+0000", 2020-08-07T15:07:29.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.165e-06 2020-08-07T15:07:29.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:29.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.131414+0000", 2020-08-07T15:07:29.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001679982 2020-08-07T15:07:29.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:29.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.133094+0000", 2020-08-07T15:07:29.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106226 2020-08-07T15:07:29.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:29.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.133200+0000", 2020-08-07T15:07:29.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000182027 2020-08-07T15:07:29.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:07:29.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.133382+0000", 2020-08-07T15:07:29.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044431330000000002 2020-08-07T15:07:29.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:29.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.137825+0000", 2020-08-07T15:07:29.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.04e-07 2020-08-07T15:07:29.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:29.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.137826+0000", 2020-08-07T15:07:29.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7651999999999999e-05 2020-08-07T15:07:29.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:29.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.137853+0000", 2020-08-07T15:07:29.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00058934600000000003 2020-08-07T15:07:29.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.138443+0000", 2020-08-07T15:07:29.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18053654499999999 2020-08-07T15:07:29.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.318979+0000", 2020-08-07T15:07:29.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1100000000000004e-07 2020-08-07T15:07:29.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:29.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.318980+0000", 2020-08-07T15:07:29.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1746999999999998e-05 2020-08-07T15:07:29.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:29.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.319032+0000", 2020-08-07T15:07:29.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.187653446 2020-08-07T15:07:29.608 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.609 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:29.609 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.609 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:64670 3.c 3:3eb1e7f5:::benchmark_data_smithi074_35447_object8083:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e41)", 2020-08-07T15:07:29.610 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:20.131957+0000", 2020-08-07T15:07:29.610 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.0489203269999994, 2020-08-07T15:07:29.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18793116900000001, 2020-08-07T15:07:29.610 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:29.611 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:29.611 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:29.611 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:29.611 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:29.611 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 64670 2020-08-07T15:07:29.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.612 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:29.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:29.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.131957+0000", 2020-08-07T15:07:29.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:29.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:29.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.131957+0000", 2020-08-07T15:07:29.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4656e-05 2020-08-07T15:07:29.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:29.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.132012+0000", 2020-08-07T15:07:29.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0870000000000003e-06 2020-08-07T15:07:29.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:29.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.132018+0000", 2020-08-07T15:07:29.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4285e-05 2020-08-07T15:07:29.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:29.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.132032+0000", 2020-08-07T15:07:29.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9079999999999998e-06 2020-08-07T15:07:29.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:29.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.132039+0000", 2020-08-07T15:07:29.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017141630000000001 2020-08-07T15:07:29.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:29.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.133753+0000", 2020-08-07T15:07:29.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010596500000000001 2020-08-07T15:07:29.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:29.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.133859+0000", 2020-08-07T15:07:29.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016834599999999999 2020-08-07T15:07:29.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:07:29.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.134028+0000", 2020-08-07T15:07:29.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038290139999999999 2020-08-07T15:07:29.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:29.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.137857+0000", 2020-08-07T15:07:29.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.92e-07 2020-08-07T15:07:29.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:29.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.137857+0000", 2020-08-07T15:07:29.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5840999999999999e-05 2020-08-07T15:07:29.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:29.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.137883+0000", 2020-08-07T15:07:29.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00058536700000000005 2020-08-07T15:07:29.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.138468+0000", 2020-08-07T15:07:29.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18138337400000001 2020-08-07T15:07:29.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.319852+0000", 2020-08-07T15:07:29.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3900000000000005e-07 2020-08-07T15:07:29.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:29.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.319852+0000", 2020-08-07T15:07:29.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6431999999999999e-05 2020-08-07T15:07:29.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:29.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.319889+0000", 2020-08-07T15:07:29.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18793116900000001 2020-08-07T15:07:29.633 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.633 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:29.633 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.634 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:64671 3.c 3:3eb1e7f5:::benchmark_data_smithi074_35447_object8083:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e41)", 2020-08-07T15:07:29.634 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:20.132813+0000", 2020-08-07T15:07:29.634 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.0480644869999995, 2020-08-07T15:07:29.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18768011800000001, 2020-08-07T15:07:29.634 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:29.634 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:29.635 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:29.635 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:29.635 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:29.635 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 64671 2020-08-07T15:07:29.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.636 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:29.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:29.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.132813+0000", 2020-08-07T15:07:29.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:29.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:29.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.132813+0000", 2020-08-07T15:07:29.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8739999999999999e-05 2020-08-07T15:07:29.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:29.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.132842+0000", 2020-08-07T15:07:29.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8349999999999997e-06 2020-08-07T15:07:29.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:29.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.132846+0000", 2020-08-07T15:07:29.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1182999999999999e-05 2020-08-07T15:07:29.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:29.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.132857+0000", 2020-08-07T15:07:29.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.677e-06 2020-08-07T15:07:29.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:29.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.132863+0000", 2020-08-07T15:07:29.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015546640000000001 2020-08-07T15:07:29.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:29.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.134417+0000", 2020-08-07T15:07:29.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2009999999999998e-05 2020-08-07T15:07:29.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:29.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.134479+0000", 2020-08-07T15:07:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4976999999999995e-05 2020-08-07T15:07:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:07:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.134574+0000", 2020-08-07T15:07:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003311863 2020-08-07T15:07:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.137886+0000", 2020-08-07T15:07:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.05e-07 2020-08-07T15:07:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.137886+0000", 2020-08-07T15:07:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5508999999999999e-05 2020-08-07T15:07:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.137912+0000", 2020-08-07T15:07:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00079057800000000001 2020-08-07T15:07:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.138702+0000", 2020-08-07T15:07:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.181741875 2020-08-07T15:07:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.320444+0000", 2020-08-07T15:07:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9400000000000005e-07 2020-08-07T15:07:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.320445+0000", 2020-08-07T15:07:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8307999999999999e-05 2020-08-07T15:07:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.320493+0000", 2020-08-07T15:07:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18768011800000001 2020-08-07T15:07:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:64672 3.c 3:3eb1e7f5:::benchmark_data_smithi074_35447_object8083:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e41)", 2020-08-07T15:07:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:20.135363+0000", 2020-08-07T15:07:29.654 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.045514786, 2020-08-07T15:07:29.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20645291299999999, 2020-08-07T15:07:29.654 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:29.654 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:29.654 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 64672 2020-08-07T15:07:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.135363+0000", 2020-08-07T15:07:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.135363+0000", 2020-08-07T15:07:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7699e-05 2020-08-07T15:07:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.135391+0000", 2020-08-07T15:07:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5379999999999998e-06 2020-08-07T15:07:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.135394+0000", 2020-08-07T15:07:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5040000000000002e-06 2020-08-07T15:07:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.135403+0000", 2020-08-07T15:07:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3459999999999998e-06 2020-08-07T15:07:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:29.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.135406+0000", 2020-08-07T15:07:29.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7472000000000001e-05 2020-08-07T15:07:29.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:29.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.135433+0000", 2020-08-07T15:07:29.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3967999999999995e-05 2020-08-07T15:07:29.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:29.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.135497+0000", 2020-08-07T15:07:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105528 2020-08-07T15:07:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:07:29.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.135603+0000", 2020-08-07T15:07:29.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0047311139999999998 2020-08-07T15:07:29.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:29.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.140334+0000", 2020-08-07T15:07:29.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3799999999999996e-07 2020-08-07T15:07:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.140335+0000", 2020-08-07T15:07:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8843000000000001e-05 2020-08-07T15:07:29.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:29.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.140374+0000", 2020-08-07T15:07:29.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18015176699999999 2020-08-07T15:07:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.320525+0000", 2020-08-07T15:07:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021207816000000001 2020-08-07T15:07:29.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.341733+0000", 2020-08-07T15:07:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5999999999999999e-06 2020-08-07T15:07:29.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:29.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.341735+0000", 2020-08-07T15:07:29.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0980000000000001e-05 2020-08-07T15:07:29.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.341816+0000", 2020-08-07T15:07:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20645291299999999 2020-08-07T15:07:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:64672 3.c e41/20 ondisk, result = 0)", 2020-08-07T15:07:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:20.138617+0000", 2020-08-07T15:07:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.0422603699999993, 2020-08-07T15:07:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20320521899999999, 2020-08-07T15:07:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:29.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:29.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.138617+0000", 2020-08-07T15:07:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.138617+0000", 2020-08-07T15:07:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7751e-05 2020-08-07T15:07:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.138635+0000", 2020-08-07T15:07:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6820000000000001e-06 2020-08-07T15:07:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.138639+0000", 2020-08-07T15:07:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20300644700000001 2020-08-07T15:07:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.341645+0000", 2020-08-07T15:07:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6890000000000004e-06 2020-08-07T15:07:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.341653+0000", 2020-08-07T15:07:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7142999999999998e-05 2020-08-07T15:07:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.341710+0000", 2020-08-07T15:07:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.221e-05 2020-08-07T15:07:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.341722+0000", 2020-08-07T15:07:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010029700000000001 2020-08-07T15:07:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.341823+0000", 2020-08-07T15:07:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20320521899999999 2020-08-07T15:07:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:64544 3.5 e41/20 ondisk, result = 0)", 2020-08-07T15:07:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:20.151417+0000", 2020-08-07T15:07:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.0294607809999992, 2020-08-07T15:07:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.199960365, 2020-08-07T15:07:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.151417+0000", 2020-08-07T15:07:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.151417+0000", 2020-08-07T15:07:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7178999999999999e-05 2020-08-07T15:07:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.151434+0000", 2020-08-07T15:07:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7680000000000002e-06 2020-08-07T15:07:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.151438+0000", 2020-08-07T15:07:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19983706200000001 2020-08-07T15:07:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.351275+0000", 2020-08-07T15:07:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8020000000000002e-06 2020-08-07T15:07:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.351279+0000", 2020-08-07T15:07:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3838999999999999e-05 2020-08-07T15:07:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.351313+0000", 2020-08-07T15:07:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2980000000000003e-06 2020-08-07T15:07:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.351320+0000", 2020-08-07T15:07:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7417000000000003e-05 2020-08-07T15:07:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.351377+0000", 2020-08-07T15:07:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.199960365 2020-08-07T15:07:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:69728 3.4 e43/20 3:2b6ce21b:::benchmark_data_smithi074_35447_object8715:head v 43'4456, mlcod=43'4447)", 2020-08-07T15:07:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:23.424713+0000", 2020-08-07T15:07:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.7561644430000003, 2020-08-07T15:07:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18766691399999999, 2020-08-07T15:07:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.424713+0000", 2020-08-07T15:07:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.424713+0000", 2020-08-07T15:07:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2834000000000003e-05 2020-08-07T15:07:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.424766+0000", 2020-08-07T15:07:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2889999999999997e-06 2020-08-07T15:07:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.424771+0000", 2020-08-07T15:07:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18580354800000001 2020-08-07T15:07:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.610575+0000", 2020-08-07T15:07:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9530000000000001e-06 2020-08-07T15:07:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.610579+0000", 2020-08-07T15:07:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9508999999999997e-05 2020-08-07T15:07:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.610618+0000", 2020-08-07T15:07:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7308000000000001e-05 2020-08-07T15:07:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.610636+0000", 2020-08-07T15:07:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017020539999999999 2020-08-07T15:07:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.612338+0000", 2020-08-07T15:07:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2419000000000002e-05 2020-08-07T15:07:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.612380+0000", 2020-08-07T15:07:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18766691399999999 2020-08-07T15:07:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:70512 3.d 3:bc6e6ca0:::benchmark_data_smithi074_35447_object8813:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e43)", 2020-08-07T15:07:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:23.917242+0000", 2020-08-07T15:07:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.2636358789999997, 2020-08-07T15:07:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20030519199999999, 2020-08-07T15:07:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 70512 2020-08-07T15:07:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.917242+0000", 2020-08-07T15:07:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.917242+0000", 2020-08-07T15:07:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9010000000000001e-05 2020-08-07T15:07:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.917281+0000", 2020-08-07T15:07:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5159999999999996e-06 2020-08-07T15:07:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.917285+0000", 2020-08-07T15:07:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9540000000000004e-06 2020-08-07T15:07:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.917293+0000", 2020-08-07T15:07:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0380000000000001e-06 2020-08-07T15:07:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.917297+0000", 2020-08-07T15:07:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016440770000000001 2020-08-07T15:07:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.918941+0000", 2020-08-07T15:07:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117478 2020-08-07T15:07:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.919059+0000", 2020-08-07T15:07:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000222368 2020-08-07T15:07:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,4,6,7", 2020-08-07T15:07:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.919281+0000", 2020-08-07T15:07:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002471595 2020-08-07T15:07:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.921753+0000", 2020-08-07T15:07:29.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049365900000000002 2020-08-07T15:07:29.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.922246+0000", 2020-08-07T15:07:29.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4924e-05 2020-08-07T15:07:29.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:29.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.922261+0000", 2020-08-07T15:07:29.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5400000000000001e-07 2020-08-07T15:07:29.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:29.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.922262+0000", 2020-08-07T15:07:29.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7467999999999998e-05 2020-08-07T15:07:29.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:29.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.922339+0000", 2020-08-07T15:07:29.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.060382534000000002 2020-08-07T15:07:29.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.982722+0000", 2020-08-07T15:07:29.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13463893299999999 2020-08-07T15:07:29.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.117361+0000", 2020-08-07T15:07:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.85e-07 2020-08-07T15:07:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.117362+0000", 2020-08-07T15:07:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018519899999999999 2020-08-07T15:07:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.117547+0000", 2020-08-07T15:07:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20030519199999999 2020-08-07T15:07:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:70512 3.d e43/35 ondisk, result = 0)", 2020-08-07T15:07:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:23.923092+0000", 2020-08-07T15:07:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.2577855539999998, 2020-08-07T15:07:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19446363999999999, 2020-08-07T15:07:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.923092+0000", 2020-08-07T15:07:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.923092+0000", 2020-08-07T15:07:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9359000000000001e-05 2020-08-07T15:07:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.923121+0000", 2020-08-07T15:07:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.977e-06 2020-08-07T15:07:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.923126+0000", 2020-08-07T15:07:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.193142337 2020-08-07T15:07:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.116269+0000", 2020-08-07T15:07:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1889999999999998e-06 2020-08-07T15:07:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.116274+0000", 2020-08-07T15:07:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001066516 2020-08-07T15:07:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.117340+0000", 2020-08-07T15:07:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0695e-05 2020-08-07T15:07:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.117351+0000", 2020-08-07T15:07:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020456700000000001 2020-08-07T15:07:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.117556+0000", 2020-08-07T15:07:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19446363999999999 2020-08-07T15:07:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:29.748 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:29.749 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:07:29.749 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:07:29.751 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:29.192+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:29.751 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 53 16 78735 78719 11.6027 11.4922 0.00571358 0.0107693 2020-08-07T15:07:29.752 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:29.750+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:29.868 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:29.855+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:29.943 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:29.943 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:29.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.944 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:79825 3.2 e47/20 3:4d722573:::benchmark_data_smithi074_35447_object9978:head v 47'4945, mlcod=47'4936)", 2020-08-07T15:07:29.944 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:29.933369+0000", 2020-08-07T15:07:29.944 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0055081449999999999, 2020-08-07T15:07:29.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055376100000000001, 2020-08-07T15:07:29.944 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:29.945 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:29.945 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:29.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:29.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.933369+0000", 2020-08-07T15:07:29.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:29.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:29.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.933369+0000", 2020-08-07T15:07:29.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8911e-05 2020-08-07T15:07:29.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:29.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.933418+0000", 2020-08-07T15:07:29.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2559999999999999e-06 2020-08-07T15:07:29.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:29.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.933424+0000", 2020-08-07T15:07:29.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2306999999999999e-05 2020-08-07T15:07:29.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:29.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.933476+0000", 2020-08-07T15:07:29.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0329999999999997e-06 2020-08-07T15:07:29.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:29.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.933482+0000", 2020-08-07T15:07:29.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5189999999999998e-05 2020-08-07T15:07:29.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:29.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.933538+0000", 2020-08-07T15:07:29.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1611999999999999e-05 2020-08-07T15:07:29.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:29.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.933559+0000", 2020-08-07T15:07:29.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000190309 2020-08-07T15:07:29.973 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.973 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:29.973 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:29.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.974 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:79826 3.2 e47/20 3:4d722573:::benchmark_data_smithi074_35447_object9978:head v 47'4946, mlcod=47'4936)", 2020-08-07T15:07:29.974 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:29.933825+0000", 2020-08-07T15:07:29.974 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0050516429999999998, 2020-08-07T15:07:29.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051634719999999997, 2020-08-07T15:07:29.975 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:29.975 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:29.975 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:29.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:29.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.933825+0000", 2020-08-07T15:07:29.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:29.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:29.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.933825+0000", 2020-08-07T15:07:29.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0158000000000003e-05 2020-08-07T15:07:29.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:29.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.933876+0000", 2020-08-07T15:07:29.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6690000000000006e-06 2020-08-07T15:07:29.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:29.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.933883+0000", 2020-08-07T15:07:29.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5869999999999999e-05 2020-08-07T15:07:29.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:29.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:29.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.933939+0000", 2020-08-07T15:07:29.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4650000000000002e-06 2020-08-07T15:07:29.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:29.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:30.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.933945+0000", 2020-08-07T15:07:30.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000164191 2020-08-07T15:07:30.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:30.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.934109+0000", 2020-08-07T15:07:30.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0959e-05 2020-08-07T15:07:30.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:30.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.934130+0000", 2020-08-07T15:07:30.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000304312 2020-08-07T15:07:30.008 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.008 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:30.009 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.009 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:79827 3.2 e47/20 3:4d722573:::benchmark_data_smithi074_35447_object9978:head v 47'4947, mlcod=47'4936)", 2020-08-07T15:07:30.009 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:29.934457+0000", 2020-08-07T15:07:30.010 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0044199599999999997, 2020-08-07T15:07:30.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046181269999999996, 2020-08-07T15:07:30.010 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:30.010 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:30.010 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:30.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:30.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.934457+0000", 2020-08-07T15:07:30.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:30.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:30.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.934457+0000", 2020-08-07T15:07:30.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4141000000000002e-05 2020-08-07T15:07:30.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:30.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.934511+0000", 2020-08-07T15:07:30.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3559999999999998e-06 2020-08-07T15:07:30.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:30.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.934518+0000", 2020-08-07T15:07:30.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8480000000000003e-05 2020-08-07T15:07:30.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:30.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.934576+0000", 2020-08-07T15:07:30.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.968e-06 2020-08-07T15:07:30.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:30.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.934583+0000", 2020-08-07T15:07:30.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5555000000000001e-05 2020-08-07T15:07:30.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:30.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.934619+0000", 2020-08-07T15:07:30.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4487e-05 2020-08-07T15:07:30.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:30.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.934633+0000", 2020-08-07T15:07:30.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017598700000000001 2020-08-07T15:07:30.056 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.056 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:30.057 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.057 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:79828 3.2 e47/20 3:4d722573:::benchmark_data_smithi074_35447_object9978:head v 47'4948, mlcod=47'4936)", 2020-08-07T15:07:30.057 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:29.934984+0000", 2020-08-07T15:07:30.068 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0038929189999999999, 2020-08-07T15:07:30.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004162954, 2020-08-07T15:07:30.069 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:30.069 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:30.069 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:30.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:30.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.934984+0000", 2020-08-07T15:07:30.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:30.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.934984+0000", 2020-08-07T15:07:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5303999999999998e-05 2020-08-07T15:07:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:30.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.935039+0000", 2020-08-07T15:07:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0550000000000005e-06 2020-08-07T15:07:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:30.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.935048+0000", 2020-08-07T15:07:30.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2456e-05 2020-08-07T15:07:30.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:30.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.935111+0000", 2020-08-07T15:07:30.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0779999999999998e-06 2020-08-07T15:07:30.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:30.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.935118+0000", 2020-08-07T15:07:30.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8148000000000001e-05 2020-08-07T15:07:30.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:30.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.935156+0000", 2020-08-07T15:07:30.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1736e-05 2020-08-07T15:07:30.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:30.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.935178+0000", 2020-08-07T15:07:30.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019377700000000001 2020-08-07T15:07:30.102 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.102 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:30.103 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.103 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:79829 3.2 e47/20 3:4d722573:::benchmark_data_smithi074_35447_object9978:head v 47'4949, mlcod=47'4936)", 2020-08-07T15:07:30.103 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:29.935690+0000", 2020-08-07T15:07:30.103 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0031872430000000002, 2020-08-07T15:07:30.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003524906, 2020-08-07T15:07:30.104 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:30.104 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:30.104 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:30.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:30.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.935690+0000", 2020-08-07T15:07:30.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:30.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:30.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.935690+0000", 2020-08-07T15:07:30.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9968e-05 2020-08-07T15:07:30.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:30.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.935740+0000", 2020-08-07T15:07:30.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5350000000000001e-06 2020-08-07T15:07:30.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:30.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.935746+0000", 2020-08-07T15:07:30.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1922999999999997e-05 2020-08-07T15:07:30.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:30.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.935798+0000", 2020-08-07T15:07:30.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6180000000000003e-06 2020-08-07T15:07:30.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:30.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.935804+0000", 2020-08-07T15:07:30.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9032000000000002e-05 2020-08-07T15:07:30.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:30.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.935843+0000", 2020-08-07T15:07:30.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0907000000000002e-05 2020-08-07T15:07:30.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:30.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.935864+0000", 2020-08-07T15:07:30.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000173983 2020-08-07T15:07:30.114 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.114 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:30.115 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.115 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:79834 3.3 e47/20 3:ce1368a9:::benchmark_data_smithi074_35447_object9979:head v 47'4674, mlcod=47'4671)", 2020-08-07T15:07:30.115 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:29.936032+0000", 2020-08-07T15:07:30.116 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0028447910000000002, 2020-08-07T15:07:30.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003229886, 2020-08-07T15:07:30.117 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:30.117 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:30.117 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:30.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:30.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936032+0000", 2020-08-07T15:07:30.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:30.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:30.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936032+0000", 2020-08-07T15:07:30.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6384000000000001e-05 2020-08-07T15:07:30.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:30.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936089+0000", 2020-08-07T15:07:30.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2020-08-07T15:07:30.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:30.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936095+0000", 2020-08-07T15:07:30.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028677900000000003 2020-08-07T15:07:30.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:30.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936382+0000", 2020-08-07T15:07:30.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.468e-06 2020-08-07T15:07:30.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:30.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936388+0000", 2020-08-07T15:07:30.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000155852 2020-08-07T15:07:30.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:30.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936543+0000", 2020-08-07T15:07:30.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2390000000000001e-05 2020-08-07T15:07:30.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:30.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936566+0000", 2020-08-07T15:07:30.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000533582 2020-08-07T15:07:30.126 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.126 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:30.126 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.127 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:79835 3.3 e47/20 3:ce1368a9:::benchmark_data_smithi074_35447_object9979:head v 47'4675, mlcod=47'4671)", 2020-08-07T15:07:30.127 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:29.936133+0000", 2020-08-07T15:07:30.128 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0027436040000000002, 2020-08-07T15:07:30.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031764250000000001, 2020-08-07T15:07:30.128 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:30.128 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:30.128 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:30.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:30.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936133+0000", 2020-08-07T15:07:30.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:30.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:30.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936133+0000", 2020-08-07T15:07:30.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7623999999999997e-05 2020-08-07T15:07:30.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:30.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936181+0000", 2020-08-07T15:07:30.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4249999999999997e-06 2020-08-07T15:07:30.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:30.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936186+0000", 2020-08-07T15:07:30.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000215713 2020-08-07T15:07:30.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:30.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936402+0000", 2020-08-07T15:07:30.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1609999999999999e-06 2020-08-07T15:07:30.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:30.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936405+0000", 2020-08-07T15:07:30.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00043639699999999999 2020-08-07T15:07:30.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:30.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936842+0000", 2020-08-07T15:07:30.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0630000000000001e-05 2020-08-07T15:07:30.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:30.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936862+0000", 2020-08-07T15:07:30.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00072895000000000002 2020-08-07T15:07:30.137 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.138 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:30.138 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.139 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:79836 3.3 e47/20 3:ce1368a9:::benchmark_data_smithi074_35447_object9979:head v 47'4676, mlcod=47'4671)", 2020-08-07T15:07:30.139 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:29.936215+0000", 2020-08-07T15:07:30.139 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0026623269999999999, 2020-08-07T15:07:30.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031741909999999998, 2020-08-07T15:07:30.139 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:30.139 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:30.140 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:30.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:30.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936215+0000", 2020-08-07T15:07:30.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:30.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:30.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936215+0000", 2020-08-07T15:07:30.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7135999999999997e-05 2020-08-07T15:07:30.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:30.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936262+0000", 2020-08-07T15:07:30.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3879999999999999e-06 2020-08-07T15:07:30.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:30.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936267+0000", 2020-08-07T15:07:30.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000160271 2020-08-07T15:07:30.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:30.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936427+0000", 2020-08-07T15:07:30.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1970000000000001e-06 2020-08-07T15:07:30.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:30.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936431+0000", 2020-08-07T15:07:30.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00066301800000000003 2020-08-07T15:07:30.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:30.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.937094+0000", 2020-08-07T15:07:30.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2166e-05 2020-08-07T15:07:30.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:30.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.937106+0000", 2020-08-07T15:07:30.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00089117600000000003 2020-08-07T15:07:30.150 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.150 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:30.150 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.151 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:79837 3.3 e47/20 3:ce1368a9:::benchmark_data_smithi074_35447_object9979:head v 47'4677, mlcod=47'4671)", 2020-08-07T15:07:30.152 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:29.936295+0000", 2020-08-07T15:07:30.152 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.002582417, 2020-08-07T15:07:30.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031565909999999998, 2020-08-07T15:07:30.153 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:30.153 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:30.153 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:30.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:30.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936295+0000", 2020-08-07T15:07:30.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:30.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:30.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936295+0000", 2020-08-07T15:07:30.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9984000000000001e-05 2020-08-07T15:07:30.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:30.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936345+0000", 2020-08-07T15:07:30.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3140000000000003e-06 2020-08-07T15:07:30.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:30.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936350+0000", 2020-08-07T15:07:30.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1883000000000001e-05 2020-08-07T15:07:30.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:30.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936442+0000", 2020-08-07T15:07:30.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.954e-06 2020-08-07T15:07:30.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:30.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936445+0000", 2020-08-07T15:07:30.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00082031300000000003 2020-08-07T15:07:30.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:30.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.937265+0000", 2020-08-07T15:07:30.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1776e-05 2020-08-07T15:07:30.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:30.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.937277+0000", 2020-08-07T15:07:30.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00098222400000000007 2020-08-07T15:07:30.192 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.193 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:30.193 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.194 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:79830 3.2 e47/20 3:4d722573:::benchmark_data_smithi074_35447_object9978:head v 47'4950, mlcod=47'4936)", 2020-08-07T15:07:30.194 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:29.936537+0000", 2020-08-07T15:07:30.194 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0023403349999999998, 2020-08-07T15:07:30.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029524149999999999, 2020-08-07T15:07:30.194 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:30.195 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:30.195 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:30.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:30.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936537+0000", 2020-08-07T15:07:30.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:30.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:30.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936537+0000", 2020-08-07T15:07:30.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9848000000000001e-05 2020-08-07T15:07:30.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:30.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936587+0000", 2020-08-07T15:07:30.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1e-06 2020-08-07T15:07:30.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:30.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936593+0000", 2020-08-07T15:07:30.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4390999999999999e-05 2020-08-07T15:07:30.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:30.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936677+0000", 2020-08-07T15:07:30.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3399999999999997e-06 2020-08-07T15:07:30.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:30.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936682+0000", 2020-08-07T15:07:30.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7567000000000001e-05 2020-08-07T15:07:30.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:30.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936720+0000", 2020-08-07T15:07:30.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9888999999999999e-05 2020-08-07T15:07:30.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:30.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936740+0000", 2020-08-07T15:07:30.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020313500000000001 2020-08-07T15:07:30.212 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.212 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:30.212 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.213 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:79831 3.2 e47/20 3:4d722573:::benchmark_data_smithi074_35447_object9978:head v 47'4951, mlcod=47'4936)", 2020-08-07T15:07:30.213 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:29.936822+0000", 2020-08-07T15:07:30.213 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0020545490000000001, 2020-08-07T15:07:30.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002701799, 2020-08-07T15:07:30.214 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:30.214 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:30.214 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:30.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:30.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936822+0000", 2020-08-07T15:07:30.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:30.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:30.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936822+0000", 2020-08-07T15:07:30.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5179000000000002e-05 2020-08-07T15:07:30.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:30.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936868+0000", 2020-08-07T15:07:30.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0750000000000003e-06 2020-08-07T15:07:30.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:30.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936874+0000", 2020-08-07T15:07:30.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2960999999999997e-05 2020-08-07T15:07:30.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:30.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936917+0000", 2020-08-07T15:07:30.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0980000000000001e-06 2020-08-07T15:07:30.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:30.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936922+0000", 2020-08-07T15:07:30.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8133000000000006e-05 2020-08-07T15:07:30.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:30.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.936990+0000", 2020-08-07T15:07:30.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2147999999999999e-05 2020-08-07T15:07:30.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:30.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.937012+0000", 2020-08-07T15:07:30.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018959399999999999 2020-08-07T15:07:30.222 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.223 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:30.223 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.223 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:79832 3.2 e47/20 3:4d722573:::benchmark_data_smithi074_35447_object9978:head v 47'4952, mlcod=47'4936)", 2020-08-07T15:07:30.224 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:29.937252+0000", 2020-08-07T15:07:30.224 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00162492, 2020-08-07T15:07:30.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023104570000000001, 2020-08-07T15:07:30.224 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:30.224 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:30.225 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:30.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:30.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.937252+0000", 2020-08-07T15:07:30.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:30.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:30.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.937252+0000", 2020-08-07T15:07:30.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8174999999999999e-05 2020-08-07T15:07:30.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:30.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.937300+0000", 2020-08-07T15:07:30.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.003e-06 2020-08-07T15:07:30.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:30.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.937306+0000", 2020-08-07T15:07:30.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4579000000000001e-05 2020-08-07T15:07:30.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:30.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.937351+0000", 2020-08-07T15:07:30.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9189999999999998e-06 2020-08-07T15:07:30.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:30.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.937356+0000", 2020-08-07T15:07:30.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5042000000000001e-05 2020-08-07T15:07:30.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:30.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.937391+0000", 2020-08-07T15:07:30.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0716999999999998e-05 2020-08-07T15:07:30.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:30.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.937412+0000", 2020-08-07T15:07:30.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015943500000000001 2020-08-07T15:07:30.236 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.236 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:30.236 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.237 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:79838 3.3 e47/20 3:ce1368a9:::benchmark_data_smithi074_35447_object9979:head v 47'4678, mlcod=47'4671)", 2020-08-07T15:07:30.237 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:29.937625+0000", 2020-08-07T15:07:30.238 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.001251963, 2020-08-07T15:07:30.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020097470000000001, 2020-08-07T15:07:30.238 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:30.238 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:30.238 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:30.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:30.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.937625+0000", 2020-08-07T15:07:30.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:30.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:30.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.937625+0000", 2020-08-07T15:07:30.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0126000000000001e-05 2020-08-07T15:07:30.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:30.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.937675+0000", 2020-08-07T15:07:30.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5239999999999997e-06 2020-08-07T15:07:30.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:30.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.937682+0000", 2020-08-07T15:07:30.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8300999999999999e-05 2020-08-07T15:07:30.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:30.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.937710+0000", 2020-08-07T15:07:30.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9499999999999998e-06 2020-08-07T15:07:30.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:30.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.937716+0000", 2020-08-07T15:07:30.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2039999999999997e-05 2020-08-07T15:07:30.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:30.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.937758+0000", 2020-08-07T15:07:30.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0917e-05 2020-08-07T15:07:30.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:30.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.937779+0000", 2020-08-07T15:07:30.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015385799999999999 2020-08-07T15:07:30.248 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.248 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:30.248 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.249 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:79839 3.3 e47/20 3:ce1368a9:::benchmark_data_smithi074_35447_object9979:head v 47'4679, mlcod=47'4671)", 2020-08-07T15:07:30.249 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:29.938735+0000", 2020-08-07T15:07:30.249 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.000141545, 2020-08-07T15:07:30.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00097159299999999998, 2020-08-07T15:07:30.250 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:30.250 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:30.250 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:30.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:30.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.938735+0000", 2020-08-07T15:07:30.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:30.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:30.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.938735+0000", 2020-08-07T15:07:30.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2491000000000003e-05 2020-08-07T15:07:30.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:30.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.938788+0000", 2020-08-07T15:07:30.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0350000000000004e-06 2020-08-07T15:07:30.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:30.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.938795+0000", 2020-08-07T15:07:30.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7520000000000001e-05 2020-08-07T15:07:30.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:30.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.938822+0000", 2020-08-07T15:07:30.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7150000000000003e-06 2020-08-07T15:07:30.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:30.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.938828+0000", 2020-08-07T15:07:30.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5327999999999998e-05 2020-08-07T15:07:30.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:30.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.938864+0000", 2020-08-07T15:07:30.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0975000000000002e-05 2020-08-07T15:07:30.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:30.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.938885+0000", 2020-08-07T15:07:30.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014906399999999999 2020-08-07T15:07:30.258 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.258 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:30.258 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.258 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:79840 3.3 e47/20)", 2020-08-07T15:07:30.259 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:29.939594+0000", 2020-08-07T15:07:30.259 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4294967295.9992828, 2020-08-07T15:07:30.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000181156, 2020-08-07T15:07:30.259 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:30.259 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:07:30.260 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:30.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:30.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.939594+0000", 2020-08-07T15:07:30.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:30.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:30.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.939594+0000", 2020-08-07T15:07:30.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8626999999999999e-05 2020-08-07T15:07:30.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:30.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.939623+0000", 2020-08-07T15:07:30.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.42e-06 2020-08-07T15:07:30.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:30.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.939627+0000", 2020-08-07T15:07:30.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6175999999999999e-05 2020-08-07T15:07:30.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:30.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.939643+0000", 2020-08-07T15:07:30.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9910000000000004e-06 2020-08-07T15:07:30.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:30.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.939647+0000", 2020-08-07T15:07:30.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3214000000000002e-05 2020-08-07T15:07:30.266 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.267 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:30.267 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.267 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.267 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:07:30.268 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 15 2020-08-07T15:07:30.268 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:30.269 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 54 16 80379 80363 11.6256 12.8438 0.00902427 0.0107479 2020-08-07T15:07:30.270 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:07:30.272 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:30.268+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:30.370 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:30.368+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:30.392 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:07:30.397 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 0 1 2020-08-07T15:07:30.456 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:30.459 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:07:30.459 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:07:30.460 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:07:30.469 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:30.471 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:07:30.472 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:30.470+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:30.574 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:30.573+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:30.698 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:30.699 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:07:30.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:07:30.699 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:30.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.699 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:11192 3.d e24/20 3:bb990748:::benchmark_data_smithi074_35447_object1398:head v 24'680, mlcod=24'671)", 2020-08-07T15:07:30.700 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:43.006852+0000", 2020-08-07T15:07:30.700 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.680162631000002, 2020-08-07T15:07:30.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15926784399999999, 2020-08-07T15:07:30.700 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:30.700 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:30.701 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:30.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:30.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.006852+0000", 2020-08-07T15:07:30.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:30.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:30.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.006852+0000", 2020-08-07T15:07:30.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0237e-05 2020-08-07T15:07:30.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:30.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.006902+0000", 2020-08-07T15:07:30.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2750000000000001e-06 2020-08-07T15:07:30.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:30.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.006908+0000", 2020-08-07T15:07:30.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15839128599999999 2020-08-07T15:07:30.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:30.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.165300+0000", 2020-08-07T15:07:30.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4450000000000004e-06 2020-08-07T15:07:30.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:30.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.165305+0000", 2020-08-07T15:07:30.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3010999999999999e-05 2020-08-07T15:07:30.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:30.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.165348+0000", 2020-08-07T15:07:30.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3289e-05 2020-08-07T15:07:30.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:30.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.165371+0000", 2020-08-07T15:07:30.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00069550299999999998 2020-08-07T15:07:30.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:30.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.166067+0000", 2020-08-07T15:07:30.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2797999999999998e-05 2020-08-07T15:07:30.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:30.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.166120+0000", 2020-08-07T15:07:30.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15926784399999999 2020-08-07T15:07:30.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.764 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:30.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.765 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:17824 3.d e27/20 3:bd4e0935:::benchmark_data_smithi074_35447_object2227:head v 27'1168, mlcod=27'1163)", 2020-08-07T15:07:30.765 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:07:30.765 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.543574114000002, 2020-08-07T15:07:30.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25249224199999998, 2020-08-07T15:07:30.782 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:30.783 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:30.783 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:30.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:30.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:07:30.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:30.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:30.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:07:30.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9348000000000003e-05 2020-08-07T15:07:30.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:30.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143490+0000", 2020-08-07T15:07:30.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7800000000000003e-06 2020-08-07T15:07:30.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:30.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143496+0000", 2020-08-07T15:07:30.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25095783399999999 2020-08-07T15:07:30.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:30.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394454+0000", 2020-08-07T15:07:30.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6550000000000001e-06 2020-08-07T15:07:30.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:30.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394460+0000", 2020-08-07T15:07:30.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8814999999999999e-05 2020-08-07T15:07:30.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:30.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394509+0000", 2020-08-07T15:07:30.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3482999999999999e-05 2020-08-07T15:07:30.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:30.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394532+0000", 2020-08-07T15:07:30.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013496199999999999 2020-08-07T15:07:30.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:30.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.395882+0000", 2020-08-07T15:07:30.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0707000000000001e-05 2020-08-07T15:07:30.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:30.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.395932+0000", 2020-08-07T15:07:30.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25249224199999998 2020-08-07T15:07:30.851 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.851 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:30.852 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.853 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:18478 3.a 3:524412cd:::benchmark_data_smithi074_35447_object2309:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:30.853 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.592837+0000", 2020-08-07T15:07:30.853 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.094177035000001, 2020-08-07T15:07:30.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21005600699999999, 2020-08-07T15:07:30.854 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:30.854 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:30.854 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:30.854 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:30.854 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:30.855 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 18478 2020-08-07T15:07:30.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.855 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:30.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:30.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592837+0000", 2020-08-07T15:07:30.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:30.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:30.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592837+0000", 2020-08-07T15:07:30.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6718999999999998e-05 2020-08-07T15:07:30.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:30.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592884+0000", 2020-08-07T15:07:30.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2749999999999997e-06 2020-08-07T15:07:30.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:30.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592888+0000", 2020-08-07T15:07:30.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0831e-05 2020-08-07T15:07:30.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:30.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592899+0000", 2020-08-07T15:07:30.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9440000000000002e-06 2020-08-07T15:07:30.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:30.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592905+0000", 2020-08-07T15:07:30.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00088115000000000003 2020-08-07T15:07:30.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:30.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.593786+0000", 2020-08-07T15:07:30.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1789000000000002e-05 2020-08-07T15:07:30.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:30.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.593838+0000", 2020-08-07T15:07:30.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011097799999999999 2020-08-07T15:07:30.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:30.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.593949+0000", 2020-08-07T15:07:30.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00088680400000000004 2020-08-07T15:07:30.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:30.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.594836+0000", 2020-08-07T15:07:30.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1500000000000005e-07 2020-08-07T15:07:30.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:30.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.594836+0000", 2020-08-07T15:07:30.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9746000000000002e-05 2020-08-07T15:07:30.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:30.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.594876+0000", 2020-08-07T15:07:30.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045589070000000001 2020-08-07T15:07:30.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:30.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.599435+0000", 2020-08-07T15:07:30.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.203361244 2020-08-07T15:07:30.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:30.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802796+0000", 2020-08-07T15:07:30.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.178e-06 2020-08-07T15:07:30.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:30.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802797+0000", 2020-08-07T15:07:30.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5927000000000005e-05 2020-08-07T15:07:30.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:30.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802893+0000", 2020-08-07T15:07:30.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21005600699999999 2020-08-07T15:07:30.872 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.873 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:30.873 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.874 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:18478 3.a e27/20 ondisk, result = 0)", 2020-08-07T15:07:30.874 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.595603+0000", 2020-08-07T15:07:30.874 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.091411137000001, 2020-08-07T15:07:30.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.207367254, 2020-08-07T15:07:30.875 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:30.875 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:30.875 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:30.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:30.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.595603+0000", 2020-08-07T15:07:30.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:30.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:30.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.595603+0000", 2020-08-07T15:07:30.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8887999999999999e-05 2020-08-07T15:07:30.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:30.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.595622+0000", 2020-08-07T15:07:30.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8469999999999998e-06 2020-08-07T15:07:30.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:30.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.595626+0000", 2020-08-07T15:07:30.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20636312700000001 2020-08-07T15:07:30.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:30.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.801989+0000", 2020-08-07T15:07:30.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0889999999999999e-06 2020-08-07T15:07:30.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:30.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.801994+0000", 2020-08-07T15:07:30.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00078913099999999999 2020-08-07T15:07:30.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:30.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802783+0000", 2020-08-07T15:07:30.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8919999999999997e-06 2020-08-07T15:07:30.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:30.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802789+0000", 2020-08-07T15:07:30.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018128 2020-08-07T15:07:30.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:30.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802970+0000", 2020-08-07T15:07:30.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.207367254 2020-08-07T15:07:30.908 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.908 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:30.908 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.909 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:33488 3.c e30/20 3:3d1929c7:::benchmark_data_smithi074_35447_object4185:head v 30'2120, mlcod=30'2110)", 2020-08-07T15:07:30.909 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:58.023101+0000", 2020-08-07T15:07:30.909 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.663913346000001, 2020-08-07T15:07:30.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22118623300000001, 2020-08-07T15:07:30.909 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:30.916 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:30.916 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:30.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:30.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023101+0000", 2020-08-07T15:07:30.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:30.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:30.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023101+0000", 2020-08-07T15:07:30.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8794000000000001e-05 2020-08-07T15:07:30.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:30.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023150+0000", 2020-08-07T15:07:30.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.545e-06 2020-08-07T15:07:30.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:30.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023156+0000", 2020-08-07T15:07:30.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.218926757 2020-08-07T15:07:30.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:30.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242083+0000", 2020-08-07T15:07:30.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1200000000000004e-06 2020-08-07T15:07:30.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:30.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242087+0000", 2020-08-07T15:07:30.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9561999999999998e-05 2020-08-07T15:07:30.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:30.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242127+0000", 2020-08-07T15:07:30.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3655999999999998e-05 2020-08-07T15:07:30.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:30.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242150+0000", 2020-08-07T15:07:30.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002089273 2020-08-07T15:07:30.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:30.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.244240+0000", 2020-08-07T15:07:30.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7525999999999999e-05 2020-08-07T15:07:30.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:30.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.244287+0000", 2020-08-07T15:07:30.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22118623300000001 2020-08-07T15:07:30.931 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.931 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:30.931 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.932 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:35856 3.a 3:5b5556e8:::benchmark_data_smithi074_35447_object4481:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-08-07T15:07:30.932 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:59.440060+0000", 2020-08-07T15:07:30.932 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.246954290000001, 2020-08-07T15:07:30.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16587832199999999, 2020-08-07T15:07:30.933 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:30.933 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:30.933 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:30.933 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:30.933 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:30.934 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 35856 2020-08-07T15:07:30.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.934 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:30.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:30.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440060+0000", 2020-08-07T15:07:30.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:30.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:30.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440060+0000", 2020-08-07T15:07:30.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018524399999999999 2020-08-07T15:07:30.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:30.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440245+0000", 2020-08-07T15:07:30.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9519999999999999e-06 2020-08-07T15:07:30.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:30.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440251+0000", 2020-08-07T15:07:30.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1705999999999999e-05 2020-08-07T15:07:30.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:30.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440263+0000", 2020-08-07T15:07:30.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2820000000000004e-06 2020-08-07T15:07:30.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:30.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440268+0000", 2020-08-07T15:07:30.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023611449999999998 2020-08-07T15:07:30.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:30.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.442629+0000", 2020-08-07T15:07:30.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010644799999999999 2020-08-07T15:07:30.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:30.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.442736+0000", 2020-08-07T15:07:30.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017566000000000001 2020-08-07T15:07:30.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:30.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.442912+0000", 2020-08-07T15:07:30.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026960399999999998 2020-08-07T15:07:30.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:30.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.445608+0000", 2020-08-07T15:07:30.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0809999999999999e-06 2020-08-07T15:07:30.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:30.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.445609+0000", 2020-08-07T15:07:30.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0878999999999997e-05 2020-08-07T15:07:30.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:30.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.445680+0000", 2020-08-07T15:07:30.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00065560499999999997 2020-08-07T15:07:30.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:30.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.446335+0000", 2020-08-07T15:07:30.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.159534921 2020-08-07T15:07:30.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:30.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.605870+0000", 2020-08-07T15:07:30.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.187e-06 2020-08-07T15:07:30.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:30.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.605871+0000", 2020-08-07T15:07:30.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7172e-05 2020-08-07T15:07:30.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:30.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.605938+0000", 2020-08-07T15:07:30.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16587832199999999 2020-08-07T15:07:30.954 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.954 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:30.954 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.955 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:42104 3.d e32/20 3:bcc35439:::benchmark_data_smithi074_35447_object5262:head v 32'2864, mlcod=32'2858)", 2020-08-07T15:07:30.955 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:07:30.955 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.443988817000001, 2020-08-07T15:07:30.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28296418899999998, 2020-08-07T15:07:30.956 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:30.956 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:30.956 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:30.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:30.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:07:30.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:30.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:30.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:07:30.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5550000000000002e-05 2020-08-07T15:07:30.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:30.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243081+0000", 2020-08-07T15:07:30.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8139999999999997e-06 2020-08-07T15:07:30.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:30.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243087+0000", 2020-08-07T15:07:30.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27968446800000002 2020-08-07T15:07:30.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:30.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522771+0000", 2020-08-07T15:07:30.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9459999999999997e-06 2020-08-07T15:07:30.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:30.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522776+0000", 2020-08-07T15:07:30.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0778000000000006e-05 2020-08-07T15:07:30.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:30.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522847+0000", 2020-08-07T15:07:30.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0622e-05 2020-08-07T15:07:30.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:30.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522868+0000", 2020-08-07T15:07:30.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030627580000000001 2020-08-07T15:07:30.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:30.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.525930+0000", 2020-08-07T15:07:30.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9252999999999997e-05 2020-08-07T15:07:30.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:30.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.525990+0000", 2020-08-07T15:07:30.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28296418899999998 2020-08-07T15:07:30.977 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.978 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:30.978 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.980 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:42704 3.2 e32/20 3:435b82af:::benchmark_data_smithi074_35447_object5337:head v 32'2440, mlcod=32'2430)", 2020-08-07T15:07:30.980 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.590578+0000", 2020-08-07T15:07:30.981 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.096436331, 2020-08-07T15:07:30.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.156827727, 2020-08-07T15:07:30.981 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:30.981 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:30.981 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:30.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:30.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.590578+0000", 2020-08-07T15:07:30.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:30.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:30.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.590578+0000", 2020-08-07T15:07:30.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0890000000000002e-05 2020-08-07T15:07:30.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:30.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.590629+0000", 2020-08-07T15:07:30.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9349999999999999e-06 2020-08-07T15:07:30.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:30.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.590635+0000", 2020-08-07T15:07:30.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15515162199999999 2020-08-07T15:07:30.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:30.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.745786+0000", 2020-08-07T15:07:30.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6650000000000001e-06 2020-08-07T15:07:30.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:30.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.745790+0000", 2020-08-07T15:07:30.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1458999999999997e-05 2020-08-07T15:07:30.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:30.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.745832+0000", 2020-08-07T15:07:30.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7506000000000001e-05 2020-08-07T15:07:30.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:30.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.745849+0000", 2020-08-07T15:07:30.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001453008 2020-08-07T15:07:30.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:30.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.747302+0000", 2020-08-07T15:07:30.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010364199999999999 2020-08-07T15:07:30.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:30.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.747406+0000", 2020-08-07T15:07:30.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.156827727 2020-08-07T15:07:30.994 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.994 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:30.994 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:30.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.995 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:42944 3.2 e32/20 3:45b0b8d5:::benchmark_data_smithi074_35447_object5367:head v 32'2464, mlcod=32'2452)", 2020-08-07T15:07:30.995 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.763245+0000", 2020-08-07T15:07:30.995 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.923769849999999, 2020-08-07T15:07:30.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.154831946, 2020-08-07T15:07:30.996 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:30.996 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:30.996 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:30.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:30.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.763245+0000", 2020-08-07T15:07:30.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:30.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:30.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.763245+0000", 2020-08-07T15:07:30.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1844e-05 2020-08-07T15:07:30.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:30.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:30.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:30.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.763296+0000", 2020-08-07T15:07:31.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4489999999999996e-06 2020-08-07T15:07:31.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:31.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.763303+0000", 2020-08-07T15:07:31.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.150368538 2020-08-07T15:07:31.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:31.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.913671+0000", 2020-08-07T15:07:31.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8499999999999998e-06 2020-08-07T15:07:31.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:31.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.913674+0000", 2020-08-07T15:07:31.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002168096 2020-08-07T15:07:31.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:31.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.915842+0000", 2020-08-07T15:07:31.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1338000000000001e-05 2020-08-07T15:07:31.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:31.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.915864+0000", 2020-08-07T15:07:31.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021568550000000001 2020-08-07T15:07:31.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:31.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.918020+0000", 2020-08-07T15:07:31.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5976000000000001e-05 2020-08-07T15:07:31.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:31.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.918076+0000", 2020-08-07T15:07:31.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.154831946 2020-08-07T15:07:31.010 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.010 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:31.010 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.011 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:44744 3.2 e32/20 3:4dba534b:::benchmark_data_smithi074_35447_object5592:head v 32'2608, mlcod=32'2602)", 2020-08-07T15:07:31.012 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:04.868608+0000", 2020-08-07T15:07:31.012 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.818406623000001, 2020-08-07T15:07:31.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16039398499999999, 2020-08-07T15:07:31.015 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:31.016 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:31.016 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:31.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:31.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.868608+0000", 2020-08-07T15:07:31.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:31.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:31.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.868608+0000", 2020-08-07T15:07:31.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1788e-05 2020-08-07T15:07:31.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:31.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.868660+0000", 2020-08-07T15:07:31.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2920000000000001e-06 2020-08-07T15:07:31.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:31.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.868666+0000", 2020-08-07T15:07:31.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15842231200000001 2020-08-07T15:07:31.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:31.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:05.027088+0000", 2020-08-07T15:07:31.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5329999999999996e-06 2020-08-07T15:07:31.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:31.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:05.027093+0000", 2020-08-07T15:07:31.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0534999999999998e-05 2020-08-07T15:07:31.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:31.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:05.027123+0000", 2020-08-07T15:07:31.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6172999999999999e-05 2020-08-07T15:07:31.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:31.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:05.027139+0000", 2020-08-07T15:07:31.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001801813 2020-08-07T15:07:31.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:31.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:05.028941+0000", 2020-08-07T15:07:31.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0538999999999998e-05 2020-08-07T15:07:31.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:31.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:05.029002+0000", 2020-08-07T15:07:31.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16039398499999999 2020-08-07T15:07:31.030 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.031 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:31.031 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.031 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:48144 3.0 e36/20 3:0cec1d53:::benchmark_data_smithi074_35447_object6017:head v 36'3128, mlcod=36'3120)", 2020-08-07T15:07:31.032 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:08.897123+0000", 2020-08-07T15:07:31.033 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.789891106999999, 2020-08-07T15:07:31.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25221023399999998, 2020-08-07T15:07:31.034 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:31.034 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:31.034 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:31.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:31.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897123+0000", 2020-08-07T15:07:31.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:31.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:31.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897123+0000", 2020-08-07T15:07:31.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0655999999999997e-05 2020-08-07T15:07:31.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:31.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897174+0000", 2020-08-07T15:07:31.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6769999999999998e-06 2020-08-07T15:07:31.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:31.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897181+0000", 2020-08-07T15:07:31.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25090537899999998 2020-08-07T15:07:31.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:31.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148086+0000", 2020-08-07T15:07:31.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5520000000000004e-06 2020-08-07T15:07:31.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:31.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148092+0000", 2020-08-07T15:07:31.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6391999999999999e-05 2020-08-07T15:07:31.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:31.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148138+0000", 2020-08-07T15:07:31.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8124999999999999e-05 2020-08-07T15:07:31.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:31.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148166+0000", 2020-08-07T15:07:31.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011152009999999999 2020-08-07T15:07:31.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:31.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.149281+0000", 2020-08-07T15:07:31.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2252e-05 2020-08-07T15:07:31.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:31.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.149333+0000", 2020-08-07T15:07:31.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25221023399999998 2020-08-07T15:07:31.060 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.061 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:31.061 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.061 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:48776 3.2 e36/20 3:47029331:::benchmark_data_smithi074_35447_object6096:head v 36'2888, mlcod=36'2881)", 2020-08-07T15:07:31.062 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:07:31.062 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.282494291999999, 2020-08-07T15:07:31.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27006165399999998, 2020-08-07T15:07:31.062 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:31.062 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:31.063 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:31.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:31.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:07:31.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:31.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:31.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:07:31.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8454000000000001e-05 2020-08-07T15:07:31.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:31.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404569+0000", 2020-08-07T15:07:31.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2349999999999996e-06 2020-08-07T15:07:31.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:31.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404575+0000", 2020-08-07T15:07:31.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26921690999999998 2020-08-07T15:07:31.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:31.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673792+0000", 2020-08-07T15:07:31.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3330000000000002e-06 2020-08-07T15:07:31.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:31.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673798+0000", 2020-08-07T15:07:31.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1013000000000001e-05 2020-08-07T15:07:31.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:31.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673849+0000", 2020-08-07T15:07:31.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7044999999999999e-05 2020-08-07T15:07:31.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:31.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673876+0000", 2020-08-07T15:07:31.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00065173100000000001 2020-08-07T15:07:31.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:31.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.674528+0000", 2020-08-07T15:07:31.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3933e-05 2020-08-07T15:07:31.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:31.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.674582+0000", 2020-08-07T15:07:31.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27006165399999998 2020-08-07T15:07:31.080 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.081 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:31.081 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.081 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:49208 3.2 e36/20 3:4d72b982:::benchmark_data_smithi074_35447_object6150:head v 36'2904, mlcod=36'2897)", 2020-08-07T15:07:31.082 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:09.692013+0000", 2020-08-07T15:07:31.082 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.995001263999999, 2020-08-07T15:07:31.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.234386975, 2020-08-07T15:07:31.082 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:31.082 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:31.083 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:31.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:31.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.692013+0000", 2020-08-07T15:07:31.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:31.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:31.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.692013+0000", 2020-08-07T15:07:31.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0998000000000002e-05 2020-08-07T15:07:31.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:31.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.692064+0000", 2020-08-07T15:07:31.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0210000000000001e-05 2020-08-07T15:07:31.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:31.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.692074+0000", 2020-08-07T15:07:31.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.233053807 2020-08-07T15:07:31.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:31.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.925128+0000", 2020-08-07T15:07:31.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4879999999999998e-06 2020-08-07T15:07:31.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:31.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.925134+0000", 2020-08-07T15:07:31.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5154999999999997e-05 2020-08-07T15:07:31.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:31.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.925179+0000", 2020-08-07T15:07:31.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2710000000000001e-05 2020-08-07T15:07:31.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:31.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.925201+0000", 2020-08-07T15:07:31.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001143725 2020-08-07T15:07:31.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:31.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.926345+0000", 2020-08-07T15:07:31.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4882000000000001e-05 2020-08-07T15:07:31.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:31.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.926400+0000", 2020-08-07T15:07:31.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.234386975 2020-08-07T15:07:31.110 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.110 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:31.110 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.111 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:55192 3.2 e37/20 3:4a96099f:::benchmark_data_smithi074_35447_object6898:head v 37'3336, mlcod=37'3326)", 2020-08-07T15:07:31.111 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:13.771422+0000", 2020-08-07T15:07:31.112 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.915592595, 2020-08-07T15:07:31.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16800543000000001, 2020-08-07T15:07:31.112 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:31.112 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:31.112 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:31.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:31.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.771422+0000", 2020-08-07T15:07:31.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:31.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:31.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.771422+0000", 2020-08-07T15:07:31.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9376000000000001e-05 2020-08-07T15:07:31.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:31.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.771451+0000", 2020-08-07T15:07:31.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8789999999999996e-06 2020-08-07T15:07:31.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:31.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.771455+0000", 2020-08-07T15:07:31.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16555510500000001 2020-08-07T15:07:31.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:31.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.937010+0000", 2020-08-07T15:07:31.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3770000000000004e-06 2020-08-07T15:07:31.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:31.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.937015+0000", 2020-08-07T15:07:31.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6736999999999997e-05 2020-08-07T15:07:31.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:31.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.937052+0000", 2020-08-07T15:07:31.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3654000000000001e-05 2020-08-07T15:07:31.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:31.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.937076+0000", 2020-08-07T15:07:31.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022779929999999999 2020-08-07T15:07:31.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:31.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.939354+0000", 2020-08-07T15:07:31.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3308999999999999e-05 2020-08-07T15:07:31.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:31.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.939427+0000", 2020-08-07T15:07:31.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16800543000000001 2020-08-07T15:07:31.123 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.123 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:31.124 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.125 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:71904 3.a 3:5e6d3be8:::benchmark_data_smithi074_35447_object8987:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e43)", 2020-08-07T15:07:31.125 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:24.780800+0000", 2020-08-07T15:07:31.126 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.9062143840000001, 2020-08-07T15:07:31.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.159596976, 2020-08-07T15:07:31.126 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:31.126 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:31.126 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:31.127 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:31.127 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:31.127 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 71904 2020-08-07T15:07:31.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.127 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:31.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:31.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.780800+0000", 2020-08-07T15:07:31.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:31.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:31.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.780800+0000", 2020-08-07T15:07:31.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3742999999999998e-05 2020-08-07T15:07:31.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:31.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.780844+0000", 2020-08-07T15:07:31.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9300000000000002e-06 2020-08-07T15:07:31.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:31.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.780849+0000", 2020-08-07T15:07:31.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9189999999999994e-06 2020-08-07T15:07:31.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:31.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.780859+0000", 2020-08-07T15:07:31.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3789999999999999e-06 2020-08-07T15:07:31.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:31.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.780863+0000", 2020-08-07T15:07:31.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016907109999999999 2020-08-07T15:07:31.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:31.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.782554+0000", 2020-08-07T15:07:31.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0457999999999993e-05 2020-08-07T15:07:31.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:31.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.782644+0000", 2020-08-07T15:07:31.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021834600000000001 2020-08-07T15:07:31.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:31.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.782862+0000", 2020-08-07T15:07:31.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022362290000000002 2020-08-07T15:07:31.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:31.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.785099+0000", 2020-08-07T15:07:31.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2699999999999998e-07 2020-08-07T15:07:31.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:31.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.785100+0000", 2020-08-07T15:07:31.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5136999999999997e-05 2020-08-07T15:07:31.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:31.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.785165+0000", 2020-08-07T15:07:31.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053074109999999997 2020-08-07T15:07:31.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:31.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.790472+0000", 2020-08-07T15:07:31.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14985902700000001 2020-08-07T15:07:31.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:31.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.940331+0000", 2020-08-07T15:07:31.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1399999999999996e-07 2020-08-07T15:07:31.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:31.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.940332+0000", 2020-08-07T15:07:31.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4944999999999996e-05 2020-08-07T15:07:31.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:31.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.940397+0000", 2020-08-07T15:07:31.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.159596976 2020-08-07T15:07:31.155 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.155 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:31.156 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.156 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:71904 3.a e43/20 ondisk, result = 0)", 2020-08-07T15:07:31.157 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:24.785667+0000", 2020-08-07T15:07:31.157 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.9013470000000003, 2020-08-07T15:07:31.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.154732906, 2020-08-07T15:07:31.157 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:31.157 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:31.158 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:31.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:31.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.785667+0000", 2020-08-07T15:07:31.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:31.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:31.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.785667+0000", 2020-08-07T15:07:31.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2913999999999997e-05 2020-08-07T15:07:31.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:31.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.785700+0000", 2020-08-07T15:07:31.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.156e-06 2020-08-07T15:07:31.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:31.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.785706+0000", 2020-08-07T15:07:31.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.153932454 2020-08-07T15:07:31.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:31.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.939639+0000", 2020-08-07T15:07:31.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.524e-06 2020-08-07T15:07:31.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:31.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.939644+0000", 2020-08-07T15:07:31.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000669489 2020-08-07T15:07:31.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:31.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.940314+0000", 2020-08-07T15:07:31.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.994e-06 2020-08-07T15:07:31.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:31.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.940322+0000", 2020-08-07T15:07:31.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8374999999999994e-05 2020-08-07T15:07:31.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:31.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.940400+0000", 2020-08-07T15:07:31.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.154732906 2020-08-07T15:07:31.167 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.168 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:31.168 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.168 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:77967 3.2 e46/20 3:48f61a4b:::benchmark_data_smithi074_35447_object9745:head v 46'4807, mlcod=46'4798)", 2020-08-07T15:07:31.169 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:28.702144+0000", 2020-08-07T15:07:31.169 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.9848701019999999, 2020-08-07T15:07:31.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26780679000000002, 2020-08-07T15:07:31.169 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:31.170 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:31.170 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:31.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:31.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702144+0000", 2020-08-07T15:07:31.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:31.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:31.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702144+0000", 2020-08-07T15:07:31.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5482000000000002e-05 2020-08-07T15:07:31.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:31.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702200+0000", 2020-08-07T15:07:31.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.216e-06 2020-08-07T15:07:31.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:31.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702207+0000", 2020-08-07T15:07:31.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26634085400000002 2020-08-07T15:07:31.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:31.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.968548+0000", 2020-08-07T15:07:31.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2490000000000001e-06 2020-08-07T15:07:31.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:31.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.968553+0000", 2020-08-07T15:07:31.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7738999999999998e-05 2020-08-07T15:07:31.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:31.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.968601+0000", 2020-08-07T15:07:31.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5783999999999999e-05 2020-08-07T15:07:31.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:31.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.968627+0000", 2020-08-07T15:07:31.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001270471 2020-08-07T15:07:31.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:31.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.969897+0000", 2020-08-07T15:07:31.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3995000000000001e-05 2020-08-07T15:07:31.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:31.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.969951+0000", 2020-08-07T15:07:31.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26780679000000002 2020-08-07T15:07:31.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.183 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:31.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.184 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:77968 3.2 e46/20 3:48f61a4b:::benchmark_data_smithi074_35447_object9745:head v 46'4808, mlcod=46'4799)", 2020-08-07T15:07:31.185 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:28.702836+0000", 2020-08-07T15:07:31.185 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.9841782619999999, 2020-08-07T15:07:31.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27322339400000001, 2020-08-07T15:07:31.186 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:31.186 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:31.186 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:31.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:31.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702836+0000", 2020-08-07T15:07:31.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:31.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:31.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702836+0000", 2020-08-07T15:07:31.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1822e-05 2020-08-07T15:07:31.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:31.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702888+0000", 2020-08-07T15:07:31.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4110000000000001e-06 2020-08-07T15:07:31.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:31.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702894+0000", 2020-08-07T15:07:31.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26750942999999999 2020-08-07T15:07:31.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:31.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970404+0000", 2020-08-07T15:07:31.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1150000000000002e-06 2020-08-07T15:07:31.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:31.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970409+0000", 2020-08-07T15:07:31.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2898000000000007e-05 2020-08-07T15:07:31.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:31.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970472+0000", 2020-08-07T15:07:31.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2311999999999999e-05 2020-08-07T15:07:31.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:31.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970494+0000", 2020-08-07T15:07:31.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055286800000000002 2020-08-07T15:07:31.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:31.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.976023+0000", 2020-08-07T15:07:31.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6726e-05 2020-08-07T15:07:31.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:31.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.976059+0000", 2020-08-07T15:07:31.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27322339400000001 2020-08-07T15:07:31.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.202 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:31.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.203 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:79751 3.0 e47/20 3:010604f4:::benchmark_data_smithi074_35447_object9968:head v 47'4983, mlcod=47'4975)", 2020-08-07T15:07:31.203 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:29.891721+0000", 2020-08-07T15:07:31.203 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.79529383300000001, 2020-08-07T15:07:31.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15340316800000001, 2020-08-07T15:07:31.204 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:31.204 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:31.204 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:31.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:31.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.891721+0000", 2020-08-07T15:07:31.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:31.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:31.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.891721+0000", 2020-08-07T15:07:31.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4506999999999998e-05 2020-08-07T15:07:31.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:31.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.891775+0000", 2020-08-07T15:07:31.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9439999999999999e-06 2020-08-07T15:07:31.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:31.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.891782+0000", 2020-08-07T15:07:31.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.149474843 2020-08-07T15:07:31.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:31.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:30.041257+0000", 2020-08-07T15:07:31.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6669999999999999e-06 2020-08-07T15:07:31.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:31.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:30.041263+0000", 2020-08-07T15:07:31.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8452000000000001e-05 2020-08-07T15:07:31.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:31.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:30.041332+0000", 2020-08-07T15:07:31.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2325999999999999e-05 2020-08-07T15:07:31.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:31.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:30.041354+0000", 2020-08-07T15:07:31.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037205559999999999 2020-08-07T15:07:31.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:31.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:30.045075+0000", 2020-08-07T15:07:31.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8872999999999998e-05 2020-08-07T15:07:31.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:31.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:30.045124+0000", 2020-08-07T15:07:31.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15340316800000001 2020-08-07T15:07:31.225 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.225 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:31.225 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.226 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:79752 3.0 e47/20 3:010604f4:::benchmark_data_smithi074_35447_object9968:head v 47'4984, mlcod=47'4975)", 2020-08-07T15:07:31.226 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:29.892380+0000", 2020-08-07T15:07:31.226 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.79463452099999998, 2020-08-07T15:07:31.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15277944199999999, 2020-08-07T15:07:31.226 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:31.227 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:31.227 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:31.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:31.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.892380+0000", 2020-08-07T15:07:31.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:31.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:31.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.892380+0000", 2020-08-07T15:07:31.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3628000000000002e-05 2020-08-07T15:07:31.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:31.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.892433+0000", 2020-08-07T15:07:31.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4409999999999998e-06 2020-08-07T15:07:31.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:31.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.892440+0000", 2020-08-07T15:07:31.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.149381548 2020-08-07T15:07:31.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:31.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:30.041821+0000", 2020-08-07T15:07:31.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.921e-06 2020-08-07T15:07:31.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:31.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:30.041826+0000", 2020-08-07T15:07:31.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8850000000000002e-05 2020-08-07T15:07:31.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:31.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:30.041865+0000", 2020-08-07T15:07:31.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2025999999999999e-05 2020-08-07T15:07:31.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:31.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:30.041887+0000", 2020-08-07T15:07:31.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00323829 2020-08-07T15:07:31.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:31.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:30.045126+0000", 2020-08-07T15:07:31.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3738000000000001e-05 2020-08-07T15:07:31.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:31.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:30.045159+0000", 2020-08-07T15:07:31.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15277944199999999 2020-08-07T15:07:31.238 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.238 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:31.238 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.239 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.239 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:31.239 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:31.239 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:30.697+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:31.240 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:07:31.242 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.0 to 1 (10000) 2020-08-07T15:07:31.243 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:07:31.243 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:07:31.244 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:31.240+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:31.246 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 55 16 81967 81951 11.6398 12.4062 0.0148484 0.0107331 2020-08-07T15:07:31.255 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 1 1 2020-08-07T15:07:31.380 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:31.370+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:31.444 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:31.443+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:31.453 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:07:31.462 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:31.463 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:31.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.463 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:82135 3.e 3:764ddde7:::benchmark_data_smithi074_35447_object10266:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e49)", 2020-08-07T15:07:31.463 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:31.354404+0000", 2020-08-07T15:07:31.464 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.102227025, 2020-08-07T15:07:31.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.10225282300000001, 2020-08-07T15:07:31.464 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:31.464 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:31.464 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:31.465 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:31.465 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:31.465 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 82135 2020-08-07T15:07:31.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.465 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:31.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:31.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:31.354404+0000", 2020-08-07T15:07:31.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:31.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:31.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:31.354404+0000", 2020-08-07T15:07:31.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8433e-05 2020-08-07T15:07:31.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:31.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:31.354432+0000", 2020-08-07T15:07:31.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.087e-06 2020-08-07T15:07:31.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:31.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:31.354436+0000", 2020-08-07T15:07:31.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4989999999999997e-06 2020-08-07T15:07:31.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:31.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:31.354444+0000", 2020-08-07T15:07:31.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.912e-06 2020-08-07T15:07:31.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:31.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:31.354448+0000", 2020-08-07T15:07:31.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024474570000000001 2020-08-07T15:07:31.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:31.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:31.356895+0000", 2020-08-07T15:07:31.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2540999999999995e-05 2020-08-07T15:07:31.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:31.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:31.356978+0000", 2020-08-07T15:07:31.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101406 2020-08-07T15:07:31.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:31.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:31.357079+0000", 2020-08-07T15:07:31.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027386009999999998 2020-08-07T15:07:31.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:31.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:31.359818+0000", 2020-08-07T15:07:31.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9100000000000003e-07 2020-08-07T15:07:31.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:31.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:31.359818+0000", 2020-08-07T15:07:31.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2376e-05 2020-08-07T15:07:31.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:31.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:31.359891+0000", 2020-08-07T15:07:31.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010525827999999999 2020-08-07T15:07:31.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:31.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:31.370417+0000", 2020-08-07T15:07:31.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016012831000000002 2020-08-07T15:07:31.530 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.530 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:31.530 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.531 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:82136 3.e 3:764ddde7:::benchmark_data_smithi074_35447_object10266:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e49)", 2020-08-07T15:07:31.531 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:31.355189+0000", 2020-08-07T15:07:31.531 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.10144226000000001, 2020-08-07T15:07:31.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.101542276, 2020-08-07T15:07:31.532 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:31.532 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:31.532 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:31.532 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:31.533 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:31.533 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 82136 2020-08-07T15:07:31.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.533 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:31.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:31.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:31.355189+0000", 2020-08-07T15:07:31.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:31.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:31.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:31.355189+0000", 2020-08-07T15:07:31.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9960000000000003e-05 2020-08-07T15:07:31.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:31.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:31.355239+0000", 2020-08-07T15:07:31.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0390000000000004e-06 2020-08-07T15:07:31.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:31.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:31.355244+0000", 2020-08-07T15:07:31.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7710000000000006e-06 2020-08-07T15:07:31.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:31.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:31.355251+0000", 2020-08-07T15:07:31.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1389999999999997e-06 2020-08-07T15:07:31.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:31.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:31.355255+0000", 2020-08-07T15:07:31.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020636140000000001 2020-08-07T15:07:31.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:31.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:31.357319+0000", 2020-08-07T15:07:31.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0467999999999999e-05 2020-08-07T15:07:31.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:31.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:31.357370+0000", 2020-08-07T15:07:31.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2666999999999999e-05 2020-08-07T15:07:31.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:31.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:31.357462+0000", 2020-08-07T15:07:31.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024417269999999999 2020-08-07T15:07:31.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:31.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:31.359904+0000", 2020-08-07T15:07:31.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5300000000000003e-07 2020-08-07T15:07:31.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:31.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:31.359905+0000", 2020-08-07T15:07:31.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8205999999999998e-05 2020-08-07T15:07:31.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:31.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:31.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:31.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:31.359963+0000", 2020-08-07T15:07:31.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0047744440000000001 2020-08-07T15:07:31.588 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.588 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:31.588 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.588 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:31.588 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:07:31.589 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 2 2020-08-07T15:07:31.589 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:31.590 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:07:31.591 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:07:31.591 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:07:31.591 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:31.589+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:31.592 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:07:31.603 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:07:31.641 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:07:31.682 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:07:31.692 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:31.691+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:31.727 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:07:31.730 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:07:31.730 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:07:31.730 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:07:31.730 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:07:31.731 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:07:31.731 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:07:31.731 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:07:31.731 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:07:31.792 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:31.791+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:31.812 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:31.812 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:31.829 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:07:31.932 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:31.920+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:32.002 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:31.996+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:32.046 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:32.047 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:32.054 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:07:32.108 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:32.107+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:32.154 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.1 to 1 (10000) 2020-08-07T15:07:32.171 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 2 1 2020-08-07T15:07:32.213 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:32.211+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:32.247 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:32.247 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:32.248 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 56 16 83601 83585 11.6599 12.7656 0.0102614 0.0107166 2020-08-07T15:07:32.256 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:07:32.304 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:32.303+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:32.407 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:32.406+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:32.486 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:32.487 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:32.500 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:07:32.542 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:32.540+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:32.609 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:32.608+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:32.672 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:32.673 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:32.683 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:07:32.710 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:32.710+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:32.811 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:32.811+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:32.910 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:32.910 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:32.912 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:32.912+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:32.915 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:07:33.013 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:33.012+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:33.115 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:33.114+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:33.128 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:33.128 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:33.139 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:07:33.154 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.2 to 1 (10000) 2020-08-07T15:07:33.168 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 3 1 2020-08-07T15:07:33.245 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 57 16 85050 85034 11.6539 11.3203 0.00959801 0.0107212 2020-08-07T15:07:33.246 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:33.244+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:33.255 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:07:33.317 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:33.317+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:33.369 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:33.370 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:33.383 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:07:33.419 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:33.418+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:33.520 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:33.520+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:33.593 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:33.595 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:33.607 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:07:33.651 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:33.650+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:33.723 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:33.722+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:33.743 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:07:33.812 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:33.815 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:33.825 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:07:33.827 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:33.824+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:33.930 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:33.930+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:33.964 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:07:34.000 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:07:34.000 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:34.001 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:34.013 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:07:34.037 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:34.034+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:34.040 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:07:34.090 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:07:34.124 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:07:34.135 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:34.135+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:34.163 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:07:34.202 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:34.203 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:07:34.203 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.203 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:86528 3.7 3:ea012d6b:::benchmark_data_smithi074_35447_object10815:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e51)", 2020-08-07T15:07:34.204 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.191637+0000", 2020-08-07T15:07:34.204 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.010287418, 2020-08-07T15:07:34.204 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.010307748, 2020-08-07T15:07:34.204 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:34.204 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:34.204 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:34.205 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:34.205 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:34.205 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 86528 2020-08-07T15:07:34.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.205 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:34.206 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.206 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:34.206 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.191637+0000", 2020-08-07T15:07:34.206 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:34.206 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.207 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.207 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:34.207 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.191637+0000", 2020-08-07T15:07:34.207 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2617e-05 2020-08-07T15:07:34.207 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.208 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.208 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:34.208 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.191700+0000", 2020-08-07T15:07:34.208 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7020000000000004e-06 2020-08-07T15:07:34.208 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.208 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.209 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:34.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.191707+0000", 2020-08-07T15:07:34.209 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8938000000000001e-05 2020-08-07T15:07:34.209 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.209 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.210 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:34.210 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.191726+0000", 2020-08-07T15:07:34.210 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9260000000000003e-06 2020-08-07T15:07:34.210 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.210 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.211 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:34.211 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.191732+0000", 2020-08-07T15:07:34.211 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00033361699999999997 2020-08-07T15:07:34.211 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.211 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.256 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:34.256 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.192066+0000", 2020-08-07T15:07:34.256 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000118405 2020-08-07T15:07:34.257 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.257 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.257 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:34.257 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.192184+0000", 2020-08-07T15:07:34.257 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00025296900000000002 2020-08-07T15:07:34.258 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.258 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.258 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:07:34.258 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.192437+0000", 2020-08-07T15:07:34.259 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033533980000000001 2020-08-07T15:07:34.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.259 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.259 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:34.259 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.195791+0000", 2020-08-07T15:07:34.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0900000000000002e-07 2020-08-07T15:07:34.260 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.260 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.260 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:34.260 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.195791+0000", 2020-08-07T15:07:34.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5361e-05 2020-08-07T15:07:34.261 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.261 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.261 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:34.261 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.195857+0000", 2020-08-07T15:07:34.261 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042194420000000003 2020-08-07T15:07:34.262 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.262 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:34.262 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.262 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.262 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:07:34.263 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 1 2020-08-07T15:07:34.264 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:34.264 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:07:34.267 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:07:34.268 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:07:34.268 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.3 to 1 (10000) 2020-08-07T15:07:34.268 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:07:34.269 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:07:34.269 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:07:34.269 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:07:34.269 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:07:34.269 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:07:34.270 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:07:34.270 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:07:34.270 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:07:34.270 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 58 16 86603 86587 11.6622 12.1328 0.011181 0.0107116 2020-08-07T15:07:34.271 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 4 1 2020-08-07T15:07:34.272 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:34.264+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:34.368 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:34.367+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:34.457 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:34.458 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:07:34.458 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:07:34.458 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:07:34.459 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:34.467 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:07:34.468 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:34.467+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:34.573 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:34.571+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:34.646 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:34.646 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:07:34.646 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:07:34.647 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:07:34.647 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.647 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:2600 3.b 3:d6a1a0d3:::benchmark_data_smithi074_35447_object324:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e23)", 2020-08-07T15:07:34.647 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:37.800755+0000", 2020-08-07T15:07:34.648 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 56.841032230000003, 2020-08-07T15:07:34.648 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.215895015, 2020-08-07T15:07:34.648 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:34.648 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:34.649 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:34.650 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:34.650 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:34.650 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 2600 2020-08-07T15:07:34.650 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.651 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:34.651 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.651 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:34.651 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.800755+0000", 2020-08-07T15:07:34.651 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:34.652 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.652 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.652 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:34.652 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.800755+0000", 2020-08-07T15:07:34.652 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7878999999999999e-05 2020-08-07T15:07:34.653 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.653 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.653 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:34.653 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.800782+0000", 2020-08-07T15:07:34.653 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0520000000000003e-06 2020-08-07T15:07:34.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:34.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.800787+0000", 2020-08-07T15:07:34.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0587999999999999e-05 2020-08-07T15:07:34.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.670 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:34.670 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.800797+0000", 2020-08-07T15:07:34.670 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4970000000000001e-06 2020-08-07T15:07:34.670 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.671 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.671 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:34.671 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.800803+0000", 2020-08-07T15:07:34.671 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0059548989999999996 2020-08-07T15:07:34.671 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.671 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.672 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:34.672 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.806757+0000", 2020-08-07T15:07:34.672 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000111375 2020-08-07T15:07:34.672 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.672 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.673 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:34.673 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.806869+0000", 2020-08-07T15:07:34.673 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000159345 2020-08-07T15:07:34.673 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.673 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.678 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:34.678 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.807028+0000", 2020-08-07T15:07:34.679 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002275329 2020-08-07T15:07:34.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.679 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:34.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.809304+0000", 2020-08-07T15:07:34.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2900000000000002e-07 2020-08-07T15:07:34.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.680 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:34.680 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.809304+0000", 2020-08-07T15:07:34.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8708000000000002e-05 2020-08-07T15:07:34.681 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.681 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:34.681 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.809353+0000", 2020-08-07T15:07:34.682 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.026379923 2020-08-07T15:07:34.682 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.683 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.683 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.683 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:37.835733+0000", 2020-08-07T15:07:34.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18084716200000001 2020-08-07T15:07:34.683 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.684 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.684 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.684 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.016580+0000", 2020-08-07T15:07:34.684 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9299999999999997e-07 2020-08-07T15:07:34.684 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.685 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.685 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:34.685 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.016581+0000", 2020-08-07T15:07:34.685 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8535999999999996e-05 2020-08-07T15:07:34.685 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.686 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.686 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:34.686 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:38.016650+0000", 2020-08-07T15:07:34.686 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.215895015 2020-08-07T15:07:34.686 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.687 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:34.687 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.687 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.687 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.687 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:18166 3.8 3:15dfcb22:::benchmark_data_smithi074_35447_object2270:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:34.688 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:48.342194+0000", 2020-08-07T15:07:34.688 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 46.299592777999997, 2020-08-07T15:07:34.688 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25725213099999999, 2020-08-07T15:07:34.689 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:34.689 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:34.690 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:34.690 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:34.690 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:34.690 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18166 2020-08-07T15:07:34.691 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.691 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:34.691 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.691 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:34.692 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.342194+0000", 2020-08-07T15:07:34.692 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:34.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.692 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:34.693 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.342194+0000", 2020-08-07T15:07:34.693 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6729e-05 2020-08-07T15:07:34.693 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.693 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.694 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:34.694 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.342231+0000", 2020-08-07T15:07:34.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7229999999999998e-06 2020-08-07T15:07:34.695 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.695 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:34.695 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.342237+0000", 2020-08-07T15:07:34.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0136000000000002e-05 2020-08-07T15:07:34.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.696 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:34.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.342258+0000", 2020-08-07T15:07:34.697 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9989999999999997e-06 2020-08-07T15:07:34.697 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:34.698 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.342264+0000", 2020-08-07T15:07:34.698 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012111499999999999 2020-08-07T15:07:34.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.698 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.698 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:34.699 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.342385+0000", 2020-08-07T15:07:34.699 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000103878 2020-08-07T15:07:34.699 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.699 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.699 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:34.700 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.342489+0000", 2020-08-07T15:07:34.700 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00022590199999999999 2020-08-07T15:07:34.700 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.700 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:07:34.701 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.342715+0000", 2020-08-07T15:07:34.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003627019 2020-08-07T15:07:34.701 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.701 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:34.702 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.346342+0000", 2020-08-07T15:07:34.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.158e-06 2020-08-07T15:07:34.702 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.702 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.702 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:34.703 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.346343+0000", 2020-08-07T15:07:34.703 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7310000000000004e-05 2020-08-07T15:07:34.703 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.705 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:34.706 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.346410+0000", 2020-08-07T15:07:34.706 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001407782 2020-08-07T15:07:34.706 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.707 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.707 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.347818+0000", 2020-08-07T15:07:34.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.251560536 2020-08-07T15:07:34.708 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.708 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.709 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.599378+0000", 2020-08-07T15:07:34.709 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2920000000000001e-06 2020-08-07T15:07:34.709 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.709 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.709 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:34.710 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.599380+0000", 2020-08-07T15:07:34.710 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6551999999999996e-05 2020-08-07T15:07:34.710 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.710 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.710 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:34.711 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.599446+0000", 2020-08-07T15:07:34.711 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25725213099999999 2020-08-07T15:07:34.711 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.711 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:34.712 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.712 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.712 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.712 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:18167 3.8 3:15dfcb22:::benchmark_data_smithi074_35447_object2270:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:34.712 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:48.342989+0000", 2020-08-07T15:07:34.713 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 46.298797327000003, 2020-08-07T15:07:34.713 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25656989899999999, 2020-08-07T15:07:34.713 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:34.713 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:34.713 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:34.714 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:34.715 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:34.715 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18167 2020-08-07T15:07:34.716 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.716 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:34.716 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.716 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:34.716 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.342989+0000", 2020-08-07T15:07:34.717 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:34.717 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.717 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.717 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:34.718 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.342989+0000", 2020-08-07T15:07:34.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7729999999999999e-05 2020-08-07T15:07:34.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.718 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.718 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:34.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.343037+0000", 2020-08-07T15:07:34.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9149999999999999e-06 2020-08-07T15:07:34.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:34.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.343044+0000", 2020-08-07T15:07:34.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9283999999999996e-05 2020-08-07T15:07:34.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:34.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.343113+0000", 2020-08-07T15:07:34.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3370000000000006e-06 2020-08-07T15:07:34.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.721 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:34.722 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.343123+0000", 2020-08-07T15:07:34.722 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0014698319999999999 2020-08-07T15:07:34.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.723 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:34.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.344593+0000", 2020-08-07T15:07:34.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000110857 2020-08-07T15:07:34.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.724 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.724 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:34.724 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.344703+0000", 2020-08-07T15:07:34.724 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017357700000000001 2020-08-07T15:07:34.724 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.725 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:07:34.725 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.344877+0000", 2020-08-07T15:07:34.725 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015451519999999999 2020-08-07T15:07:34.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.727 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.727 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:34.727 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.346422+0000", 2020-08-07T15:07:34.727 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5000000000000003e-07 2020-08-07T15:07:34.727 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:34.728 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.346423+0000", 2020-08-07T15:07:34.728 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0982e-05 2020-08-07T15:07:34.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.729 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.729 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:34.729 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.346484+0000", 2020-08-07T15:07:34.729 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0046474940000000003 2020-08-07T15:07:34.729 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.730 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.730 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.730 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.351131+0000", 2020-08-07T15:07:34.730 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24836395 2020-08-07T15:07:34.730 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.731 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.731 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.599495+0000", 2020-08-07T15:07:34.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7899999999999999e-07 2020-08-07T15:07:34.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:34.732 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.599496+0000", 2020-08-07T15:07:34.732 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3759999999999999e-05 2020-08-07T15:07:34.732 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.732 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.733 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:34.733 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.599559+0000", 2020-08-07T15:07:34.733 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25656989899999999 2020-08-07T15:07:34.733 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.733 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:34.733 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.734 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.734 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.734 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:18168 3.8 3:15dfcb22:::benchmark_data_smithi074_35447_object2270:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:34.734 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:48.343981+0000", 2020-08-07T15:07:34.734 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 46.297806059999999, 2020-08-07T15:07:34.735 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25567693400000002, 2020-08-07T15:07:34.735 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:34.735 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:34.735 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:34.735 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:34.736 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:34.736 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18168 2020-08-07T15:07:34.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.736 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:34.736 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.737 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:34.737 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.343981+0000", 2020-08-07T15:07:34.737 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:34.737 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.737 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.737 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:34.738 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.343981+0000", 2020-08-07T15:07:34.738 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2184e-05 2020-08-07T15:07:34.738 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.738 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.738 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:34.739 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.344033+0000", 2020-08-07T15:07:34.739 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3769999999999998e-06 2020-08-07T15:07:34.739 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.739 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.739 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:34.740 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.344037+0000", 2020-08-07T15:07:34.740 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5410000000000006e-06 2020-08-07T15:07:34.740 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.740 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.740 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:34.740 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.344047+0000", 2020-08-07T15:07:34.741 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8470000000000003e-06 2020-08-07T15:07:34.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.741 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.741 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:34.741 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.344052+0000", 2020-08-07T15:07:34.742 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011857199999999999 2020-08-07T15:07:34.742 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.742 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.742 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:34.742 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.345237+0000", 2020-08-07T15:07:34.743 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0568e-05 2020-08-07T15:07:34.743 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.743 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:34.743 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.345298+0000", 2020-08-07T15:07:34.743 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012783200000000001 2020-08-07T15:07:34.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:07:34.744 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.345426+0000", 2020-08-07T15:07:34.744 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001066858 2020-08-07T15:07:34.745 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.745 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:34.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.346493+0000", 2020-08-07T15:07:34.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.75e-07 2020-08-07T15:07:34.746 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.746 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:34.746 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.346493+0000", 2020-08-07T15:07:34.746 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013222400000000001 2020-08-07T15:07:34.747 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.747 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.747 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:34.747 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.346625+0000", 2020-08-07T15:07:34.747 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.073894926999999999 2020-08-07T15:07:34.748 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.748 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.748 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.748 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.420520+0000", 2020-08-07T15:07:34.748 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.179097231 2020-08-07T15:07:34.749 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.749 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.749 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.749 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.599618+0000", 2020-08-07T15:07:34.749 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9299999999999998e-07 2020-08-07T15:07:34.750 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.750 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.750 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:34.750 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.599618+0000", 2020-08-07T15:07:34.751 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9456999999999998e-05 2020-08-07T15:07:34.751 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.751 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.751 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:34.751 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:48.599658+0000", 2020-08-07T15:07:34.752 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25567693400000002 2020-08-07T15:07:34.752 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.752 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:34.752 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.752 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.753 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.753 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:24264 3.8 3:1d3fdbda:::benchmark_data_smithi074_35447_object3032:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e28)", 2020-08-07T15:07:34.753 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.377939+0000", 2020-08-07T15:07:34.753 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 42.263848201999998, 2020-08-07T15:07:34.754 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.425246179, 2020-08-07T15:07:34.754 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:34.754 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:34.755 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:34.755 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:34.755 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:34.755 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 24264 2020-08-07T15:07:34.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.756 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:34.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:34.757 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.377939+0000", 2020-08-07T15:07:34.757 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:34.757 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.757 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:34.758 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.377939+0000", 2020-08-07T15:07:34.758 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8847999999999997e-05 2020-08-07T15:07:34.758 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.758 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.758 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:34.758 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.377987+0000", 2020-08-07T15:07:34.759 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7389999999999996e-06 2020-08-07T15:07:34.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.759 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:34.759 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.377993+0000", 2020-08-07T15:07:34.760 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3533000000000001e-05 2020-08-07T15:07:34.760 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.760 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.760 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:34.760 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.378007+0000", 2020-08-07T15:07:34.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7910000000000001e-06 2020-08-07T15:07:34.761 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.761 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:34.761 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.378013+0000", 2020-08-07T15:07:34.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0046691550000000004 2020-08-07T15:07:34.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.762 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.762 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:34.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.382682+0000", 2020-08-07T15:07:34.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7563000000000003e-05 2020-08-07T15:07:34.763 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.763 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:34.763 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.382759+0000", 2020-08-07T15:07:34.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016565800000000001 2020-08-07T15:07:34.764 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.764 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:07:34.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.382925+0000", 2020-08-07T15:07:34.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026446130000000001 2020-08-07T15:07:34.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.765 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.765 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.385569+0000", 2020-08-07T15:07:34.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00070027000000000002 2020-08-07T15:07:34.766 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.766 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:34.766 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.386270+0000", 2020-08-07T15:07:34.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41678485799999998 2020-08-07T15:07:34.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.767 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.803055+0000", 2020-08-07T15:07:34.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3900000000000003e-07 2020-08-07T15:07:34.768 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.768 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.768 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:34.768 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.803056+0000", 2020-08-07T15:07:34.768 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0357000000000003e-05 2020-08-07T15:07:34.769 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.769 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.769 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:34.769 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.803136+0000", 2020-08-07T15:07:34.769 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8854999999999999e-05 2020-08-07T15:07:34.770 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.770 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.770 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:34.770 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.803185+0000", 2020-08-07T15:07:34.770 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.425246179 2020-08-07T15:07:34.771 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.771 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:34.771 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.771 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.771 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.772 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:27156 3.2 3:4fef2231:::benchmark_data_smithi074_35447_object3394:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:34.772 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:54.104778+0000", 2020-08-07T15:07:34.772 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 40.537008948999997, 2020-08-07T15:07:34.772 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21340421700000001, 2020-08-07T15:07:34.772 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:34.773 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:34.773 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:34.773 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:34.773 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:34.773 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 27156 2020-08-07T15:07:34.774 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.774 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:34.774 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.774 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:34.774 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.104778+0000", 2020-08-07T15:07:34.775 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:34.775 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.775 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:34.775 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.104778+0000", 2020-08-07T15:07:34.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1062e-05 2020-08-07T15:07:34.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:34.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.104829+0000", 2020-08-07T15:07:34.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6799999999999998e-06 2020-08-07T15:07:34.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:34.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.104835+0000", 2020-08-07T15:07:34.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.201e-05 2020-08-07T15:07:34.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:34.779 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.104847+0000", 2020-08-07T15:07:34.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7790000000000001e-06 2020-08-07T15:07:34.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.779 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:34.780 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.104852+0000", 2020-08-07T15:07:34.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00028754399999999998 2020-08-07T15:07:34.780 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.780 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:34.781 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105140+0000", 2020-08-07T15:07:34.781 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000116425 2020-08-07T15:07:34.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.782 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:34.782 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105256+0000", 2020-08-07T15:07:34.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000190729 2020-08-07T15:07:34.782 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:34.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105447+0000", 2020-08-07T15:07:34.783 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0022271819999999999 2020-08-07T15:07:34.783 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:34.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.107674+0000", 2020-08-07T15:07:34.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.285e-06 2020-08-07T15:07:34.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:34.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.107676+0000", 2020-08-07T15:07:34.785 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7856999999999998e-05 2020-08-07T15:07:34.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.786 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:34.787 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.107743+0000", 2020-08-07T15:07:34.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039706129999999996 2020-08-07T15:07:34.787 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.787 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.788 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.111714+0000", 2020-08-07T15:07:34.788 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20642095499999999 2020-08-07T15:07:34.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.788 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.789 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.318135+0000", 2020-08-07T15:07:34.789 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0100000000000001e-06 2020-08-07T15:07:34.789 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.789 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.789 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:34.790 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.318136+0000", 2020-08-07T15:07:34.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6085999999999999e-05 2020-08-07T15:07:34.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.790 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:34.791 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.318182+0000", 2020-08-07T15:07:34.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21340421700000001 2020-08-07T15:07:34.791 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.791 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:34.791 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.792 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.792 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:27157 3.2 3:4fef2231:::benchmark_data_smithi074_35447_object3394:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:34.792 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:54.105085+0000", 2020-08-07T15:07:34.792 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 40.536701721999997, 2020-08-07T15:07:34.793 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.213177318, 2020-08-07T15:07:34.793 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:34.793 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:34.793 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:34.793 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:34.794 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:34.794 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 27157 2020-08-07T15:07:34.794 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.794 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:34.794 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.795 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:34.795 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105085+0000", 2020-08-07T15:07:34.795 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:34.795 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.795 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.796 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:34.796 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105085+0000", 2020-08-07T15:07:34.796 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4802000000000001e-05 2020-08-07T15:07:34.796 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.797 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:34.797 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105140+0000", 2020-08-07T15:07:34.797 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.998e-06 2020-08-07T15:07:34.797 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.797 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.797 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:34.798 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105146+0000", 2020-08-07T15:07:34.798 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2187e-05 2020-08-07T15:07:34.798 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.798 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.798 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:34.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105158+0000", 2020-08-07T15:07:34.799 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9739999999999999e-06 2020-08-07T15:07:34.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:34.800 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105164+0000", 2020-08-07T15:07:34.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000735339 2020-08-07T15:07:34.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:34.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105899+0000", 2020-08-07T15:07:34.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0789999999999999e-05 2020-08-07T15:07:34.801 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.801 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.801 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:34.802 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105960+0000", 2020-08-07T15:07:34.802 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000114586 2020-08-07T15:07:34.802 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.802 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.802 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:34.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.106075+0000", 2020-08-07T15:07:34.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043816419999999998 2020-08-07T15:07:34.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:34.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.110456+0000", 2020-08-07T15:07:34.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5799999999999998e-07 2020-08-07T15:07:34.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:34.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.110457+0000", 2020-08-07T15:07:34.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7607e-05 2020-08-07T15:07:34.814 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.815 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:34.815 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.110505+0000", 2020-08-07T15:07:34.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012407989999999999 2020-08-07T15:07:34.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.111746+0000", 2020-08-07T15:07:34.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20647121099999999 2020-08-07T15:07:34.816 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.817 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.318217+0000", 2020-08-07T15:07:34.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7599999999999996e-07 2020-08-07T15:07:34.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.818 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:34.818 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.318218+0000", 2020-08-07T15:07:34.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4749000000000001e-05 2020-08-07T15:07:34.819 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:34.819 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.318262+0000", 2020-08-07T15:07:34.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.213177318 2020-08-07T15:07:34.820 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.820 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:34.820 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.821 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:27158 3.2 3:4fef2231:::benchmark_data_smithi074_35447_object3394:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:34.821 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:54.105200+0000", 2020-08-07T15:07:34.821 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 40.536586522, 2020-08-07T15:07:34.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21352283999999999, 2020-08-07T15:07:34.821 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:34.823 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:34.823 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:34.823 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:34.823 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:34.824 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 27158 2020-08-07T15:07:34.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.824 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:34.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:34.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105200+0000", 2020-08-07T15:07:34.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:34.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:34.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105200+0000", 2020-08-07T15:07:34.826 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8102999999999997e-05 2020-08-07T15:07:34.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:34.826 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105248+0000", 2020-08-07T15:07:34.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6820000000000002e-06 2020-08-07T15:07:34.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.827 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:34.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105253+0000", 2020-08-07T15:07:34.828 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3109999999999995e-06 2020-08-07T15:07:34.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:34.828 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105262+0000", 2020-08-07T15:07:34.828 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8519999999999997e-06 2020-08-07T15:07:34.829 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.829 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.829 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:34.829 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.105266+0000", 2020-08-07T15:07:34.829 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001222133 2020-08-07T15:07:34.831 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.831 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.831 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:34.831 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.106488+0000", 2020-08-07T15:07:34.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4035999999999995e-05 2020-08-07T15:07:34.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.832 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:34.832 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.106552+0000", 2020-08-07T15:07:34.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000117271 2020-08-07T15:07:34.833 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:34.833 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.106670+0000", 2020-08-07T15:07:34.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038427420000000001 2020-08-07T15:07:34.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:34.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.110512+0000", 2020-08-07T15:07:34.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3300000000000002e-07 2020-08-07T15:07:34.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:34.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.110513+0000", 2020-08-07T15:07:34.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4688999999999998e-05 2020-08-07T15:07:34.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.836 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:34.836 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.110568+0000", 2020-08-07T15:07:34.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001217187 2020-08-07T15:07:34.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.111785+0000", 2020-08-07T15:07:34.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20686242099999999 2020-08-07T15:07:34.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.838 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.838 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.318647+0000", 2020-08-07T15:07:34.838 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4199999999999995e-07 2020-08-07T15:07:34.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.839 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:34.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.318648+0000", 2020-08-07T15:07:34.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5238e-05 2020-08-07T15:07:34.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.840 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.840 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:34.840 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.318723+0000", 2020-08-07T15:07:34.840 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21352283999999999 2020-08-07T15:07:34.840 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.841 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:34.841 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.841 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.841 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:27159 3.2 3:4fef2231:::benchmark_data_smithi074_35447_object3394:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:34.842 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:54.106350+0000", 2020-08-07T15:07:34.842 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 40.535436607000001, 2020-08-07T15:07:34.842 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21249198999999999, 2020-08-07T15:07:34.842 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:34.842 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:34.842 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:34.843 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:34.843 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:34.843 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 27159 2020-08-07T15:07:34.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.843 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:34.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.844 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:34.844 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.106350+0000", 2020-08-07T15:07:34.844 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:34.845 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.846 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:34.846 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.106350+0000", 2020-08-07T15:07:34.846 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3527000000000003e-05 2020-08-07T15:07:34.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.846 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.847 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:34.847 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.106414+0000", 2020-08-07T15:07:34.847 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6329999999999999e-06 2020-08-07T15:07:34.847 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.847 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.848 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:34.848 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.106420+0000", 2020-08-07T15:07:34.848 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3470000000000001e-05 2020-08-07T15:07:34.848 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.848 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.848 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:34.849 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.106444+0000", 2020-08-07T15:07:34.849 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3289999999999999e-06 2020-08-07T15:07:34.849 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.849 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.850 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:34.850 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.106450+0000", 2020-08-07T15:07:34.850 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00053049499999999997 2020-08-07T15:07:34.850 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.850 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:34.851 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.106981+0000", 2020-08-07T15:07:34.851 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8999000000000002e-05 2020-08-07T15:07:34.851 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.851 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.851 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:34.852 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.107040+0000", 2020-08-07T15:07:34.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000105346 2020-08-07T15:07:34.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:34.853 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.107145+0000", 2020-08-07T15:07:34.853 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034308810000000002 2020-08-07T15:07:34.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.853 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:34.854 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.110576+0000", 2020-08-07T15:07:34.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3e-07 2020-08-07T15:07:34.854 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.854 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.855 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:34.855 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.110577+0000", 2020-08-07T15:07:34.855 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9908000000000003e-05 2020-08-07T15:07:34.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.855 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:34.856 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.110636+0000", 2020-08-07T15:07:34.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011884739999999999 2020-08-07T15:07:34.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.856 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.856 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.111825+0000", 2020-08-07T15:07:34.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.206949144 2020-08-07T15:07:34.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.858 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.318774+0000", 2020-08-07T15:07:34.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8700000000000005e-07 2020-08-07T15:07:34.858 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.858 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:34.858 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.318775+0000", 2020-08-07T15:07:34.859 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7366999999999994e-05 2020-08-07T15:07:34.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.859 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.859 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:34.859 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.318842+0000", 2020-08-07T15:07:34.860 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21249198999999999 2020-08-07T15:07:34.860 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.860 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:34.860 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.860 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.861 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:27296 3.b 3:de3f8919:::benchmark_data_smithi074_35447_object3411:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e29)", 2020-08-07T15:07:34.861 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:54.195758+0000", 2020-08-07T15:07:34.861 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 40.446028407, 2020-08-07T15:07:34.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28811428300000003, 2020-08-07T15:07:34.861 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:34.862 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:34.862 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:34.862 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:34.862 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:34.862 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 27296 2020-08-07T15:07:34.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.863 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:34.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:34.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.195758+0000", 2020-08-07T15:07:34.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:34.864 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:34.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.195758+0000", 2020-08-07T15:07:34.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9490999999999997e-05 2020-08-07T15:07:34.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.865 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:34.865 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.195818+0000", 2020-08-07T15:07:34.865 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7470000000000002e-06 2020-08-07T15:07:34.866 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.866 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.866 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:34.866 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.195824+0000", 2020-08-07T15:07:34.866 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.167e-05 2020-08-07T15:07:34.867 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.867 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.867 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:34.867 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.195835+0000", 2020-08-07T15:07:34.867 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3129999999999998e-06 2020-08-07T15:07:34.867 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.868 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.868 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:34.868 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.195841+0000", 2020-08-07T15:07:34.868 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0061607880000000004 2020-08-07T15:07:34.868 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.869 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.869 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:34.869 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.202001+0000", 2020-08-07T15:07:34.869 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6767000000000005e-05 2020-08-07T15:07:34.869 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.870 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.870 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:34.870 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.202088+0000", 2020-08-07T15:07:34.870 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000180087 2020-08-07T15:07:34.870 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.870 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.871 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:34.871 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.202268+0000", 2020-08-07T15:07:34.871 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0055746279999999999 2020-08-07T15:07:34.871 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.872 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:34.872 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.207843+0000", 2020-08-07T15:07:34.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4099999999999998e-07 2020-08-07T15:07:34.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.872 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.872 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:34.873 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.207844+0000", 2020-08-07T15:07:34.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9927999999999999e-05 2020-08-07T15:07:34.873 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.873 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:34.874 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.207914+0000", 2020-08-07T15:07:34.874 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.019254090000000001 2020-08-07T15:07:34.874 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.874 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.874 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.875 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.227168+0000", 2020-08-07T15:07:34.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.256637636 2020-08-07T15:07:34.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.483805+0000", 2020-08-07T15:07:34.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7700000000000003e-07 2020-08-07T15:07:34.876 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.876 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:34.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.483806+0000", 2020-08-07T15:07:34.877 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6519999999999993e-05 2020-08-07T15:07:34.877 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.877 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.877 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:34.878 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.483873+0000", 2020-08-07T15:07:34.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28811428300000003 2020-08-07T15:07:34.879 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.879 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:34.879 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.879 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.880 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:27296 3.b e29/20 ondisk, result = 0)", 2020-08-07T15:07:34.880 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:54.207466+0000", 2020-08-07T15:07:34.880 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 40.434321101000002, 2020-08-07T15:07:34.880 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27641121299999999, 2020-08-07T15:07:34.881 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:34.881 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:34.881 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:34.881 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.881 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:34.882 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.207466+0000", 2020-08-07T15:07:34.882 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:34.882 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.882 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.882 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:34.883 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.207466+0000", 2020-08-07T15:07:34.883 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3191999999999997e-05 2020-08-07T15:07:34.883 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.883 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.883 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:34.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.207499+0000", 2020-08-07T15:07:34.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1140000000000002e-06 2020-08-07T15:07:34.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.884 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.884 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:34.885 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.207505+0000", 2020-08-07T15:07:34.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.275903079 2020-08-07T15:07:34.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:34.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.483408+0000", 2020-08-07T15:07:34.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6809999999999997e-06 2020-08-07T15:07:34.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.886 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:34.887 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.483413+0000", 2020-08-07T15:07:34.887 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00037547699999999998 2020-08-07T15:07:34.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:34.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.483788+0000", 2020-08-07T15:07:34.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5450000000000003e-06 2020-08-07T15:07:34.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:34.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.483797+0000", 2020-08-07T15:07:34.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0124999999999996e-05 2020-08-07T15:07:34.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:34.890 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:54.483877+0000", 2020-08-07T15:07:34.890 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27641121299999999 2020-08-07T15:07:34.890 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.890 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:34.890 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.891 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.891 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:47336 3.8 3:1652c683:::benchmark_data_smithi074_35447_object5916:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e36)", 2020-08-07T15:07:34.891 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:08.384949+0000", 2020-08-07T15:07:34.891 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 26.256838054999999, 2020-08-07T15:07:34.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.232023073, 2020-08-07T15:07:34.892 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:34.892 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:34.892 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:34.892 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:34.892 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:34.893 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 47336 2020-08-07T15:07:34.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.893 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:34.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.894 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:34.894 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.384949+0000", 2020-08-07T15:07:34.894 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:34.894 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.894 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.894 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:34.895 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.384949+0000", 2020-08-07T15:07:34.895 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.545e-05 2020-08-07T15:07:34.895 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.895 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.895 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:34.896 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.384994+0000", 2020-08-07T15:07:34.896 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3240000000000002e-06 2020-08-07T15:07:34.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.896 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:34.897 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.385000+0000", 2020-08-07T15:07:34.897 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0817e-05 2020-08-07T15:07:34.897 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.897 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.897 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:34.897 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.385010+0000", 2020-08-07T15:07:34.898 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.011e-06 2020-08-07T15:07:34.898 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.898 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.898 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:34.898 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.385015+0000", 2020-08-07T15:07:34.899 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0029727579999999998 2020-08-07T15:07:34.899 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.899 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.899 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:34.899 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.387988+0000", 2020-08-07T15:07:34.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8638000000000003e-05 2020-08-07T15:07:34.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.900 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:34.900 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.388047+0000", 2020-08-07T15:07:34.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000142495 2020-08-07T15:07:34.901 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.901 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.901 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:07:34.901 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.388189+0000", 2020-08-07T15:07:34.902 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00210209 2020-08-07T15:07:34.902 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.902 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.902 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.902 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.390291+0000", 2020-08-07T15:07:34.903 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00089285899999999997 2020-08-07T15:07:34.903 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.903 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:34.903 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.391184+0000", 2020-08-07T15:07:34.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.225717942 2020-08-07T15:07:34.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.904 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.904 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.616902+0000", 2020-08-07T15:07:34.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7000000000000004e-07 2020-08-07T15:07:34.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:34.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.616903+0000", 2020-08-07T15:07:34.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0258000000000001e-05 2020-08-07T15:07:34.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.906 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.906 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:34.906 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.616933+0000", 2020-08-07T15:07:34.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8661000000000001e-05 2020-08-07T15:07:34.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:34.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.616972+0000", 2020-08-07T15:07:34.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.232023073 2020-08-07T15:07:34.908 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.908 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:34.908 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.909 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:48776 3.2 3:47029331:::benchmark_data_smithi074_35447_object6096:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e36)", 2020-08-07T15:07:34.909 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:09.403100+0000", 2020-08-07T15:07:34.909 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 25.238686874999999, 2020-08-07T15:07:34.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27318728799999997, 2020-08-07T15:07:34.910 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:34.910 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:34.910 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:34.910 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:34.911 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:34.911 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48776 2020-08-07T15:07:34.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.911 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:34.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:34.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.403100+0000", 2020-08-07T15:07:34.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:34.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:34.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.403100+0000", 2020-08-07T15:07:34.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9872e-05 2020-08-07T15:07:34.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:34.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.403150+0000", 2020-08-07T15:07:34.914 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4469999999999997e-06 2020-08-07T15:07:34.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:34.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.403155+0000", 2020-08-07T15:07:34.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7580000000000004e-06 2020-08-07T15:07:34.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:34.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.403165+0000", 2020-08-07T15:07:34.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5980000000000001e-06 2020-08-07T15:07:34.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:34.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.403169+0000", 2020-08-07T15:07:34.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000890933 2020-08-07T15:07:34.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:34.918 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.404060+0000", 2020-08-07T15:07:34.918 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000102286 2020-08-07T15:07:34.918 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.918 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:34.919 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.404162+0000", 2020-08-07T15:07:34.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018597700000000001 2020-08-07T15:07:34.919 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.919 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:34.920 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.404348+0000", 2020-08-07T15:07:34.920 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010765029999999999 2020-08-07T15:07:34.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:34.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.405424+0000", 2020-08-07T15:07:34.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9600000000000001e-07 2020-08-07T15:07:34.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:34.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.405424+0000", 2020-08-07T15:07:34.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7682e-05 2020-08-07T15:07:34.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.922 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:34.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.405452+0000", 2020-08-07T15:07:34.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045217419999999996 2020-08-07T15:07:34.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.409974+0000", 2020-08-07T15:07:34.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26623468500000003 2020-08-07T15:07:34.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.676209+0000", 2020-08-07T15:07:34.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5099999999999998e-07 2020-08-07T15:07:34.925 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:34.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.676210+0000", 2020-08-07T15:07:34.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7657999999999994e-05 2020-08-07T15:07:34.926 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.926 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:34.926 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.676287+0000", 2020-08-07T15:07:34.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27318728799999997 2020-08-07T15:07:34.927 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.927 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:34.927 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.928 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:49208 3.2 3:4d72b982:::benchmark_data_smithi074_35447_object6150:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e36)", 2020-08-07T15:07:34.928 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:09.690130+0000", 2020-08-07T15:07:34.928 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 24.951656388, 2020-08-07T15:07:34.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.237499918, 2020-08-07T15:07:34.928 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:34.929 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:34.929 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:34.929 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:34.929 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:34.929 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 49208 2020-08-07T15:07:34.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.930 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:34.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:34.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.690130+0000", 2020-08-07T15:07:34.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:34.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:34.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.690130+0000", 2020-08-07T15:07:34.931 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3445000000000003e-05 2020-08-07T15:07:34.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.932 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.932 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:34.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.690164+0000", 2020-08-07T15:07:34.932 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1869999999999999e-06 2020-08-07T15:07:34.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.933 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.933 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:34.933 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.690168+0000", 2020-08-07T15:07:34.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2430000000000005e-06 2020-08-07T15:07:34.934 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.934 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:34.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.690176+0000", 2020-08-07T15:07:34.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9579999999999998e-06 2020-08-07T15:07:34.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:34.936 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.690181+0000", 2020-08-07T15:07:34.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010947610000000001 2020-08-07T15:07:34.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:34.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.691276+0000", 2020-08-07T15:07:34.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000104084 2020-08-07T15:07:34.937 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:34.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.691380+0000", 2020-08-07T15:07:34.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020709799999999999 2020-08-07T15:07:34.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:34.940 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.691587+0000", 2020-08-07T15:07:34.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024913349999999999 2020-08-07T15:07:34.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.940 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.940 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.694079+0000", 2020-08-07T15:07:34.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0031218000000000001 2020-08-07T15:07:34.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:34.942 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.697200+0000", 2020-08-07T15:07:34.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23029393400000001 2020-08-07T15:07:34.942 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.942 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.942 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.927494+0000", 2020-08-07T15:07:34.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5799999999999998e-07 2020-08-07T15:07:34.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:34.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.927495+0000", 2020-08-07T15:07:34.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2087000000000004e-05 2020-08-07T15:07:34.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.944 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:34.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.927557+0000", 2020-08-07T15:07:34.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3127999999999995e-05 2020-08-07T15:07:34.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:34.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.927630+0000", 2020-08-07T15:07:34.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.237499918 2020-08-07T15:07:34.947 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.947 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:34.948 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.948 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:64679 3.8 3:1566a0dc:::benchmark_data_smithi074_35447_object8084:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e41)", 2020-08-07T15:07:34.948 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:20.140669+0000", 2020-08-07T15:07:34.949 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 14.501118146, 2020-08-07T15:07:34.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.212216606, 2020-08-07T15:07:34.949 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:34.949 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:34.949 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:34.950 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:34.950 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:34.950 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 64679 2020-08-07T15:07:34.950 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.950 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:34.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:34.951 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.140669+0000", 2020-08-07T15:07:34.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:34.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.952 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:34.952 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.140669+0000", 2020-08-07T15:07:34.952 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1898999999999999e-05 2020-08-07T15:07:34.952 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.952 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.953 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:34.953 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.140721+0000", 2020-08-07T15:07:34.953 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6669999999999999e-06 2020-08-07T15:07:34.953 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.953 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.955 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:34.955 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.140727+0000", 2020-08-07T15:07:34.956 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1564e-05 2020-08-07T15:07:34.956 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.956 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.956 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:34.956 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.140739+0000", 2020-08-07T15:07:34.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6749999999999999e-06 2020-08-07T15:07:34.957 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.957 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:34.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.140744+0000", 2020-08-07T15:07:34.958 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00075668199999999995 2020-08-07T15:07:34.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:34.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.141501+0000", 2020-08-07T15:07:34.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.836e-05 2020-08-07T15:07:34.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:34.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.141600+0000", 2020-08-07T15:07:34.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018370499999999999 2020-08-07T15:07:34.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:07:34.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.141783+0000", 2020-08-07T15:07:34.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0050479419999999997 2020-08-07T15:07:34.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.146831+0000", 2020-08-07T15:07:34.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00053047700000000001 2020-08-07T15:07:34.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.962 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:34.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.147362+0000", 2020-08-07T15:07:34.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20540308700000001 2020-08-07T15:07:34.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.352765+0000", 2020-08-07T15:07:34.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1999999999999998e-07 2020-08-07T15:07:34.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:34.964 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.352766+0000", 2020-08-07T15:07:34.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8413999999999999e-05 2020-08-07T15:07:34.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:34.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.352834+0000", 2020-08-07T15:07:34.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1313999999999997e-05 2020-08-07T15:07:34.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:34.966 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.352885+0000", 2020-08-07T15:07:34.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.212216606 2020-08-07T15:07:34.967 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.967 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:34.967 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.968 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:71720 3.8 3:131e443b:::benchmark_data_smithi074_35447_object8964:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e43)", 2020-08-07T15:07:34.968 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:24.674770+0000", 2020-08-07T15:07:34.968 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 9.9670169239999993, 2020-08-07T15:07:34.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26101218100000001, 2020-08-07T15:07:34.968 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:34.968 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:34.969 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:34.969 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:34.969 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:34.969 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 71720 2020-08-07T15:07:34.969 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.970 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:34.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.970 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:34.970 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.674770+0000", 2020-08-07T15:07:34.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:34.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:34.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.674770+0000", 2020-08-07T15:07:34.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7911999999999998e-05 2020-08-07T15:07:34.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:34.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.674818+0000", 2020-08-07T15:07:34.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4269999999999999e-06 2020-08-07T15:07:34.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:34.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.674823+0000", 2020-08-07T15:07:34.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1259e-05 2020-08-07T15:07:34.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:34.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.674834+0000", 2020-08-07T15:07:34.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9470000000000002e-06 2020-08-07T15:07:34.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.975 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:34.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.674839+0000", 2020-08-07T15:07:34.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0031316320000000001 2020-08-07T15:07:34.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:34.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.677971+0000", 2020-08-07T15:07:34.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5804999999999995e-05 2020-08-07T15:07:34.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:34.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.678067+0000", 2020-08-07T15:07:34.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017964599999999999 2020-08-07T15:07:34.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:07:34.978 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.678247+0000", 2020-08-07T15:07:34.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00082650400000000004 2020-08-07T15:07:34.979 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.979 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.979 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:34.979 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.679073+0000", 2020-08-07T15:07:34.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2399999999999997e-07 2020-08-07T15:07:34.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:34.980 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.679074+0000", 2020-08-07T15:07:34.980 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8667000000000001e-05 2020-08-07T15:07:34.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.981 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:34.981 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.679112+0000", 2020-08-07T15:07:34.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.027758574000000001 2020-08-07T15:07:34.982 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.982 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.983 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.706871+0000", 2020-08-07T15:07:34.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22885607399999999 2020-08-07T15:07:34.983 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.983 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.983 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:34.984 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.935727+0000", 2020-08-07T15:07:34.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0860000000000001e-06 2020-08-07T15:07:34.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.984 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:34.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.935728+0000", 2020-08-07T15:07:34.985 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3823999999999998e-05 2020-08-07T15:07:34.985 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.985 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:34.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.935782+0000", 2020-08-07T15:07:34.986 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26101218100000001 2020-08-07T15:07:34.986 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.986 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:34.986 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:34.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.987 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:71720 3.8 e43/20 ondisk, result = 0)", 2020-08-07T15:07:34.987 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:24.681833+0000", 2020-08-07T15:07:34.987 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 9.9599537110000007, 2020-08-07T15:07:34.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.253953653, 2020-08-07T15:07:34.988 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:34.988 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:34.988 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:34.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:34.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.681833+0000", 2020-08-07T15:07:34.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:34.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:34.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.681833+0000", 2020-08-07T15:07:34.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2709000000000001e-05 2020-08-07T15:07:34.990 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.990 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.990 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:34.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.681866+0000", 2020-08-07T15:07:34.991 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6409999999999999e-06 2020-08-07T15:07:34.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:34.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.681871+0000", 2020-08-07T15:07:34.991 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25379821800000002 2020-08-07T15:07:34.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:34.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.935670+0000", 2020-08-07T15:07:34.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7799999999999998e-06 2020-08-07T15:07:34.993 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.993 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.993 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:34.993 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.935673+0000", 2020-08-07T15:07:34.993 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5441000000000002e-05 2020-08-07T15:07:34.994 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:34.994 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:34.994 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:34.994 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.935709+0000", 2020-08-07T15:07:34.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9549999999999998e-06 2020-08-07T15:07:34.995 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.002 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.002 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:35.002 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.935718+0000", 2020-08-07T15:07:35.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8909e-05 2020-08-07T15:07:35.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:35.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.935787+0000", 2020-08-07T15:07:35.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.253953653 2020-08-07T15:07:35.004 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:35.004 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:35.004 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:35.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.005 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:80120 3.2 3:44f11057:::benchmark_data_smithi074_35447_object10014:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e47)", 2020-08-07T15:07:35.005 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:30.095082+0000", 2020-08-07T15:07:35.005 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4.546704847, 2020-08-07T15:07:35.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25415911299999999, 2020-08-07T15:07:35.006 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:35.006 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:35.006 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:35.006 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:35.006 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:35.007 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 80120 2020-08-07T15:07:35.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.007 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:35.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:35.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.095082+0000", 2020-08-07T15:07:35.008 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:35.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:35.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.095082+0000", 2020-08-07T15:07:35.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0065999999999998e-05 2020-08-07T15:07:35.009 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.009 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.009 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:35.010 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.095132+0000", 2020-08-07T15:07:35.010 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2530000000000004e-06 2020-08-07T15:07:35.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.010 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:35.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.095137+0000", 2020-08-07T15:07:35.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0429e-05 2020-08-07T15:07:35.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.011 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:35.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.095148+0000", 2020-08-07T15:07:35.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4599999999999996e-06 2020-08-07T15:07:35.012 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.012 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:35.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.095152+0000", 2020-08-07T15:07:35.013 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0047980260000000004 2020-08-07T15:07:35.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.013 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:35.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.099950+0000", 2020-08-07T15:07:35.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000106982 2020-08-07T15:07:35.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:35.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.100057+0000", 2020-08-07T15:07:35.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016820499999999999 2020-08-07T15:07:35.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:35.026 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.100225+0000", 2020-08-07T15:07:35.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0027400089999999998 2020-08-07T15:07:35.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:35.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.102965+0000", 2020-08-07T15:07:35.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6199999999999997e-07 2020-08-07T15:07:35.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:35.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.102966+0000", 2020-08-07T15:07:35.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8827999999999999e-05 2020-08-07T15:07:35.030 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.030 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:35.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.103035+0000", 2020-08-07T15:07:35.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0025656839999999999 2020-08-07T15:07:35.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:35.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.105601+0000", 2020-08-07T15:07:35.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.243580142 2020-08-07T15:07:35.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.034 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:35.034 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.349181+0000", 2020-08-07T15:07:35.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9500000000000001e-07 2020-08-07T15:07:35.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:35.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.349182+0000", 2020-08-07T15:07:35.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9472000000000003e-05 2020-08-07T15:07:35.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:35.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.349241+0000", 2020-08-07T15:07:35.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25415911299999999 2020-08-07T15:07:35.036 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:35.037 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:35.037 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:35.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.037 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:81543 3.7 3:e9bb655d:::benchmark_data_smithi074_35447_object10192:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e48)", 2020-08-07T15:07:35.038 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:30.892746+0000", 2020-08-07T15:07:35.038 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 3.749040355, 2020-08-07T15:07:35.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21284497899999999, 2020-08-07T15:07:35.050 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:35.051 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:35.051 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:35.054 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:35.054 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:35.054 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 81543 2020-08-07T15:07:35.055 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.055 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:35.055 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:35.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.892746+0000", 2020-08-07T15:07:35.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:35.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:35.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.892746+0000", 2020-08-07T15:07:35.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.774e-05 2020-08-07T15:07:35.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:35.057 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.892794+0000", 2020-08-07T15:07:35.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2540000000000003e-06 2020-08-07T15:07:35.058 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.058 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.058 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:35.058 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.892798+0000", 2020-08-07T15:07:35.058 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0851e-05 2020-08-07T15:07:35.059 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.059 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.059 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:35.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.892809+0000", 2020-08-07T15:07:35.059 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.101e-06 2020-08-07T15:07:35.059 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:35.060 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.892814+0000", 2020-08-07T15:07:35.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006876877 2020-08-07T15:07:35.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.061 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:35.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.899691+0000", 2020-08-07T15:07:35.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6452000000000006e-05 2020-08-07T15:07:35.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.062 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.062 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:35.062 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.899768+0000", 2020-08-07T15:07:35.062 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012544300000000001 2020-08-07T15:07:35.062 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.063 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:07:35.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.899893+0000", 2020-08-07T15:07:35.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024793129999999999 2020-08-07T15:07:35.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.064 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:35.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.902372+0000", 2020-08-07T15:07:35.065 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0029778109999999999 2020-08-07T15:07:35.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.065 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:35.065 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.905350+0000", 2020-08-07T15:07:35.066 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20016350499999999 2020-08-07T15:07:35.066 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.066 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.066 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:35.067 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.105514+0000", 2020-08-07T15:07:35.067 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3349999999999999e-06 2020-08-07T15:07:35.067 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.067 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.067 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:35.068 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.105515+0000", 2020-08-07T15:07:35.068 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3166e-05 2020-08-07T15:07:35.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.068 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.068 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:35.069 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.105558+0000", 2020-08-07T15:07:35.069 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3130999999999998e-05 2020-08-07T15:07:35.070 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.070 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.070 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:35.070 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.105591+0000", 2020-08-07T15:07:35.070 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21284497899999999 2020-08-07T15:07:35.071 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:35.071 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:35.071 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:35.071 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.071 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.072 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:81544 3.7 3:e9bb655d:::benchmark_data_smithi074_35447_object10192:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e48)", 2020-08-07T15:07:35.072 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:30.894388+0000", 2020-08-07T15:07:35.072 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 3.7473986840000002, 2020-08-07T15:07:35.072 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.219041878, 2020-08-07T15:07:35.072 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:35.073 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:35.073 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:35.073 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:35.073 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:35.073 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 81544 2020-08-07T15:07:35.074 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.074 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:35.074 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.075 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:35.075 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.894388+0000", 2020-08-07T15:07:35.075 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:35.075 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.075 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.076 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:35.076 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.894388+0000", 2020-08-07T15:07:35.076 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0058e-05 2020-08-07T15:07:35.076 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.077 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.077 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:35.077 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.894448+0000", 2020-08-07T15:07:35.077 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5119999999999997e-06 2020-08-07T15:07:35.077 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.078 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.085 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:35.085 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.894455+0000", 2020-08-07T15:07:35.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3634e-05 2020-08-07T15:07:35.086 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.086 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.086 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:35.086 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.894468+0000", 2020-08-07T15:07:35.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4309999999999999e-06 2020-08-07T15:07:35.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:35.087 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.894475+0000", 2020-08-07T15:07:35.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0068177489999999997 2020-08-07T15:07:35.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:35.095 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.901292+0000", 2020-08-07T15:07:35.095 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4939999999999996e-05 2020-08-07T15:07:35.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.096 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.096 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:35.096 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.901387+0000", 2020-08-07T15:07:35.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00022803400000000001 2020-08-07T15:07:35.096 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.097 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.097 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:07:35.097 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.901615+0000", 2020-08-07T15:07:35.097 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013569739999999999 2020-08-07T15:07:35.097 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.098 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.098 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:35.098 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.902972+0000", 2020-08-07T15:07:35.098 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002387962 2020-08-07T15:07:35.098 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.106 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.106 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:35.106 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.905360+0000", 2020-08-07T15:07:35.106 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20792416599999999 2020-08-07T15:07:35.107 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.107 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:35.114 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.113285+0000", 2020-08-07T15:07:35.114 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0500000000000003e-07 2020-08-07T15:07:35.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.115 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:35.115 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.113285+0000", 2020-08-07T15:07:35.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.4794000000000002e-05 2020-08-07T15:07:35.115 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.116 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.116 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:35.116 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.113370+0000", 2020-08-07T15:07:35.117 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9919e-05 2020-08-07T15:07:35.117 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:35.117 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:35.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:35.124 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.113430+0000", 2020-08-07T15:07:35.124 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.219041878 2020-08-07T15:07:35.124 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:35.125 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:35.125 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:35.125 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:35.125 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:35.125 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:35.154 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:07:35.155 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:35.140+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:35.243 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:35.242+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:35.252 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 59 16 87518 87502 11.5856 7.14844 0.0125589 0.0107827 2020-08-07T15:07:35.258 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.4 to 1 (10000) 2020-08-07T15:07:35.269 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 5 1 2020-08-07T15:07:35.319 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:35.319 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:35.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.320 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:87583 3.3 3:cec40526:::benchmark_data_smithi074_35447_object10947:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e53)", 2020-08-07T15:07:35.320 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:35.294924+0000", 2020-08-07T15:07:35.320 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.02300054, 2020-08-07T15:07:35.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023020572, 2020-08-07T15:07:35.321 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:35.321 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:35.321 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:35.321 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:35.321 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:35.334 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 87583 2020-08-07T15:07:35.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.335 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:35.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:35.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.294924+0000", 2020-08-07T15:07:35.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:35.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:35.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.294924+0000", 2020-08-07T15:07:35.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3062999999999997e-05 2020-08-07T15:07:35.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:35.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.294978+0000", 2020-08-07T15:07:35.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6470000000000001e-06 2020-08-07T15:07:35.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:35.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.294984+0000", 2020-08-07T15:07:35.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4249e-05 2020-08-07T15:07:35.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:35.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.294998+0000", 2020-08-07T15:07:35.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4810000000000003e-06 2020-08-07T15:07:35.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:35.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.295005+0000", 2020-08-07T15:07:35.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003774379 2020-08-07T15:07:35.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:35.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.298779+0000", 2020-08-07T15:07:35.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7485999999999997e-05 2020-08-07T15:07:35.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:35.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.298847+0000", 2020-08-07T15:07:35.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1218999999999995e-05 2020-08-07T15:07:35.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,6", 2020-08-07T15:07:35.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.298938+0000", 2020-08-07T15:07:35.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001615666 2020-08-07T15:07:35.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:35.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.300554+0000", 2020-08-07T15:07:35.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2500000000000005e-07 2020-08-07T15:07:35.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:35.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.300554+0000", 2020-08-07T15:07:35.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8600999999999997e-05 2020-08-07T15:07:35.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:35.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.300613+0000", 2020-08-07T15:07:35.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005688416 2020-08-07T15:07:35.349 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:35.349 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:35.349 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:35.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.350 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:87584 3.3 3:cec40526:::benchmark_data_smithi074_35447_object10947:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e53)", 2020-08-07T15:07:35.350 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:35.295397+0000", 2020-08-07T15:07:35.350 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.022528339000000001, 2020-08-07T15:07:35.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.02260993, 2020-08-07T15:07:35.351 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:35.351 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:35.351 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:35.351 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:35.352 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:35.352 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 87584 2020-08-07T15:07:35.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.352 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:35.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:35.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.295397+0000", 2020-08-07T15:07:35.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:35.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:35.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.295397+0000", 2020-08-07T15:07:35.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1400999999999998e-05 2020-08-07T15:07:35.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:35.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.295448+0000", 2020-08-07T15:07:35.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2659999999999998e-06 2020-08-07T15:07:35.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:35.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.295454+0000", 2020-08-07T15:07:35.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2421e-05 2020-08-07T15:07:35.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:35.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.295467+0000", 2020-08-07T15:07:35.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1029999999999998e-06 2020-08-07T15:07:35.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:35.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.295473+0000", 2020-08-07T15:07:35.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038432309999999999 2020-08-07T15:07:35.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:35.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.299316+0000", 2020-08-07T15:07:35.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3503000000000004e-05 2020-08-07T15:07:35.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:35.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.299390+0000", 2020-08-07T15:07:35.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0365000000000006e-05 2020-08-07T15:07:35.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,6", 2020-08-07T15:07:35.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.299480+0000", 2020-08-07T15:07:35.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028929889999999999 2020-08-07T15:07:35.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:35.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.302373+0000", 2020-08-07T15:07:35.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6700000000000002e-07 2020-08-07T15:07:35.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:35.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.302374+0000", 2020-08-07T15:07:35.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7267000000000005e-05 2020-08-07T15:07:35.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:35.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.302441+0000", 2020-08-07T15:07:35.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070444130000000002 2020-08-07T15:07:35.374 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:35.374 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:35.374 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:35.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.375 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:87576 3.9 3:93d8a7c1:::benchmark_data_smithi074_35447_object10946:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e53)", 2020-08-07T15:07:35.375 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:35.290279+0000", 2020-08-07T15:07:35.375 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.027646041999999999, 2020-08-07T15:07:35.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.027784092999999999, 2020-08-07T15:07:35.376 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:35.376 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:35.376 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:35.376 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:35.376 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:35.377 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 87576 2020-08-07T15:07:35.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.377 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:35.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:35.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.290279+0000", 2020-08-07T15:07:35.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:35.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:35.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.290279+0000", 2020-08-07T15:07:35.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5337000000000003e-05 2020-08-07T15:07:35.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:35.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.290324+0000", 2020-08-07T15:07:35.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0119999999999996e-06 2020-08-07T15:07:35.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:35.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.290329+0000", 2020-08-07T15:07:35.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0373e-05 2020-08-07T15:07:35.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:35.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.290340+0000", 2020-08-07T15:07:35.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2399999999999998e-06 2020-08-07T15:07:35.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:35.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.290345+0000", 2020-08-07T15:07:35.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032059789999999999 2020-08-07T15:07:35.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:35.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.293551+0000", 2020-08-07T15:07:35.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2892000000000004e-05 2020-08-07T15:07:35.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:35.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.293644+0000", 2020-08-07T15:07:35.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000159307 2020-08-07T15:07:35.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:35.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.293803+0000", 2020-08-07T15:07:35.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00094019999999999998 2020-08-07T15:07:35.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:35.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.294743+0000", 2020-08-07T15:07:35.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4099999999999999e-07 2020-08-07T15:07:35.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:35.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.294744+0000", 2020-08-07T15:07:35.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7828999999999999e-05 2020-08-07T15:07:35.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:35.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.294792+0000", 2020-08-07T15:07:35.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000673421 2020-08-07T15:07:35.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:35.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.295465+0000", 2020-08-07T15:07:35.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051861310000000001 2020-08-07T15:07:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:87617 3.6 e53/26 3:63b5ce00:::benchmark_data_smithi074_35447_object10952:head v 53'5937, mlcod=53'5933)", 2020-08-07T15:07:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:35.313281+0000", 2020-08-07T15:07:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0046441119999999997, 2020-08-07T15:07:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048466400000000001, 2020-08-07T15:07:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.313281+0000", 2020-08-07T15:07:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.313281+0000", 2020-08-07T15:07:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4790000000000002e-05 2020-08-07T15:07:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:35.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.313336+0000", 2020-08-07T15:07:35.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3039999999999999e-06 2020-08-07T15:07:35.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:35.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.313343+0000", 2020-08-07T15:07:35.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017971809999999999 2020-08-07T15:07:35.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:35.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.315140+0000", 2020-08-07T15:07:35.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3139999999999998e-06 2020-08-07T15:07:35.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:35.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.315147+0000", 2020-08-07T15:07:35.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1557000000000002e-05 2020-08-07T15:07:35.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:35.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.315189+0000", 2020-08-07T15:07:35.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3978999999999999e-05 2020-08-07T15:07:35.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:35.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.315213+0000", 2020-08-07T15:07:35.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001932125 2020-08-07T15:07:35.407 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:35.407 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:35.407 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:35.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.407 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:87618 3.6 e53/26 3:63b5ce00:::benchmark_data_smithi074_35447_object10952:head v 53'5938, mlcod=53'5933)", 2020-08-07T15:07:35.408 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:35.315050+0000", 2020-08-07T15:07:35.408 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.002874503, 2020-08-07T15:07:35.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031107109999999999, 2020-08-07T15:07:35.408 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:35.408 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:35.409 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:35.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:35.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.315050+0000", 2020-08-07T15:07:35.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:35.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:35.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.315050+0000", 2020-08-07T15:07:35.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5491000000000001e-05 2020-08-07T15:07:35.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:35.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.315106+0000", 2020-08-07T15:07:35.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5980000000000002e-06 2020-08-07T15:07:35.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:35.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.315113+0000", 2020-08-07T15:07:35.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00035602099999999999 2020-08-07T15:07:35.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:35.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.315469+0000", 2020-08-07T15:07:35.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1170000000000003e-06 2020-08-07T15:07:35.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:35.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.315474+0000", 2020-08-07T15:07:35.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00053659300000000003 2020-08-07T15:07:35.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:35.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.316010+0000", 2020-08-07T15:07:35.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3447000000000001e-05 2020-08-07T15:07:35.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:35.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.316034+0000", 2020-08-07T15:07:35.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00098326699999999991 2020-08-07T15:07:35.419 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:35.420 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:35.420 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:35.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.420 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:87619 3.6 e53/26 3:63b5ce00:::benchmark_data_smithi074_35447_object10952:head v 53'5939, mlcod=53'5933)", 2020-08-07T15:07:35.421 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:35.315389+0000", 2020-08-07T15:07:35.421 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0025359979999999998, 2020-08-07T15:07:35.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002805648, 2020-08-07T15:07:35.421 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:35.421 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:35.431 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:35.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:35.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.315389+0000", 2020-08-07T15:07:35.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:35.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:35.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.315389+0000", 2020-08-07T15:07:35.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9076999999999998e-05 2020-08-07T15:07:35.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:35.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.315438+0000", 2020-08-07T15:07:35.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.409e-06 2020-08-07T15:07:35.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:35.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.315444+0000", 2020-08-07T15:07:35.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002151334 2020-08-07T15:07:35.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:35.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.317596+0000", 2020-08-07T15:07:35.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8789999999999996e-06 2020-08-07T15:07:35.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:35.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.317600+0000", 2020-08-07T15:07:35.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9462e-05 2020-08-07T15:07:35.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:35.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.317629+0000", 2020-08-07T15:07:35.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2889e-05 2020-08-07T15:07:35.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:35.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.317642+0000", 2020-08-07T15:07:35.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022530499999999999 2020-08-07T15:07:35.460 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:35.460 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:35.460 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:35.461 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:35.461 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:07:35.461 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 6 2020-08-07T15:07:35.461 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:35.463 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:07:35.464 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:35.462+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:35.590 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:35.582+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:35.682 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:35.683 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:07:35.683 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:07:35.683 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:07:35.684 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:35.688 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:35.688+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:35.692 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:07:35.791 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:35.790+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:35.901 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:35.900+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:35.922 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:35.934 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:07:35.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:07:35.940 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:35.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.940 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:8112 3.9 3:9b275d48:::benchmark_data_smithi074_35447_object1013:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:35.941 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.083981+0000", 2020-08-07T15:07:35.941 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.834658245999996, 2020-08-07T15:07:35.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31911657999999998, 2020-08-07T15:07:35.942 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:35.942 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:35.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:35.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:35.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:35.944 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 8112 2020-08-07T15:07:35.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.944 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:35.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:35.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.083981+0000", 2020-08-07T15:07:35.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:35.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:35.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.083981+0000", 2020-08-07T15:07:35.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4051999999999999e-05 2020-08-07T15:07:35.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:35.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084015+0000", 2020-08-07T15:07:35.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7290000000000002e-06 2020-08-07T15:07:35.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:35.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084019+0000", 2020-08-07T15:07:35.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2280000000000001e-06 2020-08-07T15:07:35.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:35.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084026+0000", 2020-08-07T15:07:35.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7780000000000001e-06 2020-08-07T15:07:35.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:35.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084030+0000", 2020-08-07T15:07:35.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00077372300000000003 2020-08-07T15:07:35.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:35.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084803+0000", 2020-08-07T15:07:35.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1502000000000003e-05 2020-08-07T15:07:35.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:35.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084855+0000", 2020-08-07T15:07:35.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5110000000000002e-05 2020-08-07T15:07:35.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:35.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084950+0000", 2020-08-07T15:07:35.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012531370000000001 2020-08-07T15:07:35.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:35.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.086203+0000", 2020-08-07T15:07:35.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4799999999999998e-07 2020-08-07T15:07:35.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:35.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.086204+0000", 2020-08-07T15:07:35.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.905e-05 2020-08-07T15:07:35.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:35.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.086253+0000", 2020-08-07T15:07:35.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00062940999999999995 2020-08-07T15:07:35.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:35.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.086882+0000", 2020-08-07T15:07:35.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31612944199999998 2020-08-07T15:07:35.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:35.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.403012+0000", 2020-08-07T15:07:35.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9099999999999991e-07 2020-08-07T15:07:35.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:35.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.403013+0000", 2020-08-07T15:07:35.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.488e-05 2020-08-07T15:07:35.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:35.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.403097+0000", 2020-08-07T15:07:35.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31911657999999998 2020-08-07T15:07:35.986 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:35.986 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:35.986 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:35.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.987 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:16223 3.9 3:9c72cb85:::benchmark_data_smithi074_35447_object2027:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:35.987 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:47.106937+0000", 2020-08-07T15:07:35.987 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.811701640000003, 2020-08-07T15:07:35.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.48645846999999998, 2020-08-07T15:07:35.988 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:35.988 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:35.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:35.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:35.989 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:35.989 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 16223 2020-08-07T15:07:35.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:35.999 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:35.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:35.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:35.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.106937+0000", 2020-08-07T15:07:36.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:36.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:36.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.106937+0000", 2020-08-07T15:07:36.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8390999999999998e-05 2020-08-07T15:07:36.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:36.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.106986+0000", 2020-08-07T15:07:36.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0120000000000002e-06 2020-08-07T15:07:36.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:36.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.106992+0000", 2020-08-07T15:07:36.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3148e-05 2020-08-07T15:07:36.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:36.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.107005+0000", 2020-08-07T15:07:36.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5719999999999999e-06 2020-08-07T15:07:36.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:36.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.107012+0000", 2020-08-07T15:07:36.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043495110000000003 2020-08-07T15:07:36.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:36.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.111361+0000", 2020-08-07T15:07:36.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0105999999999997e-05 2020-08-07T15:07:36.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:36.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.111421+0000", 2020-08-07T15:07:36.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012232399999999999 2020-08-07T15:07:36.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:36.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.111544+0000", 2020-08-07T15:07:36.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003099367 2020-08-07T15:07:36.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:36.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.114643+0000", 2020-08-07T15:07:36.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3099999999999997e-07 2020-08-07T15:07:36.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:36.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.114644+0000", 2020-08-07T15:07:36.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7627000000000003e-05 2020-08-07T15:07:36.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:36.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.114691+0000", 2020-08-07T15:07:36.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005765905 2020-08-07T15:07:36.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:36.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.120457+0000", 2020-08-07T15:07:36.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.47285765200000002 2020-08-07T15:07:36.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:36.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.593315+0000", 2020-08-07T15:07:36.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1969999999999999e-06 2020-08-07T15:07:36.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:36.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.593316+0000", 2020-08-07T15:07:36.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0026999999999998e-05 2020-08-07T15:07:36.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:36.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.593396+0000", 2020-08-07T15:07:36.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.48645846999999998 2020-08-07T15:07:36.056 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.057 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:36.057 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.057 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:16224 3.9 3:9c72cb85:::benchmark_data_smithi074_35447_object2027:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:36.058 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:47.108015+0000", 2020-08-07T15:07:36.058 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.810623903, 2020-08-07T15:07:36.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.512944332, 2020-08-07T15:07:36.058 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:36.058 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:36.060 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:36.060 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:36.060 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:36.061 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 16224 2020-08-07T15:07:36.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.062 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:36.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:36.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108015+0000", 2020-08-07T15:07:36.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:36.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:36.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108015+0000", 2020-08-07T15:07:36.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7508e-05 2020-08-07T15:07:36.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:36.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108063+0000", 2020-08-07T15:07:36.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.485e-06 2020-08-07T15:07:36.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:36.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108068+0000", 2020-08-07T15:07:36.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3387000000000001e-05 2020-08-07T15:07:36.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:36.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108082+0000", 2020-08-07T15:07:36.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1129999999999997e-06 2020-08-07T15:07:36.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:36.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108088+0000", 2020-08-07T15:07:36.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056051920000000002 2020-08-07T15:07:36.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:36.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.113693+0000", 2020-08-07T15:07:36.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9973000000000001e-05 2020-08-07T15:07:36.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:36.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.113783+0000", 2020-08-07T15:07:36.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001649 2020-08-07T15:07:36.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:36.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.113948+0000", 2020-08-07T15:07:36.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001023007 2020-08-07T15:07:36.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:36.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.114971+0000", 2020-08-07T15:07:36.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2600000000000001e-07 2020-08-07T15:07:36.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:36.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.114972+0000", 2020-08-07T15:07:36.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9228999999999998e-05 2020-08-07T15:07:36.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:36.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.115031+0000", 2020-08-07T15:07:36.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0063345550000000004 2020-08-07T15:07:36.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:36.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.121365+0000", 2020-08-07T15:07:36.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.49952866099999998 2020-08-07T15:07:36.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:36.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.620894+0000", 2020-08-07T15:07:36.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9299999999999996e-07 2020-08-07T15:07:36.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:36.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.620895+0000", 2020-08-07T15:07:36.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4503000000000002e-05 2020-08-07T15:07:36.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:36.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.620960+0000", 2020-08-07T15:07:36.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.512944332 2020-08-07T15:07:36.105 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.105 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:36.105 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.106 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:23896 3.a e28/20 3:55e62b6c:::benchmark_data_smithi074_35447_object2986:head v 28'1648, mlcod=28'1639)", 2020-08-07T15:07:36.106 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:52.053957+0000", 2020-08-07T15:07:36.106 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 43.864681628, 2020-08-07T15:07:36.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26202785699999998, 2020-08-07T15:07:36.109 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:36.109 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:36.109 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:36.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:36.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.053957+0000", 2020-08-07T15:07:36.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:36.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:36.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.053957+0000", 2020-08-07T15:07:36.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4489e-05 2020-08-07T15:07:36.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:36.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.054012+0000", 2020-08-07T15:07:36.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.314e-06 2020-08-07T15:07:36.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:36.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.054018+0000", 2020-08-07T15:07:36.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26134933700000001 2020-08-07T15:07:36.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:36.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.315368+0000", 2020-08-07T15:07:36.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.783e-06 2020-08-07T15:07:36.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:36.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.315371+0000", 2020-08-07T15:07:36.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0726000000000003e-05 2020-08-07T15:07:36.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:36.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.315412+0000", 2020-08-07T15:07:36.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7337999999999999e-05 2020-08-07T15:07:36.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:36.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.315429+0000", 2020-08-07T15:07:36.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051664600000000005 2020-08-07T15:07:36.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:36.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.315946+0000", 2020-08-07T15:07:36.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9224000000000003e-05 2020-08-07T15:07:36.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:36.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:52.315985+0000", 2020-08-07T15:07:36.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26202785699999998 2020-08-07T15:07:36.127 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.127 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:36.128 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.135 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:33728 3.1 3:8fce2265:::benchmark_data_smithi074_35447_object4215:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-08-07T15:07:36.136 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:58.156056+0000", 2020-08-07T15:07:36.136 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.762583501999998, 2020-08-07T15:07:36.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26861316299999999, 2020-08-07T15:07:36.136 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:36.137 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:36.137 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:36.137 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:36.137 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:36.137 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 33728 2020-08-07T15:07:36.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.138 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:36.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:36.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156056+0000", 2020-08-07T15:07:36.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:36.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:36.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156056+0000", 2020-08-07T15:07:36.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6795999999999998e-05 2020-08-07T15:07:36.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:36.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156092+0000", 2020-08-07T15:07:36.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4690000000000002e-06 2020-08-07T15:07:36.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:36.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156096+0000", 2020-08-07T15:07:36.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5270000000000004e-06 2020-08-07T15:07:36.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:36.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156102+0000", 2020-08-07T15:07:36.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3129999999999999e-06 2020-08-07T15:07:36.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:36.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156106+0000", 2020-08-07T15:07:36.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00337032 2020-08-07T15:07:36.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:36.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.159476+0000", 2020-08-07T15:07:36.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104874 2020-08-07T15:07:36.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:36.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.159581+0000", 2020-08-07T15:07:36.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016822300000000001 2020-08-07T15:07:36.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,3", 2020-08-07T15:07:36.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.159749+0000", 2020-08-07T15:07:36.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020156620000000001 2020-08-07T15:07:36.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:36.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.161765+0000", 2020-08-07T15:07:36.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1699999999999997e-07 2020-08-07T15:07:36.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:36.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.161765+0000", 2020-08-07T15:07:36.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6606e-05 2020-08-07T15:07:36.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:36.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.161822+0000", 2020-08-07T15:07:36.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0060344459999999997 2020-08-07T15:07:36.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:36.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.167857+0000", 2020-08-07T15:07:36.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25676037099999999 2020-08-07T15:07:36.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:36.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424617+0000", 2020-08-07T15:07:36.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3370000000000001e-06 2020-08-07T15:07:36.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:36.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424618+0000", 2020-08-07T15:07:36.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0501999999999999e-05 2020-08-07T15:07:36.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:36.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424669+0000", 2020-08-07T15:07:36.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26861316299999999 2020-08-07T15:07:36.161 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.161 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:36.162 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.162 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:33728 3.1 e30/26 ondisk, result = 0)", 2020-08-07T15:07:36.162 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:58.167159+0000", 2020-08-07T15:07:36.163 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.751480469999997, 2020-08-07T15:07:36.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25751397999999998, 2020-08-07T15:07:36.163 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:36.163 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:36.163 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:36.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:36.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.167159+0000", 2020-08-07T15:07:36.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:36.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:36.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.167159+0000", 2020-08-07T15:07:36.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3473999999999998e-05 2020-08-07T15:07:36.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:36.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.167192+0000", 2020-08-07T15:07:36.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1480000000000003e-06 2020-08-07T15:07:36.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:36.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.167198+0000", 2020-08-07T15:07:36.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25735533700000002 2020-08-07T15:07:36.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:36.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424554+0000", 2020-08-07T15:07:36.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2959999999999998e-06 2020-08-07T15:07:36.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:36.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424559+0000", 2020-08-07T15:07:36.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0877000000000002e-05 2020-08-07T15:07:36.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:36.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424600+0000", 2020-08-07T15:07:36.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5580000000000001e-06 2020-08-07T15:07:36.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:36.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424608+0000", 2020-08-07T15:07:36.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4289999999999996e-05 2020-08-07T15:07:36.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:36.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424673+0000", 2020-08-07T15:07:36.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25751397999999998 2020-08-07T15:07:36.187 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.187 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:36.187 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.188 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:34624 3.9 3:946cdb13:::benchmark_data_smithi074_35447_object4327:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-08-07T15:07:36.188 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:58.744745+0000", 2020-08-07T15:07:36.188 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.173894382999997, 2020-08-07T15:07:36.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28897188600000001, 2020-08-07T15:07:36.189 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:36.189 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:36.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:36.212 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:36.213 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:36.224 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 34624 2020-08-07T15:07:36.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.230 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:36.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:36.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744745+0000", 2020-08-07T15:07:36.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:36.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:36.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744745+0000", 2020-08-07T15:07:36.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1117e-05 2020-08-07T15:07:36.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:36.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744776+0000", 2020-08-07T15:07:36.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2140000000000001e-06 2020-08-07T15:07:36.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:36.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744782+0000", 2020-08-07T15:07:36.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1756999999999999e-05 2020-08-07T15:07:36.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:36.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744794+0000", 2020-08-07T15:07:36.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.674e-06 2020-08-07T15:07:36.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:36.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744800+0000", 2020-08-07T15:07:36.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012818110000000001 2020-08-07T15:07:36.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:36.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.746082+0000", 2020-08-07T15:07:36.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2163999999999997e-05 2020-08-07T15:07:36.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:36.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.746134+0000", 2020-08-07T15:07:36.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107853 2020-08-07T15:07:36.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:36.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.746242+0000", 2020-08-07T15:07:36.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00119732 2020-08-07T15:07:36.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:36.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.747440+0000", 2020-08-07T15:07:36.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5799999999999998e-07 2020-08-07T15:07:36.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:36.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.747440+0000", 2020-08-07T15:07:36.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0883000000000001e-05 2020-08-07T15:07:36.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:36.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.747481+0000", 2020-08-07T15:07:36.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000272355 2020-08-07T15:07:36.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:36.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.747754+0000", 2020-08-07T15:07:36.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28589292999999999 2020-08-07T15:07:36.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:36.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.033647+0000", 2020-08-07T15:07:36.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6600000000000005e-07 2020-08-07T15:07:36.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:36.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.033647+0000", 2020-08-07T15:07:36.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9183999999999994e-05 2020-08-07T15:07:36.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:36.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.033717+0000", 2020-08-07T15:07:36.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28897188600000001 2020-08-07T15:07:36.338 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.338 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:36.338 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.339 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:54199 3.3 3:cc40637e:::benchmark_data_smithi074_35447_object6774:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e37)", 2020-08-07T15:07:36.339 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:13.096518+0000", 2020-08-07T15:07:36.339 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.822121101, 2020-08-07T15:07:36.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.242907547, 2020-08-07T15:07:36.340 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:36.340 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:36.340 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:36.340 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:36.341 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:36.341 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 54199 2020-08-07T15:07:36.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.341 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:36.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:36.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.096518+0000", 2020-08-07T15:07:36.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:36.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:36.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.096518+0000", 2020-08-07T15:07:36.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9442e-05 2020-08-07T15:07:36.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:36.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.096547+0000", 2020-08-07T15:07:36.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9689999999999996e-06 2020-08-07T15:07:36.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:36.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.096551+0000", 2020-08-07T15:07:36.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7849999999999997e-06 2020-08-07T15:07:36.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:36.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.096557+0000", 2020-08-07T15:07:36.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.475e-06 2020-08-07T15:07:36.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:36.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.096560+0000", 2020-08-07T15:07:36.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001230412 2020-08-07T15:07:36.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:36.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.097790+0000", 2020-08-07T15:07:36.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1904999999999995e-05 2020-08-07T15:07:36.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:36.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.097882+0000", 2020-08-07T15:07:36.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000137162 2020-08-07T15:07:36.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,6", 2020-08-07T15:07:36.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.098019+0000", 2020-08-07T15:07:36.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051773610000000001 2020-08-07T15:07:36.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:36.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.103197+0000", 2020-08-07T15:07:36.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8100000000000001e-07 2020-08-07T15:07:36.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:36.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.103197+0000", 2020-08-07T15:07:36.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4860000000000001e-05 2020-08-07T15:07:36.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:36.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.103242+0000", 2020-08-07T15:07:36.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14002068500000001 2020-08-07T15:07:36.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:36.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.243263+0000", 2020-08-07T15:07:36.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.096123423 2020-08-07T15:07:36.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:36.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.339386+0000", 2020-08-07T15:07:36.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9500000000000001e-07 2020-08-07T15:07:36.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:36.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.339387+0000", 2020-08-07T15:07:36.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8291999999999999e-05 2020-08-07T15:07:36.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:36.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.339426+0000", 2020-08-07T15:07:36.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.242907547 2020-08-07T15:07:36.373 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.373 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:36.373 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.374 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:54200 3.3 3:cc40637e:::benchmark_data_smithi074_35447_object6774:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e37)", 2020-08-07T15:07:36.374 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:13.097118+0000", 2020-08-07T15:07:36.374 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.821521047000001, 2020-08-07T15:07:36.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24303496599999999, 2020-08-07T15:07:36.375 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:36.375 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:36.375 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:36.375 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:36.375 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:36.376 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 54200 2020-08-07T15:07:36.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.376 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:36.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:36.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.097118+0000", 2020-08-07T15:07:36.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:36.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:36.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.097118+0000", 2020-08-07T15:07:36.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6094999999999997e-05 2020-08-07T15:07:36.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:36.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.097174+0000", 2020-08-07T15:07:36.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0040000000000002e-06 2020-08-07T15:07:36.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:36.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.097181+0000", 2020-08-07T15:07:36.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1990999999999999e-05 2020-08-07T15:07:36.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:36.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.097193+0000", 2020-08-07T15:07:36.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3580000000000002e-06 2020-08-07T15:07:36.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:36.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.097198+0000", 2020-08-07T15:07:36.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001099661 2020-08-07T15:07:36.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:36.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.098298+0000", 2020-08-07T15:07:36.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3878999999999997e-05 2020-08-07T15:07:36.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:36.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.098352+0000", 2020-08-07T15:07:36.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104274 2020-08-07T15:07:36.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,6", 2020-08-07T15:07:36.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.098456+0000", 2020-08-07T15:07:36.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0047920360000000004 2020-08-07T15:07:36.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:36.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.103248+0000", 2020-08-07T15:07:36.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4700000000000002e-07 2020-08-07T15:07:36.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:36.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.103249+0000", 2020-08-07T15:07:36.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8408e-05 2020-08-07T15:07:36.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:36.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.103277+0000", 2020-08-07T15:07:36.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14003026499999999 2020-08-07T15:07:36.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:36.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.243307+0000", 2020-08-07T15:07:36.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.096787082999999996 2020-08-07T15:07:36.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:36.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.340095+0000", 2020-08-07T15:07:36.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7999999999999995e-07 2020-08-07T15:07:36.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:36.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.340095+0000", 2020-08-07T15:07:36.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7785000000000003e-05 2020-08-07T15:07:36.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:36.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.340153+0000", 2020-08-07T15:07:36.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24303496599999999 2020-08-07T15:07:36.394 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.395 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:36.395 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.395 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:54199 3.3 e37/20 ondisk, result = 0)", 2020-08-07T15:07:36.395 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:13.099517+0000", 2020-08-07T15:07:36.396 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.819121758000001, 2020-08-07T15:07:36.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23991633600000001, 2020-08-07T15:07:36.396 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:36.396 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:36.396 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:36.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:36.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.099517+0000", 2020-08-07T15:07:36.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:36.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:36.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.099517+0000", 2020-08-07T15:07:36.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9927999999999999e-05 2020-08-07T15:07:36.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:36.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.099547+0000", 2020-08-07T15:07:36.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4090000000000002e-06 2020-08-07T15:07:36.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:36.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.099553+0000", 2020-08-07T15:07:36.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23978838499999999 2020-08-07T15:07:36.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:36.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.339341+0000", 2020-08-07T15:07:36.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.489e-06 2020-08-07T15:07:36.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:36.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.339345+0000", 2020-08-07T15:07:36.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0859999999999999e-05 2020-08-07T15:07:36.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:36.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.339375+0000", 2020-08-07T15:07:36.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5999999999999997e-06 2020-08-07T15:07:36.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:36.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.339381+0000", 2020-08-07T15:07:36.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2664999999999998e-05 2020-08-07T15:07:36.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:36.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.339434+0000", 2020-08-07T15:07:36.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23991633600000001 2020-08-07T15:07:36.438 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.438 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:36.438 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.439 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:63008 3.3 3:c682708a:::benchmark_data_smithi074_35447_object7875:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e41)", 2020-08-07T15:07:36.439 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:18.881861+0000", 2020-08-07T15:07:36.439 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.036778549000001, 2020-08-07T15:07:36.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21975826000000001, 2020-08-07T15:07:36.439 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:36.440 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:36.440 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:36.440 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:36.440 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:36.440 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 63008 2020-08-07T15:07:36.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.441 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:36.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:36.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.881861+0000", 2020-08-07T15:07:36.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:36.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:36.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.881861+0000", 2020-08-07T15:07:36.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8640000000000002e-05 2020-08-07T15:07:36.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:36.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.881909+0000", 2020-08-07T15:07:36.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6439999999999997e-06 2020-08-07T15:07:36.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:36.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.881915+0000", 2020-08-07T15:07:36.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1005999999999999e-05 2020-08-07T15:07:36.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:36.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.881926+0000", 2020-08-07T15:07:36.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0039999999999999e-06 2020-08-07T15:07:36.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:36.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.881931+0000", 2020-08-07T15:07:36.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013591289999999999 2020-08-07T15:07:36.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:36.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.883290+0000", 2020-08-07T15:07:36.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1316999999999996e-05 2020-08-07T15:07:36.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:36.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.883351+0000", 2020-08-07T15:07:36.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6556000000000002e-05 2020-08-07T15:07:36.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,6", 2020-08-07T15:07:36.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.883448+0000", 2020-08-07T15:07:36.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033434799999999998 2020-08-07T15:07:36.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:36.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.886791+0000", 2020-08-07T15:07:36.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9800000000000003e-07 2020-08-07T15:07:36.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:36.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.886792+0000", 2020-08-07T15:07:36.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9125000000000001e-05 2020-08-07T15:07:36.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:36.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.886841+0000", 2020-08-07T15:07:36.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031574899999999999 2020-08-07T15:07:36.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:36.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.887157+0000", 2020-08-07T15:07:36.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21441148800000001 2020-08-07T15:07:36.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:36.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:19.101568+0000", 2020-08-07T15:07:36.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8700000000000004e-07 2020-08-07T15:07:36.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:36.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:19.101569+0000", 2020-08-07T15:07:36.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9536999999999997e-05 2020-08-07T15:07:36.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:36.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:19.101619+0000", 2020-08-07T15:07:36.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21975826000000001 2020-08-07T15:07:36.461 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.461 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:36.461 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.462 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:64656 3.9 3:92365776:::benchmark_data_smithi074_35447_object8081:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e41)", 2020-08-07T15:07:36.463 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:20.118422+0000", 2020-08-07T15:07:36.464 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.800217543, 2020-08-07T15:07:36.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27269932899999999, 2020-08-07T15:07:36.465 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:36.465 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:36.465 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:36.465 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:36.465 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:36.466 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 64656 2020-08-07T15:07:36.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.466 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:36.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:36.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.118422+0000", 2020-08-07T15:07:36.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:36.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:36.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.118422+0000", 2020-08-07T15:07:36.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9726e-05 2020-08-07T15:07:36.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:36.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.118451+0000", 2020-08-07T15:07:36.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3759999999999999e-06 2020-08-07T15:07:36.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:36.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.118457+0000", 2020-08-07T15:07:36.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2139999999999996e-06 2020-08-07T15:07:36.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:36.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.118465+0000", 2020-08-07T15:07:36.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2420000000000002e-06 2020-08-07T15:07:36.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:36.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.118469+0000", 2020-08-07T15:07:36.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031527040000000001 2020-08-07T15:07:36.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:36.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.121622+0000", 2020-08-07T15:07:36.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4674999999999998e-05 2020-08-07T15:07:36.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:36.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.121706+0000", 2020-08-07T15:07:36.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000185775 2020-08-07T15:07:36.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:36.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.121892+0000", 2020-08-07T15:07:36.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003359384 2020-08-07T15:07:36.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:36.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.125252+0000", 2020-08-07T15:07:36.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2099999999999995e-07 2020-08-07T15:07:36.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:36.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.125252+0000", 2020-08-07T15:07:36.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4799999999999997e-05 2020-08-07T15:07:36.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:36.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.125307+0000", 2020-08-07T15:07:36.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3406000000000003e-05 2020-08-07T15:07:36.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:36.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.125361+0000", 2020-08-07T15:07:36.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26572362599999999 2020-08-07T15:07:36.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:36.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.391084+0000", 2020-08-07T15:07:36.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6899999999999996e-07 2020-08-07T15:07:36.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:36.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.391085+0000", 2020-08-07T15:07:36.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5811e-05 2020-08-07T15:07:36.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:36.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.391121+0000", 2020-08-07T15:07:36.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27269932899999999 2020-08-07T15:07:36.488 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.488 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:36.488 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.489 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:64656 3.9 e41/20 ondisk, result = 0)", 2020-08-07T15:07:36.489 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:20.122964+0000", 2020-08-07T15:07:36.489 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.795675481, 2020-08-07T15:07:36.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26816000699999998, 2020-08-07T15:07:36.490 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:36.490 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:36.490 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:36.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:36.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.122964+0000", 2020-08-07T15:07:36.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:36.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:36.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.122964+0000", 2020-08-07T15:07:36.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8997999999999999e-05 2020-08-07T15:07:36.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:36.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.122993+0000", 2020-08-07T15:07:36.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.395e-06 2020-08-07T15:07:36.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:36.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.122998+0000", 2020-08-07T15:07:36.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.266724181 2020-08-07T15:07:36.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:36.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.389722+0000", 2020-08-07T15:07:36.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7139999999999999e-06 2020-08-07T15:07:36.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:36.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.389726+0000", 2020-08-07T15:07:36.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001342408 2020-08-07T15:07:36.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:36.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.391068+0000", 2020-08-07T15:07:36.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0597e-05 2020-08-07T15:07:36.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:36.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.391079+0000", 2020-08-07T15:07:36.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4713999999999999e-05 2020-08-07T15:07:36.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:36.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.391124+0000", 2020-08-07T15:07:36.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26816000699999998 2020-08-07T15:07:36.520 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.520 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:36.521 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.521 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:72616 3.6 e44/26 3:64e83676:::benchmark_data_smithi074_35447_object9076:head v 44'4848, mlcod=44'4843)", 2020-08-07T15:07:36.521 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:25.249399+0000", 2020-08-07T15:07:36.522 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.669240224999999, 2020-08-07T15:07:36.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27428339600000001, 2020-08-07T15:07:36.522 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:36.522 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:36.522 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:36.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:36.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.249399+0000", 2020-08-07T15:07:36.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:36.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:36.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.249399+0000", 2020-08-07T15:07:36.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0565000000000003e-05 2020-08-07T15:07:36.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:36.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.249429+0000", 2020-08-07T15:07:36.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3390000000000003e-06 2020-08-07T15:07:36.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:36.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.249434+0000", 2020-08-07T15:07:36.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27367070199999999 2020-08-07T15:07:36.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:36.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.523104+0000", 2020-08-07T15:07:36.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2100000000000001e-06 2020-08-07T15:07:36.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:36.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.523110+0000", 2020-08-07T15:07:36.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6372999999999997e-05 2020-08-07T15:07:36.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:36.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.523166+0000", 2020-08-07T15:07:36.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5656000000000001e-05 2020-08-07T15:07:36.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:36.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.523182+0000", 2020-08-07T15:07:36.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00046263499999999998 2020-08-07T15:07:36.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:36.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.523644+0000", 2020-08-07T15:07:36.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7916000000000001e-05 2020-08-07T15:07:36.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:36.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.523682+0000", 2020-08-07T15:07:36.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27428339600000001 2020-08-07T15:07:36.542 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.543 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:36.543 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.543 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:78743 3.6 e47/26 3:6151aa5d:::benchmark_data_smithi074_35447_object9842:head v 47'5303, mlcod=47'5297)", 2020-08-07T15:07:36.544 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:29.251886+0000", 2020-08-07T15:07:36.544 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.6667526989999999, 2020-08-07T15:07:36.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.267414384, 2020-08-07T15:07:36.544 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:36.544 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:36.545 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:36.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:36.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.251886+0000", 2020-08-07T15:07:36.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:36.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:36.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.251886+0000", 2020-08-07T15:07:36.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8955e-05 2020-08-07T15:07:36.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:36.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.251915+0000", 2020-08-07T15:07:36.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2549999999999999e-06 2020-08-07T15:07:36.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:36.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.251920+0000", 2020-08-07T15:07:36.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26582320300000001 2020-08-07T15:07:36.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:36.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.517743+0000", 2020-08-07T15:07:36.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.585e-06 2020-08-07T15:07:36.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:36.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.517746+0000", 2020-08-07T15:07:36.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4413000000000003e-05 2020-08-07T15:07:36.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:36.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.517791+0000", 2020-08-07T15:07:36.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3614000000000001e-05 2020-08-07T15:07:36.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:36.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.517814+0000", 2020-08-07T15:07:36.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001433915 2020-08-07T15:07:36.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:36.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.519248+0000", 2020-08-07T15:07:36.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2444000000000001e-05 2020-08-07T15:07:36.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:36.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.519301+0000", 2020-08-07T15:07:36.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.267414384 2020-08-07T15:07:36.559 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.559 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:36.559 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.562 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:78744 3.6 e47/26 3:6151aa5d:::benchmark_data_smithi074_35447_object9842:head v 47'5304, mlcod=47'5297)", 2020-08-07T15:07:36.562 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:29.252471+0000", 2020-08-07T15:07:36.562 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.6661678880000004, 2020-08-07T15:07:36.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26934327600000002, 2020-08-07T15:07:36.562 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:36.563 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:36.563 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:36.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:36.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.252471+0000", 2020-08-07T15:07:36.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:36.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:36.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.252471+0000", 2020-08-07T15:07:36.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2329e-05 2020-08-07T15:07:36.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:36.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.252524+0000", 2020-08-07T15:07:36.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0229999999999998e-06 2020-08-07T15:07:36.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:36.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.252530+0000", 2020-08-07T15:07:36.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26582335099999999 2020-08-07T15:07:36.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:36.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.518353+0000", 2020-08-07T15:07:36.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9689999999999999e-06 2020-08-07T15:07:36.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:36.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.518356+0000", 2020-08-07T15:07:36.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5710999999999997e-05 2020-08-07T15:07:36.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:36.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.518392+0000", 2020-08-07T15:07:36.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5387e-05 2020-08-07T15:07:36.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:36.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.518407+0000", 2020-08-07T15:07:36.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033617870000000002 2020-08-07T15:07:36.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:36.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.521769+0000", 2020-08-07T15:07:36.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5719000000000001e-05 2020-08-07T15:07:36.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:36.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.521814+0000", 2020-08-07T15:07:36.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26934327600000002 2020-08-07T15:07:36.575 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.575 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:36.575 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.576 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:86696 3.a e52/20 3:5058ba7c:::benchmark_data_smithi074_35447_object10836:head v 52'5704, mlcod=51'5695)", 2020-08-07T15:07:36.576 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.329685+0000", 2020-08-07T15:07:36.576 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.588954483, 2020-08-07T15:07:36.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.34216058999999999, 2020-08-07T15:07:36.577 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:36.577 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:36.577 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:36.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:36.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.329685+0000", 2020-08-07T15:07:36.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:36.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:36.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.329685+0000", 2020-08-07T15:07:36.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8568e-05 2020-08-07T15:07:36.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:36.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.329733+0000", 2020-08-07T15:07:36.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5509999999999999e-06 2020-08-07T15:07:36.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:36.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.329739+0000", 2020-08-07T15:07:36.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.337284427 2020-08-07T15:07:36.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:36.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.667023+0000", 2020-08-07T15:07:36.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7069999999999997e-06 2020-08-07T15:07:36.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:36.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.667029+0000", 2020-08-07T15:07:36.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5142999999999998e-05 2020-08-07T15:07:36.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:36.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.667074+0000", 2020-08-07T15:07:36.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7229000000000001e-05 2020-08-07T15:07:36.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:36.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.667091+0000", 2020-08-07T15:07:36.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046970060000000001 2020-08-07T15:07:36.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:36.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.671788+0000", 2020-08-07T15:07:36.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6959000000000001e-05 2020-08-07T15:07:36.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:36.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.671845+0000", 2020-08-07T15:07:36.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.34216058999999999 2020-08-07T15:07:36.589 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.589 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:36.589 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.590 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:86776 3.d e52/35 3:bfcf83b1:::benchmark_data_smithi074_35447_object10846:head v 52'5528, mlcod=52'5523)", 2020-08-07T15:07:36.590 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.389403+0000", 2020-08-07T15:07:36.590 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.529236558, 2020-08-07T15:07:36.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24959447200000001, 2020-08-07T15:07:36.591 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:36.591 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:36.591 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:36.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:36.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.389403+0000", 2020-08-07T15:07:36.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:36.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:36.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.389403+0000", 2020-08-07T15:07:36.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1182e-05 2020-08-07T15:07:36.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:36.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.389444+0000", 2020-08-07T15:07:36.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1430000000000003e-06 2020-08-07T15:07:36.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:36.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.389450+0000", 2020-08-07T15:07:36.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24890837800000001 2020-08-07T15:07:36.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:36.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.638358+0000", 2020-08-07T15:07:36.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5929999999999996e-06 2020-08-07T15:07:36.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:36.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.638364+0000", 2020-08-07T15:07:36.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8807000000000002e-05 2020-08-07T15:07:36.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:36.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.638413+0000", 2020-08-07T15:07:36.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7064e-05 2020-08-07T15:07:36.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:36.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.638440+0000", 2020-08-07T15:07:36.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050356499999999998 2020-08-07T15:07:36.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:36.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.638943+0000", 2020-08-07T15:07:36.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3739999999999997e-05 2020-08-07T15:07:36.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:36.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.638997+0000", 2020-08-07T15:07:36.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24959447200000001 2020-08-07T15:07:36.605 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.606 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:36.606 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.606 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:87903 3.6 e53/26 3:62fb2bea:::benchmark_data_smithi074_35447_object10987:head v 53'5975, mlcod=53'5967)", 2020-08-07T15:07:36.607 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:35.494954+0000", 2020-08-07T15:07:36.607 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.42368482299999999, 2020-08-07T15:07:36.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.216277147, 2020-08-07T15:07:36.609 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:36.609 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:36.609 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:36.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:36.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.494954+0000", 2020-08-07T15:07:36.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:36.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:36.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.494954+0000", 2020-08-07T15:07:36.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2147999999999998e-05 2020-08-07T15:07:36.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:36.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.494986+0000", 2020-08-07T15:07:36.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2509999999999996e-06 2020-08-07T15:07:36.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:36.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.494991+0000", 2020-08-07T15:07:36.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21534300200000001 2020-08-07T15:07:36.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:36.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.710334+0000", 2020-08-07T15:07:36.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6420000000000004e-06 2020-08-07T15:07:36.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:36.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.710339+0000", 2020-08-07T15:07:36.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1507000000000002e-05 2020-08-07T15:07:36.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:36.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.710371+0000", 2020-08-07T15:07:36.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4253999999999999e-05 2020-08-07T15:07:36.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:36.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.710385+0000", 2020-08-07T15:07:36.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00079023899999999998 2020-08-07T15:07:36.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:36.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.711175+0000", 2020-08-07T15:07:36.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6103999999999997e-05 2020-08-07T15:07:36.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:36.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.711231+0000", 2020-08-07T15:07:36.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.216277147 2020-08-07T15:07:36.638 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.638 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:36.638 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.639 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:87904 3.6 e53/26 3:62fb2bea:::benchmark_data_smithi074_35447_object10987:head v 53'5976, mlcod=53'5967)", 2020-08-07T15:07:36.639 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:35.496088+0000", 2020-08-07T15:07:36.639 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.422550857, 2020-08-07T15:07:36.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.215553139, 2020-08-07T15:07:36.640 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:36.640 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:36.640 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:36.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:36.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.496088+0000", 2020-08-07T15:07:36.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:36.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:36.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.496088+0000", 2020-08-07T15:07:36.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6458000000000002e-05 2020-08-07T15:07:36.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:36.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.496125+0000", 2020-08-07T15:07:36.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9650000000000002e-06 2020-08-07T15:07:36.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:36.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.496129+0000", 2020-08-07T15:07:36.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21477223500000001 2020-08-07T15:07:36.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:36.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.710901+0000", 2020-08-07T15:07:36.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5680000000000001e-06 2020-08-07T15:07:36.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:36.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.710905+0000", 2020-08-07T15:07:36.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6606999999999997e-05 2020-08-07T15:07:36.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:36.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.710942+0000", 2020-08-07T15:07:36.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4463e-05 2020-08-07T15:07:36.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:36.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.710957+0000", 2020-08-07T15:07:36.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00065154099999999995 2020-08-07T15:07:36.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:36.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.711608+0000", 2020-08-07T15:07:36.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3302000000000001e-05 2020-08-07T15:07:36.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:36.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.711641+0000", 2020-08-07T15:07:36.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.215553139 2020-08-07T15:07:36.664 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.664 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:36.664 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.664 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.664 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:36.665 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:36.665 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:07:36.668 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:36.143+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:36.669 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:07:36.669 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:07:36.672 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:2020-08-07T15:07:36.245142+0000 min lat: 0.000913402 max lat: 1.95066 avg lat: 0.0107532 2020-08-07T15:07:36.672 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-08-07T15:07:36.672 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 60 16 89264 89248 11.6198 13.6406 0.00854682 0.0107532 2020-08-07T15:07:36.673 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.5 to 1 (10000) 2020-08-07T15:07:36.673 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 6 1 2020-08-07T15:07:36.674 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:36.667+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:36.773 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:36.772+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:36.868 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:36.869 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:36.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.869 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:90246 3.d 3:be4bcd37:::benchmark_data_smithi074_35447_object11280:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e54)", 2020-08-07T15:07:36.870 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:36.854774+0000", 2020-08-07T15:07:36.870 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.012666396999999999, 2020-08-07T15:07:36.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012685706999999999, 2020-08-07T15:07:36.879 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:36.879 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:36.879 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:36.890 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:36.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:36.891 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 90246 2020-08-07T15:07:36.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.892 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:36.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:36.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.854774+0000", 2020-08-07T15:07:36.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:36.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:36.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.854774+0000", 2020-08-07T15:07:36.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3081000000000002e-05 2020-08-07T15:07:36.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:36.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.854827+0000", 2020-08-07T15:07:36.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1569999999999997e-06 2020-08-07T15:07:36.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:36.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.854833+0000", 2020-08-07T15:07:36.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2468e-05 2020-08-07T15:07:36.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:36.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.854846+0000", 2020-08-07T15:07:36.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8860000000000001e-06 2020-08-07T15:07:36.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:36.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.854852+0000", 2020-08-07T15:07:36.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039263329999999997 2020-08-07T15:07:36.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:36.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.858778+0000", 2020-08-07T15:07:36.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013176299999999999 2020-08-07T15:07:36.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:36.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.858910+0000", 2020-08-07T15:07:36.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023561200000000001 2020-08-07T15:07:36.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,4,6,7", 2020-08-07T15:07:36.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.859145+0000", 2020-08-07T15:07:36.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033949150000000001 2020-08-07T15:07:36.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:36.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.862540+0000", 2020-08-07T15:07:36.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00073258500000000001 2020-08-07T15:07:36.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:36.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.863273+0000", 2020-08-07T15:07:36.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000844694 2020-08-07T15:07:36.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:36.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.864118+0000", 2020-08-07T15:07:36.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4799999999999997e-07 2020-08-07T15:07:36.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:36.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.864118+0000", 2020-08-07T15:07:36.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0575999999999996e-05 2020-08-07T15:07:36.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:36.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.864199+0000", 2020-08-07T15:07:36.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026067400000000002 2020-08-07T15:07:36.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:36.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.864460+0000", 2020-08-07T15:07:36.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0096855919999999998 2020-08-07T15:07:36.929 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.929 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:36.929 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.930 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:90247 3.d 3:be4bcd37:::benchmark_data_smithi074_35447_object11280:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e54)", 2020-08-07T15:07:36.930 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:36.856680+0000", 2020-08-07T15:07:36.930 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.010760746, 2020-08-07T15:07:36.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010851655, 2020-08-07T15:07:36.930 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:36.931 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:36.931 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:36.932 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:36.932 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:36.932 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 90247 2020-08-07T15:07:36.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.933 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:36.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:36.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.856680+0000", 2020-08-07T15:07:36.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:36.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:36.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.856680+0000", 2020-08-07T15:07:36.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2026000000000006e-05 2020-08-07T15:07:36.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:36.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.856742+0000", 2020-08-07T15:07:36.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4930000000000003e-06 2020-08-07T15:07:36.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:36.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.856748+0000", 2020-08-07T15:07:36.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3162999999999999e-05 2020-08-07T15:07:36.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:36.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.856761+0000", 2020-08-07T15:07:36.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2020000000000001e-06 2020-08-07T15:07:36.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:36.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.856768+0000", 2020-08-07T15:07:36.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036048539999999998 2020-08-07T15:07:36.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:36.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.860372+0000", 2020-08-07T15:07:36.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000114753 2020-08-07T15:07:36.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:36.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.860487+0000", 2020-08-07T15:07:36.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00041930299999999999 2020-08-07T15:07:36.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,4,6,7", 2020-08-07T15:07:36.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.860906+0000", 2020-08-07T15:07:36.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023115119999999999 2020-08-07T15:07:36.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:36.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.863218+0000", 2020-08-07T15:07:36.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000205532 2020-08-07T15:07:36.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:36.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.863424+0000", 2020-08-07T15:07:36.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003518195 2020-08-07T15:07:36.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:36.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.866942+0000", 2020-08-07T15:07:36.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9999999999999996e-07 2020-08-07T15:07:36.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:36.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.866943+0000", 2020-08-07T15:07:36.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9682999999999998e-05 2020-08-07T15:07:36.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:36.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.866992+0000", 2020-08-07T15:07:36.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8621999999999995e-05 2020-08-07T15:07:36.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:36.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.867061+0000", 2020-08-07T15:07:36.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010381237999999999 2020-08-07T15:07:36.984 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.984 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:36.984 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:36.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.985 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:90248 3.d 3:be4bcd37:::benchmark_data_smithi074_35447_object11280:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e54)", 2020-08-07T15:07:36.985 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:36.856808+0000", 2020-08-07T15:07:36.986 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.010632625, 2020-08-07T15:07:36.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010788205, 2020-08-07T15:07:36.986 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:36.986 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:36.986 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:36.987 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:36.987 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:36.987 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 90248 2020-08-07T15:07:36.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.987 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:36.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:36.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.856808+0000", 2020-08-07T15:07:36.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:36.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:36.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:36.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.856808+0000", 2020-08-07T15:07:36.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6589999999999999e-05 2020-08-07T15:07:36.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:36.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:37.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.856864+0000", 2020-08-07T15:07:37.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5330000000000002e-06 2020-08-07T15:07:37.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:37.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.856870+0000", 2020-08-07T15:07:37.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0869999999999999e-05 2020-08-07T15:07:37.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:37.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.856881+0000", 2020-08-07T15:07:37.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6120000000000001e-06 2020-08-07T15:07:37.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:37.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.856885+0000", 2020-08-07T15:07:37.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0047645980000000001 2020-08-07T15:07:37.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:37.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.861650+0000", 2020-08-07T15:07:37.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117564 2020-08-07T15:07:37.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:37.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.861768+0000", 2020-08-07T15:07:37.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000212642 2020-08-07T15:07:37.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,4,6,7", 2020-08-07T15:07:37.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.861980+0000", 2020-08-07T15:07:37.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001474616 2020-08-07T15:07:37.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:37.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.863455+0000", 2020-08-07T15:07:37.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035474920000000002 2020-08-07T15:07:37.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:37.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.867002+0000", 2020-08-07T15:07:37.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4796999999999994e-05 2020-08-07T15:07:37.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:37.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.867097+0000", 2020-08-07T15:07:37.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5499999999999999e-07 2020-08-07T15:07:37.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:37.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.867097+0000", 2020-08-07T15:07:37.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1199000000000003e-05 2020-08-07T15:07:37.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:37.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.867139+0000", 2020-08-07T15:07:37.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010330868 2020-08-07T15:07:37.043 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:37.043 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:37.044 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:37.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.044 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:90244 3.d 3:be4bcd37:::benchmark_data_smithi074_35447_object11280:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e54)", 2020-08-07T15:07:37.044 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:36.852646+0000", 2020-08-07T15:07:37.045 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.014794167, 2020-08-07T15:07:37.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015009392999999999, 2020-08-07T15:07:37.045 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:37.045 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:07:37.045 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:37.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:37.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:37.046 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 90244 2020-08-07T15:07:37.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.052 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:37.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:37.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.852646+0000", 2020-08-07T15:07:37.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:37.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:37.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.852646+0000", 2020-08-07T15:07:37.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6980000000000001e-05 2020-08-07T15:07:37.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:37.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.852693+0000", 2020-08-07T15:07:37.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1540000000000001e-06 2020-08-07T15:07:37.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:37.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.852698+0000", 2020-08-07T15:07:37.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0183999999999999e-05 2020-08-07T15:07:37.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:37.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.852709+0000", 2020-08-07T15:07:37.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.279e-06 2020-08-07T15:07:37.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:37.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.852713+0000", 2020-08-07T15:07:37.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036700779999999998 2020-08-07T15:07:37.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:37.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.856383+0000", 2020-08-07T15:07:37.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011466799999999999 2020-08-07T15:07:37.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.856498+0000", 2020-08-07T15:07:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027469499999999998 2020-08-07T15:07:37.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,4,6,7", 2020-08-07T15:07:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.856772+0000", 2020-08-07T15:07:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056627489999999999 2020-08-07T15:07:37.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:37.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.862435+0000", 2020-08-07T15:07:37.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2191000000000001e-05 2020-08-07T15:07:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.862457+0000", 2020-08-07T15:07:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103071 2020-08-07T15:07:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.862560+0000", 2020-08-07T15:07:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00072353999999999995 2020-08-07T15:07:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.863284+0000", 2020-08-07T15:07:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010637588999999999 2020-08-07T15:07:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:90245 3.d 3:be4bcd37:::benchmark_data_smithi074_35447_object11280:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e54)", 2020-08-07T15:07:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:36.854495+0000", 2020-08-07T15:07:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.012945171, 2020-08-07T15:07:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013216488, 2020-08-07T15:07:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 90245 2020-08-07T15:07:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.193 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.854495+0000", 2020-08-07T15:07:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.854495+0000", 2020-08-07T15:07:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3838000000000003e-05 2020-08-07T15:07:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.854539+0000", 2020-08-07T15:07:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2519999999999996e-06 2020-08-07T15:07:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.854545+0000", 2020-08-07T15:07:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3227999999999999e-05 2020-08-07T15:07:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.854559+0000", 2020-08-07T15:07:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1340000000000001e-06 2020-08-07T15:07:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.854565+0000", 2020-08-07T15:07:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030264110000000001 2020-08-07T15:07:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.857591+0000", 2020-08-07T15:07:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121304 2020-08-07T15:07:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.857712+0000", 2020-08-07T15:07:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050046900000000002 2020-08-07T15:07:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,4,6,7", 2020-08-07T15:07:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.858213+0000", 2020-08-07T15:07:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043070469999999996 2020-08-07T15:07:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.862520+0000", 2020-08-07T15:07:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012566399999999999 2020-08-07T15:07:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.862646+0000", 2020-08-07T15:07:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00064341099999999998 2020-08-07T15:07:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.863289+0000", 2020-08-07T15:07:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9700000000000002e-07 2020-08-07T15:07:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.863289+0000", 2020-08-07T15:07:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4122999999999998e-05 2020-08-07T15:07:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.863334+0000", 2020-08-07T15:07:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011007160000000001 2020-08-07T15:07:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:36.864434+0000", 2020-08-07T15:07:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0099389939999999996 2020-08-07T15:07:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:07:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 5 2020-08-07T15:07:37.209 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:37.211 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:07:37.212 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:07:37.212 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:07:37.212 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:07:37.213 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:07:37.213 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:07:37.213 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:07:37.214 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:07:37.215 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:37.209+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:37.227 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:07:37.231 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:07:37.232 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:07:37.232 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:07:37.232 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:07:37.232 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:07:37.232 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:07:37.233 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:07:37.233 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:07:37.248 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 61 16 90762 90746 11.6212 11.7031 0.00480023 0.0107457 2020-08-07T15:07:37.317 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:37.311+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:37.415 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:37.413+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:37.445 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:37.445 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:37.462 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:07:37.551 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:37.550+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:37.618 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:37.616+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:37.695 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:37.696 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:37.710 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:07:37.726 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:37.719+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:37.824 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:37.822+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:37.928 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:37.925+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:37.931 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:37.931 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:37.944 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:07:38.040 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:38.038+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:38.132 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:38.130+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:38.155 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.6 to 1 (10000) 2020-08-07T15:07:38.160 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:38.169 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:38.170 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:07:38.172 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 7 1 2020-08-07T15:07:38.173 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:07:38.246 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 62 16 92425 92409 11.6433 12.9922 0.00450173 0.0107321 2020-08-07T15:07:38.261 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:38.257+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:38.335 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:38.333+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:38.394 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:38.396 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:38.408 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:07:38.449 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:38.448+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:38.538 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:38.536+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:38.639 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:38.637+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:38.640 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:38.641 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:38.655 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:07:38.746 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:38.741+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:38.826 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:38.826 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:38.834 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:07:38.882 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:38.880+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:38.948 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:38.945+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:39.017 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:39.017 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:39.029 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:07:39.080 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:39.072+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:39.246 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:39.247 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:39.247 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:07:39.248 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 63 16 93832 93816 11.6329 10.9922 0.00674055 0.0107409 2020-08-07T15:07:39.253 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:07:39.262 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:39.251+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:39.289 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.7 to 1 (10000) 2020-08-07T15:07:39.307 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2020-08-07T15:07:39.307 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:07:39.356 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:39.349+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:39.464 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:39.462+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:39.468 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:39.469 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:39.481 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:07:39.513 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:07:39.559 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:07:39.564 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:39.560+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:39.601 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:07:39.645 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:07:39.657 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:39.653+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:39.684 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:07:39.730 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:39.731 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:07:39.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:07:39.731 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:07:39.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.732 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:9416 3.6 3:6d5d7913:::benchmark_data_smithi074_35447_object1176:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:39.732 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.922485+0000", 2020-08-07T15:07:39.732 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.787932378999997, 2020-08-07T15:07:39.732 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16975183499999999, 2020-08-07T15:07:39.732 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:39.733 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:39.733 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:39.733 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:39.733 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:39.733 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 9416 2020-08-07T15:07:39.734 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.734 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:39.734 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.734 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:39.734 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922485+0000", 2020-08-07T15:07:39.735 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:39.735 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.735 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.735 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:39.735 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922485+0000", 2020-08-07T15:07:39.736 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6072999999999998e-05 2020-08-07T15:07:39.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.736 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.736 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:39.736 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922521+0000", 2020-08-07T15:07:39.737 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.241e-06 2020-08-07T15:07:39.737 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.737 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.737 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:39.737 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922527+0000", 2020-08-07T15:07:39.749 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2075e-05 2020-08-07T15:07:39.749 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.749 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.749 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:39.750 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922540+0000", 2020-08-07T15:07:39.750 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0589999999999999e-06 2020-08-07T15:07:39.750 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.750 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.750 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:39.751 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.922546+0000", 2020-08-07T15:07:39.751 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00063213199999999998 2020-08-07T15:07:39.751 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.751 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.751 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:39.752 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.923178+0000", 2020-08-07T15:07:39.752 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9913e-05 2020-08-07T15:07:39.752 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.752 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:39.753 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.923238+0000", 2020-08-07T15:07:39.753 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010683099999999999 2020-08-07T15:07:39.753 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.753 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.753 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:39.753 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.923344+0000", 2020-08-07T15:07:39.754 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004139663 2020-08-07T15:07:39.754 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.754 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.754 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:39.754 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.927484+0000", 2020-08-07T15:07:39.755 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001063149 2020-08-07T15:07:39.755 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.755 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:39.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.928547+0000", 2020-08-07T15:07:39.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16358086099999999 2020-08-07T15:07:39.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:39.756 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092128+0000", 2020-08-07T15:07:39.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3699999999999999e-07 2020-08-07T15:07:39.757 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.757 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:39.757 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092129+0000", 2020-08-07T15:07:39.757 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3859e-05 2020-08-07T15:07:39.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.760 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:39.760 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092193+0000", 2020-08-07T15:07:39.760 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4141999999999999e-05 2020-08-07T15:07:39.760 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.760 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.761 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:39.761 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092237+0000", 2020-08-07T15:07:39.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16975183499999999 2020-08-07T15:07:39.761 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:39.761 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:39.762 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:39.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.763 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.763 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:9416 3.6 e24/20 ondisk, result = 0)", 2020-08-07T15:07:39.763 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.926716+0000", 2020-08-07T15:07:39.763 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.783701985999997, 2020-08-07T15:07:39.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16552591899999999, 2020-08-07T15:07:39.764 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:39.764 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:39.764 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:39.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.765 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:39.765 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.926716+0000", 2020-08-07T15:07:39.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:39.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.766 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:39.766 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.926716+0000", 2020-08-07T15:07:39.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9971000000000002e-05 2020-08-07T15:07:39.766 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:39.767 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.926745+0000", 2020-08-07T15:07:39.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.51e-06 2020-08-07T15:07:39.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:39.768 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.926750+0000", 2020-08-07T15:07:39.768 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.165288254 2020-08-07T15:07:39.769 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.770 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.770 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:39.770 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092038+0000", 2020-08-07T15:07:39.770 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1109999999999999e-06 2020-08-07T15:07:39.771 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.772 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.772 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:39.772 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092043+0000", 2020-08-07T15:07:39.773 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7359000000000003e-05 2020-08-07T15:07:39.773 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.773 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.773 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:39.774 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092111+0000", 2020-08-07T15:07:39.774 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6889999999999993e-06 2020-08-07T15:07:39.774 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.774 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.774 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:39.774 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092119+0000", 2020-08-07T15:07:39.775 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000122025 2020-08-07T15:07:39.775 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.775 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:39.775 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.092241+0000", 2020-08-07T15:07:39.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16552591899999999 2020-08-07T15:07:39.776 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:39.776 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:39.776 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:39.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.777 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:10240 3.6 3:665d71ff:::benchmark_data_smithi074_35447_object1279:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:39.777 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:07:39.777 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.317198548999997, 2020-08-07T15:07:39.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25191569699999999, 2020-08-07T15:07:39.778 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:39.778 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:39.778 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:39.779 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:39.779 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:39.779 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 10240 2020-08-07T15:07:39.780 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.780 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:39.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.780 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:39.780 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:07:39.781 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:39.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.782 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.782 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:39.782 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:07:39.783 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5463000000000002e-05 2020-08-07T15:07:39.783 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:39.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393264+0000", 2020-08-07T15:07:39.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9219999999999999e-06 2020-08-07T15:07:39.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:39.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393268+0000", 2020-08-07T15:07:39.785 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6149999999999997e-06 2020-08-07T15:07:39.785 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.786 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:39.786 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393277+0000", 2020-08-07T15:07:39.786 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2039999999999996e-06 2020-08-07T15:07:39.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:39.787 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393282+0000", 2020-08-07T15:07:39.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001225243 2020-08-07T15:07:39.787 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.787 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.788 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:39.788 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394507+0000", 2020-08-07T15:07:39.788 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7108999999999995e-05 2020-08-07T15:07:39.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.789 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:39.790 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394605+0000", 2020-08-07T15:07:39.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017490800000000001 2020-08-07T15:07:39.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.790 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:39.791 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394779+0000", 2020-08-07T15:07:39.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032608699999999999 2020-08-07T15:07:39.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:39.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.398040+0000", 2020-08-07T15:07:39.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013015259999999999 2020-08-07T15:07:39.793 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.794 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.794 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:39.794 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.399342+0000", 2020-08-07T15:07:39.794 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24566893400000001 2020-08-07T15:07:39.795 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.795 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.795 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:39.795 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645011+0000", 2020-08-07T15:07:39.795 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.217e-06 2020-08-07T15:07:39.796 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.796 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:39.796 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645012+0000", 2020-08-07T15:07:39.796 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0803000000000006e-05 2020-08-07T15:07:39.797 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.797 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.797 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:39.797 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645083+0000", 2020-08-07T15:07:39.797 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1882999999999998e-05 2020-08-07T15:07:39.798 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.798 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.798 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:39.798 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645135+0000", 2020-08-07T15:07:39.799 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25191569699999999 2020-08-07T15:07:39.799 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:39.799 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:39.799 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:39.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.800 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:31760 3.6 e30/26 3:6672be19:::benchmark_data_smithi074_35447_object3969:head v 30'2104, mlcod=30'2094)", 2020-08-07T15:07:39.801 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:56.999757+0000", 2020-08-07T15:07:39.801 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 42.710660627999999, 2020-08-07T15:07:39.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17823482099999999, 2020-08-07T15:07:39.801 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:39.801 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:39.802 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:39.802 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.802 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:39.802 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:56.999757+0000", 2020-08-07T15:07:39.802 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:39.802 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:39.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:56.999757+0000", 2020-08-07T15:07:39.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2955999999999999e-05 2020-08-07T15:07:39.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:39.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:56.999810+0000", 2020-08-07T15:07:39.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5509999999999996e-06 2020-08-07T15:07:39.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:39.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:56.999816+0000", 2020-08-07T15:07:39.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.177361504 2020-08-07T15:07:39.806 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:39.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177178+0000", 2020-08-07T15:07:39.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6979999999999997e-06 2020-08-07T15:07:39.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:39.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177183+0000", 2020-08-07T15:07:39.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1974999999999997e-05 2020-08-07T15:07:39.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.808 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:39.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177225+0000", 2020-08-07T15:07:39.809 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4612000000000001e-05 2020-08-07T15:07:39.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.809 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:39.809 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177249+0000", 2020-08-07T15:07:39.809 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00069063000000000002 2020-08-07T15:07:39.810 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.810 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.811 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:39.811 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177940+0000", 2020-08-07T15:07:39.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1894999999999997e-05 2020-08-07T15:07:39.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:39.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177992+0000", 2020-08-07T15:07:39.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17823482099999999 2020-08-07T15:07:39.812 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:39.812 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:39.813 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:39.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.813 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:43335 3.6 e32/26 3:65a57ed3:::benchmark_data_smithi074_35447_object5416:head v 32'2831, mlcod=32'2822)", 2020-08-07T15:07:39.813 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:03.998850+0000", 2020-08-07T15:07:39.814 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 35.711567477999999, 2020-08-07T15:07:39.814 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.221384421, 2020-08-07T15:07:39.815 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:39.815 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:39.815 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:39.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:39.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998850+0000", 2020-08-07T15:07:39.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:39.816 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.817 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:39.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998850+0000", 2020-08-07T15:07:39.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4788000000000004e-05 2020-08-07T15:07:39.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.818 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:39.818 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998915+0000", 2020-08-07T15:07:39.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7909999999999999e-06 2020-08-07T15:07:39.818 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.818 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.818 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:39.819 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998922+0000", 2020-08-07T15:07:39.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22050499700000001 2020-08-07T15:07:39.819 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:39.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219427+0000", 2020-08-07T15:07:39.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0950000000000001e-06 2020-08-07T15:07:39.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:39.821 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219433+0000", 2020-08-07T15:07:39.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6342999999999997e-05 2020-08-07T15:07:39.821 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.822 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.822 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:39.822 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219509+0000", 2020-08-07T15:07:39.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4173999999999999e-05 2020-08-07T15:07:39.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:39.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219533+0000", 2020-08-07T15:07:39.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00064214899999999995 2020-08-07T15:07:39.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:39.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.220175+0000", 2020-08-07T15:07:39.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9083999999999999e-05 2020-08-07T15:07:39.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:39.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.220234+0000", 2020-08-07T15:07:39.826 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.221384421 2020-08-07T15:07:39.826 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:39.826 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:39.826 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:39.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.827 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:43336 3.6 e32/26 3:65a57ed3:::benchmark_data_smithi074_35447_object5416:head v 32'2832, mlcod=32'2822)", 2020-08-07T15:07:39.827 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:03.999509+0000", 2020-08-07T15:07:39.828 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 35.710908148999998, 2020-08-07T15:07:39.828 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22632864799999999, 2020-08-07T15:07:39.828 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:39.828 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:39.828 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:39.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.829 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:39.829 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999509+0000", 2020-08-07T15:07:39.829 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:39.829 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.829 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.830 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:39.830 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999509+0000", 2020-08-07T15:07:39.830 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0107e-05 2020-08-07T15:07:39.831 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.831 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.832 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:39.832 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999559+0000", 2020-08-07T15:07:39.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0889999999999996e-06 2020-08-07T15:07:39.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:39.833 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999566+0000", 2020-08-07T15:07:39.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22332094599999999 2020-08-07T15:07:39.833 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:39.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222887+0000", 2020-08-07T15:07:39.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5990000000000001e-06 2020-08-07T15:07:39.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:39.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222892+0000", 2020-08-07T15:07:39.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2342999999999998e-05 2020-08-07T15:07:39.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.836 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:39.836 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222924+0000", 2020-08-07T15:07:39.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9108999999999999e-05 2020-08-07T15:07:39.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:39.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222944+0000", 2020-08-07T15:07:39.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028409289999999999 2020-08-07T15:07:39.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.839 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:39.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.225784+0000", 2020-08-07T15:07:39.840 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3526000000000002e-05 2020-08-07T15:07:39.840 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.840 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.840 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:39.840 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.225838+0000", 2020-08-07T15:07:39.841 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22632864799999999 2020-08-07T15:07:39.841 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:39.841 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:39.841 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:39.842 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.842 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.843 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:43784 3.6 e32/26 3:6da53356:::benchmark_data_smithi074_35447_object5472:head v 32'2848, mlcod=32'2839)", 2020-08-07T15:07:39.843 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:04.253355+0000", 2020-08-07T15:07:39.843 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 35.457062577000002, 2020-08-07T15:07:39.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21456700500000001, 2020-08-07T15:07:39.843 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:39.844 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:39.844 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:39.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.844 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:39.844 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253355+0000", 2020-08-07T15:07:39.845 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:39.845 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.846 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:39.847 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253355+0000", 2020-08-07T15:07:39.847 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6168000000000002e-05 2020-08-07T15:07:39.847 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.847 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.847 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:39.848 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253401+0000", 2020-08-07T15:07:39.848 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3179999999999997e-06 2020-08-07T15:07:39.848 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.848 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.848 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:39.849 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253406+0000", 2020-08-07T15:07:39.849 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21194338600000001 2020-08-07T15:07:39.849 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.849 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.849 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:39.850 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465350+0000", 2020-08-07T15:07:39.855 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3759999999999999e-06 2020-08-07T15:07:39.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.856 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.856 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:39.856 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465355+0000", 2020-08-07T15:07:39.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8250999999999997e-05 2020-08-07T15:07:39.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:39.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465393+0000", 2020-08-07T15:07:39.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0296999999999999e-05 2020-08-07T15:07:39.858 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:39.861 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465414+0000", 2020-08-07T15:07:39.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024515209999999999 2020-08-07T15:07:39.870 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.870 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.871 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:39.871 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.467865+0000", 2020-08-07T15:07:39.871 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6687999999999997e-05 2020-08-07T15:07:39.871 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.872 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:39.872 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.467922+0000", 2020-08-07T15:07:39.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21456700500000001 2020-08-07T15:07:39.873 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:39.873 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:39.873 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:39.874 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.874 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.874 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:44152 3.6 e32/26 3:6c90cb4b:::benchmark_data_smithi074_35447_object5518:head v 32'2856, mlcod=32'2851)", 2020-08-07T15:07:39.874 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:04.473943+0000", 2020-08-07T15:07:39.874 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 35.236474755000003, 2020-08-07T15:07:39.874 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18875260999999999, 2020-08-07T15:07:39.875 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:39.875 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:39.875 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:39.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:39.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.473943+0000", 2020-08-07T15:07:39.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:39.877 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.877 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.877 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:39.877 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.473943+0000", 2020-08-07T15:07:39.878 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8267999999999999e-05 2020-08-07T15:07:39.878 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.878 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.878 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:39.878 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.473991+0000", 2020-08-07T15:07:39.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9689999999999999e-06 2020-08-07T15:07:39.879 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.882 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:39.882 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.473997+0000", 2020-08-07T15:07:39.882 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18142124000000001 2020-08-07T15:07:39.883 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.883 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.883 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:39.883 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.655418+0000", 2020-08-07T15:07:39.883 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7439999999999998e-06 2020-08-07T15:07:39.883 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.884 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.884 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:39.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.655423+0000", 2020-08-07T15:07:39.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0052999999999997e-05 2020-08-07T15:07:39.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:39.885 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.655473+0000", 2020-08-07T15:07:39.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3739000000000001e-05 2020-08-07T15:07:39.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.886 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.886 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:39.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.655497+0000", 2020-08-07T15:07:39.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0071399030000000004 2020-08-07T15:07:39.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:39.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.662637+0000", 2020-08-07T15:07:39.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8693999999999997e-05 2020-08-07T15:07:39.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:39.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.662695+0000", 2020-08-07T15:07:39.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18875260999999999 2020-08-07T15:07:39.889 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:39.889 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:39.889 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:39.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.890 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:54200 3.3 e37/20 3:cc40637e:::benchmark_data_smithi074_35447_object6774:head v 37'3248, mlcod=37'3236)", 2020-08-07T15:07:39.891 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:07:39.891 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 26.611855329000001, 2020-08-07T15:07:39.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24065872499999999, 2020-08-07T15:07:39.892 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:39.892 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:39.892 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:39.892 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.893 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:39.893 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:07:39.893 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:39.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.894 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.894 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:39.894 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:07:39.895 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1671000000000002e-05 2020-08-07T15:07:39.895 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.895 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.896 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:39.896 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098594+0000", 2020-08-07T15:07:39.896 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0890000000000002e-06 2020-08-07T15:07:39.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.897 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:39.897 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098598+0000", 2020-08-07T15:07:39.898 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23875592300000001 2020-08-07T15:07:39.898 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.898 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.899 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:39.899 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337354+0000", 2020-08-07T15:07:39.899 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0420000000000002e-06 2020-08-07T15:07:39.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.902 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:39.902 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337359+0000", 2020-08-07T15:07:39.903 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0586000000000001e-05 2020-08-07T15:07:39.903 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.903 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:39.903 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337399+0000", 2020-08-07T15:07:39.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9415999999999998e-05 2020-08-07T15:07:39.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.904 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:39.904 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337419+0000", 2020-08-07T15:07:39.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017568390000000001 2020-08-07T15:07:39.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:39.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.339176+0000", 2020-08-07T15:07:39.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5158999999999999e-05 2020-08-07T15:07:39.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:39.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.339221+0000", 2020-08-07T15:07:39.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24065872499999999 2020-08-07T15:07:39.909 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:39.909 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:39.909 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:39.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.910 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:54728 3.7 e37/20 3:ee999259:::benchmark_data_smithi074_35447_object6840:head v 37'3328, mlcod=37'3319)", 2020-08-07T15:07:39.910 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:13.452195+0000", 2020-08-07T15:07:39.910 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 26.258222547999999, 2020-08-07T15:07:39.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.187979807, 2020-08-07T15:07:39.911 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:39.911 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:39.912 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:39.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:39.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.452195+0000", 2020-08-07T15:07:39.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:39.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:39.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.452195+0000", 2020-08-07T15:07:39.914 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9023000000000001e-05 2020-08-07T15:07:39.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:39.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.452244+0000", 2020-08-07T15:07:39.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.733e-06 2020-08-07T15:07:39.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:39.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.452250+0000", 2020-08-07T15:07:39.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18453694300000001 2020-08-07T15:07:39.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:39.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.636787+0000", 2020-08-07T15:07:39.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5679999999999999e-06 2020-08-07T15:07:39.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.918 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:39.918 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.636792+0000", 2020-08-07T15:07:39.918 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0119999999999999e-05 2020-08-07T15:07:39.918 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:39.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.636872+0000", 2020-08-07T15:07:39.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6356999999999999e-05 2020-08-07T15:07:39.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.922 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:39.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.636899+0000", 2020-08-07T15:07:39.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003233737 2020-08-07T15:07:39.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:39.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.640132+0000", 2020-08-07T15:07:39.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2326000000000001e-05 2020-08-07T15:07:39.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:39.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.640175+0000", 2020-08-07T15:07:39.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.187979807 2020-08-07T15:07:39.925 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:39.926 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:39.926 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:39.926 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.926 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:64280 3.1 e41/26 3:889ec893:::benchmark_data_smithi074_35447_object8034:head v 41'3824, mlcod=41'3819)", 2020-08-07T15:07:39.927 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:19.860260+0000", 2020-08-07T15:07:39.927 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 19.850157346, 2020-08-07T15:07:39.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16225524899999999, 2020-08-07T15:07:39.927 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:39.927 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:39.928 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:39.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:39.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.860260+0000", 2020-08-07T15:07:39.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:39.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:39.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.860260+0000", 2020-08-07T15:07:39.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8782000000000001e-05 2020-08-07T15:07:39.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.946 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.946 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:39.946 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.860319+0000", 2020-08-07T15:07:39.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8560000000000007e-06 2020-08-07T15:07:39.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.947 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.947 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:39.947 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.860327+0000", 2020-08-07T15:07:39.948 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.15885376900000001 2020-08-07T15:07:39.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:39.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.019181+0000", 2020-08-07T15:07:39.958 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3629999999999999e-06 2020-08-07T15:07:39.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:39.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.019187+0000", 2020-08-07T15:07:39.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9097e-05 2020-08-07T15:07:39.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:39.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.019246+0000", 2020-08-07T15:07:39.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1652000000000001e-05 2020-08-07T15:07:39.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:39.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.019278+0000", 2020-08-07T15:07:39.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0031829330000000002 2020-08-07T15:07:39.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.962 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:39.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.022461+0000", 2020-08-07T15:07:39.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4796999999999997e-05 2020-08-07T15:07:39.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:39.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.022515+0000", 2020-08-07T15:07:39.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16225524899999999 2020-08-07T15:07:39.963 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:39.964 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:39.964 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:39.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.964 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:71136 3.6 e43/26 3:64cb7acb:::benchmark_data_smithi074_35447_object8891:head v 43'4744, mlcod=43'4739)", 2020-08-07T15:07:39.965 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:07:39.965 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 15.391803962999999, 2020-08-07T15:07:39.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26135482999999998, 2020-08-07T15:07:39.965 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:39.965 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:39.965 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:39.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:39.966 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:07:39.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:39.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:39.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:07:39.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3702999999999997e-05 2020-08-07T15:07:39.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:39.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318667+0000", 2020-08-07T15:07:39.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3119999999999999e-06 2020-08-07T15:07:39.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.969 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:39.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318674+0000", 2020-08-07T15:07:39.969 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.255173292 2020-08-07T15:07:39.969 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.969 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.970 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:39.970 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.573847+0000", 2020-08-07T15:07:39.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1140000000000002e-06 2020-08-07T15:07:39.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:39.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.573853+0000", 2020-08-07T15:07:39.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00057603799999999996 2020-08-07T15:07:39.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:39.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.574429+0000", 2020-08-07T15:07:39.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4776e-05 2020-08-07T15:07:39.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:39.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.574444+0000", 2020-08-07T15:07:39.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054577080000000004 2020-08-07T15:07:39.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.978 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:39.978 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.579901+0000", 2020-08-07T15:07:39.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6886999999999998e-05 2020-08-07T15:07:39.979 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.979 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.979 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:39.979 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.579968+0000", 2020-08-07T15:07:39.980 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26135482999999998 2020-08-07T15:07:39.980 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:39.980 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:39.990 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:39.990 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.991 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:71760 3.f e43/20 3:f213ad20:::benchmark_data_smithi074_35447_object8969:head v 43'4784, mlcod=43'4776)", 2020-08-07T15:07:39.991 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:24.699138+0000", 2020-08-07T15:07:39.991 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 15.011279485999999, 2020-08-07T15:07:39.991 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17350447599999999, 2020-08-07T15:07:39.992 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:39.992 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:39.992 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:39.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:39.993 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.699138+0000", 2020-08-07T15:07:39.993 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:39.993 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.993 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.994 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:39.994 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.699138+0000", 2020-08-07T15:07:39.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1221000000000001e-05 2020-08-07T15:07:39.994 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.994 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.995 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:39.995 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.699209+0000", 2020-08-07T15:07:39.995 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6989999999999997e-06 2020-08-07T15:07:39.995 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.995 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.996 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:39.996 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.699216+0000", 2020-08-07T15:07:39.996 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17275928900000001 2020-08-07T15:07:39.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.996 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.996 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:39.997 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.871975+0000", 2020-08-07T15:07:39.997 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0990000000000001e-06 2020-08-07T15:07:39.997 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.997 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.997 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:39.998 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.871979+0000", 2020-08-07T15:07:39.998 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8105999999999997e-05 2020-08-07T15:07:39.998 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.998 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.998 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:39.999 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.872017+0000", 2020-08-07T15:07:39.999 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7921e-05 2020-08-07T15:07:39.999 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:39.999 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:39.999 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:39.999 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.872035+0000", 2020-08-07T15:07:40.000 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00055102099999999995 2020-08-07T15:07:40.000 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.000 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.000 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:40.000 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.872586+0000", 2020-08-07T15:07:40.001 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6119999999999998e-05 2020-08-07T15:07:40.001 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.001 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.001 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:40.001 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.872642+0000", 2020-08-07T15:07:40.002 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17350447599999999 2020-08-07T15:07:40.002 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.002 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:40.002 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.002 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.002 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.003 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:78480 3.7 e46/20 3:e01a31d6:::benchmark_data_smithi074_35447_object9809:head v 46'4768, mlcod=46'4754)", 2020-08-07T15:07:40.003 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:29.037712+0000", 2020-08-07T15:07:40.003 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 10.672705249, 2020-08-07T15:07:40.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17142612199999999, 2020-08-07T15:07:40.003 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:40.004 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:40.004 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:40.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:40.004 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.037712+0000", 2020-08-07T15:07:40.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:40.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:40.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.037712+0000", 2020-08-07T15:07:40.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2291e-05 2020-08-07T15:07:40.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:40.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.037745+0000", 2020-08-07T15:07:40.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1330000000000001e-06 2020-08-07T15:07:40.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:40.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.037749+0000", 2020-08-07T15:07:40.008 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16534796600000001 2020-08-07T15:07:40.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.009 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:40.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.203097+0000", 2020-08-07T15:07:40.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6579999999999999e-06 2020-08-07T15:07:40.009 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.009 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.010 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:40.010 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.203102+0000", 2020-08-07T15:07:40.010 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7601999999999997e-05 2020-08-07T15:07:40.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.011 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:40.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.203140+0000", 2020-08-07T15:07:40.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5035999999999999e-05 2020-08-07T15:07:40.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.011 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:40.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.203165+0000", 2020-08-07T15:07:40.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0058662300000000001 2020-08-07T15:07:40.012 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.012 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:40.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.209031+0000", 2020-08-07T15:07:40.013 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000107206 2020-08-07T15:07:40.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.013 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:40.014 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.209138+0000", 2020-08-07T15:07:40.014 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17142612199999999 2020-08-07T15:07:40.015 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.015 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:40.015 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.016 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.016 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:79920 3.1 e47/26 3:89f6b906:::benchmark_data_smithi074_35447_object9989:head v 47'4792, mlcod=47'4784)", 2020-08-07T15:07:40.016 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:29.985182+0000", 2020-08-07T15:07:40.016 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 9.7252353780000007, 2020-08-07T15:07:40.017 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17172180300000001, 2020-08-07T15:07:40.017 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:40.017 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:40.017 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:40.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.018 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:40.018 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.985182+0000", 2020-08-07T15:07:40.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:40.018 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.018 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.018 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:40.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.985182+0000", 2020-08-07T15:07:40.019 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1389999999999993e-05 2020-08-07T15:07:40.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.019 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.019 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:40.020 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.985244+0000", 2020-08-07T15:07:40.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0020000000000006e-06 2020-08-07T15:07:40.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.020 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.020 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:40.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.985252+0000", 2020-08-07T15:07:40.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16828369000000001 2020-08-07T15:07:40.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:40.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.153535+0000", 2020-08-07T15:07:40.022 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6480000000000002e-06 2020-08-07T15:07:40.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:40.022 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.153540+0000", 2020-08-07T15:07:40.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7845000000000001e-05 2020-08-07T15:07:40.023 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.023 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:40.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.153578+0000", 2020-08-07T15:07:40.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1129e-05 2020-08-07T15:07:40.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:40.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.153599+0000", 2020-08-07T15:07:40.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032692620000000002 2020-08-07T15:07:40.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:40.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.156868+0000", 2020-08-07T15:07:40.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5837000000000003e-05 2020-08-07T15:07:40.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.034 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.034 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:40.034 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.156904+0000", 2020-08-07T15:07:40.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17172180300000001 2020-08-07T15:07:40.035 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.035 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:40.035 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.036 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:81544 3.7 e48/20 3:e9bb655d:::benchmark_data_smithi074_35447_object10192:head v 48'4944, mlcod=48'4935)", 2020-08-07T15:07:40.036 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:30.902099+0000", 2020-08-07T15:07:40.036 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 8.8083190039999995, 2020-08-07T15:07:40.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20669163500000001, 2020-08-07T15:07:40.036 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:40.037 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:40.037 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:40.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.037 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:40.037 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.902099+0000", 2020-08-07T15:07:40.038 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:40.038 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.038 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:40.038 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.902099+0000", 2020-08-07T15:07:40.038 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1317999999999999e-05 2020-08-07T15:07:40.039 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.039 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.039 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:40.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.902150+0000", 2020-08-07T15:07:40.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2079999999999997e-06 2020-08-07T15:07:40.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:40.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.902156+0000", 2020-08-07T15:07:40.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20264375100000001 2020-08-07T15:07:40.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.042 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:40.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.104800+0000", 2020-08-07T15:07:40.042 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2390000000000002e-06 2020-08-07T15:07:40.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:40.043 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.104805+0000", 2020-08-07T15:07:40.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3696000000000003e-05 2020-08-07T15:07:40.050 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.050 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.050 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:40.050 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.104849+0000", 2020-08-07T15:07:40.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4179e-05 2020-08-07T15:07:40.051 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.051 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:40.051 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.104873+0000", 2020-08-07T15:07:40.051 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038653839999999999 2020-08-07T15:07:40.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.052 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.052 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:40.052 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.108738+0000", 2020-08-07T15:07:40.052 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1860000000000002e-05 2020-08-07T15:07:40.053 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.053 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:40.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.108790+0000", 2020-08-07T15:07:40.053 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20669163500000001 2020-08-07T15:07:40.054 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.054 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:40.054 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.054 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.054 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.054 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:85696 3.f e51/20 3:f99fcea4:::benchmark_data_smithi074_35447_object10711:head v 51'5688, mlcod=51'5679)", 2020-08-07T15:07:40.055 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:07:40.055 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.0578125500000004, 2020-08-07T15:07:40.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49655717300000002, 2020-08-07T15:07:40.055 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:40.055 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:40.056 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:40.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:40.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:07:40.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:40.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:40.057 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:07:40.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9543000000000002e-05 2020-08-07T15:07:40.058 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.058 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.058 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:40.058 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652665+0000", 2020-08-07T15:07:40.058 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8700000000000003e-06 2020-08-07T15:07:40.058 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.070 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.070 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:40.070 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652671+0000", 2020-08-07T15:07:40.071 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49579595799999998 2020-08-07T15:07:40.071 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.071 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.071 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:40.071 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148467+0000", 2020-08-07T15:07:40.072 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2509999999999997e-06 2020-08-07T15:07:40.072 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.072 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.072 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:40.072 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148475+0000", 2020-08-07T15:07:40.073 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5603999999999998e-05 2020-08-07T15:07:40.073 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.073 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.073 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:40.073 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148530+0000", 2020-08-07T15:07:40.074 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5486999999999998e-05 2020-08-07T15:07:40.074 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.074 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.074 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:40.074 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148556+0000", 2020-08-07T15:07:40.075 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00055666600000000004 2020-08-07T15:07:40.075 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.075 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.075 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:40.075 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.149112+0000", 2020-08-07T15:07:40.076 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9793999999999998e-05 2020-08-07T15:07:40.076 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.076 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.076 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:40.076 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.149162+0000", 2020-08-07T15:07:40.076 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49655717300000002 2020-08-07T15:07:40.077 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.077 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:40.077 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.077 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.077 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.099 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:86792 3.f e52/20 3:fc5386ee:::benchmark_data_smithi074_35447_object10848:head v 52'5752, mlcod=52'5743)", 2020-08-07T15:07:40.100 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:07:40.100 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.3079667749999997, 2020-08-07T15:07:40.100 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.295270005, 2020-08-07T15:07:40.100 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:40.101 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:40.101 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:40.101 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.101 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:40.101 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:07:40.102 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:40.102 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.102 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.102 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:40.102 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:07:40.103 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9712000000000001e-05 2020-08-07T15:07:40.103 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.103 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.103 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:40.103 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402500+0000", 2020-08-07T15:07:40.104 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6789999999999999e-06 2020-08-07T15:07:40.104 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.104 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.104 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:40.104 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402507+0000", 2020-08-07T15:07:40.105 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29317074999999998 2020-08-07T15:07:40.123 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.123 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.124 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:40.124 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695678+0000", 2020-08-07T15:07:40.124 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8120000000000002e-06 2020-08-07T15:07:40.124 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.124 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.125 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:40.125 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695685+0000", 2020-08-07T15:07:40.125 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1270999999999998e-05 2020-08-07T15:07:40.125 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.126 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.126 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:40.126 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695726+0000", 2020-08-07T15:07:40.126 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2739e-05 2020-08-07T15:07:40.126 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.127 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.127 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:40.127 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695749+0000", 2020-08-07T15:07:40.156 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019333899999999999 2020-08-07T15:07:40.157 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.157 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.157 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:40.157 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.697682+0000", 2020-08-07T15:07:40.157 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8652000000000002e-05 2020-08-07T15:07:40.158 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.158 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.158 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:40.158 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.697721+0000", 2020-08-07T15:07:40.158 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.295270005 2020-08-07T15:07:40.159 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.159 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:40.159 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.159 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.159 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.160 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:87088 3.6 e52/26 3:6b0320df:::benchmark_data_smithi074_35447_object10885:head v 52'5912, mlcod=52'5902)", 2020-08-07T15:07:40.160 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:07:40.160 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4.7437335349999996, 2020-08-07T15:07:40.160 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29522189799999998, 2020-08-07T15:07:40.160 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:40.161 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:40.161 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:40.161 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.161 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:40.161 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:07:40.168 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:40.168 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.168 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.169 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:40.169 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:07:40.169 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0751000000000002e-05 2020-08-07T15:07:40.169 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.169 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.175 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:40.175 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966735+0000", 2020-08-07T15:07:40.175 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3080000000000004e-06 2020-08-07T15:07:40.176 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.176 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.176 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:40.176 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966741+0000", 2020-08-07T15:07:40.176 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29365034699999998 2020-08-07T15:07:40.177 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.177 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.177 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:40.177 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260391+0000", 2020-08-07T15:07:40.177 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2720000000000003e-06 2020-08-07T15:07:40.178 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.178 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.178 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:40.178 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260398+0000", 2020-08-07T15:07:40.178 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8819000000000005e-05 2020-08-07T15:07:40.189 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.189 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.190 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:40.190 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260476+0000", 2020-08-07T15:07:40.190 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4958999999999999e-05 2020-08-07T15:07:40.190 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.190 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.191 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:40.191 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260501+0000", 2020-08-07T15:07:40.191 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001353641 2020-08-07T15:07:40.191 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.191 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.192 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:40.192 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.261855+0000", 2020-08-07T15:07:40.192 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0800999999999997e-05 2020-08-07T15:07:40.192 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.192 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.193 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:40.193 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.261906+0000", 2020-08-07T15:07:40.193 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29522189799999998 2020-08-07T15:07:40.193 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.193 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:40.194 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.194 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.194 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.194 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:94024 3.1 e56/26 3:87245d0e:::benchmark_data_smithi074_35447_object11752:head v 56'5560, mlcod=56'5551)", 2020-08-07T15:07:40.194 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.398599+0000", 2020-08-07T15:07:40.194 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.311818076, 2020-08-07T15:07:40.195 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16220536999999999, 2020-08-07T15:07:40.195 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:40.195 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:40.195 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:40.195 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.196 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:40.196 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.398599+0000", 2020-08-07T15:07:40.196 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:40.196 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.196 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.197 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:40.197 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.398599+0000", 2020-08-07T15:07:40.197 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2380000000000003e-05 2020-08-07T15:07:40.197 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.197 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.208 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:40.208 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.398652+0000", 2020-08-07T15:07:40.208 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0256e-05 2020-08-07T15:07:40.208 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.209 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.209 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:40.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.398662+0000", 2020-08-07T15:07:40.209 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16129753599999999 2020-08-07T15:07:40.209 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.210 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.210 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:40.210 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.559960+0000", 2020-08-07T15:07:40.210 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5899999999999998e-06 2020-08-07T15:07:40.210 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.211 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.211 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:40.211 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.559965+0000", 2020-08-07T15:07:40.220 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4124e-05 2020-08-07T15:07:40.220 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.220 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.222 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:40.222 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.560009+0000", 2020-08-07T15:07:40.222 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5664999999999999e-05 2020-08-07T15:07:40.223 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.223 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.224 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:40.224 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.560035+0000", 2020-08-07T15:07:40.224 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00071728700000000005 2020-08-07T15:07:40.225 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.225 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.225 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:40.225 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.560752+0000", 2020-08-07T15:07:40.226 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2531999999999998e-05 2020-08-07T15:07:40.226 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.226 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.226 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:40.226 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.560805+0000", 2020-08-07T15:07:40.226 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16220536999999999 2020-08-07T15:07:40.227 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.227 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:40.227 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.227 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.227 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:40.228 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:40.228 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:07:40.228 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:07:40.228 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:07:40.229 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:07:40.229 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:07:40.229 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:07:40.229 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:07:40.229 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:07:40.230 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:07:40.230 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:07:40.230 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:07:40.230 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:07:40.231 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2020-08-07T15:07:39.280472+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":738156544,"num_objects":11266,"num_object_clones":0,"num_object_copies":33798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1432,"num_objects_unfound":0,"num_objects_dirty":11266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90108,"num_write_kb":720857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":105,"acting":105,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":113,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":10544164,"kb_used_data":2155508,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":744430556,"statfs":{"total":773094113280,"available":762296889344,"internally_reserved":8589934592,"allocated":2207240192,"data_stored":2204372030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,2,7,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":60596224,"num_objects":925,"num_object_clones":0,"num_object_copies":2775,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":108,"num_objects_unfound":0,"num_objects_dirty":925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7396,"num_write_kb":59175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.653702"},"pg_stats":[{"pgid":"3.f","version":"54'5944","reported_seq":"5986","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.501758+0000","last_change":"2020-08-07T15:06:35.163498+0000","last_active":"2020-08-07T15:07:36.501758+0000","last_peered":"2020-08-07T15:07:36.501758+0000","last_clean":"2020-08-07T15:07:36.501758+0000","last_became_active":"2020-08-07T15:06:35.162764+0000","last_became_peered":"2020-08-07T15:06:35.162764+0000","last_unstale":"2020-08-07T15:07:36.501758+0000","last_undegraded":"2020-08-07T15:07:36.501758+0000","last_fullsized":"2020-08-07T15:07:36.501758+0000","mapping_epoch":20,"log_start":"54'5936","ondisk_log_start":"54'5936","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48693248,"num_objects":743,"num_object_clones":0,"num_object_copies":2229,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5944,"num_write_kb":47552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"54'5840","reported_seq":"5882","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.699649+0000","last_change":"2020-08-07T15:06:35.169137+0000","last_active":"2020-08-07T15:07:36.699649+0000","last_peered":"2020-08-07T15:07:36.699649+0000","last_clean":"2020-08-07T15:07:36.699649+0000","last_became_active":"2020-08-07T15:06:35.168689+0000","last_became_peered":"2020-08-07T15:06:35.168689+0000","last_unstale":"2020-08-07T15:07:36.699649+0000","last_undegraded":"2020-08-07T15:07:36.699649+0000","last_fullsized":"2020-08-07T15:07:36.699649+0000","mapping_epoch":20,"log_start":"54'5835","ondisk_log_start":"54'5835","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47841280,"num_objects":730,"num_object_clones":0,"num_object_copies":2190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5840,"num_write_kb":46720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"54'5727","reported_seq":"5769","reported_epoch":"54","state":"active+remapped+backfill_toofull","last_fresh":"2020-08-07T15:07:36.702674+0000","last_change":"2020-08-07T15:07:08.296959+0000","last_active":"2020-08-07T15:07:36.702674+0000","last_peered":"2020-08-07T15:07:36.702674+0000","last_clean":"2020-08-07T15:07:06.228184+0000","last_became_active":"2020-08-07T15:07:08.269740+0000","last_became_peered":"2020-08-07T15:07:08.269740+0000","last_unstale":"2020-08-07T15:07:36.702674+0000","last_undegraded":"2020-08-07T15:07:36.702674+0000","last_fullsized":"2020-08-07T15:07:36.702674+0000","mapping_epoch":35,"log_start":"54'5720","ondisk_log_start":"54'5720","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":46915584,"num_objects":716,"num_object_clones":0,"num_object_copies":2148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1432,"num_objects_unfound":0,"num_objects_dirty":716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5727,"num_write_kb":45816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[5,6,7],"avail_no_missing":["5","6","7"],"object_location_counts":[{"shards":"5,6,7","objects":716}],"blocked_by":[],"up_primary":3,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"54'5784","reported_seq":"5826","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.540800+0000","last_change":"2020-08-07T15:06:35.167480+0000","last_active":"2020-08-07T15:07:36.540800+0000","last_peered":"2020-08-07T15:07:36.540800+0000","last_clean":"2020-08-07T15:07:36.540800+0000","last_became_active":"2020-08-07T15:06:35.167101+0000","last_became_peered":"2020-08-07T15:06:35.167101+0000","last_unstale":"2020-08-07T15:07:36.540800+0000","last_undegraded":"2020-08-07T15:07:36.540800+0000","last_fullsized":"2020-08-07T15:07:36.540800+0000","mapping_epoch":20,"log_start":"54'5776","ondisk_log_start":"54'5776","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47382528,"num_objects":723,"num_object_clones":0,"num_object_copies":2169,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5784,"num_write_kb":46272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"81","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.248696+0000","last_change":"2020-08-07T15:07:33.160980+0000","last_active":"2020-08-07T15:07:35.248696+0000","last_peered":"2020-08-07T15:07:35.248696+0000","last_clean":"2020-08-07T15:07:35.248696+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:07:35.248696+0000","last_undegraded":"2020-08-07T15:07:35.248696+0000","last_fullsized":"2020-08-07T15:07:35.248696+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:18.518049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:18.518049+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.b","version":"54'5188","reported_seq":"5227","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.644183+0000","last_change":"2020-08-07T15:06:35.170504+0000","last_active":"2020-08-07T15:07:36.644183+0000","last_peered":"2020-08-07T15:07:36.644183+0000","last_clean":"2020-08-07T15:07:36.644183+0000","last_became_active":"2020-08-07T15:06:35.169532+0000","last_became_peered":"2020-08-07T15:06:35.169532+0000","last_unstale":"2020-08-07T15:07:36.644183+0000","last_undegraded":"2020-08-07T15:07:36.644183+0000","last_fullsized":"2020-08-07T15:07:36.644183+0000","mapping_epoch":20,"log_start":"54'5184","ondisk_log_start":"54'5184","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":42500096,"num_objects":649,"num_object_clones":0,"num_object_copies":1947,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5188,"num_write_kb":41504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"20","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.245950+0000","last_change":"2020-08-07T15:07:33.177491+0000","last_active":"2020-08-07T15:07:35.245950+0000","last_peered":"2020-08-07T15:07:35.245950+0000","last_clean":"2020-08-07T15:07:35.245950+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:07:35.245950+0000","last_undegraded":"2020-08-07T15:07:35.245950+0000","last_fullsized":"2020-08-07T15:07:35.245950+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:33.177412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:33.177412+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.a","version":"54'5920","reported_seq":"5962","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:37.661446+0000","last_change":"2020-08-07T15:06:35.170105+0000","last_active":"2020-08-07T15:07:37.661446+0000","last_peered":"2020-08-07T15:07:37.661446+0000","last_clean":"2020-08-07T15:07:37.661446+0000","last_became_active":"2020-08-07T15:06:35.169494+0000","last_became_peered":"2020-08-07T15:06:35.169494+0000","last_unstale":"2020-08-07T15:07:37.661446+0000","last_undegraded":"2020-08-07T15:07:37.661446+0000","last_fullsized":"2020-08-07T15:07:37.661446+0000","mapping_epoch":20,"log_start":"54'5912","ondisk_log_start":"54'5912","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48496640,"num_objects":740,"num_object_clones":0,"num_object_copies":2220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5920,"num_write_kb":47360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"54'5272","reported_seq":"5314","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.649275+0000","last_change":"2020-08-07T15:06:35.163109+0000","last_active":"2020-08-07T15:07:36.649275+0000","last_peered":"2020-08-07T15:07:36.649275+0000","last_clean":"2020-08-07T15:07:36.649275+0000","last_became_active":"2020-08-07T15:06:35.162721+0000","last_became_peered":"2020-08-07T15:06:35.162721+0000","last_unstale":"2020-08-07T15:07:36.649275+0000","last_undegraded":"2020-08-07T15:07:36.649275+0000","last_fullsized":"2020-08-07T15:07:36.649275+0000","mapping_epoch":20,"log_start":"54'5265","ondisk_log_start":"54'5265","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":43188224,"num_objects":659,"num_object_clones":0,"num_object_copies":1977,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5272,"num_write_kb":42176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"20","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:36.143434+0000","last_change":"2020-08-07T15:07:36.143434+0000","last_active":"2020-08-07T15:07:36.143434+0000","last_peered":"2020-08-07T15:07:36.143434+0000","last_clean":"2020-08-07T15:07:36.143434+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:07:36.143434+0000","last_undegraded":"2020-08-07T15:07:36.143434+0000","last_fullsized":"2020-08-07T15:07:36.143434+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.143353+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"54'5728","reported_seq":"5769","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.665562+0000","last_change":"2020-08-07T15:06:35.173504+0000","last_active":"2020-08-07T15:07:36.665562+0000","last_peered":"2020-08-07T15:07:36.665562+0000","last_clean":"2020-08-07T15:07:36.665562+0000","last_became_active":"2020-08-07T15:06:35.172437+0000","last_became_peered":"2020-08-07T15:06:35.172437+0000","last_unstale":"2020-08-07T15:07:36.665562+0000","last_undegraded":"2020-08-07T15:07:36.665562+0000","last_fullsized":"2020-08-07T15:07:36.665562+0000","mapping_epoch":20,"log_start":"54'5721","ondisk_log_start":"54'5721","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":46923776,"num_objects":716,"num_object_clones":0,"num_object_copies":2148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5728,"num_write_kb":45824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"41","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.927825+0000","last_change":"2020-08-07T15:07:36.927879+0000","last_active":"2020-08-07T15:07:36.927825+0000","last_peered":"2020-08-07T15:07:36.927825+0000","last_clean":"2020-08-07T15:07:36.927825+0000","last_became_active":"2020-08-07T15:07:33.166297+0000","last_became_peered":"2020-08-07T15:07:33.166297+0000","last_unstale":"2020-08-07T15:07:36.927825+0000","last_undegraded":"2020-08-07T15:07:36.927825+0000","last_fullsized":"2020-08-07T15:07:36.927825+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.927824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.927824+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"42","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.248877+0000","last_change":"2020-08-07T15:06:46.199001+0000","last_active":"2020-08-07T15:07:35.248877+0000","last_peered":"2020-08-07T15:07:35.248877+0000","last_clean":"2020-08-07T15:07:35.248877+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:07:35.248877+0000","last_undegraded":"2020-08-07T15:07:35.248877+0000","last_fullsized":"2020-08-07T15:07:35.248877+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.5","version":"54'5528","reported_seq":"5570","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.660638+0000","last_change":"2020-08-07T15:06:35.169307+0000","last_active":"2020-08-07T15:07:36.660638+0000","last_peered":"2020-08-07T15:07:36.660638+0000","last_clean":"2020-08-07T15:07:36.660638+0000","last_became_active":"2020-08-07T15:06:35.168887+0000","last_became_peered":"2020-08-07T15:06:35.168887+0000","last_unstale":"2020-08-07T15:07:36.660638+0000","last_undegraded":"2020-08-07T15:07:36.660638+0000","last_fullsized":"2020-08-07T15:07:36.660638+0000","mapping_epoch":20,"log_start":"54'5521","ondisk_log_start":"54'5521","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":45285376,"num_objects":691,"num_object_clones":0,"num_object_copies":2073,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5528,"num_write_kb":44224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"31","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.245961+0000","last_change":"2020-08-07T15:07:33.171050+0000","last_active":"2020-08-07T15:07:35.245961+0000","last_peered":"2020-08-07T15:07:35.245961+0000","last_clean":"2020-08-07T15:07:35.245961+0000","last_became_active":"2020-08-07T15:07:33.170597+0000","last_became_peered":"2020-08-07T15:07:33.170597+0000","last_unstale":"2020-08-07T15:07:35.245961+0000","last_undegraded":"2020-08-07T15:07:35.245961+0000","last_fullsized":"2020-08-07T15:07:35.245961+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:18.518049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:18.518049+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"45","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.246128+0000","last_change":"2020-08-07T15:06:23.074962+0000","last_active":"2020-08-07T15:07:35.246128+0000","last_peered":"2020-08-07T15:07:35.246128+0000","last_clean":"2020-08-07T15:07:35.246128+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:07:35.246128+0000","last_undegraded":"2020-08-07T15:07:35.246128+0000","last_fullsized":"2020-08-07T15:07:35.246128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"54'5216","reported_seq":"5258","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.635055+0000","last_change":"2020-08-07T15:06:35.164593+0000","last_active":"2020-08-07T15:07:36.635055+0000","last_peered":"2020-08-07T15:07:36.635055+0000","last_clean":"2020-08-07T15:07:36.635055+0000","last_became_active":"2020-08-07T15:06:35.164108+0000","last_became_peered":"2020-08-07T15:06:35.164108+0000","last_unstale":"2020-08-07T15:07:36.635055+0000","last_undegraded":"2020-08-07T15:07:36.635055+0000","last_fullsized":"2020-08-07T15:07:36.635055+0000","mapping_epoch":20,"log_start":"54'5208","ondisk_log_start":"54'5208","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":42729472,"num_objects":652,"num_object_clones":0,"num_object_copies":1956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5216,"num_write_kb":41728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"84","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.358799+0000","last_change":"2020-08-07T15:07:36.358882+0000","last_active":"2020-08-07T15:07:36.358799+0000","last_peered":"2020-08-07T15:07:36.358799+0000","last_clean":"2020-08-07T15:07:36.358799+0000","last_became_active":"2020-08-07T15:07:33.171449+0000","last_became_peered":"2020-08-07T15:07:33.171449+0000","last_unstale":"2020-08-07T15:07:36.358799+0000","last_undegraded":"2020-08-07T15:07:36.358799+0000","last_fullsized":"2020-08-07T15:07:36.358799+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.358798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.358798+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"45","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.247910+0000","last_change":"2020-08-07T15:06:23.223924+0000","last_active":"2020-08-07T15:07:35.247910+0000","last_peered":"2020-08-07T15:07:35.247910+0000","last_clean":"2020-08-07T15:07:35.247910+0000","last_became_active":"2020-08-07T15:06:23.223489+0000","last_became_peered":"2020-08-07T15:06:23.223489+0000","last_unstale":"2020-08-07T15:07:35.247910+0000","last_undegraded":"2020-08-07T15:07:35.247910+0000","last_fullsized":"2020-08-07T15:07:35.247910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"54'5688","reported_seq":"5730","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.607352+0000","last_change":"2020-08-07T15:06:35.170553+0000","last_active":"2020-08-07T15:07:36.607352+0000","last_peered":"2020-08-07T15:07:36.607352+0000","last_clean":"2020-08-07T15:07:36.607352+0000","last_became_active":"2020-08-07T15:06:35.170037+0000","last_became_peered":"2020-08-07T15:06:35.170037+0000","last_unstale":"2020-08-07T15:07:36.607352+0000","last_undegraded":"2020-08-07T15:07:36.607352+0000","last_fullsized":"2020-08-07T15:07:36.607352+0000","mapping_epoch":20,"log_start":"54'5680","ondisk_log_start":"54'5680","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":46596096,"num_objects":711,"num_object_clones":0,"num_object_copies":2133,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5688,"num_write_kb":45504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"45","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.246154+0000","last_change":"2020-08-07T15:06:23.075958+0000","last_active":"2020-08-07T15:07:35.246154+0000","last_peered":"2020-08-07T15:07:35.246154+0000","last_clean":"2020-08-07T15:07:35.246154+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:07:35.246154+0000","last_undegraded":"2020-08-07T15:07:35.246154+0000","last_fullsized":"2020-08-07T15:07:35.246154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"68","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.248680+0000","last_change":"2020-08-07T15:07:32.461010+0000","last_active":"2020-08-07T15:07:35.248680+0000","last_peered":"2020-08-07T15:07:35.248680+0000","last_clean":"2020-08-07T15:07:35.248680+0000","last_became_active":"2020-08-07T15:07:28.210245+0000","last_became_peered":"2020-08-07T15:07:28.210245+0000","last_unstale":"2020-08-07T15:07:35.248680+0000","last_undegraded":"2020-08-07T15:07:35.248680+0000","last_fullsized":"2020-08-07T15:07:35.248680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:32.460945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:32.460945+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"54'5608","reported_seq":"5649","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.685391+0000","last_change":"2020-08-07T15:06:35.169888+0000","last_active":"2020-08-07T15:07:36.685391+0000","last_peered":"2020-08-07T15:07:36.685391+0000","last_clean":"2020-08-07T15:07:36.685391+0000","last_became_active":"2020-08-07T15:06:35.169385+0000","last_became_peered":"2020-08-07T15:06:35.169385+0000","last_unstale":"2020-08-07T15:07:36.685391+0000","last_undegraded":"2020-08-07T15:07:36.685391+0000","last_fullsized":"2020-08-07T15:07:36.685391+0000","mapping_epoch":20,"log_start":"54'5600","ondisk_log_start":"54'5600","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":45940736,"num_objects":701,"num_object_clones":0,"num_object_copies":2103,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5608,"num_write_kb":44864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"38","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.296041+0000","last_change":"2020-08-07T15:07:33.157062+0000","last_active":"2020-08-07T15:07:35.296041+0000","last_peered":"2020-08-07T15:07:35.296041+0000","last_clean":"2020-08-07T15:07:35.296041+0000","last_became_active":"2020-08-07T15:07:33.156569+0000","last_became_peered":"2020-08-07T15:07:33.156569+0000","last_unstale":"2020-08-07T15:07:35.296041+0000","last_undegraded":"2020-08-07T15:07:35.296041+0000","last_fullsized":"2020-08-07T15:07:35.296041+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:18.518049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:18.518049+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"45","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.248722+0000","last_change":"2020-08-07T15:06:23.224115+0000","last_active":"2020-08-07T15:07:35.248722+0000","last_peered":"2020-08-07T15:07:35.248722+0000","last_clean":"2020-08-07T15:07:35.248722+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:07:35.248722+0000","last_undegraded":"2020-08-07T15:07:35.248722+0000","last_fullsized":"2020-08-07T15:07:35.248722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"54'5344","reported_seq":"5383","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.696701+0000","last_change":"2020-08-07T15:06:46.214618+0000","last_active":"2020-08-07T15:07:36.696701+0000","last_peered":"2020-08-07T15:07:36.696701+0000","last_clean":"2020-08-07T15:07:36.696701+0000","last_became_active":"2020-08-07T15:06:46.202192+0000","last_became_peered":"2020-08-07T15:06:46.202192+0000","last_unstale":"2020-08-07T15:07:36.696701+0000","last_undegraded":"2020-08-07T15:07:36.696701+0000","last_fullsized":"2020-08-07T15:07:36.696701+0000","mapping_epoch":26,"log_start":"54'5338","ondisk_log_start":"54'5338","created":20,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":43778048,"num_objects":668,"num_object_clones":0,"num_object_copies":2004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5344,"num_write_kb":42752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3],"acting":[4,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"38","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.296060+0000","last_change":"2020-08-07T15:07:33.170029+0000","last_active":"2020-08-07T15:07:35.296060+0000","last_peered":"2020-08-07T15:07:35.296060+0000","last_clean":"2020-08-07T15:07:35.296060+0000","last_became_active":"2020-08-07T15:07:33.169518+0000","last_became_peered":"2020-08-07T15:07:33.169518+0000","last_unstale":"2020-08-07T15:07:35.296060+0000","last_undegraded":"2020-08-07T15:07:35.296060+0000","last_fullsized":"2020-08-07T15:07:35.296060+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:18.518049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:18.518049+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"45","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.248799+0000","last_change":"2020-08-07T15:06:23.225886+0000","last_active":"2020-08-07T15:07:35.248799+0000","last_peered":"2020-08-07T15:07:35.248799+0000","last_clean":"2020-08-07T15:07:35.248799+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:07:35.248799+0000","last_undegraded":"2020-08-07T15:07:35.248799+0000","last_fullsized":"2020-08-07T15:07:35.248799+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"54'5704","reported_seq":"5744","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.691281+0000","last_change":"2020-08-07T15:06:35.169626+0000","last_active":"2020-08-07T15:07:36.691281+0000","last_peered":"2020-08-07T15:07:36.691281+0000","last_clean":"2020-08-07T15:07:36.691281+0000","last_became_active":"2020-08-07T15:06:35.169127+0000","last_became_peered":"2020-08-07T15:06:35.169127+0000","last_unstale":"2020-08-07T15:07:36.691281+0000","last_undegraded":"2020-08-07T15:07:36.691281+0000","last_fullsized":"2020-08-07T15:07:36.691281+0000","mapping_epoch":20,"log_start":"54'5698","ondisk_log_start":"54'5698","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":46727168,"num_objects":713,"num_object_clones":0,"num_object_copies":2139,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5704,"num_write_kb":45632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"16'2","reported_seq":"17","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.243767+0000","last_change":"2020-08-07T15:07:33.155878+0000","last_active":"2020-08-07T15:07:35.243767+0000","last_peered":"2020-08-07T15:07:35.243767+0000","last_clean":"2020-08-07T15:07:35.243767+0000","last_became_active":"2020-08-07T15:07:33.155139+0000","last_became_peered":"2020-08-07T15:07:33.155139+0000","last_unstale":"2020-08-07T15:07:35.243767+0000","last_undegraded":"2020-08-07T15:07:35.243767+0000","last_fullsized":"2020-08-07T15:07:35.243767+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:18.518049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:18.518049+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"54'5504","reported_seq":"5545","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.680215+0000","last_change":"2020-08-07T15:06:35.172780+0000","last_active":"2020-08-07T15:07:36.680215+0000","last_peered":"2020-08-07T15:07:36.680215+0000","last_clean":"2020-08-07T15:07:36.680215+0000","last_became_active":"2020-08-07T15:06:35.172360+0000","last_became_peered":"2020-08-07T15:06:35.172360+0000","last_unstale":"2020-08-07T15:07:36.680215+0000","last_undegraded":"2020-08-07T15:07:36.680215+0000","last_fullsized":"2020-08-07T15:07:36.680215+0000","mapping_epoch":20,"log_start":"54'5496","ondisk_log_start":"54'5496","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":45088768,"num_objects":688,"num_object_clones":0,"num_object_copies":2064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5504,"num_write_kb":44032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"48","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.253223+0000","last_change":"2020-08-07T15:07:34.423236+0000","last_active":"2020-08-07T15:07:35.253223+0000","last_peered":"2020-08-07T15:07:35.253223+0000","last_clean":"2020-08-07T15:07:35.253223+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:07:35.253223+0000","last_undegraded":"2020-08-07T15:07:35.253223+0000","last_fullsized":"2020-08-07T15:07:35.253223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:07:34.423115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:07:34.423115+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"27","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.248451+0000","last_change":"2020-08-07T15:07:34.653928+0000","last_active":"2020-08-07T15:07:35.248451+0000","last_peered":"2020-08-07T15:07:35.248451+0000","last_clean":"2020-08-07T15:07:35.248451+0000","last_became_active":"2020-08-07T15:07:33.161761+0000","last_became_peered":"2020-08-07T15:07:33.161761+0000","last_unstale":"2020-08-07T15:07:35.248451+0000","last_undegraded":"2020-08-07T15:07:35.248451+0000","last_fullsized":"2020-08-07T15:07:35.248451+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:34.653843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:34.653843+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"45","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.296358+0000","last_change":"2020-08-07T15:06:23.073702+0000","last_active":"2020-08-07T15:07:35.296358+0000","last_peered":"2020-08-07T15:07:35.296358+0000","last_clean":"2020-08-07T15:07:35.296358+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:07:35.296358+0000","last_undegraded":"2020-08-07T15:07:35.296358+0000","last_fullsized":"2020-08-07T15:07:35.296358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"54'6112","reported_seq":"6156","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.642119+0000","last_change":"2020-08-07T15:06:46.199635+0000","last_active":"2020-08-07T15:07:36.642119+0000","last_peered":"2020-08-07T15:07:36.642119+0000","last_clean":"2020-08-07T15:07:36.642119+0000","last_became_active":"2020-08-07T15:06:46.199201+0000","last_became_peered":"2020-08-07T15:06:46.199201+0000","last_unstale":"2020-08-07T15:07:36.642119+0000","last_undegraded":"2020-08-07T15:07:36.642119+0000","last_fullsized":"2020-08-07T15:07:36.642119+0000","mapping_epoch":26,"log_start":"54'6105","ondisk_log_start":"54'6105","created":20,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50069504,"num_objects":764,"num_object_clones":0,"num_object_copies":2292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6112,"num_write_kb":48896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":738156544,"num_objects":11264,"num_object_clones":0,"num_object_copies":33792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1432,"num_objects_unfound":0,"num_objects_dirty":11264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90107,"num_write_kb":720856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2217771008,"data_stored":2217771008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22,"ondisk_log_size":22,"up":33,"acting":33,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542162,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1321028,"kb_used_data":272444,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93050812,"statfs":{"total":96636764160,"available":95284031488,"internally_reserved":1073741824,"allocated":278982656,"data_stored":278620165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.008},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":1.2}]},{"osd":1,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":5.7519999999999998,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.887}]},{"osd":2,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":1.038}]},{"osd":3,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":5.5019999999999998,"5min":5.5019999999999998,"15min":5.5019999999999998},"last":1.222},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":1.25}]},{"osd":4,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.9790000000000001,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":1.345},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":1.236}]},{"osd":5,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":37.506999999999998,"5min":37.506999999999998,"15min":37.506999999999998},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":831.88099999999997,"5min":831.88099999999997,"15min":831.88099999999997},"last":7.4429999999999996},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":7.5350000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":3.1389999999999998,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":3.048,"5min":3.048,"15min":3.048},"last":1.8100000000000001}]}]},{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1319356,"kb_used_data":270772,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93052484,"statfs":{"total":96636764160,"available":95285743616,"internally_reserved":1073741824,"allocated":277270528,"data_stored":276908037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":3.4020000000000001,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.383,"5min":2.383,"15min":2.383},"last":0.84999999999999998}]},{"osd":1,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.212},{"interface":"front","average":{"1min":1.387,"5min":1.387,"15min":1.387},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.5939999999999999,"5min":3.5939999999999999,"15min":3.5939999999999999},"last":1.1990000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.272,"15min":1.272},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":4.0529999999999999,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":1.175}]},{"osd":3,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":3.3580000000000001,"5min":3.3580000000000001,"15min":3.3580000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":3.004,"5min":3.004,"15min":3.004},"last":1.1220000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.663,"15min":1.663},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":5.2729999999999997,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"min":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":5.2990000000000004,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":0.93100000000000005}]},{"osd":5,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.766,"5min":1.766,"15min":1.766},"min":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":5.327,"5min":5.327,"15min":5.327},"last":1.097}]},{"osd":6,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":3.3809999999999998,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.2310000000000001},{"interface":"front","average":{"1min":18.326000000000001,"5min":18.326000000000001,"15min":18.326000000000001},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":266.26999999999998,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.1459999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1326936,"kb_used_data":278344,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93044904,"statfs":{"total":96636764160,"available":95277981696,"internally_reserved":1073741824,"allocated":285024256,"data_stored":284665033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":34.186,"5min":34.186,"15min":34.186},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":497.04000000000002,"5min":497.04000000000002,"15min":497.04000000000002},"last":1.754},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":4.1980000000000004,"5min":4.1980000000000004,"15min":4.1980000000000004},"last":2.7029999999999998}]},{"osd":2,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":2.7869999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":4.0919999999999996,"5min":4.0919999999999996,"15min":4.0919999999999996},"last":2.1640000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.454,"15min":1.454},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.9889999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":3.7330000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":3.7040000000000002},{"interface":"front","average":{"1min":1.2,"5min":1.2,"15min":1.2},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":2.8809999999999998}]},{"osd":5,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":13.007999999999999,"5min":13.007999999999999,"15min":13.007999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":213.63800000000001,"5min":213.63800000000001,"15min":213.63800000000001},"last":3.577},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":2.851}]},{"osd":6,"last update":"Fri Aug 7 15:07:22 2020","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.552,"15min":1.552},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":3.234,"5min":3.234,"15min":3.234},"last":2.577},{"interface":"front","average":{"1min":1.361,"5min":1.361,"15min":1.361},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":3.6120000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.615,"5min":1.615,"15min":1.615},"min":{"1min":1.046,"5min":1.046,"15min":1.046},"max":{"1min":3.2810000000000001,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":2.8180000000000001},{"interface":"front","average":{"1min":1.48,"5min":1.48,"15min":1.48},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.8759999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":3.9089999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1314832,"kb_used_data":266248,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93057008,"statfs":{"total":96636764160,"available":95290376192,"internally_reserved":1073741824,"allocated":272637952,"data_stored":272278979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.2440000000000002,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":0.57699999999999996}]},{"osd":2,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":32.740000000000002,"5min":32.740000000000002,"15min":32.740000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":606.18499999999995,"5min":606.18499999999995,"15min":606.18499999999995},"last":0.62}]},{"osd":3,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":0.83199999999999996}]},{"osd":4,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.6949999999999998,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":2.4060000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":0.79000000000000004}]},{"osd":5,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":0.88},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.96199999999999997}]},{"osd":6,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":8.6140000000000008,"5min":8.6140000000000008,"15min":8.6140000000000008},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":124.738,"5min":124.738,"15min":124.738},"last":0.86499999999999999},{"interface":"front","average":{"1min":18.652999999999999,"5min":18.652999999999999,"15min":18.652999999999999},"min":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"max":{"1min":294.94,"5min":294.94,"15min":294.94},"last":0.90000000000000002}]},{"osd":7,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":6.468,"5min":6.468,"15min":6.468},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.883,"5min":2.883,"15min":2.883},"last":0.82099999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1223416,"kb_used_data":174832,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93148424,"statfs":{"total":96636764160,"available":95383986176,"internally_reserved":1073741824,"allocated":179027968,"data_stored":178679726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":3.2240000000000002,"5min":3.2240000000000002,"15min":3.2240000000000002},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1.022}]},{"osd":1,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":3.1320000000000001,"5min":3.1320000000000001,"15min":3.1320000000000001},"last":1.794},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":3.3069999999999999,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":1.1930000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.3759999999999999,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.556,"5min":1.556,"15min":1.556},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":8.2210000000000001,"5min":8.2210000000000001,"15min":8.2210000000000001},"last":1.1719999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.206},{"interface":"front","average":{"1min":1.643,"5min":1.643,"15min":1.643},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":3.2749999999999999,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":0.80100000000000005}]},{"osd":5,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":2.1629999999999998},{"interface":"front","average":{"1min":25.524999999999999,"5min":25.524999999999999,"15min":25.524999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":415.40699999999998,"5min":415.40699999999998,"15min":415.40699999999998},"last":1.1339999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.252},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":1.4079999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.742,"5min":1.742,"15min":1.742},"min":{"1min":1.093,"5min":1.093,"15min":1.093},"max":{"1min":4.056,"5min":4.056,"15min":4.056},"last":3.1190000000000002},{"interface":"front","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":4.4539999999999997,"5min":4.4539999999999997,"15min":4.4539999999999997},"last":1.238}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1358024,"kb_used_data":309440,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93013816,"statfs":{"total":96636764160,"available":95246147584,"internally_reserved":1073741824,"allocated":316866560,"data_stored":316507337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":2.8220000000000001,"5min":2.8220000000000001,"15min":2.8220000000000001},"last":1.179},{"interface":"front","average":{"1min":1.637,"5min":1.637,"15min":1.637},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":13.067,"5min":13.067,"15min":13.067},"last":1.74}]},{"osd":1,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.899,"5min":1.899,"15min":1.899},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":6.6829999999999998,"5min":6.6829999999999998,"15min":6.6829999999999998},"last":1.0469999999999999},{"interface":"front","average":{"1min":2.012,"5min":2.012,"15min":2.012},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":11.936,"5min":11.936,"15min":11.936},"last":1.133}]},{"osd":2,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.589,"5min":3.589,"15min":3.589},"last":2.5630000000000002},{"interface":"front","average":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":5.5890000000000004,"5min":5.5890000000000004,"15min":5.5890000000000004},"last":1.6759999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:07:19 2020","interfaces":[{"interface":"back","average":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":4.556,"5min":4.556,"15min":4.556},"last":3.2389999999999999},{"interface":"front","average":{"1min":32.100000000000001,"5min":32.100000000000001,"15min":32.100000000000001},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":677.88599999999997,"5min":677.88599999999997,"15min":677.88599999999997},"last":2.5979999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":3.6600000000000001,"5min":3.6600000000000001,"15min":3.6600000000000001},"last":2.6400000000000001},{"interface":"front","average":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"min":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":5.4329999999999998,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":2.6960000000000002}]},{"osd":6,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":3.621,"5min":3.621,"15min":3.621},"last":2.665},{"interface":"front","average":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":5.6189999999999998,"5min":5.6189999999999998,"15min":5.6189999999999998},"last":1.7110000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":5.734,"5min":5.734,"15min":5.734},"last":2.1659999999999999},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":4.8540000000000001,"5min":4.8540000000000001,"15min":4.8540000000000001},"last":3.2719999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1406460,"kb_used_data":357900,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92965380,"statfs":{"total":96636764160,"available":95196549120,"internally_reserved":1073741824,"allocated":366489600,"data_stored":366127109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,2,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.181,"15min":1.181},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":4.9480000000000004,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":1.165},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.5289999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":4.9829999999999997,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.377,"5min":1.377,"15min":1.377},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":5.0430000000000001,"5min":5.0430000000000001,"15min":5.0430000000000001},"last":3.2010000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.452,"15min":1.452},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":4.415,"5min":4.415,"15min":4.415},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":4.4340000000000002,"5min":4.4340000000000002,"15min":4.4340000000000002},"last":3.032}]},{"osd":3,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.508,"15min":1.508},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":5.0099999999999998,"5min":5.0099999999999998,"15min":5.0099999999999998},"last":3.2429999999999999},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":3.2029999999999998,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":2.8260000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":5.157,"5min":5.157,"15min":5.157},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.627,"5min":1.627,"15min":1.627},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":5.0979999999999999,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":3.4510000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.79,"5min":1.79,"15min":1.79},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":5.1310000000000002,"5min":5.1310000000000002,"15min":5.1310000000000002},"last":3.4849999999999999},{"interface":"front","average":{"1min":19.109000000000002,"5min":19.109000000000002,"15min":19.109000000000002},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":389.14600000000002,"5min":389.14600000000002,"15min":389.14600000000002},"last":2.855}]},{"osd":7,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":3.3540000000000001,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":4.1189999999999998},{"interface":"front","average":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":4.4569999999999999,"5min":4.4569999999999999,"15min":4.4569999999999999},"last":0.95399999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1274112,"kb_used_data":225528,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93097728,"statfs":{"total":96636764160,"available":95332073472,"internally_reserved":1073741824,"allocated":230940672,"data_stored":230585644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.766,"5min":2.766,"15min":2.766},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.998}]},{"osd":1,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.6509999999999998,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":2.746,"5min":2.746,"15min":2.746},"last":0.89800000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":9.2200000000000006,"5min":9.2200000000000006,"15min":9.2200000000000006},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":168.56899999999999,"5min":168.56899999999999,"15min":168.56899999999999},"last":1.1259999999999999},{"interface":"front","average":{"1min":33.131,"5min":33.131,"15min":33.131},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":676.27099999999996,"5min":676.27099999999996,"15min":676.27099999999996},"last":0.79700000000000004}]},{"osd":3,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.7669999999999999,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":1.02},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.964,"5min":2.964,"15min":2.964},"last":0.873}]},{"osd":4,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":50.893000000000001,"5min":50.893000000000001,"15min":50.893000000000001},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":530.30799999999999,"5min":530.30799999999999,"15min":530.30799999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":3.1080000000000001,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":1.0629999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":50.408999999999999,"5min":50.408999999999999,"15min":50.408999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":754.31299999999999,"5min":754.31299999999999,"15min":754.31299999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":26.016999999999999,"5min":26.016999999999999,"15min":26.016999999999999},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":517.26700000000005,"5min":517.26700000000005,"15min":517.26700000000005},"last":1.1040000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"min":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":3.5649999999999999,"5min":3.5649999999999999,"15min":3.5649999999999999},"last":1.085}]}]}]}} 2020-08-07T15:07:40.233 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:07:40.238 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:07:40.239 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:39.971+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:40.240 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:07:40.241 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:07:40.242 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:40.240+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:40.253 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 64 16 95322 95306 11.633 11.6406 0.0192574 0.010738 2020-08-07T15:07:40.345 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:40.342+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:40.433 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:40.433 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:07:40.434 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.434 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:95423 3.0 3:000c7cfc:::benchmark_data_smithi074_35447_object11927:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e56)", 2020-08-07T15:07:40.434 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:40.315101+0000", 2020-08-07T15:07:40.434 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.116748348, 2020-08-07T15:07:40.434 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.116778564, 2020-08-07T15:07:40.435 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:40.435 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:40.436 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:40.436 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:40.436 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:40.436 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 95423 2020-08-07T15:07:40.436 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.437 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:40.437 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.437 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:40.437 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.315101+0000", 2020-08-07T15:07:40.437 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:40.438 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.438 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.438 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:40.438 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.315101+0000", 2020-08-07T15:07:40.438 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3483000000000003e-05 2020-08-07T15:07:40.439 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.439 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.439 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:40.439 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.315155+0000", 2020-08-07T15:07:40.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9489999999999996e-06 2020-08-07T15:07:40.440 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.440 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:40.440 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.315163+0000", 2020-08-07T15:07:40.440 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1657e-05 2020-08-07T15:07:40.441 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.441 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.441 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:40.441 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.315175+0000", 2020-08-07T15:07:40.441 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.994e-06 2020-08-07T15:07:40.442 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.442 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.442 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:40.442 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.315180+0000", 2020-08-07T15:07:40.442 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0022456300000000002 2020-08-07T15:07:40.443 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.443 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.443 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:40.443 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.317425+0000", 2020-08-07T15:07:40.443 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4123999999999995e-05 2020-08-07T15:07:40.444 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.444 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:40.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.317519+0000", 2020-08-07T15:07:40.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000170056 2020-08-07T15:07:40.445 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.445 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.445 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:07:40.446 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.317689+0000", 2020-08-07T15:07:40.446 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028671180000000001 2020-08-07T15:07:40.446 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.446 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.447 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:40.447 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.320556+0000", 2020-08-07T15:07:40.447 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4600000000000004e-07 2020-08-07T15:07:40.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:40.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.320557+0000", 2020-08-07T15:07:40.448 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0730999999999999e-05 2020-08-07T15:07:40.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.449 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.449 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:40.449 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.320608+0000", 2020-08-07T15:07:40.450 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00098521200000000007 2020-08-07T15:07:40.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.450 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:40.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.321593+0000", 2020-08-07T15:07:40.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0064917000000000004 2020-08-07T15:07:40.451 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.451 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:40.452 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.452 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:95424 3.0 3:000c7cfc:::benchmark_data_smithi074_35447_object11927:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e56)", 2020-08-07T15:07:40.452 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:40.315214+0000", 2020-08-07T15:07:40.453 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.116635459, 2020-08-07T15:07:40.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11679281399999999, 2020-08-07T15:07:40.453 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:40.453 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:07:40.453 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:40.454 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:40.454 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:40.454 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 95424 2020-08-07T15:07:40.454 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.454 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:40.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:40.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.315214+0000", 2020-08-07T15:07:40.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:40.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:40.456 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.315214+0000", 2020-08-07T15:07:40.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0911000000000001e-05 2020-08-07T15:07:40.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:40.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.315265+0000", 2020-08-07T15:07:40.458 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4890000000000003e-06 2020-08-07T15:07:40.458 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.458 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.458 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:40.458 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.315271+0000", 2020-08-07T15:07:40.459 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0817e-05 2020-08-07T15:07:40.459 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:40.459 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.315282+0000", 2020-08-07T15:07:40.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0599999999999998e-06 2020-08-07T15:07:40.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.460 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.460 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:40.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.315287+0000", 2020-08-07T15:07:40.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042505759999999998 2020-08-07T15:07:40.461 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.461 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.461 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:40.461 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.319537+0000", 2020-08-07T15:07:40.462 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5648000000000003e-05 2020-08-07T15:07:40.462 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.462 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.462 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:40.462 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.319633+0000", 2020-08-07T15:07:40.463 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0001649 2020-08-07T15:07:40.463 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.463 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.463 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:07:40.463 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.319798+0000", 2020-08-07T15:07:40.464 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0021373189999999999 2020-08-07T15:07:40.464 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.465 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.465 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:40.465 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.321935+0000", 2020-08-07T15:07:40.465 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00054533499999999998 2020-08-07T15:07:40.466 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.466 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.467 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:40.468 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.322480+0000", 2020-08-07T15:07:40.468 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0072660550000000004 2020-08-07T15:07:40.468 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.468 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:40.468 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.469 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.469 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:07:40.469 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 2 2020-08-07T15:07:40.469 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:40.471 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:07:40.472 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:40.469+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:40.575 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:40.570+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:40.659 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:40.659 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:07:40.660 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:07:40.660 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:07:40.661 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:40.676 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:40.671+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:40.677 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:07:40.716 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:07:40.717 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:07:40.730 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2020-08-07T15:07:40.627888+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":738156544,"num_objects":11266,"num_object_clones":0,"num_object_copies":33798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1432,"num_objects_unfound":0,"num_objects_dirty":11266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90108,"num_write_kb":720857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":105,"acting":105,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":113,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":10544164,"kb_used_data":2155508,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":744430556,"statfs":{"total":773094113280,"available":762296889344,"internally_reserved":8589934592,"allocated":2207240192,"data_stored":2204372030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,0,2,7,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":60596224,"num_objects":925,"num_object_clones":0,"num_object_copies":2775,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":108,"num_objects_unfound":0,"num_objects_dirty":925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7396,"num_write_kb":59175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.386929"},"pg_stats":[{"pgid":"3.f","version":"54'5944","reported_seq":"5986","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.501758+0000","last_change":"2020-08-07T15:06:35.163498+0000","last_active":"2020-08-07T15:07:36.501758+0000","last_peered":"2020-08-07T15:07:36.501758+0000","last_clean":"2020-08-07T15:07:36.501758+0000","last_became_active":"2020-08-07T15:06:35.162764+0000","last_became_peered":"2020-08-07T15:06:35.162764+0000","last_unstale":"2020-08-07T15:07:36.501758+0000","last_undegraded":"2020-08-07T15:07:36.501758+0000","last_fullsized":"2020-08-07T15:07:36.501758+0000","mapping_epoch":20,"log_start":"54'5936","ondisk_log_start":"54'5936","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48693248,"num_objects":743,"num_object_clones":0,"num_object_copies":2229,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5944,"num_write_kb":47552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"54'5840","reported_seq":"5882","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.699649+0000","last_change":"2020-08-07T15:06:35.169137+0000","last_active":"2020-08-07T15:07:36.699649+0000","last_peered":"2020-08-07T15:07:36.699649+0000","last_clean":"2020-08-07T15:07:36.699649+0000","last_became_active":"2020-08-07T15:06:35.168689+0000","last_became_peered":"2020-08-07T15:06:35.168689+0000","last_unstale":"2020-08-07T15:07:36.699649+0000","last_undegraded":"2020-08-07T15:07:36.699649+0000","last_fullsized":"2020-08-07T15:07:36.699649+0000","mapping_epoch":20,"log_start":"54'5835","ondisk_log_start":"54'5835","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47841280,"num_objects":730,"num_object_clones":0,"num_object_copies":2190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5840,"num_write_kb":46720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"54'5727","reported_seq":"5769","reported_epoch":"54","state":"active+remapped+backfill_toofull","last_fresh":"2020-08-07T15:07:36.702674+0000","last_change":"2020-08-07T15:07:08.296959+0000","last_active":"2020-08-07T15:07:36.702674+0000","last_peered":"2020-08-07T15:07:36.702674+0000","last_clean":"2020-08-07T15:07:06.228184+0000","last_became_active":"2020-08-07T15:07:08.269740+0000","last_became_peered":"2020-08-07T15:07:08.269740+0000","last_unstale":"2020-08-07T15:07:36.702674+0000","last_undegraded":"2020-08-07T15:07:36.702674+0000","last_fullsized":"2020-08-07T15:07:36.702674+0000","mapping_epoch":35,"log_start":"54'5720","ondisk_log_start":"54'5720","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":46915584,"num_objects":716,"num_object_clones":0,"num_object_copies":2148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1432,"num_objects_unfound":0,"num_objects_dirty":716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5727,"num_write_kb":45816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[5,6,7],"avail_no_missing":["5","6","7"],"object_location_counts":[{"shards":"5,6,7","objects":716}],"blocked_by":[],"up_primary":3,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"54'5784","reported_seq":"5826","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.540800+0000","last_change":"2020-08-07T15:06:35.167480+0000","last_active":"2020-08-07T15:07:36.540800+0000","last_peered":"2020-08-07T15:07:36.540800+0000","last_clean":"2020-08-07T15:07:36.540800+0000","last_became_active":"2020-08-07T15:06:35.167101+0000","last_became_peered":"2020-08-07T15:06:35.167101+0000","last_unstale":"2020-08-07T15:07:36.540800+0000","last_undegraded":"2020-08-07T15:07:36.540800+0000","last_fullsized":"2020-08-07T15:07:36.540800+0000","mapping_epoch":20,"log_start":"54'5776","ondisk_log_start":"54'5776","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47382528,"num_objects":723,"num_object_clones":0,"num_object_copies":2169,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5784,"num_write_kb":46272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"81","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.248696+0000","last_change":"2020-08-07T15:07:33.160980+0000","last_active":"2020-08-07T15:07:35.248696+0000","last_peered":"2020-08-07T15:07:35.248696+0000","last_clean":"2020-08-07T15:07:35.248696+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:07:35.248696+0000","last_undegraded":"2020-08-07T15:07:35.248696+0000","last_fullsized":"2020-08-07T15:07:35.248696+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:18.518049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:18.518049+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.b","version":"54'5188","reported_seq":"5227","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.644183+0000","last_change":"2020-08-07T15:06:35.170504+0000","last_active":"2020-08-07T15:07:36.644183+0000","last_peered":"2020-08-07T15:07:36.644183+0000","last_clean":"2020-08-07T15:07:36.644183+0000","last_became_active":"2020-08-07T15:06:35.169532+0000","last_became_peered":"2020-08-07T15:06:35.169532+0000","last_unstale":"2020-08-07T15:07:36.644183+0000","last_undegraded":"2020-08-07T15:07:36.644183+0000","last_fullsized":"2020-08-07T15:07:36.644183+0000","mapping_epoch":20,"log_start":"54'5184","ondisk_log_start":"54'5184","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":42500096,"num_objects":649,"num_object_clones":0,"num_object_copies":1947,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5188,"num_write_kb":41504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"20","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.245950+0000","last_change":"2020-08-07T15:07:33.177491+0000","last_active":"2020-08-07T15:07:35.245950+0000","last_peered":"2020-08-07T15:07:35.245950+0000","last_clean":"2020-08-07T15:07:35.245950+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:07:35.245950+0000","last_undegraded":"2020-08-07T15:07:35.245950+0000","last_fullsized":"2020-08-07T15:07:35.245950+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:33.177412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:33.177412+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.a","version":"54'5920","reported_seq":"5962","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:37.661446+0000","last_change":"2020-08-07T15:06:35.170105+0000","last_active":"2020-08-07T15:07:37.661446+0000","last_peered":"2020-08-07T15:07:37.661446+0000","last_clean":"2020-08-07T15:07:37.661446+0000","last_became_active":"2020-08-07T15:06:35.169494+0000","last_became_peered":"2020-08-07T15:06:35.169494+0000","last_unstale":"2020-08-07T15:07:37.661446+0000","last_undegraded":"2020-08-07T15:07:37.661446+0000","last_fullsized":"2020-08-07T15:07:37.661446+0000","mapping_epoch":20,"log_start":"54'5912","ondisk_log_start":"54'5912","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48496640,"num_objects":740,"num_object_clones":0,"num_object_copies":2220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5920,"num_write_kb":47360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"54'5272","reported_seq":"5314","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.649275+0000","last_change":"2020-08-07T15:06:35.163109+0000","last_active":"2020-08-07T15:07:36.649275+0000","last_peered":"2020-08-07T15:07:36.649275+0000","last_clean":"2020-08-07T15:07:36.649275+0000","last_became_active":"2020-08-07T15:06:35.162721+0000","last_became_peered":"2020-08-07T15:06:35.162721+0000","last_unstale":"2020-08-07T15:07:36.649275+0000","last_undegraded":"2020-08-07T15:07:36.649275+0000","last_fullsized":"2020-08-07T15:07:36.649275+0000","mapping_epoch":20,"log_start":"54'5265","ondisk_log_start":"54'5265","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":43188224,"num_objects":659,"num_object_clones":0,"num_object_copies":1977,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5272,"num_write_kb":42176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"20","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:36.143434+0000","last_change":"2020-08-07T15:07:36.143434+0000","last_active":"2020-08-07T15:07:36.143434+0000","last_peered":"2020-08-07T15:07:36.143434+0000","last_clean":"2020-08-07T15:07:36.143434+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:07:36.143434+0000","last_undegraded":"2020-08-07T15:07:36.143434+0000","last_fullsized":"2020-08-07T15:07:36.143434+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.143353+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"54'5728","reported_seq":"5769","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.665562+0000","last_change":"2020-08-07T15:06:35.173504+0000","last_active":"2020-08-07T15:07:36.665562+0000","last_peered":"2020-08-07T15:07:36.665562+0000","last_clean":"2020-08-07T15:07:36.665562+0000","last_became_active":"2020-08-07T15:06:35.172437+0000","last_became_peered":"2020-08-07T15:06:35.172437+0000","last_unstale":"2020-08-07T15:07:36.665562+0000","last_undegraded":"2020-08-07T15:07:36.665562+0000","last_fullsized":"2020-08-07T15:07:36.665562+0000","mapping_epoch":20,"log_start":"54'5721","ondisk_log_start":"54'5721","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":46923776,"num_objects":716,"num_object_clones":0,"num_object_copies":2148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5728,"num_write_kb":45824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"41","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.927825+0000","last_change":"2020-08-07T15:07:36.927879+0000","last_active":"2020-08-07T15:07:36.927825+0000","last_peered":"2020-08-07T15:07:36.927825+0000","last_clean":"2020-08-07T15:07:36.927825+0000","last_became_active":"2020-08-07T15:07:33.166297+0000","last_became_peered":"2020-08-07T15:07:33.166297+0000","last_unstale":"2020-08-07T15:07:36.927825+0000","last_undegraded":"2020-08-07T15:07:36.927825+0000","last_fullsized":"2020-08-07T15:07:36.927825+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.927824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.927824+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"42","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.248877+0000","last_change":"2020-08-07T15:06:46.199001+0000","last_active":"2020-08-07T15:07:35.248877+0000","last_peered":"2020-08-07T15:07:35.248877+0000","last_clean":"2020-08-07T15:07:35.248877+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:07:35.248877+0000","last_undegraded":"2020-08-07T15:07:35.248877+0000","last_fullsized":"2020-08-07T15:07:35.248877+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.5","version":"54'5528","reported_seq":"5570","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.660638+0000","last_change":"2020-08-07T15:06:35.169307+0000","last_active":"2020-08-07T15:07:36.660638+0000","last_peered":"2020-08-07T15:07:36.660638+0000","last_clean":"2020-08-07T15:07:36.660638+0000","last_became_active":"2020-08-07T15:06:35.168887+0000","last_became_peered":"2020-08-07T15:06:35.168887+0000","last_unstale":"2020-08-07T15:07:36.660638+0000","last_undegraded":"2020-08-07T15:07:36.660638+0000","last_fullsized":"2020-08-07T15:07:36.660638+0000","mapping_epoch":20,"log_start":"54'5521","ondisk_log_start":"54'5521","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":45285376,"num_objects":691,"num_object_clones":0,"num_object_copies":2073,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5528,"num_write_kb":44224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"31","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.245961+0000","last_change":"2020-08-07T15:07:33.171050+0000","last_active":"2020-08-07T15:07:35.245961+0000","last_peered":"2020-08-07T15:07:35.245961+0000","last_clean":"2020-08-07T15:07:35.245961+0000","last_became_active":"2020-08-07T15:07:33.170597+0000","last_became_peered":"2020-08-07T15:07:33.170597+0000","last_unstale":"2020-08-07T15:07:35.245961+0000","last_undegraded":"2020-08-07T15:07:35.245961+0000","last_fullsized":"2020-08-07T15:07:35.245961+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:18.518049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:18.518049+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"45","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.246128+0000","last_change":"2020-08-07T15:06:23.074962+0000","last_active":"2020-08-07T15:07:35.246128+0000","last_peered":"2020-08-07T15:07:35.246128+0000","last_clean":"2020-08-07T15:07:35.246128+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:07:35.246128+0000","last_undegraded":"2020-08-07T15:07:35.246128+0000","last_fullsized":"2020-08-07T15:07:35.246128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"54'5216","reported_seq":"5258","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.635055+0000","last_change":"2020-08-07T15:06:35.164593+0000","last_active":"2020-08-07T15:07:36.635055+0000","last_peered":"2020-08-07T15:07:36.635055+0000","last_clean":"2020-08-07T15:07:36.635055+0000","last_became_active":"2020-08-07T15:06:35.164108+0000","last_became_peered":"2020-08-07T15:06:35.164108+0000","last_unstale":"2020-08-07T15:07:36.635055+0000","last_undegraded":"2020-08-07T15:07:36.635055+0000","last_fullsized":"2020-08-07T15:07:36.635055+0000","mapping_epoch":20,"log_start":"54'5208","ondisk_log_start":"54'5208","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":42729472,"num_objects":652,"num_object_clones":0,"num_object_copies":1956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5216,"num_write_kb":41728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"84","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.358799+0000","last_change":"2020-08-07T15:07:36.358882+0000","last_active":"2020-08-07T15:07:36.358799+0000","last_peered":"2020-08-07T15:07:36.358799+0000","last_clean":"2020-08-07T15:07:36.358799+0000","last_became_active":"2020-08-07T15:07:33.171449+0000","last_became_peered":"2020-08-07T15:07:33.171449+0000","last_unstale":"2020-08-07T15:07:36.358799+0000","last_undegraded":"2020-08-07T15:07:36.358799+0000","last_fullsized":"2020-08-07T15:07:36.358799+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.358798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.358798+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"45","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.247910+0000","last_change":"2020-08-07T15:06:23.223924+0000","last_active":"2020-08-07T15:07:35.247910+0000","last_peered":"2020-08-07T15:07:35.247910+0000","last_clean":"2020-08-07T15:07:35.247910+0000","last_became_active":"2020-08-07T15:06:23.223489+0000","last_became_peered":"2020-08-07T15:06:23.223489+0000","last_unstale":"2020-08-07T15:07:35.247910+0000","last_undegraded":"2020-08-07T15:07:35.247910+0000","last_fullsized":"2020-08-07T15:07:35.247910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"54'5688","reported_seq":"5730","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.607352+0000","last_change":"2020-08-07T15:06:35.170553+0000","last_active":"2020-08-07T15:07:36.607352+0000","last_peered":"2020-08-07T15:07:36.607352+0000","last_clean":"2020-08-07T15:07:36.607352+0000","last_became_active":"2020-08-07T15:06:35.170037+0000","last_became_peered":"2020-08-07T15:06:35.170037+0000","last_unstale":"2020-08-07T15:07:36.607352+0000","last_undegraded":"2020-08-07T15:07:36.607352+0000","last_fullsized":"2020-08-07T15:07:36.607352+0000","mapping_epoch":20,"log_start":"54'5680","ondisk_log_start":"54'5680","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":46596096,"num_objects":711,"num_object_clones":0,"num_object_copies":2133,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5688,"num_write_kb":45504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"45","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.246154+0000","last_change":"2020-08-07T15:06:23.075958+0000","last_active":"2020-08-07T15:07:35.246154+0000","last_peered":"2020-08-07T15:07:35.246154+0000","last_clean":"2020-08-07T15:07:35.246154+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:07:35.246154+0000","last_undegraded":"2020-08-07T15:07:35.246154+0000","last_fullsized":"2020-08-07T15:07:35.246154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"68","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.248680+0000","last_change":"2020-08-07T15:07:32.461010+0000","last_active":"2020-08-07T15:07:35.248680+0000","last_peered":"2020-08-07T15:07:35.248680+0000","last_clean":"2020-08-07T15:07:35.248680+0000","last_became_active":"2020-08-07T15:07:28.210245+0000","last_became_peered":"2020-08-07T15:07:28.210245+0000","last_unstale":"2020-08-07T15:07:35.248680+0000","last_undegraded":"2020-08-07T15:07:35.248680+0000","last_fullsized":"2020-08-07T15:07:35.248680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:32.460945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:32.460945+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"54'5608","reported_seq":"5649","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.685391+0000","last_change":"2020-08-07T15:06:35.169888+0000","last_active":"2020-08-07T15:07:36.685391+0000","last_peered":"2020-08-07T15:07:36.685391+0000","last_clean":"2020-08-07T15:07:36.685391+0000","last_became_active":"2020-08-07T15:06:35.169385+0000","last_became_peered":"2020-08-07T15:06:35.169385+0000","last_unstale":"2020-08-07T15:07:36.685391+0000","last_undegraded":"2020-08-07T15:07:36.685391+0000","last_fullsized":"2020-08-07T15:07:36.685391+0000","mapping_epoch":20,"log_start":"54'5600","ondisk_log_start":"54'5600","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":45940736,"num_objects":701,"num_object_clones":0,"num_object_copies":2103,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5608,"num_write_kb":44864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"38","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.296041+0000","last_change":"2020-08-07T15:07:33.157062+0000","last_active":"2020-08-07T15:07:35.296041+0000","last_peered":"2020-08-07T15:07:35.296041+0000","last_clean":"2020-08-07T15:07:35.296041+0000","last_became_active":"2020-08-07T15:07:33.156569+0000","last_became_peered":"2020-08-07T15:07:33.156569+0000","last_unstale":"2020-08-07T15:07:35.296041+0000","last_undegraded":"2020-08-07T15:07:35.296041+0000","last_fullsized":"2020-08-07T15:07:35.296041+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:18.518049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:18.518049+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"45","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.248722+0000","last_change":"2020-08-07T15:06:23.224115+0000","last_active":"2020-08-07T15:07:35.248722+0000","last_peered":"2020-08-07T15:07:35.248722+0000","last_clean":"2020-08-07T15:07:35.248722+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:07:35.248722+0000","last_undegraded":"2020-08-07T15:07:35.248722+0000","last_fullsized":"2020-08-07T15:07:35.248722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"54'5344","reported_seq":"5383","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.696701+0000","last_change":"2020-08-07T15:06:46.214618+0000","last_active":"2020-08-07T15:07:36.696701+0000","last_peered":"2020-08-07T15:07:36.696701+0000","last_clean":"2020-08-07T15:07:36.696701+0000","last_became_active":"2020-08-07T15:06:46.202192+0000","last_became_peered":"2020-08-07T15:06:46.202192+0000","last_unstale":"2020-08-07T15:07:36.696701+0000","last_undegraded":"2020-08-07T15:07:36.696701+0000","last_fullsized":"2020-08-07T15:07:36.696701+0000","mapping_epoch":26,"log_start":"54'5338","ondisk_log_start":"54'5338","created":20,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":43778048,"num_objects":668,"num_object_clones":0,"num_object_copies":2004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5344,"num_write_kb":42752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3],"acting":[4,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"38","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.296060+0000","last_change":"2020-08-07T15:07:33.170029+0000","last_active":"2020-08-07T15:07:35.296060+0000","last_peered":"2020-08-07T15:07:35.296060+0000","last_clean":"2020-08-07T15:07:35.296060+0000","last_became_active":"2020-08-07T15:07:33.169518+0000","last_became_peered":"2020-08-07T15:07:33.169518+0000","last_unstale":"2020-08-07T15:07:35.296060+0000","last_undegraded":"2020-08-07T15:07:35.296060+0000","last_fullsized":"2020-08-07T15:07:35.296060+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:18.518049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:18.518049+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"45","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.248799+0000","last_change":"2020-08-07T15:06:23.225886+0000","last_active":"2020-08-07T15:07:35.248799+0000","last_peered":"2020-08-07T15:07:35.248799+0000","last_clean":"2020-08-07T15:07:35.248799+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:07:35.248799+0000","last_undegraded":"2020-08-07T15:07:35.248799+0000","last_fullsized":"2020-08-07T15:07:35.248799+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"54'5704","reported_seq":"5744","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.691281+0000","last_change":"2020-08-07T15:06:35.169626+0000","last_active":"2020-08-07T15:07:36.691281+0000","last_peered":"2020-08-07T15:07:36.691281+0000","last_clean":"2020-08-07T15:07:36.691281+0000","last_became_active":"2020-08-07T15:06:35.169127+0000","last_became_peered":"2020-08-07T15:06:35.169127+0000","last_unstale":"2020-08-07T15:07:36.691281+0000","last_undegraded":"2020-08-07T15:07:36.691281+0000","last_fullsized":"2020-08-07T15:07:36.691281+0000","mapping_epoch":20,"log_start":"54'5698","ondisk_log_start":"54'5698","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":46727168,"num_objects":713,"num_object_clones":0,"num_object_copies":2139,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5704,"num_write_kb":45632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"16'2","reported_seq":"17","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.243767+0000","last_change":"2020-08-07T15:07:33.155878+0000","last_active":"2020-08-07T15:07:35.243767+0000","last_peered":"2020-08-07T15:07:35.243767+0000","last_clean":"2020-08-07T15:07:35.243767+0000","last_became_active":"2020-08-07T15:07:33.155139+0000","last_became_peered":"2020-08-07T15:07:33.155139+0000","last_unstale":"2020-08-07T15:07:35.243767+0000","last_undegraded":"2020-08-07T15:07:35.243767+0000","last_fullsized":"2020-08-07T15:07:35.243767+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:18.518049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:18.518049+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"54'5504","reported_seq":"5545","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.680215+0000","last_change":"2020-08-07T15:06:35.172780+0000","last_active":"2020-08-07T15:07:36.680215+0000","last_peered":"2020-08-07T15:07:36.680215+0000","last_clean":"2020-08-07T15:07:36.680215+0000","last_became_active":"2020-08-07T15:06:35.172360+0000","last_became_peered":"2020-08-07T15:06:35.172360+0000","last_unstale":"2020-08-07T15:07:36.680215+0000","last_undegraded":"2020-08-07T15:07:36.680215+0000","last_fullsized":"2020-08-07T15:07:36.680215+0000","mapping_epoch":20,"log_start":"54'5496","ondisk_log_start":"54'5496","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":45088768,"num_objects":688,"num_object_clones":0,"num_object_copies":2064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5504,"num_write_kb":44032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"48","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.253223+0000","last_change":"2020-08-07T15:07:34.423236+0000","last_active":"2020-08-07T15:07:35.253223+0000","last_peered":"2020-08-07T15:07:35.253223+0000","last_clean":"2020-08-07T15:07:35.253223+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:07:35.253223+0000","last_undegraded":"2020-08-07T15:07:35.253223+0000","last_fullsized":"2020-08-07T15:07:35.253223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:07:34.423115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:07:34.423115+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"27","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.248451+0000","last_change":"2020-08-07T15:07:34.653928+0000","last_active":"2020-08-07T15:07:35.248451+0000","last_peered":"2020-08-07T15:07:35.248451+0000","last_clean":"2020-08-07T15:07:35.248451+0000","last_became_active":"2020-08-07T15:07:33.161761+0000","last_became_peered":"2020-08-07T15:07:33.161761+0000","last_unstale":"2020-08-07T15:07:35.248451+0000","last_undegraded":"2020-08-07T15:07:35.248451+0000","last_fullsized":"2020-08-07T15:07:35.248451+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:34.653843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:34.653843+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"45","reported_epoch":"53","state":"active+clean","last_fresh":"2020-08-07T15:07:35.296358+0000","last_change":"2020-08-07T15:06:23.073702+0000","last_active":"2020-08-07T15:07:35.296358+0000","last_peered":"2020-08-07T15:07:35.296358+0000","last_clean":"2020-08-07T15:07:35.296358+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:07:35.296358+0000","last_undegraded":"2020-08-07T15:07:35.296358+0000","last_fullsized":"2020-08-07T15:07:35.296358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"54'6112","reported_seq":"6156","reported_epoch":"54","state":"active+clean","last_fresh":"2020-08-07T15:07:36.642119+0000","last_change":"2020-08-07T15:06:46.199635+0000","last_active":"2020-08-07T15:07:36.642119+0000","last_peered":"2020-08-07T15:07:36.642119+0000","last_clean":"2020-08-07T15:07:36.642119+0000","last_became_active":"2020-08-07T15:06:46.199201+0000","last_became_peered":"2020-08-07T15:06:46.199201+0000","last_unstale":"2020-08-07T15:07:36.642119+0000","last_undegraded":"2020-08-07T15:07:36.642119+0000","last_fullsized":"2020-08-07T15:07:36.642119+0000","mapping_epoch":26,"log_start":"54'6105","ondisk_log_start":"54'6105","created":20,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50069504,"num_objects":764,"num_object_clones":0,"num_object_copies":2292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6112,"num_write_kb":48896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":738156544,"num_objects":11264,"num_object_clones":0,"num_object_copies":33792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1432,"num_objects_unfound":0,"num_objects_dirty":11264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90107,"num_write_kb":720856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2217771008,"data_stored":2217771008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22,"ondisk_log_size":22,"up":33,"acting":33,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542162,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1321028,"kb_used_data":272444,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93050812,"statfs":{"total":96636764160,"available":95284031488,"internally_reserved":1073741824,"allocated":278982656,"data_stored":278620165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.008},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":1.2}]},{"osd":1,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":5.7519999999999998,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.887}]},{"osd":2,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":1.038}]},{"osd":3,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":5.5019999999999998,"5min":5.5019999999999998,"15min":5.5019999999999998},"last":1.222},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":1.25}]},{"osd":4,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.9790000000000001,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":1.345},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":1.236}]},{"osd":5,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":37.506999999999998,"5min":37.506999999999998,"15min":37.506999999999998},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":831.88099999999997,"5min":831.88099999999997,"15min":831.88099999999997},"last":7.4429999999999996},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":7.5350000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":3.1389999999999998,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":3.048,"5min":3.048,"15min":3.048},"last":1.8100000000000001}]}]},{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1319356,"kb_used_data":270772,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93052484,"statfs":{"total":96636764160,"available":95285743616,"internally_reserved":1073741824,"allocated":277270528,"data_stored":276908037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":3.4020000000000001,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.383,"5min":2.383,"15min":2.383},"last":0.84999999999999998}]},{"osd":1,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.212},{"interface":"front","average":{"1min":1.387,"5min":1.387,"15min":1.387},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.5939999999999999,"5min":3.5939999999999999,"15min":3.5939999999999999},"last":1.1990000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.272,"15min":1.272},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":4.0529999999999999,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":1.175}]},{"osd":3,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":3.3580000000000001,"5min":3.3580000000000001,"15min":3.3580000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":3.004,"5min":3.004,"15min":3.004},"last":1.1220000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.663,"15min":1.663},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":5.2729999999999997,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"min":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":5.2990000000000004,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":0.93100000000000005}]},{"osd":5,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.766,"5min":1.766,"15min":1.766},"min":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":5.327,"5min":5.327,"15min":5.327},"last":1.097}]},{"osd":6,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":3.3809999999999998,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.2310000000000001},{"interface":"front","average":{"1min":18.326000000000001,"5min":18.326000000000001,"15min":18.326000000000001},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":266.26999999999998,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.1459999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1326936,"kb_used_data":278344,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93044904,"statfs":{"total":96636764160,"available":95277981696,"internally_reserved":1073741824,"allocated":285024256,"data_stored":284665033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":34.186,"5min":34.186,"15min":34.186},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":497.04000000000002,"5min":497.04000000000002,"15min":497.04000000000002},"last":1.754},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":4.1980000000000004,"5min":4.1980000000000004,"15min":4.1980000000000004},"last":2.7029999999999998}]},{"osd":2,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":2.7869999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":4.0919999999999996,"5min":4.0919999999999996,"15min":4.0919999999999996},"last":2.1640000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.454,"15min":1.454},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.9889999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":3.7330000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":3.7040000000000002},{"interface":"front","average":{"1min":1.2,"5min":1.2,"15min":1.2},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":2.8809999999999998}]},{"osd":5,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":13.007999999999999,"5min":13.007999999999999,"15min":13.007999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":213.63800000000001,"5min":213.63800000000001,"15min":213.63800000000001},"last":3.577},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":2.851}]},{"osd":6,"last update":"Fri Aug 7 15:07:22 2020","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.552,"15min":1.552},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":3.234,"5min":3.234,"15min":3.234},"last":2.577},{"interface":"front","average":{"1min":1.361,"5min":1.361,"15min":1.361},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":3.6120000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.615,"5min":1.615,"15min":1.615},"min":{"1min":1.046,"5min":1.046,"15min":1.046},"max":{"1min":3.2810000000000001,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":2.8180000000000001},{"interface":"front","average":{"1min":1.48,"5min":1.48,"15min":1.48},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.8759999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":3.9089999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1314832,"kb_used_data":266248,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93057008,"statfs":{"total":96636764160,"available":95290376192,"internally_reserved":1073741824,"allocated":272637952,"data_stored":272278979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.2440000000000002,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":0.57699999999999996}]},{"osd":2,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":32.740000000000002,"5min":32.740000000000002,"15min":32.740000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":606.18499999999995,"5min":606.18499999999995,"15min":606.18499999999995},"last":0.62}]},{"osd":3,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":0.83199999999999996}]},{"osd":4,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.6949999999999998,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":2.4060000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":0.79000000000000004}]},{"osd":5,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":0.88},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.96199999999999997}]},{"osd":6,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":8.6140000000000008,"5min":8.6140000000000008,"15min":8.6140000000000008},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":124.738,"5min":124.738,"15min":124.738},"last":0.86499999999999999},{"interface":"front","average":{"1min":18.652999999999999,"5min":18.652999999999999,"15min":18.652999999999999},"min":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"max":{"1min":294.94,"5min":294.94,"15min":294.94},"last":0.90000000000000002}]},{"osd":7,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":6.468,"5min":6.468,"15min":6.468},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.883,"5min":2.883,"15min":2.883},"last":0.82099999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1223416,"kb_used_data":174832,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93148424,"statfs":{"total":96636764160,"available":95383986176,"internally_reserved":1073741824,"allocated":179027968,"data_stored":178679726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":3.2240000000000002,"5min":3.2240000000000002,"15min":3.2240000000000002},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1.022}]},{"osd":1,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":3.1320000000000001,"5min":3.1320000000000001,"15min":3.1320000000000001},"last":1.794},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":3.3069999999999999,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":1.1930000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.3759999999999999,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.556,"5min":1.556,"15min":1.556},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":8.2210000000000001,"5min":8.2210000000000001,"15min":8.2210000000000001},"last":1.1719999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.206},{"interface":"front","average":{"1min":1.643,"5min":1.643,"15min":1.643},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":3.2749999999999999,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":0.80100000000000005}]},{"osd":5,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":2.1629999999999998},{"interface":"front","average":{"1min":25.524999999999999,"5min":25.524999999999999,"15min":25.524999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":415.40699999999998,"5min":415.40699999999998,"15min":415.40699999999998},"last":1.1339999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.252},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":1.4079999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.742,"5min":1.742,"15min":1.742},"min":{"1min":1.093,"5min":1.093,"15min":1.093},"max":{"1min":4.056,"5min":4.056,"15min":4.056},"last":3.1190000000000002},{"interface":"front","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":4.4539999999999997,"5min":4.4539999999999997,"15min":4.4539999999999997},"last":1.238}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1358024,"kb_used_data":309440,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93013816,"statfs":{"total":96636764160,"available":95246147584,"internally_reserved":1073741824,"allocated":316866560,"data_stored":316507337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":2.8220000000000001,"5min":2.8220000000000001,"15min":2.8220000000000001},"last":1.179},{"interface":"front","average":{"1min":1.637,"5min":1.637,"15min":1.637},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":13.067,"5min":13.067,"15min":13.067},"last":1.74}]},{"osd":1,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.899,"5min":1.899,"15min":1.899},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":6.6829999999999998,"5min":6.6829999999999998,"15min":6.6829999999999998},"last":1.0469999999999999},{"interface":"front","average":{"1min":2.012,"5min":2.012,"15min":2.012},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":11.936,"5min":11.936,"15min":11.936},"last":1.133}]},{"osd":2,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.589,"5min":3.589,"15min":3.589},"last":2.5630000000000002},{"interface":"front","average":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":5.5890000000000004,"5min":5.5890000000000004,"15min":5.5890000000000004},"last":1.6759999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:07:19 2020","interfaces":[{"interface":"back","average":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":4.556,"5min":4.556,"15min":4.556},"last":3.2389999999999999},{"interface":"front","average":{"1min":32.100000000000001,"5min":32.100000000000001,"15min":32.100000000000001},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":677.88599999999997,"5min":677.88599999999997,"15min":677.88599999999997},"last":2.5979999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":3.6600000000000001,"5min":3.6600000000000001,"15min":3.6600000000000001},"last":2.6400000000000001},{"interface":"front","average":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"min":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":5.4329999999999998,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":2.6960000000000002}]},{"osd":6,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":3.621,"5min":3.621,"15min":3.621},"last":2.665},{"interface":"front","average":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":5.6189999999999998,"5min":5.6189999999999998,"15min":5.6189999999999998},"last":1.7110000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":5.734,"5min":5.734,"15min":5.734},"last":2.1659999999999999},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":4.8540000000000001,"5min":4.8540000000000001,"15min":4.8540000000000001},"last":3.2719999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1406460,"kb_used_data":357900,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92965380,"statfs":{"total":96636764160,"available":95196549120,"internally_reserved":1073741824,"allocated":366489600,"data_stored":366127109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,2,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.181,"15min":1.181},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":4.9480000000000004,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":1.165},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.5289999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":4.9829999999999997,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.377,"5min":1.377,"15min":1.377},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":5.0430000000000001,"5min":5.0430000000000001,"15min":5.0430000000000001},"last":3.2010000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.452,"15min":1.452},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":4.415,"5min":4.415,"15min":4.415},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":4.4340000000000002,"5min":4.4340000000000002,"15min":4.4340000000000002},"last":3.032}]},{"osd":3,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.508,"15min":1.508},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":5.0099999999999998,"5min":5.0099999999999998,"15min":5.0099999999999998},"last":3.2429999999999999},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":3.2029999999999998,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":2.8260000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":5.157,"5min":5.157,"15min":5.157},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.627,"5min":1.627,"15min":1.627},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":5.0979999999999999,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":3.4510000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.79,"5min":1.79,"15min":1.79},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":5.1310000000000002,"5min":5.1310000000000002,"15min":5.1310000000000002},"last":3.4849999999999999},{"interface":"front","average":{"1min":19.109000000000002,"5min":19.109000000000002,"15min":19.109000000000002},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":389.14600000000002,"5min":389.14600000000002,"15min":389.14600000000002},"last":2.855}]},{"osd":7,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":3.3540000000000001,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":4.1189999999999998},{"interface":"front","average":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":4.4569999999999999,"5min":4.4569999999999999,"15min":4.4569999999999999},"last":0.95399999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1274112,"kb_used_data":225528,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93097728,"statfs":{"total":96636764160,"available":95332073472,"internally_reserved":1073741824,"allocated":230940672,"data_stored":230585644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.766,"5min":2.766,"15min":2.766},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.998}]},{"osd":1,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.6509999999999998,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":2.746,"5min":2.746,"15min":2.746},"last":0.89800000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":9.2200000000000006,"5min":9.2200000000000006,"15min":9.2200000000000006},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":168.56899999999999,"5min":168.56899999999999,"15min":168.56899999999999},"last":1.1259999999999999},{"interface":"front","average":{"1min":33.131,"5min":33.131,"15min":33.131},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":676.27099999999996,"5min":676.27099999999996,"15min":676.27099999999996},"last":0.79700000000000004}]},{"osd":3,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.7669999999999999,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":1.02},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.964,"5min":2.964,"15min":2.964},"last":0.873}]},{"osd":4,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":50.893000000000001,"5min":50.893000000000001,"15min":50.893000000000001},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":530.30799999999999,"5min":530.30799999999999,"15min":530.30799999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":3.1080000000000001,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":1.0629999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":50.408999999999999,"5min":50.408999999999999,"15min":50.408999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":754.31299999999999,"5min":754.31299999999999,"15min":754.31299999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":26.016999999999999,"5min":26.016999999999999,"15min":26.016999999999999},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":517.26700000000005,"5min":517.26700000000005,"15min":517.26700000000005},"last":1.1040000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"min":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":3.5649999999999999,"5min":3.5649999999999999,"15min":3.5649999999999999},"last":1.085}]}]}]}} 2020-08-07T15:07:40.735 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2020-08-07T15:07:40.735 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:07:40.736 INFO:tasks.thrashosds.thrasher:inject_pause on 7 2020-08-07T15:07:40.736 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2020-08-07T15:07:40.736 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2020-08-07T15:07:40.737 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2020-08-07T15:07:40.782 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:40.780+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:40.879 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:40.877+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:40.897 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:40.897 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:07:40.898 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:07:40.898 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:07:40.899 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.899 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:9352 3.e e24/20 3:7113579b:::benchmark_data_smithi074_35447_object1168:head v 24'656, mlcod=24'648)", 2020-08-07T15:07:40.899 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.889020+0000", 2020-08-07T15:07:40.899 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 59.002512906, 2020-08-07T15:07:40.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24471712600000001, 2020-08-07T15:07:40.900 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:40.901 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:40.901 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:40.902 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.902 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:40.903 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.889020+0000", 2020-08-07T15:07:40.903 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:40.903 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.904 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:40.904 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.889020+0000", 2020-08-07T15:07:40.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7401999999999998e-05 2020-08-07T15:07:40.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:40.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.889068+0000", 2020-08-07T15:07:40.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.669e-06 2020-08-07T15:07:40.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:40.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.889075+0000", 2020-08-07T15:07:40.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23823541500000001 2020-08-07T15:07:40.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:40.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.127310+0000", 2020-08-07T15:07:40.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.895e-06 2020-08-07T15:07:40.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:40.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.127314+0000", 2020-08-07T15:07:40.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3778000000000001e-05 2020-08-07T15:07:40.918 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.918 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:40.919 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.127348+0000", 2020-08-07T15:07:40.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6517000000000001e-05 2020-08-07T15:07:40.919 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:40.920 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.127364+0000", 2020-08-07T15:07:40.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063235399999999999 2020-08-07T15:07:40.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.922 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:40.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.133688+0000", 2020-08-07T15:07:40.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9910000000000002e-05 2020-08-07T15:07:40.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:40.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.133738+0000", 2020-08-07T15:07:40.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24471712600000001 2020-08-07T15:07:40.925 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.925 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:40.935 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.937 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:10240 3.6 e24/20 3:665d71ff:::benchmark_data_smithi074_35447_object1279:head v 24'728, mlcod=24'718)", 2020-08-07T15:07:40.937 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.394891+0000", 2020-08-07T15:07:40.937 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 58.496642547999997, 2020-08-07T15:07:40.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24904923400000001, 2020-08-07T15:07:40.938 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:40.940 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:40.940 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:40.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:40.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394891+0000", 2020-08-07T15:07:40.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:40.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.944 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:40.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394891+0000", 2020-08-07T15:07:40.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9815000000000003e-05 2020-08-07T15:07:40.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:40.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394941+0000", 2020-08-07T15:07:40.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5100000000000004e-06 2020-08-07T15:07:40.956 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.956 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.957 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:40.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394947+0000", 2020-08-07T15:07:40.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24715435899999999 2020-08-07T15:07:40.957 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:40.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.642101+0000", 2020-08-07T15:07:40.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7069999999999992e-06 2020-08-07T15:07:40.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:40.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.642109+0000", 2020-08-07T15:07:40.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9301000000000002e-05 2020-08-07T15:07:40.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:40.964 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.642148+0000", 2020-08-07T15:07:40.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7227999999999998e-05 2020-08-07T15:07:40.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:40.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.642166+0000", 2020-08-07T15:07:40.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017236300000000001 2020-08-07T15:07:40.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:40.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.643889+0000", 2020-08-07T15:07:40.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0683999999999998e-05 2020-08-07T15:07:40.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:40.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.643940+0000", 2020-08-07T15:07:40.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24904923400000001 2020-08-07T15:07:40.969 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.969 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:40.969 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.969 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.970 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:15576 3.e e27/20 3:727724a5:::benchmark_data_smithi074_35447_object1946:head v 27'1032, mlcod=27'1023)", 2020-08-07T15:07:40.970 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:07:40.971 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 54.222050695999997, 2020-08-07T15:07:40.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.46091973000000003, 2020-08-07T15:07:40.971 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:40.972 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:40.972 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:40.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:40.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:07:40.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:40.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:40.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:07:40.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9592000000000001e-05 2020-08-07T15:07:40.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.975 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:40.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669542+0000", 2020-08-07T15:07:40.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.113e-06 2020-08-07T15:07:40.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:40.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669547+0000", 2020-08-07T15:07:40.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45151617799999999 2020-08-07T15:07:40.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.978 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:40.979 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121064+0000", 2020-08-07T15:07:40.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.377e-06 2020-08-07T15:07:40.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:40.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121067+0000", 2020-08-07T15:07:40.982 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9195e-05 2020-08-07T15:07:40.982 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.983 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:40.983 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121106+0000", 2020-08-07T15:07:40.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7509000000000001e-05 2020-08-07T15:07:40.983 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.984 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:40.984 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121124+0000", 2020-08-07T15:07:40.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0091783100000000003 2020-08-07T15:07:40.985 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.985 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:40.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.130302+0000", 2020-08-07T15:07:40.986 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000100456 2020-08-07T15:07:40.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.986 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:40.987 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.130402+0000", 2020-08-07T15:07:40.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.46091973000000003 2020-08-07T15:07:40.987 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.988 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:40.988 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:40.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:40.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.989 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:24264 3.8 e28/20 3:1d3fdbda:::benchmark_data_smithi074_35447_object3032:head v 28'1520, mlcod=28'1515)", 2020-08-07T15:07:40.989 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:07:40.989 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 48.508420659000002, 2020-08-07T15:07:40.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41856254199999998, 2020-08-07T15:07:40.990 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:40.994 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:40.997 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:40.997 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:40.997 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:40.998 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:07:41.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:41.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:41.004 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:07:41.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1348999999999998e-05 2020-08-07T15:07:41.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:41.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383174+0000", 2020-08-07T15:07:41.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6579999999999996e-06 2020-08-07T15:07:41.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:41.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383181+0000", 2020-08-07T15:07:41.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41194352699999998 2020-08-07T15:07:41.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:41.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.795124+0000", 2020-08-07T15:07:41.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5729999999999999e-06 2020-08-07T15:07:41.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:41.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.795128+0000", 2020-08-07T15:07:41.008 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002170384 2020-08-07T15:07:41.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.009 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:41.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.797298+0000", 2020-08-07T15:07:41.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5412e-05 2020-08-07T15:07:41.009 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.010 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:41.010 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.797324+0000", 2020-08-07T15:07:41.010 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042943360000000002 2020-08-07T15:07:41.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.011 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:41.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.801618+0000", 2020-08-07T15:07:41.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7302999999999997e-05 2020-08-07T15:07:41.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:41.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.801675+0000", 2020-08-07T15:07:41.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41856254199999998 2020-08-07T15:07:41.014 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.015 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:41.015 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.015 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.016 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:43335 3.6 3:65a57ed3:::benchmark_data_smithi074_35447_object5416:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:07:41.016 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:03.997487+0000", 2020-08-07T15:07:41.016 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 36.89404648, 2020-08-07T15:07:41.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22588636300000001, 2020-08-07T15:07:41.016 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:41.017 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:41.017 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:41.017 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:41.017 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:41.017 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 43335 2020-08-07T15:07:41.018 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.018 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:41.018 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.018 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:41.018 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.997487+0000", 2020-08-07T15:07:41.019 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:41.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.019 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.019 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:41.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.997487+0000", 2020-08-07T15:07:41.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9805000000000002e-05 2020-08-07T15:07:41.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.020 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.020 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:41.020 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.997537+0000", 2020-08-07T15:07:41.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4040000000000003e-06 2020-08-07T15:07:41.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:41.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.997542+0000", 2020-08-07T15:07:41.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.061e-05 2020-08-07T15:07:41.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:41.022 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.997553+0000", 2020-08-07T15:07:41.022 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4039999999999997e-06 2020-08-07T15:07:41.023 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.023 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:41.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.997557+0000", 2020-08-07T15:07:41.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00085322200000000005 2020-08-07T15:07:41.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:41.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998410+0000", 2020-08-07T15:07:41.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9388999999999994e-05 2020-08-07T15:07:41.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:41.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998500+0000", 2020-08-07T15:07:41.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00022212999999999999 2020-08-07T15:07:41.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:41.026 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998722+0000", 2020-08-07T15:07:41.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0036367819999999999 2020-08-07T15:07:41.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:41.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.002359+0000", 2020-08-07T15:07:41.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2700000000000001e-07 2020-08-07T15:07:41.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:41.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.002359+0000", 2020-08-07T15:07:41.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5210000000000002e-05 2020-08-07T15:07:41.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:41.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.002414+0000", 2020-08-07T15:07:41.030 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0073127519999999996 2020-08-07T15:07:41.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:41.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.009727+0000", 2020-08-07T15:07:41.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21357918200000001 2020-08-07T15:07:41.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:41.033 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.223306+0000", 2020-08-07T15:07:41.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.08e-06 2020-08-07T15:07:41.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:41.034 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.223307+0000", 2020-08-07T15:07:41.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6166000000000003e-05 2020-08-07T15:07:41.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.034 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.034 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:41.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.223373+0000", 2020-08-07T15:07:41.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22588636300000001 2020-08-07T15:07:41.035 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.035 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:41.035 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.036 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:43336 3.6 3:65a57ed3:::benchmark_data_smithi074_35447_object5416:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:07:41.036 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:03.997813+0000", 2020-08-07T15:07:41.036 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 36.893720031999997, 2020-08-07T15:07:41.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22842448100000001, 2020-08-07T15:07:41.037 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:41.037 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:41.037 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:41.037 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:41.037 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:41.038 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 43336 2020-08-07T15:07:41.038 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.038 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:41.038 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:41.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.997813+0000", 2020-08-07T15:07:41.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:41.039 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.039 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.039 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:41.040 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.997813+0000", 2020-08-07T15:07:41.040 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3659000000000002e-05 2020-08-07T15:07:41.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:41.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.997867+0000", 2020-08-07T15:07:41.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9199999999999997e-06 2020-08-07T15:07:41.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.041 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:41.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.997871+0000", 2020-08-07T15:07:41.042 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6510000000000001e-06 2020-08-07T15:07:41.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.042 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:41.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.997879+0000", 2020-08-07T15:07:41.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3329999999999996e-06 2020-08-07T15:07:41.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:41.043 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.997884+0000", 2020-08-07T15:07:41.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012298560000000001 2020-08-07T15:07:41.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.044 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:41.044 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999114+0000", 2020-08-07T15:07:41.045 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4678000000000004e-05 2020-08-07T15:07:41.045 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.045 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:41.045 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999208+0000", 2020-08-07T15:07:41.046 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016877099999999999 2020-08-07T15:07:41.046 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.046 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.046 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:41.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999377+0000", 2020-08-07T15:07:41.047 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0048515140000000003 2020-08-07T15:07:41.047 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.047 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:41.047 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.004229+0000", 2020-08-07T15:07:41.047 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1100000000000005e-07 2020-08-07T15:07:41.048 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.048 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.048 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:41.048 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.004230+0000", 2020-08-07T15:07:41.048 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0355000000000002e-05 2020-08-07T15:07:41.049 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.049 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.049 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:41.049 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.004270+0000", 2020-08-07T15:07:41.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054997459999999998 2020-08-07T15:07:41.050 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.050 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.050 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:41.050 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.009770+0000", 2020-08-07T15:07:41.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21640456999999999 2020-08-07T15:07:41.050 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.051 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:41.051 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.226174+0000", 2020-08-07T15:07:41.051 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2200000000000003e-07 2020-08-07T15:07:41.051 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.052 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.052 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:41.052 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.226175+0000", 2020-08-07T15:07:41.052 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2895e-05 2020-08-07T15:07:41.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.053 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:41.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.226238+0000", 2020-08-07T15:07:41.053 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22842448100000001 2020-08-07T15:07:41.053 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.053 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:41.054 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.054 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.054 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.054 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:44120 3.e e32/20 3:7e6f7bb9:::benchmark_data_smithi074_35447_object5514:head v 32'2752, mlcod=32'2742)", 2020-08-07T15:07:41.055 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:04.453943+0000", 2020-08-07T15:07:41.055 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 36.437590432999997, 2020-08-07T15:07:41.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26204982500000001, 2020-08-07T15:07:41.055 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:41.055 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:41.055 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:41.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:41.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.453943+0000", 2020-08-07T15:07:41.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:41.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:41.057 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.453943+0000", 2020-08-07T15:07:41.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2278000000000003e-05 2020-08-07T15:07:41.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.058 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.058 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:41.058 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.453995+0000", 2020-08-07T15:07:41.058 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7050000000000002e-06 2020-08-07T15:07:41.058 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.058 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.059 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:41.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.454002+0000", 2020-08-07T15:07:41.059 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26083199299999998 2020-08-07T15:07:41.059 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.059 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:41.060 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.714834+0000", 2020-08-07T15:07:41.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3139999999999997e-06 2020-08-07T15:07:41.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:41.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.714838+0000", 2020-08-07T15:07:41.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8711000000000002e-05 2020-08-07T15:07:41.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.061 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:41.062 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.714887+0000", 2020-08-07T15:07:41.062 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7895e-05 2020-08-07T15:07:41.062 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.062 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.062 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:41.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.714915+0000", 2020-08-07T15:07:41.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010364 2020-08-07T15:07:41.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.063 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:41.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.715951+0000", 2020-08-07T15:07:41.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1529000000000002e-05 2020-08-07T15:07:41.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.064 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:41.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.715993+0000", 2020-08-07T15:07:41.065 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26204982500000001 2020-08-07T15:07:41.065 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.065 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:41.065 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.066 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.066 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:47336 3.8 e36/20 3:1652c683:::benchmark_data_smithi074_35447_object5916:head v 36'2992, mlcod=36'2984)", 2020-08-07T15:07:41.066 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:08.389243+0000", 2020-08-07T15:07:41.066 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 32.502290412000001, 2020-08-07T15:07:41.066 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22662983, 2020-08-07T15:07:41.067 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:41.067 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:41.067 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:41.067 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.067 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:41.067 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.389243+0000", 2020-08-07T15:07:41.068 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:41.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.068 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.068 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:41.068 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.389243+0000", 2020-08-07T15:07:41.069 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0009e-05 2020-08-07T15:07:41.069 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.069 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.069 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:41.069 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.389273+0000", 2020-08-07T15:07:41.069 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1219999999999997e-06 2020-08-07T15:07:41.070 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.070 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.070 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:41.070 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.389277+0000", 2020-08-07T15:07:41.070 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.225964585 2020-08-07T15:07:41.071 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.071 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.071 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:41.071 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.615242+0000", 2020-08-07T15:07:41.071 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3490000000000003e-06 2020-08-07T15:07:41.072 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.072 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.072 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:41.072 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.615249+0000", 2020-08-07T15:07:41.072 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5593000000000001e-05 2020-08-07T15:07:41.072 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.073 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.073 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:41.073 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.615305+0000", 2020-08-07T15:07:41.073 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4199e-05 2020-08-07T15:07:41.073 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.074 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.074 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:41.074 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.615329+0000", 2020-08-07T15:07:41.074 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00050547700000000005 2020-08-07T15:07:41.074 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.075 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.075 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:41.075 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.615834+0000", 2020-08-07T15:07:41.075 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8495999999999999e-05 2020-08-07T15:07:41.075 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.075 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.076 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:41.076 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.615873+0000", 2020-08-07T15:07:41.076 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22662983 2020-08-07T15:07:41.076 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.076 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:41.077 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.077 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.077 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.077 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:48144 3.0 3:0cec1d53:::benchmark_data_smithi074_35447_object6017:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e36)", 2020-08-07T15:07:41.077 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:08.892002+0000", 2020-08-07T15:07:41.078 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 31.999530913000001, 2020-08-07T15:07:41.078 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.257647242, 2020-08-07T15:07:41.078 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:41.078 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:41.078 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:41.079 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:41.079 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:41.079 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48144 2020-08-07T15:07:41.079 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.080 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:41.080 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.081 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:41.081 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.892002+0000", 2020-08-07T15:07:41.081 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:41.082 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.082 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:41.082 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.892002+0000", 2020-08-07T15:07:41.082 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5709999999999999e-05 2020-08-07T15:07:41.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.083 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.083 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:41.083 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.892028+0000", 2020-08-07T15:07:41.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9110000000000002e-06 2020-08-07T15:07:41.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.084 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:41.084 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.892031+0000", 2020-08-07T15:07:41.084 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3320000000000003e-06 2020-08-07T15:07:41.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.085 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:41.086 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.892038+0000", 2020-08-07T15:07:41.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.356e-06 2020-08-07T15:07:41.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:41.088 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.892041+0000", 2020-08-07T15:07:41.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045711179999999999 2020-08-07T15:07:41.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.089 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:41.089 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.896612+0000", 2020-08-07T15:07:41.089 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012007599999999999 2020-08-07T15:07:41.089 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.089 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.090 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:41.090 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.896732+0000", 2020-08-07T15:07:41.090 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014412499999999999 2020-08-07T15:07:41.090 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.090 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.090 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:07:41.091 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.896876+0000", 2020-08-07T15:07:41.091 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054785700000000003 2020-08-07T15:07:41.091 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.091 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.091 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:41.092 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.902355+0000", 2020-08-07T15:07:41.092 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1990000000000001e-06 2020-08-07T15:07:41.092 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.092 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.092 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:41.093 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.902356+0000", 2020-08-07T15:07:41.093 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4178999999999997e-05 2020-08-07T15:07:41.093 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.093 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.093 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:41.094 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.902420+0000", 2020-08-07T15:07:41.094 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.011879413 2020-08-07T15:07:41.094 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.094 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.094 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:41.094 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.914299+0000", 2020-08-07T15:07:41.095 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23528454700000001 2020-08-07T15:07:41.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.095 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.095 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:41.095 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.149584+0000", 2020-08-07T15:07:41.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1379999999999999e-06 2020-08-07T15:07:41.096 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.096 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.096 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:41.096 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.149585+0000", 2020-08-07T15:07:41.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4567999999999996e-05 2020-08-07T15:07:41.097 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.097 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.097 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:41.097 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.149650+0000", 2020-08-07T15:07:41.097 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.257647242 2020-08-07T15:07:41.098 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.098 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:41.098 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.098 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.098 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.099 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:64072 3.6 3:650e778f:::benchmark_data_smithi074_35447_object8008:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e41)", 2020-08-07T15:07:41.100 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:19.718097+0000", 2020-08-07T15:07:41.100 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 21.173436443, 2020-08-07T15:07:41.100 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25959740599999998, 2020-08-07T15:07:41.100 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:41.100 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:41.102 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:41.102 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:41.102 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:41.102 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 64072 2020-08-07T15:07:41.102 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.103 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:41.103 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.103 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:41.104 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.718097+0000", 2020-08-07T15:07:41.104 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:41.104 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.104 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.104 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:41.105 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.718097+0000", 2020-08-07T15:07:41.105 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0025000000000003e-05 2020-08-07T15:07:41.105 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.106 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.106 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:41.106 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.718147+0000", 2020-08-07T15:07:41.107 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2469999999999997e-06 2020-08-07T15:07:41.107 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.107 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.107 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:41.107 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.718153+0000", 2020-08-07T15:07:41.107 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3059e-05 2020-08-07T15:07:41.108 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.108 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.108 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:41.108 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.718166+0000", 2020-08-07T15:07:41.108 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4210000000000003e-06 2020-08-07T15:07:41.109 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.109 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.109 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:41.109 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.718172+0000", 2020-08-07T15:07:41.109 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002189683 2020-08-07T15:07:41.110 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.110 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.110 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:41.110 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.720361+0000", 2020-08-07T15:07:41.110 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9148999999999995e-05 2020-08-07T15:07:41.111 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.111 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.111 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:41.111 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.720460+0000", 2020-08-07T15:07:41.111 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019332600000000001 2020-08-07T15:07:41.111 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.112 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:41.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.720654+0000", 2020-08-07T15:07:41.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0064242470000000001 2020-08-07T15:07:41.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.113 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:41.113 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.727078+0000", 2020-08-07T15:07:41.113 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023503629999999998 2020-08-07T15:07:41.113 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.114 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:41.114 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.729428+0000", 2020-08-07T15:07:41.114 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24817636700000001 2020-08-07T15:07:41.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.115 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.115 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:41.115 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.977605+0000", 2020-08-07T15:07:41.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6299999999999993e-07 2020-08-07T15:07:41.115 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.115 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.116 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:41.116 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.977606+0000", 2020-08-07T15:07:41.116 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7691e-05 2020-08-07T15:07:41.116 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.116 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:41.117 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.977663+0000", 2020-08-07T15:07:41.117 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0864999999999997e-05 2020-08-07T15:07:41.117 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.117 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:41.118 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.977694+0000", 2020-08-07T15:07:41.118 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25959740599999998 2020-08-07T15:07:41.118 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.118 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:41.118 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.119 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.119 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.119 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:64072 3.6 e41/26 ondisk, result = 0)", 2020-08-07T15:07:41.119 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:19.722636+0000", 2020-08-07T15:07:41.119 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 21.168897507000001, 2020-08-07T15:07:41.120 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25506327899999998, 2020-08-07T15:07:41.120 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:41.120 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:41.120 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:41.120 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.120 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:41.121 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.722636+0000", 2020-08-07T15:07:41.121 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:41.121 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.121 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.121 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:41.122 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.722636+0000", 2020-08-07T15:07:41.122 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7552000000000002e-05 2020-08-07T15:07:41.122 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.122 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.122 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:41.123 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.722673+0000", 2020-08-07T15:07:41.123 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8279999999999999e-06 2020-08-07T15:07:41.123 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.123 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.123 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:41.124 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.722679+0000", 2020-08-07T15:07:41.124 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25481403499999999 2020-08-07T15:07:41.124 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.124 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.124 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:41.124 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.977493+0000", 2020-08-07T15:07:41.125 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0319999999999997e-06 2020-08-07T15:07:41.125 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.125 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.125 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:41.125 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.977497+0000", 2020-08-07T15:07:41.126 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8094e-05 2020-08-07T15:07:41.126 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.126 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.126 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:41.126 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.977595+0000", 2020-08-07T15:07:41.127 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6219999999999998e-06 2020-08-07T15:07:41.127 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.127 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.127 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:41.127 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.977599+0000", 2020-08-07T15:07:41.128 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000100116 2020-08-07T15:07:41.128 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.128 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.128 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:41.128 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.977699+0000", 2020-08-07T15:07:41.128 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25506327899999998 2020-08-07T15:07:41.129 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.129 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:41.129 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.129 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.129 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.130 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:71136 3.6 3:64cb7acb:::benchmark_data_smithi074_35447_object8891:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e43)", 2020-08-07T15:07:41.130 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:24.312984+0000", 2020-08-07T15:07:41.130 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.578549241000001, 2020-08-07T15:07:41.130 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26986886799999998, 2020-08-07T15:07:41.130 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:41.131 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:41.131 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:41.131 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:41.131 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:41.131 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 71136 2020-08-07T15:07:41.131 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.132 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:41.132 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.132 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:41.132 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.312984+0000", 2020-08-07T15:07:41.132 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:41.133 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.133 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.133 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:41.133 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.312984+0000", 2020-08-07T15:07:41.133 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6711999999999997e-05 2020-08-07T15:07:41.134 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.134 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.134 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:41.134 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.313021+0000", 2020-08-07T15:07:41.134 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1999999999999999e-06 2020-08-07T15:07:41.134 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.135 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.135 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:41.135 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.313024+0000", 2020-08-07T15:07:41.136 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7479999999999993e-06 2020-08-07T15:07:41.136 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.136 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.136 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:41.136 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.313032+0000", 2020-08-07T15:07:41.136 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7379999999999999e-06 2020-08-07T15:07:41.137 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.137 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.137 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:41.137 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.313035+0000", 2020-08-07T15:07:41.137 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043807819999999997 2020-08-07T15:07:41.138 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.138 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.138 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:41.138 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.317415+0000", 2020-08-07T15:07:41.138 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00042567899999999999 2020-08-07T15:07:41.139 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.139 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.139 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:41.139 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.317841+0000", 2020-08-07T15:07:41.139 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000173048 2020-08-07T15:07:41.139 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.140 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.140 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:41.140 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318014+0000", 2020-08-07T15:07:41.140 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00099031100000000001 2020-08-07T15:07:41.140 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.141 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.141 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:41.141 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.319004+0000", 2020-08-07T15:07:41.141 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.75e-07 2020-08-07T15:07:41.141 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.142 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.142 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:41.142 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.319005+0000", 2020-08-07T15:07:41.142 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0659000000000003e-05 2020-08-07T15:07:41.143 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.143 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.143 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:41.145 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.319066+0000", 2020-08-07T15:07:41.145 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023319909999999998 2020-08-07T15:07:41.145 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.146 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.146 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:41.146 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.321398+0000", 2020-08-07T15:07:41.146 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26138581700000002 2020-08-07T15:07:41.167 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.168 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.168 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:41.168 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.582783+0000", 2020-08-07T15:07:41.169 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.19e-06 2020-08-07T15:07:41.169 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.169 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.169 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:41.169 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.582785+0000", 2020-08-07T15:07:41.170 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8417999999999994e-05 2020-08-07T15:07:41.170 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.170 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.170 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:41.170 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.582853+0000", 2020-08-07T15:07:41.171 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26986886799999998 2020-08-07T15:07:41.171 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.171 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:41.171 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.171 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.172 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.172 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:72616 3.6 3:64e83676:::benchmark_data_smithi074_35447_object9076:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e44)", 2020-08-07T15:07:41.172 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:25.246439+0000", 2020-08-07T15:07:41.172 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 15.645094005000001, 2020-08-07T15:07:41.172 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27754408000000003, 2020-08-07T15:07:41.173 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:41.173 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:41.173 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:41.173 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:41.173 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:41.173 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72616 2020-08-07T15:07:41.174 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.174 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:41.174 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.174 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:41.174 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.246439+0000", 2020-08-07T15:07:41.175 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:41.175 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.175 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.175 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:41.175 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.246439+0000", 2020-08-07T15:07:41.176 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0911000000000003e-05 2020-08-07T15:07:41.176 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.176 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.176 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:41.176 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.246470+0000", 2020-08-07T15:07:41.176 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7080000000000002e-06 2020-08-07T15:07:41.177 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.177 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.177 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:41.177 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.246473+0000", 2020-08-07T15:07:41.177 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4199999999999998e-06 2020-08-07T15:07:41.178 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.178 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.178 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:41.178 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.246478+0000", 2020-08-07T15:07:41.178 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5229999999999998e-06 2020-08-07T15:07:41.178 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.179 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.179 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:41.179 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.246481+0000", 2020-08-07T15:07:41.179 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024729550000000002 2020-08-07T15:07:41.179 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.180 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.180 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:41.180 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.248954+0000", 2020-08-07T15:07:41.180 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0921999999999994e-05 2020-08-07T15:07:41.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.181 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.181 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:41.181 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.249045+0000", 2020-08-07T15:07:41.181 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000158875 2020-08-07T15:07:41.181 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:41.182 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.249204+0000", 2020-08-07T15:07:41.182 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003988068 2020-08-07T15:07:41.182 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.184 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:41.184 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.253192+0000", 2020-08-07T15:07:41.184 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7400000000000003e-07 2020-08-07T15:07:41.184 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.184 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.185 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:41.185 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.253192+0000", 2020-08-07T15:07:41.185 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7210000000000003e-05 2020-08-07T15:07:41.185 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.185 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.186 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:41.186 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.253250+0000", 2020-08-07T15:07:41.186 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00069997899999999999 2020-08-07T15:07:41.186 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.186 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.187 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:41.188 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.253949+0000", 2020-08-07T15:07:41.188 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26998122600000002 2020-08-07T15:07:41.189 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.189 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.189 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:41.189 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.523931+0000", 2020-08-07T15:07:41.190 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6339999999999999e-06 2020-08-07T15:07:41.190 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.190 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.190 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:41.202 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.523932+0000", 2020-08-07T15:07:41.202 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1075000000000001e-05 2020-08-07T15:07:41.203 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.204 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.204 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:41.204 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.523983+0000", 2020-08-07T15:07:41.204 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27754408000000003 2020-08-07T15:07:41.204 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.205 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:41.205 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.205 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.205 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:78743 3.6 3:6151aa5d:::benchmark_data_smithi074_35447_object9842:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e47)", 2020-08-07T15:07:41.206 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:29.249535+0000", 2020-08-07T15:07:41.206 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.641997888000001, 2020-08-07T15:07:41.207 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.270038371, 2020-08-07T15:07:41.207 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:41.207 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:41.211 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:41.211 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:41.211 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:41.212 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 78743 2020-08-07T15:07:41.212 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.212 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:41.212 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.212 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:41.213 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249535+0000", 2020-08-07T15:07:41.213 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:41.213 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.213 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.213 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:41.214 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249535+0000", 2020-08-07T15:07:41.214 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1631000000000001e-05 2020-08-07T15:07:41.214 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.214 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.215 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:41.215 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249587+0000", 2020-08-07T15:07:41.215 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3969999999999999e-06 2020-08-07T15:07:41.215 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.215 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.215 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:41.216 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249593+0000", 2020-08-07T15:07:41.216 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2312e-05 2020-08-07T15:07:41.216 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.216 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.216 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:41.217 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249606+0000", 2020-08-07T15:07:41.217 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7379999999999997e-06 2020-08-07T15:07:41.217 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.217 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.217 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:41.219 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249613+0000", 2020-08-07T15:07:41.219 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0018089250000000001 2020-08-07T15:07:41.220 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.220 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.220 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:41.220 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.251421+0000", 2020-08-07T15:07:41.221 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4716999999999995e-05 2020-08-07T15:07:41.221 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.221 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.221 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:41.221 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.251516+0000", 2020-08-07T15:07:41.222 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000190061 2020-08-07T15:07:41.222 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.222 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.222 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:41.223 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.251706+0000", 2020-08-07T15:07:41.223 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013539520000000001 2020-08-07T15:07:41.223 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.223 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.223 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:41.224 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.253060+0000", 2020-08-07T15:07:41.224 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019342299999999999 2020-08-07T15:07:41.224 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.224 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.225 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:41.225 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.253254+0000", 2020-08-07T15:07:41.225 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26620518599999998 2020-08-07T15:07:41.225 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.225 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.227 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:41.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.519459+0000", 2020-08-07T15:07:41.228 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2109999999999999e-06 2020-08-07T15:07:41.228 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.228 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.229 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:41.229 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.519460+0000", 2020-08-07T15:07:41.229 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6767000000000006e-05 2020-08-07T15:07:41.229 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.230 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.230 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:41.230 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.519527+0000", 2020-08-07T15:07:41.230 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7051000000000001e-05 2020-08-07T15:07:41.231 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.231 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.231 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:41.231 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.519574+0000", 2020-08-07T15:07:41.232 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.270038371 2020-08-07T15:07:41.232 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.232 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:41.232 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.233 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.233 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.233 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:78744 3.6 3:6151aa5d:::benchmark_data_smithi074_35447_object9842:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e47)", 2020-08-07T15:07:41.233 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:29.249649+0000", 2020-08-07T15:07:41.234 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.641884238999999, 2020-08-07T15:07:41.234 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27244544500000001, 2020-08-07T15:07:41.235 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:41.235 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:41.235 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:41.235 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:41.236 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:41.236 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 78744 2020-08-07T15:07:41.236 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.236 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:41.237 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.237 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:41.237 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249649+0000", 2020-08-07T15:07:41.237 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:41.237 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.238 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.238 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:41.238 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249649+0000", 2020-08-07T15:07:41.238 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6628000000000001e-05 2020-08-07T15:07:41.239 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.239 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.239 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:41.239 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249696+0000", 2020-08-07T15:07:41.240 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4209999999999998e-06 2020-08-07T15:07:41.240 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.240 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.240 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:41.241 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249703+0000", 2020-08-07T15:07:41.241 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2456999999999999e-05 2020-08-07T15:07:41.241 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.242 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.242 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:41.242 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249716+0000", 2020-08-07T15:07:41.242 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1030000000000001e-06 2020-08-07T15:07:41.243 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.243 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.243 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:41.243 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249721+0000", 2020-08-07T15:07:41.244 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024347980000000002 2020-08-07T15:07:41.244 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.245 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.246 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:41.247 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.252156+0000", 2020-08-07T15:07:41.247 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3114e-05 2020-08-07T15:07:41.247 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.247 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.248 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:41.248 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.252209+0000", 2020-08-07T15:07:41.248 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3654999999999997e-05 2020-08-07T15:07:41.249 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.249 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.249 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:41.249 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.252302+0000", 2020-08-07T15:07:41.250 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001130437 2020-08-07T15:07:41.250 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.250 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.250 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:41.251 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.253433+0000", 2020-08-07T15:07:41.251 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5849999999999997e-05 2020-08-07T15:07:41.251 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.251 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.252 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:41.252 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.253469+0000", 2020-08-07T15:07:41.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26856134399999998 2020-08-07T15:07:41.252 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.253 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:41.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.522030+0000", 2020-08-07T15:07:41.253 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0500000000000002e-07 2020-08-07T15:07:41.254 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.254 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.255 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:41.255 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.522031+0000", 2020-08-07T15:07:41.255 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8105000000000002e-05 2020-08-07T15:07:41.256 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.256 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.256 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:41.257 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.522069+0000", 2020-08-07T15:07:41.257 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5728000000000002e-05 2020-08-07T15:07:41.257 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.258 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.258 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:41.258 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.522095+0000", 2020-08-07T15:07:41.259 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27244544500000001 2020-08-07T15:07:41.259 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.259 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:41.259 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.260 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.260 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.260 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:86712 3.8 e52/20 3:14925daa:::benchmark_data_smithi074_35447_object10838:head v 52'5488, mlcod=52'5479)", 2020-08-07T15:07:41.260 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.337908+0000", 2020-08-07T15:07:41.260 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.5536249250000003, 2020-08-07T15:07:41.267 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.305870904, 2020-08-07T15:07:41.268 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:41.268 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:41.268 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:41.268 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.269 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:41.269 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.337908+0000", 2020-08-07T15:07:41.269 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:41.269 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.269 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.270 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:41.270 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.337908+0000", 2020-08-07T15:07:41.270 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1465999999999999e-05 2020-08-07T15:07:41.270 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.271 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:41.271 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.337950+0000", 2020-08-07T15:07:41.271 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1810000000000003e-06 2020-08-07T15:07:41.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:41.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.337954+0000", 2020-08-07T15:07:41.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30500923899999999 2020-08-07T15:07:41.273 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.273 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:41.273 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.642963+0000", 2020-08-07T15:07:41.274 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5389999999999998e-06 2020-08-07T15:07:41.274 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.274 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.274 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:41.275 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.642969+0000", 2020-08-07T15:07:41.275 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8265e-05 2020-08-07T15:07:41.277 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.277 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.277 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:41.278 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.643017+0000", 2020-08-07T15:07:41.278 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6812999999999999e-05 2020-08-07T15:07:41.278 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.278 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.278 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:41.279 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.643044+0000", 2020-08-07T15:07:41.279 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00067619800000000003 2020-08-07T15:07:41.280 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.280 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.280 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:41.280 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.643720+0000", 2020-08-07T15:07:41.280 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9203000000000002e-05 2020-08-07T15:07:41.281 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.282 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.282 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:41.282 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.643779+0000", 2020-08-07T15:07:41.282 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.305870904 2020-08-07T15:07:41.283 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.283 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:41.283 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.283 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.283 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.284 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:87087 3.6 3:6b0320df:::benchmark_data_smithi074_35447_object10885:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:07:41.284 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.958471+0000", 2020-08-07T15:07:41.284 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.9330619880000004, 2020-08-07T15:07:41.284 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30431862500000001, 2020-08-07T15:07:41.284 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:41.285 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:41.285 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:41.285 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:41.285 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:41.285 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 87087 2020-08-07T15:07:41.286 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.286 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:41.286 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.286 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:41.287 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958471+0000", 2020-08-07T15:07:41.289 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:41.289 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.290 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.290 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:41.290 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958471+0000", 2020-08-07T15:07:41.291 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3241999999999997e-05 2020-08-07T15:07:41.291 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.291 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.292 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:41.292 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958515+0000", 2020-08-07T15:07:41.292 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4930000000000003e-06 2020-08-07T15:07:41.292 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.292 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.293 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:41.293 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958521+0000", 2020-08-07T15:07:41.293 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2308e-05 2020-08-07T15:07:41.293 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.293 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.294 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:41.294 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958533+0000", 2020-08-07T15:07:41.294 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.105e-06 2020-08-07T15:07:41.294 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.295 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.295 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:41.295 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958540+0000", 2020-08-07T15:07:41.295 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0064000089999999999 2020-08-07T15:07:41.295 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.296 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.296 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:41.296 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.964940+0000", 2020-08-07T15:07:41.296 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000159581 2020-08-07T15:07:41.297 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.297 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.297 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:41.297 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.965099+0000", 2020-08-07T15:07:41.300 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000160743 2020-08-07T15:07:41.300 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.300 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.300 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:41.301 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.965260+0000", 2020-08-07T15:07:41.301 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002913559 2020-08-07T15:07:41.301 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.301 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.302 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:41.302 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.968173+0000", 2020-08-07T15:07:41.302 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5000000000000001e-07 2020-08-07T15:07:41.303 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.303 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.303 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:41.303 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.968174+0000", 2020-08-07T15:07:41.303 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6161999999999994e-05 2020-08-07T15:07:41.304 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.304 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.304 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:41.305 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.968241+0000", 2020-08-07T15:07:41.306 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.010959649 2020-08-07T15:07:41.306 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.306 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.306 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:41.307 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.979200+0000", 2020-08-07T15:07:41.307 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28352280699999999 2020-08-07T15:07:41.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.307 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:41.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262723+0000", 2020-08-07T15:07:41.308 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.19e-06 2020-08-07T15:07:41.308 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.309 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.309 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:41.309 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262724+0000", 2020-08-07T15:07:41.309 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5827000000000005e-05 2020-08-07T15:07:41.310 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.321 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.321 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:41.321 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262790+0000", 2020-08-07T15:07:41.321 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30431862500000001 2020-08-07T15:07:41.322 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.322 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:41.322 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.322 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.322 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.323 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:87088 3.6 3:6b0320df:::benchmark_data_smithi074_35447_object10885:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:07:41.323 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.958567+0000", 2020-08-07T15:07:41.323 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.9329663669999997, 2020-08-07T15:07:41.323 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30442327000000002, 2020-08-07T15:07:41.324 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:41.324 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:41.324 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:41.324 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:41.325 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:41.325 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 87088 2020-08-07T15:07:41.325 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.326 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:41.326 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.326 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:41.326 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958567+0000", 2020-08-07T15:07:41.327 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:41.327 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.327 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.327 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:41.328 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958567+0000", 2020-08-07T15:07:41.328 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5040000000000002e-05 2020-08-07T15:07:41.328 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.328 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.328 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:41.329 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958612+0000", 2020-08-07T15:07:41.329 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4519999999999996e-06 2020-08-07T15:07:41.329 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.329 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.329 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:41.330 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958617+0000", 2020-08-07T15:07:41.330 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3974e-05 2020-08-07T15:07:41.330 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.331 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.332 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:41.332 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958631+0000", 2020-08-07T15:07:41.332 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5619999999999997e-06 2020-08-07T15:07:41.333 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.334 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.334 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:41.334 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958636+0000", 2020-08-07T15:07:41.334 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0071073999999999998 2020-08-07T15:07:41.335 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.335 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:41.336 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.965743+0000", 2020-08-07T15:07:41.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5477999999999996e-05 2020-08-07T15:07:41.336 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.336 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.336 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:41.337 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.965839+0000", 2020-08-07T15:07:41.337 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000168061 2020-08-07T15:07:41.337 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.337 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.338 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:41.338 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966007+0000", 2020-08-07T15:07:41.338 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054997550000000003 2020-08-07T15:07:41.339 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.339 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.339 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:41.339 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971507+0000", 2020-08-07T15:07:41.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6200000000000006e-07 2020-08-07T15:07:41.340 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.340 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:41.340 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971508+0000", 2020-08-07T15:07:41.341 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012566300000000001 2020-08-07T15:07:41.341 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.341 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:41.343 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971633+0000", 2020-08-07T15:07:41.343 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0075941899999999998 2020-08-07T15:07:41.343 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.343 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.345 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:41.345 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.979228+0000", 2020-08-07T15:07:41.345 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28372433899999999 2020-08-07T15:07:41.345 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.346 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.346 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:41.346 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262952+0000", 2020-08-07T15:07:41.347 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2799999999999995e-07 2020-08-07T15:07:41.347 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.348 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:41.348 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262953+0000", 2020-08-07T15:07:41.348 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7666000000000002e-05 2020-08-07T15:07:41.350 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.350 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.350 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:41.350 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262990+0000", 2020-08-07T15:07:41.353 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30442327000000002 2020-08-07T15:07:41.364 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.365 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:41.365 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.365 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.365 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.366 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:87087 3.6 e52/26 ondisk, result = 0)", 2020-08-07T15:07:41.366 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.971041+0000", 2020-08-07T15:07:41.366 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.9204924569999999, 2020-08-07T15:07:41.366 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29175585500000001, 2020-08-07T15:07:41.367 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:41.367 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:41.367 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:41.367 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.368 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:41.368 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971041+0000", 2020-08-07T15:07:41.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:41.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.369 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:41.370 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971041+0000", 2020-08-07T15:07:41.370 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7841999999999999e-05 2020-08-07T15:07:41.370 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.370 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.371 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:41.371 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971069+0000", 2020-08-07T15:07:41.371 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7110000000000003e-06 2020-08-07T15:07:41.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.372 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:41.373 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971073+0000", 2020-08-07T15:07:41.373 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28965561299999998 2020-08-07T15:07:41.373 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.374 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.374 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:41.374 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260729+0000", 2020-08-07T15:07:41.375 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8010000000000003e-06 2020-08-07T15:07:41.375 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.376 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:41.376 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260734+0000", 2020-08-07T15:07:41.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019719500000000001 2020-08-07T15:07:41.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:41.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262706+0000", 2020-08-07T15:07:41.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7399999999999993e-06 2020-08-07T15:07:41.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:41.380 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262715+0000", 2020-08-07T15:07:41.380 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2198000000000001e-05 2020-08-07T15:07:41.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:41.382 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262797+0000", 2020-08-07T15:07:41.382 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29175585500000001 2020-08-07T15:07:41.383 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.383 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:41.383 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.384 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.384 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.385 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:93920 3.6 e56/26 ondisk, result = 0)", 2020-08-07T15:07:41.385 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:07:41.385 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 1.561057396, 2020-08-07T15:07:41.385 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.397576347, 2020-08-07T15:07:41.386 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:41.386 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:41.386 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:41.386 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.387 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:41.387 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:07:41.387 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:41.387 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.388 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.388 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:41.388 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:07:41.388 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2571999999999999e-05 2020-08-07T15:07:41.389 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.389 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.389 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:41.390 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330509+0000", 2020-08-07T15:07:41.390 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2059999999999998e-06 2020-08-07T15:07:41.390 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.391 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.393 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:41.393 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330514+0000", 2020-08-07T15:07:41.394 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39740134599999999 2020-08-07T15:07:41.394 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.395 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.395 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:41.395 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727915+0000", 2020-08-07T15:07:41.395 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.677e-06 2020-08-07T15:07:41.395 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.396 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.396 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:41.396 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727921+0000", 2020-08-07T15:07:41.396 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5431e-05 2020-08-07T15:07:41.396 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.397 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.397 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:41.397 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727956+0000", 2020-08-07T15:07:41.397 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5790000000000004e-06 2020-08-07T15:07:41.398 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.398 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.398 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:41.399 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727965+0000", 2020-08-07T15:07:41.399 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7535999999999997e-05 2020-08-07T15:07:41.399 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:41.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:41.400 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:41.400 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.728052+0000", 2020-08-07T15:07:41.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.397576347 2020-08-07T15:07:41.400 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.401 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:41.401 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.401 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:41.401 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:41.402 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:41.406 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:41.406 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2020-08-07T15:07:41.407 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:41.407 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:41.403+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:41.408 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:07:41.409 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 65 16 96692 96676 11.6187 10.7031 0.006888 0.0107547 2020-08-07T15:07:41.410 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:07:41.510 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:41.509+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:41.593 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:41.597 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:07:41.599 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:07:41.599 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:41.609 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:07:41.620 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:41.617+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:41.722 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:41.719+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:41.801 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:41.801 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:07:41.801 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:07:41.802 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:07:41.803 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:41.819 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:07:41.821 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:41.820+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:41.924 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:41.923+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:42.015 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:42.015 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:07:42.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:07:42.016 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:07:42.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.016 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:7848 3.9 e24/20 3:905b1512:::benchmark_data_smithi074_35447_object980:head v 24'408, mlcod=24'399)", 2020-08-07T15:07:42.016 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:40.898821+0000", 2020-08-07T15:07:42.017 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 61.109370179999999, 2020-08-07T15:07:42.017 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.182786849, 2020-08-07T15:07:42.017 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.017 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:42.017 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.018 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.018 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.018 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.898821+0000", 2020-08-07T15:07:42.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.019 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.019 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.898821+0000", 2020-08-07T15:07:42.019 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5016000000000002e-05 2020-08-07T15:07:42.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.020 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.020 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.020 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.898886+0000", 2020-08-07T15:07:42.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2740000000000007e-06 2020-08-07T15:07:42.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:40.898894+0000", 2020-08-07T15:07:42.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.181521612 2020-08-07T15:07:42.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.037 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.080416+0000", 2020-08-07T15:07:42.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8980000000000004e-06 2020-08-07T15:07:42.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.037 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.080421+0000", 2020-08-07T15:07:42.046 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6742000000000001e-05 2020-08-07T15:07:42.047 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.047 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.048 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.080468+0000", 2020-08-07T15:07:42.048 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3512000000000001e-05 2020-08-07T15:07:42.048 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.049 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.049 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.049 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.080491+0000", 2020-08-07T15:07:42.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010625750000000001 2020-08-07T15:07:42.054 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.054 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:42.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.081554+0000", 2020-08-07T15:07:42.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4219999999999999e-05 2020-08-07T15:07:42.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:42.057 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.081608+0000", 2020-08-07T15:07:42.058 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.182786849 2020-08-07T15:07:42.058 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.058 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.059 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.059 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.059 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.060 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:8112 3.9 e24/20 3:9b275d48:::benchmark_data_smithi074_35447_object1013:head v 24'416, mlcod=24'408)", 2020-08-07T15:07:42.060 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.085042+0000", 2020-08-07T15:07:42.060 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 60.923149727000002, 2020-08-07T15:07:42.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31443637000000002, 2020-08-07T15:07:42.061 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.061 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:42.073 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.073 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.074 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.074 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085042+0000", 2020-08-07T15:07:42.074 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.074 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.074 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.075 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.075 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085042+0000", 2020-08-07T15:07:42.075 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4534000000000003e-05 2020-08-07T15:07:42.075 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.075 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.076 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.076 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085096+0000", 2020-08-07T15:07:42.076 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5789999999999998e-06 2020-08-07T15:07:42.076 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.076 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.077 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.077 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085104+0000", 2020-08-07T15:07:42.077 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31316244700000001 2020-08-07T15:07:42.077 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.077 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.086 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.086 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398266+0000", 2020-08-07T15:07:42.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3019999999999996e-06 2020-08-07T15:07:42.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.096 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.096 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.096 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398270+0000", 2020-08-07T15:07:42.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3467000000000002e-05 2020-08-07T15:07:42.097 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.097 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.097 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.100 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398314+0000", 2020-08-07T15:07:42.102 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8184000000000001e-05 2020-08-07T15:07:42.102 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.102 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.104 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.104 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398332+0000", 2020-08-07T15:07:42.104 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010908059999999999 2020-08-07T15:07:42.104 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.105 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.105 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:42.106 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.399423+0000", 2020-08-07T15:07:42.106 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5050999999999999e-05 2020-08-07T15:07:42.106 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.106 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.106 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:42.107 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.399478+0000", 2020-08-07T15:07:42.107 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31443637000000002 2020-08-07T15:07:42.107 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.107 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.107 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.108 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.110 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.110 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:8760 3.9 e24/20 3:9f02d263:::benchmark_data_smithi074_35447_object1094:head v 24'448, mlcod=24'441)", 2020-08-07T15:07:42.110 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.535293+0000", 2020-08-07T15:07:42.110 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 60.472898534999999, 2020-08-07T15:07:42.111 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.194025799, 2020-08-07T15:07:42.111 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.111 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:42.111 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.112 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.535293+0000", 2020-08-07T15:07:42.113 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.113 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.113 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.113 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.535293+0000", 2020-08-07T15:07:42.114 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1352e-05 2020-08-07T15:07:42.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.118 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.535334+0000", 2020-08-07T15:07:42.118 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6660000000000002e-06 2020-08-07T15:07:42.119 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.120 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.120 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.120 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.535341+0000", 2020-08-07T15:07:42.120 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19233371499999999 2020-08-07T15:07:42.120 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.121 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.121 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.121 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.727675+0000", 2020-08-07T15:07:42.121 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1819999999999997e-06 2020-08-07T15:07:42.121 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.122 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.122 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.122 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.727680+0000", 2020-08-07T15:07:42.122 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0288999999999999e-05 2020-08-07T15:07:42.122 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.123 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.123 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.123 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.727730+0000", 2020-08-07T15:07:42.123 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0340000000000001e-05 2020-08-07T15:07:42.123 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.124 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.124 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.124 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.727760+0000", 2020-08-07T15:07:42.124 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015059489999999999 2020-08-07T15:07:42.124 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.124 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.125 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:42.125 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.729266+0000", 2020-08-07T15:07:42.125 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2305999999999997e-05 2020-08-07T15:07:42.149 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.149 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.149 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:42.149 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.729319+0000", 2020-08-07T15:07:42.150 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.194025799 2020-08-07T15:07:42.150 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.150 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.151 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.151 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:16223 3.9 e27/20 3:9c72cb85:::benchmark_data_smithi074_35447_object2027:head v 27'887, mlcod=27'881)", 2020-08-07T15:07:42.152 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.111666+0000", 2020-08-07T15:07:42.152 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 54.896524874999997, 2020-08-07T15:07:42.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.48138790999999997, 2020-08-07T15:07:42.153 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.154 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:42.155 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.155 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.155 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.155 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111666+0000", 2020-08-07T15:07:42.155 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.156 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.156 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111666+0000", 2020-08-07T15:07:42.156 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3168999999999999e-05 2020-08-07T15:07:42.157 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.157 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.157 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.157 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111720+0000", 2020-08-07T15:07:42.157 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6309999999999997e-06 2020-08-07T15:07:42.157 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.158 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.158 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.158 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111726+0000", 2020-08-07T15:07:42.158 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.47995092 2020-08-07T15:07:42.158 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.159 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.160 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.160 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591677+0000", 2020-08-07T15:07:42.160 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9879999999999998e-06 2020-08-07T15:07:42.160 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.160 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.161 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.161 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591684+0000", 2020-08-07T15:07:42.162 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5562000000000002e-05 2020-08-07T15:07:42.162 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.162 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.162 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.162 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591730+0000", 2020-08-07T15:07:42.163 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4731000000000001e-05 2020-08-07T15:07:42.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.163 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.163 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.163 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591754+0000", 2020-08-07T15:07:42.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012412739999999999 2020-08-07T15:07:42.164 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:42.164 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.592996+0000", 2020-08-07T15:07:42.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8634999999999997e-05 2020-08-07T15:07:42.165 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.165 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.165 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:42.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.593054+0000", 2020-08-07T15:07:42.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.48138790999999997 2020-08-07T15:07:42.166 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.166 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.166 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.166 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.166 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.167 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:16224 3.9 e27/20 3:9c72cb85:::benchmark_data_smithi074_35447_object2027:head v 27'888, mlcod=27'881)", 2020-08-07T15:07:42.167 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.114073+0000", 2020-08-07T15:07:42.167 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 54.894118147999997, 2020-08-07T15:07:42.167 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.50478892600000003, 2020-08-07T15:07:42.167 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.167 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:42.168 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.168 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.168 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.168 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114073+0000", 2020-08-07T15:07:42.168 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.169 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.169 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.169 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.170 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114073+0000", 2020-08-07T15:07:42.170 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3088999999999999e-05 2020-08-07T15:07:42.170 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.170 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.171 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.171 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114126+0000", 2020-08-07T15:07:42.171 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1629999999999999e-06 2020-08-07T15:07:42.171 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.172 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.174 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.174 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114133+0000", 2020-08-07T15:07:42.174 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.50357425199999994 2020-08-07T15:07:42.174 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.174 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.175 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.175 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617708+0000", 2020-08-07T15:07:42.175 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9160000000000004e-06 2020-08-07T15:07:42.175 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.175 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.176 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.176 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617715+0000", 2020-08-07T15:07:42.176 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8770999999999998e-05 2020-08-07T15:07:42.176 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.176 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.177 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.177 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617763+0000", 2020-08-07T15:07:42.177 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.544e-05 2020-08-07T15:07:42.177 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.177 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.178 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.178 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617789+0000", 2020-08-07T15:07:42.178 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010210359999999999 2020-08-07T15:07:42.179 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.179 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.179 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:42.179 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.618810+0000", 2020-08-07T15:07:42.179 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2259000000000002e-05 2020-08-07T15:07:42.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.180 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.181 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:42.181 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.618862+0000", 2020-08-07T15:07:42.181 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.50478892600000003 2020-08-07T15:07:42.181 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.181 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.182 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.182 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.182 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:24992 3.0 e28/20 3:008a8fe9:::benchmark_data_smithi074_35447_object3123:head v 28'1704, mlcod=28'1694)", 2020-08-07T15:07:42.182 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.852063+0000", 2020-08-07T15:07:42.183 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 49.156128013, 2020-08-07T15:07:42.183 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20540171199999999, 2020-08-07T15:07:42.183 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.184 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:42.184 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.184 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.185 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.185 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.852063+0000", 2020-08-07T15:07:42.185 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.185 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.185 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.185 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.186 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.852063+0000", 2020-08-07T15:07:42.186 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1409999999999997e-05 2020-08-07T15:07:42.186 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.186 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.186 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.187 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.852115+0000", 2020-08-07T15:07:42.187 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6100000000000002e-06 2020-08-07T15:07:42.187 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.187 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.187 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.188 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.852121+0000", 2020-08-07T15:07:42.188 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20455925999999999 2020-08-07T15:07:42.188 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.188 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.188 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.189 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.056681+0000", 2020-08-07T15:07:42.189 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8739999999999995e-06 2020-08-07T15:07:42.189 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.207 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.207 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.208 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.056689+0000", 2020-08-07T15:07:42.208 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2753999999999998e-05 2020-08-07T15:07:42.208 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.208 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.208 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.056731+0000", 2020-08-07T15:07:42.209 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9171e-05 2020-08-07T15:07:42.209 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.209 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.209 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.210 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.056750+0000", 2020-08-07T15:07:42.210 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00065542300000000003 2020-08-07T15:07:42.210 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.210 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.211 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:42.211 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.057406+0000", 2020-08-07T15:07:42.211 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9209999999999997e-05 2020-08-07T15:07:42.211 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.211 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.211 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:42.212 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.057465+0000", 2020-08-07T15:07:42.212 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20540171199999999 2020-08-07T15:07:42.213 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.213 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.213 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.214 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.215 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.215 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:34000 3.4 3:22a8b4e3:::benchmark_data_smithi074_35447_object4249:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-08-07T15:07:42.215 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:58.316850+0000", 2020-08-07T15:07:42.215 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 43.691341307000002, 2020-08-07T15:07:42.216 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19523959799999999, 2020-08-07T15:07:42.216 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.216 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:42.216 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:42.217 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:42.217 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:42.217 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 34000 2020-08-07T15:07:42.217 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.217 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.218 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.218 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.218 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316850+0000", 2020-08-07T15:07:42.218 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.218 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.219 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.219 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.219 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316850+0000", 2020-08-07T15:07:42.219 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7274999999999997e-05 2020-08-07T15:07:42.219 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.219 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.220 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.220 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316897+0000", 2020-08-07T15:07:42.220 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2249999999999999e-06 2020-08-07T15:07:42.220 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.221 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.221 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.221 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316903+0000", 2020-08-07T15:07:42.221 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0665e-05 2020-08-07T15:07:42.223 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.224 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.224 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.224 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316913+0000", 2020-08-07T15:07:42.224 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3089999999999997e-06 2020-08-07T15:07:42.225 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.225 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.225 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.225 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316918+0000", 2020-08-07T15:07:42.225 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041515279999999998 2020-08-07T15:07:42.226 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.226 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.226 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.226 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.321069+0000", 2020-08-07T15:07:42.227 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6084999999999999e-05 2020-08-07T15:07:42.227 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.227 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.227 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.227 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.321165+0000", 2020-08-07T15:07:42.228 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000185076 2020-08-07T15:07:42.228 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.228 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.228 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:07:42.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.321350+0000", 2020-08-07T15:07:42.229 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00079810799999999996 2020-08-07T15:07:42.229 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.229 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.229 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:42.229 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.322148+0000", 2020-08-07T15:07:42.230 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9299999999999997e-07 2020-08-07T15:07:42.231 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.231 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.231 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:42.231 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.322149+0000", 2020-08-07T15:07:42.231 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1654000000000003e-05 2020-08-07T15:07:42.232 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.232 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.232 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:42.232 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.322211+0000", 2020-08-07T15:07:42.232 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044003000000000002 2020-08-07T15:07:42.233 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.233 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.233 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:42.233 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.326611+0000", 2020-08-07T15:07:42.233 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18541316599999999 2020-08-07T15:07:42.234 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.234 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.234 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:42.234 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.512024+0000", 2020-08-07T15:07:42.234 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5290000000000001e-06 2020-08-07T15:07:42.235 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.235 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.235 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:42.235 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.512026+0000", 2020-08-07T15:07:42.235 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3985000000000005e-05 2020-08-07T15:07:42.235 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.236 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.236 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:42.236 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.512090+0000", 2020-08-07T15:07:42.236 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19523959799999999 2020-08-07T15:07:42.237 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.237 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.237 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.237 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.237 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.238 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:34000 3.4 e30/20 ondisk, result = 0)", 2020-08-07T15:07:42.238 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:58.324486+0000", 2020-08-07T15:07:42.238 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 43.683705379000003, 2020-08-07T15:07:42.238 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.187608989, 2020-08-07T15:07:42.239 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.239 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:42.239 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.239 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.239 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.240 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.324486+0000", 2020-08-07T15:07:42.240 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.240 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.240 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.240 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.241 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.324486+0000", 2020-08-07T15:07:42.241 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5866999999999999e-05 2020-08-07T15:07:42.241 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.241 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.241 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.242 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.324502+0000", 2020-08-07T15:07:42.242 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4119999999999999e-06 2020-08-07T15:07:42.242 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.242 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.242 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.243 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.324504+0000", 2020-08-07T15:07:42.243 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18677271500000001 2020-08-07T15:07:42.243 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.243 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.243 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.244 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.511277+0000", 2020-08-07T15:07:42.244 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5489999999999997e-06 2020-08-07T15:07:42.244 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.245 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.245 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.245 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.511283+0000", 2020-08-07T15:07:42.246 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00072544799999999996 2020-08-07T15:07:42.246 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.246 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.246 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.247 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.512008+0000", 2020-08-07T15:07:42.247 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7689999999999996e-06 2020-08-07T15:07:42.247 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.247 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.247 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.248 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.512016+0000", 2020-08-07T15:07:42.248 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9228999999999996e-05 2020-08-07T15:07:42.248 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.248 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.248 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:42.248 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.512095+0000", 2020-08-07T15:07:42.249 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.187608989 2020-08-07T15:07:42.249 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.249 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.249 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.249 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.250 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.250 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:34624 3.9 e30/20 3:946cdb13:::benchmark_data_smithi074_35447_object4327:head v 30'2016, mlcod=30'2009)", 2020-08-07T15:07:42.250 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:58.746375+0000", 2020-08-07T15:07:42.250 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 43.261816504000002, 2020-08-07T15:07:42.250 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28564679799999998, 2020-08-07T15:07:42.251 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.251 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:42.251 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.251 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.251 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.252 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746375+0000", 2020-08-07T15:07:42.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.252 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.252 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.252 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.252 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746375+0000", 2020-08-07T15:07:42.253 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3674000000000001e-05 2020-08-07T15:07:42.253 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.253 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746429+0000", 2020-08-07T15:07:42.254 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9349999999999997e-06 2020-08-07T15:07:42.254 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.254 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.254 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.254 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746435+0000", 2020-08-07T15:07:42.255 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28165817700000001 2020-08-07T15:07:42.255 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.255 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.255 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.255 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028094+0000", 2020-08-07T15:07:42.255 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.512e-06 2020-08-07T15:07:42.256 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.256 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.256 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.256 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028097+0000", 2020-08-07T15:07:42.256 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7904999999999997e-05 2020-08-07T15:07:42.257 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.257 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.257 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.257 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028135+0000", 2020-08-07T15:07:42.257 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6243e-05 2020-08-07T15:07:42.258 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.258 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.258 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.258 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028151+0000", 2020-08-07T15:07:42.258 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003818372 2020-08-07T15:07:42.258 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.259 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.259 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:42.259 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.031970+0000", 2020-08-07T15:07:42.259 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.198e-05 2020-08-07T15:07:42.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.260 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.260 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:42.260 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.032022+0000", 2020-08-07T15:07:42.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28564679799999998 2020-08-07T15:07:42.260 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.261 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.261 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.261 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.261 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.261 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:48216 3.4 3:29f3c7f3:::benchmark_data_smithi074_35447_object6026:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e36)", 2020-08-07T15:07:42.262 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:08.937978+0000", 2020-08-07T15:07:42.262 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 33.070213271999997, 2020-08-07T15:07:42.262 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17762572800000001, 2020-08-07T15:07:42.262 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.262 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:42.263 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:42.263 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:42.263 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:42.263 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48216 2020-08-07T15:07:42.264 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.264 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.265 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.267 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.268 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.937978+0000", 2020-08-07T15:07:42.268 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.269 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.269 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.270 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.270 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.937978+0000", 2020-08-07T15:07:42.270 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9499999999999997e-05 2020-08-07T15:07:42.271 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.271 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.938028+0000", 2020-08-07T15:07:42.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3929999999999999e-06 2020-08-07T15:07:42.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.273 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.274 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.938033+0000", 2020-08-07T15:07:42.274 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0659000000000001e-05 2020-08-07T15:07:42.274 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.275 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.275 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.275 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.938044+0000", 2020-08-07T15:07:42.276 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9569999999999999e-06 2020-08-07T15:07:42.276 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.277 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.277 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.277 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.938050+0000", 2020-08-07T15:07:42.278 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00063126600000000001 2020-08-07T15:07:42.278 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.278 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.279 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.279 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.938681+0000", 2020-08-07T15:07:42.280 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5762999999999998e-05 2020-08-07T15:07:42.280 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.280 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.281 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.281 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.938777+0000", 2020-08-07T15:07:42.281 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015617199999999999 2020-08-07T15:07:42.282 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.282 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.282 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:07:42.283 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.938933+0000", 2020-08-07T15:07:42.283 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0018355559999999999 2020-08-07T15:07:42.284 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.284 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.284 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:42.286 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.940768+0000", 2020-08-07T15:07:42.287 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5700000000000001e-07 2020-08-07T15:07:42.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.288 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:42.288 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.940769+0000", 2020-08-07T15:07:42.288 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0513000000000002e-05 2020-08-07T15:07:42.289 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.289 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.289 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:42.290 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.940830+0000", 2020-08-07T15:07:42.290 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0029664779999999998 2020-08-07T15:07:42.291 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.291 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.291 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:42.292 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:08.943796+0000", 2020-08-07T15:07:42.292 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17176112800000001 2020-08-07T15:07:42.292 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.293 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.293 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:42.293 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.115557+0000", 2020-08-07T15:07:42.294 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0899999999999999e-06 2020-08-07T15:07:42.295 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.295 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.295 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:42.296 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.115558+0000", 2020-08-07T15:07:42.297 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5395999999999997e-05 2020-08-07T15:07:42.297 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.297 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.298 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:42.298 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:09.115604+0000", 2020-08-07T15:07:42.298 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17762572800000001 2020-08-07T15:07:42.299 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.299 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.299 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.300 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.300 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.300 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:63016 3.4 3:218829e8:::benchmark_data_smithi074_35447_object7876:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e41)", 2020-08-07T15:07:42.301 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:18.893754+0000", 2020-08-07T15:07:42.301 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 23.114436995999998, 2020-08-07T15:07:42.301 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18301139899999999, 2020-08-07T15:07:42.302 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.302 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:42.302 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:42.303 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:42.303 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:42.304 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 63016 2020-08-07T15:07:42.304 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.304 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.305 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.305 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.305 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.893754+0000", 2020-08-07T15:07:42.306 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.306 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.306 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.307 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.307 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.893754+0000", 2020-08-07T15:07:42.307 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6196000000000001e-05 2020-08-07T15:07:42.308 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.308 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.309 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.893781+0000", 2020-08-07T15:07:42.309 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3400000000000002e-06 2020-08-07T15:07:42.309 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.310 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.310 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.310 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.893784+0000", 2020-08-07T15:07:42.311 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7850000000000003e-06 2020-08-07T15:07:42.312 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.312 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.313 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.313 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.893791+0000", 2020-08-07T15:07:42.313 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3210000000000001e-06 2020-08-07T15:07:42.314 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.314 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.314 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.315 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.893794+0000", 2020-08-07T15:07:42.315 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00053106699999999998 2020-08-07T15:07:42.315 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.316 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.316 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.316 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.894325+0000", 2020-08-07T15:07:42.317 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1616000000000001e-05 2020-08-07T15:07:42.317 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.318 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.318 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.318 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.894387+0000", 2020-08-07T15:07:42.318 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000104379 2020-08-07T15:07:42.319 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.319 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.319 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:07:42.319 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.894491+0000", 2020-08-07T15:07:42.320 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0018720900000000001 2020-08-07T15:07:42.320 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.320 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.320 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:42.320 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.896363+0000", 2020-08-07T15:07:42.321 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2999999999999999e-07 2020-08-07T15:07:42.321 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.321 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.321 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:42.321 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.896363+0000", 2020-08-07T15:07:42.322 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7368e-05 2020-08-07T15:07:42.322 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.322 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.322 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:42.322 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.896391+0000", 2020-08-07T15:07:42.323 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004266527 2020-08-07T15:07:42.323 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.323 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.323 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:42.323 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.900657+0000", 2020-08-07T15:07:42.324 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.176062467 2020-08-07T15:07:42.324 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.324 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.324 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:42.324 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.076720+0000", 2020-08-07T15:07:42.325 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.147e-06 2020-08-07T15:07:42.325 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.325 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.325 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:42.325 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.076721+0000", 2020-08-07T15:07:42.326 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4866e-05 2020-08-07T15:07:42.326 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.326 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.326 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:42.326 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.076766+0000", 2020-08-07T15:07:42.327 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18301139899999999 2020-08-07T15:07:42.327 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.327 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.327 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.328 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.328 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.328 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:63016 3.4 e41/20 ondisk, result = 0)", 2020-08-07T15:07:42.328 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:18.896305+0000", 2020-08-07T15:07:42.328 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 23.111886859999998, 2020-08-07T15:07:42.329 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18046616200000001, 2020-08-07T15:07:42.329 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.329 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:42.329 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.329 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.330 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.330 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.896305+0000", 2020-08-07T15:07:42.330 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.330 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.330 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.330 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.331 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.896305+0000", 2020-08-07T15:07:42.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9275999999999999e-05 2020-08-07T15:07:42.331 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.332 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.333 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.333 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.896334+0000", 2020-08-07T15:07:42.333 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8609999999999997e-06 2020-08-07T15:07:42.333 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.333 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.334 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.334 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:18.896339+0000", 2020-08-07T15:07:42.334 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18032953400000001 2020-08-07T15:07:42.334 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.334 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.335 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.335 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.076668+0000", 2020-08-07T15:07:42.335 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.053e-06 2020-08-07T15:07:42.335 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.335 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.336 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.076672+0000", 2020-08-07T15:07:42.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3062999999999998e-05 2020-08-07T15:07:42.336 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.336 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.336 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.337 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.076705+0000", 2020-08-07T15:07:42.337 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6040000000000004e-06 2020-08-07T15:07:42.337 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.337 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.337 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.337 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.076713+0000", 2020-08-07T15:07:42.338 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7771e-05 2020-08-07T15:07:42.338 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.338 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.338 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:42.338 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:19.076771+0000", 2020-08-07T15:07:42.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18046616200000001 2020-08-07T15:07:42.339 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.339 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.339 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.339 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.340 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:64688 3.4 3:221784a5:::benchmark_data_smithi074_35447_object8085:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e41)", 2020-08-07T15:07:42.340 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:20.149588+0000", 2020-08-07T15:07:42.340 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 21.858602873999999, 2020-08-07T15:07:42.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20387735500000001, 2020-08-07T15:07:42.341 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.341 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:42.341 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:42.341 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:42.341 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:42.341 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 64688 2020-08-07T15:07:42.342 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.342 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.342 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.342 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.342 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.149588+0000", 2020-08-07T15:07:42.343 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.343 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.343 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.343 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.343 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.149588+0000", 2020-08-07T15:07:42.344 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0269999999999998e-05 2020-08-07T15:07:42.344 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.344 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.344 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.344 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.149639+0000", 2020-08-07T15:07:42.344 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1489999999999999e-06 2020-08-07T15:07:42.345 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.345 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.346 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.347 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.149645+0000", 2020-08-07T15:07:42.347 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1691999999999999e-05 2020-08-07T15:07:42.347 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.347 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.348 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.149657+0000", 2020-08-07T15:07:42.348 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0409e-05 2020-08-07T15:07:42.348 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.348 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.348 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.349 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.149667+0000", 2020-08-07T15:07:42.349 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00061991800000000001 2020-08-07T15:07:42.349 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.349 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.349 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.350 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.150287+0000", 2020-08-07T15:07:42.350 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8415e-05 2020-08-07T15:07:42.350 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.350 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.350 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.351 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.150375+0000", 2020-08-07T15:07:42.351 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000180729 2020-08-07T15:07:42.351 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.351 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.351 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:07:42.352 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.150556+0000", 2020-08-07T15:07:42.352 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0052814489999999997 2020-08-07T15:07:42.352 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.352 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.352 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:42.353 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.155838+0000", 2020-08-07T15:07:42.353 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6000000000000003e-07 2020-08-07T15:07:42.353 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.353 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.367 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:42.367 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.155838+0000", 2020-08-07T15:07:42.367 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5048999999999993e-05 2020-08-07T15:07:42.367 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.368 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.368 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:42.368 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.155903+0000", 2020-08-07T15:07:42.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18448816500000001 2020-08-07T15:07:42.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.369 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:42.369 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.340391+0000", 2020-08-07T15:07:42.369 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.013000151 2020-08-07T15:07:42.370 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.370 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.370 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:42.370 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.353392+0000", 2020-08-07T15:07:42.370 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0580000000000001e-06 2020-08-07T15:07:42.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.371 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.371 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:42.371 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.353393+0000", 2020-08-07T15:07:42.371 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3140999999999996e-05 2020-08-07T15:07:42.372 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.372 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:42.372 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.353466+0000", 2020-08-07T15:07:42.372 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20387735500000001 2020-08-07T15:07:42.373 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.373 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.373 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.373 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.374 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.374 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:64688 3.4 e41/20 ondisk, result = 0)", 2020-08-07T15:07:42.374 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:20.154166+0000", 2020-08-07T15:07:42.374 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 21.854025619000002, 2020-08-07T15:07:42.374 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18622893900000001, 2020-08-07T15:07:42.375 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.375 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:42.375 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.375 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.375 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.154166+0000", 2020-08-07T15:07:42.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.376 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.376 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.154166+0000", 2020-08-07T15:07:42.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3429999999999997e-05 2020-08-07T15:07:42.377 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.154199+0000", 2020-08-07T15:07:42.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.792e-06 2020-08-07T15:07:42.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.154204+0000", 2020-08-07T15:07:42.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.186071086 2020-08-07T15:07:42.379 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.340275+0000", 2020-08-07T15:07:42.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2570000000000007e-06 2020-08-07T15:07:42.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.380 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.340283+0000", 2020-08-07T15:07:42.380 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5538000000000007e-05 2020-08-07T15:07:42.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.340369+0000", 2020-08-07T15:07:42.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1582e-05 2020-08-07T15:07:42.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.382 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.382 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.382 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.340380+0000", 2020-08-07T15:07:42.382 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4253999999999999e-05 2020-08-07T15:07:42.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.383 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.383 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:42.383 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.340395+0000", 2020-08-07T15:07:42.383 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18622893900000001 2020-08-07T15:07:42.383 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.384 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.384 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.385 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.385 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.385 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:69728 3.4 3:2b6ce21b:::benchmark_data_smithi074_35447_object8715:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e43)", 2020-08-07T15:07:42.391 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:23.422315+0000", 2020-08-07T15:07:42.392 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 18.585876021000001, 2020-08-07T15:07:42.392 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19060555100000001, 2020-08-07T15:07:42.392 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.392 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:42.393 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:42.393 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:42.393 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:42.393 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 69728 2020-08-07T15:07:42.393 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.394 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.394 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.394 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.394 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:23.422315+0000", 2020-08-07T15:07:42.394 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.395 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.395 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.395 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.395 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:23.422315+0000", 2020-08-07T15:07:42.395 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5822000000000001e-05 2020-08-07T15:07:42.395 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.396 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.396 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.396 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:23.422341+0000", 2020-08-07T15:07:42.396 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1070000000000001e-06 2020-08-07T15:07:42.396 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.397 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.397 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.397 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:23.422344+0000", 2020-08-07T15:07:42.397 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9380000000000003e-06 2020-08-07T15:07:42.397 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.398 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.398 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.398 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:23.422351+0000", 2020-08-07T15:07:42.398 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.861e-06 2020-08-07T15:07:42.398 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.399 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.399 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:23.422355+0000", 2020-08-07T15:07:42.399 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019616619999999999 2020-08-07T15:07:42.399 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.400 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.400 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:23.424317+0000", 2020-08-07T15:07:42.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2576000000000004e-05 2020-08-07T15:07:42.400 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.401 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.401 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:23.424389+0000", 2020-08-07T15:07:42.401 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016833899999999999 2020-08-07T15:07:42.401 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.401 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.402 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:07:42.402 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:23.424558+0000", 2020-08-07T15:07:42.402 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002343103 2020-08-07T15:07:42.402 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.402 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.402 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:42.403 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:23.426901+0000", 2020-08-07T15:07:42.403 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7000000000000003e-07 2020-08-07T15:07:42.403 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.403 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.403 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:42.404 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:23.426902+0000", 2020-08-07T15:07:42.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3007999999999998e-05 2020-08-07T15:07:42.404 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.404 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.405 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:42.405 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:23.426955+0000", 2020-08-07T15:07:42.405 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.072434643000000007 2020-08-07T15:07:42.405 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.406 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:42.407 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:23.499389+0000", 2020-08-07T15:07:42.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.113464353 2020-08-07T15:07:42.407 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.407 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.407 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:42.408 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:23.612854+0000", 2020-08-07T15:07:42.408 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0100000000000001e-06 2020-08-07T15:07:42.408 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.408 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.408 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:42.409 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:23.612855+0000", 2020-08-07T15:07:42.409 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6259000000000004e-05 2020-08-07T15:07:42.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.409 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.409 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:42.409 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:23.612921+0000", 2020-08-07T15:07:42.410 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19060555100000001 2020-08-07T15:07:42.410 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.410 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.411 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.411 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.412 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.412 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:78016 3.9 e46/20 3:921b23e9:::benchmark_data_smithi074_35447_object9751:head v 46'4512, mlcod=46'4506)", 2020-08-07T15:07:42.412 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:28.734571+0000", 2020-08-07T15:07:42.412 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 13.273620102000001, 2020-08-07T15:07:42.412 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23216004300000001, 2020-08-07T15:07:42.413 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.413 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:42.413 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.413 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.414 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.734571+0000", 2020-08-07T15:07:42.414 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.414 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.414 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.414 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.415 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.734571+0000", 2020-08-07T15:07:42.415 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7376000000000003e-05 2020-08-07T15:07:42.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.415 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.415 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.734609+0000", 2020-08-07T15:07:42.416 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3640000000000004e-06 2020-08-07T15:07:42.416 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.416 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.416 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.416 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.734615+0000", 2020-08-07T15:07:42.417 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.228692642 2020-08-07T15:07:42.417 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.417 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.417 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.417 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.963308+0000", 2020-08-07T15:07:42.418 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9639999999999997e-06 2020-08-07T15:07:42.418 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.418 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.418 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.418 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.963312+0000", 2020-08-07T15:07:42.419 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5215000000000001e-05 2020-08-07T15:07:42.419 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.419 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.420 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.420 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.963347+0000", 2020-08-07T15:07:42.420 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8610999999999999e-05 2020-08-07T15:07:42.420 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.420 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.421 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.421 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.963365+0000", 2020-08-07T15:07:42.421 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033060149999999998 2020-08-07T15:07:42.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.421 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.422 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:42.422 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.966671+0000", 2020-08-07T15:07:42.422 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9855999999999997e-05 2020-08-07T15:07:42.422 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.423 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:42.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.966731+0000", 2020-08-07T15:07:42.423 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23216004300000001 2020-08-07T15:07:42.423 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.423 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.423 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.424 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.425 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:86415 3.4 3:2516cb0e:::benchmark_data_smithi074_35447_object10801:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e51)", 2020-08-07T15:07:42.425 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.113622+0000", 2020-08-07T15:07:42.425 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 7.8945689469999998, 2020-08-07T15:07:42.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18193646199999999, 2020-08-07T15:07:42.425 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.426 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:42.426 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:42.426 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:42.426 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:42.426 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 86415 2020-08-07T15:07:42.427 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.427 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.427 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.113622+0000", 2020-08-07T15:07:42.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.113622+0000", 2020-08-07T15:07:42.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2716999999999997e-05 2020-08-07T15:07:42.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.429 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.113675+0000", 2020-08-07T15:07:42.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2900000000000002e-06 2020-08-07T15:07:42.430 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.430 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.430 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.430 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.113680+0000", 2020-08-07T15:07:42.430 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0011e-05 2020-08-07T15:07:42.430 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.431 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.431 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.113690+0000", 2020-08-07T15:07:42.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1999999999999996e-06 2020-08-07T15:07:42.431 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.432 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.432 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.432 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.113695+0000", 2020-08-07T15:07:42.432 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0078346300000000004 2020-08-07T15:07:42.432 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.433 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.433 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.433 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.121529+0000", 2020-08-07T15:07:42.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000110596 2020-08-07T15:07:42.433 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.433 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.434 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.434 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.121640+0000", 2020-08-07T15:07:42.434 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00025067500000000001 2020-08-07T15:07:42.434 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.434 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.435 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:07:42.435 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.121891+0000", 2020-08-07T15:07:42.435 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002320749 2020-08-07T15:07:42.435 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.435 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.436 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:42.436 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.124211+0000", 2020-08-07T15:07:42.436 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0040629280000000004 2020-08-07T15:07:42.436 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.436 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.436 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:42.437 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.128274+0000", 2020-08-07T15:07:42.437 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16718783800000001 2020-08-07T15:07:42.437 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.437 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.437 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:42.438 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.295462+0000", 2020-08-07T15:07:42.438 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1100000000000005e-07 2020-08-07T15:07:42.438 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.438 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.438 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:42.439 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.295463+0000", 2020-08-07T15:07:42.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6582999999999996e-05 2020-08-07T15:07:42.439 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.440 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:42.440 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.295529+0000", 2020-08-07T15:07:42.440 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9434e-05 2020-08-07T15:07:42.441 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.441 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.441 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:42.441 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.295559+0000", 2020-08-07T15:07:42.441 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18193646199999999 2020-08-07T15:07:42.442 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.442 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.442 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.442 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.443 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.443 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:86416 3.4 3:2516cb0e:::benchmark_data_smithi074_35447_object10801:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e51)", 2020-08-07T15:07:42.443 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.113726+0000", 2020-08-07T15:07:42.443 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 7.8944655570000002, 2020-08-07T15:07:42.443 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.182016452, 2020-08-07T15:07:42.444 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.444 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:42.444 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:42.444 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:42.444 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:42.444 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 86416 2020-08-07T15:07:42.445 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.445 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.445 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.445 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.445 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.113726+0000", 2020-08-07T15:07:42.446 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.446 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.446 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.446 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.447 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.113726+0000", 2020-08-07T15:07:42.447 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00092336199999999995 2020-08-07T15:07:42.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.447 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.447 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.114649+0000", 2020-08-07T15:07:42.448 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.566e-06 2020-08-07T15:07:42.448 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.114654+0000", 2020-08-07T15:07:42.449 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2649e-05 2020-08-07T15:07:42.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.449 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.449 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.449 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.114666+0000", 2020-08-07T15:07:42.449 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2559999999999998e-06 2020-08-07T15:07:42.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.450 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.450 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.114670+0000", 2020-08-07T15:07:42.450 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0077534750000000001 2020-08-07T15:07:42.451 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.451 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.451 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.122423+0000", 2020-08-07T15:07:42.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4734999999999993e-05 2020-08-07T15:07:42.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.453 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.453 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.122518+0000", 2020-08-07T15:07:42.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016884199999999999 2020-08-07T15:07:42.454 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.454 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.454 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:07:42.454 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.122687+0000", 2020-08-07T15:07:42.454 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0020836919999999998 2020-08-07T15:07:42.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:42.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.124770+0000", 2020-08-07T15:07:42.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003510549 2020-08-07T15:07:42.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:42.456 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.128281+0000", 2020-08-07T15:07:42.456 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.16739548900000001 2020-08-07T15:07:42.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:42.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.295676+0000", 2020-08-07T15:07:42.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6899999999999997e-07 2020-08-07T15:07:42.458 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.458 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.458 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:42.458 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.295677+0000", 2020-08-07T15:07:42.458 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8934999999999999e-05 2020-08-07T15:07:42.458 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:42.459 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.295716+0000", 2020-08-07T15:07:42.459 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6333e-05 2020-08-07T15:07:42.459 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.460 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.460 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:42.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.295742+0000", 2020-08-07T15:07:42.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.182016452 2020-08-07T15:07:42.460 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.461 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.461 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.461 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.461 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.461 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:96015 3.4 3:2bc10706:::benchmark_data_smithi074_35447_object12001:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e56)", 2020-08-07T15:07:42.462 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:40.755925+0000", 2020-08-07T15:07:42.462 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 1.252266189, 2020-08-07T15:07:42.462 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20304256600000001, 2020-08-07T15:07:42.463 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.463 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:42.463 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:42.463 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:42.463 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:42.464 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 96015 2020-08-07T15:07:42.464 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.464 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.464 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.464 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.465 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.755925+0000", 2020-08-07T15:07:42.465 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.465 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.465 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.465 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.466 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.755925+0000", 2020-08-07T15:07:42.466 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0268000000000001e-05 2020-08-07T15:07:42.466 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.466 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.466 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.467 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.755965+0000", 2020-08-07T15:07:42.467 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5780000000000004e-06 2020-08-07T15:07:42.467 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.467 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.467 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.467 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.755972+0000", 2020-08-07T15:07:42.468 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1082e-05 2020-08-07T15:07:42.468 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.468 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.468 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.468 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.755983+0000", 2020-08-07T15:07:42.469 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3429999999999998e-06 2020-08-07T15:07:42.469 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.469 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.469 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.469 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.755987+0000", 2020-08-07T15:07:42.470 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034211799999999998 2020-08-07T15:07:42.470 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.470 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.470 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.470 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.759409+0000", 2020-08-07T15:07:42.470 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1285000000000004e-05 2020-08-07T15:07:42.471 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.471 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.471 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.471 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.759500+0000", 2020-08-07T15:07:42.471 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000155134 2020-08-07T15:07:42.472 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.472 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.472 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:07:42.472 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.759655+0000", 2020-08-07T15:07:42.472 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0031454679999999998 2020-08-07T15:07:42.473 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.473 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.473 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:42.473 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.762801+0000", 2020-08-07T15:07:42.473 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7299999999999996e-07 2020-08-07T15:07:42.474 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.474 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.474 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:42.474 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.762801+0000", 2020-08-07T15:07:42.474 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9682999999999998e-05 2020-08-07T15:07:42.474 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.475 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.475 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:42.475 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.762861+0000", 2020-08-07T15:07:42.475 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.10233099900000001 2020-08-07T15:07:42.475 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.476 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.476 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:42.476 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.865192+0000", 2020-08-07T15:07:42.476 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.093708455999999996 2020-08-07T15:07:42.476 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.476 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.477 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:42.477 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.958900+0000", 2020-08-07T15:07:42.477 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3600000000000002e-07 2020-08-07T15:07:42.477 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.477 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.478 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:42.478 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.958901+0000", 2020-08-07T15:07:42.478 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6581000000000005e-05 2020-08-07T15:07:42.478 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.478 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.479 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:42.479 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.958968+0000", 2020-08-07T15:07:42.479 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20304256600000001 2020-08-07T15:07:42.479 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.479 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.480 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.480 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.480 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:96016 3.4 3:2bc10706:::benchmark_data_smithi074_35447_object12001:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e56)", 2020-08-07T15:07:42.480 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:40.757829+0000", 2020-08-07T15:07:42.480 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 1.2503626160000001, 2020-08-07T15:07:42.481 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.201245428, 2020-08-07T15:07:42.481 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.481 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:42.481 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:42.482 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:42.482 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:42.482 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 96016 2020-08-07T15:07:42.482 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.483 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.483 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.484 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.484 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.757829+0000", 2020-08-07T15:07:42.484 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.484 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.484 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.485 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.485 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.757829+0000", 2020-08-07T15:07:42.485 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00035768300000000002 2020-08-07T15:07:42.485 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.485 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.486 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.486 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.758186+0000", 2020-08-07T15:07:42.486 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2670000000000003e-06 2020-08-07T15:07:42.486 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.486 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.487 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.487 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.758193+0000", 2020-08-07T15:07:42.487 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3022e-05 2020-08-07T15:07:42.487 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.487 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.488 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.758206+0000", 2020-08-07T15:07:42.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9680000000000003e-06 2020-08-07T15:07:42.488 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.489 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.758212+0000", 2020-08-07T15:07:42.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038084989999999999 2020-08-07T15:07:42.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.489 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.489 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.490 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.762020+0000", 2020-08-07T15:07:42.490 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7879000000000003e-05 2020-08-07T15:07:42.490 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.490 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.490 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.491 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.762118+0000", 2020-08-07T15:07:42.491 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017300900000000001 2020-08-07T15:07:42.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.491 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.491 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:07:42.492 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.762291+0000", 2020-08-07T15:07:42.492 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026504240000000002 2020-08-07T15:07:42.492 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.492 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.492 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:42.493 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.764941+0000", 2020-08-07T15:07:42.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8700000000000004e-07 2020-08-07T15:07:42.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.493 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:42.494 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.764942+0000", 2020-08-07T15:07:42.494 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3740999999999998e-05 2020-08-07T15:07:42.494 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.495 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:42.495 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.765006+0000", 2020-08-07T15:07:42.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.109174536 2020-08-07T15:07:42.495 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.496 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.496 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:42.496 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.874181+0000", 2020-08-07T15:07:42.496 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.084829826999999997 2020-08-07T15:07:42.496 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.496 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.497 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:42.497 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.959010+0000", 2020-08-07T15:07:42.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9800000000000004e-07 2020-08-07T15:07:42.497 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.497 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.498 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:42.498 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.959011+0000", 2020-08-07T15:07:42.498 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3188000000000006e-05 2020-08-07T15:07:42.498 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.498 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.499 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:42.499 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.959074+0000", 2020-08-07T15:07:42.499 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.201245428 2020-08-07T15:07:42.499 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.499 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.499 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.500 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.500 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.500 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:42.501 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:07:42.503 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:07:42.503 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 66 15 98509 98494 11.6579 14.2031 0.00577629 0.0107174 2020-08-07T15:07:42.504 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:42.500+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:42.602 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:42.601+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:42.710 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:42.708+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:42.723 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:42.723 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:07:42.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.724 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:99306 3.5 e57/20 3:a43ada7b:::benchmark_data_smithi074_35447_object12413:head v 57'6034, mlcod=57'6031)", 2020-08-07T15:07:42.724 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:42.716961+0000", 2020-08-07T15:07:42.724 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0041112509999999998, 2020-08-07T15:07:42.724 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041370790000000001, 2020-08-07T15:07:42.725 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.725 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:42.727 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.727 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.728 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.716961+0000", 2020-08-07T15:07:42.728 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.729 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.729 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.729 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.716961+0000", 2020-08-07T15:07:42.730 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9499e-05 2020-08-07T15:07:42.730 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.730 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.730 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.731 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.716990+0000", 2020-08-07T15:07:42.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2640000000000002e-06 2020-08-07T15:07:42.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.732 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.734 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.734 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.716995+0000", 2020-08-07T15:07:42.734 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010605440000000001 2020-08-07T15:07:42.735 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.735 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.735 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.737 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.718055+0000", 2020-08-07T15:07:42.741 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1930000000000001e-06 2020-08-07T15:07:42.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.742 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.742 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.742 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.718060+0000", 2020-08-07T15:07:42.742 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7242000000000003e-05 2020-08-07T15:07:42.743 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.743 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.743 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.718138+0000", 2020-08-07T15:07:42.744 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2759999999999999e-05 2020-08-07T15:07:42.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.718160+0000", 2020-08-07T15:07:42.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011995020000000001 2020-08-07T15:07:42.745 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.745 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.758 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.760 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:99224 3.7 3:e43b6cdf:::benchmark_data_smithi074_35447_object12402:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e57)", 2020-08-07T15:07:42.760 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:42.664415+0000", 2020-08-07T15:07:42.760 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.056657073000000002, 2020-08-07T15:07:42.760 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.056767840999999999, 2020-08-07T15:07:42.761 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.761 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:07:42.761 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:42.761 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:42.762 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:42.762 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 99224 2020-08-07T15:07:42.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.762 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.762 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.763 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.664415+0000", 2020-08-07T15:07:42.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.763 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.764 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.664415+0000", 2020-08-07T15:07:42.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4811000000000002e-05 2020-08-07T15:07:42.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.765 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.765 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.664460+0000", 2020-08-07T15:07:42.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0250000000000002e-06 2020-08-07T15:07:42.766 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.766 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.767 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.664465+0000", 2020-08-07T15:07:42.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0576e-05 2020-08-07T15:07:42.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.768 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.664476+0000", 2020-08-07T15:07:42.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8389999999999997e-06 2020-08-07T15:07:42.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.779 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.664480+0000", 2020-08-07T15:07:42.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0029280220000000002 2020-08-07T15:07:42.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.779 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.780 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.667408+0000", 2020-08-07T15:07:42.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000101384 2020-08-07T15:07:42.780 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.781 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.781 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.667510+0000", 2020-08-07T15:07:42.781 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017426799999999999 2020-08-07T15:07:42.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.782 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:07:42.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.667684+0000", 2020-08-07T15:07:42.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004271731 2020-08-07T15:07:42.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:42.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.671956+0000", 2020-08-07T15:07:42.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0031664919999999999 2020-08-07T15:07:42.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:42.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.675122+0000", 2020-08-07T15:07:42.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.010707148 2020-08-07T15:07:42.806 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.806 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.806 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.807 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:99307 3.5 e57/20 3:a43ada7b:::benchmark_data_smithi074_35447_object12413:head v 57'6035, mlcod=57'6031)", 2020-08-07T15:07:42.807 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:42.717964+0000", 2020-08-07T15:07:42.807 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0031081419999999999, 2020-08-07T15:07:42.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032765350000000001, 2020-08-07T15:07:42.808 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.808 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:42.808 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.808 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.809 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.717964+0000", 2020-08-07T15:07:42.809 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.810 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.810 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.717964+0000", 2020-08-07T15:07:42.810 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5575000000000002e-05 2020-08-07T15:07:42.810 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.811 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.811 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.718020+0000", 2020-08-07T15:07:42.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5939999999999999e-06 2020-08-07T15:07:42.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.718026+0000", 2020-08-07T15:07:42.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00072513999999999999 2020-08-07T15:07:42.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.813 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.718751+0000", 2020-08-07T15:07:42.813 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4000000000000003e-06 2020-08-07T15:07:42.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.814 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.814 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.718759+0000", 2020-08-07T15:07:42.814 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9688999999999999e-05 2020-08-07T15:07:42.814 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.814 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.814 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.815 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.718798+0000", 2020-08-07T15:07:42.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1928000000000001e-05 2020-08-07T15:07:42.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.815 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.718820+0000", 2020-08-07T15:07:42.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00085632600000000003 2020-08-07T15:07:42.816 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.816 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.816 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.817 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:99308 3.5 e57/20 3:a43ada7b:::benchmark_data_smithi074_35447_object12413:head v 57'6036, mlcod=57'6031)", 2020-08-07T15:07:42.817 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:42.718646+0000", 2020-08-07T15:07:42.817 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.002426175, 2020-08-07T15:07:42.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002628624, 2020-08-07T15:07:42.818 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.819 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:42.819 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.819 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.718646+0000", 2020-08-07T15:07:42.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.718646+0000", 2020-08-07T15:07:42.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0903999999999998e-05 2020-08-07T15:07:42.821 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.821 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.718707+0000", 2020-08-07T15:07:42.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7800000000000003e-06 2020-08-07T15:07:42.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.718714+0000", 2020-08-07T15:07:42.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000814561 2020-08-07T15:07:42.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.719528+0000", 2020-08-07T15:07:42.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6320000000000008e-06 2020-08-07T15:07:42.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.719536+0000", 2020-08-07T15:07:42.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3405000000000002e-05 2020-08-07T15:07:42.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.826 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.719579+0000", 2020-08-07T15:07:42.826 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5794000000000001e-05 2020-08-07T15:07:42.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.829 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.830 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.830 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.719605+0000", 2020-08-07T15:07:42.830 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000959076 2020-08-07T15:07:42.830 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.830 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.831 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.831 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.831 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.831 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:99309 3.5 e57/20)", 2020-08-07T15:07:42.831 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:42.719405+0000", 2020-08-07T15:07:42.832 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.00166708, 2020-08-07T15:07:42.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001902611, 2020-08-07T15:07:42.832 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.832 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:07:42.832 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.719405+0000", 2020-08-07T15:07:42.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.719405+0000", 2020-08-07T15:07:42.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9402999999999999e-05 2020-08-07T15:07:42.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.836 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.836 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.719475+0000", 2020-08-07T15:07:42.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5290000000000008e-06 2020-08-07T15:07:42.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.719483+0000", 2020-08-07T15:07:42.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00081789999999999999 2020-08-07T15:07:42.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.838 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.838 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.720301+0000", 2020-08-07T15:07:42.838 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3519999999999997e-06 2020-08-07T15:07:42.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.839 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.720306+0000", 2020-08-07T15:07:42.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00090118399999999997 2020-08-07T15:07:42.839 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.839 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.840 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.840 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.840 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.840 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:99310 3.5 e57/20)", 2020-08-07T15:07:42.852 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:42.720229+0000", 2020-08-07T15:07:42.852 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.00084324499999999995, 2020-08-07T15:07:42.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001104343, 2020-08-07T15:07:42.853 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.853 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:07:42.853 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.854 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.854 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.720229+0000", 2020-08-07T15:07:42.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.854 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.855 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.856 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.720229+0000", 2020-08-07T15:07:42.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6950999999999999e-05 2020-08-07T15:07:42.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.856 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.720266+0000", 2020-08-07T15:07:42.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3990000000000001e-06 2020-08-07T15:07:42.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.858 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.858 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.720272+0000", 2020-08-07T15:07:42.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00063505600000000005 2020-08-07T15:07:42.858 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.859 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.859 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.720907+0000", 2020-08-07T15:07:42.859 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0090000000000001e-06 2020-08-07T15:07:42.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.859 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.720911+0000", 2020-08-07T15:07:42.860 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00068241500000000002 2020-08-07T15:07:42.860 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.860 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.861 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.861 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:99278 3.2 3:41c8d219:::benchmark_data_smithi074_35447_object12409:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e57)", 2020-08-07T15:07:42.861 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:42.698740+0000", 2020-08-07T15:07:42.862 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.022332088999999999, 2020-08-07T15:07:42.862 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.022624459, 2020-08-07T15:07:42.862 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.862 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:42.863 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:42.863 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:42.863 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:42.863 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 99278 2020-08-07T15:07:42.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.864 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.866 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.698740+0000", 2020-08-07T15:07:42.866 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.866 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.867 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.867 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.867 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.698740+0000", 2020-08-07T15:07:42.867 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4489e-05 2020-08-07T15:07:42.867 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.868 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.868 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.868 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.698795+0000", 2020-08-07T15:07:42.868 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.311e-06 2020-08-07T15:07:42.868 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.869 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.869 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.869 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.698802+0000", 2020-08-07T15:07:42.869 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7679000000000001e-05 2020-08-07T15:07:42.870 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.870 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.870 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.870 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.698820+0000", 2020-08-07T15:07:42.870 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1580000000000002e-06 2020-08-07T15:07:42.871 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.871 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.871 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.698826+0000", 2020-08-07T15:07:42.871 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00030759199999999998 2020-08-07T15:07:42.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.872 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.872 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.872 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.699133+0000", 2020-08-07T15:07:42.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6947999999999997e-05 2020-08-07T15:07:42.873 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.873 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.873 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.699200+0000", 2020-08-07T15:07:42.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013033599999999999 2020-08-07T15:07:42.874 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.874 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.874 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:42.874 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.699331+0000", 2020-08-07T15:07:42.874 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032221670000000002 2020-08-07T15:07:42.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:42.875 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.702553+0000", 2020-08-07T15:07:42.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2900000000000003e-07 2020-08-07T15:07:42.876 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.877 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:42.877 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.702553+0000", 2020-08-07T15:07:42.877 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9206000000000003e-05 2020-08-07T15:07:42.877 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.878 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.878 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:42.878 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.702603+0000", 2020-08-07T15:07:42.878 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.014120832 2020-08-07T15:07:42.878 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.879 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:42.879 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.716723+0000", 2020-08-07T15:07:42.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.017983347 2020-08-07T15:07:42.879 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.880 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.880 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.880 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.880 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:99279 3.2 3:41c8d219:::benchmark_data_smithi074_35447_object12409:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e57)", 2020-08-07T15:07:42.881 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:42.698864+0000", 2020-08-07T15:07:42.881 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.022207891, 2020-08-07T15:07:42.881 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.022568122999999999, 2020-08-07T15:07:42.881 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.881 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:42.882 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:42.882 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:42.882 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:42.882 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 99279 2020-08-07T15:07:42.883 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.883 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.883 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.883 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.883 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.698864+0000", 2020-08-07T15:07:42.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.884 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.884 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.698864+0000", 2020-08-07T15:07:42.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0518e-05 2020-08-07T15:07:42.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.885 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.698915+0000", 2020-08-07T15:07:42.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7389999999999996e-06 2020-08-07T15:07:42.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.886 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.887 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.698921+0000", 2020-08-07T15:07:42.887 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3072e-05 2020-08-07T15:07:42.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.698934+0000", 2020-08-07T15:07:42.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.485e-06 2020-08-07T15:07:42.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.698939+0000", 2020-08-07T15:07:42.890 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00077173300000000003 2020-08-07T15:07:42.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.891 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.891 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.699711+0000", 2020-08-07T15:07:42.891 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010624 2020-08-07T15:07:42.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.892 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.892 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.892 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.699817+0000", 2020-08-07T15:07:42.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017705600000000001 2020-08-07T15:07:42.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.893 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:42.893 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.699994+0000", 2020-08-07T15:07:42.893 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026156679999999998 2020-08-07T15:07:42.894 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.894 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.894 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:42.894 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.702610+0000", 2020-08-07T15:07:42.894 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.099e-06 2020-08-07T15:07:42.895 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.895 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.895 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:42.895 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.702611+0000", 2020-08-07T15:07:42.896 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6199999999999997e-05 2020-08-07T15:07:42.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.896 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:42.896 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.702667+0000", 2020-08-07T15:07:42.897 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.014082905 2020-08-07T15:07:42.897 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.898 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.898 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:42.898 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.716750+0000", 2020-08-07T15:07:42.898 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.017885715 2020-08-07T15:07:42.899 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.899 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.899 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.899 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.899 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.900 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:99280 3.2 3:41c8d219:::benchmark_data_smithi074_35447_object12409:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e57)", 2020-08-07T15:07:42.900 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:42.698977+0000", 2020-08-07T15:07:42.900 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.022095317999999999, 2020-08-07T15:07:42.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.022545374, 2020-08-07T15:07:42.901 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.901 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:42.901 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:42.901 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:42.901 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:42.902 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 99280 2020-08-07T15:07:42.902 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.902 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.902 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.903 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.903 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.698977+0000", 2020-08-07T15:07:42.903 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.903 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.904 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.904 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.698977+0000", 2020-08-07T15:07:42.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2763999999999998e-05 2020-08-07T15:07:42.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.699030+0000", 2020-08-07T15:07:42.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.48e-06 2020-08-07T15:07:42.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.906 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.906 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.906 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.699035+0000", 2020-08-07T15:07:42.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1494999999999999e-05 2020-08-07T15:07:42.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.699047+0000", 2020-08-07T15:07:42.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.763e-06 2020-08-07T15:07:42.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.699051+0000", 2020-08-07T15:07:42.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00129054 2020-08-07T15:07:42.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.910 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.700342+0000", 2020-08-07T15:07:42.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011335700000000001 2020-08-07T15:07:42.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.700455+0000", 2020-08-07T15:07:42.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012254900000000001 2020-08-07T15:07:42.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:42.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.700578+0000", 2020-08-07T15:07:42.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0020971710000000001 2020-08-07T15:07:42.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:42.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.702675+0000", 2020-08-07T15:07:42.914 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9599999999999999e-07 2020-08-07T15:07:42.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:42.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.702676+0000", 2020-08-07T15:07:42.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3411000000000001e-05 2020-08-07T15:07:42.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:42.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.702729+0000", 2020-08-07T15:07:42.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.014040604999999999 2020-08-07T15:07:42.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:42.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.716770+0000", 2020-08-07T15:07:42.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.017792830999999999 2020-08-07T15:07:42.918 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.918 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.919 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.919 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.920 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:99231 3.8 3:1dd05e77:::benchmark_data_smithi074_35447_object12403:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e57)", 2020-08-07T15:07:42.920 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:42.666843+0000", 2020-08-07T15:07:42.920 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.054228819999999997, 2020-08-07T15:07:42.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.054829206999999998, 2020-08-07T15:07:42.921 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.921 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:42.922 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:42.922 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:42.922 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:42.923 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 99231 2020-08-07T15:07:42.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.923 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.666843+0000", 2020-08-07T15:07:42.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.666843+0000", 2020-08-07T15:07:42.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8296e-05 2020-08-07T15:07:42.925 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.926 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.926 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.666892+0000", 2020-08-07T15:07:42.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1200000000000001e-06 2020-08-07T15:07:42.926 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.666897+0000", 2020-08-07T15:07:42.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8700000000000004e-06 2020-08-07T15:07:42.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.666907+0000", 2020-08-07T15:07:42.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4100000000000001e-06 2020-08-07T15:07:42.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.947 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.666911+0000", 2020-08-07T15:07:42.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002088838 2020-08-07T15:07:42.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.947 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.948 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:42.948 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.669000+0000", 2020-08-07T15:07:42.948 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010493300000000001 2020-08-07T15:07:42.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.949 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:42.949 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.669105+0000", 2020-08-07T15:07:42.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017290200000000001 2020-08-07T15:07:42.949 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:07:42.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.669278+0000", 2020-08-07T15:07:42.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0072958329999999998 2020-08-07T15:07:42.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:42.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.676574+0000", 2020-08-07T15:07:42.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6700000000000001e-06 2020-08-07T15:07:42.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:42.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.676575+0000", 2020-08-07T15:07:42.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0615000000000001e-05 2020-08-07T15:07:42.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:42.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.676646+0000", 2020-08-07T15:07:42.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0090892549999999992 2020-08-07T15:07:42.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.978 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:42.978 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.685735+0000", 2020-08-07T15:07:42.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.018891742 2020-08-07T15:07:42.979 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.980 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:42.981 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:42.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.982 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:99232 3.8 3:1dd05e77:::benchmark_data_smithi074_35447_object12403:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e57)", 2020-08-07T15:07:42.982 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:42.666943+0000", 2020-08-07T15:07:42.982 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.054129328999999997, 2020-08-07T15:07:42.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.054851798, 2020-08-07T15:07:42.983 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:42.983 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:42.984 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:42.984 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:42.984 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:42.985 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 99232 2020-08-07T15:07:42.985 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.985 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:42.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.986 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:42.987 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.666943+0000", 2020-08-07T15:07:42.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:42.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:42.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.666943+0000", 2020-08-07T15:07:42.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9908999999999999e-05 2020-08-07T15:07:42.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:42.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.666993+0000", 2020-08-07T15:07:42.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3199999999999999e-06 2020-08-07T15:07:42.990 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:42.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.666998+0000", 2020-08-07T15:07:42.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0674e-05 2020-08-07T15:07:42.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.993 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:42.993 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.667009+0000", 2020-08-07T15:07:42.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.493e-06 2020-08-07T15:07:42.994 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.994 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:42.995 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:42.995 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.667013+0000", 2020-08-07T15:07:42.995 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003140173 2020-08-07T15:07:42.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:42.996 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:43.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:43.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.670153+0000", 2020-08-07T15:07:43.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2635999999999999e-05 2020-08-07T15:07:43.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:43.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:43.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:43.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.670246+0000", 2020-08-07T15:07:43.008 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015670300000000001 2020-08-07T15:07:43.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:43.009 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:43.009 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:07:43.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.670403+0000", 2020-08-07T15:07:43.010 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0062499610000000001 2020-08-07T15:07:43.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:43.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:43.011 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:43.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.676653+0000", 2020-08-07T15:07:43.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.111e-06 2020-08-07T15:07:43.012 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:43.012 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:43.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:43.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.676654+0000", 2020-08-07T15:07:43.013 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7525000000000002e-05 2020-08-07T15:07:43.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:43.014 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:43.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:43.014 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.676711+0000", 2020-08-07T15:07:43.015 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0090643089999999996 2020-08-07T15:07:43.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:43.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:43.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:43.017 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.685776+0000", 2020-08-07T15:07:43.017 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.018832814 2020-08-07T15:07:43.017 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:43.018 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:43.019 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:43.019 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:43.019 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:07:43.020 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 11 2020-08-07T15:07:43.020 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:43.025 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:07:43.026 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:07:43.027 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:07:43.027 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:07:43.027 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:07:43.027 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:07:43.028 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:07:43.028 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:07:43.028 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:07:43.028 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:07:43.029 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:07:43.029 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:07:43.029 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:07:43.029 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:07:43.029 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:07:43.030 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:07:43.030 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:07:43.030 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:43.020+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:43.123 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:43.122+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:43.224 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:43.223+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:43.233 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:43.233 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:43.246 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:07:43.247 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 67 16 100133 100117 11.6731 12.6797 0.00397244 0.0107042 2020-08-07T15:07:43.326 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:43.323+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:43.422 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:07:43.424 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:43.424 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:43.425 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:07:43.467 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:43.464+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:43.527 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:43.525+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:43.624 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:43.624 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:43.628 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:43.627+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:43.633 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:07:43.729 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:43.728+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:43.830 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:43.828+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:43.861 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:43.861 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:43.877 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:07:43.973 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:43.970+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:44.033 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:44.031+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:44.100 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:44.101 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:44.114 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:07:44.158 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:44.153+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:44.242 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:44.237+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:44.246 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 68 16 101667 101651 11.6777 11.9844 0.00799108 0.0106996 2020-08-07T15:07:44.334 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:07:44.342 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:44.340+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:44.354 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:44.355 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:44.369 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:07:44.444 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:44.442+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:44.548 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:44.543+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:44.562 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:44.563 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:44.579 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:07:44.666 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:44.664+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:44.749 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:44.744+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:44.798 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:44.798 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:44.804 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:07:44.852 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:44.850+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:44.958 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:44.957+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:45.022 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:45.022 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:45.033 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:07:45.035 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:07:45.096 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:45.092+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:45.171 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:45.166+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:45.235 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:07:45.246 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 69 15 103064 103049 11.6667 10.9219 0.0105524 0.0107099 2020-08-07T15:07:45.262 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:45.262 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:45.278 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:07:45.278 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:45.271+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:45.280 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:07:45.309 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:07:45.356 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:07:45.373 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:45.372+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:45.392 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:07:45.432 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:07:45.468 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:07:45.476 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:45.474+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:45.508 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:45.509 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:07:45.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:07:45.509 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:45.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.509 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:11192 3.d e24/20 3:bb990748:::benchmark_data_smithi074_35447_object1398:head v 24'680, mlcod=24'671)", 2020-08-07T15:07:45.510 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:43.006852+0000", 2020-08-07T15:07:45.510 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.498205572000003, 2020-08-07T15:07:45.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15926784399999999, 2020-08-07T15:07:45.511 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:45.511 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:45.511 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:45.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:45.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.006852+0000", 2020-08-07T15:07:45.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:45.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:45.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.006852+0000", 2020-08-07T15:07:45.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0237e-05 2020-08-07T15:07:45.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:45.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.006902+0000", 2020-08-07T15:07:45.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2750000000000001e-06 2020-08-07T15:07:45.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:45.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.006908+0000", 2020-08-07T15:07:45.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15839128599999999 2020-08-07T15:07:45.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:45.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.165300+0000", 2020-08-07T15:07:45.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4450000000000004e-06 2020-08-07T15:07:45.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:45.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.165305+0000", 2020-08-07T15:07:45.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3010999999999999e-05 2020-08-07T15:07:45.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:45.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.165348+0000", 2020-08-07T15:07:45.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3289e-05 2020-08-07T15:07:45.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:45.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.165371+0000", 2020-08-07T15:07:45.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00069550299999999998 2020-08-07T15:07:45.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:45.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.166067+0000", 2020-08-07T15:07:45.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2797999999999998e-05 2020-08-07T15:07:45.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:45.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:43.166120+0000", 2020-08-07T15:07:45.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15926784399999999 2020-08-07T15:07:45.523 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.524 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:45.524 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.524 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:17824 3.d e27/20 3:bd4e0935:::benchmark_data_smithi074_35447_object2227:head v 27'1168, mlcod=27'1163)", 2020-08-07T15:07:45.525 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:07:45.525 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 57.361617055000004, 2020-08-07T15:07:45.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25249224199999998, 2020-08-07T15:07:45.525 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:45.525 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:45.526 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:45.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:45.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:07:45.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:45.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:45.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:07:45.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9348000000000003e-05 2020-08-07T15:07:45.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:45.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143490+0000", 2020-08-07T15:07:45.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7800000000000003e-06 2020-08-07T15:07:45.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:45.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143496+0000", 2020-08-07T15:07:45.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25095783399999999 2020-08-07T15:07:45.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:45.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394454+0000", 2020-08-07T15:07:45.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6550000000000001e-06 2020-08-07T15:07:45.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:45.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394460+0000", 2020-08-07T15:07:45.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8814999999999999e-05 2020-08-07T15:07:45.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:45.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394509+0000", 2020-08-07T15:07:45.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3482999999999999e-05 2020-08-07T15:07:45.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:45.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394532+0000", 2020-08-07T15:07:45.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013496199999999999 2020-08-07T15:07:45.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:45.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.395882+0000", 2020-08-07T15:07:45.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0707000000000001e-05 2020-08-07T15:07:45.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:45.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.395932+0000", 2020-08-07T15:07:45.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25249224199999998 2020-08-07T15:07:45.537 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.537 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:45.537 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.538 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:18478 3.a 3:524412cd:::benchmark_data_smithi074_35447_object2309:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:45.538 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.592837+0000", 2020-08-07T15:07:45.538 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.912219976000003, 2020-08-07T15:07:45.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21005600699999999, 2020-08-07T15:07:45.538 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:45.539 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:45.539 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:45.539 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:45.539 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:45.539 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 18478 2020-08-07T15:07:45.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.540 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:45.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:45.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592837+0000", 2020-08-07T15:07:45.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:45.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:45.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592837+0000", 2020-08-07T15:07:45.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6718999999999998e-05 2020-08-07T15:07:45.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:45.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592884+0000", 2020-08-07T15:07:45.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2749999999999997e-06 2020-08-07T15:07:45.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:45.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592888+0000", 2020-08-07T15:07:45.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0831e-05 2020-08-07T15:07:45.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:45.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592899+0000", 2020-08-07T15:07:45.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9440000000000002e-06 2020-08-07T15:07:45.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:45.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592905+0000", 2020-08-07T15:07:45.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00088115000000000003 2020-08-07T15:07:45.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:45.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.593786+0000", 2020-08-07T15:07:45.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1789000000000002e-05 2020-08-07T15:07:45.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:45.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.593838+0000", 2020-08-07T15:07:45.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011097799999999999 2020-08-07T15:07:45.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:45.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.593949+0000", 2020-08-07T15:07:45.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00088680400000000004 2020-08-07T15:07:45.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:45.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.594836+0000", 2020-08-07T15:07:45.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1500000000000005e-07 2020-08-07T15:07:45.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:45.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.594836+0000", 2020-08-07T15:07:45.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9746000000000002e-05 2020-08-07T15:07:45.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:45.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.594876+0000", 2020-08-07T15:07:45.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045589070000000001 2020-08-07T15:07:45.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:45.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.599435+0000", 2020-08-07T15:07:45.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.203361244 2020-08-07T15:07:45.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:45.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802796+0000", 2020-08-07T15:07:45.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.178e-06 2020-08-07T15:07:45.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:45.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802797+0000", 2020-08-07T15:07:45.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5927000000000005e-05 2020-08-07T15:07:45.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:45.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802893+0000", 2020-08-07T15:07:45.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21005600699999999 2020-08-07T15:07:45.556 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.557 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:45.557 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.557 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:18478 3.a e27/20 ondisk, result = 0)", 2020-08-07T15:07:45.558 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.595603+0000", 2020-08-07T15:07:45.558 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.909454078000003, 2020-08-07T15:07:45.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.207367254, 2020-08-07T15:07:45.558 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:45.558 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:45.558 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:45.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:45.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.595603+0000", 2020-08-07T15:07:45.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:45.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:45.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.595603+0000", 2020-08-07T15:07:45.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8887999999999999e-05 2020-08-07T15:07:45.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:45.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.595622+0000", 2020-08-07T15:07:45.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8469999999999998e-06 2020-08-07T15:07:45.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:45.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.595626+0000", 2020-08-07T15:07:45.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20636312700000001 2020-08-07T15:07:45.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:45.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.801989+0000", 2020-08-07T15:07:45.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0889999999999999e-06 2020-08-07T15:07:45.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:45.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.801994+0000", 2020-08-07T15:07:45.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00078913099999999999 2020-08-07T15:07:45.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:45.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802783+0000", 2020-08-07T15:07:45.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8919999999999997e-06 2020-08-07T15:07:45.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:45.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802789+0000", 2020-08-07T15:07:45.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018128 2020-08-07T15:07:45.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:45.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802970+0000", 2020-08-07T15:07:45.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.207367254 2020-08-07T15:07:45.567 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.567 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:45.568 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.568 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:33488 3.c e30/20 3:3d1929c7:::benchmark_data_smithi074_35447_object4185:head v 30'2120, mlcod=30'2110)", 2020-08-07T15:07:45.568 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:58.023101+0000", 2020-08-07T15:07:45.568 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.481956287000003, 2020-08-07T15:07:45.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22118623300000001, 2020-08-07T15:07:45.569 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:45.569 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:45.569 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:45.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:45.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023101+0000", 2020-08-07T15:07:45.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:45.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:45.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023101+0000", 2020-08-07T15:07:45.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8794000000000001e-05 2020-08-07T15:07:45.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:45.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023150+0000", 2020-08-07T15:07:45.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.545e-06 2020-08-07T15:07:45.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:45.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023156+0000", 2020-08-07T15:07:45.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.218926757 2020-08-07T15:07:45.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:45.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242083+0000", 2020-08-07T15:07:45.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1200000000000004e-06 2020-08-07T15:07:45.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:45.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242087+0000", 2020-08-07T15:07:45.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9561999999999998e-05 2020-08-07T15:07:45.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:45.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242127+0000", 2020-08-07T15:07:45.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3655999999999998e-05 2020-08-07T15:07:45.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:45.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242150+0000", 2020-08-07T15:07:45.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002089273 2020-08-07T15:07:45.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:45.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.244240+0000", 2020-08-07T15:07:45.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7525999999999999e-05 2020-08-07T15:07:45.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:45.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.244287+0000", 2020-08-07T15:07:45.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22118623300000001 2020-08-07T15:07:45.580 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.580 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:45.580 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.581 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:35856 3.a 3:5b5556e8:::benchmark_data_smithi074_35447_object4481:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-08-07T15:07:45.581 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:59.440060+0000", 2020-08-07T15:07:45.581 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.064997231, 2020-08-07T15:07:45.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16587832199999999, 2020-08-07T15:07:45.582 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:45.583 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:45.583 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:45.583 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:45.583 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:45.584 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 35856 2020-08-07T15:07:45.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.585 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:45.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:45.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440060+0000", 2020-08-07T15:07:45.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:45.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:45.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440060+0000", 2020-08-07T15:07:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018524399999999999 2020-08-07T15:07:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440245+0000", 2020-08-07T15:07:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9519999999999999e-06 2020-08-07T15:07:45.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:45.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440251+0000", 2020-08-07T15:07:45.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1705999999999999e-05 2020-08-07T15:07:45.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:45.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440263+0000", 2020-08-07T15:07:45.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2820000000000004e-06 2020-08-07T15:07:45.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:45.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.440268+0000", 2020-08-07T15:07:45.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023611449999999998 2020-08-07T15:07:45.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:45.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.442629+0000", 2020-08-07T15:07:45.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010644799999999999 2020-08-07T15:07:45.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:45.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.442736+0000", 2020-08-07T15:07:45.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017566000000000001 2020-08-07T15:07:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.442912+0000", 2020-08-07T15:07:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026960399999999998 2020-08-07T15:07:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.445608+0000", 2020-08-07T15:07:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0809999999999999e-06 2020-08-07T15:07:45.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:45.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.445609+0000", 2020-08-07T15:07:45.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0878999999999997e-05 2020-08-07T15:07:45.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:45.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.445680+0000", 2020-08-07T15:07:45.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00065560499999999997 2020-08-07T15:07:45.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:45.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.446335+0000", 2020-08-07T15:07:45.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.159534921 2020-08-07T15:07:45.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:45.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.605870+0000", 2020-08-07T15:07:45.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.187e-06 2020-08-07T15:07:45.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:45.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.605871+0000", 2020-08-07T15:07:45.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7172e-05 2020-08-07T15:07:45.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:45.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.605938+0000", 2020-08-07T15:07:45.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16587832199999999 2020-08-07T15:07:45.603 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.603 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:45.604 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.605 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:42104 3.d e32/20 3:bcc35439:::benchmark_data_smithi074_35447_object5262:head v 32'2864, mlcod=32'2858)", 2020-08-07T15:07:45.605 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:07:45.605 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.262031757999999, 2020-08-07T15:07:45.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28296418899999998, 2020-08-07T15:07:45.605 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:45.606 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:45.606 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:45.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:45.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:07:45.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:45.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:45.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:07:45.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5550000000000002e-05 2020-08-07T15:07:45.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:45.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243081+0000", 2020-08-07T15:07:45.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8139999999999997e-06 2020-08-07T15:07:45.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:45.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243087+0000", 2020-08-07T15:07:45.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27968446800000002 2020-08-07T15:07:45.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:45.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522771+0000", 2020-08-07T15:07:45.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9459999999999997e-06 2020-08-07T15:07:45.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:45.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522776+0000", 2020-08-07T15:07:45.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0778000000000006e-05 2020-08-07T15:07:45.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:45.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522847+0000", 2020-08-07T15:07:45.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0622e-05 2020-08-07T15:07:45.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:45.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522868+0000", 2020-08-07T15:07:45.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030627580000000001 2020-08-07T15:07:45.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:45.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.525930+0000", 2020-08-07T15:07:45.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9252999999999997e-05 2020-08-07T15:07:45.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:45.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.525990+0000", 2020-08-07T15:07:45.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28296418899999998 2020-08-07T15:07:45.616 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.616 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:45.617 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.617 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:42704 3.2 e32/20 3:435b82af:::benchmark_data_smithi074_35447_object5337:head v 32'2440, mlcod=32'2430)", 2020-08-07T15:07:45.618 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.590578+0000", 2020-08-07T15:07:45.618 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.914479272000001, 2020-08-07T15:07:45.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.156827727, 2020-08-07T15:07:45.619 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:45.619 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:45.619 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:45.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:45.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.590578+0000", 2020-08-07T15:07:45.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:45.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:45.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.590578+0000", 2020-08-07T15:07:45.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0890000000000002e-05 2020-08-07T15:07:45.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:45.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.590629+0000", 2020-08-07T15:07:45.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9349999999999999e-06 2020-08-07T15:07:45.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:45.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.590635+0000", 2020-08-07T15:07:45.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15515162199999999 2020-08-07T15:07:45.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:45.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.745786+0000", 2020-08-07T15:07:45.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6650000000000001e-06 2020-08-07T15:07:45.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:45.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.745790+0000", 2020-08-07T15:07:45.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1458999999999997e-05 2020-08-07T15:07:45.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:45.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.745832+0000", 2020-08-07T15:07:45.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7506000000000001e-05 2020-08-07T15:07:45.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:45.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.745849+0000", 2020-08-07T15:07:45.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001453008 2020-08-07T15:07:45.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:45.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.747302+0000", 2020-08-07T15:07:45.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010364199999999999 2020-08-07T15:07:45.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:45.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.747406+0000", 2020-08-07T15:07:45.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.156827727 2020-08-07T15:07:45.646 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.647 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:45.647 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.647 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:44744 3.2 e32/20 3:4dba534b:::benchmark_data_smithi074_35447_object5592:head v 32'2608, mlcod=32'2602)", 2020-08-07T15:07:45.648 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:04.868608+0000", 2020-08-07T15:07:45.649 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.636449564000003, 2020-08-07T15:07:45.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16039398499999999, 2020-08-07T15:07:45.650 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:45.650 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:45.650 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:45.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:45.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.868608+0000", 2020-08-07T15:07:45.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:45.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:45.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.868608+0000", 2020-08-07T15:07:45.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1788e-05 2020-08-07T15:07:45.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:45.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.868660+0000", 2020-08-07T15:07:45.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2920000000000001e-06 2020-08-07T15:07:45.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:45.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.868666+0000", 2020-08-07T15:07:45.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15842231200000001 2020-08-07T15:07:45.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:05.027088+0000", 2020-08-07T15:07:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5329999999999996e-06 2020-08-07T15:07:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:45.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:05.027093+0000", 2020-08-07T15:07:45.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0534999999999998e-05 2020-08-07T15:07:45.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:45.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:05.027123+0000", 2020-08-07T15:07:45.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6172999999999999e-05 2020-08-07T15:07:45.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:45.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:05.027139+0000", 2020-08-07T15:07:45.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001801813 2020-08-07T15:07:45.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:45.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:05.028941+0000", 2020-08-07T15:07:45.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0538999999999998e-05 2020-08-07T15:07:45.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:45.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:05.029002+0000", 2020-08-07T15:07:45.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16039398499999999 2020-08-07T15:07:45.671 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.671 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:45.671 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.674 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:48144 3.0 e36/20 3:0cec1d53:::benchmark_data_smithi074_35447_object6017:head v 36'3128, mlcod=36'3120)", 2020-08-07T15:07:45.674 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:08.897123+0000", 2020-08-07T15:07:45.674 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.607934047999997, 2020-08-07T15:07:45.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25221023399999998, 2020-08-07T15:07:45.675 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:45.675 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:45.676 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:45.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:45.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897123+0000", 2020-08-07T15:07:45.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:45.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:45.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897123+0000", 2020-08-07T15:07:45.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0655999999999997e-05 2020-08-07T15:07:45.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:45.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897174+0000", 2020-08-07T15:07:45.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6769999999999998e-06 2020-08-07T15:07:45.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:45.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897181+0000", 2020-08-07T15:07:45.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25090537899999998 2020-08-07T15:07:45.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:45.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148086+0000", 2020-08-07T15:07:45.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5520000000000004e-06 2020-08-07T15:07:45.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:45.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148092+0000", 2020-08-07T15:07:45.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6391999999999999e-05 2020-08-07T15:07:45.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:45.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148138+0000", 2020-08-07T15:07:45.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8124999999999999e-05 2020-08-07T15:07:45.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:45.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148166+0000", 2020-08-07T15:07:45.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011152009999999999 2020-08-07T15:07:45.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:45.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.149281+0000", 2020-08-07T15:07:45.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2252e-05 2020-08-07T15:07:45.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:45.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.149333+0000", 2020-08-07T15:07:45.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25221023399999998 2020-08-07T15:07:45.687 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.687 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:45.688 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.688 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:48776 3.2 e36/20 3:47029331:::benchmark_data_smithi074_35447_object6096:head v 36'2888, mlcod=36'2881)", 2020-08-07T15:07:45.688 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:07:45.689 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.100537232999997, 2020-08-07T15:07:45.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27006165399999998, 2020-08-07T15:07:45.689 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:45.689 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:45.689 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:45.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:45.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:07:45.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:45.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:45.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:07:45.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8454000000000001e-05 2020-08-07T15:07:45.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:45.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404569+0000", 2020-08-07T15:07:45.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2349999999999996e-06 2020-08-07T15:07:45.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:45.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404575+0000", 2020-08-07T15:07:45.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26921690999999998 2020-08-07T15:07:45.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:45.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673792+0000", 2020-08-07T15:07:45.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3330000000000002e-06 2020-08-07T15:07:45.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:45.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673798+0000", 2020-08-07T15:07:45.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1013000000000001e-05 2020-08-07T15:07:45.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:45.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673849+0000", 2020-08-07T15:07:45.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7044999999999999e-05 2020-08-07T15:07:45.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:45.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673876+0000", 2020-08-07T15:07:45.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00065173100000000001 2020-08-07T15:07:45.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:45.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.674528+0000", 2020-08-07T15:07:45.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3933e-05 2020-08-07T15:07:45.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:45.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.674582+0000", 2020-08-07T15:07:45.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27006165399999998 2020-08-07T15:07:45.706 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.706 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:45.706 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.707 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:49208 3.2 e36/20 3:4d72b982:::benchmark_data_smithi074_35447_object6150:head v 36'2904, mlcod=36'2897)", 2020-08-07T15:07:45.707 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:09.692013+0000", 2020-08-07T15:07:45.707 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.813044204999997, 2020-08-07T15:07:45.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.234386975, 2020-08-07T15:07:45.708 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:45.708 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:45.708 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:45.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:45.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.692013+0000", 2020-08-07T15:07:45.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:45.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:45.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.692013+0000", 2020-08-07T15:07:45.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0998000000000002e-05 2020-08-07T15:07:45.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:45.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.692064+0000", 2020-08-07T15:07:45.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0210000000000001e-05 2020-08-07T15:07:45.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:45.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.692074+0000", 2020-08-07T15:07:45.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.233053807 2020-08-07T15:07:45.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:45.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.925128+0000", 2020-08-07T15:07:45.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4879999999999998e-06 2020-08-07T15:07:45.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:45.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.925134+0000", 2020-08-07T15:07:45.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5154999999999997e-05 2020-08-07T15:07:45.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:45.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.925179+0000", 2020-08-07T15:07:45.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2710000000000001e-05 2020-08-07T15:07:45.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:45.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.925201+0000", 2020-08-07T15:07:45.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001143725 2020-08-07T15:07:45.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:45.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.926345+0000", 2020-08-07T15:07:45.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4882000000000001e-05 2020-08-07T15:07:45.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:45.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.926400+0000", 2020-08-07T15:07:45.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.234386975 2020-08-07T15:07:45.719 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.719 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:45.719 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.720 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:55192 3.2 e37/20 3:4a96099f:::benchmark_data_smithi074_35447_object6898:head v 37'3336, mlcod=37'3326)", 2020-08-07T15:07:45.721 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:13.771422+0000", 2020-08-07T15:07:45.721 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.733635536000001, 2020-08-07T15:07:45.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16800543000000001, 2020-08-07T15:07:45.721 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:45.721 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:45.722 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:45.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:45.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.771422+0000", 2020-08-07T15:07:45.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:45.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:45.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.771422+0000", 2020-08-07T15:07:45.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9376000000000001e-05 2020-08-07T15:07:45.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:45.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.771451+0000", 2020-08-07T15:07:45.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8789999999999996e-06 2020-08-07T15:07:45.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:45.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.771455+0000", 2020-08-07T15:07:45.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16555510500000001 2020-08-07T15:07:45.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:45.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.937010+0000", 2020-08-07T15:07:45.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3770000000000004e-06 2020-08-07T15:07:45.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:45.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.937015+0000", 2020-08-07T15:07:45.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6736999999999997e-05 2020-08-07T15:07:45.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:45.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.937052+0000", 2020-08-07T15:07:45.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3654000000000001e-05 2020-08-07T15:07:45.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:45.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.937076+0000", 2020-08-07T15:07:45.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022779929999999999 2020-08-07T15:07:45.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:45.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.939354+0000", 2020-08-07T15:07:45.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3308999999999999e-05 2020-08-07T15:07:45.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:45.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:13.939427+0000", 2020-08-07T15:07:45.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16800543000000001 2020-08-07T15:07:45.739 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.740 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:45.740 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.740 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:71904 3.a 3:5e6d3be8:::benchmark_data_smithi074_35447_object8987:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e43)", 2020-08-07T15:07:45.741 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:24.780800+0000", 2020-08-07T15:07:45.741 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.724257325, 2020-08-07T15:07:45.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.159596976, 2020-08-07T15:07:45.741 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:45.741 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:45.741 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:45.742 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:45.742 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:45.742 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 71904 2020-08-07T15:07:45.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.742 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:45.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:45.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.780800+0000", 2020-08-07T15:07:45.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:45.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:45.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.780800+0000", 2020-08-07T15:07:45.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3742999999999998e-05 2020-08-07T15:07:45.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:45.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.780844+0000", 2020-08-07T15:07:45.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9300000000000002e-06 2020-08-07T15:07:45.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:45.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.780849+0000", 2020-08-07T15:07:45.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9189999999999994e-06 2020-08-07T15:07:45.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:45.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.780859+0000", 2020-08-07T15:07:45.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3789999999999999e-06 2020-08-07T15:07:45.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:45.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.780863+0000", 2020-08-07T15:07:45.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016907109999999999 2020-08-07T15:07:45.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:45.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.782554+0000", 2020-08-07T15:07:45.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0457999999999993e-05 2020-08-07T15:07:45.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:45.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.782644+0000", 2020-08-07T15:07:45.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021834600000000001 2020-08-07T15:07:45.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:45.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.782862+0000", 2020-08-07T15:07:45.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022362290000000002 2020-08-07T15:07:45.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:45.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.785099+0000", 2020-08-07T15:07:45.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2699999999999998e-07 2020-08-07T15:07:45.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:45.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.785100+0000", 2020-08-07T15:07:45.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5136999999999997e-05 2020-08-07T15:07:45.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:45.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.785165+0000", 2020-08-07T15:07:45.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053074109999999997 2020-08-07T15:07:45.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:45.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.790472+0000", 2020-08-07T15:07:45.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14985902700000001 2020-08-07T15:07:45.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:45.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.940331+0000", 2020-08-07T15:07:45.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1399999999999996e-07 2020-08-07T15:07:45.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:45.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.940332+0000", 2020-08-07T15:07:45.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4944999999999996e-05 2020-08-07T15:07:45.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:45.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:24.940397+0000", 2020-08-07T15:07:45.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.159596976 2020-08-07T15:07:45.758 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.758 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:45.758 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.759 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:77967 3.2 e46/20 3:48f61a4b:::benchmark_data_smithi074_35447_object9745:head v 46'4807, mlcod=46'4798)", 2020-08-07T15:07:45.759 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:28.702144+0000", 2020-08-07T15:07:45.759 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.802913043, 2020-08-07T15:07:45.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26780679000000002, 2020-08-07T15:07:45.759 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:45.760 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:45.760 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:45.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:45.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702144+0000", 2020-08-07T15:07:45.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:45.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:45.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702144+0000", 2020-08-07T15:07:45.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5482000000000002e-05 2020-08-07T15:07:45.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:45.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702200+0000", 2020-08-07T15:07:45.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.216e-06 2020-08-07T15:07:45.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:45.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702207+0000", 2020-08-07T15:07:45.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26634085400000002 2020-08-07T15:07:45.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:45.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.968548+0000", 2020-08-07T15:07:45.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2490000000000001e-06 2020-08-07T15:07:45.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:45.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.968553+0000", 2020-08-07T15:07:45.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7738999999999998e-05 2020-08-07T15:07:45.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:45.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.968601+0000", 2020-08-07T15:07:45.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5783999999999999e-05 2020-08-07T15:07:45.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:45.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.968627+0000", 2020-08-07T15:07:45.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001270471 2020-08-07T15:07:45.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:45.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.969897+0000", 2020-08-07T15:07:45.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3995000000000001e-05 2020-08-07T15:07:45.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:45.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.969951+0000", 2020-08-07T15:07:45.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26780679000000002 2020-08-07T15:07:45.777 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.778 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:45.778 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.778 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:77968 3.2 e46/20 3:48f61a4b:::benchmark_data_smithi074_35447_object9745:head v 46'4808, mlcod=46'4799)", 2020-08-07T15:07:45.778 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:28.702836+0000", 2020-08-07T15:07:45.779 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.802221202999998, 2020-08-07T15:07:45.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27322339400000001, 2020-08-07T15:07:45.779 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:45.779 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:45.779 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:45.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:45.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702836+0000", 2020-08-07T15:07:45.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:45.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:45.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702836+0000", 2020-08-07T15:07:45.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1822e-05 2020-08-07T15:07:45.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:45.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702888+0000", 2020-08-07T15:07:45.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4110000000000001e-06 2020-08-07T15:07:45.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:45.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702894+0000", 2020-08-07T15:07:45.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26750942999999999 2020-08-07T15:07:45.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:45.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970404+0000", 2020-08-07T15:07:45.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1150000000000002e-06 2020-08-07T15:07:45.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:45.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970409+0000", 2020-08-07T15:07:45.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2898000000000007e-05 2020-08-07T15:07:45.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:45.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970472+0000", 2020-08-07T15:07:45.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2311999999999999e-05 2020-08-07T15:07:45.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:45.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970494+0000", 2020-08-07T15:07:45.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055286800000000002 2020-08-07T15:07:45.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:45.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.976023+0000", 2020-08-07T15:07:45.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6726e-05 2020-08-07T15:07:45.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:45.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.976059+0000", 2020-08-07T15:07:45.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27322339400000001 2020-08-07T15:07:45.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.791 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:45.791 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.791 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86695 3.a 3:5058ba7c:::benchmark_data_smithi074_35447_object10836:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:07:45.791 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:07:45.792 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.179729786999999, 2020-08-07T15:07:45.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31490022699999998, 2020-08-07T15:07:45.792 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:45.792 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:45.792 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:45.793 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:45.793 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:45.793 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86695 2020-08-07T15:07:45.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.793 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:45.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:45.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:07:45.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:45.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:45.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:07:45.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4560000000000005e-05 2020-08-07T15:07:45.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:45.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325392+0000", 2020-08-07T15:07:45.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3180000000000003e-06 2020-08-07T15:07:45.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:45.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325398+0000", 2020-08-07T15:07:45.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.258e-05 2020-08-07T15:07:45.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:45.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325411+0000", 2020-08-07T15:07:45.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6550000000000001e-06 2020-08-07T15:07:45.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:45.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325417+0000", 2020-08-07T15:07:45.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024282879999999998 2020-08-07T15:07:45.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:45.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.327845+0000", 2020-08-07T15:07:45.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1203000000000001e-05 2020-08-07T15:07:45.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:45.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.327936+0000", 2020-08-07T15:07:45.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000186505 2020-08-07T15:07:45.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:45.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328123+0000", 2020-08-07T15:07:45.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020062890000000001 2020-08-07T15:07:45.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:45.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.330129+0000", 2020-08-07T15:07:45.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121513 2020-08-07T15:07:45.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:45.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.330250+0000", 2020-08-07T15:07:45.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30989746899999998 2020-08-07T15:07:45.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:45.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640148+0000", 2020-08-07T15:07:45.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2139999999999999e-06 2020-08-07T15:07:45.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:45.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640149+0000", 2020-08-07T15:07:45.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8081999999999998e-05 2020-08-07T15:07:45.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:45.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640197+0000", 2020-08-07T15:07:45.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0550999999999999e-05 2020-08-07T15:07:45.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:45.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640228+0000", 2020-08-07T15:07:45.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31490022699999998 2020-08-07T15:07:45.808 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.808 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:45.808 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.809 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86696 3.a 3:5058ba7c:::benchmark_data_smithi074_35447_object10836:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:07:45.809 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:07:45.809 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.179605108000001, 2020-08-07T15:07:45.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.347347556, 2020-08-07T15:07:45.809 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:45.810 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:45.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:45.811 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:45.811 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:45.811 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86696 2020-08-07T15:07:45.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.811 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:45.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:45.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:07:45.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:45.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:45.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:07:45.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0436999999999998e-05 2020-08-07T15:07:45.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:45.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325503+0000", 2020-08-07T15:07:45.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4659999999999998e-06 2020-08-07T15:07:45.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:45.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325508+0000", 2020-08-07T15:07:45.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1177e-05 2020-08-07T15:07:45.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:45.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325519+0000", 2020-08-07T15:07:45.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7809999999999996e-06 2020-08-07T15:07:45.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:45.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325524+0000", 2020-08-07T15:07:45.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002989219 2020-08-07T15:07:45.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:45.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328513+0000", 2020-08-07T15:07:45.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0649000000000006e-05 2020-08-07T15:07:45.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:45.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328604+0000", 2020-08-07T15:07:45.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001652 2020-08-07T15:07:45.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:45.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328769+0000", 2020-08-07T15:07:45.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023470639999999998 2020-08-07T15:07:45.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:45.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331116+0000", 2020-08-07T15:07:45.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6099999999999999e-07 2020-08-07T15:07:45.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:45.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331117+0000", 2020-08-07T15:07:45.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9150000000000001e-05 2020-08-07T15:07:45.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:45.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331176+0000", 2020-08-07T15:07:45.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065402581000000001 2020-08-07T15:07:45.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:45.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.396579+0000", 2020-08-07T15:07:45.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27618104599999999 2020-08-07T15:07:45.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:45.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672760+0000", 2020-08-07T15:07:45.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2020-08-07T15:07:45.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:45.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672761+0000", 2020-08-07T15:07:45.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9211000000000001e-05 2020-08-07T15:07:45.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:45.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672800+0000", 2020-08-07T15:07:45.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.347347556 2020-08-07T15:07:45.827 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.827 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:45.827 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.828 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:86695 3.a e52/20 ondisk, result = 0)", 2020-08-07T15:07:45.828 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:07:45.828 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.171648411, 2020-08-07T15:07:45.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.306823501, 2020-08-07T15:07:45.828 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:45.829 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:45.829 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:45.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:45.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:07:45.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:45.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:45.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:07:45.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.048e-05 2020-08-07T15:07:45.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:45.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333439+0000", 2020-08-07T15:07:45.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9990000000000002e-06 2020-08-07T15:07:45.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:45.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333443+0000", 2020-08-07T15:07:45.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30664195100000002 2020-08-07T15:07:45.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:45.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640085+0000", 2020-08-07T15:07:45.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8559999999999998e-06 2020-08-07T15:07:45.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:45.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640090+0000", 2020-08-07T15:07:45.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5166000000000001e-05 2020-08-07T15:07:45.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:45.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640135+0000", 2020-08-07T15:07:45.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4160000000000001e-06 2020-08-07T15:07:45.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:45.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640142+0000", 2020-08-07T15:07:45.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0632999999999998e-05 2020-08-07T15:07:45.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:45.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640232+0000", 2020-08-07T15:07:45.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.306823501 2020-08-07T15:07:45.838 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.838 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:45.838 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.839 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:87360 3.2 e52/20 3:45522a70:::benchmark_data_smithi074_35447_object10919:head v 52'5480, mlcod=52'5472)", 2020-08-07T15:07:45.839 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:35.140295+0000", 2020-08-07T15:07:45.839 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.364762181, 2020-08-07T15:07:45.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16390410399999999, 2020-08-07T15:07:45.840 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:45.840 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:45.840 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:45.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:45.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.140295+0000", 2020-08-07T15:07:45.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:45.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:45.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.140295+0000", 2020-08-07T15:07:45.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3275000000000001e-05 2020-08-07T15:07:45.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:45.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.140348+0000", 2020-08-07T15:07:45.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7580000000000003e-06 2020-08-07T15:07:45.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:45.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.140355+0000", 2020-08-07T15:07:45.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.162238204 2020-08-07T15:07:45.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:45.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.302593+0000", 2020-08-07T15:07:45.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.602e-06 2020-08-07T15:07:45.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:45.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.302597+0000", 2020-08-07T15:07:45.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6308000000000002e-05 2020-08-07T15:07:45.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:45.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.302673+0000", 2020-08-07T15:07:45.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3665000000000001e-05 2020-08-07T15:07:45.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:45.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.302697+0000", 2020-08-07T15:07:45.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014522090000000001 2020-08-07T15:07:45.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:45.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.304149+0000", 2020-08-07T15:07:45.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0083000000000001e-05 2020-08-07T15:07:45.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:45.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:45.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:45.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:35.304199+0000", 2020-08-07T15:07:45.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.16390410399999999 2020-08-07T15:07:45.851 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.852 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:45.852 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.852 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:45.852 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:45.852 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:45.853 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:07:45.855 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:07:45.856 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:07:45.856 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:07:45.856 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:07:45.857 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:07:45.857 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:07:45.857 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:07:45.857 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:07:45.857 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:07:45.858 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:45.852+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:45.954 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:45.953+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:46.043 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:46.044 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:46.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.044 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:104128 3.e 3:78158f44:::benchmark_data_smithi074_35447_object13015:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:07:46.044 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:45.870701+0000", 2020-08-07T15:07:46.044 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.17211415799999999, 2020-08-07T15:07:46.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.17213966, 2020-08-07T15:07:46.045 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:46.045 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:46.045 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:46.045 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:46.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:46.046 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 104128 2020-08-07T15:07:46.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.046 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:46.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:46.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.870701+0000", 2020-08-07T15:07:46.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:46.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:46.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.870701+0000", 2020-08-07T15:07:46.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8121000000000001e-05 2020-08-07T15:07:46.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:46.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.870729+0000", 2020-08-07T15:07:46.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3649999999999997e-06 2020-08-07T15:07:46.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:46.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.870734+0000", 2020-08-07T15:07:46.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4924e-05 2020-08-07T15:07:46.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:46.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.870748+0000", 2020-08-07T15:07:46.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1729999999999998e-06 2020-08-07T15:07:46.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:46.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.870756+0000", 2020-08-07T15:07:46.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3569000000000002e-05 2020-08-07T15:07:46.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:46.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.870799+0000", 2020-08-07T15:07:46.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102191 2020-08-07T15:07:46.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:46.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.870901+0000", 2020-08-07T15:07:46.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018755599999999999 2020-08-07T15:07:46.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:46.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.871089+0000", 2020-08-07T15:07:46.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002616125 2020-08-07T15:07:46.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:46.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.873705+0000", 2020-08-07T15:07:46.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0100000000000004e-07 2020-08-07T15:07:46.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:46.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.873706+0000", 2020-08-07T15:07:46.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9969999999999997e-05 2020-08-07T15:07:46.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:46.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.873766+0000", 2020-08-07T15:07:46.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037339550000000002 2020-08-07T15:07:46.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:46.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.877500+0000", 2020-08-07T15:07:46.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0067986499999999998 2020-08-07T15:07:46.081 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.082 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:46.082 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.082 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.082 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:07:46.083 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 1 2020-08-07T15:07:46.083 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:46.091 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:07:46.092 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:46.088+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:46.190 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:46.188+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:46.246 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 70 16 104485 104469 11.6585 11.0938 0.0128489 0.0107176 2020-08-07T15:07:46.293 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:46.291+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:46.313 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:46.314 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:07:46.314 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:07:46.314 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:07:46.314 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:46.331 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:07:46.347 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:07:46.398 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:46.394+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:46.414 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:07:46.414 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:07:46.414 INFO:tasks.thrashosds.thrasher:Reweighting osd 3 to 0.7917969780468398 2020-08-07T15:07:46.415 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 3 0.7917969780468398 2020-08-07T15:07:46.537 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:46.497+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:46.538 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:46.538 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:07:46.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:07:46.538 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:46.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.539 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:9352 3.e 3:7113579b:::benchmark_data_smithi074_35447_object1168:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:46.539 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.885925+0000", 2020-08-07T15:07:46.539 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.614004123000001, 2020-08-07T15:07:46.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24955413800000001, 2020-08-07T15:07:46.539 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:46.540 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:46.542 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:46.542 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:46.542 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:46.542 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 9352 2020-08-07T15:07:46.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.543 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:46.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:46.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.885925+0000", 2020-08-07T15:07:46.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:46.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:46.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.885925+0000", 2020-08-07T15:07:46.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4308999999999998e-05 2020-08-07T15:07:46.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:46.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.885979+0000", 2020-08-07T15:07:46.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0440000000000001e-06 2020-08-07T15:07:46.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:46.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.885985+0000", 2020-08-07T15:07:46.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9462000000000001e-05 2020-08-07T15:07:46.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:46.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.886005+0000", 2020-08-07T15:07:46.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2530000000000001e-06 2020-08-07T15:07:46.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:46.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.886011+0000", 2020-08-07T15:07:46.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025236080000000001 2020-08-07T15:07:46.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:46.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.888535+0000", 2020-08-07T15:07:46.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112871 2020-08-07T15:07:46.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:46.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.888648+0000", 2020-08-07T15:07:46.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020764399999999999 2020-08-07T15:07:46.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:46.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.888855+0000", 2020-08-07T15:07:46.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048523799999999999 2020-08-07T15:07:46.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:46.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.893708+0000", 2020-08-07T15:07:46.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030190300000000001 2020-08-07T15:07:46.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:46.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.894010+0000", 2020-08-07T15:07:46.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24135353300000001 2020-08-07T15:07:46.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:46.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.135363+0000", 2020-08-07T15:07:46.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9899999999999999e-07 2020-08-07T15:07:46.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:46.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.135364+0000", 2020-08-07T15:07:46.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7601999999999994e-05 2020-08-07T15:07:46.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:46.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.135431+0000", 2020-08-07T15:07:46.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7729999999999999e-05 2020-08-07T15:07:46.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:46.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.135479+0000", 2020-08-07T15:07:46.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24955413800000001 2020-08-07T15:07:46.570 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.571 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:46.571 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:9976 3.e 3:7fb2e2e5:::benchmark_data_smithi074_35447_object1246:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:07:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.262731235999993, 2020-08-07T15:07:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28420182999999999, 2020-08-07T15:07:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 9976 2020-08-07T15:07:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:07:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:07:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7113e-05 2020-08-07T15:07:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237255+0000", 2020-08-07T15:07:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5869999999999998e-06 2020-08-07T15:07:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237262+0000", 2020-08-07T15:07:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4467999999999999e-05 2020-08-07T15:07:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237276+0000", 2020-08-07T15:07:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0749999999999997e-06 2020-08-07T15:07:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237281+0000", 2020-08-07T15:07:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013795109999999999 2020-08-07T15:07:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238661+0000", 2020-08-07T15:07:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100309 2020-08-07T15:07:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238761+0000", 2020-08-07T15:07:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000141784 2020-08-07T15:07:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238903+0000", 2020-08-07T15:07:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00070626599999999999 2020-08-07T15:07:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239609+0000", 2020-08-07T15:07:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3499999999999996e-07 2020-08-07T15:07:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:46.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239610+0000", 2020-08-07T15:07:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0846000000000006e-05 2020-08-07T15:07:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239680+0000", 2020-08-07T15:07:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022037080000000001 2020-08-07T15:07:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.261718+0000", 2020-08-07T15:07:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25961784500000001 2020-08-07T15:07:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521335+0000", 2020-08-07T15:07:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2839999999999999e-06 2020-08-07T15:07:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521337+0000", 2020-08-07T15:07:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3126999999999994e-05 2020-08-07T15:07:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521400+0000", 2020-08-07T15:07:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28420182999999999 2020-08-07T15:07:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:9976 3.e e24/20 ondisk, result = 0)", 2020-08-07T15:07:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.269400+0000", 2020-08-07T15:07:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.230529012000005, 2020-08-07T15:07:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25200699900000001, 2020-08-07T15:07:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.269400+0000", 2020-08-07T15:07:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.269400+0000", 2020-08-07T15:07:46.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1279999999999999e-05 2020-08-07T15:07:46.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:46.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.269431+0000", 2020-08-07T15:07:46.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.071e-06 2020-08-07T15:07:46.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:46.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.269438+0000", 2020-08-07T15:07:46.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24671454900000001 2020-08-07T15:07:46.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:46.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.516152+0000", 2020-08-07T15:07:46.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4140000000000002e-06 2020-08-07T15:07:46.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:46.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.516157+0000", 2020-08-07T15:07:46.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051603170000000002 2020-08-07T15:07:46.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:46.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521318+0000", 2020-08-07T15:07:46.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7479999999999999e-06 2020-08-07T15:07:46.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:46.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521327+0000", 2020-08-07T15:07:46.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0619999999999997e-05 2020-08-07T15:07:46.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:46.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521407+0000", 2020-08-07T15:07:46.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25200699900000001 2020-08-07T15:07:46.629 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.629 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:46.635 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.635 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15576 3.e 3:727724a5:::benchmark_data_smithi074_35447_object1946:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:46.636 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:07:46.636 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 59.833294273, 2020-08-07T15:07:46.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999, 2020-08-07T15:07:46.636 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:46.636 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:46.637 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:46.637 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:46.637 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:46.637 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15576 2020-08-07T15:07:46.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.638 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:46.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:46.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:07:46.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:46.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:46.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:07:46.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7156e-05 2020-08-07T15:07:46.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:46.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666682+0000", 2020-08-07T15:07:46.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0209999999999999e-06 2020-08-07T15:07:46.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:46.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666687+0000", 2020-08-07T15:07:46.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0730999999999999e-05 2020-08-07T15:07:46.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:46.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666698+0000", 2020-08-07T15:07:46.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2020-08-07T15:07:46.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:46.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666702+0000", 2020-08-07T15:07:46.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023978459999999999 2020-08-07T15:07:46.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:46.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669100+0000", 2020-08-07T15:07:46.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9664000000000001e-05 2020-08-07T15:07:46.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:46.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669190+0000", 2020-08-07T15:07:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015957100000000001 2020-08-07T15:07:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669350+0000", 2020-08-07T15:07:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016470149999999999 2020-08-07T15:07:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:07:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1299999999999999e-07 2020-08-07T15:07:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:07:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5269000000000002e-05 2020-08-07T15:07:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.671022+0000", 2020-08-07T15:07:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031414400000000001 2020-08-07T15:07:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.674163+0000", 2020-08-07T15:07:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45641417400000001 2020-08-07T15:07:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:07:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0399999999999995e-07 2020-08-07T15:07:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:07:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3343999999999997e-05 2020-08-07T15:07:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:46.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130622+0000", 2020-08-07T15:07:46.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999 2020-08-07T15:07:46.689 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.690 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:46.690 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.691 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:17824 3.d 3:bd4e0935:::benchmark_data_smithi074_35447_object2227:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:46.691 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.140708+0000", 2020-08-07T15:07:46.691 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.359221349999999, 2020-08-07T15:07:46.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25688457100000001, 2020-08-07T15:07:46.692 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:46.692 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:46.692 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:46.692 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:46.692 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:46.692 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 17824 2020-08-07T15:07:46.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.693 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:46.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:46.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140708+0000", 2020-08-07T15:07:46.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:46.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:46.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140708+0000", 2020-08-07T15:07:46.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0522000000000002e-05 2020-08-07T15:07:46.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:46.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140758+0000", 2020-08-07T15:07:46.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1680000000000001e-06 2020-08-07T15:07:46.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:46.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140765+0000", 2020-08-07T15:07:46.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5791999999999998e-05 2020-08-07T15:07:46.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:46.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140810+0000", 2020-08-07T15:07:46.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2389999999999996e-06 2020-08-07T15:07:46.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:46.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140818+0000", 2020-08-07T15:07:46.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001721544 2020-08-07T15:07:46.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:46.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.142539+0000", 2020-08-07T15:07:46.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020908100000000001 2020-08-07T15:07:46.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:46.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.142748+0000", 2020-08-07T15:07:46.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019000000000000001 2020-08-07T15:07:46.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:46.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.142938+0000", 2020-08-07T15:07:46.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001381279 2020-08-07T15:07:46.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:46.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.144319+0000", 2020-08-07T15:07:46.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010975570000000001 2020-08-07T15:07:46.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:46.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.145417+0000", 2020-08-07T15:07:46.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.252064128 2020-08-07T15:07:46.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:46.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397481+0000", 2020-08-07T15:07:46.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003e-06 2020-08-07T15:07:46.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:46.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397482+0000", 2020-08-07T15:07:46.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3081000000000001e-05 2020-08-07T15:07:46.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:46.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397545+0000", 2020-08-07T15:07:46.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7176999999999999e-05 2020-08-07T15:07:46.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:46.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397592+0000", 2020-08-07T15:07:46.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25688457100000001 2020-08-07T15:07:46.711 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.711 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:46.711 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.712 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:35576 3.f 3:fca2de08:::benchmark_data_smithi074_35447_object4446:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-08-07T15:07:46.712 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:59.287747+0000", 2020-08-07T15:07:46.712 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.212182546999998, 2020-08-07T15:07:46.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25861668500000001, 2020-08-07T15:07:46.713 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:46.713 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:46.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:46.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:46.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:46.714 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 35576 2020-08-07T15:07:46.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.714 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:46.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:46.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287747+0000", 2020-08-07T15:07:46.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:46.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:46.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287747+0000", 2020-08-07T15:07:46.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0089000000000001e-05 2020-08-07T15:07:46.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:46.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287797+0000", 2020-08-07T15:07:46.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4219999999999997e-06 2020-08-07T15:07:46.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:46.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287803+0000", 2020-08-07T15:07:46.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4132e-05 2020-08-07T15:07:46.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:46.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287817+0000", 2020-08-07T15:07:46.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1600000000000003e-06 2020-08-07T15:07:46.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:46.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287823+0000", 2020-08-07T15:07:46.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019418230000000001 2020-08-07T15:07:46.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:46.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.289765+0000", 2020-08-07T15:07:46.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110597 2020-08-07T15:07:46.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:46.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.289876+0000", 2020-08-07T15:07:46.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017674800000000001 2020-08-07T15:07:46.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:46.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.290053+0000", 2020-08-07T15:07:46.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052995730000000001 2020-08-07T15:07:46.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:46.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295352+0000", 2020-08-07T15:07:46.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9999999999999997e-07 2020-08-07T15:07:46.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:46.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295353+0000", 2020-08-07T15:07:46.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6803000000000004e-05 2020-08-07T15:07:46.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:46.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295420+0000", 2020-08-07T15:07:46.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015611539999999999 2020-08-07T15:07:46.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:46.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.296981+0000", 2020-08-07T15:07:46.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.249314866 2020-08-07T15:07:46.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:46.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546296+0000", 2020-08-07T15:07:46.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7300000000000005e-07 2020-08-07T15:07:46.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:46.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546296+0000", 2020-08-07T15:07:46.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6845000000000001e-05 2020-08-07T15:07:46.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:46.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546363+0000", 2020-08-07T15:07:46.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25861668500000001 2020-08-07T15:07:46.731 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.732 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:46.732 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.732 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:35576 3.f e31/20 ondisk, result = 0)", 2020-08-07T15:07:46.733 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:59.295024+0000", 2020-08-07T15:07:46.733 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.204905373999999, 2020-08-07T15:07:46.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25134726499999999, 2020-08-07T15:07:46.733 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:46.733 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:46.734 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:46.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:46.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295024+0000", 2020-08-07T15:07:46.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:46.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:46.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295024+0000", 2020-08-07T15:07:46.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7844999999999999e-05 2020-08-07T15:07:46.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:46.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295052+0000", 2020-08-07T15:07:46.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7740000000000004e-06 2020-08-07T15:07:46.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:46.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295056+0000", 2020-08-07T15:07:46.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25017799099999999 2020-08-07T15:07:46.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:46.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.545234+0000", 2020-08-07T15:07:46.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5140000000000003e-06 2020-08-07T15:07:46.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:46.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.545239+0000", 2020-08-07T15:07:46.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010386989999999999 2020-08-07T15:07:46.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:46.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546278+0000", 2020-08-07T15:07:46.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0480000000000004e-06 2020-08-07T15:07:46.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:46.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546287+0000", 2020-08-07T15:07:46.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4394000000000005e-05 2020-08-07T15:07:46.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:46.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546371+0000", 2020-08-07T15:07:46.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25134726499999999 2020-08-07T15:07:46.756 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.756 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:46.756 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.757 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:41431 3.d 3:b5c8a160:::benchmark_data_smithi074_35447_object5178:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:07:46.758 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.791939+0000", 2020-08-07T15:07:46.758 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 43.707989744000002, 2020-08-07T15:07:46.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28436496, 2020-08-07T15:07:46.758 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:46.759 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:46.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:46.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:46.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:46.759 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 41431 2020-08-07T15:07:46.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.760 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:46.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:46.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791939+0000", 2020-08-07T15:07:46.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:46.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:46.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791939+0000", 2020-08-07T15:07:46.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6304999999999998e-05 2020-08-07T15:07:46.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:46.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791986+0000", 2020-08-07T15:07:46.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6339999999999998e-06 2020-08-07T15:07:46.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:46.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791991+0000", 2020-08-07T15:07:46.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.026e-05 2020-08-07T15:07:46.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:46.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792002+0000", 2020-08-07T15:07:46.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6240000000000001e-06 2020-08-07T15:07:46.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:46.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792006+0000", 2020-08-07T15:07:46.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015600220000000001 2020-08-07T15:07:46.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:46.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.793566+0000", 2020-08-07T15:07:46.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3628999999999994e-05 2020-08-07T15:07:46.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:46.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.793660+0000", 2020-08-07T15:07:46.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000168197 2020-08-07T15:07:46.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:46.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.793828+0000", 2020-08-07T15:07:46.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001273251 2020-08-07T15:07:46.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:46.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795101+0000", 2020-08-07T15:07:46.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5899999999999998e-07 2020-08-07T15:07:46.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:46.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795102+0000", 2020-08-07T15:07:46.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4904000000000001e-05 2020-08-07T15:07:46.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:46.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795127+0000", 2020-08-07T15:07:46.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00670237 2020-08-07T15:07:46.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:46.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801829+0000", 2020-08-07T15:07:46.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27442024999999998 2020-08-07T15:07:46.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:46.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076249+0000", 2020-08-07T15:07:46.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999997e-07 2020-08-07T15:07:46.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:46.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076250+0000", 2020-08-07T15:07:46.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4369000000000001e-05 2020-08-07T15:07:46.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:46.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076304+0000", 2020-08-07T15:07:46.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28436496 2020-08-07T15:07:46.784 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.784 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:46.784 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.785 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:41432 3.d 3:b5c8a160:::benchmark_data_smithi074_35447_object5178:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:07:46.785 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:07:46.785 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 43.707639047999997, 2020-08-07T15:07:46.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.37720701699999998, 2020-08-07T15:07:46.786 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:46.786 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:46.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:46.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:46.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:46.787 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 41432 2020-08-07T15:07:46.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.787 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:46.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:46.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:07:46.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:46.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:46.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:07:46.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4787999999999999e-05 2020-08-07T15:07:46.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:46.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792335+0000", 2020-08-07T15:07:46.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5999999999999997e-06 2020-08-07T15:07:46.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:46.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792341+0000", 2020-08-07T15:07:46.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0998e-05 2020-08-07T15:07:46.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:46.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792352+0000", 2020-08-07T15:07:46.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2989999999999996e-06 2020-08-07T15:07:46.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:46.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792357+0000", 2020-08-07T15:07:46.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001852752 2020-08-07T15:07:46.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:46.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794210+0000", 2020-08-07T15:07:46.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5771000000000002e-05 2020-08-07T15:07:46.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:46.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794305+0000", 2020-08-07T15:07:46.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012776399999999999 2020-08-07T15:07:46.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:46.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794433+0000", 2020-08-07T15:07:46.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00069673599999999999 2020-08-07T15:07:46.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:46.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795130+0000", 2020-08-07T15:07:46.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1500000000000001e-07 2020-08-07T15:07:46.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:46.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795130+0000", 2020-08-07T15:07:46.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7319e-05 2020-08-07T15:07:46.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:46.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795167+0000", 2020-08-07T15:07:46.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27910026700000001 2020-08-07T15:07:46.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:46.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074268+0000", 2020-08-07T15:07:46.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.095150660999999997 2020-08-07T15:07:46.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:46.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169418+0000", 2020-08-07T15:07:46.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.423e-06 2020-08-07T15:07:46.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:46.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169420+0000", 2020-08-07T15:07:46.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7423999999999996e-05 2020-08-07T15:07:46.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:46.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169497+0000", 2020-08-07T15:07:46.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.37720701699999998 2020-08-07T15:07:46.804 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.804 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:46.804 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.805 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:41431 3.d e32/20 ondisk, result = 0)", 2020-08-07T15:07:46.805 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.796229+0000", 2020-08-07T15:07:46.806 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 43.703700146000003, 2020-08-07T15:07:46.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28007903299999998, 2020-08-07T15:07:46.806 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:46.806 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:46.806 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:46.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:46.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.796229+0000", 2020-08-07T15:07:46.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:46.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:46.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.796229+0000", 2020-08-07T15:07:46.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3272000000000003e-05 2020-08-07T15:07:46.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:46.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.796262+0000", 2020-08-07T15:07:46.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9579999999999998e-06 2020-08-07T15:07:46.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:46.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.796267+0000", 2020-08-07T15:07:46.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27993393599999999 2020-08-07T15:07:46.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:46.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076201+0000", 2020-08-07T15:07:46.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8709999999999999e-06 2020-08-07T15:07:46.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:46.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076205+0000", 2020-08-07T15:07:46.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9128e-05 2020-08-07T15:07:46.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:46.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076234+0000", 2020-08-07T15:07:46.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9800000000000001e-06 2020-08-07T15:07:46.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:46.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076241+0000", 2020-08-07T15:07:46.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6888000000000001e-05 2020-08-07T15:07:46.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:46.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076308+0000", 2020-08-07T15:07:46.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28007903299999998 2020-08-07T15:07:46.815 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.815 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:46.816 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.816 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:41432 3.d e32/20 ondisk, result = 0)", 2020-08-07T15:07:46.816 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.801630+0000", 2020-08-07T15:07:46.817 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 43.698299245000001, 2020-08-07T15:07:46.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27264033999999998, 2020-08-07T15:07:46.818 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:46.818 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:46.818 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:46.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:46.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801630+0000", 2020-08-07T15:07:46.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:46.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:46.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801630+0000", 2020-08-07T15:07:46.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6257e-05 2020-08-07T15:07:46.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:46.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801646+0000", 2020-08-07T15:07:46.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5450000000000002e-06 2020-08-07T15:07:46.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:46.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801649+0000", 2020-08-07T15:07:46.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27254806999999998 2020-08-07T15:07:46.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:46.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074197+0000", 2020-08-07T15:07:46.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4079999999999997e-06 2020-08-07T15:07:46.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:46.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074202+0000", 2020-08-07T15:07:46.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5222000000000002e-05 2020-08-07T15:07:46.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:46.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074247+0000", 2020-08-07T15:07:46.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9459999999999993e-06 2020-08-07T15:07:46.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:46.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074257+0000", 2020-08-07T15:07:46.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2892e-05 2020-08-07T15:07:46.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:46.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074270+0000", 2020-08-07T15:07:46.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27264033999999998 2020-08-07T15:07:46.827 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.828 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:46.828 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.828 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:42104 3.d 3:bcc35439:::benchmark_data_smithi074_35447_object5262:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:07:46.829 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:07:46.829 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 43.261006877, 2020-08-07T15:07:46.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28812091400000001, 2020-08-07T15:07:46.829 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:46.829 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:46.830 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:46.830 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:46.830 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:46.830 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42104 2020-08-07T15:07:46.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.830 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:46.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:46.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:07:46.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:46.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:46.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:07:46.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9734000000000002e-05 2020-08-07T15:07:46.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:46.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238972+0000", 2020-08-07T15:07:46.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6910000000000002e-06 2020-08-07T15:07:46.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:46.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238978+0000", 2020-08-07T15:07:46.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1864e-05 2020-08-07T15:07:46.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:46.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238990+0000", 2020-08-07T15:07:46.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.737e-06 2020-08-07T15:07:46.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:46.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238996+0000", 2020-08-07T15:07:46.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00283943 2020-08-07T15:07:46.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:46.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.241836+0000", 2020-08-07T15:07:46.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4712999999999999e-05 2020-08-07T15:07:46.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:46.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.241930+0000", 2020-08-07T15:07:46.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015328499999999999 2020-08-07T15:07:46.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:07:46.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.242084+0000", 2020-08-07T15:07:46.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016104400000000001 2020-08-07T15:07:46.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:46.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243694+0000", 2020-08-07T15:07:46.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1399999999999997e-07 2020-08-07T15:07:46.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:46.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243695+0000", 2020-08-07T15:07:46.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7800999999999998e-05 2020-08-07T15:07:46.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:46.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243753+0000", 2020-08-07T15:07:46.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090405661999999998 2020-08-07T15:07:46.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:46.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.334158+0000", 2020-08-07T15:07:46.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19283929299999999 2020-08-07T15:07:46.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:46.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.526997+0000", 2020-08-07T15:07:46.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.012e-06 2020-08-07T15:07:46.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:46.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.526998+0000", 2020-08-07T15:07:46.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4737999999999998e-05 2020-08-07T15:07:46.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:46.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.527043+0000", 2020-08-07T15:07:46.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28812091400000001 2020-08-07T15:07:46.847 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.848 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:46.848 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.848 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:44120 3.e 3:7e6f7bb9:::benchmark_data_smithi074_35447_object5514:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:07:46.849 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:04.450106+0000", 2020-08-07T15:07:46.849 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.049823343, 2020-08-07T15:07:46.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26702573699999999, 2020-08-07T15:07:46.849 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:46.861 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:46.861 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:46.861 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:46.861 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:46.861 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 44120 2020-08-07T15:07:46.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.862 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:46.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:46.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450106+0000", 2020-08-07T15:07:46.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:46.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:46.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450106+0000", 2020-08-07T15:07:46.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9926000000000003e-05 2020-08-07T15:07:46.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:46.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450156+0000", 2020-08-07T15:07:46.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4079999999999997e-06 2020-08-07T15:07:46.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:46.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450161+0000", 2020-08-07T15:07:46.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0494e-05 2020-08-07T15:07:46.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:46.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450172+0000", 2020-08-07T15:07:46.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3950000000000003e-06 2020-08-07T15:07:46.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:46.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450176+0000", 2020-08-07T15:07:46.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033213689999999998 2020-08-07T15:07:46.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:46.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.453497+0000", 2020-08-07T15:07:46.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102414 2020-08-07T15:07:46.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:46.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.453600+0000", 2020-08-07T15:07:46.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019776 2020-08-07T15:07:46.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:46.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.453798+0000", 2020-08-07T15:07:46.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001075456 2020-08-07T15:07:46.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:46.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.454873+0000", 2020-08-07T15:07:46.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7999999999999995e-07 2020-08-07T15:07:46.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:46.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.454874+0000", 2020-08-07T15:07:46.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3912000000000002e-05 2020-08-07T15:07:46.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:46.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.454928+0000", 2020-08-07T15:07:46.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058537550000000004 2020-08-07T15:07:46.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:46.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.460781+0000", 2020-08-07T15:07:46.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25630591899999999 2020-08-07T15:07:46.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:46.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.717087+0000", 2020-08-07T15:07:46.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.54e-07 2020-08-07T15:07:46.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:46.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.717088+0000", 2020-08-07T15:07:46.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3594999999999998e-05 2020-08-07T15:07:46.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:46.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.717132+0000", 2020-08-07T15:07:46.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26702573699999999 2020-08-07T15:07:46.879 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.879 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:46.880 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.880 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:49800 3.e 3:7cb4a069:::benchmark_data_smithi074_35447_object6224:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e36)", 2020-08-07T15:07:46.880 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:10.076186+0000", 2020-08-07T15:07:46.880 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.423743604999999, 2020-08-07T15:07:46.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25479637100000002, 2020-08-07T15:07:46.881 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:46.881 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:46.881 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:46.881 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:46.882 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:46.882 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 49800 2020-08-07T15:07:46.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.882 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:46.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:46.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076186+0000", 2020-08-07T15:07:46.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:46.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:46.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076186+0000", 2020-08-07T15:07:46.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3579000000000003e-05 2020-08-07T15:07:46.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:46.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076239+0000", 2020-08-07T15:07:46.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0499999999999997e-06 2020-08-07T15:07:46.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:46.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076245+0000", 2020-08-07T15:07:46.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1756999999999999e-05 2020-08-07T15:07:46.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:46.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076257+0000", 2020-08-07T15:07:46.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8220000000000003e-06 2020-08-07T15:07:46.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:46.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076263+0000", 2020-08-07T15:07:46.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9265999999999999e-05 2020-08-07T15:07:46.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:46.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076312+0000", 2020-08-07T15:07:46.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2280000000000004e-05 2020-08-07T15:07:46.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:46.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076404+0000", 2020-08-07T15:07:46.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015721499999999999 2020-08-07T15:07:46.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:46.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076562+0000", 2020-08-07T15:07:46.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041832179999999998 2020-08-07T15:07:46.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:46.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.080745+0000", 2020-08-07T15:07:46.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.091e-06 2020-08-07T15:07:46.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:46.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.080746+0000", 2020-08-07T15:07:46.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3623999999999999e-05 2020-08-07T15:07:46.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:46.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.080809+0000", 2020-08-07T15:07:46.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11945386299999999 2020-08-07T15:07:46.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:46.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.200263+0000", 2020-08-07T15:07:46.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.130675596 2020-08-07T15:07:46.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:46.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.330939+0000", 2020-08-07T15:07:46.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.508e-06 2020-08-07T15:07:46.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:46.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.330940+0000", 2020-08-07T15:07:46.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1501999999999997e-05 2020-08-07T15:07:46.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:46.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.330982+0000", 2020-08-07T15:07:46.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25479637100000002 2020-08-07T15:07:46.915 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.916 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:46.916 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.916 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:69535 3.e 3:7dba033a:::benchmark_data_smithi074_35447_object8691:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e43)", 2020-08-07T15:07:46.916 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:23.311281+0000", 2020-08-07T15:07:46.917 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.188648532999999, 2020-08-07T15:07:46.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24762457700000001, 2020-08-07T15:07:46.917 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:46.917 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:46.920 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:46.921 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:46.921 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:46.921 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 69535 2020-08-07T15:07:46.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.922 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:46.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:46.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.311281+0000", 2020-08-07T15:07:46.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:46.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:46.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.311281+0000", 2020-08-07T15:07:46.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1261999999999998e-05 2020-08-07T15:07:46.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:46.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.311332+0000", 2020-08-07T15:07:46.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.198e-06 2020-08-07T15:07:46.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:46.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.311337+0000", 2020-08-07T15:07:46.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0242e-05 2020-08-07T15:07:46.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:46.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.311347+0000", 2020-08-07T15:07:46.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2220000000000004e-06 2020-08-07T15:07:46.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:46.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.311352+0000", 2020-08-07T15:07:46.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044450189999999997 2020-08-07T15:07:46.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:46.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.315797+0000", 2020-08-07T15:07:46.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9657999999999997e-05 2020-08-07T15:07:46.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:46.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.315856+0000", 2020-08-07T15:07:46.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010547700000000001 2020-08-07T15:07:46.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:46.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.315962+0000", 2020-08-07T15:07:46.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050302630000000001 2020-08-07T15:07:46.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:46.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.320992+0000", 2020-08-07T15:07:46.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.105e-06 2020-08-07T15:07:46.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:46.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.320993+0000", 2020-08-07T15:07:46.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9343000000000003e-05 2020-08-07T15:07:46.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:46.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.321062+0000", 2020-08-07T15:07:46.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001560566 2020-08-07T15:07:46.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:46.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.322623+0000", 2020-08-07T15:07:46.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23620960299999999 2020-08-07T15:07:46.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:46.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.558833+0000", 2020-08-07T15:07:46.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5499999999999996e-07 2020-08-07T15:07:46.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:46.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.558834+0000", 2020-08-07T15:07:46.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1663999999999997e-05 2020-08-07T15:07:46.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:46.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.558905+0000", 2020-08-07T15:07:46.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24762457700000001 2020-08-07T15:07:46.938 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.938 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:46.939 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.939 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:69536 3.e 3:7dba033a:::benchmark_data_smithi074_35447_object8691:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e43)", 2020-08-07T15:07:46.939 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:23.311385+0000", 2020-08-07T15:07:46.940 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.188543911, 2020-08-07T15:07:46.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24764810000000001, 2020-08-07T15:07:46.940 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:46.940 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:46.940 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:46.940 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:46.941 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:46.941 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 69536 2020-08-07T15:07:46.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.941 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:46.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:46.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.311385+0000", 2020-08-07T15:07:46.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:46.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:46.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.311385+0000", 2020-08-07T15:07:46.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.761e-05 2020-08-07T15:07:46.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:46.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.311433+0000", 2020-08-07T15:07:46.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0599999999999998e-06 2020-08-07T15:07:46.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:46.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.311438+0000", 2020-08-07T15:07:46.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0456e-05 2020-08-07T15:07:46.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:46.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.311448+0000", 2020-08-07T15:07:46.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2020-08-07T15:07:46.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:46.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.311453+0000", 2020-08-07T15:07:46.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049773370000000001 2020-08-07T15:07:46.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:46.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.316430+0000", 2020-08-07T15:07:46.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000108507 2020-08-07T15:07:46.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:46.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.316538+0000", 2020-08-07T15:07:46.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000166706 2020-08-07T15:07:46.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:46.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.316705+0000", 2020-08-07T15:07:46.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004371425 2020-08-07T15:07:46.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:46.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.321076+0000", 2020-08-07T15:07:46.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.92e-07 2020-08-07T15:07:46.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:46.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.321077+0000", 2020-08-07T15:07:46.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6020999999999999e-05 2020-08-07T15:07:46.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:46.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.321123+0000", 2020-08-07T15:07:46.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017699650000000001 2020-08-07T15:07:46.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:46.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.322893+0000", 2020-08-07T15:07:46.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23607164999999999 2020-08-07T15:07:46.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:46.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.558965+0000", 2020-08-07T15:07:46.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6199999999999997e-07 2020-08-07T15:07:46.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:46.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.558966+0000", 2020-08-07T15:07:46.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7787000000000007e-05 2020-08-07T15:07:46.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:46.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:23.559033+0000", 2020-08-07T15:07:46.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24764810000000001 2020-08-07T15:07:46.958 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.958 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:46.958 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.959 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:86712 3.8 e52/20 3:14925daa:::benchmark_data_smithi074_35447_object10838:head v 52'5488, mlcod=52'5479)", 2020-08-07T15:07:46.959 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.338631+0000", 2020-08-07T15:07:46.959 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 12.161297779, 2020-08-07T15:07:46.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.310229594, 2020-08-07T15:07:46.960 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:46.960 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:46.960 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:46.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:46.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338631+0000", 2020-08-07T15:07:46.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:46.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:46.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338631+0000", 2020-08-07T15:07:46.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6180000000000001e-05 2020-08-07T15:07:46.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:46.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338688+0000", 2020-08-07T15:07:46.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4949999999999996e-06 2020-08-07T15:07:46.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:46.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338694+0000", 2020-08-07T15:07:46.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30438292099999997 2020-08-07T15:07:46.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:46.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643077+0000", 2020-08-07T15:07:46.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9389999999999996e-06 2020-08-07T15:07:46.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:46.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643082+0000", 2020-08-07T15:07:46.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5096999999999999e-05 2020-08-07T15:07:46.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:46.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643127+0000", 2020-08-07T15:07:46.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4323000000000001e-05 2020-08-07T15:07:46.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:46.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643151+0000", 2020-08-07T15:07:46.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055847300000000004 2020-08-07T15:07:46.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:46.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.648736+0000", 2020-08-07T15:07:46.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012490900000000001 2020-08-07T15:07:46.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:46.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.648861+0000", 2020-08-07T15:07:46.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.310229594 2020-08-07T15:07:46.970 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.970 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:46.971 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:46.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.971 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86792 3.f 3:fc5386ee:::benchmark_data_smithi074_35447_object10848:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:07:46.971 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.395402+0000", 2020-08-07T15:07:46.971 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 12.104527668999999, 2020-08-07T15:07:46.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30259831999999998, 2020-08-07T15:07:46.972 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:46.972 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:46.972 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:46.972 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:46.973 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:46.973 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86792 2020-08-07T15:07:46.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.973 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:46.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:46.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395402+0000", 2020-08-07T15:07:46.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:46.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:46.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395402+0000", 2020-08-07T15:07:46.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2262999999999998e-05 2020-08-07T15:07:46.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:46.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395444+0000", 2020-08-07T15:07:46.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1800000000000004e-06 2020-08-07T15:07:46.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:46.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395449+0000", 2020-08-07T15:07:46.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0738e-05 2020-08-07T15:07:46.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:46.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395460+0000", 2020-08-07T15:07:46.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7219999999999999e-06 2020-08-07T15:07:46.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:46.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395464+0000", 2020-08-07T15:07:46.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0063169089999999999 2020-08-07T15:07:46.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:46.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.401781+0000", 2020-08-07T15:07:46.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010702200000000001 2020-08-07T15:07:46.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:46.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:46.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:46.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.401888+0000", 2020-08-07T15:07:47.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019171999999999999 2020-08-07T15:07:47.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:07:47.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.402080+0000", 2020-08-07T15:07:47.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042866839999999998 2020-08-07T15:07:47.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:47.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.406367+0000", 2020-08-07T15:07:47.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000001e-07 2020-08-07T15:07:47.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:47.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.406367+0000", 2020-08-07T15:07:47.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4720000000000004e-05 2020-08-07T15:07:47.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:47.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.406432+0000", 2020-08-07T15:07:47.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050405365000000001 2020-08-07T15:07:47.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:47.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.456837+0000", 2020-08-07T15:07:47.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24111702700000001 2020-08-07T15:07:47.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:47.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.697954+0000", 2020-08-07T15:07:47.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.22e-06 2020-08-07T15:07:47.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:47.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.697955+0000", 2020-08-07T15:07:47.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4336999999999999e-05 2020-08-07T15:07:47.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:47.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.698000+0000", 2020-08-07T15:07:47.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30259831999999998 2020-08-07T15:07:47.012 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:47.012 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:47.012 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:47.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.013 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:97576 3.d e57/35 3:b9e86280:::benchmark_data_smithi074_35447_object12196:head v 57'6304, mlcod=57'6295)", 2020-08-07T15:07:47.013 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:41.740780+0000", 2020-08-07T15:07:47.013 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.7591495510000001, 2020-08-07T15:07:47.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27784160200000002, 2020-08-07T15:07:47.014 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:47.014 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:47.014 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:47.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:47.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.740780+0000", 2020-08-07T15:07:47.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:47.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:47.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.740780+0000", 2020-08-07T15:07:47.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0355999999999997e-05 2020-08-07T15:07:47.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:47.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.740830+0000", 2020-08-07T15:07:47.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6680000000000004e-06 2020-08-07T15:07:47.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:47.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.740837+0000", 2020-08-07T15:07:47.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27346419799999999 2020-08-07T15:07:47.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:47.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.014301+0000", 2020-08-07T15:07:47.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.852e-06 2020-08-07T15:07:47.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:47.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.014307+0000", 2020-08-07T15:07:47.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6504e-05 2020-08-07T15:07:47.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:47.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.014363+0000", 2020-08-07T15:07:47.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8759999999999999e-05 2020-08-07T15:07:47.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:47.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.014392+0000", 2020-08-07T15:07:47.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041857459999999997 2020-08-07T15:07:47.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:47.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.018578+0000", 2020-08-07T15:07:47.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3517999999999999e-05 2020-08-07T15:07:47.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:47.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.018621+0000", 2020-08-07T15:07:47.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27784160200000002 2020-08-07T15:07:47.034 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:47.034 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:47.034 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:47.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.035 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:104104 3.8 e58/20 3:11c276ea:::benchmark_data_smithi074_35447_object13012:head v 58'6496, mlcod=58'6487)", 2020-08-07T15:07:47.035 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:07:47.035 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.64144352699999996, 2020-08-07T15:07:47.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33178497099999998, 2020-08-07T15:07:47.035 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:47.036 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:47.036 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:47.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:47.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:07:47.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:47.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:47.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:07:47.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9961999999999999e-05 2020-08-07T15:07:47.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:47.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858516+0000", 2020-08-07T15:07:47.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3200000000000001e-06 2020-08-07T15:07:47.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:47.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858520+0000", 2020-08-07T15:07:47.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32756790000000002 2020-08-07T15:07:47.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:47.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186088+0000", 2020-08-07T15:07:47.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0260000000000004e-06 2020-08-07T15:07:47.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:47.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186094+0000", 2020-08-07T15:07:47.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7919000000000001e-05 2020-08-07T15:07:47.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:47.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186132+0000", 2020-08-07T15:07:47.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4968e-05 2020-08-07T15:07:47.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:47.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186147+0000", 2020-08-07T15:07:47.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040671520000000001 2020-08-07T15:07:47.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:47.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.190214+0000", 2020-08-07T15:07:47.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6724000000000001e-05 2020-08-07T15:07:47.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:47.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:47.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:47.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.190271+0000", 2020-08-07T15:07:47.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33178497099999998 2020-08-07T15:07:47.046 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:47.046 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:47.046 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:47.047 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:47.047 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:47.047 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:47.047 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:07:47.050 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:47.049+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:47.151 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:47.149+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:47.234 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:47.234 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:07:47.235 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:07:47.235 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:47.247 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:07:47.247 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 71 16 106091 106075 11.671 12.5469 0.00421908 0.0107064 2020-08-07T15:07:47.259 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:47.258+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:47.350 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:07:47.351 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.3 to 0.791797 (cab3) 2020-08-07T15:07:47.364 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:47.362+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:47.438 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:47.440 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:07:47.440 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:07:47.440 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:07:47.440 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:47.449 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:07:47.503 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:47.499+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:47.574 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:47.574 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:07:47.574 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:07:47.575 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:07:47.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.575 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:10240 3.6 3:665d71ff:::benchmark_data_smithi074_35447_object1279:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:47.575 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:07:47.575 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 65.178370792999999, 2020-08-07T15:07:47.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25191569699999999, 2020-08-07T15:07:47.576 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:47.576 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:47.576 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:47.576 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:47.577 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:47.577 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 10240 2020-08-07T15:07:47.577 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.577 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:47.577 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.587 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:47.587 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:07:47.587 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:47.587 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.588 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:47.588 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:07:47.588 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5463000000000002e-05 2020-08-07T15:07:47.588 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.589 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:47.589 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393264+0000", 2020-08-07T15:07:47.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9219999999999999e-06 2020-08-07T15:07:47.589 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.590 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.590 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:47.590 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393268+0000", 2020-08-07T15:07:47.590 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6149999999999997e-06 2020-08-07T15:07:47.591 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.591 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.591 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:47.591 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393277+0000", 2020-08-07T15:07:47.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2039999999999996e-06 2020-08-07T15:07:47.599 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.605 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:47.605 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393282+0000", 2020-08-07T15:07:47.605 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001225243 2020-08-07T15:07:47.605 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.606 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.606 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:47.606 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394507+0000", 2020-08-07T15:07:47.606 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7108999999999995e-05 2020-08-07T15:07:47.606 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.606 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.610 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:47.611 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394605+0000", 2020-08-07T15:07:47.611 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017490800000000001 2020-08-07T15:07:47.611 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.614 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.614 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:47.615 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394779+0000", 2020-08-07T15:07:47.615 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032608699999999999 2020-08-07T15:07:47.615 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.615 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.616 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:47.616 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.398040+0000", 2020-08-07T15:07:47.616 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013015259999999999 2020-08-07T15:07:47.616 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.616 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.617 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:47.617 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.399342+0000", 2020-08-07T15:07:47.617 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24566893400000001 2020-08-07T15:07:47.617 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.617 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.618 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:47.618 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645011+0000", 2020-08-07T15:07:47.618 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.217e-06 2020-08-07T15:07:47.619 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.619 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.619 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:47.619 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645012+0000", 2020-08-07T15:07:47.619 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0803000000000006e-05 2020-08-07T15:07:47.620 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.620 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.620 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:47.620 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645083+0000", 2020-08-07T15:07:47.620 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1882999999999998e-05 2020-08-07T15:07:47.621 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.621 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.621 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:47.621 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645135+0000", 2020-08-07T15:07:47.621 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25191569699999999 2020-08-07T15:07:47.622 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.622 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:47.622 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.623 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.623 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.623 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:31760 3.6 e30/26 3:6672be19:::benchmark_data_smithi074_35447_object3969:head v 30'2104, mlcod=30'2094)", 2020-08-07T15:07:47.623 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:56.999757+0000", 2020-08-07T15:07:47.623 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 50.571832872000002, 2020-08-07T15:07:47.624 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17823482099999999, 2020-08-07T15:07:47.624 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:47.624 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:47.624 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:47.624 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.625 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:47.625 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:56.999757+0000", 2020-08-07T15:07:47.625 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:47.625 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.625 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.626 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:47.626 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:56.999757+0000", 2020-08-07T15:07:47.626 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2955999999999999e-05 2020-08-07T15:07:47.626 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.627 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:47.627 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:56.999810+0000", 2020-08-07T15:07:47.627 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5509999999999996e-06 2020-08-07T15:07:47.627 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.628 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.628 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:47.628 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:56.999816+0000", 2020-08-07T15:07:47.628 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.177361504 2020-08-07T15:07:47.628 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.628 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.629 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:47.632 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177178+0000", 2020-08-07T15:07:47.634 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6979999999999997e-06 2020-08-07T15:07:47.634 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.634 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.634 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:47.634 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177183+0000", 2020-08-07T15:07:47.635 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1974999999999997e-05 2020-08-07T15:07:47.635 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.635 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.635 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:47.635 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177225+0000", 2020-08-07T15:07:47.636 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4612000000000001e-05 2020-08-07T15:07:47.636 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.636 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.636 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:47.636 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177249+0000", 2020-08-07T15:07:47.637 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00069063000000000002 2020-08-07T15:07:47.637 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.637 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.637 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:47.637 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177940+0000", 2020-08-07T15:07:47.638 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1894999999999997e-05 2020-08-07T15:07:47.638 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.638 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.638 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:47.638 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177992+0000", 2020-08-07T15:07:47.638 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17823482099999999 2020-08-07T15:07:47.639 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.639 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:47.639 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.639 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.639 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.640 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:43335 3.6 e32/26 3:65a57ed3:::benchmark_data_smithi074_35447_object5416:head v 32'2831, mlcod=32'2822)", 2020-08-07T15:07:47.640 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:03.998850+0000", 2020-08-07T15:07:47.640 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 43.572739722000001, 2020-08-07T15:07:47.640 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.221384421, 2020-08-07T15:07:47.640 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:47.641 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:47.641 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:47.641 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.641 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:47.641 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998850+0000", 2020-08-07T15:07:47.642 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:47.642 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.642 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.642 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:47.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998850+0000", 2020-08-07T15:07:47.643 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4788000000000004e-05 2020-08-07T15:07:47.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.643 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.643 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:47.643 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998915+0000", 2020-08-07T15:07:47.643 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7909999999999999e-06 2020-08-07T15:07:47.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.644 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.644 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:47.644 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998922+0000", 2020-08-07T15:07:47.644 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22050499700000001 2020-08-07T15:07:47.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:47.645 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219427+0000", 2020-08-07T15:07:47.645 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0950000000000001e-06 2020-08-07T15:07:47.646 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.646 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.646 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:47.646 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219433+0000", 2020-08-07T15:07:47.646 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6342999999999997e-05 2020-08-07T15:07:47.647 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.647 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.647 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:47.647 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219509+0000", 2020-08-07T15:07:47.647 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4173999999999999e-05 2020-08-07T15:07:47.648 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.649 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.649 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:47.649 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219533+0000", 2020-08-07T15:07:47.649 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00064214899999999995 2020-08-07T15:07:47.649 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.650 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.650 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:47.650 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.220175+0000", 2020-08-07T15:07:47.650 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9083999999999999e-05 2020-08-07T15:07:47.650 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.651 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.651 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:47.651 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.220234+0000", 2020-08-07T15:07:47.651 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.221384421 2020-08-07T15:07:47.651 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.652 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:47.652 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.652 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.652 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.652 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:43336 3.6 e32/26 3:65a57ed3:::benchmark_data_smithi074_35447_object5416:head v 32'2832, mlcod=32'2822)", 2020-08-07T15:07:47.653 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:03.999509+0000", 2020-08-07T15:07:47.653 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 43.572080393, 2020-08-07T15:07:47.653 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22632864799999999, 2020-08-07T15:07:47.653 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:47.653 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:47.655 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:47.655 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.655 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:47.656 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999509+0000", 2020-08-07T15:07:47.656 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:47.656 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.656 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.657 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:47.657 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999509+0000", 2020-08-07T15:07:47.657 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0107e-05 2020-08-07T15:07:47.657 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.657 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.658 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:47.658 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999559+0000", 2020-08-07T15:07:47.658 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0889999999999996e-06 2020-08-07T15:07:47.661 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.661 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.661 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:47.661 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999566+0000", 2020-08-07T15:07:47.661 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22332094599999999 2020-08-07T15:07:47.662 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.662 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.672 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:47.672 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222887+0000", 2020-08-07T15:07:47.673 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5990000000000001e-06 2020-08-07T15:07:47.673 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.673 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.673 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:47.673 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222892+0000", 2020-08-07T15:07:47.674 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2342999999999998e-05 2020-08-07T15:07:47.674 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.674 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.674 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:47.674 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222924+0000", 2020-08-07T15:07:47.675 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9108999999999999e-05 2020-08-07T15:07:47.675 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.675 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.675 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:47.675 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222944+0000", 2020-08-07T15:07:47.676 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028409289999999999 2020-08-07T15:07:47.676 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.676 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.676 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:47.676 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.225784+0000", 2020-08-07T15:07:47.677 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3526000000000002e-05 2020-08-07T15:07:47.677 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:47.678 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.225838+0000", 2020-08-07T15:07:47.678 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22632864799999999 2020-08-07T15:07:47.678 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.678 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:47.678 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.679 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:43784 3.6 e32/26 3:6da53356:::benchmark_data_smithi074_35447_object5472:head v 32'2848, mlcod=32'2839)", 2020-08-07T15:07:47.679 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:04.253355+0000", 2020-08-07T15:07:47.679 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 43.318234820999997, 2020-08-07T15:07:47.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21456700500000001, 2020-08-07T15:07:47.680 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:47.680 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:47.680 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:47.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:47.681 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253355+0000", 2020-08-07T15:07:47.682 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:47.682 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.682 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.682 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:47.682 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253355+0000", 2020-08-07T15:07:47.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6168000000000002e-05 2020-08-07T15:07:47.683 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.683 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.683 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:47.683 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253401+0000", 2020-08-07T15:07:47.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3179999999999997e-06 2020-08-07T15:07:47.684 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.684 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.684 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:47.684 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253406+0000", 2020-08-07T15:07:47.684 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21194338600000001 2020-08-07T15:07:47.685 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.685 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.685 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:47.685 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465350+0000", 2020-08-07T15:07:47.686 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3759999999999999e-06 2020-08-07T15:07:47.687 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.688 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.688 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:47.688 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465355+0000", 2020-08-07T15:07:47.688 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8250999999999997e-05 2020-08-07T15:07:47.689 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.689 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.689 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:47.690 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465393+0000", 2020-08-07T15:07:47.690 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0296999999999999e-05 2020-08-07T15:07:47.690 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.690 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.690 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:47.691 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465414+0000", 2020-08-07T15:07:47.691 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024515209999999999 2020-08-07T15:07:47.691 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.691 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.691 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:47.692 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.467865+0000", 2020-08-07T15:07:47.692 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6687999999999997e-05 2020-08-07T15:07:47.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.692 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:47.693 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.467922+0000", 2020-08-07T15:07:47.693 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21456700500000001 2020-08-07T15:07:47.693 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.693 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:47.693 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.708 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.708 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:44152 3.6 e32/26 3:6c90cb4b:::benchmark_data_smithi074_35447_object5518:head v 32'2856, mlcod=32'2851)", 2020-08-07T15:07:47.709 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:04.473943+0000", 2020-08-07T15:07:47.709 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 43.097646998999998, 2020-08-07T15:07:47.709 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18875260999999999, 2020-08-07T15:07:47.709 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:47.710 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:47.710 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:47.710 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.710 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:47.710 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.473943+0000", 2020-08-07T15:07:47.711 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:47.711 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.711 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:47.712 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.473943+0000", 2020-08-07T15:07:47.712 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8267999999999999e-05 2020-08-07T15:07:47.713 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.713 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.713 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:47.713 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.473991+0000", 2020-08-07T15:07:47.713 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9689999999999999e-06 2020-08-07T15:07:47.714 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.714 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.714 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:47.714 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.473997+0000", 2020-08-07T15:07:47.714 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18142124000000001 2020-08-07T15:07:47.715 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.715 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.715 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:47.715 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.655418+0000", 2020-08-07T15:07:47.715 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7439999999999998e-06 2020-08-07T15:07:47.716 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.716 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.716 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:47.716 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.655423+0000", 2020-08-07T15:07:47.716 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0052999999999997e-05 2020-08-07T15:07:47.717 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.717 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.717 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:47.717 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.655473+0000", 2020-08-07T15:07:47.717 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3739000000000001e-05 2020-08-07T15:07:47.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.722 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:47.722 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.655497+0000", 2020-08-07T15:07:47.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0071399030000000004 2020-08-07T15:07:47.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.723 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:47.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.662637+0000", 2020-08-07T15:07:47.724 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8693999999999997e-05 2020-08-07T15:07:47.724 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.724 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.724 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:47.724 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.662695+0000", 2020-08-07T15:07:47.725 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18875260999999999 2020-08-07T15:07:47.725 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.725 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:47.725 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.735 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.735 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:54200 3.3 e37/20 3:cc40637e:::benchmark_data_smithi074_35447_object6774:head v 37'3248, mlcod=37'3236)", 2020-08-07T15:07:47.736 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:07:47.736 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 34.473027573000003, 2020-08-07T15:07:47.736 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24065872499999999, 2020-08-07T15:07:47.736 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:47.736 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:47.737 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:47.737 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.737 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:47.737 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:07:47.737 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:47.738 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.738 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.738 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:47.738 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:07:47.738 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1671000000000002e-05 2020-08-07T15:07:47.739 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.739 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.739 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:47.739 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098594+0000", 2020-08-07T15:07:47.739 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0890000000000002e-06 2020-08-07T15:07:47.740 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.740 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.740 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:47.740 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098598+0000", 2020-08-07T15:07:47.740 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23875592300000001 2020-08-07T15:07:47.740 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.741 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.741 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:47.741 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337354+0000", 2020-08-07T15:07:47.741 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0420000000000002e-06 2020-08-07T15:07:47.754 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.754 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.754 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:47.754 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337359+0000", 2020-08-07T15:07:47.755 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0586000000000001e-05 2020-08-07T15:07:47.755 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.755 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:47.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337399+0000", 2020-08-07T15:07:47.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9415999999999998e-05 2020-08-07T15:07:47.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:47.756 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337419+0000", 2020-08-07T15:07:47.757 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017568390000000001 2020-08-07T15:07:47.757 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.757 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:47.757 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.339176+0000", 2020-08-07T15:07:47.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5158999999999999e-05 2020-08-07T15:07:47.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.763 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:47.763 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.339221+0000", 2020-08-07T15:07:47.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24065872499999999 2020-08-07T15:07:47.763 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.764 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:47.764 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.764 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.764 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:54728 3.7 e37/20 3:ee999259:::benchmark_data_smithi074_35447_object6840:head v 37'3328, mlcod=37'3319)", 2020-08-07T15:07:47.764 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:13.452195+0000", 2020-08-07T15:07:47.765 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 34.119394792000001, 2020-08-07T15:07:47.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.187979807, 2020-08-07T15:07:47.765 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:47.765 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:47.765 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:47.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.766 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:47.766 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.452195+0000", 2020-08-07T15:07:47.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:47.766 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:47.767 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.452195+0000", 2020-08-07T15:07:47.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9023000000000001e-05 2020-08-07T15:07:47.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.768 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.768 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:47.768 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.452244+0000", 2020-08-07T15:07:47.768 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.733e-06 2020-08-07T15:07:47.768 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.769 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.769 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:47.769 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.452250+0000", 2020-08-07T15:07:47.769 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18453694300000001 2020-08-07T15:07:47.769 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.782 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:47.782 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.636787+0000", 2020-08-07T15:07:47.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5679999999999999e-06 2020-08-07T15:07:47.782 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:47.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.636792+0000", 2020-08-07T15:07:47.783 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0119999999999999e-05 2020-08-07T15:07:47.783 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:47.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.636872+0000", 2020-08-07T15:07:47.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6356999999999999e-05 2020-08-07T15:07:47.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:47.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.636899+0000", 2020-08-07T15:07:47.786 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003233737 2020-08-07T15:07:47.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.786 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:47.786 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.640132+0000", 2020-08-07T15:07:47.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2326000000000001e-05 2020-08-07T15:07:47.787 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.789 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.790 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:47.790 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.640175+0000", 2020-08-07T15:07:47.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.187979807 2020-08-07T15:07:47.790 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.791 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:47.791 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.791 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:71136 3.6 e43/26 3:64cb7acb:::benchmark_data_smithi074_35447_object8891:head v 43'4744, mlcod=43'4739)", 2020-08-07T15:07:47.792 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:07:47.792 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 23.252976207, 2020-08-07T15:07:47.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26135482999999998, 2020-08-07T15:07:47.792 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:47.792 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:47.793 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:47.793 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.793 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:47.793 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:07:47.793 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:47.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:47.800 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:07:47.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3702999999999997e-05 2020-08-07T15:07:47.801 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.801 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.801 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:47.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318667+0000", 2020-08-07T15:07:47.802 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3119999999999999e-06 2020-08-07T15:07:47.802 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.802 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.802 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:47.802 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318674+0000", 2020-08-07T15:07:47.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.255173292 2020-08-07T15:07:47.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:47.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.573847+0000", 2020-08-07T15:07:47.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1140000000000002e-06 2020-08-07T15:07:47.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:47.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.573853+0000", 2020-08-07T15:07:47.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00057603799999999996 2020-08-07T15:07:47.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:47.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.574429+0000", 2020-08-07T15:07:47.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4776e-05 2020-08-07T15:07:47.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:47.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.574444+0000", 2020-08-07T15:07:47.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054577080000000004 2020-08-07T15:07:47.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.809 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:47.809 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.579901+0000", 2020-08-07T15:07:47.810 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6886999999999998e-05 2020-08-07T15:07:47.810 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.810 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.810 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:47.810 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.579968+0000", 2020-08-07T15:07:47.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26135482999999998 2020-08-07T15:07:47.811 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.811 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:47.811 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.812 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:71760 3.f e43/20 3:f213ad20:::benchmark_data_smithi074_35447_object8969:head v 43'4784, mlcod=43'4776)", 2020-08-07T15:07:47.812 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:24.699138+0000", 2020-08-07T15:07:47.813 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 22.872451730000002, 2020-08-07T15:07:47.813 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17350447599999999, 2020-08-07T15:07:47.813 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:47.814 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:47.814 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:47.814 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.815 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:47.815 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.699138+0000", 2020-08-07T15:07:47.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:47.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:47.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.699138+0000", 2020-08-07T15:07:47.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1221000000000001e-05 2020-08-07T15:07:47.816 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.817 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:47.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.699209+0000", 2020-08-07T15:07:47.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6989999999999997e-06 2020-08-07T15:07:47.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.818 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:47.818 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.699216+0000", 2020-08-07T15:07:47.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17275928900000001 2020-08-07T15:07:47.818 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.818 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:47.819 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.871975+0000", 2020-08-07T15:07:47.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0990000000000001e-06 2020-08-07T15:07:47.819 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:47.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.871979+0000", 2020-08-07T15:07:47.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8105999999999997e-05 2020-08-07T15:07:47.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:47.821 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.872017+0000", 2020-08-07T15:07:47.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7921e-05 2020-08-07T15:07:47.821 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:47.822 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.872035+0000", 2020-08-07T15:07:47.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00055102099999999995 2020-08-07T15:07:47.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.822 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.822 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:47.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.872586+0000", 2020-08-07T15:07:47.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6119999999999998e-05 2020-08-07T15:07:47.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:47.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.872642+0000", 2020-08-07T15:07:47.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17350447599999999 2020-08-07T15:07:47.824 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.824 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:47.824 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.825 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:81544 3.7 e48/20 3:e9bb655d:::benchmark_data_smithi074_35447_object10192:head v 48'4944, mlcod=48'4935)", 2020-08-07T15:07:47.825 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:30.902099+0000", 2020-08-07T15:07:47.825 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.669491248, 2020-08-07T15:07:47.826 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20669163500000001, 2020-08-07T15:07:47.826 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:47.826 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:47.826 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:47.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.827 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:47.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.902099+0000", 2020-08-07T15:07:47.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:47.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:47.828 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.902099+0000", 2020-08-07T15:07:47.828 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1317999999999999e-05 2020-08-07T15:07:47.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.829 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:47.829 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.902150+0000", 2020-08-07T15:07:47.838 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2079999999999997e-06 2020-08-07T15:07:47.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.839 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:47.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.902156+0000", 2020-08-07T15:07:47.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20264375100000001 2020-08-07T15:07:47.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.840 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:47.840 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.104800+0000", 2020-08-07T15:07:47.840 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2390000000000002e-06 2020-08-07T15:07:47.840 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.840 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:47.841 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.104805+0000", 2020-08-07T15:07:47.841 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3696000000000003e-05 2020-08-07T15:07:47.841 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.842 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:47.842 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.104849+0000", 2020-08-07T15:07:47.842 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4179e-05 2020-08-07T15:07:47.842 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.842 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.843 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:47.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.104873+0000", 2020-08-07T15:07:47.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038653839999999999 2020-08-07T15:07:47.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.844 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:47.844 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.108738+0000", 2020-08-07T15:07:47.844 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1860000000000002e-05 2020-08-07T15:07:47.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:47.845 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.108790+0000", 2020-08-07T15:07:47.845 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20669163500000001 2020-08-07T15:07:47.845 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.845 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:47.846 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.846 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.846 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:85696 3.f e51/20 3:f99fcea4:::benchmark_data_smithi074_35447_object10711:head v 51'5688, mlcod=51'5679)", 2020-08-07T15:07:47.846 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:07:47.846 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 13.918984794, 2020-08-07T15:07:47.847 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49655717300000002, 2020-08-07T15:07:47.847 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:47.847 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:47.847 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:47.847 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.848 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:47.848 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:07:47.848 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:47.848 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.849 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.850 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:47.850 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:07:47.850 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9543000000000002e-05 2020-08-07T15:07:47.850 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.851 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.851 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:47.851 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652665+0000", 2020-08-07T15:07:47.851 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8700000000000003e-06 2020-08-07T15:07:47.851 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.851 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:47.852 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652671+0000", 2020-08-07T15:07:47.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49579595799999998 2020-08-07T15:07:47.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.853 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:47.853 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148467+0000", 2020-08-07T15:07:47.853 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2509999999999997e-06 2020-08-07T15:07:47.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.854 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:47.854 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148475+0000", 2020-08-07T15:07:47.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5603999999999998e-05 2020-08-07T15:07:47.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.855 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:47.855 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148530+0000", 2020-08-07T15:07:47.855 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5486999999999998e-05 2020-08-07T15:07:47.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.856 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.856 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:47.856 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148556+0000", 2020-08-07T15:07:47.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00055666600000000004 2020-08-07T15:07:47.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:47.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.149112+0000", 2020-08-07T15:07:47.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9793999999999998e-05 2020-08-07T15:07:47.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.858 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:47.858 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.149162+0000", 2020-08-07T15:07:47.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49655717300000002 2020-08-07T15:07:47.858 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.859 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:47.859 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.859 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.859 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:86792 3.f e52/20 3:fc5386ee:::benchmark_data_smithi074_35447_object10848:head v 52'5752, mlcod=52'5743)", 2020-08-07T15:07:47.860 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:07:47.861 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 13.169139018999999, 2020-08-07T15:07:47.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.295270005, 2020-08-07T15:07:47.862 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:47.862 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:47.862 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:47.862 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.862 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:47.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:07:47.863 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:47.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:47.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:07:47.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9712000000000001e-05 2020-08-07T15:07:47.864 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:47.865 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402500+0000", 2020-08-07T15:07:47.865 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6789999999999999e-06 2020-08-07T15:07:47.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.866 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:47.866 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402507+0000", 2020-08-07T15:07:47.866 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29317074999999998 2020-08-07T15:07:47.867 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.867 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.867 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:47.867 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695678+0000", 2020-08-07T15:07:47.867 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8120000000000002e-06 2020-08-07T15:07:47.868 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.868 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.868 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:47.868 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695685+0000", 2020-08-07T15:07:47.869 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1270999999999998e-05 2020-08-07T15:07:47.869 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.869 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.869 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:47.869 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695726+0000", 2020-08-07T15:07:47.870 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2739e-05 2020-08-07T15:07:47.870 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.870 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.871 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:47.871 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695749+0000", 2020-08-07T15:07:47.871 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019333899999999999 2020-08-07T15:07:47.871 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.872 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:47.872 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.697682+0000", 2020-08-07T15:07:47.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8652000000000002e-05 2020-08-07T15:07:47.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.872 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.873 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:47.873 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.697721+0000", 2020-08-07T15:07:47.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.295270005 2020-08-07T15:07:47.873 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.874 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:47.874 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.874 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.874 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.874 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:87088 3.6 e52/26 3:6b0320df:::benchmark_data_smithi074_35447_object10885:head v 52'5912, mlcod=52'5902)", 2020-08-07T15:07:47.875 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:07:47.875 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 12.604905778999999, 2020-08-07T15:07:47.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29522189799999998, 2020-08-07T15:07:47.875 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:47.875 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:47.875 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:47.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.876 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:47.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:07:47.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:47.876 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.877 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.877 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:47.877 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:07:47.877 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0751000000000002e-05 2020-08-07T15:07:47.877 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.878 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.878 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:47.878 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966735+0000", 2020-08-07T15:07:47.878 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3080000000000004e-06 2020-08-07T15:07:47.878 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.878 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.879 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:47.879 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966741+0000", 2020-08-07T15:07:47.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29365034699999998 2020-08-07T15:07:47.879 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.880 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:47.880 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260391+0000", 2020-08-07T15:07:47.880 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2720000000000003e-06 2020-08-07T15:07:47.880 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.881 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:47.881 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260398+0000", 2020-08-07T15:07:47.881 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8819000000000005e-05 2020-08-07T15:07:47.881 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.881 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.882 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:47.882 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260476+0000", 2020-08-07T15:07:47.882 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4958999999999999e-05 2020-08-07T15:07:47.882 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.883 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.883 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:47.883 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260501+0000", 2020-08-07T15:07:47.883 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001353641 2020-08-07T15:07:47.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.884 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.884 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:47.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.261855+0000", 2020-08-07T15:07:47.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0800999999999997e-05 2020-08-07T15:07:47.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:47.885 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.261906+0000", 2020-08-07T15:07:47.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29522189799999998 2020-08-07T15:07:47.885 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.886 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:47.886 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.886 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.887 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:93920 3.6 e56/26 3:631393d3:::benchmark_data_smithi074_35447_object11739:head v 56'6352, mlcod=55'6342)", 2020-08-07T15:07:47.887 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:07:47.887 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 8.2447903480000004, 2020-08-07T15:07:47.887 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39765404300000001, 2020-08-07T15:07:47.888 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:47.888 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:47.888 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:47.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:47.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:07:47.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:47.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.891 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:47.891 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:07:47.891 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3035999999999998e-05 2020-08-07T15:07:47.901 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.901 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.902 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:47.902 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326852+0000", 2020-08-07T15:07:47.902 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5379999999999999e-06 2020-08-07T15:07:47.902 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:47.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326859+0000", 2020-08-07T15:07:47.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.393917568 2020-08-07T15:07:47.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:47.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.720777+0000", 2020-08-07T15:07:47.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2430000000000005e-06 2020-08-07T15:07:47.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:47.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.720785+0000", 2020-08-07T15:07:47.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00065658700000000003 2020-08-07T15:07:47.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:47.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.721441+0000", 2020-08-07T15:07:47.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8585000000000002e-05 2020-08-07T15:07:47.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:47.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.721470+0000", 2020-08-07T15:07:47.918 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00293349 2020-08-07T15:07:47.918 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.918 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:47.919 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.724403+0000", 2020-08-07T15:07:47.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9996e-05 2020-08-07T15:07:47.919 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.919 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:47.920 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.724453+0000", 2020-08-07T15:07:47.920 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39765404300000001 2020-08-07T15:07:47.920 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.920 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:47.920 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.921 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:94384 3.3 e56/20 3:ce533213:::benchmark_data_smithi074_35447_object11797:head v 56'5488, mlcod=56'5479)", 2020-08-07T15:07:47.921 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.637245+0000", 2020-08-07T15:07:47.921 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 7.934344844, 2020-08-07T15:07:47.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.247824459, 2020-08-07T15:07:47.922 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:47.922 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:47.922 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:47.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:47.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637245+0000", 2020-08-07T15:07:47.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:47.925 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.926 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:47.926 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637245+0000", 2020-08-07T15:07:47.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0782999999999997e-05 2020-08-07T15:07:47.926 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:47.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637306+0000", 2020-08-07T15:07:47.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8699999999999992e-06 2020-08-07T15:07:47.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:47.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637314+0000", 2020-08-07T15:07:47.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24469571800000001 2020-08-07T15:07:47.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:47.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882009+0000", 2020-08-07T15:07:47.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8510000000000004e-06 2020-08-07T15:07:47.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:47.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882015+0000", 2020-08-07T15:07:47.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6708999999999997e-05 2020-08-07T15:07:47.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:47.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882052+0000", 2020-08-07T15:07:47.931 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0280000000000001e-05 2020-08-07T15:07:47.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.932 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.932 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:47.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882082+0000", 2020-08-07T15:07:47.932 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002934767 2020-08-07T15:07:47.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.933 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.933 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:47.933 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.885017+0000", 2020-08-07T15:07:47.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2481000000000001e-05 2020-08-07T15:07:47.934 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.934 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.934 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:47.934 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.885069+0000", 2020-08-07T15:07:47.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.247824459 2020-08-07T15:07:47.935 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.935 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:47.935 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.936 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:96832 3.6 e57/26 3:6d818ad0:::benchmark_data_smithi074_35447_object12103:head v 57'6536, mlcod=56'6526)", 2020-08-07T15:07:47.936 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:41.323206+0000", 2020-08-07T15:07:47.936 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.2483839489999999, 2020-08-07T15:07:47.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.188700062, 2020-08-07T15:07:47.937 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:47.937 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:47.937 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:47.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:47.938 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.323206+0000", 2020-08-07T15:07:47.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:47.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:47.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.323206+0000", 2020-08-07T15:07:47.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6401999999999999e-05 2020-08-07T15:07:47.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:47.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.323232+0000", 2020-08-07T15:07:47.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3380000000000001e-06 2020-08-07T15:07:47.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.944 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:47.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.323236+0000", 2020-08-07T15:07:47.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18768147800000001 2020-08-07T15:07:47.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:47.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.510917+0000", 2020-08-07T15:07:47.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.536e-06 2020-08-07T15:07:47.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.948 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:47.948 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.510923+0000", 2020-08-07T15:07:47.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3084999999999998e-05 2020-08-07T15:07:47.949 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.950 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:47.950 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.510966+0000", 2020-08-07T15:07:47.950 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8317e-05 2020-08-07T15:07:47.950 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.950 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:47.951 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.510994+0000", 2020-08-07T15:07:47.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00087632200000000002 2020-08-07T15:07:47.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.952 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:47.952 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.511870+0000", 2020-08-07T15:07:47.952 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5583999999999997e-05 2020-08-07T15:07:47.952 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.952 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.953 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:47.953 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.511906+0000", 2020-08-07T15:07:47.953 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.188700062 2020-08-07T15:07:47.953 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.953 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:47.954 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.954 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.954 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.954 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:98624 3.6 e57/26 3:6fe31b68:::benchmark_data_smithi074_35447_object12327:head v 57'6648, mlcod=57'6641)", 2020-08-07T15:07:47.954 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:42.318711+0000", 2020-08-07T15:07:47.955 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.2528786609999996, 2020-08-07T15:07:47.955 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18025650700000001, 2020-08-07T15:07:47.955 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:47.955 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:47.955 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:47.955 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.956 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:47.956 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.318711+0000", 2020-08-07T15:07:47.956 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:47.956 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.956 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.957 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:47.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.318711+0000", 2020-08-07T15:07:47.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7558000000000001e-05 2020-08-07T15:07:47.957 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:47.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.318759+0000", 2020-08-07T15:07:47.958 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7010000000000001e-06 2020-08-07T15:07:47.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:47.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.318764+0000", 2020-08-07T15:07:47.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.178359881 2020-08-07T15:07:47.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:47.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.497124+0000", 2020-08-07T15:07:47.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0409999999999997e-06 2020-08-07T15:07:47.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:47.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.497129+0000", 2020-08-07T15:07:47.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0988000000000001e-05 2020-08-07T15:07:47.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:47.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.497170+0000", 2020-08-07T15:07:47.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3748999999999999e-05 2020-08-07T15:07:47.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.962 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:47.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.497194+0000", 2020-08-07T15:07:47.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001726357 2020-08-07T15:07:47.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:47.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.498920+0000", 2020-08-07T15:07:47.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7231999999999998e-05 2020-08-07T15:07:47.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:47.964 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.498968+0000", 2020-08-07T15:07:47.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18025650700000001 2020-08-07T15:07:47.965 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.965 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:47.965 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.966 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:104184 3.1 e58/26 3:841b6818:::benchmark_data_smithi074_35447_object13022:head v 58'6200, mlcod=58'6196)", 2020-08-07T15:07:47.966 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:45.910556+0000", 2020-08-07T15:07:47.966 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 1.66103398, 2020-08-07T15:07:47.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.184209715, 2020-08-07T15:07:47.967 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:47.967 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:47.967 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:47.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:47.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.910556+0000", 2020-08-07T15:07:47.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:47.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:47.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.910556+0000", 2020-08-07T15:07:47.969 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7603e-05 2020-08-07T15:07:47.969 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.970 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:47.970 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.910583+0000", 2020-08-07T15:07:47.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0899999999999998e-06 2020-08-07T15:07:47.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:47.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.910587+0000", 2020-08-07T15:07:47.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17963316800000001 2020-08-07T15:07:47.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:47.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.090221+0000", 2020-08-07T15:07:47.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2870000000000001e-06 2020-08-07T15:07:47.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:47.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.090227+0000", 2020-08-07T15:07:47.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9820000000000002e-05 2020-08-07T15:07:47.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:47.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.090267+0000", 2020-08-07T15:07:47.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4604e-05 2020-08-07T15:07:47.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.975 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:47.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.090291+0000", 2020-08-07T15:07:47.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044229129999999997 2020-08-07T15:07:47.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:47.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.094714+0000", 2020-08-07T15:07:47.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1230000000000002e-05 2020-08-07T15:07:47.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:47.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.094765+0000", 2020-08-07T15:07:47.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.184209715 2020-08-07T15:07:47.977 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.977 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:47.978 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.978 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:104560 3.f e58/20 3:fb4383ae:::benchmark_data_smithi074_35447_object13069:head v 58'6888, mlcod=58'6878)", 2020-08-07T15:07:47.978 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:46.301207+0000", 2020-08-07T15:07:47.979 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 1.2703830270000001, 2020-08-07T15:07:47.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23687350400000001, 2020-08-07T15:07:47.979 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:47.979 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:47.979 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:47.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:47.980 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301207+0000", 2020-08-07T15:07:47.980 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:47.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.981 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:47.981 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301207+0000", 2020-08-07T15:07:47.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7945000000000003e-05 2020-08-07T15:07:47.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.982 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:47.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301255+0000", 2020-08-07T15:07:47.982 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2030000000000002e-06 2020-08-07T15:07:47.982 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.983 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:47.983 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301259+0000", 2020-08-07T15:07:47.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23565420500000001 2020-08-07T15:07:47.983 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.983 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.983 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:47.984 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.536913+0000", 2020-08-07T15:07:47.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2290000000000003e-06 2020-08-07T15:07:47.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.984 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:47.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.536921+0000", 2020-08-07T15:07:47.985 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0955e-05 2020-08-07T15:07:47.985 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.985 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:47.986 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.537002+0000", 2020-08-07T15:07:47.986 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8977999999999999e-05 2020-08-07T15:07:47.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.987 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:47.987 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.537031+0000", 2020-08-07T15:07:47.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00099831000000000008 2020-08-07T15:07:47.987 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:47.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.538030+0000", 2020-08-07T15:07:47.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0679e-05 2020-08-07T15:07:47.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:47.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:47.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:47.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.538080+0000", 2020-08-07T15:07:47.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23687350400000001 2020-08-07T15:07:47.989 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.989 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:47.990 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.990 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:47.990 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:47.990 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:47.990 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:47.571+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:47.991 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:07:47.993 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:47.991+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:47.993 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:07:48.097 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:48.095+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:48.165 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:48.166 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:07:48.166 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.167 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106261 3.8 3.281b4ff8 (undecoded) ondisk+write+known_if_redirected e59)", 2020-08-07T15:07:48.167 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.369315+0000", 2020-08-07T15:07:48.167 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.79494002100000005, 2020-08-07T15:07:48.167 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.79496059200000002, 2020-08-07T15:07:48.167 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:48.168 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "delayed", 2020-08-07T15:07:48.168 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:48.168 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:48.168 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:48.168 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106261 2020-08-07T15:07:48.169 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.169 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:48.169 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.169 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:48.170 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.369315+0000", 2020-08-07T15:07:48.170 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:48.170 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.170 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.170 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:48.171 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.369315+0000", 2020-08-07T15:07:48.171 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4261999999999997e-05 2020-08-07T15:07:48.171 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.171 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.171 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:48.172 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.369359+0000", 2020-08-07T15:07:48.172 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9479999999999996e-06 2020-08-07T15:07:48.172 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.172 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.172 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:48.173 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.369365+0000", 2020-08-07T15:07:48.173 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.226e-05 2020-08-07T15:07:48.173 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.173 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.173 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:48.174 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.369377+0000", 2020-08-07T15:07:48.174 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6149999999999996e-06 2020-08-07T15:07:48.174 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.174 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.174 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:48.175 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.369383+0000", 2020-08-07T15:07:48.175 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3016000000000002e-05 2020-08-07T15:07:48.175 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.175 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.175 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:48.177 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.369436+0000", 2020-08-07T15:07:48.177 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5499999999999995e-06 2020-08-07T15:07:48.177 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.177 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.178 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:07:48.178 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.369444+0000", 2020-08-07T15:07:48.178 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000129651 2020-08-07T15:07:48.178 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.179 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:48.179 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.179 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.179 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.179 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106262 3.8 3.281b4ff8 (undecoded) ondisk+write+known_if_redirected e59)", 2020-08-07T15:07:48.180 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.369663+0000", 2020-08-07T15:07:48.180 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.79459159000000001, 2020-08-07T15:07:48.180 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.79465878099999998, 2020-08-07T15:07:48.181 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:48.181 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "delayed", 2020-08-07T15:07:48.181 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:48.181 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:48.182 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:48.182 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106262 2020-08-07T15:07:48.182 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.182 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:48.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.183 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:48.183 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.369663+0000", 2020-08-07T15:07:48.183 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:48.183 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.183 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.184 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:48.184 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.369663+0000", 2020-08-07T15:07:48.184 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5272000000000003e-05 2020-08-07T15:07:48.184 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.184 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.185 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:48.185 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.369708+0000", 2020-08-07T15:07:48.185 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8209999999999998e-06 2020-08-07T15:07:48.185 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.185 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.186 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:48.186 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.369714+0000", 2020-08-07T15:07:48.186 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.403e-05 2020-08-07T15:07:48.187 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.187 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.187 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:48.187 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.369728+0000", 2020-08-07T15:07:48.187 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5770000000000001e-06 2020-08-07T15:07:48.188 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.188 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.188 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:48.188 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.369734+0000", 2020-08-07T15:07:48.188 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8550000000000002e-05 2020-08-07T15:07:48.189 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.189 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.189 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:48.189 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.369772+0000", 2020-08-07T15:07:48.189 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9110000000000005e-06 2020-08-07T15:07:48.189 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.190 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.190 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:07:48.190 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.369782+0000", 2020-08-07T15:07:48.190 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011916100000000001 2020-08-07T15:07:48.190 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.191 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:48.191 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.191 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.191 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.191 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106263 3.8 3.281b4ff8 (undecoded) ondisk+write+known_if_redirected e59)", 2020-08-07T15:07:48.192 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.536923+0000", 2020-08-07T15:07:48.192 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.62733172199999998, 2020-08-07T15:07:48.192 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.62743787500000003, 2020-08-07T15:07:48.192 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:48.192 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "delayed", 2020-08-07T15:07:48.193 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:48.193 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:48.193 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:48.193 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106263 2020-08-07T15:07:48.194 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.194 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:48.194 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.194 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:48.194 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.536923+0000", 2020-08-07T15:07:48.195 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:48.195 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.195 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.195 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:48.195 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.536923+0000", 2020-08-07T15:07:48.196 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1606000000000002e-05 2020-08-07T15:07:48.196 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.196 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.196 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:48.196 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.536955+0000", 2020-08-07T15:07:48.197 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1310000000000003e-06 2020-08-07T15:07:48.197 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.197 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.197 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:48.197 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.536961+0000", 2020-08-07T15:07:48.198 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4630999999999999e-05 2020-08-07T15:07:48.198 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.198 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.198 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:48.198 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.536975+0000", 2020-08-07T15:07:48.199 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5499999999999997e-06 2020-08-07T15:07:48.199 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.199 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.199 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:48.199 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.536983+0000", 2020-08-07T15:07:48.200 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8721999999999997e-05 2020-08-07T15:07:48.200 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.200 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.200 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:48.200 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.537052+0000", 2020-08-07T15:07:48.201 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3978000000000001e-05 2020-08-07T15:07:48.201 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.201 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.201 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:07:48.201 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.537066+0000", 2020-08-07T15:07:48.202 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014261800000000001 2020-08-07T15:07:48.202 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.202 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:48.202 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.202 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.203 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.203 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106264 3.8 3.281b4ff8 (undecoded) ondisk+write+known_if_redirected e59)", 2020-08-07T15:07:48.203 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.694163+0000", 2020-08-07T15:07:48.203 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.47009201299999998, 2020-08-07T15:07:48.203 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.47023615499999999, 2020-08-07T15:07:48.204 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:48.204 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "delayed", 2020-08-07T15:07:48.204 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:48.204 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:48.204 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:48.205 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106264 2020-08-07T15:07:48.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.205 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:48.205 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.205 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:48.206 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694163+0000", 2020-08-07T15:07:48.206 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:48.206 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.206 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.206 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:48.207 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694163+0000", 2020-08-07T15:07:48.207 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6906e-05 2020-08-07T15:07:48.207 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.207 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.207 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:48.207 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694190+0000", 2020-08-07T15:07:48.208 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4669999999999998e-06 2020-08-07T15:07:48.208 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.208 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.208 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:48.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694194+0000", 2020-08-07T15:07:48.209 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0490000000000004e-06 2020-08-07T15:07:48.209 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.209 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.209 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:48.210 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694202+0000", 2020-08-07T15:07:48.210 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0599999999999998e-06 2020-08-07T15:07:48.210 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.210 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.210 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:48.211 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694207+0000", 2020-08-07T15:07:48.211 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9650999999999997e-05 2020-08-07T15:07:48.211 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.211 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.211 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:48.212 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694247+0000", 2020-08-07T15:07:48.212 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6060000000000003e-06 2020-08-07T15:07:48.212 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.212 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.212 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:07:48.213 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694255+0000", 2020-08-07T15:07:48.213 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2738999999999994e-05 2020-08-07T15:07:48.213 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.213 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:48.213 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.214 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.214 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.214 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106205 3.8 3.53585198 (undecoded) ondisk+retry+write+known_if_redirected e59)", 2020-08-07T15:07:48.214 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.332257+0000", 2020-08-07T15:07:48.214 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.83199746200000002, 2020-08-07T15:07:48.215 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.83218098799999995, 2020-08-07T15:07:48.215 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:48.215 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "delayed", 2020-08-07T15:07:48.216 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:48.216 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:48.217 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:48.217 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106205 2020-08-07T15:07:48.217 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.217 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:48.217 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.218 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:48.218 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332257+0000", 2020-08-07T15:07:48.218 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:48.219 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.224 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.224 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:48.224 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332257+0000", 2020-08-07T15:07:48.225 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6199999999999997e-05 2020-08-07T15:07:48.225 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.225 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.225 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:48.225 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332313+0000", 2020-08-07T15:07:48.226 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5886e-05 2020-08-07T15:07:48.226 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.226 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.226 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:48.227 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332329+0000", 2020-08-07T15:07:48.227 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6392000000000002e-05 2020-08-07T15:07:48.227 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.227 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.227 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:48.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332346+0000", 2020-08-07T15:07:48.228 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2790000000000001e-06 2020-08-07T15:07:48.230 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.230 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.230 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:48.230 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332353+0000", 2020-08-07T15:07:48.230 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5106999999999999e-05 2020-08-07T15:07:48.231 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.231 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.231 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:48.231 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332408+0000", 2020-08-07T15:07:48.232 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7410000000000004e-06 2020-08-07T15:07:48.232 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.232 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.232 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op must wait for map", 2020-08-07T15:07:48.232 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332418+0000", 2020-08-07T15:07:48.233 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0073256750000000002 2020-08-07T15:07:48.233 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.233 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.233 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:48.233 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.339743+0000", 2020-08-07T15:07:48.234 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0033e-05 2020-08-07T15:07:48.234 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.234 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.234 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:07:48.235 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.339753+0000", 2020-08-07T15:07:48.235 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074963130000000001 2020-08-07T15:07:48.235 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.235 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:48.235 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.236 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.236 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.236 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106206 3.8 3.53585198 (undecoded) ondisk+retry+write+known_if_redirected e59)", 2020-08-07T15:07:48.236 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.332396+0000", 2020-08-07T15:07:48.236 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.83185888900000005, 2020-08-07T15:07:48.237 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.83208649199999996, 2020-08-07T15:07:48.237 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:48.237 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "delayed", 2020-08-07T15:07:48.237 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:48.237 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:48.238 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:48.238 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106206 2020-08-07T15:07:48.238 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.238 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:48.238 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.239 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:48.239 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332396+0000", 2020-08-07T15:07:48.239 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:48.239 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.239 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.239 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:48.240 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332396+0000", 2020-08-07T15:07:48.240 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6459999999999999e-05 2020-08-07T15:07:48.240 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.240 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.240 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:48.241 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332442+0000", 2020-08-07T15:07:48.241 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1529999999999996e-06 2020-08-07T15:07:48.241 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.241 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.241 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:48.242 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332447+0000", 2020-08-07T15:07:48.242 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0299000000000001e-05 2020-08-07T15:07:48.242 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.242 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.242 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:48.243 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332458+0000", 2020-08-07T15:07:48.243 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5060000000000003e-06 2020-08-07T15:07:48.243 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.243 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.243 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:48.244 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332463+0000", 2020-08-07T15:07:48.244 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2706e-05 2020-08-07T15:07:48.244 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.244 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.244 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:48.245 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332516+0000", 2020-08-07T15:07:48.245 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2640000000000002e-06 2020-08-07T15:07:48.245 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.245 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.245 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:07:48.246 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332523+0000", 2020-08-07T15:07:48.246 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0072592120000000001 2020-08-07T15:07:48.246 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.246 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.246 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:48.246 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.339782+0000", 2020-08-07T15:07:48.247 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8709999999999999e-06 2020-08-07T15:07:48.247 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.247 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.247 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:07:48.248 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.339789+0000", 2020-08-07T15:07:48.248 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0073934710000000004 2020-08-07T15:07:48.248 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.248 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:48.248 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.249 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.249 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.249 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106353 3.8 3.d4d09778 (undecoded) ondisk+write+known_if_redirected e59)", 2020-08-07T15:07:48.249 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:48.081964+0000", 2020-08-07T15:07:48.249 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.082290990999999994, 2020-08-07T15:07:48.250 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.082563056999999995, 2020-08-07T15:07:48.250 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:48.250 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "delayed", 2020-08-07T15:07:48.250 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:48.250 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:48.251 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:48.251 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106353 2020-08-07T15:07:48.251 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.251 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:48.251 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.251 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:48.252 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.081964+0000", 2020-08-07T15:07:48.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:48.252 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.252 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.252 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:48.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.081964+0000", 2020-08-07T15:07:48.253 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4215999999999996e-05 2020-08-07T15:07:48.253 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.253 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:48.254 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.082028+0000", 2020-08-07T15:07:48.254 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4019999999999999e-06 2020-08-07T15:07:48.254 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.254 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.254 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:48.255 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.082034+0000", 2020-08-07T15:07:48.255 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6957999999999999e-05 2020-08-07T15:07:48.255 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.256 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.256 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:48.256 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.082051+0000", 2020-08-07T15:07:48.256 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8760000000000005e-06 2020-08-07T15:07:48.256 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.257 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.257 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:48.257 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.082059+0000", 2020-08-07T15:07:48.257 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4391000000000002e-05 2020-08-07T15:07:48.257 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.258 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.258 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:48.258 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.082103+0000", 2020-08-07T15:07:48.258 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0494e-05 2020-08-07T15:07:48.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.259 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.259 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:07:48.259 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.082114+0000", 2020-08-07T15:07:48.259 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015033700000000001 2020-08-07T15:07:48.260 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.260 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:48.260 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.260 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.260 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.262 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106207 3.8 3.53585198 (undecoded) ondisk+retry+write+known_if_redirected e59)", 2020-08-07T15:07:48.262 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.332496+0000", 2020-08-07T15:07:48.263 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.83175897099999996, 2020-08-07T15:07:48.263 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.83207051899999995, 2020-08-07T15:07:48.263 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:48.263 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "delayed", 2020-08-07T15:07:48.263 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:48.264 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:48.264 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:48.264 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106207 2020-08-07T15:07:48.264 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.265 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:48.266 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.266 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:48.266 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332496+0000", 2020-08-07T15:07:48.267 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:48.267 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.267 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.267 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:48.267 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332496+0000", 2020-08-07T15:07:48.268 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5994e-05 2020-08-07T15:07:48.268 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.268 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.268 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:48.269 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332542+0000", 2020-08-07T15:07:48.269 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8590000000000004e-06 2020-08-07T15:07:48.269 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.269 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.269 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:48.270 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332547+0000", 2020-08-07T15:07:48.270 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.145e-05 2020-08-07T15:07:48.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:48.273 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332558+0000", 2020-08-07T15:07:48.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4089999999999996e-06 2020-08-07T15:07:48.273 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.273 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.274 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:48.274 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332562+0000", 2020-08-07T15:07:48.274 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.757e-05 2020-08-07T15:07:48.274 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.274 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.275 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:48.275 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332610+0000", 2020-08-07T15:07:48.275 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5879999999999999e-06 2020-08-07T15:07:48.275 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.275 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.276 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:07:48.276 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332615+0000", 2020-08-07T15:07:48.276 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0071977849999999999 2020-08-07T15:07:48.276 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.276 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.277 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:48.277 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.339812+0000", 2020-08-07T15:07:48.277 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5340000000000002e-06 2020-08-07T15:07:48.277 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.277 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.278 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:07:48.278 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.339820+0000", 2020-08-07T15:07:48.278 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.007324189 2020-08-07T15:07:48.278 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.279 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:48.279 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.279 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.279 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.279 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106208 3.8 3.53585198 (undecoded) ondisk+retry+write+known_if_redirected e59)", 2020-08-07T15:07:48.280 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.332606+0000", 2020-08-07T15:07:48.280 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.83164897199999999, 2020-08-07T15:07:48.280 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.83200476899999998, 2020-08-07T15:07:48.280 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:48.280 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "delayed", 2020-08-07T15:07:48.281 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:48.281 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:48.281 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:48.281 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106208 2020-08-07T15:07:48.281 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.282 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:48.282 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.282 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:48.282 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332606+0000", 2020-08-07T15:07:48.282 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:48.283 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.283 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.283 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:48.283 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332606+0000", 2020-08-07T15:07:48.283 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0093999999999998e-05 2020-08-07T15:07:48.284 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.284 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.284 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:48.284 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332666+0000", 2020-08-07T15:07:48.284 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1070000000000004e-06 2020-08-07T15:07:48.284 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.285 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.285 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:48.285 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332671+0000", 2020-08-07T15:07:48.285 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1035e-05 2020-08-07T15:07:48.285 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.286 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.286 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:48.286 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332682+0000", 2020-08-07T15:07:48.286 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9529999999999998e-06 2020-08-07T15:07:48.286 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.287 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:48.287 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332687+0000", 2020-08-07T15:07:48.287 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016443799999999999 2020-08-07T15:07:48.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.288 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.288 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:48.288 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332851+0000", 2020-08-07T15:07:48.288 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8569999999999995e-06 2020-08-07T15:07:48.288 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.289 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.289 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:07:48.289 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.332859+0000", 2020-08-07T15:07:48.289 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.011897909 2020-08-07T15:07:48.289 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.290 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.290 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:48.290 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344757+0000", 2020-08-07T15:07:48.290 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7170000000000003e-06 2020-08-07T15:07:48.290 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.290 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.291 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:07:48.291 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344767+0000", 2020-08-07T15:07:48.291 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.012161109999999999 2020-08-07T15:07:48.292 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.292 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:48.292 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.292 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.292 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.293 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106257 3.8 3.281b4ff8 (undecoded) ondisk+write+known_if_redirected e59)", 2020-08-07T15:07:48.293 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.367552+0000", 2020-08-07T15:07:48.293 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.79670293199999997, 2020-08-07T15:07:48.293 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.79710273300000001, 2020-08-07T15:07:48.294 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:48.294 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "delayed", 2020-08-07T15:07:48.294 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:48.294 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:48.294 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:48.295 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106257 2020-08-07T15:07:48.295 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.295 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:48.295 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.295 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:48.296 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.367552+0000", 2020-08-07T15:07:48.296 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:48.296 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.296 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.296 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:48.297 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.367552+0000", 2020-08-07T15:07:48.297 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6276000000000001e-05 2020-08-07T15:07:48.297 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.297 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.297 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:48.298 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.367608+0000", 2020-08-07T15:07:48.298 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2930000000000006e-06 2020-08-07T15:07:48.298 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.298 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.298 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:48.299 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.367617+0000", 2020-08-07T15:07:48.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7737e-05 2020-08-07T15:07:48.299 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.299 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.299 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:48.299 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.367635+0000", 2020-08-07T15:07:48.300 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2289999999999992e-06 2020-08-07T15:07:48.300 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.300 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.300 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:48.301 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.367644+0000", 2020-08-07T15:07:48.301 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5332000000000004e-05 2020-08-07T15:07:48.301 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.301 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.301 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:48.302 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.367710+0000", 2020-08-07T15:07:48.302 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1045e-05 2020-08-07T15:07:48.302 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.302 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.302 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:07:48.303 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.367721+0000", 2020-08-07T15:07:48.303 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016891200000000001 2020-08-07T15:07:48.303 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.303 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:48.303 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.304 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.304 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.304 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106258 3.8 3.281b4ff8 (undecoded) ondisk+write+known_if_redirected e59)", 2020-08-07T15:07:48.304 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.368494+0000", 2020-08-07T15:07:48.304 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.79576026499999997, 2020-08-07T15:07:48.305 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.79620568800000002, 2020-08-07T15:07:48.305 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:48.305 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "delayed", 2020-08-07T15:07:48.305 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:48.305 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:48.306 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:48.306 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106258 2020-08-07T15:07:48.306 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.306 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:48.306 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.307 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:48.307 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.368494+0000", 2020-08-07T15:07:48.307 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:48.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.307 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:48.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.368494+0000", 2020-08-07T15:07:48.308 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7286999999999997e-05 2020-08-07T15:07:48.308 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.308 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.309 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:48.309 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.368532+0000", 2020-08-07T15:07:48.309 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2020000000000001e-06 2020-08-07T15:07:48.309 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.310 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.310 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:48.310 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.368538+0000", 2020-08-07T15:07:48.310 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0923e-05 2020-08-07T15:07:48.310 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.310 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.311 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:48.311 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.368549+0000", 2020-08-07T15:07:48.311 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3000000000000003e-06 2020-08-07T15:07:48.311 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.311 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.312 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:48.313 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.368553+0000", 2020-08-07T15:07:48.313 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5601e-05 2020-08-07T15:07:48.314 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.314 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.315 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:48.325 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.368589+0000", 2020-08-07T15:07:48.325 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0669999999999992e-06 2020-08-07T15:07:48.326 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.326 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.326 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:07:48.326 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.368597+0000", 2020-08-07T15:07:48.326 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010238000000000001 2020-08-07T15:07:48.327 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.327 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:48.327 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.327 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.327 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.328 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106259 3.8 3.281b4ff8 (undecoded) ondisk+write+known_if_redirected e59)", 2020-08-07T15:07:48.328 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.368577+0000", 2020-08-07T15:07:48.328 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.79567801699999996, 2020-08-07T15:07:48.328 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.79616060200000005, 2020-08-07T15:07:48.329 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:48.329 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "delayed", 2020-08-07T15:07:48.329 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:48.331 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:48.331 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:48.331 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106259 2020-08-07T15:07:48.332 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.341 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:48.342 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.342 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:48.342 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.368577+0000", 2020-08-07T15:07:48.343 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:48.343 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.343 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.343 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:48.344 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.368577+0000", 2020-08-07T15:07:48.344 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1504000000000002e-05 2020-08-07T15:07:48.344 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.344 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.344 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:48.344 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.368608+0000", 2020-08-07T15:07:48.349 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.168e-06 2020-08-07T15:07:48.350 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.350 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.350 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:48.351 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.368611+0000", 2020-08-07T15:07:48.351 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9959999999999999e-06 2020-08-07T15:07:48.351 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.351 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.351 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:48.352 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.368617+0000", 2020-08-07T15:07:48.352 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6639999999999998e-06 2020-08-07T15:07:48.352 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.352 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.352 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:48.353 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.368620+0000", 2020-08-07T15:07:48.353 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7848e-05 2020-08-07T15:07:48.353 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.353 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.353 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:48.353 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.368668+0000", 2020-08-07T15:07:48.354 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0690000000000004e-06 2020-08-07T15:07:48.354 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.354 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.354 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:07:48.354 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.368675+0000", 2020-08-07T15:07:48.355 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8249000000000001e-05 2020-08-07T15:07:48.355 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.355 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:48.355 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.355 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.356 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.356 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106260 3.8 3.281b4ff8 (undecoded) ondisk+write+known_if_redirected e59)", 2020-08-07T15:07:48.356 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.368640+0000", 2020-08-07T15:07:48.356 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.795614496, 2020-08-07T15:07:48.356 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.79613630499999999, 2020-08-07T15:07:48.357 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:48.357 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "delayed", 2020-08-07T15:07:48.357 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:48.357 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:48.357 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:48.358 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106260 2020-08-07T15:07:48.358 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.358 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:48.358 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.358 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:48.359 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.368640+0000", 2020-08-07T15:07:48.359 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:48.359 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.359 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.359 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:48.360 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.368640+0000", 2020-08-07T15:07:48.360 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0678e-05 2020-08-07T15:07:48.360 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.360 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.360 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:48.360 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.368671+0000", 2020-08-07T15:07:48.361 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0000000000000001e-06 2020-08-07T15:07:48.361 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.361 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.361 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:48.361 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.368674+0000", 2020-08-07T15:07:48.362 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7570000000000001e-06 2020-08-07T15:07:48.362 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.362 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.362 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:48.362 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.368680+0000", 2020-08-07T15:07:48.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5629999999999999e-06 2020-08-07T15:07:48.363 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.363 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.363 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:48.363 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.368682+0000", 2020-08-07T15:07:48.364 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3431e-05 2020-08-07T15:07:48.364 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.365 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:48.365 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.368706+0000", 2020-08-07T15:07:48.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3080000000000001e-06 2020-08-07T15:07:48.365 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:48.365 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:48.366 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:07:48.366 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.368710+0000", 2020-08-07T15:07:48.366 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9737e-05 2020-08-07T15:07:48.367 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.367 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:48.367 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.367 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:48.367 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:07:48.368 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 13 2020-08-07T15:07:48.368 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:48.371 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:07:48.372 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:07:48.372 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:07:48.372 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:07:48.372 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:07:48.372 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:07:48.373 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:07:48.373 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:07:48.373 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:07:48.373 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 72 16 106354 106338 11.5374 2.05469 0.131629 0.0107231 2020-08-07T15:07:48.373 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:07:48.374 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:07:48.374 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:07:48.374 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:07:48.374 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:07:48.374 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:07:48.375 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:07:48.375 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:07:48.375 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:48.368+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:48.476 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:48.473+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:48.559 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:48.560 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:48.560 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:07:48.579 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:48.578+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:48.690 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:48.680+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:48.850 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:48.851 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:48.862 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:07:48.863 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:48.855+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:48.955 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:48.953+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:49.032 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:49.032 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:49.042 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:07:49.086 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:49.081+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:49.156 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:49.155+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:49.209 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:49.209 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:49.218 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:07:49.247 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 73 16 106356 106340 11.3796 0.015625 0.414204 0.0107353 2020-08-07T15:07:49.257 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:49.255+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:49.346 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:49.346 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:49.365 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:07:49.368 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:07:49.370 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:49.361+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:49.471 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:49.468+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:49.579 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:49.580 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:49.580 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:49.570+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:49.592 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:07:49.675 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:49.673+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:49.772 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:49.772 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:49.782 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:49.777+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:49.783 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:07:49.883 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:49.880+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:49.986 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:49.985+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:50.019 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:50.019 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:50.041 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:07:50.103 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:50.101+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:50.194 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:50.193+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:50.246 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:50.247 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:50.247 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 74 16 107828 107812 11.3812 11.5 0.0117949 0.0109796 2020-08-07T15:07:50.259 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:07:50.298 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:50.294+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:50.347 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:07:50.387 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:07:50.411 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:50.399+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:50.492 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:50.493 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:50.508 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:07:50.509 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:50.507+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:50.603 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:07:50.631 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:50.628+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:50.638 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:07:50.668 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:07:50.699 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:07:50.711 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:50.709+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:50.719 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:50.720 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:50.757 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:07:50.773 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:07:50.774 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:07:50.799 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:07:50.816 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:50.814+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:50.930 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:50.922+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:50.931 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:50.931 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:07:50.931 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2020-08-07T15:07:50.931 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:50.943 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:07:51.025 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:07:51.025 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:51.023+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:51.026 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:07:51.026 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:07:51.026 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:07:51.027 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:07:51.027 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:07:51.027 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:07:51.027 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:07:51.027 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:07:51.106 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:51.107 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:07:51.118 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:07:51.119 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:07:51.119 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:51.120 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:07:51.130 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:51.129+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:51.231 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:51.229+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:51.246 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:07:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:07:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:15744 3.d e27/20 3:b5661143:::benchmark_data_smithi074_35447_object1967:head v 27'984, mlcod=27'977)", 2020-08-07T15:07:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.777088+0000", 2020-08-07T15:07:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.461412703999997, 2020-08-07T15:07:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23236351199999999, 2020-08-07T15:07:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777088+0000", 2020-08-07T15:07:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777088+0000", 2020-08-07T15:07:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9495e-05 2020-08-07T15:07:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777137+0000", 2020-08-07T15:07:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3049999999999998e-06 2020-08-07T15:07:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.777143+0000", 2020-08-07T15:07:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23074995500000001 2020-08-07T15:07:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.007893+0000", 2020-08-07T15:07:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4390000000000003e-06 2020-08-07T15:07:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.007901+0000", 2020-08-07T15:07:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0071000000000003e-05 2020-08-07T15:07:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.007941+0000", 2020-08-07T15:07:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2722e-05 2020-08-07T15:07:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.007964+0000", 2020-08-07T15:07:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014374870000000001 2020-08-07T15:07:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.009401+0000", 2020-08-07T15:07:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0037999999999998e-05 2020-08-07T15:07:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.009451+0000", 2020-08-07T15:07:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23236351199999999 2020-08-07T15:07:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:33488 3.c 3:3d1929c7:::benchmark_data_smithi074_35447_object4185:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-08-07T15:07:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:58.021233+0000", 2020-08-07T15:07:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 53.217267434, 2020-08-07T15:07:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.2237055, 2020-08-07T15:07:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 33488 2020-08-07T15:07:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.021233+0000", 2020-08-07T15:07:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.021233+0000", 2020-08-07T15:07:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7682000000000002e-05 2020-08-07T15:07:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.021281+0000", 2020-08-07T15:07:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.955e-06 2020-08-07T15:07:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.021285+0000", 2020-08-07T15:07:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2132e-05 2020-08-07T15:07:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.021298+0000", 2020-08-07T15:07:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1390000000000003e-06 2020-08-07T15:07:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.021303+0000", 2020-08-07T15:07:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011846789999999999 2020-08-07T15:07:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.022487+0000", 2020-08-07T15:07:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3844000000000002e-05 2020-08-07T15:07:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.022541+0000", 2020-08-07T15:07:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013274700000000001 2020-08-07T15:07:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:07:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.022674+0000", 2020-08-07T15:07:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00161185 2020-08-07T15:07:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.024286+0000", 2020-08-07T15:07:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8800000000000003e-07 2020-08-07T15:07:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.024286+0000", 2020-08-07T15:07:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8810999999999997e-05 2020-08-07T15:07:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.024335+0000", 2020-08-07T15:07:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025354919999999999 2020-08-07T15:07:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.026871+0000", 2020-08-07T15:07:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21800207099999999 2020-08-07T15:07:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.244873+0000", 2020-08-07T15:07:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2279999999999999e-06 2020-08-07T15:07:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.244874+0000", 2020-08-07T15:07:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4381999999999994e-05 2020-08-07T15:07:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.244938+0000", 2020-08-07T15:07:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.2237055 2020-08-07T15:07:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:41432 3.d e32/20 3:b5c8a160:::benchmark_data_smithi074_35447_object5178:head v 32'2848, mlcod=32'2838)", 2020-08-07T15:07:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.794575+0000", 2020-08-07T15:07:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.443925301999997, 2020-08-07T15:07:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36243746199999999, 2020-08-07T15:07:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794575+0000", 2020-08-07T15:07:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794575+0000", 2020-08-07T15:07:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0696999999999999e-05 2020-08-07T15:07:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794626+0000", 2020-08-07T15:07:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4189999999999999e-06 2020-08-07T15:07:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794632+0000", 2020-08-07T15:07:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36145730399999998 2020-08-07T15:07:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.156089+0000", 2020-08-07T15:07:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2329999999999997e-06 2020-08-07T15:07:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.156095+0000", 2020-08-07T15:07:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0054999999999999e-05 2020-08-07T15:07:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.156125+0000", 2020-08-07T15:07:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5369000000000001e-05 2020-08-07T15:07:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.156140+0000", 2020-08-07T15:07:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00076810300000000001 2020-08-07T15:07:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.156908+0000", 2020-08-07T15:07:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104282 2020-08-07T15:07:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.157012+0000", 2020-08-07T15:07:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36243746199999999 2020-08-07T15:07:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:62248 3.d 3:bc63bb1c:::benchmark_data_smithi074_35447_object7780:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e41)", 2020-08-07T15:07:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:18.423100+0000", 2020-08-07T15:07:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.815400578999999, 2020-08-07T15:07:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.236870208, 2020-08-07T15:07:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 62248 2020-08-07T15:07:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.423100+0000", 2020-08-07T15:07:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.423100+0000", 2020-08-07T15:07:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0126000000000001e-05 2020-08-07T15:07:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.423150+0000", 2020-08-07T15:07:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8900000000000004e-06 2020-08-07T15:07:51.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:51.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.423156+0000", 2020-08-07T15:07:51.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1775e-05 2020-08-07T15:07:51.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:51.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.423167+0000", 2020-08-07T15:07:51.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0509999999999999e-06 2020-08-07T15:07:51.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:51.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.423175+0000", 2020-08-07T15:07:51.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0047868310000000001 2020-08-07T15:07:51.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:51.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.427961+0000", 2020-08-07T15:07:51.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012365499999999999 2020-08-07T15:07:51.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:51.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.428085+0000", 2020-08-07T15:07:51.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022334000000000001 2020-08-07T15:07:51.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,4,6,7", 2020-08-07T15:07:51.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.428308+0000", 2020-08-07T15:07:51.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011845499999999999 2020-08-07T15:07:51.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:51.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.429493+0000", 2020-08-07T15:07:51.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.020861682999999999 2020-08-07T15:07:51.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.450355+0000", 2020-08-07T15:07:51.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.13080476599999999 2020-08-07T15:07:51.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.581159+0000", 2020-08-07T15:07:51.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.319e-06 2020-08-07T15:07:51.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:51.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.581161+0000", 2020-08-07T15:07:51.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4919000000000005e-05 2020-08-07T15:07:51.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:51.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.581246+0000", 2020-08-07T15:07:51.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.076846029999999996 2020-08-07T15:07:51.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.658092+0000", 2020-08-07T15:07:51.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017504 2020-08-07T15:07:51.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.659842+0000", 2020-08-07T15:07:51.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9500000000000003e-07 2020-08-07T15:07:51.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:51.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.659843+0000", 2020-08-07T15:07:51.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012737800000000001 2020-08-07T15:07:51.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:51.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.659970+0000", 2020-08-07T15:07:51.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.236870208 2020-08-07T15:07:51.355 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.356 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:51.356 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.356 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:62248 3.d e41/35 ondisk, result = 0)", 2020-08-07T15:07:51.357 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:18.430063+0000", 2020-08-07T15:07:51.357 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.808437572000003, 2020-08-07T15:07:51.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22803188599999999, 2020-08-07T15:07:51.357 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:51.358 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:51.358 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:51.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:51.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.430063+0000", 2020-08-07T15:07:51.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:51.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:51.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.430063+0000", 2020-08-07T15:07:51.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7802000000000002e-05 2020-08-07T15:07:51.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:51.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.430101+0000", 2020-08-07T15:07:51.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2480000000000001e-06 2020-08-07T15:07:51.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:51.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.430107+0000", 2020-08-07T15:07:51.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.226864811 2020-08-07T15:07:51.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:51.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.656972+0000", 2020-08-07T15:07:51.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6069999999999999e-06 2020-08-07T15:07:51.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:51.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.656975+0000", 2020-08-07T15:07:51.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001086587 2020-08-07T15:07:51.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:51.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.658062+0000", 2020-08-07T15:07:51.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7053000000000001e-05 2020-08-07T15:07:51.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:51.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.658079+0000", 2020-08-07T15:07:51.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5778e-05 2020-08-07T15:07:51.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:51.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.658095+0000", 2020-08-07T15:07:51.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22803188599999999 2020-08-07T15:07:51.395 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.395 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:51.395 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.396 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:62248 3.d e41/35 ondisk, result = 0)", 2020-08-07T15:07:51.396 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:18.430203+0000", 2020-08-07T15:07:51.396 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.808296966999997, 2020-08-07T15:07:51.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.229772009, 2020-08-07T15:07:51.396 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:51.397 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:51.397 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:51.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:51.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.430203+0000", 2020-08-07T15:07:51.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:51.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:51.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.430203+0000", 2020-08-07T15:07:51.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9278e-05 2020-08-07T15:07:51.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:51.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.430233+0000", 2020-08-07T15:07:51.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1120000000000004e-06 2020-08-07T15:07:51.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:51.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.430238+0000", 2020-08-07T15:07:51.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22955323399999999 2020-08-07T15:07:51.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:51.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.659791+0000", 2020-08-07T15:07:51.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.891e-06 2020-08-07T15:07:51.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:51.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.659797+0000", 2020-08-07T15:07:51.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3538000000000003e-05 2020-08-07T15:07:51.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:51.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.659830+0000", 2020-08-07T15:07:51.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3770000000000001e-06 2020-08-07T15:07:51.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:51.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.659837+0000", 2020-08-07T15:07:51.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000138579 2020-08-07T15:07:51.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:51.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:18.659975+0000", 2020-08-07T15:07:51.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.229772009 2020-08-07T15:07:51.407 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.407 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:51.407 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:64544 3.5 3:a49c5714:::benchmark_data_smithi074_35447_object8067:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e41)", 2020-08-07T15:07:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:20.050676+0000", 2020-08-07T15:07:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.187823937000001, 2020-08-07T15:07:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30069369800000001, 2020-08-07T15:07:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 64544 2020-08-07T15:07:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.050676+0000", 2020-08-07T15:07:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.050676+0000", 2020-08-07T15:07:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7633000000000003e-05 2020-08-07T15:07:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.050724+0000", 2020-08-07T15:07:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.967e-06 2020-08-07T15:07:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.050729+0000", 2020-08-07T15:07:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1659999999999997e-06 2020-08-07T15:07:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.050736+0000", 2020-08-07T15:07:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6909999999999999e-06 2020-08-07T15:07:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.050740+0000", 2020-08-07T15:07:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052275669999999998 2020-08-07T15:07:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.055967+0000", 2020-08-07T15:07:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5745000000000002e-05 2020-08-07T15:07:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.056023+0000", 2020-08-07T15:07:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012673099999999999 2020-08-07T15:07:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:07:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.056150+0000", 2020-08-07T15:07:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016914740000000001 2020-08-07T15:07:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.057841+0000", 2020-08-07T15:07:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7100000000000001e-07 2020-08-07T15:07:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.057842+0000", 2020-08-07T15:07:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3665999999999998e-05 2020-08-07T15:07:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.057886+0000", 2020-08-07T15:07:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018919662 2020-08-07T15:07:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.076805+0000", 2020-08-07T15:07:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27452132699999998 2020-08-07T15:07:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.351327+0000", 2020-08-07T15:07:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0219999999999999e-06 2020-08-07T15:07:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.351328+0000", 2020-08-07T15:07:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2276e-05 2020-08-07T15:07:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.351370+0000", 2020-08-07T15:07:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30069369800000001 2020-08-07T15:07:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86776 3.d 3:bfcf83b1:::benchmark_data_smithi074_35447_object10846:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:07:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.385927+0000", 2020-08-07T15:07:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.852573151000001, 2020-08-07T15:07:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25591075200000002, 2020-08-07T15:07:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86776 2020-08-07T15:07:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.385927+0000", 2020-08-07T15:07:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.385927+0000", 2020-08-07T15:07:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1490999999999999e-05 2020-08-07T15:07:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.385979+0000", 2020-08-07T15:07:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2759999999999997e-06 2020-08-07T15:07:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.385985+0000", 2020-08-07T15:07:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1790999999999999e-05 2020-08-07T15:07:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.385997+0000", 2020-08-07T15:07:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7640000000000002e-06 2020-08-07T15:07:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.386002+0000", 2020-08-07T15:07:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002866244 2020-08-07T15:07:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.388869+0000", 2020-08-07T15:07:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4080999999999999e-05 2020-08-07T15:07:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.388933+0000", 2020-08-07T15:07:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000224515 2020-08-07T15:07:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,4,6,7", 2020-08-07T15:07:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.389157+0000", 2020-08-07T15:07:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028461960000000001 2020-08-07T15:07:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.392003+0000", 2020-08-07T15:07:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012122820000000001 2020-08-07T15:07:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.393216+0000", 2020-08-07T15:07:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0072762800000000004 2020-08-07T15:07:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.400492+0000", 2020-08-07T15:07:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4099999999999998e-07 2020-08-07T15:07:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.400493+0000", 2020-08-07T15:07:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0433999999999997e-05 2020-08-07T15:07:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.400563+0000", 2020-08-07T15:07:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21751274800000001 2020-08-07T15:07:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.618076+0000", 2020-08-07T15:07:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023555118 2020-08-07T15:07:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.641631+0000", 2020-08-07T15:07:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7400000000000002e-07 2020-08-07T15:07:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.641632+0000", 2020-08-07T15:07:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020601699999999999 2020-08-07T15:07:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.641838+0000", 2020-08-07T15:07:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25591075200000002 2020-08-07T15:07:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:86776 3.d e52/35 ondisk, result = 0)", 2020-08-07T15:07:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.393108+0000", 2020-08-07T15:07:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.845392311000001, 2020-08-07T15:07:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.224970906, 2020-08-07T15:07:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.393108+0000", 2020-08-07T15:07:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.393108+0000", 2020-08-07T15:07:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1553000000000001e-05 2020-08-07T15:07:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.393140+0000", 2020-08-07T15:07:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8799999999999999e-06 2020-08-07T15:07:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.393144+0000", 2020-08-07T15:07:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22476732699999999 2020-08-07T15:07:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.617912+0000", 2020-08-07T15:07:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5219999999999998e-06 2020-08-07T15:07:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.617917+0000", 2020-08-07T15:07:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013144 2020-08-07T15:07:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.618049+0000", 2020-08-07T15:07:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2763e-05 2020-08-07T15:07:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.618061+0000", 2020-08-07T15:07:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7421000000000001e-05 2020-08-07T15:07:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.618079+0000", 2020-08-07T15:07:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.224970906 2020-08-07T15:07:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86928 3.d 3:baaab9f3:::benchmark_data_smithi074_35447_object10865:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:07:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.680599+0000", 2020-08-07T15:07:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.557901342000001, 2020-08-07T15:07:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.244713969, 2020-08-07T15:07:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86928 2020-08-07T15:07:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.680599+0000", 2020-08-07T15:07:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.680599+0000", 2020-08-07T15:07:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3895e-05 2020-08-07T15:07:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.680633+0000", 2020-08-07T15:07:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.178e-06 2020-08-07T15:07:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.680639+0000", 2020-08-07T15:07:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7785e-05 2020-08-07T15:07:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.680657+0000", 2020-08-07T15:07:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6330000000000001e-06 2020-08-07T15:07:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.680662+0000", 2020-08-07T15:07:51.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005569755 2020-08-07T15:07:51.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:51.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.686232+0000", 2020-08-07T15:07:51.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012559100000000001 2020-08-07T15:07:51.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:51.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.686358+0000", 2020-08-07T15:07:51.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00047514600000000002 2020-08-07T15:07:51.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,4,6,7", 2020-08-07T15:07:51.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.686833+0000", 2020-08-07T15:07:51.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026029410000000001 2020-08-07T15:07:51.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:51.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.689436+0000", 2020-08-07T15:07:51.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013454630000000001 2020-08-07T15:07:51.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.507 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 75 16 108510 108494 11.3005 5.32812 0.235366 0.0109675 2020-08-07T15:07:51.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.690781+0000", 2020-08-07T15:07:51.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5797000000000001e-05 2020-08-07T15:07:51.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.690837+0000", 2020-08-07T15:07:51.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8000000000000001e-07 2020-08-07T15:07:51.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:51.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.690838+0000", 2020-08-07T15:07:51.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4367999999999999e-05 2020-08-07T15:07:51.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:51.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.690892+0000", 2020-08-07T15:07:51.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00158982 2020-08-07T15:07:51.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.692482+0000", 2020-08-07T15:07:51.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.232608799 2020-08-07T15:07:51.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.925090+0000", 2020-08-07T15:07:51.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.454e-06 2020-08-07T15:07:51.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:51.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.925092+0000", 2020-08-07T15:07:51.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022096400000000001 2020-08-07T15:07:51.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:51.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.925313+0000", 2020-08-07T15:07:51.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.244713969 2020-08-07T15:07:51.517 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.517 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:51.517 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.518 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:96390 3.d 3:be0da2f7:::benchmark_data_smithi074_35447_object12048:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e56)", 2020-08-07T15:07:51.518 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:41.003500+0000", 2020-08-07T15:07:51.518 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.234999927, 2020-08-07T15:07:51.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.216617751, 2020-08-07T15:07:51.518 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:51.519 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:51.519 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:51.519 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:51.519 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:51.520 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 96390 2020-08-07T15:07:51.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.520 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:51.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:51.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.003500+0000", 2020-08-07T15:07:51.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:51.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:51.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.003500+0000", 2020-08-07T15:07:51.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1910000000000003e-05 2020-08-07T15:07:51.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:51.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.003552+0000", 2020-08-07T15:07:51.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3399999999999997e-06 2020-08-07T15:07:51.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:51.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.003558+0000", 2020-08-07T15:07:51.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9720000000000004e-06 2020-08-07T15:07:51.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:51.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.003568+0000", 2020-08-07T15:07:51.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7369999999999997e-06 2020-08-07T15:07:51.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:51.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.003572+0000", 2020-08-07T15:07:51.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001611807 2020-08-07T15:07:51.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.005184+0000", 2020-08-07T15:07:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012318600000000001 2020-08-07T15:07:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.005307+0000", 2020-08-07T15:07:51.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023712900000000001 2020-08-07T15:07:51.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,4,6,7", 2020-08-07T15:07:51.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.005544+0000", 2020-08-07T15:07:51.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043724269999999999 2020-08-07T15:07:51.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:51.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.009917+0000", 2020-08-07T15:07:51.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027935759999999999 2020-08-07T15:07:51.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.012710+0000", 2020-08-07T15:07:51.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2942000000000006e-05 2020-08-07T15:07:51.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.012803+0000", 2020-08-07T15:07:51.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4600000000000004e-07 2020-08-07T15:07:51.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:51.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.012804+0000", 2020-08-07T15:07:51.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1198000000000004e-05 2020-08-07T15:07:51.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:51.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.012885+0000", 2020-08-07T15:07:51.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010288859999999999 2020-08-07T15:07:51.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.013914+0000", 2020-08-07T15:07:51.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20596002299999999 2020-08-07T15:07:51.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.219874+0000", 2020-08-07T15:07:51.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.353e-06 2020-08-07T15:07:51.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:51.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.219876+0000", 2020-08-07T15:07:51.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024251899999999999 2020-08-07T15:07:51.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:51.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.220118+0000", 2020-08-07T15:07:51.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.216617751 2020-08-07T15:07:51.540 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.540 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:51.540 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.541 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:96391 3.d 3:be0da2f7:::benchmark_data_smithi074_35447_object12048:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e56)", 2020-08-07T15:07:51.557 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:41.003881+0000", 2020-08-07T15:07:51.558 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.234619015, 2020-08-07T15:07:51.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21961502199999999, 2020-08-07T15:07:51.558 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:51.558 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:51.558 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:51.559 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:51.559 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:51.563 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 96391 2020-08-07T15:07:51.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.563 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:51.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:51.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.003881+0000", 2020-08-07T15:07:51.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:51.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:51.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.003881+0000", 2020-08-07T15:07:51.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0179999999999999e-05 2020-08-07T15:07:51.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:51.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.003911+0000", 2020-08-07T15:07:51.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9280000000000003e-06 2020-08-07T15:07:51.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:51.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.003915+0000", 2020-08-07T15:07:51.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4830000000000002e-06 2020-08-07T15:07:51.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:51.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.003923+0000", 2020-08-07T15:07:51.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0060000000000003e-06 2020-08-07T15:07:51.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:51.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.003927+0000", 2020-08-07T15:07:51.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002298409 2020-08-07T15:07:51.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:51.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.006225+0000", 2020-08-07T15:07:51.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016317099999999999 2020-08-07T15:07:51.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:51.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.006388+0000", 2020-08-07T15:07:51.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000229294 2020-08-07T15:07:51.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,4,6,7", 2020-08-07T15:07:51.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.006618+0000", 2020-08-07T15:07:51.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033077739999999999 2020-08-07T15:07:51.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:51.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.009926+0000", 2020-08-07T15:07:51.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028341960000000002 2020-08-07T15:07:51.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.012760+0000", 2020-08-07T15:07:51.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018228 2020-08-07T15:07:51.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.012942+0000", 2020-08-07T15:07:51.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.89e-07 2020-08-07T15:07:51.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:51.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.012942+0000", 2020-08-07T15:07:51.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7465999999999994e-05 2020-08-07T15:07:51.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:51.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.013010+0000", 2020-08-07T15:07:51.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00095542800000000003 2020-08-07T15:07:51.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.013965+0000", 2020-08-07T15:07:51.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20928491099999999 2020-08-07T15:07:51.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.223250+0000", 2020-08-07T15:07:51.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7199999999999997e-07 2020-08-07T15:07:51.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:51.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.223251+0000", 2020-08-07T15:07:51.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000245135 2020-08-07T15:07:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.223496+0000", 2020-08-07T15:07:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21961502199999999 2020-08-07T15:07:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:96392 3.d 3:be0da2f7:::benchmark_data_smithi074_35447_object12048:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e56)", 2020-08-07T15:07:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:41.003966+0000", 2020-08-07T15:07:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.23453417, 2020-08-07T15:07:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.222971327, 2020-08-07T15:07:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 96392 2020-08-07T15:07:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.003966+0000", 2020-08-07T15:07:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.003966+0000", 2020-08-07T15:07:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0985000000000002e-05 2020-08-07T15:07:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.003997+0000", 2020-08-07T15:07:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.117e-06 2020-08-07T15:07:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.004000+0000", 2020-08-07T15:07:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0690000000000002e-06 2020-08-07T15:07:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.004008+0000", 2020-08-07T15:07:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.189e-06 2020-08-07T15:07:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.004012+0000", 2020-08-07T15:07:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033159249999999999 2020-08-07T15:07:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.007328+0000", 2020-08-07T15:07:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013308699999999999 2020-08-07T15:07:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.007461+0000", 2020-08-07T15:07:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032424899999999998 2020-08-07T15:07:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,4,6,7", 2020-08-07T15:07:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.007786+0000", 2020-08-07T15:07:51.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00214445 2020-08-07T15:07:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.009930+0000", 2020-08-07T15:07:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031342430000000001 2020-08-07T15:07:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.013064+0000", 2020-08-07T15:07:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016188979999999999 2020-08-07T15:07:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.014683+0000", 2020-08-07T15:07:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.23e-07 2020-08-07T15:07:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.014684+0000", 2020-08-07T15:07:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7357999999999999e-05 2020-08-07T15:07:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.014771+0000", 2020-08-07T15:07:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00078108400000000003 2020-08-07T15:07:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.015552+0000", 2020-08-07T15:07:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.211224474 2020-08-07T15:07:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.226777+0000", 2020-08-07T15:07:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7400000000000002e-07 2020-08-07T15:07:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.226778+0000", 2020-08-07T15:07:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015970200000000001 2020-08-07T15:07:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.226937+0000", 2020-08-07T15:07:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.222971327 2020-08-07T15:07:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:97576 3.d 3:b9e86280:::benchmark_data_smithi074_35447_object12196:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e57)", 2020-08-07T15:07:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:41.732971+0000", 2020-08-07T15:07:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.5055292779999991, 2020-08-07T15:07:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28857946099999998, 2020-08-07T15:07:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 97576 2020-08-07T15:07:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.732971+0000", 2020-08-07T15:07:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.732971+0000", 2020-08-07T15:07:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1149000000000001e-05 2020-08-07T15:07:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.733022+0000", 2020-08-07T15:07:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5110000000000003e-06 2020-08-07T15:07:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.733031+0000", 2020-08-07T15:07:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3125000000000001e-05 2020-08-07T15:07:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.733044+0000", 2020-08-07T15:07:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9290000000000003e-06 2020-08-07T15:07:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.733050+0000", 2020-08-07T15:07:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059384989999999999 2020-08-07T15:07:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.738988+0000", 2020-08-07T15:07:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012748499999999999 2020-08-07T15:07:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.739116+0000", 2020-08-07T15:07:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000241604 2020-08-07T15:07:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,4,6,7", 2020-08-07T15:07:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.739357+0000", 2020-08-07T15:07:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0060933430000000002 2020-08-07T15:07:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.745451+0000", 2020-08-07T15:07:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001449812 2020-08-07T15:07:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.746900+0000", 2020-08-07T15:07:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001219773 2020-08-07T15:07:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.748120+0000", 2020-08-07T15:07:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0599999999999999e-07 2020-08-07T15:07:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.748121+0000", 2020-08-07T15:07:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5364000000000006e-05 2020-08-07T15:07:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.748197+0000", 2020-08-07T15:07:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.068992297999999994 2020-08-07T15:07:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.817189+0000", 2020-08-07T15:07:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20413072299999999 2020-08-07T15:07:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.021320+0000", 2020-08-07T15:07:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8599999999999997e-07 2020-08-07T15:07:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.021320+0000", 2020-08-07T15:07:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023015399999999999 2020-08-07T15:07:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.021550+0000", 2020-08-07T15:07:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28857946099999998 2020-08-07T15:07:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "MOSDPGPushReply(3.d 58/35 [PushReplyOp(3:baa598e9:::benchmark_data_smithi074_35447_object8718:head)])", 2020-08-07T15:07:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:46.894916+0000", 2020-08-07T15:07:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.3435841880000003, 2020-08-07T15:07:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21159749999999999, 2020-08-07T15:07:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:07:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.894916+0000", 2020-08-07T15:07:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.894916+0000", 2020-08-07T15:07:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5832999999999999e-05 2020-08-07T15:07:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.894932+0000", 2020-08-07T15:07:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.194e-06 2020-08-07T15:07:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.894936+0000", 2020-08-07T15:07:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.211149067 2020-08-07T15:07:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.106085+0000", 2020-08-07T15:07:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3260000000000003e-06 2020-08-07T15:07:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.106091+0000", 2020-08-07T15:07:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2143e-05 2020-08-07T15:07:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.106123+0000", 2020-08-07T15:07:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000390937 2020-08-07T15:07:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.106514+0000", 2020-08-07T15:07:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21159749999999999 2020-08-07T15:07:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "MOSDPGPushReply(3.d 59/35 [PushReplyOp(3:bb99ee5d:::benchmark_data_smithi074_35447_object6726:head)])", 2020-08-07T15:07:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.498949+0000", 2020-08-07T15:07:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.7395514040000002, 2020-08-07T15:07:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22920950600000001, 2020-08-07T15:07:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:07:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.498949+0000", 2020-08-07T15:07:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.498949+0000", 2020-08-07T15:07:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9456999999999999e-05 2020-08-07T15:07:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.498978+0000", 2020-08-07T15:07:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4119999999999998e-06 2020-08-07T15:07:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.498985+0000", 2020-08-07T15:07:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22885186499999999 2020-08-07T15:07:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.727837+0000", 2020-08-07T15:07:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5619999999999998e-06 2020-08-07T15:07:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.727844+0000", 2020-08-07T15:07:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6341999999999998e-05 2020-08-07T15:07:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.727891+0000", 2020-08-07T15:07:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026786799999999999 2020-08-07T15:07:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.728158+0000", 2020-08-07T15:07:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22920950600000001 2020-08-07T15:07:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106320 3.d 3:b8d3b03d:::benchmark_data_smithi074_35447_object13289:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:07:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.834704+0000", 2020-08-07T15:07:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.403796635, 2020-08-07T15:07:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22145492899999999, 2020-08-07T15:07:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106320 2020-08-07T15:07:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.834704+0000", 2020-08-07T15:07:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.834704+0000", 2020-08-07T15:07:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8297999999999999e-05 2020-08-07T15:07:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.834732+0000", 2020-08-07T15:07:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1350000000000002e-06 2020-08-07T15:07:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.834736+0000", 2020-08-07T15:07:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6090000000000007e-06 2020-08-07T15:07:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.834746+0000", 2020-08-07T15:07:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4500000000000003e-06 2020-08-07T15:07:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.834751+0000", 2020-08-07T15:07:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4035999999999997e-05 2020-08-07T15:07:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.834785+0000", 2020-08-07T15:07:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1001999999999995e-05 2020-08-07T15:07:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.834866+0000", 2020-08-07T15:07:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000154971 2020-08-07T15:07:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,4,6,7", 2020-08-07T15:07:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.835021+0000", 2020-08-07T15:07:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00080727700000000002 2020-08-07T15:07:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.835828+0000", 2020-08-07T15:07:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00069431600000000005 2020-08-07T15:07:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.836523+0000", 2020-08-07T15:07:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002626162 2020-08-07T15:07:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.839149+0000", 2020-08-07T15:07:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.99e-07 2020-08-07T15:07:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.839150+0000", 2020-08-07T15:07:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9016e-05 2020-08-07T15:07:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.839199+0000", 2020-08-07T15:07:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000425256 2020-08-07T15:07:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.839624+0000", 2020-08-07T15:07:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21638104599999999 2020-08-07T15:07:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.056005+0000", 2020-08-07T15:07:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.35e-06 2020-08-07T15:07:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.056006+0000", 2020-08-07T15:07:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000152306 2020-08-07T15:07:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.056159+0000", 2020-08-07T15:07:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22145492899999999 2020-08-07T15:07:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106320 3.d e59/35 ondisk, result = 0)", 2020-08-07T15:07:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.836304+0000", 2020-08-07T15:07:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.4021966159999999, 2020-08-07T15:07:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21986075299999999, 2020-08-07T15:07:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.836304+0000", 2020-08-07T15:07:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.836304+0000", 2020-08-07T15:07:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.307e-05 2020-08-07T15:07:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.836337+0000", 2020-08-07T15:07:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8799999999999996e-06 2020-08-07T15:07:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.836343+0000", 2020-08-07T15:07:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.219551887 2020-08-07T15:07:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.055895+0000", 2020-08-07T15:07:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.95e-06 2020-08-07T15:07:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.055899+0000", 2020-08-07T15:07:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8163000000000003e-05 2020-08-07T15:07:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.055988+0000", 2020-08-07T15:07:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7790000000000007e-06 2020-08-07T15:07:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.055997+0000", 2020-08-07T15:07:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016702399999999999 2020-08-07T15:07:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.056164+0000", 2020-08-07T15:07:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21986075299999999 2020-08-07T15:07:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "MOSDPGPushReply(3.d 59/35 [PushReplyOp(3:bbd08cec:::benchmark_data_smithi074_35447_object8039:head)])", 2020-08-07T15:07:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.839005+0000", 2020-08-07T15:07:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.3994949970000001, 2020-08-07T15:07:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42127482700000002, 2020-08-07T15:07:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:07:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.839005+0000", 2020-08-07T15:07:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.839005+0000", 2020-08-07T15:07:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2351000000000002e-05 2020-08-07T15:07:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.839038+0000", 2020-08-07T15:07:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3849999999999996e-06 2020-08-07T15:07:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.839045+0000", 2020-08-07T15:07:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42104101399999999 2020-08-07T15:07:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.260086+0000", 2020-08-07T15:07:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.605e-06 2020-08-07T15:07:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.260091+0000", 2020-08-07T15:07:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1946000000000006e-05 2020-08-07T15:07:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.260153+0000", 2020-08-07T15:07:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012752600000000001 2020-08-07T15:07:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.260280+0000", 2020-08-07T15:07:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42127482700000002 2020-08-07T15:07:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "pg_backfill(progress 3.1 e 62/62 lb 3:82e41841:::benchmark_data_smithi074_35447_object8611:head)", 2020-08-07T15:07:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.555833+0000", 2020-08-07T15:07:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.68266757300000003, 2020-08-07T15:07:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27281116999999999, 2020-08-07T15:07:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.555833+0000", 2020-08-07T15:07:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.555833+0000", 2020-08-07T15:07:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0040999999999999e-05 2020-08-07T15:07:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.555863+0000", 2020-08-07T15:07:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4109999999999996e-06 2020-08-07T15:07:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.555871+0000", 2020-08-07T15:07:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27259343400000002 2020-08-07T15:07:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.828465+0000", 2020-08-07T15:07:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7959999999999996e-06 2020-08-07T15:07:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.828472+0000", 2020-08-07T15:07:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5361e-05 2020-08-07T15:07:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.828538+0000", 2020-08-07T15:07:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5410999999999999e-05 2020-08-07T15:07:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.828553+0000", 2020-08-07T15:07:51.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0716000000000004e-05 2020-08-07T15:07:51.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:51.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.828644+0000", 2020-08-07T15:07:51.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27281116999999999 2020-08-07T15:07:51.777 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.777 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:51.777 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.777 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:51.778 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:51.778 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:51.778 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:07:51.779 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:51.505+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:51.781 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:51.780+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:51.888 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:51.885+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:51.959 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:51.960 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:51.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.960 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:109473 3.a 3:5c3a9606:::benchmark_data_smithi074_35447_object13684:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e63)", 2020-08-07T15:07:51.960 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:51.948394+0000", 2020-08-07T15:07:51.960 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0052495670000000001, 2020-08-07T15:07:51.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052769289999999996, 2020-08-07T15:07:51.961 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:51.961 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:51.961 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:51.961 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:51.972 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:51.972 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 109473 2020-08-07T15:07:51.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.977 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:51.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:51.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948394+0000", 2020-08-07T15:07:51.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:51.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:51.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948394+0000", 2020-08-07T15:07:51.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6306000000000001e-05 2020-08-07T15:07:51.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:51.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948430+0000", 2020-08-07T15:07:51.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6899999999999998e-06 2020-08-07T15:07:51.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:51.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948434+0000", 2020-08-07T15:07:51.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2717e-05 2020-08-07T15:07:51.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:51.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948447+0000", 2020-08-07T15:07:51.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0760000000000002e-06 2020-08-07T15:07:51.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:51.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948452+0000", 2020-08-07T15:07:51.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5911e-05 2020-08-07T15:07:51.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:51.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948478+0000", 2020-08-07T15:07:51.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6052999999999997e-05 2020-08-07T15:07:51.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:51.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948574+0000", 2020-08-07T15:07:51.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110846 2020-08-07T15:07:51.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:51.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948685+0000", 2020-08-07T15:07:51.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019912749999999998 2020-08-07T15:07:51.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:51.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:51.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:51.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.950676+0000", 2020-08-07T15:07:52.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4600000000000004e-07 2020-08-07T15:07:52.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:52.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.950677+0000", 2020-08-07T15:07:52.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9113999999999999e-05 2020-08-07T15:07:52.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:52.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.950716+0000", 2020-08-07T15:07:52.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6647999999999997e-05 2020-08-07T15:07:52.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:52.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.950772+0000", 2020-08-07T15:07:52.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023783820000000001 2020-08-07T15:07:52.003 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.003 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.003 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.004 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:109474 3.a 3:5c3a9606:::benchmark_data_smithi074_35447_object13684:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e63)", 2020-08-07T15:07:52.004 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:51.948480+0000", 2020-08-07T15:07:52.004 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0051636030000000001, 2020-08-07T15:07:52.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052631199999999996, 2020-08-07T15:07:52.004 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.005 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:07:52.006 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:52.007 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:52.007 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:52.007 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 109474 2020-08-07T15:07:52.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.008 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948480+0000", 2020-08-07T15:07:52.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948480+0000", 2020-08-07T15:07:52.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8878999999999999e-05 2020-08-07T15:07:52.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948509+0000", 2020-08-07T15:07:52.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1069999999999998e-06 2020-08-07T15:07:52.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948513+0000", 2020-08-07T15:07:52.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0039999999999996e-06 2020-08-07T15:07:52.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948519+0000", 2020-08-07T15:07:52.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6690000000000002e-06 2020-08-07T15:07:52.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948522+0000", 2020-08-07T15:07:52.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00057323599999999997 2020-08-07T15:07:52.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:52.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.949095+0000", 2020-08-07T15:07:52.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2277e-05 2020-08-07T15:07:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.949157+0000", 2020-08-07T15:07:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112317 2020-08-07T15:07:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:52.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.949269+0000", 2020-08-07T15:07:52.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001530331 2020-08-07T15:07:52.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:52.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.950800+0000", 2020-08-07T15:07:52.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002754683 2020-08-07T15:07:52.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:52.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953554+0000", 2020-08-07T15:07:52.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050745030000000002 2020-08-07T15:07:52.020 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.020 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.021 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.021 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:109475 3.a 3:5c3a9606:::benchmark_data_smithi074_35447_object13684:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e63)", 2020-08-07T15:07:52.021 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:51.948545+0000", 2020-08-07T15:07:52.022 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0050984649999999999, 2020-08-07T15:07:52.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053151300000000004, 2020-08-07T15:07:52.022 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.022 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:07:52.022 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:52.023 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:52.023 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:52.023 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 109475 2020-08-07T15:07:52.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.023 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948545+0000", 2020-08-07T15:07:52.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948545+0000", 2020-08-07T15:07:52.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4799999999999999e-05 2020-08-07T15:07:52.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948580+0000", 2020-08-07T15:07:52.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0960000000000001e-06 2020-08-07T15:07:52.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948583+0000", 2020-08-07T15:07:52.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9020000000000004e-06 2020-08-07T15:07:52.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948589+0000", 2020-08-07T15:07:52.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6259999999999999e-06 2020-08-07T15:07:52.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948592+0000", 2020-08-07T15:07:52.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00098632400000000001 2020-08-07T15:07:52.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:52.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.949578+0000", 2020-08-07T15:07:52.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4246000000000005e-05 2020-08-07T15:07:52.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:52.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.949672+0000", 2020-08-07T15:07:52.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019091799999999999 2020-08-07T15:07:52.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:52.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.949863+0000", 2020-08-07T15:07:52.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036800420000000001 2020-08-07T15:07:52.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:52.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953543+0000", 2020-08-07T15:07:52.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0265e-05 2020-08-07T15:07:52.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:52.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953563+0000", 2020-08-07T15:07:52.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005018219 2020-08-07T15:07:52.052 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.052 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.052 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.053 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:109476 3.a 3:5c3a9606:::benchmark_data_smithi074_35447_object13684:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e63)", 2020-08-07T15:07:52.053 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:51.948613+0000", 2020-08-07T15:07:52.053 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0050310650000000004, 2020-08-07T15:07:52.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053640069999999996, 2020-08-07T15:07:52.053 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.054 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:52.054 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:52.054 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:52.054 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:52.054 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 109476 2020-08-07T15:07:52.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.055 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948613+0000", 2020-08-07T15:07:52.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948613+0000", 2020-08-07T15:07:52.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6149e-05 2020-08-07T15:07:52.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948639+0000", 2020-08-07T15:07:52.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.926e-06 2020-08-07T15:07:52.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948642+0000", 2020-08-07T15:07:52.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5969999999999999e-06 2020-08-07T15:07:52.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948647+0000", 2020-08-07T15:07:52.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4940000000000002e-06 2020-08-07T15:07:52.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948650+0000", 2020-08-07T15:07:52.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015763940000000001 2020-08-07T15:07:52.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:52.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.950226+0000", 2020-08-07T15:07:52.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4157999999999999e-05 2020-08-07T15:07:52.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:52.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.950280+0000", 2020-08-07T15:07:52.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101228 2020-08-07T15:07:52.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:52.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.950381+0000", 2020-08-07T15:07:52.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031903579999999999 2020-08-07T15:07:52.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:52.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953572+0000", 2020-08-07T15:07:52.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2020-08-07T15:07:52.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:52.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953572+0000", 2020-08-07T15:07:52.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021034100000000001 2020-08-07T15:07:52.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:52.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953783+0000", 2020-08-07T15:07:52.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051700909999999999 2020-08-07T15:07:52.069 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.069 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.069 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.069 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:109477 3.a 3:5c3a9606:::benchmark_data_smithi074_35447_object13684:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e63)", 2020-08-07T15:07:52.070 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:51.948667+0000", 2020-08-07T15:07:52.070 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0049765670000000003, 2020-08-07T15:07:52.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054402340000000004, 2020-08-07T15:07:52.070 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.070 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:52.071 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:52.071 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:52.071 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:52.071 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 109477 2020-08-07T15:07:52.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.072 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948667+0000", 2020-08-07T15:07:52.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948667+0000", 2020-08-07T15:07:52.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5559e-05 2020-08-07T15:07:52.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948693+0000", 2020-08-07T15:07:52.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.571e-06 2020-08-07T15:07:52.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948695+0000", 2020-08-07T15:07:52.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.237e-06 2020-08-07T15:07:52.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948700+0000", 2020-08-07T15:07:52.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4269999999999998e-06 2020-08-07T15:07:52.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948703+0000", 2020-08-07T15:07:52.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021332360000000002 2020-08-07T15:07:52.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:52.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.950836+0000", 2020-08-07T15:07:52.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5624000000000006e-05 2020-08-07T15:07:52.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:52.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.950922+0000", 2020-08-07T15:07:52.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9999000000000003e-05 2020-08-07T15:07:52.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:52.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.951022+0000", 2020-08-07T15:07:52.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027746329999999999 2020-08-07T15:07:52.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:52.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953796+0000", 2020-08-07T15:07:52.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2099999999999996e-07 2020-08-07T15:07:52.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:52.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953797+0000", 2020-08-07T15:07:52.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7294999999999999e-05 2020-08-07T15:07:52.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:52.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953864+0000", 2020-08-07T15:07:52.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051972019999999997 2020-08-07T15:07:52.084 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.084 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.084 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.085 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:109478 3.a 3:5c3a9606:::benchmark_data_smithi074_35447_object13684:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e63)", 2020-08-07T15:07:52.085 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:51.948724+0000", 2020-08-07T15:07:52.085 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0049198030000000004, 2020-08-07T15:07:52.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055030890000000001, 2020-08-07T15:07:52.086 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.086 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:52.087 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:52.087 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:52.087 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:52.087 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 109478 2020-08-07T15:07:52.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.088 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948724+0000", 2020-08-07T15:07:52.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948724+0000", 2020-08-07T15:07:52.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1307000000000002e-05 2020-08-07T15:07:52.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948775+0000", 2020-08-07T15:07:52.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3170000000000003e-06 2020-08-07T15:07:52.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948779+0000", 2020-08-07T15:07:52.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3579999999999999e-06 2020-08-07T15:07:52.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948786+0000", 2020-08-07T15:07:52.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7809999999999999e-06 2020-08-07T15:07:52.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948790+0000", 2020-08-07T15:07:52.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002592291 2020-08-07T15:07:52.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:52.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.951382+0000", 2020-08-07T15:07:52.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1709000000000001e-05 2020-08-07T15:07:52.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:52.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.951454+0000", 2020-08-07T15:07:52.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024384299999999999 2020-08-07T15:07:52.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:52.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.951697+0000", 2020-08-07T15:07:52.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021762140000000001 2020-08-07T15:07:52.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:52.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953874+0000", 2020-08-07T15:07:52.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3499999999999996e-07 2020-08-07T15:07:52.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:52.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953874+0000", 2020-08-07T15:07:52.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000118119 2020-08-07T15:07:52.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:52.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953992+0000", 2020-08-07T15:07:52.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052685739999999998 2020-08-07T15:07:52.106 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.106 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.106 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.107 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:109479 3.a 3.60695c3a (undecoded) ondisk+write+known_if_redirected e63)", 2020-08-07T15:07:52.109 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:51.948820+0000", 2020-08-07T15:07:52.113 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0048239010000000002, 2020-08-07T15:07:52.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055216859999999996, 2020-08-07T15:07:52.114 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.114 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:07:52.114 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:52.115 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:52.115 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:52.115 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 109479 2020-08-07T15:07:52.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.115 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948820+0000", 2020-08-07T15:07:52.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948820+0000", 2020-08-07T15:07:52.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8764000000000003e-05 2020-08-07T15:07:52.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948868+0000", 2020-08-07T15:07:52.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1440000000000002e-06 2020-08-07T15:07:52.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948874+0000", 2020-08-07T15:07:52.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0827e-05 2020-08-07T15:07:52.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948884+0000", 2020-08-07T15:07:52.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3189999999999996e-06 2020-08-07T15:07:52.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948889+0000", 2020-08-07T15:07:52.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9054000000000007e-05 2020-08-07T15:07:52.122 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.122 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.122 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.123 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:109480 3.a 3.60695c3a (undecoded) ondisk+write+known_if_redirected e63)", 2020-08-07T15:07:52.123 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:51.948924+0000", 2020-08-07T15:07:52.123 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.004719333, 2020-08-07T15:07:52.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005485055, 2020-08-07T15:07:52.124 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.124 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:07:52.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:52.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:52.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:52.124 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 109480 2020-08-07T15:07:52.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.125 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948924+0000", 2020-08-07T15:07:52.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948924+0000", 2020-08-07T15:07:52.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7948000000000002e-05 2020-08-07T15:07:52.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948972+0000", 2020-08-07T15:07:52.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2310000000000001e-06 2020-08-07T15:07:52.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948975+0000", 2020-08-07T15:07:52.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7349999999999999e-06 2020-08-07T15:07:52.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948982+0000", 2020-08-07T15:07:52.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3340000000000002e-06 2020-08-07T15:07:52.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.948985+0000", 2020-08-07T15:07:52.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1247999999999994e-05 2020-08-07T15:07:52.134 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.135 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.135 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.135 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:109475 3.a e63/20 ondisk, result = 0)", 2020-08-07T15:07:52.136 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:51.953124+0000", 2020-08-07T15:07:52.136 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00051917700000000001, 2020-08-07T15:07:52.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013479690000000001, 2020-08-07T15:07:52.136 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.136 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:52.137 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953124+0000", 2020-08-07T15:07:52.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953124+0000", 2020-08-07T15:07:52.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8215000000000001e-05 2020-08-07T15:07:52.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953143+0000", 2020-08-07T15:07:52.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5600000000000004e-06 2020-08-07T15:07:52.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953147+0000", 2020-08-07T15:07:52.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151858 2020-08-07T15:07:52.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953299+0000", 2020-08-07T15:07:52.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.005e-06 2020-08-07T15:07:52.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953302+0000", 2020-08-07T15:07:52.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011486490000000001 2020-08-07T15:07:52.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:52.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.954451+0000", 2020-08-07T15:07:52.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.859e-06 2020-08-07T15:07:52.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:52.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.954461+0000", 2020-08-07T15:07:52.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013361460000000001 2020-08-07T15:07:52.149 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.150 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.150 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.150 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:109476 3.a e63/20)", 2020-08-07T15:07:52.151 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:51.953176+0000", 2020-08-07T15:07:52.151 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00046801999999999998, 2020-08-07T15:07:52.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013735710000000001, 2020-08-07T15:07:52.151 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.151 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:07:52.151 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953176+0000", 2020-08-07T15:07:52.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953176+0000", 2020-08-07T15:07:52.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8507000000000001e-05 2020-08-07T15:07:52.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953194+0000", 2020-08-07T15:07:52.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5749999999999999e-06 2020-08-07T15:07:52.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953197+0000", 2020-08-07T15:07:52.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000114885 2020-08-07T15:07:52.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953312+0000", 2020-08-07T15:07:52.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0709999999999999e-06 2020-08-07T15:07:52.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953315+0000", 2020-08-07T15:07:52.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013903799999999999 2020-08-07T15:07:52.165 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.165 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.165 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.166 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:109477 3.a e63/20)", 2020-08-07T15:07:52.166 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:51.953211+0000", 2020-08-07T15:07:52.177 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00043257300000000001, 2020-08-07T15:07:52.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001394752, 2020-08-07T15:07:52.178 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.178 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:07:52.178 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953211+0000", 2020-08-07T15:07:52.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953211+0000", 2020-08-07T15:07:52.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5234e-05 2020-08-07T15:07:52.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953226+0000", 2020-08-07T15:07:52.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6370000000000001e-06 2020-08-07T15:07:52.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953230+0000", 2020-08-07T15:07:52.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3579000000000006e-05 2020-08-07T15:07:52.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953323+0000", 2020-08-07T15:07:52.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1489999999999998e-06 2020-08-07T15:07:52.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953327+0000", 2020-08-07T15:07:52.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000115599 2020-08-07T15:07:52.207 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.207 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.208 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.208 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:109477 3.a e63/20)", 2020-08-07T15:07:52.208 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:51.953360+0000", 2020-08-07T15:07:52.209 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00028322699999999999, 2020-08-07T15:07:52.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013061520000000001, 2020-08-07T15:07:52.209 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.209 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:07:52.209 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953360+0000", 2020-08-07T15:07:52.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953360+0000", 2020-08-07T15:07:52.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9371000000000001e-05 2020-08-07T15:07:52.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953390+0000", 2020-08-07T15:07:52.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2020-08-07T15:07:52.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953395+0000", 2020-08-07T15:07:52.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3032000000000002e-05 2020-08-07T15:07:52.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953458+0000", 2020-08-07T15:07:52.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8779999999999997e-06 2020-08-07T15:07:52.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953463+0000", 2020-08-07T15:07:52.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102291 2020-08-07T15:07:52.221 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.221 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.221 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.223 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:109478 3.a e63/20)", 2020-08-07T15:07:52.223 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:51.953666+0000", 2020-08-07T15:07:52.223 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4294967295.9999776, 2020-08-07T15:07:52.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010550920000000001, 2020-08-07T15:07:52.226 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.226 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:07:52.231 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953666+0000", 2020-08-07T15:07:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953666+0000", 2020-08-07T15:07:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7129000000000002e-05 2020-08-07T15:07:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953683+0000", 2020-08-07T15:07:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.365e-06 2020-08-07T15:07:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953686+0000", 2020-08-07T15:07:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6708000000000002e-05 2020-08-07T15:07:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953723+0000", 2020-08-07T15:07:52.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0119999999999996e-06 2020-08-07T15:07:52.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.953728+0000", 2020-08-07T15:07:52.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2213999999999998e-05 2020-08-07T15:07:52.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.270 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.271 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.271 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:07:52.271 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 13 2020-08-07T15:07:52.271 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:52.273 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 76 16 109981 109965 11.303 11.4922 0.00219391 0.0110522 2020-08-07T15:07:52.273 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:52.272+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:52.289 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:07:52.367 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:07:52.368 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:07:52.368 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:07:52.369 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:07:52.375 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:52.374+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:52.480 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:52.480 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:07:52.481 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:07:52.481 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:07:52.481 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:52.481 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:52.479+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:52.496 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:07:52.592 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:52.584+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:52.683 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:52.681+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:52.729 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:52.729 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:07:52.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:07:52.730 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:52.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.730 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:17824 3.d e27/20 3:bd4e0935:::benchmark_data_smithi074_35447_object2227:head v 27'1168, mlcod=27'1163)", 2020-08-07T15:07:52.731 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:07:52.731 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.581010315, 2020-08-07T15:07:52.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25249224199999998, 2020-08-07T15:07:52.733 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.733 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:52.733 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:07:52.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:07:52.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9348000000000003e-05 2020-08-07T15:07:52.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143490+0000", 2020-08-07T15:07:52.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7800000000000003e-06 2020-08-07T15:07:52.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143496+0000", 2020-08-07T15:07:52.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25095783399999999 2020-08-07T15:07:52.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394454+0000", 2020-08-07T15:07:52.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6550000000000001e-06 2020-08-07T15:07:52.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394460+0000", 2020-08-07T15:07:52.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8814999999999999e-05 2020-08-07T15:07:52.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:52.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394509+0000", 2020-08-07T15:07:52.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3482999999999999e-05 2020-08-07T15:07:52.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:52.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394532+0000", 2020-08-07T15:07:52.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013496199999999999 2020-08-07T15:07:52.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:52.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.395882+0000", 2020-08-07T15:07:52.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0707000000000001e-05 2020-08-07T15:07:52.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:52.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.395932+0000", 2020-08-07T15:07:52.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25249224199999998 2020-08-07T15:07:52.743 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.743 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.743 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.745 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:18478 3.a 3:524412cd:::benchmark_data_smithi074_35447_object2309:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:52.745 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.592837+0000", 2020-08-07T15:07:52.746 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.131613236000007, 2020-08-07T15:07:52.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21005600699999999, 2020-08-07T15:07:52.746 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.746 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:52.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:52.747 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:52.747 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:52.747 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 18478 2020-08-07T15:07:52.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.747 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592837+0000", 2020-08-07T15:07:52.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592837+0000", 2020-08-07T15:07:52.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6718999999999998e-05 2020-08-07T15:07:52.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592884+0000", 2020-08-07T15:07:52.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2749999999999997e-06 2020-08-07T15:07:52.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592888+0000", 2020-08-07T15:07:52.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0831e-05 2020-08-07T15:07:52.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592899+0000", 2020-08-07T15:07:52.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9440000000000002e-06 2020-08-07T15:07:52.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592905+0000", 2020-08-07T15:07:52.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00088115000000000003 2020-08-07T15:07:52.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:52.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.593786+0000", 2020-08-07T15:07:52.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1789000000000002e-05 2020-08-07T15:07:52.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:52.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.593838+0000", 2020-08-07T15:07:52.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011097799999999999 2020-08-07T15:07:52.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:52.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.593949+0000", 2020-08-07T15:07:52.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00088680400000000004 2020-08-07T15:07:52.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:52.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.594836+0000", 2020-08-07T15:07:52.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1500000000000005e-07 2020-08-07T15:07:52.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:52.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.594836+0000", 2020-08-07T15:07:52.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9746000000000002e-05 2020-08-07T15:07:52.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:52.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.594876+0000", 2020-08-07T15:07:52.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045589070000000001 2020-08-07T15:07:52.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:52.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.599435+0000", 2020-08-07T15:07:52.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.203361244 2020-08-07T15:07:52.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:52.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802796+0000", 2020-08-07T15:07:52.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.178e-06 2020-08-07T15:07:52.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:52.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802797+0000", 2020-08-07T15:07:52.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5927000000000005e-05 2020-08-07T15:07:52.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:52.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802893+0000", 2020-08-07T15:07:52.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21005600699999999 2020-08-07T15:07:52.767 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.767 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.767 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.768 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:18478 3.a e27/20 ondisk, result = 0)", 2020-08-07T15:07:52.768 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.595603+0000", 2020-08-07T15:07:52.768 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.128847338, 2020-08-07T15:07:52.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.207367254, 2020-08-07T15:07:52.769 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.769 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:52.769 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.595603+0000", 2020-08-07T15:07:52.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.595603+0000", 2020-08-07T15:07:52.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8887999999999999e-05 2020-08-07T15:07:52.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.595622+0000", 2020-08-07T15:07:52.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8469999999999998e-06 2020-08-07T15:07:52.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.595626+0000", 2020-08-07T15:07:52.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.20636312700000001 2020-08-07T15:07:52.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.801989+0000", 2020-08-07T15:07:52.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0889999999999999e-06 2020-08-07T15:07:52.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.801994+0000", 2020-08-07T15:07:52.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00078913099999999999 2020-08-07T15:07:52.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:52.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802783+0000", 2020-08-07T15:07:52.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8919999999999997e-06 2020-08-07T15:07:52.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:52.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802789+0000", 2020-08-07T15:07:52.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018128 2020-08-07T15:07:52.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:52.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802970+0000", 2020-08-07T15:07:52.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.207367254 2020-08-07T15:07:52.778 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.778 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.778 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.779 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:33488 3.c e30/20 3:3d1929c7:::benchmark_data_smithi074_35447_object4185:head v 30'2120, mlcod=30'2110)", 2020-08-07T15:07:52.779 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:58.023101+0000", 2020-08-07T15:07:52.779 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.701349547, 2020-08-07T15:07:52.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22118623300000001, 2020-08-07T15:07:52.780 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.780 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:52.780 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023101+0000", 2020-08-07T15:07:52.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023101+0000", 2020-08-07T15:07:52.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8794000000000001e-05 2020-08-07T15:07:52.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023150+0000", 2020-08-07T15:07:52.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.545e-06 2020-08-07T15:07:52.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023156+0000", 2020-08-07T15:07:52.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.218926757 2020-08-07T15:07:52.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242083+0000", 2020-08-07T15:07:52.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1200000000000004e-06 2020-08-07T15:07:52.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242087+0000", 2020-08-07T15:07:52.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9561999999999998e-05 2020-08-07T15:07:52.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:52.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242127+0000", 2020-08-07T15:07:52.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3655999999999998e-05 2020-08-07T15:07:52.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:52.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242150+0000", 2020-08-07T15:07:52.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002089273 2020-08-07T15:07:52.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:52.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.244240+0000", 2020-08-07T15:07:52.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7525999999999999e-05 2020-08-07T15:07:52.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:52.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.244287+0000", 2020-08-07T15:07:52.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22118623300000001 2020-08-07T15:07:52.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.790 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.791 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:42104 3.d e32/20 3:bcc35439:::benchmark_data_smithi074_35447_object5262:head v 32'2864, mlcod=32'2858)", 2020-08-07T15:07:52.791 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:07:52.791 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 49.481425018000003, 2020-08-07T15:07:52.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28296418899999998, 2020-08-07T15:07:52.792 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.792 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:52.792 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:07:52.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:07:52.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5550000000000002e-05 2020-08-07T15:07:52.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243081+0000", 2020-08-07T15:07:52.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8139999999999997e-06 2020-08-07T15:07:52.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243087+0000", 2020-08-07T15:07:52.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27968446800000002 2020-08-07T15:07:52.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522771+0000", 2020-08-07T15:07:52.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9459999999999997e-06 2020-08-07T15:07:52.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522776+0000", 2020-08-07T15:07:52.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0778000000000006e-05 2020-08-07T15:07:52.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:52.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522847+0000", 2020-08-07T15:07:52.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0622e-05 2020-08-07T15:07:52.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:52.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522868+0000", 2020-08-07T15:07:52.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030627580000000001 2020-08-07T15:07:52.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:52.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.525930+0000", 2020-08-07T15:07:52.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9252999999999997e-05 2020-08-07T15:07:52.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:52.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.525990+0000", 2020-08-07T15:07:52.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28296418899999998 2020-08-07T15:07:52.803 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:48144 3.0 e36/20 3:0cec1d53:::benchmark_data_smithi074_35447_object6017:head v 36'3128, mlcod=36'3120)", 2020-08-07T15:07:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:08.897123+0000", 2020-08-07T15:07:52.805 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 43.827327308000001, 2020-08-07T15:07:52.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25221023399999998, 2020-08-07T15:07:52.805 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.805 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:52.805 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897123+0000", 2020-08-07T15:07:52.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897123+0000", 2020-08-07T15:07:52.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0655999999999997e-05 2020-08-07T15:07:52.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897174+0000", 2020-08-07T15:07:52.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6769999999999998e-06 2020-08-07T15:07:52.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897181+0000", 2020-08-07T15:07:52.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25090537899999998 2020-08-07T15:07:52.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148086+0000", 2020-08-07T15:07:52.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5520000000000004e-06 2020-08-07T15:07:52.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148092+0000", 2020-08-07T15:07:52.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6391999999999999e-05 2020-08-07T15:07:52.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:52.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148138+0000", 2020-08-07T15:07:52.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8124999999999999e-05 2020-08-07T15:07:52.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:52.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148166+0000", 2020-08-07T15:07:52.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011152009999999999 2020-08-07T15:07:52.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:52.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.149281+0000", 2020-08-07T15:07:52.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2252e-05 2020-08-07T15:07:52.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:52.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.149333+0000", 2020-08-07T15:07:52.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25221023399999998 2020-08-07T15:07:52.831 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.831 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.832 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.832 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:48776 3.2 e36/20 3:47029331:::benchmark_data_smithi074_35447_object6096:head v 36'2888, mlcod=36'2881)", 2020-08-07T15:07:52.833 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:07:52.833 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 43.319930493000001, 2020-08-07T15:07:52.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27006165399999998, 2020-08-07T15:07:52.833 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.834 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:52.834 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:07:52.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:07:52.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8454000000000001e-05 2020-08-07T15:07:52.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404569+0000", 2020-08-07T15:07:52.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2349999999999996e-06 2020-08-07T15:07:52.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404575+0000", 2020-08-07T15:07:52.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26921690999999998 2020-08-07T15:07:52.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673792+0000", 2020-08-07T15:07:52.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3330000000000002e-06 2020-08-07T15:07:52.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673798+0000", 2020-08-07T15:07:52.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1013000000000001e-05 2020-08-07T15:07:52.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:52.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673849+0000", 2020-08-07T15:07:52.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7044999999999999e-05 2020-08-07T15:07:52.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:52.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673876+0000", 2020-08-07T15:07:52.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00065173100000000001 2020-08-07T15:07:52.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:52.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.674528+0000", 2020-08-07T15:07:52.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3933e-05 2020-08-07T15:07:52.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:52.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.674582+0000", 2020-08-07T15:07:52.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27006165399999998 2020-08-07T15:07:52.843 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.844 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.844 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.844 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:49208 3.2 e36/20 3:4d72b982:::benchmark_data_smithi074_35447_object6150:head v 36'2904, mlcod=36'2897)", 2020-08-07T15:07:52.845 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:09.692013+0000", 2020-08-07T15:07:52.845 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 43.032437465000001, 2020-08-07T15:07:52.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.234386975, 2020-08-07T15:07:52.845 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.846 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:52.846 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.692013+0000", 2020-08-07T15:07:52.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.692013+0000", 2020-08-07T15:07:52.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0998000000000002e-05 2020-08-07T15:07:52.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.692064+0000", 2020-08-07T15:07:52.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0210000000000001e-05 2020-08-07T15:07:52.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.692074+0000", 2020-08-07T15:07:52.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.233053807 2020-08-07T15:07:52.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.925128+0000", 2020-08-07T15:07:52.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4879999999999998e-06 2020-08-07T15:07:52.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.925134+0000", 2020-08-07T15:07:52.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5154999999999997e-05 2020-08-07T15:07:52.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:52.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.925179+0000", 2020-08-07T15:07:52.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2710000000000001e-05 2020-08-07T15:07:52.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:52.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.925201+0000", 2020-08-07T15:07:52.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001143725 2020-08-07T15:07:52.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:52.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.926345+0000", 2020-08-07T15:07:52.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4882000000000001e-05 2020-08-07T15:07:52.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:52.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.926400+0000", 2020-08-07T15:07:52.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.234386975 2020-08-07T15:07:52.856 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.856 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.856 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.857 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:77967 3.2 e46/20 3:48f61a4b:::benchmark_data_smithi074_35447_object9745:head v 46'4807, mlcod=46'4798)", 2020-08-07T15:07:52.858 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:28.702144+0000", 2020-08-07T15:07:52.858 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.022306303000001, 2020-08-07T15:07:52.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26780679000000002, 2020-08-07T15:07:52.859 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.860 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:52.860 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702144+0000", 2020-08-07T15:07:52.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702144+0000", 2020-08-07T15:07:52.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5482000000000002e-05 2020-08-07T15:07:52.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702200+0000", 2020-08-07T15:07:52.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.216e-06 2020-08-07T15:07:52.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702207+0000", 2020-08-07T15:07:52.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26634085400000002 2020-08-07T15:07:52.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.968548+0000", 2020-08-07T15:07:52.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2490000000000001e-06 2020-08-07T15:07:52.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.968553+0000", 2020-08-07T15:07:52.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7738999999999998e-05 2020-08-07T15:07:52.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:52.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.968601+0000", 2020-08-07T15:07:52.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5783999999999999e-05 2020-08-07T15:07:52.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:52.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.968627+0000", 2020-08-07T15:07:52.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001270471 2020-08-07T15:07:52.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:52.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.969897+0000", 2020-08-07T15:07:52.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3995000000000001e-05 2020-08-07T15:07:52.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:52.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.969951+0000", 2020-08-07T15:07:52.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26780679000000002 2020-08-07T15:07:52.870 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.870 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.870 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.871 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:77968 3.2 e46/20 3:48f61a4b:::benchmark_data_smithi074_35447_object9745:head v 46'4808, mlcod=46'4799)", 2020-08-07T15:07:52.871 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:28.702836+0000", 2020-08-07T15:07:52.871 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.021614462999999, 2020-08-07T15:07:52.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27322339400000001, 2020-08-07T15:07:52.871 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.871 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:52.872 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702836+0000", 2020-08-07T15:07:52.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702836+0000", 2020-08-07T15:07:52.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1822e-05 2020-08-07T15:07:52.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702888+0000", 2020-08-07T15:07:52.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4110000000000001e-06 2020-08-07T15:07:52.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702894+0000", 2020-08-07T15:07:52.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26750942999999999 2020-08-07T15:07:52.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970404+0000", 2020-08-07T15:07:52.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1150000000000002e-06 2020-08-07T15:07:52.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970409+0000", 2020-08-07T15:07:52.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2898000000000007e-05 2020-08-07T15:07:52.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:52.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970472+0000", 2020-08-07T15:07:52.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2311999999999999e-05 2020-08-07T15:07:52.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:52.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970494+0000", 2020-08-07T15:07:52.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055286800000000002 2020-08-07T15:07:52.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:52.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.976023+0000", 2020-08-07T15:07:52.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6726e-05 2020-08-07T15:07:52.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:52.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.976059+0000", 2020-08-07T15:07:52.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27322339400000001 2020-08-07T15:07:52.883 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.883 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.883 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.884 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86695 3.a 3:5058ba7c:::benchmark_data_smithi074_35447_object10836:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:07:52.884 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:07:52.884 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.399123047, 2020-08-07T15:07:52.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31490022699999998, 2020-08-07T15:07:52.885 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.885 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:52.885 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:52.885 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:52.888 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:52.888 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86695 2020-08-07T15:07:52.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.888 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:07:52.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:07:52.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4560000000000005e-05 2020-08-07T15:07:52.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325392+0000", 2020-08-07T15:07:52.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3180000000000003e-06 2020-08-07T15:07:52.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325398+0000", 2020-08-07T15:07:52.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.258e-05 2020-08-07T15:07:52.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325411+0000", 2020-08-07T15:07:52.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6550000000000001e-06 2020-08-07T15:07:52.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325417+0000", 2020-08-07T15:07:52.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024282879999999998 2020-08-07T15:07:52.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:52.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.327845+0000", 2020-08-07T15:07:52.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1203000000000001e-05 2020-08-07T15:07:52.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:52.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.327936+0000", 2020-08-07T15:07:52.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000186505 2020-08-07T15:07:52.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:52.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328123+0000", 2020-08-07T15:07:52.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020062890000000001 2020-08-07T15:07:52.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:52.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.330129+0000", 2020-08-07T15:07:52.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121513 2020-08-07T15:07:52.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:52.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.330250+0000", 2020-08-07T15:07:52.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30989746899999998 2020-08-07T15:07:52.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:52.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640148+0000", 2020-08-07T15:07:52.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2139999999999999e-06 2020-08-07T15:07:52.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:52.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640149+0000", 2020-08-07T15:07:52.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8081999999999998e-05 2020-08-07T15:07:52.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:52.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640197+0000", 2020-08-07T15:07:52.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0550999999999999e-05 2020-08-07T15:07:52.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:52.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640228+0000", 2020-08-07T15:07:52.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31490022699999998 2020-08-07T15:07:52.932 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.933 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.933 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.933 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86696 3.a 3:5058ba7c:::benchmark_data_smithi074_35447_object10836:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:07:52.934 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:07:52.934 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.398998368000001, 2020-08-07T15:07:52.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.347347556, 2020-08-07T15:07:52.934 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.934 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:52.935 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:52.935 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:52.935 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:52.935 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86696 2020-08-07T15:07:52.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.936 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:07:52.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:07:52.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0436999999999998e-05 2020-08-07T15:07:52.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325503+0000", 2020-08-07T15:07:52.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4659999999999998e-06 2020-08-07T15:07:52.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325508+0000", 2020-08-07T15:07:52.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1177e-05 2020-08-07T15:07:52.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325519+0000", 2020-08-07T15:07:52.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7809999999999996e-06 2020-08-07T15:07:52.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325524+0000", 2020-08-07T15:07:52.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002989219 2020-08-07T15:07:52.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:52.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328513+0000", 2020-08-07T15:07:52.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0649000000000006e-05 2020-08-07T15:07:52.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:52.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328604+0000", 2020-08-07T15:07:52.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001652 2020-08-07T15:07:52.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:52.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328769+0000", 2020-08-07T15:07:52.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023470639999999998 2020-08-07T15:07:52.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:52.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331116+0000", 2020-08-07T15:07:52.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6099999999999999e-07 2020-08-07T15:07:52.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:52.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331117+0000", 2020-08-07T15:07:52.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9150000000000001e-05 2020-08-07T15:07:52.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:52.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331176+0000", 2020-08-07T15:07:52.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065402581000000001 2020-08-07T15:07:52.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:52.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.396579+0000", 2020-08-07T15:07:52.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27618104599999999 2020-08-07T15:07:52.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:52.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672760+0000", 2020-08-07T15:07:52.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2020-08-07T15:07:52.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:52.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672761+0000", 2020-08-07T15:07:52.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9211000000000001e-05 2020-08-07T15:07:52.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:52.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672800+0000", 2020-08-07T15:07:52.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.347347556 2020-08-07T15:07:52.952 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.952 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.952 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.953 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:86695 3.a e52/20 ondisk, result = 0)", 2020-08-07T15:07:52.953 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:07:52.953 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.391041671, 2020-08-07T15:07:52.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.306823501, 2020-08-07T15:07:52.954 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.954 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:52.954 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:07:52.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:07:52.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.048e-05 2020-08-07T15:07:52.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333439+0000", 2020-08-07T15:07:52.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9990000000000002e-06 2020-08-07T15:07:52.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333443+0000", 2020-08-07T15:07:52.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30664195100000002 2020-08-07T15:07:52.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640085+0000", 2020-08-07T15:07:52.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8559999999999998e-06 2020-08-07T15:07:52.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640090+0000", 2020-08-07T15:07:52.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5166000000000001e-05 2020-08-07T15:07:52.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:52.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640135+0000", 2020-08-07T15:07:52.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4160000000000001e-06 2020-08-07T15:07:52.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:52.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640142+0000", 2020-08-07T15:07:52.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0632999999999998e-05 2020-08-07T15:07:52.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:52.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640232+0000", 2020-08-07T15:07:52.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.306823501 2020-08-07T15:07:52.963 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.963 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.963 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.964 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:103800 3.a 3:518bcc79:::benchmark_data_smithi074_35447_object12974:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:07:52.964 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:45.671709+0000", 2020-08-07T15:07:52.964 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 7.052741707, 2020-08-07T15:07:52.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19979493100000001, 2020-08-07T15:07:52.965 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.965 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:52.965 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:52.965 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:52.966 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:52.966 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103800 2020-08-07T15:07:52.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.966 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.671709+0000", 2020-08-07T15:07:52.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.671709+0000", 2020-08-07T15:07:52.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9119000000000002e-05 2020-08-07T15:07:52.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.671758+0000", 2020-08-07T15:07:52.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2329999999999997e-06 2020-08-07T15:07:52.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.671763+0000", 2020-08-07T15:07:52.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.047e-05 2020-08-07T15:07:52.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.671774+0000", 2020-08-07T15:07:52.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2660000000000003e-06 2020-08-07T15:07:52.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.671778+0000", 2020-08-07T15:07:52.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029805809999999999 2020-08-07T15:07:52.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:52.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.674759+0000", 2020-08-07T15:07:52.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4988999999999998e-05 2020-08-07T15:07:52.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:52.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.674814+0000", 2020-08-07T15:07:52.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012427400000000001 2020-08-07T15:07:52.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:52.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.674938+0000", 2020-08-07T15:07:52.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001952121 2020-08-07T15:07:52.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:52.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.676890+0000", 2020-08-07T15:07:52.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.68e-07 2020-08-07T15:07:52.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:52.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.676890+0000", 2020-08-07T15:07:52.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6968000000000001e-05 2020-08-07T15:07:52.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:52.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.676947+0000", 2020-08-07T15:07:52.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039264970000000001 2020-08-07T15:07:52.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:52.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.680874+0000", 2020-08-07T15:07:52.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19055907799999999 2020-08-07T15:07:52.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:52.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.871433+0000", 2020-08-07T15:07:52.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2619999999999999e-06 2020-08-07T15:07:52.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:52.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.871434+0000", 2020-08-07T15:07:52.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9505000000000006e-05 2020-08-07T15:07:52.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:52.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.871504+0000", 2020-08-07T15:07:52.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19979493100000001 2020-08-07T15:07:52.984 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.984 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.985 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.985 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:103856 3.3 e58/20 3:c90e818c:::benchmark_data_smithi074_35447_object12981:head v 58'6128, mlcod=58'6121)", 2020-08-07T15:07:52.985 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:45.703362+0000", 2020-08-07T15:07:52.985 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 7.0210883989999999, 2020-08-07T15:07:52.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18069433600000001, 2020-08-07T15:07:52.986 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:52.986 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:52.986 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:52.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:52.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.703362+0000", 2020-08-07T15:07:52.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:52.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:52.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.703362+0000", 2020-08-07T15:07:52.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6607000000000003e-05 2020-08-07T15:07:52.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:52.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.703409+0000", 2020-08-07T15:07:52.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.172e-06 2020-08-07T15:07:52.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:52.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.703413+0000", 2020-08-07T15:07:52.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.17746437200000001 2020-08-07T15:07:52.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:52.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.880877+0000", 2020-08-07T15:07:52.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5820000000000001e-06 2020-08-07T15:07:52.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:52.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.880883+0000", 2020-08-07T15:07:52.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5605999999999995e-05 2020-08-07T15:07:52.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:52.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.880948+0000", 2020-08-07T15:07:52.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9317000000000002e-05 2020-08-07T15:07:52.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:52.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.880968+0000", 2020-08-07T15:07:52.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030343229999999998 2020-08-07T15:07:52.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:52.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.884002+0000", 2020-08-07T15:07:52.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4357000000000002e-05 2020-08-07T15:07:52.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:52.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.884056+0000", 2020-08-07T15:07:52.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18069433600000001 2020-08-07T15:07:52.998 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.998 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:52.998 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:52.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:52.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:52.999 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:104144 3.a 3:5c46cbda:::benchmark_data_smithi074_35447_object13017:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:07:52.999 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:45.878313+0000", 2020-08-07T15:07:52.999 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.8461376119999997, 2020-08-07T15:07:52.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.251430239, 2020-08-07T15:07:52.999 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:53.000 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:53.000 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:53.000 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:53.000 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:53.000 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 104144 2020-08-07T15:07:53.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.001 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:53.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:53.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.878313+0000", 2020-08-07T15:07:53.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:53.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:53.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.878313+0000", 2020-08-07T15:07:53.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5476000000000001e-05 2020-08-07T15:07:53.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:53.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.878338+0000", 2020-08-07T15:07:53.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.241e-06 2020-08-07T15:07:53.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:53.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.878342+0000", 2020-08-07T15:07:53.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1210000000000007e-06 2020-08-07T15:07:53.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:53.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.878350+0000", 2020-08-07T15:07:53.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5470000000000001e-06 2020-08-07T15:07:53.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:53.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.878353+0000", 2020-08-07T15:07:53.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002441992 2020-08-07T15:07:53.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:53.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.880795+0000", 2020-08-07T15:07:53.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4197999999999995e-05 2020-08-07T15:07:53.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:53.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.880890+0000", 2020-08-07T15:07:53.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000163864 2020-08-07T15:07:53.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:53.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.881053+0000", 2020-08-07T15:07:53.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005483768 2020-08-07T15:07:53.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:53.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886537+0000", 2020-08-07T15:07:53.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7000000000000003e-07 2020-08-07T15:07:53.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:53.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886538+0000", 2020-08-07T15:07:53.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6101999999999998e-05 2020-08-07T15:07:53.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:53.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886604+0000", 2020-08-07T15:07:53.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19508814899999999 2020-08-07T15:07:53.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:53.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.081692+0000", 2020-08-07T15:07:53.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047975891999999999 2020-08-07T15:07:53.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:53.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.129668+0000", 2020-08-07T15:07:53.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.079e-06 2020-08-07T15:07:53.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:53.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.129669+0000", 2020-08-07T15:07:53.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.394e-05 2020-08-07T15:07:53.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:53.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.129743+0000", 2020-08-07T15:07:53.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.251430239 2020-08-07T15:07:53.031 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:53.031 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:53.031 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:53.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.032 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:104144 3.a e58/20 ondisk, result = 0)", 2020-08-07T15:07:53.032 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:45.886622+0000", 2020-08-07T15:07:53.032 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.8378280629999999, 2020-08-07T15:07:53.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24340332200000001, 2020-08-07T15:07:53.033 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:53.033 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:53.033 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:53.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:53.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886622+0000", 2020-08-07T15:07:53.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:53.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:53.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886622+0000", 2020-08-07T15:07:53.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2391999999999998e-05 2020-08-07T15:07:53.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:53.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886655+0000", 2020-08-07T15:07:53.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6959999999999996e-06 2020-08-07T15:07:53.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:53.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886660+0000", 2020-08-07T15:07:53.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.240856029 2020-08-07T15:07:53.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:53.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.127516+0000", 2020-08-07T15:07:53.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9840000000000001e-06 2020-08-07T15:07:53.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:53.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.127521+0000", 2020-08-07T15:07:53.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002130032 2020-08-07T15:07:53.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:53.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.129651+0000", 2020-08-07T15:07:53.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6780000000000006e-06 2020-08-07T15:07:53.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:53.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.129659+0000", 2020-08-07T15:07:53.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00036651099999999997 2020-08-07T15:07:53.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:53.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.130026+0000", 2020-08-07T15:07:53.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24340332200000001 2020-08-07T15:07:53.046 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:53.046 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:53.046 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:53.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.047 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106232 3.a 3:5599d7e6:::benchmark_data_smithi074_35447_object13278:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:07:53.048 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:07:53.048 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.378843421, 2020-08-07T15:07:53.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52578228699999996, 2020-08-07T15:07:53.049 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:53.049 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106232 2020-08-07T15:07:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:07:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:07:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4214999999999997e-05 2020-08-07T15:07:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345641+0000", 2020-08-07T15:07:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9279999999999993e-06 2020-08-07T15:07:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345649+0000", 2020-08-07T15:07:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4391e-05 2020-08-07T15:07:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345664+0000", 2020-08-07T15:07:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7170000000000002e-06 2020-08-07T15:07:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345670+0000", 2020-08-07T15:07:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.459e-05 2020-08-07T15:07:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345695+0000", 2020-08-07T15:07:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6070999999999999e-05 2020-08-07T15:07:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345751+0000", 2020-08-07T15:07:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010069300000000001 2020-08-07T15:07:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345852+0000", 2020-08-07T15:07:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013512299999999999 2020-08-07T15:07:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.347203+0000", 2020-08-07T15:07:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0844000000000001e-05 2020-08-07T15:07:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.347274+0000", 2020-08-07T15:07:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52399471600000003 2020-08-07T15:07:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871269+0000", 2020-08-07T15:07:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.311e-06 2020-08-07T15:07:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871270+0000", 2020-08-07T15:07:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3509000000000004e-05 2020-08-07T15:07:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871343+0000", 2020-08-07T15:07:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6072000000000002e-05 2020-08-07T15:07:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871389+0000", 2020-08-07T15:07:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52578228699999996 2020-08-07T15:07:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106232 3.a e59/20 ondisk, result = 0)", 2020-08-07T15:07:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:07:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.3757972460000003, 2020-08-07T15:07:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52274359299999995, 2020-08-07T15:07:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:07:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:07:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5011000000000001e-05 2020-08-07T15:07:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348668+0000", 2020-08-07T15:07:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1959999999999998e-06 2020-08-07T15:07:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348671+0000", 2020-08-07T15:07:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52253456499999995 2020-08-07T15:07:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871205+0000", 2020-08-07T15:07:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0230000000000003e-06 2020-08-07T15:07:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871209+0000", 2020-08-07T15:07:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0708999999999999e-05 2020-08-07T15:07:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871250+0000", 2020-08-07T15:07:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0158e-05 2020-08-07T15:07:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871260+0000", 2020-08-07T15:07:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013693099999999999 2020-08-07T15:07:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871397+0000", 2020-08-07T15:07:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52274359299999995 2020-08-07T15:07:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:109160 3.a 3:536e7918:::benchmark_data_smithi074_35447_object13644:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e63)", 2020-08-07T15:07:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:51.732858+0000", 2020-08-07T15:07:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.99159288599999995, 2020-08-07T15:07:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18591481100000001, 2020-08-07T15:07:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 109160 2020-08-07T15:07:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.732858+0000", 2020-08-07T15:07:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.732858+0000", 2020-08-07T15:07:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4933e-05 2020-08-07T15:07:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.732883+0000", 2020-08-07T15:07:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3700000000000002e-06 2020-08-07T15:07:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.732885+0000", 2020-08-07T15:07:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.977e-06 2020-08-07T15:07:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.732890+0000", 2020-08-07T15:07:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1110000000000002e-06 2020-08-07T15:07:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.732892+0000", 2020-08-07T15:07:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042819980000000004 2020-08-07T15:07:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:53.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.737174+0000", 2020-08-07T15:07:53.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010377100000000001 2020-08-07T15:07:53.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:53.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.737278+0000", 2020-08-07T15:07:53.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018085099999999999 2020-08-07T15:07:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.737459+0000", 2020-08-07T15:07:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052943529999999999 2020-08-07T15:07:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.742753+0000", 2020-08-07T15:07:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.82e-07 2020-08-07T15:07:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.742754+0000", 2020-08-07T15:07:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116147 2020-08-07T15:07:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.742870+0000", 2020-08-07T15:07:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024778719999999999 2020-08-07T15:07:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.745348+0000", 2020-08-07T15:07:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.173358763 2020-08-07T15:07:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.918706+0000", 2020-08-07T15:07:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.624e-06 2020-08-07T15:07:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.918708+0000", 2020-08-07T15:07:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4559000000000003e-05 2020-08-07T15:07:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.918772+0000", 2020-08-07T15:07:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.18591481100000001 2020-08-07T15:07:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:53.133 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:53.134 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:07:53.138 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:07:53.139 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2020-08-07T15:07:52.630234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":892076032,"num_objects":13614,"num_object_clones":0,"num_object_copies":40842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1471,"num_objects_unfound":0,"num_objects_dirty":13614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108897,"num_write_kb":871169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":990,"num_bytes_recovered":64815104,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":105,"acting":105,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":109,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":11115384,"kb_used_data":2726712,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":743859336,"statfs":{"total":773094113280,"available":761711960064,"internally_reserved":8589934592,"allocated":2792153088,"data_stored":2788788223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29966336,"num_objects":457,"num_object_clones":0,"num_object_copies":1371,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":843,"num_objects_unfound":0,"num_objects_dirty":457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3658,"num_write_kb":29264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":31588352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.297082"},"pg_stats":[{"pgid":"3.f","version":"63'7176","reported_seq":"7227","reported_epoch":"63","state":"active+clean","last_fresh":"2020-08-07T15:07:51.575383+0000","last_change":"2020-08-07T15:06:35.163498+0000","last_active":"2020-08-07T15:07:51.575383+0000","last_peered":"2020-08-07T15:07:51.575383+0000","last_clean":"2020-08-07T15:07:51.575383+0000","last_became_active":"2020-08-07T15:06:35.162764+0000","last_became_peered":"2020-08-07T15:06:35.162764+0000","last_unstale":"2020-08-07T15:07:51.575383+0000","last_undegraded":"2020-08-07T15:07:51.575383+0000","last_fullsized":"2020-08-07T15:07:51.575383+0000","mapping_epoch":20,"log_start":"63'7168","ondisk_log_start":"63'7168","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":58785792,"num_objects":897,"num_object_clones":0,"num_object_copies":2691,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7176,"num_write_kb":57408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"63'7064","reported_seq":"7113","reported_epoch":"63","state":"active+clean","last_fresh":"2020-08-07T15:07:51.690803+0000","last_change":"2020-08-07T15:06:35.169137+0000","last_active":"2020-08-07T15:07:51.690803+0000","last_peered":"2020-08-07T15:07:51.690803+0000","last_clean":"2020-08-07T15:07:51.690803+0000","last_became_active":"2020-08-07T15:06:35.168689+0000","last_became_peered":"2020-08-07T15:06:35.168689+0000","last_unstale":"2020-08-07T15:07:51.690803+0000","last_undegraded":"2020-08-07T15:07:51.690803+0000","last_fullsized":"2020-08-07T15:07:51.690803+0000","mapping_epoch":20,"log_start":"63'7052","ondisk_log_start":"63'7052","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57868288,"num_objects":883,"num_object_clones":0,"num_object_copies":2649,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7064,"num_write_kb":56512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"63'7040","reported_seq":"9566","reported_epoch":"63","state":"active+clean","last_fresh":"2020-08-07T15:07:51.685477+0000","last_change":"2020-08-07T15:07:51.369178+0000","last_active":"2020-08-07T15:07:51.685477+0000","last_peered":"2020-08-07T15:07:51.685477+0000","last_clean":"2020-08-07T15:07:51.685477+0000","last_became_active":"2020-08-07T15:07:51.356141+0000","last_became_peered":"2020-08-07T15:07:51.356141+0000","last_unstale":"2020-08-07T15:07:51.685477+0000","last_undegraded":"2020-08-07T15:07:51.685477+0000","last_fullsized":"2020-08-07T15:07:51.685477+0000","mapping_epoch":62,"log_start":"63'7031","ondisk_log_start":"63'7031","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57671680,"num_objects":880,"num_object_clones":0,"num_object_copies":2640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7040,"num_write_kb":56320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":821,"num_bytes_recovered":53805056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"63'7096","reported_seq":"7147","reported_epoch":"63","state":"active+clean","last_fresh":"2020-08-07T15:07:51.592552+0000","last_change":"2020-08-07T15:06:35.167480+0000","last_active":"2020-08-07T15:07:51.592552+0000","last_peered":"2020-08-07T15:07:51.592552+0000","last_clean":"2020-08-07T15:07:51.592552+0000","last_became_active":"2020-08-07T15:06:35.167101+0000","last_became_peered":"2020-08-07T15:06:35.167101+0000","last_unstale":"2020-08-07T15:07:51.592552+0000","last_undegraded":"2020-08-07T15:07:51.592552+0000","last_fullsized":"2020-08-07T15:07:51.592552+0000","mapping_epoch":20,"log_start":"63'7091","ondisk_log_start":"63'7091","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":58130432,"num_objects":887,"num_object_clones":0,"num_object_copies":2661,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7096,"num_write_kb":56768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"90","reported_epoch":"62","state":"active+clean","last_fresh":"2020-08-07T15:07:50.350257+0000","last_change":"2020-08-07T15:07:33.160980+0000","last_active":"2020-08-07T15:07:50.350257+0000","last_peered":"2020-08-07T15:07:50.350257+0000","last_clean":"2020-08-07T15:07:50.350257+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:07:50.350257+0000","last_undegraded":"2020-08-07T15:07:50.350257+0000","last_fullsized":"2020-08-07T15:07:50.350257+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:18.518049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:18.518049+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.b","version":"63'6200","reported_seq":"6251","reported_epoch":"63","state":"active+clean","last_fresh":"2020-08-07T15:07:51.677087+0000","last_change":"2020-08-07T15:06:35.170504+0000","last_active":"2020-08-07T15:07:51.677087+0000","last_peered":"2020-08-07T15:07:51.677087+0000","last_clean":"2020-08-07T15:07:51.677087+0000","last_became_active":"2020-08-07T15:06:35.169532+0000","last_became_peered":"2020-08-07T15:06:35.169532+0000","last_unstale":"2020-08-07T15:07:51.677087+0000","last_undegraded":"2020-08-07T15:07:51.677087+0000","last_fullsized":"2020-08-07T15:07:51.677087+0000","mapping_epoch":20,"log_start":"63'6193","ondisk_log_start":"63'6193","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50790400,"num_objects":775,"num_object_clones":0,"num_object_copies":2325,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6200,"num_write_kb":49600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"29","reported_epoch":"62","state":"active+clean","last_fresh":"2020-08-07T15:07:50.343405+0000","last_change":"2020-08-07T15:07:33.177491+0000","last_active":"2020-08-07T15:07:50.343405+0000","last_peered":"2020-08-07T15:07:50.343405+0000","last_clean":"2020-08-07T15:07:50.343405+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:07:50.343405+0000","last_undegraded":"2020-08-07T15:07:50.343405+0000","last_fullsized":"2020-08-07T15:07:50.343405+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:33.177412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:33.177412+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.a","version":"59'6800","reported_seq":"6846","reported_epoch":"59","state":"active+clean","last_fresh":"2020-08-07T15:07:47.348904+0000","last_change":"2020-08-07T15:06:35.170105+0000","last_active":"2020-08-07T15:07:47.348904+0000","last_peered":"2020-08-07T15:07:47.348904+0000","last_clean":"2020-08-07T15:07:47.348904+0000","last_became_active":"2020-08-07T15:06:35.169494+0000","last_became_peered":"2020-08-07T15:06:35.169494+0000","last_unstale":"2020-08-07T15:07:47.348904+0000","last_undegraded":"2020-08-07T15:07:47.348904+0000","last_fullsized":"2020-08-07T15:07:47.348904+0000","mapping_epoch":20,"log_start":"59'6793","ondisk_log_start":"59'6793","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":55705600,"num_objects":850,"num_object_clones":0,"num_object_copies":2550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6800,"num_write_kb":54400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"63'6568","reported_seq":"6615","reported_epoch":"63","state":"active+clean","last_fresh":"2020-08-07T15:07:51.700396+0000","last_change":"2020-08-07T15:06:35.163109+0000","last_active":"2020-08-07T15:07:51.700396+0000","last_peered":"2020-08-07T15:07:51.700396+0000","last_clean":"2020-08-07T15:07:51.700396+0000","last_became_active":"2020-08-07T15:06:35.162721+0000","last_became_peered":"2020-08-07T15:06:35.162721+0000","last_unstale":"2020-08-07T15:07:51.700396+0000","last_undegraded":"2020-08-07T15:07:51.700396+0000","last_fullsized":"2020-08-07T15:07:51.700396+0000","mapping_epoch":20,"log_start":"63'6560","ondisk_log_start":"63'6560","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53805056,"num_objects":821,"num_object_clones":0,"num_object_copies":2463,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6568,"num_write_kb":52544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"29","reported_epoch":"62","state":"active+clean","last_fresh":"2020-08-07T15:07:50.343538+0000","last_change":"2020-08-07T15:07:36.143434+0000","last_active":"2020-08-07T15:07:50.343538+0000","last_peered":"2020-08-07T15:07:50.343538+0000","last_clean":"2020-08-07T15:07:50.343538+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:07:50.343538+0000","last_undegraded":"2020-08-07T15:07:50.343538+0000","last_fullsized":"2020-08-07T15:07:50.343538+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.143353+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"63'6760","reported_seq":"6817","reported_epoch":"63","state":"active+remapped+backfill_toofull","last_fresh":"2020-08-07T15:07:51.709200+0000","last_change":"2020-08-07T15:07:49.373818+0000","last_active":"2020-08-07T15:07:51.709200+0000","last_peered":"2020-08-07T15:07:51.709200+0000","last_clean":"2020-08-07T15:07:47.323755+0000","last_became_active":"2020-08-07T15:07:49.354652+0000","last_became_peered":"2020-08-07T15:07:49.354652+0000","last_unstale":"2020-08-07T15:07:51.709200+0000","last_undegraded":"2020-08-07T15:07:51.709200+0000","last_fullsized":"2020-08-07T15:07:51.709200+0000","mapping_epoch":60,"log_start":"63'6756","ondisk_log_start":"63'6756","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":55377920,"num_objects":845,"num_object_clones":0,"num_object_copies":2535,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":845,"num_objects_unfound":0,"num_objects_dirty":845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6760,"num_write_kb":54080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,4],"acting":[1,7,3],"avail_no_missing":["1","3","7"],"object_location_counts":[{"shards":"1,3,7","objects":845}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"47","reported_epoch":"59","state":"peering","last_fresh":"2020-08-07T15:07:47.337336+0000","last_change":"2020-08-07T15:07:47.337530+0000","last_active":"2020-08-07T15:07:47.337336+0000","last_peered":"2020-08-07T15:07:44.333517+0000","last_clean":"2020-08-07T15:07:44.333517+0000","last_became_active":"2020-08-07T15:07:33.166297+0000","last_became_peered":"2020-08-07T15:07:33.166297+0000","last_unstale":"2020-08-07T15:07:47.337336+0000","last_undegraded":"2020-08-07T15:07:47.337336+0000","last_fullsized":"2020-08-07T15:07:47.337336+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.927824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.927824+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"51","reported_epoch":"62","state":"active+clean","last_fresh":"2020-08-07T15:07:50.349996+0000","last_change":"2020-08-07T15:06:46.199001+0000","last_active":"2020-08-07T15:07:50.349996+0000","last_peered":"2020-08-07T15:07:50.349996+0000","last_clean":"2020-08-07T15:07:50.349996+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:07:50.349996+0000","last_undegraded":"2020-08-07T15:07:50.349996+0000","last_fullsized":"2020-08-07T15:07:50.349996+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.5","version":"63'6704","reported_seq":"6755","reported_epoch":"63","state":"active+clean","last_fresh":"2020-08-07T15:07:51.639447+0000","last_change":"2020-08-07T15:06:35.169307+0000","last_active":"2020-08-07T15:07:51.639447+0000","last_peered":"2020-08-07T15:07:51.639447+0000","last_clean":"2020-08-07T15:07:51.639447+0000","last_became_active":"2020-08-07T15:06:35.168887+0000","last_became_peered":"2020-08-07T15:06:35.168887+0000","last_unstale":"2020-08-07T15:07:51.639447+0000","last_undegraded":"2020-08-07T15:07:51.639447+0000","last_fullsized":"2020-08-07T15:07:51.639447+0000","mapping_epoch":20,"log_start":"63'6698","ondisk_log_start":"63'6698","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":54919168,"num_objects":838,"num_object_clones":0,"num_object_copies":2514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6704,"num_write_kb":53632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"40","reported_epoch":"62","state":"active+clean","last_fresh":"2020-08-07T15:07:50.343534+0000","last_change":"2020-08-07T15:07:33.171050+0000","last_active":"2020-08-07T15:07:50.343534+0000","last_peered":"2020-08-07T15:07:50.343534+0000","last_clean":"2020-08-07T15:07:50.343534+0000","last_became_active":"2020-08-07T15:07:33.170597+0000","last_became_peered":"2020-08-07T15:07:33.170597+0000","last_unstale":"2020-08-07T15:07:50.343534+0000","last_undegraded":"2020-08-07T15:07:50.343534+0000","last_fullsized":"2020-08-07T15:07:50.343534+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:18.518049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:18.518049+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"54","reported_epoch":"62","state":"active+clean","last_fresh":"2020-08-07T15:07:50.343643+0000","last_change":"2020-08-07T15:06:23.074962+0000","last_active":"2020-08-07T15:07:50.343643+0000","last_peered":"2020-08-07T15:07:50.343643+0000","last_clean":"2020-08-07T15:07:50.343643+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:07:50.343643+0000","last_undegraded":"2020-08-07T15:07:50.343643+0000","last_fullsized":"2020-08-07T15:07:50.343643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"63'6528","reported_seq":"6579","reported_epoch":"63","state":"active+clean","last_fresh":"2020-08-07T15:07:51.626984+0000","last_change":"2020-08-07T15:06:35.164593+0000","last_active":"2020-08-07T15:07:51.626984+0000","last_peered":"2020-08-07T15:07:51.626984+0000","last_clean":"2020-08-07T15:07:51.626984+0000","last_became_active":"2020-08-07T15:06:35.164108+0000","last_became_peered":"2020-08-07T15:06:35.164108+0000","last_unstale":"2020-08-07T15:07:51.626984+0000","last_undegraded":"2020-08-07T15:07:51.626984+0000","last_fullsized":"2020-08-07T15:07:51.626984+0000","mapping_epoch":20,"log_start":"63'6514","ondisk_log_start":"63'6514","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53477376,"num_objects":816,"num_object_clones":0,"num_object_copies":2448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6528,"num_write_kb":52224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"93","reported_epoch":"62","state":"active+clean","last_fresh":"2020-08-07T15:07:50.349800+0000","last_change":"2020-08-07T15:07:36.358882+0000","last_active":"2020-08-07T15:07:50.349800+0000","last_peered":"2020-08-07T15:07:50.349800+0000","last_clean":"2020-08-07T15:07:50.349800+0000","last_became_active":"2020-08-07T15:07:33.171449+0000","last_became_peered":"2020-08-07T15:07:33.171449+0000","last_unstale":"2020-08-07T15:07:50.349800+0000","last_undegraded":"2020-08-07T15:07:50.349800+0000","last_fullsized":"2020-08-07T15:07:50.349800+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.358798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.358798+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"57","reported_epoch":"62","state":"active+clean","last_fresh":"2020-08-07T15:07:50.348109+0000","last_change":"2020-08-07T15:07:36.864560+0000","last_active":"2020-08-07T15:07:50.348109+0000","last_peered":"2020-08-07T15:07:50.348109+0000","last_clean":"2020-08-07T15:07:50.348109+0000","last_became_active":"2020-08-07T15:06:23.223489+0000","last_became_peered":"2020-08-07T15:06:23.223489+0000","last_unstale":"2020-08-07T15:07:50.348109+0000","last_undegraded":"2020-08-07T15:07:50.348109+0000","last_fullsized":"2020-08-07T15:07:50.348109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:07:36.864483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.864483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"63'7016","reported_seq":"7062","reported_epoch":"63","state":"active+clean","last_fresh":"2020-08-07T15:07:51.710299+0000","last_change":"2020-08-07T15:06:35.170553+0000","last_active":"2020-08-07T15:07:51.710299+0000","last_peered":"2020-08-07T15:07:51.710299+0000","last_clean":"2020-08-07T15:07:51.710299+0000","last_became_active":"2020-08-07T15:06:35.170037+0000","last_became_peered":"2020-08-07T15:06:35.170037+0000","last_unstale":"2020-08-07T15:07:51.710299+0000","last_undegraded":"2020-08-07T15:07:51.710299+0000","last_fullsized":"2020-08-07T15:07:51.710299+0000","mapping_epoch":20,"log_start":"63'7010","ondisk_log_start":"63'7010","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57475072,"num_objects":877,"num_object_clones":0,"num_object_copies":2631,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7016,"num_write_kb":56128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"54","reported_epoch":"62","state":"active+clean","last_fresh":"2020-08-07T15:07:50.343819+0000","last_change":"2020-08-07T15:06:23.075958+0000","last_active":"2020-08-07T15:07:50.343819+0000","last_peered":"2020-08-07T15:07:50.343819+0000","last_clean":"2020-08-07T15:07:50.343819+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:07:50.343819+0000","last_undegraded":"2020-08-07T15:07:50.343819+0000","last_fullsized":"2020-08-07T15:07:50.343819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"77","reported_epoch":"62","state":"active+clean","last_fresh":"2020-08-07T15:07:50.349596+0000","last_change":"2020-08-07T15:07:32.461010+0000","last_active":"2020-08-07T15:07:50.349596+0000","last_peered":"2020-08-07T15:07:50.349596+0000","last_clean":"2020-08-07T15:07:50.349596+0000","last_became_active":"2020-08-07T15:07:28.210245+0000","last_became_peered":"2020-08-07T15:07:28.210245+0000","last_unstale":"2020-08-07T15:07:50.349596+0000","last_undegraded":"2020-08-07T15:07:50.349596+0000","last_fullsized":"2020-08-07T15:07:50.349596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:32.460945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:32.460945+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"63'6872","reported_seq":"6923","reported_epoch":"63","state":"active+clean","last_fresh":"2020-08-07T15:07:51.564804+0000","last_change":"2020-08-07T15:06:35.169888+0000","last_active":"2020-08-07T15:07:51.564804+0000","last_peered":"2020-08-07T15:07:51.564804+0000","last_clean":"2020-08-07T15:07:51.564804+0000","last_became_active":"2020-08-07T15:06:35.169385+0000","last_became_peered":"2020-08-07T15:06:35.169385+0000","last_unstale":"2020-08-07T15:07:51.564804+0000","last_undegraded":"2020-08-07T15:07:51.564804+0000","last_fullsized":"2020-08-07T15:07:51.564804+0000","mapping_epoch":20,"log_start":"63'6866","ondisk_log_start":"63'6866","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56295424,"num_objects":859,"num_object_clones":0,"num_object_copies":2577,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6872,"num_write_kb":54976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"43","reported_epoch":"58","state":"active+clean","last_fresh":"2020-08-07T15:07:44.333320+0000","last_change":"2020-08-07T15:07:33.157062+0000","last_active":"2020-08-07T15:07:44.333320+0000","last_peered":"2020-08-07T15:07:44.333320+0000","last_clean":"2020-08-07T15:07:44.333320+0000","last_became_active":"2020-08-07T15:07:33.156569+0000","last_became_peered":"2020-08-07T15:07:33.156569+0000","last_unstale":"2020-08-07T15:07:44.333320+0000","last_undegraded":"2020-08-07T15:07:44.333320+0000","last_fullsized":"2020-08-07T15:07:44.333320+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:18.518049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:18.518049+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"54","reported_epoch":"62","state":"active+clean","last_fresh":"2020-08-07T15:07:50.349694+0000","last_change":"2020-08-07T15:06:23.224115+0000","last_active":"2020-08-07T15:07:50.349694+0000","last_peered":"2020-08-07T15:07:50.349694+0000","last_clean":"2020-08-07T15:07:50.349694+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:07:50.349694+0000","last_undegraded":"2020-08-07T15:07:50.349694+0000","last_fullsized":"2020-08-07T15:07:50.349694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"63'6376","reported_seq":"6774","reported_epoch":"63","state":"active+remapped+backfilling","last_fresh":"2020-08-07T15:07:51.691846+0000","last_change":"2020-08-07T15:07:49.359056+0000","last_active":"2020-08-07T15:07:51.691846+0000","last_peered":"2020-08-07T15:07:51.691846+0000","last_clean":"2020-08-07T15:07:47.181800+0000","last_became_active":"2020-08-07T15:07:49.355379+0000","last_became_peered":"2020-08-07T15:07:49.355379+0000","last_unstale":"2020-08-07T15:07:51.691846+0000","last_undegraded":"2020-08-07T15:07:51.691846+0000","last_fullsized":"2020-08-07T15:07:51.691846+0000","mapping_epoch":60,"log_start":"63'6369","ondisk_log_start":"63'6369","created":20,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":52232192,"num_objects":797,"num_object_clones":0,"num_object_copies":2391,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":626,"num_objects_unfound":0,"num_objects_dirty":797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6376,"num_write_kb":51008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":11010048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,3],"avail_no_missing":["4","0","3"],"object_location_counts":[{"shards":"0,3,4","objects":797}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"43","reported_epoch":"58","state":"active+clean","last_fresh":"2020-08-07T15:07:44.333374+0000","last_change":"2020-08-07T15:07:33.170029+0000","last_active":"2020-08-07T15:07:44.333374+0000","last_peered":"2020-08-07T15:07:44.333374+0000","last_clean":"2020-08-07T15:07:44.333374+0000","last_became_active":"2020-08-07T15:07:33.169518+0000","last_became_peered":"2020-08-07T15:07:33.169518+0000","last_unstale":"2020-08-07T15:07:44.333374+0000","last_undegraded":"2020-08-07T15:07:44.333374+0000","last_fullsized":"2020-08-07T15:07:44.333374+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:18.518049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:18.518049+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"54","reported_epoch":"62","state":"active+clean","last_fresh":"2020-08-07T15:07:50.350092+0000","last_change":"2020-08-07T15:06:23.225886+0000","last_active":"2020-08-07T15:07:50.350092+0000","last_peered":"2020-08-07T15:07:50.350092+0000","last_clean":"2020-08-07T15:07:50.350092+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:07:50.350092+0000","last_undegraded":"2020-08-07T15:07:50.350092+0000","last_fullsized":"2020-08-07T15:07:50.350092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"63'6856","reported_seq":"6910","reported_epoch":"63","state":"active+clean","last_fresh":"2020-08-07T15:07:51.670434+0000","last_change":"2020-08-07T15:07:38.065908+0000","last_active":"2020-08-07T15:07:51.670434+0000","last_peered":"2020-08-07T15:07:51.670434+0000","last_clean":"2020-08-07T15:07:51.670434+0000","last_became_active":"2020-08-07T15:06:35.169127+0000","last_became_peered":"2020-08-07T15:06:35.169127+0000","last_unstale":"2020-08-07T15:07:51.670434+0000","last_undegraded":"2020-08-07T15:07:51.670434+0000","last_fullsized":"2020-08-07T15:07:51.670434+0000","mapping_epoch":20,"log_start":"63'6846","ondisk_log_start":"63'6846","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'5784","last_scrub_stamp":"2020-08-07T15:07:38.065851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:07:38.065851+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56164352,"num_objects":857,"num_object_clones":0,"num_object_copies":2571,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6856,"num_write_kb":54848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"16'2","reported_seq":"26","reported_epoch":"62","state":"active+clean","last_fresh":"2020-08-07T15:07:50.345315+0000","last_change":"2020-08-07T15:07:33.155878+0000","last_active":"2020-08-07T15:07:50.345315+0000","last_peered":"2020-08-07T15:07:50.345315+0000","last_clean":"2020-08-07T15:07:50.345315+0000","last_became_active":"2020-08-07T15:07:33.155139+0000","last_became_peered":"2020-08-07T15:07:33.155139+0000","last_unstale":"2020-08-07T15:07:50.345315+0000","last_undegraded":"2020-08-07T15:07:50.345315+0000","last_fullsized":"2020-08-07T15:07:50.345315+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:18.518049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:18.518049+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"63'6640","reported_seq":"6691","reported_epoch":"63","state":"active+clean","last_fresh":"2020-08-07T15:07:51.537800+0000","last_change":"2020-08-07T15:06:35.172780+0000","last_active":"2020-08-07T15:07:51.537800+0000","last_peered":"2020-08-07T15:07:51.537800+0000","last_clean":"2020-08-07T15:07:51.537800+0000","last_became_active":"2020-08-07T15:06:35.172360+0000","last_became_peered":"2020-08-07T15:06:35.172360+0000","last_unstale":"2020-08-07T15:07:51.537800+0000","last_undegraded":"2020-08-07T15:07:51.537800+0000","last_fullsized":"2020-08-07T15:07:51.537800+0000","mapping_epoch":20,"log_start":"63'6633","ondisk_log_start":"63'6633","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":54394880,"num_objects":830,"num_object_clones":0,"num_object_copies":2490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6640,"num_write_kb":53120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"57","reported_epoch":"62","state":"active+clean","last_fresh":"2020-08-07T15:07:50.348103+0000","last_change":"2020-08-07T15:07:34.423236+0000","last_active":"2020-08-07T15:07:50.348103+0000","last_peered":"2020-08-07T15:07:50.348103+0000","last_clean":"2020-08-07T15:07:50.348103+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:07:50.348103+0000","last_undegraded":"2020-08-07T15:07:50.348103+0000","last_fullsized":"2020-08-07T15:07:50.348103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:07:34.423115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:07:34.423115+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"36","reported_epoch":"62","state":"active+clean","last_fresh":"2020-08-07T15:07:50.342108+0000","last_change":"2020-08-07T15:07:34.653928+0000","last_active":"2020-08-07T15:07:50.342108+0000","last_peered":"2020-08-07T15:07:50.342108+0000","last_clean":"2020-08-07T15:07:50.342108+0000","last_became_active":"2020-08-07T15:07:33.161761+0000","last_became_peered":"2020-08-07T15:07:33.161761+0000","last_unstale":"2020-08-07T15:07:50.342108+0000","last_undegraded":"2020-08-07T15:07:50.342108+0000","last_fullsized":"2020-08-07T15:07:50.342108+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:34.653843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:34.653843+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"50","reported_epoch":"58","state":"active+clean","last_fresh":"2020-08-07T15:07:44.333618+0000","last_change":"2020-08-07T15:06:23.073702+0000","last_active":"2020-08-07T15:07:44.333618+0000","last_peered":"2020-08-07T15:07:44.333618+0000","last_clean":"2020-08-07T15:07:44.333618+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:07:44.333618+0000","last_undegraded":"2020-08-07T15:07:44.333618+0000","last_fullsized":"2020-08-07T15:07:44.333618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"62'7200","reported_seq":"7253","reported_epoch":"62","state":"active+clean","last_fresh":"2020-08-07T15:07:51.076823+0000","last_change":"2020-08-07T15:06:46.199635+0000","last_active":"2020-08-07T15:07:51.076823+0000","last_peered":"2020-08-07T15:07:51.076823+0000","last_clean":"2020-08-07T15:07:51.076823+0000","last_became_active":"2020-08-07T15:06:46.199201+0000","last_became_peered":"2020-08-07T15:06:46.199201+0000","last_unstale":"2020-08-07T15:07:51.076823+0000","last_undegraded":"2020-08-07T15:07:51.076823+0000","last_fullsized":"2020-08-07T15:07:51.076823+0000","mapping_epoch":26,"log_start":"62'7195","ondisk_log_start":"62'7195","created":20,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":58982400,"num_objects":900,"num_object_clones":0,"num_object_copies":2700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7200,"num_write_kb":57600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":892076032,"num_objects":13612,"num_object_clones":0,"num_object_copies":40836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1471,"num_objects_unfound":0,"num_objects_dirty":13612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108896,"num_write_kb":871168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":989,"num_bytes_recovered":64815104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2793816064,"data_stored":2793816064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22,"ondisk_log_size":22,"up":33,"acting":33,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542164,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1369148,"kb_used_data":320564,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93002692,"statfs":{"total":96636764160,"available":95234756608,"internally_reserved":1073741824,"allocated":328257536,"data_stored":327865177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.5629999999999999},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.63300000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":5.7519999999999998,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":1.319},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.6679999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":1.6439999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":5.5019999999999998,"5min":5.5019999999999998,"15min":5.5019999999999998},"last":1.135},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":1.0089999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.9790000000000001,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":3.887},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.96699999999999997}]},{"osd":5,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":37.506999999999998,"5min":37.506999999999998,"15min":37.506999999999998},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":831.88099999999997,"5min":831.88099999999997,"15min":831.88099999999997},"last":2.6469999999999998},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":2.6739999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":3.1389999999999998,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":3.048,"5min":3.048,"15min":3.048},"last":1.2849999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1376536,"kb_used_data":327952,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92995304,"statfs":{"total":96636764160,"available":95227191296,"internally_reserved":1073741824,"allocated":335822848,"data_stored":335393678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":3.4020000000000001,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.383,"5min":2.383,"15min":2.383},"last":1.1020000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":1.387,"5min":1.387,"15min":1.387},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.5939999999999999,"5min":3.5939999999999999,"15min":3.5939999999999999},"last":1.137}]},{"osd":2,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.272,"5min":1.272,"15min":1.272},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":4.0529999999999999,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":0.72199999999999998}]},{"osd":3,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":3.3580000000000001,"5min":3.3580000000000001,"15min":3.3580000000000001},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":3.004,"5min":3.004,"15min":3.004},"last":0.92200000000000004}]},{"osd":4,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.663,"15min":1.663},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":5.2729999999999997,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"min":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":5.2990000000000004,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.242}]},{"osd":5,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.766,"5min":1.766,"15min":1.766},"min":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":5.327,"5min":5.327,"15min":5.327},"last":1.2150000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":3.3809999999999998,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.4630000000000001},{"interface":"front","average":{"1min":18.326000000000001,"5min":18.326000000000001,"15min":18.326000000000001},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":266.26999999999998,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.4930000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1382568,"kb_used_data":333984,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92989272,"statfs":{"total":96636764160,"available":95221014528,"internally_reserved":1073741824,"allocated":341999616,"data_stored":341581136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":34.186,"5min":34.186,"15min":34.186},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":497.04000000000002,"5min":497.04000000000002,"15min":497.04000000000002},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":4.1980000000000004,"5min":4.1980000000000004,"15min":4.1980000000000004},"last":0.95399999999999996}]},{"osd":2,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":4.0919999999999996,"5min":4.0919999999999996,"15min":4.0919999999999996},"last":0.623}]},{"osd":3,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":1.038},{"interface":"front","average":{"1min":1.454,"5min":1.454,"15min":1.454},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.9889999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":1.0660000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.2,"5min":1.2,"15min":1.2},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.2390000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":13.007999999999999,"5min":13.007999999999999,"15min":13.007999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":213.63800000000001,"5min":213.63800000000001,"15min":213.63800000000001},"last":1.113},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":1.2769999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:07:22 2020","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.552,"15min":1.552},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":3.234,"5min":3.234,"15min":3.234},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.361,"5min":1.361,"15min":1.361},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":1.22}]},{"osd":7,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.615,"5min":1.615,"15min":1.615},"min":{"1min":1.046,"5min":1.046,"15min":1.046},"max":{"1min":3.2810000000000001,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":0.997},{"interface":"front","average":{"1min":1.48,"5min":1.48,"15min":1.48},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.8759999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.26}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1370164,"kb_used_data":321580,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93001676,"statfs":{"total":96636764160,"available":95233716224,"internally_reserved":1073741824,"allocated":329297920,"data_stored":328872268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.2440000000000002,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":0.70399999999999996}]},{"osd":2,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":32.740000000000002,"5min":32.740000000000002,"15min":32.740000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":606.18499999999995,"5min":606.18499999999995,"15min":606.18499999999995},"last":0.85599999999999998}]},{"osd":3,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":0.78700000000000003}]},{"osd":4,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.6949999999999998,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":2.4060000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":0.84099999999999997}]},{"osd":5,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":1.133},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.212}]},{"osd":6,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":8.6140000000000008,"5min":8.6140000000000008,"15min":8.6140000000000008},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":124.738,"5min":124.738,"15min":124.738},"last":0.81100000000000005},{"interface":"front","average":{"1min":18.652999999999999,"5min":18.652999999999999,"15min":18.652999999999999},"min":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"max":{"1min":294.94,"5min":294.94,"15min":294.94},"last":1.016}]},{"osd":7,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":6.468,"5min":6.468,"15min":6.468},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.883,"5min":2.883,"15min":2.883},"last":1.2390000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1266288,"kb_used_data":217704,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93105552,"statfs":{"total":96636764160,"available":95340085248,"internally_reserved":1073741824,"allocated":222928896,"data_stored":222506512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":3.2240000000000002,"5min":3.2240000000000002,"15min":3.2240000000000002},"last":1.4530000000000001},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":3.6520000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":3.1320000000000001,"5min":3.1320000000000001,"15min":3.1320000000000001},"last":2.9540000000000002},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":3.3069999999999999,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":1.472}]},{"osd":3,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.3759999999999999,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.556,"5min":1.556,"15min":1.556},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":8.2210000000000001,"5min":8.2210000000000001,"15min":8.2210000000000001},"last":2.891}]},{"osd":4,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":4.5140000000000002},{"interface":"front","average":{"1min":1.643,"5min":1.643,"15min":1.643},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":3.2749999999999999,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":1.635}]},{"osd":5,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.696},{"interface":"front","average":{"1min":25.524999999999999,"5min":25.524999999999999,"15min":25.524999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":415.40699999999998,"5min":415.40699999999998,"15min":415.40699999999998},"last":4.4790000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":2.9249999999999998},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":1.1819999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.742,"5min":1.742,"15min":1.742},"min":{"1min":1.093,"5min":1.093,"15min":1.093},"max":{"1min":4.056,"5min":4.056,"15min":4.056},"last":1.9239999999999999},{"interface":"front","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":4.4539999999999997,"5min":4.4539999999999997,"15min":4.4539999999999997},"last":1.665}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1478196,"kb_used_data":429612,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92893644,"statfs":{"total":96636764160,"available":95123091456,"internally_reserved":1073741824,"allocated":439922688,"data_stored":439496786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":2.8220000000000001,"5min":2.8220000000000001,"15min":2.8220000000000001},"last":1.895},{"interface":"front","average":{"1min":1.637,"5min":1.637,"15min":1.637},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":13.067,"5min":13.067,"15min":13.067},"last":0.66400000000000003}]},{"osd":1,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.899,"5min":1.899,"15min":1.899},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":6.6829999999999998,"5min":6.6829999999999998,"15min":6.6829999999999998},"last":1.6719999999999999},{"interface":"front","average":{"1min":2.012,"5min":2.012,"15min":2.012},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":11.936,"5min":11.936,"15min":11.936},"last":2.4870000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.589,"5min":3.589,"15min":3.589},"last":1.635},{"interface":"front","average":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":5.5890000000000004,"5min":5.5890000000000004,"15min":5.5890000000000004},"last":1.756}]},{"osd":4,"last update":"Fri Aug 7 15:07:19 2020","interfaces":[{"interface":"back","average":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":4.556,"5min":4.556,"15min":4.556},"last":2.0880000000000001},{"interface":"front","average":{"1min":32.100000000000001,"5min":32.100000000000001,"15min":32.100000000000001},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":677.88599999999997,"5min":677.88599999999997,"15min":677.88599999999997},"last":0.68899999999999995}]},{"osd":5,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":3.6600000000000001,"5min":3.6600000000000001,"15min":3.6600000000000001},"last":1.575},{"interface":"front","average":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"min":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":5.4329999999999998,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":1.603}]},{"osd":6,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":3.621,"5min":3.621,"15min":3.621},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":5.6189999999999998,"5min":5.6189999999999998,"15min":5.6189999999999998},"last":1.7230000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":5.734,"5min":5.734,"15min":5.734},"last":1.7},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":4.8540000000000001,"5min":4.8540000000000001,"15min":4.8540000000000001},"last":2.1960000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1538496,"kb_used_data":489912,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92833344,"statfs":{"total":96636764160,"available":95061344256,"internally_reserved":1073741824,"allocated":501669888,"data_stored":501240718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.181,"15min":1.181},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":4.9480000000000004,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.78200000000000003}]},{"osd":1,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":4.9829999999999997,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.377,"5min":1.377,"15min":1.377},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":5.0430000000000001,"5min":5.0430000000000001,"15min":5.0430000000000001},"last":0.67000000000000004}]},{"osd":2,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.452,"15min":1.452},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":4.415,"5min":4.415,"15min":4.415},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":4.4340000000000002,"5min":4.4340000000000002,"15min":4.4340000000000002},"last":0.93700000000000006}]},{"osd":3,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.508,"15min":1.508},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":5.0099999999999998,"5min":5.0099999999999998,"15min":5.0099999999999998},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":3.2029999999999998,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":0.83299999999999996}]},{"osd":5,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":5.157,"5min":5.157,"15min":5.157},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.627,"5min":1.627,"15min":1.627},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":5.0979999999999999,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":0.96699999999999997}]},{"osd":6,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.79,"5min":1.79,"15min":1.79},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":5.1310000000000002,"5min":5.1310000000000002,"15min":5.1310000000000002},"last":1.0149999999999999},{"interface":"front","average":{"1min":19.109000000000002,"5min":19.109000000000002,"15min":19.109000000000002},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":389.14600000000002,"5min":389.14600000000002,"15min":389.14600000000002},"last":1.073}]},{"osd":7,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":3.3540000000000001,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":4.4569999999999999,"5min":4.4569999999999999,"15min":4.4569999999999999},"last":1.458}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1333988,"kb_used_data":285404,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93037852,"statfs":{"total":96636764160,"available":95270760448,"internally_reserved":1073741824,"allocated":292253696,"data_stored":291831948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.766,"5min":2.766,"15min":2.766},"last":1.1619999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.56100000000000005}]},{"osd":1,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.6509999999999998,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":2.746,"5min":2.746,"15min":2.746},"last":1.0760000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":9.2200000000000006,"5min":9.2200000000000006,"15min":9.2200000000000006},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":168.56899999999999,"5min":168.56899999999999,"15min":168.56899999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":33.131,"5min":33.131,"15min":33.131},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":676.27099999999996,"5min":676.27099999999996,"15min":676.27099999999996},"last":1.05}]},{"osd":3,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.7669999999999999,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.964,"5min":2.964,"15min":2.964},"last":0.65800000000000003}]},{"osd":4,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":50.893000000000001,"5min":50.893000000000001,"15min":50.893000000000001},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":530.30799999999999,"5min":530.30799999999999,"15min":530.30799999999999},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":3.1080000000000001,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.67300000000000004}]},{"osd":6,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":50.408999999999999,"5min":50.408999999999999,"15min":50.408999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":754.31299999999999,"5min":754.31299999999999,"15min":754.31299999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":26.016999999999999,"5min":26.016999999999999,"15min":26.016999999999999},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":517.26700000000005,"5min":517.26700000000005,"15min":517.26700000000005},"last":1.1970000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.282},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"min":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":3.5649999999999999,"5min":3.5649999999999999,"15min":3.5649999999999999},"last":0.81299999999999994}]}]}]}} 2020-08-07T15:07:53.142 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:07:53.148 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:07:53.148 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:53.133+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:53.236 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:53.235+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:53.251 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 77 16 111410 111394 11.3012 11.1641 0.0079228 0.0110572 2020-08-07T15:07:53.337 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:53.336+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:53.373 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:07:53.375 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:53.376 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:53.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.377 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:111625 3.9 e64/20 3:9b5013fb:::benchmark_data_smithi074_35447_object13953:head v 64'6689, mlcod=63'6687)", 2020-08-07T15:07:53.377 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:53.371466+0000", 2020-08-07T15:07:53.378 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.001639869, 2020-08-07T15:07:53.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016608630000000001, 2020-08-07T15:07:53.379 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:53.379 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:53.380 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:53.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:53.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.371466+0000", 2020-08-07T15:07:53.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:53.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:53.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.371466+0000", 2020-08-07T15:07:53.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5254000000000003e-05 2020-08-07T15:07:53.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:53.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.371521+0000", 2020-08-07T15:07:53.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8900000000000004e-06 2020-08-07T15:07:53.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:53.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.371527+0000", 2020-08-07T15:07:53.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000165432 2020-08-07T15:07:53.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:53.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.371693+0000", 2020-08-07T15:07:53.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7009999999999996e-06 2020-08-07T15:07:53.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:53.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.371700+0000", 2020-08-07T15:07:53.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2027000000000003e-05 2020-08-07T15:07:53.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:53.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.371742+0000", 2020-08-07T15:07:53.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1934e-05 2020-08-07T15:07:53.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:53.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.371764+0000", 2020-08-07T15:07:53.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029823799999999999 2020-08-07T15:07:53.401 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:53.401 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:53.406 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:53.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.406 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:111626 3.9 e64/20 3:9b5013fb:::benchmark_data_smithi074_35447_object13953:head v 64'6690, mlcod=63'6687)", 2020-08-07T15:07:53.407 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:53.372145+0000", 2020-08-07T15:07:53.407 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00096111099999999995, 2020-08-07T15:07:53.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001056743, 2020-08-07T15:07:53.407 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:53.407 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:53.408 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:53.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:53.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.372145+0000", 2020-08-07T15:07:53.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:53.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:53.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.372145+0000", 2020-08-07T15:07:53.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2491000000000003e-05 2020-08-07T15:07:53.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:53.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.372197+0000", 2020-08-07T15:07:53.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8800000000000002e-06 2020-08-07T15:07:53.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:53.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.372204+0000", 2020-08-07T15:07:53.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1556000000000005e-05 2020-08-07T15:07:53.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:53.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.372266+0000", 2020-08-07T15:07:53.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9909999999999999e-06 2020-08-07T15:07:53.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:53.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.372272+0000", 2020-08-07T15:07:53.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8590000000000002e-05 2020-08-07T15:07:53.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:53.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.372310+0000", 2020-08-07T15:07:53.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5134999999999999e-05 2020-08-07T15:07:53.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:53.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.372335+0000", 2020-08-07T15:07:53.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000190643 2020-08-07T15:07:53.416 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:53.416 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:53.416 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:53.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.417 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:111627 3.9 e64/20 3:9b5013fb:::benchmark_data_smithi074_35447_object13953:head v 64'6691, mlcod=63'6687)", 2020-08-07T15:07:53.417 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:53.372751+0000", 2020-08-07T15:07:53.417 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00035509300000000001, 2020-08-07T15:07:53.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00053579699999999997, 2020-08-07T15:07:53.418 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:53.428 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:53.428 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:53.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:53.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.372751+0000", 2020-08-07T15:07:53.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:53.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:53.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.372751+0000", 2020-08-07T15:07:53.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1831000000000001e-05 2020-08-07T15:07:53.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:53.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.372783+0000", 2020-08-07T15:07:53.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3039999999999998e-06 2020-08-07T15:07:53.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:53.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.372787+0000", 2020-08-07T15:07:53.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8141e-05 2020-08-07T15:07:53.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:53.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.372825+0000", 2020-08-07T15:07:53.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0219999999999998e-06 2020-08-07T15:07:53.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:53.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.372829+0000", 2020-08-07T15:07:53.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0142e-05 2020-08-07T15:07:53.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:53.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.372859+0000", 2020-08-07T15:07:53.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3073e-05 2020-08-07T15:07:53.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:53.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.372872+0000", 2020-08-07T15:07:53.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121513 2020-08-07T15:07:53.436 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:53.436 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:53.436 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:53.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.437 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:111568 3.e 3:7431d577:::benchmark_data_smithi074_35447_object13945:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e63)", 2020-08-07T15:07:53.437 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:53.336938+0000", 2020-08-07T15:07:53.437 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.036167965000000003, 2020-08-07T15:07:53.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036426750000000001, 2020-08-07T15:07:53.437 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:53.438 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:53.438 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:53.439 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:53.439 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:53.439 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 111568 2020-08-07T15:07:53.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.439 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:53.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:53.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.336938+0000", 2020-08-07T15:07:53.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:53.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:53.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.336938+0000", 2020-08-07T15:07:53.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2938e-05 2020-08-07T15:07:53.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:53.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.336991+0000", 2020-08-07T15:07:53.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2840000000000003e-06 2020-08-07T15:07:53.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:53.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.336997+0000", 2020-08-07T15:07:53.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3774999999999999e-05 2020-08-07T15:07:53.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:53.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.337011+0000", 2020-08-07T15:07:53.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5690000000000001e-06 2020-08-07T15:07:53.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:53.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.337018+0000", 2020-08-07T15:07:53.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00066849699999999999 2020-08-07T15:07:53.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:53.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.337686+0000", 2020-08-07T15:07:53.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9555999999999994e-05 2020-08-07T15:07:53.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:53.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.337776+0000", 2020-08-07T15:07:53.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000175637 2020-08-07T15:07:53.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:07:53.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.337951+0000", 2020-08-07T15:07:53.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013179839999999999 2020-08-07T15:07:53.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:53.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.339269+0000", 2020-08-07T15:07:53.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5799999999999999e-07 2020-08-07T15:07:53.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:53.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.339270+0000", 2020-08-07T15:07:53.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9268000000000003e-05 2020-08-07T15:07:53.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:53.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.339319+0000", 2020-08-07T15:07:53.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002332803 2020-08-07T15:07:53.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:53.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:53.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:53.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:53.341652+0000", 2020-08-07T15:07:53.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0047138689999999999 2020-08-07T15:07:53.453 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:53.453 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:53.453 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:53.453 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:53.453 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:07:53.454 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 4 2020-08-07T15:07:53.454 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:53.456 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:07:53.456 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:07:53.457 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:07:53.457 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:53.454+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:53.470 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:07:53.518 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:07:53.535 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:07:53.565 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:07:53.566 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:53.560+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:53.602 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:07:53.639 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:07:53.639 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:07:53.640 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:07:53.640 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:07:53.640 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:07:53.641 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:07:53.641 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:07:53.641 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:07:53.641 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:07:53.663 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:53.664 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:53.664 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:53.661+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:53.679 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:07:53.783 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:53.782+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:53.887 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:53.886+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:53.898 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:53.898 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:53.913 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:07:53.988 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:53.987+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:54.094 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:54.091+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:54.098 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:54.099 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:54.123 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:07:54.197 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:54.194+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:54.248 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 78 16 113005 112989 11.316 12.4609 0.0115628 0.0110421 2020-08-07T15:07:54.297 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:54.296+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:54.326 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:54.327 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:54.346 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:07:54.439 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:54.435+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:54.506 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:54.499+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:54.543 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:54.544 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:54.556 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:07:54.640 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:54.639+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:54.701 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:54.701+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:54.763 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:54.764 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:54.773 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:07:54.815 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:54.813+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:54.903 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:54.901+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:54.990 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:54.991 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:55.016 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:07:55.017 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:55.002+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:55.104 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:55.103+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:55.206 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:55.205+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:55.234 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:55.235 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:55.247 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 79 16 114403 114387 11.311 10.9219 0.0125565 0.0110475 2020-08-07T15:07:55.248 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:07:55.307 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:55.306+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:55.415 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:55.414+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:55.428 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:55.428 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:55.438 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:07:55.456 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:07:55.522 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:55.521+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:55.617 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:55.616+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:55.656 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:07:55.689 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:55.690 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:55.709 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:07:55.721 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:55.720+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:55.823 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:55.821+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:55.898 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:07:55.924 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:55.922+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:55.938 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:07:55.983 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:55.983 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:55.984 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:07:55.992 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:07:56.019 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:07:56.027 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:56.026+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:56.060 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:07:56.106 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:07:56.129 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:56.129+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:56.139 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:07:56.183 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:07:56.195 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:07:56.196 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:07:56.196 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:07:56.196 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:07:56.196 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:07:56.197 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:07:56.197 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:07:56.197 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:07:56.210 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:56.210 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:07:56.210 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.211 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:115857 3.8 e64/60 3:10d185c1:::benchmark_data_smithi074_35447_object14482:head v 64'7193, mlcod=64'7191)", 2020-08-07T15:07:56.211 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:56.208498+0000", 2020-08-07T15:07:56.211 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.00055701299999999997, 2020-08-07T15:07:56.211 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00058185300000000001, 2020-08-07T15:07:56.212 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:56.212 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:56.212 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:56.212 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.212 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:56.216 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.208498+0000", 2020-08-07T15:07:56.217 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:56.217 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.217 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.218 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:56.219 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.208498+0000", 2020-08-07T15:07:56.219 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1617999999999999e-05 2020-08-07T15:07:56.220 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.220 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.220 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:56.220 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.208549+0000", 2020-08-07T15:07:56.221 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4620000000000001e-06 2020-08-07T15:07:56.221 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.221 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.221 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:56.221 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.208556+0000", 2020-08-07T15:07:56.222 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8983000000000001e-05 2020-08-07T15:07:56.222 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.222 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.222 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:56.222 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.208615+0000", 2020-08-07T15:07:56.223 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5849999999999999e-06 2020-08-07T15:07:56.223 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.223 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.224 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:56.225 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.208620+0000", 2020-08-07T15:07:56.225 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2509999999999997e-05 2020-08-07T15:07:56.225 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.225 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.225 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:56.226 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.208673+0000", 2020-08-07T15:07:56.226 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1146999999999998e-05 2020-08-07T15:07:56.226 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.227 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.227 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:56.227 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.208704+0000", 2020-08-07T15:07:56.228 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000206305 2020-08-07T15:07:56.228 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:56.228 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:56.229 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:56.229 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.234 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.234 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:115855 3.1 e64/60)", 2020-08-07T15:07:56.235 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:56.208998+0000", 2020-08-07T15:07:56.235 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.6336999999999999e-05, 2020-08-07T15:07:56.235 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000152845, 2020-08-07T15:07:56.235 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:56.235 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:07:56.236 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:56.236 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.236 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:56.237 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.208998+0000", 2020-08-07T15:07:56.237 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:56.237 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.238 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.238 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:56.238 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.208998+0000", 2020-08-07T15:07:56.238 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0766999999999997e-05 2020-08-07T15:07:56.238 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.239 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.249 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:56.250 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.209049+0000", 2020-08-07T15:07:56.250 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2650000000000002e-06 2020-08-07T15:07:56.250 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.250 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.251 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:56.251 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.209055+0000", 2020-08-07T15:07:56.251 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0491000000000002e-05 2020-08-07T15:07:56.251 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.252 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.252 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:56.252 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.209106+0000", 2020-08-07T15:07:56.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2950000000000001e-06 2020-08-07T15:07:56.252 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.253 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:56.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.209111+0000", 2020-08-07T15:07:56.253 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2242e-05 2020-08-07T15:07:56.253 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.254 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.255 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:56.255 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.209153+0000", 2020-08-07T15:07:56.256 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015506 2020-08-07T15:07:56.256 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:56.256 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:56.257 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:56.257 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.257 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.258 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:115792 3.7 3:e0749bac:::benchmark_data_smithi074_35447_object14473:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2020-08-07T15:07:56.258 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:56.164009+0000", 2020-08-07T15:07:56.258 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.045045474000000002, 2020-08-07T15:07:56.258 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.045216826000000002, 2020-08-07T15:07:56.258 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:56.259 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:56.259 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:56.259 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:56.259 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:56.259 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 115792 2020-08-07T15:07:56.260 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.260 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:56.260 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.260 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:56.260 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.164009+0000", 2020-08-07T15:07:56.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:56.261 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.261 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.261 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:56.261 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.164009+0000", 2020-08-07T15:07:56.262 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2652999999999999e-05 2020-08-07T15:07:56.262 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.262 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.262 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:56.263 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.164062+0000", 2020-08-07T15:07:56.263 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.643e-06 2020-08-07T15:07:56.263 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.263 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.263 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:56.263 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.164067+0000", 2020-08-07T15:07:56.264 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1572e-05 2020-08-07T15:07:56.264 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.264 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.265 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:56.265 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.164079+0000", 2020-08-07T15:07:56.265 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0379999999999999e-06 2020-08-07T15:07:56.268 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.269 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.269 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:56.269 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.164084+0000", 2020-08-07T15:07:56.270 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0046655150000000003 2020-08-07T15:07:56.270 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.270 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.270 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:56.271 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.168750+0000", 2020-08-07T15:07:56.271 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7472000000000001e-05 2020-08-07T15:07:56.271 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.271 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:56.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.168807+0000", 2020-08-07T15:07:56.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000109623 2020-08-07T15:07:56.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:07:56.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.168917+0000", 2020-08-07T15:07:56.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001594067 2020-08-07T15:07:56.273 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.273 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:56.291 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.170511+0000", 2020-08-07T15:07:56.292 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9999999999999998e-07 2020-08-07T15:07:56.292 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.292 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.292 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:56.292 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.170511+0000", 2020-08-07T15:07:56.293 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3886e-05 2020-08-07T15:07:56.293 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.297 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.297 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:56.300 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.170545+0000", 2020-08-07T15:07:56.300 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0065358689999999997 2020-08-07T15:07:56.303 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:56.313 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:56.313 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:56.314 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:56.314 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:07:56.314 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 3 2020-08-07T15:07:56.314 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:56.316 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:07:56.317 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:2020-08-07T15:07:56.246816+0000 min lat: 0.000913402 max lat: 2.03718 avg lat: 0.0110394 2020-08-07T15:07:56.317 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-08-07T15:07:56.317 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 80 16 115924 115908 11.3182 11.8828 0.00454192 0.0110394 2020-08-07T15:07:56.317 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:56.314+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:56.381 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:07:56.419 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:56.417+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:56.520 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:56.519+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:56.524 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:56.525 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:07:56.525 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:07:56.525 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:07:56.525 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:56.543 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:07:56.626 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:56.620+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:56.727 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:56.722+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:56.763 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:07:56.763 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:07:56.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:07:56.764 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:07:56.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.764 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:104103 3.8 3:11c276ea:::benchmark_data_smithi074_35447_object13012:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:07:56.764 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:45.853251+0000", 2020-08-07T15:07:56.765 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 10.903218848, 2020-08-07T15:07:56.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51821152400000003, 2020-08-07T15:07:56.765 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:56.765 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:56.765 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:56.779 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:56.779 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:56.779 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 104103 2020-08-07T15:07:56.780 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.780 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:56.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.780 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:56.781 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.853251+0000", 2020-08-07T15:07:56.781 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:56.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.781 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:56.782 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.853251+0000", 2020-08-07T15:07:56.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3990000000000003e-05 2020-08-07T15:07:56.782 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:56.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.853305+0000", 2020-08-07T15:07:56.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6409999999999996e-06 2020-08-07T15:07:56.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:56.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.853312+0000", 2020-08-07T15:07:56.785 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3594999999999999e-05 2020-08-07T15:07:56.787 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.790 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:56.790 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.853325+0000", 2020-08-07T15:07:56.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7919999999999998e-06 2020-08-07T15:07:56.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:56.791 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.853331+0000", 2020-08-07T15:07:56.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039587290000000002 2020-08-07T15:07:56.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.792 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.792 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:56.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.857290+0000", 2020-08-07T15:07:56.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010191699999999999 2020-08-07T15:07:56.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.793 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.793 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:56.793 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.857392+0000", 2020-08-07T15:07:56.793 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000203654 2020-08-07T15:07:56.793 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:07:56.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.857595+0000", 2020-08-07T15:07:56.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038121600000000002 2020-08-07T15:07:56.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:56.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861408+0000", 2020-08-07T15:07:56.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3499999999999996e-07 2020-08-07T15:07:56.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.808 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:56.809 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861408+0000", 2020-08-07T15:07:56.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7356999999999999e-05 2020-08-07T15:07:56.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.815 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:56.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861476+0000", 2020-08-07T15:07:56.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00067373200000000002 2020-08-07T15:07:56.816 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:56.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.862149+0000", 2020-08-07T15:07:56.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.50923365300000001 2020-08-07T15:07:56.818 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:56.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.371383+0000", 2020-08-07T15:07:56.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5000000000000001e-07 2020-08-07T15:07:56.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:56.826 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.371384+0000", 2020-08-07T15:07:56.826 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8819000000000005e-05 2020-08-07T15:07:56.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:56.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.371463+0000", 2020-08-07T15:07:56.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51821152400000003 2020-08-07T15:07:56.827 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:56.827 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:56.827 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:56.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.833 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:104104 3.8 3:11c276ea:::benchmark_data_smithi074_35447_object13012:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:07:56.833 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:45.853881+0000", 2020-08-07T15:07:56.833 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 10.902589131999999, 2020-08-07T15:07:56.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51834839399999999, 2020-08-07T15:07:56.833 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:56.834 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:56.834 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:56.834 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:56.834 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:56.835 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 104104 2020-08-07T15:07:56.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.835 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:56.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:56.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.853881+0000", 2020-08-07T15:07:56.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:56.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.836 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:56.836 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.853881+0000", 2020-08-07T15:07:56.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0507999999999998e-05 2020-08-07T15:07:56.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:56.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.853931+0000", 2020-08-07T15:07:56.838 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5810000000000002e-06 2020-08-07T15:07:56.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.838 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:56.838 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.853938+0000", 2020-08-07T15:07:56.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3549e-05 2020-08-07T15:07:56.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.839 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:56.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.853951+0000", 2020-08-07T15:07:56.840 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7849999999999997e-06 2020-08-07T15:07:56.840 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.840 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.840 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:56.840 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.853957+0000", 2020-08-07T15:07:56.840 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004119775 2020-08-07T15:07:56.841 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:56.841 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.858077+0000", 2020-08-07T15:07:56.841 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2958000000000001e-05 2020-08-07T15:07:56.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:56.845 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.858160+0000", 2020-08-07T15:07:56.846 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000116242 2020-08-07T15:07:56.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.846 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.847 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:07:56.850 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.858276+0000", 2020-08-07T15:07:56.850 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003211036 2020-08-07T15:07:56.850 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.851 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:56.851 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861487+0000", 2020-08-07T15:07:56.851 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1900000000000002e-07 2020-08-07T15:07:56.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:56.852 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861488+0000", 2020-08-07T15:07:56.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2969999999999997e-05 2020-08-07T15:07:56.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.853 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:56.853 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861531+0000", 2020-08-07T15:07:56.853 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32897901600000001 2020-08-07T15:07:56.854 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.854 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.854 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:56.854 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.190510+0000", 2020-08-07T15:07:56.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18163950300000001 2020-08-07T15:07:56.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.855 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:56.855 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.372149+0000", 2020-08-07T15:07:56.855 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0699999999999996e-07 2020-08-07T15:07:56.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.856 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.856 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:56.856 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.372150+0000", 2020-08-07T15:07:56.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9045e-05 2020-08-07T15:07:56.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:56.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.372229+0000", 2020-08-07T15:07:56.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51834839399999999 2020-08-07T15:07:56.858 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:56.858 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:56.858 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:56.858 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.859 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:104103 3.8 e58/20 ondisk, result = 0)", 2020-08-07T15:07:56.859 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:45.860696+0000", 2020-08-07T15:07:56.859 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 10.895773720999999, 2020-08-07T15:07:56.859 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51077444900000002, 2020-08-07T15:07:56.859 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:56.860 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:56.860 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:56.860 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:56.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.860696+0000", 2020-08-07T15:07:56.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:56.864 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:56.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.860696+0000", 2020-08-07T15:07:56.865 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8182e-05 2020-08-07T15:07:56.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.865 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:56.865 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.860714+0000", 2020-08-07T15:07:56.866 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5739999999999998e-06 2020-08-07T15:07:56.866 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.866 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.866 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:56.866 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.860717+0000", 2020-08-07T15:07:56.867 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.510595935 2020-08-07T15:07:56.867 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.867 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.867 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:56.867 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.371313+0000", 2020-08-07T15:07:56.867 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4790000000000004e-06 2020-08-07T15:07:56.868 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.868 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.868 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:56.868 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.371318+0000", 2020-08-07T15:07:56.868 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5068000000000003e-05 2020-08-07T15:07:56.869 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.869 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.869 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:56.869 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.371363+0000", 2020-08-07T15:07:56.869 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0251999999999999e-05 2020-08-07T15:07:56.870 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.870 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.870 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:56.870 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.371374+0000", 2020-08-07T15:07:56.870 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6959000000000005e-05 2020-08-07T15:07:56.871 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.871 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:56.871 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.371471+0000", 2020-08-07T15:07:56.871 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51077444900000002 2020-08-07T15:07:56.871 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:56.872 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:56.872 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:56.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.872 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.872 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:104104 3.8 e58/20 ondisk, result = 0)", 2020-08-07T15:07:56.873 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:45.860731+0000", 2020-08-07T15:07:56.873 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 10.895738443000001, 2020-08-07T15:07:56.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51150493600000002, 2020-08-07T15:07:56.873 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:56.873 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:56.874 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:56.874 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.874 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:56.874 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.860731+0000", 2020-08-07T15:07:56.874 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:56.874 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:56.875 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.860731+0000", 2020-08-07T15:07:56.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.487e-05 2020-08-07T15:07:56.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.876 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:56.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.860746+0000", 2020-08-07T15:07:56.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2919999999999998e-06 2020-08-07T15:07:56.876 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.877 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.877 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:56.877 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.860749+0000", 2020-08-07T15:07:56.877 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51133623699999997 2020-08-07T15:07:56.877 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.878 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.878 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:56.878 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.372085+0000", 2020-08-07T15:07:56.878 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9600000000000002e-06 2020-08-07T15:07:56.878 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.879 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:56.879 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.372089+0000", 2020-08-07T15:07:56.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1868e-05 2020-08-07T15:07:56.879 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.880 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:56.880 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.372131+0000", 2020-08-07T15:07:56.880 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1870000000000003e-06 2020-08-07T15:07:56.880 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.881 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:56.881 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.372140+0000", 2020-08-07T15:07:56.881 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6521999999999995e-05 2020-08-07T15:07:56.881 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.881 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.882 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:56.882 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.372236+0000", 2020-08-07T15:07:56.882 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51150493600000002 2020-08-07T15:07:56.882 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:56.882 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:56.882 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:56.883 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.883 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.883 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106239 3.7 3:ebdd8242:::benchmark_data_smithi074_35447_object13279:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:07:56.883 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.349775+0000", 2020-08-07T15:07:56.883 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 9.4066947919999997, 2020-08-07T15:07:56.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.48901084099999997, 2020-08-07T15:07:56.884 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:56.884 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:56.884 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:56.884 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:56.885 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:56.885 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106239 2020-08-07T15:07:56.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.885 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:56.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.886 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:56.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.349775+0000", 2020-08-07T15:07:56.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:56.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:56.887 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.349775+0000", 2020-08-07T15:07:56.887 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4729999999999999e-05 2020-08-07T15:07:56.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:56.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.349830+0000", 2020-08-07T15:07:56.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.254e-06 2020-08-07T15:07:56.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:56.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.349835+0000", 2020-08-07T15:07:56.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0451999999999999e-05 2020-08-07T15:07:56.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.890 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:56.890 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.349846+0000", 2020-08-07T15:07:56.890 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5569999999999997e-06 2020-08-07T15:07:56.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.891 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.891 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:56.891 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.349850+0000", 2020-08-07T15:07:56.891 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034527009999999999 2020-08-07T15:07:56.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.892 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.892 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:56.892 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.353303+0000", 2020-08-07T15:07:56.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000111487 2020-08-07T15:07:56.892 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.893 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:56.893 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.353414+0000", 2020-08-07T15:07:56.893 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018956600000000001 2020-08-07T15:07:56.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.894 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.894 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:07:56.894 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.353604+0000", 2020-08-07T15:07:56.894 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0037590079999999999 2020-08-07T15:07:56.894 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.894 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.895 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:56.895 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.357363+0000", 2020-08-07T15:07:56.895 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1500000000000003e-07 2020-08-07T15:07:56.895 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.895 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:56.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.357364+0000", 2020-08-07T15:07:56.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000113244 2020-08-07T15:07:56.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:56.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.357477+0000", 2020-08-07T15:07:56.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00075726500000000002 2020-08-07T15:07:56.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:56.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.358234+0000", 2020-08-07T15:07:56.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.480494105 2020-08-07T15:07:56.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:56.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.838728+0000", 2020-08-07T15:07:56.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.052e-06 2020-08-07T15:07:56.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:56.920 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.838729+0000", 2020-08-07T15:07:56.920 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6505000000000002e-05 2020-08-07T15:07:56.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:56.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.838786+0000", 2020-08-07T15:07:56.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.48901084099999997 2020-08-07T15:07:56.922 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:56.922 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:56.922 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:56.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.923 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106240 3.7 3:ebdd8242:::benchmark_data_smithi074_35447_object13279:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:07:56.923 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:07:56.923 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 9.4014041400000004, 2020-08-07T15:07:56.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.90079396, 2020-08-07T15:07:56.923 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:56.924 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:56.924 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:56.924 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:56.924 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:56.924 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106240 2020-08-07T15:07:56.925 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.925 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:56.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:56.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:07:56.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:56.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:56.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:07:56.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2522000000000003e-05 2020-08-07T15:07:56.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:56.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355118+0000", 2020-08-07T15:07:56.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6159999999999998e-06 2020-08-07T15:07:56.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:56.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355125+0000", 2020-08-07T15:07:56.931 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2699e-05 2020-08-07T15:07:56.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:56.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355138+0000", 2020-08-07T15:07:56.932 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2310000000000001e-06 2020-08-07T15:07:56.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.932 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.932 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:56.933 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355144+0000", 2020-08-07T15:07:56.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9855000000000003e-05 2020-08-07T15:07:56.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.933 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.933 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:56.938 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355194+0000", 2020-08-07T15:07:56.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112713 2020-08-07T15:07:56.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:56.940 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355306+0000", 2020-08-07T15:07:56.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019794600000000001 2020-08-07T15:07:56.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:07:56.942 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355504+0000", 2020-08-07T15:07:56.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0048346029999999998 2020-08-07T15:07:56.942 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:56.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.360339+0000", 2020-08-07T15:07:56.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00033646400000000002 2020-08-07T15:07:56.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.944 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:56.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.360675+0000", 2020-08-07T15:07:56.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89508481200000001 2020-08-07T15:07:56.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:56.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255760+0000", 2020-08-07T15:07:56.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5659999999999999e-06 2020-08-07T15:07:56.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.946 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.946 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:56.946 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255762+0000", 2020-08-07T15:07:56.946 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6595000000000003e-05 2020-08-07T15:07:56.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.947 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.947 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:56.947 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255818+0000", 2020-08-07T15:07:56.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1338000000000003e-05 2020-08-07T15:07:56.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.948 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:56.949 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255860+0000", 2020-08-07T15:07:56.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.90079396 2020-08-07T15:07:56.949 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:56.950 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:56.950 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:56.950 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.950 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.951 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106240 3.7 e59/20 ondisk, result = 0)", 2020-08-07T15:07:56.951 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:07:56.951 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 9.3050081880000004, 2020-08-07T15:07:56.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80440215100000001, 2020-08-07T15:07:56.951 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:56.952 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:07:56.952 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:56.952 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.953 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:56.953 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:07:56.953 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:56.953 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.954 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.954 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:56.954 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:07:56.954 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0052999999999999e-05 2020-08-07T15:07:56.954 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.955 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.955 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:56.955 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451482+0000", 2020-08-07T15:07:56.955 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3229999999999999e-06 2020-08-07T15:07:56.955 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.955 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.956 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:56.956 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451486+0000", 2020-08-07T15:07:56.956 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80420304799999998 2020-08-07T15:07:56.956 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.956 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.957 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:56.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255689+0000", 2020-08-07T15:07:56.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7759999999999997e-06 2020-08-07T15:07:56.957 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:56.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255694+0000", 2020-08-07T15:07:56.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5828999999999998e-05 2020-08-07T15:07:56.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:56.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255740+0000", 2020-08-07T15:07:56.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1466e-05 2020-08-07T15:07:56.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:56.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255751+0000", 2020-08-07T15:07:56.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112656 2020-08-07T15:07:56.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.962 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:56.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255864+0000", 2020-08-07T15:07:56.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80440215100000001 2020-08-07T15:07:56.963 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:56.963 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:56.963 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:56.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.964 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:107951 3.d 3:bf34718c:::benchmark_data_smithi074_35447_object13493:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e62)", 2020-08-07T15:07:56.964 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.691353+0000", 2020-08-07T15:07:56.964 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.0651170309999998, 2020-08-07T15:07:56.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.665804014, 2020-08-07T15:07:56.964 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:56.965 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:07:56.965 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:56.965 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:56.965 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:56.965 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 107951 2020-08-07T15:07:56.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.966 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:56.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:56.966 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691353+0000", 2020-08-07T15:07:56.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:56.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:56.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691353+0000", 2020-08-07T15:07:56.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2561999999999998e-05 2020-08-07T15:07:56.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.969 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:56.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691385+0000", 2020-08-07T15:07:56.969 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3030000000000001e-06 2020-08-07T15:07:56.969 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.970 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:56.970 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691390+0000", 2020-08-07T15:07:56.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5640000000000002e-06 2020-08-07T15:07:56.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:56.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691399+0000", 2020-08-07T15:07:56.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.425e-06 2020-08-07T15:07:56.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:56.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691404+0000", 2020-08-07T15:07:56.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0949999999999999e-05 2020-08-07T15:07:56.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:56.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691445+0000", 2020-08-07T15:07:56.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0067e-05 2020-08-07T15:07:56.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:07:56.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691455+0000", 2020-08-07T15:07:56.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66557516000000005 2020-08-07T15:07:56.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.975 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:56.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.357030+0000", 2020-08-07T15:07:56.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000126983 2020-08-07T15:07:56.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:56.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.357157+0000", 2020-08-07T15:07:56.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.665804014 2020-08-07T15:07:56.977 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:56.977 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:56.977 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:56.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.978 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:107952 3.d 3:bf34718c:::benchmark_data_smithi074_35447_object13493:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e62)", 2020-08-07T15:07:56.978 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.691432+0000", 2020-08-07T15:07:56.978 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.0650383019999996, 2020-08-07T15:07:56.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66585522200000002, 2020-08-07T15:07:56.979 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:56.979 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:07:56.979 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:56.979 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:56.979 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:56.980 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 107952 2020-08-07T15:07:56.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.981 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:56.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.981 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:56.981 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691432+0000", 2020-08-07T15:07:56.982 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:56.982 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.982 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:56.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691432+0000", 2020-08-07T15:07:56.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6483e-05 2020-08-07T15:07:56.983 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.985 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:56.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691458+0000", 2020-08-07T15:07:56.986 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0259999999999999e-06 2020-08-07T15:07:56.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.986 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:56.986 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691461+0000", 2020-08-07T15:07:56.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9619999999999998e-06 2020-08-07T15:07:56.987 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.987 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:56.987 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691467+0000", 2020-08-07T15:07:56.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7429999999999998e-06 2020-08-07T15:07:56.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:56.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691470+0000", 2020-08-07T15:07:56.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4148e-05 2020-08-07T15:07:56.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.990 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:56.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691494+0000", 2020-08-07T15:07:56.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4599999999999996e-06 2020-08-07T15:07:56.990 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:56.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:56.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:07:56.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691498+0000", 2020-08-07T15:07:56.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66568584500000005 2020-08-07T15:07:56.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.001 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.002 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:57.002 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.357184+0000", 2020-08-07T15:07:57.002 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000102555 2020-08-07T15:07:57.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:57.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.357287+0000", 2020-08-07T15:07:57.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66585522200000002 2020-08-07T15:07:57.004 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:57.004 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:57.004 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:57.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.004 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108233 3.d 3:be154549:::benchmark_data_smithi074_35447_object13529:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e62)", 2020-08-07T15:07:57.005 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.691491+0000", 2020-08-07T15:07:57.005 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.0649784950000001, 2020-08-07T15:07:57.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67823205099999995, 2020-08-07T15:07:57.005 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:57.005 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:57.006 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:57.012 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:57.013 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:57.013 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108233 2020-08-07T15:07:57.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.013 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:57.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:57.014 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691491+0000", 2020-08-07T15:07:57.014 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:57.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.015 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.015 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:57.015 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691491+0000", 2020-08-07T15:07:57.015 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6054000000000002e-05 2020-08-07T15:07:57.016 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.016 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:57.016 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691517+0000", 2020-08-07T15:07:57.017 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7949999999999999e-06 2020-08-07T15:07:57.017 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:57.017 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691520+0000", 2020-08-07T15:07:57.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4539999999999998e-06 2020-08-07T15:07:57.018 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.018 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.018 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:57.018 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691526+0000", 2020-08-07T15:07:57.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4109999999999999e-06 2020-08-07T15:07:57.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.019 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.019 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:57.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691528+0000", 2020-08-07T15:07:57.019 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1905e-05 2020-08-07T15:07:57.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:57.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691560+0000", 2020-08-07T15:07:57.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3219999999999998e-06 2020-08-07T15:07:57.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:07:57.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691566+0000", 2020-08-07T15:07:57.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66574513700000004 2020-08-07T15:07:57.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:57.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.357312+0000", 2020-08-07T15:07:57.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000216388 2020-08-07T15:07:57.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.037 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:57.037 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.357528+0000", 2020-08-07T15:07:57.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000184489 2020-08-07T15:07:57.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.038 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:07:57.038 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.357712+0000", 2020-08-07T15:07:57.038 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023584119999999998 2020-08-07T15:07:57.039 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.039 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.039 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:57.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.360071+0000", 2020-08-07T15:07:57.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1629999999999999e-06 2020-08-07T15:07:57.039 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:57.040 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.360072+0000", 2020-08-07T15:07:57.040 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2478999999999997e-05 2020-08-07T15:07:57.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.041 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:57.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.360124+0000", 2020-08-07T15:07:57.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0094617999999999994 2020-08-07T15:07:57.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.042 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:57.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.369586+0000", 2020-08-07T15:07:57.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7409e-05 2020-08-07T15:07:57.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:57.043 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.369634+0000", 2020-08-07T15:07:57.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2500000000000004e-07 2020-08-07T15:07:57.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.044 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:57.044 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.369635+0000", 2020-08-07T15:07:57.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8907999999999996e-05 2020-08-07T15:07:57.058 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.058 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.058 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:57.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.369723+0000", 2020-08-07T15:07:57.059 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67823205099999995 2020-08-07T15:07:57.059 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:57.059 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:57.059 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:57.059 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.060 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108234 3.d 3:be154549:::benchmark_data_smithi074_35447_object13529:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e62)", 2020-08-07T15:07:57.070 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.691556+0000", 2020-08-07T15:07:57.070 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.0649136319999997, 2020-08-07T15:07:57.071 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67838000700000001, 2020-08-07T15:07:57.071 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:57.071 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:57.071 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:57.072 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:57.072 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:57.072 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108234 2020-08-07T15:07:57.072 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.072 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:57.073 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.073 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:57.073 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691556+0000", 2020-08-07T15:07:57.073 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:57.073 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.078 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.078 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:57.079 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691556+0000", 2020-08-07T15:07:57.079 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5318999999999998e-05 2020-08-07T15:07:57.079 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.079 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.079 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:57.080 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691602+0000", 2020-08-07T15:07:57.080 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.912e-06 2020-08-07T15:07:57.080 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.080 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.080 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:57.081 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691606+0000", 2020-08-07T15:07:57.081 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0250000000000002e-06 2020-08-07T15:07:57.081 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.081 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.081 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:57.082 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691614+0000", 2020-08-07T15:07:57.082 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1640000000000001e-06 2020-08-07T15:07:57.082 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:57.088 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691617+0000", 2020-08-07T15:07:57.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0720000000000002e-05 2020-08-07T15:07:57.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.089 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.089 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:57.089 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691637+0000", 2020-08-07T15:07:57.089 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5660000000000001e-06 2020-08-07T15:07:57.089 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.090 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.090 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:07:57.090 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691645+0000", 2020-08-07T15:07:57.090 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66654076600000001 2020-08-07T15:07:57.091 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.091 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.091 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:57.091 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.358186+0000", 2020-08-07T15:07:57.091 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3206000000000003e-05 2020-08-07T15:07:57.092 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.092 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.092 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:57.092 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.358259+0000", 2020-08-07T15:07:57.093 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000149815 2020-08-07T15:07:57.093 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.093 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.108 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:07:57.109 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.358409+0000", 2020-08-07T15:07:57.109 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0065748500000000001 2020-08-07T15:07:57.109 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.109 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.110 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:57.110 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.364984+0000", 2020-08-07T15:07:57.110 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4589999999999999e-06 2020-08-07T15:07:57.110 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.111 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.111 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:57.111 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.364985+0000", 2020-08-07T15:07:57.111 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5142000000000007e-05 2020-08-07T15:07:57.111 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.112 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:57.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365060+0000", 2020-08-07T15:07:57.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0047306620000000001 2020-08-07T15:07:57.113 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.113 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:57.113 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.369791+0000", 2020-08-07T15:07:57.162 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0852000000000005e-05 2020-08-07T15:07:57.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.163 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.163 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:57.163 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.369862+0000", 2020-08-07T15:07:57.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.133e-06 2020-08-07T15:07:57.164 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:57.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.369863+0000", 2020-08-07T15:07:57.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3416000000000003e-05 2020-08-07T15:07:57.167 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.167 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.167 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:57.167 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.369936+0000", 2020-08-07T15:07:57.168 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67838000700000001 2020-08-07T15:07:57.168 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:57.168 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:57.169 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:57.169 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.169 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.169 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108235 3.d 3:be154549:::benchmark_data_smithi074_35447_object13529:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e62)", 2020-08-07T15:07:57.170 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.691645+0000", 2020-08-07T15:07:57.170 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.0648253580000002, 2020-08-07T15:07:57.170 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67849062100000002, 2020-08-07T15:07:57.170 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:57.170 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:57.171 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:57.171 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:57.171 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:57.171 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108235 2020-08-07T15:07:57.172 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.172 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:57.172 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.172 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:57.172 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691645+0000", 2020-08-07T15:07:57.173 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:57.173 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.173 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.175 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:57.175 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691645+0000", 2020-08-07T15:07:57.175 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7483e-05 2020-08-07T15:07:57.175 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.176 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.176 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:57.176 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691682+0000", 2020-08-07T15:07:57.177 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2610000000000002e-06 2020-08-07T15:07:57.177 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.177 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.177 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:57.177 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691685+0000", 2020-08-07T15:07:57.178 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3719999999999994e-06 2020-08-07T15:07:57.178 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.178 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.178 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:57.179 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691695+0000", 2020-08-07T15:07:57.179 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2339999999999999e-06 2020-08-07T15:07:57.179 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.179 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.180 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:57.180 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691698+0000", 2020-08-07T15:07:57.180 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4182e-05 2020-08-07T15:07:57.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.181 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.181 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:57.181 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691722+0000", 2020-08-07T15:07:57.181 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3889999999999998e-06 2020-08-07T15:07:57.182 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:07:57.183 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691727+0000", 2020-08-07T15:07:57.183 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.667110651 2020-08-07T15:07:57.183 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.183 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.184 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:57.185 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.358837+0000", 2020-08-07T15:07:57.186 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000102965 2020-08-07T15:07:57.186 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.187 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.187 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:57.187 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.358940+0000", 2020-08-07T15:07:57.187 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00022451700000000001 2020-08-07T15:07:57.188 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.188 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.188 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:07:57.188 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.359165+0000", 2020-08-07T15:07:57.189 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0059109999999999996 2020-08-07T15:07:57.189 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.189 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.189 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:57.190 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365076+0000", 2020-08-07T15:07:57.190 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5300000000000002e-07 2020-08-07T15:07:57.190 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.190 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.191 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:57.191 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365077+0000", 2020-08-07T15:07:57.191 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7922000000000005e-05 2020-08-07T15:07:57.192 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.192 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.192 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:57.193 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365145+0000", 2020-08-07T15:07:57.193 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004866085 2020-08-07T15:07:57.193 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.193 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.194 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:57.194 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370011+0000", 2020-08-07T15:07:57.194 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3035999999999998e-05 2020-08-07T15:07:57.195 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.195 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.195 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:57.195 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370064+0000", 2020-08-07T15:07:57.195 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.062e-06 2020-08-07T15:07:57.196 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.196 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.196 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:57.196 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370065+0000", 2020-08-07T15:07:57.197 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0508999999999999e-05 2020-08-07T15:07:57.197 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.197 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.197 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:57.198 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370135+0000", 2020-08-07T15:07:57.198 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67849062100000002 2020-08-07T15:07:57.198 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:57.198 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:57.198 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:57.199 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.199 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.199 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108236 3.d 3:be154549:::benchmark_data_smithi074_35447_object13529:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e62)", 2020-08-07T15:07:57.199 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.691723+0000", 2020-08-07T15:07:57.200 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.0647474299999997, 2020-08-07T15:07:57.200 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67855064399999998, 2020-08-07T15:07:57.200 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:57.201 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:57.201 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:57.202 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:57.202 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:57.202 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108236 2020-08-07T15:07:57.202 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.203 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:57.203 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.203 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:57.204 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691723+0000", 2020-08-07T15:07:57.204 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:57.204 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.204 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.204 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:57.205 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691723+0000", 2020-08-07T15:07:57.205 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4675999999999999e-05 2020-08-07T15:07:57.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.206 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.206 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:57.206 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691747+0000", 2020-08-07T15:07:57.207 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8499999999999998e-06 2020-08-07T15:07:57.207 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.207 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.207 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:57.208 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691750+0000", 2020-08-07T15:07:57.208 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4760000000000001e-06 2020-08-07T15:07:57.208 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.208 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.208 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:57.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691758+0000", 2020-08-07T15:07:57.209 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4909999999999996e-06 2020-08-07T15:07:57.209 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.209 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.210 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:57.210 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691763+0000", 2020-08-07T15:07:57.210 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7676000000000001e-05 2020-08-07T15:07:57.210 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.211 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.214 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:57.215 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691791+0000", 2020-08-07T15:07:57.215 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5159999999999996e-06 2020-08-07T15:07:57.216 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.216 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.217 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:07:57.218 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691795+0000", 2020-08-07T15:07:57.218 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.667739164 2020-08-07T15:07:57.218 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.218 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.219 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:57.219 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.359534+0000", 2020-08-07T15:07:57.219 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8170000000000006e-05 2020-08-07T15:07:57.219 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.220 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.220 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:57.220 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.359603+0000", 2020-08-07T15:07:57.223 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013336399999999999 2020-08-07T15:07:57.223 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.224 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.224 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:07:57.224 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.359736+0000", 2020-08-07T15:07:57.225 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00542254 2020-08-07T15:07:57.225 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.225 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.225 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:57.226 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365158+0000", 2020-08-07T15:07:57.226 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8000000000000005e-07 2020-08-07T15:07:57.226 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.227 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.227 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:57.227 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365159+0000", 2020-08-07T15:07:57.227 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4726000000000003e-05 2020-08-07T15:07:57.228 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.228 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.228 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:57.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365224+0000", 2020-08-07T15:07:57.228 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004972537 2020-08-07T15:07:57.229 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.229 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.229 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:57.230 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370196+0000", 2020-08-07T15:07:57.230 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3263999999999999e-05 2020-08-07T15:07:57.230 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.231 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.231 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:57.231 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370230+0000", 2020-08-07T15:07:57.232 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5700000000000001e-07 2020-08-07T15:07:57.232 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.232 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.233 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:57.233 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370231+0000", 2020-08-07T15:07:57.233 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2556999999999999e-05 2020-08-07T15:07:57.233 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.234 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.234 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:57.234 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370273+0000", 2020-08-07T15:07:57.234 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67855064399999998 2020-08-07T15:07:57.235 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:57.235 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:57.235 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:57.235 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.236 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.236 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108237 3.d 3:be154549:::benchmark_data_smithi074_35447_object13529:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e62)", 2020-08-07T15:07:57.236 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.691798+0000", 2020-08-07T15:07:57.236 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.0646723150000001, 2020-08-07T15:07:57.237 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67865340900000004, 2020-08-07T15:07:57.237 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:57.237 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:57.237 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:57.238 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:57.238 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:57.238 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108237 2020-08-07T15:07:57.239 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.239 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:57.240 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.240 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:57.240 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691798+0000", 2020-08-07T15:07:57.240 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:57.241 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.241 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.241 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:57.241 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691798+0000", 2020-08-07T15:07:57.242 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9239000000000002e-05 2020-08-07T15:07:57.242 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.242 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.243 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:57.243 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691837+0000", 2020-08-07T15:07:57.243 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1839999999999999e-06 2020-08-07T15:07:57.243 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.244 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.244 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:57.244 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691840+0000", 2020-08-07T15:07:57.244 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9400000000000002e-06 2020-08-07T15:07:57.245 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.245 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.245 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:57.245 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691848+0000", 2020-08-07T15:07:57.245 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2740000000000001e-06 2020-08-07T15:07:57.246 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.246 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.246 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:57.246 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691852+0000", 2020-08-07T15:07:57.247 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0021e-05 2020-08-07T15:07:57.247 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.247 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.248 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:57.248 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691872+0000", 2020-08-07T15:07:57.248 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3810000000000001e-06 2020-08-07T15:07:57.249 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.249 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.249 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:07:57.250 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691877+0000", 2020-08-07T15:07:57.250 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66829760299999996 2020-08-07T15:07:57.250 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.250 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.251 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:57.251 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.360174+0000", 2020-08-07T15:07:57.251 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012061999999999999 2020-08-07T15:07:57.251 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.252 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.252 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:57.252 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.360295+0000", 2020-08-07T15:07:57.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020822399999999999 2020-08-07T15:07:57.253 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.253 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:07:57.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.360503+0000", 2020-08-07T15:07:57.256 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0047338340000000001 2020-08-07T15:07:57.256 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.256 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.256 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:57.256 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365237+0000", 2020-08-07T15:07:57.258 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3499999999999995e-07 2020-08-07T15:07:57.258 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.258 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.258 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:57.259 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365238+0000", 2020-08-07T15:07:57.259 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7602999999999996e-05 2020-08-07T15:07:57.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.259 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.259 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:57.260 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365305+0000", 2020-08-07T15:07:57.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0050325420000000001 2020-08-07T15:07:57.260 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.260 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.260 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:57.261 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370338+0000", 2020-08-07T15:07:57.261 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5608000000000001e-05 2020-08-07T15:07:57.261 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.261 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.261 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:57.262 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370383+0000", 2020-08-07T15:07:57.262 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7499999999999998e-07 2020-08-07T15:07:57.262 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.262 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.262 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:57.263 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370384+0000", 2020-08-07T15:07:57.263 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6625999999999995e-05 2020-08-07T15:07:57.263 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.263 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.263 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:57.263 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370451+0000", 2020-08-07T15:07:57.264 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67865340900000004 2020-08-07T15:07:57.264 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:57.264 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:57.264 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:57.265 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.265 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.265 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108238 3.d 3:be154549:::benchmark_data_smithi074_35447_object13529:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e62)", 2020-08-07T15:07:57.265 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.691877+0000", 2020-08-07T15:07:57.265 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.0645932619999998, 2020-08-07T15:07:57.266 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67875236900000002, 2020-08-07T15:07:57.266 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:57.266 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:57.266 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:57.266 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:57.267 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:57.267 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108238 2020-08-07T15:07:57.267 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.267 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:57.267 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.268 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:57.268 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691877+0000", 2020-08-07T15:07:57.268 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:57.268 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.268 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.268 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:57.269 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691877+0000", 2020-08-07T15:07:57.269 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5123999999999999e-05 2020-08-07T15:07:57.269 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.269 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.269 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:57.270 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691902+0000", 2020-08-07T15:07:57.270 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8270000000000002e-06 2020-08-07T15:07:57.270 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.270 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.270 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:57.271 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691905+0000", 2020-08-07T15:07:57.271 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.614e-06 2020-08-07T15:07:57.271 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.271 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:57.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691910+0000", 2020-08-07T15:07:57.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8260000000000001e-06 2020-08-07T15:07:57.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:57.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691913+0000", 2020-08-07T15:07:57.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7557999999999999e-05 2020-08-07T15:07:57.273 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.273 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:57.290 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691941+0000", 2020-08-07T15:07:57.290 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2230000000000001e-06 2020-08-07T15:07:57.290 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.291 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.291 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:07:57.291 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691945+0000", 2020-08-07T15:07:57.291 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66907557799999995 2020-08-07T15:07:57.291 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.292 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.292 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:57.292 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.361020+0000", 2020-08-07T15:07:57.292 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013186799999999999 2020-08-07T15:07:57.292 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.293 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.293 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:57.293 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.361152+0000", 2020-08-07T15:07:57.293 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024732399999999999 2020-08-07T15:07:57.293 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.293 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.294 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:07:57.294 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.361400+0000", 2020-08-07T15:07:57.294 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00391735 2020-08-07T15:07:57.294 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.294 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.295 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:57.295 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365317+0000", 2020-08-07T15:07:57.295 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6700000000000003e-07 2020-08-07T15:07:57.295 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.295 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.296 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:57.296 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365318+0000", 2020-08-07T15:07:57.296 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6130000000000006e-05 2020-08-07T15:07:57.296 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.296 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.297 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:57.297 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365384+0000", 2020-08-07T15:07:57.297 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005119129 2020-08-07T15:07:57.297 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.297 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.299 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:57.300 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370503+0000", 2020-08-07T15:07:57.300 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0532000000000003e-05 2020-08-07T15:07:57.301 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.301 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.302 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:57.302 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370563+0000", 2020-08-07T15:07:57.302 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7600000000000006e-07 2020-08-07T15:07:57.302 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.302 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.303 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:57.303 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370564+0000", 2020-08-07T15:07:57.303 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4743e-05 2020-08-07T15:07:57.303 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.303 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.303 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:57.304 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370629+0000", 2020-08-07T15:07:57.304 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67875236900000002 2020-08-07T15:07:57.304 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:57.304 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:57.304 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:57.305 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.305 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.305 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108239 3.d 3:be154549:::benchmark_data_smithi074_35447_object13529:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e62)", 2020-08-07T15:07:57.305 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.691951+0000", 2020-08-07T15:07:57.305 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.0645191890000003, 2020-08-07T15:07:57.306 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67888694500000002, 2020-08-07T15:07:57.306 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:57.306 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:57.306 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:57.306 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:57.307 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:57.307 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108239 2020-08-07T15:07:57.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.307 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:57.307 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:57.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691951+0000", 2020-08-07T15:07:57.308 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:57.308 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.308 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.309 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:57.309 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691951+0000", 2020-08-07T15:07:57.309 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7484999999999998e-05 2020-08-07T15:07:57.309 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.310 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.310 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:57.310 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691988+0000", 2020-08-07T15:07:57.310 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2739999999999999e-06 2020-08-07T15:07:57.310 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.310 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.311 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:57.311 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691992+0000", 2020-08-07T15:07:57.311 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6389999999999992e-06 2020-08-07T15:07:57.326 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.327 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.327 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:57.327 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691999+0000", 2020-08-07T15:07:57.328 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4410000000000002e-06 2020-08-07T15:07:57.328 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.328 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.328 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:57.329 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.692003+0000", 2020-08-07T15:07:57.329 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9826999999999998e-05 2020-08-07T15:07:57.329 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.329 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.329 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:57.330 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.692022+0000", 2020-08-07T15:07:57.330 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0990000000000001e-06 2020-08-07T15:07:57.330 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.330 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.330 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:07:57.331 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.692027+0000", 2020-08-07T15:07:57.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66992547099999999 2020-08-07T15:07:57.331 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.331 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.331 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:57.332 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.361952+0000", 2020-08-07T15:07:57.332 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010414199999999999 2020-08-07T15:07:57.332 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.332 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.332 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:57.332 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.362056+0000", 2020-08-07T15:07:57.333 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015952499999999999 2020-08-07T15:07:57.333 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.333 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.333 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:07:57.333 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.362216+0000", 2020-08-07T15:07:57.345 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045867449999999997 2020-08-07T15:07:57.345 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.346 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.346 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:57.346 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.366802+0000", 2020-08-07T15:07:57.346 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.178e-06 2020-08-07T15:07:57.347 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.347 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:57.347 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.366804+0000", 2020-08-07T15:07:57.347 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5803999999999995e-05 2020-08-07T15:07:57.348 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.348 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.348 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:57.348 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.366879+0000", 2020-08-07T15:07:57.348 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003807286 2020-08-07T15:07:57.348 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.349 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.349 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:57.349 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370687+0000", 2020-08-07T15:07:57.349 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1903000000000005e-05 2020-08-07T15:07:57.349 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.350 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.350 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:57.350 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370769+0000", 2020-08-07T15:07:57.350 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6899999999999997e-07 2020-08-07T15:07:57.350 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.351 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.351 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:57.351 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370769+0000", 2020-08-07T15:07:57.351 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8557000000000001e-05 2020-08-07T15:07:57.351 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.352 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.352 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:57.352 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370838+0000", 2020-08-07T15:07:57.352 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67888694500000002 2020-08-07T15:07:57.352 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:57.353 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:57.353 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:57.353 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.354 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.354 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108240 3.d 3:be154549:::benchmark_data_smithi074_35447_object13529:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e62)", 2020-08-07T15:07:57.354 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.692026+0000", 2020-08-07T15:07:57.355 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.0644443360000002, 2020-08-07T15:07:57.355 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67892993700000004, 2020-08-07T15:07:57.355 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:57.355 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:57.355 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:57.355 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:57.356 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:57.356 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108240 2020-08-07T15:07:57.356 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.356 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:57.356 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.357 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:57.357 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.692026+0000", 2020-08-07T15:07:57.357 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:57.357 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.357 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.363 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:57.363 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.692026+0000", 2020-08-07T15:07:57.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6211e-05 2020-08-07T15:07:57.363 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:57.364 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.692052+0000", 2020-08-07T15:07:57.364 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8830000000000002e-06 2020-08-07T15:07:57.364 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.365 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.365 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:57.365 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.692055+0000", 2020-08-07T15:07:57.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.609e-06 2020-08-07T15:07:57.365 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.366 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:57.366 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.692060+0000", 2020-08-07T15:07:57.366 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4889999999999998e-06 2020-08-07T15:07:57.366 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.367 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:57.367 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.692063+0000", 2020-08-07T15:07:57.367 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9899e-05 2020-08-07T15:07:57.367 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.367 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.368 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:57.368 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.692083+0000", 2020-08-07T15:07:57.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2910000000000001e-06 2020-08-07T15:07:57.368 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.368 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.369 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:07:57.369 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.692087+0000", 2020-08-07T15:07:57.369 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67051431100000003 2020-08-07T15:07:57.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.369 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:57.370 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.362601+0000", 2020-08-07T15:07:57.370 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000118664 2020-08-07T15:07:57.370 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.370 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.370 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:57.371 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.362720+0000", 2020-08-07T15:07:57.371 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000194902 2020-08-07T15:07:57.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.371 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.371 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:07:57.372 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.362915+0000", 2020-08-07T15:07:57.372 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039819479999999999 2020-08-07T15:07:57.372 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.372 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:57.372 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.366897+0000", 2020-08-07T15:07:57.373 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8800000000000002e-07 2020-08-07T15:07:57.373 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.373 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.373 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:57.373 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.366898+0000", 2020-08-07T15:07:57.374 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3904000000000003e-05 2020-08-07T15:07:57.374 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.374 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.374 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:57.374 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.366972+0000", 2020-08-07T15:07:57.375 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003757608 2020-08-07T15:07:57.375 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.375 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:57.375 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370729+0000", 2020-08-07T15:07:57.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015924999999999999 2020-08-07T15:07:57.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:57.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370888+0000", 2020-08-07T15:07:57.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7199999999999999e-07 2020-08-07T15:07:57.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:57.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370889+0000", 2020-08-07T15:07:57.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6607999999999997e-05 2020-08-07T15:07:57.379 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:57.390 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370956+0000", 2020-08-07T15:07:57.390 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67892993700000004 2020-08-07T15:07:57.391 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:57.391 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:57.391 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:57.391 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.391 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.392 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108505 3.d 3:b25992ab:::benchmark_data_smithi074_35447_object13563:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e62)", 2020-08-07T15:07:57.392 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.820278+0000", 2020-08-07T15:07:57.392 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.9361917100000001, 2020-08-07T15:07:57.392 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.55083482699999997, 2020-08-07T15:07:57.392 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:57.393 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:57.393 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:57.393 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:57.393 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:57.393 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108505 2020-08-07T15:07:57.394 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.394 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:57.395 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.395 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:57.395 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.820278+0000", 2020-08-07T15:07:57.395 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:57.396 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.396 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.397 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:57.397 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.820278+0000", 2020-08-07T15:07:57.397 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8326999999999998e-05 2020-08-07T15:07:57.397 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.397 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.398 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:57.398 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.820307+0000", 2020-08-07T15:07:57.398 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5059999999999997e-06 2020-08-07T15:07:57.398 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.398 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.399 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:57.399 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.820311+0000", 2020-08-07T15:07:57.399 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2397e-05 2020-08-07T15:07:57.399 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.399 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:57.400 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.820323+0000", 2020-08-07T15:07:57.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4620000000000004e-06 2020-08-07T15:07:57.404 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.404 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.404 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:57.404 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.820329+0000", 2020-08-07T15:07:57.405 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0113e-05 2020-08-07T15:07:57.405 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.405 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:57.405 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.820369+0000", 2020-08-07T15:07:57.406 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.79e-06 2020-08-07T15:07:57.406 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.406 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.406 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:07:57.407 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.820377+0000", 2020-08-07T15:07:57.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.54296667899999995 2020-08-07T15:07:57.407 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.407 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.407 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:57.407 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.363344+0000", 2020-08-07T15:07:57.408 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015576300000000001 2020-08-07T15:07:57.408 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.408 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.408 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:57.409 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.363499+0000", 2020-08-07T15:07:57.410 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00021215700000000001 2020-08-07T15:07:57.410 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.410 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.410 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:07:57.410 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.363711+0000", 2020-08-07T15:07:57.411 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003271422 2020-08-07T15:07:57.411 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.411 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.411 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:57.411 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.366983+0000", 2020-08-07T15:07:57.412 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4899999999999995e-07 2020-08-07T15:07:57.412 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.412 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.412 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:57.412 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.366983+0000", 2020-08-07T15:07:57.413 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5705999999999997e-05 2020-08-07T15:07:57.413 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.413 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:57.413 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.367049+0000", 2020-08-07T15:07:57.414 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003955554 2020-08-07T15:07:57.414 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.414 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.414 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:57.414 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.371005+0000", 2020-08-07T15:07:57.415 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0803999999999998e-05 2020-08-07T15:07:57.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.415 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:57.415 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.371046+0000", 2020-08-07T15:07:57.415 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8100000000000003e-07 2020-08-07T15:07:57.416 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.416 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.416 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:57.416 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.371046+0000", 2020-08-07T15:07:57.416 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7016999999999999e-05 2020-08-07T15:07:57.417 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.417 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.417 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:57.417 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.371113+0000", 2020-08-07T15:07:57.417 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.55083482699999997 2020-08-07T15:07:57.418 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:57.418 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:57.418 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:57.418 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.418 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.419 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108506 3.d 3:b25992ab:::benchmark_data_smithi074_35447_object13563:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e62)", 2020-08-07T15:07:57.419 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.821020+0000", 2020-08-07T15:07:57.420 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.9354499250000003, 2020-08-07T15:07:57.420 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.55317842500000003, 2020-08-07T15:07:57.420 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:57.420 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:57.420 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:57.421 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:57.421 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:57.421 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108506 2020-08-07T15:07:57.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.421 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:57.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.422 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:57.422 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.821020+0000", 2020-08-07T15:07:57.422 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:57.422 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.423 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.423 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:57.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.821020+0000", 2020-08-07T15:07:57.423 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8736000000000001e-05 2020-08-07T15:07:57.423 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.424 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.424 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:57.424 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.821049+0000", 2020-08-07T15:07:57.424 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.72e-06 2020-08-07T15:07:57.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.424 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:57.425 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.821052+0000", 2020-08-07T15:07:57.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3499999999999997e-06 2020-08-07T15:07:57.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.425 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.426 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:57.426 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.821061+0000", 2020-08-07T15:07:57.426 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8620000000000002e-06 2020-08-07T15:07:57.426 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.426 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:57.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.821066+0000", 2020-08-07T15:07:57.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5120000000000003e-05 2020-08-07T15:07:57.427 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.427 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:57.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.821101+0000", 2020-08-07T15:07:57.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5800000000000003e-06 2020-08-07T15:07:57.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.429 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:07:57.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.821108+0000", 2020-08-07T15:07:57.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.54305088899999998 2020-08-07T15:07:57.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.430 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:57.431 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.364159+0000", 2020-08-07T15:07:57.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000116743 2020-08-07T15:07:57.431 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.431 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:57.432 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.364276+0000", 2020-08-07T15:07:57.432 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00022238499999999999 2020-08-07T15:07:57.432 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.432 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.432 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:07:57.433 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.364498+0000", 2020-08-07T15:07:57.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042165290000000001 2020-08-07T15:07:57.433 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.433 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.433 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:57.434 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.368715+0000", 2020-08-07T15:07:57.434 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2529999999999999e-06 2020-08-07T15:07:57.434 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.434 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.434 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:57.435 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.368716+0000", 2020-08-07T15:07:57.435 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7997999999999995e-05 2020-08-07T15:07:57.435 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.435 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.435 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:57.436 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.368784+0000", 2020-08-07T15:07:57.436 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023803190000000001 2020-08-07T15:07:57.436 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.436 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.436 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:57.436 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.371164+0000", 2020-08-07T15:07:57.437 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0029631760000000001 2020-08-07T15:07:57.437 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.437 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.437 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:57.437 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.374128+0000", 2020-08-07T15:07:57.438 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0240000000000001e-06 2020-08-07T15:07:57.438 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.438 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.438 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:57.438 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.374129+0000", 2020-08-07T15:07:57.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9740999999999995e-05 2020-08-07T15:07:57.439 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.439 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.439 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:57.439 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.374198+0000", 2020-08-07T15:07:57.440 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.55317842500000003 2020-08-07T15:07:57.440 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:57.440 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:57.441 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:57.441 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.441 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.441 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108507 3.d 3:b25992ab:::benchmark_data_smithi074_35447_object13563:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e62)", 2020-08-07T15:07:57.442 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.823955+0000", 2020-08-07T15:07:57.442 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.9325153039999998, 2020-08-07T15:07:57.442 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.55129072499999998, 2020-08-07T15:07:57.442 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:07:57.443 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:57.443 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:07:57.443 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:07:57.444 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:57.444 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108507 2020-08-07T15:07:57.444 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.444 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:07:57.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.445 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:07:57.445 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.823955+0000", 2020-08-07T15:07:57.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:07:57.445 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.445 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.446 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:07:57.446 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.823955+0000", 2020-08-07T15:07:57.446 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.955e-05 2020-08-07T15:07:57.446 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.446 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.447 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:07:57.447 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.823984+0000", 2020-08-07T15:07:57.447 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2259999999999999e-06 2020-08-07T15:07:57.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.447 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:07:57.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.823988+0000", 2020-08-07T15:07:57.448 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6879999999999994e-06 2020-08-07T15:07:57.448 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:07:57.449 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.823998+0000", 2020-08-07T15:07:57.449 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5119999999999999e-06 2020-08-07T15:07:57.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.449 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.449 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:07:57.450 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.824004+0000", 2020-08-07T15:07:57.450 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8118000000000003e-05 2020-08-07T15:07:57.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.450 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:57.450 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.824042+0000", 2020-08-07T15:07:57.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0639999999999996e-06 2020-08-07T15:07:57.451 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:07:57.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.824049+0000", 2020-08-07T15:07:57.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.541388706 2020-08-07T15:07:57.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.453 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:07:57.453 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365438+0000", 2020-08-07T15:07:57.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000125785 2020-08-07T15:07:57.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.454 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.454 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:07:57.454 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365563+0000", 2020-08-07T15:07:57.454 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016269799999999999 2020-08-07T15:07:57.454 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:07:57.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365726+0000", 2020-08-07T15:07:57.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0035814530000000001 2020-08-07T15:07:57.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:07:57.456 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.369307+0000", 2020-08-07T15:07:57.456 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.17e-06 2020-08-07T15:07:57.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:57.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.369309+0000", 2020-08-07T15:07:57.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1161000000000005e-05 2020-08-07T15:07:57.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.458 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:07:57.458 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.369380+0000", 2020-08-07T15:07:57.458 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0051009820000000004 2020-08-07T15:07:57.458 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.458 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.458 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:57.459 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.374481+0000", 2020-08-07T15:07:57.459 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00067293099999999998 2020-08-07T15:07:57.459 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:57.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.375154+0000", 2020-08-07T15:07:57.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9600000000000008e-07 2020-08-07T15:07:57.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.460 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.460 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:07:57.461 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.375155+0000", 2020-08-07T15:07:57.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0685000000000003e-05 2020-08-07T15:07:57.461 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:07:57.462 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:07:57.462 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:07:57.463 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.375245+0000", 2020-08-07T15:07:57.463 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.55129072499999998 2020-08-07T15:07:57.463 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:57.463 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:57.463 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:57.464 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:07:57.464 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:07:57.464 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:07:57.466 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:07:57.467 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 81 16 117126 117110 11.2944 9.39062 0.00961404 0.011062 2020-08-07T15:07:57.467 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:07:57.464+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:07:57.568 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:57.565+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:57.665 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:57.665 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:57.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.669 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:117754 3.e e65/20 3:7e187785:::benchmark_data_smithi074_35447_object14719:head v 65'7514, mlcod=65'7506)", 2020-08-07T15:07:57.671 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:57.656008+0000", 2020-08-07T15:07:57.678 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0065920880000000003, 2020-08-07T15:07:57.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0066093200000000001, 2020-08-07T15:07:57.679 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:57.679 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:57.681 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:57.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:57.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656008+0000", 2020-08-07T15:07:57.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:57.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:57.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656008+0000", 2020-08-07T15:07:57.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0939000000000001e-05 2020-08-07T15:07:57.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:57.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656059+0000", 2020-08-07T15:07:57.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9920000000000004e-06 2020-08-07T15:07:57.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:57.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656065+0000", 2020-08-07T15:07:57.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014341500000000001 2020-08-07T15:07:57.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:57.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656208+0000", 2020-08-07T15:07:57.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8260000000000003e-06 2020-08-07T15:07:57.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:57.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656212+0000", 2020-08-07T15:07:57.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000380987 2020-08-07T15:07:57.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:57.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656593+0000", 2020-08-07T15:07:57.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2640999999999999e-05 2020-08-07T15:07:57.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:57.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656616+0000", 2020-08-07T15:07:57.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00060780000000000003 2020-08-07T15:07:57.703 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.703 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:57.703 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.725 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:117755 3.e e65/20 3:7e187785:::benchmark_data_smithi074_35447_object14719:head v 65'7515, mlcod=65'7506)", 2020-08-07T15:07:57.726 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:57.656102+0000", 2020-08-07T15:07:57.726 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0064980669999999997, 2020-08-07T15:07:57.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0065554949999999997, 2020-08-07T15:07:57.726 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:57.727 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:57.727 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:57.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:57.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656102+0000", 2020-08-07T15:07:57.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:57.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:57.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656102+0000", 2020-08-07T15:07:57.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3192000000000002e-05 2020-08-07T15:07:57.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:57.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656155+0000", 2020-08-07T15:07:57.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3349999999999997e-06 2020-08-07T15:07:57.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:57.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656161+0000", 2020-08-07T15:07:57.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3693000000000002e-05 2020-08-07T15:07:57.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:57.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656224+0000", 2020-08-07T15:07:57.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3979999999999999e-06 2020-08-07T15:07:57.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:57.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656228+0000", 2020-08-07T15:07:57.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00063651699999999996 2020-08-07T15:07:57.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:57.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656864+0000", 2020-08-07T15:07:57.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1075999999999999e-05 2020-08-07T15:07:57.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:57.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656885+0000", 2020-08-07T15:07:57.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00078321100000000002 2020-08-07T15:07:57.748 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.748 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:57.748 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.749 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:117711 3.9 3:966a7651:::benchmark_data_smithi074_35447_object14713:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e65)", 2020-08-07T15:07:57.749 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:57.630579+0000", 2020-08-07T15:07:57.749 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.032020977999999999, 2020-08-07T15:07:57.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032117879000000002, 2020-08-07T15:07:57.749 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:57.750 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:57.750 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:57.750 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:57.750 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:57.750 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 117711 2020-08-07T15:07:57.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.751 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:57.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:57.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.630579+0000", 2020-08-07T15:07:57.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:57.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:57.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.630579+0000", 2020-08-07T15:07:57.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6910999999999999e-05 2020-08-07T15:07:57.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:57.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.630626+0000", 2020-08-07T15:07:57.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1930000000000001e-06 2020-08-07T15:07:57.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:57.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.630631+0000", 2020-08-07T15:07:57.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3689999999999996e-06 2020-08-07T15:07:57.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:57.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.630641+0000", 2020-08-07T15:07:57.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4690000000000002e-06 2020-08-07T15:07:57.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:57.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.630644+0000", 2020-08-07T15:07:57.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0072504220000000003 2020-08-07T15:07:57.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:57.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.637894+0000", 2020-08-07T15:07:57.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023697400000000001 2020-08-07T15:07:57.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:57.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.638131+0000", 2020-08-07T15:07:57.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018769699999999999 2020-08-07T15:07:57.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:57.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.638319+0000", 2020-08-07T15:07:57.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030852620000000001 2020-08-07T15:07:57.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:57.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.641404+0000", 2020-08-07T15:07:57.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2699999999999999e-07 2020-08-07T15:07:57.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:57.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.641405+0000", 2020-08-07T15:07:57.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8118999999999997e-05 2020-08-07T15:07:57.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:57.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.641463+0000", 2020-08-07T15:07:57.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042949540000000001 2020-08-07T15:07:57.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:57.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.645758+0000", 2020-08-07T15:07:57.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015179097000000001 2020-08-07T15:07:57.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.764 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:57.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.765 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:117712 3.9 3:966a7651:::benchmark_data_smithi074_35447_object14713:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e65)", 2020-08-07T15:07:57.765 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:57.630675+0000", 2020-08-07T15:07:57.766 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.031925479, 2020-08-07T15:07:57.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032082339000000001, 2020-08-07T15:07:57.766 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:57.766 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:57.766 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:57.766 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:57.767 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:57.767 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 117712 2020-08-07T15:07:57.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.767 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:57.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:57.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.630675+0000", 2020-08-07T15:07:57.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:57.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:57.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.630675+0000", 2020-08-07T15:07:57.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9595000000000002e-05 2020-08-07T15:07:57.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:57.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.630724+0000", 2020-08-07T15:07:57.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4330000000000003e-06 2020-08-07T15:07:57.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:57.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.630730+0000", 2020-08-07T15:07:57.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0730000000000001e-05 2020-08-07T15:07:57.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:57.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.630740+0000", 2020-08-07T15:07:57.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7110000000000003e-06 2020-08-07T15:07:57.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:57.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.630745+0000", 2020-08-07T15:07:57.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081744150000000008 2020-08-07T15:07:57.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:57.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.638919+0000", 2020-08-07T15:07:57.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4364999999999995e-05 2020-08-07T15:07:57.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:57.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.639014+0000", 2020-08-07T15:07:57.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017228499999999999 2020-08-07T15:07:57.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:57.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.639186+0000", 2020-08-07T15:07:57.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038958230000000001 2020-08-07T15:07:57.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:57.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.643082+0000", 2020-08-07T15:07:57.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1200000000000001e-07 2020-08-07T15:07:57.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:57.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.643083+0000", 2020-08-07T15:07:57.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7172e-05 2020-08-07T15:07:57.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:57.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.643150+0000", 2020-08-07T15:07:57.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002661067 2020-08-07T15:07:57.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:57.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.645811+0000", 2020-08-07T15:07:57.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015136508 2020-08-07T15:07:57.781 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.781 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:57.782 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.782 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:117763 3.3 3:c6e35c65:::benchmark_data_smithi074_35447_object14720:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e65)", 2020-08-07T15:07:57.783 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:57.658227+0000", 2020-08-07T15:07:57.783 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0043734029999999997, 2020-08-07T15:07:57.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045869309999999998, 2020-08-07T15:07:57.783 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:57.783 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:07:57.784 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:57.784 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:57.784 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:57.785 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 117763 2020-08-07T15:07:57.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.785 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:57.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:57.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.658227+0000", 2020-08-07T15:07:57.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:57.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:57.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.658227+0000", 2020-08-07T15:07:57.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6618e-05 2020-08-07T15:07:57.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:57.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.658273+0000", 2020-08-07T15:07:57.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9469999999999996e-06 2020-08-07T15:07:57.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:57.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.658277+0000", 2020-08-07T15:07:57.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7449999999999995e-06 2020-08-07T15:07:57.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:57.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.658285+0000", 2020-08-07T15:07:57.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9389999999999999e-06 2020-08-07T15:07:57.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:57.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.658289+0000", 2020-08-07T15:07:57.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5395999999999995e-05 2020-08-07T15:07:57.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:57.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.658354+0000", 2020-08-07T15:07:57.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2640999999999996e-05 2020-08-07T15:07:57.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:57.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.658447+0000", 2020-08-07T15:07:57.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000178355 2020-08-07T15:07:57.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,6", 2020-08-07T15:07:57.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.658625+0000", 2020-08-07T15:07:57.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024385489999999999 2020-08-07T15:07:57.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:57.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.661064+0000", 2020-08-07T15:07:57.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000417706 2020-08-07T15:07:57.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:57.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.661482+0000", 2020-08-07T15:07:57.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032548960000000002 2020-08-07T15:07:57.804 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.814 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:57.815 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.815 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:117764 3.3 3:c6e35c65:::benchmark_data_smithi074_35447_object14720:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e65)", 2020-08-07T15:07:57.815 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:57.659114+0000", 2020-08-07T15:07:57.816 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0034861559999999998, 2020-08-07T15:07:57.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037487610000000002, 2020-08-07T15:07:57.816 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:57.816 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:07:57.816 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:57.817 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:57.817 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:57.817 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 117764 2020-08-07T15:07:57.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.817 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:57.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:57.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.659114+0000", 2020-08-07T15:07:57.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:57.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:57.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.659114+0000", 2020-08-07T15:07:57.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8432e-05 2020-08-07T15:07:57.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:57.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.659162+0000", 2020-08-07T15:07:57.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3409999999999997e-06 2020-08-07T15:07:57.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:57.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.659170+0000", 2020-08-07T15:07:57.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2858e-05 2020-08-07T15:07:57.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:57.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.659183+0000", 2020-08-07T15:07:57.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6219999999999997e-06 2020-08-07T15:07:57.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:57.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.659188+0000", 2020-08-07T15:07:57.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2818000000000002e-05 2020-08-07T15:07:57.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:57.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.659241+0000", 2020-08-07T15:07:57.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5507999999999994e-05 2020-08-07T15:07:57.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:57.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.659336+0000", 2020-08-07T15:07:57.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016968199999999999 2020-08-07T15:07:57.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,6", 2020-08-07T15:07:57.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.659506+0000", 2020-08-07T15:07:57.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039226099999999998 2020-08-07T15:07:57.847 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.847 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:57.847 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.848 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:117756 3.e e65/20 3:7e187785:::benchmark_data_smithi074_35447_object14719:head v 65'7516, mlcod=65'7506)", 2020-08-07T15:07:57.848 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:57.658860+0000", 2020-08-07T15:07:57.848 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0037402759999999998, 2020-08-07T15:07:57.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040453420000000004, 2020-08-07T15:07:57.849 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:57.849 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:57.849 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:57.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:57.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.658860+0000", 2020-08-07T15:07:57.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:57.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:57.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.658860+0000", 2020-08-07T15:07:57.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1445e-05 2020-08-07T15:07:57.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:57.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.658911+0000", 2020-08-07T15:07:57.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4350000000000002e-06 2020-08-07T15:07:57.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:57.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.658918+0000", 2020-08-07T15:07:57.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029560900000000001 2020-08-07T15:07:57.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:57.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.659213+0000", 2020-08-07T15:07:57.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1480000000000003e-06 2020-08-07T15:07:57.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:57.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.659219+0000", 2020-08-07T15:07:57.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5108999999999998e-05 2020-08-07T15:07:57.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:57.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.659265+0000", 2020-08-07T15:07:57.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5746999999999999e-05 2020-08-07T15:07:57.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:57.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.659290+0000", 2020-08-07T15:07:57.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000430493 2020-08-07T15:07:57.857 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.857 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:57.858 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.858 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:117757 3.e e65/20 3:7e187785:::benchmark_data_smithi074_35447_object14719:head v 65'7517, mlcod=65'7511)", 2020-08-07T15:07:57.858 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:57.658951+0000", 2020-08-07T15:07:57.859 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0036495059999999998, 2020-08-07T15:07:57.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003988246, 2020-08-07T15:07:57.859 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:57.859 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:57.859 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:57.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:57.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.658951+0000", 2020-08-07T15:07:57.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:57.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:57.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.658951+0000", 2020-08-07T15:07:57.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2686999999999999e-05 2020-08-07T15:07:57.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:57.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.659003+0000", 2020-08-07T15:07:57.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.772e-06 2020-08-07T15:07:57.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:57.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.659009+0000", 2020-08-07T15:07:57.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000226018 2020-08-07T15:07:57.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:57.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.659235+0000", 2020-08-07T15:07:57.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.634e-06 2020-08-07T15:07:57.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:57.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.659240+0000", 2020-08-07T15:07:57.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000344481 2020-08-07T15:07:57.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:57.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.659584+0000", 2020-08-07T15:07:57.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1974e-05 2020-08-07T15:07:57.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:57.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.659606+0000", 2020-08-07T15:07:57.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000655566 2020-08-07T15:07:57.868 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.868 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:57.868 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.869 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:117758 3.e e65/20 3:7e187785:::benchmark_data_smithi074_35447_object14719:head v 65'7518, mlcod=65'7511)", 2020-08-07T15:07:57.869 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:57.661267+0000", 2020-08-07T15:07:57.869 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.001332661, 2020-08-07T15:07:57.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001706167, 2020-08-07T15:07:57.870 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:57.870 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:57.870 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:57.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:57.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.661267+0000", 2020-08-07T15:07:57.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:57.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:57.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.661267+0000", 2020-08-07T15:07:57.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5783e-05 2020-08-07T15:07:57.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:57.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.661293+0000", 2020-08-07T15:07:57.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0369999999999999e-06 2020-08-07T15:07:57.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:57.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.661296+0000", 2020-08-07T15:07:57.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014535500000000001 2020-08-07T15:07:57.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:57.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.661442+0000", 2020-08-07T15:07:57.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2689999999999998e-06 2020-08-07T15:07:57.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:57.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.661444+0000", 2020-08-07T15:07:57.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5562999999999997e-05 2020-08-07T15:07:57.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:57.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.661489+0000", 2020-08-07T15:07:57.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2929999999999999e-05 2020-08-07T15:07:57.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:57.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.661512+0000", 2020-08-07T15:07:57.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024493699999999998 2020-08-07T15:07:57.897 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.897 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:57.898 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.898 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:117759 3.e e65/20 3:7e187785:::benchmark_data_smithi074_35447_object14719:head v 65'7519, mlcod=65'7511)", 2020-08-07T15:07:57.898 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:57.661311+0000", 2020-08-07T15:07:57.899 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.001289185, 2020-08-07T15:07:57.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016995249999999999, 2020-08-07T15:07:57.899 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:57.899 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:57.899 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:57.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:57.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.661311+0000", 2020-08-07T15:07:57.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:57.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:57.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.661311+0000", 2020-08-07T15:07:57.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7842999999999997e-05 2020-08-07T15:07:57.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:57.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.661349+0000", 2020-08-07T15:07:57.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0509999999999996e-06 2020-08-07T15:07:57.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:57.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.661354+0000", 2020-08-07T15:07:57.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104079 2020-08-07T15:07:57.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:57.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.661458+0000", 2020-08-07T15:07:57.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7810000000000001e-06 2020-08-07T15:07:57.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:57.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.661461+0000", 2020-08-07T15:07:57.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034489900000000002 2020-08-07T15:07:57.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:57.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.661806+0000", 2020-08-07T15:07:57.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5821e-05 2020-08-07T15:07:57.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:57.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.661821+0000", 2020-08-07T15:07:57.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051047400000000002 2020-08-07T15:07:57.908 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.908 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:57.909 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.909 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:117760 3.e e65/20 3:7e187785:::benchmark_data_smithi074_35447_object14719:head v 65'7520, mlcod=65'7511)", 2020-08-07T15:07:57.909 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:57.661381+0000", 2020-08-07T15:07:57.910 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.001219386, 2020-08-07T15:07:57.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001663005, 2020-08-07T15:07:57.910 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:57.910 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:57.910 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:57.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:57.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.661381+0000", 2020-08-07T15:07:57.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:57.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:57.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.661381+0000", 2020-08-07T15:07:57.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7253000000000001e-05 2020-08-07T15:07:57.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:57.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.661408+0000", 2020-08-07T15:07:57.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3079999999999999e-06 2020-08-07T15:07:57.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:57.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.661411+0000", 2020-08-07T15:07:57.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4986000000000003e-05 2020-08-07T15:07:57.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:57.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.661486+0000", 2020-08-07T15:07:57.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5820000000000002e-06 2020-08-07T15:07:57.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:57.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.661490+0000", 2020-08-07T15:07:57.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00054236099999999999 2020-08-07T15:07:57.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:57.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.662032+0000", 2020-08-07T15:07:57.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4989000000000001e-05 2020-08-07T15:07:57.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:57.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.662057+0000", 2020-08-07T15:07:57.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067647900000000001 2020-08-07T15:07:57.918 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.918 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:57.919 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.921 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:117746 3.6 e65/26 3:6b1ad806:::benchmark_data_smithi074_35447_object14718:head v 65'7714, mlcod=65'7711)", 2020-08-07T15:07:57.922 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:57.650139+0000", 2020-08-07T15:07:57.922 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.012461539000000001, 2020-08-07T15:07:57.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012945185, 2020-08-07T15:07:57.922 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:57.922 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:57.923 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:57.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:57.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.650139+0000", 2020-08-07T15:07:57.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:57.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:57.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.650139+0000", 2020-08-07T15:07:57.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7783e-05 2020-08-07T15:07:57.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:57.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.650186+0000", 2020-08-07T15:07:57.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9599999999999997e-06 2020-08-07T15:07:57.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:57.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.650192+0000", 2020-08-07T15:07:57.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025988259999999998 2020-08-07T15:07:57.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:57.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.652791+0000", 2020-08-07T15:07:57.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2859999999999999e-06 2020-08-07T15:07:57.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:57.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.652796+0000", 2020-08-07T15:07:57.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2821000000000002e-05 2020-08-07T15:07:57.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:57.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.652839+0000", 2020-08-07T15:07:57.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2949999999999999e-05 2020-08-07T15:07:57.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:57.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.652862+0000", 2020-08-07T15:07:57.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027236259999999998 2020-08-07T15:07:57.941 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.942 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:57.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.942 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:117747 3.6 e65/26 3:6b1ad806:::benchmark_data_smithi074_35447_object14718:head v 65'7715, mlcod=65'7711)", 2020-08-07T15:07:57.943 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:57.651200+0000", 2020-08-07T15:07:57.943 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.011399759000000001, 2020-08-07T15:07:57.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011917308, 2020-08-07T15:07:57.943 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:57.943 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:57.944 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:57.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:57.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.651200+0000", 2020-08-07T15:07:57.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:57.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:57.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.651200+0000", 2020-08-07T15:07:57.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1656999999999998e-05 2020-08-07T15:07:57.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:57.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.651242+0000", 2020-08-07T15:07:57.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9750000000000001e-06 2020-08-07T15:07:57.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:57.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.651246+0000", 2020-08-07T15:07:57.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015660699999999999 2020-08-07T15:07:57.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:57.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.652812+0000", 2020-08-07T15:07:57.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3220000000000001e-06 2020-08-07T15:07:57.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:57.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.652815+0000", 2020-08-07T15:07:57.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023654790000000002 2020-08-07T15:07:57.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:57.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.655181+0000", 2020-08-07T15:07:57.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2169000000000001e-05 2020-08-07T15:07:57.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:57.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.655203+0000", 2020-08-07T15:07:57.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040026719999999997 2020-08-07T15:07:57.953 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.953 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:57.953 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.954 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:117748 3.6 e65/26 3:6b1ad806:::benchmark_data_smithi074_35447_object14718:head v 65'7716, mlcod=65'7711)", 2020-08-07T15:07:57.954 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:57.652507+0000", 2020-08-07T15:07:57.954 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.010092568, 2020-08-07T15:07:57.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010657975, 2020-08-07T15:07:57.955 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:57.955 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:57.955 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:57.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:57.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.652507+0000", 2020-08-07T15:07:57.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:57.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:57.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.652507+0000", 2020-08-07T15:07:57.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2618999999999999e-05 2020-08-07T15:07:57.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:57.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.652560+0000", 2020-08-07T15:07:57.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1079999999999998e-06 2020-08-07T15:07:57.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:57.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.652566+0000", 2020-08-07T15:07:57.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000843985 2020-08-07T15:07:57.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:57.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.653410+0000", 2020-08-07T15:07:57.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.619e-06 2020-08-07T15:07:57.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:57.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.653414+0000", 2020-08-07T15:07:57.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020827160000000001 2020-08-07T15:07:57.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:57.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.655497+0000", 2020-08-07T15:07:57.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0863999999999999e-05 2020-08-07T15:07:57.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:57.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.655517+0000", 2020-08-07T15:07:57.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030099110000000001 2020-08-07T15:07:57.964 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.964 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:57.965 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.972 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:117749 3.6 e65/26 3:6b1ad806:::benchmark_data_smithi074_35447_object14718:head v 65'7717, mlcod=65'7711)", 2020-08-07T15:07:57.982 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:57.652594+0000", 2020-08-07T15:07:57.982 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.010005712, 2020-08-07T15:07:57.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010614037, 2020-08-07T15:07:57.983 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:57.983 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:57.983 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:57.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:57.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.652594+0000", 2020-08-07T15:07:57.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:57.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:57.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.652594+0000", 2020-08-07T15:07:57.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2981000000000001e-05 2020-08-07T15:07:57.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:57.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.652637+0000", 2020-08-07T15:07:57.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4790000000000004e-06 2020-08-07T15:07:57.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:57.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.652643+0000", 2020-08-07T15:07:57.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012326349999999999 2020-08-07T15:07:57.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:57.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.653875+0000", 2020-08-07T15:07:57.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6480000000000001e-06 2020-08-07T15:07:57.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:57.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.653879+0000", 2020-08-07T15:07:57.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001947839 2020-08-07T15:07:57.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:57.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.655827+0000", 2020-08-07T15:07:57.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1671e-05 2020-08-07T15:07:57.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:57.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.655849+0000", 2020-08-07T15:07:57.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032542529999999999 2020-08-07T15:07:57.996 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.996 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:57.996 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:57.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:57.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.997 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:117750 3.6 e65/26 3:6b1ad806:::benchmark_data_smithi074_35447_object14718:head v 65'7718, mlcod=65'7711)", 2020-08-07T15:07:57.997 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:57.653353+0000", 2020-08-07T15:07:57.997 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0092474489999999996, 2020-08-07T15:07:57.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0098897829999999992, 2020-08-07T15:07:57.998 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:57.998 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:57.998 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:57.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:57.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:57.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.653353+0000", 2020-08-07T15:07:57.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:58.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:58.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.653353+0000", 2020-08-07T15:07:58.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7920999999999998e-05 2020-08-07T15:07:58.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:58.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.653391+0000", 2020-08-07T15:07:58.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2339999999999996e-06 2020-08-07T15:07:58.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:58.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.653395+0000", 2020-08-07T15:07:58.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011137709999999999 2020-08-07T15:07:58.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:58.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.654509+0000", 2020-08-07T15:07:58.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1619999999999999e-06 2020-08-07T15:07:58.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:58.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.654514+0000", 2020-08-07T15:07:58.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016517820000000001 2020-08-07T15:07:58.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:58.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656165+0000", 2020-08-07T15:07:58.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4341e-05 2020-08-07T15:07:58.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:58.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656190+0000", 2020-08-07T15:07:58.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028372110000000001 2020-08-07T15:07:58.014 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.014 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:58.015 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.016 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:117761 3.3 3:c6e35c65:::benchmark_data_smithi074_35447_object14720:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e65)", 2020-08-07T15:07:58.016 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:57.655901+0000", 2020-08-07T15:07:58.016 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0066994059999999998, 2020-08-07T15:07:58.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0073785229999999997, 2020-08-07T15:07:58.017 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:58.017 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:07:58.018 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:58.019 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:58.019 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:58.019 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 117761 2020-08-07T15:07:58.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.020 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:58.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:58.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.655901+0000", 2020-08-07T15:07:58.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:58.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:58.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.655901+0000", 2020-08-07T15:07:58.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8451000000000001e-05 2020-08-07T15:07:58.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:58.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.655949+0000", 2020-08-07T15:07:58.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2900000000000002e-06 2020-08-07T15:07:58.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:58.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.655954+0000", 2020-08-07T15:07:58.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2028000000000001e-05 2020-08-07T15:07:58.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:58.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.655966+0000", 2020-08-07T15:07:58.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0929999999999999e-06 2020-08-07T15:07:58.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:58.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.655973+0000", 2020-08-07T15:07:58.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1567999999999998e-05 2020-08-07T15:07:58.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:58.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656024+0000", 2020-08-07T15:07:58.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016097999999999999 2020-08-07T15:07:58.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:58.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656185+0000", 2020-08-07T15:07:58.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028485799999999998 2020-08-07T15:07:58.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,6", 2020-08-07T15:07:58.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656470+0000", 2020-08-07T15:07:58.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00157339 2020-08-07T15:07:58.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:58.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.658043+0000", 2020-08-07T15:07:58.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029562519999999999 2020-08-07T15:07:58.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:58.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.661000+0000", 2020-08-07T15:07:58.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050989099999999999 2020-08-07T15:07:58.038 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.038 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:58.038 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.039 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:117762 3.3 3:c6e35c65:::benchmark_data_smithi074_35447_object14720:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e65)", 2020-08-07T15:07:58.040 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:57.656028+0000", 2020-08-07T15:07:58.040 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0065720249999999996, 2020-08-07T15:07:58.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.007301213, 2020-08-07T15:07:58.040 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:58.040 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:07:58.041 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:58.041 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:58.041 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:58.041 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 117762 2020-08-07T15:07:58.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.042 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:58.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:58.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656028+0000", 2020-08-07T15:07:58.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:58.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:58.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656028+0000", 2020-08-07T15:07:58.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2446000000000005e-05 2020-08-07T15:07:58.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:58.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656090+0000", 2020-08-07T15:07:58.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4420000000000003e-06 2020-08-07T15:07:58.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:58.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656097+0000", 2020-08-07T15:07:58.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3118999999999999e-05 2020-08-07T15:07:58.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:58.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656110+0000", 2020-08-07T15:07:58.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5150000000000003e-06 2020-08-07T15:07:58.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:58.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656117+0000", 2020-08-07T15:07:58.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00079630499999999997 2020-08-07T15:07:58.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:58.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656913+0000", 2020-08-07T15:07:58.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9740999999999993e-05 2020-08-07T15:07:58.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:58.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.657003+0000", 2020-08-07T15:07:58.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017715 2020-08-07T15:07:58.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,6", 2020-08-07T15:07:58.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.657180+0000", 2020-08-07T15:07:58.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018871339999999999 2020-08-07T15:07:58.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:58.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.659067+0000", 2020-08-07T15:07:58.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019723150000000001 2020-08-07T15:07:58.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:58.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.661039+0000", 2020-08-07T15:07:58.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050111670000000004 2020-08-07T15:07:58.053 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.053 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:58.053 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.054 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:117753 3.e e65/20 3:7e187785:::benchmark_data_smithi074_35447_object14719:head v 65'7513, mlcod=65'7506)", 2020-08-07T15:07:58.054 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:57.655894+0000", 2020-08-07T15:07:58.054 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0067065199999999997, 2020-08-07T15:07:58.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.007483684, 2020-08-07T15:07:58.055 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:58.055 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:58.055 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:58.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:58.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.655894+0000", 2020-08-07T15:07:58.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:58.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:58.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.655894+0000", 2020-08-07T15:07:58.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3644999999999999e-05 2020-08-07T15:07:58.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:58.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.655947+0000", 2020-08-07T15:07:58.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4989999999999999e-06 2020-08-07T15:07:58.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:58.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.655954+0000", 2020-08-07T15:07:58.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023431099999999999 2020-08-07T15:07:58.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:58.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656188+0000", 2020-08-07T15:07:58.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7479999999999999e-06 2020-08-07T15:07:58.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:58.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656194+0000", 2020-08-07T15:07:58.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2363999999999997e-05 2020-08-07T15:07:58.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:58.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656226+0000", 2020-08-07T15:07:58.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1543999999999999e-05 2020-08-07T15:07:58.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:57.656248+0000", 2020-08-07T15:07:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000354111 2020-08-07T15:07:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.066 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:58.066 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.066 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.066 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:07:58.066 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 19 2020-08-07T15:07:58.067 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:58.068 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:07:58.069 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:07:58.067+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:07:58.156 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:07:58.157 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:07:58.157 INFO:tasks.thrashosds.thrasher:inject_pause on 7 2020-08-07T15:07:58.157 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2020-08-07T15:07:58.158 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2020-08-07T15:07:58.158 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2020-08-07T15:07:58.171 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:58.170+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:58.202 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:07:58.250 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 82 16 118677 118661 11.3044 12.1172 0.00761337 0.0110533 2020-08-07T15:07:58.275 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:58.274+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:58.302 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:58.303 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:07:58.303 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:07:58.303 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:07:58.303 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:58.320 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:07:58.372 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:58.373 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "" 2020-08-07T15:07:58.373 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:58.392 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:07:58.430 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:58.425+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:58.444 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:07:58.488 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:58.486+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:58.490 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:07:58.523 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:07:58.553 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:07:58.554 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:07:58.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:07:58.554 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:07:58.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.555 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:8112 3.9 3:9b275d48:::benchmark_data_smithi074_35447_object1013:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:07:58.555 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.083981+0000", 2020-08-07T15:07:58.555 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.451272949, 2020-08-07T15:07:58.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31911657999999998, 2020-08-07T15:07:58.555 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:58.556 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:58.560 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:58.560 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:58.561 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:58.561 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 8112 2020-08-07T15:07:58.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.561 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:58.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:58.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.083981+0000", 2020-08-07T15:07:58.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:58.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:58.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.083981+0000", 2020-08-07T15:07:58.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4051999999999999e-05 2020-08-07T15:07:58.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:58.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084015+0000", 2020-08-07T15:07:58.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7290000000000002e-06 2020-08-07T15:07:58.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:58.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084019+0000", 2020-08-07T15:07:58.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2280000000000001e-06 2020-08-07T15:07:58.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:58.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084026+0000", 2020-08-07T15:07:58.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7780000000000001e-06 2020-08-07T15:07:58.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:58.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084030+0000", 2020-08-07T15:07:58.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00077372300000000003 2020-08-07T15:07:58.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:58.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084803+0000", 2020-08-07T15:07:58.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1502000000000003e-05 2020-08-07T15:07:58.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:58.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084855+0000", 2020-08-07T15:07:58.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5110000000000002e-05 2020-08-07T15:07:58.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:58.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084950+0000", 2020-08-07T15:07:58.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012531370000000001 2020-08-07T15:07:58.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:58.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.086203+0000", 2020-08-07T15:07:58.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4799999999999998e-07 2020-08-07T15:07:58.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:58.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.086204+0000", 2020-08-07T15:07:58.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.905e-05 2020-08-07T15:07:58.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:58.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.086253+0000", 2020-08-07T15:07:58.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00062940999999999995 2020-08-07T15:07:58.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:58.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.086882+0000", 2020-08-07T15:07:58.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31612944199999998 2020-08-07T15:07:58.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:58.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.403012+0000", 2020-08-07T15:07:58.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9099999999999991e-07 2020-08-07T15:07:58.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:58.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.403013+0000", 2020-08-07T15:07:58.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.488e-05 2020-08-07T15:07:58.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:58.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.403097+0000", 2020-08-07T15:07:58.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31911657999999998 2020-08-07T15:07:58.620 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.621 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:58.621 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.621 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:16223 3.9 3:9c72cb85:::benchmark_data_smithi074_35447_object2027:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:58.622 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:47.106937+0000", 2020-08-07T15:07:58.622 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 71.428316343000006, 2020-08-07T15:07:58.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.48645846999999998, 2020-08-07T15:07:58.623 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:58.623 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:58.623 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:58.624 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:58.624 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:58.624 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 16223 2020-08-07T15:07:58.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.624 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:58.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:58.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.106937+0000", 2020-08-07T15:07:58.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:58.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:58.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.106937+0000", 2020-08-07T15:07:58.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8390999999999998e-05 2020-08-07T15:07:58.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:58.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.106986+0000", 2020-08-07T15:07:58.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0120000000000002e-06 2020-08-07T15:07:58.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:58.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.106992+0000", 2020-08-07T15:07:58.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3148e-05 2020-08-07T15:07:58.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:58.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.107005+0000", 2020-08-07T15:07:58.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5719999999999999e-06 2020-08-07T15:07:58.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:58.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.107012+0000", 2020-08-07T15:07:58.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043495110000000003 2020-08-07T15:07:58.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:58.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.111361+0000", 2020-08-07T15:07:58.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0105999999999997e-05 2020-08-07T15:07:58.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:58.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.111421+0000", 2020-08-07T15:07:58.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012232399999999999 2020-08-07T15:07:58.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:58.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.111544+0000", 2020-08-07T15:07:58.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003099367 2020-08-07T15:07:58.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:58.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.114643+0000", 2020-08-07T15:07:58.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3099999999999997e-07 2020-08-07T15:07:58.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:58.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.114644+0000", 2020-08-07T15:07:58.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7627000000000003e-05 2020-08-07T15:07:58.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:58.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.114691+0000", 2020-08-07T15:07:58.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005765905 2020-08-07T15:07:58.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:58.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.120457+0000", 2020-08-07T15:07:58.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.47285765200000002 2020-08-07T15:07:58.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:58.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.593315+0000", 2020-08-07T15:07:58.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1969999999999999e-06 2020-08-07T15:07:58.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:58.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.593316+0000", 2020-08-07T15:07:58.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0026999999999998e-05 2020-08-07T15:07:58.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:58.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.593396+0000", 2020-08-07T15:07:58.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.48645846999999998 2020-08-07T15:07:58.715 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.715 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:58.715 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.715 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:16224 3.9 3:9c72cb85:::benchmark_data_smithi074_35447_object2027:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:07:58.716 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:47.108015+0000", 2020-08-07T15:07:58.716 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 71.427238606000003, 2020-08-07T15:07:58.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.512944332, 2020-08-07T15:07:58.716 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:58.716 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:58.717 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:58.717 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:58.717 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:58.718 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 16224 2020-08-07T15:07:58.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.718 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:58.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:58.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108015+0000", 2020-08-07T15:07:58.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:58.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:58.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108015+0000", 2020-08-07T15:07:58.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7508e-05 2020-08-07T15:07:58.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:58.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108063+0000", 2020-08-07T15:07:58.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.485e-06 2020-08-07T15:07:58.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:58.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108068+0000", 2020-08-07T15:07:58.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3387000000000001e-05 2020-08-07T15:07:58.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:58.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108082+0000", 2020-08-07T15:07:58.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1129999999999997e-06 2020-08-07T15:07:58.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:58.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108088+0000", 2020-08-07T15:07:58.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056051920000000002 2020-08-07T15:07:58.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:58.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.113693+0000", 2020-08-07T15:07:58.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9973000000000001e-05 2020-08-07T15:07:58.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:58.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.113783+0000", 2020-08-07T15:07:58.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001649 2020-08-07T15:07:58.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:58.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.113948+0000", 2020-08-07T15:07:58.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001023007 2020-08-07T15:07:58.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:58.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.114971+0000", 2020-08-07T15:07:58.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2600000000000001e-07 2020-08-07T15:07:58.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:58.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.114972+0000", 2020-08-07T15:07:58.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9228999999999998e-05 2020-08-07T15:07:58.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:58.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.115031+0000", 2020-08-07T15:07:58.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0063345550000000004 2020-08-07T15:07:58.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:58.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.121365+0000", 2020-08-07T15:07:58.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.49952866099999998 2020-08-07T15:07:58.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:58.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.620894+0000", 2020-08-07T15:07:58.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9299999999999996e-07 2020-08-07T15:07:58.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:58.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.620895+0000", 2020-08-07T15:07:58.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4503000000000002e-05 2020-08-07T15:07:58.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:58.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.620960+0000", 2020-08-07T15:07:58.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.512944332 2020-08-07T15:07:58.749 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.749 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:58.749 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.750 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:33728 3.1 3:8fce2265:::benchmark_data_smithi074_35447_object4215:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-08-07T15:07:58.750 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:58.156056+0000", 2020-08-07T15:07:58.750 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.379198205000002, 2020-08-07T15:07:58.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26861316299999999, 2020-08-07T15:07:58.751 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:58.751 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:58.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:58.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:58.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:58.752 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 33728 2020-08-07T15:07:58.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.752 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:58.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:58.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156056+0000", 2020-08-07T15:07:58.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:58.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:58.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156056+0000", 2020-08-07T15:07:58.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6795999999999998e-05 2020-08-07T15:07:58.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:58.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156092+0000", 2020-08-07T15:07:58.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4690000000000002e-06 2020-08-07T15:07:58.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:58.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156096+0000", 2020-08-07T15:07:58.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5270000000000004e-06 2020-08-07T15:07:58.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:58.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156102+0000", 2020-08-07T15:07:58.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3129999999999999e-06 2020-08-07T15:07:58.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:58.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156106+0000", 2020-08-07T15:07:58.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00337032 2020-08-07T15:07:58.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:58.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.159476+0000", 2020-08-07T15:07:58.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104874 2020-08-07T15:07:58.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:58.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.159581+0000", 2020-08-07T15:07:58.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016822300000000001 2020-08-07T15:07:58.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,3", 2020-08-07T15:07:58.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.159749+0000", 2020-08-07T15:07:58.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020156620000000001 2020-08-07T15:07:58.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:58.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.161765+0000", 2020-08-07T15:07:58.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1699999999999997e-07 2020-08-07T15:07:58.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:58.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.161765+0000", 2020-08-07T15:07:58.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6606e-05 2020-08-07T15:07:58.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:58.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.161822+0000", 2020-08-07T15:07:58.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0060344459999999997 2020-08-07T15:07:58.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:58.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.167857+0000", 2020-08-07T15:07:58.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25676037099999999 2020-08-07T15:07:58.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:58.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424617+0000", 2020-08-07T15:07:58.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3370000000000001e-06 2020-08-07T15:07:58.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:58.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424618+0000", 2020-08-07T15:07:58.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0501999999999999e-05 2020-08-07T15:07:58.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:58.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424669+0000", 2020-08-07T15:07:58.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26861316299999999 2020-08-07T15:07:58.780 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.780 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:58.780 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.781 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:34624 3.9 3:946cdb13:::benchmark_data_smithi074_35447_object4327:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-08-07T15:07:58.781 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:58.744745+0000", 2020-08-07T15:07:58.782 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 59.790509086, 2020-08-07T15:07:58.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28897188600000001, 2020-08-07T15:07:58.782 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:58.782 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:58.782 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:58.782 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:58.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:58.783 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 34624 2020-08-07T15:07:58.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.783 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:58.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:58.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744745+0000", 2020-08-07T15:07:58.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:58.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:58.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744745+0000", 2020-08-07T15:07:58.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1117e-05 2020-08-07T15:07:58.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:58.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744776+0000", 2020-08-07T15:07:58.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2140000000000001e-06 2020-08-07T15:07:58.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:58.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744782+0000", 2020-08-07T15:07:58.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1756999999999999e-05 2020-08-07T15:07:58.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:58.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744794+0000", 2020-08-07T15:07:58.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.674e-06 2020-08-07T15:07:58.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:58.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744800+0000", 2020-08-07T15:07:58.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012818110000000001 2020-08-07T15:07:58.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:58.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.746082+0000", 2020-08-07T15:07:58.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2163999999999997e-05 2020-08-07T15:07:58.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:58.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.746134+0000", 2020-08-07T15:07:58.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107853 2020-08-07T15:07:58.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:58.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.746242+0000", 2020-08-07T15:07:58.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00119732 2020-08-07T15:07:58.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:58.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.747440+0000", 2020-08-07T15:07:58.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5799999999999998e-07 2020-08-07T15:07:58.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:58.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.747440+0000", 2020-08-07T15:07:58.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0883000000000001e-05 2020-08-07T15:07:58.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:58.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.747481+0000", 2020-08-07T15:07:58.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000272355 2020-08-07T15:07:58.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:58.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.747754+0000", 2020-08-07T15:07:58.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28589292999999999 2020-08-07T15:07:58.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:58.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.033647+0000", 2020-08-07T15:07:58.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6600000000000005e-07 2020-08-07T15:07:58.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:58.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.033647+0000", 2020-08-07T15:07:58.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9183999999999994e-05 2020-08-07T15:07:58.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:58.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.033717+0000", 2020-08-07T15:07:58.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28897188600000001 2020-08-07T15:07:58.844 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.844 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:58.844 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.845 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:64656 3.9 3:92365776:::benchmark_data_smithi074_35447_object8081:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e41)", 2020-08-07T15:07:58.845 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:20.118422+0000", 2020-08-07T15:07:58.846 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 38.416832245999998, 2020-08-07T15:07:58.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27269932899999999, 2020-08-07T15:07:58.846 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:58.846 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:58.847 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:58.847 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:58.847 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:58.847 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 64656 2020-08-07T15:07:58.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.847 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:58.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:58.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.118422+0000", 2020-08-07T15:07:58.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:58.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:58.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.118422+0000", 2020-08-07T15:07:58.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9726e-05 2020-08-07T15:07:58.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:58.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.118451+0000", 2020-08-07T15:07:58.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3759999999999999e-06 2020-08-07T15:07:58.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:58.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.118457+0000", 2020-08-07T15:07:58.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2139999999999996e-06 2020-08-07T15:07:58.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:58.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.118465+0000", 2020-08-07T15:07:58.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2420000000000002e-06 2020-08-07T15:07:58.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:58.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.118469+0000", 2020-08-07T15:07:58.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031527040000000001 2020-08-07T15:07:58.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:58.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.121622+0000", 2020-08-07T15:07:58.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4674999999999998e-05 2020-08-07T15:07:58.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:58.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.121706+0000", 2020-08-07T15:07:58.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000185775 2020-08-07T15:07:58.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:58.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.121892+0000", 2020-08-07T15:07:58.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003359384 2020-08-07T15:07:58.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:58.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.125252+0000", 2020-08-07T15:07:58.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2099999999999995e-07 2020-08-07T15:07:58.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:58.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.125252+0000", 2020-08-07T15:07:58.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4799999999999997e-05 2020-08-07T15:07:58.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:58.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.125307+0000", 2020-08-07T15:07:58.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3406000000000003e-05 2020-08-07T15:07:58.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:58.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.125361+0000", 2020-08-07T15:07:58.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26572362599999999 2020-08-07T15:07:58.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:58.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.391084+0000", 2020-08-07T15:07:58.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6899999999999996e-07 2020-08-07T15:07:58.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:58.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.391085+0000", 2020-08-07T15:07:58.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5811e-05 2020-08-07T15:07:58.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:58.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.391121+0000", 2020-08-07T15:07:58.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27269932899999999 2020-08-07T15:07:58.863 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.863 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:58.863 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.864 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:64656 3.9 e41/20 ondisk, result = 0)", 2020-08-07T15:07:58.864 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:20.122964+0000", 2020-08-07T15:07:58.864 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 38.412290184, 2020-08-07T15:07:58.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26816000699999998, 2020-08-07T15:07:58.865 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:58.865 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:58.865 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:58.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:58.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.122964+0000", 2020-08-07T15:07:58.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:58.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:58.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.122964+0000", 2020-08-07T15:07:58.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8997999999999999e-05 2020-08-07T15:07:58.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:58.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.122993+0000", 2020-08-07T15:07:58.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.395e-06 2020-08-07T15:07:58.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:58.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.122998+0000", 2020-08-07T15:07:58.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.266724181 2020-08-07T15:07:58.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:58.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.389722+0000", 2020-08-07T15:07:58.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7139999999999999e-06 2020-08-07T15:07:58.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:58.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.389726+0000", 2020-08-07T15:07:58.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001342408 2020-08-07T15:07:58.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:58.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.391068+0000", 2020-08-07T15:07:58.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0597e-05 2020-08-07T15:07:58.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:58.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.391079+0000", 2020-08-07T15:07:58.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4713999999999999e-05 2020-08-07T15:07:58.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:58.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.391124+0000", 2020-08-07T15:07:58.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26816000699999998 2020-08-07T15:07:58.874 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.874 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:58.874 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.883 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:72616 3.6 e44/26 3:64e83676:::benchmark_data_smithi074_35447_object9076:head v 44'4848, mlcod=44'4843)", 2020-08-07T15:07:58.884 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:25.249399+0000", 2020-08-07T15:07:58.884 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.285854927999999, 2020-08-07T15:07:58.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27428339600000001, 2020-08-07T15:07:58.893 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:58.893 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:58.894 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:58.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:58.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.249399+0000", 2020-08-07T15:07:58.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:58.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:58.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.249399+0000", 2020-08-07T15:07:58.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0565000000000003e-05 2020-08-07T15:07:58.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:58.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.249429+0000", 2020-08-07T15:07:58.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3390000000000003e-06 2020-08-07T15:07:58.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:58.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.249434+0000", 2020-08-07T15:07:58.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27367070199999999 2020-08-07T15:07:58.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:58.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.523104+0000", 2020-08-07T15:07:58.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2100000000000001e-06 2020-08-07T15:07:58.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:58.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.523110+0000", 2020-08-07T15:07:58.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6372999999999997e-05 2020-08-07T15:07:58.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:58.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.523166+0000", 2020-08-07T15:07:58.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5656000000000001e-05 2020-08-07T15:07:58.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:58.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.523182+0000", 2020-08-07T15:07:58.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00046263499999999998 2020-08-07T15:07:58.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:58.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.523644+0000", 2020-08-07T15:07:58.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7916000000000001e-05 2020-08-07T15:07:58.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:58.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.523682+0000", 2020-08-07T15:07:58.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27428339600000001 2020-08-07T15:07:58.913 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.913 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:58.913 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.914 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:78743 3.6 e47/26 3:6151aa5d:::benchmark_data_smithi074_35447_object9842:head v 47'5303, mlcod=47'5297)", 2020-08-07T15:07:58.914 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:29.251886+0000", 2020-08-07T15:07:58.914 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.283367402, 2020-08-07T15:07:58.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.267414384, 2020-08-07T15:07:58.915 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:58.915 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:58.915 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:58.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:58.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.251886+0000", 2020-08-07T15:07:58.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:58.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:58.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.251886+0000", 2020-08-07T15:07:58.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8955e-05 2020-08-07T15:07:58.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:58.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.251915+0000", 2020-08-07T15:07:58.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2549999999999999e-06 2020-08-07T15:07:58.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:58.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.251920+0000", 2020-08-07T15:07:58.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26582320300000001 2020-08-07T15:07:58.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:58.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.517743+0000", 2020-08-07T15:07:58.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.585e-06 2020-08-07T15:07:58.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:58.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.517746+0000", 2020-08-07T15:07:58.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4413000000000003e-05 2020-08-07T15:07:58.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:58.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.517791+0000", 2020-08-07T15:07:58.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3614000000000001e-05 2020-08-07T15:07:58.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:58.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.517814+0000", 2020-08-07T15:07:58.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001433915 2020-08-07T15:07:58.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:58.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.519248+0000", 2020-08-07T15:07:58.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2444000000000001e-05 2020-08-07T15:07:58.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:58.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.519301+0000", 2020-08-07T15:07:58.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.267414384 2020-08-07T15:07:58.925 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.925 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:58.925 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.926 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:78744 3.6 e47/26 3:6151aa5d:::benchmark_data_smithi074_35447_object9842:head v 47'5304, mlcod=47'5297)", 2020-08-07T15:07:58.926 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:29.252471+0000", 2020-08-07T15:07:58.927 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.282782591, 2020-08-07T15:07:58.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26934327600000002, 2020-08-07T15:07:58.927 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:58.927 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:58.927 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:58.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:58.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.252471+0000", 2020-08-07T15:07:58.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:58.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:58.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.252471+0000", 2020-08-07T15:07:58.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2329e-05 2020-08-07T15:07:58.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:58.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.252524+0000", 2020-08-07T15:07:58.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0229999999999998e-06 2020-08-07T15:07:58.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:58.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.252530+0000", 2020-08-07T15:07:58.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26582335099999999 2020-08-07T15:07:58.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:58.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.518353+0000", 2020-08-07T15:07:58.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9689999999999999e-06 2020-08-07T15:07:58.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:58.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.518356+0000", 2020-08-07T15:07:58.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5710999999999997e-05 2020-08-07T15:07:58.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:58.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.518392+0000", 2020-08-07T15:07:58.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5387e-05 2020-08-07T15:07:58.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:58.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.518407+0000", 2020-08-07T15:07:58.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033617870000000002 2020-08-07T15:07:58.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:58.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.521769+0000", 2020-08-07T15:07:58.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5719000000000001e-05 2020-08-07T15:07:58.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:58.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.521814+0000", 2020-08-07T15:07:58.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26934327600000002 2020-08-07T15:07:58.950 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.950 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:58.950 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.951 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:86696 3.a e52/20 3:5058ba7c:::benchmark_data_smithi074_35447_object10836:head v 52'5704, mlcod=51'5695)", 2020-08-07T15:07:58.951 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.329685+0000", 2020-08-07T15:07:58.951 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.205569186000002, 2020-08-07T15:07:58.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.34216058999999999, 2020-08-07T15:07:58.952 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:58.952 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:58.952 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:58.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:58.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.329685+0000", 2020-08-07T15:07:58.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:58.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:58.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.329685+0000", 2020-08-07T15:07:58.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8568e-05 2020-08-07T15:07:58.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:58.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.329733+0000", 2020-08-07T15:07:58.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5509999999999999e-06 2020-08-07T15:07:58.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:58.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.329739+0000", 2020-08-07T15:07:58.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.337284427 2020-08-07T15:07:58.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:58.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.667023+0000", 2020-08-07T15:07:58.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7069999999999997e-06 2020-08-07T15:07:58.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:58.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.667029+0000", 2020-08-07T15:07:58.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5142999999999998e-05 2020-08-07T15:07:58.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:58.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.667074+0000", 2020-08-07T15:07:58.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7229000000000001e-05 2020-08-07T15:07:58.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:58.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.667091+0000", 2020-08-07T15:07:58.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046970060000000001 2020-08-07T15:07:58.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:58.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.671788+0000", 2020-08-07T15:07:58.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6959000000000001e-05 2020-08-07T15:07:58.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:58.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.671845+0000", 2020-08-07T15:07:58.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.34216058999999999 2020-08-07T15:07:58.961 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.962 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:58.962 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.962 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:104111 3.9 3:99770d45:::benchmark_data_smithi074_35447_object13013:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:07:58.963 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:45.858953+0000", 2020-08-07T15:07:58.963 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 12.676300939000001, 2020-08-07T15:07:58.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.29313854499999997, 2020-08-07T15:07:58.963 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:58.963 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:58.964 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:58.964 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:58.964 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:58.964 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 104111 2020-08-07T15:07:58.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.964 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:58.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:58.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858953+0000", 2020-08-07T15:07:58.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:58.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:58.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858953+0000", 2020-08-07T15:07:58.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8607000000000003e-05 2020-08-07T15:07:58.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:58.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.859011+0000", 2020-08-07T15:07:58.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9560000000000002e-06 2020-08-07T15:07:58.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:58.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.859017+0000", 2020-08-07T15:07:58.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2743e-05 2020-08-07T15:07:58.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:58.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.859030+0000", 2020-08-07T15:07:58.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2010000000000002e-06 2020-08-07T15:07:58.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:58.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.859037+0000", 2020-08-07T15:07:58.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001837827 2020-08-07T15:07:58.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:58.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.860875+0000", 2020-08-07T15:07:58.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2448999999999998e-05 2020-08-07T15:07:58.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:58.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.860938+0000", 2020-08-07T15:07:58.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017472500000000001 2020-08-07T15:07:58.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:58.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.861112+0000", 2020-08-07T15:07:58.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027810859999999999 2020-08-07T15:07:58.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:58.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.863893+0000", 2020-08-07T15:07:58.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2020-08-07T15:07:58.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:58.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.863894+0000", 2020-08-07T15:07:58.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7246e-05 2020-08-07T15:07:58.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:58.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.863921+0000", 2020-08-07T15:07:58.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.217031574 2020-08-07T15:07:58.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:58.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.080953+0000", 2020-08-07T15:07:58.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.071052727999999996 2020-08-07T15:07:58.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:58.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.152005+0000", 2020-08-07T15:07:58.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1769999999999999e-06 2020-08-07T15:07:58.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:58.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.152007+0000", 2020-08-07T15:07:58.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4748999999999998e-05 2020-08-07T15:07:58.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:58.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.152091+0000", 2020-08-07T15:07:58.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.29313854499999997 2020-08-07T15:07:58.980 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.980 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:58.980 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.981 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:104112 3.9 3:99770d45:::benchmark_data_smithi074_35447_object13013:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:07:58.981 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:45.859287+0000", 2020-08-07T15:07:58.981 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 12.675967059, 2020-08-07T15:07:58.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.29682865600000002, 2020-08-07T15:07:58.981 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:58.982 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:58.982 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:58.982 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:58.982 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:58.982 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 104112 2020-08-07T15:07:58.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.983 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:58.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:58.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.859287+0000", 2020-08-07T15:07:58.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:58.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:58.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.859287+0000", 2020-08-07T15:07:58.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9147999999999999e-05 2020-08-07T15:07:58.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:58.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.859316+0000", 2020-08-07T15:07:58.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.641e-06 2020-08-07T15:07:58.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:58.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.859320+0000", 2020-08-07T15:07:58.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8990000000000004e-06 2020-08-07T15:07:58.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:58.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.859326+0000", 2020-08-07T15:07:58.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8879999999999999e-06 2020-08-07T15:07:58.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:58.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.859330+0000", 2020-08-07T15:07:58.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025379999999999999 2020-08-07T15:07:58.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:58.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.861868+0000", 2020-08-07T15:07:58.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017909100000000001 2020-08-07T15:07:58.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:58.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.862047+0000", 2020-08-07T15:07:58.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000248946 2020-08-07T15:07:58.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:58.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.862296+0000", 2020-08-07T15:07:58.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035645329999999999 2020-08-07T15:07:58.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:58.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.865861+0000", 2020-08-07T15:07:58.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1119999999999999e-06 2020-08-07T15:07:58.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:58.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.865862+0000", 2020-08-07T15:07:58.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6837999999999999e-05 2020-08-07T15:07:58.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:58.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.865929+0000", 2020-08-07T15:07:58.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21515225399999999 2020-08-07T15:07:58.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:58.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.081081+0000", 2020-08-07T15:07:58.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.074966868000000006 2020-08-07T15:07:58.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:58.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.156048+0000", 2020-08-07T15:07:58.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4099999999999997e-07 2020-08-07T15:07:58.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:58.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.156049+0000", 2020-08-07T15:07:58.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6597e-05 2020-08-07T15:07:58.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:58.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.156115+0000", 2020-08-07T15:07:58.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.29682865600000002 2020-08-07T15:07:58.998 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.998 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:58.998 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:58.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:58.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:58.999 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106192 3.9 3:9094e6df:::benchmark_data_smithi074_35447_object13273:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:07:58.999 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.308893+0000", 2020-08-07T15:07:58.999 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.226360529000001, 2020-08-07T15:07:58.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43331256600000001, 2020-08-07T15:07:58.999 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:59.000 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:59.000 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:07:59.000 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:07:59.000 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:07:59.000 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106192 2020-08-07T15:07:59.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.001 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:59.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:59.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.308893+0000", 2020-08-07T15:07:59.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:59.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:59.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.308893+0000", 2020-08-07T15:07:59.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7664999999999998e-05 2020-08-07T15:07:59.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:59.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.308941+0000", 2020-08-07T15:07:59.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1030000000000001e-06 2020-08-07T15:07:59.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:59.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.308946+0000", 2020-08-07T15:07:59.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0556e-05 2020-08-07T15:07:59.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:59.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.308957+0000", 2020-08-07T15:07:59.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3900000000000003e-06 2020-08-07T15:07:59.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:59.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.308961+0000", 2020-08-07T15:07:59.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040638360000000004 2020-08-07T15:07:59.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:59.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.313025+0000", 2020-08-07T15:07:59.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010189800000000001 2020-08-07T15:07:59.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:59.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.313127+0000", 2020-08-07T15:07:59.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121432 2020-08-07T15:07:59.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:07:59.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.313248+0000", 2020-08-07T15:07:59.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046267490000000003 2020-08-07T15:07:59.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:07:59.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.317875+0000", 2020-08-07T15:07:59.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5600000000000005e-07 2020-08-07T15:07:59.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:07:59.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.317876+0000", 2020-08-07T15:07:59.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7885000000000005e-05 2020-08-07T15:07:59.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:59.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.317943+0000", 2020-08-07T15:07:59.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043631985999999998 2020-08-07T15:07:59.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:59.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.361575+0000", 2020-08-07T15:07:59.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38057095099999999 2020-08-07T15:07:59.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:07:59.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.742146+0000", 2020-08-07T15:07:59.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1939999999999999e-06 2020-08-07T15:07:59.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:07:59.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.742148+0000", 2020-08-07T15:07:59.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8264999999999999e-05 2020-08-07T15:07:59.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:59.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.742206+0000", 2020-08-07T15:07:59.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43331256600000001 2020-08-07T15:07:59.017 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:59.017 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:59.018 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:59.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.018 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106192 3.9 e58/20 ondisk, result = 0)", 2020-08-07T15:07:59.018 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.316710+0000", 2020-08-07T15:07:59.019 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.218544290000001, 2020-08-07T15:07:59.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42550088699999999, 2020-08-07T15:07:59.019 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:59.019 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:59.019 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:59.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:59.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.316710+0000", 2020-08-07T15:07:59.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:59.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:59.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.316710+0000", 2020-08-07T15:07:59.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5000999999999999e-05 2020-08-07T15:07:59.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:59.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.316745+0000", 2020-08-07T15:07:59.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1419999999999998e-06 2020-08-07T15:07:59.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:59.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.316751+0000", 2020-08-07T15:07:59.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42489234599999998 2020-08-07T15:07:59.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:59.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.741643+0000", 2020-08-07T15:07:59.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.89e-06 2020-08-07T15:07:59.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:59.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.741647+0000", 2020-08-07T15:07:59.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00048394600000000002 2020-08-07T15:07:59.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:59.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.742131+0000", 2020-08-07T15:07:59.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0099999999999998e-06 2020-08-07T15:07:59.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:59.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.742138+0000", 2020-08-07T15:07:59.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2552000000000006e-05 2020-08-07T15:07:59.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:59.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.742210+0000", 2020-08-07T15:07:59.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42550088699999999 2020-08-07T15:07:59.073 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:59.073 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:59.073 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:59.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.074 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "MOSDPGPushReply(3.1 62/60 [PushReplyOp(3:82e44256:::benchmark_data_smithi074_35447_object9802:head)])", 2020-08-07T15:07:59.074 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.559885+0000", 2020-08-07T15:07:59.074 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 7.9753692200000001, 2020-08-07T15:07:59.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26743896499999997, 2020-08-07T15:07:59.074 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:59.075 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:07:59.075 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:59.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:59.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.559885+0000", 2020-08-07T15:07:59.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:59.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:59.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.559885+0000", 2020-08-07T15:07:59.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5777000000000002e-05 2020-08-07T15:07:59.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:59.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.559900+0000", 2020-08-07T15:07:59.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7500000000000001e-06 2020-08-07T15:07:59.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:59.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.559904+0000", 2020-08-07T15:07:59.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26718243699999999 2020-08-07T15:07:59.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:59.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.827087+0000", 2020-08-07T15:07:59.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9779999999999996e-06 2020-08-07T15:07:59.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:59.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.827092+0000", 2020-08-07T15:07:59.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2234999999999998e-05 2020-08-07T15:07:59.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:59.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.827134+0000", 2020-08-07T15:07:59.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018978800000000001 2020-08-07T15:07:59.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:59.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.827324+0000", 2020-08-07T15:07:59.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26743896499999997 2020-08-07T15:07:59.083 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:59.083 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:59.083 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:59.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.084 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:108376 3.3 e62/20 ondisk, result = 0)", 2020-08-07T15:07:59.084 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:07:59.084 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 7.8522116369999999, 2020-08-07T15:07:59.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62926872599999994, 2020-08-07T15:07:59.085 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:59.085 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:07:59.085 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:59.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:59.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:07:59.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:59.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:59.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:07:59.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8491999999999998e-05 2020-08-07T15:07:59.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:59.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683061+0000", 2020-08-07T15:07:59.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9099999999999998e-06 2020-08-07T15:07:59.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:59.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683065+0000", 2020-08-07T15:07:59.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62906670600000003 2020-08-07T15:07:59.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:59.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312131+0000", 2020-08-07T15:07:59.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2147000000000001e-05 2020-08-07T15:07:59.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:59.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312143+0000", 2020-08-07T15:07:59.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5152e-05 2020-08-07T15:07:59.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:59.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312239+0000", 2020-08-07T15:07:59.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5089999999999993e-06 2020-08-07T15:07:59.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:59.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312247+0000", 2020-08-07T15:07:59.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3810000000000001e-05 2020-08-07T15:07:59.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:59.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312311+0000", 2020-08-07T15:07:59.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62926872599999994 2020-08-07T15:07:59.095 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:59.095 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:59.095 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:59.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.095 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:108400 3.b e62/20 3:de5f1917:::benchmark_data_smithi074_35447_object13549:head v 62'6168, mlcod=62'6160)", 2020-08-07T15:07:59.096 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.712575+0000", 2020-08-07T15:07:59.096 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 7.8226791899999997, 2020-08-07T15:07:59.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42039355899999997, 2020-08-07T15:07:59.096 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:59.096 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:59.097 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:59.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:59.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.712575+0000", 2020-08-07T15:07:59.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:59.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:59.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.712575+0000", 2020-08-07T15:07:59.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8982e-05 2020-08-07T15:07:59.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:59.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.712624+0000", 2020-08-07T15:07:59.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3940000000000001e-06 2020-08-07T15:07:59.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:59.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.712630+0000", 2020-08-07T15:07:59.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.41948886000000002 2020-08-07T15:07:59.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:59.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.132119+0000", 2020-08-07T15:07:59.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3150000000000003e-06 2020-08-07T15:07:59.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:59.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.132126+0000", 2020-08-07T15:07:59.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2264000000000001e-05 2020-08-07T15:07:59.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:59.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.132168+0000", 2020-08-07T15:07:59.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2549e-05 2020-08-07T15:07:59.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:59.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.132191+0000", 2020-08-07T15:07:59.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00072222300000000003 2020-08-07T15:07:59.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:59.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.132913+0000", 2020-08-07T15:07:59.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4972000000000002e-05 2020-08-07T15:07:59.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:59.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.132968+0000", 2020-08-07T15:07:59.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42039355899999997 2020-08-07T15:07:59.106 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:59.106 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:59.106 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:59.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.107 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:108464 3.6 e62/26 3:62704f92:::benchmark_data_smithi074_35447_object13557:head v 62'7200, mlcod=62'7194)", 2020-08-07T15:07:59.108 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.771334+0000", 2020-08-07T15:07:59.108 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 7.7639198250000003, 2020-08-07T15:07:59.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30528281699999998, 2020-08-07T15:07:59.108 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:59.108 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:07:59.109 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:59.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:59.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.771334+0000", 2020-08-07T15:07:59.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:59.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:59.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.771334+0000", 2020-08-07T15:07:59.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2753999999999998e-05 2020-08-07T15:07:59.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:59.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.771377+0000", 2020-08-07T15:07:59.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0269999999999997e-06 2020-08-07T15:07:59.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:59.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.771381+0000", 2020-08-07T15:07:59.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30419457500000002 2020-08-07T15:07:59.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:59.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.075575+0000", 2020-08-07T15:07:59.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.412e-06 2020-08-07T15:07:59.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:59.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.075581+0000", 2020-08-07T15:07:59.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7296000000000003e-05 2020-08-07T15:07:59.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:59.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.075628+0000", 2020-08-07T15:07:59.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5247000000000001e-05 2020-08-07T15:07:59.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:07:59.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.075653+0000", 2020-08-07T15:07:59.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00090736300000000001 2020-08-07T15:07:59.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:07:59.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.076561+0000", 2020-08-07T15:07:59.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6143000000000001e-05 2020-08-07T15:07:59.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:59.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.076617+0000", 2020-08-07T15:07:59.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30528281699999998 2020-08-07T15:07:59.118 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:59.118 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:59.118 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:59.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.119 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "MOSDPGPushReply(3.1 62/60 [PushReplyOp(3:8329f37e:::benchmark_data_smithi074_35447_object11915:head)])", 2020-08-07T15:07:59.119 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:07:59.119 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 7.6768536210000002, 2020-08-07T15:07:59.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996, 2020-08-07T15:07:59.120 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:07:59.120 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:07:59.120 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:07:59.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:07:59.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:07:59.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:07:59.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:07:59.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:07:59.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5372000000000001e-05 2020-08-07T15:07:59.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:07:59.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858416+0000", 2020-08-07T15:07:59.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6899999999999998e-06 2020-08-07T15:07:59.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:07:59.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858419+0000", 2020-08-07T15:07:59.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63907904299999996 2020-08-07T15:07:59.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:07:59.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497498+0000", 2020-08-07T15:07:59.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3332999999999999e-05 2020-08-07T15:07:59.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:07:59.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497512+0000", 2020-08-07T15:07:59.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000450875 2020-08-07T15:07:59.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:07:59.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497962+0000", 2020-08-07T15:07:59.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000267779 2020-08-07T15:07:59.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:07:59.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:07:59.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:07:59.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.498230+0000", 2020-08-07T15:07:59.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996 2020-08-07T15:07:59.139 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:59.139 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:59.140 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:59.140 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:07:59.140 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:07:59.140 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:07:59.141 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:07:59.154 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:07:59.154 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:07:59.155 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:07:59.155 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:07:59.155 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:07:59.155 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:07:59.155 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:07:59.156 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:07:59.156 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:07:59.156 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:07:59.156 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:07:59.157 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:07:59.157 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:07:59.158 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:07:59.153+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:07:59.247 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 83 16 120293 120277 11.3203 12.625 0.00562674 0.0110371 2020-08-07T15:07:59.254 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:59.253+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:59.359 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:59.359 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:59.364 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:59.360+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:07:59.369 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:07:59.423 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:07:59.479 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:59.463+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:59.564 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:07:59.560+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:07:59.582 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:59.583 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:59.595 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:07:59.676 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:07:59.667+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:07:59.775 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:07:59.774+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:07:59.831 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:07:59.831 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:07:59.845 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:07:59.899 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:07:59.887+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:07:59.983 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:07:59.981+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:00.042 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:00.042 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:00.044 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:08:00.091 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:00.084+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:00.202 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:00.193+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:00.248 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 84 16 122011 121995 11.3453 13.4219 0.00609137 0.0110122 2020-08-07T15:08:00.268 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:00.268 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:00.290 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:08:00.315 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:00.314+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:00.419 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:00.416+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:00.494 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:00.495 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:00.503 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:08:00.557 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:00.551+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:00.622 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:00.623 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:00.631 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:00.625+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:00.631 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:08:00.731 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:00.729+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:00.787 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:00.788 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:00.796 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:08:00.845 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:00.839+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:00.924 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:00.924 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:00.934 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:00.933+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:00.935 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:08:01.036 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:01.034+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:01.067 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:01.067 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:01.083 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:08:01.137 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:01.135+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:01.159 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:08:01.204 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:01.205 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:01.212 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:08:01.238 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:01.236+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:01.249 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 85 16 122384 122368 11.2461 2.91406 0.723474 0.0110325 2020-08-07T15:08:01.328 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:08:01.358 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:08:01.359 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:01.347+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:01.375 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:08:01.393 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:01.394 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:08:01.394 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:08:01.394 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:08:01.394 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.426 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:10240 3.6 3:665d71ff:::benchmark_data_smithi074_35447_object1279:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:08:01.426 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:08:01.426 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 78.985130854999994, 2020-08-07T15:08:01.476 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25191569699999999, 2020-08-07T15:08:01.476 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:01.477 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:01.487 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:01.490 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:01.490 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:01.500 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 10240 2020-08-07T15:08:01.508 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.508 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:01.508 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.508 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:01.508 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:08:01.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:01.514 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.515 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:01.515 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:08:01.515 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5463000000000002e-05 2020-08-07T15:08:01.515 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.515 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.516 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:01.516 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393264+0000", 2020-08-07T15:08:01.516 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9219999999999999e-06 2020-08-07T15:08:01.516 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.516 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.516 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:01.517 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393268+0000", 2020-08-07T15:08:01.517 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6149999999999997e-06 2020-08-07T15:08:01.524 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.524 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.524 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:01.524 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393277+0000", 2020-08-07T15:08:01.525 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2039999999999996e-06 2020-08-07T15:08:01.525 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.525 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.525 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:01.526 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393282+0000", 2020-08-07T15:08:01.526 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001225243 2020-08-07T15:08:01.526 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.526 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.526 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:01.527 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394507+0000", 2020-08-07T15:08:01.527 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7108999999999995e-05 2020-08-07T15:08:01.527 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.527 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.527 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:01.528 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394605+0000", 2020-08-07T15:08:01.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017490800000000001 2020-08-07T15:08:01.529 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.529 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.529 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:01.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394779+0000", 2020-08-07T15:08:01.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032608699999999999 2020-08-07T15:08:01.533 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.533 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.534 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:01.534 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.398040+0000", 2020-08-07T15:08:01.534 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013015259999999999 2020-08-07T15:08:01.534 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:01.535 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.399342+0000", 2020-08-07T15:08:01.535 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24566893400000001 2020-08-07T15:08:01.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.541 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:01.541 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645011+0000", 2020-08-07T15:08:01.541 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.217e-06 2020-08-07T15:08:01.541 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.542 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.542 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:01.542 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645012+0000", 2020-08-07T15:08:01.543 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0803000000000006e-05 2020-08-07T15:08:01.544 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.544 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.544 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:01.544 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645083+0000", 2020-08-07T15:08:01.544 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1882999999999998e-05 2020-08-07T15:08:01.545 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.545 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:01.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645135+0000", 2020-08-07T15:08:01.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25191569699999999 2020-08-07T15:08:01.546 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.546 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:01.546 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.565 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.565 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.565 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:31760 3.6 e30/26 3:6672be19:::benchmark_data_smithi074_35447_object3969:head v 30'2104, mlcod=30'2094)", 2020-08-07T15:08:01.565 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:56.999757+0000", 2020-08-07T15:08:01.566 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 64.378592933999997, 2020-08-07T15:08:01.566 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17823482099999999, 2020-08-07T15:08:01.566 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:01.566 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:01.595 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:01.617 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.617 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:01.617 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:56.999757+0000", 2020-08-07T15:08:01.617 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:01.635 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.635 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.636 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:01.636 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:56.999757+0000", 2020-08-07T15:08:01.636 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2955999999999999e-05 2020-08-07T15:08:01.636 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.636 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.637 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:01.637 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:56.999810+0000", 2020-08-07T15:08:01.637 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5509999999999996e-06 2020-08-07T15:08:01.637 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.637 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.638 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:01.638 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:56.999816+0000", 2020-08-07T15:08:01.638 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.177361504 2020-08-07T15:08:01.638 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.638 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.639 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:01.639 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177178+0000", 2020-08-07T15:08:01.639 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6979999999999997e-06 2020-08-07T15:08:01.639 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.639 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.640 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:01.640 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177183+0000", 2020-08-07T15:08:01.640 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1974999999999997e-05 2020-08-07T15:08:01.640 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.640 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.641 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:01.641 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177225+0000", 2020-08-07T15:08:01.641 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4612000000000001e-05 2020-08-07T15:08:01.641 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.641 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.655 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:01.656 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177249+0000", 2020-08-07T15:08:01.656 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00069063000000000002 2020-08-07T15:08:01.656 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.656 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.656 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:01.657 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177940+0000", 2020-08-07T15:08:01.657 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1894999999999997e-05 2020-08-07T15:08:01.657 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.657 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.657 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:01.658 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:57.177992+0000", 2020-08-07T15:08:01.658 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17823482099999999 2020-08-07T15:08:01.658 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.658 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:01.658 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.659 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.659 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.659 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:43335 3.6 e32/26 3:65a57ed3:::benchmark_data_smithi074_35447_object5416:head v 32'2831, mlcod=32'2822)", 2020-08-07T15:08:01.659 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:03.998850+0000", 2020-08-07T15:08:01.659 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.379499783999997, 2020-08-07T15:08:01.659 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.221384421, 2020-08-07T15:08:01.660 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:01.660 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:01.660 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:01.660 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.660 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:01.661 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998850+0000", 2020-08-07T15:08:01.661 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:01.661 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.661 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.661 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:01.663 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998850+0000", 2020-08-07T15:08:01.663 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4788000000000004e-05 2020-08-07T15:08:01.663 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.663 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.663 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:01.664 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998915+0000", 2020-08-07T15:08:01.664 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7909999999999999e-06 2020-08-07T15:08:01.664 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.666 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:01.666 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998922+0000", 2020-08-07T15:08:01.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22050499700000001 2020-08-07T15:08:01.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.667 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:01.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219427+0000", 2020-08-07T15:08:01.668 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0950000000000001e-06 2020-08-07T15:08:01.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:01.668 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219433+0000", 2020-08-07T15:08:01.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6342999999999997e-05 2020-08-07T15:08:01.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:01.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219509+0000", 2020-08-07T15:08:01.670 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4173999999999999e-05 2020-08-07T15:08:01.670 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.670 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.670 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:01.670 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219533+0000", 2020-08-07T15:08:01.670 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00064214899999999995 2020-08-07T15:08:01.671 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.671 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.671 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:01.672 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.220175+0000", 2020-08-07T15:08:01.672 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9083999999999999e-05 2020-08-07T15:08:01.672 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.672 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.673 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:01.673 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.220234+0000", 2020-08-07T15:08:01.673 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.221384421 2020-08-07T15:08:01.673 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.673 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:01.674 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.674 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.674 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.674 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:43336 3.6 e32/26 3:65a57ed3:::benchmark_data_smithi074_35447_object5416:head v 32'2832, mlcod=32'2822)", 2020-08-07T15:08:01.674 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:03.999509+0000", 2020-08-07T15:08:01.675 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.378840455000002, 2020-08-07T15:08:01.677 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22632864799999999, 2020-08-07T15:08:01.677 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:01.677 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:01.677 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:01.678 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.678 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:01.678 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999509+0000", 2020-08-07T15:08:01.678 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:01.678 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.679 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:01.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999509+0000", 2020-08-07T15:08:01.679 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0107e-05 2020-08-07T15:08:01.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.680 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:01.680 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999559+0000", 2020-08-07T15:08:01.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0889999999999996e-06 2020-08-07T15:08:01.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:01.681 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999566+0000", 2020-08-07T15:08:01.681 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22332094599999999 2020-08-07T15:08:01.681 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.681 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.682 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:01.682 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222887+0000", 2020-08-07T15:08:01.682 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5990000000000001e-06 2020-08-07T15:08:01.682 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.682 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.683 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:01.683 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222892+0000", 2020-08-07T15:08:01.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2342999999999998e-05 2020-08-07T15:08:01.683 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.683 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.683 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:01.684 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222924+0000", 2020-08-07T15:08:01.684 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9108999999999999e-05 2020-08-07T15:08:01.684 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.684 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.685 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:01.685 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222944+0000", 2020-08-07T15:08:01.685 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028409289999999999 2020-08-07T15:08:01.685 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.690 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.691 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:01.691 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.225784+0000", 2020-08-07T15:08:01.691 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3526000000000002e-05 2020-08-07T15:08:01.691 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.692 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:01.692 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.225838+0000", 2020-08-07T15:08:01.692 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22632864799999999 2020-08-07T15:08:01.693 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.693 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:01.694 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.694 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.694 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.695 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:43784 3.6 e32/26 3:6da53356:::benchmark_data_smithi074_35447_object5472:head v 32'2848, mlcod=32'2839)", 2020-08-07T15:08:01.695 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:04.253355+0000", 2020-08-07T15:08:01.695 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.124994882999999, 2020-08-07T15:08:01.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21456700500000001, 2020-08-07T15:08:01.695 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:01.696 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:01.696 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:01.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:01.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253355+0000", 2020-08-07T15:08:01.697 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:01.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.698 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.698 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:01.699 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253355+0000", 2020-08-07T15:08:01.699 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6168000000000002e-05 2020-08-07T15:08:01.699 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.699 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.699 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:01.700 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253401+0000", 2020-08-07T15:08:01.700 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3179999999999997e-06 2020-08-07T15:08:01.700 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.700 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:01.701 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253406+0000", 2020-08-07T15:08:01.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21194338600000001 2020-08-07T15:08:01.701 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.701 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:01.702 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465350+0000", 2020-08-07T15:08:01.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3759999999999999e-06 2020-08-07T15:08:01.702 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.702 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.702 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:01.703 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465355+0000", 2020-08-07T15:08:01.703 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8250999999999997e-05 2020-08-07T15:08:01.703 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.703 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:01.703 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465393+0000", 2020-08-07T15:08:01.704 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0296999999999999e-05 2020-08-07T15:08:01.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.704 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:01.704 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465414+0000", 2020-08-07T15:08:01.705 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024515209999999999 2020-08-07T15:08:01.705 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.705 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.705 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:01.705 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.467865+0000", 2020-08-07T15:08:01.712 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6687999999999997e-05 2020-08-07T15:08:01.712 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.712 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.712 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:01.713 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.467922+0000", 2020-08-07T15:08:01.713 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21456700500000001 2020-08-07T15:08:01.713 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.713 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:01.713 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.714 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.714 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.714 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:44152 3.6 e32/26 3:6c90cb4b:::benchmark_data_smithi074_35447_object5518:head v 32'2856, mlcod=32'2851)", 2020-08-07T15:08:01.714 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:04.473943+0000", 2020-08-07T15:08:01.715 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 56.904407061000001, 2020-08-07T15:08:01.715 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18875260999999999, 2020-08-07T15:08:01.715 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:01.715 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:01.715 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:01.716 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.716 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:01.716 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.473943+0000", 2020-08-07T15:08:01.716 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:01.716 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.718 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.718 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:01.718 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.473943+0000", 2020-08-07T15:08:01.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8267999999999999e-05 2020-08-07T15:08:01.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:01.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.473991+0000", 2020-08-07T15:08:01.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9689999999999999e-06 2020-08-07T15:08:01.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:01.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.473997+0000", 2020-08-07T15:08:01.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18142124000000001 2020-08-07T15:08:01.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.724 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:01.724 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.655418+0000", 2020-08-07T15:08:01.724 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7439999999999998e-06 2020-08-07T15:08:01.724 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.724 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.725 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:01.725 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.655423+0000", 2020-08-07T15:08:01.725 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0052999999999997e-05 2020-08-07T15:08:01.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.726 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:01.726 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.655473+0000", 2020-08-07T15:08:01.726 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3739000000000001e-05 2020-08-07T15:08:01.727 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.727 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.727 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:01.727 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.655497+0000", 2020-08-07T15:08:01.727 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0071399030000000004 2020-08-07T15:08:01.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:01.728 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.662637+0000", 2020-08-07T15:08:01.728 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8693999999999997e-05 2020-08-07T15:08:01.729 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.729 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.729 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:01.729 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.662695+0000", 2020-08-07T15:08:01.729 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18875260999999999 2020-08-07T15:08:01.730 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.730 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:01.730 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.730 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.731 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:54200 3.3 e37/20 3:cc40637e:::benchmark_data_smithi074_35447_object6774:head v 37'3248, mlcod=37'3236)", 2020-08-07T15:08:01.731 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:08:01.731 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 48.279787634999998, 2020-08-07T15:08:01.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24065872499999999, 2020-08-07T15:08:01.732 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:01.732 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:01.732 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:01.732 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:01.734 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:08:01.734 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:01.734 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.734 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.734 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:01.735 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:08:01.735 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1671000000000002e-05 2020-08-07T15:08:01.735 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.735 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.735 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:01.736 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098594+0000", 2020-08-07T15:08:01.736 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0890000000000002e-06 2020-08-07T15:08:01.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.736 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.736 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:01.737 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098598+0000", 2020-08-07T15:08:01.737 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23875592300000001 2020-08-07T15:08:01.737 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.737 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.737 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:01.737 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337354+0000", 2020-08-07T15:08:01.738 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0420000000000002e-06 2020-08-07T15:08:01.738 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.738 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.738 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:01.738 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337359+0000", 2020-08-07T15:08:01.739 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0586000000000001e-05 2020-08-07T15:08:01.739 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.739 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.739 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:01.739 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337399+0000", 2020-08-07T15:08:01.740 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9415999999999998e-05 2020-08-07T15:08:01.740 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.740 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.740 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:01.740 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337419+0000", 2020-08-07T15:08:01.741 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017568390000000001 2020-08-07T15:08:01.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.741 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.741 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:01.741 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.339176+0000", 2020-08-07T15:08:01.742 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5158999999999999e-05 2020-08-07T15:08:01.743 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:01.744 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.339221+0000", 2020-08-07T15:08:01.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24065872499999999 2020-08-07T15:08:01.745 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.745 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:01.745 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.745 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.746 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:54728 3.7 e37/20 3:ee999259:::benchmark_data_smithi074_35447_object6840:head v 37'3328, mlcod=37'3319)", 2020-08-07T15:08:01.746 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:13.452195+0000", 2020-08-07T15:08:01.746 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 47.926154854000004, 2020-08-07T15:08:01.746 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.187979807, 2020-08-07T15:08:01.747 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:01.747 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:01.747 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:01.747 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.747 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:01.748 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.452195+0000", 2020-08-07T15:08:01.748 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:01.748 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.748 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.748 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:01.749 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.452195+0000", 2020-08-07T15:08:01.749 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9023000000000001e-05 2020-08-07T15:08:01.749 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.749 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.750 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:01.750 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.452244+0000", 2020-08-07T15:08:01.750 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.733e-06 2020-08-07T15:08:01.750 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.750 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.751 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:01.751 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.452250+0000", 2020-08-07T15:08:01.751 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18453694300000001 2020-08-07T15:08:01.751 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.751 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.751 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:01.752 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.636787+0000", 2020-08-07T15:08:01.752 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5679999999999999e-06 2020-08-07T15:08:01.752 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.752 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:01.753 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.636792+0000", 2020-08-07T15:08:01.753 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0119999999999999e-05 2020-08-07T15:08:01.753 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.753 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.753 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:01.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.636872+0000", 2020-08-07T15:08:01.755 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6356999999999999e-05 2020-08-07T15:08:01.755 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.755 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:01.756 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.636899+0000", 2020-08-07T15:08:01.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003233737 2020-08-07T15:08:01.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:01.757 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.640132+0000", 2020-08-07T15:08:01.757 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2326000000000001e-05 2020-08-07T15:08:01.757 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.757 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:01.758 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.640175+0000", 2020-08-07T15:08:01.758 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.187979807 2020-08-07T15:08:01.759 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.759 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:01.759 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.760 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:71136 3.6 e43/26 3:64cb7acb:::benchmark_data_smithi074_35447_object8891:head v 43'4744, mlcod=43'4739)", 2020-08-07T15:08:01.760 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:08:01.760 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 37.059736268999998, 2020-08-07T15:08:01.760 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26135482999999998, 2020-08-07T15:08:01.760 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:01.761 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:01.761 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:01.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.761 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:01.761 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:08:01.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:01.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.762 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.762 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:01.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:08:01.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3702999999999997e-05 2020-08-07T15:08:01.763 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.763 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:01.763 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318667+0000", 2020-08-07T15:08:01.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3119999999999999e-06 2020-08-07T15:08:01.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.766 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:01.766 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318674+0000", 2020-08-07T15:08:01.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.255173292 2020-08-07T15:08:01.766 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:01.767 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.573847+0000", 2020-08-07T15:08:01.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1140000000000002e-06 2020-08-07T15:08:01.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.768 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.768 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:01.768 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.573853+0000", 2020-08-07T15:08:01.768 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00057603799999999996 2020-08-07T15:08:01.768 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.768 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.769 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:01.769 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.574429+0000", 2020-08-07T15:08:01.769 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4776e-05 2020-08-07T15:08:01.769 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.770 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.770 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:01.770 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.574444+0000", 2020-08-07T15:08:01.771 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054577080000000004 2020-08-07T15:08:01.771 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.771 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.771 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:01.772 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.579901+0000", 2020-08-07T15:08:01.772 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6886999999999998e-05 2020-08-07T15:08:01.772 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.772 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.772 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:01.773 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.579968+0000", 2020-08-07T15:08:01.773 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26135482999999998 2020-08-07T15:08:01.773 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.773 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:01.773 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.774 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.774 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.774 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:81544 3.7 e48/20 3:e9bb655d:::benchmark_data_smithi074_35447_object10192:head v 48'4944, mlcod=48'4935)", 2020-08-07T15:08:01.774 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:30.902099+0000", 2020-08-07T15:08:01.776 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 30.476251309999999, 2020-08-07T15:08:01.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20669163500000001, 2020-08-07T15:08:01.776 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:01.776 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:01.777 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:01.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:01.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.902099+0000", 2020-08-07T15:08:01.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:01.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:01.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.902099+0000", 2020-08-07T15:08:01.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1317999999999999e-05 2020-08-07T15:08:01.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.779 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:01.779 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.902150+0000", 2020-08-07T15:08:01.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2079999999999997e-06 2020-08-07T15:08:01.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.780 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:01.780 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.902156+0000", 2020-08-07T15:08:01.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20264375100000001 2020-08-07T15:08:01.780 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.781 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:01.781 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.104800+0000", 2020-08-07T15:08:01.781 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2390000000000002e-06 2020-08-07T15:08:01.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.782 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:01.782 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.104805+0000", 2020-08-07T15:08:01.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3696000000000003e-05 2020-08-07T15:08:01.782 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:01.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.104849+0000", 2020-08-07T15:08:01.783 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4179e-05 2020-08-07T15:08:01.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:01.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.104873+0000", 2020-08-07T15:08:01.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038653839999999999 2020-08-07T15:08:01.785 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:01.789 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.108738+0000", 2020-08-07T15:08:01.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1860000000000002e-05 2020-08-07T15:08:01.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.794 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.794 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:01.795 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.108790+0000", 2020-08-07T15:08:01.795 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20669163500000001 2020-08-07T15:08:01.795 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.795 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:01.795 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.796 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.797 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:85696 3.f e51/20 3:f99fcea4:::benchmark_data_smithi074_35447_object10711:head v 51'5688, mlcod=51'5679)", 2020-08-07T15:08:01.797 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:08:01.797 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 27.725744855999999, 2020-08-07T15:08:01.798 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49655717300000002, 2020-08-07T15:08:01.798 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:01.798 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:01.799 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:01.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:01.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:08:01.799 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:01.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:01.800 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:08:01.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9543000000000002e-05 2020-08-07T15:08:01.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.801 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.801 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:01.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652665+0000", 2020-08-07T15:08:01.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8700000000000003e-06 2020-08-07T15:08:01.801 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.802 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.802 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:01.802 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652671+0000", 2020-08-07T15:08:01.802 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49579595799999998 2020-08-07T15:08:01.802 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:01.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148467+0000", 2020-08-07T15:08:01.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2509999999999997e-06 2020-08-07T15:08:01.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:01.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148475+0000", 2020-08-07T15:08:01.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5603999999999998e-05 2020-08-07T15:08:01.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:01.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148530+0000", 2020-08-07T15:08:01.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5486999999999998e-05 2020-08-07T15:08:01.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:01.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148556+0000", 2020-08-07T15:08:01.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00055666600000000004 2020-08-07T15:08:01.806 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:01.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.149112+0000", 2020-08-07T15:08:01.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9793999999999998e-05 2020-08-07T15:08:01.821 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:01.821 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.149162+0000", 2020-08-07T15:08:01.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49655717300000002 2020-08-07T15:08:01.822 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.822 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:01.822 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.823 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:86792 3.f e52/20 3:fc5386ee:::benchmark_data_smithi074_35447_object10848:head v 52'5752, mlcod=52'5743)", 2020-08-07T15:08:01.823 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:08:01.823 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 26.975899081000001, 2020-08-07T15:08:01.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.295270005, 2020-08-07T15:08:01.824 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:01.824 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:01.824 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:01.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:01.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:08:01.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:01.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.827 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:01.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:08:01.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9712000000000001e-05 2020-08-07T15:08:01.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:01.829 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402500+0000", 2020-08-07T15:08:01.829 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6789999999999999e-06 2020-08-07T15:08:01.829 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.830 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.830 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:01.830 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402507+0000", 2020-08-07T15:08:01.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29317074999999998 2020-08-07T15:08:01.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:01.833 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695678+0000", 2020-08-07T15:08:01.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8120000000000002e-06 2020-08-07T15:08:01.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:01.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695685+0000", 2020-08-07T15:08:01.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1270999999999998e-05 2020-08-07T15:08:01.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:01.836 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695726+0000", 2020-08-07T15:08:01.842 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2739e-05 2020-08-07T15:08:01.842 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.842 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.842 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:01.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695749+0000", 2020-08-07T15:08:01.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019333899999999999 2020-08-07T15:08:01.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.843 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:01.844 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.697682+0000", 2020-08-07T15:08:01.844 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8652000000000002e-05 2020-08-07T15:08:01.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.844 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:01.844 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.697721+0000", 2020-08-07T15:08:01.845 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.295270005 2020-08-07T15:08:01.845 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.845 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:01.845 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.853 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:87088 3.6 e52/26 3:6b0320df:::benchmark_data_smithi074_35447_object10885:head v 52'5912, mlcod=52'5902)", 2020-08-07T15:08:01.854 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:08:01.854 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 26.411665841000001, 2020-08-07T15:08:01.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29522189799999998, 2020-08-07T15:08:01.854 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:01.854 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:01.855 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:01.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.873 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:01.874 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:08:01.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:01.879 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.880 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:01.880 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:08:01.880 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0751000000000002e-05 2020-08-07T15:08:01.880 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.881 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:01.881 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966735+0000", 2020-08-07T15:08:01.881 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3080000000000004e-06 2020-08-07T15:08:01.881 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.881 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.882 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:01.882 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966741+0000", 2020-08-07T15:08:01.882 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29365034699999998 2020-08-07T15:08:01.882 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.882 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.882 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:01.883 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260391+0000", 2020-08-07T15:08:01.883 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2720000000000003e-06 2020-08-07T15:08:01.883 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.883 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.883 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:01.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260398+0000", 2020-08-07T15:08:01.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8819000000000005e-05 2020-08-07T15:08:01.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.884 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.884 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:01.885 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260476+0000", 2020-08-07T15:08:01.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4958999999999999e-05 2020-08-07T15:08:01.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.890 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:01.890 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260501+0000", 2020-08-07T15:08:01.890 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001353641 2020-08-07T15:08:01.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.891 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:01.891 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.261855+0000", 2020-08-07T15:08:01.891 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0800999999999997e-05 2020-08-07T15:08:01.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.891 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.892 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:01.892 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.261906+0000", 2020-08-07T15:08:01.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29522189799999998 2020-08-07T15:08:01.892 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.900 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:01.901 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.901 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.901 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.901 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:93920 3.6 e56/26 3:631393d3:::benchmark_data_smithi074_35447_object11739:head v 56'6352, mlcod=55'6342)", 2020-08-07T15:08:01.901 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:08:01.902 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 22.051550410000001, 2020-08-07T15:08:01.902 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39765404300000001, 2020-08-07T15:08:01.902 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:01.902 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:01.902 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:01.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.903 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:01.903 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:08:01.903 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:01.903 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.904 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:01.904 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:08:01.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3035999999999998e-05 2020-08-07T15:08:01.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:01.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326852+0000", 2020-08-07T15:08:01.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5379999999999999e-06 2020-08-07T15:08:01.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.908 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:01.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326859+0000", 2020-08-07T15:08:01.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.393917568 2020-08-07T15:08:01.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:01.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.720777+0000", 2020-08-07T15:08:01.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2430000000000005e-06 2020-08-07T15:08:01.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:01.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.720785+0000", 2020-08-07T15:08:01.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00065658700000000003 2020-08-07T15:08:01.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:01.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.721441+0000", 2020-08-07T15:08:01.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8585000000000002e-05 2020-08-07T15:08:01.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:01.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.721470+0000", 2020-08-07T15:08:01.914 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00293349 2020-08-07T15:08:01.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:01.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.724403+0000", 2020-08-07T15:08:01.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9996e-05 2020-08-07T15:08:01.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:01.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.724453+0000", 2020-08-07T15:08:01.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39765404300000001 2020-08-07T15:08:01.916 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.916 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:01.916 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.917 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:94384 3.3 e56/20 3:ce533213:::benchmark_data_smithi074_35447_object11797:head v 56'5488, mlcod=56'5479)", 2020-08-07T15:08:01.917 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.637245+0000", 2020-08-07T15:08:01.918 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 21.741104906, 2020-08-07T15:08:01.918 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.247824459, 2020-08-07T15:08:01.918 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:01.918 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:01.918 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:01.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.919 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:01.919 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637245+0000", 2020-08-07T15:08:01.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:01.919 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:01.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637245+0000", 2020-08-07T15:08:01.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0782999999999997e-05 2020-08-07T15:08:01.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:01.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637306+0000", 2020-08-07T15:08:01.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8699999999999992e-06 2020-08-07T15:08:01.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:01.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637314+0000", 2020-08-07T15:08:01.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24469571800000001 2020-08-07T15:08:01.925 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:01.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882009+0000", 2020-08-07T15:08:01.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8510000000000004e-06 2020-08-07T15:08:01.926 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.926 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:01.926 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882015+0000", 2020-08-07T15:08:01.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6708999999999997e-05 2020-08-07T15:08:01.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:01.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882052+0000", 2020-08-07T15:08:01.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0280000000000001e-05 2020-08-07T15:08:01.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:01.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882082+0000", 2020-08-07T15:08:01.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002934767 2020-08-07T15:08:01.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:01.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.885017+0000", 2020-08-07T15:08:01.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2481000000000001e-05 2020-08-07T15:08:01.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:01.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.885069+0000", 2020-08-07T15:08:01.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.247824459 2020-08-07T15:08:01.930 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.931 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:01.931 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.932 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.932 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:96832 3.6 e57/26 3:6d818ad0:::benchmark_data_smithi074_35447_object12103:head v 57'6536, mlcod=56'6526)", 2020-08-07T15:08:01.933 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:41.323206+0000", 2020-08-07T15:08:01.933 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 20.055144010999999, 2020-08-07T15:08:01.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.188700062, 2020-08-07T15:08:01.934 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:01.934 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:01.934 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:01.934 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.934 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:01.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.323206+0000", 2020-08-07T15:08:01.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:01.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:01.936 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.323206+0000", 2020-08-07T15:08:01.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6401999999999999e-05 2020-08-07T15:08:01.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:01.936 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.323232+0000", 2020-08-07T15:08:01.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3380000000000001e-06 2020-08-07T15:08:01.937 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:01.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.323236+0000", 2020-08-07T15:08:01.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18768147800000001 2020-08-07T15:08:01.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.938 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:01.938 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.510917+0000", 2020-08-07T15:08:01.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.536e-06 2020-08-07T15:08:01.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:01.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.510923+0000", 2020-08-07T15:08:01.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3084999999999998e-05 2020-08-07T15:08:01.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.940 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.940 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:01.940 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.510966+0000", 2020-08-07T15:08:01.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8317e-05 2020-08-07T15:08:01.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:01.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.510994+0000", 2020-08-07T15:08:01.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00087632200000000002 2020-08-07T15:08:01.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:01.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.511870+0000", 2020-08-07T15:08:01.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5583999999999997e-05 2020-08-07T15:08:01.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.944 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:01.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.511906+0000", 2020-08-07T15:08:01.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.188700062 2020-08-07T15:08:01.945 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.945 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:01.945 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.946 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:98624 3.6 e57/26 3:6fe31b68:::benchmark_data_smithi074_35447_object12327:head v 57'6648, mlcod=57'6641)", 2020-08-07T15:08:01.946 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:42.318711+0000", 2020-08-07T15:08:01.946 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 19.059638722999999, 2020-08-07T15:08:01.946 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18025650700000001, 2020-08-07T15:08:01.946 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:01.947 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:01.947 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:01.947 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.947 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:01.947 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.318711+0000", 2020-08-07T15:08:01.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:01.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.948 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:01.948 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.318711+0000", 2020-08-07T15:08:01.948 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7558000000000001e-05 2020-08-07T15:08:01.949 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.949 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:01.949 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.318759+0000", 2020-08-07T15:08:01.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7010000000000001e-06 2020-08-07T15:08:01.950 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.950 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.950 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:01.950 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.318764+0000", 2020-08-07T15:08:01.950 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.178359881 2020-08-07T15:08:01.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:01.951 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.497124+0000", 2020-08-07T15:08:01.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0409999999999997e-06 2020-08-07T15:08:01.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.952 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.952 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:01.952 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.497129+0000", 2020-08-07T15:08:01.954 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0988000000000001e-05 2020-08-07T15:08:01.955 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.955 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.955 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:01.956 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.497170+0000", 2020-08-07T15:08:01.956 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3748999999999999e-05 2020-08-07T15:08:01.956 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.956 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.956 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:01.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.497194+0000", 2020-08-07T15:08:01.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001726357 2020-08-07T15:08:01.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:01.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.498920+0000", 2020-08-07T15:08:01.958 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7231999999999998e-05 2020-08-07T15:08:01.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:01.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:42.498968+0000", 2020-08-07T15:08:01.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18025650700000001 2020-08-07T15:08:01.959 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.960 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:01.960 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.960 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:104184 3.1 e58/26 3:841b6818:::benchmark_data_smithi074_35447_object13022:head v 58'6200, mlcod=58'6196)", 2020-08-07T15:08:01.961 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:45.910556+0000", 2020-08-07T15:08:01.961 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 15.467794042, 2020-08-07T15:08:01.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.184209715, 2020-08-07T15:08:01.961 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:01.961 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:01.962 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:01.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.962 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:01.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.910556+0000", 2020-08-07T15:08:01.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:01.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:01.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.910556+0000", 2020-08-07T15:08:01.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7603e-05 2020-08-07T15:08:01.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:01.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.910583+0000", 2020-08-07T15:08:01.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0899999999999998e-06 2020-08-07T15:08:01.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:01.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.910587+0000", 2020-08-07T15:08:01.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.17963316800000001 2020-08-07T15:08:01.969 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.969 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.969 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:01.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.090221+0000", 2020-08-07T15:08:01.969 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2870000000000001e-06 2020-08-07T15:08:01.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.970 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:01.970 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.090227+0000", 2020-08-07T15:08:01.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9820000000000002e-05 2020-08-07T15:08:01.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:01.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.090267+0000", 2020-08-07T15:08:01.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4604e-05 2020-08-07T15:08:01.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:01.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.090291+0000", 2020-08-07T15:08:01.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044229129999999997 2020-08-07T15:08:01.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:01.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.094714+0000", 2020-08-07T15:08:01.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1230000000000002e-05 2020-08-07T15:08:01.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:01.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.094765+0000", 2020-08-07T15:08:01.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.184209715 2020-08-07T15:08:01.975 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.975 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:01.985 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:01.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.986 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:104560 3.f e58/20 3:fb4383ae:::benchmark_data_smithi074_35447_object13069:head v 58'6888, mlcod=58'6878)", 2020-08-07T15:08:01.986 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:46.301207+0000", 2020-08-07T15:08:01.987 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 15.077143089, 2020-08-07T15:08:01.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23687350400000001, 2020-08-07T15:08:01.987 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:01.987 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:01.987 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:01.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:01.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301207+0000", 2020-08-07T15:08:01.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:01.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:01.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:01.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:01.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301207+0000", 2020-08-07T15:08:01.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7945000000000003e-05 2020-08-07T15:08:01.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.009 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:08:02.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:02.033 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301255+0000", 2020-08-07T15:08:02.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2030000000000002e-06 2020-08-07T15:08:02.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.034 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:02.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301259+0000", 2020-08-07T15:08:02.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23565420500000001 2020-08-07T15:08:02.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:02.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.536913+0000", 2020-08-07T15:08:02.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2290000000000003e-06 2020-08-07T15:08:02.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.045 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:02.045 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.536921+0000", 2020-08-07T15:08:02.046 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0955e-05 2020-08-07T15:08:02.046 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.046 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.046 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:02.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.537002+0000", 2020-08-07T15:08:02.047 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8977999999999999e-05 2020-08-07T15:08:02.047 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.048 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:02.048 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.537031+0000", 2020-08-07T15:08:02.048 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00099831000000000008 2020-08-07T15:08:02.048 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.048 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.049 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:02.049 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.538030+0000", 2020-08-07T15:08:02.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0679e-05 2020-08-07T15:08:02.049 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.049 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.050 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:02.050 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.538080+0000", 2020-08-07T15:08:02.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23687350400000001 2020-08-07T15:08:02.050 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:02.050 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:02.051 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:02.051 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.051 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:116064 3.6 e64/26 3:6cc03ffa:::benchmark_data_smithi074_35447_object14507:head v 64'7632, mlcod=64'7623)", 2020-08-07T15:08:02.051 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:08:02.052 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.015589007, 2020-08-07T15:08:02.052 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25728607799999997, 2020-08-07T15:08:02.052 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:02.052 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:02.052 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:02.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.053 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:02.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:08:02.053 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:02.053 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.054 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.054 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:02.054 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:08:02.054 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6042000000000003e-05 2020-08-07T15:08:02.055 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.055 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:02.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362817+0000", 2020-08-07T15:08:02.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9399999999999999e-06 2020-08-07T15:08:02.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:02.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362823+0000", 2020-08-07T15:08:02.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.254635583 2020-08-07T15:08:02.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:02.057 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617458+0000", 2020-08-07T15:08:02.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5100000000000004e-06 2020-08-07T15:08:02.058 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.058 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.058 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:02.058 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617465+0000", 2020-08-07T15:08:02.058 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4439999999999999e-05 2020-08-07T15:08:02.058 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.059 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.059 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:02.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617539+0000", 2020-08-07T15:08:02.059 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3527e-05 2020-08-07T15:08:02.059 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:02.060 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617563+0000", 2020-08-07T15:08:02.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024301069999999999 2020-08-07T15:08:02.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.061 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:02.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.619993+0000", 2020-08-07T15:08:02.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3928999999999998e-05 2020-08-07T15:08:02.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.063 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:02.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.620047+0000", 2020-08-07T15:08:02.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25728607799999997 2020-08-07T15:08:02.063 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:02.064 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:02.064 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:02.064 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:02.064 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:02.065 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:02.065 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:08:02.066 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:08:02.066 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:08:02.066 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:08:02.067 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:08:02.067 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:08:02.067 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:08:02.070 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:08:02.070 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:08:02.071 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:08:02.071 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:08:02.072 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:08:02.072 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:08:02.073 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:08:02.073 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:02.020+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:02.130 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:02.128+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:02.260 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 86 16 123876 123860 11.2508 11.6562 0.00680075 0.0111059 2020-08-07T15:08:02.260 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:02.253+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:02.278 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:02.279 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:08:02.286 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.287 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:123800 3.6 3:684fe972:::benchmark_data_smithi074_35447_object15474:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e69)", 2020-08-07T15:08:02.287 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:02.204802+0000", 2020-08-07T15:08:02.288 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.066303847999999999, 2020-08-07T15:08:02.288 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.066326138000000007, 2020-08-07T15:08:02.288 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:02.289 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:08:02.289 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:02.289 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:02.289 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:02.298 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 123800 2020-08-07T15:08:02.298 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.303 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:02.304 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.304 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:02.304 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.204802+0000", 2020-08-07T15:08:02.304 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:02.305 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.305 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.305 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:02.314 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.204802+0000", 2020-08-07T15:08:02.314 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5443e-05 2020-08-07T15:08:02.315 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.315 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.315 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:02.315 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.204827+0000", 2020-08-07T15:08:02.316 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8720000000000002e-06 2020-08-07T15:08:02.324 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.339 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.339 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:02.339 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.204830+0000", 2020-08-07T15:08:02.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7579999999999998e-06 2020-08-07T15:08:02.340 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.340 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:02.340 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.204836+0000", 2020-08-07T15:08:02.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4370000000000001e-06 2020-08-07T15:08:02.341 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:02.364 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.204839+0000", 2020-08-07T15:08:02.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041793380000000003 2020-08-07T15:08:02.365 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.365 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.365 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:02.366 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.209018+0000", 2020-08-07T15:08:02.366 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5193999999999997e-05 2020-08-07T15:08:02.366 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.366 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:02.367 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.209113+0000", 2020-08-07T15:08:02.367 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015423100000000001 2020-08-07T15:08:02.367 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.367 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.367 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:02.368 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.209267+0000", 2020-08-07T15:08:02.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0025621350000000001 2020-08-07T15:08:02.368 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.368 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.368 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:02.369 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.211829+0000", 2020-08-07T15:08:02.369 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001155047 2020-08-07T15:08:02.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.369 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:02.389 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.212984+0000", 2020-08-07T15:08:02.390 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0081824549999999999 2020-08-07T15:08:02.391 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:02.391 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:02.391 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:02.391 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.392 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.392 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:123880 3.0 3:04cf2fe9:::benchmark_data_smithi074_35447_object15484:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e69)", 2020-08-07T15:08:02.392 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:02.255362+0000", 2020-08-07T15:08:02.392 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.015743819999999999, 2020-08-07T15:08:02.392 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.015826553, 2020-08-07T15:08:02.393 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:02.393 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:02.393 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:02.393 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:02.393 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:02.405 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 123880 2020-08-07T15:08:02.406 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.406 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:02.407 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.407 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:02.407 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.255362+0000", 2020-08-07T15:08:02.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:02.408 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.408 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.408 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:02.408 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.255362+0000", 2020-08-07T15:08:02.408 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9993000000000001e-05 2020-08-07T15:08:02.414 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.414 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.415 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:02.415 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.255412+0000", 2020-08-07T15:08:02.415 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3949999999999998e-06 2020-08-07T15:08:02.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.416 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:02.416 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.255418+0000", 2020-08-07T15:08:02.416 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4457999999999999e-05 2020-08-07T15:08:02.416 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.416 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.417 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:02.417 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.255433+0000", 2020-08-07T15:08:02.417 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5309999999999998e-06 2020-08-07T15:08:02.417 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.417 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.418 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:02.418 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.255439+0000", 2020-08-07T15:08:02.418 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0027539309999999998 2020-08-07T15:08:02.418 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.418 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.419 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:02.419 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.258193+0000", 2020-08-07T15:08:02.419 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4655999999999996e-05 2020-08-07T15:08:02.419 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.419 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.420 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:02.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.258288+0000", 2020-08-07T15:08:02.423 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016215000000000001 2020-08-07T15:08:02.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.438 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.439 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:08:02.439 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.258450+0000", 2020-08-07T15:08:02.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0029743170000000002 2020-08-07T15:08:02.439 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.439 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.440 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:02.440 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.261424+0000", 2020-08-07T15:08:02.440 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4300000000000002e-07 2020-08-07T15:08:02.440 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.441 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:02.441 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.261425+0000", 2020-08-07T15:08:02.441 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0927999999999998e-05 2020-08-07T15:08:02.441 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.441 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.450 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:02.450 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.261476+0000", 2020-08-07T15:08:02.450 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0029306599999999999 2020-08-07T15:08:02.451 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.451 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.451 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:02.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.264407+0000", 2020-08-07T15:08:02.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0090447619999999996 2020-08-07T15:08:02.451 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:02.452 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:02.452 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:02.452 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:02.452 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:08:02.452 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 2 2020-08-07T15:08:02.453 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:02.454 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:08:02.455 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:02.454+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:02.562 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:02.558+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:02.662 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:02.660+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:02.690 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:02.690 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:08:02.691 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:08:02.691 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:08:02.691 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:02.708 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:08:02.770 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:02.765+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:02.882 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:02.869+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:02.933 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:02.933 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:08:02.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:08:02.935 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:08:02.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.935 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:15576 3.e e27/20 3:727724a5:::benchmark_data_smithi074_35447_object1946:head v 27'1032, mlcod=27'1023)", 2020-08-07T15:08:02.935 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:08:02.935 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 76.254482159999995, 2020-08-07T15:08:02.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.46091973000000003, 2020-08-07T15:08:02.936 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:02.936 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:02.936 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:02.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:02.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:08:02.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:02.937 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.938 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:02.938 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:08:02.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9592000000000001e-05 2020-08-07T15:08:02.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:02.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669542+0000", 2020-08-07T15:08:02.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.113e-06 2020-08-07T15:08:02.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.940 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:02.940 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669547+0000", 2020-08-07T15:08:02.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45151617799999999 2020-08-07T15:08:02.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.940 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:02.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121064+0000", 2020-08-07T15:08:02.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.377e-06 2020-08-07T15:08:02.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:02.942 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121067+0000", 2020-08-07T15:08:02.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9195e-05 2020-08-07T15:08:02.942 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.942 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:02.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121106+0000", 2020-08-07T15:08:02.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7509000000000001e-05 2020-08-07T15:08:02.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:02.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121124+0000", 2020-08-07T15:08:02.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0091783100000000003 2020-08-07T15:08:02.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.944 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:02.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.130302+0000", 2020-08-07T15:08:02.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000100456 2020-08-07T15:08:02.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:02.956 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.130402+0000", 2020-08-07T15:08:02.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.46091973000000003 2020-08-07T15:08:02.957 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:02.957 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:02.957 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:02.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.958 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:24264 3.8 e28/20 3:1d3fdbda:::benchmark_data_smithi074_35447_object3032:head v 28'1520, mlcod=28'1515)", 2020-08-07T15:08:02.958 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:08:02.958 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 70.540852122999993, 2020-08-07T15:08:02.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41856254199999998, 2020-08-07T15:08:02.959 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:02.959 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:02.959 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:02.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:02.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:08:02.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:02.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:02.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:08:02.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1348999999999998e-05 2020-08-07T15:08:02.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.962 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:02.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383174+0000", 2020-08-07T15:08:02.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6579999999999996e-06 2020-08-07T15:08:02.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:02.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383181+0000", 2020-08-07T15:08:02.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41194352699999998 2020-08-07T15:08:02.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:02.964 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.795124+0000", 2020-08-07T15:08:02.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5729999999999999e-06 2020-08-07T15:08:02.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:02.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.795128+0000", 2020-08-07T15:08:02.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002170384 2020-08-07T15:08:02.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:02.966 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.797298+0000", 2020-08-07T15:08:02.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5412e-05 2020-08-07T15:08:02.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:02.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.797324+0000", 2020-08-07T15:08:02.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042943360000000002 2020-08-07T15:08:02.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:02.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.801618+0000", 2020-08-07T15:08:02.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7302999999999997e-05 2020-08-07T15:08:02.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.969 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:02.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.801675+0000", 2020-08-07T15:08:02.969 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41856254199999998 2020-08-07T15:08:02.969 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:02.969 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:02.970 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:02.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.970 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:44120 3.e e32/20 3:7e6f7bb9:::benchmark_data_smithi074_35447_object5514:head v 32'2752, mlcod=32'2742)", 2020-08-07T15:08:02.970 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:04.453943+0000", 2020-08-07T15:08:02.971 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 58.470021897000002, 2020-08-07T15:08:02.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26204982500000001, 2020-08-07T15:08:02.971 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:02.971 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:02.971 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:02.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:02.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.453943+0000", 2020-08-07T15:08:02.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:02.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:02.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.453943+0000", 2020-08-07T15:08:02.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2278000000000003e-05 2020-08-07T15:08:02.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:02.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.453995+0000", 2020-08-07T15:08:02.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7050000000000002e-06 2020-08-07T15:08:02.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.975 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:02.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.454002+0000", 2020-08-07T15:08:02.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26083199299999998 2020-08-07T15:08:02.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:02.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.714834+0000", 2020-08-07T15:08:02.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3139999999999997e-06 2020-08-07T15:08:02.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:02.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.714838+0000", 2020-08-07T15:08:02.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8711000000000002e-05 2020-08-07T15:08:02.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.978 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:02.978 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.714887+0000", 2020-08-07T15:08:02.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7895e-05 2020-08-07T15:08:02.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.979 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:02.979 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.714915+0000", 2020-08-07T15:08:02.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010364 2020-08-07T15:08:02.979 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.979 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.979 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:02.980 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.715951+0000", 2020-08-07T15:08:02.980 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1529000000000002e-05 2020-08-07T15:08:02.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:02.981 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.715993+0000", 2020-08-07T15:08:02.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26204982500000001 2020-08-07T15:08:02.981 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:02.981 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:02.981 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:02.982 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.982 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:71136 3.6 3:64cb7acb:::benchmark_data_smithi074_35447_object8891:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e43)", 2020-08-07T15:08:02.982 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:24.312984+0000", 2020-08-07T15:08:02.982 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 38.610980705000003, 2020-08-07T15:08:02.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26986886799999998, 2020-08-07T15:08:02.983 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:02.983 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:02.983 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:02.983 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:02.984 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:02.984 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 71136 2020-08-07T15:08:02.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.984 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:02.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:02.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.312984+0000", 2020-08-07T15:08:02.985 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:02.985 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.985 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.986 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:02.986 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.312984+0000", 2020-08-07T15:08:02.986 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6711999999999997e-05 2020-08-07T15:08:02.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.986 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:02.987 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.313021+0000", 2020-08-07T15:08:02.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1999999999999999e-06 2020-08-07T15:08:02.987 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.987 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:02.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.313024+0000", 2020-08-07T15:08:02.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7479999999999993e-06 2020-08-07T15:08:02.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:02.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.313032+0000", 2020-08-07T15:08:02.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7379999999999999e-06 2020-08-07T15:08:02.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:02.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.313035+0000", 2020-08-07T15:08:02.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043807819999999997 2020-08-07T15:08:02.990 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.990 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.990 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:02.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.317415+0000", 2020-08-07T15:08:02.991 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00042567899999999999 2020-08-07T15:08:02.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:02.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.317841+0000", 2020-08-07T15:08:02.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000173048 2020-08-07T15:08:02.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:02.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318014+0000", 2020-08-07T15:08:02.993 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00099031100000000001 2020-08-07T15:08:02.993 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.993 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.993 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:02.993 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.319004+0000", 2020-08-07T15:08:02.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.75e-07 2020-08-07T15:08:02.994 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.994 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.994 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:02.994 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.319005+0000", 2020-08-07T15:08:02.995 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0659000000000003e-05 2020-08-07T15:08:02.995 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.995 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.995 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:02.996 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.319066+0000", 2020-08-07T15:08:02.996 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023319909999999998 2020-08-07T15:08:02.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.996 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.997 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:02.997 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.321398+0000", 2020-08-07T15:08:02.997 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26138581700000002 2020-08-07T15:08:02.997 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.997 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.998 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:02.998 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.582783+0000", 2020-08-07T15:08:02.998 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.19e-06 2020-08-07T15:08:02.998 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.998 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:02.999 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:02.999 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.582785+0000", 2020-08-07T15:08:02.999 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8417999999999994e-05 2020-08-07T15:08:02.999 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:02.999 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.000 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:03.000 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.582853+0000", 2020-08-07T15:08:03.000 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26986886799999998 2020-08-07T15:08:03.000 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.000 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:03.000 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.001 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.001 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.001 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:72616 3.6 3:64e83676:::benchmark_data_smithi074_35447_object9076:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e44)", 2020-08-07T15:08:03.001 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:25.246439+0000", 2020-08-07T15:08:03.003 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 37.677525469000003, 2020-08-07T15:08:03.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27754408000000003, 2020-08-07T15:08:03.004 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:03.004 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:03.004 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:03.004 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:03.004 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:03.005 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72616 2020-08-07T15:08:03.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.005 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:03.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:03.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.246439+0000", 2020-08-07T15:08:03.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:03.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:03.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.246439+0000", 2020-08-07T15:08:03.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0911000000000003e-05 2020-08-07T15:08:03.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:03.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.246470+0000", 2020-08-07T15:08:03.008 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7080000000000002e-06 2020-08-07T15:08:03.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:03.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.246473+0000", 2020-08-07T15:08:03.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4199999999999998e-06 2020-08-07T15:08:03.009 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.009 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.009 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:03.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.246478+0000", 2020-08-07T15:08:03.010 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5229999999999998e-06 2020-08-07T15:08:03.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.010 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:03.010 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.246481+0000", 2020-08-07T15:08:03.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024729550000000002 2020-08-07T15:08:03.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.011 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:03.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.248954+0000", 2020-08-07T15:08:03.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0921999999999994e-05 2020-08-07T15:08:03.012 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.012 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:03.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.249045+0000", 2020-08-07T15:08:03.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000158875 2020-08-07T15:08:03.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.013 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:03.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.249204+0000", 2020-08-07T15:08:03.013 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003988068 2020-08-07T15:08:03.014 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.014 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:03.016 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.253192+0000", 2020-08-07T15:08:03.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7400000000000003e-07 2020-08-07T15:08:03.016 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:03.017 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.253192+0000", 2020-08-07T15:08:03.017 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7210000000000003e-05 2020-08-07T15:08:03.017 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.018 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:03.018 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.253250+0000", 2020-08-07T15:08:03.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00069997899999999999 2020-08-07T15:08:03.018 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.018 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.019 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:03.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.253949+0000", 2020-08-07T15:08:03.019 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26998122600000002 2020-08-07T15:08:03.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.019 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.020 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:03.020 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.523931+0000", 2020-08-07T15:08:03.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6339999999999999e-06 2020-08-07T15:08:03.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.020 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.020 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:03.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.523932+0000", 2020-08-07T15:08:03.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1075000000000001e-05 2020-08-07T15:08:03.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:03.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.523983+0000", 2020-08-07T15:08:03.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27754408000000003 2020-08-07T15:08:03.028 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.028 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:03.028 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.029 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:78743 3.6 3:6151aa5d:::benchmark_data_smithi074_35447_object9842:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e47)", 2020-08-07T15:08:03.029 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:29.249535+0000", 2020-08-07T15:08:03.029 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 33.674429351999997, 2020-08-07T15:08:03.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.270038371, 2020-08-07T15:08:03.029 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:03.030 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:03.030 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:03.030 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:03.030 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:03.030 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 78743 2020-08-07T15:08:03.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.031 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:03.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:03.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249535+0000", 2020-08-07T15:08:03.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:03.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.034 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.034 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:03.034 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249535+0000", 2020-08-07T15:08:03.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1631000000000001e-05 2020-08-07T15:08:03.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:03.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249587+0000", 2020-08-07T15:08:03.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3969999999999999e-06 2020-08-07T15:08:03.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:03.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249593+0000", 2020-08-07T15:08:03.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2312e-05 2020-08-07T15:08:03.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.037 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:03.037 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249606+0000", 2020-08-07T15:08:03.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7379999999999997e-06 2020-08-07T15:08:03.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:03.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249613+0000", 2020-08-07T15:08:03.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0018089250000000001 2020-08-07T15:08:03.039 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.039 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.039 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:03.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.251421+0000", 2020-08-07T15:08:03.040 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4716999999999995e-05 2020-08-07T15:08:03.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:03.040 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.251516+0000", 2020-08-07T15:08:03.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000190061 2020-08-07T15:08:03.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.041 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:03.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.251706+0000", 2020-08-07T15:08:03.042 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013539520000000001 2020-08-07T15:08:03.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.042 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:03.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.253060+0000", 2020-08-07T15:08:03.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019342299999999999 2020-08-07T15:08:03.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:03.043 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.253254+0000", 2020-08-07T15:08:03.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26620518599999998 2020-08-07T15:08:03.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.044 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:03.044 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.519459+0000", 2020-08-07T15:08:03.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2109999999999999e-06 2020-08-07T15:08:03.045 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.045 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:03.045 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.519460+0000", 2020-08-07T15:08:03.045 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6767000000000006e-05 2020-08-07T15:08:03.046 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.046 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.046 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:03.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.519527+0000", 2020-08-07T15:08:03.046 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7051000000000001e-05 2020-08-07T15:08:03.047 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.047 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:03.047 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.519574+0000", 2020-08-07T15:08:03.047 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.270038371 2020-08-07T15:08:03.047 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.048 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:03.048 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.048 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.048 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.048 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:78744 3.6 3:6151aa5d:::benchmark_data_smithi074_35447_object9842:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e47)", 2020-08-07T15:08:03.049 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:29.249649+0000", 2020-08-07T15:08:03.049 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 33.674315702999998, 2020-08-07T15:08:03.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27244544500000001, 2020-08-07T15:08:03.049 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:03.049 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:03.050 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:03.050 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:03.050 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:03.050 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 78744 2020-08-07T15:08:03.050 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.051 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:03.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.051 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:03.051 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249649+0000", 2020-08-07T15:08:03.051 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:03.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.052 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.052 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:03.052 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249649+0000", 2020-08-07T15:08:03.052 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6628000000000001e-05 2020-08-07T15:08:03.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.053 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:03.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249696+0000", 2020-08-07T15:08:03.053 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4209999999999998e-06 2020-08-07T15:08:03.054 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.054 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.054 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:03.054 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249703+0000", 2020-08-07T15:08:03.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2456999999999999e-05 2020-08-07T15:08:03.055 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.055 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:03.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249716+0000", 2020-08-07T15:08:03.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1030000000000001e-06 2020-08-07T15:08:03.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:03.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249721+0000", 2020-08-07T15:08:03.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024347980000000002 2020-08-07T15:08:03.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:03.057 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.252156+0000", 2020-08-07T15:08:03.058 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3114e-05 2020-08-07T15:08:03.058 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.058 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.058 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:03.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.252209+0000", 2020-08-07T15:08:03.059 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3654999999999997e-05 2020-08-07T15:08:03.059 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.059 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:03.060 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.252302+0000", 2020-08-07T15:08:03.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001130437 2020-08-07T15:08:03.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.061 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:03.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.253433+0000", 2020-08-07T15:08:03.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5849999999999997e-05 2020-08-07T15:08:03.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.062 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:03.062 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.253469+0000", 2020-08-07T15:08:03.062 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26856134399999998 2020-08-07T15:08:03.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.063 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:03.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.522030+0000", 2020-08-07T15:08:03.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0500000000000002e-07 2020-08-07T15:08:03.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.064 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:03.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.522031+0000", 2020-08-07T15:08:03.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8105000000000002e-05 2020-08-07T15:08:03.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.065 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:03.065 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.522069+0000", 2020-08-07T15:08:03.065 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5728000000000002e-05 2020-08-07T15:08:03.066 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.066 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.066 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:03.066 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.522095+0000", 2020-08-07T15:08:03.066 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27244544500000001 2020-08-07T15:08:03.066 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.067 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:03.067 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.067 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.067 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.068 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:86712 3.8 e52/20 3:14925daa:::benchmark_data_smithi074_35447_object10838:head v 52'5488, mlcod=52'5479)", 2020-08-07T15:08:03.068 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.337908+0000", 2020-08-07T15:08:03.068 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 28.586056388999999, 2020-08-07T15:08:03.068 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.305870904, 2020-08-07T15:08:03.068 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:03.069 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:03.069 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:03.069 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.069 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:03.069 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.337908+0000", 2020-08-07T15:08:03.070 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:03.070 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.070 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.071 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:03.071 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.337908+0000", 2020-08-07T15:08:03.071 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1465999999999999e-05 2020-08-07T15:08:03.071 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.071 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.072 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:03.072 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.337950+0000", 2020-08-07T15:08:03.072 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1810000000000003e-06 2020-08-07T15:08:03.072 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.072 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.072 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:03.073 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.337954+0000", 2020-08-07T15:08:03.073 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30500923899999999 2020-08-07T15:08:03.073 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.073 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.073 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:03.074 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.642963+0000", 2020-08-07T15:08:03.074 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5389999999999998e-06 2020-08-07T15:08:03.074 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.074 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.074 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:03.075 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.642969+0000", 2020-08-07T15:08:03.075 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8265e-05 2020-08-07T15:08:03.075 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.075 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.075 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:03.075 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.643017+0000", 2020-08-07T15:08:03.076 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6812999999999999e-05 2020-08-07T15:08:03.076 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.076 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.076 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:03.076 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.643044+0000", 2020-08-07T15:08:03.077 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00067619800000000003 2020-08-07T15:08:03.077 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.077 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.077 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:03.078 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.643720+0000", 2020-08-07T15:08:03.078 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9203000000000002e-05 2020-08-07T15:08:03.078 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.078 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.078 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:03.079 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.643779+0000", 2020-08-07T15:08:03.079 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.305870904 2020-08-07T15:08:03.079 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.079 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:03.079 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.080 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.080 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.080 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:87087 3.6 3:6b0320df:::benchmark_data_smithi074_35447_object10885:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:08:03.080 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.958471+0000", 2020-08-07T15:08:03.080 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 27.965493452, 2020-08-07T15:08:03.081 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30431862500000001, 2020-08-07T15:08:03.081 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:03.081 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:03.081 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:03.081 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:03.081 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:03.082 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 87087 2020-08-07T15:08:03.082 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.082 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:03.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.082 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:03.083 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958471+0000", 2020-08-07T15:08:03.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:03.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.083 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.083 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:03.084 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958471+0000", 2020-08-07T15:08:03.084 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3241999999999997e-05 2020-08-07T15:08:03.084 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.084 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.085 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:03.085 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958515+0000", 2020-08-07T15:08:03.085 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4930000000000003e-06 2020-08-07T15:08:03.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.086 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:03.086 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958521+0000", 2020-08-07T15:08:03.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2308e-05 2020-08-07T15:08:03.086 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.086 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:03.087 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958533+0000", 2020-08-07T15:08:03.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.105e-06 2020-08-07T15:08:03.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:03.088 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958540+0000", 2020-08-07T15:08:03.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0064000089999999999 2020-08-07T15:08:03.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:03.089 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.964940+0000", 2020-08-07T15:08:03.089 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000159581 2020-08-07T15:08:03.089 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.089 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.089 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:03.090 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.965099+0000", 2020-08-07T15:08:03.090 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000160743 2020-08-07T15:08:03.090 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.090 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.090 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:03.091 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.965260+0000", 2020-08-07T15:08:03.091 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002913559 2020-08-07T15:08:03.091 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.091 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.091 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:03.092 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.968173+0000", 2020-08-07T15:08:03.092 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5000000000000001e-07 2020-08-07T15:08:03.092 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.092 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.092 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:03.092 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.968174+0000", 2020-08-07T15:08:03.093 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6161999999999994e-05 2020-08-07T15:08:03.093 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.093 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.093 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:03.093 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.968241+0000", 2020-08-07T15:08:03.094 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.010959649 2020-08-07T15:08:03.094 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.094 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.094 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:03.094 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.979200+0000", 2020-08-07T15:08:03.095 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28352280699999999 2020-08-07T15:08:03.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.095 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.095 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:03.095 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262723+0000", 2020-08-07T15:08:03.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.19e-06 2020-08-07T15:08:03.096 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.096 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.096 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:03.096 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262724+0000", 2020-08-07T15:08:03.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5827000000000005e-05 2020-08-07T15:08:03.097 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.097 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.097 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:03.097 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262790+0000", 2020-08-07T15:08:03.097 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30431862500000001 2020-08-07T15:08:03.098 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.098 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:03.098 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.098 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.098 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.099 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:87088 3.6 3:6b0320df:::benchmark_data_smithi074_35447_object10885:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:08:03.099 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.958567+0000", 2020-08-07T15:08:03.099 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 27.965397831000001, 2020-08-07T15:08:03.099 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30442327000000002, 2020-08-07T15:08:03.099 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:03.100 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:03.100 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:03.100 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:03.101 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:03.101 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 87088 2020-08-07T15:08:03.101 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.101 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:03.101 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.102 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:03.102 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958567+0000", 2020-08-07T15:08:03.102 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:03.102 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.102 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.102 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:03.103 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958567+0000", 2020-08-07T15:08:03.103 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5040000000000002e-05 2020-08-07T15:08:03.103 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.103 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.103 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:03.104 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958612+0000", 2020-08-07T15:08:03.104 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4519999999999996e-06 2020-08-07T15:08:03.104 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.104 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.104 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:03.105 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958617+0000", 2020-08-07T15:08:03.105 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3974e-05 2020-08-07T15:08:03.105 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.105 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.105 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:03.106 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958631+0000", 2020-08-07T15:08:03.106 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5619999999999997e-06 2020-08-07T15:08:03.106 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.106 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.107 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:03.107 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958636+0000", 2020-08-07T15:08:03.107 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0071073999999999998 2020-08-07T15:08:03.107 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.107 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.108 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:03.108 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.965743+0000", 2020-08-07T15:08:03.108 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5477999999999996e-05 2020-08-07T15:08:03.108 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.108 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.108 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:03.109 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.965839+0000", 2020-08-07T15:08:03.109 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000168061 2020-08-07T15:08:03.109 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.109 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.109 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:03.110 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966007+0000", 2020-08-07T15:08:03.110 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054997550000000003 2020-08-07T15:08:03.110 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.110 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.110 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:03.111 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971507+0000", 2020-08-07T15:08:03.111 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6200000000000006e-07 2020-08-07T15:08:03.111 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.111 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.111 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:03.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971508+0000", 2020-08-07T15:08:03.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012566300000000001 2020-08-07T15:08:03.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.112 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:03.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971633+0000", 2020-08-07T15:08:03.113 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0075941899999999998 2020-08-07T15:08:03.113 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.113 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:03.113 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.979228+0000", 2020-08-07T15:08:03.114 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28372433899999999 2020-08-07T15:08:03.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.114 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:03.114 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262952+0000", 2020-08-07T15:08:03.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2799999999999995e-07 2020-08-07T15:08:03.115 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.115 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.116 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:03.116 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262953+0000", 2020-08-07T15:08:03.117 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7666000000000002e-05 2020-08-07T15:08:03.117 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.117 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:03.118 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262990+0000", 2020-08-07T15:08:03.118 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30442327000000002 2020-08-07T15:08:03.118 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.118 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:03.118 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.119 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.119 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.119 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:87087 3.6 e52/26 ondisk, result = 0)", 2020-08-07T15:08:03.119 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.971041+0000", 2020-08-07T15:08:03.119 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 27.952923921, 2020-08-07T15:08:03.120 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29175585500000001, 2020-08-07T15:08:03.120 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:03.120 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:03.121 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:03.121 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.121 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:03.121 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971041+0000", 2020-08-07T15:08:03.122 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:03.122 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.122 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.122 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:03.123 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971041+0000", 2020-08-07T15:08:03.123 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7841999999999999e-05 2020-08-07T15:08:03.123 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.123 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.123 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:03.124 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971069+0000", 2020-08-07T15:08:03.124 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7110000000000003e-06 2020-08-07T15:08:03.124 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.124 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.124 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:03.124 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971073+0000", 2020-08-07T15:08:03.125 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28965561299999998 2020-08-07T15:08:03.125 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.125 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.125 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:03.125 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260729+0000", 2020-08-07T15:08:03.126 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8010000000000003e-06 2020-08-07T15:08:03.126 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.126 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.126 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:03.126 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260734+0000", 2020-08-07T15:08:03.127 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019719500000000001 2020-08-07T15:08:03.127 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.127 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.127 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:03.127 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262706+0000", 2020-08-07T15:08:03.127 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7399999999999993e-06 2020-08-07T15:08:03.128 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.128 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.128 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:03.129 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262715+0000", 2020-08-07T15:08:03.129 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2198000000000001e-05 2020-08-07T15:08:03.129 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.129 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.130 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:03.130 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262797+0000", 2020-08-07T15:08:03.130 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29175585500000001 2020-08-07T15:08:03.130 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.130 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:03.130 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.131 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.131 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.131 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:93920 3.6 e56/26 ondisk, result = 0)", 2020-08-07T15:08:03.131 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:08:03.131 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 23.593488860000001, 2020-08-07T15:08:03.132 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.397576347, 2020-08-07T15:08:03.132 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:03.132 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:03.132 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:03.132 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.133 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:03.134 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:08:03.134 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:03.134 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.134 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.135 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:03.135 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:08:03.135 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2571999999999999e-05 2020-08-07T15:08:03.135 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.135 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.136 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:03.136 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330509+0000", 2020-08-07T15:08:03.136 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2059999999999998e-06 2020-08-07T15:08:03.136 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.136 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.137 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:03.137 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330514+0000", 2020-08-07T15:08:03.137 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39740134599999999 2020-08-07T15:08:03.137 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.137 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.139 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:03.139 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727915+0000", 2020-08-07T15:08:03.139 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.677e-06 2020-08-07T15:08:03.139 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.139 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.140 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:03.140 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727921+0000", 2020-08-07T15:08:03.140 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5431e-05 2020-08-07T15:08:03.140 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.140 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.141 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:03.141 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727956+0000", 2020-08-07T15:08:03.141 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5790000000000004e-06 2020-08-07T15:08:03.141 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.141 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.142 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:03.142 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727965+0000", 2020-08-07T15:08:03.142 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7535999999999997e-05 2020-08-07T15:08:03.142 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.142 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.143 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:03.143 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.728052+0000", 2020-08-07T15:08:03.143 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.397576347 2020-08-07T15:08:03.143 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.143 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:03.144 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.144 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.144 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.144 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:104696 3.6 3:680df183:::benchmark_data_smithi074_35447_object13086:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:08:03.144 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:46.386692+0000", 2020-08-07T15:08:03.144 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.537273124999999, 2020-08-07T15:08:03.145 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26505562500000002, 2020-08-07T15:08:03.145 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:03.145 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:03.145 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:03.145 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:03.146 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:03.146 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 104696 2020-08-07T15:08:03.146 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.146 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:03.148 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.148 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:03.148 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.386692+0000", 2020-08-07T15:08:03.149 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:03.149 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.149 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.151 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:03.151 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.386692+0000", 2020-08-07T15:08:03.151 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0942000000000001e-05 2020-08-07T15:08:03.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:03.152 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.386743+0000", 2020-08-07T15:08:03.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9279999999999998e-06 2020-08-07T15:08:03.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.153 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:03.153 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.386749+0000", 2020-08-07T15:08:03.153 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1221e-05 2020-08-07T15:08:03.153 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.154 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.154 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:03.154 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.386760+0000", 2020-08-07T15:08:03.154 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:08:03.155 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.155 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.155 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:03.155 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.386765+0000", 2020-08-07T15:08:03.155 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001868219 2020-08-07T15:08:03.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.156 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:03.156 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.388633+0000", 2020-08-07T15:08:03.156 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010592800000000001 2020-08-07T15:08:03.157 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.157 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.157 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:03.157 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.388739+0000", 2020-08-07T15:08:03.157 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017264599999999999 2020-08-07T15:08:03.158 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.158 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.158 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:03.159 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.388912+0000", 2020-08-07T15:08:03.159 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0030961729999999998 2020-08-07T15:08:03.159 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.159 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.159 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:03.160 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.392008+0000", 2020-08-07T15:08:03.160 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1909999999999999e-06 2020-08-07T15:08:03.160 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.160 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.160 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:03.161 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.392009+0000", 2020-08-07T15:08:03.161 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8398000000000005e-05 2020-08-07T15:08:03.162 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.162 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.162 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:03.162 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.392077+0000", 2020-08-07T15:08:03.162 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001373434 2020-08-07T15:08:03.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.163 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.163 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:03.163 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.393451+0000", 2020-08-07T15:08:03.163 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25771527799999999 2020-08-07T15:08:03.164 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:03.164 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.651166+0000", 2020-08-07T15:08:03.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.322e-06 2020-08-07T15:08:03.165 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.165 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.165 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:03.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.651167+0000", 2020-08-07T15:08:03.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00058001099999999996 2020-08-07T15:08:03.166 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.166 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.166 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:03.166 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.651747+0000", 2020-08-07T15:08:03.166 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26505562500000002 2020-08-07T15:08:03.167 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.167 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:03.167 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.167 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.167 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.168 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106104 3.0 3:0d7a2a89:::benchmark_data_smithi074_35447_object13262:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:08:03.168 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.252296+0000", 2020-08-07T15:08:03.168 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 15.671668971000001, 2020-08-07T15:08:03.168 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.44182183200000003, 2020-08-07T15:08:03.168 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:03.169 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:03.169 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:03.169 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:03.169 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:03.169 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106104 2020-08-07T15:08:03.170 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.170 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:03.170 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.170 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:03.170 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252296+0000", 2020-08-07T15:08:03.170 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:03.171 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.171 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.171 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:03.171 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252296+0000", 2020-08-07T15:08:03.171 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4728000000000001e-05 2020-08-07T15:08:03.172 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.172 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.172 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:03.172 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252321+0000", 2020-08-07T15:08:03.172 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9529999999999999e-06 2020-08-07T15:08:03.173 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.173 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.173 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:03.173 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252324+0000", 2020-08-07T15:08:03.173 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5759999999999996e-06 2020-08-07T15:08:03.174 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.174 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.174 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:03.174 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252329+0000", 2020-08-07T15:08:03.174 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.582e-06 2020-08-07T15:08:03.174 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.175 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.175 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:03.175 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252332+0000", 2020-08-07T15:08:03.175 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012958519999999999 2020-08-07T15:08:03.175 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.176 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.176 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:03.176 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.253628+0000", 2020-08-07T15:08:03.176 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7871e-05 2020-08-07T15:08:03.176 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.177 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.177 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:03.177 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.253705+0000", 2020-08-07T15:08:03.177 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017801199999999999 2020-08-07T15:08:03.177 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.178 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.178 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:08:03.178 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.253883+0000", 2020-08-07T15:08:03.178 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001869051 2020-08-07T15:08:03.178 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.178 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.179 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:03.179 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255752+0000", 2020-08-07T15:08:03.179 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3e-07 2020-08-07T15:08:03.179 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.179 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.180 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:03.180 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255753+0000", 2020-08-07T15:08:03.180 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5321000000000001e-05 2020-08-07T15:08:03.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.180 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.181 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:03.181 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255808+0000", 2020-08-07T15:08:03.181 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.091917211999999998 2020-08-07T15:08:03.181 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.181 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:03.182 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.347726+0000", 2020-08-07T15:08:03.182 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.34634211500000001 2020-08-07T15:08:03.182 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:03.183 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694068+0000", 2020-08-07T15:08:03.183 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3030000000000001e-06 2020-08-07T15:08:03.183 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.183 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.183 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:03.184 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694069+0000", 2020-08-07T15:08:03.184 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8625999999999998e-05 2020-08-07T15:08:03.184 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.184 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.185 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:03.185 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694118+0000", 2020-08-07T15:08:03.185 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.44182183200000003 2020-08-07T15:08:03.185 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.186 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:03.186 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.186 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.186 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.186 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106104 3.0 e58/20 ondisk, result = 0)", 2020-08-07T15:08:03.187 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.257196+0000", 2020-08-07T15:08:03.187 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 15.666768464, 2020-08-07T15:08:03.187 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43692689000000001, 2020-08-07T15:08:03.187 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:03.187 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:03.188 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:03.188 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.196 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:03.196 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257196+0000", 2020-08-07T15:08:03.196 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:03.197 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.197 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.197 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:03.197 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257196+0000", 2020-08-07T15:08:03.197 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2021999999999999e-05 2020-08-07T15:08:03.198 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.198 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.198 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:03.198 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257228+0000", 2020-08-07T15:08:03.198 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9549999999999997e-06 2020-08-07T15:08:03.199 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.199 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.199 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:03.199 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257234+0000", 2020-08-07T15:08:03.199 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43671786400000001 2020-08-07T15:08:03.200 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.200 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.200 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:03.200 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693952+0000", 2020-08-07T15:08:03.200 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7690000000000004e-06 2020-08-07T15:08:03.201 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.201 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.201 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:03.201 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693957+0000", 2020-08-07T15:08:03.201 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3294000000000006e-05 2020-08-07T15:08:03.202 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.202 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.202 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:03.202 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694040+0000", 2020-08-07T15:08:03.202 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3436e-05 2020-08-07T15:08:03.203 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.203 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.203 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:03.203 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694054+0000", 2020-08-07T15:08:03.203 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9549999999999996e-05 2020-08-07T15:08:03.204 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.204 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.204 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:03.204 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694123+0000", 2020-08-07T15:08:03.204 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43692689000000001 2020-08-07T15:08:03.204 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.205 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:03.205 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.205 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.205 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106144 3.6 3:629055a5:::benchmark_data_smithi074_35447_object13267:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:08:03.206 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.280142+0000", 2020-08-07T15:08:03.206 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 15.643823002, 2020-08-07T15:08:03.206 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.57968657199999996, 2020-08-07T15:08:03.206 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:03.206 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:03.207 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:03.207 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:03.207 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:03.207 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106144 2020-08-07T15:08:03.207 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.208 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:03.208 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.208 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:03.208 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280142+0000", 2020-08-07T15:08:03.208 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:03.208 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.209 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.209 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:03.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280142+0000", 2020-08-07T15:08:03.209 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.905e-05 2020-08-07T15:08:03.209 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.210 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.210 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:03.210 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280191+0000", 2020-08-07T15:08:03.210 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6729999999999997e-06 2020-08-07T15:08:03.210 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.211 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.211 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:03.211 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280197+0000", 2020-08-07T15:08:03.211 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0747999999999999e-05 2020-08-07T15:08:03.211 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.212 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.212 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:03.212 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280207+0000", 2020-08-07T15:08:03.212 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7700000000000001e-06 2020-08-07T15:08:03.212 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.212 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.213 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:03.213 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280212+0000", 2020-08-07T15:08:03.213 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00060223399999999999 2020-08-07T15:08:03.213 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.213 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.214 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:03.214 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280814+0000", 2020-08-07T15:08:03.214 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7831000000000006e-05 2020-08-07T15:08:03.214 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.214 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.215 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:03.215 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280912+0000", 2020-08-07T15:08:03.215 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000137119 2020-08-07T15:08:03.215 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.215 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.216 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:03.216 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.281049+0000", 2020-08-07T15:08:03.217 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023794369999999999 2020-08-07T15:08:03.217 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.217 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.217 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:03.218 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283429+0000", 2020-08-07T15:08:03.218 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.173e-06 2020-08-07T15:08:03.218 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.218 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.218 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:03.219 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283430+0000", 2020-08-07T15:08:03.219 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5530000000000004e-05 2020-08-07T15:08:03.219 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.219 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.219 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:03.220 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283495+0000", 2020-08-07T15:08:03.220 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5649000000000003e-05 2020-08-07T15:08:03.220 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.220 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.220 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:03.221 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283541+0000", 2020-08-07T15:08:03.221 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.57624206300000003 2020-08-07T15:08:03.221 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.221 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.221 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:03.222 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859783+0000", 2020-08-07T15:08:03.222 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.421e-06 2020-08-07T15:08:03.222 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.222 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.223 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:03.223 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859785+0000", 2020-08-07T15:08:03.223 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3874e-05 2020-08-07T15:08:03.223 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.223 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.223 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:03.224 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859828+0000", 2020-08-07T15:08:03.224 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.57968657199999996 2020-08-07T15:08:03.224 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.224 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:03.224 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.225 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.225 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.225 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106144 3.6 e59/26 ondisk, result = 0)", 2020-08-07T15:08:03.225 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.344536+0000", 2020-08-07T15:08:03.225 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 15.57942866, 2020-08-07T15:08:03.226 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51529786099999997, 2020-08-07T15:08:03.226 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:03.226 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:03.226 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:03.226 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.227 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:03.227 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344536+0000", 2020-08-07T15:08:03.227 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:03.227 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.227 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.227 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:03.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344536+0000", 2020-08-07T15:08:03.228 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7266999999999999e-05 2020-08-07T15:08:03.228 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.228 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.228 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:03.229 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344553+0000", 2020-08-07T15:08:03.229 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1070000000000001e-06 2020-08-07T15:08:03.229 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.229 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.229 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:03.230 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344557+0000", 2020-08-07T15:08:03.230 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.515143201 2020-08-07T15:08:03.230 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.230 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.230 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:03.230 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859700+0000", 2020-08-07T15:08:03.231 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.271e-05 2020-08-07T15:08:03.239 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.239 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.239 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:03.239 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859712+0000", 2020-08-07T15:08:03.240 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5436999999999998e-05 2020-08-07T15:08:03.240 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.240 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.240 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:03.240 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859768+0000", 2020-08-07T15:08:03.241 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1540000000000002e-06 2020-08-07T15:08:03.241 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.241 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.241 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:03.241 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859776+0000", 2020-08-07T15:08:03.245 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7985000000000001e-05 2020-08-07T15:08:03.245 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.245 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.245 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:03.246 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859834+0000", 2020-08-07T15:08:03.246 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51529786099999997 2020-08-07T15:08:03.246 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.246 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:03.246 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.247 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.247 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.247 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108352 3.8 3:1aac8373:::benchmark_data_smithi074_35447_object13543:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e62)", 2020-08-07T15:08:03.247 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.642093+0000", 2020-08-07T15:08:03.247 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 12.281871970999999, 2020-08-07T15:08:03.248 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39397409900000002, 2020-08-07T15:08:03.248 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:03.248 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:03.248 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:03.248 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:03.249 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:03.249 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108352 2020-08-07T15:08:03.249 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.249 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:03.250 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.250 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:03.250 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642093+0000", 2020-08-07T15:08:03.250 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:03.250 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.251 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.251 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:03.251 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642093+0000", 2020-08-07T15:08:03.251 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9475000000000002e-05 2020-08-07T15:08:03.251 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.252 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.252 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:03.252 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642122+0000", 2020-08-07T15:08:03.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0439999999999997e-06 2020-08-07T15:08:03.252 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.253 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:03.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642126+0000", 2020-08-07T15:08:03.253 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9689999999999994e-06 2020-08-07T15:08:03.253 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.254 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:03.254 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642134+0000", 2020-08-07T15:08:03.254 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1219999999999997e-06 2020-08-07T15:08:03.254 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.254 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.255 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:03.255 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642138+0000", 2020-08-07T15:08:03.255 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011580399999999999 2020-08-07T15:08:03.255 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.255 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.256 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:03.256 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.643296+0000", 2020-08-07T15:08:03.256 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3791999999999999e-05 2020-08-07T15:08:03.256 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.256 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.257 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:03.257 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.643390+0000", 2020-08-07T15:08:03.257 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011882700000000001 2020-08-07T15:08:03.257 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.257 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.257 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4,7", 2020-08-07T15:08:03.258 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.643509+0000", 2020-08-07T15:08:03.258 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034736799999999998 2020-08-07T15:08:03.258 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.258 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.258 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:03.259 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.646983+0000", 2020-08-07T15:08:03.259 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0077748490000000003 2020-08-07T15:08:03.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.259 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.259 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:03.260 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.654758+0000", 2020-08-07T15:08:03.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000005e-07 2020-08-07T15:08:03.260 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.260 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.260 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:03.261 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.654758+0000", 2020-08-07T15:08:03.261 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4258000000000002e-05 2020-08-07T15:08:03.261 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.261 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.261 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:03.263 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.654802+0000", 2020-08-07T15:08:03.263 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11166224399999999 2020-08-07T15:08:03.263 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.263 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.263 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:03.264 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.766465+0000", 2020-08-07T15:08:03.264 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26946740899999999 2020-08-07T15:08:03.264 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.264 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.264 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:03.265 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.035932+0000", 2020-08-07T15:08:03.265 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5829999999999999e-06 2020-08-07T15:08:03.265 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.265 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.265 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:03.266 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.035934+0000", 2020-08-07T15:08:03.266 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000133306 2020-08-07T15:08:03.266 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.266 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.266 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:03.267 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.036067+0000", 2020-08-07T15:08:03.267 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39397409900000002 2020-08-07T15:08:03.267 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.267 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:03.267 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.267 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.268 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.268 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108464 3.6 3:62704f92:::benchmark_data_smithi074_35447_object13557:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e62)", 2020-08-07T15:08:03.268 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.770928+0000", 2020-08-07T15:08:03.268 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 12.15303683, 2020-08-07T15:08:03.268 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30592449999999999, 2020-08-07T15:08:03.269 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:03.269 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:03.269 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:03.269 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:03.269 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:03.271 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108464 2020-08-07T15:08:03.271 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.271 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:03.271 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:03.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770928+0000", 2020-08-07T15:08:03.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:03.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:03.273 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770928+0000", 2020-08-07T15:08:03.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7257e-05 2020-08-07T15:08:03.273 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.273 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:03.274 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770955+0000", 2020-08-07T15:08:03.275 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0080000000000004e-06 2020-08-07T15:08:03.275 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.275 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.275 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:03.275 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770959+0000", 2020-08-07T15:08:03.276 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2810000000000003e-06 2020-08-07T15:08:03.276 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.276 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.276 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:03.276 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770967+0000", 2020-08-07T15:08:03.277 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3850000000000004e-06 2020-08-07T15:08:03.277 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.277 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.277 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:03.277 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770971+0000", 2020-08-07T15:08:03.278 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4960000000000002e-05 2020-08-07T15:08:03.279 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.279 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.279 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:03.279 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770996+0000", 2020-08-07T15:08:03.279 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4565999999999999e-05 2020-08-07T15:08:03.280 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.280 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.280 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:03.280 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.771050+0000", 2020-08-07T15:08:03.280 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010631799999999999 2020-08-07T15:08:03.281 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.281 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.281 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:03.281 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.771157+0000", 2020-08-07T15:08:03.281 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0049076099999999998 2020-08-07T15:08:03.282 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.282 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.283 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:03.283 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.776064+0000", 2020-08-07T15:08:03.283 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3700000000000001e-07 2020-08-07T15:08:03.283 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.283 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.284 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:03.284 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.776065+0000", 2020-08-07T15:08:03.284 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8387999999999998e-05 2020-08-07T15:08:03.284 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.284 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.285 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:03.285 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.776113+0000", 2020-08-07T15:08:03.285 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00068335599999999998 2020-08-07T15:08:03.285 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.285 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.286 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:03.286 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.776797+0000", 2020-08-07T15:08:03.286 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30001175499999999 2020-08-07T15:08:03.286 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.287 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:03.287 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.076808+0000", 2020-08-07T15:08:03.287 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4200000000000004e-07 2020-08-07T15:08:03.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.288 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.288 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:03.288 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.076809+0000", 2020-08-07T15:08:03.288 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3237e-05 2020-08-07T15:08:03.288 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.289 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.289 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:03.289 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.076852+0000", 2020-08-07T15:08:03.289 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30592449999999999 2020-08-07T15:08:03.289 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.290 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:03.290 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.290 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.290 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.290 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:116064 3.6 3:6cc03ffa:::benchmark_data_smithi074_35447_object14507:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e64)", 2020-08-07T15:08:03.291 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:56.361361+0000", 2020-08-07T15:08:03.291 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.5626034549999996, 2020-08-07T15:08:03.291 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26063054099999999, 2020-08-07T15:08:03.291 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:03.291 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:03.292 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:03.292 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:03.292 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:03.292 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 116064 2020-08-07T15:08:03.292 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.293 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:03.293 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.293 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:03.293 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.361361+0000", 2020-08-07T15:08:03.293 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:03.294 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.294 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.294 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:03.294 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.361361+0000", 2020-08-07T15:08:03.294 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8124999999999999e-05 2020-08-07T15:08:03.294 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.295 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.295 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:03.295 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.361389+0000", 2020-08-07T15:08:03.295 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0439999999999997e-06 2020-08-07T15:08:03.295 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.296 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.296 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:03.296 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.361394+0000", 2020-08-07T15:08:03.296 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8110000000000003e-06 2020-08-07T15:08:03.296 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.297 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.297 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:03.297 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.361401+0000", 2020-08-07T15:08:03.297 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0509999999999998e-06 2020-08-07T15:08:03.297 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.298 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.298 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:03.298 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.361405+0000", 2020-08-07T15:08:03.298 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00090700899999999996 2020-08-07T15:08:03.298 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.298 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.299 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:03.299 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362312+0000", 2020-08-07T15:08:03.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000105777 2020-08-07T15:08:03.299 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.299 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.300 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:03.300 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362418+0000", 2020-08-07T15:08:03.300 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000205498 2020-08-07T15:08:03.300 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.300 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.301 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:03.301 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362624+0000", 2020-08-07T15:08:03.301 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012514150000000001 2020-08-07T15:08:03.301 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.301 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.301 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:03.302 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.363875+0000", 2020-08-07T15:08:03.302 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6300000000000004e-07 2020-08-07T15:08:03.302 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.302 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.302 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:03.303 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.363876+0000", 2020-08-07T15:08:03.303 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5851000000000004e-05 2020-08-07T15:08:03.303 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.303 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.303 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:03.304 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.363942+0000", 2020-08-07T15:08:03.304 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.062455701000000002 2020-08-07T15:08:03.304 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.304 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.304 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:03.304 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.426398+0000", 2020-08-07T15:08:03.305 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19555302599999999 2020-08-07T15:08:03.305 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.305 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.305 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:03.305 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.621951+0000", 2020-08-07T15:08:03.307 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.063e-06 2020-08-07T15:08:03.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.308 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:03.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.621952+0000", 2020-08-07T15:08:03.308 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0306999999999999e-05 2020-08-07T15:08:03.308 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.309 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.309 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:03.309 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.621992+0000", 2020-08-07T15:08:03.309 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26063054099999999 2020-08-07T15:08:03.309 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.310 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:03.310 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.310 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.310 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:03.310 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:03.312 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:08:03.313 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 87 16 125417 125401 11.2599 12.0391 0.00901401 0.0110956 2020-08-07T15:08:03.313 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:03.311+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:03.386 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:08:03.386 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:08:03.387 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_0 2020-08-07T15:08:03.387 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:08:03.414 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:03.413+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:03.504 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:08:03.514 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:03.513+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:03.530 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:03.531 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:08:03.531 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.531 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:125528 3.4 3:2c06ab2d:::benchmark_data_smithi074_35447_object15690:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e69)", 2020-08-07T15:08:03.531 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:03.323266+0000", 2020-08-07T15:08:03.532 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.20654414300000001, 2020-08-07T15:08:03.532 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20656849699999999, 2020-08-07T15:08:03.532 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:03.532 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:03.532 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:03.532 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:03.533 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:03.533 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 125528 2020-08-07T15:08:03.533 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.533 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:03.533 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.534 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:03.534 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.323266+0000", 2020-08-07T15:08:03.534 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:03.534 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.534 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:03.535 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.323266+0000", 2020-08-07T15:08:03.535 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1621000000000001e-05 2020-08-07T15:08:03.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:03.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.323298+0000", 2020-08-07T15:08:03.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1820000000000002e-06 2020-08-07T15:08:03.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.536 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.537 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:03.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.323301+0000", 2020-08-07T15:08:03.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4389999999999997e-06 2020-08-07T15:08:03.537 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.537 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.538 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:03.538 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.323307+0000", 2020-08-07T15:08:03.538 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9500000000000001e-06 2020-08-07T15:08:03.538 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.538 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.539 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:03.539 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.323310+0000", 2020-08-07T15:08:03.539 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010126709999999999 2020-08-07T15:08:03.539 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.539 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.539 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:03.540 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.324323+0000", 2020-08-07T15:08:03.540 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010536500000000001 2020-08-07T15:08:03.540 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.540 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.540 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:03.541 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.324428+0000", 2020-08-07T15:08:03.541 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017499900000000001 2020-08-07T15:08:03.541 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.542 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:08:03.542 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.324603+0000", 2020-08-07T15:08:03.542 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023473299999999999 2020-08-07T15:08:03.542 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.543 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.543 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:03.543 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.326951+0000", 2020-08-07T15:08:03.543 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.23e-07 2020-08-07T15:08:03.544 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.544 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.544 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:03.544 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.326951+0000", 2020-08-07T15:08:03.544 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3559999999999997e-05 2020-08-07T15:08:03.545 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.545 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:03.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.326985+0000", 2020-08-07T15:08:03.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00041237000000000001 2020-08-07T15:08:03.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:03.546 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:03.546 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:03.546 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.327397+0000", 2020-08-07T15:08:03.546 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041312099999999997 2020-08-07T15:08:03.546 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.547 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:03.547 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.547 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:03.547 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:08:03.547 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 1 2020-08-07T15:08:03.548 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:03.549 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:08:03.649 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:03.642+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:03.720 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:03.718+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:03.776 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:03.776 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:08:03.776 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:08:03.777 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:08:03.777 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:03.794 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:08:03.820 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:03.818+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:03.868 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:08:03.869 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:08:03.882 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2020-08-07T15:08:03.490120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1007779840,"num_objects":15380,"num_object_clones":0,"num_object_copies":46140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":959,"num_objects_unfound":0,"num_objects_dirty":15380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123021,"num_write_kb":984161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1651,"num_bytes_recovered":108134400,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":105,"acting":105,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":109,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":11467560,"kb_used_data":3078888,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":743507160,"statfs":{"total":773094113280,"available":761351331840,"internally_reserved":8589934592,"allocated":3152781312,"data_stored":3149089429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":52854784,"num_objects":807,"num_object_clones":0,"num_object_copies":2421,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-171,"num_objects_unfound":0,"num_objects_dirty":807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6452,"num_write_kb":51616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":15204352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.100711"},"pg_stats":[{"pgid":"3.f","version":"68'8084","reported_seq":"8133","reported_epoch":"68","state":"active+clean","last_fresh":"2020-08-07T15:08:01.705514+0000","last_change":"2020-08-07T15:06:35.163498+0000","last_active":"2020-08-07T15:08:01.705514+0000","last_peered":"2020-08-07T15:08:01.705514+0000","last_clean":"2020-08-07T15:08:01.705514+0000","last_became_active":"2020-08-07T15:06:35.162764+0000","last_became_peered":"2020-08-07T15:06:35.162764+0000","last_unstale":"2020-08-07T15:08:01.705514+0000","last_undegraded":"2020-08-07T15:08:01.705514+0000","last_fullsized":"2020-08-07T15:08:01.705514+0000","mapping_epoch":20,"log_start":"68'8073","ondisk_log_start":"68'8073","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":66224128,"num_objects":1011,"num_object_clones":0,"num_object_copies":3033,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8084,"num_write_kb":64672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"68'7864","reported_seq":"7920","reported_epoch":"68","state":"active+clean","last_fresh":"2020-08-07T15:08:01.649568+0000","last_change":"2020-08-07T15:06:35.169137+0000","last_active":"2020-08-07T15:08:01.649568+0000","last_peered":"2020-08-07T15:08:01.649568+0000","last_clean":"2020-08-07T15:08:01.649568+0000","last_became_active":"2020-08-07T15:06:35.168689+0000","last_became_peered":"2020-08-07T15:06:35.168689+0000","last_unstale":"2020-08-07T15:08:01.649568+0000","last_undegraded":"2020-08-07T15:08:01.649568+0000","last_fullsized":"2020-08-07T15:08:01.649568+0000","mapping_epoch":20,"log_start":"68'7856","ondisk_log_start":"68'7856","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":64421888,"num_objects":983,"num_object_clones":0,"num_object_copies":2949,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7864,"num_write_kb":62912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"68'7848","reported_seq":"10381","reported_epoch":"68","state":"active+clean","last_fresh":"2020-08-07T15:08:01.636141+0000","last_change":"2020-08-07T15:08:00.913767+0000","last_active":"2020-08-07T15:08:01.636141+0000","last_peered":"2020-08-07T15:08:01.636141+0000","last_clean":"2020-08-07T15:08:01.636141+0000","last_became_active":"2020-08-07T15:07:51.356141+0000","last_became_peered":"2020-08-07T15:07:51.356141+0000","last_unstale":"2020-08-07T15:08:01.636141+0000","last_undegraded":"2020-08-07T15:08:01.636141+0000","last_fullsized":"2020-08-07T15:08:01.636141+0000","mapping_epoch":62,"log_start":"68'7843","ondisk_log_start":"68'7843","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'7824","last_scrub_stamp":"2020-08-07T15:08:00.913676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:08:00.913676+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":64290816,"num_objects":981,"num_object_clones":0,"num_object_copies":2943,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7848,"num_write_kb":62784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":821,"num_bytes_recovered":53805056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"68'8088","reported_seq":"8144","reported_epoch":"68","state":"active+clean","last_fresh":"2020-08-07T15:08:01.488630+0000","last_change":"2020-08-07T15:06:35.167480+0000","last_active":"2020-08-07T15:08:01.488630+0000","last_peered":"2020-08-07T15:08:01.488630+0000","last_clean":"2020-08-07T15:08:01.488630+0000","last_became_active":"2020-08-07T15:06:35.167101+0000","last_became_peered":"2020-08-07T15:06:35.167101+0000","last_unstale":"2020-08-07T15:08:01.488630+0000","last_undegraded":"2020-08-07T15:08:01.488630+0000","last_fullsized":"2020-08-07T15:08:01.488630+0000","mapping_epoch":20,"log_start":"68'8080","ondisk_log_start":"68'8080","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":66256896,"num_objects":1011,"num_object_clones":0,"num_object_copies":3033,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8088,"num_write_kb":64704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"98","reported_epoch":"67","state":"active+clean","last_fresh":"2020-08-07T15:08:00.415582+0000","last_change":"2020-08-07T15:07:56.631170+0000","last_active":"2020-08-07T15:08:00.415582+0000","last_peered":"2020-08-07T15:08:00.415582+0000","last_clean":"2020-08-07T15:08:00.415582+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:08:00.415582+0000","last_undegraded":"2020-08-07T15:08:00.415582+0000","last_fullsized":"2020-08-07T15:08:00.415582+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:56.631091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:56.631091+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.b","version":"68'7056","reported_seq":"7112","reported_epoch":"68","state":"active+clean","last_fresh":"2020-08-07T15:08:01.656088+0000","last_change":"2020-08-07T15:06:35.170504+0000","last_active":"2020-08-07T15:08:01.656088+0000","last_peered":"2020-08-07T15:08:01.656088+0000","last_clean":"2020-08-07T15:08:01.656088+0000","last_became_active":"2020-08-07T15:06:35.169532+0000","last_became_peered":"2020-08-07T15:06:35.169532+0000","last_unstale":"2020-08-07T15:08:01.656088+0000","last_undegraded":"2020-08-07T15:08:01.656088+0000","last_fullsized":"2020-08-07T15:08:01.656088+0000","mapping_epoch":20,"log_start":"68'7048","ondisk_log_start":"68'7048","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57802752,"num_objects":882,"num_object_clones":0,"num_object_copies":2646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7056,"num_write_kb":56448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"34","reported_epoch":"67","state":"active+clean","last_fresh":"2020-08-07T15:08:00.405465+0000","last_change":"2020-08-07T15:07:33.177491+0000","last_active":"2020-08-07T15:08:00.405465+0000","last_peered":"2020-08-07T15:08:00.405465+0000","last_clean":"2020-08-07T15:08:00.405465+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:08:00.405465+0000","last_undegraded":"2020-08-07T15:08:00.405465+0000","last_fullsized":"2020-08-07T15:08:00.405465+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:33.177412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:33.177412+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.a","version":"69'7944","reported_seq":"8001","reported_epoch":"69","state":"active+clean","last_fresh":"2020-08-07T15:08:02.643769+0000","last_change":"2020-08-07T15:06:35.170105+0000","last_active":"2020-08-07T15:08:02.643769+0000","last_peered":"2020-08-07T15:08:02.643769+0000","last_clean":"2020-08-07T15:08:02.643769+0000","last_became_active":"2020-08-07T15:06:35.169494+0000","last_became_peered":"2020-08-07T15:06:35.169494+0000","last_unstale":"2020-08-07T15:08:02.643769+0000","last_undegraded":"2020-08-07T15:08:02.643769+0000","last_fullsized":"2020-08-07T15:08:02.643769+0000","mapping_epoch":20,"log_start":"69'7936","ondisk_log_start":"69'7936","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":65077248,"num_objects":993,"num_object_clones":0,"num_object_copies":2979,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7944,"num_write_kb":63552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"68'7520","reported_seq":"7575","reported_epoch":"68","state":"active+clean","last_fresh":"2020-08-07T15:08:01.694029+0000","last_change":"2020-08-07T15:06:35.163109+0000","last_active":"2020-08-07T15:08:01.694029+0000","last_peered":"2020-08-07T15:08:01.694029+0000","last_clean":"2020-08-07T15:08:01.694029+0000","last_became_active":"2020-08-07T15:06:35.162721+0000","last_became_peered":"2020-08-07T15:06:35.162721+0000","last_unstale":"2020-08-07T15:08:01.694029+0000","last_undegraded":"2020-08-07T15:08:01.694029+0000","last_fullsized":"2020-08-07T15:08:01.694029+0000","mapping_epoch":20,"log_start":"68'7508","ondisk_log_start":"68'7508","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61603840,"num_objects":940,"num_object_clones":0,"num_object_copies":2820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7520,"num_write_kb":60160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"34","reported_epoch":"67","state":"active+clean","last_fresh":"2020-08-07T15:08:00.405489+0000","last_change":"2020-08-07T15:07:36.143434+0000","last_active":"2020-08-07T15:08:00.405489+0000","last_peered":"2020-08-07T15:08:00.405489+0000","last_clean":"2020-08-07T15:08:00.405489+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:08:00.405489+0000","last_undegraded":"2020-08-07T15:08:00.405489+0000","last_fullsized":"2020-08-07T15:08:00.405489+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.143353+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"68'7672","reported_seq":"7734","reported_epoch":"68","state":"active+remapped+backfill_toofull","last_fresh":"2020-08-07T15:08:01.699363+0000","last_change":"2020-08-07T15:07:49.373818+0000","last_active":"2020-08-07T15:08:01.699363+0000","last_peered":"2020-08-07T15:08:01.699363+0000","last_clean":"2020-08-07T15:07:47.323755+0000","last_became_active":"2020-08-07T15:07:49.354652+0000","last_became_peered":"2020-08-07T15:07:49.354652+0000","last_unstale":"2020-08-07T15:08:01.699363+0000","last_undegraded":"2020-08-07T15:08:01.699363+0000","last_fullsized":"2020-08-07T15:08:01.699363+0000","mapping_epoch":60,"log_start":"68'7664","ondisk_log_start":"68'7664","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":62849024,"num_objects":959,"num_object_clones":0,"num_object_copies":2877,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":959,"num_objects_unfound":0,"num_objects_dirty":959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7672,"num_write_kb":61376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,4],"acting":[1,7,3],"avail_no_missing":["1","3","7"],"object_location_counts":[{"shards":"1,3,7","objects":959}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"73","reported_epoch":"68","state":"active+clean","last_fresh":"2020-08-07T15:08:01.412146+0000","last_change":"2020-08-07T15:07:51.357310+0000","last_active":"2020-08-07T15:08:01.412146+0000","last_peered":"2020-08-07T15:08:01.412146+0000","last_clean":"2020-08-07T15:08:01.412146+0000","last_became_active":"2020-08-07T15:07:51.356798+0000","last_became_peered":"2020-08-07T15:07:51.356798+0000","last_unstale":"2020-08-07T15:08:01.412146+0000","last_undegraded":"2020-08-07T15:08:01.412146+0000","last_fullsized":"2020-08-07T15:08:01.412146+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.927824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.927824+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"56","reported_epoch":"67","state":"active+clean","last_fresh":"2020-08-07T15:08:00.415943+0000","last_change":"2020-08-07T15:06:46.199001+0000","last_active":"2020-08-07T15:08:00.415943+0000","last_peered":"2020-08-07T15:08:00.415943+0000","last_clean":"2020-08-07T15:08:00.415943+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:08:00.415943+0000","last_undegraded":"2020-08-07T15:08:00.415943+0000","last_fullsized":"2020-08-07T15:08:00.415943+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.5","version":"68'7608","reported_seq":"7664","reported_epoch":"68","state":"active+clean","last_fresh":"2020-08-07T15:08:01.649282+0000","last_change":"2020-08-07T15:06:35.169307+0000","last_active":"2020-08-07T15:08:01.649282+0000","last_peered":"2020-08-07T15:08:01.649282+0000","last_clean":"2020-08-07T15:08:01.649282+0000","last_became_active":"2020-08-07T15:06:35.168887+0000","last_became_peered":"2020-08-07T15:06:35.168887+0000","last_unstale":"2020-08-07T15:08:01.649282+0000","last_undegraded":"2020-08-07T15:08:01.649282+0000","last_fullsized":"2020-08-07T15:08:01.649282+0000","mapping_epoch":20,"log_start":"68'7601","ondisk_log_start":"68'7601","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":62324736,"num_objects":951,"num_object_clones":0,"num_object_copies":2853,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7608,"num_write_kb":60864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"45","reported_epoch":"67","state":"active+clean","last_fresh":"2020-08-07T15:08:00.405624+0000","last_change":"2020-08-07T15:07:33.171050+0000","last_active":"2020-08-07T15:08:00.405624+0000","last_peered":"2020-08-07T15:08:00.405624+0000","last_clean":"2020-08-07T15:08:00.405624+0000","last_became_active":"2020-08-07T15:07:33.170597+0000","last_became_peered":"2020-08-07T15:07:33.170597+0000","last_unstale":"2020-08-07T15:08:00.405624+0000","last_undegraded":"2020-08-07T15:08:00.405624+0000","last_fullsized":"2020-08-07T15:08:00.405624+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:18.518049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:18.518049+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"59","reported_epoch":"67","state":"active+clean","last_fresh":"2020-08-07T15:08:00.405598+0000","last_change":"2020-08-07T15:06:23.074962+0000","last_active":"2020-08-07T15:08:00.405598+0000","last_peered":"2020-08-07T15:08:00.405598+0000","last_clean":"2020-08-07T15:08:00.405598+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:08:00.405598+0000","last_undegraded":"2020-08-07T15:08:00.405598+0000","last_fullsized":"2020-08-07T15:08:00.405598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"68'7400","reported_seq":"7456","reported_epoch":"68","state":"active+clean","last_fresh":"2020-08-07T15:08:01.665287+0000","last_change":"2020-08-07T15:06:35.164593+0000","last_active":"2020-08-07T15:08:01.665287+0000","last_peered":"2020-08-07T15:08:01.665287+0000","last_clean":"2020-08-07T15:08:01.665287+0000","last_became_active":"2020-08-07T15:06:35.164108+0000","last_became_peered":"2020-08-07T15:06:35.164108+0000","last_unstale":"2020-08-07T15:08:01.665287+0000","last_undegraded":"2020-08-07T15:08:01.665287+0000","last_fullsized":"2020-08-07T15:08:01.665287+0000","mapping_epoch":20,"log_start":"68'7394","ondisk_log_start":"68'7394","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60620800,"num_objects":925,"num_object_clones":0,"num_object_copies":2775,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7400,"num_write_kb":59200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"98","reported_epoch":"67","state":"active+clean","last_fresh":"2020-08-07T15:08:00.415665+0000","last_change":"2020-08-07T15:07:36.358882+0000","last_active":"2020-08-07T15:08:00.415665+0000","last_peered":"2020-08-07T15:08:00.415665+0000","last_clean":"2020-08-07T15:08:00.415665+0000","last_became_active":"2020-08-07T15:07:33.171449+0000","last_became_peered":"2020-08-07T15:07:33.171449+0000","last_unstale":"2020-08-07T15:08:00.415665+0000","last_undegraded":"2020-08-07T15:08:00.415665+0000","last_fullsized":"2020-08-07T15:08:00.415665+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.358798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.358798+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"62","reported_epoch":"67","state":"active+clean","last_fresh":"2020-08-07T15:08:00.422647+0000","last_change":"2020-08-07T15:07:36.864560+0000","last_active":"2020-08-07T15:08:00.422647+0000","last_peered":"2020-08-07T15:08:00.422647+0000","last_clean":"2020-08-07T15:08:00.422647+0000","last_became_active":"2020-08-07T15:06:23.223489+0000","last_became_peered":"2020-08-07T15:06:23.223489+0000","last_unstale":"2020-08-07T15:08:00.422647+0000","last_undegraded":"2020-08-07T15:08:00.422647+0000","last_fullsized":"2020-08-07T15:08:00.422647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:07:36.864483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.864483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"68'7784","reported_seq":"7840","reported_epoch":"68","state":"active+clean","last_fresh":"2020-08-07T15:08:01.560051+0000","last_change":"2020-08-07T15:06:35.170553+0000","last_active":"2020-08-07T15:08:01.560051+0000","last_peered":"2020-08-07T15:08:01.560051+0000","last_clean":"2020-08-07T15:08:01.560051+0000","last_became_active":"2020-08-07T15:06:35.170037+0000","last_became_peered":"2020-08-07T15:06:35.170037+0000","last_unstale":"2020-08-07T15:08:01.560051+0000","last_undegraded":"2020-08-07T15:08:01.560051+0000","last_fullsized":"2020-08-07T15:08:01.560051+0000","mapping_epoch":20,"log_start":"68'7780","ondisk_log_start":"68'7780","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63766528,"num_objects":973,"num_object_clones":0,"num_object_copies":2919,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7784,"num_write_kb":62272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"59","reported_epoch":"67","state":"active+clean","last_fresh":"2020-08-07T15:08:00.405608+0000","last_change":"2020-08-07T15:06:23.075958+0000","last_active":"2020-08-07T15:08:00.405608+0000","last_peered":"2020-08-07T15:08:00.405608+0000","last_clean":"2020-08-07T15:08:00.405608+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:08:00.405608+0000","last_undegraded":"2020-08-07T15:08:00.405608+0000","last_fullsized":"2020-08-07T15:08:00.405608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"82","reported_epoch":"67","state":"active+clean","last_fresh":"2020-08-07T15:08:00.415561+0000","last_change":"2020-08-07T15:07:32.461010+0000","last_active":"2020-08-07T15:08:00.415561+0000","last_peered":"2020-08-07T15:08:00.415561+0000","last_clean":"2020-08-07T15:08:00.415561+0000","last_became_active":"2020-08-07T15:07:28.210245+0000","last_became_peered":"2020-08-07T15:07:28.210245+0000","last_unstale":"2020-08-07T15:08:00.415561+0000","last_undegraded":"2020-08-07T15:08:00.415561+0000","last_fullsized":"2020-08-07T15:08:00.415561+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:32.460945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:32.460945+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"68'7800","reported_seq":"7856","reported_epoch":"68","state":"active+clean","last_fresh":"2020-08-07T15:08:01.476168+0000","last_change":"2020-08-07T15:06:35.169888+0000","last_active":"2020-08-07T15:08:01.476168+0000","last_peered":"2020-08-07T15:08:01.476168+0000","last_clean":"2020-08-07T15:08:01.476168+0000","last_became_active":"2020-08-07T15:06:35.169385+0000","last_became_peered":"2020-08-07T15:06:35.169385+0000","last_unstale":"2020-08-07T15:08:01.476168+0000","last_undegraded":"2020-08-07T15:08:01.476168+0000","last_fullsized":"2020-08-07T15:08:01.476168+0000","mapping_epoch":20,"log_start":"68'7793","ondisk_log_start":"68'7793","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63897600,"num_objects":975,"num_object_clones":0,"num_object_copies":2925,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7800,"num_write_kb":62400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"53","reported_epoch":"68","state":"active+clean","last_fresh":"2020-08-07T15:08:01.412218+0000","last_change":"2020-08-07T15:07:33.157062+0000","last_active":"2020-08-07T15:08:01.412218+0000","last_peered":"2020-08-07T15:08:01.412218+0000","last_clean":"2020-08-07T15:08:01.412218+0000","last_became_active":"2020-08-07T15:07:33.156569+0000","last_became_peered":"2020-08-07T15:07:33.156569+0000","last_unstale":"2020-08-07T15:08:01.412218+0000","last_undegraded":"2020-08-07T15:08:01.412218+0000","last_fullsized":"2020-08-07T15:08:01.412218+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:18.518049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:18.518049+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"59","reported_epoch":"67","state":"active+clean","last_fresh":"2020-08-07T15:08:00.415619+0000","last_change":"2020-08-07T15:06:23.224115+0000","last_active":"2020-08-07T15:08:00.415619+0000","last_peered":"2020-08-07T15:08:00.415619+0000","last_clean":"2020-08-07T15:08:00.415619+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:08:00.415619+0000","last_undegraded":"2020-08-07T15:08:00.415619+0000","last_fullsized":"2020-08-07T15:08:00.415619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"68'7264","reported_seq":"9000","reported_epoch":"68","state":"active+clean","last_fresh":"2020-08-07T15:08:01.683025+0000","last_change":"2020-08-07T15:08:01.432640+0000","last_active":"2020-08-07T15:08:01.683025+0000","last_peered":"2020-08-07T15:08:01.683025+0000","last_clean":"2020-08-07T15:08:01.683025+0000","last_became_active":"2020-08-07T15:08:01.414124+0000","last_became_peered":"2020-08-07T15:08:01.414124+0000","last_unstale":"2020-08-07T15:08:01.683025+0000","last_undegraded":"2020-08-07T15:08:01.683025+0000","last_fullsized":"2020-08-07T15:08:01.683025+0000","mapping_epoch":67,"log_start":"68'7255","ondisk_log_start":"68'7255","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":59506688,"num_objects":908,"num_object_clones":0,"num_object_copies":2724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7264,"num_write_kb":58112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":829,"num_bytes_recovered":54329344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"53","reported_epoch":"68","state":"active+clean","last_fresh":"2020-08-07T15:08:01.412120+0000","last_change":"2020-08-07T15:07:33.170029+0000","last_active":"2020-08-07T15:08:01.412120+0000","last_peered":"2020-08-07T15:08:01.412120+0000","last_clean":"2020-08-07T15:08:01.412120+0000","last_became_active":"2020-08-07T15:07:33.169518+0000","last_became_peered":"2020-08-07T15:07:33.169518+0000","last_unstale":"2020-08-07T15:08:01.412120+0000","last_undegraded":"2020-08-07T15:08:01.412120+0000","last_fullsized":"2020-08-07T15:08:01.412120+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:18.518049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:18.518049+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"59","reported_epoch":"67","state":"active+clean","last_fresh":"2020-08-07T15:08:00.415754+0000","last_change":"2020-08-07T15:06:23.225886+0000","last_active":"2020-08-07T15:08:00.415754+0000","last_peered":"2020-08-07T15:08:00.415754+0000","last_clean":"2020-08-07T15:08:00.415754+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:08:00.415754+0000","last_undegraded":"2020-08-07T15:08:00.415754+0000","last_fullsized":"2020-08-07T15:08:00.415754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"68'7640","reported_seq":"7698","reported_epoch":"68","state":"active+clean","last_fresh":"2020-08-07T15:08:01.640338+0000","last_change":"2020-08-07T15:07:38.065908+0000","last_active":"2020-08-07T15:08:01.640338+0000","last_peered":"2020-08-07T15:08:01.640338+0000","last_clean":"2020-08-07T15:08:01.640338+0000","last_became_active":"2020-08-07T15:06:35.169127+0000","last_became_peered":"2020-08-07T15:06:35.169127+0000","last_unstale":"2020-08-07T15:08:01.640338+0000","last_undegraded":"2020-08-07T15:08:01.640338+0000","last_fullsized":"2020-08-07T15:08:01.640338+0000","mapping_epoch":20,"log_start":"68'7632","ondisk_log_start":"68'7632","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'5784","last_scrub_stamp":"2020-08-07T15:07:38.065851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:07:38.065851+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":62586880,"num_objects":955,"num_object_clones":0,"num_object_copies":2865,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7640,"num_write_kb":61120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"16'2","reported_seq":"31","reported_epoch":"67","state":"active+clean","last_fresh":"2020-08-07T15:08:00.409917+0000","last_change":"2020-08-07T15:07:33.155878+0000","last_active":"2020-08-07T15:08:00.409917+0000","last_peered":"2020-08-07T15:08:00.409917+0000","last_clean":"2020-08-07T15:08:00.409917+0000","last_became_active":"2020-08-07T15:07:33.155139+0000","last_became_peered":"2020-08-07T15:07:33.155139+0000","last_unstale":"2020-08-07T15:08:00.409917+0000","last_undegraded":"2020-08-07T15:08:00.409917+0000","last_fullsized":"2020-08-07T15:08:00.409917+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:18.518049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:18.518049+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"68'7432","reported_seq":"7488","reported_epoch":"68","state":"active+clean","last_fresh":"2020-08-07T15:08:01.673320+0000","last_change":"2020-08-07T15:06:35.172780+0000","last_active":"2020-08-07T15:08:01.673320+0000","last_peered":"2020-08-07T15:08:01.673320+0000","last_clean":"2020-08-07T15:08:01.673320+0000","last_became_active":"2020-08-07T15:06:35.172360+0000","last_became_peered":"2020-08-07T15:06:35.172360+0000","last_unstale":"2020-08-07T15:08:01.673320+0000","last_undegraded":"2020-08-07T15:08:01.673320+0000","last_fullsized":"2020-08-07T15:08:01.673320+0000","mapping_epoch":20,"log_start":"68'7424","ondisk_log_start":"68'7424","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60882944,"num_objects":929,"num_object_clones":0,"num_object_copies":2787,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7432,"num_write_kb":59456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"62","reported_epoch":"67","state":"active+clean","last_fresh":"2020-08-07T15:08:00.409288+0000","last_change":"2020-08-07T15:07:34.423236+0000","last_active":"2020-08-07T15:08:00.409288+0000","last_peered":"2020-08-07T15:08:00.409288+0000","last_clean":"2020-08-07T15:08:00.409288+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:08:00.409288+0000","last_undegraded":"2020-08-07T15:08:00.409288+0000","last_fullsized":"2020-08-07T15:08:00.409288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:07:34.423115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:07:34.423115+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"41","reported_epoch":"67","state":"active+clean","last_fresh":"2020-08-07T15:08:00.413980+0000","last_change":"2020-08-07T15:07:34.653928+0000","last_active":"2020-08-07T15:08:00.413980+0000","last_peered":"2020-08-07T15:08:00.413980+0000","last_clean":"2020-08-07T15:08:00.413980+0000","last_became_active":"2020-08-07T15:07:33.161761+0000","last_became_peered":"2020-08-07T15:07:33.161761+0000","last_unstale":"2020-08-07T15:08:00.413980+0000","last_undegraded":"2020-08-07T15:08:00.413980+0000","last_fullsized":"2020-08-07T15:08:00.413980+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:34.653843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:34.653843+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"60","reported_epoch":"68","state":"active+clean","last_fresh":"2020-08-07T15:08:01.412278+0000","last_change":"2020-08-07T15:06:23.073702+0000","last_active":"2020-08-07T15:08:01.412278+0000","last_peered":"2020-08-07T15:08:01.412278+0000","last_clean":"2020-08-07T15:08:01.412278+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:08:01.412278+0000","last_undegraded":"2020-08-07T15:08:01.412278+0000","last_fullsized":"2020-08-07T15:08:01.412278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"68'8016","reported_seq":"8075","reported_epoch":"68","state":"active+clean+scrubbing","last_fresh":"2020-08-07T15:08:01.698273+0000","last_change":"2020-08-07T15:08:01.232712+0000","last_active":"2020-08-07T15:08:01.698273+0000","last_peered":"2020-08-07T15:08:01.698273+0000","last_clean":"2020-08-07T15:08:01.698273+0000","last_became_active":"2020-08-07T15:06:46.199201+0000","last_became_peered":"2020-08-07T15:06:46.199201+0000","last_unstale":"2020-08-07T15:08:01.698273+0000","last_undegraded":"2020-08-07T15:08:01.698273+0000","last_fullsized":"2020-08-07T15:08:01.698273+0000","mapping_epoch":26,"log_start":"68'8009","ondisk_log_start":"68'8009","created":20,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":65667072,"num_objects":1002,"num_object_clones":0,"num_object_copies":3006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8016,"num_write_kb":64128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":1007779840,"num_objects":15378,"num_object_clones":0,"num_object_copies":46134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":959,"num_objects_unfound":0,"num_objects_dirty":15378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123020,"num_write_kb":984160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1650,"num_bytes_recovered":108134400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3155492864,"data_stored":3155492864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124,"ondisk_log_size":124,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22,"ondisk_log_size":22,"up":33,"acting":33,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542167,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1423788,"kb_used_data":375204,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92948052,"statfs":{"total":96636764160,"available":95178805248,"internally_reserved":1073741824,"allocated":384208896,"data_stored":383742507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.96199999999999997}]},{"osd":1,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":5.7519999999999998,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":0.995},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.0229999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":1.2569999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":5.5019999999999998,"5min":5.5019999999999998,"15min":5.5019999999999998},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":1.2390000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.9790000000000001,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":1.325},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":1.3740000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":37.506999999999998,"5min":37.506999999999998,"15min":37.506999999999998},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":831.88099999999997,"5min":831.88099999999997,"15min":831.88099999999997},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":1.3480000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":3.1389999999999998,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":1.304},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":3.048,"5min":3.048,"15min":3.048},"last":1.4299999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1393308,"kb_used_data":344724,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92978532,"statfs":{"total":96636764160,"available":95210016768,"internally_reserved":1073741824,"allocated":352997376,"data_stored":352530987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":3.4020000000000001,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.383,"5min":2.383,"15min":2.383},"last":0.73299999999999998}]},{"osd":1,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.387,"5min":1.387,"15min":1.387},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.5939999999999999,"5min":3.5939999999999999,"15min":3.5939999999999999},"last":1.129}]},{"osd":2,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.272,"15min":1.272},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":4.0529999999999999,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":1.048}]},{"osd":3,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":3.3580000000000001,"5min":3.3580000000000001,"15min":3.3580000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":3.004,"5min":3.004,"15min":3.004},"last":1.2430000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.663,"15min":1.663},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":5.2729999999999997,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"min":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":5.2990000000000004,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":2.0649999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.766,"5min":1.766,"15min":1.766},"min":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":5.327,"5min":5.327,"15min":5.327},"last":1.1479999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":3.3809999999999998,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":0.97999999999999998},{"interface":"front","average":{"1min":18.326000000000001,"5min":18.326000000000001,"15min":18.326000000000001},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":266.26999999999998,"5min":266.26999999999998,"15min":266.26999999999998},"last":0.997}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1422372,"kb_used_data":373788,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92949468,"statfs":{"total":96636764160,"available":95180255232,"internally_reserved":1073741824,"allocated":382758912,"data_stored":382303254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":34.186,"5min":34.186,"15min":34.186},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":497.04000000000002,"5min":497.04000000000002,"15min":497.04000000000002},"last":0.622},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":4.1980000000000004,"5min":4.1980000000000004,"15min":4.1980000000000004},"last":0.90400000000000003}]},{"osd":2,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":4.0919999999999996,"5min":4.0919999999999996,"15min":4.0919999999999996},"last":0.72999999999999998}]},{"osd":3,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.454,"5min":1.454,"15min":1.454},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.9889999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":0.97099999999999997}]},{"osd":4,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.2,"5min":1.2,"15min":1.2},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.1679999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":13.007999999999999,"5min":13.007999999999999,"15min":13.007999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":213.63800000000001,"5min":213.63800000000001,"15min":213.63800000000001},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":1.1200000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:07:22 2020","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.552,"15min":1.552},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":3.234,"5min":3.234,"15min":3.234},"last":1.153},{"interface":"front","average":{"1min":1.361,"5min":1.361,"15min":1.361},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":1.381}]},{"osd":7,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.615,"5min":1.615,"15min":1.615},"min":{"1min":1.046,"5min":1.046,"15min":1.046},"max":{"1min":3.2810000000000001,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":1.038},{"interface":"front","average":{"1min":1.48,"5min":1.48,"15min":1.48},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.8759999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":1.429}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1409316,"kb_used_data":360732,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92962524,"statfs":{"total":96636764160,"available":95193624576,"internally_reserved":1073741824,"allocated":369389568,"data_stored":368934160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.2440000000000002,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":0.71699999999999997}]},{"osd":2,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":32.740000000000002,"5min":32.740000000000002,"15min":32.740000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":606.18499999999995,"5min":606.18499999999995,"15min":606.18499999999995},"last":0.65800000000000003}]},{"osd":3,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":0.89300000000000002}]},{"osd":4,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.6949999999999998,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":2.4060000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.0169999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.874}]},{"osd":6,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":8.6140000000000008,"5min":8.6140000000000008,"15min":8.6140000000000008},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":124.738,"5min":124.738,"15min":124.738},"last":0.92600000000000005},{"interface":"front","average":{"1min":18.652999999999999,"5min":18.652999999999999,"15min":18.652999999999999},"min":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"max":{"1min":294.94,"5min":294.94,"15min":294.94},"last":0.99399999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":6.468,"5min":6.468,"15min":6.468},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.883,"5min":2.883,"15min":2.883},"last":0.91400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1293324,"kb_used_data":244740,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93078516,"statfs":{"total":96636764160,"available":95312400384,"internally_reserved":1073741824,"allocated":250613760,"data_stored":250154157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":3.2240000000000002,"5min":3.2240000000000002,"15min":3.2240000000000002},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.56499999999999995}]},{"osd":1,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":3.1320000000000001,"5min":3.1320000000000001,"15min":3.1320000000000001},"last":3.7829999999999999},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":3.3069999999999999,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":1.9530000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.3759999999999999,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.556,"5min":1.556,"15min":1.556},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":8.2210000000000001,"5min":8.2210000000000001,"15min":8.2210000000000001},"last":0.67400000000000004}]},{"osd":4,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.643,"5min":1.643,"15min":1.643},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":3.2749999999999999,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":1.891}]},{"osd":5,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":0.76800000000000002},{"interface":"front","average":{"1min":25.524999999999999,"5min":25.524999999999999,"15min":25.524999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":415.40699999999998,"5min":415.40699999999998,"15min":415.40699999999998},"last":1.0129999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":1.054}]},{"osd":7,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.742,"5min":1.742,"15min":1.742},"min":{"1min":1.093,"5min":1.093,"15min":1.093},"max":{"1min":4.056,"5min":4.056,"15min":4.056},"last":1.925},{"interface":"front","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":4.4539999999999997,"5min":4.4539999999999997,"15min":4.4539999999999997},"last":1.085}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1532608,"kb_used_data":484024,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92839232,"statfs":{"total":96636764160,"available":95067373568,"internally_reserved":1073741824,"allocated":495640576,"data_stored":495177455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":2.8220000000000001,"5min":2.8220000000000001,"15min":2.8220000000000001},"last":2.3199999999999998},{"interface":"front","average":{"1min":1.637,"5min":1.637,"15min":1.637},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":13.067,"5min":13.067,"15min":13.067},"last":0.53600000000000003}]},{"osd":1,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.899,"5min":1.899,"15min":1.899},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":6.6829999999999998,"5min":6.6829999999999998,"15min":6.6829999999999998},"last":1.155},{"interface":"front","average":{"1min":2.012,"5min":2.012,"15min":2.012},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":11.936,"5min":11.936,"15min":11.936},"last":2.613}]},{"osd":2,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.589,"5min":3.589,"15min":3.589},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":5.5890000000000004,"5min":5.5890000000000004,"15min":5.5890000000000004},"last":1.119}]},{"osd":4,"last update":"Fri Aug 7 15:07:19 2020","interfaces":[{"interface":"back","average":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":4.556,"5min":4.556,"15min":4.556},"last":3.1360000000000001},{"interface":"front","average":{"1min":32.100000000000001,"5min":32.100000000000001,"15min":32.100000000000001},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":677.88599999999997,"5min":677.88599999999997,"15min":677.88599999999997},"last":0.64200000000000002}]},{"osd":5,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":3.6600000000000001,"5min":3.6600000000000001,"15min":3.6600000000000001},"last":1.002},{"interface":"front","average":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"min":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":5.4329999999999998,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":1.0669999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":3.621,"5min":3.621,"15min":3.621},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":5.6189999999999998,"5min":5.6189999999999998,"15min":5.6189999999999998},"last":1.343}]},{"osd":7,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":5.734,"5min":5.734,"15min":5.734},"last":1.373},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":4.8540000000000001,"5min":4.8540000000000001,"15min":4.8540000000000001},"last":3.6150000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1600292,"kb_used_data":551708,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92771548,"statfs":{"total":96636764160,"available":94998065152,"internally_reserved":1073741824,"allocated":564948992,"data_stored":564482603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.181,"15min":1.181},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":4.9480000000000004,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.86499999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":4.9829999999999997,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":1.137},{"interface":"front","average":{"1min":1.377,"5min":1.377,"15min":1.377},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":5.0430000000000001,"5min":5.0430000000000001,"15min":5.0430000000000001},"last":0.93799999999999994}]},{"osd":2,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.452,"15min":1.452},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":4.415,"5min":4.415,"15min":4.415},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":4.4340000000000002,"5min":4.4340000000000002,"15min":4.4340000000000002},"last":1.1020000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.508,"15min":1.508},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":5.0099999999999998,"5min":5.0099999999999998,"15min":5.0099999999999998},"last":1.073},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":3.2029999999999998,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":0.88700000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":5.157,"5min":5.157,"15min":5.157},"last":0.995},{"interface":"front","average":{"1min":1.627,"5min":1.627,"15min":1.627},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":5.0979999999999999,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":1.1699999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.79,"5min":1.79,"15min":1.79},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":5.1310000000000002,"5min":5.1310000000000002,"15min":5.1310000000000002},"last":1.2050000000000001},{"interface":"front","average":{"1min":19.109000000000002,"5min":19.109000000000002,"15min":19.109000000000002},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":389.14600000000002,"5min":389.14600000000002,"15min":389.14600000000002},"last":0.90600000000000003}]},{"osd":7,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":3.3540000000000001,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":4.4569999999999999,"5min":4.4569999999999999,"15min":4.4569999999999999},"last":1.232}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1392552,"kb_used_data":343968,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92979288,"statfs":{"total":96636764160,"available":95210790912,"internally_reserved":1073741824,"allocated":352223232,"data_stored":351764306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.766,"5min":2.766,"15min":2.766},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.93100000000000005}]},{"osd":1,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.6509999999999998,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":2.746,"5min":2.746,"15min":2.746},"last":0.84999999999999998}]},{"osd":2,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":9.2200000000000006,"5min":9.2200000000000006,"15min":9.2200000000000006},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":168.56899999999999,"5min":168.56899999999999,"15min":168.56899999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":33.131,"5min":33.131,"15min":33.131},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":676.27099999999996,"5min":676.27099999999996,"15min":676.27099999999996},"last":0.97199999999999998}]},{"osd":3,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.7669999999999999,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.964,"5min":2.964,"15min":2.964},"last":1.423}]},{"osd":4,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":50.893000000000001,"5min":50.893000000000001,"15min":50.893000000000001},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":530.30799999999999,"5min":530.30799999999999,"15min":530.30799999999999},"last":0.875},{"interface":"front","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":3.1080000000000001,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":1.395}]},{"osd":6,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":50.408999999999999,"5min":50.408999999999999,"15min":50.408999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":754.31299999999999,"5min":754.31299999999999,"15min":754.31299999999999},"last":1.478},{"interface":"front","average":{"1min":26.016999999999999,"5min":26.016999999999999,"15min":26.016999999999999},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":517.26700000000005,"5min":517.26700000000005,"15min":517.26700000000005},"last":0.998}]},{"osd":7,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.177},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"min":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":3.5649999999999999,"5min":3.5649999999999999,"15min":3.5649999999999999},"last":1.4510000000000001}]}]}]}} 2020-08-07T15:08:04.016 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 pgp_num 16 2020-08-07T15:08:04.020 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:04.020 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:08:04.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:08:04.021 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:08:04.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.022 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:8112 3.9 e24/20 3:9b275d48:::benchmark_data_smithi074_35447_object1013:head v 24'416, mlcod=24'408)", 2020-08-07T15:08:04.022 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.085042+0000", 2020-08-07T15:08:04.023 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 82.929436682000002, 2020-08-07T15:08:04.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31443637000000002, 2020-08-07T15:08:04.023 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.024 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:04.024 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085042+0000", 2020-08-07T15:08:04.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085042+0000", 2020-08-07T15:08:04.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4534000000000003e-05 2020-08-07T15:08:04.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085096+0000", 2020-08-07T15:08:04.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5789999999999998e-06 2020-08-07T15:08:04.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.039 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085104+0000", 2020-08-07T15:08:04.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31316244700000001 2020-08-07T15:08:04.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.040 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398266+0000", 2020-08-07T15:08:04.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3019999999999996e-06 2020-08-07T15:08:04.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.043 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398270+0000", 2020-08-07T15:08:04.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3467000000000002e-05 2020-08-07T15:08:04.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.045 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.045 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398314+0000", 2020-08-07T15:08:04.046 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8184000000000001e-05 2020-08-07T15:08:04.047 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.047 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.048 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398332+0000", 2020-08-07T15:08:04.048 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010908059999999999 2020-08-07T15:08:04.048 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.049 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.049 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:04.049 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.399423+0000", 2020-08-07T15:08:04.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5050999999999999e-05 2020-08-07T15:08:04.050 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.050 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.051 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:04.051 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.399478+0000", 2020-08-07T15:08:04.051 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31443637000000002 2020-08-07T15:08:04.052 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.052 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.052 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.053 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:8760 3.9 e24/20 3:9f02d263:::benchmark_data_smithi074_35447_object1094:head v 24'448, mlcod=24'441)", 2020-08-07T15:08:04.053 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.535293+0000", 2020-08-07T15:08:04.054 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 82.479185490000006, 2020-08-07T15:08:04.054 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.194025799, 2020-08-07T15:08:04.055 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.055 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:04.055 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.535293+0000", 2020-08-07T15:08:04.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.058 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.535293+0000", 2020-08-07T15:08:04.058 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1352e-05 2020-08-07T15:08:04.058 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.059 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.059 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.535334+0000", 2020-08-07T15:08:04.059 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6660000000000002e-06 2020-08-07T15:08:04.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.060 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.535341+0000", 2020-08-07T15:08:04.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19233371499999999 2020-08-07T15:08:04.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.064 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.065 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.727675+0000", 2020-08-07T15:08:04.065 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1819999999999997e-06 2020-08-07T15:08:04.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.066 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.066 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.066 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.727680+0000", 2020-08-07T15:08:04.066 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0288999999999999e-05 2020-08-07T15:08:04.067 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.067 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.067 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.067 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.727730+0000", 2020-08-07T15:08:04.068 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0340000000000001e-05 2020-08-07T15:08:04.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.068 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.081 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.081 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.727760+0000", 2020-08-07T15:08:04.081 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015059489999999999 2020-08-07T15:08:04.082 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.082 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:04.082 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.729266+0000", 2020-08-07T15:08:04.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2305999999999997e-05 2020-08-07T15:08:04.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.083 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.083 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:04.084 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.729319+0000", 2020-08-07T15:08:04.084 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.194025799 2020-08-07T15:08:04.084 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.085 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.085 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.086 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:16223 3.9 e27/20 3:9c72cb85:::benchmark_data_smithi074_35447_object2027:head v 27'887, mlcod=27'881)", 2020-08-07T15:08:04.086 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.111666+0000", 2020-08-07T15:08:04.087 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 76.902811830000005, 2020-08-07T15:08:04.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.48138790999999997, 2020-08-07T15:08:04.088 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.088 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:04.088 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.089 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.089 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.089 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111666+0000", 2020-08-07T15:08:04.090 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.090 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.091 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.091 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.091 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111666+0000", 2020-08-07T15:08:04.092 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3168999999999999e-05 2020-08-07T15:08:04.092 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.092 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.092 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.093 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111720+0000", 2020-08-07T15:08:04.093 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6309999999999997e-06 2020-08-07T15:08:04.093 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.093 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.094 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.094 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111726+0000", 2020-08-07T15:08:04.095 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.47995092 2020-08-07T15:08:04.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.095 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.096 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.096 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591677+0000", 2020-08-07T15:08:04.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9879999999999998e-06 2020-08-07T15:08:04.096 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.097 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.097 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.097 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591684+0000", 2020-08-07T15:08:04.097 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5562000000000002e-05 2020-08-07T15:08:04.098 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.098 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.099 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.099 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591730+0000", 2020-08-07T15:08:04.099 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4731000000000001e-05 2020-08-07T15:08:04.099 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.100 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.100 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.102 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591754+0000", 2020-08-07T15:08:04.103 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012412739999999999 2020-08-07T15:08:04.103 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.103 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.104 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:04.104 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.592996+0000", 2020-08-07T15:08:04.104 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8634999999999997e-05 2020-08-07T15:08:04.104 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.105 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.105 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:04.106 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.593054+0000", 2020-08-07T15:08:04.106 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.48138790999999997 2020-08-07T15:08:04.106 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.107 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.107 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.107 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.107 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.108 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:16224 3.9 e27/20 3:9c72cb85:::benchmark_data_smithi074_35447_object2027:head v 27'888, mlcod=27'881)", 2020-08-07T15:08:04.108 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.114073+0000", 2020-08-07T15:08:04.108 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 76.900405102999997, 2020-08-07T15:08:04.108 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.50478892600000003, 2020-08-07T15:08:04.109 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.109 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:04.109 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.109 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.110 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.110 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114073+0000", 2020-08-07T15:08:04.110 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.111 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.111 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.111 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114073+0000", 2020-08-07T15:08:04.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3088999999999999e-05 2020-08-07T15:08:04.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.113 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114126+0000", 2020-08-07T15:08:04.113 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1629999999999999e-06 2020-08-07T15:08:04.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.114 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.114 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114133+0000", 2020-08-07T15:08:04.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.50357425199999994 2020-08-07T15:08:04.115 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.115 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.115 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.115 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617708+0000", 2020-08-07T15:08:04.116 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9160000000000004e-06 2020-08-07T15:08:04.116 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.116 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.117 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617715+0000", 2020-08-07T15:08:04.117 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8770999999999998e-05 2020-08-07T15:08:04.118 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.118 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.118 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.118 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617763+0000", 2020-08-07T15:08:04.119 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.544e-05 2020-08-07T15:08:04.119 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.119 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.120 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.120 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617789+0000", 2020-08-07T15:08:04.120 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010210359999999999 2020-08-07T15:08:04.120 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.121 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.121 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:04.121 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.618810+0000", 2020-08-07T15:08:04.122 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2259000000000002e-05 2020-08-07T15:08:04.122 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.122 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.122 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:04.130 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.618862+0000", 2020-08-07T15:08:04.131 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.50478892600000003 2020-08-07T15:08:04.131 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.131 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.131 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.132 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.132 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.132 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:24992 3.0 e28/20 3:008a8fe9:::benchmark_data_smithi074_35447_object3123:head v 28'1704, mlcod=28'1694)", 2020-08-07T15:08:04.133 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.852063+0000", 2020-08-07T15:08:04.133 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 71.162414967999993, 2020-08-07T15:08:04.133 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20540171199999999, 2020-08-07T15:08:04.133 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.142 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:04.142 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.142 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.143 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.143 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.852063+0000", 2020-08-07T15:08:04.143 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.144 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.144 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.144 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.144 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.852063+0000", 2020-08-07T15:08:04.145 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1409999999999997e-05 2020-08-07T15:08:04.145 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.145 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.146 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.146 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.852115+0000", 2020-08-07T15:08:04.147 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6100000000000002e-06 2020-08-07T15:08:04.147 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.147 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.147 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.148 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.852121+0000", 2020-08-07T15:08:04.148 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20455925999999999 2020-08-07T15:08:04.148 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.149 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.149 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.149 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.056681+0000", 2020-08-07T15:08:04.149 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8739999999999995e-06 2020-08-07T15:08:04.150 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.150 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.150 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.151 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.056689+0000", 2020-08-07T15:08:04.151 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2753999999999998e-05 2020-08-07T15:08:04.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.152 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.056731+0000", 2020-08-07T15:08:04.153 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9171e-05 2020-08-07T15:08:04.153 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.154 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.154 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.155 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.056750+0000", 2020-08-07T15:08:04.155 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00065542300000000003 2020-08-07T15:08:04.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.156 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:04.156 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.057406+0000", 2020-08-07T15:08:04.157 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9209999999999997e-05 2020-08-07T15:08:04.157 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.157 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.158 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:04.158 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.057465+0000", 2020-08-07T15:08:04.158 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20540171199999999 2020-08-07T15:08:04.158 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.159 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.159 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.159 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.159 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.160 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:34000 3.4 3:22a8b4e3:::benchmark_data_smithi074_35447_object4249:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-08-07T15:08:04.160 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:58.316850+0000", 2020-08-07T15:08:04.160 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 65.697628261999995, 2020-08-07T15:08:04.160 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19523959799999999, 2020-08-07T15:08:04.160 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.161 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:04.161 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:04.161 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:04.161 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:04.161 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 34000 2020-08-07T15:08:04.162 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.162 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.162 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.163 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.163 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316850+0000", 2020-08-07T15:08:04.163 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.164 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316850+0000", 2020-08-07T15:08:04.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7274999999999997e-05 2020-08-07T15:08:04.165 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.165 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.165 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316897+0000", 2020-08-07T15:08:04.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2249999999999999e-06 2020-08-07T15:08:04.166 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.166 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.166 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.167 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316903+0000", 2020-08-07T15:08:04.167 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0665e-05 2020-08-07T15:08:04.167 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.167 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.168 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.168 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316913+0000", 2020-08-07T15:08:04.168 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3089999999999997e-06 2020-08-07T15:08:04.168 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.169 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.169 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.169 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316918+0000", 2020-08-07T15:08:04.169 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041515279999999998 2020-08-07T15:08:04.170 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.170 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.170 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.170 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.321069+0000", 2020-08-07T15:08:04.171 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6084999999999999e-05 2020-08-07T15:08:04.171 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.171 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.172 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.172 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.321165+0000", 2020-08-07T15:08:04.172 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000185076 2020-08-07T15:08:04.172 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.173 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.173 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:08:04.173 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.321350+0000", 2020-08-07T15:08:04.173 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00079810799999999996 2020-08-07T15:08:04.174 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.174 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.174 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:04.174 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.322148+0000", 2020-08-07T15:08:04.175 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9299999999999997e-07 2020-08-07T15:08:04.175 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.175 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.176 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:04.176 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.322149+0000", 2020-08-07T15:08:04.176 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1654000000000003e-05 2020-08-07T15:08:04.177 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.177 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.177 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:04.178 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.322211+0000", 2020-08-07T15:08:04.178 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044003000000000002 2020-08-07T15:08:04.178 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.178 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.179 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:04.179 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.326611+0000", 2020-08-07T15:08:04.179 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18541316599999999 2020-08-07T15:08:04.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.181 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.181 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:04.181 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.512024+0000", 2020-08-07T15:08:04.181 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5290000000000001e-06 2020-08-07T15:08:04.182 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:04.183 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.512026+0000", 2020-08-07T15:08:04.183 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3985000000000005e-05 2020-08-07T15:08:04.183 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.183 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.184 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:04.184 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.512090+0000", 2020-08-07T15:08:04.184 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19523959799999999 2020-08-07T15:08:04.185 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.185 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.186 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.186 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.186 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.187 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:34624 3.9 e30/20 3:946cdb13:::benchmark_data_smithi074_35447_object4327:head v 30'2016, mlcod=30'2009)", 2020-08-07T15:08:04.187 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:58.746375+0000", 2020-08-07T15:08:04.187 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 65.268103459000002, 2020-08-07T15:08:04.187 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28564679799999998, 2020-08-07T15:08:04.188 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.188 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:04.188 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.188 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.189 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.189 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746375+0000", 2020-08-07T15:08:04.189 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.189 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.190 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.190 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.190 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746375+0000", 2020-08-07T15:08:04.191 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3674000000000001e-05 2020-08-07T15:08:04.191 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.192 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.192 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.192 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746429+0000", 2020-08-07T15:08:04.194 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9349999999999997e-06 2020-08-07T15:08:04.194 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.194 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.195 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.195 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746435+0000", 2020-08-07T15:08:04.195 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28165817700000001 2020-08-07T15:08:04.195 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.196 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.196 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.196 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028094+0000", 2020-08-07T15:08:04.196 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.512e-06 2020-08-07T15:08:04.197 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.197 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.197 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.197 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028097+0000", 2020-08-07T15:08:04.198 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7904999999999997e-05 2020-08-07T15:08:04.198 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.199 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.199 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.200 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028135+0000", 2020-08-07T15:08:04.200 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6243e-05 2020-08-07T15:08:04.201 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.201 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.201 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.202 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028151+0000", 2020-08-07T15:08:04.202 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003818372 2020-08-07T15:08:04.202 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.203 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.203 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:04.203 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.031970+0000", 2020-08-07T15:08:04.203 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.198e-05 2020-08-07T15:08:04.203 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.204 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.204 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:04.204 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.032022+0000", 2020-08-07T15:08:04.204 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28564679799999998 2020-08-07T15:08:04.205 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.205 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.205 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.206 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.206 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.206 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:64688 3.4 3:221784a5:::benchmark_data_smithi074_35447_object8085:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e41)", 2020-08-07T15:08:04.206 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:20.149588+0000", 2020-08-07T15:08:04.207 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 43.864889828999999, 2020-08-07T15:08:04.207 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20387735500000001, 2020-08-07T15:08:04.207 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.207 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:04.208 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:04.208 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:04.208 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:04.209 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 64688 2020-08-07T15:08:04.209 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.209 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.210 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.210 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.210 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.149588+0000", 2020-08-07T15:08:04.210 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.210 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.211 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.211 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.211 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.149588+0000", 2020-08-07T15:08:04.211 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0269999999999998e-05 2020-08-07T15:08:04.212 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.212 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.212 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.212 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.149639+0000", 2020-08-07T15:08:04.213 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1489999999999999e-06 2020-08-07T15:08:04.213 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.213 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.214 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.214 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.149645+0000", 2020-08-07T15:08:04.214 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1691999999999999e-05 2020-08-07T15:08:04.214 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.215 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.215 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.215 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.149657+0000", 2020-08-07T15:08:04.215 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0409e-05 2020-08-07T15:08:04.216 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.216 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.216 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.217 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.149667+0000", 2020-08-07T15:08:04.217 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00061991800000000001 2020-08-07T15:08:04.217 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.218 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.218 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.218 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.150287+0000", 2020-08-07T15:08:04.218 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8415e-05 2020-08-07T15:08:04.219 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.219 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.219 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.219 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.150375+0000", 2020-08-07T15:08:04.220 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000180729 2020-08-07T15:08:04.220 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.220 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.221 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:08:04.221 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.150556+0000", 2020-08-07T15:08:04.221 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0052814489999999997 2020-08-07T15:08:04.221 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.222 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.222 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:04.222 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.155838+0000", 2020-08-07T15:08:04.223 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6000000000000003e-07 2020-08-07T15:08:04.223 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.223 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.223 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:04.224 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.155838+0000", 2020-08-07T15:08:04.224 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5048999999999993e-05 2020-08-07T15:08:04.224 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.224 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.225 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:04.225 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.155903+0000", 2020-08-07T15:08:04.225 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18448816500000001 2020-08-07T15:08:04.226 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.226 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.226 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:04.227 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.340391+0000", 2020-08-07T15:08:04.227 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.013000151 2020-08-07T15:08:04.227 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.227 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.228 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:04.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.353392+0000", 2020-08-07T15:08:04.229 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0580000000000001e-06 2020-08-07T15:08:04.229 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.229 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.230 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:04.230 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.353393+0000", 2020-08-07T15:08:04.231 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3140999999999996e-05 2020-08-07T15:08:04.231 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.231 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.231 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:04.231 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.353466+0000", 2020-08-07T15:08:04.232 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20387735500000001 2020-08-07T15:08:04.232 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.232 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.242 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.243 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.243 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.243 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:78016 3.9 e46/20 3:921b23e9:::benchmark_data_smithi074_35447_object9751:head v 46'4512, mlcod=46'4506)", 2020-08-07T15:08:04.243 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:28.734571+0000", 2020-08-07T15:08:04.244 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 35.279907057000003, 2020-08-07T15:08:04.244 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23216004300000001, 2020-08-07T15:08:04.247 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.247 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:04.247 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.247 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.248 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.248 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.734571+0000", 2020-08-07T15:08:04.249 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.249 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.250 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.250 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.250 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.734571+0000", 2020-08-07T15:08:04.250 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7376000000000003e-05 2020-08-07T15:08:04.251 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.251 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.251 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.251 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.734609+0000", 2020-08-07T15:08:04.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3640000000000004e-06 2020-08-07T15:08:04.252 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.252 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.252 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.734615+0000", 2020-08-07T15:08:04.253 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.228692642 2020-08-07T15:08:04.253 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.254 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.254 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.254 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.963308+0000", 2020-08-07T15:08:04.254 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9639999999999997e-06 2020-08-07T15:08:04.255 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.256 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.256 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.257 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.963312+0000", 2020-08-07T15:08:04.257 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5215000000000001e-05 2020-08-07T15:08:04.257 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.258 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.258 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.258 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.963347+0000", 2020-08-07T15:08:04.258 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8610999999999999e-05 2020-08-07T15:08:04.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.259 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.259 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.260 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.963365+0000", 2020-08-07T15:08:04.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033060149999999998 2020-08-07T15:08:04.260 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.260 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.261 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:04.261 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.966671+0000", 2020-08-07T15:08:04.261 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9855999999999997e-05 2020-08-07T15:08:04.261 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.262 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.262 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:04.262 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.966731+0000", 2020-08-07T15:08:04.262 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23216004300000001 2020-08-07T15:08:04.263 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.263 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.263 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.263 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.264 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.264 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:96015 3.4 3:2bc10706:::benchmark_data_smithi074_35447_object12001:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e56)", 2020-08-07T15:08:04.264 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:40.755925+0000", 2020-08-07T15:08:04.265 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 23.258553144, 2020-08-07T15:08:04.265 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20304256600000001, 2020-08-07T15:08:04.265 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.265 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:04.266 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:04.266 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:04.266 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:04.267 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 96015 2020-08-07T15:08:04.267 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.267 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.267 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.267 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.268 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.755925+0000", 2020-08-07T15:08:04.268 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.268 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.269 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.269 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.269 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.755925+0000", 2020-08-07T15:08:04.269 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0268000000000001e-05 2020-08-07T15:08:04.270 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.270 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.270 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.270 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.755965+0000", 2020-08-07T15:08:04.271 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5780000000000004e-06 2020-08-07T15:08:04.271 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.271 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.755972+0000", 2020-08-07T15:08:04.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1082e-05 2020-08-07T15:08:04.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.273 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.755983+0000", 2020-08-07T15:08:04.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3429999999999998e-06 2020-08-07T15:08:04.274 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.274 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.274 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.275 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.755987+0000", 2020-08-07T15:08:04.275 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034211799999999998 2020-08-07T15:08:04.275 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.275 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.276 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.276 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.759409+0000", 2020-08-07T15:08:04.276 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1285000000000004e-05 2020-08-07T15:08:04.276 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.277 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.277 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.277 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.759500+0000", 2020-08-07T15:08:04.278 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000155134 2020-08-07T15:08:04.278 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.278 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.278 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:08:04.279 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.759655+0000", 2020-08-07T15:08:04.279 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0031454679999999998 2020-08-07T15:08:04.279 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.279 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.280 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:04.280 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.762801+0000", 2020-08-07T15:08:04.280 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7299999999999996e-07 2020-08-07T15:08:04.280 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.281 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.281 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:04.281 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.762801+0000", 2020-08-07T15:08:04.282 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9682999999999998e-05 2020-08-07T15:08:04.282 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.283 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.283 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:04.283 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.762861+0000", 2020-08-07T15:08:04.283 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.10233099900000001 2020-08-07T15:08:04.284 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.284 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.284 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:04.284 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.865192+0000", 2020-08-07T15:08:04.285 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.093708455999999996 2020-08-07T15:08:04.285 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.285 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.285 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:04.286 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.958900+0000", 2020-08-07T15:08:04.286 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3600000000000002e-07 2020-08-07T15:08:04.286 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.286 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.287 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:04.287 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.958901+0000", 2020-08-07T15:08:04.287 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6581000000000005e-05 2020-08-07T15:08:04.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.288 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.288 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:04.288 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.958968+0000", 2020-08-07T15:08:04.288 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20304256600000001 2020-08-07T15:08:04.289 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.289 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.289 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.289 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.290 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.290 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:96016 3.4 3:2bc10706:::benchmark_data_smithi074_35447_object12001:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e56)", 2020-08-07T15:08:04.290 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:40.757829+0000", 2020-08-07T15:08:04.290 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 23.256649571000001, 2020-08-07T15:08:04.290 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.201245428, 2020-08-07T15:08:04.291 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.292 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:04.292 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:04.292 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:04.292 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:04.293 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 96016 2020-08-07T15:08:04.293 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.293 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.293 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.293 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.294 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.757829+0000", 2020-08-07T15:08:04.294 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.294 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.294 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.295 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.295 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.757829+0000", 2020-08-07T15:08:04.295 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00035768300000000002 2020-08-07T15:08:04.295 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.296 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.296 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.296 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.758186+0000", 2020-08-07T15:08:04.296 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2670000000000003e-06 2020-08-07T15:08:04.297 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.297 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.297 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.297 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.758193+0000", 2020-08-07T15:08:04.298 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3022e-05 2020-08-07T15:08:04.298 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.299 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.299 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.299 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.758206+0000", 2020-08-07T15:08:04.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9680000000000003e-06 2020-08-07T15:08:04.300 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.300 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.300 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.300 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.758212+0000", 2020-08-07T15:08:04.301 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038084989999999999 2020-08-07T15:08:04.301 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.302 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.302 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.302 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.762020+0000", 2020-08-07T15:08:04.303 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7879000000000003e-05 2020-08-07T15:08:04.303 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.303 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.303 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.304 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.762118+0000", 2020-08-07T15:08:04.304 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017300900000000001 2020-08-07T15:08:04.304 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.305 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.305 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:08:04.306 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.762291+0000", 2020-08-07T15:08:04.306 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026504240000000002 2020-08-07T15:08:04.306 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.306 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.307 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:04.307 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.764941+0000", 2020-08-07T15:08:04.307 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8700000000000004e-07 2020-08-07T15:08:04.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.308 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:04.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.764942+0000", 2020-08-07T15:08:04.308 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3740999999999998e-05 2020-08-07T15:08:04.309 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.309 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.309 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:04.309 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.765006+0000", 2020-08-07T15:08:04.310 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.109174536 2020-08-07T15:08:04.310 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.310 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.310 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:04.311 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.874181+0000", 2020-08-07T15:08:04.311 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.084829826999999997 2020-08-07T15:08:04.311 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.311 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.311 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:04.312 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.959010+0000", 2020-08-07T15:08:04.312 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9800000000000004e-07 2020-08-07T15:08:04.312 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.312 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.313 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:04.313 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.959011+0000", 2020-08-07T15:08:04.313 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3188000000000006e-05 2020-08-07T15:08:04.313 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.314 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.314 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:04.314 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.959074+0000", 2020-08-07T15:08:04.314 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.201245428 2020-08-07T15:08:04.315 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.315 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.315 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.324 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.324 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.325 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:104111 3.9 e58/20 3:99770d45:::benchmark_data_smithi074_35447_object13013:head v 58'6207, mlcod=58'6197)", 2020-08-07T15:08:04.325 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:45.861256+0000", 2020-08-07T15:08:04.325 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 18.153222242999998, 2020-08-07T15:08:04.326 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29017704, 2020-08-07T15:08:04.326 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.327 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:04.327 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.327 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.328 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.328 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861256+0000", 2020-08-07T15:08:04.328 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.329 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.329 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.329 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.330 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861256+0000", 2020-08-07T15:08:04.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013044700000000001 2020-08-07T15:08:04.331 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.331 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.332 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.332 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861387+0000", 2020-08-07T15:08:04.332 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7533000000000001e-05 2020-08-07T15:08:04.333 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.334 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.334 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.334 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861424+0000", 2020-08-07T15:08:04.335 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28788376900000001 2020-08-07T15:08:04.335 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.336 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.336 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.149308+0000", 2020-08-07T15:08:04.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1890000000000004e-06 2020-08-07T15:08:04.337 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.338 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.338 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.338 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.149314+0000", 2020-08-07T15:08:04.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8288999999999999e-05 2020-08-07T15:08:04.339 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.339 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.340 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.340 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.149352+0000", 2020-08-07T15:08:04.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6765999999999998e-05 2020-08-07T15:08:04.350 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.351 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.351 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.352 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.149369+0000", 2020-08-07T15:08:04.352 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0020281000000000001 2020-08-07T15:08:04.352 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.352 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.352 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:04.353 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.151397+0000", 2020-08-07T15:08:04.353 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5947e-05 2020-08-07T15:08:04.353 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.353 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.353 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:04.354 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.151433+0000", 2020-08-07T15:08:04.354 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29017704 2020-08-07T15:08:04.354 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.355 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.355 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.355 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.355 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.356 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:104112 3.9 e58/20 3:99770d45:::benchmark_data_smithi074_35447_object13013:head v 58'6208, mlcod=58'6198)", 2020-08-07T15:08:04.356 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:45.862801+0000", 2020-08-07T15:08:04.356 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 18.151677061000001, 2020-08-07T15:08:04.356 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29018473099999997, 2020-08-07T15:08:04.356 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.357 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:04.357 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.357 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.357 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.357 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.862801+0000", 2020-08-07T15:08:04.358 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.358 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.358 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.358 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.358 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.862801+0000", 2020-08-07T15:08:04.359 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0234e-05 2020-08-07T15:08:04.359 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.359 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.359 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.359 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.862851+0000", 2020-08-07T15:08:04.360 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2700000000000001e-06 2020-08-07T15:08:04.360 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.360 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.360 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.360 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.862858+0000", 2020-08-07T15:08:04.360 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28821313100000001 2020-08-07T15:08:04.361 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.361 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.361 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.361 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.151071+0000", 2020-08-07T15:08:04.370 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4140000000000002e-06 2020-08-07T15:08:04.370 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.370 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.371 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.371 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.151076+0000", 2020-08-07T15:08:04.371 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7945999999999999e-05 2020-08-07T15:08:04.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.371 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.372 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.372 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.151114+0000", 2020-08-07T15:08:04.372 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2900000000000001e-05 2020-08-07T15:08:04.372 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.373 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.373 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.151137+0000", 2020-08-07T15:08:04.373 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001801415 2020-08-07T15:08:04.373 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.373 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.374 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:04.374 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.152939+0000", 2020-08-07T15:08:04.374 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7420999999999999e-05 2020-08-07T15:08:04.374 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.374 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.374 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:04.375 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.152986+0000", 2020-08-07T15:08:04.375 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29018473099999997 2020-08-07T15:08:04.375 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.375 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.375 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.376 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106080 3.4 3:208b1d5b:::benchmark_data_smithi074_35447_object13259:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:08:04.376 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.241181+0000", 2020-08-07T15:08:04.376 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.773297613, 2020-08-07T15:08:04.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45274565500000002, 2020-08-07T15:08:04.377 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.377 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:04.377 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:04.377 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:04.378 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:04.378 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106080 2020-08-07T15:08:04.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.378 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.241181+0000", 2020-08-07T15:08:04.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.241181+0000", 2020-08-07T15:08:04.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3634999999999998e-05 2020-08-07T15:08:04.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.382 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.382 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.382 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.241224+0000", 2020-08-07T15:08:04.382 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2310000000000001e-06 2020-08-07T15:08:04.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.383 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.383 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.383 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.241228+0000", 2020-08-07T15:08:04.383 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7490000000000001e-06 2020-08-07T15:08:04.383 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.383 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.384 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.384 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.241234+0000", 2020-08-07T15:08:04.384 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1320000000000004e-06 2020-08-07T15:08:04.384 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.384 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.385 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.385 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.241238+0000", 2020-08-07T15:08:04.385 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001563659 2020-08-07T15:08:04.385 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.385 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.386 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.386 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.242802+0000", 2020-08-07T15:08:04.386 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0303999999999995e-05 2020-08-07T15:08:04.386 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.386 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.387 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.387 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.242892+0000", 2020-08-07T15:08:04.387 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013412 2020-08-07T15:08:04.387 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.387 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.388 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:08:04.388 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.243027+0000", 2020-08-07T15:08:04.388 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001917291 2020-08-07T15:08:04.388 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.388 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.389 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:04.389 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.244944+0000", 2020-08-07T15:08:04.389 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9647999999999995e-05 2020-08-07T15:08:04.390 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.391 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.391 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:04.391 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.245013+0000", 2020-08-07T15:08:04.392 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.44882515299999998 2020-08-07T15:08:04.392 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.392 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.392 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:04.392 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693839+0000", 2020-08-07T15:08:04.393 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5290000000000001e-06 2020-08-07T15:08:04.393 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.393 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.393 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:04.393 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693840+0000", 2020-08-07T15:08:04.394 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7976999999999998e-05 2020-08-07T15:08:04.394 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.394 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.394 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:04.394 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693888+0000", 2020-08-07T15:08:04.395 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8226999999999998e-05 2020-08-07T15:08:04.395 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.395 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.395 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:04.395 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693926+0000", 2020-08-07T15:08:04.395 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45274565500000002 2020-08-07T15:08:04.396 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.396 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.396 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.396 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.396 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.397 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106080 3.4 e58/20 ondisk, result = 0)", 2020-08-07T15:08:04.397 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.255977+0000", 2020-08-07T15:08:04.397 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.758500991999998, 2020-08-07T15:08:04.397 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43795860599999997, 2020-08-07T15:08:04.397 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.398 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:04.398 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.398 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.398 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.398 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255977+0000", 2020-08-07T15:08:04.398 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.399 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.399 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.399 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255977+0000", 2020-08-07T15:08:04.399 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2820000000000001e-05 2020-08-07T15:08:04.400 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.400 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.401 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.256010+0000", 2020-08-07T15:08:04.401 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8660000000000002e-06 2020-08-07T15:08:04.402 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.402 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.402 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.402 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.256016+0000", 2020-08-07T15:08:04.403 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.437661209 2020-08-07T15:08:04.403 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.403 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.403 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.403 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693677+0000", 2020-08-07T15:08:04.403 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6000000000000003e-06 2020-08-07T15:08:04.404 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.404 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.404 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.404 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693684+0000", 2020-08-07T15:08:04.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000138572 2020-08-07T15:08:04.405 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.405 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.405 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693822+0000", 2020-08-07T15:08:04.405 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6339999999999996e-06 2020-08-07T15:08:04.406 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.406 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.406 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.406 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693832+0000", 2020-08-07T15:08:04.406 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000103905 2020-08-07T15:08:04.407 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.407 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.407 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:04.407 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693936+0000", 2020-08-07T15:08:04.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43795860599999997 2020-08-07T15:08:04.407 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.408 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.408 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.408 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.408 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.408 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:108232 3.0 e62/20 3:0c86ca21:::benchmark_data_smithi074_35447_object13528:head v 62'6928, mlcod=62'6924)", 2020-08-07T15:08:04.409 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.505590+0000", 2020-08-07T15:08:04.409 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 13.508887955000001, 2020-08-07T15:08:04.409 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19165691800000001, 2020-08-07T15:08:04.409 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.409 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:04.410 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.410 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.410 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.410 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.505590+0000", 2020-08-07T15:08:04.410 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.411 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.411 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.412 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.412 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.505590+0000", 2020-08-07T15:08:04.412 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5963e-05 2020-08-07T15:08:04.413 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.413 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.414 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.505636+0000", 2020-08-07T15:08:04.414 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1609999999999996e-06 2020-08-07T15:08:04.414 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.415 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.415 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.505640+0000", 2020-08-07T15:08:04.415 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.190200166 2020-08-07T15:08:04.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.416 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.416 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.416 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.695841+0000", 2020-08-07T15:08:04.416 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3850000000000004e-06 2020-08-07T15:08:04.416 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.417 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.417 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.417 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.695845+0000", 2020-08-07T15:08:04.417 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6428999999999999e-05 2020-08-07T15:08:04.417 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.418 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.418 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.418 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.695891+0000", 2020-08-07T15:08:04.418 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8402e-05 2020-08-07T15:08:04.418 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.419 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.419 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.419 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.695920+0000", 2020-08-07T15:08:04.419 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012769190000000001 2020-08-07T15:08:04.419 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.419 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.420 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:04.420 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.697197+0000", 2020-08-07T15:08:04.420 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0492999999999999e-05 2020-08-07T15:08:04.420 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.420 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.421 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:04.421 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.697247+0000", 2020-08-07T15:08:04.421 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19165691800000001 2020-08-07T15:08:04.421 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.422 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.423 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.423 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.423 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.423 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:116623 3.9 e65/20 3:93e94c7d:::benchmark_data_smithi074_35447_object14577:head v 65'7111, mlcod=65'7102)", 2020-08-07T15:08:04.424 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:56.780860+0000", 2020-08-07T15:08:04.424 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 7.2336178430000002, 2020-08-07T15:08:04.424 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22065456899999999, 2020-08-07T15:08:04.425 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.425 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:04.425 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.425 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.426 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.780860+0000", 2020-08-07T15:08:04.426 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.426 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.426 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.426 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.780860+0000", 2020-08-07T15:08:04.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2389000000000003e-05 2020-08-07T15:08:04.427 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.427 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.780913+0000", 2020-08-07T15:08:04.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7039999999999997e-06 2020-08-07T15:08:04.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.780920+0000", 2020-08-07T15:08:04.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.219753378 2020-08-07T15:08:04.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.429 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.000673+0000", 2020-08-07T15:08:04.430 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8869999999999997e-06 2020-08-07T15:08:04.430 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.430 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.430 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.430 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.000679+0000", 2020-08-07T15:08:04.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0431000000000004e-05 2020-08-07T15:08:04.431 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.431 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.431 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.000749+0000", 2020-08-07T15:08:04.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8459e-05 2020-08-07T15:08:04.432 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.432 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.433 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.433 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.000778+0000", 2020-08-07T15:08:04.434 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00068167900000000003 2020-08-07T15:08:04.434 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.434 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.434 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:04.434 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.001459+0000", 2020-08-07T15:08:04.435 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5642e-05 2020-08-07T15:08:04.435 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.435 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.436 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:04.436 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.001515+0000", 2020-08-07T15:08:04.436 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22065456899999999 2020-08-07T15:08:04.436 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.437 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.437 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.437 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.437 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.438 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:116624 3.9 e65/20 3:93e94c7d:::benchmark_data_smithi074_35447_object14577:head v 65'7112, mlcod=65'7103)", 2020-08-07T15:08:04.438 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:56.781405+0000", 2020-08-07T15:08:04.438 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 7.2330731840000002, 2020-08-07T15:08:04.438 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.248173752, 2020-08-07T15:08:04.438 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.439 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:04.439 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.439 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.439 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.439 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.781405+0000", 2020-08-07T15:08:04.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.440 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.440 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.440 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.781405+0000", 2020-08-07T15:08:04.440 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3173999999999998e-05 2020-08-07T15:08:04.441 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.441 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.441 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.441 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.781438+0000", 2020-08-07T15:08:04.441 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1790000000000001e-06 2020-08-07T15:08:04.442 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.442 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.442 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.442 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.781442+0000", 2020-08-07T15:08:04.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24601534999999999 2020-08-07T15:08:04.444 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.444 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.445 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.027458+0000", 2020-08-07T15:08:04.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8599999999999998e-06 2020-08-07T15:08:04.445 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.446 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.446 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.446 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.027464+0000", 2020-08-07T15:08:04.446 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3399000000000003e-05 2020-08-07T15:08:04.446 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.447 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.447 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.027497+0000", 2020-08-07T15:08:04.447 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6073999999999999e-05 2020-08-07T15:08:04.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.027513+0000", 2020-08-07T15:08:04.448 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0020137979999999998 2020-08-07T15:08:04.448 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.449 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:04.449 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.029527+0000", 2020-08-07T15:08:04.449 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1918e-05 2020-08-07T15:08:04.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.449 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.450 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:04.450 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.029579+0000", 2020-08-07T15:08:04.450 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.248173752 2020-08-07T15:08:04.450 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.450 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.451 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.451 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.451 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.451 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:117016 3.4 3:2f619d09:::benchmark_data_smithi074_35447_object14626:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e65)", 2020-08-07T15:08:04.452 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:57.118050+0000", 2020-08-07T15:08:04.452 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.8964281239999998, 2020-08-07T15:08:04.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.255829902, 2020-08-07T15:08:04.452 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.452 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:04.453 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:04.453 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:04.454 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:04.454 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 117016 2020-08-07T15:08:04.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.455 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118050+0000", 2020-08-07T15:08:04.456 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.456 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118050+0000", 2020-08-07T15:08:04.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2957999999999999e-05 2020-08-07T15:08:04.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118083+0000", 2020-08-07T15:08:04.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8829999999999997e-06 2020-08-07T15:08:04.458 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.458 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.458 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.458 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118088+0000", 2020-08-07T15:08:04.458 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4660000000000005e-06 2020-08-07T15:08:04.459 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.459 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118098+0000", 2020-08-07T15:08:04.459 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.768e-06 2020-08-07T15:08:04.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.460 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.460 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118100+0000", 2020-08-07T15:08:04.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00054986900000000003 2020-08-07T15:08:04.461 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.461 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.461 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.462 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118650+0000", 2020-08-07T15:08:04.462 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7521e-05 2020-08-07T15:08:04.462 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.463 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.463 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.463 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118708+0000", 2020-08-07T15:08:04.465 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012445499999999999 2020-08-07T15:08:04.465 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.466 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.466 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:08:04.466 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118832+0000", 2020-08-07T15:08:04.467 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002948806 2020-08-07T15:08:04.467 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.467 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.468 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:04.472 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.121781+0000", 2020-08-07T15:08:04.472 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8299999999999996e-07 2020-08-07T15:08:04.473 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.473 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.473 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:04.474 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.121782+0000", 2020-08-07T15:08:04.476 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4300000000000004e-05 2020-08-07T15:08:04.477 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.477 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.477 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:04.478 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.121846+0000", 2020-08-07T15:08:04.478 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.044326363000000001 2020-08-07T15:08:04.478 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.479 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.479 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:04.479 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.166172+0000", 2020-08-07T15:08:04.480 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20763515900000001 2020-08-07T15:08:04.480 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.481 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:04.481 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.373808+0000", 2020-08-07T15:08:04.481 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6339999999999999e-06 2020-08-07T15:08:04.482 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.482 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.482 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:04.483 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.373809+0000", 2020-08-07T15:08:04.483 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0937000000000002e-05 2020-08-07T15:08:04.483 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.484 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.484 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:04.484 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.373880+0000", 2020-08-07T15:08:04.485 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.255829902 2020-08-07T15:08:04.485 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.486 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.487 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.487 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.487 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.487 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:125528 3.4 3:2c06ab2d:::benchmark_data_smithi074_35447_object15690:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e69)", 2020-08-07T15:08:04.488 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:03.323266+0000", 2020-08-07T15:08:04.488 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.69121222800000004, 2020-08-07T15:08:04.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21366406900000001, 2020-08-07T15:08:04.488 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.488 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:04.489 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:04.489 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:04.489 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:04.489 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 125528 2020-08-07T15:08:04.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.490 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.490 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.490 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.490 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.323266+0000", 2020-08-07T15:08:04.490 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.491 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.491 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.491 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.323266+0000", 2020-08-07T15:08:04.491 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1621000000000001e-05 2020-08-07T15:08:04.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.492 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.492 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.492 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.323298+0000", 2020-08-07T15:08:04.492 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1820000000000002e-06 2020-08-07T15:08:04.492 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.493 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.493 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.323301+0000", 2020-08-07T15:08:04.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4389999999999997e-06 2020-08-07T15:08:04.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.494 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.494 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.323307+0000", 2020-08-07T15:08:04.505 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9500000000000001e-06 2020-08-07T15:08:04.505 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.505 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.506 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.506 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.323310+0000", 2020-08-07T15:08:04.506 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010126709999999999 2020-08-07T15:08:04.506 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.507 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.507 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.507 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.324323+0000", 2020-08-07T15:08:04.507 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010536500000000001 2020-08-07T15:08:04.507 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.507 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.508 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.508 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.324428+0000", 2020-08-07T15:08:04.508 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017499900000000001 2020-08-07T15:08:04.508 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.508 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.509 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:08:04.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.324603+0000", 2020-08-07T15:08:04.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023473299999999999 2020-08-07T15:08:04.509 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.509 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.510 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:04.510 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.326951+0000", 2020-08-07T15:08:04.510 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.23e-07 2020-08-07T15:08:04.511 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:04.512 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.326951+0000", 2020-08-07T15:08:04.512 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3559999999999997e-05 2020-08-07T15:08:04.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.512 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.512 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:04.512 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.326985+0000", 2020-08-07T15:08:04.513 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00041237000000000001 2020-08-07T15:08:04.513 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.513 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.513 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:04.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.327397+0000", 2020-08-07T15:08:04.514 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20945345100000001 2020-08-07T15:08:04.514 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.514 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:04.514 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.536851+0000", 2020-08-07T15:08:04.515 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.621e-06 2020-08-07T15:08:04.515 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.516 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.516 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:04.516 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.536852+0000", 2020-08-07T15:08:04.516 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7787000000000006e-05 2020-08-07T15:08:04.516 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.517 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.517 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:04.517 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.536930+0000", 2020-08-07T15:08:04.518 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21366406900000001 2020-08-07T15:08:04.518 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.518 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.518 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.518 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.519 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.519 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:04.519 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:04.017+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:04.520 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:08:04.523 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:08:04.523 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 88 16 126982 126966 11.2709 12.2266 0.00851922 0.011086 2020-08-07T15:08:04.524 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:04.520+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:04.628 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:04.625+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:04.728 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:04.727+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:04.779 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:04.780 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:08:04.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.780 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:127735 3.7 3:edc06f24:::benchmark_data_smithi074_35447_object15966:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e70)", 2020-08-07T15:08:04.780 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:04.716411+0000", 2020-08-07T15:08:04.780 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.059722174000000003, 2020-08-07T15:08:04.781 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.059752115000000001, 2020-08-07T15:08:04.781 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.781 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:08:04.781 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:04.781 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:04.782 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:04.782 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 127735 2020-08-07T15:08:04.782 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.782 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.716411+0000", 2020-08-07T15:08:04.783 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.783 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.716411+0000", 2020-08-07T15:08:04.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6208000000000001e-05 2020-08-07T15:08:04.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.716467+0000", 2020-08-07T15:08:04.785 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4249999999999997e-06 2020-08-07T15:08:04.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.786 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.786 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.716473+0000", 2020-08-07T15:08:04.786 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0298e-05 2020-08-07T15:08:04.787 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.787 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.787 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.716483+0000", 2020-08-07T15:08:04.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4499999999999997e-06 2020-08-07T15:08:04.789 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.789 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.789 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.790 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.716487+0000", 2020-08-07T15:08:04.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0047761430000000001 2020-08-07T15:08:04.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.791 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.721264+0000", 2020-08-07T15:08:04.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9061999999999994e-05 2020-08-07T15:08:04.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.792 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.792 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.721363+0000", 2020-08-07T15:08:04.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019368099999999999 2020-08-07T15:08:04.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.793 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.793 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:08:04.793 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.721556+0000", 2020-08-07T15:08:04.793 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0018921299999999999 2020-08-07T15:08:04.793 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.794 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.794 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:04.794 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.723448+0000", 2020-08-07T15:08:04.794 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0025356100000000002 2020-08-07T15:08:04.794 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.794 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.795 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:04.795 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.725984+0000", 2020-08-07T15:08:04.795 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0095730069999999997 2020-08-07T15:08:04.795 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.795 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.796 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.796 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.796 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:127736 3.7 3:edc06f24:::benchmark_data_smithi074_35447_object15966:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e70)", 2020-08-07T15:08:04.796 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:04.721525+0000", 2020-08-07T15:08:04.797 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.054608516000000003, 2020-08-07T15:08:04.797 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.054752332000000001, 2020-08-07T15:08:04.797 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.797 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:08:04.797 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:04.798 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:04.798 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:04.798 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 127736 2020-08-07T15:08:04.798 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.798 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.721525+0000", 2020-08-07T15:08:04.810 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.810 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.811 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.811 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.721525+0000", 2020-08-07T15:08:04.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0627999999999997e-05 2020-08-07T15:08:04.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.721575+0000", 2020-08-07T15:08:04.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4779999999999996e-06 2020-08-07T15:08:04.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.838 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.838 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.721582+0000", 2020-08-07T15:08:04.838 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3932e-05 2020-08-07T15:08:04.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.839 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.721596+0000", 2020-08-07T15:08:04.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9750000000000001e-06 2020-08-07T15:08:04.840 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.840 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.840 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.840 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.721603+0000", 2020-08-07T15:08:04.840 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000427646 2020-08-07T15:08:04.841 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.841 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.722030+0000", 2020-08-07T15:08:04.841 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000101809 2020-08-07T15:08:04.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.843 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.722132+0000", 2020-08-07T15:08:04.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017808400000000001 2020-08-07T15:08:04.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.844 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:08:04.844 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.722310+0000", 2020-08-07T15:08:04.844 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001184874 2020-08-07T15:08:04.845 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:04.845 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.723495+0000", 2020-08-07T15:08:04.845 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024973109999999999 2020-08-07T15:08:04.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.846 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.846 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:04.846 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.725992+0000", 2020-08-07T15:08:04.846 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044677370000000003 2020-08-07T15:08:04.847 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.847 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.847 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.847 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.847 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.847 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:127817 3.5 e70/20 3:a8ad16b8:::benchmark_data_smithi074_35447_object15977:head v 70'7905, mlcod=70'7903)", 2020-08-07T15:08:04.848 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:04.774038+0000", 2020-08-07T15:08:04.848 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0020953249999999999, 2020-08-07T15:08:04.848 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023291599999999998, 2020-08-07T15:08:04.848 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.848 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:04.849 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.849 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.849 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.849 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.774038+0000", 2020-08-07T15:08:04.849 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.850 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.850 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.850 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.774038+0000", 2020-08-07T15:08:04.850 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8120000000000001e-05 2020-08-07T15:08:04.851 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.851 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.851 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.851 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.774076+0000", 2020-08-07T15:08:04.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2948e-05 2020-08-07T15:08:04.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.852 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.774089+0000", 2020-08-07T15:08:04.853 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00092453299999999995 2020-08-07T15:08:04.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.853 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.853 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.775013+0000", 2020-08-07T15:08:04.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5559999999999998e-06 2020-08-07T15:08:04.854 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.854 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.854 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.854 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.775019+0000", 2020-08-07T15:08:04.855 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9531999999999999e-05 2020-08-07T15:08:04.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.855 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.855 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.775069+0000", 2020-08-07T15:08:04.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5510000000000001e-05 2020-08-07T15:08:04.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.856 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.856 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.856 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.775094+0000", 2020-08-07T15:08:04.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010561990000000001 2020-08-07T15:08:04.857 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.857 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.857 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.858 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:127756 3.d 3:b346e464:::benchmark_data_smithi074_35447_object15969:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e70)", 2020-08-07T15:08:04.858 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:04.736357+0000", 2020-08-07T15:08:04.858 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.039776236, 2020-08-07T15:08:04.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.040087511999999999, 2020-08-07T15:08:04.858 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.859 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:04.859 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:04.859 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:04.859 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:04.860 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 127756 2020-08-07T15:08:04.860 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.860 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.860 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.861 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.736357+0000", 2020-08-07T15:08:04.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.861 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.862 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.736357+0000", 2020-08-07T15:08:04.862 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6558999999999999e-05 2020-08-07T15:08:04.862 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.862 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.862 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.736404+0000", 2020-08-07T15:08:04.863 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0419999999999999e-06 2020-08-07T15:08:04.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.736410+0000", 2020-08-07T15:08:04.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3896e-05 2020-08-07T15:08:04.864 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.736423+0000", 2020-08-07T15:08:04.865 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1149999999999996e-06 2020-08-07T15:08:04.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.865 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.865 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.736431+0000", 2020-08-07T15:08:04.866 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9345999999999995e-05 2020-08-07T15:08:04.866 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.866 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.866 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.866 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.736520+0000", 2020-08-07T15:08:04.867 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010855500000000001 2020-08-07T15:08:04.867 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.867 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.867 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.867 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.736628+0000", 2020-08-07T15:08:04.868 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020104399999999999 2020-08-07T15:08:04.868 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.868 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.868 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:08:04.868 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.736830+0000", 2020-08-07T15:08:04.868 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0035156580000000001 2020-08-07T15:08:04.869 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.869 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.869 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:04.869 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.740345+0000", 2020-08-07T15:08:04.869 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3540000000000001e-06 2020-08-07T15:08:04.870 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.870 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.870 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:04.870 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.740347+0000", 2020-08-07T15:08:04.870 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6638999999999996e-05 2020-08-07T15:08:04.871 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.871 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:04.871 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.740423+0000", 2020-08-07T15:08:04.871 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015111079999999999 2020-08-07T15:08:04.871 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.872 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.872 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:04.872 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.741934+0000", 2020-08-07T15:08:04.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0055773159999999997 2020-08-07T15:08:04.872 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.873 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.873 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.873 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.873 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:127757 3.d 3:b346e464:::benchmark_data_smithi074_35447_object15969:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e70)", 2020-08-07T15:08:04.874 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:04.737430+0000", 2020-08-07T15:08:04.874 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.038703020999999997, 2020-08-07T15:08:04.874 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.039129828999999998, 2020-08-07T15:08:04.874 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.874 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:04.875 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:04.875 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:04.875 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:04.875 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 127757 2020-08-07T15:08:04.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.875 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.876 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.737430+0000", 2020-08-07T15:08:04.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.876 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.877 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.877 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.877 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.737430+0000", 2020-08-07T15:08:04.877 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6520000000000002e-05 2020-08-07T15:08:04.877 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.880 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.881 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.737477+0000", 2020-08-07T15:08:04.881 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8950000000000001e-06 2020-08-07T15:08:04.881 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.882 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.882 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.882 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.737484+0000", 2020-08-07T15:08:04.882 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2449999999999999e-05 2020-08-07T15:08:04.883 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.883 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.883 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.883 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.737496+0000", 2020-08-07T15:08:04.883 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8579999999999996e-06 2020-08-07T15:08:04.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.884 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.884 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.737502+0000", 2020-08-07T15:08:04.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00023869700000000001 2020-08-07T15:08:04.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.885 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.737741+0000", 2020-08-07T15:08:04.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6706000000000005e-05 2020-08-07T15:08:04.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.886 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.886 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.737837+0000", 2020-08-07T15:08:04.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016585699999999999 2020-08-07T15:08:04.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:08:04.887 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.738003+0000", 2020-08-07T15:08:04.887 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024313389999999998 2020-08-07T15:08:04.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:04.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.740434+0000", 2020-08-07T15:08:04.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0999999999999998e-07 2020-08-07T15:08:04.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:04.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.740435+0000", 2020-08-07T15:08:04.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3898000000000004e-05 2020-08-07T15:08:04.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.890 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:04.890 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.740499+0000", 2020-08-07T15:08:04.890 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001475002 2020-08-07T15:08:04.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.890 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:04.891 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.741974+0000", 2020-08-07T15:08:04.891 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045439319999999997 2020-08-07T15:08:04.891 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.891 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.891 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.892 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.892 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.892 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:127758 3.d 3:b346e464:::benchmark_data_smithi074_35447_object15969:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e70)", 2020-08-07T15:08:04.892 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:04.737537+0000", 2020-08-07T15:08:04.892 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.038595891, 2020-08-07T15:08:04.893 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.039097270000000003, 2020-08-07T15:08:04.893 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.893 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:04.893 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:04.893 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:04.893 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:04.894 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 127758 2020-08-07T15:08:04.894 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.894 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.894 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.894 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.895 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.737537+0000", 2020-08-07T15:08:04.895 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.895 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.895 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.895 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.896 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.737537+0000", 2020-08-07T15:08:04.896 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2317000000000002e-05 2020-08-07T15:08:04.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.896 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.896 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.737580+0000", 2020-08-07T15:08:04.897 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0050000000000004e-06 2020-08-07T15:08:04.897 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.897 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.897 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.897 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.737585+0000", 2020-08-07T15:08:04.898 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7499999999999998e-06 2020-08-07T15:08:04.898 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.898 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.898 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.898 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.737594+0000", 2020-08-07T15:08:04.899 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9559999999999999e-06 2020-08-07T15:08:04.899 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.899 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.899 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.899 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.737598+0000", 2020-08-07T15:08:04.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00087193399999999999 2020-08-07T15:08:04.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.900 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.900 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.738470+0000", 2020-08-07T15:08:04.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1550999999999994e-05 2020-08-07T15:08:04.901 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.901 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.901 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.901 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.738532+0000", 2020-08-07T15:08:04.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000101309 2020-08-07T15:08:04.902 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.902 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.902 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:08:04.902 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.738633+0000", 2020-08-07T15:08:04.902 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0018743 2020-08-07T15:08:04.903 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.903 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:04.903 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.740507+0000", 2020-08-07T15:08:04.903 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.06e-07 2020-08-07T15:08:04.903 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.904 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:04.904 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.740508+0000", 2020-08-07T15:08:04.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9002999999999997e-05 2020-08-07T15:08:04.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:04.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.740567+0000", 2020-08-07T15:08:04.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0014445149999999999 2020-08-07T15:08:04.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.906 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.906 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:04.906 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.742011+0000", 2020-08-07T15:08:04.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004474146 2020-08-07T15:08:04.907 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.907 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.907 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.908 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:127759 3.d 3:b346e464:::benchmark_data_smithi074_35447_object15969:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e70)", 2020-08-07T15:08:04.914 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:04.737629+0000", 2020-08-07T15:08:04.914 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.038504534, 2020-08-07T15:08:04.914 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.039073832000000003, 2020-08-07T15:08:04.918 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.918 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:04.918 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:04.918 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:04.919 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:04.919 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 127759 2020-08-07T15:08:04.919 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.919 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.920 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.737629+0000", 2020-08-07T15:08:04.920 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.737629+0000", 2020-08-07T15:08:04.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7274000000000002e-05 2020-08-07T15:08:04.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.737676+0000", 2020-08-07T15:08:04.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6879999999999998e-06 2020-08-07T15:08:04.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.737681+0000", 2020-08-07T15:08:04.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2399999999999996e-06 2020-08-07T15:08:04.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.737690+0000", 2020-08-07T15:08:04.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.827e-06 2020-08-07T15:08:04.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.737694+0000", 2020-08-07T15:08:04.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012555380000000001 2020-08-07T15:08:04.925 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.926 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.738949+0000", 2020-08-07T15:08:04.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000106174 2020-08-07T15:08:04.926 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.926 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.739055+0000", 2020-08-07T15:08:04.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018923099999999999 2020-08-07T15:08:04.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:08:04.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.739245+0000", 2020-08-07T15:08:04.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013318119999999999 2020-08-07T15:08:04.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:04.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.740576+0000", 2020-08-07T15:08:04.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9400000000000005e-07 2020-08-07T15:08:04.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:04.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.740577+0000", 2020-08-07T15:08:04.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8469999999999998e-05 2020-08-07T15:08:04.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:04.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.740666+0000", 2020-08-07T15:08:04.931 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013828309999999999 2020-08-07T15:08:04.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.932 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.932 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:04.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.742048+0000", 2020-08-07T15:08:04.932 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044196790000000001 2020-08-07T15:08:04.932 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.932 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.933 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.933 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.933 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:127760 3.d 3:b346e464:::benchmark_data_smithi074_35447_object15969:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e70)", 2020-08-07T15:08:04.933 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:04.739288+0000", 2020-08-07T15:08:04.934 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.036845423000000002, 2020-08-07T15:08:04.934 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.037513721, 2020-08-07T15:08:04.934 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:04.934 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:04.934 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:04.935 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:04.935 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:04.935 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 127760 2020-08-07T15:08:04.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.935 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:04.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:04.936 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.739288+0000", 2020-08-07T15:08:04.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:04.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:04.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.739288+0000", 2020-08-07T15:08:04.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4832e-05 2020-08-07T15:08:04.937 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.938 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:04.938 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.739333+0000", 2020-08-07T15:08:04.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0560000000000001e-06 2020-08-07T15:08:04.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.938 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:04.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.739339+0000", 2020-08-07T15:08:04.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1545e-05 2020-08-07T15:08:04.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:04.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.739350+0000", 2020-08-07T15:08:04.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7030000000000003e-06 2020-08-07T15:08:04.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.942 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:04.942 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.739356+0000", 2020-08-07T15:08:04.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00034598300000000001 2020-08-07T15:08:04.942 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:04.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.739702+0000", 2020-08-07T15:08:04.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000103557 2020-08-07T15:08:04.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.944 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:04.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.739805+0000", 2020-08-07T15:08:04.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017546400000000001 2020-08-07T15:08:04.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:08:04.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.739981+0000", 2020-08-07T15:08:04.946 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000952231 2020-08-07T15:08:04.946 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.946 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.946 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:04.947 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.740933+0000", 2020-08-07T15:08:04.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6200000000000006e-07 2020-08-07T15:08:04.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.947 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.947 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:04.948 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.740934+0000", 2020-08-07T15:08:04.948 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3689999999999995e-05 2020-08-07T15:08:04.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.948 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:04.949 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.740998+0000", 2020-08-07T15:08:04.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001335131 2020-08-07T15:08:04.949 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:04.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:04.949 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:04.949 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.742333+0000", 2020-08-07T15:08:04.950 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003045154 2020-08-07T15:08:04.950 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.950 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:04.950 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.950 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:04.951 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:08:04.951 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 8 2020-08-07T15:08:04.951 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:04.953 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:08:04.954 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:08:04.954 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:08:04.954 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:08:05.014 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:04.952+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:05.055 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:08:05.071 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:05.070+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:05.115 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:05.116 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:05.124 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:08:05.125 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:08:05.152 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:08:05.175 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:05.173+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:05.195 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:08:05.236 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:08:05.250 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:08:05.251 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:08:05.251 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:08:05.251 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:08:05.252 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:08:05.252 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:08:05.252 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:08:05.252 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:08:05.252 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 89 16 128249 128233 11.2554 9.89844 0.00470594 0.0111014 2020-08-07T15:08:05.296 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:05.295+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:05.342 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:05.342 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:05.367 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:08:05.393 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:05.392+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:05.437 INFO:teuthology.orchestra.run.smithi074.stderr:set pool 3 pgp_num to 16 2020-08-07T15:08:05.496 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:05.495+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:05.513 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:08:05.541 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:05.542 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:05.556 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:08:05.674 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:05.607+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:05.686 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:05.685+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:05.694 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:05.694 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:05.711 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:08:05.788 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:05.786+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:05.892 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:05.889+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:05.920 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:05.920 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:05.941 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:08:05.994 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:05.993+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:06.095 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:06.093+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:06.119 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:06.120 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:06.129 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:08:06.221 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:06.220+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:06.249 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 90 5 129606 129601 11.2491 10.6875 0.00791328 0.0111065 2020-08-07T15:08:06.313 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:06.314 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:06.314 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:06.313+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:06.325 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:08:06.422 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:06.417+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:06.438 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:08:06.492 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:06.493 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:06.501 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:08:06.547 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:06.541+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:06.624 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:06.622+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:06.666 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:06.666 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:06.673 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:08:06.727 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:06.724+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:06.831 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:06.829+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:06.840 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:06.840 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:06.847 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:08:06.935 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:06.933+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:06.966 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:06.967 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:06.972 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:08:07.059 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:07.057+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:07.132 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:07.133 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:07.145 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:08:07.146 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:07.143+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:07.257 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:08:07.258 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Total time run: 90.4951 2020-08-07T15:08:07.259 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Total writes made: 129606 2020-08-07T15:08:07.259 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Write size: 8192 2020-08-07T15:08:07.259 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Object size: 65536 2020-08-07T15:08:07.260 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Bandwidth (MB/sec): 11.189 2020-08-07T15:08:07.260 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Stddev Bandwidth: 3.06898 2020-08-07T15:08:07.260 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Max bandwidth (MB/sec): 14.3594 2020-08-07T15:08:07.261 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Min bandwidth (MB/sec): 0 2020-08-07T15:08:07.261 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Average IOPS: 1432 2020-08-07T15:08:07.261 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Stddev IOPS: 392.83 2020-08-07T15:08:07.263 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Max IOPS: 1838 2020-08-07T15:08:07.264 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Min IOPS: 0 2020-08-07T15:08:07.264 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Average Latency(s): 0.0111206 2020-08-07T15:08:07.264 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Stddev Latency(s): 0.0368216 2020-08-07T15:08:07.267 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Max latency(s): 2.03718 2020-08-07T15:08:07.267 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Min latency(s): 0.000913402 2020-08-07T15:08:07.268 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Cleaning up (deleting benchmark objects) 2020-08-07T15:08:07.269 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:07.256+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:07.342 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:07.343 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:07.356 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:08:07.358 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:07.357+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:07.459 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:07.457+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:07.558 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:08:07.566 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:07.561+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:07.575 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:07.576 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:07.592 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:08:07.612 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:08:07.667 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:08:07.668 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:07.667+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:07.727 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:08:07.769 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:07.767+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:07.789 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:08:07.843 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:07.843 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:08:07.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:07.844 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:131060 3.1 e72/67 3:804dde19:::benchmark_data_smithi074_35447_object1451:head v 72'7838, mlcod=72'7835)", 2020-08-07T15:08:07.844 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:07.816102+0000", 2020-08-07T15:08:07.844 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.026238609999999999, 2020-08-07T15:08:07.844 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.026263501000000002, 2020-08-07T15:08:07.844 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:07.845 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:07.845 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:07.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:07.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:07.846 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:07.816102+0000", 2020-08-07T15:08:07.846 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:07.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:07.846 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:07.846 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:07.847 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:07.816102+0000", 2020-08-07T15:08:07.847 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0921999999999999e-05 2020-08-07T15:08:07.847 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:07.847 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:07.847 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:07.847 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:07.816143+0000", 2020-08-07T15:08:07.848 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2389999999999999e-06 2020-08-07T15:08:07.848 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:07.848 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:07.848 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:07.849 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:07.816149+0000", 2020-08-07T15:08:07.849 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.025339225999999999 2020-08-07T15:08:07.849 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:07.849 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:07.849 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:07.849 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:07.841488+0000", 2020-08-07T15:08:07.850 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3970000000000002e-06 2020-08-07T15:08:07.850 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:07.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:07.850 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:07.850 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:07.841494+0000", 2020-08-07T15:08:07.851 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4202999999999997e-05 2020-08-07T15:08:07.851 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:07.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:07.853 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:07.853 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:07.841538+0000", 2020-08-07T15:08:07.853 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3538000000000001e-05 2020-08-07T15:08:07.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:07.854 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:07.854 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:07.854 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:07.841562+0000", 2020-08-07T15:08:07.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.025459525 2020-08-07T15:08:07.854 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:07.855 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:07.855 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:07.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:07.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:07.855 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:131102 3.7 e72/20 3:e25d83b5:::benchmark_data_smithi074_35447_object1493:head v 72'7943, mlcod=72'7941)", 2020-08-07T15:08:07.856 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:07.841697+0000", 2020-08-07T15:08:07.856 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.00064400500000000001, 2020-08-07T15:08:07.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00074459099999999996, 2020-08-07T15:08:07.856 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:07.856 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:07.857 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:07.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:07.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:07.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:07.841697+0000", 2020-08-07T15:08:07.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:07.858 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:07.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:07.858 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:07.858 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:07.841697+0000", 2020-08-07T15:08:07.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0189999999999999e-05 2020-08-07T15:08:07.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:07.859 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:07.859 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:07.859 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:07.841747+0000", 2020-08-07T15:08:07.859 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6649999999999997e-06 2020-08-07T15:08:07.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:07.860 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:07.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:07.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:07.841753+0000", 2020-08-07T15:08:07.860 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7104e-05 2020-08-07T15:08:07.860 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:07.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:07.861 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:07.861 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:07.841781+0000", 2020-08-07T15:08:07.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.339e-06 2020-08-07T15:08:07.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:07.862 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:07.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:07.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:07.841786+0000", 2020-08-07T15:08:07.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2120000000000003e-05 2020-08-07T15:08:07.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:07.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:07.865 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:07.865 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:07.841828+0000", 2020-08-07T15:08:07.865 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1443999999999999e-05 2020-08-07T15:08:07.866 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:07.866 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:07.866 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:07.866 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:07.841859+0000", 2020-08-07T15:08:07.866 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000162862 2020-08-07T15:08:07.867 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:07.867 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:07.867 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:07.867 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:07.867 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:08:07.868 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 2 2020-08-07T15:08:07.868 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:07.869 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:08:07.870 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:08:07.871 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:07.868+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:07.897 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:08:07.949 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:08:07.954 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:08:07.955 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:08:07.955 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:08:07.955 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:08:07.955 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:08:07.955 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:08:07.956 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:08:07.956 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:08:07.973 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:07.970+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:08.074 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:08.071+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:08.122 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:08.123 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:08:08.123 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:08:08.123 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:08:08.123 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:08.136 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:08:08.194 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:08.193+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:08.277 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:08.275+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:08.349 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:08.350 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:08:08.350 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:08:08.350 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:08:08.350 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.350 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:10240 3.6 3:665d71ff:::benchmark_data_smithi074_35447_object1279:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:08:08.360 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:08:08.360 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 85.952458684000007, 2020-08-07T15:08:08.360 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25191569699999999, 2020-08-07T15:08:08.361 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:08.361 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:08.361 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:08.361 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:08.361 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:08.362 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 10240 2020-08-07T15:08:08.362 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.362 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:08.363 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.363 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:08.363 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:08:08.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:08.363 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:08.364 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:08:08.364 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5463000000000002e-05 2020-08-07T15:08:08.364 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.365 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.365 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:08.365 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393264+0000", 2020-08-07T15:08:08.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9219999999999999e-06 2020-08-07T15:08:08.365 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.366 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:08.366 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393268+0000", 2020-08-07T15:08:08.366 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6149999999999997e-06 2020-08-07T15:08:08.367 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.367 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.367 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:08.369 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393277+0000", 2020-08-07T15:08:08.369 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2039999999999996e-06 2020-08-07T15:08:08.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.370 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:08.370 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393282+0000", 2020-08-07T15:08:08.370 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001225243 2020-08-07T15:08:08.370 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.370 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.371 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:08.371 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394507+0000", 2020-08-07T15:08:08.371 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7108999999999995e-05 2020-08-07T15:08:08.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.372 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:08.372 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394605+0000", 2020-08-07T15:08:08.372 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017490800000000001 2020-08-07T15:08:08.373 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.373 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.373 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:08.373 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394779+0000", 2020-08-07T15:08:08.373 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032608699999999999 2020-08-07T15:08:08.374 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.374 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.374 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:08.374 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.398040+0000", 2020-08-07T15:08:08.374 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013015259999999999 2020-08-07T15:08:08.375 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.375 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:08.375 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.399342+0000", 2020-08-07T15:08:08.375 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24566893400000001 2020-08-07T15:08:08.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.376 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:08.376 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645011+0000", 2020-08-07T15:08:08.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.217e-06 2020-08-07T15:08:08.377 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:08.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645012+0000", 2020-08-07T15:08:08.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0803000000000006e-05 2020-08-07T15:08:08.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:08.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645083+0000", 2020-08-07T15:08:08.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1882999999999998e-05 2020-08-07T15:08:08.379 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:08.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645135+0000", 2020-08-07T15:08:08.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25191569699999999 2020-08-07T15:08:08.380 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.380 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:08.380 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.381 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:43335 3.6 e32/26 3:65a57ed3:::benchmark_data_smithi074_35447_object5416:head v 32'2831, mlcod=32'2822)", 2020-08-07T15:08:08.381 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:03.998850+0000", 2020-08-07T15:08:08.381 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 64.346827613000002, 2020-08-07T15:08:08.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.221384421, 2020-08-07T15:08:08.382 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:08.382 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:08.382 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:08.382 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.382 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:08.383 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998850+0000", 2020-08-07T15:08:08.383 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:08.383 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.383 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.383 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:08.384 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998850+0000", 2020-08-07T15:08:08.384 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4788000000000004e-05 2020-08-07T15:08:08.384 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.384 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.384 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:08.385 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998915+0000", 2020-08-07T15:08:08.385 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7909999999999999e-06 2020-08-07T15:08:08.385 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.385 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.385 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:08.386 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998922+0000", 2020-08-07T15:08:08.386 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22050499700000001 2020-08-07T15:08:08.386 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.386 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.386 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:08.387 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219427+0000", 2020-08-07T15:08:08.387 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0950000000000001e-06 2020-08-07T15:08:08.387 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.387 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.387 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:08.388 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219433+0000", 2020-08-07T15:08:08.388 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6342999999999997e-05 2020-08-07T15:08:08.388 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.388 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.388 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:08.389 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219509+0000", 2020-08-07T15:08:08.389 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4173999999999999e-05 2020-08-07T15:08:08.389 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.389 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.390 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:08.390 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219533+0000", 2020-08-07T15:08:08.390 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00064214899999999995 2020-08-07T15:08:08.390 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.390 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.391 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:08.391 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.220175+0000", 2020-08-07T15:08:08.391 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9083999999999999e-05 2020-08-07T15:08:08.391 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.391 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.392 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:08.392 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.220234+0000", 2020-08-07T15:08:08.392 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.221384421 2020-08-07T15:08:08.392 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.392 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:08.393 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.393 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.393 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.393 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:43336 3.6 e32/26 3:65a57ed3:::benchmark_data_smithi074_35447_object5416:head v 32'2832, mlcod=32'2822)", 2020-08-07T15:08:08.393 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:03.999509+0000", 2020-08-07T15:08:08.394 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 64.346168284000001, 2020-08-07T15:08:08.394 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22632864799999999, 2020-08-07T15:08:08.394 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:08.394 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:08.395 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:08.395 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.395 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:08.395 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999509+0000", 2020-08-07T15:08:08.395 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:08.396 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.396 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.396 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:08.396 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999509+0000", 2020-08-07T15:08:08.396 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0107e-05 2020-08-07T15:08:08.397 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.397 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.397 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:08.397 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999559+0000", 2020-08-07T15:08:08.397 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0889999999999996e-06 2020-08-07T15:08:08.398 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.398 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.398 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:08.398 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999566+0000", 2020-08-07T15:08:08.399 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22332094599999999 2020-08-07T15:08:08.399 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.399 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:08.400 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222887+0000", 2020-08-07T15:08:08.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5990000000000001e-06 2020-08-07T15:08:08.400 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.401 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:08.401 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222892+0000", 2020-08-07T15:08:08.401 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2342999999999998e-05 2020-08-07T15:08:08.401 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.401 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.402 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:08.402 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222924+0000", 2020-08-07T15:08:08.402 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9108999999999999e-05 2020-08-07T15:08:08.402 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.403 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.403 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:08.403 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222944+0000", 2020-08-07T15:08:08.403 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028409289999999999 2020-08-07T15:08:08.403 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.404 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.404 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:08.404 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.225784+0000", 2020-08-07T15:08:08.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3526000000000002e-05 2020-08-07T15:08:08.404 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.405 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:08.405 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.225838+0000", 2020-08-07T15:08:08.405 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22632864799999999 2020-08-07T15:08:08.405 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.406 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:08.406 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.406 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.406 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.406 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:43784 3.6 e32/26 3:6da53356:::benchmark_data_smithi074_35447_object5472:head v 32'2848, mlcod=32'2839)", 2020-08-07T15:08:08.407 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:04.253355+0000", 2020-08-07T15:08:08.407 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 64.092322711999998, 2020-08-07T15:08:08.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21456700500000001, 2020-08-07T15:08:08.407 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:08.407 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:08.408 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:08.408 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.408 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:08.408 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253355+0000", 2020-08-07T15:08:08.408 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:08.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.409 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.411 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:08.411 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253355+0000", 2020-08-07T15:08:08.412 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6168000000000002e-05 2020-08-07T15:08:08.412 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.413 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:08.414 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253401+0000", 2020-08-07T15:08:08.414 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3179999999999997e-06 2020-08-07T15:08:08.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.415 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:08.415 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253406+0000", 2020-08-07T15:08:08.415 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21194338600000001 2020-08-07T15:08:08.416 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.416 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.416 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:08.416 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465350+0000", 2020-08-07T15:08:08.417 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3759999999999999e-06 2020-08-07T15:08:08.417 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.417 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.417 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:08.417 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465355+0000", 2020-08-07T15:08:08.418 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8250999999999997e-05 2020-08-07T15:08:08.418 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.418 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.418 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:08.418 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465393+0000", 2020-08-07T15:08:08.419 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0296999999999999e-05 2020-08-07T15:08:08.419 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.419 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.420 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:08.420 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465414+0000", 2020-08-07T15:08:08.420 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024515209999999999 2020-08-07T15:08:08.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.421 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.421 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:08.422 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.467865+0000", 2020-08-07T15:08:08.426 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6687999999999997e-05 2020-08-07T15:08:08.426 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.427 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:08.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.467922+0000", 2020-08-07T15:08:08.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21456700500000001 2020-08-07T15:08:08.429 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.429 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:08.429 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.430 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.430 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.432 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:44152 3.6 e32/26 3:6c90cb4b:::benchmark_data_smithi074_35447_object5518:head v 32'2856, mlcod=32'2851)", 2020-08-07T15:08:08.433 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:04.473943+0000", 2020-08-07T15:08:08.434 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 63.871734889999999, 2020-08-07T15:08:08.434 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18875260999999999, 2020-08-07T15:08:08.435 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:08.435 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:08.435 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:08.436 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.436 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:08.436 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.473943+0000", 2020-08-07T15:08:08.436 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:08.437 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.437 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.437 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:08.438 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.473943+0000", 2020-08-07T15:08:08.438 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8267999999999999e-05 2020-08-07T15:08:08.438 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.439 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.439 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:08.439 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.473991+0000", 2020-08-07T15:08:08.440 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9689999999999999e-06 2020-08-07T15:08:08.440 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.442 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:08.442 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.473997+0000", 2020-08-07T15:08:08.442 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18142124000000001 2020-08-07T15:08:08.443 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.443 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.443 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:08.443 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.655418+0000", 2020-08-07T15:08:08.443 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7439999999999998e-06 2020-08-07T15:08:08.444 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.444 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:08.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.655423+0000", 2020-08-07T15:08:08.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0052999999999997e-05 2020-08-07T15:08:08.445 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.445 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.445 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:08.445 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.655473+0000", 2020-08-07T15:08:08.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3739000000000001e-05 2020-08-07T15:08:08.446 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.446 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.446 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:08.446 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.655497+0000", 2020-08-07T15:08:08.446 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0071399030000000004 2020-08-07T15:08:08.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:08.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.662637+0000", 2020-08-07T15:08:08.448 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8693999999999997e-05 2020-08-07T15:08:08.448 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.449 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.449 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:08.449 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.662695+0000", 2020-08-07T15:08:08.449 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18875260999999999 2020-08-07T15:08:08.449 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.449 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:08.450 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.450 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:54200 3.3 e37/20 3:cc40637e:::benchmark_data_smithi074_35447_object6774:head v 37'3248, mlcod=37'3236)", 2020-08-07T15:08:08.450 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:08:08.451 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 55.247115463999997, 2020-08-07T15:08:08.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24065872499999999, 2020-08-07T15:08:08.451 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:08.451 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:08.451 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:08.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:08.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:08:08.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:08.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.453 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:08.453 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:08:08.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1671000000000002e-05 2020-08-07T15:08:08.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.454 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.454 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:08.454 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098594+0000", 2020-08-07T15:08:08.454 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0890000000000002e-06 2020-08-07T15:08:08.454 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:08.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098598+0000", 2020-08-07T15:08:08.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23875592300000001 2020-08-07T15:08:08.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:08.456 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337354+0000", 2020-08-07T15:08:08.456 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0420000000000002e-06 2020-08-07T15:08:08.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:08.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337359+0000", 2020-08-07T15:08:08.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0586000000000001e-05 2020-08-07T15:08:08.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.458 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.458 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:08.458 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337399+0000", 2020-08-07T15:08:08.458 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9415999999999998e-05 2020-08-07T15:08:08.459 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:08.459 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337419+0000", 2020-08-07T15:08:08.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017568390000000001 2020-08-07T15:08:08.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.460 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.460 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:08.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.339176+0000", 2020-08-07T15:08:08.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5158999999999999e-05 2020-08-07T15:08:08.461 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.461 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.461 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:08.461 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.339221+0000", 2020-08-07T15:08:08.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24065872499999999 2020-08-07T15:08:08.462 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.462 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:08.462 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.462 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.462 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.463 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:54728 3.7 e37/20 3:ee999259:::benchmark_data_smithi074_35447_object6840:head v 37'3328, mlcod=37'3319)", 2020-08-07T15:08:08.463 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:13.452195+0000", 2020-08-07T15:08:08.463 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 54.893482683000002, 2020-08-07T15:08:08.463 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.187979807, 2020-08-07T15:08:08.463 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:08.464 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:08.464 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:08.464 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.464 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:08.464 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.452195+0000", 2020-08-07T15:08:08.465 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:08.465 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.465 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.465 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:08.465 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.452195+0000", 2020-08-07T15:08:08.466 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9023000000000001e-05 2020-08-07T15:08:08.466 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.466 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.466 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:08.466 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.452244+0000", 2020-08-07T15:08:08.467 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.733e-06 2020-08-07T15:08:08.467 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.467 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.467 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:08.467 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.452250+0000", 2020-08-07T15:08:08.467 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18453694300000001 2020-08-07T15:08:08.468 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.468 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.468 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:08.468 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.636787+0000", 2020-08-07T15:08:08.468 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5679999999999999e-06 2020-08-07T15:08:08.469 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.469 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.469 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:08.469 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.636792+0000", 2020-08-07T15:08:08.470 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0119999999999999e-05 2020-08-07T15:08:08.470 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.470 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.470 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:08.470 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.636872+0000", 2020-08-07T15:08:08.471 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6356999999999999e-05 2020-08-07T15:08:08.471 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.471 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.471 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:08.471 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.636899+0000", 2020-08-07T15:08:08.472 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003233737 2020-08-07T15:08:08.472 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.472 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.472 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:08.472 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.640132+0000", 2020-08-07T15:08:08.472 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2326000000000001e-05 2020-08-07T15:08:08.473 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.473 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.474 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:08.474 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.640175+0000", 2020-08-07T15:08:08.474 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.187979807 2020-08-07T15:08:08.474 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.474 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:08.474 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.475 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.475 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.475 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:71136 3.6 e43/26 3:64cb7acb:::benchmark_data_smithi074_35447_object8891:head v 43'4744, mlcod=43'4739)", 2020-08-07T15:08:08.475 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:08:08.475 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 44.027064097999997, 2020-08-07T15:08:08.476 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26135482999999998, 2020-08-07T15:08:08.476 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:08.476 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:08.476 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:08.476 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.477 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:08.477 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:08:08.477 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:08.477 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.477 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.478 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:08.478 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:08:08.478 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3702999999999997e-05 2020-08-07T15:08:08.479 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.479 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.479 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:08.479 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318667+0000", 2020-08-07T15:08:08.479 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3119999999999999e-06 2020-08-07T15:08:08.479 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.480 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:08.480 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318674+0000", 2020-08-07T15:08:08.480 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.255173292 2020-08-07T15:08:08.480 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.481 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.481 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:08.481 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.573847+0000", 2020-08-07T15:08:08.482 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1140000000000002e-06 2020-08-07T15:08:08.482 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.482 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.482 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:08.482 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.573853+0000", 2020-08-07T15:08:08.483 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00057603799999999996 2020-08-07T15:08:08.483 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.483 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.483 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:08.483 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.574429+0000", 2020-08-07T15:08:08.484 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4776e-05 2020-08-07T15:08:08.484 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.484 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.484 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:08.484 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.574444+0000", 2020-08-07T15:08:08.485 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054577080000000004 2020-08-07T15:08:08.485 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.485 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.485 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:08.485 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.579901+0000", 2020-08-07T15:08:08.486 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6886999999999998e-05 2020-08-07T15:08:08.486 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.486 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.486 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:08.488 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.579968+0000", 2020-08-07T15:08:08.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26135482999999998 2020-08-07T15:08:08.488 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.488 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:08.488 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.489 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.489 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:81544 3.7 e48/20 3:e9bb655d:::benchmark_data_smithi074_35447_object10192:head v 48'4944, mlcod=48'4935)", 2020-08-07T15:08:08.489 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:30.902099+0000", 2020-08-07T15:08:08.489 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 37.443579139000001, 2020-08-07T15:08:08.490 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20669163500000001, 2020-08-07T15:08:08.490 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:08.490 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:08.490 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:08.490 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.491 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:08.491 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.902099+0000", 2020-08-07T15:08:08.491 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:08.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.491 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.492 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:08.492 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.902099+0000", 2020-08-07T15:08:08.492 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1317999999999999e-05 2020-08-07T15:08:08.492 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.492 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:08.493 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.902150+0000", 2020-08-07T15:08:08.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2079999999999997e-06 2020-08-07T15:08:08.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.493 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:08.494 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.902156+0000", 2020-08-07T15:08:08.494 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20264375100000001 2020-08-07T15:08:08.494 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.494 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:08.495 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.104800+0000", 2020-08-07T15:08:08.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2390000000000002e-06 2020-08-07T15:08:08.495 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.495 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.495 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:08.496 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.104805+0000", 2020-08-07T15:08:08.496 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3696000000000003e-05 2020-08-07T15:08:08.496 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.496 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.496 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:08.497 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.104849+0000", 2020-08-07T15:08:08.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4179e-05 2020-08-07T15:08:08.497 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.497 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.497 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:08.498 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.104873+0000", 2020-08-07T15:08:08.498 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038653839999999999 2020-08-07T15:08:08.498 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.498 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.498 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:08.498 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.108738+0000", 2020-08-07T15:08:08.499 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1860000000000002e-05 2020-08-07T15:08:08.499 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.499 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.500 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:08.500 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.108790+0000", 2020-08-07T15:08:08.501 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20669163500000001 2020-08-07T15:08:08.501 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.501 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:08.501 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.501 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.502 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.502 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:85696 3.f e51/20 3:f99fcea4:::benchmark_data_smithi074_35447_object10711:head v 51'5688, mlcod=51'5679)", 2020-08-07T15:08:08.502 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:08:08.502 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 34.693072684999997, 2020-08-07T15:08:08.502 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49655717300000002, 2020-08-07T15:08:08.503 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:08.503 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:08.503 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:08.503 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.503 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:08.504 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:08:08.504 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:08.504 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.504 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.504 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:08.505 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:08:08.505 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9543000000000002e-05 2020-08-07T15:08:08.505 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.505 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.505 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:08.506 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652665+0000", 2020-08-07T15:08:08.506 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8700000000000003e-06 2020-08-07T15:08:08.506 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.506 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.506 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:08.507 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652671+0000", 2020-08-07T15:08:08.507 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49579595799999998 2020-08-07T15:08:08.507 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.507 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.507 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:08.508 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148467+0000", 2020-08-07T15:08:08.508 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2509999999999997e-06 2020-08-07T15:08:08.508 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.508 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.509 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:08.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148475+0000", 2020-08-07T15:08:08.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5603999999999998e-05 2020-08-07T15:08:08.509 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.509 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.510 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:08.510 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148530+0000", 2020-08-07T15:08:08.510 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5486999999999998e-05 2020-08-07T15:08:08.510 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:08.511 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148556+0000", 2020-08-07T15:08:08.511 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00055666600000000004 2020-08-07T15:08:08.511 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.512 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:08.512 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.149112+0000", 2020-08-07T15:08:08.512 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9793999999999998e-05 2020-08-07T15:08:08.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.512 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.513 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:08.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.149162+0000", 2020-08-07T15:08:08.513 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49655717300000002 2020-08-07T15:08:08.513 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.513 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:08.513 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.514 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.514 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:86792 3.f e52/20 3:fc5386ee:::benchmark_data_smithi074_35447_object10848:head v 52'5752, mlcod=52'5743)", 2020-08-07T15:08:08.514 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:08:08.514 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 33.94322691, 2020-08-07T15:08:08.515 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.295270005, 2020-08-07T15:08:08.515 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:08.515 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:08.515 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:08.515 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.516 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:08.516 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:08:08.516 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:08.516 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.516 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.517 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:08.517 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:08:08.517 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9712000000000001e-05 2020-08-07T15:08:08.517 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.517 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.518 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:08.518 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402500+0000", 2020-08-07T15:08:08.518 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6789999999999999e-06 2020-08-07T15:08:08.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.519 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:08.519 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402507+0000", 2020-08-07T15:08:08.519 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29317074999999998 2020-08-07T15:08:08.519 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.519 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:08.520 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695678+0000", 2020-08-07T15:08:08.520 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8120000000000002e-06 2020-08-07T15:08:08.520 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.520 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.521 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:08.521 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695685+0000", 2020-08-07T15:08:08.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1270999999999998e-05 2020-08-07T15:08:08.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.521 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.522 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:08.522 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695726+0000", 2020-08-07T15:08:08.522 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2739e-05 2020-08-07T15:08:08.522 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.523 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.523 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:08.523 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695749+0000", 2020-08-07T15:08:08.523 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019333899999999999 2020-08-07T15:08:08.523 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.524 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.524 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:08.524 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.697682+0000", 2020-08-07T15:08:08.524 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8652000000000002e-05 2020-08-07T15:08:08.524 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.525 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.525 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:08.525 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.697721+0000", 2020-08-07T15:08:08.525 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.295270005 2020-08-07T15:08:08.525 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.526 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:08.526 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.526 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.526 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.526 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:87088 3.6 e52/26 3:6b0320df:::benchmark_data_smithi074_35447_object10885:head v 52'5912, mlcod=52'5902)", 2020-08-07T15:08:08.526 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:08:08.527 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 33.37899367, 2020-08-07T15:08:08.527 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29522189799999998, 2020-08-07T15:08:08.527 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:08.527 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:08.527 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:08.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.528 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:08.528 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:08:08.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:08.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.529 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.529 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:08.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:08:08.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0751000000000002e-05 2020-08-07T15:08:08.529 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.530 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.530 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:08.530 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966735+0000", 2020-08-07T15:08:08.530 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3080000000000004e-06 2020-08-07T15:08:08.531 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.531 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.531 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:08.531 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966741+0000", 2020-08-07T15:08:08.531 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29365034699999998 2020-08-07T15:08:08.532 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.532 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.532 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:08.532 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260391+0000", 2020-08-07T15:08:08.532 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2720000000000003e-06 2020-08-07T15:08:08.533 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.533 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.533 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:08.533 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260398+0000", 2020-08-07T15:08:08.533 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8819000000000005e-05 2020-08-07T15:08:08.533 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.534 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.534 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:08.534 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260476+0000", 2020-08-07T15:08:08.534 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4958999999999999e-05 2020-08-07T15:08:08.534 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:08.535 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260501+0000", 2020-08-07T15:08:08.535 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001353641 2020-08-07T15:08:08.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.536 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:08.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.261855+0000", 2020-08-07T15:08:08.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0800999999999997e-05 2020-08-07T15:08:08.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.537 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.537 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:08.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.261906+0000", 2020-08-07T15:08:08.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29522189799999998 2020-08-07T15:08:08.537 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.538 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:08.538 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.538 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.538 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.538 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:93920 3.6 e56/26 3:631393d3:::benchmark_data_smithi074_35447_object11739:head v 56'6352, mlcod=55'6342)", 2020-08-07T15:08:08.539 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:08:08.539 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 29.018878238999999, 2020-08-07T15:08:08.539 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39765404300000001, 2020-08-07T15:08:08.539 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:08.539 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:08.540 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:08.540 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.540 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:08.540 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:08:08.540 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:08.540 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.541 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:08.541 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:08:08.541 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3035999999999998e-05 2020-08-07T15:08:08.542 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.542 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.542 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:08.542 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326852+0000", 2020-08-07T15:08:08.542 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5379999999999999e-06 2020-08-07T15:08:08.543 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.543 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.543 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:08.543 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326859+0000", 2020-08-07T15:08:08.543 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.393917568 2020-08-07T15:08:08.544 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.544 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.544 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:08.544 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.720777+0000", 2020-08-07T15:08:08.544 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2430000000000005e-06 2020-08-07T15:08:08.545 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.545 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:08.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.720785+0000", 2020-08-07T15:08:08.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00065658700000000003 2020-08-07T15:08:08.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.546 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.546 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:08.546 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.721441+0000", 2020-08-07T15:08:08.546 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8585000000000002e-05 2020-08-07T15:08:08.547 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.547 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:08.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.721470+0000", 2020-08-07T15:08:08.547 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00293349 2020-08-07T15:08:08.547 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.548 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:08.548 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.724403+0000", 2020-08-07T15:08:08.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9996e-05 2020-08-07T15:08:08.548 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.549 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.549 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:08.549 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.724453+0000", 2020-08-07T15:08:08.549 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39765404300000001 2020-08-07T15:08:08.549 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.550 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:08.550 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.550 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.550 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.550 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:94384 3.3 e56/20 3:ce533213:::benchmark_data_smithi074_35447_object11797:head v 56'5488, mlcod=56'5479)", 2020-08-07T15:08:08.551 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.637245+0000", 2020-08-07T15:08:08.551 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 28.708432734999999, 2020-08-07T15:08:08.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.247824459, 2020-08-07T15:08:08.551 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:08.551 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:08.551 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:08.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.552 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:08.552 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637245+0000", 2020-08-07T15:08:08.552 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:08.552 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.553 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.553 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:08.553 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637245+0000", 2020-08-07T15:08:08.553 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0782999999999997e-05 2020-08-07T15:08:08.553 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.554 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.560 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:08.561 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637306+0000", 2020-08-07T15:08:08.561 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8699999999999992e-06 2020-08-07T15:08:08.561 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.561 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.561 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:08.562 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637314+0000", 2020-08-07T15:08:08.562 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24469571800000001 2020-08-07T15:08:08.562 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.562 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.562 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:08.563 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882009+0000", 2020-08-07T15:08:08.563 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8510000000000004e-06 2020-08-07T15:08:08.563 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.563 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.563 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:08.564 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882015+0000", 2020-08-07T15:08:08.564 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6708999999999997e-05 2020-08-07T15:08:08.564 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.564 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:08.565 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882052+0000", 2020-08-07T15:08:08.565 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0280000000000001e-05 2020-08-07T15:08:08.565 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.565 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.565 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:08.565 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882082+0000", 2020-08-07T15:08:08.566 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002934767 2020-08-07T15:08:08.566 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.566 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.566 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:08.566 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.885017+0000", 2020-08-07T15:08:08.567 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2481000000000001e-05 2020-08-07T15:08:08.567 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.567 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.567 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:08.567 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.885069+0000", 2020-08-07T15:08:08.568 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.247824459 2020-08-07T15:08:08.568 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.568 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:08.568 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.568 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.569 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.569 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:96832 3.6 e57/26 3:6d818ad0:::benchmark_data_smithi074_35447_object12103:head v 57'6536, mlcod=56'6526)", 2020-08-07T15:08:08.569 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:41.323206+0000", 2020-08-07T15:08:08.569 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 27.022471840000001, 2020-08-07T15:08:08.569 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.188700062, 2020-08-07T15:08:08.570 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:08.570 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:08.570 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:08.570 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.570 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:08.571 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.323206+0000", 2020-08-07T15:08:08.571 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:08.571 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.571 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.571 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:08.572 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.323206+0000", 2020-08-07T15:08:08.572 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6401999999999999e-05 2020-08-07T15:08:08.572 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.572 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.572 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:08.572 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.323232+0000", 2020-08-07T15:08:08.573 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3380000000000001e-06 2020-08-07T15:08:08.573 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.573 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.573 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:08.573 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.323236+0000", 2020-08-07T15:08:08.574 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18768147800000001 2020-08-07T15:08:08.574 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.574 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.574 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:08.574 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.510917+0000", 2020-08-07T15:08:08.575 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.536e-06 2020-08-07T15:08:08.575 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.575 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:08.575 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.510923+0000", 2020-08-07T15:08:08.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3084999999999998e-05 2020-08-07T15:08:08.576 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.576 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.576 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:08.576 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.510966+0000", 2020-08-07T15:08:08.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8317e-05 2020-08-07T15:08:08.577 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.577 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.577 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:08.577 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.510994+0000", 2020-08-07T15:08:08.577 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00087632200000000002 2020-08-07T15:08:08.578 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.578 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.578 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:08.578 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.511870+0000", 2020-08-07T15:08:08.578 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5583999999999997e-05 2020-08-07T15:08:08.579 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.579 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.579 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:08.579 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:41.511906+0000", 2020-08-07T15:08:08.579 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.188700062 2020-08-07T15:08:08.579 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.580 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:08.580 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.580 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.580 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.580 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:104560 3.f e58/20 3:fb4383ae:::benchmark_data_smithi074_35447_object13069:head v 58'6888, mlcod=58'6878)", 2020-08-07T15:08:08.581 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:46.301207+0000", 2020-08-07T15:08:08.581 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 22.044470917999998, 2020-08-07T15:08:08.581 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23687350400000001, 2020-08-07T15:08:08.581 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:08.581 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:08.582 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:08.582 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.582 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:08.582 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301207+0000", 2020-08-07T15:08:08.582 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:08.583 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.583 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.583 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:08.583 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301207+0000", 2020-08-07T15:08:08.583 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7945000000000003e-05 2020-08-07T15:08:08.583 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.584 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.584 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:08.584 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301255+0000", 2020-08-07T15:08:08.584 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2030000000000002e-06 2020-08-07T15:08:08.584 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.585 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.585 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:08.585 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301259+0000", 2020-08-07T15:08:08.585 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23565420500000001 2020-08-07T15:08:08.585 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.586 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.586 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:08.586 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.536913+0000", 2020-08-07T15:08:08.586 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2290000000000003e-06 2020-08-07T15:08:08.586 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.587 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.587 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:08.587 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.536921+0000", 2020-08-07T15:08:08.587 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0955e-05 2020-08-07T15:08:08.588 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.588 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:08.588 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.537002+0000", 2020-08-07T15:08:08.588 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8977999999999999e-05 2020-08-07T15:08:08.589 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.589 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:08.589 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.537031+0000", 2020-08-07T15:08:08.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00099831000000000008 2020-08-07T15:08:08.590 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.590 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.590 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:08.590 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.538030+0000", 2020-08-07T15:08:08.590 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0679e-05 2020-08-07T15:08:08.590 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.591 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.591 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:08.591 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.538080+0000", 2020-08-07T15:08:08.591 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23687350400000001 2020-08-07T15:08:08.591 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.592 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:08.592 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.592 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.592 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.592 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:116064 3.6 e64/26 3:6cc03ffa:::benchmark_data_smithi074_35447_object14507:head v 64'7632, mlcod=64'7623)", 2020-08-07T15:08:08.593 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:08:08.593 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.982916835999999, 2020-08-07T15:08:08.593 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25728607799999997, 2020-08-07T15:08:08.593 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:08.593 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:08.594 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:08.594 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.594 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:08.594 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:08:08.594 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:08.595 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.595 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.595 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:08.595 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:08:08.595 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6042000000000003e-05 2020-08-07T15:08:08.595 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.596 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.596 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:08.596 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362817+0000", 2020-08-07T15:08:08.597 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9399999999999999e-06 2020-08-07T15:08:08.597 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.597 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.597 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:08.597 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362823+0000", 2020-08-07T15:08:08.598 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.254635583 2020-08-07T15:08:08.598 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.598 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.598 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:08.598 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617458+0000", 2020-08-07T15:08:08.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5100000000000004e-06 2020-08-07T15:08:08.599 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.599 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.599 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:08.599 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617465+0000", 2020-08-07T15:08:08.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4439999999999999e-05 2020-08-07T15:08:08.600 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.600 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.600 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:08.600 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617539+0000", 2020-08-07T15:08:08.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3527e-05 2020-08-07T15:08:08.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.602 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:08.602 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617563+0000", 2020-08-07T15:08:08.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024301069999999999 2020-08-07T15:08:08.603 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.603 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:08.603 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.619993+0000", 2020-08-07T15:08:08.603 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3928999999999998e-05 2020-08-07T15:08:08.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:08.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.620047+0000", 2020-08-07T15:08:08.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25728607799999997 2020-08-07T15:08:08.605 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.605 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:08.605 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.605 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.605 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.606 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:123096 3.f e68/20 3:fe990d4f:::benchmark_data_smithi074_35447_object15386:head v 68'8120, mlcod=68'8112)", 2020-08-07T15:08:08.606 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:01.823387+0000", 2020-08-07T15:08:08.606 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.5222904479999997, 2020-08-07T15:08:08.606 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.196095347, 2020-08-07T15:08:08.606 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:08.607 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:08.607 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:08.607 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.607 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:08.607 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:01.823387+0000", 2020-08-07T15:08:08.607 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:08.608 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.608 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.608 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:08.608 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:01.823387+0000", 2020-08-07T15:08:08.608 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1274999999999999e-05 2020-08-07T15:08:08.609 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.609 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.609 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:08.609 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:01.823448+0000", 2020-08-07T15:08:08.609 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5830000000000003e-06 2020-08-07T15:08:08.610 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.610 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.610 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:08.610 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:01.823455+0000", 2020-08-07T15:08:08.611 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19422482299999999 2020-08-07T15:08:08.611 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.611 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.611 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:08.612 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.017680+0000", 2020-08-07T15:08:08.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8530000000000003e-06 2020-08-07T15:08:08.612 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.612 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.612 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:08.613 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.017687+0000", 2020-08-07T15:08:08.613 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6105e-05 2020-08-07T15:08:08.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.613 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:08.614 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.017733+0000", 2020-08-07T15:08:08.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2348e-05 2020-08-07T15:08:08.614 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.614 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.614 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:08.615 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.017755+0000", 2020-08-07T15:08:08.615 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0016718480000000001 2020-08-07T15:08:08.615 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.615 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.615 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:08.616 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.019427+0000", 2020-08-07T15:08:08.616 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5512e-05 2020-08-07T15:08:08.616 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.616 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.616 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:08.617 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.019483+0000", 2020-08-07T15:08:08.617 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.196095347 2020-08-07T15:08:08.617 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.617 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:08.617 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.618 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.618 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.618 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:127528 3.6 e70/26 3:6d26a4f4:::benchmark_data_smithi074_35447_object15940:head v 70'8264, mlcod=70'8256)", 2020-08-07T15:08:08.618 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:04.535579+0000", 2020-08-07T15:08:08.618 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 3.8100987869999998, 2020-08-07T15:08:08.619 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22956602400000001, 2020-08-07T15:08:08.619 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:08.619 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:08.619 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:08.619 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.619 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:08.620 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.535579+0000", 2020-08-07T15:08:08.620 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:08.620 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.621 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.622 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:08.622 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.535579+0000", 2020-08-07T15:08:08.622 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0343999999999997e-05 2020-08-07T15:08:08.622 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.631 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.631 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:08.631 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.535639+0000", 2020-08-07T15:08:08.631 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5889999999999999e-06 2020-08-07T15:08:08.632 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.632 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.632 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:08.632 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.535646+0000", 2020-08-07T15:08:08.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22504058599999999 2020-08-07T15:08:08.633 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.633 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.633 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:08.633 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.760686+0000", 2020-08-07T15:08:08.633 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7790000000000003e-06 2020-08-07T15:08:08.634 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.634 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.634 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:08.634 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.760691+0000", 2020-08-07T15:08:08.646 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2592999999999997e-05 2020-08-07T15:08:08.647 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.647 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.648 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:08.649 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.760734+0000", 2020-08-07T15:08:08.649 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3947999999999998e-05 2020-08-07T15:08:08.649 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.649 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.650 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:08.650 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.760758+0000", 2020-08-07T15:08:08.650 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043316020000000004 2020-08-07T15:08:08.650 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.650 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.651 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:08.651 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.765089+0000", 2020-08-07T15:08:08.651 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5583e-05 2020-08-07T15:08:08.651 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.651 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.652 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:08.652 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.765145+0000", 2020-08-07T15:08:08.653 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22956602400000001 2020-08-07T15:08:08.655 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.655 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:08.656 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.656 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.656 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.656 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:127560 3.3 e70/20 3:cd6ba668:::benchmark_data_smithi074_35447_object15944:head v 70'7720, mlcod=70'7711)", 2020-08-07T15:08:08.656 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:04.553268+0000", 2020-08-07T15:08:08.657 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 3.792409889, 2020-08-07T15:08:08.657 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.193559553, 2020-08-07T15:08:08.657 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:08.657 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:08.657 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:08.658 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.658 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:08.658 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.553268+0000", 2020-08-07T15:08:08.658 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:08.658 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.659 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.659 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:08.659 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.553268+0000", 2020-08-07T15:08:08.659 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7871000000000001e-05 2020-08-07T15:08:08.659 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.660 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.660 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:08.660 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.553296+0000", 2020-08-07T15:08:08.660 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2980000000000002e-06 2020-08-07T15:08:08.660 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.661 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.661 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:08.661 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.553300+0000", 2020-08-07T15:08:08.661 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.191834846 2020-08-07T15:08:08.665 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.666 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.666 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:08.666 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.745135+0000", 2020-08-07T15:08:08.666 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7130000000000001e-06 2020-08-07T15:08:08.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.667 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:08.668 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.745137+0000", 2020-08-07T15:08:08.668 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4369000000000002e-05 2020-08-07T15:08:08.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:08.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.745182+0000", 2020-08-07T15:08:08.670 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4066e-05 2020-08-07T15:08:08.670 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.670 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.670 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:08.671 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.745206+0000", 2020-08-07T15:08:08.671 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015765480000000001 2020-08-07T15:08:08.672 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.672 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.672 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:08.673 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.746782+0000", 2020-08-07T15:08:08.673 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4842000000000002e-05 2020-08-07T15:08:08.674 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:08.678 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.746827+0000", 2020-08-07T15:08:08.678 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.193559553 2020-08-07T15:08:08.678 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.679 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:08.679 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.679 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.680 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:08.680 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:08.681 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:08:08.683 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:08:08.684 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:08.680+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:08.787 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:08.785+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:08.893 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:08.887+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:08.955 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:08.956 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:08:08.956 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.957 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:133578 3.6 3:6672be19:::benchmark_data_smithi074_35447_object3969:head [delete] snapc 0=[] ondisk+write+known_if_redirected e72)", 2020-08-07T15:08:08.957 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:08.949789+0000", 2020-08-07T15:08:08.957 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0046935420000000002, 2020-08-07T15:08:08.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0047120529999999999, 2020-08-07T15:08:08.958 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:08.958 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:08:08.958 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:08.958 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:08.958 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:08.959 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 133578 2020-08-07T15:08:08.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.959 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:08.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:08.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.949789+0000", 2020-08-07T15:08:08.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:08.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:08.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.949789+0000", 2020-08-07T15:08:08.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1859000000000001e-05 2020-08-07T15:08:08.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:08.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.949821+0000", 2020-08-07T15:08:08.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1619999999999996e-06 2020-08-07T15:08:08.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:08.964 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.949827+0000", 2020-08-07T15:08:08.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5517999999999999e-05 2020-08-07T15:08:08.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:08.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.949842+0000", 2020-08-07T15:08:08.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6260000000000003e-06 2020-08-07T15:08:08.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:08.981 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.949850+0000", 2020-08-07T15:08:08.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9536000000000001e-05 2020-08-07T15:08:08.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.981 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:08.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.949909+0000", 2020-08-07T15:08:08.982 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016577599999999999 2020-08-07T15:08:08.982 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.983 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:08.983 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.950075+0000", 2020-08-07T15:08:08.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00026122899999999998 2020-08-07T15:08:08.983 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.984 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:08.984 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.950336+0000", 2020-08-07T15:08:08.985 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013890650000000001 2020-08-07T15:08:08.985 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.985 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:08.987 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.951725+0000", 2020-08-07T15:08:08.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000552174 2020-08-07T15:08:08.987 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:08.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.952278+0000", 2020-08-07T15:08:08.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024889449999999998 2020-08-07T15:08:08.988 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.988 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:08.989 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:08.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.989 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:133563 3.8 3:1ed5cfde:::benchmark_data_smithi074_35447_object3954:head [delete] snapc 0=[] ondisk+write+known_if_redirected e72)", 2020-08-07T15:08:08.989 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:08.938160+0000", 2020-08-07T15:08:08.990 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.016321722, 2020-08-07T15:08:08.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.016398039999999999, 2020-08-07T15:08:08.990 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:08.990 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:08.991 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:08.991 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:08.991 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:08.991 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 133563 2020-08-07T15:08:08.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.992 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:08.993 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.993 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:08.994 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.938160+0000", 2020-08-07T15:08:08.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:08.995 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.995 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.995 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:08.995 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.938160+0000", 2020-08-07T15:08:08.995 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1652000000000001e-05 2020-08-07T15:08:08.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.996 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.996 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:08.997 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.938192+0000", 2020-08-07T15:08:08.997 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8980000000000001e-06 2020-08-07T15:08:08.997 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.997 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.997 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:08.998 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.938198+0000", 2020-08-07T15:08:08.998 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2557e-05 2020-08-07T15:08:08.998 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.998 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:08.998 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:08.999 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.938211+0000", 2020-08-07T15:08:08.999 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0920000000000003e-06 2020-08-07T15:08:08.999 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:08.999 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.000 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:09.000 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.938217+0000", 2020-08-07T15:08:09.000 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1075999999999996e-05 2020-08-07T15:08:09.000 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.000 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.001 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:09.001 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.938278+0000", 2020-08-07T15:08:09.001 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018227 2020-08-07T15:08:09.001 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.002 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.002 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:09.002 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.938460+0000", 2020-08-07T15:08:09.002 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00022268200000000001 2020-08-07T15:08:09.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4,7", 2020-08-07T15:08:09.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.938683+0000", 2020-08-07T15:08:09.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0029842530000000001 2020-08-07T15:08:09.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:09.004 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.941667+0000", 2020-08-07T15:08:09.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023615609999999999 2020-08-07T15:08:09.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:09.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.944029+0000", 2020-08-07T15:08:09.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.218e-06 2020-08-07T15:08:09.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:09.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.944030+0000", 2020-08-07T15:08:09.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9496e-05 2020-08-07T15:08:09.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:09.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.944099+0000", 2020-08-07T15:08:09.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00054566100000000002 2020-08-07T15:08:09.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:09.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.944645+0000", 2020-08-07T15:08:09.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0064844159999999998 2020-08-07T15:08:09.009 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.009 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:09.009 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.009 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.010 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:133569 3.8 3:10ea78b2:::benchmark_data_smithi074_35447_object3960:head [delete] snapc 0=[] ondisk+write+known_if_redirected e72)", 2020-08-07T15:08:09.010 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:08.942030+0000", 2020-08-07T15:08:09.010 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.012452621000000001, 2020-08-07T15:08:09.010 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.012590229, 2020-08-07T15:08:09.012 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:09.012 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:08:09.013 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:09.013 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:09.013 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:09.013 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 133569 2020-08-07T15:08:09.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.014 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:09.014 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:09.014 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.942030+0000", 2020-08-07T15:08:09.015 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:09.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.015 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.015 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:09.015 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.942030+0000", 2020-08-07T15:08:09.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4678999999999998e-05 2020-08-07T15:08:09.017 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:09.018 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.942064+0000", 2020-08-07T15:08:09.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2310000000000001e-06 2020-08-07T15:08:09.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.019 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.019 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:09.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.942070+0000", 2020-08-07T15:08:09.019 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2356e-05 2020-08-07T15:08:09.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.020 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.020 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:09.020 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.942083+0000", 2020-08-07T15:08:09.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6230000000000002e-06 2020-08-07T15:08:09.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:09.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.942088+0000", 2020-08-07T15:08:09.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7123999999999997e-05 2020-08-07T15:08:09.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:09.022 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.942146+0000", 2020-08-07T15:08:09.022 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000181008 2020-08-07T15:08:09.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.023 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:09.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.942327+0000", 2020-08-07T15:08:09.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016651099999999999 2020-08-07T15:08:09.023 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4,7", 2020-08-07T15:08:09.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.942493+0000", 2020-08-07T15:08:09.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024838299999999998 2020-08-07T15:08:09.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:09.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.944977+0000", 2020-08-07T15:08:09.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017104220000000001 2020-08-07T15:08:09.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:09.026 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.946687+0000", 2020-08-07T15:08:09.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00099389599999999997 2020-08-07T15:08:09.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:09.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.947681+0000", 2020-08-07T15:08:09.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0056516800000000001 2020-08-07T15:08:09.027 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.027 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:09.028 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.028 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:133572 3.8 3:185c7fef:::benchmark_data_smithi074_35447_object3963:head [delete] snapc 0=[] ondisk+write+known_if_redirected e72)", 2020-08-07T15:08:09.028 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:08.942552+0000", 2020-08-07T15:08:09.029 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.011929762, 2020-08-07T15:08:09.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.012127529, 2020-08-07T15:08:09.029 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:09.029 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:09.029 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:09.030 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:09.030 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:09.030 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 133572 2020-08-07T15:08:09.030 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.030 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:09.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:09.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.942552+0000", 2020-08-07T15:08:09.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:09.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:09.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.942552+0000", 2020-08-07T15:08:09.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9059000000000001e-05 2020-08-07T15:08:09.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:09.033 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.942581+0000", 2020-08-07T15:08:09.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6559999999999997e-06 2020-08-07T15:08:09.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.034 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:09.034 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.942587+0000", 2020-08-07T15:08:09.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2584e-05 2020-08-07T15:08:09.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.034 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:09.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.942600+0000", 2020-08-07T15:08:09.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3210000000000004e-06 2020-08-07T15:08:09.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:09.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.942605+0000", 2020-08-07T15:08:09.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00026508499999999997 2020-08-07T15:08:09.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.037 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:09.037 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.942870+0000", 2020-08-07T15:08:09.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000164974 2020-08-07T15:08:09.038 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.039 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.039 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:09.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.943035+0000", 2020-08-07T15:08:09.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016799 2020-08-07T15:08:09.039 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4,7", 2020-08-07T15:08:09.040 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.943203+0000", 2020-08-07T15:08:09.040 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0035391139999999999 2020-08-07T15:08:09.045 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.045 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:09.045 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.946742+0000", 2020-08-07T15:08:09.046 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00094843300000000005 2020-08-07T15:08:09.046 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.046 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.046 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:09.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.947691+0000", 2020-08-07T15:08:09.046 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3900000000000004e-07 2020-08-07T15:08:09.047 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.047 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:09.047 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.947691+0000", 2020-08-07T15:08:09.047 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4884999999999999e-05 2020-08-07T15:08:09.048 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.048 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.048 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:09.048 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.947766+0000", 2020-08-07T15:08:09.048 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00521384 2020-08-07T15:08:09.049 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.049 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:09.049 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.049 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.049 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.050 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:133522 3.8 3:1c7a201f:::benchmark_data_smithi074_35447_object3913:head [delete] snapc 0=[] ondisk+write+known_if_redirected e72)", 2020-08-07T15:08:09.050 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:08.917242+0000", 2020-08-07T15:08:09.050 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.037240685000000003, 2020-08-07T15:08:09.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.037494451999999998, 2020-08-07T15:08:09.051 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:09.051 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:09.051 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:09.051 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:09.051 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:09.052 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 133522 2020-08-07T15:08:09.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.052 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:09.052 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.052 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:09.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.917242+0000", 2020-08-07T15:08:09.053 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:09.053 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.053 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:09.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.917242+0000", 2020-08-07T15:08:09.054 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1387999999999998e-05 2020-08-07T15:08:09.054 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.054 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.054 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:09.054 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.917273+0000", 2020-08-07T15:08:09.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3400000000000003e-06 2020-08-07T15:08:09.055 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.055 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:09.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.917279+0000", 2020-08-07T15:08:09.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1963999999999999e-05 2020-08-07T15:08:09.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:09.057 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.917291+0000", 2020-08-07T15:08:09.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4869999999999999e-06 2020-08-07T15:08:09.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:09.058 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.917298+0000", 2020-08-07T15:08:09.058 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00023420900000000001 2020-08-07T15:08:09.058 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.058 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.058 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:09.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.917532+0000", 2020-08-07T15:08:09.059 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000186121 2020-08-07T15:08:09.059 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.059 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:09.060 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.917718+0000", 2020-08-07T15:08:09.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000181681 2020-08-07T15:08:09.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.061 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4,7", 2020-08-07T15:08:09.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.917900+0000", 2020-08-07T15:08:09.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019976529999999998 2020-08-07T15:08:09.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.062 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:09.062 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.919897+0000", 2020-08-07T15:08:09.062 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2160000000000006e-06 2020-08-07T15:08:09.062 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.062 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.063 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:09.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.919906+0000", 2020-08-07T15:08:09.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9200000000000001e-07 2020-08-07T15:08:09.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.063 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:09.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.919906+0000", 2020-08-07T15:08:09.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6588999999999994e-05 2020-08-07T15:08:09.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.064 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:09.065 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.919993+0000", 2020-08-07T15:08:09.065 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002018621 2020-08-07T15:08:09.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.065 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:09.066 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:08.922011+0000", 2020-08-07T15:08:09.066 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0047698610000000002 2020-08-07T15:08:09.066 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.066 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:09.066 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.067 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.067 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:08:09.067 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 5 2020-08-07T15:08:09.067 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:09.069 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:08:09.069 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:09.067+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:09.187 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:09.169+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:09.273 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:09.272+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:09.328 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:09.328 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:08:09.329 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:08:09.329 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:08:09.329 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:09.340 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:08:09.377 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:09.373+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:09.477 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:08:09.491 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:09.475+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:09.591 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:09.578+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:09.593 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:09.593 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:08:09.594 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:08:09.594 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:08:09.594 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.594 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:15576 3.e e27/20 3:727724a5:::benchmark_data_smithi074_35447_object1946:head v 27'1032, mlcod=27'1023)", 2020-08-07T15:08:09.595 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:08:09.595 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 82.919515387000004, 2020-08-07T15:08:09.595 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.46091973000000003, 2020-08-07T15:08:09.595 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:09.595 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:09.596 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:09.596 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.596 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:09.596 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:08:09.596 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:09.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.601 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:09.602 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:08:09.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9592000000000001e-05 2020-08-07T15:08:09.602 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.602 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.602 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:09.603 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669542+0000", 2020-08-07T15:08:09.603 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.113e-06 2020-08-07T15:08:09.603 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:09.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669547+0000", 2020-08-07T15:08:09.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45151617799999999 2020-08-07T15:08:09.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.630 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:09.630 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121064+0000", 2020-08-07T15:08:09.630 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.377e-06 2020-08-07T15:08:09.631 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.631 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.631 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:09.631 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121067+0000", 2020-08-07T15:08:09.631 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9195e-05 2020-08-07T15:08:09.632 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.632 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.632 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:09.632 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121106+0000", 2020-08-07T15:08:09.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7509000000000001e-05 2020-08-07T15:08:09.633 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.633 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.633 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:09.633 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121124+0000", 2020-08-07T15:08:09.633 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0091783100000000003 2020-08-07T15:08:09.634 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.634 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.634 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:09.634 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.130302+0000", 2020-08-07T15:08:09.634 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000100456 2020-08-07T15:08:09.635 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.636 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.636 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:09.636 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.130402+0000", 2020-08-07T15:08:09.636 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.46091973000000003 2020-08-07T15:08:09.636 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.637 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:09.637 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.637 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.637 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.637 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:24264 3.8 e28/20 3:1d3fdbda:::benchmark_data_smithi074_35447_object3032:head v 28'1520, mlcod=28'1515)", 2020-08-07T15:08:09.638 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:08:09.638 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 77.205885350000003, 2020-08-07T15:08:09.638 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41856254199999998, 2020-08-07T15:08:09.638 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:09.638 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:09.639 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:09.639 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.639 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:09.639 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:08:09.639 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:09.640 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.642 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.642 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:09.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:08:09.643 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1348999999999998e-05 2020-08-07T15:08:09.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.643 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.643 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:09.644 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383174+0000", 2020-08-07T15:08:09.644 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6579999999999996e-06 2020-08-07T15:08:09.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.644 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.644 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:09.644 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383181+0000", 2020-08-07T15:08:09.645 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41194352699999998 2020-08-07T15:08:09.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:09.646 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.795124+0000", 2020-08-07T15:08:09.647 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5729999999999999e-06 2020-08-07T15:08:09.647 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.647 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.647 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:09.648 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.795128+0000", 2020-08-07T15:08:09.648 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002170384 2020-08-07T15:08:09.648 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.649 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.649 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:09.649 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.797298+0000", 2020-08-07T15:08:09.649 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5412e-05 2020-08-07T15:08:09.650 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.650 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.650 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:09.650 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.797324+0000", 2020-08-07T15:08:09.651 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042943360000000002 2020-08-07T15:08:09.651 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.651 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.651 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:09.652 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.801618+0000", 2020-08-07T15:08:09.652 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7302999999999997e-05 2020-08-07T15:08:09.652 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.653 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.653 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:09.653 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.801675+0000", 2020-08-07T15:08:09.653 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41856254199999998 2020-08-07T15:08:09.654 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.654 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:09.654 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.654 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.655 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.655 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:44120 3.e e32/20 3:7e6f7bb9:::benchmark_data_smithi074_35447_object5514:head v 32'2752, mlcod=32'2742)", 2020-08-07T15:08:09.655 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:04.453943+0000", 2020-08-07T15:08:09.655 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 65.135055124000004, 2020-08-07T15:08:09.656 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26204982500000001, 2020-08-07T15:08:09.656 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:09.656 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:09.656 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:09.657 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.657 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:09.657 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.453943+0000", 2020-08-07T15:08:09.657 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:09.657 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.658 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.658 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:09.658 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.453943+0000", 2020-08-07T15:08:09.658 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2278000000000003e-05 2020-08-07T15:08:09.659 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.659 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.660 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:09.660 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.453995+0000", 2020-08-07T15:08:09.660 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7050000000000002e-06 2020-08-07T15:08:09.660 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.661 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.661 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:09.661 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.454002+0000", 2020-08-07T15:08:09.661 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26083199299999998 2020-08-07T15:08:09.662 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.662 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.662 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:09.662 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.714834+0000", 2020-08-07T15:08:09.663 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3139999999999997e-06 2020-08-07T15:08:09.663 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.663 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.663 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:09.664 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.714838+0000", 2020-08-07T15:08:09.664 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8711000000000002e-05 2020-08-07T15:08:09.664 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.665 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:09.665 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.714887+0000", 2020-08-07T15:08:09.665 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7895e-05 2020-08-07T15:08:09.665 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.666 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.666 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:09.666 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.714915+0000", 2020-08-07T15:08:09.666 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010364 2020-08-07T15:08:09.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.667 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:09.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.715951+0000", 2020-08-07T15:08:09.668 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1529000000000002e-05 2020-08-07T15:08:09.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:09.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.715993+0000", 2020-08-07T15:08:09.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26204982500000001 2020-08-07T15:08:09.669 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.670 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:09.670 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.670 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.672 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.672 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:71136 3.6 3:64cb7acb:::benchmark_data_smithi074_35447_object8891:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e43)", 2020-08-07T15:08:09.672 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:24.312984+0000", 2020-08-07T15:08:09.672 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 45.276013931999998, 2020-08-07T15:08:09.673 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26986886799999998, 2020-08-07T15:08:09.673 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:09.673 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:09.673 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:09.673 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:09.674 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:09.675 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 71136 2020-08-07T15:08:09.675 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.675 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:09.676 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.676 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:09.676 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.312984+0000", 2020-08-07T15:08:09.676 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:09.676 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:09.677 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.312984+0000", 2020-08-07T15:08:09.677 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6711999999999997e-05 2020-08-07T15:08:09.677 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.678 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.678 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:09.678 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.313021+0000", 2020-08-07T15:08:09.678 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1999999999999999e-06 2020-08-07T15:08:09.678 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.679 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:09.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.313024+0000", 2020-08-07T15:08:09.679 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7479999999999993e-06 2020-08-07T15:08:09.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.680 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:09.680 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.313032+0000", 2020-08-07T15:08:09.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7379999999999999e-06 2020-08-07T15:08:09.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:09.681 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.313035+0000", 2020-08-07T15:08:09.681 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043807819999999997 2020-08-07T15:08:09.681 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.681 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.682 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:09.682 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.317415+0000", 2020-08-07T15:08:09.682 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00042567899999999999 2020-08-07T15:08:09.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.695 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:09.695 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.317841+0000", 2020-08-07T15:08:09.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000173048 2020-08-07T15:08:09.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.696 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:09.696 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318014+0000", 2020-08-07T15:08:09.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00099031100000000001 2020-08-07T15:08:09.711 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.711 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:09.712 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.319004+0000", 2020-08-07T15:08:09.712 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.75e-07 2020-08-07T15:08:09.712 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.712 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.713 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:09.713 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.319005+0000", 2020-08-07T15:08:09.713 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0659000000000003e-05 2020-08-07T15:08:09.713 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.713 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.714 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:09.714 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.319066+0000", 2020-08-07T15:08:09.714 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023319909999999998 2020-08-07T15:08:09.714 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.714 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.715 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:09.715 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.321398+0000", 2020-08-07T15:08:09.715 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26138581700000002 2020-08-07T15:08:09.715 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.715 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.716 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:09.716 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.582783+0000", 2020-08-07T15:08:09.716 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.19e-06 2020-08-07T15:08:09.716 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.716 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.716 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:09.717 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.582785+0000", 2020-08-07T15:08:09.717 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8417999999999994e-05 2020-08-07T15:08:09.717 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.717 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.726 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:09.726 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.582853+0000", 2020-08-07T15:08:09.727 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26986886799999998 2020-08-07T15:08:09.727 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.727 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:09.728 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.728 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:72616 3.6 3:64e83676:::benchmark_data_smithi074_35447_object9076:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e44)", 2020-08-07T15:08:09.729 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:25.246439+0000", 2020-08-07T15:08:09.729 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 44.342558695999998, 2020-08-07T15:08:09.729 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27754408000000003, 2020-08-07T15:08:09.729 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:09.730 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:09.730 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:09.730 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:09.731 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:09.731 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72616 2020-08-07T15:08:09.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.732 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:09.732 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:09.732 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.246439+0000", 2020-08-07T15:08:09.732 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:09.733 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.733 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:09.733 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.246439+0000", 2020-08-07T15:08:09.733 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0911000000000003e-05 2020-08-07T15:08:09.734 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.734 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.734 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:09.734 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.246470+0000", 2020-08-07T15:08:09.734 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7080000000000002e-06 2020-08-07T15:08:09.735 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.735 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.735 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:09.735 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.246473+0000", 2020-08-07T15:08:09.736 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4199999999999998e-06 2020-08-07T15:08:09.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.736 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.736 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:09.736 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.246478+0000", 2020-08-07T15:08:09.737 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5229999999999998e-06 2020-08-07T15:08:09.737 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.737 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.737 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:09.737 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.246481+0000", 2020-08-07T15:08:09.738 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024729550000000002 2020-08-07T15:08:09.738 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.738 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.738 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:09.738 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.248954+0000", 2020-08-07T15:08:09.739 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0921999999999994e-05 2020-08-07T15:08:09.739 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.739 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.739 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:09.739 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.249045+0000", 2020-08-07T15:08:09.740 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000158875 2020-08-07T15:08:09.740 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.740 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.740 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:09.740 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.249204+0000", 2020-08-07T15:08:09.740 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003988068 2020-08-07T15:08:09.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.741 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.741 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:09.741 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.253192+0000", 2020-08-07T15:08:09.741 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7400000000000003e-07 2020-08-07T15:08:09.742 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.742 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.742 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:09.742 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.253192+0000", 2020-08-07T15:08:09.743 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7210000000000003e-05 2020-08-07T15:08:09.743 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.743 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:09.743 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.253250+0000", 2020-08-07T15:08:09.744 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00069997899999999999 2020-08-07T15:08:09.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:09.744 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.253949+0000", 2020-08-07T15:08:09.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26998122600000002 2020-08-07T15:08:09.745 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.745 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:09.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.523931+0000", 2020-08-07T15:08:09.746 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6339999999999999e-06 2020-08-07T15:08:09.746 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.746 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:09.746 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.523932+0000", 2020-08-07T15:08:09.747 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1075000000000001e-05 2020-08-07T15:08:09.747 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.747 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.748 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:09.748 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:25.523983+0000", 2020-08-07T15:08:09.748 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27754408000000003 2020-08-07T15:08:09.748 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.749 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:09.749 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.749 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.749 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.750 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:78743 3.6 3:6151aa5d:::benchmark_data_smithi074_35447_object9842:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e47)", 2020-08-07T15:08:09.750 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:29.249535+0000", 2020-08-07T15:08:09.750 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 40.339462578999999, 2020-08-07T15:08:09.750 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.270038371, 2020-08-07T15:08:09.751 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:09.751 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:09.751 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:09.751 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:09.751 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:09.752 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 78743 2020-08-07T15:08:09.752 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.752 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:09.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.752 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:09.753 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249535+0000", 2020-08-07T15:08:09.753 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:09.753 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.753 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.753 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:09.754 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249535+0000", 2020-08-07T15:08:09.754 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1631000000000001e-05 2020-08-07T15:08:09.754 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.755 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:09.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249587+0000", 2020-08-07T15:08:09.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3969999999999999e-06 2020-08-07T15:08:09.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:09.757 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249593+0000", 2020-08-07T15:08:09.757 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2312e-05 2020-08-07T15:08:09.757 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.757 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:09.758 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249606+0000", 2020-08-07T15:08:09.758 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7379999999999997e-06 2020-08-07T15:08:09.758 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.759 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:09.759 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249613+0000", 2020-08-07T15:08:09.759 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0018089250000000001 2020-08-07T15:08:09.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.760 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.760 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:09.760 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.251421+0000", 2020-08-07T15:08:09.760 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4716999999999995e-05 2020-08-07T15:08:09.760 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.760 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.761 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:09.761 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.251516+0000", 2020-08-07T15:08:09.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000190061 2020-08-07T15:08:09.761 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.762 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.762 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:09.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.251706+0000", 2020-08-07T15:08:09.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013539520000000001 2020-08-07T15:08:09.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.763 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:09.763 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.253060+0000", 2020-08-07T15:08:09.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019342299999999999 2020-08-07T15:08:09.764 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.764 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:09.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.253254+0000", 2020-08-07T15:08:09.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26620518599999998 2020-08-07T15:08:09.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.765 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:09.765 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.519459+0000", 2020-08-07T15:08:09.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2109999999999999e-06 2020-08-07T15:08:09.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.766 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:09.766 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.519460+0000", 2020-08-07T15:08:09.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6767000000000006e-05 2020-08-07T15:08:09.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:09.768 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.519527+0000", 2020-08-07T15:08:09.768 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7051000000000001e-05 2020-08-07T15:08:09.768 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.768 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.768 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:09.769 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.519574+0000", 2020-08-07T15:08:09.769 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.270038371 2020-08-07T15:08:09.769 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.769 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:09.769 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.770 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.770 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.770 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:78744 3.6 3:6151aa5d:::benchmark_data_smithi074_35447_object9842:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e47)", 2020-08-07T15:08:09.770 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:29.249649+0000", 2020-08-07T15:08:09.770 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 40.33934893, 2020-08-07T15:08:09.771 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27244544500000001, 2020-08-07T15:08:09.771 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:09.771 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:09.771 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:09.771 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:09.772 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:09.772 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 78744 2020-08-07T15:08:09.772 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.772 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:09.772 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.772 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:09.774 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249649+0000", 2020-08-07T15:08:09.774 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:09.774 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.774 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.774 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:09.775 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249649+0000", 2020-08-07T15:08:09.775 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6628000000000001e-05 2020-08-07T15:08:09.775 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.775 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:09.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249696+0000", 2020-08-07T15:08:09.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4209999999999998e-06 2020-08-07T15:08:09.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:09.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249703+0000", 2020-08-07T15:08:09.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2456999999999999e-05 2020-08-07T15:08:09.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:09.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249716+0000", 2020-08-07T15:08:09.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1030000000000001e-06 2020-08-07T15:08:09.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.779 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:09.779 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.249721+0000", 2020-08-07T15:08:09.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024347980000000002 2020-08-07T15:08:09.780 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.780 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:09.781 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.252156+0000", 2020-08-07T15:08:09.781 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3114e-05 2020-08-07T15:08:09.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.782 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:09.782 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.252209+0000", 2020-08-07T15:08:09.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3654999999999997e-05 2020-08-07T15:08:09.782 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:09.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.252302+0000", 2020-08-07T15:08:09.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001130437 2020-08-07T15:08:09.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:09.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.253433+0000", 2020-08-07T15:08:09.785 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5849999999999997e-05 2020-08-07T15:08:09.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.786 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:09.787 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.253469+0000", 2020-08-07T15:08:09.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26856134399999998 2020-08-07T15:08:09.787 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.787 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.788 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:09.788 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.522030+0000", 2020-08-07T15:08:09.788 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0500000000000002e-07 2020-08-07T15:08:09.789 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.789 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.789 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:09.790 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.522031+0000", 2020-08-07T15:08:09.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8105000000000002e-05 2020-08-07T15:08:09.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:09.791 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.522069+0000", 2020-08-07T15:08:09.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5728000000000002e-05 2020-08-07T15:08:09.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.792 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.792 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:09.793 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:29.522095+0000", 2020-08-07T15:08:09.793 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27244544500000001 2020-08-07T15:08:09.795 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.796 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:09.796 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.797 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.797 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.810 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:86712 3.8 e52/20 3:14925daa:::benchmark_data_smithi074_35447_object10838:head v 52'5488, mlcod=52'5479)", 2020-08-07T15:08:09.810 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.337908+0000", 2020-08-07T15:08:09.810 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 35.251089616000002, 2020-08-07T15:08:09.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.305870904, 2020-08-07T15:08:09.811 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:09.811 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:09.811 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:09.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:09.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.337908+0000", 2020-08-07T15:08:09.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:09.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.813 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:09.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.337908+0000", 2020-08-07T15:08:09.813 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1465999999999999e-05 2020-08-07T15:08:09.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:09.819 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.337950+0000", 2020-08-07T15:08:09.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1810000000000003e-06 2020-08-07T15:08:09.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:09.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.337954+0000", 2020-08-07T15:08:09.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30500923899999999 2020-08-07T15:08:09.821 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:09.821 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.642963+0000", 2020-08-07T15:08:09.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5389999999999998e-06 2020-08-07T15:08:09.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:09.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.642969+0000", 2020-08-07T15:08:09.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8265e-05 2020-08-07T15:08:09.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:09.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.643017+0000", 2020-08-07T15:08:09.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6812999999999999e-05 2020-08-07T15:08:09.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:09.826 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.643044+0000", 2020-08-07T15:08:09.826 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00067619800000000003 2020-08-07T15:08:09.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:09.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.643720+0000", 2020-08-07T15:08:09.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9203000000000002e-05 2020-08-07T15:08:09.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.827 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:09.829 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.643779+0000", 2020-08-07T15:08:09.829 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.305870904 2020-08-07T15:08:09.829 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.830 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:09.830 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.830 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.830 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.830 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:87087 3.6 3:6b0320df:::benchmark_data_smithi074_35447_object10885:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:08:09.831 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.958471+0000", 2020-08-07T15:08:09.831 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 34.630526678999999, 2020-08-07T15:08:09.831 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30431862500000001, 2020-08-07T15:08:09.831 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:09.831 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:09.832 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:09.832 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:09.832 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:09.833 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 87087 2020-08-07T15:08:09.833 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.833 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:09.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:09.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958471+0000", 2020-08-07T15:08:09.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:09.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:09.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958471+0000", 2020-08-07T15:08:09.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3241999999999997e-05 2020-08-07T15:08:09.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:09.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958515+0000", 2020-08-07T15:08:09.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4930000000000003e-06 2020-08-07T15:08:09.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.836 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:09.836 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958521+0000", 2020-08-07T15:08:09.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2308e-05 2020-08-07T15:08:09.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:09.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958533+0000", 2020-08-07T15:08:09.838 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.105e-06 2020-08-07T15:08:09.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.840 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:09.840 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958540+0000", 2020-08-07T15:08:09.840 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0064000089999999999 2020-08-07T15:08:09.840 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.840 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:09.841 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.964940+0000", 2020-08-07T15:08:09.841 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000159581 2020-08-07T15:08:09.841 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.842 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:09.842 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.965099+0000", 2020-08-07T15:08:09.842 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000160743 2020-08-07T15:08:09.842 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.842 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.843 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:09.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.965260+0000", 2020-08-07T15:08:09.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002913559 2020-08-07T15:08:09.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.844 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:09.844 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.968173+0000", 2020-08-07T15:08:09.844 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5000000000000001e-07 2020-08-07T15:08:09.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:09.845 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.968174+0000", 2020-08-07T15:08:09.845 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6161999999999994e-05 2020-08-07T15:08:09.845 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:09.846 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.968241+0000", 2020-08-07T15:08:09.846 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.010959649 2020-08-07T15:08:09.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.846 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.846 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:09.847 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.979200+0000", 2020-08-07T15:08:09.847 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28352280699999999 2020-08-07T15:08:09.847 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.847 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.847 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:09.848 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262723+0000", 2020-08-07T15:08:09.848 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.19e-06 2020-08-07T15:08:09.848 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.848 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.848 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:09.851 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262724+0000", 2020-08-07T15:08:09.851 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5827000000000005e-05 2020-08-07T15:08:09.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:09.852 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262790+0000", 2020-08-07T15:08:09.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30431862500000001 2020-08-07T15:08:09.853 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.853 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:09.853 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.854 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:87088 3.6 3:6b0320df:::benchmark_data_smithi074_35447_object10885:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:08:09.854 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.958567+0000", 2020-08-07T15:08:09.854 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 34.630431057999999, 2020-08-07T15:08:09.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30442327000000002, 2020-08-07T15:08:09.854 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:09.855 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:09.855 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:09.855 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:09.855 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:09.855 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 87088 2020-08-07T15:08:09.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.856 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:09.856 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.856 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:09.856 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958567+0000", 2020-08-07T15:08:09.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:09.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:09.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958567+0000", 2020-08-07T15:08:09.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5040000000000002e-05 2020-08-07T15:08:09.858 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.858 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:09.858 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958612+0000", 2020-08-07T15:08:09.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4519999999999996e-06 2020-08-07T15:08:09.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.859 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:09.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958617+0000", 2020-08-07T15:08:09.860 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3974e-05 2020-08-07T15:08:09.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.861 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:09.862 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958631+0000", 2020-08-07T15:08:09.862 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5619999999999997e-06 2020-08-07T15:08:09.862 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:09.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958636+0000", 2020-08-07T15:08:09.863 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0071073999999999998 2020-08-07T15:08:09.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:09.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.965743+0000", 2020-08-07T15:08:09.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5477999999999996e-05 2020-08-07T15:08:09.864 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.865 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:09.865 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.965839+0000", 2020-08-07T15:08:09.865 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000168061 2020-08-07T15:08:09.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.867 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.867 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:09.878 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966007+0000", 2020-08-07T15:08:09.878 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054997550000000003 2020-08-07T15:08:09.878 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.879 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:09.879 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971507+0000", 2020-08-07T15:08:09.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6200000000000006e-07 2020-08-07T15:08:09.879 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.880 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:09.881 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971508+0000", 2020-08-07T15:08:09.881 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012566300000000001 2020-08-07T15:08:09.882 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.882 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.882 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:09.882 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971633+0000", 2020-08-07T15:08:09.883 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0075941899999999998 2020-08-07T15:08:09.883 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.883 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.884 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:09.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.979228+0000", 2020-08-07T15:08:09.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28372433899999999 2020-08-07T15:08:09.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.884 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:09.885 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262952+0000", 2020-08-07T15:08:09.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2799999999999995e-07 2020-08-07T15:08:09.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:09.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262953+0000", 2020-08-07T15:08:09.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7666000000000002e-05 2020-08-07T15:08:09.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.886 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:09.887 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262990+0000", 2020-08-07T15:08:09.887 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30442327000000002 2020-08-07T15:08:09.887 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.888 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:09.888 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.888 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:87087 3.6 e52/26 ondisk, result = 0)", 2020-08-07T15:08:09.889 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.971041+0000", 2020-08-07T15:08:09.889 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 34.617957148000002, 2020-08-07T15:08:09.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29175585500000001, 2020-08-07T15:08:09.889 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:09.889 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:09.890 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:09.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.890 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:09.890 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971041+0000", 2020-08-07T15:08:09.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:09.892 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.892 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.892 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:09.892 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971041+0000", 2020-08-07T15:08:09.893 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7841999999999999e-05 2020-08-07T15:08:09.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.894 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.894 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:09.894 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971069+0000", 2020-08-07T15:08:09.894 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7110000000000003e-06 2020-08-07T15:08:09.894 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.895 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.895 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:09.895 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971073+0000", 2020-08-07T15:08:09.895 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28965561299999998 2020-08-07T15:08:09.895 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.896 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:09.896 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260729+0000", 2020-08-07T15:08:09.896 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8010000000000003e-06 2020-08-07T15:08:09.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.897 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.897 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:09.897 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260734+0000", 2020-08-07T15:08:09.897 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019719500000000001 2020-08-07T15:08:09.897 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.897 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.898 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:09.898 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262706+0000", 2020-08-07T15:08:09.898 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7399999999999993e-06 2020-08-07T15:08:09.898 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.898 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.899 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:09.899 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262715+0000", 2020-08-07T15:08:09.899 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2198000000000001e-05 2020-08-07T15:08:09.899 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.899 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.900 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:09.900 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262797+0000", 2020-08-07T15:08:09.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29175585500000001 2020-08-07T15:08:09.901 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.901 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:09.902 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.903 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.905 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:93920 3.6 e56/26 ondisk, result = 0)", 2020-08-07T15:08:09.905 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:08:09.906 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 30.258522086999999, 2020-08-07T15:08:09.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.397576347, 2020-08-07T15:08:09.906 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:09.907 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:09.907 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:09.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:09.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:08:09.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:09.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:09.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:08:09.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2571999999999999e-05 2020-08-07T15:08:09.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:09.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330509+0000", 2020-08-07T15:08:09.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2059999999999998e-06 2020-08-07T15:08:09.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:09.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330514+0000", 2020-08-07T15:08:09.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39740134599999999 2020-08-07T15:08:09.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:09.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727915+0000", 2020-08-07T15:08:09.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.677e-06 2020-08-07T15:08:09.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:09.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727921+0000", 2020-08-07T15:08:09.918 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5431e-05 2020-08-07T15:08:09.918 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.919 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:09.919 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727956+0000", 2020-08-07T15:08:09.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5790000000000004e-06 2020-08-07T15:08:09.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:09.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727965+0000", 2020-08-07T15:08:09.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7535999999999997e-05 2020-08-07T15:08:09.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.922 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:09.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.728052+0000", 2020-08-07T15:08:09.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.397576347 2020-08-07T15:08:09.924 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.930 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:09.931 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.933 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.937 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:104696 3.6 3:680df183:::benchmark_data_smithi074_35447_object13086:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:08:09.937 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:46.386692+0000", 2020-08-07T15:08:09.944 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 23.202306352000001, 2020-08-07T15:08:09.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26505562500000002, 2020-08-07T15:08:09.945 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:09.945 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:09.945 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:09.946 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:09.947 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:09.947 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 104696 2020-08-07T15:08:09.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.947 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:09.947 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.948 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:09.948 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.386692+0000", 2020-08-07T15:08:09.948 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:09.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.949 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:09.949 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.386692+0000", 2020-08-07T15:08:09.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0942000000000001e-05 2020-08-07T15:08:09.954 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.957 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:09.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.386743+0000", 2020-08-07T15:08:09.958 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9279999999999998e-06 2020-08-07T15:08:09.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:09.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.386749+0000", 2020-08-07T15:08:09.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1221e-05 2020-08-07T15:08:09.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:09.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.386760+0000", 2020-08-07T15:08:09.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:08:09.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:09.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.386765+0000", 2020-08-07T15:08:09.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001868219 2020-08-07T15:08:09.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:09.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.388633+0000", 2020-08-07T15:08:09.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010592800000000001 2020-08-07T15:08:09.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.962 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:09.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.388739+0000", 2020-08-07T15:08:09.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017264599999999999 2020-08-07T15:08:09.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:09.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.388912+0000", 2020-08-07T15:08:09.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0030961729999999998 2020-08-07T15:08:09.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:09.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.392008+0000", 2020-08-07T15:08:09.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1909999999999999e-06 2020-08-07T15:08:09.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:09.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.392009+0000", 2020-08-07T15:08:09.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8398000000000005e-05 2020-08-07T15:08:09.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:09.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.392077+0000", 2020-08-07T15:08:09.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001373434 2020-08-07T15:08:09.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.969 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.969 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:09.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.393451+0000", 2020-08-07T15:08:09.969 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25771527799999999 2020-08-07T15:08:09.969 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:09.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.651166+0000", 2020-08-07T15:08:09.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.322e-06 2020-08-07T15:08:09.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.975 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:09.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.651167+0000", 2020-08-07T15:08:09.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00058001099999999996 2020-08-07T15:08:09.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:09.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.651747+0000", 2020-08-07T15:08:09.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26505562500000002 2020-08-07T15:08:09.976 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.977 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:09.977 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:09.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.977 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106104 3.0 3:0d7a2a89:::benchmark_data_smithi074_35447_object13262:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:08:09.978 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.252296+0000", 2020-08-07T15:08:09.978 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 22.336702198000001, 2020-08-07T15:08:09.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.44182183200000003, 2020-08-07T15:08:09.978 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:09.978 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:09.979 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:09.979 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:09.979 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:09.979 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106104 2020-08-07T15:08:09.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.980 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:09.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.981 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:09.981 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252296+0000", 2020-08-07T15:08:09.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:09.982 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.982 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:09.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252296+0000", 2020-08-07T15:08:09.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4728000000000001e-05 2020-08-07T15:08:09.983 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.983 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.983 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:09.983 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252321+0000", 2020-08-07T15:08:09.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9529999999999999e-06 2020-08-07T15:08:09.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.984 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:09.984 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252324+0000", 2020-08-07T15:08:09.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5759999999999996e-06 2020-08-07T15:08:09.985 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.985 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:09.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252329+0000", 2020-08-07T15:08:09.985 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.582e-06 2020-08-07T15:08:09.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.986 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:09.986 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252332+0000", 2020-08-07T15:08:09.986 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012958519999999999 2020-08-07T15:08:09.987 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.987 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:09.987 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.253628+0000", 2020-08-07T15:08:09.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7871e-05 2020-08-07T15:08:09.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:09.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.253705+0000", 2020-08-07T15:08:09.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017801199999999999 2020-08-07T15:08:09.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:08:09.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.253883+0000", 2020-08-07T15:08:09.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001869051 2020-08-07T15:08:09.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.990 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.990 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:09.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255752+0000", 2020-08-07T15:08:09.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3e-07 2020-08-07T15:08:09.990 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:09.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255753+0000", 2020-08-07T15:08:09.991 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5321000000000001e-05 2020-08-07T15:08:09.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:09.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255808+0000", 2020-08-07T15:08:09.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.091917211999999998 2020-08-07T15:08:09.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.993 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.993 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:09.993 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.347726+0000", 2020-08-07T15:08:09.993 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.34634211500000001 2020-08-07T15:08:09.993 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:09.994 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:09.994 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:09.994 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694068+0000", 2020-08-07T15:08:10.000 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3030000000000001e-06 2020-08-07T15:08:10.000 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.001 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.001 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:10.001 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694069+0000", 2020-08-07T15:08:10.001 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8625999999999998e-05 2020-08-07T15:08:10.002 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.002 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.002 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:10.002 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694118+0000", 2020-08-07T15:08:10.002 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.44182183200000003 2020-08-07T15:08:10.003 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:10.003 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:10.003 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:10.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.004 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106104 3.0 e58/20 ondisk, result = 0)", 2020-08-07T15:08:10.004 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.257196+0000", 2020-08-07T15:08:10.004 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 22.331801690999999, 2020-08-07T15:08:10.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43692689000000001, 2020-08-07T15:08:10.005 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:10.005 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:10.005 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:10.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:10.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257196+0000", 2020-08-07T15:08:10.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:10.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:10.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257196+0000", 2020-08-07T15:08:10.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2021999999999999e-05 2020-08-07T15:08:10.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:10.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257228+0000", 2020-08-07T15:08:10.008 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9549999999999997e-06 2020-08-07T15:08:10.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:10.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257234+0000", 2020-08-07T15:08:10.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43671786400000001 2020-08-07T15:08:10.009 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.009 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.009 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:10.010 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693952+0000", 2020-08-07T15:08:10.010 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7690000000000004e-06 2020-08-07T15:08:10.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.010 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:10.010 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693957+0000", 2020-08-07T15:08:10.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3294000000000006e-05 2020-08-07T15:08:10.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.011 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:10.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694040+0000", 2020-08-07T15:08:10.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3436e-05 2020-08-07T15:08:10.012 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.012 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:10.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694054+0000", 2020-08-07T15:08:10.013 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9549999999999996e-05 2020-08-07T15:08:10.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.013 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:10.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694123+0000", 2020-08-07T15:08:10.014 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43692689000000001 2020-08-07T15:08:10.014 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:10.014 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:10.014 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:10.014 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.014 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.015 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106144 3.6 3:629055a5:::benchmark_data_smithi074_35447_object13267:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:08:10.015 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.280142+0000", 2020-08-07T15:08:10.015 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 22.308856229, 2020-08-07T15:08:10.015 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.57968657199999996, 2020-08-07T15:08:10.016 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:10.016 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:10.016 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:10.017 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:10.017 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:10.017 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106144 2020-08-07T15:08:10.017 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.017 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:10.018 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.018 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:10.018 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280142+0000", 2020-08-07T15:08:10.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:10.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.019 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.019 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:10.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280142+0000", 2020-08-07T15:08:10.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.905e-05 2020-08-07T15:08:10.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.020 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.020 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:10.020 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280191+0000", 2020-08-07T15:08:10.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6729999999999997e-06 2020-08-07T15:08:10.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:10.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280197+0000", 2020-08-07T15:08:10.022 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0747999999999999e-05 2020-08-07T15:08:10.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:10.022 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280207+0000", 2020-08-07T15:08:10.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7700000000000001e-06 2020-08-07T15:08:10.023 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.023 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:10.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280212+0000", 2020-08-07T15:08:10.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00060223399999999999 2020-08-07T15:08:10.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:10.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280814+0000", 2020-08-07T15:08:10.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7831000000000006e-05 2020-08-07T15:08:10.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:10.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280912+0000", 2020-08-07T15:08:10.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000137119 2020-08-07T15:08:10.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:10.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.281049+0000", 2020-08-07T15:08:10.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023794369999999999 2020-08-07T15:08:10.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:10.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283429+0000", 2020-08-07T15:08:10.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.173e-06 2020-08-07T15:08:10.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.030 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:10.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283430+0000", 2020-08-07T15:08:10.030 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5530000000000004e-05 2020-08-07T15:08:10.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:10.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283495+0000", 2020-08-07T15:08:10.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5649000000000003e-05 2020-08-07T15:08:10.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:10.033 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283541+0000", 2020-08-07T15:08:10.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.57624206300000003 2020-08-07T15:08:10.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.034 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:10.034 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859783+0000", 2020-08-07T15:08:10.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.421e-06 2020-08-07T15:08:10.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:10.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859785+0000", 2020-08-07T15:08:10.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3874e-05 2020-08-07T15:08:10.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:10.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859828+0000", 2020-08-07T15:08:10.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.57968657199999996 2020-08-07T15:08:10.036 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:10.036 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:10.037 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:10.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.037 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106144 3.6 e59/26 ondisk, result = 0)", 2020-08-07T15:08:10.037 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.344536+0000", 2020-08-07T15:08:10.038 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 22.244461887, 2020-08-07T15:08:10.038 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51529786099999997, 2020-08-07T15:08:10.038 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:10.038 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:10.038 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:10.039 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.039 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:10.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344536+0000", 2020-08-07T15:08:10.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:10.039 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:10.040 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344536+0000", 2020-08-07T15:08:10.040 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7266999999999999e-05 2020-08-07T15:08:10.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.041 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:10.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344553+0000", 2020-08-07T15:08:10.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1070000000000001e-06 2020-08-07T15:08:10.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:10.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344557+0000", 2020-08-07T15:08:10.042 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.515143201 2020-08-07T15:08:10.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.042 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:10.043 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859700+0000", 2020-08-07T15:08:10.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.271e-05 2020-08-07T15:08:10.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:10.044 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859712+0000", 2020-08-07T15:08:10.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5436999999999998e-05 2020-08-07T15:08:10.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.045 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:10.045 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859768+0000", 2020-08-07T15:08:10.046 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1540000000000002e-06 2020-08-07T15:08:10.046 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.046 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.046 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:10.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859776+0000", 2020-08-07T15:08:10.047 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7985000000000001e-05 2020-08-07T15:08:10.047 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.047 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:10.047 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859834+0000", 2020-08-07T15:08:10.048 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51529786099999997 2020-08-07T15:08:10.048 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:10.048 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:10.048 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:10.048 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.048 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.049 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108352 3.8 3:1aac8373:::benchmark_data_smithi074_35447_object13543:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e62)", 2020-08-07T15:08:10.049 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.642093+0000", 2020-08-07T15:08:10.049 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 18.946905198, 2020-08-07T15:08:10.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39397409900000002, 2020-08-07T15:08:10.050 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:10.050 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:10.050 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:10.050 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:10.050 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:10.050 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108352 2020-08-07T15:08:10.051 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.051 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:10.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.051 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:10.051 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642093+0000", 2020-08-07T15:08:10.052 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:10.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.052 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.052 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:10.052 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642093+0000", 2020-08-07T15:08:10.053 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9475000000000002e-05 2020-08-07T15:08:10.053 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.053 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:10.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642122+0000", 2020-08-07T15:08:10.054 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0439999999999997e-06 2020-08-07T15:08:10.054 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.054 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.054 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:10.054 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642126+0000", 2020-08-07T15:08:10.054 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9689999999999994e-06 2020-08-07T15:08:10.055 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.055 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:10.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642134+0000", 2020-08-07T15:08:10.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1219999999999997e-06 2020-08-07T15:08:10.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:10.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642138+0000", 2020-08-07T15:08:10.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011580399999999999 2020-08-07T15:08:10.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:10.057 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.643296+0000", 2020-08-07T15:08:10.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3791999999999999e-05 2020-08-07T15:08:10.058 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.058 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.058 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:10.058 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.643390+0000", 2020-08-07T15:08:10.058 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011882700000000001 2020-08-07T15:08:10.058 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.059 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.059 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4,7", 2020-08-07T15:08:10.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.643509+0000", 2020-08-07T15:08:10.059 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034736799999999998 2020-08-07T15:08:10.059 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:10.060 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.646983+0000", 2020-08-07T15:08:10.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0077748490000000003 2020-08-07T15:08:10.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.061 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:10.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.654758+0000", 2020-08-07T15:08:10.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000005e-07 2020-08-07T15:08:10.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.062 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:10.062 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.654758+0000", 2020-08-07T15:08:10.062 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4258000000000002e-05 2020-08-07T15:08:10.062 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.062 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.063 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:10.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.654802+0000", 2020-08-07T15:08:10.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11166224399999999 2020-08-07T15:08:10.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:10.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.766465+0000", 2020-08-07T15:08:10.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26946740899999999 2020-08-07T15:08:10.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.064 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:10.065 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.035932+0000", 2020-08-07T15:08:10.065 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5829999999999999e-06 2020-08-07T15:08:10.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.065 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:10.066 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.035934+0000", 2020-08-07T15:08:10.066 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000133306 2020-08-07T15:08:10.066 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.066 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.066 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:10.067 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.036067+0000", 2020-08-07T15:08:10.067 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39397409900000002 2020-08-07T15:08:10.067 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:10.067 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:10.067 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:10.067 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.068 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.068 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108464 3.6 3:62704f92:::benchmark_data_smithi074_35447_object13557:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e62)", 2020-08-07T15:08:10.068 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.770928+0000", 2020-08-07T15:08:10.068 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 18.818070057, 2020-08-07T15:08:10.068 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30592449999999999, 2020-08-07T15:08:10.069 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:10.069 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:10.069 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:10.069 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:10.069 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:10.070 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108464 2020-08-07T15:08:10.070 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.070 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:10.071 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.071 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:10.071 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770928+0000", 2020-08-07T15:08:10.071 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:10.071 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.072 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.072 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:10.072 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770928+0000", 2020-08-07T15:08:10.072 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7257e-05 2020-08-07T15:08:10.072 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.073 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.073 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:10.073 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770955+0000", 2020-08-07T15:08:10.073 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0080000000000004e-06 2020-08-07T15:08:10.073 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.073 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.074 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:10.074 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770959+0000", 2020-08-07T15:08:10.074 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2810000000000003e-06 2020-08-07T15:08:10.074 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.074 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.075 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:10.075 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770967+0000", 2020-08-07T15:08:10.075 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3850000000000004e-06 2020-08-07T15:08:10.076 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.076 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.076 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:10.076 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770971+0000", 2020-08-07T15:08:10.076 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4960000000000002e-05 2020-08-07T15:08:10.077 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.077 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.077 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:10.077 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770996+0000", 2020-08-07T15:08:10.077 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4565999999999999e-05 2020-08-07T15:08:10.078 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.078 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.078 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:10.078 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.771050+0000", 2020-08-07T15:08:10.078 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010631799999999999 2020-08-07T15:08:10.079 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.079 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.079 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:10.079 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.771157+0000", 2020-08-07T15:08:10.079 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0049076099999999998 2020-08-07T15:08:10.079 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.080 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.080 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:10.081 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.776064+0000", 2020-08-07T15:08:10.081 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3700000000000001e-07 2020-08-07T15:08:10.081 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.081 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.082 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:10.082 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.776065+0000", 2020-08-07T15:08:10.082 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8387999999999998e-05 2020-08-07T15:08:10.082 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.083 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:10.083 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.776113+0000", 2020-08-07T15:08:10.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00068335599999999998 2020-08-07T15:08:10.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.083 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:10.084 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.776797+0000", 2020-08-07T15:08:10.084 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30001175499999999 2020-08-07T15:08:10.084 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.084 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:10.085 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.076808+0000", 2020-08-07T15:08:10.085 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4200000000000004e-07 2020-08-07T15:08:10.086 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.086 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.086 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:10.086 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.076809+0000", 2020-08-07T15:08:10.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3237e-05 2020-08-07T15:08:10.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:10.087 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.076852+0000", 2020-08-07T15:08:10.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30592449999999999 2020-08-07T15:08:10.088 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:10.088 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:10.088 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:10.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.089 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:116064 3.6 3:6cc03ffa:::benchmark_data_smithi074_35447_object14507:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e64)", 2020-08-07T15:08:10.089 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:56.361361+0000", 2020-08-07T15:08:10.089 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 13.227636682, 2020-08-07T15:08:10.089 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26063054099999999, 2020-08-07T15:08:10.089 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:10.089 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:10.090 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:10.090 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:10.090 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:10.090 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 116064 2020-08-07T15:08:10.090 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.091 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:10.091 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.091 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:10.091 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.361361+0000", 2020-08-07T15:08:10.091 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:10.092 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.092 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.092 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:10.092 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.361361+0000", 2020-08-07T15:08:10.092 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8124999999999999e-05 2020-08-07T15:08:10.093 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.093 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.093 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:10.093 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.361389+0000", 2020-08-07T15:08:10.093 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0439999999999997e-06 2020-08-07T15:08:10.094 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.094 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.094 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:10.094 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.361394+0000", 2020-08-07T15:08:10.094 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8110000000000003e-06 2020-08-07T15:08:10.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.095 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.095 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:10.096 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.361401+0000", 2020-08-07T15:08:10.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0509999999999998e-06 2020-08-07T15:08:10.096 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.096 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.096 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:10.097 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.361405+0000", 2020-08-07T15:08:10.097 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00090700899999999996 2020-08-07T15:08:10.097 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.097 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.097 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:10.097 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362312+0000", 2020-08-07T15:08:10.098 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000105777 2020-08-07T15:08:10.098 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.098 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.098 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:10.098 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362418+0000", 2020-08-07T15:08:10.099 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000205498 2020-08-07T15:08:10.099 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.099 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.100 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:10.100 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362624+0000", 2020-08-07T15:08:10.101 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012514150000000001 2020-08-07T15:08:10.101 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.101 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.102 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:10.102 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.363875+0000", 2020-08-07T15:08:10.102 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6300000000000004e-07 2020-08-07T15:08:10.102 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.102 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.103 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:10.103 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.363876+0000", 2020-08-07T15:08:10.103 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5851000000000004e-05 2020-08-07T15:08:10.104 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.104 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.104 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:10.104 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.363942+0000", 2020-08-07T15:08:10.104 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.062455701000000002 2020-08-07T15:08:10.105 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.105 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.105 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:10.105 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.426398+0000", 2020-08-07T15:08:10.105 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19555302599999999 2020-08-07T15:08:10.106 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.106 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.106 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:10.106 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.621951+0000", 2020-08-07T15:08:10.106 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.063e-06 2020-08-07T15:08:10.107 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.107 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.107 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:10.107 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.621952+0000", 2020-08-07T15:08:10.107 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0306999999999999e-05 2020-08-07T15:08:10.107 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.108 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.108 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:10.108 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.621992+0000", 2020-08-07T15:08:10.108 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26063054099999999 2020-08-07T15:08:10.108 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:10.109 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:10.109 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:10.109 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:10.109 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:10.109 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:10.110 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:08:10.112 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:08:10.113 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:10.110+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:10.213 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:10.211+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:10.326 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:10.318+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:10.414 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:10.523 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:08:10.523 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.524 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:136709 3.4 3:2b527fc9:::benchmark_data_smithi074_35447_object7100:head [delete] snapc 0=[] ondisk+write+known_if_redirected e73)", 2020-08-07T15:08:10.524 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:10.404047+0000", 2020-08-07T15:08:10.524 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0037239930000000001, 2020-08-07T15:08:10.525 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0037524310000000001, 2020-08-07T15:08:10.525 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:10.525 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:10.526 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:10.526 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:10.526 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:10.527 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 136709 2020-08-07T15:08:10.527 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.538 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:10.539 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.539 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:10.539 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:10.404047+0000", 2020-08-07T15:08:10.540 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:10.540 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.540 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.540 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:10.540 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:10.404047+0000", 2020-08-07T15:08:10.541 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9181000000000001e-05 2020-08-07T15:08:10.541 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.541 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:10.541 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:10.404076+0000", 2020-08-07T15:08:10.556 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0460000000000002e-06 2020-08-07T15:08:10.556 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.557 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.557 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:10.561 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:10.404082+0000", 2020-08-07T15:08:10.561 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1943000000000001e-05 2020-08-07T15:08:10.561 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.561 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.561 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:10.562 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:10.404094+0000", 2020-08-07T15:08:10.562 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5999999999999997e-06 2020-08-07T15:08:10.562 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.562 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.562 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:10.563 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:10.404099+0000", 2020-08-07T15:08:10.563 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2970999999999999e-05 2020-08-07T15:08:10.563 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.563 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.563 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:10.564 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:10.404142+0000", 2020-08-07T15:08:10.564 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013155099999999999 2020-08-07T15:08:10.564 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.564 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:10.564 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:10.404274+0000", 2020-08-07T15:08:10.565 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000108039 2020-08-07T15:08:10.565 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.565 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.565 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:08:10.565 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:10.404382+0000", 2020-08-07T15:08:10.578 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00074576899999999997 2020-08-07T15:08:10.579 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.579 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.579 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:10.579 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:10.405128+0000", 2020-08-07T15:08:10.579 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9100000000000003e-07 2020-08-07T15:08:10.580 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.580 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.580 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:10.580 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:10.405129+0000", 2020-08-07T15:08:10.580 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6449000000000003e-05 2020-08-07T15:08:10.581 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.581 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.581 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:10.581 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:10.405165+0000", 2020-08-07T15:08:10.581 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011177959999999999 2020-08-07T15:08:10.584 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.584 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.584 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:10.584 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:10.406283+0000", 2020-08-07T15:08:10.585 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0022361360000000001 2020-08-07T15:08:10.585 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:10.586 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:10.586 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:10.586 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.586 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.586 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:136709 3.4 e73/20 ondisk, result = 0)", 2020-08-07T15:08:10.587 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:10.407627+0000", 2020-08-07T15:08:10.587 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.00014341899999999999, 2020-08-07T15:08:10.587 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000294806, 2020-08-07T15:08:10.587 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:10.587 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:10.588 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:10.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.588 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:10.588 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:10.407627+0000", 2020-08-07T15:08:10.588 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:10.589 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.589 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:10.589 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:10.407627+0000", 2020-08-07T15:08:10.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5837999999999998e-05 2020-08-07T15:08:10.589 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.590 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.590 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:10.590 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:10.407663+0000", 2020-08-07T15:08:10.590 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3249999999999996e-06 2020-08-07T15:08:10.590 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.591 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.591 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:10.591 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:10.407669+0000", 2020-08-07T15:08:10.591 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2218000000000006e-05 2020-08-07T15:08:10.591 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.592 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.592 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:10.592 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:10.407732+0000", 2020-08-07T15:08:10.592 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6450000000000002e-06 2020-08-07T15:08:10.592 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.593 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.593 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:10.593 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:10.407737+0000", 2020-08-07T15:08:10.593 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000147439 2020-08-07T15:08:10.593 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.594 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.594 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:10.594 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:10.407885+0000", 2020-08-07T15:08:10.594 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0725e-05 2020-08-07T15:08:10.594 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:10.594 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:10.595 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:10.595 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:10.407895+0000", 2020-08-07T15:08:10.595 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00026819000000000002 2020-08-07T15:08:10.595 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:10.595 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:10.596 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:10.596 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:10.596 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:08:10.596 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 2 2020-08-07T15:08:10.596 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:10.597 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:08:10.597 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:08:10.598 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump -f json-pretty 2020-08-07T15:08:10.599 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:08:10.600 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:08:10.600 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:08:10.613 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:08:10.614 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:08:10.615 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:10.600+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:10.646 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:08:10.694 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:08:10.707 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:10.699+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:10.749 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:08:10.809 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:10.802+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:10.903 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:10.904 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:10.904 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:10.902+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:10.922 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:08:11.007 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:11.004+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:11.108 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:11.105+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:11.209 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:11.208+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:11.235 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:11.235 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:11.238 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:08:11.239 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:11.239 INFO:teuthology.orchestra.run.smithi074.stdout: "epoch": 73, 2020-08-07T15:08:11.239 INFO:teuthology.orchestra.run.smithi074.stdout: "fsid": "a01d3e82-8d24-4288-9e42-576ab3ff4957", 2020-08-07T15:08:11.240 INFO:teuthology.orchestra.run.smithi074.stdout: "created": "2020-08-07T15:06:07.342912+0000", 2020-08-07T15:08:11.240 INFO:teuthology.orchestra.run.smithi074.stdout: "modified": "2020-08-07T15:08:09.452352+0000", 2020-08-07T15:08:11.240 INFO:teuthology.orchestra.run.smithi074.stdout: "last_up_change": "2020-08-07T15:06:17.674292+0000", 2020-08-07T15:08:11.240 INFO:teuthology.orchestra.run.smithi074.stdout: "last_in_change": "2020-08-07T15:06:17.674292+0000", 2020-08-07T15:08:11.240 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-08-07T15:08:11.241 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_num": 5804032, 2020-08-07T15:08:11.241 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_set": [ 2020-08-07T15:08:11.241 INFO:teuthology.orchestra.run.smithi074.stdout: "nodeep-scrub", 2020-08-07T15:08:11.241 INFO:teuthology.orchestra.run.smithi074.stdout: "pglog_hardlimit", 2020-08-07T15:08:11.241 INFO:teuthology.orchestra.run.smithi074.stdout: "purged_snapdirs", 2020-08-07T15:08:11.242 INFO:teuthology.orchestra.run.smithi074.stdout: "recovery_deletes", 2020-08-07T15:08:11.242 INFO:teuthology.orchestra.run.smithi074.stdout: "sortbitwise" 2020-08-07T15:08:11.242 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:08:11.242 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_version": 5, 2020-08-07T15:08:11.242 INFO:teuthology.orchestra.run.smithi074.stdout: "full_ratio": 0.94999998807907104, 2020-08-07T15:08:11.242 INFO:teuthology.orchestra.run.smithi074.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-08-07T15:08:11.243 INFO:teuthology.orchestra.run.smithi074.stdout: "nearfull_ratio": 0.85000002384185791, 2020-08-07T15:08:11.243 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_snapshot": "", 2020-08-07T15:08:11.243 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_max": 3, 2020-08-07T15:08:11.243 INFO:teuthology.orchestra.run.smithi074.stdout: "max_osd": 8, 2020-08-07T15:08:11.243 INFO:teuthology.orchestra.run.smithi074.stdout: "require_min_compat_client": "luminous", 2020-08-07T15:08:11.244 INFO:teuthology.orchestra.run.smithi074.stdout: "min_compat_client": "luminous", 2020-08-07T15:08:11.244 INFO:teuthology.orchestra.run.smithi074.stdout: "require_osd_release": "pacific", 2020-08-07T15:08:11.244 INFO:teuthology.orchestra.run.smithi074.stdout: "pools": [ 2020-08-07T15:08:11.244 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.245 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 1, 2020-08-07T15:08:11.245 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "device_health_metrics", 2020-08-07T15:08:11.245 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:06:10.648876+0000", 2020-08-07T15:08:11.245 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:08:11.246 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:08:11.246 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:08:11.246 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:08:11.246 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:08:11.246 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:08:11.246 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:08:11.247 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:08:11.247 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 11, 2020-08-07T15:08:11.247 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 11, 2020-08-07T15:08:11.247 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 11, 2020-08-07T15:08:11.247 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 11, 2020-08-07T15:08:11.248 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 11, 2020-08-07T15:08:11.248 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:08:11.248 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:08:11.248 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:08:11.248 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:08:11.249 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:08:11.251 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:08:11.251 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:08:11.251 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.251 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "47", 2020-08-07T15:08:11.251 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:08:11.252 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:08:11.252 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "45", 2020-08-07T15:08:11.252 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:08:11.254 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:08:11.254 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:08:11.254 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:08:11.254 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:08:11.254 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:08:11.255 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:08:11.255 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:08:11.255 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:08:11.255 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:08:11.255 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:08:11.256 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:08:11.256 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:08:11.256 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:08:11.256 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:08:11.257 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:08:11.257 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:08:11.257 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:08:11.257 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:08:11.257 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:08:11.258 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:08:11.258 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:08:11.258 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:08:11.258 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.258 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:08:11.258 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:08:11.259 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:08:11.259 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:08:11.259 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:08:11.260 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:08:11.260 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:08:11.260 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:08:11.260 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:08:11.261 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:08:11.261 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:08:11.261 INFO:teuthology.orchestra.run.smithi074.stdout: "options": { 2020-08-07T15:08:11.261 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_min": 1 2020-08-07T15:08:11.261 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.262 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:08:11.262 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_devicehealth": {} 2020-08-07T15:08:11.262 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.262 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.262 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.262 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 2, 2020-08-07T15:08:11.263 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "rbd", 2020-08-07T15:08:11.263 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:06:21.474648+0000", 2020-08-07T15:08:11.263 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:08:11.263 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:08:11.263 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:08:11.264 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:08:11.264 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:08:11.264 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:08:11.264 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:08:11.264 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:08:11.265 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 8, 2020-08-07T15:08:11.274 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 8, 2020-08-07T15:08:11.274 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 8, 2020-08-07T15:08:11.275 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 8, 2020-08-07T15:08:11.275 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 8, 2020-08-07T15:08:11.275 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:08:11.275 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:08:11.275 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:08:11.276 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:08:11.276 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:08:11.276 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:08:11.276 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:08:11.276 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.277 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "19", 2020-08-07T15:08:11.277 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:08:11.277 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:08:11.277 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "0", 2020-08-07T15:08:11.277 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:08:11.296 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:08:11.296 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:08:11.296 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:08:11.297 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:08:11.297 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:08:11.297 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:08:11.297 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:08:11.297 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:08:11.298 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:08:11.298 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:08:11.298 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:08:11.298 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:08:11.298 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:08:11.299 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:08:11.300 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:08:11.300 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:08:11.300 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:08:11.300 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:08:11.301 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:08:11.301 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:08:11.301 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:08:11.302 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:08:11.302 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.302 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:08:11.302 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:08:11.303 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:08:11.303 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:08:11.303 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:08:11.303 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:08:11.303 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:08:11.304 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:08:11.304 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:08:11.304 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:08:11.304 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:08:11.304 INFO:teuthology.orchestra.run.smithi074.stdout: "options": {}, 2020-08-07T15:08:11.305 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:08:11.306 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd": {} 2020-08-07T15:08:11.306 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.306 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.306 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.307 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 3, 2020-08-07T15:08:11.307 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "unique_pool_0", 2020-08-07T15:08:11.307 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:06:33.421788+0000", 2020-08-07T15:08:11.308 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:08:11.308 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:08:11.308 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:08:11.308 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:08:11.308 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:08:11.309 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:08:11.309 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:08:11.309 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:08:11.309 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 16, 2020-08-07T15:08:11.309 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 16, 2020-08-07T15:08:11.310 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 16, 2020-08-07T15:08:11.310 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 16, 2020-08-07T15:08:11.310 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 16, 2020-08-07T15:08:11.310 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:08:11.310 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:08:11.311 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:08:11.311 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:08:11.311 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:08:11.311 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:08:11.311 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:08:11.311 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.312 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "71", 2020-08-07T15:08:11.312 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:08:11.313 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:08:11.313 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "0", 2020-08-07T15:08:11.313 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:08:11.313 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:08:11.313 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:08:11.314 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:08:11.314 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:08:11.314 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:08:11.314 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:08:11.314 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:08:11.315 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:08:11.315 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:08:11.315 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:08:11.315 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:08:11.315 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:08:11.316 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:08:11.316 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:08:11.316 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:08:11.316 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:08:11.316 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:08:11.317 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:08:11.317 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:08:11.317 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:08:11.317 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:08:11.317 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:08:11.317 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.318 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:08:11.318 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:08:11.318 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:08:11.318 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:08:11.318 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:08:11.319 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:08:11.319 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:08:11.319 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:08:11.319 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:08:11.319 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:08:11.320 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:08:11.320 INFO:teuthology.orchestra.run.smithi074.stdout: "options": {}, 2020-08-07T15:08:11.320 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:08:11.320 INFO:teuthology.orchestra.run.smithi074.stdout: "rados": {} 2020-08-07T15:08:11.320 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.320 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.321 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:08:11.321 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:08:11.321 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.321 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 0, 2020-08-07T15:08:11.321 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "fc1ebb23-3ef4-4fc0-b395-382e385885a3", 2020-08-07T15:08:11.322 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:08:11.322 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:08:11.322 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:08:11.323 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0, 2020-08-07T15:08:11.323 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:08:11.323 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:08:11.323 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:08:11.324 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 49, 2020-08-07T15:08:11.324 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:08:11.324 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:08:11.324 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:08:11.324 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.324 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.325 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.325 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6801", 2020-08-07T15:08:11.325 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:08:11.326 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.326 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.326 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.326 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:08:11.326 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.327 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.327 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.327 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6802", 2020-08-07T15:08:11.327 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:08:11.327 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.327 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.328 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.328 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:08:11.328 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.328 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.328 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.329 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6804", 2020-08-07T15:08:11.329 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:08:11.329 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.329 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.329 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.330 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:08:11.330 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.330 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.330 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.331 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6803", 2020-08-07T15:08:11.331 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:08:11.331 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.331 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.331 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.332 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6801/31577", 2020-08-07T15:08:11.332 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6802/31577", 2020-08-07T15:08:11.332 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6804/31577", 2020-08-07T15:08:11.332 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6803/31577", 2020-08-07T15:08:11.332 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:08:11.335 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:08:11.335 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:08:11.336 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.336 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.336 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.336 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 1, 2020-08-07T15:08:11.337 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "1fbe530a-63ce-4dc8-82ab-18793d6ebc0b", 2020-08-07T15:08:11.337 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:08:11.337 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:08:11.338 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:08:11.338 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0.7196044921875, 2020-08-07T15:08:11.339 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:08:11.339 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:08:11.339 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:08:11.339 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 60, 2020-08-07T15:08:11.340 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:08:11.340 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:08:11.341 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:08:11.341 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.341 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.341 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6809", 2020-08-07T15:08:11.345 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:08:11.345 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.345 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.347 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.347 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:08:11.347 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.347 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.348 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6810", 2020-08-07T15:08:11.348 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:08:11.348 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.349 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.349 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.349 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:08:11.349 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.350 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.350 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.350 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6812", 2020-08-07T15:08:11.350 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:08:11.351 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.351 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.351 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.351 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:08:11.352 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.352 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.352 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.352 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6811", 2020-08-07T15:08:11.352 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:08:11.353 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.353 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.354 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.354 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6809/31578", 2020-08-07T15:08:11.355 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6810/31578", 2020-08-07T15:08:11.355 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6812/31578", 2020-08-07T15:08:11.355 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6811/31578", 2020-08-07T15:08:11.355 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:08:11.356 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:08:11.356 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:08:11.356 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.356 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.356 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.356 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 2, 2020-08-07T15:08:11.357 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "d6fe4338-3e11-4f73-9cbb-e1b046ce286a", 2020-08-07T15:08:11.357 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:08:11.357 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:08:11.357 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:08:11.357 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:08:11.358 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:08:11.358 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:08:11.358 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:08:11.358 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 49, 2020-08-07T15:08:11.358 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:08:11.359 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:08:11.359 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:08:11.359 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.359 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.359 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.360 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6805", 2020-08-07T15:08:11.360 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:08:11.360 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.360 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.360 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.360 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:08:11.361 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.361 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.361 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.361 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6806", 2020-08-07T15:08:11.361 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:08:11.362 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.362 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.362 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.362 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:08:11.362 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.363 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.363 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.363 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6808", 2020-08-07T15:08:11.363 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:08:11.363 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.364 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.365 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.366 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:08:11.367 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.367 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.367 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.367 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6807", 2020-08-07T15:08:11.367 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:08:11.367 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.368 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.368 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.368 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6805/31576", 2020-08-07T15:08:11.369 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6806/31576", 2020-08-07T15:08:11.369 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6808/31576", 2020-08-07T15:08:11.369 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6807/31576", 2020-08-07T15:08:11.369 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:08:11.369 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:08:11.370 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:08:11.370 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.370 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.370 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.370 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 3, 2020-08-07T15:08:11.371 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9", 2020-08-07T15:08:11.371 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:08:11.371 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:08:11.371 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 0.7917938232421875, 2020-08-07T15:08:11.371 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:08:11.371 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:08:11.372 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:08:11.372 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:08:11.372 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 62, 2020-08-07T15:08:11.372 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:08:11.372 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:08:11.373 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:08:11.373 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.373 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.373 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.373 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6813", 2020-08-07T15:08:11.374 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:08:11.374 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.374 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.374 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.374 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:08:11.374 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.377 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.377 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6814", 2020-08-07T15:08:11.377 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:08:11.377 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.377 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.378 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:08:11.378 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.378 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.379 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6816", 2020-08-07T15:08:11.379 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:08:11.379 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.379 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.387 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.387 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:08:11.391 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.392 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.392 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.392 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6815", 2020-08-07T15:08:11.392 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:08:11.392 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.393 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.393 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.393 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6813/31579", 2020-08-07T15:08:11.403 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6814/31579", 2020-08-07T15:08:11.404 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6816/31579", 2020-08-07T15:08:11.404 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6815/31579", 2020-08-07T15:08:11.406 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:08:11.406 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:08:11.406 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:08:11.408 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.409 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.409 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 4, 2020-08-07T15:08:11.409 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "01aadc13-29c5-4ddd-9bb9-de24b2b20412", 2020-08-07T15:08:11.409 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:08:11.410 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:08:11.410 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:08:11.410 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:08:11.410 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:08:11.410 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:08:11.411 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:08:11.411 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 67, 2020-08-07T15:08:11.411 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:08:11.411 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:08:11.411 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:08:11.412 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.412 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.412 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.412 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6800", 2020-08-07T15:08:11.412 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:08:11.412 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.413 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.413 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.413 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:08:11.413 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.429 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.429 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6801", 2020-08-07T15:08:11.430 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:08:11.432 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.432 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.444 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.444 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:08:11.444 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.445 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.445 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6803", 2020-08-07T15:08:11.445 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:08:11.445 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.447 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.448 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:08:11.448 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.448 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.448 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6802", 2020-08-07T15:08:11.449 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:08:11.449 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.449 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.449 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6800/31545", 2020-08-07T15:08:11.450 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6801/31545", 2020-08-07T15:08:11.450 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6803/31545", 2020-08-07T15:08:11.450 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6802/31545", 2020-08-07T15:08:11.450 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:08:11.450 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:08:11.463 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:08:11.463 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.463 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.464 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.464 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 5, 2020-08-07T15:08:11.464 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "f5556a82-73b4-47b1-8e29-1d2fe031d8fd", 2020-08-07T15:08:11.464 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:08:11.473 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:08:11.473 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:08:11.474 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:08:11.474 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:08:11.474 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:08:11.474 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:08:11.474 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 61, 2020-08-07T15:08:11.475 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:08:11.475 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:08:11.475 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:08:11.475 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.475 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.476 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.476 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6804", 2020-08-07T15:08:11.476 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:08:11.476 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.476 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.476 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.477 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:08:11.477 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.477 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.477 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.482 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6805", 2020-08-07T15:08:11.490 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:08:11.490 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.491 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.492 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.492 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:08:11.495 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.496 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.496 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.496 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6807", 2020-08-07T15:08:11.496 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:08:11.496 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.497 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.497 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.497 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:08:11.497 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.502 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.503 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.503 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6806", 2020-08-07T15:08:11.504 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:08:11.504 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.504 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.504 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.505 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6804/31547", 2020-08-07T15:08:11.505 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6805/31547", 2020-08-07T15:08:11.505 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6807/31547", 2020-08-07T15:08:11.505 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6806/31547", 2020-08-07T15:08:11.514 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:08:11.517 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:08:11.518 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:08:11.518 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.518 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 6, 2020-08-07T15:08:11.519 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "9b98b337-e1cc-43cd-9e9b-f21ae3f20899", 2020-08-07T15:08:11.519 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:08:11.519 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:08:11.519 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:08:11.519 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:08:11.520 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:08:11.520 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:08:11.520 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 14, 2020-08-07T15:08:11.520 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 62, 2020-08-07T15:08:11.520 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:08:11.521 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:08:11.521 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:08:11.522 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.522 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.522 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.522 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6808", 2020-08-07T15:08:11.522 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:08:11.523 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.523 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.523 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.523 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:08:11.523 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.523 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.524 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.524 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6809", 2020-08-07T15:08:11.524 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:08:11.532 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.533 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.533 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.533 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:08:11.533 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.533 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.534 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.534 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6811", 2020-08-07T15:08:11.534 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:08:11.534 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.534 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.535 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:08:11.535 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.535 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.535 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6810", 2020-08-07T15:08:11.536 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:08:11.536 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.536 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.536 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6808/31546", 2020-08-07T15:08:11.537 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6809/31546", 2020-08-07T15:08:11.537 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6811/31546", 2020-08-07T15:08:11.537 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6810/31546", 2020-08-07T15:08:11.537 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:08:11.537 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:08:11.574 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:08:11.574 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.574 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.575 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 7, 2020-08-07T15:08:11.575 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "777662d8-bd34-48e8-83e1-24ce9e61f14b", 2020-08-07T15:08:11.575 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:08:11.575 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:08:11.576 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:08:11.576 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:08:11.576 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:08:11.576 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:08:11.576 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:08:11.577 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 50, 2020-08-07T15:08:11.577 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:08:11.577 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:08:11.577 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:08:11.577 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.596 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.599 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.600 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6812", 2020-08-07T15:08:11.600 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:08:11.600 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.600 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.601 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:08:11.601 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.602 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.602 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6813", 2020-08-07T15:08:11.602 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:08:11.602 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.602 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.603 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.603 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:08:11.603 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.603 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.604 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6815", 2020-08-07T15:08:11.604 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:08:11.606 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.606 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.606 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.607 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:08:11.607 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:11.607 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.607 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:11.607 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6814", 2020-08-07T15:08:11.608 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:08:11.608 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.608 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.608 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.608 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6812/31570", 2020-08-07T15:08:11.609 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6813/31570", 2020-08-07T15:08:11.609 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6815/31570", 2020-08-07T15:08:11.609 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6814/31570", 2020-08-07T15:08:11.609 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:08:11.614 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:08:11.614 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:08:11.614 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.617 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.618 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:08:11.618 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_xinfo": [ 2020-08-07T15:08:11.618 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.618 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 0, 2020-08-07T15:08:11.620 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:08:11.620 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:08:11.621 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:08:11.621 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:08:11.621 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:08:11.621 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:07:16.136193+0000", 2020-08-07T15:08:11.621 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:08:11.622 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.622 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.622 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 1, 2020-08-07T15:08:11.622 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:08:11.622 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:08:11.623 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:08:11.623 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:08:11.623 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:08:11.623 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:07:15.797452+0000", 2020-08-07T15:08:11.651 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:08:11.651 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.652 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.652 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 2, 2020-08-07T15:08:11.652 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:08:11.652 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:08:11.652 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:08:11.653 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:08:11.653 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:08:11.653 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:07:16.224551+0000", 2020-08-07T15:08:11.653 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:08:11.653 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.654 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.654 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 3, 2020-08-07T15:08:11.654 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:08:11.654 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:08:11.654 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:08:11.655 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:08:11.655 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:08:11.655 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:07:16.395481+0000", 2020-08-07T15:08:11.655 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:08:11.655 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.656 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.656 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 4, 2020-08-07T15:08:11.656 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:08:11.656 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:08:11.656 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:08:11.656 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:08:11.657 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:08:11.657 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:07:16.310712+0000", 2020-08-07T15:08:11.657 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:08:11.708 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.708 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 5, 2020-08-07T15:08:11.708 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:08:11.709 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:08:11.709 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:08:11.709 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:08:11.709 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:08:11.709 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:07:16.580462+0000", 2020-08-07T15:08:11.710 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:08:11.710 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.710 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.710 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 6, 2020-08-07T15:08:11.710 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:08:11.711 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:08:11.711 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:08:11.711 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:08:11.711 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:08:11.711 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:07:16.264199+0000", 2020-08-07T15:08:11.712 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:08:11.712 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.712 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.712 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 7, 2020-08-07T15:08:11.712 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:08:11.712 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:08:11.713 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:08:11.713 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:08:11.713 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:08:11.713 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:07:16.087894+0000", 2020-08-07T15:08:11.713 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:08:11.730 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.730 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:08:11.731 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_upmap": [ 2020-08-07T15:08:11.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.731 INFO:teuthology.orchestra.run.smithi074.stdout: "pgid": "3.d", 2020-08-07T15:08:11.731 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:08:11.732 INFO:teuthology.orchestra.run.smithi074.stdout: 3, 2020-08-07T15:08:11.732 INFO:teuthology.orchestra.run.smithi074.stdout: 4, 2020-08-07T15:08:11.732 INFO:teuthology.orchestra.run.smithi074.stdout: 6 2020-08-07T15:08:11.732 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.732 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.732 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:08:11.733 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_upmap_items": [ 2020-08-07T15:08:11.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.733 INFO:teuthology.orchestra.run.smithi074.stdout: "pgid": "3.7", 2020-08-07T15:08:11.733 INFO:teuthology.orchestra.run.smithi074.stdout: "mappings": [ 2020-08-07T15:08:11.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.739 INFO:teuthology.orchestra.run.smithi074.stdout: "from": 0, 2020-08-07T15:08:11.739 INFO:teuthology.orchestra.run.smithi074.stdout: "to": 7 2020-08-07T15:08:11.740 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.759 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.759 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.759 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:08:11.759 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_temp": [ 2020-08-07T15:08:11.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:11.761 INFO:teuthology.orchestra.run.smithi074.stdout: "pgid": "3.8", 2020-08-07T15:08:11.761 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:08:11.761 INFO:teuthology.orchestra.run.smithi074.stdout: 1, 2020-08-07T15:08:11.761 INFO:teuthology.orchestra.run.smithi074.stdout: 7, 2020-08-07T15:08:11.762 INFO:teuthology.orchestra.run.smithi074.stdout: 3 2020-08-07T15:08:11.762 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:11.762 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.762 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:08:11.762 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_temp": [], 2020-08-07T15:08:11.762 INFO:teuthology.orchestra.run.smithi074.stdout: "blacklist": {}, 2020-08-07T15:08:11.763 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profiles": { 2020-08-07T15:08:11.763 INFO:teuthology.orchestra.run.smithi074.stdout: "default": { 2020-08-07T15:08:11.763 INFO:teuthology.orchestra.run.smithi074.stdout: "crush-failure-domain": "osd", 2020-08-07T15:08:11.763 INFO:teuthology.orchestra.run.smithi074.stdout: "k": "2", 2020-08-07T15:08:11.763 INFO:teuthology.orchestra.run.smithi074.stdout: "m": "1", 2020-08-07T15:08:11.764 INFO:teuthology.orchestra.run.smithi074.stdout: "plugin": "jerasure", 2020-08-07T15:08:11.764 INFO:teuthology.orchestra.run.smithi074.stdout: "ruleset-failure-domain": "osd", 2020-08-07T15:08:11.764 INFO:teuthology.orchestra.run.smithi074.stdout: "technique": "reed_sol_van" 2020-08-07T15:08:11.764 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:11.764 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:11.765 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps_queue": [], 2020-08-07T15:08:11.765 INFO:teuthology.orchestra.run.smithi074.stdout: "new_removed_snaps": [], 2020-08-07T15:08:11.765 INFO:teuthology.orchestra.run.smithi074.stdout: "new_purged_snaps": [], 2020-08-07T15:08:11.765 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_node_flags": {}, 2020-08-07T15:08:11.765 INFO:teuthology.orchestra.run.smithi074.stdout: "device_class_flags": {} 2020-08-07T15:08:11.778 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:11.779 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:08:11.781 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:08:11.783 INFO:tasks.thrashosds.thrasher:j is {'epoch': 73, 'fsid': 'a01d3e82-8d24-4288-9e42-576ab3ff4957', 'created': '2020-08-07T15:06:07.342912+0000', 'modified': '2020-08-07T15:08:09.452352+0000', 'last_up_change': '2020-08-07T15:06:17.674292+0000', 'last_in_change': '2020-08-07T15:06:17.674292+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-08-07T15:06:10.648876+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 11, 'pg_placement_num': 11, 'pg_placement_num_target': 11, 'pg_num_target': 11, 'pg_num_pending': 11, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '45', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-08-07T15:06:21.474648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2020-08-07T15:06:33.421788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': 'fc1ebb23-3ef4-4fc0-b395-382e385885a3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6801', 'nonce': 31577}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6802', 'nonce': 31577}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6804', 'nonce': 31577}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6803', 'nonce': 31577}]}, 'public_addr': '172.21.15.74:6801/31577', 'cluster_addr': '172.21.15.74:6802/31577', 'heartbeat_back_addr': '172.21.15.74:6804/31577', 'heartbeat_front_addr': '172.21.15.74:6803/31577', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '1fbe530a-63ce-4dc8-82ab-18793d6ebc0b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.7196044921875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 60, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6809', 'nonce': 31578}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6810', 'nonce': 31578}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6812', 'nonce': 31578}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6811', 'nonce': 31578}]}, 'public_addr': '172.21.15.74:6809/31578', 'cluster_addr': '172.21.15.74:6810/31578', 'heartbeat_back_addr': '172.21.15.74:6812/31578', 'heartbeat_front_addr': '172.21.15.74:6811/31578', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd6fe4338-3e11-4f73-9cbb-e1b046ce286a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6805', 'nonce': 31576}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6806', 'nonce': 31576}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6808', 'nonce': 31576}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6807', 'nonce': 31576}]}, 'public_addr': '172.21.15.74:6805/31576', 'cluster_addr': '172.21.15.74:6806/31576', 'heartbeat_back_addr': '172.21.15.74:6808/31576', 'heartbeat_front_addr': '172.21.15.74:6807/31576', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9', 'up': 1, 'in': 1, 'weight': 0.7917938232421875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 62, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6813', 'nonce': 31579}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6814', 'nonce': 31579}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6816', 'nonce': 31579}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6815', 'nonce': 31579}]}, 'public_addr': '172.21.15.74:6813/31579', 'cluster_addr': '172.21.15.74:6814/31579', 'heartbeat_back_addr': '172.21.15.74:6816/31579', 'heartbeat_front_addr': '172.21.15.74:6815/31579', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '01aadc13-29c5-4ddd-9bb9-de24b2b20412', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 67, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6800', 'nonce': 31545}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6801', 'nonce': 31545}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6803', 'nonce': 31545}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6802', 'nonce': 31545}]}, 'public_addr': '172.21.15.96:6800/31545', 'cluster_addr': '172.21.15.96:6801/31545', 'heartbeat_back_addr': '172.21.15.96:6803/31545', 'heartbeat_front_addr': '172.21.15.96:6802/31545', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'f5556a82-73b4-47b1-8e29-1d2fe031d8fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 61, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6804', 'nonce': 31547}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6805', 'nonce': 31547}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6807', 'nonce': 31547}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6806', 'nonce': 31547}]}, 'public_addr': '172.21.15.96:6804/31547', 'cluster_addr': '172.21.15.96:6805/31547', 'heartbeat_back_addr': '172.21.15.96:6807/31547', 'heartbeat_front_addr': '172.21.15.96:6806/31547', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9b98b337-e1cc-43cd-9e9b-f21ae3f20899', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 62, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6808', 'nonce': 31546}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6809', 'nonce': 31546}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6811', 'nonce': 31546}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6810', 'nonce': 31546}]}, 'public_addr': '172.21.15.96:6808/31546', 'cluster_addr': '172.21.15.96:6809/31546', 'heartbeat_back_addr': '172.21.15.96:6811/31546', 'heartbeat_front_addr': '172.21.15.96:6810/31546', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '777662d8-bd34-48e8-83e1-24ce9e61f14b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6812', 'nonce': 31570}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6813', 'nonce': 31570}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6815', 'nonce': 31570}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6814', 'nonce': 31570}]}, 'public_addr': '172.21.15.96:6812/31570', 'cluster_addr': '172.21.15.96:6813/31570', 'heartbeat_back_addr': '172.21.15.96:6815/31570', 'heartbeat_front_addr': '172.21.15.96:6814/31570', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:07:16.136193+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:07:15.797452+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:07:16.224551+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:07:16.395481+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:07:16.310712+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:07:16.580462+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:07:16.264199+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:07:16.087894+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.d', 'osds': [3, 4, 6]}], 'pg_upmap_items': [{'pgid': '3.7', 'mappings': [{'from': 0, 'to': 7}]}], 'pg_temp': [{'pgid': '3.8', 'osds': [1, 7, 3]}], 'primary_temp': [], 'blacklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}} 2020-08-07T15:08:11.783 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:08:11.784 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:08:11.784 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:08:11.784 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:08:11.785 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:08:11.785 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:08:11.785 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:08:11.785 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:08:11.785 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:08:11.802 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:08:11.802 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:11.778+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:11.887 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:11.881+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:11.988 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:11.985+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:12.046 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:12.046 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:12.059 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:08:12.103 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:12.100+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:12.202 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:12.201+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:12.313 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:12.305+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:12.372 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:12.372 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:12.373 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:08:12.385 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:08:12.395 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:08:12.401 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2020-08-07T15:08:12.156715+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":433766424,"num_objects":6622,"num_object_clones":0,"num_object_copies":19866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":374,"num_objects_unfound":0,"num_objects_dirty":6622,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":139190,"num_write_kb":1036850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1651,"num_bytes_recovered":108134400,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":105,"acting":105,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":109,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":10017768,"kb_used_data":1628456,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":744956952,"statfs":{"total":773094113280,"available":762835918848,"internally_reserved":8589934592,"allocated":1667538944,"data_stored":1663527879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,3,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-574013416,"num_objects":-8758,"num_object_clones":0,"num_object_copies":-26274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-585,"num_objects_unfound":0,"num_objects_dirty":-8758,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16169,"num_write_kb":52689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.727692"},"pg_stats":[{"pgid":"3.f","version":"73'9195","reported_seq":"9256","reported_epoch":"73","state":"active+clean","last_fresh":"2020-08-07T15:08:11.679748+0000","last_change":"2020-08-07T15:06:35.163498+0000","last_active":"2020-08-07T15:08:11.679748+0000","last_peered":"2020-08-07T15:08:11.679748+0000","last_clean":"2020-08-07T15:08:11.679748+0000","last_became_active":"2020-08-07T15:06:35.162764+0000","last_became_peered":"2020-08-07T15:06:35.162764+0000","last_unstale":"2020-08-07T15:08:11.679748+0000","last_undegraded":"2020-08-07T15:08:11.679748+0000","last_fullsized":"2020-08-07T15:08:11.679748+0000","mapping_epoch":20,"log_start":"73'9192","ondisk_log_start":"73'9192","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25559040,"num_objects":390,"num_object_clones":0,"num_object_copies":1170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9195,"num_write_kb":68160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"73'8942","reported_seq":"9003","reported_epoch":"73","state":"active+clean","last_fresh":"2020-08-07T15:08:11.691385+0000","last_change":"2020-08-07T15:06:35.169137+0000","last_active":"2020-08-07T15:08:11.691385+0000","last_peered":"2020-08-07T15:08:11.691385+0000","last_clean":"2020-08-07T15:08:11.691385+0000","last_became_active":"2020-08-07T15:06:35.168689+0000","last_became_peered":"2020-08-07T15:06:35.168689+0000","last_unstale":"2020-08-07T15:08:11.691385+0000","last_undegraded":"2020-08-07T15:08:11.691385+0000","last_fullsized":"2020-08-07T15:08:11.691385+0000","mapping_epoch":20,"log_start":"73'8936","ondisk_log_start":"73'8936","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24887296,"num_objects":380,"num_object_clones":0,"num_object_copies":1140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8942,"num_write_kb":66288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"73'8895","reported_seq":"11436","reported_epoch":"73","state":"active+clean","last_fresh":"2020-08-07T15:08:11.702402+0000","last_change":"2020-08-07T15:08:00.913767+0000","last_active":"2020-08-07T15:08:11.702402+0000","last_peered":"2020-08-07T15:08:11.702402+0000","last_clean":"2020-08-07T15:08:11.702402+0000","last_became_active":"2020-08-07T15:07:51.356141+0000","last_became_peered":"2020-08-07T15:07:51.356141+0000","last_unstale":"2020-08-07T15:08:11.702402+0000","last_undegraded":"2020-08-07T15:08:11.702402+0000","last_fullsized":"2020-08-07T15:08:11.702402+0000","mapping_epoch":62,"log_start":"73'8892","ondisk_log_start":"73'8892","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'7824","last_scrub_stamp":"2020-08-07T15:08:00.913676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:08:00.913676+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24641560,"num_objects":377,"num_object_clones":0,"num_object_copies":1131,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":377,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8895,"num_write_kb":65921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":821,"num_bytes_recovered":53805056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"73'9180","reported_seq":"9241","reported_epoch":"73","state":"active+clean","last_fresh":"2020-08-07T15:08:11.690121+0000","last_change":"2020-08-07T15:06:35.167480+0000","last_active":"2020-08-07T15:08:11.690121+0000","last_peered":"2020-08-07T15:08:11.690121+0000","last_clean":"2020-08-07T15:08:11.690121+0000","last_became_active":"2020-08-07T15:06:35.167101+0000","last_became_peered":"2020-08-07T15:06:35.167101+0000","last_unstale":"2020-08-07T15:08:11.690121+0000","last_undegraded":"2020-08-07T15:08:11.690121+0000","last_fullsized":"2020-08-07T15:08:11.690121+0000","mapping_epoch":20,"log_start":"73'9177","ondisk_log_start":"73'9177","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27131904,"num_objects":414,"num_object_clones":0,"num_object_copies":1242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9180,"num_write_kb":68224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"103","reported_epoch":"72","state":"active+clean","last_fresh":"2020-08-07T15:08:06.477539+0000","last_change":"2020-08-07T15:07:56.631170+0000","last_active":"2020-08-07T15:08:06.477539+0000","last_peered":"2020-08-07T15:08:06.477539+0000","last_clean":"2020-08-07T15:08:06.477539+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:08:06.477539+0000","last_undegraded":"2020-08-07T15:08:06.477539+0000","last_fullsized":"2020-08-07T15:08:06.477539+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:56.631091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:56.631091+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.b","version":"73'8132","reported_seq":"8193","reported_epoch":"73","state":"active+clean","last_fresh":"2020-08-07T15:08:11.706045+0000","last_change":"2020-08-07T15:06:35.170504+0000","last_active":"2020-08-07T15:08:11.706045+0000","last_peered":"2020-08-07T15:08:11.706045+0000","last_clean":"2020-08-07T15:08:11.706045+0000","last_became_active":"2020-08-07T15:06:35.169532+0000","last_became_peered":"2020-08-07T15:06:35.169532+0000","last_unstale":"2020-08-07T15:08:11.706045+0000","last_undegraded":"2020-08-07T15:08:11.706045+0000","last_fullsized":"2020-08-07T15:08:11.706045+0000","mapping_epoch":20,"log_start":"73'8129","ondisk_log_start":"73'8129","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23265280,"num_objects":355,"num_object_clones":0,"num_object_copies":1065,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8132,"num_write_kb":60352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"39","reported_epoch":"72","state":"active+clean","last_fresh":"2020-08-07T15:08:06.443257+0000","last_change":"2020-08-07T15:07:33.177491+0000","last_active":"2020-08-07T15:08:06.443257+0000","last_peered":"2020-08-07T15:08:06.443257+0000","last_clean":"2020-08-07T15:08:06.443257+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:08:06.443257+0000","last_undegraded":"2020-08-07T15:08:06.443257+0000","last_fullsized":"2020-08-07T15:08:06.443257+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:33.177412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:33.177412+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.a","version":"72'8419","reported_seq":"8479","reported_epoch":"72","state":"active+clean","last_fresh":"2020-08-07T15:08:07.705748+0000","last_change":"2020-08-07T15:06:35.170105+0000","last_active":"2020-08-07T15:08:07.705748+0000","last_peered":"2020-08-07T15:08:07.705748+0000","last_clean":"2020-08-07T15:08:07.705748+0000","last_became_active":"2020-08-07T15:06:35.169494+0000","last_became_peered":"2020-08-07T15:06:35.169494+0000","last_unstale":"2020-08-07T15:08:07.705748+0000","last_undegraded":"2020-08-07T15:08:07.705748+0000","last_fullsized":"2020-08-07T15:08:07.705748+0000","mapping_epoch":20,"log_start":"72'8414","ondisk_log_start":"72'8414","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":62259200,"num_objects":950,"num_object_clones":0,"num_object_copies":2850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8419,"num_write_kb":66624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"73'8516","reported_seq":"8577","reported_epoch":"73","state":"active+clean","last_fresh":"2020-08-07T15:08:11.649190+0000","last_change":"2020-08-07T15:06:35.163109+0000","last_active":"2020-08-07T15:08:11.649190+0000","last_peered":"2020-08-07T15:08:11.649190+0000","last_clean":"2020-08-07T15:08:11.649190+0000","last_became_active":"2020-08-07T15:06:35.162721+0000","last_became_peered":"2020-08-07T15:06:35.162721+0000","last_unstale":"2020-08-07T15:08:11.649190+0000","last_undegraded":"2020-08-07T15:08:11.649190+0000","last_fullsized":"2020-08-07T15:08:11.649190+0000","mapping_epoch":20,"log_start":"73'8513","ondisk_log_start":"73'8513","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27000832,"num_objects":412,"num_object_clones":0,"num_object_copies":1236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8516,"num_write_kb":63488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"39","reported_epoch":"72","state":"active+clean","last_fresh":"2020-08-07T15:08:06.443313+0000","last_change":"2020-08-07T15:07:36.143434+0000","last_active":"2020-08-07T15:08:06.443313+0000","last_peered":"2020-08-07T15:08:06.443313+0000","last_clean":"2020-08-07T15:08:06.443313+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:08:06.443313+0000","last_undegraded":"2020-08-07T15:08:06.443313+0000","last_fullsized":"2020-08-07T15:08:06.443313+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.143353+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"73'8707","reported_seq":"8771","reported_epoch":"73","state":"active+remapped+backfill_toofull","last_fresh":"2020-08-07T15:08:11.697313+0000","last_change":"2020-08-07T15:07:49.373818+0000","last_active":"2020-08-07T15:08:11.697313+0000","last_peered":"2020-08-07T15:08:11.697313+0000","last_clean":"2020-08-07T15:07:47.323755+0000","last_became_active":"2020-08-07T15:07:49.354652+0000","last_became_peered":"2020-08-07T15:07:49.354652+0000","last_unstale":"2020-08-07T15:08:11.697313+0000","last_undegraded":"2020-08-07T15:08:11.697313+0000","last_fullsized":"2020-08-07T15:08:11.697313+0000","mapping_epoch":60,"log_start":"73'8702","ondisk_log_start":"73'8702","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24510464,"num_objects":374,"num_object_clones":0,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":374,"num_objects_unfound":0,"num_objects_dirty":374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8707,"num_write_kb":64576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,4],"acting":[1,7,3],"avail_no_missing":["1","3","7"],"object_location_counts":[{"shards":"1,3,7","objects":374}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"76","reported_epoch":"71","state":"active+clean","last_fresh":"2020-08-07T15:08:05.437156+0000","last_change":"2020-08-07T15:07:51.357310+0000","last_active":"2020-08-07T15:08:05.437156+0000","last_peered":"2020-08-07T15:08:05.437156+0000","last_clean":"2020-08-07T15:08:05.437156+0000","last_became_active":"2020-08-07T15:07:51.356798+0000","last_became_peered":"2020-08-07T15:07:51.356798+0000","last_unstale":"2020-08-07T15:08:05.437156+0000","last_undegraded":"2020-08-07T15:08:05.437156+0000","last_fullsized":"2020-08-07T15:08:05.437156+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.927824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.927824+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"61","reported_epoch":"72","state":"active+clean","last_fresh":"2020-08-07T15:08:06.477767+0000","last_change":"2020-08-07T15:06:46.199001+0000","last_active":"2020-08-07T15:08:06.477767+0000","last_peered":"2020-08-07T15:08:06.477767+0000","last_clean":"2020-08-07T15:08:06.477767+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:08:06.477767+0000","last_undegraded":"2020-08-07T15:08:06.477767+0000","last_fullsized":"2020-08-07T15:08:06.477767+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.5","version":"73'8622","reported_seq":"8682","reported_epoch":"73","state":"active+clean","last_fresh":"2020-08-07T15:08:11.694175+0000","last_change":"2020-08-07T15:06:35.169307+0000","last_active":"2020-08-07T15:08:11.694175+0000","last_peered":"2020-08-07T15:08:11.694175+0000","last_clean":"2020-08-07T15:08:11.694175+0000","last_became_active":"2020-08-07T15:06:35.168887+0000","last_became_peered":"2020-08-07T15:06:35.168887+0000","last_unstale":"2020-08-07T15:08:11.694175+0000","last_undegraded":"2020-08-07T15:08:11.694175+0000","last_fullsized":"2020-08-07T15:08:11.694175+0000","mapping_epoch":20,"log_start":"73'8619","ondisk_log_start":"73'8619","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24772608,"num_objects":378,"num_object_clones":0,"num_object_copies":1134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8622,"num_write_kb":64000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"50","reported_epoch":"72","state":"active+clean","last_fresh":"2020-08-07T15:08:06.443260+0000","last_change":"2020-08-07T15:07:33.171050+0000","last_active":"2020-08-07T15:08:06.443260+0000","last_peered":"2020-08-07T15:08:06.443260+0000","last_clean":"2020-08-07T15:08:06.443260+0000","last_became_active":"2020-08-07T15:07:33.170597+0000","last_became_peered":"2020-08-07T15:07:33.170597+0000","last_unstale":"2020-08-07T15:08:06.443260+0000","last_undegraded":"2020-08-07T15:08:06.443260+0000","last_fullsized":"2020-08-07T15:08:06.443260+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:18.518049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:18.518049+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"64","reported_epoch":"72","state":"active+clean","last_fresh":"2020-08-07T15:08:06.443412+0000","last_change":"2020-08-07T15:06:23.074962+0000","last_active":"2020-08-07T15:08:06.443412+0000","last_peered":"2020-08-07T15:08:06.443412+0000","last_clean":"2020-08-07T15:08:06.443412+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:08:06.443412+0000","last_undegraded":"2020-08-07T15:08:06.443412+0000","last_fullsized":"2020-08-07T15:08:06.443412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"73'8394","reported_seq":"8455","reported_epoch":"73","state":"active+clean","last_fresh":"2020-08-07T15:08:11.687214+0000","last_change":"2020-08-07T15:06:35.164593+0000","last_active":"2020-08-07T15:08:11.687214+0000","last_peered":"2020-08-07T15:08:11.687214+0000","last_clean":"2020-08-07T15:08:11.687214+0000","last_became_active":"2020-08-07T15:06:35.164108+0000","last_became_peered":"2020-08-07T15:06:35.164108+0000","last_unstale":"2020-08-07T15:08:11.687214+0000","last_undegraded":"2020-08-07T15:08:11.687214+0000","last_fullsized":"2020-08-07T15:08:11.687214+0000","mapping_epoch":20,"log_start":"73'8391","ondisk_log_start":"73'8391","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24969216,"num_objects":381,"num_object_clones":0,"num_object_copies":1143,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8394,"num_write_kb":62400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"103","reported_epoch":"72","state":"active+clean","last_fresh":"2020-08-07T15:08:06.477878+0000","last_change":"2020-08-07T15:07:36.358882+0000","last_active":"2020-08-07T15:08:06.477878+0000","last_peered":"2020-08-07T15:08:06.477878+0000","last_clean":"2020-08-07T15:08:06.477878+0000","last_became_active":"2020-08-07T15:07:33.171449+0000","last_became_peered":"2020-08-07T15:07:33.171449+0000","last_unstale":"2020-08-07T15:08:06.477878+0000","last_undegraded":"2020-08-07T15:08:06.477878+0000","last_fullsized":"2020-08-07T15:08:06.477878+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.358798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.358798+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"67","reported_epoch":"72","state":"active+clean","last_fresh":"2020-08-07T15:08:06.430638+0000","last_change":"2020-08-07T15:07:36.864560+0000","last_active":"2020-08-07T15:08:06.430638+0000","last_peered":"2020-08-07T15:08:06.430638+0000","last_clean":"2020-08-07T15:08:06.430638+0000","last_became_active":"2020-08-07T15:06:23.223489+0000","last_became_peered":"2020-08-07T15:06:23.223489+0000","last_unstale":"2020-08-07T15:08:06.430638+0000","last_undegraded":"2020-08-07T15:08:06.430638+0000","last_fullsized":"2020-08-07T15:08:06.430638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:07:36.864483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.864483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"73'8778","reported_seq":"8841","reported_epoch":"73","state":"active+clean","last_fresh":"2020-08-07T15:08:11.717379+0000","last_change":"2020-08-07T15:08:11.473066+0000","last_active":"2020-08-07T15:08:11.717379+0000","last_peered":"2020-08-07T15:08:11.717379+0000","last_clean":"2020-08-07T15:08:11.717379+0000","last_became_active":"2020-08-07T15:06:35.170037+0000","last_became_peered":"2020-08-07T15:06:35.170037+0000","last_unstale":"2020-08-07T15:08:11.717379+0000","last_undegraded":"2020-08-07T15:08:11.717379+0000","last_fullsized":"2020-08-07T15:08:11.717379+0000","mapping_epoch":20,"log_start":"73'8774","ondisk_log_start":"73'8774","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'8736","last_scrub_stamp":"2020-08-07T15:08:11.472996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:08:11.472996+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24576000,"num_objects":375,"num_object_clones":0,"num_object_copies":1125,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8778,"num_write_kb":65088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"64","reported_epoch":"72","state":"active+clean","last_fresh":"2020-08-07T15:08:06.443294+0000","last_change":"2020-08-07T15:06:23.075958+0000","last_active":"2020-08-07T15:08:06.443294+0000","last_peered":"2020-08-07T15:08:06.443294+0000","last_clean":"2020-08-07T15:08:06.443294+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:08:06.443294+0000","last_undegraded":"2020-08-07T15:08:06.443294+0000","last_fullsized":"2020-08-07T15:08:06.443294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"87","reported_epoch":"72","state":"active+clean","last_fresh":"2020-08-07T15:08:06.477512+0000","last_change":"2020-08-07T15:07:32.461010+0000","last_active":"2020-08-07T15:08:06.477512+0000","last_peered":"2020-08-07T15:08:06.477512+0000","last_clean":"2020-08-07T15:08:06.477512+0000","last_became_active":"2020-08-07T15:07:28.210245+0000","last_became_peered":"2020-08-07T15:07:28.210245+0000","last_unstale":"2020-08-07T15:08:06.477512+0000","last_undegraded":"2020-08-07T15:08:06.477512+0000","last_fullsized":"2020-08-07T15:08:06.477512+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:32.460945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:32.460945+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"73'8788","reported_seq":"8849","reported_epoch":"73","state":"active+clean","last_fresh":"2020-08-07T15:08:11.711343+0000","last_change":"2020-08-07T15:06:35.169888+0000","last_active":"2020-08-07T15:08:11.711343+0000","last_peered":"2020-08-07T15:08:11.711343+0000","last_clean":"2020-08-07T15:08:11.711343+0000","last_became_active":"2020-08-07T15:06:35.169385+0000","last_became_peered":"2020-08-07T15:06:35.169385+0000","last_unstale":"2020-08-07T15:08:11.711343+0000","last_undegraded":"2020-08-07T15:08:11.711343+0000","last_fullsized":"2020-08-07T15:08:11.711343+0000","mapping_epoch":20,"log_start":"73'8785","ondisk_log_start":"73'8785","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25100288,"num_objects":383,"num_object_clones":0,"num_object_copies":1149,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8788,"num_write_kb":65216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"59","reported_epoch":"71","state":"active+clean","last_fresh":"2020-08-07T15:08:05.436788+0000","last_change":"2020-08-07T15:08:02.901198+0000","last_active":"2020-08-07T15:08:05.436788+0000","last_peered":"2020-08-07T15:08:05.436788+0000","last_clean":"2020-08-07T15:08:05.436788+0000","last_became_active":"2020-08-07T15:07:33.156569+0000","last_became_peered":"2020-08-07T15:07:33.156569+0000","last_unstale":"2020-08-07T15:08:05.436788+0000","last_undegraded":"2020-08-07T15:08:05.436788+0000","last_fullsized":"2020-08-07T15:08:05.436788+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:02.901115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:02.901115+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"64","reported_epoch":"72","state":"active+clean","last_fresh":"2020-08-07T15:08:06.477593+0000","last_change":"2020-08-07T15:06:23.224115+0000","last_active":"2020-08-07T15:08:06.477593+0000","last_peered":"2020-08-07T15:08:06.477593+0000","last_clean":"2020-08-07T15:08:06.477593+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:08:06.477593+0000","last_undegraded":"2020-08-07T15:08:06.477593+0000","last_fullsized":"2020-08-07T15:08:06.477593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"73'8353","reported_seq":"10093","reported_epoch":"73","state":"active+clean","last_fresh":"2020-08-07T15:08:11.692269+0000","last_change":"2020-08-07T15:08:01.432640+0000","last_active":"2020-08-07T15:08:11.692269+0000","last_peered":"2020-08-07T15:08:11.692269+0000","last_clean":"2020-08-07T15:08:11.692269+0000","last_became_active":"2020-08-07T15:08:01.414124+0000","last_became_peered":"2020-08-07T15:08:01.414124+0000","last_unstale":"2020-08-07T15:08:11.692269+0000","last_undegraded":"2020-08-07T15:08:11.692269+0000","last_fullsized":"2020-08-07T15:08:11.692269+0000","mapping_epoch":67,"log_start":"73'8349","ondisk_log_start":"73'8349","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:06:34.146731+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23527424,"num_objects":359,"num_object_clones":0,"num_object_copies":1077,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8353,"num_write_kb":61952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":829,"num_bytes_recovered":54329344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"56","reported_epoch":"71","state":"active+clean","last_fresh":"2020-08-07T15:08:05.436806+0000","last_change":"2020-08-07T15:07:33.170029+0000","last_active":"2020-08-07T15:08:05.436806+0000","last_peered":"2020-08-07T15:08:05.436806+0000","last_clean":"2020-08-07T15:08:05.436806+0000","last_became_active":"2020-08-07T15:07:33.169518+0000","last_became_peered":"2020-08-07T15:07:33.169518+0000","last_unstale":"2020-08-07T15:08:05.436806+0000","last_undegraded":"2020-08-07T15:08:05.436806+0000","last_fullsized":"2020-08-07T15:08:05.436806+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:18.518049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:18.518049+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"64","reported_epoch":"72","state":"active+clean","last_fresh":"2020-08-07T15:08:06.477761+0000","last_change":"2020-08-07T15:06:23.225886+0000","last_active":"2020-08-07T15:08:06.477761+0000","last_peered":"2020-08-07T15:08:06.477761+0000","last_clean":"2020-08-07T15:08:06.477761+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:08:06.477761+0000","last_undegraded":"2020-08-07T15:08:06.477761+0000","last_fullsized":"2020-08-07T15:08:06.477761+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"73'8748","reported_seq":"8811","reported_epoch":"73","state":"active+clean","last_fresh":"2020-08-07T15:08:11.694794+0000","last_change":"2020-08-07T15:07:38.065908+0000","last_active":"2020-08-07T15:08:11.694794+0000","last_peered":"2020-08-07T15:08:11.694794+0000","last_clean":"2020-08-07T15:08:11.694794+0000","last_became_active":"2020-08-07T15:06:35.169127+0000","last_became_peered":"2020-08-07T15:06:35.169127+0000","last_unstale":"2020-08-07T15:08:11.694794+0000","last_undegraded":"2020-08-07T15:08:11.694794+0000","last_fullsized":"2020-08-07T15:08:11.694794+0000","mapping_epoch":20,"log_start":"73'8745","ondisk_log_start":"73'8745","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"54'5784","last_scrub_stamp":"2020-08-07T15:07:38.065851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:07:38.065851+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24182784,"num_objects":369,"num_object_clones":0,"num_object_copies":1107,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8748,"num_write_kb":64832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"16'2","reported_seq":"36","reported_epoch":"72","state":"active+clean","last_fresh":"2020-08-07T15:08:06.430801+0000","last_change":"2020-08-07T15:07:33.155878+0000","last_active":"2020-08-07T15:08:06.430801+0000","last_peered":"2020-08-07T15:08:06.430801+0000","last_clean":"2020-08-07T15:08:06.430801+0000","last_became_active":"2020-08-07T15:07:33.155139+0000","last_became_peered":"2020-08-07T15:07:33.155139+0000","last_unstale":"2020-08-07T15:08:06.430801+0000","last_undegraded":"2020-08-07T15:08:06.430801+0000","last_fullsized":"2020-08-07T15:08:06.430801+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:18.518049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:18.518049+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"73'8473","reported_seq":"8537","reported_epoch":"73","state":"active+clean","last_fresh":"2020-08-07T15:08:11.713533+0000","last_change":"2020-08-07T15:08:09.808326+0000","last_active":"2020-08-07T15:08:11.713533+0000","last_peered":"2020-08-07T15:08:11.713533+0000","last_clean":"2020-08-07T15:08:11.713533+0000","last_became_active":"2020-08-07T15:06:35.172360+0000","last_became_peered":"2020-08-07T15:06:35.172360+0000","last_unstale":"2020-08-07T15:08:11.713533+0000","last_undegraded":"2020-08-07T15:08:11.713533+0000","last_fullsized":"2020-08-07T15:08:11.713533+0000","mapping_epoch":20,"log_start":"73'8470","ondisk_log_start":"73'8470","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'8195","last_scrub_stamp":"2020-08-07T15:08:09.808242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:08:09.808242+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23920640,"num_objects":365,"num_object_clones":0,"num_object_copies":1095,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8473,"num_write_kb":62848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"67","reported_epoch":"72","state":"active+clean","last_fresh":"2020-08-07T15:08:06.443437+0000","last_change":"2020-08-07T15:07:34.423236+0000","last_active":"2020-08-07T15:08:06.443437+0000","last_peered":"2020-08-07T15:08:06.443437+0000","last_clean":"2020-08-07T15:08:06.443437+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:08:06.443437+0000","last_undegraded":"2020-08-07T15:08:06.443437+0000","last_fullsized":"2020-08-07T15:08:06.443437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:07:34.423115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:07:34.423115+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"46","reported_epoch":"72","state":"active+clean","last_fresh":"2020-08-07T15:08:06.430028+0000","last_change":"2020-08-07T15:07:34.653928+0000","last_active":"2020-08-07T15:08:06.430028+0000","last_peered":"2020-08-07T15:08:06.430028+0000","last_clean":"2020-08-07T15:08:06.430028+0000","last_became_active":"2020-08-07T15:07:33.161761+0000","last_became_peered":"2020-08-07T15:07:33.161761+0000","last_unstale":"2020-08-07T15:08:06.430028+0000","last_undegraded":"2020-08-07T15:08:06.430028+0000","last_fullsized":"2020-08-07T15:08:06.430028+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:34.653843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:34.653843+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"63","reported_epoch":"71","state":"active+clean","last_fresh":"2020-08-07T15:08:05.437075+0000","last_change":"2020-08-07T15:06:23.073702+0000","last_active":"2020-08-07T15:08:05.437075+0000","last_peered":"2020-08-07T15:08:05.437075+0000","last_clean":"2020-08-07T15:08:05.437075+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:08:05.437075+0000","last_undegraded":"2020-08-07T15:08:05.437075+0000","last_fullsized":"2020-08-07T15:08:05.437075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"73'9047","reported_seq":"9111","reported_epoch":"73","state":"active+clean","last_fresh":"2020-08-07T15:08:11.699961+0000","last_change":"2020-08-07T15:08:01.801466+0000","last_active":"2020-08-07T15:08:11.699961+0000","last_peered":"2020-08-07T15:08:11.699961+0000","last_clean":"2020-08-07T15:08:11.699961+0000","last_became_active":"2020-08-07T15:06:46.199201+0000","last_became_peered":"2020-08-07T15:06:46.199201+0000","last_unstale":"2020-08-07T15:08:11.699961+0000","last_undegraded":"2020-08-07T15:08:11.699961+0000","last_fullsized":"2020-08-07T15:08:11.699961+0000","mapping_epoch":26,"log_start":"73'9044","ondisk_log_start":"73'9044","created":20,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"68'8029","last_scrub_stamp":"2020-08-07T15:08:01.801405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:34.146731+0000","last_clean_scrub_stamp":"2020-08-07T15:08:01.801405+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23461888,"num_objects":358,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9047,"num_write_kb":66880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":433766424,"num_objects":6620,"num_object_clones":0,"num_object_copies":19860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":374,"num_objects_unfound":0,"num_objects_dirty":6620,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":139189,"num_write_kb":1036849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1650,"num_bytes_recovered":108134400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1481650176,"data_stored":1481637960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57,"ondisk_log_size":57,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22,"ondisk_log_size":22,"up":33,"acting":33,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542168,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1442852,"kb_used_data":394268,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92928988,"statfs":{"total":96636764160,"available":95159283712,"internally_reserved":1073741824,"allocated":403730432,"data_stored":403234499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":5.7519999999999998,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":1.2090000000000001},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.1539999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":1.4690000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":5.5019999999999998,"5min":5.5019999999999998,"15min":5.5019999999999998},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.76200000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.9790000000000001,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":1.242},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":1.371}]},{"osd":5,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":37.506999999999998,"5min":37.506999999999998,"15min":37.506999999999998},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":831.88099999999997,"5min":831.88099999999997,"15min":831.88099999999997},"last":1.647},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":1.679}]},{"osd":7,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":3.1389999999999998,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":3.048,"5min":3.048,"15min":3.048},"last":1.5249999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1231760,"kb_used_data":183112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93140080,"statfs":{"total":96636764160,"available":95375441920,"internally_reserved":1073741824,"allocated":187506688,"data_stored":187003284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":3.4020000000000001,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.383,"5min":2.383,"15min":2.383},"last":0.84899999999999998}]},{"osd":1,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.9890000000000001},{"interface":"front","average":{"1min":1.387,"5min":1.387,"15min":1.387},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.5939999999999999,"5min":3.5939999999999999,"15min":3.5939999999999999},"last":0.79500000000000004}]},{"osd":2,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.4129999999999998,"5min":2.4129999999999998,"15min":2.4129999999999998},"last":2.2000000000000002},{"interface":"front","average":{"1min":1.272,"5min":1.272,"15min":1.272},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":4.0529999999999999,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":2.7440000000000002}]},{"osd":3,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":3.3580000000000001,"5min":3.3580000000000001,"15min":3.3580000000000001},"last":3.46},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":3.004,"5min":3.004,"15min":3.004},"last":2.8010000000000002}]},{"osd":4,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.663,"15min":1.663},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":5.2729999999999997,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":1.107},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"min":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"max":{"1min":5.2990000000000004,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":0.82399999999999995}]},{"osd":5,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":2.6360000000000001},{"interface":"front","average":{"1min":1.766,"5min":1.766,"15min":1.766},"min":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"max":{"1min":5.327,"5min":5.327,"15min":5.327},"last":1.4790000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":3.3809999999999998,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.143},{"interface":"front","average":{"1min":18.326000000000001,"5min":18.326000000000001,"15min":18.326000000000001},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":266.26999999999998,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.1719999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1225372,"kb_used_data":176596,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93146468,"statfs":{"total":96636764160,"available":95381983232,"internally_reserved":1073741824,"allocated":180834304,"data_stored":180334168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,3,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":34.186,"5min":34.186,"15min":34.186},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":497.04000000000002,"5min":497.04000000000002,"15min":497.04000000000002},"last":1.115},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":4.1980000000000004,"5min":4.1980000000000004,"15min":4.1980000000000004},"last":1.155}]},{"osd":2,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":1.371},{"interface":"front","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":4.0919999999999996,"5min":4.0919999999999996,"15min":4.0919999999999996},"last":3.0369999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":2.1419999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.454,"15min":1.454},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.9889999999999999,"5min":2.9889999999999999,"15min":2.9889999999999999},"last":2.1629999999999998}]},{"osd":4,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":2.0779999999999998},{"interface":"front","average":{"1min":1.2,"5min":1.2,"15min":1.2},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.6299999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":13.007999999999999,"5min":13.007999999999999,"15min":13.007999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":213.63800000000001,"5min":213.63800000000001,"15min":213.63800000000001},"last":2.1160000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":1.736}]},{"osd":6,"last update":"Fri Aug 7 15:07:22 2020","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.552,"15min":1.552},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":3.234,"5min":3.234,"15min":3.234},"last":3.0089999999999999},{"interface":"front","average":{"1min":1.361,"5min":1.361,"15min":1.361},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":2.4079999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.615,"5min":1.615,"15min":1.615},"min":{"1min":1.046,"5min":1.046,"15min":1.046},"max":{"1min":3.2810000000000001,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":3.0739999999999998},{"interface":"front","average":{"1min":1.48,"5min":1.48,"15min":1.48},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.8759999999999999,"5min":2.8759999999999999,"15min":2.8759999999999999},"last":2.1909999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1198380,"kb_used_data":149796,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93173460,"statfs":{"total":96636764160,"available":95409623040,"internally_reserved":1073741824,"allocated":153391104,"data_stored":152891218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":3.2080000000000002},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.2440000000000002,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":1.0880000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.53},{"interface":"front","average":{"1min":32.740000000000002,"5min":32.740000000000002,"15min":32.740000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":606.18499999999995,"5min":606.18499999999995,"15min":606.18499999999995},"last":0.67900000000000005}]},{"osd":3,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":2.7290000000000001},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":2.887}]},{"osd":4,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.6949999999999998,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":2.4060000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":0.74199999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":2.0179999999999998}]},{"osd":6,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":8.6140000000000008,"5min":8.6140000000000008,"15min":8.6140000000000008},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":124.738,"5min":124.738,"15min":124.738},"last":1.458},{"interface":"front","average":{"1min":18.652999999999999,"5min":18.652999999999999,"15min":18.652999999999999},"min":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"max":{"1min":294.94,"5min":294.94,"15min":294.94},"last":1.244}]},{"osd":7,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":6.468,"5min":6.468,"15min":6.468},"last":1.125},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.883,"5min":2.883,"15min":2.883},"last":1.1060000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1151784,"kb_used_data":103136,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93220056,"statfs":{"total":96636764160,"available":95457337344,"internally_reserved":1073741824,"allocated":105611264,"data_stored":105114646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.117,"15min":1.117},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":3.2240000000000002,"5min":3.2240000000000002,"15min":3.2240000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.999}]},{"osd":1,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":3.1320000000000001,"5min":3.1320000000000001,"15min":3.1320000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":3.3069999999999999,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.80600000000000005}]},{"osd":3,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.3759999999999999,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":1.556,"5min":1.556,"15min":1.556},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":8.2210000000000001,"5min":8.2210000000000001,"15min":8.2210000000000001},"last":1.3819999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.643,"5min":1.643,"15min":1.643},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":3.2749999999999999,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":1.038}]},{"osd":5,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":0.84899999999999998},{"interface":"front","average":{"1min":25.524999999999999,"5min":25.524999999999999,"15min":25.524999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":415.40699999999998,"5min":415.40699999999998,"15min":415.40699999999998},"last":1.069}]},{"osd":6,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.339},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":1.179}]},{"osd":7,"last update":"Fri Aug 7 15:07:16 2020","interfaces":[{"interface":"back","average":{"1min":1.742,"5min":1.742,"15min":1.742},"min":{"1min":1.093,"5min":1.093,"15min":1.093},"max":{"1min":4.056,"5min":4.056,"15min":4.056},"last":35.548999999999999},{"interface":"front","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":4.4539999999999997,"5min":4.4539999999999997,"15min":4.4539999999999997},"last":35.646999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1274032,"kb_used_data":225320,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93097808,"statfs":{"total":96636764160,"available":95332155392,"internally_reserved":1073741824,"allocated":230727680,"data_stored":230223472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":2.8220000000000001,"5min":2.8220000000000001,"15min":2.8220000000000001},"last":8.6020000000000003},{"interface":"front","average":{"1min":1.637,"5min":1.637,"15min":1.637},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":13.067,"5min":13.067,"15min":13.067},"last":4.9119999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.899,"5min":1.899,"15min":1.899},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":6.6829999999999998,"5min":6.6829999999999998,"15min":6.6829999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":2.012,"5min":2.012,"15min":2.012},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":11.936,"5min":11.936,"15min":11.936},"last":14.349}]},{"osd":2,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.589,"5min":3.589,"15min":3.589},"last":4.9580000000000002},{"interface":"front","average":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":5.5890000000000004,"5min":5.5890000000000004,"15min":5.5890000000000004},"last":0.72399999999999998}]},{"osd":4,"last update":"Fri Aug 7 15:07:19 2020","interfaces":[{"interface":"back","average":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":4.556,"5min":4.556,"15min":4.556},"last":8.6419999999999995},{"interface":"front","average":{"1min":32.100000000000001,"5min":32.100000000000001,"15min":32.100000000000001},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":677.88599999999997,"5min":677.88599999999997,"15min":677.88599999999997},"last":5.0670000000000002}]},{"osd":5,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":3.6600000000000001,"5min":3.6600000000000001,"15min":3.6600000000000001},"last":5.0049999999999999},{"interface":"front","average":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"min":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":5.4329999999999998,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":5.226}]},{"osd":6,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":3.621,"5min":3.621,"15min":3.621},"last":5.0389999999999997},{"interface":"front","average":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":5.6189999999999998,"5min":5.6189999999999998,"15min":5.6189999999999998},"last":1.0369999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":5.734,"5min":5.734,"15min":5.734},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":4.8540000000000001,"5min":4.8540000000000001,"15min":4.8540000000000001},"last":8.6790000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1276516,"kb_used_data":227740,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93095324,"statfs":{"total":96636764160,"available":95329611776,"internally_reserved":1073741824,"allocated":233205760,"data_stored":232698284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.181,"15min":1.181},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":4.9480000000000004,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.80200000000000005}]},{"osd":1,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":4.9829999999999997,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":0.58199999999999996},{"interface":"front","average":{"1min":1.377,"5min":1.377,"15min":1.377},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":5.0430000000000001,"5min":5.0430000000000001,"15min":5.0430000000000001},"last":1.2949999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.452,"15min":1.452},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":4.415,"5min":4.415,"15min":4.415},"last":1.028},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":4.4340000000000002,"5min":4.4340000000000002,"15min":4.4340000000000002},"last":1.0980000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.508,"15min":1.508},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":5.0099999999999998,"5min":5.0099999999999998,"15min":5.0099999999999998},"last":1.76},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":3.2029999999999998,"5min":3.2029999999999998,"15min":3.2029999999999998},"last":1.8009999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":5.157,"5min":5.157,"15min":5.157},"last":0.999},{"interface":"front","average":{"1min":1.627,"5min":1.627,"15min":1.627},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":5.0979999999999999,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":1.3280000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.79,"5min":1.79,"15min":1.79},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":5.1310000000000002,"5min":5.1310000000000002,"15min":5.1310000000000002},"last":1.3560000000000001},{"interface":"front","average":{"1min":19.109000000000002,"5min":19.109000000000002,"15min":19.109000000000002},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":389.14600000000002,"5min":389.14600000000002,"15min":389.14600000000002},"last":1.1459999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:07:17 2020","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":3.3540000000000001,"5min":3.3540000000000001,"15min":3.3540000000000001},"last":1.173},{"interface":"front","average":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"min":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"max":{"1min":4.4569999999999999,"5min":4.4569999999999999,"15min":4.4569999999999999},"last":1.052}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1217072,"kb_used_data":168488,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93154768,"statfs":{"total":96636764160,"available":95390482432,"internally_reserved":1073741824,"allocated":172531712,"data_stored":172028308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.766,"5min":2.766,"15min":2.766},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.69999999999999996}]},{"osd":1,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.6509999999999998,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":2.746,"5min":2.746,"15min":2.746},"last":0.98199999999999998}]},{"osd":2,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":9.2200000000000006,"5min":9.2200000000000006,"15min":9.2200000000000006},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":168.56899999999999,"5min":168.56899999999999,"15min":168.56899999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":33.131,"5min":33.131,"15min":33.131},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":676.27099999999996,"5min":676.27099999999996,"15min":676.27099999999996},"last":0.86499999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.7669999999999999,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.964,"5min":2.964,"15min":2.964},"last":0.95799999999999996}]},{"osd":4,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":50.893000000000001,"5min":50.893000000000001,"15min":50.893000000000001},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":530.30799999999999,"5min":530.30799999999999,"15min":530.30799999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":3.1080000000000001,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.91800000000000004}]},{"osd":6,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":50.408999999999999,"5min":50.408999999999999,"15min":50.408999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":754.31299999999999,"5min":754.31299999999999,"15min":754.31299999999999},"last":1.05},{"interface":"front","average":{"1min":26.016999999999999,"5min":26.016999999999999,"15min":26.016999999999999},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":517.26700000000005,"5min":517.26700000000005,"15min":517.26700000000005},"last":0.80800000000000005}]},{"osd":7,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"min":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":3.5649999999999999,"5min":3.5649999999999999,"15min":3.5649999999999999},"last":1.1220000000000001}]}]}]}} 2020-08-07T15:08:12.427 INFO:tasks.thrashosds.thrasher:Setting 1.4 to [2, 3, 6, 4, 0, 7] 2020-08-07T15:08:12.427 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '1.4', '2', '3', '6', '4', '0', '7'] 2020-08-07T15:08:12.428 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pg-upmap-items 1.4 2 3 6 4 0 7 2020-08-07T15:08:12.450 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:12.448+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:12.543 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:12.541+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:12.643 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:12.639+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:12.696 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:12.696 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:12.711 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:08:12.746 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:12.743+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:12.849 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:12.847+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:12.942 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:12.942 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:12.954 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:12.951+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:12.955 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:08:13.075 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:13.073+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:13.155 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:13.154+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:13.176 INFO:teuthology.orchestra.run.smithi074.stderr:set 1.4 pg_upmap_items mapping to [2->3,6->4,0->7] 2020-08-07T15:08:13.176 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:08:13.208 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:13.209 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:13.222 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:08:13.275 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:13.271+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:13.351 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:13.350+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:13.465 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:13.455+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:13.465 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:13.466 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:13.478 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:08:13.567 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:13.558+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:13.666 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:13.662+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:13.740 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:13.740 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:13.758 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:08:13.802 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:13.798+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:13.809 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:08:13.885 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:13.868+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:13.974 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:13.970+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:14.006 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:14.007 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:14.019 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:08:14.049 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:08:14.087 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:14.075+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:14.099 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:08:14.137 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:08:14.170 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:08:14.182 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:14.180+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:14.202 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:14.203 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:14.210 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:08:14.217 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:08:14.246 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:08:14.293 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:08:14.313 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:14.292+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:14.319 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:08:14.320 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:08:14.327 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:08:14.327 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:08:14.328 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:08:14.328 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:08:14.342 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:08:14.342 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:08:14.343 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:08:14.395 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:14.393+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:14.417 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:08:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:145143 3.a 3:52d594e1:::benchmark_data_smithi074_35447_object15534:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:08:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.382992+0000", 2020-08-07T15:08:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.016675987, 2020-08-07T15:08:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016704554, 2020-08-07T15:08:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 145143 2020-08-07T15:08:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.382992+0000", 2020-08-07T15:08:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:14.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.382992+0000", 2020-08-07T15:08:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9050000000000001e-05 2020-08-07T15:08:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.383021+0000", 2020-08-07T15:08:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6400000000000002e-06 2020-08-07T15:08:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.383026+0000", 2020-08-07T15:08:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2157999999999999e-05 2020-08-07T15:08:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.383039+0000", 2020-08-07T15:08:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2269999999999998e-06 2020-08-07T15:08:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.383045+0000", 2020-08-07T15:08:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00063113100000000005 2020-08-07T15:08:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.383676+0000", 2020-08-07T15:08:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012576199999999999 2020-08-07T15:08:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.383802+0000", 2020-08-07T15:08:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4464999999999999e-05 2020-08-07T15:08:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.383886+0000", 2020-08-07T15:08:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024218899999999999 2020-08-07T15:08:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.386308+0000", 2020-08-07T15:08:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1399999999999996e-07 2020-08-07T15:08:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.386309+0000", 2020-08-07T15:08:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7834000000000002e-05 2020-08-07T15:08:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.386367+0000", 2020-08-07T15:08:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001011351 2020-08-07T15:08:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.387378+0000", 2020-08-07T15:08:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043863219999999998 2020-08-07T15:08:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:145148 3.a 3:51815525:::benchmark_data_smithi074_35447_object15539:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:08:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.386281+0000", 2020-08-07T15:08:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.013387125999999999, 2020-08-07T15:08:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013557626999999999, 2020-08-07T15:08:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 145148 2020-08-07T15:08:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.386281+0000", 2020-08-07T15:08:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.386281+0000", 2020-08-07T15:08:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1368000000000002e-05 2020-08-07T15:08:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.386312+0000", 2020-08-07T15:08:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0870000000000003e-06 2020-08-07T15:08:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:14.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.386318+0000", 2020-08-07T15:08:14.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2345e-05 2020-08-07T15:08:14.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:14.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.386330+0000", 2020-08-07T15:08:14.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9179999999999999e-06 2020-08-07T15:08:14.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.386336+0000", 2020-08-07T15:08:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9147999999999997e-05 2020-08-07T15:08:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.386395+0000", 2020-08-07T15:08:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016076 2020-08-07T15:08:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.386556+0000", 2020-08-07T15:08:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016536599999999999 2020-08-07T15:08:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.386722+0000", 2020-08-07T15:08:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011294409999999999 2020-08-07T15:08:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.387851+0000", 2020-08-07T15:08:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4300000000000001e-07 2020-08-07T15:08:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.387852+0000", 2020-08-07T15:08:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0229000000000002e-05 2020-08-07T15:08:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.387912+0000", 2020-08-07T15:08:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011038945999999999 2020-08-07T15:08:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.398951+0000", 2020-08-07T15:08:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012670551 2020-08-07T15:08:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:145171 3.0 e75/20 3:00e40ba4:::benchmark_data_smithi074_35447_object15562:head v 75'9120, mlcod=75'9118)", 2020-08-07T15:08:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.396395+0000", 2020-08-07T15:08:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.003272437, 2020-08-07T15:08:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035729009999999999, 2020-08-07T15:08:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.396395+0000", 2020-08-07T15:08:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.396395+0000", 2020-08-07T15:08:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8866999999999997e-05 2020-08-07T15:08:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.396454+0000", 2020-08-07T15:08:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8189999999999992e-06 2020-08-07T15:08:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.396462+0000", 2020-08-07T15:08:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2898999999999998e-05 2020-08-07T15:08:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.396495+0000", 2020-08-07T15:08:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6479999999999997e-06 2020-08-07T15:08:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.396502+0000", 2020-08-07T15:08:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6467000000000001e-05 2020-08-07T15:08:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.396548+0000", 2020-08-07T15:08:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5624000000000001e-05 2020-08-07T15:08:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.396574+0000", 2020-08-07T15:08:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000178324 2020-08-07T15:08:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:145175 3.a 3:5ae4d117:::benchmark_data_smithi074_35447_object15566:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:08:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.397141+0000", 2020-08-07T15:08:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0025268529999999999, 2020-08-07T15:08:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002902675, 2020-08-07T15:08:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:08:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 145175 2020-08-07T15:08:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.397141+0000", 2020-08-07T15:08:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.397141+0000", 2020-08-07T15:08:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7005999999999999e-05 2020-08-07T15:08:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.397158+0000", 2020-08-07T15:08:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2130000000000002e-06 2020-08-07T15:08:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.397162+0000", 2020-08-07T15:08:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9999999999999996e-06 2020-08-07T15:08:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.397170+0000", 2020-08-07T15:08:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2290000000000002e-06 2020-08-07T15:08:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.397175+0000", 2020-08-07T15:08:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8056999999999998e-05 2020-08-07T15:08:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.397213+0000", 2020-08-07T15:08:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014452399999999999 2020-08-07T15:08:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.397358+0000", 2020-08-07T15:08:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.153e-05 2020-08-07T15:08:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.397439+0000", 2020-08-07T15:08:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000298559 2020-08-07T15:08:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:145174 3.2 e75/20 3:448a4098:::benchmark_data_smithi074_35447_object15565:head v 75'9138, mlcod=75'9135)", 2020-08-07T15:08:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.397684+0000", 2020-08-07T15:08:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0019836150000000002, 2020-08-07T15:08:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024499600000000002, 2020-08-07T15:08:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.397684+0000", 2020-08-07T15:08:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:14.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:14.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.397684+0000", 2020-08-07T15:08:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2524000000000001e-05 2020-08-07T15:08:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.397727+0000", 2020-08-07T15:08:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0349999999999998e-06 2020-08-07T15:08:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.397733+0000", 2020-08-07T15:08:14.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2006000000000003e-05 2020-08-07T15:08:14.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:14.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.397785+0000", 2020-08-07T15:08:14.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.989e-06 2020-08-07T15:08:14.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.397790+0000", 2020-08-07T15:08:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3259000000000001e-05 2020-08-07T15:08:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.397823+0000", 2020-08-07T15:08:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.154e-05 2020-08-07T15:08:14.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:14.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.397844+0000", 2020-08-07T15:08:14.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016035299999999999 2020-08-07T15:08:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:145180 3.0 e75/20 3:0890399e:::benchmark_data_smithi074_35447_object15571:head v 75'9121, mlcod=75'9118)", 2020-08-07T15:08:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.399609+0000", 2020-08-07T15:08:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.8727000000000002e-05, 2020-08-07T15:08:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000594226, 2020-08-07T15:08:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.399609+0000", 2020-08-07T15:08:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.399609+0000", 2020-08-07T15:08:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5345e-05 2020-08-07T15:08:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.399654+0000", 2020-08-07T15:08:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3329999999999996e-06 2020-08-07T15:08:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.399660+0000", 2020-08-07T15:08:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7694e-05 2020-08-07T15:08:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.399717+0000", 2020-08-07T15:08:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1280000000000001e-06 2020-08-07T15:08:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.399722+0000", 2020-08-07T15:08:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8554999999999999e-05 2020-08-07T15:08:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.399760+0000", 2020-08-07T15:08:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2767000000000001e-05 2020-08-07T15:08:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.399783+0000", 2020-08-07T15:08:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017382199999999999 2020-08-07T15:08:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:08:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 6 2020-08-07T15:08:14.522 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:14.522 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:08:14.528 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:14.527+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:14.632 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:14.629+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:14.707 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:08:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:08:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:08:14.711 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:14.716 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:08:14.732 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:14.731+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:14.839 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:14.838+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:14.881 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:14.882 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:08:14.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:08:14.882 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:08:14.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.891 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:17824 3.d e27/20 3:bd4e0935:::benchmark_data_smithi074_35447_object2227:head v 27'1168, mlcod=27'1163)", 2020-08-07T15:08:14.891 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:08:14.891 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.727769593999994, 2020-08-07T15:08:14.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25249224199999998, 2020-08-07T15:08:14.892 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:14.892 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:14.892 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:14.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:14.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:08:14.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:14.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:14.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:08:14.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9348000000000003e-05 2020-08-07T15:08:14.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:14.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143490+0000", 2020-08-07T15:08:14.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7800000000000003e-06 2020-08-07T15:08:14.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:14.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143496+0000", 2020-08-07T15:08:14.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25095783399999999 2020-08-07T15:08:14.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:14.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394454+0000", 2020-08-07T15:08:14.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6550000000000001e-06 2020-08-07T15:08:14.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394460+0000", 2020-08-07T15:08:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8814999999999999e-05 2020-08-07T15:08:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394509+0000", 2020-08-07T15:08:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3482999999999999e-05 2020-08-07T15:08:14.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:14.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394532+0000", 2020-08-07T15:08:14.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013496199999999999 2020-08-07T15:08:14.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.395882+0000", 2020-08-07T15:08:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0707000000000001e-05 2020-08-07T15:08:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.395932+0000", 2020-08-07T15:08:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25249224199999998 2020-08-07T15:08:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:18478 3.a 3:524412cd:::benchmark_data_smithi074_35447_object2309:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:08:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.592837+0000", 2020-08-07T15:08:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.278372515000001, 2020-08-07T15:08:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21005600699999999, 2020-08-07T15:08:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 18478 2020-08-07T15:08:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592837+0000", 2020-08-07T15:08:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592837+0000", 2020-08-07T15:08:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6718999999999998e-05 2020-08-07T15:08:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592884+0000", 2020-08-07T15:08:14.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2749999999999997e-06 2020-08-07T15:08:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592888+0000", 2020-08-07T15:08:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0831e-05 2020-08-07T15:08:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592899+0000", 2020-08-07T15:08:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9440000000000002e-06 2020-08-07T15:08:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.592905+0000", 2020-08-07T15:08:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00088115000000000003 2020-08-07T15:08:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.593786+0000", 2020-08-07T15:08:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1789000000000002e-05 2020-08-07T15:08:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.593838+0000", 2020-08-07T15:08:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011097799999999999 2020-08-07T15:08:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.593949+0000", 2020-08-07T15:08:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00088680400000000004 2020-08-07T15:08:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.594836+0000", 2020-08-07T15:08:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1500000000000005e-07 2020-08-07T15:08:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.594836+0000", 2020-08-07T15:08:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9746000000000002e-05 2020-08-07T15:08:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.594876+0000", 2020-08-07T15:08:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045589070000000001 2020-08-07T15:08:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.599435+0000", 2020-08-07T15:08:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.203361244 2020-08-07T15:08:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802796+0000", 2020-08-07T15:08:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.178e-06 2020-08-07T15:08:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802797+0000", 2020-08-07T15:08:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5927000000000005e-05 2020-08-07T15:08:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:14.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.802893+0000", 2020-08-07T15:08:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21005600699999999 2020-08-07T15:08:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:33488 3.c e30/20 3:3d1929c7:::benchmark_data_smithi074_35447_object4185:head v 30'2120, mlcod=30'2110)", 2020-08-07T15:08:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:58.023101+0000", 2020-08-07T15:08:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 76.848108826000001, 2020-08-07T15:08:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22118623300000001, 2020-08-07T15:08:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023101+0000", 2020-08-07T15:08:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023101+0000", 2020-08-07T15:08:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8794000000000001e-05 2020-08-07T15:08:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023150+0000", 2020-08-07T15:08:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.545e-06 2020-08-07T15:08:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.023156+0000", 2020-08-07T15:08:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.218926757 2020-08-07T15:08:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242083+0000", 2020-08-07T15:08:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1200000000000004e-06 2020-08-07T15:08:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242087+0000", 2020-08-07T15:08:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9561999999999998e-05 2020-08-07T15:08:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242127+0000", 2020-08-07T15:08:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3655999999999998e-05 2020-08-07T15:08:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.242150+0000", 2020-08-07T15:08:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002089273 2020-08-07T15:08:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.244240+0000", 2020-08-07T15:08:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7525999999999999e-05 2020-08-07T15:08:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.244287+0000", 2020-08-07T15:08:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.22118623300000001 2020-08-07T15:08:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:42104 3.d e32/20 3:bcc35439:::benchmark_data_smithi074_35447_object5262:head v 32'2864, mlcod=32'2858)", 2020-08-07T15:08:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:08:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 71.628184297000004, 2020-08-07T15:08:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28296418899999998, 2020-08-07T15:08:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:08:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:08:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5550000000000002e-05 2020-08-07T15:08:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243081+0000", 2020-08-07T15:08:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8139999999999997e-06 2020-08-07T15:08:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243087+0000", 2020-08-07T15:08:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27968446800000002 2020-08-07T15:08:15.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:15.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522771+0000", 2020-08-07T15:08:15.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9459999999999997e-06 2020-08-07T15:08:15.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522776+0000", 2020-08-07T15:08:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0778000000000006e-05 2020-08-07T15:08:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:15.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522847+0000", 2020-08-07T15:08:15.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0622e-05 2020-08-07T15:08:15.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522868+0000", 2020-08-07T15:08:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030627580000000001 2020-08-07T15:08:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.525930+0000", 2020-08-07T15:08:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9252999999999997e-05 2020-08-07T15:08:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.525990+0000", 2020-08-07T15:08:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28296418899999998 2020-08-07T15:08:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:15.020 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:48144 3.0 e36/20 3:0cec1d53:::benchmark_data_smithi074_35447_object6017:head v 36'3128, mlcod=36'3120)", 2020-08-07T15:08:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:08.897123+0000", 2020-08-07T15:08:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.974086587000002, 2020-08-07T15:08:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25221023399999998, 2020-08-07T15:08:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897123+0000", 2020-08-07T15:08:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897123+0000", 2020-08-07T15:08:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0655999999999997e-05 2020-08-07T15:08:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897174+0000", 2020-08-07T15:08:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6769999999999998e-06 2020-08-07T15:08:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897181+0000", 2020-08-07T15:08:15.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25090537899999998 2020-08-07T15:08:15.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:15.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148086+0000", 2020-08-07T15:08:15.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5520000000000004e-06 2020-08-07T15:08:15.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:15.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148092+0000", 2020-08-07T15:08:15.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6391999999999999e-05 2020-08-07T15:08:15.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:15.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148138+0000", 2020-08-07T15:08:15.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8124999999999999e-05 2020-08-07T15:08:15.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:15.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148166+0000", 2020-08-07T15:08:15.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011152009999999999 2020-08-07T15:08:15.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:15.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.149281+0000", 2020-08-07T15:08:15.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2252e-05 2020-08-07T15:08:15.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:15.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.149333+0000", 2020-08-07T15:08:15.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25221023399999998 2020-08-07T15:08:15.039 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.039 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:15.039 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.040 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:48776 3.2 e36/20 3:47029331:::benchmark_data_smithi074_35447_object6096:head v 36'2888, mlcod=36'2881)", 2020-08-07T15:08:15.040 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:08:15.040 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.466689771999995, 2020-08-07T15:08:15.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27006165399999998, 2020-08-07T15:08:15.041 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:15.041 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:15.041 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:15.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:15.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:08:15.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:15.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:15.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:08:15.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8454000000000001e-05 2020-08-07T15:08:15.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:15.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404569+0000", 2020-08-07T15:08:15.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2349999999999996e-06 2020-08-07T15:08:15.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:15.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404575+0000", 2020-08-07T15:08:15.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26921690999999998 2020-08-07T15:08:15.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:15.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673792+0000", 2020-08-07T15:08:15.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3330000000000002e-06 2020-08-07T15:08:15.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:15.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673798+0000", 2020-08-07T15:08:15.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1013000000000001e-05 2020-08-07T15:08:15.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:15.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673849+0000", 2020-08-07T15:08:15.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7044999999999999e-05 2020-08-07T15:08:15.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:15.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673876+0000", 2020-08-07T15:08:15.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00065173100000000001 2020-08-07T15:08:15.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:15.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.674528+0000", 2020-08-07T15:08:15.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3933e-05 2020-08-07T15:08:15.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:15.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.674582+0000", 2020-08-07T15:08:15.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27006165399999998 2020-08-07T15:08:15.055 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.055 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:15.055 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.056 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:49208 3.2 e36/20 3:4d72b982:::benchmark_data_smithi074_35447_object6150:head v 36'2904, mlcod=36'2897)", 2020-08-07T15:08:15.056 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:09.692013+0000", 2020-08-07T15:08:15.057 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.179196743999995, 2020-08-07T15:08:15.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.234386975, 2020-08-07T15:08:15.057 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:15.057 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:15.057 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:15.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:15.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.692013+0000", 2020-08-07T15:08:15.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:15.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:15.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.692013+0000", 2020-08-07T15:08:15.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0998000000000002e-05 2020-08-07T15:08:15.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:15.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.692064+0000", 2020-08-07T15:08:15.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0210000000000001e-05 2020-08-07T15:08:15.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:15.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.692074+0000", 2020-08-07T15:08:15.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.233053807 2020-08-07T15:08:15.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:15.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.925128+0000", 2020-08-07T15:08:15.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4879999999999998e-06 2020-08-07T15:08:15.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:15.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.925134+0000", 2020-08-07T15:08:15.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5154999999999997e-05 2020-08-07T15:08:15.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:15.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.925179+0000", 2020-08-07T15:08:15.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2710000000000001e-05 2020-08-07T15:08:15.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:15.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.925201+0000", 2020-08-07T15:08:15.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001143725 2020-08-07T15:08:15.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:15.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.926345+0000", 2020-08-07T15:08:15.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4882000000000001e-05 2020-08-07T15:08:15.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:15.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.926400+0000", 2020-08-07T15:08:15.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.234386975 2020-08-07T15:08:15.069 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.070 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:15.070 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.070 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:77967 3.2 e46/20 3:48f61a4b:::benchmark_data_smithi074_35447_object9745:head v 46'4807, mlcod=46'4798)", 2020-08-07T15:08:15.071 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:28.702144+0000", 2020-08-07T15:08:15.071 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.169065582000002, 2020-08-07T15:08:15.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26780679000000002, 2020-08-07T15:08:15.071 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:15.071 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:15.072 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:15.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:15.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702144+0000", 2020-08-07T15:08:15.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:15.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:15.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702144+0000", 2020-08-07T15:08:15.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5482000000000002e-05 2020-08-07T15:08:15.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:15.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702200+0000", 2020-08-07T15:08:15.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.216e-06 2020-08-07T15:08:15.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:15.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702207+0000", 2020-08-07T15:08:15.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26634085400000002 2020-08-07T15:08:15.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:15.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.968548+0000", 2020-08-07T15:08:15.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2490000000000001e-06 2020-08-07T15:08:15.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:15.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.968553+0000", 2020-08-07T15:08:15.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7738999999999998e-05 2020-08-07T15:08:15.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:15.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.968601+0000", 2020-08-07T15:08:15.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5783999999999999e-05 2020-08-07T15:08:15.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:15.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.968627+0000", 2020-08-07T15:08:15.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001270471 2020-08-07T15:08:15.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:15.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.969897+0000", 2020-08-07T15:08:15.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3995000000000001e-05 2020-08-07T15:08:15.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:15.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.969951+0000", 2020-08-07T15:08:15.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26780679000000002 2020-08-07T15:08:15.117 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.117 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:15.117 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:77968 3.2 e46/20 3:48f61a4b:::benchmark_data_smithi074_35447_object9745:head v 46'4808, mlcod=46'4799)", 2020-08-07T15:08:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:28.702836+0000", 2020-08-07T15:08:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.168373742, 2020-08-07T15:08:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27322339400000001, 2020-08-07T15:08:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702836+0000", 2020-08-07T15:08:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702836+0000", 2020-08-07T15:08:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1822e-05 2020-08-07T15:08:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702888+0000", 2020-08-07T15:08:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4110000000000001e-06 2020-08-07T15:08:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702894+0000", 2020-08-07T15:08:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26750942999999999 2020-08-07T15:08:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970404+0000", 2020-08-07T15:08:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1150000000000002e-06 2020-08-07T15:08:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970409+0000", 2020-08-07T15:08:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2898000000000007e-05 2020-08-07T15:08:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970472+0000", 2020-08-07T15:08:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2311999999999999e-05 2020-08-07T15:08:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970494+0000", 2020-08-07T15:08:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055286800000000002 2020-08-07T15:08:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.976023+0000", 2020-08-07T15:08:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6726e-05 2020-08-07T15:08:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.976059+0000", 2020-08-07T15:08:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27322339400000001 2020-08-07T15:08:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86695 3.a 3:5058ba7c:::benchmark_data_smithi074_35447_object10836:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:08:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:08:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.545882325999997, 2020-08-07T15:08:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31490022699999998, 2020-08-07T15:08:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86695 2020-08-07T15:08:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:08:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:15.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:15.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:08:15.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4560000000000005e-05 2020-08-07T15:08:15.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:15.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325392+0000", 2020-08-07T15:08:15.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3180000000000003e-06 2020-08-07T15:08:15.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:15.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325398+0000", 2020-08-07T15:08:15.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.258e-05 2020-08-07T15:08:15.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:15.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325411+0000", 2020-08-07T15:08:15.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6550000000000001e-06 2020-08-07T15:08:15.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:15.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325417+0000", 2020-08-07T15:08:15.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024282879999999998 2020-08-07T15:08:15.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:15.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.327845+0000", 2020-08-07T15:08:15.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1203000000000001e-05 2020-08-07T15:08:15.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:15.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.327936+0000", 2020-08-07T15:08:15.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000186505 2020-08-07T15:08:15.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:15.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328123+0000", 2020-08-07T15:08:15.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020062890000000001 2020-08-07T15:08:15.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:15.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.330129+0000", 2020-08-07T15:08:15.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121513 2020-08-07T15:08:15.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:15.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.330250+0000", 2020-08-07T15:08:15.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30989746899999998 2020-08-07T15:08:15.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:15.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640148+0000", 2020-08-07T15:08:15.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2139999999999999e-06 2020-08-07T15:08:15.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:15.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640149+0000", 2020-08-07T15:08:15.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8081999999999998e-05 2020-08-07T15:08:15.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:15.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640197+0000", 2020-08-07T15:08:15.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0550999999999999e-05 2020-08-07T15:08:15.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:15.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640228+0000", 2020-08-07T15:08:15.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31490022699999998 2020-08-07T15:08:15.273 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.274 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:15.274 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.282 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86696 3.a 3:5058ba7c:::benchmark_data_smithi074_35447_object10836:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:08:15.282 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:08:15.282 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.545757647000002, 2020-08-07T15:08:15.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.347347556, 2020-08-07T15:08:15.283 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:15.283 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:15.283 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:15.283 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:15.284 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:15.284 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86696 2020-08-07T15:08:15.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.284 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:15.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:15.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:08:15.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:15.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:15.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:08:15.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0436999999999998e-05 2020-08-07T15:08:15.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:15.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325503+0000", 2020-08-07T15:08:15.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4659999999999998e-06 2020-08-07T15:08:15.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:15.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325508+0000", 2020-08-07T15:08:15.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1177e-05 2020-08-07T15:08:15.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:15.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325519+0000", 2020-08-07T15:08:15.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7809999999999996e-06 2020-08-07T15:08:15.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:15.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325524+0000", 2020-08-07T15:08:15.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002989219 2020-08-07T15:08:15.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:15.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328513+0000", 2020-08-07T15:08:15.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0649000000000006e-05 2020-08-07T15:08:15.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:15.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328604+0000", 2020-08-07T15:08:15.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001652 2020-08-07T15:08:15.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:15.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328769+0000", 2020-08-07T15:08:15.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023470639999999998 2020-08-07T15:08:15.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:15.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331116+0000", 2020-08-07T15:08:15.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6099999999999999e-07 2020-08-07T15:08:15.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:15.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331117+0000", 2020-08-07T15:08:15.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9150000000000001e-05 2020-08-07T15:08:15.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:15.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331176+0000", 2020-08-07T15:08:15.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065402581000000001 2020-08-07T15:08:15.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:15.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.396579+0000", 2020-08-07T15:08:15.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27618104599999999 2020-08-07T15:08:15.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:15.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672760+0000", 2020-08-07T15:08:15.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2020-08-07T15:08:15.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:15.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672761+0000", 2020-08-07T15:08:15.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9211000000000001e-05 2020-08-07T15:08:15.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:15.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672800+0000", 2020-08-07T15:08:15.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.347347556 2020-08-07T15:08:15.371 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.371 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:15.372 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.385 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:86695 3.a e52/20 ondisk, result = 0)", 2020-08-07T15:08:15.386 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:08:15.386 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.537800949999998, 2020-08-07T15:08:15.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.306823501, 2020-08-07T15:08:15.387 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:15.387 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:15.387 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:15.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:15.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:08:15.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:15.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:15.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:08:15.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.048e-05 2020-08-07T15:08:15.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:15.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333439+0000", 2020-08-07T15:08:15.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9990000000000002e-06 2020-08-07T15:08:15.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:15.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333443+0000", 2020-08-07T15:08:15.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30664195100000002 2020-08-07T15:08:15.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:15.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640085+0000", 2020-08-07T15:08:15.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8559999999999998e-06 2020-08-07T15:08:15.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:15.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640090+0000", 2020-08-07T15:08:15.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5166000000000001e-05 2020-08-07T15:08:15.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:15.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640135+0000", 2020-08-07T15:08:15.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4160000000000001e-06 2020-08-07T15:08:15.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:15.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640142+0000", 2020-08-07T15:08:15.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0632999999999998e-05 2020-08-07T15:08:15.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:15.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640232+0000", 2020-08-07T15:08:15.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.306823501 2020-08-07T15:08:15.421 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.421 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:15.421 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.422 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:104144 3.a 3:5c46cbda:::benchmark_data_smithi074_35447_object13017:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:08:15.422 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:45.878313+0000", 2020-08-07T15:08:15.422 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.992896891000001, 2020-08-07T15:08:15.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.251430239, 2020-08-07T15:08:15.423 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:15.423 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:15.423 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:15.455 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:15.462 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:15.462 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 104144 2020-08-07T15:08:15.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.463 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:15.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:15.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.878313+0000", 2020-08-07T15:08:15.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:15.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:15.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.878313+0000", 2020-08-07T15:08:15.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5476000000000001e-05 2020-08-07T15:08:15.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:15.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.878338+0000", 2020-08-07T15:08:15.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.241e-06 2020-08-07T15:08:15.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:15.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.878342+0000", 2020-08-07T15:08:15.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1210000000000007e-06 2020-08-07T15:08:15.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:15.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.878350+0000", 2020-08-07T15:08:15.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5470000000000001e-06 2020-08-07T15:08:15.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:15.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.878353+0000", 2020-08-07T15:08:15.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002441992 2020-08-07T15:08:15.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:15.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.880795+0000", 2020-08-07T15:08:15.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4197999999999995e-05 2020-08-07T15:08:15.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.880890+0000", 2020-08-07T15:08:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000163864 2020-08-07T15:08:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.881053+0000", 2020-08-07T15:08:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005483768 2020-08-07T15:08:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886537+0000", 2020-08-07T15:08:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7000000000000003e-07 2020-08-07T15:08:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886538+0000", 2020-08-07T15:08:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6101999999999998e-05 2020-08-07T15:08:15.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886604+0000", 2020-08-07T15:08:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19508814899999999 2020-08-07T15:08:15.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.081692+0000", 2020-08-07T15:08:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047975891999999999 2020-08-07T15:08:15.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.129668+0000", 2020-08-07T15:08:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.079e-06 2020-08-07T15:08:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.129669+0000", 2020-08-07T15:08:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.394e-05 2020-08-07T15:08:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.129743+0000", 2020-08-07T15:08:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.251430239 2020-08-07T15:08:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.521 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:104144 3.a e58/20 ondisk, result = 0)", 2020-08-07T15:08:15.522 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:45.886622+0000", 2020-08-07T15:08:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.984587342000001, 2020-08-07T15:08:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24340332200000001, 2020-08-07T15:08:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:15.523 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886622+0000", 2020-08-07T15:08:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886622+0000", 2020-08-07T15:08:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2391999999999998e-05 2020-08-07T15:08:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886655+0000", 2020-08-07T15:08:15.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6959999999999996e-06 2020-08-07T15:08:15.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:15.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886660+0000", 2020-08-07T15:08:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.240856029 2020-08-07T15:08:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:15.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.127516+0000", 2020-08-07T15:08:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9840000000000001e-06 2020-08-07T15:08:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:15.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.127521+0000", 2020-08-07T15:08:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002130032 2020-08-07T15:08:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:15.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.129651+0000", 2020-08-07T15:08:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6780000000000006e-06 2020-08-07T15:08:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:15.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.129659+0000", 2020-08-07T15:08:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00036651099999999997 2020-08-07T15:08:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.130026+0000", 2020-08-07T15:08:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24340332200000001 2020-08-07T15:08:15.547 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.548 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106232 3.a 3:5599d7e6:::benchmark_data_smithi074_35447_object13278:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:08:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:08:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.5256027, 2020-08-07T15:08:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52578228699999996, 2020-08-07T15:08:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:15.549 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:15.550 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:15.559 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106232 2020-08-07T15:08:15.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.560 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:15.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:15.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:08:15.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:15.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:15.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:08:15.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4214999999999997e-05 2020-08-07T15:08:15.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:15.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345641+0000", 2020-08-07T15:08:15.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9279999999999993e-06 2020-08-07T15:08:15.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:15.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345649+0000", 2020-08-07T15:08:15.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4391e-05 2020-08-07T15:08:15.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:15.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345664+0000", 2020-08-07T15:08:15.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7170000000000002e-06 2020-08-07T15:08:15.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:15.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345670+0000", 2020-08-07T15:08:15.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.459e-05 2020-08-07T15:08:15.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:15.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345695+0000", 2020-08-07T15:08:15.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6070999999999999e-05 2020-08-07T15:08:15.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:15.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345751+0000", 2020-08-07T15:08:15.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010069300000000001 2020-08-07T15:08:15.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:15.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345852+0000", 2020-08-07T15:08:15.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013512299999999999 2020-08-07T15:08:15.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:15.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.347203+0000", 2020-08-07T15:08:15.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0844000000000001e-05 2020-08-07T15:08:15.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:15.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.347274+0000", 2020-08-07T15:08:15.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52399471600000003 2020-08-07T15:08:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871269+0000", 2020-08-07T15:08:15.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.311e-06 2020-08-07T15:08:15.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871270+0000", 2020-08-07T15:08:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3509000000000004e-05 2020-08-07T15:08:15.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871343+0000", 2020-08-07T15:08:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6072000000000002e-05 2020-08-07T15:08:15.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871389+0000", 2020-08-07T15:08:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52578228699999996 2020-08-07T15:08:15.605 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106232 3.a e59/20 ondisk, result = 0)", 2020-08-07T15:08:15.611 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:08:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.522556524999999, 2020-08-07T15:08:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52274359299999995, 2020-08-07T15:08:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:15.612 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:08:15.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:15.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:15.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:08:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5011000000000001e-05 2020-08-07T15:08:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:15.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348668+0000", 2020-08-07T15:08:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1959999999999998e-06 2020-08-07T15:08:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:15.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348671+0000", 2020-08-07T15:08:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52253456499999995 2020-08-07T15:08:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:15.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871205+0000", 2020-08-07T15:08:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0230000000000003e-06 2020-08-07T15:08:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871209+0000", 2020-08-07T15:08:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0708999999999999e-05 2020-08-07T15:08:15.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871250+0000", 2020-08-07T15:08:15.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0158e-05 2020-08-07T15:08:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871260+0000", 2020-08-07T15:08:15.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013693099999999999 2020-08-07T15:08:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871397+0000", 2020-08-07T15:08:15.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52274359299999995 2020-08-07T15:08:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.659 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:127864 3.a 3:58746ca8:::benchmark_data_smithi074_35447_object15982:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e70)", 2020-08-07T15:08:15.678 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:04.819979+0000", 2020-08-07T15:08:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.051230816, 2020-08-07T15:08:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27732920700000002, 2020-08-07T15:08:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:15.679 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 127864 2020-08-07T15:08:15.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.819979+0000", 2020-08-07T15:08:15.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.819979+0000", 2020-08-07T15:08:15.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7188000000000001e-05 2020-08-07T15:08:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820006+0000", 2020-08-07T15:08:15.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0940000000000001e-06 2020-08-07T15:08:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820010+0000", 2020-08-07T15:08:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0869999999999996e-06 2020-08-07T15:08:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820019+0000", 2020-08-07T15:08:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7509999999999999e-06 2020-08-07T15:08:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:15.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820024+0000", 2020-08-07T15:08:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5089e-05 2020-08-07T15:08:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820049+0000", 2020-08-07T15:08:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0068999999999995e-05 2020-08-07T15:08:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820129+0000", 2020-08-07T15:08:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013952100000000001 2020-08-07T15:08:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820269+0000", 2020-08-07T15:08:15.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031575290000000001 2020-08-07T15:08:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.823426+0000", 2020-08-07T15:08:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4900000000000004e-07 2020-08-07T15:08:15.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:15.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.823427+0000", 2020-08-07T15:08:15.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2324999999999998e-05 2020-08-07T15:08:15.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:15.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.823480+0000", 2020-08-07T15:08:15.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.272535737 2020-08-07T15:08:15.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:15.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.096015+0000", 2020-08-07T15:08:15.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012371560000000001 2020-08-07T15:08:15.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:15.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097253+0000", 2020-08-07T15:08:15.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4000000000000001e-07 2020-08-07T15:08:15.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:15.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097253+0000", 2020-08-07T15:08:15.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5071999999999997e-05 2020-08-07T15:08:15.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:15.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097308+0000", 2020-08-07T15:08:15.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27732920700000002 2020-08-07T15:08:15.700 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.701 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:15.701 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.701 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:127864 3.a e70/20 ondisk, result = 0)", 2020-08-07T15:08:15.701 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:04.822639+0000", 2020-08-07T15:08:15.702 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.048571254000001, 2020-08-07T15:08:15.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27467429799999998, 2020-08-07T15:08:15.702 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:15.702 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:15.702 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:15.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:15.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.822639+0000", 2020-08-07T15:08:15.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:15.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:15.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.822639+0000", 2020-08-07T15:08:15.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3278999999999998e-05 2020-08-07T15:08:15.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:15.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.822672+0000", 2020-08-07T15:08:15.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1489999999999999e-06 2020-08-07T15:08:15.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:15.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.822678+0000", 2020-08-07T15:08:15.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.274520551 2020-08-07T15:08:15.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:15.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097199+0000", 2020-08-07T15:08:15.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6940000000000001e-06 2020-08-07T15:08:15.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:15.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097202+0000", 2020-08-07T15:08:15.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3429999999999997e-05 2020-08-07T15:08:15.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:15.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097236+0000", 2020-08-07T15:08:15.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7390000000000005e-06 2020-08-07T15:08:15.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:15.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097244+0000", 2020-08-07T15:08:15.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8455999999999996e-05 2020-08-07T15:08:15.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:15.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097313+0000", 2020-08-07T15:08:15.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27467429799999998 2020-08-07T15:08:15.723 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.723 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:15.723 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.724 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:127871 3.2 e70/20 3:405b2522:::benchmark_data_smithi074_35447_object15983:head v 70'8079, mlcod=70'8071)", 2020-08-07T15:08:15.724 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:04.826119+0000", 2020-08-07T15:08:15.724 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.045090925, 2020-08-07T15:08:15.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25781899600000002, 2020-08-07T15:08:15.725 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:15.725 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:15.725 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:15.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:15.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.826119+0000", 2020-08-07T15:08:15.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:15.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:15.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.826119+0000", 2020-08-07T15:08:15.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1702000000000001e-05 2020-08-07T15:08:15.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:15.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.826171+0000", 2020-08-07T15:08:15.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9230000000000002e-06 2020-08-07T15:08:15.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:15.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.826179+0000", 2020-08-07T15:08:15.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25632726 2020-08-07T15:08:15.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:15.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.082506+0000", 2020-08-07T15:08:15.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8739999999999997e-06 2020-08-07T15:08:15.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:15.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.082513+0000", 2020-08-07T15:08:15.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2123000000000003e-05 2020-08-07T15:08:15.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:15.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.082555+0000", 2020-08-07T15:08:15.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6835e-05 2020-08-07T15:08:15.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:15.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.082582+0000", 2020-08-07T15:08:15.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012973889999999999 2020-08-07T15:08:15.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:15.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.083879+0000", 2020-08-07T15:08:15.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.889e-05 2020-08-07T15:08:15.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:15.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.083938+0000", 2020-08-07T15:08:15.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25781899600000002 2020-08-07T15:08:15.761 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.762 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:15.762 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.762 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:127872 3.2 e70/20 3:405b2522:::benchmark_data_smithi074_35447_object15983:head v 70'8080, mlcod=70'8071)", 2020-08-07T15:08:15.763 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:04.826865+0000", 2020-08-07T15:08:15.763 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.044345158, 2020-08-07T15:08:15.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26030899299999999, 2020-08-07T15:08:15.763 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:15.763 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:15.764 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:15.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:15.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.826865+0000", 2020-08-07T15:08:15.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:15.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:15.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.826865+0000", 2020-08-07T15:08:15.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2008000000000001e-05 2020-08-07T15:08:15.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:15.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.826917+0000", 2020-08-07T15:08:15.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0510000000000002e-06 2020-08-07T15:08:15.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:15.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.826923+0000", 2020-08-07T15:08:15.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25807582499999998 2020-08-07T15:08:15.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:15.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.084999+0000", 2020-08-07T15:08:15.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8619999999999999e-06 2020-08-07T15:08:15.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:15.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.085004+0000", 2020-08-07T15:08:15.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106156 2020-08-07T15:08:15.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:15.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.085111+0000", 2020-08-07T15:08:15.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3410000000000001e-05 2020-08-07T15:08:15.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:15.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.085134+0000", 2020-08-07T15:08:15.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019880459999999998 2020-08-07T15:08:15.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:15.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.087122+0000", 2020-08-07T15:08:15.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1635000000000003e-05 2020-08-07T15:08:15.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:15.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:15.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:15.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.087174+0000", 2020-08-07T15:08:15.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26030899299999999 2020-08-07T15:08:15.796 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.796 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:15.796 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.796 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:15.796 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:15.797 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:15.797 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:08:15.801 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:08:15.803 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Removed 16201 objects 2020-08-07T15:08:15.803 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Clean up completed and total clean up time :8.27163 2020-08-07T15:08:15.804 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:15.800+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:15.902 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:15.900+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:15.933 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:15.935 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:08:15.935 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2020-08-07T15:08:15.935 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:15.949 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:08:16.003 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:16.001+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:16.108 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:16.102+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:16.113 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:16.113 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:08:16.113 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:08:16.114 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:08:16.114 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:16.156 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:08:16.218 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:16.213+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:16.311 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:16.311 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:08:16.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:08:16.312 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:08:16.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.312 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:9976 3.e 3:7fb2e2e5:::benchmark_data_smithi074_35447_object1246:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:08:16.313 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:08:16.313 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 94.068278688000007, 2020-08-07T15:08:16.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28420182999999999, 2020-08-07T15:08:16.313 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:16.313 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:16.314 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:16.314 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:16.314 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:16.314 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 9976 2020-08-07T15:08:16.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.315 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:16.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:16.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:08:16.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:16.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:16.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:08:16.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7113e-05 2020-08-07T15:08:16.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:16.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237255+0000", 2020-08-07T15:08:16.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5869999999999998e-06 2020-08-07T15:08:16.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:16.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237262+0000", 2020-08-07T15:08:16.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4467999999999999e-05 2020-08-07T15:08:16.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:16.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237276+0000", 2020-08-07T15:08:16.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0749999999999997e-06 2020-08-07T15:08:16.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:16.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237281+0000", 2020-08-07T15:08:16.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013795109999999999 2020-08-07T15:08:16.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:16.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238661+0000", 2020-08-07T15:08:16.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100309 2020-08-07T15:08:16.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:16.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238761+0000", 2020-08-07T15:08:16.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000141784 2020-08-07T15:08:16.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:16.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238903+0000", 2020-08-07T15:08:16.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00070626599999999999 2020-08-07T15:08:16.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:16.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239609+0000", 2020-08-07T15:08:16.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3499999999999996e-07 2020-08-07T15:08:16.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:16.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239610+0000", 2020-08-07T15:08:16.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0846000000000006e-05 2020-08-07T15:08:16.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:16.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239680+0000", 2020-08-07T15:08:16.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022037080000000001 2020-08-07T15:08:16.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:16.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.261718+0000", 2020-08-07T15:08:16.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25961784500000001 2020-08-07T15:08:16.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:16.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521335+0000", 2020-08-07T15:08:16.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2839999999999999e-06 2020-08-07T15:08:16.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:16.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521337+0000", 2020-08-07T15:08:16.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3126999999999994e-05 2020-08-07T15:08:16.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:16.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521400+0000", 2020-08-07T15:08:16.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28420182999999999 2020-08-07T15:08:16.377 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.377 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:16.377 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.378 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15576 3.e 3:727724a5:::benchmark_data_smithi074_35447_object1946:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:08:16.378 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:08:16.378 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 89.638841725000006, 2020-08-07T15:08:16.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999, 2020-08-07T15:08:16.379 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:16.379 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:16.379 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:16.379 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:16.379 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:16.379 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15576 2020-08-07T15:08:16.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.380 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:16.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:16.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:08:16.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:16.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:16.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:08:16.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7156e-05 2020-08-07T15:08:16.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:16.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666682+0000", 2020-08-07T15:08:16.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0209999999999999e-06 2020-08-07T15:08:16.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:16.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666687+0000", 2020-08-07T15:08:16.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0730999999999999e-05 2020-08-07T15:08:16.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:16.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666698+0000", 2020-08-07T15:08:16.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2020-08-07T15:08:16.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:16.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666702+0000", 2020-08-07T15:08:16.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023978459999999999 2020-08-07T15:08:16.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:16.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669100+0000", 2020-08-07T15:08:16.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9664000000000001e-05 2020-08-07T15:08:16.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:16.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669190+0000", 2020-08-07T15:08:16.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015957100000000001 2020-08-07T15:08:16.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:16.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669350+0000", 2020-08-07T15:08:16.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016470149999999999 2020-08-07T15:08:16.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:16.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:08:16.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1299999999999999e-07 2020-08-07T15:08:16.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:16.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:08:16.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5269000000000002e-05 2020-08-07T15:08:16.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:16.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.671022+0000", 2020-08-07T15:08:16.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031414400000000001 2020-08-07T15:08:16.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:16.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.674163+0000", 2020-08-07T15:08:16.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45641417400000001 2020-08-07T15:08:16.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:16.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:08:16.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0399999999999995e-07 2020-08-07T15:08:16.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:16.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:08:16.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3343999999999997e-05 2020-08-07T15:08:16.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:16.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130622+0000", 2020-08-07T15:08:16.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999 2020-08-07T15:08:16.440 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.440 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:16.440 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.441 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:17824 3.d 3:bd4e0935:::benchmark_data_smithi074_35447_object2227:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:08:16.441 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.140708+0000", 2020-08-07T15:08:16.441 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.164768801999998, 2020-08-07T15:08:16.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25688457100000001, 2020-08-07T15:08:16.442 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:16.442 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:16.442 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:16.442 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:16.442 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:16.443 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 17824 2020-08-07T15:08:16.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.443 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:16.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:16.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140708+0000", 2020-08-07T15:08:16.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:16.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:16.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140708+0000", 2020-08-07T15:08:16.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0522000000000002e-05 2020-08-07T15:08:16.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:16.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140758+0000", 2020-08-07T15:08:16.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1680000000000001e-06 2020-08-07T15:08:16.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:16.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140765+0000", 2020-08-07T15:08:16.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5791999999999998e-05 2020-08-07T15:08:16.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:16.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140810+0000", 2020-08-07T15:08:16.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2389999999999996e-06 2020-08-07T15:08:16.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:16.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140818+0000", 2020-08-07T15:08:16.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001721544 2020-08-07T15:08:16.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:16.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.142539+0000", 2020-08-07T15:08:16.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020908100000000001 2020-08-07T15:08:16.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:16.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.142748+0000", 2020-08-07T15:08:16.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019000000000000001 2020-08-07T15:08:16.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:08:16.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.142938+0000", 2020-08-07T15:08:16.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001381279 2020-08-07T15:08:16.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:16.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.144319+0000", 2020-08-07T15:08:16.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010975570000000001 2020-08-07T15:08:16.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:16.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.145417+0000", 2020-08-07T15:08:16.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.252064128 2020-08-07T15:08:16.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:16.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397481+0000", 2020-08-07T15:08:16.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003e-06 2020-08-07T15:08:16.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:16.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397482+0000", 2020-08-07T15:08:16.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3081000000000001e-05 2020-08-07T15:08:16.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:16.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397545+0000", 2020-08-07T15:08:16.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7176999999999999e-05 2020-08-07T15:08:16.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:16.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397592+0000", 2020-08-07T15:08:16.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25688457100000001 2020-08-07T15:08:16.504 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.504 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:16.505 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.505 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:35576 3.f 3:fca2de08:::benchmark_data_smithi074_35447_object4446:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-08-07T15:08:16.505 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:59.287747+0000", 2020-08-07T15:08:16.506 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.017729998999997, 2020-08-07T15:08:16.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25861668500000001, 2020-08-07T15:08:16.506 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:16.506 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:16.506 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:16.507 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:16.507 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:16.507 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 35576 2020-08-07T15:08:16.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.507 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:16.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:16.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287747+0000", 2020-08-07T15:08:16.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:16.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:16.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287747+0000", 2020-08-07T15:08:16.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0089000000000001e-05 2020-08-07T15:08:16.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:16.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287797+0000", 2020-08-07T15:08:16.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4219999999999997e-06 2020-08-07T15:08:16.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:16.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287803+0000", 2020-08-07T15:08:16.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4132e-05 2020-08-07T15:08:16.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:16.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287817+0000", 2020-08-07T15:08:16.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1600000000000003e-06 2020-08-07T15:08:16.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:16.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287823+0000", 2020-08-07T15:08:16.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019418230000000001 2020-08-07T15:08:16.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:16.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.289765+0000", 2020-08-07T15:08:16.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110597 2020-08-07T15:08:16.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:16.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.289876+0000", 2020-08-07T15:08:16.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017674800000000001 2020-08-07T15:08:16.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:16.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.290053+0000", 2020-08-07T15:08:16.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052995730000000001 2020-08-07T15:08:16.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:16.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295352+0000", 2020-08-07T15:08:16.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9999999999999997e-07 2020-08-07T15:08:16.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:16.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295353+0000", 2020-08-07T15:08:16.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6803000000000004e-05 2020-08-07T15:08:16.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:16.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295420+0000", 2020-08-07T15:08:16.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015611539999999999 2020-08-07T15:08:16.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:16.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.296981+0000", 2020-08-07T15:08:16.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.249314866 2020-08-07T15:08:16.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:16.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546296+0000", 2020-08-07T15:08:16.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7300000000000005e-07 2020-08-07T15:08:16.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:16.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546296+0000", 2020-08-07T15:08:16.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6845000000000001e-05 2020-08-07T15:08:16.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:16.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546363+0000", 2020-08-07T15:08:16.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25861668500000001 2020-08-07T15:08:16.556 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.556 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:16.556 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.557 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:41431 3.d 3:b5c8a160:::benchmark_data_smithi074_35447_object5178:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:08:16.557 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.791939+0000", 2020-08-07T15:08:16.558 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.513537196000001, 2020-08-07T15:08:16.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28436496, 2020-08-07T15:08:16.558 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:16.558 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:16.558 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:16.559 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:16.559 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:16.559 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 41431 2020-08-07T15:08:16.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.559 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:16.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:16.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791939+0000", 2020-08-07T15:08:16.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:16.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:16.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791939+0000", 2020-08-07T15:08:16.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6304999999999998e-05 2020-08-07T15:08:16.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:16.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791986+0000", 2020-08-07T15:08:16.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6339999999999998e-06 2020-08-07T15:08:16.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:16.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791991+0000", 2020-08-07T15:08:16.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.026e-05 2020-08-07T15:08:16.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:16.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792002+0000", 2020-08-07T15:08:16.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6240000000000001e-06 2020-08-07T15:08:16.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:16.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792006+0000", 2020-08-07T15:08:16.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015600220000000001 2020-08-07T15:08:16.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:16.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.793566+0000", 2020-08-07T15:08:16.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3628999999999994e-05 2020-08-07T15:08:16.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:16.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.793660+0000", 2020-08-07T15:08:16.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000168197 2020-08-07T15:08:16.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:08:16.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.793828+0000", 2020-08-07T15:08:16.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001273251 2020-08-07T15:08:16.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:16.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795101+0000", 2020-08-07T15:08:16.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5899999999999998e-07 2020-08-07T15:08:16.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:16.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795102+0000", 2020-08-07T15:08:16.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4904000000000001e-05 2020-08-07T15:08:16.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:16.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795127+0000", 2020-08-07T15:08:16.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00670237 2020-08-07T15:08:16.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:16.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801829+0000", 2020-08-07T15:08:16.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27442024999999998 2020-08-07T15:08:16.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:16.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076249+0000", 2020-08-07T15:08:16.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999997e-07 2020-08-07T15:08:16.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:16.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076250+0000", 2020-08-07T15:08:16.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4369000000000001e-05 2020-08-07T15:08:16.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:16.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076304+0000", 2020-08-07T15:08:16.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28436496 2020-08-07T15:08:16.620 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.620 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:16.620 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.621 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:41432 3.d 3:b5c8a160:::benchmark_data_smithi074_35447_object5178:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:08:16.621 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:08:16.621 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.513186500000003, 2020-08-07T15:08:16.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.37720701699999998, 2020-08-07T15:08:16.622 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:16.622 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:16.622 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:16.622 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:16.623 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:16.623 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 41432 2020-08-07T15:08:16.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.623 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:16.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:16.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:08:16.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:16.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:16.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:08:16.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4787999999999999e-05 2020-08-07T15:08:16.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:16.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792335+0000", 2020-08-07T15:08:16.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5999999999999997e-06 2020-08-07T15:08:16.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:16.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792341+0000", 2020-08-07T15:08:16.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0998e-05 2020-08-07T15:08:16.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:16.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792352+0000", 2020-08-07T15:08:16.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2989999999999996e-06 2020-08-07T15:08:16.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:16.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792357+0000", 2020-08-07T15:08:16.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001852752 2020-08-07T15:08:16.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:16.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794210+0000", 2020-08-07T15:08:16.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5771000000000002e-05 2020-08-07T15:08:16.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:16.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794305+0000", 2020-08-07T15:08:16.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012776399999999999 2020-08-07T15:08:16.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:08:16.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794433+0000", 2020-08-07T15:08:16.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00069673599999999999 2020-08-07T15:08:16.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:16.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795130+0000", 2020-08-07T15:08:16.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1500000000000001e-07 2020-08-07T15:08:16.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:16.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795130+0000", 2020-08-07T15:08:16.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7319e-05 2020-08-07T15:08:16.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:16.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795167+0000", 2020-08-07T15:08:16.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27910026700000001 2020-08-07T15:08:16.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:16.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074268+0000", 2020-08-07T15:08:16.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.095150660999999997 2020-08-07T15:08:16.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169418+0000", 2020-08-07T15:08:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.423e-06 2020-08-07T15:08:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:16.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169420+0000", 2020-08-07T15:08:16.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7423999999999996e-05 2020-08-07T15:08:16.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:16.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169497+0000", 2020-08-07T15:08:16.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.37720701699999998 2020-08-07T15:08:16.652 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.654 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:16.654 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.655 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:41431 3.d e32/20 ondisk, result = 0)", 2020-08-07T15:08:16.655 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.796229+0000", 2020-08-07T15:08:16.655 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.509247598000002, 2020-08-07T15:08:16.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28007903299999998, 2020-08-07T15:08:16.656 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:16.656 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:16.656 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:16.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:16.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.796229+0000", 2020-08-07T15:08:16.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:16.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:16.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.796229+0000", 2020-08-07T15:08:16.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3272000000000003e-05 2020-08-07T15:08:16.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:16.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.796262+0000", 2020-08-07T15:08:16.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9579999999999998e-06 2020-08-07T15:08:16.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:16.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.796267+0000", 2020-08-07T15:08:16.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27993393599999999 2020-08-07T15:08:16.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:16.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076201+0000", 2020-08-07T15:08:16.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8709999999999999e-06 2020-08-07T15:08:16.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:16.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076205+0000", 2020-08-07T15:08:16.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9128e-05 2020-08-07T15:08:16.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:16.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076234+0000", 2020-08-07T15:08:16.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9800000000000001e-06 2020-08-07T15:08:16.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:16.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076241+0000", 2020-08-07T15:08:16.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6888000000000001e-05 2020-08-07T15:08:16.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:16.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076308+0000", 2020-08-07T15:08:16.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28007903299999998 2020-08-07T15:08:16.666 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.667 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:16.667 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.667 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:41432 3.d e32/20 ondisk, result = 0)", 2020-08-07T15:08:16.668 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.801630+0000", 2020-08-07T15:08:16.668 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.503846697, 2020-08-07T15:08:16.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27264033999999998, 2020-08-07T15:08:16.668 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:16.668 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:16.669 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:16.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:16.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801630+0000", 2020-08-07T15:08:16.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:16.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:16.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801630+0000", 2020-08-07T15:08:16.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6257e-05 2020-08-07T15:08:16.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:16.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801646+0000", 2020-08-07T15:08:16.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5450000000000002e-06 2020-08-07T15:08:16.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:16.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801649+0000", 2020-08-07T15:08:16.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27254806999999998 2020-08-07T15:08:16.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:16.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074197+0000", 2020-08-07T15:08:16.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4079999999999997e-06 2020-08-07T15:08:16.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:16.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074202+0000", 2020-08-07T15:08:16.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5222000000000002e-05 2020-08-07T15:08:16.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:16.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074247+0000", 2020-08-07T15:08:16.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9459999999999993e-06 2020-08-07T15:08:16.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:16.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074257+0000", 2020-08-07T15:08:16.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2892e-05 2020-08-07T15:08:16.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:16.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074270+0000", 2020-08-07T15:08:16.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27264033999999998 2020-08-07T15:08:16.679 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.679 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:16.680 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.680 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:42104 3.d 3:bcc35439:::benchmark_data_smithi074_35447_object5262:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:08:16.680 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:08:16.681 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.066554328999999, 2020-08-07T15:08:16.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28812091400000001, 2020-08-07T15:08:16.681 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:16.681 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:16.681 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:16.682 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:16.682 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:16.682 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42104 2020-08-07T15:08:16.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.682 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:16.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:16.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:08:16.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:16.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:16.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:08:16.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9734000000000002e-05 2020-08-07T15:08:16.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:16.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238972+0000", 2020-08-07T15:08:16.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6910000000000002e-06 2020-08-07T15:08:16.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:16.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238978+0000", 2020-08-07T15:08:16.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1864e-05 2020-08-07T15:08:16.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:16.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238990+0000", 2020-08-07T15:08:16.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.737e-06 2020-08-07T15:08:16.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:16.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238996+0000", 2020-08-07T15:08:16.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00283943 2020-08-07T15:08:16.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:16.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.241836+0000", 2020-08-07T15:08:16.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4712999999999999e-05 2020-08-07T15:08:16.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:16.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.241930+0000", 2020-08-07T15:08:16.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015328499999999999 2020-08-07T15:08:16.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:08:16.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.242084+0000", 2020-08-07T15:08:16.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016104400000000001 2020-08-07T15:08:16.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:16.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243694+0000", 2020-08-07T15:08:16.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1399999999999997e-07 2020-08-07T15:08:16.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:16.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243695+0000", 2020-08-07T15:08:16.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7800999999999998e-05 2020-08-07T15:08:16.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:16.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243753+0000", 2020-08-07T15:08:16.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090405661999999998 2020-08-07T15:08:16.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:16.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.334158+0000", 2020-08-07T15:08:16.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19283929299999999 2020-08-07T15:08:16.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:16.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.526997+0000", 2020-08-07T15:08:16.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.012e-06 2020-08-07T15:08:16.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:16.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.526998+0000", 2020-08-07T15:08:16.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4737999999999998e-05 2020-08-07T15:08:16.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:16.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.527043+0000", 2020-08-07T15:08:16.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28812091400000001 2020-08-07T15:08:16.701 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.701 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:16.701 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.702 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:44120 3.e 3:7e6f7bb9:::benchmark_data_smithi074_35447_object5514:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:08:16.702 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:04.450106+0000", 2020-08-07T15:08:16.702 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 71.855370794999999, 2020-08-07T15:08:16.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26702573699999999, 2020-08-07T15:08:16.703 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:16.704 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:16.704 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:16.704 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:16.704 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:16.704 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 44120 2020-08-07T15:08:16.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.705 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:16.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:16.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450106+0000", 2020-08-07T15:08:16.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:16.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:16.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450106+0000", 2020-08-07T15:08:16.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9926000000000003e-05 2020-08-07T15:08:16.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:16.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450156+0000", 2020-08-07T15:08:16.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4079999999999997e-06 2020-08-07T15:08:16.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:16.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450161+0000", 2020-08-07T15:08:16.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0494e-05 2020-08-07T15:08:16.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:16.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450172+0000", 2020-08-07T15:08:16.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3950000000000003e-06 2020-08-07T15:08:16.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:16.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450176+0000", 2020-08-07T15:08:16.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033213689999999998 2020-08-07T15:08:16.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:16.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.453497+0000", 2020-08-07T15:08:16.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102414 2020-08-07T15:08:16.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:16.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.453600+0000", 2020-08-07T15:08:16.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019776 2020-08-07T15:08:16.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:16.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.453798+0000", 2020-08-07T15:08:16.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001075456 2020-08-07T15:08:16.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:16.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.454873+0000", 2020-08-07T15:08:16.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7999999999999995e-07 2020-08-07T15:08:16.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:16.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.454874+0000", 2020-08-07T15:08:16.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3912000000000002e-05 2020-08-07T15:08:16.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:16.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.454928+0000", 2020-08-07T15:08:16.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058537550000000004 2020-08-07T15:08:16.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:16.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.460781+0000", 2020-08-07T15:08:16.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25630591899999999 2020-08-07T15:08:16.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:16.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.717087+0000", 2020-08-07T15:08:16.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.54e-07 2020-08-07T15:08:16.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:16.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.717088+0000", 2020-08-07T15:08:16.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3594999999999998e-05 2020-08-07T15:08:16.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:16.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.717132+0000", 2020-08-07T15:08:16.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26702573699999999 2020-08-07T15:08:16.724 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.724 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:16.725 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.725 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:49800 3.e 3:7cb4a069:::benchmark_data_smithi074_35447_object6224:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e36)", 2020-08-07T15:08:16.725 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:10.076186+0000", 2020-08-07T15:08:16.726 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.229291056999998, 2020-08-07T15:08:16.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25479637100000002, 2020-08-07T15:08:16.726 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:16.726 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:16.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:16.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:16.727 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:16.728 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 49800 2020-08-07T15:08:16.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.729 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:16.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:16.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076186+0000", 2020-08-07T15:08:16.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:16.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:16.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076186+0000", 2020-08-07T15:08:16.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3579000000000003e-05 2020-08-07T15:08:16.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:16.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076239+0000", 2020-08-07T15:08:16.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0499999999999997e-06 2020-08-07T15:08:16.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:16.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076245+0000", 2020-08-07T15:08:16.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1756999999999999e-05 2020-08-07T15:08:16.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:16.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076257+0000", 2020-08-07T15:08:16.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8220000000000003e-06 2020-08-07T15:08:16.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:16.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076263+0000", 2020-08-07T15:08:16.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9265999999999999e-05 2020-08-07T15:08:16.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:16.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076312+0000", 2020-08-07T15:08:16.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2280000000000004e-05 2020-08-07T15:08:16.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:16.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076404+0000", 2020-08-07T15:08:16.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015721499999999999 2020-08-07T15:08:16.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:16.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076562+0000", 2020-08-07T15:08:16.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041832179999999998 2020-08-07T15:08:16.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:16.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.080745+0000", 2020-08-07T15:08:16.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.091e-06 2020-08-07T15:08:16.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:16.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.080746+0000", 2020-08-07T15:08:16.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3623999999999999e-05 2020-08-07T15:08:16.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:16.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.080809+0000", 2020-08-07T15:08:16.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11945386299999999 2020-08-07T15:08:16.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:16.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.200263+0000", 2020-08-07T15:08:16.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.130675596 2020-08-07T15:08:16.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:16.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.330939+0000", 2020-08-07T15:08:16.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.508e-06 2020-08-07T15:08:16.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:16.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.330940+0000", 2020-08-07T15:08:16.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1501999999999997e-05 2020-08-07T15:08:16.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:16.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.330982+0000", 2020-08-07T15:08:16.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25479637100000002 2020-08-07T15:08:16.757 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.757 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:16.758 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.758 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:86712 3.8 e52/20 3:14925daa:::benchmark_data_smithi074_35447_object10838:head v 52'5488, mlcod=52'5479)", 2020-08-07T15:08:16.762 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.338631+0000", 2020-08-07T15:08:16.762 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.966845231000001, 2020-08-07T15:08:16.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.310229594, 2020-08-07T15:08:16.763 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:16.763 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:16.763 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:16.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:16.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338631+0000", 2020-08-07T15:08:16.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:16.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:16.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338631+0000", 2020-08-07T15:08:16.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6180000000000001e-05 2020-08-07T15:08:16.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:16.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338688+0000", 2020-08-07T15:08:16.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4949999999999996e-06 2020-08-07T15:08:16.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:16.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338694+0000", 2020-08-07T15:08:16.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30438292099999997 2020-08-07T15:08:16.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:16.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643077+0000", 2020-08-07T15:08:16.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9389999999999996e-06 2020-08-07T15:08:16.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:16.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643082+0000", 2020-08-07T15:08:16.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5096999999999999e-05 2020-08-07T15:08:16.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:16.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643127+0000", 2020-08-07T15:08:16.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4323000000000001e-05 2020-08-07T15:08:16.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:16.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643151+0000", 2020-08-07T15:08:16.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055847300000000004 2020-08-07T15:08:16.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:16.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.648736+0000", 2020-08-07T15:08:16.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012490900000000001 2020-08-07T15:08:16.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:16.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.648861+0000", 2020-08-07T15:08:16.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.310229594 2020-08-07T15:08:16.789 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.789 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:16.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.791 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86792 3.f 3:fc5386ee:::benchmark_data_smithi074_35447_object10848:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:08:16.791 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.395402+0000", 2020-08-07T15:08:16.791 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.910075120999998, 2020-08-07T15:08:16.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30259831999999998, 2020-08-07T15:08:16.792 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:16.792 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:16.792 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:16.792 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:16.793 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:16.793 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86792 2020-08-07T15:08:16.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.794 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:16.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:16.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395402+0000", 2020-08-07T15:08:16.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:16.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:16.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395402+0000", 2020-08-07T15:08:16.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2262999999999998e-05 2020-08-07T15:08:16.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:16.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395444+0000", 2020-08-07T15:08:16.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1800000000000004e-06 2020-08-07T15:08:16.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:16.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395449+0000", 2020-08-07T15:08:16.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0738e-05 2020-08-07T15:08:16.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:16.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395460+0000", 2020-08-07T15:08:16.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7219999999999999e-06 2020-08-07T15:08:16.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:16.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395464+0000", 2020-08-07T15:08:16.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0063169089999999999 2020-08-07T15:08:16.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:16.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.401781+0000", 2020-08-07T15:08:16.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010702200000000001 2020-08-07T15:08:16.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:16.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.401888+0000", 2020-08-07T15:08:16.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019171999999999999 2020-08-07T15:08:16.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:16.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.402080+0000", 2020-08-07T15:08:16.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042866839999999998 2020-08-07T15:08:16.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:16.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.406367+0000", 2020-08-07T15:08:16.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000001e-07 2020-08-07T15:08:16.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:16.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.406367+0000", 2020-08-07T15:08:16.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4720000000000004e-05 2020-08-07T15:08:16.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:16.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.406432+0000", 2020-08-07T15:08:16.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050405365000000001 2020-08-07T15:08:16.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:16.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.456837+0000", 2020-08-07T15:08:16.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24111702700000001 2020-08-07T15:08:16.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:16.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.697954+0000", 2020-08-07T15:08:16.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.22e-06 2020-08-07T15:08:16.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:16.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.697955+0000", 2020-08-07T15:08:16.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4336999999999999e-05 2020-08-07T15:08:16.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:16.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.698000+0000", 2020-08-07T15:08:16.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30259831999999998 2020-08-07T15:08:16.826 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.827 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:16.827 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.827 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:97576 3.d e57/35 3:b9e86280:::benchmark_data_smithi074_35447_object12196:head v 57'6304, mlcod=57'6295)", 2020-08-07T15:08:16.828 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:41.740780+0000", 2020-08-07T15:08:16.828 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.564697002999999, 2020-08-07T15:08:16.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27784160200000002, 2020-08-07T15:08:16.830 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:16.830 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:16.830 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:16.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:16.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.740780+0000", 2020-08-07T15:08:16.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:16.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:16.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.740780+0000", 2020-08-07T15:08:16.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0355999999999997e-05 2020-08-07T15:08:16.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:16.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.740830+0000", 2020-08-07T15:08:16.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6680000000000004e-06 2020-08-07T15:08:16.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:16.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.740837+0000", 2020-08-07T15:08:16.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27346419799999999 2020-08-07T15:08:16.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:16.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.014301+0000", 2020-08-07T15:08:16.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.852e-06 2020-08-07T15:08:16.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:16.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.014307+0000", 2020-08-07T15:08:16.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6504e-05 2020-08-07T15:08:16.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:16.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.014363+0000", 2020-08-07T15:08:16.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8759999999999999e-05 2020-08-07T15:08:16.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:16.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.014392+0000", 2020-08-07T15:08:16.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041857459999999997 2020-08-07T15:08:16.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:16.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.018578+0000", 2020-08-07T15:08:16.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3517999999999999e-05 2020-08-07T15:08:16.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:16.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.018621+0000", 2020-08-07T15:08:16.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27784160200000002 2020-08-07T15:08:16.844 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.844 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:16.845 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.845 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:104104 3.8 e58/20 3:11c276ea:::benchmark_data_smithi074_35447_object13012:head v 58'6496, mlcod=58'6487)", 2020-08-07T15:08:16.845 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:08:16.846 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 30.446990978999999, 2020-08-07T15:08:16.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33178497099999998, 2020-08-07T15:08:16.846 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:16.847 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:16.847 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:16.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:16.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:08:16.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:16.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:16.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:08:16.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9961999999999999e-05 2020-08-07T15:08:16.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:16.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858516+0000", 2020-08-07T15:08:16.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3200000000000001e-06 2020-08-07T15:08:16.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:16.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858520+0000", 2020-08-07T15:08:16.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32756790000000002 2020-08-07T15:08:16.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:16.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186088+0000", 2020-08-07T15:08:16.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0260000000000004e-06 2020-08-07T15:08:16.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:16.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186094+0000", 2020-08-07T15:08:16.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7919000000000001e-05 2020-08-07T15:08:16.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:16.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186132+0000", 2020-08-07T15:08:16.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4968e-05 2020-08-07T15:08:16.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:16.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186147+0000", 2020-08-07T15:08:16.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040671520000000001 2020-08-07T15:08:16.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:16.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.190214+0000", 2020-08-07T15:08:16.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6724000000000001e-05 2020-08-07T15:08:16.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:16.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.190271+0000", 2020-08-07T15:08:16.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33178497099999998 2020-08-07T15:08:16.859 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.859 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:16.859 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.860 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106248 3.f 3:f950fd9d:::benchmark_data_smithi074_35447_object13280:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:08:16.861 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:08:16.861 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.945312184999999, 2020-08-07T15:08:16.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002, 2020-08-07T15:08:16.862 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:16.862 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:16.862 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:16.863 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:16.863 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:16.863 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106248 2020-08-07T15:08:16.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.865 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:16.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:16.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:08:16.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:16.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:16.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:08:16.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7184e-05 2020-08-07T15:08:16.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:16.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360222+0000", 2020-08-07T15:08:16.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0906e-05 2020-08-07T15:08:16.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:16.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360233+0000", 2020-08-07T15:08:16.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2972999999999999e-05 2020-08-07T15:08:16.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:16.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360246+0000", 2020-08-07T15:08:16.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0114e-05 2020-08-07T15:08:16.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:16.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360256+0000", 2020-08-07T15:08:16.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1303999999999997e-05 2020-08-07T15:08:16.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:16.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360287+0000", 2020-08-07T15:08:16.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3798000000000001e-05 2020-08-07T15:08:16.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:16.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360351+0000", 2020-08-07T15:08:16.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111308 2020-08-07T15:08:16.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:16.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360462+0000", 2020-08-07T15:08:16.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003480674 2020-08-07T15:08:16.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:16.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:08:16.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0900000000000001e-07 2020-08-07T15:08:16.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:16.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:08:16.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8899999999999994e-05 2020-08-07T15:08:16.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:16.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.364012+0000", 2020-08-07T15:08:16.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35291637199999998 2020-08-07T15:08:16.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:16.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716929+0000", 2020-08-07T15:08:16.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.076244233999999994 2020-08-07T15:08:16.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:16.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793173+0000", 2020-08-07T15:08:16.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6109999999999999e-06 2020-08-07T15:08:16.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:16.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793175+0000", 2020-08-07T15:08:16.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7492999999999999e-05 2020-08-07T15:08:16.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:16.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793242+0000", 2020-08-07T15:08:16.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002 2020-08-07T15:08:16.908 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.908 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:16.908 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.909 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106248 3.f e59/20 ondisk, result = 0)", 2020-08-07T15:08:16.909 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.365323+0000", 2020-08-07T15:08:16.909 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.940153629000001, 2020-08-07T15:08:16.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35160734999999999, 2020-08-07T15:08:16.910 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:16.910 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:16.910 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:16.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:16.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365323+0000", 2020-08-07T15:08:16.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:16.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:16.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365323+0000", 2020-08-07T15:08:16.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4981000000000003e-05 2020-08-07T15:08:16.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:16.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365358+0000", 2020-08-07T15:08:16.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1090000000000003e-06 2020-08-07T15:08:16.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:16.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365364+0000", 2020-08-07T15:08:16.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35151899399999997 2020-08-07T15:08:16.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:16.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716883+0000", 2020-08-07T15:08:16.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0090000000000001e-06 2020-08-07T15:08:16.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:16.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716887+0000", 2020-08-07T15:08:16.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9238999999999999e-05 2020-08-07T15:08:16.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:16.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716916+0000", 2020-08-07T15:08:16.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0390000000000001e-06 2020-08-07T15:08:16.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:16.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716922+0000", 2020-08-07T15:08:16.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9789999999999993e-06 2020-08-07T15:08:16.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:16.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716930+0000", 2020-08-07T15:08:16.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35160734999999999 2020-08-07T15:08:16.921 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.921 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:16.922 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.922 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106248 3.f e59/20 ondisk, result = 0)", 2020-08-07T15:08:16.922 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:08:16.923 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.940048900000001, 2020-08-07T15:08:16.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001, 2020-08-07T15:08:16.924 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:16.924 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:16.924 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:16.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:16.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:08:16.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:16.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:16.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:08:16.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7518999999999999e-05 2020-08-07T15:08:16.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:16.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365445+0000", 2020-08-07T15:08:16.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-06 2020-08-07T15:08:16.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:16.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365449+0000", 2020-08-07T15:08:16.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42764147600000002 2020-08-07T15:08:16.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:16.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793091+0000", 2020-08-07T15:08:16.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6500000000000002e-06 2020-08-07T15:08:16.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:16.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793099+0000", 2020-08-07T15:08:16.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3664e-05 2020-08-07T15:08:16.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:16.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793153+0000", 2020-08-07T15:08:16.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1078e-05 2020-08-07T15:08:16.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:16.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793164+0000", 2020-08-07T15:08:16.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3404000000000003e-05 2020-08-07T15:08:16.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:16.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793248+0000", 2020-08-07T15:08:16.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001 2020-08-07T15:08:16.959 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.960 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:16.960 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:16.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.961 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106344 3.e 3:7abbcafa:::benchmark_data_smithi074_35447_object13292:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:08:16.961 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:48.059687+0000", 2020-08-07T15:08:16.962 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.245789798000001, 2020-08-07T15:08:16.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28541866100000002, 2020-08-07T15:08:16.963 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:16.963 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:16.964 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:16.964 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:16.964 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:16.964 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106344 2020-08-07T15:08:16.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.965 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:16.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:16.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059687+0000", 2020-08-07T15:08:16.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:16.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:16.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059687+0000", 2020-08-07T15:08:16.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5778999999999998e-05 2020-08-07T15:08:16.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:16.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059713+0000", 2020-08-07T15:08:16.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.422e-06 2020-08-07T15:08:16.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:16.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059716+0000", 2020-08-07T15:08:16.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7679999999999997e-06 2020-08-07T15:08:16.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:16.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059725+0000", 2020-08-07T15:08:16.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2020-08-07T15:08:16.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:16.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059729+0000", 2020-08-07T15:08:16.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6205000000000003e-05 2020-08-07T15:08:16.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:16.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059765+0000", 2020-08-07T15:08:16.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3615999999999995e-05 2020-08-07T15:08:16.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:16.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059829+0000", 2020-08-07T15:08:16.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000118903 2020-08-07T15:08:16.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:16.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059948+0000", 2020-08-07T15:08:16.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00070666299999999995 2020-08-07T15:08:16.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:16.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.060654+0000", 2020-08-07T15:08:16.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5199999999999995e-07 2020-08-07T15:08:16.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:16.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.060655+0000", 2020-08-07T15:08:16.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2092999999999998e-05 2020-08-07T15:08:16.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:16.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.060697+0000", 2020-08-07T15:08:16.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00038454500000000003 2020-08-07T15:08:16.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:16.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.061082+0000", 2020-08-07T15:08:16.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.283939936 2020-08-07T15:08:16.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:16.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:16.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.345022+0000", 2020-08-07T15:08:16.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.288e-06 2020-08-07T15:08:16.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:16.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:17.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:17.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.345023+0000", 2020-08-07T15:08:17.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2468999999999999e-05 2020-08-07T15:08:17.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:17.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:17.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:17.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.345105+0000", 2020-08-07T15:08:17.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28541866100000002 2020-08-07T15:08:17.019 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:17.019 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:17.019 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:17.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:17.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:17.019 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:107952 3.d e61/35)", 2020-08-07T15:08:17.020 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:08:17.020 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.979119191999999, 2020-08-07T15:08:17.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001, 2020-08-07T15:08:17.020 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:17.020 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:08:17.021 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:17.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:17.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:17.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:08:17.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:17.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:17.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:17.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:17.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:08:17.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8826000000000003e-05 2020-08-07T15:08:17.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:17.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:17.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:17.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326406+0000", 2020-08-07T15:08:17.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9199999999999998e-06 2020-08-07T15:08:17.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:17.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:17.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:17.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326413+0000", 2020-08-07T15:08:17.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75082853500000002 2020-08-07T15:08:17.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:17.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:17.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:17.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077242+0000", 2020-08-07T15:08:17.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1479999999999999e-06 2020-08-07T15:08:17.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:17.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:17.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:17.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077246+0000", 2020-08-07T15:08:17.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3331999999999998e-05 2020-08-07T15:08:17.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:17.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:17.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:17.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077289+0000", 2020-08-07T15:08:17.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8977000000000001e-05 2020-08-07T15:08:17.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:17.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:17.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:17.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077308+0000", 2020-08-07T15:08:17.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001 2020-08-07T15:08:17.035 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:17.035 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:17.035 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:17.036 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:17.036 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:17.036 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:17.037 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:08:17.041 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:08:17.041 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:17.037+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:17.042 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:08:17.142 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:17.140+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:17.225 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:17.225 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:08:17.226 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:08:17.226 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:17.240 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:08:17.248 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:08:17.248 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:17.244+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:17.271 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:08:17.314 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:08:17.335 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:08:17.343 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:17.343+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:17.373 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:08:17.383 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:17.384 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:17.386 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:08:17.387 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:08:17.414 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:08:17.439 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:08:17.440 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:08:17.440 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:08:17.440 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:08:17.441 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:08:17.441 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:08:17.441 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:08:17.442 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:08:17.442 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:08:17.448 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:17.444+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:17.538 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:17.539 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:17.547 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:17.546+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:17.549 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:08:17.648 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:17.647+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:17.673 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:17.674 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:17.679 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:08:17.751 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:17.750+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:17.800 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:17.800 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:17.809 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:08:17.855 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:17.853+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:17.931 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:17.931 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:17.940 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:08:17.987 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:17.985+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:18.059 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:18.060 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:18.060 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:18.059+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:18.071 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:08:18.168 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:18.166+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:18.197 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:08:18.197 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:08:18.197 INFO:tasks.thrashosds.thrasher:inject_pause on 0 2020-08-07T15:08:18.198 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2020-08-07T15:08:18.198 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2020-08-07T15:08:18.198 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2020-08-07T15:08:18.229 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:18.230 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:18.239 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:08:18.284 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:18.279+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:18.320 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:18.320 INFO:teuthology.orchestra.run.smithi074.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2020-08-07T15:08:18.321 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:18.383 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:18.374+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:18.401 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:18.403 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:18.411 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:08:18.482 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:18.475+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:18.533 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:18.534 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:18.543 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:08:18.584 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:18.576+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:18.701 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:18.702 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:18.702 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:18.680+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:18.710 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:08:18.790 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:18.786+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:18.868 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:18.869 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:18.877 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:08:18.898 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:18.889+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:18.998 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:18.991+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:19.010 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:19.010 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:19.019 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:08:19.042 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:08:19.092 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:19.091+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:19.147 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:19.148 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:19.148 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:08:19.200 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:19.199+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:19.277 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:19.277 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:19.278 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:08:19.318 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:19.318+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:19.406 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:19.403+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:19.436 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:19.437 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:19.447 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:08:19.458 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:08:19.507 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:19.503+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:19.527 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:08:19.570 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:19.571 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:19.581 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:08:19.605 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:19.604+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:19.613 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:08:19.637 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:08:19.661 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:08:19.690 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:08:19.702 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:19.703 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:19.705 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:19.704+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:19.706 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:08:19.709 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:08:19.735 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:08:19.760 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:08:19.787 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:08:19.788 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:08:19.788 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:08:19.788 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:08:19.789 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:08:19.789 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:08:19.789 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:08:19.789 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:08:19.789 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:08:19.807 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:19.805+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:19.871 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:08:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2020-08-07T15:08:19.872 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:19.873 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:08:19.918 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:19.913+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:20.032 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:20.022+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:20.047 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:20.047 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:08:20.047 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:08:20.048 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:08:20.048 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:20.048 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:08:20.196 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:20.137+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:20.211 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:08:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:08:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:08:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.227 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:17824 3.d e27/20 3:bd4e0935:::benchmark_data_smithi074_35447_object2227:head v 27'1168, mlcod=27'1163)", 2020-08-07T15:08:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:08:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 92.062269729999997, 2020-08-07T15:08:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25249224199999998, 2020-08-07T15:08:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:08:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:08:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9348000000000003e-05 2020-08-07T15:08:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143490+0000", 2020-08-07T15:08:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7800000000000003e-06 2020-08-07T15:08:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143496+0000", 2020-08-07T15:08:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25095783399999999 2020-08-07T15:08:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394454+0000", 2020-08-07T15:08:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6550000000000001e-06 2020-08-07T15:08:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394460+0000", 2020-08-07T15:08:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8814999999999999e-05 2020-08-07T15:08:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394509+0000", 2020-08-07T15:08:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3482999999999999e-05 2020-08-07T15:08:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394532+0000", 2020-08-07T15:08:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013496199999999999 2020-08-07T15:08:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.395882+0000", 2020-08-07T15:08:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0707000000000001e-05 2020-08-07T15:08:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.395932+0000", 2020-08-07T15:08:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25249224199999998 2020-08-07T15:08:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:42104 3.d e32/20 3:bcc35439:::benchmark_data_smithi074_35447_object5262:head v 32'2864, mlcod=32'2858)", 2020-08-07T15:08:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:08:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 76.962684433000007, 2020-08-07T15:08:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28296418899999998, 2020-08-07T15:08:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:08:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:08:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5550000000000002e-05 2020-08-07T15:08:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243081+0000", 2020-08-07T15:08:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8139999999999997e-06 2020-08-07T15:08:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243087+0000", 2020-08-07T15:08:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27968446800000002 2020-08-07T15:08:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522771+0000", 2020-08-07T15:08:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9459999999999997e-06 2020-08-07T15:08:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522776+0000", 2020-08-07T15:08:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0778000000000006e-05 2020-08-07T15:08:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522847+0000", 2020-08-07T15:08:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0622e-05 2020-08-07T15:08:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522868+0000", 2020-08-07T15:08:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030627580000000001 2020-08-07T15:08:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.525930+0000", 2020-08-07T15:08:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9252999999999997e-05 2020-08-07T15:08:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.525990+0000", 2020-08-07T15:08:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28296418899999998 2020-08-07T15:08:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:48144 3.0 e36/20 3:0cec1d53:::benchmark_data_smithi074_35447_object6017:head v 36'3128, mlcod=36'3120)", 2020-08-07T15:08:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:08.897123+0000", 2020-08-07T15:08:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 71.308586723000005, 2020-08-07T15:08:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25221023399999998, 2020-08-07T15:08:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897123+0000", 2020-08-07T15:08:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897123+0000", 2020-08-07T15:08:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0655999999999997e-05 2020-08-07T15:08:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897174+0000", 2020-08-07T15:08:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6769999999999998e-06 2020-08-07T15:08:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897181+0000", 2020-08-07T15:08:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25090537899999998 2020-08-07T15:08:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148086+0000", 2020-08-07T15:08:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5520000000000004e-06 2020-08-07T15:08:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148092+0000", 2020-08-07T15:08:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6391999999999999e-05 2020-08-07T15:08:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148138+0000", 2020-08-07T15:08:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8124999999999999e-05 2020-08-07T15:08:20.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:20.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148166+0000", 2020-08-07T15:08:20.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011152009999999999 2020-08-07T15:08:20.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:20.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.149281+0000", 2020-08-07T15:08:20.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2252e-05 2020-08-07T15:08:20.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:20.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.149333+0000", 2020-08-07T15:08:20.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25221023399999998 2020-08-07T15:08:20.286 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.286 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:20.286 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:48776 3.2 e36/20 3:47029331:::benchmark_data_smithi074_35447_object6096:head v 36'2888, mlcod=36'2881)", 2020-08-07T15:08:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:08:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 70.801189907999998, 2020-08-07T15:08:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27006165399999998, 2020-08-07T15:08:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:20.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:08:20.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:20.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:20.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:08:20.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8454000000000001e-05 2020-08-07T15:08:20.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:20.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404569+0000", 2020-08-07T15:08:20.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2349999999999996e-06 2020-08-07T15:08:20.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:20.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404575+0000", 2020-08-07T15:08:20.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26921690999999998 2020-08-07T15:08:20.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:20.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673792+0000", 2020-08-07T15:08:20.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3330000000000002e-06 2020-08-07T15:08:20.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:20.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673798+0000", 2020-08-07T15:08:20.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1013000000000001e-05 2020-08-07T15:08:20.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:20.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673849+0000", 2020-08-07T15:08:20.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7044999999999999e-05 2020-08-07T15:08:20.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:20.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673876+0000", 2020-08-07T15:08:20.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00065173100000000001 2020-08-07T15:08:20.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:20.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.674528+0000", 2020-08-07T15:08:20.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3933e-05 2020-08-07T15:08:20.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:20.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.674582+0000", 2020-08-07T15:08:20.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27006165399999998 2020-08-07T15:08:20.308 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.309 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:20.309 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.309 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:77967 3.2 e46/20 3:48f61a4b:::benchmark_data_smithi074_35447_object9745:head v 46'4807, mlcod=46'4798)", 2020-08-07T15:08:20.310 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:28.702144+0000", 2020-08-07T15:08:20.310 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 51.503565717999997, 2020-08-07T15:08:20.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26780679000000002, 2020-08-07T15:08:20.310 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:20.310 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:20.311 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:20.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:20.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702144+0000", 2020-08-07T15:08:20.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:20.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:20.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702144+0000", 2020-08-07T15:08:20.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5482000000000002e-05 2020-08-07T15:08:20.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:20.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702200+0000", 2020-08-07T15:08:20.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.216e-06 2020-08-07T15:08:20.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:20.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702207+0000", 2020-08-07T15:08:20.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26634085400000002 2020-08-07T15:08:20.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.968548+0000", 2020-08-07T15:08:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2490000000000001e-06 2020-08-07T15:08:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.968553+0000", 2020-08-07T15:08:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7738999999999998e-05 2020-08-07T15:08:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.968601+0000", 2020-08-07T15:08:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5783999999999999e-05 2020-08-07T15:08:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.968627+0000", 2020-08-07T15:08:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001270471 2020-08-07T15:08:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.969897+0000", 2020-08-07T15:08:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3995000000000001e-05 2020-08-07T15:08:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.969951+0000", 2020-08-07T15:08:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26780679000000002 2020-08-07T15:08:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.322 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:77968 3.2 e46/20 3:48f61a4b:::benchmark_data_smithi074_35447_object9745:head v 46'4808, mlcod=46'4799)", 2020-08-07T15:08:20.322 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:28.702836+0000", 2020-08-07T15:08:20.322 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 51.502873878000003, 2020-08-07T15:08:20.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27322339400000001, 2020-08-07T15:08:20.322 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:20.323 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:20.323 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:20.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:20.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702836+0000", 2020-08-07T15:08:20.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:20.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:20.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702836+0000", 2020-08-07T15:08:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1822e-05 2020-08-07T15:08:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702888+0000", 2020-08-07T15:08:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4110000000000001e-06 2020-08-07T15:08:20.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:20.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702894+0000", 2020-08-07T15:08:20.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26750942999999999 2020-08-07T15:08:20.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:20.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970404+0000", 2020-08-07T15:08:20.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1150000000000002e-06 2020-08-07T15:08:20.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:20.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970409+0000", 2020-08-07T15:08:20.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2898000000000007e-05 2020-08-07T15:08:20.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:20.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970472+0000", 2020-08-07T15:08:20.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2311999999999999e-05 2020-08-07T15:08:20.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:20.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970494+0000", 2020-08-07T15:08:20.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055286800000000002 2020-08-07T15:08:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.976023+0000", 2020-08-07T15:08:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6726e-05 2020-08-07T15:08:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:20.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.976059+0000", 2020-08-07T15:08:20.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27322339400000001 2020-08-07T15:08:20.332 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.333 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:20.333 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.333 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86695 3.a 3:5058ba7c:::benchmark_data_smithi074_35447_object10836:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:08:20.334 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:08:20.334 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.880382462, 2020-08-07T15:08:20.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31490022699999998, 2020-08-07T15:08:20.351 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:20.351 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:20.352 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:20.352 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:20.353 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:20.354 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86695 2020-08-07T15:08:20.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.360 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:20.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:20.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:08:20.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:20.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:20.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:08:20.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4560000000000005e-05 2020-08-07T15:08:20.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:20.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325392+0000", 2020-08-07T15:08:20.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3180000000000003e-06 2020-08-07T15:08:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325398+0000", 2020-08-07T15:08:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.258e-05 2020-08-07T15:08:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325411+0000", 2020-08-07T15:08:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6550000000000001e-06 2020-08-07T15:08:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325417+0000", 2020-08-07T15:08:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024282879999999998 2020-08-07T15:08:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.327845+0000", 2020-08-07T15:08:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1203000000000001e-05 2020-08-07T15:08:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.327936+0000", 2020-08-07T15:08:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000186505 2020-08-07T15:08:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328123+0000", 2020-08-07T15:08:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020062890000000001 2020-08-07T15:08:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.330129+0000", 2020-08-07T15:08:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121513 2020-08-07T15:08:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.330250+0000", 2020-08-07T15:08:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30989746899999998 2020-08-07T15:08:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640148+0000", 2020-08-07T15:08:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2139999999999999e-06 2020-08-07T15:08:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640149+0000", 2020-08-07T15:08:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8081999999999998e-05 2020-08-07T15:08:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640197+0000", 2020-08-07T15:08:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0550999999999999e-05 2020-08-07T15:08:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640228+0000", 2020-08-07T15:08:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31490022699999998 2020-08-07T15:08:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86696 3.a 3:5058ba7c:::benchmark_data_smithi074_35447_object10836:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:08:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:08:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.880257782999998, 2020-08-07T15:08:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.347347556, 2020-08-07T15:08:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86696 2020-08-07T15:08:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:08:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:08:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0436999999999998e-05 2020-08-07T15:08:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325503+0000", 2020-08-07T15:08:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4659999999999998e-06 2020-08-07T15:08:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325508+0000", 2020-08-07T15:08:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1177e-05 2020-08-07T15:08:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325519+0000", 2020-08-07T15:08:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7809999999999996e-06 2020-08-07T15:08:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325524+0000", 2020-08-07T15:08:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002989219 2020-08-07T15:08:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328513+0000", 2020-08-07T15:08:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0649000000000006e-05 2020-08-07T15:08:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328604+0000", 2020-08-07T15:08:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001652 2020-08-07T15:08:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328769+0000", 2020-08-07T15:08:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023470639999999998 2020-08-07T15:08:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331116+0000", 2020-08-07T15:08:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6099999999999999e-07 2020-08-07T15:08:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331117+0000", 2020-08-07T15:08:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9150000000000001e-05 2020-08-07T15:08:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331176+0000", 2020-08-07T15:08:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065402581000000001 2020-08-07T15:08:20.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.396579+0000", 2020-08-07T15:08:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27618104599999999 2020-08-07T15:08:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:20.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672760+0000", 2020-08-07T15:08:20.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2020-08-07T15:08:20.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:20.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672761+0000", 2020-08-07T15:08:20.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9211000000000001e-05 2020-08-07T15:08:20.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:20.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672800+0000", 2020-08-07T15:08:20.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.347347556 2020-08-07T15:08:20.484 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.484 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:20.484 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:86695 3.a e52/20 ondisk, result = 0)", 2020-08-07T15:08:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:08:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.872301086, 2020-08-07T15:08:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.306823501, 2020-08-07T15:08:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:08:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:08:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.048e-05 2020-08-07T15:08:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333439+0000", 2020-08-07T15:08:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9990000000000002e-06 2020-08-07T15:08:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:20.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333443+0000", 2020-08-07T15:08:20.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30664195100000002 2020-08-07T15:08:20.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640085+0000", 2020-08-07T15:08:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8559999999999998e-06 2020-08-07T15:08:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:20.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640090+0000", 2020-08-07T15:08:20.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5166000000000001e-05 2020-08-07T15:08:20.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:20.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640135+0000", 2020-08-07T15:08:20.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4160000000000001e-06 2020-08-07T15:08:20.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:20.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640142+0000", 2020-08-07T15:08:20.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0632999999999998e-05 2020-08-07T15:08:20.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:20.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640232+0000", 2020-08-07T15:08:20.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.306823501 2020-08-07T15:08:20.501 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.501 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:20.501 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.502 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:104144 3.a 3:5c46cbda:::benchmark_data_smithi074_35447_object13017:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:08:20.503 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:45.878313+0000", 2020-08-07T15:08:20.504 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.327397027000004, 2020-08-07T15:08:20.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.251430239, 2020-08-07T15:08:20.504 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:20.504 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:20.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:20.505 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:20.505 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:20.505 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 104144 2020-08-07T15:08:20.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.505 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:20.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:20.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.878313+0000", 2020-08-07T15:08:20.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:20.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:20.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.878313+0000", 2020-08-07T15:08:20.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5476000000000001e-05 2020-08-07T15:08:20.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:20.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.878338+0000", 2020-08-07T15:08:20.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.241e-06 2020-08-07T15:08:20.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:20.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.878342+0000", 2020-08-07T15:08:20.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1210000000000007e-06 2020-08-07T15:08:20.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:20.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.878350+0000", 2020-08-07T15:08:20.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5470000000000001e-06 2020-08-07T15:08:20.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:20.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.878353+0000", 2020-08-07T15:08:20.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002441992 2020-08-07T15:08:20.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:20.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.880795+0000", 2020-08-07T15:08:20.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4197999999999995e-05 2020-08-07T15:08:20.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:20.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.880890+0000", 2020-08-07T15:08:20.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000163864 2020-08-07T15:08:20.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:20.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.881053+0000", 2020-08-07T15:08:20.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005483768 2020-08-07T15:08:20.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:20.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886537+0000", 2020-08-07T15:08:20.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7000000000000003e-07 2020-08-07T15:08:20.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:20.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886538+0000", 2020-08-07T15:08:20.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6101999999999998e-05 2020-08-07T15:08:20.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:20.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886604+0000", 2020-08-07T15:08:20.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19508814899999999 2020-08-07T15:08:20.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:20.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.081692+0000", 2020-08-07T15:08:20.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047975891999999999 2020-08-07T15:08:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.129668+0000", 2020-08-07T15:08:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.079e-06 2020-08-07T15:08:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.129669+0000", 2020-08-07T15:08:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.394e-05 2020-08-07T15:08:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.129743+0000", 2020-08-07T15:08:20.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.251430239 2020-08-07T15:08:20.611 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.611 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:20.611 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:104144 3.a e58/20 ondisk, result = 0)", 2020-08-07T15:08:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:45.886622+0000", 2020-08-07T15:08:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.319087478, 2020-08-07T15:08:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24340332200000001, 2020-08-07T15:08:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886622+0000", 2020-08-07T15:08:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886622+0000", 2020-08-07T15:08:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2391999999999998e-05 2020-08-07T15:08:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886655+0000", 2020-08-07T15:08:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6959999999999996e-06 2020-08-07T15:08:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886660+0000", 2020-08-07T15:08:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.240856029 2020-08-07T15:08:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.127516+0000", 2020-08-07T15:08:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9840000000000001e-06 2020-08-07T15:08:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.127521+0000", 2020-08-07T15:08:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002130032 2020-08-07T15:08:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.129651+0000", 2020-08-07T15:08:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6780000000000006e-06 2020-08-07T15:08:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.129659+0000", 2020-08-07T15:08:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00036651099999999997 2020-08-07T15:08:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.130026+0000", 2020-08-07T15:08:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24340332200000001 2020-08-07T15:08:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106232 3.a 3:5599d7e6:::benchmark_data_smithi074_35447_object13278:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:08:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:08:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.860102836000003, 2020-08-07T15:08:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52578228699999996, 2020-08-07T15:08:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106232 2020-08-07T15:08:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:08:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:08:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4214999999999997e-05 2020-08-07T15:08:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345641+0000", 2020-08-07T15:08:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9279999999999993e-06 2020-08-07T15:08:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345649+0000", 2020-08-07T15:08:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4391e-05 2020-08-07T15:08:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345664+0000", 2020-08-07T15:08:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7170000000000002e-06 2020-08-07T15:08:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345670+0000", 2020-08-07T15:08:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.459e-05 2020-08-07T15:08:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345695+0000", 2020-08-07T15:08:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6070999999999999e-05 2020-08-07T15:08:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345751+0000", 2020-08-07T15:08:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010069300000000001 2020-08-07T15:08:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345852+0000", 2020-08-07T15:08:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013512299999999999 2020-08-07T15:08:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.347203+0000", 2020-08-07T15:08:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0844000000000001e-05 2020-08-07T15:08:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.347274+0000", 2020-08-07T15:08:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52399471600000003 2020-08-07T15:08:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871269+0000", 2020-08-07T15:08:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.311e-06 2020-08-07T15:08:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871270+0000", 2020-08-07T15:08:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3509000000000004e-05 2020-08-07T15:08:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871343+0000", 2020-08-07T15:08:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6072000000000002e-05 2020-08-07T15:08:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871389+0000", 2020-08-07T15:08:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52578228699999996 2020-08-07T15:08:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106232 3.a e59/20 ondisk, result = 0)", 2020-08-07T15:08:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:08:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.857056661000001, 2020-08-07T15:08:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52274359299999995, 2020-08-07T15:08:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:08:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:08:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5011000000000001e-05 2020-08-07T15:08:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348668+0000", 2020-08-07T15:08:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1959999999999998e-06 2020-08-07T15:08:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348671+0000", 2020-08-07T15:08:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52253456499999995 2020-08-07T15:08:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871205+0000", 2020-08-07T15:08:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0230000000000003e-06 2020-08-07T15:08:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871209+0000", 2020-08-07T15:08:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0708999999999999e-05 2020-08-07T15:08:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871250+0000", 2020-08-07T15:08:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0158e-05 2020-08-07T15:08:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871260+0000", 2020-08-07T15:08:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013693099999999999 2020-08-07T15:08:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871397+0000", 2020-08-07T15:08:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52274359299999995 2020-08-07T15:08:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:127864 3.a 3:58746ca8:::benchmark_data_smithi074_35447_object15982:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e70)", 2020-08-07T15:08:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:04.819979+0000", 2020-08-07T15:08:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.385730951999999, 2020-08-07T15:08:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27732920700000002, 2020-08-07T15:08:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 127864 2020-08-07T15:08:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.819979+0000", 2020-08-07T15:08:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.819979+0000", 2020-08-07T15:08:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7188000000000001e-05 2020-08-07T15:08:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820006+0000", 2020-08-07T15:08:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0940000000000001e-06 2020-08-07T15:08:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820010+0000", 2020-08-07T15:08:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0869999999999996e-06 2020-08-07T15:08:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820019+0000", 2020-08-07T15:08:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7509999999999999e-06 2020-08-07T15:08:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820024+0000", 2020-08-07T15:08:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5089e-05 2020-08-07T15:08:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820049+0000", 2020-08-07T15:08:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0068999999999995e-05 2020-08-07T15:08:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820129+0000", 2020-08-07T15:08:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013952100000000001 2020-08-07T15:08:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820269+0000", 2020-08-07T15:08:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031575290000000001 2020-08-07T15:08:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.823426+0000", 2020-08-07T15:08:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4900000000000004e-07 2020-08-07T15:08:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.823427+0000", 2020-08-07T15:08:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2324999999999998e-05 2020-08-07T15:08:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.823480+0000", 2020-08-07T15:08:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.272535737 2020-08-07T15:08:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.096015+0000", 2020-08-07T15:08:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012371560000000001 2020-08-07T15:08:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097253+0000", 2020-08-07T15:08:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4000000000000001e-07 2020-08-07T15:08:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097253+0000", 2020-08-07T15:08:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5071999999999997e-05 2020-08-07T15:08:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097308+0000", 2020-08-07T15:08:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27732920700000002 2020-08-07T15:08:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:127864 3.a e70/20 ondisk, result = 0)", 2020-08-07T15:08:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:04.822639+0000", 2020-08-07T15:08:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.38307139, 2020-08-07T15:08:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27467429799999998, 2020-08-07T15:08:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.822639+0000", 2020-08-07T15:08:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.822639+0000", 2020-08-07T15:08:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3278999999999998e-05 2020-08-07T15:08:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.822672+0000", 2020-08-07T15:08:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1489999999999999e-06 2020-08-07T15:08:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.822678+0000", 2020-08-07T15:08:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.274520551 2020-08-07T15:08:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097199+0000", 2020-08-07T15:08:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6940000000000001e-06 2020-08-07T15:08:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097202+0000", 2020-08-07T15:08:20.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3429999999999997e-05 2020-08-07T15:08:20.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:20.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097236+0000", 2020-08-07T15:08:20.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7390000000000005e-06 2020-08-07T15:08:20.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:20.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097244+0000", 2020-08-07T15:08:20.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8455999999999996e-05 2020-08-07T15:08:20.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:20.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097313+0000", 2020-08-07T15:08:20.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27467429799999998 2020-08-07T15:08:20.843 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.844 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:20.844 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.844 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:127871 3.2 e70/20 3:405b2522:::benchmark_data_smithi074_35447_object15983:head v 70'8079, mlcod=70'8071)", 2020-08-07T15:08:20.845 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:04.826119+0000", 2020-08-07T15:08:20.845 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.379591060999999, 2020-08-07T15:08:20.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25781899600000002, 2020-08-07T15:08:20.845 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:20.845 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:20.846 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:20.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:20.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.826119+0000", 2020-08-07T15:08:20.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:20.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:20.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.826119+0000", 2020-08-07T15:08:20.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1702000000000001e-05 2020-08-07T15:08:20.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:20.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.826171+0000", 2020-08-07T15:08:20.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9230000000000002e-06 2020-08-07T15:08:20.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:20.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.826179+0000", 2020-08-07T15:08:20.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25632726 2020-08-07T15:08:20.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:20.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.082506+0000", 2020-08-07T15:08:20.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8739999999999997e-06 2020-08-07T15:08:20.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:20.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.082513+0000", 2020-08-07T15:08:20.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2123000000000003e-05 2020-08-07T15:08:20.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:20.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.082555+0000", 2020-08-07T15:08:20.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6835e-05 2020-08-07T15:08:20.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:20.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.082582+0000", 2020-08-07T15:08:20.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012973889999999999 2020-08-07T15:08:20.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:20.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.083879+0000", 2020-08-07T15:08:20.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.889e-05 2020-08-07T15:08:20.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:20.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.083938+0000", 2020-08-07T15:08:20.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25781899600000002 2020-08-07T15:08:20.876 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.876 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:20.876 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:127872 3.2 e70/20 3:405b2522:::benchmark_data_smithi074_35447_object15983:head v 70'8080, mlcod=70'8071)", 2020-08-07T15:08:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:04.826865+0000", 2020-08-07T15:08:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.378845294, 2020-08-07T15:08:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26030899299999999, 2020-08-07T15:08:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:20.878 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:20.878 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:20.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:20.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.826865+0000", 2020-08-07T15:08:20.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:20.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:20.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.826865+0000", 2020-08-07T15:08:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2008000000000001e-05 2020-08-07T15:08:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.826917+0000", 2020-08-07T15:08:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0510000000000002e-06 2020-08-07T15:08:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.826923+0000", 2020-08-07T15:08:20.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25807582499999998 2020-08-07T15:08:20.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:20.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.084999+0000", 2020-08-07T15:08:20.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8619999999999999e-06 2020-08-07T15:08:20.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:20.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.085004+0000", 2020-08-07T15:08:20.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106156 2020-08-07T15:08:20.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:20.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.085111+0000", 2020-08-07T15:08:20.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3410000000000001e-05 2020-08-07T15:08:20.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:20.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.085134+0000", 2020-08-07T15:08:20.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019880459999999998 2020-08-07T15:08:20.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:20.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.087122+0000", 2020-08-07T15:08:20.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1635000000000003e-05 2020-08-07T15:08:20.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:20.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.087174+0000", 2020-08-07T15:08:20.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26030899299999999 2020-08-07T15:08:20.888 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.888 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:145796 3.a 3:54d72af7:::benchmark_data_smithi074_35447_object16187:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:08:20.890 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.668714+0000", 2020-08-07T15:08:20.890 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.536995932, 2020-08-07T15:08:20.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24066032200000001, 2020-08-07T15:08:20.890 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 145796 2020-08-07T15:08:20.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.902 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:20.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:20.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.668714+0000", 2020-08-07T15:08:20.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:20.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:20.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.668714+0000", 2020-08-07T15:08:20.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100949 2020-08-07T15:08:20.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:20.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.668815+0000", 2020-08-07T15:08:20.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.99e-06 2020-08-07T15:08:20.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:20.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.668822+0000", 2020-08-07T15:08:20.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.349e-05 2020-08-07T15:08:20.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:20.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.668835+0000", 2020-08-07T15:08:20.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.489e-06 2020-08-07T15:08:20.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:20.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.668842+0000", 2020-08-07T15:08:20.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2604999999999997e-05 2020-08-07T15:08:20.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:20.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.668875+0000", 2020-08-07T15:08:20.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015340500000000001 2020-08-07T15:08:20.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:20.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.669028+0000", 2020-08-07T15:08:20.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013534199999999999 2020-08-07T15:08:20.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:20.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.669163+0000", 2020-08-07T15:08:20.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000575414 2020-08-07T15:08:20.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:20.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.669739+0000", 2020-08-07T15:08:20.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2500000000000004e-07 2020-08-07T15:08:20.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:20.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.669740+0000", 2020-08-07T15:08:20.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9196e-05 2020-08-07T15:08:20.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:20.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.669799+0000", 2020-08-07T15:08:20.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016268109999999999 2020-08-07T15:08:20.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:20.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671426+0000", 2020-08-07T15:08:20.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23782094200000001 2020-08-07T15:08:20.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:20.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.909246+0000", 2020-08-07T15:08:20.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6699999999999998e-06 2020-08-07T15:08:20.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:20.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.909249+0000", 2020-08-07T15:08:20.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012519399999999999 2020-08-07T15:08:20.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:20.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.909374+0000", 2020-08-07T15:08:20.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24066032200000001 2020-08-07T15:08:20.959 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.959 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:20.959 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.960 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:145796 3.a e75/20 ondisk, result = 0)", 2020-08-07T15:08:20.961 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.673330+0000", 2020-08-07T15:08:20.961 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.5323798359999996, 2020-08-07T15:08:20.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23606771200000001, 2020-08-07T15:08:20.961 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:20.962 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:20.962 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:20.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:20.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673330+0000", 2020-08-07T15:08:20.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:20.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:20.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673330+0000", 2020-08-07T15:08:20.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2091000000000002e-05 2020-08-07T15:08:20.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:20.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673362+0000", 2020-08-07T15:08:20.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1720000000000004e-06 2020-08-07T15:08:20.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:20.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673368+0000", 2020-08-07T15:08:20.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23566768699999999 2020-08-07T15:08:20.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:20.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.909036+0000", 2020-08-07T15:08:20.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3594000000000001e-05 2020-08-07T15:08:20.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:20.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.909050+0000", 2020-08-07T15:08:20.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000156916 2020-08-07T15:08:20.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:20.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.909207+0000", 2020-08-07T15:08:20.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0985999999999999e-05 2020-08-07T15:08:20.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:20.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.909228+0000", 2020-08-07T15:08:20.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000170266 2020-08-07T15:08:20.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:20.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.909398+0000", 2020-08-07T15:08:20.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23606771200000001 2020-08-07T15:08:20.977 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.978 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:20.978 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.979 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:145807 3.c e75/20 3:39741db6:::benchmark_data_smithi074_35447_object16198:head v 75'9594, mlcod=75'9590)", 2020-08-07T15:08:20.979 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.674086+0000", 2020-08-07T15:08:20.979 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.5316244440000002, 2020-08-07T15:08:20.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40268798500000003, 2020-08-07T15:08:20.980 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:20.980 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:20.980 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:20.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:20.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674086+0000", 2020-08-07T15:08:20.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:20.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:20.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674086+0000", 2020-08-07T15:08:20.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3321000000000001e-05 2020-08-07T15:08:20.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:20.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674129+0000", 2020-08-07T15:08:20.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1919999999999999e-06 2020-08-07T15:08:20.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:20.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674136+0000", 2020-08-07T15:08:20.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40193062899999998 2020-08-07T15:08:20.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:20.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076067+0000", 2020-08-07T15:08:20.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3579e-05 2020-08-07T15:08:20.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:20.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076080+0000", 2020-08-07T15:08:20.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3733999999999998e-05 2020-08-07T15:08:20.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:20.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076124+0000", 2020-08-07T15:08:20.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0145e-05 2020-08-07T15:08:20.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:20.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076154+0000", 2020-08-07T15:08:20.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00055768600000000005 2020-08-07T15:08:20.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:20.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076712+0000", 2020-08-07T15:08:20.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1699000000000007e-05 2020-08-07T15:08:20.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:20.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:20.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:20.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076774+0000", 2020-08-07T15:08:20.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40268798500000003 2020-08-07T15:08:20.990 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.991 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:20.991 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.991 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:20.991 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:20.992 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:20.994 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2020-08-07T15:08:20.995 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2020-08-07T15:08:20.996 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:08:20.996 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:20.993+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:21.099 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:21.097+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:21.128 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:08:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2020-08-07T15:08:21.129 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:21.137 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:08:21.228 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:21.227+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:21.297 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:21.298 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:08:21.298 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:08:21.298 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:08:21.299 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:21.305 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:08:21.306 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:21.302+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:21.415 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:21.406+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:21.465 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:21.466 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:08:21.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:08:21.466 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:08:21.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:9976 3.e 3:7fb2e2e5:::benchmark_data_smithi074_35447_object1246:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:08:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:08:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 99.224465828999996, 2020-08-07T15:08:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28420182999999999, 2020-08-07T15:08:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 9976 2020-08-07T15:08:21.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.469 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:21.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:21.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:08:21.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:21.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:08:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7113e-05 2020-08-07T15:08:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:21.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237255+0000", 2020-08-07T15:08:21.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5869999999999998e-06 2020-08-07T15:08:21.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:21.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237262+0000", 2020-08-07T15:08:21.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4467999999999999e-05 2020-08-07T15:08:21.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:21.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237276+0000", 2020-08-07T15:08:21.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0749999999999997e-06 2020-08-07T15:08:21.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:21.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237281+0000", 2020-08-07T15:08:21.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013795109999999999 2020-08-07T15:08:21.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:21.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238661+0000", 2020-08-07T15:08:21.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100309 2020-08-07T15:08:21.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:21.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238761+0000", 2020-08-07T15:08:21.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000141784 2020-08-07T15:08:21.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:21.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238903+0000", 2020-08-07T15:08:21.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00070626599999999999 2020-08-07T15:08:21.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:21.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239609+0000", 2020-08-07T15:08:21.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3499999999999996e-07 2020-08-07T15:08:21.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:21.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239610+0000", 2020-08-07T15:08:21.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0846000000000006e-05 2020-08-07T15:08:21.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:21.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239680+0000", 2020-08-07T15:08:21.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022037080000000001 2020-08-07T15:08:21.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:21.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.261718+0000", 2020-08-07T15:08:21.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25961784500000001 2020-08-07T15:08:21.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:21.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521335+0000", 2020-08-07T15:08:21.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2839999999999999e-06 2020-08-07T15:08:21.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:21.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521337+0000", 2020-08-07T15:08:21.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3126999999999994e-05 2020-08-07T15:08:21.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:21.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521400+0000", 2020-08-07T15:08:21.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28420182999999999 2020-08-07T15:08:21.569 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:21.569 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:21.569 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:21.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.590 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15576 3.e 3:727724a5:::benchmark_data_smithi074_35447_object1946:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:08:21.590 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:08:21.590 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 94.795028865999996, 2020-08-07T15:08:21.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999, 2020-08-07T15:08:21.591 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:21.591 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:21.591 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:21.591 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:21.591 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:21.592 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15576 2020-08-07T15:08:21.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.592 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:21.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:21.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:08:21.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:21.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:21.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:08:21.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7156e-05 2020-08-07T15:08:21.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:21.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666682+0000", 2020-08-07T15:08:21.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0209999999999999e-06 2020-08-07T15:08:21.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:21.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666687+0000", 2020-08-07T15:08:21.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0730999999999999e-05 2020-08-07T15:08:21.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:21.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666698+0000", 2020-08-07T15:08:21.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2020-08-07T15:08:21.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:21.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666702+0000", 2020-08-07T15:08:21.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023978459999999999 2020-08-07T15:08:21.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:21.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669100+0000", 2020-08-07T15:08:21.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9664000000000001e-05 2020-08-07T15:08:21.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:21.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669190+0000", 2020-08-07T15:08:21.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015957100000000001 2020-08-07T15:08:21.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:21.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669350+0000", 2020-08-07T15:08:21.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016470149999999999 2020-08-07T15:08:21.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:21.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:08:21.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1299999999999999e-07 2020-08-07T15:08:21.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:21.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:08:21.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5269000000000002e-05 2020-08-07T15:08:21.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:21.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.671022+0000", 2020-08-07T15:08:21.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031414400000000001 2020-08-07T15:08:21.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:21.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.674163+0000", 2020-08-07T15:08:21.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45641417400000001 2020-08-07T15:08:21.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:21.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:08:21.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0399999999999995e-07 2020-08-07T15:08:21.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:21.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:08:21.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3343999999999997e-05 2020-08-07T15:08:21.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:21.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130622+0000", 2020-08-07T15:08:21.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999 2020-08-07T15:08:21.610 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:21.610 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:21.611 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:21.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.611 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:17824 3.d 3:bd4e0935:::benchmark_data_smithi074_35447_object2227:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:08:21.611 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.140708+0000", 2020-08-07T15:08:21.612 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.320955943000001, 2020-08-07T15:08:21.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25688457100000001, 2020-08-07T15:08:21.612 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:21.612 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:21.612 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:21.613 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:21.613 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:21.613 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 17824 2020-08-07T15:08:21.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.613 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:21.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:21.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140708+0000", 2020-08-07T15:08:21.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:21.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:21.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140708+0000", 2020-08-07T15:08:21.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0522000000000002e-05 2020-08-07T15:08:21.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:21.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140758+0000", 2020-08-07T15:08:21.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1680000000000001e-06 2020-08-07T15:08:21.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:21.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140765+0000", 2020-08-07T15:08:21.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5791999999999998e-05 2020-08-07T15:08:21.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:21.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140810+0000", 2020-08-07T15:08:21.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2389999999999996e-06 2020-08-07T15:08:21.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:21.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140818+0000", 2020-08-07T15:08:21.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001721544 2020-08-07T15:08:21.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:21.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.142539+0000", 2020-08-07T15:08:21.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020908100000000001 2020-08-07T15:08:21.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:21.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.142748+0000", 2020-08-07T15:08:21.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019000000000000001 2020-08-07T15:08:21.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:08:21.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.142938+0000", 2020-08-07T15:08:21.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001381279 2020-08-07T15:08:21.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:21.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.144319+0000", 2020-08-07T15:08:21.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010975570000000001 2020-08-07T15:08:21.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:21.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.145417+0000", 2020-08-07T15:08:21.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.252064128 2020-08-07T15:08:21.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:21.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397481+0000", 2020-08-07T15:08:21.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003e-06 2020-08-07T15:08:21.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:21.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397482+0000", 2020-08-07T15:08:21.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3081000000000001e-05 2020-08-07T15:08:21.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:21.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397545+0000", 2020-08-07T15:08:21.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7176999999999999e-05 2020-08-07T15:08:21.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:21.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397592+0000", 2020-08-07T15:08:21.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25688457100000001 2020-08-07T15:08:21.632 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:21.632 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:21.632 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:21.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.633 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:35576 3.f 3:fca2de08:::benchmark_data_smithi074_35447_object4446:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-08-07T15:08:21.633 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:59.287747+0000", 2020-08-07T15:08:21.633 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.17391714, 2020-08-07T15:08:21.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25861668500000001, 2020-08-07T15:08:21.633 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:21.634 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:21.634 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:21.634 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:21.634 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:21.635 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 35576 2020-08-07T15:08:21.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.635 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:21.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:21.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287747+0000", 2020-08-07T15:08:21.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:21.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:21.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287747+0000", 2020-08-07T15:08:21.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0089000000000001e-05 2020-08-07T15:08:21.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:21.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287797+0000", 2020-08-07T15:08:21.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4219999999999997e-06 2020-08-07T15:08:21.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:21.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287803+0000", 2020-08-07T15:08:21.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4132e-05 2020-08-07T15:08:21.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:21.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287817+0000", 2020-08-07T15:08:21.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1600000000000003e-06 2020-08-07T15:08:21.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:21.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287823+0000", 2020-08-07T15:08:21.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019418230000000001 2020-08-07T15:08:21.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:21.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.289765+0000", 2020-08-07T15:08:21.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110597 2020-08-07T15:08:21.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:21.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.289876+0000", 2020-08-07T15:08:21.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017674800000000001 2020-08-07T15:08:21.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:21.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.290053+0000", 2020-08-07T15:08:21.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052995730000000001 2020-08-07T15:08:21.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295352+0000", 2020-08-07T15:08:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9999999999999997e-07 2020-08-07T15:08:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:21.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295353+0000", 2020-08-07T15:08:21.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6803000000000004e-05 2020-08-07T15:08:21.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295420+0000", 2020-08-07T15:08:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015611539999999999 2020-08-07T15:08:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:21.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.296981+0000", 2020-08-07T15:08:21.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.249314866 2020-08-07T15:08:21.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:21.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546296+0000", 2020-08-07T15:08:21.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7300000000000005e-07 2020-08-07T15:08:21.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:21.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546296+0000", 2020-08-07T15:08:21.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6845000000000001e-05 2020-08-07T15:08:21.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:21.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546363+0000", 2020-08-07T15:08:21.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25861668500000001 2020-08-07T15:08:21.664 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:21.664 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:21.665 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:21.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.665 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:41431 3.d 3:b5c8a160:::benchmark_data_smithi074_35447_object5178:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:08:21.665 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.791939+0000", 2020-08-07T15:08:21.666 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.669724337000005, 2020-08-07T15:08:21.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28436496, 2020-08-07T15:08:21.666 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:21.667 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:21.668 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:21.668 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:21.668 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:21.668 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 41431 2020-08-07T15:08:21.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.669 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:21.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:21.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791939+0000", 2020-08-07T15:08:21.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:21.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:21.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791939+0000", 2020-08-07T15:08:21.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6304999999999998e-05 2020-08-07T15:08:21.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:21.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791986+0000", 2020-08-07T15:08:21.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6339999999999998e-06 2020-08-07T15:08:21.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:21.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791991+0000", 2020-08-07T15:08:21.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.026e-05 2020-08-07T15:08:21.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:21.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792002+0000", 2020-08-07T15:08:21.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6240000000000001e-06 2020-08-07T15:08:21.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:21.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792006+0000", 2020-08-07T15:08:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015600220000000001 2020-08-07T15:08:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.793566+0000", 2020-08-07T15:08:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3628999999999994e-05 2020-08-07T15:08:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:21.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.793660+0000", 2020-08-07T15:08:21.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000168197 2020-08-07T15:08:21.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:08:21.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.793828+0000", 2020-08-07T15:08:21.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001273251 2020-08-07T15:08:21.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:21.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795101+0000", 2020-08-07T15:08:21.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5899999999999998e-07 2020-08-07T15:08:21.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:21.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795102+0000", 2020-08-07T15:08:21.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4904000000000001e-05 2020-08-07T15:08:21.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:21.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795127+0000", 2020-08-07T15:08:21.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00670237 2020-08-07T15:08:21.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:21.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801829+0000", 2020-08-07T15:08:21.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27442024999999998 2020-08-07T15:08:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076249+0000", 2020-08-07T15:08:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999997e-07 2020-08-07T15:08:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:21.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076250+0000", 2020-08-07T15:08:21.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4369000000000001e-05 2020-08-07T15:08:21.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076304+0000", 2020-08-07T15:08:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28436496 2020-08-07T15:08:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:21.701 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:21.701 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:21.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.711 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:41432 3.d 3:b5c8a160:::benchmark_data_smithi074_35447_object5178:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:08:21.711 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:08:21.712 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.669373641000007, 2020-08-07T15:08:21.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.37720701699999998, 2020-08-07T15:08:21.712 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:21.713 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:21.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:21.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:21.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:21.714 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 41432 2020-08-07T15:08:21.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.714 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:21.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:21.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:08:21.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:21.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:21.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:08:21.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4787999999999999e-05 2020-08-07T15:08:21.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:21.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792335+0000", 2020-08-07T15:08:21.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5999999999999997e-06 2020-08-07T15:08:21.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:21.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792341+0000", 2020-08-07T15:08:21.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0998e-05 2020-08-07T15:08:21.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:21.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792352+0000", 2020-08-07T15:08:21.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2989999999999996e-06 2020-08-07T15:08:21.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:21.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792357+0000", 2020-08-07T15:08:21.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001852752 2020-08-07T15:08:21.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:21.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794210+0000", 2020-08-07T15:08:21.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5771000000000002e-05 2020-08-07T15:08:21.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:21.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794305+0000", 2020-08-07T15:08:21.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012776399999999999 2020-08-07T15:08:21.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:08:21.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794433+0000", 2020-08-07T15:08:21.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00069673599999999999 2020-08-07T15:08:21.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:21.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795130+0000", 2020-08-07T15:08:21.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1500000000000001e-07 2020-08-07T15:08:21.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:21.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795130+0000", 2020-08-07T15:08:21.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7319e-05 2020-08-07T15:08:21.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:21.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795167+0000", 2020-08-07T15:08:21.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27910026700000001 2020-08-07T15:08:21.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:21.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074268+0000", 2020-08-07T15:08:21.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.095150660999999997 2020-08-07T15:08:21.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:21.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169418+0000", 2020-08-07T15:08:21.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.423e-06 2020-08-07T15:08:21.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:21.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169420+0000", 2020-08-07T15:08:21.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7423999999999996e-05 2020-08-07T15:08:21.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:21.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169497+0000", 2020-08-07T15:08:21.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.37720701699999998 2020-08-07T15:08:21.756 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:21.756 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:21.757 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:21.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.757 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:41431 3.d e32/20 ondisk, result = 0)", 2020-08-07T15:08:21.757 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.796229+0000", 2020-08-07T15:08:21.758 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.665434739000005, 2020-08-07T15:08:21.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28007903299999998, 2020-08-07T15:08:21.758 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:21.759 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:21.759 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:21.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:21.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.796229+0000", 2020-08-07T15:08:21.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:21.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:21.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.796229+0000", 2020-08-07T15:08:21.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3272000000000003e-05 2020-08-07T15:08:21.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:21.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.796262+0000", 2020-08-07T15:08:21.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9579999999999998e-06 2020-08-07T15:08:21.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:21.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.796267+0000", 2020-08-07T15:08:21.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27993393599999999 2020-08-07T15:08:21.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:21.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076201+0000", 2020-08-07T15:08:21.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8709999999999999e-06 2020-08-07T15:08:21.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:21.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076205+0000", 2020-08-07T15:08:21.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9128e-05 2020-08-07T15:08:21.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:21.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076234+0000", 2020-08-07T15:08:21.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9800000000000001e-06 2020-08-07T15:08:21.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:21.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076241+0000", 2020-08-07T15:08:21.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6888000000000001e-05 2020-08-07T15:08:21.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:21.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076308+0000", 2020-08-07T15:08:21.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28007903299999998 2020-08-07T15:08:21.770 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:21.770 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:21.770 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:21.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.771 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:41432 3.d e32/20 ondisk, result = 0)", 2020-08-07T15:08:21.771 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.801630+0000", 2020-08-07T15:08:21.771 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.660033838000004, 2020-08-07T15:08:21.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27264033999999998, 2020-08-07T15:08:21.771 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:21.771 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:21.772 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:21.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:21.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801630+0000", 2020-08-07T15:08:21.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:21.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:21.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801630+0000", 2020-08-07T15:08:21.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6257e-05 2020-08-07T15:08:21.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:21.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801646+0000", 2020-08-07T15:08:21.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5450000000000002e-06 2020-08-07T15:08:21.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:21.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801649+0000", 2020-08-07T15:08:21.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27254806999999998 2020-08-07T15:08:21.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:21.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074197+0000", 2020-08-07T15:08:21.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4079999999999997e-06 2020-08-07T15:08:21.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:21.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074202+0000", 2020-08-07T15:08:21.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5222000000000002e-05 2020-08-07T15:08:21.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:21.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074247+0000", 2020-08-07T15:08:21.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9459999999999993e-06 2020-08-07T15:08:21.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:21.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074257+0000", 2020-08-07T15:08:21.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2892e-05 2020-08-07T15:08:21.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:21.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074270+0000", 2020-08-07T15:08:21.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27264033999999998 2020-08-07T15:08:21.782 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:21.782 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:21.782 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:21.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.783 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:42104 3.d 3:bcc35439:::benchmark_data_smithi074_35447_object5262:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:08:21.783 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:08:21.783 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.222741470000003, 2020-08-07T15:08:21.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28812091400000001, 2020-08-07T15:08:21.783 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:21.784 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:21.784 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:21.784 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:21.798 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:21.798 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42104 2020-08-07T15:08:21.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.810 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:21.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:21.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:08:21.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:21.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:21.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:08:21.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9734000000000002e-05 2020-08-07T15:08:21.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:21.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238972+0000", 2020-08-07T15:08:21.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6910000000000002e-06 2020-08-07T15:08:21.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:21.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238978+0000", 2020-08-07T15:08:21.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1864e-05 2020-08-07T15:08:21.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:21.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238990+0000", 2020-08-07T15:08:21.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.737e-06 2020-08-07T15:08:21.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:21.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238996+0000", 2020-08-07T15:08:21.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00283943 2020-08-07T15:08:21.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:21.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.241836+0000", 2020-08-07T15:08:21.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4712999999999999e-05 2020-08-07T15:08:21.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:21.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.241930+0000", 2020-08-07T15:08:21.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015328499999999999 2020-08-07T15:08:21.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:08:21.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.242084+0000", 2020-08-07T15:08:21.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016104400000000001 2020-08-07T15:08:21.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:21.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243694+0000", 2020-08-07T15:08:21.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1399999999999997e-07 2020-08-07T15:08:21.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:21.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243695+0000", 2020-08-07T15:08:21.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7800999999999998e-05 2020-08-07T15:08:21.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:21.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243753+0000", 2020-08-07T15:08:21.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090405661999999998 2020-08-07T15:08:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.334158+0000", 2020-08-07T15:08:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19283929299999999 2020-08-07T15:08:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.526997+0000", 2020-08-07T15:08:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.012e-06 2020-08-07T15:08:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.526998+0000", 2020-08-07T15:08:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4737999999999998e-05 2020-08-07T15:08:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.527043+0000", 2020-08-07T15:08:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28812091400000001 2020-08-07T15:08:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:44120 3.e 3:7e6f7bb9:::benchmark_data_smithi074_35447_object5514:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:08:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:04.450106+0000", 2020-08-07T15:08:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.011557936000003, 2020-08-07T15:08:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26702573699999999, 2020-08-07T15:08:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 44120 2020-08-07T15:08:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450106+0000", 2020-08-07T15:08:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450106+0000", 2020-08-07T15:08:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9926000000000003e-05 2020-08-07T15:08:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450156+0000", 2020-08-07T15:08:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4079999999999997e-06 2020-08-07T15:08:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450161+0000", 2020-08-07T15:08:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0494e-05 2020-08-07T15:08:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450172+0000", 2020-08-07T15:08:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3950000000000003e-06 2020-08-07T15:08:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450176+0000", 2020-08-07T15:08:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033213689999999998 2020-08-07T15:08:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.453497+0000", 2020-08-07T15:08:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102414 2020-08-07T15:08:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.453600+0000", 2020-08-07T15:08:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019776 2020-08-07T15:08:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.453798+0000", 2020-08-07T15:08:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001075456 2020-08-07T15:08:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.454873+0000", 2020-08-07T15:08:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7999999999999995e-07 2020-08-07T15:08:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.454874+0000", 2020-08-07T15:08:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3912000000000002e-05 2020-08-07T15:08:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.454928+0000", 2020-08-07T15:08:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058537550000000004 2020-08-07T15:08:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.460781+0000", 2020-08-07T15:08:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25630591899999999 2020-08-07T15:08:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.717087+0000", 2020-08-07T15:08:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.54e-07 2020-08-07T15:08:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.717088+0000", 2020-08-07T15:08:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3594999999999998e-05 2020-08-07T15:08:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.717132+0000", 2020-08-07T15:08:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26702573699999999 2020-08-07T15:08:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:49800 3.e 3:7cb4a069:::benchmark_data_smithi074_35447_object6224:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e36)", 2020-08-07T15:08:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:10.076186+0000", 2020-08-07T15:08:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 71.385478198000001, 2020-08-07T15:08:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25479637100000002, 2020-08-07T15:08:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 49800 2020-08-07T15:08:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076186+0000", 2020-08-07T15:08:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:21.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076186+0000", 2020-08-07T15:08:21.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3579000000000003e-05 2020-08-07T15:08:21.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076239+0000", 2020-08-07T15:08:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0499999999999997e-06 2020-08-07T15:08:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:21.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076245+0000", 2020-08-07T15:08:21.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1756999999999999e-05 2020-08-07T15:08:21.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:21.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076257+0000", 2020-08-07T15:08:21.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8220000000000003e-06 2020-08-07T15:08:21.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:21.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076263+0000", 2020-08-07T15:08:21.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9265999999999999e-05 2020-08-07T15:08:21.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:21.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076312+0000", 2020-08-07T15:08:21.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2280000000000004e-05 2020-08-07T15:08:21.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:21.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076404+0000", 2020-08-07T15:08:21.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015721499999999999 2020-08-07T15:08:21.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:21.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076562+0000", 2020-08-07T15:08:21.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041832179999999998 2020-08-07T15:08:21.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:21.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.080745+0000", 2020-08-07T15:08:21.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.091e-06 2020-08-07T15:08:21.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:21.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.080746+0000", 2020-08-07T15:08:21.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3623999999999999e-05 2020-08-07T15:08:21.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:21.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.080809+0000", 2020-08-07T15:08:21.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11945386299999999 2020-08-07T15:08:21.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:21.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.200263+0000", 2020-08-07T15:08:21.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.130675596 2020-08-07T15:08:21.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:21.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.330939+0000", 2020-08-07T15:08:21.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.508e-06 2020-08-07T15:08:21.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:21.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.330940+0000", 2020-08-07T15:08:21.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1501999999999997e-05 2020-08-07T15:08:21.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:21.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.330982+0000", 2020-08-07T15:08:21.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25479637100000002 2020-08-07T15:08:21.990 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:21.990 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:21.990 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:21.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.991 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:86712 3.8 e52/20 3:14925daa:::benchmark_data_smithi074_35447_object10838:head v 52'5488, mlcod=52'5479)", 2020-08-07T15:08:21.991 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.338631+0000", 2020-08-07T15:08:21.991 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.123032371999997, 2020-08-07T15:08:21.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.310229594, 2020-08-07T15:08:21.992 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:21.992 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:21.992 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:21.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:21.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338631+0000", 2020-08-07T15:08:21.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:21.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:21.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338631+0000", 2020-08-07T15:08:21.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6180000000000001e-05 2020-08-07T15:08:21.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:21.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338688+0000", 2020-08-07T15:08:21.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4949999999999996e-06 2020-08-07T15:08:21.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:21.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338694+0000", 2020-08-07T15:08:21.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30438292099999997 2020-08-07T15:08:21.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:21.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643077+0000", 2020-08-07T15:08:21.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9389999999999996e-06 2020-08-07T15:08:21.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:21.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:21.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:21.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643082+0000", 2020-08-07T15:08:22.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5096999999999999e-05 2020-08-07T15:08:22.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:22.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643127+0000", 2020-08-07T15:08:22.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4323000000000001e-05 2020-08-07T15:08:22.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:22.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643151+0000", 2020-08-07T15:08:22.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055847300000000004 2020-08-07T15:08:22.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:22.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.648736+0000", 2020-08-07T15:08:22.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012490900000000001 2020-08-07T15:08:22.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:22.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.648861+0000", 2020-08-07T15:08:22.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.310229594 2020-08-07T15:08:22.004 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:22.004 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:22.005 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:22.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.005 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86792 3.f 3:fc5386ee:::benchmark_data_smithi074_35447_object10848:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:08:22.005 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.395402+0000", 2020-08-07T15:08:22.005 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.066262262000002, 2020-08-07T15:08:22.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30259831999999998, 2020-08-07T15:08:22.006 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:22.006 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:22.006 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:22.006 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:22.007 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:22.007 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86792 2020-08-07T15:08:22.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.007 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:22.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:22.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395402+0000", 2020-08-07T15:08:22.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:22.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:22.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395402+0000", 2020-08-07T15:08:22.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2262999999999998e-05 2020-08-07T15:08:22.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:22.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395444+0000", 2020-08-07T15:08:22.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1800000000000004e-06 2020-08-07T15:08:22.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:22.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395449+0000", 2020-08-07T15:08:22.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0738e-05 2020-08-07T15:08:22.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:22.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395460+0000", 2020-08-07T15:08:22.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7219999999999999e-06 2020-08-07T15:08:22.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:22.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395464+0000", 2020-08-07T15:08:22.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0063169089999999999 2020-08-07T15:08:22.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:22.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.401781+0000", 2020-08-07T15:08:22.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010702200000000001 2020-08-07T15:08:22.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:22.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.401888+0000", 2020-08-07T15:08:22.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019171999999999999 2020-08-07T15:08:22.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:22.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.402080+0000", 2020-08-07T15:08:22.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042866839999999998 2020-08-07T15:08:22.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:22.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.406367+0000", 2020-08-07T15:08:22.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000001e-07 2020-08-07T15:08:22.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:22.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.406367+0000", 2020-08-07T15:08:22.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4720000000000004e-05 2020-08-07T15:08:22.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:22.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.406432+0000", 2020-08-07T15:08:22.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050405365000000001 2020-08-07T15:08:22.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:22.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.456837+0000", 2020-08-07T15:08:22.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24111702700000001 2020-08-07T15:08:22.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:22.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.697954+0000", 2020-08-07T15:08:22.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.22e-06 2020-08-07T15:08:22.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:22.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.697955+0000", 2020-08-07T15:08:22.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4336999999999999e-05 2020-08-07T15:08:22.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:22.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.698000+0000", 2020-08-07T15:08:22.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30259831999999998 2020-08-07T15:08:22.061 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:22.061 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:22.061 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:22.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.072 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:97576 3.d e57/35 3:b9e86280:::benchmark_data_smithi074_35447_object12196:head v 57'6304, mlcod=57'6295)", 2020-08-07T15:08:22.073 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:41.740780+0000", 2020-08-07T15:08:22.073 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.720884144000003, 2020-08-07T15:08:22.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27784160200000002, 2020-08-07T15:08:22.073 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:22.074 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:22.074 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:22.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:22.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.740780+0000", 2020-08-07T15:08:22.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:22.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:22.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.740780+0000", 2020-08-07T15:08:22.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0355999999999997e-05 2020-08-07T15:08:22.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:22.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.740830+0000", 2020-08-07T15:08:22.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6680000000000004e-06 2020-08-07T15:08:22.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:22.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.740837+0000", 2020-08-07T15:08:22.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27346419799999999 2020-08-07T15:08:22.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:22.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.014301+0000", 2020-08-07T15:08:22.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.852e-06 2020-08-07T15:08:22.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:22.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.014307+0000", 2020-08-07T15:08:22.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6504e-05 2020-08-07T15:08:22.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:22.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.014363+0000", 2020-08-07T15:08:22.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8759999999999999e-05 2020-08-07T15:08:22.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:22.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.014392+0000", 2020-08-07T15:08:22.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041857459999999997 2020-08-07T15:08:22.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:22.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.018578+0000", 2020-08-07T15:08:22.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3517999999999999e-05 2020-08-07T15:08:22.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:22.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.018621+0000", 2020-08-07T15:08:22.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27784160200000002 2020-08-07T15:08:22.099 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:22.099 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:22.099 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:22.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.100 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:104104 3.8 e58/20 3:11c276ea:::benchmark_data_smithi074_35447_object13012:head v 58'6496, mlcod=58'6487)", 2020-08-07T15:08:22.100 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:08:22.100 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.603178120000003, 2020-08-07T15:08:22.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33178497099999998, 2020-08-07T15:08:22.100 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:22.101 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:22.101 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:22.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:22.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:08:22.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:22.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:22.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:08:22.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9961999999999999e-05 2020-08-07T15:08:22.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:22.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858516+0000", 2020-08-07T15:08:22.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3200000000000001e-06 2020-08-07T15:08:22.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:22.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858520+0000", 2020-08-07T15:08:22.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32756790000000002 2020-08-07T15:08:22.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:22.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186088+0000", 2020-08-07T15:08:22.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0260000000000004e-06 2020-08-07T15:08:22.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:22.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186094+0000", 2020-08-07T15:08:22.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7919000000000001e-05 2020-08-07T15:08:22.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:22.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186132+0000", 2020-08-07T15:08:22.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4968e-05 2020-08-07T15:08:22.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:22.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186147+0000", 2020-08-07T15:08:22.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040671520000000001 2020-08-07T15:08:22.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:22.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.190214+0000", 2020-08-07T15:08:22.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6724000000000001e-05 2020-08-07T15:08:22.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:22.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.190271+0000", 2020-08-07T15:08:22.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33178497099999998 2020-08-07T15:08:22.112 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:22.113 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:22.113 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:22.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.113 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106248 3.f 3:f950fd9d:::benchmark_data_smithi074_35447_object13280:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:08:22.114 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:08:22.114 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.101499326000003, 2020-08-07T15:08:22.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002, 2020-08-07T15:08:22.114 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:22.114 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:22.114 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:22.115 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:22.115 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:22.115 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106248 2020-08-07T15:08:22.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.117 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:22.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:22.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:08:22.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:22.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:22.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:08:22.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7184e-05 2020-08-07T15:08:22.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:22.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360222+0000", 2020-08-07T15:08:22.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0906e-05 2020-08-07T15:08:22.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:22.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360233+0000", 2020-08-07T15:08:22.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2972999999999999e-05 2020-08-07T15:08:22.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:22.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360246+0000", 2020-08-07T15:08:22.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0114e-05 2020-08-07T15:08:22.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:22.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360256+0000", 2020-08-07T15:08:22.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1303999999999997e-05 2020-08-07T15:08:22.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:22.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360287+0000", 2020-08-07T15:08:22.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3798000000000001e-05 2020-08-07T15:08:22.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:22.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360351+0000", 2020-08-07T15:08:22.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111308 2020-08-07T15:08:22.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:22.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360462+0000", 2020-08-07T15:08:22.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003480674 2020-08-07T15:08:22.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:22.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:08:22.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0900000000000001e-07 2020-08-07T15:08:22.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:22.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:08:22.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8899999999999994e-05 2020-08-07T15:08:22.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:22.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.364012+0000", 2020-08-07T15:08:22.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35291637199999998 2020-08-07T15:08:22.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:22.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716929+0000", 2020-08-07T15:08:22.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.076244233999999994 2020-08-07T15:08:22.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:22.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793173+0000", 2020-08-07T15:08:22.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6109999999999999e-06 2020-08-07T15:08:22.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:22.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793175+0000", 2020-08-07T15:08:22.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7492999999999999e-05 2020-08-07T15:08:22.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:22.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793242+0000", 2020-08-07T15:08:22.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002 2020-08-07T15:08:22.143 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:22.143 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:22.143 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:22.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.144 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106248 3.f e59/20 ondisk, result = 0)", 2020-08-07T15:08:22.144 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.365323+0000", 2020-08-07T15:08:22.144 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.096340769999998, 2020-08-07T15:08:22.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35160734999999999, 2020-08-07T15:08:22.144 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:22.145 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:22.145 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:22.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:22.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365323+0000", 2020-08-07T15:08:22.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:22.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:22.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365323+0000", 2020-08-07T15:08:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4981000000000003e-05 2020-08-07T15:08:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365358+0000", 2020-08-07T15:08:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1090000000000003e-06 2020-08-07T15:08:22.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:22.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365364+0000", 2020-08-07T15:08:22.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35151899399999997 2020-08-07T15:08:22.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:22.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716883+0000", 2020-08-07T15:08:22.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0090000000000001e-06 2020-08-07T15:08:22.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:22.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716887+0000", 2020-08-07T15:08:22.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9238999999999999e-05 2020-08-07T15:08:22.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:22.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716916+0000", 2020-08-07T15:08:22.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0390000000000001e-06 2020-08-07T15:08:22.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:22.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716922+0000", 2020-08-07T15:08:22.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9789999999999993e-06 2020-08-07T15:08:22.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:22.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716930+0000", 2020-08-07T15:08:22.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35160734999999999 2020-08-07T15:08:22.157 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:22.157 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:22.157 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:22.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.158 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106248 3.f e59/20 ondisk, result = 0)", 2020-08-07T15:08:22.158 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:08:22.158 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.096236040999997, 2020-08-07T15:08:22.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001, 2020-08-07T15:08:22.158 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:22.159 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:22.159 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:22.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:22.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:08:22.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:22.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:22.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:08:22.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7518999999999999e-05 2020-08-07T15:08:22.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:22.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365445+0000", 2020-08-07T15:08:22.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-06 2020-08-07T15:08:22.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:22.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365449+0000", 2020-08-07T15:08:22.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42764147600000002 2020-08-07T15:08:22.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:22.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793091+0000", 2020-08-07T15:08:22.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6500000000000002e-06 2020-08-07T15:08:22.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:22.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793099+0000", 2020-08-07T15:08:22.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3664e-05 2020-08-07T15:08:22.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:22.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793153+0000", 2020-08-07T15:08:22.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1078e-05 2020-08-07T15:08:22.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:22.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793164+0000", 2020-08-07T15:08:22.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3404000000000003e-05 2020-08-07T15:08:22.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:22.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793248+0000", 2020-08-07T15:08:22.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001 2020-08-07T15:08:22.169 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:22.169 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:22.169 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:22.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.170 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106344 3.e 3:7abbcafa:::benchmark_data_smithi074_35447_object13292:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:08:22.170 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:48.059687+0000", 2020-08-07T15:08:22.170 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.401976939000001, 2020-08-07T15:08:22.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28541866100000002, 2020-08-07T15:08:22.171 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:22.171 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:22.171 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:22.172 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:22.172 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:22.172 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106344 2020-08-07T15:08:22.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.172 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:22.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:22.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059687+0000", 2020-08-07T15:08:22.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:22.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:22.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059687+0000", 2020-08-07T15:08:22.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5778999999999998e-05 2020-08-07T15:08:22.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:22.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059713+0000", 2020-08-07T15:08:22.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.422e-06 2020-08-07T15:08:22.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:22.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059716+0000", 2020-08-07T15:08:22.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7679999999999997e-06 2020-08-07T15:08:22.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:22.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059725+0000", 2020-08-07T15:08:22.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2020-08-07T15:08:22.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:22.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059729+0000", 2020-08-07T15:08:22.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6205000000000003e-05 2020-08-07T15:08:22.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:22.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059765+0000", 2020-08-07T15:08:22.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3615999999999995e-05 2020-08-07T15:08:22.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:22.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059829+0000", 2020-08-07T15:08:22.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000118903 2020-08-07T15:08:22.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:22.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059948+0000", 2020-08-07T15:08:22.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00070666299999999995 2020-08-07T15:08:22.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:22.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.060654+0000", 2020-08-07T15:08:22.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5199999999999995e-07 2020-08-07T15:08:22.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:22.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.060655+0000", 2020-08-07T15:08:22.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2092999999999998e-05 2020-08-07T15:08:22.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:22.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.060697+0000", 2020-08-07T15:08:22.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00038454500000000003 2020-08-07T15:08:22.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:22.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.061082+0000", 2020-08-07T15:08:22.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.283939936 2020-08-07T15:08:22.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:22.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.345022+0000", 2020-08-07T15:08:22.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.288e-06 2020-08-07T15:08:22.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:22.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.345023+0000", 2020-08-07T15:08:22.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2468999999999999e-05 2020-08-07T15:08:22.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:22.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.345105+0000", 2020-08-07T15:08:22.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28541866100000002 2020-08-07T15:08:22.191 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:22.191 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:22.191 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:22.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.192 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:107952 3.d e61/35)", 2020-08-07T15:08:22.192 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:08:22.192 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.135306332999999, 2020-08-07T15:08:22.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001, 2020-08-07T15:08:22.193 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:22.193 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:08:22.193 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:22.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:22.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:08:22.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:22.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:22.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:08:22.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8826000000000003e-05 2020-08-07T15:08:22.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:22.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326406+0000", 2020-08-07T15:08:22.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9199999999999998e-06 2020-08-07T15:08:22.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:22.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326413+0000", 2020-08-07T15:08:22.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75082853500000002 2020-08-07T15:08:22.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:22.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077242+0000", 2020-08-07T15:08:22.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1479999999999999e-06 2020-08-07T15:08:22.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:22.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077246+0000", 2020-08-07T15:08:22.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3331999999999998e-05 2020-08-07T15:08:22.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:22.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077289+0000", 2020-08-07T15:08:22.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8977000000000001e-05 2020-08-07T15:08:22.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:22.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:22.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:22.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077308+0000", 2020-08-07T15:08:22.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001 2020-08-07T15:08:22.205 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:22.205 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:22.205 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:22.205 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:22.206 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:22.206 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:22.207 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:08:22.208 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:08:22.209 INFO:teuthology.orchestra.run.smithi074.stderr:pool 'unique_pool_0' removed 2020-08-07T15:08:22.210 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2020-08-07T15:08:22.211 INFO:tasks.radosbench:Beginning radosbench... 2020-08-07T15:08:22.211 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_1 2020-08-07T15:08:22.212 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create unique_pool_1 16 2020-08-07T15:08:22.213 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:22.206+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:22.213 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:08:22.311 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:22.309+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:22.398 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:22.400 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:08:22.400 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:08:22.400 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:22.411 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:08:22.414 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:22.413+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:22.414 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:08:22.442 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:08:22.476 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:08:22.506 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:08:22.520 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:22.517+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:22.528 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:08:22.555 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:22.556 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:22.571 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:08:22.571 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:08:22.598 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:08:22.624 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:22.622+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:22.629 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:08:22.636 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:08:22.636 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:08:22.637 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:08:22.637 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:08:22.637 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:08:22.637 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:08:22.637 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:08:22.637 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:08:22.724 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:22.724 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:22.725 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:22.723+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:22.742 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:08:22.835 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:22.834+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:22.866 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:22.866 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:22.900 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:08:22.941 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:22.940+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:23.026 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:23.027 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:23.034 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:08:23.044 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:23.042+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:23.145 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:23.142+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:23.351 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:08:23.352 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:08:23.352 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump -f json-pretty 2020-08-07T15:08:23.364 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:23.366 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:23.367 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:08:23.368 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:23.356+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:23.454 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:23.452+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:23.530 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:23.530 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:23.546 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:08:23.549 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:08:23.551 INFO:teuthology.orchestra.run.smithi074.stderr:pool 'unique_pool_1' created 2020-08-07T15:08:23.576 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:23.574+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:23.576 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool application enable unique_pool_1 rados --yes-i-really-mean-it || true 2020-08-07T15:08:23.680 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:23.676+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:23.700 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:08:23.700 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:23.701 INFO:teuthology.orchestra.run.smithi074.stdout: "epoch": 81, 2020-08-07T15:08:23.701 INFO:teuthology.orchestra.run.smithi074.stdout: "fsid": "a01d3e82-8d24-4288-9e42-576ab3ff4957", 2020-08-07T15:08:23.701 INFO:teuthology.orchestra.run.smithi074.stdout: "created": "2020-08-07T15:06:07.342912+0000", 2020-08-07T15:08:23.701 INFO:teuthology.orchestra.run.smithi074.stdout: "modified": "2020-08-07T15:08:23.520041+0000", 2020-08-07T15:08:23.702 INFO:teuthology.orchestra.run.smithi074.stdout: "last_up_change": "2020-08-07T15:06:17.674292+0000", 2020-08-07T15:08:23.702 INFO:teuthology.orchestra.run.smithi074.stdout: "last_in_change": "2020-08-07T15:06:17.674292+0000", 2020-08-07T15:08:23.703 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-08-07T15:08:23.703 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_num": 5804032, 2020-08-07T15:08:23.703 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_set": [ 2020-08-07T15:08:23.703 INFO:teuthology.orchestra.run.smithi074.stdout: "nodeep-scrub", 2020-08-07T15:08:23.704 INFO:teuthology.orchestra.run.smithi074.stdout: "pglog_hardlimit", 2020-08-07T15:08:23.704 INFO:teuthology.orchestra.run.smithi074.stdout: "purged_snapdirs", 2020-08-07T15:08:23.704 INFO:teuthology.orchestra.run.smithi074.stdout: "recovery_deletes", 2020-08-07T15:08:23.704 INFO:teuthology.orchestra.run.smithi074.stdout: "sortbitwise" 2020-08-07T15:08:23.704 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:08:23.705 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_version": 5, 2020-08-07T15:08:23.705 INFO:teuthology.orchestra.run.smithi074.stdout: "full_ratio": 0.94999998807907104, 2020-08-07T15:08:23.705 INFO:teuthology.orchestra.run.smithi074.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-08-07T15:08:23.705 INFO:teuthology.orchestra.run.smithi074.stdout: "nearfull_ratio": 0.85000002384185791, 2020-08-07T15:08:23.705 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_snapshot": "", 2020-08-07T15:08:23.706 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_max": 4, 2020-08-07T15:08:23.706 INFO:teuthology.orchestra.run.smithi074.stdout: "max_osd": 8, 2020-08-07T15:08:23.706 INFO:teuthology.orchestra.run.smithi074.stdout: "require_min_compat_client": "luminous", 2020-08-07T15:08:23.706 INFO:teuthology.orchestra.run.smithi074.stdout: "min_compat_client": "jewel", 2020-08-07T15:08:23.706 INFO:teuthology.orchestra.run.smithi074.stdout: "require_osd_release": "pacific", 2020-08-07T15:08:23.707 INFO:teuthology.orchestra.run.smithi074.stdout: "pools": [ 2020-08-07T15:08:23.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.707 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 1, 2020-08-07T15:08:23.708 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "device_health_metrics", 2020-08-07T15:08:23.708 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:06:10.648876+0000", 2020-08-07T15:08:23.708 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:08:23.708 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:08:23.709 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:08:23.709 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:08:23.709 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:08:23.709 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:08:23.709 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:08:23.710 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:08:23.711 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 11, 2020-08-07T15:08:23.711 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 11, 2020-08-07T15:08:23.711 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 11, 2020-08-07T15:08:23.711 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 11, 2020-08-07T15:08:23.712 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 11, 2020-08-07T15:08:23.712 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:08:23.712 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:08:23.712 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:08:23.713 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:08:23.713 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:08:23.713 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:08:23.713 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:08:23.713 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.714 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "47", 2020-08-07T15:08:23.714 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:08:23.714 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:08:23.714 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "45", 2020-08-07T15:08:23.714 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:08:23.715 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:08:23.715 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:08:23.715 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:08:23.715 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:08:23.715 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:08:23.716 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:08:23.716 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:08:23.716 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:08:23.716 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:08:23.716 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:08:23.717 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:08:23.717 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:08:23.717 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:08:23.717 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:08:23.717 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:08:23.718 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:08:23.719 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:08:23.720 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:08:23.720 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:08:23.720 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:08:23.720 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:08:23.720 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:08:23.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.721 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:08:23.721 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:08:23.721 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:08:23.721 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:08:23.722 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:08:23.722 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:08:23.722 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:08:23.722 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:08:23.723 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:08:23.723 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:08:23.723 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:08:23.723 INFO:teuthology.orchestra.run.smithi074.stdout: "options": { 2020-08-07T15:08:23.723 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_min": 1 2020-08-07T15:08:23.724 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.724 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:08:23.724 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_devicehealth": {} 2020-08-07T15:08:23.724 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.724 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.725 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 2, 2020-08-07T15:08:23.725 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "rbd", 2020-08-07T15:08:23.725 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:06:21.474648+0000", 2020-08-07T15:08:23.726 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:08:23.726 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:08:23.726 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:08:23.727 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:08:23.728 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:08:23.728 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:08:23.728 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:08:23.729 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:08:23.729 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 8, 2020-08-07T15:08:23.729 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 8, 2020-08-07T15:08:23.729 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 8, 2020-08-07T15:08:23.729 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 8, 2020-08-07T15:08:23.730 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 8, 2020-08-07T15:08:23.730 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:08:23.730 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:08:23.730 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:08:23.730 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:08:23.731 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:08:23.731 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:08:23.731 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:08:23.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.733 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "19", 2020-08-07T15:08:23.734 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:08:23.734 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:08:23.734 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "0", 2020-08-07T15:08:23.734 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:08:23.734 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:08:23.735 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:08:23.735 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:08:23.735 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:08:23.735 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:08:23.735 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:08:23.736 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:08:23.736 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:08:23.737 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:08:23.737 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:08:23.737 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:08:23.738 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:08:23.738 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:08:23.738 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:08:23.738 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:08:23.738 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:08:23.738 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:08:23.739 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:08:23.739 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:08:23.740 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:08:23.740 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:08:23.740 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:08:23.740 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.740 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:08:23.741 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:08:23.741 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:08:23.741 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:08:23.741 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:08:23.741 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:08:23.742 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:08:23.743 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:08:23.744 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:08:23.744 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:08:23.744 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:08:23.744 INFO:teuthology.orchestra.run.smithi074.stdout: "options": {}, 2020-08-07T15:08:23.744 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:08:23.745 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd": {} 2020-08-07T15:08:23.745 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.745 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.745 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 4, 2020-08-07T15:08:23.746 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "unique_pool_1", 2020-08-07T15:08:23.746 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:08:22.604810+0000", 2020-08-07T15:08:23.746 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 32769, 2020-08-07T15:08:23.746 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool,creating", 2020-08-07T15:08:23.746 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:08:23.747 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:08:23.747 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:08:23.748 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:08:23.748 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:08:23.748 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:08:23.748 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 16, 2020-08-07T15:08:23.749 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 16, 2020-08-07T15:08:23.749 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 16, 2020-08-07T15:08:23.749 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 16, 2020-08-07T15:08:23.749 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 16, 2020-08-07T15:08:23.749 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:08:23.750 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:08:23.750 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:08:23.750 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:08:23.750 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:08:23.750 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:08:23.751 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:08:23.751 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.751 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "81", 2020-08-07T15:08:23.751 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:08:23.751 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:08:23.752 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "0", 2020-08-07T15:08:23.752 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:08:23.752 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:08:23.752 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:08:23.753 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:08:23.754 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:08:23.754 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:08:23.755 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:08:23.755 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:08:23.755 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:08:23.755 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:08:23.755 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:08:23.756 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:08:23.756 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:08:23.756 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:08:23.756 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:08:23.756 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:08:23.756 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:08:23.757 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:08:23.757 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:08:23.758 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:08:23.758 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:08:23.758 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:08:23.758 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:08:23.758 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.759 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:08:23.759 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:08:23.759 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:08:23.759 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:08:23.759 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:08:23.760 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:08:23.776 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:08:23.776 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:08:23.776 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:08:23.776 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:08:23.777 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:08:23.777 INFO:teuthology.orchestra.run.smithi074.stdout: "options": {}, 2020-08-07T15:08:23.777 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": {} 2020-08-07T15:08:23.784 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.784 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:08:23.784 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:08:23.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.785 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 0, 2020-08-07T15:08:23.785 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "fc1ebb23-3ef4-4fc0-b395-382e385885a3", 2020-08-07T15:08:23.785 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:08:23.786 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:08:23.786 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:08:23.786 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0, 2020-08-07T15:08:23.786 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:08:23.786 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:08:23.787 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:08:23.787 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 49, 2020-08-07T15:08:23.787 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:08:23.787 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:08:23.787 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:08:23.788 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.811 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.811 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6801", 2020-08-07T15:08:23.811 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:08:23.811 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.812 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.813 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:08:23.813 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.813 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.814 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6802", 2020-08-07T15:08:23.814 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:08:23.814 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.814 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.814 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.815 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:08:23.815 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.815 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.815 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6804", 2020-08-07T15:08:23.815 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:08:23.816 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.816 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.816 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.816 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:08:23.817 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.817 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.818 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6803", 2020-08-07T15:08:23.818 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:08:23.818 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.818 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.818 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.819 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6801/31577", 2020-08-07T15:08:23.819 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6802/31577", 2020-08-07T15:08:23.819 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6804/31577", 2020-08-07T15:08:23.819 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6803/31577", 2020-08-07T15:08:23.822 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:08:23.823 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:08:23.823 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:08:23.823 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.824 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 1, 2020-08-07T15:08:23.824 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "1fbe530a-63ce-4dc8-82ab-18793d6ebc0b", 2020-08-07T15:08:23.824 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:08:23.825 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:08:23.825 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:08:23.825 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0.7196044921875, 2020-08-07T15:08:23.825 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:08:23.825 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:08:23.826 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:08:23.826 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 78, 2020-08-07T15:08:23.826 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:08:23.826 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:08:23.826 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:08:23.827 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.827 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.827 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6809", 2020-08-07T15:08:23.827 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:08:23.828 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.828 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.828 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:08:23.828 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.829 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.829 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.829 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6810", 2020-08-07T15:08:23.829 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:08:23.829 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.830 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.831 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.831 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:08:23.832 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.832 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.832 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6812", 2020-08-07T15:08:23.832 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:08:23.832 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.833 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.834 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:08:23.834 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.845 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.845 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6811", 2020-08-07T15:08:23.848 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:08:23.848 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.848 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.849 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.849 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6809/31578", 2020-08-07T15:08:23.856 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6810/31578", 2020-08-07T15:08:23.856 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6812/31578", 2020-08-07T15:08:23.856 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6811/31578", 2020-08-07T15:08:23.857 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:08:23.857 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:08:23.857 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:08:23.857 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.862 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.864 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 2, 2020-08-07T15:08:23.864 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "d6fe4338-3e11-4f73-9cbb-e1b046ce286a", 2020-08-07T15:08:23.864 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:08:23.864 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:08:23.865 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:08:23.870 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:08:23.870 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:08:23.870 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:08:23.870 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:08:23.870 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 49, 2020-08-07T15:08:23.871 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:08:23.871 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:08:23.871 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:08:23.871 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.872 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.872 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6805", 2020-08-07T15:08:23.872 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:08:23.872 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.872 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.873 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.873 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:08:23.873 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.883 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.884 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6806", 2020-08-07T15:08:23.884 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:08:23.885 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.885 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.885 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:08:23.886 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.888 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.889 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6808", 2020-08-07T15:08:23.889 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:08:23.889 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.889 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.890 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:08:23.890 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.890 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.890 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6807", 2020-08-07T15:08:23.891 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:08:23.891 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.891 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.891 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6805/31576", 2020-08-07T15:08:23.891 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6806/31576", 2020-08-07T15:08:23.892 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6808/31576", 2020-08-07T15:08:23.892 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6807/31576", 2020-08-07T15:08:23.892 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:08:23.892 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:08:23.892 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:08:23.893 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.893 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 3, 2020-08-07T15:08:23.893 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9", 2020-08-07T15:08:23.894 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:08:23.894 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:08:23.894 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 0.7917938232421875, 2020-08-07T15:08:23.894 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:08:23.894 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:08:23.896 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:08:23.896 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:08:23.896 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 62, 2020-08-07T15:08:23.911 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:08:23.911 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:08:23.912 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:08:23.912 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.912 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.912 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6813", 2020-08-07T15:08:23.913 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:08:23.913 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.913 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.913 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:08:23.914 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.914 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.914 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6814", 2020-08-07T15:08:23.914 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:08:23.915 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.915 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.915 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:08:23.915 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.916 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.916 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6816", 2020-08-07T15:08:23.916 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:08:23.916 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.916 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.917 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:08:23.917 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.917 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.918 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6815", 2020-08-07T15:08:23.919 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:08:23.919 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.920 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.920 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6813/31579", 2020-08-07T15:08:23.920 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6814/31579", 2020-08-07T15:08:23.921 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6816/31579", 2020-08-07T15:08:23.921 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6815/31579", 2020-08-07T15:08:23.921 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:08:23.922 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:08:23.923 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:08:23.923 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.924 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 4, 2020-08-07T15:08:23.924 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "01aadc13-29c5-4ddd-9bb9-de24b2b20412", 2020-08-07T15:08:23.924 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:08:23.924 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:08:23.924 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:08:23.925 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:08:23.925 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:08:23.925 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:08:23.925 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:08:23.925 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 67, 2020-08-07T15:08:23.926 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:08:23.926 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:08:23.926 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:08:23.926 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.927 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.927 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6800", 2020-08-07T15:08:23.927 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:08:23.927 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.927 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.928 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:08:23.928 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.929 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.930 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6801", 2020-08-07T15:08:23.930 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:08:23.930 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.930 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.931 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:08:23.931 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.932 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.932 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.932 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6803", 2020-08-07T15:08:23.932 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:08:23.933 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.933 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.934 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.934 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:08:23.934 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.934 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.934 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.935 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6802", 2020-08-07T15:08:23.935 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:08:23.935 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.935 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.935 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6800/31545", 2020-08-07T15:08:23.936 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6801/31545", 2020-08-07T15:08:23.936 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6803/31545", 2020-08-07T15:08:23.936 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6802/31545", 2020-08-07T15:08:23.936 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:08:23.936 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:08:23.937 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:08:23.937 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.937 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.937 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 5, 2020-08-07T15:08:23.938 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "f5556a82-73b4-47b1-8e29-1d2fe031d8fd", 2020-08-07T15:08:23.938 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:08:23.938 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:08:23.938 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:08:23.938 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:08:23.939 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:08:23.939 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:08:23.940 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:08:23.940 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 61, 2020-08-07T15:08:23.940 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:08:23.941 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:08:23.941 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:08:23.941 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.941 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.942 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6804", 2020-08-07T15:08:23.943 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:08:23.943 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.943 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.944 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:08:23.944 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.944 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.945 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6805", 2020-08-07T15:08:23.945 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:08:23.945 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.945 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.946 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:08:23.946 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.946 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.946 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.946 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6807", 2020-08-07T15:08:23.947 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:08:23.947 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.947 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.947 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:08:23.948 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.948 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.948 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6806", 2020-08-07T15:08:23.948 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:08:23.949 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.949 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.949 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.949 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6804/31547", 2020-08-07T15:08:23.950 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6805/31547", 2020-08-07T15:08:23.951 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6807/31547", 2020-08-07T15:08:23.951 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6806/31547", 2020-08-07T15:08:23.951 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:08:23.951 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:08:23.951 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:08:23.952 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.952 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.952 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.953 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 6, 2020-08-07T15:08:23.953 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "9b98b337-e1cc-43cd-9e9b-f21ae3f20899", 2020-08-07T15:08:23.953 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:08:23.953 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:08:23.954 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:08:23.955 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:08:23.955 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:08:23.955 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:08:23.955 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 14, 2020-08-07T15:08:23.956 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 62, 2020-08-07T15:08:23.956 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:08:23.956 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:08:23.956 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:08:23.956 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.957 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.957 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6808", 2020-08-07T15:08:23.957 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:08:23.957 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.958 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.958 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:08:23.958 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.959 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.959 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6809", 2020-08-07T15:08:23.959 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:08:23.959 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.959 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.960 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:08:23.961 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.961 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.962 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6811", 2020-08-07T15:08:23.962 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:08:23.962 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.962 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.964 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:08:23.964 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.965 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.965 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6810", 2020-08-07T15:08:23.965 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:08:23.965 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.965 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.966 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6808/31546", 2020-08-07T15:08:23.966 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6809/31546", 2020-08-07T15:08:23.966 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6811/31546", 2020-08-07T15:08:23.967 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6810/31546", 2020-08-07T15:08:23.967 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:08:23.967 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:08:23.968 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:08:23.968 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.968 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 7, 2020-08-07T15:08:23.969 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "777662d8-bd34-48e8-83e1-24ce9e61f14b", 2020-08-07T15:08:23.969 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:08:23.969 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:08:23.969 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:08:23.969 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:08:23.970 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:08:23.970 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:08:23.970 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:08:23.970 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 50, 2020-08-07T15:08:23.971 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:08:23.972 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:08:23.972 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:08:23.972 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.972 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.973 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6812", 2020-08-07T15:08:23.973 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:08:23.973 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.973 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.976 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:08:23.976 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.976 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.977 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6813", 2020-08-07T15:08:23.977 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:08:23.977 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.978 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.978 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:08:23.978 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.979 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.979 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6815", 2020-08-07T15:08:23.979 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:08:23.979 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.979 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.980 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:08:23.980 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:08:23.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.980 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:08:23.981 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6814", 2020-08-07T15:08:23.983 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:08:23.983 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.984 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.984 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6812/31570", 2020-08-07T15:08:23.985 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6813/31570", 2020-08-07T15:08:23.986 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6815/31570", 2020-08-07T15:08:23.986 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6814/31570", 2020-08-07T15:08:23.986 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:08:23.987 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:08:23.987 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:08:23.988 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:23.988 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:23.988 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:08:23.988 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_xinfo": [ 2020-08-07T15:08:23.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.989 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 0, 2020-08-07T15:08:23.989 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:08:23.989 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:08:23.989 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:08:23.990 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:08:23.990 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:08:23.990 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:08:17.127270+0000", 2020-08-07T15:08:23.990 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:08:23.990 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:23.991 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 1, 2020-08-07T15:08:23.991 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:08:23.991 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:08:23.991 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:08:23.994 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:08:23.994 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:08:23.994 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:08:15.853980+0000", 2020-08-07T15:08:23.994 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:08:23.995 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:23.995 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:24.004 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 2, 2020-08-07T15:08:24.005 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:08:24.005 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:08:24.005 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:08:24.006 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:08:24.006 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:08:24.006 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:08:16.235887+0000", 2020-08-07T15:08:24.007 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:08:24.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:24.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:24.007 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 3, 2020-08-07T15:08:24.007 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:08:24.008 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:08:24.008 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:08:24.008 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:08:24.008 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:08:24.008 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:08:17.320589+0000", 2020-08-07T15:08:24.008 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:08:24.009 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:24.009 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:24.009 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 4, 2020-08-07T15:08:24.009 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:08:24.009 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:08:24.010 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:08:24.010 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:08:24.010 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:08:24.010 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:08:16.349218+0000", 2020-08-07T15:08:24.011 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:08:24.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:24.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:24.011 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 5, 2020-08-07T15:08:24.011 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:08:24.012 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:08:24.012 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:08:24.012 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:08:24.012 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:08:24.012 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:08:16.727337+0000", 2020-08-07T15:08:24.014 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:08:24.014 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:24.014 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:24.015 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 6, 2020-08-07T15:08:24.015 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:08:24.015 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:08:24.015 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:08:24.015 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:08:24.015 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:08:24.016 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:08:16.299215+0000", 2020-08-07T15:08:24.016 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:08:24.016 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:24.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:24.018 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 7, 2020-08-07T15:08:24.018 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:08:24.018 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:08:24.018 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:08:24.018 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:08:24.019 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:08:24.019 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:08:16.172523+0000", 2020-08-07T15:08:24.019 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:08:24.019 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:24.019 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:08:24.020 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_upmap": [], 2020-08-07T15:08:24.020 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_upmap_items": [], 2020-08-07T15:08:24.020 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_temp": [], 2020-08-07T15:08:24.020 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_temp": [], 2020-08-07T15:08:24.020 INFO:teuthology.orchestra.run.smithi074.stdout: "blacklist": {}, 2020-08-07T15:08:24.021 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profiles": { 2020-08-07T15:08:24.021 INFO:teuthology.orchestra.run.smithi074.stdout: "default": { 2020-08-07T15:08:24.021 INFO:teuthology.orchestra.run.smithi074.stdout: "crush-failure-domain": "osd", 2020-08-07T15:08:24.021 INFO:teuthology.orchestra.run.smithi074.stdout: "k": "2", 2020-08-07T15:08:24.021 INFO:teuthology.orchestra.run.smithi074.stdout: "m": "1", 2020-08-07T15:08:24.022 INFO:teuthology.orchestra.run.smithi074.stdout: "plugin": "jerasure", 2020-08-07T15:08:24.022 INFO:teuthology.orchestra.run.smithi074.stdout: "ruleset-failure-domain": "osd", 2020-08-07T15:08:24.022 INFO:teuthology.orchestra.run.smithi074.stdout: "technique": "reed_sol_van" 2020-08-07T15:08:24.022 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:24.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:24.022 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps_queue": [], 2020-08-07T15:08:24.023 INFO:teuthology.orchestra.run.smithi074.stdout: "new_removed_snaps": [], 2020-08-07T15:08:24.023 INFO:teuthology.orchestra.run.smithi074.stdout: "new_purged_snaps": [], 2020-08-07T15:08:24.023 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_node_flags": {}, 2020-08-07T15:08:24.024 INFO:teuthology.orchestra.run.smithi074.stdout: "device_class_flags": {} 2020-08-07T15:08:24.024 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:24.027 INFO:tasks.thrashosds.thrasher:j is {'epoch': 81, 'fsid': 'a01d3e82-8d24-4288-9e42-576ab3ff4957', 'created': '2020-08-07T15:06:07.342912+0000', 'modified': '2020-08-07T15:08:23.520041+0000', 'last_up_change': '2020-08-07T15:06:17.674292+0000', 'last_in_change': '2020-08-07T15:06:17.674292+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-08-07T15:06:10.648876+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 11, 'pg_placement_num': 11, 'pg_placement_num_target': 11, 'pg_num_target': 11, 'pg_num_pending': 11, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '45', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-08-07T15:06:21.474648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'unique_pool_1', 'create_time': '2020-08-07T15:08:22.604810+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}}], 'osds': [{'osd': 0, 'uuid': 'fc1ebb23-3ef4-4fc0-b395-382e385885a3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6801', 'nonce': 31577}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6802', 'nonce': 31577}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6804', 'nonce': 31577}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6803', 'nonce': 31577}]}, 'public_addr': '172.21.15.74:6801/31577', 'cluster_addr': '172.21.15.74:6802/31577', 'heartbeat_back_addr': '172.21.15.74:6804/31577', 'heartbeat_front_addr': '172.21.15.74:6803/31577', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '1fbe530a-63ce-4dc8-82ab-18793d6ebc0b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.7196044921875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 78, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6809', 'nonce': 31578}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6810', 'nonce': 31578}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6812', 'nonce': 31578}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6811', 'nonce': 31578}]}, 'public_addr': '172.21.15.74:6809/31578', 'cluster_addr': '172.21.15.74:6810/31578', 'heartbeat_back_addr': '172.21.15.74:6812/31578', 'heartbeat_front_addr': '172.21.15.74:6811/31578', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd6fe4338-3e11-4f73-9cbb-e1b046ce286a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6805', 'nonce': 31576}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6806', 'nonce': 31576}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6808', 'nonce': 31576}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6807', 'nonce': 31576}]}, 'public_addr': '172.21.15.74:6805/31576', 'cluster_addr': '172.21.15.74:6806/31576', 'heartbeat_back_addr': '172.21.15.74:6808/31576', 'heartbeat_front_addr': '172.21.15.74:6807/31576', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9', 'up': 1, 'in': 1, 'weight': 0.7917938232421875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 62, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6813', 'nonce': 31579}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6814', 'nonce': 31579}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6816', 'nonce': 31579}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6815', 'nonce': 31579}]}, 'public_addr': '172.21.15.74:6813/31579', 'cluster_addr': '172.21.15.74:6814/31579', 'heartbeat_back_addr': '172.21.15.74:6816/31579', 'heartbeat_front_addr': '172.21.15.74:6815/31579', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '01aadc13-29c5-4ddd-9bb9-de24b2b20412', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 67, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6800', 'nonce': 31545}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6801', 'nonce': 31545}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6803', 'nonce': 31545}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6802', 'nonce': 31545}]}, 'public_addr': '172.21.15.96:6800/31545', 'cluster_addr': '172.21.15.96:6801/31545', 'heartbeat_back_addr': '172.21.15.96:6803/31545', 'heartbeat_front_addr': '172.21.15.96:6802/31545', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'f5556a82-73b4-47b1-8e29-1d2fe031d8fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 61, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6804', 'nonce': 31547}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6805', 'nonce': 31547}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6807', 'nonce': 31547}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6806', 'nonce': 31547}]}, 'public_addr': '172.21.15.96:6804/31547', 'cluster_addr': '172.21.15.96:6805/31547', 'heartbeat_back_addr': '172.21.15.96:6807/31547', 'heartbeat_front_addr': '172.21.15.96:6806/31547', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9b98b337-e1cc-43cd-9e9b-f21ae3f20899', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 62, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6808', 'nonce': 31546}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6809', 'nonce': 31546}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6811', 'nonce': 31546}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6810', 'nonce': 31546}]}, 'public_addr': '172.21.15.96:6808/31546', 'cluster_addr': '172.21.15.96:6809/31546', 'heartbeat_back_addr': '172.21.15.96:6811/31546', 'heartbeat_front_addr': '172.21.15.96:6810/31546', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '777662d8-bd34-48e8-83e1-24ce9e61f14b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6812', 'nonce': 31570}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6813', 'nonce': 31570}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6815', 'nonce': 31570}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6814', 'nonce': 31570}]}, 'public_addr': '172.21.15.96:6812/31570', 'cluster_addr': '172.21.15.96:6813/31570', 'heartbeat_back_addr': '172.21.15.96:6815/31570', 'heartbeat_front_addr': '172.21.15.96:6814/31570', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:08:17.127270+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:08:15.853980+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:08:16.235887+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:08:17.320589+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:08:16.349218+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:08:16.727337+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:08:16.299215+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:08:16.172523+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blacklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}} 2020-08-07T15:08:24.027 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:08:24.028 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:24.028 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:24.029 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:08:24.031 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:24.024+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:24.126 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:24.124+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:24.195 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:24.195 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:24.205 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:08:24.246 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:24.244+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:24.346 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:24.335+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:24.355 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:08:24.356 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:08:24.366 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2020-08-07T15:08:23.530336+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":595,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":57,"acting":57,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":108,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8414732,"kb_used_data":26060,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746559988,"statfs":{"total":773094113280,"available":764477427712,"internally_reserved":8589934592,"allocated":26685440,"data_stored":22296304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-145809,"num_write_kb":-1036849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1650,"num_bytes_recovered":-108134400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.894912"},"pg_stats":[{"pgid":"4.e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:08:23.520041+0000","last_change":"2020-08-07T15:08:23.520041+0000","last_active":"2020-08-07T15:08:23.520041+0000","last_peered":"2020-08-07T15:08:23.520041+0000","last_clean":"2020-08-07T15:08:23.520041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:08:23.520041+0000","last_undegraded":"2020-08-07T15:08:23.520041+0000","last_fullsized":"2020-08-07T15:08:23.520041+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:08:23.520041+0000","last_change":"2020-08-07T15:08:23.520041+0000","last_active":"2020-08-07T15:08:23.520041+0000","last_peered":"2020-08-07T15:08:23.520041+0000","last_clean":"2020-08-07T15:08:23.520041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:08:23.520041+0000","last_undegraded":"2020-08-07T15:08:23.520041+0000","last_fullsized":"2020-08-07T15:08:23.520041+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:08:23.520041+0000","last_change":"2020-08-07T15:08:23.520041+0000","last_active":"2020-08-07T15:08:23.520041+0000","last_peered":"2020-08-07T15:08:23.520041+0000","last_clean":"2020-08-07T15:08:23.520041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:08:23.520041+0000","last_undegraded":"2020-08-07T15:08:23.520041+0000","last_fullsized":"2020-08-07T15:08:23.520041+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:08:23.520041+0000","last_change":"2020-08-07T15:08:23.520041+0000","last_active":"2020-08-07T15:08:23.520041+0000","last_peered":"2020-08-07T15:08:23.520041+0000","last_clean":"2020-08-07T15:08:23.520041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:08:23.520041+0000","last_undegraded":"2020-08-07T15:08:23.520041+0000","last_fullsized":"2020-08-07T15:08:23.520041+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:08:23.520041+0000","last_change":"2020-08-07T15:08:23.520041+0000","last_active":"2020-08-07T15:08:23.520041+0000","last_peered":"2020-08-07T15:08:23.520041+0000","last_clean":"2020-08-07T15:08:23.520041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:08:23.520041+0000","last_undegraded":"2020-08-07T15:08:23.520041+0000","last_fullsized":"2020-08-07T15:08:23.520041+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:08:23.520041+0000","last_change":"2020-08-07T15:08:23.520041+0000","last_active":"2020-08-07T15:08:23.520041+0000","last_peered":"2020-08-07T15:08:23.520041+0000","last_clean":"2020-08-07T15:08:23.520041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:08:23.520041+0000","last_undegraded":"2020-08-07T15:08:23.520041+0000","last_fullsized":"2020-08-07T15:08:23.520041+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"109","reported_epoch":"78","state":"active+clean","last_fresh":"2020-08-07T15:08:19.514213+0000","last_change":"2020-08-07T15:07:56.631170+0000","last_active":"2020-08-07T15:08:19.514213+0000","last_peered":"2020-08-07T15:08:19.514213+0000","last_clean":"2020-08-07T15:08:19.514213+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:08:19.514213+0000","last_undegraded":"2020-08-07T15:08:19.514213+0000","last_fullsized":"2020-08-07T15:08:19.514213+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:56.631091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:56.631091+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:08:23.520041+0000","last_change":"2020-08-07T15:08:23.520041+0000","last_active":"2020-08-07T15:08:23.520041+0000","last_peered":"2020-08-07T15:08:23.520041+0000","last_clean":"2020-08-07T15:08:23.520041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:08:23.520041+0000","last_undegraded":"2020-08-07T15:08:23.520041+0000","last_fullsized":"2020-08-07T15:08:23.520041+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"45","reported_epoch":"78","state":"active+clean","last_fresh":"2020-08-07T15:08:19.718521+0000","last_change":"2020-08-07T15:07:33.177491+0000","last_active":"2020-08-07T15:08:19.718521+0000","last_peered":"2020-08-07T15:08:19.718521+0000","last_clean":"2020-08-07T15:08:19.718521+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:08:19.718521+0000","last_undegraded":"2020-08-07T15:08:19.718521+0000","last_fullsized":"2020-08-07T15:08:19.718521+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:33.177412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:33.177412+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:08:23.520041+0000","last_change":"2020-08-07T15:08:23.520041+0000","last_active":"2020-08-07T15:08:23.520041+0000","last_peered":"2020-08-07T15:08:23.520041+0000","last_clean":"2020-08-07T15:08:23.520041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:08:23.520041+0000","last_undegraded":"2020-08-07T15:08:23.520041+0000","last_fullsized":"2020-08-07T15:08:23.520041+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"45","reported_epoch":"78","state":"active+clean","last_fresh":"2020-08-07T15:08:19.718600+0000","last_change":"2020-08-07T15:07:36.143434+0000","last_active":"2020-08-07T15:08:19.718600+0000","last_peered":"2020-08-07T15:08:19.718600+0000","last_clean":"2020-08-07T15:08:19.718600+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:08:19.718600+0000","last_undegraded":"2020-08-07T15:08:19.718600+0000","last_fullsized":"2020-08-07T15:08:19.718600+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.143353+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:08:23.520041+0000","last_change":"2020-08-07T15:08:23.520041+0000","last_active":"2020-08-07T15:08:23.520041+0000","last_peered":"2020-08-07T15:08:23.520041+0000","last_clean":"2020-08-07T15:08:23.520041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:08:23.520041+0000","last_undegraded":"2020-08-07T15:08:23.520041+0000","last_fullsized":"2020-08-07T15:08:23.520041+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"83","reported_epoch":"78","state":"active+clean","last_fresh":"2020-08-07T15:08:19.719122+0000","last_change":"2020-08-07T15:07:51.357310+0000","last_active":"2020-08-07T15:08:19.719122+0000","last_peered":"2020-08-07T15:08:19.719122+0000","last_clean":"2020-08-07T15:08:19.719122+0000","last_became_active":"2020-08-07T15:07:51.356798+0000","last_became_peered":"2020-08-07T15:07:51.356798+0000","last_unstale":"2020-08-07T15:08:19.719122+0000","last_undegraded":"2020-08-07T15:08:19.719122+0000","last_fullsized":"2020-08-07T15:08:19.719122+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.927824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.927824+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"70","reported_epoch":"78","state":"active+clean","last_fresh":"2020-08-07T15:08:19.514134+0000","last_change":"2020-08-07T15:08:15.887617+0000","last_active":"2020-08-07T15:08:19.514134+0000","last_peered":"2020-08-07T15:08:19.514134+0000","last_clean":"2020-08-07T15:08:19.514134+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:08:19.514134+0000","last_undegraded":"2020-08-07T15:08:19.514134+0000","last_fullsized":"2020-08-07T15:08:19.514134+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:15.887513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:15.887513+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:08:23.520041+0000","last_change":"2020-08-07T15:08:23.520041+0000","last_active":"2020-08-07T15:08:23.520041+0000","last_peered":"2020-08-07T15:08:23.520041+0000","last_clean":"2020-08-07T15:08:23.520041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:08:23.520041+0000","last_undegraded":"2020-08-07T15:08:23.520041+0000","last_fullsized":"2020-08-07T15:08:23.520041+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"59","reported_epoch":"78","state":"active+clean","last_fresh":"2020-08-07T15:08:19.718529+0000","last_change":"2020-08-07T15:08:15.279484+0000","last_active":"2020-08-07T15:08:19.718529+0000","last_peered":"2020-08-07T15:08:19.718529+0000","last_clean":"2020-08-07T15:08:19.718529+0000","last_became_active":"2020-08-07T15:07:33.170597+0000","last_became_peered":"2020-08-07T15:07:33.170597+0000","last_unstale":"2020-08-07T15:08:19.718529+0000","last_undegraded":"2020-08-07T15:08:19.718529+0000","last_fullsized":"2020-08-07T15:08:19.718529+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:15.279411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:15.279411+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"70","reported_epoch":"78","state":"active+clean","last_fresh":"2020-08-07T15:08:19.718645+0000","last_change":"2020-08-07T15:06:23.074962+0000","last_active":"2020-08-07T15:08:19.718645+0000","last_peered":"2020-08-07T15:08:19.718645+0000","last_clean":"2020-08-07T15:08:19.718645+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:08:19.718645+0000","last_undegraded":"2020-08-07T15:08:19.718645+0000","last_fullsized":"2020-08-07T15:08:19.718645+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:08:23.520041+0000","last_change":"2020-08-07T15:08:23.520041+0000","last_active":"2020-08-07T15:08:23.520041+0000","last_peered":"2020-08-07T15:08:23.520041+0000","last_clean":"2020-08-07T15:08:23.520041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:08:23.520041+0000","last_undegraded":"2020-08-07T15:08:23.520041+0000","last_fullsized":"2020-08-07T15:08:23.520041+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"109","reported_epoch":"78","state":"active+clean","last_fresh":"2020-08-07T15:08:19.514026+0000","last_change":"2020-08-07T15:07:36.358882+0000","last_active":"2020-08-07T15:08:19.514026+0000","last_peered":"2020-08-07T15:08:19.514026+0000","last_clean":"2020-08-07T15:08:19.514026+0000","last_became_active":"2020-08-07T15:07:33.171449+0000","last_became_peered":"2020-08-07T15:07:33.171449+0000","last_unstale":"2020-08-07T15:08:19.514026+0000","last_undegraded":"2020-08-07T15:08:19.514026+0000","last_fullsized":"2020-08-07T15:08:19.514026+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.358798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.358798+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"74","reported_epoch":"79","state":"active+clean","last_fresh":"2020-08-07T15:08:21.278940+0000","last_change":"2020-08-07T15:07:36.864560+0000","last_active":"2020-08-07T15:08:21.278940+0000","last_peered":"2020-08-07T15:08:21.278940+0000","last_clean":"2020-08-07T15:08:21.278940+0000","last_became_active":"2020-08-07T15:06:23.223489+0000","last_became_peered":"2020-08-07T15:06:23.223489+0000","last_unstale":"2020-08-07T15:08:21.278940+0000","last_undegraded":"2020-08-07T15:08:21.278940+0000","last_fullsized":"2020-08-07T15:08:21.278940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:07:36.864483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.864483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:08:23.520041+0000","last_change":"2020-08-07T15:08:23.520041+0000","last_active":"2020-08-07T15:08:23.520041+0000","last_peered":"2020-08-07T15:08:23.520041+0000","last_clean":"2020-08-07T15:08:23.520041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:08:23.520041+0000","last_undegraded":"2020-08-07T15:08:23.520041+0000","last_fullsized":"2020-08-07T15:08:23.520041+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"70","reported_epoch":"78","state":"active+clean","last_fresh":"2020-08-07T15:08:19.718614+0000","last_change":"2020-08-07T15:06:23.075958+0000","last_active":"2020-08-07T15:08:19.718614+0000","last_peered":"2020-08-07T15:08:19.718614+0000","last_clean":"2020-08-07T15:08:19.718614+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:08:19.718614+0000","last_undegraded":"2020-08-07T15:08:19.718614+0000","last_fullsized":"2020-08-07T15:08:19.718614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"93","reported_epoch":"78","state":"active+clean","last_fresh":"2020-08-07T15:08:19.514541+0000","last_change":"2020-08-07T15:07:32.461010+0000","last_active":"2020-08-07T15:08:19.514541+0000","last_peered":"2020-08-07T15:08:19.514541+0000","last_clean":"2020-08-07T15:08:19.514541+0000","last_became_active":"2020-08-07T15:07:28.210245+0000","last_became_peered":"2020-08-07T15:07:28.210245+0000","last_unstale":"2020-08-07T15:08:19.514541+0000","last_undegraded":"2020-08-07T15:08:19.514541+0000","last_fullsized":"2020-08-07T15:08:19.514541+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:32.460945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:32.460945+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:08:23.520041+0000","last_change":"2020-08-07T15:08:23.520041+0000","last_active":"2020-08-07T15:08:23.520041+0000","last_peered":"2020-08-07T15:08:23.520041+0000","last_clean":"2020-08-07T15:08:23.520041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:08:23.520041+0000","last_undegraded":"2020-08-07T15:08:23.520041+0000","last_fullsized":"2020-08-07T15:08:23.520041+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"66","reported_epoch":"78","state":"active+clean","last_fresh":"2020-08-07T15:08:19.718766+0000","last_change":"2020-08-07T15:08:02.901198+0000","last_active":"2020-08-07T15:08:19.718766+0000","last_peered":"2020-08-07T15:08:19.718766+0000","last_clean":"2020-08-07T15:08:19.718766+0000","last_became_active":"2020-08-07T15:07:33.156569+0000","last_became_peered":"2020-08-07T15:07:33.156569+0000","last_unstale":"2020-08-07T15:08:19.718766+0000","last_undegraded":"2020-08-07T15:08:19.718766+0000","last_fullsized":"2020-08-07T15:08:19.718766+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:02.901115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:02.901115+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"73","reported_epoch":"78","state":"active+clean","last_fresh":"2020-08-07T15:08:19.514488+0000","last_change":"2020-08-07T15:08:13.867355+0000","last_active":"2020-08-07T15:08:19.514488+0000","last_peered":"2020-08-07T15:08:19.514488+0000","last_clean":"2020-08-07T15:08:19.514488+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:08:19.514488+0000","last_undegraded":"2020-08-07T15:08:19.514488+0000","last_fullsized":"2020-08-07T15:08:19.514488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:13.867148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:13.867148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:08:23.520041+0000","last_change":"2020-08-07T15:08:23.520041+0000","last_active":"2020-08-07T15:08:23.520041+0000","last_peered":"2020-08-07T15:08:23.520041+0000","last_clean":"2020-08-07T15:08:23.520041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:08:23.520041+0000","last_undegraded":"2020-08-07T15:08:23.520041+0000","last_fullsized":"2020-08-07T15:08:23.520041+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"66","reported_epoch":"78","state":"active+clean","last_fresh":"2020-08-07T15:08:19.718975+0000","last_change":"2020-08-07T15:08:09.841524+0000","last_active":"2020-08-07T15:08:19.718975+0000","last_peered":"2020-08-07T15:08:19.718975+0000","last_clean":"2020-08-07T15:08:19.718975+0000","last_became_active":"2020-08-07T15:07:33.169518+0000","last_became_peered":"2020-08-07T15:07:33.169518+0000","last_unstale":"2020-08-07T15:08:19.718975+0000","last_undegraded":"2020-08-07T15:08:19.718975+0000","last_fullsized":"2020-08-07T15:08:19.718975+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:09.841428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:09.841428+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"70","reported_epoch":"78","state":"active+clean","last_fresh":"2020-08-07T15:08:19.514348+0000","last_change":"2020-08-07T15:06:23.225886+0000","last_active":"2020-08-07T15:08:19.514348+0000","last_peered":"2020-08-07T15:08:19.514348+0000","last_clean":"2020-08-07T15:08:19.514348+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:08:19.514348+0000","last_undegraded":"2020-08-07T15:08:19.514348+0000","last_fullsized":"2020-08-07T15:08:19.514348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:08:23.520041+0000","last_change":"2020-08-07T15:08:23.520041+0000","last_active":"2020-08-07T15:08:23.520041+0000","last_peered":"2020-08-07T15:08:23.520041+0000","last_clean":"2020-08-07T15:08:23.520041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:08:23.520041+0000","last_undegraded":"2020-08-07T15:08:23.520041+0000","last_fullsized":"2020-08-07T15:08:23.520041+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"47","reported_epoch":"79","state":"active+clean","last_fresh":"2020-08-07T15:08:20.513196+0000","last_change":"2020-08-07T15:08:15.585668+0000","last_active":"2020-08-07T15:08:20.513196+0000","last_peered":"2020-08-07T15:08:20.513196+0000","last_clean":"2020-08-07T15:08:20.513196+0000","last_became_active":"2020-08-07T15:07:33.155139+0000","last_became_peered":"2020-08-07T15:07:33.155139+0000","last_unstale":"2020-08-07T15:08:20.513196+0000","last_undegraded":"2020-08-07T15:08:20.513196+0000","last_fullsized":"2020-08-07T15:08:20.513196+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:15.585597+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":595,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"73","reported_epoch":"78","state":"active+clean","last_fresh":"2020-08-07T15:08:19.510926+0000","last_change":"2020-08-07T15:07:34.423236+0000","last_active":"2020-08-07T15:08:19.510926+0000","last_peered":"2020-08-07T15:08:19.510926+0000","last_clean":"2020-08-07T15:08:19.510926+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:08:19.510926+0000","last_undegraded":"2020-08-07T15:08:19.510926+0000","last_fullsized":"2020-08-07T15:08:19.510926+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:07:34.423115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:07:34.423115+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:08:23.520041+0000","last_change":"2020-08-07T15:08:23.520041+0000","last_active":"2020-08-07T15:08:23.520041+0000","last_peered":"2020-08-07T15:08:23.520041+0000","last_clean":"2020-08-07T15:08:23.520041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:08:23.520041+0000","last_undegraded":"2020-08-07T15:08:23.520041+0000","last_fullsized":"2020-08-07T15:08:23.520041+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"53","reported_epoch":"79","state":"active+clean","last_fresh":"2020-08-07T15:08:20.658280+0000","last_change":"2020-08-07T15:07:34.653928+0000","last_active":"2020-08-07T15:08:20.658280+0000","last_peered":"2020-08-07T15:08:20.658280+0000","last_clean":"2020-08-07T15:08:20.658280+0000","last_became_active":"2020-08-07T15:07:33.161761+0000","last_became_peered":"2020-08-07T15:07:33.161761+0000","last_unstale":"2020-08-07T15:08:20.658280+0000","last_undegraded":"2020-08-07T15:08:20.658280+0000","last_fullsized":"2020-08-07T15:08:20.658280+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:34.653843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:34.653843+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"73","reported_epoch":"78","state":"active+clean","last_fresh":"2020-08-07T15:08:19.719212+0000","last_change":"2020-08-07T15:08:11.840861+0000","last_active":"2020-08-07T15:08:19.719212+0000","last_peered":"2020-08-07T15:08:19.719212+0000","last_clean":"2020-08-07T15:08:19.719212+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:08:19.719212+0000","last_undegraded":"2020-08-07T15:08:19.719212+0000","last_fullsized":"2020-08-07T15:08:19.719212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:11.840807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:11.840807+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":595,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":33,"acting":33,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542171,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049464,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322376,"statfs":{"total":96636764160,"available":95562113024,"internally_reserved":1073741824,"allocated":901120,"data_stored":353278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.875},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.83299999999999996}]},{"osd":1,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":5.7519999999999998,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":0.621},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":0.79200000000000004}]},{"osd":2,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":1.056}]},{"osd":3,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":5.5019999999999998,"5min":5.5019999999999998,"15min":5.5019999999999998},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.97999999999999998}]},{"osd":4,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":2.9790000000000001,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":1.264}]},{"osd":5,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":37.506999999999998,"5min":37.506999999999998,"15min":37.506999999999998},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":831.88099999999997,"5min":831.88099999999997,"15min":831.88099999999997},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":1.0860000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.518,"15min":1.518},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":3.1389999999999998,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":3.048,"5min":3.048,"15min":3.048},"last":1.232}]}]},{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049656,"kb_used_data":1072,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322184,"statfs":{"total":96636764160,"available":95561916416,"internally_reserved":1073741824,"allocated":1097728,"data_stored":549886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.2490000000000001,"15min":1.3049999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.6439999999999999,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.079,"5min":1.135,"15min":1.145},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":2.9950000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":0.95699999999999996}]},{"osd":1,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.95999999999999996,"15min":0.95799999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.9890000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":19.103999999999999,"5min":4.9299999999999997,"15min":2.5680000000000001},"min":{"1min":0.75600000000000001,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":372.50799999999998,"5min":372.50799999999998,"15min":372.50799999999998},"last":0.90100000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2,"15min":1.196},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":3.036,"5min":3.036,"15min":3.036},"last":0.65700000000000003},{"interface":"front","average":{"1min":1.093,"5min":1.236,"15min":1.26},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.7440000000000002,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":0.872}]},{"osd":3,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.4870000000000001,"15min":1.5129999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":3.46,"5min":3.46,"15min":3.46},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.268,"5min":1.3640000000000001,"15min":1.3799999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":3.1970000000000001,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.85099999999999998}]},{"osd":4,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.5880000000000001,"15min":1.6379999999999999},"min":{"1min":0.72499999999999998,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.7999999999999998,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.653,"5min":1.6679999999999999,"15min":1.6699999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":3.694,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":0.98099999999999998}]},{"osd":5,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.3260000000000001,"15min":1.351},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.6360000000000001,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.393,"5min":1.6910000000000001,"15min":1.7410000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.8279999999999998,"5min":5.327,"15min":5.327},"last":1.149}]},{"osd":6,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.446,"15min":1.4370000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.7799999999999998,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.4730000000000001,"5min":14.955,"15min":17.202000000000002},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":3.7309999999999999,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.202}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049460,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322380,"statfs":{"total":96636764160,"available":95562117120,"internally_reserved":1073741824,"allocated":897024,"data_stored":352450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":27.559999999999999,"15min":31.977},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.1820000000000004,"5min":497.04000000000002,"15min":497.04000000000002},"last":1.1519999999999999},{"interface":"front","average":{"1min":44.500999999999998,"5min":9.8049999999999997,"15min":4.0220000000000002},"min":{"1min":0.64200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":993.84500000000003,"5min":993.84500000000003,"15min":993.84500000000003},"last":1.204}]},{"osd":2,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":0.98799999999999999,"15min":0.92800000000000005},"min":{"1min":0.57399999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":4.585,"5min":4.585,"15min":4.585},"last":1.252},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.2410000000000001,"15min":1.216},"min":{"1min":0.51800000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":4.149,"5min":4.149,"15min":4.149},"last":1.131}]},{"osd":3,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.962,"5min":1.4410000000000001,"15min":1.3540000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":10.603999999999999,"5min":10.603999999999999,"15min":10.603999999999999},"last":1.5609999999999999},{"interface":"front","average":{"1min":2.1909999999999998,"5min":1.601,"15min":1.5029999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":10.638999999999999,"5min":10.638999999999999,"15min":10.638999999999999},"last":1.5880000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":2.1869999999999998,"5min":1.464,"15min":1.3440000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":10.69,"5min":10.69,"15min":10.69},"last":1.23},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.264,"15min":1.2210000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":4.9500000000000002,"5min":4.9500000000000002,"15min":4.9500000000000002},"last":1.4530000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":2.0630000000000002,"5min":10.819000000000001,"15min":12.278},"min":{"1min":0.73399999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":10.720000000000001,"5min":213.63800000000001,"15min":213.63800000000001},"last":1.173},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.258,"15min":1.212},"min":{"1min":0.82899999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":4.6269999999999998,"5min":4.6269999999999998,"15min":4.6269999999999998},"last":1.49}]},{"osd":6,"last update":"Fri Aug 7 15:07:22 2020","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.552,"15min":1.552},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":3.234,"5min":3.234,"15min":3.234},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.361,"5min":1.361,"15min":1.361},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":1.5289999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.8129999999999999,"5min":1.6539999999999999,"15min":1.6279999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":4.5549999999999997,"5min":4.5549999999999997,"15min":4.5549999999999997},"last":1.278},{"interface":"front","average":{"1min":2.2719999999999998,"5min":1.6379999999999999,"15min":1.532},"min":{"1min":0.998,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":10.752000000000001,"5min":10.752000000000001,"15min":10.752000000000001},"last":1.6879999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049460,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322380,"statfs":{"total":96636764160,"available":95562117120,"internally_reserved":1073741824,"allocated":897024,"data_stored":352700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.4500000000000002,"5min":2.4500000000000002,"15min":2.4500000000000002},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.2440000000000002,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":0.81599999999999995}]},{"osd":2,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":32.740000000000002,"5min":32.740000000000002,"15min":32.740000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":606.18499999999995,"5min":606.18499999999995,"15min":606.18499999999995},"last":0.90600000000000003}]},{"osd":3,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.0029999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.6949999999999998,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":1.6339999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":2.4060000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.4650000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":1.373},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.661}]},{"osd":6,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":8.6140000000000008,"5min":8.6140000000000008,"15min":8.6140000000000008},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":124.738,"5min":124.738,"15min":124.738},"last":1.4450000000000001},{"interface":"front","average":{"1min":18.652999999999999,"5min":18.652999999999999,"15min":18.652999999999999},"min":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"max":{"1min":294.94,"5min":294.94,"15min":294.94},"last":1.4119999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:07:20 2020","interfaces":[{"interface":"back","average":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":6.468,"5min":6.468,"15min":6.468},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.883,"5min":2.883,"15min":2.883},"last":1.6000000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049472,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322368,"statfs":{"total":96636764160,"available":95562104832,"internally_reserved":1073741824,"allocated":909312,"data_stored":356636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.153,"15min":1.129},"min":{"1min":0.505,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.4909999999999997,"5min":4.4909999999999997,"15min":4.4909999999999997},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.284,"5min":1.036,"15min":0.995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":3.6520000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.88200000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":2.5219999999999998,"5min":1.3779999999999999,"15min":1.1870000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":18.672000000000001,"5min":18.672000000000001,"15min":18.672000000000001},"last":0.995},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.357,"15min":1.359},"min":{"1min":0.70699999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":5.1130000000000004,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":1.04}]},{"osd":3,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.216,"15min":1.1799999999999999},"min":{"1min":0.65300000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.419,"5min":1.528,"15min":1.546},"min":{"1min":0.59999999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":5.3339999999999996,"5min":8.2210000000000001,"15min":8.2210000000000001},"last":1.085}]},{"osd":4,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":2.3319999999999999,"5min":1.3540000000000001,"15min":1.1910000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":19.783000000000001,"5min":19.783000000000001,"15min":19.783000000000001},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.5589999999999999,"15min":1.615},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.891,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":1.1180000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.2789999999999999,"15min":1.2669999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.7730000000000001,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.5249999999999999,"5min":20.725000000000001,"15min":23.925000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":4.4790000000000001,"5min":415.40699999999998,"15min":415.40699999999998},"last":1.431}]},{"osd":6,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.331},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":1.266}]},{"osd":7,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":3.5409999999999999,"5min":2.101,"15min":1.861},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":35.548999999999999,"5min":35.548999999999999,"15min":35.548999999999999},"last":1.2330000000000001},{"interface":"front","average":{"1min":3.4460000000000002,"5min":1.8859999999999999,"15min":1.6259999999999999},"min":{"1min":1.004,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":35.646999999999998,"5min":35.646999999999998,"15min":35.646999999999998},"last":1.381}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1068100,"kb_used_data":19516,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93303740,"statfs":{"total":96636764160,"available":95543029760,"internally_reserved":1073741824,"allocated":19984384,"data_stored":19428190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.044,"5min":1.411,"15min":1.3049999999999999},"min":{"1min":0.55600000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":8.6020000000000003,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.599,"15min":1.6240000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":4.9119999999999999,"5min":13.067,"15min":13.067},"last":0.50600000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":16.382999999999999,"5min":4.7949999999999999,"15min":2.8639999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":270.68700000000001,"5min":270.68700000000001,"15min":270.68700000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":2.7210000000000001,"5min":2.153,"15min":2.0590000000000002},"min":{"1min":0.71599999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":14.349,"5min":14.349,"15min":14.349},"last":0.82299999999999995}]},{"osd":2,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":1.7470000000000001,"5min":1.2669999999999999,"15min":1.1870000000000001},"min":{"1min":0.64900000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":4.9580000000000002,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":0.86199999999999999},{"interface":"front","average":{"1min":4.5220000000000002,"5min":2.2519999999999998,"15min":1.8740000000000001},"min":{"1min":0.72399999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":51.530000000000001,"5min":51.530000000000001,"15min":51.530000000000001},"last":0.78200000000000003}]},{"osd":4,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":47.652999999999999,"5min":10.888,"15min":4.7599999999999998},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":506.21300000000002,"5min":506.21300000000002,"15min":506.21300000000002},"last":0.94099999999999995},{"interface":"front","average":{"1min":29.771000000000001,"5min":31.634,"15min":31.943999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":505.738,"5min":677.88599999999997,"15min":677.88599999999997},"last":0.91600000000000004}]},{"osd":5,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.012,"5min":1.6160000000000001,"15min":1.55},"min":{"1min":0.96399999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":5.0049999999999999,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":2.2210000000000001,"5min":1.859,"15min":1.7989999999999999},"min":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":5.226,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":1.234}]},{"osd":6,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":1.99,"5min":1.623,"15min":1.5620000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":5.0389999999999997,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":1.27},{"interface":"front","average":{"1min":1.9710000000000001,"5min":1.9830000000000001,"15min":1.9850000000000001},"min":{"1min":1.0369999999999999,"5min":0.876,"15min":0.876},"max":{"1min":4.266,"5min":5.6189999999999998,"15min":5.6189999999999998},"last":1.113}]},{"osd":7,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.0110000000000001,"5min":2.1309999999999998,"15min":2.1509999999999998},"min":{"1min":0.96999999999999997,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":3.79,"5min":5.734,"15min":5.734},"last":0.99199999999999999},{"interface":"front","average":{"1min":2.6200000000000001,"5min":2.105,"15min":2.0190000000000001},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":8.6790000000000003,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":1.0580000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049464,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322376,"statfs":{"total":96636764160,"available":95562113024,"internally_reserved":1073741824,"allocated":901120,"data_stored":353278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":1.125,"15min":1.1619999999999999},"min":{"1min":0.47699999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.3999999999999999,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.349,"5min":0.94899999999999995,"15min":0.88200000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":6.3970000000000002,"5min":6.3970000000000002,"15min":6.3970000000000002},"last":1.0820000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.304,"15min":1.337},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.4380000000000002,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":0.93700000000000006},{"interface":"front","average":{"1min":32.585999999999999,"5min":7.6180000000000003,"15min":3.4569999999999999},"min":{"1min":0.67000000000000004,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":689.846,"5min":689.846,"15min":689.846},"last":1.036}]},{"osd":2,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.46,"15min":1.454},"min":{"1min":0.56399999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":6.8730000000000002,"5min":6.8730000000000002,"15min":6.8730000000000002},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.5669999999999999,"15min":1.532},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":7.1360000000000001,"5min":7.1360000000000001,"15min":7.1360000000000001},"last":1.4390000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":12.619999999999999,"5min":3.73,"15min":2.2480000000000002},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":247.202,"5min":247.202,"15min":247.202},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.583,"5min":1.3169999999999999,"15min":1.2729999999999999},"min":{"1min":0.67800000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":6.431,"5min":6.431,"15min":6.431},"last":0.96199999999999997}]},{"osd":5,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":46.786999999999999,"5min":10.659000000000001,"15min":4.6379999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":990.71600000000001,"5min":990.71600000000001,"15min":990.71600000000001},"last":1.337},{"interface":"front","average":{"1min":1.748,"5min":1.651,"15min":1.635},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":3.9329999999999998,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":1.361}]},{"osd":6,"last update":"Fri Aug 7 15:07:18 2020","interfaces":[{"interface":"back","average":{"1min":1.79,"5min":1.79,"15min":1.79},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":5.1310000000000002,"5min":5.1310000000000002,"15min":5.1310000000000002},"last":1.482},{"interface":"front","average":{"1min":19.109000000000002,"5min":19.109000000000002,"15min":19.109000000000002},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":389.14600000000002,"5min":389.14600000000002,"15min":389.14600000000002},"last":1.3120000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":13.378,"5min":3.802,"15min":2.206},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":252.798,"5min":252.798,"15min":252.798},"last":1.3959999999999999},{"interface":"front","average":{"1min":44.676000000000002,"5min":10.252000000000001,"15min":4.5140000000000002},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":501.93000000000001,"5min":501.93000000000001,"15min":501.93000000000001},"last":1.5860000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049656,"kb_used_data":1072,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322184,"statfs":{"total":96636764160,"available":95561916416,"internally_reserved":1073741824,"allocated":1097728,"data_stored":549886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.766,"5min":2.766,"15min":2.766},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.86899999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.6509999999999998,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":2.746,"5min":2.746,"15min":2.746},"last":0.72399999999999998}]},{"osd":2,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":9.2200000000000006,"5min":9.2200000000000006,"15min":9.2200000000000006},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":168.56899999999999,"5min":168.56899999999999,"15min":168.56899999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":33.131,"5min":33.131,"15min":33.131},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":676.27099999999996,"5min":676.27099999999996,"15min":676.27099999999996},"last":0.82199999999999995}]},{"osd":3,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.7669999999999999,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.964,"5min":2.964,"15min":2.964},"last":1.018}]},{"osd":4,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":50.893000000000001,"5min":50.893000000000001,"15min":50.893000000000001},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":530.30799999999999,"5min":530.30799999999999,"15min":530.30799999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":3.1080000000000001,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.98999999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":50.408999999999999,"5min":50.408999999999999,"15min":50.408999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":754.31299999999999,"5min":754.31299999999999,"15min":754.31299999999999},"last":1.1459999999999999},{"interface":"front","average":{"1min":26.016999999999999,"5min":26.016999999999999,"15min":26.016999999999999},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":517.26700000000005,"5min":517.26700000000005,"15min":517.26700000000005},"last":1.002}]},{"osd":7,"last update":"Fri Aug 7 15:07:21 2020","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.034},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"min":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":3.5649999999999999,"5min":3.5649999999999999,"15min":3.5649999999999999},"last":1.1279999999999999}]}]}]}} 2020-08-07T15:08:24.368 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:24.369 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:24.372 INFO:tasks.thrashosds.thrasher:Setting 4.9 to [6, 7, 2, 3, 4, 5] 2020-08-07T15:08:24.373 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '4.9', '6', '7', '2', '3', '4', '5'] 2020-08-07T15:08:24.373 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pg-upmap-items 4.9 6 7 2 3 4 5 2020-08-07T15:08:24.376 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:08:24.463 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:24.461+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:24.551 INFO:teuthology.orchestra.run.smithi074.stderr:enabled application 'rados' on pool 'unique_pool_1' 2020-08-07T15:08:24.551 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:24.552 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:24.555 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:24.549+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:24.567 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:08:24.654 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:24.651+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:24.662 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:08:24.728 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:24.729 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:24.739 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:08:24.788 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:24.784+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:24.811 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:08:24.833 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:08:24.856 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:08:24.856 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:24.854+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:24.869 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:24.870 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:24.878 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:08:24.879 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:08:24.903 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:08:24.930 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:08:24.953 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:08:24.980 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:08:24.998 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:08:24.998 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:08:24.998 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:08:24.999 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:08:24.999 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:08:24.999 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:08:24.999 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:08:24.999 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:08:25.000 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:24.972+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:25.047 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:25.047 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:08:25.048 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:08:25.048 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:08:25.048 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:25.059 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:08:25.059 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:25.057+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:25.161 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:25.160+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:25.237 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:25.238 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:08:25.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:08:25.238 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:08:25.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.239 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:8112 3.9 3:9b275d48:::benchmark_data_smithi074_35447_object1013:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:08:25.239 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:41.083981+0000", 2020-08-07T15:08:25.239 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 104.137921418, 2020-08-07T15:08:25.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31911657999999998, 2020-08-07T15:08:25.242 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:25.242 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:25.243 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:25.243 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:25.245 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:25.245 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 8112 2020-08-07T15:08:25.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.246 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:25.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:25.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.083981+0000", 2020-08-07T15:08:25.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:25.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:25.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.083981+0000", 2020-08-07T15:08:25.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4051999999999999e-05 2020-08-07T15:08:25.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:25.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084015+0000", 2020-08-07T15:08:25.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7290000000000002e-06 2020-08-07T15:08:25.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:25.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084019+0000", 2020-08-07T15:08:25.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2280000000000001e-06 2020-08-07T15:08:25.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:25.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084026+0000", 2020-08-07T15:08:25.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7780000000000001e-06 2020-08-07T15:08:25.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:25.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084030+0000", 2020-08-07T15:08:25.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00077372300000000003 2020-08-07T15:08:25.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:25.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084803+0000", 2020-08-07T15:08:25.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1502000000000003e-05 2020-08-07T15:08:25.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:25.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084855+0000", 2020-08-07T15:08:25.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5110000000000002e-05 2020-08-07T15:08:25.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:08:25.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.084950+0000", 2020-08-07T15:08:25.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012531370000000001 2020-08-07T15:08:25.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:25.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.086203+0000", 2020-08-07T15:08:25.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4799999999999998e-07 2020-08-07T15:08:25.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:25.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.086204+0000", 2020-08-07T15:08:25.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.905e-05 2020-08-07T15:08:25.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:25.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.086253+0000", 2020-08-07T15:08:25.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00062940999999999995 2020-08-07T15:08:25.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:25.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.086882+0000", 2020-08-07T15:08:25.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31612944199999998 2020-08-07T15:08:25.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:25.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.403012+0000", 2020-08-07T15:08:25.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9099999999999991e-07 2020-08-07T15:08:25.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:25.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.403013+0000", 2020-08-07T15:08:25.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.488e-05 2020-08-07T15:08:25.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:25.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:41.403097+0000", 2020-08-07T15:08:25.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31911657999999998 2020-08-07T15:08:25.294 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:25.294 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:25.295 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:25.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.296 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:16223 3.9 3:9c72cb85:::benchmark_data_smithi074_35447_object2027:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:08:25.306 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:47.106937+0000", 2020-08-07T15:08:25.307 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 98.114964811999997, 2020-08-07T15:08:25.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.48645846999999998, 2020-08-07T15:08:25.307 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:25.308 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:25.308 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:25.308 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:25.309 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:25.309 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 16223 2020-08-07T15:08:25.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.310 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:25.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:25.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.106937+0000", 2020-08-07T15:08:25.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:25.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:25.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.106937+0000", 2020-08-07T15:08:25.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8390999999999998e-05 2020-08-07T15:08:25.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:25.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.106986+0000", 2020-08-07T15:08:25.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0120000000000002e-06 2020-08-07T15:08:25.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:25.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.106992+0000", 2020-08-07T15:08:25.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3148e-05 2020-08-07T15:08:25.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:25.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.107005+0000", 2020-08-07T15:08:25.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5719999999999999e-06 2020-08-07T15:08:25.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:25.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.107012+0000", 2020-08-07T15:08:25.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043495110000000003 2020-08-07T15:08:25.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:25.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.111361+0000", 2020-08-07T15:08:25.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0105999999999997e-05 2020-08-07T15:08:25.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:25.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.111421+0000", 2020-08-07T15:08:25.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012232399999999999 2020-08-07T15:08:25.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:08:25.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.111544+0000", 2020-08-07T15:08:25.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003099367 2020-08-07T15:08:25.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:25.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.114643+0000", 2020-08-07T15:08:25.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3099999999999997e-07 2020-08-07T15:08:25.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:25.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.114644+0000", 2020-08-07T15:08:25.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7627000000000003e-05 2020-08-07T15:08:25.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:25.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.114691+0000", 2020-08-07T15:08:25.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005765905 2020-08-07T15:08:25.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:25.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.120457+0000", 2020-08-07T15:08:25.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.47285765200000002 2020-08-07T15:08:25.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:25.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.593315+0000", 2020-08-07T15:08:25.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1969999999999999e-06 2020-08-07T15:08:25.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:25.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.593316+0000", 2020-08-07T15:08:25.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0026999999999998e-05 2020-08-07T15:08:25.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:25.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.593396+0000", 2020-08-07T15:08:25.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.48645846999999998 2020-08-07T15:08:25.396 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:25.396 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:25.397 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:25.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.397 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:16224 3.9 3:9c72cb85:::benchmark_data_smithi074_35447_object2027:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:08:25.397 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:47.108015+0000", 2020-08-07T15:08:25.398 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 98.113887074999994, 2020-08-07T15:08:25.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.512944332, 2020-08-07T15:08:25.398 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:25.398 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:25.398 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:25.399 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:25.399 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:25.408 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 16224 2020-08-07T15:08:25.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.409 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:25.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:25.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108015+0000", 2020-08-07T15:08:25.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:25.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:25.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108015+0000", 2020-08-07T15:08:25.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7508e-05 2020-08-07T15:08:25.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:25.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108063+0000", 2020-08-07T15:08:25.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.485e-06 2020-08-07T15:08:25.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:25.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108068+0000", 2020-08-07T15:08:25.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3387000000000001e-05 2020-08-07T15:08:25.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:25.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108082+0000", 2020-08-07T15:08:25.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1129999999999997e-06 2020-08-07T15:08:25.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:25.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.108088+0000", 2020-08-07T15:08:25.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056051920000000002 2020-08-07T15:08:25.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:25.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.113693+0000", 2020-08-07T15:08:25.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9973000000000001e-05 2020-08-07T15:08:25.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:25.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.113783+0000", 2020-08-07T15:08:25.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001649 2020-08-07T15:08:25.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:08:25.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.113948+0000", 2020-08-07T15:08:25.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001023007 2020-08-07T15:08:25.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:25.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.114971+0000", 2020-08-07T15:08:25.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2600000000000001e-07 2020-08-07T15:08:25.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:25.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.114972+0000", 2020-08-07T15:08:25.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9228999999999998e-05 2020-08-07T15:08:25.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:25.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.115031+0000", 2020-08-07T15:08:25.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0063345550000000004 2020-08-07T15:08:25.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:25.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.121365+0000", 2020-08-07T15:08:25.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.49952866099999998 2020-08-07T15:08:25.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:25.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.620894+0000", 2020-08-07T15:08:25.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9299999999999996e-07 2020-08-07T15:08:25.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:25.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.620895+0000", 2020-08-07T15:08:25.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4503000000000002e-05 2020-08-07T15:08:25.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:25.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.620960+0000", 2020-08-07T15:08:25.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.512944332 2020-08-07T15:08:25.466 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:25.466 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:25.466 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:25.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.474 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:33728 3.1 3:8fce2265:::benchmark_data_smithi074_35447_object4215:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-08-07T15:08:25.475 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:58.156056+0000", 2020-08-07T15:08:25.475 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.065846673999999, 2020-08-07T15:08:25.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26861316299999999, 2020-08-07T15:08:25.477 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:25.477 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:25.478 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:25.478 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:25.517 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:25.517 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 33728 2020-08-07T15:08:25.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.519 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:25.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:25.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156056+0000", 2020-08-07T15:08:25.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:25.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:25.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156056+0000", 2020-08-07T15:08:25.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6795999999999998e-05 2020-08-07T15:08:25.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:25.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156092+0000", 2020-08-07T15:08:25.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4690000000000002e-06 2020-08-07T15:08:25.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:25.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156096+0000", 2020-08-07T15:08:25.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5270000000000004e-06 2020-08-07T15:08:25.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:25.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156102+0000", 2020-08-07T15:08:25.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3129999999999999e-06 2020-08-07T15:08:25.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:25.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.156106+0000", 2020-08-07T15:08:25.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00337032 2020-08-07T15:08:25.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:25.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.159476+0000", 2020-08-07T15:08:25.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104874 2020-08-07T15:08:25.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:25.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.159581+0000", 2020-08-07T15:08:25.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016822300000000001 2020-08-07T15:08:25.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,3", 2020-08-07T15:08:25.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.159749+0000", 2020-08-07T15:08:25.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020156620000000001 2020-08-07T15:08:25.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:25.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.161765+0000", 2020-08-07T15:08:25.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1699999999999997e-07 2020-08-07T15:08:25.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:25.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.161765+0000", 2020-08-07T15:08:25.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6606e-05 2020-08-07T15:08:25.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:25.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.161822+0000", 2020-08-07T15:08:25.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0060344459999999997 2020-08-07T15:08:25.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:25.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.167857+0000", 2020-08-07T15:08:25.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25676037099999999 2020-08-07T15:08:25.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:25.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424617+0000", 2020-08-07T15:08:25.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3370000000000001e-06 2020-08-07T15:08:25.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:25.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424618+0000", 2020-08-07T15:08:25.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0501999999999999e-05 2020-08-07T15:08:25.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:25.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.424669+0000", 2020-08-07T15:08:25.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26861316299999999 2020-08-07T15:08:25.603 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:25.604 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:25.604 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:25.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.605 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:34624 3.9 3:946cdb13:::benchmark_data_smithi074_35447_object4327:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-08-07T15:08:25.605 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:58.744745+0000", 2020-08-07T15:08:25.605 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.477157555000005, 2020-08-07T15:08:25.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28897188600000001, 2020-08-07T15:08:25.606 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:25.606 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:25.607 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:25.607 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:25.607 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:25.608 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 34624 2020-08-07T15:08:25.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.608 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:25.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:25.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744745+0000", 2020-08-07T15:08:25.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:25.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:25.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744745+0000", 2020-08-07T15:08:25.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1117e-05 2020-08-07T15:08:25.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:25.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744776+0000", 2020-08-07T15:08:25.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2140000000000001e-06 2020-08-07T15:08:25.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:25.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744782+0000", 2020-08-07T15:08:25.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1756999999999999e-05 2020-08-07T15:08:25.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:25.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744794+0000", 2020-08-07T15:08:25.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.674e-06 2020-08-07T15:08:25.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:25.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.744800+0000", 2020-08-07T15:08:25.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012818110000000001 2020-08-07T15:08:25.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:25.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.746082+0000", 2020-08-07T15:08:25.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2163999999999997e-05 2020-08-07T15:08:25.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:25.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.746134+0000", 2020-08-07T15:08:25.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107853 2020-08-07T15:08:25.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:08:25.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.746242+0000", 2020-08-07T15:08:25.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00119732 2020-08-07T15:08:25.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:25.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.747440+0000", 2020-08-07T15:08:25.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5799999999999998e-07 2020-08-07T15:08:25.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:25.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.747440+0000", 2020-08-07T15:08:25.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0883000000000001e-05 2020-08-07T15:08:25.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:25.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.747481+0000", 2020-08-07T15:08:25.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000272355 2020-08-07T15:08:25.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:25.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:58.747754+0000", 2020-08-07T15:08:25.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28589292999999999 2020-08-07T15:08:25.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:25.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.033647+0000", 2020-08-07T15:08:25.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6600000000000005e-07 2020-08-07T15:08:25.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:25.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.033647+0000", 2020-08-07T15:08:25.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9183999999999994e-05 2020-08-07T15:08:25.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:25.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.033717+0000", 2020-08-07T15:08:25.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28897188600000001 2020-08-07T15:08:25.677 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:25.684 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:25.685 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:25.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.685 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:64656 3.9 3:92365776:::benchmark_data_smithi074_35447_object8081:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e41)", 2020-08-07T15:08:25.686 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:20.118422+0000", 2020-08-07T15:08:25.686 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.103480715000003, 2020-08-07T15:08:25.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27269932899999999, 2020-08-07T15:08:25.686 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:25.686 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:25.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:25.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:25.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:25.687 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 64656 2020-08-07T15:08:25.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.688 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:25.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:25.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.118422+0000", 2020-08-07T15:08:25.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:25.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:25.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.118422+0000", 2020-08-07T15:08:25.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9726e-05 2020-08-07T15:08:25.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:25.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.118451+0000", 2020-08-07T15:08:25.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3759999999999999e-06 2020-08-07T15:08:25.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:25.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.118457+0000", 2020-08-07T15:08:25.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2139999999999996e-06 2020-08-07T15:08:25.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:25.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.118465+0000", 2020-08-07T15:08:25.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2420000000000002e-06 2020-08-07T15:08:25.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:25.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.118469+0000", 2020-08-07T15:08:25.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031527040000000001 2020-08-07T15:08:25.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.121622+0000", 2020-08-07T15:08:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4674999999999998e-05 2020-08-07T15:08:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:25.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.121706+0000", 2020-08-07T15:08:25.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000185775 2020-08-07T15:08:25.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:08:25.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.121892+0000", 2020-08-07T15:08:25.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003359384 2020-08-07T15:08:25.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:25.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.125252+0000", 2020-08-07T15:08:25.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2099999999999995e-07 2020-08-07T15:08:25.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:25.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.125252+0000", 2020-08-07T15:08:25.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4799999999999997e-05 2020-08-07T15:08:25.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:25.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.125307+0000", 2020-08-07T15:08:25.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3406000000000003e-05 2020-08-07T15:08:25.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:25.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.125361+0000", 2020-08-07T15:08:25.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26572362599999999 2020-08-07T15:08:25.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:25.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.391084+0000", 2020-08-07T15:08:25.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6899999999999996e-07 2020-08-07T15:08:25.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:25.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.391085+0000", 2020-08-07T15:08:25.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5811e-05 2020-08-07T15:08:25.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:25.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:20.391121+0000", 2020-08-07T15:08:25.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27269932899999999 2020-08-07T15:08:25.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:25.731 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:25.731 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:25.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.731 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:72616 3.6 e44/26 3:64e83676:::benchmark_data_smithi074_35447_object9076:head v 44'4848, mlcod=44'4843)", 2020-08-07T15:08:25.732 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:25.249399+0000", 2020-08-07T15:08:25.732 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 59.972503396999997, 2020-08-07T15:08:25.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27428339600000001, 2020-08-07T15:08:25.732 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:25.732 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:25.732 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:25.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:25.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.249399+0000", 2020-08-07T15:08:25.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:25.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:25.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.249399+0000", 2020-08-07T15:08:25.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0565000000000003e-05 2020-08-07T15:08:25.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:25.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.249429+0000", 2020-08-07T15:08:25.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3390000000000003e-06 2020-08-07T15:08:25.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:25.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.249434+0000", 2020-08-07T15:08:25.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27367070199999999 2020-08-07T15:08:25.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:25.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.523104+0000", 2020-08-07T15:08:25.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2100000000000001e-06 2020-08-07T15:08:25.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:25.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.523110+0000", 2020-08-07T15:08:25.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6372999999999997e-05 2020-08-07T15:08:25.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:25.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.523166+0000", 2020-08-07T15:08:25.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5656000000000001e-05 2020-08-07T15:08:25.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:25.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.523182+0000", 2020-08-07T15:08:25.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00046263499999999998 2020-08-07T15:08:25.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:25.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.523644+0000", 2020-08-07T15:08:25.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7916000000000001e-05 2020-08-07T15:08:25.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:25.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:25.523682+0000", 2020-08-07T15:08:25.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27428339600000001 2020-08-07T15:08:25.755 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:25.755 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:25.756 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:25.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.756 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:78744 3.6 e47/26 3:6151aa5d:::benchmark_data_smithi074_35447_object9842:head v 47'5304, mlcod=47'5297)", 2020-08-07T15:08:25.756 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:29.252471+0000", 2020-08-07T15:08:25.756 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.969431059999998, 2020-08-07T15:08:25.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26934327600000002, 2020-08-07T15:08:25.757 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:25.757 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:25.757 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:25.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:25.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.252471+0000", 2020-08-07T15:08:25.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:25.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:25.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.252471+0000", 2020-08-07T15:08:25.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2329e-05 2020-08-07T15:08:25.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:25.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.252524+0000", 2020-08-07T15:08:25.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0229999999999998e-06 2020-08-07T15:08:25.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:25.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.252530+0000", 2020-08-07T15:08:25.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26582335099999999 2020-08-07T15:08:25.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:25.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.518353+0000", 2020-08-07T15:08:25.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9689999999999999e-06 2020-08-07T15:08:25.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:25.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.518356+0000", 2020-08-07T15:08:25.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5710999999999997e-05 2020-08-07T15:08:25.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:25.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.518392+0000", 2020-08-07T15:08:25.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5387e-05 2020-08-07T15:08:25.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:25.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.518407+0000", 2020-08-07T15:08:25.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033617870000000002 2020-08-07T15:08:25.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:25.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.521769+0000", 2020-08-07T15:08:25.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5719000000000001e-05 2020-08-07T15:08:25.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:25.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:29.521814+0000", 2020-08-07T15:08:25.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26934327600000002 2020-08-07T15:08:25.767 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:25.768 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:25.768 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:25.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.769 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:86696 3.a e52/20 3:5058ba7c:::benchmark_data_smithi074_35447_object10836:head v 52'5704, mlcod=51'5695)", 2020-08-07T15:08:25.769 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.329685+0000", 2020-08-07T15:08:25.770 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.892217655000003, 2020-08-07T15:08:25.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.34216058999999999, 2020-08-07T15:08:25.770 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:25.770 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:25.771 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:25.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:25.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.329685+0000", 2020-08-07T15:08:25.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:25.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:25.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.329685+0000", 2020-08-07T15:08:25.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8568e-05 2020-08-07T15:08:25.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:25.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.329733+0000", 2020-08-07T15:08:25.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5509999999999999e-06 2020-08-07T15:08:25.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:25.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.329739+0000", 2020-08-07T15:08:25.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.337284427 2020-08-07T15:08:25.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:25.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.667023+0000", 2020-08-07T15:08:25.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7069999999999997e-06 2020-08-07T15:08:25.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:25.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.667029+0000", 2020-08-07T15:08:25.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5142999999999998e-05 2020-08-07T15:08:25.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:25.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.667074+0000", 2020-08-07T15:08:25.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7229000000000001e-05 2020-08-07T15:08:25.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:25.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.667091+0000", 2020-08-07T15:08:25.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046970060000000001 2020-08-07T15:08:25.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:25.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.671788+0000", 2020-08-07T15:08:25.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6959000000000001e-05 2020-08-07T15:08:25.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:25.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.671845+0000", 2020-08-07T15:08:25.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.34216058999999999 2020-08-07T15:08:25.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:25.790 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:25.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:25.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.791 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:104111 3.9 3:99770d45:::benchmark_data_smithi074_35447_object13013:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:08:25.791 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:45.858953+0000", 2020-08-07T15:08:25.791 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.362949407999999, 2020-08-07T15:08:25.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.29313854499999997, 2020-08-07T15:08:25.792 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:25.792 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:25.792 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:25.792 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:25.793 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:25.793 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 104111 2020-08-07T15:08:25.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.793 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:25.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:25.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858953+0000", 2020-08-07T15:08:25.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:25.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:25.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858953+0000", 2020-08-07T15:08:25.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8607000000000003e-05 2020-08-07T15:08:25.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:25.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.859011+0000", 2020-08-07T15:08:25.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9560000000000002e-06 2020-08-07T15:08:25.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:25.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.859017+0000", 2020-08-07T15:08:25.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2743e-05 2020-08-07T15:08:25.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:25.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.859030+0000", 2020-08-07T15:08:25.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2010000000000002e-06 2020-08-07T15:08:25.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:25.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.859037+0000", 2020-08-07T15:08:25.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001837827 2020-08-07T15:08:25.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:25.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.860875+0000", 2020-08-07T15:08:25.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2448999999999998e-05 2020-08-07T15:08:25.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:25.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.860938+0000", 2020-08-07T15:08:25.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017472500000000001 2020-08-07T15:08:25.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:08:25.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.861112+0000", 2020-08-07T15:08:25.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027810859999999999 2020-08-07T15:08:25.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:25.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.863893+0000", 2020-08-07T15:08:25.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2020-08-07T15:08:25.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:25.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.863894+0000", 2020-08-07T15:08:25.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7246e-05 2020-08-07T15:08:25.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:25.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.863921+0000", 2020-08-07T15:08:25.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.217031574 2020-08-07T15:08:25.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:25.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.080953+0000", 2020-08-07T15:08:25.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.071052727999999996 2020-08-07T15:08:25.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:25.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.152005+0000", 2020-08-07T15:08:25.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1769999999999999e-06 2020-08-07T15:08:25.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:25.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.152007+0000", 2020-08-07T15:08:25.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4748999999999998e-05 2020-08-07T15:08:25.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:25.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.152091+0000", 2020-08-07T15:08:25.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.29313854499999997 2020-08-07T15:08:25.818 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:25.821 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:25.821 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:25.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.827 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:104112 3.9 3:99770d45:::benchmark_data_smithi074_35447_object13013:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:08:25.827 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:45.859287+0000", 2020-08-07T15:08:25.827 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.362615527999999, 2020-08-07T15:08:25.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.29682865600000002, 2020-08-07T15:08:25.844 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:25.844 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:25.844 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:25.845 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:25.845 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:25.845 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 104112 2020-08-07T15:08:25.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.846 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:25.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:25.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.859287+0000", 2020-08-07T15:08:25.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:25.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:25.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.859287+0000", 2020-08-07T15:08:25.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9147999999999999e-05 2020-08-07T15:08:25.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:25.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.859316+0000", 2020-08-07T15:08:25.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.641e-06 2020-08-07T15:08:25.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:25.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.859320+0000", 2020-08-07T15:08:25.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8990000000000004e-06 2020-08-07T15:08:25.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:25.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.859326+0000", 2020-08-07T15:08:25.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8879999999999999e-06 2020-08-07T15:08:25.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:25.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.859330+0000", 2020-08-07T15:08:25.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025379999999999999 2020-08-07T15:08:25.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:25.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.861868+0000", 2020-08-07T15:08:25.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017909100000000001 2020-08-07T15:08:25.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:25.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.862047+0000", 2020-08-07T15:08:25.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000248946 2020-08-07T15:08:25.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:08:25.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.862296+0000", 2020-08-07T15:08:25.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035645329999999999 2020-08-07T15:08:25.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:25.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.865861+0000", 2020-08-07T15:08:25.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1119999999999999e-06 2020-08-07T15:08:25.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:25.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.865862+0000", 2020-08-07T15:08:25.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6837999999999999e-05 2020-08-07T15:08:25.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:25.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.865929+0000", 2020-08-07T15:08:25.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.21515225399999999 2020-08-07T15:08:25.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:25.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.081081+0000", 2020-08-07T15:08:25.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.074966868000000006 2020-08-07T15:08:25.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:25.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.156048+0000", 2020-08-07T15:08:25.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4099999999999997e-07 2020-08-07T15:08:25.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:25.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.156049+0000", 2020-08-07T15:08:25.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6597e-05 2020-08-07T15:08:25.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:25.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.156115+0000", 2020-08-07T15:08:25.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.29682865600000002 2020-08-07T15:08:25.908 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:25.908 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:25.909 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:25.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.909 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106192 3.9 3:9094e6df:::benchmark_data_smithi074_35447_object13273:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:08:25.910 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.308893+0000", 2020-08-07T15:08:25.910 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.913008998000002, 2020-08-07T15:08:25.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43331256600000001, 2020-08-07T15:08:25.910 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:25.911 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:25.911 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:25.911 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:25.912 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:25.913 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106192 2020-08-07T15:08:25.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.913 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:25.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:25.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.308893+0000", 2020-08-07T15:08:25.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:25.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:25.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.308893+0000", 2020-08-07T15:08:25.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7664999999999998e-05 2020-08-07T15:08:25.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:25.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.308941+0000", 2020-08-07T15:08:25.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1030000000000001e-06 2020-08-07T15:08:25.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:25.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.308946+0000", 2020-08-07T15:08:25.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0556e-05 2020-08-07T15:08:25.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:25.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.308957+0000", 2020-08-07T15:08:25.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3900000000000003e-06 2020-08-07T15:08:25.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:25.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.308961+0000", 2020-08-07T15:08:25.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040638360000000004 2020-08-07T15:08:25.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:25.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.313025+0000", 2020-08-07T15:08:25.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010189800000000001 2020-08-07T15:08:25.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:25.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.313127+0000", 2020-08-07T15:08:25.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121432 2020-08-07T15:08:25.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:08:25.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.313248+0000", 2020-08-07T15:08:25.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046267490000000003 2020-08-07T15:08:25.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:25.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.317875+0000", 2020-08-07T15:08:25.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5600000000000005e-07 2020-08-07T15:08:25.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:25.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.317876+0000", 2020-08-07T15:08:25.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7885000000000005e-05 2020-08-07T15:08:25.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:25.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.317943+0000", 2020-08-07T15:08:25.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043631985999999998 2020-08-07T15:08:25.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:25.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.361575+0000", 2020-08-07T15:08:25.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38057095099999999 2020-08-07T15:08:25.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:25.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.742146+0000", 2020-08-07T15:08:25.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1939999999999999e-06 2020-08-07T15:08:25.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:25.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.742148+0000", 2020-08-07T15:08:25.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8264999999999999e-05 2020-08-07T15:08:25.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:25.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.742206+0000", 2020-08-07T15:08:25.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43331256600000001 2020-08-07T15:08:25.967 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:25.967 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:25.967 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:25.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.968 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106192 3.9 e58/20 ondisk, result = 0)", 2020-08-07T15:08:25.968 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.316710+0000", 2020-08-07T15:08:25.968 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.905192759000002, 2020-08-07T15:08:25.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42550088699999999, 2020-08-07T15:08:25.969 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:25.969 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:25.969 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:25.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:25.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.316710+0000", 2020-08-07T15:08:25.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:25.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:25.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.316710+0000", 2020-08-07T15:08:25.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5000999999999999e-05 2020-08-07T15:08:25.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:25.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.316745+0000", 2020-08-07T15:08:25.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1419999999999998e-06 2020-08-07T15:08:25.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:25.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.316751+0000", 2020-08-07T15:08:25.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42489234599999998 2020-08-07T15:08:25.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:25.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.741643+0000", 2020-08-07T15:08:25.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.89e-06 2020-08-07T15:08:25.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:25.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.741647+0000", 2020-08-07T15:08:25.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00048394600000000002 2020-08-07T15:08:25.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:25.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.742131+0000", 2020-08-07T15:08:25.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0099999999999998e-06 2020-08-07T15:08:25.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:25.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.742138+0000", 2020-08-07T15:08:25.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2552000000000006e-05 2020-08-07T15:08:25.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:25.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.742210+0000", 2020-08-07T15:08:25.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42550088699999999 2020-08-07T15:08:25.993 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:25.993 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:25.993 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:25.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.994 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:108376 3.3 e62/20 ondisk, result = 0)", 2020-08-07T15:08:25.994 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:08:25.994 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.538860106000001, 2020-08-07T15:08:25.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62926872599999994, 2020-08-07T15:08:25.995 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:25.995 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:25.995 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:25.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:25.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:08:25.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:25.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:25.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:08:25.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8491999999999998e-05 2020-08-07T15:08:25.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:25.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683061+0000", 2020-08-07T15:08:25.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9099999999999998e-06 2020-08-07T15:08:25.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:25.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:25.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:25.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683065+0000", 2020-08-07T15:08:25.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62906670600000003 2020-08-07T15:08:25.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:26.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312131+0000", 2020-08-07T15:08:26.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2147000000000001e-05 2020-08-07T15:08:26.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:26.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312143+0000", 2020-08-07T15:08:26.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5152e-05 2020-08-07T15:08:26.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:26.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312239+0000", 2020-08-07T15:08:26.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5089999999999993e-06 2020-08-07T15:08:26.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:26.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312247+0000", 2020-08-07T15:08:26.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3810000000000001e-05 2020-08-07T15:08:26.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:26.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312311+0000", 2020-08-07T15:08:26.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62926872599999994 2020-08-07T15:08:26.022 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.022 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:26.022 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.023 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:108400 3.b e62/20 3:de5f1917:::benchmark_data_smithi074_35447_object13549:head v 62'6168, mlcod=62'6160)", 2020-08-07T15:08:26.023 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.712575+0000", 2020-08-07T15:08:26.023 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.509327659, 2020-08-07T15:08:26.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42039355899999997, 2020-08-07T15:08:26.023 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:26.024 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:26.024 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:26.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:26.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.712575+0000", 2020-08-07T15:08:26.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:26.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:26.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.712575+0000", 2020-08-07T15:08:26.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8982e-05 2020-08-07T15:08:26.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:26.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.712624+0000", 2020-08-07T15:08:26.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3940000000000001e-06 2020-08-07T15:08:26.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:26.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.712630+0000", 2020-08-07T15:08:26.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.41948886000000002 2020-08-07T15:08:26.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:26.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.132119+0000", 2020-08-07T15:08:26.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3150000000000003e-06 2020-08-07T15:08:26.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:26.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.132126+0000", 2020-08-07T15:08:26.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2264000000000001e-05 2020-08-07T15:08:26.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:26.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.132168+0000", 2020-08-07T15:08:26.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2549e-05 2020-08-07T15:08:26.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:26.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.132191+0000", 2020-08-07T15:08:26.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00072222300000000003 2020-08-07T15:08:26.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:26.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.132913+0000", 2020-08-07T15:08:26.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4972000000000002e-05 2020-08-07T15:08:26.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:26.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.132968+0000", 2020-08-07T15:08:26.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42039355899999997 2020-08-07T15:08:26.035 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.035 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:26.035 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.036 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:108464 3.6 e62/26 3:62704f92:::benchmark_data_smithi074_35447_object13557:head v 62'7200, mlcod=62'7194)", 2020-08-07T15:08:26.036 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.771334+0000", 2020-08-07T15:08:26.036 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.450568294, 2020-08-07T15:08:26.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30528281699999998, 2020-08-07T15:08:26.037 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:26.037 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:26.037 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:26.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:26.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.771334+0000", 2020-08-07T15:08:26.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:26.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:26.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.771334+0000", 2020-08-07T15:08:26.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2753999999999998e-05 2020-08-07T15:08:26.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:26.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.771377+0000", 2020-08-07T15:08:26.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0269999999999997e-06 2020-08-07T15:08:26.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:26.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.771381+0000", 2020-08-07T15:08:26.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30419457500000002 2020-08-07T15:08:26.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:26.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.075575+0000", 2020-08-07T15:08:26.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.412e-06 2020-08-07T15:08:26.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:26.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.075581+0000", 2020-08-07T15:08:26.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7296000000000003e-05 2020-08-07T15:08:26.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:26.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.075628+0000", 2020-08-07T15:08:26.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5247000000000001e-05 2020-08-07T15:08:26.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:26.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.075653+0000", 2020-08-07T15:08:26.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00090736300000000001 2020-08-07T15:08:26.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:26.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.076561+0000", 2020-08-07T15:08:26.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6143000000000001e-05 2020-08-07T15:08:26.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:26.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.076617+0000", 2020-08-07T15:08:26.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30528281699999998 2020-08-07T15:08:26.050 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.050 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:26.050 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.051 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "MOSDPGPushReply(3.1 62/60 [PushReplyOp(3:8329f37e:::benchmark_data_smithi074_35447_object11915:head)])", 2020-08-07T15:08:26.051 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:08:26.052 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.363502089999997, 2020-08-07T15:08:26.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996, 2020-08-07T15:08:26.052 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:26.053 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:08:26.053 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:26.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:26.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:08:26.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:26.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:26.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:08:26.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5372000000000001e-05 2020-08-07T15:08:26.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:26.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858416+0000", 2020-08-07T15:08:26.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6899999999999998e-06 2020-08-07T15:08:26.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:26.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858419+0000", 2020-08-07T15:08:26.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63907904299999996 2020-08-07T15:08:26.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:26.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497498+0000", 2020-08-07T15:08:26.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3332999999999999e-05 2020-08-07T15:08:26.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:26.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497512+0000", 2020-08-07T15:08:26.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000450875 2020-08-07T15:08:26.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:26.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497962+0000", 2020-08-07T15:08:26.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000267779 2020-08-07T15:08:26.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:26.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.498230+0000", 2020-08-07T15:08:26.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996 2020-08-07T15:08:26.082 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.083 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:26.083 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.083 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:145792 3.1 3:881e0420:::benchmark_data_smithi074_35447_object16183:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:08:26.084 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:08:26.084 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.555096209, 2020-08-07T15:08:26.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997, 2020-08-07T15:08:26.084 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:26.084 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:26.085 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:26.085 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:26.085 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:26.085 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 145792 2020-08-07T15:08:26.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.086 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:26.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:26.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:08:26.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:26.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:26.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:08:26.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7489000000000001e-05 2020-08-07T15:08:26.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:26.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666824+0000", 2020-08-07T15:08:26.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0169999999999998e-06 2020-08-07T15:08:26.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:26.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666828+0000", 2020-08-07T15:08:26.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9829999999999996e-06 2020-08-07T15:08:26.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:26.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666836+0000", 2020-08-07T15:08:26.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.481e-06 2020-08-07T15:08:26.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:26.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666840+0000", 2020-08-07T15:08:26.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5299000000000001e-05 2020-08-07T15:08:26.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:26.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666885+0000", 2020-08-07T15:08:26.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014456500000000001 2020-08-07T15:08:26.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:26.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667030+0000", 2020-08-07T15:08:26.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017677799999999999 2020-08-07T15:08:26.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,5", 2020-08-07T15:08:26.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667207+0000", 2020-08-07T15:08:26.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004149649 2020-08-07T15:08:26.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:26.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671356+0000", 2020-08-07T15:08:26.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019339649999999999 2020-08-07T15:08:26.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:26.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673290+0000", 2020-08-07T15:08:26.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73346620500000004 2020-08-07T15:08:26.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:26.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406756+0000", 2020-08-07T15:08:26.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4700000000000001e-06 2020-08-07T15:08:26.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:26.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406759+0000", 2020-08-07T15:08:26.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6258999999999998e-05 2020-08-07T15:08:26.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:26.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406815+0000", 2020-08-07T15:08:26.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0126000000000002e-05 2020-08-07T15:08:26.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:26.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406855+0000", 2020-08-07T15:08:26.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997 2020-08-07T15:08:26.119 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.119 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:26.119 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.120 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:145792 3.1 e75/67 ondisk, result = 0)", 2020-08-07T15:08:26.120 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:08:26.120 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.550711321, 2020-08-07T15:08:26.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003, 2020-08-07T15:08:26.121 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:26.121 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:26.121 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:26.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:26.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:08:26.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:26.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:26.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:08:26.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2830999999999998e-05 2020-08-07T15:08:26.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:26.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671224+0000", 2020-08-07T15:08:26.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:08:26.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:26.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671229+0000", 2020-08-07T15:08:26.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.735414864 2020-08-07T15:08:26.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:26.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406644+0000", 2020-08-07T15:08:26.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1250000000000004e-06 2020-08-07T15:08:26.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:26.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406651+0000", 2020-08-07T15:08:26.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6567e-05 2020-08-07T15:08:26.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:26.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406737+0000", 2020-08-07T15:08:26.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0054000000000001e-05 2020-08-07T15:08:26.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:26.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406747+0000", 2020-08-07T15:08:26.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011391999999999999 2020-08-07T15:08:26.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:26.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406861+0000", 2020-08-07T15:08:26.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003 2020-08-07T15:08:26.136 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.136 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:26.136 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.137 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:145806 3.8 e75/60 3:16669ddb:::benchmark_data_smithi074_35447_object16197:head v 75'9081, mlcod=75'9076)", 2020-08-07T15:08:26.137 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.674747+0000", 2020-08-07T15:08:26.138 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.547155503999999, 2020-08-07T15:08:26.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40259677300000002, 2020-08-07T15:08:26.138 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:26.138 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:26.139 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:26.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:26.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674747+0000", 2020-08-07T15:08:26.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:26.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:26.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674747+0000", 2020-08-07T15:08:26.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4858000000000003e-05 2020-08-07T15:08:26.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:26.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674792+0000", 2020-08-07T15:08:26.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8449999999999999e-06 2020-08-07T15:08:26.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:26.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674797+0000", 2020-08-07T15:08:26.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40178211699999999 2020-08-07T15:08:26.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:26.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076580+0000", 2020-08-07T15:08:26.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1260000000000004e-06 2020-08-07T15:08:26.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:26.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076589+0000", 2020-08-07T15:08:26.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0612999999999997e-05 2020-08-07T15:08:26.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:26.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076649+0000", 2020-08-07T15:08:26.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3822e-05 2020-08-07T15:08:26.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:26.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076673+0000", 2020-08-07T15:08:26.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00060830200000000002 2020-08-07T15:08:26.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:26.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.077281+0000", 2020-08-07T15:08:26.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2089999999999997e-05 2020-08-07T15:08:26.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:26.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.077344+0000", 2020-08-07T15:08:26.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40259677300000002 2020-08-07T15:08:26.164 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.165 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:26.165 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.165 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.165 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:26.166 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:26.169 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:08:26.170 INFO:teuthology.orchestra.run.smithi074:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 8192 --object-size 65536 -p unique_pool_1 bench 90 write' 2020-08-07T15:08:26.171 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:08:26.174 INFO:teuthology.orchestra.run.smithi074.stderr:set 4.9 pg_upmap_items mapping to [6->7,2->3,4->5] 2020-08-07T15:08:26.175 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:26.169+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:26.216 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2020-08-07T15:08:26.217 INFO:teuthology.orchestra.run:waiting for 3000 2020-08-07T15:08:26.272 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:hints = 1 2020-08-07T15:08:26.272 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Maintaining 16 concurrent writes of 8192 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2020-08-07T15:08:26.272 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Object prefix: benchmark_data_smithi074_44586 2020-08-07T15:08:26.273 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-08-07T15:08:26.273 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 0 0 0 0 0 0 - 0 2020-08-07T15:08:26.273 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:26.270+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:26.315 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:26.316 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:08:26.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.316 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:38 4.3 4:cfe8251d:::benchmark_data_smithi074_44586_object4:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:26.317 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.295760+0000", 2020-08-07T15:08:26.317 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.017928303, 2020-08-07T15:08:26.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017950952999999999, 2020-08-07T15:08:26.318 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:26.318 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:08:26.318 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:26.319 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:26.319 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:26.322 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 38 2020-08-07T15:08:26.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.323 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:26.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:26.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.295760+0000", 2020-08-07T15:08:26.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:26.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:26.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.295760+0000", 2020-08-07T15:08:26.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0985000000000001e-05 2020-08-07T15:08:26.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:26.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.295811+0000", 2020-08-07T15:08:26.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3330000000000002e-06 2020-08-07T15:08:26.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:26.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.295817+0000", 2020-08-07T15:08:26.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2371000000000001e-05 2020-08-07T15:08:26.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:26.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.295829+0000", 2020-08-07T15:08:26.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7440000000000004e-06 2020-08-07T15:08:26.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:26.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.295835+0000", 2020-08-07T15:08:26.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9088000000000002e-05 2020-08-07T15:08:26.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:26.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.295884+0000", 2020-08-07T15:08:26.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107095 2020-08-07T15:08:26.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:26.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.295991+0000", 2020-08-07T15:08:26.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00060810200000000001 2020-08-07T15:08:26.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:08:26.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.296599+0000", 2020-08-07T15:08:26.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036102830000000002 2020-08-07T15:08:26.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:26.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.300210+0000", 2020-08-07T15:08:26.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027910779999999998 2020-08-07T15:08:26.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:26.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.303001+0000", 2020-08-07T15:08:26.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0072410790000000001 2020-08-07T15:08:26.386 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.387 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:26.387 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.388 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:39 4.3 4:cfe8251d:::benchmark_data_smithi074_44586_object4:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:26.388 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.295877+0000", 2020-08-07T15:08:26.389 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.017810605, 2020-08-07T15:08:26.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017892538999999999, 2020-08-07T15:08:26.398 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:26.398 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:08:26.399 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:26.399 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:26.399 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:26.400 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 39 2020-08-07T15:08:26.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.401 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:26.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:26.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.295877+0000", 2020-08-07T15:08:26.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:26.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:26.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.295877+0000", 2020-08-07T15:08:26.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4514e-05 2020-08-07T15:08:26.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:26.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.295932+0000", 2020-08-07T15:08:26.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8309999999999997e-06 2020-08-07T15:08:26.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:26.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.295938+0000", 2020-08-07T15:08:26.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1573000000000001e-05 2020-08-07T15:08:26.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:26.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.295949+0000", 2020-08-07T15:08:26.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1889999999999998e-06 2020-08-07T15:08:26.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:26.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.295954+0000", 2020-08-07T15:08:26.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001549555 2020-08-07T15:08:26.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:26.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.297504+0000", 2020-08-07T15:08:26.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8122999999999995e-05 2020-08-07T15:08:26.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:26.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.297602+0000", 2020-08-07T15:08:26.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017441099999999999 2020-08-07T15:08:26.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:08:26.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.297777+0000", 2020-08-07T15:08:26.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051840430000000002 2020-08-07T15:08:26.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:26.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.302961+0000", 2020-08-07T15:08:26.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2776999999999998e-05 2020-08-07T15:08:26.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:26.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.303033+0000", 2020-08-07T15:08:26.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071560160000000003 2020-08-07T15:08:26.506 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.506 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:26.506 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.507 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:40 4.3 4:cfe8251d:::benchmark_data_smithi074_44586_object4:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:26.507 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.296061+0000", 2020-08-07T15:08:26.507 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.017627334000000001, 2020-08-07T15:08:26.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017761178999999998, 2020-08-07T15:08:26.508 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:26.508 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:08:26.508 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:26.508 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:26.508 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:26.509 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 40 2020-08-07T15:08:26.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.523 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:26.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:26.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.296061+0000", 2020-08-07T15:08:26.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:26.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:26.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.296061+0000", 2020-08-07T15:08:26.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5751000000000002e-05 2020-08-07T15:08:26.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:26.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.296116+0000", 2020-08-07T15:08:26.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8470000000000001e-06 2020-08-07T15:08:26.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:26.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.296122+0000", 2020-08-07T15:08:26.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1806e-05 2020-08-07T15:08:26.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:26.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.296134+0000", 2020-08-07T15:08:26.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9429999999999999e-06 2020-08-07T15:08:26.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:26.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.296139+0000", 2020-08-07T15:08:26.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032058310000000001 2020-08-07T15:08:26.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:26.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.299345+0000", 2020-08-07T15:08:26.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100578 2020-08-07T15:08:26.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:26.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.299445+0000", 2020-08-07T15:08:26.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033627100000000002 2020-08-07T15:08:26.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:08:26.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.299782+0000", 2020-08-07T15:08:26.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046106719999999997 2020-08-07T15:08:26.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:26.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.304392+0000", 2020-08-07T15:08:26.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001193745 2020-08-07T15:08:26.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:26.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.305586+0000", 2020-08-07T15:08:26.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0095254439999999992 2020-08-07T15:08:26.539 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.539 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:26.539 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.539 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:33 4.3 4:cfe8251d:::benchmark_data_smithi074_44586_object4:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:26.540 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.290294+0000", 2020-08-07T15:08:26.540 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.023394313, 2020-08-07T15:08:26.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023580034999999999, 2020-08-07T15:08:26.540 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:26.540 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:08:26.541 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:26.541 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:26.541 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:26.541 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 33 2020-08-07T15:08:26.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.542 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:26.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:26.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.290294+0000", 2020-08-07T15:08:26.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:26.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:26.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.290294+0000", 2020-08-07T15:08:26.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3192999999999997e-05 2020-08-07T15:08:26.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:26.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.290347+0000", 2020-08-07T15:08:26.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4559999999999997e-06 2020-08-07T15:08:26.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:26.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.290353+0000", 2020-08-07T15:08:26.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.323e-05 2020-08-07T15:08:26.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:26.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.290367+0000", 2020-08-07T15:08:26.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2840000000000003e-06 2020-08-07T15:08:26.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:26.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.290373+0000", 2020-08-07T15:08:26.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6081000000000001e-05 2020-08-07T15:08:26.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:26.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.290429+0000", 2020-08-07T15:08:26.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016448600000000001 2020-08-07T15:08:26.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:26.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.290593+0000", 2020-08-07T15:08:26.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000164186 2020-08-07T15:08:26.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:08:26.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.290758+0000", 2020-08-07T15:08:26.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039275029999999997 2020-08-07T15:08:26.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:26.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.294685+0000", 2020-08-07T15:08:26.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054421189999999996 2020-08-07T15:08:26.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:26.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.300127+0000", 2020-08-07T15:08:26.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0098335379999999993 2020-08-07T15:08:26.607 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.608 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:26.608 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.609 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:34 4.3 4:cfe8251d:::benchmark_data_smithi074_44586_object4:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:26.609 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.290419+0000", 2020-08-07T15:08:26.609 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.023268825, 2020-08-07T15:08:26.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023506580999999999, 2020-08-07T15:08:26.610 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:26.610 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:08:26.610 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:26.611 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:26.612 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:26.612 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 34 2020-08-07T15:08:26.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.612 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:26.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:26.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.290419+0000", 2020-08-07T15:08:26.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:26.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:26.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.290419+0000", 2020-08-07T15:08:26.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4148999999999999e-05 2020-08-07T15:08:26.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:26.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.290473+0000", 2020-08-07T15:08:26.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8780000000000003e-06 2020-08-07T15:08:26.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:26.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.290479+0000", 2020-08-07T15:08:26.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2532e-05 2020-08-07T15:08:26.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:26.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.290492+0000", 2020-08-07T15:08:26.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4330000000000003e-06 2020-08-07T15:08:26.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:26.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.290497+0000", 2020-08-07T15:08:26.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00069468799999999995 2020-08-07T15:08:26.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:26.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.291192+0000", 2020-08-07T15:08:26.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7257999999999997e-05 2020-08-07T15:08:26.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:26.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.291279+0000", 2020-08-07T15:08:26.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016417299999999999 2020-08-07T15:08:26.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:08:26.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.291443+0000", 2020-08-07T15:08:26.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003290332 2020-08-07T15:08:26.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:26.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.294734+0000", 2020-08-07T15:08:26.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054117719999999996 2020-08-07T15:08:26.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:26.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.300145+0000", 2020-08-07T15:08:26.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0097262149999999999 2020-08-07T15:08:26.625 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.626 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:26.626 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.626 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:35 4.3 4:cfe8251d:::benchmark_data_smithi074_44586_object4:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:26.627 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.290980+0000", 2020-08-07T15:08:26.627 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.02270805, 2020-08-07T15:08:26.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022995543, 2020-08-07T15:08:26.627 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:26.627 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:08:26.627 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:26.628 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:26.628 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:26.628 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 35 2020-08-07T15:08:26.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.629 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:26.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:26.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.290980+0000", 2020-08-07T15:08:26.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:26.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:26.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.290980+0000", 2020-08-07T15:08:26.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7828000000000003e-05 2020-08-07T15:08:26.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:26.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.291038+0000", 2020-08-07T15:08:26.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8839999999999999e-06 2020-08-07T15:08:26.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:26.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.291044+0000", 2020-08-07T15:08:26.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4049000000000001e-05 2020-08-07T15:08:26.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:26.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.291058+0000", 2020-08-07T15:08:26.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7989999999999999e-06 2020-08-07T15:08:26.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:26.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.291063+0000", 2020-08-07T15:08:26.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00084594800000000001 2020-08-07T15:08:26.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:26.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.291909+0000", 2020-08-07T15:08:26.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100442 2020-08-07T15:08:26.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:26.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.292010+0000", 2020-08-07T15:08:26.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018260499999999999 2020-08-07T15:08:26.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:08:26.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.292192+0000", 2020-08-07T15:08:26.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034355319999999998 2020-08-07T15:08:26.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:26.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.295628+0000", 2020-08-07T15:08:26.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045347599999999997 2020-08-07T15:08:26.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:26.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.300163+0000", 2020-08-07T15:08:26.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0091828469999999992 2020-08-07T15:08:26.640 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.648 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:26.648 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.649 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:36 4.3 4:cfe8251d:::benchmark_data_smithi074_44586_object4:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:26.650 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.291897+0000", 2020-08-07T15:08:26.650 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021791080000000001, 2020-08-07T15:08:26.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022130153, 2020-08-07T15:08:26.650 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:26.650 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:08:26.651 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:26.651 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:26.651 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:26.651 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 36 2020-08-07T15:08:26.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.652 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:26.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:26.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.291897+0000", 2020-08-07T15:08:26.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:26.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:26.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.291897+0000", 2020-08-07T15:08:26.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5312000000000002e-05 2020-08-07T15:08:26.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:26.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.291952+0000", 2020-08-07T15:08:26.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1850000000000001e-06 2020-08-07T15:08:26.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:26.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.291958+0000", 2020-08-07T15:08:26.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2986999999999999e-05 2020-08-07T15:08:26.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:26.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.291971+0000", 2020-08-07T15:08:26.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5999999999999997e-06 2020-08-07T15:08:26.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:26.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.291977+0000", 2020-08-07T15:08:26.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00062190300000000004 2020-08-07T15:08:26.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:26.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.292599+0000", 2020-08-07T15:08:26.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0822999999999994e-05 2020-08-07T15:08:26.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:26.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.292690+0000", 2020-08-07T15:08:26.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000161306 2020-08-07T15:08:26.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:08:26.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.292851+0000", 2020-08-07T15:08:26.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028120110000000001 2020-08-07T15:08:26.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:26.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.295663+0000", 2020-08-07T15:08:26.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045153820000000001 2020-08-07T15:08:26.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:26.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.300178+0000", 2020-08-07T15:08:26.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0082815089999999994 2020-08-07T15:08:26.683 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.684 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:26.684 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.684 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:37 4.3 4:cfe8251d:::benchmark_data_smithi074_44586_object4:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:26.685 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.293904+0000", 2020-08-07T15:08:26.685 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.019784385000000002, 2020-08-07T15:08:26.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.020183089000000001, 2020-08-07T15:08:26.685 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:26.685 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:08:26.686 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:26.686 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:26.686 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:26.686 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 37 2020-08-07T15:08:26.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.687 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:26.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:26.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.293904+0000", 2020-08-07T15:08:26.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:26.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:26.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.293904+0000", 2020-08-07T15:08:26.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9401999999999999e-05 2020-08-07T15:08:26.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:26.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.293953+0000", 2020-08-07T15:08:26.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3559999999999998e-06 2020-08-07T15:08:26.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:26.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.293959+0000", 2020-08-07T15:08:26.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3114e-05 2020-08-07T15:08:26.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:26.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.293972+0000", 2020-08-07T15:08:26.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4559999999999997e-06 2020-08-07T15:08:26.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:26.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.293979+0000", 2020-08-07T15:08:26.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0306e-05 2020-08-07T15:08:26.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:26.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.294059+0000", 2020-08-07T15:08:26.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9878999999999997e-05 2020-08-07T15:08:26.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:26.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.294159+0000", 2020-08-07T15:08:26.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016127700000000001 2020-08-07T15:08:26.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:08:26.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.294320+0000", 2020-08-07T15:08:26.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058736789999999997 2020-08-07T15:08:26.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:26.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.300194+0000", 2020-08-07T15:08:26.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00035481100000000002 2020-08-07T15:08:26.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:26.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.300549+0000", 2020-08-07T15:08:26.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0066452799999999999 2020-08-07T15:08:26.718 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.718 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:26.718 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.719 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "rep_scrubmap(2.3 e17 from shard 2)", 2020-08-07T15:08:26.719 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.312353+0000", 2020-08-07T15:08:26.719 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0013345099999999999, 2020-08-07T15:08:26.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001780282, 2020-08-07T15:08:26.719 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:26.720 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:08:26.720 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:26.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:26.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.312353+0000", 2020-08-07T15:08:26.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:26.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:26.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.312353+0000", 2020-08-07T15:08:26.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4474000000000001e-05 2020-08-07T15:08:26.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:26.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.312398+0000", 2020-08-07T15:08:26.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1229999999999997e-06 2020-08-07T15:08:26.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:26.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.312404+0000", 2020-08-07T15:08:26.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5047999999999999e-05 2020-08-07T15:08:26.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:26.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.312459+0000", 2020-08-07T15:08:26.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2949999999999996e-06 2020-08-07T15:08:26.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:26.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:26.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:26.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.312466+0000", 2020-08-07T15:08:26.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011294 2020-08-07T15:08:26.729 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.729 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:26.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:26.731 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:08:26.731 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 9 2020-08-07T15:08:26.731 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:26.733 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:08:26.734 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:26.731+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:26.837 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:26.833+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:26.926 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:26.926 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:08:26.927 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:08:26.927 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:08:26.927 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:26.940 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:08:26.941 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:26.934+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:27.004 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:08:27.036 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:27.035+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:27.149 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:27.139+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:27.157 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:27.157 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:08:27.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:08:27.160 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:08:27.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.161 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:41432 3.d e32/20 3:b5c8a160:::benchmark_data_smithi074_35447_object5178:head v 32'2848, mlcod=32'2838)", 2020-08-07T15:08:27.161 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.794575+0000", 2020-08-07T15:08:27.164 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 84.355403588000001, 2020-08-07T15:08:27.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36243746199999999, 2020-08-07T15:08:27.165 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:27.165 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:27.165 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:27.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:27.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794575+0000", 2020-08-07T15:08:27.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:27.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:27.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794575+0000", 2020-08-07T15:08:27.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0696999999999999e-05 2020-08-07T15:08:27.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:27.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794626+0000", 2020-08-07T15:08:27.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4189999999999999e-06 2020-08-07T15:08:27.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:27.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794632+0000", 2020-08-07T15:08:27.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36145730399999998 2020-08-07T15:08:27.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:27.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.156089+0000", 2020-08-07T15:08:27.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2329999999999997e-06 2020-08-07T15:08:27.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:27.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.156095+0000", 2020-08-07T15:08:27.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0054999999999999e-05 2020-08-07T15:08:27.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.156125+0000", 2020-08-07T15:08:27.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5369000000000001e-05 2020-08-07T15:08:27.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:27.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.156140+0000", 2020-08-07T15:08:27.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00076810300000000001 2020-08-07T15:08:27.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:27.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.156908+0000", 2020-08-07T15:08:27.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104282 2020-08-07T15:08:27.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:27.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.157012+0000", 2020-08-07T15:08:27.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36243746199999999 2020-08-07T15:08:27.246 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.246 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:27.246 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.247 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "MOSDPGPushReply(3.d 59/35 [PushReplyOp(3:bbd08cec:::benchmark_data_smithi074_35447_object8039:head)])", 2020-08-07T15:08:27.247 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.839005+0000", 2020-08-07T15:08:27.248 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.310973283000003, 2020-08-07T15:08:27.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42127482700000002, 2020-08-07T15:08:27.248 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:27.248 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:08:27.249 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:27.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:27.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.839005+0000", 2020-08-07T15:08:27.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:27.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:27.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.839005+0000", 2020-08-07T15:08:27.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2351000000000002e-05 2020-08-07T15:08:27.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:27.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.839038+0000", 2020-08-07T15:08:27.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3849999999999996e-06 2020-08-07T15:08:27.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:27.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.839045+0000", 2020-08-07T15:08:27.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42104101399999999 2020-08-07T15:08:27.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:27.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.260086+0000", 2020-08-07T15:08:27.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.605e-06 2020-08-07T15:08:27.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:27.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.260091+0000", 2020-08-07T15:08:27.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1946000000000006e-05 2020-08-07T15:08:27.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.260153+0000", 2020-08-07T15:08:27.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012752600000000001 2020-08-07T15:08:27.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:27.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.260280+0000", 2020-08-07T15:08:27.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42127482700000002 2020-08-07T15:08:27.258 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.259 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:27.259 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.259 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:108496 3.2 e62/20 3:49ad53c2:::benchmark_data_smithi074_35447_object13561:head v 62'6848, mlcod=62'6843)", 2020-08-07T15:08:27.260 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.809757+0000", 2020-08-07T15:08:27.260 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.340221835999998, 2020-08-07T15:08:27.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.48234552600000002, 2020-08-07T15:08:27.261 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:27.261 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:27.261 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:27.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:27.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.809757+0000", 2020-08-07T15:08:27.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:27.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:27.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.809757+0000", 2020-08-07T15:08:27.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2111000000000003e-05 2020-08-07T15:08:27.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:27.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.809809+0000", 2020-08-07T15:08:27.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8590000000000005e-06 2020-08-07T15:08:27.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:27.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.809817+0000", 2020-08-07T15:08:27.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.48126780000000002 2020-08-07T15:08:27.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:27.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.291084+0000", 2020-08-07T15:08:27.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8420000000000004e-06 2020-08-07T15:08:27.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:27.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.291089+0000", 2020-08-07T15:08:27.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2973999999999995e-05 2020-08-07T15:08:27.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.291172+0000", 2020-08-07T15:08:27.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2328999999999999e-05 2020-08-07T15:08:27.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:27.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.291195+0000", 2020-08-07T15:08:27.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00085884999999999998 2020-08-07T15:08:27.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:27.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.292053+0000", 2020-08-07T15:08:27.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8761000000000003e-05 2020-08-07T15:08:27.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:27.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.292102+0000", 2020-08-07T15:08:27.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.48234552600000002 2020-08-07T15:08:27.273 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.274 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:27.274 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.276 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "pg_backfill(progress 3.1 e 62/62 lb 3:8328459b:::benchmark_data_smithi074_35447_object1550:head)", 2020-08-07T15:08:27.276 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.857351+0000", 2020-08-07T15:08:27.276 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.292627318000001, 2020-08-07T15:08:27.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46585928100000001, 2020-08-07T15:08:27.277 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:27.277 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:27.277 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:27.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:27.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.857351+0000", 2020-08-07T15:08:27.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:27.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:27.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.857351+0000", 2020-08-07T15:08:27.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4912e-05 2020-08-07T15:08:27.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:27.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.857386+0000", 2020-08-07T15:08:27.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2329999999999995e-06 2020-08-07T15:08:27.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:27.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.857395+0000", 2020-08-07T15:08:27.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.465707918 2020-08-07T15:08:27.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:27.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.323103+0000", 2020-08-07T15:08:27.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4059999999999998e-06 2020-08-07T15:08:27.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.323108+0000", 2020-08-07T15:08:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1260999999999997e-05 2020-08-07T15:08:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.323149+0000", 2020-08-07T15:08:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4552999999999999e-05 2020-08-07T15:08:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.323164+0000", 2020-08-07T15:08:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6998e-05 2020-08-07T15:08:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.323211+0000", 2020-08-07T15:08:27.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46585928100000001 2020-08-07T15:08:27.359 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.359 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:27.359 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.362 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:145807 3.c 3:39741db6:::benchmark_data_smithi074_35447_object16198:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:08:27.363 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.673539+0000", 2020-08-07T15:08:27.363 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 12.476439736, 2020-08-07T15:08:27.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40356883700000001, 2020-08-07T15:08:27.364 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:27.364 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:27.365 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:27.394 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:27.395 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:27.402 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 145807 2020-08-07T15:08:27.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.403 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:27.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:27.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673539+0000", 2020-08-07T15:08:27.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:27.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:27.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673539+0000", 2020-08-07T15:08:27.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1495999999999998e-05 2020-08-07T15:08:27.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:27.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673570+0000", 2020-08-07T15:08:27.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7729999999999996e-06 2020-08-07T15:08:27.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:27.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673576+0000", 2020-08-07T15:08:27.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2119e-05 2020-08-07T15:08:27.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:27.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673588+0000", 2020-08-07T15:08:27.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6010000000000002e-06 2020-08-07T15:08:27.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:27.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673594+0000", 2020-08-07T15:08:27.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1732e-05 2020-08-07T15:08:27.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673636+0000", 2020-08-07T15:08:27.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015816199999999999 2020-08-07T15:08:27.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:27.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673794+0000", 2020-08-07T15:08:27.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010452 2020-08-07T15:08:27.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:08:27.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673898+0000", 2020-08-07T15:08:27.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041817169999999997 2020-08-07T15:08:27.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:27.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.678080+0000", 2020-08-07T15:08:27.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0979999999999999e-06 2020-08-07T15:08:27.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:27.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.678081+0000", 2020-08-07T15:08:27.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1353999999999996e-05 2020-08-07T15:08:27.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:27.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.678142+0000", 2020-08-07T15:08:27.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.081619948999999997 2020-08-07T15:08:27.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.759762+0000", 2020-08-07T15:08:27.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31727375899999999 2020-08-07T15:08:27.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.077036+0000", 2020-08-07T15:08:27.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.378e-06 2020-08-07T15:08:27.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:27.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.077037+0000", 2020-08-07T15:08:27.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0178999999999993e-05 2020-08-07T15:08:27.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:27.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.077108+0000", 2020-08-07T15:08:27.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40356883700000001 2020-08-07T15:08:27.425 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.425 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:27.425 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.426 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:81 4.9 4:9d72df7e:::benchmark_data_smithi074_44586_object10:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:27.426 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.332553+0000", 2020-08-07T15:08:27.426 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.81742556099999997, 2020-08-07T15:08:27.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52066251100000005, 2020-08-07T15:08:27.427 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:27.427 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:27.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:27.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:27.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:27.428 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 81 2020-08-07T15:08:27.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.428 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:27.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:27.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332553+0000", 2020-08-07T15:08:27.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:27.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:27.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332553+0000", 2020-08-07T15:08:27.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4428999999999997e-05 2020-08-07T15:08:27.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:27.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332607+0000", 2020-08-07T15:08:27.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9009999999999997e-06 2020-08-07T15:08:27.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:27.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332614+0000", 2020-08-07T15:08:27.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4676e-05 2020-08-07T15:08:27.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:27.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332629+0000", 2020-08-07T15:08:27.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4560000000000003e-06 2020-08-07T15:08:27.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:27.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332636+0000", 2020-08-07T15:08:27.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9703000000000002e-05 2020-08-07T15:08:27.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332686+0000", 2020-08-07T15:08:27.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0727000000000001e-05 2020-08-07T15:08:27.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:27.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332697+0000", 2020-08-07T15:08:27.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.50871030100000003 2020-08-07T15:08:27.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.841407+0000", 2020-08-07T15:08:27.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015651400000000001 2020-08-07T15:08:27.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:27.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.841564+0000", 2020-08-07T15:08:27.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134673 2020-08-07T15:08:27.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:27.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.841698+0000", 2020-08-07T15:08:27.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003074313 2020-08-07T15:08:27.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:27.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.844773+0000", 2020-08-07T15:08:27.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.136e-06 2020-08-07T15:08:27.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:27.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.844774+0000", 2020-08-07T15:08:27.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1717e-05 2020-08-07T15:08:27.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:27.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.844826+0000", 2020-08-07T15:08:27.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083120510000000009 2020-08-07T15:08:27.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853138+0000", 2020-08-07T15:08:27.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8754e-05 2020-08-07T15:08:27.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853156+0000", 2020-08-07T15:08:27.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8599999999999998e-07 2020-08-07T15:08:27.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:27.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853157+0000", 2020-08-07T15:08:27.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8474000000000003e-05 2020-08-07T15:08:27.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:27.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853216+0000", 2020-08-07T15:08:27.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52066251100000005 2020-08-07T15:08:27.458 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.458 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:27.458 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.459 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:82 4.9 4:9d72df7e:::benchmark_data_smithi074_44586_object10:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:27.460 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.332680+0000", 2020-08-07T15:08:27.460 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.81729812700000004, 2020-08-07T15:08:27.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52064592399999998, 2020-08-07T15:08:27.461 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:27.461 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:27.461 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:27.461 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:27.461 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:27.470 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 82 2020-08-07T15:08:27.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.470 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:27.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:27.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332680+0000", 2020-08-07T15:08:27.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:27.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:27.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332680+0000", 2020-08-07T15:08:27.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.168e-05 2020-08-07T15:08:27.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:27.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332732+0000", 2020-08-07T15:08:27.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4600000000000002e-06 2020-08-07T15:08:27.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:27.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332738+0000", 2020-08-07T15:08:27.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1202e-05 2020-08-07T15:08:27.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:27.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332749+0000", 2020-08-07T15:08:27.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.164e-06 2020-08-07T15:08:27.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:27.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332754+0000", 2020-08-07T15:08:27.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0890000000000002e-05 2020-08-07T15:08:27.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332805+0000", 2020-08-07T15:08:27.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0029999999999994e-06 2020-08-07T15:08:27.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:27.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332813+0000", 2020-08-07T15:08:27.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.50934248400000004 2020-08-07T15:08:27.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.842155+0000", 2020-08-07T15:08:27.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011158399999999999 2020-08-07T15:08:27.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:27.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.842267+0000", 2020-08-07T15:08:27.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019425200000000001 2020-08-07T15:08:27.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:27.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.842461+0000", 2020-08-07T15:08:27.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023764670000000002 2020-08-07T15:08:27.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:27.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.844838+0000", 2020-08-07T15:08:27.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000005e-07 2020-08-07T15:08:27.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:27.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.844838+0000", 2020-08-07T15:08:27.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3197e-05 2020-08-07T15:08:27.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:27.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.844881+0000", 2020-08-07T15:08:27.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083592779999999995 2020-08-07T15:08:27.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853241+0000", 2020-08-07T15:08:27.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3352e-05 2020-08-07T15:08:27.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853294+0000", 2020-08-07T15:08:27.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4699999999999998e-07 2020-08-07T15:08:27.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:27.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853294+0000", 2020-08-07T15:08:27.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2162999999999997e-05 2020-08-07T15:08:27.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:27.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853326+0000", 2020-08-07T15:08:27.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52064592399999998 2020-08-07T15:08:27.494 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.494 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:27.494 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.495 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:83 4.9 4:9d72df7e:::benchmark_data_smithi074_44586_object10:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:27.495 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.332842+0000", 2020-08-07T15:08:27.495 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.81713619800000004, 2020-08-07T15:08:27.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52054723000000003, 2020-08-07T15:08:27.495 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:27.496 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:27.496 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:27.496 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:27.496 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:27.496 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 83 2020-08-07T15:08:27.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.497 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:27.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:27.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332842+0000", 2020-08-07T15:08:27.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:27.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:27.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332842+0000", 2020-08-07T15:08:27.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2209000000000001e-05 2020-08-07T15:08:27.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:27.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332895+0000", 2020-08-07T15:08:27.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4070000000000004e-06 2020-08-07T15:08:27.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:27.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332902+0000", 2020-08-07T15:08:27.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6030000000000001e-05 2020-08-07T15:08:27.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:27.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332918+0000", 2020-08-07T15:08:27.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3189999999999995e-06 2020-08-07T15:08:27.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:27.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332926+0000", 2020-08-07T15:08:27.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2171000000000001e-05 2020-08-07T15:08:27.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332959+0000", 2020-08-07T15:08:27.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8200000000000002e-06 2020-08-07T15:08:27.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:27.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332964+0000", 2020-08-07T15:08:27.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.50992941300000005 2020-08-07T15:08:27.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.842894+0000", 2020-08-07T15:08:27.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9205999999999999e-05 2020-08-07T15:08:27.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:27.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.842993+0000", 2020-08-07T15:08:27.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000194155 2020-08-07T15:08:27.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:27.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.843187+0000", 2020-08-07T15:08:27.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029782989999999998 2020-08-07T15:08:27.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:27.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.846165+0000", 2020-08-07T15:08:27.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9699999999999995e-07 2020-08-07T15:08:27.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:27.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.846166+0000", 2020-08-07T15:08:27.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5018999999999997e-05 2020-08-07T15:08:27.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:27.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.846211+0000", 2020-08-07T15:08:27.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070467719999999998 2020-08-07T15:08:27.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853258+0000", 2020-08-07T15:08:27.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010171 2020-08-07T15:08:27.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853360+0000", 2020-08-07T15:08:27.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5699999999999999e-07 2020-08-07T15:08:27.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:27.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853360+0000", 2020-08-07T15:08:27.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9646e-05 2020-08-07T15:08:27.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:27.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853390+0000", 2020-08-07T15:08:27.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52054723000000003 2020-08-07T15:08:27.518 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.518 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:27.518 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.519 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:84 4.9 4:9d72df7e:::benchmark_data_smithi074_44586_object10:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:27.519 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.332964+0000", 2020-08-07T15:08:27.519 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.81701429000000003, 2020-08-07T15:08:27.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52047439900000003, 2020-08-07T15:08:27.520 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:27.520 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:27.520 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:27.521 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:27.521 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:27.521 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 84 2020-08-07T15:08:27.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.521 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:27.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:27.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332964+0000", 2020-08-07T15:08:27.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:27.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:27.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332964+0000", 2020-08-07T15:08:27.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9694000000000002e-05 2020-08-07T15:08:27.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:27.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333014+0000", 2020-08-07T15:08:27.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2000000000000002e-06 2020-08-07T15:08:27.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:27.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333019+0000", 2020-08-07T15:08:27.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3052e-05 2020-08-07T15:08:27.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:27.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333032+0000", 2020-08-07T15:08:27.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8559999999999998e-06 2020-08-07T15:08:27.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:27.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333037+0000", 2020-08-07T15:08:27.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5210000000000003e-05 2020-08-07T15:08:27.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333082+0000", 2020-08-07T15:08:27.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.594e-06 2020-08-07T15:08:27.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:27.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333092+0000", 2020-08-07T15:08:27.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.51043333000000002 2020-08-07T15:08:27.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.843525+0000", 2020-08-07T15:08:27.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106469 2020-08-07T15:08:27.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:27.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.843632+0000", 2020-08-07T15:08:27.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019589400000000001 2020-08-07T15:08:27.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:27.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.843828+0000", 2020-08-07T15:08:27.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003896208 2020-08-07T15:08:27.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:27.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.847724+0000", 2020-08-07T15:08:27.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6700000000000002e-07 2020-08-07T15:08:27.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:27.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.847725+0000", 2020-08-07T15:08:27.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4647999999999995e-05 2020-08-07T15:08:27.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:27.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.847789+0000", 2020-08-07T15:08:27.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054861290000000002 2020-08-07T15:08:27.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853276+0000", 2020-08-07T15:08:27.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013245999999999999 2020-08-07T15:08:27.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853408+0000", 2020-08-07T15:08:27.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8000000000000002e-07 2020-08-07T15:08:27.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:27.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853408+0000", 2020-08-07T15:08:27.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0408000000000001e-05 2020-08-07T15:08:27.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:27.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853439+0000", 2020-08-07T15:08:27.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52047439900000003 2020-08-07T15:08:27.541 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.541 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:27.541 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.542 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:85 4.9 4:9d72df7e:::benchmark_data_smithi074_44586_object10:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:27.542 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.333658+0000", 2020-08-07T15:08:27.543 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.81632072899999997, 2020-08-07T15:08:27.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.51984521400000006, 2020-08-07T15:08:27.543 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:27.543 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:27.543 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:27.544 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:27.544 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:27.544 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 85 2020-08-07T15:08:27.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.544 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:27.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:27.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333658+0000", 2020-08-07T15:08:27.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:27.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:27.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333658+0000", 2020-08-07T15:08:27.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6450999999999999e-05 2020-08-07T15:08:27.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:27.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333714+0000", 2020-08-07T15:08:27.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4449999999999999e-06 2020-08-07T15:08:27.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:27.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333722+0000", 2020-08-07T15:08:27.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.491e-05 2020-08-07T15:08:27.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:27.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333737+0000", 2020-08-07T15:08:27.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7570000000000001e-06 2020-08-07T15:08:27.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:27.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333742+0000", 2020-08-07T15:08:27.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2324000000000003e-05 2020-08-07T15:08:27.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333785+0000", 2020-08-07T15:08:27.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4694e-05 2020-08-07T15:08:27.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:27.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333799+0000", 2020-08-07T15:08:27.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.51044864400000001 2020-08-07T15:08:27.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.844248+0000", 2020-08-07T15:08:27.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9263000000000003e-05 2020-08-07T15:08:27.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:27.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.844327+0000", 2020-08-07T15:08:27.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012624700000000001 2020-08-07T15:08:27.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:27.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.844454+0000", 2020-08-07T15:08:27.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003347362 2020-08-07T15:08:27.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:27.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.847801+0000", 2020-08-07T15:08:27.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.37e-07 2020-08-07T15:08:27.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:27.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.847802+0000", 2020-08-07T15:08:27.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4914000000000003e-05 2020-08-07T15:08:27.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:27.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.847857+0000", 2020-08-07T15:08:27.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056010640000000002 2020-08-07T15:08:27.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853458+0000", 2020-08-07T15:08:27.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6331999999999999e-05 2020-08-07T15:08:27.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853474+0000", 2020-08-07T15:08:27.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1e-07 2020-08-07T15:08:27.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:27.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853474+0000", 2020-08-07T15:08:27.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8860000000000002e-05 2020-08-07T15:08:27.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:27.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853503+0000", 2020-08-07T15:08:27.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.51984521400000006 2020-08-07T15:08:27.578 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.578 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:27.579 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.580 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:86 4.9 4:9d72df7e:::benchmark_data_smithi074_44586_object10:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:27.580 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.334385+0000", 2020-08-07T15:08:27.580 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.815593872, 2020-08-07T15:08:27.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.51921272600000001, 2020-08-07T15:08:27.581 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:27.594 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:27.594 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:27.594 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:27.595 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:27.595 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86 2020-08-07T15:08:27.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.595 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:27.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:27.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334385+0000", 2020-08-07T15:08:27.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:27.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:27.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334385+0000", 2020-08-07T15:08:27.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.757e-05 2020-08-07T15:08:27.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:27.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334432+0000", 2020-08-07T15:08:27.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8680000000000001e-06 2020-08-07T15:08:27.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:27.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334439+0000", 2020-08-07T15:08:27.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.291e-05 2020-08-07T15:08:27.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:27.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334452+0000", 2020-08-07T15:08:27.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0889999999999996e-06 2020-08-07T15:08:27.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:27.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334458+0000", 2020-08-07T15:08:27.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4972000000000003e-05 2020-08-07T15:08:27.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334493+0000", 2020-08-07T15:08:27.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9689999999999999e-06 2020-08-07T15:08:27.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:27.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334499+0000", 2020-08-07T15:08:27.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.51041587300000002 2020-08-07T15:08:27.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.844915+0000", 2020-08-07T15:08:27.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4506999999999995e-05 2020-08-07T15:08:27.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:27.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.845009+0000", 2020-08-07T15:08:27.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000171695 2020-08-07T15:08:27.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:27.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.845181+0000", 2020-08-07T15:08:27.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003417546 2020-08-07T15:08:27.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:27.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.848599+0000", 2020-08-07T15:08:27.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7999999999999993e-07 2020-08-07T15:08:27.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:27.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.848600+0000", 2020-08-07T15:08:27.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4317000000000001e-05 2020-08-07T15:08:27.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:27.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.848664+0000", 2020-08-07T15:08:27.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048569909999999997 2020-08-07T15:08:27.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853521+0000", 2020-08-07T15:08:27.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7061000000000002e-05 2020-08-07T15:08:27.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853568+0000", 2020-08-07T15:08:27.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.85e-07 2020-08-07T15:08:27.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:27.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853568+0000", 2020-08-07T15:08:27.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9193e-05 2020-08-07T15:08:27.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:27.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853597+0000", 2020-08-07T15:08:27.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.51921272600000001 2020-08-07T15:08:27.637 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.637 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:27.639 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.640 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:87 4.9 4:9d72df7e:::benchmark_data_smithi074_44586_object10:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:27.641 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.334494+0000", 2020-08-07T15:08:27.641 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.81548428299999998, 2020-08-07T15:08:27.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.51915305300000003, 2020-08-07T15:08:27.642 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:27.642 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:27.643 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:27.643 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:27.644 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:27.644 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 87 2020-08-07T15:08:27.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.645 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:27.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:27.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334494+0000", 2020-08-07T15:08:27.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:27.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:27.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334494+0000", 2020-08-07T15:08:27.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7128e-05 2020-08-07T15:08:27.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:27.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334541+0000", 2020-08-07T15:08:27.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7169999999999996e-06 2020-08-07T15:08:27.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:27.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334547+0000", 2020-08-07T15:08:27.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1586999999999999e-05 2020-08-07T15:08:27.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:27.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334559+0000", 2020-08-07T15:08:27.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4620000000000004e-06 2020-08-07T15:08:27.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:27.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334564+0000", 2020-08-07T15:08:27.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5139999999999999e-05 2020-08-07T15:08:27.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334599+0000", 2020-08-07T15:08:27.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2600000000000005e-06 2020-08-07T15:08:27.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:27.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334608+0000", 2020-08-07T15:08:27.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.51096861900000001 2020-08-07T15:08:27.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.845576+0000", 2020-08-07T15:08:27.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5854999999999996e-05 2020-08-07T15:08:27.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:27.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.845672+0000", 2020-08-07T15:08:27.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016852800000000001 2020-08-07T15:08:27.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:27.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.845841+0000", 2020-08-07T15:08:27.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028329470000000002 2020-08-07T15:08:27.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:27.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.848674+0000", 2020-08-07T15:08:27.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0200000000000001e-07 2020-08-07T15:08:27.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:27.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.848674+0000", 2020-08-07T15:08:27.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0541000000000002e-05 2020-08-07T15:08:27.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:27.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.848735+0000", 2020-08-07T15:08:27.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048022710000000003 2020-08-07T15:08:27.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853537+0000", 2020-08-07T15:08:27.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7730000000000003e-05 2020-08-07T15:08:27.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853615+0000", 2020-08-07T15:08:27.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8199999999999999e-07 2020-08-07T15:08:27.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:27.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853615+0000", 2020-08-07T15:08:27.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2384e-05 2020-08-07T15:08:27.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:27.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853647+0000", 2020-08-07T15:08:27.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.51915305300000003 2020-08-07T15:08:27.690 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.690 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:27.691 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.692 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:88 4.9 4:9d72df7e:::benchmark_data_smithi074_44586_object10:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:27.692 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.335168+0000", 2020-08-07T15:08:27.692 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.81481100500000003, 2020-08-07T15:08:27.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.51856037899999996, 2020-08-07T15:08:27.693 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:27.693 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:27.693 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:27.694 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:27.694 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:27.695 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 88 2020-08-07T15:08:27.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.718 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:27.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:27.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.335168+0000", 2020-08-07T15:08:27.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:27.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:27.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.335168+0000", 2020-08-07T15:08:27.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7526e-05 2020-08-07T15:08:27.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:27.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.335205+0000", 2020-08-07T15:08:27.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.19e-06 2020-08-07T15:08:27.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:27.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.335208+0000", 2020-08-07T15:08:27.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8259999999999993e-06 2020-08-07T15:08:27.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:27.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.335216+0000", 2020-08-07T15:08:27.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9170000000000002e-06 2020-08-07T15:08:27.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:27.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.335219+0000", 2020-08-07T15:08:27.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7576000000000002e-05 2020-08-07T15:08:27.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.335247+0000", 2020-08-07T15:08:27.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4499999999999997e-06 2020-08-07T15:08:27.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:27.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.335251+0000", 2020-08-07T15:08:27.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.51102096600000002 2020-08-07T15:08:27.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.846272+0000", 2020-08-07T15:08:27.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104205 2020-08-07T15:08:27.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:27.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.846376+0000", 2020-08-07T15:08:27.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017541 2020-08-07T15:08:27.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:27.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.846552+0000", 2020-08-07T15:08:27.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029493729999999999 2020-08-07T15:08:27.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:27.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.849501+0000", 2020-08-07T15:08:27.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1299999999999999e-07 2020-08-07T15:08:27.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:27.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.849502+0000", 2020-08-07T15:08:27.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7806999999999997e-05 2020-08-07T15:08:27.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:27.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.849560+0000", 2020-08-07T15:08:27.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039933019999999998 2020-08-07T15:08:27.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853553+0000", 2020-08-07T15:08:27.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000126184 2020-08-07T15:08:27.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853679+0000", 2020-08-07T15:08:27.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9900000000000002e-07 2020-08-07T15:08:27.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:27.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853679+0000", 2020-08-07T15:08:27.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8535000000000002e-05 2020-08-07T15:08:27.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:27.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853728+0000", 2020-08-07T15:08:27.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.51856037899999996 2020-08-07T15:08:27.743 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.744 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:27.744 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.744 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:177 4.9 4:96543f2a:::benchmark_data_smithi074_44586_object22:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:27.745 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.418082+0000", 2020-08-07T15:08:27.745 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.73189691899999998, 2020-08-07T15:08:27.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44218349899999998, 2020-08-07T15:08:27.746 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:27.746 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:27.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:27.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:27.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:27.747 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 177 2020-08-07T15:08:27.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.747 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:27.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:27.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418082+0000", 2020-08-07T15:08:27.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:27.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:27.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418082+0000", 2020-08-07T15:08:27.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7248999999999999e-05 2020-08-07T15:08:27.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:27.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418109+0000", 2020-08-07T15:08:27.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0729999999999998e-06 2020-08-07T15:08:27.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:27.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418113+0000", 2020-08-07T15:08:27.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6090000000000001e-06 2020-08-07T15:08:27.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:27.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418122+0000", 2020-08-07T15:08:27.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.809e-06 2020-08-07T15:08:27.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:27.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418126+0000", 2020-08-07T15:08:27.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2879000000000001e-05 2020-08-07T15:08:27.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418159+0000", 2020-08-07T15:08:27.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7880000000000003e-06 2020-08-07T15:08:27.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:27.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418165+0000", 2020-08-07T15:08:27.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42881086299999999 2020-08-07T15:08:27.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.846976+0000", 2020-08-07T15:08:27.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018687200000000001 2020-08-07T15:08:27.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:27.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.847163+0000", 2020-08-07T15:08:27.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000176665 2020-08-07T15:08:27.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:27.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.847339+0000", 2020-08-07T15:08:27.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022285949999999999 2020-08-07T15:08:27.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:27.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.849568+0000", 2020-08-07T15:08:27.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6000000000000003e-07 2020-08-07T15:08:27.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:27.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.849569+0000", 2020-08-07T15:08:27.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1811000000000003e-05 2020-08-07T15:08:27.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:27.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.849621+0000", 2020-08-07T15:08:27.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004143178 2020-08-07T15:08:27.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853764+0000", 2020-08-07T15:08:27.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064402749999999996 2020-08-07T15:08:27.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860204+0000", 2020-08-07T15:08:27.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1710000000000001e-06 2020-08-07T15:08:27.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:27.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860205+0000", 2020-08-07T15:08:27.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0001999999999999e-05 2020-08-07T15:08:27.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:27.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860265+0000", 2020-08-07T15:08:27.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44218349899999998 2020-08-07T15:08:27.777 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.777 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:27.777 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.778 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:178 4.9 4:96543f2a:::benchmark_data_smithi074_44586_object22:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:27.778 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.418151+0000", 2020-08-07T15:08:27.779 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.73182772900000004, 2020-08-07T15:08:27.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44221660699999998, 2020-08-07T15:08:27.779 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:27.779 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:27.804 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:27.804 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:27.804 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:27.805 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 178 2020-08-07T15:08:27.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.805 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:27.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:27.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418151+0000", 2020-08-07T15:08:27.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:27.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:27.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418151+0000", 2020-08-07T15:08:27.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9198000000000001e-05 2020-08-07T15:08:27.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:27.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418180+0000", 2020-08-07T15:08:27.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0199999999999999e-06 2020-08-07T15:08:27.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:27.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418183+0000", 2020-08-07T15:08:27.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9129999999999997e-06 2020-08-07T15:08:27.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:27.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418190+0000", 2020-08-07T15:08:27.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.655e-06 2020-08-07T15:08:27.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:27.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418193+0000", 2020-08-07T15:08:27.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0032999999999999e-05 2020-08-07T15:08:27.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418223+0000", 2020-08-07T15:08:27.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5979999999999996e-06 2020-08-07T15:08:27.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:27.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418228+0000", 2020-08-07T15:08:27.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42965734799999999 2020-08-07T15:08:27.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.847886+0000", 2020-08-07T15:08:27.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116982 2020-08-07T15:08:27.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:27.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.848003+0000", 2020-08-07T15:08:27.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021092100000000001 2020-08-07T15:08:27.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:27.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.848214+0000", 2020-08-07T15:08:27.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055933939999999998 2020-08-07T15:08:27.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:27.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853807+0000", 2020-08-07T15:08:27.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.178e-06 2020-08-07T15:08:27.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:27.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853808+0000", 2020-08-07T15:08:27.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7018000000000002e-05 2020-08-07T15:08:27.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:27.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853865+0000", 2020-08-07T15:08:27.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000335068 2020-08-07T15:08:27.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854200+0000", 2020-08-07T15:08:27.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061043219999999997 2020-08-07T15:08:27.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860304+0000", 2020-08-07T15:08:27.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.54e-07 2020-08-07T15:08:27.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:27.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860305+0000", 2020-08-07T15:08:27.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2205000000000005e-05 2020-08-07T15:08:27.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:27.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860367+0000", 2020-08-07T15:08:27.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44221660699999998 2020-08-07T15:08:27.831 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.831 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:27.831 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.832 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:179 4.9 4:96543f2a:::benchmark_data_smithi074_44586_object22:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:27.832 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.418212+0000", 2020-08-07T15:08:27.832 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.73176617200000005, 2020-08-07T15:08:27.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.442386379, 2020-08-07T15:08:27.833 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:27.833 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:27.833 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:27.833 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:27.834 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:27.834 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 179 2020-08-07T15:08:27.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.834 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:27.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:27.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418212+0000", 2020-08-07T15:08:27.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:27.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:27.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418212+0000", 2020-08-07T15:08:27.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6448000000000002e-05 2020-08-07T15:08:27.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:27.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418239+0000", 2020-08-07T15:08:27.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.695e-06 2020-08-07T15:08:27.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:27.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418242+0000", 2020-08-07T15:08:27.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2939999999999996e-06 2020-08-07T15:08:27.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:27.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418247+0000", 2020-08-07T15:08:27.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3810000000000002e-06 2020-08-07T15:08:27.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:27.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418249+0000", 2020-08-07T15:08:27.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7589e-05 2020-08-07T15:08:27.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418267+0000", 2020-08-07T15:08:27.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3139999999999999e-06 2020-08-07T15:08:27.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:27.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418270+0000", 2020-08-07T15:08:27.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43049494300000002 2020-08-07T15:08:27.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.848765+0000", 2020-08-07T15:08:27.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013311900000000001 2020-08-07T15:08:27.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:27.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.848898+0000", 2020-08-07T15:08:27.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026578799999999998 2020-08-07T15:08:27.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:27.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.849164+0000", 2020-08-07T15:08:27.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050703370000000003 2020-08-07T15:08:27.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854234+0000", 2020-08-07T15:08:27.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4944999999999997e-05 2020-08-07T15:08:27.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:27.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854279+0000", 2020-08-07T15:08:27.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0062045219999999996 2020-08-07T15:08:27.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860484+0000", 2020-08-07T15:08:27.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8199999999999998e-07 2020-08-07T15:08:27.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:27.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860485+0000", 2020-08-07T15:08:27.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6585e-05 2020-08-07T15:08:27.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:27.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860551+0000", 2020-08-07T15:08:27.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7537000000000002e-05 2020-08-07T15:08:27.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:27.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860599+0000", 2020-08-07T15:08:27.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.442386379 2020-08-07T15:08:27.861 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.861 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:27.861 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.862 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:180 4.9 4:96543f2a:::benchmark_data_smithi074_44586_object22:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:27.862 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.418982+0000", 2020-08-07T15:08:27.862 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.73099663999999998, 2020-08-07T15:08:27.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.441772624, 2020-08-07T15:08:27.862 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:27.863 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:27.863 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:27.863 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:27.863 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:27.863 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 180 2020-08-07T15:08:27.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.880 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:27.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:27.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418982+0000", 2020-08-07T15:08:27.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:27.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:27.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418982+0000", 2020-08-07T15:08:27.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9762000000000001e-05 2020-08-07T15:08:27.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:27.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.419012+0000", 2020-08-07T15:08:27.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1910000000000002e-06 2020-08-07T15:08:27.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:27.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.419016+0000", 2020-08-07T15:08:27.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1570000000000006e-06 2020-08-07T15:08:27.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:27.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.419025+0000", 2020-08-07T15:08:27.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3480000000000003e-06 2020-08-07T15:08:27.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:27.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.419030+0000", 2020-08-07T15:08:27.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2101000000000001e-05 2020-08-07T15:08:27.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.419052+0000", 2020-08-07T15:08:27.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4489999999999999e-06 2020-08-07T15:08:27.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:27.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.419056+0000", 2020-08-07T15:08:27.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43059229599999999 2020-08-07T15:08:27.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.849648+0000", 2020-08-07T15:08:27.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3301999999999995e-05 2020-08-07T15:08:27.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:27.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.849742+0000", 2020-08-07T15:08:27.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016384599999999999 2020-08-07T15:08:27.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:27.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.849905+0000", 2020-08-07T15:08:27.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043603569999999996 2020-08-07T15:08:27.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854266+0000", 2020-08-07T15:08:27.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.997e-05 2020-08-07T15:08:27.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:27.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854286+0000", 2020-08-07T15:08:27.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006358982 2020-08-07T15:08:27.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860645+0000", 2020-08-07T15:08:27.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.75e-07 2020-08-07T15:08:27.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:27.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860645+0000", 2020-08-07T15:08:27.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3958e-05 2020-08-07T15:08:27.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:27.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860709+0000", 2020-08-07T15:08:27.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5330000000000001e-05 2020-08-07T15:08:27.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:27.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860755+0000", 2020-08-07T15:08:27.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.441772624 2020-08-07T15:08:27.934 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.934 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:27.934 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.935 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:181 4.9 4:96543f2a:::benchmark_data_smithi074_44586_object22:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:27.935 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.420446+0000", 2020-08-07T15:08:27.936 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.72953270100000001, 2020-08-07T15:08:27.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44049454599999999, 2020-08-07T15:08:27.936 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:27.936 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:27.936 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:27.936 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:27.937 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:27.937 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 181 2020-08-07T15:08:27.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.937 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:27.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:27.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420446+0000", 2020-08-07T15:08:27.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:27.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:27.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420446+0000", 2020-08-07T15:08:27.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7520000000000001e-05 2020-08-07T15:08:27.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:27.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420473+0000", 2020-08-07T15:08:27.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9609999999999999e-06 2020-08-07T15:08:27.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:27.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420477+0000", 2020-08-07T15:08:27.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0050000000000004e-06 2020-08-07T15:08:27.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:27.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420485+0000", 2020-08-07T15:08:27.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1629999999999998e-06 2020-08-07T15:08:27.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:27.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420490+0000", 2020-08-07T15:08:27.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3699999999999999e-05 2020-08-07T15:08:27.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420523+0000", 2020-08-07T15:08:27.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.293e-06 2020-08-07T15:08:27.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:27.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420529+0000", 2020-08-07T15:08:27.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42978759300000002 2020-08-07T15:08:27.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.850316+0000", 2020-08-07T15:08:27.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101647 2020-08-07T15:08:27.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:27.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.850418+0000", 2020-08-07T15:08:27.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019151800000000001 2020-08-07T15:08:27.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:27.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.850609+0000", 2020-08-07T15:08:27.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037621099999999999 2020-08-07T15:08:27.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:27.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854371+0000", 2020-08-07T15:08:27.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9999999999999997e-07 2020-08-07T15:08:27.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:27.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854372+0000", 2020-08-07T15:08:27.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7818999999999998e-05 2020-08-07T15:08:27.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:27.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854410+0000", 2020-08-07T15:08:27.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002193459 2020-08-07T15:08:27.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.856603+0000", 2020-08-07T15:08:27.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042760380000000002 2020-08-07T15:08:27.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860879+0000", 2020-08-07T15:08:27.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2020-08-07T15:08:27.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:27.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860880+0000", 2020-08-07T15:08:27.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0449999999999999e-05 2020-08-07T15:08:27.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:27.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860940+0000", 2020-08-07T15:08:27.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44049454599999999 2020-08-07T15:08:27.960 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.961 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:27.961 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:27.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.961 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:182 4.9 4:96543f2a:::benchmark_data_smithi074_44586_object22:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:27.962 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.420527+0000", 2020-08-07T15:08:27.963 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.72945107499999995, 2020-08-07T15:08:27.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44055363400000003, 2020-08-07T15:08:27.963 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:27.964 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:27.964 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:27.964 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:27.965 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:27.965 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 182 2020-08-07T15:08:27.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.965 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:27.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:27.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420527+0000", 2020-08-07T15:08:27.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:27.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:27.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420527+0000", 2020-08-07T15:08:27.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3171999999999999e-05 2020-08-07T15:08:27.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:27.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420571+0000", 2020-08-07T15:08:27.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3900000000000001e-06 2020-08-07T15:08:27.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:27.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420576+0000", 2020-08-07T15:08:27.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6989999999999997e-06 2020-08-07T15:08:27.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:27.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420583+0000", 2020-08-07T15:08:27.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4410000000000002e-06 2020-08-07T15:08:27.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:27.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420586+0000", 2020-08-07T15:08:27.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8172e-05 2020-08-07T15:08:27.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420624+0000", 2020-08-07T15:08:27.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2298e-05 2020-08-07T15:08:27.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:27.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420637+0000", 2020-08-07T15:08:27.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43048289299999998 2020-08-07T15:08:27.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:27.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.851120+0000", 2020-08-07T15:08:27.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5534999999999999e-05 2020-08-07T15:08:27.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:27.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.851215+0000", 2020-08-07T15:08:27.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016653100000000001 2020-08-07T15:08:27.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:27.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.851382+0000", 2020-08-07T15:08:27.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030351480000000001 2020-08-07T15:08:27.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:27.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854417+0000", 2020-08-07T15:08:27.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1399999999999997e-07 2020-08-07T15:08:27.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:27.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854417+0000", 2020-08-07T15:08:27.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0976000000000001e-05 2020-08-07T15:08:27.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:27.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854468+0000", 2020-08-07T15:08:27.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039349759999999998 2020-08-07T15:08:27.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.858403+0000", 2020-08-07T15:08:27.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026043580000000002 2020-08-07T15:08:27.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:27.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.861008+0000", 2020-08-07T15:08:27.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7199999999999998e-07 2020-08-07T15:08:27.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:27.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.861008+0000", 2020-08-07T15:08:27.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2758999999999999e-05 2020-08-07T15:08:27.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:27.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:27.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:27.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.861081+0000", 2020-08-07T15:08:27.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44055363400000003 2020-08-07T15:08:27.999 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:28.000 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:28.000 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:28.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:28.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:28.001 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:183 4.9 4:96543f2a:::benchmark_data_smithi074_44586_object22:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:28.001 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.420764+0000", 2020-08-07T15:08:28.001 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.72921407800000004, 2020-08-07T15:08:28.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44042149400000002, 2020-08-07T15:08:28.002 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:28.002 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:28.003 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:28.003 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:28.003 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:28.003 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 183 2020-08-07T15:08:28.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:28.004 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:28.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:28.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:28.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420764+0000", 2020-08-07T15:08:28.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:28.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:28.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:28.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:28.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420764+0000", 2020-08-07T15:08:28.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.546e-05 2020-08-07T15:08:28.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:28.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:28.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:28.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420790+0000", 2020-08-07T15:08:28.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8459999999999999e-06 2020-08-07T15:08:28.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:28.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:28.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:28.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420793+0000", 2020-08-07T15:08:28.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7740000000000001e-06 2020-08-07T15:08:28.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:28.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:28.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:28.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420799+0000", 2020-08-07T15:08:28.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8710000000000001e-06 2020-08-07T15:08:28.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:28.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:28.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:28.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420801+0000", 2020-08-07T15:08:28.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6449e-05 2020-08-07T15:08:28.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:28.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:28.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:28.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420828+0000", 2020-08-07T15:08:28.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.882e-06 2020-08-07T15:08:28.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:28.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:28.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:28.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420833+0000", 2020-08-07T15:08:28.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43095108799999998 2020-08-07T15:08:28.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:28.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:28.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:28.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.851784+0000", 2020-08-07T15:08:28.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7198000000000002e-05 2020-08-07T15:08:28.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:28.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:28.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:28.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.851861+0000", 2020-08-07T15:08:28.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019497500000000001 2020-08-07T15:08:28.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:28.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:28.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:28.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.852056+0000", 2020-08-07T15:08:28.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024200630000000001 2020-08-07T15:08:28.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:28.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:28.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:28.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854476+0000", 2020-08-07T15:08:28.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9599999999999999e-07 2020-08-07T15:08:28.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:28.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:28.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:28.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854477+0000", 2020-08-07T15:08:28.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4832e-05 2020-08-07T15:08:28.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:28.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:28.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:28.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854522+0000", 2020-08-07T15:08:28.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039150890000000001 2020-08-07T15:08:28.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:28.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:28.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:28.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.858437+0000", 2020-08-07T15:08:28.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026888260000000001 2020-08-07T15:08:28.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:28.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:28.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:28.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.861125+0000", 2020-08-07T15:08:28.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6400000000000002e-07 2020-08-07T15:08:28.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:28.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:28.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:28.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.861126+0000", 2020-08-07T15:08:28.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9981000000000001e-05 2020-08-07T15:08:28.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:28.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:28.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:28.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.861186+0000", 2020-08-07T15:08:28.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44042149400000002 2020-08-07T15:08:28.043 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:28.043 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:28.043 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:28.043 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:28.044 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:28.044 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:28.047 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:08:28.048 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:08:28.048 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:08:28.048 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:08:28.048 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:08:28.049 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:08:28.050 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:08:28.050 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:08:28.050 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 1 16 867 851 6.64812 6.64844 0.00502367 0.0186035 2020-08-07T15:08:28.051 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:08:28.051 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:08:28.051 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:08:28.051 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:08:28.051 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:08:28.052 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:08:28.052 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:08:28.052 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:08:28.053 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:08:28.053 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:08:28.054 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:28.044+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:28.147 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:28.145+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:28.264 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:28.262+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:28.274 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 2 16 2655 2639 10.3079 13.9688 0.0119997 0.0120665 2020-08-07T15:08:28.287 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:28.288 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:28.338 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:08:28.409 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:28.407+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:28.472 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:28.468+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:28.522 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:28.522 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:28.543 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:08:28.594 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:28.592+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:28.679 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:28.677+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:28.758 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:28.758 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:28.772 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:08:28.822 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:28.819+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:28.902 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:28.889+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:28.995 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:28.993+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:29.024 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:29.025 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:29.035 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:08:29.102 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:29.097+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:29.206 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:29.202+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:29.246 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:29.246 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:29.254 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:08:29.285 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 3 16 4211 4195 10.9237 12.1562 0.0108294 0.0113439 2020-08-07T15:08:29.314 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:29.309+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:29.437 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:29.413+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:29.438 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:29.438 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:29.439 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:08:29.517 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:29.515+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:29.614 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:29.615 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:29.617 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:29.616+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:29.626 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:08:29.722 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:29.720+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:29.819 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:29.818+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:29.829 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:29.830 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:29.850 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:08:29.940 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:29.935+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:30.053 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:30.023+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:30.054 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:08:30.062 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:08:30.064 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:30.064 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:30.065 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:08:30.131 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:30.129+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:30.231 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:30.229+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:30.272 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 4 16 5761 5745 11.2199 12.1094 0.00996996 0.0110864 2020-08-07T15:08:30.315 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:30.316 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:30.319 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:08:30.325 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:08:30.342 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:30.335+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:30.359 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:08:30.402 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:08:30.451 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:30.438+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:30.452 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:08:30.489 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:08:30.530 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:08:30.546 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:30.545+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:30.559 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:30.559 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:08:30.560 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:08:30.560 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:08:30.560 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:30.568 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:08:30.569 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:08:30.604 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:08:30.608 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:08:30.608 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:08:30.609 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:08:30.609 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:08:30.609 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:08:30.609 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:08:30.610 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:08:30.610 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:08:30.666 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:30.663+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:30.758 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:30.756+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:30.792 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:30.796 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:08:30.796 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:08:30.796 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:08:30.797 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.797 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:15576 3.e e27/20 3:727724a5:::benchmark_data_smithi074_35447_object1946:head v 27'1032, mlcod=27'1023)", 2020-08-07T15:08:30.797 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:08:30.797 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 104.10078812899999, 2020-08-07T15:08:30.798 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.46091973000000003, 2020-08-07T15:08:30.798 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:30.798 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:30.798 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:30.798 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:30.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:08:30.799 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:30.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:30.800 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:08:30.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9592000000000001e-05 2020-08-07T15:08:30.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.801 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:30.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669542+0000", 2020-08-07T15:08:30.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.113e-06 2020-08-07T15:08:30.801 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.801 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.802 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:30.802 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669547+0000", 2020-08-07T15:08:30.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45151617799999999 2020-08-07T15:08:30.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:30.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121064+0000", 2020-08-07T15:08:30.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.377e-06 2020-08-07T15:08:30.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:30.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121067+0000", 2020-08-07T15:08:30.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9195e-05 2020-08-07T15:08:30.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:30.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121106+0000", 2020-08-07T15:08:30.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7509000000000001e-05 2020-08-07T15:08:30.806 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:30.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121124+0000", 2020-08-07T15:08:30.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0091783100000000003 2020-08-07T15:08:30.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:30.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.130302+0000", 2020-08-07T15:08:30.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000100456 2020-08-07T15:08:30.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.809 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:30.810 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.130402+0000", 2020-08-07T15:08:30.810 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.46091973000000003 2020-08-07T15:08:30.810 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:30.810 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:30.811 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:30.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.812 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:24264 3.8 e28/20 3:1d3fdbda:::benchmark_data_smithi074_35447_object3032:head v 28'1520, mlcod=28'1515)", 2020-08-07T15:08:30.812 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:08:30.812 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 98.387158092000007, 2020-08-07T15:08:30.813 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41856254199999998, 2020-08-07T15:08:30.813 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:30.813 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:30.814 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:30.814 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.814 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:30.815 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:08:30.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:30.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:30.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:08:30.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1348999999999998e-05 2020-08-07T15:08:30.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.818 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:30.818 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383174+0000", 2020-08-07T15:08:30.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6579999999999996e-06 2020-08-07T15:08:30.819 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:30.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383181+0000", 2020-08-07T15:08:30.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41194352699999998 2020-08-07T15:08:30.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:30.821 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.795124+0000", 2020-08-07T15:08:30.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5729999999999999e-06 2020-08-07T15:08:30.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.822 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:30.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.795128+0000", 2020-08-07T15:08:30.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002170384 2020-08-07T15:08:30.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:30.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.797298+0000", 2020-08-07T15:08:30.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5412e-05 2020-08-07T15:08:30.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:30.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.797324+0000", 2020-08-07T15:08:30.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042943360000000002 2020-08-07T15:08:30.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:30.826 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.801618+0000", 2020-08-07T15:08:30.826 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7302999999999997e-05 2020-08-07T15:08:30.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.827 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:30.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.801675+0000", 2020-08-07T15:08:30.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41856254199999998 2020-08-07T15:08:30.828 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:30.828 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:30.828 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:30.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.828 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:86712 3.8 e52/20 3:14925daa:::benchmark_data_smithi074_35447_object10838:head v 52'5488, mlcod=52'5479)", 2020-08-07T15:08:30.829 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.337908+0000", 2020-08-07T15:08:30.829 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 56.432362357999999, 2020-08-07T15:08:30.829 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.305870904, 2020-08-07T15:08:30.831 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:30.831 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:30.832 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:30.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.832 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:30.832 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.337908+0000", 2020-08-07T15:08:30.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:30.833 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:30.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.337908+0000", 2020-08-07T15:08:30.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1465999999999999e-05 2020-08-07T15:08:30.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.843 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:30.844 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.337950+0000", 2020-08-07T15:08:30.844 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1810000000000003e-06 2020-08-07T15:08:30.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.844 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:30.845 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.337954+0000", 2020-08-07T15:08:30.845 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30500923899999999 2020-08-07T15:08:30.845 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:30.846 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.642963+0000", 2020-08-07T15:08:30.846 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5389999999999998e-06 2020-08-07T15:08:30.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.846 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.846 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:30.846 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.642969+0000", 2020-08-07T15:08:30.847 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8265e-05 2020-08-07T15:08:30.847 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.848 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.848 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:30.848 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.643017+0000", 2020-08-07T15:08:30.849 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6812999999999999e-05 2020-08-07T15:08:30.849 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.849 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.850 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:30.851 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.643044+0000", 2020-08-07T15:08:30.851 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00067619800000000003 2020-08-07T15:08:30.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:30.852 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.643720+0000", 2020-08-07T15:08:30.853 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9203000000000002e-05 2020-08-07T15:08:30.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.853 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:30.854 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.643779+0000", 2020-08-07T15:08:30.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.305870904 2020-08-07T15:08:30.854 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:30.855 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:30.855 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:30.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.856 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:87088 3.6 3:6b0320df:::benchmark_data_smithi074_35447_object10885:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:08:30.856 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.958567+0000", 2020-08-07T15:08:30.856 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 55.811703799999997, 2020-08-07T15:08:30.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30442327000000002, 2020-08-07T15:08:30.857 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:30.857 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:30.857 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:30.857 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:30.857 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:30.858 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 87088 2020-08-07T15:08:30.858 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.858 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:30.859 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.859 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:30.859 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958567+0000", 2020-08-07T15:08:30.859 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:30.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.860 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:30.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958567+0000", 2020-08-07T15:08:30.860 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5040000000000002e-05 2020-08-07T15:08:30.860 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.861 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:30.861 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958612+0000", 2020-08-07T15:08:30.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4519999999999996e-06 2020-08-07T15:08:30.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.862 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.862 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:30.862 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958617+0000", 2020-08-07T15:08:30.862 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3974e-05 2020-08-07T15:08:30.862 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:30.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958631+0000", 2020-08-07T15:08:30.863 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5619999999999997e-06 2020-08-07T15:08:30.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:30.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958636+0000", 2020-08-07T15:08:30.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0071073999999999998 2020-08-07T15:08:30.864 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.865 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:30.865 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.965743+0000", 2020-08-07T15:08:30.865 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5477999999999996e-05 2020-08-07T15:08:30.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.866 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:30.866 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.965839+0000", 2020-08-07T15:08:30.866 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000168061 2020-08-07T15:08:30.866 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.866 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.867 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:30.867 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966007+0000", 2020-08-07T15:08:30.867 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054997550000000003 2020-08-07T15:08:30.867 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.867 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.868 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:30.868 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971507+0000", 2020-08-07T15:08:30.868 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6200000000000006e-07 2020-08-07T15:08:30.868 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.868 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.869 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:30.869 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971508+0000", 2020-08-07T15:08:30.869 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012566300000000001 2020-08-07T15:08:30.869 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.869 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.869 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:30.870 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971633+0000", 2020-08-07T15:08:30.870 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0075941899999999998 2020-08-07T15:08:30.870 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.870 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.870 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:30.871 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.979228+0000", 2020-08-07T15:08:30.871 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28372433899999999 2020-08-07T15:08:30.871 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.871 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:30.872 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262952+0000", 2020-08-07T15:08:30.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2799999999999995e-07 2020-08-07T15:08:30.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.872 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.872 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:30.873 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262953+0000", 2020-08-07T15:08:30.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7666000000000002e-05 2020-08-07T15:08:30.873 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.874 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:30.874 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262990+0000", 2020-08-07T15:08:30.874 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30442327000000002 2020-08-07T15:08:30.874 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:30.874 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:30.875 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:30.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.875 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:93920 3.6 e56/26 ondisk, result = 0)", 2020-08-07T15:08:30.875 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:08:30.876 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 51.439794829, 2020-08-07T15:08:30.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.397576347, 2020-08-07T15:08:30.876 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:30.876 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:30.876 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:30.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.877 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:30.877 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:08:30.877 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:30.877 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.877 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.878 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:30.878 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:08:30.878 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2571999999999999e-05 2020-08-07T15:08:30.878 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.878 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.879 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:30.879 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330509+0000", 2020-08-07T15:08:30.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2059999999999998e-06 2020-08-07T15:08:30.879 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.880 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:30.880 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330514+0000", 2020-08-07T15:08:30.880 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39740134599999999 2020-08-07T15:08:30.880 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.880 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:30.881 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727915+0000", 2020-08-07T15:08:30.881 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.677e-06 2020-08-07T15:08:30.881 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.881 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.883 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:30.883 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727921+0000", 2020-08-07T15:08:30.883 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5431e-05 2020-08-07T15:08:30.883 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.884 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.884 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:30.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727956+0000", 2020-08-07T15:08:30.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5790000000000004e-06 2020-08-07T15:08:30.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:30.885 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727965+0000", 2020-08-07T15:08:30.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7535999999999997e-05 2020-08-07T15:08:30.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.891 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:30.891 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.728052+0000", 2020-08-07T15:08:30.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.397576347 2020-08-07T15:08:30.892 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:30.892 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:30.892 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:30.892 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.893 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106104 3.0 3:0d7a2a89:::benchmark_data_smithi074_35447_object13262:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:08:30.893 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.252296+0000", 2020-08-07T15:08:30.896 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 43.517974940000002, 2020-08-07T15:08:30.896 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.44182183200000003, 2020-08-07T15:08:30.897 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:30.897 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:30.897 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:30.897 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:30.898 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:30.898 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106104 2020-08-07T15:08:30.898 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.898 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:30.898 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.899 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:30.899 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252296+0000", 2020-08-07T15:08:30.899 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:30.899 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.900 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:30.900 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252296+0000", 2020-08-07T15:08:30.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4728000000000001e-05 2020-08-07T15:08:30.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.901 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.901 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:30.901 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252321+0000", 2020-08-07T15:08:30.902 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9529999999999999e-06 2020-08-07T15:08:30.903 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.903 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:30.903 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252324+0000", 2020-08-07T15:08:30.903 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5759999999999996e-06 2020-08-07T15:08:30.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.904 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:30.904 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252329+0000", 2020-08-07T15:08:30.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.582e-06 2020-08-07T15:08:30.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:30.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252332+0000", 2020-08-07T15:08:30.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012958519999999999 2020-08-07T15:08:30.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.906 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.906 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:30.906 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.253628+0000", 2020-08-07T15:08:30.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7871e-05 2020-08-07T15:08:30.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:30.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.253705+0000", 2020-08-07T15:08:30.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017801199999999999 2020-08-07T15:08:30.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.908 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:08:30.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.253883+0000", 2020-08-07T15:08:30.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001869051 2020-08-07T15:08:30.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:30.910 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255752+0000", 2020-08-07T15:08:30.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3e-07 2020-08-07T15:08:30.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.910 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:30.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255753+0000", 2020-08-07T15:08:30.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5321000000000001e-05 2020-08-07T15:08:30.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:30.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255808+0000", 2020-08-07T15:08:30.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.091917211999999998 2020-08-07T15:08:30.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:30.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.347726+0000", 2020-08-07T15:08:30.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.34634211500000001 2020-08-07T15:08:30.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:30.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694068+0000", 2020-08-07T15:08:30.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3030000000000001e-06 2020-08-07T15:08:30.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:30.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694069+0000", 2020-08-07T15:08:30.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8625999999999998e-05 2020-08-07T15:08:30.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:30.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694118+0000", 2020-08-07T15:08:30.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.44182183200000003 2020-08-07T15:08:30.917 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:30.917 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:30.917 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:30.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.918 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106104 3.0 e58/20 ondisk, result = 0)", 2020-08-07T15:08:30.918 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.257196+0000", 2020-08-07T15:08:30.918 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 43.513074433, 2020-08-07T15:08:30.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43692689000000001, 2020-08-07T15:08:30.919 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:30.919 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:30.919 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:30.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:30.920 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257196+0000", 2020-08-07T15:08:30.920 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:30.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:30.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257196+0000", 2020-08-07T15:08:30.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2021999999999999e-05 2020-08-07T15:08:30.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.922 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:30.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257228+0000", 2020-08-07T15:08:30.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9549999999999997e-06 2020-08-07T15:08:30.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:30.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257234+0000", 2020-08-07T15:08:30.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43671786400000001 2020-08-07T15:08:30.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:30.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693952+0000", 2020-08-07T15:08:30.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7690000000000004e-06 2020-08-07T15:08:30.925 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.926 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:30.926 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693957+0000", 2020-08-07T15:08:30.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3294000000000006e-05 2020-08-07T15:08:30.926 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:30.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694040+0000", 2020-08-07T15:08:30.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3436e-05 2020-08-07T15:08:30.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:30.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694054+0000", 2020-08-07T15:08:30.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9549999999999996e-05 2020-08-07T15:08:30.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:30.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694123+0000", 2020-08-07T15:08:30.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43692689000000001 2020-08-07T15:08:30.929 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:30.930 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:30.930 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:30.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.930 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106144 3.6 3:629055a5:::benchmark_data_smithi074_35447_object13267:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:08:30.931 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.280142+0000", 2020-08-07T15:08:30.931 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 43.490128970999997, 2020-08-07T15:08:30.931 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.57968657199999996, 2020-08-07T15:08:30.931 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:30.931 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:30.932 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:30.932 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:30.932 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:30.932 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106144 2020-08-07T15:08:30.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.933 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:30.933 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.934 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:30.934 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280142+0000", 2020-08-07T15:08:30.934 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:30.934 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.934 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:30.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280142+0000", 2020-08-07T15:08:30.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.905e-05 2020-08-07T15:08:30.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:30.936 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280191+0000", 2020-08-07T15:08:30.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6729999999999997e-06 2020-08-07T15:08:30.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:30.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280197+0000", 2020-08-07T15:08:30.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0747999999999999e-05 2020-08-07T15:08:30.937 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.938 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:30.938 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280207+0000", 2020-08-07T15:08:30.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7700000000000001e-06 2020-08-07T15:08:30.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:30.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280212+0000", 2020-08-07T15:08:30.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00060223399999999999 2020-08-07T15:08:30.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.940 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.940 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:30.940 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280814+0000", 2020-08-07T15:08:30.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7831000000000006e-05 2020-08-07T15:08:30.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:30.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280912+0000", 2020-08-07T15:08:30.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000137119 2020-08-07T15:08:30.942 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.942 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:30.942 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.281049+0000", 2020-08-07T15:08:30.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023794369999999999 2020-08-07T15:08:30.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:30.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283429+0000", 2020-08-07T15:08:30.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.173e-06 2020-08-07T15:08:30.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:30.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283430+0000", 2020-08-07T15:08:30.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5530000000000004e-05 2020-08-07T15:08:30.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.946 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.946 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:30.946 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283495+0000", 2020-08-07T15:08:30.946 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5649000000000003e-05 2020-08-07T15:08:30.946 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.947 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.947 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:30.947 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283541+0000", 2020-08-07T15:08:30.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.57624206300000003 2020-08-07T15:08:30.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.948 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:30.948 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859783+0000", 2020-08-07T15:08:30.948 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.421e-06 2020-08-07T15:08:30.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.949 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:30.949 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859785+0000", 2020-08-07T15:08:30.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3874e-05 2020-08-07T15:08:30.950 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.950 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.950 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:30.950 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859828+0000", 2020-08-07T15:08:30.950 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.57968657199999996 2020-08-07T15:08:30.951 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:30.951 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:30.951 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:30.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.952 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106144 3.6 e59/26 ondisk, result = 0)", 2020-08-07T15:08:30.952 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.344536+0000", 2020-08-07T15:08:30.952 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 43.425734628999997, 2020-08-07T15:08:30.952 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51529786099999997, 2020-08-07T15:08:30.952 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:30.953 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:30.953 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:30.953 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.953 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:30.953 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344536+0000", 2020-08-07T15:08:30.954 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:30.955 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.955 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.955 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:30.956 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344536+0000", 2020-08-07T15:08:30.956 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7266999999999999e-05 2020-08-07T15:08:30.956 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.956 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.956 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:30.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344553+0000", 2020-08-07T15:08:30.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1070000000000001e-06 2020-08-07T15:08:30.957 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:30.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344557+0000", 2020-08-07T15:08:30.958 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.515143201 2020-08-07T15:08:30.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:30.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859700+0000", 2020-08-07T15:08:30.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.271e-05 2020-08-07T15:08:30.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:30.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859712+0000", 2020-08-07T15:08:30.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5436999999999998e-05 2020-08-07T15:08:30.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:30.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859768+0000", 2020-08-07T15:08:30.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1540000000000002e-06 2020-08-07T15:08:30.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.962 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:30.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859776+0000", 2020-08-07T15:08:30.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7985000000000001e-05 2020-08-07T15:08:30.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:30.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859834+0000", 2020-08-07T15:08:30.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51529786099999997 2020-08-07T15:08:30.963 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:30.964 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:30.964 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:30.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.964 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108352 3.8 3:1aac8373:::benchmark_data_smithi074_35447_object13543:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e62)", 2020-08-07T15:08:30.965 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.642093+0000", 2020-08-07T15:08:30.966 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 40.12817794, 2020-08-07T15:08:30.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39397409900000002, 2020-08-07T15:08:30.966 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:30.966 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:30.967 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:30.967 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:30.967 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:30.967 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108352 2020-08-07T15:08:30.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.968 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:30.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:30.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642093+0000", 2020-08-07T15:08:30.969 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:30.969 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.969 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.969 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:30.970 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642093+0000", 2020-08-07T15:08:30.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9475000000000002e-05 2020-08-07T15:08:30.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.970 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:30.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642122+0000", 2020-08-07T15:08:30.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0439999999999997e-06 2020-08-07T15:08:30.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:30.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642126+0000", 2020-08-07T15:08:30.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9689999999999994e-06 2020-08-07T15:08:30.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:30.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642134+0000", 2020-08-07T15:08:30.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1219999999999997e-06 2020-08-07T15:08:30.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:30.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642138+0000", 2020-08-07T15:08:30.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011580399999999999 2020-08-07T15:08:30.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.975 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:30.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.643296+0000", 2020-08-07T15:08:30.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3791999999999999e-05 2020-08-07T15:08:30.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:30.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.643390+0000", 2020-08-07T15:08:30.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011882700000000001 2020-08-07T15:08:30.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4,7", 2020-08-07T15:08:30.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.643509+0000", 2020-08-07T15:08:30.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034736799999999998 2020-08-07T15:08:30.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.979 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:30.979 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.646983+0000", 2020-08-07T15:08:30.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0077748490000000003 2020-08-07T15:08:30.979 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:30.980 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.654758+0000", 2020-08-07T15:08:30.980 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000005e-07 2020-08-07T15:08:30.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.981 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:30.981 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.654758+0000", 2020-08-07T15:08:30.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4258000000000002e-05 2020-08-07T15:08:30.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.982 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:30.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.654802+0000", 2020-08-07T15:08:30.982 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11166224399999999 2020-08-07T15:08:30.982 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.983 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.983 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:30.983 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.766465+0000", 2020-08-07T15:08:30.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26946740899999999 2020-08-07T15:08:30.983 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.984 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:30.984 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.035932+0000", 2020-08-07T15:08:30.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5829999999999999e-06 2020-08-07T15:08:30.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.985 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:30.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.035934+0000", 2020-08-07T15:08:30.985 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000133306 2020-08-07T15:08:30.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.987 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:30.987 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.036067+0000", 2020-08-07T15:08:30.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39397409900000002 2020-08-07T15:08:30.987 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:30.988 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:30.988 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:30.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.989 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108464 3.6 3:62704f92:::benchmark_data_smithi074_35447_object13557:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e62)", 2020-08-07T15:08:30.989 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.770928+0000", 2020-08-07T15:08:30.989 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 39.999342798999997, 2020-08-07T15:08:30.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30592449999999999, 2020-08-07T15:08:30.989 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:30.990 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:30.990 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:30.990 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:30.990 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:30.990 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108464 2020-08-07T15:08:30.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.991 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:30.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:30.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770928+0000", 2020-08-07T15:08:30.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:30.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:30.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770928+0000", 2020-08-07T15:08:30.993 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7257e-05 2020-08-07T15:08:30.993 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.993 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.993 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:30.994 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770955+0000", 2020-08-07T15:08:30.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0080000000000004e-06 2020-08-07T15:08:30.994 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.994 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.994 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:30.995 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770959+0000", 2020-08-07T15:08:30.995 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2810000000000003e-06 2020-08-07T15:08:30.995 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.995 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.995 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:30.996 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770967+0000", 2020-08-07T15:08:30.996 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3850000000000004e-06 2020-08-07T15:08:30.997 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.997 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.997 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:30.998 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770971+0000", 2020-08-07T15:08:30.998 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4960000000000002e-05 2020-08-07T15:08:30.998 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.998 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:30.999 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:30.999 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770996+0000", 2020-08-07T15:08:30.999 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4565999999999999e-05 2020-08-07T15:08:30.999 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:30.999 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.000 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:31.000 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.771050+0000", 2020-08-07T15:08:31.000 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010631799999999999 2020-08-07T15:08:31.000 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.000 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.001 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:31.001 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.771157+0000", 2020-08-07T15:08:31.001 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0049076099999999998 2020-08-07T15:08:31.001 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.001 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.002 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:31.002 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.776064+0000", 2020-08-07T15:08:31.002 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3700000000000001e-07 2020-08-07T15:08:31.002 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:31.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.776065+0000", 2020-08-07T15:08:31.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8387999999999998e-05 2020-08-07T15:08:31.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:31.004 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.776113+0000", 2020-08-07T15:08:31.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00068335599999999998 2020-08-07T15:08:31.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:31.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.776797+0000", 2020-08-07T15:08:31.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30001175499999999 2020-08-07T15:08:31.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:31.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.076808+0000", 2020-08-07T15:08:31.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4200000000000004e-07 2020-08-07T15:08:31.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:31.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.076809+0000", 2020-08-07T15:08:31.008 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3237e-05 2020-08-07T15:08:31.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.009 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.009 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:31.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.076852+0000", 2020-08-07T15:08:31.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30592449999999999 2020-08-07T15:08:31.010 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.010 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:31.010 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.011 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145767 3.6 3:62fe5854:::benchmark_data_smithi074_35447_object16158:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:08:31.011 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.654765+0000", 2020-08-07T15:08:31.011 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.115505754000001, 2020-08-07T15:08:31.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.62108287699999998, 2020-08-07T15:08:31.011 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:31.012 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:31.012 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:31.012 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:31.012 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:31.013 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145767 2020-08-07T15:08:31.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.013 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:31.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:31.014 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654765+0000", 2020-08-07T15:08:31.014 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:31.014 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.015 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.015 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:31.015 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654765+0000", 2020-08-07T15:08:31.015 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1977999999999998e-05 2020-08-07T15:08:31.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.016 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:31.016 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654797+0000", 2020-08-07T15:08:31.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6799999999999998e-06 2020-08-07T15:08:31.016 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:31.017 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654803+0000", 2020-08-07T15:08:31.017 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1263e-05 2020-08-07T15:08:31.018 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.018 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.018 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:31.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654814+0000", 2020-08-07T15:08:31.019 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2129999999999999e-06 2020-08-07T15:08:31.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.019 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.020 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:31.020 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654819+0000", 2020-08-07T15:08:31.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0324999999999997e-05 2020-08-07T15:08:31.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.020 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:31.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654869+0000", 2020-08-07T15:08:31.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015424199999999999 2020-08-07T15:08:31.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:31.022 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.655024+0000", 2020-08-07T15:08:31.022 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000163573 2020-08-07T15:08:31.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.023 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:31.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.655187+0000", 2020-08-07T15:08:31.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001341645 2020-08-07T15:08:31.023 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:31.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.656529+0000", 2020-08-07T15:08:31.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9599999999999998e-07 2020-08-07T15:08:31.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:31.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.656530+0000", 2020-08-07T15:08:31.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4381e-05 2020-08-07T15:08:31.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:31.026 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.656584+0000", 2020-08-07T15:08:31.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0057298979999999998 2020-08-07T15:08:31.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:31.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.662314+0000", 2020-08-07T15:08:31.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.61347776300000001 2020-08-07T15:08:31.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:31.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275792+0000", 2020-08-07T15:08:31.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9740000000000001e-06 2020-08-07T15:08:31.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:31.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275794+0000", 2020-08-07T15:08:31.030 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4146e-05 2020-08-07T15:08:31.030 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.030 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.030 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:31.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275848+0000", 2020-08-07T15:08:31.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.62108287699999998 2020-08-07T15:08:31.031 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.031 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:31.032 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.032 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145801 3.6 3:69e4cb22:::benchmark_data_smithi074_35447_object16192:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:08:31.032 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671123+0000", 2020-08-07T15:08:31.033 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.099147879, 2020-08-07T15:08:31.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74411769999999999, 2020-08-07T15:08:31.033 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:31.033 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:31.033 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:31.034 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:31.034 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:31.034 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145801 2020-08-07T15:08:31.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.035 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:31.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:31.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671123+0000", 2020-08-07T15:08:31.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:31.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:31.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671123+0000", 2020-08-07T15:08:31.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3691e-05 2020-08-07T15:08:31.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.037 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:31.037 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671157+0000", 2020-08-07T15:08:31.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0249999999999999e-06 2020-08-07T15:08:31.038 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.038 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:31.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671163+0000", 2020-08-07T15:08:31.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2339e-05 2020-08-07T15:08:31.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:31.040 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671175+0000", 2020-08-07T15:08:31.040 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5249999999999996e-06 2020-08-07T15:08:31.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.041 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:31.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671180+0000", 2020-08-07T15:08:31.042 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6826000000000002e-05 2020-08-07T15:08:31.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.042 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:31.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671227+0000", 2020-08-07T15:08:31.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014152699999999999 2020-08-07T15:08:31.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:31.043 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671369+0000", 2020-08-07T15:08:31.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8021000000000002e-05 2020-08-07T15:08:31.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.044 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:31.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671467+0000", 2020-08-07T15:08:31.048 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0016293760000000001 2020-08-07T15:08:31.049 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.052 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:31.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673096+0000", 2020-08-07T15:08:31.053 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.048e-06 2020-08-07T15:08:31.053 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.053 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:31.054 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673097+0000", 2020-08-07T15:08:31.054 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8841999999999998e-05 2020-08-07T15:08:31.054 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.054 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.054 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:31.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673146+0000", 2020-08-07T15:08:31.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002829935 2020-08-07T15:08:31.055 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.055 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:31.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675976+0000", 2020-08-07T15:08:31.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73917870399999996 2020-08-07T15:08:31.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:31.057 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.415155+0000", 2020-08-07T15:08:31.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.193e-06 2020-08-07T15:08:31.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:31.058 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.415157+0000", 2020-08-07T15:08:31.058 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3647999999999996e-05 2020-08-07T15:08:31.058 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.058 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.058 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:31.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.415241+0000", 2020-08-07T15:08:31.059 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74411769999999999 2020-08-07T15:08:31.059 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.059 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:31.059 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.059 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.060 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145803 3.0 3:0c939450:::benchmark_data_smithi074_35447_object16194:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:08:31.060 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671678+0000", 2020-08-07T15:08:31.060 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.098593084000001, 2020-08-07T15:08:31.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.882157211, 2020-08-07T15:08:31.061 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:31.061 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:31.061 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:31.061 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:31.061 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:31.062 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145803 2020-08-07T15:08:31.062 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.062 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:31.062 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.063 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:31.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671678+0000", 2020-08-07T15:08:31.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:31.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.064 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:31.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671678+0000", 2020-08-07T15:08:31.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4097999999999998e-05 2020-08-07T15:08:31.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.065 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:31.065 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671712+0000", 2020-08-07T15:08:31.065 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2439999999999998e-06 2020-08-07T15:08:31.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.066 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.067 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:31.067 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671718+0000", 2020-08-07T15:08:31.067 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2629999999999999e-05 2020-08-07T15:08:31.067 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.067 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.068 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:31.068 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671731+0000", 2020-08-07T15:08:31.068 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9589999999999998e-06 2020-08-07T15:08:31.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.068 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.069 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:31.069 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671738+0000", 2020-08-07T15:08:31.069 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2495000000000004e-05 2020-08-07T15:08:31.069 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.069 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.070 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:31.070 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671800+0000", 2020-08-07T15:08:31.070 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013640700000000001 2020-08-07T15:08:31.070 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.070 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.071 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:31.071 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671937+0000", 2020-08-07T15:08:31.071 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013672200000000001 2020-08-07T15:08:31.071 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.071 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.072 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:08:31.072 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.672073+0000", 2020-08-07T15:08:31.072 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001179982 2020-08-07T15:08:31.072 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.072 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.073 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:31.073 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673253+0000", 2020-08-07T15:08:31.073 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1900000000000001e-07 2020-08-07T15:08:31.073 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.073 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.074 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:31.074 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673254+0000", 2020-08-07T15:08:31.074 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2104000000000001e-05 2020-08-07T15:08:31.074 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.074 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.074 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:31.075 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673306+0000", 2020-08-07T15:08:31.075 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002264369 2020-08-07T15:08:31.075 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.075 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.075 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:31.076 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675571+0000", 2020-08-07T15:08:31.076 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.87820796899999998 2020-08-07T15:08:31.076 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.076 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.076 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:31.077 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553778+0000", 2020-08-07T15:08:31.077 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3629999999999999e-06 2020-08-07T15:08:31.077 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.077 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.077 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:31.077 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553780+0000", 2020-08-07T15:08:31.078 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5050000000000003e-05 2020-08-07T15:08:31.078 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.078 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.078 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:31.078 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553835+0000", 2020-08-07T15:08:31.079 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.882157211 2020-08-07T15:08:31.079 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.079 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:31.079 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.079 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.080 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.080 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145805 3.8 3:13b8e273:::benchmark_data_smithi074_35447_object16196:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:08:31.080 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671847+0000", 2020-08-07T15:08:31.080 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.098423764, 2020-08-07T15:08:31.080 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.340134453, 2020-08-07T15:08:31.081 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:31.081 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:31.081 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:31.081 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:31.081 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:31.081 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145805 2020-08-07T15:08:31.082 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.082 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:31.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.082 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:31.082 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671847+0000", 2020-08-07T15:08:31.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:31.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.083 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.083 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:31.083 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671847+0000", 2020-08-07T15:08:31.084 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1139999999999997e-05 2020-08-07T15:08:31.084 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.084 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:31.084 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671878+0000", 2020-08-07T15:08:31.085 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.784e-06 2020-08-07T15:08:31.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.085 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:31.085 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671884+0000", 2020-08-07T15:08:31.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2802999999999999e-05 2020-08-07T15:08:31.086 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.086 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.086 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:31.086 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671897+0000", 2020-08-07T15:08:31.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9909999999999999e-06 2020-08-07T15:08:31.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:31.087 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671903+0000", 2020-08-07T15:08:31.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011861 2020-08-07T15:08:31.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:31.088 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.672021+0000", 2020-08-07T15:08:31.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017702399999999999 2020-08-07T15:08:31.089 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.089 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.089 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:31.089 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.672198+0000", 2020-08-07T15:08:31.090 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020546400000000001 2020-08-07T15:08:31.090 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.090 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.090 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4,7", 2020-08-07T15:08:31.090 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.672404+0000", 2020-08-07T15:08:31.091 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024385019999999999 2020-08-07T15:08:31.091 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.091 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.091 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:31.091 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674842+0000", 2020-08-07T15:08:31.091 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00028594999999999998 2020-08-07T15:08:31.092 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.092 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.092 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:31.092 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675128+0000", 2020-08-07T15:08:31.092 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8100000000000003e-07 2020-08-07T15:08:31.093 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.093 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.093 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:31.093 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675129+0000", 2020-08-07T15:08:31.093 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7973999999999997e-05 2020-08-07T15:08:31.094 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.094 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.094 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:31.095 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675197+0000", 2020-08-07T15:08:31.095 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032171220000000002 2020-08-07T15:08:31.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.095 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.095 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:31.095 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678414+0000", 2020-08-07T15:08:31.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33343769299999998 2020-08-07T15:08:31.096 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.096 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.096 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:31.096 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011852+0000", 2020-08-07T15:08:31.097 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3480000000000001e-06 2020-08-07T15:08:31.097 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.097 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.098 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:31.098 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011853+0000", 2020-08-07T15:08:31.099 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000128467 2020-08-07T15:08:31.099 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.099 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.099 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:31.099 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011981+0000", 2020-08-07T15:08:31.100 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.340134453 2020-08-07T15:08:31.100 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.100 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:31.100 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.100 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.101 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.101 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145806 3.8 3:16669ddb:::benchmark_data_smithi074_35447_object16197:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:08:31.101 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.673992+0000", 2020-08-07T15:08:31.101 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.096279238000001, 2020-08-07T15:08:31.101 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40687891900000001, 2020-08-07T15:08:31.102 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:31.102 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:31.102 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:31.102 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:31.102 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:31.102 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145806 2020-08-07T15:08:31.103 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.103 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:31.103 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.103 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:31.103 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673992+0000", 2020-08-07T15:08:31.104 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:31.104 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.104 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.104 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:31.104 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673992+0000", 2020-08-07T15:08:31.105 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5429000000000002e-05 2020-08-07T15:08:31.105 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.105 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.105 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:31.105 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674027+0000", 2020-08-07T15:08:31.106 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3199999999999996e-06 2020-08-07T15:08:31.106 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.106 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.106 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:31.106 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674033+0000", 2020-08-07T15:08:31.106 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2835e-05 2020-08-07T15:08:31.107 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.107 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.107 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:31.107 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674046+0000", 2020-08-07T15:08:31.107 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9030000000000001e-06 2020-08-07T15:08:31.108 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.108 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.108 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:31.108 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674052+0000", 2020-08-07T15:08:31.108 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4181000000000003e-05 2020-08-07T15:08:31.109 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.109 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.109 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:31.109 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674096+0000", 2020-08-07T15:08:31.109 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00022310599999999999 2020-08-07T15:08:31.110 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.110 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.110 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:31.110 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674319+0000", 2020-08-07T15:08:31.110 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012681300000000001 2020-08-07T15:08:31.111 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.111 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.111 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4,7", 2020-08-07T15:08:31.111 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674446+0000", 2020-08-07T15:08:31.111 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002882754 2020-08-07T15:08:31.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.112 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:31.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.677329+0000", 2020-08-07T15:08:31.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9466e-05 2020-08-07T15:08:31.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.113 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:31.113 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.677418+0000", 2020-08-07T15:08:31.113 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0299999999999998e-07 2020-08-07T15:08:31.113 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.114 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:31.114 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.677419+0000", 2020-08-07T15:08:31.114 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5274000000000001e-05 2020-08-07T15:08:31.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.115 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.115 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:31.115 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.677464+0000", 2020-08-07T15:08:31.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40011127099999999 2020-08-07T15:08:31.115 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.116 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.116 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:31.116 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.077576+0000", 2020-08-07T15:08:31.116 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0031778370000000002 2020-08-07T15:08:31.116 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.116 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:31.117 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080754+0000", 2020-08-07T15:08:31.117 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.92e-07 2020-08-07T15:08:31.117 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.117 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.118 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:31.118 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080754+0000", 2020-08-07T15:08:31.118 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000116135 2020-08-07T15:08:31.119 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.119 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.119 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:31.119 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080870+0000", 2020-08-07T15:08:31.119 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40687891900000001 2020-08-07T15:08:31.120 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.120 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:31.120 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.120 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.121 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.121 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:145767 3.6 e75/26 ondisk, result = 0)", 2020-08-07T15:08:31.121 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.674108+0000", 2020-08-07T15:08:31.121 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.096162813999999, 2020-08-07T15:08:31.121 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.60174556899999998, 2020-08-07T15:08:31.122 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:31.122 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:31.122 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:31.122 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.122 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:31.123 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674108+0000", 2020-08-07T15:08:31.123 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:31.123 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.123 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.123 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:31.124 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674108+0000", 2020-08-07T15:08:31.124 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0558000000000001e-05 2020-08-07T15:08:31.124 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.124 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.124 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:31.124 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674139+0000", 2020-08-07T15:08:31.125 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6620000000000002e-06 2020-08-07T15:08:31.125 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.125 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.125 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:31.125 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674144+0000", 2020-08-07T15:08:31.126 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.60156723899999998 2020-08-07T15:08:31.126 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.126 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.126 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:31.127 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275711+0000", 2020-08-07T15:08:31.127 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4070000000000004e-06 2020-08-07T15:08:31.127 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.127 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.127 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:31.127 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275719+0000", 2020-08-07T15:08:31.128 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3205999999999998e-05 2020-08-07T15:08:31.128 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.128 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.128 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:31.128 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275772+0000", 2020-08-07T15:08:31.129 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0624e-05 2020-08-07T15:08:31.129 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.129 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.129 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:31.129 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275783+0000", 2020-08-07T15:08:31.130 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0872999999999997e-05 2020-08-07T15:08:31.130 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.130 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.130 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:31.130 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275854+0000", 2020-08-07T15:08:31.131 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.60174556899999998 2020-08-07T15:08:31.131 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.131 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:31.131 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.131 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.131 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.132 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:145803 3.0 e75/20 ondisk, result = 0)", 2020-08-07T15:08:31.132 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.674488+0000", 2020-08-07T15:08:31.132 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.095782832000001, 2020-08-07T15:08:31.132 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.87935229699999995, 2020-08-07T15:08:31.132 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:31.133 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:31.133 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:31.133 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.133 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:31.133 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674488+0000", 2020-08-07T15:08:31.134 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:31.134 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.134 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.134 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:31.134 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674488+0000", 2020-08-07T15:08:31.135 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3062000000000003e-05 2020-08-07T15:08:31.135 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.135 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.135 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:31.135 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674521+0000", 2020-08-07T15:08:31.136 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4911e-05 2020-08-07T15:08:31.136 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.136 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.136 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:31.136 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674536+0000", 2020-08-07T15:08:31.137 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.87914350600000002 2020-08-07T15:08:31.137 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.137 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.137 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:31.137 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553679+0000", 2020-08-07T15:08:31.138 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5950000000000001e-06 2020-08-07T15:08:31.138 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.138 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.138 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:31.138 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553687+0000", 2020-08-07T15:08:31.139 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2565999999999996e-05 2020-08-07T15:08:31.139 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.139 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.139 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:31.139 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553760+0000", 2020-08-07T15:08:31.139 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0417999999999999e-05 2020-08-07T15:08:31.140 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.140 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.140 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:31.140 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553770+0000", 2020-08-07T15:08:31.140 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0239000000000003e-05 2020-08-07T15:08:31.141 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.141 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.141 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:31.141 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553840+0000", 2020-08-07T15:08:31.141 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.87935229699999995 2020-08-07T15:08:31.142 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.142 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:31.142 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.142 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.142 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.142 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:145805 3.8 e75/60 ondisk, result = 0)", 2020-08-07T15:08:31.143 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.676843+0000", 2020-08-07T15:08:31.143 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.093427362, 2020-08-07T15:08:31.143 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.335144041, 2020-08-07T15:08:31.143 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:31.143 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:31.144 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:31.144 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.144 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:31.144 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.676843+0000", 2020-08-07T15:08:31.144 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:31.145 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.145 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.145 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:31.145 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.676843+0000", 2020-08-07T15:08:31.146 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7853e-05 2020-08-07T15:08:31.146 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.146 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.146 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:31.147 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.676861+0000", 2020-08-07T15:08:31.147 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7359999999999999e-06 2020-08-07T15:08:31.147 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.147 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.147 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:31.148 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.676865+0000", 2020-08-07T15:08:31.148 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33484658099999998 2020-08-07T15:08:31.148 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.148 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.148 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:31.149 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011712+0000", 2020-08-07T15:08:31.149 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1613e-05 2020-08-07T15:08:31.149 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.150 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.150 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:31.150 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011723+0000", 2020-08-07T15:08:31.150 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011038 2020-08-07T15:08:31.150 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.151 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:31.151 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011834+0000", 2020-08-07T15:08:31.151 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0097e-05 2020-08-07T15:08:31.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:31.152 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011844+0000", 2020-08-07T15:08:31.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000143781 2020-08-07T15:08:31.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.153 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:31.153 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011987+0000", 2020-08-07T15:08:31.153 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.335144041 2020-08-07T15:08:31.153 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.153 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:31.154 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.154 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.154 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.154 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:145806 3.8 e75/60 ondisk, result = 0)", 2020-08-07T15:08:31.154 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.678157+0000", 2020-08-07T15:08:31.155 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.092113309999998, 2020-08-07T15:08:31.155 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.402718152, 2020-08-07T15:08:31.155 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:31.155 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:31.155 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:31.155 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.156 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:31.156 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678157+0000", 2020-08-07T15:08:31.156 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:31.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.157 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:31.157 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678157+0000", 2020-08-07T15:08:31.157 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8496e-05 2020-08-07T15:08:31.157 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.157 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.158 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:31.158 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678176+0000", 2020-08-07T15:08:31.158 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.676e-06 2020-08-07T15:08:31.158 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.158 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.158 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:31.159 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678180+0000", 2020-08-07T15:08:31.159 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40250966999999999 2020-08-07T15:08:31.159 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.159 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.159 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:31.160 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080689+0000", 2020-08-07T15:08:31.160 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7110000000000003e-06 2020-08-07T15:08:31.160 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.160 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.160 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:31.160 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080694+0000", 2020-08-07T15:08:31.161 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4122999999999998e-05 2020-08-07T15:08:31.161 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.161 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.161 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:31.161 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080738+0000", 2020-08-07T15:08:31.162 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5809999999999999e-06 2020-08-07T15:08:31.162 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.162 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.162 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:31.162 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080746+0000", 2020-08-07T15:08:31.163 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012989499999999999 2020-08-07T15:08:31.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.163 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.163 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:31.163 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080876+0000", 2020-08-07T15:08:31.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.402718152 2020-08-07T15:08:31.164 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.164 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:31.164 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.164 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.165 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:31.165 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:31.166 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:08:31.170 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:31.168+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:31.177 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:08:31.177 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:08:31.178 INFO:tasks.thrashosds.thrasher:Growing pool device_health_metrics 2020-08-07T15:08:31.178 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:08:31.271 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:31.269+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:31.272 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 5 15 7475 7460 11.6554 13.3984 0.0129486 0.0107121 2020-08-07T15:08:31.342 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:31.342 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:08:31.342 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.343 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:7304 4.4 4:2da9e9a1:::benchmark_data_smithi074_44586_object912:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e85)", 2020-08-07T15:08:31.343 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:31.182209+0000", 2020-08-07T15:08:31.343 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.15333691799999999, 2020-08-07T15:08:31.343 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.15336155400000001, 2020-08-07T15:08:31.343 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:31.344 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:31.344 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:31.344 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:08:31.344 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:31.359 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7304 2020-08-07T15:08:31.359 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.359 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:31.360 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.360 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:31.360 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.182209+0000", 2020-08-07T15:08:31.360 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:31.360 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.361 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.361 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:31.361 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.182209+0000", 2020-08-07T15:08:31.361 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7351999999999998e-05 2020-08-07T15:08:31.361 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.362 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.362 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:31.362 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.182236+0000", 2020-08-07T15:08:31.362 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.512e-06 2020-08-07T15:08:31.362 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.363 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.363 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:31.363 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.182240+0000", 2020-08-07T15:08:31.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1302e-05 2020-08-07T15:08:31.363 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:31.364 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.182251+0000", 2020-08-07T15:08:31.364 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1370000000000004e-06 2020-08-07T15:08:31.364 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.365 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.365 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:31.365 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.182255+0000", 2020-08-07T15:08:31.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024353510000000001 2020-08-07T15:08:31.365 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.375 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:31.375 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.184690+0000", 2020-08-07T15:08:31.375 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5316999999999996e-05 2020-08-07T15:08:31.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.382 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:31.382 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.184786+0000", 2020-08-07T15:08:31.382 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018412300000000001 2020-08-07T15:08:31.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.382 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.383 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:08:31.383 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.184970+0000", 2020-08-07T15:08:31.383 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0025507059999999998 2020-08-07T15:08:31.383 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.383 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.384 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:31.384 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.187521+0000", 2020-08-07T15:08:31.384 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.96e-07 2020-08-07T15:08:31.384 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.384 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.385 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:31.385 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.187521+0000", 2020-08-07T15:08:31.385 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2775000000000002e-05 2020-08-07T15:08:31.385 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.385 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.386 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:31.386 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.187574+0000", 2020-08-07T15:08:31.387 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.040609377000000002 2020-08-07T15:08:31.387 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.387 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.387 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:31.387 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.228183+0000", 2020-08-07T15:08:31.388 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.045974347999999998 2020-08-07T15:08:31.388 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.388 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:31.389 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.389 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.390 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:08:31.391 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 1 2020-08-07T15:08:31.391 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:31.393 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:08:31.393 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:31.390+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:31.495 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:31.492+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:31.589 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:08:31.594 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:31.593+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:31.618 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:31.641 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:08:31.641 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:08:31.650 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:08:31.650 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:31.681 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:08:31.703 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2020-08-07T15:08:31.581602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2555904,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":595,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":105,"acting":105,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8406040,"kb_used_data":17368,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746568680,"statfs":{"total":773094113280,"available":764486328320,"internally_reserved":8589934592,"allocated":17784832,"data_stored":14490990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2555904,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.041214"},"pg_stats":[{"pgid":"4.e","version":"0'0","reported_seq":"11","reported_epoch":"83","state":"active+clean","last_fresh":"2020-08-07T15:08:25.541881+0000","last_change":"2020-08-07T15:08:24.548299+0000","last_active":"2020-08-07T15:08:25.541881+0000","last_peered":"2020-08-07T15:08:25.541881+0000","last_clean":"2020-08-07T15:08:25.541881+0000","last_became_active":"2020-08-07T15:08:24.547773+0000","last_became_peered":"2020-08-07T15:08:24.547773+0000","last_unstale":"2020-08-07T15:08:25.541881+0000","last_undegraded":"2020-08-07T15:08:25.541881+0000","last_fullsized":"2020-08-07T15:08:25.541881+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"83'16","reported_seq":"27","reported_epoch":"83","state":"active+clean","last_fresh":"2020-08-07T15:08:26.388880+0000","last_change":"2020-08-07T15:08:24.552802+0000","last_active":"2020-08-07T15:08:26.388880+0000","last_peered":"2020-08-07T15:08:26.388880+0000","last_clean":"2020-08-07T15:08:26.388880+0000","last_became_active":"2020-08-07T15:08:24.552127+0000","last_became_peered":"2020-08-07T15:08:24.552127+0000","last_unstale":"2020-08-07T15:08:26.388880+0000","last_undegraded":"2020-08-07T15:08:26.388880+0000","last_fullsized":"2020-08-07T15:08:26.388880+0000","mapping_epoch":81,"log_start":"83'10","ondisk_log_start":"83'10","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"85'80","reported_seq":"92","reported_epoch":"85","state":"active+clean","last_fresh":"2020-08-07T15:08:27.641680+0000","last_change":"2020-08-07T15:08:24.554472+0000","last_active":"2020-08-07T15:08:27.641680+0000","last_peered":"2020-08-07T15:08:27.641680+0000","last_clean":"2020-08-07T15:08:27.641680+0000","last_became_active":"2020-08-07T15:08:24.554021+0000","last_became_peered":"2020-08-07T15:08:24.554021+0000","last_unstale":"2020-08-07T15:08:27.641680+0000","last_undegraded":"2020-08-07T15:08:27.641680+0000","last_fullsized":"2020-08-07T15:08:27.641680+0000","mapping_epoch":81,"log_start":"85'71","ondisk_log_start":"85'71","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":"4","reported_epoch":"84","state":"activating","last_fresh":"2020-08-07T15:08:26.544513+0000","last_change":"2020-08-07T15:08:26.544513+0000","last_active":"2020-08-07T15:08:23.520041+0000","last_peered":"2020-08-07T15:08:23.520041+0000","last_clean":"2020-08-07T15:08:23.520041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:08:26.544513+0000","last_undegraded":"2020-08-07T15:08:26.544513+0000","last_fullsized":"2020-08-07T15:08:26.544513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":["5","1","4","6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"83'8","reported_seq":"19","reported_epoch":"83","state":"active+clean","last_fresh":"2020-08-07T15:08:26.304528+0000","last_change":"2020-08-07T15:08:24.550496+0000","last_active":"2020-08-07T15:08:26.304528+0000","last_peered":"2020-08-07T15:08:26.304528+0000","last_clean":"2020-08-07T15:08:26.304528+0000","last_became_active":"2020-08-07T15:08:24.549868+0000","last_became_peered":"2020-08-07T15:08:24.549868+0000","last_unstale":"2020-08-07T15:08:26.304528+0000","last_undegraded":"2020-08-07T15:08:26.304528+0000","last_fullsized":"2020-08-07T15:08:26.304528+0000","mapping_epoch":81,"log_start":"83'3","ondisk_log_start":"83'3","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.c","version":"83'16","reported_seq":"27","reported_epoch":"83","state":"active+clean","last_fresh":"2020-08-07T15:08:26.394105+0000","last_change":"2020-08-07T15:08:24.550212+0000","last_active":"2020-08-07T15:08:26.394105+0000","last_peered":"2020-08-07T15:08:26.394105+0000","last_clean":"2020-08-07T15:08:26.394105+0000","last_became_active":"2020-08-07T15:08:24.548660+0000","last_became_peered":"2020-08-07T15:08:24.548660+0000","last_unstale":"2020-08-07T15:08:26.394105+0000","last_undegraded":"2020-08-07T15:08:26.394105+0000","last_fullsized":"2020-08-07T15:08:26.394105+0000","mapping_epoch":81,"log_start":"83'10","ondisk_log_start":"83'10","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"113","reported_epoch":"82","state":"active+clean","last_fresh":"2020-08-07T15:08:24.546066+0000","last_change":"2020-08-07T15:07:56.631170+0000","last_active":"2020-08-07T15:08:24.546066+0000","last_peered":"2020-08-07T15:08:24.546066+0000","last_clean":"2020-08-07T15:08:24.546066+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:08:24.546066+0000","last_undegraded":"2020-08-07T15:08:24.546066+0000","last_fullsized":"2020-08-07T15:08:24.546066+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:56.631091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:56.631091+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"83'8","reported_seq":"19","reported_epoch":"83","state":"active+clean","last_fresh":"2020-08-07T15:08:26.336717+0000","last_change":"2020-08-07T15:08:24.550962+0000","last_active":"2020-08-07T15:08:26.336717+0000","last_peered":"2020-08-07T15:08:26.336717+0000","last_clean":"2020-08-07T15:08:26.336717+0000","last_became_active":"2020-08-07T15:08:24.550523+0000","last_became_peered":"2020-08-07T15:08:24.550523+0000","last_unstale":"2020-08-07T15:08:26.336717+0000","last_undegraded":"2020-08-07T15:08:26.336717+0000","last_fullsized":"2020-08-07T15:08:26.336717+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"50","reported_epoch":"83","state":"active+clean","last_fresh":"2020-08-07T15:08:25.542489+0000","last_change":"2020-08-07T15:07:33.177491+0000","last_active":"2020-08-07T15:08:25.542489+0000","last_peered":"2020-08-07T15:08:25.542489+0000","last_clean":"2020-08-07T15:08:25.542489+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:08:25.542489+0000","last_undegraded":"2020-08-07T15:08:25.542489+0000","last_fullsized":"2020-08-07T15:08:25.542489+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:33.177412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:33.177412+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"83'32","reported_seq":"43","reported_epoch":"83","state":"active+clean","last_fresh":"2020-08-07T15:08:26.473081+0000","last_change":"2020-08-07T15:08:24.554063+0000","last_active":"2020-08-07T15:08:26.473081+0000","last_peered":"2020-08-07T15:08:26.473081+0000","last_clean":"2020-08-07T15:08:26.473081+0000","last_became_active":"2020-08-07T15:08:24.553668+0000","last_became_peered":"2020-08-07T15:08:24.553668+0000","last_unstale":"2020-08-07T15:08:26.473081+0000","last_undegraded":"2020-08-07T15:08:26.473081+0000","last_fullsized":"2020-08-07T15:08:26.473081+0000","mapping_epoch":81,"log_start":"83'25","ondisk_log_start":"83'25","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"50","reported_epoch":"83","state":"active+clean","last_fresh":"2020-08-07T15:08:25.542672+0000","last_change":"2020-08-07T15:07:36.143434+0000","last_active":"2020-08-07T15:08:25.542672+0000","last_peered":"2020-08-07T15:08:25.542672+0000","last_clean":"2020-08-07T15:08:25.542672+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:08:25.542672+0000","last_undegraded":"2020-08-07T15:08:25.542672+0000","last_fullsized":"2020-08-07T15:08:25.542672+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.143353+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"83'8","reported_seq":"19","reported_epoch":"83","state":"active+clean","last_fresh":"2020-08-07T15:08:26.411257+0000","last_change":"2020-08-07T15:08:24.552962+0000","last_active":"2020-08-07T15:08:26.411257+0000","last_peered":"2020-08-07T15:08:26.411257+0000","last_clean":"2020-08-07T15:08:26.411257+0000","last_became_active":"2020-08-07T15:08:24.552070+0000","last_became_peered":"2020-08-07T15:08:24.552070+0000","last_unstale":"2020-08-07T15:08:26.411257+0000","last_undegraded":"2020-08-07T15:08:26.411257+0000","last_fullsized":"2020-08-07T15:08:26.411257+0000","mapping_epoch":81,"log_start":"83'4","ondisk_log_start":"83'4","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"89","reported_epoch":"84","state":"active+clean","last_fresh":"2020-08-07T15:08:26.848467+0000","last_change":"2020-08-07T15:07:51.357310+0000","last_active":"2020-08-07T15:08:26.848467+0000","last_peered":"2020-08-07T15:08:26.848467+0000","last_clean":"2020-08-07T15:08:26.848467+0000","last_became_active":"2020-08-07T15:07:51.356798+0000","last_became_peered":"2020-08-07T15:07:51.356798+0000","last_unstale":"2020-08-07T15:08:26.848467+0000","last_undegraded":"2020-08-07T15:08:26.848467+0000","last_fullsized":"2020-08-07T15:08:26.848467+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.927824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.927824+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"74","reported_epoch":"82","state":"active+clean","last_fresh":"2020-08-07T15:08:24.546194+0000","last_change":"2020-08-07T15:08:15.887617+0000","last_active":"2020-08-07T15:08:24.546194+0000","last_peered":"2020-08-07T15:08:24.546194+0000","last_clean":"2020-08-07T15:08:24.546194+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:08:24.546194+0000","last_undegraded":"2020-08-07T15:08:24.546194+0000","last_fullsized":"2020-08-07T15:08:24.546194+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:15.887513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:15.887513+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"83'24","reported_seq":"34","reported_epoch":"83","state":"active+clean","last_fresh":"2020-08-07T15:08:26.368246+0000","last_change":"2020-08-07T15:08:24.558242+0000","last_active":"2020-08-07T15:08:26.368246+0000","last_peered":"2020-08-07T15:08:26.368246+0000","last_clean":"2020-08-07T15:08:26.368246+0000","last_became_active":"2020-08-07T15:08:24.557354+0000","last_became_peered":"2020-08-07T15:08:24.557354+0000","last_unstale":"2020-08-07T15:08:26.368246+0000","last_undegraded":"2020-08-07T15:08:26.368246+0000","last_fullsized":"2020-08-07T15:08:26.368246+0000","mapping_epoch":81,"log_start":"83'20","ondisk_log_start":"83'20","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"64","reported_epoch":"83","state":"active+clean","last_fresh":"2020-08-07T15:08:25.542603+0000","last_change":"2020-08-07T15:08:15.279484+0000","last_active":"2020-08-07T15:08:25.542603+0000","last_peered":"2020-08-07T15:08:25.542603+0000","last_clean":"2020-08-07T15:08:25.542603+0000","last_became_active":"2020-08-07T15:07:33.170597+0000","last_became_peered":"2020-08-07T15:07:33.170597+0000","last_unstale":"2020-08-07T15:08:25.542603+0000","last_undegraded":"2020-08-07T15:08:25.542603+0000","last_fullsized":"2020-08-07T15:08:25.542603+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:15.279411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:15.279411+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"75","reported_epoch":"83","state":"active+clean","last_fresh":"2020-08-07T15:08:25.542766+0000","last_change":"2020-08-07T15:06:23.074962+0000","last_active":"2020-08-07T15:08:25.542766+0000","last_peered":"2020-08-07T15:08:25.542766+0000","last_clean":"2020-08-07T15:08:25.542766+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:08:25.542766+0000","last_undegraded":"2020-08-07T15:08:25.542766+0000","last_fullsized":"2020-08-07T15:08:25.542766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"83'16","reported_seq":"27","reported_epoch":"83","state":"active+clean","last_fresh":"2020-08-07T15:08:26.299805+0000","last_change":"2020-08-07T15:08:24.550789+0000","last_active":"2020-08-07T15:08:26.299805+0000","last_peered":"2020-08-07T15:08:26.299805+0000","last_clean":"2020-08-07T15:08:26.299805+0000","last_became_active":"2020-08-07T15:08:24.550222+0000","last_became_peered":"2020-08-07T15:08:24.550222+0000","last_unstale":"2020-08-07T15:08:26.299805+0000","last_undegraded":"2020-08-07T15:08:26.299805+0000","last_fullsized":"2020-08-07T15:08:26.299805+0000","mapping_epoch":81,"log_start":"83'6","ondisk_log_start":"83'6","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"113","reported_epoch":"82","state":"active+clean","last_fresh":"2020-08-07T15:08:24.546263+0000","last_change":"2020-08-07T15:07:36.358882+0000","last_active":"2020-08-07T15:08:24.546263+0000","last_peered":"2020-08-07T15:08:24.546263+0000","last_clean":"2020-08-07T15:08:24.546263+0000","last_became_active":"2020-08-07T15:07:33.171449+0000","last_became_peered":"2020-08-07T15:07:33.171449+0000","last_unstale":"2020-08-07T15:08:24.546263+0000","last_undegraded":"2020-08-07T15:08:24.546263+0000","last_fullsized":"2020-08-07T15:08:24.546263+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.358798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.358798+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"77","reported_epoch":"82","state":"active+clean","last_fresh":"2020-08-07T15:08:24.546366+0000","last_change":"2020-08-07T15:07:36.864560+0000","last_active":"2020-08-07T15:08:24.546366+0000","last_peered":"2020-08-07T15:08:24.546366+0000","last_clean":"2020-08-07T15:08:24.546366+0000","last_became_active":"2020-08-07T15:06:23.223489+0000","last_became_peered":"2020-08-07T15:06:23.223489+0000","last_unstale":"2020-08-07T15:08:24.546366+0000","last_undegraded":"2020-08-07T15:08:24.546366+0000","last_fullsized":"2020-08-07T15:08:24.546366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:07:36.864483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.864483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"85'72","reported_seq":"83","reported_epoch":"85","state":"active+clean","last_fresh":"2020-08-07T15:08:27.651722+0000","last_change":"2020-08-07T15:08:24.552570+0000","last_active":"2020-08-07T15:08:27.651722+0000","last_peered":"2020-08-07T15:08:27.651722+0000","last_clean":"2020-08-07T15:08:27.651722+0000","last_became_active":"2020-08-07T15:08:24.552105+0000","last_became_peered":"2020-08-07T15:08:24.552105+0000","last_unstale":"2020-08-07T15:08:27.651722+0000","last_undegraded":"2020-08-07T15:08:27.651722+0000","last_fullsized":"2020-08-07T15:08:27.651722+0000","mapping_epoch":81,"log_start":"85'65","ondisk_log_start":"85'65","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":589824,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":72,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"78","reported_epoch":"83","state":"active+clean","last_fresh":"2020-08-07T15:08:26.317430+0000","last_change":"2020-08-07T15:08:26.317430+0000","last_active":"2020-08-07T15:08:26.317430+0000","last_peered":"2020-08-07T15:08:26.317430+0000","last_clean":"2020-08-07T15:08:26.317430+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:08:26.317430+0000","last_undegraded":"2020-08-07T15:08:26.317430+0000","last_fullsized":"2020-08-07T15:08:26.317430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:26.317351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:26.317351+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"97","reported_epoch":"82","state":"active+clean","last_fresh":"2020-08-07T15:08:24.547033+0000","last_change":"2020-08-07T15:07:32.461010+0000","last_active":"2020-08-07T15:08:24.547033+0000","last_peered":"2020-08-07T15:08:24.547033+0000","last_clean":"2020-08-07T15:08:24.547033+0000","last_became_active":"2020-08-07T15:07:28.210245+0000","last_became_peered":"2020-08-07T15:07:28.210245+0000","last_unstale":"2020-08-07T15:08:24.547033+0000","last_undegraded":"2020-08-07T15:08:24.547033+0000","last_fullsized":"2020-08-07T15:08:24.547033+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:32.460945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:32.460945+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"83'8","reported_seq":"18","reported_epoch":"83","state":"active+clean","last_fresh":"2020-08-07T15:08:26.321473+0000","last_change":"2020-08-07T15:08:24.557398+0000","last_active":"2020-08-07T15:08:26.321473+0000","last_peered":"2020-08-07T15:08:26.321473+0000","last_clean":"2020-08-07T15:08:26.321473+0000","last_became_active":"2020-08-07T15:08:24.557022+0000","last_became_peered":"2020-08-07T15:08:24.557022+0000","last_unstale":"2020-08-07T15:08:26.321473+0000","last_undegraded":"2020-08-07T15:08:26.321473+0000","last_fullsized":"2020-08-07T15:08:26.321473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"72","reported_epoch":"84","state":"active+clean","last_fresh":"2020-08-07T15:08:26.848089+0000","last_change":"2020-08-07T15:08:02.901198+0000","last_active":"2020-08-07T15:08:26.848089+0000","last_peered":"2020-08-07T15:08:26.848089+0000","last_clean":"2020-08-07T15:08:26.848089+0000","last_became_active":"2020-08-07T15:07:33.156569+0000","last_became_peered":"2020-08-07T15:07:33.156569+0000","last_unstale":"2020-08-07T15:08:26.848089+0000","last_undegraded":"2020-08-07T15:08:26.848089+0000","last_fullsized":"2020-08-07T15:08:26.848089+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:02.901115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:02.901115+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"77","reported_epoch":"82","state":"active+clean","last_fresh":"2020-08-07T15:08:24.547138+0000","last_change":"2020-08-07T15:08:13.867355+0000","last_active":"2020-08-07T15:08:24.547138+0000","last_peered":"2020-08-07T15:08:24.547138+0000","last_clean":"2020-08-07T15:08:24.547138+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:08:24.547138+0000","last_undegraded":"2020-08-07T15:08:24.547138+0000","last_fullsized":"2020-08-07T15:08:24.547138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:13.867148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:13.867148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"83'16","reported_seq":"27","reported_epoch":"83","state":"active+clean","last_fresh":"2020-08-07T15:08:26.417919+0000","last_change":"2020-08-07T15:08:24.549715+0000","last_active":"2020-08-07T15:08:26.417919+0000","last_peered":"2020-08-07T15:08:26.417919+0000","last_clean":"2020-08-07T15:08:26.417919+0000","last_became_active":"2020-08-07T15:08:24.549245+0000","last_became_peered":"2020-08-07T15:08:24.549245+0000","last_unstale":"2020-08-07T15:08:26.417919+0000","last_undegraded":"2020-08-07T15:08:26.417919+0000","last_fullsized":"2020-08-07T15:08:26.417919+0000","mapping_epoch":81,"log_start":"83'11","ondisk_log_start":"83'11","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"72","reported_epoch":"84","state":"active+clean","last_fresh":"2020-08-07T15:08:26.848339+0000","last_change":"2020-08-07T15:08:09.841524+0000","last_active":"2020-08-07T15:08:26.848339+0000","last_peered":"2020-08-07T15:08:26.848339+0000","last_clean":"2020-08-07T15:08:26.848339+0000","last_became_active":"2020-08-07T15:07:33.169518+0000","last_became_peered":"2020-08-07T15:07:33.169518+0000","last_unstale":"2020-08-07T15:08:26.848339+0000","last_undegraded":"2020-08-07T15:08:26.848339+0000","last_fullsized":"2020-08-07T15:08:26.848339+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:09.841428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:09.841428+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"74","reported_epoch":"82","state":"active+clean","last_fresh":"2020-08-07T15:08:24.546247+0000","last_change":"2020-08-07T15:06:23.225886+0000","last_active":"2020-08-07T15:08:24.546247+0000","last_peered":"2020-08-07T15:08:24.546247+0000","last_clean":"2020-08-07T15:08:24.546247+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:08:24.546247+0000","last_undegraded":"2020-08-07T15:08:24.546247+0000","last_fullsized":"2020-08-07T15:08:24.546247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:06:22.062496+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"83'8","reported_seq":"18","reported_epoch":"83","state":"active+clean","last_fresh":"2020-08-07T15:08:26.336606+0000","last_change":"2020-08-07T15:08:24.554330+0000","last_active":"2020-08-07T15:08:26.336606+0000","last_peered":"2020-08-07T15:08:26.336606+0000","last_clean":"2020-08-07T15:08:26.336606+0000","last_became_active":"2020-08-07T15:08:24.553881+0000","last_became_peered":"2020-08-07T15:08:24.553881+0000","last_unstale":"2020-08-07T15:08:26.336606+0000","last_undegraded":"2020-08-07T15:08:26.336606+0000","last_fullsized":"2020-08-07T15:08:26.336606+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"51","reported_epoch":"83","state":"active+clean","last_fresh":"2020-08-07T15:08:25.541945+0000","last_change":"2020-08-07T15:08:15.585668+0000","last_active":"2020-08-07T15:08:25.541945+0000","last_peered":"2020-08-07T15:08:25.541945+0000","last_clean":"2020-08-07T15:08:25.541945+0000","last_became_active":"2020-08-07T15:07:33.155139+0000","last_became_peered":"2020-08-07T15:07:33.155139+0000","last_unstale":"2020-08-07T15:08:25.541945+0000","last_undegraded":"2020-08-07T15:08:25.541945+0000","last_fullsized":"2020-08-07T15:08:25.541945+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:15.585597+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":595,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"78","reported_epoch":"83","state":"active+clean","last_fresh":"2020-08-07T15:08:25.545903+0000","last_change":"2020-08-07T15:07:34.423236+0000","last_active":"2020-08-07T15:08:25.545903+0000","last_peered":"2020-08-07T15:08:25.545903+0000","last_clean":"2020-08-07T15:08:25.545903+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:08:25.545903+0000","last_undegraded":"2020-08-07T15:08:25.545903+0000","last_fullsized":"2020-08-07T15:08:25.545903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:07:34.423115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:07:34.423115+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"11","reported_epoch":"83","state":"active+clean","last_fresh":"2020-08-07T15:08:25.541693+0000","last_change":"2020-08-07T15:08:24.550895+0000","last_active":"2020-08-07T15:08:25.541693+0000","last_peered":"2020-08-07T15:08:25.541693+0000","last_clean":"2020-08-07T15:08:25.541693+0000","last_became_active":"2020-08-07T15:08:24.550490+0000","last_became_peered":"2020-08-07T15:08:24.550490+0000","last_unstale":"2020-08-07T15:08:25.541693+0000","last_undegraded":"2020-08-07T15:08:25.541693+0000","last_fullsized":"2020-08-07T15:08:25.541693+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"56","reported_epoch":"82","state":"active+clean","last_fresh":"2020-08-07T15:08:24.539976+0000","last_change":"2020-08-07T15:07:34.653928+0000","last_active":"2020-08-07T15:08:24.539976+0000","last_peered":"2020-08-07T15:08:24.539976+0000","last_clean":"2020-08-07T15:08:24.539976+0000","last_became_active":"2020-08-07T15:07:33.161761+0000","last_became_peered":"2020-08-07T15:07:33.161761+0000","last_unstale":"2020-08-07T15:08:24.539976+0000","last_undegraded":"2020-08-07T15:08:24.539976+0000","last_fullsized":"2020-08-07T15:08:24.539976+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:34.653843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:34.653843+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"79","reported_epoch":"84","state":"active+clean","last_fresh":"2020-08-07T15:08:26.848566+0000","last_change":"2020-08-07T15:08:11.840861+0000","last_active":"2020-08-07T15:08:26.848566+0000","last_peered":"2020-08-07T15:08:26.848566+0000","last_clean":"2020-08-07T15:08:26.848566+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:08:26.848566+0000","last_undegraded":"2020-08-07T15:08:26.848566+0000","last_fullsized":"2020-08-07T15:08:26.848566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:11.840807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:11.840807+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":2555904,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":2496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8282112,"data_stored":8282112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":595,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":33,"acting":33,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049664,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322176,"statfs":{"total":96636764160,"available":95561908224,"internally_reserved":1073741824,"allocated":1105920,"data_stored":688549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":0.95099999999999996,"15min":0.91200000000000003},"min":{"1min":0.48199999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":3.4159999999999999,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.0940000000000001,"5min":0.98899999999999999,"15min":0.97099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.3559999999999999,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.66400000000000003}]},{"osd":1,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3069999999999999,"15min":1.2989999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":4.4829999999999997,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.044,"15min":0.99199999999999999},"min":{"1min":0.61799999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":3.4620000000000002,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":1.1519999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.179,"15min":1.1439999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":4.423,"5min":4.423,"15min":4.423},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.5,"5min":1.194,"15min":1.143},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":3.5099999999999998,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":1.216}]},{"osd":3,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.5900000000000001,"15min":1.6439999999999999},"min":{"1min":0.73799999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.5430000000000001,"5min":5.5019999999999998,"15min":5.5019999999999998},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.329,"15min":1.3380000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.4590000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.93799999999999994}]},{"osd":4,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.7849999999999999,"5min":1.377,"15min":1.3089999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":4.835,"5min":4.835,"15min":4.835},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.3300000000000001,"15min":1.3180000000000001},"min":{"1min":0.76900000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.4470000000000001,"5min":2.778,"15min":2.778},"last":1.238}]},{"osd":5,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":2.3940000000000001,"5min":30.484000000000002,"15min":35.165999999999997},"min":{"1min":0.96799999999999997,"5min":0.875,"15min":0.875},"max":{"1min":7.4429999999999996,"5min":831.88099999999997,"15min":831.88099999999997},"last":1.3069999999999999},{"interface":"front","average":{"1min":2.456,"5min":1.6870000000000001,"15min":1.5589999999999999},"min":{"1min":1,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":7.5350000000000001,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":1.3360000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.542,"15min":1.526},"min":{"1min":0.83599999999999997,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.9900000000000002,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.9219999999999999,"5min":1.5900000000000001,"15min":1.534},"min":{"1min":1.115,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":4.5170000000000003,"5min":4.5170000000000003,"15min":4.5170000000000003},"last":1.4610000000000001}]}]},{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049244,"kb_used_data":660,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322596,"statfs":{"total":96636764160,"available":95562338304,"internally_reserved":1073741824,"allocated":675840,"data_stored":265990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.2490000000000001,"15min":1.3049999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.6439999999999999,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":1.038},{"interface":"front","average":{"1min":1.079,"5min":1.135,"15min":1.145},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":2.9950000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.3120000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.95999999999999996,"15min":0.95799999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.9890000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.121},{"interface":"front","average":{"1min":19.103999999999999,"5min":4.9299999999999997,"15min":2.5680000000000001},"min":{"1min":0.75600000000000001,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":372.50799999999998,"5min":372.50799999999998,"15min":372.50799999999998},"last":0.89200000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2,"15min":1.196},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":3.036,"5min":3.036,"15min":3.036},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.236,"15min":1.26},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.7440000000000002,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":1.0780000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.4870000000000001,"15min":1.5129999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":3.46,"5min":3.46,"15min":3.46},"last":1.238},{"interface":"front","average":{"1min":1.268,"5min":1.3640000000000001,"15min":1.3799999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":3.1970000000000001,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":1.1020000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.5880000000000001,"15min":1.6379999999999999},"min":{"1min":0.72499999999999998,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.7999999999999998,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.653,"5min":1.6679999999999999,"15min":1.6699999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":3.694,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":0.96199999999999997}]},{"osd":5,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.3260000000000001,"15min":1.351},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.6360000000000001,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.393,"5min":1.6910000000000001,"15min":1.7410000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.8279999999999998,"5min":5.327,"15min":5.327},"last":1.292}]},{"osd":6,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.446,"15min":1.4370000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.7799999999999998,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.262},{"interface":"front","average":{"1min":1.4730000000000001,"5min":14.955,"15min":17.202000000000002},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":3.7309999999999999,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.333}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049244,"kb_used_data":660,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322596,"statfs":{"total":96636764160,"available":95562338304,"internally_reserved":1073741824,"allocated":675840,"data_stored":265990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":27.559999999999999,"15min":31.977},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.1820000000000004,"5min":497.04000000000002,"15min":497.04000000000002},"last":0.65400000000000003},{"interface":"front","average":{"1min":44.500999999999998,"5min":9.8049999999999997,"15min":4.0220000000000002},"min":{"1min":0.64200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":993.84500000000003,"5min":993.84500000000003,"15min":993.84500000000003},"last":0.73299999999999998}]},{"osd":2,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":0.98799999999999999,"15min":0.92800000000000005},"min":{"1min":0.57399999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":4.585,"5min":4.585,"15min":4.585},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.2410000000000001,"15min":1.216},"min":{"1min":0.51800000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":4.149,"5min":4.149,"15min":4.149},"last":0.88200000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.962,"5min":1.4410000000000001,"15min":1.3540000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":10.603999999999999,"5min":10.603999999999999,"15min":10.603999999999999},"last":1.381},{"interface":"front","average":{"1min":2.1909999999999998,"5min":1.601,"15min":1.5029999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":10.638999999999999,"5min":10.638999999999999,"15min":10.638999999999999},"last":1.4339999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":2.1869999999999998,"5min":1.464,"15min":1.3440000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":10.69,"5min":10.69,"15min":10.69},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.264,"15min":1.2210000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":4.9500000000000002,"5min":4.9500000000000002,"15min":4.9500000000000002},"last":1.093}]},{"osd":5,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":2.0630000000000002,"5min":10.819000000000001,"15min":12.278},"min":{"1min":0.73399999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":10.720000000000001,"5min":213.63800000000001,"15min":213.63800000000001},"last":1.004},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.258,"15min":1.212},"min":{"1min":0.82899999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":4.6269999999999998,"5min":4.6269999999999998,"15min":4.6269999999999998},"last":326.01400000000001}]},{"osd":6,"last update":"Fri Aug 7 15:08:24 2020","interfaces":[{"interface":"back","average":{"1min":1.7589999999999999,"5min":1.593,"15min":1.5649999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":4.5259999999999998,"5min":4.5259999999999998,"15min":4.5259999999999998},"last":1.1879999999999999},{"interface":"front","average":{"1min":2.3690000000000002,"5min":1.5620000000000001,"15min":1.4279999999999999},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":10.792,"5min":10.792,"15min":10.792},"last":1.0649999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.8129999999999999,"5min":1.6539999999999999,"15min":1.6279999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":4.5549999999999997,"5min":4.5549999999999997,"15min":4.5549999999999997},"last":1.212},{"interface":"front","average":{"1min":2.2719999999999998,"5min":1.6379999999999999,"15min":1.532},"min":{"1min":0.998,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":10.752000000000001,"5min":10.752000000000001,"15min":10.752000000000001},"last":1.3480000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049756,"kb_used_data":1172,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322084,"statfs":{"total":96636764160,"available":95561814016,"internally_reserved":1073741824,"allocated":1200128,"data_stored":790278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:08:24 2020","interfaces":[{"interface":"back","average":{"1min":81.266000000000005,"5min":17.033999999999999,"15min":6.3289999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":845.57100000000003,"5min":845.57100000000003,"15min":845.57100000000003},"last":845.57100000000003},{"interface":"front","average":{"1min":1.083,"5min":0.96299999999999997,"15min":0.94299999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":5.5970000000000004,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":1.02}]},{"osd":2,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.0069999999999999,"15min":0.95399999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":4.1600000000000001,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":26.393000000000001,"15min":30.623999999999999},"min":{"1min":0.54200000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.1280000000000001,"5min":606.18499999999995,"15min":606.18499999999995},"last":1.056}]},{"osd":3,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.0549999999999999,"15min":0.97499999999999998},"min":{"1min":0.64000000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":3.1779999999999999,"5min":3.1779999999999999,"15min":3.1779999999999999},"last":1.677},{"interface":"front","average":{"1min":1.927,"5min":1.319,"15min":1.218},"min":{"1min":0.78700000000000003,"5min":0.621,"15min":0.621},"max":{"1min":4.21,"5min":4.21,"15min":4.21},"last":1.516}]},{"osd":4,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.238,"15min":1.1899999999999999},"min":{"1min":0.73899999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":3.0870000000000002,"5min":3.0870000000000002,"15min":3.0870000000000002},"last":1.752},{"interface":"front","average":{"1min":1.6890000000000001,"5min":1.3440000000000001,"15min":1.286},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":4.0960000000000001,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":1.5349999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":39.531999999999996,"5min":8.8879999999999999,"15min":3.7799999999999998},"min":{"1min":0.76600000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":501.267,"5min":501.267,"15min":501.267},"last":1.411},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.24,"15min":1.1499999999999999},"min":{"1min":0.874,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":6.0010000000000003,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":1.72}]},{"osd":6,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.7729999999999999,"5min":7.2450000000000001,"15min":8.157},"min":{"1min":0.81100000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":4.1310000000000002,"5min":124.738,"15min":124.738},"last":1.611},{"interface":"front","average":{"1min":1.47,"5min":15.215999999999999,"15min":17.507000000000001},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":3.7120000000000002,"5min":294.94,"15min":294.94},"last":1.4470000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.552,"15min":1.5740000000000001},"min":{"1min":0.96199999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.3929999999999998,"5min":6.468,"15min":6.468},"last":1.49},{"interface":"front","average":{"1min":1.6879999999999999,"5min":1.446,"15min":1.4059999999999999},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":3.117,"5min":3.117,"15min":3.117},"last":1.7849999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049564,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322276,"statfs":{"total":96636764160,"available":95562010624,"internally_reserved":1073741824,"allocated":1003520,"data_stored":593670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.153,"15min":1.129},"min":{"1min":0.505,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.4909999999999997,"5min":4.4909999999999997,"15min":4.4909999999999997},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.284,"5min":1.036,"15min":0.995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":3.6520000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.91000000000000003}]},{"osd":1,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":2.5219999999999998,"5min":1.3779999999999999,"15min":1.1870000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":18.672000000000001,"5min":18.672000000000001,"15min":18.672000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.357,"15min":1.359},"min":{"1min":0.70699999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":5.1130000000000004,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":0.88600000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.216,"15min":1.1799999999999999},"min":{"1min":0.65300000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.419,"5min":1.528,"15min":1.546},"min":{"1min":0.59999999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":5.3339999999999996,"5min":8.2210000000000001,"15min":8.2210000000000001},"last":0.997}]},{"osd":4,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":2.3319999999999999,"5min":1.3540000000000001,"15min":1.1910000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":19.783000000000001,"5min":19.783000000000001,"15min":19.783000000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.5589999999999999,"15min":1.615},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.891,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":1.0780000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.2789999999999999,"15min":1.2669999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.7730000000000001,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.5249999999999999,"5min":20.725000000000001,"15min":23.925000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":4.4790000000000001,"5min":415.40699999999998,"15min":415.40699999999998},"last":1.25}]},{"osd":6,"last update":"Fri Aug 7 15:08:25 2020","interfaces":[{"interface":"back","average":{"1min":2.5089999999999999,"5min":1.478,"15min":1.306},"min":{"1min":0.89000000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":20.035,"5min":20.035,"15min":20.035},"last":1.198},{"interface":"front","average":{"1min":44.033000000000001,"5min":9.8629999999999995,"15min":4.1680000000000001},"min":{"1min":0.93700000000000006,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":751.70000000000005,"5min":751.70000000000005,"15min":751.70000000000005},"last":1.228}]},{"osd":7,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":3.5409999999999999,"5min":2.101,"15min":1.861},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":35.548999999999999,"5min":35.548999999999999,"15min":35.548999999999999},"last":1.282},{"interface":"front","average":{"1min":3.4460000000000002,"5min":1.8859999999999999,"15min":1.6259999999999999},"min":{"1min":1.004,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":35.646999999999998,"5min":35.646999999999998,"15min":35.646999999999998},"last":1.411}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1059164,"kb_used_data":10580,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93312676,"statfs":{"total":96636764160,"available":95552180224,"internally_reserved":1073741824,"allocated":10833920,"data_stored":10424070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.044,"5min":1.411,"15min":1.3049999999999999},"min":{"1min":0.55600000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":8.6020000000000003,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":0.72099999999999997},{"interface":"front","average":{"1min":1.45,"5min":1.599,"15min":1.6240000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":4.9119999999999999,"5min":13.067,"15min":13.067},"last":0.79700000000000004}]},{"osd":1,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":16.382999999999999,"5min":4.7949999999999999,"15min":2.8639999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":270.68700000000001,"5min":270.68700000000001,"15min":270.68700000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":2.7210000000000001,"5min":2.153,"15min":2.0590000000000002},"min":{"1min":0.71599999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":14.349,"5min":14.349,"15min":14.349},"last":0.76600000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":1.7470000000000001,"5min":1.2669999999999999,"15min":1.1870000000000001},"min":{"1min":0.64900000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":4.9580000000000002,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":0.82099999999999995},{"interface":"front","average":{"1min":4.5220000000000002,"5min":2.2519999999999998,"15min":1.8740000000000001},"min":{"1min":0.72399999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":51.530000000000001,"5min":51.530000000000001,"15min":51.530000000000001},"last":0.752}]},{"osd":4,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":47.652999999999999,"5min":10.888,"15min":4.7599999999999998},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":506.21300000000002,"5min":506.21300000000002,"15min":506.21300000000002},"last":1.3029999999999999},{"interface":"front","average":{"1min":29.771000000000001,"5min":31.634,"15min":31.943999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":505.738,"5min":677.88599999999997,"15min":677.88599999999997},"last":1.056}]},{"osd":5,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.012,"5min":1.6160000000000001,"15min":1.55},"min":{"1min":0.96399999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":5.0049999999999999,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.1899999999999999},{"interface":"front","average":{"1min":2.2210000000000001,"5min":1.859,"15min":1.7989999999999999},"min":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":5.226,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":1.21}]},{"osd":6,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":1.99,"5min":1.623,"15min":1.5620000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":5.0389999999999997,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.9710000000000001,"5min":1.9830000000000001,"15min":1.9850000000000001},"min":{"1min":1.0369999999999999,"5min":0.876,"15min":0.876},"max":{"1min":4.266,"5min":5.6189999999999998,"15min":5.6189999999999998},"last":1.173}]},{"osd":7,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.0110000000000001,"5min":2.1309999999999998,"15min":2.1509999999999998},"min":{"1min":0.96999999999999997,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":3.79,"5min":5.734,"15min":5.734},"last":1.071},{"interface":"front","average":{"1min":2.6200000000000001,"5min":2.105,"15min":2.0190000000000001},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":8.6790000000000003,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":1.4419999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049952,"kb_used_data":1368,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321888,"statfs":{"total":96636764160,"available":95561613312,"internally_reserved":1073741824,"allocated":1400832,"data_stored":983461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":1.125,"15min":1.1619999999999999},"min":{"1min":0.47699999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.3999999999999999,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.349,"5min":0.94899999999999995,"15min":0.88200000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":6.3970000000000002,"5min":6.3970000000000002,"15min":6.3970000000000002},"last":0.70499999999999996}]},{"osd":1,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.304,"15min":1.337},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.4380000000000002,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":0.84999999999999998},{"interface":"front","average":{"1min":32.585999999999999,"5min":7.6180000000000003,"15min":3.4569999999999999},"min":{"1min":0.67000000000000004,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":689.846,"5min":689.846,"15min":689.846},"last":0.89600000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.46,"15min":1.454},"min":{"1min":0.56399999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":6.8730000000000002,"5min":6.8730000000000002,"15min":6.8730000000000002},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.5669999999999999,"15min":1.532},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":7.1360000000000001,"5min":7.1360000000000001,"15min":7.1360000000000001},"last":1.181}]},{"osd":3,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":12.619999999999999,"5min":3.73,"15min":2.2480000000000002},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":247.202,"5min":247.202,"15min":247.202},"last":1.107},{"interface":"front","average":{"1min":1.583,"5min":1.3169999999999999,"15min":1.2729999999999999},"min":{"1min":0.67800000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":6.431,"5min":6.431,"15min":6.431},"last":1.0600000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":46.786999999999999,"5min":10.659000000000001,"15min":4.6379999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":990.71600000000001,"5min":990.71600000000001,"15min":990.71600000000001},"last":1.218},{"interface":"front","average":{"1min":1.748,"5min":1.651,"15min":1.635},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":3.9329999999999998,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":1.165}]},{"osd":6,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.7070000000000001,"5min":1.7729999999999999,"15min":1.784},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":6.9749999999999996,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":1.206},{"interface":"front","average":{"1min":1.524,"5min":15.592000000000001,"15min":17.936},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":6.7729999999999997,"5min":389.14600000000002,"15min":389.14600000000002},"last":1.0309999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":13.378,"5min":3.802,"15min":2.206},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":252.798,"5min":252.798,"15min":252.798},"last":1.0820000000000001},{"interface":"front","average":{"1min":44.676000000000002,"5min":10.252000000000001,"15min":4.5140000000000002},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":501.93000000000001,"5min":501.93000000000001,"15min":501.93000000000001},"last":1.246}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049452,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322388,"statfs":{"total":96636764160,"available":95562125312,"internally_reserved":1073741824,"allocated":888832,"data_stored":478982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0580000000000001,"15min":1.0700000000000001},"min":{"1min":0.49199999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.5169999999999999,"5min":2.766,"15min":2.766},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.2250000000000001,"5min":0.88900000000000001,"15min":0.83299999999999996},"min":{"1min":0.54900000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.321}]},{"osd":1,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":47.365000000000002,"5min":10.215999999999999,"15min":4.024},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":787.22900000000004,"5min":787.22900000000004,"15min":787.22900000000004},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.071,"5min":1.1000000000000001,"15min":1.105},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.8380000000000001,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":0.93899999999999995}]},{"osd":2,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":7.5659999999999998,"15min":8.6679999999999993},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.4950000000000001,"5min":168.56899999999999,"15min":168.56899999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.2929999999999999,"5min":26.763000000000002,"15min":31.007999999999999},"min":{"1min":0.79700000000000004,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":3.2389999999999999,"5min":676.27099999999996,"15min":676.27099999999996},"last":1.373}]},{"osd":3,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.196,"15min":1.1439999999999999},"min":{"1min":0.89000000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":3.5649999999999999,"5min":3.5649999999999999,"15min":3.5649999999999999},"last":1.353},{"interface":"front","average":{"1min":43.073,"5min":9.5850000000000009,"15min":4.0030000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":712.98699999999997,"5min":712.98699999999997,"15min":712.98699999999997},"last":1.121}]},{"osd":4,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":40.962000000000003,"15min":47.582000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.4980000000000002,"5min":530.30799999999999,"15min":530.30799999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.278,"15min":1.296},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.6259999999999999,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":1.2130000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":40.569000000000003,"15min":47.128999999999998},"min":{"1min":0.76800000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.6699999999999999,"5min":754.31299999999999,"15min":754.31299999999999},"last":1.099},{"interface":"front","average":{"1min":1.1899999999999999,"5min":21.050999999999998,"15min":24.361000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.7809999999999999,"5min":517.26700000000005,"15min":517.26700000000005},"last":1.1619999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.3560000000000001,"15min":1.3240000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":3.9079999999999999,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.589},{"interface":"front","average":{"1min":1.411,"5min":1.5349999999999999,"15min":1.556},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":3.9390000000000001,"5min":3.9390000000000001,"15min":3.9390000000000001},"last":1.075}]}]}]}} 2020-08-07T15:08:31.716 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:08:31.717 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:08:31.721 INFO:tasks.ceph.ceph_manager.ceph:increase pool size by 10 2020-08-07T15:08:31.721 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set device_health_metrics pg_num 21 2020-08-07T15:08:31.731 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:31.729+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:31.841 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:31.839+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:31.919 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:31.919 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:08:31.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:08:31.920 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:08:31.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.920 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:8112 3.9 e24/20 3:9b275d48:::benchmark_data_smithi074_35447_object1013:head v 24'416, mlcod=24'408)", 2020-08-07T15:08:31.920 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.085042+0000", 2020-08-07T15:08:31.920 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 110.81910222800001, 2020-08-07T15:08:31.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31443637000000002, 2020-08-07T15:08:31.921 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:31.921 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:31.921 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:31.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.922 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:31.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085042+0000", 2020-08-07T15:08:31.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:31.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:31.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085042+0000", 2020-08-07T15:08:31.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4534000000000003e-05 2020-08-07T15:08:31.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:31.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085096+0000", 2020-08-07T15:08:31.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5789999999999998e-06 2020-08-07T15:08:31.925 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:31.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085104+0000", 2020-08-07T15:08:31.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31316244700000001 2020-08-07T15:08:31.926 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.926 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:31.926 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398266+0000", 2020-08-07T15:08:31.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3019999999999996e-06 2020-08-07T15:08:31.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:31.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398270+0000", 2020-08-07T15:08:31.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3467000000000002e-05 2020-08-07T15:08:31.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:31.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398314+0000", 2020-08-07T15:08:31.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8184000000000001e-05 2020-08-07T15:08:31.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:31.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398332+0000", 2020-08-07T15:08:31.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010908059999999999 2020-08-07T15:08:31.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:31.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.399423+0000", 2020-08-07T15:08:31.931 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5050999999999999e-05 2020-08-07T15:08:31.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:31.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.399478+0000", 2020-08-07T15:08:31.932 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31443637000000002 2020-08-07T15:08:31.932 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.932 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:31.932 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.933 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.933 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:8760 3.9 e24/20 3:9f02d263:::benchmark_data_smithi074_35447_object1094:head v 24'448, mlcod=24'441)", 2020-08-07T15:08:31.933 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.535293+0000", 2020-08-07T15:08:31.933 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 110.368851036, 2020-08-07T15:08:31.934 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.194025799, 2020-08-07T15:08:31.934 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:31.934 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:31.934 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:31.934 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:31.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.535293+0000", 2020-08-07T15:08:31.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:31.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:31.936 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.535293+0000", 2020-08-07T15:08:31.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1352e-05 2020-08-07T15:08:31.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:31.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.535334+0000", 2020-08-07T15:08:31.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6660000000000002e-06 2020-08-07T15:08:31.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.938 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:31.938 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.535341+0000", 2020-08-07T15:08:31.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19233371499999999 2020-08-07T15:08:31.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:31.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.727675+0000", 2020-08-07T15:08:31.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1819999999999997e-06 2020-08-07T15:08:31.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.940 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.940 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:31.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.727680+0000", 2020-08-07T15:08:31.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0288999999999999e-05 2020-08-07T15:08:31.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:31.942 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.727730+0000", 2020-08-07T15:08:31.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0340000000000001e-05 2020-08-07T15:08:31.942 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.942 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:31.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.727760+0000", 2020-08-07T15:08:31.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015059489999999999 2020-08-07T15:08:31.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:31.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.729266+0000", 2020-08-07T15:08:31.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2305999999999997e-05 2020-08-07T15:08:31.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.944 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:31.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.729319+0000", 2020-08-07T15:08:31.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.194025799 2020-08-07T15:08:31.945 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.945 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:31.945 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.946 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.946 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:16223 3.9 e27/20 3:9c72cb85:::benchmark_data_smithi074_35447_object2027:head v 27'887, mlcod=27'881)", 2020-08-07T15:08:31.946 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.111666+0000", 2020-08-07T15:08:31.946 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 104.79247737599999, 2020-08-07T15:08:31.946 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.48138790999999997, 2020-08-07T15:08:31.947 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:31.947 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:31.947 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:31.947 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.947 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:31.947 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111666+0000", 2020-08-07T15:08:31.948 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:31.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.948 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:31.948 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111666+0000", 2020-08-07T15:08:31.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3168999999999999e-05 2020-08-07T15:08:31.949 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.949 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:31.949 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111720+0000", 2020-08-07T15:08:31.950 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6309999999999997e-06 2020-08-07T15:08:31.950 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.950 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.950 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:31.950 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111726+0000", 2020-08-07T15:08:31.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.47995092 2020-08-07T15:08:31.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:31.951 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591677+0000", 2020-08-07T15:08:31.952 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9879999999999998e-06 2020-08-07T15:08:31.952 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.953 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.953 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:31.953 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591684+0000", 2020-08-07T15:08:31.953 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5562000000000002e-05 2020-08-07T15:08:31.953 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.954 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.954 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:31.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591730+0000", 2020-08-07T15:08:31.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4731000000000001e-05 2020-08-07T15:08:31.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:31.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591754+0000", 2020-08-07T15:08:31.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012412739999999999 2020-08-07T15:08:31.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:31.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.592996+0000", 2020-08-07T15:08:31.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8634999999999997e-05 2020-08-07T15:08:31.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:31.966 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.593054+0000", 2020-08-07T15:08:31.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.48138790999999997 2020-08-07T15:08:31.967 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.967 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:31.968 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.969 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:16224 3.9 e27/20 3:9c72cb85:::benchmark_data_smithi074_35447_object2027:head v 27'888, mlcod=27'881)", 2020-08-07T15:08:31.969 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.114073+0000", 2020-08-07T15:08:31.969 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 104.790070649, 2020-08-07T15:08:31.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.50478892600000003, 2020-08-07T15:08:31.970 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:31.970 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:31.971 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:31.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:31.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114073+0000", 2020-08-07T15:08:31.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:31.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:31.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114073+0000", 2020-08-07T15:08:31.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3088999999999999e-05 2020-08-07T15:08:31.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.975 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:31.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114126+0000", 2020-08-07T15:08:31.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1629999999999999e-06 2020-08-07T15:08:31.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:31.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114133+0000", 2020-08-07T15:08:31.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.50357425199999994 2020-08-07T15:08:31.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:31.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617708+0000", 2020-08-07T15:08:31.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9160000000000004e-06 2020-08-07T15:08:31.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.979 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:31.979 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617715+0000", 2020-08-07T15:08:31.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8770999999999998e-05 2020-08-07T15:08:31.979 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:31.980 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617763+0000", 2020-08-07T15:08:31.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.544e-05 2020-08-07T15:08:31.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.982 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:31.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617789+0000", 2020-08-07T15:08:31.982 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010210359999999999 2020-08-07T15:08:31.983 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.983 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.983 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:31.984 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.618810+0000", 2020-08-07T15:08:31.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2259000000000002e-05 2020-08-07T15:08:31.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.984 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:31.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.618862+0000", 2020-08-07T15:08:31.985 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.50478892600000003 2020-08-07T15:08:31.985 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.986 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:31.986 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:31.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.986 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:24992 3.0 e28/20 3:008a8fe9:::benchmark_data_smithi074_35447_object3123:head v 28'1704, mlcod=28'1694)", 2020-08-07T15:08:31.987 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.852063+0000", 2020-08-07T15:08:31.987 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 99.052080513999996, 2020-08-07T15:08:31.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20540171199999999, 2020-08-07T15:08:31.987 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:31.987 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:31.987 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:31.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:31.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.852063+0000", 2020-08-07T15:08:31.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:31.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:31.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.852063+0000", 2020-08-07T15:08:31.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1409999999999997e-05 2020-08-07T15:08:31.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.990 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.990 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:31.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.852115+0000", 2020-08-07T15:08:31.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6100000000000002e-06 2020-08-07T15:08:31.990 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:31.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.852121+0000", 2020-08-07T15:08:31.991 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20455925999999999 2020-08-07T15:08:31.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:31.996 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:31.997 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:32.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.056681+0000", 2020-08-07T15:08:32.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8739999999999995e-06 2020-08-07T15:08:32.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:32.004 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.056689+0000", 2020-08-07T15:08:32.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2753999999999998e-05 2020-08-07T15:08:32.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:32.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.056731+0000", 2020-08-07T15:08:32.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9171e-05 2020-08-07T15:08:32.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:32.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.056750+0000", 2020-08-07T15:08:32.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00065542300000000003 2020-08-07T15:08:32.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:32.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.057406+0000", 2020-08-07T15:08:32.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9209999999999997e-05 2020-08-07T15:08:32.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:32.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:53.057465+0000", 2020-08-07T15:08:32.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20540171199999999 2020-08-07T15:08:32.009 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.009 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:32.010 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.011 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:34000 3.4 3:22a8b4e3:::benchmark_data_smithi074_35447_object4249:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e30)", 2020-08-07T15:08:32.011 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:58.316850+0000", 2020-08-07T15:08:32.011 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 93.587293807999998, 2020-08-07T15:08:32.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19523959799999999, 2020-08-07T15:08:32.012 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:32.012 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:32.013 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:32.013 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:32.013 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:32.014 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 34000 2020-08-07T15:08:32.014 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.014 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:32.015 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.015 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:32.015 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316850+0000", 2020-08-07T15:08:32.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:32.016 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:32.017 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316850+0000", 2020-08-07T15:08:32.017 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7274999999999997e-05 2020-08-07T15:08:32.018 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.018 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.018 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:32.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316897+0000", 2020-08-07T15:08:32.019 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2249999999999999e-06 2020-08-07T15:08:32.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.020 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.020 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:32.020 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316903+0000", 2020-08-07T15:08:32.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0665e-05 2020-08-07T15:08:32.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:32.022 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316913+0000", 2020-08-07T15:08:32.022 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3089999999999997e-06 2020-08-07T15:08:32.023 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:32.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.316918+0000", 2020-08-07T15:08:32.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041515279999999998 2020-08-07T15:08:32.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:32.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.321069+0000", 2020-08-07T15:08:32.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6084999999999999e-05 2020-08-07T15:08:32.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:32.033 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.321165+0000", 2020-08-07T15:08:32.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000185076 2020-08-07T15:08:32.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.034 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.034 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:08:32.034 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.321350+0000", 2020-08-07T15:08:32.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00079810799999999996 2020-08-07T15:08:32.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:32.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.322148+0000", 2020-08-07T15:08:32.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9299999999999997e-07 2020-08-07T15:08:32.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:32.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.322149+0000", 2020-08-07T15:08:32.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1654000000000003e-05 2020-08-07T15:08:32.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.037 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:32.037 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.322211+0000", 2020-08-07T15:08:32.038 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044003000000000002 2020-08-07T15:08:32.038 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.038 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.039 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:32.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.326611+0000", 2020-08-07T15:08:32.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18541316599999999 2020-08-07T15:08:32.039 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.039 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:32.040 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.512024+0000", 2020-08-07T15:08:32.040 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5290000000000001e-06 2020-08-07T15:08:32.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.041 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:32.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.512026+0000", 2020-08-07T15:08:32.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3985000000000005e-05 2020-08-07T15:08:32.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:32.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.512090+0000", 2020-08-07T15:08:32.042 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19523959799999999 2020-08-07T15:08:32.042 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.043 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:32.043 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.044 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:34624 3.9 e30/20 3:946cdb13:::benchmark_data_smithi074_35447_object4327:head v 30'2016, mlcod=30'2009)", 2020-08-07T15:08:32.045 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:58.746375+0000", 2020-08-07T15:08:32.045 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 93.157769005000006, 2020-08-07T15:08:32.045 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28564679799999998, 2020-08-07T15:08:32.046 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:32.046 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:32.046 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:32.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.047 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:32.047 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746375+0000", 2020-08-07T15:08:32.048 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:32.048 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.054 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:32.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746375+0000", 2020-08-07T15:08:32.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3674000000000001e-05 2020-08-07T15:08:32.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:32.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746429+0000", 2020-08-07T15:08:32.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9349999999999997e-06 2020-08-07T15:08:32.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.067 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:32.067 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746435+0000", 2020-08-07T15:08:32.067 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28165817700000001 2020-08-07T15:08:32.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.068 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.068 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:32.069 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028094+0000", 2020-08-07T15:08:32.069 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.512e-06 2020-08-07T15:08:32.069 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.070 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.070 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:32.070 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028097+0000", 2020-08-07T15:08:32.070 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7904999999999997e-05 2020-08-07T15:08:32.071 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.071 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.071 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:32.072 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028135+0000", 2020-08-07T15:08:32.072 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6243e-05 2020-08-07T15:08:32.072 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.073 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.073 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:32.073 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028151+0000", 2020-08-07T15:08:32.074 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003818372 2020-08-07T15:08:32.074 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.074 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.074 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:32.075 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.031970+0000", 2020-08-07T15:08:32.075 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.198e-05 2020-08-07T15:08:32.075 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.076 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.076 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:32.076 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.032022+0000", 2020-08-07T15:08:32.077 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28564679799999998 2020-08-07T15:08:32.077 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.077 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:32.087 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.088 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:64688 3.4 3:221784a5:::benchmark_data_smithi074_35447_object8085:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e41)", 2020-08-07T15:08:32.089 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:20.149588+0000", 2020-08-07T15:08:32.089 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 71.754555374999995, 2020-08-07T15:08:32.090 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20387735500000001, 2020-08-07T15:08:32.090 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:32.090 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:32.091 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:32.091 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:32.091 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:32.092 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 64688 2020-08-07T15:08:32.092 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.092 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:32.093 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.093 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:32.094 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.149588+0000", 2020-08-07T15:08:32.094 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:32.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.095 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.095 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:32.096 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.149588+0000", 2020-08-07T15:08:32.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0269999999999998e-05 2020-08-07T15:08:32.096 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.097 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.097 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:32.098 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.149639+0000", 2020-08-07T15:08:32.098 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1489999999999999e-06 2020-08-07T15:08:32.098 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.099 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.099 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:32.100 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.149645+0000", 2020-08-07T15:08:32.100 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1691999999999999e-05 2020-08-07T15:08:32.100 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.101 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.101 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:32.101 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.149657+0000", 2020-08-07T15:08:32.102 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0409e-05 2020-08-07T15:08:32.113 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.113 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:32.114 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.149667+0000", 2020-08-07T15:08:32.114 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00061991800000000001 2020-08-07T15:08:32.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.115 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:32.115 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.150287+0000", 2020-08-07T15:08:32.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8415e-05 2020-08-07T15:08:32.115 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.116 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.116 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:32.116 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.150375+0000", 2020-08-07T15:08:32.116 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000180729 2020-08-07T15:08:32.117 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.117 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:08:32.117 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.150556+0000", 2020-08-07T15:08:32.118 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0052814489999999997 2020-08-07T15:08:32.118 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.118 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.118 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:32.119 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.155838+0000", 2020-08-07T15:08:32.119 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6000000000000003e-07 2020-08-07T15:08:32.119 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.119 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.120 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:32.120 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.155838+0000", 2020-08-07T15:08:32.120 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5048999999999993e-05 2020-08-07T15:08:32.120 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.121 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.121 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:32.121 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.155903+0000", 2020-08-07T15:08:32.122 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18448816500000001 2020-08-07T15:08:32.122 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.122 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.123 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:32.123 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.340391+0000", 2020-08-07T15:08:32.123 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.013000151 2020-08-07T15:08:32.123 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.124 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.124 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:32.124 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.353392+0000", 2020-08-07T15:08:32.125 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0580000000000001e-06 2020-08-07T15:08:32.125 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.125 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.126 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:32.126 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.353393+0000", 2020-08-07T15:08:32.126 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3140999999999996e-05 2020-08-07T15:08:32.126 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.127 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.134 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:32.134 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:20.353466+0000", 2020-08-07T15:08:32.135 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20387735500000001 2020-08-07T15:08:32.135 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.135 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:32.136 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.136 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.136 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.137 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:78016 3.9 e46/20 3:921b23e9:::benchmark_data_smithi074_35447_object9751:head v 46'4512, mlcod=46'4506)", 2020-08-07T15:08:32.137 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:28.734571+0000", 2020-08-07T15:08:32.137 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 63.169572602999999, 2020-08-07T15:08:32.137 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23216004300000001, 2020-08-07T15:08:32.143 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:32.143 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:32.143 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:32.144 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.144 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:32.144 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.734571+0000", 2020-08-07T15:08:32.144 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:32.145 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.145 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.145 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:32.145 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.734571+0000", 2020-08-07T15:08:32.145 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7376000000000003e-05 2020-08-07T15:08:32.146 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.146 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.146 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:32.147 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.734609+0000", 2020-08-07T15:08:32.147 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3640000000000004e-06 2020-08-07T15:08:32.147 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.148 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.150 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:32.150 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.734615+0000", 2020-08-07T15:08:32.150 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.228692642 2020-08-07T15:08:32.150 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.151 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:32.151 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.963308+0000", 2020-08-07T15:08:32.151 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9639999999999997e-06 2020-08-07T15:08:32.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:32.152 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.963312+0000", 2020-08-07T15:08:32.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5215000000000001e-05 2020-08-07T15:08:32.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.153 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.153 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:32.153 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.963347+0000", 2020-08-07T15:08:32.153 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8610999999999999e-05 2020-08-07T15:08:32.153 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.154 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.154 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:32.154 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.963365+0000", 2020-08-07T15:08:32.155 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033060149999999998 2020-08-07T15:08:32.155 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.155 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.155 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:32.155 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.966671+0000", 2020-08-07T15:08:32.156 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9855999999999997e-05 2020-08-07T15:08:32.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.156 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:32.156 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.966731+0000", 2020-08-07T15:08:32.157 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23216004300000001 2020-08-07T15:08:32.157 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.157 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:32.157 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.157 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.158 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.158 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:96015 3.4 3:2bc10706:::benchmark_data_smithi074_35447_object12001:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e56)", 2020-08-07T15:08:32.160 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:40.755925+0000", 2020-08-07T15:08:32.160 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 51.14821869, 2020-08-07T15:08:32.160 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20304256600000001, 2020-08-07T15:08:32.160 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:32.161 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:32.161 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:32.161 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:32.161 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:32.162 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 96015 2020-08-07T15:08:32.162 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.162 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:32.162 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.162 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:32.162 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.755925+0000", 2020-08-07T15:08:32.163 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:32.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.163 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.163 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:32.163 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.755925+0000", 2020-08-07T15:08:32.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0268000000000001e-05 2020-08-07T15:08:32.164 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.165 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:32.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.755965+0000", 2020-08-07T15:08:32.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5780000000000004e-06 2020-08-07T15:08:32.165 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.166 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.166 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:32.166 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.755972+0000", 2020-08-07T15:08:32.166 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1082e-05 2020-08-07T15:08:32.166 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.167 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.167 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:32.167 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.755983+0000", 2020-08-07T15:08:32.167 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3429999999999998e-06 2020-08-07T15:08:32.167 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.183 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:32.184 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.755987+0000", 2020-08-07T15:08:32.184 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034211799999999998 2020-08-07T15:08:32.184 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.184 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.184 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:32.185 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.759409+0000", 2020-08-07T15:08:32.185 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1285000000000004e-05 2020-08-07T15:08:32.185 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.185 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.185 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:32.186 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.759500+0000", 2020-08-07T15:08:32.186 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000155134 2020-08-07T15:08:32.186 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.186 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.187 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:08:32.187 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.759655+0000", 2020-08-07T15:08:32.187 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0031454679999999998 2020-08-07T15:08:32.187 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.187 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.188 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:32.188 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.762801+0000", 2020-08-07T15:08:32.189 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7299999999999996e-07 2020-08-07T15:08:32.189 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.189 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.189 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:32.189 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.762801+0000", 2020-08-07T15:08:32.190 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9682999999999998e-05 2020-08-07T15:08:32.190 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.190 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.190 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:32.191 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.762861+0000", 2020-08-07T15:08:32.192 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.10233099900000001 2020-08-07T15:08:32.192 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.192 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.192 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:32.192 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.865192+0000", 2020-08-07T15:08:32.193 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.093708455999999996 2020-08-07T15:08:32.193 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.193 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.193 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:32.193 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.958900+0000", 2020-08-07T15:08:32.194 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3600000000000002e-07 2020-08-07T15:08:32.194 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.194 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.194 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:32.194 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.958901+0000", 2020-08-07T15:08:32.195 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6581000000000005e-05 2020-08-07T15:08:32.195 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.195 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.195 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:32.195 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.958968+0000", 2020-08-07T15:08:32.195 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20304256600000001 2020-08-07T15:08:32.196 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.196 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:32.196 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.196 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.196 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.197 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:96016 3.4 3:2bc10706:::benchmark_data_smithi074_35447_object12001:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e56)", 2020-08-07T15:08:32.197 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:40.757829+0000", 2020-08-07T15:08:32.197 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 51.146315117, 2020-08-07T15:08:32.197 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.201245428, 2020-08-07T15:08:32.197 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:32.198 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:32.198 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:32.198 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:32.198 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:32.198 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 96016 2020-08-07T15:08:32.199 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.199 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:32.199 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.199 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:32.199 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.757829+0000", 2020-08-07T15:08:32.200 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:32.200 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.200 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.200 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:32.200 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.757829+0000", 2020-08-07T15:08:32.200 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00035768300000000002 2020-08-07T15:08:32.201 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.201 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.201 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:32.201 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.758186+0000", 2020-08-07T15:08:32.202 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2670000000000003e-06 2020-08-07T15:08:32.202 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.202 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.203 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:32.203 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.758193+0000", 2020-08-07T15:08:32.203 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3022e-05 2020-08-07T15:08:32.203 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.203 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.203 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:32.204 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.758206+0000", 2020-08-07T15:08:32.204 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9680000000000003e-06 2020-08-07T15:08:32.204 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.204 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.204 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:32.205 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.758212+0000", 2020-08-07T15:08:32.205 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038084989999999999 2020-08-07T15:08:32.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.205 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.205 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:32.206 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.762020+0000", 2020-08-07T15:08:32.206 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7879000000000003e-05 2020-08-07T15:08:32.206 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.206 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.206 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:32.207 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.762118+0000", 2020-08-07T15:08:32.207 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017300900000000001 2020-08-07T15:08:32.207 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.207 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.207 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:08:32.208 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.762291+0000", 2020-08-07T15:08:32.208 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026504240000000002 2020-08-07T15:08:32.208 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.208 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.208 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:32.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.764941+0000", 2020-08-07T15:08:32.209 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8700000000000004e-07 2020-08-07T15:08:32.209 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.209 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.209 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:32.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.764942+0000", 2020-08-07T15:08:32.210 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3740999999999998e-05 2020-08-07T15:08:32.210 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.210 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.210 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:32.210 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.765006+0000", 2020-08-07T15:08:32.211 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.109174536 2020-08-07T15:08:32.211 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.211 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.211 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:32.211 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.874181+0000", 2020-08-07T15:08:32.212 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.084829826999999997 2020-08-07T15:08:32.213 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.213 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.213 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:32.214 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.959010+0000", 2020-08-07T15:08:32.214 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9800000000000004e-07 2020-08-07T15:08:32.214 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.215 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.215 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:32.215 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.959011+0000", 2020-08-07T15:08:32.216 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3188000000000006e-05 2020-08-07T15:08:32.216 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.216 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.217 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:32.217 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:40.959074+0000", 2020-08-07T15:08:32.217 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.201245428 2020-08-07T15:08:32.218 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.218 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:32.218 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.219 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.219 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.219 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:104111 3.9 e58/20 3:99770d45:::benchmark_data_smithi074_35447_object13013:head v 58'6207, mlcod=58'6197)", 2020-08-07T15:08:32.221 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:45.861256+0000", 2020-08-07T15:08:32.221 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 46.042887788999998, 2020-08-07T15:08:32.222 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29017704, 2020-08-07T15:08:32.223 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:32.223 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:32.224 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:32.225 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.225 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:32.225 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861256+0000", 2020-08-07T15:08:32.225 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:32.225 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.226 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.226 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:32.226 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861256+0000", 2020-08-07T15:08:32.226 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013044700000000001 2020-08-07T15:08:32.226 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.227 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.227 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:32.227 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861387+0000", 2020-08-07T15:08:32.227 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7533000000000001e-05 2020-08-07T15:08:32.227 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.228 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.228 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:32.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861424+0000", 2020-08-07T15:08:32.228 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28788376900000001 2020-08-07T15:08:32.228 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.229 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.229 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:32.229 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.149308+0000", 2020-08-07T15:08:32.229 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1890000000000004e-06 2020-08-07T15:08:32.229 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.230 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.230 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:32.230 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.149314+0000", 2020-08-07T15:08:32.230 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8288999999999999e-05 2020-08-07T15:08:32.230 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.230 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.234 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:32.234 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.149352+0000", 2020-08-07T15:08:32.234 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6765999999999998e-05 2020-08-07T15:08:32.234 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.235 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.236 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:32.236 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.149369+0000", 2020-08-07T15:08:32.237 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0020281000000000001 2020-08-07T15:08:32.237 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.237 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.237 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:32.237 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.151397+0000", 2020-08-07T15:08:32.238 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5947e-05 2020-08-07T15:08:32.238 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.238 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.238 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:32.238 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.151433+0000", 2020-08-07T15:08:32.239 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29017704 2020-08-07T15:08:32.239 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.239 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:32.239 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.239 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.240 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.240 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:104112 3.9 e58/20 3:99770d45:::benchmark_data_smithi074_35447_object13013:head v 58'6208, mlcod=58'6198)", 2020-08-07T15:08:32.240 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:45.862801+0000", 2020-08-07T15:08:32.240 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 46.041342606999997, 2020-08-07T15:08:32.240 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29018473099999997, 2020-08-07T15:08:32.241 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:32.241 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:32.241 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:32.241 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.241 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:32.242 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.862801+0000", 2020-08-07T15:08:32.242 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:32.242 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.242 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.242 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:32.243 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.862801+0000", 2020-08-07T15:08:32.243 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0234e-05 2020-08-07T15:08:32.243 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.243 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.244 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:32.244 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.862851+0000", 2020-08-07T15:08:32.244 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2700000000000001e-06 2020-08-07T15:08:32.244 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.244 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.245 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:32.245 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.862858+0000", 2020-08-07T15:08:32.245 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28821313100000001 2020-08-07T15:08:32.245 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.245 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.245 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:32.246 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.151071+0000", 2020-08-07T15:08:32.246 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4140000000000002e-06 2020-08-07T15:08:32.247 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.247 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.247 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:32.247 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.151076+0000", 2020-08-07T15:08:32.247 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7945999999999999e-05 2020-08-07T15:08:32.248 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.248 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.248 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:32.248 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.151114+0000", 2020-08-07T15:08:32.248 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2900000000000001e-05 2020-08-07T15:08:32.249 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.249 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.251 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:32.251 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.151137+0000", 2020-08-07T15:08:32.251 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001801415 2020-08-07T15:08:32.251 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.251 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.252 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:32.252 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.152939+0000", 2020-08-07T15:08:32.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7420999999999999e-05 2020-08-07T15:08:32.252 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.253 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:32.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.152986+0000", 2020-08-07T15:08:32.253 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29018473099999997 2020-08-07T15:08:32.253 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.254 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:32.254 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.254 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.254 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.254 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106080 3.4 3:208b1d5b:::benchmark_data_smithi074_35447_object13259:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:08:32.255 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.241181+0000", 2020-08-07T15:08:32.255 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 44.662963159, 2020-08-07T15:08:32.255 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45274565500000002, 2020-08-07T15:08:32.255 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:32.255 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:32.256 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:32.256 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:32.256 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:32.256 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106080 2020-08-07T15:08:32.256 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.257 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:32.257 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.258 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:32.258 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.241181+0000", 2020-08-07T15:08:32.258 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:32.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.259 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.259 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:32.260 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.241181+0000", 2020-08-07T15:08:32.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3634999999999998e-05 2020-08-07T15:08:32.260 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.261 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.261 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:32.261 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.241224+0000", 2020-08-07T15:08:32.261 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2310000000000001e-06 2020-08-07T15:08:32.262 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.263 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.264 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:32.264 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.241228+0000", 2020-08-07T15:08:32.264 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7490000000000001e-06 2020-08-07T15:08:32.265 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.265 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.265 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:32.266 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.241234+0000", 2020-08-07T15:08:32.266 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1320000000000004e-06 2020-08-07T15:08:32.266 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.267 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.267 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:32.268 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.241238+0000", 2020-08-07T15:08:32.268 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001563659 2020-08-07T15:08:32.269 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.269 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.269 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:32.270 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.242802+0000", 2020-08-07T15:08:32.270 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0303999999999995e-05 2020-08-07T15:08:32.270 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.271 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:32.271 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.242892+0000", 2020-08-07T15:08:32.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013412 2020-08-07T15:08:32.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:08:32.273 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.243027+0000", 2020-08-07T15:08:32.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001917291 2020-08-07T15:08:32.274 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.274 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.274 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:32.275 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.244944+0000", 2020-08-07T15:08:32.275 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9647999999999995e-05 2020-08-07T15:08:32.275 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.275 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.277 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:32.277 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.245013+0000", 2020-08-07T15:08:32.279 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.44882515299999998 2020-08-07T15:08:32.279 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.280 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.280 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:32.280 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693839+0000", 2020-08-07T15:08:32.281 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5290000000000001e-06 2020-08-07T15:08:32.281 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.281 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.282 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:32.282 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693840+0000", 2020-08-07T15:08:32.282 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7976999999999998e-05 2020-08-07T15:08:32.283 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.283 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.283 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:32.284 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693888+0000", 2020-08-07T15:08:32.284 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8226999999999998e-05 2020-08-07T15:08:32.284 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.285 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.285 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:32.285 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693926+0000", 2020-08-07T15:08:32.286 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45274565500000002 2020-08-07T15:08:32.286 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.286 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:32.287 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.290 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106080 3.4 e58/20 ondisk, result = 0)", 2020-08-07T15:08:32.291 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.255977+0000", 2020-08-07T15:08:32.291 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 44.648166537999998, 2020-08-07T15:08:32.292 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43795860599999997, 2020-08-07T15:08:32.292 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:32.293 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:32.295 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:32.295 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.296 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:32.296 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255977+0000", 2020-08-07T15:08:32.296 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:32.296 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.297 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.297 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:32.297 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255977+0000", 2020-08-07T15:08:32.298 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2820000000000001e-05 2020-08-07T15:08:32.298 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.298 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.299 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:32.299 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.256010+0000", 2020-08-07T15:08:32.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8660000000000002e-06 2020-08-07T15:08:32.300 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.300 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.301 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:32.302 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.256016+0000", 2020-08-07T15:08:32.302 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.437661209 2020-08-07T15:08:32.302 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.303 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.303 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:32.303 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693677+0000", 2020-08-07T15:08:32.303 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6000000000000003e-06 2020-08-07T15:08:32.304 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.304 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.304 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:32.324 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693684+0000", 2020-08-07T15:08:32.325 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000138572 2020-08-07T15:08:32.325 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.325 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.325 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:32.326 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693822+0000", 2020-08-07T15:08:32.326 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6339999999999996e-06 2020-08-07T15:08:32.327 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.327 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.327 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:32.330 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693832+0000", 2020-08-07T15:08:32.330 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000103905 2020-08-07T15:08:32.331 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.331 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.331 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:32.331 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693936+0000", 2020-08-07T15:08:32.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43795860599999997 2020-08-07T15:08:32.332 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.332 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:32.332 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.332 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.332 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.333 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:116623 3.9 e65/20 3:93e94c7d:::benchmark_data_smithi074_35447_object14577:head v 65'7111, mlcod=65'7102)", 2020-08-07T15:08:32.333 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:56.780860+0000", 2020-08-07T15:08:32.333 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 35.123283389000001, 2020-08-07T15:08:32.333 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22065456899999999, 2020-08-07T15:08:32.333 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:32.334 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:32.334 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:32.334 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.334 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:32.334 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.780860+0000", 2020-08-07T15:08:32.335 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:32.335 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.335 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:32.335 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.780860+0000", 2020-08-07T15:08:32.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2389000000000003e-05 2020-08-07T15:08:32.336 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.336 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.336 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:32.336 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.780913+0000", 2020-08-07T15:08:32.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7039999999999997e-06 2020-08-07T15:08:32.337 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.337 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.337 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:32.337 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.780920+0000", 2020-08-07T15:08:32.337 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.219753378 2020-08-07T15:08:32.338 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.338 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.338 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:32.338 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.000673+0000", 2020-08-07T15:08:32.338 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8869999999999997e-06 2020-08-07T15:08:32.339 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.339 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.339 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:32.339 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.000679+0000", 2020-08-07T15:08:32.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0431000000000004e-05 2020-08-07T15:08:32.339 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.340 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:32.354 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.000749+0000", 2020-08-07T15:08:32.355 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8459e-05 2020-08-07T15:08:32.355 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.355 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.355 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:32.356 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.000778+0000", 2020-08-07T15:08:32.356 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00068167900000000003 2020-08-07T15:08:32.356 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.356 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.356 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:32.356 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.001459+0000", 2020-08-07T15:08:32.357 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5642e-05 2020-08-07T15:08:32.357 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.357 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.357 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:32.357 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.001515+0000", 2020-08-07T15:08:32.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22065456899999999 2020-08-07T15:08:32.363 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.363 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:32.363 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.364 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.364 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:116624 3.9 e65/20 3:93e94c7d:::benchmark_data_smithi074_35447_object14577:head v 65'7112, mlcod=65'7103)", 2020-08-07T15:08:32.364 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:56.781405+0000", 2020-08-07T15:08:32.364 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 35.122738730000002, 2020-08-07T15:08:32.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.248173752, 2020-08-07T15:08:32.365 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:32.365 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:32.365 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:32.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.366 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:32.366 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.781405+0000", 2020-08-07T15:08:32.366 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:32.367 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.367 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.367 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:32.367 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.781405+0000", 2020-08-07T15:08:32.367 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3173999999999998e-05 2020-08-07T15:08:32.368 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.368 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.368 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:32.368 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.781438+0000", 2020-08-07T15:08:32.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1790000000000001e-06 2020-08-07T15:08:32.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.370 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:32.370 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.781442+0000", 2020-08-07T15:08:32.370 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24601534999999999 2020-08-07T15:08:32.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.382 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.382 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:32.383 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.027458+0000", 2020-08-07T15:08:32.383 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8599999999999998e-06 2020-08-07T15:08:32.383 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.383 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.383 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:32.384 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.027464+0000", 2020-08-07T15:08:32.384 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3399000000000003e-05 2020-08-07T15:08:32.384 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.384 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.384 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:32.385 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.027497+0000", 2020-08-07T15:08:32.385 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6073999999999999e-05 2020-08-07T15:08:32.385 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.385 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.386 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:32.386 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.027513+0000", 2020-08-07T15:08:32.386 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0020137979999999998 2020-08-07T15:08:32.386 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.386 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.387 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:32.387 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.029527+0000", 2020-08-07T15:08:32.387 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1918e-05 2020-08-07T15:08:32.387 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.387 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.388 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:32.388 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.029579+0000", 2020-08-07T15:08:32.388 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.248173752 2020-08-07T15:08:32.388 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.388 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:32.389 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.389 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.389 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.389 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:117016 3.4 3:2f619d09:::benchmark_data_smithi074_35447_object14626:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e65)", 2020-08-07T15:08:32.389 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:57.118050+0000", 2020-08-07T15:08:32.393 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 34.78609367, 2020-08-07T15:08:32.393 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.255829902, 2020-08-07T15:08:32.393 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:32.393 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:32.394 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:32.394 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:32.394 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:32.394 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 117016 2020-08-07T15:08:32.394 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.395 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:32.395 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.395 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:32.395 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118050+0000", 2020-08-07T15:08:32.395 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:32.396 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.396 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.396 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:32.396 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118050+0000", 2020-08-07T15:08:32.396 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2957999999999999e-05 2020-08-07T15:08:32.397 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.397 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.398 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:32.398 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118083+0000", 2020-08-07T15:08:32.398 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8829999999999997e-06 2020-08-07T15:08:32.399 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.399 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:32.399 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118088+0000", 2020-08-07T15:08:32.399 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4660000000000005e-06 2020-08-07T15:08:32.400 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.400 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:32.400 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118098+0000", 2020-08-07T15:08:32.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.768e-06 2020-08-07T15:08:32.401 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.401 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.401 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:32.401 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118100+0000", 2020-08-07T15:08:32.401 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00054986900000000003 2020-08-07T15:08:32.402 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.402 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.402 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:32.403 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118650+0000", 2020-08-07T15:08:32.403 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7521e-05 2020-08-07T15:08:32.403 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.404 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.404 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:32.404 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118708+0000", 2020-08-07T15:08:32.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012445499999999999 2020-08-07T15:08:32.404 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.405 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:08:32.405 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118832+0000", 2020-08-07T15:08:32.405 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002948806 2020-08-07T15:08:32.406 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.406 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.406 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:32.406 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.121781+0000", 2020-08-07T15:08:32.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8299999999999996e-07 2020-08-07T15:08:32.407 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.407 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.408 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:32.408 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.121782+0000", 2020-08-07T15:08:32.408 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4300000000000004e-05 2020-08-07T15:08:32.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.409 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.409 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:32.409 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.121846+0000", 2020-08-07T15:08:32.409 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.044326363000000001 2020-08-07T15:08:32.410 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.410 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.410 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:32.410 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.166172+0000", 2020-08-07T15:08:32.410 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20763515900000001 2020-08-07T15:08:32.411 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.411 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.411 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:32.411 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.373808+0000", 2020-08-07T15:08:32.411 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6339999999999999e-06 2020-08-07T15:08:32.412 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.412 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.412 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:32.412 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.373809+0000", 2020-08-07T15:08:32.412 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0937000000000002e-05 2020-08-07T15:08:32.412 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.414 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:32.414 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.373880+0000", 2020-08-07T15:08:32.414 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.255829902 2020-08-07T15:08:32.414 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.415 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:32.415 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.416 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:125528 3.4 3:2c06ab2d:::benchmark_data_smithi074_35447_object15690:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e69)", 2020-08-07T15:08:32.416 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:03.323266+0000", 2020-08-07T15:08:32.416 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 28.580877774000001, 2020-08-07T15:08:32.416 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21366406900000001, 2020-08-07T15:08:32.416 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:32.417 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:32.417 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:32.417 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:32.417 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:32.417 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 125528 2020-08-07T15:08:32.418 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.418 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:32.418 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.418 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:32.420 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.323266+0000", 2020-08-07T15:08:32.420 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:32.420 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.420 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.420 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:32.421 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.323266+0000", 2020-08-07T15:08:32.421 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1621000000000001e-05 2020-08-07T15:08:32.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.421 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.421 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:32.422 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.323298+0000", 2020-08-07T15:08:32.422 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1820000000000002e-06 2020-08-07T15:08:32.422 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.422 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:32.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.323301+0000", 2020-08-07T15:08:32.423 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4389999999999997e-06 2020-08-07T15:08:32.423 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.423 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.424 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:32.424 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.323307+0000", 2020-08-07T15:08:32.424 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9500000000000001e-06 2020-08-07T15:08:32.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.425 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:32.425 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.323310+0000", 2020-08-07T15:08:32.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010126709999999999 2020-08-07T15:08:32.426 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.426 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.426 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:32.426 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.324323+0000", 2020-08-07T15:08:32.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010536500000000001 2020-08-07T15:08:32.427 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.427 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:32.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.324428+0000", 2020-08-07T15:08:32.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017499900000000001 2020-08-07T15:08:32.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:08:32.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.324603+0000", 2020-08-07T15:08:32.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023473299999999999 2020-08-07T15:08:32.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.429 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:32.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.326951+0000", 2020-08-07T15:08:32.430 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.23e-07 2020-08-07T15:08:32.430 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.430 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.430 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:32.430 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.326951+0000", 2020-08-07T15:08:32.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3559999999999997e-05 2020-08-07T15:08:32.431 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.431 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:32.431 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.326985+0000", 2020-08-07T15:08:32.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00041237000000000001 2020-08-07T15:08:32.432 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.432 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.432 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:32.432 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.327397+0000", 2020-08-07T15:08:32.432 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20945345100000001 2020-08-07T15:08:32.433 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.433 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.433 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:32.433 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.536851+0000", 2020-08-07T15:08:32.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.621e-06 2020-08-07T15:08:32.434 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.434 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.435 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:32.435 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.536852+0000", 2020-08-07T15:08:32.435 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7787000000000006e-05 2020-08-07T15:08:32.435 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.435 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.436 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:32.436 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:03.536930+0000", 2020-08-07T15:08:32.436 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21366406900000001 2020-08-07T15:08:32.436 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.436 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:32.437 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.437 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.437 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.437 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:7304 4.4 4:2da9e9a1:::benchmark_data_smithi074_44586_object912:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e85)", 2020-08-07T15:08:32.437 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:31.182209+0000", 2020-08-07T15:08:32.438 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.72193513899999995, 2020-08-07T15:08:32.438 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23183255, 2020-08-07T15:08:32.438 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:32.438 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:32.447 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:32.447 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:08:32.447 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:32.447 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7304 2020-08-07T15:08:32.448 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.448 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:32.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:32.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.182209+0000", 2020-08-07T15:08:32.449 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:32.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.449 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.449 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:32.449 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.182209+0000", 2020-08-07T15:08:32.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7351999999999998e-05 2020-08-07T15:08:32.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:32.456 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.182236+0000", 2020-08-07T15:08:32.456 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.512e-06 2020-08-07T15:08:32.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:32.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.182240+0000", 2020-08-07T15:08:32.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1302e-05 2020-08-07T15:08:32.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.458 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.466 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:32.466 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.182251+0000", 2020-08-07T15:08:32.466 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1370000000000004e-06 2020-08-07T15:08:32.467 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.467 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.467 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:32.467 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.182255+0000", 2020-08-07T15:08:32.467 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024353510000000001 2020-08-07T15:08:32.468 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.468 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.468 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:32.468 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.184690+0000", 2020-08-07T15:08:32.469 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5316999999999996e-05 2020-08-07T15:08:32.469 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.469 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.469 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:32.469 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.184786+0000", 2020-08-07T15:08:32.470 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018412300000000001 2020-08-07T15:08:32.470 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.470 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.470 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:08:32.471 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.184970+0000", 2020-08-07T15:08:32.471 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0025507059999999998 2020-08-07T15:08:32.471 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.471 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.472 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:32.472 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.187521+0000", 2020-08-07T15:08:32.472 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.96e-07 2020-08-07T15:08:32.472 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.472 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.473 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:32.473 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.187521+0000", 2020-08-07T15:08:32.473 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2775000000000002e-05 2020-08-07T15:08:32.473 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.474 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.474 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:32.474 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.187574+0000", 2020-08-07T15:08:32.474 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.040609377000000002 2020-08-07T15:08:32.474 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.475 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.475 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:32.475 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.228183+0000", 2020-08-07T15:08:32.475 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.185779313 2020-08-07T15:08:32.475 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.476 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.476 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:32.476 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.413962+0000", 2020-08-07T15:08:32.477 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0189999999999999e-06 2020-08-07T15:08:32.477 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.477 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.477 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:32.477 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.413963+0000", 2020-08-07T15:08:32.478 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7869999999999998e-05 2020-08-07T15:08:32.478 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.478 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.478 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:32.478 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.414041+0000", 2020-08-07T15:08:32.479 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23183255 2020-08-07T15:08:32.479 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.479 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:32.479 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.479 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.480 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:32.480 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:32.482 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 6 16 9081 9065 11.8025 12.5391 0.00621761 0.0105521 2020-08-07T15:08:32.482 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:32.479+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:32.483 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:08:32.590 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:32.580+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:32.625 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:08:32.685 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:32.681+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:32.702 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:32.702 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:08:32.702 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.702 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:9568 4.f e86/81 4:fb618276:::benchmark_data_smithi074_44586_object1195:head v 86'640, mlcod=86'631)", 2020-08-07T15:08:32.703 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:32.565147+0000", 2020-08-07T15:08:32.703 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.13229342899999999, 2020-08-07T15:08:32.703 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.132317605, 2020-08-07T15:08:32.703 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:32.703 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:32.704 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:32.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.704 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:32.704 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.565147+0000", 2020-08-07T15:08:32.704 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:32.705 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.705 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.705 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:32.705 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.565147+0000", 2020-08-07T15:08:32.705 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5937999999999999e-05 2020-08-07T15:08:32.707 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.707 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:32.707 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.565203+0000", 2020-08-07T15:08:32.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.297e-06 2020-08-07T15:08:32.708 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.708 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:32.708 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.565209+0000", 2020-08-07T15:08:32.709 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13046351 2020-08-07T15:08:32.709 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.709 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.709 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:32.709 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.695673+0000", 2020-08-07T15:08:32.710 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8789999999999997e-06 2020-08-07T15:08:32.717 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.717 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.718 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:32.718 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.695680+0000", 2020-08-07T15:08:32.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0158000000000003e-05 2020-08-07T15:08:32.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.718 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:32.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.695730+0000", 2020-08-07T15:08:32.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9775999999999998e-05 2020-08-07T15:08:32.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:32.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.695750+0000", 2020-08-07T15:08:32.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13060255800000001 2020-08-07T15:08:32.721 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.721 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:32.722 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.723 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:9825 4.f e86/81 4:ff69e10c:::benchmark_data_smithi074_44586_object1228:head v 86'641, mlcod=86'638)", 2020-08-07T15:08:32.725 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:32.695571+0000", 2020-08-07T15:08:32.725 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0018692120000000001, 2020-08-07T15:08:32.725 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019718840000000001, 2020-08-07T15:08:32.725 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:32.726 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:32.726 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:32.726 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.726 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:32.726 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.695571+0000", 2020-08-07T15:08:32.738 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:32.738 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.738 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.739 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:32.739 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.695571+0000", 2020-08-07T15:08:32.739 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3198999999999997e-05 2020-08-07T15:08:32.739 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.739 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.740 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:32.740 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.695624+0000", 2020-08-07T15:08:32.740 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8650000000000001e-06 2020-08-07T15:08:32.740 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.740 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.741 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:32.741 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.695632+0000", 2020-08-07T15:08:32.741 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00079039400000000004 2020-08-07T15:08:32.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.741 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.750 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:32.750 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.696423+0000", 2020-08-07T15:08:32.751 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3089999999999997e-06 2020-08-07T15:08:32.751 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.751 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.751 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:32.751 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.696427+0000", 2020-08-07T15:08:32.752 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8371999999999998e-05 2020-08-07T15:08:32.752 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.752 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:32.752 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.696465+0000", 2020-08-07T15:08:32.753 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7948000000000002e-05 2020-08-07T15:08:32.753 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.753 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.753 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:32.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.696483+0000", 2020-08-07T15:08:32.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00091208700000000001 2020-08-07T15:08:32.765 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.765 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:32.765 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.766 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:9826 4.f e86/81)", 2020-08-07T15:08:32.772 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:32.696372+0000", 2020-08-07T15:08:32.773 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.001068428, 2020-08-07T15:08:32.773 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012183540000000001, 2020-08-07T15:08:32.773 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:32.773 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:08:32.773 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:32.774 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.774 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:32.774 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.696372+0000", 2020-08-07T15:08:32.774 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:32.774 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.775 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:32.775 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.696372+0000", 2020-08-07T15:08:32.775 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9175000000000001e-05 2020-08-07T15:08:32.775 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:32.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.696401+0000", 2020-08-07T15:08:32.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5379999999999996e-06 2020-08-07T15:08:32.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:32.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.696406+0000", 2020-08-07T15:08:32.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001101571 2020-08-07T15:08:32.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:32.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.697507+0000", 2020-08-07T15:08:32.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2119999999999997e-06 2020-08-07T15:08:32.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:32.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:32.786 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:32.786 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.697511+0000", 2020-08-07T15:08:32.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001139496 2020-08-07T15:08:32.787 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.787 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:32.787 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.787 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:32.788 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:08:32.788 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 3 2020-08-07T15:08:32.788 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:32.790 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:08:32.791 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:32.788+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:32.835 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:08:32.888 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:08:32.920 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:32.919+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:32.941 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:08:32.967 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:08:33.006 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:08:33.007 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:33.007 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:33.019 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:08:33.021 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:33.019+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:33.052 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:08:33.114 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:08:33.134 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:08:33.138 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:08:33.138 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:08:33.139 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:08:33.139 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:08:33.139 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:08:33.139 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:08:33.139 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:08:33.140 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:08:33.140 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:33.128+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:33.171 INFO:teuthology.orchestra.run.smithi074.stderr:set pool 1 pg_num to 21 2020-08-07T15:08:33.193 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:33.199 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:33.231 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:08:33.232 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:33.229+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:33.274 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 7 16 10837 10821 12.0761 13.7188 0.00483747 0.0103387 2020-08-07T15:08:33.337 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:33.330+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:33.400 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:33.400 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:33.410 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:08:33.461 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:33.460+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:33.537 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:33.534+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:33.588 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:33.588 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:33.600 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:08:33.618 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:08:33.670 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:33.666+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:33.758 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:33.754+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:33.806 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:33.806 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:33.823 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:08:33.858 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:33.855+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:33.962 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:33.957+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:34.007 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:34.007 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:34.017 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:08:34.063 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:34.061+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:34.164 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:34.162+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:34.251 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:34.252 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:34.257 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:08:34.274 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 8 16 12348 12332 12.0421 11.8047 0.00563148 0.0103682 2020-08-07T15:08:34.306 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:34.297+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:34.372 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:34.369+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:34.472 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:34.471+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:34.496 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:34.496 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:34.511 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:08:34.600 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:34.594+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:34.613 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:08:34.682 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:34.679+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:34.729 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:34.746 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:34.747 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:08:34.776 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:34.773+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:34.877 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:34.875+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:34.959 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:34.959 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:34.975 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:08:34.984 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:34.981+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:35.098 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:35.085+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:35.159 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:08:35.178 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:35.178 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:35.187 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:08:35.188 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:35.185+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:35.273 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 9 15 14026 14011 12.1615 13.1172 0.00839246 0.0102625 2020-08-07T15:08:35.290 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:35.287+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:35.348 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:08:35.388 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:08:35.395 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:35.394+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:35.421 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:08:35.426 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:35.427 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:35.438 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:08:35.458 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:08:35.509 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:08:35.511 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:35.508+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:35.536 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:08:35.572 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:08:35.609 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:08:35.615 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:08:35.615 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:08:35.616 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:08:35.616 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:08:35.616 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:08:35.616 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:08:35.617 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:08:35.617 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:08:35.617 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:35.611+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:35.658 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:35.659 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:08:35.659 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:08:35.659 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:08:35.660 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:35.713 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:08:35.715 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:35.713+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:35.818 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:35.816+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:35.900 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:35.902 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:08:35.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:08:35.902 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:08:35.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:35.903 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:9976 3.e 3:7fb2e2e5:::benchmark_data_smithi074_35447_object1246:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:08:35.903 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:08:35.903 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 113.660011751, 2020-08-07T15:08:35.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28420182999999999, 2020-08-07T15:08:35.903 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:35.904 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:35.904 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:35.905 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:35.905 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:35.905 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 9976 2020-08-07T15:08:35.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:35.906 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:35.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:35.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:35.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:08:35.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:35.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:35.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:35.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:35.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:08:35.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7113e-05 2020-08-07T15:08:35.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:35.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:35.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:35.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237255+0000", 2020-08-07T15:08:35.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5869999999999998e-06 2020-08-07T15:08:35.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:35.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:35.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:35.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237262+0000", 2020-08-07T15:08:35.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4467999999999999e-05 2020-08-07T15:08:35.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:35.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:35.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:35.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237276+0000", 2020-08-07T15:08:35.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0749999999999997e-06 2020-08-07T15:08:35.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:35.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:35.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:35.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237281+0000", 2020-08-07T15:08:35.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013795109999999999 2020-08-07T15:08:35.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:35.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:35.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:35.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238661+0000", 2020-08-07T15:08:35.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100309 2020-08-07T15:08:35.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:35.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:35.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:35.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238761+0000", 2020-08-07T15:08:35.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000141784 2020-08-07T15:08:35.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:35.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:35.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:35.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238903+0000", 2020-08-07T15:08:35.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00070626599999999999 2020-08-07T15:08:35.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:35.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:35.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:35.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239609+0000", 2020-08-07T15:08:35.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3499999999999996e-07 2020-08-07T15:08:35.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:35.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:35.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:35.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239610+0000", 2020-08-07T15:08:35.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0846000000000006e-05 2020-08-07T15:08:35.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:35.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:35.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:35.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239680+0000", 2020-08-07T15:08:35.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022037080000000001 2020-08-07T15:08:35.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:35.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:35.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:35.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.261718+0000", 2020-08-07T15:08:35.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25961784500000001 2020-08-07T15:08:35.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:35.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:35.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:35.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521335+0000", 2020-08-07T15:08:35.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2839999999999999e-06 2020-08-07T15:08:35.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:35.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:35.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:35.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521337+0000", 2020-08-07T15:08:35.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3126999999999994e-05 2020-08-07T15:08:35.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:35.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:35.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:35.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521400+0000", 2020-08-07T15:08:35.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28420182999999999 2020-08-07T15:08:35.971 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:35.971 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:35.972 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:35.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:35.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:35.972 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15576 3.e 3:727724a5:::benchmark_data_smithi074_35447_object1946:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:08:35.972 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:08:35.973 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 109.230574788, 2020-08-07T15:08:35.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999, 2020-08-07T15:08:35.973 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:35.973 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:35.974 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:35.974 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:35.974 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:35.974 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15576 2020-08-07T15:08:35.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:35.975 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:35.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:35.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:35.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:08:35.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:35.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:35.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:35.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:35.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:08:35.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7156e-05 2020-08-07T15:08:35.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:35.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:35.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:35.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666682+0000", 2020-08-07T15:08:35.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0209999999999999e-06 2020-08-07T15:08:35.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:35.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:35.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:35.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666687+0000", 2020-08-07T15:08:35.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0730999999999999e-05 2020-08-07T15:08:35.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:35.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:35.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:35.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666698+0000", 2020-08-07T15:08:35.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2020-08-07T15:08:35.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:35.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:35.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:35.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666702+0000", 2020-08-07T15:08:35.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023978459999999999 2020-08-07T15:08:35.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:35.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:35.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:35.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669100+0000", 2020-08-07T15:08:36.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9664000000000001e-05 2020-08-07T15:08:36.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:36.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669190+0000", 2020-08-07T15:08:36.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015957100000000001 2020-08-07T15:08:36.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:36.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669350+0000", 2020-08-07T15:08:36.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016470149999999999 2020-08-07T15:08:36.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:36.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:08:36.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1299999999999999e-07 2020-08-07T15:08:36.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:36.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:08:36.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5269000000000002e-05 2020-08-07T15:08:36.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:36.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.671022+0000", 2020-08-07T15:08:36.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031414400000000001 2020-08-07T15:08:36.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:36.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.674163+0000", 2020-08-07T15:08:36.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45641417400000001 2020-08-07T15:08:36.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:36.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:08:36.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0399999999999995e-07 2020-08-07T15:08:36.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:36.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:08:36.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3343999999999997e-05 2020-08-07T15:08:36.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:36.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130622+0000", 2020-08-07T15:08:36.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999 2020-08-07T15:08:36.048 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.049 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:36.049 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.050 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:17824 3.d 3:bd4e0935:::benchmark_data_smithi074_35447_object2227:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:08:36.050 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.140708+0000", 2020-08-07T15:08:36.051 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 107.756501865, 2020-08-07T15:08:36.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25688457100000001, 2020-08-07T15:08:36.051 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:36.051 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:36.052 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:36.052 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:36.052 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:36.053 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 17824 2020-08-07T15:08:36.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.053 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:36.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:36.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140708+0000", 2020-08-07T15:08:36.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:36.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:36.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140708+0000", 2020-08-07T15:08:36.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0522000000000002e-05 2020-08-07T15:08:36.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:36.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140758+0000", 2020-08-07T15:08:36.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1680000000000001e-06 2020-08-07T15:08:36.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:36.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140765+0000", 2020-08-07T15:08:36.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5791999999999998e-05 2020-08-07T15:08:36.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:36.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140810+0000", 2020-08-07T15:08:36.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2389999999999996e-06 2020-08-07T15:08:36.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:36.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.140818+0000", 2020-08-07T15:08:36.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001721544 2020-08-07T15:08:36.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:36.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.142539+0000", 2020-08-07T15:08:36.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020908100000000001 2020-08-07T15:08:36.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:36.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.142748+0000", 2020-08-07T15:08:36.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019000000000000001 2020-08-07T15:08:36.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:08:36.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.142938+0000", 2020-08-07T15:08:36.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001381279 2020-08-07T15:08:36.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:36.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.144319+0000", 2020-08-07T15:08:36.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010975570000000001 2020-08-07T15:08:36.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:36.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.145417+0000", 2020-08-07T15:08:36.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.252064128 2020-08-07T15:08:36.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:36.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397481+0000", 2020-08-07T15:08:36.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003e-06 2020-08-07T15:08:36.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:36.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397482+0000", 2020-08-07T15:08:36.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3081000000000001e-05 2020-08-07T15:08:36.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:36.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397545+0000", 2020-08-07T15:08:36.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7176999999999999e-05 2020-08-07T15:08:36.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:36.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.397592+0000", 2020-08-07T15:08:36.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25688457100000001 2020-08-07T15:08:36.111 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.111 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:36.111 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.112 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:35576 3.f 3:fca2de08:::benchmark_data_smithi074_35447_object4446:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e31)", 2020-08-07T15:08:36.112 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:59.287747+0000", 2020-08-07T15:08:36.112 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 96.609463062000003, 2020-08-07T15:08:36.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25861668500000001, 2020-08-07T15:08:36.112 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:36.113 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:36.113 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:36.113 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:36.123 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:36.124 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 35576 2020-08-07T15:08:36.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.124 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:36.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:36.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287747+0000", 2020-08-07T15:08:36.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:36.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:36.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287747+0000", 2020-08-07T15:08:36.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0089000000000001e-05 2020-08-07T15:08:36.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:36.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287797+0000", 2020-08-07T15:08:36.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4219999999999997e-06 2020-08-07T15:08:36.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:36.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287803+0000", 2020-08-07T15:08:36.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4132e-05 2020-08-07T15:08:36.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:36.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287817+0000", 2020-08-07T15:08:36.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1600000000000003e-06 2020-08-07T15:08:36.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:36.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.287823+0000", 2020-08-07T15:08:36.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019418230000000001 2020-08-07T15:08:36.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:36.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.289765+0000", 2020-08-07T15:08:36.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110597 2020-08-07T15:08:36.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:36.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.289876+0000", 2020-08-07T15:08:36.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017674800000000001 2020-08-07T15:08:36.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:36.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.290053+0000", 2020-08-07T15:08:36.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052995730000000001 2020-08-07T15:08:36.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:36.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295352+0000", 2020-08-07T15:08:36.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9999999999999997e-07 2020-08-07T15:08:36.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:36.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295353+0000", 2020-08-07T15:08:36.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6803000000000004e-05 2020-08-07T15:08:36.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:36.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.295420+0000", 2020-08-07T15:08:36.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015611539999999999 2020-08-07T15:08:36.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:36.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.296981+0000", 2020-08-07T15:08:36.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.249314866 2020-08-07T15:08:36.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:36.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546296+0000", 2020-08-07T15:08:36.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7300000000000005e-07 2020-08-07T15:08:36.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:36.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546296+0000", 2020-08-07T15:08:36.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6845000000000001e-05 2020-08-07T15:08:36.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:36.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:59.546363+0000", 2020-08-07T15:08:36.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25861668500000001 2020-08-07T15:08:36.145 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.145 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:36.145 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.146 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:41431 3.d 3:b5c8a160:::benchmark_data_smithi074_35447_object5178:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:08:36.146 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.791939+0000", 2020-08-07T15:08:36.146 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.105270258999994, 2020-08-07T15:08:36.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28436496, 2020-08-07T15:08:36.147 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:36.147 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:36.147 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:36.147 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:36.148 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:36.148 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 41431 2020-08-07T15:08:36.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.148 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:36.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:36.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791939+0000", 2020-08-07T15:08:36.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:36.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:36.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791939+0000", 2020-08-07T15:08:36.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6304999999999998e-05 2020-08-07T15:08:36.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:36.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791986+0000", 2020-08-07T15:08:36.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6339999999999998e-06 2020-08-07T15:08:36.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:36.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791991+0000", 2020-08-07T15:08:36.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.026e-05 2020-08-07T15:08:36.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:36.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792002+0000", 2020-08-07T15:08:36.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6240000000000001e-06 2020-08-07T15:08:36.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:36.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792006+0000", 2020-08-07T15:08:36.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015600220000000001 2020-08-07T15:08:36.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:36.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.793566+0000", 2020-08-07T15:08:36.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3628999999999994e-05 2020-08-07T15:08:36.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:36.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.793660+0000", 2020-08-07T15:08:36.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000168197 2020-08-07T15:08:36.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:08:36.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.793828+0000", 2020-08-07T15:08:36.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001273251 2020-08-07T15:08:36.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:36.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795101+0000", 2020-08-07T15:08:36.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5899999999999998e-07 2020-08-07T15:08:36.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:36.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795102+0000", 2020-08-07T15:08:36.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4904000000000001e-05 2020-08-07T15:08:36.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:36.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795127+0000", 2020-08-07T15:08:36.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00670237 2020-08-07T15:08:36.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:36.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801829+0000", 2020-08-07T15:08:36.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27442024999999998 2020-08-07T15:08:36.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:36.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076249+0000", 2020-08-07T15:08:36.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999997e-07 2020-08-07T15:08:36.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:36.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076250+0000", 2020-08-07T15:08:36.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4369000000000001e-05 2020-08-07T15:08:36.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:36.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076304+0000", 2020-08-07T15:08:36.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28436496 2020-08-07T15:08:36.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.202 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:36.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.203 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:41432 3.d 3:b5c8a160:::benchmark_data_smithi074_35447_object5178:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:08:36.203 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:08:36.203 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.104919562999996, 2020-08-07T15:08:36.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.37720701699999998, 2020-08-07T15:08:36.204 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:36.204 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:36.204 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:36.204 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:36.205 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:36.205 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 41432 2020-08-07T15:08:36.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.205 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:36.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:36.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:08:36.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:36.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:36.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:08:36.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4787999999999999e-05 2020-08-07T15:08:36.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:36.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792335+0000", 2020-08-07T15:08:36.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5999999999999997e-06 2020-08-07T15:08:36.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:36.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792341+0000", 2020-08-07T15:08:36.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0998e-05 2020-08-07T15:08:36.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:36.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792352+0000", 2020-08-07T15:08:36.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2989999999999996e-06 2020-08-07T15:08:36.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:36.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792357+0000", 2020-08-07T15:08:36.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001852752 2020-08-07T15:08:36.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:36.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794210+0000", 2020-08-07T15:08:36.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5771000000000002e-05 2020-08-07T15:08:36.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:36.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794305+0000", 2020-08-07T15:08:36.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012776399999999999 2020-08-07T15:08:36.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:08:36.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794433+0000", 2020-08-07T15:08:36.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00069673599999999999 2020-08-07T15:08:36.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:36.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795130+0000", 2020-08-07T15:08:36.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1500000000000001e-07 2020-08-07T15:08:36.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:36.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795130+0000", 2020-08-07T15:08:36.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7319e-05 2020-08-07T15:08:36.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:36.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795167+0000", 2020-08-07T15:08:36.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27910026700000001 2020-08-07T15:08:36.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:36.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074268+0000", 2020-08-07T15:08:36.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.095150660999999997 2020-08-07T15:08:36.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:36.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169418+0000", 2020-08-07T15:08:36.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.423e-06 2020-08-07T15:08:36.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:36.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169420+0000", 2020-08-07T15:08:36.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7423999999999996e-05 2020-08-07T15:08:36.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:36.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169497+0000", 2020-08-07T15:08:36.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.37720701699999998 2020-08-07T15:08:36.233 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.233 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:36.233 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.234 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:41431 3.d e32/20 ondisk, result = 0)", 2020-08-07T15:08:36.234 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.796229+0000", 2020-08-07T15:08:36.235 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.100980660999994, 2020-08-07T15:08:36.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28007903299999998, 2020-08-07T15:08:36.235 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:36.235 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:36.235 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:36.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:36.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.796229+0000", 2020-08-07T15:08:36.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:36.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:36.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.796229+0000", 2020-08-07T15:08:36.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3272000000000003e-05 2020-08-07T15:08:36.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:36.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.796262+0000", 2020-08-07T15:08:36.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9579999999999998e-06 2020-08-07T15:08:36.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:36.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.796267+0000", 2020-08-07T15:08:36.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27993393599999999 2020-08-07T15:08:36.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:36.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076201+0000", 2020-08-07T15:08:36.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8709999999999999e-06 2020-08-07T15:08:36.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:36.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076205+0000", 2020-08-07T15:08:36.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9128e-05 2020-08-07T15:08:36.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:36.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076234+0000", 2020-08-07T15:08:36.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9800000000000001e-06 2020-08-07T15:08:36.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:36.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076241+0000", 2020-08-07T15:08:36.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6888000000000001e-05 2020-08-07T15:08:36.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:36.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076308+0000", 2020-08-07T15:08:36.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28007903299999998 2020-08-07T15:08:36.247 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.247 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:36.247 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.248 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:41432 3.d e32/20 ondisk, result = 0)", 2020-08-07T15:08:36.248 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.801630+0000", 2020-08-07T15:08:36.248 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.095579760000007, 2020-08-07T15:08:36.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27264033999999998, 2020-08-07T15:08:36.255 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:36.256 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:36.256 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:36.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:36.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801630+0000", 2020-08-07T15:08:36.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:36.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:36.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801630+0000", 2020-08-07T15:08:36.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6257e-05 2020-08-07T15:08:36.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:36.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801646+0000", 2020-08-07T15:08:36.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5450000000000002e-06 2020-08-07T15:08:36.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:36.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801649+0000", 2020-08-07T15:08:36.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27254806999999998 2020-08-07T15:08:36.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:36.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074197+0000", 2020-08-07T15:08:36.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4079999999999997e-06 2020-08-07T15:08:36.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:36.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074202+0000", 2020-08-07T15:08:36.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5222000000000002e-05 2020-08-07T15:08:36.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:36.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074247+0000", 2020-08-07T15:08:36.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9459999999999993e-06 2020-08-07T15:08:36.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:36.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074257+0000", 2020-08-07T15:08:36.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2892e-05 2020-08-07T15:08:36.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:36.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074270+0000", 2020-08-07T15:08:36.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27264033999999998 2020-08-07T15:08:36.269 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.269 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:36.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.270 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:42104 3.d 3:bcc35439:::benchmark_data_smithi074_35447_object5262:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:08:36.270 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:08:36.271 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 92.658287392000005, 2020-08-07T15:08:36.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28812091400000001, 2020-08-07T15:08:36.271 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:36.271 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:36.271 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:36.272 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:36.272 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:36.272 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42104 2020-08-07T15:08:36.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.274 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:36.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:36.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:08:36.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:36.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:36.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:08:36.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9734000000000002e-05 2020-08-07T15:08:36.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:36.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238972+0000", 2020-08-07T15:08:36.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6910000000000002e-06 2020-08-07T15:08:36.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:36.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238978+0000", 2020-08-07T15:08:36.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1864e-05 2020-08-07T15:08:36.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:36.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238990+0000", 2020-08-07T15:08:36.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.737e-06 2020-08-07T15:08:36.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:36.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238996+0000", 2020-08-07T15:08:36.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00283943 2020-08-07T15:08:36.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:36.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.241836+0000", 2020-08-07T15:08:36.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4712999999999999e-05 2020-08-07T15:08:36.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:36.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.241930+0000", 2020-08-07T15:08:36.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015328499999999999 2020-08-07T15:08:36.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:08:36.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.242084+0000", 2020-08-07T15:08:36.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016104400000000001 2020-08-07T15:08:36.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:36.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243694+0000", 2020-08-07T15:08:36.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1399999999999997e-07 2020-08-07T15:08:36.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:36.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243695+0000", 2020-08-07T15:08:36.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7800999999999998e-05 2020-08-07T15:08:36.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:36.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243753+0000", 2020-08-07T15:08:36.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090405661999999998 2020-08-07T15:08:36.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:36.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.334158+0000", 2020-08-07T15:08:36.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19283929299999999 2020-08-07T15:08:36.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:36.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.526997+0000", 2020-08-07T15:08:36.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.012e-06 2020-08-07T15:08:36.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:36.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.526998+0000", 2020-08-07T15:08:36.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4737999999999998e-05 2020-08-07T15:08:36.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:36.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.527043+0000", 2020-08-07T15:08:36.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28812091400000001 2020-08-07T15:08:36.291 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.292 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:36.292 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.292 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:44120 3.e 3:7e6f7bb9:::benchmark_data_smithi074_35447_object5514:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:08:36.293 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:04.450106+0000", 2020-08-07T15:08:36.293 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.447103858000006, 2020-08-07T15:08:36.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26702573699999999, 2020-08-07T15:08:36.294 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:36.294 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:36.294 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:36.294 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:36.294 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:36.295 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 44120 2020-08-07T15:08:36.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.295 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:36.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:36.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450106+0000", 2020-08-07T15:08:36.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:36.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:36.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450106+0000", 2020-08-07T15:08:36.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9926000000000003e-05 2020-08-07T15:08:36.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:36.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450156+0000", 2020-08-07T15:08:36.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4079999999999997e-06 2020-08-07T15:08:36.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:36.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450161+0000", 2020-08-07T15:08:36.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0494e-05 2020-08-07T15:08:36.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:36.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450172+0000", 2020-08-07T15:08:36.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3950000000000003e-06 2020-08-07T15:08:36.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:36.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.450176+0000", 2020-08-07T15:08:36.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033213689999999998 2020-08-07T15:08:36.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:36.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.453497+0000", 2020-08-07T15:08:36.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102414 2020-08-07T15:08:36.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:36.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.453600+0000", 2020-08-07T15:08:36.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019776 2020-08-07T15:08:36.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:36.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.453798+0000", 2020-08-07T15:08:36.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001075456 2020-08-07T15:08:36.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:36.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.454873+0000", 2020-08-07T15:08:36.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7999999999999995e-07 2020-08-07T15:08:36.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:36.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.454874+0000", 2020-08-07T15:08:36.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3912000000000002e-05 2020-08-07T15:08:36.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:36.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.454928+0000", 2020-08-07T15:08:36.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058537550000000004 2020-08-07T15:08:36.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:36.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.460781+0000", 2020-08-07T15:08:36.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25630591899999999 2020-08-07T15:08:36.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:36.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.717087+0000", 2020-08-07T15:08:36.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.54e-07 2020-08-07T15:08:36.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:36.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.717088+0000", 2020-08-07T15:08:36.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3594999999999998e-05 2020-08-07T15:08:36.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:36.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:04.717132+0000", 2020-08-07T15:08:36.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26702573699999999 2020-08-07T15:08:36.323 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.323 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:36.324 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.324 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:49800 3.e 3:7cb4a069:::benchmark_data_smithi074_35447_object6224:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e36)", 2020-08-07T15:08:36.324 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:10.076186+0000", 2020-08-07T15:08:36.325 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.821024120000004, 2020-08-07T15:08:36.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25479637100000002, 2020-08-07T15:08:36.325 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:36.325 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:36.325 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:36.326 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:36.326 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:36.326 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 49800 2020-08-07T15:08:36.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.326 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:36.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:36.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076186+0000", 2020-08-07T15:08:36.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:36.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:36.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076186+0000", 2020-08-07T15:08:36.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3579000000000003e-05 2020-08-07T15:08:36.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:36.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076239+0000", 2020-08-07T15:08:36.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0499999999999997e-06 2020-08-07T15:08:36.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:36.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076245+0000", 2020-08-07T15:08:36.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1756999999999999e-05 2020-08-07T15:08:36.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:36.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076257+0000", 2020-08-07T15:08:36.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8220000000000003e-06 2020-08-07T15:08:36.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:36.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076263+0000", 2020-08-07T15:08:36.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9265999999999999e-05 2020-08-07T15:08:36.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:36.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076312+0000", 2020-08-07T15:08:36.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2280000000000004e-05 2020-08-07T15:08:36.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:36.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076404+0000", 2020-08-07T15:08:36.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015721499999999999 2020-08-07T15:08:36.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:36.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.076562+0000", 2020-08-07T15:08:36.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041832179999999998 2020-08-07T15:08:36.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:36.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.080745+0000", 2020-08-07T15:08:36.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.091e-06 2020-08-07T15:08:36.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:36.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.080746+0000", 2020-08-07T15:08:36.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3623999999999999e-05 2020-08-07T15:08:36.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:36.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.080809+0000", 2020-08-07T15:08:36.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11945386299999999 2020-08-07T15:08:36.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:36.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.200263+0000", 2020-08-07T15:08:36.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.130675596 2020-08-07T15:08:36.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:36.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.330939+0000", 2020-08-07T15:08:36.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.508e-06 2020-08-07T15:08:36.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:36.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.330940+0000", 2020-08-07T15:08:36.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1501999999999997e-05 2020-08-07T15:08:36.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:36.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:10.330982+0000", 2020-08-07T15:08:36.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25479637100000002 2020-08-07T15:08:36.344 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.344 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:36.344 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.345 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:86712 3.8 e52/20 3:14925daa:::benchmark_data_smithi074_35447_object10838:head v 52'5488, mlcod=52'5479)", 2020-08-07T15:08:36.345 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.338631+0000", 2020-08-07T15:08:36.345 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.558578294, 2020-08-07T15:08:36.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.310229594, 2020-08-07T15:08:36.345 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:36.346 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:36.346 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:36.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:36.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338631+0000", 2020-08-07T15:08:36.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:36.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:36.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338631+0000", 2020-08-07T15:08:36.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6180000000000001e-05 2020-08-07T15:08:36.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:36.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338688+0000", 2020-08-07T15:08:36.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4949999999999996e-06 2020-08-07T15:08:36.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:36.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338694+0000", 2020-08-07T15:08:36.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30438292099999997 2020-08-07T15:08:36.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:36.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643077+0000", 2020-08-07T15:08:36.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9389999999999996e-06 2020-08-07T15:08:36.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:36.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643082+0000", 2020-08-07T15:08:36.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5096999999999999e-05 2020-08-07T15:08:36.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:36.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643127+0000", 2020-08-07T15:08:36.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4323000000000001e-05 2020-08-07T15:08:36.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:36.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643151+0000", 2020-08-07T15:08:36.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055847300000000004 2020-08-07T15:08:36.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:36.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.648736+0000", 2020-08-07T15:08:36.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012490900000000001 2020-08-07T15:08:36.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:36.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.648861+0000", 2020-08-07T15:08:36.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.310229594 2020-08-07T15:08:36.357 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.357 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:36.357 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.358 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86792 3.f 3:fc5386ee:::benchmark_data_smithi074_35447_object10848:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:08:36.358 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.395402+0000", 2020-08-07T15:08:36.358 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.501808183999998, 2020-08-07T15:08:36.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30259831999999998, 2020-08-07T15:08:36.359 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:36.359 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:36.359 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:36.359 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:36.359 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:36.360 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86792 2020-08-07T15:08:36.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.360 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:36.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:36.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395402+0000", 2020-08-07T15:08:36.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:36.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:36.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395402+0000", 2020-08-07T15:08:36.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2262999999999998e-05 2020-08-07T15:08:36.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:36.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395444+0000", 2020-08-07T15:08:36.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1800000000000004e-06 2020-08-07T15:08:36.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:36.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395449+0000", 2020-08-07T15:08:36.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0738e-05 2020-08-07T15:08:36.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:36.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395460+0000", 2020-08-07T15:08:36.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7219999999999999e-06 2020-08-07T15:08:36.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:36.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395464+0000", 2020-08-07T15:08:36.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0063169089999999999 2020-08-07T15:08:36.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:36.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.401781+0000", 2020-08-07T15:08:36.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010702200000000001 2020-08-07T15:08:36.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:36.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.401888+0000", 2020-08-07T15:08:36.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019171999999999999 2020-08-07T15:08:36.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:36.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.402080+0000", 2020-08-07T15:08:36.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042866839999999998 2020-08-07T15:08:36.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:36.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.406367+0000", 2020-08-07T15:08:36.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000001e-07 2020-08-07T15:08:36.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:36.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.406367+0000", 2020-08-07T15:08:36.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4720000000000004e-05 2020-08-07T15:08:36.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:36.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.406432+0000", 2020-08-07T15:08:36.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050405365000000001 2020-08-07T15:08:36.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:36.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.456837+0000", 2020-08-07T15:08:36.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24111702700000001 2020-08-07T15:08:36.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:36.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.697954+0000", 2020-08-07T15:08:36.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.22e-06 2020-08-07T15:08:36.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:36.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.697955+0000", 2020-08-07T15:08:36.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4336999999999999e-05 2020-08-07T15:08:36.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:36.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.698000+0000", 2020-08-07T15:08:36.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30259831999999998 2020-08-07T15:08:36.387 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.387 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:36.388 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.388 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:97576 3.d e57/35 3:b9e86280:::benchmark_data_smithi074_35447_object12196:head v 57'6304, mlcod=57'6295)", 2020-08-07T15:08:36.388 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:41.740780+0000", 2020-08-07T15:08:36.389 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.156430065999999, 2020-08-07T15:08:36.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27784160200000002, 2020-08-07T15:08:36.389 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:36.389 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:36.390 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:36.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:36.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.740780+0000", 2020-08-07T15:08:36.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:36.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:36.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.740780+0000", 2020-08-07T15:08:36.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0355999999999997e-05 2020-08-07T15:08:36.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:36.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.740830+0000", 2020-08-07T15:08:36.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6680000000000004e-06 2020-08-07T15:08:36.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:36.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:41.740837+0000", 2020-08-07T15:08:36.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27346419799999999 2020-08-07T15:08:36.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:36.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.014301+0000", 2020-08-07T15:08:36.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.852e-06 2020-08-07T15:08:36.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:36.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.014307+0000", 2020-08-07T15:08:36.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6504e-05 2020-08-07T15:08:36.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:36.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.014363+0000", 2020-08-07T15:08:36.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8759999999999999e-05 2020-08-07T15:08:36.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:36.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.014392+0000", 2020-08-07T15:08:36.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041857459999999997 2020-08-07T15:08:36.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:36.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.018578+0000", 2020-08-07T15:08:36.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3517999999999999e-05 2020-08-07T15:08:36.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:36.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:42.018621+0000", 2020-08-07T15:08:36.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27784160200000002 2020-08-07T15:08:36.426 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.426 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:36.426 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.427 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:104104 3.8 e58/20 3:11c276ea:::benchmark_data_smithi074_35447_object13012:head v 58'6496, mlcod=58'6487)", 2020-08-07T15:08:36.427 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:08:36.427 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.038724041999998, 2020-08-07T15:08:36.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33178497099999998, 2020-08-07T15:08:36.428 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:36.428 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:36.428 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:36.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:36.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:08:36.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:36.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:36.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:08:36.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9961999999999999e-05 2020-08-07T15:08:36.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:36.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858516+0000", 2020-08-07T15:08:36.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3200000000000001e-06 2020-08-07T15:08:36.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:36.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858520+0000", 2020-08-07T15:08:36.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32756790000000002 2020-08-07T15:08:36.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:36.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186088+0000", 2020-08-07T15:08:36.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0260000000000004e-06 2020-08-07T15:08:36.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:36.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186094+0000", 2020-08-07T15:08:36.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7919000000000001e-05 2020-08-07T15:08:36.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:36.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186132+0000", 2020-08-07T15:08:36.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4968e-05 2020-08-07T15:08:36.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:36.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186147+0000", 2020-08-07T15:08:36.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040671520000000001 2020-08-07T15:08:36.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:36.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.190214+0000", 2020-08-07T15:08:36.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6724000000000001e-05 2020-08-07T15:08:36.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:36.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.190271+0000", 2020-08-07T15:08:36.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33178497099999998 2020-08-07T15:08:36.440 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.440 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:36.441 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.441 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106248 3.f 3:f950fd9d:::benchmark_data_smithi074_35447_object13280:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:08:36.441 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:08:36.442 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.537045247999998, 2020-08-07T15:08:36.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002, 2020-08-07T15:08:36.442 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:36.442 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:36.442 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:36.443 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:36.443 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:36.443 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106248 2020-08-07T15:08:36.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.444 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:36.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:36.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:08:36.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:36.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:36.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:08:36.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7184e-05 2020-08-07T15:08:36.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:36.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360222+0000", 2020-08-07T15:08:36.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0906e-05 2020-08-07T15:08:36.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:36.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360233+0000", 2020-08-07T15:08:36.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2972999999999999e-05 2020-08-07T15:08:36.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:36.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360246+0000", 2020-08-07T15:08:36.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0114e-05 2020-08-07T15:08:36.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:36.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360256+0000", 2020-08-07T15:08:36.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1303999999999997e-05 2020-08-07T15:08:36.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:36.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360287+0000", 2020-08-07T15:08:36.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3798000000000001e-05 2020-08-07T15:08:36.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:36.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360351+0000", 2020-08-07T15:08:36.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111308 2020-08-07T15:08:36.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:36.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360462+0000", 2020-08-07T15:08:36.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003480674 2020-08-07T15:08:36.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:36.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:08:36.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0900000000000001e-07 2020-08-07T15:08:36.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:36.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:08:36.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8899999999999994e-05 2020-08-07T15:08:36.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:36.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.364012+0000", 2020-08-07T15:08:36.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35291637199999998 2020-08-07T15:08:36.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:36.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716929+0000", 2020-08-07T15:08:36.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.076244233999999994 2020-08-07T15:08:36.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:36.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793173+0000", 2020-08-07T15:08:36.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6109999999999999e-06 2020-08-07T15:08:36.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:36.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793175+0000", 2020-08-07T15:08:36.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7492999999999999e-05 2020-08-07T15:08:36.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:36.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793242+0000", 2020-08-07T15:08:36.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002 2020-08-07T15:08:36.461 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.461 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:36.461 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.462 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106248 3.f e59/20 ondisk, result = 0)", 2020-08-07T15:08:36.462 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.365323+0000", 2020-08-07T15:08:36.462 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.531886692, 2020-08-07T15:08:36.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35160734999999999, 2020-08-07T15:08:36.463 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:36.463 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:36.464 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:36.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:36.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365323+0000", 2020-08-07T15:08:36.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:36.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:36.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365323+0000", 2020-08-07T15:08:36.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4981000000000003e-05 2020-08-07T15:08:36.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:36.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365358+0000", 2020-08-07T15:08:36.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1090000000000003e-06 2020-08-07T15:08:36.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:36.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365364+0000", 2020-08-07T15:08:36.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35151899399999997 2020-08-07T15:08:36.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:36.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716883+0000", 2020-08-07T15:08:36.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0090000000000001e-06 2020-08-07T15:08:36.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:36.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716887+0000", 2020-08-07T15:08:36.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9238999999999999e-05 2020-08-07T15:08:36.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:36.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716916+0000", 2020-08-07T15:08:36.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0390000000000001e-06 2020-08-07T15:08:36.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:36.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716922+0000", 2020-08-07T15:08:36.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9789999999999993e-06 2020-08-07T15:08:36.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:36.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716930+0000", 2020-08-07T15:08:36.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35160734999999999 2020-08-07T15:08:36.474 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.475 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:36.475 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.476 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106248 3.f e59/20 ondisk, result = 0)", 2020-08-07T15:08:36.476 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:08:36.476 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.531781963, 2020-08-07T15:08:36.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001, 2020-08-07T15:08:36.477 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:36.477 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:36.477 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:36.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:36.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:08:36.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:36.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:36.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:08:36.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7518999999999999e-05 2020-08-07T15:08:36.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:36.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365445+0000", 2020-08-07T15:08:36.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-06 2020-08-07T15:08:36.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:36.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365449+0000", 2020-08-07T15:08:36.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42764147600000002 2020-08-07T15:08:36.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:36.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793091+0000", 2020-08-07T15:08:36.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6500000000000002e-06 2020-08-07T15:08:36.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:36.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793099+0000", 2020-08-07T15:08:36.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3664e-05 2020-08-07T15:08:36.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:36.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793153+0000", 2020-08-07T15:08:36.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1078e-05 2020-08-07T15:08:36.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:36.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793164+0000", 2020-08-07T15:08:36.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3404000000000003e-05 2020-08-07T15:08:36.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:36.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793248+0000", 2020-08-07T15:08:36.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001 2020-08-07T15:08:36.487 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.488 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:36.488 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.488 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106344 3.e 3:7abbcafa:::benchmark_data_smithi074_35447_object13292:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:08:36.489 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:48.059687+0000", 2020-08-07T15:08:36.489 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.837522860999997, 2020-08-07T15:08:36.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28541866100000002, 2020-08-07T15:08:36.489 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:36.489 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:36.490 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:36.490 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:36.490 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:36.490 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106344 2020-08-07T15:08:36.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.490 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:36.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:36.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059687+0000", 2020-08-07T15:08:36.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:36.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:36.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059687+0000", 2020-08-07T15:08:36.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5778999999999998e-05 2020-08-07T15:08:36.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:36.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059713+0000", 2020-08-07T15:08:36.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.422e-06 2020-08-07T15:08:36.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:36.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059716+0000", 2020-08-07T15:08:36.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7679999999999997e-06 2020-08-07T15:08:36.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:36.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059725+0000", 2020-08-07T15:08:36.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2020-08-07T15:08:36.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:36.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059729+0000", 2020-08-07T15:08:36.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6205000000000003e-05 2020-08-07T15:08:36.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:36.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059765+0000", 2020-08-07T15:08:36.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3615999999999995e-05 2020-08-07T15:08:36.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:36.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059829+0000", 2020-08-07T15:08:36.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000118903 2020-08-07T15:08:36.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:36.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059948+0000", 2020-08-07T15:08:36.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00070666299999999995 2020-08-07T15:08:36.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:36.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.060654+0000", 2020-08-07T15:08:36.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5199999999999995e-07 2020-08-07T15:08:36.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:36.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.060655+0000", 2020-08-07T15:08:36.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2092999999999998e-05 2020-08-07T15:08:36.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:36.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.060697+0000", 2020-08-07T15:08:36.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00038454500000000003 2020-08-07T15:08:36.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:36.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.061082+0000", 2020-08-07T15:08:36.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.283939936 2020-08-07T15:08:36.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:36.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.345022+0000", 2020-08-07T15:08:36.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.288e-06 2020-08-07T15:08:36.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:36.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.345023+0000", 2020-08-07T15:08:36.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2468999999999999e-05 2020-08-07T15:08:36.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:36.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.345105+0000", 2020-08-07T15:08:36.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28541866100000002 2020-08-07T15:08:36.512 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.513 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:36.513 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.513 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:107952 3.d e61/35)", 2020-08-07T15:08:36.514 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:08:36.514 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.570852254999998, 2020-08-07T15:08:36.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001, 2020-08-07T15:08:36.514 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:36.515 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:08:36.515 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:36.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:36.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:08:36.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:36.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:36.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:08:36.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8826000000000003e-05 2020-08-07T15:08:36.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:36.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326406+0000", 2020-08-07T15:08:36.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9199999999999998e-06 2020-08-07T15:08:36.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:36.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326413+0000", 2020-08-07T15:08:36.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75082853500000002 2020-08-07T15:08:36.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:36.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077242+0000", 2020-08-07T15:08:36.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1479999999999999e-06 2020-08-07T15:08:36.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:36.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077246+0000", 2020-08-07T15:08:36.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3331999999999998e-05 2020-08-07T15:08:36.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:36.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077289+0000", 2020-08-07T15:08:36.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8977000000000001e-05 2020-08-07T15:08:36.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:36.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:36.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:36.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077308+0000", 2020-08-07T15:08:36.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001 2020-08-07T15:08:36.527 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.528 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:36.528 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.528 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:36.528 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:36.528 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:36.534 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 10 16 15721 15705 12.2687 13.2344 0.00311523 0.010172 2020-08-07T15:08:36.535 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:36.528+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:36.536 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:08:36.635 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:36.633+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:36.641 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:08:36.710 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:36.710 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:08:36.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.711 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:16408 4.3 e90/81 4:c4625da2:::benchmark_data_smithi074_44586_object2050:head v 90'984, mlcod=89'975)", 2020-08-07T15:08:36.711 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:36.701436+0000", 2020-08-07T15:08:36.711 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0074256139999999997, 2020-08-07T15:08:36.712 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074516560000000001, 2020-08-07T15:08:36.712 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:36.713 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:36.713 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:36.713 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.714 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:36.714 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.701436+0000", 2020-08-07T15:08:36.714 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:36.714 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.715 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.715 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:36.715 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.701436+0000", 2020-08-07T15:08:36.715 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2636000000000002e-05 2020-08-07T15:08:36.716 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.716 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.716 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:36.718 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.701489+0000", 2020-08-07T15:08:36.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4529999999999999e-06 2020-08-07T15:08:36.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:36.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.701495+0000", 2020-08-07T15:08:36.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5999000000000003e-05 2020-08-07T15:08:36.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.721 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:36.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.701561+0000", 2020-08-07T15:08:36.722 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4090000000000002e-06 2020-08-07T15:08:36.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.722 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:36.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.701567+0000", 2020-08-07T15:08:36.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4165000000000002e-05 2020-08-07T15:08:36.724 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.724 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.724 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:36.724 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.701601+0000", 2020-08-07T15:08:36.725 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1469999999999999e-05 2020-08-07T15:08:36.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.727 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:36.728 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.701622+0000", 2020-08-07T15:08:36.730 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0072283440000000003 2020-08-07T15:08:36.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.734 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.734 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:36.735 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.708851+0000", 2020-08-07T15:08:36.735 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074144759999999997 2020-08-07T15:08:36.735 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:36.735 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:36.735 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:36.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.736 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.736 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:16411 4.7 e90/81 4:eba9853c:::benchmark_data_smithi074_44586_object2051:head v 90'1187, mlcod=90'1182)", 2020-08-07T15:08:36.736 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:36.701775+0000", 2020-08-07T15:08:36.737 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0070874270000000003, 2020-08-07T15:08:36.737 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0072530210000000001, 2020-08-07T15:08:36.737 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:36.737 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:36.738 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:36.738 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.738 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:36.738 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.701775+0000", 2020-08-07T15:08:36.739 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:36.739 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.745 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:36.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.701775+0000", 2020-08-07T15:08:36.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9013e-05 2020-08-07T15:08:36.746 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.746 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:36.746 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.701824+0000", 2020-08-07T15:08:36.746 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0090000000000004e-06 2020-08-07T15:08:36.747 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.747 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.747 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:36.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.701830+0000", 2020-08-07T15:08:36.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7158999999999999e-05 2020-08-07T15:08:36.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.757 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:36.757 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.701887+0000", 2020-08-07T15:08:36.789 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5230000000000003e-06 2020-08-07T15:08:36.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.790 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:36.791 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.701892+0000", 2020-08-07T15:08:36.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8748000000000002e-05 2020-08-07T15:08:36.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.792 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:36.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.701931+0000", 2020-08-07T15:08:36.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.065e-05 2020-08-07T15:08:36.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.792 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.793 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:36.793 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.701952+0000", 2020-08-07T15:08:36.793 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000177102 2020-08-07T15:08:36.793 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:36.794 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:36.794 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:36.794 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.794 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.794 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:16412 4.7 e90/81 4:eba9853c:::benchmark_data_smithi074_44586_object2051:head v 90'1188, mlcod=90'1182)", 2020-08-07T15:08:36.795 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:36.702552+0000", 2020-08-07T15:08:36.795 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0063096209999999996, 2020-08-07T15:08:36.795 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0065471820000000003, 2020-08-07T15:08:36.795 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:36.796 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:36.796 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:36.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:36.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.702552+0000", 2020-08-07T15:08:36.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:36.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.836 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:36.836 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.702552+0000", 2020-08-07T15:08:36.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8412000000000003e-05 2020-08-07T15:08:36.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:36.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.702601+0000", 2020-08-07T15:08:36.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0399999999999998e-06 2020-08-07T15:08:36.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.856 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:36.856 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.702607+0000", 2020-08-07T15:08:36.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2759e-05 2020-08-07T15:08:36.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:36.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.702670+0000", 2020-08-07T15:08:36.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.507e-06 2020-08-07T15:08:36.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.860 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:36.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.702675+0000", 2020-08-07T15:08:36.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1532000000000001e-05 2020-08-07T15:08:36.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.862 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:36.862 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.702717+0000", 2020-08-07T15:08:36.862 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3278e-05 2020-08-07T15:08:36.862 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.862 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:36.866 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.702740+0000", 2020-08-07T15:08:36.867 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018752800000000001 2020-08-07T15:08:36.867 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:36.867 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:36.868 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:36.868 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.868 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.868 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:16413 4.7 e90/81 4:eba9853c:::benchmark_data_smithi074_44586_object2051:head v 90'1189, mlcod=90'1182)", 2020-08-07T15:08:36.869 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:36.703352+0000", 2020-08-07T15:08:36.869 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.00550959, 2020-08-07T15:08:36.869 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0058254259999999999, 2020-08-07T15:08:36.869 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:36.870 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:36.870 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:36.870 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.870 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:36.871 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.703352+0000", 2020-08-07T15:08:36.871 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:36.871 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.872 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:36.872 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.703352+0000", 2020-08-07T15:08:36.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0284999999999997e-05 2020-08-07T15:08:36.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.872 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.873 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:36.873 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.703403+0000", 2020-08-07T15:08:36.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4350000000000002e-06 2020-08-07T15:08:36.873 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:36.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.703409+0000", 2020-08-07T15:08:36.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6518000000000003e-05 2020-08-07T15:08:36.877 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.877 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.877 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:36.877 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.703466+0000", 2020-08-07T15:08:36.878 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5289999999999999e-06 2020-08-07T15:08:36.878 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.879 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:36.879 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.703471+0000", 2020-08-07T15:08:36.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7694999999999997e-05 2020-08-07T15:08:36.879 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.880 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:36.880 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.703509+0000", 2020-08-07T15:08:36.880 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3621e-05 2020-08-07T15:08:36.881 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.881 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.881 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:36.881 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.703532+0000", 2020-08-07T15:08:36.882 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018008299999999999 2020-08-07T15:08:36.882 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:36.882 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:36.882 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:36.883 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.883 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.883 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:16414 4.7 e90/81 4:eba9853c:::benchmark_data_smithi074_44586_object2051:head v 90'1190, mlcod=90'1182)", 2020-08-07T15:08:36.883 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:36.703836+0000", 2020-08-07T15:08:36.884 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.005025926, 2020-08-07T15:08:36.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054092289999999998, 2020-08-07T15:08:36.885 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:36.885 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:36.885 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:36.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.886 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:36.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.703836+0000", 2020-08-07T15:08:36.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:36.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:36.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.703836+0000", 2020-08-07T15:08:36.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0546e-05 2020-08-07T15:08:36.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.890 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:36.890 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.703887+0000", 2020-08-07T15:08:36.890 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4130000000000003e-06 2020-08-07T15:08:36.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.891 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.891 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:36.892 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.703893+0000", 2020-08-07T15:08:36.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3405000000000006e-05 2020-08-07T15:08:36.892 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.893 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:36.893 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.703966+0000", 2020-08-07T15:08:36.893 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2050000000000001e-06 2020-08-07T15:08:36.894 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.894 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.894 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:36.894 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.703972+0000", 2020-08-07T15:08:36.897 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6706999999999999e-05 2020-08-07T15:08:36.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.900 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:36.901 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.704008+0000", 2020-08-07T15:08:36.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6878e-05 2020-08-07T15:08:36.901 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.903 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:36.903 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.704025+0000", 2020-08-07T15:08:36.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000189154 2020-08-07T15:08:36.904 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:36.904 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:36.904 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:36.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.906 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:16415 4.7 e90/81 4:eba9853c:::benchmark_data_smithi074_44586_object2051:head v 90'1191, mlcod=90'1182)", 2020-08-07T15:08:36.906 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:36.704427+0000", 2020-08-07T15:08:36.906 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0044348019999999998, 2020-08-07T15:08:36.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0048601139999999996, 2020-08-07T15:08:36.907 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:36.907 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:36.907 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:36.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.908 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:36.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.704427+0000", 2020-08-07T15:08:36.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:36.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.910 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:36.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.704427+0000", 2020-08-07T15:08:36.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0175e-05 2020-08-07T15:08:36.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:36.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.704477+0000", 2020-08-07T15:08:36.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.567e-06 2020-08-07T15:08:36.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:36.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.704484+0000", 2020-08-07T15:08:36.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5228e-05 2020-08-07T15:08:36.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:36.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.704539+0000", 2020-08-07T15:08:36.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6879999999999996e-06 2020-08-07T15:08:36.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:36.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.704545+0000", 2020-08-07T15:08:36.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7327999999999999e-05 2020-08-07T15:08:36.918 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.918 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:36.919 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.704582+0000", 2020-08-07T15:08:36.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0993e-05 2020-08-07T15:08:36.919 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:36.920 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.704603+0000", 2020-08-07T15:08:36.920 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017597899999999999 2020-08-07T15:08:36.921 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:36.921 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:36.921 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:36.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.922 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:16416 4.7 e90/81 4:eba9853c:::benchmark_data_smithi074_44586_object2051:head v 90'1192, mlcod=90'1184)", 2020-08-07T15:08:36.924 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:36.705760+0000", 2020-08-07T15:08:36.924 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0031017100000000001, 2020-08-07T15:08:36.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0035675809999999998, 2020-08-07T15:08:36.925 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:36.926 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:36.926 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:36.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:36.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.705760+0000", 2020-08-07T15:08:36.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:36.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:36.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.705760+0000", 2020-08-07T15:08:36.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7732999999999998e-05 2020-08-07T15:08:36.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.932 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.933 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:36.933 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.705818+0000", 2020-08-07T15:08:36.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3080000000000001e-06 2020-08-07T15:08:36.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.934 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.934 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:36.934 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.705822+0000", 2020-08-07T15:08:36.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9142999999999999e-05 2020-08-07T15:08:36.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:36.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.705881+0000", 2020-08-07T15:08:36.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1770000000000001e-06 2020-08-07T15:08:36.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:36.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.705889+0000", 2020-08-07T15:08:36.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2425999999999997e-05 2020-08-07T15:08:36.937 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:36.940 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.705931+0000", 2020-08-07T15:08:36.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.73e-05 2020-08-07T15:08:36.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:36.940 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:36.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:36.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.705948+0000", 2020-08-07T15:08:36.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000188087 2020-08-07T15:08:36.941 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:36.942 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:36.943 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:36.943 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:36.944 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:08:36.944 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 7 2020-08-07T15:08:36.944 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:36.946 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:08:36.947 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:36.944+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:37.049 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:37.045+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:37.118 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:37.119 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:08:37.119 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:08:37.119 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:08:37.119 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:37.131 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:08:37.182 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:37.169+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:37.263 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:37.260+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:37.280 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 11 16 17228 17212 12.2236 11.7734 0.00606433 0.0101973 2020-08-07T15:08:37.356 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:37.357 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:08:37.357 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:08:37.357 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:08:37.357 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.358 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:10240 3.6 3:665d71ff:::benchmark_data_smithi074_35447_object1279:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:08:37.358 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:08:37.359 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 114.961094217, 2020-08-07T15:08:37.359 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25191569699999999, 2020-08-07T15:08:37.359 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:37.359 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:37.359 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:37.360 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:37.360 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:37.360 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 10240 2020-08-07T15:08:37.360 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.360 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:37.361 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.361 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:37.361 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:08:37.361 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:37.361 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.362 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.362 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:37.362 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:08:37.362 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5463000000000002e-05 2020-08-07T15:08:37.362 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.363 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.363 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:37.363 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393264+0000", 2020-08-07T15:08:37.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9219999999999999e-06 2020-08-07T15:08:37.363 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.363 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:37.364 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393268+0000", 2020-08-07T15:08:37.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6149999999999997e-06 2020-08-07T15:08:37.365 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.365 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.365 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:37.365 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393277+0000", 2020-08-07T15:08:37.366 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2039999999999996e-06 2020-08-07T15:08:37.366 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.366 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:37.366 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393282+0000", 2020-08-07T15:08:37.367 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001225243 2020-08-07T15:08:37.367 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.367 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.367 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:37.367 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394507+0000", 2020-08-07T15:08:37.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7108999999999995e-05 2020-08-07T15:08:37.368 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.368 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.368 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:37.368 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394605+0000", 2020-08-07T15:08:37.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017490800000000001 2020-08-07T15:08:37.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.369 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:37.369 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394779+0000", 2020-08-07T15:08:37.369 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032608699999999999 2020-08-07T15:08:37.370 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.370 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.370 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:37.370 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.398040+0000", 2020-08-07T15:08:37.370 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013015259999999999 2020-08-07T15:08:37.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.371 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.371 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:37.371 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.399342+0000", 2020-08-07T15:08:37.371 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24566893400000001 2020-08-07T15:08:37.372 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.372 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:37.372 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645011+0000", 2020-08-07T15:08:37.372 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.217e-06 2020-08-07T15:08:37.372 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.373 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.373 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:37.373 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645012+0000", 2020-08-07T15:08:37.373 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0803000000000006e-05 2020-08-07T15:08:37.373 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.374 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.374 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:37.374 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645083+0000", 2020-08-07T15:08:37.374 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1882999999999998e-05 2020-08-07T15:08:37.374 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.375 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:37.375 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645135+0000", 2020-08-07T15:08:37.375 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25191569699999999 2020-08-07T15:08:37.375 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.376 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:37.376 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.376 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:43335 3.6 e32/26 3:65a57ed3:::benchmark_data_smithi074_35447_object5416:head v 32'2831, mlcod=32'2822)", 2020-08-07T15:08:37.377 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:03.998850+0000", 2020-08-07T15:08:37.377 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 93.355463146000005, 2020-08-07T15:08:37.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.221384421, 2020-08-07T15:08:37.377 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:37.377 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:37.378 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:37.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:37.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998850+0000", 2020-08-07T15:08:37.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:37.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:37.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998850+0000", 2020-08-07T15:08:37.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4788000000000004e-05 2020-08-07T15:08:37.379 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:37.380 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998915+0000", 2020-08-07T15:08:37.380 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7909999999999999e-06 2020-08-07T15:08:37.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:37.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998922+0000", 2020-08-07T15:08:37.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22050499700000001 2020-08-07T15:08:37.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.382 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.382 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:37.382 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219427+0000", 2020-08-07T15:08:37.382 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0950000000000001e-06 2020-08-07T15:08:37.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.383 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.383 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:37.383 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219433+0000", 2020-08-07T15:08:37.383 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6342999999999997e-05 2020-08-07T15:08:37.383 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.384 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.384 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:37.384 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219509+0000", 2020-08-07T15:08:37.384 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4173999999999999e-05 2020-08-07T15:08:37.384 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.385 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.385 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:37.385 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219533+0000", 2020-08-07T15:08:37.385 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00064214899999999995 2020-08-07T15:08:37.385 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.386 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.386 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:37.386 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.220175+0000", 2020-08-07T15:08:37.386 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9083999999999999e-05 2020-08-07T15:08:37.386 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.386 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.387 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:37.387 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.220234+0000", 2020-08-07T15:08:37.387 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.221384421 2020-08-07T15:08:37.387 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.387 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:37.388 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.388 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.388 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.388 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:43336 3.6 e32/26 3:65a57ed3:::benchmark_data_smithi074_35447_object5416:head v 32'2832, mlcod=32'2822)", 2020-08-07T15:08:37.388 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:03.999509+0000", 2020-08-07T15:08:37.389 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 93.354803817000004, 2020-08-07T15:08:37.389 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22632864799999999, 2020-08-07T15:08:37.389 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:37.390 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:37.390 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:37.390 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.390 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:37.390 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999509+0000", 2020-08-07T15:08:37.391 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:37.391 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.391 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.391 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:37.391 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999509+0000", 2020-08-07T15:08:37.392 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0107e-05 2020-08-07T15:08:37.392 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.392 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.392 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:37.392 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999559+0000", 2020-08-07T15:08:37.393 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0889999999999996e-06 2020-08-07T15:08:37.393 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.393 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.393 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:37.403 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999566+0000", 2020-08-07T15:08:37.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22332094599999999 2020-08-07T15:08:37.404 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.404 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.404 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:37.404 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222887+0000", 2020-08-07T15:08:37.405 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5990000000000001e-06 2020-08-07T15:08:37.405 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.405 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:37.406 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222892+0000", 2020-08-07T15:08:37.406 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2342999999999998e-05 2020-08-07T15:08:37.406 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.406 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.406 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:37.407 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222924+0000", 2020-08-07T15:08:37.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9108999999999999e-05 2020-08-07T15:08:37.407 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.407 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.407 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:37.408 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222944+0000", 2020-08-07T15:08:37.408 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028409289999999999 2020-08-07T15:08:37.408 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.408 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.409 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:37.409 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.225784+0000", 2020-08-07T15:08:37.409 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3526000000000002e-05 2020-08-07T15:08:37.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.409 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.410 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:37.410 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.225838+0000", 2020-08-07T15:08:37.410 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22632864799999999 2020-08-07T15:08:37.410 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.410 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:37.411 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.411 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.411 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.411 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:43784 3.6 e32/26 3:6da53356:::benchmark_data_smithi074_35447_object5472:head v 32'2848, mlcod=32'2839)", 2020-08-07T15:08:37.411 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:04.253355+0000", 2020-08-07T15:08:37.412 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 93.100958245000001, 2020-08-07T15:08:37.412 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21456700500000001, 2020-08-07T15:08:37.412 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:37.412 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:37.412 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:37.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.413 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:37.413 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253355+0000", 2020-08-07T15:08:37.413 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:37.413 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.414 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:37.414 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253355+0000", 2020-08-07T15:08:37.414 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6168000000000002e-05 2020-08-07T15:08:37.414 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.414 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.415 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:37.415 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253401+0000", 2020-08-07T15:08:37.415 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3179999999999997e-06 2020-08-07T15:08:37.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.416 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:37.416 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253406+0000", 2020-08-07T15:08:37.416 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21194338600000001 2020-08-07T15:08:37.416 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.416 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.417 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:37.417 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465350+0000", 2020-08-07T15:08:37.417 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3759999999999999e-06 2020-08-07T15:08:37.417 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.418 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.418 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:37.418 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465355+0000", 2020-08-07T15:08:37.418 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8250999999999997e-05 2020-08-07T15:08:37.419 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.419 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.419 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:37.419 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465393+0000", 2020-08-07T15:08:37.419 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0296999999999999e-05 2020-08-07T15:08:37.420 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.420 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.420 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:37.420 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465414+0000", 2020-08-07T15:08:37.420 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024515209999999999 2020-08-07T15:08:37.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.421 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.421 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:37.421 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.467865+0000", 2020-08-07T15:08:37.421 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6687999999999997e-05 2020-08-07T15:08:37.422 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.422 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:37.422 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.467922+0000", 2020-08-07T15:08:37.422 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21456700500000001 2020-08-07T15:08:37.423 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.423 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:37.423 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.423 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.423 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.424 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:54200 3.3 e37/20 3:cc40637e:::benchmark_data_smithi074_35447_object6774:head v 37'3248, mlcod=37'3236)", 2020-08-07T15:08:37.424 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:08:37.424 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 84.255750997000007, 2020-08-07T15:08:37.424 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24065872499999999, 2020-08-07T15:08:37.424 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:37.424 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:37.425 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:37.425 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:37.425 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:08:37.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:37.426 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.426 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.426 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:37.426 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:08:37.426 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1671000000000002e-05 2020-08-07T15:08:37.427 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.427 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:37.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098594+0000", 2020-08-07T15:08:37.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0890000000000002e-06 2020-08-07T15:08:37.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:37.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098598+0000", 2020-08-07T15:08:37.430 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23875592300000001 2020-08-07T15:08:37.430 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.430 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.430 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:37.430 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337354+0000", 2020-08-07T15:08:37.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0420000000000002e-06 2020-08-07T15:08:37.431 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.431 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:37.431 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337359+0000", 2020-08-07T15:08:37.432 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0586000000000001e-05 2020-08-07T15:08:37.432 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.432 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.432 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:37.432 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337399+0000", 2020-08-07T15:08:37.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9415999999999998e-05 2020-08-07T15:08:37.433 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.433 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.433 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:37.433 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337419+0000", 2020-08-07T15:08:37.434 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017568390000000001 2020-08-07T15:08:37.434 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.434 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.434 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:37.434 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.339176+0000", 2020-08-07T15:08:37.435 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5158999999999999e-05 2020-08-07T15:08:37.435 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.435 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.435 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:37.436 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.339221+0000", 2020-08-07T15:08:37.436 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24065872499999999 2020-08-07T15:08:37.436 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.436 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:37.436 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.437 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.437 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.437 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:71136 3.6 e43/26 3:64cb7acb:::benchmark_data_smithi074_35447_object8891:head v 43'4744, mlcod=43'4739)", 2020-08-07T15:08:37.437 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:08:37.437 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 73.035699631, 2020-08-07T15:08:37.438 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26135482999999998, 2020-08-07T15:08:37.438 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:37.438 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:37.438 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:37.438 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.439 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:37.439 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:08:37.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:37.439 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.445 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.446 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:37.446 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:08:37.446 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3702999999999997e-05 2020-08-07T15:08:37.446 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.447 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:37.447 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318667+0000", 2020-08-07T15:08:37.447 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3119999999999999e-06 2020-08-07T15:08:37.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:37.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318674+0000", 2020-08-07T15:08:37.448 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.255173292 2020-08-07T15:08:37.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.449 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.449 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:37.450 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.573847+0000", 2020-08-07T15:08:37.450 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1140000000000002e-06 2020-08-07T15:08:37.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.450 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:37.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.573853+0000", 2020-08-07T15:08:37.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00057603799999999996 2020-08-07T15:08:37.451 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.451 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.451 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:37.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.574429+0000", 2020-08-07T15:08:37.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4776e-05 2020-08-07T15:08:37.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:37.462 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.574444+0000", 2020-08-07T15:08:37.469 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054577080000000004 2020-08-07T15:08:37.469 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.469 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.470 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:37.492 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.579901+0000", 2020-08-07T15:08:37.492 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6886999999999998e-05 2020-08-07T15:08:37.494 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.495 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.495 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:37.495 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.579968+0000", 2020-08-07T15:08:37.502 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26135482999999998 2020-08-07T15:08:37.502 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.502 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:37.502 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.503 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.503 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.503 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:81544 3.7 e48/20 3:e9bb655d:::benchmark_data_smithi074_35447_object10192:head v 48'4944, mlcod=48'4935)", 2020-08-07T15:08:37.503 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:30.902099+0000", 2020-08-07T15:08:37.503 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 66.452214671999997, 2020-08-07T15:08:37.504 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20669163500000001, 2020-08-07T15:08:37.505 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:37.505 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:37.506 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:37.506 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.506 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:37.506 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.902099+0000", 2020-08-07T15:08:37.507 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:37.507 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.507 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.508 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:37.508 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.902099+0000", 2020-08-07T15:08:37.508 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1317999999999999e-05 2020-08-07T15:08:37.509 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.509 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.509 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:37.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.902150+0000", 2020-08-07T15:08:37.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2079999999999997e-06 2020-08-07T15:08:37.510 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.510 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:37.510 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.902156+0000", 2020-08-07T15:08:37.510 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20264375100000001 2020-08-07T15:08:37.511 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:37.511 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.104800+0000", 2020-08-07T15:08:37.511 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2390000000000002e-06 2020-08-07T15:08:37.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.512 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.512 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:37.512 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.104805+0000", 2020-08-07T15:08:37.512 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3696000000000003e-05 2020-08-07T15:08:37.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.513 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.513 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:37.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.104849+0000", 2020-08-07T15:08:37.513 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4179e-05 2020-08-07T15:08:37.514 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.514 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:37.514 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.104873+0000", 2020-08-07T15:08:37.514 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038653839999999999 2020-08-07T15:08:37.515 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.515 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.515 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:37.515 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.108738+0000", 2020-08-07T15:08:37.515 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1860000000000002e-05 2020-08-07T15:08:37.516 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.516 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.516 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:37.516 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.108790+0000", 2020-08-07T15:08:37.516 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20669163500000001 2020-08-07T15:08:37.517 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.517 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:37.517 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.517 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.517 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.518 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:85696 3.f e51/20 3:f99fcea4:::benchmark_data_smithi074_35447_object10711:head v 51'5688, mlcod=51'5679)", 2020-08-07T15:08:37.518 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:08:37.518 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 63.701708218, 2020-08-07T15:08:37.519 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49655717300000002, 2020-08-07T15:08:37.519 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:37.519 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:37.519 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:37.519 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:37.520 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:08:37.520 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:37.520 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.520 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.521 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:37.521 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:08:37.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9543000000000002e-05 2020-08-07T15:08:37.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.521 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.522 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:37.522 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652665+0000", 2020-08-07T15:08:37.522 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8700000000000003e-06 2020-08-07T15:08:37.522 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.522 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.523 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:37.523 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652671+0000", 2020-08-07T15:08:37.523 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49579595799999998 2020-08-07T15:08:37.523 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.523 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.523 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:37.524 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148467+0000", 2020-08-07T15:08:37.524 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2509999999999997e-06 2020-08-07T15:08:37.524 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.524 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.524 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:37.525 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148475+0000", 2020-08-07T15:08:37.525 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5603999999999998e-05 2020-08-07T15:08:37.525 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.525 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.525 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:37.526 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148530+0000", 2020-08-07T15:08:37.526 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5486999999999998e-05 2020-08-07T15:08:37.526 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.526 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.526 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:37.527 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148556+0000", 2020-08-07T15:08:37.527 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00055666600000000004 2020-08-07T15:08:37.527 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.527 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.527 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:37.528 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.149112+0000", 2020-08-07T15:08:37.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9793999999999998e-05 2020-08-07T15:08:37.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.528 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:37.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.149162+0000", 2020-08-07T15:08:37.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49655717300000002 2020-08-07T15:08:37.529 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.529 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:37.529 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.530 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.530 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.530 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:86792 3.f e52/20 3:fc5386ee:::benchmark_data_smithi074_35447_object10848:head v 52'5752, mlcod=52'5743)", 2020-08-07T15:08:37.531 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:08:37.531 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 62.951862443000003, 2020-08-07T15:08:37.531 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.295270005, 2020-08-07T15:08:37.531 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:37.531 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:37.532 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:37.532 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.532 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:37.532 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:08:37.532 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:37.533 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.533 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.533 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:37.533 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:08:37.533 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9712000000000001e-05 2020-08-07T15:08:37.534 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.534 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.534 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:37.534 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402500+0000", 2020-08-07T15:08:37.534 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6789999999999999e-06 2020-08-07T15:08:37.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:37.535 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402507+0000", 2020-08-07T15:08:37.535 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29317074999999998 2020-08-07T15:08:37.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.536 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:37.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695678+0000", 2020-08-07T15:08:37.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8120000000000002e-06 2020-08-07T15:08:37.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.537 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.537 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:37.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695685+0000", 2020-08-07T15:08:37.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1270999999999998e-05 2020-08-07T15:08:37.537 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.538 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.538 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:37.538 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695726+0000", 2020-08-07T15:08:37.538 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2739e-05 2020-08-07T15:08:37.538 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.539 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.539 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:37.539 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695749+0000", 2020-08-07T15:08:37.539 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019333899999999999 2020-08-07T15:08:37.539 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.540 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.540 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:37.540 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.697682+0000", 2020-08-07T15:08:37.540 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8652000000000002e-05 2020-08-07T15:08:37.540 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.541 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:37.541 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.697721+0000", 2020-08-07T15:08:37.541 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.295270005 2020-08-07T15:08:37.541 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.542 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:37.542 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.542 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.543 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.543 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:87088 3.6 e52/26 3:6b0320df:::benchmark_data_smithi074_35447_object10885:head v 52'5912, mlcod=52'5902)", 2020-08-07T15:08:37.543 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:08:37.543 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 62.387629203000003, 2020-08-07T15:08:37.543 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29522189799999998, 2020-08-07T15:08:37.544 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:37.544 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:37.544 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:37.544 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.544 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:37.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:08:37.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:37.545 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.545 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:37.546 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:08:37.546 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0751000000000002e-05 2020-08-07T15:08:37.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.546 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.546 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:37.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966735+0000", 2020-08-07T15:08:37.547 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3080000000000004e-06 2020-08-07T15:08:37.547 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.547 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:37.548 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966741+0000", 2020-08-07T15:08:37.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29365034699999998 2020-08-07T15:08:37.548 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.549 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.549 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:37.549 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260391+0000", 2020-08-07T15:08:37.549 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2720000000000003e-06 2020-08-07T15:08:37.549 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.550 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.551 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:37.551 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260398+0000", 2020-08-07T15:08:37.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8819000000000005e-05 2020-08-07T15:08:37.551 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.551 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.554 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:37.554 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260476+0000", 2020-08-07T15:08:37.554 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4958999999999999e-05 2020-08-07T15:08:37.554 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.554 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.555 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:37.555 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260501+0000", 2020-08-07T15:08:37.555 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001353641 2020-08-07T15:08:37.555 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.555 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:37.556 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.261855+0000", 2020-08-07T15:08:37.556 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0800999999999997e-05 2020-08-07T15:08:37.556 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.557 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:37.557 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.261906+0000", 2020-08-07T15:08:37.557 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29522189799999998 2020-08-07T15:08:37.557 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.557 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:37.558 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.558 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.558 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.558 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:93920 3.6 e56/26 3:631393d3:::benchmark_data_smithi074_35447_object11739:head v 56'6352, mlcod=55'6342)", 2020-08-07T15:08:37.558 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:08:37.559 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 58.027513771999999, 2020-08-07T15:08:37.559 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39765404300000001, 2020-08-07T15:08:37.559 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:37.559 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:37.559 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:37.559 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.560 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:37.560 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:08:37.560 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:37.560 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.560 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.561 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:37.561 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:08:37.561 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3035999999999998e-05 2020-08-07T15:08:37.561 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.561 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.562 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:37.562 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326852+0000", 2020-08-07T15:08:37.562 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5379999999999999e-06 2020-08-07T15:08:37.562 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.562 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.563 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:37.563 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326859+0000", 2020-08-07T15:08:37.563 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.393917568 2020-08-07T15:08:37.564 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.564 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:37.564 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.720777+0000", 2020-08-07T15:08:37.565 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2430000000000005e-06 2020-08-07T15:08:37.565 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.565 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.565 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:37.565 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.720785+0000", 2020-08-07T15:08:37.566 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00065658700000000003 2020-08-07T15:08:37.566 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.566 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.566 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:37.566 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.721441+0000", 2020-08-07T15:08:37.567 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8585000000000002e-05 2020-08-07T15:08:37.567 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.567 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.567 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:37.568 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.721470+0000", 2020-08-07T15:08:37.568 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00293349 2020-08-07T15:08:37.568 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.568 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.568 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:37.569 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.724403+0000", 2020-08-07T15:08:37.569 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9996e-05 2020-08-07T15:08:37.569 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.569 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.569 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:37.570 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.724453+0000", 2020-08-07T15:08:37.570 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39765404300000001 2020-08-07T15:08:37.570 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.570 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:37.570 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.571 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.571 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.571 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:94384 3.3 e56/20 3:ce533213:::benchmark_data_smithi074_35447_object11797:head v 56'5488, mlcod=56'5479)", 2020-08-07T15:08:37.571 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.637245+0000", 2020-08-07T15:08:37.571 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.717068267999998, 2020-08-07T15:08:37.572 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.247824459, 2020-08-07T15:08:37.572 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:37.572 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:37.572 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:37.572 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.572 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:37.573 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637245+0000", 2020-08-07T15:08:37.573 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:37.573 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.573 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.574 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:37.574 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637245+0000", 2020-08-07T15:08:37.574 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0782999999999997e-05 2020-08-07T15:08:37.575 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.575 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:37.575 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637306+0000", 2020-08-07T15:08:37.575 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8699999999999992e-06 2020-08-07T15:08:37.576 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.576 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.576 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:37.576 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637314+0000", 2020-08-07T15:08:37.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24469571800000001 2020-08-07T15:08:37.577 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.577 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.577 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:37.577 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882009+0000", 2020-08-07T15:08:37.577 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8510000000000004e-06 2020-08-07T15:08:37.578 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.578 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.578 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:37.578 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882015+0000", 2020-08-07T15:08:37.579 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6708999999999997e-05 2020-08-07T15:08:37.579 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.579 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.579 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:37.579 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882052+0000", 2020-08-07T15:08:37.580 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0280000000000001e-05 2020-08-07T15:08:37.580 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.580 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.580 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:37.580 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882082+0000", 2020-08-07T15:08:37.581 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002934767 2020-08-07T15:08:37.581 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.581 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.581 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:37.582 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.885017+0000", 2020-08-07T15:08:37.582 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2481000000000001e-05 2020-08-07T15:08:37.582 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.582 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.582 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:37.583 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.885069+0000", 2020-08-07T15:08:37.583 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.247824459 2020-08-07T15:08:37.583 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.583 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:37.583 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.584 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.584 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.584 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:104560 3.f e58/20 3:fb4383ae:::benchmark_data_smithi074_35447_object13069:head v 58'6888, mlcod=58'6878)", 2020-08-07T15:08:37.584 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:46.301207+0000", 2020-08-07T15:08:37.585 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 51.053106450999998, 2020-08-07T15:08:37.585 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23687350400000001, 2020-08-07T15:08:37.585 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:37.585 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:37.586 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:37.586 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.586 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:37.586 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301207+0000", 2020-08-07T15:08:37.586 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:37.587 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.587 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.587 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:37.587 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301207+0000", 2020-08-07T15:08:37.587 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7945000000000003e-05 2020-08-07T15:08:37.588 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.588 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:37.588 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301255+0000", 2020-08-07T15:08:37.588 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2030000000000002e-06 2020-08-07T15:08:37.589 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.589 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:37.589 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301259+0000", 2020-08-07T15:08:37.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23565420500000001 2020-08-07T15:08:37.590 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.590 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.590 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:37.590 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.536913+0000", 2020-08-07T15:08:37.590 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2290000000000003e-06 2020-08-07T15:08:37.591 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.591 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.591 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:37.591 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.536921+0000", 2020-08-07T15:08:37.591 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0955e-05 2020-08-07T15:08:37.592 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.592 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.592 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:37.592 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.537002+0000", 2020-08-07T15:08:37.592 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8977999999999999e-05 2020-08-07T15:08:37.592 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.593 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.593 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:37.593 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.537031+0000", 2020-08-07T15:08:37.593 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00099831000000000008 2020-08-07T15:08:37.593 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:37.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.538030+0000", 2020-08-07T15:08:37.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0679e-05 2020-08-07T15:08:37.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.605 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.605 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:37.605 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.538080+0000", 2020-08-07T15:08:37.606 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23687350400000001 2020-08-07T15:08:37.606 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.606 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:37.606 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.606 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.607 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.607 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:116064 3.6 e64/26 3:6cc03ffa:::benchmark_data_smithi074_35447_object14507:head v 64'7632, mlcod=64'7623)", 2020-08-07T15:08:37.607 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:08:37.607 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 40.991552368999997, 2020-08-07T15:08:37.607 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25728607799999997, 2020-08-07T15:08:37.607 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:37.608 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:37.608 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:37.608 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.608 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:37.608 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:08:37.609 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:37.609 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.609 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.609 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:37.609 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:08:37.610 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6042000000000003e-05 2020-08-07T15:08:37.610 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.610 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.610 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:37.610 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362817+0000", 2020-08-07T15:08:37.611 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9399999999999999e-06 2020-08-07T15:08:37.611 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.611 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.611 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:37.611 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362823+0000", 2020-08-07T15:08:37.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.254635583 2020-08-07T15:08:37.612 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.612 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.612 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:37.612 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617458+0000", 2020-08-07T15:08:37.613 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5100000000000004e-06 2020-08-07T15:08:37.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.613 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:37.614 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617465+0000", 2020-08-07T15:08:37.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4439999999999999e-05 2020-08-07T15:08:37.614 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.614 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.615 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:37.615 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617539+0000", 2020-08-07T15:08:37.615 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3527e-05 2020-08-07T15:08:37.615 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.615 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.616 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:37.616 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617563+0000", 2020-08-07T15:08:37.616 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024301069999999999 2020-08-07T15:08:37.616 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.616 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.617 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:37.617 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.619993+0000", 2020-08-07T15:08:37.617 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3928999999999998e-05 2020-08-07T15:08:37.618 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.618 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.618 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:37.618 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.620047+0000", 2020-08-07T15:08:37.618 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25728607799999997 2020-08-07T15:08:37.619 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.619 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:37.619 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.619 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.619 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.620 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:123096 3.f e68/20 3:fe990d4f:::benchmark_data_smithi074_35447_object15386:head v 68'8120, mlcod=68'8112)", 2020-08-07T15:08:37.620 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:01.823387+0000", 2020-08-07T15:08:37.620 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 35.530925981000003, 2020-08-07T15:08:37.620 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.196095347, 2020-08-07T15:08:37.620 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:37.621 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:37.621 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:37.621 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.621 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:37.621 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:01.823387+0000", 2020-08-07T15:08:37.621 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:37.622 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.622 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.622 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:37.622 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:01.823387+0000", 2020-08-07T15:08:37.622 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1274999999999999e-05 2020-08-07T15:08:37.623 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.623 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.623 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:37.623 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:01.823448+0000", 2020-08-07T15:08:37.623 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5830000000000003e-06 2020-08-07T15:08:37.624 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.625 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.626 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:37.626 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:01.823455+0000", 2020-08-07T15:08:37.626 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19422482299999999 2020-08-07T15:08:37.626 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.627 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:37.627 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.017680+0000", 2020-08-07T15:08:37.627 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8530000000000003e-06 2020-08-07T15:08:37.627 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.628 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:37.628 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.017687+0000", 2020-08-07T15:08:37.628 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6105e-05 2020-08-07T15:08:37.629 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.629 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.629 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:37.629 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.017733+0000", 2020-08-07T15:08:37.629 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2348e-05 2020-08-07T15:08:37.629 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.630 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.630 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:37.630 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.017755+0000", 2020-08-07T15:08:37.630 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0016718480000000001 2020-08-07T15:08:37.630 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.631 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.631 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:37.631 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.019427+0000", 2020-08-07T15:08:37.631 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5512e-05 2020-08-07T15:08:37.631 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.632 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.632 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:37.632 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:02.019483+0000", 2020-08-07T15:08:37.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.196095347 2020-08-07T15:08:37.633 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.633 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:37.634 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.634 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.634 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.634 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:127528 3.6 e70/26 3:6d26a4f4:::benchmark_data_smithi074_35447_object15940:head v 70'8264, mlcod=70'8256)", 2020-08-07T15:08:37.635 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:04.535579+0000", 2020-08-07T15:08:37.635 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 32.818734319999997, 2020-08-07T15:08:37.635 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22956602400000001, 2020-08-07T15:08:37.635 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:37.636 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:37.636 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:37.636 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.636 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:37.637 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.535579+0000", 2020-08-07T15:08:37.637 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:37.637 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.637 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.638 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:37.638 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.535579+0000", 2020-08-07T15:08:37.638 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0343999999999997e-05 2020-08-07T15:08:37.638 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.638 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.638 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:37.639 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.535639+0000", 2020-08-07T15:08:37.648 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5889999999999999e-06 2020-08-07T15:08:37.649 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.652 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.652 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:37.652 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.535646+0000", 2020-08-07T15:08:37.652 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22504058599999999 2020-08-07T15:08:37.654 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.655 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.655 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:37.655 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.760686+0000", 2020-08-07T15:08:37.655 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7790000000000003e-06 2020-08-07T15:08:37.655 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.656 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.656 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:37.656 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.760691+0000", 2020-08-07T15:08:37.656 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2592999999999997e-05 2020-08-07T15:08:37.663 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.663 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.664 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:37.664 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.760734+0000", 2020-08-07T15:08:37.664 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3947999999999998e-05 2020-08-07T15:08:37.664 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.665 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:37.665 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.760758+0000", 2020-08-07T15:08:37.665 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043316020000000004 2020-08-07T15:08:37.665 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.665 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.666 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:37.666 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.765089+0000", 2020-08-07T15:08:37.666 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5583e-05 2020-08-07T15:08:37.666 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.667 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:37.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.765145+0000", 2020-08-07T15:08:37.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22956602400000001 2020-08-07T15:08:37.667 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.668 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:37.668 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.668 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:145801 3.6 e75/26 3:69e4cb22:::benchmark_data_smithi074_35447_object16192:head v 75'9405, mlcod=75'9401)", 2020-08-07T15:08:37.669 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:08:37.669 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 22.682557076999998, 2020-08-07T15:08:37.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639, 2020-08-07T15:08:37.669 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:37.669 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:37.670 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:37.670 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.670 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:37.671 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:08:37.671 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:37.671 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.671 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.671 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:37.672 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:08:37.672 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6511000000000002e-05 2020-08-07T15:08:37.672 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.672 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.672 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:37.673 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671803+0000", 2020-08-07T15:08:37.673 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7190000000000004e-06 2020-08-07T15:08:37.673 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.673 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.673 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:37.674 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671809+0000", 2020-08-07T15:08:37.674 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74210928300000001 2020-08-07T15:08:37.674 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.674 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.674 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:37.675 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413919+0000", 2020-08-07T15:08:37.675 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1759999999999999e-05 2020-08-07T15:08:37.675 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.675 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.675 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:37.676 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413930+0000", 2020-08-07T15:08:37.676 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000108601 2020-08-07T15:08:37.676 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.676 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:37.677 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414039+0000", 2020-08-07T15:08:37.677 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9948999999999999e-05 2020-08-07T15:08:37.677 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.678 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.678 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:37.678 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414089+0000", 2020-08-07T15:08:37.678 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00079656499999999995 2020-08-07T15:08:37.678 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.679 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:37.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414885+0000", 2020-08-07T15:08:37.679 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2250999999999998e-05 2020-08-07T15:08:37.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.680 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:37.680 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414938+0000", 2020-08-07T15:08:37.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639 2020-08-07T15:08:37.681 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.681 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:37.682 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.682 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.682 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.682 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:8616 4.6 e86/81 4:67238642:::benchmark_data_smithi074_44586_object1076:head v 86'520, mlcod=86'513)", 2020-08-07T15:08:37.683 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:31.984864+0000", 2020-08-07T15:08:37.683 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.3694490789999998, 2020-08-07T15:08:37.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20134179199999999, 2020-08-07T15:08:37.683 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:37.683 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:37.683 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:37.684 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.684 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:37.684 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.984864+0000", 2020-08-07T15:08:37.685 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:37.685 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.685 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.685 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:37.685 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.984864+0000", 2020-08-07T15:08:37.686 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0510000000000003e-05 2020-08-07T15:08:37.686 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.686 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.686 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:37.687 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.984915+0000", 2020-08-07T15:08:37.687 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4810000000000003e-06 2020-08-07T15:08:37.687 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.687 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.687 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:37.687 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.984921+0000", 2020-08-07T15:08:37.688 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19571144600000001 2020-08-07T15:08:37.688 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.688 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.688 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:37.688 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.180633+0000", 2020-08-07T15:08:37.689 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5609999999999998e-06 2020-08-07T15:08:37.689 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.689 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.689 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:37.689 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.180638+0000", 2020-08-07T15:08:37.690 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000145508 2020-08-07T15:08:37.690 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.690 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.690 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:37.690 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.180784+0000", 2020-08-07T15:08:37.691 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3981e-05 2020-08-07T15:08:37.691 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.691 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.691 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:37.691 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.180808+0000", 2020-08-07T15:08:37.692 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005345565 2020-08-07T15:08:37.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.692 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:37.692 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.186153+0000", 2020-08-07T15:08:37.693 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.274e-05 2020-08-07T15:08:37.693 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.693 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.693 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:37.693 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.186206+0000", 2020-08-07T15:08:37.694 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20134179199999999 2020-08-07T15:08:37.694 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.694 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:37.694 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.694 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.695 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:9112 4.6 e86/81 4:6004be0b:::benchmark_data_smithi074_44586_object1138:head v 86'544, mlcod=86'533)", 2020-08-07T15:08:37.695 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:32.296890+0000", 2020-08-07T15:08:37.695 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.0574236680000002, 2020-08-07T15:08:37.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26263388100000001, 2020-08-07T15:08:37.696 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:37.696 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:37.696 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:37.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.696 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:37.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296890+0000", 2020-08-07T15:08:37.697 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:37.697 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:37.698 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296890+0000", 2020-08-07T15:08:37.698 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9907999999999997e-05 2020-08-07T15:08:37.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.698 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.698 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:37.699 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296939+0000", 2020-08-07T15:08:37.699 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7630000000000003e-06 2020-08-07T15:08:37.699 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.699 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.700 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:37.700 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296946+0000", 2020-08-07T15:08:37.700 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26058471900000002 2020-08-07T15:08:37.700 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.701 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:37.701 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557531+0000", 2020-08-07T15:08:37.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5270000000000004e-06 2020-08-07T15:08:37.701 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.702 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:37.702 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557537+0000", 2020-08-07T15:08:37.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2843000000000003e-05 2020-08-07T15:08:37.702 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.702 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.702 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:37.703 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557580+0000", 2020-08-07T15:08:37.703 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3833999999999999e-05 2020-08-07T15:08:37.703 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.703 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:37.704 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557604+0000", 2020-08-07T15:08:37.704 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001875196 2020-08-07T15:08:37.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.704 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:37.705 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.559479+0000", 2020-08-07T15:08:37.705 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4091000000000002e-05 2020-08-07T15:08:37.705 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.705 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.705 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:37.706 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.559523+0000", 2020-08-07T15:08:37.706 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26263388100000001 2020-08-07T15:08:37.706 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.706 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:37.706 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.707 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.707 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:13784 4.6 e88/81 4:66f64b92:::benchmark_data_smithi074_44586_object1722:head v 88'832, mlcod=88'827)", 2020-08-07T15:08:37.707 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:35.137315+0000", 2020-08-07T15:08:37.707 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 2.2169979350000002, 2020-08-07T15:08:37.707 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32459016899999998, 2020-08-07T15:08:37.708 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:37.708 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:37.708 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:37.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.708 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:37.709 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137315+0000", 2020-08-07T15:08:37.709 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:37.709 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.709 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.709 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:37.710 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137315+0000", 2020-08-07T15:08:37.710 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0362999999999998e-05 2020-08-07T15:08:37.710 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.710 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.710 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:37.711 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137376+0000", 2020-08-07T15:08:37.711 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7020000000000004e-06 2020-08-07T15:08:37.711 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:37.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137382+0000", 2020-08-07T15:08:37.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32172605500000001 2020-08-07T15:08:37.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:37.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459108+0000", 2020-08-07T15:08:37.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7710000000000003e-06 2020-08-07T15:08:37.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.721 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:37.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459114+0000", 2020-08-07T15:08:37.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4297e-05 2020-08-07T15:08:37.739 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.743 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:37.744 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459158+0000", 2020-08-07T15:08:37.744 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5058999999999998e-05 2020-08-07T15:08:37.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.745 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:37.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459183+0000", 2020-08-07T15:08:37.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026624740000000002 2020-08-07T15:08:37.745 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.752 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:37.752 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.461846+0000", 2020-08-07T15:08:37.753 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9447999999999997e-05 2020-08-07T15:08:37.754 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:37.754 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:37.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:37.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.461905+0000", 2020-08-07T15:08:37.755 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32459016899999998 2020-08-07T15:08:37.755 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.755 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:37.756 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.756 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:37.756 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:37.756 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:37.757 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:08:37.759 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:08:37.760 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:37.756+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:37.761 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:08:37.861 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:37.858+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:37.961 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:37.960+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:38.018 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:38.019 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:08:38.019 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:38.019 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:18328 4.7 4:e93bf477:::benchmark_data_smithi074_44586_object2290:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e91)", 2020-08-07T15:08:38.019 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:37.891665+0000", 2020-08-07T15:08:38.020 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.11218046, 2020-08-07T15:08:38.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.112200148, 2020-08-07T15:08:38.020 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:38.020 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:38.021 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:38.021 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:08:38.021 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:38.021 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18328 2020-08-07T15:08:38.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:38.023 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:38.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:38.023 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:38.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:37.891665+0000", 2020-08-07T15:08:38.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:38.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:38.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:38.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:38.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:37.891665+0000", 2020-08-07T15:08:38.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1940000000000001e-05 2020-08-07T15:08:38.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:38.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:38.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:38.026 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:37.891717+0000", 2020-08-07T15:08:38.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8289999999999996e-06 2020-08-07T15:08:38.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:38.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:38.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:38.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:37.891723+0000", 2020-08-07T15:08:38.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0720000000000001e-05 2020-08-07T15:08:38.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:38.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:38.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:38.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:37.891734+0000", 2020-08-07T15:08:38.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6759999999999998e-06 2020-08-07T15:08:38.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:38.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:38.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:38.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:37.891738+0000", 2020-08-07T15:08:38.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024091730000000001 2020-08-07T15:08:38.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:38.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:38.030 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:38.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:37.894148+0000", 2020-08-07T15:08:38.030 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6701999999999997e-05 2020-08-07T15:08:38.030 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:38.030 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:38.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:38.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:37.894244+0000", 2020-08-07T15:08:38.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017677 2020-08-07T15:08:38.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:38.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:38.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,5", 2020-08-07T15:08:38.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:37.894421+0000", 2020-08-07T15:08:38.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001774723 2020-08-07T15:08:38.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:38.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:38.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:38.033 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:37.896196+0000", 2020-08-07T15:08:38.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6499999999999999e-07 2020-08-07T15:08:38.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:38.034 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:38.034 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:38.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:37.896196+0000", 2020-08-07T15:08:38.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5655000000000002e-05 2020-08-07T15:08:38.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:38.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:38.037 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:38.037 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:37.896252+0000", 2020-08-07T15:08:38.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00022618399999999999 2020-08-07T15:08:38.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:38.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:38.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:38.038 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:37.896478+0000", 2020-08-07T15:08:38.038 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004813037 2020-08-07T15:08:38.038 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:38.038 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:38.039 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:38.039 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:38.039 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:08:38.039 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 1 2020-08-07T15:08:38.039 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:38.040 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:08:38.040 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:08:38.054 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:08:38.072 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:38.071+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:38.087 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:08:38.123 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:08:38.165 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:08:38.175 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:38.174+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:38.186 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:08:38.186 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:08:38.186 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_1 2020-08-07T15:08:38.187 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:08:38.202 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:08:38.240 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:38.241 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:38.241 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:08:38.270 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:08:38.279 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 12 16 18943 18927 12.3214 13.3984 0.00572933 0.0101379 2020-08-07T15:08:38.279 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:08:38.279 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:38.277+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:38.305 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:08:38.305 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:08:38.305 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:08:38.305 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:08:38.326 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:08:38.326 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:08:38.326 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:08:38.326 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:08:38.382 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:38.379+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:38.458 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:38.458 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:38.466 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:08:38.527 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:38.519+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:38.583 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:38.582+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:38.649 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:08:38.658 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:08:38.669 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2020-08-07T15:08:37.627439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":127926272,"num_objects":1954,"num_object_clones":0,"num_object_copies":5862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15616,"num_write_kb":124928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":595,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":123,"acting":123,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8748740,"kb_used_data":360068,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746225980,"statfs":{"total":773094113280,"available":764135403520,"internally_reserved":8589934592,"allocated":368709632,"data_stored":365097285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,9,6,2,1,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":59072512,"num_objects":901,"num_object_clones":0,"num_object_copies":2703,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7211,"num_write_kb":57688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.471405"},"pg_stats":[{"pgid":"1.14","version":"16'2","reported_seq":"71","reported_epoch":"90","state":"active+clean","last_fresh":"2020-08-07T15:08:36.623931+0000","last_change":"2020-08-07T15:08:36.623931+0000","last_active":"2020-08-07T15:08:36.623931+0000","last_peered":"2020-08-07T15:08:36.623931+0000","last_clean":"2020-08-07T15:08:36.623931+0000","last_became_active":"2020-08-07T15:08:36.623420+0000","last_became_peered":"2020-08-07T15:08:36.623420+0000","last_unstale":"2020-08-07T15:08:36.623931+0000","last_undegraded":"2020-08-07T15:08:36.623931+0000","last_fullsized":"2020-08-07T15:08:36.623931+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:34.653843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:34.653843+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.13","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:08:35.597707+0000","last_change":"2020-08-07T15:08:35.597707+0000","last_active":"2020-08-07T15:08:35.597707+0000","last_peered":"2020-08-07T15:08:35.597707+0000","last_clean":"2020-08-07T15:08:35.597707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:08:35.597707+0000","last_undegraded":"2020-08-07T15:08:35.597707+0000","last_fullsized":"2020-08-07T15:08:35.597707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:35.597707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:35.597707+0000","last_clean_scrub_stamp":"2020-08-07T15:08:35.597707+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"128","reported_epoch":"90","state":"active+clean","last_fresh":"2020-08-07T15:08:36.633202+0000","last_change":"2020-08-07T15:08:36.633202+0000","last_active":"2020-08-07T15:08:36.633202+0000","last_peered":"2020-08-07T15:08:36.633202+0000","last_clean":"2020-08-07T15:08:36.633202+0000","last_became_active":"2020-08-07T15:08:36.632683+0000","last_became_peered":"2020-08-07T15:08:36.632683+0000","last_unstale":"2020-08-07T15:08:36.633202+0000","last_undegraded":"2020-08-07T15:08:36.633202+0000","last_fullsized":"2020-08-07T15:08:36.633202+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.358798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.358798+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"78","reported_epoch":"90","state":"active+clean","last_fresh":"2020-08-07T15:08:36.621798+0000","last_change":"2020-08-07T15:08:36.621798+0000","last_active":"2020-08-07T15:08:36.621798+0000","last_peered":"2020-08-07T15:08:36.621798+0000","last_clean":"2020-08-07T15:08:36.621798+0000","last_became_active":"2020-08-07T15:08:36.621303+0000","last_became_peered":"2020-08-07T15:08:36.621303+0000","last_unstale":"2020-08-07T15:08:36.621798+0000","last_undegraded":"2020-08-07T15:08:36.621798+0000","last_fullsized":"2020-08-07T15:08:36.621798+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:15.279411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:15.279411+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"113","reported_epoch":"90","state":"active+clean","last_fresh":"2020-08-07T15:08:36.674808+0000","last_change":"2020-08-07T15:08:36.674808+0000","last_active":"2020-08-07T15:08:36.674808+0000","last_peered":"2020-08-07T15:08:36.674808+0000","last_clean":"2020-08-07T15:08:36.674808+0000","last_became_active":"2020-08-07T15:08:36.674329+0000","last_became_peered":"2020-08-07T15:08:36.674329+0000","last_unstale":"2020-08-07T15:08:36.674808+0000","last_undegraded":"2020-08-07T15:08:36.674808+0000","last_fullsized":"2020-08-07T15:08:36.674808+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:32.460945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:32.460945+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"80","reported_epoch":"85","state":"active+clean","last_fresh":"2020-08-07T15:08:27.566090+0000","last_change":"2020-08-07T15:08:11.840861+0000","last_active":"2020-08-07T15:08:27.566090+0000","last_peered":"2020-08-07T15:08:27.566090+0000","last_clean":"2020-08-07T15:08:27.566090+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:08:27.566090+0000","last_undegraded":"2020-08-07T15:08:27.566090+0000","last_fullsized":"2020-08-07T15:08:27.566090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:11.840807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:11.840807+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"72","reported_epoch":"90","state":"active+clean","last_fresh":"2020-08-07T15:08:36.615847+0000","last_change":"2020-08-07T15:08:36.615847+0000","last_active":"2020-08-07T15:08:36.615847+0000","last_peered":"2020-08-07T15:08:36.615847+0000","last_clean":"2020-08-07T15:08:36.615847+0000","last_became_active":"2020-08-07T15:08:36.615358+0000","last_became_peered":"2020-08-07T15:08:36.615358+0000","last_unstale":"2020-08-07T15:08:36.615847+0000","last_undegraded":"2020-08-07T15:08:36.615847+0000","last_fullsized":"2020-08-07T15:08:36.615847+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:34.653843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:34.653843+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"90'952","reported_seq":"969","reported_epoch":"90","state":"active+clean","last_fresh":"2020-08-07T15:08:36.690306+0000","last_change":"2020-08-07T15:08:24.550895+0000","last_active":"2020-08-07T15:08:36.690306+0000","last_peered":"2020-08-07T15:08:36.690306+0000","last_clean":"2020-08-07T15:08:36.690306+0000","last_became_active":"2020-08-07T15:08:24.550490+0000","last_became_peered":"2020-08-07T15:08:24.550490+0000","last_unstale":"2020-08-07T15:08:36.690306+0000","last_undegraded":"2020-08-07T15:08:36.690306+0000","last_fullsized":"2020-08-07T15:08:36.690306+0000","mapping_epoch":81,"log_start":"90'944","ondisk_log_start":"90'944","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7798784,"num_objects":119,"num_object_clones":0,"num_object_copies":357,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":952,"num_write_kb":7616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"84","reported_epoch":"89","state":"active+clean","last_fresh":"2020-08-07T15:08:35.611145+0000","last_change":"2020-08-07T15:07:34.423236+0000","last_active":"2020-08-07T15:08:35.611145+0000","last_peered":"2020-08-07T15:08:35.611145+0000","last_clean":"2020-08-07T15:08:35.611145+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:08:35.611145+0000","last_undegraded":"2020-08-07T15:08:35.611145+0000","last_fullsized":"2020-08-07T15:08:35.611145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:07:34.423115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:07:34.423115+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"67","reported_epoch":"90","state":"active+clean","last_fresh":"2020-08-07T15:08:36.625666+0000","last_change":"2020-08-07T15:08:36.625666+0000","last_active":"2020-08-07T15:08:36.625666+0000","last_peered":"2020-08-07T15:08:36.625666+0000","last_clean":"2020-08-07T15:08:36.625666+0000","last_became_active":"2020-08-07T15:08:36.625195+0000","last_became_peered":"2020-08-07T15:08:36.625195+0000","last_unstale":"2020-08-07T15:08:36.625666+0000","last_undegraded":"2020-08-07T15:08:36.625666+0000","last_fullsized":"2020-08-07T15:08:36.625666+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:15.585597+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":297,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"89'1080","reported_seq":"1097","reported_epoch":"89","state":"active+clean","last_fresh":"2020-08-07T15:08:36.597002+0000","last_change":"2020-08-07T15:08:24.554330+0000","last_active":"2020-08-07T15:08:36.597002+0000","last_peered":"2020-08-07T15:08:36.597002+0000","last_clean":"2020-08-07T15:08:36.597002+0000","last_became_active":"2020-08-07T15:08:24.553881+0000","last_became_peered":"2020-08-07T15:08:24.553881+0000","last_unstale":"2020-08-07T15:08:36.597002+0000","last_undegraded":"2020-08-07T15:08:36.597002+0000","last_fullsized":"2020-08-07T15:08:36.597002+0000","mapping_epoch":81,"log_start":"89'1076","ondisk_log_start":"89'1076","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8847360,"num_objects":135,"num_object_clones":0,"num_object_copies":405,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1080,"num_write_kb":8640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"84","reported_epoch":"89","state":"active+clean","last_fresh":"2020-08-07T15:08:35.610276+0000","last_change":"2020-08-07T15:08:27.906894+0000","last_active":"2020-08-07T15:08:35.610276+0000","last_peered":"2020-08-07T15:08:35.610276+0000","last_clean":"2020-08-07T15:08:35.610276+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:08:35.610276+0000","last_undegraded":"2020-08-07T15:08:35.610276+0000","last_fullsized":"2020-08-07T15:08:35.610276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:27.906822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:27.906822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"73","reported_epoch":"85","state":"active+clean","last_fresh":"2020-08-07T15:08:27.566008+0000","last_change":"2020-08-07T15:08:09.841524+0000","last_active":"2020-08-07T15:08:27.566008+0000","last_peered":"2020-08-07T15:08:27.566008+0000","last_clean":"2020-08-07T15:08:27.566008+0000","last_became_active":"2020-08-07T15:07:33.169518+0000","last_became_peered":"2020-08-07T15:07:33.169518+0000","last_unstale":"2020-08-07T15:08:27.566008+0000","last_undegraded":"2020-08-07T15:08:27.566008+0000","last_fullsized":"2020-08-07T15:08:27.566008+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:09.841428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:09.841428+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"90'984","reported_seq":"1001","reported_epoch":"90","state":"active+clean","last_fresh":"2020-08-07T15:08:36.703104+0000","last_change":"2020-08-07T15:08:24.549715+0000","last_active":"2020-08-07T15:08:36.703104+0000","last_peered":"2020-08-07T15:08:36.703104+0000","last_clean":"2020-08-07T15:08:36.703104+0000","last_became_active":"2020-08-07T15:08:24.549245+0000","last_became_peered":"2020-08-07T15:08:24.549245+0000","last_unstale":"2020-08-07T15:08:36.703104+0000","last_undegraded":"2020-08-07T15:08:36.703104+0000","last_fullsized":"2020-08-07T15:08:36.703104+0000","mapping_epoch":81,"log_start":"89'976","ondisk_log_start":"89'976","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8060928,"num_objects":123,"num_object_clones":0,"num_object_copies":369,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":984,"num_write_kb":7872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"84","reported_epoch":"89","state":"active+clean","last_fresh":"2020-08-07T15:08:35.610484+0000","last_change":"2020-08-07T15:08:13.867355+0000","last_active":"2020-08-07T15:08:35.610484+0000","last_peered":"2020-08-07T15:08:35.610484+0000","last_clean":"2020-08-07T15:08:35.610484+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:08:35.610484+0000","last_undegraded":"2020-08-07T15:08:35.610484+0000","last_fullsized":"2020-08-07T15:08:35.610484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:13.867148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:13.867148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"73","reported_epoch":"85","state":"active+clean","last_fresh":"2020-08-07T15:08:27.565965+0000","last_change":"2020-08-07T15:08:02.901198+0000","last_active":"2020-08-07T15:08:27.565965+0000","last_peered":"2020-08-07T15:08:27.565965+0000","last_clean":"2020-08-07T15:08:27.565965+0000","last_became_active":"2020-08-07T15:07:33.156569+0000","last_became_peered":"2020-08-07T15:07:33.156569+0000","last_unstale":"2020-08-07T15:08:27.565965+0000","last_undegraded":"2020-08-07T15:08:27.565965+0000","last_fullsized":"2020-08-07T15:08:27.565965+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:02.901115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:02.901115+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"89'984","reported_seq":"1001","reported_epoch":"89","state":"active+clean","last_fresh":"2020-08-07T15:08:36.541057+0000","last_change":"2020-08-07T15:08:24.557398+0000","last_active":"2020-08-07T15:08:36.541057+0000","last_peered":"2020-08-07T15:08:36.541057+0000","last_clean":"2020-08-07T15:08:36.541057+0000","last_became_active":"2020-08-07T15:08:24.557022+0000","last_became_peered":"2020-08-07T15:08:24.557022+0000","last_unstale":"2020-08-07T15:08:36.541057+0000","last_undegraded":"2020-08-07T15:08:36.541057+0000","last_fullsized":"2020-08-07T15:08:36.541057+0000","mapping_epoch":81,"log_start":"89'979","ondisk_log_start":"89'979","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8060928,"num_objects":123,"num_object_clones":0,"num_object_copies":369,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":984,"num_write_kb":7872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"113","reported_epoch":"90","state":"active+clean","last_fresh":"2020-08-07T15:08:36.627891+0000","last_change":"2020-08-07T15:08:36.627891+0000","last_active":"2020-08-07T15:08:36.627891+0000","last_peered":"2020-08-07T15:08:36.627891+0000","last_clean":"2020-08-07T15:08:36.627891+0000","last_became_active":"2020-08-07T15:08:36.627426+0000","last_became_peered":"2020-08-07T15:08:36.627426+0000","last_unstale":"2020-08-07T15:08:36.627891+0000","last_undegraded":"2020-08-07T15:08:36.627891+0000","last_fullsized":"2020-08-07T15:08:36.627891+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:32.460945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:32.460945+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"84","reported_epoch":"89","state":"active+clean","last_fresh":"2020-08-07T15:08:35.602039+0000","last_change":"2020-08-07T15:08:26.317430+0000","last_active":"2020-08-07T15:08:35.602039+0000","last_peered":"2020-08-07T15:08:35.602039+0000","last_clean":"2020-08-07T15:08:35.602039+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:08:35.602039+0000","last_undegraded":"2020-08-07T15:08:35.602039+0000","last_fullsized":"2020-08-07T15:08:35.602039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:26.317351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:26.317351+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"86'624","reported_seq":"637","reported_epoch":"86","state":"active+clean","last_fresh":"2020-08-07T15:08:32.705263+0000","last_change":"2020-08-07T15:08:24.552570+0000","last_active":"2020-08-07T15:08:32.705263+0000","last_peered":"2020-08-07T15:08:32.705263+0000","last_clean":"2020-08-07T15:08:32.705263+0000","last_became_active":"2020-08-07T15:08:24.552105+0000","last_became_peered":"2020-08-07T15:08:24.552105+0000","last_unstale":"2020-08-07T15:08:32.705263+0000","last_undegraded":"2020-08-07T15:08:32.705263+0000","last_fullsized":"2020-08-07T15:08:32.705263+0000","mapping_epoch":81,"log_start":"86'616","ondisk_log_start":"86'616","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5111808,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":624,"num_write_kb":4992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"84","reported_epoch":"89","state":"active+clean","last_fresh":"2020-08-07T15:08:35.608285+0000","last_change":"2020-08-07T15:07:36.864560+0000","last_active":"2020-08-07T15:08:35.608285+0000","last_peered":"2020-08-07T15:08:35.608285+0000","last_clean":"2020-08-07T15:08:35.608285+0000","last_became_active":"2020-08-07T15:06:23.223489+0000","last_became_peered":"2020-08-07T15:06:23.223489+0000","last_unstale":"2020-08-07T15:08:35.608285+0000","last_undegraded":"2020-08-07T15:08:35.608285+0000","last_fullsized":"2020-08-07T15:08:35.608285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:07:36.864483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.864483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"129","reported_epoch":"90","state":"active+clean","last_fresh":"2020-08-07T15:08:36.628812+0000","last_change":"2020-08-07T15:08:36.628812+0000","last_active":"2020-08-07T15:08:36.628812+0000","last_peered":"2020-08-07T15:08:36.628812+0000","last_clean":"2020-08-07T15:08:36.628812+0000","last_became_active":"2020-08-07T15:08:36.628365+0000","last_became_peered":"2020-08-07T15:08:36.628365+0000","last_unstale":"2020-08-07T15:08:36.628812+0000","last_undegraded":"2020-08-07T15:08:36.628812+0000","last_fullsized":"2020-08-07T15:08:36.628812+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.358798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.358798+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"90'1192","reported_seq":"1208","reported_epoch":"90","state":"active+clean","last_fresh":"2020-08-07T15:08:36.710262+0000","last_change":"2020-08-07T15:08:24.550789+0000","last_active":"2020-08-07T15:08:36.710262+0000","last_peered":"2020-08-07T15:08:36.710262+0000","last_clean":"2020-08-07T15:08:36.710262+0000","last_became_active":"2020-08-07T15:08:24.550222+0000","last_became_peered":"2020-08-07T15:08:24.550222+0000","last_unstale":"2020-08-07T15:08:36.710262+0000","last_undegraded":"2020-08-07T15:08:36.710262+0000","last_fullsized":"2020-08-07T15:08:36.710262+0000","mapping_epoch":81,"log_start":"90'1185","ondisk_log_start":"90'1185","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":9764864,"num_objects":149,"num_object_clones":0,"num_object_copies":447,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1192,"num_write_kb":9536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"84","reported_epoch":"89","state":"active+clean","last_fresh":"2020-08-07T15:08:35.601515+0000","last_change":"2020-08-07T15:08:30.339852+0000","last_active":"2020-08-07T15:08:35.601515+0000","last_peered":"2020-08-07T15:08:35.601515+0000","last_clean":"2020-08-07T15:08:35.601515+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:08:35.601515+0000","last_undegraded":"2020-08-07T15:08:35.601515+0000","last_fullsized":"2020-08-07T15:08:35.601515+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:08:30.339785+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"79","reported_epoch":"90","state":"active+clean","last_fresh":"2020-08-07T15:08:36.618625+0000","last_change":"2020-08-07T15:08:36.618625+0000","last_active":"2020-08-07T15:08:36.618625+0000","last_peered":"2020-08-07T15:08:36.618625+0000","last_clean":"2020-08-07T15:08:36.618625+0000","last_became_active":"2020-08-07T15:08:36.618164+0000","last_became_peered":"2020-08-07T15:08:36.618164+0000","last_unstale":"2020-08-07T15:08:36.618625+0000","last_undegraded":"2020-08-07T15:08:36.618625+0000","last_fullsized":"2020-08-07T15:08:36.618625+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:15.279411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:15.279411+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"89'1016","reported_seq":"1033","reported_epoch":"89","state":"active+clean","last_fresh":"2020-08-07T15:08:36.550599+0000","last_change":"2020-08-07T15:08:24.558242+0000","last_active":"2020-08-07T15:08:36.550599+0000","last_peered":"2020-08-07T15:08:36.550599+0000","last_clean":"2020-08-07T15:08:36.550599+0000","last_became_active":"2020-08-07T15:08:24.557354+0000","last_became_peered":"2020-08-07T15:08:24.557354+0000","last_unstale":"2020-08-07T15:08:36.550599+0000","last_undegraded":"2020-08-07T15:08:36.550599+0000","last_fullsized":"2020-08-07T15:08:36.550599+0000","mapping_epoch":81,"log_start":"89'1008","ondisk_log_start":"89'1008","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8323072,"num_objects":127,"num_object_clones":0,"num_object_copies":381,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1016,"num_write_kb":8128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"81","reported_epoch":"89","state":"active+clean","last_fresh":"2020-08-07T15:08:35.612641+0000","last_change":"2020-08-07T15:08:15.887617+0000","last_active":"2020-08-07T15:08:35.612641+0000","last_peered":"2020-08-07T15:08:35.612641+0000","last_clean":"2020-08-07T15:08:35.612641+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:08:35.612641+0000","last_undegraded":"2020-08-07T15:08:35.612641+0000","last_fullsized":"2020-08-07T15:08:35.612641+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:15.887513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:15.887513+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"90","reported_epoch":"85","state":"active+clean","last_fresh":"2020-08-07T15:08:27.566037+0000","last_change":"2020-08-07T15:07:51.357310+0000","last_active":"2020-08-07T15:08:27.566037+0000","last_peered":"2020-08-07T15:08:27.566037+0000","last_clean":"2020-08-07T15:08:27.566037+0000","last_became_active":"2020-08-07T15:07:51.356798+0000","last_became_peered":"2020-08-07T15:07:51.356798+0000","last_unstale":"2020-08-07T15:08:27.566037+0000","last_undegraded":"2020-08-07T15:08:27.566037+0000","last_fullsized":"2020-08-07T15:08:27.566037+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.927824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.927824+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"90'928","reported_seq":"945","reported_epoch":"90","state":"active+clean","last_fresh":"2020-08-07T15:08:36.678141+0000","last_change":"2020-08-07T15:08:24.552962+0000","last_active":"2020-08-07T15:08:36.678141+0000","last_peered":"2020-08-07T15:08:36.678141+0000","last_clean":"2020-08-07T15:08:36.678141+0000","last_became_active":"2020-08-07T15:08:24.552070+0000","last_became_peered":"2020-08-07T15:08:24.552070+0000","last_unstale":"2020-08-07T15:08:36.678141+0000","last_undegraded":"2020-08-07T15:08:36.678141+0000","last_fullsized":"2020-08-07T15:08:36.678141+0000","mapping_epoch":81,"log_start":"89'920","ondisk_log_start":"89'920","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":928,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"56","reported_epoch":"89","state":"active+clean","last_fresh":"2020-08-07T15:08:35.601627+0000","last_change":"2020-08-07T15:07:36.143434+0000","last_active":"2020-08-07T15:08:35.601627+0000","last_peered":"2020-08-07T15:08:35.601627+0000","last_clean":"2020-08-07T15:08:35.601627+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:08:35.601627+0000","last_undegraded":"2020-08-07T15:08:35.601627+0000","last_fullsized":"2020-08-07T15:08:35.601627+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.143353+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"90'1168","reported_seq":"1184","reported_epoch":"90","state":"active+clean","last_fresh":"2020-08-07T15:08:36.638307+0000","last_change":"2020-08-07T15:08:24.554063+0000","last_active":"2020-08-07T15:08:36.638307+0000","last_peered":"2020-08-07T15:08:36.638307+0000","last_clean":"2020-08-07T15:08:36.638307+0000","last_became_active":"2020-08-07T15:08:24.553668+0000","last_became_peered":"2020-08-07T15:08:24.553668+0000","last_unstale":"2020-08-07T15:08:36.638307+0000","last_undegraded":"2020-08-07T15:08:36.638307+0000","last_fullsized":"2020-08-07T15:08:36.638307+0000","mapping_epoch":81,"log_start":"89'1160","ondisk_log_start":"89'1160","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":9568256,"num_objects":146,"num_object_clones":0,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1168,"num_write_kb":9344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"56","reported_epoch":"89","state":"active+clean","last_fresh":"2020-08-07T15:08:35.601509+0000","last_change":"2020-08-07T15:07:33.177491+0000","last_active":"2020-08-07T15:08:35.601509+0000","last_peered":"2020-08-07T15:08:35.601509+0000","last_clean":"2020-08-07T15:08:35.601509+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:08:35.601509+0000","last_undegraded":"2020-08-07T15:08:35.601509+0000","last_fullsized":"2020-08-07T15:08:35.601509+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:33.177412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:33.177412+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"90'920","reported_seq":"937","reported_epoch":"90","state":"active+clean","last_fresh":"2020-08-07T15:08:36.696621+0000","last_change":"2020-08-07T15:08:24.550962+0000","last_active":"2020-08-07T15:08:36.696621+0000","last_peered":"2020-08-07T15:08:36.696621+0000","last_clean":"2020-08-07T15:08:36.696621+0000","last_became_active":"2020-08-07T15:08:24.550523+0000","last_became_peered":"2020-08-07T15:08:24.550523+0000","last_unstale":"2020-08-07T15:08:36.696621+0000","last_undegraded":"2020-08-07T15:08:36.696621+0000","last_fullsized":"2020-08-07T15:08:36.696621+0000","mapping_epoch":81,"log_start":"90'913","ondisk_log_start":"90'913","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7536640,"num_objects":115,"num_object_clones":0,"num_object_copies":345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":920,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"120","reported_epoch":"89","state":"active+clean","last_fresh":"2020-08-07T15:08:35.609391+0000","last_change":"2020-08-07T15:07:56.631170+0000","last_active":"2020-08-07T15:08:35.609391+0000","last_peered":"2020-08-07T15:08:35.609391+0000","last_clean":"2020-08-07T15:08:35.609391+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:08:35.609391+0000","last_undegraded":"2020-08-07T15:08:35.609391+0000","last_fullsized":"2020-08-07T15:08:35.609391+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:56.631091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:56.631091+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"89'928","reported_seq":"945","reported_epoch":"89","state":"active+clean","last_fresh":"2020-08-07T15:08:36.548617+0000","last_change":"2020-08-07T15:08:24.550212+0000","last_active":"2020-08-07T15:08:36.548617+0000","last_peered":"2020-08-07T15:08:36.548617+0000","last_clean":"2020-08-07T15:08:36.548617+0000","last_became_active":"2020-08-07T15:08:24.548660+0000","last_became_peered":"2020-08-07T15:08:24.548660+0000","last_unstale":"2020-08-07T15:08:36.548617+0000","last_undegraded":"2020-08-07T15:08:36.548617+0000","last_fullsized":"2020-08-07T15:08:36.548617+0000","mapping_epoch":81,"log_start":"89'920","ondisk_log_start":"89'920","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":928,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"65","reported_epoch":"90","state":"active+clean","last_fresh":"2020-08-07T15:08:36.635846+0000","last_change":"2020-08-07T15:08:36.635846+0000","last_active":"2020-08-07T15:08:36.635846+0000","last_peered":"2020-08-07T15:08:36.635846+0000","last_clean":"2020-08-07T15:08:36.635846+0000","last_became_active":"2020-08-07T15:08:36.635397+0000","last_became_peered":"2020-08-07T15:08:36.635397+0000","last_unstale":"2020-08-07T15:08:36.635846+0000","last_undegraded":"2020-08-07T15:08:36.635846+0000","last_fullsized":"2020-08-07T15:08:36.635846+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:15.585597+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":298,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"90'1080","reported_seq":"1097","reported_epoch":"90","state":"active+clean","last_fresh":"2020-08-07T15:08:36.660245+0000","last_change":"2020-08-07T15:08:24.550496+0000","last_active":"2020-08-07T15:08:36.660245+0000","last_peered":"2020-08-07T15:08:36.660245+0000","last_clean":"2020-08-07T15:08:36.660245+0000","last_became_active":"2020-08-07T15:08:24.549868+0000","last_became_peered":"2020-08-07T15:08:24.549868+0000","last_unstale":"2020-08-07T15:08:36.660245+0000","last_undegraded":"2020-08-07T15:08:36.660245+0000","last_fullsized":"2020-08-07T15:08:36.660245+0000","mapping_epoch":81,"log_start":"90'1076","ondisk_log_start":"90'1076","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8847360,"num_objects":135,"num_object_clones":0,"num_object_copies":405,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1080,"num_write_kb":8640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"71","reported_epoch":"90","state":"active+clean","last_fresh":"2020-08-07T15:08:36.618878+0000","last_change":"2020-08-07T15:08:36.618878+0000","last_active":"2020-08-07T15:08:36.618878+0000","last_peered":"2020-08-07T15:08:36.618878+0000","last_clean":"2020-08-07T15:08:36.618878+0000","last_became_active":"2020-08-07T15:08:36.618338+0000","last_became_peered":"2020-08-07T15:08:36.618338+0000","last_unstale":"2020-08-07T15:08:36.618878+0000","last_undegraded":"2020-08-07T15:08:36.618878+0000","last_fullsized":"2020-08-07T15:08:36.618878+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:34.653843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:34.653843+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"90'1136","reported_seq":"1152","reported_epoch":"90","state":"active+clean","last_fresh":"2020-08-07T15:08:36.694994+0000","last_change":"2020-08-07T15:08:26.852955+0000","last_active":"2020-08-07T15:08:36.694994+0000","last_peered":"2020-08-07T15:08:36.694994+0000","last_clean":"2020-08-07T15:08:36.694994+0000","last_became_active":"2020-08-07T15:08:26.841002+0000","last_became_peered":"2020-08-07T15:08:26.841002+0000","last_unstale":"2020-08-07T15:08:36.694994+0000","last_undegraded":"2020-08-07T15:08:36.694994+0000","last_fullsized":"2020-08-07T15:08:36.694994+0000","mapping_epoch":83,"log_start":"90'1132","ondisk_log_start":"90'1132","created":81,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":9306112,"num_objects":142,"num_object_clones":0,"num_object_copies":426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1136,"num_write_kb":9088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:08:35.597707+0000","last_change":"2020-08-07T15:08:35.597707+0000","last_active":"2020-08-07T15:08:35.597707+0000","last_peered":"2020-08-07T15:08:35.597707+0000","last_clean":"2020-08-07T15:08:35.597707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:08:35.597707+0000","last_undegraded":"2020-08-07T15:08:35.597707+0000","last_fullsized":"2020-08-07T15:08:35.597707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:35.597707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:35.597707+0000","last_clean_scrub_stamp":"2020-08-07T15:08:35.597707+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"86'680","reported_seq":"692","reported_epoch":"86","state":"active+clean","last_fresh":"2020-08-07T15:08:32.690544+0000","last_change":"2020-08-07T15:08:24.554472+0000","last_active":"2020-08-07T15:08:32.690544+0000","last_peered":"2020-08-07T15:08:32.690544+0000","last_clean":"2020-08-07T15:08:32.690544+0000","last_became_active":"2020-08-07T15:08:24.554021+0000","last_became_peered":"2020-08-07T15:08:24.554021+0000","last_unstale":"2020-08-07T15:08:32.690544+0000","last_undegraded":"2020-08-07T15:08:32.690544+0000","last_fullsized":"2020-08-07T15:08:32.690544+0000","mapping_epoch":81,"log_start":"86'672","ondisk_log_start":"86'672","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":680,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:08:35.597707+0000","last_change":"2020-08-07T15:08:35.597707+0000","last_active":"2020-08-07T15:08:35.597707+0000","last_peered":"2020-08-07T15:08:35.597707+0000","last_clean":"2020-08-07T15:08:35.597707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:08:35.597707+0000","last_undegraded":"2020-08-07T15:08:35.597707+0000","last_fullsized":"2020-08-07T15:08:35.597707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:35.597707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:35.597707+0000","last_clean_scrub_stamp":"2020-08-07T15:08:35.597707+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"89'968","reported_seq":"985","reported_epoch":"89","state":"active+clean","last_fresh":"2020-08-07T15:08:36.562565+0000","last_change":"2020-08-07T15:08:24.552802+0000","last_active":"2020-08-07T15:08:36.562565+0000","last_peered":"2020-08-07T15:08:36.562565+0000","last_clean":"2020-08-07T15:08:36.562565+0000","last_became_active":"2020-08-07T15:08:24.552127+0000","last_became_peered":"2020-08-07T15:08:24.552127+0000","last_unstale":"2020-08-07T15:08:36.562565+0000","last_undegraded":"2020-08-07T15:08:36.562565+0000","last_fullsized":"2020-08-07T15:08:36.562565+0000","mapping_epoch":81,"log_start":"89'964","ondisk_log_start":"89'964","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7929856,"num_objects":121,"num_object_clones":0,"num_object_copies":363,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":968,"num_write_kb":7744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"90'976","reported_seq":"993","reported_epoch":"90","state":"active+clean","last_fresh":"2020-08-07T15:08:36.636279+0000","last_change":"2020-08-07T15:08:24.548299+0000","last_active":"2020-08-07T15:08:36.636279+0000","last_peered":"2020-08-07T15:08:36.636279+0000","last_clean":"2020-08-07T15:08:36.636279+0000","last_became_active":"2020-08-07T15:08:24.547773+0000","last_became_peered":"2020-08-07T15:08:24.547773+0000","last_unstale":"2020-08-07T15:08:36.636279+0000","last_undegraded":"2020-08-07T15:08:36.636279+0000","last_fullsized":"2020-08-07T15:08:36.636279+0000","mapping_epoch":81,"log_start":"89'968","ondisk_log_start":"89'968","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7995392,"num_objects":122,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":976,"num_write_kb":7808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:08:35.597707+0000","last_change":"2020-08-07T15:08:35.597707+0000","last_active":"2020-08-07T15:08:35.597707+0000","last_peered":"2020-08-07T15:08:35.597707+0000","last_clean":"2020-08-07T15:08:35.597707+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:08:35.597707+0000","last_undegraded":"2020-08-07T15:08:35.597707+0000","last_fullsized":"2020-08-07T15:08:35.597707+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:35.597707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:35.597707+0000","last_clean_scrub_stamp":"2020-08-07T15:08:35.597707+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":127926272,"num_objects":1952,"num_object_clones":0,"num_object_copies":5856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15616,"num_write_kb":124928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":384524288,"data_stored":384524288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":595,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":51,"acting":51,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542173,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1079024,"kb_used_data":30440,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93292816,"statfs":{"total":96636764160,"available":95531843584,"internally_reserved":1073741824,"allocated":31170560,"data_stored":30738131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":0.95099999999999996,"15min":0.91200000000000003},"min":{"1min":0.48199999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":3.4159999999999999,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.0940000000000001,"5min":0.98899999999999999,"15min":0.97099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.3559999999999999,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":1.478}]},{"osd":1,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3069999999999999,"15min":1.2989999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":4.4829999999999997,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":1.542},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.044,"15min":0.99199999999999999},"min":{"1min":0.61799999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":3.4620000000000002,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":2.0470000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.179,"15min":1.1439999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":4.423,"5min":4.423,"15min":4.423},"last":1.78},{"interface":"front","average":{"1min":1.5,"5min":1.194,"15min":1.143},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":3.5099999999999998,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":2.0800000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.5900000000000001,"15min":1.6439999999999999},"min":{"1min":0.73799999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.5430000000000001,"5min":5.5019999999999998,"15min":5.5019999999999998},"last":1.964},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.329,"15min":1.3380000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.4590000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":1.998}]},{"osd":4,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.7849999999999999,"5min":1.377,"15min":1.3089999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":4.835,"5min":4.835,"15min":4.835},"last":2.1560000000000001},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.3300000000000001,"15min":1.3180000000000001},"min":{"1min":0.76900000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.4470000000000001,"5min":2.778,"15min":2.778},"last":866.24099999999999}]},{"osd":5,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":2.3940000000000001,"5min":30.484000000000002,"15min":35.165999999999997},"min":{"1min":0.96799999999999997,"5min":0.875,"15min":0.875},"max":{"1min":7.4429999999999996,"5min":831.88099999999997,"15min":831.88099999999997},"last":5.0279999999999996},{"interface":"front","average":{"1min":2.456,"5min":1.6870000000000001,"15min":1.5589999999999999},"min":{"1min":1,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":7.5350000000000001,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":5.0579999999999998}]},{"osd":7,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.542,"15min":1.526},"min":{"1min":0.83599999999999997,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.9900000000000002,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":2.1139999999999999},{"interface":"front","average":{"1min":1.9219999999999999,"5min":1.5900000000000001,"15min":1.534},"min":{"1min":1.115,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":4.5170000000000003,"5min":4.5170000000000003,"15min":4.5170000000000003},"last":2.3999999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574879,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1096868,"kb_used_data":48284,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93274972,"statfs":{"total":96636764160,"available":95513571328,"internally_reserved":1073741824,"allocated":49442816,"data_stored":48988542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,1,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.2490000000000001,"15min":1.3049999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.6439999999999999,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.079,"5min":1.135,"15min":1.145},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":2.9950000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":0.875}]},{"osd":1,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.95999999999999996,"15min":0.95799999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.9890000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":19.103999999999999,"5min":4.9299999999999997,"15min":2.5680000000000001},"min":{"1min":0.75600000000000001,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":372.50799999999998,"5min":372.50799999999998,"15min":372.50799999999998},"last":0.69799999999999995}]},{"osd":2,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2,"15min":1.196},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":3.036,"5min":3.036,"15min":3.036},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.236,"15min":1.26},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.7440000000000002,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":0.92800000000000005}]},{"osd":3,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.4870000000000001,"15min":1.5129999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":3.46,"5min":3.46,"15min":3.46},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.3640000000000001,"15min":1.3799999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":3.1970000000000001,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.86299999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.5880000000000001,"15min":1.6379999999999999},"min":{"1min":0.72499999999999998,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.7999999999999998,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.653,"5min":1.6679999999999999,"15min":1.6699999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":3.694,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.5880000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.3260000000000001,"15min":1.351},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.6360000000000001,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.393,"5min":1.6910000000000001,"15min":1.7410000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.8279999999999998,"5min":5.327,"15min":5.327},"last":1.208}]},{"osd":6,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.446,"15min":1.4370000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.7799999999999998,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.4730000000000001,"5min":14.955,"15min":17.202000000000002},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":3.7309999999999999,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.534}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1134588,"kb_used_data":86004,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93237252,"statfs":{"total":96636764160,"available":95474946048,"internally_reserved":1073741824,"allocated":88068096,"data_stored":87613822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":27.559999999999999,"15min":31.977},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.1820000000000004,"5min":497.04000000000002,"15min":497.04000000000002},"last":0.80400000000000005},{"interface":"front","average":{"1min":44.500999999999998,"5min":9.8049999999999997,"15min":4.0220000000000002},"min":{"1min":0.64200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":993.84500000000003,"5min":993.84500000000003,"15min":993.84500000000003},"last":0.871}]},{"osd":2,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":0.98799999999999999,"15min":0.92800000000000005},"min":{"1min":0.57399999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":4.585,"5min":4.585,"15min":4.585},"last":1.429},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.2410000000000001,"15min":1.216},"min":{"1min":0.51800000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":4.149,"5min":4.149,"15min":4.149},"last":1.075}]},{"osd":3,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.962,"5min":1.4410000000000001,"15min":1.3540000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":10.603999999999999,"5min":10.603999999999999,"15min":10.603999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":2.1909999999999998,"5min":1.601,"15min":1.5029999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":10.638999999999999,"5min":10.638999999999999,"15min":10.638999999999999},"last":1.014}]},{"osd":4,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":2.1869999999999998,"5min":1.464,"15min":1.3440000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":10.69,"5min":10.69,"15min":10.69},"last":1.026},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.264,"15min":1.2210000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":4.9500000000000002,"5min":4.9500000000000002,"15min":4.9500000000000002},"last":1.748}]},{"osd":5,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":2.0630000000000002,"5min":10.819000000000001,"15min":12.278},"min":{"1min":0.73399999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":10.720000000000001,"5min":213.63800000000001,"15min":213.63800000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.258,"15min":1.212},"min":{"1min":0.82899999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":4.6269999999999998,"5min":4.6269999999999998,"15min":4.6269999999999998},"last":1.2270000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:08:24 2020","interfaces":[{"interface":"back","average":{"1min":1.7589999999999999,"5min":1.593,"15min":1.5649999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":4.5259999999999998,"5min":4.5259999999999998,"15min":4.5259999999999998},"last":0.84699999999999998},{"interface":"front","average":{"1min":2.3690000000000002,"5min":1.5620000000000001,"15min":1.4279999999999999},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":10.792,"5min":10.792,"15min":10.792},"last":0.997}]},{"osd":7,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.8129999999999999,"5min":1.6539999999999999,"15min":1.6279999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":4.5549999999999997,"5min":4.5549999999999997,"15min":4.5549999999999997},"last":1.105},{"interface":"front","average":{"1min":2.2719999999999998,"5min":1.6379999999999999,"15min":1.532},"min":{"1min":0.998,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":10.752000000000001,"5min":10.752000000000001,"15min":10.752000000000001},"last":1.1359999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1089196,"kb_used_data":40612,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93282644,"statfs":{"total":96636764160,"available":95521427456,"internally_reserved":1073741824,"allocated":41586688,"data_stored":41132414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:08:24 2020","interfaces":[{"interface":"back","average":{"1min":81.266000000000005,"5min":17.033999999999999,"15min":6.3289999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":845.57100000000003,"5min":845.57100000000003,"15min":845.57100000000003},"last":1.724},{"interface":"front","average":{"1min":1.083,"5min":0.96299999999999997,"15min":0.94299999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":5.5970000000000004,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":1.2509999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.0069999999999999,"15min":0.95399999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":4.1600000000000001,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":26.393000000000001,"15min":30.623999999999999},"min":{"1min":0.54200000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.1280000000000001,"5min":606.18499999999995,"15min":606.18499999999995},"last":1.0069999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.0549999999999999,"15min":0.97499999999999998},"min":{"1min":0.64000000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":3.1779999999999999,"5min":3.1779999999999999,"15min":3.1779999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.927,"5min":1.319,"15min":1.218},"min":{"1min":0.78700000000000003,"5min":0.621,"15min":0.621},"max":{"1min":4.21,"5min":4.21,"15min":4.21},"last":1.5700000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.238,"15min":1.1899999999999999},"min":{"1min":0.73899999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":3.0870000000000002,"5min":3.0870000000000002,"15min":3.0870000000000002},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.6890000000000001,"5min":1.3440000000000001,"15min":1.286},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":4.0960000000000001,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":4.5899999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":39.531999999999996,"5min":8.8879999999999999,"15min":3.7799999999999998},"min":{"1min":0.76600000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":501.267,"5min":501.267,"15min":501.267},"last":3.1859999999999999},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.24,"15min":1.1499999999999999},"min":{"1min":0.874,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":6.0010000000000003,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":1.0389999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.7729999999999999,"5min":7.2450000000000001,"15min":8.157},"min":{"1min":0.81100000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":4.1310000000000002,"5min":124.738,"15min":124.738},"last":1.603},{"interface":"front","average":{"1min":1.47,"5min":15.215999999999999,"15min":17.507000000000001},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":3.7120000000000002,"5min":294.94,"15min":294.94},"last":1.8069999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.552,"15min":1.5740000000000001},"min":{"1min":0.96199999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.3929999999999998,"5min":6.468,"15min":6.468},"last":1.456},{"interface":"front","average":{"1min":1.6879999999999999,"5min":1.446,"15min":1.4059999999999999},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":3.117,"5min":3.117,"15min":3.117},"last":1.0700000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1072420,"kb_used_data":23836,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93299420,"statfs":{"total":96636764160,"available":95538606080,"internally_reserved":1073741824,"allocated":24408064,"data_stored":23953790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.153,"15min":1.129},"min":{"1min":0.505,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.4909999999999997,"5min":4.4909999999999997,"15min":4.4909999999999997},"last":1.752},{"interface":"front","average":{"1min":1.284,"5min":1.036,"15min":0.995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":3.6520000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":1.8700000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":2.5219999999999998,"5min":1.3779999999999999,"15min":1.1870000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":18.672000000000001,"5min":18.672000000000001,"15min":18.672000000000001},"last":1.8089999999999999},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.357,"15min":1.359},"min":{"1min":0.70699999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":5.1130000000000004,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":1.3380000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.216,"15min":1.1799999999999999},"min":{"1min":0.65300000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.419,"5min":1.528,"15min":1.546},"min":{"1min":0.59999999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":5.3339999999999996,"5min":8.2210000000000001,"15min":8.2210000000000001},"last":1.5940000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":2.3319999999999999,"5min":1.3540000000000001,"15min":1.1910000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":19.783000000000001,"5min":19.783000000000001,"15min":19.783000000000001},"last":1.6399999999999999},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.5589999999999999,"15min":1.615},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.891,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":1.7210000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.2789999999999999,"15min":1.2669999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.7730000000000001,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":1.8340000000000001},{"interface":"front","average":{"1min":1.5249999999999999,"5min":20.725000000000001,"15min":23.925000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":4.4790000000000001,"5min":415.40699999999998,"15min":415.40699999999998},"last":1.6719999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:08:25 2020","interfaces":[{"interface":"back","average":{"1min":2.5089999999999999,"5min":1.478,"15min":1.306},"min":{"1min":0.89000000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":20.035,"5min":20.035,"15min":20.035},"last":1.7050000000000001},{"interface":"front","average":{"1min":44.033000000000001,"5min":9.8629999999999995,"15min":4.1680000000000001},"min":{"1min":0.93700000000000006,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":751.70000000000005,"5min":751.70000000000005,"15min":751.70000000000005},"last":2.1520000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":3.5409999999999999,"5min":2.101,"15min":1.861},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":35.548999999999999,"5min":35.548999999999999,"15min":35.548999999999999},"last":1.899},{"interface":"front","average":{"1min":3.4460000000000002,"5min":1.8859999999999999,"15min":1.6259999999999999},"min":{"1min":1.004,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":35.646999999999998,"5min":35.646999999999998,"15min":35.646999999999998},"last":2.1890000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1073948,"kb_used_data":25364,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93297892,"statfs":{"total":96636764160,"available":95537041408,"internally_reserved":1073741824,"allocated":25972736,"data_stored":25518462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.044,"5min":1.411,"15min":1.3049999999999999},"min":{"1min":0.55600000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":8.6020000000000003,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.45,"5min":1.599,"15min":1.6240000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":4.9119999999999999,"5min":13.067,"15min":13.067},"last":0.52400000000000002}]},{"osd":1,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":16.382999999999999,"5min":4.7949999999999999,"15min":2.8639999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":270.68700000000001,"5min":270.68700000000001,"15min":270.68700000000001},"last":0.628},{"interface":"front","average":{"1min":2.7210000000000001,"5min":2.153,"15min":2.0590000000000002},"min":{"1min":0.71599999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":14.349,"5min":14.349,"15min":14.349},"last":0.72099999999999997}]},{"osd":2,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":1.7470000000000001,"5min":1.2669999999999999,"15min":1.1870000000000001},"min":{"1min":0.64900000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":4.9580000000000002,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":1.0660000000000001},{"interface":"front","average":{"1min":4.5220000000000002,"5min":2.2519999999999998,"15min":1.8740000000000001},"min":{"1min":0.72399999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":51.530000000000001,"5min":51.530000000000001,"15min":51.530000000000001},"last":1.079}]},{"osd":4,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":47.652999999999999,"5min":10.888,"15min":4.7599999999999998},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":506.21300000000002,"5min":506.21300000000002,"15min":506.21300000000002},"last":1.03},{"interface":"front","average":{"1min":29.771000000000001,"5min":31.634,"15min":31.943999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":505.738,"5min":677.88599999999997,"15min":677.88599999999997},"last":0.60699999999999998}]},{"osd":5,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.012,"5min":1.6160000000000001,"15min":1.55},"min":{"1min":0.96399999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":5.0049999999999999,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":2.2210000000000001,"5min":1.859,"15min":1.7989999999999999},"min":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":5.226,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":0.92000000000000004}]},{"osd":6,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":1.99,"5min":1.623,"15min":1.5620000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":5.0389999999999997,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.9710000000000001,"5min":1.9830000000000001,"15min":1.9850000000000001},"min":{"1min":1.0369999999999999,"5min":0.876,"15min":0.876},"max":{"1min":4.266,"5min":5.6189999999999998,"15min":5.6189999999999998},"last":1.0189999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.0110000000000001,"5min":2.1309999999999998,"15min":2.1509999999999998},"min":{"1min":0.96999999999999997,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":3.79,"5min":5.734,"15min":5.734},"last":0.83199999999999996},{"interface":"front","average":{"1min":2.6200000000000001,"5min":2.105,"15min":2.0190000000000001},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":8.6790000000000003,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":1.444}]}]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1105380,"kb_used_data":56796,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93266460,"statfs":{"total":96636764160,"available":95504855040,"internally_reserved":1073741824,"allocated":58159104,"data_stored":57704830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":1.125,"15min":1.1619999999999999},"min":{"1min":0.47699999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.3999999999999999,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.349,"5min":0.94899999999999995,"15min":0.88200000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":6.3970000000000002,"5min":6.3970000000000002,"15min":6.3970000000000002},"last":1.0029999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.304,"15min":1.337},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.4380000000000002,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":1.4470000000000001},{"interface":"front","average":{"1min":32.585999999999999,"5min":7.6180000000000003,"15min":3.4569999999999999},"min":{"1min":0.67000000000000004,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":689.846,"5min":689.846,"15min":689.846},"last":1.784}]},{"osd":2,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.46,"15min":1.454},"min":{"1min":0.56399999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":6.8730000000000002,"5min":6.8730000000000002,"15min":6.8730000000000002},"last":1.395},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.5669999999999999,"15min":1.532},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":7.1360000000000001,"5min":7.1360000000000001,"15min":7.1360000000000001},"last":2.2250000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":12.619999999999999,"5min":3.73,"15min":2.2480000000000002},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":247.202,"5min":247.202,"15min":247.202},"last":1.8149999999999999},{"interface":"front","average":{"1min":1.583,"5min":1.3169999999999999,"15min":1.2729999999999999},"min":{"1min":0.67800000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":6.431,"5min":6.431,"15min":6.431},"last":1.536}]},{"osd":5,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":46.786999999999999,"5min":10.659000000000001,"15min":4.6379999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":990.71600000000001,"5min":990.71600000000001,"15min":990.71600000000001},"last":2.4220000000000002},{"interface":"front","average":{"1min":1.748,"5min":1.651,"15min":1.635},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":3.9329999999999998,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":3.3519999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.7070000000000001,"5min":1.7729999999999999,"15min":1.784},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":6.9749999999999996,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":1.9159999999999999},{"interface":"front","average":{"1min":1.524,"5min":15.592000000000001,"15min":17.936},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":6.7729999999999997,"5min":389.14600000000002,"15min":389.14600000000002},"last":1.6699999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":13.378,"5min":3.802,"15min":2.206},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":252.798,"5min":252.798,"15min":252.798},"last":1.5620000000000001},{"interface":"front","average":{"1min":44.676000000000002,"5min":10.252000000000001,"15min":4.5140000000000002},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":501.93000000000001,"5min":501.93000000000001,"15min":501.93000000000001},"last":2.3050000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1097316,"kb_used_data":48732,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93274524,"statfs":{"total":96636764160,"available":95513112576,"internally_reserved":1073741824,"allocated":49901568,"data_stored":49447294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0580000000000001,"15min":1.0700000000000001},"min":{"1min":0.49199999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.5169999999999999,"5min":2.766,"15min":2.766},"last":0.61099999999999999},{"interface":"front","average":{"1min":1.2250000000000001,"5min":0.88900000000000001,"15min":0.83299999999999996},"min":{"1min":0.54900000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.024}]},{"osd":1,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":47.365000000000002,"5min":10.215999999999999,"15min":4.024},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":787.22900000000004,"5min":787.22900000000004,"15min":787.22900000000004},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.071,"5min":1.1000000000000001,"15min":1.105},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.8380000000000001,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":7.5659999999999998,"15min":8.6679999999999993},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.4950000000000001,"5min":168.56899999999999,"15min":168.56899999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.2929999999999999,"5min":26.763000000000002,"15min":31.007999999999999},"min":{"1min":0.79700000000000004,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":3.2389999999999999,"5min":676.27099999999996,"15min":676.27099999999996},"last":1.093}]},{"osd":3,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.196,"15min":1.1439999999999999},"min":{"1min":0.89000000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":3.5649999999999999,"5min":3.5649999999999999,"15min":3.5649999999999999},"last":1.0640000000000001},{"interface":"front","average":{"1min":43.073,"5min":9.5850000000000009,"15min":4.0030000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":712.98699999999997,"5min":712.98699999999997,"15min":712.98699999999997},"last":0.90500000000000003}]},{"osd":4,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":40.962000000000003,"15min":47.582000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.4980000000000002,"5min":530.30799999999999,"15min":530.30799999999999},"last":971.67999999999995},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.278,"15min":1.296},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.6259999999999999,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":0.68300000000000005}]},{"osd":6,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":40.569000000000003,"15min":47.128999999999998},"min":{"1min":0.76800000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.6699999999999999,"5min":754.31299999999999,"15min":754.31299999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.1899999999999999,"5min":21.050999999999998,"15min":24.361000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.7809999999999999,"5min":517.26700000000005,"15min":517.26700000000005},"last":0.82099999999999995}]},{"osd":7,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.3560000000000001,"15min":1.3240000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":3.9079999999999999,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.411,"5min":1.5349999999999999,"15min":1.556},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":3.9390000000000001,"5min":3.9390000000000001,"15min":3.9390000000000001},"last":0.93799999999999994}]}]}]}} 2020-08-07T15:08:38.697 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_1 pgp_num 16 2020-08-07T15:08:38.700 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:38.700 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:38.701 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:08:38.712 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:38.709+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:38.815 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:38.809+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:38.895 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:38.897 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:38.902 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:08:38.924 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:38.923+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:39.026 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:39.024+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:39.086 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:39.087 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:39.100 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:08:39.126 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:39.125+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:39.229 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:39.227+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:39.273 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 13 16 20525 20509 12.3243 12.3594 0.00754784 0.0101277 2020-08-07T15:08:39.322 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:39.322 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:39.333 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:08:39.334 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:39.332+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:39.437 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:39.435+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:39.494 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:39.495 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:39.503 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:08:39.545 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:39.543+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:39.643 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:39.638+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:39.658 INFO:teuthology.orchestra.run.smithi074.stderr:set pool 4 pgp_num to 16 2020-08-07T15:08:39.734 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:39.734 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:39.743 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:08:39.744 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:39.741+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:39.766 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:08:39.850 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:39.845+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:39.926 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:39.926 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:39.940 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:08:39.962 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:39.956+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:40.063 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:40.056+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:40.153 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:40.153 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:40.166 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:40.162+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:40.167 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:08:40.267 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:40.266+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:40.273 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 14 16 22201 22185 12.3791 13.0938 0.0114276 0.0100825 2020-08-07T15:08:40.329 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:08:40.378 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:40.369+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:40.394 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:40.394 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:40.396 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:08:40.473 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:40.471+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:40.538 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:08:40.569 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:08:40.582 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:40.577+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:40.599 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:40.599 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:40.599 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:08:40.622 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:08:40.671 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:08:40.694 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:08:40.695 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:08:40.724 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:08:40.724 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:40.722+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:40.763 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:08:40.803 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:08:40.803 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:08:40.803 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:08:40.804 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:08:40.804 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:08:40.804 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:08:40.804 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:08:40.804 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:08:40.805 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:08:40.805 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:40.802+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:40.871 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:40.871 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:08:40.871 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:08:40.872 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:08:40.872 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:40.887 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:08:40.907 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:40.906+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:41.008 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:41.008+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:41.111 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:41.110+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:41.123 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:41.123 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:08:41.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:08:41.123 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:08:41.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.124 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:41432 3.d e32/20 3:b5c8a160:::benchmark_data_smithi074_35447_object5178:head v 32'2848, mlcod=32'2838)", 2020-08-07T15:08:41.124 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.794575+0000", 2020-08-07T15:08:41.124 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 98.324382588999995, 2020-08-07T15:08:41.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36243746199999999, 2020-08-07T15:08:41.125 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:41.125 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:41.125 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:41.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:41.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794575+0000", 2020-08-07T15:08:41.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:41.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:41.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794575+0000", 2020-08-07T15:08:41.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0696999999999999e-05 2020-08-07T15:08:41.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:41.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794626+0000", 2020-08-07T15:08:41.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4189999999999999e-06 2020-08-07T15:08:41.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:41.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794632+0000", 2020-08-07T15:08:41.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36145730399999998 2020-08-07T15:08:41.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:41.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.156089+0000", 2020-08-07T15:08:41.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2329999999999997e-06 2020-08-07T15:08:41.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:41.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.156095+0000", 2020-08-07T15:08:41.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0054999999999999e-05 2020-08-07T15:08:41.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.156125+0000", 2020-08-07T15:08:41.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5369000000000001e-05 2020-08-07T15:08:41.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:41.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.156140+0000", 2020-08-07T15:08:41.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00076810300000000001 2020-08-07T15:08:41.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:41.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.156908+0000", 2020-08-07T15:08:41.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104282 2020-08-07T15:08:41.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:41.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.157012+0000", 2020-08-07T15:08:41.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36243746199999999 2020-08-07T15:08:41.156 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.157 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:41.157 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.162 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "MOSDPGPushReply(3.d 59/35 [PushReplyOp(3:bbd08cec:::benchmark_data_smithi074_35447_object8039:head)])", 2020-08-07T15:08:41.162 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.839005+0000", 2020-08-07T15:08:41.163 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 53.279952283999997, 2020-08-07T15:08:41.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42127482700000002, 2020-08-07T15:08:41.163 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:41.163 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:08:41.164 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:41.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:41.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.839005+0000", 2020-08-07T15:08:41.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:41.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:41.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.839005+0000", 2020-08-07T15:08:41.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2351000000000002e-05 2020-08-07T15:08:41.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:41.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.839038+0000", 2020-08-07T15:08:41.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3849999999999996e-06 2020-08-07T15:08:41.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:41.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.839045+0000", 2020-08-07T15:08:41.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42104101399999999 2020-08-07T15:08:41.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:41.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.260086+0000", 2020-08-07T15:08:41.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.605e-06 2020-08-07T15:08:41.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:41.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.260091+0000", 2020-08-07T15:08:41.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1946000000000006e-05 2020-08-07T15:08:41.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.260153+0000", 2020-08-07T15:08:41.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012752600000000001 2020-08-07T15:08:41.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:41.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.260280+0000", 2020-08-07T15:08:41.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42127482700000002 2020-08-07T15:08:41.178 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.179 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:41.179 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.180 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:108496 3.2 e62/20 3:49ad53c2:::benchmark_data_smithi074_35447_object13561:head v 62'6848, mlcod=62'6843)", 2020-08-07T15:08:41.180 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.809757+0000", 2020-08-07T15:08:41.180 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.309200836999999, 2020-08-07T15:08:41.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.48234552600000002, 2020-08-07T15:08:41.181 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:41.181 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:41.181 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:41.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:41.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.809757+0000", 2020-08-07T15:08:41.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:41.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:41.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.809757+0000", 2020-08-07T15:08:41.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2111000000000003e-05 2020-08-07T15:08:41.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:41.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.809809+0000", 2020-08-07T15:08:41.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8590000000000005e-06 2020-08-07T15:08:41.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:41.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.809817+0000", 2020-08-07T15:08:41.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.48126780000000002 2020-08-07T15:08:41.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:41.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.291084+0000", 2020-08-07T15:08:41.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8420000000000004e-06 2020-08-07T15:08:41.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:41.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.291089+0000", 2020-08-07T15:08:41.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2973999999999995e-05 2020-08-07T15:08:41.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.291172+0000", 2020-08-07T15:08:41.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2328999999999999e-05 2020-08-07T15:08:41.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:41.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.291195+0000", 2020-08-07T15:08:41.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00085884999999999998 2020-08-07T15:08:41.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:41.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.292053+0000", 2020-08-07T15:08:41.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8761000000000003e-05 2020-08-07T15:08:41.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:41.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.292102+0000", 2020-08-07T15:08:41.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.48234552600000002 2020-08-07T15:08:41.220 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.220 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:41.220 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.221 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "pg_backfill(progress 3.1 e 62/62 lb 3:8328459b:::benchmark_data_smithi074_35447_object1550:head)", 2020-08-07T15:08:41.221 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.857351+0000", 2020-08-07T15:08:41.221 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.261606319000002, 2020-08-07T15:08:41.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46585928100000001, 2020-08-07T15:08:41.222 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:41.222 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:41.222 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:41.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:41.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.857351+0000", 2020-08-07T15:08:41.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:41.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:41.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.857351+0000", 2020-08-07T15:08:41.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4912e-05 2020-08-07T15:08:41.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:41.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.857386+0000", 2020-08-07T15:08:41.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2329999999999995e-06 2020-08-07T15:08:41.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:41.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.857395+0000", 2020-08-07T15:08:41.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.465707918 2020-08-07T15:08:41.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:41.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.323103+0000", 2020-08-07T15:08:41.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4059999999999998e-06 2020-08-07T15:08:41.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:41.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.323108+0000", 2020-08-07T15:08:41.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1260999999999997e-05 2020-08-07T15:08:41.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.323149+0000", 2020-08-07T15:08:41.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4552999999999999e-05 2020-08-07T15:08:41.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:41.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.323164+0000", 2020-08-07T15:08:41.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6998e-05 2020-08-07T15:08:41.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:41.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.323211+0000", 2020-08-07T15:08:41.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46585928100000001 2020-08-07T15:08:41.237 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.237 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:41.237 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.238 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:145807 3.c 3:39741db6:::benchmark_data_smithi074_35447_object16198:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:08:41.238 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.673539+0000", 2020-08-07T15:08:41.238 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.445418737000001, 2020-08-07T15:08:41.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40356883700000001, 2020-08-07T15:08:41.240 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:41.240 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:41.241 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:41.241 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:41.241 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:41.241 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 145807 2020-08-07T15:08:41.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.242 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:41.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:41.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673539+0000", 2020-08-07T15:08:41.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:41.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:41.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673539+0000", 2020-08-07T15:08:41.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1495999999999998e-05 2020-08-07T15:08:41.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:41.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673570+0000", 2020-08-07T15:08:41.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7729999999999996e-06 2020-08-07T15:08:41.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:41.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673576+0000", 2020-08-07T15:08:41.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2119e-05 2020-08-07T15:08:41.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:41.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673588+0000", 2020-08-07T15:08:41.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6010000000000002e-06 2020-08-07T15:08:41.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:41.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673594+0000", 2020-08-07T15:08:41.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1732e-05 2020-08-07T15:08:41.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673636+0000", 2020-08-07T15:08:41.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015816199999999999 2020-08-07T15:08:41.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:41.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673794+0000", 2020-08-07T15:08:41.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010452 2020-08-07T15:08:41.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:08:41.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673898+0000", 2020-08-07T15:08:41.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041817169999999997 2020-08-07T15:08:41.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:41.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.678080+0000", 2020-08-07T15:08:41.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0979999999999999e-06 2020-08-07T15:08:41.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:41.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.678081+0000", 2020-08-07T15:08:41.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1353999999999996e-05 2020-08-07T15:08:41.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:41.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.678142+0000", 2020-08-07T15:08:41.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.081619948999999997 2020-08-07T15:08:41.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.759762+0000", 2020-08-07T15:08:41.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31727375899999999 2020-08-07T15:08:41.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.077036+0000", 2020-08-07T15:08:41.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.378e-06 2020-08-07T15:08:41.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:41.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.077037+0000", 2020-08-07T15:08:41.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0178999999999993e-05 2020-08-07T15:08:41.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:41.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.077108+0000", 2020-08-07T15:08:41.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40356883700000001 2020-08-07T15:08:41.264 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.265 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:41.265 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.265 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:81 4.9 4:9d72df7e:::benchmark_data_smithi074_44586_object10:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:41.266 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.332553+0000", 2020-08-07T15:08:41.266 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.786404562, 2020-08-07T15:08:41.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52066251100000005, 2020-08-07T15:08:41.266 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:41.266 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:41.267 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:41.267 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:41.267 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:41.267 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 81 2020-08-07T15:08:41.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.268 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:41.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:41.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332553+0000", 2020-08-07T15:08:41.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:41.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:41.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332553+0000", 2020-08-07T15:08:41.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4428999999999997e-05 2020-08-07T15:08:41.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:41.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332607+0000", 2020-08-07T15:08:41.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9009999999999997e-06 2020-08-07T15:08:41.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:41.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332614+0000", 2020-08-07T15:08:41.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4676e-05 2020-08-07T15:08:41.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:41.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332629+0000", 2020-08-07T15:08:41.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4560000000000003e-06 2020-08-07T15:08:41.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:41.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332636+0000", 2020-08-07T15:08:41.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9703000000000002e-05 2020-08-07T15:08:41.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332686+0000", 2020-08-07T15:08:41.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0727000000000001e-05 2020-08-07T15:08:41.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:41.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332697+0000", 2020-08-07T15:08:41.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.50871030100000003 2020-08-07T15:08:41.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.841407+0000", 2020-08-07T15:08:41.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015651400000000001 2020-08-07T15:08:41.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:41.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.841564+0000", 2020-08-07T15:08:41.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134673 2020-08-07T15:08:41.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:41.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.841698+0000", 2020-08-07T15:08:41.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003074313 2020-08-07T15:08:41.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:41.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.844773+0000", 2020-08-07T15:08:41.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.136e-06 2020-08-07T15:08:41.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:41.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.844774+0000", 2020-08-07T15:08:41.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1717e-05 2020-08-07T15:08:41.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:41.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.844826+0000", 2020-08-07T15:08:41.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083120510000000009 2020-08-07T15:08:41.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853138+0000", 2020-08-07T15:08:41.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8754e-05 2020-08-07T15:08:41.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853156+0000", 2020-08-07T15:08:41.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8599999999999998e-07 2020-08-07T15:08:41.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:41.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853157+0000", 2020-08-07T15:08:41.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8474000000000003e-05 2020-08-07T15:08:41.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:41.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853216+0000", 2020-08-07T15:08:41.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52066251100000005 2020-08-07T15:08:41.296 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.298 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:41.298 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.300 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:82 4.9 4:9d72df7e:::benchmark_data_smithi074_44586_object10:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:41.300 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.332680+0000", 2020-08-07T15:08:41.300 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.786277128, 2020-08-07T15:08:41.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52064592399999998, 2020-08-07T15:08:41.301 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:41.301 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:41.301 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:41.301 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:41.301 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:41.302 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 82 2020-08-07T15:08:41.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.302 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:41.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:41.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332680+0000", 2020-08-07T15:08:41.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:41.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:41.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332680+0000", 2020-08-07T15:08:41.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.168e-05 2020-08-07T15:08:41.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:41.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332732+0000", 2020-08-07T15:08:41.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4600000000000002e-06 2020-08-07T15:08:41.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:41.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332738+0000", 2020-08-07T15:08:41.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1202e-05 2020-08-07T15:08:41.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:41.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332749+0000", 2020-08-07T15:08:41.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.164e-06 2020-08-07T15:08:41.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:41.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332754+0000", 2020-08-07T15:08:41.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0890000000000002e-05 2020-08-07T15:08:41.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332805+0000", 2020-08-07T15:08:41.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0029999999999994e-06 2020-08-07T15:08:41.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:41.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332813+0000", 2020-08-07T15:08:41.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.50934248400000004 2020-08-07T15:08:41.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.842155+0000", 2020-08-07T15:08:41.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011158399999999999 2020-08-07T15:08:41.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:41.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.842267+0000", 2020-08-07T15:08:41.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019425200000000001 2020-08-07T15:08:41.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:41.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.842461+0000", 2020-08-07T15:08:41.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023764670000000002 2020-08-07T15:08:41.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:41.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.844838+0000", 2020-08-07T15:08:41.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000005e-07 2020-08-07T15:08:41.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:41.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.844838+0000", 2020-08-07T15:08:41.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3197e-05 2020-08-07T15:08:41.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:41.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.844881+0000", 2020-08-07T15:08:41.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083592779999999995 2020-08-07T15:08:41.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853241+0000", 2020-08-07T15:08:41.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3352e-05 2020-08-07T15:08:41.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853294+0000", 2020-08-07T15:08:41.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4699999999999998e-07 2020-08-07T15:08:41.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:41.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853294+0000", 2020-08-07T15:08:41.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2162999999999997e-05 2020-08-07T15:08:41.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:41.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853326+0000", 2020-08-07T15:08:41.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52064592399999998 2020-08-07T15:08:41.379 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.380 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:41.380 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.381 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:83 4.9 4:9d72df7e:::benchmark_data_smithi074_44586_object10:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:41.381 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.332842+0000", 2020-08-07T15:08:41.382 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.786115198999999, 2020-08-07T15:08:41.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52054723000000003, 2020-08-07T15:08:41.382 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:41.383 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:41.383 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:41.383 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:41.383 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:41.384 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 83 2020-08-07T15:08:41.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.384 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:41.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:41.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332842+0000", 2020-08-07T15:08:41.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:41.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:41.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332842+0000", 2020-08-07T15:08:41.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2209000000000001e-05 2020-08-07T15:08:41.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:41.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332895+0000", 2020-08-07T15:08:41.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4070000000000004e-06 2020-08-07T15:08:41.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:41.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332902+0000", 2020-08-07T15:08:41.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6030000000000001e-05 2020-08-07T15:08:41.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:41.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332918+0000", 2020-08-07T15:08:41.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3189999999999995e-06 2020-08-07T15:08:41.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:41.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332926+0000", 2020-08-07T15:08:41.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2171000000000001e-05 2020-08-07T15:08:41.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332959+0000", 2020-08-07T15:08:41.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8200000000000002e-06 2020-08-07T15:08:41.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:41.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332964+0000", 2020-08-07T15:08:41.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.50992941300000005 2020-08-07T15:08:41.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.842894+0000", 2020-08-07T15:08:41.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9205999999999999e-05 2020-08-07T15:08:41.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:41.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.842993+0000", 2020-08-07T15:08:41.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000194155 2020-08-07T15:08:41.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:41.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.843187+0000", 2020-08-07T15:08:41.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029782989999999998 2020-08-07T15:08:41.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:41.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.846165+0000", 2020-08-07T15:08:41.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9699999999999995e-07 2020-08-07T15:08:41.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:41.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.846166+0000", 2020-08-07T15:08:41.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5018999999999997e-05 2020-08-07T15:08:41.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:41.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.846211+0000", 2020-08-07T15:08:41.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070467719999999998 2020-08-07T15:08:41.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853258+0000", 2020-08-07T15:08:41.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010171 2020-08-07T15:08:41.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853360+0000", 2020-08-07T15:08:41.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5699999999999999e-07 2020-08-07T15:08:41.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:41.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853360+0000", 2020-08-07T15:08:41.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9646e-05 2020-08-07T15:08:41.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:41.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853390+0000", 2020-08-07T15:08:41.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52054723000000003 2020-08-07T15:08:41.447 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.447 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:41.447 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.454 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:84 4.9 4:9d72df7e:::benchmark_data_smithi074_44586_object10:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:41.455 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.332964+0000", 2020-08-07T15:08:41.456 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.785993291, 2020-08-07T15:08:41.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52047439900000003, 2020-08-07T15:08:41.457 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:41.457 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:41.458 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:41.458 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:41.459 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:41.459 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 84 2020-08-07T15:08:41.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.459 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:41.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:41.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332964+0000", 2020-08-07T15:08:41.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:41.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:41.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.332964+0000", 2020-08-07T15:08:41.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9694000000000002e-05 2020-08-07T15:08:41.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:41.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333014+0000", 2020-08-07T15:08:41.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2000000000000002e-06 2020-08-07T15:08:41.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:41.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333019+0000", 2020-08-07T15:08:41.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3052e-05 2020-08-07T15:08:41.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:41.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333032+0000", 2020-08-07T15:08:41.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8559999999999998e-06 2020-08-07T15:08:41.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:41.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333037+0000", 2020-08-07T15:08:41.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5210000000000003e-05 2020-08-07T15:08:41.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333082+0000", 2020-08-07T15:08:41.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.594e-06 2020-08-07T15:08:41.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:41.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333092+0000", 2020-08-07T15:08:41.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.51043333000000002 2020-08-07T15:08:41.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.843525+0000", 2020-08-07T15:08:41.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106469 2020-08-07T15:08:41.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:41.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.843632+0000", 2020-08-07T15:08:41.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019589400000000001 2020-08-07T15:08:41.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:41.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.843828+0000", 2020-08-07T15:08:41.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003896208 2020-08-07T15:08:41.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:41.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.847724+0000", 2020-08-07T15:08:41.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6700000000000002e-07 2020-08-07T15:08:41.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:41.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.847725+0000", 2020-08-07T15:08:41.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4647999999999995e-05 2020-08-07T15:08:41.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:41.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.847789+0000", 2020-08-07T15:08:41.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054861290000000002 2020-08-07T15:08:41.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853276+0000", 2020-08-07T15:08:41.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013245999999999999 2020-08-07T15:08:41.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853408+0000", 2020-08-07T15:08:41.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8000000000000002e-07 2020-08-07T15:08:41.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:41.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853408+0000", 2020-08-07T15:08:41.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0408000000000001e-05 2020-08-07T15:08:41.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:41.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853439+0000", 2020-08-07T15:08:41.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52047439900000003 2020-08-07T15:08:41.503 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.503 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:41.504 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.504 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:85 4.9 4:9d72df7e:::benchmark_data_smithi074_44586_object10:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:41.505 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.333658+0000", 2020-08-07T15:08:41.505 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.78529973, 2020-08-07T15:08:41.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.51984521400000006, 2020-08-07T15:08:41.505 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:41.506 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:41.507 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:41.507 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:41.507 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:41.507 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 85 2020-08-07T15:08:41.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.508 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:41.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:41.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333658+0000", 2020-08-07T15:08:41.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:41.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:41.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333658+0000", 2020-08-07T15:08:41.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6450999999999999e-05 2020-08-07T15:08:41.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:41.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333714+0000", 2020-08-07T15:08:41.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4449999999999999e-06 2020-08-07T15:08:41.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:41.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333722+0000", 2020-08-07T15:08:41.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.491e-05 2020-08-07T15:08:41.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:41.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333737+0000", 2020-08-07T15:08:41.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7570000000000001e-06 2020-08-07T15:08:41.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:41.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333742+0000", 2020-08-07T15:08:41.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2324000000000003e-05 2020-08-07T15:08:41.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333785+0000", 2020-08-07T15:08:41.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4694e-05 2020-08-07T15:08:41.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:41.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.333799+0000", 2020-08-07T15:08:41.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.51044864400000001 2020-08-07T15:08:41.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.844248+0000", 2020-08-07T15:08:41.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9263000000000003e-05 2020-08-07T15:08:41.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:41.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.844327+0000", 2020-08-07T15:08:41.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012624700000000001 2020-08-07T15:08:41.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:41.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.844454+0000", 2020-08-07T15:08:41.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003347362 2020-08-07T15:08:41.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:41.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.847801+0000", 2020-08-07T15:08:41.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.37e-07 2020-08-07T15:08:41.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:41.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.847802+0000", 2020-08-07T15:08:41.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4914000000000003e-05 2020-08-07T15:08:41.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:41.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.847857+0000", 2020-08-07T15:08:41.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056010640000000002 2020-08-07T15:08:41.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853458+0000", 2020-08-07T15:08:41.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6331999999999999e-05 2020-08-07T15:08:41.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853474+0000", 2020-08-07T15:08:41.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1e-07 2020-08-07T15:08:41.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:41.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853474+0000", 2020-08-07T15:08:41.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8860000000000002e-05 2020-08-07T15:08:41.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:41.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853503+0000", 2020-08-07T15:08:41.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.51984521400000006 2020-08-07T15:08:41.545 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.545 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:41.549 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.551 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:86 4.9 4:9d72df7e:::benchmark_data_smithi074_44586_object10:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:41.553 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.334385+0000", 2020-08-07T15:08:41.554 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.784572873, 2020-08-07T15:08:41.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.51921272600000001, 2020-08-07T15:08:41.554 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:41.555 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:41.555 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:41.555 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:41.558 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:41.558 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86 2020-08-07T15:08:41.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.559 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:41.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:41.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334385+0000", 2020-08-07T15:08:41.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:41.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:41.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334385+0000", 2020-08-07T15:08:41.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.757e-05 2020-08-07T15:08:41.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:41.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334432+0000", 2020-08-07T15:08:41.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8680000000000001e-06 2020-08-07T15:08:41.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:41.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334439+0000", 2020-08-07T15:08:41.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.291e-05 2020-08-07T15:08:41.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:41.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334452+0000", 2020-08-07T15:08:41.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0889999999999996e-06 2020-08-07T15:08:41.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:41.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334458+0000", 2020-08-07T15:08:41.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4972000000000003e-05 2020-08-07T15:08:41.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334493+0000", 2020-08-07T15:08:41.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9689999999999999e-06 2020-08-07T15:08:41.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:41.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334499+0000", 2020-08-07T15:08:41.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.51041587300000002 2020-08-07T15:08:41.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.844915+0000", 2020-08-07T15:08:41.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4506999999999995e-05 2020-08-07T15:08:41.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:41.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.845009+0000", 2020-08-07T15:08:41.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000171695 2020-08-07T15:08:41.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:41.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.845181+0000", 2020-08-07T15:08:41.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003417546 2020-08-07T15:08:41.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:41.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.848599+0000", 2020-08-07T15:08:41.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7999999999999993e-07 2020-08-07T15:08:41.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:41.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.848600+0000", 2020-08-07T15:08:41.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4317000000000001e-05 2020-08-07T15:08:41.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:41.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.848664+0000", 2020-08-07T15:08:41.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048569909999999997 2020-08-07T15:08:41.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853521+0000", 2020-08-07T15:08:41.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7061000000000002e-05 2020-08-07T15:08:41.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853568+0000", 2020-08-07T15:08:41.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.85e-07 2020-08-07T15:08:41.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:41.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853568+0000", 2020-08-07T15:08:41.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9193e-05 2020-08-07T15:08:41.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:41.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853597+0000", 2020-08-07T15:08:41.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.51921272600000001 2020-08-07T15:08:41.607 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.608 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:41.609 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.610 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:87 4.9 4:9d72df7e:::benchmark_data_smithi074_44586_object10:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:41.610 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.334494+0000", 2020-08-07T15:08:41.611 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.784463283999999, 2020-08-07T15:08:41.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.51915305300000003, 2020-08-07T15:08:41.613 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:41.613 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:41.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:41.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:41.615 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:41.615 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 87 2020-08-07T15:08:41.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.616 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:41.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:41.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334494+0000", 2020-08-07T15:08:41.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:41.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:41.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334494+0000", 2020-08-07T15:08:41.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7128e-05 2020-08-07T15:08:41.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:41.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334541+0000", 2020-08-07T15:08:41.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7169999999999996e-06 2020-08-07T15:08:41.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:41.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334547+0000", 2020-08-07T15:08:41.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1586999999999999e-05 2020-08-07T15:08:41.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:41.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334559+0000", 2020-08-07T15:08:41.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4620000000000004e-06 2020-08-07T15:08:41.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:41.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334564+0000", 2020-08-07T15:08:41.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5139999999999999e-05 2020-08-07T15:08:41.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334599+0000", 2020-08-07T15:08:41.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2600000000000005e-06 2020-08-07T15:08:41.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:41.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.334608+0000", 2020-08-07T15:08:41.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.51096861900000001 2020-08-07T15:08:41.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.845576+0000", 2020-08-07T15:08:41.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5854999999999996e-05 2020-08-07T15:08:41.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:41.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.845672+0000", 2020-08-07T15:08:41.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016852800000000001 2020-08-07T15:08:41.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:41.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.845841+0000", 2020-08-07T15:08:41.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028329470000000002 2020-08-07T15:08:41.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:41.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.848674+0000", 2020-08-07T15:08:41.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0200000000000001e-07 2020-08-07T15:08:41.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:41.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.848674+0000", 2020-08-07T15:08:41.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0541000000000002e-05 2020-08-07T15:08:41.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:41.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.848735+0000", 2020-08-07T15:08:41.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048022710000000003 2020-08-07T15:08:41.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853537+0000", 2020-08-07T15:08:41.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7730000000000003e-05 2020-08-07T15:08:41.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853615+0000", 2020-08-07T15:08:41.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8199999999999999e-07 2020-08-07T15:08:41.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:41.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853615+0000", 2020-08-07T15:08:41.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2384e-05 2020-08-07T15:08:41.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:41.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853647+0000", 2020-08-07T15:08:41.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.51915305300000003 2020-08-07T15:08:41.672 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.672 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:41.673 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.685 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:88 4.9 4:9d72df7e:::benchmark_data_smithi074_44586_object10:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:41.686 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.335168+0000", 2020-08-07T15:08:41.686 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.783790006, 2020-08-07T15:08:41.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.51856037899999996, 2020-08-07T15:08:41.687 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:41.687 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:41.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:41.688 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:41.688 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:41.688 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 88 2020-08-07T15:08:41.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.689 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:41.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:41.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.335168+0000", 2020-08-07T15:08:41.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:41.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:41.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.335168+0000", 2020-08-07T15:08:41.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7526e-05 2020-08-07T15:08:41.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:41.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.335205+0000", 2020-08-07T15:08:41.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.19e-06 2020-08-07T15:08:41.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:41.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.335208+0000", 2020-08-07T15:08:41.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8259999999999993e-06 2020-08-07T15:08:41.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:41.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.335216+0000", 2020-08-07T15:08:41.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9170000000000002e-06 2020-08-07T15:08:41.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:41.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.335219+0000", 2020-08-07T15:08:41.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7576000000000002e-05 2020-08-07T15:08:41.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.335247+0000", 2020-08-07T15:08:41.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4499999999999997e-06 2020-08-07T15:08:41.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:41.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.335251+0000", 2020-08-07T15:08:41.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.51102096600000002 2020-08-07T15:08:41.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.846272+0000", 2020-08-07T15:08:41.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104205 2020-08-07T15:08:41.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:41.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.846376+0000", 2020-08-07T15:08:41.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017541 2020-08-07T15:08:41.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:41.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.846552+0000", 2020-08-07T15:08:41.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029493729999999999 2020-08-07T15:08:41.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:41.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.849501+0000", 2020-08-07T15:08:41.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1299999999999999e-07 2020-08-07T15:08:41.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:41.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.849502+0000", 2020-08-07T15:08:41.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7806999999999997e-05 2020-08-07T15:08:41.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:41.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.849560+0000", 2020-08-07T15:08:41.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039933019999999998 2020-08-07T15:08:41.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853553+0000", 2020-08-07T15:08:41.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000126184 2020-08-07T15:08:41.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853679+0000", 2020-08-07T15:08:41.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9900000000000002e-07 2020-08-07T15:08:41.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:41.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853679+0000", 2020-08-07T15:08:41.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8535000000000002e-05 2020-08-07T15:08:41.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:41.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853728+0000", 2020-08-07T15:08:41.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.51856037899999996 2020-08-07T15:08:41.746 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.747 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:41.747 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.747 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:177 4.9 4:96543f2a:::benchmark_data_smithi074_44586_object22:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:41.748 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.418082+0000", 2020-08-07T15:08:41.748 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.70087592, 2020-08-07T15:08:41.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44218349899999998, 2020-08-07T15:08:41.748 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:41.748 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:41.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:41.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:41.758 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:41.758 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 177 2020-08-07T15:08:41.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.759 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:41.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:41.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418082+0000", 2020-08-07T15:08:41.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:41.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:41.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418082+0000", 2020-08-07T15:08:41.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7248999999999999e-05 2020-08-07T15:08:41.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:41.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418109+0000", 2020-08-07T15:08:41.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0729999999999998e-06 2020-08-07T15:08:41.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:41.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418113+0000", 2020-08-07T15:08:41.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6090000000000001e-06 2020-08-07T15:08:41.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:41.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418122+0000", 2020-08-07T15:08:41.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.809e-06 2020-08-07T15:08:41.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:41.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418126+0000", 2020-08-07T15:08:41.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2879000000000001e-05 2020-08-07T15:08:41.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418159+0000", 2020-08-07T15:08:41.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7880000000000003e-06 2020-08-07T15:08:41.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:41.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418165+0000", 2020-08-07T15:08:41.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42881086299999999 2020-08-07T15:08:41.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.846976+0000", 2020-08-07T15:08:41.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018687200000000001 2020-08-07T15:08:41.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:41.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.847163+0000", 2020-08-07T15:08:41.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000176665 2020-08-07T15:08:41.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:41.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.847339+0000", 2020-08-07T15:08:41.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022285949999999999 2020-08-07T15:08:41.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:41.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.849568+0000", 2020-08-07T15:08:41.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6000000000000003e-07 2020-08-07T15:08:41.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:41.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.849569+0000", 2020-08-07T15:08:41.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1811000000000003e-05 2020-08-07T15:08:41.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:41.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.849621+0000", 2020-08-07T15:08:41.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004143178 2020-08-07T15:08:41.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853764+0000", 2020-08-07T15:08:41.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064402749999999996 2020-08-07T15:08:41.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860204+0000", 2020-08-07T15:08:41.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1710000000000001e-06 2020-08-07T15:08:41.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:41.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860205+0000", 2020-08-07T15:08:41.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0001999999999999e-05 2020-08-07T15:08:41.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:41.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860265+0000", 2020-08-07T15:08:41.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44218349899999998 2020-08-07T15:08:41.783 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.783 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:41.783 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.784 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:178 4.9 4:96543f2a:::benchmark_data_smithi074_44586_object22:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:41.785 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.418151+0000", 2020-08-07T15:08:41.785 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.70080673, 2020-08-07T15:08:41.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44221660699999998, 2020-08-07T15:08:41.787 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:41.787 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:41.787 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:41.787 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:41.788 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:41.788 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 178 2020-08-07T15:08:41.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.788 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:41.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:41.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418151+0000", 2020-08-07T15:08:41.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:41.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:41.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418151+0000", 2020-08-07T15:08:41.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9198000000000001e-05 2020-08-07T15:08:41.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:41.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418180+0000", 2020-08-07T15:08:41.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0199999999999999e-06 2020-08-07T15:08:41.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:41.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418183+0000", 2020-08-07T15:08:41.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9129999999999997e-06 2020-08-07T15:08:41.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:41.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418190+0000", 2020-08-07T15:08:41.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.655e-06 2020-08-07T15:08:41.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:41.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418193+0000", 2020-08-07T15:08:41.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0032999999999999e-05 2020-08-07T15:08:41.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418223+0000", 2020-08-07T15:08:41.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5979999999999996e-06 2020-08-07T15:08:41.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:41.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418228+0000", 2020-08-07T15:08:41.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42965734799999999 2020-08-07T15:08:41.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.847886+0000", 2020-08-07T15:08:41.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116982 2020-08-07T15:08:41.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:41.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.848003+0000", 2020-08-07T15:08:41.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021092100000000001 2020-08-07T15:08:41.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:41.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.848214+0000", 2020-08-07T15:08:41.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055933939999999998 2020-08-07T15:08:41.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:41.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853807+0000", 2020-08-07T15:08:41.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.178e-06 2020-08-07T15:08:41.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:41.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853808+0000", 2020-08-07T15:08:41.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7018000000000002e-05 2020-08-07T15:08:41.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:41.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.853865+0000", 2020-08-07T15:08:41.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000335068 2020-08-07T15:08:41.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854200+0000", 2020-08-07T15:08:41.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061043219999999997 2020-08-07T15:08:41.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860304+0000", 2020-08-07T15:08:41.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.54e-07 2020-08-07T15:08:41.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:41.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860305+0000", 2020-08-07T15:08:41.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2205000000000005e-05 2020-08-07T15:08:41.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:41.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860367+0000", 2020-08-07T15:08:41.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44221660699999998 2020-08-07T15:08:41.833 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.833 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:41.833 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.834 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:179 4.9 4:96543f2a:::benchmark_data_smithi074_44586_object22:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:41.836 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.418212+0000", 2020-08-07T15:08:41.836 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.700745173, 2020-08-07T15:08:41.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.442386379, 2020-08-07T15:08:41.837 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:41.837 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:41.837 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:41.838 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:41.838 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:41.838 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 179 2020-08-07T15:08:41.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.838 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:41.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:41.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418212+0000", 2020-08-07T15:08:41.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:41.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:41.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418212+0000", 2020-08-07T15:08:41.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6448000000000002e-05 2020-08-07T15:08:41.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:41.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418239+0000", 2020-08-07T15:08:41.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.695e-06 2020-08-07T15:08:41.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:41.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418242+0000", 2020-08-07T15:08:41.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2939999999999996e-06 2020-08-07T15:08:41.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:41.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418247+0000", 2020-08-07T15:08:41.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3810000000000002e-06 2020-08-07T15:08:41.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:41.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418249+0000", 2020-08-07T15:08:41.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7589e-05 2020-08-07T15:08:41.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418267+0000", 2020-08-07T15:08:41.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3139999999999999e-06 2020-08-07T15:08:41.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:41.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418270+0000", 2020-08-07T15:08:41.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43049494300000002 2020-08-07T15:08:41.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.848765+0000", 2020-08-07T15:08:41.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013311900000000001 2020-08-07T15:08:41.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:41.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.848898+0000", 2020-08-07T15:08:41.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026578799999999998 2020-08-07T15:08:41.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:41.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.849164+0000", 2020-08-07T15:08:41.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050703370000000003 2020-08-07T15:08:41.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854234+0000", 2020-08-07T15:08:41.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4944999999999997e-05 2020-08-07T15:08:41.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:41.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854279+0000", 2020-08-07T15:08:41.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0062045219999999996 2020-08-07T15:08:41.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860484+0000", 2020-08-07T15:08:41.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8199999999999998e-07 2020-08-07T15:08:41.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:41.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860485+0000", 2020-08-07T15:08:41.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6585e-05 2020-08-07T15:08:41.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:41.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860551+0000", 2020-08-07T15:08:41.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7537000000000002e-05 2020-08-07T15:08:41.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:41.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860599+0000", 2020-08-07T15:08:41.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.442386379 2020-08-07T15:08:41.867 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.867 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:41.867 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.868 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:180 4.9 4:96543f2a:::benchmark_data_smithi074_44586_object22:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:41.868 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.418982+0000", 2020-08-07T15:08:41.868 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.699975641, 2020-08-07T15:08:41.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.441772624, 2020-08-07T15:08:41.869 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:41.869 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:41.872 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:41.872 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:41.872 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:41.872 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 180 2020-08-07T15:08:41.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.874 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:41.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:41.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418982+0000", 2020-08-07T15:08:41.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:41.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:41.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.418982+0000", 2020-08-07T15:08:41.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9762000000000001e-05 2020-08-07T15:08:41.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:41.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.419012+0000", 2020-08-07T15:08:41.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1910000000000002e-06 2020-08-07T15:08:41.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:41.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.419016+0000", 2020-08-07T15:08:41.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1570000000000006e-06 2020-08-07T15:08:41.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:41.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.419025+0000", 2020-08-07T15:08:41.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3480000000000003e-06 2020-08-07T15:08:41.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:41.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.419030+0000", 2020-08-07T15:08:41.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2101000000000001e-05 2020-08-07T15:08:41.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.419052+0000", 2020-08-07T15:08:41.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4489999999999999e-06 2020-08-07T15:08:41.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:41.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.419056+0000", 2020-08-07T15:08:41.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43059229599999999 2020-08-07T15:08:41.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.849648+0000", 2020-08-07T15:08:41.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3301999999999995e-05 2020-08-07T15:08:41.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:41.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.849742+0000", 2020-08-07T15:08:41.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016384599999999999 2020-08-07T15:08:41.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:41.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.849905+0000", 2020-08-07T15:08:41.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043603569999999996 2020-08-07T15:08:41.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854266+0000", 2020-08-07T15:08:41.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.997e-05 2020-08-07T15:08:41.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:41.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854286+0000", 2020-08-07T15:08:41.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006358982 2020-08-07T15:08:41.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860645+0000", 2020-08-07T15:08:41.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.75e-07 2020-08-07T15:08:41.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:41.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860645+0000", 2020-08-07T15:08:41.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3958e-05 2020-08-07T15:08:41.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:41.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860709+0000", 2020-08-07T15:08:41.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5330000000000001e-05 2020-08-07T15:08:41.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:41.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860755+0000", 2020-08-07T15:08:41.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.441772624 2020-08-07T15:08:41.919 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.919 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:41.920 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.920 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:181 4.9 4:96543f2a:::benchmark_data_smithi074_44586_object22:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:41.921 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.420446+0000", 2020-08-07T15:08:41.921 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.698511701999999, 2020-08-07T15:08:41.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44049454599999999, 2020-08-07T15:08:41.921 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:41.922 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:41.922 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:41.922 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:41.922 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:41.922 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 181 2020-08-07T15:08:41.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.923 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:41.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:41.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420446+0000", 2020-08-07T15:08:41.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:41.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:41.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420446+0000", 2020-08-07T15:08:41.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7520000000000001e-05 2020-08-07T15:08:41.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:41.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420473+0000", 2020-08-07T15:08:41.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9609999999999999e-06 2020-08-07T15:08:41.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:41.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420477+0000", 2020-08-07T15:08:41.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0050000000000004e-06 2020-08-07T15:08:41.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:41.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420485+0000", 2020-08-07T15:08:41.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1629999999999998e-06 2020-08-07T15:08:41.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:41.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420490+0000", 2020-08-07T15:08:41.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3699999999999999e-05 2020-08-07T15:08:41.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420523+0000", 2020-08-07T15:08:41.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.293e-06 2020-08-07T15:08:41.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:41.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420529+0000", 2020-08-07T15:08:41.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42978759300000002 2020-08-07T15:08:41.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.850316+0000", 2020-08-07T15:08:41.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101647 2020-08-07T15:08:41.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:41.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.850418+0000", 2020-08-07T15:08:41.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019151800000000001 2020-08-07T15:08:41.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:41.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.850609+0000", 2020-08-07T15:08:41.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037621099999999999 2020-08-07T15:08:41.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:41.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854371+0000", 2020-08-07T15:08:41.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9999999999999997e-07 2020-08-07T15:08:41.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:41.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854372+0000", 2020-08-07T15:08:41.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7818999999999998e-05 2020-08-07T15:08:41.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:41.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854410+0000", 2020-08-07T15:08:41.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002193459 2020-08-07T15:08:41.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.856603+0000", 2020-08-07T15:08:41.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042760380000000002 2020-08-07T15:08:41.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:41.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860879+0000", 2020-08-07T15:08:41.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2020-08-07T15:08:41.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:41.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860880+0000", 2020-08-07T15:08:41.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0449999999999999e-05 2020-08-07T15:08:41.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:41.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.860940+0000", 2020-08-07T15:08:41.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44049454599999999 2020-08-07T15:08:41.979 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.980 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:41.981 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:41.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.981 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:182 4.9 4:96543f2a:::benchmark_data_smithi074_44586_object22:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:41.982 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.420527+0000", 2020-08-07T15:08:41.982 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.698430075999999, 2020-08-07T15:08:41.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44055363400000003, 2020-08-07T15:08:41.983 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:41.983 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:41.984 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:41.984 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:41.984 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:41.984 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 182 2020-08-07T15:08:41.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.985 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:41.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:41.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420527+0000", 2020-08-07T15:08:41.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:41.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:41.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420527+0000", 2020-08-07T15:08:41.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3171999999999999e-05 2020-08-07T15:08:41.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:41.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420571+0000", 2020-08-07T15:08:41.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3900000000000001e-06 2020-08-07T15:08:41.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:41.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420576+0000", 2020-08-07T15:08:41.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6989999999999997e-06 2020-08-07T15:08:41.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:41.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420583+0000", 2020-08-07T15:08:41.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4410000000000002e-06 2020-08-07T15:08:41.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:41.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420586+0000", 2020-08-07T15:08:41.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8172e-05 2020-08-07T15:08:41.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420624+0000", 2020-08-07T15:08:41.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2298e-05 2020-08-07T15:08:41.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:41.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420637+0000", 2020-08-07T15:08:41.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43048289299999998 2020-08-07T15:08:41.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:41.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:41.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:41.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.851120+0000", 2020-08-07T15:08:41.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5534999999999999e-05 2020-08-07T15:08:41.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:42.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.851215+0000", 2020-08-07T15:08:42.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016653100000000001 2020-08-07T15:08:42.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:42.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.851382+0000", 2020-08-07T15:08:42.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030351480000000001 2020-08-07T15:08:42.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:42.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854417+0000", 2020-08-07T15:08:42.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1399999999999997e-07 2020-08-07T15:08:42.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:42.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854417+0000", 2020-08-07T15:08:42.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0976000000000001e-05 2020-08-07T15:08:42.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:42.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854468+0000", 2020-08-07T15:08:42.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039349759999999998 2020-08-07T15:08:42.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:42.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.858403+0000", 2020-08-07T15:08:42.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026043580000000002 2020-08-07T15:08:42.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:42.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.861008+0000", 2020-08-07T15:08:42.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7199999999999998e-07 2020-08-07T15:08:42.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:42.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.861008+0000", 2020-08-07T15:08:42.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2758999999999999e-05 2020-08-07T15:08:42.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:42.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.861081+0000", 2020-08-07T15:08:42.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44055363400000003 2020-08-07T15:08:42.010 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:42.010 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:42.010 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:42.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.011 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:183 4.9 4:96543f2a:::benchmark_data_smithi074_44586_object22:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:42.011 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.420764+0000", 2020-08-07T15:08:42.011 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.698193078999999, 2020-08-07T15:08:42.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44042149400000002, 2020-08-07T15:08:42.012 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:42.012 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:42.012 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:42.015 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:42.015 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:42.016 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 183 2020-08-07T15:08:42.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.016 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:42.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:42.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420764+0000", 2020-08-07T15:08:42.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:42.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:42.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420764+0000", 2020-08-07T15:08:42.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.546e-05 2020-08-07T15:08:42.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:42.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420790+0000", 2020-08-07T15:08:42.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8459999999999999e-06 2020-08-07T15:08:42.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:42.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420793+0000", 2020-08-07T15:08:42.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7740000000000001e-06 2020-08-07T15:08:42.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:42.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420799+0000", 2020-08-07T15:08:42.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8710000000000001e-06 2020-08-07T15:08:42.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:42.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420801+0000", 2020-08-07T15:08:42.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6449e-05 2020-08-07T15:08:42.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:42.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420828+0000", 2020-08-07T15:08:42.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.882e-06 2020-08-07T15:08:42.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:42.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.420833+0000", 2020-08-07T15:08:42.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43095108799999998 2020-08-07T15:08:42.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:42.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.851784+0000", 2020-08-07T15:08:42.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7198000000000002e-05 2020-08-07T15:08:42.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:42.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.851861+0000", 2020-08-07T15:08:42.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019497500000000001 2020-08-07T15:08:42.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:42.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.852056+0000", 2020-08-07T15:08:42.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024200630000000001 2020-08-07T15:08:42.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:42.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854476+0000", 2020-08-07T15:08:42.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9599999999999999e-07 2020-08-07T15:08:42.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:42.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854477+0000", 2020-08-07T15:08:42.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4832e-05 2020-08-07T15:08:42.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:42.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854522+0000", 2020-08-07T15:08:42.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039150890000000001 2020-08-07T15:08:42.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:42.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.858437+0000", 2020-08-07T15:08:42.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026888260000000001 2020-08-07T15:08:42.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:42.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.861125+0000", 2020-08-07T15:08:42.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6400000000000002e-07 2020-08-07T15:08:42.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:42.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.861126+0000", 2020-08-07T15:08:42.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9981000000000001e-05 2020-08-07T15:08:42.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:42.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.861186+0000", 2020-08-07T15:08:42.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44042149400000002 2020-08-07T15:08:42.055 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:42.056 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:42.056 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:42.056 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:42.057 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:42.057 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:42.059 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 15 16 23805 23789 12.3892 12.5312 0.00699065 0.010081 2020-08-07T15:08:42.060 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:08:42.061 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:42.057+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:42.179 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:42.163+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:42.310 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 16 16 25614 25598 12.4981 14.1328 0.00431724 0.00999292 2020-08-07T15:08:42.311 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:42.269+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:42.312 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:42.316 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:08:42.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.317 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:25655 4.8 e96/81 4:1c59bf4e:::benchmark_data_smithi074_44586_object3206:head v 96'1551, mlcod=96'1543)", 2020-08-07T15:08:42.317 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:42.300705+0000", 2020-08-07T15:08:42.318 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0017354880000000001, 2020-08-07T15:08:42.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017593660000000001, 2020-08-07T15:08:42.318 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:42.319 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:42.319 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:42.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:42.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.300705+0000", 2020-08-07T15:08:42.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:42.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:42.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.300705+0000", 2020-08-07T15:08:42.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1333000000000004e-05 2020-08-07T15:08:42.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:42.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.300766+0000", 2020-08-07T15:08:42.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0729999999999999e-06 2020-08-07T15:08:42.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:42.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.300773+0000", 2020-08-07T15:08:42.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7378000000000001e-05 2020-08-07T15:08:42.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:42.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.300800+0000", 2020-08-07T15:08:42.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6119999999999998e-06 2020-08-07T15:08:42.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:42.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.300806+0000", 2020-08-07T15:08:42.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0445999999999999e-05 2020-08-07T15:08:42.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:42.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.300847+0000", 2020-08-07T15:08:42.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3142e-05 2020-08-07T15:08:42.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:42.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.300870+0000", 2020-08-07T15:08:42.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016498399999999999 2020-08-07T15:08:42.335 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:42.335 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:42.335 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:42.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.336 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:25552 4.a 4:527f9702:::benchmark_data_smithi074_44586_object3193:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e96)", 2020-08-07T15:08:42.339 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:42.234646+0000", 2020-08-07T15:08:42.339 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.067794521999999996, 2020-08-07T15:08:42.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067919409, 2020-08-07T15:08:42.340 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:42.340 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:42.340 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:42.340 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:42.341 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:42.341 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25552 2020-08-07T15:08:42.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.342 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:42.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:42.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.234646+0000", 2020-08-07T15:08:42.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:42.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:42.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.234646+0000", 2020-08-07T15:08:42.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9716000000000003e-05 2020-08-07T15:08:42.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:42.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.234695+0000", 2020-08-07T15:08:42.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5699999999999998e-06 2020-08-07T15:08:42.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:42.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.234702+0000", 2020-08-07T15:08:42.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1917e-05 2020-08-07T15:08:42.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:42.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.234714+0000", 2020-08-07T15:08:42.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7359999999999998e-06 2020-08-07T15:08:42.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:42.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.234720+0000", 2020-08-07T15:08:42.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020258450000000001 2020-08-07T15:08:42.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:42.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.236745+0000", 2020-08-07T15:08:42.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0336e-05 2020-08-07T15:08:42.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:42.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.236806+0000", 2020-08-07T15:08:42.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7215999999999999e-05 2020-08-07T15:08:42.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:42.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.236903+0000", 2020-08-07T15:08:42.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050066629999999997 2020-08-07T15:08:42.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:42.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.241910+0000", 2020-08-07T15:08:42.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8599999999999996e-07 2020-08-07T15:08:42.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:42.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.241911+0000", 2020-08-07T15:08:42.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5183000000000002e-05 2020-08-07T15:08:42.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:42.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.241986+0000", 2020-08-07T15:08:42.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025372820000000001 2020-08-07T15:08:42.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:42.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.244523+0000", 2020-08-07T15:08:42.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0098774499999999994 2020-08-07T15:08:42.358 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:42.358 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:42.358 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:42.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.359 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:25607 4.5 4:a12a5d1f:::benchmark_data_smithi074_44586_object3200:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e96)", 2020-08-07T15:08:42.359 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:42.269965+0000", 2020-08-07T15:08:42.359 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.032475071000000001, 2020-08-07T15:08:42.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032728629000000002, 2020-08-07T15:08:42.359 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:42.360 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:42.360 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:42.360 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:42.360 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:42.360 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25607 2020-08-07T15:08:42.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.361 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:42.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:42.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.269965+0000", 2020-08-07T15:08:42.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:42.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:42.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.269965+0000", 2020-08-07T15:08:42.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4367999999999999e-05 2020-08-07T15:08:42.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:42.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.270019+0000", 2020-08-07T15:08:42.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.562e-06 2020-08-07T15:08:42.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:42.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.270026+0000", 2020-08-07T15:08:42.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3434999999999999e-05 2020-08-07T15:08:42.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:42.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.270039+0000", 2020-08-07T15:08:42.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2790000000000004e-06 2020-08-07T15:08:42.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:42.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.270046+0000", 2020-08-07T15:08:42.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017119629999999999 2020-08-07T15:08:42.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.271758+0000", 2020-08-07T15:08:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7961999999999996e-05 2020-08-07T15:08:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.271846+0000", 2020-08-07T15:08:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016015100000000001 2020-08-07T15:08:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,7", 2020-08-07T15:08:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.272006+0000", 2020-08-07T15:08:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051599870000000004 2020-08-07T15:08:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.277166+0000", 2020-08-07T15:08:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8100000000000001e-07 2020-08-07T15:08:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.277167+0000", 2020-08-07T15:08:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022709 2020-08-07T15:08:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.277394+0000", 2020-08-07T15:08:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039812279999999999 2020-08-07T15:08:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.281375+0000", 2020-08-07T15:08:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011410006 2020-08-07T15:08:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:25608 4.5 4:a12a5d1f:::benchmark_data_smithi074_44586_object3200:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e96)", 2020-08-07T15:08:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:42.270111+0000", 2020-08-07T15:08:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.032328765000000002, 2020-08-07T15:08:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032704723999999998, 2020-08-07T15:08:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 25608 2020-08-07T15:08:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.270111+0000", 2020-08-07T15:08:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.270111+0000", 2020-08-07T15:08:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4030999999999998e-05 2020-08-07T15:08:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.270165+0000", 2020-08-07T15:08:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7799999999999997e-06 2020-08-07T15:08:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.270171+0000", 2020-08-07T15:08:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1994000000000001e-05 2020-08-07T15:08:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.270183+0000", 2020-08-07T15:08:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1710000000000001e-06 2020-08-07T15:08:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.270188+0000", 2020-08-07T15:08:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024391790000000001 2020-08-07T15:08:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.272628+0000", 2020-08-07T15:08:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000128885 2020-08-07T15:08:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.272756+0000", 2020-08-07T15:08:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017516600000000001 2020-08-07T15:08:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,7", 2020-08-07T15:08:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.272932+0000", 2020-08-07T15:08:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004470623 2020-08-07T15:08:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.277402+0000", 2020-08-07T15:08:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9100000000000003e-07 2020-08-07T15:08:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.277403+0000", 2020-08-07T15:08:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1842999999999998e-05 2020-08-07T15:08:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.277455+0000", 2020-08-07T15:08:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039366879999999998 2020-08-07T15:08:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:42.281391+0000", 2020-08-07T15:08:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011280050999999999 2020-08-07T15:08:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:08:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 4 2020-08-07T15:08:42.414 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:42.416 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:08:42.417 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:42.414+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:42.520 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:42.518+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:42.627 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:42.626+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:42.648 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:42.649 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:08:42.649 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:08:42.649 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:08:42.649 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:42.660 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:08:42.696 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:08:42.747 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:42.745+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:42.822 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:08:42.831 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:42.831 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:08:42.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:08:42.832 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:08:42.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.832 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:17824 3.d e27/20 3:bd4e0935:::benchmark_data_smithi074_35447_object2227:head v 27'1168, mlcod=27'1163)", 2020-08-07T15:08:42.832 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:08:42.840 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 114.683070672, 2020-08-07T15:08:42.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25249224199999998, 2020-08-07T15:08:42.841 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:42.841 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:42.841 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:42.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:42.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:08:42.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:42.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:42.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143440+0000", 2020-08-07T15:08:42.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9348000000000003e-05 2020-08-07T15:08:42.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:42.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143490+0000", 2020-08-07T15:08:42.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7800000000000003e-06 2020-08-07T15:08:42.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:42.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.143496+0000", 2020-08-07T15:08:42.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25095783399999999 2020-08-07T15:08:42.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:42.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394454+0000", 2020-08-07T15:08:42.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6550000000000001e-06 2020-08-07T15:08:42.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:42.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394460+0000", 2020-08-07T15:08:42.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8814999999999999e-05 2020-08-07T15:08:42.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:42.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394509+0000", 2020-08-07T15:08:42.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3482999999999999e-05 2020-08-07T15:08:42.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:42.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.394532+0000", 2020-08-07T15:08:42.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013496199999999999 2020-08-07T15:08:42.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:42.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.395882+0000", 2020-08-07T15:08:42.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0707000000000001e-05 2020-08-07T15:08:42.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:42.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:48.395932+0000", 2020-08-07T15:08:42.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25249224199999998 2020-08-07T15:08:42.920 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:42.928 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:42.958 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:42.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.959 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:42104 3.d e32/20 3:bcc35439:::benchmark_data_smithi074_35447_object5262:head v 32'2864, mlcod=32'2858)", 2020-08-07T15:08:42.959 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:08:42.969 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 99.583485374999995, 2020-08-07T15:08:42.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28296418899999998, 2020-08-07T15:08:42.970 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:42.970 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:42.970 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:42.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:42.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:08:42.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:42.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:42.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:08:42.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5550000000000002e-05 2020-08-07T15:08:42.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:42.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243081+0000", 2020-08-07T15:08:42.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8139999999999997e-06 2020-08-07T15:08:42.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:42.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243087+0000", 2020-08-07T15:08:42.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27968446800000002 2020-08-07T15:08:42.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:42.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:42.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:42.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522771+0000", 2020-08-07T15:08:43.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9459999999999997e-06 2020-08-07T15:08:43.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:43.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522776+0000", 2020-08-07T15:08:43.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0778000000000006e-05 2020-08-07T15:08:43.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:43.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522847+0000", 2020-08-07T15:08:43.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0622e-05 2020-08-07T15:08:43.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:43.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522868+0000", 2020-08-07T15:08:43.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030627580000000001 2020-08-07T15:08:43.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:43.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.525930+0000", 2020-08-07T15:08:43.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9252999999999997e-05 2020-08-07T15:08:43.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:43.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.525990+0000", 2020-08-07T15:08:43.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28296418899999998 2020-08-07T15:08:43.019 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.020 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:43.020 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.021 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:48144 3.0 e36/20 3:0cec1d53:::benchmark_data_smithi074_35447_object6017:head v 36'3128, mlcod=36'3120)", 2020-08-07T15:08:43.021 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:08.897123+0000", 2020-08-07T15:08:43.022 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.929387664999993, 2020-08-07T15:08:43.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25221023399999998, 2020-08-07T15:08:43.023 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:43.023 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:43.024 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:43.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:43.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897123+0000", 2020-08-07T15:08:43.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:43.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:43.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897123+0000", 2020-08-07T15:08:43.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0655999999999997e-05 2020-08-07T15:08:43.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:43.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897174+0000", 2020-08-07T15:08:43.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6769999999999998e-06 2020-08-07T15:08:43.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:43.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:08.897181+0000", 2020-08-07T15:08:43.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25090537899999998 2020-08-07T15:08:43.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:43.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148086+0000", 2020-08-07T15:08:43.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5520000000000004e-06 2020-08-07T15:08:43.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:43.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148092+0000", 2020-08-07T15:08:43.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6391999999999999e-05 2020-08-07T15:08:43.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:43.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148138+0000", 2020-08-07T15:08:43.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8124999999999999e-05 2020-08-07T15:08:43.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:43.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.148166+0000", 2020-08-07T15:08:43.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011152009999999999 2020-08-07T15:08:43.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:43.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.149281+0000", 2020-08-07T15:08:43.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2252e-05 2020-08-07T15:08:43.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:43.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.149333+0000", 2020-08-07T15:08:43.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25221023399999998 2020-08-07T15:08:43.042 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.043 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:43.043 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.044 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:48776 3.2 e36/20 3:47029331:::benchmark_data_smithi074_35447_object6096:head v 36'2888, mlcod=36'2881)", 2020-08-07T15:08:43.045 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:08:43.045 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.42199085, 2020-08-07T15:08:43.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27006165399999998, 2020-08-07T15:08:43.046 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:43.046 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:43.046 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:43.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:43.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:08:43.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:43.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:43.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:08:43.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8454000000000001e-05 2020-08-07T15:08:43.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:43.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404569+0000", 2020-08-07T15:08:43.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2349999999999996e-06 2020-08-07T15:08:43.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:43.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404575+0000", 2020-08-07T15:08:43.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26921690999999998 2020-08-07T15:08:43.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:43.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673792+0000", 2020-08-07T15:08:43.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3330000000000002e-06 2020-08-07T15:08:43.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:43.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673798+0000", 2020-08-07T15:08:43.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1013000000000001e-05 2020-08-07T15:08:43.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:43.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673849+0000", 2020-08-07T15:08:43.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7044999999999999e-05 2020-08-07T15:08:43.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:43.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673876+0000", 2020-08-07T15:08:43.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00065173100000000001 2020-08-07T15:08:43.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:43.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.674528+0000", 2020-08-07T15:08:43.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3933e-05 2020-08-07T15:08:43.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:43.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.674582+0000", 2020-08-07T15:08:43.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27006165399999998 2020-08-07T15:08:43.064 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.064 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:43.064 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.066 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:77967 3.2 e46/20 3:48f61a4b:::benchmark_data_smithi074_35447_object9745:head v 46'4807, mlcod=46'4798)", 2020-08-07T15:08:43.067 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:28.702144+0000", 2020-08-07T15:08:43.067 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.124366660000007, 2020-08-07T15:08:43.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26780679000000002, 2020-08-07T15:08:43.068 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:43.068 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:43.068 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:43.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:43.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702144+0000", 2020-08-07T15:08:43.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:43.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:43.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702144+0000", 2020-08-07T15:08:43.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5482000000000002e-05 2020-08-07T15:08:43.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:43.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702200+0000", 2020-08-07T15:08:43.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.216e-06 2020-08-07T15:08:43.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:43.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702207+0000", 2020-08-07T15:08:43.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26634085400000002 2020-08-07T15:08:43.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:43.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.968548+0000", 2020-08-07T15:08:43.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2490000000000001e-06 2020-08-07T15:08:43.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:43.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.968553+0000", 2020-08-07T15:08:43.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7738999999999998e-05 2020-08-07T15:08:43.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:43.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.968601+0000", 2020-08-07T15:08:43.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5783999999999999e-05 2020-08-07T15:08:43.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:43.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.968627+0000", 2020-08-07T15:08:43.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001270471 2020-08-07T15:08:43.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:43.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.969897+0000", 2020-08-07T15:08:43.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3995000000000001e-05 2020-08-07T15:08:43.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:43.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.969951+0000", 2020-08-07T15:08:43.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26780679000000002 2020-08-07T15:08:43.107 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.107 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:43.107 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.108 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:77968 3.2 e46/20 3:48f61a4b:::benchmark_data_smithi074_35447_object9745:head v 46'4808, mlcod=46'4799)", 2020-08-07T15:08:43.108 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:28.702836+0000", 2020-08-07T15:08:43.108 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.123674820000005, 2020-08-07T15:08:43.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27322339400000001, 2020-08-07T15:08:43.111 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:43.112 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:43.112 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:43.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:43.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702836+0000", 2020-08-07T15:08:43.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:43.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:43.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702836+0000", 2020-08-07T15:08:43.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1822e-05 2020-08-07T15:08:43.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:43.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702888+0000", 2020-08-07T15:08:43.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4110000000000001e-06 2020-08-07T15:08:43.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:43.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702894+0000", 2020-08-07T15:08:43.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26750942999999999 2020-08-07T15:08:43.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:43.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970404+0000", 2020-08-07T15:08:43.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1150000000000002e-06 2020-08-07T15:08:43.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:43.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970409+0000", 2020-08-07T15:08:43.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2898000000000007e-05 2020-08-07T15:08:43.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:43.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970472+0000", 2020-08-07T15:08:43.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2311999999999999e-05 2020-08-07T15:08:43.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:43.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970494+0000", 2020-08-07T15:08:43.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055286800000000002 2020-08-07T15:08:43.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:43.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.976023+0000", 2020-08-07T15:08:43.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6726e-05 2020-08-07T15:08:43.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:43.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.976059+0000", 2020-08-07T15:08:43.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27322339400000001 2020-08-07T15:08:43.133 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.133 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:43.134 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.134 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86695 3.a 3:5058ba7c:::benchmark_data_smithi074_35447_object10836:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:08:43.134 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:08:43.135 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.501183404000002, 2020-08-07T15:08:43.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31490022699999998, 2020-08-07T15:08:43.135 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:43.135 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:43.135 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:43.136 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:43.136 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:43.136 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86695 2020-08-07T15:08:43.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.136 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:43.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:43.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:08:43.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:43.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:43.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:08:43.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4560000000000005e-05 2020-08-07T15:08:43.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:43.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325392+0000", 2020-08-07T15:08:43.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3180000000000003e-06 2020-08-07T15:08:43.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:43.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325398+0000", 2020-08-07T15:08:43.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.258e-05 2020-08-07T15:08:43.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:43.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325411+0000", 2020-08-07T15:08:43.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6550000000000001e-06 2020-08-07T15:08:43.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:43.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325417+0000", 2020-08-07T15:08:43.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024282879999999998 2020-08-07T15:08:43.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:43.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.327845+0000", 2020-08-07T15:08:43.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1203000000000001e-05 2020-08-07T15:08:43.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:43.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.327936+0000", 2020-08-07T15:08:43.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000186505 2020-08-07T15:08:43.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:43.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328123+0000", 2020-08-07T15:08:43.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020062890000000001 2020-08-07T15:08:43.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:43.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.330129+0000", 2020-08-07T15:08:43.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121513 2020-08-07T15:08:43.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:43.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.330250+0000", 2020-08-07T15:08:43.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30989746899999998 2020-08-07T15:08:43.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:43.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640148+0000", 2020-08-07T15:08:43.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2139999999999999e-06 2020-08-07T15:08:43.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:43.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640149+0000", 2020-08-07T15:08:43.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8081999999999998e-05 2020-08-07T15:08:43.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:43.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640197+0000", 2020-08-07T15:08:43.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0550999999999999e-05 2020-08-07T15:08:43.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:43.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640228+0000", 2020-08-07T15:08:43.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31490022699999998 2020-08-07T15:08:43.163 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.163 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:43.164 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.164 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86696 3.a 3:5058ba7c:::benchmark_data_smithi074_35447_object10836:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:08:43.165 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:08:43.165 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.501058724999993, 2020-08-07T15:08:43.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.347347556, 2020-08-07T15:08:43.165 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:43.166 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:43.166 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:43.166 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:43.167 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:43.167 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86696 2020-08-07T15:08:43.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.167 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:43.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:43.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:08:43.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:43.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:43.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:08:43.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0436999999999998e-05 2020-08-07T15:08:43.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:43.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325503+0000", 2020-08-07T15:08:43.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4659999999999998e-06 2020-08-07T15:08:43.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:43.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325508+0000", 2020-08-07T15:08:43.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1177e-05 2020-08-07T15:08:43.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:43.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325519+0000", 2020-08-07T15:08:43.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7809999999999996e-06 2020-08-07T15:08:43.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:43.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325524+0000", 2020-08-07T15:08:43.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002989219 2020-08-07T15:08:43.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:43.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328513+0000", 2020-08-07T15:08:43.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0649000000000006e-05 2020-08-07T15:08:43.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:43.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328604+0000", 2020-08-07T15:08:43.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001652 2020-08-07T15:08:43.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:43.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328769+0000", 2020-08-07T15:08:43.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023470639999999998 2020-08-07T15:08:43.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:43.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331116+0000", 2020-08-07T15:08:43.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6099999999999999e-07 2020-08-07T15:08:43.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:43.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331117+0000", 2020-08-07T15:08:43.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9150000000000001e-05 2020-08-07T15:08:43.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:43.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331176+0000", 2020-08-07T15:08:43.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065402581000000001 2020-08-07T15:08:43.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:43.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.396579+0000", 2020-08-07T15:08:43.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27618104599999999 2020-08-07T15:08:43.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:43.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672760+0000", 2020-08-07T15:08:43.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2020-08-07T15:08:43.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:43.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672761+0000", 2020-08-07T15:08:43.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9211000000000001e-05 2020-08-07T15:08:43.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:43.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672800+0000", 2020-08-07T15:08:43.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.347347556 2020-08-07T15:08:43.218 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.219 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:43.219 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.220 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:86695 3.a e52/20 ondisk, result = 0)", 2020-08-07T15:08:43.220 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:08:43.221 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.493102027999996, 2020-08-07T15:08:43.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.306823501, 2020-08-07T15:08:43.221 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:43.222 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:43.222 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:43.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:43.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:08:43.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:43.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:43.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:08:43.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.048e-05 2020-08-07T15:08:43.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:43.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333439+0000", 2020-08-07T15:08:43.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9990000000000002e-06 2020-08-07T15:08:43.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:43.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333443+0000", 2020-08-07T15:08:43.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30664195100000002 2020-08-07T15:08:43.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:43.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640085+0000", 2020-08-07T15:08:43.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8559999999999998e-06 2020-08-07T15:08:43.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:43.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640090+0000", 2020-08-07T15:08:43.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5166000000000001e-05 2020-08-07T15:08:43.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:43.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640135+0000", 2020-08-07T15:08:43.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4160000000000001e-06 2020-08-07T15:08:43.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:43.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640142+0000", 2020-08-07T15:08:43.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0632999999999998e-05 2020-08-07T15:08:43.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:43.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640232+0000", 2020-08-07T15:08:43.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.306823501 2020-08-07T15:08:43.253 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.253 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:43.254 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.255 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:104144 3.a 3:5c46cbda:::benchmark_data_smithi074_35447_object13017:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:08:43.256 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:45.878313+0000", 2020-08-07T15:08:43.257 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.948197968999999, 2020-08-07T15:08:43.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.251430239, 2020-08-07T15:08:43.257 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:43.258 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:43.258 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:43.258 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:43.258 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:43.259 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 104144 2020-08-07T15:08:43.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.260 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:43.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:43.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.878313+0000", 2020-08-07T15:08:43.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:43.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:43.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.878313+0000", 2020-08-07T15:08:43.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5476000000000001e-05 2020-08-07T15:08:43.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:43.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.878338+0000", 2020-08-07T15:08:43.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.241e-06 2020-08-07T15:08:43.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:43.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.878342+0000", 2020-08-07T15:08:43.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1210000000000007e-06 2020-08-07T15:08:43.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:43.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.878350+0000", 2020-08-07T15:08:43.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5470000000000001e-06 2020-08-07T15:08:43.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:43.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.878353+0000", 2020-08-07T15:08:43.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002441992 2020-08-07T15:08:43.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:43.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.880795+0000", 2020-08-07T15:08:43.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4197999999999995e-05 2020-08-07T15:08:43.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:43.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.880890+0000", 2020-08-07T15:08:43.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000163864 2020-08-07T15:08:43.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:43.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.881053+0000", 2020-08-07T15:08:43.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005483768 2020-08-07T15:08:43.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:43.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886537+0000", 2020-08-07T15:08:43.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7000000000000003e-07 2020-08-07T15:08:43.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:43.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886538+0000", 2020-08-07T15:08:43.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6101999999999998e-05 2020-08-07T15:08:43.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:43.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886604+0000", 2020-08-07T15:08:43.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19508814899999999 2020-08-07T15:08:43.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:43.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.081692+0000", 2020-08-07T15:08:43.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047975891999999999 2020-08-07T15:08:43.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:43.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.129668+0000", 2020-08-07T15:08:43.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.079e-06 2020-08-07T15:08:43.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:43.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.129669+0000", 2020-08-07T15:08:43.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.394e-05 2020-08-07T15:08:43.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:43.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.129743+0000", 2020-08-07T15:08:43.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.251430239 2020-08-07T15:08:43.292 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.292 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:43.293 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.293 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:104144 3.a e58/20 ondisk, result = 0)", 2020-08-07T15:08:43.294 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:45.886622+0000", 2020-08-07T15:08:43.294 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.939888420000003, 2020-08-07T15:08:43.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24340332200000001, 2020-08-07T15:08:43.295 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:43.295 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:43.295 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:43.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:43.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886622+0000", 2020-08-07T15:08:43.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:43.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:43.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886622+0000", 2020-08-07T15:08:43.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2391999999999998e-05 2020-08-07T15:08:43.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:43.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886655+0000", 2020-08-07T15:08:43.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6959999999999996e-06 2020-08-07T15:08:43.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:43.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.886660+0000", 2020-08-07T15:08:43.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.240856029 2020-08-07T15:08:43.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:43.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.127516+0000", 2020-08-07T15:08:43.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9840000000000001e-06 2020-08-07T15:08:43.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:43.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.127521+0000", 2020-08-07T15:08:43.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002130032 2020-08-07T15:08:43.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:43.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.129651+0000", 2020-08-07T15:08:43.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6780000000000006e-06 2020-08-07T15:08:43.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:43.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.129659+0000", 2020-08-07T15:08:43.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00036651099999999997 2020-08-07T15:08:43.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:43.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.130026+0000", 2020-08-07T15:08:43.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24340332200000001 2020-08-07T15:08:43.325 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.326 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:43.326 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.327 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106232 3.a 3:5599d7e6:::benchmark_data_smithi074_35447_object13278:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:08:43.328 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:08:43.329 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.480903777999998, 2020-08-07T15:08:43.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52578228699999996, 2020-08-07T15:08:43.330 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:43.331 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:43.331 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:43.331 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:43.332 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:43.332 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106232 2020-08-07T15:08:43.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.333 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:43.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:43.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:08:43.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:43.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:43.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:08:43.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4214999999999997e-05 2020-08-07T15:08:43.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:43.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345641+0000", 2020-08-07T15:08:43.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9279999999999993e-06 2020-08-07T15:08:43.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:43.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345649+0000", 2020-08-07T15:08:43.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4391e-05 2020-08-07T15:08:43.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:43.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345664+0000", 2020-08-07T15:08:43.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7170000000000002e-06 2020-08-07T15:08:43.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:43.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345670+0000", 2020-08-07T15:08:43.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.459e-05 2020-08-07T15:08:43.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:43.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345695+0000", 2020-08-07T15:08:43.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6070999999999999e-05 2020-08-07T15:08:43.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:43.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345751+0000", 2020-08-07T15:08:43.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010069300000000001 2020-08-07T15:08:43.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:43.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345852+0000", 2020-08-07T15:08:43.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013512299999999999 2020-08-07T15:08:43.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:43.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.347203+0000", 2020-08-07T15:08:43.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0844000000000001e-05 2020-08-07T15:08:43.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:43.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.347274+0000", 2020-08-07T15:08:43.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52399471600000003 2020-08-07T15:08:43.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:43.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871269+0000", 2020-08-07T15:08:43.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.311e-06 2020-08-07T15:08:43.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:43.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871270+0000", 2020-08-07T15:08:43.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3509000000000004e-05 2020-08-07T15:08:43.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:43.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871343+0000", 2020-08-07T15:08:43.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6072000000000002e-05 2020-08-07T15:08:43.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:43.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871389+0000", 2020-08-07T15:08:43.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52578228699999996 2020-08-07T15:08:43.358 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.359 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:43.359 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.360 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106232 3.a e59/20 ondisk, result = 0)", 2020-08-07T15:08:43.361 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:08:43.361 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.477857602999997, 2020-08-07T15:08:43.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52274359299999995, 2020-08-07T15:08:43.362 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:43.363 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:43.363 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:43.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:43.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:08:43.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:43.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:43.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:08:43.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5011000000000001e-05 2020-08-07T15:08:43.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:43.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348668+0000", 2020-08-07T15:08:43.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1959999999999998e-06 2020-08-07T15:08:43.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:43.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348671+0000", 2020-08-07T15:08:43.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52253456499999995 2020-08-07T15:08:43.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:43.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871205+0000", 2020-08-07T15:08:43.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0230000000000003e-06 2020-08-07T15:08:43.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:43.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871209+0000", 2020-08-07T15:08:43.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0708999999999999e-05 2020-08-07T15:08:43.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:43.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871250+0000", 2020-08-07T15:08:43.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0158e-05 2020-08-07T15:08:43.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:43.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871260+0000", 2020-08-07T15:08:43.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013693099999999999 2020-08-07T15:08:43.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:43.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871397+0000", 2020-08-07T15:08:43.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52274359299999995 2020-08-07T15:08:43.382 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.383 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:43.383 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.384 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:127864 3.a 3:58746ca8:::benchmark_data_smithi074_35447_object15982:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e70)", 2020-08-07T15:08:43.384 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:04.819979+0000", 2020-08-07T15:08:43.385 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 38.006531893999998, 2020-08-07T15:08:43.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27732920700000002, 2020-08-07T15:08:43.386 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:43.386 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:43.386 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:43.387 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:43.388 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:43.388 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 127864 2020-08-07T15:08:43.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.389 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:43.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:43.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.819979+0000", 2020-08-07T15:08:43.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:43.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:43.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.819979+0000", 2020-08-07T15:08:43.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7188000000000001e-05 2020-08-07T15:08:43.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:43.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820006+0000", 2020-08-07T15:08:43.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0940000000000001e-06 2020-08-07T15:08:43.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:43.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820010+0000", 2020-08-07T15:08:43.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0869999999999996e-06 2020-08-07T15:08:43.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:43.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820019+0000", 2020-08-07T15:08:43.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7509999999999999e-06 2020-08-07T15:08:43.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:43.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820024+0000", 2020-08-07T15:08:43.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5089e-05 2020-08-07T15:08:43.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:43.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820049+0000", 2020-08-07T15:08:43.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0068999999999995e-05 2020-08-07T15:08:43.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:43.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820129+0000", 2020-08-07T15:08:43.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013952100000000001 2020-08-07T15:08:43.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:43.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820269+0000", 2020-08-07T15:08:43.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031575290000000001 2020-08-07T15:08:43.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:43.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.823426+0000", 2020-08-07T15:08:43.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4900000000000004e-07 2020-08-07T15:08:43.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:43.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.823427+0000", 2020-08-07T15:08:43.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2324999999999998e-05 2020-08-07T15:08:43.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:43.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.823480+0000", 2020-08-07T15:08:43.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.272535737 2020-08-07T15:08:43.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:43.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.096015+0000", 2020-08-07T15:08:43.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012371560000000001 2020-08-07T15:08:43.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:43.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097253+0000", 2020-08-07T15:08:43.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4000000000000001e-07 2020-08-07T15:08:43.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:43.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097253+0000", 2020-08-07T15:08:43.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5071999999999997e-05 2020-08-07T15:08:43.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:43.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097308+0000", 2020-08-07T15:08:43.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27732920700000002 2020-08-07T15:08:43.468 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.468 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:43.468 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.469 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:127864 3.a e70/20 ondisk, result = 0)", 2020-08-07T15:08:43.474 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:04.822639+0000", 2020-08-07T15:08:43.474 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 38.003872332, 2020-08-07T15:08:43.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27467429799999998, 2020-08-07T15:08:43.475 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:43.475 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:43.476 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:43.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:43.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.822639+0000", 2020-08-07T15:08:43.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:43.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:43.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.822639+0000", 2020-08-07T15:08:43.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3278999999999998e-05 2020-08-07T15:08:43.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:43.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.822672+0000", 2020-08-07T15:08:43.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1489999999999999e-06 2020-08-07T15:08:43.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:43.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.822678+0000", 2020-08-07T15:08:43.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.274520551 2020-08-07T15:08:43.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:43.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097199+0000", 2020-08-07T15:08:43.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6940000000000001e-06 2020-08-07T15:08:43.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:43.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097202+0000", 2020-08-07T15:08:43.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3429999999999997e-05 2020-08-07T15:08:43.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:43.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097236+0000", 2020-08-07T15:08:43.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7390000000000005e-06 2020-08-07T15:08:43.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:43.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097244+0000", 2020-08-07T15:08:43.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8455999999999996e-05 2020-08-07T15:08:43.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:43.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097313+0000", 2020-08-07T15:08:43.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27467429799999998 2020-08-07T15:08:43.547 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.547 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:43.548 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.549 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:127871 3.2 e70/20 3:405b2522:::benchmark_data_smithi074_35447_object15983:head v 70'8079, mlcod=70'8071)", 2020-08-07T15:08:43.550 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:04.826119+0000", 2020-08-07T15:08:43.550 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 38.000392003000002, 2020-08-07T15:08:43.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25781899600000002, 2020-08-07T15:08:43.551 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:43.551 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:43.551 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:43.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:43.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.826119+0000", 2020-08-07T15:08:43.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:43.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:43.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.826119+0000", 2020-08-07T15:08:43.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1702000000000001e-05 2020-08-07T15:08:43.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:43.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.826171+0000", 2020-08-07T15:08:43.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9230000000000002e-06 2020-08-07T15:08:43.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:43.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.826179+0000", 2020-08-07T15:08:43.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25632726 2020-08-07T15:08:43.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:43.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.082506+0000", 2020-08-07T15:08:43.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8739999999999997e-06 2020-08-07T15:08:43.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:43.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.082513+0000", 2020-08-07T15:08:43.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2123000000000003e-05 2020-08-07T15:08:43.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:43.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.082555+0000", 2020-08-07T15:08:43.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6835e-05 2020-08-07T15:08:43.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:43.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.082582+0000", 2020-08-07T15:08:43.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012973889999999999 2020-08-07T15:08:43.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:43.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.083879+0000", 2020-08-07T15:08:43.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.889e-05 2020-08-07T15:08:43.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:43.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.083938+0000", 2020-08-07T15:08:43.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25781899600000002 2020-08-07T15:08:43.580 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.580 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:43.581 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.581 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:127872 3.2 e70/20 3:405b2522:::benchmark_data_smithi074_35447_object15983:head v 70'8080, mlcod=70'8071)", 2020-08-07T15:08:43.584 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:04.826865+0000", 2020-08-07T15:08:43.584 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.999646235999997, 2020-08-07T15:08:43.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26030899299999999, 2020-08-07T15:08:43.585 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:43.585 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:43.586 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:43.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:43.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.826865+0000", 2020-08-07T15:08:43.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:43.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:43.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.826865+0000", 2020-08-07T15:08:43.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2008000000000001e-05 2020-08-07T15:08:43.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:43.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.826917+0000", 2020-08-07T15:08:43.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0510000000000002e-06 2020-08-07T15:08:43.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:43.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.826923+0000", 2020-08-07T15:08:43.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25807582499999998 2020-08-07T15:08:43.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:43.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.084999+0000", 2020-08-07T15:08:43.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8619999999999999e-06 2020-08-07T15:08:43.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:43.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.085004+0000", 2020-08-07T15:08:43.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106156 2020-08-07T15:08:43.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:43.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.085111+0000", 2020-08-07T15:08:43.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3410000000000001e-05 2020-08-07T15:08:43.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:43.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.085134+0000", 2020-08-07T15:08:43.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019880459999999998 2020-08-07T15:08:43.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:43.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.087122+0000", 2020-08-07T15:08:43.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1635000000000003e-05 2020-08-07T15:08:43.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:43.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.087174+0000", 2020-08-07T15:08:43.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26030899299999999 2020-08-07T15:08:43.644 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.644 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:43.645 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.646 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:145796 3.a 3:54d72af7:::benchmark_data_smithi074_35447_object16187:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:08:43.646 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.668714+0000", 2020-08-07T15:08:43.646 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.157796873999999, 2020-08-07T15:08:43.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24066032200000001, 2020-08-07T15:08:43.647 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:43.647 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:43.648 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:43.648 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:43.648 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:43.649 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 145796 2020-08-07T15:08:43.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.649 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:43.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:43.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.668714+0000", 2020-08-07T15:08:43.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:43.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:43.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.668714+0000", 2020-08-07T15:08:43.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100949 2020-08-07T15:08:43.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:43.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.668815+0000", 2020-08-07T15:08:43.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.99e-06 2020-08-07T15:08:43.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:43.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.668822+0000", 2020-08-07T15:08:43.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.349e-05 2020-08-07T15:08:43.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:43.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.668835+0000", 2020-08-07T15:08:43.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.489e-06 2020-08-07T15:08:43.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:43.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.668842+0000", 2020-08-07T15:08:43.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2604999999999997e-05 2020-08-07T15:08:43.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:43.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.668875+0000", 2020-08-07T15:08:43.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015340500000000001 2020-08-07T15:08:43.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:43.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.669028+0000", 2020-08-07T15:08:43.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013534199999999999 2020-08-07T15:08:43.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:43.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.669163+0000", 2020-08-07T15:08:43.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000575414 2020-08-07T15:08:43.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:43.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.669739+0000", 2020-08-07T15:08:43.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2500000000000004e-07 2020-08-07T15:08:43.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:43.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.669740+0000", 2020-08-07T15:08:43.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9196e-05 2020-08-07T15:08:43.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:43.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.669799+0000", 2020-08-07T15:08:43.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016268109999999999 2020-08-07T15:08:43.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:43.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671426+0000", 2020-08-07T15:08:43.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.23782094200000001 2020-08-07T15:08:43.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:43.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.909246+0000", 2020-08-07T15:08:43.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6699999999999998e-06 2020-08-07T15:08:43.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:43.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.909249+0000", 2020-08-07T15:08:43.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012519399999999999 2020-08-07T15:08:43.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:43.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.909374+0000", 2020-08-07T15:08:43.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24066032200000001 2020-08-07T15:08:43.679 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.679 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:43.680 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.681 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:145807 3.c e75/20 3:39741db6:::benchmark_data_smithi074_35447_object16198:head v 75'9594, mlcod=75'9590)", 2020-08-07T15:08:43.681 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.674086+0000", 2020-08-07T15:08:43.681 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.152425386000001, 2020-08-07T15:08:43.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40268798500000003, 2020-08-07T15:08:43.682 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:43.683 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:43.684 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:43.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:43.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674086+0000", 2020-08-07T15:08:43.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:43.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:43.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674086+0000", 2020-08-07T15:08:43.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3321000000000001e-05 2020-08-07T15:08:43.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:43.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674129+0000", 2020-08-07T15:08:43.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1919999999999999e-06 2020-08-07T15:08:43.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:43.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674136+0000", 2020-08-07T15:08:43.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40193062899999998 2020-08-07T15:08:43.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:43.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076067+0000", 2020-08-07T15:08:43.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3579e-05 2020-08-07T15:08:43.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:43.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076080+0000", 2020-08-07T15:08:43.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3733999999999998e-05 2020-08-07T15:08:43.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:43.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076124+0000", 2020-08-07T15:08:43.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0145e-05 2020-08-07T15:08:43.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:43.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076154+0000", 2020-08-07T15:08:43.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00055768600000000005 2020-08-07T15:08:43.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:43.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076712+0000", 2020-08-07T15:08:43.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1699000000000007e-05 2020-08-07T15:08:43.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:43.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076774+0000", 2020-08-07T15:08:43.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40268798500000003 2020-08-07T15:08:43.694 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.694 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:43.694 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.696 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:144 4.5 4:aed0a56a:::benchmark_data_smithi074_44586_object17:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:08:43.696 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.396584+0000", 2020-08-07T15:08:43.696 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.429927147000001, 2020-08-07T15:08:43.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45809661000000002, 2020-08-07T15:08:43.697 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:43.697 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:43.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:43.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 144 2020-08-07T15:08:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396584+0000", 2020-08-07T15:08:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396584+0000", 2020-08-07T15:08:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4195e-05 2020-08-07T15:08:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396628+0000", 2020-08-07T15:08:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.157e-06 2020-08-07T15:08:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396631+0000", 2020-08-07T15:08:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.832e-06 2020-08-07T15:08:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396638+0000", 2020-08-07T15:08:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.745e-06 2020-08-07T15:08:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396641+0000", 2020-08-07T15:08:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001659368 2020-08-07T15:08:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.398300+0000", 2020-08-07T15:08:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2652000000000007e-05 2020-08-07T15:08:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.398393+0000", 2020-08-07T15:08:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016763699999999999 2020-08-07T15:08:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,7", 2020-08-07T15:08:43.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.398560+0000", 2020-08-07T15:08:43.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023917719999999999 2020-08-07T15:08:43.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:43.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.400952+0000", 2020-08-07T15:08:43.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999997e-07 2020-08-07T15:08:43.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:43.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.400953+0000", 2020-08-07T15:08:43.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9373999999999998e-05 2020-08-07T15:08:43.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:43.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.401012+0000", 2020-08-07T15:08:43.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003107865 2020-08-07T15:08:43.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:43.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.404120+0000", 2020-08-07T15:08:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45049484299999998 2020-08-07T15:08:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854615+0000", 2020-08-07T15:08:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.035e-06 2020-08-07T15:08:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854616+0000", 2020-08-07T15:08:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4249000000000001e-05 2020-08-07T15:08:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854680+0000", 2020-08-07T15:08:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45809661000000002 2020-08-07T15:08:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:43.766 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:43.795 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:08:43.800 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 17 16 27395 27379 12.5814 13.9141 0.0073173 0.00992508 2020-08-07T15:08:43.800 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:08:43.801 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:08:43.803 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:08:43.804 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:08:43.804 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:08:43.805 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:08:43.805 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:08:43.805 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:08:43.806 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:08:43.809 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:08:43.810 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:08:43.810 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:08:43.811 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:08:43.811 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:08:43.811 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:08:43.812 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:08:43.813 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:08:43.814 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:43.137+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:43.838 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:43.837+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:43.907 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:43.905+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:44.002 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:44.002 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:44.013 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:44.010+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:44.033 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:08:44.123 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:44.118+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:44.222 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:44.216+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:44.246 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:44.246 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:44.263 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:08:44.273 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 18 16 28986 28970 12.5729 12.4297 0.00838829 0.00993246 2020-08-07T15:08:44.360 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:44.355+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:44.420 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:44.419+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:44.481 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:44.482 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:44.496 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:08:44.555 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:44.549+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:44.623 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:44.622+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:44.677 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:44.679 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:44.695 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:08:44.696 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:08:44.696 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 0.000000 2020-08-07T15:08:44.696 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd primary-affinity 4 0 2020-08-07T15:08:44.701 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:08:44.730 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:44.725+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:44.839 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:44.833+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:44.949 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:44.936+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:44.970 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:44.971 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:44.983 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:08:45.039 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:45.037+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:45.157 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:45.172 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:45.172 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:45.138+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:45.174 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:08:45.252 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:45.248+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:45.278 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 19 16 30329 30313 12.4633 10.4922 0.00212268 0.0100137 2020-08-07T15:08:45.415 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:45.415 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:45.416 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:08:45.417 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:45.393+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:45.522 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:45.518+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:45.613 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:45.610+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:45.622 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:45.622 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:45.635 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:08:45.691 INFO:teuthology.orchestra.run.smithi074.stderr:set osd.4 primary-affinity to 0 (802) 2020-08-07T15:08:45.766 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:45.756+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:45.816 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:08:45.822 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:08:45.823 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:45.823 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:45.823 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:45.816+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:45.837 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:08:45.935 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:45.934+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:46.009 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:08:46.028 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:46.027+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:46.029 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:46.033 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:46.035 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:08:46.048 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:08:46.065 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:08:46.090 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:08:46.114 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:08:46.139 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:46.135+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:46.140 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:08:46.164 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:08:46.194 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:08:46.194 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:08:46.194 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:08:46.195 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:08:46.195 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:08:46.195 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:08:46.195 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:08:46.195 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:08:46.196 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:08:46.215 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:46.216 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:08:46.216 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:08:46.216 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:08:46.216 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:46.217 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:08:46.251 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:46.246+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:46.278 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:2020-08-07T15:08:46.273486+0000 min lat: 0.00108947 max lat: 0.516324 avg lat: 0.0100619 2020-08-07T15:08:46.279 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-08-07T15:08:46.279 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 20 16 30984 30968 12.096 5.11719 0.455498 0.0100619 2020-08-07T15:08:46.358 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:46.348+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:46.403 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:46.403 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:08:46.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:08:46.404 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:08:46.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.406 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:8112 3.9 e24/20 3:9b275d48:::benchmark_data_smithi074_35447_object1013:head v 24'416, mlcod=24'408)", 2020-08-07T15:08:46.407 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.085042+0000", 2020-08-07T15:08:46.407 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 125.297046416, 2020-08-07T15:08:46.408 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31443637000000002, 2020-08-07T15:08:46.408 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:46.408 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:46.409 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:46.412 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.413 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:46.413 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085042+0000", 2020-08-07T15:08:46.414 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:46.414 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.415 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:46.417 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085042+0000", 2020-08-07T15:08:46.417 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4534000000000003e-05 2020-08-07T15:08:46.418 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.418 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.419 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:46.419 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085096+0000", 2020-08-07T15:08:46.419 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5789999999999998e-06 2020-08-07T15:08:46.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.426 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:46.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085104+0000", 2020-08-07T15:08:46.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31316244700000001 2020-08-07T15:08:46.427 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:46.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398266+0000", 2020-08-07T15:08:46.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3019999999999996e-06 2020-08-07T15:08:46.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.429 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:46.430 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398270+0000", 2020-08-07T15:08:46.430 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3467000000000002e-05 2020-08-07T15:08:46.432 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.432 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.433 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:46.433 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398314+0000", 2020-08-07T15:08:46.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8184000000000001e-05 2020-08-07T15:08:46.434 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.434 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.434 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:46.435 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398332+0000", 2020-08-07T15:08:46.435 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010908059999999999 2020-08-07T15:08:46.435 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.437 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.438 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:46.439 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.399423+0000", 2020-08-07T15:08:46.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5050999999999999e-05 2020-08-07T15:08:46.439 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.439 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.440 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:46.440 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.399478+0000", 2020-08-07T15:08:46.440 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31443637000000002 2020-08-07T15:08:46.440 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:46.441 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:46.441 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:46.441 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.442 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.442 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:16223 3.9 e27/20 3:9c72cb85:::benchmark_data_smithi074_35447_object2027:head v 27'887, mlcod=27'881)", 2020-08-07T15:08:46.442 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.111666+0000", 2020-08-07T15:08:46.442 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 119.270421564, 2020-08-07T15:08:46.442 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.48138790999999997, 2020-08-07T15:08:46.443 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:46.443 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:46.443 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:46.443 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.443 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:46.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111666+0000", 2020-08-07T15:08:46.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:46.444 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.450 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:46.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111666+0000", 2020-08-07T15:08:46.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3168999999999999e-05 2020-08-07T15:08:46.451 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.451 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:46.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111720+0000", 2020-08-07T15:08:46.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6309999999999997e-06 2020-08-07T15:08:46.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.453 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:46.453 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111726+0000", 2020-08-07T15:08:46.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.47995092 2020-08-07T15:08:46.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.463 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:46.463 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591677+0000", 2020-08-07T15:08:46.466 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9879999999999998e-06 2020-08-07T15:08:46.466 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.467 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.467 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:46.467 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591684+0000", 2020-08-07T15:08:46.468 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5562000000000002e-05 2020-08-07T15:08:46.468 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.468 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.478 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:46.479 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591730+0000", 2020-08-07T15:08:46.480 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4731000000000001e-05 2020-08-07T15:08:46.480 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.481 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:46.481 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591754+0000", 2020-08-07T15:08:46.481 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012412739999999999 2020-08-07T15:08:46.482 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.482 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.498 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:46.498 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.592996+0000", 2020-08-07T15:08:46.499 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8634999999999997e-05 2020-08-07T15:08:46.499 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.499 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.499 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:46.499 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.593054+0000", 2020-08-07T15:08:46.500 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.48138790999999997 2020-08-07T15:08:46.500 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:46.500 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:46.500 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:46.500 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.501 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.501 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:16224 3.9 e27/20 3:9c72cb85:::benchmark_data_smithi074_35447_object2027:head v 27'888, mlcod=27'881)", 2020-08-07T15:08:46.501 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.114073+0000", 2020-08-07T15:08:46.501 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 119.268014837, 2020-08-07T15:08:46.501 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.50478892600000003, 2020-08-07T15:08:46.521 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:46.521 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:46.522 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:46.522 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.530 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:46.531 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114073+0000", 2020-08-07T15:08:46.542 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:46.543 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.543 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.544 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:46.544 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114073+0000", 2020-08-07T15:08:46.544 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3088999999999999e-05 2020-08-07T15:08:46.544 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.544 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:46.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114126+0000", 2020-08-07T15:08:46.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1629999999999999e-06 2020-08-07T15:08:46.545 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.546 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.546 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:46.546 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114133+0000", 2020-08-07T15:08:46.546 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.50357425199999994 2020-08-07T15:08:46.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.575 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:46.577 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617708+0000", 2020-08-07T15:08:46.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9160000000000004e-06 2020-08-07T15:08:46.590 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.590 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.591 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:46.591 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617715+0000", 2020-08-07T15:08:46.592 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8770999999999998e-05 2020-08-07T15:08:46.592 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.592 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.592 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:46.592 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617763+0000", 2020-08-07T15:08:46.593 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.544e-05 2020-08-07T15:08:46.593 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.593 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.593 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:46.593 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617789+0000", 2020-08-07T15:08:46.594 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010210359999999999 2020-08-07T15:08:46.594 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.594 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.594 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:46.594 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.618810+0000", 2020-08-07T15:08:46.595 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2259000000000002e-05 2020-08-07T15:08:46.595 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.595 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.595 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:46.595 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.618862+0000", 2020-08-07T15:08:46.596 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.50478892600000003 2020-08-07T15:08:46.596 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:46.596 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:46.596 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:46.596 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.596 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.597 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:34624 3.9 e30/20 3:946cdb13:::benchmark_data_smithi074_35447_object4327:head v 30'2016, mlcod=30'2009)", 2020-08-07T15:08:46.597 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:58.746375+0000", 2020-08-07T15:08:46.597 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 107.635713193, 2020-08-07T15:08:46.597 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28564679799999998, 2020-08-07T15:08:46.598 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:46.598 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:46.599 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:46.599 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.599 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:46.599 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746375+0000", 2020-08-07T15:08:46.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:46.600 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.600 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.600 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:46.601 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746375+0000", 2020-08-07T15:08:46.601 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3674000000000001e-05 2020-08-07T15:08:46.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.602 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.602 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:46.602 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746429+0000", 2020-08-07T15:08:46.603 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9349999999999997e-06 2020-08-07T15:08:46.603 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.603 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:46.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746435+0000", 2020-08-07T15:08:46.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28165817700000001 2020-08-07T15:08:46.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:46.605 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028094+0000", 2020-08-07T15:08:46.605 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.512e-06 2020-08-07T15:08:46.605 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.606 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.606 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:46.607 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028097+0000", 2020-08-07T15:08:46.607 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7904999999999997e-05 2020-08-07T15:08:46.607 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.608 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.608 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:46.608 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028135+0000", 2020-08-07T15:08:46.621 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6243e-05 2020-08-07T15:08:46.621 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.621 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:46.647 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028151+0000", 2020-08-07T15:08:46.647 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003818372 2020-08-07T15:08:46.647 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.648 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.648 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:46.648 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.031970+0000", 2020-08-07T15:08:46.648 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.198e-05 2020-08-07T15:08:46.648 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.649 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.649 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:46.649 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.032022+0000", 2020-08-07T15:08:46.649 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28564679799999998 2020-08-07T15:08:46.649 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:46.669 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:46.669 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:46.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.670 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:78016 3.9 e46/20 3:921b23e9:::benchmark_data_smithi074_35447_object9751:head v 46'4512, mlcod=46'4506)", 2020-08-07T15:08:46.670 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:28.734571+0000", 2020-08-07T15:08:46.670 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 77.647516791000001, 2020-08-07T15:08:46.670 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23216004300000001, 2020-08-07T15:08:46.670 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:46.671 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:46.671 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:46.671 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.671 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:46.671 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.734571+0000", 2020-08-07T15:08:46.672 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:46.672 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.672 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.672 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:46.672 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.734571+0000", 2020-08-07T15:08:46.673 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7376000000000003e-05 2020-08-07T15:08:46.673 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.673 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.673 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:46.673 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.734609+0000", 2020-08-07T15:08:46.711 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3640000000000004e-06 2020-08-07T15:08:46.712 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.712 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.713 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:46.717 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.734615+0000", 2020-08-07T15:08:46.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.228692642 2020-08-07T15:08:46.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.718 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:46.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.963308+0000", 2020-08-07T15:08:46.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9639999999999997e-06 2020-08-07T15:08:46.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:46.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.963312+0000", 2020-08-07T15:08:46.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5215000000000001e-05 2020-08-07T15:08:46.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.721 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:46.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.963347+0000", 2020-08-07T15:08:46.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8610999999999999e-05 2020-08-07T15:08:46.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.723 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:46.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.963365+0000", 2020-08-07T15:08:46.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033060149999999998 2020-08-07T15:08:46.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.724 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:46.724 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.966671+0000", 2020-08-07T15:08:46.724 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9855999999999997e-05 2020-08-07T15:08:46.724 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.725 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:46.725 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:28.966731+0000", 2020-08-07T15:08:46.725 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23216004300000001 2020-08-07T15:08:46.725 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:46.726 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:46.726 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:46.726 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.726 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.726 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:104111 3.9 e58/20 3:99770d45:::benchmark_data_smithi074_35447_object13013:head v 58'6207, mlcod=58'6197)", 2020-08-07T15:08:46.727 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:45.861256+0000", 2020-08-07T15:08:46.727 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 60.520831977, 2020-08-07T15:08:46.730 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29017704, 2020-08-07T15:08:46.730 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:46.731 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:46.731 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:46.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.731 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:46.732 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861256+0000", 2020-08-07T15:08:46.732 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:46.732 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.732 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:46.733 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861256+0000", 2020-08-07T15:08:46.733 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013044700000000001 2020-08-07T15:08:46.733 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.748 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.748 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:46.749 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861387+0000", 2020-08-07T15:08:46.749 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7533000000000001e-05 2020-08-07T15:08:46.749 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.749 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.749 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:46.750 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861424+0000", 2020-08-07T15:08:46.751 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28788376900000001 2020-08-07T15:08:46.752 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.752 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:46.752 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.149308+0000", 2020-08-07T15:08:46.752 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1890000000000004e-06 2020-08-07T15:08:46.753 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.753 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.753 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:46.753 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.149314+0000", 2020-08-07T15:08:46.753 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8288999999999999e-05 2020-08-07T15:08:46.754 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.754 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.754 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:46.754 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.149352+0000", 2020-08-07T15:08:46.754 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6765999999999998e-05 2020-08-07T15:08:46.755 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.755 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:46.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.149369+0000", 2020-08-07T15:08:46.755 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0020281000000000001 2020-08-07T15:08:46.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:46.756 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.151397+0000", 2020-08-07T15:08:46.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5947e-05 2020-08-07T15:08:46.760 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.760 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.760 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:46.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.151433+0000", 2020-08-07T15:08:46.771 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29017704 2020-08-07T15:08:46.773 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:46.773 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:46.773 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:46.773 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.774 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.774 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:104112 3.9 e58/20 3:99770d45:::benchmark_data_smithi074_35447_object13013:head v 58'6208, mlcod=58'6198)", 2020-08-07T15:08:46.774 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:45.862801+0000", 2020-08-07T15:08:46.774 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 60.519286794999999, 2020-08-07T15:08:46.774 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29018473099999997, 2020-08-07T15:08:46.775 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:46.775 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:46.775 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:46.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.775 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:46.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.862801+0000", 2020-08-07T15:08:46.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:46.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:46.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.862801+0000", 2020-08-07T15:08:46.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0234e-05 2020-08-07T15:08:46.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:46.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.862851+0000", 2020-08-07T15:08:46.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2700000000000001e-06 2020-08-07T15:08:46.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:46.779 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.862858+0000", 2020-08-07T15:08:46.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28821313100000001 2020-08-07T15:08:46.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.779 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:46.780 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.151071+0000", 2020-08-07T15:08:46.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4140000000000002e-06 2020-08-07T15:08:46.780 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.780 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:46.781 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.151076+0000", 2020-08-07T15:08:46.781 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7945999999999999e-05 2020-08-07T15:08:46.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.781 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:46.782 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.151114+0000", 2020-08-07T15:08:46.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2900000000000001e-05 2020-08-07T15:08:46.783 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:46.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.151137+0000", 2020-08-07T15:08:46.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001801415 2020-08-07T15:08:46.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:46.787 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.152939+0000", 2020-08-07T15:08:46.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7420999999999999e-05 2020-08-07T15:08:46.787 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.788 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:46.788 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.152986+0000", 2020-08-07T15:08:46.788 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29018473099999997 2020-08-07T15:08:46.789 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:46.789 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:46.790 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:46.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.791 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106080 3.4 3:208b1d5b:::benchmark_data_smithi074_35447_object13259:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:08:46.791 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.241181+0000", 2020-08-07T15:08:46.792 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 59.140907347000002, 2020-08-07T15:08:46.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45274565500000002, 2020-08-07T15:08:46.794 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:46.794 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:46.794 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:46.794 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:46.795 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:46.795 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106080 2020-08-07T15:08:46.796 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.796 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:46.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.797 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:46.797 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.241181+0000", 2020-08-07T15:08:46.797 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:46.798 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:46.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.241181+0000", 2020-08-07T15:08:46.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3634999999999998e-05 2020-08-07T15:08:46.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:46.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.241224+0000", 2020-08-07T15:08:46.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2310000000000001e-06 2020-08-07T15:08:46.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.813 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:46.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.241228+0000", 2020-08-07T15:08:46.813 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7490000000000001e-06 2020-08-07T15:08:46.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:46.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.241234+0000", 2020-08-07T15:08:46.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1320000000000004e-06 2020-08-07T15:08:46.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.818 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:46.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.241238+0000", 2020-08-07T15:08:46.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001563659 2020-08-07T15:08:46.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:46.821 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.242802+0000", 2020-08-07T15:08:46.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0303999999999995e-05 2020-08-07T15:08:46.821 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.822 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.822 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:46.822 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.242892+0000", 2020-08-07T15:08:46.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013412 2020-08-07T15:08:46.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:08:46.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.243027+0000", 2020-08-07T15:08:46.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001917291 2020-08-07T15:08:46.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:46.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.244944+0000", 2020-08-07T15:08:46.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9647999999999995e-05 2020-08-07T15:08:46.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:46.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.245013+0000", 2020-08-07T15:08:46.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.44882515299999998 2020-08-07T15:08:46.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:46.826 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693839+0000", 2020-08-07T15:08:46.826 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5290000000000001e-06 2020-08-07T15:08:46.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.827 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:46.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693840+0000", 2020-08-07T15:08:46.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7976999999999998e-05 2020-08-07T15:08:46.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:46.828 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693888+0000", 2020-08-07T15:08:46.828 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8226999999999998e-05 2020-08-07T15:08:46.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.829 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.829 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:46.829 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693926+0000", 2020-08-07T15:08:46.830 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45274565500000002 2020-08-07T15:08:46.830 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:46.830 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:46.831 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:46.831 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.831 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.831 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106080 3.4 e58/20 ondisk, result = 0)", 2020-08-07T15:08:46.831 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.255977+0000", 2020-08-07T15:08:46.832 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 59.126110726, 2020-08-07T15:08:46.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43795860599999997, 2020-08-07T15:08:46.832 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:46.832 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:46.832 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:46.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:46.833 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255977+0000", 2020-08-07T15:08:46.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:46.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:46.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255977+0000", 2020-08-07T15:08:46.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2820000000000001e-05 2020-08-07T15:08:46.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:46.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.256010+0000", 2020-08-07T15:08:46.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8660000000000002e-06 2020-08-07T15:08:46.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.836 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:46.836 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.256016+0000", 2020-08-07T15:08:46.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.437661209 2020-08-07T15:08:46.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.838 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:46.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693677+0000", 2020-08-07T15:08:46.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6000000000000003e-06 2020-08-07T15:08:46.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.839 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:46.841 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693684+0000", 2020-08-07T15:08:46.841 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000138572 2020-08-07T15:08:46.841 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:46.842 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693822+0000", 2020-08-07T15:08:46.842 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6339999999999996e-06 2020-08-07T15:08:46.842 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.842 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.842 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:46.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693832+0000", 2020-08-07T15:08:46.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000103905 2020-08-07T15:08:46.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.843 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:46.844 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693936+0000", 2020-08-07T15:08:46.844 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43795860599999997 2020-08-07T15:08:46.844 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:46.844 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:46.844 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:46.845 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.845 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:116623 3.9 e65/20 3:93e94c7d:::benchmark_data_smithi074_35447_object14577:head v 65'7111, mlcod=65'7102)", 2020-08-07T15:08:46.845 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:56.780860+0000", 2020-08-07T15:08:46.845 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 49.601227577000003, 2020-08-07T15:08:46.846 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22065456899999999, 2020-08-07T15:08:46.846 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:46.846 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:46.846 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:46.846 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.847 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:46.847 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.780860+0000", 2020-08-07T15:08:46.847 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:46.847 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.847 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.848 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:46.848 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.780860+0000", 2020-08-07T15:08:46.848 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2389000000000003e-05 2020-08-07T15:08:46.849 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.849 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.849 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:46.849 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.780913+0000", 2020-08-07T15:08:46.849 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7039999999999997e-06 2020-08-07T15:08:46.850 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.850 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:46.851 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.780920+0000", 2020-08-07T15:08:46.851 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.219753378 2020-08-07T15:08:46.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:46.852 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.000673+0000", 2020-08-07T15:08:46.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8869999999999997e-06 2020-08-07T15:08:46.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.853 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:46.853 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.000679+0000", 2020-08-07T15:08:46.853 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0431000000000004e-05 2020-08-07T15:08:46.854 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.854 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.854 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:46.854 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.000749+0000", 2020-08-07T15:08:46.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8459e-05 2020-08-07T15:08:46.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.855 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:46.855 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.000778+0000", 2020-08-07T15:08:46.855 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00068167900000000003 2020-08-07T15:08:46.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.856 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.856 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:46.856 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.001459+0000", 2020-08-07T15:08:46.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5642e-05 2020-08-07T15:08:46.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:46.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.001515+0000", 2020-08-07T15:08:46.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22065456899999999 2020-08-07T15:08:46.858 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:46.858 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:46.858 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:46.858 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.859 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:116624 3.9 e65/20 3:93e94c7d:::benchmark_data_smithi074_35447_object14577:head v 65'7112, mlcod=65'7103)", 2020-08-07T15:08:46.859 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:56.781405+0000", 2020-08-07T15:08:46.859 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 49.600682917999997, 2020-08-07T15:08:46.859 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.248173752, 2020-08-07T15:08:46.859 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:46.860 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:46.860 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:46.860 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:46.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.781405+0000", 2020-08-07T15:08:46.862 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:46.862 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:46.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.781405+0000", 2020-08-07T15:08:46.863 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3173999999999998e-05 2020-08-07T15:08:46.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:46.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.781438+0000", 2020-08-07T15:08:46.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1790000000000001e-06 2020-08-07T15:08:46.864 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.865 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:46.865 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.781442+0000", 2020-08-07T15:08:46.865 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24601534999999999 2020-08-07T15:08:46.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.872 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.872 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:46.872 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.027458+0000", 2020-08-07T15:08:46.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8599999999999998e-06 2020-08-07T15:08:46.873 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.873 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:46.874 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.027464+0000", 2020-08-07T15:08:46.874 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3399000000000003e-05 2020-08-07T15:08:46.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:46.875 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.027497+0000", 2020-08-07T15:08:46.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6073999999999999e-05 2020-08-07T15:08:46.876 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.876 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:46.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.027513+0000", 2020-08-07T15:08:46.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0020137979999999998 2020-08-07T15:08:46.877 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.877 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.877 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:46.877 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.029527+0000", 2020-08-07T15:08:46.877 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1918e-05 2020-08-07T15:08:46.878 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.878 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.878 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:46.878 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.029579+0000", 2020-08-07T15:08:46.878 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.248173752 2020-08-07T15:08:46.879 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:46.889 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:46.889 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:46.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.892 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:117016 3.4 3:2f619d09:::benchmark_data_smithi074_35447_object14626:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e65)", 2020-08-07T15:08:46.893 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:57.118050+0000", 2020-08-07T15:08:46.905 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 49.264037858000002, 2020-08-07T15:08:46.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.255829902, 2020-08-07T15:08:46.905 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:46.906 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:46.906 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:46.906 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:46.907 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:46.914 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 117016 2020-08-07T15:08:46.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.915 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:46.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:46.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118050+0000", 2020-08-07T15:08:46.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:46.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.918 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:46.918 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118050+0000", 2020-08-07T15:08:46.918 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2957999999999999e-05 2020-08-07T15:08:46.919 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.919 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:46.920 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118083+0000", 2020-08-07T15:08:46.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8829999999999997e-06 2020-08-07T15:08:46.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:46.966 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118088+0000", 2020-08-07T15:08:46.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4660000000000005e-06 2020-08-07T15:08:46.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:46.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118098+0000", 2020-08-07T15:08:46.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.768e-06 2020-08-07T15:08:46.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.969 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.969 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:46.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118100+0000", 2020-08-07T15:08:46.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00054986900000000003 2020-08-07T15:08:46.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:46.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118650+0000", 2020-08-07T15:08:46.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7521e-05 2020-08-07T15:08:46.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:46.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118708+0000", 2020-08-07T15:08:46.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012445499999999999 2020-08-07T15:08:46.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:08:46.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118832+0000", 2020-08-07T15:08:46.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002948806 2020-08-07T15:08:46.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:46.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.121781+0000", 2020-08-07T15:08:46.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8299999999999996e-07 2020-08-07T15:08:46.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:46.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.121782+0000", 2020-08-07T15:08:46.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4300000000000004e-05 2020-08-07T15:08:46.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.978 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:46.978 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.121846+0000", 2020-08-07T15:08:46.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.044326363000000001 2020-08-07T15:08:46.979 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.979 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.979 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:46.980 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.166172+0000", 2020-08-07T15:08:46.980 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20763515900000001 2020-08-07T15:08:46.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:46.981 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.373808+0000", 2020-08-07T15:08:46.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6339999999999999e-06 2020-08-07T15:08:46.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.981 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:46.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.373809+0000", 2020-08-07T15:08:46.982 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0937000000000002e-05 2020-08-07T15:08:46.982 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.983 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:46.984 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.373880+0000", 2020-08-07T15:08:46.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.255829902 2020-08-07T15:08:46.991 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:46.991 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:46.992 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:46.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.993 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:7304 4.4 4:2da9e9a1:::benchmark_data_smithi074_44586_object912:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e85)", 2020-08-07T15:08:46.993 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:31.182209+0000", 2020-08-07T15:08:46.993 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 15.199879327, 2020-08-07T15:08:46.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23183255, 2020-08-07T15:08:46.994 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:46.994 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:46.995 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:46.995 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:08:46.995 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:46.996 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7304 2020-08-07T15:08:46.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.997 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:46.997 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.997 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:46.998 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.182209+0000", 2020-08-07T15:08:46.998 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:46.998 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.998 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:46.999 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:46.999 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.182209+0000", 2020-08-07T15:08:46.999 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7351999999999998e-05 2020-08-07T15:08:46.999 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:46.999 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.000 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:47.000 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.182236+0000", 2020-08-07T15:08:47.000 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.512e-06 2020-08-07T15:08:47.000 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.000 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.001 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:47.001 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.182240+0000", 2020-08-07T15:08:47.001 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1302e-05 2020-08-07T15:08:47.001 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.001 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.002 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:47.002 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.182251+0000", 2020-08-07T15:08:47.002 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1370000000000004e-06 2020-08-07T15:08:47.002 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.002 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.002 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:47.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.182255+0000", 2020-08-07T15:08:47.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024353510000000001 2020-08-07T15:08:47.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:47.004 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.184690+0000", 2020-08-07T15:08:47.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5316999999999996e-05 2020-08-07T15:08:47.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:47.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.184786+0000", 2020-08-07T15:08:47.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018412300000000001 2020-08-07T15:08:47.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:08:47.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.184970+0000", 2020-08-07T15:08:47.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0025507059999999998 2020-08-07T15:08:47.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:47.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.187521+0000", 2020-08-07T15:08:47.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.96e-07 2020-08-07T15:08:47.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:47.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.187521+0000", 2020-08-07T15:08:47.008 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2775000000000002e-05 2020-08-07T15:08:47.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.009 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.009 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:47.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.187574+0000", 2020-08-07T15:08:47.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.040609377000000002 2020-08-07T15:08:47.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.010 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:47.010 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.228183+0000", 2020-08-07T15:08:47.010 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.185779313 2020-08-07T15:08:47.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:47.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.413962+0000", 2020-08-07T15:08:47.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0189999999999999e-06 2020-08-07T15:08:47.012 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.013 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:47.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.413963+0000", 2020-08-07T15:08:47.013 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7869999999999998e-05 2020-08-07T15:08:47.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.014 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:47.016 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:31.414041+0000", 2020-08-07T15:08:47.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23183255 2020-08-07T15:08:47.017 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:47.017 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:47.017 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:47.017 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.018 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:9111 4.6 4:6004be0b:::benchmark_data_smithi074_44586_object1138:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e86)", 2020-08-07T15:08:47.018 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:32.294153+0000", 2020-08-07T15:08:47.018 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 14.087935163999999, 2020-08-07T15:08:47.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26066347899999998, 2020-08-07T15:08:47.018 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:47.019 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:47.019 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:47.019 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:08:47.019 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:47.019 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 9111 2020-08-07T15:08:47.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.020 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:47.020 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:47.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.294153+0000", 2020-08-07T15:08:47.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:47.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:47.022 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.294153+0000", 2020-08-07T15:08:47.022 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5032e-05 2020-08-07T15:08:47.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.023 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:47.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.294178+0000", 2020-08-07T15:08:47.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9830000000000001e-06 2020-08-07T15:08:47.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:47.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.294181+0000", 2020-08-07T15:08:47.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5319999999999997e-06 2020-08-07T15:08:47.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.038 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:47.038 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.294186+0000", 2020-08-07T15:08:47.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7209999999999999e-06 2020-08-07T15:08:47.039 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.039 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:47.040 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.294189+0000", 2020-08-07T15:08:47.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000974397 2020-08-07T15:08:47.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:47.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.295164+0000", 2020-08-07T15:08:47.042 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000134375 2020-08-07T15:08:47.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:47.044 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.295298+0000", 2020-08-07T15:08:47.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017785000000000001 2020-08-07T15:08:47.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:08:47.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.295476+0000", 2020-08-07T15:08:47.046 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0029282259999999999 2020-08-07T15:08:47.046 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.047 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:47.047 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.298404+0000", 2020-08-07T15:08:47.048 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5199999999999996e-07 2020-08-07T15:08:47.048 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.049 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.049 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:47.049 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.298405+0000", 2020-08-07T15:08:47.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7341e-05 2020-08-07T15:08:47.049 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.050 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.050 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:47.050 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.298452+0000", 2020-08-07T15:08:47.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004868079 2020-08-07T15:08:47.050 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.062 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:47.062 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.303320+0000", 2020-08-07T15:08:47.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25142435099999999 2020-08-07T15:08:47.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.063 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:47.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.554744+0000", 2020-08-07T15:08:47.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3459999999999999e-06 2020-08-07T15:08:47.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.064 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.075 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:47.086 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.554746+0000", 2020-08-07T15:08:47.090 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0593999999999995e-05 2020-08-07T15:08:47.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.095 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.101 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:47.101 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.554816+0000", 2020-08-07T15:08:47.113 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26066347899999998 2020-08-07T15:08:47.114 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:47.114 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:47.114 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:47.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.115 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:9112 4.6 4:6004be0b:::benchmark_data_smithi074_44586_object1138:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e86)", 2020-08-07T15:08:47.115 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:32.294206+0000", 2020-08-07T15:08:47.115 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 14.087882146, 2020-08-07T15:08:47.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26582170300000002, 2020-08-07T15:08:47.115 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:47.116 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:47.116 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:47.116 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:08:47.117 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:47.126 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 9112 2020-08-07T15:08:47.127 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.127 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:47.127 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.127 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:47.127 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.294206+0000", 2020-08-07T15:08:47.128 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:47.169 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.169 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.169 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:47.170 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.294206+0000", 2020-08-07T15:08:47.170 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1359999999999998e-05 2020-08-07T15:08:47.170 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.170 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.170 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:47.171 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.294237+0000", 2020-08-07T15:08:47.171 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9879999999999998e-06 2020-08-07T15:08:47.171 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.171 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.171 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:47.172 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.294241+0000", 2020-08-07T15:08:47.172 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9900000000000002e-06 2020-08-07T15:08:47.172 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.172 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.172 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:47.173 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.294247+0000", 2020-08-07T15:08:47.173 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9079999999999999e-06 2020-08-07T15:08:47.173 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.173 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.173 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:47.201 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.294250+0000", 2020-08-07T15:08:47.201 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001905871 2020-08-07T15:08:47.202 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.202 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.202 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:47.207 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296156+0000", 2020-08-07T15:08:47.210 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8465e-05 2020-08-07T15:08:47.210 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.234 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.235 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:47.235 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296254+0000", 2020-08-07T15:08:47.235 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018382500000000001 2020-08-07T15:08:47.239 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.241 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.241 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:08:47.241 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296438+0000", 2020-08-07T15:08:47.242 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002019819 2020-08-07T15:08:47.242 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.242 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.243 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:47.243 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.298458+0000", 2020-08-07T15:08:47.243 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2500000000000001e-07 2020-08-07T15:08:47.243 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.243 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.244 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:47.244 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.298458+0000", 2020-08-07T15:08:47.244 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2608999999999997e-05 2020-08-07T15:08:47.244 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.244 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.245 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:47.245 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.298511+0000", 2020-08-07T15:08:47.245 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0048420349999999997 2020-08-07T15:08:47.245 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.245 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.246 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:47.246 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.303353+0000", 2020-08-07T15:08:47.246 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25663897600000002 2020-08-07T15:08:47.247 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.247 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.247 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:47.247 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.559992+0000", 2020-08-07T15:08:47.247 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.92e-07 2020-08-07T15:08:47.248 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.248 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.249 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:47.249 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.559993+0000", 2020-08-07T15:08:47.250 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4740000000000003e-05 2020-08-07T15:08:47.250 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.251 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.251 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:47.251 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.560028+0000", 2020-08-07T15:08:47.251 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26582170300000002 2020-08-07T15:08:47.252 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:47.252 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:47.252 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:47.252 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.252 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.252 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:9111 4.6 e86/81 ondisk, result = 0)", 2020-08-07T15:08:47.253 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:32.298867+0000", 2020-08-07T15:08:47.253 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 14.083220892, 2020-08-07T15:08:47.253 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25595633200000001, 2020-08-07T15:08:47.253 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:47.254 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:47.254 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:47.254 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.254 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:47.254 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.298867+0000", 2020-08-07T15:08:47.255 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:47.255 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.255 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.255 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:47.255 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.298867+0000", 2020-08-07T15:08:47.256 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4514e-05 2020-08-07T15:08:47.256 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.256 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.256 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:47.256 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.298882+0000", 2020-08-07T15:08:47.257 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3039999999999999e-06 2020-08-07T15:08:47.257 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.257 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.258 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:47.258 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.298884+0000", 2020-08-07T15:08:47.258 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25579149899999998 2020-08-07T15:08:47.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.259 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.259 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:47.259 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.554675+0000", 2020-08-07T15:08:47.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7690000000000002e-06 2020-08-07T15:08:47.260 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.261 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.261 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:47.261 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.554681+0000", 2020-08-07T15:08:47.262 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3312999999999997e-05 2020-08-07T15:08:47.270 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.270 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:47.271 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.554725+0000", 2020-08-07T15:08:47.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0437e-05 2020-08-07T15:08:47.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:47.273 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.554735+0000", 2020-08-07T15:08:47.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8496000000000001e-05 2020-08-07T15:08:47.273 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.273 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:47.274 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.554823+0000", 2020-08-07T15:08:47.274 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25595633200000001 2020-08-07T15:08:47.274 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:47.274 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:47.274 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:47.275 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.275 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.275 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:15736 4.4 4:29482d63:::benchmark_data_smithi074_44586_object1966:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e89)", 2020-08-07T15:08:47.275 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:36.282829+0000", 2020-08-07T15:08:47.275 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 10.099259514, 2020-08-07T15:08:47.276 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24397778000000001, 2020-08-07T15:08:47.276 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:47.276 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:47.276 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:47.277 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:08:47.278 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:47.278 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 15736 2020-08-07T15:08:47.279 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.279 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:47.279 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.279 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:47.280 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.282829+0000", 2020-08-07T15:08:47.280 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:47.280 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.281 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.281 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:47.281 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.282829+0000", 2020-08-07T15:08:47.281 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0772000000000003e-05 2020-08-07T15:08:47.281 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.284 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.284 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:47.284 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.282859+0000", 2020-08-07T15:08:47.284 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3610000000000003e-06 2020-08-07T15:08:47.284 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.285 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.285 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:47.285 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.282864+0000", 2020-08-07T15:08:47.285 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8449999999999992e-06 2020-08-07T15:08:47.285 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.286 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.286 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:47.287 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.282873+0000", 2020-08-07T15:08:47.287 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7790000000000002e-06 2020-08-07T15:08:47.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.287 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:47.288 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.282876+0000", 2020-08-07T15:08:47.288 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028449009999999999 2020-08-07T15:08:47.288 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.288 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.288 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:47.288 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.285721+0000", 2020-08-07T15:08:47.289 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000102428 2020-08-07T15:08:47.289 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.289 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.289 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:47.289 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.285824+0000", 2020-08-07T15:08:47.290 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000133869 2020-08-07T15:08:47.290 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.291 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.291 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:08:47.291 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.285957+0000", 2020-08-07T15:08:47.291 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001718898 2020-08-07T15:08:47.291 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.292 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.292 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:47.292 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.287676+0000", 2020-08-07T15:08:47.292 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7399999999999998e-07 2020-08-07T15:08:47.292 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.293 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.293 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:47.296 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.287677+0000", 2020-08-07T15:08:47.296 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3714000000000001e-05 2020-08-07T15:08:47.296 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.297 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.297 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:47.297 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.287731+0000", 2020-08-07T15:08:47.297 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0020407379999999998 2020-08-07T15:08:47.298 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.298 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.298 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:47.298 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.289771+0000", 2020-08-07T15:08:47.298 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23695730000000001 2020-08-07T15:08:47.299 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.299 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.299 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:47.299 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.526729+0000", 2020-08-07T15:08:47.300 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5799999999999998e-07 2020-08-07T15:08:47.300 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.300 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.300 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:47.302 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.526730+0000", 2020-08-07T15:08:47.302 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6742999999999994e-05 2020-08-07T15:08:47.303 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.303 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.303 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:47.303 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:36.526806+0000", 2020-08-07T15:08:47.305 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24397778000000001 2020-08-07T15:08:47.305 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:47.305 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:47.305 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:47.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.307 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.307 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:23496 4.4 4:2fe071c3:::benchmark_data_smithi074_44586_object2936:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e94)", 2020-08-07T15:08:47.308 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:41.111254+0000", 2020-08-07T15:08:47.308 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.270833573, 2020-08-07T15:08:47.308 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21461021199999999, 2020-08-07T15:08:47.309 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:47.309 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:47.309 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:47.309 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:08:47.309 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:47.310 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 23496 2020-08-07T15:08:47.310 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.310 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:47.310 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.310 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:47.311 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:41.111254+0000", 2020-08-07T15:08:47.311 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:47.312 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.312 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.312 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:47.312 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:41.111254+0000", 2020-08-07T15:08:47.312 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4122e-05 2020-08-07T15:08:47.313 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.313 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.313 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:47.313 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:41.111279+0000", 2020-08-07T15:08:47.313 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5770000000000001e-06 2020-08-07T15:08:47.313 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.314 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.315 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:47.315 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:41.111281+0000", 2020-08-07T15:08:47.315 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2379999999999997e-06 2020-08-07T15:08:47.316 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.316 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.316 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:47.317 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:41.111286+0000", 2020-08-07T15:08:47.317 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1610000000000001e-06 2020-08-07T15:08:47.317 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.318 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.318 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:47.318 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:41.111289+0000", 2020-08-07T15:08:47.318 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0083359790000000003 2020-08-07T15:08:47.318 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.319 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.319 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:47.328 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:41.119625+0000", 2020-08-07T15:08:47.328 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000109994 2020-08-07T15:08:47.328 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.328 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.328 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:47.329 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:41.119735+0000", 2020-08-07T15:08:47.335 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013498199999999999 2020-08-07T15:08:47.335 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.336 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:08:47.337 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:41.119870+0000", 2020-08-07T15:08:47.337 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0062362959999999997 2020-08-07T15:08:47.338 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.338 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.339 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:47.344 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:41.126106+0000", 2020-08-07T15:08:47.345 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5300000000000004e-07 2020-08-07T15:08:47.345 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.346 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.346 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:47.346 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:41.126106+0000", 2020-08-07T15:08:47.347 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8418999999999998e-05 2020-08-07T15:08:47.347 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.348 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:47.348 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:41.126155+0000", 2020-08-07T15:08:47.348 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0067078279999999999 2020-08-07T15:08:47.349 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.349 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.349 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:47.350 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:41.132863+0000", 2020-08-07T15:08:47.350 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.19293768 2020-08-07T15:08:47.350 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.350 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.351 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:47.351 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:41.325800+0000", 2020-08-07T15:08:47.353 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1880000000000001e-06 2020-08-07T15:08:47.353 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.354 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.354 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:47.354 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:41.325802+0000", 2020-08-07T15:08:47.355 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3095000000000005e-05 2020-08-07T15:08:47.355 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.355 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.356 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:47.356 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:41.325865+0000", 2020-08-07T15:08:47.369 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21461021199999999 2020-08-07T15:08:47.370 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:47.370 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:47.371 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:47.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.371 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.372 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:25792 4.4 4:2d04ac8e:::benchmark_data_smithi074_44586_object3223:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e96)", 2020-08-07T15:08:47.372 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:42.365230+0000", 2020-08-07T15:08:47.372 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4.0168577519999999, 2020-08-07T15:08:47.372 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25180039700000001, 2020-08-07T15:08:47.372 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:47.373 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:47.373 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:47.373 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:08:47.373 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:47.373 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 25792 2020-08-07T15:08:47.374 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.374 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:47.374 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.374 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:47.374 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.365230+0000", 2020-08-07T15:08:47.375 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:47.375 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.376 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:47.376 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.365230+0000", 2020-08-07T15:08:47.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5354000000000002e-05 2020-08-07T15:08:47.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:47.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.365256+0000", 2020-08-07T15:08:47.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.605e-06 2020-08-07T15:08:47.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:47.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.365258+0000", 2020-08-07T15:08:47.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8200000000000002e-06 2020-08-07T15:08:47.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:47.380 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.365264+0000", 2020-08-07T15:08:47.380 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1019999999999999e-06 2020-08-07T15:08:47.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:47.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.365266+0000", 2020-08-07T15:08:47.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002530577 2020-08-07T15:08:47.420 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.420 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.421 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:47.421 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.367797+0000", 2020-08-07T15:08:47.421 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3579999999999998e-05 2020-08-07T15:08:47.451 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:47.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.367860+0000", 2020-08-07T15:08:47.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000106155 2020-08-07T15:08:47.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.454 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:08:47.454 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.367966+0000", 2020-08-07T15:08:47.454 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0040722500000000003 2020-08-07T15:08:47.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:47.456 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.372039+0000", 2020-08-07T15:08:47.456 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5700000000000002e-07 2020-08-07T15:08:47.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:47.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.372039+0000", 2020-08-07T15:08:47.462 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3897999999999998e-05 2020-08-07T15:08:47.462 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.462 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.463 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:47.463 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.372093+0000", 2020-08-07T15:08:47.463 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0029951140000000001 2020-08-07T15:08:47.463 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.463 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.464 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:47.464 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.375088+0000", 2020-08-07T15:08:47.464 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24190147400000001 2020-08-07T15:08:47.464 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.464 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.465 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:47.468 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.616990+0000", 2020-08-07T15:08:47.468 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7000000000000004e-07 2020-08-07T15:08:47.469 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.469 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.469 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:47.470 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.616990+0000", 2020-08-07T15:08:47.470 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0241000000000003e-05 2020-08-07T15:08:47.473 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.473 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.473 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:47.473 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.617031+0000", 2020-08-07T15:08:47.474 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25180039700000001 2020-08-07T15:08:47.474 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:47.474 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:47.474 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:47.474 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.475 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.475 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:25792 4.4 e96/81 ondisk, result = 0)", 2020-08-07T15:08:47.475 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:42.377508+0000", 2020-08-07T15:08:47.475 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4.0045797939999996, 2020-08-07T15:08:47.475 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23952621900000001, 2020-08-07T15:08:47.476 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:47.476 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:47.476 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:47.477 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.477 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:47.477 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.377508+0000", 2020-08-07T15:08:47.477 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:47.477 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.478 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.478 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:47.479 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.377508+0000", 2020-08-07T15:08:47.479 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2602999999999999e-05 2020-08-07T15:08:47.479 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.479 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.480 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:47.480 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.377541+0000", 2020-08-07T15:08:47.480 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3059999999999997e-06 2020-08-07T15:08:47.480 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.480 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:47.481 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.377546+0000", 2020-08-07T15:08:47.481 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23936785499999999 2020-08-07T15:08:47.481 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.483 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.483 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:47.483 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.616914+0000", 2020-08-07T15:08:47.483 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0449999999999997e-06 2020-08-07T15:08:47.483 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.484 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.484 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:47.484 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.616920+0000", 2020-08-07T15:08:47.484 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1873999999999997e-05 2020-08-07T15:08:47.484 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.485 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.485 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:47.485 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.616982+0000", 2020-08-07T15:08:47.486 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.895e-06 2020-08-07T15:08:47.486 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.486 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.487 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:47.487 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.616986+0000", 2020-08-07T15:08:47.487 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8640999999999997e-05 2020-08-07T15:08:47.488 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:47.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:47.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:47.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.617034+0000", 2020-08-07T15:08:47.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23952621900000001 2020-08-07T15:08:47.489 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:47.489 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:47.490 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:47.490 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:47.490 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:47.491 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:47.492 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:08:47.513 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:08:47.514 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 21 16 31016 31000 11.5319 0.25 0.00380648 0.0102869 2020-08-07T15:08:47.514 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:47.494+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:47.617 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:47.614+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:47.700 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:47.700 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:08:47.700 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:08:47.701 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:47.714 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:08:47.756 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:47.755+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:47.837 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:47.836+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:47.903 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:47.904 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:08:47.904 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:08:47.905 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:08:47.905 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:47.912 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:08:47.950 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:47.949+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:48.061 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:48.049+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:48.074 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:48.074 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:08:48.075 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:08:48.075 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:08:48.075 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.075 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:104103 3.8 3:11c276ea:::benchmark_data_smithi074_35447_object13012:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:08:48.084 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:45.853251+0000", 2020-08-07T15:08:48.084 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 62.211406490999998, 2020-08-07T15:08:48.085 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51821152400000003, 2020-08-07T15:08:48.085 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:48.085 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:48.085 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:48.085 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:48.086 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:48.086 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 104103 2020-08-07T15:08:48.086 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.086 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:48.086 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:48.087 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.853251+0000", 2020-08-07T15:08:48.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:48.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:48.088 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.853251+0000", 2020-08-07T15:08:48.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3990000000000003e-05 2020-08-07T15:08:48.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:48.111 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.853305+0000", 2020-08-07T15:08:48.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6409999999999996e-06 2020-08-07T15:08:48.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.112 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:48.113 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.853312+0000", 2020-08-07T15:08:48.117 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3594999999999999e-05 2020-08-07T15:08:48.117 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.117 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:48.118 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.853325+0000", 2020-08-07T15:08:48.118 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7919999999999998e-06 2020-08-07T15:08:48.118 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.119 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.119 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:48.119 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.853331+0000", 2020-08-07T15:08:48.120 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039587290000000002 2020-08-07T15:08:48.120 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.120 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.120 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.120 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.857290+0000", 2020-08-07T15:08:48.121 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010191699999999999 2020-08-07T15:08:48.121 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.121 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.121 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:48.121 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.857392+0000", 2020-08-07T15:08:48.122 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000203654 2020-08-07T15:08:48.122 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.122 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.122 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:48.122 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.857595+0000", 2020-08-07T15:08:48.123 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038121600000000002 2020-08-07T15:08:48.123 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.123 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.123 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:48.124 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861408+0000", 2020-08-07T15:08:48.124 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3499999999999996e-07 2020-08-07T15:08:48.124 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.124 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.125 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:48.125 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861408+0000", 2020-08-07T15:08:48.125 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7356999999999999e-05 2020-08-07T15:08:48.125 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.125 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.126 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:48.126 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861476+0000", 2020-08-07T15:08:48.126 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00067373200000000002 2020-08-07T15:08:48.126 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.127 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.127 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:48.128 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.862149+0000", 2020-08-07T15:08:48.128 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.50923365300000001 2020-08-07T15:08:48.128 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.128 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.128 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:48.128 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.371383+0000", 2020-08-07T15:08:48.129 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5000000000000001e-07 2020-08-07T15:08:48.129 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.129 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.129 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:48.129 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.371384+0000", 2020-08-07T15:08:48.130 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8819000000000005e-05 2020-08-07T15:08:48.130 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.130 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.130 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:48.130 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.371463+0000", 2020-08-07T15:08:48.131 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51821152400000003 2020-08-07T15:08:48.131 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.131 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:48.131 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.131 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.132 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.132 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:104104 3.8 3:11c276ea:::benchmark_data_smithi074_35447_object13012:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:08:48.133 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:45.853881+0000", 2020-08-07T15:08:48.133 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 62.210776774999999, 2020-08-07T15:08:48.133 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51834839399999999, 2020-08-07T15:08:48.133 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:48.133 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:48.134 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:48.134 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:48.134 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:48.135 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 104104 2020-08-07T15:08:48.135 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.135 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:48.135 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.135 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:48.136 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.853881+0000", 2020-08-07T15:08:48.136 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:48.136 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.136 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.136 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:48.137 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.853881+0000", 2020-08-07T15:08:48.137 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0507999999999998e-05 2020-08-07T15:08:48.137 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.137 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.137 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:48.138 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.853931+0000", 2020-08-07T15:08:48.138 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5810000000000002e-06 2020-08-07T15:08:48.138 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.139 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.139 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:48.139 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.853938+0000", 2020-08-07T15:08:48.139 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3549e-05 2020-08-07T15:08:48.140 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.140 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.140 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:48.140 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.853951+0000", 2020-08-07T15:08:48.140 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7849999999999997e-06 2020-08-07T15:08:48.141 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.141 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.141 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:48.141 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.853957+0000", 2020-08-07T15:08:48.141 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004119775 2020-08-07T15:08:48.142 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.142 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.143 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.143 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.858077+0000", 2020-08-07T15:08:48.143 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2958000000000001e-05 2020-08-07T15:08:48.144 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.144 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.145 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:48.146 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.858160+0000", 2020-08-07T15:08:48.146 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000116242 2020-08-07T15:08:48.146 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.146 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.147 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:08:48.147 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.858276+0000", 2020-08-07T15:08:48.147 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003211036 2020-08-07T15:08:48.147 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.147 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.148 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:48.148 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861487+0000", 2020-08-07T15:08:48.148 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1900000000000002e-07 2020-08-07T15:08:48.149 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.149 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.149 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:48.149 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861488+0000", 2020-08-07T15:08:48.149 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2969999999999997e-05 2020-08-07T15:08:48.150 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.150 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.150 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:48.150 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861531+0000", 2020-08-07T15:08:48.150 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32897901600000001 2020-08-07T15:08:48.150 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.151 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:48.151 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.190510+0000", 2020-08-07T15:08:48.151 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.18163950300000001 2020-08-07T15:08:48.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:48.152 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.372149+0000", 2020-08-07T15:08:48.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0699999999999996e-07 2020-08-07T15:08:48.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.153 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.153 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:48.153 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.372150+0000", 2020-08-07T15:08:48.153 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9045e-05 2020-08-07T15:08:48.153 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.154 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.154 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:48.154 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.372229+0000", 2020-08-07T15:08:48.154 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51834839399999999 2020-08-07T15:08:48.154 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.155 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:48.155 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.155 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.156 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:104103 3.8 e58/20 ondisk, result = 0)", 2020-08-07T15:08:48.156 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:45.860696+0000", 2020-08-07T15:08:48.156 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 62.203961364000001, 2020-08-07T15:08:48.157 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51077444900000002, 2020-08-07T15:08:48.157 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:48.158 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:48.158 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:48.159 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.159 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:48.159 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.860696+0000", 2020-08-07T15:08:48.160 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:48.160 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.160 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.160 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:48.160 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.860696+0000", 2020-08-07T15:08:48.161 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8182e-05 2020-08-07T15:08:48.161 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.161 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.161 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:48.161 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.860714+0000", 2020-08-07T15:08:48.162 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5739999999999998e-06 2020-08-07T15:08:48.162 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.162 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.163 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:48.163 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.860717+0000", 2020-08-07T15:08:48.163 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.510595935 2020-08-07T15:08:48.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.163 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.163 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:48.164 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.371313+0000", 2020-08-07T15:08:48.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4790000000000004e-06 2020-08-07T15:08:48.164 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:48.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.371318+0000", 2020-08-07T15:08:48.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5068000000000003e-05 2020-08-07T15:08:48.165 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.165 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.165 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.166 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.371363+0000", 2020-08-07T15:08:48.166 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0251999999999999e-05 2020-08-07T15:08:48.167 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.167 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.167 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:48.167 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.371374+0000", 2020-08-07T15:08:48.167 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6959000000000005e-05 2020-08-07T15:08:48.168 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.168 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.168 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:48.169 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.371471+0000", 2020-08-07T15:08:48.169 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51077444900000002 2020-08-07T15:08:48.169 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.169 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:48.169 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.170 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.170 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.172 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:104104 3.8 e58/20 ondisk, result = 0)", 2020-08-07T15:08:48.173 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:45.860731+0000", 2020-08-07T15:08:48.174 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 62.203926086000003, 2020-08-07T15:08:48.174 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51150493600000002, 2020-08-07T15:08:48.174 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:48.174 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:48.175 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:48.175 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.175 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:48.175 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.860731+0000", 2020-08-07T15:08:48.175 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:48.176 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.176 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.176 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:48.177 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.860731+0000", 2020-08-07T15:08:48.177 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.487e-05 2020-08-07T15:08:48.178 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.178 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.178 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:48.178 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.860746+0000", 2020-08-07T15:08:48.178 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2919999999999998e-06 2020-08-07T15:08:48.179 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.179 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.179 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:48.179 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.860749+0000", 2020-08-07T15:08:48.179 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51133623699999997 2020-08-07T15:08:48.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.180 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.180 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:48.180 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.372085+0000", 2020-08-07T15:08:48.180 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9600000000000002e-06 2020-08-07T15:08:48.181 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.181 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.181 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:48.181 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.372089+0000", 2020-08-07T15:08:48.181 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1868e-05 2020-08-07T15:08:48.182 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.182 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.372131+0000", 2020-08-07T15:08:48.183 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1870000000000003e-06 2020-08-07T15:08:48.183 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.183 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.183 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:48.184 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.372140+0000", 2020-08-07T15:08:48.184 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6521999999999995e-05 2020-08-07T15:08:48.184 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.184 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.184 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:48.185 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.372236+0000", 2020-08-07T15:08:48.185 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51150493600000002 2020-08-07T15:08:48.185 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.185 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:48.185 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.186 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.186 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.186 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106239 3.7 3:ebdd8242:::benchmark_data_smithi074_35447_object13279:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:08:48.186 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.349775+0000", 2020-08-07T15:08:48.186 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 60.714882435, 2020-08-07T15:08:48.187 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.48901084099999997, 2020-08-07T15:08:48.187 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:48.187 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:48.188 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:48.188 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:48.189 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:48.189 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106239 2020-08-07T15:08:48.189 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.189 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:48.189 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.190 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:48.190 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.349775+0000", 2020-08-07T15:08:48.190 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:48.190 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.190 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.191 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:48.191 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.349775+0000", 2020-08-07T15:08:48.191 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4729999999999999e-05 2020-08-07T15:08:48.191 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.191 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.191 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:48.192 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.349830+0000", 2020-08-07T15:08:48.192 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.254e-06 2020-08-07T15:08:48.192 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.192 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.193 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:48.193 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.349835+0000", 2020-08-07T15:08:48.193 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0451999999999999e-05 2020-08-07T15:08:48.193 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.193 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.194 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:48.194 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.349846+0000", 2020-08-07T15:08:48.194 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5569999999999997e-06 2020-08-07T15:08:48.194 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.194 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.195 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:48.195 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.349850+0000", 2020-08-07T15:08:48.195 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034527009999999999 2020-08-07T15:08:48.195 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.195 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.196 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.196 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.353303+0000", 2020-08-07T15:08:48.196 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000111487 2020-08-07T15:08:48.196 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.196 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.197 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:48.197 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.353414+0000", 2020-08-07T15:08:48.197 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018956600000000001 2020-08-07T15:08:48.198 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.198 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.198 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:08:48.198 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.353604+0000", 2020-08-07T15:08:48.198 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0037590079999999999 2020-08-07T15:08:48.199 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.199 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.199 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:48.200 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.357363+0000", 2020-08-07T15:08:48.200 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1500000000000003e-07 2020-08-07T15:08:48.200 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.200 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.200 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:48.200 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.357364+0000", 2020-08-07T15:08:48.201 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000113244 2020-08-07T15:08:48.202 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.202 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.202 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:48.203 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.357477+0000", 2020-08-07T15:08:48.203 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00075726500000000002 2020-08-07T15:08:48.203 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.203 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.203 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:48.204 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.358234+0000", 2020-08-07T15:08:48.204 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.480494105 2020-08-07T15:08:48.204 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.204 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.204 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:48.205 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.838728+0000", 2020-08-07T15:08:48.205 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.052e-06 2020-08-07T15:08:48.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.206 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.206 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:48.206 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.838729+0000", 2020-08-07T15:08:48.206 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6505000000000002e-05 2020-08-07T15:08:48.206 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.207 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.212 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:48.212 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.838786+0000", 2020-08-07T15:08:48.213 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.48901084099999997 2020-08-07T15:08:48.213 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.213 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:48.213 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.213 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.214 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.214 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106240 3.7 3:ebdd8242:::benchmark_data_smithi074_35447_object13279:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:08:48.233 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:08:48.233 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 60.709591783, 2020-08-07T15:08:48.233 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.90079396, 2020-08-07T15:08:48.234 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:48.234 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:48.234 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:48.234 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:48.235 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:48.235 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106240 2020-08-07T15:08:48.235 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.235 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:48.235 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.236 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:48.236 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:08:48.236 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:48.236 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.236 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.237 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:48.237 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:08:48.237 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2522000000000003e-05 2020-08-07T15:08:48.237 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.237 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.260 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:48.261 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355118+0000", 2020-08-07T15:08:48.262 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6159999999999998e-06 2020-08-07T15:08:48.262 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.262 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.263 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:48.263 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355125+0000", 2020-08-07T15:08:48.274 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2699e-05 2020-08-07T15:08:48.274 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.274 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.275 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:48.275 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355138+0000", 2020-08-07T15:08:48.275 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2310000000000001e-06 2020-08-07T15:08:48.275 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.275 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.276 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:48.276 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355144+0000", 2020-08-07T15:08:48.277 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9855000000000003e-05 2020-08-07T15:08:48.277 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.277 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.277 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.277 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355194+0000", 2020-08-07T15:08:48.278 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112713 2020-08-07T15:08:48.278 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.278 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.278 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:48.278 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355306+0000", 2020-08-07T15:08:48.279 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019794600000000001 2020-08-07T15:08:48.279 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.279 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.279 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:08:48.290 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355504+0000", 2020-08-07T15:08:48.290 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0048346029999999998 2020-08-07T15:08:48.291 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.291 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.291 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:48.291 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.360339+0000", 2020-08-07T15:08:48.291 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00033646400000000002 2020-08-07T15:08:48.292 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.292 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.292 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:48.292 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.360675+0000", 2020-08-07T15:08:48.292 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89508481200000001 2020-08-07T15:08:48.293 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.293 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.293 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:48.293 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255760+0000", 2020-08-07T15:08:48.307 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5659999999999999e-06 2020-08-07T15:08:48.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.307 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.307 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:48.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255762+0000", 2020-08-07T15:08:48.308 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6595000000000003e-05 2020-08-07T15:08:48.308 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.308 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:48.309 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255818+0000", 2020-08-07T15:08:48.309 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1338000000000003e-05 2020-08-07T15:08:48.309 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.309 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.309 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:48.333 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255860+0000", 2020-08-07T15:08:48.333 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.90079396 2020-08-07T15:08:48.333 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.334 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:48.334 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.335 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.335 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106240 3.7 e59/20 ondisk, result = 0)", 2020-08-07T15:08:48.335 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:08:48.336 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 60.613195830999999, 2020-08-07T15:08:48.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80440215100000001, 2020-08-07T15:08:48.336 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:48.336 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:48.336 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:48.337 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.337 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:48.339 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:08:48.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:48.339 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.339 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.340 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:48.340 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:08:48.341 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0052999999999999e-05 2020-08-07T15:08:48.341 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.341 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:48.342 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451482+0000", 2020-08-07T15:08:48.342 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3229999999999999e-06 2020-08-07T15:08:48.342 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.342 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.343 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:48.343 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451486+0000", 2020-08-07T15:08:48.343 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80420304799999998 2020-08-07T15:08:48.343 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.344 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.344 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:48.344 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255689+0000", 2020-08-07T15:08:48.344 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7759999999999997e-06 2020-08-07T15:08:48.344 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.345 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.345 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:48.345 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255694+0000", 2020-08-07T15:08:48.345 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5828999999999998e-05 2020-08-07T15:08:48.345 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.346 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.346 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.346 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255740+0000", 2020-08-07T15:08:48.346 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1466e-05 2020-08-07T15:08:48.346 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.346 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.347 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:48.347 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255751+0000", 2020-08-07T15:08:48.347 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112656 2020-08-07T15:08:48.347 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.348 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:48.348 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255864+0000", 2020-08-07T15:08:48.349 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80440215100000001 2020-08-07T15:08:48.349 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.349 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:48.349 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.349 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.350 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.350 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:107951 3.d 3:bf34718c:::benchmark_data_smithi074_35447_object13493:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e62)", 2020-08-07T15:08:48.350 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.691353+0000", 2020-08-07T15:08:48.350 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.373304674000003, 2020-08-07T15:08:48.351 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.665804014, 2020-08-07T15:08:48.351 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:48.351 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:08:48.351 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:48.351 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:48.351 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:48.352 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 107951 2020-08-07T15:08:48.352 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.352 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:48.352 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.352 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:48.353 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691353+0000", 2020-08-07T15:08:48.353 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:48.353 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.353 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.353 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:48.354 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691353+0000", 2020-08-07T15:08:48.354 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2561999999999998e-05 2020-08-07T15:08:48.354 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.354 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.354 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:48.355 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691385+0000", 2020-08-07T15:08:48.355 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3030000000000001e-06 2020-08-07T15:08:48.355 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.355 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.355 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:48.356 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691390+0000", 2020-08-07T15:08:48.356 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5640000000000002e-06 2020-08-07T15:08:48.356 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.356 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.356 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:48.357 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691399+0000", 2020-08-07T15:08:48.357 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.425e-06 2020-08-07T15:08:48.357 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.357 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.357 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:48.358 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691404+0000", 2020-08-07T15:08:48.358 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0949999999999999e-05 2020-08-07T15:08:48.358 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.358 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.358 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.359 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691445+0000", 2020-08-07T15:08:48.360 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0067e-05 2020-08-07T15:08:48.361 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.361 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.362 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:08:48.362 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691455+0000", 2020-08-07T15:08:48.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66557516000000005 2020-08-07T15:08:48.363 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.363 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.365 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.357030+0000", 2020-08-07T15:08:48.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000126983 2020-08-07T15:08:48.366 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.367 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:48.368 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.357157+0000", 2020-08-07T15:08:48.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.665804014 2020-08-07T15:08:48.368 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.368 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:48.371 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.371 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.371 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:107952 3.d 3:bf34718c:::benchmark_data_smithi074_35447_object13493:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e62)", 2020-08-07T15:08:48.372 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.691432+0000", 2020-08-07T15:08:48.372 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.373225945000002, 2020-08-07T15:08:48.372 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66585522200000002, 2020-08-07T15:08:48.372 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:48.373 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:08:48.373 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:48.373 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:48.373 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:48.374 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 107952 2020-08-07T15:08:48.374 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.374 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:48.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.375 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:48.375 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691432+0000", 2020-08-07T15:08:48.375 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:48.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.376 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:48.376 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691432+0000", 2020-08-07T15:08:48.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6483e-05 2020-08-07T15:08:48.377 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:48.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691458+0000", 2020-08-07T15:08:48.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0259999999999999e-06 2020-08-07T15:08:48.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:48.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691461+0000", 2020-08-07T15:08:48.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9619999999999998e-06 2020-08-07T15:08:48.379 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:48.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691467+0000", 2020-08-07T15:08:48.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7429999999999998e-06 2020-08-07T15:08:48.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:48.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691470+0000", 2020-08-07T15:08:48.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4148e-05 2020-08-07T15:08:48.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.394 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.394 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.394 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691494+0000", 2020-08-07T15:08:48.394 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4599999999999996e-06 2020-08-07T15:08:48.394 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.395 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.395 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:08:48.395 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691498+0000", 2020-08-07T15:08:48.395 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66568584500000005 2020-08-07T15:08:48.395 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.396 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.396 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.396 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.357184+0000", 2020-08-07T15:08:48.396 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000102555 2020-08-07T15:08:48.396 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.403 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.403 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:48.404 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.357287+0000", 2020-08-07T15:08:48.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66585522200000002 2020-08-07T15:08:48.404 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.404 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:48.425 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.426 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.426 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108233 3.d 3:be154549:::benchmark_data_smithi074_35447_object13529:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e62)", 2020-08-07T15:08:48.426 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.691491+0000", 2020-08-07T15:08:48.426 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.373166138000002, 2020-08-07T15:08:48.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67823205099999995, 2020-08-07T15:08:48.427 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:48.427 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:48.427 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:48.427 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:48.428 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:48.428 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108233 2020-08-07T15:08:48.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.428 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:48.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.429 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:48.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691491+0000", 2020-08-07T15:08:48.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:48.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.431 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:48.432 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691491+0000", 2020-08-07T15:08:48.432 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6054000000000002e-05 2020-08-07T15:08:48.432 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.432 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.433 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:48.433 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691517+0000", 2020-08-07T15:08:48.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7949999999999999e-06 2020-08-07T15:08:48.433 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.433 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.434 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:48.434 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691520+0000", 2020-08-07T15:08:48.434 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4539999999999998e-06 2020-08-07T15:08:48.434 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.434 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.435 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:48.435 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691526+0000", 2020-08-07T15:08:48.435 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4109999999999999e-06 2020-08-07T15:08:48.436 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.436 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.437 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:48.437 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691528+0000", 2020-08-07T15:08:48.437 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1905e-05 2020-08-07T15:08:48.437 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.437 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.438 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.438 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691560+0000", 2020-08-07T15:08:48.438 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3219999999999998e-06 2020-08-07T15:08:48.438 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.438 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.439 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:08:48.439 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691566+0000", 2020-08-07T15:08:48.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66574513700000004 2020-08-07T15:08:48.439 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.439 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.440 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.440 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.357312+0000", 2020-08-07T15:08:48.440 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000216388 2020-08-07T15:08:48.440 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.441 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:48.441 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.357528+0000", 2020-08-07T15:08:48.441 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000184489 2020-08-07T15:08:48.441 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.442 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.442 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:08:48.442 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.357712+0000", 2020-08-07T15:08:48.442 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023584119999999998 2020-08-07T15:08:48.442 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.443 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.443 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:48.443 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.360071+0000", 2020-08-07T15:08:48.443 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1629999999999999e-06 2020-08-07T15:08:48.443 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.444 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:48.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.360072+0000", 2020-08-07T15:08:48.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2478999999999997e-05 2020-08-07T15:08:48.444 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.445 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.445 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:48.445 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.360124+0000", 2020-08-07T15:08:48.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0094617999999999994 2020-08-07T15:08:48.445 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.446 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.446 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:48.446 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.369586+0000", 2020-08-07T15:08:48.446 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7409e-05 2020-08-07T15:08:48.446 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:48.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.369634+0000", 2020-08-07T15:08:48.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2500000000000004e-07 2020-08-07T15:08:48.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.460 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.461 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:48.461 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.369635+0000", 2020-08-07T15:08:48.463 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8907999999999996e-05 2020-08-07T15:08:48.463 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.464 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.464 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:48.464 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.369723+0000", 2020-08-07T15:08:48.464 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67823205099999995 2020-08-07T15:08:48.465 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.465 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:48.465 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.465 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.465 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.466 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108234 3.d 3:be154549:::benchmark_data_smithi074_35447_object13529:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e62)", 2020-08-07T15:08:48.466 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.691556+0000", 2020-08-07T15:08:48.466 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.373101275000003, 2020-08-07T15:08:48.466 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67838000700000001, 2020-08-07T15:08:48.466 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:48.467 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:48.467 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:48.467 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:48.467 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:48.467 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108234 2020-08-07T15:08:48.467 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.468 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:48.468 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.468 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:48.468 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691556+0000", 2020-08-07T15:08:48.468 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:48.469 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.469 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.469 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:48.469 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691556+0000", 2020-08-07T15:08:48.469 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5318999999999998e-05 2020-08-07T15:08:48.470 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.470 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.470 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:48.470 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691602+0000", 2020-08-07T15:08:48.470 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.912e-06 2020-08-07T15:08:48.471 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.471 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.471 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:48.471 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691606+0000", 2020-08-07T15:08:48.471 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0250000000000002e-06 2020-08-07T15:08:48.472 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.472 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.473 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:48.473 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691614+0000", 2020-08-07T15:08:48.482 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1640000000000001e-06 2020-08-07T15:08:48.482 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.491 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.491 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:48.491 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691617+0000", 2020-08-07T15:08:48.491 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0720000000000002e-05 2020-08-07T15:08:48.492 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.492 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.492 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.493 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691637+0000", 2020-08-07T15:08:48.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5660000000000001e-06 2020-08-07T15:08:48.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.493 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.495 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:08:48.495 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691645+0000", 2020-08-07T15:08:48.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66654076600000001 2020-08-07T15:08:48.496 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.496 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.496 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.496 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.358186+0000", 2020-08-07T15:08:48.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3206000000000003e-05 2020-08-07T15:08:48.497 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.497 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.497 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:48.497 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.358259+0000", 2020-08-07T15:08:48.498 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000149815 2020-08-07T15:08:48.499 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.499 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.499 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:08:48.499 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.358409+0000", 2020-08-07T15:08:48.500 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0065748500000000001 2020-08-07T15:08:48.500 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.500 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.501 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:48.501 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.364984+0000", 2020-08-07T15:08:48.501 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4589999999999999e-06 2020-08-07T15:08:48.501 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.502 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.502 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:48.502 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.364985+0000", 2020-08-07T15:08:48.502 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5142000000000007e-05 2020-08-07T15:08:48.502 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.503 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.503 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:48.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365060+0000", 2020-08-07T15:08:48.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0047306620000000001 2020-08-07T15:08:48.510 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.510 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:48.510 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.369791+0000", 2020-08-07T15:08:48.511 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0852000000000005e-05 2020-08-07T15:08:48.511 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:48.519 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.369862+0000", 2020-08-07T15:08:48.520 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.133e-06 2020-08-07T15:08:48.520 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.520 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.521 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:48.521 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.369863+0000", 2020-08-07T15:08:48.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3416000000000003e-05 2020-08-07T15:08:48.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.522 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.522 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:48.522 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.369936+0000", 2020-08-07T15:08:48.522 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67838000700000001 2020-08-07T15:08:48.522 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.523 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:48.523 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.523 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.550 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.551 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108235 3.d 3:be154549:::benchmark_data_smithi074_35447_object13529:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e62)", 2020-08-07T15:08:48.551 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.691645+0000", 2020-08-07T15:08:48.552 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.373013000999997, 2020-08-07T15:08:48.552 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67849062100000002, 2020-08-07T15:08:48.552 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:48.553 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:48.553 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:48.553 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:48.553 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:48.554 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108235 2020-08-07T15:08:48.554 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.563 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:48.563 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.563 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:48.564 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691645+0000", 2020-08-07T15:08:48.564 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:48.564 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.564 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:48.565 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691645+0000", 2020-08-07T15:08:48.565 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7483e-05 2020-08-07T15:08:48.565 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.565 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.565 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:48.566 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691682+0000", 2020-08-07T15:08:48.566 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2610000000000002e-06 2020-08-07T15:08:48.588 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.591 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:48.592 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691685+0000", 2020-08-07T15:08:48.592 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3719999999999994e-06 2020-08-07T15:08:48.592 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.592 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.592 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:48.593 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691695+0000", 2020-08-07T15:08:48.593 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2339999999999999e-06 2020-08-07T15:08:48.593 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.611 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.611 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:48.612 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691698+0000", 2020-08-07T15:08:48.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4182e-05 2020-08-07T15:08:48.612 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.613 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.613 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691722+0000", 2020-08-07T15:08:48.631 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3889999999999998e-06 2020-08-07T15:08:48.631 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.631 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.638 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:08:48.639 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691727+0000", 2020-08-07T15:08:48.639 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.667110651 2020-08-07T15:08:48.639 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.639 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.640 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.640 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.358837+0000", 2020-08-07T15:08:48.640 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000102965 2020-08-07T15:08:48.640 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.640 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.641 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:48.641 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.358940+0000", 2020-08-07T15:08:48.641 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00022451700000000001 2020-08-07T15:08:48.641 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.641 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.649 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:08:48.650 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.359165+0000", 2020-08-07T15:08:48.650 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0059109999999999996 2020-08-07T15:08:48.651 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.651 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.651 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:48.651 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365076+0000", 2020-08-07T15:08:48.651 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5300000000000002e-07 2020-08-07T15:08:48.652 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.652 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.652 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:48.652 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365077+0000", 2020-08-07T15:08:48.652 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7922000000000005e-05 2020-08-07T15:08:48.653 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.653 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.653 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:48.653 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365145+0000", 2020-08-07T15:08:48.653 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004866085 2020-08-07T15:08:48.654 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.654 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.654 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:48.654 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370011+0000", 2020-08-07T15:08:48.654 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3035999999999998e-05 2020-08-07T15:08:48.655 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.655 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.655 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:48.655 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370064+0000", 2020-08-07T15:08:48.655 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.062e-06 2020-08-07T15:08:48.656 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.656 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.656 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:48.656 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370065+0000", 2020-08-07T15:08:48.656 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0508999999999999e-05 2020-08-07T15:08:48.657 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.657 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.657 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:48.657 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370135+0000", 2020-08-07T15:08:48.657 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67849062100000002 2020-08-07T15:08:48.658 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.658 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:48.658 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.658 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.659 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.659 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108236 3.d 3:be154549:::benchmark_data_smithi074_35447_object13529:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e62)", 2020-08-07T15:08:48.659 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.691723+0000", 2020-08-07T15:08:48.659 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.372935073000001, 2020-08-07T15:08:48.659 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67855064399999998, 2020-08-07T15:08:48.660 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:48.660 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:48.660 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:48.660 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:48.661 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:48.661 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108236 2020-08-07T15:08:48.661 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.661 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:48.661 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.662 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:48.662 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691723+0000", 2020-08-07T15:08:48.662 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:48.662 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.662 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.663 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:48.663 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691723+0000", 2020-08-07T15:08:48.663 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4675999999999999e-05 2020-08-07T15:08:48.663 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.663 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.664 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:48.664 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691747+0000", 2020-08-07T15:08:48.664 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8499999999999998e-06 2020-08-07T15:08:48.664 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.665 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:48.665 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691750+0000", 2020-08-07T15:08:48.665 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4760000000000001e-06 2020-08-07T15:08:48.665 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.665 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.666 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:48.666 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691758+0000", 2020-08-07T15:08:48.666 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4909999999999996e-06 2020-08-07T15:08:48.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.667 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:48.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691763+0000", 2020-08-07T15:08:48.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7676000000000001e-05 2020-08-07T15:08:48.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.668 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691791+0000", 2020-08-07T15:08:48.668 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5159999999999996e-06 2020-08-07T15:08:48.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:08:48.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691795+0000", 2020-08-07T15:08:48.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.667739164 2020-08-07T15:08:48.670 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.670 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.670 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.670 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.359534+0000", 2020-08-07T15:08:48.670 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8170000000000006e-05 2020-08-07T15:08:48.670 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.671 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.671 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:48.671 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.359603+0000", 2020-08-07T15:08:48.671 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013336399999999999 2020-08-07T15:08:48.671 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.672 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.672 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:08:48.672 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.359736+0000", 2020-08-07T15:08:48.672 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00542254 2020-08-07T15:08:48.688 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.690 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.691 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:48.691 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365158+0000", 2020-08-07T15:08:48.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8000000000000005e-07 2020-08-07T15:08:48.701 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.710 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:48.710 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365159+0000", 2020-08-07T15:08:48.710 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4726000000000003e-05 2020-08-07T15:08:48.711 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.711 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:48.711 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365224+0000", 2020-08-07T15:08:48.712 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004972537 2020-08-07T15:08:48.712 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.712 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.712 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:48.712 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370196+0000", 2020-08-07T15:08:48.712 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3263999999999999e-05 2020-08-07T15:08:48.713 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.713 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.713 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:48.713 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370230+0000", 2020-08-07T15:08:48.734 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5700000000000001e-07 2020-08-07T15:08:48.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.737 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.737 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:48.738 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370231+0000", 2020-08-07T15:08:48.738 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2556999999999999e-05 2020-08-07T15:08:48.740 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.741 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.742 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:48.743 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370273+0000", 2020-08-07T15:08:48.743 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67855064399999998 2020-08-07T15:08:48.744 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.744 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:48.744 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.746 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.748 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108237 3.d 3:be154549:::benchmark_data_smithi074_35447_object13529:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e62)", 2020-08-07T15:08:48.749 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.691798+0000", 2020-08-07T15:08:48.750 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.372859957999999, 2020-08-07T15:08:48.750 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67865340900000004, 2020-08-07T15:08:48.750 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:48.751 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:48.751 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:48.752 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:48.752 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:48.752 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108237 2020-08-07T15:08:48.752 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.753 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:48.753 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.753 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:48.753 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691798+0000", 2020-08-07T15:08:48.753 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:48.754 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.754 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.754 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:48.754 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691798+0000", 2020-08-07T15:08:48.754 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9239000000000002e-05 2020-08-07T15:08:48.755 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.755 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:48.756 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691837+0000", 2020-08-07T15:08:48.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1839999999999999e-06 2020-08-07T15:08:48.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:48.757 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691840+0000", 2020-08-07T15:08:48.757 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9400000000000002e-06 2020-08-07T15:08:48.757 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.757 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:48.758 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691848+0000", 2020-08-07T15:08:48.758 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2740000000000001e-06 2020-08-07T15:08:48.758 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.758 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.758 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:48.759 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691852+0000", 2020-08-07T15:08:48.759 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0021e-05 2020-08-07T15:08:48.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.762 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691872+0000", 2020-08-07T15:08:48.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3810000000000001e-06 2020-08-07T15:08:48.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.763 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:08:48.763 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691877+0000", 2020-08-07T15:08:48.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66829760299999996 2020-08-07T15:08:48.764 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.764 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.360174+0000", 2020-08-07T15:08:48.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012061999999999999 2020-08-07T15:08:48.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.765 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:48.765 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.360295+0000", 2020-08-07T15:08:48.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020822399999999999 2020-08-07T15:08:48.766 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.766 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:08:48.766 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.360503+0000", 2020-08-07T15:08:48.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0047338340000000001 2020-08-07T15:08:48.766 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.770 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:48.770 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365237+0000", 2020-08-07T15:08:48.770 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3499999999999995e-07 2020-08-07T15:08:48.770 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.771 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.771 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:48.771 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365238+0000", 2020-08-07T15:08:48.771 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7602999999999996e-05 2020-08-07T15:08:48.771 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.772 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.772 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:48.772 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365305+0000", 2020-08-07T15:08:48.772 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0050325420000000001 2020-08-07T15:08:48.772 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.773 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.773 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:48.773 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370338+0000", 2020-08-07T15:08:48.773 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5608000000000001e-05 2020-08-07T15:08:48.773 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:48.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370383+0000", 2020-08-07T15:08:48.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7499999999999998e-07 2020-08-07T15:08:48.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:48.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370384+0000", 2020-08-07T15:08:48.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6625999999999995e-05 2020-08-07T15:08:48.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.779 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:48.779 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370451+0000", 2020-08-07T15:08:48.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67865340900000004 2020-08-07T15:08:48.780 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.780 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:48.780 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.780 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.781 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108238 3.d 3:be154549:::benchmark_data_smithi074_35447_object13529:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e62)", 2020-08-07T15:08:48.781 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.691877+0000", 2020-08-07T15:08:48.781 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.372780904999999, 2020-08-07T15:08:48.781 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67875236900000002, 2020-08-07T15:08:48.781 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:48.782 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:48.782 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:48.782 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:48.782 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:48.782 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108238 2020-08-07T15:08:48.783 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.783 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:48.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:48.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691877+0000", 2020-08-07T15:08:48.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:48.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:48.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691877+0000", 2020-08-07T15:08:48.785 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5123999999999999e-05 2020-08-07T15:08:48.785 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:48.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691902+0000", 2020-08-07T15:08:48.786 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8270000000000002e-06 2020-08-07T15:08:48.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.786 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:48.786 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691905+0000", 2020-08-07T15:08:48.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.614e-06 2020-08-07T15:08:48.787 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.787 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:48.787 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691910+0000", 2020-08-07T15:08:48.788 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8260000000000001e-06 2020-08-07T15:08:48.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.788 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:48.788 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691913+0000", 2020-08-07T15:08:48.788 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7557999999999999e-05 2020-08-07T15:08:48.789 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.789 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.789 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.789 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691941+0000", 2020-08-07T15:08:48.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2230000000000001e-06 2020-08-07T15:08:48.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.790 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:08:48.791 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691945+0000", 2020-08-07T15:08:48.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66907557799999995 2020-08-07T15:08:48.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.792 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.792 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.361020+0000", 2020-08-07T15:08:48.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013186799999999999 2020-08-07T15:08:48.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.793 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.793 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:48.793 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.361152+0000", 2020-08-07T15:08:48.793 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024732399999999999 2020-08-07T15:08:48.793 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.794 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.794 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:08:48.794 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.361400+0000", 2020-08-07T15:08:48.794 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00391735 2020-08-07T15:08:48.794 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.795 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.795 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:48.795 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365317+0000", 2020-08-07T15:08:48.795 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6700000000000003e-07 2020-08-07T15:08:48.795 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.796 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:48.796 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365318+0000", 2020-08-07T15:08:48.796 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6130000000000006e-05 2020-08-07T15:08:48.796 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.797 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.797 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:48.797 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365384+0000", 2020-08-07T15:08:48.797 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005119129 2020-08-07T15:08:48.797 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.798 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.798 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:48.798 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370503+0000", 2020-08-07T15:08:48.798 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0532000000000003e-05 2020-08-07T15:08:48.798 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:48.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370563+0000", 2020-08-07T15:08:48.799 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7600000000000006e-07 2020-08-07T15:08:48.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:48.800 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370564+0000", 2020-08-07T15:08:48.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4743e-05 2020-08-07T15:08:48.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.801 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:48.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370629+0000", 2020-08-07T15:08:48.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67875236900000002 2020-08-07T15:08:48.801 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.801 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:48.802 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.802 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.802 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.802 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108239 3.d 3:be154549:::benchmark_data_smithi074_35447_object13529:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e62)", 2020-08-07T15:08:48.802 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.691951+0000", 2020-08-07T15:08:48.803 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.372706831999999, 2020-08-07T15:08:48.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67888694500000002, 2020-08-07T15:08:48.803 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:48.803 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:48.804 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:48.804 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:48.804 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:48.804 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108239 2020-08-07T15:08:48.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.805 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:48.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:48.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691951+0000", 2020-08-07T15:08:48.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:48.806 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:48.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691951+0000", 2020-08-07T15:08:48.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7484999999999998e-05 2020-08-07T15:08:48.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:48.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691988+0000", 2020-08-07T15:08:48.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2739999999999999e-06 2020-08-07T15:08:48.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.808 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:48.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691992+0000", 2020-08-07T15:08:48.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6389999999999992e-06 2020-08-07T15:08:48.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.809 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:48.809 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.691999+0000", 2020-08-07T15:08:48.809 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4410000000000002e-06 2020-08-07T15:08:48.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.810 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.810 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:48.810 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.692003+0000", 2020-08-07T15:08:48.810 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9826999999999998e-05 2020-08-07T15:08:48.810 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.811 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.811 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.692022+0000", 2020-08-07T15:08:48.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0990000000000001e-06 2020-08-07T15:08:48.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:08:48.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.692027+0000", 2020-08-07T15:08:48.813 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66992547099999999 2020-08-07T15:08:48.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.813 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.361952+0000", 2020-08-07T15:08:48.814 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010414199999999999 2020-08-07T15:08:48.814 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.814 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.814 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:48.814 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.362056+0000", 2020-08-07T15:08:48.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015952499999999999 2020-08-07T15:08:48.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.815 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:08:48.815 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.362216+0000", 2020-08-07T15:08:48.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045867449999999997 2020-08-07T15:08:48.816 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:48.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.366802+0000", 2020-08-07T15:08:48.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.178e-06 2020-08-07T15:08:48.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.817 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:48.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.366804+0000", 2020-08-07T15:08:48.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5803999999999995e-05 2020-08-07T15:08:48.850 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.851 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:48.851 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.366879+0000", 2020-08-07T15:08:48.851 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003807286 2020-08-07T15:08:48.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:48.853 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370687+0000", 2020-08-07T15:08:48.853 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1903000000000005e-05 2020-08-07T15:08:48.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:48.875 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370769+0000", 2020-08-07T15:08:48.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6899999999999997e-07 2020-08-07T15:08:48.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.876 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:48.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370769+0000", 2020-08-07T15:08:48.878 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8557000000000001e-05 2020-08-07T15:08:48.878 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.878 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.879 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:48.879 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370838+0000", 2020-08-07T15:08:48.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67888694500000002 2020-08-07T15:08:48.880 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.881 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:48.881 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:48.881 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.881 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.882 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108240 3.d 3:be154549:::benchmark_data_smithi074_35447_object13529:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e62)", 2020-08-07T15:08:48.882 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.692026+0000", 2020-08-07T15:08:48.882 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.372631978999998, 2020-08-07T15:08:48.882 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67892993700000004, 2020-08-07T15:08:48.882 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:48.883 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:48.883 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:48.883 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:48.883 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:48.883 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108240 2020-08-07T15:08:48.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.884 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:48.884 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.884 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:48.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.692026+0000", 2020-08-07T15:08:48.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:48.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.895 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.895 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:48.895 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.692026+0000", 2020-08-07T15:08:48.895 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6211e-05 2020-08-07T15:08:48.895 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.896 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:48.896 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.692052+0000", 2020-08-07T15:08:48.896 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8830000000000002e-06 2020-08-07T15:08:48.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.906 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:48.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.692055+0000", 2020-08-07T15:08:48.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.609e-06 2020-08-07T15:08:48.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.908 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:48.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.692060+0000", 2020-08-07T15:08:48.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4889999999999998e-06 2020-08-07T15:08:48.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:48.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.692063+0000", 2020-08-07T15:08:48.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9899e-05 2020-08-07T15:08:48.926 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.692083+0000", 2020-08-07T15:08:48.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2910000000000001e-06 2020-08-07T15:08:48.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.942 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:08:48.942 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.692087+0000", 2020-08-07T15:08:48.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67051431100000003 2020-08-07T15:08:48.942 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:48.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.362601+0000", 2020-08-07T15:08:48.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000118664 2020-08-07T15:08:48.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.944 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:48.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.362720+0000", 2020-08-07T15:08:48.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000194902 2020-08-07T15:08:48.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:08:48.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.362915+0000", 2020-08-07T15:08:48.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039819479999999999 2020-08-07T15:08:48.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.978 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:48.978 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.366897+0000", 2020-08-07T15:08:48.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8800000000000002e-07 2020-08-07T15:08:48.979 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.979 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.979 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:48.979 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.366898+0000", 2020-08-07T15:08:48.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3904000000000003e-05 2020-08-07T15:08:48.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:48.980 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.366972+0000", 2020-08-07T15:08:48.980 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003757608 2020-08-07T15:08:48.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.981 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:48.981 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370729+0000", 2020-08-07T15:08:48.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015924999999999999 2020-08-07T15:08:48.998 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.998 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.998 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:48.998 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370888+0000", 2020-08-07T15:08:48.999 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7199999999999999e-07 2020-08-07T15:08:48.999 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:48.999 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:48.999 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:48.999 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370889+0000", 2020-08-07T15:08:49.000 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6607999999999997e-05 2020-08-07T15:08:49.000 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.000 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.000 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:49.000 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.370956+0000", 2020-08-07T15:08:49.001 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67892993700000004 2020-08-07T15:08:49.001 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:49.001 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:49.001 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:49.001 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.002 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.002 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108505 3.d 3:b25992ab:::benchmark_data_smithi074_35447_object13563:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e62)", 2020-08-07T15:08:49.002 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.820278+0000", 2020-08-07T15:08:49.002 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.244379352999999, 2020-08-07T15:08:49.002 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.55083482699999997, 2020-08-07T15:08:49.003 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:49.003 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:49.003 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:49.003 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:49.003 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:49.004 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108505 2020-08-07T15:08:49.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.004 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:49.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:49.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.820278+0000", 2020-08-07T15:08:49.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:49.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:49.033 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.820278+0000", 2020-08-07T15:08:49.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8326999999999998e-05 2020-08-07T15:08:49.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.034 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:49.034 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.820307+0000", 2020-08-07T15:08:49.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5059999999999997e-06 2020-08-07T15:08:49.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.034 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:49.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.820311+0000", 2020-08-07T15:08:49.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2397e-05 2020-08-07T15:08:49.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:49.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.820323+0000", 2020-08-07T15:08:49.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4620000000000004e-06 2020-08-07T15:08:49.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.037 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:49.037 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.820329+0000", 2020-08-07T15:08:49.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0113e-05 2020-08-07T15:08:49.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:49.058 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.820369+0000", 2020-08-07T15:08:49.058 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.79e-06 2020-08-07T15:08:49.058 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.059 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.059 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:08:49.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.820377+0000", 2020-08-07T15:08:49.059 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.54296667899999995 2020-08-07T15:08:49.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:49.060 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.363344+0000", 2020-08-07T15:08:49.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015576300000000001 2020-08-07T15:08:49.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.061 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:49.080 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.363499+0000", 2020-08-07T15:08:49.081 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00021215700000000001 2020-08-07T15:08:49.081 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.082 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:08:49.082 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.363711+0000", 2020-08-07T15:08:49.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003271422 2020-08-07T15:08:49.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.083 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:49.084 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.366983+0000", 2020-08-07T15:08:49.085 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4899999999999995e-07 2020-08-07T15:08:49.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.106 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.106 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:49.107 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.366983+0000", 2020-08-07T15:08:49.107 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5705999999999997e-05 2020-08-07T15:08:49.107 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.108 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.108 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:49.108 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.367049+0000", 2020-08-07T15:08:49.109 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003955554 2020-08-07T15:08:49.109 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.109 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:49.117 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.371005+0000", 2020-08-07T15:08:49.118 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0803999999999998e-05 2020-08-07T15:08:49.118 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.118 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.118 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:49.119 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.371046+0000", 2020-08-07T15:08:49.119 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8100000000000003e-07 2020-08-07T15:08:49.126 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.126 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.126 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:49.127 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.371046+0000", 2020-08-07T15:08:49.127 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7016999999999999e-05 2020-08-07T15:08:49.127 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.127 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.127 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:49.128 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.371113+0000", 2020-08-07T15:08:49.128 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.55083482699999997 2020-08-07T15:08:49.128 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:49.128 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:49.128 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:49.129 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.129 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.129 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108506 3.d 3:b25992ab:::benchmark_data_smithi074_35447_object13563:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e62)", 2020-08-07T15:08:49.129 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.821020+0000", 2020-08-07T15:08:49.141 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.243637567999997, 2020-08-07T15:08:49.142 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.55317842500000003, 2020-08-07T15:08:49.142 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:49.142 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:49.150 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:49.151 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:49.151 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:49.151 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108506 2020-08-07T15:08:49.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.152 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:49.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:49.152 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.821020+0000", 2020-08-07T15:08:49.153 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:49.153 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.166 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.166 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:49.167 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.821020+0000", 2020-08-07T15:08:49.167 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8736000000000001e-05 2020-08-07T15:08:49.167 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.168 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.168 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:49.168 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.821049+0000", 2020-08-07T15:08:49.169 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.72e-06 2020-08-07T15:08:49.178 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.179 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.179 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:49.179 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.821052+0000", 2020-08-07T15:08:49.180 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3499999999999997e-06 2020-08-07T15:08:49.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.180 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.181 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:49.181 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.821061+0000", 2020-08-07T15:08:49.181 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8620000000000002e-06 2020-08-07T15:08:49.182 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:49.198 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.821066+0000", 2020-08-07T15:08:49.199 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5120000000000003e-05 2020-08-07T15:08:49.199 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.199 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.200 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:49.200 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.821101+0000", 2020-08-07T15:08:49.200 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5800000000000003e-06 2020-08-07T15:08:49.200 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.201 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.201 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:08:49.201 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.821108+0000", 2020-08-07T15:08:49.201 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.54305088899999998 2020-08-07T15:08:49.202 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.202 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.202 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:49.202 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.364159+0000", 2020-08-07T15:08:49.202 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000116743 2020-08-07T15:08:49.203 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.203 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.203 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:49.203 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.364276+0000", 2020-08-07T15:08:49.203 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00022238499999999999 2020-08-07T15:08:49.204 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.204 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.204 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:08:49.204 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.364498+0000", 2020-08-07T15:08:49.204 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042165290000000001 2020-08-07T15:08:49.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.205 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.205 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:49.205 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.368715+0000", 2020-08-07T15:08:49.228 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2529999999999999e-06 2020-08-07T15:08:49.229 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.229 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.230 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:49.230 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.368716+0000", 2020-08-07T15:08:49.230 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7997999999999995e-05 2020-08-07T15:08:49.230 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.231 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.231 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:49.231 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.368784+0000", 2020-08-07T15:08:49.231 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023803190000000001 2020-08-07T15:08:49.231 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.232 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.232 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:49.232 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.371164+0000", 2020-08-07T15:08:49.232 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0029631760000000001 2020-08-07T15:08:49.232 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.233 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.233 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:49.233 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.374128+0000", 2020-08-07T15:08:49.233 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0240000000000001e-06 2020-08-07T15:08:49.250 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.250 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.250 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:49.251 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.374129+0000", 2020-08-07T15:08:49.251 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9740999999999995e-05 2020-08-07T15:08:49.252 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.252 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.252 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:49.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.374198+0000", 2020-08-07T15:08:49.253 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.55317842500000003 2020-08-07T15:08:49.253 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:49.253 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:49.255 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:49.255 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.255 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.256 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108507 3.d 3:b25992ab:::benchmark_data_smithi074_35447_object13563:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e62)", 2020-08-07T15:08:49.256 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.823955+0000", 2020-08-07T15:08:49.256 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.240702947000003, 2020-08-07T15:08:49.256 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.55129072499999998, 2020-08-07T15:08:49.256 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:49.257 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:49.257 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:49.257 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:49.257 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:49.257 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108507 2020-08-07T15:08:49.258 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.258 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:49.258 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.258 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:49.258 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.823955+0000", 2020-08-07T15:08:49.259 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:49.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.259 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.259 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:49.259 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.823955+0000", 2020-08-07T15:08:49.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.955e-05 2020-08-07T15:08:49.260 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.260 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.260 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:49.260 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.823984+0000", 2020-08-07T15:08:49.261 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2259999999999999e-06 2020-08-07T15:08:49.261 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.261 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.261 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:49.262 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.823988+0000", 2020-08-07T15:08:49.262 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6879999999999994e-06 2020-08-07T15:08:49.262 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.262 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.262 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:49.263 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.823998+0000", 2020-08-07T15:08:49.263 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5119999999999999e-06 2020-08-07T15:08:49.263 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.263 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.263 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:49.263 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.824004+0000", 2020-08-07T15:08:49.264 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8118000000000003e-05 2020-08-07T15:08:49.264 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.264 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.264 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:49.264 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.824042+0000", 2020-08-07T15:08:49.265 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0639999999999996e-06 2020-08-07T15:08:49.265 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.265 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.265 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:08:49.265 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.824049+0000", 2020-08-07T15:08:49.266 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.541388706 2020-08-07T15:08:49.266 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.266 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.266 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:49.266 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365438+0000", 2020-08-07T15:08:49.267 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000125785 2020-08-07T15:08:49.267 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.267 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.267 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:49.267 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365563+0000", 2020-08-07T15:08:49.268 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016269799999999999 2020-08-07T15:08:49.268 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.268 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.268 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:08:49.268 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.365726+0000", 2020-08-07T15:08:49.269 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0035814530000000001 2020-08-07T15:08:49.269 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.269 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.269 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:49.269 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.369307+0000", 2020-08-07T15:08:49.270 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.17e-06 2020-08-07T15:08:49.270 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.270 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.270 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:49.270 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.369309+0000", 2020-08-07T15:08:49.271 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1161000000000005e-05 2020-08-07T15:08:49.271 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.271 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:49.271 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.369380+0000", 2020-08-07T15:08:49.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0051009820000000004 2020-08-07T15:08:49.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:49.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.374481+0000", 2020-08-07T15:08:49.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00067293099999999998 2020-08-07T15:08:49.273 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.273 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:49.273 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.375154+0000", 2020-08-07T15:08:49.274 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9600000000000008e-07 2020-08-07T15:08:49.274 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.274 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.274 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:49.274 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.375155+0000", 2020-08-07T15:08:49.274 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0685000000000003e-05 2020-08-07T15:08:49.275 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:49.275 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:49.275 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:49.275 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.375245+0000", 2020-08-07T15:08:49.275 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.55129072499999998 2020-08-07T15:08:49.276 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:49.276 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:49.276 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:49.276 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:49.276 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:49.277 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:49.278 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:08:49.280 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:08:49.280 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 22 16 31845 31829 11.302 6.47656 0.0072115 0.0110059 2020-08-07T15:08:49.281 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 23 16 31847 31831 10.8113 0.015625 0.432044 0.0110266 2020-08-07T15:08:49.281 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:49.277+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:49.401 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:49.400+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:49.420 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:49.420 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:08:49.421 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2020-08-07T15:08:49.421 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:49.427 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:08:49.469 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:08:49.491 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:08:49.505 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:49.501+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:49.519 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:08:49.520 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:08:49.587 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:49.587 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:08:49.588 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:08:49.588 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:08:49.588 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:49.599 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:08:49.604 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:49.603+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:49.710 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:49.705+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:49.723 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:49.723 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:08:49.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:08:49.724 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:08:49.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.724 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:108376 3.3 e62/20 ondisk, result = 0)", 2020-08-07T15:08:49.724 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:08:49.730 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 59.034429203000002, 2020-08-07T15:08:49.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62926872599999994, 2020-08-07T15:08:49.730 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:49.730 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:49.731 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:49.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:49.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:08:49.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:49.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:49.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:08:49.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8491999999999998e-05 2020-08-07T15:08:49.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:49.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683061+0000", 2020-08-07T15:08:49.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9099999999999998e-06 2020-08-07T15:08:49.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:49.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683065+0000", 2020-08-07T15:08:49.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62906670600000003 2020-08-07T15:08:49.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:49.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312131+0000", 2020-08-07T15:08:49.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2147000000000001e-05 2020-08-07T15:08:49.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:49.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312143+0000", 2020-08-07T15:08:49.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5152e-05 2020-08-07T15:08:49.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:49.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312239+0000", 2020-08-07T15:08:49.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5089999999999993e-06 2020-08-07T15:08:49.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:49.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312247+0000", 2020-08-07T15:08:49.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3810000000000001e-05 2020-08-07T15:08:49.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:49.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312311+0000", 2020-08-07T15:08:49.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62926872599999994 2020-08-07T15:08:49.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:49.765 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:49.765 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:49.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.778 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "MOSDPGPushReply(3.1 62/60 [PushReplyOp(3:8329f37e:::benchmark_data_smithi074_35447_object11915:head)])", 2020-08-07T15:08:49.778 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:08:49.778 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.859071186999998, 2020-08-07T15:08:49.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996, 2020-08-07T15:08:49.779 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:49.779 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:08:49.779 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:49.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:49.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:08:49.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:49.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:49.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:08:49.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5372000000000001e-05 2020-08-07T15:08:49.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:49.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858416+0000", 2020-08-07T15:08:49.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6899999999999998e-06 2020-08-07T15:08:49.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:49.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858419+0000", 2020-08-07T15:08:49.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63907904299999996 2020-08-07T15:08:49.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:49.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497498+0000", 2020-08-07T15:08:49.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3332999999999999e-05 2020-08-07T15:08:49.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:49.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497512+0000", 2020-08-07T15:08:49.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000450875 2020-08-07T15:08:49.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:49.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497962+0000", 2020-08-07T15:08:49.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000267779 2020-08-07T15:08:49.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:49.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.498230+0000", 2020-08-07T15:08:49.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996 2020-08-07T15:08:49.803 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:49.803 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:49.803 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:49.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.804 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:145792 3.1 3:881e0420:::benchmark_data_smithi074_35447_object16183:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:08:49.805 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:08:49.806 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.050665305999999, 2020-08-07T15:08:49.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997, 2020-08-07T15:08:49.806 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:49.806 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:49.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:49.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:08:49.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:49.807 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 145792 2020-08-07T15:08:49.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.808 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:49.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:49.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:08:49.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:49.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:49.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:08:49.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7489000000000001e-05 2020-08-07T15:08:49.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:49.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666824+0000", 2020-08-07T15:08:49.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0169999999999998e-06 2020-08-07T15:08:49.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:49.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666828+0000", 2020-08-07T15:08:49.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9829999999999996e-06 2020-08-07T15:08:49.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:49.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666836+0000", 2020-08-07T15:08:49.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.481e-06 2020-08-07T15:08:49.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:49.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666840+0000", 2020-08-07T15:08:49.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5299000000000001e-05 2020-08-07T15:08:49.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:49.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666885+0000", 2020-08-07T15:08:49.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014456500000000001 2020-08-07T15:08:49.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:49.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667030+0000", 2020-08-07T15:08:49.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017677799999999999 2020-08-07T15:08:49.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,5", 2020-08-07T15:08:49.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667207+0000", 2020-08-07T15:08:49.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004149649 2020-08-07T15:08:49.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:49.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671356+0000", 2020-08-07T15:08:49.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019339649999999999 2020-08-07T15:08:49.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:49.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673290+0000", 2020-08-07T15:08:49.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73346620500000004 2020-08-07T15:08:49.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:49.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406756+0000", 2020-08-07T15:08:49.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4700000000000001e-06 2020-08-07T15:08:49.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:49.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406759+0000", 2020-08-07T15:08:49.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6258999999999998e-05 2020-08-07T15:08:49.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:49.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406815+0000", 2020-08-07T15:08:49.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0126000000000002e-05 2020-08-07T15:08:49.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:49.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406855+0000", 2020-08-07T15:08:49.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997 2020-08-07T15:08:49.837 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:49.837 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:49.837 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:49.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.838 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:145792 3.1 e75/67 ondisk, result = 0)", 2020-08-07T15:08:49.838 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:08:49.838 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.046280418000002, 2020-08-07T15:08:49.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003, 2020-08-07T15:08:49.839 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:49.839 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:08:49.840 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:49.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:49.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:08:49.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:49.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:49.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:08:49.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2830999999999998e-05 2020-08-07T15:08:49.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:49.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671224+0000", 2020-08-07T15:08:49.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:08:49.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:49.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671229+0000", 2020-08-07T15:08:49.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.735414864 2020-08-07T15:08:49.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:49.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406644+0000", 2020-08-07T15:08:49.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1250000000000004e-06 2020-08-07T15:08:49.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:49.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406651+0000", 2020-08-07T15:08:49.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6567e-05 2020-08-07T15:08:49.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:49.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406737+0000", 2020-08-07T15:08:49.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0054000000000001e-05 2020-08-07T15:08:49.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:49.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406747+0000", 2020-08-07T15:08:49.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011391999999999999 2020-08-07T15:08:49.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:49.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406861+0000", 2020-08-07T15:08:49.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003 2020-08-07T15:08:49.867 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:49.867 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:49.867 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:49.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.868 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30921 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:08:49.868 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:08:49.868 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.011532044, 2020-08-07T15:08:49.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001, 2020-08-07T15:08:49.869 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:49.869 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:08:49.870 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:49.870 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:49.870 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:49.870 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30921 2020-08-07T15:08:49.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.871 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:49.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:49.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:08:49.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:49.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:49.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:08:49.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6954999999999998e-05 2020-08-07T15:08:49.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:49.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706016+0000", 2020-08-07T15:08:49.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0229999999999992e-06 2020-08-07T15:08:49.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:49.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706024+0000", 2020-08-07T15:08:49.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.432e-05 2020-08-07T15:08:49.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:49.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706039+0000", 2020-08-07T15:08:49.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8490000000000006e-06 2020-08-07T15:08:49.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:49.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706046+0000", 2020-08-07T15:08:49.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6837999999999999e-05 2020-08-07T15:08:49.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:49.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706113+0000", 2020-08-07T15:08:49.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3553000000000001e-05 2020-08-07T15:08:49.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:49.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706127+0000", 2020-08-07T15:08:49.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77277201699999998 2020-08-07T15:08:49.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:49.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.478899+0000", 2020-08-07T15:08:49.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093978599999999996 2020-08-07T15:08:49.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:08:49.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479839+0000", 2020-08-07T15:08:49.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010376029 2020-08-07T15:08:49.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:49.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490215+0000", 2020-08-07T15:08:49.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000190657 2020-08-07T15:08:49.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:49.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490405+0000", 2020-08-07T15:08:49.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001 2020-08-07T15:08:49.886 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:49.887 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:49.887 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:49.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.888 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30922 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:08:49.888 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:08:49.888 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.0113718810000001, 2020-08-07T15:08:49.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247, 2020-08-07T15:08:49.888 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:49.889 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:08:49.889 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:49.889 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:49.889 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:49.889 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30922 2020-08-07T15:08:49.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.890 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:49.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:49.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:08:49.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:49.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:49.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:08:49.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9178999999999997e-05 2020-08-07T15:08:49.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:49.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706159+0000", 2020-08-07T15:08:49.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4999999999999999e-06 2020-08-07T15:08:49.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:49.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706164+0000", 2020-08-07T15:08:49.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1792e-05 2020-08-07T15:08:49.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:49.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706176+0000", 2020-08-07T15:08:49.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.139e-06 2020-08-07T15:08:49.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:49.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706182+0000", 2020-08-07T15:08:49.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039724900000000002 2020-08-07T15:08:49.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:49.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706579+0000", 2020-08-07T15:08:49.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1191e-05 2020-08-07T15:08:49.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:49.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706591+0000", 2020-08-07T15:08:49.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329101300000003 2020-08-07T15:08:49.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:49.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479882+0000", 2020-08-07T15:08:49.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9565000000000001e-05 2020-08-07T15:08:49.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:08:49.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479971+0000", 2020-08-07T15:08:49.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010471263 2020-08-07T15:08:49.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:49.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490442+0000", 2020-08-07T15:08:49.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015035600000000001 2020-08-07T15:08:49.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:49.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490593+0000", 2020-08-07T15:08:49.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247 2020-08-07T15:08:49.924 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:49.924 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:49.924 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:49.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.925 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30923 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:08:49.925 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:08:49.925 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.0108968200000001, 2020-08-07T15:08:49.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998, 2020-08-07T15:08:49.925 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:49.926 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:08:49.926 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:49.926 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:49.926 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:49.926 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30923 2020-08-07T15:08:49.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.927 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:49.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:49.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:08:49.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:49.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:49.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:08:49.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5531000000000001e-05 2020-08-07T15:08:49.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:49.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706630+0000", 2020-08-07T15:08:49.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7689999999999999e-06 2020-08-07T15:08:49.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:49.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706637+0000", 2020-08-07T15:08:49.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3577999999999999e-05 2020-08-07T15:08:49.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:49.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706650+0000", 2020-08-07T15:08:49.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1439999999999997e-06 2020-08-07T15:08:49.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:49.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706658+0000", 2020-08-07T15:08:49.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1663999999999999e-05 2020-08-07T15:08:49.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:49.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706709+0000", 2020-08-07T15:08:49.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0149000000000001e-05 2020-08-07T15:08:49.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:49.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706719+0000", 2020-08-07T15:08:49.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329694900000001 2020-08-07T15:08:49.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:49.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480016+0000", 2020-08-07T15:08:49.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7998999999999997e-05 2020-08-07T15:08:49.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:08:49.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480084+0000", 2020-08-07T15:08:49.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010547333000000001 2020-08-07T15:08:49.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:49.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490632+0000", 2020-08-07T15:08:49.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7439e-05 2020-08-07T15:08:49.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:49.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490729+0000", 2020-08-07T15:08:49.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998 2020-08-07T15:08:49.980 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:49.980 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:49.980 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:49.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.981 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30924 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:08:49.981 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:08:49.982 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.0107695479999999, 2020-08-07T15:08:49.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995, 2020-08-07T15:08:49.982 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:49.982 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:08:49.983 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:49.983 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:49.983 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:49.983 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30924 2020-08-07T15:08:49.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.984 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:49.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:49.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:08:49.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:49.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:49.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:08:49.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9698999999999999e-05 2020-08-07T15:08:49.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:49.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706762+0000", 2020-08-07T15:08:49.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8080000000000001e-06 2020-08-07T15:08:49.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:49.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706767+0000", 2020-08-07T15:08:49.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1069e-05 2020-08-07T15:08:49.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:49.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706778+0000", 2020-08-07T15:08:49.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6969999999999998e-06 2020-08-07T15:08:49.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:49.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706784+0000", 2020-08-07T15:08:49.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8792999999999998e-05 2020-08-07T15:08:49.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:49.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706843+0000", 2020-08-07T15:08:49.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5449999999999998e-06 2020-08-07T15:08:49.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:49.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706850+0000", 2020-08-07T15:08:49.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.773278354 2020-08-07T15:08:49.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:49.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480129+0000", 2020-08-07T15:08:49.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1577999999999997e-05 2020-08-07T15:08:49.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:08:49.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480210+0000", 2020-08-07T15:08:49.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010541915000000001 2020-08-07T15:08:49.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:49.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:49.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:49.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490752+0000", 2020-08-07T15:08:49.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103937 2020-08-07T15:08:50.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:50.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490856+0000", 2020-08-07T15:08:50.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995 2020-08-07T15:08:50.001 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.001 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:50.001 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.002 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30925 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:08:50.002 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:08:50.002 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.0106457770000001, 2020-08-07T15:08:50.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995, 2020-08-07T15:08:50.003 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:50.003 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:08:50.003 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:50.003 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:50.003 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:50.004 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30925 2020-08-07T15:08:50.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.004 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:50.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:50.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:08:50.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:50.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:50.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:08:50.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4664999999999999e-05 2020-08-07T15:08:50.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:50.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706880+0000", 2020-08-07T15:08:50.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7479999999999999e-06 2020-08-07T15:08:50.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:50.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706886+0000", 2020-08-07T15:08:50.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1838000000000001e-05 2020-08-07T15:08:50.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:50.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706898+0000", 2020-08-07T15:08:50.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0089999999999998e-06 2020-08-07T15:08:50.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:50.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706903+0000", 2020-08-07T15:08:50.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3559e-05 2020-08-07T15:08:50.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706956+0000", 2020-08-07T15:08:50.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0309000000000001e-05 2020-08-07T15:08:50.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:50.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706967+0000", 2020-08-07T15:08:50.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77328290499999996 2020-08-07T15:08:50.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480250+0000", 2020-08-07T15:08:50.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8680999999999997e-05 2020-08-07T15:08:50.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:08:50.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480298+0000", 2020-08-07T15:08:50.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01059272 2020-08-07T15:08:50.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490891+0000", 2020-08-07T15:08:50.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014580300000000001 2020-08-07T15:08:50.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:50.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491037+0000", 2020-08-07T15:08:50.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995 2020-08-07T15:08:50.041 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.041 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:50.042 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.042 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30926 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:08:50.042 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:08:50.043 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.0105289079999999, 2020-08-07T15:08:50.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005, 2020-08-07T15:08:50.043 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:50.043 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:08:50.043 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:50.043 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:50.044 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:50.044 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30926 2020-08-07T15:08:50.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.044 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:50.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:50.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:08:50.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:50.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:50.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:08:50.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5223000000000003e-05 2020-08-07T15:08:50.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:50.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706998+0000", 2020-08-07T15:08:50.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4809999999999997e-06 2020-08-07T15:08:50.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:50.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707003+0000", 2020-08-07T15:08:50.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1663e-05 2020-08-07T15:08:50.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:50.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707015+0000", 2020-08-07T15:08:50.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9729999999999997e-06 2020-08-07T15:08:50.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:50.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707020+0000", 2020-08-07T15:08:50.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2565999999999999e-05 2020-08-07T15:08:50.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707062+0000", 2020-08-07T15:08:50.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8944e-05 2020-08-07T15:08:50.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:50.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707081+0000", 2020-08-07T15:08:50.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77325885000000005 2020-08-07T15:08:50.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480340+0000", 2020-08-07T15:08:50.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0321000000000006e-05 2020-08-07T15:08:50.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:08:50.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480420+0000", 2020-08-07T15:08:50.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010655958 2020-08-07T15:08:50.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491076+0000", 2020-08-07T15:08:50.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9784999999999995e-05 2020-08-07T15:08:50.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:50.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491166+0000", 2020-08-07T15:08:50.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005 2020-08-07T15:08:50.072 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.072 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:50.073 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.073 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30927 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:08:50.073 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:08:50.074 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.0104110890000002, 2020-08-07T15:08:50.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999, 2020-08-07T15:08:50.074 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:50.074 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:08:50.074 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:50.075 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:50.075 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:50.075 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30927 2020-08-07T15:08:50.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.075 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:50.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:50.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:08:50.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:50.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:50.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:08:50.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9861999999999997e-05 2020-08-07T15:08:50.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:50.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707120+0000", 2020-08-07T15:08:50.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8969999999999996e-06 2020-08-07T15:08:50.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:50.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707126+0000", 2020-08-07T15:08:50.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1248e-05 2020-08-07T15:08:50.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:50.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707137+0000", 2020-08-07T15:08:50.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.955e-06 2020-08-07T15:08:50.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:50.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707142+0000", 2020-08-07T15:08:50.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0643999999999998e-05 2020-08-07T15:08:50.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707193+0000", 2020-08-07T15:08:50.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1983e-05 2020-08-07T15:08:50.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:50.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707205+0000", 2020-08-07T15:08:50.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77324914 2020-08-07T15:08:50.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480454+0000", 2020-08-07T15:08:50.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4119999999999998e-05 2020-08-07T15:08:50.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:08:50.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480498+0000", 2020-08-07T15:08:50.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010709707000000001 2020-08-07T15:08:50.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491208+0000", 2020-08-07T15:08:50.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014433800000000001 2020-08-07T15:08:50.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:50.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491352+0000", 2020-08-07T15:08:50.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999 2020-08-07T15:08:50.120 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.121 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:50.121 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.121 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30928 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:08:50.122 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:08:50.122 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.010286732, 2020-08-07T15:08:50.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998, 2020-08-07T15:08:50.122 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:50.122 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:08:50.122 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:50.123 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:50.123 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:50.123 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30928 2020-08-07T15:08:50.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.123 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:50.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:50.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:08:50.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:50.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:50.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:08:50.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6582000000000001e-05 2020-08-07T15:08:50.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:50.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707241+0000", 2020-08-07T15:08:50.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9190000000000004e-06 2020-08-07T15:08:50.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:50.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707247+0000", 2020-08-07T15:08:50.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000188822 2020-08-07T15:08:50.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:50.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707436+0000", 2020-08-07T15:08:50.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2940000000000002e-06 2020-08-07T15:08:50.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:50.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707442+0000", 2020-08-07T15:08:50.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7800000000000003e-05 2020-08-07T15:08:50.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707490+0000", 2020-08-07T15:08:50.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2020-08-07T15:08:50.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:50.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707501+0000", 2020-08-07T15:08:50.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77303498800000003 2020-08-07T15:08:50.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480536+0000", 2020-08-07T15:08:50.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7723000000000001e-05 2020-08-07T15:08:50.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:08:50.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480614+0000", 2020-08-07T15:08:50.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010812183 2020-08-07T15:08:50.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491426+0000", 2020-08-07T15:08:50.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5502999999999999e-05 2020-08-07T15:08:50.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:50.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491502+0000", 2020-08-07T15:08:50.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998 2020-08-07T15:08:50.153 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.153 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:50.153 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.161 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31009 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:08:50.175 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:08:50.175 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 2.9863304259999999, 2020-08-07T15:08:50.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002, 2020-08-07T15:08:50.176 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:50.176 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:50.176 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:50.176 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:50.176 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:50.177 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31009 2020-08-07T15:08:50.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.177 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:50.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:50.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:08:50.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:50.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:50.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:08:50.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2429000000000002e-05 2020-08-07T15:08:50.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:50.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731193+0000", 2020-08-07T15:08:50.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9760000000000003e-06 2020-08-07T15:08:50.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:50.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731198+0000", 2020-08-07T15:08:50.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3597e-05 2020-08-07T15:08:50.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:50.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731212+0000", 2020-08-07T15:08:50.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1709999999999996e-06 2020-08-07T15:08:50.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:50.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731219+0000", 2020-08-07T15:08:50.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5402999999999997e-05 2020-08-07T15:08:50.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731285+0000", 2020-08-07T15:08:50.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3495e-05 2020-08-07T15:08:50.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:50.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731298+0000", 2020-08-07T15:08:50.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74935423599999995 2020-08-07T15:08:50.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480652+0000", 2020-08-07T15:08:50.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000137806 2020-08-07T15:08:50.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:50.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480790+0000", 2020-08-07T15:08:50.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022272399999999999 2020-08-07T15:08:50.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:08:50.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481013+0000", 2020-08-07T15:08:50.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003223381 2020-08-07T15:08:50.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:50.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484236+0000", 2020-08-07T15:08:50.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1370000000000001e-06 2020-08-07T15:08:50.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:50.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484237+0000", 2020-08-07T15:08:50.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6288e-05 2020-08-07T15:08:50.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:50.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484294+0000", 2020-08-07T15:08:50.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.007260402 2020-08-07T15:08:50.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:50.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491554+0000", 2020-08-07T15:08:50.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7358000000000003e-05 2020-08-07T15:08:50.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:50.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491601+0000", 2020-08-07T15:08:50.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1100000000000004e-07 2020-08-07T15:08:50.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:50.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491602+0000", 2020-08-07T15:08:50.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013436500000000001 2020-08-07T15:08:50.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:50.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491737+0000", 2020-08-07T15:08:50.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002 2020-08-07T15:08:50.204 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.205 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:50.205 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.205 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31010 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:08:50.206 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:08:50.206 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 2.9862044719999998, 2020-08-07T15:08:50.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001, 2020-08-07T15:08:50.222 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:50.223 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:50.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:50.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:50.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:50.223 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31010 2020-08-07T15:08:50.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.224 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:50.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:50.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:08:50.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:50.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:50.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:08:50.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2398000000000002e-05 2020-08-07T15:08:50.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:50.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731319+0000", 2020-08-07T15:08:50.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7100000000000004e-06 2020-08-07T15:08:50.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:50.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731325+0000", 2020-08-07T15:08:50.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4328000000000001e-05 2020-08-07T15:08:50.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:50.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731339+0000", 2020-08-07T15:08:50.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3310000000000003e-06 2020-08-07T15:08:50.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:50.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731345+0000", 2020-08-07T15:08:50.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8952999999999998e-05 2020-08-07T15:08:50.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731384+0000", 2020-08-07T15:08:50.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0036e-05 2020-08-07T15:08:50.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:50.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731394+0000", 2020-08-07T15:08:50.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75027681300000004 2020-08-07T15:08:50.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481670+0000", 2020-08-07T15:08:50.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016555899999999999 2020-08-07T15:08:50.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:50.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481836+0000", 2020-08-07T15:08:50.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028760999999999999 2020-08-07T15:08:50.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:08:50.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482124+0000", 2020-08-07T15:08:50.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038342680000000001 2020-08-07T15:08:50.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:50.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485958+0000", 2020-08-07T15:08:50.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2559999999999999e-06 2020-08-07T15:08:50.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:50.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485959+0000", 2020-08-07T15:08:50.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3758999999999995e-05 2020-08-07T15:08:50.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:50.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486043+0000", 2020-08-07T15:08:50.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057590419999999998 2020-08-07T15:08:50.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:50.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491802+0000", 2020-08-07T15:08:50.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5757000000000001e-05 2020-08-07T15:08:50.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:50.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:08:50.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9299999999999997e-07 2020-08-07T15:08:50.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:50.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:08:50.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012496999999999999 2020-08-07T15:08:50.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:50.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491993+0000", 2020-08-07T15:08:50.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001 2020-08-07T15:08:50.264 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.264 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:50.264 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.265 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31011 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:08:50.265 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:08:50.265 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 2.986088267, 2020-08-07T15:08:50.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997, 2020-08-07T15:08:50.265 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:50.277 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:50.277 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:50.277 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:50.277 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:50.278 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31011 2020-08-07T15:08:50.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.278 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:50.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:50.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:08:50.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:50.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:50.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:08:50.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9539999999999997e-05 2020-08-07T15:08:50.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:50.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731433+0000", 2020-08-07T15:08:50.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9209999999999997e-06 2020-08-07T15:08:50.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:50.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731439+0000", 2020-08-07T15:08:50.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4246999999999999e-05 2020-08-07T15:08:50.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:50.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731453+0000", 2020-08-07T15:08:50.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8130000000000003e-06 2020-08-07T15:08:50.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:50.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731458+0000", 2020-08-07T15:08:50.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6668000000000001e-05 2020-08-07T15:08:50.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731504+0000", 2020-08-07T15:08:50.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1112e-05 2020-08-07T15:08:50.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:50.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731515+0000", 2020-08-07T15:08:50.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75121332100000004 2020-08-07T15:08:50.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482729+0000", 2020-08-07T15:08:50.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1810000000000004e-05 2020-08-07T15:08:50.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:50.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482811+0000", 2020-08-07T15:08:50.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018001399999999999 2020-08-07T15:08:50.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:08:50.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482991+0000", 2020-08-07T15:08:50.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030701140000000001 2020-08-07T15:08:50.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:50.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486061+0000", 2020-08-07T15:08:50.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4499999999999996e-07 2020-08-07T15:08:50.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:50.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486062+0000", 2020-08-07T15:08:50.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5366999999999998e-05 2020-08-07T15:08:50.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:50.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486147+0000", 2020-08-07T15:08:50.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056872779999999996 2020-08-07T15:08:50.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:50.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491834+0000", 2020-08-07T15:08:50.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024645100000000002 2020-08-07T15:08:50.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:50.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:08:50.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1200000000000003e-07 2020-08-07T15:08:50.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:50.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:08:50.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1451000000000002e-05 2020-08-07T15:08:50.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:50.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492173+0000", 2020-08-07T15:08:50.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997 2020-08-07T15:08:50.356 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.356 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:50.356 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.357 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31012 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:08:50.357 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:08:50.358 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 2.985434148, 2020-08-07T15:08:50.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003, 2020-08-07T15:08:50.358 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:50.358 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:50.359 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:50.359 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:50.359 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:50.359 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31012 2020-08-07T15:08:50.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.360 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:50.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:50.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:08:50.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:50.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:50.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:08:50.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6344999999999997e-05 2020-08-07T15:08:50.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:50.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732094+0000", 2020-08-07T15:08:50.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2920000000000001e-06 2020-08-07T15:08:50.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:50.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732100+0000", 2020-08-07T15:08:50.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.189e-05 2020-08-07T15:08:50.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:50.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732112+0000", 2020-08-07T15:08:50.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3720000000000001e-06 2020-08-07T15:08:50.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:50.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732118+0000", 2020-08-07T15:08:50.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2514000000000004e-05 2020-08-07T15:08:50.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732191+0000", 2020-08-07T15:08:50.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4610000000000001e-05 2020-08-07T15:08:50.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:50.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732205+0000", 2020-08-07T15:08:50.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75132974100000005 2020-08-07T15:08:50.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483535+0000", 2020-08-07T15:08:50.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9185999999999996e-05 2020-08-07T15:08:50.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:50.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483624+0000", 2020-08-07T15:08:50.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015683 2020-08-07T15:08:50.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:08:50.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483781+0000", 2020-08-07T15:08:50.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046202889999999996 2020-08-07T15:08:50.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:50.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488401+0000", 2020-08-07T15:08:50.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6700000000000002e-07 2020-08-07T15:08:50.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:50.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488402+0000", 2020-08-07T15:08:50.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1487000000000001e-05 2020-08-07T15:08:50.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:50.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488484+0000", 2020-08-07T15:08:50.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035520830000000002 2020-08-07T15:08:50.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:50.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492036+0000", 2020-08-07T15:08:50.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029266600000000002 2020-08-07T15:08:50.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:50.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492328+0000", 2020-08-07T15:08:50.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5099999999999998e-07 2020-08-07T15:08:50.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:50.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492329+0000", 2020-08-07T15:08:50.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000201299 2020-08-07T15:08:50.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:50.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492531+0000", 2020-08-07T15:08:50.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003 2020-08-07T15:08:50.402 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.403 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:50.403 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.404 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31013 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:08:50.404 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:08:50.404 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 2.985222517, 2020-08-07T15:08:50.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996, 2020-08-07T15:08:50.405 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:50.405 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:50.405 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:50.405 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:50.406 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:50.406 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31013 2020-08-07T15:08:50.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.406 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:50.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:50.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:08:50.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:50.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:50.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:08:50.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8949000000000002e-05 2020-08-07T15:08:50.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:50.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732298+0000", 2020-08-07T15:08:50.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1399999999999997e-06 2020-08-07T15:08:50.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:50.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732304+0000", 2020-08-07T15:08:50.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.154e-05 2020-08-07T15:08:50.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:50.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732315+0000", 2020-08-07T15:08:50.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1319999999999999e-06 2020-08-07T15:08:50.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:50.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732322+0000", 2020-08-07T15:08:50.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7431e-05 2020-08-07T15:08:50.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732369+0000", 2020-08-07T15:08:50.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0621999999999999e-05 2020-08-07T15:08:50.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:50.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732380+0000", 2020-08-07T15:08:50.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75196845999999995 2020-08-07T15:08:50.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484348+0000", 2020-08-07T15:08:50.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000123008 2020-08-07T15:08:50.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:50.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484471+0000", 2020-08-07T15:08:50.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017572700000000001 2020-08-07T15:08:50.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:08:50.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484647+0000", 2020-08-07T15:08:50.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038557650000000001 2020-08-07T15:08:50.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:50.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:08:50.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2020-08-07T15:08:50.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:50.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:08:50.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0714e-05 2020-08-07T15:08:50.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:50.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488584+0000", 2020-08-07T15:08:50.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036298319999999999 2020-08-07T15:08:50.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:50.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492214+0000", 2020-08-07T15:08:50.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039530099999999997 2020-08-07T15:08:50.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:50.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492609+0000", 2020-08-07T15:08:50.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5300000000000002e-07 2020-08-07T15:08:50.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:50.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492610+0000", 2020-08-07T15:08:50.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197321 2020-08-07T15:08:50.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:50.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492807+0000", 2020-08-07T15:08:50.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996 2020-08-07T15:08:50.425 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.425 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:50.426 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.426 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31014 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:08:50.426 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:08:50.426 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 2.983840373, 2020-08-07T15:08:50.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998, 2020-08-07T15:08:50.427 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:50.427 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:50.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:50.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:50.428 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:50.428 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31014 2020-08-07T15:08:50.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.428 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:50.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:50.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:08:50.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:50.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:50.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:08:50.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3743999999999999e-05 2020-08-07T15:08:50.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:50.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733685+0000", 2020-08-07T15:08:50.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1720000000000007e-06 2020-08-07T15:08:50.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:50.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733693+0000", 2020-08-07T15:08:50.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3674e-05 2020-08-07T15:08:50.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:50.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733707+0000", 2020-08-07T15:08:50.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9249999999999998e-06 2020-08-07T15:08:50.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:50.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733714+0000", 2020-08-07T15:08:50.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8338000000000003e-05 2020-08-07T15:08:50.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733772+0000", 2020-08-07T15:08:50.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3019e-05 2020-08-07T15:08:50.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:50.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733785+0000", 2020-08-07T15:08:50.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75134856299999997 2020-08-07T15:08:50.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485133+0000", 2020-08-07T15:08:50.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8739999999999995e-05 2020-08-07T15:08:50.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:50.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485212+0000", 2020-08-07T15:08:50.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151244 2020-08-07T15:08:50.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:08:50.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485363+0000", 2020-08-07T15:08:50.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032929090000000001 2020-08-07T15:08:50.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:50.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488656+0000", 2020-08-07T15:08:50.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.065e-06 2020-08-07T15:08:50.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:50.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488657+0000", 2020-08-07T15:08:50.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1123000000000003e-05 2020-08-07T15:08:50.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:50.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488729+0000", 2020-08-07T15:08:50.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035508330000000002 2020-08-07T15:08:50.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:50.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492279+0000", 2020-08-07T15:08:50.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00059376000000000003 2020-08-07T15:08:50.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:50.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492873+0000", 2020-08-07T15:08:50.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5700000000000002e-07 2020-08-07T15:08:50.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:50.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492874+0000", 2020-08-07T15:08:50.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019264 2020-08-07T15:08:50.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:50.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493066+0000", 2020-08-07T15:08:50.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998 2020-08-07T15:08:50.456 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.456 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:50.456 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.457 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31015 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:08:50.457 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:08:50.457 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 2.9837144069999999, 2020-08-07T15:08:50.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002, 2020-08-07T15:08:50.458 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:50.458 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:50.458 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:50.458 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:50.459 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:50.459 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31015 2020-08-07T15:08:50.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.459 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:50.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:50.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:08:50.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:50.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:50.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:08:50.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3707000000000001e-05 2020-08-07T15:08:50.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:50.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733791+0000", 2020-08-07T15:08:50.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8120000000000001e-06 2020-08-07T15:08:50.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:50.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733794+0000", 2020-08-07T15:08:50.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0728e-05 2020-08-07T15:08:50.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:50.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733805+0000", 2020-08-07T15:08:50.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8279999999999999e-06 2020-08-07T15:08:50.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:50.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733811+0000", 2020-08-07T15:08:50.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2955999999999999e-05 2020-08-07T15:08:50.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733864+0000", 2020-08-07T15:08:50.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1389000000000001e-05 2020-08-07T15:08:50.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:50.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733875+0000", 2020-08-07T15:08:50.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75232711100000005 2020-08-07T15:08:50.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486202+0000", 2020-08-07T15:08:50.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027503400000000001 2020-08-07T15:08:50.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:50.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486478+0000", 2020-08-07T15:08:50.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033010400000000001 2020-08-07T15:08:50.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:08:50.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486808+0000", 2020-08-07T15:08:50.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019382609999999999 2020-08-07T15:08:50.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:50.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488746+0000", 2020-08-07T15:08:50.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9400000000000004e-07 2020-08-07T15:08:50.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:50.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488747+0000", 2020-08-07T15:08:50.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4324000000000002e-05 2020-08-07T15:08:50.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:50.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488821+0000", 2020-08-07T15:08:50.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043745640000000001 2020-08-07T15:08:50.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:50.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493196+0000", 2020-08-07T15:08:50.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030381000000000001 2020-08-07T15:08:50.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:50.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493499+0000", 2020-08-07T15:08:50.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4e-07 2020-08-07T15:08:50.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:50.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493500+0000", 2020-08-07T15:08:50.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017499 2020-08-07T15:08:50.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:50.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493675+0000", 2020-08-07T15:08:50.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002 2020-08-07T15:08:50.505 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.505 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:50.506 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.506 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31016 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:08:50.506 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:08:50.507 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 2.9831184319999999, 2020-08-07T15:08:50.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971, 2020-08-07T15:08:50.507 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:50.507 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:50.507 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:08:50.508 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:08:50.508 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:50.508 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31016 2020-08-07T15:08:50.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.509 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:50.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:50.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:08:50.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:50.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:50.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:08:50.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0924000000000002e-05 2020-08-07T15:08:50.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:50.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734404+0000", 2020-08-07T15:08:50.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2020-08-07T15:08:50.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:50.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734410+0000", 2020-08-07T15:08:50.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3459999999999994e-06 2020-08-07T15:08:50.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:50.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734418+0000", 2020-08-07T15:08:50.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9640000000000002e-06 2020-08-07T15:08:50.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:50.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734423+0000", 2020-08-07T15:08:50.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7253999999999999e-05 2020-08-07T15:08:50.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734470+0000", 2020-08-07T15:08:50.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0249999999999997e-06 2020-08-07T15:08:50.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:08:50.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734477+0000", 2020-08-07T15:08:50.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75294592900000001 2020-08-07T15:08:50.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:50.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487423+0000", 2020-08-07T15:08:50.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031604300000000001 2020-08-07T15:08:50.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:50.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487739+0000", 2020-08-07T15:08:50.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000171149 2020-08-07T15:08:50.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:08:50.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487910+0000", 2020-08-07T15:08:50.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024207769999999998 2020-08-07T15:08:50.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:08:50.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490331+0000", 2020-08-07T15:08:50.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1650000000000001e-06 2020-08-07T15:08:50.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:50.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490332+0000", 2020-08-07T15:08:50.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9796999999999996e-05 2020-08-07T15:08:50.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:50.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490402+0000", 2020-08-07T15:08:50.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029928849999999998 2020-08-07T15:08:50.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:50.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493395+0000", 2020-08-07T15:08:50.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034903199999999999 2020-08-07T15:08:50.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:50.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493744+0000", 2020-08-07T15:08:50.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8899999999999999e-07 2020-08-07T15:08:50.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:08:50.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493745+0000", 2020-08-07T15:08:50.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197362 2020-08-07T15:08:50.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:50.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:50.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:50.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493942+0000", 2020-08-07T15:08:50.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971 2020-08-07T15:08:50.529 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.529 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:50.529 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.529 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:50.529 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:50.529 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:08:50.544 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:08:50.544 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:08:50.544 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:08:50.544 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:08:50.545 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:08:50.545 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:08:50.545 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:08:50.545 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:08:50.545 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:08:50.546 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:08:50.546 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:08:50.546 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:08:50.546 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:08:50.547 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:08:50.547 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 24 16 31991 31975 10.4077 1.125 0.0713718 0.0120047 2020-08-07T15:08:50.547 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:50.530+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:50.548 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:08:50.638 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:50.633+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:50.712 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:08:50.713 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 0 1 2020-08-07T15:08:50.743 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:50.733+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:50.763 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:50.764 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:50.781 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:08:50.865 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:50.863+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:50.936 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:50.935+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:50.997 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:50.998 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:51.012 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:08:51.054 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:51.052+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:51.171 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:51.166+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:51.221 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.0 to 1 (10000) 2020-08-07T15:08:51.222 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:51.225 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:51.242 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:08:51.243 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 1 1 2020-08-07T15:08:51.276 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 25 16 33653 33637 10.5107 12.9844 0.061655 0.0118656 2020-08-07T15:08:51.277 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:51.273+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:51.382 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:51.380+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:51.394 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:51.395 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:51.404 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:08:51.513 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:51.482+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:51.566 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:51.566 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:51.578 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:08:51.587 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:51.587+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:51.690 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:51.688+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:51.710 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:51.710 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:51.719 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:08:51.798 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:51.790+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:51.889 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:51.889 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:51.890 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:08:51.897 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:51.890+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:51.994 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:51.992+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:52.007 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:52.007 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:52.016 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:08:52.107 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:52.106+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:52.175 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.1 to 1 (10000) 2020-08-07T15:08:52.195 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 2 1 2020-08-07T15:08:52.199 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:52.199 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:52.201 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:08:52.235 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:52.224+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:52.275 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 26 16 33818 33802 10.156 1.28906 0.00219911 0.0123013 2020-08-07T15:08:52.299 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:52.297+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:52.409 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:52.399+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:52.421 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:52.423 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:52.430 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:08:52.502 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:52.500+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:52.547 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:08:52.548 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:08:52.605 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:52.602+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:52.615 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:52.616 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:52.627 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:08:52.719 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:52.702+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:52.794 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:08:52.808 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:52.805+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:52.838 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:08:52.854 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:52.854 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:08:52.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:08:52.855 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:08:52.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.855 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:10240 3.6 3:665d71ff:::benchmark_data_smithi074_35447_object1279:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:08:52.855 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:08:52.855 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 130.449002804, 2020-08-07T15:08:52.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25191569699999999, 2020-08-07T15:08:52.856 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:52.856 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:52.856 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:52.856 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:52.857 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:52.857 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 10240 2020-08-07T15:08:52.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.857 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:52.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.871 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:52.872 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:08:52.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:52.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.872 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.873 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:52.873 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:08:52.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5463000000000002e-05 2020-08-07T15:08:52.873 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.874 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.874 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:52.874 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393264+0000", 2020-08-07T15:08:52.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9219999999999999e-06 2020-08-07T15:08:52.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:52.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393268+0000", 2020-08-07T15:08:52.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6149999999999997e-06 2020-08-07T15:08:52.876 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.877 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:52.877 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393277+0000", 2020-08-07T15:08:52.877 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2039999999999996e-06 2020-08-07T15:08:52.877 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.883 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.883 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:52.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393282+0000", 2020-08-07T15:08:52.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001225243 2020-08-07T15:08:52.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:52.885 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394507+0000", 2020-08-07T15:08:52.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7108999999999995e-05 2020-08-07T15:08:52.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.886 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.886 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:52.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394605+0000", 2020-08-07T15:08:52.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017490800000000001 2020-08-07T15:08:52.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:52.887 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394779+0000", 2020-08-07T15:08:52.887 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032608699999999999 2020-08-07T15:08:52.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:52.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.398040+0000", 2020-08-07T15:08:52.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013015259999999999 2020-08-07T15:08:52.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:52.890 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.399342+0000", 2020-08-07T15:08:52.890 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24566893400000001 2020-08-07T15:08:52.898 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.898 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.906 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:52.906 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645011+0000", 2020-08-07T15:08:52.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.217e-06 2020-08-07T15:08:52.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:52.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645012+0000", 2020-08-07T15:08:52.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0803000000000006e-05 2020-08-07T15:08:52.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.908 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:52.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645083+0000", 2020-08-07T15:08:52.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1882999999999998e-05 2020-08-07T15:08:52.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.922 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:52.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645135+0000", 2020-08-07T15:08:52.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25191569699999999 2020-08-07T15:08:52.923 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:52.923 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:52.923 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:52.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.941 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:43335 3.6 e32/26 3:65a57ed3:::benchmark_data_smithi074_35447_object5416:head v 32'2831, mlcod=32'2822)", 2020-08-07T15:08:52.941 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:03.998850+0000", 2020-08-07T15:08:52.941 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 108.843371733, 2020-08-07T15:08:52.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.221384421, 2020-08-07T15:08:52.948 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:52.949 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:52.949 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:52.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.950 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:52.950 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998850+0000", 2020-08-07T15:08:52.950 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:52.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:52.951 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998850+0000", 2020-08-07T15:08:52.952 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4788000000000004e-05 2020-08-07T15:08:52.952 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.952 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.952 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:52.953 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998915+0000", 2020-08-07T15:08:52.953 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7909999999999999e-06 2020-08-07T15:08:52.953 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.953 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.953 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:52.954 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998922+0000", 2020-08-07T15:08:52.954 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22050499700000001 2020-08-07T15:08:52.954 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.954 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.955 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:52.955 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219427+0000", 2020-08-07T15:08:52.955 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0950000000000001e-06 2020-08-07T15:08:52.956 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.956 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.956 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:52.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219433+0000", 2020-08-07T15:08:52.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6342999999999997e-05 2020-08-07T15:08:52.957 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:52.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219509+0000", 2020-08-07T15:08:52.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4173999999999999e-05 2020-08-07T15:08:52.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:52.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219533+0000", 2020-08-07T15:08:52.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00064214899999999995 2020-08-07T15:08:52.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:52.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.220175+0000", 2020-08-07T15:08:52.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9083999999999999e-05 2020-08-07T15:08:52.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.962 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:52.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.220234+0000", 2020-08-07T15:08:52.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.221384421 2020-08-07T15:08:52.964 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:52.965 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:52.965 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:52.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.967 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:43336 3.6 e32/26 3:65a57ed3:::benchmark_data_smithi074_35447_object5416:head v 32'2832, mlcod=32'2822)", 2020-08-07T15:08:52.967 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:03.999509+0000", 2020-08-07T15:08:52.968 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 108.842712404, 2020-08-07T15:08:52.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22632864799999999, 2020-08-07T15:08:52.968 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:52.969 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:52.969 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:52.969 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.969 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:52.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999509+0000", 2020-08-07T15:08:52.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:52.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.970 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:52.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999509+0000", 2020-08-07T15:08:52.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0107e-05 2020-08-07T15:08:52.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:52.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999559+0000", 2020-08-07T15:08:52.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0889999999999996e-06 2020-08-07T15:08:52.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:52.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999566+0000", 2020-08-07T15:08:52.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22332094599999999 2020-08-07T15:08:52.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:52.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222887+0000", 2020-08-07T15:08:52.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5990000000000001e-06 2020-08-07T15:08:52.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.975 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:52.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222892+0000", 2020-08-07T15:08:52.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2342999999999998e-05 2020-08-07T15:08:52.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:52.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222924+0000", 2020-08-07T15:08:52.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9108999999999999e-05 2020-08-07T15:08:52.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.978 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:52.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222944+0000", 2020-08-07T15:08:52.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028409289999999999 2020-08-07T15:08:52.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:52.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.225784+0000", 2020-08-07T15:08:52.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3526000000000002e-05 2020-08-07T15:08:52.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.993 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.993 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:52.993 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.225838+0000", 2020-08-07T15:08:52.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22632864799999999 2020-08-07T15:08:52.994 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:52.994 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:52.994 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:52.995 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.995 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.995 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:43784 3.6 e32/26 3:6da53356:::benchmark_data_smithi074_35447_object5472:head v 32'2848, mlcod=32'2839)", 2020-08-07T15:08:52.995 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:04.253355+0000", 2020-08-07T15:08:52.996 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 108.58886683199999, 2020-08-07T15:08:52.996 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21456700500000001, 2020-08-07T15:08:52.996 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:52.996 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:52.997 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:52.997 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.997 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:52.997 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253355+0000", 2020-08-07T15:08:52.998 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:52.998 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.998 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.998 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:52.999 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253355+0000", 2020-08-07T15:08:52.999 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6168000000000002e-05 2020-08-07T15:08:52.999 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:52.999 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:52.999 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.000 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253401+0000", 2020-08-07T15:08:53.000 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3179999999999997e-06 2020-08-07T15:08:53.000 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.000 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.001 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.001 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.253406+0000", 2020-08-07T15:08:53.001 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21194338600000001 2020-08-07T15:08:53.001 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.002 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.002 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.002 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465350+0000", 2020-08-07T15:08:53.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3759999999999999e-06 2020-08-07T15:08:53.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465355+0000", 2020-08-07T15:08:53.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8250999999999997e-05 2020-08-07T15:08:53.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465393+0000", 2020-08-07T15:08:53.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0296999999999999e-05 2020-08-07T15:08:53.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.465414+0000", 2020-08-07T15:08:53.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024515209999999999 2020-08-07T15:08:53.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:53.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.467865+0000", 2020-08-07T15:08:53.008 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6687999999999997e-05 2020-08-07T15:08:53.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:53.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.467922+0000", 2020-08-07T15:08:53.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21456700500000001 2020-08-07T15:08:53.009 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.009 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.009 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.010 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:54200 3.3 e37/20 3:cc40637e:::benchmark_data_smithi074_35447_object6774:head v 37'3248, mlcod=37'3236)", 2020-08-07T15:08:53.010 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:08:53.011 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 99.743659584, 2020-08-07T15:08:53.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24065872499999999, 2020-08-07T15:08:53.011 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.012 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:53.012 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.012 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:08:53.013 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.013 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.014 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:08:53.014 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1671000000000002e-05 2020-08-07T15:08:53.014 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.015 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.015 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.015 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098594+0000", 2020-08-07T15:08:53.015 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0890000000000002e-06 2020-08-07T15:08:53.016 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.016 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.016 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098598+0000", 2020-08-07T15:08:53.017 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23875592300000001 2020-08-07T15:08:53.017 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.018 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337354+0000", 2020-08-07T15:08:53.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0420000000000002e-06 2020-08-07T15:08:53.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337359+0000", 2020-08-07T15:08:53.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0586000000000001e-05 2020-08-07T15:08:53.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337399+0000", 2020-08-07T15:08:53.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9415999999999998e-05 2020-08-07T15:08:53.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.030 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337419+0000", 2020-08-07T15:08:53.030 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017568390000000001 2020-08-07T15:08:53.030 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:53.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.339176+0000", 2020-08-07T15:08:53.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5158999999999999e-05 2020-08-07T15:08:53.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:53.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.339221+0000", 2020-08-07T15:08:53.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24065872499999999 2020-08-07T15:08:53.033 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.033 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.034 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.034 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.034 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:71136 3.6 e43/26 3:64cb7acb:::benchmark_data_smithi074_35447_object8891:head v 43'4744, mlcod=43'4739)", 2020-08-07T15:08:53.035 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:08:53.035 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 88.523608218000007, 2020-08-07T15:08:53.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26135482999999998, 2020-08-07T15:08:53.035 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.036 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:53.036 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:08:53.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.038 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:08:53.038 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3702999999999997e-05 2020-08-07T15:08:53.038 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.039 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.039 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318667+0000", 2020-08-07T15:08:53.062 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3119999999999999e-06 2020-08-07T15:08:53.062 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.063 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318674+0000", 2020-08-07T15:08:53.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.255173292 2020-08-07T15:08:53.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.064 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.573847+0000", 2020-08-07T15:08:53.065 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1140000000000002e-06 2020-08-07T15:08:53.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.065 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.071 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.573853+0000", 2020-08-07T15:08:53.071 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00057603799999999996 2020-08-07T15:08:53.071 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.072 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.072 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.072 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.574429+0000", 2020-08-07T15:08:53.072 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4776e-05 2020-08-07T15:08:53.073 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.073 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.073 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.074 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.574444+0000", 2020-08-07T15:08:53.074 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054577080000000004 2020-08-07T15:08:53.074 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.074 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.075 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:53.075 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.579901+0000", 2020-08-07T15:08:53.075 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6886999999999998e-05 2020-08-07T15:08:53.075 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.076 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.076 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:53.076 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.579968+0000", 2020-08-07T15:08:53.076 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26135482999999998 2020-08-07T15:08:53.077 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.077 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.077 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.077 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.078 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.078 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:81544 3.7 e48/20 3:e9bb655d:::benchmark_data_smithi074_35447_object10192:head v 48'4944, mlcod=48'4935)", 2020-08-07T15:08:53.078 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:30.902099+0000", 2020-08-07T15:08:53.078 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 81.940123259000003, 2020-08-07T15:08:53.079 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20669163500000001, 2020-08-07T15:08:53.079 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.094 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:53.095 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.096 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.096 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.096 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.902099+0000", 2020-08-07T15:08:53.097 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.097 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.097 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.097 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.102 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.902099+0000", 2020-08-07T15:08:53.103 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1317999999999999e-05 2020-08-07T15:08:53.103 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.103 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.103 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.104 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.902150+0000", 2020-08-07T15:08:53.104 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2079999999999997e-06 2020-08-07T15:08:53.104 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.104 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.105 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.105 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:30.902156+0000", 2020-08-07T15:08:53.105 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20264375100000001 2020-08-07T15:08:53.105 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.106 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.106 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.106 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.104800+0000", 2020-08-07T15:08:53.106 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2390000000000002e-06 2020-08-07T15:08:53.107 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.107 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.107 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.107 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.104805+0000", 2020-08-07T15:08:53.108 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3696000000000003e-05 2020-08-07T15:08:53.109 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.109 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.109 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.110 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.104849+0000", 2020-08-07T15:08:53.110 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4179e-05 2020-08-07T15:08:53.110 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.110 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.110 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.111 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.104873+0000", 2020-08-07T15:08:53.111 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038653839999999999 2020-08-07T15:08:53.111 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.111 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.112 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:53.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.108738+0000", 2020-08-07T15:08:53.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1860000000000002e-05 2020-08-07T15:08:53.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:53.113 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:31.108790+0000", 2020-08-07T15:08:53.113 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20669163500000001 2020-08-07T15:08:53.113 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.114 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.114 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.115 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:85696 3.f e51/20 3:f99fcea4:::benchmark_data_smithi074_35447_object10711:head v 51'5688, mlcod=51'5679)", 2020-08-07T15:08:53.115 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:08:53.115 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 79.189616805, 2020-08-07T15:08:53.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49655717300000002, 2020-08-07T15:08:53.115 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.116 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:53.116 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.116 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.116 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.117 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:08:53.117 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.117 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.117 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.118 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:08:53.118 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9543000000000002e-05 2020-08-07T15:08:53.118 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.118 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.119 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.119 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652665+0000", 2020-08-07T15:08:53.119 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8700000000000003e-06 2020-08-07T15:08:53.125 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.125 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.125 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.126 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652671+0000", 2020-08-07T15:08:53.126 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49579595799999998 2020-08-07T15:08:53.126 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.126 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.127 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.127 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148467+0000", 2020-08-07T15:08:53.127 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2509999999999997e-06 2020-08-07T15:08:53.127 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.127 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.128 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.128 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148475+0000", 2020-08-07T15:08:53.128 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5603999999999998e-05 2020-08-07T15:08:53.129 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.129 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.129 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.129 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148530+0000", 2020-08-07T15:08:53.130 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5486999999999998e-05 2020-08-07T15:08:53.130 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.130 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.130 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.131 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148556+0000", 2020-08-07T15:08:53.131 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00055666600000000004 2020-08-07T15:08:53.131 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.131 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.132 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:53.132 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.149112+0000", 2020-08-07T15:08:53.132 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9793999999999998e-05 2020-08-07T15:08:53.132 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.133 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.133 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:53.134 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.149162+0000", 2020-08-07T15:08:53.134 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49655717300000002 2020-08-07T15:08:53.134 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.135 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.135 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.135 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.135 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.136 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:86792 3.f e52/20 3:fc5386ee:::benchmark_data_smithi074_35447_object10848:head v 52'5752, mlcod=52'5743)", 2020-08-07T15:08:53.136 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:08:53.136 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 78.439771030000003, 2020-08-07T15:08:53.137 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.295270005, 2020-08-07T15:08:53.137 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.137 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:53.137 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.138 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.138 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.138 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:08:53.138 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.139 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.139 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.139 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.139 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:08:53.140 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9712000000000001e-05 2020-08-07T15:08:53.140 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.141 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.141 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.141 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402500+0000", 2020-08-07T15:08:53.141 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6789999999999999e-06 2020-08-07T15:08:53.142 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.142 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.142 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.142 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402507+0000", 2020-08-07T15:08:53.142 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29317074999999998 2020-08-07T15:08:53.143 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.143 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.143 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.143 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695678+0000", 2020-08-07T15:08:53.144 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8120000000000002e-06 2020-08-07T15:08:53.144 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.144 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.144 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.145 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695685+0000", 2020-08-07T15:08:53.145 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1270999999999998e-05 2020-08-07T15:08:53.145 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.145 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.146 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.146 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695726+0000", 2020-08-07T15:08:53.146 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2739e-05 2020-08-07T15:08:53.146 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.147 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.147 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.147 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695749+0000", 2020-08-07T15:08:53.147 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019333899999999999 2020-08-07T15:08:53.148 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.148 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.148 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:53.148 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.697682+0000", 2020-08-07T15:08:53.149 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8652000000000002e-05 2020-08-07T15:08:53.149 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.149 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.149 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:53.150 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.697721+0000", 2020-08-07T15:08:53.150 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.295270005 2020-08-07T15:08:53.150 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.150 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.150 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.151 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:87088 3.6 e52/26 3:6b0320df:::benchmark_data_smithi074_35447_object10885:head v 52'5912, mlcod=52'5902)", 2020-08-07T15:08:53.151 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:08:53.154 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 77.875537789999996, 2020-08-07T15:08:53.154 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29522189799999998, 2020-08-07T15:08:53.154 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.155 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:53.155 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.156 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.156 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:08:53.156 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.157 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.157 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.157 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.157 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:08:53.158 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0751000000000002e-05 2020-08-07T15:08:53.158 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.159 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.159 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.159 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966735+0000", 2020-08-07T15:08:53.159 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3080000000000004e-06 2020-08-07T15:08:53.160 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.160 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.160 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.160 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966741+0000", 2020-08-07T15:08:53.161 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29365034699999998 2020-08-07T15:08:53.161 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.161 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.161 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.162 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260391+0000", 2020-08-07T15:08:53.162 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2720000000000003e-06 2020-08-07T15:08:53.162 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.162 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.163 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.163 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260398+0000", 2020-08-07T15:08:53.163 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8819000000000005e-05 2020-08-07T15:08:53.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.163 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.164 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260476+0000", 2020-08-07T15:08:53.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4958999999999999e-05 2020-08-07T15:08:53.165 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.165 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.165 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260501+0000", 2020-08-07T15:08:53.166 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001353641 2020-08-07T15:08:53.166 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.167 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.167 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:53.167 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.261855+0000", 2020-08-07T15:08:53.167 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0800999999999997e-05 2020-08-07T15:08:53.167 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.168 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.168 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:53.168 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.261906+0000", 2020-08-07T15:08:53.168 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29522189799999998 2020-08-07T15:08:53.169 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.169 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.169 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.170 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.170 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.170 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:93920 3.6 e56/26 3:631393d3:::benchmark_data_smithi074_35447_object11739:head v 56'6352, mlcod=55'6342)", 2020-08-07T15:08:53.170 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:08:53.171 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 73.515422358999999, 2020-08-07T15:08:53.171 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39765404300000001, 2020-08-07T15:08:53.171 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.171 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:53.172 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.172 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.172 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.172 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:08:53.173 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.173 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.173 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.173 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.174 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:08:53.175 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3035999999999998e-05 2020-08-07T15:08:53.175 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.175 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.175 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.175 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326852+0000", 2020-08-07T15:08:53.176 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5379999999999999e-06 2020-08-07T15:08:53.176 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.176 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.176 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.177 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326859+0000", 2020-08-07T15:08:53.177 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.393917568 2020-08-07T15:08:53.177 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.177 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.177 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.178 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.720777+0000", 2020-08-07T15:08:53.178 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2430000000000005e-06 2020-08-07T15:08:53.178 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.178 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.178 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.179 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.720785+0000", 2020-08-07T15:08:53.179 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00065658700000000003 2020-08-07T15:08:53.179 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.179 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.180 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.180 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.721441+0000", 2020-08-07T15:08:53.180 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8585000000000002e-05 2020-08-07T15:08:53.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.180 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.181 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.181 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.721470+0000", 2020-08-07T15:08:53.181 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00293349 2020-08-07T15:08:53.181 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:53.182 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.724403+0000", 2020-08-07T15:08:53.182 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9996e-05 2020-08-07T15:08:53.183 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.183 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.183 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:53.183 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.724453+0000", 2020-08-07T15:08:53.184 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39765404300000001 2020-08-07T15:08:53.184 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.184 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.184 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.184 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.185 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.185 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:94384 3.3 e56/20 3:ce533213:::benchmark_data_smithi074_35447_object11797:head v 56'5488, mlcod=56'5479)", 2020-08-07T15:08:53.185 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.637245+0000", 2020-08-07T15:08:53.185 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 73.204976854999998, 2020-08-07T15:08:53.186 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.247824459, 2020-08-07T15:08:53.186 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.186 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:53.186 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.187 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.187 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.187 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637245+0000", 2020-08-07T15:08:53.187 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.187 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.188 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.188 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.188 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637245+0000", 2020-08-07T15:08:53.188 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0782999999999997e-05 2020-08-07T15:08:53.189 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.189 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.189 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.189 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637306+0000", 2020-08-07T15:08:53.190 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8699999999999992e-06 2020-08-07T15:08:53.190 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.190 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.190 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.190 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637314+0000", 2020-08-07T15:08:53.191 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24469571800000001 2020-08-07T15:08:53.191 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.191 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.192 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.192 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882009+0000", 2020-08-07T15:08:53.192 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8510000000000004e-06 2020-08-07T15:08:53.192 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.193 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.193 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.193 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882015+0000", 2020-08-07T15:08:53.193 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6708999999999997e-05 2020-08-07T15:08:53.193 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.194 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.194 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.194 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882052+0000", 2020-08-07T15:08:53.194 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0280000000000001e-05 2020-08-07T15:08:53.195 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.196 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.196 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.196 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882082+0000", 2020-08-07T15:08:53.197 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002934767 2020-08-07T15:08:53.197 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.197 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.197 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:53.198 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.885017+0000", 2020-08-07T15:08:53.198 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2481000000000001e-05 2020-08-07T15:08:53.198 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.198 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.199 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:53.199 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.885069+0000", 2020-08-07T15:08:53.199 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.247824459 2020-08-07T15:08:53.199 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.200 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.200 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.200 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.200 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.201 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:104560 3.f e58/20 3:fb4383ae:::benchmark_data_smithi074_35447_object13069:head v 58'6888, mlcod=58'6878)", 2020-08-07T15:08:53.201 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:46.301207+0000", 2020-08-07T15:08:53.201 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 66.541015037999998, 2020-08-07T15:08:53.201 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23687350400000001, 2020-08-07T15:08:53.202 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.202 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:53.202 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.202 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.203 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.203 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301207+0000", 2020-08-07T15:08:53.203 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.203 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.204 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.204 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.204 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301207+0000", 2020-08-07T15:08:53.204 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7945000000000003e-05 2020-08-07T15:08:53.204 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.205 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.205 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.205 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301255+0000", 2020-08-07T15:08:53.205 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2030000000000002e-06 2020-08-07T15:08:53.206 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.206 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.206 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.206 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301259+0000", 2020-08-07T15:08:53.207 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23565420500000001 2020-08-07T15:08:53.207 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.207 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.208 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.208 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.536913+0000", 2020-08-07T15:08:53.208 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2290000000000003e-06 2020-08-07T15:08:53.208 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.208 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.209 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.536921+0000", 2020-08-07T15:08:53.209 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0955e-05 2020-08-07T15:08:53.209 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.210 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.210 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.210 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.537002+0000", 2020-08-07T15:08:53.211 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8977999999999999e-05 2020-08-07T15:08:53.211 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.211 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.211 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.212 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.537031+0000", 2020-08-07T15:08:53.212 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00099831000000000008 2020-08-07T15:08:53.212 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.213 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.213 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:53.213 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.538030+0000", 2020-08-07T15:08:53.214 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0679e-05 2020-08-07T15:08:53.214 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.214 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.215 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:53.215 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.538080+0000", 2020-08-07T15:08:53.215 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23687350400000001 2020-08-07T15:08:53.215 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.216 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.216 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.216 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.216 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.216 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:116064 3.6 e64/26 3:6cc03ffa:::benchmark_data_smithi074_35447_object14507:head v 64'7632, mlcod=64'7623)", 2020-08-07T15:08:53.217 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:08:53.217 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 56.479460955999997, 2020-08-07T15:08:53.217 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25728607799999997, 2020-08-07T15:08:53.217 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.218 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:53.229 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.229 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.229 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.230 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:08:53.230 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.230 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.230 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.231 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.231 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:08:53.231 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6042000000000003e-05 2020-08-07T15:08:53.231 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.231 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.232 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.232 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362817+0000", 2020-08-07T15:08:53.232 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9399999999999999e-06 2020-08-07T15:08:53.232 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.233 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.233 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.233 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362823+0000", 2020-08-07T15:08:53.234 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.254635583 2020-08-07T15:08:53.234 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.234 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.235 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.235 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617458+0000", 2020-08-07T15:08:53.235 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5100000000000004e-06 2020-08-07T15:08:53.235 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.235 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.236 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.236 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617465+0000", 2020-08-07T15:08:53.236 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4439999999999999e-05 2020-08-07T15:08:53.236 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.237 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.237 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.237 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617539+0000", 2020-08-07T15:08:53.238 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3527e-05 2020-08-07T15:08:53.238 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.239 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.239 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.240 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617563+0000", 2020-08-07T15:08:53.240 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024301069999999999 2020-08-07T15:08:53.240 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.242 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.242 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:53.242 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.619993+0000", 2020-08-07T15:08:53.242 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3928999999999998e-05 2020-08-07T15:08:53.243 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.243 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.243 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:53.243 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.620047+0000", 2020-08-07T15:08:53.244 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25728607799999997 2020-08-07T15:08:53.244 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.244 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.244 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.244 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.245 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.245 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:127528 3.6 e70/26 3:6d26a4f4:::benchmark_data_smithi074_35447_object15940:head v 70'8264, mlcod=70'8256)", 2020-08-07T15:08:53.245 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:04.535579+0000", 2020-08-07T15:08:53.245 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 48.306642906999997, 2020-08-07T15:08:53.246 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22956602400000001, 2020-08-07T15:08:53.246 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.246 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:53.246 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.247 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.247 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.247 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.535579+0000", 2020-08-07T15:08:53.247 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.248 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.248 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.248 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.248 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.535579+0000", 2020-08-07T15:08:53.248 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0343999999999997e-05 2020-08-07T15:08:53.249 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.249 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.249 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.249 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.535639+0000", 2020-08-07T15:08:53.250 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5889999999999999e-06 2020-08-07T15:08:53.250 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.250 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.250 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.250 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.535646+0000", 2020-08-07T15:08:53.251 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22504058599999999 2020-08-07T15:08:53.251 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.251 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.251 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.251 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.760686+0000", 2020-08-07T15:08:53.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7790000000000003e-06 2020-08-07T15:08:53.252 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.252 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.252 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.760691+0000", 2020-08-07T15:08:53.253 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2592999999999997e-05 2020-08-07T15:08:53.253 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.254 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.255 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.760734+0000", 2020-08-07T15:08:53.265 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3947999999999998e-05 2020-08-07T15:08:53.266 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.266 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.266 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.267 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.760758+0000", 2020-08-07T15:08:53.267 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043316020000000004 2020-08-07T15:08:53.269 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.269 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.269 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:53.270 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.765089+0000", 2020-08-07T15:08:53.270 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5583e-05 2020-08-07T15:08:53.270 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.270 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.270 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:53.271 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.765145+0000", 2020-08-07T15:08:53.271 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22956602400000001 2020-08-07T15:08:53.271 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.271 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.272 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.273 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.273 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.273 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:145801 3.6 e75/26 3:69e4cb22:::benchmark_data_smithi074_35447_object16192:head v 75'9405, mlcod=75'9401)", 2020-08-07T15:08:53.273 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:08:53.273 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 38.170465663999998, 2020-08-07T15:08:53.274 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639, 2020-08-07T15:08:53.274 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.275 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:53.276 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.276 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.276 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.277 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:08:53.277 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.277 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.277 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.278 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.278 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:08:53.280 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6511000000000002e-05 2020-08-07T15:08:53.280 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.281 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.281 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.281 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671803+0000", 2020-08-07T15:08:53.281 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7190000000000004e-06 2020-08-07T15:08:53.281 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.282 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.282 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.282 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671809+0000", 2020-08-07T15:08:53.282 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74210928300000001 2020-08-07T15:08:53.283 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.283 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.283 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.284 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413919+0000", 2020-08-07T15:08:53.284 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1759999999999999e-05 2020-08-07T15:08:53.284 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.284 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.284 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.285 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413930+0000", 2020-08-07T15:08:53.285 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000108601 2020-08-07T15:08:53.286 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.286 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.287 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.287 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414039+0000", 2020-08-07T15:08:53.287 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9948999999999999e-05 2020-08-07T15:08:53.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.288 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.288 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414089+0000", 2020-08-07T15:08:53.288 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00079656499999999995 2020-08-07T15:08:53.288 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.288 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.289 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:53.290 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414885+0000", 2020-08-07T15:08:53.290 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2250999999999998e-05 2020-08-07T15:08:53.292 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.292 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.293 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:53.296 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414938+0000", 2020-08-07T15:08:53.297 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639 2020-08-07T15:08:53.297 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.297 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.298 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.298 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.298 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.298 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:9112 4.6 e86/81 4:6004be0b:::benchmark_data_smithi074_44586_object1138:head v 86'544, mlcod=86'533)", 2020-08-07T15:08:53.298 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:32.296890+0000", 2020-08-07T15:08:53.299 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 20.545332255000002, 2020-08-07T15:08:53.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26263388100000001, 2020-08-07T15:08:53.299 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.300 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:53.300 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.301 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.306 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.307 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296890+0000", 2020-08-07T15:08:53.307 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.307 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296890+0000", 2020-08-07T15:08:53.315 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9907999999999997e-05 2020-08-07T15:08:53.315 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.315 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.316 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.316 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296939+0000", 2020-08-07T15:08:53.316 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7630000000000003e-06 2020-08-07T15:08:53.316 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.316 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.317 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.317 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296946+0000", 2020-08-07T15:08:53.317 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26058471900000002 2020-08-07T15:08:53.318 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.318 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.319 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.319 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557531+0000", 2020-08-07T15:08:53.319 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5270000000000004e-06 2020-08-07T15:08:53.319 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.319 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.320 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.320 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557537+0000", 2020-08-07T15:08:53.320 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2843000000000003e-05 2020-08-07T15:08:53.321 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.326 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.327 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.327 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557580+0000", 2020-08-07T15:08:53.327 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3833999999999999e-05 2020-08-07T15:08:53.327 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.328 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.328 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.328 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557604+0000", 2020-08-07T15:08:53.328 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001875196 2020-08-07T15:08:53.328 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.329 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.329 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:53.340 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.559479+0000", 2020-08-07T15:08:53.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4091000000000002e-05 2020-08-07T15:08:53.341 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.341 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:53.341 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.559523+0000", 2020-08-07T15:08:53.341 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26263388100000001 2020-08-07T15:08:53.342 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.342 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.342 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.342 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.342 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.343 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:13784 4.6 e88/81 4:66f64b92:::benchmark_data_smithi074_44586_object1722:head v 88'832, mlcod=88'827)", 2020-08-07T15:08:53.343 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:35.137315+0000", 2020-08-07T15:08:53.343 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 17.704906522000002, 2020-08-07T15:08:53.343 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32459016899999998, 2020-08-07T15:08:53.343 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.344 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:53.344 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.344 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.344 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.344 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137315+0000", 2020-08-07T15:08:53.345 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.345 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.345 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.345 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.345 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137315+0000", 2020-08-07T15:08:53.345 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0362999999999998e-05 2020-08-07T15:08:53.346 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.346 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.346 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.346 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137376+0000", 2020-08-07T15:08:53.346 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7020000000000004e-06 2020-08-07T15:08:53.347 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.347 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.347 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137382+0000", 2020-08-07T15:08:53.347 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32172605500000001 2020-08-07T15:08:53.348 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.348 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.350 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.350 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459108+0000", 2020-08-07T15:08:53.350 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7710000000000003e-06 2020-08-07T15:08:53.350 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.350 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.351 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.351 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459114+0000", 2020-08-07T15:08:53.351 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4297e-05 2020-08-07T15:08:53.351 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.351 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.352 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.352 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459158+0000", 2020-08-07T15:08:53.352 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5058999999999998e-05 2020-08-07T15:08:53.352 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.352 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.353 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.353 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459183+0000", 2020-08-07T15:08:53.353 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026624740000000002 2020-08-07T15:08:53.353 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.353 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.354 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:53.354 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.461846+0000", 2020-08-07T15:08:53.354 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9447999999999997e-05 2020-08-07T15:08:53.354 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.354 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.355 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:53.355 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.461905+0000", 2020-08-07T15:08:53.355 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32459016899999998 2020-08-07T15:08:53.355 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.355 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.356 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.356 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.356 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.356 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:31608 4.7 e100/81 4:e0ebdc25:::benchmark_data_smithi074_44586_object3950:head v 100'2136, mlcod=100'2126)", 2020-08-07T15:08:53.356 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:47.845848+0000", 2020-08-07T15:08:53.356 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4.9963736619999999, 2020-08-07T15:08:53.357 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27603951399999999, 2020-08-07T15:08:53.357 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.357 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:53.357 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.357 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.358 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.358 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:47.845848+0000", 2020-08-07T15:08:53.358 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.358 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.358 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.360 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.360 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:47.845848+0000", 2020-08-07T15:08:53.360 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3656999999999998e-05 2020-08-07T15:08:53.360 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.360 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.361 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.361 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:47.845902+0000", 2020-08-07T15:08:53.361 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4050000000000003e-06 2020-08-07T15:08:53.361 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.362 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.362 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.362 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:47.845909+0000", 2020-08-07T15:08:53.362 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27431144000000002 2020-08-07T15:08:53.362 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.362 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.363 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.363 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.120221+0000", 2020-08-07T15:08:53.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3640000000000004e-06 2020-08-07T15:08:53.363 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.363 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.364 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.120227+0000", 2020-08-07T15:08:53.364 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1541e-05 2020-08-07T15:08:53.364 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.365 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.365 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.120279+0000", 2020-08-07T15:08:53.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.372e-05 2020-08-07T15:08:53.365 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.365 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.366 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.366 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.120302+0000", 2020-08-07T15:08:53.366 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015294460000000001 2020-08-07T15:08:53.366 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.367 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:53.367 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.121832+0000", 2020-08-07T15:08:53.367 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5940999999999999e-05 2020-08-07T15:08:53.367 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.367 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.368 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:53.368 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.121888+0000", 2020-08-07T15:08:53.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27603951399999999 2020-08-07T15:08:53.368 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.368 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.369 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.370 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:31832 4.b e100/81 4:d005fdc5:::benchmark_data_smithi074_44586_object3978:head v 100'1872, mlcod=100'1863)", 2020-08-07T15:08:53.370 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:48.177094+0000", 2020-08-07T15:08:53.371 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4.6651273980000001, 2020-08-07T15:08:53.371 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26337036000000003, 2020-08-07T15:08:53.371 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.371 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:53.371 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.372 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.372 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.177094+0000", 2020-08-07T15:08:53.372 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.372 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.373 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.373 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.373 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.177094+0000", 2020-08-07T15:08:53.373 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3760999999999997e-05 2020-08-07T15:08:53.373 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.374 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.374 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.374 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.177128+0000", 2020-08-07T15:08:53.375 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1549999999999998e-06 2020-08-07T15:08:53.375 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.375 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.375 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.177133+0000", 2020-08-07T15:08:53.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.262460639 2020-08-07T15:08:53.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.439594+0000", 2020-08-07T15:08:53.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0943e-05 2020-08-07T15:08:53.377 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.439605+0000", 2020-08-07T15:08:53.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013696100000000001 2020-08-07T15:08:53.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.439742+0000", 2020-08-07T15:08:53.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5649000000000003e-05 2020-08-07T15:08:53.379 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.439787+0000", 2020-08-07T15:08:53.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00063599200000000005 2020-08-07T15:08:53.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.382 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.382 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:53.382 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.440423+0000", 2020-08-07T15:08:53.382 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1260000000000001e-05 2020-08-07T15:08:53.383 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.383 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.383 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:53.383 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.440465+0000", 2020-08-07T15:08:53.383 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26337036000000003 2020-08-07T15:08:53.384 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.384 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.384 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.384 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.384 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.385 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:53.388 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:08:53.389 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:08:53.389 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:08:53.389 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:08:53.389 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:08:53.389 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:08:53.390 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:08:53.390 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:08:53.390 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:08:53.391 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:08:53.392 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:08:53.392 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:08:53.392 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:08:53.392 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:08:53.393 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:08:53.393 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.2 to 1 (10000) 2020-08-07T15:08:53.393 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 3 1 2020-08-07T15:08:53.394 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 27 16 35334 35318 10.2185 11.8438 0.00886225 0.0122252 2020-08-07T15:08:53.395 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:53.385+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:53.395 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:08:53.487 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:53.486+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:53.590 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:53.589+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:53.625 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:53.626 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:08:53.626 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.626 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:35920 4.7 4:ea96bce7:::benchmark_data_smithi074_44586_object4489:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e106)", 2020-08-07T15:08:53.626 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:53.603529+0000", 2020-08-07T15:08:53.627 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.014921454000000001, 2020-08-07T15:08:53.627 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.014954622000000001, 2020-08-07T15:08:53.628 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.628 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:53.628 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:53.628 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:08:53.628 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:53.629 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 35920 2020-08-07T15:08:53.630 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.630 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.630 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.630 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.631 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.603529+0000", 2020-08-07T15:08:53.631 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.631 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.631 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.631 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.632 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.603529+0000", 2020-08-07T15:08:53.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5762e-05 2020-08-07T15:08:53.632 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.634 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.634 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.634 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.603575+0000", 2020-08-07T15:08:53.635 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4450000000000004e-06 2020-08-07T15:08:53.635 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.635 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.635 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.635 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.603580+0000", 2020-08-07T15:08:53.636 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0777e-05 2020-08-07T15:08:53.636 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.636 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.636 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.636 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.603591+0000", 2020-08-07T15:08:53.637 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8230000000000002e-06 2020-08-07T15:08:53.637 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.637 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.637 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.637 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.603596+0000", 2020-08-07T15:08:53.638 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039729680000000003 2020-08-07T15:08:53.638 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.638 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.638 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.638 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.607569+0000", 2020-08-07T15:08:53.639 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000106471 2020-08-07T15:08:53.639 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.639 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.639 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.640 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.607675+0000", 2020-08-07T15:08:53.640 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013583599999999999 2020-08-07T15:08:53.641 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.641 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.641 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,5", 2020-08-07T15:08:53.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.607811+0000", 2020-08-07T15:08:53.642 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032938120000000001 2020-08-07T15:08:53.642 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.642 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.642 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:53.645 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.611105+0000", 2020-08-07T15:08:53.645 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8299999999999996e-07 2020-08-07T15:08:53.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:53.646 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.611105+0000", 2020-08-07T15:08:53.646 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4035e-05 2020-08-07T15:08:53.646 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.647 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.647 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:53.647 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.611160+0000", 2020-08-07T15:08:53.647 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017876859999999999 2020-08-07T15:08:53.647 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.648 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.648 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:53.648 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.612947+0000", 2020-08-07T15:08:53.648 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0094182980000000003 2020-08-07T15:08:53.648 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.649 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.649 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.649 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.649 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.649 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:35937 4.0 e106/81 4:0444f2a3:::benchmark_data_smithi074_44586_object4492:head v 106'2137, mlcod=106'2135)", 2020-08-07T15:08:53.650 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:53.614183+0000", 2020-08-07T15:08:53.650 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.004267753, 2020-08-07T15:08:53.651 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044437979999999997, 2020-08-07T15:08:53.651 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.652 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:53.652 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.652 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.652 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.652 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.614183+0000", 2020-08-07T15:08:53.653 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.653 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.653 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.653 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.653 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.614183+0000", 2020-08-07T15:08:53.654 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8872999999999998e-05 2020-08-07T15:08:53.654 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.654 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.654 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.654 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.614231+0000", 2020-08-07T15:08:53.655 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0639999999999999e-06 2020-08-07T15:08:53.655 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.655 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.655 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.655 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.614238+0000", 2020-08-07T15:08:53.656 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00028171899999999999 2020-08-07T15:08:53.656 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.656 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.656 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.656 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.614519+0000", 2020-08-07T15:08:53.657 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0409999999999997e-06 2020-08-07T15:08:53.657 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.657 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.657 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.657 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.614524+0000", 2020-08-07T15:08:53.658 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5531999999999997e-05 2020-08-07T15:08:53.658 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.659 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.659 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.659 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.614570+0000", 2020-08-07T15:08:53.659 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00042775700000000002 2020-08-07T15:08:53.659 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.660 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.614998+0000", 2020-08-07T15:08:53.670 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00081498599999999997 2020-08-07T15:08:53.670 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.670 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.670 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.673 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.673 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.673 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:35934 4.a e106/81 4:540a2a4d:::benchmark_data_smithi074_44586_object4491:head v 106'2534, mlcod=106'2527)", 2020-08-07T15:08:53.677 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:53.613292+0000", 2020-08-07T15:08:53.677 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0051584020000000003, 2020-08-07T15:08:53.677 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054174239999999997, 2020-08-07T15:08:53.677 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.677 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:53.678 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.678 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.678 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.678 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.613292+0000", 2020-08-07T15:08:53.678 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.680 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.680 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.613292+0000", 2020-08-07T15:08:53.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4648999999999998e-05 2020-08-07T15:08:53.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.681 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.682 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.613347+0000", 2020-08-07T15:08:53.682 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6340000000000003e-06 2020-08-07T15:08:53.684 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.684 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.694 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.694 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.613353+0000", 2020-08-07T15:08:53.694 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012026879999999999 2020-08-07T15:08:53.695 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.695 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.695 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.614556+0000", 2020-08-07T15:08:53.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3100000000000001e-06 2020-08-07T15:08:53.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.696 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.696 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.614559+0000", 2020-08-07T15:08:53.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9040000000000003e-05 2020-08-07T15:08:53.697 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.614628+0000", 2020-08-07T15:08:53.697 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0747e-05 2020-08-07T15:08:53.699 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.699 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.699 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.699 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.614649+0000", 2020-08-07T15:08:53.700 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013570679999999999 2020-08-07T15:08:53.700 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.700 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.700 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.700 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.701 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:35935 4.a e106/81 4:540a2a4d:::benchmark_data_smithi074_44586_object4491:head v 106'2535, mlcod=106'2527)", 2020-08-07T15:08:53.701 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:53.614362+0000", 2020-08-07T15:08:53.701 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0040882319999999998, 2020-08-07T15:08:53.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044059650000000004, 2020-08-07T15:08:53.702 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.702 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:53.702 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.703 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.703 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.614362+0000", 2020-08-07T15:08:53.703 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.703 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.704 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.704 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.614362+0000", 2020-08-07T15:08:53.704 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8102999999999997e-05 2020-08-07T15:08:53.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.705 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.705 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.614410+0000", 2020-08-07T15:08:53.705 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7089999999999999e-06 2020-08-07T15:08:53.705 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.705 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.706 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.706 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.614416+0000", 2020-08-07T15:08:53.706 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000154139 2020-08-07T15:08:53.706 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.706 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.707 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.707 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.614570+0000", 2020-08-07T15:08:53.707 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4769999999999999e-06 2020-08-07T15:08:53.707 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.708 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.708 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.614574+0000", 2020-08-07T15:08:53.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00032592500000000002 2020-08-07T15:08:53.709 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.709 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.709 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.709 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.614899+0000", 2020-08-07T15:08:53.710 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9669000000000001e-05 2020-08-07T15:08:53.710 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.710 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.710 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.710 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.614919+0000", 2020-08-07T15:08:53.711 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00055702200000000001 2020-08-07T15:08:53.711 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.711 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.712 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.712 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.712 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.712 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:35924 4.f 4:f5e07511:::benchmark_data_smithi074_44586_object4490:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e106)", 2020-08-07T15:08:53.712 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:53.605884+0000", 2020-08-07T15:08:53.713 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.012566580000000001, 2020-08-07T15:08:53.713 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.012938923, 2020-08-07T15:08:53.713 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.713 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:53.713 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:53.714 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:08:53.714 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:53.714 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 35924 2020-08-07T15:08:53.714 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.714 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.715 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.715 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.715 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.605884+0000", 2020-08-07T15:08:53.715 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.715 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.716 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.716 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.716 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.605884+0000", 2020-08-07T15:08:53.716 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6749999999999998e-05 2020-08-07T15:08:53.716 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.716 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.717 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.717 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.605931+0000", 2020-08-07T15:08:53.717 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.558e-06 2020-08-07T15:08:53.717 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.717 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.718 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.718 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.605936+0000", 2020-08-07T15:08:53.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0812000000000001e-05 2020-08-07T15:08:53.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.605947+0000", 2020-08-07T15:08:53.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0239999999999997e-06 2020-08-07T15:08:53.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.605952+0000", 2020-08-07T15:08:53.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006264113 2020-08-07T15:08:53.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.721 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.612216+0000", 2020-08-07T15:08:53.722 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000111799 2020-08-07T15:08:53.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.722 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.722 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.612328+0000", 2020-08-07T15:08:53.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000158269 2020-08-07T15:08:53.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.723 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4", 2020-08-07T15:08:53.724 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.612486+0000", 2020-08-07T15:08:53.724 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0031439839999999998 2020-08-07T15:08:53.724 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.724 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.724 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:53.725 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.615630+0000", 2020-08-07T15:08:53.725 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1200000000000001e-07 2020-08-07T15:08:53.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.725 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:53.726 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.615631+0000", 2020-08-07T15:08:53.726 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4636999999999999e-05 2020-08-07T15:08:53.726 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.726 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.726 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:53.727 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.615686+0000", 2020-08-07T15:08:53.727 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0098018580000000001 2020-08-07T15:08:53.727 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.727 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.727 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.728 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:35938 4.0 e106/81 4:0444f2a3:::benchmark_data_smithi074_44586_object4492:head v 106'2138, mlcod=106'2135)", 2020-08-07T15:08:53.728 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:53.616248+0000", 2020-08-07T15:08:53.728 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0022025529999999999, 2020-08-07T15:08:53.728 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002639204, 2020-08-07T15:08:53.729 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.729 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:53.730 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.730 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.730 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.731 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.616248+0000", 2020-08-07T15:08:53.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.731 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.732 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.616248+0000", 2020-08-07T15:08:53.732 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2729000000000003e-05 2020-08-07T15:08:53.732 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.732 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.733 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.616301+0000", 2020-08-07T15:08:53.733 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0270000000000001e-06 2020-08-07T15:08:53.733 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.743 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.743 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.616307+0000", 2020-08-07T15:08:53.743 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4848999999999999e-05 2020-08-07T15:08:53.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.744 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.616331+0000", 2020-08-07T15:08:53.744 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0780000000000003e-06 2020-08-07T15:08:53.745 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.752 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.753 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.616336+0000", 2020-08-07T15:08:53.753 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00056498500000000005 2020-08-07T15:08:53.754 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.754 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.754 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.754 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.616901+0000", 2020-08-07T15:08:53.754 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1435000000000001e-05 2020-08-07T15:08:53.755 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.755 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.616923+0000", 2020-08-07T15:08:53.755 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00067510300000000003 2020-08-07T15:08:53.756 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.756 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.756 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.757 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:35939 4.0 e106/81 4:0444f2a3:::benchmark_data_smithi074_44586_object4492:head v 106'2139, mlcod=106'2135)", 2020-08-07T15:08:53.757 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:53.616801+0000", 2020-08-07T15:08:53.757 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.001649668, 2020-08-07T15:08:53.757 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0021259709999999999, 2020-08-07T15:08:53.757 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.758 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:53.758 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.758 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.758 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.758 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.616801+0000", 2020-08-07T15:08:53.759 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.759 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.759 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.616801+0000", 2020-08-07T15:08:53.760 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0890999999999998e-05 2020-08-07T15:08:53.760 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.760 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.760 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.760 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.616852+0000", 2020-08-07T15:08:53.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2929999999999997e-06 2020-08-07T15:08:53.761 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.761 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.761 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.616858+0000", 2020-08-07T15:08:53.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4383e-05 2020-08-07T15:08:53.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.762 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.762 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.616882+0000", 2020-08-07T15:08:53.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2290000000000002e-06 2020-08-07T15:08:53.763 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.763 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.765 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.616887+0000", 2020-08-07T15:08:53.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010942930000000001 2020-08-07T15:08:53.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.765 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.766 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.617982+0000", 2020-08-07T15:08:53.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1398e-05 2020-08-07T15:08:53.766 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.766 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.767 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.618013+0000", 2020-08-07T15:08:53.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012124869999999999 2020-08-07T15:08:53.767 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.767 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.767 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.768 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.768 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.768 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:35910 4.4 e106/81 4:2fa6f506:::benchmark_data_smithi074_44586_object4488:head v 106'2318, mlcod=106'2307)", 2020-08-07T15:08:53.768 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:53.599693+0000", 2020-08-07T15:08:53.769 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.018757846000000002, 2020-08-07T15:08:53.769 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.019273765000000002, 2020-08-07T15:08:53.769 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.769 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:53.769 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.770 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.770 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.770 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.599693+0000", 2020-08-07T15:08:53.770 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.771 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.771 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.771 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.771 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.599693+0000", 2020-08-07T15:08:53.772 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1840000000000002e-05 2020-08-07T15:08:53.772 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.772 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.772 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.772 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.599774+0000", 2020-08-07T15:08:53.773 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0719999999999996e-06 2020-08-07T15:08:53.773 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.773 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.773 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.773 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.599780+0000", 2020-08-07T15:08:53.774 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.01742577 2020-08-07T15:08:53.775 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.775 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.775 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.617206+0000", 2020-08-07T15:08:53.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9490000000000004e-06 2020-08-07T15:08:53.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.617211+0000", 2020-08-07T15:08:53.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4489999999999997e-05 2020-08-07T15:08:53.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.617246+0000", 2020-08-07T15:08:53.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7677999999999999e-05 2020-08-07T15:08:53.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.780 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.617263+0000", 2020-08-07T15:08:53.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.017570799000000002 2020-08-07T15:08:53.780 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.780 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.781 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.781 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:35940 4.0 e106/81)", 2020-08-07T15:08:53.781 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:53.617426+0000", 2020-08-07T15:08:53.782 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.001023908, 2020-08-07T15:08:53.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015770739999999999, 2020-08-07T15:08:53.782 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.782 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:08:53.782 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.617426+0000", 2020-08-07T15:08:53.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.617426+0000", 2020-08-07T15:08:53.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7339000000000002e-05 2020-08-07T15:08:53.787 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.787 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.787 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.617474+0000", 2020-08-07T15:08:53.788 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9859999999999999e-06 2020-08-07T15:08:53.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.788 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.788 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.617480+0000", 2020-08-07T15:08:53.789 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4394000000000001e-05 2020-08-07T15:08:53.789 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.789 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.789 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.789 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.617504+0000", 2020-08-07T15:08:53.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6700000000000002e-06 2020-08-07T15:08:53.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.790 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.790 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.617509+0000", 2020-08-07T15:08:53.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2389e-05 2020-08-07T15:08:53.791 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.791 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.791 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.792 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.792 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:35925 4.f 4:f5e07511:::benchmark_data_smithi074_44586_object4490:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e106)", 2020-08-07T15:08:53.792 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:53.609089+0000", 2020-08-07T15:08:53.792 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0093617040000000002, 2020-08-07T15:08:53.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0099583729999999995, 2020-08-07T15:08:53.793 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.801 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:53.802 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:53.802 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:08:53.802 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:53.802 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 35925 2020-08-07T15:08:53.802 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.803 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.609089+0000", 2020-08-07T15:08:53.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.806 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.609089+0000", 2020-08-07T15:08:53.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6774000000000003e-05 2020-08-07T15:08:53.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.609135+0000", 2020-08-07T15:08:53.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9739999999999999e-06 2020-08-07T15:08:53.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.609141+0000", 2020-08-07T15:08:53.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3977e-05 2020-08-07T15:08:53.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.836 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.836 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.609155+0000", 2020-08-07T15:08:53.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7440000000000004e-06 2020-08-07T15:08:53.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.836 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.609161+0000", 2020-08-07T15:08:53.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039310329999999996 2020-08-07T15:08:53.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.861 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.613092+0000", 2020-08-07T15:08:53.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011042799999999999 2020-08-07T15:08:53.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.862 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.862 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.613203+0000", 2020-08-07T15:08:53.862 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000196345 2020-08-07T15:08:53.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4", 2020-08-07T15:08:53.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.613399+0000", 2020-08-07T15:08:53.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0049100139999999999 2020-08-07T15:08:53.864 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:53.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.618309+0000", 2020-08-07T15:08:53.865 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9500000000000001e-07 2020-08-07T15:08:53.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.866 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.866 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:53.866 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.618310+0000", 2020-08-07T15:08:53.866 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7874999999999997e-05 2020-08-07T15:08:53.866 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.867 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.867 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:53.868 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.618368+0000", 2020-08-07T15:08:53.868 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0092789589999999998 2020-08-07T15:08:53.868 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.868 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.868 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.869 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.869 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.869 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:35941 4.0 e106/81)", 2020-08-07T15:08:53.869 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:53.618041+0000", 2020-08-07T15:08:53.869 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.00040887900000000001, 2020-08-07T15:08:53.870 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010939459999999999, 2020-08-07T15:08:53.870 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.870 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:08:53.870 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.870 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.871 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.871 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.618041+0000", 2020-08-07T15:08:53.871 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.871 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.871 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.872 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.618041+0000", 2020-08-07T15:08:53.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1808000000000003e-05 2020-08-07T15:08:53.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.872 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.872 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.873 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.618093+0000", 2020-08-07T15:08:53.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0009999999999998e-06 2020-08-07T15:08:53.873 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.873 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.874 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.618099+0000", 2020-08-07T15:08:53.874 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5029999999999999e-05 2020-08-07T15:08:53.874 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.874 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.874 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.875 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.618124+0000", 2020-08-07T15:08:53.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8629999999999999e-06 2020-08-07T15:08:53.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.876 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.618129+0000", 2020-08-07T15:08:53.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7701999999999995e-05 2020-08-07T15:08:53.876 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.877 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.877 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.878 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.878 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.878 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:35926 4.f 4:f5e07511:::benchmark_data_smithi074_44586_object4490:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e106)", 2020-08-07T15:08:53.878 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:53.609195+0000", 2020-08-07T15:08:53.878 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0092551409999999997, 2020-08-07T15:08:53.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0099973279999999998, 2020-08-07T15:08:53.879 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.880 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:08:53.880 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:53.880 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:08:53.880 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:53.880 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 35926 2020-08-07T15:08:53.881 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.881 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.881 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.881 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.881 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.609195+0000", 2020-08-07T15:08:53.882 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.882 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.882 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.882 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.882 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.609195+0000", 2020-08-07T15:08:53.883 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2704000000000003e-05 2020-08-07T15:08:53.883 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.883 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.883 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.883 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.609238+0000", 2020-08-07T15:08:53.883 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0750000000000003e-06 2020-08-07T15:08:53.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.884 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.884 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.609244+0000", 2020-08-07T15:08:53.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1174e-05 2020-08-07T15:08:53.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.885 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.609255+0000", 2020-08-07T15:08:53.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6870000000000002e-06 2020-08-07T15:08:53.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.886 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.887 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.609260+0000", 2020-08-07T15:08:53.887 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004625596 2020-08-07T15:08:53.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.613885+0000", 2020-08-07T15:08:53.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0954999999999999e-05 2020-08-07T15:08:53.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.890 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.890 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.613976+0000", 2020-08-07T15:08:53.890 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017874199999999999 2020-08-07T15:08:53.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.891 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4", 2020-08-07T15:08:53.891 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.614155+0000", 2020-08-07T15:08:53.891 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0049971959999999998 2020-08-07T15:08:53.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.891 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.892 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:53.892 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.619152+0000", 2020-08-07T15:08:53.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0099571290000000003 2020-08-07T15:08:53.892 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.892 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.893 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.893 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:35942 4.0 e106/81)", 2020-08-07T15:08:53.893 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:53.618605+0000", 2020-08-07T15:08:53.894 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4294967295.9998455, 2020-08-07T15:08:53.894 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00064962600000000004, 2020-08-07T15:08:53.894 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.895 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:08:53.895 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.895 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.895 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.895 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.618605+0000", 2020-08-07T15:08:53.896 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.897 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.897 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.618605+0000", 2020-08-07T15:08:53.897 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2247999999999998e-05 2020-08-07T15:08:53.897 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.898 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.898 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.898 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.618657+0000", 2020-08-07T15:08:53.899 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7569999999999999e-06 2020-08-07T15:08:53.899 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.899 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.899 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.900 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.618667+0000", 2020-08-07T15:08:53.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0256e-05 2020-08-07T15:08:53.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.900 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.900 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.618697+0000", 2020-08-07T15:08:53.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8080000000000004e-06 2020-08-07T15:08:53.901 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.901 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.901 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.901 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.618702+0000", 2020-08-07T15:08:53.902 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7069000000000002e-05 2020-08-07T15:08:53.902 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.902 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.902 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.902 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.903 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:35927 4.f 4:f5e07511:::benchmark_data_smithi074_44586_object4490:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e106)", 2020-08-07T15:08:53.903 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:53.609293+0000", 2020-08-07T15:08:53.903 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0091577789999999996, 2020-08-07T15:08:53.903 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0099951220000000004, 2020-08-07T15:08:53.904 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.904 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:08:53.904 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:53.904 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:08:53.904 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:53.905 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 35927 2020-08-07T15:08:53.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.905 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.609293+0000", 2020-08-07T15:08:53.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.609293+0000", 2020-08-07T15:08:53.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5853999999999998e-05 2020-08-07T15:08:53.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.918 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.918 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.609338+0000", 2020-08-07T15:08:53.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4020000000000001e-06 2020-08-07T15:08:53.926 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.609344+0000", 2020-08-07T15:08:53.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0866999999999999e-05 2020-08-07T15:08:53.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.609355+0000", 2020-08-07T15:08:53.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9769999999999997e-06 2020-08-07T15:08:53.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.609361+0000", 2020-08-07T15:08:53.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0055748799999999999 2020-08-07T15:08:53.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.614936+0000", 2020-08-07T15:08:53.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000101758 2020-08-07T15:08:53.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.615037+0000", 2020-08-07T15:08:53.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016602899999999999 2020-08-07T15:08:53.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.938 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4", 2020-08-07T15:08:53.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.615203+0000", 2020-08-07T15:08:53.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0040052630000000002 2020-08-07T15:08:53.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:53.942 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.619209+0000", 2020-08-07T15:08:53.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0099160299999999993 2020-08-07T15:08:53.943 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.943 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.943 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.944 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:35928 4.f 4:f5e07511:::benchmark_data_smithi074_44586_object4490:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e106)", 2020-08-07T15:08:53.944 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:53.609387+0000", 2020-08-07T15:08:53.944 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0090633220000000004, 2020-08-07T15:08:53.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0099594550000000007, 2020-08-07T15:08:53.945 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.945 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:08:53.946 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:53.947 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:08:53.947 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:53.947 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 35928 2020-08-07T15:08:53.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.948 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.949 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.949 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.609387+0000", 2020-08-07T15:08:53.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.952 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.609387+0000", 2020-08-07T15:08:53.953 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5018999999999999e-05 2020-08-07T15:08:53.953 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.953 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.953 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.953 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.609412+0000", 2020-08-07T15:08:53.954 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3170000000000002e-06 2020-08-07T15:08:53.954 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.954 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.955 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.955 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.609415+0000", 2020-08-07T15:08:53.956 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.49e-06 2020-08-07T15:08:53.956 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.956 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.957 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.609421+0000", 2020-08-07T15:08:53.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6589999999999999e-06 2020-08-07T15:08:53.957 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.609424+0000", 2020-08-07T15:08:53.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0066299849999999997 2020-08-07T15:08:53.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.962 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:53.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.616054+0000", 2020-08-07T15:08:53.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5487000000000002e-05 2020-08-07T15:08:53.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:53.964 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.616149+0000", 2020-08-07T15:08:53.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015964700000000001 2020-08-07T15:08:53.969 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.969 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.969 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4", 2020-08-07T15:08:53.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.616309+0000", 2020-08-07T15:08:53.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069216039999999996 2020-08-07T15:08:53.970 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.970 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.971 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.972 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:35943 4.0 e106/81)", 2020-08-07T15:08:53.972 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:53.619196+0000", 2020-08-07T15:08:53.973 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4294967295.9992547, 2020-08-07T15:08:53.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00023016700000000001, 2020-08-07T15:08:53.974 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.974 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:08:53.974 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.975 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.619196+0000", 2020-08-07T15:08:53.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.619196+0000", 2020-08-07T15:08:53.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6848000000000003e-05 2020-08-07T15:08:53.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.619243+0000", 2020-08-07T15:08:53.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4870000000000002e-06 2020-08-07T15:08:53.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.979 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.979 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.619248+0000", 2020-08-07T15:08:53.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1367000000000001e-05 2020-08-07T15:08:53.979 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.980 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.619269+0000", 2020-08-07T15:08:53.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0180000000000003e-06 2020-08-07T15:08:53.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.982 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:53.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.619273+0000", 2020-08-07T15:08:53.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7719999999999994e-05 2020-08-07T15:08:53.985 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.985 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:53.986 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:53.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.987 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:35917 4.7 4:ea96bce7:::benchmark_data_smithi074_44586_object4489:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e106)", 2020-08-07T15:08:53.988 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:53.602273+0000", 2020-08-07T15:08:53.988 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.016177099, 2020-08-07T15:08:53.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.017214338999999999, 2020-08-07T15:08:53.989 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:53.989 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:53.990 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:53.990 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:08:53.990 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:53.991 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 35917 2020-08-07T15:08:53.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.991 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:53.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:53.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.602273+0000", 2020-08-07T15:08:53.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:53.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.993 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.993 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:53.993 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.602273+0000", 2020-08-07T15:08:53.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0568000000000001e-05 2020-08-07T15:08:53.994 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.994 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.994 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:53.995 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.602324+0000", 2020-08-07T15:08:53.995 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2950000000000001e-06 2020-08-07T15:08:53.995 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.995 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.995 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:53.996 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.602329+0000", 2020-08-07T15:08:53.996 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1871999999999999e-05 2020-08-07T15:08:53.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:53.997 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:53.997 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:53.997 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.602341+0000", 2020-08-07T15:08:53.997 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.865e-06 2020-08-07T15:08:53.999 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.000 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.000 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:54.000 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.602346+0000", 2020-08-07T15:08:54.000 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0030395520000000001 2020-08-07T15:08:54.001 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.001 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.001 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:54.001 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.605385+0000", 2020-08-07T15:08:54.002 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3646000000000004e-05 2020-08-07T15:08:54.002 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:54.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.605479+0000", 2020-08-07T15:08:54.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000164112 2020-08-07T15:08:54.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,5", 2020-08-07T15:08:54.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.605643+0000", 2020-08-07T15:08:54.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0052889800000000004 2020-08-07T15:08:54.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:54.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.610932+0000", 2020-08-07T15:08:54.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9700000000000005e-07 2020-08-07T15:08:54.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:54.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.610933+0000", 2020-08-07T15:08:54.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3075000000000002e-05 2020-08-07T15:08:54.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:54.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.610996+0000", 2020-08-07T15:08:54.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00034073699999999997 2020-08-07T15:08:54.009 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.010 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:54.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.611337+0000", 2020-08-07T15:08:54.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0090635990000000003 2020-08-07T15:08:54.012 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.012 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:54.013 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.013 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:35918 4.7 4:ea96bce7:::benchmark_data_smithi074_44586_object4489:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e106)", 2020-08-07T15:08:54.014 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:53.603314+0000", 2020-08-07T15:08:54.014 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.015136276000000001, 2020-08-07T15:08:54.014 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.016291040999999999, 2020-08-07T15:08:54.015 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:54.015 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:54.015 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:54.016 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:08:54.016 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:54.016 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 35918 2020-08-07T15:08:54.016 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.017 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:54.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:54.017 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.603314+0000", 2020-08-07T15:08:54.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:54.018 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.018 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.019 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:54.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.603314+0000", 2020-08-07T15:08:54.019 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1211000000000001e-05 2020-08-07T15:08:54.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:54.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.603365+0000", 2020-08-07T15:08:54.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2899999999999999e-06 2020-08-07T15:08:54.023 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:54.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.603372+0000", 2020-08-07T15:08:54.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1714e-05 2020-08-07T15:08:54.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:54.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.603383+0000", 2020-08-07T15:08:54.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6949999999999997e-06 2020-08-07T15:08:54.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:54.026 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.603389+0000", 2020-08-07T15:08:54.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002715271 2020-08-07T15:08:54.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:54.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.606104+0000", 2020-08-07T15:08:54.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0121e-05 2020-08-07T15:08:54.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:54.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.606194+0000", 2020-08-07T15:08:54.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016590100000000001 2020-08-07T15:08:54.030 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.030 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.030 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,5", 2020-08-07T15:08:54.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.606360+0000", 2020-08-07T15:08:54.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0046450650000000003 2020-08-07T15:08:54.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:54.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.611005+0000", 2020-08-07T15:08:54.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.172e-06 2020-08-07T15:08:54.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:54.034 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.611007+0000", 2020-08-07T15:08:54.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6686000000000001e-05 2020-08-07T15:08:54.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.034 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:54.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.611043+0000", 2020-08-07T15:08:54.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000330596 2020-08-07T15:08:54.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:54.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.611374+0000", 2020-08-07T15:08:54.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0080597220000000001 2020-08-07T15:08:54.037 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.037 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:54.037 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.038 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.038 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.038 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:35919 4.7 4:ea96bce7:::benchmark_data_smithi074_44586_object4489:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e106)", 2020-08-07T15:08:54.038 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:53.603426+0000", 2020-08-07T15:08:54.039 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.015024473, 2020-08-07T15:08:54.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.016294129000000001, 2020-08-07T15:08:54.039 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:54.040 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:54.040 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:54.040 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:08:54.041 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:54.041 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 35919 2020-08-07T15:08:54.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.041 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:54.042 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:54.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.603426+0000", 2020-08-07T15:08:54.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:54.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.044 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:54.044 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.603426+0000", 2020-08-07T15:08:54.045 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8288999999999998e-05 2020-08-07T15:08:54.045 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.045 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:54.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.603474+0000", 2020-08-07T15:08:54.046 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3109999999999996e-06 2020-08-07T15:08:54.046 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.046 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.047 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:54.047 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.603479+0000", 2020-08-07T15:08:54.047 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0961999999999999e-05 2020-08-07T15:08:54.047 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.048 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:54.048 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.603490+0000", 2020-08-07T15:08:54.048 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8169999999999998e-06 2020-08-07T15:08:54.049 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.049 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.049 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:54.050 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.603495+0000", 2020-08-07T15:08:54.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032352520000000001 2020-08-07T15:08:54.050 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.050 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.051 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:54.051 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.606731+0000", 2020-08-07T15:08:54.051 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1051000000000006e-05 2020-08-07T15:08:54.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.052 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.052 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:54.052 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.606822+0000", 2020-08-07T15:08:54.053 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000158018 2020-08-07T15:08:54.053 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.054 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,5", 2020-08-07T15:08:54.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.606980+0000", 2020-08-07T15:08:54.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0040688549999999997 2020-08-07T15:08:54.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:54.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.611048+0000", 2020-08-07T15:08:54.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8400000000000005e-07 2020-08-07T15:08:54.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.058 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:54.058 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.611049+0000", 2020-08-07T15:08:54.058 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7005999999999997e-05 2020-08-07T15:08:54.059 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.059 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.059 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:54.060 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.611096+0000", 2020-08-07T15:08:54.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00033944900000000002 2020-08-07T15:08:54.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.061 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:54.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.611435+0000", 2020-08-07T15:08:54.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0080094940000000007 2020-08-07T15:08:54.061 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.062 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:54.062 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.062 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.062 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:08:54.063 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 19 2020-08-07T15:08:54.063 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:54.066 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:08:54.070 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:54.063+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:54.168 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:54.165+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:54.243 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:54.244 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:08:54.244 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:08:54.244 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:08:54.245 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:54.245 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.3 to 1 (10000) 2020-08-07T15:08:54.257 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:08:54.262 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 4 1 2020-08-07T15:08:54.267 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:54.265+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:54.290 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 28 16 37070 37054 10.3379 13.5625 0.0108545 0.0120725 2020-08-07T15:08:54.399 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:54.415 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:08:54.415 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:08:54.416 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:08:54.416 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.416 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:15576 3.e e27/20 3:727724a5:::benchmark_data_smithi074_35447_object1946:head v 27'1032, mlcod=27'1023)", 2020-08-07T15:08:54.416 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:08:54.417 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 127.724580194, 2020-08-07T15:08:54.417 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.46091973000000003, 2020-08-07T15:08:54.417 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:54.417 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:54.417 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:54.418 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.418 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:54.418 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:08:54.418 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:54.418 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.419 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.419 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:54.419 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:08:54.419 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9592000000000001e-05 2020-08-07T15:08:54.419 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.419 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.420 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:54.420 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669542+0000", 2020-08-07T15:08:54.420 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.113e-06 2020-08-07T15:08:54.420 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.420 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:54.426 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669547+0000", 2020-08-07T15:08:54.426 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45151617799999999 2020-08-07T15:08:54.426 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.426 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:54.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121064+0000", 2020-08-07T15:08:54.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.377e-06 2020-08-07T15:08:54.427 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:54.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121067+0000", 2020-08-07T15:08:54.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9195e-05 2020-08-07T15:08:54.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.429 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:54.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121106+0000", 2020-08-07T15:08:54.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7509000000000001e-05 2020-08-07T15:08:54.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.430 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.430 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:54.430 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121124+0000", 2020-08-07T15:08:54.430 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0091783100000000003 2020-08-07T15:08:54.430 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.431 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:54.431 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.130302+0000", 2020-08-07T15:08:54.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000100456 2020-08-07T15:08:54.437 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.437 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.437 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:54.438 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.130402+0000", 2020-08-07T15:08:54.438 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.46091973000000003 2020-08-07T15:08:54.438 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.439 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:54.439 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.439 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.439 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.439 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:24264 3.8 e28/20 3:1d3fdbda:::benchmark_data_smithi074_35447_object3032:head v 28'1520, mlcod=28'1515)", 2020-08-07T15:08:54.440 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:08:54.440 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 122.010950157, 2020-08-07T15:08:54.440 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41856254199999998, 2020-08-07T15:08:54.440 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:54.440 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:54.441 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:54.441 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.441 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:54.441 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:08:54.441 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:54.442 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.446 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.447 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:54.447 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:08:54.447 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1348999999999998e-05 2020-08-07T15:08:54.448 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.450 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:54.450 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383174+0000", 2020-08-07T15:08:54.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6579999999999996e-06 2020-08-07T15:08:54.451 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.451 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.451 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:54.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383181+0000", 2020-08-07T15:08:54.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41194352699999998 2020-08-07T15:08:54.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:54.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.795124+0000", 2020-08-07T15:08:54.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5729999999999999e-06 2020-08-07T15:08:54.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.458 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.458 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:54.458 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.795128+0000", 2020-08-07T15:08:54.459 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002170384 2020-08-07T15:08:54.459 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:54.459 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.797298+0000", 2020-08-07T15:08:54.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5412e-05 2020-08-07T15:08:54.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.460 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.460 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:54.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.797324+0000", 2020-08-07T15:08:54.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042943360000000002 2020-08-07T15:08:54.461 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.461 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.461 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:54.461 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.801618+0000", 2020-08-07T15:08:54.462 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7302999999999997e-05 2020-08-07T15:08:54.462 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.462 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.462 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:54.462 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.801675+0000", 2020-08-07T15:08:54.463 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41856254199999998 2020-08-07T15:08:54.469 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.469 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:54.469 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.469 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.469 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.470 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:86712 3.8 e52/20 3:14925daa:::benchmark_data_smithi074_35447_object10838:head v 52'5488, mlcod=52'5479)", 2020-08-07T15:08:54.470 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.337908+0000", 2020-08-07T15:08:54.470 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 80.056154422999995, 2020-08-07T15:08:54.470 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.305870904, 2020-08-07T15:08:54.470 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:54.471 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:54.471 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:54.471 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.471 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:54.483 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.337908+0000", 2020-08-07T15:08:54.483 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:54.483 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.486 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.486 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:54.499 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.337908+0000", 2020-08-07T15:08:54.500 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1465999999999999e-05 2020-08-07T15:08:54.500 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.500 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.500 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:54.501 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.337950+0000", 2020-08-07T15:08:54.501 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1810000000000003e-06 2020-08-07T15:08:54.501 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.501 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.501 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:54.502 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.337954+0000", 2020-08-07T15:08:54.502 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30500923899999999 2020-08-07T15:08:54.502 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.502 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.502 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:54.511 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.642963+0000", 2020-08-07T15:08:54.511 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5389999999999998e-06 2020-08-07T15:08:54.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.532 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.533 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:54.533 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.642969+0000", 2020-08-07T15:08:54.533 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8265e-05 2020-08-07T15:08:54.549 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.550 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.550 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:54.550 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.643017+0000", 2020-08-07T15:08:54.550 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6812999999999999e-05 2020-08-07T15:08:54.550 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.551 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.551 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:54.551 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.643044+0000", 2020-08-07T15:08:54.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00067619800000000003 2020-08-07T15:08:54.551 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.552 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:54.552 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.643720+0000", 2020-08-07T15:08:54.552 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9203000000000002e-05 2020-08-07T15:08:54.578 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.579 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.579 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:54.579 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.643779+0000", 2020-08-07T15:08:54.579 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.305870904 2020-08-07T15:08:54.579 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.580 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:54.580 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.580 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.581 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.581 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:87088 3.6 3:6b0320df:::benchmark_data_smithi074_35447_object10885:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:08:54.581 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.958567+0000", 2020-08-07T15:08:54.582 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 79.435495864999993, 2020-08-07T15:08:54.585 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30442327000000002, 2020-08-07T15:08:54.585 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:54.585 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:54.585 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:54.586 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:54.586 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:54.586 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 87088 2020-08-07T15:08:54.586 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.587 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:54.587 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.587 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:54.587 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958567+0000", 2020-08-07T15:08:54.587 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:54.588 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.588 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:54.589 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958567+0000", 2020-08-07T15:08:54.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5040000000000002e-05 2020-08-07T15:08:54.589 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.590 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:54.590 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958612+0000", 2020-08-07T15:08:54.590 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4519999999999996e-06 2020-08-07T15:08:54.590 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.590 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.591 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:54.591 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958617+0000", 2020-08-07T15:08:54.591 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3974e-05 2020-08-07T15:08:54.591 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.591 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.592 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:54.592 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958631+0000", 2020-08-07T15:08:54.592 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5619999999999997e-06 2020-08-07T15:08:54.592 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.592 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.593 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:54.593 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.958636+0000", 2020-08-07T15:08:54.593 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0071073999999999998 2020-08-07T15:08:54.593 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.597 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.597 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:54.597 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.965743+0000", 2020-08-07T15:08:54.598 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5477999999999996e-05 2020-08-07T15:08:54.598 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.598 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.599 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:54.599 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.965839+0000", 2020-08-07T15:08:54.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000168061 2020-08-07T15:08:54.599 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.599 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.600 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:54.600 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966007+0000", 2020-08-07T15:08:54.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054997550000000003 2020-08-07T15:08:54.600 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.600 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.601 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:54.601 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971507+0000", 2020-08-07T15:08:54.601 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6200000000000006e-07 2020-08-07T15:08:54.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.602 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:54.602 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971508+0000", 2020-08-07T15:08:54.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012566300000000001 2020-08-07T15:08:54.602 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.602 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.603 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:54.603 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.971633+0000", 2020-08-07T15:08:54.603 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0075941899999999998 2020-08-07T15:08:54.603 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.603 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:54.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.979228+0000", 2020-08-07T15:08:54.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28372433899999999 2020-08-07T15:08:54.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:54.605 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262952+0000", 2020-08-07T15:08:54.605 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2799999999999995e-07 2020-08-07T15:08:54.630 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.630 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.630 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:54.630 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262953+0000", 2020-08-07T15:08:54.630 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7666000000000002e-05 2020-08-07T15:08:54.631 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.631 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.631 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:54.631 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.262990+0000", 2020-08-07T15:08:54.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30442327000000002 2020-08-07T15:08:54.632 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.632 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:54.632 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.633 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.633 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.633 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:93920 3.6 e56/26 ondisk, result = 0)", 2020-08-07T15:08:54.633 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:08:54.636 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 75.063586893999997, 2020-08-07T15:08:54.637 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.397576347, 2020-08-07T15:08:54.637 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:54.637 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:54.638 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:54.638 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.638 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:54.638 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:08:54.638 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:54.639 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.639 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.639 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:54.639 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:08:54.639 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2571999999999999e-05 2020-08-07T15:08:54.640 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.640 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.640 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:54.643 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330509+0000", 2020-08-07T15:08:54.643 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2059999999999998e-06 2020-08-07T15:08:54.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.643 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.644 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:54.644 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330514+0000", 2020-08-07T15:08:54.644 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39740134599999999 2020-08-07T15:08:54.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.644 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:54.645 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727915+0000", 2020-08-07T15:08:54.645 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.677e-06 2020-08-07T15:08:54.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.646 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:54.646 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727921+0000", 2020-08-07T15:08:54.646 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5431e-05 2020-08-07T15:08:54.652 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.653 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.653 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:54.653 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727956+0000", 2020-08-07T15:08:54.654 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5790000000000004e-06 2020-08-07T15:08:54.654 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.654 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.654 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:54.655 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727965+0000", 2020-08-07T15:08:54.655 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7535999999999997e-05 2020-08-07T15:08:54.655 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.655 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.655 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:54.656 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.728052+0000", 2020-08-07T15:08:54.656 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.397576347 2020-08-07T15:08:54.656 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.656 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:54.656 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.657 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.657 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.657 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106104 3.0 3:0d7a2a89:::benchmark_data_smithi074_35447_object13262:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:08:54.657 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.252296+0000", 2020-08-07T15:08:54.658 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 67.141767005000005, 2020-08-07T15:08:54.658 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.44182183200000003, 2020-08-07T15:08:54.658 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:54.659 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:54.659 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:54.659 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:54.660 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:54.660 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106104 2020-08-07T15:08:54.660 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.661 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:54.661 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.661 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:54.661 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252296+0000", 2020-08-07T15:08:54.661 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:54.662 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.662 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.663 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:54.663 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252296+0000", 2020-08-07T15:08:54.663 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4728000000000001e-05 2020-08-07T15:08:54.663 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.664 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:54.664 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252321+0000", 2020-08-07T15:08:54.664 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9529999999999999e-06 2020-08-07T15:08:54.664 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.665 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.665 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:54.665 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252324+0000", 2020-08-07T15:08:54.665 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5759999999999996e-06 2020-08-07T15:08:54.665 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.666 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.666 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:54.666 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252329+0000", 2020-08-07T15:08:54.666 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.582e-06 2020-08-07T15:08:54.666 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.667 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:54.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252332+0000", 2020-08-07T15:08:54.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012958519999999999 2020-08-07T15:08:54.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:54.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.253628+0000", 2020-08-07T15:08:54.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7871e-05 2020-08-07T15:08:54.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:54.670 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.253705+0000", 2020-08-07T15:08:54.670 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017801199999999999 2020-08-07T15:08:54.670 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.670 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.670 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:08:54.671 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.253883+0000", 2020-08-07T15:08:54.671 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001869051 2020-08-07T15:08:54.671 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.671 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.671 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:54.672 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255752+0000", 2020-08-07T15:08:54.672 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3e-07 2020-08-07T15:08:54.672 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.672 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.672 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:54.673 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255753+0000", 2020-08-07T15:08:54.674 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5321000000000001e-05 2020-08-07T15:08:54.674 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.674 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.674 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:54.674 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255808+0000", 2020-08-07T15:08:54.675 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.091917211999999998 2020-08-07T15:08:54.675 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.675 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.675 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:54.675 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.347726+0000", 2020-08-07T15:08:54.676 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.34634211500000001 2020-08-07T15:08:54.676 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.676 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:54.677 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694068+0000", 2020-08-07T15:08:54.677 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3030000000000001e-06 2020-08-07T15:08:54.677 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.678 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.678 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:54.678 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694069+0000", 2020-08-07T15:08:54.679 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8625999999999998e-05 2020-08-07T15:08:54.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.679 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:54.680 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694118+0000", 2020-08-07T15:08:54.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.44182183200000003 2020-08-07T15:08:54.680 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.680 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:54.680 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.681 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.681 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.681 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106104 3.0 e58/20 ondisk, result = 0)", 2020-08-07T15:08:54.681 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.257196+0000", 2020-08-07T15:08:54.681 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 67.136866498000003, 2020-08-07T15:08:54.682 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43692689000000001, 2020-08-07T15:08:54.682 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:54.682 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:54.682 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:54.682 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.683 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:54.683 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257196+0000", 2020-08-07T15:08:54.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:54.684 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.684 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.684 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:54.685 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257196+0000", 2020-08-07T15:08:54.685 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2021999999999999e-05 2020-08-07T15:08:54.685 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.695 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:54.695 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257228+0000", 2020-08-07T15:08:54.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9549999999999997e-06 2020-08-07T15:08:54.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.696 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:54.696 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257234+0000", 2020-08-07T15:08:54.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43671786400000001 2020-08-07T15:08:54.697 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:54.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693952+0000", 2020-08-07T15:08:54.697 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7690000000000004e-06 2020-08-07T15:08:54.699 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.700 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:54.700 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693957+0000", 2020-08-07T15:08:54.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3294000000000006e-05 2020-08-07T15:08:54.701 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.701 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:54.701 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694040+0000", 2020-08-07T15:08:54.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3436e-05 2020-08-07T15:08:54.702 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.702 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.702 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:54.702 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694054+0000", 2020-08-07T15:08:54.703 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9549999999999996e-05 2020-08-07T15:08:54.703 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.704 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:54.704 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694123+0000", 2020-08-07T15:08:54.704 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43692689000000001 2020-08-07T15:08:54.704 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.706 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:54.712 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.712 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.713 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.713 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106144 3.6 3:629055a5:::benchmark_data_smithi074_35447_object13267:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:08:54.713 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.280142+0000", 2020-08-07T15:08:54.713 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 67.113921035999994, 2020-08-07T15:08:54.713 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.57968657199999996, 2020-08-07T15:08:54.715 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:54.716 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:54.716 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:54.717 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:54.717 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:54.717 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106144 2020-08-07T15:08:54.717 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.717 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:54.718 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.718 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:54.718 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280142+0000", 2020-08-07T15:08:54.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:54.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:54.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280142+0000", 2020-08-07T15:08:54.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.905e-05 2020-08-07T15:08:54.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:54.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280191+0000", 2020-08-07T15:08:54.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6729999999999997e-06 2020-08-07T15:08:54.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:54.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280197+0000", 2020-08-07T15:08:54.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0747999999999999e-05 2020-08-07T15:08:54.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.721 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.722 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:54.722 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280207+0000", 2020-08-07T15:08:54.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7700000000000001e-06 2020-08-07T15:08:54.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.723 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:54.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280212+0000", 2020-08-07T15:08:54.724 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00060223399999999999 2020-08-07T15:08:54.724 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.724 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.724 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:54.724 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280814+0000", 2020-08-07T15:08:54.725 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7831000000000006e-05 2020-08-07T15:08:54.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.725 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:54.725 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280912+0000", 2020-08-07T15:08:54.726 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000137119 2020-08-07T15:08:54.726 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.726 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.727 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:54.727 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.281049+0000", 2020-08-07T15:08:54.727 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023794369999999999 2020-08-07T15:08:54.727 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.727 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:54.728 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283429+0000", 2020-08-07T15:08:54.728 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.173e-06 2020-08-07T15:08:54.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.729 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:54.729 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283430+0000", 2020-08-07T15:08:54.729 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5530000000000004e-05 2020-08-07T15:08:54.729 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.729 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.730 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:54.730 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283495+0000", 2020-08-07T15:08:54.730 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5649000000000003e-05 2020-08-07T15:08:54.730 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.730 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.731 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:54.731 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283541+0000", 2020-08-07T15:08:54.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.57624206300000003 2020-08-07T15:08:54.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:54.732 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859783+0000", 2020-08-07T15:08:54.732 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.421e-06 2020-08-07T15:08:54.732 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.732 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.733 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:54.734 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859785+0000", 2020-08-07T15:08:54.734 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3874e-05 2020-08-07T15:08:54.734 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.734 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.735 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:54.735 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859828+0000", 2020-08-07T15:08:54.735 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.57968657199999996 2020-08-07T15:08:54.735 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.736 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:54.736 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.736 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.736 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106144 3.6 e59/26 ondisk, result = 0)", 2020-08-07T15:08:54.737 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.344536+0000", 2020-08-07T15:08:54.737 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 67.049526693999994, 2020-08-07T15:08:54.738 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51529786099999997, 2020-08-07T15:08:54.738 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:54.738 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:54.738 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:54.738 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.739 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:54.739 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344536+0000", 2020-08-07T15:08:54.739 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:54.739 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.739 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.740 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:54.740 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344536+0000", 2020-08-07T15:08:54.740 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7266999999999999e-05 2020-08-07T15:08:54.740 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.740 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.741 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:54.741 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344553+0000", 2020-08-07T15:08:54.741 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1070000000000001e-06 2020-08-07T15:08:54.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.742 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.742 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:54.742 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344557+0000", 2020-08-07T15:08:54.742 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.515143201 2020-08-07T15:08:54.743 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:54.744 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859700+0000", 2020-08-07T15:08:54.744 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.271e-05 2020-08-07T15:08:54.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.745 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:54.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859712+0000", 2020-08-07T15:08:54.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5436999999999998e-05 2020-08-07T15:08:54.745 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.746 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:54.747 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859768+0000", 2020-08-07T15:08:54.747 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1540000000000002e-06 2020-08-07T15:08:54.748 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.748 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.748 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:54.748 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859776+0000", 2020-08-07T15:08:54.748 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7985000000000001e-05 2020-08-07T15:08:54.749 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.749 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.749 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:54.749 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859834+0000", 2020-08-07T15:08:54.750 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51529786099999997 2020-08-07T15:08:54.750 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.750 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:54.750 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.750 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.751 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.751 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108352 3.8 3:1aac8373:::benchmark_data_smithi074_35447_object13543:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e62)", 2020-08-07T15:08:54.751 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.642093+0000", 2020-08-07T15:08:54.752 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 63.751970004999997, 2020-08-07T15:08:54.752 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39397409900000002, 2020-08-07T15:08:54.752 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:54.752 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:54.752 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:54.753 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:54.753 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:54.753 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108352 2020-08-07T15:08:54.753 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.756 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:54.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:54.756 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642093+0000", 2020-08-07T15:08:54.757 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:54.757 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.757 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:54.766 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642093+0000", 2020-08-07T15:08:54.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9475000000000002e-05 2020-08-07T15:08:54.766 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:54.767 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642122+0000", 2020-08-07T15:08:54.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0439999999999997e-06 2020-08-07T15:08:54.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.768 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:54.768 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642126+0000", 2020-08-07T15:08:54.768 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9689999999999994e-06 2020-08-07T15:08:54.769 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.769 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.769 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:54.769 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642134+0000", 2020-08-07T15:08:54.770 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1219999999999997e-06 2020-08-07T15:08:54.770 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.770 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.770 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:54.771 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642138+0000", 2020-08-07T15:08:54.771 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011580399999999999 2020-08-07T15:08:54.771 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.771 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.772 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:54.772 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.643296+0000", 2020-08-07T15:08:54.772 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3791999999999999e-05 2020-08-07T15:08:54.772 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.774 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.774 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:54.774 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.643390+0000", 2020-08-07T15:08:54.775 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011882700000000001 2020-08-07T15:08:54.775 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.775 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4,7", 2020-08-07T15:08:54.780 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.643509+0000", 2020-08-07T15:08:54.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034736799999999998 2020-08-07T15:08:54.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.781 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:54.781 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.646983+0000", 2020-08-07T15:08:54.783 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0077748490000000003 2020-08-07T15:08:54.783 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:54.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.654758+0000", 2020-08-07T15:08:54.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000005e-07 2020-08-07T15:08:54.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:54.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.654758+0000", 2020-08-07T15:08:54.785 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4258000000000002e-05 2020-08-07T15:08:54.785 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:54.786 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.654802+0000", 2020-08-07T15:08:54.786 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11166224399999999 2020-08-07T15:08:54.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.797 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:54.798 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.766465+0000", 2020-08-07T15:08:54.798 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26946740899999999 2020-08-07T15:08:54.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:54.800 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.035932+0000", 2020-08-07T15:08:54.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5829999999999999e-06 2020-08-07T15:08:54.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.801 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.801 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:54.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.035934+0000", 2020-08-07T15:08:54.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000133306 2020-08-07T15:08:54.801 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.802 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.802 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:54.802 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.036067+0000", 2020-08-07T15:08:54.802 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39397409900000002 2020-08-07T15:08:54.803 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.803 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:54.804 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.806 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108464 3.6 3:62704f92:::benchmark_data_smithi074_35447_object13557:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e62)", 2020-08-07T15:08:54.807 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.770928+0000", 2020-08-07T15:08:54.807 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 63.623134864000001, 2020-08-07T15:08:54.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30592449999999999, 2020-08-07T15:08:54.807 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:54.807 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:54.808 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:54.808 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:54.808 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:54.808 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108464 2020-08-07T15:08:54.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.809 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:54.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.809 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:54.809 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770928+0000", 2020-08-07T15:08:54.809 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:54.810 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.810 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.810 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:54.810 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770928+0000", 2020-08-07T15:08:54.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7257e-05 2020-08-07T15:08:54.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:54.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770955+0000", 2020-08-07T15:08:54.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0080000000000004e-06 2020-08-07T15:08:54.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.813 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:54.814 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770959+0000", 2020-08-07T15:08:54.814 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2810000000000003e-06 2020-08-07T15:08:54.814 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.815 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:54.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770967+0000", 2020-08-07T15:08:54.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3850000000000004e-06 2020-08-07T15:08:54.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.817 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:54.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770971+0000", 2020-08-07T15:08:54.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4960000000000002e-05 2020-08-07T15:08:54.818 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.818 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.818 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:54.818 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.770996+0000", 2020-08-07T15:08:54.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4565999999999999e-05 2020-08-07T15:08:54.819 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:54.819 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.771050+0000", 2020-08-07T15:08:54.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010631799999999999 2020-08-07T15:08:54.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:54.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.771157+0000", 2020-08-07T15:08:54.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0049076099999999998 2020-08-07T15:08:54.821 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:54.821 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.776064+0000", 2020-08-07T15:08:54.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3700000000000001e-07 2020-08-07T15:08:54.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.822 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.822 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:54.822 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.776065+0000", 2020-08-07T15:08:54.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8387999999999998e-05 2020-08-07T15:08:54.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:54.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.776113+0000", 2020-08-07T15:08:54.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00068335599999999998 2020-08-07T15:08:54.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:54.826 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.776797+0000", 2020-08-07T15:08:54.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30001175499999999 2020-08-07T15:08:54.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.840 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:54.846 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.076808+0000", 2020-08-07T15:08:54.851 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4200000000000004e-07 2020-08-07T15:08:54.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:54.852 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.076809+0000", 2020-08-07T15:08:54.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3237e-05 2020-08-07T15:08:54.898 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.898 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.898 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:54.899 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.076852+0000", 2020-08-07T15:08:54.899 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30592449999999999 2020-08-07T15:08:54.899 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.899 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:54.899 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.900 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145767 3.6 3:62fe5854:::benchmark_data_smithi074_35447_object16158:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:08:54.900 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.654765+0000", 2020-08-07T15:08:54.900 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 39.739297819000001, 2020-08-07T15:08:54.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.62108287699999998, 2020-08-07T15:08:54.901 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:54.901 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:54.901 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:54.901 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:54.902 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:54.902 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145767 2020-08-07T15:08:54.902 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.903 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:54.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.903 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:54.903 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654765+0000", 2020-08-07T15:08:54.903 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:54.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.904 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:54.904 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654765+0000", 2020-08-07T15:08:54.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1977999999999998e-05 2020-08-07T15:08:54.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:54.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654797+0000", 2020-08-07T15:08:54.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6799999999999998e-06 2020-08-07T15:08:54.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.906 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.906 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:54.906 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654803+0000", 2020-08-07T15:08:54.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1263e-05 2020-08-07T15:08:54.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:54.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654814+0000", 2020-08-07T15:08:54.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2129999999999999e-06 2020-08-07T15:08:54.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.908 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:54.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654819+0000", 2020-08-07T15:08:54.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0324999999999997e-05 2020-08-07T15:08:54.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:54.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654869+0000", 2020-08-07T15:08:54.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015424199999999999 2020-08-07T15:08:54.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.910 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:54.910 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.655024+0000", 2020-08-07T15:08:54.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000163573 2020-08-07T15:08:54.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:54.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.655187+0000", 2020-08-07T15:08:54.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001341645 2020-08-07T15:08:54.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:54.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.656529+0000", 2020-08-07T15:08:54.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9599999999999998e-07 2020-08-07T15:08:54.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:54.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.656530+0000", 2020-08-07T15:08:54.914 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4381e-05 2020-08-07T15:08:54.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:54.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.656584+0000", 2020-08-07T15:08:54.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0057298979999999998 2020-08-07T15:08:54.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:54.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.662314+0000", 2020-08-07T15:08:54.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.61347776300000001 2020-08-07T15:08:54.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:54.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275792+0000", 2020-08-07T15:08:54.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9740000000000001e-06 2020-08-07T15:08:54.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:54.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275794+0000", 2020-08-07T15:08:54.918 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4146e-05 2020-08-07T15:08:54.918 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.919 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:54.919 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275848+0000", 2020-08-07T15:08:54.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.62108287699999998 2020-08-07T15:08:54.919 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.919 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:54.920 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.920 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145801 3.6 3:69e4cb22:::benchmark_data_smithi074_35447_object16192:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:08:54.920 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671123+0000", 2020-08-07T15:08:54.921 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 39.722939943999997, 2020-08-07T15:08:54.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74411769999999999, 2020-08-07T15:08:54.921 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:54.922 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:54.922 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:54.922 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:54.922 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:54.922 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145801 2020-08-07T15:08:54.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.923 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:54.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:54.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671123+0000", 2020-08-07T15:08:54.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:54.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:54.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671123+0000", 2020-08-07T15:08:54.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3691e-05 2020-08-07T15:08:54.925 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:54.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671157+0000", 2020-08-07T15:08:54.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0249999999999999e-06 2020-08-07T15:08:54.926 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.926 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:54.926 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671163+0000", 2020-08-07T15:08:54.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2339e-05 2020-08-07T15:08:54.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:54.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671175+0000", 2020-08-07T15:08:54.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5249999999999996e-06 2020-08-07T15:08:54.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:54.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671180+0000", 2020-08-07T15:08:54.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6826000000000002e-05 2020-08-07T15:08:54.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:54.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671227+0000", 2020-08-07T15:08:54.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014152699999999999 2020-08-07T15:08:54.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:54.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671369+0000", 2020-08-07T15:08:54.931 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8021000000000002e-05 2020-08-07T15:08:54.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:08:54.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671467+0000", 2020-08-07T15:08:54.932 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0016293760000000001 2020-08-07T15:08:54.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.932 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.932 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:54.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673096+0000", 2020-08-07T15:08:54.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.048e-06 2020-08-07T15:08:54.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.933 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.933 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:54.933 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673097+0000", 2020-08-07T15:08:54.934 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8841999999999998e-05 2020-08-07T15:08:54.934 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.934 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.934 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:54.934 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673146+0000", 2020-08-07T15:08:54.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002829935 2020-08-07T15:08:54.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:54.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675976+0000", 2020-08-07T15:08:54.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73917870399999996 2020-08-07T15:08:54.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:54.936 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.415155+0000", 2020-08-07T15:08:54.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.193e-06 2020-08-07T15:08:54.937 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:54.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.415157+0000", 2020-08-07T15:08:54.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3647999999999996e-05 2020-08-07T15:08:54.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.938 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:54.938 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.415241+0000", 2020-08-07T15:08:54.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74411769999999999 2020-08-07T15:08:54.939 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.939 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:54.939 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:54.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.940 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.940 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145803 3.0 3:0c939450:::benchmark_data_smithi074_35447_object16194:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:08:54.940 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671678+0000", 2020-08-07T15:08:54.940 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 39.722385148999997, 2020-08-07T15:08:54.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.882157211, 2020-08-07T15:08:54.941 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:54.941 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:54.941 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:54.942 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:54.942 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:54.942 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145803 2020-08-07T15:08:54.942 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.942 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:54.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.950 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:54.950 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671678+0000", 2020-08-07T15:08:54.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:54.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:54.954 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671678+0000", 2020-08-07T15:08:54.955 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4097999999999998e-05 2020-08-07T15:08:54.955 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.955 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.955 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:54.956 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671712+0000", 2020-08-07T15:08:54.956 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2439999999999998e-06 2020-08-07T15:08:54.956 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.956 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.956 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:54.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671718+0000", 2020-08-07T15:08:54.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2629999999999999e-05 2020-08-07T15:08:54.957 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.957 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:54.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671731+0000", 2020-08-07T15:08:54.958 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9589999999999998e-06 2020-08-07T15:08:54.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:54.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671738+0000", 2020-08-07T15:08:54.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2495000000000004e-05 2020-08-07T15:08:54.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:54.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671800+0000", 2020-08-07T15:08:54.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013640700000000001 2020-08-07T15:08:54.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:54.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671937+0000", 2020-08-07T15:08:54.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013672200000000001 2020-08-07T15:08:54.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:08:54.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.672073+0000", 2020-08-07T15:08:54.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001179982 2020-08-07T15:08:54.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.962 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:54.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673253+0000", 2020-08-07T15:08:54.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1900000000000001e-07 2020-08-07T15:08:54.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:54.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673254+0000", 2020-08-07T15:08:54.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2104000000000001e-05 2020-08-07T15:08:54.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.970 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:54.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673306+0000", 2020-08-07T15:08:54.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002264369 2020-08-07T15:08:54.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:54.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675571+0000", 2020-08-07T15:08:54.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.87820796899999998 2020-08-07T15:08:54.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:54.981 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553778+0000", 2020-08-07T15:08:54.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3629999999999999e-06 2020-08-07T15:08:54.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.982 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:54.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553780+0000", 2020-08-07T15:08:54.982 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5050000000000003e-05 2020-08-07T15:08:54.982 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:54.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:54.999 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:54.999 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553835+0000", 2020-08-07T15:08:54.999 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.882157211 2020-08-07T15:08:55.000 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.000 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:55.000 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.012 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.012 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145805 3.8 3:13b8e273:::benchmark_data_smithi074_35447_object16196:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:08:55.012 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671847+0000", 2020-08-07T15:08:55.012 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 39.722215829, 2020-08-07T15:08:55.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.340134453, 2020-08-07T15:08:55.013 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:55.013 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:55.013 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:55.013 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:55.013 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:55.014 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145805 2020-08-07T15:08:55.014 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.014 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:55.014 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:55.015 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671847+0000", 2020-08-07T15:08:55.015 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:55.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.016 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:55.016 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671847+0000", 2020-08-07T15:08:55.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1139999999999997e-05 2020-08-07T15:08:55.017 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:55.017 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671878+0000", 2020-08-07T15:08:55.017 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.784e-06 2020-08-07T15:08:55.018 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.018 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.018 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:55.018 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671884+0000", 2020-08-07T15:08:55.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2802999999999999e-05 2020-08-07T15:08:55.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.019 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.019 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:55.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671897+0000", 2020-08-07T15:08:55.019 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9909999999999999e-06 2020-08-07T15:08:55.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.020 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.020 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:55.020 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671903+0000", 2020-08-07T15:08:55.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011861 2020-08-07T15:08:55.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:55.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.672021+0000", 2020-08-07T15:08:55.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017702399999999999 2020-08-07T15:08:55.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:55.022 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.672198+0000", 2020-08-07T15:08:55.022 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020546400000000001 2020-08-07T15:08:55.023 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.023 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4,7", 2020-08-07T15:08:55.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.672404+0000", 2020-08-07T15:08:55.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024385019999999999 2020-08-07T15:08:55.023 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:55.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674842+0000", 2020-08-07T15:08:55.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00028594999999999998 2020-08-07T15:08:55.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:55.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675128+0000", 2020-08-07T15:08:55.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8100000000000003e-07 2020-08-07T15:08:55.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:55.026 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675129+0000", 2020-08-07T15:08:55.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7973999999999997e-05 2020-08-07T15:08:55.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:55.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675197+0000", 2020-08-07T15:08:55.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032171220000000002 2020-08-07T15:08:55.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:55.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678414+0000", 2020-08-07T15:08:55.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33343769299999998 2020-08-07T15:08:55.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:55.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011852+0000", 2020-08-07T15:08:55.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3480000000000001e-06 2020-08-07T15:08:55.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.030 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.030 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:55.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011853+0000", 2020-08-07T15:08:55.030 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000128467 2020-08-07T15:08:55.030 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:55.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011981+0000", 2020-08-07T15:08:55.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.340134453 2020-08-07T15:08:55.032 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.033 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:55.033 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.033 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145806 3.8 3:16669ddb:::benchmark_data_smithi074_35447_object16197:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:08:55.034 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.673992+0000", 2020-08-07T15:08:55.034 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 39.720071302999997, 2020-08-07T15:08:55.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40687891900000001, 2020-08-07T15:08:55.034 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:55.034 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:55.035 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:55.035 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:55.035 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:55.035 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145806 2020-08-07T15:08:55.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.036 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:55.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:55.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673992+0000", 2020-08-07T15:08:55.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:55.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.037 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:55.037 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673992+0000", 2020-08-07T15:08:55.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5429000000000002e-05 2020-08-07T15:08:55.038 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.038 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:55.038 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674027+0000", 2020-08-07T15:08:55.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3199999999999996e-06 2020-08-07T15:08:55.039 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.039 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.039 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:55.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674033+0000", 2020-08-07T15:08:55.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2835e-05 2020-08-07T15:08:55.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:55.040 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674046+0000", 2020-08-07T15:08:55.040 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9030000000000001e-06 2020-08-07T15:08:55.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.042 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:55.043 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674052+0000", 2020-08-07T15:08:55.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4181000000000003e-05 2020-08-07T15:08:55.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:55.044 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674096+0000", 2020-08-07T15:08:55.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00022310599999999999 2020-08-07T15:08:55.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.044 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:55.045 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674319+0000", 2020-08-07T15:08:55.045 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012681300000000001 2020-08-07T15:08:55.045 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.045 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4,7", 2020-08-07T15:08:55.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674446+0000", 2020-08-07T15:08:55.046 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002882754 2020-08-07T15:08:55.046 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.046 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.047 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:55.048 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.677329+0000", 2020-08-07T15:08:55.048 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9466e-05 2020-08-07T15:08:55.048 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.048 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.048 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:55.049 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.677418+0000", 2020-08-07T15:08:55.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0299999999999998e-07 2020-08-07T15:08:55.049 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.049 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.049 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:55.050 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.677419+0000", 2020-08-07T15:08:55.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5274000000000001e-05 2020-08-07T15:08:55.050 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.050 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.050 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:55.051 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.677464+0000", 2020-08-07T15:08:55.051 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40011127099999999 2020-08-07T15:08:55.051 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.051 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:55.052 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.077576+0000", 2020-08-07T15:08:55.052 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0031778370000000002 2020-08-07T15:08:55.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.052 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.052 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:55.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080754+0000", 2020-08-07T15:08:55.053 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.92e-07 2020-08-07T15:08:55.053 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.053 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:55.054 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080754+0000", 2020-08-07T15:08:55.054 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000116135 2020-08-07T15:08:55.054 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.054 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:55.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080870+0000", 2020-08-07T15:08:55.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40687891900000001 2020-08-07T15:08:55.055 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.055 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:55.056 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.056 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:145767 3.6 e75/26 ondisk, result = 0)", 2020-08-07T15:08:55.056 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.674108+0000", 2020-08-07T15:08:55.057 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 39.719954878999999, 2020-08-07T15:08:55.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.60174556899999998, 2020-08-07T15:08:55.057 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:55.057 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:55.057 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:55.058 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.058 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:55.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674108+0000", 2020-08-07T15:08:55.059 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:55.059 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.059 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.059 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:55.060 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674108+0000", 2020-08-07T15:08:55.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0558000000000001e-05 2020-08-07T15:08:55.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:55.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674139+0000", 2020-08-07T15:08:55.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6620000000000002e-06 2020-08-07T15:08:55.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.061 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:55.062 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674144+0000", 2020-08-07T15:08:55.062 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.60156723899999998 2020-08-07T15:08:55.062 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.062 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.063 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:55.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275711+0000", 2020-08-07T15:08:55.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4070000000000004e-06 2020-08-07T15:08:55.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:55.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275719+0000", 2020-08-07T15:08:55.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3205999999999998e-05 2020-08-07T15:08:55.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.064 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:55.065 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275772+0000", 2020-08-07T15:08:55.065 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0624e-05 2020-08-07T15:08:55.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.065 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:55.066 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275783+0000", 2020-08-07T15:08:55.066 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0872999999999997e-05 2020-08-07T15:08:55.066 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.066 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.066 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:55.067 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275854+0000", 2020-08-07T15:08:55.067 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.60174556899999998 2020-08-07T15:08:55.067 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.067 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:55.067 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.068 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.068 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:145803 3.0 e75/20 ondisk, result = 0)", 2020-08-07T15:08:55.068 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.674488+0000", 2020-08-07T15:08:55.068 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 39.719574897000001, 2020-08-07T15:08:55.069 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.87935229699999995, 2020-08-07T15:08:55.069 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:55.069 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:55.069 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:55.069 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.069 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:55.070 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674488+0000", 2020-08-07T15:08:55.070 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:55.070 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.070 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.070 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:55.071 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674488+0000", 2020-08-07T15:08:55.071 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3062000000000003e-05 2020-08-07T15:08:55.071 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.071 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.071 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:55.072 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674521+0000", 2020-08-07T15:08:55.072 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4911e-05 2020-08-07T15:08:55.072 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.073 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.074 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:55.074 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674536+0000", 2020-08-07T15:08:55.074 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.87914350600000002 2020-08-07T15:08:55.074 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.075 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.075 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:55.075 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553679+0000", 2020-08-07T15:08:55.075 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5950000000000001e-06 2020-08-07T15:08:55.075 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.076 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.076 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:55.076 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553687+0000", 2020-08-07T15:08:55.076 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2565999999999996e-05 2020-08-07T15:08:55.076 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.076 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.077 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:55.077 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553760+0000", 2020-08-07T15:08:55.077 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0417999999999999e-05 2020-08-07T15:08:55.077 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.077 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.078 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:55.078 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553770+0000", 2020-08-07T15:08:55.078 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0239000000000003e-05 2020-08-07T15:08:55.078 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.078 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.079 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:55.079 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553840+0000", 2020-08-07T15:08:55.079 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.87935229699999995 2020-08-07T15:08:55.079 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.079 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:55.080 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.080 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.080 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.080 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:145805 3.8 e75/60 ondisk, result = 0)", 2020-08-07T15:08:55.080 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.676843+0000", 2020-08-07T15:08:55.081 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 39.717219427000003, 2020-08-07T15:08:55.081 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.335144041, 2020-08-07T15:08:55.081 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:55.081 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:55.081 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:55.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.082 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:55.082 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.676843+0000", 2020-08-07T15:08:55.082 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:55.082 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.083 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.083 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:55.083 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.676843+0000", 2020-08-07T15:08:55.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7853e-05 2020-08-07T15:08:55.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.084 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:55.084 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.676861+0000", 2020-08-07T15:08:55.084 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7359999999999999e-06 2020-08-07T15:08:55.084 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.085 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:55.085 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.676865+0000", 2020-08-07T15:08:55.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33484658099999998 2020-08-07T15:08:55.086 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.086 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.086 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:55.086 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011712+0000", 2020-08-07T15:08:55.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1613e-05 2020-08-07T15:08:55.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:55.087 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011723+0000", 2020-08-07T15:08:55.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011038 2020-08-07T15:08:55.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.089 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:55.089 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011834+0000", 2020-08-07T15:08:55.089 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0097e-05 2020-08-07T15:08:55.089 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.089 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.090 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:55.090 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011844+0000", 2020-08-07T15:08:55.090 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000143781 2020-08-07T15:08:55.090 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.090 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.091 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:55.091 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011987+0000", 2020-08-07T15:08:55.091 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.335144041 2020-08-07T15:08:55.091 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.091 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:55.092 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.092 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.092 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.092 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:145806 3.8 e75/60 ondisk, result = 0)", 2020-08-07T15:08:55.092 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.678157+0000", 2020-08-07T15:08:55.093 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 39.715905374999998, 2020-08-07T15:08:55.093 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.402718152, 2020-08-07T15:08:55.093 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:55.093 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:55.093 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:55.094 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.094 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:55.094 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678157+0000", 2020-08-07T15:08:55.094 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:55.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.095 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.095 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:55.095 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678157+0000", 2020-08-07T15:08:55.095 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8496e-05 2020-08-07T15:08:55.096 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.096 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.096 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:55.096 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678176+0000", 2020-08-07T15:08:55.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.676e-06 2020-08-07T15:08:55.097 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.097 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.097 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:55.097 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678180+0000", 2020-08-07T15:08:55.097 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40250966999999999 2020-08-07T15:08:55.098 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.098 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.098 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:55.098 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080689+0000", 2020-08-07T15:08:55.098 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7110000000000003e-06 2020-08-07T15:08:55.099 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.099 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.099 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:55.099 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080694+0000", 2020-08-07T15:08:55.099 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4122999999999998e-05 2020-08-07T15:08:55.100 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.100 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.100 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:55.100 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080738+0000", 2020-08-07T15:08:55.100 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5809999999999999e-06 2020-08-07T15:08:55.100 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.101 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.101 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:55.101 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080746+0000", 2020-08-07T15:08:55.101 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012989499999999999 2020-08-07T15:08:55.101 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.102 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.102 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:55.102 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080876+0000", 2020-08-07T15:08:55.103 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.402718152 2020-08-07T15:08:55.103 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.103 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:55.103 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.103 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.104 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:55.104 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:55.106 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:08:55.107 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:54.370+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:55.108 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:55.104+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:55.207 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:55.206+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:55.260 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.4 to 1 (10000) 2020-08-07T15:08:55.273 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 5 1 2020-08-07T15:08:55.274 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 29 16 37072 37056 9.98192 0.015625 0.540352 0.012097 2020-08-07T15:08:55.296 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:55.297 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:08:55.297 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:08:55.298 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:55.305 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:08:55.346 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:55.345+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:55.394 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:08:55.396 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:08:55.409 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:55.408+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:55.508 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:55.508 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:08:55.509 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:08:55.509 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:08:55.509 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:55.534 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:08:55.536 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:55.535+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:55.565 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:08:55.640 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:55.639+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:55.659 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:08:55.659 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:08:55.659 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:08:55.660 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:08:55.660 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.660 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:8112 3.9 e24/20 3:9b275d48:::benchmark_data_smithi074_35447_object1013:head v 24'416, mlcod=24'408)", 2020-08-07T15:08:55.660 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:41.085042+0000", 2020-08-07T15:08:55.660 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 134.571580684, 2020-08-07T15:08:55.661 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31443637000000002, 2020-08-07T15:08:55.661 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:55.661 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:55.661 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:55.661 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.662 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:55.662 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085042+0000", 2020-08-07T15:08:55.662 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:55.662 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.662 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.663 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:55.663 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085042+0000", 2020-08-07T15:08:55.663 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4534000000000003e-05 2020-08-07T15:08:55.663 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.663 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.663 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:55.664 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085096+0000", 2020-08-07T15:08:55.664 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5789999999999998e-06 2020-08-07T15:08:55.664 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.664 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:55.665 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.085104+0000", 2020-08-07T15:08:55.665 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31316244700000001 2020-08-07T15:08:55.665 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.665 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.665 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:55.666 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398266+0000", 2020-08-07T15:08:55.666 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3019999999999996e-06 2020-08-07T15:08:55.666 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.666 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.666 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:55.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398270+0000", 2020-08-07T15:08:55.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3467000000000002e-05 2020-08-07T15:08:55.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:55.668 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398314+0000", 2020-08-07T15:08:55.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8184000000000001e-05 2020-08-07T15:08:55.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:55.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.398332+0000", 2020-08-07T15:08:55.670 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010908059999999999 2020-08-07T15:08:55.670 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.670 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.670 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:55.670 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.399423+0000", 2020-08-07T15:08:55.671 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5050999999999999e-05 2020-08-07T15:08:55.671 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.671 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.671 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:55.671 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:41.399478+0000", 2020-08-07T15:08:55.672 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31443637000000002 2020-08-07T15:08:55.672 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.672 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:55.672 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.672 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.673 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.673 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:16223 3.9 e27/20 3:9c72cb85:::benchmark_data_smithi074_35447_object2027:head v 27'887, mlcod=27'881)", 2020-08-07T15:08:55.673 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.111666+0000", 2020-08-07T15:08:55.673 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 128.544955832, 2020-08-07T15:08:55.673 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.48138790999999997, 2020-08-07T15:08:55.674 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:55.674 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:55.674 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:55.674 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.674 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:55.675 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111666+0000", 2020-08-07T15:08:55.675 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:55.675 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.675 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.675 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:55.675 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111666+0000", 2020-08-07T15:08:55.676 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3168999999999999e-05 2020-08-07T15:08:55.676 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.676 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.676 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:55.676 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111720+0000", 2020-08-07T15:08:55.677 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6309999999999997e-06 2020-08-07T15:08:55.677 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.688 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:55.689 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.111726+0000", 2020-08-07T15:08:55.689 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.47995092 2020-08-07T15:08:55.689 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.689 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.690 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:55.690 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591677+0000", 2020-08-07T15:08:55.690 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9879999999999998e-06 2020-08-07T15:08:55.690 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.690 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.691 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:55.691 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591684+0000", 2020-08-07T15:08:55.691 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5562000000000002e-05 2020-08-07T15:08:55.691 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.691 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.699 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:55.699 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591730+0000", 2020-08-07T15:08:55.699 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4731000000000001e-05 2020-08-07T15:08:55.700 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.700 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:55.700 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.591754+0000", 2020-08-07T15:08:55.700 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012412739999999999 2020-08-07T15:08:55.701 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.701 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:55.701 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.592996+0000", 2020-08-07T15:08:55.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8634999999999997e-05 2020-08-07T15:08:55.702 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.702 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.702 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:55.702 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.593054+0000", 2020-08-07T15:08:55.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.48138790999999997 2020-08-07T15:08:55.703 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.703 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:55.703 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.703 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.704 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:16224 3.9 e27/20 3:9c72cb85:::benchmark_data_smithi074_35447_object2027:head v 27'888, mlcod=27'881)", 2020-08-07T15:08:55.704 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:47.114073+0000", 2020-08-07T15:08:55.704 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 128.54254910500001, 2020-08-07T15:08:55.705 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.50478892600000003, 2020-08-07T15:08:55.705 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:55.705 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:55.705 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:55.705 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.706 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:55.706 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114073+0000", 2020-08-07T15:08:55.706 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:55.706 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.707 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:55.707 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114073+0000", 2020-08-07T15:08:55.707 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3088999999999999e-05 2020-08-07T15:08:55.707 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.708 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:55.708 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114126+0000", 2020-08-07T15:08:55.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1629999999999999e-06 2020-08-07T15:08:55.709 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.717 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.718 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:55.718 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.114133+0000", 2020-08-07T15:08:55.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.50357425199999994 2020-08-07T15:08:55.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:55.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617708+0000", 2020-08-07T15:08:55.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9160000000000004e-06 2020-08-07T15:08:55.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:55.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617715+0000", 2020-08-07T15:08:55.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8770999999999998e-05 2020-08-07T15:08:55.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.721 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:55.722 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617763+0000", 2020-08-07T15:08:55.722 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.544e-05 2020-08-07T15:08:55.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.722 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:55.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.617789+0000", 2020-08-07T15:08:55.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010210359999999999 2020-08-07T15:08:55.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.723 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:55.724 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.618810+0000", 2020-08-07T15:08:55.724 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2259000000000002e-05 2020-08-07T15:08:55.724 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.724 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.724 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:55.725 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.618862+0000", 2020-08-07T15:08:55.725 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.50478892600000003 2020-08-07T15:08:55.725 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.725 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:55.725 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.726 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.726 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.726 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:34624 3.9 e30/20 3:946cdb13:::benchmark_data_smithi074_35447_object4327:head v 30'2016, mlcod=30'2009)", 2020-08-07T15:08:55.726 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:58.746375+0000", 2020-08-07T15:08:55.726 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 116.910247461, 2020-08-07T15:08:55.726 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28564679799999998, 2020-08-07T15:08:55.727 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:55.727 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:55.727 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:55.727 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.727 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:55.728 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746375+0000", 2020-08-07T15:08:55.728 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:55.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:55.729 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746375+0000", 2020-08-07T15:08:55.741 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3674000000000001e-05 2020-08-07T15:08:55.742 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.742 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.742 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:55.742 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746429+0000", 2020-08-07T15:08:55.743 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9349999999999997e-06 2020-08-07T15:08:55.743 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.743 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:55.743 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:58.746435+0000", 2020-08-07T15:08:55.744 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28165817700000001 2020-08-07T15:08:55.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:55.744 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028094+0000", 2020-08-07T15:08:55.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.512e-06 2020-08-07T15:08:55.745 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.745 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:55.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028097+0000", 2020-08-07T15:08:55.746 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7904999999999997e-05 2020-08-07T15:08:55.746 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.746 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:55.746 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028135+0000", 2020-08-07T15:08:55.747 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6243e-05 2020-08-07T15:08:55.747 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.747 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.747 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:55.747 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.028151+0000", 2020-08-07T15:08:55.747 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003818372 2020-08-07T15:08:55.748 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.748 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.748 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:55.748 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.031970+0000", 2020-08-07T15:08:55.748 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.198e-05 2020-08-07T15:08:55.749 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.749 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.749 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:55.759 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:59.032022+0000", 2020-08-07T15:08:55.759 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28564679799999998 2020-08-07T15:08:55.759 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.760 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:55.760 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.760 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.760 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.760 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:104111 3.9 e58/20 3:99770d45:::benchmark_data_smithi074_35447_object13013:head v 58'6207, mlcod=58'6197)", 2020-08-07T15:08:55.761 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:45.861256+0000", 2020-08-07T15:08:55.761 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 69.795366244999997, 2020-08-07T15:08:55.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29017704, 2020-08-07T15:08:55.761 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:55.761 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:55.770 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:55.770 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.770 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:55.770 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861256+0000", 2020-08-07T15:08:55.770 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:55.771 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.771 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.771 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:55.771 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861256+0000", 2020-08-07T15:08:55.771 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013044700000000001 2020-08-07T15:08:55.772 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.772 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.772 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:55.773 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861387+0000", 2020-08-07T15:08:55.773 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7533000000000001e-05 2020-08-07T15:08:55.774 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.774 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.774 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:55.774 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.861424+0000", 2020-08-07T15:08:55.775 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28788376900000001 2020-08-07T15:08:55.775 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.775 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:55.775 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.149308+0000", 2020-08-07T15:08:55.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1890000000000004e-06 2020-08-07T15:08:55.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:55.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.149314+0000", 2020-08-07T15:08:55.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8288999999999999e-05 2020-08-07T15:08:55.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:55.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.149352+0000", 2020-08-07T15:08:55.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6765999999999998e-05 2020-08-07T15:08:55.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:55.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.149369+0000", 2020-08-07T15:08:55.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0020281000000000001 2020-08-07T15:08:55.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.779 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:55.797 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.151397+0000", 2020-08-07T15:08:55.798 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5947e-05 2020-08-07T15:08:55.798 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:55.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.151433+0000", 2020-08-07T15:08:55.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29017704 2020-08-07T15:08:55.800 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.800 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:55.800 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.832 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:104112 3.9 e58/20 3:99770d45:::benchmark_data_smithi074_35447_object13013:head v 58'6208, mlcod=58'6198)", 2020-08-07T15:08:55.833 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:45.862801+0000", 2020-08-07T15:08:55.833 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 69.793821062999996, 2020-08-07T15:08:55.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29018473099999997, 2020-08-07T15:08:55.833 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:55.833 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:55.834 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:55.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:55.870 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.862801+0000", 2020-08-07T15:08:55.871 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:55.871 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.871 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:55.871 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.862801+0000", 2020-08-07T15:08:55.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0234e-05 2020-08-07T15:08:55.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.872 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.872 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:55.872 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.862851+0000", 2020-08-07T15:08:55.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2700000000000001e-06 2020-08-07T15:08:55.873 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.898 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:55.899 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:45.862858+0000", 2020-08-07T15:08:55.899 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28821313100000001 2020-08-07T15:08:55.899 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.899 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.900 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:55.900 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.151071+0000", 2020-08-07T15:08:55.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4140000000000002e-06 2020-08-07T15:08:55.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.901 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:55.901 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.151076+0000", 2020-08-07T15:08:55.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7945999999999999e-05 2020-08-07T15:08:55.901 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.901 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:55.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.151114+0000", 2020-08-07T15:08:55.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2900000000000001e-05 2020-08-07T15:08:55.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:55.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.151137+0000", 2020-08-07T15:08:55.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001801415 2020-08-07T15:08:55.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:55.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.152939+0000", 2020-08-07T15:08:55.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7420999999999999e-05 2020-08-07T15:08:55.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:55.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.152986+0000", 2020-08-07T15:08:55.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29018473099999997 2020-08-07T15:08:55.925 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.925 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:55.926 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:55.926 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.926 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106080 3.4 3:208b1d5b:::benchmark_data_smithi074_35447_object13259:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:08:55.926 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.241181+0000", 2020-08-07T15:08:55.927 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 68.415441615000006, 2020-08-07T15:08:55.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45274565500000002, 2020-08-07T15:08:55.939 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:55.940 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:55.940 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:55.941 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:55.941 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:55.941 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106080 2020-08-07T15:08:55.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.945 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:55.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:55.946 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.241181+0000", 2020-08-07T15:08:55.946 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:55.946 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.947 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.948 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:55.951 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.241181+0000", 2020-08-07T15:08:55.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3634999999999998e-05 2020-08-07T15:08:55.952 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.952 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.952 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:55.953 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.241224+0000", 2020-08-07T15:08:55.953 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2310000000000001e-06 2020-08-07T15:08:55.953 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:55.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.241228+0000", 2020-08-07T15:08:55.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7490000000000001e-06 2020-08-07T15:08:55.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:55.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.241234+0000", 2020-08-07T15:08:55.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1320000000000004e-06 2020-08-07T15:08:55.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:55.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.241238+0000", 2020-08-07T15:08:55.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001563659 2020-08-07T15:08:55.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.986 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:55.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.242802+0000", 2020-08-07T15:08:55.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0303999999999995e-05 2020-08-07T15:08:55.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:55.990 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:55.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.242892+0000", 2020-08-07T15:08:55.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013412 2020-08-07T15:08:55.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:55.998 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:08:56.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.243027+0000", 2020-08-07T15:08:56.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001917291 2020-08-07T15:08:56.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:56.004 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.244944+0000", 2020-08-07T15:08:56.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9647999999999995e-05 2020-08-07T15:08:56.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:56.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.245013+0000", 2020-08-07T15:08:56.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.44882515299999998 2020-08-07T15:08:56.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:56.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693839+0000", 2020-08-07T15:08:56.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5290000000000001e-06 2020-08-07T15:08:56.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:56.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693840+0000", 2020-08-07T15:08:56.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7976999999999998e-05 2020-08-07T15:08:56.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:56.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693888+0000", 2020-08-07T15:08:56.008 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8226999999999998e-05 2020-08-07T15:08:56.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.009 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:56.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693926+0000", 2020-08-07T15:08:56.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45274565500000002 2020-08-07T15:08:56.009 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:56.009 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:56.010 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:56.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.010 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106080 3.4 e58/20 ondisk, result = 0)", 2020-08-07T15:08:56.010 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.255977+0000", 2020-08-07T15:08:56.010 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 68.400644994000004, 2020-08-07T15:08:56.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43795860599999997, 2020-08-07T15:08:56.011 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:56.011 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:56.011 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:56.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:56.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255977+0000", 2020-08-07T15:08:56.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:56.058 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.058 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.058 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:56.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255977+0000", 2020-08-07T15:08:56.059 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2820000000000001e-05 2020-08-07T15:08:56.059 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.059 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.059 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:56.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.256010+0000", 2020-08-07T15:08:56.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8660000000000002e-06 2020-08-07T15:08:56.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:56.060 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.256016+0000", 2020-08-07T15:08:56.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.437661209 2020-08-07T15:08:56.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.061 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:56.079 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693677+0000", 2020-08-07T15:08:56.079 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6000000000000003e-06 2020-08-07T15:08:56.080 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.080 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.080 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:56.080 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693684+0000", 2020-08-07T15:08:56.080 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000138572 2020-08-07T15:08:56.081 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.081 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.081 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:56.081 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693822+0000", 2020-08-07T15:08:56.081 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6339999999999996e-06 2020-08-07T15:08:56.082 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.082 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:56.082 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693832+0000", 2020-08-07T15:08:56.082 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000103905 2020-08-07T15:08:56.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.083 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.083 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:56.083 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693936+0000", 2020-08-07T15:08:56.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43795860599999997 2020-08-07T15:08:56.084 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:56.084 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:56.084 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:56.084 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.084 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.085 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:116624 3.9 e65/20 3:93e94c7d:::benchmark_data_smithi074_35447_object14577:head v 65'7112, mlcod=65'7103)", 2020-08-07T15:08:56.085 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:56.781405+0000", 2020-08-07T15:08:56.085 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 58.875217186, 2020-08-07T15:08:56.085 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.248173752, 2020-08-07T15:08:56.085 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:56.121 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:56.122 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:56.122 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.122 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:56.122 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.781405+0000", 2020-08-07T15:08:56.123 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:56.123 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.124 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.124 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:56.124 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.781405+0000", 2020-08-07T15:08:56.125 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3173999999999998e-05 2020-08-07T15:08:56.135 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.137 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.137 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:56.138 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.781438+0000", 2020-08-07T15:08:56.138 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1790000000000001e-06 2020-08-07T15:08:56.138 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.138 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.139 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:56.174 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.781442+0000", 2020-08-07T15:08:56.182 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24601534999999999 2020-08-07T15:08:56.182 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.235 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.235 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:56.235 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.027458+0000", 2020-08-07T15:08:56.236 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8599999999999998e-06 2020-08-07T15:08:56.236 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.236 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.237 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:56.237 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.027464+0000", 2020-08-07T15:08:56.237 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3399000000000003e-05 2020-08-07T15:08:56.241 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.241 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.241 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:56.241 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.027497+0000", 2020-08-07T15:08:56.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6073999999999999e-05 2020-08-07T15:08:56.253 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.253 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:56.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.027513+0000", 2020-08-07T15:08:56.254 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0020137979999999998 2020-08-07T15:08:56.255 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.255 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.257 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:56.257 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.029527+0000", 2020-08-07T15:08:56.257 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1918e-05 2020-08-07T15:08:56.257 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.257 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.261 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:56.261 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.029579+0000", 2020-08-07T15:08:56.262 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.248173752 2020-08-07T15:08:56.262 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:56.262 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:56.262 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:56.262 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.263 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.264 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:117016 3.4 3:2f619d09:::benchmark_data_smithi074_35447_object14626:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e65)", 2020-08-07T15:08:56.264 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:57.118050+0000", 2020-08-07T15:08:56.264 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 58.538572125999998, 2020-08-07T15:08:56.282 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.255829902, 2020-08-07T15:08:56.283 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:56.283 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:56.283 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:56.283 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:08:56.284 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:08:56.284 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 117016 2020-08-07T15:08:56.284 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.285 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:56.286 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.286 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:56.286 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118050+0000", 2020-08-07T15:08:56.286 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:56.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.290 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:56.290 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118050+0000", 2020-08-07T15:08:56.290 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2957999999999999e-05 2020-08-07T15:08:56.290 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.291 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.291 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:56.292 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118083+0000", 2020-08-07T15:08:56.301 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8829999999999997e-06 2020-08-07T15:08:56.301 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.310 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.311 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:56.311 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118088+0000", 2020-08-07T15:08:56.311 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4660000000000005e-06 2020-08-07T15:08:56.312 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.312 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.312 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:56.313 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118098+0000", 2020-08-07T15:08:56.313 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.768e-06 2020-08-07T15:08:56.313 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.313 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.315 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:56.316 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118100+0000", 2020-08-07T15:08:56.317 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00054986900000000003 2020-08-07T15:08:56.317 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.317 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.318 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:56.319 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118650+0000", 2020-08-07T15:08:56.319 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7521e-05 2020-08-07T15:08:56.319 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.319 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.320 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:56.320 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118708+0000", 2020-08-07T15:08:56.320 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012445499999999999 2020-08-07T15:08:56.320 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.321 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.321 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,5", 2020-08-07T15:08:56.321 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.118832+0000", 2020-08-07T15:08:56.321 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002948806 2020-08-07T15:08:56.322 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.322 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.322 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:56.323 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.121781+0000", 2020-08-07T15:08:56.323 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8299999999999996e-07 2020-08-07T15:08:56.323 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.324 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.324 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:56.324 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.121782+0000", 2020-08-07T15:08:56.332 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4300000000000004e-05 2020-08-07T15:08:56.333 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.333 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.333 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:56.334 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.121846+0000", 2020-08-07T15:08:56.354 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.044326363000000001 2020-08-07T15:08:56.360 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.360 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.360 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:56.361 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.166172+0000", 2020-08-07T15:08:56.362 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20763515900000001 2020-08-07T15:08:56.362 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.362 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.362 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:56.362 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.373808+0000", 2020-08-07T15:08:56.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6339999999999999e-06 2020-08-07T15:08:56.363 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.363 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.363 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:56.363 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.373809+0000", 2020-08-07T15:08:56.364 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0937000000000002e-05 2020-08-07T15:08:56.364 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:56.364 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:57.373880+0000", 2020-08-07T15:08:56.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.255829902 2020-08-07T15:08:56.365 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:56.365 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:56.365 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:56.365 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.366 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:9111 4.6 4:6004be0b:::benchmark_data_smithi074_44586_object1138:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e86)", 2020-08-07T15:08:56.366 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:32.294153+0000", 2020-08-07T15:08:56.366 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 23.362469432000001, 2020-08-07T15:08:56.367 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26066347899999998, 2020-08-07T15:08:56.367 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:56.367 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:56.367 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:56.368 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:08:56.368 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:56.368 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 9111 2020-08-07T15:08:56.368 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.369 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:56.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.369 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:56.370 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.294153+0000", 2020-08-07T15:08:56.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:56.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.383 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:56.383 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.294153+0000", 2020-08-07T15:08:56.383 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5032e-05 2020-08-07T15:08:56.384 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.384 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.384 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:56.384 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.294178+0000", 2020-08-07T15:08:56.384 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9830000000000001e-06 2020-08-07T15:08:56.385 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.385 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.385 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:56.385 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.294181+0000", 2020-08-07T15:08:56.385 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5319999999999997e-06 2020-08-07T15:08:56.386 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.386 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.386 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:56.386 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.294186+0000", 2020-08-07T15:08:56.386 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7209999999999999e-06 2020-08-07T15:08:56.399 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.400 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:56.400 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.294189+0000", 2020-08-07T15:08:56.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000974397 2020-08-07T15:08:56.407 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.407 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.408 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:56.408 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.295164+0000", 2020-08-07T15:08:56.412 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000134375 2020-08-07T15:08:56.412 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.412 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.418 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:56.419 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.295298+0000", 2020-08-07T15:08:56.419 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017785000000000001 2020-08-07T15:08:56.420 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.420 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.420 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:08:56.421 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.295476+0000", 2020-08-07T15:08:56.421 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0029282259999999999 2020-08-07T15:08:56.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.421 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.421 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:56.422 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.298404+0000", 2020-08-07T15:08:56.422 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5199999999999996e-07 2020-08-07T15:08:56.422 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.422 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:56.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.298405+0000", 2020-08-07T15:08:56.430 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7341e-05 2020-08-07T15:08:56.430 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.431 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:56.431 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.298452+0000", 2020-08-07T15:08:56.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004868079 2020-08-07T15:08:56.432 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.432 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.432 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:56.432 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.303320+0000", 2020-08-07T15:08:56.432 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25142435099999999 2020-08-07T15:08:56.433 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.433 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.433 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:56.433 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.554744+0000", 2020-08-07T15:08:56.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3459999999999999e-06 2020-08-07T15:08:56.451 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.451 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:56.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.554746+0000", 2020-08-07T15:08:56.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0593999999999995e-05 2020-08-07T15:08:56.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.453 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:56.453 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.554816+0000", 2020-08-07T15:08:56.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26066347899999998 2020-08-07T15:08:56.453 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:56.454 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:56.454 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:56.454 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.454 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.454 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:9112 4.6 4:6004be0b:::benchmark_data_smithi074_44586_object1138:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e86)", 2020-08-07T15:08:56.455 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:32.294206+0000", 2020-08-07T15:08:56.455 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 23.362416413999998, 2020-08-07T15:08:56.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26582170300000002, 2020-08-07T15:08:56.455 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:56.455 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:56.455 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:56.456 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:08:56.456 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:56.456 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 9112 2020-08-07T15:08:56.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.456 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:56.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:56.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.294206+0000", 2020-08-07T15:08:56.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:56.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.464 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.465 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:56.465 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.294206+0000", 2020-08-07T15:08:56.465 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1359999999999998e-05 2020-08-07T15:08:56.465 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.465 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.467 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:56.467 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.294237+0000", 2020-08-07T15:08:56.467 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9879999999999998e-06 2020-08-07T15:08:56.467 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.468 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.468 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:56.468 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.294241+0000", 2020-08-07T15:08:56.468 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9900000000000002e-06 2020-08-07T15:08:56.468 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.469 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.469 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:56.469 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.294247+0000", 2020-08-07T15:08:56.470 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9079999999999999e-06 2020-08-07T15:08:56.470 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.470 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.471 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:56.471 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.294250+0000", 2020-08-07T15:08:56.471 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001905871 2020-08-07T15:08:56.471 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.471 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.472 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:56.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296156+0000", 2020-08-07T15:08:56.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8465e-05 2020-08-07T15:08:56.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.490 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.490 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:56.490 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296254+0000", 2020-08-07T15:08:56.490 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018382500000000001 2020-08-07T15:08:56.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.491 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.491 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:08:56.491 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296438+0000", 2020-08-07T15:08:56.491 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002019819 2020-08-07T15:08:56.500 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.500 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.500 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:56.500 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.298458+0000", 2020-08-07T15:08:56.500 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2500000000000001e-07 2020-08-07T15:08:56.501 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.501 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.501 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:56.501 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.298458+0000", 2020-08-07T15:08:56.501 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2608999999999997e-05 2020-08-07T15:08:56.502 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.502 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.502 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:56.510 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.298511+0000", 2020-08-07T15:08:56.511 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0048420349999999997 2020-08-07T15:08:56.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.512 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.512 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:56.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.303353+0000", 2020-08-07T15:08:56.513 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25663897600000002 2020-08-07T15:08:56.513 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.514 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:56.514 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.559992+0000", 2020-08-07T15:08:56.514 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.92e-07 2020-08-07T15:08:56.515 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.515 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.515 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:56.515 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.559993+0000", 2020-08-07T15:08:56.516 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4740000000000003e-05 2020-08-07T15:08:56.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.528 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:56.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.560028+0000", 2020-08-07T15:08:56.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26582170300000002 2020-08-07T15:08:56.529 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:56.530 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:56.530 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:56.530 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.531 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.531 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:9111 4.6 e86/81 ondisk, result = 0)", 2020-08-07T15:08:56.531 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:32.298867+0000", 2020-08-07T15:08:56.532 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 23.35775516, 2020-08-07T15:08:56.532 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25595633200000001, 2020-08-07T15:08:56.532 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:56.533 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:56.533 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:56.533 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.548 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:56.549 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.298867+0000", 2020-08-07T15:08:56.549 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:56.550 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.550 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.550 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:56.551 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.298867+0000", 2020-08-07T15:08:56.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4514e-05 2020-08-07T15:08:56.551 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.552 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:56.552 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.298882+0000", 2020-08-07T15:08:56.552 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3039999999999999e-06 2020-08-07T15:08:56.559 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.566 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.566 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:56.566 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.298884+0000", 2020-08-07T15:08:56.567 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25579149899999998 2020-08-07T15:08:56.567 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.567 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.567 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:56.567 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.554675+0000", 2020-08-07T15:08:56.568 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7690000000000002e-06 2020-08-07T15:08:56.568 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.568 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.568 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:56.568 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.554681+0000", 2020-08-07T15:08:56.569 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3312999999999997e-05 2020-08-07T15:08:56.569 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.572 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.572 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:56.572 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.554725+0000", 2020-08-07T15:08:56.573 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0437e-05 2020-08-07T15:08:56.573 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.573 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.573 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:56.573 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.554735+0000", 2020-08-07T15:08:56.574 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8496000000000001e-05 2020-08-07T15:08:56.574 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.574 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.574 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:56.574 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.554823+0000", 2020-08-07T15:08:56.575 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25595633200000001 2020-08-07T15:08:56.575 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:56.575 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:56.575 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:56.575 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.576 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:25792 4.4 4:2d04ac8e:::benchmark_data_smithi074_44586_object3223:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e96)", 2020-08-07T15:08:56.576 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:42.365230+0000", 2020-08-07T15:08:56.576 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 13.29139202, 2020-08-07T15:08:56.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25180039700000001, 2020-08-07T15:08:56.576 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:56.577 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:56.577 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:56.577 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:08:56.577 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:56.587 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 25792 2020-08-07T15:08:56.587 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.604 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:56.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:56.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.365230+0000", 2020-08-07T15:08:56.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:56.605 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.605 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.605 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:56.605 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.365230+0000", 2020-08-07T15:08:56.605 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5354000000000002e-05 2020-08-07T15:08:56.607 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.607 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.607 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:56.607 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.365256+0000", 2020-08-07T15:08:56.607 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.605e-06 2020-08-07T15:08:56.608 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.608 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.612 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:56.614 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.365258+0000", 2020-08-07T15:08:56.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8200000000000002e-06 2020-08-07T15:08:56.615 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.615 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.616 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:56.616 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.365264+0000", 2020-08-07T15:08:56.617 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1019999999999999e-06 2020-08-07T15:08:56.624 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.625 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.641 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:56.641 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.365266+0000", 2020-08-07T15:08:56.641 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002530577 2020-08-07T15:08:56.642 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.642 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.642 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:56.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.367797+0000", 2020-08-07T15:08:56.642 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3579999999999998e-05 2020-08-07T15:08:56.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.643 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.643 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:56.643 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.367860+0000", 2020-08-07T15:08:56.643 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000106155 2020-08-07T15:08:56.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.644 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.644 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:08:56.644 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.367966+0000", 2020-08-07T15:08:56.644 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0040722500000000003 2020-08-07T15:08:56.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.654 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:56.654 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.372039+0000", 2020-08-07T15:08:56.654 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5700000000000002e-07 2020-08-07T15:08:56.655 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.655 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.655 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:56.655 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.372039+0000", 2020-08-07T15:08:56.656 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3897999999999998e-05 2020-08-07T15:08:56.657 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.657 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.657 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:56.657 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.372093+0000", 2020-08-07T15:08:56.658 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0029951140000000001 2020-08-07T15:08:56.658 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.658 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.658 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:56.658 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.375088+0000", 2020-08-07T15:08:56.668 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24190147400000001 2020-08-07T15:08:56.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:56.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.616990+0000", 2020-08-07T15:08:56.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7000000000000004e-07 2020-08-07T15:08:56.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.670 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:56.680 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.616990+0000", 2020-08-07T15:08:56.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0241000000000003e-05 2020-08-07T15:08:56.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:56.681 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:42.617031+0000", 2020-08-07T15:08:56.681 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25180039700000001 2020-08-07T15:08:56.681 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:56.681 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:56.682 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:56.682 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.682 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.682 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:30968 4.4 e98/81 ondisk, result = 0)", 2020-08-07T15:08:56.682 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:08:56.683 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 9.9268340150000007, 2020-08-07T15:08:56.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96211418999999998, 2020-08-07T15:08:56.683 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:56.683 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:56.683 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:56.684 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.684 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:56.684 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:08:56.694 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:56.694 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.694 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.694 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:56.695 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:08:56.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2387e-05 2020-08-07T15:08:56.695 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.695 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:56.696 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729821+0000", 2020-08-07T15:08:56.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9909999999999999e-06 2020-08-07T15:08:56.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.696 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:56.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729827+0000", 2020-08-07T15:08:56.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96185524600000005 2020-08-07T15:08:56.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.718 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.718 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:56.718 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691682+0000", 2020-08-07T15:08:56.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8760000000000005e-06 2020-08-07T15:08:56.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:56.743 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691690+0000", 2020-08-07T15:08:56.744 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9682000000000001e-05 2020-08-07T15:08:56.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:56.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691759+0000", 2020-08-07T15:08:56.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5381000000000001e-05 2020-08-07T15:08:56.745 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.745 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:56.746 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691775+0000", 2020-08-07T15:08:56.746 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000127627 2020-08-07T15:08:56.746 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.746 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:56.747 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691902+0000", 2020-08-07T15:08:56.747 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96211418999999998 2020-08-07T15:08:56.747 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:56.747 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:56.747 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:56.748 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.748 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.773 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:36864 4.4 4:259d2907:::benchmark_data_smithi074_44586_object4607:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e106)", 2020-08-07T15:08:56.774 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.133347+0000", 2020-08-07T15:08:56.774 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 1.5232752970000001, 2020-08-07T15:08:56.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.62052594100000003, 2020-08-07T15:08:56.782 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:56.783 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:56.783 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:56.783 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:08:56.783 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:56.783 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 36864 2020-08-07T15:08:56.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.784 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:56.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:56.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.133347+0000", 2020-08-07T15:08:56.785 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:56.785 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:56.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.133347+0000", 2020-08-07T15:08:56.794 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4625000000000001e-05 2020-08-07T15:08:56.794 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.794 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.795 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:56.795 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.133382+0000", 2020-08-07T15:08:56.795 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2320000000000001e-06 2020-08-07T15:08:56.795 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.795 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.796 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:56.796 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.133387+0000", 2020-08-07T15:08:56.796 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0509999999999999e-05 2020-08-07T15:08:56.810 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.810 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.810 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:56.810 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.133397+0000", 2020-08-07T15:08:56.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1300000000000001e-06 2020-08-07T15:08:56.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.811 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:56.811 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.133401+0000", 2020-08-07T15:08:56.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0085783109999999999 2020-08-07T15:08:56.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:56.822 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.141979+0000", 2020-08-07T15:08:56.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000104498 2020-08-07T15:08:56.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.822 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:56.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.142083+0000", 2020-08-07T15:08:56.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00021400999999999999 2020-08-07T15:08:56.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:08:56.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.142297+0000", 2020-08-07T15:08:56.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023026520000000001 2020-08-07T15:08:56.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:56.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.144600+0000", 2020-08-07T15:08:56.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.142e-06 2020-08-07T15:08:56.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:08:56.862 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.144601+0000", 2020-08-07T15:08:56.863 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3246999999999999e-05 2020-08-07T15:08:56.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:56.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.144674+0000", 2020-08-07T15:08:56.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.032141511999999997 2020-08-07T15:08:56.864 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:56.865 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.176816+0000", 2020-08-07T15:08:56.865 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.57700459699999995 2020-08-07T15:08:56.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.865 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:56.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.753820+0000", 2020-08-07T15:08:56.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.254e-06 2020-08-07T15:08:56.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.886 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:56.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.753822+0000", 2020-08-07T15:08:56.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1221000000000003e-05 2020-08-07T15:08:56.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:56.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.753873+0000", 2020-08-07T15:08:56.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.62052594100000003 2020-08-07T15:08:56.888 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:56.889 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:56.889 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:56.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.889 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:36935 4.d 4:bc941c2c:::benchmark_data_smithi074_44586_object4616:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e106)", 2020-08-07T15:08:56.922 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.169304+0000", 2020-08-07T15:08:56.922 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 1.4873180399999999, 2020-08-07T15:08:56.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39166248300000001, 2020-08-07T15:08:56.923 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:56.927 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:56.928 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:56.928 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:08:56.928 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:56.928 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 36935 2020-08-07T15:08:56.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.929 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:56.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:56.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.169304+0000", 2020-08-07T15:08:56.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:56.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:56.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.169304+0000", 2020-08-07T15:08:56.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7366e-05 2020-08-07T15:08:56.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:56.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.169352+0000", 2020-08-07T15:08:56.931 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1619999999999999e-06 2020-08-07T15:08:56.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.932 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.932 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:56.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.169357+0000", 2020-08-07T15:08:56.932 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0827e-05 2020-08-07T15:08:56.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.933 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.933 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:56.933 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.169368+0000", 2020-08-07T15:08:56.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9439999999999996e-06 2020-08-07T15:08:56.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:56.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.169373+0000", 2020-08-07T15:08:56.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023484389999999999 2020-08-07T15:08:56.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:56.964 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.171721+0000", 2020-08-07T15:08:56.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6893999999999998e-05 2020-08-07T15:08:56.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:56.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.171808+0000", 2020-08-07T15:08:56.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010190400000000001 2020-08-07T15:08:56.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:56.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.171910+0000", 2020-08-07T15:08:56.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001763677 2020-08-07T15:08:56.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:56.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.173673+0000", 2020-08-07T15:08:56.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015711849999999999 2020-08-07T15:08:56.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.975 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:56.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.175245+0000", 2020-08-07T15:08:56.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.38554944299999999 2020-08-07T15:08:56.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:56.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.560794+0000", 2020-08-07T15:08:56.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8560000000000001e-06 2020-08-07T15:08:56.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:56.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.560796+0000", 2020-08-07T15:08:56.991 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2744999999999994e-05 2020-08-07T15:08:56.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:56.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.560889+0000", 2020-08-07T15:08:56.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8040999999999994e-05 2020-08-07T15:08:56.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:56.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:56.993 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:57.001 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.560967+0000", 2020-08-07T15:08:57.001 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39166248300000001 2020-08-07T15:08:57.001 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:57.002 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:57.002 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:57.002 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.006 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:36936 4.d 4:bc941c2c:::benchmark_data_smithi074_44586_object4616:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e106)", 2020-08-07T15:08:57.006 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.169407+0000", 2020-08-07T15:08:57.007 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 1.4872148300000001, 2020-08-07T15:08:57.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.54015153000000005, 2020-08-07T15:08:57.007 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:57.007 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:57.007 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:08:57.008 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:08:57.008 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:08:57.008 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 36936 2020-08-07T15:08:57.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.008 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:57.009 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.009 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:57.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.169407+0000", 2020-08-07T15:08:57.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:57.009 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:57.022 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.169407+0000", 2020-08-07T15:08:57.022 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6887e-05 2020-08-07T15:08:57.023 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.023 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:57.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.169454+0000", 2020-08-07T15:08:57.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5119999999999999e-06 2020-08-07T15:08:57.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:57.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.169460+0000", 2020-08-07T15:08:57.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1277000000000001e-05 2020-08-07T15:08:57.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:57.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.169471+0000", 2020-08-07T15:08:57.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.541e-06 2020-08-07T15:08:57.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:57.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.169477+0000", 2020-08-07T15:08:57.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026785960000000001 2020-08-07T15:08:57.030 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.030 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.030 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:57.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.172155+0000", 2020-08-07T15:08:57.030 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000118763 2020-08-07T15:08:57.030 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:57.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.172274+0000", 2020-08-07T15:08:57.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000158138 2020-08-07T15:08:57.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.041 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:08:57.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.172432+0000", 2020-08-07T15:08:57.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002374176 2020-08-07T15:08:57.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.042 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:57.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.174806+0000", 2020-08-07T15:08:57.042 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00044415699999999998 2020-08-07T15:08:57.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:08:57.043 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.175251+0000", 2020-08-07T15:08:57.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.53422133800000005 2020-08-07T15:08:57.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.044 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:08:57.044 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.709472+0000", 2020-08-07T15:08:57.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.252e-06 2020-08-07T15:08:57.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.045 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:08:57.045 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.709473+0000", 2020-08-07T15:08:57.045 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0747e-05 2020-08-07T15:08:57.045 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.048 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.048 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:08:57.048 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.709524+0000", 2020-08-07T15:08:57.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5145999999999999e-05 2020-08-07T15:08:57.049 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.049 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.049 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:57.049 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.709559+0000", 2020-08-07T15:08:57.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.54015153000000005 2020-08-07T15:08:57.050 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:57.050 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:57.051 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:57.051 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.053 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:36935 4.d e106/98 ondisk, result = 0)", 2020-08-07T15:08:57.053 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.176684+0000", 2020-08-07T15:08:57.054 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 1.479938456, 2020-08-07T15:08:57.054 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.38429434000000001, 2020-08-07T15:08:57.054 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:57.055 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:57.055 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:57.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:57.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.176684+0000", 2020-08-07T15:08:57.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:57.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.058 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:57.058 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.176684+0000", 2020-08-07T15:08:57.059 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0094999999999999e-05 2020-08-07T15:08:57.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.065 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:57.065 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.176714+0000", 2020-08-07T15:08:57.066 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4989999999999996e-06 2020-08-07T15:08:57.066 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.066 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.066 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:57.066 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.176718+0000", 2020-08-07T15:08:57.067 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.38398004099999999 2020-08-07T15:08:57.067 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.067 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.067 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:57.067 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.560698+0000", 2020-08-07T15:08:57.068 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0180000000000001e-06 2020-08-07T15:08:57.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.068 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.068 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:57.068 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.560707+0000", 2020-08-07T15:08:57.069 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0420000000000001e-05 2020-08-07T15:08:57.069 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.069 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.069 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:57.069 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.560767+0000", 2020-08-07T15:08:57.072 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4090000000000001e-05 2020-08-07T15:08:57.072 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.073 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.073 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:57.073 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.560781+0000", 2020-08-07T15:08:57.073 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019717700000000001 2020-08-07T15:08:57.082 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.083 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:57.083 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.560978+0000", 2020-08-07T15:08:57.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.38429434000000001 2020-08-07T15:08:57.083 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:57.083 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:57.084 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:57.084 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.084 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.094 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:36936 4.d e106/98 ondisk, result = 0)", 2020-08-07T15:08:57.094 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.202503+0000", 2020-08-07T15:08:57.095 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 1.4541188979999999, 2020-08-07T15:08:57.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.50706159900000003, 2020-08-07T15:08:57.096 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:08:57.096 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:08:57.102 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:08:57.102 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.103 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:08:57.103 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.202503+0000", 2020-08-07T15:08:57.103 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:08:57.103 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.103 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.104 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:08:57.104 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.202503+0000", 2020-08-07T15:08:57.104 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2388e-05 2020-08-07T15:08:57.104 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.104 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.105 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:08:57.114 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.202526+0000", 2020-08-07T15:08:57.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3749999999999999e-06 2020-08-07T15:08:57.115 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.115 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.115 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:08:57.115 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.202532+0000", 2020-08-07T15:08:57.116 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.50683397100000005 2020-08-07T15:08:57.116 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.116 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.116 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:08:57.116 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.709366+0000", 2020-08-07T15:08:57.117 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9830000000000001e-06 2020-08-07T15:08:57.117 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.117 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:08:57.117 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.709372+0000", 2020-08-07T15:08:57.118 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2395e-05 2020-08-07T15:08:57.118 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.118 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.135 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:08:57.136 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.709455+0000", 2020-08-07T15:08:57.136 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2969999999999992e-06 2020-08-07T15:08:57.136 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.136 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.136 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:08:57.137 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.709464+0000", 2020-08-07T15:08:57.137 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010119 2020-08-07T15:08:57.137 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:08:57.137 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:08:57.137 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:08:57.138 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.709565+0000", 2020-08-07T15:08:57.138 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.50706159900000003 2020-08-07T15:08:57.138 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:57.138 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:57.138 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:57.139 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:08:57.139 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:08:57.139 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:08:57.148 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:08:57.149 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:08:57.149 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:08:57.150 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:08:57.150 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:08:57.150 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:08:57.159 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:08:57.160 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:08:57.160 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:08:57.160 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:08:57.160 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:08:57.161 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:08:57.161 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:08:57.161 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:08:57.161 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:08:57.161 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:08:57.162 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.5 to 1 (10000) 2020-08-07T15:08:57.162 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:08:57.163 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 30 16 37072 37056 9.64919 0 - 0.012097 2020-08-07T15:08:57.163 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:57.139+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:57.163 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 6 1 2020-08-07T15:08:57.248 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:08:57.246+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:08:57.276 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 31 16 38293 38277 9.64561 4.76953 0.0175328 0.0129511 2020-08-07T15:08:57.350 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:57.348+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:57.393 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:57.553 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:57.554 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:08:57.563 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:57.554+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:57.655 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:57.654+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:57.755 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:57.755 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:57.756 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:08:57.759 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:57.757+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:57.868 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:57.864+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:57.967 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:57.966+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:57.968 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:57.968 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:57.981 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:08:58.074 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:58.073+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:58.138 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:58.139 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:58.154 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:08:58.179 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.6 to 1 (10000) 2020-08-07T15:08:58.193 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 7 1 2020-08-07T15:08:58.195 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:58.194+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:58.275 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 32 16 39781 39765 9.70744 11.625 0.00297887 0.0128689 2020-08-07T15:08:58.277 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:58.276+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:58.341 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:58.341 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:58.351 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:08:58.395 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:58.394+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:58.479 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:58.479+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:58.524 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:58.525 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:58.545 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:08:58.592 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:58.591+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:58.695 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:08:58.694+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:08:58.770 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:58.771 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:58.781 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:08:58.818 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:58.807+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:58.925 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:58.908+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:58.991 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:58.991 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:59.043 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:08:59.088 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:59.080+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:59.165 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:08:59.166 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:08:59.207 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:08:59.194+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:08:59.268 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:59.269 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:59.284 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:08:59.285 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 33 16 41095 41079 9.72418 10.2656 0.00757486 0.0128377 2020-08-07T15:08:59.286 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.7 to 1 (10000) 2020-08-07T15:08:59.308 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2020-08-07T15:08:59.309 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:08:59.346 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:08:59.315+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:08:59.437 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:08:59.438 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:08:59.431+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:08:59.483 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:08:59.506 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:59.506 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:59.513 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:08:59.515 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:08:59.515+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:08:59.540 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:08:59.571 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:08:59.621 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:59.616+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:59.622 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:08:59.670 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:08:59.671 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:08:59.671 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:08:59.683 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:08:59.722 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:08:59.719+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:08:59.724 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:08:59.771 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:08:59.782 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:08:59.782 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:08:59.782 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:08:59.783 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:08:59.783 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:08:59.783 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:08:59.783 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:08:59.783 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:08:59.828 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:08:59.824+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:08:59.874 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:08:59.874 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:08:59.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:08:59.875 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:08:59.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:59.875 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:42104 3.d e32/20 3:bcc35439:::benchmark_data_smithi074_35447_object5262:head v 32'2864, mlcod=32'2858)", 2020-08-07T15:08:59.875 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:08:59.876 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 116.615956105, 2020-08-07T15:08:59.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28296418899999998, 2020-08-07T15:08:59.876 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:08:59.876 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:08:59.876 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:08:59.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:59.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:08:59.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:08:59.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:08:59.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:59.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:59.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:08:59.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:08:59.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5550000000000002e-05 2020-08-07T15:08:59.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:59.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:59.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:08:59.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243081+0000", 2020-08-07T15:08:59.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8139999999999997e-06 2020-08-07T15:08:59.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:59.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:59.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:08:59.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243087+0000", 2020-08-07T15:08:59.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27968446800000002 2020-08-07T15:08:59.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:59.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:59.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:08:59.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522771+0000", 2020-08-07T15:08:59.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9459999999999997e-06 2020-08-07T15:08:59.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:59.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:59.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:08:59.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522776+0000", 2020-08-07T15:08:59.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0778000000000006e-05 2020-08-07T15:08:59.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:59.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:59.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:08:59.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522847+0000", 2020-08-07T15:08:59.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0622e-05 2020-08-07T15:08:59.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:59.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:59.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:08:59.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522868+0000", 2020-08-07T15:08:59.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030627580000000001 2020-08-07T15:08:59.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:59.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:59.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:08:59.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.525930+0000", 2020-08-07T15:08:59.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9252999999999997e-05 2020-08-07T15:08:59.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:08:59.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:08:59.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:08:59.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.525990+0000", 2020-08-07T15:08:59.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28296418899999998 2020-08-07T15:08:59.994 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:08:59.995 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:08:59.995 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:00.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.019 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:48776 3.2 e36/20 3:47029331:::benchmark_data_smithi074_35447_object6096:head v 36'2888, mlcod=36'2881)", 2020-08-07T15:09:00.019 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:09:00.019 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.45446158, 2020-08-07T15:09:00.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27006165399999998, 2020-08-07T15:09:00.020 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:00.020 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:00.020 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:00.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:00.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:09:00.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:00.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:00.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404520+0000", 2020-08-07T15:09:00.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8454000000000001e-05 2020-08-07T15:09:00.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:00.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404569+0000", 2020-08-07T15:09:00.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2349999999999996e-06 2020-08-07T15:09:00.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:00.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.404575+0000", 2020-08-07T15:09:00.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26921690999999998 2020-08-07T15:09:00.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:00.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673792+0000", 2020-08-07T15:09:00.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3330000000000002e-06 2020-08-07T15:09:00.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:00.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673798+0000", 2020-08-07T15:09:00.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1013000000000001e-05 2020-08-07T15:09:00.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:00.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673849+0000", 2020-08-07T15:09:00.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7044999999999999e-05 2020-08-07T15:09:00.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.673876+0000", 2020-08-07T15:09:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00065173100000000001 2020-08-07T15:09:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.674528+0000", 2020-08-07T15:09:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3933e-05 2020-08-07T15:09:00.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:00.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:09.674582+0000", 2020-08-07T15:09:00.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27006165399999998 2020-08-07T15:09:00.222 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:00.223 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:00.223 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:00.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.226 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:77968 3.2 e46/20 3:48f61a4b:::benchmark_data_smithi074_35447_object9745:head v 46'4808, mlcod=46'4799)", 2020-08-07T15:09:00.226 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:28.702836+0000", 2020-08-07T15:09:00.226 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.156145550000005, 2020-08-07T15:09:00.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27322339400000001, 2020-08-07T15:09:00.230 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:00.230 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:00.230 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:00.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:00.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702836+0000", 2020-08-07T15:09:00.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:00.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702836+0000", 2020-08-07T15:09:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1822e-05 2020-08-07T15:09:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702888+0000", 2020-08-07T15:09:00.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4110000000000001e-06 2020-08-07T15:09:00.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:00.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.702894+0000", 2020-08-07T15:09:00.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.26750942999999999 2020-08-07T15:09:00.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:00.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970404+0000", 2020-08-07T15:09:00.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1150000000000002e-06 2020-08-07T15:09:00.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:00.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970409+0000", 2020-08-07T15:09:00.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2898000000000007e-05 2020-08-07T15:09:00.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:00.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970472+0000", 2020-08-07T15:09:00.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2311999999999999e-05 2020-08-07T15:09:00.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:00.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.970494+0000", 2020-08-07T15:09:00.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055286800000000002 2020-08-07T15:09:00.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:00.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.976023+0000", 2020-08-07T15:09:00.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6726e-05 2020-08-07T15:09:00.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:00.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:28.976059+0000", 2020-08-07T15:09:00.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27322339400000001 2020-08-07T15:09:00.330 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:00.330 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:00.331 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:00.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.346 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86695 3.a 3:5058ba7c:::benchmark_data_smithi074_35447_object10836:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:09:00.347 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:09:00.362 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.533654134000003, 2020-08-07T15:09:00.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31490022699999998, 2020-08-07T15:09:00.363 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:00.363 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:00.363 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:00.363 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:00.363 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:00.364 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86695 2020-08-07T15:09:00.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.364 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:00.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:00.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:09:00.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:00.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:00.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:09:00.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4560000000000005e-05 2020-08-07T15:09:00.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:00.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325392+0000", 2020-08-07T15:09:00.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3180000000000003e-06 2020-08-07T15:09:00.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:00.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325398+0000", 2020-08-07T15:09:00.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.258e-05 2020-08-07T15:09:00.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:00.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325411+0000", 2020-08-07T15:09:00.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6550000000000001e-06 2020-08-07T15:09:00.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:00.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325417+0000", 2020-08-07T15:09:00.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024282879999999998 2020-08-07T15:09:00.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:00.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.327845+0000", 2020-08-07T15:09:00.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1203000000000001e-05 2020-08-07T15:09:00.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:00.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.327936+0000", 2020-08-07T15:09:00.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000186505 2020-08-07T15:09:00.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:09:00.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328123+0000", 2020-08-07T15:09:00.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020062890000000001 2020-08-07T15:09:00.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:00.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.330129+0000", 2020-08-07T15:09:00.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121513 2020-08-07T15:09:00.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:00.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.330250+0000", 2020-08-07T15:09:00.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30989746899999998 2020-08-07T15:09:00.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:00.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640148+0000", 2020-08-07T15:09:00.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2139999999999999e-06 2020-08-07T15:09:00.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:00.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640149+0000", 2020-08-07T15:09:00.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8081999999999998e-05 2020-08-07T15:09:00.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:00.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640197+0000", 2020-08-07T15:09:00.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0550999999999999e-05 2020-08-07T15:09:00.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:00.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640228+0000", 2020-08-07T15:09:00.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31490022699999998 2020-08-07T15:09:00.576 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:00.589 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:00.590 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:00.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.591 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86696 3.a 3:5058ba7c:::benchmark_data_smithi074_35447_object10836:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:09:00.591 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:09:00.591 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.533529454999993, 2020-08-07T15:09:00.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.347347556, 2020-08-07T15:09:00.592 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:00.592 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:00.593 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:00.593 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:00.623 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:00.623 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86696 2020-08-07T15:09:00.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.624 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:00.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:00.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:09:00.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:00.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:00.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:09:00.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0436999999999998e-05 2020-08-07T15:09:00.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:00.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325503+0000", 2020-08-07T15:09:00.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4659999999999998e-06 2020-08-07T15:09:00.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:00.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325508+0000", 2020-08-07T15:09:00.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1177e-05 2020-08-07T15:09:00.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:00.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325519+0000", 2020-08-07T15:09:00.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7809999999999996e-06 2020-08-07T15:09:00.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:00.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325524+0000", 2020-08-07T15:09:00.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002989219 2020-08-07T15:09:00.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:00.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328513+0000", 2020-08-07T15:09:00.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0649000000000006e-05 2020-08-07T15:09:00.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:00.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328604+0000", 2020-08-07T15:09:00.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001652 2020-08-07T15:09:00.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:09:00.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328769+0000", 2020-08-07T15:09:00.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023470639999999998 2020-08-07T15:09:00.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:00.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331116+0000", 2020-08-07T15:09:00.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6099999999999999e-07 2020-08-07T15:09:00.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:00.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331117+0000", 2020-08-07T15:09:00.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9150000000000001e-05 2020-08-07T15:09:00.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:00.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331176+0000", 2020-08-07T15:09:00.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065402581000000001 2020-08-07T15:09:00.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:00.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.396579+0000", 2020-08-07T15:09:00.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27618104599999999 2020-08-07T15:09:00.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:00.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672760+0000", 2020-08-07T15:09:00.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2020-08-07T15:09:00.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:00.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672761+0000", 2020-08-07T15:09:00.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9211000000000001e-05 2020-08-07T15:09:00.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:00.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672800+0000", 2020-08-07T15:09:00.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.347347556 2020-08-07T15:09:00.729 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:00.729 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:00.729 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:00.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.736 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:86695 3.a e52/20 ondisk, result = 0)", 2020-08-07T15:09:00.736 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:09:00.736 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.525572757999996, 2020-08-07T15:09:00.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.306823501, 2020-08-07T15:09:00.737 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:00.737 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:00.737 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:00.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:00.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:09:00.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:00.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:00.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:09:00.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.048e-05 2020-08-07T15:09:00.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:00.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333439+0000", 2020-08-07T15:09:00.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9990000000000002e-06 2020-08-07T15:09:00.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:00.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333443+0000", 2020-08-07T15:09:00.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30664195100000002 2020-08-07T15:09:00.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:00.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640085+0000", 2020-08-07T15:09:00.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8559999999999998e-06 2020-08-07T15:09:00.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:00.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640090+0000", 2020-08-07T15:09:00.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5166000000000001e-05 2020-08-07T15:09:00.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:00.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640135+0000", 2020-08-07T15:09:00.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4160000000000001e-06 2020-08-07T15:09:00.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:00.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640142+0000", 2020-08-07T15:09:00.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0632999999999998e-05 2020-08-07T15:09:00.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:00.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640232+0000", 2020-08-07T15:09:00.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.306823501 2020-08-07T15:09:00.785 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:00.785 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:00.786 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:00.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.786 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106232 3.a 3:5599d7e6:::benchmark_data_smithi074_35447_object13278:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:09:00.787 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:09:00.787 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 72.513374507999998, 2020-08-07T15:09:00.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52578228699999996, 2020-08-07T15:09:00.787 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:00.788 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:00.788 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106232 2020-08-07T15:09:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:09:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:09:00.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4214999999999997e-05 2020-08-07T15:09:00.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:00.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345641+0000", 2020-08-07T15:09:00.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9279999999999993e-06 2020-08-07T15:09:00.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:00.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345649+0000", 2020-08-07T15:09:00.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4391e-05 2020-08-07T15:09:00.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:00.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345664+0000", 2020-08-07T15:09:00.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7170000000000002e-06 2020-08-07T15:09:00.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:00.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345670+0000", 2020-08-07T15:09:00.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.459e-05 2020-08-07T15:09:00.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:00.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345695+0000", 2020-08-07T15:09:00.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6070999999999999e-05 2020-08-07T15:09:00.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345751+0000", 2020-08-07T15:09:00.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010069300000000001 2020-08-07T15:09:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:09:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345852+0000", 2020-08-07T15:09:00.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013512299999999999 2020-08-07T15:09:00.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.347203+0000", 2020-08-07T15:09:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0844000000000001e-05 2020-08-07T15:09:00.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:00.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.347274+0000", 2020-08-07T15:09:00.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52399471600000003 2020-08-07T15:09:00.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871269+0000", 2020-08-07T15:09:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.311e-06 2020-08-07T15:09:00.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:00.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871270+0000", 2020-08-07T15:09:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3509000000000004e-05 2020-08-07T15:09:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871343+0000", 2020-08-07T15:09:00.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6072000000000002e-05 2020-08-07T15:09:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:00.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871389+0000", 2020-08-07T15:09:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52578228699999996 2020-08-07T15:09:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:00.936 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:00.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.945 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106232 3.a e59/20 ondisk, result = 0)", 2020-08-07T15:09:00.945 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:09:00.945 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 72.510328333000004, 2020-08-07T15:09:00.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52274359299999995, 2020-08-07T15:09:00.946 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:00.946 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:00.946 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:00.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:00.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:09:00.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:00.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:00.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:09:00.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5011000000000001e-05 2020-08-07T15:09:00.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:00.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348668+0000", 2020-08-07T15:09:00.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1959999999999998e-06 2020-08-07T15:09:00.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:00.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348671+0000", 2020-08-07T15:09:00.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52253456499999995 2020-08-07T15:09:00.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:00.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871205+0000", 2020-08-07T15:09:00.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0230000000000003e-06 2020-08-07T15:09:00.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:00.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871209+0000", 2020-08-07T15:09:00.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0708999999999999e-05 2020-08-07T15:09:00.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:00.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871250+0000", 2020-08-07T15:09:00.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0158e-05 2020-08-07T15:09:00.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:00.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:00.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:00.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871260+0000", 2020-08-07T15:09:01.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013693099999999999 2020-08-07T15:09:01.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:01.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871397+0000", 2020-08-07T15:09:01.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52274359299999995 2020-08-07T15:09:01.009 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:01.009 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:01.010 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:01.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.010 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:127864 3.a 3:58746ca8:::benchmark_data_smithi074_35447_object15982:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e70)", 2020-08-07T15:09:01.011 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:04.819979+0000", 2020-08-07T15:09:01.011 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.039002623999998, 2020-08-07T15:09:01.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27732920700000002, 2020-08-07T15:09:01.021 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:01.038 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:01.038 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:01.038 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:01.039 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:01.039 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 127864 2020-08-07T15:09:01.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.039 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:01.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:01.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.819979+0000", 2020-08-07T15:09:01.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:01.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:01.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.819979+0000", 2020-08-07T15:09:01.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7188000000000001e-05 2020-08-07T15:09:01.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:01.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820006+0000", 2020-08-07T15:09:01.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0940000000000001e-06 2020-08-07T15:09:01.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:01.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820010+0000", 2020-08-07T15:09:01.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0869999999999996e-06 2020-08-07T15:09:01.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:01.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820019+0000", 2020-08-07T15:09:01.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7509999999999999e-06 2020-08-07T15:09:01.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:01.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820024+0000", 2020-08-07T15:09:01.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5089e-05 2020-08-07T15:09:01.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:01.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820049+0000", 2020-08-07T15:09:01.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0068999999999995e-05 2020-08-07T15:09:01.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:01.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820129+0000", 2020-08-07T15:09:01.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013952100000000001 2020-08-07T15:09:01.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:09:01.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820269+0000", 2020-08-07T15:09:01.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031575290000000001 2020-08-07T15:09:01.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:01.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.823426+0000", 2020-08-07T15:09:01.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4900000000000004e-07 2020-08-07T15:09:01.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:01.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.823427+0000", 2020-08-07T15:09:01.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2324999999999998e-05 2020-08-07T15:09:01.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:01.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.823480+0000", 2020-08-07T15:09:01.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.272535737 2020-08-07T15:09:01.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:01.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.096015+0000", 2020-08-07T15:09:01.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012371560000000001 2020-08-07T15:09:01.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:01.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097253+0000", 2020-08-07T15:09:01.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4000000000000001e-07 2020-08-07T15:09:01.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:01.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097253+0000", 2020-08-07T15:09:01.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5071999999999997e-05 2020-08-07T15:09:01.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:01.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097308+0000", 2020-08-07T15:09:01.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27732920700000002 2020-08-07T15:09:01.207 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:01.208 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:01.208 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:01.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.209 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:127864 3.a e70/20 ondisk, result = 0)", 2020-08-07T15:09:01.209 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:04.822639+0000", 2020-08-07T15:09:01.209 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.036343062, 2020-08-07T15:09:01.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27467429799999998, 2020-08-07T15:09:01.210 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:01.210 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:01.210 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:01.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:01.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.822639+0000", 2020-08-07T15:09:01.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:01.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:01.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.822639+0000", 2020-08-07T15:09:01.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3278999999999998e-05 2020-08-07T15:09:01.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:01.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.822672+0000", 2020-08-07T15:09:01.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1489999999999999e-06 2020-08-07T15:09:01.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:01.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.822678+0000", 2020-08-07T15:09:01.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.274520551 2020-08-07T15:09:01.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:01.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097199+0000", 2020-08-07T15:09:01.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6940000000000001e-06 2020-08-07T15:09:01.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:01.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097202+0000", 2020-08-07T15:09:01.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3429999999999997e-05 2020-08-07T15:09:01.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:01.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097236+0000", 2020-08-07T15:09:01.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7390000000000005e-06 2020-08-07T15:09:01.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:01.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097244+0000", 2020-08-07T15:09:01.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8455999999999996e-05 2020-08-07T15:09:01.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:01.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097313+0000", 2020-08-07T15:09:01.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27467429799999998 2020-08-07T15:09:01.245 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:01.245 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:01.245 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:01.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.249 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:145807 3.c e75/20 3:39741db6:::benchmark_data_smithi074_35447_object16198:head v 75'9594, mlcod=75'9590)", 2020-08-07T15:09:01.249 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.674086+0000", 2020-08-07T15:09:01.253 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.184896115999997, 2020-08-07T15:09:01.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40268798500000003, 2020-08-07T15:09:01.253 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:01.254 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:01.255 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:01.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:01.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674086+0000", 2020-08-07T15:09:01.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:01.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:01.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674086+0000", 2020-08-07T15:09:01.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3321000000000001e-05 2020-08-07T15:09:01.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:01.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674129+0000", 2020-08-07T15:09:01.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1919999999999999e-06 2020-08-07T15:09:01.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:01.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674136+0000", 2020-08-07T15:09:01.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40193062899999998 2020-08-07T15:09:01.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:01.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076067+0000", 2020-08-07T15:09:01.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3579e-05 2020-08-07T15:09:01.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:01.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076080+0000", 2020-08-07T15:09:01.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3733999999999998e-05 2020-08-07T15:09:01.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:01.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076124+0000", 2020-08-07T15:09:01.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0145e-05 2020-08-07T15:09:01.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:01.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076154+0000", 2020-08-07T15:09:01.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00055768600000000005 2020-08-07T15:09:01.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:01.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076712+0000", 2020-08-07T15:09:01.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1699000000000007e-05 2020-08-07T15:09:01.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:01.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076774+0000", 2020-08-07T15:09:01.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40268798500000003 2020-08-07T15:09:01.309 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:01.309 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:01.310 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:01.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.310 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:144 4.5 4:aed0a56a:::benchmark_data_smithi074_44586_object17:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:09:01.311 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.396584+0000", 2020-08-07T15:09:01.311 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.462397877000001, 2020-08-07T15:09:01.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45809661000000002, 2020-08-07T15:09:01.311 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:01.312 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:01.312 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:01.312 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:01.312 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:01.312 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 144 2020-08-07T15:09:01.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.313 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:01.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:01.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396584+0000", 2020-08-07T15:09:01.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:01.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:01.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396584+0000", 2020-08-07T15:09:01.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4195e-05 2020-08-07T15:09:01.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:01.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396628+0000", 2020-08-07T15:09:01.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.157e-06 2020-08-07T15:09:01.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:01.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396631+0000", 2020-08-07T15:09:01.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.832e-06 2020-08-07T15:09:01.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:01.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396638+0000", 2020-08-07T15:09:01.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.745e-06 2020-08-07T15:09:01.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:01.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396641+0000", 2020-08-07T15:09:01.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001659368 2020-08-07T15:09:01.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:01.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.398300+0000", 2020-08-07T15:09:01.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2652000000000007e-05 2020-08-07T15:09:01.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:01.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.398393+0000", 2020-08-07T15:09:01.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016763699999999999 2020-08-07T15:09:01.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,7", 2020-08-07T15:09:01.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.398560+0000", 2020-08-07T15:09:01.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023917719999999999 2020-08-07T15:09:01.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:01.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.400952+0000", 2020-08-07T15:09:01.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999997e-07 2020-08-07T15:09:01.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:01.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.400953+0000", 2020-08-07T15:09:01.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9373999999999998e-05 2020-08-07T15:09:01.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:01.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.401012+0000", 2020-08-07T15:09:01.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003107865 2020-08-07T15:09:01.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:01.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.404120+0000", 2020-08-07T15:09:01.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45049484299999998 2020-08-07T15:09:01.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:01.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854615+0000", 2020-08-07T15:09:01.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.035e-06 2020-08-07T15:09:01.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:01.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854616+0000", 2020-08-07T15:09:01.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4249000000000001e-05 2020-08-07T15:09:01.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:01.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854680+0000", 2020-08-07T15:09:01.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45809661000000002 2020-08-07T15:09:01.526 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:01.527 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:01.527 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:01.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.528 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:30975 4.5 e98/81 ondisk, result = 0)", 2020-08-07T15:09:01.529 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:45.734259+0000", 2020-08-07T15:09:01.529 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.124722780000001, 2020-08-07T15:09:01.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46200962800000001, 2020-08-07T15:09:01.534 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:01.535 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:01.535 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:01.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:01.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.734259+0000", 2020-08-07T15:09:01.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:01.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:01.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.734259+0000", 2020-08-07T15:09:01.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9501000000000001e-05 2020-08-07T15:09:01.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:01.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.734288+0000", 2020-08-07T15:09:01.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5990000000000004e-06 2020-08-07T15:09:01.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:01.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.734293+0000", 2020-08-07T15:09:01.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46181377000000001 2020-08-07T15:09:01.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:01.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.196107+0000", 2020-08-07T15:09:01.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6169999999999998e-06 2020-08-07T15:09:01.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:01.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.196112+0000", 2020-08-07T15:09:01.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5908999999999997e-05 2020-08-07T15:09:01.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:01.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.196158+0000", 2020-08-07T15:09:01.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1104000000000001e-05 2020-08-07T15:09:01.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:01.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.196169+0000", 2020-08-07T15:09:01.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9128000000000004e-05 2020-08-07T15:09:01.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:01.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.196268+0000", 2020-08-07T15:09:01.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46200962800000001 2020-08-07T15:09:01.616 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:01.616 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:01.616 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:01.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.622 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:30976 4.5 e98/81 ondisk, result = 0)", 2020-08-07T15:09:01.623 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:45.736493+0000", 2020-08-07T15:09:01.623 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.122488607999999, 2020-08-07T15:09:01.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54677504399999999, 2020-08-07T15:09:01.623 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:01.623 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:01.624 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:01.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:01.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736493+0000", 2020-08-07T15:09:01.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:01.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:01.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736493+0000", 2020-08-07T15:09:01.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.012e-05 2020-08-07T15:09:01.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:01.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736523+0000", 2020-08-07T15:09:01.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6669999999999998e-06 2020-08-07T15:09:01.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:01.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736527+0000", 2020-08-07T15:09:01.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54657333200000002 2020-08-07T15:09:01.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:01.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283100+0000", 2020-08-07T15:09:01.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.549e-06 2020-08-07T15:09:01.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:01.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283105+0000", 2020-08-07T15:09:01.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1853000000000006e-05 2020-08-07T15:09:01.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:01.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283167+0000", 2020-08-07T15:09:01.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015e-05 2020-08-07T15:09:01.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:01.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283177+0000", 2020-08-07T15:09:01.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1508000000000005e-05 2020-08-07T15:09:01.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:01.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283268+0000", 2020-08-07T15:09:01.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54677504399999999 2020-08-07T15:09:01.695 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:01.696 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:01.696 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:01.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.705 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31528 4.5 4:a883779b:::benchmark_data_smithi074_44586_object3940:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e100)", 2020-08-07T15:09:01.706 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:47.795320+0000", 2020-08-07T15:09:01.706 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 12.063662012, 2020-08-07T15:09:01.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33015256900000001, 2020-08-07T15:09:01.714 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:01.715 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:01.715 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:01.715 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:01.715 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:01.716 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31528 2020-08-07T15:09:01.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.716 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:01.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:01.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.795320+0000", 2020-08-07T15:09:01.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:01.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:01.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.795320+0000", 2020-08-07T15:09:01.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9305000000000003e-05 2020-08-07T15:09:01.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:01.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.795369+0000", 2020-08-07T15:09:01.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1659999999999999e-06 2020-08-07T15:09:01.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:01.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.795375+0000", 2020-08-07T15:09:01.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2284e-05 2020-08-07T15:09:01.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:01.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.795387+0000", 2020-08-07T15:09:01.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.789e-06 2020-08-07T15:09:01.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:01.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.795393+0000", 2020-08-07T15:09:01.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025794009999999998 2020-08-07T15:09:01.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:01.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.797973+0000", 2020-08-07T15:09:01.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011622599999999999 2020-08-07T15:09:01.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:01.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.798089+0000", 2020-08-07T15:09:01.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021524900000000001 2020-08-07T15:09:01.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,7", 2020-08-07T15:09:01.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.798304+0000", 2020-08-07T15:09:01.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042865830000000001 2020-08-07T15:09:01.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:01.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.802591+0000", 2020-08-07T15:09:01.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0088220250000000007 2020-08-07T15:09:01.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:01.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.811413+0000", 2020-08-07T15:09:01.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31390628199999998 2020-08-07T15:09:01.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:01.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125319+0000", 2020-08-07T15:09:01.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.037e-06 2020-08-07T15:09:01.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:01.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125320+0000", 2020-08-07T15:09:01.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0878000000000003e-05 2020-08-07T15:09:01.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:01.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125371+0000", 2020-08-07T15:09:01.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101344 2020-08-07T15:09:01.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:01.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125472+0000", 2020-08-07T15:09:01.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33015256900000001 2020-08-07T15:09:01.814 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:01.815 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:01.815 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:01.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.818 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:31536 4.c e100/99 4:326df2a4:::benchmark_data_smithi074_44586_object3941:head v 100'2008, mlcod=100'1999)", 2020-08-07T15:09:01.819 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:47.804640+0000", 2020-08-07T15:09:01.819 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 12.05434144, 2020-08-07T15:09:01.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32071613199999999, 2020-08-07T15:09:01.827 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:01.839 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:01.839 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:01.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:01.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.804640+0000", 2020-08-07T15:09:01.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:01.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:01.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.804640+0000", 2020-08-07T15:09:01.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9385999999999998e-05 2020-08-07T15:09:01.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:01.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.804690+0000", 2020-08-07T15:09:01.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2319999999999998e-06 2020-08-07T15:09:01.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:01.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.804696+0000", 2020-08-07T15:09:01.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.319733934 2020-08-07T15:09:01.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:01.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.124430+0000", 2020-08-07T15:09:01.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1089999999999997e-06 2020-08-07T15:09:01.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:01.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.124435+0000", 2020-08-07T15:09:01.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9981999999999998e-05 2020-08-07T15:09:01.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:01.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.124475+0000", 2020-08-07T15:09:01.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0684e-05 2020-08-07T15:09:01.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:01.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.124506+0000", 2020-08-07T15:09:01.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00079925600000000003 2020-08-07T15:09:01.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:01.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125305+0000", 2020-08-07T15:09:01.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1548999999999997e-05 2020-08-07T15:09:01.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:01.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125356+0000", 2020-08-07T15:09:01.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32071613199999999 2020-08-07T15:09:01.890 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:01.891 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:01.891 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:01.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.891 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:31528 4.5 e100/81 ondisk, result = 0)", 2020-08-07T15:09:01.892 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:47.812815+0000", 2020-08-07T15:09:01.892 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 12.046166512999999, 2020-08-07T15:09:01.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.312664363, 2020-08-07T15:09:01.922 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:01.922 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:01.922 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:01.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:01.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.812815+0000", 2020-08-07T15:09:01.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:01.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:01.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.812815+0000", 2020-08-07T15:09:01.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2224999999999997e-05 2020-08-07T15:09:01.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:01.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.812847+0000", 2020-08-07T15:09:01.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9460000000000003e-06 2020-08-07T15:09:01.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:01.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.812853+0000", 2020-08-07T15:09:01.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31240245100000003 2020-08-07T15:09:01.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:01.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125256+0000", 2020-08-07T15:09:01.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7509999999999997e-06 2020-08-07T15:09:01.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:01.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125262+0000", 2020-08-07T15:09:01.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9220000000000001e-05 2020-08-07T15:09:01.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:01.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125301+0000", 2020-08-07T15:09:01.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3700000000000001e-06 2020-08-07T15:09:01.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:01.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125310+0000", 2020-08-07T15:09:01.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001694 2020-08-07T15:09:01.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:01.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125479+0000", 2020-08-07T15:09:01.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.312664363 2020-08-07T15:09:01.958 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:01.965 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:01.965 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:01.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.966 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31528 4.5 4:a883779b:::benchmark_data_smithi074_44586_object3940:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e100)", 2020-08-07T15:09:01.966 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:47.842454+0000", 2020-08-07T15:09:01.966 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 12.016527926, 2020-08-07T15:09:01.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28293349200000001, 2020-08-07T15:09:01.967 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:01.967 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:09:01.967 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:01.973 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:01.973 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:01.973 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31528 2020-08-07T15:09:01.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.980 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:01.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:01.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.842454+0000", 2020-08-07T15:09:01.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:01.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:01.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.842454+0000", 2020-08-07T15:09:01.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8015000000000004e-05 2020-08-07T15:09:01.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:01.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:01.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:01.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.842532+0000", 2020-08-07T15:09:02.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0839000000000001e-05 2020-08-07T15:09:02.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:02.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.842543+0000", 2020-08-07T15:09:02.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2580000000000001e-05 2020-08-07T15:09:02.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:02.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.842565+0000", 2020-08-07T15:09:02.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1283e-05 2020-08-07T15:09:02.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:02.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.842576+0000", 2020-08-07T15:09:02.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0846000000000004e-05 2020-08-07T15:09:02.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:02.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.842667+0000", 2020-08-07T15:09:02.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7760000000000003e-05 2020-08-07T15:09:02.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for ondisk", 2020-08-07T15:09:02.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.842725+0000", 2020-08-07T15:09:02.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28266216900000002 2020-08-07T15:09:02.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:02.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125387+0000", 2020-08-07T15:09:02.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28293349200000001 2020-08-07T15:09:02.051 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:02.051 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:02.052 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:02.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.052 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:31816 4.1 e100/98 4:8a2ebf91:::benchmark_data_smithi074_44586_object3976:head v 100'1872, mlcod=100'1866)", 2020-08-07T15:09:02.052 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.169272+0000", 2020-08-07T15:09:02.053 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.689709401, 2020-08-07T15:09:02.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38326060099999998, 2020-08-07T15:09:02.063 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:02.064 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:02.064 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:02.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:02.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.169272+0000", 2020-08-07T15:09:02.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:02.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:02.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.169272+0000", 2020-08-07T15:09:02.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.93e-05 2020-08-07T15:09:02.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:02.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.169312+0000", 2020-08-07T15:09:02.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0230000000000003e-06 2020-08-07T15:09:02.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:02.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.169316+0000", 2020-08-07T15:09:02.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38224064899999999 2020-08-07T15:09:02.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:02.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.551556+0000", 2020-08-07T15:09:02.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.673e-06 2020-08-07T15:09:02.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:02.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.551561+0000", 2020-08-07T15:09:02.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0315999999999996e-05 2020-08-07T15:09:02.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:02.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.551631+0000", 2020-08-07T15:09:02.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6877999999999999e-05 2020-08-07T15:09:02.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:02.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.551658+0000", 2020-08-07T15:09:02.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00081916599999999997 2020-08-07T15:09:02.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:02.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.552477+0000", 2020-08-07T15:09:02.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5596000000000001e-05 2020-08-07T15:09:02.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:02.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.552533+0000", 2020-08-07T15:09:02.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38326060099999998 2020-08-07T15:09:02.077 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:02.078 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:02.078 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:02.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.079 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:37048 4.5 4:a403bd70:::benchmark_data_smithi074_44586_object4630:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e106)", 2020-08-07T15:09:02.080 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:54.237330+0000", 2020-08-07T15:09:02.080 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.6216515310000004, 2020-08-07T15:09:02.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.474059283, 2020-08-07T15:09:02.080 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:02.080 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:02.081 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:02.081 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:02.081 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:02.081 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 37048 2020-08-07T15:09:02.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.082 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:02.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:02.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237330+0000", 2020-08-07T15:09:02.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:02.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:02.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237330+0000", 2020-08-07T15:09:02.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0713999999999998e-05 2020-08-07T15:09:02.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:02.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237361+0000", 2020-08-07T15:09:02.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.756e-06 2020-08-07T15:09:02.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:02.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237364+0000", 2020-08-07T15:09:02.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.694e-06 2020-08-07T15:09:02.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:02.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237369+0000", 2020-08-07T15:09:02.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3709999999999998e-06 2020-08-07T15:09:02.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:02.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237372+0000", 2020-08-07T15:09:02.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015644700000000001 2020-08-07T15:09:02.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:02.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.238936+0000", 2020-08-07T15:09:02.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9940999999999998e-05 2020-08-07T15:09:02.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:02.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.239026+0000", 2020-08-07T15:09:02.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017423800000000001 2020-08-07T15:09:02.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,7", 2020-08-07T15:09:02.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.239200+0000", 2020-08-07T15:09:02.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041267530000000004 2020-08-07T15:09:02.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:02.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.243327+0000", 2020-08-07T15:09:02.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7700000000000004e-07 2020-08-07T15:09:02.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:02.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.243328+0000", 2020-08-07T15:09:02.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1211999999999997e-05 2020-08-07T15:09:02.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:02.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.243379+0000", 2020-08-07T15:09:02.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075155150000000004 2020-08-07T15:09:02.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:02.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.250894+0000", 2020-08-07T15:09:02.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.460450313 2020-08-07T15:09:02.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:02.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.711345+0000", 2020-08-07T15:09:02.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1990000000000001e-06 2020-08-07T15:09:02.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:02.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.711346+0000", 2020-08-07T15:09:02.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3529999999999998e-05 2020-08-07T15:09:02.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:02.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:02.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:02.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.711389+0000", 2020-08-07T15:09:02.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.474059283 2020-08-07T15:09:02.107 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:02.107 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:02.107 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:02.107 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:02.107 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:02.108 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:09:02.108 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:09:02.127 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:09:02.132 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:09:02.143 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2020-08-07T15:08:59.266587+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":311132160,"num_objects":4750,"num_object_clones":0,"num_object_copies":14250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":542,"num_objects_unfound":0,"num_objects_dirty":4750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37980,"num_write_kb":303840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":3514368,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":135,"acting":134,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9297472,"kb_used_data":908800,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":745677248,"statfs":{"total":773094113280,"available":763573501952,"internally_reserved":8589934592,"allocated":930611200,"data_stored":925810562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,13,7,7,1,0,1,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":34004992,"num_objects":519,"num_object_clones":0,"num_object_copies":1557,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":542,"num_objects_unfound":0,"num_objects_dirty":519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4150,"num_write_kb":33207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":3473408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.018792"},"pg_stats":[{"pgid":"1.14","version":"16'2","reported_seq":"38","reported_epoch":"108","state":"active+clean","last_fresh":"2020-08-07T15:08:55.494295+0000","last_change":"2020-08-07T15:08:42.533999+0000","last_active":"2020-08-07T15:08:55.494295+0000","last_peered":"2020-08-07T15:08:55.494295+0000","last_clean":"2020-08-07T15:08:55.494295+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:08:55.494295+0000","last_undegraded":"2020-08-07T15:08:55.494295+0000","last_fullsized":"2020-08-07T15:08:55.494295+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:42.533921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:42.533921+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"65","reported_epoch":"109","state":"active+undersized+remapped","last_fresh":"2020-08-07T15:08:56.263398+0000","last_change":"2020-08-07T15:08:56.263250+0000","last_active":"2020-08-07T15:08:56.263398+0000","last_peered":"2020-08-07T15:08:56.263398+0000","last_clean":"2020-08-07T15:08:53.236400+0000","last_became_active":"2020-08-07T15:08:56.263250+0000","last_became_peered":"2020-08-07T15:08:56.263250+0000","last_unstale":"2020-08-07T15:08:56.263398+0000","last_undegraded":"2020-08-07T15:08:56.263398+0000","last_fullsized":"2020-08-07T15:08:56.257727+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.141010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.141010+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,4],"avail_no_missing":["1","0","3","4"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"38","reported_epoch":"108","state":"active+clean","last_fresh":"2020-08-07T15:08:55.252092+0000","last_change":"2020-08-07T15:08:55.078015+0000","last_active":"2020-08-07T15:08:55.252092+0000","last_peered":"2020-08-07T15:08:55.252092+0000","last_clean":"2020-08-07T15:08:55.252092+0000","last_became_active":"2020-08-07T15:08:41.685106+0000","last_became_peered":"2020-08-07T15:08:41.685106+0000","last_unstale":"2020-08-07T15:08:55.252092+0000","last_undegraded":"2020-08-07T15:08:55.252092+0000","last_fullsized":"2020-08-07T15:08:55.252092+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:55.077924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:55.077924+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"117","reported_epoch":"108","state":"active+clean","last_fresh":"2020-08-07T15:08:55.251703+0000","last_change":"2020-08-07T15:08:43.544399+0000","last_active":"2020-08-07T15:08:55.251703+0000","last_peered":"2020-08-07T15:08:55.251703+0000","last_clean":"2020-08-07T15:08:55.251703+0000","last_became_active":"2020-08-07T15:08:41.675776+0000","last_became_peered":"2020-08-07T15:08:41.675776+0000","last_unstale":"2020-08-07T15:08:55.251703+0000","last_undegraded":"2020-08-07T15:08:55.251703+0000","last_fullsized":"2020-08-07T15:08:55.251703+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.544324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.544324+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"145","reported_epoch":"108","state":"active+clean","last_fresh":"2020-08-07T15:08:55.251652+0000","last_change":"2020-08-07T15:08:51.599032+0000","last_active":"2020-08-07T15:08:55.251652+0000","last_peered":"2020-08-07T15:08:55.251652+0000","last_clean":"2020-08-07T15:08:55.251652+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:08:55.251652+0000","last_undegraded":"2020-08-07T15:08:55.251652+0000","last_fullsized":"2020-08-07T15:08:55.251652+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:51.598947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.598947+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"104","reported_epoch":"109","state":"active+clean","last_fresh":"2020-08-07T15:08:56.259493+0000","last_change":"2020-08-07T15:08:11.840861+0000","last_active":"2020-08-07T15:08:56.259493+0000","last_peered":"2020-08-07T15:08:56.259493+0000","last_clean":"2020-08-07T15:08:56.259493+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:08:56.259493+0000","last_undegraded":"2020-08-07T15:08:56.259493+0000","last_fullsized":"2020-08-07T15:08:56.259493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:11.840807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:11.840807+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"94","reported_epoch":"108","state":"active+clean","last_fresh":"2020-08-07T15:08:55.250302+0000","last_change":"2020-08-07T15:08:39.558760+0000","last_active":"2020-08-07T15:08:55.250302+0000","last_peered":"2020-08-07T15:08:55.250302+0000","last_clean":"2020-08-07T15:08:55.250302+0000","last_became_active":"2020-08-07T15:08:36.615358+0000","last_became_peered":"2020-08-07T15:08:36.615358+0000","last_unstale":"2020-08-07T15:08:55.250302+0000","last_undegraded":"2020-08-07T15:08:55.250302+0000","last_fullsized":"2020-08-07T15:08:55.250302+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:39.558677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:39.558677+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"109'2224","reported_seq":"2260","reported_epoch":"109","state":"active+clean","last_fresh":"2020-08-07T15:08:56.639742+0000","last_change":"2020-08-07T15:08:46.715869+0000","last_active":"2020-08-07T15:08:56.639742+0000","last_peered":"2020-08-07T15:08:56.639742+0000","last_clean":"2020-08-07T15:08:56.639742+0000","last_became_active":"2020-08-07T15:08:46.708112+0000","last_became_peered":"2020-08-07T15:08:46.708112+0000","last_unstale":"2020-08-07T15:08:56.639742+0000","last_undegraded":"2020-08-07T15:08:56.639742+0000","last_fullsized":"2020-08-07T15:08:56.639742+0000","mapping_epoch":98,"log_start":"109'2218","ondisk_log_start":"109'2218","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18219008,"num_objects":278,"num_object_clones":0,"num_object_copies":834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2224,"num_write_kb":17792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"106","reported_epoch":"108","state":"active+clean","last_fresh":"2020-08-07T15:08:55.253935+0000","last_change":"2020-08-07T15:08:40.057705+0000","last_active":"2020-08-07T15:08:55.253935+0000","last_peered":"2020-08-07T15:08:55.253935+0000","last_clean":"2020-08-07T15:08:55.253935+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:08:55.253935+0000","last_undegraded":"2020-08-07T15:08:55.253935+0000","last_fullsized":"2020-08-07T15:08:55.253935+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:40.057622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:40.057622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"63","reported_epoch":"108","state":"active+clean","last_fresh":"2020-08-07T15:08:55.494258+0000","last_change":"2020-08-07T15:08:53.756554+0000","last_active":"2020-08-07T15:08:55.494258+0000","last_peered":"2020-08-07T15:08:55.494258+0000","last_clean":"2020-08-07T15:08:55.494258+0000","last_became_active":"2020-08-07T15:08:46.710404+0000","last_became_peered":"2020-08-07T15:08:46.710404+0000","last_unstale":"2020-08-07T15:08:55.494258+0000","last_undegraded":"2020-08-07T15:08:55.494258+0000","last_fullsized":"2020-08-07T15:08:55.494258+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:53.756343+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:53.756343+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"109'2380","reported_seq":"2468","reported_epoch":"109","state":"active+remapped+backfilling","last_fresh":"2020-08-07T15:08:56.709793+0000","last_change":"2020-08-07T15:08:56.301510+0000","last_active":"2020-08-07T15:08:56.709793+0000","last_peered":"2020-08-07T15:08:56.709793+0000","last_clean":"2020-08-07T15:08:54.203386+0000","last_became_active":"2020-08-07T15:08:56.280473+0000","last_became_peered":"2020-08-07T15:08:56.280473+0000","last_unstale":"2020-08-07T15:08:56.709793+0000","last_undegraded":"2020-08-07T15:08:56.709793+0000","last_fullsized":"2020-08-07T15:08:56.709793+0000","mapping_epoch":108,"log_start":"109'2376","ondisk_log_start":"109'2376","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19496960,"num_objects":298,"num_object_clones":0,"num_object_copies":894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":270,"num_objects_unfound":0,"num_objects_dirty":298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2380,"num_write_kb":19040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":1769472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[0,7,1],"avail_no_missing":["0","1","7"],"object_location_counts":[{"shards":"0,1,7","objects":298}],"blocked_by":[],"up_primary":3,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"103","reported_epoch":"108","state":"active+clean","last_fresh":"2020-08-07T15:08:55.255322+0000","last_change":"2020-08-07T15:08:27.906894+0000","last_active":"2020-08-07T15:08:55.255322+0000","last_peered":"2020-08-07T15:08:55.255322+0000","last_clean":"2020-08-07T15:08:55.255322+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:08:55.255322+0000","last_undegraded":"2020-08-07T15:08:55.255322+0000","last_fullsized":"2020-08-07T15:08:55.255322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:27.906822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:27.906822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"109","reported_epoch":"109","state":"active+clean","last_fresh":"2020-08-07T15:08:56.259472+0000","last_change":"2020-08-07T15:08:43.899342+0000","last_active":"2020-08-07T15:08:56.259472+0000","last_peered":"2020-08-07T15:08:56.259472+0000","last_clean":"2020-08-07T15:08:56.259472+0000","last_became_active":"2020-08-07T15:08:36.622209+0000","last_became_peered":"2020-08-07T15:08:36.622209+0000","last_unstale":"2020-08-07T15:08:56.259472+0000","last_undegraded":"2020-08-07T15:08:56.259472+0000","last_fullsized":"2020-08-07T15:08:56.259472+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.899280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.899280+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"109'2352","reported_seq":"2388","reported_epoch":"109","state":"active+clean","last_fresh":"2020-08-07T15:08:56.701825+0000","last_change":"2020-08-07T15:08:24.549715+0000","last_active":"2020-08-07T15:08:56.701825+0000","last_peered":"2020-08-07T15:08:56.701825+0000","last_clean":"2020-08-07T15:08:56.701825+0000","last_became_active":"2020-08-07T15:08:24.549245+0000","last_became_peered":"2020-08-07T15:08:24.549245+0000","last_unstale":"2020-08-07T15:08:56.701825+0000","last_undegraded":"2020-08-07T15:08:56.701825+0000","last_fullsized":"2020-08-07T15:08:56.701825+0000","mapping_epoch":81,"log_start":"109'2344","ondisk_log_start":"109'2344","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19267584,"num_objects":294,"num_object_clones":0,"num_object_copies":882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2352,"num_write_kb":18816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"103","reported_epoch":"108","state":"active+clean","last_fresh":"2020-08-07T15:08:55.255740+0000","last_change":"2020-08-07T15:08:13.867355+0000","last_active":"2020-08-07T15:08:55.255740+0000","last_peered":"2020-08-07T15:08:55.255740+0000","last_clean":"2020-08-07T15:08:55.255740+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:08:55.255740+0000","last_undegraded":"2020-08-07T15:08:55.255740+0000","last_fullsized":"2020-08-07T15:08:55.255740+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:13.867148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:13.867148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"109","reported_epoch":"109","state":"active+clean","last_fresh":"2020-08-07T15:08:56.259193+0000","last_change":"2020-08-07T15:08:50.686578+0000","last_active":"2020-08-07T15:08:56.259193+0000","last_peered":"2020-08-07T15:08:56.259193+0000","last_clean":"2020-08-07T15:08:56.259193+0000","last_became_active":"2020-08-07T15:08:36.633406+0000","last_became_peered":"2020-08-07T15:08:36.633406+0000","last_unstale":"2020-08-07T15:08:56.259193+0000","last_undegraded":"2020-08-07T15:08:56.259193+0000","last_fullsized":"2020-08-07T15:08:56.259193+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:50.686496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:50.686496+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"109'2176","reported_seq":"2212","reported_epoch":"109","state":"active+clean","last_fresh":"2020-08-07T15:08:56.666805+0000","last_change":"2020-08-07T15:08:24.557398+0000","last_active":"2020-08-07T15:08:56.666805+0000","last_peered":"2020-08-07T15:08:56.666805+0000","last_clean":"2020-08-07T15:08:56.666805+0000","last_became_active":"2020-08-07T15:08:24.557022+0000","last_became_peered":"2020-08-07T15:08:24.557022+0000","last_unstale":"2020-08-07T15:08:56.666805+0000","last_undegraded":"2020-08-07T15:08:56.666805+0000","last_fullsized":"2020-08-07T15:08:56.666805+0000","mapping_epoch":81,"log_start":"109'2168","ondisk_log_start":"109'2168","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":17825792,"num_objects":272,"num_object_clones":0,"num_object_copies":816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2176,"num_write_kb":17408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"135","reported_epoch":"108","state":"active+clean","last_fresh":"2020-08-07T15:08:55.255706+0000","last_change":"2020-08-07T15:08:43.036067+0000","last_active":"2020-08-07T15:08:55.255706+0000","last_peered":"2020-08-07T15:08:55.255706+0000","last_clean":"2020-08-07T15:08:55.255706+0000","last_became_active":"2020-08-07T15:08:36.627426+0000","last_became_peered":"2020-08-07T15:08:36.627426+0000","last_unstale":"2020-08-07T15:08:55.255706+0000","last_undegraded":"2020-08-07T15:08:55.255706+0000","last_fullsized":"2020-08-07T15:08:55.255706+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.035987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.035987+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"103","reported_epoch":"108","state":"active+clean","last_fresh":"2020-08-07T15:08:55.251800+0000","last_change":"2020-08-07T15:08:26.317430+0000","last_active":"2020-08-07T15:08:55.251800+0000","last_peered":"2020-08-07T15:08:55.251800+0000","last_clean":"2020-08-07T15:08:55.251800+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:08:55.251800+0000","last_undegraded":"2020-08-07T15:08:55.251800+0000","last_fullsized":"2020-08-07T15:08:55.251800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:26.317351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:26.317351+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"110'2496","reported_seq":"2533","reported_epoch":"110","state":"active+clean","last_fresh":"2020-08-07T15:08:57.667912+0000","last_change":"2020-08-07T15:08:24.552570+0000","last_active":"2020-08-07T15:08:57.667912+0000","last_peered":"2020-08-07T15:08:57.667912+0000","last_clean":"2020-08-07T15:08:57.667912+0000","last_became_active":"2020-08-07T15:08:24.552105+0000","last_became_peered":"2020-08-07T15:08:24.552105+0000","last_unstale":"2020-08-07T15:08:57.667912+0000","last_undegraded":"2020-08-07T15:08:57.667912+0000","last_fullsized":"2020-08-07T15:08:57.667912+0000","mapping_epoch":81,"log_start":"110'2489","ondisk_log_start":"110'2489","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20447232,"num_objects":312,"num_object_clones":0,"num_object_copies":936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2496,"num_write_kb":19968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"103","reported_epoch":"108","state":"active+clean","last_fresh":"2020-08-07T15:08:55.494214+0000","last_change":"2020-08-07T15:07:36.864560+0000","last_active":"2020-08-07T15:08:55.494214+0000","last_peered":"2020-08-07T15:08:55.494214+0000","last_clean":"2020-08-07T15:08:55.494214+0000","last_became_active":"2020-08-07T15:06:23.223489+0000","last_became_peered":"2020-08-07T15:06:23.223489+0000","last_unstale":"2020-08-07T15:08:55.494214+0000","last_undegraded":"2020-08-07T15:08:55.494214+0000","last_fullsized":"2020-08-07T15:08:55.494214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:07:36.864483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.864483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"151","reported_epoch":"108","state":"active+clean","last_fresh":"2020-08-07T15:08:55.254878+0000","last_change":"2020-08-07T15:08:42.069637+0000","last_active":"2020-08-07T15:08:55.254878+0000","last_peered":"2020-08-07T15:08:55.254878+0000","last_clean":"2020-08-07T15:08:55.254878+0000","last_became_active":"2020-08-07T15:08:36.628365+0000","last_became_peered":"2020-08-07T15:08:36.628365+0000","last_unstale":"2020-08-07T15:08:55.254878+0000","last_undegraded":"2020-08-07T15:08:55.254878+0000","last_fullsized":"2020-08-07T15:08:55.254878+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:42.069585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:42.069585+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"109'2512","reported_seq":"2548","reported_epoch":"109","state":"active+clean","last_fresh":"2020-08-07T15:08:56.472515+0000","last_change":"2020-08-07T15:08:24.550789+0000","last_active":"2020-08-07T15:08:56.472515+0000","last_peered":"2020-08-07T15:08:56.472515+0000","last_clean":"2020-08-07T15:08:56.472515+0000","last_became_active":"2020-08-07T15:08:24.550222+0000","last_became_peered":"2020-08-07T15:08:24.550222+0000","last_unstale":"2020-08-07T15:08:56.472515+0000","last_undegraded":"2020-08-07T15:08:56.472515+0000","last_fullsized":"2020-08-07T15:08:56.472515+0000","mapping_epoch":81,"log_start":"109'2505","ondisk_log_start":"109'2505","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20578304,"num_objects":314,"num_object_clones":0,"num_object_copies":942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2512,"num_write_kb":20096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"103","reported_epoch":"108","state":"active+clean","last_fresh":"2020-08-07T15:08:55.252008+0000","last_change":"2020-08-07T15:08:30.339852+0000","last_active":"2020-08-07T15:08:55.252008+0000","last_peered":"2020-08-07T15:08:55.252008+0000","last_clean":"2020-08-07T15:08:55.252008+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:08:55.252008+0000","last_undegraded":"2020-08-07T15:08:55.252008+0000","last_fullsized":"2020-08-07T15:08:55.252008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:08:30.339785+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"101","reported_epoch":"108","state":"active+clean","last_fresh":"2020-08-07T15:08:55.252216+0000","last_change":"2020-08-07T15:08:38.386792+0000","last_active":"2020-08-07T15:08:55.252216+0000","last_peered":"2020-08-07T15:08:55.252216+0000","last_clean":"2020-08-07T15:08:55.252216+0000","last_became_active":"2020-08-07T15:08:36.618164+0000","last_became_peered":"2020-08-07T15:08:36.618164+0000","last_unstale":"2020-08-07T15:08:55.252216+0000","last_undegraded":"2020-08-07T15:08:55.252216+0000","last_fullsized":"2020-08-07T15:08:55.252216+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:38.386739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:38.386739+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"109'2448","reported_seq":"2484","reported_epoch":"109","state":"active+clean","last_fresh":"2020-08-07T15:08:56.695130+0000","last_change":"2020-08-07T15:08:24.558242+0000","last_active":"2020-08-07T15:08:56.695130+0000","last_peered":"2020-08-07T15:08:56.695130+0000","last_clean":"2020-08-07T15:08:56.695130+0000","last_became_active":"2020-08-07T15:08:24.557354+0000","last_became_peered":"2020-08-07T15:08:24.557354+0000","last_unstale":"2020-08-07T15:08:56.695130+0000","last_undegraded":"2020-08-07T15:08:56.695130+0000","last_fullsized":"2020-08-07T15:08:56.695130+0000","mapping_epoch":81,"log_start":"109'2441","ondisk_log_start":"109'2441","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20054016,"num_objects":306,"num_object_clones":0,"num_object_copies":918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2448,"num_write_kb":19584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"100","reported_epoch":"108","state":"active+clean","last_fresh":"2020-08-07T15:08:55.255157+0000","last_change":"2020-08-07T15:08:15.887617+0000","last_active":"2020-08-07T15:08:55.255157+0000","last_peered":"2020-08-07T15:08:55.255157+0000","last_clean":"2020-08-07T15:08:55.255157+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:08:55.255157+0000","last_undegraded":"2020-08-07T15:08:55.255157+0000","last_fullsized":"2020-08-07T15:08:55.255157+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:15.887513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:15.887513+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"132","reported_epoch":"110","state":"peering","last_fresh":"2020-08-07T15:08:57.169641+0000","last_change":"2020-08-07T15:08:57.169893+0000","last_active":"2020-08-07T15:08:57.169641+0000","last_peered":"2020-08-07T15:08:56.264303+0000","last_clean":"2020-08-07T15:08:53.224498+0000","last_became_active":"2020-08-07T15:08:56.263952+0000","last_became_peered":"2020-08-07T15:08:56.263952+0000","last_unstale":"2020-08-07T15:08:57.169641+0000","last_undegraded":"2020-08-07T15:08:57.169641+0000","last_fullsized":"2020-08-07T15:08:57.169641+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.927824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.927824+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"109'2232","reported_seq":"2266","reported_epoch":"109","state":"active+clean","last_fresh":"2020-08-07T15:08:56.649817+0000","last_change":"2020-08-07T15:08:24.552962+0000","last_active":"2020-08-07T15:08:56.649817+0000","last_peered":"2020-08-07T15:08:56.649817+0000","last_clean":"2020-08-07T15:08:56.649817+0000","last_became_active":"2020-08-07T15:08:24.552070+0000","last_became_peered":"2020-08-07T15:08:24.552070+0000","last_unstale":"2020-08-07T15:08:56.649817+0000","last_undegraded":"2020-08-07T15:08:56.649817+0000","last_fullsized":"2020-08-07T15:08:56.649817+0000","mapping_epoch":81,"log_start":"109'2226","ondisk_log_start":"109'2226","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18284544,"num_objects":279,"num_object_clones":0,"num_object_copies":837,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2232,"num_write_kb":17856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"78","reported_epoch":"108","state":"active+clean","last_fresh":"2020-08-07T15:08:55.251901+0000","last_change":"2020-08-07T15:08:39.409159+0000","last_active":"2020-08-07T15:08:55.251901+0000","last_peered":"2020-08-07T15:08:55.251901+0000","last_clean":"2020-08-07T15:08:55.251901+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:08:55.251901+0000","last_undegraded":"2020-08-07T15:08:55.251901+0000","last_fullsized":"2020-08-07T15:08:55.251901+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:39.409063+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_clean_scrub_stamp":"2020-08-07T15:08:39.409063+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"109'2504","reported_seq":"2540","reported_epoch":"109","state":"active+clean","last_fresh":"2020-08-07T15:08:56.459433+0000","last_change":"2020-08-07T15:08:24.554063+0000","last_active":"2020-08-07T15:08:56.459433+0000","last_peered":"2020-08-07T15:08:56.459433+0000","last_clean":"2020-08-07T15:08:56.459433+0000","last_became_active":"2020-08-07T15:08:24.553668+0000","last_became_peered":"2020-08-07T15:08:24.553668+0000","last_unstale":"2020-08-07T15:08:56.459433+0000","last_undegraded":"2020-08-07T15:08:56.459433+0000","last_fullsized":"2020-08-07T15:08:56.459433+0000","mapping_epoch":81,"log_start":"109'2498","ondisk_log_start":"109'2498","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20512768,"num_objects":313,"num_object_clones":0,"num_object_copies":939,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2504,"num_write_kb":20032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"75","reported_epoch":"108","state":"active+clean","last_fresh":"2020-08-07T15:08:55.251670+0000","last_change":"2020-08-07T15:07:33.177491+0000","last_active":"2020-08-07T15:08:55.251670+0000","last_peered":"2020-08-07T15:08:55.251670+0000","last_clean":"2020-08-07T15:08:55.251670+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:08:55.251670+0000","last_undegraded":"2020-08-07T15:08:55.251670+0000","last_fullsized":"2020-08-07T15:08:55.251670+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:33.177412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:33.177412+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"109'2200","reported_seq":"2236","reported_epoch":"109","state":"active+clean","last_fresh":"2020-08-07T15:08:56.633877+0000","last_change":"2020-08-07T15:08:46.709789+0000","last_active":"2020-08-07T15:08:56.633877+0000","last_peered":"2020-08-07T15:08:56.633877+0000","last_clean":"2020-08-07T15:08:56.633877+0000","last_became_active":"2020-08-07T15:08:46.708674+0000","last_became_peered":"2020-08-07T15:08:46.708674+0000","last_unstale":"2020-08-07T15:08:56.633877+0000","last_undegraded":"2020-08-07T15:08:56.633877+0000","last_fullsized":"2020-08-07T15:08:56.633877+0000","mapping_epoch":98,"log_start":"109'2190","ondisk_log_start":"109'2190","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18022400,"num_objects":275,"num_object_clones":0,"num_object_copies":825,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2200,"num_write_kb":17600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"139","reported_epoch":"108","state":"active+clean","last_fresh":"2020-08-07T15:08:55.255572+0000","last_change":"2020-08-07T15:07:56.631170+0000","last_active":"2020-08-07T15:08:55.255572+0000","last_peered":"2020-08-07T15:08:55.255572+0000","last_clean":"2020-08-07T15:08:55.255572+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:08:55.255572+0000","last_undegraded":"2020-08-07T15:08:55.255572+0000","last_fullsized":"2020-08-07T15:08:55.255572+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:56.631091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:56.631091+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"109'2416","reported_seq":"2474","reported_epoch":"109","state":"active+clean","last_fresh":"2020-08-07T15:08:56.696932+0000","last_change":"2020-08-07T15:08:52.193226+0000","last_active":"2020-08-07T15:08:56.696932+0000","last_peered":"2020-08-07T15:08:56.696932+0000","last_clean":"2020-08-07T15:08:56.696932+0000","last_became_active":"2020-08-07T15:08:52.166336+0000","last_became_peered":"2020-08-07T15:08:52.166336+0000","last_unstale":"2020-08-07T15:08:56.696932+0000","last_undegraded":"2020-08-07T15:08:56.696932+0000","last_fullsized":"2020-08-07T15:08:56.696932+0000","mapping_epoch":104,"log_start":"109'2410","ondisk_log_start":"109'2410","created":81,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19791872,"num_objects":302,"num_object_clones":0,"num_object_copies":906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2416,"num_write_kb":19328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":40960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"97","reported_epoch":"108","state":"active+clean","last_fresh":"2020-08-07T15:08:55.255198+0000","last_change":"2020-08-07T15:08:51.513398+0000","last_active":"2020-08-07T15:08:55.255198+0000","last_peered":"2020-08-07T15:08:55.255198+0000","last_clean":"2020-08-07T15:08:55.255198+0000","last_became_active":"2020-08-07T15:08:41.680448+0000","last_became_peered":"2020-08-07T15:08:41.680448+0000","last_unstale":"2020-08-07T15:08:55.255198+0000","last_undegraded":"2020-08-07T15:08:55.255198+0000","last_fullsized":"2020-08-07T15:08:55.255198+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:51.513319+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.513319+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"109'2360","reported_seq":"2396","reported_epoch":"109","state":"active+clean","last_fresh":"2020-08-07T15:08:56.591199+0000","last_change":"2020-08-07T15:08:24.550496+0000","last_active":"2020-08-07T15:08:56.591199+0000","last_peered":"2020-08-07T15:08:56.591199+0000","last_clean":"2020-08-07T15:08:56.591199+0000","last_became_active":"2020-08-07T15:08:24.549868+0000","last_became_peered":"2020-08-07T15:08:24.549868+0000","last_unstale":"2020-08-07T15:08:56.591199+0000","last_undegraded":"2020-08-07T15:08:56.591199+0000","last_fullsized":"2020-08-07T15:08:56.591199+0000","mapping_epoch":81,"log_start":"109'2353","ondisk_log_start":"109'2353","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19333120,"num_objects":295,"num_object_clones":0,"num_object_copies":885,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2360,"num_write_kb":18880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"107","reported_epoch":"108","state":"active+clean","last_fresh":"2020-08-07T15:08:55.249996+0000","last_change":"2020-08-07T15:08:41.679850+0000","last_active":"2020-08-07T15:08:55.249996+0000","last_peered":"2020-08-07T15:08:55.249996+0000","last_clean":"2020-08-07T15:08:55.249996+0000","last_became_active":"2020-08-07T15:08:41.679321+0000","last_became_peered":"2020-08-07T15:08:41.679321+0000","last_unstale":"2020-08-07T15:08:55.249996+0000","last_undegraded":"2020-08-07T15:08:55.249996+0000","last_fullsized":"2020-08-07T15:08:55.249996+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:34.653843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:34.653843+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"109'2384","reported_seq":"2482","reported_epoch":"109","state":"active+remapped+backfilling","last_fresh":"2020-08-07T15:08:56.504622+0000","last_change":"2020-08-07T15:08:56.283755+0000","last_active":"2020-08-07T15:08:56.504622+0000","last_peered":"2020-08-07T15:08:56.504622+0000","last_clean":"2020-08-07T15:08:54.209887+0000","last_became_active":"2020-08-07T15:08:56.281990+0000","last_became_peered":"2020-08-07T15:08:56.281990+0000","last_unstale":"2020-08-07T15:08:56.504622+0000","last_undegraded":"2020-08-07T15:08:56.504622+0000","last_fullsized":"2020-08-07T15:08:56.504622+0000","mapping_epoch":108,"log_start":"109'2378","ondisk_log_start":"109'2378","created":81,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19529728,"num_objects":298,"num_object_clones":0,"num_object_copies":894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":272,"num_objects_unfound":0,"num_objects_dirty":298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2384,"num_write_kb":19072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1703936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,7],"avail_no_missing":["5","1","7"],"object_location_counts":[{"shards":"1,5,7","objects":298}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"139","reported_epoch":"109","state":"active+clean","last_fresh":"2020-08-07T15:08:56.259368+0000","last_change":"2020-08-07T15:08:41.689973+0000","last_active":"2020-08-07T15:08:56.259368+0000","last_peered":"2020-08-07T15:08:56.259368+0000","last_clean":"2020-08-07T15:08:56.259368+0000","last_became_active":"2020-08-07T15:08:41.689435+0000","last_became_peered":"2020-08-07T15:08:41.689435+0000","last_unstale":"2020-08-07T15:08:56.259368+0000","last_undegraded":"2020-08-07T15:08:56.259368+0000","last_fullsized":"2020-08-07T15:08:56.259368+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.927824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.927824+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"110'2728","reported_seq":"2764","reported_epoch":"110","state":"active+clean","last_fresh":"2020-08-07T15:08:57.692287+0000","last_change":"2020-08-07T15:08:24.554472+0000","last_active":"2020-08-07T15:08:57.692287+0000","last_peered":"2020-08-07T15:08:57.692287+0000","last_clean":"2020-08-07T15:08:57.692287+0000","last_became_active":"2020-08-07T15:08:24.554021+0000","last_became_peered":"2020-08-07T15:08:24.554021+0000","last_unstale":"2020-08-07T15:08:57.692287+0000","last_undegraded":"2020-08-07T15:08:57.692287+0000","last_fullsized":"2020-08-07T15:08:57.692287+0000","mapping_epoch":81,"log_start":"110'2720","ondisk_log_start":"110'2720","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22347776,"num_objects":341,"num_object_clones":0,"num_object_copies":1023,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2728,"num_write_kb":21824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"25","reported_epoch":"108","state":"active+clean","last_fresh":"2020-08-07T15:08:55.255230+0000","last_change":"2020-08-07T15:08:46.710556+0000","last_active":"2020-08-07T15:08:55.255230+0000","last_peered":"2020-08-07T15:08:55.255230+0000","last_clean":"2020-08-07T15:08:55.255230+0000","last_became_active":"2020-08-07T15:08:46.710059+0000","last_became_peered":"2020-08-07T15:08:46.710059+0000","last_unstale":"2020-08-07T15:08:55.255230+0000","last_undegraded":"2020-08-07T15:08:55.255230+0000","last_fullsized":"2020-08-07T15:08:55.255230+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:09.841428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:09.841428+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"109'2248","reported_seq":"2284","reported_epoch":"109","state":"active+clean","last_fresh":"2020-08-07T15:08:56.565955+0000","last_change":"2020-08-07T15:08:24.552802+0000","last_active":"2020-08-07T15:08:56.565955+0000","last_peered":"2020-08-07T15:08:56.565955+0000","last_clean":"2020-08-07T15:08:56.565955+0000","last_became_active":"2020-08-07T15:08:24.552127+0000","last_became_peered":"2020-08-07T15:08:24.552127+0000","last_unstale":"2020-08-07T15:08:56.565955+0000","last_undegraded":"2020-08-07T15:08:56.565955+0000","last_fullsized":"2020-08-07T15:08:56.565955+0000","mapping_epoch":81,"log_start":"109'2240","ondisk_log_start":"109'2240","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18415616,"num_objects":281,"num_object_clones":0,"num_object_copies":843,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2248,"num_write_kb":17984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"110'2320","reported_seq":"2360","reported_epoch":"110","state":"active+clean","last_fresh":"2020-08-07T15:08:57.613236+0000","last_change":"2020-08-07T15:08:46.710643+0000","last_active":"2020-08-07T15:08:57.613236+0000","last_peered":"2020-08-07T15:08:57.613236+0000","last_clean":"2020-08-07T15:08:57.613236+0000","last_became_active":"2020-08-07T15:08:46.710100+0000","last_became_peered":"2020-08-07T15:08:46.710100+0000","last_unstale":"2020-08-07T15:08:57.613236+0000","last_undegraded":"2020-08-07T15:08:57.613236+0000","last_fullsized":"2020-08-07T15:08:57.613236+0000","mapping_epoch":98,"log_start":"110'2312","ondisk_log_start":"110'2312","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19005440,"num_objects":290,"num_object_clones":0,"num_object_copies":870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2320,"num_write_kb":18560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"47","reported_epoch":"108","state":"active+clean","last_fresh":"2020-08-07T15:08:55.251803+0000","last_change":"2020-08-07T15:08:41.678395+0000","last_active":"2020-08-07T15:08:55.251803+0000","last_peered":"2020-08-07T15:08:55.251803+0000","last_clean":"2020-08-07T15:08:55.251803+0000","last_became_active":"2020-08-07T15:08:41.677944+0000","last_became_peered":"2020-08-07T15:08:41.677944+0000","last_unstale":"2020-08-07T15:08:55.251803+0000","last_undegraded":"2020-08-07T15:08:55.251803+0000","last_fullsized":"2020-08-07T15:08:55.251803+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:02.901115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:02.901115+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":311132160,"num_objects":4748,"num_object_clones":0,"num_object_copies":14244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":542,"num_objects_unfound":0,"num_objects_dirty":4748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37980,"num_write_kb":303840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":3514368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":934592512,"data_stored":934592512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":63,"acting":62,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542178,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1166640,"kb_used_data":118056,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93205200,"statfs":{"total":96636764160,"available":95442124800,"internally_reserved":1073741824,"allocated":120889344,"data_stored":120280013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":0.95099999999999996,"15min":0.91200000000000003},"min":{"1min":0.48199999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":3.4159999999999999,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.0940000000000001,"5min":0.98899999999999999,"15min":0.97099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.3559999999999999,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":1.669}]},{"osd":1,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3069999999999999,"15min":1.2989999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":4.4829999999999997,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":1.8180000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.044,"15min":0.99199999999999999},"min":{"1min":0.61799999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":3.4620000000000002,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":1.071}]},{"osd":2,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.179,"15min":1.1439999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":4.423,"5min":4.423,"15min":4.423},"last":1.7849999999999999},{"interface":"front","average":{"1min":1.5,"5min":1.194,"15min":1.143},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":3.5099999999999998,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":0.92200000000000004}]},{"osd":3,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.5900000000000001,"15min":1.6439999999999999},"min":{"1min":0.73799999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.5430000000000001,"5min":5.5019999999999998,"15min":5.5019999999999998},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.329,"15min":1.3380000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.4590000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.86399999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.7849999999999999,"5min":1.377,"15min":1.3089999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":4.835,"5min":4.835,"15min":4.835},"last":1.986},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.3300000000000001,"15min":1.3180000000000001},"min":{"1min":0.76900000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.4470000000000001,"5min":2.778,"15min":2.778},"last":1.4219999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":2.3940000000000001,"5min":30.484000000000002,"15min":35.165999999999997},"min":{"1min":0.96799999999999997,"5min":0.875,"15min":0.875},"max":{"1min":7.4429999999999996,"5min":831.88099999999997,"15min":831.88099999999997},"last":1.869},{"interface":"front","average":{"1min":2.456,"5min":1.6870000000000001,"15min":1.5589999999999999},"min":{"1min":1,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":7.5350000000000001,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":1.7569999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.542,"15min":1.526},"min":{"1min":0.83599999999999997,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.9900000000000002,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":2.9039999999999999},{"interface":"front","average":{"1min":1.9219999999999999,"5min":1.5900000000000001,"15min":1.534},"min":{"1min":1.115,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":4.5170000000000003,"5min":4.5170000000000003,"15min":4.5170000000000003},"last":2.1880000000000002}]}]},{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1163272,"kb_used_data":114688,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93208568,"statfs":{"total":96636764160,"available":95445573632,"internally_reserved":1073741824,"allocated":117440512,"data_stored":116845967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.2490000000000001,"15min":1.3049999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.6439999999999999,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.079,"5min":1.135,"15min":1.145},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":2.9950000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.05}]},{"osd":1,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.95999999999999996,"15min":0.95799999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.9890000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.153},{"interface":"front","average":{"1min":19.103999999999999,"5min":4.9299999999999997,"15min":2.5680000000000001},"min":{"1min":0.75600000000000001,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":372.50799999999998,"5min":372.50799999999998,"15min":372.50799999999998},"last":1.0069999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2,"15min":1.196},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":3.036,"5min":3.036,"15min":3.036},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.093,"5min":1.236,"15min":1.26},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.7440000000000002,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":1.252}]},{"osd":3,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.4870000000000001,"15min":1.5129999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":3.46,"5min":3.46,"15min":3.46},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.3640000000000001,"15min":1.3799999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":3.1970000000000001,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":1.2230000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.5880000000000001,"15min":1.6379999999999999},"min":{"1min":0.72499999999999998,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.7999999999999998,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":1.077},{"interface":"front","average":{"1min":1.653,"5min":1.6679999999999999,"15min":1.6699999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":3.694,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.4259999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.3260000000000001,"15min":1.351},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.6360000000000001,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.393,"5min":1.6910000000000001,"15min":1.7410000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.8279999999999998,"5min":5.327,"15min":5.327},"last":1.4570000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.446,"15min":1.4370000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.7799999999999998,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":14.955,"15min":17.202000000000002},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":3.7309999999999999,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.476}]}]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1253768,"kb_used_data":205184,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93118072,"statfs":{"total":96636764160,"available":95352905728,"internally_reserved":1073741824,"allocated":210108416,"data_stored":209513871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":27.559999999999999,"15min":31.977},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.1820000000000004,"5min":497.04000000000002,"15min":497.04000000000002},"last":0.78200000000000003},{"interface":"front","average":{"1min":44.500999999999998,"5min":9.8049999999999997,"15min":4.0220000000000002},"min":{"1min":0.64200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":993.84500000000003,"5min":993.84500000000003,"15min":993.84500000000003},"last":0.81999999999999995}]},{"osd":2,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":0.98799999999999999,"15min":0.92800000000000005},"min":{"1min":0.57399999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":4.585,"5min":4.585,"15min":4.585},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.2410000000000001,"15min":1.216},"min":{"1min":0.51800000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":4.149,"5min":4.149,"15min":4.149},"last":0.50600000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.962,"5min":1.4410000000000001,"15min":1.3540000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":10.603999999999999,"5min":10.603999999999999,"15min":10.603999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":2.1909999999999998,"5min":1.601,"15min":1.5029999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":10.638999999999999,"5min":10.638999999999999,"15min":10.638999999999999},"last":0.86599999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":2.1869999999999998,"5min":1.464,"15min":1.3440000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":10.69,"5min":10.69,"15min":10.69},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.264,"15min":1.2210000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":4.9500000000000002,"5min":4.9500000000000002,"15min":4.9500000000000002},"last":0.93700000000000006}]},{"osd":5,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":2.0630000000000002,"5min":10.819000000000001,"15min":12.278},"min":{"1min":0.73399999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":10.720000000000001,"5min":213.63800000000001,"15min":213.63800000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.258,"15min":1.212},"min":{"1min":0.82899999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":4.6269999999999998,"5min":4.6269999999999998,"15min":4.6269999999999998},"last":0.95499999999999996}]},{"osd":6,"last update":"Fri Aug 7 15:08:24 2020","interfaces":[{"interface":"back","average":{"1min":1.7589999999999999,"5min":1.593,"15min":1.5649999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":4.5259999999999998,"5min":4.5259999999999998,"15min":4.5259999999999998},"last":0.88500000000000001},{"interface":"front","average":{"1min":2.3690000000000002,"5min":1.5620000000000001,"15min":1.4279999999999999},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":10.792,"5min":10.792,"15min":10.792},"last":0.79600000000000004}]},{"osd":7,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.8129999999999999,"5min":1.6539999999999999,"15min":1.6279999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":4.5549999999999997,"5min":4.5549999999999997,"15min":4.5549999999999997},"last":0.99399999999999999},{"interface":"front","average":{"1min":2.2719999999999998,"5min":1.6379999999999999,"15min":1.532},"min":{"1min":0.998,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":10.752000000000001,"5min":10.752000000000001,"15min":10.752000000000001},"last":1.0349999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1141320,"kb_used_data":92736,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93230520,"statfs":{"total":96636764160,"available":95468052480,"internally_reserved":1073741824,"allocated":94961664,"data_stored":94367119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:08:24 2020","interfaces":[{"interface":"back","average":{"1min":81.266000000000005,"5min":17.033999999999999,"15min":6.3289999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":845.57100000000003,"5min":845.57100000000003,"15min":845.57100000000003},"last":0.61399999999999999},{"interface":"front","average":{"1min":1.083,"5min":0.96299999999999997,"15min":0.94299999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":5.5970000000000004,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":0.63200000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.0069999999999999,"15min":0.95399999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":4.1600000000000001,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.0049999999999999,"5min":26.393000000000001,"15min":30.623999999999999},"min":{"1min":0.54200000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.1280000000000001,"5min":606.18499999999995,"15min":606.18499999999995},"last":0.69899999999999995}]},{"osd":3,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.0549999999999999,"15min":0.97499999999999998},"min":{"1min":0.64000000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":3.1779999999999999,"5min":3.1779999999999999,"15min":3.1779999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.927,"5min":1.319,"15min":1.218},"min":{"1min":0.78700000000000003,"5min":0.621,"15min":0.621},"max":{"1min":4.21,"5min":4.21,"15min":4.21},"last":0.82499999999999996}]},{"osd":4,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.238,"15min":1.1899999999999999},"min":{"1min":0.73899999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":3.0870000000000002,"5min":3.0870000000000002,"15min":3.0870000000000002},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.6890000000000001,"5min":1.3440000000000001,"15min":1.286},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":4.0960000000000001,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":1.131}]},{"osd":5,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":39.531999999999996,"5min":8.8879999999999999,"15min":3.7799999999999998},"min":{"1min":0.76600000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":501.267,"5min":501.267,"15min":501.267},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.24,"15min":1.1499999999999999},"min":{"1min":0.874,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":6.0010000000000003,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":1.292}]},{"osd":6,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.7729999999999999,"5min":7.2450000000000001,"15min":8.157},"min":{"1min":0.81100000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":4.1310000000000002,"5min":124.738,"15min":124.738},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.47,"5min":15.215999999999999,"15min":17.507000000000001},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":3.7120000000000002,"5min":294.94,"15min":294.94},"last":1.252}]},{"osd":7,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.552,"15min":1.5740000000000001},"min":{"1min":0.96199999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.3929999999999998,"5min":6.468,"15min":6.468},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.6879999999999999,"5min":1.446,"15min":1.4059999999999999},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":3.117,"5min":3.117,"15min":3.117},"last":1.357}]}]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1103956,"kb_used_data":55372,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93267884,"statfs":{"total":96636764160,"available":95506313216,"internally_reserved":1073741824,"allocated":56700928,"data_stored":56099010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6,2,4,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.153,"15min":1.129},"min":{"1min":0.505,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.4909999999999997,"5min":4.4909999999999997,"15min":4.4909999999999997},"last":3.5219999999999998},{"interface":"front","average":{"1min":1.284,"5min":1.036,"15min":0.995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":3.6520000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":1.2889999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":2.5219999999999998,"5min":1.3779999999999999,"15min":1.1870000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":18.672000000000001,"5min":18.672000000000001,"15min":18.672000000000001},"last":2.2170000000000001},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.357,"15min":1.359},"min":{"1min":0.70699999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":5.1130000000000004,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":3.585}]},{"osd":3,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.216,"15min":1.1799999999999999},"min":{"1min":0.65300000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":2.661},{"interface":"front","average":{"1min":1.419,"5min":1.528,"15min":1.546},"min":{"1min":0.59999999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":5.3339999999999996,"5min":8.2210000000000001,"15min":8.2210000000000001},"last":1.7470000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":2.3319999999999999,"5min":1.3540000000000001,"15min":1.1910000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":19.783000000000001,"5min":19.783000000000001,"15min":19.783000000000001},"last":2.1880000000000002},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.5589999999999999,"15min":1.615},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.891,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":3.5569999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.2789999999999999,"15min":1.2669999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.7730000000000001,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.5249999999999999,"5min":20.725000000000001,"15min":23.925000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":4.4790000000000001,"5min":415.40699999999998,"15min":415.40699999999998},"last":2.532}]},{"osd":6,"last update":"Fri Aug 7 15:08:25 2020","interfaces":[{"interface":"back","average":{"1min":2.5089999999999999,"5min":1.478,"15min":1.306},"min":{"1min":0.89000000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":20.035,"5min":20.035,"15min":20.035},"last":2.4929999999999999},{"interface":"front","average":{"1min":44.033000000000001,"5min":9.8629999999999995,"15min":4.1680000000000001},"min":{"1min":0.93700000000000006,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":751.70000000000005,"5min":751.70000000000005,"15min":751.70000000000005},"last":1.605}]},{"osd":7,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":3.5409999999999999,"5min":2.101,"15min":1.861},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":35.548999999999999,"5min":35.548999999999999,"15min":35.548999999999999},"last":5.2610000000000001},{"interface":"front","average":{"1min":3.4460000000000002,"5min":1.8859999999999999,"15min":1.6259999999999999},"min":{"1min":1.004,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":35.646999999999998,"5min":35.646999999999998,"15min":35.646999999999998},"last":1.722}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1110676,"kb_used_data":62092,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93261164,"statfs":{"total":96636764160,"available":95499431936,"internally_reserved":1073741824,"allocated":63582208,"data_stored":62980290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.044,"5min":1.411,"15min":1.3049999999999999},"min":{"1min":0.55600000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":8.6020000000000003,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.45,"5min":1.599,"15min":1.6240000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":4.9119999999999999,"5min":13.067,"15min":13.067},"last":0.65400000000000003}]},{"osd":1,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":16.382999999999999,"5min":4.7949999999999999,"15min":2.8639999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":270.68700000000001,"5min":270.68700000000001,"15min":270.68700000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":2.7210000000000001,"5min":2.153,"15min":2.0590000000000002},"min":{"1min":0.71599999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":14.349,"5min":14.349,"15min":14.349},"last":0.55100000000000005}]},{"osd":2,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":1.7470000000000001,"5min":1.2669999999999999,"15min":1.1870000000000001},"min":{"1min":0.64900000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":4.9580000000000002,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":0.89500000000000002},{"interface":"front","average":{"1min":4.5220000000000002,"5min":2.2519999999999998,"15min":1.8740000000000001},"min":{"1min":0.72399999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":51.530000000000001,"5min":51.530000000000001,"15min":51.530000000000001},"last":0.96399999999999997}]},{"osd":4,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":47.652999999999999,"5min":10.888,"15min":4.7599999999999998},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":506.21300000000002,"5min":506.21300000000002,"15min":506.21300000000002},"last":0.76200000000000001},{"interface":"front","average":{"1min":29.771000000000001,"5min":31.634,"15min":31.943999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":505.738,"5min":677.88599999999997,"15min":677.88599999999997},"last":1.0129999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.012,"5min":1.6160000000000001,"15min":1.55},"min":{"1min":0.96399999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":5.0049999999999999,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.03},{"interface":"front","average":{"1min":2.2210000000000001,"5min":1.859,"15min":1.7989999999999999},"min":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":5.226,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":1.155}]},{"osd":6,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":1.99,"5min":1.623,"15min":1.5620000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":5.0389999999999997,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.9710000000000001,"5min":1.9830000000000001,"15min":1.9850000000000001},"min":{"1min":1.0369999999999999,"5min":0.876,"15min":0.876},"max":{"1min":4.266,"5min":5.6189999999999998,"15min":5.6189999999999998},"last":1.0009999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.0110000000000001,"5min":2.1309999999999998,"15min":2.1509999999999998},"min":{"1min":0.96999999999999997,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":3.79,"5min":5.734,"15min":5.734},"last":0.92400000000000004},{"interface":"front","average":{"1min":2.6200000000000001,"5min":2.105,"15min":2.0190000000000001},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":8.6790000000000003,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":0.98299999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1196196,"kb_used_data":147612,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93175644,"statfs":{"total":96636764160,"available":95411859456,"internally_reserved":1073741824,"allocated":151154688,"data_stored":150552770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":1.125,"15min":1.1619999999999999},"min":{"1min":0.47699999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.3999999999999999,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":1.26},{"interface":"front","average":{"1min":1.349,"5min":0.94899999999999995,"15min":0.88200000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":6.3970000000000002,"5min":6.3970000000000002,"15min":6.3970000000000002},"last":1.2869999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.304,"15min":1.337},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.4380000000000002,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":1.339},{"interface":"front","average":{"1min":32.585999999999999,"5min":7.6180000000000003,"15min":3.4569999999999999},"min":{"1min":0.67000000000000004,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":689.846,"5min":689.846,"15min":689.846},"last":1.3089999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.46,"15min":1.454},"min":{"1min":0.56399999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":6.8730000000000002,"5min":6.8730000000000002,"15min":6.8730000000000002},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.5669999999999999,"15min":1.532},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":7.1360000000000001,"5min":7.1360000000000001,"15min":7.1360000000000001},"last":1.218}]},{"osd":3,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":12.619999999999999,"5min":3.73,"15min":2.2480000000000002},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":247.202,"5min":247.202,"15min":247.202},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.583,"5min":1.3169999999999999,"15min":1.2729999999999999},"min":{"1min":0.67800000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":6.431,"5min":6.431,"15min":6.431},"last":0.91400000000000003}]},{"osd":5,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":46.786999999999999,"5min":10.659000000000001,"15min":4.6379999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":990.71600000000001,"5min":990.71600000000001,"15min":990.71600000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.748,"5min":1.651,"15min":1.635},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":3.9329999999999998,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":1.3740000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.7070000000000001,"5min":1.7729999999999999,"15min":1.784},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":6.9749999999999996,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.524,"5min":15.592000000000001,"15min":17.936},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":6.7729999999999997,"5min":389.14600000000002,"15min":389.14600000000002},"last":0.83299999999999996}]},{"osd":7,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":13.378,"5min":3.802,"15min":2.206},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":252.798,"5min":252.798,"15min":252.798},"last":1.173},{"interface":"front","average":{"1min":44.676000000000002,"5min":10.252000000000001,"15min":4.5140000000000002},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":501.93000000000001,"5min":501.93000000000001,"15min":501.93000000000001},"last":0.94199999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1161644,"kb_used_data":113060,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93210196,"statfs":{"total":96636764160,"available":95447240704,"internally_reserved":1073741824,"allocated":115773440,"data_stored":115171522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,7,5,0,1,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0580000000000001,"15min":1.0700000000000001},"min":{"1min":0.49199999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.5169999999999999,"5min":2.766,"15min":2.766},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.2250000000000001,"5min":0.88900000000000001,"15min":0.83299999999999996},"min":{"1min":0.54900000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.393}]},{"osd":1,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":47.365000000000002,"5min":10.215999999999999,"15min":4.024},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":787.22900000000004,"5min":787.22900000000004,"15min":787.22900000000004},"last":1.349},{"interface":"front","average":{"1min":1.071,"5min":1.1000000000000001,"15min":1.105},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.8380000000000001,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":1.893}]},{"osd":2,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":7.5659999999999998,"15min":8.6679999999999993},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.4950000000000001,"5min":168.56899999999999,"15min":168.56899999999999},"last":1.78},{"interface":"front","average":{"1min":1.2929999999999999,"5min":26.763000000000002,"15min":31.007999999999999},"min":{"1min":0.79700000000000004,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":3.2389999999999999,"5min":676.27099999999996,"15min":676.27099999999996},"last":1.444}]},{"osd":3,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.196,"15min":1.1439999999999999},"min":{"1min":0.89000000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":3.5649999999999999,"5min":3.5649999999999999,"15min":3.5649999999999999},"last":1.472},{"interface":"front","average":{"1min":43.073,"5min":9.5850000000000009,"15min":4.0030000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":712.98699999999997,"5min":712.98699999999997,"15min":712.98699999999997},"last":2.1040000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":40.962000000000003,"15min":47.582000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.4980000000000002,"5min":530.30799999999999,"15min":530.30799999999999},"last":1.9570000000000001},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.278,"15min":1.296},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.6259999999999999,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":2.1749999999999998}]},{"osd":6,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":40.569000000000003,"15min":47.128999999999998},"min":{"1min":0.76800000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.6699999999999999,"5min":754.31299999999999,"15min":754.31299999999999},"last":2.141},{"interface":"front","average":{"1min":1.1899999999999999,"5min":21.050999999999998,"15min":24.361000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.7809999999999999,"5min":517.26700000000005,"15min":517.26700000000005},"last":1.925}]},{"osd":7,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.3560000000000001,"15min":1.3240000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":3.9079999999999999,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.641},{"interface":"front","average":{"1min":1.411,"5min":1.5349999999999999,"15min":1.556},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":3.9390000000000001,"5min":3.9390000000000001,"15min":3.9390000000000001},"last":4.5}]}]}]}} 2020-08-07T15:09:02.159 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:09:02.176 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:09:02.177 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 34 16 42487 42471 9.758 10.875 0.00472154 0.0127959 2020-08-07T15:09:02.177 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 35 16 42532 42516 9.48924 0.351562 0.69662 0.0128802 2020-08-07T15:09:02.177 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:09:02.212 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:02.110+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:02.231 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:02.215+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:02.278 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 36 16 43046 43030 9.33719 4.01562 0.00963199 0.0133824 2020-08-07T15:09:02.324 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:02.314+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:02.398 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:09:02.399 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:09:02.399 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2020-08-07T15:09:02.399 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:09:02.407 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:09:02.442 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:09:02.454 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:02.448+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:02.478 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:09:02.514 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:09:02.532 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:02.527+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:02.559 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:09:02.590 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:09:02.626 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:09:02.630 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:02.629+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:02.639 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:09:02.639 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:09:02.639 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:09:02.640 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:09:02.640 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:09:02.651 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:09:02.661 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:09:02.674 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:09:02.675 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:09:02.686 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2020-08-07T15:09:02.643633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":345702400,"num_objects":5277,"num_object_clones":0,"num_object_copies":15831,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42200,"num_write_kb":337600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":635,"num_bytes_recovered":41525248,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":135,"acting":134,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9453604,"kb_used_data":1064932,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":745521116,"statfs":{"total":773094113280,"available":763413622784,"internally_reserved":8589934592,"allocated":1090490368,"data_stored":1085429145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":34570240,"num_objects":527,"num_object_clones":0,"num_object_copies":1581,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-542,"num_objects_unfound":0,"num_objects_dirty":527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4220,"num_write_kb":33760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":580,"num_bytes_recovered":38010880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000844"},"pg_stats":[{"pgid":"1.14","version":"16'2","reported_seq":"43","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.515159+0000","last_change":"2020-08-07T15:08:42.533999+0000","last_active":"2020-08-07T15:09:00.515159+0000","last_peered":"2020-08-07T15:09:00.515159+0000","last_clean":"2020-08-07T15:09:00.515159+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:09:00.515159+0000","last_undegraded":"2020-08-07T15:09:00.515159+0000","last_fullsized":"2020-08-07T15:09:00.515159+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:42.533921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:42.533921+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"79","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.292021+0000","last_change":"2020-08-07T15:08:58.212828+0000","last_active":"2020-08-07T15:09:00.292021+0000","last_peered":"2020-08-07T15:09:00.292021+0000","last_clean":"2020-08-07T15:09:00.292021+0000","last_became_active":"2020-08-07T15:08:58.211785+0000","last_became_peered":"2020-08-07T15:08:58.211785+0000","last_unstale":"2020-08-07T15:09:00.292021+0000","last_undegraded":"2020-08-07T15:09:00.292021+0000","last_fullsized":"2020-08-07T15:09:00.292021+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.141010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.141010+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"43","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.289984+0000","last_change":"2020-08-07T15:08:55.078015+0000","last_active":"2020-08-07T15:09:00.289984+0000","last_peered":"2020-08-07T15:09:00.289984+0000","last_clean":"2020-08-07T15:09:00.289984+0000","last_became_active":"2020-08-07T15:08:41.685106+0000","last_became_peered":"2020-08-07T15:08:41.685106+0000","last_unstale":"2020-08-07T15:09:00.289984+0000","last_undegraded":"2020-08-07T15:09:00.289984+0000","last_fullsized":"2020-08-07T15:09:00.289984+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:55.077924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:55.077924+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"122","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.290319+0000","last_change":"2020-08-07T15:08:43.544399+0000","last_active":"2020-08-07T15:09:00.290319+0000","last_peered":"2020-08-07T15:09:00.290319+0000","last_clean":"2020-08-07T15:09:00.290319+0000","last_became_active":"2020-08-07T15:08:41.675776+0000","last_became_peered":"2020-08-07T15:08:41.675776+0000","last_unstale":"2020-08-07T15:09:00.290319+0000","last_undegraded":"2020-08-07T15:09:00.290319+0000","last_fullsized":"2020-08-07T15:09:00.290319+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.544324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.544324+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"150","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.289931+0000","last_change":"2020-08-07T15:08:51.599032+0000","last_active":"2020-08-07T15:09:00.289931+0000","last_peered":"2020-08-07T15:09:00.289931+0000","last_clean":"2020-08-07T15:09:00.289931+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:09:00.289931+0000","last_undegraded":"2020-08-07T15:09:00.289931+0000","last_fullsized":"2020-08-07T15:09:00.289931+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:51.598947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.598947+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"104","reported_epoch":"109","state":"active+clean","last_fresh":"2020-08-07T15:08:56.259493+0000","last_change":"2020-08-07T15:08:11.840861+0000","last_active":"2020-08-07T15:08:56.259493+0000","last_peered":"2020-08-07T15:08:56.259493+0000","last_clean":"2020-08-07T15:08:56.259493+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:08:56.259493+0000","last_undegraded":"2020-08-07T15:08:56.259493+0000","last_fullsized":"2020-08-07T15:08:56.259493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:11.840807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:11.840807+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"99","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.286421+0000","last_change":"2020-08-07T15:08:39.558760+0000","last_active":"2020-08-07T15:09:00.286421+0000","last_peered":"2020-08-07T15:09:00.286421+0000","last_clean":"2020-08-07T15:09:00.286421+0000","last_became_active":"2020-08-07T15:08:36.615358+0000","last_became_peered":"2020-08-07T15:08:36.615358+0000","last_unstale":"2020-08-07T15:09:00.286421+0000","last_undegraded":"2020-08-07T15:09:00.286421+0000","last_fullsized":"2020-08-07T15:09:00.286421+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:39.558677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:39.558677+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"114'2584","reported_seq":"2626","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.680980+0000","last_change":"2020-08-07T15:08:46.715869+0000","last_active":"2020-08-07T15:09:01.680980+0000","last_peered":"2020-08-07T15:09:01.680980+0000","last_clean":"2020-08-07T15:09:01.680980+0000","last_became_active":"2020-08-07T15:08:46.708112+0000","last_became_peered":"2020-08-07T15:08:46.708112+0000","last_unstale":"2020-08-07T15:09:01.680980+0000","last_undegraded":"2020-08-07T15:09:01.680980+0000","last_fullsized":"2020-08-07T15:09:01.680980+0000","mapping_epoch":98,"log_start":"114'2581","ondisk_log_start":"114'2581","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21168128,"num_objects":323,"num_object_clones":0,"num_object_copies":969,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2584,"num_write_kb":20672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"111","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.292329+0000","last_change":"2020-08-07T15:08:40.057705+0000","last_active":"2020-08-07T15:09:00.292329+0000","last_peered":"2020-08-07T15:09:00.292329+0000","last_clean":"2020-08-07T15:09:00.292329+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:09:00.292329+0000","last_undegraded":"2020-08-07T15:09:00.292329+0000","last_fullsized":"2020-08-07T15:09:00.292329+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:40.057622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:40.057622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"68","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.515303+0000","last_change":"2020-08-07T15:08:53.756554+0000","last_active":"2020-08-07T15:09:00.515303+0000","last_peered":"2020-08-07T15:09:00.515303+0000","last_clean":"2020-08-07T15:09:00.515303+0000","last_became_active":"2020-08-07T15:08:46.710404+0000","last_became_peered":"2020-08-07T15:08:46.710404+0000","last_unstale":"2020-08-07T15:09:00.515303+0000","last_undegraded":"2020-08-07T15:09:00.515303+0000","last_fullsized":"2020-08-07T15:09:00.515303+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:53.756343+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:53.756343+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"114'2824","reported_seq":"3509","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.322637+0000","last_change":"2020-08-07T15:09:01.304273+0000","last_active":"2020-08-07T15:09:01.322637+0000","last_peered":"2020-08-07T15:09:01.322637+0000","last_clean":"2020-08-07T15:09:01.322637+0000","last_became_active":"2020-08-07T15:09:01.293427+0000","last_became_peered":"2020-08-07T15:09:01.293427+0000","last_unstale":"2020-08-07T15:09:01.322637+0000","last_undegraded":"2020-08-07T15:09:01.322637+0000","last_fullsized":"2020-08-07T15:09:01.322637+0000","mapping_epoch":113,"log_start":"114'2815","ondisk_log_start":"114'2815","created":81,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23134208,"num_objects":353,"num_object_clones":0,"num_object_copies":1059,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2824,"num_write_kb":22592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":20971520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"108","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.296117+0000","last_change":"2020-08-07T15:08:27.906894+0000","last_active":"2020-08-07T15:09:00.296117+0000","last_peered":"2020-08-07T15:09:00.296117+0000","last_clean":"2020-08-07T15:09:00.296117+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:09:00.296117+0000","last_undegraded":"2020-08-07T15:09:00.296117+0000","last_fullsized":"2020-08-07T15:09:00.296117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:27.906822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:27.906822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"109","reported_epoch":"109","state":"active+clean","last_fresh":"2020-08-07T15:08:56.259472+0000","last_change":"2020-08-07T15:08:43.899342+0000","last_active":"2020-08-07T15:08:56.259472+0000","last_peered":"2020-08-07T15:08:56.259472+0000","last_clean":"2020-08-07T15:08:56.259472+0000","last_became_active":"2020-08-07T15:08:36.622209+0000","last_became_peered":"2020-08-07T15:08:36.622209+0000","last_unstale":"2020-08-07T15:08:56.259472+0000","last_undegraded":"2020-08-07T15:08:56.259472+0000","last_fullsized":"2020-08-07T15:08:56.259472+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.899280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.899280+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"114'2616","reported_seq":"2657","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.421157+0000","last_change":"2020-08-07T15:08:24.549715+0000","last_active":"2020-08-07T15:09:01.421157+0000","last_peered":"2020-08-07T15:09:01.421157+0000","last_clean":"2020-08-07T15:09:01.421157+0000","last_became_active":"2020-08-07T15:08:24.549245+0000","last_became_peered":"2020-08-07T15:08:24.549245+0000","last_unstale":"2020-08-07T15:09:01.421157+0000","last_undegraded":"2020-08-07T15:09:01.421157+0000","last_fullsized":"2020-08-07T15:09:01.421157+0000","mapping_epoch":81,"log_start":"114'2610","ondisk_log_start":"114'2610","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21430272,"num_objects":327,"num_object_clones":0,"num_object_copies":981,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2616,"num_write_kb":20928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"108","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.295286+0000","last_change":"2020-08-07T15:08:13.867355+0000","last_active":"2020-08-07T15:09:00.295286+0000","last_peered":"2020-08-07T15:09:00.295286+0000","last_clean":"2020-08-07T15:09:00.295286+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:09:00.295286+0000","last_undegraded":"2020-08-07T15:09:00.295286+0000","last_fullsized":"2020-08-07T15:09:00.295286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:13.867148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:13.867148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"109","reported_epoch":"109","state":"active+clean","last_fresh":"2020-08-07T15:08:56.259193+0000","last_change":"2020-08-07T15:08:50.686578+0000","last_active":"2020-08-07T15:08:56.259193+0000","last_peered":"2020-08-07T15:08:56.259193+0000","last_clean":"2020-08-07T15:08:56.259193+0000","last_became_active":"2020-08-07T15:08:36.633406+0000","last_became_peered":"2020-08-07T15:08:36.633406+0000","last_unstale":"2020-08-07T15:08:56.259193+0000","last_undegraded":"2020-08-07T15:08:56.259193+0000","last_fullsized":"2020-08-07T15:08:56.259193+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:50.686496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:50.686496+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"114'2512","reported_seq":"2552","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.594921+0000","last_change":"2020-08-07T15:08:24.557398+0000","last_active":"2020-08-07T15:09:01.594921+0000","last_peered":"2020-08-07T15:09:01.594921+0000","last_clean":"2020-08-07T15:09:01.594921+0000","last_became_active":"2020-08-07T15:08:24.557022+0000","last_became_peered":"2020-08-07T15:08:24.557022+0000","last_unstale":"2020-08-07T15:09:01.594921+0000","last_undegraded":"2020-08-07T15:09:01.594921+0000","last_fullsized":"2020-08-07T15:09:01.594921+0000","mapping_epoch":81,"log_start":"114'2509","ondisk_log_start":"114'2509","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20578304,"num_objects":314,"num_object_clones":0,"num_object_copies":942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2512,"num_write_kb":20096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"140","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.295184+0000","last_change":"2020-08-07T15:08:43.036067+0000","last_active":"2020-08-07T15:09:00.295184+0000","last_peered":"2020-08-07T15:09:00.295184+0000","last_clean":"2020-08-07T15:09:00.295184+0000","last_became_active":"2020-08-07T15:08:36.627426+0000","last_became_peered":"2020-08-07T15:08:36.627426+0000","last_unstale":"2020-08-07T15:09:00.295184+0000","last_undegraded":"2020-08-07T15:09:00.295184+0000","last_fullsized":"2020-08-07T15:09:00.295184+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.035987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.035987+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"108","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.290414+0000","last_change":"2020-08-07T15:08:26.317430+0000","last_active":"2020-08-07T15:09:00.290414+0000","last_peered":"2020-08-07T15:09:00.290414+0000","last_clean":"2020-08-07T15:09:00.290414+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:09:00.290414+0000","last_undegraded":"2020-08-07T15:09:00.290414+0000","last_fullsized":"2020-08-07T15:09:00.290414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:26.317351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:26.317351+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"110'2496","reported_seq":"2533","reported_epoch":"110","state":"active+clean","last_fresh":"2020-08-07T15:08:57.667912+0000","last_change":"2020-08-07T15:08:24.552570+0000","last_active":"2020-08-07T15:08:57.667912+0000","last_peered":"2020-08-07T15:08:57.667912+0000","last_clean":"2020-08-07T15:08:57.667912+0000","last_became_active":"2020-08-07T15:08:24.552105+0000","last_became_peered":"2020-08-07T15:08:24.552105+0000","last_unstale":"2020-08-07T15:08:57.667912+0000","last_undegraded":"2020-08-07T15:08:57.667912+0000","last_fullsized":"2020-08-07T15:08:57.667912+0000","mapping_epoch":81,"log_start":"110'2489","ondisk_log_start":"110'2489","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20447232,"num_objects":312,"num_object_clones":0,"num_object_copies":936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2496,"num_write_kb":19968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"108","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.515189+0000","last_change":"2020-08-07T15:07:36.864560+0000","last_active":"2020-08-07T15:09:00.515189+0000","last_peered":"2020-08-07T15:09:00.515189+0000","last_clean":"2020-08-07T15:09:00.515189+0000","last_became_active":"2020-08-07T15:06:23.223489+0000","last_became_peered":"2020-08-07T15:06:23.223489+0000","last_unstale":"2020-08-07T15:09:00.515189+0000","last_undegraded":"2020-08-07T15:09:00.515189+0000","last_fullsized":"2020-08-07T15:09:00.515189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:07:36.864483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.864483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"156","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.294032+0000","last_change":"2020-08-07T15:08:42.069637+0000","last_active":"2020-08-07T15:09:00.294032+0000","last_peered":"2020-08-07T15:09:00.294032+0000","last_clean":"2020-08-07T15:09:00.294032+0000","last_became_active":"2020-08-07T15:08:36.628365+0000","last_became_peered":"2020-08-07T15:08:36.628365+0000","last_unstale":"2020-08-07T15:09:00.294032+0000","last_undegraded":"2020-08-07T15:09:00.294032+0000","last_fullsized":"2020-08-07T15:09:00.294032+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:42.069585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:42.069585+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"114'2824","reported_seq":"2865","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.426013+0000","last_change":"2020-08-07T15:08:24.550789+0000","last_active":"2020-08-07T15:09:01.426013+0000","last_peered":"2020-08-07T15:09:01.426013+0000","last_clean":"2020-08-07T15:09:01.426013+0000","last_became_active":"2020-08-07T15:08:24.550222+0000","last_became_peered":"2020-08-07T15:08:24.550222+0000","last_unstale":"2020-08-07T15:09:01.426013+0000","last_undegraded":"2020-08-07T15:09:01.426013+0000","last_fullsized":"2020-08-07T15:09:01.426013+0000","mapping_epoch":81,"log_start":"114'2818","ondisk_log_start":"114'2818","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23134208,"num_objects":353,"num_object_clones":0,"num_object_copies":1059,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2824,"num_write_kb":22592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"108","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.290321+0000","last_change":"2020-08-07T15:08:30.339852+0000","last_active":"2020-08-07T15:09:00.290321+0000","last_peered":"2020-08-07T15:09:00.290321+0000","last_clean":"2020-08-07T15:09:00.290321+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:09:00.290321+0000","last_undegraded":"2020-08-07T15:09:00.290321+0000","last_fullsized":"2020-08-07T15:09:00.290321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:08:30.339785+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"106","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.292512+0000","last_change":"2020-08-07T15:08:38.386792+0000","last_active":"2020-08-07T15:09:00.292512+0000","last_peered":"2020-08-07T15:09:00.292512+0000","last_clean":"2020-08-07T15:09:00.292512+0000","last_became_active":"2020-08-07T15:08:36.618164+0000","last_became_peered":"2020-08-07T15:08:36.618164+0000","last_unstale":"2020-08-07T15:09:00.292512+0000","last_undegraded":"2020-08-07T15:09:00.292512+0000","last_fullsized":"2020-08-07T15:09:00.292512+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:38.386739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:38.386739+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"114'2768","reported_seq":"2809","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.674515+0000","last_change":"2020-08-07T15:08:24.558242+0000","last_active":"2020-08-07T15:09:01.674515+0000","last_peered":"2020-08-07T15:09:01.674515+0000","last_clean":"2020-08-07T15:09:01.674515+0000","last_became_active":"2020-08-07T15:08:24.557354+0000","last_became_peered":"2020-08-07T15:08:24.557354+0000","last_unstale":"2020-08-07T15:09:01.674515+0000","last_undegraded":"2020-08-07T15:09:01.674515+0000","last_fullsized":"2020-08-07T15:09:01.674515+0000","mapping_epoch":81,"log_start":"114'2765","ondisk_log_start":"114'2765","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22675456,"num_objects":346,"num_object_clones":0,"num_object_copies":1038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2768,"num_write_kb":22144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"105","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.295909+0000","last_change":"2020-08-07T15:08:15.887617+0000","last_active":"2020-08-07T15:09:00.295909+0000","last_peered":"2020-08-07T15:09:00.295909+0000","last_clean":"2020-08-07T15:09:00.295909+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:09:00.295909+0000","last_undegraded":"2020-08-07T15:09:00.295909+0000","last_fullsized":"2020-08-07T15:09:00.295909+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:15.887513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:15.887513+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"132","reported_epoch":"110","state":"peering","last_fresh":"2020-08-07T15:08:57.169641+0000","last_change":"2020-08-07T15:08:57.169893+0000","last_active":"2020-08-07T15:08:57.169641+0000","last_peered":"2020-08-07T15:08:56.264303+0000","last_clean":"2020-08-07T15:08:53.224498+0000","last_became_active":"2020-08-07T15:08:56.263952+0000","last_became_peered":"2020-08-07T15:08:56.263952+0000","last_unstale":"2020-08-07T15:08:57.169641+0000","last_undegraded":"2020-08-07T15:08:57.169641+0000","last_fullsized":"2020-08-07T15:08:57.169641+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.927824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.927824+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"114'2464","reported_seq":"2504","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.426624+0000","last_change":"2020-08-07T15:08:24.552962+0000","last_active":"2020-08-07T15:09:01.426624+0000","last_peered":"2020-08-07T15:09:01.426624+0000","last_clean":"2020-08-07T15:09:01.426624+0000","last_became_active":"2020-08-07T15:08:24.552070+0000","last_became_peered":"2020-08-07T15:08:24.552070+0000","last_unstale":"2020-08-07T15:09:01.426624+0000","last_undegraded":"2020-08-07T15:09:01.426624+0000","last_fullsized":"2020-08-07T15:09:01.426624+0000","mapping_epoch":81,"log_start":"114'2456","ondisk_log_start":"114'2456","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20185088,"num_objects":308,"num_object_clones":0,"num_object_copies":924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2464,"num_write_kb":19712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"83","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.290206+0000","last_change":"2020-08-07T15:08:39.409159+0000","last_active":"2020-08-07T15:09:00.290206+0000","last_peered":"2020-08-07T15:09:00.290206+0000","last_clean":"2020-08-07T15:09:00.290206+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:09:00.290206+0000","last_undegraded":"2020-08-07T15:09:00.290206+0000","last_fullsized":"2020-08-07T15:09:00.290206+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:39.409063+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_clean_scrub_stamp":"2020-08-07T15:08:39.409063+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"114'2760","reported_seq":"2801","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.409800+0000","last_change":"2020-08-07T15:08:24.554063+0000","last_active":"2020-08-07T15:09:01.409800+0000","last_peered":"2020-08-07T15:09:01.409800+0000","last_clean":"2020-08-07T15:09:01.409800+0000","last_became_active":"2020-08-07T15:08:24.553668+0000","last_became_peered":"2020-08-07T15:08:24.553668+0000","last_unstale":"2020-08-07T15:09:01.409800+0000","last_undegraded":"2020-08-07T15:09:01.409800+0000","last_fullsized":"2020-08-07T15:09:01.409800+0000","mapping_epoch":81,"log_start":"114'2754","ondisk_log_start":"114'2754","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22609920,"num_objects":345,"num_object_clones":0,"num_object_copies":1035,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2760,"num_write_kb":22080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"80","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.289975+0000","last_change":"2020-08-07T15:07:33.177491+0000","last_active":"2020-08-07T15:09:00.289975+0000","last_peered":"2020-08-07T15:09:00.289975+0000","last_clean":"2020-08-07T15:09:00.289975+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:09:00.289975+0000","last_undegraded":"2020-08-07T15:09:00.289975+0000","last_fullsized":"2020-08-07T15:09:00.289975+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:33.177412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:33.177412+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"114'2488","reported_seq":"2529","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.467392+0000","last_change":"2020-08-07T15:08:46.709789+0000","last_active":"2020-08-07T15:09:01.467392+0000","last_peered":"2020-08-07T15:09:01.467392+0000","last_clean":"2020-08-07T15:09:01.467392+0000","last_became_active":"2020-08-07T15:08:46.708674+0000","last_became_peered":"2020-08-07T15:08:46.708674+0000","last_unstale":"2020-08-07T15:09:01.467392+0000","last_undegraded":"2020-08-07T15:09:01.467392+0000","last_fullsized":"2020-08-07T15:09:01.467392+0000","mapping_epoch":98,"log_start":"114'2484","ondisk_log_start":"114'2484","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20381696,"num_objects":311,"num_object_clones":0,"num_object_copies":933,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2488,"num_write_kb":19904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"144","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.296026+0000","last_change":"2020-08-07T15:07:56.631170+0000","last_active":"2020-08-07T15:09:00.296026+0000","last_peered":"2020-08-07T15:09:00.296026+0000","last_clean":"2020-08-07T15:09:00.296026+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:09:00.296026+0000","last_undegraded":"2020-08-07T15:09:00.296026+0000","last_fullsized":"2020-08-07T15:09:00.296026+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:56.631091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:56.631091+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"114'2792","reported_seq":"2855","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.589955+0000","last_change":"2020-08-07T15:08:52.193226+0000","last_active":"2020-08-07T15:09:01.589955+0000","last_peered":"2020-08-07T15:09:01.589955+0000","last_clean":"2020-08-07T15:09:01.589955+0000","last_became_active":"2020-08-07T15:08:52.166336+0000","last_became_peered":"2020-08-07T15:08:52.166336+0000","last_unstale":"2020-08-07T15:09:01.589955+0000","last_undegraded":"2020-08-07T15:09:01.589955+0000","last_fullsized":"2020-08-07T15:09:01.589955+0000","mapping_epoch":104,"log_start":"114'2789","ondisk_log_start":"114'2789","created":81,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22872064,"num_objects":349,"num_object_clones":0,"num_object_copies":1047,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2792,"num_write_kb":22336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":40960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"102","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.296057+0000","last_change":"2020-08-07T15:08:51.513398+0000","last_active":"2020-08-07T15:09:00.296057+0000","last_peered":"2020-08-07T15:09:00.296057+0000","last_clean":"2020-08-07T15:09:00.296057+0000","last_became_active":"2020-08-07T15:08:41.680448+0000","last_became_peered":"2020-08-07T15:08:41.680448+0000","last_unstale":"2020-08-07T15:09:00.296057+0000","last_undegraded":"2020-08-07T15:09:00.296057+0000","last_fullsized":"2020-08-07T15:09:00.296057+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:51.513319+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.513319+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"114'2640","reported_seq":"2681","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.406450+0000","last_change":"2020-08-07T15:08:24.550496+0000","last_active":"2020-08-07T15:09:01.406450+0000","last_peered":"2020-08-07T15:09:01.406450+0000","last_clean":"2020-08-07T15:09:01.406450+0000","last_became_active":"2020-08-07T15:08:24.549868+0000","last_became_peered":"2020-08-07T15:08:24.549868+0000","last_unstale":"2020-08-07T15:09:01.406450+0000","last_undegraded":"2020-08-07T15:09:01.406450+0000","last_fullsized":"2020-08-07T15:09:01.406450+0000","mapping_epoch":81,"log_start":"114'2625","ondisk_log_start":"114'2625","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21626880,"num_objects":330,"num_object_clones":0,"num_object_copies":990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2640,"num_write_kb":21120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"112","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.286220+0000","last_change":"2020-08-07T15:08:41.679850+0000","last_active":"2020-08-07T15:09:00.286220+0000","last_peered":"2020-08-07T15:09:00.286220+0000","last_clean":"2020-08-07T15:09:00.286220+0000","last_became_active":"2020-08-07T15:08:41.679321+0000","last_became_peered":"2020-08-07T15:08:41.679321+0000","last_unstale":"2020-08-07T15:09:00.286220+0000","last_undegraded":"2020-08-07T15:09:00.286220+0000","last_fullsized":"2020-08-07T15:09:00.286220+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:34.653843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:34.653843+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"112'2776","reported_seq":"3450","reported_epoch":"114","state":"remapped+peering","last_fresh":"2020-08-07T15:09:01.291140+0000","last_change":"2020-08-07T15:09:01.291140+0000","last_active":"2020-08-07T15:09:00.290771+0000","last_peered":"2020-08-07T15:09:00.287039+0000","last_clean":"2020-08-07T15:09:00.287039+0000","last_became_active":"2020-08-07T15:08:56.281990+0000","last_became_peered":"2020-08-07T15:08:56.281990+0000","last_unstale":"2020-08-07T15:09:01.291140+0000","last_undegraded":"2020-08-07T15:09:01.291140+0000","last_fullsized":"2020-08-07T15:09:01.291140+0000","mapping_epoch":114,"log_start":"112'2767","ondisk_log_start":"112'2767","created":81,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22740992,"num_objects":347,"num_object_clones":0,"num_object_copies":1041,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2776,"num_write_kb":22208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":20512768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"139","reported_epoch":"109","state":"active+clean","last_fresh":"2020-08-07T15:08:56.259368+0000","last_change":"2020-08-07T15:08:41.689973+0000","last_active":"2020-08-07T15:08:56.259368+0000","last_peered":"2020-08-07T15:08:56.259368+0000","last_clean":"2020-08-07T15:08:56.259368+0000","last_became_active":"2020-08-07T15:08:41.689435+0000","last_became_peered":"2020-08-07T15:08:41.689435+0000","last_unstale":"2020-08-07T15:08:56.259368+0000","last_undegraded":"2020-08-07T15:08:56.259368+0000","last_fullsized":"2020-08-07T15:08:56.259368+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.927824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.927824+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"110'2728","reported_seq":"2764","reported_epoch":"110","state":"active+clean","last_fresh":"2020-08-07T15:08:57.692287+0000","last_change":"2020-08-07T15:08:24.554472+0000","last_active":"2020-08-07T15:08:57.692287+0000","last_peered":"2020-08-07T15:08:57.692287+0000","last_clean":"2020-08-07T15:08:57.692287+0000","last_became_active":"2020-08-07T15:08:24.554021+0000","last_became_peered":"2020-08-07T15:08:24.554021+0000","last_unstale":"2020-08-07T15:08:57.692287+0000","last_undegraded":"2020-08-07T15:08:57.692287+0000","last_fullsized":"2020-08-07T15:08:57.692287+0000","mapping_epoch":81,"log_start":"110'2720","ondisk_log_start":"110'2720","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22347776,"num_objects":341,"num_object_clones":0,"num_object_copies":1023,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2728,"num_write_kb":21824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"30","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.295927+0000","last_change":"2020-08-07T15:08:46.710556+0000","last_active":"2020-08-07T15:09:00.295927+0000","last_peered":"2020-08-07T15:09:00.295927+0000","last_clean":"2020-08-07T15:09:00.295927+0000","last_became_active":"2020-08-07T15:08:46.710059+0000","last_became_peered":"2020-08-07T15:08:46.710059+0000","last_unstale":"2020-08-07T15:09:00.295927+0000","last_undegraded":"2020-08-07T15:09:00.295927+0000","last_fullsized":"2020-08-07T15:09:00.295927+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:09.841428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:09.841428+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"114'2608","reported_seq":"2649","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.643925+0000","last_change":"2020-08-07T15:08:24.552802+0000","last_active":"2020-08-07T15:09:01.643925+0000","last_peered":"2020-08-07T15:09:01.643925+0000","last_clean":"2020-08-07T15:09:01.643925+0000","last_became_active":"2020-08-07T15:08:24.552127+0000","last_became_peered":"2020-08-07T15:08:24.552127+0000","last_unstale":"2020-08-07T15:09:01.643925+0000","last_undegraded":"2020-08-07T15:09:01.643925+0000","last_fullsized":"2020-08-07T15:09:01.643925+0000","mapping_epoch":81,"log_start":"114'2605","ondisk_log_start":"114'2605","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21364736,"num_objects":326,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2608,"num_write_kb":20864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"110'2320","reported_seq":"2360","reported_epoch":"110","state":"active+clean","last_fresh":"2020-08-07T15:08:57.613236+0000","last_change":"2020-08-07T15:08:46.710643+0000","last_active":"2020-08-07T15:08:57.613236+0000","last_peered":"2020-08-07T15:08:57.613236+0000","last_clean":"2020-08-07T15:08:57.613236+0000","last_became_active":"2020-08-07T15:08:46.710100+0000","last_became_peered":"2020-08-07T15:08:46.710100+0000","last_unstale":"2020-08-07T15:08:57.613236+0000","last_undegraded":"2020-08-07T15:08:57.613236+0000","last_fullsized":"2020-08-07T15:08:57.613236+0000","mapping_epoch":98,"log_start":"110'2312","ondisk_log_start":"110'2312","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19005440,"num_objects":290,"num_object_clones":0,"num_object_copies":870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2320,"num_write_kb":18560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"52","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.290299+0000","last_change":"2020-08-07T15:08:41.678395+0000","last_active":"2020-08-07T15:09:00.290299+0000","last_peered":"2020-08-07T15:09:00.290299+0000","last_clean":"2020-08-07T15:09:00.290299+0000","last_became_active":"2020-08-07T15:08:41.677944+0000","last_became_peered":"2020-08-07T15:08:41.677944+0000","last_unstale":"2020-08-07T15:09:00.290299+0000","last_undegraded":"2020-08-07T15:09:00.290299+0000","last_fullsized":"2020-08-07T15:09:00.290299+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:02.901115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:02.901115+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":345702400,"num_objects":5275,"num_object_clones":0,"num_object_copies":15825,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42200,"num_write_kb":337600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":635,"num_bytes_recovered":41525248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1085702144,"data_stored":1085702144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101,"ondisk_log_size":101,"up":48,"acting":47,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":63,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542178,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1166640,"kb_used_data":118056,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93205200,"statfs":{"total":96636764160,"available":95442124800,"internally_reserved":1073741824,"allocated":120889344,"data_stored":120280013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":0.95099999999999996,"15min":0.91200000000000003},"min":{"1min":0.48199999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":3.4159999999999999,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.0940000000000001,"5min":0.98899999999999999,"15min":0.97099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.3559999999999999,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":1.669}]},{"osd":1,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3069999999999999,"15min":1.2989999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":4.4829999999999997,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":1.8180000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.044,"15min":0.99199999999999999},"min":{"1min":0.61799999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":3.4620000000000002,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":1.071}]},{"osd":2,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.179,"15min":1.1439999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":4.423,"5min":4.423,"15min":4.423},"last":1.7849999999999999},{"interface":"front","average":{"1min":1.5,"5min":1.194,"15min":1.143},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":3.5099999999999998,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":0.92200000000000004}]},{"osd":3,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.5900000000000001,"15min":1.6439999999999999},"min":{"1min":0.73799999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.5430000000000001,"5min":5.5019999999999998,"15min":5.5019999999999998},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.329,"15min":1.3380000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.4590000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":0.86399999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.7849999999999999,"5min":1.377,"15min":1.3089999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":4.835,"5min":4.835,"15min":4.835},"last":1.986},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.3300000000000001,"15min":1.3180000000000001},"min":{"1min":0.76900000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.4470000000000001,"5min":2.778,"15min":2.778},"last":1.4219999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":2.3940000000000001,"5min":30.484000000000002,"15min":35.165999999999997},"min":{"1min":0.96799999999999997,"5min":0.875,"15min":0.875},"max":{"1min":7.4429999999999996,"5min":831.88099999999997,"15min":831.88099999999997},"last":1.869},{"interface":"front","average":{"1min":2.456,"5min":1.6870000000000001,"15min":1.5589999999999999},"min":{"1min":1,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":7.5350000000000001,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":1.7569999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.542,"15min":1.526},"min":{"1min":0.83599999999999997,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.9900000000000002,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":2.9039999999999999},{"interface":"front","average":{"1min":1.9219999999999999,"5min":1.5900000000000001,"15min":1.534},"min":{"1min":1.115,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":4.5170000000000003,"5min":4.5170000000000003,"15min":4.5170000000000003},"last":2.1880000000000002}]}]},{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1180916,"kb_used_data":132332,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93190924,"statfs":{"total":96636764160,"available":95427506176,"internally_reserved":1073741824,"allocated":135507968,"data_stored":134876248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.2490000000000001,"15min":1.3049999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.6439999999999999,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":1.081},{"interface":"front","average":{"1min":1.079,"5min":1.135,"15min":1.145},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":2.9950000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.3740000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.95999999999999996,"15min":0.95799999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.9890000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":19.103999999999999,"5min":4.9299999999999997,"15min":2.5680000000000001},"min":{"1min":0.75600000000000001,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":372.50799999999998,"5min":372.50799999999998,"15min":372.50799999999998},"last":1.391}]},{"osd":2,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2,"15min":1.196},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":3.036,"5min":3.036,"15min":3.036},"last":0.876},{"interface":"front","average":{"1min":1.093,"5min":1.236,"15min":1.26},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.7440000000000002,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":1.2150000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.4870000000000001,"15min":1.5129999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":3.46,"5min":3.46,"15min":3.46},"last":1.054},{"interface":"front","average":{"1min":1.268,"5min":1.3640000000000001,"15min":1.3799999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":3.1970000000000001,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.91900000000000004}]},{"osd":4,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.5880000000000001,"15min":1.6379999999999999},"min":{"1min":0.72499999999999998,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.7999999999999998,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":0.996},{"interface":"front","average":{"1min":1.653,"5min":1.6679999999999999,"15min":1.6699999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":3.694,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.1419999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.3260000000000001,"15min":1.351},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.6360000000000001,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":1.107},{"interface":"front","average":{"1min":1.393,"5min":1.6910000000000001,"15min":1.7410000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.8279999999999998,"5min":5.327,"15min":5.327},"last":1.357}]},{"osd":6,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.446,"15min":1.4370000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.7799999999999998,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.4730000000000001,"5min":14.955,"15min":17.202000000000002},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":3.7309999999999999,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.3360000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1284228,"kb_used_data":235644,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93087612,"statfs":{"total":96636764160,"available":95321714688,"internally_reserved":1073741824,"allocated":241299456,"data_stored":240667736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":27.559999999999999,"15min":31.977},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.1820000000000004,"5min":497.04000000000002,"15min":497.04000000000002},"last":0.61899999999999999},{"interface":"front","average":{"1min":44.500999999999998,"5min":9.8049999999999997,"15min":4.0220000000000002},"min":{"1min":0.64200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":993.84500000000003,"5min":993.84500000000003,"15min":993.84500000000003},"last":0.71099999999999997}]},{"osd":2,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":0.98799999999999999,"15min":0.92800000000000005},"min":{"1min":0.57399999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":4.585,"5min":4.585,"15min":4.585},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.2410000000000001,"15min":1.216},"min":{"1min":0.51800000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":4.149,"5min":4.149,"15min":4.149},"last":0.67400000000000004}]},{"osd":3,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.962,"5min":1.4410000000000001,"15min":1.3540000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":10.603999999999999,"5min":10.603999999999999,"15min":10.603999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":2.1909999999999998,"5min":1.601,"15min":1.5029999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":10.638999999999999,"5min":10.638999999999999,"15min":10.638999999999999},"last":0.874}]},{"osd":4,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":2.1869999999999998,"5min":1.464,"15min":1.3440000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":10.69,"5min":10.69,"15min":10.69},"last":1.089},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.264,"15min":1.2210000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":4.9500000000000002,"5min":4.9500000000000002,"15min":4.9500000000000002},"last":1.141}]},{"osd":5,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":2.0630000000000002,"5min":10.819000000000001,"15min":12.278},"min":{"1min":0.73399999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":10.720000000000001,"5min":213.63800000000001,"15min":213.63800000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.258,"15min":1.212},"min":{"1min":0.82899999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":4.6269999999999998,"5min":4.6269999999999998,"15min":4.6269999999999998},"last":0.99199999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:08:24 2020","interfaces":[{"interface":"back","average":{"1min":1.7589999999999999,"5min":1.593,"15min":1.5649999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":4.5259999999999998,"5min":4.5259999999999998,"15min":4.5259999999999998},"last":0.90000000000000002},{"interface":"front","average":{"1min":2.3690000000000002,"5min":1.5620000000000001,"15min":1.4279999999999999},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":10.792,"5min":10.792,"15min":10.792},"last":1.1100000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.8129999999999999,"5min":1.6539999999999999,"15min":1.6279999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":4.5549999999999997,"5min":4.5549999999999997,"15min":4.5549999999999997},"last":1.0640000000000001},{"interface":"front","average":{"1min":2.2719999999999998,"5min":1.6379999999999999,"15min":1.532},"min":{"1min":0.998,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":10.752000000000001,"5min":10.752000000000001,"15min":10.752000000000001},"last":1.1719999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1155716,"kb_used_data":107132,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93216124,"statfs":{"total":96636764160,"available":95453310976,"internally_reserved":1073741824,"allocated":109703168,"data_stored":109071448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:08:24 2020","interfaces":[{"interface":"back","average":{"1min":81.266000000000005,"5min":17.033999999999999,"15min":6.3289999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":845.57100000000003,"5min":845.57100000000003,"15min":845.57100000000003},"last":1.327},{"interface":"front","average":{"1min":1.083,"5min":0.96299999999999997,"15min":0.94299999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":5.5970000000000004,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":1.74}]},{"osd":2,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.0069999999999999,"15min":0.95399999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":4.1600000000000001,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":1.036},{"interface":"front","average":{"1min":1.0049999999999999,"5min":26.393000000000001,"15min":30.623999999999999},"min":{"1min":0.54200000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.1280000000000001,"5min":606.18499999999995,"15min":606.18499999999995},"last":1.7929999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.0549999999999999,"15min":0.97499999999999998},"min":{"1min":0.64000000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":3.1779999999999999,"5min":3.1779999999999999,"15min":3.1779999999999999},"last":1.6859999999999999},{"interface":"front","average":{"1min":1.927,"5min":1.319,"15min":1.218},"min":{"1min":0.78700000000000003,"5min":0.621,"15min":0.621},"max":{"1min":4.21,"5min":4.21,"15min":4.21},"last":1.657}]},{"osd":4,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.238,"15min":1.1899999999999999},"min":{"1min":0.73899999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":3.0870000000000002,"5min":3.0870000000000002,"15min":3.0870000000000002},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.6890000000000001,"5min":1.3440000000000001,"15min":1.286},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":4.0960000000000001,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":2.0449999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":39.531999999999996,"5min":8.8879999999999999,"15min":3.7799999999999998},"min":{"1min":0.76600000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":501.267,"5min":501.267,"15min":501.267},"last":2.246},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.24,"15min":1.1499999999999999},"min":{"1min":0.874,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":6.0010000000000003,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":2.0939999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.7729999999999999,"5min":7.2450000000000001,"15min":8.157},"min":{"1min":0.81100000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":4.1310000000000002,"5min":124.738,"15min":124.738},"last":2.0059999999999998},{"interface":"front","average":{"1min":1.47,"5min":15.215999999999999,"15min":17.507000000000001},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":3.7120000000000002,"5min":294.94,"15min":294.94},"last":2.2789999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.552,"15min":1.5740000000000001},"min":{"1min":0.96199999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.3929999999999998,"5min":6.468,"15min":6.468},"last":2.3069999999999999},{"interface":"front","average":{"1min":1.6879999999999999,"5min":1.446,"15min":1.4059999999999999},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":3.117,"5min":3.117,"15min":3.117},"last":2.0659999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1111696,"kb_used_data":63112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93260144,"statfs":{"total":96636764160,"available":95498387456,"internally_reserved":1073741824,"allocated":64626688,"data_stored":63987505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.153,"15min":1.129},"min":{"1min":0.505,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.4909999999999997,"5min":4.4909999999999997,"15min":4.4909999999999997},"last":8.4930000000000003},{"interface":"front","average":{"1min":1.284,"5min":1.036,"15min":0.995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":3.6520000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":3.5}]},{"osd":1,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":2.5219999999999998,"5min":1.3779999999999999,"15min":1.1870000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":18.672000000000001,"5min":18.672000000000001,"15min":18.672000000000001},"last":5.6820000000000004},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.357,"15min":1.359},"min":{"1min":0.70699999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":5.1130000000000004,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":2.2250000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.216,"15min":1.1799999999999999},"min":{"1min":0.65300000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":1.736},{"interface":"front","average":{"1min":1.419,"5min":1.528,"15min":1.546},"min":{"1min":0.59999999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":5.3339999999999996,"5min":8.2210000000000001,"15min":8.2210000000000001},"last":1.4590000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":2.3319999999999999,"5min":1.3540000000000001,"15min":1.1910000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":19.783000000000001,"5min":19.783000000000001,"15min":19.783000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.5589999999999999,"15min":1.615},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.891,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":2.2829999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.2789999999999999,"15min":1.2669999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.7730000000000001,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.5249999999999999,"5min":20.725000000000001,"15min":23.925000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":4.4790000000000001,"5min":415.40699999999998,"15min":415.40699999999998},"last":2.532}]},{"osd":6,"last update":"Fri Aug 7 15:08:25 2020","interfaces":[{"interface":"back","average":{"1min":2.5089999999999999,"5min":1.478,"15min":1.306},"min":{"1min":0.89000000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":20.035,"5min":20.035,"15min":20.035},"last":0.85999999999999999},{"interface":"front","average":{"1min":44.033000000000001,"5min":9.8629999999999995,"15min":4.1680000000000001},"min":{"1min":0.93700000000000006,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":751.70000000000005,"5min":751.70000000000005,"15min":751.70000000000005},"last":1.6719999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":3.5409999999999999,"5min":2.101,"15min":1.861},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":35.548999999999999,"5min":35.548999999999999,"15min":35.548999999999999},"last":2.254},{"interface":"front","average":{"1min":3.4460000000000002,"5min":1.8859999999999999,"15min":1.6259999999999999},"min":{"1min":1.004,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":35.646999999999998,"5min":35.646999999999998,"15min":35.646999999999998},"last":1.7070000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1160944,"kb_used_data":112360,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93210896,"statfs":{"total":96636764160,"available":95447957504,"internally_reserved":1073741824,"allocated":115056640,"data_stored":114417457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.044,"5min":1.411,"15min":1.3049999999999999},"min":{"1min":0.55600000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":8.6020000000000003,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":1.383},{"interface":"front","average":{"1min":1.45,"5min":1.599,"15min":1.6240000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":4.9119999999999999,"5min":13.067,"15min":13.067},"last":0.79500000000000004}]},{"osd":1,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":16.382999999999999,"5min":4.7949999999999999,"15min":2.8639999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":270.68700000000001,"5min":270.68700000000001,"15min":270.68700000000001},"last":1.093},{"interface":"front","average":{"1min":2.7210000000000001,"5min":2.153,"15min":2.0590000000000002},"min":{"1min":0.71599999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":14.349,"5min":14.349,"15min":14.349},"last":7.0940000000000003}]},{"osd":2,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":1.7470000000000001,"5min":1.2669999999999999,"15min":1.1870000000000001},"min":{"1min":0.64900000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":4.9580000000000002,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":1.198},{"interface":"front","average":{"1min":4.5220000000000002,"5min":2.2519999999999998,"15min":1.8740000000000001},"min":{"1min":0.72399999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":51.530000000000001,"5min":51.530000000000001,"15min":51.530000000000001},"last":1.526}]},{"osd":4,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":47.652999999999999,"5min":10.888,"15min":4.7599999999999998},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":506.21300000000002,"5min":506.21300000000002,"15min":506.21300000000002},"last":1.752},{"interface":"front","average":{"1min":29.771000000000001,"5min":31.634,"15min":31.943999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":505.738,"5min":677.88599999999997,"15min":677.88599999999997},"last":1.498}]},{"osd":5,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.012,"5min":1.6160000000000001,"15min":1.55},"min":{"1min":0.96399999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":5.0049999999999999,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.5589999999999999},{"interface":"front","average":{"1min":2.2210000000000001,"5min":1.859,"15min":1.7989999999999999},"min":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":5.226,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":1.595}]},{"osd":6,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":1.99,"5min":1.623,"15min":1.5620000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":5.0389999999999997,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":2.1850000000000001},{"interface":"front","average":{"1min":1.9710000000000001,"5min":1.9830000000000001,"15min":1.9850000000000001},"min":{"1min":1.0369999999999999,"5min":0.876,"15min":0.876},"max":{"1min":4.266,"5min":5.6189999999999998,"15min":5.6189999999999998},"last":1.96}]},{"osd":7,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.0110000000000001,"5min":2.1309999999999998,"15min":2.1509999999999998},"min":{"1min":0.96999999999999997,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":3.79,"5min":5.734,"15min":5.734},"last":1.3200000000000001},{"interface":"front","average":{"1min":2.6200000000000001,"5min":2.105,"15min":2.0190000000000001},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":8.6790000000000003,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":1.7889999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1216808,"kb_used_data":168224,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93155032,"statfs":{"total":96636764160,"available":95390752768,"internally_reserved":1073741824,"allocated":172261376,"data_stored":171622193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":1.125,"15min":1.1619999999999999},"min":{"1min":0.47699999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.3999999999999999,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.349,"5min":0.94899999999999995,"15min":0.88200000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":6.3970000000000002,"5min":6.3970000000000002,"15min":6.3970000000000002},"last":1.194}]},{"osd":1,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.304,"15min":1.337},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.4380000000000002,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":1.397},{"interface":"front","average":{"1min":32.585999999999999,"5min":7.6180000000000003,"15min":3.4569999999999999},"min":{"1min":0.67000000000000004,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":689.846,"5min":689.846,"15min":689.846},"last":1.379}]},{"osd":2,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.46,"15min":1.454},"min":{"1min":0.56399999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":6.8730000000000002,"5min":6.8730000000000002,"15min":6.8730000000000002},"last":1.165},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.5669999999999999,"15min":1.532},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":7.1360000000000001,"5min":7.1360000000000001,"15min":7.1360000000000001},"last":1.222}]},{"osd":3,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":12.619999999999999,"5min":3.73,"15min":2.2480000000000002},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":247.202,"5min":247.202,"15min":247.202},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.583,"5min":1.3169999999999999,"15min":1.2729999999999999},"min":{"1min":0.67800000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":6.431,"5min":6.431,"15min":6.431},"last":1.244}]},{"osd":5,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":46.786999999999999,"5min":10.659000000000001,"15min":4.6379999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":990.71600000000001,"5min":990.71600000000001,"15min":990.71600000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.748,"5min":1.651,"15min":1.635},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":3.9329999999999998,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":1.3169999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.7070000000000001,"5min":1.7729999999999999,"15min":1.784},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":6.9749999999999996,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.524,"5min":15.592000000000001,"15min":17.936},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":6.7729999999999997,"5min":389.14600000000002,"15min":389.14600000000002},"last":0.99299999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":13.378,"5min":3.802,"15min":2.206},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":252.798,"5min":252.798,"15min":252.798},"last":0.95299999999999996},{"interface":"front","average":{"1min":44.676000000000002,"5min":10.252000000000001,"15min":4.5140000000000002},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":501.93000000000001,"5min":501.93000000000001,"15min":501.93000000000001},"last":1.03}]}]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1176656,"kb_used_data":128072,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93195184,"statfs":{"total":96636764160,"available":95431868416,"internally_reserved":1073741824,"allocated":131145728,"data_stored":130506545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0580000000000001,"15min":1.0700000000000001},"min":{"1min":0.49199999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.5169999999999999,"5min":2.766,"15min":2.766},"last":0.51700000000000002},{"interface":"front","average":{"1min":1.2250000000000001,"5min":0.88900000000000001,"15min":0.83299999999999996},"min":{"1min":0.54900000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":0.71599999999999997}]},{"osd":1,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":47.365000000000002,"5min":10.215999999999999,"15min":4.024},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":787.22900000000004,"5min":787.22900000000004,"15min":787.22900000000004},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.071,"5min":1.1000000000000001,"15min":1.105},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.8380000000000001,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":0.60999999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":7.5659999999999998,"15min":8.6679999999999993},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.4950000000000001,"5min":168.56899999999999,"15min":168.56899999999999},"last":1.026},{"interface":"front","average":{"1min":1.2929999999999999,"5min":26.763000000000002,"15min":31.007999999999999},"min":{"1min":0.79700000000000004,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":3.2389999999999999,"5min":676.27099999999996,"15min":676.27099999999996},"last":1.099}]},{"osd":3,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.196,"15min":1.1439999999999999},"min":{"1min":0.89000000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":3.5649999999999999,"5min":3.5649999999999999,"15min":3.5649999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":43.073,"5min":9.5850000000000009,"15min":4.0030000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":712.98699999999997,"5min":712.98699999999997,"15min":712.98699999999997},"last":2.2010000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":40.962000000000003,"15min":47.582000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.4980000000000002,"5min":530.30799999999999,"15min":530.30799999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.278,"15min":1.296},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.6259999999999999,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":1.137}]},{"osd":6,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":40.569000000000003,"15min":47.128999999999998},"min":{"1min":0.76800000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.6699999999999999,"5min":754.31299999999999,"15min":754.31299999999999},"last":966.60900000000004},{"interface":"front","average":{"1min":1.1899999999999999,"5min":21.050999999999998,"15min":24.361000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.7809999999999999,"5min":517.26700000000005,"15min":517.26700000000005},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.3560000000000001,"15min":1.3240000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":3.9079999999999999,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.23},{"interface":"front","average":{"1min":1.411,"5min":1.5349999999999999,"15min":1.556},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":3.9390000000000001,"5min":3.9390000000000001,"15min":3.9390000000000001},"last":1.288}]}]}]}} 2020-08-07T15:09:02.691 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early status --format=json 2020-08-07T15:09:02.696 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:09:02.701 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:09:02.701 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:09:02.701 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:09:02.701 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:09:02.702 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:09:02.702 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:09:02.702 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:09:02.702 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:09:02.760 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:02.756+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:02.844 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:02.842+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:02.894 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:02.895 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:02.903 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:09:02.961 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:02.958+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:03.054 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:03.053+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:03.115 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:03.116 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:03.117 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:09:03.143 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:09:03.143 INFO:teuthology.orchestra.run.smithi074.stdout:{"fsid":"a01d3e82-8d24-4288-9e42-576ab3ff4957","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub,nodeep-scrub flag(s) set","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":175,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":115,"num_osds":8,"num_up_osds":8,"osd_up_since":1596812777,"num_in_osds":8,"osd_in_since":1596812777,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":41},{"state_name":"active+remapped+backfilling","count":2},{"state_name":"active+undersized+remapped","count":1},{"state_name":"peering","count":1}],"num_pgs":45,"num_pools":3,"num_objects":4750,"data_bytes":311132160,"bytes_used":9520611328,"bytes_avail":763573501952,"bytes_total":773094113280,"inactive_pgs_ratio":0.02222222276031971,"misplaced_objects":542,"misplaced_total":14250,"misplaced_ratio":0.038035087719298248,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":868140,"recovering_keys_per_sec":0,"num_objects_recovered":53,"num_bytes_recovered":3473408,"num_keys_recovered":0,"write_bytes_sec":8498925,"read_op_per_sec":0,"write_op_per_sec":1037},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2020-08-07T15:07:18.623632+0000","services":{}},"progress_events":{}} 2020-08-07T15:09:03.159 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2020-08-07T15:09:03.160 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:09:03.169 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:03.158+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:03.263 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:03.258+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:03.276 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 37 16 44462 44446 9.3838 11.0625 0.0112504 0.0133158 2020-08-07T15:09:03.346 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:03.346 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:03.347 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:09:03.388 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:03.386+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:03.461 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:03.459+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:03.507 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:03.511 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:03.517 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:09:03.550 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:09:03.551 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:09:03.559 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":200,"stamp":"2020-08-07T15:09:03.174464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":353501184,"num_objects":5396,"num_object_clones":0,"num_object_copies":16188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43152,"num_write_kb":345216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":635,"num_bytes_recovered":41525248,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":135,"acting":134,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9467936,"kb_used_data":1079264,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":745506784,"statfs":{"total":773094113280,"available":763398946816,"internally_reserved":8589934592,"allocated":1105166336,"data_stored":1100067761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42369024,"num_objects":646,"num_object_clones":0,"num_object_copies":1938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-542,"num_objects_unfound":0,"num_objects_dirty":646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5172,"num_write_kb":41376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":580,"num_bytes_recovered":38010880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.907877"},"pg_stats":[{"pgid":"1.14","version":"16'2","reported_seq":"43","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.515159+0000","last_change":"2020-08-07T15:08:42.533999+0000","last_active":"2020-08-07T15:09:00.515159+0000","last_peered":"2020-08-07T15:09:00.515159+0000","last_clean":"2020-08-07T15:09:00.515159+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:09:00.515159+0000","last_undegraded":"2020-08-07T15:09:00.515159+0000","last_fullsized":"2020-08-07T15:09:00.515159+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:42.533921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:42.533921+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"79","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.292021+0000","last_change":"2020-08-07T15:08:58.212828+0000","last_active":"2020-08-07T15:09:00.292021+0000","last_peered":"2020-08-07T15:09:00.292021+0000","last_clean":"2020-08-07T15:09:00.292021+0000","last_became_active":"2020-08-07T15:08:58.211785+0000","last_became_peered":"2020-08-07T15:08:58.211785+0000","last_unstale":"2020-08-07T15:09:00.292021+0000","last_undegraded":"2020-08-07T15:09:00.292021+0000","last_fullsized":"2020-08-07T15:09:00.292021+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.141010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.141010+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"43","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.289984+0000","last_change":"2020-08-07T15:08:55.078015+0000","last_active":"2020-08-07T15:09:00.289984+0000","last_peered":"2020-08-07T15:09:00.289984+0000","last_clean":"2020-08-07T15:09:00.289984+0000","last_became_active":"2020-08-07T15:08:41.685106+0000","last_became_peered":"2020-08-07T15:08:41.685106+0000","last_unstale":"2020-08-07T15:09:00.289984+0000","last_undegraded":"2020-08-07T15:09:00.289984+0000","last_fullsized":"2020-08-07T15:09:00.289984+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:55.077924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:55.077924+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"122","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.290319+0000","last_change":"2020-08-07T15:08:43.544399+0000","last_active":"2020-08-07T15:09:00.290319+0000","last_peered":"2020-08-07T15:09:00.290319+0000","last_clean":"2020-08-07T15:09:00.290319+0000","last_became_active":"2020-08-07T15:08:41.675776+0000","last_became_peered":"2020-08-07T15:08:41.675776+0000","last_unstale":"2020-08-07T15:09:00.290319+0000","last_undegraded":"2020-08-07T15:09:00.290319+0000","last_fullsized":"2020-08-07T15:09:00.290319+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.544324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.544324+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"150","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.289931+0000","last_change":"2020-08-07T15:08:51.599032+0000","last_active":"2020-08-07T15:09:00.289931+0000","last_peered":"2020-08-07T15:09:00.289931+0000","last_clean":"2020-08-07T15:09:00.289931+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:09:00.289931+0000","last_undegraded":"2020-08-07T15:09:00.289931+0000","last_fullsized":"2020-08-07T15:09:00.289931+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:51.598947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.598947+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"109","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.289452+0000","last_change":"2020-08-07T15:08:11.840861+0000","last_active":"2020-08-07T15:09:01.289452+0000","last_peered":"2020-08-07T15:09:01.289452+0000","last_clean":"2020-08-07T15:09:01.289452+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:09:01.289452+0000","last_undegraded":"2020-08-07T15:09:01.289452+0000","last_fullsized":"2020-08-07T15:09:01.289452+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:11.840807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:11.840807+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"99","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.286421+0000","last_change":"2020-08-07T15:08:39.558760+0000","last_active":"2020-08-07T15:09:00.286421+0000","last_peered":"2020-08-07T15:09:00.286421+0000","last_clean":"2020-08-07T15:09:00.286421+0000","last_became_active":"2020-08-07T15:08:36.615358+0000","last_became_peered":"2020-08-07T15:08:36.615358+0000","last_unstale":"2020-08-07T15:09:00.286421+0000","last_undegraded":"2020-08-07T15:09:00.286421+0000","last_fullsized":"2020-08-07T15:09:00.286421+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:39.558677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:39.558677+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"114'2584","reported_seq":"2626","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.680980+0000","last_change":"2020-08-07T15:08:46.715869+0000","last_active":"2020-08-07T15:09:01.680980+0000","last_peered":"2020-08-07T15:09:01.680980+0000","last_clean":"2020-08-07T15:09:01.680980+0000","last_became_active":"2020-08-07T15:08:46.708112+0000","last_became_peered":"2020-08-07T15:08:46.708112+0000","last_unstale":"2020-08-07T15:09:01.680980+0000","last_undegraded":"2020-08-07T15:09:01.680980+0000","last_fullsized":"2020-08-07T15:09:01.680980+0000","mapping_epoch":98,"log_start":"114'2581","ondisk_log_start":"114'2581","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21168128,"num_objects":323,"num_object_clones":0,"num_object_copies":969,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2584,"num_write_kb":20672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"111","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.292329+0000","last_change":"2020-08-07T15:08:40.057705+0000","last_active":"2020-08-07T15:09:00.292329+0000","last_peered":"2020-08-07T15:09:00.292329+0000","last_clean":"2020-08-07T15:09:00.292329+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:09:00.292329+0000","last_undegraded":"2020-08-07T15:09:00.292329+0000","last_fullsized":"2020-08-07T15:09:00.292329+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:40.057622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:40.057622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"68","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.515303+0000","last_change":"2020-08-07T15:08:53.756554+0000","last_active":"2020-08-07T15:09:00.515303+0000","last_peered":"2020-08-07T15:09:00.515303+0000","last_clean":"2020-08-07T15:09:00.515303+0000","last_became_active":"2020-08-07T15:08:46.710404+0000","last_became_peered":"2020-08-07T15:08:46.710404+0000","last_unstale":"2020-08-07T15:09:00.515303+0000","last_undegraded":"2020-08-07T15:09:00.515303+0000","last_fullsized":"2020-08-07T15:09:00.515303+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:53.756343+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:53.756343+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"114'2824","reported_seq":"3509","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.322637+0000","last_change":"2020-08-07T15:09:01.304273+0000","last_active":"2020-08-07T15:09:01.322637+0000","last_peered":"2020-08-07T15:09:01.322637+0000","last_clean":"2020-08-07T15:09:01.322637+0000","last_became_active":"2020-08-07T15:09:01.293427+0000","last_became_peered":"2020-08-07T15:09:01.293427+0000","last_unstale":"2020-08-07T15:09:01.322637+0000","last_undegraded":"2020-08-07T15:09:01.322637+0000","last_fullsized":"2020-08-07T15:09:01.322637+0000","mapping_epoch":113,"log_start":"114'2815","ondisk_log_start":"114'2815","created":81,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23134208,"num_objects":353,"num_object_clones":0,"num_object_copies":1059,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2824,"num_write_kb":22592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":20971520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"108","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.296117+0000","last_change":"2020-08-07T15:08:27.906894+0000","last_active":"2020-08-07T15:09:00.296117+0000","last_peered":"2020-08-07T15:09:00.296117+0000","last_clean":"2020-08-07T15:09:00.296117+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:09:00.296117+0000","last_undegraded":"2020-08-07T15:09:00.296117+0000","last_fullsized":"2020-08-07T15:09:00.296117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:27.906822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:27.906822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"114","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.289746+0000","last_change":"2020-08-07T15:08:43.899342+0000","last_active":"2020-08-07T15:09:01.289746+0000","last_peered":"2020-08-07T15:09:01.289746+0000","last_clean":"2020-08-07T15:09:01.289746+0000","last_became_active":"2020-08-07T15:08:36.622209+0000","last_became_peered":"2020-08-07T15:08:36.622209+0000","last_unstale":"2020-08-07T15:09:01.289746+0000","last_undegraded":"2020-08-07T15:09:01.289746+0000","last_fullsized":"2020-08-07T15:09:01.289746+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.899280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.899280+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"114'2616","reported_seq":"2657","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.421157+0000","last_change":"2020-08-07T15:08:24.549715+0000","last_active":"2020-08-07T15:09:01.421157+0000","last_peered":"2020-08-07T15:09:01.421157+0000","last_clean":"2020-08-07T15:09:01.421157+0000","last_became_active":"2020-08-07T15:08:24.549245+0000","last_became_peered":"2020-08-07T15:08:24.549245+0000","last_unstale":"2020-08-07T15:09:01.421157+0000","last_undegraded":"2020-08-07T15:09:01.421157+0000","last_fullsized":"2020-08-07T15:09:01.421157+0000","mapping_epoch":81,"log_start":"114'2610","ondisk_log_start":"114'2610","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21430272,"num_objects":327,"num_object_clones":0,"num_object_copies":981,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2616,"num_write_kb":20928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"108","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.295286+0000","last_change":"2020-08-07T15:08:13.867355+0000","last_active":"2020-08-07T15:09:00.295286+0000","last_peered":"2020-08-07T15:09:00.295286+0000","last_clean":"2020-08-07T15:09:00.295286+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:09:00.295286+0000","last_undegraded":"2020-08-07T15:09:00.295286+0000","last_fullsized":"2020-08-07T15:09:00.295286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:13.867148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:13.867148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"114","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.289247+0000","last_change":"2020-08-07T15:08:50.686578+0000","last_active":"2020-08-07T15:09:01.289247+0000","last_peered":"2020-08-07T15:09:01.289247+0000","last_clean":"2020-08-07T15:09:01.289247+0000","last_became_active":"2020-08-07T15:08:36.633406+0000","last_became_peered":"2020-08-07T15:08:36.633406+0000","last_unstale":"2020-08-07T15:09:01.289247+0000","last_undegraded":"2020-08-07T15:09:01.289247+0000","last_fullsized":"2020-08-07T15:09:01.289247+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:50.686496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:50.686496+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"114'2512","reported_seq":"2552","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.594921+0000","last_change":"2020-08-07T15:08:24.557398+0000","last_active":"2020-08-07T15:09:01.594921+0000","last_peered":"2020-08-07T15:09:01.594921+0000","last_clean":"2020-08-07T15:09:01.594921+0000","last_became_active":"2020-08-07T15:08:24.557022+0000","last_became_peered":"2020-08-07T15:08:24.557022+0000","last_unstale":"2020-08-07T15:09:01.594921+0000","last_undegraded":"2020-08-07T15:09:01.594921+0000","last_fullsized":"2020-08-07T15:09:01.594921+0000","mapping_epoch":81,"log_start":"114'2509","ondisk_log_start":"114'2509","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20578304,"num_objects":314,"num_object_clones":0,"num_object_copies":942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2512,"num_write_kb":20096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"140","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.295184+0000","last_change":"2020-08-07T15:08:43.036067+0000","last_active":"2020-08-07T15:09:00.295184+0000","last_peered":"2020-08-07T15:09:00.295184+0000","last_clean":"2020-08-07T15:09:00.295184+0000","last_became_active":"2020-08-07T15:08:36.627426+0000","last_became_peered":"2020-08-07T15:08:36.627426+0000","last_unstale":"2020-08-07T15:09:00.295184+0000","last_undegraded":"2020-08-07T15:09:00.295184+0000","last_fullsized":"2020-08-07T15:09:00.295184+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.035987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.035987+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"108","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.290414+0000","last_change":"2020-08-07T15:08:26.317430+0000","last_active":"2020-08-07T15:09:00.290414+0000","last_peered":"2020-08-07T15:09:00.290414+0000","last_clean":"2020-08-07T15:09:00.290414+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:09:00.290414+0000","last_undegraded":"2020-08-07T15:09:00.290414+0000","last_fullsized":"2020-08-07T15:09:00.290414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:26.317351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:26.317351+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"115'2768","reported_seq":"2804","reported_epoch":"115","state":"active+clean","last_fresh":"2020-08-07T15:09:02.714450+0000","last_change":"2020-08-07T15:08:24.552570+0000","last_active":"2020-08-07T15:09:02.714450+0000","last_peered":"2020-08-07T15:09:02.714450+0000","last_clean":"2020-08-07T15:09:02.714450+0000","last_became_active":"2020-08-07T15:08:24.552105+0000","last_became_peered":"2020-08-07T15:08:24.552105+0000","last_unstale":"2020-08-07T15:09:02.714450+0000","last_undegraded":"2020-08-07T15:09:02.714450+0000","last_fullsized":"2020-08-07T15:09:02.714450+0000","mapping_epoch":81,"log_start":"115'2761","ondisk_log_start":"115'2761","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22675456,"num_objects":346,"num_object_clones":0,"num_object_copies":1038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2768,"num_write_kb":22144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"108","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.515189+0000","last_change":"2020-08-07T15:07:36.864560+0000","last_active":"2020-08-07T15:09:00.515189+0000","last_peered":"2020-08-07T15:09:00.515189+0000","last_clean":"2020-08-07T15:09:00.515189+0000","last_became_active":"2020-08-07T15:06:23.223489+0000","last_became_peered":"2020-08-07T15:06:23.223489+0000","last_unstale":"2020-08-07T15:09:00.515189+0000","last_undegraded":"2020-08-07T15:09:00.515189+0000","last_fullsized":"2020-08-07T15:09:00.515189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:07:36.864483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.864483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"156","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.294032+0000","last_change":"2020-08-07T15:08:42.069637+0000","last_active":"2020-08-07T15:09:00.294032+0000","last_peered":"2020-08-07T15:09:00.294032+0000","last_clean":"2020-08-07T15:09:00.294032+0000","last_became_active":"2020-08-07T15:08:36.628365+0000","last_became_peered":"2020-08-07T15:08:36.628365+0000","last_unstale":"2020-08-07T15:09:00.294032+0000","last_undegraded":"2020-08-07T15:09:00.294032+0000","last_fullsized":"2020-08-07T15:09:00.294032+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:42.069585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:42.069585+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"114'2824","reported_seq":"2865","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.426013+0000","last_change":"2020-08-07T15:08:24.550789+0000","last_active":"2020-08-07T15:09:01.426013+0000","last_peered":"2020-08-07T15:09:01.426013+0000","last_clean":"2020-08-07T15:09:01.426013+0000","last_became_active":"2020-08-07T15:08:24.550222+0000","last_became_peered":"2020-08-07T15:08:24.550222+0000","last_unstale":"2020-08-07T15:09:01.426013+0000","last_undegraded":"2020-08-07T15:09:01.426013+0000","last_fullsized":"2020-08-07T15:09:01.426013+0000","mapping_epoch":81,"log_start":"114'2818","ondisk_log_start":"114'2818","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23134208,"num_objects":353,"num_object_clones":0,"num_object_copies":1059,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2824,"num_write_kb":22592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"108","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.290321+0000","last_change":"2020-08-07T15:08:30.339852+0000","last_active":"2020-08-07T15:09:00.290321+0000","last_peered":"2020-08-07T15:09:00.290321+0000","last_clean":"2020-08-07T15:09:00.290321+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:09:00.290321+0000","last_undegraded":"2020-08-07T15:09:00.290321+0000","last_fullsized":"2020-08-07T15:09:00.290321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:08:30.339785+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"106","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.292512+0000","last_change":"2020-08-07T15:08:38.386792+0000","last_active":"2020-08-07T15:09:00.292512+0000","last_peered":"2020-08-07T15:09:00.292512+0000","last_clean":"2020-08-07T15:09:00.292512+0000","last_became_active":"2020-08-07T15:08:36.618164+0000","last_became_peered":"2020-08-07T15:08:36.618164+0000","last_unstale":"2020-08-07T15:09:00.292512+0000","last_undegraded":"2020-08-07T15:09:00.292512+0000","last_fullsized":"2020-08-07T15:09:00.292512+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:38.386739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:38.386739+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"114'2768","reported_seq":"2809","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.674515+0000","last_change":"2020-08-07T15:08:24.558242+0000","last_active":"2020-08-07T15:09:01.674515+0000","last_peered":"2020-08-07T15:09:01.674515+0000","last_clean":"2020-08-07T15:09:01.674515+0000","last_became_active":"2020-08-07T15:08:24.557354+0000","last_became_peered":"2020-08-07T15:08:24.557354+0000","last_unstale":"2020-08-07T15:09:01.674515+0000","last_undegraded":"2020-08-07T15:09:01.674515+0000","last_fullsized":"2020-08-07T15:09:01.674515+0000","mapping_epoch":81,"log_start":"114'2765","ondisk_log_start":"114'2765","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22675456,"num_objects":346,"num_object_clones":0,"num_object_copies":1038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2768,"num_write_kb":22144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"105","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.295909+0000","last_change":"2020-08-07T15:08:15.887617+0000","last_active":"2020-08-07T15:09:00.295909+0000","last_peered":"2020-08-07T15:09:00.295909+0000","last_clean":"2020-08-07T15:09:00.295909+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:09:00.295909+0000","last_undegraded":"2020-08-07T15:09:00.295909+0000","last_fullsized":"2020-08-07T15:09:00.295909+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:15.887513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:15.887513+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"145","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.289473+0000","last_change":"2020-08-07T15:08:58.183263+0000","last_active":"2020-08-07T15:09:01.289473+0000","last_peered":"2020-08-07T15:09:01.289473+0000","last_clean":"2020-08-07T15:09:01.289473+0000","last_became_active":"2020-08-07T15:08:58.182722+0000","last_became_peered":"2020-08-07T15:08:58.182722+0000","last_unstale":"2020-08-07T15:09:01.289473+0000","last_undegraded":"2020-08-07T15:09:01.289473+0000","last_fullsized":"2020-08-07T15:09:01.289473+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.927824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.927824+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"114'2464","reported_seq":"2504","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.426624+0000","last_change":"2020-08-07T15:08:24.552962+0000","last_active":"2020-08-07T15:09:01.426624+0000","last_peered":"2020-08-07T15:09:01.426624+0000","last_clean":"2020-08-07T15:09:01.426624+0000","last_became_active":"2020-08-07T15:08:24.552070+0000","last_became_peered":"2020-08-07T15:08:24.552070+0000","last_unstale":"2020-08-07T15:09:01.426624+0000","last_undegraded":"2020-08-07T15:09:01.426624+0000","last_fullsized":"2020-08-07T15:09:01.426624+0000","mapping_epoch":81,"log_start":"114'2456","ondisk_log_start":"114'2456","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20185088,"num_objects":308,"num_object_clones":0,"num_object_copies":924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2464,"num_write_kb":19712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"83","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.290206+0000","last_change":"2020-08-07T15:08:39.409159+0000","last_active":"2020-08-07T15:09:00.290206+0000","last_peered":"2020-08-07T15:09:00.290206+0000","last_clean":"2020-08-07T15:09:00.290206+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:09:00.290206+0000","last_undegraded":"2020-08-07T15:09:00.290206+0000","last_fullsized":"2020-08-07T15:09:00.290206+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:39.409063+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_clean_scrub_stamp":"2020-08-07T15:08:39.409063+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"114'2760","reported_seq":"2801","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.409800+0000","last_change":"2020-08-07T15:08:24.554063+0000","last_active":"2020-08-07T15:09:01.409800+0000","last_peered":"2020-08-07T15:09:01.409800+0000","last_clean":"2020-08-07T15:09:01.409800+0000","last_became_active":"2020-08-07T15:08:24.553668+0000","last_became_peered":"2020-08-07T15:08:24.553668+0000","last_unstale":"2020-08-07T15:09:01.409800+0000","last_undegraded":"2020-08-07T15:09:01.409800+0000","last_fullsized":"2020-08-07T15:09:01.409800+0000","mapping_epoch":81,"log_start":"114'2754","ondisk_log_start":"114'2754","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22609920,"num_objects":345,"num_object_clones":0,"num_object_copies":1035,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2760,"num_write_kb":22080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"80","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.289975+0000","last_change":"2020-08-07T15:07:33.177491+0000","last_active":"2020-08-07T15:09:00.289975+0000","last_peered":"2020-08-07T15:09:00.289975+0000","last_clean":"2020-08-07T15:09:00.289975+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:09:00.289975+0000","last_undegraded":"2020-08-07T15:09:00.289975+0000","last_fullsized":"2020-08-07T15:09:00.289975+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:33.177412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:33.177412+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"114'2488","reported_seq":"2529","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.467392+0000","last_change":"2020-08-07T15:08:46.709789+0000","last_active":"2020-08-07T15:09:01.467392+0000","last_peered":"2020-08-07T15:09:01.467392+0000","last_clean":"2020-08-07T15:09:01.467392+0000","last_became_active":"2020-08-07T15:08:46.708674+0000","last_became_peered":"2020-08-07T15:08:46.708674+0000","last_unstale":"2020-08-07T15:09:01.467392+0000","last_undegraded":"2020-08-07T15:09:01.467392+0000","last_fullsized":"2020-08-07T15:09:01.467392+0000","mapping_epoch":98,"log_start":"114'2484","ondisk_log_start":"114'2484","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20381696,"num_objects":311,"num_object_clones":0,"num_object_copies":933,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2488,"num_write_kb":19904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"144","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.296026+0000","last_change":"2020-08-07T15:07:56.631170+0000","last_active":"2020-08-07T15:09:00.296026+0000","last_peered":"2020-08-07T15:09:00.296026+0000","last_clean":"2020-08-07T15:09:00.296026+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:09:00.296026+0000","last_undegraded":"2020-08-07T15:09:00.296026+0000","last_fullsized":"2020-08-07T15:09:00.296026+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:56.631091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:56.631091+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"114'2792","reported_seq":"2855","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.589955+0000","last_change":"2020-08-07T15:08:52.193226+0000","last_active":"2020-08-07T15:09:01.589955+0000","last_peered":"2020-08-07T15:09:01.589955+0000","last_clean":"2020-08-07T15:09:01.589955+0000","last_became_active":"2020-08-07T15:08:52.166336+0000","last_became_peered":"2020-08-07T15:08:52.166336+0000","last_unstale":"2020-08-07T15:09:01.589955+0000","last_undegraded":"2020-08-07T15:09:01.589955+0000","last_fullsized":"2020-08-07T15:09:01.589955+0000","mapping_epoch":104,"log_start":"114'2789","ondisk_log_start":"114'2789","created":81,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22872064,"num_objects":349,"num_object_clones":0,"num_object_copies":1047,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2792,"num_write_kb":22336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":40960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"102","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.296057+0000","last_change":"2020-08-07T15:08:51.513398+0000","last_active":"2020-08-07T15:09:00.296057+0000","last_peered":"2020-08-07T15:09:00.296057+0000","last_clean":"2020-08-07T15:09:00.296057+0000","last_became_active":"2020-08-07T15:08:41.680448+0000","last_became_peered":"2020-08-07T15:08:41.680448+0000","last_unstale":"2020-08-07T15:09:00.296057+0000","last_undegraded":"2020-08-07T15:09:00.296057+0000","last_fullsized":"2020-08-07T15:09:00.296057+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:51.513319+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.513319+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"114'2640","reported_seq":"2681","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.406450+0000","last_change":"2020-08-07T15:08:24.550496+0000","last_active":"2020-08-07T15:09:01.406450+0000","last_peered":"2020-08-07T15:09:01.406450+0000","last_clean":"2020-08-07T15:09:01.406450+0000","last_became_active":"2020-08-07T15:08:24.549868+0000","last_became_peered":"2020-08-07T15:08:24.549868+0000","last_unstale":"2020-08-07T15:09:01.406450+0000","last_undegraded":"2020-08-07T15:09:01.406450+0000","last_fullsized":"2020-08-07T15:09:01.406450+0000","mapping_epoch":81,"log_start":"114'2625","ondisk_log_start":"114'2625","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21626880,"num_objects":330,"num_object_clones":0,"num_object_copies":990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2640,"num_write_kb":21120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"112","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.286220+0000","last_change":"2020-08-07T15:08:41.679850+0000","last_active":"2020-08-07T15:09:00.286220+0000","last_peered":"2020-08-07T15:09:00.286220+0000","last_clean":"2020-08-07T15:09:00.286220+0000","last_became_active":"2020-08-07T15:08:41.679321+0000","last_became_peered":"2020-08-07T15:08:41.679321+0000","last_unstale":"2020-08-07T15:09:00.286220+0000","last_undegraded":"2020-08-07T15:09:00.286220+0000","last_fullsized":"2020-08-07T15:09:00.286220+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:34.653843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:34.653843+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"112'2776","reported_seq":"3450","reported_epoch":"114","state":"remapped+peering","last_fresh":"2020-08-07T15:09:01.291140+0000","last_change":"2020-08-07T15:09:01.291140+0000","last_active":"2020-08-07T15:09:00.290771+0000","last_peered":"2020-08-07T15:09:00.287039+0000","last_clean":"2020-08-07T15:09:00.287039+0000","last_became_active":"2020-08-07T15:08:56.281990+0000","last_became_peered":"2020-08-07T15:08:56.281990+0000","last_unstale":"2020-08-07T15:09:01.291140+0000","last_undegraded":"2020-08-07T15:09:01.291140+0000","last_fullsized":"2020-08-07T15:09:01.291140+0000","mapping_epoch":114,"log_start":"112'2767","ondisk_log_start":"112'2767","created":81,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22740992,"num_objects":347,"num_object_clones":0,"num_object_copies":1041,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2776,"num_write_kb":22208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":20512768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"144","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.289331+0000","last_change":"2020-08-07T15:08:41.689973+0000","last_active":"2020-08-07T15:09:01.289331+0000","last_peered":"2020-08-07T15:09:01.289331+0000","last_clean":"2020-08-07T15:09:01.289331+0000","last_became_active":"2020-08-07T15:08:41.689435+0000","last_became_peered":"2020-08-07T15:08:41.689435+0000","last_unstale":"2020-08-07T15:09:01.289331+0000","last_undegraded":"2020-08-07T15:09:01.289331+0000","last_fullsized":"2020-08-07T15:09:01.289331+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.927824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.927824+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"115'3016","reported_seq":"3058","reported_epoch":"115","state":"active+clean","last_fresh":"2020-08-07T15:09:02.713039+0000","last_change":"2020-08-07T15:08:24.554472+0000","last_active":"2020-08-07T15:09:02.713039+0000","last_peered":"2020-08-07T15:09:02.713039+0000","last_clean":"2020-08-07T15:09:02.713039+0000","last_became_active":"2020-08-07T15:08:24.554021+0000","last_became_peered":"2020-08-07T15:08:24.554021+0000","last_unstale":"2020-08-07T15:09:02.713039+0000","last_undegraded":"2020-08-07T15:09:02.713039+0000","last_fullsized":"2020-08-07T15:09:02.713039+0000","mapping_epoch":81,"log_start":"115'3008","ondisk_log_start":"115'3008","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24707072,"num_objects":377,"num_object_clones":0,"num_object_copies":1131,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3016,"num_write_kb":24128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"30","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.295927+0000","last_change":"2020-08-07T15:08:46.710556+0000","last_active":"2020-08-07T15:09:00.295927+0000","last_peered":"2020-08-07T15:09:00.295927+0000","last_clean":"2020-08-07T15:09:00.295927+0000","last_became_active":"2020-08-07T15:08:46.710059+0000","last_became_peered":"2020-08-07T15:08:46.710059+0000","last_unstale":"2020-08-07T15:09:00.295927+0000","last_undegraded":"2020-08-07T15:09:00.295927+0000","last_fullsized":"2020-08-07T15:09:00.295927+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:09.841428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:09.841428+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"114'2608","reported_seq":"2649","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.643925+0000","last_change":"2020-08-07T15:08:24.552802+0000","last_active":"2020-08-07T15:09:01.643925+0000","last_peered":"2020-08-07T15:09:01.643925+0000","last_clean":"2020-08-07T15:09:01.643925+0000","last_became_active":"2020-08-07T15:08:24.552127+0000","last_became_peered":"2020-08-07T15:08:24.552127+0000","last_unstale":"2020-08-07T15:09:01.643925+0000","last_undegraded":"2020-08-07T15:09:01.643925+0000","last_fullsized":"2020-08-07T15:09:01.643925+0000","mapping_epoch":81,"log_start":"114'2605","ondisk_log_start":"114'2605","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21364736,"num_objects":326,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2608,"num_write_kb":20864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"115'2712","reported_seq":"2757","reported_epoch":"115","state":"active+clean","last_fresh":"2020-08-07T15:09:02.712781+0000","last_change":"2020-08-07T15:08:46.710643+0000","last_active":"2020-08-07T15:09:02.712781+0000","last_peered":"2020-08-07T15:09:02.712781+0000","last_clean":"2020-08-07T15:09:02.712781+0000","last_became_active":"2020-08-07T15:08:46.710100+0000","last_became_peered":"2020-08-07T15:08:46.710100+0000","last_unstale":"2020-08-07T15:09:02.712781+0000","last_undegraded":"2020-08-07T15:09:02.712781+0000","last_fullsized":"2020-08-07T15:09:02.712781+0000","mapping_epoch":98,"log_start":"115'2704","ondisk_log_start":"115'2704","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22216704,"num_objects":339,"num_object_clones":0,"num_object_copies":1017,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2712,"num_write_kb":21696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"52","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.290299+0000","last_change":"2020-08-07T15:08:41.678395+0000","last_active":"2020-08-07T15:09:00.290299+0000","last_peered":"2020-08-07T15:09:00.290299+0000","last_clean":"2020-08-07T15:09:00.290299+0000","last_became_active":"2020-08-07T15:08:41.677944+0000","last_became_peered":"2020-08-07T15:08:41.677944+0000","last_unstale":"2020-08-07T15:09:00.290299+0000","last_undegraded":"2020-08-07T15:09:00.290299+0000","last_fullsized":"2020-08-07T15:09:00.290299+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:02.901115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:02.901115+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":353501184,"num_objects":5394,"num_object_clones":0,"num_object_copies":16182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5394,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43152,"num_write_kb":345216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":635,"num_bytes_recovered":41525248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1101365248,"data_stored":1101365248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101,"ondisk_log_size":101,"up":48,"acting":47,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":63,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542179,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1180972,"kb_used_data":132388,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93190868,"statfs":{"total":96636764160,"available":95427448832,"internally_reserved":1073741824,"allocated":135565312,"data_stored":134918629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":0.95099999999999996,"15min":0.91200000000000003},"min":{"1min":0.48199999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":3.4159999999999999,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.0940000000000001,"5min":0.98899999999999999,"15min":0.97099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.3559999999999999,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":1.163}]},{"osd":1,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3069999999999999,"15min":1.2989999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":4.4829999999999997,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.044,"15min":0.99199999999999999},"min":{"1min":0.61799999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":3.4620000000000002,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":1.069}]},{"osd":2,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.179,"15min":1.1439999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":4.423,"5min":4.423,"15min":4.423},"last":1.6339999999999999},{"interface":"front","average":{"1min":1.5,"5min":1.194,"15min":1.143},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":3.5099999999999998,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":5.1319999999999997}]},{"osd":3,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.5900000000000001,"15min":1.6439999999999999},"min":{"1min":0.73799999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.5430000000000001,"5min":5.5019999999999998,"15min":5.5019999999999998},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.329,"15min":1.3380000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.4590000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":1.355}]},{"osd":4,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.7849999999999999,"5min":1.377,"15min":1.3089999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":4.835,"5min":4.835,"15min":4.835},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.3300000000000001,"15min":1.3180000000000001},"min":{"1min":0.76900000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.4470000000000001,"5min":2.778,"15min":2.778},"last":1.006}]},{"osd":5,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":2.3940000000000001,"5min":30.484000000000002,"15min":35.165999999999997},"min":{"1min":0.96799999999999997,"5min":0.875,"15min":0.875},"max":{"1min":7.4429999999999996,"5min":831.88099999999997,"15min":831.88099999999997},"last":1.044},{"interface":"front","average":{"1min":2.456,"5min":1.6870000000000001,"15min":1.5589999999999999},"min":{"1min":1,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":7.5350000000000001,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":1.1279999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.542,"15min":1.526},"min":{"1min":0.83599999999999997,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.9900000000000002,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.9219999999999999,"5min":1.5900000000000001,"15min":1.534},"min":{"1min":1.115,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":4.5170000000000003,"5min":4.5170000000000003,"15min":4.5170000000000003},"last":1.478}]}]},{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1180916,"kb_used_data":132332,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93190924,"statfs":{"total":96636764160,"available":95427506176,"internally_reserved":1073741824,"allocated":135507968,"data_stored":134876248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.2490000000000001,"15min":1.3049999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.6439999999999999,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":1.081},{"interface":"front","average":{"1min":1.079,"5min":1.135,"15min":1.145},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":2.9950000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.3740000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.95999999999999996,"15min":0.95799999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.9890000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":19.103999999999999,"5min":4.9299999999999997,"15min":2.5680000000000001},"min":{"1min":0.75600000000000001,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":372.50799999999998,"5min":372.50799999999998,"15min":372.50799999999998},"last":1.391}]},{"osd":2,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2,"15min":1.196},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":3.036,"5min":3.036,"15min":3.036},"last":0.876},{"interface":"front","average":{"1min":1.093,"5min":1.236,"15min":1.26},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.7440000000000002,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":1.2150000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.4870000000000001,"15min":1.5129999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":3.46,"5min":3.46,"15min":3.46},"last":1.054},{"interface":"front","average":{"1min":1.268,"5min":1.3640000000000001,"15min":1.3799999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":3.1970000000000001,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.91900000000000004}]},{"osd":4,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.5880000000000001,"15min":1.6379999999999999},"min":{"1min":0.72499999999999998,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.7999999999999998,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":0.996},{"interface":"front","average":{"1min":1.653,"5min":1.6679999999999999,"15min":1.6699999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":3.694,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.1419999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.3260000000000001,"15min":1.351},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.6360000000000001,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":1.107},{"interface":"front","average":{"1min":1.393,"5min":1.6910000000000001,"15min":1.7410000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.8279999999999998,"5min":5.327,"15min":5.327},"last":1.357}]},{"osd":6,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.446,"15min":1.4370000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.7799999999999998,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.4730000000000001,"5min":14.955,"15min":17.202000000000002},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":3.7309999999999999,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.3360000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1284228,"kb_used_data":235644,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93087612,"statfs":{"total":96636764160,"available":95321714688,"internally_reserved":1073741824,"allocated":241299456,"data_stored":240667736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":27.559999999999999,"15min":31.977},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.1820000000000004,"5min":497.04000000000002,"15min":497.04000000000002},"last":0.61899999999999999},{"interface":"front","average":{"1min":44.500999999999998,"5min":9.8049999999999997,"15min":4.0220000000000002},"min":{"1min":0.64200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":993.84500000000003,"5min":993.84500000000003,"15min":993.84500000000003},"last":0.71099999999999997}]},{"osd":2,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":0.98799999999999999,"15min":0.92800000000000005},"min":{"1min":0.57399999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":4.585,"5min":4.585,"15min":4.585},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.2410000000000001,"15min":1.216},"min":{"1min":0.51800000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":4.149,"5min":4.149,"15min":4.149},"last":0.67400000000000004}]},{"osd":3,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.962,"5min":1.4410000000000001,"15min":1.3540000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":10.603999999999999,"5min":10.603999999999999,"15min":10.603999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":2.1909999999999998,"5min":1.601,"15min":1.5029999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":10.638999999999999,"5min":10.638999999999999,"15min":10.638999999999999},"last":0.874}]},{"osd":4,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":2.1869999999999998,"5min":1.464,"15min":1.3440000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":10.69,"5min":10.69,"15min":10.69},"last":1.089},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.264,"15min":1.2210000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":4.9500000000000002,"5min":4.9500000000000002,"15min":4.9500000000000002},"last":1.141}]},{"osd":5,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":2.0630000000000002,"5min":10.819000000000001,"15min":12.278},"min":{"1min":0.73399999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":10.720000000000001,"5min":213.63800000000001,"15min":213.63800000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.258,"15min":1.212},"min":{"1min":0.82899999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":4.6269999999999998,"5min":4.6269999999999998,"15min":4.6269999999999998},"last":0.99199999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:08:24 2020","interfaces":[{"interface":"back","average":{"1min":1.7589999999999999,"5min":1.593,"15min":1.5649999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":4.5259999999999998,"5min":4.5259999999999998,"15min":4.5259999999999998},"last":0.90000000000000002},{"interface":"front","average":{"1min":2.3690000000000002,"5min":1.5620000000000001,"15min":1.4279999999999999},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":10.792,"5min":10.792,"15min":10.792},"last":1.1100000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.8129999999999999,"5min":1.6539999999999999,"15min":1.6279999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":4.5549999999999997,"5min":4.5549999999999997,"15min":4.5549999999999997},"last":1.0640000000000001},{"interface":"front","average":{"1min":2.2719999999999998,"5min":1.6379999999999999,"15min":1.532},"min":{"1min":0.998,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":10.752000000000001,"5min":10.752000000000001,"15min":10.752000000000001},"last":1.1719999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1155716,"kb_used_data":107132,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93216124,"statfs":{"total":96636764160,"available":95453310976,"internally_reserved":1073741824,"allocated":109703168,"data_stored":109071448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:08:24 2020","interfaces":[{"interface":"back","average":{"1min":81.266000000000005,"5min":17.033999999999999,"15min":6.3289999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":845.57100000000003,"5min":845.57100000000003,"15min":845.57100000000003},"last":1.327},{"interface":"front","average":{"1min":1.083,"5min":0.96299999999999997,"15min":0.94299999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":5.5970000000000004,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":1.74}]},{"osd":2,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.0069999999999999,"15min":0.95399999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":4.1600000000000001,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":1.036},{"interface":"front","average":{"1min":1.0049999999999999,"5min":26.393000000000001,"15min":30.623999999999999},"min":{"1min":0.54200000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.1280000000000001,"5min":606.18499999999995,"15min":606.18499999999995},"last":1.7929999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.0549999999999999,"15min":0.97499999999999998},"min":{"1min":0.64000000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":3.1779999999999999,"5min":3.1779999999999999,"15min":3.1779999999999999},"last":1.6859999999999999},{"interface":"front","average":{"1min":1.927,"5min":1.319,"15min":1.218},"min":{"1min":0.78700000000000003,"5min":0.621,"15min":0.621},"max":{"1min":4.21,"5min":4.21,"15min":4.21},"last":1.657}]},{"osd":4,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.238,"15min":1.1899999999999999},"min":{"1min":0.73899999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":3.0870000000000002,"5min":3.0870000000000002,"15min":3.0870000000000002},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.6890000000000001,"5min":1.3440000000000001,"15min":1.286},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":4.0960000000000001,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":2.0449999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":39.531999999999996,"5min":8.8879999999999999,"15min":3.7799999999999998},"min":{"1min":0.76600000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":501.267,"5min":501.267,"15min":501.267},"last":2.246},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.24,"15min":1.1499999999999999},"min":{"1min":0.874,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":6.0010000000000003,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":2.0939999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.7729999999999999,"5min":7.2450000000000001,"15min":8.157},"min":{"1min":0.81100000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":4.1310000000000002,"5min":124.738,"15min":124.738},"last":2.0059999999999998},{"interface":"front","average":{"1min":1.47,"5min":15.215999999999999,"15min":17.507000000000001},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":3.7120000000000002,"5min":294.94,"15min":294.94},"last":2.2789999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.552,"15min":1.5740000000000001},"min":{"1min":0.96199999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.3929999999999998,"5min":6.468,"15min":6.468},"last":2.3069999999999999},{"interface":"front","average":{"1min":1.6879999999999999,"5min":1.446,"15min":1.4059999999999999},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":3.117,"5min":3.117,"15min":3.117},"last":2.0659999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1111696,"kb_used_data":63112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93260144,"statfs":{"total":96636764160,"available":95498387456,"internally_reserved":1073741824,"allocated":64626688,"data_stored":63987505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.153,"15min":1.129},"min":{"1min":0.505,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.4909999999999997,"5min":4.4909999999999997,"15min":4.4909999999999997},"last":8.4930000000000003},{"interface":"front","average":{"1min":1.284,"5min":1.036,"15min":0.995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":3.6520000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":3.5}]},{"osd":1,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":2.5219999999999998,"5min":1.3779999999999999,"15min":1.1870000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":18.672000000000001,"5min":18.672000000000001,"15min":18.672000000000001},"last":5.6820000000000004},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.357,"15min":1.359},"min":{"1min":0.70699999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":5.1130000000000004,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":2.2250000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.216,"15min":1.1799999999999999},"min":{"1min":0.65300000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":1.736},{"interface":"front","average":{"1min":1.419,"5min":1.528,"15min":1.546},"min":{"1min":0.59999999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":5.3339999999999996,"5min":8.2210000000000001,"15min":8.2210000000000001},"last":1.4590000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":2.3319999999999999,"5min":1.3540000000000001,"15min":1.1910000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":19.783000000000001,"5min":19.783000000000001,"15min":19.783000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.5589999999999999,"15min":1.615},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.891,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":2.2829999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.2789999999999999,"15min":1.2669999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.7730000000000001,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.5249999999999999,"5min":20.725000000000001,"15min":23.925000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":4.4790000000000001,"5min":415.40699999999998,"15min":415.40699999999998},"last":2.532}]},{"osd":6,"last update":"Fri Aug 7 15:08:25 2020","interfaces":[{"interface":"back","average":{"1min":2.5089999999999999,"5min":1.478,"15min":1.306},"min":{"1min":0.89000000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":20.035,"5min":20.035,"15min":20.035},"last":0.85999999999999999},{"interface":"front","average":{"1min":44.033000000000001,"5min":9.8629999999999995,"15min":4.1680000000000001},"min":{"1min":0.93700000000000006,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":751.70000000000005,"5min":751.70000000000005,"15min":751.70000000000005},"last":1.6719999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":3.5409999999999999,"5min":2.101,"15min":1.861},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":35.548999999999999,"5min":35.548999999999999,"15min":35.548999999999999},"last":2.254},{"interface":"front","average":{"1min":3.4460000000000002,"5min":1.8859999999999999,"15min":1.6259999999999999},"min":{"1min":1.004,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":35.646999999999998,"5min":35.646999999999998,"15min":35.646999999999998},"last":1.7070000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1160944,"kb_used_data":112360,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93210896,"statfs":{"total":96636764160,"available":95447957504,"internally_reserved":1073741824,"allocated":115056640,"data_stored":114417457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.044,"5min":1.411,"15min":1.3049999999999999},"min":{"1min":0.55600000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":8.6020000000000003,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":1.383},{"interface":"front","average":{"1min":1.45,"5min":1.599,"15min":1.6240000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":4.9119999999999999,"5min":13.067,"15min":13.067},"last":0.79500000000000004}]},{"osd":1,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":16.382999999999999,"5min":4.7949999999999999,"15min":2.8639999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":270.68700000000001,"5min":270.68700000000001,"15min":270.68700000000001},"last":1.093},{"interface":"front","average":{"1min":2.7210000000000001,"5min":2.153,"15min":2.0590000000000002},"min":{"1min":0.71599999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":14.349,"5min":14.349,"15min":14.349},"last":7.0940000000000003}]},{"osd":2,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":1.7470000000000001,"5min":1.2669999999999999,"15min":1.1870000000000001},"min":{"1min":0.64900000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":4.9580000000000002,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":1.198},{"interface":"front","average":{"1min":4.5220000000000002,"5min":2.2519999999999998,"15min":1.8740000000000001},"min":{"1min":0.72399999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":51.530000000000001,"5min":51.530000000000001,"15min":51.530000000000001},"last":1.526}]},{"osd":4,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":47.652999999999999,"5min":10.888,"15min":4.7599999999999998},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":506.21300000000002,"5min":506.21300000000002,"15min":506.21300000000002},"last":1.752},{"interface":"front","average":{"1min":29.771000000000001,"5min":31.634,"15min":31.943999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":505.738,"5min":677.88599999999997,"15min":677.88599999999997},"last":1.498}]},{"osd":5,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.012,"5min":1.6160000000000001,"15min":1.55},"min":{"1min":0.96399999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":5.0049999999999999,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.5589999999999999},{"interface":"front","average":{"1min":2.2210000000000001,"5min":1.859,"15min":1.7989999999999999},"min":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":5.226,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":1.595}]},{"osd":6,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":1.99,"5min":1.623,"15min":1.5620000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":5.0389999999999997,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":2.1850000000000001},{"interface":"front","average":{"1min":1.9710000000000001,"5min":1.9830000000000001,"15min":1.9850000000000001},"min":{"1min":1.0369999999999999,"5min":0.876,"15min":0.876},"max":{"1min":4.266,"5min":5.6189999999999998,"15min":5.6189999999999998},"last":1.96}]},{"osd":7,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.0110000000000001,"5min":2.1309999999999998,"15min":2.1509999999999998},"min":{"1min":0.96999999999999997,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":3.79,"5min":5.734,"15min":5.734},"last":1.3200000000000001},{"interface":"front","average":{"1min":2.6200000000000001,"5min":2.105,"15min":2.0190000000000001},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":8.6790000000000003,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":1.7889999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1216808,"kb_used_data":168224,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93155032,"statfs":{"total":96636764160,"available":95390752768,"internally_reserved":1073741824,"allocated":172261376,"data_stored":171622193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":1.125,"15min":1.1619999999999999},"min":{"1min":0.47699999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.3999999999999999,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.349,"5min":0.94899999999999995,"15min":0.88200000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":6.3970000000000002,"5min":6.3970000000000002,"15min":6.3970000000000002},"last":1.194}]},{"osd":1,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.304,"15min":1.337},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.4380000000000002,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":1.397},{"interface":"front","average":{"1min":32.585999999999999,"5min":7.6180000000000003,"15min":3.4569999999999999},"min":{"1min":0.67000000000000004,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":689.846,"5min":689.846,"15min":689.846},"last":1.379}]},{"osd":2,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.46,"15min":1.454},"min":{"1min":0.56399999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":6.8730000000000002,"5min":6.8730000000000002,"15min":6.8730000000000002},"last":1.165},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.5669999999999999,"15min":1.532},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":7.1360000000000001,"5min":7.1360000000000001,"15min":7.1360000000000001},"last":1.222}]},{"osd":3,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":12.619999999999999,"5min":3.73,"15min":2.2480000000000002},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":247.202,"5min":247.202,"15min":247.202},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.583,"5min":1.3169999999999999,"15min":1.2729999999999999},"min":{"1min":0.67800000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":6.431,"5min":6.431,"15min":6.431},"last":1.244}]},{"osd":5,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":46.786999999999999,"5min":10.659000000000001,"15min":4.6379999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":990.71600000000001,"5min":990.71600000000001,"15min":990.71600000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.748,"5min":1.651,"15min":1.635},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":3.9329999999999998,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":1.3169999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.7070000000000001,"5min":1.7729999999999999,"15min":1.784},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":6.9749999999999996,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.524,"5min":15.592000000000001,"15min":17.936},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":6.7729999999999997,"5min":389.14600000000002,"15min":389.14600000000002},"last":0.99299999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":13.378,"5min":3.802,"15min":2.206},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":252.798,"5min":252.798,"15min":252.798},"last":0.95299999999999996},{"interface":"front","average":{"1min":44.676000000000002,"5min":10.252000000000001,"15min":4.5140000000000002},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":501.93000000000001,"5min":501.93000000000001,"15min":501.93000000000001},"last":1.03}]}]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1176656,"kb_used_data":128072,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93195184,"statfs":{"total":96636764160,"available":95431868416,"internally_reserved":1073741824,"allocated":131145728,"data_stored":130506545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0580000000000001,"15min":1.0700000000000001},"min":{"1min":0.49199999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.5169999999999999,"5min":2.766,"15min":2.766},"last":0.51700000000000002},{"interface":"front","average":{"1min":1.2250000000000001,"5min":0.88900000000000001,"15min":0.83299999999999996},"min":{"1min":0.54900000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":0.71599999999999997}]},{"osd":1,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":47.365000000000002,"5min":10.215999999999999,"15min":4.024},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":787.22900000000004,"5min":787.22900000000004,"15min":787.22900000000004},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.071,"5min":1.1000000000000001,"15min":1.105},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.8380000000000001,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":0.60999999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":7.5659999999999998,"15min":8.6679999999999993},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.4950000000000001,"5min":168.56899999999999,"15min":168.56899999999999},"last":1.026},{"interface":"front","average":{"1min":1.2929999999999999,"5min":26.763000000000002,"15min":31.007999999999999},"min":{"1min":0.79700000000000004,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":3.2389999999999999,"5min":676.27099999999996,"15min":676.27099999999996},"last":1.099}]},{"osd":3,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.196,"15min":1.1439999999999999},"min":{"1min":0.89000000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":3.5649999999999999,"5min":3.5649999999999999,"15min":3.5649999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":43.073,"5min":9.5850000000000009,"15min":4.0030000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":712.98699999999997,"5min":712.98699999999997,"15min":712.98699999999997},"last":2.2010000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":40.962000000000003,"15min":47.582000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.4980000000000002,"5min":530.30799999999999,"15min":530.30799999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.278,"15min":1.296},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.6259999999999999,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":1.137}]},{"osd":6,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":40.569000000000003,"15min":47.128999999999998},"min":{"1min":0.76800000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.6699999999999999,"5min":754.31299999999999,"15min":754.31299999999999},"last":966.60900000000004},{"interface":"front","average":{"1min":1.1899999999999999,"5min":21.050999999999998,"15min":24.361000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.7809999999999999,"5min":517.26700000000005,"15min":517.26700000000005},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.3560000000000001,"15min":1.3240000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":3.9079999999999999,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.23},{"interface":"front","average":{"1min":1.411,"5min":1.5349999999999999,"15min":1.556},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":3.9390000000000001,"5min":3.9390000000000001,"15min":3.9390000000000001},"last":1.288}]}]}]}} 2020-08-07T15:09:03.600 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:03.599+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:03.667 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:03.668 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:03.684 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:03.683+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:03.685 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:09:03.794 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:03.793+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:03.854 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:03.854 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:03.864 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:09:03.896 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:03.893+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:04.002 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:03.998+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:04.023 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:04.023 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:04.035 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:09:04.122 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:04.120+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:04.204 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:04.202+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:04.205 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:04.205 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:04.213 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:09:04.219 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:09:04.278 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 38 16 44733 44717 9.19257 2.11719 0.0117973 0.0135922 2020-08-07T15:09:04.324 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:04.316+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:04.414 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:04.405+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:04.448 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:04.449 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:04.467 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:09:04.511 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:04.509+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:04.614 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:04.613+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:04.693 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:04.694 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:04.707 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:09:04.716 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:09:04.717 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:04.715+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:04.822 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:04.821+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:04.896 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:04.896 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:04.911 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:09:04.963 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:04.951+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:04.963 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:09:05.003 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:09:05.039 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:05.037+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:05.048 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:09:05.086 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:09:05.122 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:09:05.141 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:09:05.141 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:09:05.141 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:09:05.141 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:09:05.142 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:09:05.142 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:05.140+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:05.171 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:09:05.172 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:09:05.203 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:09:05.243 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:09:05.243 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:09:05.243 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:09:05.244 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:09:05.244 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:09:05.244 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:09:05.244 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:09:05.244 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:09:05.245 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:05.242+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:05.284 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 39 16 46217 46201 9.25411 11.5938 0.00747676 0.0135011 2020-08-07T15:09:05.327 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:09:05.383 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:09:05.383 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:09:05.384 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:09:05.384 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:09:05.384 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.384 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106240 3.7 3:ebdd8242:::benchmark_data_smithi074_35447_object13279:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:09:05.384 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:09:05.385 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 78.022370387999999, 2020-08-07T15:09:05.385 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.90079396, 2020-08-07T15:09:05.385 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:05.385 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:05.385 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:05.414 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:09:05.414 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:05.415 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106240 2020-08-07T15:09:05.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.415 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:05.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.415 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:05.416 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:09:05.416 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:05.416 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.416 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.416 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:05.417 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:09:05.417 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2522000000000003e-05 2020-08-07T15:09:05.417 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.417 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.417 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:05.424 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355118+0000", 2020-08-07T15:09:05.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6159999999999998e-06 2020-08-07T15:09:05.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.425 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:05.425 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355125+0000", 2020-08-07T15:09:05.426 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2699e-05 2020-08-07T15:09:05.426 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.426 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.426 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:05.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355138+0000", 2020-08-07T15:09:05.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2310000000000001e-06 2020-08-07T15:09:05.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.429 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:05.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355144+0000", 2020-08-07T15:09:05.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9855000000000003e-05 2020-08-07T15:09:05.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.430 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.430 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355194+0000", 2020-08-07T15:09:05.430 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112713 2020-08-07T15:09:05.430 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.430 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.431 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:05.431 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355306+0000", 2020-08-07T15:09:05.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019794600000000001 2020-08-07T15:09:05.431 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.432 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:09:05.432 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355504+0000", 2020-08-07T15:09:05.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0048346029999999998 2020-08-07T15:09:05.433 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.433 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.433 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:05.433 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.360339+0000", 2020-08-07T15:09:05.434 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00033646400000000002 2020-08-07T15:09:05.434 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.434 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.434 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:05.434 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.360675+0000", 2020-08-07T15:09:05.435 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89508481200000001 2020-08-07T15:09:05.435 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.435 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.435 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:05.435 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255760+0000", 2020-08-07T15:09:05.436 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5659999999999999e-06 2020-08-07T15:09:05.436 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.436 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.436 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:05.436 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255762+0000", 2020-08-07T15:09:05.437 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6595000000000003e-05 2020-08-07T15:09:05.437 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.437 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.437 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:05.437 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255818+0000", 2020-08-07T15:09:05.438 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1338000000000003e-05 2020-08-07T15:09:05.438 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.438 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.438 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:05.438 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255860+0000", 2020-08-07T15:09:05.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.90079396 2020-08-07T15:09:05.439 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.439 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:05.439 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.439 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.440 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106240 3.7 e59/20 ondisk, result = 0)", 2020-08-07T15:09:05.440 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:09:05.440 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 77.925974436000004, 2020-08-07T15:09:05.440 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80440215100000001, 2020-08-07T15:09:05.441 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:05.441 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:09:05.441 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:05.441 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.441 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:05.441 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:09:05.442 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:05.442 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.442 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.442 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:05.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:09:05.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0052999999999999e-05 2020-08-07T15:09:05.444 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.445 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:05.445 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451482+0000", 2020-08-07T15:09:05.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3229999999999999e-06 2020-08-07T15:09:05.445 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.445 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.446 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:05.446 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451486+0000", 2020-08-07T15:09:05.446 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80420304799999998 2020-08-07T15:09:05.446 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.446 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.447 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:05.447 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255689+0000", 2020-08-07T15:09:05.447 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7759999999999997e-06 2020-08-07T15:09:05.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:05.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255694+0000", 2020-08-07T15:09:05.448 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5828999999999998e-05 2020-08-07T15:09:05.448 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.449 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.449 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255740+0000", 2020-08-07T15:09:05.449 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1466e-05 2020-08-07T15:09:05.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.449 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.450 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:05.450 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255751+0000", 2020-08-07T15:09:05.450 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112656 2020-08-07T15:09:05.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.451 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:05.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255864+0000", 2020-08-07T15:09:05.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80440215100000001 2020-08-07T15:09:05.451 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.451 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:05.452 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.452 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37057 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:05.452 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:09:05.454 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.133865726, 2020-08-07T15:09:05.454 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007248691, 2020-08-07T15:09:05.454 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:05.454 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:05.455 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:05.455 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:05.455 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:05.455 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37057 2020-08-07T15:09:05.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.456 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:05.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:05.456 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:09:05.456 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:05.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:05.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:09:05.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3661e-05 2020-08-07T15:09:05.458 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.458 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.458 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:05.458 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243624+0000", 2020-08-07T15:09:05.459 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7519999999999996e-06 2020-08-07T15:09:05.459 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:05.459 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243632+0000", 2020-08-07T15:09:05.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6392000000000002e-05 2020-08-07T15:09:05.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.460 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.460 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:05.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243648+0000", 2020-08-07T15:09:05.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6659999999999997e-06 2020-08-07T15:09:05.461 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.461 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.462 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:05.462 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243657+0000", 2020-08-07T15:09:05.462 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0070484609999999998 2020-08-07T15:09:05.462 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.462 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.463 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.463 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250705+0000", 2020-08-07T15:09:05.463 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0083e-05 2020-08-07T15:09:05.463 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.464 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.464 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:05.464 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250715+0000", 2020-08-07T15:09:05.465 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000086695 2020-08-07T15:09:05.465 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.465 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.465 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.466 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.250802+0000", 2020-08-07T15:09:05.466 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6980999999999999e-05 2020-08-07T15:09:05.466 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.466 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.466 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:05.467 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.250819+0000", 2020-08-07T15:09:05.467 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007248691 2020-08-07T15:09:05.467 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.467 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:05.467 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.468 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.468 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.468 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37058 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:05.469 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:09:05.469 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.133723992, 2020-08-07T15:09:05.469 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007368357, 2020-08-07T15:09:05.469 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:05.470 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:05.470 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:05.470 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:05.470 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:05.470 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37058 2020-08-07T15:09:05.471 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.471 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:05.471 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.471 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:05.472 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:09:05.472 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:05.472 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.472 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.472 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:05.473 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:09:05.473 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8838999999999998e-05 2020-08-07T15:09:05.473 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.473 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.474 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:05.474 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243761+0000", 2020-08-07T15:09:05.474 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2079999999999999e-06 2020-08-07T15:09:05.475 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.475 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.475 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:05.475 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243766+0000", 2020-08-07T15:09:05.476 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0567999999999999e-05 2020-08-07T15:09:05.476 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.476 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.476 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:05.477 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243777+0000", 2020-08-07T15:09:05.477 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7210000000000002e-06 2020-08-07T15:09:05.477 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.477 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.477 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:05.478 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243782+0000", 2020-08-07T15:09:05.478 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069508749999999996 2020-08-07T15:09:05.478 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.478 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.478 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.478 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250732+0000", 2020-08-07T15:09:05.479 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7899999999999999e-06 2020-08-07T15:09:05.479 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.479 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.479 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:05.479 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250737+0000", 2020-08-07T15:09:05.480 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003272409999999 2020-08-07T15:09:05.480 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.482 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.483 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251064+0000", 2020-08-07T15:09:05.483 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6115000000000001e-05 2020-08-07T15:09:05.483 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.483 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.484 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:05.484 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251081+0000", 2020-08-07T15:09:05.484 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007368357 2020-08-07T15:09:05.484 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.484 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:05.485 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.485 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.486 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.486 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37059 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:05.486 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:09:05.486 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.133480943, 2020-08-07T15:09:05.486 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0071655319999999, 2020-08-07T15:09:05.487 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:05.487 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:05.487 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:05.487 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:05.487 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:05.488 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37059 2020-08-07T15:09:05.488 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.488 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:05.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:05.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:09:05.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:05.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.489 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.489 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:05.490 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:09:05.490 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6533000000000002e-05 2020-08-07T15:09:05.490 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.490 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.491 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:05.491 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244012+0000", 2020-08-07T15:09:05.491 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3860000000000001e-06 2020-08-07T15:09:05.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.491 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.492 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:05.492 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244019+0000", 2020-08-07T15:09:05.492 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6982000000000001e-05 2020-08-07T15:09:05.492 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.492 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.492 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:05.493 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244036+0000", 2020-08-07T15:09:05.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0310000000000001e-06 2020-08-07T15:09:05.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.493 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:05.494 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244043+0000", 2020-08-07T15:09:05.494 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0067169259999999998 2020-08-07T15:09:05.494 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.494 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.495 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250760+0000", 2020-08-07T15:09:05.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4499999999999998e-06 2020-08-07T15:09:05.495 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.496 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.496 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:05.496 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250768+0000", 2020-08-07T15:09:05.496 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000341691 2020-08-07T15:09:05.497 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.497 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.497 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.497 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251109+0000", 2020-08-07T15:09:05.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1533e-05 2020-08-07T15:09:05.498 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.498 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.498 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:05.498 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251121+0000", 2020-08-07T15:09:05.498 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0071655319999999 2020-08-07T15:09:05.499 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.499 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:05.499 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.499 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.499 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.500 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37060 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:05.500 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:09:05.500 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.133357006000001, 2020-08-07T15:09:05.500 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070762280000001, 2020-08-07T15:09:05.500 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:05.500 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:05.501 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:05.501 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:05.502 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:05.502 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37060 2020-08-07T15:09:05.503 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.503 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:05.503 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.503 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:05.503 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:09:05.504 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:05.504 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.504 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.504 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:05.504 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:09:05.505 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5739000000000003e-05 2020-08-07T15:09:05.505 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.505 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.505 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:05.505 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244135+0000", 2020-08-07T15:09:05.506 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7740000000000001e-06 2020-08-07T15:09:05.507 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.507 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.507 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:05.507 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244141+0000", 2020-08-07T15:09:05.507 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3817e-05 2020-08-07T15:09:05.508 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.508 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.508 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:05.508 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244155+0000", 2020-08-07T15:09:05.508 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3829999999999997e-06 2020-08-07T15:09:05.509 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.509 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.509 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:05.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244161+0000", 2020-08-07T15:09:05.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0066303669999999999 2020-08-07T15:09:05.510 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.510 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.510 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250791+0000", 2020-08-07T15:09:05.510 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.392e-06 2020-08-07T15:09:05.511 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:05.511 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250798+0000", 2020-08-07T15:09:05.511 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003485860000001 2020-08-07T15:09:05.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.512 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.512 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.512 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251146+0000", 2020-08-07T15:09:05.512 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1700000000000003e-06 2020-08-07T15:09:05.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.513 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.513 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:05.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251155+0000", 2020-08-07T15:09:05.513 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070762280000001 2020-08-07T15:09:05.513 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.514 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:05.514 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.514 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.514 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37061 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:05.515 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:09:05.515 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.133242379, 2020-08-07T15:09:05.515 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070006920000001, 2020-08-07T15:09:05.515 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:05.515 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:05.516 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:05.516 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:05.516 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:05.516 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37061 2020-08-07T15:09:05.517 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.517 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:05.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.518 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:05.518 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:09:05.518 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:05.519 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.519 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.519 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:05.519 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:09:05.520 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5453e-05 2020-08-07T15:09:05.520 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.520 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:05.521 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244239+0000", 2020-08-07T15:09:05.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3179999999999997e-06 2020-08-07T15:09:05.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.521 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.521 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:05.522 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244245+0000", 2020-08-07T15:09:05.522 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2138e-05 2020-08-07T15:09:05.522 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.522 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.522 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:05.523 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244257+0000", 2020-08-07T15:09:05.523 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.69e-06 2020-08-07T15:09:05.523 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.523 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.523 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:05.524 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244261+0000", 2020-08-07T15:09:05.524 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0065598940000000001 2020-08-07T15:09:05.524 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.524 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.524 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.524 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250821+0000", 2020-08-07T15:09:05.525 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4799999999999998e-06 2020-08-07T15:09:05.525 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.525 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.525 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:05.525 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250828+0000", 2020-08-07T15:09:05.526 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003567310000001 2020-08-07T15:09:05.526 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.526 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.526 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.527 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251185+0000", 2020-08-07T15:09:05.527 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9879999999999999e-06 2020-08-07T15:09:05.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.528 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:05.528 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251195+0000", 2020-08-07T15:09:05.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070006920000001 2020-08-07T15:09:05.529 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.529 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:05.529 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.529 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.538 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.540 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37062 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:05.540 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:09:05.541 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.133142161, 2020-08-07T15:09:05.541 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006933662, 2020-08-07T15:09:05.541 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:05.541 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:05.541 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:05.542 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:05.542 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:05.542 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37062 2020-08-07T15:09:05.542 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.542 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:05.543 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.543 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:05.543 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:09:05.543 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:05.543 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.544 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.544 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:05.544 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:09:05.544 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4041e-05 2020-08-07T15:09:05.544 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.545 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:05.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244348+0000", 2020-08-07T15:09:05.546 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2079999999999997e-06 2020-08-07T15:09:05.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.546 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.546 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:05.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244354+0000", 2020-08-07T15:09:05.547 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1483e-05 2020-08-07T15:09:05.547 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.547 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:05.548 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244366+0000", 2020-08-07T15:09:05.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7989999999999999e-06 2020-08-07T15:09:05.549 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.549 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.549 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:05.549 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244372+0000", 2020-08-07T15:09:05.549 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0064783699999999998 2020-08-07T15:09:05.550 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.550 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.550 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.550 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250850+0000", 2020-08-07T15:09:05.550 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4729999999999997e-06 2020-08-07T15:09:05.551 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.551 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.551 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:05.551 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250856+0000", 2020-08-07T15:09:05.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003630240000001 2020-08-07T15:09:05.552 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.552 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.552 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251219+0000", 2020-08-07T15:09:05.552 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2640000000000008e-06 2020-08-07T15:09:05.553 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.553 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.553 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:05.553 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251228+0000", 2020-08-07T15:09:05.554 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006933662 2020-08-07T15:09:05.554 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.554 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:05.554 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.555 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.555 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.555 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37063 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:05.555 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:09:05.555 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.133029827, 2020-08-07T15:09:05.556 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006858051, 2020-08-07T15:09:05.556 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:05.556 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:05.556 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:05.557 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:05.557 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:05.557 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37063 2020-08-07T15:09:05.557 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.557 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:05.558 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.558 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:05.558 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:09:05.560 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:05.561 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.561 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.561 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:05.561 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:09:05.561 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5043000000000001e-05 2020-08-07T15:09:05.562 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.562 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.562 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:05.562 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244461+0000", 2020-08-07T15:09:05.562 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4600000000000002e-06 2020-08-07T15:09:05.563 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.567 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.567 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:05.567 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244467+0000", 2020-08-07T15:09:05.568 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1435e-05 2020-08-07T15:09:05.568 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.595 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.596 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:05.596 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244478+0000", 2020-08-07T15:09:05.596 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0190000000000003e-06 2020-08-07T15:09:05.596 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.597 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.597 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:05.597 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244484+0000", 2020-08-07T15:09:05.597 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006392895 2020-08-07T15:09:05.598 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.598 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.598 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.598 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250877+0000", 2020-08-07T15:09:05.598 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4280000000000001e-06 2020-08-07T15:09:05.599 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.599 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.599 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:05.599 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250884+0000", 2020-08-07T15:09:05.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000370414 2020-08-07T15:09:05.600 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.600 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.600 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.600 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251254+0000", 2020-08-07T15:09:05.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0356999999999999e-05 2020-08-07T15:09:05.600 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.601 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:05.601 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251264+0000", 2020-08-07T15:09:05.601 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006858051 2020-08-07T15:09:05.607 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.608 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:05.609 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.609 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.609 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.610 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37064 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:05.610 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:09:05.610 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.132917694, 2020-08-07T15:09:05.610 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0067798189999999, 2020-08-07T15:09:05.610 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:05.611 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:05.611 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:05.611 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:05.619 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:05.620 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37064 2020-08-07T15:09:05.620 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.620 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:05.620 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.621 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:05.621 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:09:05.621 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:05.621 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.622 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.622 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:05.622 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:09:05.622 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4007e-05 2020-08-07T15:09:05.622 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.623 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.623 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:05.623 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244573+0000", 2020-08-07T15:09:05.623 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.831e-06 2020-08-07T15:09:05.623 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.624 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.624 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:05.624 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244577+0000", 2020-08-07T15:09:05.624 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2269000000000001e-05 2020-08-07T15:09:05.624 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.625 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.625 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:05.625 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244590+0000", 2020-08-07T15:09:05.625 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7149999999999997e-06 2020-08-07T15:09:05.637 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.638 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.638 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:05.640 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244594+0000", 2020-08-07T15:09:05.640 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063105670000000004 2020-08-07T15:09:05.641 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.641 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.641 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.641 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250905+0000", 2020-08-07T15:09:05.641 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1970000000000001e-06 2020-08-07T15:09:05.642 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.642 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.642 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:05.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250911+0000", 2020-08-07T15:09:05.642 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003783020000001 2020-08-07T15:09:05.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.643 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.643 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.643 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251289+0000", 2020-08-07T15:09:05.672 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9309999999999997e-06 2020-08-07T15:09:05.672 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.673 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.673 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:05.673 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251298+0000", 2020-08-07T15:09:05.673 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0067798189999999 2020-08-07T15:09:05.673 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.674 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:05.674 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.674 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.674 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.674 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37065 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:05.675 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:09:05.675 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.132807475, 2020-08-07T15:09:05.675 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006708545, 2020-08-07T15:09:05.675 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:05.675 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:05.676 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:05.676 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:05.676 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:05.676 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37065 2020-08-07T15:09:05.676 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.676 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:05.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.707 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:05.708 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:09:05.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:05.708 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.709 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:05.709 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:09:05.709 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2750000000000001e-05 2020-08-07T15:09:05.709 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.709 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.717 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:05.717 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244681+0000", 2020-08-07T15:09:05.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0950000000000003e-06 2020-08-07T15:09:05.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.718 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.718 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:05.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244687+0000", 2020-08-07T15:09:05.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2485e-05 2020-08-07T15:09:05.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:05.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244699+0000", 2020-08-07T15:09:05.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8800000000000002e-06 2020-08-07T15:09:05.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:05.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244706+0000", 2020-08-07T15:09:05.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0062281569999999998 2020-08-07T15:09:05.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.723 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250934+0000", 2020-08-07T15:09:05.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1719999999999998e-06 2020-08-07T15:09:05.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.724 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.725 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:05.725 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250939+0000", 2020-08-07T15:09:05.725 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003861810000001 2020-08-07T15:09:05.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.726 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.726 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.726 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251325+0000", 2020-08-07T15:09:05.726 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1824999999999999e-05 2020-08-07T15:09:05.726 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.726 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.727 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:05.727 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251337+0000", 2020-08-07T15:09:05.727 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006708545 2020-08-07T15:09:05.727 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.727 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:05.728 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.728 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37066 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:05.728 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:09:05.729 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.128138505000001, 2020-08-07T15:09:05.729 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002077205, 2020-08-07T15:09:05.729 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:05.729 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:05.729 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:05.730 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:05.730 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:05.730 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37066 2020-08-07T15:09:05.730 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.730 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:05.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.731 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:05.731 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:09:05.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:05.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.738 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.738 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:05.738 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:09:05.739 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4654000000000002e-05 2020-08-07T15:09:05.739 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.739 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.739 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:05.739 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249352+0000", 2020-08-07T15:09:05.740 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6210000000000004e-06 2020-08-07T15:09:05.740 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.740 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.740 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:05.740 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249360+0000", 2020-08-07T15:09:05.741 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1895999999999998e-05 2020-08-07T15:09:05.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.741 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.741 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:05.741 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249382+0000", 2020-08-07T15:09:05.749 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1479999999999995e-06 2020-08-07T15:09:05.749 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.750 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.750 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:05.750 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249391+0000", 2020-08-07T15:09:05.750 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015731510000000001 2020-08-07T15:09:05.753 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.754 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.754 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.754 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250964+0000", 2020-08-07T15:09:05.754 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5400000000000001e-06 2020-08-07T15:09:05.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.759 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:05.759 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250971+0000", 2020-08-07T15:09:05.760 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000392379 2020-08-07T15:09:05.760 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.760 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.760 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.760 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251363+0000", 2020-08-07T15:09:05.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1816e-05 2020-08-07T15:09:05.761 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.761 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:05.761 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251375+0000", 2020-08-07T15:09:05.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002077205 2020-08-07T15:09:05.762 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.762 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:05.762 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.778 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37067 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:05.778 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:09:05.783 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.127997811, 2020-08-07T15:09:05.783 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0019761119999999, 2020-08-07T15:09:05.784 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:05.784 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:05.784 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:05.785 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:05.785 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:05.786 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37067 2020-08-07T15:09:05.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.786 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:05.787 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:05.787 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:09:05.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:05.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.790 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:05.791 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:09:05.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5961999999999997e-05 2020-08-07T15:09:05.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:05.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249494+0000", 2020-08-07T15:09:05.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3210000000000004e-06 2020-08-07T15:09:05.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.792 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.792 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:05.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249500+0000", 2020-08-07T15:09:05.793 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1012000000000001e-05 2020-08-07T15:09:05.793 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.793 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.793 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:05.793 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249511+0000", 2020-08-07T15:09:05.794 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.809e-06 2020-08-07T15:09:05.794 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:05.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249515+0000", 2020-08-07T15:09:05.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0014777670000000001 2020-08-07T15:09:05.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250993+0000", 2020-08-07T15:09:05.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9499999999999998e-06 2020-08-07T15:09:05.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.808 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:05.814 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250999+0000", 2020-08-07T15:09:05.814 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0004032519999999 2020-08-07T15:09:05.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.818 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.818 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251402+0000", 2020-08-07T15:09:05.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2038999999999999e-05 2020-08-07T15:09:05.818 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.818 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:05.819 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251414+0000", 2020-08-07T15:09:05.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0019761119999999 2020-08-07T15:09:05.821 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.821 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:05.821 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.822 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.822 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37068 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:05.822 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:09:05.822 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.12575324, 2020-08-07T15:09:05.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99977072, 2020-08-07T15:09:05.823 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:05.823 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:05.823 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:05.823 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:05.824 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:05.824 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37068 2020-08-07T15:09:05.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.824 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:05.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:05.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:09:05.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:05.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:05.826 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:09:05.828 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1369000000000003e-05 2020-08-07T15:09:05.829 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.829 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.829 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:05.830 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251724+0000", 2020-08-07T15:09:05.830 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.19e-06 2020-08-07T15:09:05.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:05.833 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251731+0000", 2020-08-07T15:09:05.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1368e-05 2020-08-07T15:09:05.833 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:05.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251742+0000", 2020-08-07T15:09:05.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.682e-06 2020-08-07T15:09:05.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:05.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251748+0000", 2020-08-07T15:09:05.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5275000000000003e-05 2020-08-07T15:09:05.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.836 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.836 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251793+0000", 2020-08-07T15:09:05.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5000000000000005e-06 2020-08-07T15:09:05.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:05.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251802+0000", 2020-08-07T15:09:05.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99963953500000002 2020-08-07T15:09:05.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.838 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.841 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251442+0000", 2020-08-07T15:09:05.841 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1800999999999999e-05 2020-08-07T15:09:05.841 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.843 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:05.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251454+0000", 2020-08-07T15:09:05.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99977072 2020-08-07T15:09:05.844 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.844 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:05.844 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.845 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37069 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:05.845 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:09:05.845 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.124649442000001, 2020-08-07T15:09:05.845 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99870607899999997, 2020-08-07T15:09:05.845 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:05.846 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:05.846 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:05.846 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:05.847 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:05.847 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37069 2020-08-07T15:09:05.847 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.847 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:05.847 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.847 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:05.848 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:09:05.848 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:05.848 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.849 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.849 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:05.850 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:09:05.850 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3755999999999999e-05 2020-08-07T15:09:05.851 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.851 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.851 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:05.851 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252831+0000", 2020-08-07T15:09:05.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7919999999999998e-06 2020-08-07T15:09:05.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.853 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:05.853 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252836+0000", 2020-08-07T15:09:05.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1409999999999999e-05 2020-08-07T15:09:05.854 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.854 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.854 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:05.854 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252848+0000", 2020-08-07T15:09:05.855 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6380000000000001e-06 2020-08-07T15:09:05.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.855 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:05.855 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252853+0000", 2020-08-07T15:09:05.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3244e-05 2020-08-07T15:09:05.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.856 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.856 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.856 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252907+0000", 2020-08-07T15:09:05.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1470000000000007e-06 2020-08-07T15:09:05.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:05.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252916+0000", 2020-08-07T15:09:05.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99856497499999997 2020-08-07T15:09:05.858 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.858 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.858 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251481+0000", 2020-08-07T15:09:05.859 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2116999999999999e-05 2020-08-07T15:09:05.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.895 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:05.902 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251493+0000", 2020-08-07T15:09:05.902 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99870607899999997 2020-08-07T15:09:05.903 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.903 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:05.912 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.912 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37070 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:05.913 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:09:05.913 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.124545179, 2020-08-07T15:09:05.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99863946699999995, 2020-08-07T15:09:05.915 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:05.916 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:05.916 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:05.916 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:05.916 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:05.917 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37070 2020-08-07T15:09:05.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.917 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:05.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:05.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:09:05.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:05.937 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.938 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:05.938 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:09:05.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9984000000000001e-05 2020-08-07T15:09:05.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:05.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252941+0000", 2020-08-07T15:09:05.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7010000000000004e-06 2020-08-07T15:09:05.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.940 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:05.940 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252946+0000", 2020-08-07T15:09:05.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1144999999999999e-05 2020-08-07T15:09:05.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:05.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252957+0000", 2020-08-07T15:09:05.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6920000000000001e-06 2020-08-07T15:09:05.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:05.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252962+0000", 2020-08-07T15:09:05.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6739000000000002e-05 2020-08-07T15:09:05.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.946 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.946 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.946 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252998+0000", 2020-08-07T15:09:05.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.503e-06 2020-08-07T15:09:05.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:05.964 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.253006+0000", 2020-08-07T15:09:05.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99851375499999995 2020-08-07T15:09:05.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:05.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251520+0000", 2020-08-07T15:09:05.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0947999999999999e-05 2020-08-07T15:09:05.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:05.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251530+0000", 2020-08-07T15:09:05.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99863946699999995 2020-08-07T15:09:05.973 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.986 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:05.986 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:05.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.987 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42505 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:09:05.987 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:09:05.987 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.0752046489999998, 2020-08-07T15:09:05.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0024736009999999, 2020-08-07T15:09:05.987 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:05.988 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:05.988 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:05.988 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:05.988 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:05.988 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42505 2020-08-07T15:09:05.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:05.989 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:05.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:05.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:05.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:09:05.999 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:06.001 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.001 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.001 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:06.001 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:09:06.002 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011052600000000001 2020-08-07T15:09:06.002 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.002 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.002 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:06.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302342+0000", 2020-08-07T15:09:06.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.781e-06 2020-08-07T15:09:06.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:06.016 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302352+0000", 2020-08-07T15:09:06.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7646e-05 2020-08-07T15:09:06.016 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:06.017 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302369+0000", 2020-08-07T15:09:06.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3629999999999994e-06 2020-08-07T15:09:06.018 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.018 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.020 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:06.020 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302378+0000", 2020-08-07T15:09:06.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8007000000000003e-05 2020-08-07T15:09:06.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:06.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302426+0000", 2020-08-07T15:09:06.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5159999999999994e-06 2020-08-07T15:09:06.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:06.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302434+0000", 2020-08-07T15:09:06.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99151562699999995 2020-08-07T15:09:06.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:06.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.293950+0000", 2020-08-07T15:09:06.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020138099999999999 2020-08-07T15:09:06.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:06.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294151+0000", 2020-08-07T15:09:06.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024761 2020-08-07T15:09:06.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:09:06.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294399+0000", 2020-08-07T15:09:06.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0027714369999999999 2020-08-07T15:09:06.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:06.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297170+0000", 2020-08-07T15:09:06.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.234e-06 2020-08-07T15:09:06.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:06.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297172+0000", 2020-08-07T15:09:06.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5384999999999998e-05 2020-08-07T15:09:06.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:06.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297237+0000", 2020-08-07T15:09:06.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0073576980000000002 2020-08-07T15:09:06.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:06.033 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304595+0000", 2020-08-07T15:09:06.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1006000000000001e-05 2020-08-07T15:09:06.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.037 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:06.037 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304626+0000", 2020-08-07T15:09:06.038 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6000000000000003e-07 2020-08-07T15:09:06.038 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.038 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:06.038 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304626+0000", 2020-08-07T15:09:06.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8724e-05 2020-08-07T15:09:06.039 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.039 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.039 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:06.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304705+0000", 2020-08-07T15:09:06.040 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0024736009999999 2020-08-07T15:09:06.040 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:06.040 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:06.040 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:06.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.041 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42506 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:09:06.041 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:09:06.041 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.0750134359999999, 2020-08-07T15:09:06.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002465226, 2020-08-07T15:09:06.073 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:06.073 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:06.073 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:06.078 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:06.078 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:06.079 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42506 2020-08-07T15:09:06.079 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.079 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:06.080 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.080 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:06.080 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:09:06.080 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:06.080 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.081 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.081 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:06.081 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:09:06.081 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8780999999999999e-05 2020-08-07T15:09:06.081 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.082 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:06.082 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302472+0000", 2020-08-07T15:09:06.082 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3740000000000003e-06 2020-08-07T15:09:06.082 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.083 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.083 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:06.083 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302478+0000", 2020-08-07T15:09:06.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1155000000000001e-05 2020-08-07T15:09:06.084 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.084 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:06.084 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302489+0000", 2020-08-07T15:09:06.084 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7559999999999996e-06 2020-08-07T15:09:06.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.096 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:06.097 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302495+0000", 2020-08-07T15:09:06.097 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4128000000000003e-05 2020-08-07T15:09:06.097 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.098 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.098 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:06.098 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302529+0000", 2020-08-07T15:09:06.098 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.516e-06 2020-08-07T15:09:06.098 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.099 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.099 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:06.099 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302535+0000", 2020-08-07T15:09:06.099 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99240899299999996 2020-08-07T15:09:06.099 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.100 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.100 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:06.100 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294944+0000", 2020-08-07T15:09:06.100 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000115806 2020-08-07T15:09:06.100 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.101 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.101 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:06.101 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295060+0000", 2020-08-07T15:09:06.101 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000205002 2020-08-07T15:09:06.118 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.118 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.118 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:09:06.118 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295265+0000", 2020-08-07T15:09:06.119 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043414400000000002 2020-08-07T15:09:06.125 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.125 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.125 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:06.125 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299607+0000", 2020-08-07T15:09:06.125 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2229999999999998e-06 2020-08-07T15:09:06.126 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.126 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.126 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:06.128 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299612+0000", 2020-08-07T15:09:06.129 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5797000000000002e-05 2020-08-07T15:09:06.129 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.129 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.129 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:06.130 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299658+0000", 2020-08-07T15:09:06.130 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005095044 2020-08-07T15:09:06.130 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.130 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.130 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:06.131 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304753+0000", 2020-08-07T15:09:06.131 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3121999999999996e-05 2020-08-07T15:09:06.131 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.131 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.131 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:06.132 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304816+0000", 2020-08-07T15:09:06.132 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5700000000000001e-07 2020-08-07T15:09:06.132 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.132 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.132 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:06.132 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304817+0000", 2020-08-07T15:09:06.133 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1231999999999998e-05 2020-08-07T15:09:06.133 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.133 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.133 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:06.133 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304888+0000", 2020-08-07T15:09:06.134 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002465226 2020-08-07T15:09:06.134 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:06.134 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:06.134 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:06.134 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.135 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.135 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42507 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:09:06.135 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:09:06.135 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.0749026659999998, 2020-08-07T15:09:06.135 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002567097, 2020-08-07T15:09:06.136 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:06.136 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:06.136 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:06.136 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:06.136 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:06.137 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42507 2020-08-07T15:09:06.137 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.137 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:06.137 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.137 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:06.137 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:09:06.138 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:06.138 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.138 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.139 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:06.140 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:09:06.140 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7679000000000002e-05 2020-08-07T15:09:06.140 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.140 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.140 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:06.141 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302581+0000", 2020-08-07T15:09:06.141 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2299999999999999e-06 2020-08-07T15:09:06.141 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.141 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.141 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:06.141 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302586+0000", 2020-08-07T15:09:06.142 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0886999999999999e-05 2020-08-07T15:09:06.142 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.142 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.142 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:06.142 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302597+0000", 2020-08-07T15:09:06.143 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4129999999999997e-06 2020-08-07T15:09:06.143 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.143 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.143 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:06.143 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302603+0000", 2020-08-07T15:09:06.144 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3963000000000001e-05 2020-08-07T15:09:06.144 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.144 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.144 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:06.146 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302627+0000", 2020-08-07T15:09:06.146 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.33e-06 2020-08-07T15:09:06.146 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.146 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.146 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:06.147 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302631+0000", 2020-08-07T15:09:06.147 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99319860900000001 2020-08-07T15:09:06.147 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.147 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.147 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:06.148 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295830+0000", 2020-08-07T15:09:06.148 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000116012 2020-08-07T15:09:06.148 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.148 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.148 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:06.149 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295946+0000", 2020-08-07T15:09:06.150 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000230242 2020-08-07T15:09:06.150 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.150 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.150 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:09:06.151 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296176+0000", 2020-08-07T15:09:06.151 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003494455 2020-08-07T15:09:06.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.151 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:06.152 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299670+0000", 2020-08-07T15:09:06.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0299999999999998e-07 2020-08-07T15:09:06.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:06.153 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299671+0000", 2020-08-07T15:09:06.153 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7298000000000001e-05 2020-08-07T15:09:06.153 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.153 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.153 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:06.154 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299708+0000", 2020-08-07T15:09:06.154 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0052628839999999998 2020-08-07T15:09:06.154 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.154 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.154 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:06.155 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304971+0000", 2020-08-07T15:09:06.155 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3754e-05 2020-08-07T15:09:06.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.156 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:06.156 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305035+0000", 2020-08-07T15:09:06.156 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9500000000000001e-07 2020-08-07T15:09:06.157 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.157 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.157 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:06.157 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305036+0000", 2020-08-07T15:09:06.160 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4843000000000002e-05 2020-08-07T15:09:06.161 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.161 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.161 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:06.161 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305101+0000", 2020-08-07T15:09:06.161 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002567097 2020-08-07T15:09:06.162 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:06.173 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:06.174 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:06.174 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.175 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.175 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42508 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:09:06.175 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:09:06.176 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.0643776469999997, 2020-08-07T15:09:06.176 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99221695600000004, 2020-08-07T15:09:06.176 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:06.176 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:06.177 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:06.177 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:06.177 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:06.177 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42508 2020-08-07T15:09:06.177 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.178 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:06.178 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.178 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:06.178 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:09:06.179 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:06.179 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.179 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.179 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:06.179 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:09:06.180 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3692e-05 2020-08-07T15:09:06.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:06.182 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313112+0000", 2020-08-07T15:09:06.183 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7399999999999998e-06 2020-08-07T15:09:06.183 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.183 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.183 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:06.183 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313119+0000", 2020-08-07T15:09:06.184 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5153999999999999e-05 2020-08-07T15:09:06.184 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.184 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.184 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:06.184 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313134+0000", 2020-08-07T15:09:06.185 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3039999999999999e-06 2020-08-07T15:09:06.185 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.185 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.185 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:06.186 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313141+0000", 2020-08-07T15:09:06.186 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6516000000000001e-05 2020-08-07T15:09:06.187 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.187 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.187 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:06.187 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313178+0000", 2020-08-07T15:09:06.187 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0949999999999998e-06 2020-08-07T15:09:06.188 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.188 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.188 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:06.188 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313185+0000", 2020-08-07T15:09:06.189 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.98342834999999995 2020-08-07T15:09:06.189 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.189 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.189 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:06.190 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296613+0000", 2020-08-07T15:09:06.193 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2230999999999993e-05 2020-08-07T15:09:06.193 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.193 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.193 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:06.194 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296686+0000", 2020-08-07T15:09:06.194 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013121300000000001 2020-08-07T15:09:06.194 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.194 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.194 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:09:06.195 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296817+0000", 2020-08-07T15:09:06.195 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044228050000000001 2020-08-07T15:09:06.195 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.195 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.196 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:06.196 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301240+0000", 2020-08-07T15:09:06.196 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0500000000000002e-07 2020-08-07T15:09:06.196 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.196 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.197 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:06.197 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301241+0000", 2020-08-07T15:09:06.197 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2434000000000006e-05 2020-08-07T15:09:06.197 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.198 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.198 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:06.198 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301303+0000", 2020-08-07T15:09:06.198 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038622919999999998 2020-08-07T15:09:06.198 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.199 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.199 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:06.199 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305165+0000", 2020-08-07T15:09:06.199 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8012000000000001e-05 2020-08-07T15:09:06.199 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.200 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.200 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:06.200 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305203+0000", 2020-08-07T15:09:06.200 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5299999999999999e-07 2020-08-07T15:09:06.204 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.204 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.204 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:06.204 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305204+0000", 2020-08-07T15:09:06.205 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1760000000000004e-05 2020-08-07T15:09:06.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:06.205 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:06.205 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:06.205 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305275+0000", 2020-08-07T15:09:06.206 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99221695600000004 2020-08-07T15:09:06.206 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:06.206 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:06.206 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:06.207 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:06.207 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:06.207 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:09:06.207 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:05.378+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:06.209 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:09:06.217 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:06.208+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:06.282 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:2020-08-07T15:09:06.275898+0000 min lat: 0.00108831 max lat: 2.05564 avg lat: 0.0133134 2020-08-07T15:09:06.288 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-08-07T15:09:06.289 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 40 16 48053 48037 9.38132 14.3438 0.0015216 0.0133134 2020-08-07T15:09:06.322 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:06.320+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:06.397 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:09:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:09:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:48262 4.2 e118/81 4:40fa5f17:::benchmark_data_smithi074_44586_object6032:head v 118'2878, mlcod=118'2871)", 2020-08-07T15:09:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:06.389249+0000", 2020-08-07T15:09:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.004810361, 2020-08-07T15:09:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048327980000000001, 2020-08-07T15:09:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:06.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.389249+0000", 2020-08-07T15:09:06.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:06.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:06.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.389249+0000", 2020-08-07T15:09:06.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9407999999999999e-05 2020-08-07T15:09:06.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:06.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.389298+0000", 2020-08-07T15:09:06.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1680000000000001e-06 2020-08-07T15:09:06.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:06.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.389304+0000", 2020-08-07T15:09:06.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002352037 2020-08-07T15:09:06.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:06.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.391656+0000", 2020-08-07T15:09:06.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.631e-06 2020-08-07T15:09:06.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:06.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.391660+0000", 2020-08-07T15:09:06.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1469e-05 2020-08-07T15:09:06.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:06.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.391691+0000", 2020-08-07T15:09:06.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9335999999999999e-05 2020-08-07T15:09:06.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:06.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.391711+0000", 2020-08-07T15:09:06.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002462049 2020-08-07T15:09:06.503 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:06.503 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:06.503 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:06.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.504 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:48263 4.2 e118/81 4:40fa5f17:::benchmark_data_smithi074_44586_object6032:head v 118'2879, mlcod=118'2871)", 2020-08-07T15:09:06.504 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:06.391384+0000", 2020-08-07T15:09:06.504 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0026744389999999998, 2020-08-07T15:09:06.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002742965, 2020-08-07T15:09:06.520 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:06.520 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:06.520 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:06.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:06.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.391384+0000", 2020-08-07T15:09:06.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:06.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:06.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.391384+0000", 2020-08-07T15:09:06.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9123000000000003e-05 2020-08-07T15:09:06.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:06.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.391444+0000", 2020-08-07T15:09:06.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0750000000000003e-06 2020-08-07T15:09:06.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:06.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.391450+0000", 2020-08-07T15:09:06.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022555200000000001 2020-08-07T15:09:06.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:06.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.391675+0000", 2020-08-07T15:09:06.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4309999999999998e-06 2020-08-07T15:09:06.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:06.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.391679+0000", 2020-08-07T15:09:06.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027590399999999999 2020-08-07T15:09:06.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:06.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.391955+0000", 2020-08-07T15:09:06.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.999e-05 2020-08-07T15:09:06.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:06.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.391975+0000", 2020-08-07T15:09:06.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00059007500000000002 2020-08-07T15:09:06.529 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:06.529 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:06.529 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:06.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.530 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:48268 4.5 e118/81 4:ab999553:::benchmark_data_smithi074_44586_object6033:head v 118'3052, mlcod=118'3047)", 2020-08-07T15:09:06.530 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:06.391585+0000", 2020-08-07T15:09:06.530 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0024743489999999998, 2020-08-07T15:09:06.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002585305, 2020-08-07T15:09:06.531 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:06.531 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:06.531 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:06.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:06.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.391585+0000", 2020-08-07T15:09:06.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:06.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:06.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.391585+0000", 2020-08-07T15:09:06.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8031000000000001e-05 2020-08-07T15:09:06.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:06.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.391633+0000", 2020-08-07T15:09:06.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8799999999999996e-06 2020-08-07T15:09:06.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:06.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.391638+0000", 2020-08-07T15:09:06.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9502000000000001e-05 2020-08-07T15:09:06.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:06.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.391688+0000", 2020-08-07T15:09:06.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.224e-06 2020-08-07T15:09:06.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:06.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.391691+0000", 2020-08-07T15:09:06.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00045171399999999998 2020-08-07T15:09:06.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:06.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.392143+0000", 2020-08-07T15:09:06.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9879000000000001e-05 2020-08-07T15:09:06.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:06.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.392163+0000", 2020-08-07T15:09:06.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00057823 2020-08-07T15:09:06.541 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:06.541 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:06.541 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:06.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.542 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:48269 4.5 e118/81 4:ab999553:::benchmark_data_smithi074_44586_object6033:head v 118'3053, mlcod=118'3047)", 2020-08-07T15:09:06.542 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:06.391953+0000", 2020-08-07T15:09:06.542 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0021062059999999998, 2020-08-07T15:09:06.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002257284, 2020-08-07T15:09:06.542 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:06.542 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:06.543 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:06.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:06.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.391953+0000", 2020-08-07T15:09:06.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:06.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:06.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.391953+0000", 2020-08-07T15:09:06.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8150999999999999e-05 2020-08-07T15:09:06.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:06.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.392011+0000", 2020-08-07T15:09:06.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3659999999999997e-06 2020-08-07T15:09:06.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:06.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.392017+0000", 2020-08-07T15:09:06.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4190999999999999e-05 2020-08-07T15:09:06.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:06.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.392041+0000", 2020-08-07T15:09:06.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8250000000000001e-06 2020-08-07T15:09:06.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:06.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.392047+0000", 2020-08-07T15:09:06.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00036830800000000003 2020-08-07T15:09:06.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:06.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.392416+0000", 2020-08-07T15:09:06.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2328999999999999e-05 2020-08-07T15:09:06.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:06.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.392438+0000", 2020-08-07T15:09:06.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00048516999999999999 2020-08-07T15:09:06.552 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:06.552 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:06.552 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:06.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.553 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:48270 4.5 e118/81 4:ab999553:::benchmark_data_smithi074_44586_object6033:head v 118'3054, mlcod=118'3047)", 2020-08-07T15:09:06.553 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:06.392244+0000", 2020-08-07T15:09:06.553 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.001815154, 2020-08-07T15:09:06.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00200096, 2020-08-07T15:09:06.554 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:06.554 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:06.554 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:06.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:06.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.392244+0000", 2020-08-07T15:09:06.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:06.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:06.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.392244+0000", 2020-08-07T15:09:06.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2573e-05 2020-08-07T15:09:06.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:06.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.392296+0000", 2020-08-07T15:09:06.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1260000000000003e-06 2020-08-07T15:09:06.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:06.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.392302+0000", 2020-08-07T15:09:06.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5222e-05 2020-08-07T15:09:06.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:06.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.392328+0000", 2020-08-07T15:09:06.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6699999999999997e-06 2020-08-07T15:09:06.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:06.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.392334+0000", 2020-08-07T15:09:06.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00035466499999999999 2020-08-07T15:09:06.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:06.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.392689+0000", 2020-08-07T15:09:06.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2340999999999999e-05 2020-08-07T15:09:06.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:06.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.392701+0000", 2020-08-07T15:09:06.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00045759700000000002 2020-08-07T15:09:06.564 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:06.564 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:06.564 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:06.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.565 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:48271 4.5 e118/81 4:ab999553:::benchmark_data_smithi074_44586_object6033:head v 118'3055, mlcod=118'3047)", 2020-08-07T15:09:06.565 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:06.392807+0000", 2020-08-07T15:09:06.573 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0012524000000000001, 2020-08-07T15:09:06.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001473198, 2020-08-07T15:09:06.573 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:06.574 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:06.574 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:06.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:06.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.392807+0000", 2020-08-07T15:09:06.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:06.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:06.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.392807+0000", 2020-08-07T15:09:06.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0056999999999999e-05 2020-08-07T15:09:06.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:06.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.392857+0000", 2020-08-07T15:09:06.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5910000000000001e-06 2020-08-07T15:09:06.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:06.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.392863+0000", 2020-08-07T15:09:06.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5411000000000001e-05 2020-08-07T15:09:06.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:06.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.392889+0000", 2020-08-07T15:09:06.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3669999999999996e-06 2020-08-07T15:09:06.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:06.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.392894+0000", 2020-08-07T15:09:06.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025395099999999998 2020-08-07T15:09:06.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:06.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.393148+0000", 2020-08-07T15:09:06.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1509e-05 2020-08-07T15:09:06.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.393169+0000", 2020-08-07T15:09:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00036288599999999998 2020-08-07T15:09:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:06.620 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:06.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.620 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:48272 4.5 e118/81)", 2020-08-07T15:09:06.620 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:06.393554+0000", 2020-08-07T15:09:06.621 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00050527000000000005, 2020-08-07T15:09:06.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075984700000000002, 2020-08-07T15:09:06.621 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:06.621 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:09:06.621 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:06.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:06.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.393554+0000", 2020-08-07T15:09:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.393554+0000", 2020-08-07T15:09:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0688e-05 2020-08-07T15:09:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.393604+0000", 2020-08-07T15:09:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.905e-06 2020-08-07T15:09:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.393611+0000", 2020-08-07T15:09:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4794e-05 2020-08-07T15:09:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.393636+0000", 2020-08-07T15:09:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0540000000000002e-06 2020-08-07T15:09:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.393641+0000", 2020-08-07T15:09:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7441000000000005e-05 2020-08-07T15:09:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:06.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.649 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:48260 4.2 e118/81 4:40fa5f17:::benchmark_data_smithi074_44586_object6032:head v 118'2876, mlcod=118'2871)", 2020-08-07T15:09:06.649 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:06.388500+0000", 2020-08-07T15:09:06.649 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0055584550000000003, 2020-08-07T15:09:06.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058462480000000001, 2020-08-07T15:09:06.649 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:06.650 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:06.650 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:06.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:06.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.388500+0000", 2020-08-07T15:09:06.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:06.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:06.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.388500+0000", 2020-08-07T15:09:06.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7123999999999997e-05 2020-08-07T15:09:06.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:06.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.388558+0000", 2020-08-07T15:09:06.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2310000000000001e-06 2020-08-07T15:09:06.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:06.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.388564+0000", 2020-08-07T15:09:06.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014206599999999999 2020-08-07T15:09:06.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:06.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.388706+0000", 2020-08-07T15:09:06.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6040000000000001e-06 2020-08-07T15:09:06.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:06.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.388709+0000", 2020-08-07T15:09:06.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022038200000000001 2020-08-07T15:09:06.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:06.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.388930+0000", 2020-08-07T15:09:06.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1270000000000001e-05 2020-08-07T15:09:06.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:06.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.388951+0000", 2020-08-07T15:09:06.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00045067700000000002 2020-08-07T15:09:06.671 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:06.671 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:06.671 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:06.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.672 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:48265 4.5 e118/81 4:ab999553:::benchmark_data_smithi074_44586_object6033:head v 118'3049, mlcod=118'3047)", 2020-08-07T15:09:06.672 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:06.388774+0000", 2020-08-07T15:09:06.672 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.005285344, 2020-08-07T15:09:06.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056189289999999999, 2020-08-07T15:09:06.672 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:06.672 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:06.673 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:06.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:06.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.388774+0000", 2020-08-07T15:09:06.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:06.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:06.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.388774+0000", 2020-08-07T15:09:06.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8019999999999998e-05 2020-08-07T15:09:06.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:06.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.388822+0000", 2020-08-07T15:09:06.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9769999999999997e-06 2020-08-07T15:09:06.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:06.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.388828+0000", 2020-08-07T15:09:06.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5999e-05 2020-08-07T15:09:06.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:06.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.388854+0000", 2020-08-07T15:09:06.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.843e-06 2020-08-07T15:09:06.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:06.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.388858+0000", 2020-08-07T15:09:06.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6869000000000002e-05 2020-08-07T15:09:06.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:06.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.388895+0000", 2020-08-07T15:09:06.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0965999999999999e-05 2020-08-07T15:09:06.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:06.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.388916+0000", 2020-08-07T15:09:06.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000142674 2020-08-07T15:09:06.682 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:06.682 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:06.683 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:06.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.683 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:48261 4.2 e118/81 4:40fa5f17:::benchmark_data_smithi074_44586_object6032:head v 118'2877, mlcod=118'2871)", 2020-08-07T15:09:06.683 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:06.388604+0000", 2020-08-07T15:09:06.684 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0054552610000000003, 2020-08-07T15:09:06.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006173062, 2020-08-07T15:09:06.684 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:06.684 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:06.684 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:06.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:06.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.388604+0000", 2020-08-07T15:09:06.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:06.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:06.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.388604+0000", 2020-08-07T15:09:06.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8721999999999999e-05 2020-08-07T15:09:06.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:06.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.388652+0000", 2020-08-07T15:09:06.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1760000000000001e-06 2020-08-07T15:09:06.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:06.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.388658+0000", 2020-08-07T15:09:06.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067873300000000005 2020-08-07T15:09:06.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:06.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.389336+0000", 2020-08-07T15:09:06.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4650000000000002e-06 2020-08-07T15:09:06.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:06.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.389342+0000", 2020-08-07T15:09:06.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019820850000000002 2020-08-07T15:09:06.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:06.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.391324+0000", 2020-08-07T15:09:06.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1906999999999999e-05 2020-08-07T15:09:06.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:06.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.391346+0000", 2020-08-07T15:09:06.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027420880000000002 2020-08-07T15:09:06.694 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:06.695 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:06.695 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:06.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.695 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:48266 4.5 e118/81 4:ab999553:::benchmark_data_smithi074_44586_object6033:head v 118'3050, mlcod=118'3047)", 2020-08-07T15:09:06.696 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:06.389695+0000", 2020-08-07T15:09:06.696 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0043637349999999997, 2020-08-07T15:09:06.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051624640000000003, 2020-08-07T15:09:06.696 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:06.696 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:06.696 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:06.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:06.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.389695+0000", 2020-08-07T15:09:06.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:06.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:06.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.389695+0000", 2020-08-07T15:09:06.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2398999999999998e-05 2020-08-07T15:09:06.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:06.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.389748+0000", 2020-08-07T15:09:06.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8580000000000002e-06 2020-08-07T15:09:06.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:06.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.389754+0000", 2020-08-07T15:09:06.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5695e-05 2020-08-07T15:09:06.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:06.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.389780+0000", 2020-08-07T15:09:06.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3789999999999997e-06 2020-08-07T15:09:06.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:06.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.389786+0000", 2020-08-07T15:09:06.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3169000000000004e-05 2020-08-07T15:09:06.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:06.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.389859+0000", 2020-08-07T15:09:06.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1608999999999999e-05 2020-08-07T15:09:06.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:06.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.389880+0000", 2020-08-07T15:09:06.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018510900000000001 2020-08-07T15:09:06.704 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:06.704 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:06.705 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:06.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.705 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:48267 4.5 e118/81 4:ab999553:::benchmark_data_smithi074_44586_object6033:head v 118'3051, mlcod=118'3047)", 2020-08-07T15:09:06.705 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:06.390369+0000", 2020-08-07T15:09:06.705 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0036897319999999998, 2020-08-07T15:09:06.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045652959999999999, 2020-08-07T15:09:06.706 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:06.706 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:06.706 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:06.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:06.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.390369+0000", 2020-08-07T15:09:06.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:06.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:06.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.390369+0000", 2020-08-07T15:09:06.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7759999999999997e-05 2020-08-07T15:09:06.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:06.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.390417+0000", 2020-08-07T15:09:06.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3300000000000004e-06 2020-08-07T15:09:06.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.390423+0000", 2020-08-07T15:09:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0045000000000001e-05 2020-08-07T15:09:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.390453+0000", 2020-08-07T15:09:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7590000000000003e-06 2020-08-07T15:09:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.390459+0000", 2020-08-07T15:09:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00087792600000000001 2020-08-07T15:09:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:06.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.391337+0000", 2020-08-07T15:09:06.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2147999999999999e-05 2020-08-07T15:09:06.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:06.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:06.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:06.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:06.391359+0000", 2020-08-07T15:09:06.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00098996799999999992 2020-08-07T15:09:06.714 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:06.714 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:06.714 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:06.714 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:06.715 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:09:06.715 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 12 2020-08-07T15:09:06.715 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:09:06.716 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:09:06.717 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:06.715+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:06.718 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:09:06.824 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:06.817+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:06.898 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:09:06.898 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:09:06.899 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:09:06.899 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:09:06.899 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:09:06.910 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:09:06.940 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:06.939+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:07.048 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:07.047+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:07.123 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:09:07.124 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:09:07.136 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2020-08-07T15:09:06.644286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":353501184,"num_objects":5396,"num_object_clones":0,"num_object_copies":16188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43152,"num_write_kb":345216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":635,"num_bytes_recovered":41525248,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":135,"acting":134,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9467936,"kb_used_data":1079264,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":745506784,"statfs":{"total":773094113280,"available":763398946816,"internally_reserved":8589934592,"allocated":1105166336,"data_stored":1100067761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7798784,"num_objects":119,"num_object_clones":0,"num_object_copies":357,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":952,"num_write_kb":7616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.480312"},"pg_stats":[{"pgid":"1.14","version":"16'2","reported_seq":"43","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.515159+0000","last_change":"2020-08-07T15:08:42.533999+0000","last_active":"2020-08-07T15:09:00.515159+0000","last_peered":"2020-08-07T15:09:00.515159+0000","last_clean":"2020-08-07T15:09:00.515159+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:09:00.515159+0000","last_undegraded":"2020-08-07T15:09:00.515159+0000","last_fullsized":"2020-08-07T15:09:00.515159+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:42.533921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:42.533921+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"79","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.292021+0000","last_change":"2020-08-07T15:08:58.212828+0000","last_active":"2020-08-07T15:09:00.292021+0000","last_peered":"2020-08-07T15:09:00.292021+0000","last_clean":"2020-08-07T15:09:00.292021+0000","last_became_active":"2020-08-07T15:08:58.211785+0000","last_became_peered":"2020-08-07T15:08:58.211785+0000","last_unstale":"2020-08-07T15:09:00.292021+0000","last_undegraded":"2020-08-07T15:09:00.292021+0000","last_fullsized":"2020-08-07T15:09:00.292021+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.141010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.141010+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"43","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.289984+0000","last_change":"2020-08-07T15:08:55.078015+0000","last_active":"2020-08-07T15:09:00.289984+0000","last_peered":"2020-08-07T15:09:00.289984+0000","last_clean":"2020-08-07T15:09:00.289984+0000","last_became_active":"2020-08-07T15:08:41.685106+0000","last_became_peered":"2020-08-07T15:08:41.685106+0000","last_unstale":"2020-08-07T15:09:00.289984+0000","last_undegraded":"2020-08-07T15:09:00.289984+0000","last_fullsized":"2020-08-07T15:09:00.289984+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:55.077924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:55.077924+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"122","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.290319+0000","last_change":"2020-08-07T15:08:43.544399+0000","last_active":"2020-08-07T15:09:00.290319+0000","last_peered":"2020-08-07T15:09:00.290319+0000","last_clean":"2020-08-07T15:09:00.290319+0000","last_became_active":"2020-08-07T15:08:41.675776+0000","last_became_peered":"2020-08-07T15:08:41.675776+0000","last_unstale":"2020-08-07T15:09:00.290319+0000","last_undegraded":"2020-08-07T15:09:00.290319+0000","last_fullsized":"2020-08-07T15:09:00.290319+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.544324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.544324+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"150","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.289931+0000","last_change":"2020-08-07T15:08:51.599032+0000","last_active":"2020-08-07T15:09:00.289931+0000","last_peered":"2020-08-07T15:09:00.289931+0000","last_clean":"2020-08-07T15:09:00.289931+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:09:00.289931+0000","last_undegraded":"2020-08-07T15:09:00.289931+0000","last_fullsized":"2020-08-07T15:09:00.289931+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:51.598947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.598947+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"109","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.289452+0000","last_change":"2020-08-07T15:08:11.840861+0000","last_active":"2020-08-07T15:09:01.289452+0000","last_peered":"2020-08-07T15:09:01.289452+0000","last_clean":"2020-08-07T15:09:01.289452+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:09:01.289452+0000","last_undegraded":"2020-08-07T15:09:01.289452+0000","last_fullsized":"2020-08-07T15:09:01.289452+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:11.840807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:11.840807+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"99","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.286421+0000","last_change":"2020-08-07T15:08:39.558760+0000","last_active":"2020-08-07T15:09:00.286421+0000","last_peered":"2020-08-07T15:09:00.286421+0000","last_clean":"2020-08-07T15:09:00.286421+0000","last_became_active":"2020-08-07T15:08:36.615358+0000","last_became_peered":"2020-08-07T15:08:36.615358+0000","last_unstale":"2020-08-07T15:09:00.286421+0000","last_undegraded":"2020-08-07T15:09:00.286421+0000","last_fullsized":"2020-08-07T15:09:00.286421+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:39.558677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:39.558677+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"114'2584","reported_seq":"2626","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.680980+0000","last_change":"2020-08-07T15:08:46.715869+0000","last_active":"2020-08-07T15:09:01.680980+0000","last_peered":"2020-08-07T15:09:01.680980+0000","last_clean":"2020-08-07T15:09:01.680980+0000","last_became_active":"2020-08-07T15:08:46.708112+0000","last_became_peered":"2020-08-07T15:08:46.708112+0000","last_unstale":"2020-08-07T15:09:01.680980+0000","last_undegraded":"2020-08-07T15:09:01.680980+0000","last_fullsized":"2020-08-07T15:09:01.680980+0000","mapping_epoch":98,"log_start":"114'2581","ondisk_log_start":"114'2581","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21168128,"num_objects":323,"num_object_clones":0,"num_object_copies":969,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2584,"num_write_kb":20672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"111","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.292329+0000","last_change":"2020-08-07T15:08:40.057705+0000","last_active":"2020-08-07T15:09:00.292329+0000","last_peered":"2020-08-07T15:09:00.292329+0000","last_clean":"2020-08-07T15:09:00.292329+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:09:00.292329+0000","last_undegraded":"2020-08-07T15:09:00.292329+0000","last_fullsized":"2020-08-07T15:09:00.292329+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:40.057622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:40.057622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"68","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.515303+0000","last_change":"2020-08-07T15:08:53.756554+0000","last_active":"2020-08-07T15:09:00.515303+0000","last_peered":"2020-08-07T15:09:00.515303+0000","last_clean":"2020-08-07T15:09:00.515303+0000","last_became_active":"2020-08-07T15:08:46.710404+0000","last_became_peered":"2020-08-07T15:08:46.710404+0000","last_unstale":"2020-08-07T15:09:00.515303+0000","last_undegraded":"2020-08-07T15:09:00.515303+0000","last_fullsized":"2020-08-07T15:09:00.515303+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:53.756343+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:53.756343+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"114'2824","reported_seq":"3509","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.322637+0000","last_change":"2020-08-07T15:09:01.304273+0000","last_active":"2020-08-07T15:09:01.322637+0000","last_peered":"2020-08-07T15:09:01.322637+0000","last_clean":"2020-08-07T15:09:01.322637+0000","last_became_active":"2020-08-07T15:09:01.293427+0000","last_became_peered":"2020-08-07T15:09:01.293427+0000","last_unstale":"2020-08-07T15:09:01.322637+0000","last_undegraded":"2020-08-07T15:09:01.322637+0000","last_fullsized":"2020-08-07T15:09:01.322637+0000","mapping_epoch":113,"log_start":"114'2815","ondisk_log_start":"114'2815","created":81,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23134208,"num_objects":353,"num_object_clones":0,"num_object_copies":1059,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2824,"num_write_kb":22592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":20971520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"108","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.296117+0000","last_change":"2020-08-07T15:08:27.906894+0000","last_active":"2020-08-07T15:09:00.296117+0000","last_peered":"2020-08-07T15:09:00.296117+0000","last_clean":"2020-08-07T15:09:00.296117+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:09:00.296117+0000","last_undegraded":"2020-08-07T15:09:00.296117+0000","last_fullsized":"2020-08-07T15:09:00.296117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:27.906822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:27.906822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"114","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.289746+0000","last_change":"2020-08-07T15:08:43.899342+0000","last_active":"2020-08-07T15:09:01.289746+0000","last_peered":"2020-08-07T15:09:01.289746+0000","last_clean":"2020-08-07T15:09:01.289746+0000","last_became_active":"2020-08-07T15:08:36.622209+0000","last_became_peered":"2020-08-07T15:08:36.622209+0000","last_unstale":"2020-08-07T15:09:01.289746+0000","last_undegraded":"2020-08-07T15:09:01.289746+0000","last_fullsized":"2020-08-07T15:09:01.289746+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.899280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.899280+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"114'2616","reported_seq":"2657","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.421157+0000","last_change":"2020-08-07T15:08:24.549715+0000","last_active":"2020-08-07T15:09:01.421157+0000","last_peered":"2020-08-07T15:09:01.421157+0000","last_clean":"2020-08-07T15:09:01.421157+0000","last_became_active":"2020-08-07T15:08:24.549245+0000","last_became_peered":"2020-08-07T15:08:24.549245+0000","last_unstale":"2020-08-07T15:09:01.421157+0000","last_undegraded":"2020-08-07T15:09:01.421157+0000","last_fullsized":"2020-08-07T15:09:01.421157+0000","mapping_epoch":81,"log_start":"114'2610","ondisk_log_start":"114'2610","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21430272,"num_objects":327,"num_object_clones":0,"num_object_copies":981,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2616,"num_write_kb":20928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"108","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.295286+0000","last_change":"2020-08-07T15:08:13.867355+0000","last_active":"2020-08-07T15:09:00.295286+0000","last_peered":"2020-08-07T15:09:00.295286+0000","last_clean":"2020-08-07T15:09:00.295286+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:09:00.295286+0000","last_undegraded":"2020-08-07T15:09:00.295286+0000","last_fullsized":"2020-08-07T15:09:00.295286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:13.867148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:13.867148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"114","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.289247+0000","last_change":"2020-08-07T15:08:50.686578+0000","last_active":"2020-08-07T15:09:01.289247+0000","last_peered":"2020-08-07T15:09:01.289247+0000","last_clean":"2020-08-07T15:09:01.289247+0000","last_became_active":"2020-08-07T15:08:36.633406+0000","last_became_peered":"2020-08-07T15:08:36.633406+0000","last_unstale":"2020-08-07T15:09:01.289247+0000","last_undegraded":"2020-08-07T15:09:01.289247+0000","last_fullsized":"2020-08-07T15:09:01.289247+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:50.686496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:50.686496+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"114'2512","reported_seq":"2552","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.594921+0000","last_change":"2020-08-07T15:08:24.557398+0000","last_active":"2020-08-07T15:09:01.594921+0000","last_peered":"2020-08-07T15:09:01.594921+0000","last_clean":"2020-08-07T15:09:01.594921+0000","last_became_active":"2020-08-07T15:08:24.557022+0000","last_became_peered":"2020-08-07T15:08:24.557022+0000","last_unstale":"2020-08-07T15:09:01.594921+0000","last_undegraded":"2020-08-07T15:09:01.594921+0000","last_fullsized":"2020-08-07T15:09:01.594921+0000","mapping_epoch":81,"log_start":"114'2509","ondisk_log_start":"114'2509","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20578304,"num_objects":314,"num_object_clones":0,"num_object_copies":942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2512,"num_write_kb":20096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"140","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.295184+0000","last_change":"2020-08-07T15:08:43.036067+0000","last_active":"2020-08-07T15:09:00.295184+0000","last_peered":"2020-08-07T15:09:00.295184+0000","last_clean":"2020-08-07T15:09:00.295184+0000","last_became_active":"2020-08-07T15:08:36.627426+0000","last_became_peered":"2020-08-07T15:08:36.627426+0000","last_unstale":"2020-08-07T15:09:00.295184+0000","last_undegraded":"2020-08-07T15:09:00.295184+0000","last_fullsized":"2020-08-07T15:09:00.295184+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.035987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.035987+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"108","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.290414+0000","last_change":"2020-08-07T15:08:26.317430+0000","last_active":"2020-08-07T15:09:00.290414+0000","last_peered":"2020-08-07T15:09:00.290414+0000","last_clean":"2020-08-07T15:09:00.290414+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:09:00.290414+0000","last_undegraded":"2020-08-07T15:09:00.290414+0000","last_fullsized":"2020-08-07T15:09:00.290414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:26.317351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:26.317351+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"115'2768","reported_seq":"2804","reported_epoch":"115","state":"active+clean","last_fresh":"2020-08-07T15:09:02.714450+0000","last_change":"2020-08-07T15:08:24.552570+0000","last_active":"2020-08-07T15:09:02.714450+0000","last_peered":"2020-08-07T15:09:02.714450+0000","last_clean":"2020-08-07T15:09:02.714450+0000","last_became_active":"2020-08-07T15:08:24.552105+0000","last_became_peered":"2020-08-07T15:08:24.552105+0000","last_unstale":"2020-08-07T15:09:02.714450+0000","last_undegraded":"2020-08-07T15:09:02.714450+0000","last_fullsized":"2020-08-07T15:09:02.714450+0000","mapping_epoch":81,"log_start":"115'2761","ondisk_log_start":"115'2761","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22675456,"num_objects":346,"num_object_clones":0,"num_object_copies":1038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2768,"num_write_kb":22144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"108","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.515189+0000","last_change":"2020-08-07T15:07:36.864560+0000","last_active":"2020-08-07T15:09:00.515189+0000","last_peered":"2020-08-07T15:09:00.515189+0000","last_clean":"2020-08-07T15:09:00.515189+0000","last_became_active":"2020-08-07T15:06:23.223489+0000","last_became_peered":"2020-08-07T15:06:23.223489+0000","last_unstale":"2020-08-07T15:09:00.515189+0000","last_undegraded":"2020-08-07T15:09:00.515189+0000","last_fullsized":"2020-08-07T15:09:00.515189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:07:36.864483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.864483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"156","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.294032+0000","last_change":"2020-08-07T15:08:42.069637+0000","last_active":"2020-08-07T15:09:00.294032+0000","last_peered":"2020-08-07T15:09:00.294032+0000","last_clean":"2020-08-07T15:09:00.294032+0000","last_became_active":"2020-08-07T15:08:36.628365+0000","last_became_peered":"2020-08-07T15:08:36.628365+0000","last_unstale":"2020-08-07T15:09:00.294032+0000","last_undegraded":"2020-08-07T15:09:00.294032+0000","last_fullsized":"2020-08-07T15:09:00.294032+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:42.069585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:42.069585+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"114'2824","reported_seq":"2865","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.426013+0000","last_change":"2020-08-07T15:08:24.550789+0000","last_active":"2020-08-07T15:09:01.426013+0000","last_peered":"2020-08-07T15:09:01.426013+0000","last_clean":"2020-08-07T15:09:01.426013+0000","last_became_active":"2020-08-07T15:08:24.550222+0000","last_became_peered":"2020-08-07T15:08:24.550222+0000","last_unstale":"2020-08-07T15:09:01.426013+0000","last_undegraded":"2020-08-07T15:09:01.426013+0000","last_fullsized":"2020-08-07T15:09:01.426013+0000","mapping_epoch":81,"log_start":"114'2818","ondisk_log_start":"114'2818","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23134208,"num_objects":353,"num_object_clones":0,"num_object_copies":1059,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2824,"num_write_kb":22592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"108","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.290321+0000","last_change":"2020-08-07T15:08:30.339852+0000","last_active":"2020-08-07T15:09:00.290321+0000","last_peered":"2020-08-07T15:09:00.290321+0000","last_clean":"2020-08-07T15:09:00.290321+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:09:00.290321+0000","last_undegraded":"2020-08-07T15:09:00.290321+0000","last_fullsized":"2020-08-07T15:09:00.290321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:08:30.339785+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"106","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.292512+0000","last_change":"2020-08-07T15:08:38.386792+0000","last_active":"2020-08-07T15:09:00.292512+0000","last_peered":"2020-08-07T15:09:00.292512+0000","last_clean":"2020-08-07T15:09:00.292512+0000","last_became_active":"2020-08-07T15:08:36.618164+0000","last_became_peered":"2020-08-07T15:08:36.618164+0000","last_unstale":"2020-08-07T15:09:00.292512+0000","last_undegraded":"2020-08-07T15:09:00.292512+0000","last_fullsized":"2020-08-07T15:09:00.292512+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:38.386739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:38.386739+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"114'2768","reported_seq":"2809","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.674515+0000","last_change":"2020-08-07T15:08:24.558242+0000","last_active":"2020-08-07T15:09:01.674515+0000","last_peered":"2020-08-07T15:09:01.674515+0000","last_clean":"2020-08-07T15:09:01.674515+0000","last_became_active":"2020-08-07T15:08:24.557354+0000","last_became_peered":"2020-08-07T15:08:24.557354+0000","last_unstale":"2020-08-07T15:09:01.674515+0000","last_undegraded":"2020-08-07T15:09:01.674515+0000","last_fullsized":"2020-08-07T15:09:01.674515+0000","mapping_epoch":81,"log_start":"114'2765","ondisk_log_start":"114'2765","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22675456,"num_objects":346,"num_object_clones":0,"num_object_copies":1038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2768,"num_write_kb":22144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"105","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.295909+0000","last_change":"2020-08-07T15:08:15.887617+0000","last_active":"2020-08-07T15:09:00.295909+0000","last_peered":"2020-08-07T15:09:00.295909+0000","last_clean":"2020-08-07T15:09:00.295909+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:09:00.295909+0000","last_undegraded":"2020-08-07T15:09:00.295909+0000","last_fullsized":"2020-08-07T15:09:00.295909+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:15.887513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:15.887513+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"145","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.289473+0000","last_change":"2020-08-07T15:08:58.183263+0000","last_active":"2020-08-07T15:09:01.289473+0000","last_peered":"2020-08-07T15:09:01.289473+0000","last_clean":"2020-08-07T15:09:01.289473+0000","last_became_active":"2020-08-07T15:08:58.182722+0000","last_became_peered":"2020-08-07T15:08:58.182722+0000","last_unstale":"2020-08-07T15:09:01.289473+0000","last_undegraded":"2020-08-07T15:09:01.289473+0000","last_fullsized":"2020-08-07T15:09:01.289473+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.927824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.927824+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"114'2464","reported_seq":"2504","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.426624+0000","last_change":"2020-08-07T15:08:24.552962+0000","last_active":"2020-08-07T15:09:01.426624+0000","last_peered":"2020-08-07T15:09:01.426624+0000","last_clean":"2020-08-07T15:09:01.426624+0000","last_became_active":"2020-08-07T15:08:24.552070+0000","last_became_peered":"2020-08-07T15:08:24.552070+0000","last_unstale":"2020-08-07T15:09:01.426624+0000","last_undegraded":"2020-08-07T15:09:01.426624+0000","last_fullsized":"2020-08-07T15:09:01.426624+0000","mapping_epoch":81,"log_start":"114'2456","ondisk_log_start":"114'2456","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20185088,"num_objects":308,"num_object_clones":0,"num_object_copies":924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2464,"num_write_kb":19712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"83","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.290206+0000","last_change":"2020-08-07T15:08:39.409159+0000","last_active":"2020-08-07T15:09:00.290206+0000","last_peered":"2020-08-07T15:09:00.290206+0000","last_clean":"2020-08-07T15:09:00.290206+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:09:00.290206+0000","last_undegraded":"2020-08-07T15:09:00.290206+0000","last_fullsized":"2020-08-07T15:09:00.290206+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:39.409063+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_clean_scrub_stamp":"2020-08-07T15:08:39.409063+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"114'2760","reported_seq":"2801","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.409800+0000","last_change":"2020-08-07T15:08:24.554063+0000","last_active":"2020-08-07T15:09:01.409800+0000","last_peered":"2020-08-07T15:09:01.409800+0000","last_clean":"2020-08-07T15:09:01.409800+0000","last_became_active":"2020-08-07T15:08:24.553668+0000","last_became_peered":"2020-08-07T15:08:24.553668+0000","last_unstale":"2020-08-07T15:09:01.409800+0000","last_undegraded":"2020-08-07T15:09:01.409800+0000","last_fullsized":"2020-08-07T15:09:01.409800+0000","mapping_epoch":81,"log_start":"114'2754","ondisk_log_start":"114'2754","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22609920,"num_objects":345,"num_object_clones":0,"num_object_copies":1035,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2760,"num_write_kb":22080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"80","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.289975+0000","last_change":"2020-08-07T15:07:33.177491+0000","last_active":"2020-08-07T15:09:00.289975+0000","last_peered":"2020-08-07T15:09:00.289975+0000","last_clean":"2020-08-07T15:09:00.289975+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:09:00.289975+0000","last_undegraded":"2020-08-07T15:09:00.289975+0000","last_fullsized":"2020-08-07T15:09:00.289975+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:33.177412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:33.177412+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"114'2488","reported_seq":"2529","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.467392+0000","last_change":"2020-08-07T15:08:46.709789+0000","last_active":"2020-08-07T15:09:01.467392+0000","last_peered":"2020-08-07T15:09:01.467392+0000","last_clean":"2020-08-07T15:09:01.467392+0000","last_became_active":"2020-08-07T15:08:46.708674+0000","last_became_peered":"2020-08-07T15:08:46.708674+0000","last_unstale":"2020-08-07T15:09:01.467392+0000","last_undegraded":"2020-08-07T15:09:01.467392+0000","last_fullsized":"2020-08-07T15:09:01.467392+0000","mapping_epoch":98,"log_start":"114'2484","ondisk_log_start":"114'2484","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20381696,"num_objects":311,"num_object_clones":0,"num_object_copies":933,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2488,"num_write_kb":19904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"144","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.296026+0000","last_change":"2020-08-07T15:07:56.631170+0000","last_active":"2020-08-07T15:09:00.296026+0000","last_peered":"2020-08-07T15:09:00.296026+0000","last_clean":"2020-08-07T15:09:00.296026+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:09:00.296026+0000","last_undegraded":"2020-08-07T15:09:00.296026+0000","last_fullsized":"2020-08-07T15:09:00.296026+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:56.631091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:56.631091+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"114'2792","reported_seq":"2855","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.589955+0000","last_change":"2020-08-07T15:08:52.193226+0000","last_active":"2020-08-07T15:09:01.589955+0000","last_peered":"2020-08-07T15:09:01.589955+0000","last_clean":"2020-08-07T15:09:01.589955+0000","last_became_active":"2020-08-07T15:08:52.166336+0000","last_became_peered":"2020-08-07T15:08:52.166336+0000","last_unstale":"2020-08-07T15:09:01.589955+0000","last_undegraded":"2020-08-07T15:09:01.589955+0000","last_fullsized":"2020-08-07T15:09:01.589955+0000","mapping_epoch":104,"log_start":"114'2789","ondisk_log_start":"114'2789","created":81,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22872064,"num_objects":349,"num_object_clones":0,"num_object_copies":1047,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2792,"num_write_kb":22336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":40960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"102","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.296057+0000","last_change":"2020-08-07T15:08:51.513398+0000","last_active":"2020-08-07T15:09:00.296057+0000","last_peered":"2020-08-07T15:09:00.296057+0000","last_clean":"2020-08-07T15:09:00.296057+0000","last_became_active":"2020-08-07T15:08:41.680448+0000","last_became_peered":"2020-08-07T15:08:41.680448+0000","last_unstale":"2020-08-07T15:09:00.296057+0000","last_undegraded":"2020-08-07T15:09:00.296057+0000","last_fullsized":"2020-08-07T15:09:00.296057+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:51.513319+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.513319+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"114'2640","reported_seq":"2681","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.406450+0000","last_change":"2020-08-07T15:08:24.550496+0000","last_active":"2020-08-07T15:09:01.406450+0000","last_peered":"2020-08-07T15:09:01.406450+0000","last_clean":"2020-08-07T15:09:01.406450+0000","last_became_active":"2020-08-07T15:08:24.549868+0000","last_became_peered":"2020-08-07T15:08:24.549868+0000","last_unstale":"2020-08-07T15:09:01.406450+0000","last_undegraded":"2020-08-07T15:09:01.406450+0000","last_fullsized":"2020-08-07T15:09:01.406450+0000","mapping_epoch":81,"log_start":"114'2625","ondisk_log_start":"114'2625","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21626880,"num_objects":330,"num_object_clones":0,"num_object_copies":990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2640,"num_write_kb":21120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"112","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.286220+0000","last_change":"2020-08-07T15:08:41.679850+0000","last_active":"2020-08-07T15:09:00.286220+0000","last_peered":"2020-08-07T15:09:00.286220+0000","last_clean":"2020-08-07T15:09:00.286220+0000","last_became_active":"2020-08-07T15:08:41.679321+0000","last_became_peered":"2020-08-07T15:08:41.679321+0000","last_unstale":"2020-08-07T15:09:00.286220+0000","last_undegraded":"2020-08-07T15:09:00.286220+0000","last_fullsized":"2020-08-07T15:09:00.286220+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:34.653843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:34.653843+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"112'2776","reported_seq":"3450","reported_epoch":"114","state":"remapped+peering","last_fresh":"2020-08-07T15:09:01.291140+0000","last_change":"2020-08-07T15:09:01.291140+0000","last_active":"2020-08-07T15:09:00.290771+0000","last_peered":"2020-08-07T15:09:00.287039+0000","last_clean":"2020-08-07T15:09:00.287039+0000","last_became_active":"2020-08-07T15:08:56.281990+0000","last_became_peered":"2020-08-07T15:08:56.281990+0000","last_unstale":"2020-08-07T15:09:01.291140+0000","last_undegraded":"2020-08-07T15:09:01.291140+0000","last_fullsized":"2020-08-07T15:09:01.291140+0000","mapping_epoch":114,"log_start":"112'2767","ondisk_log_start":"112'2767","created":81,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22740992,"num_objects":347,"num_object_clones":0,"num_object_copies":1041,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2776,"num_write_kb":22208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":20512768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"144","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.289331+0000","last_change":"2020-08-07T15:08:41.689973+0000","last_active":"2020-08-07T15:09:01.289331+0000","last_peered":"2020-08-07T15:09:01.289331+0000","last_clean":"2020-08-07T15:09:01.289331+0000","last_became_active":"2020-08-07T15:08:41.689435+0000","last_became_peered":"2020-08-07T15:08:41.689435+0000","last_unstale":"2020-08-07T15:09:01.289331+0000","last_undegraded":"2020-08-07T15:09:01.289331+0000","last_fullsized":"2020-08-07T15:09:01.289331+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.927824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.927824+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"115'3016","reported_seq":"3058","reported_epoch":"115","state":"active+clean","last_fresh":"2020-08-07T15:09:02.713039+0000","last_change":"2020-08-07T15:08:24.554472+0000","last_active":"2020-08-07T15:09:02.713039+0000","last_peered":"2020-08-07T15:09:02.713039+0000","last_clean":"2020-08-07T15:09:02.713039+0000","last_became_active":"2020-08-07T15:08:24.554021+0000","last_became_peered":"2020-08-07T15:08:24.554021+0000","last_unstale":"2020-08-07T15:09:02.713039+0000","last_undegraded":"2020-08-07T15:09:02.713039+0000","last_fullsized":"2020-08-07T15:09:02.713039+0000","mapping_epoch":81,"log_start":"115'3008","ondisk_log_start":"115'3008","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24707072,"num_objects":377,"num_object_clones":0,"num_object_copies":1131,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3016,"num_write_kb":24128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"30","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.295927+0000","last_change":"2020-08-07T15:08:46.710556+0000","last_active":"2020-08-07T15:09:00.295927+0000","last_peered":"2020-08-07T15:09:00.295927+0000","last_clean":"2020-08-07T15:09:00.295927+0000","last_became_active":"2020-08-07T15:08:46.710059+0000","last_became_peered":"2020-08-07T15:08:46.710059+0000","last_unstale":"2020-08-07T15:09:00.295927+0000","last_undegraded":"2020-08-07T15:09:00.295927+0000","last_fullsized":"2020-08-07T15:09:00.295927+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:09.841428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:09.841428+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"114'2608","reported_seq":"2649","reported_epoch":"114","state":"active+clean","last_fresh":"2020-08-07T15:09:01.643925+0000","last_change":"2020-08-07T15:08:24.552802+0000","last_active":"2020-08-07T15:09:01.643925+0000","last_peered":"2020-08-07T15:09:01.643925+0000","last_clean":"2020-08-07T15:09:01.643925+0000","last_became_active":"2020-08-07T15:08:24.552127+0000","last_became_peered":"2020-08-07T15:08:24.552127+0000","last_unstale":"2020-08-07T15:09:01.643925+0000","last_undegraded":"2020-08-07T15:09:01.643925+0000","last_fullsized":"2020-08-07T15:09:01.643925+0000","mapping_epoch":81,"log_start":"114'2605","ondisk_log_start":"114'2605","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21364736,"num_objects":326,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2608,"num_write_kb":20864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"115'2712","reported_seq":"2757","reported_epoch":"115","state":"active+clean","last_fresh":"2020-08-07T15:09:02.712781+0000","last_change":"2020-08-07T15:08:46.710643+0000","last_active":"2020-08-07T15:09:02.712781+0000","last_peered":"2020-08-07T15:09:02.712781+0000","last_clean":"2020-08-07T15:09:02.712781+0000","last_became_active":"2020-08-07T15:08:46.710100+0000","last_became_peered":"2020-08-07T15:08:46.710100+0000","last_unstale":"2020-08-07T15:09:02.712781+0000","last_undegraded":"2020-08-07T15:09:02.712781+0000","last_fullsized":"2020-08-07T15:09:02.712781+0000","mapping_epoch":98,"log_start":"115'2704","ondisk_log_start":"115'2704","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22216704,"num_objects":339,"num_object_clones":0,"num_object_copies":1017,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2712,"num_write_kb":21696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"52","reported_epoch":"113","state":"active+clean","last_fresh":"2020-08-07T15:09:00.290299+0000","last_change":"2020-08-07T15:08:41.678395+0000","last_active":"2020-08-07T15:09:00.290299+0000","last_peered":"2020-08-07T15:09:00.290299+0000","last_clean":"2020-08-07T15:09:00.290299+0000","last_became_active":"2020-08-07T15:08:41.677944+0000","last_became_peered":"2020-08-07T15:08:41.677944+0000","last_unstale":"2020-08-07T15:09:00.290299+0000","last_undegraded":"2020-08-07T15:09:00.290299+0000","last_fullsized":"2020-08-07T15:09:00.290299+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:02.901115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:02.901115+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":353501184,"num_objects":5394,"num_object_clones":0,"num_object_copies":16182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5394,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43152,"num_write_kb":345216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":635,"num_bytes_recovered":41525248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1101365248,"data_stored":1101365248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101,"ondisk_log_size":101,"up":48,"acting":47,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":63,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542179,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1180972,"kb_used_data":132388,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93190868,"statfs":{"total":96636764160,"available":95427448832,"internally_reserved":1073741824,"allocated":135565312,"data_stored":134918629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":0.95099999999999996,"15min":0.91200000000000003},"min":{"1min":0.48199999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":3.4159999999999999,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.0940000000000001,"5min":0.98899999999999999,"15min":0.97099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.3559999999999999,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":1.163}]},{"osd":1,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3069999999999999,"15min":1.2989999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":4.4829999999999997,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.044,"15min":0.99199999999999999},"min":{"1min":0.61799999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":3.4620000000000002,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":1.069}]},{"osd":2,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.179,"15min":1.1439999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":4.423,"5min":4.423,"15min":4.423},"last":1.6339999999999999},{"interface":"front","average":{"1min":1.5,"5min":1.194,"15min":1.143},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":3.5099999999999998,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":5.1319999999999997}]},{"osd":3,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.5900000000000001,"15min":1.6439999999999999},"min":{"1min":0.73799999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.5430000000000001,"5min":5.5019999999999998,"15min":5.5019999999999998},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.329,"15min":1.3380000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.4590000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":1.355}]},{"osd":4,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.7849999999999999,"5min":1.377,"15min":1.3089999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":4.835,"5min":4.835,"15min":4.835},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.3300000000000001,"15min":1.3180000000000001},"min":{"1min":0.76900000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.4470000000000001,"5min":2.778,"15min":2.778},"last":1.006}]},{"osd":5,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":2.3940000000000001,"5min":30.484000000000002,"15min":35.165999999999997},"min":{"1min":0.96799999999999997,"5min":0.875,"15min":0.875},"max":{"1min":7.4429999999999996,"5min":831.88099999999997,"15min":831.88099999999997},"last":1.044},{"interface":"front","average":{"1min":2.456,"5min":1.6870000000000001,"15min":1.5589999999999999},"min":{"1min":1,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":7.5350000000000001,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":1.1279999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.542,"15min":1.526},"min":{"1min":0.83599999999999997,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.9900000000000002,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.9219999999999999,"5min":1.5900000000000001,"15min":1.534},"min":{"1min":1.115,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":4.5170000000000003,"5min":4.5170000000000003,"15min":4.5170000000000003},"last":1.478}]}]},{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1180916,"kb_used_data":132332,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93190924,"statfs":{"total":96636764160,"available":95427506176,"internally_reserved":1073741824,"allocated":135507968,"data_stored":134876248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.2490000000000001,"15min":1.3049999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.6439999999999999,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":1.081},{"interface":"front","average":{"1min":1.079,"5min":1.135,"15min":1.145},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":2.9950000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.3740000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.95999999999999996,"15min":0.95799999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.9890000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":19.103999999999999,"5min":4.9299999999999997,"15min":2.5680000000000001},"min":{"1min":0.75600000000000001,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":372.50799999999998,"5min":372.50799999999998,"15min":372.50799999999998},"last":1.391}]},{"osd":2,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2,"15min":1.196},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":3.036,"5min":3.036,"15min":3.036},"last":0.876},{"interface":"front","average":{"1min":1.093,"5min":1.236,"15min":1.26},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.7440000000000002,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":1.2150000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.4870000000000001,"15min":1.5129999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":3.46,"5min":3.46,"15min":3.46},"last":1.054},{"interface":"front","average":{"1min":1.268,"5min":1.3640000000000001,"15min":1.3799999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":3.1970000000000001,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.91900000000000004}]},{"osd":4,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.5880000000000001,"15min":1.6379999999999999},"min":{"1min":0.72499999999999998,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.7999999999999998,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":0.996},{"interface":"front","average":{"1min":1.653,"5min":1.6679999999999999,"15min":1.6699999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":3.694,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.1419999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.3260000000000001,"15min":1.351},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.6360000000000001,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":1.107},{"interface":"front","average":{"1min":1.393,"5min":1.6910000000000001,"15min":1.7410000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.8279999999999998,"5min":5.327,"15min":5.327},"last":1.357}]},{"osd":6,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.446,"15min":1.4370000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.7799999999999998,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.4730000000000001,"5min":14.955,"15min":17.202000000000002},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":3.7309999999999999,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.3360000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1284228,"kb_used_data":235644,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93087612,"statfs":{"total":96636764160,"available":95321714688,"internally_reserved":1073741824,"allocated":241299456,"data_stored":240667736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":27.559999999999999,"15min":31.977},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.1820000000000004,"5min":497.04000000000002,"15min":497.04000000000002},"last":0.61899999999999999},{"interface":"front","average":{"1min":44.500999999999998,"5min":9.8049999999999997,"15min":4.0220000000000002},"min":{"1min":0.64200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":993.84500000000003,"5min":993.84500000000003,"15min":993.84500000000003},"last":0.71099999999999997}]},{"osd":2,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":0.98799999999999999,"15min":0.92800000000000005},"min":{"1min":0.57399999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":4.585,"5min":4.585,"15min":4.585},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.2410000000000001,"15min":1.216},"min":{"1min":0.51800000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":4.149,"5min":4.149,"15min":4.149},"last":0.67400000000000004}]},{"osd":3,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.962,"5min":1.4410000000000001,"15min":1.3540000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":10.603999999999999,"5min":10.603999999999999,"15min":10.603999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":2.1909999999999998,"5min":1.601,"15min":1.5029999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":10.638999999999999,"5min":10.638999999999999,"15min":10.638999999999999},"last":0.874}]},{"osd":4,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":2.1869999999999998,"5min":1.464,"15min":1.3440000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":10.69,"5min":10.69,"15min":10.69},"last":1.089},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.264,"15min":1.2210000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":4.9500000000000002,"5min":4.9500000000000002,"15min":4.9500000000000002},"last":1.141}]},{"osd":5,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":2.0630000000000002,"5min":10.819000000000001,"15min":12.278},"min":{"1min":0.73399999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":10.720000000000001,"5min":213.63800000000001,"15min":213.63800000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.258,"15min":1.212},"min":{"1min":0.82899999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":4.6269999999999998,"5min":4.6269999999999998,"15min":4.6269999999999998},"last":0.99199999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:08:24 2020","interfaces":[{"interface":"back","average":{"1min":1.7589999999999999,"5min":1.593,"15min":1.5649999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":4.5259999999999998,"5min":4.5259999999999998,"15min":4.5259999999999998},"last":0.90000000000000002},{"interface":"front","average":{"1min":2.3690000000000002,"5min":1.5620000000000001,"15min":1.4279999999999999},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":10.792,"5min":10.792,"15min":10.792},"last":1.1100000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.8129999999999999,"5min":1.6539999999999999,"15min":1.6279999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":4.5549999999999997,"5min":4.5549999999999997,"15min":4.5549999999999997},"last":1.0640000000000001},{"interface":"front","average":{"1min":2.2719999999999998,"5min":1.6379999999999999,"15min":1.532},"min":{"1min":0.998,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":10.752000000000001,"5min":10.752000000000001,"15min":10.752000000000001},"last":1.1719999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1155716,"kb_used_data":107132,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93216124,"statfs":{"total":96636764160,"available":95453310976,"internally_reserved":1073741824,"allocated":109703168,"data_stored":109071448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:08:24 2020","interfaces":[{"interface":"back","average":{"1min":81.266000000000005,"5min":17.033999999999999,"15min":6.3289999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":845.57100000000003,"5min":845.57100000000003,"15min":845.57100000000003},"last":1.327},{"interface":"front","average":{"1min":1.083,"5min":0.96299999999999997,"15min":0.94299999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":5.5970000000000004,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":1.74}]},{"osd":2,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.0069999999999999,"15min":0.95399999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":4.1600000000000001,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":1.036},{"interface":"front","average":{"1min":1.0049999999999999,"5min":26.393000000000001,"15min":30.623999999999999},"min":{"1min":0.54200000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.1280000000000001,"5min":606.18499999999995,"15min":606.18499999999995},"last":1.7929999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.0549999999999999,"15min":0.97499999999999998},"min":{"1min":0.64000000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":3.1779999999999999,"5min":3.1779999999999999,"15min":3.1779999999999999},"last":1.6859999999999999},{"interface":"front","average":{"1min":1.927,"5min":1.319,"15min":1.218},"min":{"1min":0.78700000000000003,"5min":0.621,"15min":0.621},"max":{"1min":4.21,"5min":4.21,"15min":4.21},"last":1.657}]},{"osd":4,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.238,"15min":1.1899999999999999},"min":{"1min":0.73899999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":3.0870000000000002,"5min":3.0870000000000002,"15min":3.0870000000000002},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.6890000000000001,"5min":1.3440000000000001,"15min":1.286},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":4.0960000000000001,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":2.0449999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":39.531999999999996,"5min":8.8879999999999999,"15min":3.7799999999999998},"min":{"1min":0.76600000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":501.267,"5min":501.267,"15min":501.267},"last":2.246},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.24,"15min":1.1499999999999999},"min":{"1min":0.874,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":6.0010000000000003,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":2.0939999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.7729999999999999,"5min":7.2450000000000001,"15min":8.157},"min":{"1min":0.81100000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":4.1310000000000002,"5min":124.738,"15min":124.738},"last":2.0059999999999998},{"interface":"front","average":{"1min":1.47,"5min":15.215999999999999,"15min":17.507000000000001},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":3.7120000000000002,"5min":294.94,"15min":294.94},"last":2.2789999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.552,"15min":1.5740000000000001},"min":{"1min":0.96199999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.3929999999999998,"5min":6.468,"15min":6.468},"last":2.3069999999999999},{"interface":"front","average":{"1min":1.6879999999999999,"5min":1.446,"15min":1.4059999999999999},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":3.117,"5min":3.117,"15min":3.117},"last":2.0659999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1111696,"kb_used_data":63112,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93260144,"statfs":{"total":96636764160,"available":95498387456,"internally_reserved":1073741824,"allocated":64626688,"data_stored":63987505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.153,"15min":1.129},"min":{"1min":0.505,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.4909999999999997,"5min":4.4909999999999997,"15min":4.4909999999999997},"last":8.4930000000000003},{"interface":"front","average":{"1min":1.284,"5min":1.036,"15min":0.995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":3.6520000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":3.5}]},{"osd":1,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":2.5219999999999998,"5min":1.3779999999999999,"15min":1.1870000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":18.672000000000001,"5min":18.672000000000001,"15min":18.672000000000001},"last":5.6820000000000004},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.357,"15min":1.359},"min":{"1min":0.70699999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":5.1130000000000004,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":2.2250000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.216,"15min":1.1799999999999999},"min":{"1min":0.65300000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":1.736},{"interface":"front","average":{"1min":1.419,"5min":1.528,"15min":1.546},"min":{"1min":0.59999999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":5.3339999999999996,"5min":8.2210000000000001,"15min":8.2210000000000001},"last":1.4590000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":2.3319999999999999,"5min":1.3540000000000001,"15min":1.1910000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":19.783000000000001,"5min":19.783000000000001,"15min":19.783000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.5589999999999999,"15min":1.615},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.891,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":2.2829999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.2789999999999999,"15min":1.2669999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.7730000000000001,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.5249999999999999,"5min":20.725000000000001,"15min":23.925000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":4.4790000000000001,"5min":415.40699999999998,"15min":415.40699999999998},"last":2.532}]},{"osd":6,"last update":"Fri Aug 7 15:08:25 2020","interfaces":[{"interface":"back","average":{"1min":2.5089999999999999,"5min":1.478,"15min":1.306},"min":{"1min":0.89000000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":20.035,"5min":20.035,"15min":20.035},"last":0.85999999999999999},{"interface":"front","average":{"1min":44.033000000000001,"5min":9.8629999999999995,"15min":4.1680000000000001},"min":{"1min":0.93700000000000006,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":751.70000000000005,"5min":751.70000000000005,"15min":751.70000000000005},"last":1.6719999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":3.5409999999999999,"5min":2.101,"15min":1.861},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":35.548999999999999,"5min":35.548999999999999,"15min":35.548999999999999},"last":2.254},{"interface":"front","average":{"1min":3.4460000000000002,"5min":1.8859999999999999,"15min":1.6259999999999999},"min":{"1min":1.004,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":35.646999999999998,"5min":35.646999999999998,"15min":35.646999999999998},"last":1.7070000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1160944,"kb_used_data":112360,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93210896,"statfs":{"total":96636764160,"available":95447957504,"internally_reserved":1073741824,"allocated":115056640,"data_stored":114417457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.044,"5min":1.411,"15min":1.3049999999999999},"min":{"1min":0.55600000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":8.6020000000000003,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":1.383},{"interface":"front","average":{"1min":1.45,"5min":1.599,"15min":1.6240000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":4.9119999999999999,"5min":13.067,"15min":13.067},"last":0.79500000000000004}]},{"osd":1,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":16.382999999999999,"5min":4.7949999999999999,"15min":2.8639999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":270.68700000000001,"5min":270.68700000000001,"15min":270.68700000000001},"last":1.093},{"interface":"front","average":{"1min":2.7210000000000001,"5min":2.153,"15min":2.0590000000000002},"min":{"1min":0.71599999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":14.349,"5min":14.349,"15min":14.349},"last":7.0940000000000003}]},{"osd":2,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":1.7470000000000001,"5min":1.2669999999999999,"15min":1.1870000000000001},"min":{"1min":0.64900000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":4.9580000000000002,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":1.198},{"interface":"front","average":{"1min":4.5220000000000002,"5min":2.2519999999999998,"15min":1.8740000000000001},"min":{"1min":0.72399999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":51.530000000000001,"5min":51.530000000000001,"15min":51.530000000000001},"last":1.526}]},{"osd":4,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":47.652999999999999,"5min":10.888,"15min":4.7599999999999998},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":506.21300000000002,"5min":506.21300000000002,"15min":506.21300000000002},"last":1.752},{"interface":"front","average":{"1min":29.771000000000001,"5min":31.634,"15min":31.943999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":505.738,"5min":677.88599999999997,"15min":677.88599999999997},"last":1.498}]},{"osd":5,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.012,"5min":1.6160000000000001,"15min":1.55},"min":{"1min":0.96399999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":5.0049999999999999,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.5589999999999999},{"interface":"front","average":{"1min":2.2210000000000001,"5min":1.859,"15min":1.7989999999999999},"min":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":5.226,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":1.595}]},{"osd":6,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":1.99,"5min":1.623,"15min":1.5620000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":5.0389999999999997,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":2.1850000000000001},{"interface":"front","average":{"1min":1.9710000000000001,"5min":1.9830000000000001,"15min":1.9850000000000001},"min":{"1min":1.0369999999999999,"5min":0.876,"15min":0.876},"max":{"1min":4.266,"5min":5.6189999999999998,"15min":5.6189999999999998},"last":1.96}]},{"osd":7,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.0110000000000001,"5min":2.1309999999999998,"15min":2.1509999999999998},"min":{"1min":0.96999999999999997,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":3.79,"5min":5.734,"15min":5.734},"last":1.3200000000000001},{"interface":"front","average":{"1min":2.6200000000000001,"5min":2.105,"15min":2.0190000000000001},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":8.6790000000000003,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":1.7889999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1216808,"kb_used_data":168224,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93155032,"statfs":{"total":96636764160,"available":95390752768,"internally_reserved":1073741824,"allocated":172261376,"data_stored":171622193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":1.125,"15min":1.1619999999999999},"min":{"1min":0.47699999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.3999999999999999,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.349,"5min":0.94899999999999995,"15min":0.88200000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":6.3970000000000002,"5min":6.3970000000000002,"15min":6.3970000000000002},"last":1.194}]},{"osd":1,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.304,"15min":1.337},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.4380000000000002,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":1.397},{"interface":"front","average":{"1min":32.585999999999999,"5min":7.6180000000000003,"15min":3.4569999999999999},"min":{"1min":0.67000000000000004,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":689.846,"5min":689.846,"15min":689.846},"last":1.379}]},{"osd":2,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.46,"15min":1.454},"min":{"1min":0.56399999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":6.8730000000000002,"5min":6.8730000000000002,"15min":6.8730000000000002},"last":1.165},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.5669999999999999,"15min":1.532},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":7.1360000000000001,"5min":7.1360000000000001,"15min":7.1360000000000001},"last":1.222}]},{"osd":3,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":12.619999999999999,"5min":3.73,"15min":2.2480000000000002},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":247.202,"5min":247.202,"15min":247.202},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.583,"5min":1.3169999999999999,"15min":1.2729999999999999},"min":{"1min":0.67800000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":6.431,"5min":6.431,"15min":6.431},"last":1.244}]},{"osd":5,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":46.786999999999999,"5min":10.659000000000001,"15min":4.6379999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":990.71600000000001,"5min":990.71600000000001,"15min":990.71600000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.748,"5min":1.651,"15min":1.635},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":3.9329999999999998,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":1.3169999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.7070000000000001,"5min":1.7729999999999999,"15min":1.784},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":6.9749999999999996,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.524,"5min":15.592000000000001,"15min":17.936},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":6.7729999999999997,"5min":389.14600000000002,"15min":389.14600000000002},"last":0.99299999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":13.378,"5min":3.802,"15min":2.206},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":252.798,"5min":252.798,"15min":252.798},"last":0.95299999999999996},{"interface":"front","average":{"1min":44.676000000000002,"5min":10.252000000000001,"15min":4.5140000000000002},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":501.93000000000001,"5min":501.93000000000001,"15min":501.93000000000001},"last":1.03}]}]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1176656,"kb_used_data":128072,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93195184,"statfs":{"total":96636764160,"available":95431868416,"internally_reserved":1073741824,"allocated":131145728,"data_stored":130506545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0580000000000001,"15min":1.0700000000000001},"min":{"1min":0.49199999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.5169999999999999,"5min":2.766,"15min":2.766},"last":0.51700000000000002},{"interface":"front","average":{"1min":1.2250000000000001,"5min":0.88900000000000001,"15min":0.83299999999999996},"min":{"1min":0.54900000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":0.71599999999999997}]},{"osd":1,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":47.365000000000002,"5min":10.215999999999999,"15min":4.024},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":787.22900000000004,"5min":787.22900000000004,"15min":787.22900000000004},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.071,"5min":1.1000000000000001,"15min":1.105},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.8380000000000001,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":0.60999999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":7.5659999999999998,"15min":8.6679999999999993},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.4950000000000001,"5min":168.56899999999999,"15min":168.56899999999999},"last":1.026},{"interface":"front","average":{"1min":1.2929999999999999,"5min":26.763000000000002,"15min":31.007999999999999},"min":{"1min":0.79700000000000004,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":3.2389999999999999,"5min":676.27099999999996,"15min":676.27099999999996},"last":1.099}]},{"osd":3,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.196,"15min":1.1439999999999999},"min":{"1min":0.89000000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":3.5649999999999999,"5min":3.5649999999999999,"15min":3.5649999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":43.073,"5min":9.5850000000000009,"15min":4.0030000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":712.98699999999997,"5min":712.98699999999997,"15min":712.98699999999997},"last":2.2010000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":40.962000000000003,"15min":47.582000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.4980000000000002,"5min":530.30799999999999,"15min":530.30799999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.278,"15min":1.296},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.6259999999999999,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":1.137}]},{"osd":6,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":40.569000000000003,"15min":47.128999999999998},"min":{"1min":0.76800000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.6699999999999999,"5min":754.31299999999999,"15min":754.31299999999999},"last":966.60900000000004},{"interface":"front","average":{"1min":1.1899999999999999,"5min":21.050999999999998,"15min":24.361000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.7809999999999999,"5min":517.26700000000005,"15min":517.26700000000005},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.3560000000000001,"15min":1.3240000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":3.9079999999999999,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.23},{"interface":"front","average":{"1min":1.411,"5min":1.5349999999999999,"15min":1.556},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":3.9390000000000001,"5min":3.9390000000000001,"15min":3.9390000000000001},"last":1.288}]}]}]}} 2020-08-07T15:09:07.143 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early status --format=json 2020-08-07T15:09:07.145 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:09:07.147 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:09:07.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:09:07.147 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:09:07.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.148 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:108376 3.3 e62/20 ondisk, result = 0)", 2020-08-07T15:09:07.148 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:09:07.148 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 76.449385882000001, 2020-08-07T15:09:07.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62926872599999994, 2020-08-07T15:09:07.148 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:07.149 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:07.149 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:07.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:07.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:09:07.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:07.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:07.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:09:07.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8491999999999998e-05 2020-08-07T15:09:07.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:07.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683061+0000", 2020-08-07T15:09:07.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9099999999999998e-06 2020-08-07T15:09:07.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:07.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683065+0000", 2020-08-07T15:09:07.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62906670600000003 2020-08-07T15:09:07.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:07.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312131+0000", 2020-08-07T15:09:07.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2147000000000001e-05 2020-08-07T15:09:07.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:07.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312143+0000", 2020-08-07T15:09:07.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5152e-05 2020-08-07T15:09:07.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312239+0000", 2020-08-07T15:09:07.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5089999999999993e-06 2020-08-07T15:09:07.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:07.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312247+0000", 2020-08-07T15:09:07.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3810000000000001e-05 2020-08-07T15:09:07.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:07.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312311+0000", 2020-08-07T15:09:07.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62926872599999994 2020-08-07T15:09:07.158 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.159 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:07.159 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.159 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "MOSDPGPushReply(3.1 62/60 [PushReplyOp(3:8329f37e:::benchmark_data_smithi074_35447_object11915:head)])", 2020-08-07T15:09:07.160 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:09:07.160 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 76.274027865999997, 2020-08-07T15:09:07.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996, 2020-08-07T15:09:07.160 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:07.160 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:07.161 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:07.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:07.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:09:07.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:07.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:07.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:09:07.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5372000000000001e-05 2020-08-07T15:09:07.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:07.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858416+0000", 2020-08-07T15:09:07.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6899999999999998e-06 2020-08-07T15:09:07.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:07.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858419+0000", 2020-08-07T15:09:07.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63907904299999996 2020-08-07T15:09:07.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:07.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497498+0000", 2020-08-07T15:09:07.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3332999999999999e-05 2020-08-07T15:09:07.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:07.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497512+0000", 2020-08-07T15:09:07.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000450875 2020-08-07T15:09:07.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497962+0000", 2020-08-07T15:09:07.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000267779 2020-08-07T15:09:07.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:07.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.498230+0000", 2020-08-07T15:09:07.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996 2020-08-07T15:09:07.169 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.169 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:07.170 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.170 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:145792 3.1 3:881e0420:::benchmark_data_smithi074_35447_object16183:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:09:07.170 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:09:07.170 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 52.465621984999999, 2020-08-07T15:09:07.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997, 2020-08-07T15:09:07.171 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:07.171 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:07.172 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:07.173 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:07.173 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:07.173 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 145792 2020-08-07T15:09:07.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.173 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:07.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:07.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:09:07.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:07.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:07.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:09:07.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7489000000000001e-05 2020-08-07T15:09:07.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:07.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666824+0000", 2020-08-07T15:09:07.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0169999999999998e-06 2020-08-07T15:09:07.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:07.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666828+0000", 2020-08-07T15:09:07.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9829999999999996e-06 2020-08-07T15:09:07.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:07.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666836+0000", 2020-08-07T15:09:07.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.481e-06 2020-08-07T15:09:07.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:07.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666840+0000", 2020-08-07T15:09:07.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5299000000000001e-05 2020-08-07T15:09:07.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666885+0000", 2020-08-07T15:09:07.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014456500000000001 2020-08-07T15:09:07.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:07.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667030+0000", 2020-08-07T15:09:07.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017677799999999999 2020-08-07T15:09:07.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,5", 2020-08-07T15:09:07.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667207+0000", 2020-08-07T15:09:07.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004149649 2020-08-07T15:09:07.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:07.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671356+0000", 2020-08-07T15:09:07.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019339649999999999 2020-08-07T15:09:07.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:07.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673290+0000", 2020-08-07T15:09:07.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73346620500000004 2020-08-07T15:09:07.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:07.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406756+0000", 2020-08-07T15:09:07.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4700000000000001e-06 2020-08-07T15:09:07.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:07.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406759+0000", 2020-08-07T15:09:07.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6258999999999998e-05 2020-08-07T15:09:07.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:07.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406815+0000", 2020-08-07T15:09:07.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0126000000000002e-05 2020-08-07T15:09:07.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:07.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406855+0000", 2020-08-07T15:09:07.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997 2020-08-07T15:09:07.191 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.192 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:07.192 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.193 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:145792 3.1 e75/67 ondisk, result = 0)", 2020-08-07T15:09:07.193 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:09:07.193 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 52.461237097000001, 2020-08-07T15:09:07.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003, 2020-08-07T15:09:07.193 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:07.194 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:07.194 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:07.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:07.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:09:07.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:07.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:07.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:09:07.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2830999999999998e-05 2020-08-07T15:09:07.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:07.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671224+0000", 2020-08-07T15:09:07.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:09:07.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:07.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671229+0000", 2020-08-07T15:09:07.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.735414864 2020-08-07T15:09:07.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:07.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406644+0000", 2020-08-07T15:09:07.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1250000000000004e-06 2020-08-07T15:09:07.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:07.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406651+0000", 2020-08-07T15:09:07.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6567e-05 2020-08-07T15:09:07.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406737+0000", 2020-08-07T15:09:07.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0054000000000001e-05 2020-08-07T15:09:07.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:07.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406747+0000", 2020-08-07T15:09:07.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011391999999999999 2020-08-07T15:09:07.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:07.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406861+0000", 2020-08-07T15:09:07.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003 2020-08-07T15:09:07.203 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.203 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:07.203 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.204 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30921 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:09:07.204 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:09:07.204 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.426488722999999, 2020-08-07T15:09:07.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001, 2020-08-07T15:09:07.205 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:07.205 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:07.205 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:07.205 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:07.206 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:07.206 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30921 2020-08-07T15:09:07.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.206 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:07.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:07.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:09:07.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:07.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:07.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:09:07.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6954999999999998e-05 2020-08-07T15:09:07.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:07.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706016+0000", 2020-08-07T15:09:07.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0229999999999992e-06 2020-08-07T15:09:07.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:07.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706024+0000", 2020-08-07T15:09:07.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.432e-05 2020-08-07T15:09:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706039+0000", 2020-08-07T15:09:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8490000000000006e-06 2020-08-07T15:09:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706046+0000", 2020-08-07T15:09:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6837999999999999e-05 2020-08-07T15:09:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706113+0000", 2020-08-07T15:09:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3553000000000001e-05 2020-08-07T15:09:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:07.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706127+0000", 2020-08-07T15:09:07.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77277201699999998 2020-08-07T15:09:07.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.478899+0000", 2020-08-07T15:09:07.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093978599999999996 2020-08-07T15:09:07.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:09:07.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479839+0000", 2020-08-07T15:09:07.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010376029 2020-08-07T15:09:07.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490215+0000", 2020-08-07T15:09:07.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000190657 2020-08-07T15:09:07.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:07.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490405+0000", 2020-08-07T15:09:07.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001 2020-08-07T15:09:07.229 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.230 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:07.230 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.230 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30922 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:09:07.231 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:09:07.231 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.426328560000002, 2020-08-07T15:09:07.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247, 2020-08-07T15:09:07.231 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:07.231 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:07.231 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:07.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:07.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:07.232 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30922 2020-08-07T15:09:07.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.232 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:07.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:07.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:09:07.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:07.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:07.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:09:07.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9178999999999997e-05 2020-08-07T15:09:07.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:07.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706159+0000", 2020-08-07T15:09:07.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4999999999999999e-06 2020-08-07T15:09:07.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:07.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706164+0000", 2020-08-07T15:09:07.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1792e-05 2020-08-07T15:09:07.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:07.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706176+0000", 2020-08-07T15:09:07.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.139e-06 2020-08-07T15:09:07.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:07.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706182+0000", 2020-08-07T15:09:07.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039724900000000002 2020-08-07T15:09:07.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706579+0000", 2020-08-07T15:09:07.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1191e-05 2020-08-07T15:09:07.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:07.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706591+0000", 2020-08-07T15:09:07.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329101300000003 2020-08-07T15:09:07.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479882+0000", 2020-08-07T15:09:07.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9565000000000001e-05 2020-08-07T15:09:07.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:09:07.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479971+0000", 2020-08-07T15:09:07.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010471263 2020-08-07T15:09:07.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490442+0000", 2020-08-07T15:09:07.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015035600000000001 2020-08-07T15:09:07.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:07.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490593+0000", 2020-08-07T15:09:07.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247 2020-08-07T15:09:07.247 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.247 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:07.247 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.248 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30923 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:09:07.249 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:09:07.249 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.425853498999999, 2020-08-07T15:09:07.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998, 2020-08-07T15:09:07.250 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:07.250 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:07.250 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:07.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:07.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:07.257 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30923 2020-08-07T15:09:07.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.258 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:07.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:07.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:09:07.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:07.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:07.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:09:07.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5531000000000001e-05 2020-08-07T15:09:07.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:07.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706630+0000", 2020-08-07T15:09:07.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7689999999999999e-06 2020-08-07T15:09:07.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:07.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706637+0000", 2020-08-07T15:09:07.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3577999999999999e-05 2020-08-07T15:09:07.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:07.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706650+0000", 2020-08-07T15:09:07.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1439999999999997e-06 2020-08-07T15:09:07.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:07.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706658+0000", 2020-08-07T15:09:07.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1663999999999999e-05 2020-08-07T15:09:07.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706709+0000", 2020-08-07T15:09:07.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0149000000000001e-05 2020-08-07T15:09:07.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:07.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706719+0000", 2020-08-07T15:09:07.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329694900000001 2020-08-07T15:09:07.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480016+0000", 2020-08-07T15:09:07.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7998999999999997e-05 2020-08-07T15:09:07.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:09:07.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480084+0000", 2020-08-07T15:09:07.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010547333000000001 2020-08-07T15:09:07.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490632+0000", 2020-08-07T15:09:07.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7439e-05 2020-08-07T15:09:07.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:07.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490729+0000", 2020-08-07T15:09:07.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998 2020-08-07T15:09:07.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.270 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:07.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.271 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30924 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:09:07.271 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:09:07.271 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.425726226999998, 2020-08-07T15:09:07.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995, 2020-08-07T15:09:07.272 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:07.272 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:07.272 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:07.272 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:07.272 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:07.273 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30924 2020-08-07T15:09:07.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.273 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:07.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:07.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:09:07.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:07.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:07.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:09:07.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9698999999999999e-05 2020-08-07T15:09:07.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:07.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706762+0000", 2020-08-07T15:09:07.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8080000000000001e-06 2020-08-07T15:09:07.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:07.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706767+0000", 2020-08-07T15:09:07.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1069e-05 2020-08-07T15:09:07.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:07.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706778+0000", 2020-08-07T15:09:07.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6969999999999998e-06 2020-08-07T15:09:07.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:07.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706784+0000", 2020-08-07T15:09:07.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8792999999999998e-05 2020-08-07T15:09:07.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706843+0000", 2020-08-07T15:09:07.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5449999999999998e-06 2020-08-07T15:09:07.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:07.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706850+0000", 2020-08-07T15:09:07.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.773278354 2020-08-07T15:09:07.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480129+0000", 2020-08-07T15:09:07.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1577999999999997e-05 2020-08-07T15:09:07.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:09:07.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480210+0000", 2020-08-07T15:09:07.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010541915000000001 2020-08-07T15:09:07.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490752+0000", 2020-08-07T15:09:07.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103937 2020-08-07T15:09:07.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:07.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490856+0000", 2020-08-07T15:09:07.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995 2020-08-07T15:09:07.284 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.285 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:07.285 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.285 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30925 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:09:07.286 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:09:07.286 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.425602456, 2020-08-07T15:09:07.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995, 2020-08-07T15:09:07.286 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:07.286 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:07.287 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:07.287 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:07.287 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:07.287 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30925 2020-08-07T15:09:07.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.288 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:07.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:07.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:09:07.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:07.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:07.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:09:07.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4664999999999999e-05 2020-08-07T15:09:07.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:07.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706880+0000", 2020-08-07T15:09:07.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7479999999999999e-06 2020-08-07T15:09:07.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:07.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706886+0000", 2020-08-07T15:09:07.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1838000000000001e-05 2020-08-07T15:09:07.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:07.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706898+0000", 2020-08-07T15:09:07.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0089999999999998e-06 2020-08-07T15:09:07.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:07.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706903+0000", 2020-08-07T15:09:07.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3559e-05 2020-08-07T15:09:07.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706956+0000", 2020-08-07T15:09:07.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0309000000000001e-05 2020-08-07T15:09:07.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:07.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706967+0000", 2020-08-07T15:09:07.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77328290499999996 2020-08-07T15:09:07.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480250+0000", 2020-08-07T15:09:07.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8680999999999997e-05 2020-08-07T15:09:07.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:09:07.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480298+0000", 2020-08-07T15:09:07.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01059272 2020-08-07T15:09:07.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490891+0000", 2020-08-07T15:09:07.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014580300000000001 2020-08-07T15:09:07.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:07.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491037+0000", 2020-08-07T15:09:07.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995 2020-08-07T15:09:07.299 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.299 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:07.300 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.300 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30926 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:09:07.300 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:09:07.301 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.425485587000001, 2020-08-07T15:09:07.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005, 2020-08-07T15:09:07.301 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:07.301 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:07.301 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:07.302 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:07.302 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:07.302 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30926 2020-08-07T15:09:07.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.302 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:07.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:07.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:09:07.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:07.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:07.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:09:07.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5223000000000003e-05 2020-08-07T15:09:07.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:07.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706998+0000", 2020-08-07T15:09:07.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4809999999999997e-06 2020-08-07T15:09:07.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:07.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707003+0000", 2020-08-07T15:09:07.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1663e-05 2020-08-07T15:09:07.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:07.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707015+0000", 2020-08-07T15:09:07.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9729999999999997e-06 2020-08-07T15:09:07.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:07.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707020+0000", 2020-08-07T15:09:07.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2565999999999999e-05 2020-08-07T15:09:07.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707062+0000", 2020-08-07T15:09:07.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8944e-05 2020-08-07T15:09:07.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:07.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707081+0000", 2020-08-07T15:09:07.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77325885000000005 2020-08-07T15:09:07.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480340+0000", 2020-08-07T15:09:07.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0321000000000006e-05 2020-08-07T15:09:07.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:09:07.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480420+0000", 2020-08-07T15:09:07.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010655958 2020-08-07T15:09:07.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491076+0000", 2020-08-07T15:09:07.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9784999999999995e-05 2020-08-07T15:09:07.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:07.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491166+0000", 2020-08-07T15:09:07.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005 2020-08-07T15:09:07.315 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.315 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:07.315 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.316 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30927 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:09:07.316 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:09:07.316 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.425367768000001, 2020-08-07T15:09:07.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999, 2020-08-07T15:09:07.317 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:07.317 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:07.317 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:07.317 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:07.317 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:07.318 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30927 2020-08-07T15:09:07.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.318 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:07.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:07.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:09:07.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:07.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:07.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:09:07.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9861999999999997e-05 2020-08-07T15:09:07.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:07.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707120+0000", 2020-08-07T15:09:07.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8969999999999996e-06 2020-08-07T15:09:07.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:07.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707126+0000", 2020-08-07T15:09:07.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1248e-05 2020-08-07T15:09:07.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:07.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707137+0000", 2020-08-07T15:09:07.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.955e-06 2020-08-07T15:09:07.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:07.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707142+0000", 2020-08-07T15:09:07.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0643999999999998e-05 2020-08-07T15:09:07.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707193+0000", 2020-08-07T15:09:07.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1983e-05 2020-08-07T15:09:07.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:07.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707205+0000", 2020-08-07T15:09:07.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77324914 2020-08-07T15:09:07.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480454+0000", 2020-08-07T15:09:07.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4119999999999998e-05 2020-08-07T15:09:07.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:09:07.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480498+0000", 2020-08-07T15:09:07.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010709707000000001 2020-08-07T15:09:07.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491208+0000", 2020-08-07T15:09:07.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014433800000000001 2020-08-07T15:09:07.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:07.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491352+0000", 2020-08-07T15:09:07.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999 2020-08-07T15:09:07.332 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.332 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:07.332 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.333 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30928 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:09:07.333 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:09:07.333 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.425243411, 2020-08-07T15:09:07.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998, 2020-08-07T15:09:07.333 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:07.334 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:07.334 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:07.334 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:07.334 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:07.334 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30928 2020-08-07T15:09:07.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.335 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:07.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:07.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:09:07.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:07.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:07.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:09:07.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6582000000000001e-05 2020-08-07T15:09:07.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:07.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707241+0000", 2020-08-07T15:09:07.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9190000000000004e-06 2020-08-07T15:09:07.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:07.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707247+0000", 2020-08-07T15:09:07.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000188822 2020-08-07T15:09:07.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:07.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707436+0000", 2020-08-07T15:09:07.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2940000000000002e-06 2020-08-07T15:09:07.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:07.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707442+0000", 2020-08-07T15:09:07.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7800000000000003e-05 2020-08-07T15:09:07.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707490+0000", 2020-08-07T15:09:07.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2020-08-07T15:09:07.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:07.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707501+0000", 2020-08-07T15:09:07.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77303498800000003 2020-08-07T15:09:07.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480536+0000", 2020-08-07T15:09:07.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7723000000000001e-05 2020-08-07T15:09:07.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:09:07.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480614+0000", 2020-08-07T15:09:07.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010812183 2020-08-07T15:09:07.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491426+0000", 2020-08-07T15:09:07.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5502999999999999e-05 2020-08-07T15:09:07.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:07.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491502+0000", 2020-08-07T15:09:07.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998 2020-08-07T15:09:07.350 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.350 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:07.350 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.351 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31009 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:09:07.351 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:09:07.351 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.401287105000002, 2020-08-07T15:09:07.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002, 2020-08-07T15:09:07.352 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:07.352 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:07.352 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:07.352 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:07.352 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:07.353 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31009 2020-08-07T15:09:07.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.353 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:07.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:07.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:09:07.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:07.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:07.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:09:07.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2429000000000002e-05 2020-08-07T15:09:07.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:07.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731193+0000", 2020-08-07T15:09:07.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9760000000000003e-06 2020-08-07T15:09:07.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:07.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731198+0000", 2020-08-07T15:09:07.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3597e-05 2020-08-07T15:09:07.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:07.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731212+0000", 2020-08-07T15:09:07.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1709999999999996e-06 2020-08-07T15:09:07.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:07.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731219+0000", 2020-08-07T15:09:07.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5402999999999997e-05 2020-08-07T15:09:07.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731285+0000", 2020-08-07T15:09:07.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3495e-05 2020-08-07T15:09:07.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:07.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731298+0000", 2020-08-07T15:09:07.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74935423599999995 2020-08-07T15:09:07.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480652+0000", 2020-08-07T15:09:07.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000137806 2020-08-07T15:09:07.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:07.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480790+0000", 2020-08-07T15:09:07.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022272399999999999 2020-08-07T15:09:07.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:09:07.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481013+0000", 2020-08-07T15:09:07.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003223381 2020-08-07T15:09:07.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:07.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484236+0000", 2020-08-07T15:09:07.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1370000000000001e-06 2020-08-07T15:09:07.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:07.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484237+0000", 2020-08-07T15:09:07.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6288e-05 2020-08-07T15:09:07.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:07.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484294+0000", 2020-08-07T15:09:07.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.007260402 2020-08-07T15:09:07.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:07.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491554+0000", 2020-08-07T15:09:07.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7358000000000003e-05 2020-08-07T15:09:07.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:07.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491601+0000", 2020-08-07T15:09:07.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1100000000000004e-07 2020-08-07T15:09:07.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:07.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491602+0000", 2020-08-07T15:09:07.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013436500000000001 2020-08-07T15:09:07.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:07.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491737+0000", 2020-08-07T15:09:07.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002 2020-08-07T15:09:07.372 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.372 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:07.372 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.373 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31010 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:09:07.373 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:09:07.373 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.401161151, 2020-08-07T15:09:07.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001, 2020-08-07T15:09:07.373 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:07.374 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:07.374 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:07.374 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:07.374 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:07.374 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31010 2020-08-07T15:09:07.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.375 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:07.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:07.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:09:07.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:07.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:07.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:09:07.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2398000000000002e-05 2020-08-07T15:09:07.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:07.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731319+0000", 2020-08-07T15:09:07.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7100000000000004e-06 2020-08-07T15:09:07.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:07.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731325+0000", 2020-08-07T15:09:07.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4328000000000001e-05 2020-08-07T15:09:07.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:07.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731339+0000", 2020-08-07T15:09:07.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3310000000000003e-06 2020-08-07T15:09:07.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:07.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731345+0000", 2020-08-07T15:09:07.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8952999999999998e-05 2020-08-07T15:09:07.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731384+0000", 2020-08-07T15:09:07.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0036e-05 2020-08-07T15:09:07.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:07.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731394+0000", 2020-08-07T15:09:07.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75027681300000004 2020-08-07T15:09:07.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481670+0000", 2020-08-07T15:09:07.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016555899999999999 2020-08-07T15:09:07.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:07.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481836+0000", 2020-08-07T15:09:07.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028760999999999999 2020-08-07T15:09:07.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:09:07.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482124+0000", 2020-08-07T15:09:07.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038342680000000001 2020-08-07T15:09:07.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:07.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485958+0000", 2020-08-07T15:09:07.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2559999999999999e-06 2020-08-07T15:09:07.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:07.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485959+0000", 2020-08-07T15:09:07.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3758999999999995e-05 2020-08-07T15:09:07.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:07.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486043+0000", 2020-08-07T15:09:07.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057590419999999998 2020-08-07T15:09:07.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:07.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491802+0000", 2020-08-07T15:09:07.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5757000000000001e-05 2020-08-07T15:09:07.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:07.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:09:07.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9299999999999997e-07 2020-08-07T15:09:07.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:07.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:09:07.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012496999999999999 2020-08-07T15:09:07.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:07.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491993+0000", 2020-08-07T15:09:07.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001 2020-08-07T15:09:07.394 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.394 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:07.394 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.395 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31011 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:09:07.395 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:09:07.395 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.401044945999999, 2020-08-07T15:09:07.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997, 2020-08-07T15:09:07.396 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:07.396 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:07.396 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:07.396 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:07.396 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:07.397 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31011 2020-08-07T15:09:07.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.397 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:07.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:07.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:09:07.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:07.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:07.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:09:07.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9539999999999997e-05 2020-08-07T15:09:07.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:07.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731433+0000", 2020-08-07T15:09:07.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9209999999999997e-06 2020-08-07T15:09:07.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:07.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731439+0000", 2020-08-07T15:09:07.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4246999999999999e-05 2020-08-07T15:09:07.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:07.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731453+0000", 2020-08-07T15:09:07.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8130000000000003e-06 2020-08-07T15:09:07.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:07.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731458+0000", 2020-08-07T15:09:07.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6668000000000001e-05 2020-08-07T15:09:07.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731504+0000", 2020-08-07T15:09:07.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1112e-05 2020-08-07T15:09:07.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:07.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731515+0000", 2020-08-07T15:09:07.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75121332100000004 2020-08-07T15:09:07.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482729+0000", 2020-08-07T15:09:07.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1810000000000004e-05 2020-08-07T15:09:07.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:07.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482811+0000", 2020-08-07T15:09:07.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018001399999999999 2020-08-07T15:09:07.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:09:07.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482991+0000", 2020-08-07T15:09:07.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030701140000000001 2020-08-07T15:09:07.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:07.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486061+0000", 2020-08-07T15:09:07.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4499999999999996e-07 2020-08-07T15:09:07.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:07.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486062+0000", 2020-08-07T15:09:07.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5366999999999998e-05 2020-08-07T15:09:07.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:07.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486147+0000", 2020-08-07T15:09:07.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056872779999999996 2020-08-07T15:09:07.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:07.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491834+0000", 2020-08-07T15:09:07.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024645100000000002 2020-08-07T15:09:07.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:07.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:09:07.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1200000000000003e-07 2020-08-07T15:09:07.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:07.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:09:07.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1451000000000002e-05 2020-08-07T15:09:07.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:07.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492173+0000", 2020-08-07T15:09:07.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997 2020-08-07T15:09:07.419 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.420 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:07.420 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.420 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31012 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:09:07.421 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:09:07.421 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.400390826999999, 2020-08-07T15:09:07.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003, 2020-08-07T15:09:07.421 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:07.421 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:07.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:07.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:07.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:07.422 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31012 2020-08-07T15:09:07.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.423 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:07.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:07.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:09:07.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:07.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:07.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:09:07.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6344999999999997e-05 2020-08-07T15:09:07.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:07.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732094+0000", 2020-08-07T15:09:07.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2920000000000001e-06 2020-08-07T15:09:07.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:07.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732100+0000", 2020-08-07T15:09:07.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.189e-05 2020-08-07T15:09:07.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:07.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732112+0000", 2020-08-07T15:09:07.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3720000000000001e-06 2020-08-07T15:09:07.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:07.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732118+0000", 2020-08-07T15:09:07.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2514000000000004e-05 2020-08-07T15:09:07.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732191+0000", 2020-08-07T15:09:07.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4610000000000001e-05 2020-08-07T15:09:07.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:07.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732205+0000", 2020-08-07T15:09:07.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75132974100000005 2020-08-07T15:09:07.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483535+0000", 2020-08-07T15:09:07.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9185999999999996e-05 2020-08-07T15:09:07.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:07.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483624+0000", 2020-08-07T15:09:07.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015683 2020-08-07T15:09:07.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:09:07.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483781+0000", 2020-08-07T15:09:07.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046202889999999996 2020-08-07T15:09:07.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:07.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488401+0000", 2020-08-07T15:09:07.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6700000000000002e-07 2020-08-07T15:09:07.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:07.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488402+0000", 2020-08-07T15:09:07.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1487000000000001e-05 2020-08-07T15:09:07.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:07.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488484+0000", 2020-08-07T15:09:07.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035520830000000002 2020-08-07T15:09:07.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:07.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492036+0000", 2020-08-07T15:09:07.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029266600000000002 2020-08-07T15:09:07.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:07.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492328+0000", 2020-08-07T15:09:07.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5099999999999998e-07 2020-08-07T15:09:07.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:07.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492329+0000", 2020-08-07T15:09:07.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000201299 2020-08-07T15:09:07.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:07.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492531+0000", 2020-08-07T15:09:07.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003 2020-08-07T15:09:07.450 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.450 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:07.451 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.452 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31013 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:09:07.452 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:09:07.453 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.400179196, 2020-08-07T15:09:07.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996, 2020-08-07T15:09:07.453 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:07.454 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:07.454 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:07.454 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:07.454 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:07.454 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31013 2020-08-07T15:09:07.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.455 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:07.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:07.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:09:07.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:07.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:07.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:09:07.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8949000000000002e-05 2020-08-07T15:09:07.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:07.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732298+0000", 2020-08-07T15:09:07.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1399999999999997e-06 2020-08-07T15:09:07.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:07.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732304+0000", 2020-08-07T15:09:07.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.154e-05 2020-08-07T15:09:07.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:07.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732315+0000", 2020-08-07T15:09:07.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1319999999999999e-06 2020-08-07T15:09:07.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:07.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732322+0000", 2020-08-07T15:09:07.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7431e-05 2020-08-07T15:09:07.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732369+0000", 2020-08-07T15:09:07.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0621999999999999e-05 2020-08-07T15:09:07.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:07.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732380+0000", 2020-08-07T15:09:07.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75196845999999995 2020-08-07T15:09:07.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484348+0000", 2020-08-07T15:09:07.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000123008 2020-08-07T15:09:07.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:07.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484471+0000", 2020-08-07T15:09:07.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017572700000000001 2020-08-07T15:09:07.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:09:07.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484647+0000", 2020-08-07T15:09:07.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038557650000000001 2020-08-07T15:09:07.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:07.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:09:07.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2020-08-07T15:09:07.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:07.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:09:07.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0714e-05 2020-08-07T15:09:07.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:07.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488584+0000", 2020-08-07T15:09:07.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036298319999999999 2020-08-07T15:09:07.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:07.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492214+0000", 2020-08-07T15:09:07.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039530099999999997 2020-08-07T15:09:07.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:07.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492609+0000", 2020-08-07T15:09:07.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5300000000000002e-07 2020-08-07T15:09:07.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:07.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492610+0000", 2020-08-07T15:09:07.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197321 2020-08-07T15:09:07.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:07.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492807+0000", 2020-08-07T15:09:07.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996 2020-08-07T15:09:07.475 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.475 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:07.475 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.476 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31014 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:09:07.476 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:09:07.476 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.398797051999999, 2020-08-07T15:09:07.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998, 2020-08-07T15:09:07.477 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:07.477 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:07.484 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:07.484 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:07.484 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:07.484 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31014 2020-08-07T15:09:07.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.485 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:07.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:07.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:09:07.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:07.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:07.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:09:07.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3743999999999999e-05 2020-08-07T15:09:07.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:07.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733685+0000", 2020-08-07T15:09:07.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1720000000000007e-06 2020-08-07T15:09:07.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:07.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733693+0000", 2020-08-07T15:09:07.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3674e-05 2020-08-07T15:09:07.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:07.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733707+0000", 2020-08-07T15:09:07.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9249999999999998e-06 2020-08-07T15:09:07.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:07.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733714+0000", 2020-08-07T15:09:07.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8338000000000003e-05 2020-08-07T15:09:07.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733772+0000", 2020-08-07T15:09:07.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3019e-05 2020-08-07T15:09:07.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:07.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733785+0000", 2020-08-07T15:09:07.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75134856299999997 2020-08-07T15:09:07.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485133+0000", 2020-08-07T15:09:07.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8739999999999995e-05 2020-08-07T15:09:07.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:07.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485212+0000", 2020-08-07T15:09:07.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151244 2020-08-07T15:09:07.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:09:07.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485363+0000", 2020-08-07T15:09:07.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032929090000000001 2020-08-07T15:09:07.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:07.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488656+0000", 2020-08-07T15:09:07.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.065e-06 2020-08-07T15:09:07.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:07.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488657+0000", 2020-08-07T15:09:07.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1123000000000003e-05 2020-08-07T15:09:07.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:07.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488729+0000", 2020-08-07T15:09:07.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035508330000000002 2020-08-07T15:09:07.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:07.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492279+0000", 2020-08-07T15:09:07.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00059376000000000003 2020-08-07T15:09:07.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:07.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492873+0000", 2020-08-07T15:09:07.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5700000000000002e-07 2020-08-07T15:09:07.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:07.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492874+0000", 2020-08-07T15:09:07.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019264 2020-08-07T15:09:07.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:07.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493066+0000", 2020-08-07T15:09:07.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998 2020-08-07T15:09:07.545 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.546 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:07.546 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.546 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31015 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:09:07.547 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:09:07.547 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.398671086, 2020-08-07T15:09:07.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002, 2020-08-07T15:09:07.547 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:07.547 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:07.548 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:07.548 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:07.548 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:07.548 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31015 2020-08-07T15:09:07.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.549 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:07.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:07.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:09:07.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:07.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:07.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:09:07.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3707000000000001e-05 2020-08-07T15:09:07.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:07.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733791+0000", 2020-08-07T15:09:07.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8120000000000001e-06 2020-08-07T15:09:07.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:07.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733794+0000", 2020-08-07T15:09:07.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0728e-05 2020-08-07T15:09:07.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:07.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733805+0000", 2020-08-07T15:09:07.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8279999999999999e-06 2020-08-07T15:09:07.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:07.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733811+0000", 2020-08-07T15:09:07.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2955999999999999e-05 2020-08-07T15:09:07.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733864+0000", 2020-08-07T15:09:07.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1389000000000001e-05 2020-08-07T15:09:07.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:07.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733875+0000", 2020-08-07T15:09:07.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75232711100000005 2020-08-07T15:09:07.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486202+0000", 2020-08-07T15:09:07.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027503400000000001 2020-08-07T15:09:07.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:07.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486478+0000", 2020-08-07T15:09:07.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033010400000000001 2020-08-07T15:09:07.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:09:07.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486808+0000", 2020-08-07T15:09:07.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019382609999999999 2020-08-07T15:09:07.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:07.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488746+0000", 2020-08-07T15:09:07.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9400000000000004e-07 2020-08-07T15:09:07.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:07.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488747+0000", 2020-08-07T15:09:07.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4324000000000002e-05 2020-08-07T15:09:07.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:07.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488821+0000", 2020-08-07T15:09:07.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043745640000000001 2020-08-07T15:09:07.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:07.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493196+0000", 2020-08-07T15:09:07.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030381000000000001 2020-08-07T15:09:07.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:07.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493499+0000", 2020-08-07T15:09:07.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4e-07 2020-08-07T15:09:07.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:07.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493500+0000", 2020-08-07T15:09:07.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017499 2020-08-07T15:09:07.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:07.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493675+0000", 2020-08-07T15:09:07.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002 2020-08-07T15:09:07.569 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.569 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:07.569 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.570 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31016 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:09:07.570 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:09:07.570 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.398075111000001, 2020-08-07T15:09:07.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971, 2020-08-07T15:09:07.570 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:07.571 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:07.571 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:07.571 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:07.571 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:07.572 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31016 2020-08-07T15:09:07.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.573 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:07.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:07.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:09:07.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:07.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:07.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:09:07.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0924000000000002e-05 2020-08-07T15:09:07.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:07.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734404+0000", 2020-08-07T15:09:07.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2020-08-07T15:09:07.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:07.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734410+0000", 2020-08-07T15:09:07.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3459999999999994e-06 2020-08-07T15:09:07.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:07.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734418+0000", 2020-08-07T15:09:07.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9640000000000002e-06 2020-08-07T15:09:07.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:07.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734423+0000", 2020-08-07T15:09:07.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7253999999999999e-05 2020-08-07T15:09:07.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734470+0000", 2020-08-07T15:09:07.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0249999999999997e-06 2020-08-07T15:09:07.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:07.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734477+0000", 2020-08-07T15:09:07.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75294592900000001 2020-08-07T15:09:07.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487423+0000", 2020-08-07T15:09:07.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031604300000000001 2020-08-07T15:09:07.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:07.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487739+0000", 2020-08-07T15:09:07.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000171149 2020-08-07T15:09:07.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:09:07.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487910+0000", 2020-08-07T15:09:07.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024207769999999998 2020-08-07T15:09:07.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:07.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490331+0000", 2020-08-07T15:09:07.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1650000000000001e-06 2020-08-07T15:09:07.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:07.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490332+0000", 2020-08-07T15:09:07.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9796999999999996e-05 2020-08-07T15:09:07.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:07.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490402+0000", 2020-08-07T15:09:07.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029928849999999998 2020-08-07T15:09:07.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:07.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493395+0000", 2020-08-07T15:09:07.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034903199999999999 2020-08-07T15:09:07.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:07.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493744+0000", 2020-08-07T15:09:07.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8899999999999999e-07 2020-08-07T15:09:07.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:07.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493745+0000", 2020-08-07T15:09:07.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197362 2020-08-07T15:09:07.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:07.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493942+0000", 2020-08-07T15:09:07.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971 2020-08-07T15:09:07.685 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.685 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:07.686 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.686 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.695 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:07.695 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:09:07.696 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:09:07.697 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:09:07.740 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 41 16 49750 49734 9.47584 13.2578 0.00527989 0.0131865 2020-08-07T15:09:07.747 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:07.695+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:07.752 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:09:07.810 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:07.807+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:07.899 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:07.897+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:07.925 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:09:07.925 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:09:07.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.925 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:50631 4.8 4.e1c7e248 (undecoded) ondisk+write+known_if_redirected e119)", 2020-08-07T15:09:07.926 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:07.921544+0000", 2020-08-07T15:09:07.926 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.001380472, 2020-08-07T15:09:07.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001402698, 2020-08-07T15:09:07.926 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:07.926 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:09:07.927 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:07.927 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:07.927 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:07.927 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 50631 2020-08-07T15:09:07.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.928 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:07.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:07.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.921544+0000", 2020-08-07T15:09:07.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:07.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:07.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.921544+0000", 2020-08-07T15:09:07.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7830000000000001e-05 2020-08-07T15:09:07.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:07.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.921592+0000", 2020-08-07T15:09:07.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4090000000000002e-06 2020-08-07T15:09:07.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:07.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.921597+0000", 2020-08-07T15:09:07.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0403e-05 2020-08-07T15:09:07.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:07.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.921608+0000", 2020-08-07T15:09:07.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0490000000000003e-06 2020-08-07T15:09:07.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:07.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.921613+0000", 2020-08-07T15:09:07.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8690999999999997e-05 2020-08-07T15:09:07.959 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.959 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:07.959 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.960 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:50627 4.8 e119/81)", 2020-08-07T15:09:07.960 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:07.922860+0000", 2020-08-07T15:09:07.961 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.4028000000000004e-05, 2020-08-07T15:09:07.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000158257, 2020-08-07T15:09:07.961 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:07.962 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2020-08-07T15:09:07.962 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:07.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:07.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.922860+0000", 2020-08-07T15:09:07.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:07.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:07.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.922860+0000", 2020-08-07T15:09:07.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2101000000000003e-05 2020-08-07T15:09:07.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:07.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.922892+0000", 2020-08-07T15:09:07.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0170000000000002e-06 2020-08-07T15:09:07.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:07.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.922898+0000", 2020-08-07T15:09:07.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8627000000000004e-05 2020-08-07T15:09:07.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:07.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.922987+0000", 2020-08-07T15:09:07.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3800000000000002e-06 2020-08-07T15:09:07.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:07.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.922992+0000", 2020-08-07T15:09:07.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000132125 2020-08-07T15:09:07.970 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.970 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:07.971 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:07.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.971 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:50600 4.9 4:9d0b2236:::benchmark_data_smithi074_44586_object6324:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e119)", 2020-08-07T15:09:07.971 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:07.901773+0000", 2020-08-07T15:09:07.972 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021150896999999998, 2020-08-07T15:09:07.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0213163, 2020-08-07T15:09:07.972 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:07.973 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:07.973 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:07.974 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:07.974 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:07.974 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 50600 2020-08-07T15:09:07.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.974 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:07.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:07.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.901773+0000", 2020-08-07T15:09:07.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:07.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:07.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.901773+0000", 2020-08-07T15:09:07.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6418999999999999e-05 2020-08-07T15:09:07.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:07.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.901800+0000", 2020-08-07T15:09:07.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8840000000000004e-06 2020-08-07T15:09:07.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:07.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.901804+0000", 2020-08-07T15:09:07.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5220000000000007e-06 2020-08-07T15:09:07.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:07.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.901812+0000", 2020-08-07T15:09:07.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000001e-06 2020-08-07T15:09:07.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:07.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.901817+0000", 2020-08-07T15:09:07.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5442999999999999e-05 2020-08-07T15:09:07.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:07.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.901852+0000", 2020-08-07T15:09:07.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8400999999999995e-05 2020-08-07T15:09:07.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:07.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.901951+0000", 2020-08-07T15:09:07.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000176262 2020-08-07T15:09:07.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:09:07.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.902127+0000", 2020-08-07T15:09:07.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067708099999999995 2020-08-07T15:09:07.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:07.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.902804+0000", 2020-08-07T15:09:07.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4600000000000004e-07 2020-08-07T15:09:07.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:07.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.902805+0000", 2020-08-07T15:09:07.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1377e-05 2020-08-07T15:09:07.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:07.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:07.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:07.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.902856+0000", 2020-08-07T15:09:07.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041132629999999998 2020-08-07T15:09:08.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:08.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.906970+0000", 2020-08-07T15:09:08.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051960679999999999 2020-08-07T15:09:08.004 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:08.004 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:08.004 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:08.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.005 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:50408 4.1 4:8e421dda:::benchmark_data_smithi074_44586_object6300:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e119)", 2020-08-07T15:09:08.005 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:07.767595+0000", 2020-08-07T15:09:08.005 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.15532890799999999, 2020-08-07T15:09:08.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.15561876099999999, 2020-08-07T15:09:08.013 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:08.013 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:08.013 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:08.013 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:08.014 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:08.014 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 50408 2020-08-07T15:09:08.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.014 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:08.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:08.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.767595+0000", 2020-08-07T15:09:08.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:08.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:08.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.767595+0000", 2020-08-07T15:09:08.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8065999999999997e-05 2020-08-07T15:09:08.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:08.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.767644+0000", 2020-08-07T15:09:08.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9790000000000001e-06 2020-08-07T15:09:08.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:08.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.767648+0000", 2020-08-07T15:09:08.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6770000000000007e-06 2020-08-07T15:09:08.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:08.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.767658+0000", 2020-08-07T15:09:08.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1339999999999997e-06 2020-08-07T15:09:08.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:08.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.767662+0000", 2020-08-07T15:09:08.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057820279999999998 2020-08-07T15:09:08.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:08.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.773444+0000", 2020-08-07T15:09:08.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4965999999999998e-05 2020-08-07T15:09:08.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:08.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.773539+0000", 2020-08-07T15:09:08.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018655700000000001 2020-08-07T15:09:08.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:09:08.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.773726+0000", 2020-08-07T15:09:08.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00161378 2020-08-07T15:09:08.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:08.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.775340+0000", 2020-08-07T15:09:08.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8899999999999998e-07 2020-08-07T15:09:08.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:08.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.775341+0000", 2020-08-07T15:09:08.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6833999999999998e-05 2020-08-07T15:09:08.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:08.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.775397+0000", 2020-08-07T15:09:08.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027591500000000001 2020-08-07T15:09:08.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:08.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.778157+0000", 2020-08-07T15:09:08.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01056116 2020-08-07T15:09:08.079 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:08.085 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:08.086 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:08.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.086 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:50628 4.8 4:1247e387:::benchmark_data_smithi074_44586_object6328:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e119)", 2020-08-07T15:09:08.087 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:07.918898+0000", 2020-08-07T15:09:08.087 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0040266520000000004, 2020-08-07T15:09:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004440215, 2020-08-07T15:09:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:09:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 50628 2020-08-07T15:09:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.918898+0000", 2020-08-07T15:09:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.918898+0000", 2020-08-07T15:09:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7778999999999998e-05 2020-08-07T15:09:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.918945+0000", 2020-08-07T15:09:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9599999999999997e-06 2020-08-07T15:09:08.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:08.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.918951+0000", 2020-08-07T15:09:08.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2323e-05 2020-08-07T15:09:08.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:08.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.918964+0000", 2020-08-07T15:09:08.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5849999999999999e-06 2020-08-07T15:09:08.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:08.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.918969+0000", 2020-08-07T15:09:08.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000146711 2020-08-07T15:09:08.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:08.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.919116+0000", 2020-08-07T15:09:08.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4141999999999995e-05 2020-08-07T15:09:08.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:08.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.919200+0000", 2020-08-07T15:09:08.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000152273 2020-08-07T15:09:08.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6,7", 2020-08-07T15:09:08.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.919352+0000", 2020-08-07T15:09:08.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031451460000000001 2020-08-07T15:09:08.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:08.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.922498+0000", 2020-08-07T15:09:08.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003599919 2020-08-07T15:09:08.130 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:08.130 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:08.130 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:08.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.131 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:50629 4.8 4:1247e387:::benchmark_data_smithi074_44586_object6328:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e119)", 2020-08-07T15:09:08.132 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:07.920404+0000", 2020-08-07T15:09:08.132 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0025199509999999999, 2020-08-07T15:09:08.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030354330000000001, 2020-08-07T15:09:08.132 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:08.132 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:09:08.133 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:08.133 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:08.133 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:08.133 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 50629 2020-08-07T15:09:08.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.134 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:08.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:08.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.920404+0000", 2020-08-07T15:09:08.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:08.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:08.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.920404+0000", 2020-08-07T15:09:08.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9626000000000002e-05 2020-08-07T15:09:08.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:08.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.920454+0000", 2020-08-07T15:09:08.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2700000000000001e-06 2020-08-07T15:09:08.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:08.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.920460+0000", 2020-08-07T15:09:08.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2309e-05 2020-08-07T15:09:08.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:08.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.920473+0000", 2020-08-07T15:09:08.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9599999999999997e-06 2020-08-07T15:09:08.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.920479+0000", 2020-08-07T15:09:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012176629999999999 2020-08-07T15:09:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:08.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.921696+0000", 2020-08-07T15:09:08.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2247e-05 2020-08-07T15:09:08.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:08.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.921788+0000", 2020-08-07T15:09:08.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017268599999999999 2020-08-07T15:09:08.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6,7", 2020-08-07T15:09:08.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.921961+0000", 2020-08-07T15:09:08.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001556761 2020-08-07T15:09:08.146 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:08.146 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:08.146 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:08.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.147 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:50630 4.8 4:1247e387:::benchmark_data_smithi074_44586_object6328:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e119)", 2020-08-07T15:09:08.147 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:07.921437+0000", 2020-08-07T15:09:08.147 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.001487398, 2020-08-07T15:09:08.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020991030000000002, 2020-08-07T15:09:08.148 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:08.148 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:09:08.148 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:08.148 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:08.149 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:08.149 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 50630 2020-08-07T15:09:08.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.149 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:08.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:08.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.921437+0000", 2020-08-07T15:09:08.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:08.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:08.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.921437+0000", 2020-08-07T15:09:08.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8769e-05 2020-08-07T15:09:08.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:08.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.921486+0000", 2020-08-07T15:09:08.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9000000000000003e-06 2020-08-07T15:09:08.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:08.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.921492+0000", 2020-08-07T15:09:08.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2306999999999999e-05 2020-08-07T15:09:08.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:08.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.921504+0000", 2020-08-07T15:09:08.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7810000000000002e-06 2020-08-07T15:09:08.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:08.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.921510+0000", 2020-08-07T15:09:08.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010208579999999999 2020-08-07T15:09:08.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:08.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.922531+0000", 2020-08-07T15:09:08.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6187000000000007e-05 2020-08-07T15:09:08.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:08.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.922617+0000", 2020-08-07T15:09:08.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019121200000000001 2020-08-07T15:09:08.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:08.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:08.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6,7", 2020-08-07T15:09:08.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:07.922808+0000", 2020-08-07T15:09:08.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001371014 2020-08-07T15:09:08.171 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:08.171 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:08.172 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:08.172 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:08.172 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:09:08.172 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 7 2020-08-07T15:09:08.174 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:09:08.176 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:09:08.176 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:09:08.176 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:09:08.176 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:09:08.177 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:08.174+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:08.177 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:09:08.195 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:09:08.237 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:09:08.278 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 42 16 51203 51187 9.52048 11.3516 0.00466965 0.0131242 2020-08-07T15:09:08.287 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:08.277+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:08.287 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:09:08.330 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:09:08.331 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:09:08.331 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:09:08.331 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:09:08.331 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:09:08.334 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:09:08.334 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:09:08.335 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:09:08.339 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:09:08.340 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:09:08.340 INFO:teuthology.orchestra.run.smithi074.stdout:{"fsid":"a01d3e82-8d24-4288-9e42-576ab3ff4957","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"nodeep-scrub flag(s) set","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":180,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":119,"num_osds":8,"num_up_osds":8,"osd_up_since":1596812777,"num_in_osds":8,"osd_in_since":1596812777,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":44},{"state_name":"remapped+peering","count":1}],"num_pgs":45,"num_pools":3,"num_objects":5396,"data_bytes":353501184,"bytes_used":9695166464,"bytes_avail":763398946816,"bytes_total":773094113280,"inactive_pgs_ratio":0.02222222276031971,"write_bytes_sec":1740678,"read_op_per_sec":0,"write_op_per_sec":212},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2020-08-07T15:07:18.623632+0000","services":{}},"progress_events":{}} 2020-08-07T15:09:08.349 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:09:08.363 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2020-08-07T15:09:08.364 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:09:08.387 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:08.382+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:08.393 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:08.394 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:08.424 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:09:08.486 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:08.482+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:08.602 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:08.585+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:08.629 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:08.630 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:08.641 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:09:08.692 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:08.689+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:08.796 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:08.793+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:08.846 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:09:08.867 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:08.869 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:08.869 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:09:08.871 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:09:08.871 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2020-08-07T15:09:08.644572+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":402391040,"num_objects":6142,"num_object_clones":0,"num_object_copies":18426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49120,"num_write_kb":392960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":636,"num_bytes_recovered":41558016,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":135,"acting":135,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9587788,"kb_used_data":1199116,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":745386932,"statfs":{"total":773094113280,"available":763276218368,"internally_reserved":8589934592,"allocated":1227894784,"data_stored":1222533191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,3,9,0,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":48889856,"num_objects":746,"num_object_clones":0,"num_object_copies":2238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5968,"num_write_kb":47744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":32768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.471929"},"pg_stats":[{"pgid":"1.14","version":"16'2","reported_seq":"47","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.268218+0000","last_change":"2020-08-07T15:08:42.533999+0000","last_active":"2020-08-07T15:09:04.268218+0000","last_peered":"2020-08-07T15:09:04.268218+0000","last_clean":"2020-08-07T15:09:04.268218+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:09:04.268218+0000","last_undegraded":"2020-08-07T15:09:04.268218+0000","last_fullsized":"2020-08-07T15:09:04.268218+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:42.533921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:42.533921+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"83","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.179395+0000","last_change":"2020-08-07T15:08:58.212828+0000","last_active":"2020-08-07T15:09:04.179395+0000","last_peered":"2020-08-07T15:09:04.179395+0000","last_clean":"2020-08-07T15:09:04.179395+0000","last_became_active":"2020-08-07T15:08:58.211785+0000","last_became_peered":"2020-08-07T15:08:58.211785+0000","last_unstale":"2020-08-07T15:09:04.179395+0000","last_undegraded":"2020-08-07T15:09:04.179395+0000","last_fullsized":"2020-08-07T15:09:04.179395+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.141010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.141010+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"47","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.177181+0000","last_change":"2020-08-07T15:08:55.078015+0000","last_active":"2020-08-07T15:09:04.177181+0000","last_peered":"2020-08-07T15:09:04.177181+0000","last_clean":"2020-08-07T15:09:04.177181+0000","last_became_active":"2020-08-07T15:08:41.685106+0000","last_became_peered":"2020-08-07T15:08:41.685106+0000","last_unstale":"2020-08-07T15:09:04.177181+0000","last_undegraded":"2020-08-07T15:09:04.177181+0000","last_fullsized":"2020-08-07T15:09:04.177181+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:55.077924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:55.077924+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"126","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.177119+0000","last_change":"2020-08-07T15:08:43.544399+0000","last_active":"2020-08-07T15:09:04.177119+0000","last_peered":"2020-08-07T15:09:04.177119+0000","last_clean":"2020-08-07T15:09:04.177119+0000","last_became_active":"2020-08-07T15:08:41.675776+0000","last_became_peered":"2020-08-07T15:08:41.675776+0000","last_unstale":"2020-08-07T15:09:04.177119+0000","last_undegraded":"2020-08-07T15:09:04.177119+0000","last_fullsized":"2020-08-07T15:09:04.177119+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.544324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.544324+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"154","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.177040+0000","last_change":"2020-08-07T15:08:51.599032+0000","last_active":"2020-08-07T15:09:04.177040+0000","last_peered":"2020-08-07T15:09:04.177040+0000","last_clean":"2020-08-07T15:09:04.177040+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:09:04.177040+0000","last_undegraded":"2020-08-07T15:09:04.177040+0000","last_fullsized":"2020-08-07T15:09:04.177040+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:51.598947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.598947+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"113","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:05.318396+0000","last_change":"2020-08-07T15:08:11.840861+0000","last_active":"2020-08-07T15:09:05.318396+0000","last_peered":"2020-08-07T15:09:05.318396+0000","last_clean":"2020-08-07T15:09:05.318396+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:09:05.318396+0000","last_undegraded":"2020-08-07T15:09:05.318396+0000","last_fullsized":"2020-08-07T15:09:05.318396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:11.840807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:11.840807+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"103","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.175603+0000","last_change":"2020-08-07T15:08:39.558760+0000","last_active":"2020-08-07T15:09:04.175603+0000","last_peered":"2020-08-07T15:09:04.175603+0000","last_clean":"2020-08-07T15:09:04.175603+0000","last_became_active":"2020-08-07T15:08:36.615358+0000","last_became_peered":"2020-08-07T15:08:36.615358+0000","last_unstale":"2020-08-07T15:09:04.175603+0000","last_undegraded":"2020-08-07T15:09:04.175603+0000","last_fullsized":"2020-08-07T15:09:04.175603+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:39.558677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:39.558677+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"118'2992","reported_seq":"3037","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:06.667165+0000","last_change":"2020-08-07T15:08:46.715869+0000","last_active":"2020-08-07T15:09:06.667165+0000","last_peered":"2020-08-07T15:09:06.667165+0000","last_clean":"2020-08-07T15:09:06.667165+0000","last_became_active":"2020-08-07T15:08:46.708112+0000","last_became_peered":"2020-08-07T15:08:46.708112+0000","last_unstale":"2020-08-07T15:09:06.667165+0000","last_undegraded":"2020-08-07T15:09:06.667165+0000","last_fullsized":"2020-08-07T15:09:06.667165+0000","mapping_epoch":98,"log_start":"118'2984","ondisk_log_start":"118'2984","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24510464,"num_objects":374,"num_object_clones":0,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2992,"num_write_kb":23936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"115","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.179579+0000","last_change":"2020-08-07T15:08:40.057705+0000","last_active":"2020-08-07T15:09:04.179579+0000","last_peered":"2020-08-07T15:09:04.179579+0000","last_clean":"2020-08-07T15:09:04.179579+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:09:04.179579+0000","last_undegraded":"2020-08-07T15:09:04.179579+0000","last_fullsized":"2020-08-07T15:09:04.179579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:40.057622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:40.057622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"72","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.268511+0000","last_change":"2020-08-07T15:08:53.756554+0000","last_active":"2020-08-07T15:09:04.268511+0000","last_peered":"2020-08-07T15:09:04.268511+0000","last_clean":"2020-08-07T15:09:04.268511+0000","last_became_active":"2020-08-07T15:08:46.710404+0000","last_became_peered":"2020-08-07T15:08:46.710404+0000","last_unstale":"2020-08-07T15:09:04.268511+0000","last_undegraded":"2020-08-07T15:09:04.268511+0000","last_fullsized":"2020-08-07T15:09:04.268511+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:53.756343+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:53.756343+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"118'3232","reported_seq":"3922","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:06.568365+0000","last_change":"2020-08-07T15:09:01.304273+0000","last_active":"2020-08-07T15:09:06.568365+0000","last_peered":"2020-08-07T15:09:06.568365+0000","last_clean":"2020-08-07T15:09:06.568365+0000","last_became_active":"2020-08-07T15:09:01.293427+0000","last_became_peered":"2020-08-07T15:09:01.293427+0000","last_unstale":"2020-08-07T15:09:06.568365+0000","last_undegraded":"2020-08-07T15:09:06.568365+0000","last_fullsized":"2020-08-07T15:09:06.568365+0000","mapping_epoch":113,"log_start":"118'3229","ondisk_log_start":"118'3229","created":81,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":26476544,"num_objects":404,"num_object_clones":0,"num_object_copies":1212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3232,"num_write_kb":25856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":20971520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"112","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.212644+0000","last_change":"2020-08-07T15:08:27.906894+0000","last_active":"2020-08-07T15:09:04.212644+0000","last_peered":"2020-08-07T15:09:04.212644+0000","last_clean":"2020-08-07T15:09:04.212644+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:09:04.212644+0000","last_undegraded":"2020-08-07T15:09:04.212644+0000","last_fullsized":"2020-08-07T15:09:04.212644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:27.906822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:27.906822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"118","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:05.318412+0000","last_change":"2020-08-07T15:08:43.899342+0000","last_active":"2020-08-07T15:09:05.318412+0000","last_peered":"2020-08-07T15:09:05.318412+0000","last_clean":"2020-08-07T15:09:05.318412+0000","last_became_active":"2020-08-07T15:08:36.622209+0000","last_became_peered":"2020-08-07T15:08:36.622209+0000","last_unstale":"2020-08-07T15:09:05.318412+0000","last_undegraded":"2020-08-07T15:09:05.318412+0000","last_fullsized":"2020-08-07T15:09:05.318412+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.899280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.899280+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"118'2888","reported_seq":"2933","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:06.693411+0000","last_change":"2020-08-07T15:08:24.549715+0000","last_active":"2020-08-07T15:09:06.693411+0000","last_peered":"2020-08-07T15:09:06.693411+0000","last_clean":"2020-08-07T15:09:06.693411+0000","last_became_active":"2020-08-07T15:08:24.549245+0000","last_became_peered":"2020-08-07T15:08:24.549245+0000","last_unstale":"2020-08-07T15:09:06.693411+0000","last_undegraded":"2020-08-07T15:09:06.693411+0000","last_fullsized":"2020-08-07T15:09:06.693411+0000","mapping_epoch":81,"log_start":"118'2878","ondisk_log_start":"118'2878","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23658496,"num_objects":361,"num_object_clones":0,"num_object_copies":1083,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2888,"num_write_kb":23104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"112","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.212863+0000","last_change":"2020-08-07T15:08:13.867355+0000","last_active":"2020-08-07T15:09:04.212863+0000","last_peered":"2020-08-07T15:09:04.212863+0000","last_clean":"2020-08-07T15:09:04.212863+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:09:04.212863+0000","last_undegraded":"2020-08-07T15:09:04.212863+0000","last_fullsized":"2020-08-07T15:09:04.212863+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:13.867148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:13.867148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"118","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:05.318235+0000","last_change":"2020-08-07T15:08:50.686578+0000","last_active":"2020-08-07T15:09:05.318235+0000","last_peered":"2020-08-07T15:09:05.318235+0000","last_clean":"2020-08-07T15:09:05.318235+0000","last_became_active":"2020-08-07T15:08:36.633406+0000","last_became_peered":"2020-08-07T15:08:36.633406+0000","last_unstale":"2020-08-07T15:09:05.318235+0000","last_undegraded":"2020-08-07T15:09:05.318235+0000","last_fullsized":"2020-08-07T15:09:05.318235+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:50.686496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:50.686496+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"118'2888","reported_seq":"2933","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:06.702365+0000","last_change":"2020-08-07T15:08:24.557398+0000","last_active":"2020-08-07T15:09:06.702365+0000","last_peered":"2020-08-07T15:09:06.702365+0000","last_clean":"2020-08-07T15:09:06.702365+0000","last_became_active":"2020-08-07T15:08:24.557022+0000","last_became_peered":"2020-08-07T15:08:24.557022+0000","last_unstale":"2020-08-07T15:09:06.702365+0000","last_undegraded":"2020-08-07T15:09:06.702365+0000","last_fullsized":"2020-08-07T15:09:06.702365+0000","mapping_epoch":81,"log_start":"118'2882","ondisk_log_start":"118'2882","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23658496,"num_objects":361,"num_object_clones":0,"num_object_copies":1083,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2888,"num_write_kb":23104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"144","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.212806+0000","last_change":"2020-08-07T15:08:43.036067+0000","last_active":"2020-08-07T15:09:04.212806+0000","last_peered":"2020-08-07T15:09:04.212806+0000","last_clean":"2020-08-07T15:09:04.212806+0000","last_became_active":"2020-08-07T15:08:36.627426+0000","last_became_peered":"2020-08-07T15:08:36.627426+0000","last_unstale":"2020-08-07T15:09:04.212806+0000","last_undegraded":"2020-08-07T15:09:04.212806+0000","last_fullsized":"2020-08-07T15:09:04.212806+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.035987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.035987+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"112","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.177378+0000","last_change":"2020-08-07T15:08:26.317430+0000","last_active":"2020-08-07T15:09:04.177378+0000","last_peered":"2020-08-07T15:09:04.177378+0000","last_clean":"2020-08-07T15:09:04.177378+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:09:04.177378+0000","last_undegraded":"2020-08-07T15:09:04.177378+0000","last_fullsized":"2020-08-07T15:09:04.177378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:26.317351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:26.317351+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"119'3192","reported_seq":"3237","reported_epoch":"119","state":"active+clean","last_fresh":"2020-08-07T15:09:07.693635+0000","last_change":"2020-08-07T15:08:24.552570+0000","last_active":"2020-08-07T15:09:07.693635+0000","last_peered":"2020-08-07T15:09:07.693635+0000","last_clean":"2020-08-07T15:09:07.693635+0000","last_became_active":"2020-08-07T15:08:24.552105+0000","last_became_peered":"2020-08-07T15:08:24.552105+0000","last_unstale":"2020-08-07T15:09:07.693635+0000","last_undegraded":"2020-08-07T15:09:07.693635+0000","last_fullsized":"2020-08-07T15:09:07.693635+0000","mapping_epoch":81,"log_start":"119'3185","ondisk_log_start":"119'3185","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":26148864,"num_objects":399,"num_object_clones":0,"num_object_copies":1197,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3192,"num_write_kb":25536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"115","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:05.606603+0000","last_change":"2020-08-07T15:09:05.606686+0000","last_active":"2020-08-07T15:09:05.606603+0000","last_peered":"2020-08-07T15:09:05.606603+0000","last_clean":"2020-08-07T15:09:05.606603+0000","last_became_active":"2020-08-07T15:06:23.223489+0000","last_became_peered":"2020-08-07T15:06:23.223489+0000","last_unstale":"2020-08-07T15:09:05.606603+0000","last_undegraded":"2020-08-07T15:09:05.606603+0000","last_fullsized":"2020-08-07T15:09:05.606603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:05.606603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:05.606603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"160","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.211952+0000","last_change":"2020-08-07T15:08:42.069637+0000","last_active":"2020-08-07T15:09:04.211952+0000","last_peered":"2020-08-07T15:09:04.211952+0000","last_clean":"2020-08-07T15:09:04.211952+0000","last_became_active":"2020-08-07T15:08:36.628365+0000","last_became_peered":"2020-08-07T15:08:36.628365+0000","last_unstale":"2020-08-07T15:09:04.211952+0000","last_undegraded":"2020-08-07T15:09:04.211952+0000","last_fullsized":"2020-08-07T15:09:04.211952+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:42.069585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:42.069585+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"118'3176","reported_seq":"3221","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:06.671020+0000","last_change":"2020-08-07T15:08:24.550789+0000","last_active":"2020-08-07T15:09:06.671020+0000","last_peered":"2020-08-07T15:09:06.671020+0000","last_clean":"2020-08-07T15:09:06.671020+0000","last_became_active":"2020-08-07T15:08:24.550222+0000","last_became_peered":"2020-08-07T15:08:24.550222+0000","last_unstale":"2020-08-07T15:09:06.671020+0000","last_undegraded":"2020-08-07T15:09:06.671020+0000","last_fullsized":"2020-08-07T15:09:06.671020+0000","mapping_epoch":81,"log_start":"118'3169","ondisk_log_start":"118'3169","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":26017792,"num_objects":397,"num_object_clones":0,"num_object_copies":1191,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3176,"num_write_kb":25408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"112","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.177353+0000","last_change":"2020-08-07T15:08:30.339852+0000","last_active":"2020-08-07T15:09:04.177353+0000","last_peered":"2020-08-07T15:09:04.177353+0000","last_clean":"2020-08-07T15:09:04.177353+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:09:04.177353+0000","last_undegraded":"2020-08-07T15:09:04.177353+0000","last_fullsized":"2020-08-07T15:09:04.177353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:08:30.339785+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"110","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.177808+0000","last_change":"2020-08-07T15:08:38.386792+0000","last_active":"2020-08-07T15:09:04.177808+0000","last_peered":"2020-08-07T15:09:04.177808+0000","last_clean":"2020-08-07T15:09:04.177808+0000","last_became_active":"2020-08-07T15:08:36.618164+0000","last_became_peered":"2020-08-07T15:08:36.618164+0000","last_unstale":"2020-08-07T15:09:04.177808+0000","last_undegraded":"2020-08-07T15:09:04.177808+0000","last_fullsized":"2020-08-07T15:09:04.177808+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:38.386739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:38.386739+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"118'3040","reported_seq":"3085","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:06.645289+0000","last_change":"2020-08-07T15:08:24.558242+0000","last_active":"2020-08-07T15:09:06.645289+0000","last_peered":"2020-08-07T15:09:06.645289+0000","last_clean":"2020-08-07T15:09:06.645289+0000","last_became_active":"2020-08-07T15:08:24.557354+0000","last_became_peered":"2020-08-07T15:08:24.557354+0000","last_unstale":"2020-08-07T15:09:06.645289+0000","last_undegraded":"2020-08-07T15:09:06.645289+0000","last_fullsized":"2020-08-07T15:09:06.645289+0000","mapping_epoch":81,"log_start":"118'3035","ondisk_log_start":"118'3035","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24903680,"num_objects":380,"num_object_clones":0,"num_object_copies":1140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3040,"num_write_kb":24320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"109","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.212117+0000","last_change":"2020-08-07T15:08:15.887617+0000","last_active":"2020-08-07T15:09:04.212117+0000","last_peered":"2020-08-07T15:09:04.212117+0000","last_clean":"2020-08-07T15:09:04.212117+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:09:04.212117+0000","last_undegraded":"2020-08-07T15:09:04.212117+0000","last_fullsized":"2020-08-07T15:09:04.212117+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:15.887513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:15.887513+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"149","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:05.318455+0000","last_change":"2020-08-07T15:08:58.183263+0000","last_active":"2020-08-07T15:09:05.318455+0000","last_peered":"2020-08-07T15:09:05.318455+0000","last_clean":"2020-08-07T15:09:05.318455+0000","last_became_active":"2020-08-07T15:08:58.182722+0000","last_became_peered":"2020-08-07T15:08:58.182722+0000","last_unstale":"2020-08-07T15:09:05.318455+0000","last_undegraded":"2020-08-07T15:09:05.318455+0000","last_fullsized":"2020-08-07T15:09:05.318455+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.927824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.927824+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"118'2928","reported_seq":"2973","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:06.588467+0000","last_change":"2020-08-07T15:08:24.552962+0000","last_active":"2020-08-07T15:09:06.588467+0000","last_peered":"2020-08-07T15:09:06.588467+0000","last_clean":"2020-08-07T15:09:06.588467+0000","last_became_active":"2020-08-07T15:08:24.552070+0000","last_became_peered":"2020-08-07T15:08:24.552070+0000","last_unstale":"2020-08-07T15:09:06.588467+0000","last_undegraded":"2020-08-07T15:09:06.588467+0000","last_fullsized":"2020-08-07T15:09:06.588467+0000","mapping_epoch":81,"log_start":"118'2918","ondisk_log_start":"118'2918","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23986176,"num_objects":366,"num_object_clones":0,"num_object_copies":1098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2928,"num_write_kb":23424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"87","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.177304+0000","last_change":"2020-08-07T15:08:39.409159+0000","last_active":"2020-08-07T15:09:04.177304+0000","last_peered":"2020-08-07T15:09:04.177304+0000","last_clean":"2020-08-07T15:09:04.177304+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:09:04.177304+0000","last_undegraded":"2020-08-07T15:09:04.177304+0000","last_fullsized":"2020-08-07T15:09:04.177304+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:39.409063+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_clean_scrub_stamp":"2020-08-07T15:08:39.409063+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"118'3144","reported_seq":"3187","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:06.687044+0000","last_change":"2020-08-07T15:08:24.554063+0000","last_active":"2020-08-07T15:09:06.687044+0000","last_peered":"2020-08-07T15:09:06.687044+0000","last_clean":"2020-08-07T15:09:06.687044+0000","last_became_active":"2020-08-07T15:08:24.553668+0000","last_became_peered":"2020-08-07T15:08:24.553668+0000","last_unstale":"2020-08-07T15:09:06.687044+0000","last_undegraded":"2020-08-07T15:09:06.687044+0000","last_fullsized":"2020-08-07T15:09:06.687044+0000","mapping_epoch":81,"log_start":"118'3136","ondisk_log_start":"118'3136","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25755648,"num_objects":393,"num_object_clones":0,"num_object_copies":1179,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3144,"num_write_kb":25152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"84","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.177067+0000","last_change":"2020-08-07T15:07:33.177491+0000","last_active":"2020-08-07T15:09:04.177067+0000","last_peered":"2020-08-07T15:09:04.177067+0000","last_clean":"2020-08-07T15:09:04.177067+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:09:04.177067+0000","last_undegraded":"2020-08-07T15:09:04.177067+0000","last_fullsized":"2020-08-07T15:09:04.177067+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:33.177412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:33.177412+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"118'2840","reported_seq":"2885","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:06.639751+0000","last_change":"2020-08-07T15:08:46.709789+0000","last_active":"2020-08-07T15:09:06.639751+0000","last_peered":"2020-08-07T15:09:06.639751+0000","last_clean":"2020-08-07T15:09:06.639751+0000","last_became_active":"2020-08-07T15:08:46.708674+0000","last_became_peered":"2020-08-07T15:08:46.708674+0000","last_unstale":"2020-08-07T15:09:06.639751+0000","last_undegraded":"2020-08-07T15:09:06.639751+0000","last_fullsized":"2020-08-07T15:09:06.639751+0000","mapping_epoch":98,"log_start":"118'2837","ondisk_log_start":"118'2837","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23265280,"num_objects":355,"num_object_clones":0,"num_object_copies":1065,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2840,"num_write_kb":22720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"148","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.212102+0000","last_change":"2020-08-07T15:07:56.631170+0000","last_active":"2020-08-07T15:09:04.212102+0000","last_peered":"2020-08-07T15:09:04.212102+0000","last_clean":"2020-08-07T15:09:04.212102+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:09:04.212102+0000","last_undegraded":"2020-08-07T15:09:04.212102+0000","last_fullsized":"2020-08-07T15:09:04.212102+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:56.631091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:56.631091+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"118'3144","reported_seq":"3210","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:06.716112+0000","last_change":"2020-08-07T15:08:52.193226+0000","last_active":"2020-08-07T15:09:06.716112+0000","last_peered":"2020-08-07T15:09:06.716112+0000","last_clean":"2020-08-07T15:09:06.716112+0000","last_became_active":"2020-08-07T15:08:52.166336+0000","last_became_peered":"2020-08-07T15:08:52.166336+0000","last_unstale":"2020-08-07T15:09:06.716112+0000","last_undegraded":"2020-08-07T15:09:06.716112+0000","last_fullsized":"2020-08-07T15:09:06.716112+0000","mapping_epoch":104,"log_start":"118'3140","ondisk_log_start":"118'3140","created":81,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25755648,"num_objects":393,"num_object_clones":0,"num_object_copies":1179,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3144,"num_write_kb":25152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":40960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"106","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.212105+0000","last_change":"2020-08-07T15:08:51.513398+0000","last_active":"2020-08-07T15:09:04.212105+0000","last_peered":"2020-08-07T15:09:04.212105+0000","last_clean":"2020-08-07T15:09:04.212105+0000","last_became_active":"2020-08-07T15:08:41.680448+0000","last_became_peered":"2020-08-07T15:08:41.680448+0000","last_unstale":"2020-08-07T15:09:04.212105+0000","last_undegraded":"2020-08-07T15:09:04.212105+0000","last_fullsized":"2020-08-07T15:09:04.212105+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:51.513319+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.513319+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"118'3064","reported_seq":"3107","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:06.698937+0000","last_change":"2020-08-07T15:08:24.550496+0000","last_active":"2020-08-07T15:09:06.698937+0000","last_peered":"2020-08-07T15:09:06.698937+0000","last_clean":"2020-08-07T15:09:06.698937+0000","last_became_active":"2020-08-07T15:08:24.549868+0000","last_became_peered":"2020-08-07T15:08:24.549868+0000","last_unstale":"2020-08-07T15:09:06.698937+0000","last_undegraded":"2020-08-07T15:09:06.698937+0000","last_fullsized":"2020-08-07T15:09:06.698937+0000","mapping_epoch":81,"log_start":"118'3056","ondisk_log_start":"118'3056","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25100288,"num_objects":383,"num_object_clones":0,"num_object_copies":1149,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3064,"num_write_kb":24512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"119","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:05.865996+0000","last_change":"2020-08-07T15:09:05.866078+0000","last_active":"2020-08-07T15:09:05.865996+0000","last_peered":"2020-08-07T15:09:05.865996+0000","last_clean":"2020-08-07T15:09:05.865996+0000","last_became_active":"2020-08-07T15:08:41.679321+0000","last_became_peered":"2020-08-07T15:08:41.679321+0000","last_unstale":"2020-08-07T15:09:05.865996+0000","last_undegraded":"2020-08-07T15:09:05.865996+0000","last_fullsized":"2020-08-07T15:09:05.865996+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:05.865996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:05.865996+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"118'3136","reported_seq":"3824","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:06.444154+0000","last_change":"2020-08-07T15:09:04.194221+0000","last_active":"2020-08-07T15:09:06.444154+0000","last_peered":"2020-08-07T15:09:06.444154+0000","last_clean":"2020-08-07T15:09:06.444154+0000","last_became_active":"2020-08-07T15:09:04.180570+0000","last_became_peered":"2020-08-07T15:09:04.180570+0000","last_unstale":"2020-08-07T15:09:06.444154+0000","last_undegraded":"2020-08-07T15:09:06.444154+0000","last_fullsized":"2020-08-07T15:09:06.444154+0000","mapping_epoch":116,"log_start":"118'3127","ondisk_log_start":"118'3127","created":81,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25690112,"num_objects":392,"num_object_clones":0,"num_object_copies":1176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3136,"num_write_kb":25088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":20545536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"148","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:05.318391+0000","last_change":"2020-08-07T15:08:41.689973+0000","last_active":"2020-08-07T15:09:05.318391+0000","last_peered":"2020-08-07T15:09:05.318391+0000","last_clean":"2020-08-07T15:09:05.318391+0000","last_became_active":"2020-08-07T15:08:41.689435+0000","last_became_peered":"2020-08-07T15:08:41.689435+0000","last_unstale":"2020-08-07T15:09:05.318391+0000","last_undegraded":"2020-08-07T15:09:05.318391+0000","last_fullsized":"2020-08-07T15:09:05.318391+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.927824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.927824+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"119'3512","reported_seq":"3558","reported_epoch":"119","state":"active+clean","last_fresh":"2020-08-07T15:09:07.416147+0000","last_change":"2020-08-07T15:08:24.554472+0000","last_active":"2020-08-07T15:09:07.416147+0000","last_peered":"2020-08-07T15:09:07.416147+0000","last_clean":"2020-08-07T15:09:07.416147+0000","last_became_active":"2020-08-07T15:08:24.554021+0000","last_became_peered":"2020-08-07T15:08:24.554021+0000","last_unstale":"2020-08-07T15:09:07.416147+0000","last_undegraded":"2020-08-07T15:09:07.416147+0000","last_fullsized":"2020-08-07T15:09:07.416147+0000","mapping_epoch":81,"log_start":"119'3503","ondisk_log_start":"119'3503","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":28770304,"num_objects":439,"num_object_clones":0,"num_object_copies":1317,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3512,"num_write_kb":28096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"34","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.212512+0000","last_change":"2020-08-07T15:08:46.710556+0000","last_active":"2020-08-07T15:09:04.212512+0000","last_peered":"2020-08-07T15:09:04.212512+0000","last_clean":"2020-08-07T15:09:04.212512+0000","last_became_active":"2020-08-07T15:08:46.710059+0000","last_became_peered":"2020-08-07T15:08:46.710059+0000","last_unstale":"2020-08-07T15:09:04.212512+0000","last_undegraded":"2020-08-07T15:09:04.212512+0000","last_fullsized":"2020-08-07T15:09:04.212512+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:09.841428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:09.841428+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"118'2904","reported_seq":"2949","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:06.318633+0000","last_change":"2020-08-07T15:08:24.552802+0000","last_active":"2020-08-07T15:09:06.318633+0000","last_peered":"2020-08-07T15:09:06.318633+0000","last_clean":"2020-08-07T15:09:06.318633+0000","last_became_active":"2020-08-07T15:08:24.552127+0000","last_became_peered":"2020-08-07T15:08:24.552127+0000","last_unstale":"2020-08-07T15:09:06.318633+0000","last_undegraded":"2020-08-07T15:09:06.318633+0000","last_fullsized":"2020-08-07T15:09:06.318633+0000","mapping_epoch":81,"log_start":"118'2896","ondisk_log_start":"118'2896","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23789568,"num_objects":363,"num_object_clones":0,"num_object_copies":1089,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2904,"num_write_kb":23232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"119'3040","reported_seq":"3087","reported_epoch":"119","state":"active+clean","last_fresh":"2020-08-07T15:09:07.645818+0000","last_change":"2020-08-07T15:08:46.710643+0000","last_active":"2020-08-07T15:09:07.645818+0000","last_peered":"2020-08-07T15:09:07.645818+0000","last_clean":"2020-08-07T15:09:07.645818+0000","last_became_active":"2020-08-07T15:08:46.710100+0000","last_became_peered":"2020-08-07T15:08:46.710100+0000","last_unstale":"2020-08-07T15:09:07.645818+0000","last_undegraded":"2020-08-07T15:09:07.645818+0000","last_fullsized":"2020-08-07T15:09:07.645818+0000","mapping_epoch":98,"log_start":"119'3031","ondisk_log_start":"119'3031","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24903680,"num_objects":380,"num_object_clones":0,"num_object_copies":1140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3040,"num_write_kb":24320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"56","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.177238+0000","last_change":"2020-08-07T15:08:41.678395+0000","last_active":"2020-08-07T15:09:04.177238+0000","last_peered":"2020-08-07T15:09:04.177238+0000","last_clean":"2020-08-07T15:09:04.177238+0000","last_became_active":"2020-08-07T15:08:41.677944+0000","last_became_peered":"2020-08-07T15:08:41.677944+0000","last_unstale":"2020-08-07T15:09:04.177238+0000","last_undegraded":"2020-08-07T15:09:04.177238+0000","last_fullsized":"2020-08-07T15:09:04.177238+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:02.901115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:02.901115+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":402391040,"num_objects":6140,"num_object_clones":0,"num_object_copies":18420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49120,"num_write_kb":392960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":636,"num_bytes_recovered":41558016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1233715200,"data_stored":1233715200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":63,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542180,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1201988,"kb_used_data":153404,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93169852,"statfs":{"total":96636764160,"available":95405928448,"internally_reserved":1073741824,"allocated":157085696,"data_stored":156408902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,7,0,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":0.95099999999999996,"15min":0.91200000000000003},"min":{"1min":0.48199999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":3.4159999999999999,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":2.1739999999999999},{"interface":"front","average":{"1min":1.0940000000000001,"5min":0.98899999999999999,"15min":0.97099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.3559999999999999,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.90800000000000003}]},{"osd":1,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3069999999999999,"15min":1.2989999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":4.4829999999999997,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.044,"15min":0.99199999999999999},"min":{"1min":0.61799999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":3.4620000000000002,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":2.206}]},{"osd":2,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.179,"15min":1.1439999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":4.423,"5min":4.423,"15min":4.423},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.5,"5min":1.194,"15min":1.143},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":3.5099999999999998,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":3.4260000000000002}]},{"osd":3,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.5900000000000001,"15min":1.6439999999999999},"min":{"1min":0.73799999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.5430000000000001,"5min":5.5019999999999998,"15min":5.5019999999999998},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.329,"15min":1.3380000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.4590000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":1.1140000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.7849999999999999,"5min":1.377,"15min":1.3089999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":4.835,"5min":4.835,"15min":4.835},"last":4.5250000000000004},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.3300000000000001,"15min":1.3180000000000001},"min":{"1min":0.76900000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.4470000000000001,"5min":2.778,"15min":2.778},"last":2.399}]},{"osd":5,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":2.3940000000000001,"5min":30.484000000000002,"15min":35.165999999999997},"min":{"1min":0.96799999999999997,"5min":0.875,"15min":0.875},"max":{"1min":7.4429999999999996,"5min":831.88099999999997,"15min":831.88099999999997},"last":6.149},{"interface":"front","average":{"1min":2.456,"5min":1.6870000000000001,"15min":1.5589999999999999},"min":{"1min":1,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":7.5350000000000001,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":4.6699999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.542,"15min":1.526},"min":{"1min":0.83599999999999997,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.9900000000000002,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":1.631},{"interface":"front","average":{"1min":1.9219999999999999,"5min":1.5900000000000001,"15min":1.534},"min":{"1min":1.115,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":4.5170000000000003,"5min":4.5170000000000003,"15min":4.5170000000000003},"last":1.7310000000000001}]}]},{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1190200,"kb_used_data":141616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93181640,"statfs":{"total":96636764160,"available":95417999360,"internally_reserved":1073741824,"allocated":145014784,"data_stored":144345527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.2490000000000001,"15min":1.3049999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.6439999999999999,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.135,"15min":1.145},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":2.9950000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":0.84099999999999997}]},{"osd":1,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.95999999999999996,"15min":0.95799999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.9890000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":19.103999999999999,"5min":4.9299999999999997,"15min":2.5680000000000001},"min":{"1min":0.75600000000000001,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":372.50799999999998,"5min":372.50799999999998,"15min":372.50799999999998},"last":0.92800000000000005}]},{"osd":2,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2,"15min":1.196},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":3.036,"5min":3.036,"15min":3.036},"last":0.502},{"interface":"front","average":{"1min":1.093,"5min":1.236,"15min":1.26},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.7440000000000002,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":6.0490000000000004}]},{"osd":3,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.4870000000000001,"15min":1.5129999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":3.46,"5min":3.46,"15min":3.46},"last":0.878},{"interface":"front","average":{"1min":1.268,"5min":1.3640000000000001,"15min":1.3799999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":3.1970000000000001,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.72299999999999998}]},{"osd":4,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.5880000000000001,"15min":1.6379999999999999},"min":{"1min":0.72499999999999998,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.7999999999999998,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.653,"5min":1.6679999999999999,"15min":1.6699999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":3.694,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.2629999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.3260000000000001,"15min":1.351},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.6360000000000001,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.393,"5min":1.6910000000000001,"15min":1.7410000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.8279999999999998,"5min":5.327,"15min":5.327},"last":1.2130000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.446,"15min":1.4370000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.7799999999999998,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.232},{"interface":"front","average":{"1min":1.4730000000000001,"5min":14.955,"15min":17.202000000000002},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":3.7309999999999999,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.2470000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1301376,"kb_used_data":252792,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93070464,"statfs":{"total":96636764160,"available":95304155136,"internally_reserved":1073741824,"allocated":258859008,"data_stored":258189751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,2,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":27.559999999999999,"15min":31.977},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.1820000000000004,"5min":497.04000000000002,"15min":497.04000000000002},"last":0.96999999999999997},{"interface":"front","average":{"1min":44.500999999999998,"5min":9.8049999999999997,"15min":4.0220000000000002},"min":{"1min":0.64200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":993.84500000000003,"5min":993.84500000000003,"15min":993.84500000000003},"last":0.59299999999999997}]},{"osd":2,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":0.98799999999999999,"15min":0.92800000000000005},"min":{"1min":0.57399999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":4.585,"5min":4.585,"15min":4.585},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.2410000000000001,"15min":1.216},"min":{"1min":0.51800000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":4.149,"5min":4.149,"15min":4.149},"last":0.77400000000000002}]},{"osd":3,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.962,"5min":1.4410000000000001,"15min":1.3540000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":10.603999999999999,"5min":10.603999999999999,"15min":10.603999999999999},"last":1.357},{"interface":"front","average":{"1min":2.1909999999999998,"5min":1.601,"15min":1.5029999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":10.638999999999999,"5min":10.638999999999999,"15min":10.638999999999999},"last":1.4059999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":2.1869999999999998,"5min":1.464,"15min":1.3440000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":10.69,"5min":10.69,"15min":10.69},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.264,"15min":1.2210000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":4.9500000000000002,"5min":4.9500000000000002,"15min":4.9500000000000002},"last":0.92100000000000004}]},{"osd":5,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":2.0630000000000002,"5min":10.819000000000001,"15min":12.278},"min":{"1min":0.73399999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":10.720000000000001,"5min":213.63800000000001,"15min":213.63800000000001},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.258,"15min":1.212},"min":{"1min":0.82899999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":4.6269999999999998,"5min":4.6269999999999998,"15min":4.6269999999999998},"last":1.0800000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:08:24 2020","interfaces":[{"interface":"back","average":{"1min":1.7589999999999999,"5min":1.593,"15min":1.5649999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":4.5259999999999998,"5min":4.5259999999999998,"15min":4.5259999999999998},"last":1.1919999999999999},{"interface":"front","average":{"1min":2.3690000000000002,"5min":1.5620000000000001,"15min":1.4279999999999999},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":10.792,"5min":10.792,"15min":10.792},"last":207.137}]},{"osd":7,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.8129999999999999,"5min":1.6539999999999999,"15min":1.6279999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":4.5549999999999997,"5min":4.5549999999999997,"15min":4.5549999999999997},"last":0.94099999999999995},{"interface":"front","average":{"1min":2.2719999999999998,"5min":1.6379999999999999,"15min":1.532},"min":{"1min":0.998,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":10.752000000000001,"5min":10.752000000000001,"15min":10.752000000000001},"last":1.4630000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1168944,"kb_used_data":120360,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93202896,"statfs":{"total":96636764160,"available":95439765504,"internally_reserved":1073741824,"allocated":123248640,"data_stored":122579383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:08:24 2020","interfaces":[{"interface":"back","average":{"1min":81.266000000000005,"5min":17.033999999999999,"15min":6.3289999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":845.57100000000003,"5min":845.57100000000003,"15min":845.57100000000003},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.083,"5min":0.96299999999999997,"15min":0.94299999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":5.5970000000000004,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":4.8209999999999997}]},{"osd":2,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.0069999999999999,"15min":0.95399999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":4.1600000000000001,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.0049999999999999,"5min":26.393000000000001,"15min":30.623999999999999},"min":{"1min":0.54200000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.1280000000000001,"5min":606.18499999999995,"15min":606.18499999999995},"last":4.7889999999999997}]},{"osd":3,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.0549999999999999,"15min":0.97499999999999998},"min":{"1min":0.64000000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":3.1779999999999999,"5min":3.1779999999999999,"15min":3.1779999999999999},"last":1.087},{"interface":"front","average":{"1min":1.927,"5min":1.319,"15min":1.218},"min":{"1min":0.78700000000000003,"5min":0.621,"15min":0.621},"max":{"1min":4.21,"5min":4.21,"15min":4.21},"last":1.0389999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.238,"15min":1.1899999999999999},"min":{"1min":0.73899999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":3.0870000000000002,"5min":3.0870000000000002,"15min":3.0870000000000002},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.6890000000000001,"5min":1.3440000000000001,"15min":1.286},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":4.0960000000000001,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.97699999999999998}]},{"osd":5,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":39.531999999999996,"5min":8.8879999999999999,"15min":3.7799999999999998},"min":{"1min":0.76600000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":501.267,"5min":501.267,"15min":501.267},"last":4.6470000000000002},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.24,"15min":1.1499999999999999},"min":{"1min":0.874,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":6.0010000000000003,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":1.0620000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.7729999999999999,"5min":7.2450000000000001,"15min":8.157},"min":{"1min":0.81100000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":4.1310000000000002,"5min":124.738,"15min":124.738},"last":1.264},{"interface":"front","average":{"1min":1.47,"5min":15.215999999999999,"15min":17.507000000000001},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":3.7120000000000002,"5min":294.94,"15min":294.94},"last":4.6879999999999997}]},{"osd":7,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.552,"15min":1.5740000000000001},"min":{"1min":0.96199999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.3929999999999998,"5min":6.468,"15min":6.468},"last":4.7169999999999996},{"interface":"front","average":{"1min":1.6879999999999999,"5min":1.446,"15min":1.4059999999999999},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":3.117,"5min":3.117,"15min":3.117},"last":1.135}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1119384,"kb_used_data":70800,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93252456,"statfs":{"total":96636764160,"available":95490514944,"internally_reserved":1073741824,"allocated":72499200,"data_stored":71829943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.153,"15min":1.129},"min":{"1min":0.505,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.4909999999999997,"5min":4.4909999999999997,"15min":4.4909999999999997},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.284,"5min":1.036,"15min":0.995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":3.6520000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":2.9470000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":2.5219999999999998,"5min":1.3779999999999999,"15min":1.1870000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":18.672000000000001,"5min":18.672000000000001,"15min":18.672000000000001},"last":4.1660000000000004},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.357,"15min":1.359},"min":{"1min":0.70699999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":5.1130000000000004,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":0.80700000000000005}]},{"osd":3,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.216,"15min":1.1799999999999999},"min":{"1min":0.65300000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":2.8740000000000001},{"interface":"front","average":{"1min":1.419,"5min":1.528,"15min":1.546},"min":{"1min":0.59999999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":5.3339999999999996,"5min":8.2210000000000001,"15min":8.2210000000000001},"last":0.76700000000000002}]},{"osd":4,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":2.3319999999999999,"5min":1.3540000000000001,"15min":1.1910000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":19.783000000000001,"5min":19.783000000000001,"15min":19.783000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.5589999999999999,"15min":1.615},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.891,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":0.97799999999999998}]},{"osd":5,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.2789999999999999,"15min":1.2669999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.7730000000000001,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":2.323},{"interface":"front","average":{"1min":1.5249999999999999,"5min":20.725000000000001,"15min":23.925000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":4.4790000000000001,"5min":415.40699999999998,"15min":415.40699999999998},"last":2.9140000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:08:25 2020","interfaces":[{"interface":"back","average":{"1min":2.5089999999999999,"5min":1.478,"15min":1.306},"min":{"1min":0.89000000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":20.035,"5min":20.035,"15min":20.035},"last":1.4219999999999999},{"interface":"front","average":{"1min":44.033000000000001,"5min":9.8629999999999995,"15min":4.1680000000000001},"min":{"1min":0.93700000000000006,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":751.70000000000005,"5min":751.70000000000005,"15min":751.70000000000005},"last":2.976}]},{"osd":7,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":3.5409999999999999,"5min":2.101,"15min":1.861},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":35.548999999999999,"5min":35.548999999999999,"15min":35.548999999999999},"last":1.5529999999999999},{"interface":"front","average":{"1min":3.4460000000000002,"5min":1.8859999999999999,"15min":1.6259999999999999},"min":{"1min":1.004,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":35.646999999999998,"5min":35.646999999999998,"15min":35.646999999999998},"last":3.0030000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1171152,"kb_used_data":122568,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93200688,"statfs":{"total":96636764160,"available":95437504512,"internally_reserved":1073741824,"allocated":125509632,"data_stored":124840375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.044,"5min":1.411,"15min":1.3049999999999999},"min":{"1min":0.55600000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":8.6020000000000003,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":1.8500000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.599,"15min":1.6240000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":4.9119999999999999,"5min":13.067,"15min":13.067},"last":0.74099999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":16.382999999999999,"5min":4.7949999999999999,"15min":2.8639999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":270.68700000000001,"5min":270.68700000000001,"15min":270.68700000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":2.7210000000000001,"5min":2.153,"15min":2.0590000000000002},"min":{"1min":0.71599999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":14.349,"5min":14.349,"15min":14.349},"last":0.69199999999999995}]},{"osd":2,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":1.7470000000000001,"5min":1.2669999999999999,"15min":1.1870000000000001},"min":{"1min":0.64900000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":4.9580000000000002,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":1.198},{"interface":"front","average":{"1min":4.5220000000000002,"5min":2.2519999999999998,"15min":1.8740000000000001},"min":{"1min":0.72399999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":51.530000000000001,"5min":51.530000000000001,"15min":51.530000000000001},"last":1.526}]},{"osd":4,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":47.652999999999999,"5min":10.888,"15min":4.7599999999999998},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":506.21300000000002,"5min":506.21300000000002,"15min":506.21300000000002},"last":1.802},{"interface":"front","average":{"1min":29.771000000000001,"5min":31.634,"15min":31.943999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":505.738,"5min":677.88599999999997,"15min":677.88599999999997},"last":1.1970000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.012,"5min":1.6160000000000001,"15min":1.55},"min":{"1min":0.96399999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":5.0049999999999999,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.23},{"interface":"front","average":{"1min":2.2210000000000001,"5min":1.859,"15min":1.7989999999999999},"min":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":5.226,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":1.593}]},{"osd":6,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":1.99,"5min":1.623,"15min":1.5620000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":5.0389999999999997,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.9710000000000001,"5min":1.9830000000000001,"15min":1.9850000000000001},"min":{"1min":1.0369999999999999,"5min":0.876,"15min":0.876},"max":{"1min":4.266,"5min":5.6189999999999998,"15min":5.6189999999999998},"last":2.0499999999999998}]},{"osd":7,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.0110000000000001,"5min":2.1309999999999998,"15min":2.1509999999999998},"min":{"1min":0.96999999999999997,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":3.79,"5min":5.734,"15min":5.734},"last":2.0110000000000001},{"interface":"front","average":{"1min":2.6200000000000001,"5min":2.105,"15min":2.0190000000000001},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":8.6790000000000003,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":1.9239999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1240144,"kb_used_data":191560,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93131696,"statfs":{"total":96636764160,"available":95366856704,"internally_reserved":1073741824,"allocated":196157440,"data_stored":195488183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":1.125,"15min":1.1619999999999999},"min":{"1min":0.47699999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.3999999999999999,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":0.57999999999999996},{"interface":"front","average":{"1min":1.349,"5min":0.94899999999999995,"15min":0.88200000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":6.3970000000000002,"5min":6.3970000000000002,"15min":6.3970000000000002},"last":0.77400000000000002}]},{"osd":1,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.304,"15min":1.337},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.4380000000000002,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":0.61299999999999999},{"interface":"front","average":{"1min":32.585999999999999,"5min":7.6180000000000003,"15min":3.4569999999999999},"min":{"1min":0.67000000000000004,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":689.846,"5min":689.846,"15min":689.846},"last":0.873}]},{"osd":2,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.46,"15min":1.454},"min":{"1min":0.56399999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":6.8730000000000002,"5min":6.8730000000000002,"15min":6.8730000000000002},"last":2.0899999999999999},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.5669999999999999,"15min":1.532},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":7.1360000000000001,"5min":7.1360000000000001,"15min":7.1360000000000001},"last":2.3159999999999998}]},{"osd":3,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":12.619999999999999,"5min":3.73,"15min":2.2480000000000002},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":247.202,"5min":247.202,"15min":247.202},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.583,"5min":1.3169999999999999,"15min":1.2729999999999999},"min":{"1min":0.67800000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":6.431,"5min":6.431,"15min":6.431},"last":0.72299999999999998}]},{"osd":5,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":46.786999999999999,"5min":10.659000000000001,"15min":4.6379999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":990.71600000000001,"5min":990.71600000000001,"15min":990.71600000000001},"last":2.431},{"interface":"front","average":{"1min":1.748,"5min":1.651,"15min":1.635},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":3.9329999999999998,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":0.90900000000000003}]},{"osd":6,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.7070000000000001,"5min":1.7729999999999999,"15min":1.784},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":6.9749999999999996,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.524,"5min":15.592000000000001,"15min":17.936},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":6.7729999999999997,"5min":389.14600000000002,"15min":389.14600000000002},"last":0.99299999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":13.378,"5min":3.802,"15min":2.206},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":252.798,"5min":252.798,"15min":252.798},"last":0.753},{"interface":"front","average":{"1min":44.676000000000002,"5min":10.252000000000001,"15min":4.5140000000000002},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":501.93000000000001,"5min":501.93000000000001,"15min":501.93000000000001},"last":2.0529999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1194600,"kb_used_data":146016,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93177240,"statfs":{"total":96636764160,"available":95413493760,"internally_reserved":1073741824,"allocated":149520384,"data_stored":148851127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0580000000000001,"15min":1.0700000000000001},"min":{"1min":0.49199999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.5169999999999999,"5min":2.766,"15min":2.766},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.2250000000000001,"5min":0.88900000000000001,"15min":0.83299999999999996},"min":{"1min":0.54900000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":2.3340000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":47.365000000000002,"5min":10.215999999999999,"15min":4.024},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":787.22900000000004,"5min":787.22900000000004,"15min":787.22900000000004},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.071,"5min":1.1000000000000001,"15min":1.105},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.8380000000000001,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":1.2629999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":7.5659999999999998,"15min":8.6679999999999993},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.4950000000000001,"5min":168.56899999999999,"15min":168.56899999999999},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.2929999999999999,"5min":26.763000000000002,"15min":31.007999999999999},"min":{"1min":0.79700000000000004,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":3.2389999999999999,"5min":676.27099999999996,"15min":676.27099999999996},"last":2.665}]},{"osd":3,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.196,"15min":1.1439999999999999},"min":{"1min":0.89000000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":3.5649999999999999,"5min":3.5649999999999999,"15min":3.5649999999999999},"last":2.7829999999999999},{"interface":"front","average":{"1min":43.073,"5min":9.5850000000000009,"15min":4.0030000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":712.98699999999997,"5min":712.98699999999997,"15min":712.98699999999997},"last":1.2889999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":40.962000000000003,"15min":47.582000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.4980000000000002,"5min":530.30799999999999,"15min":530.30799999999999},"last":1.9239999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.278,"15min":1.296},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.6259999999999999,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":1.819}]},{"osd":6,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":40.569000000000003,"15min":47.128999999999998},"min":{"1min":0.76800000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.6699999999999999,"5min":754.31299999999999,"15min":754.31299999999999},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":21.050999999999998,"15min":24.361000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.7809999999999999,"5min":517.26700000000005,"15min":517.26700000000005},"last":1.5900000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.3560000000000001,"15min":1.3240000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":3.9079999999999999,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":2.8940000000000001},{"interface":"front","average":{"1min":1.411,"5min":1.5349999999999999,"15min":1.556},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":3.9390000000000001,"5min":3.9390000000000001,"15min":3.9390000000000001},"last":1.7070000000000001}]}]}]}} 2020-08-07T15:09:08.910 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:08.908+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:08.998 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:08.996+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:09.064 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:09.065 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:09.079 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:09:09.125 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:09.119+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:09.209 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:09.198+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:09.278 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 43 16 52863 52847 9.60065 12.9688 0.0143126 0.0130112 2020-08-07T15:09:09.286 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:09.287 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:09.299 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:09:09.300 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:09.299+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:09.405 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:09.400+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:09.470 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:09.471 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:09.492 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:09:09.502 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:09.501+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:09.610 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:09.604+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:09.705 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:09.704+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:09.715 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:09.715 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:09.728 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:09:09.806 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:09.805+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:09.911 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:09.909+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:09.933 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:09.934 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:09.947 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:09:10.014 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:10.011+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:10.114 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:10.112+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:10.122 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:10.123 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:10.141 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:09:10.234 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:10.229+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:10.278 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 44 16 54407 54391 9.65658 12.0625 0.00969959 0.012941 2020-08-07T15:09:10.324 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:10.321+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:10.350 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:09:10.375 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:09:10.378 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:10.378 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:10.389 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:09:10.424 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:10.423+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:10.538 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:10.527+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:10.597 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:09:10.646 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:10.633+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:10.650 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:09:10.661 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:09:10.664 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:09:10.664 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:09:10.665 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:09:10.665 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.665 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:10240 3.6 3:665d71ff:::benchmark_data_smithi074_35447_object1279:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:09:10.665 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:09:10.666 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 148.26261145999999, 2020-08-07T15:09:10.666 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25191569699999999, 2020-08-07T15:09:10.667 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:10.667 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:10.667 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:10.668 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:09:10.668 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:10.668 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 10240 2020-08-07T15:09:10.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.668 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:10.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:10.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:09:10.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:10.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.670 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.670 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:10.670 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:09:10.670 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5463000000000002e-05 2020-08-07T15:09:10.670 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.671 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.671 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:10.671 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393264+0000", 2020-08-07T15:09:10.671 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9219999999999999e-06 2020-08-07T15:09:10.671 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.672 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.672 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:10.672 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393268+0000", 2020-08-07T15:09:10.672 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6149999999999997e-06 2020-08-07T15:09:10.672 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.672 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.673 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:10.673 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393277+0000", 2020-08-07T15:09:10.673 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2039999999999996e-06 2020-08-07T15:09:10.673 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.690 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.715 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:10.715 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393282+0000", 2020-08-07T15:09:10.716 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001225243 2020-08-07T15:09:10.716 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.718 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.718 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:10.718 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394507+0000", 2020-08-07T15:09:10.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7108999999999995e-05 2020-08-07T15:09:10.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:10.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394605+0000", 2020-08-07T15:09:10.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017490800000000001 2020-08-07T15:09:10.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.726 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:09:10.726 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394779+0000", 2020-08-07T15:09:10.726 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032608699999999999 2020-08-07T15:09:10.726 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.727 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.731 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:10.732 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.398040+0000", 2020-08-07T15:09:10.732 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013015259999999999 2020-08-07T15:09:10.732 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.732 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:10.733 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.399342+0000", 2020-08-07T15:09:10.733 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24566893400000001 2020-08-07T15:09:10.733 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.733 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:10.734 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645011+0000", 2020-08-07T15:09:10.734 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.217e-06 2020-08-07T15:09:10.735 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.735 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.735 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:10.735 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645012+0000", 2020-08-07T15:09:10.735 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0803000000000006e-05 2020-08-07T15:09:10.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.736 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.736 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:10.736 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645083+0000", 2020-08-07T15:09:10.736 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1882999999999998e-05 2020-08-07T15:09:10.737 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.737 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.737 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:10.737 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645135+0000", 2020-08-07T15:09:10.737 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25191569699999999 2020-08-07T15:09:10.738 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:10.738 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:10.738 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:10.738 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.738 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.739 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:43335 3.6 e32/26 3:65a57ed3:::benchmark_data_smithi074_35447_object5416:head v 32'2831, mlcod=32'2822)", 2020-08-07T15:09:10.739 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:03.998850+0000", 2020-08-07T15:09:10.739 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 126.656980389, 2020-08-07T15:09:10.739 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.221384421, 2020-08-07T15:09:10.739 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:10.740 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:10.740 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:10.740 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.740 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:10.740 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998850+0000", 2020-08-07T15:09:10.741 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:10.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.741 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.761 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:10.761 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998850+0000", 2020-08-07T15:09:10.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4788000000000004e-05 2020-08-07T15:09:10.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.765 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:10.765 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998915+0000", 2020-08-07T15:09:10.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7909999999999999e-06 2020-08-07T15:09:10.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:10.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.998922+0000", 2020-08-07T15:09:10.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22050499700000001 2020-08-07T15:09:10.798 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.798 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.798 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:10.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219427+0000", 2020-08-07T15:09:10.799 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0950000000000001e-06 2020-08-07T15:09:10.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:10.800 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219433+0000", 2020-08-07T15:09:10.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6342999999999997e-05 2020-08-07T15:09:10.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:10.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219509+0000", 2020-08-07T15:09:10.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4173999999999999e-05 2020-08-07T15:09:10.801 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.801 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.814 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:10.814 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.219533+0000", 2020-08-07T15:09:10.814 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00064214899999999995 2020-08-07T15:09:10.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.817 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:10.818 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.220175+0000", 2020-08-07T15:09:10.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9083999999999999e-05 2020-08-07T15:09:10.818 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:10.819 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.220234+0000", 2020-08-07T15:09:10.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.221384421 2020-08-07T15:09:10.819 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:10.820 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:10.820 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:10.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.820 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:43336 3.6 e32/26 3:65a57ed3:::benchmark_data_smithi074_35447_object5416:head v 32'2832, mlcod=32'2822)", 2020-08-07T15:09:10.821 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:03.999509+0000", 2020-08-07T15:09:10.821 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 126.65632106, 2020-08-07T15:09:10.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22632864799999999, 2020-08-07T15:09:10.821 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:10.821 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:10.827 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:10.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:10.828 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999509+0000", 2020-08-07T15:09:10.828 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:10.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.829 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:10.829 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999509+0000", 2020-08-07T15:09:10.829 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0107e-05 2020-08-07T15:09:10.829 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.829 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:10.841 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999559+0000", 2020-08-07T15:09:10.841 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0889999999999996e-06 2020-08-07T15:09:10.841 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.842 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.842 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:10.842 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:03.999566+0000", 2020-08-07T15:09:10.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22332094599999999 2020-08-07T15:09:10.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.843 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:10.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222887+0000", 2020-08-07T15:09:10.844 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5990000000000001e-06 2020-08-07T15:09:10.854 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.854 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.854 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:10.854 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222892+0000", 2020-08-07T15:09:10.855 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2342999999999998e-05 2020-08-07T15:09:10.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.855 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:10.855 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222924+0000", 2020-08-07T15:09:10.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9108999999999999e-05 2020-08-07T15:09:10.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.856 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.874 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:10.875 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.222944+0000", 2020-08-07T15:09:10.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028409289999999999 2020-08-07T15:09:10.876 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.878 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.883 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:10.883 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.225784+0000", 2020-08-07T15:09:10.883 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3526000000000002e-05 2020-08-07T15:09:10.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.884 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.884 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:10.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:04.225838+0000", 2020-08-07T15:09:10.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22632864799999999 2020-08-07T15:09:10.885 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:10.885 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:10.885 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:10.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.886 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:54200 3.3 e37/20 3:cc40637e:::benchmark_data_smithi074_35447_object6774:head v 37'3248, mlcod=37'3236)", 2020-08-07T15:09:10.886 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:09:10.886 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 117.55726824, 2020-08-07T15:09:10.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24065872499999999, 2020-08-07T15:09:10.886 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:10.887 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:10.887 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:10.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:10.887 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:09:10.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:10.892 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.892 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.892 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:10.893 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:09:10.893 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1671000000000002e-05 2020-08-07T15:09:10.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.893 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:10.903 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098594+0000", 2020-08-07T15:09:10.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0890000000000002e-06 2020-08-07T15:09:10.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.904 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:10.904 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098598+0000", 2020-08-07T15:09:10.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23875592300000001 2020-08-07T15:09:10.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:10.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337354+0000", 2020-08-07T15:09:10.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0420000000000002e-06 2020-08-07T15:09:10.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:10.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337359+0000", 2020-08-07T15:09:10.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0586000000000001e-05 2020-08-07T15:09:10.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:10.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337399+0000", 2020-08-07T15:09:10.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9415999999999998e-05 2020-08-07T15:09:10.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:10.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337419+0000", 2020-08-07T15:09:10.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017568390000000001 2020-08-07T15:09:10.918 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.918 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:10.919 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.339176+0000", 2020-08-07T15:09:10.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5158999999999999e-05 2020-08-07T15:09:10.919 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.919 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:10.920 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.339221+0000", 2020-08-07T15:09:10.920 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24065872499999999 2020-08-07T15:09:10.920 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:10.920 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:10.920 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:10.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.921 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:71136 3.6 e43/26 3:64cb7acb:::benchmark_data_smithi074_35447_object8891:head v 43'4744, mlcod=43'4739)", 2020-08-07T15:09:10.921 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:09:10.921 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 106.33721687400001, 2020-08-07T15:09:10.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26135482999999998, 2020-08-07T15:09:10.922 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:10.922 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:10.923 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:10.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:10.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:09:10.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:10.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:10.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:09:10.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3702999999999997e-05 2020-08-07T15:09:10.925 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:10.926 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318667+0000", 2020-08-07T15:09:10.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3119999999999999e-06 2020-08-07T15:09:10.926 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.926 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:10.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318674+0000", 2020-08-07T15:09:10.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.255173292 2020-08-07T15:09:10.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:10.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.573847+0000", 2020-08-07T15:09:10.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1140000000000002e-06 2020-08-07T15:09:10.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:10.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.573853+0000", 2020-08-07T15:09:10.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00057603799999999996 2020-08-07T15:09:10.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:10.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.574429+0000", 2020-08-07T15:09:10.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4776e-05 2020-08-07T15:09:10.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:10.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.574444+0000", 2020-08-07T15:09:10.931 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054577080000000004 2020-08-07T15:09:10.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:10.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.579901+0000", 2020-08-07T15:09:10.932 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6886999999999998e-05 2020-08-07T15:09:10.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.932 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.932 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:10.933 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.579968+0000", 2020-08-07T15:09:10.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26135482999999998 2020-08-07T15:09:10.933 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:10.933 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:10.933 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:10.934 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.934 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.935 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:85696 3.f e51/20 3:f99fcea4:::benchmark_data_smithi074_35447_object10711:head v 51'5688, mlcod=51'5679)", 2020-08-07T15:09:10.935 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:09:10.935 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 97.003225461, 2020-08-07T15:09:10.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49655717300000002, 2020-08-07T15:09:10.936 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:10.936 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:10.936 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:10.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:10.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:09:10.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:10.937 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:10.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:09:10.946 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9543000000000002e-05 2020-08-07T15:09:10.946 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.946 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.946 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:10.947 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652665+0000", 2020-08-07T15:09:10.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8700000000000003e-06 2020-08-07T15:09:10.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.947 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.947 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:10.948 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652671+0000", 2020-08-07T15:09:10.948 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49579595799999998 2020-08-07T15:09:10.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.948 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:10.949 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148467+0000", 2020-08-07T15:09:10.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2509999999999997e-06 2020-08-07T15:09:10.949 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.949 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:10.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148475+0000", 2020-08-07T15:09:10.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5603999999999998e-05 2020-08-07T15:09:10.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:10.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148530+0000", 2020-08-07T15:09:10.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5486999999999998e-05 2020-08-07T15:09:10.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.975 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:10.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148556+0000", 2020-08-07T15:09:10.982 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00055666600000000004 2020-08-07T15:09:10.983 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.983 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.983 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:10.983 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.149112+0000", 2020-08-07T15:09:10.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9793999999999998e-05 2020-08-07T15:09:10.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:10.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:10.984 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:10.984 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.149162+0000", 2020-08-07T15:09:10.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49655717300000002 2020-08-07T15:09:10.985 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:10.985 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:10.985 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:10.985 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.007 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:86792 3.f e52/20 3:fc5386ee:::benchmark_data_smithi074_35447_object10848:head v 52'5752, mlcod=52'5743)", 2020-08-07T15:09:11.007 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:09:11.008 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 96.253379686000002, 2020-08-07T15:09:11.008 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.295270005, 2020-08-07T15:09:11.011 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:11.011 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:11.011 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:11.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:11.020 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:09:11.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:11.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:11.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:09:11.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9712000000000001e-05 2020-08-07T15:09:11.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:11.022 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402500+0000", 2020-08-07T15:09:11.022 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6789999999999999e-06 2020-08-07T15:09:11.023 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:11.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402507+0000", 2020-08-07T15:09:11.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29317074999999998 2020-08-07T15:09:11.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:11.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695678+0000", 2020-08-07T15:09:11.038 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8120000000000002e-06 2020-08-07T15:09:11.038 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.038 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:11.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695685+0000", 2020-08-07T15:09:11.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1270999999999998e-05 2020-08-07T15:09:11.039 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.039 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.039 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:11.040 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695726+0000", 2020-08-07T15:09:11.040 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2739e-05 2020-08-07T15:09:11.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:11.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695749+0000", 2020-08-07T15:09:11.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019333899999999999 2020-08-07T15:09:11.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.041 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:11.067 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.697682+0000", 2020-08-07T15:09:11.072 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8652000000000002e-05 2020-08-07T15:09:11.073 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.081 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.082 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:11.082 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.697721+0000", 2020-08-07T15:09:11.082 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.295270005 2020-08-07T15:09:11.083 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.083 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:11.083 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.084 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.084 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:87088 3.6 e52/26 3:6b0320df:::benchmark_data_smithi074_35447_object10885:head v 52'5912, mlcod=52'5902)", 2020-08-07T15:09:11.084 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:09:11.084 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 95.689146445999995, 2020-08-07T15:09:11.084 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29522189799999998, 2020-08-07T15:09:11.084 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:11.085 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:11.085 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:11.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.085 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:11.111 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:09:11.111 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:11.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.112 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:11.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:09:11.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0751000000000002e-05 2020-08-07T15:09:11.113 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.129 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.132 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:11.133 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966735+0000", 2020-08-07T15:09:11.133 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3080000000000004e-06 2020-08-07T15:09:11.133 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.133 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.134 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:11.134 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966741+0000", 2020-08-07T15:09:11.134 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29365034699999998 2020-08-07T15:09:11.134 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.134 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.135 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:11.135 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260391+0000", 2020-08-07T15:09:11.135 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2720000000000003e-06 2020-08-07T15:09:11.136 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.136 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.137 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:11.137 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260398+0000", 2020-08-07T15:09:11.137 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8819000000000005e-05 2020-08-07T15:09:11.137 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.137 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.138 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:11.144 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260476+0000", 2020-08-07T15:09:11.145 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4958999999999999e-05 2020-08-07T15:09:11.145 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.145 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.145 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:11.145 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260501+0000", 2020-08-07T15:09:11.150 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001353641 2020-08-07T15:09:11.150 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.150 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:11.152 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.261855+0000", 2020-08-07T15:09:11.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0800999999999997e-05 2020-08-07T15:09:11.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.166 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:11.167 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.261906+0000", 2020-08-07T15:09:11.167 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29522189799999998 2020-08-07T15:09:11.167 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.167 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:11.167 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.168 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.168 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.168 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:93920 3.6 e56/26 3:631393d3:::benchmark_data_smithi074_35447_object11739:head v 56'6352, mlcod=55'6342)", 2020-08-07T15:09:11.168 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:09:11.168 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 91.329031014999998, 2020-08-07T15:09:11.169 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39765404300000001, 2020-08-07T15:09:11.169 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:11.169 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:11.169 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:11.179 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.179 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:11.179 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:09:11.183 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:11.185 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:11.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:09:11.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3035999999999998e-05 2020-08-07T15:09:11.379 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:11.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326852+0000", 2020-08-07T15:09:11.380 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5379999999999999e-06 2020-08-07T15:09:11.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:11.380 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326859+0000", 2020-08-07T15:09:11.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.393917568 2020-08-07T15:09:11.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.393 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:11.408 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.720777+0000", 2020-08-07T15:09:11.409 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2430000000000005e-06 2020-08-07T15:09:11.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.409 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.409 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:11.409 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.720785+0000", 2020-08-07T15:09:11.410 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00065658700000000003 2020-08-07T15:09:11.410 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.410 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.410 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:11.410 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.721441+0000", 2020-08-07T15:09:11.411 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8585000000000002e-05 2020-08-07T15:09:11.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.421 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.421 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:11.422 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.721470+0000", 2020-08-07T15:09:11.422 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00293349 2020-08-07T15:09:11.422 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.458 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:11.458 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.724403+0000", 2020-08-07T15:09:11.458 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9996e-05 2020-08-07T15:09:11.459 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:11.459 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.724453+0000", 2020-08-07T15:09:11.459 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39765404300000001 2020-08-07T15:09:11.460 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.460 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:11.460 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.460 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.460 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:94384 3.3 e56/20 3:ce533213:::benchmark_data_smithi074_35447_object11797:head v 56'5488, mlcod=56'5479)", 2020-08-07T15:09:11.461 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.637245+0000", 2020-08-07T15:09:11.461 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 91.018585510999998, 2020-08-07T15:09:11.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.247824459, 2020-08-07T15:09:11.461 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:11.474 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:11.475 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:11.475 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.475 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:11.475 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637245+0000", 2020-08-07T15:09:11.476 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:11.476 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.476 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.476 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:11.476 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637245+0000", 2020-08-07T15:09:11.477 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0782999999999997e-05 2020-08-07T15:09:11.477 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.477 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.477 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:11.477 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637306+0000", 2020-08-07T15:09:11.478 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8699999999999992e-06 2020-08-07T15:09:11.479 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.479 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.480 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:11.480 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637314+0000", 2020-08-07T15:09:11.480 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24469571800000001 2020-08-07T15:09:11.480 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.481 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.481 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:11.481 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882009+0000", 2020-08-07T15:09:11.481 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8510000000000004e-06 2020-08-07T15:09:11.481 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.482 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.482 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:11.482 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882015+0000", 2020-08-07T15:09:11.482 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6708999999999997e-05 2020-08-07T15:09:11.482 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.483 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.483 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:11.483 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882052+0000", 2020-08-07T15:09:11.483 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0280000000000001e-05 2020-08-07T15:09:11.483 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.484 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.484 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:11.484 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882082+0000", 2020-08-07T15:09:11.484 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002934767 2020-08-07T15:09:11.484 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.484 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.485 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:11.485 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.885017+0000", 2020-08-07T15:09:11.485 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2481000000000001e-05 2020-08-07T15:09:11.485 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.485 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.486 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:11.486 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.885069+0000", 2020-08-07T15:09:11.486 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.247824459 2020-08-07T15:09:11.486 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.486 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:11.487 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.487 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.487 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.487 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:104560 3.f e58/20 3:fb4383ae:::benchmark_data_smithi074_35447_object13069:head v 58'6888, mlcod=58'6878)", 2020-08-07T15:09:11.487 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:46.301207+0000", 2020-08-07T15:09:11.488 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 84.354623693999997, 2020-08-07T15:09:11.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23687350400000001, 2020-08-07T15:09:11.488 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:11.488 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:11.488 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:11.489 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.489 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:11.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301207+0000", 2020-08-07T15:09:11.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:11.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.489 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.490 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:11.490 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301207+0000", 2020-08-07T15:09:11.490 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7945000000000003e-05 2020-08-07T15:09:11.490 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.490 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.491 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:11.491 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301255+0000", 2020-08-07T15:09:11.491 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2030000000000002e-06 2020-08-07T15:09:11.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.491 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.492 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:11.492 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301259+0000", 2020-08-07T15:09:11.492 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23565420500000001 2020-08-07T15:09:11.492 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.492 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:11.493 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.536913+0000", 2020-08-07T15:09:11.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2290000000000003e-06 2020-08-07T15:09:11.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.493 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:11.494 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.536921+0000", 2020-08-07T15:09:11.494 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0955e-05 2020-08-07T15:09:11.494 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.494 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:11.495 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.537002+0000", 2020-08-07T15:09:11.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8977999999999999e-05 2020-08-07T15:09:11.495 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.495 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.495 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:11.496 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.537031+0000", 2020-08-07T15:09:11.496 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00099831000000000008 2020-08-07T15:09:11.496 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.496 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.496 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:11.497 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.538030+0000", 2020-08-07T15:09:11.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0679e-05 2020-08-07T15:09:11.497 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.497 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.497 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:11.497 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.538080+0000", 2020-08-07T15:09:11.498 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23687350400000001 2020-08-07T15:09:11.498 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.498 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:11.498 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.498 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.499 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.499 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:116064 3.6 e64/26 3:6cc03ffa:::benchmark_data_smithi074_35447_object14507:head v 64'7632, mlcod=64'7623)", 2020-08-07T15:09:11.499 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:09:11.499 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 74.293069611999996, 2020-08-07T15:09:11.499 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25728607799999997, 2020-08-07T15:09:11.500 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:11.500 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:11.500 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:11.500 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.500 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:11.501 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:09:11.501 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:11.501 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.501 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.501 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:11.501 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:09:11.502 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6042000000000003e-05 2020-08-07T15:09:11.502 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.502 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.502 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:11.502 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362817+0000", 2020-08-07T15:09:11.503 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9399999999999999e-06 2020-08-07T15:09:11.503 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.503 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.503 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:11.503 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362823+0000", 2020-08-07T15:09:11.504 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.254635583 2020-08-07T15:09:11.504 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.504 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.504 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:11.504 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617458+0000", 2020-08-07T15:09:11.505 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5100000000000004e-06 2020-08-07T15:09:11.505 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.505 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.505 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:11.505 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617465+0000", 2020-08-07T15:09:11.505 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4439999999999999e-05 2020-08-07T15:09:11.506 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.515 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.515 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:11.515 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617539+0000", 2020-08-07T15:09:11.515 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3527e-05 2020-08-07T15:09:11.515 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.516 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.516 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:11.516 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617563+0000", 2020-08-07T15:09:11.516 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024301069999999999 2020-08-07T15:09:11.516 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.517 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.517 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:11.517 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.619993+0000", 2020-08-07T15:09:11.517 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3928999999999998e-05 2020-08-07T15:09:11.517 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.518 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:11.518 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.620047+0000", 2020-08-07T15:09:11.518 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25728607799999997 2020-08-07T15:09:11.518 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.519 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:11.519 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.519 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.519 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.519 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:127528 3.6 e70/26 3:6d26a4f4:::benchmark_data_smithi074_35447_object15940:head v 70'8264, mlcod=70'8256)", 2020-08-07T15:09:11.520 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:04.535579+0000", 2020-08-07T15:09:11.520 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 66.120251562999997, 2020-08-07T15:09:11.520 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22956602400000001, 2020-08-07T15:09:11.520 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:11.520 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:11.521 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:11.521 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.521 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:11.521 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.535579+0000", 2020-08-07T15:09:11.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:11.522 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.522 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.522 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:11.522 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.535579+0000", 2020-08-07T15:09:11.522 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0343999999999997e-05 2020-08-07T15:09:11.522 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.523 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.523 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:11.523 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.535639+0000", 2020-08-07T15:09:11.523 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5889999999999999e-06 2020-08-07T15:09:11.523 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.524 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.524 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:11.524 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.535646+0000", 2020-08-07T15:09:11.524 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22504058599999999 2020-08-07T15:09:11.524 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.525 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.525 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:11.525 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.760686+0000", 2020-08-07T15:09:11.525 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7790000000000003e-06 2020-08-07T15:09:11.525 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.526 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.526 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:11.526 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.760691+0000", 2020-08-07T15:09:11.526 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2592999999999997e-05 2020-08-07T15:09:11.526 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.526 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.527 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:11.527 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.760734+0000", 2020-08-07T15:09:11.527 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3947999999999998e-05 2020-08-07T15:09:11.527 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.527 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.528 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:11.528 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.760758+0000", 2020-08-07T15:09:11.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043316020000000004 2020-08-07T15:09:11.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.529 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:11.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.765089+0000", 2020-08-07T15:09:11.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5583e-05 2020-08-07T15:09:11.529 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.529 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.530 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:11.530 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:04.765145+0000", 2020-08-07T15:09:11.530 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22956602400000001 2020-08-07T15:09:11.530 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.530 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:11.530 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.531 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.531 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.532 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:145801 3.6 e75/26 3:69e4cb22:::benchmark_data_smithi074_35447_object16192:head v 75'9405, mlcod=75'9401)", 2020-08-07T15:09:11.532 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:09:11.532 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 55.984074319999998, 2020-08-07T15:09:11.532 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639, 2020-08-07T15:09:11.532 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:11.532 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:11.533 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:11.533 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.533 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:11.533 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:09:11.533 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:11.534 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.534 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.534 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:11.534 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:09:11.534 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6511000000000002e-05 2020-08-07T15:09:11.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:11.535 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671803+0000", 2020-08-07T15:09:11.535 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7190000000000004e-06 2020-08-07T15:09:11.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.536 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:11.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671809+0000", 2020-08-07T15:09:11.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74210928300000001 2020-08-07T15:09:11.537 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.537 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.537 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:11.538 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413919+0000", 2020-08-07T15:09:11.538 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1759999999999999e-05 2020-08-07T15:09:11.538 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.538 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.538 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:11.539 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413930+0000", 2020-08-07T15:09:11.539 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000108601 2020-08-07T15:09:11.539 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.539 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.539 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:11.540 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414039+0000", 2020-08-07T15:09:11.540 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9948999999999999e-05 2020-08-07T15:09:11.540 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.540 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.540 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:11.541 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414089+0000", 2020-08-07T15:09:11.541 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00079656499999999995 2020-08-07T15:09:11.541 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.541 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:11.542 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414885+0000", 2020-08-07T15:09:11.544 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2250999999999998e-05 2020-08-07T15:09:11.544 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.544 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:11.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414938+0000", 2020-08-07T15:09:11.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639 2020-08-07T15:09:11.545 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.545 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:11.546 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.546 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.546 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:9112 4.6 e86/81 4:6004be0b:::benchmark_data_smithi074_44586_object1138:head v 86'544, mlcod=86'533)", 2020-08-07T15:09:11.546 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:32.296890+0000", 2020-08-07T15:09:11.547 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 38.358940910999998, 2020-08-07T15:09:11.547 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26263388100000001, 2020-08-07T15:09:11.547 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:11.547 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:11.547 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:11.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.548 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:11.548 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296890+0000", 2020-08-07T15:09:11.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:11.548 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.549 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:11.549 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296890+0000", 2020-08-07T15:09:11.549 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9907999999999997e-05 2020-08-07T15:09:11.549 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.549 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.550 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:11.550 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296939+0000", 2020-08-07T15:09:11.550 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7630000000000003e-06 2020-08-07T15:09:11.550 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.550 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.551 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:11.551 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296946+0000", 2020-08-07T15:09:11.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26058471900000002 2020-08-07T15:09:11.551 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.551 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.552 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:11.552 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557531+0000", 2020-08-07T15:09:11.552 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5270000000000004e-06 2020-08-07T15:09:11.552 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.552 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:11.553 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557537+0000", 2020-08-07T15:09:11.553 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2843000000000003e-05 2020-08-07T15:09:11.553 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.553 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.553 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:11.554 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557580+0000", 2020-08-07T15:09:11.554 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3833999999999999e-05 2020-08-07T15:09:11.554 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.554 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.554 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:11.555 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557604+0000", 2020-08-07T15:09:11.555 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001875196 2020-08-07T15:09:11.555 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.555 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.555 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:11.556 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.559479+0000", 2020-08-07T15:09:11.556 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4091000000000002e-05 2020-08-07T15:09:11.556 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:11.556 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.559523+0000", 2020-08-07T15:09:11.557 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26263388100000001 2020-08-07T15:09:11.557 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.557 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:11.557 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.557 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.558 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.558 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:13784 4.6 e88/81 4:66f64b92:::benchmark_data_smithi074_44586_object1722:head v 88'832, mlcod=88'827)", 2020-08-07T15:09:11.558 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:35.137315+0000", 2020-08-07T15:09:11.558 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 35.518515178000001, 2020-08-07T15:09:11.558 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32459016899999998, 2020-08-07T15:09:11.559 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:11.559 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:11.559 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:11.559 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.559 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:11.560 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137315+0000", 2020-08-07T15:09:11.560 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:11.560 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.560 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.560 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:11.561 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137315+0000", 2020-08-07T15:09:11.561 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0362999999999998e-05 2020-08-07T15:09:11.561 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.561 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.561 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:11.562 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137376+0000", 2020-08-07T15:09:11.562 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7020000000000004e-06 2020-08-07T15:09:11.562 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.562 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.562 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:11.563 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137382+0000", 2020-08-07T15:09:11.563 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32172605500000001 2020-08-07T15:09:11.563 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.563 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.563 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:11.564 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459108+0000", 2020-08-07T15:09:11.564 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7710000000000003e-06 2020-08-07T15:09:11.564 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.565 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:11.565 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459114+0000", 2020-08-07T15:09:11.565 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4297e-05 2020-08-07T15:09:11.565 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.566 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.567 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:11.567 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459158+0000", 2020-08-07T15:09:11.568 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5058999999999998e-05 2020-08-07T15:09:11.568 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.568 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.568 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:11.568 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459183+0000", 2020-08-07T15:09:11.569 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026624740000000002 2020-08-07T15:09:11.569 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.569 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.579 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:11.580 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.461846+0000", 2020-08-07T15:09:11.580 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9447999999999997e-05 2020-08-07T15:09:11.581 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.584 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.584 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:11.585 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.461905+0000", 2020-08-07T15:09:11.585 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32459016899999998 2020-08-07T15:09:11.585 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.585 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:11.585 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.586 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.586 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.586 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:31608 4.7 e100/81 4:e0ebdc25:::benchmark_data_smithi074_44586_object3950:head v 100'2136, mlcod=100'2126)", 2020-08-07T15:09:11.586 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:47.845848+0000", 2020-08-07T15:09:11.586 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 22.809982317999999, 2020-08-07T15:09:11.587 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27603951399999999, 2020-08-07T15:09:11.587 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:11.587 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:11.587 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:11.587 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.588 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:11.588 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:47.845848+0000", 2020-08-07T15:09:11.588 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:11.588 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.589 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:11.589 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:47.845848+0000", 2020-08-07T15:09:11.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3656999999999998e-05 2020-08-07T15:09:11.589 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.590 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:11.590 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:47.845902+0000", 2020-08-07T15:09:11.590 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4050000000000003e-06 2020-08-07T15:09:11.590 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.590 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.591 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:11.591 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:47.845909+0000", 2020-08-07T15:09:11.591 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27431144000000002 2020-08-07T15:09:11.591 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.592 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.592 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:11.592 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.120221+0000", 2020-08-07T15:09:11.592 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3640000000000004e-06 2020-08-07T15:09:11.592 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.592 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.593 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:11.593 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.120227+0000", 2020-08-07T15:09:11.593 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1541e-05 2020-08-07T15:09:11.593 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.593 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.594 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:11.594 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.120279+0000", 2020-08-07T15:09:11.594 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.372e-05 2020-08-07T15:09:11.594 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.594 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.595 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:11.595 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.120302+0000", 2020-08-07T15:09:11.595 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015294460000000001 2020-08-07T15:09:11.595 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.595 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.596 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:11.596 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.121832+0000", 2020-08-07T15:09:11.596 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5940999999999999e-05 2020-08-07T15:09:11.596 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.596 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.596 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:11.597 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.121888+0000", 2020-08-07T15:09:11.597 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27603951399999999 2020-08-07T15:09:11.597 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.597 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:11.597 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.598 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.598 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.598 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:31832 4.b e100/81 4:d005fdc5:::benchmark_data_smithi074_44586_object3978:head v 100'1872, mlcod=100'1863)", 2020-08-07T15:09:11.598 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:48.177094+0000", 2020-08-07T15:09:11.598 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 22.478736053999999, 2020-08-07T15:09:11.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26337036000000003, 2020-08-07T15:09:11.599 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:11.599 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:11.599 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:11.599 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.600 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:11.600 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.177094+0000", 2020-08-07T15:09:11.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:11.600 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.600 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.601 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:11.601 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.177094+0000", 2020-08-07T15:09:11.601 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3760999999999997e-05 2020-08-07T15:09:11.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.601 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:11.602 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.177128+0000", 2020-08-07T15:09:11.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1549999999999998e-06 2020-08-07T15:09:11.602 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.602 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.602 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:11.603 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.177133+0000", 2020-08-07T15:09:11.603 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.262460639 2020-08-07T15:09:11.603 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.603 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:11.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.439594+0000", 2020-08-07T15:09:11.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0943e-05 2020-08-07T15:09:11.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:11.605 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.439605+0000", 2020-08-07T15:09:11.605 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013696100000000001 2020-08-07T15:09:11.605 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.605 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.605 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:11.606 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.439742+0000", 2020-08-07T15:09:11.606 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5649000000000003e-05 2020-08-07T15:09:11.606 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.606 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.606 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:11.606 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.439787+0000", 2020-08-07T15:09:11.607 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00063599200000000005 2020-08-07T15:09:11.607 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.607 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.607 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:11.607 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.440423+0000", 2020-08-07T15:09:11.608 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1260000000000001e-05 2020-08-07T15:09:11.608 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.610 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.610 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:11.610 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.440465+0000", 2020-08-07T15:09:11.610 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26337036000000003 2020-08-07T15:09:11.611 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.611 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:11.611 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.611 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.611 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.611 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:36344 4.6 e106/81 4:6d7fae61:::benchmark_data_smithi074_44586_object4542:head v 106'2144, mlcod=106'2135)", 2020-08-07T15:09:11.612 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:53.851275+0000", 2020-08-07T15:09:11.612 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.804555751999999, 2020-08-07T15:09:11.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21501221100000001, 2020-08-07T15:09:11.612 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:11.612 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:11.614 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:11.614 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.614 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:11.614 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.851275+0000", 2020-08-07T15:09:11.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:11.615 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.615 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.615 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:11.615 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.851275+0000", 2020-08-07T15:09:11.615 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9171000000000001e-05 2020-08-07T15:09:11.615 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.616 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.616 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:11.616 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.851324+0000", 2020-08-07T15:09:11.616 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2299999999999996e-06 2020-08-07T15:09:11.619 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.619 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.619 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:11.619 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:53.851330+0000", 2020-08-07T15:09:11.619 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21380696099999999 2020-08-07T15:09:11.620 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.620 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.620 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:11.620 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.065137+0000", 2020-08-07T15:09:11.620 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6879999999999996e-06 2020-08-07T15:09:11.621 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.622 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.622 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:11.623 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.065143+0000", 2020-08-07T15:09:11.623 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9782000000000003e-05 2020-08-07T15:09:11.633 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.633 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.634 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:11.634 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.065222+0000", 2020-08-07T15:09:11.634 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6562000000000001e-05 2020-08-07T15:09:11.634 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.634 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.635 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:11.635 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.065249+0000", 2020-08-07T15:09:11.635 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00099917499999999993 2020-08-07T15:09:11.635 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.636 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.636 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:11.636 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.066248+0000", 2020-08-07T15:09:11.636 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8642e-05 2020-08-07T15:09:11.636 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.636 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.637 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:11.637 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.066287+0000", 2020-08-07T15:09:11.637 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21501221100000001 2020-08-07T15:09:11.637 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.638 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:11.638 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.638 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.639 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.639 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:42280 4.6 e112/81 4:6fc8ffaf:::benchmark_data_smithi074_44586_object5284:head v 112'2464, mlcod=112'2455)", 2020-08-07T15:09:11.639 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.113698+0000", 2020-08-07T15:09:11.639 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 10.542132661, 2020-08-07T15:09:11.639 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43055294, 2020-08-07T15:09:11.640 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:11.640 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:11.640 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:11.640 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.640 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:11.641 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.113698+0000", 2020-08-07T15:09:11.641 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:11.641 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.641 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.641 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:11.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.113698+0000", 2020-08-07T15:09:11.642 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2302000000000003e-05 2020-08-07T15:09:11.642 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.642 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.642 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:11.643 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.113740+0000", 2020-08-07T15:09:11.643 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3669999999999999e-06 2020-08-07T15:09:11.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.643 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.644 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:11.644 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.113744+0000", 2020-08-07T15:09:11.644 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.42958231000000002 2020-08-07T15:09:11.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:11.645 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.543327+0000", 2020-08-07T15:09:11.645 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2373e-05 2020-08-07T15:09:11.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.646 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.646 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:11.646 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.543339+0000", 2020-08-07T15:09:11.646 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014704099999999999 2020-08-07T15:09:11.646 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.647 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.647 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:11.647 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.543486+0000", 2020-08-07T15:09:11.647 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4834999999999999e-05 2020-08-07T15:09:11.647 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.648 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.648 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:11.648 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.543511+0000", 2020-08-07T15:09:11.648 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00069499500000000001 2020-08-07T15:09:11.648 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.648 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.649 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:11.649 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.544206+0000", 2020-08-07T15:09:11.649 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4716999999999999e-05 2020-08-07T15:09:11.649 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.649 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.650 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:11.650 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.544251+0000", 2020-08-07T15:09:11.650 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43055294 2020-08-07T15:09:11.650 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.650 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:11.651 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.651 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.651 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:11.651 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:09:11.656 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:09:11.657 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:09:11.657 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:09:11.657 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:09:11.658 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:09:11.658 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:09:11.658 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:09:11.658 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:09:11.658 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:09:11.659 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:09:11.659 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:09:11.659 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:09:11.659 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:09:11.659 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:09:11.660 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:09:11.660 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 45 16 56147 56131 9.74405 13.5938 0.00914469 0.0128243 2020-08-07T15:09:11.660 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:09:11.661 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:11.651+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:11.761 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:11.759+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:11.866 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:11.856+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:11.889 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:09:11.890 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:09:11.891 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:09:11.891 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.891 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:57170 4.a e121/81 4:5dde08c6:::benchmark_data_smithi074_44586_object7146:head v 121'3986, mlcod=121'3983)", 2020-08-07T15:09:11.891 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:11.885087+0000", 2020-08-07T15:09:11.891 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0032179439999999999, 2020-08-07T15:09:11.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032357100000000001, 2020-08-07T15:09:11.892 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:11.892 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:09:11.892 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:11.892 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.893 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:11.893 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.885087+0000", 2020-08-07T15:09:11.893 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:11.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.908 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:11.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.885087+0000", 2020-08-07T15:09:11.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0796999999999997e-05 2020-08-07T15:09:11.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:11.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.885118+0000", 2020-08-07T15:09:11.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6190000000000002e-06 2020-08-07T15:09:11.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.910 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:11.910 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.885123+0000", 2020-08-07T15:09:11.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1087999999999998e-05 2020-08-07T15:09:11.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:11.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.885154+0000", 2020-08-07T15:09:11.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4999999999999996e-06 2020-08-07T15:09:11.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:11.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.885160+0000", 2020-08-07T15:09:11.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2429999999999999e-05 2020-08-07T15:09:11.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.938 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:11.938 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.885203+0000", 2020-08-07T15:09:11.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5330999999999999e-05 2020-08-07T15:09:11.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:11.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.885218+0000", 2020-08-07T15:09:11.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000130765 2020-08-07T15:09:11.940 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.940 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:11.940 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.940 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.941 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:57171 4.a e121/81 4:5dde08c6:::benchmark_data_smithi074_44586_object7146:head v 121'3987, mlcod=121'3983)", 2020-08-07T15:09:11.941 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:11.885738+0000", 2020-08-07T15:09:11.941 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0025672189999999999, 2020-08-07T15:09:11.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026254249999999998, 2020-08-07T15:09:11.941 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:11.964 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:09:11.965 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:11.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:11.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.885738+0000", 2020-08-07T15:09:11.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:11.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:11.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.885738+0000", 2020-08-07T15:09:11.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3142e-05 2020-08-07T15:09:11.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:11.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.885791+0000", 2020-08-07T15:09:11.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2979999999999997e-06 2020-08-07T15:09:11.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:11.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.885797+0000", 2020-08-07T15:09:11.969 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9928e-05 2020-08-07T15:09:11.969 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.969 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.969 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:11.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.885817+0000", 2020-08-07T15:09:11.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5910000000000003e-06 2020-08-07T15:09:11.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.970 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:11.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.885823+0000", 2020-08-07T15:09:11.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6040000000000001e-05 2020-08-07T15:09:11.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:11.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.885859+0000", 2020-08-07T15:09:11.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.02e-05 2020-08-07T15:09:11.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:11.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.885879+0000", 2020-08-07T15:09:11.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000141199 2020-08-07T15:09:11.973 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.974 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:11.974 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:11.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.975 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:57172 4.a e121/81 4:5dde08c6:::benchmark_data_smithi074_44586_object7146:head v 121'3988, mlcod=121'3983)", 2020-08-07T15:09:11.975 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:11.886439+0000", 2020-08-07T15:09:11.975 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.001866525, 2020-08-07T15:09:11.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019621679999999998, 2020-08-07T15:09:11.975 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:11.976 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:09:11.976 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:11.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:11.979 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.886439+0000", 2020-08-07T15:09:11.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:11.979 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.979 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.979 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:11.981 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.886439+0000", 2020-08-07T15:09:11.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9342000000000001e-05 2020-08-07T15:09:11.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.982 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:11.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.886468+0000", 2020-08-07T15:09:11.982 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.89e-06 2020-08-07T15:09:11.982 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.983 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:11.983 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.886472+0000", 2020-08-07T15:09:11.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4535999999999999e-05 2020-08-07T15:09:11.983 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.983 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.984 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:11.984 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.886486+0000", 2020-08-07T15:09:11.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5750000000000001e-06 2020-08-07T15:09:11.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.985 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:11.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.886490+0000", 2020-08-07T15:09:11.985 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8558999999999999e-05 2020-08-07T15:09:11.985 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:11.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:11.986 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:11.986 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.886519+0000", 2020-08-07T15:09:11.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5998000000000002e-05 2020-08-07T15:09:11.994 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:12.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.886535+0000", 2020-08-07T15:09:12.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.59e-05 2020-08-07T15:09:12.004 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:12.004 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:12.004 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:12.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.005 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:57152 4.7 4:e36be368:::benchmark_data_smithi074_44586_object7143:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e121)", 2020-08-07T15:09:12.007 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:11.868351+0000", 2020-08-07T15:09:12.007 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.019954209000000001, 2020-08-07T15:09:12.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.020088812000000001, 2020-08-07T15:09:12.008 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:12.008 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:12.013 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:12.013 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:12.013 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:12.017 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 57152 2020-08-07T15:09:12.017 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.017 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:12.018 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.018 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:12.018 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.868351+0000", 2020-08-07T15:09:12.019 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:12.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.019 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.019 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:12.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.868351+0000", 2020-08-07T15:09:12.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1548999999999997e-05 2020-08-07T15:09:12.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.020 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.020 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:12.020 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.868403+0000", 2020-08-07T15:09:12.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1789999999999996e-06 2020-08-07T15:09:12.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:12.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.868409+0000", 2020-08-07T15:09:12.022 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0508e-05 2020-08-07T15:09:12.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:12.022 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.868419+0000", 2020-08-07T15:09:12.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4499999999999997e-06 2020-08-07T15:09:12.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:12.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.868424+0000", 2020-08-07T15:09:12.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044104219999999998 2020-08-07T15:09:12.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:12.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.872834+0000", 2020-08-07T15:09:12.030 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0912999999999998e-05 2020-08-07T15:09:12.030 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.030 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.030 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:12.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.872895+0000", 2020-08-07T15:09:12.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010487199999999999 2020-08-07T15:09:12.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.041 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,5", 2020-08-07T15:09:12.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.873000+0000", 2020-08-07T15:09:12.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0036982640000000002 2020-08-07T15:09:12.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.042 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:12.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.876698+0000", 2020-08-07T15:09:12.042 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5199999999999996e-07 2020-08-07T15:09:12.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:12.044 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.876699+0000", 2020-08-07T15:09:12.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4793000000000002e-05 2020-08-07T15:09:12.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:12.045 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.876754+0000", 2020-08-07T15:09:12.045 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0050885820000000003 2020-08-07T15:09:12.045 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.045 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.046 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:12.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.881842+0000", 2020-08-07T15:09:12.046 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.013491283999999999 2020-08-07T15:09:12.046 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:12.046 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:12.047 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:12.047 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.047 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:57168 4.b 4:df54b906:::benchmark_data_smithi074_44586_object7145:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e121)", 2020-08-07T15:09:12.047 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:11.881964+0000", 2020-08-07T15:09:12.048 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0063416380000000001, 2020-08-07T15:09:12.048 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0065365709999999997, 2020-08-07T15:09:12.048 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:12.048 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:12.048 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:12.049 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:12.049 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:12.049 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 57168 2020-08-07T15:09:12.049 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.049 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:12.050 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.050 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:12.050 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.881964+0000", 2020-08-07T15:09:12.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:12.050 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.051 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:12.051 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.881964+0000", 2020-08-07T15:09:12.051 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0719000000000002e-05 2020-08-07T15:09:12.051 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.052 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:12.052 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.881994+0000", 2020-08-07T15:09:12.052 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0149999999999997e-06 2020-08-07T15:09:12.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.052 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.053 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:12.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.881998+0000", 2020-08-07T15:09:12.053 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4139999999999997e-06 2020-08-07T15:09:12.053 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.054 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:12.054 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.882006+0000", 2020-08-07T15:09:12.054 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.349e-06 2020-08-07T15:09:12.054 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.054 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:12.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.882009+0000", 2020-08-07T15:09:12.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000160242 2020-08-07T15:09:12.055 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.055 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:12.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.882169+0000", 2020-08-07T15:09:12.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9050999999999997e-05 2020-08-07T15:09:12.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:12.057 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.882268+0000", 2020-08-07T15:09:12.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020008299999999999 2020-08-07T15:09:12.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.058 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:09:12.058 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.882468+0000", 2020-08-07T15:09:12.058 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0014685239999999999 2020-08-07T15:09:12.058 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.058 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.058 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:12.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.883937+0000", 2020-08-07T15:09:12.059 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3e-07 2020-08-07T15:09:12.059 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.059 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.059 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:12.060 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.883938+0000", 2020-08-07T15:09:12.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1148999999999993e-05 2020-08-07T15:09:12.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:12.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.883999+0000", 2020-08-07T15:09:12.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019583909999999999 2020-08-07T15:09:12.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.061 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:12.062 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:11.885957+0000", 2020-08-07T15:09:12.062 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039936670000000002 2020-08-07T15:09:12.062 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:12.062 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:12.062 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:12.063 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:12.063 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:09:12.063 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 5 2020-08-07T15:09:12.063 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:09:12.066 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:09:12.067 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:12.063+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:12.168 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:12.165+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:12.289 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 46 16 57823 57807 9.81685 13.0938 0.0162854 0.0127294 2020-08-07T15:09:12.290 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:12.283+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:12.353 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:09:12.353 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:09:12.354 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:09:12.354 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:09:12.354 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:09:12.355 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:09:12.395 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:12.392+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:12.517 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:12.488+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:12.549 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:09:12.556 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:09:12.566 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2020-08-07T15:09:11.348869+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":402391040,"num_objects":6142,"num_object_clones":0,"num_object_copies":18426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49120,"num_write_kb":392960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":636,"num_bytes_recovered":41558016,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":135,"acting":135,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9587788,"kb_used_data":1199116,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":745386932,"statfs":{"total":773094113280,"available":763276218368,"internally_reserved":8589934592,"allocated":1227894784,"data_stored":1222533191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,3,9,0,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":48889856,"num_objects":746,"num_object_clones":0,"num_object_copies":2238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5968,"num_write_kb":47744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":32768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.036711"},"pg_stats":[{"pgid":"1.14","version":"16'2","reported_seq":"47","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.268218+0000","last_change":"2020-08-07T15:08:42.533999+0000","last_active":"2020-08-07T15:09:04.268218+0000","last_peered":"2020-08-07T15:09:04.268218+0000","last_clean":"2020-08-07T15:09:04.268218+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:09:04.268218+0000","last_undegraded":"2020-08-07T15:09:04.268218+0000","last_fullsized":"2020-08-07T15:09:04.268218+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:42.533921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:42.533921+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"83","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.179395+0000","last_change":"2020-08-07T15:08:58.212828+0000","last_active":"2020-08-07T15:09:04.179395+0000","last_peered":"2020-08-07T15:09:04.179395+0000","last_clean":"2020-08-07T15:09:04.179395+0000","last_became_active":"2020-08-07T15:08:58.211785+0000","last_became_peered":"2020-08-07T15:08:58.211785+0000","last_unstale":"2020-08-07T15:09:04.179395+0000","last_undegraded":"2020-08-07T15:09:04.179395+0000","last_fullsized":"2020-08-07T15:09:04.179395+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.141010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.141010+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"47","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.177181+0000","last_change":"2020-08-07T15:08:55.078015+0000","last_active":"2020-08-07T15:09:04.177181+0000","last_peered":"2020-08-07T15:09:04.177181+0000","last_clean":"2020-08-07T15:09:04.177181+0000","last_became_active":"2020-08-07T15:08:41.685106+0000","last_became_peered":"2020-08-07T15:08:41.685106+0000","last_unstale":"2020-08-07T15:09:04.177181+0000","last_undegraded":"2020-08-07T15:09:04.177181+0000","last_fullsized":"2020-08-07T15:09:04.177181+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:55.077924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:55.077924+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"126","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.177119+0000","last_change":"2020-08-07T15:08:43.544399+0000","last_active":"2020-08-07T15:09:04.177119+0000","last_peered":"2020-08-07T15:09:04.177119+0000","last_clean":"2020-08-07T15:09:04.177119+0000","last_became_active":"2020-08-07T15:08:41.675776+0000","last_became_peered":"2020-08-07T15:08:41.675776+0000","last_unstale":"2020-08-07T15:09:04.177119+0000","last_undegraded":"2020-08-07T15:09:04.177119+0000","last_fullsized":"2020-08-07T15:09:04.177119+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.544324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.544324+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"154","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.177040+0000","last_change":"2020-08-07T15:08:51.599032+0000","last_active":"2020-08-07T15:09:04.177040+0000","last_peered":"2020-08-07T15:09:04.177040+0000","last_clean":"2020-08-07T15:09:04.177040+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:09:04.177040+0000","last_undegraded":"2020-08-07T15:09:04.177040+0000","last_fullsized":"2020-08-07T15:09:04.177040+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:51.598947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.598947+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"113","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:05.318396+0000","last_change":"2020-08-07T15:08:11.840861+0000","last_active":"2020-08-07T15:09:05.318396+0000","last_peered":"2020-08-07T15:09:05.318396+0000","last_clean":"2020-08-07T15:09:05.318396+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:09:05.318396+0000","last_undegraded":"2020-08-07T15:09:05.318396+0000","last_fullsized":"2020-08-07T15:09:05.318396+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:11.840807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:11.840807+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"103","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.175603+0000","last_change":"2020-08-07T15:08:39.558760+0000","last_active":"2020-08-07T15:09:04.175603+0000","last_peered":"2020-08-07T15:09:04.175603+0000","last_clean":"2020-08-07T15:09:04.175603+0000","last_became_active":"2020-08-07T15:08:36.615358+0000","last_became_peered":"2020-08-07T15:08:36.615358+0000","last_unstale":"2020-08-07T15:09:04.175603+0000","last_undegraded":"2020-08-07T15:09:04.175603+0000","last_fullsized":"2020-08-07T15:09:04.175603+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:39.558677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:39.558677+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"118'2992","reported_seq":"3037","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:06.667165+0000","last_change":"2020-08-07T15:08:46.715869+0000","last_active":"2020-08-07T15:09:06.667165+0000","last_peered":"2020-08-07T15:09:06.667165+0000","last_clean":"2020-08-07T15:09:06.667165+0000","last_became_active":"2020-08-07T15:08:46.708112+0000","last_became_peered":"2020-08-07T15:08:46.708112+0000","last_unstale":"2020-08-07T15:09:06.667165+0000","last_undegraded":"2020-08-07T15:09:06.667165+0000","last_fullsized":"2020-08-07T15:09:06.667165+0000","mapping_epoch":98,"log_start":"118'2984","ondisk_log_start":"118'2984","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24510464,"num_objects":374,"num_object_clones":0,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2992,"num_write_kb":23936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"115","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.179579+0000","last_change":"2020-08-07T15:08:40.057705+0000","last_active":"2020-08-07T15:09:04.179579+0000","last_peered":"2020-08-07T15:09:04.179579+0000","last_clean":"2020-08-07T15:09:04.179579+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:09:04.179579+0000","last_undegraded":"2020-08-07T15:09:04.179579+0000","last_fullsized":"2020-08-07T15:09:04.179579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:40.057622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:40.057622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"72","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.268511+0000","last_change":"2020-08-07T15:08:53.756554+0000","last_active":"2020-08-07T15:09:04.268511+0000","last_peered":"2020-08-07T15:09:04.268511+0000","last_clean":"2020-08-07T15:09:04.268511+0000","last_became_active":"2020-08-07T15:08:46.710404+0000","last_became_peered":"2020-08-07T15:08:46.710404+0000","last_unstale":"2020-08-07T15:09:04.268511+0000","last_undegraded":"2020-08-07T15:09:04.268511+0000","last_fullsized":"2020-08-07T15:09:04.268511+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:53.756343+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:53.756343+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"118'3232","reported_seq":"3922","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:06.568365+0000","last_change":"2020-08-07T15:09:01.304273+0000","last_active":"2020-08-07T15:09:06.568365+0000","last_peered":"2020-08-07T15:09:06.568365+0000","last_clean":"2020-08-07T15:09:06.568365+0000","last_became_active":"2020-08-07T15:09:01.293427+0000","last_became_peered":"2020-08-07T15:09:01.293427+0000","last_unstale":"2020-08-07T15:09:06.568365+0000","last_undegraded":"2020-08-07T15:09:06.568365+0000","last_fullsized":"2020-08-07T15:09:06.568365+0000","mapping_epoch":113,"log_start":"118'3229","ondisk_log_start":"118'3229","created":81,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":26476544,"num_objects":404,"num_object_clones":0,"num_object_copies":1212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3232,"num_write_kb":25856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":20971520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"112","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.212644+0000","last_change":"2020-08-07T15:08:27.906894+0000","last_active":"2020-08-07T15:09:04.212644+0000","last_peered":"2020-08-07T15:09:04.212644+0000","last_clean":"2020-08-07T15:09:04.212644+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:09:04.212644+0000","last_undegraded":"2020-08-07T15:09:04.212644+0000","last_fullsized":"2020-08-07T15:09:04.212644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:27.906822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:27.906822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"118","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:05.318412+0000","last_change":"2020-08-07T15:08:43.899342+0000","last_active":"2020-08-07T15:09:05.318412+0000","last_peered":"2020-08-07T15:09:05.318412+0000","last_clean":"2020-08-07T15:09:05.318412+0000","last_became_active":"2020-08-07T15:08:36.622209+0000","last_became_peered":"2020-08-07T15:08:36.622209+0000","last_unstale":"2020-08-07T15:09:05.318412+0000","last_undegraded":"2020-08-07T15:09:05.318412+0000","last_fullsized":"2020-08-07T15:09:05.318412+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.899280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.899280+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"118'2888","reported_seq":"2933","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:06.693411+0000","last_change":"2020-08-07T15:08:24.549715+0000","last_active":"2020-08-07T15:09:06.693411+0000","last_peered":"2020-08-07T15:09:06.693411+0000","last_clean":"2020-08-07T15:09:06.693411+0000","last_became_active":"2020-08-07T15:08:24.549245+0000","last_became_peered":"2020-08-07T15:08:24.549245+0000","last_unstale":"2020-08-07T15:09:06.693411+0000","last_undegraded":"2020-08-07T15:09:06.693411+0000","last_fullsized":"2020-08-07T15:09:06.693411+0000","mapping_epoch":81,"log_start":"118'2878","ondisk_log_start":"118'2878","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23658496,"num_objects":361,"num_object_clones":0,"num_object_copies":1083,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2888,"num_write_kb":23104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"112","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.212863+0000","last_change":"2020-08-07T15:08:13.867355+0000","last_active":"2020-08-07T15:09:04.212863+0000","last_peered":"2020-08-07T15:09:04.212863+0000","last_clean":"2020-08-07T15:09:04.212863+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:09:04.212863+0000","last_undegraded":"2020-08-07T15:09:04.212863+0000","last_fullsized":"2020-08-07T15:09:04.212863+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:13.867148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:13.867148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"118","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:05.318235+0000","last_change":"2020-08-07T15:08:50.686578+0000","last_active":"2020-08-07T15:09:05.318235+0000","last_peered":"2020-08-07T15:09:05.318235+0000","last_clean":"2020-08-07T15:09:05.318235+0000","last_became_active":"2020-08-07T15:08:36.633406+0000","last_became_peered":"2020-08-07T15:08:36.633406+0000","last_unstale":"2020-08-07T15:09:05.318235+0000","last_undegraded":"2020-08-07T15:09:05.318235+0000","last_fullsized":"2020-08-07T15:09:05.318235+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:50.686496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:50.686496+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"118'2888","reported_seq":"2933","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:06.702365+0000","last_change":"2020-08-07T15:08:24.557398+0000","last_active":"2020-08-07T15:09:06.702365+0000","last_peered":"2020-08-07T15:09:06.702365+0000","last_clean":"2020-08-07T15:09:06.702365+0000","last_became_active":"2020-08-07T15:08:24.557022+0000","last_became_peered":"2020-08-07T15:08:24.557022+0000","last_unstale":"2020-08-07T15:09:06.702365+0000","last_undegraded":"2020-08-07T15:09:06.702365+0000","last_fullsized":"2020-08-07T15:09:06.702365+0000","mapping_epoch":81,"log_start":"118'2882","ondisk_log_start":"118'2882","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23658496,"num_objects":361,"num_object_clones":0,"num_object_copies":1083,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2888,"num_write_kb":23104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"144","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.212806+0000","last_change":"2020-08-07T15:08:43.036067+0000","last_active":"2020-08-07T15:09:04.212806+0000","last_peered":"2020-08-07T15:09:04.212806+0000","last_clean":"2020-08-07T15:09:04.212806+0000","last_became_active":"2020-08-07T15:08:36.627426+0000","last_became_peered":"2020-08-07T15:08:36.627426+0000","last_unstale":"2020-08-07T15:09:04.212806+0000","last_undegraded":"2020-08-07T15:09:04.212806+0000","last_fullsized":"2020-08-07T15:09:04.212806+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.035987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.035987+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"112","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.177378+0000","last_change":"2020-08-07T15:08:26.317430+0000","last_active":"2020-08-07T15:09:04.177378+0000","last_peered":"2020-08-07T15:09:04.177378+0000","last_clean":"2020-08-07T15:09:04.177378+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:09:04.177378+0000","last_undegraded":"2020-08-07T15:09:04.177378+0000","last_fullsized":"2020-08-07T15:09:04.177378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:26.317351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:26.317351+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"119'3192","reported_seq":"3237","reported_epoch":"119","state":"active+clean","last_fresh":"2020-08-07T15:09:07.693635+0000","last_change":"2020-08-07T15:08:24.552570+0000","last_active":"2020-08-07T15:09:07.693635+0000","last_peered":"2020-08-07T15:09:07.693635+0000","last_clean":"2020-08-07T15:09:07.693635+0000","last_became_active":"2020-08-07T15:08:24.552105+0000","last_became_peered":"2020-08-07T15:08:24.552105+0000","last_unstale":"2020-08-07T15:09:07.693635+0000","last_undegraded":"2020-08-07T15:09:07.693635+0000","last_fullsized":"2020-08-07T15:09:07.693635+0000","mapping_epoch":81,"log_start":"119'3185","ondisk_log_start":"119'3185","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":26148864,"num_objects":399,"num_object_clones":0,"num_object_copies":1197,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3192,"num_write_kb":25536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"115","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:05.606603+0000","last_change":"2020-08-07T15:09:05.606686+0000","last_active":"2020-08-07T15:09:05.606603+0000","last_peered":"2020-08-07T15:09:05.606603+0000","last_clean":"2020-08-07T15:09:05.606603+0000","last_became_active":"2020-08-07T15:06:23.223489+0000","last_became_peered":"2020-08-07T15:06:23.223489+0000","last_unstale":"2020-08-07T15:09:05.606603+0000","last_undegraded":"2020-08-07T15:09:05.606603+0000","last_fullsized":"2020-08-07T15:09:05.606603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:05.606603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:05.606603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"160","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.211952+0000","last_change":"2020-08-07T15:08:42.069637+0000","last_active":"2020-08-07T15:09:04.211952+0000","last_peered":"2020-08-07T15:09:04.211952+0000","last_clean":"2020-08-07T15:09:04.211952+0000","last_became_active":"2020-08-07T15:08:36.628365+0000","last_became_peered":"2020-08-07T15:08:36.628365+0000","last_unstale":"2020-08-07T15:09:04.211952+0000","last_undegraded":"2020-08-07T15:09:04.211952+0000","last_fullsized":"2020-08-07T15:09:04.211952+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:42.069585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:42.069585+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"118'3176","reported_seq":"3221","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:06.671020+0000","last_change":"2020-08-07T15:08:24.550789+0000","last_active":"2020-08-07T15:09:06.671020+0000","last_peered":"2020-08-07T15:09:06.671020+0000","last_clean":"2020-08-07T15:09:06.671020+0000","last_became_active":"2020-08-07T15:08:24.550222+0000","last_became_peered":"2020-08-07T15:08:24.550222+0000","last_unstale":"2020-08-07T15:09:06.671020+0000","last_undegraded":"2020-08-07T15:09:06.671020+0000","last_fullsized":"2020-08-07T15:09:06.671020+0000","mapping_epoch":81,"log_start":"118'3169","ondisk_log_start":"118'3169","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":26017792,"num_objects":397,"num_object_clones":0,"num_object_copies":1191,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3176,"num_write_kb":25408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"112","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.177353+0000","last_change":"2020-08-07T15:08:30.339852+0000","last_active":"2020-08-07T15:09:04.177353+0000","last_peered":"2020-08-07T15:09:04.177353+0000","last_clean":"2020-08-07T15:09:04.177353+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:09:04.177353+0000","last_undegraded":"2020-08-07T15:09:04.177353+0000","last_fullsized":"2020-08-07T15:09:04.177353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:08:30.339785+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"110","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.177808+0000","last_change":"2020-08-07T15:08:38.386792+0000","last_active":"2020-08-07T15:09:04.177808+0000","last_peered":"2020-08-07T15:09:04.177808+0000","last_clean":"2020-08-07T15:09:04.177808+0000","last_became_active":"2020-08-07T15:08:36.618164+0000","last_became_peered":"2020-08-07T15:08:36.618164+0000","last_unstale":"2020-08-07T15:09:04.177808+0000","last_undegraded":"2020-08-07T15:09:04.177808+0000","last_fullsized":"2020-08-07T15:09:04.177808+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:38.386739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:38.386739+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"118'3040","reported_seq":"3085","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:06.645289+0000","last_change":"2020-08-07T15:08:24.558242+0000","last_active":"2020-08-07T15:09:06.645289+0000","last_peered":"2020-08-07T15:09:06.645289+0000","last_clean":"2020-08-07T15:09:06.645289+0000","last_became_active":"2020-08-07T15:08:24.557354+0000","last_became_peered":"2020-08-07T15:08:24.557354+0000","last_unstale":"2020-08-07T15:09:06.645289+0000","last_undegraded":"2020-08-07T15:09:06.645289+0000","last_fullsized":"2020-08-07T15:09:06.645289+0000","mapping_epoch":81,"log_start":"118'3035","ondisk_log_start":"118'3035","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24903680,"num_objects":380,"num_object_clones":0,"num_object_copies":1140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3040,"num_write_kb":24320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"109","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.212117+0000","last_change":"2020-08-07T15:08:15.887617+0000","last_active":"2020-08-07T15:09:04.212117+0000","last_peered":"2020-08-07T15:09:04.212117+0000","last_clean":"2020-08-07T15:09:04.212117+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:09:04.212117+0000","last_undegraded":"2020-08-07T15:09:04.212117+0000","last_fullsized":"2020-08-07T15:09:04.212117+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:15.887513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:15.887513+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"149","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:05.318455+0000","last_change":"2020-08-07T15:08:58.183263+0000","last_active":"2020-08-07T15:09:05.318455+0000","last_peered":"2020-08-07T15:09:05.318455+0000","last_clean":"2020-08-07T15:09:05.318455+0000","last_became_active":"2020-08-07T15:08:58.182722+0000","last_became_peered":"2020-08-07T15:08:58.182722+0000","last_unstale":"2020-08-07T15:09:05.318455+0000","last_undegraded":"2020-08-07T15:09:05.318455+0000","last_fullsized":"2020-08-07T15:09:05.318455+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.927824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.927824+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"118'2928","reported_seq":"2973","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:06.588467+0000","last_change":"2020-08-07T15:08:24.552962+0000","last_active":"2020-08-07T15:09:06.588467+0000","last_peered":"2020-08-07T15:09:06.588467+0000","last_clean":"2020-08-07T15:09:06.588467+0000","last_became_active":"2020-08-07T15:08:24.552070+0000","last_became_peered":"2020-08-07T15:08:24.552070+0000","last_unstale":"2020-08-07T15:09:06.588467+0000","last_undegraded":"2020-08-07T15:09:06.588467+0000","last_fullsized":"2020-08-07T15:09:06.588467+0000","mapping_epoch":81,"log_start":"118'2918","ondisk_log_start":"118'2918","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23986176,"num_objects":366,"num_object_clones":0,"num_object_copies":1098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2928,"num_write_kb":23424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"87","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.177304+0000","last_change":"2020-08-07T15:08:39.409159+0000","last_active":"2020-08-07T15:09:04.177304+0000","last_peered":"2020-08-07T15:09:04.177304+0000","last_clean":"2020-08-07T15:09:04.177304+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:09:04.177304+0000","last_undegraded":"2020-08-07T15:09:04.177304+0000","last_fullsized":"2020-08-07T15:09:04.177304+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:39.409063+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_clean_scrub_stamp":"2020-08-07T15:08:39.409063+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"118'3144","reported_seq":"3187","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:06.687044+0000","last_change":"2020-08-07T15:08:24.554063+0000","last_active":"2020-08-07T15:09:06.687044+0000","last_peered":"2020-08-07T15:09:06.687044+0000","last_clean":"2020-08-07T15:09:06.687044+0000","last_became_active":"2020-08-07T15:08:24.553668+0000","last_became_peered":"2020-08-07T15:08:24.553668+0000","last_unstale":"2020-08-07T15:09:06.687044+0000","last_undegraded":"2020-08-07T15:09:06.687044+0000","last_fullsized":"2020-08-07T15:09:06.687044+0000","mapping_epoch":81,"log_start":"118'3136","ondisk_log_start":"118'3136","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25755648,"num_objects":393,"num_object_clones":0,"num_object_copies":1179,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3144,"num_write_kb":25152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"84","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.177067+0000","last_change":"2020-08-07T15:07:33.177491+0000","last_active":"2020-08-07T15:09:04.177067+0000","last_peered":"2020-08-07T15:09:04.177067+0000","last_clean":"2020-08-07T15:09:04.177067+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:09:04.177067+0000","last_undegraded":"2020-08-07T15:09:04.177067+0000","last_fullsized":"2020-08-07T15:09:04.177067+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:33.177412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:33.177412+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"118'2840","reported_seq":"2885","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:06.639751+0000","last_change":"2020-08-07T15:08:46.709789+0000","last_active":"2020-08-07T15:09:06.639751+0000","last_peered":"2020-08-07T15:09:06.639751+0000","last_clean":"2020-08-07T15:09:06.639751+0000","last_became_active":"2020-08-07T15:08:46.708674+0000","last_became_peered":"2020-08-07T15:08:46.708674+0000","last_unstale":"2020-08-07T15:09:06.639751+0000","last_undegraded":"2020-08-07T15:09:06.639751+0000","last_fullsized":"2020-08-07T15:09:06.639751+0000","mapping_epoch":98,"log_start":"118'2837","ondisk_log_start":"118'2837","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23265280,"num_objects":355,"num_object_clones":0,"num_object_copies":1065,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2840,"num_write_kb":22720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"148","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.212102+0000","last_change":"2020-08-07T15:07:56.631170+0000","last_active":"2020-08-07T15:09:04.212102+0000","last_peered":"2020-08-07T15:09:04.212102+0000","last_clean":"2020-08-07T15:09:04.212102+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:09:04.212102+0000","last_undegraded":"2020-08-07T15:09:04.212102+0000","last_fullsized":"2020-08-07T15:09:04.212102+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:56.631091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:56.631091+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"118'3144","reported_seq":"3210","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:06.716112+0000","last_change":"2020-08-07T15:08:52.193226+0000","last_active":"2020-08-07T15:09:06.716112+0000","last_peered":"2020-08-07T15:09:06.716112+0000","last_clean":"2020-08-07T15:09:06.716112+0000","last_became_active":"2020-08-07T15:08:52.166336+0000","last_became_peered":"2020-08-07T15:08:52.166336+0000","last_unstale":"2020-08-07T15:09:06.716112+0000","last_undegraded":"2020-08-07T15:09:06.716112+0000","last_fullsized":"2020-08-07T15:09:06.716112+0000","mapping_epoch":104,"log_start":"118'3140","ondisk_log_start":"118'3140","created":81,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25755648,"num_objects":393,"num_object_clones":0,"num_object_copies":1179,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3144,"num_write_kb":25152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":40960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"106","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.212105+0000","last_change":"2020-08-07T15:08:51.513398+0000","last_active":"2020-08-07T15:09:04.212105+0000","last_peered":"2020-08-07T15:09:04.212105+0000","last_clean":"2020-08-07T15:09:04.212105+0000","last_became_active":"2020-08-07T15:08:41.680448+0000","last_became_peered":"2020-08-07T15:08:41.680448+0000","last_unstale":"2020-08-07T15:09:04.212105+0000","last_undegraded":"2020-08-07T15:09:04.212105+0000","last_fullsized":"2020-08-07T15:09:04.212105+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:51.513319+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.513319+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"118'3064","reported_seq":"3107","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:06.698937+0000","last_change":"2020-08-07T15:08:24.550496+0000","last_active":"2020-08-07T15:09:06.698937+0000","last_peered":"2020-08-07T15:09:06.698937+0000","last_clean":"2020-08-07T15:09:06.698937+0000","last_became_active":"2020-08-07T15:08:24.549868+0000","last_became_peered":"2020-08-07T15:08:24.549868+0000","last_unstale":"2020-08-07T15:09:06.698937+0000","last_undegraded":"2020-08-07T15:09:06.698937+0000","last_fullsized":"2020-08-07T15:09:06.698937+0000","mapping_epoch":81,"log_start":"118'3056","ondisk_log_start":"118'3056","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25100288,"num_objects":383,"num_object_clones":0,"num_object_copies":1149,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3064,"num_write_kb":24512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"119","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:05.865996+0000","last_change":"2020-08-07T15:09:05.866078+0000","last_active":"2020-08-07T15:09:05.865996+0000","last_peered":"2020-08-07T15:09:05.865996+0000","last_clean":"2020-08-07T15:09:05.865996+0000","last_became_active":"2020-08-07T15:08:41.679321+0000","last_became_peered":"2020-08-07T15:08:41.679321+0000","last_unstale":"2020-08-07T15:09:05.865996+0000","last_undegraded":"2020-08-07T15:09:05.865996+0000","last_fullsized":"2020-08-07T15:09:05.865996+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:05.865996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:05.865996+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"118'3136","reported_seq":"3824","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:06.444154+0000","last_change":"2020-08-07T15:09:04.194221+0000","last_active":"2020-08-07T15:09:06.444154+0000","last_peered":"2020-08-07T15:09:06.444154+0000","last_clean":"2020-08-07T15:09:06.444154+0000","last_became_active":"2020-08-07T15:09:04.180570+0000","last_became_peered":"2020-08-07T15:09:04.180570+0000","last_unstale":"2020-08-07T15:09:06.444154+0000","last_undegraded":"2020-08-07T15:09:06.444154+0000","last_fullsized":"2020-08-07T15:09:06.444154+0000","mapping_epoch":116,"log_start":"118'3127","ondisk_log_start":"118'3127","created":81,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25690112,"num_objects":392,"num_object_clones":0,"num_object_copies":1176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3136,"num_write_kb":25088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":20545536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"148","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:05.318391+0000","last_change":"2020-08-07T15:08:41.689973+0000","last_active":"2020-08-07T15:09:05.318391+0000","last_peered":"2020-08-07T15:09:05.318391+0000","last_clean":"2020-08-07T15:09:05.318391+0000","last_became_active":"2020-08-07T15:08:41.689435+0000","last_became_peered":"2020-08-07T15:08:41.689435+0000","last_unstale":"2020-08-07T15:09:05.318391+0000","last_undegraded":"2020-08-07T15:09:05.318391+0000","last_fullsized":"2020-08-07T15:09:05.318391+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:07:36.927824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:07:36.927824+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"119'3512","reported_seq":"3558","reported_epoch":"119","state":"active+clean","last_fresh":"2020-08-07T15:09:07.416147+0000","last_change":"2020-08-07T15:08:24.554472+0000","last_active":"2020-08-07T15:09:07.416147+0000","last_peered":"2020-08-07T15:09:07.416147+0000","last_clean":"2020-08-07T15:09:07.416147+0000","last_became_active":"2020-08-07T15:08:24.554021+0000","last_became_peered":"2020-08-07T15:08:24.554021+0000","last_unstale":"2020-08-07T15:09:07.416147+0000","last_undegraded":"2020-08-07T15:09:07.416147+0000","last_fullsized":"2020-08-07T15:09:07.416147+0000","mapping_epoch":81,"log_start":"119'3503","ondisk_log_start":"119'3503","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":28770304,"num_objects":439,"num_object_clones":0,"num_object_copies":1317,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3512,"num_write_kb":28096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"34","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.212512+0000","last_change":"2020-08-07T15:08:46.710556+0000","last_active":"2020-08-07T15:09:04.212512+0000","last_peered":"2020-08-07T15:09:04.212512+0000","last_clean":"2020-08-07T15:09:04.212512+0000","last_became_active":"2020-08-07T15:08:46.710059+0000","last_became_peered":"2020-08-07T15:08:46.710059+0000","last_unstale":"2020-08-07T15:09:04.212512+0000","last_undegraded":"2020-08-07T15:09:04.212512+0000","last_fullsized":"2020-08-07T15:09:04.212512+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:09.841428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:09.841428+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"118'2904","reported_seq":"2949","reported_epoch":"118","state":"active+clean","last_fresh":"2020-08-07T15:09:06.318633+0000","last_change":"2020-08-07T15:08:24.552802+0000","last_active":"2020-08-07T15:09:06.318633+0000","last_peered":"2020-08-07T15:09:06.318633+0000","last_clean":"2020-08-07T15:09:06.318633+0000","last_became_active":"2020-08-07T15:08:24.552127+0000","last_became_peered":"2020-08-07T15:08:24.552127+0000","last_unstale":"2020-08-07T15:09:06.318633+0000","last_undegraded":"2020-08-07T15:09:06.318633+0000","last_fullsized":"2020-08-07T15:09:06.318633+0000","mapping_epoch":81,"log_start":"118'2896","ondisk_log_start":"118'2896","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23789568,"num_objects":363,"num_object_clones":0,"num_object_copies":1089,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2904,"num_write_kb":23232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"119'3040","reported_seq":"3087","reported_epoch":"119","state":"active+clean","last_fresh":"2020-08-07T15:09:07.645818+0000","last_change":"2020-08-07T15:08:46.710643+0000","last_active":"2020-08-07T15:09:07.645818+0000","last_peered":"2020-08-07T15:09:07.645818+0000","last_clean":"2020-08-07T15:09:07.645818+0000","last_became_active":"2020-08-07T15:08:46.710100+0000","last_became_peered":"2020-08-07T15:08:46.710100+0000","last_unstale":"2020-08-07T15:09:07.645818+0000","last_undegraded":"2020-08-07T15:09:07.645818+0000","last_fullsized":"2020-08-07T15:09:07.645818+0000","mapping_epoch":98,"log_start":"119'3031","ondisk_log_start":"119'3031","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24903680,"num_objects":380,"num_object_clones":0,"num_object_copies":1140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3040,"num_write_kb":24320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"56","reported_epoch":"117","state":"active+clean","last_fresh":"2020-08-07T15:09:04.177238+0000","last_change":"2020-08-07T15:08:41.678395+0000","last_active":"2020-08-07T15:09:04.177238+0000","last_peered":"2020-08-07T15:09:04.177238+0000","last_clean":"2020-08-07T15:09:04.177238+0000","last_became_active":"2020-08-07T15:08:41.677944+0000","last_became_peered":"2020-08-07T15:08:41.677944+0000","last_unstale":"2020-08-07T15:09:04.177238+0000","last_undegraded":"2020-08-07T15:09:04.177238+0000","last_fullsized":"2020-08-07T15:09:04.177238+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:02.901115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:02.901115+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":402391040,"num_objects":6140,"num_object_clones":0,"num_object_copies":18420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49120,"num_write_kb":392960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":636,"num_bytes_recovered":41558016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1233715200,"data_stored":1233715200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":63,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542180,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1201988,"kb_used_data":153404,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93169852,"statfs":{"total":96636764160,"available":95405928448,"internally_reserved":1073741824,"allocated":157085696,"data_stored":156408902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,7,0,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":0.95099999999999996,"15min":0.91200000000000003},"min":{"1min":0.48199999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":3.4159999999999999,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":2.1739999999999999},{"interface":"front","average":{"1min":1.0940000000000001,"5min":0.98899999999999999,"15min":0.97099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.3559999999999999,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.90800000000000003}]},{"osd":1,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3069999999999999,"15min":1.2989999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":4.4829999999999997,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.044,"15min":0.99199999999999999},"min":{"1min":0.61799999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":3.4620000000000002,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":2.206}]},{"osd":2,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.179,"15min":1.1439999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":4.423,"5min":4.423,"15min":4.423},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.5,"5min":1.194,"15min":1.143},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":3.5099999999999998,"5min":3.5099999999999998,"15min":3.5099999999999998},"last":3.4260000000000002}]},{"osd":3,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.5900000000000001,"15min":1.6439999999999999},"min":{"1min":0.73799999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.5430000000000001,"5min":5.5019999999999998,"15min":5.5019999999999998},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.329,"15min":1.3380000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.4590000000000001,"5min":2.7450000000000001,"15min":2.7450000000000001},"last":1.1140000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.7849999999999999,"5min":1.377,"15min":1.3089999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":4.835,"5min":4.835,"15min":4.835},"last":4.5250000000000004},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.3300000000000001,"15min":1.3180000000000001},"min":{"1min":0.76900000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.4470000000000001,"5min":2.778,"15min":2.778},"last":2.399}]},{"osd":5,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":2.3940000000000001,"5min":30.484000000000002,"15min":35.165999999999997},"min":{"1min":0.96799999999999997,"5min":0.875,"15min":0.875},"max":{"1min":7.4429999999999996,"5min":831.88099999999997,"15min":831.88099999999997},"last":6.149},{"interface":"front","average":{"1min":2.456,"5min":1.6870000000000001,"15min":1.5589999999999999},"min":{"1min":1,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":7.5350000000000001,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":4.6699999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.542,"15min":1.526},"min":{"1min":0.83599999999999997,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.9900000000000002,"5min":3.1389999999999998,"15min":3.1389999999999998},"last":1.631},{"interface":"front","average":{"1min":1.9219999999999999,"5min":1.5900000000000001,"15min":1.534},"min":{"1min":1.115,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":4.5170000000000003,"5min":4.5170000000000003,"15min":4.5170000000000003},"last":1.7310000000000001}]}]},{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1190200,"kb_used_data":141616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93181640,"statfs":{"total":96636764160,"available":95417999360,"internally_reserved":1073741824,"allocated":145014784,"data_stored":144345527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.2490000000000001,"15min":1.3049999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.6439999999999999,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.135,"15min":1.145},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":2.9950000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":0.84099999999999997}]},{"osd":1,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.95999999999999996,"15min":0.95799999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.9890000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":19.103999999999999,"5min":4.9299999999999997,"15min":2.5680000000000001},"min":{"1min":0.75600000000000001,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":372.50799999999998,"5min":372.50799999999998,"15min":372.50799999999998},"last":0.92800000000000005}]},{"osd":2,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2,"15min":1.196},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":3.036,"5min":3.036,"15min":3.036},"last":0.502},{"interface":"front","average":{"1min":1.093,"5min":1.236,"15min":1.26},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.7440000000000002,"5min":4.0529999999999999,"15min":4.0529999999999999},"last":6.0490000000000004}]},{"osd":3,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.4870000000000001,"15min":1.5129999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":3.46,"5min":3.46,"15min":3.46},"last":0.878},{"interface":"front","average":{"1min":1.268,"5min":1.3640000000000001,"15min":1.3799999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":3.1970000000000001,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.72299999999999998}]},{"osd":4,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.5880000000000001,"15min":1.6379999999999999},"min":{"1min":0.72499999999999998,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.7999999999999998,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.653,"5min":1.6679999999999999,"15min":1.6699999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":3.694,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.2629999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.3260000000000001,"15min":1.351},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.6360000000000001,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.393,"5min":1.6910000000000001,"15min":1.7410000000000001},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.8279999999999998,"5min":5.327,"15min":5.327},"last":1.2130000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.446,"15min":1.4370000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.7799999999999998,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.232},{"interface":"front","average":{"1min":1.4730000000000001,"5min":14.955,"15min":17.202000000000002},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":3.7309999999999999,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.2470000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1301376,"kb_used_data":252792,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93070464,"statfs":{"total":96636764160,"available":95304155136,"internally_reserved":1073741824,"allocated":258859008,"data_stored":258189751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,2,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":27.559999999999999,"15min":31.977},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.1820000000000004,"5min":497.04000000000002,"15min":497.04000000000002},"last":0.96999999999999997},{"interface":"front","average":{"1min":44.500999999999998,"5min":9.8049999999999997,"15min":4.0220000000000002},"min":{"1min":0.64200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":993.84500000000003,"5min":993.84500000000003,"15min":993.84500000000003},"last":0.59299999999999997}]},{"osd":2,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":0.98799999999999999,"15min":0.92800000000000005},"min":{"1min":0.57399999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":4.585,"5min":4.585,"15min":4.585},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.2410000000000001,"15min":1.216},"min":{"1min":0.51800000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":4.149,"5min":4.149,"15min":4.149},"last":0.77400000000000002}]},{"osd":3,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.962,"5min":1.4410000000000001,"15min":1.3540000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":10.603999999999999,"5min":10.603999999999999,"15min":10.603999999999999},"last":1.357},{"interface":"front","average":{"1min":2.1909999999999998,"5min":1.601,"15min":1.5029999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":10.638999999999999,"5min":10.638999999999999,"15min":10.638999999999999},"last":1.4059999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":2.1869999999999998,"5min":1.464,"15min":1.3440000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":10.69,"5min":10.69,"15min":10.69},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.264,"15min":1.2210000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":4.9500000000000002,"5min":4.9500000000000002,"15min":4.9500000000000002},"last":0.92100000000000004}]},{"osd":5,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":2.0630000000000002,"5min":10.819000000000001,"15min":12.278},"min":{"1min":0.73399999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":10.720000000000001,"5min":213.63800000000001,"15min":213.63800000000001},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.258,"15min":1.212},"min":{"1min":0.82899999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":4.6269999999999998,"5min":4.6269999999999998,"15min":4.6269999999999998},"last":1.0800000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:08:24 2020","interfaces":[{"interface":"back","average":{"1min":1.7589999999999999,"5min":1.593,"15min":1.5649999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":4.5259999999999998,"5min":4.5259999999999998,"15min":4.5259999999999998},"last":1.1919999999999999},{"interface":"front","average":{"1min":2.3690000000000002,"5min":1.5620000000000001,"15min":1.4279999999999999},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":10.792,"5min":10.792,"15min":10.792},"last":207.137}]},{"osd":7,"last update":"Fri Aug 7 15:08:18 2020","interfaces":[{"interface":"back","average":{"1min":1.8129999999999999,"5min":1.6539999999999999,"15min":1.6279999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":4.5549999999999997,"5min":4.5549999999999997,"15min":4.5549999999999997},"last":0.94099999999999995},{"interface":"front","average":{"1min":2.2719999999999998,"5min":1.6379999999999999,"15min":1.532},"min":{"1min":0.998,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":10.752000000000001,"5min":10.752000000000001,"15min":10.752000000000001},"last":1.4630000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1168944,"kb_used_data":120360,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93202896,"statfs":{"total":96636764160,"available":95439765504,"internally_reserved":1073741824,"allocated":123248640,"data_stored":122579383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:08:24 2020","interfaces":[{"interface":"back","average":{"1min":81.266000000000005,"5min":17.033999999999999,"15min":6.3289999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":845.57100000000003,"5min":845.57100000000003,"15min":845.57100000000003},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.083,"5min":0.96299999999999997,"15min":0.94299999999999995},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":5.5970000000000004,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":4.8209999999999997}]},{"osd":2,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.0069999999999999,"15min":0.95399999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":4.1600000000000001,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.0049999999999999,"5min":26.393000000000001,"15min":30.623999999999999},"min":{"1min":0.54200000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":2.1280000000000001,"5min":606.18499999999995,"15min":606.18499999999995},"last":4.7889999999999997}]},{"osd":3,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.0549999999999999,"15min":0.97499999999999998},"min":{"1min":0.64000000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":3.1779999999999999,"5min":3.1779999999999999,"15min":3.1779999999999999},"last":1.087},{"interface":"front","average":{"1min":1.927,"5min":1.319,"15min":1.218},"min":{"1min":0.78700000000000003,"5min":0.621,"15min":0.621},"max":{"1min":4.21,"5min":4.21,"15min":4.21},"last":1.0389999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.238,"15min":1.1899999999999999},"min":{"1min":0.73899999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":3.0870000000000002,"5min":3.0870000000000002,"15min":3.0870000000000002},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.6890000000000001,"5min":1.3440000000000001,"15min":1.286},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":4.0960000000000001,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.97699999999999998}]},{"osd":5,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":39.531999999999996,"5min":8.8879999999999999,"15min":3.7799999999999998},"min":{"1min":0.76600000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":501.267,"5min":501.267,"15min":501.267},"last":4.6470000000000002},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.24,"15min":1.1499999999999999},"min":{"1min":0.874,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":6.0010000000000003,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":1.0620000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.7729999999999999,"5min":7.2450000000000001,"15min":8.157},"min":{"1min":0.81100000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":4.1310000000000002,"5min":124.738,"15min":124.738},"last":1.264},{"interface":"front","average":{"1min":1.47,"5min":15.215999999999999,"15min":17.507000000000001},"min":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":3.7120000000000002,"5min":294.94,"15min":294.94},"last":4.6879999999999997}]},{"osd":7,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.552,"15min":1.5740000000000001},"min":{"1min":0.96199999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.3929999999999998,"5min":6.468,"15min":6.468},"last":4.7169999999999996},{"interface":"front","average":{"1min":1.6879999999999999,"5min":1.446,"15min":1.4059999999999999},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":3.117,"5min":3.117,"15min":3.117},"last":1.135}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1119384,"kb_used_data":70800,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93252456,"statfs":{"total":96636764160,"available":95490514944,"internally_reserved":1073741824,"allocated":72499200,"data_stored":71829943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.153,"15min":1.129},"min":{"1min":0.505,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":4.4909999999999997,"5min":4.4909999999999997,"15min":4.4909999999999997},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.284,"5min":1.036,"15min":0.995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":3.6520000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":2.9470000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":2.5219999999999998,"5min":1.3779999999999999,"15min":1.1870000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":18.672000000000001,"5min":18.672000000000001,"15min":18.672000000000001},"last":4.1660000000000004},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.357,"15min":1.359},"min":{"1min":0.70699999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":5.1130000000000004,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":0.80700000000000005}]},{"osd":3,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.216,"15min":1.1799999999999999},"min":{"1min":0.65300000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":2.8740000000000001},{"interface":"front","average":{"1min":1.419,"5min":1.528,"15min":1.546},"min":{"1min":0.59999999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":5.3339999999999996,"5min":8.2210000000000001,"15min":8.2210000000000001},"last":0.76700000000000002}]},{"osd":4,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":2.3319999999999999,"5min":1.3540000000000001,"15min":1.1910000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":19.783000000000001,"5min":19.783000000000001,"15min":19.783000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.5589999999999999,"15min":1.615},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.891,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":0.97799999999999998}]},{"osd":5,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.2789999999999999,"15min":1.2669999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.7730000000000001,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":2.323},{"interface":"front","average":{"1min":1.5249999999999999,"5min":20.725000000000001,"15min":23.925000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":4.4790000000000001,"5min":415.40699999999998,"15min":415.40699999999998},"last":2.9140000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:08:25 2020","interfaces":[{"interface":"back","average":{"1min":2.5089999999999999,"5min":1.478,"15min":1.306},"min":{"1min":0.89000000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":20.035,"5min":20.035,"15min":20.035},"last":1.4219999999999999},{"interface":"front","average":{"1min":44.033000000000001,"5min":9.8629999999999995,"15min":4.1680000000000001},"min":{"1min":0.93700000000000006,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":751.70000000000005,"5min":751.70000000000005,"15min":751.70000000000005},"last":2.976}]},{"osd":7,"last update":"Fri Aug 7 15:08:19 2020","interfaces":[{"interface":"back","average":{"1min":3.5409999999999999,"5min":2.101,"15min":1.861},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":35.548999999999999,"5min":35.548999999999999,"15min":35.548999999999999},"last":1.5529999999999999},{"interface":"front","average":{"1min":3.4460000000000002,"5min":1.8859999999999999,"15min":1.6259999999999999},"min":{"1min":1.004,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":35.646999999999998,"5min":35.646999999999998,"15min":35.646999999999998},"last":3.0030000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1171152,"kb_used_data":122568,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93200688,"statfs":{"total":96636764160,"available":95437504512,"internally_reserved":1073741824,"allocated":125509632,"data_stored":124840375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.044,"5min":1.411,"15min":1.3049999999999999},"min":{"1min":0.55600000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":8.6020000000000003,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":1.8500000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.599,"15min":1.6240000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":4.9119999999999999,"5min":13.067,"15min":13.067},"last":0.74099999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":16.382999999999999,"5min":4.7949999999999999,"15min":2.8639999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":270.68700000000001,"5min":270.68700000000001,"15min":270.68700000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":2.7210000000000001,"5min":2.153,"15min":2.0590000000000002},"min":{"1min":0.71599999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":14.349,"5min":14.349,"15min":14.349},"last":0.69199999999999995}]},{"osd":2,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":1.7470000000000001,"5min":1.2669999999999999,"15min":1.1870000000000001},"min":{"1min":0.64900000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":4.9580000000000002,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":1.198},{"interface":"front","average":{"1min":4.5220000000000002,"5min":2.2519999999999998,"15min":1.8740000000000001},"min":{"1min":0.72399999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":51.530000000000001,"5min":51.530000000000001,"15min":51.530000000000001},"last":1.526}]},{"osd":4,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":47.652999999999999,"5min":10.888,"15min":4.7599999999999998},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":506.21300000000002,"5min":506.21300000000002,"15min":506.21300000000002},"last":1.802},{"interface":"front","average":{"1min":29.771000000000001,"5min":31.634,"15min":31.943999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":505.738,"5min":677.88599999999997,"15min":677.88599999999997},"last":1.1970000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.012,"5min":1.6160000000000001,"15min":1.55},"min":{"1min":0.96399999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":5.0049999999999999,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.23},{"interface":"front","average":{"1min":2.2210000000000001,"5min":1.859,"15min":1.7989999999999999},"min":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":5.226,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":1.593}]},{"osd":6,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":1.99,"5min":1.623,"15min":1.5620000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":5.0389999999999997,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.9710000000000001,"5min":1.9830000000000001,"15min":1.9850000000000001},"min":{"1min":1.0369999999999999,"5min":0.876,"15min":0.876},"max":{"1min":4.266,"5min":5.6189999999999998,"15min":5.6189999999999998},"last":2.0499999999999998}]},{"osd":7,"last update":"Fri Aug 7 15:08:21 2020","interfaces":[{"interface":"back","average":{"1min":2.0110000000000001,"5min":2.1309999999999998,"15min":2.1509999999999998},"min":{"1min":0.96999999999999997,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":3.79,"5min":5.734,"15min":5.734},"last":2.0110000000000001},{"interface":"front","average":{"1min":2.6200000000000001,"5min":2.105,"15min":2.0190000000000001},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":8.6790000000000003,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":1.9239999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1240144,"kb_used_data":191560,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93131696,"statfs":{"total":96636764160,"available":95366856704,"internally_reserved":1073741824,"allocated":196157440,"data_stored":195488183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":1.125,"15min":1.1619999999999999},"min":{"1min":0.47699999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.3999999999999999,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":0.57999999999999996},{"interface":"front","average":{"1min":1.349,"5min":0.94899999999999995,"15min":0.88200000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":6.3970000000000002,"5min":6.3970000000000002,"15min":6.3970000000000002},"last":0.77400000000000002}]},{"osd":1,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.304,"15min":1.337},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.4380000000000002,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":0.61299999999999999},{"interface":"front","average":{"1min":32.585999999999999,"5min":7.6180000000000003,"15min":3.4569999999999999},"min":{"1min":0.67000000000000004,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":689.846,"5min":689.846,"15min":689.846},"last":0.873}]},{"osd":2,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.46,"15min":1.454},"min":{"1min":0.56399999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":6.8730000000000002,"5min":6.8730000000000002,"15min":6.8730000000000002},"last":2.0899999999999999},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.5669999999999999,"15min":1.532},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":7.1360000000000001,"5min":7.1360000000000001,"15min":7.1360000000000001},"last":2.3159999999999998}]},{"osd":3,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":12.619999999999999,"5min":3.73,"15min":2.2480000000000002},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":247.202,"5min":247.202,"15min":247.202},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.583,"5min":1.3169999999999999,"15min":1.2729999999999999},"min":{"1min":0.67800000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":6.431,"5min":6.431,"15min":6.431},"last":0.72299999999999998}]},{"osd":5,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":46.786999999999999,"5min":10.659000000000001,"15min":4.6379999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":990.71600000000001,"5min":990.71600000000001,"15min":990.71600000000001},"last":2.431},{"interface":"front","average":{"1min":1.748,"5min":1.651,"15min":1.635},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":3.9329999999999998,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":0.90900000000000003}]},{"osd":6,"last update":"Fri Aug 7 15:08:23 2020","interfaces":[{"interface":"back","average":{"1min":1.7070000000000001,"5min":1.7729999999999999,"15min":1.784},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":6.9749999999999996,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.524,"5min":15.592000000000001,"15min":17.936},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":6.7729999999999997,"5min":389.14600000000002,"15min":389.14600000000002},"last":0.99299999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:08:17 2020","interfaces":[{"interface":"back","average":{"1min":13.378,"5min":3.802,"15min":2.206},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":252.798,"5min":252.798,"15min":252.798},"last":0.753},{"interface":"front","average":{"1min":44.676000000000002,"5min":10.252000000000001,"15min":4.5140000000000002},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":501.93000000000001,"5min":501.93000000000001,"15min":501.93000000000001},"last":2.0529999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1194600,"kb_used_data":146016,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93177240,"statfs":{"total":96636764160,"available":95413493760,"internally_reserved":1073741824,"allocated":149520384,"data_stored":148851127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0580000000000001,"15min":1.0700000000000001},"min":{"1min":0.49199999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.5169999999999999,"5min":2.766,"15min":2.766},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.2250000000000001,"5min":0.88900000000000001,"15min":0.83299999999999996},"min":{"1min":0.54900000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":2.3340000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":47.365000000000002,"5min":10.215999999999999,"15min":4.024},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":787.22900000000004,"5min":787.22900000000004,"15min":787.22900000000004},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.071,"5min":1.1000000000000001,"15min":1.105},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.8380000000000001,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":1.2629999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":7.5659999999999998,"15min":8.6679999999999993},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.4950000000000001,"5min":168.56899999999999,"15min":168.56899999999999},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.2929999999999999,"5min":26.763000000000002,"15min":31.007999999999999},"min":{"1min":0.79700000000000004,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":3.2389999999999999,"5min":676.27099999999996,"15min":676.27099999999996},"last":2.665}]},{"osd":3,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.196,"15min":1.1439999999999999},"min":{"1min":0.89000000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":3.5649999999999999,"5min":3.5649999999999999,"15min":3.5649999999999999},"last":2.7829999999999999},{"interface":"front","average":{"1min":43.073,"5min":9.5850000000000009,"15min":4.0030000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":712.98699999999997,"5min":712.98699999999997,"15min":712.98699999999997},"last":1.2889999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":40.962000000000003,"15min":47.582000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.4980000000000002,"5min":530.30799999999999,"15min":530.30799999999999},"last":1.9239999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.278,"15min":1.296},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.6259999999999999,"5min":3.1080000000000001,"15min":3.1080000000000001},"last":1.819}]},{"osd":6,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":40.569000000000003,"15min":47.128999999999998},"min":{"1min":0.76800000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.6699999999999999,"5min":754.31299999999999,"15min":754.31299999999999},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":21.050999999999998,"15min":24.361000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.7809999999999999,"5min":517.26700000000005,"15min":517.26700000000005},"last":1.5900000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:08:22 2020","interfaces":[{"interface":"back","average":{"1min":1.5469999999999999,"5min":1.3560000000000001,"15min":1.3240000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":3.9079999999999999,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":2.8940000000000001},{"interface":"front","average":{"1min":1.411,"5min":1.5349999999999999,"15min":1.556},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":3.9390000000000001,"5min":3.9390000000000001,"15min":3.9390000000000001},"last":1.7070000000000001}]}]}]}} 2020-08-07T15:09:12.585 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2020-08-07T15:09:12.586 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:09:12.616 INFO:tasks.thrashosds.thrasher:Reweighting osd 3 to 0.5298903471145886 2020-08-07T15:09:12.631 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 3 0.5298903471145886 2020-08-07T15:09:12.633 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:09:12.649 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:09:12.649 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:09:12.650 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:09:12.650 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.650 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:15576 3.e e27/20 3:727724a5:::benchmark_data_smithi074_35447_object1946:head v 27'1032, mlcod=27'1023)", 2020-08-07T15:09:12.650 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:09:12.650 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 145.92127623499999, 2020-08-07T15:09:12.651 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.46091973000000003, 2020-08-07T15:09:12.651 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:12.651 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:12.651 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:12.651 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.652 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:12.652 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:09:12.652 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:12.652 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.652 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.653 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:12.653 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:09:12.653 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9592000000000001e-05 2020-08-07T15:09:12.653 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.653 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.658 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:12.658 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669542+0000", 2020-08-07T15:09:12.659 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.113e-06 2020-08-07T15:09:12.659 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.659 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.659 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:12.659 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669547+0000", 2020-08-07T15:09:12.660 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45151617799999999 2020-08-07T15:09:12.660 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.660 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.660 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:12.660 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121064+0000", 2020-08-07T15:09:12.661 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.377e-06 2020-08-07T15:09:12.661 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.661 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.693 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:12.693 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121067+0000", 2020-08-07T15:09:12.693 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9195e-05 2020-08-07T15:09:12.693 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.693 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.694 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:12.694 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121106+0000", 2020-08-07T15:09:12.694 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7509000000000001e-05 2020-08-07T15:09:12.694 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.694 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.695 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:12.695 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121124+0000", 2020-08-07T15:09:12.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0091783100000000003 2020-08-07T15:09:12.695 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.696 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:12.696 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.130302+0000", 2020-08-07T15:09:12.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000100456 2020-08-07T15:09:12.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:12.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.130402+0000", 2020-08-07T15:09:12.697 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.46091973000000003 2020-08-07T15:09:12.697 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:12.697 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:12.702 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:12.703 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.703 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:24264 3.8 e28/20 3:1d3fdbda:::benchmark_data_smithi074_35447_object3032:head v 28'1520, mlcod=28'1515)", 2020-08-07T15:09:12.703 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:09:12.703 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 140.20764619799999, 2020-08-07T15:09:12.704 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41856254199999998, 2020-08-07T15:09:12.704 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:12.704 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:12.704 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:12.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.705 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:12.705 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:09:12.705 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:12.705 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.705 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.706 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:12.706 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:09:12.706 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1348999999999998e-05 2020-08-07T15:09:12.706 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.706 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.707 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:12.707 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383174+0000", 2020-08-07T15:09:12.707 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6579999999999996e-06 2020-08-07T15:09:12.739 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.740 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.740 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:12.740 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383181+0000", 2020-08-07T15:09:12.740 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41194352699999998 2020-08-07T15:09:12.740 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.741 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.741 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:12.741 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.795124+0000", 2020-08-07T15:09:12.741 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5729999999999999e-06 2020-08-07T15:09:12.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.742 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.742 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:12.742 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.795128+0000", 2020-08-07T15:09:12.742 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002170384 2020-08-07T15:09:12.742 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.743 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:12.743 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.797298+0000", 2020-08-07T15:09:12.743 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5412e-05 2020-08-07T15:09:12.743 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:12.744 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.797324+0000", 2020-08-07T15:09:12.744 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042943360000000002 2020-08-07T15:09:12.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.745 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:12.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.801618+0000", 2020-08-07T15:09:12.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7302999999999997e-05 2020-08-07T15:09:12.745 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.763 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:12.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.801675+0000", 2020-08-07T15:09:12.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41856254199999998 2020-08-07T15:09:12.764 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:12.764 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:12.764 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:12.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.765 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:93920 3.6 e56/26 ondisk, result = 0)", 2020-08-07T15:09:12.765 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:09:12.765 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 93.260282935000006, 2020-08-07T15:09:12.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.397576347, 2020-08-07T15:09:12.766 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:12.766 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:09:12.766 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:12.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:12.767 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:09:12.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:12.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:12.768 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:09:12.768 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2571999999999999e-05 2020-08-07T15:09:12.768 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.768 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.768 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:12.769 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330509+0000", 2020-08-07T15:09:12.769 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2059999999999998e-06 2020-08-07T15:09:12.769 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.769 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.769 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:12.775 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330514+0000", 2020-08-07T15:09:12.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39740134599999999 2020-08-07T15:09:12.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:12.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727915+0000", 2020-08-07T15:09:12.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.677e-06 2020-08-07T15:09:12.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:12.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727921+0000", 2020-08-07T15:09:12.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5431e-05 2020-08-07T15:09:12.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:12.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727956+0000", 2020-08-07T15:09:12.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5790000000000004e-06 2020-08-07T15:09:12.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.786 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:12.787 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727965+0000", 2020-08-07T15:09:12.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7535999999999997e-05 2020-08-07T15:09:12.787 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.787 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:12.788 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.728052+0000", 2020-08-07T15:09:12.788 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.397576347 2020-08-07T15:09:12.788 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:12.788 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:12.788 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:12.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.789 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.789 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106104 3.0 3:0d7a2a89:::benchmark_data_smithi074_35447_object13262:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:09:12.789 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.252296+0000", 2020-08-07T15:09:12.795 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 85.338463046000001, 2020-08-07T15:09:12.795 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.44182183200000003, 2020-08-07T15:09:12.795 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:12.795 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:12.796 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:12.797 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:09:12.797 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:12.797 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106104 2020-08-07T15:09:12.797 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.798 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:12.798 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.798 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:12.798 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252296+0000", 2020-08-07T15:09:12.799 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:12.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:12.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252296+0000", 2020-08-07T15:09:12.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4728000000000001e-05 2020-08-07T15:09:12.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:12.800 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252321+0000", 2020-08-07T15:09:12.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9529999999999999e-06 2020-08-07T15:09:12.801 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.801 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.801 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:12.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252324+0000", 2020-08-07T15:09:12.802 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5759999999999996e-06 2020-08-07T15:09:12.802 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.802 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.802 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:12.802 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252329+0000", 2020-08-07T15:09:12.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.582e-06 2020-08-07T15:09:12.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:12.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252332+0000", 2020-08-07T15:09:12.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012958519999999999 2020-08-07T15:09:12.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:12.826 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.253628+0000", 2020-08-07T15:09:12.826 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7871e-05 2020-08-07T15:09:12.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.827 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:12.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.253705+0000", 2020-08-07T15:09:12.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017801199999999999 2020-08-07T15:09:12.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:09:12.840 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.253883+0000", 2020-08-07T15:09:12.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001869051 2020-08-07T15:09:12.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.861 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:12.862 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255752+0000", 2020-08-07T15:09:12.862 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3e-07 2020-08-07T15:09:12.862 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.862 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:12.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255753+0000", 2020-08-07T15:09:12.863 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5321000000000001e-05 2020-08-07T15:09:12.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:12.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255808+0000", 2020-08-07T15:09:12.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.091917211999999998 2020-08-07T15:09:12.864 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.865 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:12.865 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.347726+0000", 2020-08-07T15:09:12.865 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.34634211500000001 2020-08-07T15:09:12.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.891 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.891 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:12.891 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694068+0000", 2020-08-07T15:09:12.891 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3030000000000001e-06 2020-08-07T15:09:12.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.895 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.895 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:12.895 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694069+0000", 2020-08-07T15:09:12.896 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8625999999999998e-05 2020-08-07T15:09:12.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.896 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:12.896 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694118+0000", 2020-08-07T15:09:12.897 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.44182183200000003 2020-08-07T15:09:12.898 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:12.898 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:12.898 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:12.898 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.899 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.899 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106104 3.0 e58/20 ondisk, result = 0)", 2020-08-07T15:09:12.899 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.257196+0000", 2020-08-07T15:09:12.899 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 85.333562538999999, 2020-08-07T15:09:12.899 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43692689000000001, 2020-08-07T15:09:12.900 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:12.900 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:09:12.900 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:12.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.900 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:12.900 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257196+0000", 2020-08-07T15:09:12.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:12.901 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.901 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.901 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:12.901 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257196+0000", 2020-08-07T15:09:12.902 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2021999999999999e-05 2020-08-07T15:09:12.918 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.918 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:12.919 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257228+0000", 2020-08-07T15:09:12.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9549999999999997e-06 2020-08-07T15:09:12.919 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:12.920 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257234+0000", 2020-08-07T15:09:12.920 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43671786400000001 2020-08-07T15:09:12.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:12.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693952+0000", 2020-08-07T15:09:12.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7690000000000004e-06 2020-08-07T15:09:12.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.922 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:12.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693957+0000", 2020-08-07T15:09:12.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3294000000000006e-05 2020-08-07T15:09:12.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:12.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694040+0000", 2020-08-07T15:09:12.932 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3436e-05 2020-08-07T15:09:12.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.932 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.932 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:12.933 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694054+0000", 2020-08-07T15:09:12.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9549999999999996e-05 2020-08-07T15:09:12.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.933 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.933 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:12.934 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694123+0000", 2020-08-07T15:09:12.934 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43692689000000001 2020-08-07T15:09:12.934 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:12.934 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:12.934 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:12.934 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.935 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106144 3.6 3:629055a5:::benchmark_data_smithi074_35447_object13267:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:09:12.935 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.280142+0000", 2020-08-07T15:09:12.935 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 85.310617077000003, 2020-08-07T15:09:12.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.57968657199999996, 2020-08-07T15:09:12.936 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:12.936 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:12.936 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:12.936 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:09:12.936 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:12.937 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106144 2020-08-07T15:09:12.937 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.963 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:12.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:12.964 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280142+0000", 2020-08-07T15:09:12.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:12.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:12.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280142+0000", 2020-08-07T15:09:12.985 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.905e-05 2020-08-07T15:09:12.985 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.986 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:12.986 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280191+0000", 2020-08-07T15:09:12.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6729999999999997e-06 2020-08-07T15:09:12.987 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.987 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:12.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280197+0000", 2020-08-07T15:09:12.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0747999999999999e-05 2020-08-07T15:09:12.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:12.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280207+0000", 2020-08-07T15:09:12.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7700000000000001e-06 2020-08-07T15:09:12.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:12.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280212+0000", 2020-08-07T15:09:12.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00060223399999999999 2020-08-07T15:09:12.990 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.990 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.990 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:12.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280814+0000", 2020-08-07T15:09:12.991 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7831000000000006e-05 2020-08-07T15:09:12.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:12.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280912+0000", 2020-08-07T15:09:12.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000137119 2020-08-07T15:09:12.998 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:12.999 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:12.999 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:09:12.999 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.281049+0000", 2020-08-07T15:09:12.999 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023794369999999999 2020-08-07T15:09:13.000 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.000 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.000 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:13.000 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283429+0000", 2020-08-07T15:09:13.000 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.173e-06 2020-08-07T15:09:13.000 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.001 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:13.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283430+0000", 2020-08-07T15:09:13.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5530000000000004e-05 2020-08-07T15:09:13.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:13.022 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283495+0000", 2020-08-07T15:09:13.022 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5649000000000003e-05 2020-08-07T15:09:13.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.023 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:13.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283541+0000", 2020-08-07T15:09:13.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.57624206300000003 2020-08-07T15:09:13.023 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:13.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859783+0000", 2020-08-07T15:09:13.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.421e-06 2020-08-07T15:09:13.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:13.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859785+0000", 2020-08-07T15:09:13.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3874e-05 2020-08-07T15:09:13.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:13.026 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859828+0000", 2020-08-07T15:09:13.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.57968657199999996 2020-08-07T15:09:13.027 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:13.027 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:13.027 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:13.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.028 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106144 3.6 e59/26 ondisk, result = 0)", 2020-08-07T15:09:13.028 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.344536+0000", 2020-08-07T15:09:13.029 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 85.246222735000003, 2020-08-07T15:09:13.030 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51529786099999997, 2020-08-07T15:09:13.030 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:13.030 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:09:13.030 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:13.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:13.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344536+0000", 2020-08-07T15:09:13.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:13.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:13.033 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344536+0000", 2020-08-07T15:09:13.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7266999999999999e-05 2020-08-07T15:09:13.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.034 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:13.034 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344553+0000", 2020-08-07T15:09:13.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1070000000000001e-06 2020-08-07T15:09:13.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.034 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:13.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344557+0000", 2020-08-07T15:09:13.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.515143201 2020-08-07T15:09:13.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:13.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859700+0000", 2020-08-07T15:09:13.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.271e-05 2020-08-07T15:09:13.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:13.058 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859712+0000", 2020-08-07T15:09:13.058 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5436999999999998e-05 2020-08-07T15:09:13.058 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.058 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.058 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:13.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859768+0000", 2020-08-07T15:09:13.059 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1540000000000002e-06 2020-08-07T15:09:13.059 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.059 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.059 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:13.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859776+0000", 2020-08-07T15:09:13.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7985000000000001e-05 2020-08-07T15:09:13.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:13.060 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859834+0000", 2020-08-07T15:09:13.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51529786099999997 2020-08-07T15:09:13.061 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:13.061 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:13.061 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:13.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.070 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.070 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108352 3.8 3:1aac8373:::benchmark_data_smithi074_35447_object13543:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e62)", 2020-08-07T15:09:13.070 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.642093+0000", 2020-08-07T15:09:13.071 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 81.948666046, 2020-08-07T15:09:13.071 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39397409900000002, 2020-08-07T15:09:13.071 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:13.071 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:13.072 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:13.072 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:09:13.072 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:13.072 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108352 2020-08-07T15:09:13.072 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.073 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:13.073 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.073 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:13.073 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642093+0000", 2020-08-07T15:09:13.073 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:13.074 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.074 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.074 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:13.074 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642093+0000", 2020-08-07T15:09:13.074 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9475000000000002e-05 2020-08-07T15:09:13.075 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.075 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.075 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:13.075 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642122+0000", 2020-08-07T15:09:13.075 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0439999999999997e-06 2020-08-07T15:09:13.076 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.076 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.076 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:13.076 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642126+0000", 2020-08-07T15:09:13.076 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9689999999999994e-06 2020-08-07T15:09:13.077 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.077 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.077 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:13.077 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642134+0000", 2020-08-07T15:09:13.126 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1219999999999997e-06 2020-08-07T15:09:13.126 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.126 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.127 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:13.127 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642138+0000", 2020-08-07T15:09:13.127 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011580399999999999 2020-08-07T15:09:13.127 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.127 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.128 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:13.128 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.643296+0000", 2020-08-07T15:09:13.128 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3791999999999999e-05 2020-08-07T15:09:13.145 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.145 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.145 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:13.145 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.643390+0000", 2020-08-07T15:09:13.145 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011882700000000001 2020-08-07T15:09:13.146 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.146 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.146 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4,7", 2020-08-07T15:09:13.146 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.643509+0000", 2020-08-07T15:09:13.151 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034736799999999998 2020-08-07T15:09:13.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.151 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:13.152 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.646983+0000", 2020-08-07T15:09:13.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0077748490000000003 2020-08-07T15:09:13.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:13.153 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.654758+0000", 2020-08-07T15:09:13.153 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000005e-07 2020-08-07T15:09:13.153 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.153 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:13.164 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.654758+0000", 2020-08-07T15:09:13.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4258000000000002e-05 2020-08-07T15:09:13.178 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.178 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.178 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:13.178 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.654802+0000", 2020-08-07T15:09:13.179 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11166224399999999 2020-08-07T15:09:13.179 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.179 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.179 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:13.179 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.766465+0000", 2020-08-07T15:09:13.180 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26946740899999999 2020-08-07T15:09:13.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.180 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.180 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:13.180 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.035932+0000", 2020-08-07T15:09:13.180 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5829999999999999e-06 2020-08-07T15:09:13.181 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.181 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.181 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:13.181 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.035934+0000", 2020-08-07T15:09:13.182 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000133306 2020-08-07T15:09:13.182 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:13.182 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.036067+0000", 2020-08-07T15:09:13.182 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39397409900000002 2020-08-07T15:09:13.204 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:13.204 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:13.205 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:13.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.207 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.207 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145767 3.6 3:62fe5854:::benchmark_data_smithi074_35447_object16158:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:09:13.208 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.654765+0000", 2020-08-07T15:09:13.208 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.935993860000004, 2020-08-07T15:09:13.208 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.62108287699999998, 2020-08-07T15:09:13.208 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:13.209 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:13.209 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:13.210 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:09:13.210 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:13.210 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145767 2020-08-07T15:09:13.210 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.211 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:13.211 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.211 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:13.211 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654765+0000", 2020-08-07T15:09:13.211 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:13.212 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.212 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.212 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:13.212 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654765+0000", 2020-08-07T15:09:13.212 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1977999999999998e-05 2020-08-07T15:09:13.212 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.213 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.213 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:13.213 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654797+0000", 2020-08-07T15:09:13.213 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6799999999999998e-06 2020-08-07T15:09:13.213 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.214 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.214 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:13.214 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654803+0000", 2020-08-07T15:09:13.214 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1263e-05 2020-08-07T15:09:13.214 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.215 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.215 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:13.215 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654814+0000", 2020-08-07T15:09:13.215 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2129999999999999e-06 2020-08-07T15:09:13.215 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.216 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.216 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:13.220 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654819+0000", 2020-08-07T15:09:13.221 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0324999999999997e-05 2020-08-07T15:09:13.221 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.221 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.221 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:13.221 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654869+0000", 2020-08-07T15:09:13.222 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015424199999999999 2020-08-07T15:09:13.222 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.222 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.222 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:13.222 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.655024+0000", 2020-08-07T15:09:13.223 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000163573 2020-08-07T15:09:13.223 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.223 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.223 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:09:13.223 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.655187+0000", 2020-08-07T15:09:13.224 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001341645 2020-08-07T15:09:13.224 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.224 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.224 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:13.231 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.656529+0000", 2020-08-07T15:09:13.231 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9599999999999998e-07 2020-08-07T15:09:13.231 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.232 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.232 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:13.232 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.656530+0000", 2020-08-07T15:09:13.232 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4381e-05 2020-08-07T15:09:13.232 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.233 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.233 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:13.233 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.656584+0000", 2020-08-07T15:09:13.233 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0057298979999999998 2020-08-07T15:09:13.233 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.234 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.234 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:13.234 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.662314+0000", 2020-08-07T15:09:13.234 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.61347776300000001 2020-08-07T15:09:13.234 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.235 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.235 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:13.235 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275792+0000", 2020-08-07T15:09:13.235 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9740000000000001e-06 2020-08-07T15:09:13.235 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.236 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.236 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:13.236 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275794+0000", 2020-08-07T15:09:13.236 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4146e-05 2020-08-07T15:09:13.236 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.237 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.237 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:13.237 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275848+0000", 2020-08-07T15:09:13.237 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.62108287699999998 2020-08-07T15:09:13.237 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:13.252 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:13.264 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:13.264 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.265 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.265 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145801 3.6 3:69e4cb22:::benchmark_data_smithi074_35447_object16192:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:09:13.266 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671123+0000", 2020-08-07T15:09:13.266 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.919635984999999, 2020-08-07T15:09:13.266 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74411769999999999, 2020-08-07T15:09:13.266 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:13.266 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:13.267 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:13.267 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:09:13.267 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:13.267 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145801 2020-08-07T15:09:13.267 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.268 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:13.268 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.268 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:13.268 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671123+0000", 2020-08-07T15:09:13.268 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:13.269 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.269 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.269 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:13.269 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671123+0000", 2020-08-07T15:09:13.285 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3691e-05 2020-08-07T15:09:13.285 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.285 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.286 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:13.286 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671157+0000", 2020-08-07T15:09:13.286 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0249999999999999e-06 2020-08-07T15:09:13.286 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.286 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.287 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:13.287 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671163+0000", 2020-08-07T15:09:13.287 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2339e-05 2020-08-07T15:09:13.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.288 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.288 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:13.288 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671175+0000", 2020-08-07T15:09:13.288 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5249999999999996e-06 2020-08-07T15:09:13.288 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.289 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.289 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:13.289 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671180+0000", 2020-08-07T15:09:13.289 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6826000000000002e-05 2020-08-07T15:09:13.289 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.290 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.290 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:13.290 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671227+0000", 2020-08-07T15:09:13.290 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014152699999999999 2020-08-07T15:09:13.290 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.291 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.291 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:13.291 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671369+0000", 2020-08-07T15:09:13.291 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8021000000000002e-05 2020-08-07T15:09:13.291 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.292 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.292 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:09:13.292 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671467+0000", 2020-08-07T15:09:13.292 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0016293760000000001 2020-08-07T15:09:13.292 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.293 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.293 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:13.293 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673096+0000", 2020-08-07T15:09:13.293 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.048e-06 2020-08-07T15:09:13.293 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.297 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.298 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:13.298 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673097+0000", 2020-08-07T15:09:13.298 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8841999999999998e-05 2020-08-07T15:09:13.298 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.298 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.299 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:13.299 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673146+0000", 2020-08-07T15:09:13.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002829935 2020-08-07T15:09:13.299 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.305 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.306 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:13.306 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675976+0000", 2020-08-07T15:09:13.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73917870399999996 2020-08-07T15:09:13.337 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.337 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.337 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:13.337 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.415155+0000", 2020-08-07T15:09:13.337 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.193e-06 2020-08-07T15:09:13.338 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.338 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.338 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:13.338 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.415157+0000", 2020-08-07T15:09:13.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3647999999999996e-05 2020-08-07T15:09:13.339 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.339 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.339 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:13.339 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.415241+0000", 2020-08-07T15:09:13.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74411769999999999 2020-08-07T15:09:13.340 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:13.340 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:13.340 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:13.340 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.341 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145803 3.0 3:0c939450:::benchmark_data_smithi074_35447_object16194:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:09:13.341 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671678+0000", 2020-08-07T15:09:13.341 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.91908119, 2020-08-07T15:09:13.341 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.882157211, 2020-08-07T15:09:13.390 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:13.390 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:13.390 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:13.391 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:09:13.391 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:13.391 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145803 2020-08-07T15:09:13.391 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.391 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:13.392 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.392 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:13.392 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671678+0000", 2020-08-07T15:09:13.392 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:13.392 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.393 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.393 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:13.393 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671678+0000", 2020-08-07T15:09:13.393 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4097999999999998e-05 2020-08-07T15:09:13.393 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.416 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.417 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:13.417 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671712+0000", 2020-08-07T15:09:13.417 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2439999999999998e-06 2020-08-07T15:09:13.417 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.427 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:13.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671718+0000", 2020-08-07T15:09:13.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2629999999999999e-05 2020-08-07T15:09:13.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:13.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671731+0000", 2020-08-07T15:09:13.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9589999999999998e-06 2020-08-07T15:09:13.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.429 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:13.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671738+0000", 2020-08-07T15:09:13.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2495000000000004e-05 2020-08-07T15:09:13.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:13.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671800+0000", 2020-08-07T15:09:13.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013640700000000001 2020-08-07T15:09:13.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.453 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:13.453 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671937+0000", 2020-08-07T15:09:13.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013672200000000001 2020-08-07T15:09:13.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:09:13.456 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.672073+0000", 2020-08-07T15:09:13.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001179982 2020-08-07T15:09:13.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:13.462 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673253+0000", 2020-08-07T15:09:13.462 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1900000000000001e-07 2020-08-07T15:09:13.462 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.462 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.463 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:13.463 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673254+0000", 2020-08-07T15:09:13.463 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2104000000000001e-05 2020-08-07T15:09:13.463 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.463 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.464 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:13.464 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673306+0000", 2020-08-07T15:09:13.464 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002264369 2020-08-07T15:09:13.464 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.464 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.465 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:13.465 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675571+0000", 2020-08-07T15:09:13.481 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.87820796899999998 2020-08-07T15:09:13.481 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.481 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.481 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:13.482 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553778+0000", 2020-08-07T15:09:13.482 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3629999999999999e-06 2020-08-07T15:09:13.482 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.482 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.482 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:13.483 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553780+0000", 2020-08-07T15:09:13.483 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5050000000000003e-05 2020-08-07T15:09:13.483 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.495 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.495 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:13.496 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553835+0000", 2020-08-07T15:09:13.496 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.882157211 2020-08-07T15:09:13.496 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:13.496 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:13.497 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:13.497 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.497 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.497 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145805 3.8 3:13b8e273:::benchmark_data_smithi074_35447_object16196:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:09:13.497 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671847+0000", 2020-08-07T15:09:13.498 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.918911870000002, 2020-08-07T15:09:13.498 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.340134453, 2020-08-07T15:09:13.498 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:13.498 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:13.498 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:13.499 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:09:13.499 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:13.499 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145805 2020-08-07T15:09:13.499 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.514 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:13.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.533 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:13.534 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671847+0000", 2020-08-07T15:09:13.534 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:13.534 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.534 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:13.535 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671847+0000", 2020-08-07T15:09:13.535 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1139999999999997e-05 2020-08-07T15:09:13.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:13.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671878+0000", 2020-08-07T15:09:13.559 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.784e-06 2020-08-07T15:09:13.560 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.560 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.560 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:13.560 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671884+0000", 2020-08-07T15:09:13.561 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2802999999999999e-05 2020-08-07T15:09:13.561 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.561 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.561 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:13.561 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671897+0000", 2020-08-07T15:09:13.568 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9909999999999999e-06 2020-08-07T15:09:13.569 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.569 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.569 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:13.596 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671903+0000", 2020-08-07T15:09:13.596 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011861 2020-08-07T15:09:13.602 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.606 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:13.606 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.672021+0000", 2020-08-07T15:09:13.607 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017702399999999999 2020-08-07T15:09:13.607 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.607 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.608 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:13.647 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.672198+0000", 2020-08-07T15:09:13.647 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020546400000000001 2020-08-07T15:09:13.648 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.658 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.659 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4,7", 2020-08-07T15:09:13.659 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.672404+0000", 2020-08-07T15:09:13.660 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024385019999999999 2020-08-07T15:09:13.660 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.661 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.661 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:13.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674842+0000", 2020-08-07T15:09:13.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00028594999999999998 2020-08-07T15:09:13.675 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.675 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.676 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:13.676 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675128+0000", 2020-08-07T15:09:13.676 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8100000000000003e-07 2020-08-07T15:09:13.677 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:13.696 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675129+0000", 2020-08-07T15:09:13.697 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7973999999999997e-05 2020-08-07T15:09:13.697 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:13.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675197+0000", 2020-08-07T15:09:13.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032171220000000002 2020-08-07T15:09:13.702 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.703 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:13.718 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678414+0000", 2020-08-07T15:09:13.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33343769299999998 2020-08-07T15:09:13.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:13.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011852+0000", 2020-08-07T15:09:13.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3480000000000001e-06 2020-08-07T15:09:13.730 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.738 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.738 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:13.738 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011853+0000", 2020-08-07T15:09:13.739 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000128467 2020-08-07T15:09:13.739 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.739 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.740 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:13.740 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011981+0000", 2020-08-07T15:09:13.740 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.340134453 2020-08-07T15:09:13.741 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:13.752 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:13.753 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:13.758 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.770 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145806 3.8 3:16669ddb:::benchmark_data_smithi074_35447_object16197:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:09:13.771 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.673992+0000", 2020-08-07T15:09:13.771 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.916767344, 2020-08-07T15:09:13.772 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40687891900000001, 2020-08-07T15:09:13.772 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:13.772 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:13.772 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:13.797 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:09:13.797 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:13.797 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145806 2020-08-07T15:09:13.798 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.798 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:13.798 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.798 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:13.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673992+0000", 2020-08-07T15:09:13.799 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:13.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:13.800 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673992+0000", 2020-08-07T15:09:13.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5429000000000002e-05 2020-08-07T15:09:13.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:13.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674027+0000", 2020-08-07T15:09:13.813 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3199999999999996e-06 2020-08-07T15:09:13.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:13.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674033+0000", 2020-08-07T15:09:13.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2835e-05 2020-08-07T15:09:13.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:13.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674046+0000", 2020-08-07T15:09:13.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9030000000000001e-06 2020-08-07T15:09:13.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:13.826 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674052+0000", 2020-08-07T15:09:13.826 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4181000000000003e-05 2020-08-07T15:09:13.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:13.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674096+0000", 2020-08-07T15:09:13.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00022310599999999999 2020-08-07T15:09:13.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.827 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:13.828 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674319+0000", 2020-08-07T15:09:13.828 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012681300000000001 2020-08-07T15:09:13.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4,7", 2020-08-07T15:09:13.829 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674446+0000", 2020-08-07T15:09:13.874 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002882754 2020-08-07T15:09:13.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:13.875 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.677329+0000", 2020-08-07T15:09:13.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9466e-05 2020-08-07T15:09:13.876 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.876 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:13.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.677418+0000", 2020-08-07T15:09:13.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0299999999999998e-07 2020-08-07T15:09:13.877 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.877 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.877 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:13.877 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.677419+0000", 2020-08-07T15:09:13.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5274000000000001e-05 2020-08-07T15:09:13.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:13.899 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.677464+0000", 2020-08-07T15:09:13.899 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40011127099999999 2020-08-07T15:09:13.899 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.900 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:13.900 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.077576+0000", 2020-08-07T15:09:13.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0031778370000000002 2020-08-07T15:09:13.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.908 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:13.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080754+0000", 2020-08-07T15:09:13.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.92e-07 2020-08-07T15:09:13.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:13.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080754+0000", 2020-08-07T15:09:13.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000116135 2020-08-07T15:09:13.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.910 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:13.910 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080870+0000", 2020-08-07T15:09:13.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40687891900000001 2020-08-07T15:09:13.911 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:13.911 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:13.911 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:13.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.912 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:145767 3.6 e75/26 ondisk, result = 0)", 2020-08-07T15:09:13.912 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.674108+0000", 2020-08-07T15:09:13.912 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.916650920000002, 2020-08-07T15:09:13.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.60174556899999998, 2020-08-07T15:09:13.913 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:13.929 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:09:13.929 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:13.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:13.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674108+0000", 2020-08-07T15:09:13.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:13.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:13.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674108+0000", 2020-08-07T15:09:13.931 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0558000000000001e-05 2020-08-07T15:09:13.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.932 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:13.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674139+0000", 2020-08-07T15:09:13.932 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6620000000000002e-06 2020-08-07T15:09:13.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.932 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.933 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:13.933 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674144+0000", 2020-08-07T15:09:13.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.60156723899999998 2020-08-07T15:09:13.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.933 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.950 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:13.950 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275711+0000", 2020-08-07T15:09:13.950 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4070000000000004e-06 2020-08-07T15:09:13.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:13.951 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275719+0000", 2020-08-07T15:09:13.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3205999999999998e-05 2020-08-07T15:09:13.952 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.952 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.952 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:13.952 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275772+0000", 2020-08-07T15:09:13.952 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0624e-05 2020-08-07T15:09:13.953 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:13.981 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275783+0000", 2020-08-07T15:09:13.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0872999999999997e-05 2020-08-07T15:09:13.995 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:13.996 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:13.996 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:13.996 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275854+0000", 2020-08-07T15:09:13.996 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.60174556899999998 2020-08-07T15:09:13.997 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:13.997 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:14.011 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:14.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.012 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:145803 3.0 e75/20 ondisk, result = 0)", 2020-08-07T15:09:14.012 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.674488+0000", 2020-08-07T15:09:14.012 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.916270937999997, 2020-08-07T15:09:14.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.87935229699999995, 2020-08-07T15:09:14.012 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:14.013 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:09:14.013 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:14.014 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:14.014 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674488+0000", 2020-08-07T15:09:14.014 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:14.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.015 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.015 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:14.015 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674488+0000", 2020-08-07T15:09:14.015 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3062000000000003e-05 2020-08-07T15:09:14.016 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.016 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:14.016 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674521+0000", 2020-08-07T15:09:14.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4911e-05 2020-08-07T15:09:14.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.019 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.019 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:14.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674536+0000", 2020-08-07T15:09:14.070 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.87914350600000002 2020-08-07T15:09:14.071 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.076 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.076 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:14.077 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553679+0000", 2020-08-07T15:09:14.077 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5950000000000001e-06 2020-08-07T15:09:14.077 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.079 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.080 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:14.080 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553687+0000", 2020-08-07T15:09:14.092 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2565999999999996e-05 2020-08-07T15:09:14.092 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.092 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.092 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:14.092 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553760+0000", 2020-08-07T15:09:14.093 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0417999999999999e-05 2020-08-07T15:09:14.093 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.093 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.093 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:14.093 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553770+0000", 2020-08-07T15:09:14.094 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0239000000000003e-05 2020-08-07T15:09:14.094 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.094 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.094 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:14.094 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553840+0000", 2020-08-07T15:09:14.095 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.87935229699999995 2020-08-07T15:09:14.095 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:14.095 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:14.095 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:14.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.096 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.096 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:145805 3.8 e75/60 ondisk, result = 0)", 2020-08-07T15:09:14.110 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.676843+0000", 2020-08-07T15:09:14.110 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.913915467999999, 2020-08-07T15:09:14.111 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.335144041, 2020-08-07T15:09:14.111 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:14.112 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:09:14.112 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:14.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.135 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:14.135 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.676843+0000", 2020-08-07T15:09:14.136 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:14.136 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.136 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.136 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:14.136 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.676843+0000", 2020-08-07T15:09:14.137 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7853e-05 2020-08-07T15:09:14.137 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.137 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.137 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:14.138 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.676861+0000", 2020-08-07T15:09:14.138 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7359999999999999e-06 2020-08-07T15:09:14.138 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.154 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.154 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:14.156 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.676865+0000", 2020-08-07T15:09:14.156 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33484658099999998 2020-08-07T15:09:14.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.157 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:14.157 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011712+0000", 2020-08-07T15:09:14.157 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1613e-05 2020-08-07T15:09:14.157 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.157 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.158 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:14.158 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011723+0000", 2020-08-07T15:09:14.158 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011038 2020-08-07T15:09:14.158 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.158 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.216 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:14.216 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011834+0000", 2020-08-07T15:09:14.217 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0097e-05 2020-08-07T15:09:14.234 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.234 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.235 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:14.239 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011844+0000", 2020-08-07T15:09:14.239 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000143781 2020-08-07T15:09:14.239 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.239 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.240 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:14.240 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011987+0000", 2020-08-07T15:09:14.240 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.335144041 2020-08-07T15:09:14.240 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:14.241 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:14.241 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:14.241 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.241 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.269 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:145806 3.8 e75/60 ondisk, result = 0)", 2020-08-07T15:09:14.269 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.678157+0000", 2020-08-07T15:09:14.270 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.912601416000001, 2020-08-07T15:09:14.270 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.402718152, 2020-08-07T15:09:14.270 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:14.270 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:09:14.272 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:14.273 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:14.273 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678157+0000", 2020-08-07T15:09:14.294 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:14.295 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.295 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.295 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:14.296 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678157+0000", 2020-08-07T15:09:14.296 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8496e-05 2020-08-07T15:09:14.296 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.297 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.304 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:14.304 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678176+0000", 2020-08-07T15:09:14.343 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.676e-06 2020-08-07T15:09:14.344 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.345 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.345 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:14.345 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678180+0000", 2020-08-07T15:09:14.346 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40250966999999999 2020-08-07T15:09:14.346 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.346 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.347 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:14.347 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080689+0000", 2020-08-07T15:09:14.347 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7110000000000003e-06 2020-08-07T15:09:14.348 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.348 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.348 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:14.348 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080694+0000", 2020-08-07T15:09:14.349 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4122999999999998e-05 2020-08-07T15:09:14.349 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.349 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.349 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:14.351 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080738+0000", 2020-08-07T15:09:14.351 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5809999999999999e-06 2020-08-07T15:09:14.352 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.352 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.352 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:14.352 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080746+0000", 2020-08-07T15:09:14.353 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012989499999999999 2020-08-07T15:09:14.353 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.353 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.353 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:14.354 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080876+0000", 2020-08-07T15:09:14.354 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.402718152 2020-08-07T15:09:14.356 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:14.356 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:14.357 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:14.357 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.357 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.358 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42288 4.2 4:4cfde355:::benchmark_data_smithi074_44586_object5285:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:09:14.358 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.115516+0000", 2020-08-07T15:09:14.359 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 12.475243318, 2020-08-07T15:09:14.359 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.53902283500000003, 2020-08-07T15:09:14.359 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:14.360 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:14.360 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:14.361 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:14.361 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:14.362 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42288 2020-08-07T15:09:14.363 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.363 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:14.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:14.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.115516+0000", 2020-08-07T15:09:14.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:14.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.401 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:14.402 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.115516+0000", 2020-08-07T15:09:14.402 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5785999999999997e-05 2020-08-07T15:09:14.402 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.403 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.423 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:14.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.115571+0000", 2020-08-07T15:09:14.424 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3130000000000004e-06 2020-08-07T15:09:14.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.433 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.433 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:14.434 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.115578+0000", 2020-08-07T15:09:14.434 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2514000000000001e-05 2020-08-07T15:09:14.434 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.435 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.435 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:14.435 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.115590+0000", 2020-08-07T15:09:14.436 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7130000000000002e-06 2020-08-07T15:09:14.436 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.437 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.437 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:14.437 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.115596+0000", 2020-08-07T15:09:14.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0036423369999999998 2020-08-07T15:09:14.462 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.462 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.465 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:14.466 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.119238+0000", 2020-08-07T15:09:14.466 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7732999999999995e-05 2020-08-07T15:09:14.466 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.467 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.467 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:14.467 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.119326+0000", 2020-08-07T15:09:14.467 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000102612 2020-08-07T15:09:14.468 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.468 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.468 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:09:14.468 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.119429+0000", 2020-08-07T15:09:14.469 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001842621 2020-08-07T15:09:14.469 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.469 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.470 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:14.470 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.121271+0000", 2020-08-07T15:09:14.470 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00096829399999999997 2020-08-07T15:09:14.471 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.481 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.482 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:14.482 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.122239+0000", 2020-08-07T15:09:14.482 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.53219874700000003 2020-08-07T15:09:14.483 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.483 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.483 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:14.484 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.654438+0000", 2020-08-07T15:09:14.484 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7290000000000001e-06 2020-08-07T15:09:14.484 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.484 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.485 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:14.485 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.654440+0000", 2020-08-07T15:09:14.485 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0956999999999999e-05 2020-08-07T15:09:14.486 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.486 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.487 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:14.487 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.654501+0000", 2020-08-07T15:09:14.487 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7478999999999998e-05 2020-08-07T15:09:14.488 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:14.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.654538+0000", 2020-08-07T15:09:14.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.53902283500000003 2020-08-07T15:09:14.489 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:14.490 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:14.491 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:14.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.492 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.492 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42720 4.2 4:46db6a58:::benchmark_data_smithi074_44586_object5339:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e114)", 2020-08-07T15:09:14.492 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:01.420605+0000", 2020-08-07T15:09:14.492 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.170154010999999, 2020-08-07T15:09:14.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.328978938, 2020-08-07T15:09:14.493 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:14.493 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:14.494 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:14.494 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:14.494 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:14.495 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42720 2020-08-07T15:09:14.495 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.495 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:14.496 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.496 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:14.496 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.420605+0000", 2020-08-07T15:09:14.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:14.497 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.498 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.498 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:14.499 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.420605+0000", 2020-08-07T15:09:14.499 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3278999999999997e-05 2020-08-07T15:09:14.499 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.500 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.500 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:14.501 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.420648+0000", 2020-08-07T15:09:14.501 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8870000000000002e-06 2020-08-07T15:09:14.501 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.501 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.501 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:14.502 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.420652+0000", 2020-08-07T15:09:14.502 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2867999999999999e-05 2020-08-07T15:09:14.502 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.502 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.502 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:14.503 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.420665+0000", 2020-08-07T15:09:14.503 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6909999999999996e-06 2020-08-07T15:09:14.503 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.504 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.504 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:14.504 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.420670+0000", 2020-08-07T15:09:14.504 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024499790000000001 2020-08-07T15:09:14.504 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.505 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.505 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:14.505 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.423120+0000", 2020-08-07T15:09:14.505 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015649199999999999 2020-08-07T15:09:14.505 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.506 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.506 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:14.506 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.423276+0000", 2020-08-07T15:09:14.506 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012563400000000001 2020-08-07T15:09:14.506 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.507 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.507 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:09:14.507 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.423402+0000", 2020-08-07T15:09:14.507 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002402132 2020-08-07T15:09:14.507 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.508 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.508 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:14.508 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.425804+0000", 2020-08-07T15:09:14.508 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6899999999999997e-07 2020-08-07T15:09:14.508 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.508 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.509 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:14.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.425805+0000", 2020-08-07T15:09:14.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6680999999999994e-05 2020-08-07T15:09:14.509 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.510 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:14.510 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.425871+0000", 2020-08-07T15:09:14.511 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015392139999999999 2020-08-07T15:09:14.511 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:14.511 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.427410+0000", 2020-08-07T15:09:14.512 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32205140399999999 2020-08-07T15:09:14.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.512 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.512 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:14.512 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.749462+0000", 2020-08-07T15:09:14.513 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.967e-06 2020-08-07T15:09:14.513 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.513 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.513 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:14.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.749464+0000", 2020-08-07T15:09:14.514 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000120041 2020-08-07T15:09:14.514 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.514 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:14.514 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.749584+0000", 2020-08-07T15:09:14.523 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.328978938 2020-08-07T15:09:14.523 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:14.524 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:14.524 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:14.525 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.525 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.525 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:42720 4.2 e114/81 ondisk, result = 0)", 2020-08-07T15:09:14.525 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:01.425319+0000", 2020-08-07T15:09:14.529 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.165440166, 2020-08-07T15:09:14.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32427655300000002, 2020-08-07T15:09:14.529 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:14.529 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:09:14.530 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:14.530 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.530 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:14.530 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.425319+0000", 2020-08-07T15:09:14.530 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:14.531 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.531 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.531 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:14.531 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.425319+0000", 2020-08-07T15:09:14.532 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6407999999999999e-05 2020-08-07T15:09:14.532 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.532 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.533 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:14.533 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.425335+0000", 2020-08-07T15:09:14.533 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5550000000000001e-06 2020-08-07T15:09:14.533 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.533 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.534 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:14.534 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.425338+0000", 2020-08-07T15:09:14.541 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32394559000000001 2020-08-07T15:09:14.541 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:14.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.749283+0000", 2020-08-07T15:09:14.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5123000000000001e-05 2020-08-07T15:09:14.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.546 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.546 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:14.546 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.749298+0000", 2020-08-07T15:09:14.547 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012721999999999999 2020-08-07T15:09:14.547 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.547 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:14.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.749426+0000", 2020-08-07T15:09:14.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8253999999999998e-05 2020-08-07T15:09:14.548 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.548 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:14.548 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.749444+0000", 2020-08-07T15:09:14.549 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015140299999999999 2020-08-07T15:09:14.549 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.549 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.549 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:14.550 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.749595+0000", 2020-08-07T15:09:14.550 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32427655300000002 2020-08-07T15:09:14.551 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:14.551 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:14.551 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:14.551 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:14.551 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:14.552 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:09:14.552 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:09:14.553 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:12.631+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:14.554 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:09:14.556 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:09:14.558 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 47 16 59427 59411 9.87458 12.5312 0.00510816 0.0126549 2020-08-07T15:09:14.558 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 48 16 61001 60985 9.92499 12.2969 0.00327024 0.0125904 2020-08-07T15:09:14.558 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:14.553+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:14.734 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:14.729+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:14.791 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:09:14.791 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:09:14.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.791 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:61445 4.d 4:b0b5fc9d:::benchmark_data_smithi074_44586_object7680:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e121)", 2020-08-07T15:09:14.792 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:14.778875+0000", 2020-08-07T15:09:14.794 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0025502530000000002, 2020-08-07T15:09:14.795 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002582572, 2020-08-07T15:09:14.795 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:14.795 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:09:14.796 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:14.796 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:14.796 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:14.796 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 61445 2020-08-07T15:09:14.796 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.797 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:14.797 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.797 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:14.797 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.778875+0000", 2020-08-07T15:09:14.797 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:14.798 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.798 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.798 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:14.798 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.778875+0000", 2020-08-07T15:09:14.798 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9628e-05 2020-08-07T15:09:14.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:14.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.778925+0000", 2020-08-07T15:09:14.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5520000000000001e-06 2020-08-07T15:09:14.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.818 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:14.818 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.778932+0000", 2020-08-07T15:09:14.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3307e-05 2020-08-07T15:09:14.819 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:14.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.778945+0000", 2020-08-07T15:09:14.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0214000000000001e-05 2020-08-07T15:09:14.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:14.821 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.778955+0000", 2020-08-07T15:09:14.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4858000000000003e-05 2020-08-07T15:09:14.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.822 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:14.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.779000+0000", 2020-08-07T15:09:14.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7937999999999996e-05 2020-08-07T15:09:14.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:14.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.779098+0000", 2020-08-07T15:09:14.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015851999999999999 2020-08-07T15:09:14.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:09:14.838 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.779256+0000", 2020-08-07T15:09:14.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00038101699999999999 2020-08-07T15:09:14.839 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:14.839 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:14.839 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:14.840 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.840 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.840 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:61446 4.d 4:b0b5fc9d:::benchmark_data_smithi074_44586_object7680:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e121)", 2020-08-07T15:09:14.840 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:14.778999+0000", 2020-08-07T15:09:14.840 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0024267529999999998, 2020-08-07T15:09:14.841 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026025879999999999, 2020-08-07T15:09:14.841 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:14.841 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:09:14.841 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:14.841 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:14.848 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:14.848 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 61446 2020-08-07T15:09:14.849 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.849 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:14.849 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.849 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:14.849 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.778999+0000", 2020-08-07T15:09:14.850 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:14.850 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.862 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.873 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:14.873 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.778999+0000", 2020-08-07T15:09:14.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3736999999999997e-05 2020-08-07T15:09:14.874 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.874 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.874 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:14.874 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.779053+0000", 2020-08-07T15:09:14.874 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8880000000000002e-06 2020-08-07T15:09:14.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:14.875 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.779059+0000", 2020-08-07T15:09:14.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1503e-05 2020-08-07T15:09:14.899 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.901 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:14.901 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.779070+0000", 2020-08-07T15:09:14.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8619999999999999e-06 2020-08-07T15:09:14.901 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.901 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.902 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:14.902 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.779076+0000", 2020-08-07T15:09:14.902 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00068211400000000005 2020-08-07T15:09:14.902 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.902 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.903 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:14.903 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.779758+0000", 2020-08-07T15:09:14.903 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000106427 2020-08-07T15:09:14.903 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.904 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:14.904 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.779864+0000", 2020-08-07T15:09:14.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018849000000000001 2020-08-07T15:09:14.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:09:14.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.780053+0000", 2020-08-07T15:09:14.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001054021 2020-08-07T15:09:14.905 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:14.905 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:14.913 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:14.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.914 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:61447 4.d 4:b0b5fc9d:::benchmark_data_smithi074_44586_object7680:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e121)", 2020-08-07T15:09:14.914 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:14.779576+0000", 2020-08-07T15:09:14.914 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0018500190000000001, 2020-08-07T15:09:14.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0021244189999999998, 2020-08-07T15:09:14.915 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:14.915 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:09:14.915 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:14.915 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:14.916 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:14.916 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 61447 2020-08-07T15:09:14.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.916 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:14.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:14.933 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.779576+0000", 2020-08-07T15:09:14.934 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:14.934 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:14.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.779576+0000", 2020-08-07T15:09:14.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3022999999999997e-05 2020-08-07T15:09:14.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:14.936 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.779629+0000", 2020-08-07T15:09:14.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.438e-06 2020-08-07T15:09:14.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:14.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.779635+0000", 2020-08-07T15:09:14.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2679e-05 2020-08-07T15:09:14.937 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.938 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:14.938 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.779648+0000", 2020-08-07T15:09:14.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.144e-06 2020-08-07T15:09:14.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:14.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.779654+0000", 2020-08-07T15:09:14.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010515050000000001 2020-08-07T15:09:14.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.940 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.940 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:14.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.780705+0000", 2020-08-07T15:09:14.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8667999999999999e-05 2020-08-07T15:09:14.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.949 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:14.950 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.780794+0000", 2020-08-07T15:09:14.950 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000160159 2020-08-07T15:09:14.950 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.950 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:09:14.951 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.780954+0000", 2020-08-07T15:09:14.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001378616 2020-08-07T15:09:14.952 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:14.954 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:14.954 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:14.954 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.954 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.955 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:61296 4.d 4:babc9939:::benchmark_data_smithi074_44586_object7661:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e121)", 2020-08-07T15:09:14.955 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:14.677405+0000", 2020-08-07T15:09:14.955 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.104020449, 2020-08-07T15:09:14.955 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.10440055600000001, 2020-08-07T15:09:14.955 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:14.956 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:14.956 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:14.956 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:14.956 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:14.956 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 61296 2020-08-07T15:09:14.957 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.957 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:14.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:14.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.677405+0000", 2020-08-07T15:09:14.958 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:14.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:14.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.677405+0000", 2020-08-07T15:09:14.969 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0859999999999998e-05 2020-08-07T15:09:14.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.982 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:14.983 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.677456+0000", 2020-08-07T15:09:14.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7060000000000001e-06 2020-08-07T15:09:14.983 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.983 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.983 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:14.984 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.677462+0000", 2020-08-07T15:09:14.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1409999999999999e-05 2020-08-07T15:09:14.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:14.984 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:14.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.677473+0000", 2020-08-07T15:09:14.985 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0080000000000002e-06 2020-08-07T15:09:14.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:14.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:15.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.677478+0000", 2020-08-07T15:09:15.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010491459999999999 2020-08-07T15:09:15.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.042 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:15.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.678527+0000", 2020-08-07T15:09:15.042 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000100018 2020-08-07T15:09:15.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:15.043 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.678627+0000", 2020-08-07T15:09:15.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017861400000000001 2020-08-07T15:09:15.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.044 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:09:15.044 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.678806+0000", 2020-08-07T15:09:15.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013869419999999999 2020-08-07T15:09:15.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:15.045 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.680193+0000", 2020-08-07T15:09:15.045 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0200000000000001e-07 2020-08-07T15:09:15.045 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.072 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.072 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:15.073 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.680194+0000", 2020-08-07T15:09:15.074 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3361999999999994e-05 2020-08-07T15:09:15.074 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.075 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.075 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:15.075 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.680257+0000", 2020-08-07T15:09:15.075 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.010539629 2020-08-07T15:09:15.075 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.076 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.076 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:15.076 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.690797+0000", 2020-08-07T15:09:15.076 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.013391396999999999 2020-08-07T15:09:15.076 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:15.077 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:15.077 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:15.077 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.077 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.077 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:61448 4.d 4:b0b5fc9d:::benchmark_data_smithi074_44586_object7680:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e121)", 2020-08-07T15:09:15.081 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:14.779696+0000", 2020-08-07T15:09:15.081 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0017294599999999999, 2020-08-07T15:09:15.081 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0022280080000000001, 2020-08-07T15:09:15.081 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:15.083 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:09:15.083 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:15.083 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:15.086 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:15.086 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 61448 2020-08-07T15:09:15.086 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.086 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:15.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:15.087 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.779696+0000", 2020-08-07T15:09:15.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:15.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:15.088 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.779696+0000", 2020-08-07T15:09:15.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3631999999999997e-05 2020-08-07T15:09:15.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.089 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.089 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:15.089 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.779750+0000", 2020-08-07T15:09:15.089 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.942e-06 2020-08-07T15:09:15.089 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.090 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.090 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:15.090 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.779756+0000", 2020-08-07T15:09:15.090 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1754e-05 2020-08-07T15:09:15.090 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.108 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.108 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:15.113 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.779768+0000", 2020-08-07T15:09:15.114 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.904e-06 2020-08-07T15:09:15.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.115 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:15.115 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.779772+0000", 2020-08-07T15:09:15.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001451492 2020-08-07T15:09:15.115 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.115 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.116 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:15.116 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.781224+0000", 2020-08-07T15:09:15.116 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4109e-05 2020-08-07T15:09:15.116 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.116 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:15.117 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.781278+0000", 2020-08-07T15:09:15.131 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013429300000000001 2020-08-07T15:09:15.131 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.131 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.132 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:09:15.132 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.781412+0000", 2020-08-07T15:09:15.150 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017161259999999999 2020-08-07T15:09:15.151 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:15.151 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:15.151 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:15.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.152 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:61449 4.d 4:b83cb311:::benchmark_data_smithi074_44586_object7681:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e121)", 2020-08-07T15:09:15.152 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:14.781782+0000", 2020-08-07T15:09:15.152 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4294967295.9996433, 2020-08-07T15:09:15.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000237638, 2020-08-07T15:09:15.153 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:15.153 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:09:15.153 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:15.153 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:15.172 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:15.210 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 61449 2020-08-07T15:09:15.211 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.211 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:15.211 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.211 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:15.211 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.781782+0000", 2020-08-07T15:09:15.212 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:15.220 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.221 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.221 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:15.221 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.781782+0000", 2020-08-07T15:09:15.221 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0893999999999998e-05 2020-08-07T15:09:15.222 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.222 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.222 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:15.222 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.781833+0000", 2020-08-07T15:09:15.222 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1500000000000004e-06 2020-08-07T15:09:15.223 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.223 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.223 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:15.223 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.781839+0000", 2020-08-07T15:09:15.223 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2968e-05 2020-08-07T15:09:15.224 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.235 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.246 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:15.247 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.781852+0000", 2020-08-07T15:09:15.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0279999999999997e-06 2020-08-07T15:09:15.253 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.253 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:15.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.781858+0000", 2020-08-07T15:09:15.254 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3531e-05 2020-08-07T15:09:15.254 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.254 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.254 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:15.260 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.781902+0000", 2020-08-07T15:09:15.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000109481 2020-08-07T15:09:15.261 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.261 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.267 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:15.267 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.782011+0000", 2020-08-07T15:09:15.267 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00022905199999999999 2020-08-07T15:09:15.268 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:15.268 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:15.268 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:15.268 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.268 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.269 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:61450 4.d 4.88cd3c1d (undecoded) ondisk+write+known_if_redirected e121)", 2020-08-07T15:09:15.269 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:14.781900+0000", 2020-08-07T15:09:15.269 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4294967295.9995255, 2020-08-07T15:09:15.269 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020064200000000001, 2020-08-07T15:09:15.270 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:15.283 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:09:15.286 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:15.286 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:15.286 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:15.287 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 61450 2020-08-07T15:09:15.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.294 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:15.294 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.295 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:15.295 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.781900+0000", 2020-08-07T15:09:15.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:15.300 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.300 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.301 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:15.301 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.781900+0000", 2020-08-07T15:09:15.301 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9768999999999997e-05 2020-08-07T15:09:15.301 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.301 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.302 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:15.302 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.781950+0000", 2020-08-07T15:09:15.302 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2809999999999999e-06 2020-08-07T15:09:15.302 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.302 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.303 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:15.303 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.781955+0000", 2020-08-07T15:09:15.303 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0706999999999999e-05 2020-08-07T15:09:15.312 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.312 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.317 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:15.317 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.781966+0000", 2020-08-07T15:09:15.326 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5179999999999998e-06 2020-08-07T15:09:15.326 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.326 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.330 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:15.331 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.781970+0000", 2020-08-07T15:09:15.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0275000000000001e-05 2020-08-07T15:09:15.331 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:15.332 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:15.332 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:15.332 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.332 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.332 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:61451 4.d 4.88cd3c1d (undecoded) ondisk+write+known_if_redirected e121)", 2020-08-07T15:09:15.333 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:14.782004+0000", 2020-08-07T15:09:15.333 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4294967295.9994216, 2020-08-07T15:09:15.333 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016008400000000001, 2020-08-07T15:09:15.333 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:15.333 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:09:15.334 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:15.334 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:15.334 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:15.334 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 61451 2020-08-07T15:09:15.334 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.335 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:15.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.352 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:15.353 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.782004+0000", 2020-08-07T15:09:15.353 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:15.353 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.353 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.353 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:15.363 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.782004+0000", 2020-08-07T15:09:15.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6059e-05 2020-08-07T15:09:15.363 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:15.364 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.782060+0000", 2020-08-07T15:09:15.364 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3179999999999997e-06 2020-08-07T15:09:15.364 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.365 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.365 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:15.365 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.782066+0000", 2020-08-07T15:09:15.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0927999999999999e-05 2020-08-07T15:09:15.365 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.412 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.412 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:15.413 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.782076+0000", 2020-08-07T15:09:15.413 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4109999999999998e-06 2020-08-07T15:09:15.413 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.414 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:15.414 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.782081+0000", 2020-08-07T15:09:15.414 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6716000000000002e-05 2020-08-07T15:09:15.414 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:15.414 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:15.415 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:15.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.471 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.471 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:61452 4.d 4.88cd3c1d (undecoded) ondisk+write+known_if_redirected e121)", 2020-08-07T15:09:15.472 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:14.782115+0000", 2020-08-07T15:09:15.483 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4294967295.999311, 2020-08-07T15:09:15.484 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000110986, 2020-08-07T15:09:15.484 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:15.484 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:09:15.484 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:15.484 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:15.485 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:15.485 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 61452 2020-08-07T15:09:15.485 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.485 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:15.485 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.486 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:15.486 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.782115+0000", 2020-08-07T15:09:15.486 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:15.486 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.486 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.487 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:15.487 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.782115+0000", 2020-08-07T15:09:15.487 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9262999999999999e-05 2020-08-07T15:09:15.487 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.487 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.487 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:15.488 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.782164+0000", 2020-08-07T15:09:15.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9969999999999998e-06 2020-08-07T15:09:15.488 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:15.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.782169+0000", 2020-08-07T15:09:15.511 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0399e-05 2020-08-07T15:09:15.511 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.515 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:15.515 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.782179+0000", 2020-08-07T15:09:15.515 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1579999999999998e-06 2020-08-07T15:09:15.516 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.516 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.516 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:15.517 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.782183+0000", 2020-08-07T15:09:15.517 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8817000000000002e-05 2020-08-07T15:09:15.517 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:15.518 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:15.518 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:15.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.519 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.519 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:61296 4.d e121/98)", 2020-08-07T15:09:15.519 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:14.681805+0000", 2020-08-07T15:09:15.520 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.099620676000000005, 2020-08-07T15:09:15.520 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.100479311, 2020-08-07T15:09:15.572 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:15.573 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:09:15.573 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:15.590 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.614 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:15.615 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.681805+0000", 2020-08-07T15:09:15.659 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:15.660 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.661 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.661 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:15.661 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.681805+0000", 2020-08-07T15:09:15.673 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8952e-05 2020-08-07T15:09:15.674 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.674 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.674 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:15.725 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.681834+0000", 2020-08-07T15:09:15.727 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1760000000000001e-06 2020-08-07T15:09:15.727 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.733 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:15.735 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.681839+0000", 2020-08-07T15:09:15.735 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.100366182 2020-08-07T15:09:15.740 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.741 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.741 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:15.746 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.782205+0000", 2020-08-07T15:09:15.746 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9870000000000001e-06 2020-08-07T15:09:15.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.762 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:15.763 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.782209+0000", 2020-08-07T15:09:15.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.100404297 2020-08-07T15:09:15.764 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:15.764 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:15.764 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:15.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.765 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:61441 4.d e121/98)", 2020-08-07T15:09:15.765 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:14.781599+0000", 2020-08-07T15:09:15.766 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4294967295.9998269, 2020-08-07T15:09:15.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00073776600000000005, 2020-08-07T15:09:15.766 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:15.767 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:09:15.767 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:15.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.768 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:15.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.781599+0000", 2020-08-07T15:09:15.802 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:15.802 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.802 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.802 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:15.810 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.781599+0000", 2020-08-07T15:09:15.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3587999999999998e-05 2020-08-07T15:09:15.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:15.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.781633+0000", 2020-08-07T15:09:15.813 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4089999999999997e-06 2020-08-07T15:09:15.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.814 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:15.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.781640+0000", 2020-08-07T15:09:15.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00058026799999999999 2020-08-07T15:09:15.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.844 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:15.844 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.782220+0000", 2020-08-07T15:09:15.844 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1030000000000002e-06 2020-08-07T15:09:15.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:15.883 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.782223+0000", 2020-08-07T15:09:15.883 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00062436800000000004 2020-08-07T15:09:15.883 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:15.884 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:15.884 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:15.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.884 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.885 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:61441 4.d 4:b0b5fc9d:::benchmark_data_smithi074_44586_object7680:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e121)", 2020-08-07T15:09:15.885 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:14.775396+0000", 2020-08-07T15:09:15.885 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0060300730000000004, 2020-08-07T15:09:15.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.007012211, 2020-08-07T15:09:15.885 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:15.886 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:15.886 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:15.886 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:15.886 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:15.886 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 61441 2020-08-07T15:09:15.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.887 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:15.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:15.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.775396+0000", 2020-08-07T15:09:15.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:15.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:15.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.775396+0000", 2020-08-07T15:09:15.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3904000000000003e-05 2020-08-07T15:09:15.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.901 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:15.901 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.775460+0000", 2020-08-07T15:09:15.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0115000000000001e-05 2020-08-07T15:09:15.901 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.902 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.902 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:15.902 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.775470+0000", 2020-08-07T15:09:15.902 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6495999999999999e-05 2020-08-07T15:09:15.902 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.903 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:15.904 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.775486+0000", 2020-08-07T15:09:15.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8269999999999993e-06 2020-08-07T15:09:15.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.904 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:15.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.775496+0000", 2020-08-07T15:09:15.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5037999999999997e-05 2020-08-07T15:09:15.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:15.906 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.775551+0000", 2020-08-07T15:09:15.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019527999999999999 2020-08-07T15:09:15.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:15.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.775746+0000", 2020-08-07T15:09:15.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00022507899999999999 2020-08-07T15:09:15.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:09:15.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.775971+0000", 2020-08-07T15:09:15.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001703488 2020-08-07T15:09:15.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:15.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.777675+0000", 2020-08-07T15:09:15.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5600000000000004e-07 2020-08-07T15:09:15.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:15.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.777676+0000", 2020-08-07T15:09:15.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3365000000000001e-05 2020-08-07T15:09:15.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.926 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:15.926 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.777739+0000", 2020-08-07T15:09:15.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023434480000000001 2020-08-07T15:09:15.926 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:15.927 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:15.927 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:15.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.927 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:61442 4.d 4:b0b5fc9d:::benchmark_data_smithi074_44586_object7680:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e121)", 2020-08-07T15:09:15.928 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:14.775542+0000", 2020-08-07T15:09:15.928 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0058836829999999998, 2020-08-07T15:09:15.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069753380000000002, 2020-08-07T15:09:15.928 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:15.933 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:15.933 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:15.933 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:15.933 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:15.934 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 61442 2020-08-07T15:09:15.934 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.934 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:15.934 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.934 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:15.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.775542+0000", 2020-08-07T15:09:15.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:15.942 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.942 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:15.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.775542+0000", 2020-08-07T15:09:15.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1914e-05 2020-08-07T15:09:15.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.944 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:15.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.775574+0000", 2020-08-07T15:09:15.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.427e-06 2020-08-07T15:09:15.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:15.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.775577+0000", 2020-08-07T15:09:15.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7290000000000003e-06 2020-08-07T15:09:15.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.946 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:15.946 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.775584+0000", 2020-08-07T15:09:15.946 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7500000000000001e-06 2020-08-07T15:09:15.946 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.946 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.947 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:15.947 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.775588+0000", 2020-08-07T15:09:15.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00082416499999999997 2020-08-07T15:09:15.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.947 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.948 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:15.948 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.776412+0000", 2020-08-07T15:09:15.948 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8675e-05 2020-08-07T15:09:15.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.949 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:15.949 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.776511+0000", 2020-08-07T15:09:15.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018247700000000001 2020-08-07T15:09:15.949 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.957 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:09:15.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.776693+0000", 2020-08-07T15:09:15.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001809609 2020-08-07T15:09:15.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:15.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.778503+0000", 2020-08-07T15:09:15.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2669999999999999e-06 2020-08-07T15:09:15.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:15.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.778504+0000", 2020-08-07T15:09:15.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8922000000000003e-05 2020-08-07T15:09:15.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:15.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.778563+0000", 2020-08-07T15:09:15.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0030209350000000002 2020-08-07T15:09:15.968 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:15.968 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:15.968 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:15.969 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:15.969 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:15.969 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:61443 4.d 4:b0b5fc9d:::benchmark_data_smithi074_44586_object7680:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e121)", 2020-08-07T15:09:15.969 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:14.775615+0000", 2020-08-07T15:09:16.003 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0058102830000000003, 2020-08-07T15:09:16.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0070121710000000002, 2020-08-07T15:09:16.004 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:16.004 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:16.004 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:16.011 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:16.011 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:16.012 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 61443 2020-08-07T15:09:16.012 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:16.012 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:16.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:16.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:16.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.775615+0000", 2020-08-07T15:09:16.022 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:16.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:16.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:16.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:16.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.775615+0000", 2020-08-07T15:09:16.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6162000000000002e-05 2020-08-07T15:09:16.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:16.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:16.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:16.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.775672+0000", 2020-08-07T15:09:16.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1429999999999997e-06 2020-08-07T15:09:16.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:16.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:16.051 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:16.052 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.775677+0000", 2020-08-07T15:09:16.052 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0519e-05 2020-08-07T15:09:16.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:16.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:16.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:16.057 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.775687+0000", 2020-08-07T15:09:16.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7570000000000004e-06 2020-08-07T15:09:16.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:16.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:16.062 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:16.062 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.775692+0000", 2020-08-07T15:09:16.062 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001405817 2020-08-07T15:09:16.062 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:16.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:16.063 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:16.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.777098+0000", 2020-08-07T15:09:16.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6837999999999998e-05 2020-08-07T15:09:16.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:16.064 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:16.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:16.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.777185+0000", 2020-08-07T15:09:16.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000132862 2020-08-07T15:09:16.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:16.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:16.079 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:09:16.079 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.777318+0000", 2020-08-07T15:09:16.080 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012571030000000001 2020-08-07T15:09:16.080 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:16.080 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:16.080 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:16.081 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.778575+0000", 2020-08-07T15:09:16.081 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4900000000000005e-07 2020-08-07T15:09:16.081 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:16.081 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:16.082 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:16.083 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.778575+0000", 2020-08-07T15:09:16.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5016999999999999e-05 2020-08-07T15:09:16.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:16.083 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:16.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:16.084 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.778630+0000", 2020-08-07T15:09:16.084 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0030149669999999999 2020-08-07T15:09:16.084 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:16.084 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:16.085 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:16.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:16.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:16.085 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:61444 4.d 4:b0b5fc9d:::benchmark_data_smithi074_44586_object7680:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e121)", 2020-08-07T15:09:16.085 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:14.776561+0000", 2020-08-07T15:09:16.103 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0048648349999999996, 2020-08-07T15:09:16.104 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006186301, 2020-08-07T15:09:16.147 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:16.147 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:16.148 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:16.148 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:16.148 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:16.152 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 61444 2020-08-07T15:09:16.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:16.153 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:16.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:16.165 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:16.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.776561+0000", 2020-08-07T15:09:16.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:16.166 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:16.166 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:16.166 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:16.167 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.776561+0000", 2020-08-07T15:09:16.167 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3618000000000001e-05 2020-08-07T15:09:16.167 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:16.174 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:16.174 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:16.174 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.776614+0000", 2020-08-07T15:09:16.175 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9550000000000003e-06 2020-08-07T15:09:16.175 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:16.175 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:16.176 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:16.176 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.776621+0000", 2020-08-07T15:09:16.176 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5798999999999999e-05 2020-08-07T15:09:16.177 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:16.177 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:16.177 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:16.178 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.776637+0000", 2020-08-07T15:09:16.178 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6780000000000003e-06 2020-08-07T15:09:16.178 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:16.179 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:16.179 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:16.179 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.776644+0000", 2020-08-07T15:09:16.180 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012164719999999999 2020-08-07T15:09:16.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:16.180 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:16.181 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:16.181 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.777860+0000", 2020-08-07T15:09:16.181 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6618000000000002e-05 2020-08-07T15:09:16.182 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:16.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:16.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:16.183 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.777957+0000", 2020-08-07T15:09:16.183 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017252599999999999 2020-08-07T15:09:16.183 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:16.184 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:16.185 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:09:16.202 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.778130+0000", 2020-08-07T15:09:16.203 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015144900000000001 2020-08-07T15:09:16.203 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:16.203 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:16.204 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:16.204 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.779644+0000", 2020-08-07T15:09:16.204 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0499999999999999e-06 2020-08-07T15:09:16.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:16.205 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:16.206 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:16.206 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.779645+0000", 2020-08-07T15:09:16.206 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6743999999999997e-05 2020-08-07T15:09:16.206 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:16.224 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:16.225 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:16.231 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.779712+0000", 2020-08-07T15:09:16.231 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00315095 2020-08-07T15:09:16.231 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:16.231 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:16.232 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:16.232 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:16.232 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:09:16.232 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 15 2020-08-07T15:09:16.232 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:09:16.236 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:09:16.237 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:09:16.237 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:09:16.237 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:09:16.237 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:09:16.237 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:09:16.262 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:09:16.263 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:09:16.263 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:09:16.264 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:09:16.264 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:09:16.264 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:09:16.265 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:09:16.265 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:09:16.265 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:09:16.279 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:09:16.280 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 49 16 62156 62140 9.90656 9.02344 0.00758569 0.0126143 2020-08-07T15:09:16.280 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:09:16.280 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.3 to 0.52989 (87a6) 2020-08-07T15:09:16.281 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:16.233+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:16.293 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:09:16.295 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 50 16 62352 62336 9.73906 1.53125 0.793328 0.0126154 2020-08-07T15:09:16.357 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:16.353+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:16.434 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:16.434 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:16.443 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:09:16.444 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:16.441+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:16.550 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:16.542+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:16.602 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:16.603 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:16.650 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:09:16.657 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:16.656+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:16.762 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:16.757+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:16.774 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:16.775 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:16.783 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:09:16.861 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:16.859+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:16.914 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:16.915 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:16.923 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:09:16.970 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:16.967+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:17.041 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:17.042 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:17.051 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:09:17.104 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:17.103+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:17.182 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:17.178+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:17.190 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:17.190 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:17.199 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:09:17.278 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 51 16 62352 62336 9.54809 0 - 0.0126154 2020-08-07T15:09:17.289 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:17.288+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:17.359 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:17.360 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:17.368 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:09:17.410 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:17.408+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:17.492 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:17.491+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:17.593 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:17.592+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:17.595 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:17.595 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:17.610 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:09:17.699 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:17.698+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:17.802 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:17.797+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:17.872 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:17.873 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:17.902 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:09:17.915 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:17.914+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:18.018 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:18.014+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:18.097 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:18.097 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:18.119 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:09:18.139 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:18.122+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:18.230 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:18.224+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:18.281 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:09:18.289 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 52 16 63562 63546 9.54625 4.72656 0.0134011 0.013087 2020-08-07T15:09:18.293 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:09:18.337 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:18.338 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:18.339 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:18.336+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:18.352 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:09:18.438 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:18.436+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:18.533 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:09:18.544 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:18.540+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:18.565 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:18.565 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:18.575 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:09:18.582 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:09:18.618 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:09:18.645 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:18.642+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:18.666 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:09:18.696 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:09:18.740 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:09:18.749 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:18.745+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:18.754 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:18.754 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:18.772 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:09:18.782 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:09:18.815 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:09:18.815 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:09:18.816 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:09:18.816 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:09:18.816 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:09:18.817 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:09:18.817 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:09:18.817 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:09:18.817 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:09:18.889 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:18.884+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:18.958 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:18.954+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:18.996 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:09:18.998 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:09:18.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:09:18.999 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:09:18.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:18.999 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:42104 3.d e32/20 3:bcc35439:::benchmark_data_smithi074_35447_object5262:head v 32'2864, mlcod=32'2858)", 2020-08-07T15:09:19.000 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:09:19.000 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 135.75014077899999, 2020-08-07T15:09:19.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28296418899999998, 2020-08-07T15:09:19.005 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:19.005 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:19.005 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:19.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:19.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:09:19.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:19.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:19.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243026+0000", 2020-08-07T15:09:19.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5550000000000002e-05 2020-08-07T15:09:19.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:19.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243081+0000", 2020-08-07T15:09:19.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8139999999999997e-06 2020-08-07T15:09:19.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:19.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243087+0000", 2020-08-07T15:09:19.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27968446800000002 2020-08-07T15:09:19.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:19.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522771+0000", 2020-08-07T15:09:19.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9459999999999997e-06 2020-08-07T15:09:19.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:19.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522776+0000", 2020-08-07T15:09:19.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0778000000000006e-05 2020-08-07T15:09:19.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:19.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522847+0000", 2020-08-07T15:09:19.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0622e-05 2020-08-07T15:09:19.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:19.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.522868+0000", 2020-08-07T15:09:19.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030627580000000001 2020-08-07T15:09:19.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:19.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.525930+0000", 2020-08-07T15:09:19.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9252999999999997e-05 2020-08-07T15:09:19.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:19.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.525990+0000", 2020-08-07T15:09:19.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28296418899999998 2020-08-07T15:09:19.035 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.035 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:19.035 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.051 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86695 3.a 3:5058ba7c:::benchmark_data_smithi074_35447_object10836:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:09:19.051 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:09:19.051 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 104.667838808, 2020-08-07T15:09:19.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31490022699999998, 2020-08-07T15:09:19.052 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:19.052 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:19.052 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:19.052 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:19.053 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:19.053 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86695 2020-08-07T15:09:19.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.053 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:19.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:19.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:09:19.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:19.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:19.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:09:19.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4560000000000005e-05 2020-08-07T15:09:19.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:19.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325392+0000", 2020-08-07T15:09:19.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3180000000000003e-06 2020-08-07T15:09:19.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:19.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325398+0000", 2020-08-07T15:09:19.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.258e-05 2020-08-07T15:09:19.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:19.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325411+0000", 2020-08-07T15:09:19.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6550000000000001e-06 2020-08-07T15:09:19.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:19.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325417+0000", 2020-08-07T15:09:19.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024282879999999998 2020-08-07T15:09:19.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:19.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.327845+0000", 2020-08-07T15:09:19.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1203000000000001e-05 2020-08-07T15:09:19.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:19.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.327936+0000", 2020-08-07T15:09:19.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000186505 2020-08-07T15:09:19.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:09:19.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328123+0000", 2020-08-07T15:09:19.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020062890000000001 2020-08-07T15:09:19.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:19.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.330129+0000", 2020-08-07T15:09:19.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121513 2020-08-07T15:09:19.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:19.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.330250+0000", 2020-08-07T15:09:19.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30989746899999998 2020-08-07T15:09:19.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:19.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640148+0000", 2020-08-07T15:09:19.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2139999999999999e-06 2020-08-07T15:09:19.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:19.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640149+0000", 2020-08-07T15:09:19.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8081999999999998e-05 2020-08-07T15:09:19.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:19.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640197+0000", 2020-08-07T15:09:19.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0550999999999999e-05 2020-08-07T15:09:19.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:19.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640228+0000", 2020-08-07T15:09:19.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31490022699999998 2020-08-07T15:09:19.103 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.103 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:19.103 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.104 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86696 3.a 3:5058ba7c:::benchmark_data_smithi074_35447_object10836:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:09:19.105 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:09:19.105 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 104.667714129, 2020-08-07T15:09:19.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.347347556, 2020-08-07T15:09:19.106 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:19.106 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:19.107 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:19.108 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:19.108 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:19.108 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86696 2020-08-07T15:09:19.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.109 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:19.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:19.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:09:19.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:19.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:19.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:09:19.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0436999999999998e-05 2020-08-07T15:09:19.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:19.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325503+0000", 2020-08-07T15:09:19.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4659999999999998e-06 2020-08-07T15:09:19.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:19.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325508+0000", 2020-08-07T15:09:19.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1177e-05 2020-08-07T15:09:19.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:19.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325519+0000", 2020-08-07T15:09:19.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7809999999999996e-06 2020-08-07T15:09:19.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:19.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325524+0000", 2020-08-07T15:09:19.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002989219 2020-08-07T15:09:19.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:19.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328513+0000", 2020-08-07T15:09:19.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0649000000000006e-05 2020-08-07T15:09:19.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:19.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328604+0000", 2020-08-07T15:09:19.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001652 2020-08-07T15:09:19.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:09:19.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328769+0000", 2020-08-07T15:09:19.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023470639999999998 2020-08-07T15:09:19.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:19.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331116+0000", 2020-08-07T15:09:19.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6099999999999999e-07 2020-08-07T15:09:19.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:19.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331117+0000", 2020-08-07T15:09:19.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9150000000000001e-05 2020-08-07T15:09:19.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:19.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331176+0000", 2020-08-07T15:09:19.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065402581000000001 2020-08-07T15:09:19.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:19.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.396579+0000", 2020-08-07T15:09:19.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27618104599999999 2020-08-07T15:09:19.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:19.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672760+0000", 2020-08-07T15:09:19.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2020-08-07T15:09:19.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:19.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672761+0000", 2020-08-07T15:09:19.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9211000000000001e-05 2020-08-07T15:09:19.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:19.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672800+0000", 2020-08-07T15:09:19.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.347347556 2020-08-07T15:09:19.128 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.128 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:19.128 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.129 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:86695 3.a e52/20 ondisk, result = 0)", 2020-08-07T15:09:19.129 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:09:19.129 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 104.65975743200001, 2020-08-07T15:09:19.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.306823501, 2020-08-07T15:09:19.131 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:19.131 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:19.131 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:19.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:19.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:09:19.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:19.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:19.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:09:19.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.048e-05 2020-08-07T15:09:19.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:19.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333439+0000", 2020-08-07T15:09:19.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9990000000000002e-06 2020-08-07T15:09:19.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:19.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333443+0000", 2020-08-07T15:09:19.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30664195100000002 2020-08-07T15:09:19.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:19.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640085+0000", 2020-08-07T15:09:19.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8559999999999998e-06 2020-08-07T15:09:19.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:19.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640090+0000", 2020-08-07T15:09:19.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5166000000000001e-05 2020-08-07T15:09:19.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:19.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640135+0000", 2020-08-07T15:09:19.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4160000000000001e-06 2020-08-07T15:09:19.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:19.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640142+0000", 2020-08-07T15:09:19.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0632999999999998e-05 2020-08-07T15:09:19.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:19.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640232+0000", 2020-08-07T15:09:19.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.306823501 2020-08-07T15:09:19.141 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.141 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:19.141 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.142 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106232 3.a 3:5599d7e6:::benchmark_data_smithi074_35447_object13278:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:09:19.142 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:09:19.142 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.647559181999995, 2020-08-07T15:09:19.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52578228699999996, 2020-08-07T15:09:19.143 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:19.143 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:19.143 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:19.143 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:19.144 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:19.144 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106232 2020-08-07T15:09:19.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.144 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:19.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:19.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:09:19.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:19.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:19.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:09:19.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4214999999999997e-05 2020-08-07T15:09:19.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:19.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345641+0000", 2020-08-07T15:09:19.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9279999999999993e-06 2020-08-07T15:09:19.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:19.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345649+0000", 2020-08-07T15:09:19.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4391e-05 2020-08-07T15:09:19.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:19.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345664+0000", 2020-08-07T15:09:19.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7170000000000002e-06 2020-08-07T15:09:19.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:19.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345670+0000", 2020-08-07T15:09:19.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.459e-05 2020-08-07T15:09:19.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:19.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345695+0000", 2020-08-07T15:09:19.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6070999999999999e-05 2020-08-07T15:09:19.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:19.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345751+0000", 2020-08-07T15:09:19.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010069300000000001 2020-08-07T15:09:19.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:09:19.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345852+0000", 2020-08-07T15:09:19.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013512299999999999 2020-08-07T15:09:19.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:19.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.347203+0000", 2020-08-07T15:09:19.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0844000000000001e-05 2020-08-07T15:09:19.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:19.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.347274+0000", 2020-08-07T15:09:19.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52399471600000003 2020-08-07T15:09:19.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:19.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871269+0000", 2020-08-07T15:09:19.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.311e-06 2020-08-07T15:09:19.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:19.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871270+0000", 2020-08-07T15:09:19.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3509000000000004e-05 2020-08-07T15:09:19.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:19.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871343+0000", 2020-08-07T15:09:19.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6072000000000002e-05 2020-08-07T15:09:19.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:19.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871389+0000", 2020-08-07T15:09:19.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52578228699999996 2020-08-07T15:09:19.170 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.170 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:19.170 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.171 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106232 3.a e59/20 ondisk, result = 0)", 2020-08-07T15:09:19.172 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:09:19.172 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.644513007, 2020-08-07T15:09:19.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52274359299999995, 2020-08-07T15:09:19.174 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:19.175 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:19.175 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:19.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:19.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:09:19.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:19.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:19.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:09:19.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5011000000000001e-05 2020-08-07T15:09:19.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:19.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348668+0000", 2020-08-07T15:09:19.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1959999999999998e-06 2020-08-07T15:09:19.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:19.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348671+0000", 2020-08-07T15:09:19.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52253456499999995 2020-08-07T15:09:19.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:19.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871205+0000", 2020-08-07T15:09:19.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0230000000000003e-06 2020-08-07T15:09:19.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:19.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871209+0000", 2020-08-07T15:09:19.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0708999999999999e-05 2020-08-07T15:09:19.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:19.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871250+0000", 2020-08-07T15:09:19.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0158e-05 2020-08-07T15:09:19.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:19.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871260+0000", 2020-08-07T15:09:19.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013693099999999999 2020-08-07T15:09:19.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871397+0000", 2020-08-07T15:09:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52274359299999995 2020-08-07T15:09:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:127864 3.a 3:58746ca8:::benchmark_data_smithi074_35447_object15982:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e70)", 2020-08-07T15:09:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:04.819979+0000", 2020-08-07T15:09:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.173187298000002, 2020-08-07T15:09:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27732920700000002, 2020-08-07T15:09:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 127864 2020-08-07T15:09:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.819979+0000", 2020-08-07T15:09:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.819979+0000", 2020-08-07T15:09:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7188000000000001e-05 2020-08-07T15:09:19.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:19.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820006+0000", 2020-08-07T15:09:19.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0940000000000001e-06 2020-08-07T15:09:19.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:19.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820010+0000", 2020-08-07T15:09:19.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0869999999999996e-06 2020-08-07T15:09:19.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:19.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820019+0000", 2020-08-07T15:09:19.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7509999999999999e-06 2020-08-07T15:09:19.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:19.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820024+0000", 2020-08-07T15:09:19.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5089e-05 2020-08-07T15:09:19.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:19.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820049+0000", 2020-08-07T15:09:19.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0068999999999995e-05 2020-08-07T15:09:19.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:19.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820129+0000", 2020-08-07T15:09:19.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013952100000000001 2020-08-07T15:09:19.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:09:19.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.820269+0000", 2020-08-07T15:09:19.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031575290000000001 2020-08-07T15:09:19.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:19.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.823426+0000", 2020-08-07T15:09:19.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4900000000000004e-07 2020-08-07T15:09:19.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:19.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.823427+0000", 2020-08-07T15:09:19.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2324999999999998e-05 2020-08-07T15:09:19.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:19.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:04.823480+0000", 2020-08-07T15:09:19.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.272535737 2020-08-07T15:09:19.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:19.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.096015+0000", 2020-08-07T15:09:19.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012371560000000001 2020-08-07T15:09:19.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:19.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097253+0000", 2020-08-07T15:09:19.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4000000000000001e-07 2020-08-07T15:09:19.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:19.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097253+0000", 2020-08-07T15:09:19.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5071999999999997e-05 2020-08-07T15:09:19.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:19.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:05.097308+0000", 2020-08-07T15:09:19.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27732920700000002 2020-08-07T15:09:19.455 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.455 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:19.455 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.456 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:145807 3.c e75/20 3:39741db6:::benchmark_data_smithi074_35447_object16198:head v 75'9594, mlcod=75'9590)", 2020-08-07T15:09:19.456 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.674086+0000", 2020-08-07T15:09:19.456 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.319080790000001, 2020-08-07T15:09:19.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40268798500000003, 2020-08-07T15:09:19.456 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:19.457 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:19.457 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:19.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:19.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674086+0000", 2020-08-07T15:09:19.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:19.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:19.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674086+0000", 2020-08-07T15:09:19.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3321000000000001e-05 2020-08-07T15:09:19.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:19.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674129+0000", 2020-08-07T15:09:19.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1919999999999999e-06 2020-08-07T15:09:19.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:19.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674136+0000", 2020-08-07T15:09:19.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40193062899999998 2020-08-07T15:09:19.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:19.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076067+0000", 2020-08-07T15:09:19.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3579e-05 2020-08-07T15:09:19.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:19.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076080+0000", 2020-08-07T15:09:19.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3733999999999998e-05 2020-08-07T15:09:19.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:19.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076124+0000", 2020-08-07T15:09:19.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0145e-05 2020-08-07T15:09:19.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:19.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076154+0000", 2020-08-07T15:09:19.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00055768600000000005 2020-08-07T15:09:19.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:19.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076712+0000", 2020-08-07T15:09:19.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1699000000000007e-05 2020-08-07T15:09:19.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:19.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076774+0000", 2020-08-07T15:09:19.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40268798500000003 2020-08-07T15:09:19.471 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.479 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:19.479 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.480 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:144 4.5 4:aed0a56a:::benchmark_data_smithi074_44586_object17:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:09:19.480 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.396584+0000", 2020-08-07T15:09:19.480 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 52.596582550999997, 2020-08-07T15:09:19.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45809661000000002, 2020-08-07T15:09:19.481 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:19.481 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:19.481 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:19.481 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:19.481 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:19.482 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 144 2020-08-07T15:09:19.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.482 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:19.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:19.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396584+0000", 2020-08-07T15:09:19.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:19.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:19.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396584+0000", 2020-08-07T15:09:19.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4195e-05 2020-08-07T15:09:19.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:19.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396628+0000", 2020-08-07T15:09:19.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.157e-06 2020-08-07T15:09:19.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:19.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396631+0000", 2020-08-07T15:09:19.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.832e-06 2020-08-07T15:09:19.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:19.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396638+0000", 2020-08-07T15:09:19.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.745e-06 2020-08-07T15:09:19.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:19.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396641+0000", 2020-08-07T15:09:19.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001659368 2020-08-07T15:09:19.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:19.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.398300+0000", 2020-08-07T15:09:19.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2652000000000007e-05 2020-08-07T15:09:19.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:19.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.398393+0000", 2020-08-07T15:09:19.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016763699999999999 2020-08-07T15:09:19.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,7", 2020-08-07T15:09:19.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.398560+0000", 2020-08-07T15:09:19.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023917719999999999 2020-08-07T15:09:19.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:19.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.400952+0000", 2020-08-07T15:09:19.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999997e-07 2020-08-07T15:09:19.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:19.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.400953+0000", 2020-08-07T15:09:19.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9373999999999998e-05 2020-08-07T15:09:19.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:19.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.401012+0000", 2020-08-07T15:09:19.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003107865 2020-08-07T15:09:19.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:19.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.404120+0000", 2020-08-07T15:09:19.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45049484299999998 2020-08-07T15:09:19.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:19.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854615+0000", 2020-08-07T15:09:19.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.035e-06 2020-08-07T15:09:19.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:19.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854616+0000", 2020-08-07T15:09:19.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4249000000000001e-05 2020-08-07T15:09:19.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:19.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854680+0000", 2020-08-07T15:09:19.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45809661000000002 2020-08-07T15:09:19.513 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.514 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:19.514 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.514 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:30975 4.5 e98/81 ondisk, result = 0)", 2020-08-07T15:09:19.515 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:45.734259+0000", 2020-08-07T15:09:19.515 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.258907454000003, 2020-08-07T15:09:19.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46200962800000001, 2020-08-07T15:09:19.515 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:19.515 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:19.516 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:19.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:19.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.734259+0000", 2020-08-07T15:09:19.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:19.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:19.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.734259+0000", 2020-08-07T15:09:19.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9501000000000001e-05 2020-08-07T15:09:19.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:19.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.734288+0000", 2020-08-07T15:09:19.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5990000000000004e-06 2020-08-07T15:09:19.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:19.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.734293+0000", 2020-08-07T15:09:19.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46181377000000001 2020-08-07T15:09:19.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:19.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.196107+0000", 2020-08-07T15:09:19.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6169999999999998e-06 2020-08-07T15:09:19.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:19.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.196112+0000", 2020-08-07T15:09:19.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5908999999999997e-05 2020-08-07T15:09:19.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:19.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.196158+0000", 2020-08-07T15:09:19.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1104000000000001e-05 2020-08-07T15:09:19.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:19.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.196169+0000", 2020-08-07T15:09:19.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9128000000000004e-05 2020-08-07T15:09:19.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:19.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.196268+0000", 2020-08-07T15:09:19.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46200962800000001 2020-08-07T15:09:19.524 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.524 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:19.525 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.525 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:30976 4.5 e98/81 ondisk, result = 0)", 2020-08-07T15:09:19.525 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:45.736493+0000", 2020-08-07T15:09:19.526 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.256673282000001, 2020-08-07T15:09:19.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54677504399999999, 2020-08-07T15:09:19.526 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:19.526 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:19.526 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:19.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:19.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736493+0000", 2020-08-07T15:09:19.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:19.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:19.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736493+0000", 2020-08-07T15:09:19.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.012e-05 2020-08-07T15:09:19.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:19.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736523+0000", 2020-08-07T15:09:19.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6669999999999998e-06 2020-08-07T15:09:19.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:19.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736527+0000", 2020-08-07T15:09:19.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54657333200000002 2020-08-07T15:09:19.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:19.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283100+0000", 2020-08-07T15:09:19.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.549e-06 2020-08-07T15:09:19.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:19.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283105+0000", 2020-08-07T15:09:19.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1853000000000006e-05 2020-08-07T15:09:19.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:19.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283167+0000", 2020-08-07T15:09:19.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015e-05 2020-08-07T15:09:19.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:19.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283177+0000", 2020-08-07T15:09:19.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1508000000000005e-05 2020-08-07T15:09:19.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:19.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283268+0000", 2020-08-07T15:09:19.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54677504399999999 2020-08-07T15:09:19.538 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.538 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:19.538 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.539 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31528 4.5 4:a883779b:::benchmark_data_smithi074_44586_object3940:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e100)", 2020-08-07T15:09:19.540 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:47.795320+0000", 2020-08-07T15:09:19.540 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.197846685999998, 2020-08-07T15:09:19.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33015256900000001, 2020-08-07T15:09:19.541 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:19.541 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:19.541 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:19.542 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:19.542 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:19.542 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31528 2020-08-07T15:09:19.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.543 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:19.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:19.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.795320+0000", 2020-08-07T15:09:19.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:19.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:19.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.795320+0000", 2020-08-07T15:09:19.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9305000000000003e-05 2020-08-07T15:09:19.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:19.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.795369+0000", 2020-08-07T15:09:19.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1659999999999999e-06 2020-08-07T15:09:19.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:19.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.795375+0000", 2020-08-07T15:09:19.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2284e-05 2020-08-07T15:09:19.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:19.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.795387+0000", 2020-08-07T15:09:19.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.789e-06 2020-08-07T15:09:19.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:19.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.795393+0000", 2020-08-07T15:09:19.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025794009999999998 2020-08-07T15:09:19.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:19.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.797973+0000", 2020-08-07T15:09:19.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011622599999999999 2020-08-07T15:09:19.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:19.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.798089+0000", 2020-08-07T15:09:19.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021524900000000001 2020-08-07T15:09:19.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,7", 2020-08-07T15:09:19.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.798304+0000", 2020-08-07T15:09:19.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042865830000000001 2020-08-07T15:09:19.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:19.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.802591+0000", 2020-08-07T15:09:19.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0088220250000000007 2020-08-07T15:09:19.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:19.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.811413+0000", 2020-08-07T15:09:19.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31390628199999998 2020-08-07T15:09:19.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:19.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125319+0000", 2020-08-07T15:09:19.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.037e-06 2020-08-07T15:09:19.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:19.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125320+0000", 2020-08-07T15:09:19.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0878000000000003e-05 2020-08-07T15:09:19.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:19.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125371+0000", 2020-08-07T15:09:19.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101344 2020-08-07T15:09:19.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:19.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125472+0000", 2020-08-07T15:09:19.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33015256900000001 2020-08-07T15:09:19.756 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.756 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:19.756 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.757 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:31536 4.c e100/99 4:326df2a4:::benchmark_data_smithi074_44586_object3941:head v 100'2008, mlcod=100'1999)", 2020-08-07T15:09:19.758 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:47.804640+0000", 2020-08-07T15:09:19.758 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.188526113999998, 2020-08-07T15:09:19.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32071613199999999, 2020-08-07T15:09:19.759 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:19.759 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:19.760 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:19.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:19.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.804640+0000", 2020-08-07T15:09:19.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:19.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:19.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.804640+0000", 2020-08-07T15:09:19.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9385999999999998e-05 2020-08-07T15:09:19.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:19.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.804690+0000", 2020-08-07T15:09:19.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2319999999999998e-06 2020-08-07T15:09:19.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:19.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.804696+0000", 2020-08-07T15:09:19.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.319733934 2020-08-07T15:09:19.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:19.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.124430+0000", 2020-08-07T15:09:19.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1089999999999997e-06 2020-08-07T15:09:19.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:19.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.124435+0000", 2020-08-07T15:09:19.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9981999999999998e-05 2020-08-07T15:09:19.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:19.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.124475+0000", 2020-08-07T15:09:19.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0684e-05 2020-08-07T15:09:19.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.124506+0000", 2020-08-07T15:09:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00079925600000000003 2020-08-07T15:09:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125305+0000", 2020-08-07T15:09:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1548999999999997e-05 2020-08-07T15:09:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125356+0000", 2020-08-07T15:09:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32071613199999999 2020-08-07T15:09:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:31528 4.5 e100/81 ondisk, result = 0)", 2020-08-07T15:09:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:47.812815+0000", 2020-08-07T15:09:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.180351186999999, 2020-08-07T15:09:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.312664363, 2020-08-07T15:09:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.812815+0000", 2020-08-07T15:09:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.812815+0000", 2020-08-07T15:09:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2224999999999997e-05 2020-08-07T15:09:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:19.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.812847+0000", 2020-08-07T15:09:19.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9460000000000003e-06 2020-08-07T15:09:19.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:19.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.812853+0000", 2020-08-07T15:09:19.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31240245100000003 2020-08-07T15:09:19.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:19.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125256+0000", 2020-08-07T15:09:19.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7509999999999997e-06 2020-08-07T15:09:19.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:19.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125262+0000", 2020-08-07T15:09:19.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9220000000000001e-05 2020-08-07T15:09:19.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:19.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125301+0000", 2020-08-07T15:09:19.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3700000000000001e-06 2020-08-07T15:09:19.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:19.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125310+0000", 2020-08-07T15:09:19.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001694 2020-08-07T15:09:19.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:19.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125479+0000", 2020-08-07T15:09:19.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.312664363 2020-08-07T15:09:19.955 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.955 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:19.955 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:19.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.971 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31528 4.5 4:a883779b:::benchmark_data_smithi074_44586_object3940:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e100)", 2020-08-07T15:09:19.971 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:47.842454+0000", 2020-08-07T15:09:19.972 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.150712599999999, 2020-08-07T15:09:19.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28293349200000001, 2020-08-07T15:09:19.990 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:19.990 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:09:19.991 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:19.991 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:19.991 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:19.991 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31528 2020-08-07T15:09:19.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.992 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:19.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:19.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.842454+0000", 2020-08-07T15:09:19.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:19.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:19.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:19.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:19.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.842454+0000", 2020-08-07T15:09:19.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8015000000000004e-05 2020-08-07T15:09:20.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:20.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.842532+0000", 2020-08-07T15:09:20.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0839000000000001e-05 2020-08-07T15:09:20.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:20.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.842543+0000", 2020-08-07T15:09:20.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2580000000000001e-05 2020-08-07T15:09:20.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:20.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.842565+0000", 2020-08-07T15:09:20.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1283e-05 2020-08-07T15:09:20.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:20.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.842576+0000", 2020-08-07T15:09:20.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0846000000000004e-05 2020-08-07T15:09:20.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:20.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.842667+0000", 2020-08-07T15:09:20.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7760000000000003e-05 2020-08-07T15:09:20.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for ondisk", 2020-08-07T15:09:20.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.842725+0000", 2020-08-07T15:09:20.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28266216900000002 2020-08-07T15:09:20.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:20.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125387+0000", 2020-08-07T15:09:20.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28293349200000001 2020-08-07T15:09:20.044 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:20.045 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:20.045 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:20.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.046 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:31816 4.1 e100/98 4:8a2ebf91:::benchmark_data_smithi074_44586_object3976:head v 100'1872, mlcod=100'1866)", 2020-08-07T15:09:20.046 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.169272+0000", 2020-08-07T15:09:20.046 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 30.823894074999998, 2020-08-07T15:09:20.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38326060099999998, 2020-08-07T15:09:20.046 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:20.047 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:20.047 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:20.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:20.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.169272+0000", 2020-08-07T15:09:20.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:20.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:20.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.169272+0000", 2020-08-07T15:09:20.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.93e-05 2020-08-07T15:09:20.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:20.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.169312+0000", 2020-08-07T15:09:20.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0230000000000003e-06 2020-08-07T15:09:20.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:20.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.169316+0000", 2020-08-07T15:09:20.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38224064899999999 2020-08-07T15:09:20.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:20.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.551556+0000", 2020-08-07T15:09:20.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.673e-06 2020-08-07T15:09:20.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:20.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.551561+0000", 2020-08-07T15:09:20.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0315999999999996e-05 2020-08-07T15:09:20.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:20.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.551631+0000", 2020-08-07T15:09:20.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6877999999999999e-05 2020-08-07T15:09:20.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:20.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.551658+0000", 2020-08-07T15:09:20.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00081916599999999997 2020-08-07T15:09:20.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:20.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.552477+0000", 2020-08-07T15:09:20.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5596000000000001e-05 2020-08-07T15:09:20.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:20.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.552533+0000", 2020-08-07T15:09:20.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38326060099999998 2020-08-07T15:09:20.143 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:20.143 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:20.144 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:20.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.144 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:37048 4.5 4:a403bd70:::benchmark_data_smithi074_44586_object4630:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e106)", 2020-08-07T15:09:20.144 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:54.237330+0000", 2020-08-07T15:09:20.145 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.755836205000001, 2020-08-07T15:09:20.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.474059283, 2020-08-07T15:09:20.146 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:20.146 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:20.147 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:20.147 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:20.147 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:20.147 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 37048 2020-08-07T15:09:20.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.148 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:20.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:20.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237330+0000", 2020-08-07T15:09:20.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:20.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:20.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237330+0000", 2020-08-07T15:09:20.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0713999999999998e-05 2020-08-07T15:09:20.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:20.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237361+0000", 2020-08-07T15:09:20.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.756e-06 2020-08-07T15:09:20.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:20.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237364+0000", 2020-08-07T15:09:20.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.694e-06 2020-08-07T15:09:20.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:20.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237369+0000", 2020-08-07T15:09:20.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3709999999999998e-06 2020-08-07T15:09:20.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:20.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237372+0000", 2020-08-07T15:09:20.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015644700000000001 2020-08-07T15:09:20.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:20.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.238936+0000", 2020-08-07T15:09:20.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9940999999999998e-05 2020-08-07T15:09:20.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:20.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.239026+0000", 2020-08-07T15:09:20.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017423800000000001 2020-08-07T15:09:20.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,7", 2020-08-07T15:09:20.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.239200+0000", 2020-08-07T15:09:20.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041267530000000004 2020-08-07T15:09:20.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:20.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.243327+0000", 2020-08-07T15:09:20.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7700000000000004e-07 2020-08-07T15:09:20.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:20.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.243328+0000", 2020-08-07T15:09:20.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1211999999999997e-05 2020-08-07T15:09:20.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:20.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.243379+0000", 2020-08-07T15:09:20.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075155150000000004 2020-08-07T15:09:20.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:20.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.250894+0000", 2020-08-07T15:09:20.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.460450313 2020-08-07T15:09:20.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:20.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.711345+0000", 2020-08-07T15:09:20.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1990000000000001e-06 2020-08-07T15:09:20.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:20.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.711346+0000", 2020-08-07T15:09:20.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3529999999999998e-05 2020-08-07T15:09:20.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.711389+0000", 2020-08-07T15:09:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.474059283 2020-08-07T15:09:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:20.170 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:20.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.171 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:46032 4.c e117/104 4:3eb0d548:::benchmark_data_smithi074_44586_object5753:head v 117'2968, mlcod=117'2958)", 2020-08-07T15:09:20.171 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:05.163259+0000", 2020-08-07T15:09:20.171 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.829906857999999, 2020-08-07T15:09:20.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31522351999999998, 2020-08-07T15:09:20.172 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:20.172 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:20.173 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:20.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:20.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.163259+0000", 2020-08-07T15:09:20.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:20.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:20.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.163259+0000", 2020-08-07T15:09:20.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9008000000000002e-05 2020-08-07T15:09:20.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:20.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.163318+0000", 2020-08-07T15:09:20.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2689999999999996e-06 2020-08-07T15:09:20.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:20.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.163325+0000", 2020-08-07T15:09:20.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.312753646 2020-08-07T15:09:20.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:20.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.476078+0000", 2020-08-07T15:09:20.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4379999999999999e-06 2020-08-07T15:09:20.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:20.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.476082+0000", 2020-08-07T15:09:20.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2913999999999998e-05 2020-08-07T15:09:20.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:20.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.476175+0000", 2020-08-07T15:09:20.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5939e-05 2020-08-07T15:09:20.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:20.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.476201+0000", 2020-08-07T15:09:20.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022305070000000001 2020-08-07T15:09:20.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:20.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.478431+0000", 2020-08-07T15:09:20.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1799000000000003e-05 2020-08-07T15:09:20.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:20.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.478483+0000", 2020-08-07T15:09:20.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31522351999999998 2020-08-07T15:09:20.186 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:20.186 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:20.186 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:20.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.187 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:60960 4.e 4:73b21c89:::benchmark_data_smithi074_44586_object7619:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e121)", 2020-08-07T15:09:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:14.254254+0000", 2020-08-07T15:09:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.7389123370000004, 2020-08-07T15:09:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.29219231800000001, 2020-08-07T15:09:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:20.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:20.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:20.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:20.189 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 60960 2020-08-07T15:09:20.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.190 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.254254+0000", 2020-08-07T15:09:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.254254+0000", 2020-08-07T15:09:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9551e-05 2020-08-07T15:09:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.254304+0000", 2020-08-07T15:09:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7420000000000003e-06 2020-08-07T15:09:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.254309+0000", 2020-08-07T15:09:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0723000000000001e-05 2020-08-07T15:09:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.254320+0000", 2020-08-07T15:09:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1070000000000004e-06 2020-08-07T15:09:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.254325+0000", 2020-08-07T15:09:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00072346499999999996 2020-08-07T15:09:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.255049+0000", 2020-08-07T15:09:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010703400000000001 2020-08-07T15:09:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.255156+0000", 2020-08-07T15:09:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000179338 2020-08-07T15:09:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:09:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.255335+0000", 2020-08-07T15:09:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041513380000000001 2020-08-07T15:09:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.259486+0000", 2020-08-07T15:09:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9700000000000002e-07 2020-08-07T15:09:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.259487+0000", 2020-08-07T15:09:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3769999999999994e-05 2020-08-07T15:09:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.259550+0000", 2020-08-07T15:09:20.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.069610947000000006 2020-08-07T15:09:20.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:20.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.329161+0000", 2020-08-07T15:09:20.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.217207706 2020-08-07T15:09:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.546369+0000", 2020-08-07T15:09:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.612e-06 2020-08-07T15:09:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.546371+0000", 2020-08-07T15:09:20.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5587999999999995e-05 2020-08-07T15:09:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.546446+0000", 2020-08-07T15:09:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.29219231800000001 2020-08-07T15:09:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:20.216 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:20.216 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:20.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.218 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:60960 4.e e121/98 ondisk, result = 0)", 2020-08-07T15:09:20.218 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:14.257727+0000", 2020-08-07T15:09:20.220 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.7354396660000004, 2020-08-07T15:09:20.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28872502300000003, 2020-08-07T15:09:20.221 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:20.221 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:20.222 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:20.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:20.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.257727+0000", 2020-08-07T15:09:20.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:20.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:20.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.257727+0000", 2020-08-07T15:09:20.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5155e-05 2020-08-07T15:09:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:20.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.257742+0000", 2020-08-07T15:09:20.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4930000000000001e-06 2020-08-07T15:09:20.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.257744+0000", 2020-08-07T15:09:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28856093399999999 2020-08-07T15:09:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.546305+0000", 2020-08-07T15:09:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.045e-06 2020-08-07T15:09:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.546310+0000", 2020-08-07T15:09:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0689999999999998e-05 2020-08-07T15:09:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.546351+0000", 2020-08-07T15:09:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3400000000000004e-06 2020-08-07T15:09:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.546360+0000", 2020-08-07T15:09:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1366000000000006e-05 2020-08-07T15:09:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.546452+0000", 2020-08-07T15:09:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28872502300000003 2020-08-07T15:09:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:20.326 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:20.327 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:09:20.335 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 53 16 65011 64995 9.5797 11.3203 0.00582415 0.0130425 2020-08-07T15:09:20.335 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 54 16 66599 66583 9.63203 12.4062 0.00326137 0.0129719 2020-08-07T15:09:20.336 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:09:20.339 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:20.333+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:20.339 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:09:20.438 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:20.435+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:20.539 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:20.538+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:20.552 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:09:20.553 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:09:20.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.556 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:66696 4.0 4:04cd115d:::benchmark_data_smithi074_44586_object8336:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e126)", 2020-08-07T15:09:20.556 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:20.340922+0000", 2020-08-07T15:09:20.556 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.209918509, 2020-08-07T15:09:20.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.209955064, 2020-08-07T15:09:20.557 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:20.557 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 66696 2020-08-07T15:09:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.340922+0000", 2020-08-07T15:09:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.340922+0000", 2020-08-07T15:09:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8118999999999998e-05 2020-08-07T15:09:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.340971+0000", 2020-08-07T15:09:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9900000000000002e-06 2020-08-07T15:09:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.340977+0000", 2020-08-07T15:09:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1779e-05 2020-08-07T15:09:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.340988+0000", 2020-08-07T15:09:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8799999999999996e-06 2020-08-07T15:09:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.340994+0000", 2020-08-07T15:09:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011116979999999999 2020-08-07T15:09:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.342106+0000", 2020-08-07T15:09:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000124084 2020-08-07T15:09:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.342230+0000", 2020-08-07T15:09:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014733300000000001 2020-08-07T15:09:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,1,3", 2020-08-07T15:09:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.342377+0000", 2020-08-07T15:09:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006015378 2020-08-07T15:09:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.348393+0000", 2020-08-07T15:09:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000147004 2020-08-07T15:09:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.348540+0000", 2020-08-07T15:09:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1099999999999995e-07 2020-08-07T15:09:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.348540+0000", 2020-08-07T15:09:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7929000000000006e-05 2020-08-07T15:09:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.348618+0000", 2020-08-07T15:09:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030249249999999999 2020-08-07T15:09:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.351643+0000", 2020-08-07T15:09:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010720830000000001 2020-08-07T15:09:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:09:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 1 2020-08-07T15:09:20.756 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:09:20.758 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:09:20.759 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:20.756+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:20.856 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:09:20.866 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:20.859+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:20.966 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:20.964+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:20.986 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:09:20.987 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:09:20.987 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:09:20.987 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:09:20.987 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:09:20.999 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:09:21.061 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:09:21.072 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:21.070+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:21.093 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:09:21.125 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:09:21.174 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:09:21.176 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:09:21.177 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:09:21.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:09:21.177 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:09:21.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.179 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:9976 3.e 3:7fb2e2e5:::benchmark_data_smithi074_35447_object1246:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:09:21.179 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:09:21.180 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 158.90689271700001, 2020-08-07T15:09:21.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28420182999999999, 2020-08-07T15:09:21.180 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:21.181 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:21.181 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:21.181 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:21.182 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:21.182 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 9976 2020-08-07T15:09:21.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.183 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:21.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:21.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:09:21.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:21.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:21.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:09:21.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7113e-05 2020-08-07T15:09:21.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:21.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237255+0000", 2020-08-07T15:09:21.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5869999999999998e-06 2020-08-07T15:09:21.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:21.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237262+0000", 2020-08-07T15:09:21.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4467999999999999e-05 2020-08-07T15:09:21.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:21.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237276+0000", 2020-08-07T15:09:21.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0749999999999997e-06 2020-08-07T15:09:21.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:21.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237281+0000", 2020-08-07T15:09:21.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013795109999999999 2020-08-07T15:09:21.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:21.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238661+0000", 2020-08-07T15:09:21.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100309 2020-08-07T15:09:21.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:21.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238761+0000", 2020-08-07T15:09:21.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000141784 2020-08-07T15:09:21.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:09:21.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238903+0000", 2020-08-07T15:09:21.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00070626599999999999 2020-08-07T15:09:21.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:21.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239609+0000", 2020-08-07T15:09:21.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3499999999999996e-07 2020-08-07T15:09:21.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:21.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239610+0000", 2020-08-07T15:09:21.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0846000000000006e-05 2020-08-07T15:09:21.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:21.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239680+0000", 2020-08-07T15:09:21.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022037080000000001 2020-08-07T15:09:21.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:21.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.261718+0000", 2020-08-07T15:09:21.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25961784500000001 2020-08-07T15:09:21.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:21.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521335+0000", 2020-08-07T15:09:21.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2839999999999999e-06 2020-08-07T15:09:21.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:21.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521337+0000", 2020-08-07T15:09:21.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3126999999999994e-05 2020-08-07T15:09:21.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:21.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521400+0000", 2020-08-07T15:09:21.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28420182999999999 2020-08-07T15:09:21.224 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.225 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:21.225 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.225 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15576 3.e 3:727724a5:::benchmark_data_smithi074_35447_object1946:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:09:21.227 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:09:21.228 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 154.477455754, 2020-08-07T15:09:21.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999, 2020-08-07T15:09:21.228 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:21.228 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:21.229 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:21.229 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:21.229 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:21.229 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15576 2020-08-07T15:09:21.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.230 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:21.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:21.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:09:21.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:21.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:21.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:09:21.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7156e-05 2020-08-07T15:09:21.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:21.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666682+0000", 2020-08-07T15:09:21.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0209999999999999e-06 2020-08-07T15:09:21.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:21.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666687+0000", 2020-08-07T15:09:21.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0730999999999999e-05 2020-08-07T15:09:21.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:21.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666698+0000", 2020-08-07T15:09:21.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2020-08-07T15:09:21.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:21.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666702+0000", 2020-08-07T15:09:21.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023978459999999999 2020-08-07T15:09:21.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:21.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669100+0000", 2020-08-07T15:09:21.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9664000000000001e-05 2020-08-07T15:09:21.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:21.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669190+0000", 2020-08-07T15:09:21.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015957100000000001 2020-08-07T15:09:21.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:09:21.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669350+0000", 2020-08-07T15:09:21.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016470149999999999 2020-08-07T15:09:21.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:21.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:09:21.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1299999999999999e-07 2020-08-07T15:09:21.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:21.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:09:21.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5269000000000002e-05 2020-08-07T15:09:21.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:21.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.671022+0000", 2020-08-07T15:09:21.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031414400000000001 2020-08-07T15:09:21.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:21.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.674163+0000", 2020-08-07T15:09:21.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45641417400000001 2020-08-07T15:09:21.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:21.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:09:21.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0399999999999995e-07 2020-08-07T15:09:21.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:21.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:09:21.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3343999999999997e-05 2020-08-07T15:09:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130622+0000", 2020-08-07T15:09:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999 2020-08-07T15:09:21.254 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.254 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:21.254 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.255 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:41431 3.d 3:b5c8a160:::benchmark_data_smithi074_35447_object5178:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:09:21.255 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.791939+0000", 2020-08-07T15:09:21.255 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 138.352151225, 2020-08-07T15:09:21.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28436496, 2020-08-07T15:09:21.255 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:21.256 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:21.256 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:21.256 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:21.256 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:21.256 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 41431 2020-08-07T15:09:21.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.257 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:21.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:21.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791939+0000", 2020-08-07T15:09:21.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:21.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:21.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791939+0000", 2020-08-07T15:09:21.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6304999999999998e-05 2020-08-07T15:09:21.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:21.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791986+0000", 2020-08-07T15:09:21.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6339999999999998e-06 2020-08-07T15:09:21.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:21.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791991+0000", 2020-08-07T15:09:21.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.026e-05 2020-08-07T15:09:21.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:21.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792002+0000", 2020-08-07T15:09:21.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6240000000000001e-06 2020-08-07T15:09:21.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:21.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792006+0000", 2020-08-07T15:09:21.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015600220000000001 2020-08-07T15:09:21.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:21.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.793566+0000", 2020-08-07T15:09:21.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3628999999999994e-05 2020-08-07T15:09:21.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:21.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.793660+0000", 2020-08-07T15:09:21.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000168197 2020-08-07T15:09:21.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:09:21.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.793828+0000", 2020-08-07T15:09:21.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001273251 2020-08-07T15:09:21.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:21.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795101+0000", 2020-08-07T15:09:21.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5899999999999998e-07 2020-08-07T15:09:21.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:21.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795102+0000", 2020-08-07T15:09:21.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4904000000000001e-05 2020-08-07T15:09:21.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:21.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795127+0000", 2020-08-07T15:09:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00670237 2020-08-07T15:09:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801829+0000", 2020-08-07T15:09:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27442024999999998 2020-08-07T15:09:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076249+0000", 2020-08-07T15:09:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999997e-07 2020-08-07T15:09:21.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076250+0000", 2020-08-07T15:09:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4369000000000001e-05 2020-08-07T15:09:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076304+0000", 2020-08-07T15:09:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28436496 2020-08-07T15:09:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:41432 3.d 3:b5c8a160:::benchmark_data_smithi074_35447_object5178:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:09:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:09:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 138.351800529, 2020-08-07T15:09:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.37720701699999998, 2020-08-07T15:09:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 41432 2020-08-07T15:09:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:09:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:09:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4787999999999999e-05 2020-08-07T15:09:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792335+0000", 2020-08-07T15:09:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5999999999999997e-06 2020-08-07T15:09:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:21.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792341+0000", 2020-08-07T15:09:21.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0998e-05 2020-08-07T15:09:21.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:21.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792352+0000", 2020-08-07T15:09:21.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2989999999999996e-06 2020-08-07T15:09:21.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:21.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792357+0000", 2020-08-07T15:09:21.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001852752 2020-08-07T15:09:21.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:21.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794210+0000", 2020-08-07T15:09:21.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5771000000000002e-05 2020-08-07T15:09:21.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:21.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794305+0000", 2020-08-07T15:09:21.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012776399999999999 2020-08-07T15:09:21.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:09:21.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794433+0000", 2020-08-07T15:09:21.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00069673599999999999 2020-08-07T15:09:21.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:21.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795130+0000", 2020-08-07T15:09:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1500000000000001e-07 2020-08-07T15:09:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795130+0000", 2020-08-07T15:09:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7319e-05 2020-08-07T15:09:21.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:21.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795167+0000", 2020-08-07T15:09:21.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27910026700000001 2020-08-07T15:09:21.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:21.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074268+0000", 2020-08-07T15:09:21.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.095150660999999997 2020-08-07T15:09:21.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:21.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169418+0000", 2020-08-07T15:09:21.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.423e-06 2020-08-07T15:09:21.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:21.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169420+0000", 2020-08-07T15:09:21.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7423999999999996e-05 2020-08-07T15:09:21.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:21.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169497+0000", 2020-08-07T15:09:21.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.37720701699999998 2020-08-07T15:09:21.461 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.461 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:21.461 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.462 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:42104 3.d 3:bcc35439:::benchmark_data_smithi074_35447_object5262:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:09:21.463 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:09:21.463 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 137.905168358, 2020-08-07T15:09:21.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28812091400000001, 2020-08-07T15:09:21.464 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:21.464 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:21.464 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:21.464 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:21.464 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:21.465 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42104 2020-08-07T15:09:21.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.465 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:21.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:21.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:09:21.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:09:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9734000000000002e-05 2020-08-07T15:09:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238972+0000", 2020-08-07T15:09:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6910000000000002e-06 2020-08-07T15:09:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:21.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238978+0000", 2020-08-07T15:09:21.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1864e-05 2020-08-07T15:09:21.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:21.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238990+0000", 2020-08-07T15:09:21.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.737e-06 2020-08-07T15:09:21.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:21.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238996+0000", 2020-08-07T15:09:21.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00283943 2020-08-07T15:09:21.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:21.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.241836+0000", 2020-08-07T15:09:21.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4712999999999999e-05 2020-08-07T15:09:21.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:21.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.241930+0000", 2020-08-07T15:09:21.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015328499999999999 2020-08-07T15:09:21.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:09:21.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.242084+0000", 2020-08-07T15:09:21.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016104400000000001 2020-08-07T15:09:21.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:21.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243694+0000", 2020-08-07T15:09:21.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1399999999999997e-07 2020-08-07T15:09:21.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:21.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243695+0000", 2020-08-07T15:09:21.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7800999999999998e-05 2020-08-07T15:09:21.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:21.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243753+0000", 2020-08-07T15:09:21.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090405661999999998 2020-08-07T15:09:21.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:21.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.334158+0000", 2020-08-07T15:09:21.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19283929299999999 2020-08-07T15:09:21.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:21.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.526997+0000", 2020-08-07T15:09:21.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.012e-06 2020-08-07T15:09:21.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:21.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.526998+0000", 2020-08-07T15:09:21.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4737999999999998e-05 2020-08-07T15:09:21.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:21.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.527043+0000", 2020-08-07T15:09:21.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28812091400000001 2020-08-07T15:09:21.483 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.484 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:21.484 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.485 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:86712 3.8 e52/20 3:14925daa:::benchmark_data_smithi074_35447_object10838:head v 52'5488, mlcod=52'5479)", 2020-08-07T15:09:21.485 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.338631+0000", 2020-08-07T15:09:21.485 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 106.80545926000001, 2020-08-07T15:09:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.310229594, 2020-08-07T15:09:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:21.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338631+0000", 2020-08-07T15:09:21.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338631+0000", 2020-08-07T15:09:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6180000000000001e-05 2020-08-07T15:09:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338688+0000", 2020-08-07T15:09:21.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4949999999999996e-06 2020-08-07T15:09:21.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:21.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338694+0000", 2020-08-07T15:09:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30438292099999997 2020-08-07T15:09:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643077+0000", 2020-08-07T15:09:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9389999999999996e-06 2020-08-07T15:09:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643082+0000", 2020-08-07T15:09:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5096999999999999e-05 2020-08-07T15:09:21.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:21.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643127+0000", 2020-08-07T15:09:21.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4323000000000001e-05 2020-08-07T15:09:21.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:21.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643151+0000", 2020-08-07T15:09:21.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055847300000000004 2020-08-07T15:09:21.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:21.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.648736+0000", 2020-08-07T15:09:21.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012490900000000001 2020-08-07T15:09:21.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.648861+0000", 2020-08-07T15:09:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.310229594 2020-08-07T15:09:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.498 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:21.498 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.502 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86792 3.f 3:fc5386ee:::benchmark_data_smithi074_35447_object10848:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:09:21.502 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.395402+0000", 2020-08-07T15:09:21.502 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 106.74868915, 2020-08-07T15:09:21.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30259831999999998, 2020-08-07T15:09:21.503 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:21.503 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:21.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:21.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:21.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:21.504 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86792 2020-08-07T15:09:21.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.504 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:21.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:21.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395402+0000", 2020-08-07T15:09:21.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:21.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:21.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395402+0000", 2020-08-07T15:09:21.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2262999999999998e-05 2020-08-07T15:09:21.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:21.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395444+0000", 2020-08-07T15:09:21.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1800000000000004e-06 2020-08-07T15:09:21.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:21.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395449+0000", 2020-08-07T15:09:21.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0738e-05 2020-08-07T15:09:21.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:21.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395460+0000", 2020-08-07T15:09:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7219999999999999e-06 2020-08-07T15:09:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395464+0000", 2020-08-07T15:09:21.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0063169089999999999 2020-08-07T15:09:21.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:21.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.401781+0000", 2020-08-07T15:09:21.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010702200000000001 2020-08-07T15:09:21.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:21.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.401888+0000", 2020-08-07T15:09:21.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019171999999999999 2020-08-07T15:09:21.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:09:21.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.402080+0000", 2020-08-07T15:09:21.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042866839999999998 2020-08-07T15:09:21.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:21.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.406367+0000", 2020-08-07T15:09:21.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000001e-07 2020-08-07T15:09:21.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:21.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.406367+0000", 2020-08-07T15:09:21.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4720000000000004e-05 2020-08-07T15:09:21.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:21.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.406432+0000", 2020-08-07T15:09:21.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050405365000000001 2020-08-07T15:09:21.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:21.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.456837+0000", 2020-08-07T15:09:21.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24111702700000001 2020-08-07T15:09:21.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:21.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.697954+0000", 2020-08-07T15:09:21.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.22e-06 2020-08-07T15:09:21.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:21.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.697955+0000", 2020-08-07T15:09:21.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4336999999999999e-05 2020-08-07T15:09:21.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:21.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.698000+0000", 2020-08-07T15:09:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30259831999999998 2020-08-07T15:09:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:104104 3.8 e58/20 3:11c276ea:::benchmark_data_smithi074_35447_object13012:head v 58'6496, mlcod=58'6487)", 2020-08-07T15:09:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:09:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 95.285605008000005, 2020-08-07T15:09:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33178497099999998, 2020-08-07T15:09:21.658 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:21.658 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:21.658 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:09:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:21.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:21.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:09:21.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9961999999999999e-05 2020-08-07T15:09:21.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:21.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858516+0000", 2020-08-07T15:09:21.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3200000000000001e-06 2020-08-07T15:09:21.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:21.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858520+0000", 2020-08-07T15:09:21.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32756790000000002 2020-08-07T15:09:21.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:21.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186088+0000", 2020-08-07T15:09:21.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0260000000000004e-06 2020-08-07T15:09:21.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:21.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186094+0000", 2020-08-07T15:09:21.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7919000000000001e-05 2020-08-07T15:09:21.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:21.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186132+0000", 2020-08-07T15:09:21.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4968e-05 2020-08-07T15:09:21.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:21.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186147+0000", 2020-08-07T15:09:21.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040671520000000001 2020-08-07T15:09:21.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:21.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.190214+0000", 2020-08-07T15:09:21.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6724000000000001e-05 2020-08-07T15:09:21.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:21.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.190271+0000", 2020-08-07T15:09:21.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33178497099999998 2020-08-07T15:09:21.671 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.671 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:21.671 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.672 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106248 3.f 3:f950fd9d:::benchmark_data_smithi074_35447_object13280:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:09:21.672 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:09:21.673 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.783926214000005, 2020-08-07T15:09:21.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002, 2020-08-07T15:09:21.673 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:21.673 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:21.673 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:21.674 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:21.674 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:21.674 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106248 2020-08-07T15:09:21.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.674 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:21.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:21.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:09:21.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:21.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:09:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7184e-05 2020-08-07T15:09:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360222+0000", 2020-08-07T15:09:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0906e-05 2020-08-07T15:09:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:21.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360233+0000", 2020-08-07T15:09:21.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2972999999999999e-05 2020-08-07T15:09:21.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:21.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360246+0000", 2020-08-07T15:09:21.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0114e-05 2020-08-07T15:09:21.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:21.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360256+0000", 2020-08-07T15:09:21.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1303999999999997e-05 2020-08-07T15:09:21.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:21.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360287+0000", 2020-08-07T15:09:21.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3798000000000001e-05 2020-08-07T15:09:21.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:21.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360351+0000", 2020-08-07T15:09:21.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111308 2020-08-07T15:09:21.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:09:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360462+0000", 2020-08-07T15:09:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003480674 2020-08-07T15:09:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:09:21.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0900000000000001e-07 2020-08-07T15:09:21.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:21.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:09:21.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8899999999999994e-05 2020-08-07T15:09:21.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.364012+0000", 2020-08-07T15:09:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35291637199999998 2020-08-07T15:09:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716929+0000", 2020-08-07T15:09:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.076244233999999994 2020-08-07T15:09:21.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:21.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793173+0000", 2020-08-07T15:09:21.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6109999999999999e-06 2020-08-07T15:09:21.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:21.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793175+0000", 2020-08-07T15:09:21.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7492999999999999e-05 2020-08-07T15:09:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793242+0000", 2020-08-07T15:09:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002 2020-08-07T15:09:21.701 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.701 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:21.701 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.702 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106248 3.f e59/20 ondisk, result = 0)", 2020-08-07T15:09:21.702 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.365323+0000", 2020-08-07T15:09:21.702 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.778767658000007, 2020-08-07T15:09:21.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35160734999999999, 2020-08-07T15:09:21.703 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:21.704 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:21.704 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:21.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:21.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365323+0000", 2020-08-07T15:09:21.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:21.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:21.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365323+0000", 2020-08-07T15:09:21.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4981000000000003e-05 2020-08-07T15:09:21.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:21.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365358+0000", 2020-08-07T15:09:21.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1090000000000003e-06 2020-08-07T15:09:21.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:21.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365364+0000", 2020-08-07T15:09:21.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35151899399999997 2020-08-07T15:09:21.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:21.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716883+0000", 2020-08-07T15:09:21.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0090000000000001e-06 2020-08-07T15:09:21.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:21.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716887+0000", 2020-08-07T15:09:21.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9238999999999999e-05 2020-08-07T15:09:21.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:21.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716916+0000", 2020-08-07T15:09:21.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0390000000000001e-06 2020-08-07T15:09:21.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:21.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716922+0000", 2020-08-07T15:09:21.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9789999999999993e-06 2020-08-07T15:09:21.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:21.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716930+0000", 2020-08-07T15:09:21.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35160734999999999 2020-08-07T15:09:21.715 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.715 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:21.715 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.716 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106248 3.f e59/20 ondisk, result = 0)", 2020-08-07T15:09:21.716 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:09:21.716 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.778662929000006, 2020-08-07T15:09:21.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001, 2020-08-07T15:09:21.717 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:21.747 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:21.747 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:21.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:21.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:09:21.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:21.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:21.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:09:21.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7518999999999999e-05 2020-08-07T15:09:21.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:21.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365445+0000", 2020-08-07T15:09:21.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-06 2020-08-07T15:09:21.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:21.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365449+0000", 2020-08-07T15:09:21.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42764147600000002 2020-08-07T15:09:21.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:21.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793091+0000", 2020-08-07T15:09:21.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6500000000000002e-06 2020-08-07T15:09:21.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:21.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793099+0000", 2020-08-07T15:09:21.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3664e-05 2020-08-07T15:09:21.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:21.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793153+0000", 2020-08-07T15:09:21.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1078e-05 2020-08-07T15:09:21.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:21.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793164+0000", 2020-08-07T15:09:21.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3404000000000003e-05 2020-08-07T15:09:21.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:21.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793248+0000", 2020-08-07T15:09:21.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001 2020-08-07T15:09:21.780 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.781 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:21.781 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.783 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106344 3.e 3:7abbcafa:::benchmark_data_smithi074_35447_object13292:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:09:21.783 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:48.059687+0000", 2020-08-07T15:09:21.784 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.084403827000003, 2020-08-07T15:09:21.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28541866100000002, 2020-08-07T15:09:21.784 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:21.785 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:21.785 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:21.785 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:21.785 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:21.785 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106344 2020-08-07T15:09:21.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.786 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:21.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:21.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059687+0000", 2020-08-07T15:09:21.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:21.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:21.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059687+0000", 2020-08-07T15:09:21.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5778999999999998e-05 2020-08-07T15:09:21.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:21.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059713+0000", 2020-08-07T15:09:21.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.422e-06 2020-08-07T15:09:21.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:21.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059716+0000", 2020-08-07T15:09:21.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7679999999999997e-06 2020-08-07T15:09:21.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:21.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059725+0000", 2020-08-07T15:09:21.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2020-08-07T15:09:21.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:21.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059729+0000", 2020-08-07T15:09:21.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6205000000000003e-05 2020-08-07T15:09:21.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:21.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059765+0000", 2020-08-07T15:09:21.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3615999999999995e-05 2020-08-07T15:09:21.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:21.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059829+0000", 2020-08-07T15:09:21.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000118903 2020-08-07T15:09:21.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:09:21.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059948+0000", 2020-08-07T15:09:21.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00070666299999999995 2020-08-07T15:09:21.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:21.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.060654+0000", 2020-08-07T15:09:21.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5199999999999995e-07 2020-08-07T15:09:21.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:21.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.060655+0000", 2020-08-07T15:09:21.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2092999999999998e-05 2020-08-07T15:09:21.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:21.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.060697+0000", 2020-08-07T15:09:21.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00038454500000000003 2020-08-07T15:09:21.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:21.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.061082+0000", 2020-08-07T15:09:21.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.283939936 2020-08-07T15:09:21.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:21.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.345022+0000", 2020-08-07T15:09:21.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.288e-06 2020-08-07T15:09:21.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:21.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.345023+0000", 2020-08-07T15:09:21.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2468999999999999e-05 2020-08-07T15:09:21.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:21.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.345105+0000", 2020-08-07T15:09:21.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28541866100000002 2020-08-07T15:09:21.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.806 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:21.806 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.806 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:107952 3.d e61/35)", 2020-08-07T15:09:21.807 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:09:21.807 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 90.817733220999997, 2020-08-07T15:09:21.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001, 2020-08-07T15:09:21.807 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:21.807 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:21.808 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:21.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:21.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:09:21.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:21.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:21.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:09:21.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8826000000000003e-05 2020-08-07T15:09:21.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:21.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326406+0000", 2020-08-07T15:09:21.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9199999999999998e-06 2020-08-07T15:09:21.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:21.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326413+0000", 2020-08-07T15:09:21.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75082853500000002 2020-08-07T15:09:21.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:21.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077242+0000", 2020-08-07T15:09:21.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1479999999999999e-06 2020-08-07T15:09:21.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:21.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077246+0000", 2020-08-07T15:09:21.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3331999999999998e-05 2020-08-07T15:09:21.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:21.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077289+0000", 2020-08-07T15:09:21.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8977000000000001e-05 2020-08-07T15:09:21.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:21.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077308+0000", 2020-08-07T15:09:21.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001 2020-08-07T15:09:21.821 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.821 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:21.822 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.823 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31696 4.0 4:0335eed9:::benchmark_data_smithi074_44586_object3961:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e100)", 2020-08-07T15:09:21.823 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:09:21.824 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.039825661999998, 2020-08-07T15:09:21.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237, 2020-08-07T15:09:21.825 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:21.825 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:21.825 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:21.826 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:21.826 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:21.826 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31696 2020-08-07T15:09:21.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.827 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:21.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:21.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:09:21.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:21.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:21.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:09:21.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7654e-05 2020-08-07T15:09:21.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:21.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104293+0000", 2020-08-07T15:09:21.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.061e-06 2020-08-07T15:09:21.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:21.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104296+0000", 2020-08-07T15:09:21.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0859999999999998e-06 2020-08-07T15:09:21.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:21.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104302+0000", 2020-08-07T15:09:21.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8030000000000001e-06 2020-08-07T15:09:21.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:21.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104305+0000", 2020-08-07T15:09:21.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037634769999999999 2020-08-07T15:09:21.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:21.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108068+0000", 2020-08-07T15:09:21.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8814999999999998e-05 2020-08-07T15:09:21.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:21.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108137+0000", 2020-08-07T15:09:21.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111455 2020-08-07T15:09:21.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:09:21.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108248+0000", 2020-08-07T15:09:21.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028756820000000001 2020-08-07T15:09:21.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:21.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111124+0000", 2020-08-07T15:09:21.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061054499999999999 2020-08-07T15:09:21.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:21.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111735+0000", 2020-08-07T15:09:21.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.423966167 2020-08-07T15:09:21.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:21.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535701+0000", 2020-08-07T15:09:21.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4929999999999999e-06 2020-08-07T15:09:21.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:21.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535702+0000", 2020-08-07T15:09:21.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2571999999999997e-05 2020-08-07T15:09:21.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:21.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535755+0000", 2020-08-07T15:09:21.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1427000000000003e-05 2020-08-07T15:09:21.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:21.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535786+0000", 2020-08-07T15:09:21.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237 2020-08-07T15:09:21.851 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.851 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:21.851 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.852 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:31696 4.0 e100/81 ondisk, result = 0)", 2020-08-07T15:09:21.853 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:09:21.860 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.030211104000003, 2020-08-07T15:09:21.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954, 2020-08-07T15:09:21.862 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:21.862 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:21.862 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:21.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:21.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:09:21.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:21.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:21.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:09:21.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1185000000000001e-05 2020-08-07T15:09:21.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:21.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113901+0000", 2020-08-07T15:09:21.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2930000000000002e-06 2020-08-07T15:09:21.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:21.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113905+0000", 2020-08-07T15:09:21.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42168640400000001 2020-08-07T15:09:21.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:21.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535591+0000", 2020-08-07T15:09:21.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2820000000000002e-06 2020-08-07T15:09:21.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:21.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535598+0000", 2020-08-07T15:09:21.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6578999999999999e-05 2020-08-07T15:09:21.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:21.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535684+0000", 2020-08-07T15:09:21.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8449999999999992e-06 2020-08-07T15:09:21.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:21.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535693+0000", 2020-08-07T15:09:21.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7366000000000003e-05 2020-08-07T15:09:21.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:21.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535791+0000", 2020-08-07T15:09:21.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954 2020-08-07T15:09:21.879 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.879 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:21.879 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.881 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:37024 4.0 e106/81)", 2020-08-07T15:09:21.881 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:09:21.882 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.915422209999999, 2020-08-07T15:09:21.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999, 2020-08-07T15:09:21.882 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:21.882 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:21.882 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:21.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:21.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:09:21.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:21.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:09:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4926000000000001e-05 2020-08-07T15:09:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228683+0000", 2020-08-07T15:09:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1280000000000002e-06 2020-08-07T15:09:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:21.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228686+0000", 2020-08-07T15:09:21.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69740012100000004 2020-08-07T15:09:21.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:21.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926086+0000", 2020-08-07T15:09:21.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000004e-06 2020-08-07T15:09:21.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:21.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926090+0000", 2020-08-07T15:09:21.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113526 2020-08-07T15:09:21.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:21.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926204+0000", 2020-08-07T15:09:21.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8824000000000001e-05 2020-08-07T15:09:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926222+0000", 2020-08-07T15:09:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999 2020-08-07T15:09:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42503 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2615, mlcod=112'2607)", 2020-08-07T15:09:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:09:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.847850294000001, 2020-08-07T15:09:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998, 2020-08-07T15:09:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:09:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:09:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270999999999998e-05 2020-08-07T15:09:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296292+0000", 2020-08-07T15:09:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4729999999999997e-06 2020-08-07T15:09:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296298+0000", 2020-08-07T15:09:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.448795049 2020-08-07T15:09:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745093+0000", 2020-08-07T15:09:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0140000000000001e-06 2020-08-07T15:09:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745100+0000", 2020-08-07T15:09:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2596999999999998e-05 2020-08-07T15:09:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745153+0000", 2020-08-07T15:09:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1662000000000003e-05 2020-08-07T15:09:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745184+0000", 2020-08-07T15:09:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093174800000000004 2020-08-07T15:09:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746116+0000", 2020-08-07T15:09:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0926999999999994e-05 2020-08-07T15:09:21.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:21.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746187+0000", 2020-08-07T15:09:21.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998 2020-08-07T15:09:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42504 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2616, mlcod=113'2609)", 2020-08-07T15:09:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:09:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.844513401, 2020-08-07T15:09:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998, 2020-08-07T15:09:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:09:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:09:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2154000000000002e-05 2020-08-07T15:09:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299629+0000", 2020-08-07T15:09:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1929999999999998e-06 2020-08-07T15:09:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299636+0000", 2020-08-07T15:09:21.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44766315400000001 2020-08-07T15:09:21.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:21.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:21.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:21.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747299+0000", 2020-08-07T15:09:21.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5889999999999999e-06 2020-08-07T15:09:21.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:22.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:22.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747305+0000", 2020-08-07T15:09:22.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3094999999999999e-05 2020-08-07T15:09:22.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:22.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:22.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747358+0000", 2020-08-07T15:09:22.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3283e-05 2020-08-07T15:09:22.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:22.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:22.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747392+0000", 2020-08-07T15:09:22.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015319509999999999 2020-08-07T15:09:22.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:22.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:22.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748924+0000", 2020-08-07T15:09:22.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5235999999999996e-05 2020-08-07T15:09:22.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:22.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:22.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748999+0000", 2020-08-07T15:09:22.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998 2020-08-07T15:09:22.022 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:22.023 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:22.023 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:22.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:22.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.024 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42519 4.8 e113/81 4:13071de5:::benchmark_data_smithi074_44586_object5314:head v 113'2623, mlcod=113'2617)", 2020-08-07T15:09:22.024 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.762041+0000", 2020-08-07T15:09:22.025 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.382049340999998, 2020-08-07T15:09:22.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.34605073400000003, 2020-08-07T15:09:22.031 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:22.031 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:22.031 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:22.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:22.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.762041+0000", 2020-08-07T15:09:22.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:22.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:22.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:22.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.762041+0000", 2020-08-07T15:09:22.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9067000000000002e-05 2020-08-07T15:09:22.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:22.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:22.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.762100+0000", 2020-08-07T15:09:22.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3529999999999997e-06 2020-08-07T15:09:22.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:22.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:22.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.762108+0000", 2020-08-07T15:09:22.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.34498205900000001 2020-08-07T15:09:22.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:22.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:22.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.107090+0000", 2020-08-07T15:09:22.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7420000000000003e-06 2020-08-07T15:09:22.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:22.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:22.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.107096+0000", 2020-08-07T15:09:22.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1074000000000004e-05 2020-08-07T15:09:22.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:22.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:22.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.107167+0000", 2020-08-07T15:09:22.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4423999999999999e-05 2020-08-07T15:09:22.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:22.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:22.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.107191+0000", 2020-08-07T15:09:22.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00084080199999999998 2020-08-07T15:09:22.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:22.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:22.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.108032+0000", 2020-08-07T15:09:22.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0213000000000001e-05 2020-08-07T15:09:22.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:22.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:22.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.108092+0000", 2020-08-07T15:09:22.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.34605073400000003 2020-08-07T15:09:22.088 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:22.089 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:22.089 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:22.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:22.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.089 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42520 4.8 e113/81 4:13071de5:::benchmark_data_smithi074_44586_object5314:head v 113'2624, mlcod=113'2618)", 2020-08-07T15:09:22.090 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:09:22.090 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.378569192, 2020-08-07T15:09:22.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001, 2020-08-07T15:09:22.090 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:22.090 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:22.091 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:22.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:22.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:09:22.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:22.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:22.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:22.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:09:22.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3967999999999995e-05 2020-08-07T15:09:22.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:22.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:22.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765585+0000", 2020-08-07T15:09:22.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0949999999999998e-06 2020-08-07T15:09:22.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:22.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:22.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765593+0000", 2020-08-07T15:09:22.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52703464200000005 2020-08-07T15:09:22.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:22.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:22.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292627+0000", 2020-08-07T15:09:22.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6929999999999996e-06 2020-08-07T15:09:22.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:22.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:22.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292636+0000", 2020-08-07T15:09:22.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9292000000000001e-05 2020-08-07T15:09:22.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:22.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:22.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292695+0000", 2020-08-07T15:09:22.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5395999999999999e-05 2020-08-07T15:09:22.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:22.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:22.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292721+0000", 2020-08-07T15:09:22.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00090361399999999996 2020-08-07T15:09:22.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:22.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:22.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293624+0000", 2020-08-07T15:09:22.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1761000000000002e-05 2020-08-07T15:09:22.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:22.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:22.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:22.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293666+0000", 2020-08-07T15:09:22.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001 2020-08-07T15:09:22.104 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:22.104 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:22.104 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:22.104 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:22.104 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:22.105 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:09:22.106 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:09:22.107 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:09:22.107 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 0 1 2020-08-07T15:09:22.109 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:21.173+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:22.109 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:22.106+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:22.110 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:09:22.110 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:09:22.110 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:09:22.110 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:09:22.110 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 55 16 67962 67946 9.65049 10.6484 0.0235557 0.0129487 2020-08-07T15:09:22.111 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:09:22.111 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:09:22.111 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:09:22.111 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:09:22.112 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:09:22.112 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:09:22.112 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:09:22.112 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:09:22.211 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:22.209+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:22.278 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 56 16 69323 69307 9.66802 10.6328 0.00592794 0.0129247 2020-08-07T15:09:22.315 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:22.313+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:22.340 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:09:22.341 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:22.343 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:22.365 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:09:22.418 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:22.414+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:22.517 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:22.516+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:22.610 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:22.611 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:22.621 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:22.618+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:22.631 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:09:22.723 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:22.721+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:22.828 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:22.826+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:22.867 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:22.867 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:22.882 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:09:22.972 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:22.967+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:23.047 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:23.038+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:23.099 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:23.100 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:23.116 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:09:23.164 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:23.155+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:23.255 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:23.253+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:23.286 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 57 16 70712 70696 9.68877 10.8516 0.00759945 0.0128932 2020-08-07T15:09:23.329 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:23.330 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:23.341 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:09:23.387 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:23.357+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:23.412 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.0 to 1 (10000) 2020-08-07T15:09:23.413 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:09:23.421 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 1 1 2020-08-07T15:09:23.463 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:23.460+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:23.564 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:23.564 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:23.565 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:23.563+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:23.574 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:09:23.667 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:23.664+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:23.758 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:23.759 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:23.772 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:09:23.773 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:23.768+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:23.870 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:23.868+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:23.907 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:23.907 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:23.916 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:09:23.975 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:23.972+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:24.038 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:24.038 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:24.047 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:09:24.100 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:24.099+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:24.130 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:09:24.220 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:24.219+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:24.230 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:24.230 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:24.238 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:09:24.278 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 58 16 71175 71159 9.58409 3.61719 0.4374 0.0129019 2020-08-07T15:09:24.310 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:24.306+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:24.330 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:09:24.353 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:09:24.385 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:09:24.414 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:24.409+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:24.416 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:09:24.417 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.1 to 1 (10000) 2020-08-07T15:09:24.432 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:09:24.432 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:09:24.432 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:09:24.433 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:09:24.433 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:09:24.442 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 2 1 2020-08-07T15:09:24.456 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:09:24.458 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:09:24.505 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:09:24.521 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:24.510+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:24.606 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:09:24.606 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:09:24.607 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:09:24.607 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:09:24.607 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:09:24.607 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:09:24.608 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:09:24.608 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:09:24.608 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:09:24.608 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:09:24.638 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:24.628+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:24.670 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:09:24.670 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:09:24.677 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:09:24.677 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:09:24.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.678 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106240 3.7 3:ebdd8242:::benchmark_data_smithi074_35447_object13279:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:09:24.678 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:09:24.690 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 97.295604693000001, 2020-08-07T15:09:24.690 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.90079396, 2020-08-07T15:09:24.691 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:24.691 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:24.691 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:24.691 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:09:24.692 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:24.692 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106240 2020-08-07T15:09:24.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.692 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:24.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.693 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:24.693 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:09:24.693 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:24.693 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.693 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.703 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:24.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:09:24.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2522000000000003e-05 2020-08-07T15:09:24.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:24.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355118+0000", 2020-08-07T15:09:24.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6159999999999998e-06 2020-08-07T15:09:24.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.721 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:24.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355125+0000", 2020-08-07T15:09:24.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2699e-05 2020-08-07T15:09:24.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.722 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:24.722 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355138+0000", 2020-08-07T15:09:24.722 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2310000000000001e-06 2020-08-07T15:09:24.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.723 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:24.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355144+0000", 2020-08-07T15:09:24.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9855000000000003e-05 2020-08-07T15:09:24.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.724 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.724 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:24.724 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355194+0000", 2020-08-07T15:09:24.724 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112713 2020-08-07T15:09:24.724 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.725 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:24.725 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355306+0000", 2020-08-07T15:09:24.725 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019794600000000001 2020-08-07T15:09:24.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.726 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.726 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:09:24.726 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355504+0000", 2020-08-07T15:09:24.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0048346029999999998 2020-08-07T15:09:24.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.746 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:24.746 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.360339+0000", 2020-08-07T15:09:24.746 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00033646400000000002 2020-08-07T15:09:24.747 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.747 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.747 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:24.748 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.360675+0000", 2020-08-07T15:09:24.748 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89508481200000001 2020-08-07T15:09:24.748 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.748 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.749 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:24.749 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255760+0000", 2020-08-07T15:09:24.749 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5659999999999999e-06 2020-08-07T15:09:24.749 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.750 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.750 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:24.750 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255762+0000", 2020-08-07T15:09:24.751 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6595000000000003e-05 2020-08-07T15:09:24.751 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.751 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.752 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:24.752 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255818+0000", 2020-08-07T15:09:24.752 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1338000000000003e-05 2020-08-07T15:09:24.752 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.753 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.753 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:24.753 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255860+0000", 2020-08-07T15:09:24.753 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.90079396 2020-08-07T15:09:24.755 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:24.756 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:24.756 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:24.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.758 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106240 3.7 e59/20 ondisk, result = 0)", 2020-08-07T15:09:24.758 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:09:24.758 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 97.199208741000007, 2020-08-07T15:09:24.759 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80440215100000001, 2020-08-07T15:09:24.759 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:24.759 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:09:24.760 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:24.760 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.760 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:24.760 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:09:24.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:24.761 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.762 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:24.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:09:24.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0052999999999999e-05 2020-08-07T15:09:24.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.762 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:24.763 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451482+0000", 2020-08-07T15:09:24.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3229999999999999e-06 2020-08-07T15:09:24.763 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.765 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:24.766 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451486+0000", 2020-08-07T15:09:24.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80420304799999998 2020-08-07T15:09:24.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:24.768 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255689+0000", 2020-08-07T15:09:24.769 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7759999999999997e-06 2020-08-07T15:09:24.769 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.769 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.770 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:24.770 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255694+0000", 2020-08-07T15:09:24.770 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5828999999999998e-05 2020-08-07T15:09:24.770 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.771 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.771 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:24.771 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255740+0000", 2020-08-07T15:09:24.772 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1466e-05 2020-08-07T15:09:24.772 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.772 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.772 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:24.772 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255751+0000", 2020-08-07T15:09:24.773 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112656 2020-08-07T15:09:24.773 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.773 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.773 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:24.774 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255864+0000", 2020-08-07T15:09:24.774 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80440215100000001 2020-08-07T15:09:24.774 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:24.774 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:24.775 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:24.775 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.777 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37057 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:24.777 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:09:24.777 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 30.407100030999999, 2020-08-07T15:09:24.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007248691, 2020-08-07T15:09:24.778 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:24.778 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:24.778 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:24.778 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:24.779 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:24.779 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37057 2020-08-07T15:09:24.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.780 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:24.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.780 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:24.780 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:09:24.781 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:24.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.789 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.789 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:24.790 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:09:24.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3661e-05 2020-08-07T15:09:24.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:24.791 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243624+0000", 2020-08-07T15:09:24.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7519999999999996e-06 2020-08-07T15:09:24.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.792 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.792 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:24.797 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243632+0000", 2020-08-07T15:09:24.797 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6392000000000002e-05 2020-08-07T15:09:24.797 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.798 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.798 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:24.798 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243648+0000", 2020-08-07T15:09:24.799 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6659999999999997e-06 2020-08-07T15:09:24.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:24.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243657+0000", 2020-08-07T15:09:24.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0070484609999999998 2020-08-07T15:09:24.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:24.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250705+0000", 2020-08-07T15:09:24.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0083e-05 2020-08-07T15:09:24.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:24.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250715+0000", 2020-08-07T15:09:24.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000086695 2020-08-07T15:09:24.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.808 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:24.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.250802+0000", 2020-08-07T15:09:24.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6980999999999999e-05 2020-08-07T15:09:24.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.809 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:24.809 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.250819+0000", 2020-08-07T15:09:24.809 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007248691 2020-08-07T15:09:24.812 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:24.812 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:24.812 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:24.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.813 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37058 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:24.814 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:09:24.814 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 30.406958296999999, 2020-08-07T15:09:24.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007368357, 2020-08-07T15:09:24.816 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:24.816 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:24.816 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:24.816 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:24.817 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:24.817 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37058 2020-08-07T15:09:24.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.817 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:24.818 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.818 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:24.818 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:09:24.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:24.818 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:24.819 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:09:24.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8838999999999998e-05 2020-08-07T15:09:24.819 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:24.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243761+0000", 2020-08-07T15:09:24.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2079999999999999e-06 2020-08-07T15:09:24.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:24.821 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243766+0000", 2020-08-07T15:09:24.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0567999999999999e-05 2020-08-07T15:09:24.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.822 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:24.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243777+0000", 2020-08-07T15:09:24.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7210000000000002e-06 2020-08-07T15:09:24.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:24.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243782+0000", 2020-08-07T15:09:24.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069508749999999996 2020-08-07T15:09:24.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:24.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250732+0000", 2020-08-07T15:09:24.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7899999999999999e-06 2020-08-07T15:09:24.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:24.826 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250737+0000", 2020-08-07T15:09:24.826 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003272409999999 2020-08-07T15:09:24.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.827 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:24.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251064+0000", 2020-08-07T15:09:24.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6115000000000001e-05 2020-08-07T15:09:24.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:24.828 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251081+0000", 2020-08-07T15:09:24.828 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007368357 2020-08-07T15:09:24.829 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:24.829 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:24.829 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:24.829 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.829 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.830 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37059 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:24.830 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:09:24.830 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 30.406715248000001, 2020-08-07T15:09:24.830 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0071655319999999, 2020-08-07T15:09:24.830 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:24.831 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:24.831 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:24.831 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:24.831 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:24.831 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37059 2020-08-07T15:09:24.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.832 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:24.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:24.833 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:09:24.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:24.833 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:24.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:09:24.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6533000000000002e-05 2020-08-07T15:09:24.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:24.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244012+0000", 2020-08-07T15:09:24.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3860000000000001e-06 2020-08-07T15:09:24.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.836 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:24.836 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244019+0000", 2020-08-07T15:09:24.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6982000000000001e-05 2020-08-07T15:09:24.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:24.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244036+0000", 2020-08-07T15:09:24.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0310000000000001e-06 2020-08-07T15:09:24.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.838 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:24.838 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244043+0000", 2020-08-07T15:09:24.838 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0067169259999999998 2020-08-07T15:09:24.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.839 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:24.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250760+0000", 2020-08-07T15:09:24.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4499999999999998e-06 2020-08-07T15:09:24.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.849 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.849 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:24.849 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250768+0000", 2020-08-07T15:09:24.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000341691 2020-08-07T15:09:24.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.853 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:24.853 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251109+0000", 2020-08-07T15:09:24.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1533e-05 2020-08-07T15:09:24.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.856 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:24.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251121+0000", 2020-08-07T15:09:24.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0071655319999999 2020-08-07T15:09:24.857 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:24.857 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:24.858 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:24.858 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.859 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37060 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:24.859 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:09:24.859 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 30.406591311, 2020-08-07T15:09:24.859 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070762280000001, 2020-08-07T15:09:24.859 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:24.860 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:24.860 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:24.860 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:24.860 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:24.860 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37060 2020-08-07T15:09:24.869 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.869 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:24.870 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.870 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:24.870 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:09:24.870 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:24.870 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.871 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:24.871 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:09:24.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5739000000000003e-05 2020-08-07T15:09:24.873 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.873 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:24.873 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244135+0000", 2020-08-07T15:09:24.874 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7740000000000001e-06 2020-08-07T15:09:24.874 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.874 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.874 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:24.874 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244141+0000", 2020-08-07T15:09:24.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3817e-05 2020-08-07T15:09:24.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:24.875 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244155+0000", 2020-08-07T15:09:24.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3829999999999997e-06 2020-08-07T15:09:24.876 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.876 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:24.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244161+0000", 2020-08-07T15:09:24.877 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0066303669999999999 2020-08-07T15:09:24.878 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.878 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.878 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:24.878 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250791+0000", 2020-08-07T15:09:24.878 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.392e-06 2020-08-07T15:09:24.879 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.880 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:24.880 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250798+0000", 2020-08-07T15:09:24.880 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003485860000001 2020-08-07T15:09:24.880 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.881 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.881 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:24.881 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251146+0000", 2020-08-07T15:09:24.881 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1700000000000003e-06 2020-08-07T15:09:24.881 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.882 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.882 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:24.882 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251155+0000", 2020-08-07T15:09:24.882 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070762280000001 2020-08-07T15:09:24.882 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:24.883 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:24.883 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:24.883 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.883 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.883 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37061 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:24.884 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:09:24.884 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 30.406476684000001, 2020-08-07T15:09:24.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070006920000001, 2020-08-07T15:09:24.884 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:24.886 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:24.886 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:24.886 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:24.887 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:24.887 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37061 2020-08-07T15:09:24.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.887 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:24.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:24.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:09:24.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:24.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:24.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:09:24.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5453e-05 2020-08-07T15:09:24.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.890 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:24.892 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244239+0000", 2020-08-07T15:09:24.893 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3179999999999997e-06 2020-08-07T15:09:24.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.896 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:24.897 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244245+0000", 2020-08-07T15:09:24.897 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2138e-05 2020-08-07T15:09:24.898 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.906 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:24.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244257+0000", 2020-08-07T15:09:24.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.69e-06 2020-08-07T15:09:24.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.908 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:24.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244261+0000", 2020-08-07T15:09:24.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0065598940000000001 2020-08-07T15:09:24.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:24.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250821+0000", 2020-08-07T15:09:24.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4799999999999998e-06 2020-08-07T15:09:24.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.910 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:24.910 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250828+0000", 2020-08-07T15:09:24.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003567310000001 2020-08-07T15:09:24.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:24.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251185+0000", 2020-08-07T15:09:24.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9879999999999999e-06 2020-08-07T15:09:24.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:24.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251195+0000", 2020-08-07T15:09:24.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070006920000001 2020-08-07T15:09:24.914 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:24.914 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:24.914 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:24.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.917 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37062 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:24.918 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:09:24.918 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 30.406376466000001, 2020-08-07T15:09:24.918 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006933662, 2020-08-07T15:09:24.919 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:24.919 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:24.919 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:24.919 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:24.919 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:24.920 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37062 2020-08-07T15:09:24.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.920 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:24.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:24.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:09:24.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:24.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.922 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:24.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:09:24.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4041e-05 2020-08-07T15:09:24.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:24.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244348+0000", 2020-08-07T15:09:24.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2079999999999997e-06 2020-08-07T15:09:24.926 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.926 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:24.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244354+0000", 2020-08-07T15:09:24.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1483e-05 2020-08-07T15:09:24.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:24.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244366+0000", 2020-08-07T15:09:24.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7989999999999999e-06 2020-08-07T15:09:24.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:24.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244372+0000", 2020-08-07T15:09:24.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0064783699999999998 2020-08-07T15:09:24.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:24.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250850+0000", 2020-08-07T15:09:24.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4729999999999997e-06 2020-08-07T15:09:24.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:24.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250856+0000", 2020-08-07T15:09:24.931 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003630240000001 2020-08-07T15:09:24.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:24.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251219+0000", 2020-08-07T15:09:24.932 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2640000000000008e-06 2020-08-07T15:09:24.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:24.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251228+0000", 2020-08-07T15:09:24.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006933662 2020-08-07T15:09:24.936 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:24.936 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:24.936 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:24.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.937 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37063 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:24.937 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:09:24.937 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 30.406264132, 2020-08-07T15:09:24.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006858051, 2020-08-07T15:09:24.937 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:24.938 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:24.938 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:24.938 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:24.938 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:24.938 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37063 2020-08-07T15:09:24.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.939 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:24.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:24.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:09:24.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:24.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.940 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.940 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:24.940 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:09:24.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5043000000000001e-05 2020-08-07T15:09:24.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:24.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244461+0000", 2020-08-07T15:09:24.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4600000000000002e-06 2020-08-07T15:09:24.942 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.950 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:24.950 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244467+0000", 2020-08-07T15:09:24.950 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1435e-05 2020-08-07T15:09:24.950 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:24.951 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244478+0000", 2020-08-07T15:09:24.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0190000000000003e-06 2020-08-07T15:09:24.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.952 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.952 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:24.952 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244484+0000", 2020-08-07T15:09:24.952 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006392895 2020-08-07T15:09:24.955 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.956 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.956 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:24.956 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250877+0000", 2020-08-07T15:09:24.956 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4280000000000001e-06 2020-08-07T15:09:24.957 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.957 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:24.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250884+0000", 2020-08-07T15:09:24.958 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000370414 2020-08-07T15:09:24.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:24.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251254+0000", 2020-08-07T15:09:24.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0356999999999999e-05 2020-08-07T15:09:24.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:24.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251264+0000", 2020-08-07T15:09:24.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006858051 2020-08-07T15:09:24.960 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:24.960 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:24.960 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:24.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.961 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37064 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:24.961 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:09:24.962 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 30.406151998999999, 2020-08-07T15:09:24.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0067798189999999, 2020-08-07T15:09:24.962 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:24.963 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:24.963 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:24.963 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:24.963 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:24.964 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37064 2020-08-07T15:09:24.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.964 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:24.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:24.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:09:24.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:24.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:24.966 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:09:24.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4007e-05 2020-08-07T15:09:24.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:24.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244573+0000", 2020-08-07T15:09:24.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.831e-06 2020-08-07T15:09:24.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:24.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244577+0000", 2020-08-07T15:09:24.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2269000000000001e-05 2020-08-07T15:09:24.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.969 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:24.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244590+0000", 2020-08-07T15:09:24.969 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7149999999999997e-06 2020-08-07T15:09:24.969 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.969 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.970 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:24.970 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244594+0000", 2020-08-07T15:09:24.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063105670000000004 2020-08-07T15:09:24.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:24.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250905+0000", 2020-08-07T15:09:24.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1970000000000001e-06 2020-08-07T15:09:24.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:24.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250911+0000", 2020-08-07T15:09:24.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003783020000001 2020-08-07T15:09:24.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:24.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251289+0000", 2020-08-07T15:09:24.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9309999999999997e-06 2020-08-07T15:09:24.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:24.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251298+0000", 2020-08-07T15:09:24.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0067798189999999 2020-08-07T15:09:24.975 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:24.976 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:24.977 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:24.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.978 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37065 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:24.978 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:09:24.978 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 30.406041779999999, 2020-08-07T15:09:24.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006708545, 2020-08-07T15:09:24.979 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:24.979 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:24.979 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:24.979 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:24.979 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:24.980 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37065 2020-08-07T15:09:24.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.980 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:24.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:24.980 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:09:24.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:24.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.981 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:24.981 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:09:24.982 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2750000000000001e-05 2020-08-07T15:09:24.982 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.982 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:24.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244681+0000", 2020-08-07T15:09:24.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0950000000000003e-06 2020-08-07T15:09:24.983 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.985 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:24.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244687+0000", 2020-08-07T15:09:24.986 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2485e-05 2020-08-07T15:09:24.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.987 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:24.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244699+0000", 2020-08-07T15:09:24.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8800000000000002e-06 2020-08-07T15:09:24.993 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.993 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.993 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:24.994 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244706+0000", 2020-08-07T15:09:24.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0062281569999999998 2020-08-07T15:09:24.994 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.994 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.994 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:24.995 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250934+0000", 2020-08-07T15:09:24.996 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1719999999999998e-06 2020-08-07T15:09:24.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:24.996 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:24.997 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:24.997 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250939+0000", 2020-08-07T15:09:24.999 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003861810000001 2020-08-07T15:09:25.001 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.001 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.001 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:25.001 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251325+0000", 2020-08-07T15:09:25.002 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1824999999999999e-05 2020-08-07T15:09:25.002 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.002 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.002 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:25.002 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251337+0000", 2020-08-07T15:09:25.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006708545 2020-08-07T15:09:25.003 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:25.004 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:25.004 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:25.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.005 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37066 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:25.005 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:09:25.005 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 30.401372810000002, 2020-08-07T15:09:25.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002077205, 2020-08-07T15:09:25.006 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:25.011 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:25.012 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:25.013 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:25.013 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:25.013 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37066 2020-08-07T15:09:25.014 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.014 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:25.014 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:25.015 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:09:25.015 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:25.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.016 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:25.018 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:09:25.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4654000000000002e-05 2020-08-07T15:09:25.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:25.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249352+0000", 2020-08-07T15:09:25.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6210000000000004e-06 2020-08-07T15:09:25.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.039 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:25.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249360+0000", 2020-08-07T15:09:25.040 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1895999999999998e-05 2020-08-07T15:09:25.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.050 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.050 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:25.051 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249382+0000", 2020-08-07T15:09:25.051 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1479999999999995e-06 2020-08-07T15:09:25.051 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.051 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:25.052 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249391+0000", 2020-08-07T15:09:25.052 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015731510000000001 2020-08-07T15:09:25.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.052 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.053 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:25.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250964+0000", 2020-08-07T15:09:25.053 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5400000000000001e-06 2020-08-07T15:09:25.053 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.054 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:25.054 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250971+0000", 2020-08-07T15:09:25.054 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000392379 2020-08-07T15:09:25.054 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.054 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:25.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251363+0000", 2020-08-07T15:09:25.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1816e-05 2020-08-07T15:09:25.055 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.055 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:25.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251375+0000", 2020-08-07T15:09:25.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002077205 2020-08-07T15:09:25.056 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:25.056 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:25.057 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:25.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.057 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37067 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:25.057 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:09:25.058 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 30.401232115999999, 2020-08-07T15:09:25.058 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0019761119999999, 2020-08-07T15:09:25.058 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:25.058 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:25.058 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:25.059 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:25.059 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:25.060 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37067 2020-08-07T15:09:25.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.060 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:25.090 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.090 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:25.090 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:09:25.091 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:25.091 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.091 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.091 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:25.092 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:09:25.092 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5961999999999997e-05 2020-08-07T15:09:25.092 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.092 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.092 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:25.093 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249494+0000", 2020-08-07T15:09:25.094 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3210000000000004e-06 2020-08-07T15:09:25.094 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.094 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.095 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:25.095 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249500+0000", 2020-08-07T15:09:25.095 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1012000000000001e-05 2020-08-07T15:09:25.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.095 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.096 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:25.096 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249511+0000", 2020-08-07T15:09:25.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.809e-06 2020-08-07T15:09:25.096 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.097 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.097 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:25.097 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249515+0000", 2020-08-07T15:09:25.097 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0014777670000000001 2020-08-07T15:09:25.097 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.098 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.098 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:25.098 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250993+0000", 2020-08-07T15:09:25.098 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9499999999999998e-06 2020-08-07T15:09:25.098 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.100 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.100 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:25.100 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250999+0000", 2020-08-07T15:09:25.100 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0004032519999999 2020-08-07T15:09:25.100 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.101 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.101 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:25.101 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251402+0000", 2020-08-07T15:09:25.101 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2038999999999999e-05 2020-08-07T15:09:25.103 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.103 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.104 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:25.105 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251414+0000", 2020-08-07T15:09:25.105 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0019761119999999 2020-08-07T15:09:25.106 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:25.106 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:25.107 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:25.108 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.108 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.108 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37068 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:25.108 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:09:25.109 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 30.398987545000001, 2020-08-07T15:09:25.109 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99977072, 2020-08-07T15:09:25.109 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:25.109 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:25.109 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:25.110 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:25.110 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:25.110 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37068 2020-08-07T15:09:25.110 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.112 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:25.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:25.113 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:09:25.113 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:25.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.115 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:25.115 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:09:25.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1369000000000003e-05 2020-08-07T15:09:25.115 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.115 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.116 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:25.116 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251724+0000", 2020-08-07T15:09:25.116 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.19e-06 2020-08-07T15:09:25.117 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.118 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.118 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:25.119 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251731+0000", 2020-08-07T15:09:25.119 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1368e-05 2020-08-07T15:09:25.119 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.119 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.120 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:25.120 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251742+0000", 2020-08-07T15:09:25.120 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.682e-06 2020-08-07T15:09:25.120 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.120 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.121 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:25.122 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251748+0000", 2020-08-07T15:09:25.122 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5275000000000003e-05 2020-08-07T15:09:25.123 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.123 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.123 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:25.123 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251793+0000", 2020-08-07T15:09:25.123 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5000000000000005e-06 2020-08-07T15:09:25.123 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.124 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.124 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:25.124 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251802+0000", 2020-08-07T15:09:25.125 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99963953500000002 2020-08-07T15:09:25.125 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.126 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.126 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:25.126 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251442+0000", 2020-08-07T15:09:25.126 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1800999999999999e-05 2020-08-07T15:09:25.126 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.127 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.127 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:25.127 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251454+0000", 2020-08-07T15:09:25.127 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99977072 2020-08-07T15:09:25.127 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:25.128 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:25.128 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:25.128 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.128 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.128 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37069 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:25.135 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:09:25.136 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 30.397883747000002, 2020-08-07T15:09:25.136 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99870607899999997, 2020-08-07T15:09:25.136 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:25.136 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:25.136 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:25.137 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:25.137 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:25.137 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37069 2020-08-07T15:09:25.137 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.137 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:25.138 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.138 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:25.138 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:09:25.138 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:25.138 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.139 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.139 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:25.139 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:09:25.139 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3755999999999999e-05 2020-08-07T15:09:25.139 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.140 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.140 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:25.140 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252831+0000", 2020-08-07T15:09:25.140 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7919999999999998e-06 2020-08-07T15:09:25.140 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.141 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.141 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:25.141 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252836+0000", 2020-08-07T15:09:25.141 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1409999999999999e-05 2020-08-07T15:09:25.141 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.142 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.142 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:25.142 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252848+0000", 2020-08-07T15:09:25.142 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6380000000000001e-06 2020-08-07T15:09:25.143 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.143 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.143 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:25.143 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252853+0000", 2020-08-07T15:09:25.143 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3244e-05 2020-08-07T15:09:25.144 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.144 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.144 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:25.144 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252907+0000", 2020-08-07T15:09:25.144 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1470000000000007e-06 2020-08-07T15:09:25.145 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.146 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.147 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:25.147 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252916+0000", 2020-08-07T15:09:25.148 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99856497499999997 2020-08-07T15:09:25.148 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.149 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.149 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:25.149 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251481+0000", 2020-08-07T15:09:25.149 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2116999999999999e-05 2020-08-07T15:09:25.150 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.150 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.150 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:25.150 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251493+0000", 2020-08-07T15:09:25.150 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99870607899999997 2020-08-07T15:09:25.152 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:25.152 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:25.152 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:25.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.153 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37070 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:25.153 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:09:25.153 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 30.397779484000001, 2020-08-07T15:09:25.153 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99863946699999995, 2020-08-07T15:09:25.153 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:25.154 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:25.154 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:25.154 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:25.154 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:25.154 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37070 2020-08-07T15:09:25.155 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.155 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:25.155 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.155 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:25.155 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:09:25.156 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:25.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.156 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:25.157 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:09:25.157 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9984000000000001e-05 2020-08-07T15:09:25.157 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.158 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.158 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:25.158 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252941+0000", 2020-08-07T15:09:25.158 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7010000000000004e-06 2020-08-07T15:09:25.158 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.159 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.159 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:25.159 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252946+0000", 2020-08-07T15:09:25.159 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1144999999999999e-05 2020-08-07T15:09:25.159 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.160 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.160 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:25.160 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252957+0000", 2020-08-07T15:09:25.160 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6920000000000001e-06 2020-08-07T15:09:25.160 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.161 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.161 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:25.161 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252962+0000", 2020-08-07T15:09:25.161 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6739000000000002e-05 2020-08-07T15:09:25.161 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.162 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.162 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:25.162 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252998+0000", 2020-08-07T15:09:25.162 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.503e-06 2020-08-07T15:09:25.162 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.163 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.163 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:25.163 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.253006+0000", 2020-08-07T15:09:25.163 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99851375499999995 2020-08-07T15:09:25.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.163 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:25.164 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251520+0000", 2020-08-07T15:09:25.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0947999999999999e-05 2020-08-07T15:09:25.164 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.166 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.166 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:25.168 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251530+0000", 2020-08-07T15:09:25.168 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99863946699999995 2020-08-07T15:09:25.168 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:25.168 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:25.168 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:25.169 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.169 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.169 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42505 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:09:25.169 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:09:25.171 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 24.348438953999999, 2020-08-07T15:09:25.172 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0024736009999999, 2020-08-07T15:09:25.172 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:25.172 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:25.174 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:25.174 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:25.175 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:25.175 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42505 2020-08-07T15:09:25.175 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.175 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:25.175 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.176 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:25.176 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:09:25.176 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:25.179 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.180 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.180 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:25.180 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:09:25.180 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011052600000000001 2020-08-07T15:09:25.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.181 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.181 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:25.181 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302342+0000", 2020-08-07T15:09:25.181 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.781e-06 2020-08-07T15:09:25.182 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:25.182 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302352+0000", 2020-08-07T15:09:25.182 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7646e-05 2020-08-07T15:09:25.183 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.183 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.183 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:25.183 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302369+0000", 2020-08-07T15:09:25.184 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3629999999999994e-06 2020-08-07T15:09:25.184 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.184 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.184 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:25.184 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302378+0000", 2020-08-07T15:09:25.185 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8007000000000003e-05 2020-08-07T15:09:25.185 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.185 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.186 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:25.186 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302426+0000", 2020-08-07T15:09:25.186 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5159999999999994e-06 2020-08-07T15:09:25.187 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.187 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.187 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:25.187 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302434+0000", 2020-08-07T15:09:25.187 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99151562699999995 2020-08-07T15:09:25.188 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.188 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.188 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:25.189 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.293950+0000", 2020-08-07T15:09:25.189 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020138099999999999 2020-08-07T15:09:25.189 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.211 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.212 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:25.212 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294151+0000", 2020-08-07T15:09:25.213 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024761 2020-08-07T15:09:25.213 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.213 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.213 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:09:25.232 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294399+0000", 2020-08-07T15:09:25.232 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0027714369999999999 2020-08-07T15:09:25.240 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.241 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.241 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:25.241 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297170+0000", 2020-08-07T15:09:25.242 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.234e-06 2020-08-07T15:09:25.242 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.242 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.242 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:25.242 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297172+0000", 2020-08-07T15:09:25.250 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5384999999999998e-05 2020-08-07T15:09:25.251 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.251 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.251 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:25.251 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297237+0000", 2020-08-07T15:09:25.251 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0073576980000000002 2020-08-07T15:09:25.252 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.252 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.252 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:25.252 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304595+0000", 2020-08-07T15:09:25.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1006000000000001e-05 2020-08-07T15:09:25.253 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.253 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:25.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304626+0000", 2020-08-07T15:09:25.253 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6000000000000003e-07 2020-08-07T15:09:25.262 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.263 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.263 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:25.263 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304626+0000", 2020-08-07T15:09:25.263 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8724e-05 2020-08-07T15:09:25.264 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.264 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.264 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:25.264 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304705+0000", 2020-08-07T15:09:25.264 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0024736009999999 2020-08-07T15:09:25.266 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:25.266 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:25.267 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:25.267 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.267 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.267 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42506 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:09:25.267 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:09:25.268 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 24.348247741000002, 2020-08-07T15:09:25.268 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002465226, 2020-08-07T15:09:25.268 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:25.269 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:25.269 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:25.269 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:25.269 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:25.270 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42506 2020-08-07T15:09:25.277 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.278 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:25.278 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.279 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:25.279 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:09:25.279 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:25.279 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.279 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.280 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:25.280 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:09:25.280 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8780999999999999e-05 2020-08-07T15:09:25.280 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.281 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.281 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:25.281 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302472+0000", 2020-08-07T15:09:25.281 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3740000000000003e-06 2020-08-07T15:09:25.281 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.282 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.282 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:25.282 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302478+0000", 2020-08-07T15:09:25.282 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1155000000000001e-05 2020-08-07T15:09:25.282 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.283 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.283 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:25.283 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302489+0000", 2020-08-07T15:09:25.283 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7559999999999996e-06 2020-08-07T15:09:25.284 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.284 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.284 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:25.285 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302495+0000", 2020-08-07T15:09:25.285 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4128000000000003e-05 2020-08-07T15:09:25.285 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.285 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.285 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:25.286 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302529+0000", 2020-08-07T15:09:25.286 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.516e-06 2020-08-07T15:09:25.286 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.286 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.286 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:25.287 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302535+0000", 2020-08-07T15:09:25.287 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99240899299999996 2020-08-07T15:09:25.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.287 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:25.289 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294944+0000", 2020-08-07T15:09:25.290 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000115806 2020-08-07T15:09:25.290 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.290 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.291 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:25.291 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295060+0000", 2020-08-07T15:09:25.291 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000205002 2020-08-07T15:09:25.292 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.292 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.293 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:09:25.293 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295265+0000", 2020-08-07T15:09:25.293 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043414400000000002 2020-08-07T15:09:25.294 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.294 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.296 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:25.296 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299607+0000", 2020-08-07T15:09:25.296 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2229999999999998e-06 2020-08-07T15:09:25.297 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.297 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.298 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:25.298 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299612+0000", 2020-08-07T15:09:25.298 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5797000000000002e-05 2020-08-07T15:09:25.299 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.299 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.300 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:25.300 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299658+0000", 2020-08-07T15:09:25.301 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005095044 2020-08-07T15:09:25.301 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.301 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.302 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:25.302 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304753+0000", 2020-08-07T15:09:25.302 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3121999999999996e-05 2020-08-07T15:09:25.303 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.303 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.303 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:25.304 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304816+0000", 2020-08-07T15:09:25.305 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5700000000000001e-07 2020-08-07T15:09:25.305 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.305 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.306 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:25.306 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304817+0000", 2020-08-07T15:09:25.306 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1231999999999998e-05 2020-08-07T15:09:25.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.307 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.307 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:25.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304888+0000", 2020-08-07T15:09:25.308 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002465226 2020-08-07T15:09:25.308 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:25.309 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:25.313 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:25.313 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.313 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.314 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42507 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:09:25.314 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:09:25.314 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 24.348136970999999, 2020-08-07T15:09:25.314 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002567097, 2020-08-07T15:09:25.314 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:25.315 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:25.315 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:25.315 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:25.315 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:25.315 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42507 2020-08-07T15:09:25.316 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.316 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:25.316 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.316 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:25.316 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:09:25.317 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:25.317 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.317 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.317 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:25.317 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:09:25.318 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7679000000000002e-05 2020-08-07T15:09:25.318 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.318 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.318 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:25.320 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302581+0000", 2020-08-07T15:09:25.320 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2299999999999999e-06 2020-08-07T15:09:25.320 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.320 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.321 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:25.321 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302586+0000", 2020-08-07T15:09:25.526 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0886999999999999e-05 2020-08-07T15:09:25.526 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.526 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.527 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:25.527 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302597+0000", 2020-08-07T15:09:25.527 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4129999999999997e-06 2020-08-07T15:09:25.527 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.527 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.528 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:25.528 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302603+0000", 2020-08-07T15:09:25.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3963000000000001e-05 2020-08-07T15:09:25.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.529 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:25.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302627+0000", 2020-08-07T15:09:25.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.33e-06 2020-08-07T15:09:25.541 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.542 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:25.542 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302631+0000", 2020-08-07T15:09:25.542 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99319860900000001 2020-08-07T15:09:25.562 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.562 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.563 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:25.563 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295830+0000", 2020-08-07T15:09:25.563 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000116012 2020-08-07T15:09:25.563 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.564 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:25.564 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295946+0000", 2020-08-07T15:09:25.564 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000230242 2020-08-07T15:09:25.564 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.565 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.565 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:09:25.598 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296176+0000", 2020-08-07T15:09:25.598 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003494455 2020-08-07T15:09:25.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:25.605 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299670+0000", 2020-08-07T15:09:25.605 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0299999999999998e-07 2020-08-07T15:09:25.605 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.605 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.605 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:25.606 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299671+0000", 2020-08-07T15:09:25.606 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7298000000000001e-05 2020-08-07T15:09:25.606 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.606 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.606 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:25.607 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299708+0000", 2020-08-07T15:09:25.636 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0052628839999999998 2020-08-07T15:09:25.650 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.651 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.651 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:25.651 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304971+0000", 2020-08-07T15:09:25.660 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3754e-05 2020-08-07T15:09:25.661 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.661 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.661 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:25.661 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305035+0000", 2020-08-07T15:09:25.661 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9500000000000001e-07 2020-08-07T15:09:25.662 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.662 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.662 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:25.662 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305036+0000", 2020-08-07T15:09:25.662 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4843000000000002e-05 2020-08-07T15:09:25.674 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.675 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.676 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:25.676 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305101+0000", 2020-08-07T15:09:25.676 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002567097 2020-08-07T15:09:25.677 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:25.677 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:25.684 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:25.691 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.691 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.691 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42508 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:09:25.692 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:09:25.692 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 24.337611952, 2020-08-07T15:09:25.693 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99221695600000004, 2020-08-07T15:09:25.693 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:25.693 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:25.702 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:25.702 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:25.703 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:25.703 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42508 2020-08-07T15:09:25.703 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.703 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:25.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.704 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:25.704 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:09:25.704 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:25.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.705 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:25.705 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:09:25.705 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3692e-05 2020-08-07T15:09:25.705 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.721 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.722 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:25.722 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313112+0000", 2020-08-07T15:09:25.722 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7399999999999998e-06 2020-08-07T15:09:25.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.725 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:25.725 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313119+0000", 2020-08-07T15:09:25.726 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5153999999999999e-05 2020-08-07T15:09:25.726 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.726 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.726 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:25.727 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313134+0000", 2020-08-07T15:09:25.727 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3039999999999999e-06 2020-08-07T15:09:25.727 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.727 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.727 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:25.728 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313141+0000", 2020-08-07T15:09:25.728 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6516000000000001e-05 2020-08-07T15:09:25.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:25.729 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313178+0000", 2020-08-07T15:09:25.729 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0949999999999998e-06 2020-08-07T15:09:25.729 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.729 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.743 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:25.743 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313185+0000", 2020-08-07T15:09:25.744 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.98342834999999995 2020-08-07T15:09:25.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:25.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296613+0000", 2020-08-07T15:09:25.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2230999999999993e-05 2020-08-07T15:09:25.745 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.755 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:25.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296686+0000", 2020-08-07T15:09:25.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013121300000000001 2020-08-07T15:09:25.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:09:25.757 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296817+0000", 2020-08-07T15:09:25.781 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044228050000000001 2020-08-07T15:09:25.782 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:25.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301240+0000", 2020-08-07T15:09:25.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0500000000000002e-07 2020-08-07T15:09:25.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.792 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.792 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:25.793 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301241+0000", 2020-08-07T15:09:25.793 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2434000000000006e-05 2020-08-07T15:09:25.793 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.814 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.814 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:25.815 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301303+0000", 2020-08-07T15:09:25.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038622919999999998 2020-08-07T15:09:25.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.815 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:25.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305165+0000", 2020-08-07T15:09:25.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8012000000000001e-05 2020-08-07T15:09:25.816 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:25.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305203+0000", 2020-08-07T15:09:25.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5299999999999999e-07 2020-08-07T15:09:25.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:25.828 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305204+0000", 2020-08-07T15:09:25.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1760000000000004e-05 2020-08-07T15:09:25.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:25.846 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:25.846 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:25.846 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305275+0000", 2020-08-07T15:09:25.847 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99221695600000004 2020-08-07T15:09:25.847 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:25.847 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:25.847 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:25.847 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:25.848 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:25.848 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:09:25.849 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:09:25.858 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:09:25.860 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 59 16 72507 72491 9.59801 10.4062 0.00290919 0.0130159 2020-08-07T15:09:25.861 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.2 to 1 (10000) 2020-08-07T15:09:25.861 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 3 1 2020-08-07T15:09:25.869 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:25.858+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:25.960 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:25.959+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:25.999 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:09:26.002 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:09:26.002 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2020-08-07T15:09:26.002 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:09:26.003 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:09:26.061 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:26.060+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:26.122 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:09:26.122 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:09:26.123 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:09:26.123 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:09:26.123 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:09:26.136 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:09:26.179 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:26.178+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:26.280 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:2020-08-07T15:09:26.277742+0000 min lat: 0.00103494 max lat: 2.05564 avg lat: 0.0130176 2020-08-07T15:09:26.281 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-08-07T15:09:26.281 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 60 16 73023 73007 9.50522 4.03125 0.340117 0.0130176 2020-08-07T15:09:26.282 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:26.279+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:26.307 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:09:26.308 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:09:26.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:09:26.309 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:09:26.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.309 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:108376 3.3 e62/20 ondisk, result = 0)", 2020-08-07T15:09:26.359 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:09:26.368 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 95.620007870999999, 2020-08-07T15:09:26.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62926872599999994, 2020-08-07T15:09:26.372 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:26.372 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:26.372 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:26.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:26.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:09:26.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:26.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:26.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:09:26.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8491999999999998e-05 2020-08-07T15:09:26.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:26.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683061+0000", 2020-08-07T15:09:26.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9099999999999998e-06 2020-08-07T15:09:26.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:26.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683065+0000", 2020-08-07T15:09:26.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62906670600000003 2020-08-07T15:09:26.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:26.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312131+0000", 2020-08-07T15:09:26.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2147000000000001e-05 2020-08-07T15:09:26.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:26.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312143+0000", 2020-08-07T15:09:26.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5152e-05 2020-08-07T15:09:26.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:26.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312239+0000", 2020-08-07T15:09:26.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5089999999999993e-06 2020-08-07T15:09:26.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:26.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312247+0000", 2020-08-07T15:09:26.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3810000000000001e-05 2020-08-07T15:09:26.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:26.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312311+0000", 2020-08-07T15:09:26.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62926872599999994 2020-08-07T15:09:26.417 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:26.418 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:26.418 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:26.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.419 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "MOSDPGPushReply(3.1 62/60 [PushReplyOp(3:8329f37e:::benchmark_data_smithi074_35447_object11915:head)])", 2020-08-07T15:09:26.419 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:09:26.420 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 95.444649854999994, 2020-08-07T15:09:26.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996, 2020-08-07T15:09:26.421 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:26.421 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:26.445 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:26.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:26.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:09:26.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:26.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:26.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:09:26.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5372000000000001e-05 2020-08-07T15:09:26.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:26.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858416+0000", 2020-08-07T15:09:26.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6899999999999998e-06 2020-08-07T15:09:26.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:26.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858419+0000", 2020-08-07T15:09:26.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63907904299999996 2020-08-07T15:09:26.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:26.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497498+0000", 2020-08-07T15:09:26.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3332999999999999e-05 2020-08-07T15:09:26.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:26.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497512+0000", 2020-08-07T15:09:26.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000450875 2020-08-07T15:09:26.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:26.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497962+0000", 2020-08-07T15:09:26.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000267779 2020-08-07T15:09:26.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:26.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.498230+0000", 2020-08-07T15:09:26.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996 2020-08-07T15:09:26.580 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:26.581 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:26.581 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:26.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.588 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:145792 3.1 3:881e0420:::benchmark_data_smithi074_35447_object16183:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:09:26.588 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:09:26.589 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 71.636243973999996, 2020-08-07T15:09:26.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997, 2020-08-07T15:09:26.589 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:26.589 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:26.589 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:26.590 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:26.590 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:26.590 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 145792 2020-08-07T15:09:26.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.590 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:26.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:26.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:09:26.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:26.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:26.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:09:26.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7489000000000001e-05 2020-08-07T15:09:26.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:26.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666824+0000", 2020-08-07T15:09:26.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0169999999999998e-06 2020-08-07T15:09:26.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:26.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666828+0000", 2020-08-07T15:09:26.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9829999999999996e-06 2020-08-07T15:09:26.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:26.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666836+0000", 2020-08-07T15:09:26.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.481e-06 2020-08-07T15:09:26.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:26.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666840+0000", 2020-08-07T15:09:26.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5299000000000001e-05 2020-08-07T15:09:26.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:26.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666885+0000", 2020-08-07T15:09:26.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014456500000000001 2020-08-07T15:09:26.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:26.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667030+0000", 2020-08-07T15:09:26.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017677799999999999 2020-08-07T15:09:26.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,5", 2020-08-07T15:09:26.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667207+0000", 2020-08-07T15:09:26.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004149649 2020-08-07T15:09:26.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:26.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671356+0000", 2020-08-07T15:09:26.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019339649999999999 2020-08-07T15:09:26.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:26.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673290+0000", 2020-08-07T15:09:26.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73346620500000004 2020-08-07T15:09:26.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:26.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406756+0000", 2020-08-07T15:09:26.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4700000000000001e-06 2020-08-07T15:09:26.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:26.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406759+0000", 2020-08-07T15:09:26.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6258999999999998e-05 2020-08-07T15:09:26.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:26.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406815+0000", 2020-08-07T15:09:26.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0126000000000002e-05 2020-08-07T15:09:26.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:26.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406855+0000", 2020-08-07T15:09:26.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997 2020-08-07T15:09:26.663 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:26.664 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:26.679 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:26.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.679 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:145792 3.1 e75/67 ondisk, result = 0)", 2020-08-07T15:09:26.680 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:09:26.680 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 71.631859086000006, 2020-08-07T15:09:26.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003, 2020-08-07T15:09:26.680 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:26.680 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:26.681 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:26.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:26.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:09:26.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:26.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:26.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:09:26.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2830999999999998e-05 2020-08-07T15:09:26.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:26.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671224+0000", 2020-08-07T15:09:26.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:09:26.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:26.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671229+0000", 2020-08-07T15:09:26.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.735414864 2020-08-07T15:09:26.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:26.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406644+0000", 2020-08-07T15:09:26.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1250000000000004e-06 2020-08-07T15:09:26.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:26.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406651+0000", 2020-08-07T15:09:26.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6567e-05 2020-08-07T15:09:26.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:26.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406737+0000", 2020-08-07T15:09:26.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0054000000000001e-05 2020-08-07T15:09:26.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:26.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406747+0000", 2020-08-07T15:09:26.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011391999999999999 2020-08-07T15:09:26.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:26.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406861+0000", 2020-08-07T15:09:26.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003 2020-08-07T15:09:26.754 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:26.754 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:26.755 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:26.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.755 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30921 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:09:26.755 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:09:26.756 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.597110712000003, 2020-08-07T15:09:26.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001, 2020-08-07T15:09:26.756 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:26.757 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:26.757 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:26.757 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:26.757 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:26.757 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30921 2020-08-07T15:09:26.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.758 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:26.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:26.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:09:26.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:26.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:26.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:09:26.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6954999999999998e-05 2020-08-07T15:09:26.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:26.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706016+0000", 2020-08-07T15:09:26.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0229999999999992e-06 2020-08-07T15:09:26.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:26.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706024+0000", 2020-08-07T15:09:26.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.432e-05 2020-08-07T15:09:26.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:26.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706039+0000", 2020-08-07T15:09:26.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8490000000000006e-06 2020-08-07T15:09:26.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:26.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706046+0000", 2020-08-07T15:09:26.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6837999999999999e-05 2020-08-07T15:09:26.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:26.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706113+0000", 2020-08-07T15:09:26.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3553000000000001e-05 2020-08-07T15:09:26.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:26.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706127+0000", 2020-08-07T15:09:26.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77277201699999998 2020-08-07T15:09:26.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:26.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.478899+0000", 2020-08-07T15:09:26.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093978599999999996 2020-08-07T15:09:26.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:09:26.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479839+0000", 2020-08-07T15:09:26.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010376029 2020-08-07T15:09:26.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:26.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490215+0000", 2020-08-07T15:09:26.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000190657 2020-08-07T15:09:26.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:26.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490405+0000", 2020-08-07T15:09:26.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001 2020-08-07T15:09:26.807 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:26.807 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:26.807 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:26.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.808 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30922 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:09:26.808 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:09:26.808 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.596950548999999, 2020-08-07T15:09:26.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247, 2020-08-07T15:09:26.808 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:26.809 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:26.809 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:26.809 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:26.809 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:26.810 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30922 2020-08-07T15:09:26.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.811 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:26.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:26.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:09:26.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:26.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:26.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:09:26.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9178999999999997e-05 2020-08-07T15:09:26.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:26.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706159+0000", 2020-08-07T15:09:26.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4999999999999999e-06 2020-08-07T15:09:26.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:26.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706164+0000", 2020-08-07T15:09:26.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1792e-05 2020-08-07T15:09:26.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:26.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706176+0000", 2020-08-07T15:09:26.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.139e-06 2020-08-07T15:09:26.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:26.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706182+0000", 2020-08-07T15:09:26.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039724900000000002 2020-08-07T15:09:26.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:26.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706579+0000", 2020-08-07T15:09:26.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1191e-05 2020-08-07T15:09:26.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:26.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706591+0000", 2020-08-07T15:09:26.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329101300000003 2020-08-07T15:09:26.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:26.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479882+0000", 2020-08-07T15:09:26.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9565000000000001e-05 2020-08-07T15:09:26.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:09:26.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479971+0000", 2020-08-07T15:09:26.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010471263 2020-08-07T15:09:26.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:26.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490442+0000", 2020-08-07T15:09:26.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015035600000000001 2020-08-07T15:09:26.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:26.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490593+0000", 2020-08-07T15:09:26.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247 2020-08-07T15:09:26.872 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:26.872 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:26.872 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:26.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.873 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30923 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:09:26.873 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:09:26.873 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.596475488000003, 2020-08-07T15:09:26.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998, 2020-08-07T15:09:26.874 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:26.883 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:26.883 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:26.884 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:26.893 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:26.893 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30923 2020-08-07T15:09:26.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.894 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:26.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:26.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:09:26.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:26.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:26.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:09:26.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5531000000000001e-05 2020-08-07T15:09:26.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:26.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706630+0000", 2020-08-07T15:09:26.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7689999999999999e-06 2020-08-07T15:09:26.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:26.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706637+0000", 2020-08-07T15:09:26.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3577999999999999e-05 2020-08-07T15:09:26.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:26.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706650+0000", 2020-08-07T15:09:26.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1439999999999997e-06 2020-08-07T15:09:26.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:26.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706658+0000", 2020-08-07T15:09:26.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1663999999999999e-05 2020-08-07T15:09:26.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:26.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706709+0000", 2020-08-07T15:09:26.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0149000000000001e-05 2020-08-07T15:09:26.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:26.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706719+0000", 2020-08-07T15:09:26.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329694900000001 2020-08-07T15:09:26.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:26.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480016+0000", 2020-08-07T15:09:26.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7998999999999997e-05 2020-08-07T15:09:26.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:09:26.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480084+0000", 2020-08-07T15:09:26.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010547333000000001 2020-08-07T15:09:26.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:26.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490632+0000", 2020-08-07T15:09:26.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7439e-05 2020-08-07T15:09:26.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:26.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490729+0000", 2020-08-07T15:09:26.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998 2020-08-07T15:09:26.974 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:26.984 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:26.984 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:26.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.986 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30924 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:09:26.986 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:09:26.987 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.596348216000003, 2020-08-07T15:09:26.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995, 2020-08-07T15:09:26.987 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:26.987 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:26.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:26.989 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:26.989 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:26.989 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30924 2020-08-07T15:09:26.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.990 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:26.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:26.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:09:26.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:26.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:26.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:09:26.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9698999999999999e-05 2020-08-07T15:09:26.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:26.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706762+0000", 2020-08-07T15:09:26.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8080000000000001e-06 2020-08-07T15:09:26.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:26.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:26.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:27.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706767+0000", 2020-08-07T15:09:27.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1069e-05 2020-08-07T15:09:27.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:27.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706778+0000", 2020-08-07T15:09:27.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6969999999999998e-06 2020-08-07T15:09:27.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:27.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706784+0000", 2020-08-07T15:09:27.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8792999999999998e-05 2020-08-07T15:09:27.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706843+0000", 2020-08-07T15:09:27.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5449999999999998e-06 2020-08-07T15:09:27.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:27.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706850+0000", 2020-08-07T15:09:27.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.773278354 2020-08-07T15:09:27.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480129+0000", 2020-08-07T15:09:27.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1577999999999997e-05 2020-08-07T15:09:27.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:09:27.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480210+0000", 2020-08-07T15:09:27.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010541915000000001 2020-08-07T15:09:27.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490752+0000", 2020-08-07T15:09:27.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103937 2020-08-07T15:09:27.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:27.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490856+0000", 2020-08-07T15:09:27.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995 2020-08-07T15:09:27.218 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.218 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:27.218 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.231 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30925 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:09:27.231 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:09:27.231 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.596224444999997, 2020-08-07T15:09:27.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995, 2020-08-07T15:09:27.232 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:27.232 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:27.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:27.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:27.233 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:27.233 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30925 2020-08-07T15:09:27.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.234 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:27.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:27.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:09:27.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:27.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:27.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:09:27.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4664999999999999e-05 2020-08-07T15:09:27.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:27.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706880+0000", 2020-08-07T15:09:27.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7479999999999999e-06 2020-08-07T15:09:27.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:27.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706886+0000", 2020-08-07T15:09:27.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1838000000000001e-05 2020-08-07T15:09:27.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:27.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706898+0000", 2020-08-07T15:09:27.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0089999999999998e-06 2020-08-07T15:09:27.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:27.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706903+0000", 2020-08-07T15:09:27.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3559e-05 2020-08-07T15:09:27.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706956+0000", 2020-08-07T15:09:27.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0309000000000001e-05 2020-08-07T15:09:27.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:27.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706967+0000", 2020-08-07T15:09:27.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77328290499999996 2020-08-07T15:09:27.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480250+0000", 2020-08-07T15:09:27.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8680999999999997e-05 2020-08-07T15:09:27.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:09:27.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480298+0000", 2020-08-07T15:09:27.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01059272 2020-08-07T15:09:27.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490891+0000", 2020-08-07T15:09:27.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014580300000000001 2020-08-07T15:09:27.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:27.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491037+0000", 2020-08-07T15:09:27.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995 2020-08-07T15:09:27.292 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.292 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:27.293 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.300 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30926 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:09:27.300 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:09:27.301 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.596107576000001, 2020-08-07T15:09:27.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005, 2020-08-07T15:09:27.301 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:27.301 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:27.301 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:27.302 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:27.302 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:27.302 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30926 2020-08-07T15:09:27.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.302 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:27.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:27.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:09:27.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:27.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:09:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5223000000000003e-05 2020-08-07T15:09:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:27.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706998+0000", 2020-08-07T15:09:27.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4809999999999997e-06 2020-08-07T15:09:27.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:27.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707003+0000", 2020-08-07T15:09:27.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1663e-05 2020-08-07T15:09:27.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:27.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707015+0000", 2020-08-07T15:09:27.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9729999999999997e-06 2020-08-07T15:09:27.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:27.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707020+0000", 2020-08-07T15:09:27.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2565999999999999e-05 2020-08-07T15:09:27.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707062+0000", 2020-08-07T15:09:27.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8944e-05 2020-08-07T15:09:27.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:27.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707081+0000", 2020-08-07T15:09:27.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77325885000000005 2020-08-07T15:09:27.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480340+0000", 2020-08-07T15:09:27.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0321000000000006e-05 2020-08-07T15:09:27.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:09:27.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480420+0000", 2020-08-07T15:09:27.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010655958 2020-08-07T15:09:27.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491076+0000", 2020-08-07T15:09:27.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9784999999999995e-05 2020-08-07T15:09:27.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:27.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491166+0000", 2020-08-07T15:09:27.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005 2020-08-07T15:09:27.360 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.360 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:27.360 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.361 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30927 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:09:27.361 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:09:27.361 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.595989756999998, 2020-08-07T15:09:27.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999, 2020-08-07T15:09:27.361 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:27.427 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:27.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:27.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:27.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:27.428 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30927 2020-08-07T15:09:27.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.428 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:27.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:27.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:09:27.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:27.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:27.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:09:27.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9861999999999997e-05 2020-08-07T15:09:27.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:27.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707120+0000", 2020-08-07T15:09:27.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8969999999999996e-06 2020-08-07T15:09:27.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:27.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707126+0000", 2020-08-07T15:09:27.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1248e-05 2020-08-07T15:09:27.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:27.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707137+0000", 2020-08-07T15:09:27.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.955e-06 2020-08-07T15:09:27.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:27.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707142+0000", 2020-08-07T15:09:27.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0643999999999998e-05 2020-08-07T15:09:27.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707193+0000", 2020-08-07T15:09:27.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1983e-05 2020-08-07T15:09:27.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:27.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707205+0000", 2020-08-07T15:09:27.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77324914 2020-08-07T15:09:27.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480454+0000", 2020-08-07T15:09:27.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4119999999999998e-05 2020-08-07T15:09:27.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:09:27.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480498+0000", 2020-08-07T15:09:27.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010709707000000001 2020-08-07T15:09:27.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491208+0000", 2020-08-07T15:09:27.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014433800000000001 2020-08-07T15:09:27.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:27.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491352+0000", 2020-08-07T15:09:27.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999 2020-08-07T15:09:27.527 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.528 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:27.528 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.528 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30928 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:09:27.529 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:09:27.529 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.595865400000001, 2020-08-07T15:09:27.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998, 2020-08-07T15:09:27.529 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:27.529 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:27.530 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:27.530 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:27.530 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:27.530 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30928 2020-08-07T15:09:27.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.531 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:27.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:27.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:09:27.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:27.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:27.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:09:27.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6582000000000001e-05 2020-08-07T15:09:27.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:27.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707241+0000", 2020-08-07T15:09:27.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9190000000000004e-06 2020-08-07T15:09:27.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:27.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707247+0000", 2020-08-07T15:09:27.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000188822 2020-08-07T15:09:27.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:27.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707436+0000", 2020-08-07T15:09:27.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2940000000000002e-06 2020-08-07T15:09:27.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:27.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707442+0000", 2020-08-07T15:09:27.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7800000000000003e-05 2020-08-07T15:09:27.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707490+0000", 2020-08-07T15:09:27.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2020-08-07T15:09:27.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:27.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707501+0000", 2020-08-07T15:09:27.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77303498800000003 2020-08-07T15:09:27.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480536+0000", 2020-08-07T15:09:27.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7723000000000001e-05 2020-08-07T15:09:27.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:09:27.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480614+0000", 2020-08-07T15:09:27.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010812183 2020-08-07T15:09:27.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491426+0000", 2020-08-07T15:09:27.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5502999999999999e-05 2020-08-07T15:09:27.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:27.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491502+0000", 2020-08-07T15:09:27.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998 2020-08-07T15:09:27.551 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.551 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:27.551 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.552 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31009 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:09:27.552 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:09:27.552 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.571909093999999, 2020-08-07T15:09:27.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002, 2020-08-07T15:09:27.553 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:27.553 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:27.553 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:27.553 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:27.553 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:27.554 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31009 2020-08-07T15:09:27.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.554 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:27.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:27.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:09:27.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:27.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:27.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:09:27.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2429000000000002e-05 2020-08-07T15:09:27.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:27.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731193+0000", 2020-08-07T15:09:27.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9760000000000003e-06 2020-08-07T15:09:27.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:27.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731198+0000", 2020-08-07T15:09:27.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3597e-05 2020-08-07T15:09:27.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:27.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731212+0000", 2020-08-07T15:09:27.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1709999999999996e-06 2020-08-07T15:09:27.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:27.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731219+0000", 2020-08-07T15:09:27.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5402999999999997e-05 2020-08-07T15:09:27.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731285+0000", 2020-08-07T15:09:27.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3495e-05 2020-08-07T15:09:27.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:27.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731298+0000", 2020-08-07T15:09:27.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74935423599999995 2020-08-07T15:09:27.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480652+0000", 2020-08-07T15:09:27.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000137806 2020-08-07T15:09:27.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:27.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480790+0000", 2020-08-07T15:09:27.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022272399999999999 2020-08-07T15:09:27.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:09:27.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481013+0000", 2020-08-07T15:09:27.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003223381 2020-08-07T15:09:27.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:27.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484236+0000", 2020-08-07T15:09:27.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1370000000000001e-06 2020-08-07T15:09:27.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:27.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484237+0000", 2020-08-07T15:09:27.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6288e-05 2020-08-07T15:09:27.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:27.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484294+0000", 2020-08-07T15:09:27.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.007260402 2020-08-07T15:09:27.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:27.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491554+0000", 2020-08-07T15:09:27.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7358000000000003e-05 2020-08-07T15:09:27.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:27.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491601+0000", 2020-08-07T15:09:27.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1100000000000004e-07 2020-08-07T15:09:27.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:27.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491602+0000", 2020-08-07T15:09:27.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013436500000000001 2020-08-07T15:09:27.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:27.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491737+0000", 2020-08-07T15:09:27.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002 2020-08-07T15:09:27.589 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.589 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:27.590 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.590 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31010 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:09:27.590 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:09:27.591 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.571783140000001, 2020-08-07T15:09:27.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001, 2020-08-07T15:09:27.591 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:27.591 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:27.591 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:27.592 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:27.592 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:27.592 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31010 2020-08-07T15:09:27.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.592 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:27.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:27.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:09:27.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:27.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:27.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:09:27.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2398000000000002e-05 2020-08-07T15:09:27.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:27.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731319+0000", 2020-08-07T15:09:27.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7100000000000004e-06 2020-08-07T15:09:27.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:27.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731325+0000", 2020-08-07T15:09:27.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4328000000000001e-05 2020-08-07T15:09:27.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:27.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731339+0000", 2020-08-07T15:09:27.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3310000000000003e-06 2020-08-07T15:09:27.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:27.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731345+0000", 2020-08-07T15:09:27.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8952999999999998e-05 2020-08-07T15:09:27.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731384+0000", 2020-08-07T15:09:27.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0036e-05 2020-08-07T15:09:27.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:27.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731394+0000", 2020-08-07T15:09:27.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75027681300000004 2020-08-07T15:09:27.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481670+0000", 2020-08-07T15:09:27.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016555899999999999 2020-08-07T15:09:27.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:27.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481836+0000", 2020-08-07T15:09:27.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028760999999999999 2020-08-07T15:09:27.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:09:27.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482124+0000", 2020-08-07T15:09:27.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038342680000000001 2020-08-07T15:09:27.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:27.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485958+0000", 2020-08-07T15:09:27.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2559999999999999e-06 2020-08-07T15:09:27.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:27.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485959+0000", 2020-08-07T15:09:27.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3758999999999995e-05 2020-08-07T15:09:27.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:27.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486043+0000", 2020-08-07T15:09:27.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057590419999999998 2020-08-07T15:09:27.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:27.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491802+0000", 2020-08-07T15:09:27.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5757000000000001e-05 2020-08-07T15:09:27.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:27.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:09:27.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9299999999999997e-07 2020-08-07T15:09:27.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:27.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:09:27.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012496999999999999 2020-08-07T15:09:27.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:27.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491993+0000", 2020-08-07T15:09:27.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001 2020-08-07T15:09:27.612 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.612 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:27.612 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.613 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31011 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:09:27.613 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:09:27.613 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.571666935000003, 2020-08-07T15:09:27.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997, 2020-08-07T15:09:27.613 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:27.614 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:27.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:27.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:27.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:27.614 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31011 2020-08-07T15:09:27.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.615 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:27.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:27.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:09:27.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:27.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:27.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:09:27.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9539999999999997e-05 2020-08-07T15:09:27.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:27.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731433+0000", 2020-08-07T15:09:27.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9209999999999997e-06 2020-08-07T15:09:27.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:27.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731439+0000", 2020-08-07T15:09:27.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4246999999999999e-05 2020-08-07T15:09:27.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:27.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731453+0000", 2020-08-07T15:09:27.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8130000000000003e-06 2020-08-07T15:09:27.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:27.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731458+0000", 2020-08-07T15:09:27.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6668000000000001e-05 2020-08-07T15:09:27.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731504+0000", 2020-08-07T15:09:27.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1112e-05 2020-08-07T15:09:27.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:27.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731515+0000", 2020-08-07T15:09:27.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75121332100000004 2020-08-07T15:09:27.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482729+0000", 2020-08-07T15:09:27.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1810000000000004e-05 2020-08-07T15:09:27.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:27.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482811+0000", 2020-08-07T15:09:27.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018001399999999999 2020-08-07T15:09:27.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:09:27.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482991+0000", 2020-08-07T15:09:27.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030701140000000001 2020-08-07T15:09:27.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:27.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486061+0000", 2020-08-07T15:09:27.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4499999999999996e-07 2020-08-07T15:09:27.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:27.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486062+0000", 2020-08-07T15:09:27.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5366999999999998e-05 2020-08-07T15:09:27.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:27.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486147+0000", 2020-08-07T15:09:27.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056872779999999996 2020-08-07T15:09:27.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:27.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491834+0000", 2020-08-07T15:09:27.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024645100000000002 2020-08-07T15:09:27.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:27.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:09:27.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1200000000000003e-07 2020-08-07T15:09:27.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:27.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:09:27.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1451000000000002e-05 2020-08-07T15:09:27.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:27.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492173+0000", 2020-08-07T15:09:27.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997 2020-08-07T15:09:27.636 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.636 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:27.636 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.637 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31012 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:09:27.637 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:09:27.638 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.571012816, 2020-08-07T15:09:27.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003, 2020-08-07T15:09:27.638 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:27.638 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:27.638 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:27.639 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:27.639 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:27.639 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31012 2020-08-07T15:09:27.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.639 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:27.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:27.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:09:27.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:27.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:27.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:09:27.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6344999999999997e-05 2020-08-07T15:09:27.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:27.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732094+0000", 2020-08-07T15:09:27.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2920000000000001e-06 2020-08-07T15:09:27.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:27.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732100+0000", 2020-08-07T15:09:27.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.189e-05 2020-08-07T15:09:27.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:27.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732112+0000", 2020-08-07T15:09:27.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3720000000000001e-06 2020-08-07T15:09:27.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:27.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732118+0000", 2020-08-07T15:09:27.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2514000000000004e-05 2020-08-07T15:09:27.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732191+0000", 2020-08-07T15:09:27.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4610000000000001e-05 2020-08-07T15:09:27.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:27.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732205+0000", 2020-08-07T15:09:27.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75132974100000005 2020-08-07T15:09:27.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483535+0000", 2020-08-07T15:09:27.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9185999999999996e-05 2020-08-07T15:09:27.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:27.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483624+0000", 2020-08-07T15:09:27.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015683 2020-08-07T15:09:27.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:09:27.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483781+0000", 2020-08-07T15:09:27.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046202889999999996 2020-08-07T15:09:27.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:27.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488401+0000", 2020-08-07T15:09:27.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6700000000000002e-07 2020-08-07T15:09:27.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:27.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488402+0000", 2020-08-07T15:09:27.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1487000000000001e-05 2020-08-07T15:09:27.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:27.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488484+0000", 2020-08-07T15:09:27.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035520830000000002 2020-08-07T15:09:27.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:27.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492036+0000", 2020-08-07T15:09:27.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029266600000000002 2020-08-07T15:09:27.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:27.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492328+0000", 2020-08-07T15:09:27.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5099999999999998e-07 2020-08-07T15:09:27.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:27.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492329+0000", 2020-08-07T15:09:27.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000201299 2020-08-07T15:09:27.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:27.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492531+0000", 2020-08-07T15:09:27.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003 2020-08-07T15:09:27.671 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.686 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:27.686 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.700 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31013 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:09:27.701 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:09:27.701 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.570801185000001, 2020-08-07T15:09:27.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996, 2020-08-07T15:09:27.701 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:27.701 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:27.702 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:27.702 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:27.702 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:27.702 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31013 2020-08-07T15:09:27.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.703 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:27.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:27.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:09:27.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:27.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:27.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:09:27.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8949000000000002e-05 2020-08-07T15:09:27.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:27.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732298+0000", 2020-08-07T15:09:27.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1399999999999997e-06 2020-08-07T15:09:27.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:27.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732304+0000", 2020-08-07T15:09:27.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.154e-05 2020-08-07T15:09:27.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:27.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732315+0000", 2020-08-07T15:09:27.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1319999999999999e-06 2020-08-07T15:09:27.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:27.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732322+0000", 2020-08-07T15:09:27.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7431e-05 2020-08-07T15:09:27.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732369+0000", 2020-08-07T15:09:27.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0621999999999999e-05 2020-08-07T15:09:27.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:27.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732380+0000", 2020-08-07T15:09:27.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75196845999999995 2020-08-07T15:09:27.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484348+0000", 2020-08-07T15:09:27.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000123008 2020-08-07T15:09:27.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:27.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484471+0000", 2020-08-07T15:09:27.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017572700000000001 2020-08-07T15:09:27.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:09:27.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484647+0000", 2020-08-07T15:09:27.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038557650000000001 2020-08-07T15:09:27.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:27.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:09:27.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2020-08-07T15:09:27.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:27.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:09:27.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0714e-05 2020-08-07T15:09:27.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:27.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488584+0000", 2020-08-07T15:09:27.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036298319999999999 2020-08-07T15:09:27.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:27.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492214+0000", 2020-08-07T15:09:27.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039530099999999997 2020-08-07T15:09:27.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:27.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492609+0000", 2020-08-07T15:09:27.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5300000000000002e-07 2020-08-07T15:09:27.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:27.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492610+0000", 2020-08-07T15:09:27.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197321 2020-08-07T15:09:27.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:27.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492807+0000", 2020-08-07T15:09:27.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996 2020-08-07T15:09:27.844 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.844 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:27.844 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.845 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31014 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:09:27.845 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:09:27.845 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.569419041000003, 2020-08-07T15:09:27.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998, 2020-08-07T15:09:27.856 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:27.857 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:27.857 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:27.857 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:27.858 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:27.858 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31014 2020-08-07T15:09:27.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.859 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:27.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:27.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:09:27.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:27.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:27.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:09:27.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3743999999999999e-05 2020-08-07T15:09:27.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:27.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733685+0000", 2020-08-07T15:09:27.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1720000000000007e-06 2020-08-07T15:09:27.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:27.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733693+0000", 2020-08-07T15:09:27.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3674e-05 2020-08-07T15:09:27.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:27.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733707+0000", 2020-08-07T15:09:27.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9249999999999998e-06 2020-08-07T15:09:27.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:27.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733714+0000", 2020-08-07T15:09:27.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8338000000000003e-05 2020-08-07T15:09:27.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733772+0000", 2020-08-07T15:09:27.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3019e-05 2020-08-07T15:09:27.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:27.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733785+0000", 2020-08-07T15:09:27.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75134856299999997 2020-08-07T15:09:27.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485133+0000", 2020-08-07T15:09:27.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8739999999999995e-05 2020-08-07T15:09:27.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:27.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485212+0000", 2020-08-07T15:09:27.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151244 2020-08-07T15:09:27.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:09:27.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485363+0000", 2020-08-07T15:09:27.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032929090000000001 2020-08-07T15:09:27.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:27.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488656+0000", 2020-08-07T15:09:27.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.065e-06 2020-08-07T15:09:27.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:27.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488657+0000", 2020-08-07T15:09:27.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1123000000000003e-05 2020-08-07T15:09:27.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:27.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488729+0000", 2020-08-07T15:09:27.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035508330000000002 2020-08-07T15:09:27.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:27.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492279+0000", 2020-08-07T15:09:27.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00059376000000000003 2020-08-07T15:09:27.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:27.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492873+0000", 2020-08-07T15:09:27.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5700000000000002e-07 2020-08-07T15:09:27.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:27.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492874+0000", 2020-08-07T15:09:27.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019264 2020-08-07T15:09:27.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:27.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493066+0000", 2020-08-07T15:09:27.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998 2020-08-07T15:09:27.886 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.887 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:27.887 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.887 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31015 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:09:27.888 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:09:27.888 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.569293074999997, 2020-08-07T15:09:27.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002, 2020-08-07T15:09:27.888 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:27.888 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:27.889 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:27.889 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:27.889 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:27.889 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31015 2020-08-07T15:09:27.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.890 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:27.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:27.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:09:27.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:27.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:27.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:09:27.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3707000000000001e-05 2020-08-07T15:09:27.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:27.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733791+0000", 2020-08-07T15:09:27.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8120000000000001e-06 2020-08-07T15:09:27.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:27.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733794+0000", 2020-08-07T15:09:27.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0728e-05 2020-08-07T15:09:27.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:27.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733805+0000", 2020-08-07T15:09:27.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8279999999999999e-06 2020-08-07T15:09:27.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:27.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733811+0000", 2020-08-07T15:09:27.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2955999999999999e-05 2020-08-07T15:09:27.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733864+0000", 2020-08-07T15:09:27.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1389000000000001e-05 2020-08-07T15:09:27.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:27.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733875+0000", 2020-08-07T15:09:27.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75232711100000005 2020-08-07T15:09:27.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486202+0000", 2020-08-07T15:09:27.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027503400000000001 2020-08-07T15:09:27.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:27.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486478+0000", 2020-08-07T15:09:27.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033010400000000001 2020-08-07T15:09:27.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:09:27.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486808+0000", 2020-08-07T15:09:27.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019382609999999999 2020-08-07T15:09:27.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:27.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488746+0000", 2020-08-07T15:09:27.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9400000000000004e-07 2020-08-07T15:09:27.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:27.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488747+0000", 2020-08-07T15:09:27.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4324000000000002e-05 2020-08-07T15:09:27.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:27.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488821+0000", 2020-08-07T15:09:27.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043745640000000001 2020-08-07T15:09:27.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:27.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493196+0000", 2020-08-07T15:09:27.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030381000000000001 2020-08-07T15:09:27.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:27.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493499+0000", 2020-08-07T15:09:27.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4e-07 2020-08-07T15:09:27.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:27.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493500+0000", 2020-08-07T15:09:27.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017499 2020-08-07T15:09:27.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:27.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493675+0000", 2020-08-07T15:09:27.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002 2020-08-07T15:09:27.913 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.913 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:27.914 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.914 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31016 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:09:27.914 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:09:27.915 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.568697100000001, 2020-08-07T15:09:27.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971, 2020-08-07T15:09:27.915 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:27.915 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:27.915 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:27.916 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:27.916 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:27.916 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31016 2020-08-07T15:09:27.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.916 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:27.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:27.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:09:27.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:27.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:27.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:09:27.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0924000000000002e-05 2020-08-07T15:09:27.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:27.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734404+0000", 2020-08-07T15:09:27.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2020-08-07T15:09:27.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:27.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734410+0000", 2020-08-07T15:09:27.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3459999999999994e-06 2020-08-07T15:09:27.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:27.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734418+0000", 2020-08-07T15:09:27.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9640000000000002e-06 2020-08-07T15:09:27.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:27.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734423+0000", 2020-08-07T15:09:27.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7253999999999999e-05 2020-08-07T15:09:27.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734470+0000", 2020-08-07T15:09:27.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0249999999999997e-06 2020-08-07T15:09:27.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:27.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734477+0000", 2020-08-07T15:09:27.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75294592900000001 2020-08-07T15:09:27.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:27.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487423+0000", 2020-08-07T15:09:27.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031604300000000001 2020-08-07T15:09:27.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:27.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487739+0000", 2020-08-07T15:09:27.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000171149 2020-08-07T15:09:27.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:09:27.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487910+0000", 2020-08-07T15:09:27.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024207769999999998 2020-08-07T15:09:27.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:27.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490331+0000", 2020-08-07T15:09:27.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1650000000000001e-06 2020-08-07T15:09:27.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:27.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490332+0000", 2020-08-07T15:09:27.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9796999999999996e-05 2020-08-07T15:09:27.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:27.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490402+0000", 2020-08-07T15:09:27.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029928849999999998 2020-08-07T15:09:27.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:27.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493395+0000", 2020-08-07T15:09:27.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034903199999999999 2020-08-07T15:09:27.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:27.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493744+0000", 2020-08-07T15:09:27.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8899999999999999e-07 2020-08-07T15:09:27.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:27.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493745+0000", 2020-08-07T15:09:27.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197362 2020-08-07T15:09:27.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:27.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:27.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:27.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493942+0000", 2020-08-07T15:09:27.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971 2020-08-07T15:09:27.992 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.992 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:27.992 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.992 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:27.993 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:27.993 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:09:27.994 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:09:27.997 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:09:27.997 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.3 to 1 (10000) 2020-08-07T15:09:27.998 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 4 1 2020-08-07T15:09:27.999 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 61 16 73136 73120 9.36386 0.882812 0.00719211 0.0133399 2020-08-07T15:09:28.003 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:09:28.007 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:27.994+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:28.096 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:28.095+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:28.142 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:09:28.143 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:09:28.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.143 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73129 4.9 4.a3ada059 (undecoded) ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:28.143 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.267471+0000", 2020-08-07T15:09:28.144 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.86778845500000001, 2020-08-07T15:09:28.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.86781305500000006, 2020-08-07T15:09:28.144 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:28.144 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:09:28.144 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:28.144 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:28.145 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:28.145 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73129 2020-08-07T15:09:28.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.145 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:28.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:28.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.267471+0000", 2020-08-07T15:09:28.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:28.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:28.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.267471+0000", 2020-08-07T15:09:28.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8472000000000004e-05 2020-08-07T15:09:28.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:28.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.267549+0000", 2020-08-07T15:09:28.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.217e-05 2020-08-07T15:09:28.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:28.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.267561+0000", 2020-08-07T15:09:28.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3754999999999999e-05 2020-08-07T15:09:28.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:28.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.267585+0000", 2020-08-07T15:09:28.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2286999999999999e-05 2020-08-07T15:09:28.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:28.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.267598+0000", 2020-08-07T15:09:28.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012384399999999999 2020-08-07T15:09:28.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:28.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.267721+0000", 2020-08-07T15:09:28.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5849e-05 2020-08-07T15:09:28.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:28.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.267737+0000", 2020-08-07T15:09:28.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026637699999999999 2020-08-07T15:09:28.190 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.190 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:28.191 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.191 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73130 4.9 4.a3ada059 (undecoded) ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:28.191 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.267875+0000", 2020-08-07T15:09:28.192 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.86738450899999997, 2020-08-07T15:09:28.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.86747244700000004, 2020-08-07T15:09:28.192 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:28.192 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:09:28.218 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:28.218 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:28.218 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:28.230 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73130 2020-08-07T15:09:28.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.230 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:28.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:28.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.267875+0000", 2020-08-07T15:09:28.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:28.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:28.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.267875+0000", 2020-08-07T15:09:28.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6733e-05 2020-08-07T15:09:28.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:28.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.267902+0000", 2020-08-07T15:09:28.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5250000000000001e-06 2020-08-07T15:09:28.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:28.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.267905+0000", 2020-08-07T15:09:28.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.303e-06 2020-08-07T15:09:28.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:28.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.267913+0000", 2020-08-07T15:09:28.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5190000000000001e-06 2020-08-07T15:09:28.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:28.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.267917+0000", 2020-08-07T15:09:28.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0717000000000003e-05 2020-08-07T15:09:28.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:28.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.267958+0000", 2020-08-07T15:09:28.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1912e-05 2020-08-07T15:09:28.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:28.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.267969+0000", 2020-08-07T15:09:28.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4709000000000004e-05 2020-08-07T15:09:28.295 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.295 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:28.295 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.296 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73131 4.9 4.a3ada059 (undecoded) ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:28.296 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.268109+0000", 2020-08-07T15:09:28.296 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.867150172, 2020-08-07T15:09:28.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.86730485999999996, 2020-08-07T15:09:28.297 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:28.297 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:09:28.297 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:28.297 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:28.297 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:28.314 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73131 2020-08-07T15:09:28.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.315 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:28.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:28.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268109+0000", 2020-08-07T15:09:28.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:28.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:28.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268109+0000", 2020-08-07T15:09:28.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.743e-05 2020-08-07T15:09:28.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:28.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268137+0000", 2020-08-07T15:09:28.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6119999999999999e-06 2020-08-07T15:09:28.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:28.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268140+0000", 2020-08-07T15:09:28.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9160000000000004e-06 2020-08-07T15:09:28.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:28.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268147+0000", 2020-08-07T15:09:28.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.177e-06 2020-08-07T15:09:28.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:28.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268151+0000", 2020-08-07T15:09:28.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1678999999999998e-05 2020-08-07T15:09:28.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:28.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268203+0000", 2020-08-07T15:09:28.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3597e-05 2020-08-07T15:09:28.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:28.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268217+0000", 2020-08-07T15:09:28.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010741100000000001 2020-08-07T15:09:28.353 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.353 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:28.353 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.354 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73132 4.9 4.a3ada059 (undecoded) ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:28.366 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.268493+0000", 2020-08-07T15:09:28.366 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.86676582999999996, 2020-08-07T15:09:28.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.866973349, 2020-08-07T15:09:28.392 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:28.392 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:09:28.392 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:28.392 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:28.392 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:28.393 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73132 2020-08-07T15:09:28.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.393 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:28.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:28.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268493+0000", 2020-08-07T15:09:28.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:28.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:28.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268493+0000", 2020-08-07T15:09:28.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8850999999999999e-05 2020-08-07T15:09:28.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:28.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268522+0000", 2020-08-07T15:09:28.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7890000000000001e-06 2020-08-07T15:09:28.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:28.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268526+0000", 2020-08-07T15:09:28.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6899999999999998e-06 2020-08-07T15:09:28.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:28.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268535+0000", 2020-08-07T15:09:28.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8210000000000003e-06 2020-08-07T15:09:28.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:28.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268539+0000", 2020-08-07T15:09:28.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8762999999999999e-05 2020-08-07T15:09:28.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:28.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268567+0000", 2020-08-07T15:09:28.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1409999999999999e-06 2020-08-07T15:09:28.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:28.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268575+0000", 2020-08-07T15:09:28.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1055000000000002e-05 2020-08-07T15:09:28.484 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.484 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:28.484 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.485 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73133 4.9 4.a3ada059 (undecoded) ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:28.485 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.268740+0000", 2020-08-07T15:09:28.507 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.86651931400000004, 2020-08-07T15:09:28.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.86676472599999999, 2020-08-07T15:09:28.508 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:28.508 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:09:28.508 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:28.508 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:28.508 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:28.509 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73133 2020-08-07T15:09:28.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.509 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:28.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:28.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268740+0000", 2020-08-07T15:09:28.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:28.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:28.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268740+0000", 2020-08-07T15:09:28.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7990000000000001e-05 2020-08-07T15:09:28.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:28.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268768+0000", 2020-08-07T15:09:28.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2540000000000001e-06 2020-08-07T15:09:28.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:28.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268771+0000", 2020-08-07T15:09:28.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5530000000000004e-06 2020-08-07T15:09:28.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:28.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268779+0000", 2020-08-07T15:09:28.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8700000000000001e-06 2020-08-07T15:09:28.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:28.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268782+0000", 2020-08-07T15:09:28.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9869000000000001e-05 2020-08-07T15:09:28.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:28.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268822+0000", 2020-08-07T15:09:28.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.004e-05 2020-08-07T15:09:28.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:28.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268832+0000", 2020-08-07T15:09:28.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1576000000000005e-05 2020-08-07T15:09:28.534 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.534 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:28.535 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.535 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73134 4.9 4.a3ada059 (undecoded) ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:28.535 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.268819+0000", 2020-08-07T15:09:28.536 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.86644040700000002, 2020-08-07T15:09:28.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.86672283699999997, 2020-08-07T15:09:28.536 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:28.536 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:09:28.536 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:28.537 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:28.537 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:28.537 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73134 2020-08-07T15:09:28.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.537 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:28.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:28.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268819+0000", 2020-08-07T15:09:28.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:28.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:28.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268819+0000", 2020-08-07T15:09:28.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8215999999999999e-05 2020-08-07T15:09:28.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:28.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268847+0000", 2020-08-07T15:09:28.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0309999999999999e-06 2020-08-07T15:09:28.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:28.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268850+0000", 2020-08-07T15:09:28.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.908e-06 2020-08-07T15:09:28.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:28.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268856+0000", 2020-08-07T15:09:28.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9000000000000002e-06 2020-08-07T15:09:28.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:28.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268859+0000", 2020-08-07T15:09:28.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6261000000000001e-05 2020-08-07T15:09:28.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:28.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268885+0000", 2020-08-07T15:09:28.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3600000000000004e-06 2020-08-07T15:09:28.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:28.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268891+0000", 2020-08-07T15:09:28.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1675999999999996e-05 2020-08-07T15:09:28.564 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.567 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:28.567 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.568 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73135 4.9 4.a3ada059 (undecoded) ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:28.568 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.268948+0000", 2020-08-07T15:09:28.568 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.86631097999999995, 2020-08-07T15:09:28.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.86663031000000001, 2020-08-07T15:09:28.569 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:28.569 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:09:28.569 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:28.570 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:28.570 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:28.570 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73135 2020-08-07T15:09:28.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.597 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:28.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:28.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268948+0000", 2020-08-07T15:09:28.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:28.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:28.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268948+0000", 2020-08-07T15:09:28.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1641000000000003e-05 2020-08-07T15:09:28.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:28.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268990+0000", 2020-08-07T15:09:28.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9349999999999999e-06 2020-08-07T15:09:28.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:28.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268993+0000", 2020-08-07T15:09:28.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6440000000000003e-06 2020-08-07T15:09:28.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:28.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269000+0000", 2020-08-07T15:09:28.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1190000000000001e-06 2020-08-07T15:09:28.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:28.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269003+0000", 2020-08-07T15:09:28.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6940000000000002e-05 2020-08-07T15:09:28.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:28.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269040+0000", 2020-08-07T15:09:28.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0988999999999999e-05 2020-08-07T15:09:28.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:28.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269051+0000", 2020-08-07T15:09:28.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102268 2020-08-07T15:09:28.617 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.626 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:28.626 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.627 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73136 4.9 4.a3ada059 (undecoded) ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:28.627 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.269092+0000", 2020-08-07T15:09:28.627 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.86616719399999997, 2020-08-07T15:09:28.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.86652463099999999, 2020-08-07T15:09:28.628 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:28.628 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:09:28.628 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:28.628 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:28.628 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:28.629 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73136 2020-08-07T15:09:28.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.629 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:28.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:28.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269092+0000", 2020-08-07T15:09:28.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:28.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:28.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269092+0000", 2020-08-07T15:09:28.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9740000000000002e-05 2020-08-07T15:09:28.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:28.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269132+0000", 2020-08-07T15:09:28.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.117e-06 2020-08-07T15:09:28.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:28.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269135+0000", 2020-08-07T15:09:28.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0129999999999999e-06 2020-08-07T15:09:28.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:28.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269141+0000", 2020-08-07T15:09:28.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1769999999999998e-06 2020-08-07T15:09:28.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:28.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269144+0000", 2020-08-07T15:09:28.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3272999999999999e-05 2020-08-07T15:09:28.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:28.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269167+0000", 2020-08-07T15:09:28.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8849999999999998e-06 2020-08-07T15:09:28.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:28.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269172+0000", 2020-08-07T15:09:28.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0204999999999995e-05 2020-08-07T15:09:28.659 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.660 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:28.660 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.682 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73081 4.9 4.eae1aa89 (undecoded) ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:28.683 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:09:28.683 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.90869356099999998, 2020-08-07T15:09:28.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.90908759400000005, 2020-08-07T15:09:28.683 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:28.684 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:09:28.692 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:28.692 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:28.693 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:28.693 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73081 2020-08-07T15:09:28.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.694 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:28.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:28.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:09:28.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:28.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:28.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:09:28.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3016000000000003e-05 2020-08-07T15:09:28.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:28.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226609+0000", 2020-08-07T15:09:28.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4479999999999996e-06 2020-08-07T15:09:28.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:28.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226613+0000", 2020-08-07T15:09:28.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0189999999999995e-06 2020-08-07T15:09:28.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:28.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226622+0000", 2020-08-07T15:09:28.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5980000000000001e-06 2020-08-07T15:09:28.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:28.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226626+0000", 2020-08-07T15:09:28.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8088999999999999e-05 2020-08-07T15:09:28.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:28.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226684+0000", 2020-08-07T15:09:28.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2162e-05 2020-08-07T15:09:28.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:28.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226696+0000", 2020-08-07T15:09:28.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013033200000000001 2020-08-07T15:09:28.727 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.727 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:28.727 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.728 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73082 4.9 4.eae1aa89 (undecoded) ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:28.728 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:09:28.728 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.90832309700000002, 2020-08-07T15:09:28.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.90875378900000003, 2020-08-07T15:09:28.729 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:28.729 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:09:28.729 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:28.730 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:28.730 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:28.731 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73082 2020-08-07T15:09:28.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.731 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:28.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:28.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:09:28.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:28.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:28.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:09:28.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6928000000000003e-05 2020-08-07T15:09:28.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:28.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226973+0000", 2020-08-07T15:09:28.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3630000000000004e-06 2020-08-07T15:09:28.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:28.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226977+0000", 2020-08-07T15:09:28.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9659999999999996e-06 2020-08-07T15:09:28.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:28.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226985+0000", 2020-08-07T15:09:28.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5800000000000002e-06 2020-08-07T15:09:28.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:28.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226990+0000", 2020-08-07T15:09:28.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9101999999999998e-05 2020-08-07T15:09:28.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:28.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227039+0000", 2020-08-07T15:09:28.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9620000000000005e-06 2020-08-07T15:09:28.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:28.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227049+0000", 2020-08-07T15:09:28.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112901 2020-08-07T15:09:28.761 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.762 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:28.762 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.762 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73083 4.9 4.eae1aa89 (undecoded) ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:28.763 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:09:28.763 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.90822883799999998, 2020-08-07T15:09:28.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.908700709, 2020-08-07T15:09:28.763 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:28.763 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:09:28.764 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:28.764 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:28.764 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:28.764 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73083 2020-08-07T15:09:28.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.764 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:28.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:28.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:09:28.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:28.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:28.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:09:28.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1627000000000001e-05 2020-08-07T15:09:28.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:28.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227062+0000", 2020-08-07T15:09:28.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.253e-06 2020-08-07T15:09:28.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:28.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227065+0000", 2020-08-07T15:09:28.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3168e-05 2020-08-07T15:09:28.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:28.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227079+0000", 2020-08-07T15:09:28.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1549999999999999e-06 2020-08-07T15:09:28.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:28.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227082+0000", 2020-08-07T15:09:28.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5534999999999997e-05 2020-08-07T15:09:28.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:28.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227127+0000", 2020-08-07T15:09:28.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6823000000000001e-05 2020-08-07T15:09:28.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:28.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227144+0000", 2020-08-07T15:09:28.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011356100000000001 2020-08-07T15:09:28.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.773 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:28.774 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.775 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73084 4.9 4.eae1aa89 (undecoded) ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:28.775 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:09:28.775 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.90813989900000003, 2020-08-07T15:09:28.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.90864861799999996, 2020-08-07T15:09:28.776 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:28.776 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:09:28.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:28.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:28.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:28.777 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73084 2020-08-07T15:09:28.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.777 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:28.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:28.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:09:28.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:28.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:28.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:09:28.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5565999999999997e-05 2020-08-07T15:09:28.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:28.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227165+0000", 2020-08-07T15:09:28.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2020-08-07T15:09:28.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:28.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227169+0000", 2020-08-07T15:09:28.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6850000000000001e-06 2020-08-07T15:09:28.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:28.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227176+0000", 2020-08-07T15:09:28.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1949999999999999e-06 2020-08-07T15:09:28.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:28.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227179+0000", 2020-08-07T15:09:28.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0318000000000003e-05 2020-08-07T15:09:28.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:28.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227220+0000", 2020-08-07T15:09:28.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0349e-05 2020-08-07T15:09:28.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:28.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227230+0000", 2020-08-07T15:09:28.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110708 2020-08-07T15:09:28.796 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.797 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:28.797 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73085 4.9 4.eae1aa89 (undecoded) ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:28.804 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:09:28.805 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.90800408399999999, 2020-08-07T15:09:28.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.90854918699999998, 2020-08-07T15:09:28.805 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:28.805 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:09:28.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:28.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:28.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:28.806 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73085 2020-08-07T15:09:28.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.807 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:28.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:28.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:09:28.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:28.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:28.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:09:28.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6570999999999997e-05 2020-08-07T15:09:28.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:28.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227322+0000", 2020-08-07T15:09:28.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5509999999999996e-06 2020-08-07T15:09:28.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:28.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227328+0000", 2020-08-07T15:09:28.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2498e-05 2020-08-07T15:09:28.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:28.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227341+0000", 2020-08-07T15:09:28.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4389999999999999e-06 2020-08-07T15:09:28.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:28.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227346+0000", 2020-08-07T15:09:28.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3711999999999998e-05 2020-08-07T15:09:28.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:28.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227390+0000", 2020-08-07T15:09:28.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1868999999999999e-05 2020-08-07T15:09:28.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:28.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227402+0000", 2020-08-07T15:09:28.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014663999999999999 2020-08-07T15:09:28.843 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.843 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:28.843 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.844 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73086 4.9 4.eae1aa89 (undecoded) ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:28.844 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:09:28.844 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.90787158800000001, 2020-08-07T15:09:28.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.90845330899999999, 2020-08-07T15:09:28.845 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:28.845 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:09:28.845 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:28.845 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:28.847 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:28.848 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73086 2020-08-07T15:09:28.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.848 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:28.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:28.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:09:28.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:28.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:28.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:09:28.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000238426 2020-08-07T15:09:28.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:28.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227626+0000", 2020-08-07T15:09:28.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9609999999999999e-06 2020-08-07T15:09:28.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:28.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227633+0000", 2020-08-07T15:09:28.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2727e-05 2020-08-07T15:09:28.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:28.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227646+0000", 2020-08-07T15:09:28.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1739999999999997e-06 2020-08-07T15:09:28.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:28.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227652+0000", 2020-08-07T15:09:28.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5004000000000001e-05 2020-08-07T15:09:28.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:28.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227737+0000", 2020-08-07T15:09:28.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2744e-05 2020-08-07T15:09:28.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:28.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227750+0000", 2020-08-07T15:09:28.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00036203599999999999 2020-08-07T15:09:28.859 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.859 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:28.859 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.860 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73087 4.9 4.eae1aa89 (undecoded) ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:28.860 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:09:28.860 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.90693043500000003, 2020-08-07T15:09:28.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.90756597400000005, 2020-08-07T15:09:28.861 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:28.861 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:09:28.861 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:28.862 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:28.862 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:28.862 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73087 2020-08-07T15:09:28.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.862 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:28.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:28.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:09:28.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:28.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:28.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:09:28.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7354e-05 2020-08-07T15:09:28.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:28.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228386+0000", 2020-08-07T15:09:28.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8000000000000001e-06 2020-08-07T15:09:28.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:28.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228393+0000", 2020-08-07T15:09:28.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3784999999999999e-05 2020-08-07T15:09:28.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:28.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228407+0000", 2020-08-07T15:09:28.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8090000000000004e-06 2020-08-07T15:09:28.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:28.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228414+0000", 2020-08-07T15:09:28.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6907000000000003e-05 2020-08-07T15:09:28.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:28.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228461+0000", 2020-08-07T15:09:28.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8929999999999994e-06 2020-08-07T15:09:28.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:28.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228469+0000", 2020-08-07T15:09:28.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000140548 2020-08-07T15:09:28.871 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.871 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:28.872 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.873 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73088 4.9 4.eae1aa89 (undecoded) ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:28.873 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:09:28.873 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.90653974500000001, 2020-08-07T15:09:28.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.90721376200000003, 2020-08-07T15:09:28.874 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:28.874 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:09:28.875 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:28.875 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:28.875 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:28.875 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73088 2020-08-07T15:09:28.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.876 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:28.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:28.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:09:28.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:28.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:28.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:09:28.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7741000000000003e-05 2020-08-07T15:09:28.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:28.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228767+0000", 2020-08-07T15:09:28.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6409999999999999e-06 2020-08-07T15:09:28.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:28.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228773+0000", 2020-08-07T15:09:28.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0874e-05 2020-08-07T15:09:28.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:28.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228784+0000", 2020-08-07T15:09:28.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1529999999999996e-06 2020-08-07T15:09:28.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:28.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228789+0000", 2020-08-07T15:09:28.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3945e-05 2020-08-07T15:09:28.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:28.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228833+0000", 2020-08-07T15:09:28.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4020000000000002e-06 2020-08-07T15:09:28.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:28.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:28.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:28.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228841+0000", 2020-08-07T15:09:28.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121756 2020-08-07T15:09:28.885 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.885 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:28.886 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.886 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:28.886 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:09:28.886 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 16 2020-08-07T15:09:28.887 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:09:28.890 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:09:28.890 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:09:28.891 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:09:28.891 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:09:28.891 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:09:28.891 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:09:28.891 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:09:28.892 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:09:28.892 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 62 16 73223 73207 9.22379 0.679688 0.00918765 0.0135485 2020-08-07T15:09:28.894 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:09:28.894 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:09:28.894 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:09:28.894 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:09:28.895 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:09:28.895 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:09:28.896 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:09:28.896 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:09:28.897 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:09:28.897 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:28.887+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:28.990 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:28.989+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:29.094 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:29.093+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:29.119 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:29.119 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:29.131 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:09:29.193 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.4 to 1 (10000) 2020-08-07T15:09:29.204 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 5 1 2020-08-07T15:09:29.222 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:29.219+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:29.284 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 63 16 74806 74790 9.27367 12.3672 0.00594507 0.0134748 2020-08-07T15:09:29.307 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:29.295+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:29.336 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:29.337 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:29.350 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:09:29.415 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:29.398+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:29.522 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:29.505+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:29.593 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:29.593 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:29.604 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:09:29.614 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:29.612+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:29.718 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:29.713+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:29.801 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:29.802 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:29.815 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:09:29.816 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:29.814+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:29.960 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:29.953+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:30.011 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:09:30.029 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:30.029 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:30.035 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:09:30.071 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:30.069+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:30.157 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:30.153+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:30.196 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.5 to 1 (10000) 2020-08-07T15:09:30.212 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 6 1 2020-08-07T15:09:30.259 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:30.258+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:30.271 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:30.272 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:30.284 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:09:30.284 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 64 16 76192 76176 9.29795 10.8281 0.00551522 0.0134385 2020-08-07T15:09:30.374 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:30.371+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:30.467 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:30.462+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:30.489 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:30.489 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:30.503 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:09:30.586 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:30.579+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:30.685 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:30.681+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:30.758 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:30.759 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:30.801 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:30.799+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:30.802 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:09:30.898 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:09:30.902 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:30.900+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:30.987 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:30.987 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:31.023 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:09:31.034 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:31.021+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:31.114 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:09:31.148 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:31.147+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:31.162 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:09:31.192 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:09:31.234 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:09:31.235 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:09:31.237 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:09:31.237 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:09:31.238 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:09:31.240 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:09:31.268 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:09:31.283 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:09:31.284 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 65 16 77759 77743 9.34323 12.2422 0.00462195 0.013372 2020-08-07T15:09:31.285 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:31.268+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:31.310 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:09:31.345 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:09:31.371 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:09:31.372 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:31.370+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:31.375 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:09:31.375 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:09:31.376 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:09:31.376 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:09:31.376 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:09:31.377 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:09:31.377 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:09:31.377 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:09:31.439 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:09:31.439 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:09:31.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:09:31.439 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:09:31.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.440 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:10240 3.6 3:665d71ff:::benchmark_data_smithi074_35447_object1279:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:09:31.440 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:09:31.440 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 169.04128055499999, 2020-08-07T15:09:31.440 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25191569699999999, 2020-08-07T15:09:31.441 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:31.441 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:31.441 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:31.441 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:09:31.441 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:31.443 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 10240 2020-08-07T15:09:31.443 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.443 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:31.443 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.444 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:31.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:09:31.450 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:31.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.451 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.451 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:31.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:09:31.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5463000000000002e-05 2020-08-07T15:09:31.451 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:31.465 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393264+0000", 2020-08-07T15:09:31.478 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9219999999999999e-06 2020-08-07T15:09:31.478 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.478 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.486 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:31.487 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393268+0000", 2020-08-07T15:09:31.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6149999999999997e-06 2020-08-07T15:09:31.488 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:31.488 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393277+0000", 2020-08-07T15:09:31.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2039999999999996e-06 2020-08-07T15:09:31.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.489 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.489 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:31.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393282+0000", 2020-08-07T15:09:31.490 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001225243 2020-08-07T15:09:31.490 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.490 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.490 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:31.490 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394507+0000", 2020-08-07T15:09:31.491 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7108999999999995e-05 2020-08-07T15:09:31.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.491 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.491 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:31.491 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394605+0000", 2020-08-07T15:09:31.492 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017490800000000001 2020-08-07T15:09:31.492 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.493 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:09:31.493 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394779+0000", 2020-08-07T15:09:31.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032608699999999999 2020-08-07T15:09:31.494 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.494 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:31.494 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.398040+0000", 2020-08-07T15:09:31.494 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013015259999999999 2020-08-07T15:09:31.495 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.503 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.503 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:31.503 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.399342+0000", 2020-08-07T15:09:31.503 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24566893400000001 2020-08-07T15:09:31.504 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.504 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.504 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:31.504 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645011+0000", 2020-08-07T15:09:31.504 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.217e-06 2020-08-07T15:09:31.505 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.505 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.505 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:31.505 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645012+0000", 2020-08-07T15:09:31.505 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0803000000000006e-05 2020-08-07T15:09:31.508 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.508 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.508 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:31.508 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645083+0000", 2020-08-07T15:09:31.508 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1882999999999998e-05 2020-08-07T15:09:31.509 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.509 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.509 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:31.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645135+0000", 2020-08-07T15:09:31.510 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25191569699999999 2020-08-07T15:09:31.510 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:31.510 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:31.510 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:31.510 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.534 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:54200 3.3 e37/20 3:cc40637e:::benchmark_data_smithi074_35447_object6774:head v 37'3248, mlcod=37'3236)", 2020-08-07T15:09:31.536 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:09:31.536 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 138.33593733500001, 2020-08-07T15:09:31.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24065872499999999, 2020-08-07T15:09:31.536 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:31.537 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:31.538 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:31.538 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.538 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:31.538 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:09:31.538 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:31.539 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.539 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.539 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:31.539 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098562+0000", 2020-08-07T15:09:31.539 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1671000000000002e-05 2020-08-07T15:09:31.540 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.540 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.540 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:31.540 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098594+0000", 2020-08-07T15:09:31.541 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0890000000000002e-06 2020-08-07T15:09:31.542 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.542 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.542 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:31.542 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.098598+0000", 2020-08-07T15:09:31.542 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23875592300000001 2020-08-07T15:09:31.544 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.545 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:31.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337354+0000", 2020-08-07T15:09:31.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0420000000000002e-06 2020-08-07T15:09:31.545 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.546 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.546 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:31.546 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337359+0000", 2020-08-07T15:09:31.546 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0586000000000001e-05 2020-08-07T15:09:31.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.547 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:31.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337399+0000", 2020-08-07T15:09:31.547 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9415999999999998e-05 2020-08-07T15:09:31.547 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.548 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:31.548 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.337419+0000", 2020-08-07T15:09:31.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017568390000000001 2020-08-07T15:09:31.548 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.549 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:31.562 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.339176+0000", 2020-08-07T15:09:31.562 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5158999999999999e-05 2020-08-07T15:09:31.563 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.563 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.563 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:31.563 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:13.339221+0000", 2020-08-07T15:09:31.563 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24065872499999999 2020-08-07T15:09:31.564 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:31.564 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:31.581 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:31.581 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.581 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.581 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:71136 3.6 e43/26 3:64cb7acb:::benchmark_data_smithi074_35447_object8891:head v 43'4744, mlcod=43'4739)", 2020-08-07T15:09:31.582 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:09:31.582 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 127.115885969, 2020-08-07T15:09:31.582 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26135482999999998, 2020-08-07T15:09:31.582 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:31.583 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:31.583 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:31.583 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.583 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:31.583 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:09:31.584 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:31.584 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.584 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.584 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:31.584 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:09:31.585 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3702999999999997e-05 2020-08-07T15:09:31.585 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.585 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.594 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:31.594 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318667+0000", 2020-08-07T15:09:31.594 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3119999999999999e-06 2020-08-07T15:09:31.595 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.595 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.595 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:31.595 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318674+0000", 2020-08-07T15:09:31.595 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.255173292 2020-08-07T15:09:31.596 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.625 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.626 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:31.626 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.573847+0000", 2020-08-07T15:09:31.626 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1140000000000002e-06 2020-08-07T15:09:31.626 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.627 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:31.627 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.573853+0000", 2020-08-07T15:09:31.627 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00057603799999999996 2020-08-07T15:09:31.627 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.628 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.628 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:31.628 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.574429+0000", 2020-08-07T15:09:31.628 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4776e-05 2020-08-07T15:09:31.629 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.629 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.629 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:31.629 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.574444+0000", 2020-08-07T15:09:31.629 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054577080000000004 2020-08-07T15:09:31.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.644 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.644 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:31.644 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.579901+0000", 2020-08-07T15:09:31.645 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6886999999999998e-05 2020-08-07T15:09:31.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:31.645 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.579968+0000", 2020-08-07T15:09:31.646 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26135482999999998 2020-08-07T15:09:31.646 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:31.646 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:31.646 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:31.647 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.647 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.647 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:85696 3.f e51/20 3:f99fcea4:::benchmark_data_smithi074_35447_object10711:head v 51'5688, mlcod=51'5679)", 2020-08-07T15:09:31.647 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:09:31.647 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 117.781894556, 2020-08-07T15:09:31.674 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49655717300000002, 2020-08-07T15:09:31.674 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:31.674 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:31.675 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:31.675 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.675 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:31.675 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:09:31.675 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:31.676 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.676 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.676 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:31.676 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:09:31.676 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9543000000000002e-05 2020-08-07T15:09:31.677 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:31.677 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652665+0000", 2020-08-07T15:09:31.726 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8700000000000003e-06 2020-08-07T15:09:31.726 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.727 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.729 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:31.729 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652671+0000", 2020-08-07T15:09:31.730 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49579595799999998 2020-08-07T15:09:31.730 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.730 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.731 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:31.731 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148467+0000", 2020-08-07T15:09:31.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2509999999999997e-06 2020-08-07T15:09:31.732 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.732 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:31.733 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148475+0000", 2020-08-07T15:09:31.733 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5603999999999998e-05 2020-08-07T15:09:31.733 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.734 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:31.734 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148530+0000", 2020-08-07T15:09:31.734 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5486999999999998e-05 2020-08-07T15:09:31.734 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.735 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.735 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:31.735 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148556+0000", 2020-08-07T15:09:31.736 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00055666600000000004 2020-08-07T15:09:31.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.737 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.737 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:31.737 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.149112+0000", 2020-08-07T15:09:31.737 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9793999999999998e-05 2020-08-07T15:09:31.738 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.738 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.738 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:31.738 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.149162+0000", 2020-08-07T15:09:31.739 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49655717300000002 2020-08-07T15:09:31.739 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:31.740 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:31.740 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:31.740 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.740 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.741 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:86792 3.f e52/20 3:fc5386ee:::benchmark_data_smithi074_35447_object10848:head v 52'5752, mlcod=52'5743)", 2020-08-07T15:09:31.741 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:09:31.741 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 117.032048781, 2020-08-07T15:09:31.742 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.295270005, 2020-08-07T15:09:31.742 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:31.742 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:31.742 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:31.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.743 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:31.743 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:09:31.743 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:31.743 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:31.744 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:09:31.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9712000000000001e-05 2020-08-07T15:09:31.745 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.746 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:31.746 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402500+0000", 2020-08-07T15:09:31.746 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6789999999999999e-06 2020-08-07T15:09:31.747 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.747 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.747 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:31.748 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402507+0000", 2020-08-07T15:09:31.748 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29317074999999998 2020-08-07T15:09:31.748 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.749 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.749 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:31.750 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695678+0000", 2020-08-07T15:09:31.751 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8120000000000002e-06 2020-08-07T15:09:31.752 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.752 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:31.752 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695685+0000", 2020-08-07T15:09:31.753 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1270999999999998e-05 2020-08-07T15:09:31.753 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.753 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.753 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:31.754 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695726+0000", 2020-08-07T15:09:31.754 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2739e-05 2020-08-07T15:09:31.754 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.755 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:31.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695749+0000", 2020-08-07T15:09:31.755 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019333899999999999 2020-08-07T15:09:31.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:31.757 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.697682+0000", 2020-08-07T15:09:31.757 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8652000000000002e-05 2020-08-07T15:09:31.757 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:31.779 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.697721+0000", 2020-08-07T15:09:31.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.295270005 2020-08-07T15:09:31.779 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:31.779 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:31.780 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:31.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.789 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.789 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:87088 3.6 e52/26 3:6b0320df:::benchmark_data_smithi074_35447_object10885:head v 52'5912, mlcod=52'5902)", 2020-08-07T15:09:31.789 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:09:31.790 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 116.46781554099999, 2020-08-07T15:09:31.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29522189799999998, 2020-08-07T15:09:31.790 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:31.791 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:31.791 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:31.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.792 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:31.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:09:31.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:31.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.793 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.793 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:31.793 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:09:31.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0751000000000002e-05 2020-08-07T15:09:31.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.818 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:31.819 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966735+0000", 2020-08-07T15:09:31.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3080000000000004e-06 2020-08-07T15:09:31.819 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:31.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966741+0000", 2020-08-07T15:09:31.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29365034699999998 2020-08-07T15:09:31.821 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:31.833 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260391+0000", 2020-08-07T15:09:31.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2720000000000003e-06 2020-08-07T15:09:31.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:31.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260398+0000", 2020-08-07T15:09:31.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8819000000000005e-05 2020-08-07T15:09:31.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.842 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:31.842 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260476+0000", 2020-08-07T15:09:31.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4958999999999999e-05 2020-08-07T15:09:31.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.843 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:31.844 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260501+0000", 2020-08-07T15:09:31.844 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001353641 2020-08-07T15:09:31.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:31.845 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.261855+0000", 2020-08-07T15:09:31.845 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0800999999999997e-05 2020-08-07T15:09:31.847 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.847 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.848 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:31.848 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.261906+0000", 2020-08-07T15:09:31.848 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29522189799999998 2020-08-07T15:09:31.848 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:31.849 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:31.849 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:31.850 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.851 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:93920 3.6 e56/26 3:631393d3:::benchmark_data_smithi074_35447_object11739:head v 56'6352, mlcod=55'6342)", 2020-08-07T15:09:31.851 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:09:31.851 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 112.10770011, 2020-08-07T15:09:31.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39765404300000001, 2020-08-07T15:09:31.853 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:31.853 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:31.853 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:31.854 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.854 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:31.854 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:09:31.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:31.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.855 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:31.855 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:09:31.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3035999999999998e-05 2020-08-07T15:09:31.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.856 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:31.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326852+0000", 2020-08-07T15:09:31.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5379999999999999e-06 2020-08-07T15:09:31.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.858 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:31.858 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326859+0000", 2020-08-07T15:09:31.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.393917568 2020-08-07T15:09:31.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.859 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.859 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:31.859 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.720777+0000", 2020-08-07T15:09:31.860 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2430000000000005e-06 2020-08-07T15:09:31.860 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.860 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:31.861 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.720785+0000", 2020-08-07T15:09:31.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00065658700000000003 2020-08-07T15:09:31.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.862 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:31.862 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.721441+0000", 2020-08-07T15:09:31.862 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8585000000000002e-05 2020-08-07T15:09:31.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:31.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.721470+0000", 2020-08-07T15:09:31.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00293349 2020-08-07T15:09:31.864 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.865 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:31.865 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.724403+0000", 2020-08-07T15:09:31.865 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9996e-05 2020-08-07T15:09:31.866 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.866 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.866 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:31.866 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.724453+0000", 2020-08-07T15:09:31.867 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39765404300000001 2020-08-07T15:09:31.867 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:31.867 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:31.868 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:31.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.875 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:94384 3.3 e56/20 3:ce533213:::benchmark_data_smithi074_35447_object11797:head v 56'5488, mlcod=56'5479)", 2020-08-07T15:09:31.876 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.637245+0000", 2020-08-07T15:09:31.876 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 111.797254606, 2020-08-07T15:09:31.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.247824459, 2020-08-07T15:09:31.876 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:31.883 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:31.883 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:31.883 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.883 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:31.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637245+0000", 2020-08-07T15:09:31.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:31.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:31.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637245+0000", 2020-08-07T15:09:31.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0782999999999997e-05 2020-08-07T15:09:31.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.886 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:31.887 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637306+0000", 2020-08-07T15:09:31.887 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8699999999999992e-06 2020-08-07T15:09:31.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:31.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.637314+0000", 2020-08-07T15:09:31.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24469571800000001 2020-08-07T15:09:31.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:31.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882009+0000", 2020-08-07T15:09:31.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8510000000000004e-06 2020-08-07T15:09:31.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.890 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:31.890 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882015+0000", 2020-08-07T15:09:31.891 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6708999999999997e-05 2020-08-07T15:09:31.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.891 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.892 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:31.892 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882052+0000", 2020-08-07T15:09:31.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0280000000000001e-05 2020-08-07T15:09:31.892 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.893 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:31.893 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.882082+0000", 2020-08-07T15:09:31.894 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002934767 2020-08-07T15:09:31.894 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.904 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:31.904 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.885017+0000", 2020-08-07T15:09:31.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2481000000000001e-05 2020-08-07T15:09:31.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:31.906 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.885069+0000", 2020-08-07T15:09:31.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.247824459 2020-08-07T15:09:31.906 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:31.907 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:31.907 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:31.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.909 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:104560 3.f e58/20 3:fb4383ae:::benchmark_data_smithi074_35447_object13069:head v 58'6888, mlcod=58'6878)", 2020-08-07T15:09:31.909 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:46.301207+0000", 2020-08-07T15:09:31.909 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 105.133292789, 2020-08-07T15:09:31.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23687350400000001, 2020-08-07T15:09:31.910 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:31.911 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:31.911 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:31.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:31.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301207+0000", 2020-08-07T15:09:31.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:31.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:31.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301207+0000", 2020-08-07T15:09:31.914 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7945000000000003e-05 2020-08-07T15:09:31.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:31.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301255+0000", 2020-08-07T15:09:31.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2030000000000002e-06 2020-08-07T15:09:31.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.922 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:31.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.301259+0000", 2020-08-07T15:09:31.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23565420500000001 2020-08-07T15:09:31.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:31.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.536913+0000", 2020-08-07T15:09:31.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2290000000000003e-06 2020-08-07T15:09:31.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:31.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.536921+0000", 2020-08-07T15:09:31.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0955e-05 2020-08-07T15:09:31.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.940 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.940 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:31.940 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.537002+0000", 2020-08-07T15:09:31.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8977999999999999e-05 2020-08-07T15:09:31.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:31.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.537031+0000", 2020-08-07T15:09:31.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00099831000000000008 2020-08-07T15:09:31.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.942 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:31.942 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.538030+0000", 2020-08-07T15:09:31.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0679e-05 2020-08-07T15:09:31.942 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:31.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:46.538080+0000", 2020-08-07T15:09:31.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23687350400000001 2020-08-07T15:09:31.944 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:31.944 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:31.944 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:31.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.945 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:116064 3.6 e64/26 3:6cc03ffa:::benchmark_data_smithi074_35447_object14507:head v 64'7632, mlcod=64'7623)", 2020-08-07T15:09:31.945 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:09:31.945 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 95.071738706999994, 2020-08-07T15:09:31.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25728607799999997, 2020-08-07T15:09:31.946 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:31.946 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:31.946 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:31.946 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.947 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:31.947 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:09:31.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:31.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.948 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:31.949 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:09:31.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6042000000000003e-05 2020-08-07T15:09:31.949 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.950 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.950 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:31.950 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362817+0000", 2020-08-07T15:09:31.950 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9399999999999999e-06 2020-08-07T15:09:31.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.952 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:31.952 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362823+0000", 2020-08-07T15:09:31.952 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.254635583 2020-08-07T15:09:31.953 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.953 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.953 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:31.954 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617458+0000", 2020-08-07T15:09:31.954 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5100000000000004e-06 2020-08-07T15:09:31.954 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.954 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.954 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:31.955 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617465+0000", 2020-08-07T15:09:31.955 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4439999999999999e-05 2020-08-07T15:09:31.955 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.955 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.956 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:31.956 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617539+0000", 2020-08-07T15:09:31.956 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3527e-05 2020-08-07T15:09:31.956 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.957 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:31.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617563+0000", 2020-08-07T15:09:31.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024301069999999999 2020-08-07T15:09:31.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:31.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.619993+0000", 2020-08-07T15:09:31.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3928999999999998e-05 2020-08-07T15:09:31.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:31.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.620047+0000", 2020-08-07T15:09:31.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25728607799999997 2020-08-07T15:09:31.960 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:31.960 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:31.960 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:31.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.961 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:145801 3.6 e75/26 3:69e4cb22:::benchmark_data_smithi074_35447_object16192:head v 75'9405, mlcod=75'9401)", 2020-08-07T15:09:31.961 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:09:31.961 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 76.762743415000003, 2020-08-07T15:09:31.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639, 2020-08-07T15:09:31.963 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:31.964 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:31.964 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:31.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:31.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:09:31.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:31.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:31.966 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:09:31.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6511000000000002e-05 2020-08-07T15:09:31.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:31.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671803+0000", 2020-08-07T15:09:31.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7190000000000004e-06 2020-08-07T15:09:31.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:31.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671809+0000", 2020-08-07T15:09:31.969 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74210928300000001 2020-08-07T15:09:31.969 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.969 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.970 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:31.970 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413919+0000", 2020-08-07T15:09:31.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1759999999999999e-05 2020-08-07T15:09:31.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:31.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413930+0000", 2020-08-07T15:09:31.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000108601 2020-08-07T15:09:31.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:31.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414039+0000", 2020-08-07T15:09:31.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9948999999999999e-05 2020-08-07T15:09:31.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:31.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414089+0000", 2020-08-07T15:09:31.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00079656499999999995 2020-08-07T15:09:31.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:31.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414885+0000", 2020-08-07T15:09:31.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2250999999999998e-05 2020-08-07T15:09:31.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:31.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414938+0000", 2020-08-07T15:09:31.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639 2020-08-07T15:09:31.978 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:31.978 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:31.978 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:31.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.979 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.979 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:9112 4.6 e86/81 4:6004be0b:::benchmark_data_smithi074_44586_object1138:head v 86'544, mlcod=86'533)", 2020-08-07T15:09:31.979 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:32.296890+0000", 2020-08-07T15:09:31.980 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 59.137610006000003, 2020-08-07T15:09:31.980 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26263388100000001, 2020-08-07T15:09:31.980 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:31.981 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:31.981 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:31.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.981 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:31.981 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296890+0000", 2020-08-07T15:09:31.982 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:31.982 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:31.986 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296890+0000", 2020-08-07T15:09:31.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9907999999999997e-05 2020-08-07T15:09:31.987 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.987 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:31.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296939+0000", 2020-08-07T15:09:31.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7630000000000003e-06 2020-08-07T15:09:31.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:31.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296946+0000", 2020-08-07T15:09:31.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26058471900000002 2020-08-07T15:09:31.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.990 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.990 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:31.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557531+0000", 2020-08-07T15:09:31.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5270000000000004e-06 2020-08-07T15:09:31.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:31.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557537+0000", 2020-08-07T15:09:31.991 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2843000000000003e-05 2020-08-07T15:09:31.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:31.993 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557580+0000", 2020-08-07T15:09:31.993 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3833999999999999e-05 2020-08-07T15:09:31.993 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.994 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.994 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:31.994 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557604+0000", 2020-08-07T15:09:31.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001875196 2020-08-07T15:09:31.995 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.995 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.995 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:31.995 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.559479+0000", 2020-08-07T15:09:31.995 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4091000000000002e-05 2020-08-07T15:09:31.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:31.996 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:31.996 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:31.999 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.559523+0000", 2020-08-07T15:09:31.999 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26263388100000001 2020-08-07T15:09:31.999 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:31.999 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:32.000 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:32.000 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.001 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.002 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:13784 4.6 e88/81 4:66f64b92:::benchmark_data_smithi074_44586_object1722:head v 88'832, mlcod=88'827)", 2020-08-07T15:09:32.002 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:35.137315+0000", 2020-08-07T15:09:32.002 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 56.297184272999999, 2020-08-07T15:09:32.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32459016899999998, 2020-08-07T15:09:32.003 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:32.003 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:32.003 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:32.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:32.014 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137315+0000", 2020-08-07T15:09:32.014 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:32.018 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.018 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.018 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:32.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137315+0000", 2020-08-07T15:09:32.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0362999999999998e-05 2020-08-07T15:09:32.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.023 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:32.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137376+0000", 2020-08-07T15:09:32.030 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7020000000000004e-06 2020-08-07T15:09:32.030 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:32.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137382+0000", 2020-08-07T15:09:32.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32172605500000001 2020-08-07T15:09:32.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:32.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459108+0000", 2020-08-07T15:09:32.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7710000000000003e-06 2020-08-07T15:09:32.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:32.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459114+0000", 2020-08-07T15:09:32.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4297e-05 2020-08-07T15:09:32.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:32.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459158+0000", 2020-08-07T15:09:32.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5058999999999998e-05 2020-08-07T15:09:32.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.037 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:32.045 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459183+0000", 2020-08-07T15:09:32.046 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026624740000000002 2020-08-07T15:09:32.050 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.051 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:32.051 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.461846+0000", 2020-08-07T15:09:32.052 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9447999999999997e-05 2020-08-07T15:09:32.059 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.059 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:32.060 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.461905+0000", 2020-08-07T15:09:32.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32459016899999998 2020-08-07T15:09:32.060 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:32.062 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:32.062 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:32.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.063 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:31608 4.7 e100/81 4:e0ebdc25:::benchmark_data_smithi074_44586_object3950:head v 100'2136, mlcod=100'2126)", 2020-08-07T15:09:32.064 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:47.845848+0000", 2020-08-07T15:09:32.064 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 43.588651413000001, 2020-08-07T15:09:32.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27603951399999999, 2020-08-07T15:09:32.064 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:32.065 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:32.065 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:32.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.065 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:32.086 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:47.845848+0000", 2020-08-07T15:09:32.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:32.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:32.088 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:47.845848+0000", 2020-08-07T15:09:32.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3656999999999998e-05 2020-08-07T15:09:32.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.089 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:32.089 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:47.845902+0000", 2020-08-07T15:09:32.089 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4050000000000003e-06 2020-08-07T15:09:32.108 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.109 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.109 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:32.109 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:47.845909+0000", 2020-08-07T15:09:32.110 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27431144000000002 2020-08-07T15:09:32.110 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.110 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.111 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:32.111 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.120221+0000", 2020-08-07T15:09:32.111 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3640000000000004e-06 2020-08-07T15:09:32.111 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.112 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:32.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.120227+0000", 2020-08-07T15:09:32.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1541e-05 2020-08-07T15:09:32.113 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.113 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:32.113 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.120279+0000", 2020-08-07T15:09:32.128 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.372e-05 2020-08-07T15:09:32.129 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.129 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.131 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:32.131 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.120302+0000", 2020-08-07T15:09:32.131 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015294460000000001 2020-08-07T15:09:32.132 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.132 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.132 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:32.132 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.121832+0000", 2020-08-07T15:09:32.132 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5940999999999999e-05 2020-08-07T15:09:32.133 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.133 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.133 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:32.134 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.121888+0000", 2020-08-07T15:09:32.134 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27603951399999999 2020-08-07T15:09:32.134 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:32.134 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:32.135 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:32.135 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.135 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.135 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:31832 4.b e100/81 4:d005fdc5:::benchmark_data_smithi074_44586_object3978:head v 100'1872, mlcod=100'1863)", 2020-08-07T15:09:32.136 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:48.177094+0000", 2020-08-07T15:09:32.136 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 43.257405149, 2020-08-07T15:09:32.136 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26337036000000003, 2020-08-07T15:09:32.136 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:32.137 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:32.137 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:32.137 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.138 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:32.138 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.177094+0000", 2020-08-07T15:09:32.138 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:32.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.153 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:32.153 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.177094+0000", 2020-08-07T15:09:32.154 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3760999999999997e-05 2020-08-07T15:09:32.154 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.154 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.154 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:32.155 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.177128+0000", 2020-08-07T15:09:32.155 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1549999999999998e-06 2020-08-07T15:09:32.155 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.156 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:32.156 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.177133+0000", 2020-08-07T15:09:32.156 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.262460639 2020-08-07T15:09:32.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.157 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.157 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:32.157 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.439594+0000", 2020-08-07T15:09:32.157 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0943e-05 2020-08-07T15:09:32.157 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.162 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.163 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:32.163 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.439605+0000", 2020-08-07T15:09:32.163 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013696100000000001 2020-08-07T15:09:32.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:32.164 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.439742+0000", 2020-08-07T15:09:32.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5649000000000003e-05 2020-08-07T15:09:32.165 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.165 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.165 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:32.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.439787+0000", 2020-08-07T15:09:32.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00063599200000000005 2020-08-07T15:09:32.166 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.166 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.166 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:32.166 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.440423+0000", 2020-08-07T15:09:32.166 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1260000000000001e-05 2020-08-07T15:09:32.183 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.184 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.184 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:32.184 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.440465+0000", 2020-08-07T15:09:32.185 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26337036000000003 2020-08-07T15:09:32.185 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:32.185 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:32.185 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:32.186 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.186 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.194 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:42280 4.6 e112/81 4:6fc8ffaf:::benchmark_data_smithi074_44586_object5284:head v 112'2464, mlcod=112'2455)", 2020-08-07T15:09:32.194 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.113698+0000", 2020-08-07T15:09:32.194 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 31.320801756000002, 2020-08-07T15:09:32.195 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43055294, 2020-08-07T15:09:32.195 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:32.195 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:32.195 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:32.195 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.196 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:32.196 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.113698+0000", 2020-08-07T15:09:32.196 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:32.196 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.197 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.197 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:32.197 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.113698+0000", 2020-08-07T15:09:32.197 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2302000000000003e-05 2020-08-07T15:09:32.197 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.198 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.198 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:32.198 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.113740+0000", 2020-08-07T15:09:32.198 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3669999999999999e-06 2020-08-07T15:09:32.198 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.199 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.199 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:32.199 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.113744+0000", 2020-08-07T15:09:32.199 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.42958231000000002 2020-08-07T15:09:32.199 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.200 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.200 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:32.200 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.543327+0000", 2020-08-07T15:09:32.200 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2373e-05 2020-08-07T15:09:32.200 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.201 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.201 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:32.201 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.543339+0000", 2020-08-07T15:09:32.201 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014704099999999999 2020-08-07T15:09:32.201 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.202 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.202 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:32.202 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.543486+0000", 2020-08-07T15:09:32.202 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4834999999999999e-05 2020-08-07T15:09:32.202 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.203 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.203 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:32.203 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.543511+0000", 2020-08-07T15:09:32.203 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00069499500000000001 2020-08-07T15:09:32.234 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.234 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.235 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:32.235 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.544206+0000", 2020-08-07T15:09:32.235 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4716999999999999e-05 2020-08-07T15:09:32.235 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.235 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.236 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:32.236 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.544251+0000", 2020-08-07T15:09:32.236 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43055294 2020-08-07T15:09:32.236 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:32.236 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:32.237 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:32.241 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.241 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.241 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:60280 4.3 e121/81 4:c4915982:::benchmark_data_smithi074_44586_object7534:head v 121'3528, mlcod=121'3519)", 2020-08-07T15:09:32.242 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:13.849777+0000", 2020-08-07T15:09:32.242 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 17.584723014000001, 2020-08-07T15:09:32.242 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25718668300000003, 2020-08-07T15:09:32.242 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:32.242 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:32.243 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:32.243 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.243 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:32.243 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:13.849777+0000", 2020-08-07T15:09:32.243 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:32.244 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.244 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.244 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:32.244 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:13.849777+0000", 2020-08-07T15:09:32.244 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2033999999999997e-05 2020-08-07T15:09:32.245 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.245 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.245 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:32.245 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:13.849829+0000", 2020-08-07T15:09:32.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5230000000000001e-06 2020-08-07T15:09:32.273 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.274 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.274 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:32.274 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:13.849835+0000", 2020-08-07T15:09:32.274 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.256119861 2020-08-07T15:09:32.274 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.275 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.275 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:32.275 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.105955+0000", 2020-08-07T15:09:32.275 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.13e-06 2020-08-07T15:09:32.275 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.276 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.276 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:32.276 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.105960+0000", 2020-08-07T15:09:32.276 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8306000000000001e-05 2020-08-07T15:09:32.276 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.277 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.277 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:32.277 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.106008+0000", 2020-08-07T15:09:32.277 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4477e-05 2020-08-07T15:09:32.277 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.278 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.278 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:32.278 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.106033+0000", 2020-08-07T15:09:32.278 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00089500699999999996 2020-08-07T15:09:32.278 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.279 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.279 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:32.279 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.106928+0000", 2020-08-07T15:09:32.279 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5345000000000001e-05 2020-08-07T15:09:32.280 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.280 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.280 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:32.280 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.106963+0000", 2020-08-07T15:09:32.280 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25718668300000003 2020-08-07T15:09:32.281 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:32.281 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:32.281 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:32.281 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.302 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.302 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:61072 4.b e121/81 4:da2d02c2:::benchmark_data_smithi074_44586_object7633:head v 121'3760, mlcod=121'3755)", 2020-08-07T15:09:32.303 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:14.333829+0000", 2020-08-07T15:09:32.303 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 17.100670811000001, 2020-08-07T15:09:32.303 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25622643699999997, 2020-08-07T15:09:32.303 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:32.303 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:32.304 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:32.304 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.304 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:32.304 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.333829+0000", 2020-08-07T15:09:32.304 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:32.305 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.305 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.305 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:32.305 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.333829+0000", 2020-08-07T15:09:32.305 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8536000000000003e-05 2020-08-07T15:09:32.306 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.306 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.306 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:32.306 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.333887+0000", 2020-08-07T15:09:32.306 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1959999999999996e-06 2020-08-07T15:09:32.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.307 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.307 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:32.325 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.333893+0000", 2020-08-07T15:09:32.327 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25306922599999998 2020-08-07T15:09:32.327 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.327 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.327 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:32.328 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.586963+0000", 2020-08-07T15:09:32.328 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6869999999999997e-06 2020-08-07T15:09:32.328 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.328 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.328 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:32.329 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.586969+0000", 2020-08-07T15:09:32.329 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017141300000000001 2020-08-07T15:09:32.329 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.329 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.329 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:32.330 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.587141+0000", 2020-08-07T15:09:32.330 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5887000000000001e-05 2020-08-07T15:09:32.330 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.330 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.330 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:32.331 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.587167+0000", 2020-08-07T15:09:32.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028273999999999999 2020-08-07T15:09:32.331 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.331 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.331 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:32.332 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.589994+0000", 2020-08-07T15:09:32.353 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1092000000000004e-05 2020-08-07T15:09:32.354 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.354 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.354 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:32.355 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.590055+0000", 2020-08-07T15:09:32.355 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25622643699999997 2020-08-07T15:09:32.355 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:32.355 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:32.355 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:32.356 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.356 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.356 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:63840 4.3 e125/81 4:cc87ec72:::benchmark_data_smithi074_44586_object7979:head v 125'3776, mlcod=125'3767)", 2020-08-07T15:09:32.356 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:18.486755+0000", 2020-08-07T15:09:32.356 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 12.947744262000001, 2020-08-07T15:09:32.357 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23910667499999999, 2020-08-07T15:09:32.357 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:32.357 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:32.378 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:32.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:32.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:18.486755+0000", 2020-08-07T15:09:32.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:32.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:32.380 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:18.486755+0000", 2020-08-07T15:09:32.380 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5102000000000002e-05 2020-08-07T15:09:32.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:32.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:18.486810+0000", 2020-08-07T15:09:32.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8850000000000001e-06 2020-08-07T15:09:32.393 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.393 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.393 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:32.395 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:18.486817+0000", 2020-08-07T15:09:32.395 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.237866138 2020-08-07T15:09:32.395 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.395 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.396 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:32.396 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:18.724683+0000", 2020-08-07T15:09:32.396 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5829999999999997e-06 2020-08-07T15:09:32.396 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.396 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.397 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:32.397 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:18.724689+0000", 2020-08-07T15:09:32.397 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3324000000000001e-05 2020-08-07T15:09:32.397 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.397 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.398 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:32.398 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:18.724732+0000", 2020-08-07T15:09:32.398 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2892999999999999e-05 2020-08-07T15:09:32.398 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.398 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.408 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:32.408 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:18.724755+0000", 2020-08-07T15:09:32.409 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001054934 2020-08-07T15:09:32.410 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.410 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.410 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:32.410 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:18.725810+0000", 2020-08-07T15:09:32.411 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1816e-05 2020-08-07T15:09:32.411 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.411 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.411 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:32.412 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:18.725862+0000", 2020-08-07T15:09:32.412 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23910667499999999 2020-08-07T15:09:32.412 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:32.412 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:32.412 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:32.413 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.413 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:72952 4.b e130/81 4:dee63f23:::benchmark_data_smithi074_44586_object9118:head v 130'4592, mlcod=130'4584)", 2020-08-07T15:09:32.413 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.589220+0000", 2020-08-07T15:09:32.413 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 5.8452793600000001, 2020-08-07T15:09:32.414 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40730996899999999, 2020-08-07T15:09:32.414 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:32.414 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:32.414 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:32.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.415 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:32.415 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.589220+0000", 2020-08-07T15:09:32.415 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:32.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.444 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:32.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.589220+0000", 2020-08-07T15:09:32.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9273000000000001e-05 2020-08-07T15:09:32.445 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.445 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.445 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:32.446 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.589269+0000", 2020-08-07T15:09:32.446 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.417e-06 2020-08-07T15:09:32.448 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:32.449 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.589275+0000", 2020-08-07T15:09:32.449 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40641516700000002 2020-08-07T15:09:32.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.449 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.449 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:32.450 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.995690+0000", 2020-08-07T15:09:32.450 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7780000000000002e-06 2020-08-07T15:09:32.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.451 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:32.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.995697+0000", 2020-08-07T15:09:32.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3010999999999999e-05 2020-08-07T15:09:32.451 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.451 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:32.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.995740+0000", 2020-08-07T15:09:32.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1878999999999998e-05 2020-08-07T15:09:32.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.453 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:32.453 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.995762+0000", 2020-08-07T15:09:32.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00069480500000000005 2020-08-07T15:09:32.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.476 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:32.476 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.996456+0000", 2020-08-07T15:09:32.477 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3639000000000004e-05 2020-08-07T15:09:32.477 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.477 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.477 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:32.478 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.996530+0000", 2020-08-07T15:09:32.478 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40730996899999999 2020-08-07T15:09:32.478 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:32.478 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:32.478 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:32.479 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:32.479 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:32.479 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:09:32.482 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:09:32.482 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.6 to 1 (10000) 2020-08-07T15:09:32.483 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:09:32.483 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 7 1 2020-08-07T15:09:32.484 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 66 16 79284 79268 9.38217 11.9141 0.030637 0.0133176 2020-08-07T15:09:32.484 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:32.480+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:32.583 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:32.582+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:32.686 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:32.684+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:32.698 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:09:32.699 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:09:32.699 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.699 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:79871 4.b 4.6158f7cb (undecoded) ondisk+write+known_if_redirected e137)", 2020-08-07T15:09:32.699 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:32.686105+0000", 2020-08-07T15:09:32.699 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0027343760000000002, 2020-08-07T15:09:32.700 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002760715, 2020-08-07T15:09:32.700 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:32.700 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:09:32.700 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:32.700 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:32.701 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:32.701 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 79871 2020-08-07T15:09:32.701 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.701 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:32.702 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.702 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:32.702 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.686105+0000", 2020-08-07T15:09:32.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:32.702 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.703 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:32.703 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.686105+0000", 2020-08-07T15:09:32.703 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7886e-05 2020-08-07T15:09:32.703 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.704 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:32.704 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.686133+0000", 2020-08-07T15:09:32.704 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3920000000000002e-06 2020-08-07T15:09:32.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.705 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.705 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:32.706 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.686139+0000", 2020-08-07T15:09:32.706 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2583e-05 2020-08-07T15:09:32.706 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.707 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:32.707 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.686151+0000", 2020-08-07T15:09:32.707 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2550000000000003e-06 2020-08-07T15:09:32.707 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.708 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:32.708 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.686158+0000", 2020-08-07T15:09:32.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2116e-05 2020-08-07T15:09:32.708 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:32.709 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:32.709 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:32.709 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.709 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.709 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:79866 4.b e137/81)", 2020-08-07T15:09:32.710 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:32.686211+0000", 2020-08-07T15:09:32.710 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0026285779999999999, 2020-08-07T15:09:32.710 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0027277949999999999, 2020-08-07T15:09:32.710 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:32.710 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:09:32.711 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:32.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.711 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:32.711 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.686211+0000", 2020-08-07T15:09:32.711 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:32.712 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.712 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.713 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:32.713 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.686211+0000", 2020-08-07T15:09:32.713 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5908000000000001e-05 2020-08-07T15:09:32.714 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.715 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.715 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:32.718 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.686227+0000", 2020-08-07T15:09:32.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9040000000000002e-06 2020-08-07T15:09:32.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:32.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.686231+0000", 2020-08-07T15:09:32.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1575000000000002e-05 2020-08-07T15:09:32.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:32.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.686263+0000", 2020-08-07T15:09:32.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.552e-06 2020-08-07T15:09:32.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:32.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.686266+0000", 2020-08-07T15:09:32.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4938999999999997e-05 2020-08-07T15:09:32.721 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:32.721 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:32.722 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:32.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.723 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:79867 4.b e137/81)", 2020-08-07T15:09:32.723 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:32.688676+0000", 2020-08-07T15:09:32.723 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.000163497, 2020-08-07T15:09:32.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00031580999999999997, 2020-08-07T15:09:32.724 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:32.724 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:09:32.724 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:32.724 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.724 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:32.725 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.688676+0000", 2020-08-07T15:09:32.725 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:32.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.726 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:32.726 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.688676+0000", 2020-08-07T15:09:32.726 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0486999999999999e-05 2020-08-07T15:09:32.727 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.727 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.729 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:32.730 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.688697+0000", 2020-08-07T15:09:32.730 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8619999999999999e-06 2020-08-07T15:09:32.730 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.731 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:32.732 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.688703+0000", 2020-08-07T15:09:32.732 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2200999999999995e-05 2020-08-07T15:09:32.732 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.733 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:32.733 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.688775+0000", 2020-08-07T15:09:32.734 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7069999999999997e-06 2020-08-07T15:09:32.734 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.734 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.734 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:32.734 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.688781+0000", 2020-08-07T15:09:32.735 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000104257 2020-08-07T15:09:32.735 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:32.735 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:32.735 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:32.735 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.736 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.736 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:79839 4.2 4:45e180bc:::benchmark_data_smithi074_44586_object9979:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e137)", 2020-08-07T15:09:32.736 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:32.669336+0000", 2020-08-07T15:09:32.736 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.019503598, 2020-08-07T15:09:32.736 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.019719181999999998, 2020-08-07T15:09:32.737 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:32.737 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:32.737 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:32.737 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:32.737 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:32.738 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 79839 2020-08-07T15:09:32.738 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.738 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:32.738 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.738 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:32.740 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.669336+0000", 2020-08-07T15:09:32.740 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:32.740 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.740 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.740 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:32.741 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.669336+0000", 2020-08-07T15:09:32.741 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4046e-05 2020-08-07T15:09:32.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.741 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.741 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:32.742 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.669360+0000", 2020-08-07T15:09:32.742 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5610000000000001e-06 2020-08-07T15:09:32.742 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.742 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.742 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:32.743 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.669363+0000", 2020-08-07T15:09:32.743 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1390000000000003e-06 2020-08-07T15:09:32.743 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:32.744 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.669368+0000", 2020-08-07T15:09:32.744 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1519999999999999e-06 2020-08-07T15:09:32.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.745 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:32.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.669370+0000", 2020-08-07T15:09:32.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0040462759999999997 2020-08-07T15:09:32.746 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.746 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:32.746 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.673416+0000", 2020-08-07T15:09:32.757 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.109e-05 2020-08-07T15:09:32.758 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.758 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.758 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:32.758 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.673478+0000", 2020-08-07T15:09:32.759 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016562700000000001 2020-08-07T15:09:32.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.759 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:09:32.759 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.673643+0000", 2020-08-07T15:09:32.759 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001681248 2020-08-07T15:09:32.760 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.760 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.760 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:32.760 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.675324+0000", 2020-08-07T15:09:32.760 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3e-07 2020-08-07T15:09:32.761 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.761 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:32.761 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.675325+0000", 2020-08-07T15:09:32.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6332000000000002e-05 2020-08-07T15:09:32.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.762 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.762 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:32.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.675381+0000", 2020-08-07T15:09:32.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002858354 2020-08-07T15:09:32.763 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.763 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:32.763 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.678240+0000", 2020-08-07T15:09:32.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0089034549999999994 2020-08-07T15:09:32.773 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:32.774 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:32.774 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:32.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.786 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:79840 4.2 4:45e180bc:::benchmark_data_smithi074_44586_object9979:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e137)", 2020-08-07T15:09:32.787 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:32.669386+0000", 2020-08-07T15:09:32.787 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.019453860999999999, 2020-08-07T15:09:32.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.019786405, 2020-08-07T15:09:32.787 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:32.787 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:32.788 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:32.798 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:32.799 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:32.799 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 79840 2020-08-07T15:09:32.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.799 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:32.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:32.800 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.669386+0000", 2020-08-07T15:09:32.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:32.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.818 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:32.818 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.669386+0000", 2020-08-07T15:09:32.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3543000000000001e-05 2020-08-07T15:09:32.819 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:32.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.669410+0000", 2020-08-07T15:09:32.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4990000000000001e-06 2020-08-07T15:09:32.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:32.870 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.669412+0000", 2020-08-07T15:09:32.871 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.101e-06 2020-08-07T15:09:32.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.895 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:32.895 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.669417+0000", 2020-08-07T15:09:32.895 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0360000000000001e-06 2020-08-07T15:09:32.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.896 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:32.897 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.669419+0000", 2020-08-07T15:09:32.897 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0046062289999999999 2020-08-07T15:09:32.897 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:32.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.674025+0000", 2020-08-07T15:09:32.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3442000000000005e-05 2020-08-07T15:09:32.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:32.964 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.674109+0000", 2020-08-07T15:09:32.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014888799999999999 2020-08-07T15:09:32.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:32.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:32.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:09:32.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.674258+0000", 2020-08-07T15:09:33.002 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011301499999999999 2020-08-07T15:09:33.002 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:33.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.675388+0000", 2020-08-07T15:09:33.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8000000000000001e-07 2020-08-07T15:09:33.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:33.004 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.675388+0000", 2020-08-07T15:09:33.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8918000000000002e-05 2020-08-07T15:09:33.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:33.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.675427+0000", 2020-08-07T15:09:33.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002831563 2020-08-07T15:09:33.009 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.009 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.010 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:33.010 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.678259+0000", 2020-08-07T15:09:33.010 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0088727489999999992 2020-08-07T15:09:33.011 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:33.011 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:33.011 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:33.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.012 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:79865 4.b 4:d3ef1a86:::benchmark_data_smithi074_44586_object9983:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e137)", 2020-08-07T15:09:33.012 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:32.683298+0000", 2020-08-07T15:09:33.012 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0055422570000000001, 2020-08-07T15:09:33.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0059968529999999999, 2020-08-07T15:09:33.012 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:33.013 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:09:33.013 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:33.013 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:33.013 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:33.013 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 79865 2020-08-07T15:09:33.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.020 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:33.020 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.020 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:33.020 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.683298+0000", 2020-08-07T15:09:33.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:33.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:33.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.683298+0000", 2020-08-07T15:09:33.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6536999999999998e-05 2020-08-07T15:09:33.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:33.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.683354+0000", 2020-08-07T15:09:33.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0530000000000001e-06 2020-08-07T15:09:33.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:33.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.683361+0000", 2020-08-07T15:09:33.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4929999999999999e-05 2020-08-07T15:09:33.030 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.030 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.030 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:33.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.683376+0000", 2020-08-07T15:09:33.030 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7749999999999993e-06 2020-08-07T15:09:33.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:33.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.683384+0000", 2020-08-07T15:09:33.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1882000000000002e-05 2020-08-07T15:09:33.038 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.038 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:33.038 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.683436+0000", 2020-08-07T15:09:33.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000187326 2020-08-07T15:09:33.039 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.039 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.039 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:33.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.683623+0000", 2020-08-07T15:09:33.040 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000183418 2020-08-07T15:09:33.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:09:33.040 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.683807+0000", 2020-08-07T15:09:33.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00050892099999999996 2020-08-07T15:09:33.041 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:33.041 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:33.041 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:33.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.048 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.049 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:79866 4.b 4:d3ef1a86:::benchmark_data_smithi074_44586_object9983:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e137)", 2020-08-07T15:09:33.049 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:32.683896+0000", 2020-08-07T15:09:33.049 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0049441329999999999, 2020-08-07T15:09:33.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054811199999999999, 2020-08-07T15:09:33.050 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:33.050 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:09:33.050 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:33.050 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:33.050 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:33.051 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 79866 2020-08-07T15:09:33.051 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.051 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:33.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.051 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:33.052 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.683896+0000", 2020-08-07T15:09:33.052 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:33.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.052 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.052 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:33.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.683896+0000", 2020-08-07T15:09:33.053 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9734000000000002e-05 2020-08-07T15:09:33.053 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.053 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:33.080 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.683945+0000", 2020-08-07T15:09:33.080 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4840000000000001e-06 2020-08-07T15:09:33.080 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.081 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.081 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:33.081 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.683952+0000", 2020-08-07T15:09:33.081 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3404000000000001e-05 2020-08-07T15:09:33.081 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.082 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:33.082 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.683965+0000", 2020-08-07T15:09:33.082 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.207e-06 2020-08-07T15:09:33.082 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.083 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.083 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:33.083 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.683972+0000", 2020-08-07T15:09:33.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00029369699999999999 2020-08-07T15:09:33.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.084 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:33.084 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.684265+0000", 2020-08-07T15:09:33.104 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8817e-05 2020-08-07T15:09:33.105 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.105 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.105 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:33.114 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.684354+0000", 2020-08-07T15:09:33.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000174177 2020-08-07T15:09:33.115 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.115 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.116 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:09:33.116 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.684528+0000", 2020-08-07T15:09:33.117 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00063252000000000002 2020-08-07T15:09:33.117 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:33.132 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:33.132 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:33.133 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.133 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.134 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:79867 4.b 4:d3ef1a86:::benchmark_data_smithi074_44586_object9983:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e137)", 2020-08-07T15:09:33.134 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:32.684013+0000", 2020-08-07T15:09:33.134 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0048270819999999999, 2020-08-07T15:09:33.135 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054238690000000004, 2020-08-07T15:09:33.135 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:33.136 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:09:33.136 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:33.136 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:33.137 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:33.137 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 79867 2020-08-07T15:09:33.158 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.159 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:33.159 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.159 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:33.159 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.684013+0000", 2020-08-07T15:09:33.159 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:33.160 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.160 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.160 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:33.160 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.684013+0000", 2020-08-07T15:09:33.160 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7827000000000001e-05 2020-08-07T15:09:33.161 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.161 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.161 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:33.161 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.684061+0000", 2020-08-07T15:09:33.170 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3519999999999997e-06 2020-08-07T15:09:33.170 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.170 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.171 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:33.171 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.684066+0000", 2020-08-07T15:09:33.171 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1388e-05 2020-08-07T15:09:33.173 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.174 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:33.182 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.684077+0000", 2020-08-07T15:09:33.182 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2249999999999999e-06 2020-08-07T15:09:33.182 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.183 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.183 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:33.183 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.684083+0000", 2020-08-07T15:09:33.183 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00081256600000000003 2020-08-07T15:09:33.183 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.184 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.184 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:33.184 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.684895+0000", 2020-08-07T15:09:33.184 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2255000000000004e-05 2020-08-07T15:09:33.185 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.185 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.187 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:33.188 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.684987+0000", 2020-08-07T15:09:33.188 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017351399999999999 2020-08-07T15:09:33.188 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.188 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.188 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:09:33.189 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.685161+0000", 2020-08-07T15:09:33.189 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011481270000000001 2020-08-07T15:09:33.189 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:33.189 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:33.189 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:33.190 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.190 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.190 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:79868 4.b 4:d3ef1a86:::benchmark_data_smithi074_44586_object9983:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e137)", 2020-08-07T15:09:33.190 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:32.684610+0000", 2020-08-07T15:09:33.190 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0042293859999999999, 2020-08-07T15:09:33.191 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0048696709999999999, 2020-08-07T15:09:33.191 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:33.191 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:09:33.191 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:33.191 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:33.192 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:33.192 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 79868 2020-08-07T15:09:33.218 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.218 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:33.221 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.221 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:33.221 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.684610+0000", 2020-08-07T15:09:33.221 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:33.222 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.222 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.222 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:33.222 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.684610+0000", 2020-08-07T15:09:33.223 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6752999999999998e-05 2020-08-07T15:09:33.223 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.223 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.223 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:33.223 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.684657+0000", 2020-08-07T15:09:33.228 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1046000000000001e-05 2020-08-07T15:09:33.228 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.228 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.229 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:33.229 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.684668+0000", 2020-08-07T15:09:33.231 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2147000000000001e-05 2020-08-07T15:09:33.231 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.231 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.232 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:33.233 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.684680+0000", 2020-08-07T15:09:33.233 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.75e-06 2020-08-07T15:09:33.233 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.234 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.234 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:33.235 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.684686+0000", 2020-08-07T15:09:33.235 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00080632100000000003 2020-08-07T15:09:33.235 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.235 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.237 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:33.238 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.685492+0000", 2020-08-07T15:09:33.238 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1712999999999994e-05 2020-08-07T15:09:33.238 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.238 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.239 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:33.239 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.685584+0000", 2020-08-07T15:09:33.239 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017830999999999999 2020-08-07T15:09:33.239 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.240 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.240 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:09:33.240 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.685763+0000", 2020-08-07T15:09:33.240 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00115204 2020-08-07T15:09:33.240 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:33.241 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:33.255 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:33.255 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.255 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.256 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:79869 4.b 4:d3ef1a86:::benchmark_data_smithi074_44586_object9983:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e137)", 2020-08-07T15:09:33.256 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:32.685220+0000", 2020-08-07T15:09:33.256 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0036197289999999999, 2020-08-07T15:09:33.256 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004302592, 2020-08-07T15:09:33.257 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:33.257 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:09:33.257 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:33.257 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:33.258 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:33.258 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 79869 2020-08-07T15:09:33.258 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.270 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:33.271 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:33.271 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.685220+0000", 2020-08-07T15:09:33.271 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:33.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:33.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.685220+0000", 2020-08-07T15:09:33.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1583999999999999e-05 2020-08-07T15:09:33.273 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.274 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.274 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:33.274 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.685272+0000", 2020-08-07T15:09:33.274 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3500000000000002e-06 2020-08-07T15:09:33.275 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.276 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.276 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:33.276 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.685278+0000", 2020-08-07T15:09:33.276 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3463e-05 2020-08-07T15:09:33.277 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.277 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.277 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:33.277 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.685292+0000", 2020-08-07T15:09:33.278 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0809999999999999e-06 2020-08-07T15:09:33.278 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.278 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.278 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:33.278 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.685298+0000", 2020-08-07T15:09:33.279 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0029916109999999999 2020-08-07T15:09:33.279 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.279 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.280 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:33.285 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.688289+0000", 2020-08-07T15:09:33.294 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4865999999999996e-05 2020-08-07T15:09:33.294 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.295 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.295 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:33.295 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.688384+0000", 2020-08-07T15:09:33.295 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00021751499999999999 2020-08-07T15:09:33.296 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.296 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.296 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:09:33.296 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.688602+0000", 2020-08-07T15:09:33.297 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033814700000000001 2020-08-07T15:09:33.297 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:33.297 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:33.297 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:33.298 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.298 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.298 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:79870 4.b 4.6158f7cb (undecoded) ondisk+write+known_if_redirected e137)", 2020-08-07T15:09:33.298 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:32.685336+0000", 2020-08-07T15:09:33.299 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.003503553, 2020-08-07T15:09:33.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042269789999999996, 2020-08-07T15:09:33.300 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:33.300 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:09:33.300 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:33.321 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:33.321 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:33.321 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 79870 2020-08-07T15:09:33.321 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.322 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:33.322 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.322 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:33.322 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.685336+0000", 2020-08-07T15:09:33.323 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:33.323 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.323 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.323 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:33.323 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.685336+0000", 2020-08-07T15:09:33.324 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0868999999999997e-05 2020-08-07T15:09:33.324 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.324 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.324 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:33.326 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.685387+0000", 2020-08-07T15:09:33.326 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5809999999999996e-06 2020-08-07T15:09:33.326 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.326 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.327 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:33.327 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.685393+0000", 2020-08-07T15:09:33.327 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1445e-05 2020-08-07T15:09:33.327 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.327 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.328 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:33.328 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.685404+0000", 2020-08-07T15:09:33.328 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0980000000000001e-06 2020-08-07T15:09:33.329 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.331 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.331 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:33.334 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.685409+0000", 2020-08-07T15:09:33.335 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2992999999999997e-05 2020-08-07T15:09:33.335 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:33.336 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:33.336 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:33.336 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.336 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.337 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:79865 4.b e137/81)", 2020-08-07T15:09:33.337 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:32.685891+0000", 2020-08-07T15:09:33.337 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0029486349999999998, 2020-08-07T15:09:33.337 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0037173990000000001, 2020-08-07T15:09:33.338 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:33.338 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:09:33.338 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:33.338 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.339 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:33.339 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.685891+0000", 2020-08-07T15:09:33.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:33.340 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.341 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:33.341 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.685891+0000", 2020-08-07T15:09:33.341 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5055999999999997e-05 2020-08-07T15:09:33.342 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.373 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:33.373 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.685936+0000", 2020-08-07T15:09:33.374 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5270000000000001e-06 2020-08-07T15:09:33.374 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.374 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.374 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:33.374 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.685944+0000", 2020-08-07T15:09:33.375 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6026999999999996e-05 2020-08-07T15:09:33.375 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.375 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:33.376 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.686020+0000", 2020-08-07T15:09:33.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6779999999999997e-06 2020-08-07T15:09:33.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:33.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:33.376 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:33.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:32.686026+0000", 2020-08-07T15:09:33.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013428800000000001 2020-08-07T15:09:33.377 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:33.377 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:33.377 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:33.393 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:33.422 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:09:33.423 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 12 2020-08-07T15:09:33.423 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:09:33.439 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:09:33.463 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:09:33.479 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 67 16 80848 80832 9.42449 12.2188 0.00709684 0.0132587 2020-08-07T15:09:33.480 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.7 to 1 (10000) 2020-08-07T15:09:33.481 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2020-08-07T15:09:33.489 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:09:33.496 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:33.438+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:33.580 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:33.559+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:33.654 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:33.643+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:33.709 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:09:33.710 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:09:33.711 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:09:33.711 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:09:33.711 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:09:33.720 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:09:33.747 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:09:33.747 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:33.746+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:33.785 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:09:33.809 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:09:33.853 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:33.850+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:33.854 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:09:33.888 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:09:33.932 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:33.932 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:33.947 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:09:33.948 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:09:33.958 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:33.956+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:33.992 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:09:33.994 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:09:33.994 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:09:33.997 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2020-08-07T15:09:33.439329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":645054464,"num_objects":9846,"num_object_clones":0,"num_object_copies":29538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":641,"num_objects_unfound":0,"num_objects_dirty":9846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78742,"num_write_kb":629936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2275,"num_bytes_recovered":148971520,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":135,"acting":135,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":10361580,"kb_used_data":1972908,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":744613140,"statfs":{"total":773094113280,"available":762483855360,"internally_reserved":8589934592,"allocated":2020257792,"data_stored":2013868752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46776320,"num_objects":715,"num_object_clones":0,"num_object_copies":2145,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":641,"num_objects_unfound":0,"num_objects_dirty":715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5710,"num_write_kb":45680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.256508"},"pg_stats":[{"pgid":"1.14","version":"16'2","reported_seq":"65","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199857+0000","last_change":"2020-08-07T15:08:42.533999+0000","last_active":"2020-08-07T15:09:30.199857+0000","last_peered":"2020-08-07T15:09:30.199857+0000","last_clean":"2020-08-07T15:09:30.199857+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:09:30.199857+0000","last_undegraded":"2020-08-07T15:09:30.199857+0000","last_fullsized":"2020-08-07T15:09:30.199857+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:42.533921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:42.533921+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"135","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.206027+0000","last_change":"2020-08-07T15:09:30.205031+0000","last_active":"2020-08-07T15:09:30.206027+0000","last_peered":"2020-08-07T15:09:30.206027+0000","last_clean":"2020-08-07T15:09:30.206027+0000","last_became_active":"2020-08-07T15:09:30.203592+0000","last_became_peered":"2020-08-07T15:09:30.203592+0000","last_unstale":"2020-08-07T15:09:30.206027+0000","last_undegraded":"2020-08-07T15:09:30.206027+0000","last_fullsized":"2020-08-07T15:09:30.206027+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.141010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.141010+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"65","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.200795+0000","last_change":"2020-08-07T15:08:55.078015+0000","last_active":"2020-08-07T15:09:30.200795+0000","last_peered":"2020-08-07T15:09:30.200795+0000","last_clean":"2020-08-07T15:09:30.200795+0000","last_became_active":"2020-08-07T15:08:41.685106+0000","last_became_peered":"2020-08-07T15:08:41.685106+0000","last_unstale":"2020-08-07T15:09:30.200795+0000","last_undegraded":"2020-08-07T15:09:30.200795+0000","last_fullsized":"2020-08-07T15:09:30.200795+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:55.077924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:55.077924+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"178","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.213864+0000","last_change":"2020-08-07T15:09:30.210669+0000","last_active":"2020-08-07T15:09:30.213864+0000","last_peered":"2020-08-07T15:09:30.213864+0000","last_clean":"2020-08-07T15:09:30.213864+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:09:30.213864+0000","last_undegraded":"2020-08-07T15:09:30.213864+0000","last_fullsized":"2020-08-07T15:09:30.213864+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.544324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.544324+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"172","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.200414+0000","last_change":"2020-08-07T15:08:51.599032+0000","last_active":"2020-08-07T15:09:30.200414+0000","last_peered":"2020-08-07T15:09:30.200414+0000","last_clean":"2020-08-07T15:09:30.200414+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:09:30.200414+0000","last_undegraded":"2020-08-07T15:09:30.200414+0000","last_fullsized":"2020-08-07T15:09:30.200414+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:51.598947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.598947+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"131","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.503476+0000","last_change":"2020-08-07T15:08:11.840861+0000","last_active":"2020-08-07T15:09:31.503476+0000","last_peered":"2020-08-07T15:09:31.503476+0000","last_clean":"2020-08-07T15:09:31.503476+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:09:31.503476+0000","last_undegraded":"2020-08-07T15:09:31.503476+0000","last_fullsized":"2020-08-07T15:09:31.503476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:11.840807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:11.840807+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"49","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.284670+0000","last_change":"2020-08-07T15:09:30.283753+0000","last_active":"2020-08-07T15:09:30.284670+0000","last_peered":"2020-08-07T15:09:30.284670+0000","last_clean":"2020-08-07T15:09:30.284670+0000","last_became_active":"2020-08-07T15:09:30.283176+0000","last_became_peered":"2020-08-07T15:09:30.283176+0000","last_unstale":"2020-08-07T15:09:30.284670+0000","last_undegraded":"2020-08-07T15:09:30.284670+0000","last_fullsized":"2020-08-07T15:09:30.284670+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:25.606757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:25.606757+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"136'4608","reported_seq":"4671","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.580740+0000","last_change":"2020-08-07T15:08:46.715869+0000","last_active":"2020-08-07T15:09:31.580740+0000","last_peered":"2020-08-07T15:09:31.580740+0000","last_clean":"2020-08-07T15:09:31.580740+0000","last_became_active":"2020-08-07T15:08:46.708112+0000","last_became_peered":"2020-08-07T15:08:46.708112+0000","last_unstale":"2020-08-07T15:09:31.580740+0000","last_undegraded":"2020-08-07T15:09:31.580740+0000","last_fullsized":"2020-08-07T15:09:31.580740+0000","mapping_epoch":98,"log_start":"135'4600","ondisk_log_start":"135'4600","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":37748736,"num_objects":576,"num_object_clones":0,"num_object_copies":1728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4608,"num_write_kb":36864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"133","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.194839+0000","last_change":"2020-08-07T15:08:40.057705+0000","last_active":"2020-08-07T15:09:30.194839+0000","last_peered":"2020-08-07T15:09:30.194839+0000","last_clean":"2020-08-07T15:09:30.194839+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:09:30.194839+0000","last_undegraded":"2020-08-07T15:09:30.194839+0000","last_fullsized":"2020-08-07T15:09:30.194839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:40.057622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:40.057622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"90","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.200120+0000","last_change":"2020-08-07T15:08:53.756554+0000","last_active":"2020-08-07T15:09:30.200120+0000","last_peered":"2020-08-07T15:09:30.200120+0000","last_clean":"2020-08-07T15:09:30.200120+0000","last_became_active":"2020-08-07T15:08:46.710404+0000","last_became_peered":"2020-08-07T15:08:46.710404+0000","last_unstale":"2020-08-07T15:09:30.200120+0000","last_undegraded":"2020-08-07T15:09:30.200120+0000","last_fullsized":"2020-08-07T15:09:30.200120+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:53.756343+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:53.756343+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"136'5176","reported_seq":"7001","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.717108+0000","last_change":"2020-08-07T15:09:27.180782+0000","last_active":"2020-08-07T15:09:31.717108+0000","last_peered":"2020-08-07T15:09:31.717108+0000","last_clean":"2020-08-07T15:09:31.717108+0000","last_became_active":"2020-08-07T15:09:27.180158+0000","last_became_peered":"2020-08-07T15:09:27.180158+0000","last_unstale":"2020-08-07T15:09:31.717108+0000","last_undegraded":"2020-08-07T15:09:31.717108+0000","last_fullsized":"2020-08-07T15:09:31.717108+0000","mapping_epoch":131,"log_start":"136'5169","ondisk_log_start":"136'5169","created":81,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":42401792,"num_objects":647,"num_object_clones":0,"num_object_copies":1941,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5176,"num_write_kb":41408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":875,"num_bytes_recovered":57344000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"130","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199794+0000","last_change":"2020-08-07T15:08:27.906894+0000","last_active":"2020-08-07T15:09:30.199794+0000","last_peered":"2020-08-07T15:09:30.199794+0000","last_clean":"2020-08-07T15:09:30.199794+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:09:30.199794+0000","last_undegraded":"2020-08-07T15:09:30.199794+0000","last_fullsized":"2020-08-07T15:09:30.199794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:27.906822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:27.906822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"170","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.503477+0000","last_change":"2020-08-07T15:09:30.202363+0000","last_active":"2020-08-07T15:09:31.503477+0000","last_peered":"2020-08-07T15:09:31.503477+0000","last_clean":"2020-08-07T15:09:31.503477+0000","last_became_active":"2020-08-07T15:09:30.201829+0000","last_became_peered":"2020-08-07T15:09:30.201829+0000","last_unstale":"2020-08-07T15:09:31.503477+0000","last_undegraded":"2020-08-07T15:09:31.503477+0000","last_fullsized":"2020-08-07T15:09:31.503477+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.899280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.899280+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"136'4672","reported_seq":"4735","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.621604+0000","last_change":"2020-08-07T15:08:24.549715+0000","last_active":"2020-08-07T15:09:31.621604+0000","last_peered":"2020-08-07T15:09:31.621604+0000","last_clean":"2020-08-07T15:09:31.621604+0000","last_became_active":"2020-08-07T15:08:24.549245+0000","last_became_peered":"2020-08-07T15:08:24.549245+0000","last_unstale":"2020-08-07T15:09:31.621604+0000","last_undegraded":"2020-08-07T15:09:31.621604+0000","last_fullsized":"2020-08-07T15:09:31.621604+0000","mapping_epoch":81,"log_start":"136'4666","ondisk_log_start":"136'4666","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":38273024,"num_objects":584,"num_object_clones":0,"num_object_copies":1752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4672,"num_write_kb":37376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"130","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199214+0000","last_change":"2020-08-07T15:08:13.867355+0000","last_active":"2020-08-07T15:09:30.199214+0000","last_peered":"2020-08-07T15:09:30.199214+0000","last_clean":"2020-08-07T15:09:30.199214+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:09:30.199214+0000","last_undegraded":"2020-08-07T15:09:30.199214+0000","last_fullsized":"2020-08-07T15:09:30.199214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:13.867148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:13.867148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"136","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.503099+0000","last_change":"2020-08-07T15:08:50.686578+0000","last_active":"2020-08-07T15:09:31.503099+0000","last_peered":"2020-08-07T15:09:31.503099+0000","last_clean":"2020-08-07T15:09:31.503099+0000","last_became_active":"2020-08-07T15:08:36.633406+0000","last_became_peered":"2020-08-07T15:08:36.633406+0000","last_unstale":"2020-08-07T15:09:31.503099+0000","last_undegraded":"2020-08-07T15:09:31.503099+0000","last_fullsized":"2020-08-07T15:09:31.503099+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:50.686496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:50.686496+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"136'4536","reported_seq":"4598","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.571269+0000","last_change":"2020-08-07T15:08:24.557398+0000","last_active":"2020-08-07T15:09:31.571269+0000","last_peered":"2020-08-07T15:09:31.571269+0000","last_clean":"2020-08-07T15:09:31.571269+0000","last_became_active":"2020-08-07T15:08:24.557022+0000","last_became_peered":"2020-08-07T15:08:24.557022+0000","last_unstale":"2020-08-07T15:09:31.571269+0000","last_undegraded":"2020-08-07T15:09:31.571269+0000","last_fullsized":"2020-08-07T15:09:31.571269+0000","mapping_epoch":81,"log_start":"136'4531","ondisk_log_start":"136'4531","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":37158912,"num_objects":567,"num_object_clones":0,"num_object_copies":1701,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4536,"num_write_kb":36288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"162","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.197478+0000","last_change":"2020-08-07T15:08:43.036067+0000","last_active":"2020-08-07T15:09:30.197478+0000","last_peered":"2020-08-07T15:09:30.197478+0000","last_clean":"2020-08-07T15:09:30.197478+0000","last_became_active":"2020-08-07T15:08:36.627426+0000","last_became_peered":"2020-08-07T15:08:36.627426+0000","last_unstale":"2020-08-07T15:09:30.197478+0000","last_undegraded":"2020-08-07T15:09:30.197478+0000","last_fullsized":"2020-08-07T15:09:30.197478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.035987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.035987+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"130","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.200827+0000","last_change":"2020-08-07T15:08:26.317430+0000","last_active":"2020-08-07T15:09:30.200827+0000","last_peered":"2020-08-07T15:09:30.200827+0000","last_clean":"2020-08-07T15:09:30.200827+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:09:30.200827+0000","last_undegraded":"2020-08-07T15:09:30.200827+0000","last_fullsized":"2020-08-07T15:09:30.200827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:26.317351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:26.317351+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"137'5027","reported_seq":"5088","reported_epoch":"137","state":"active+clean","last_fresh":"2020-08-07T15:09:32.697664+0000","last_change":"2020-08-07T15:08:24.552570+0000","last_active":"2020-08-07T15:09:32.697664+0000","last_peered":"2020-08-07T15:09:32.697664+0000","last_clean":"2020-08-07T15:09:32.697664+0000","last_became_active":"2020-08-07T15:08:24.552105+0000","last_became_peered":"2020-08-07T15:08:24.552105+0000","last_unstale":"2020-08-07T15:09:32.697664+0000","last_undegraded":"2020-08-07T15:09:32.697664+0000","last_fullsized":"2020-08-07T15:09:32.697664+0000","mapping_epoch":81,"log_start":"137'5023","ondisk_log_start":"137'5023","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41181184,"num_objects":629,"num_object_clones":0,"num_object_copies":1887,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5027,"num_write_kb":40216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"151","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199951+0000","last_change":"2020-08-07T15:09:27.181676+0000","last_active":"2020-08-07T15:09:30.199951+0000","last_peered":"2020-08-07T15:09:30.199951+0000","last_clean":"2020-08-07T15:09:30.199951+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:09:30.199951+0000","last_undegraded":"2020-08-07T15:09:30.199951+0000","last_fullsized":"2020-08-07T15:09:30.199951+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:05.606603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:05.606603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"178","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199259+0000","last_change":"2020-08-07T15:08:42.069637+0000","last_active":"2020-08-07T15:09:30.199259+0000","last_peered":"2020-08-07T15:09:30.199259+0000","last_clean":"2020-08-07T15:09:30.199259+0000","last_became_active":"2020-08-07T15:08:36.628365+0000","last_became_peered":"2020-08-07T15:08:36.628365+0000","last_unstale":"2020-08-07T15:09:30.199259+0000","last_undegraded":"2020-08-07T15:09:30.199259+0000","last_fullsized":"2020-08-07T15:09:30.199259+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:42.069585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:42.069585+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"136'5016","reported_seq":"5075","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.699090+0000","last_change":"2020-08-07T15:08:24.550789+0000","last_active":"2020-08-07T15:09:31.699090+0000","last_peered":"2020-08-07T15:09:31.699090+0000","last_clean":"2020-08-07T15:09:31.699090+0000","last_became_active":"2020-08-07T15:08:24.550222+0000","last_became_peered":"2020-08-07T15:08:24.550222+0000","last_unstale":"2020-08-07T15:09:31.699090+0000","last_undegraded":"2020-08-07T15:09:31.699090+0000","last_fullsized":"2020-08-07T15:09:31.699090+0000","mapping_epoch":81,"log_start":"136'5011","ondisk_log_start":"136'5011","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41091072,"num_objects":627,"num_object_clones":0,"num_object_copies":1881,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5016,"num_write_kb":40128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"130","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.201090+0000","last_change":"2020-08-07T15:08:30.339852+0000","last_active":"2020-08-07T15:09:30.201090+0000","last_peered":"2020-08-07T15:09:30.201090+0000","last_clean":"2020-08-07T15:09:30.201090+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:09:30.201090+0000","last_undegraded":"2020-08-07T15:09:30.201090+0000","last_fullsized":"2020-08-07T15:09:30.201090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:08:30.339785+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"128","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.202603+0000","last_change":"2020-08-07T15:08:38.386792+0000","last_active":"2020-08-07T15:09:30.202603+0000","last_peered":"2020-08-07T15:09:30.202603+0000","last_clean":"2020-08-07T15:09:30.202603+0000","last_became_active":"2020-08-07T15:08:36.618164+0000","last_became_peered":"2020-08-07T15:08:36.618164+0000","last_unstale":"2020-08-07T15:09:30.202603+0000","last_undegraded":"2020-08-07T15:09:30.202603+0000","last_fullsized":"2020-08-07T15:09:30.202603+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:38.386739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:38.386739+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"136'4880","reported_seq":"6034","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.635996+0000","last_change":"2020-08-07T15:09:27.199178+0000","last_active":"2020-08-07T15:09:31.635996+0000","last_peered":"2020-08-07T15:09:31.635996+0000","last_clean":"2020-08-07T15:09:31.635996+0000","last_became_active":"2020-08-07T15:09:27.183115+0000","last_became_peered":"2020-08-07T15:09:27.183115+0000","last_unstale":"2020-08-07T15:09:31.635996+0000","last_undegraded":"2020-08-07T15:09:31.635996+0000","last_fullsized":"2020-08-07T15:09:31.635996+0000","mapping_epoch":131,"log_start":"136'4872","ondisk_log_start":"136'4872","created":81,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":39976960,"num_objects":610,"num_object_clones":0,"num_object_copies":1830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4880,"num_write_kb":39040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":536,"num_bytes_recovered":35127296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"127","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199444+0000","last_change":"2020-08-07T15:08:15.887617+0000","last_active":"2020-08-07T15:09:30.199444+0000","last_peered":"2020-08-07T15:09:30.199444+0000","last_clean":"2020-08-07T15:09:30.199444+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:09:30.199444+0000","last_undegraded":"2020-08-07T15:09:30.199444+0000","last_fullsized":"2020-08-07T15:09:30.199444+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:15.887513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:15.887513+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"204","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.503550+0000","last_change":"2020-08-07T15:09:30.208873+0000","last_active":"2020-08-07T15:09:31.503550+0000","last_peered":"2020-08-07T15:09:31.503550+0000","last_clean":"2020-08-07T15:09:31.503550+0000","last_became_active":"2020-08-07T15:09:30.208345+0000","last_became_peered":"2020-08-07T15:09:30.208345+0000","last_unstale":"2020-08-07T15:09:31.503550+0000","last_undegraded":"2020-08-07T15:09:31.503550+0000","last_fullsized":"2020-08-07T15:09:31.503550+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.381325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.381325+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"136'4707","reported_seq":"4763","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.727732+0000","last_change":"2020-08-07T15:08:24.552962+0000","last_active":"2020-08-07T15:09:31.727732+0000","last_peered":"2020-08-07T15:09:31.727732+0000","last_clean":"2020-08-07T15:09:31.727732+0000","last_became_active":"2020-08-07T15:08:24.552070+0000","last_became_peered":"2020-08-07T15:08:24.552070+0000","last_unstale":"2020-08-07T15:09:31.727732+0000","last_undegraded":"2020-08-07T15:09:31.727732+0000","last_fullsized":"2020-08-07T15:09:31.727732+0000","mapping_epoch":81,"log_start":"136'4698","ondisk_log_start":"136'4698","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":38559744,"num_objects":589,"num_object_clones":0,"num_object_copies":1767,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4707,"num_write_kb":37656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"105","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.200989+0000","last_change":"2020-08-07T15:08:39.409159+0000","last_active":"2020-08-07T15:09:30.200989+0000","last_peered":"2020-08-07T15:09:30.200989+0000","last_clean":"2020-08-07T15:09:30.200989+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:09:30.200989+0000","last_undegraded":"2020-08-07T15:09:30.200989+0000","last_fullsized":"2020-08-07T15:09:30.200989+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:39.409063+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_clean_scrub_stamp":"2020-08-07T15:08:39.409063+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"136'4936","reported_seq":"4999","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.718382+0000","last_change":"2020-08-07T15:08:24.554063+0000","last_active":"2020-08-07T15:09:31.718382+0000","last_peered":"2020-08-07T15:09:31.718382+0000","last_clean":"2020-08-07T15:09:31.718382+0000","last_became_active":"2020-08-07T15:08:24.553668+0000","last_became_peered":"2020-08-07T15:08:24.553668+0000","last_unstale":"2020-08-07T15:09:31.718382+0000","last_undegraded":"2020-08-07T15:09:31.718382+0000","last_fullsized":"2020-08-07T15:09:31.718382+0000","mapping_epoch":81,"log_start":"136'4930","ondisk_log_start":"136'4930","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":40435712,"num_objects":617,"num_object_clones":0,"num_object_copies":1851,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4936,"num_write_kb":39488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"105","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.198725+0000","last_change":"2020-08-07T15:09:09.691838+0000","last_active":"2020-08-07T15:09:30.198725+0000","last_peered":"2020-08-07T15:09:30.198725+0000","last_clean":"2020-08-07T15:09:30.198725+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:09:30.198725+0000","last_undegraded":"2020-08-07T15:09:30.198725+0000","last_fullsized":"2020-08-07T15:09:30.198725+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.691753+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"136'4864","reported_seq":"4927","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.634461+0000","last_change":"2020-08-07T15:08:46.709789+0000","last_active":"2020-08-07T15:09:31.634461+0000","last_peered":"2020-08-07T15:09:31.634461+0000","last_clean":"2020-08-07T15:09:31.634461+0000","last_became_active":"2020-08-07T15:08:46.708674+0000","last_became_peered":"2020-08-07T15:08:46.708674+0000","last_unstale":"2020-08-07T15:09:31.634461+0000","last_undegraded":"2020-08-07T15:09:31.634461+0000","last_fullsized":"2020-08-07T15:09:31.634461+0000","mapping_epoch":98,"log_start":"136'4856","ondisk_log_start":"136'4856","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":39845888,"num_objects":608,"num_object_clones":0,"num_object_copies":1824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4864,"num_write_kb":38912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"169","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199320+0000","last_change":"2020-08-07T15:09:25.984108+0000","last_active":"2020-08-07T15:09:30.199320+0000","last_peered":"2020-08-07T15:09:30.199320+0000","last_clean":"2020-08-07T15:09:30.199320+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:09:30.199320+0000","last_undegraded":"2020-08-07T15:09:30.199320+0000","last_fullsized":"2020-08-07T15:09:30.199320+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:25.984018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:25.984018+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"136'5008","reported_seq":"5091","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.689702+0000","last_change":"2020-08-07T15:08:52.193226+0000","last_active":"2020-08-07T15:09:31.689702+0000","last_peered":"2020-08-07T15:09:31.689702+0000","last_clean":"2020-08-07T15:09:31.689702+0000","last_became_active":"2020-08-07T15:08:52.166336+0000","last_became_peered":"2020-08-07T15:08:52.166336+0000","last_unstale":"2020-08-07T15:09:31.689702+0000","last_undegraded":"2020-08-07T15:09:31.689702+0000","last_fullsized":"2020-08-07T15:09:31.689702+0000","mapping_epoch":104,"log_start":"136'5002","ondisk_log_start":"136'5002","created":81,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41025536,"num_objects":626,"num_object_clones":0,"num_object_copies":1878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5008,"num_write_kb":40064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":40960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"124","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199606+0000","last_change":"2020-08-07T15:08:51.513398+0000","last_active":"2020-08-07T15:09:30.199606+0000","last_peered":"2020-08-07T15:09:30.199606+0000","last_clean":"2020-08-07T15:09:30.199606+0000","last_became_active":"2020-08-07T15:08:41.680448+0000","last_became_peered":"2020-08-07T15:08:41.680448+0000","last_unstale":"2020-08-07T15:09:30.199606+0000","last_undegraded":"2020-08-07T15:09:30.199606+0000","last_fullsized":"2020-08-07T15:09:30.199606+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:51.513319+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.513319+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"136'4856","reported_seq":"4917","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.644606+0000","last_change":"2020-08-07T15:08:24.550496+0000","last_active":"2020-08-07T15:09:31.644606+0000","last_peered":"2020-08-07T15:09:31.644606+0000","last_clean":"2020-08-07T15:09:31.644606+0000","last_became_active":"2020-08-07T15:08:24.549868+0000","last_became_peered":"2020-08-07T15:08:24.549868+0000","last_unstale":"2020-08-07T15:09:31.644606+0000","last_undegraded":"2020-08-07T15:09:31.644606+0000","last_fullsized":"2020-08-07T15:09:31.644606+0000","mapping_epoch":81,"log_start":"136'4847","ondisk_log_start":"136'4847","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":39780352,"num_objects":607,"num_object_clones":0,"num_object_copies":1821,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4856,"num_write_kb":38848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"92","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.206452+0000","last_change":"2020-08-07T15:09:30.205290+0000","last_active":"2020-08-07T15:09:30.206452+0000","last_peered":"2020-08-07T15:09:30.206452+0000","last_clean":"2020-08-07T15:09:30.206452+0000","last_became_active":"2020-08-07T15:09:30.204655+0000","last_became_peered":"2020-08-07T15:09:30.204655+0000","last_unstale":"2020-08-07T15:09:30.206452+0000","last_undegraded":"2020-08-07T15:09:30.206452+0000","last_fullsized":"2020-08-07T15:09:30.206452+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:05.865996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:05.865996+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"135'5128","reported_seq":"6972","reported_epoch":"136","state":"active+remapped+backfill_toofull","last_fresh":"2020-08-07T15:09:31.518434+0000","last_change":"2020-08-07T15:09:28.235048+0000","last_active":"2020-08-07T15:09:31.518434+0000","last_peered":"2020-08-07T15:09:31.518434+0000","last_clean":"2020-08-07T15:09:25.573158+0000","last_became_active":"2020-08-07T15:09:28.212799+0000","last_became_peered":"2020-08-07T15:09:28.212799+0000","last_unstale":"2020-08-07T15:09:31.518434+0000","last_undegraded":"2020-08-07T15:09:31.518434+0000","last_fullsized":"2020-08-07T15:09:31.518434+0000","mapping_epoch":132,"log_start":"135'5121","ondisk_log_start":"135'5121","created":81,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":42008576,"num_objects":641,"num_object_clones":0,"num_object_copies":1923,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":641,"num_objects_unfound":0,"num_objects_dirty":641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5128,"num_write_kb":41024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":862,"num_bytes_recovered":56459264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,6],"avail_no_missing":["5","1","6"],"object_location_counts":[{"shards":"1,5,6","objects":641}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"169","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.503478+0000","last_change":"2020-08-07T15:09:08.399507+0000","last_active":"2020-08-07T15:09:31.503478+0000","last_peered":"2020-08-07T15:09:31.503478+0000","last_clean":"2020-08-07T15:09:31.503478+0000","last_became_active":"2020-08-07T15:08:41.689435+0000","last_became_peered":"2020-08-07T15:08:41.689435+0000","last_unstale":"2020-08-07T15:09:31.503478+0000","last_undegraded":"2020-08-07T15:09:31.503478+0000","last_fullsized":"2020-08-07T15:09:31.503478+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:08.399439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:08.399439+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"137'5480","reported_seq":"5544","reported_epoch":"137","state":"active+clean","last_fresh":"2020-08-07T15:09:32.709539+0000","last_change":"2020-08-07T15:08:24.554472+0000","last_active":"2020-08-07T15:09:32.709539+0000","last_peered":"2020-08-07T15:09:32.709539+0000","last_clean":"2020-08-07T15:09:32.709539+0000","last_became_active":"2020-08-07T15:08:24.554021+0000","last_became_peered":"2020-08-07T15:08:24.554021+0000","last_unstale":"2020-08-07T15:09:32.709539+0000","last_undegraded":"2020-08-07T15:09:32.709539+0000","last_fullsized":"2020-08-07T15:09:32.709539+0000","mapping_epoch":81,"log_start":"137'5472","ondisk_log_start":"137'5472","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44892160,"num_objects":685,"num_object_clones":0,"num_object_copies":2055,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5480,"num_write_kb":43840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"55","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199904+0000","last_change":"2020-08-07T15:09:09.868283+0000","last_active":"2020-08-07T15:09:30.199904+0000","last_peered":"2020-08-07T15:09:30.199904+0000","last_clean":"2020-08-07T15:09:30.199904+0000","last_became_active":"2020-08-07T15:08:46.710059+0000","last_became_peered":"2020-08-07T15:08:46.710059+0000","last_unstale":"2020-08-07T15:09:30.199904+0000","last_undegraded":"2020-08-07T15:09:30.199904+0000","last_fullsized":"2020-08-07T15:09:30.199904+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.868198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.868198+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"136'4984","reported_seq":"5050","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.679946+0000","last_change":"2020-08-07T15:09:26.248248+0000","last_active":"2020-08-07T15:09:31.679946+0000","last_peered":"2020-08-07T15:09:31.679946+0000","last_clean":"2020-08-07T15:09:31.679946+0000","last_became_active":"2020-08-07T15:08:24.552127+0000","last_became_peered":"2020-08-07T15:08:24.552127+0000","last_unstale":"2020-08-07T15:09:31.679946+0000","last_undegraded":"2020-08-07T15:09:31.679946+0000","last_fullsized":"2020-08-07T15:09:31.679946+0000","mapping_epoch":81,"log_start":"136'4974","ondisk_log_start":"136'4974","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"130'4592","last_scrub_stamp":"2020-08-07T15:09:26.248174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:26.248174+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":40828928,"num_objects":623,"num_object_clones":0,"num_object_copies":1869,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4984,"num_write_kb":39872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"137'4864","reported_seq":"4931","reported_epoch":"137","state":"active+clean","last_fresh":"2020-08-07T15:09:32.446888+0000","last_change":"2020-08-07T15:08:46.710643+0000","last_active":"2020-08-07T15:09:32.446888+0000","last_peered":"2020-08-07T15:09:32.446888+0000","last_clean":"2020-08-07T15:09:32.446888+0000","last_became_active":"2020-08-07T15:08:46.710100+0000","last_became_peered":"2020-08-07T15:08:46.710100+0000","last_unstale":"2020-08-07T15:09:32.446888+0000","last_undegraded":"2020-08-07T15:09:32.446888+0000","last_fullsized":"2020-08-07T15:09:32.446888+0000","mapping_epoch":98,"log_start":"137'4856","ondisk_log_start":"137'4856","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":39845888,"num_objects":608,"num_object_clones":0,"num_object_copies":1824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4864,"num_write_kb":38912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"77","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.200940+0000","last_change":"2020-08-07T15:09:08.653773+0000","last_active":"2020-08-07T15:09:30.200940+0000","last_peered":"2020-08-07T15:09:30.200940+0000","last_clean":"2020-08-07T15:09:30.200940+0000","last_became_active":"2020-08-07T15:08:41.677944+0000","last_became_peered":"2020-08-07T15:08:41.677944+0000","last_unstale":"2020-08-07T15:09:30.200940+0000","last_undegraded":"2020-08-07T15:09:30.200940+0000","last_fullsized":"2020-08-07T15:09:30.200940+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:08.653722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:08.653722+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":645054464,"num_objects":9844,"num_object_clones":0,"num_object_copies":29532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":641,"num_objects_unfound":0,"num_objects_dirty":9844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78742,"num_write_kb":629936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2275,"num_bytes_recovered":148971520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2024382464,"data_stored":2024382464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":63,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542185,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1356196,"kb_used_data":307612,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93015644,"statfs":{"total":96636764160,"available":95248019456,"internally_reserved":1073741824,"allocated":314994688,"data_stored":314185744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.95099999999999996,"15min":0.91200000000000003},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.1739999999999999,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.0149999999999999,"15min":0.97999999999999998},"min":{"1min":0.54300000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.47}]},{"osd":1,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.357,"15min":1.3149999999999999},"min":{"1min":0.68700000000000006,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":5.6239999999999997,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":1.105,"5min":1.0720000000000001,"15min":1.0009999999999999},"min":{"1min":0.64100000000000001,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.206,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":1.028}]},{"osd":2,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.272,"15min":1.175},"min":{"1min":0.71999999999999997,"5min":0.628,"15min":0.628},"max":{"1min":5.6619999999999999,"5min":5.6619999999999999,"15min":5.6619999999999999},"last":0.747},{"interface":"front","average":{"1min":1.446,"5min":1.26,"15min":1.165},"min":{"1min":0.66800000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":5.1319999999999997,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":0.86799999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":75.045000000000002,"5min":16.263999999999999,"15min":6.5359999999999996},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":917.38699999999994,"5min":917.38699999999994,"15min":917.38699999999994},"last":1.506},{"interface":"front","average":{"1min":19.670999999999999,"5min":4.9939999999999998,"15min":2.5600000000000001},"min":{"1min":0.80000000000000004,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":333.59199999999998,"5min":333.59199999999998,"15min":333.59199999999998},"last":1.5329999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.411,"15min":1.321},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":4.5250000000000004,"5min":4.835,"15min":4.835},"last":0.76400000000000001},{"interface":"front","average":{"1min":52.259999999999998,"5min":11.52,"15min":4.7140000000000004},"min":{"1min":0.77000000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":866.24099999999999,"5min":866.24099999999999,"15min":866.24099999999999},"last":1.5589999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.2869999999999999,"5min":23.440000000000001,"15min":32.817999999999998},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":6.6509999999999998,"5min":831.88099999999997,"15min":831.88099999999997},"last":1.0089999999999999},{"interface":"front","average":{"1min":2.2519999999999998,"5min":1.8380000000000001,"15min":1.609},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":6.6790000000000003,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":1.1060000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":13.818,"5min":4.0019999999999998,"15min":2.3460000000000001},"min":{"1min":0.82999999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":224.14500000000001,"5min":224.14500000000001,"15min":224.14500000000001},"last":1.9219999999999999},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.6839999999999999,"15min":1.5660000000000001},"min":{"1min":0.90600000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":7.4359999999999999,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":1.1439999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1246260,"kb_used_data":197676,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93125580,"statfs":{"total":96636764160,"available":95360593920,"internally_reserved":1073741824,"allocated":202420224,"data_stored":201626280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.179,"15min":1.2809999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8149999999999999,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":1.024},{"interface":"front","average":{"1min":15.239000000000001,"5min":3.9529999999999998,"15min":2.0840000000000001},"min":{"1min":0.78100000000000003,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":320.31299999999999,"5min":320.31299999999999,"15min":320.31299999999999},"last":0.95699999999999996}]},{"osd":1,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":24.821999999999999,"5min":5.7329999999999997,"15min":2.5489999999999999},"min":{"1min":0.46700000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":547.65300000000002,"5min":547.65300000000002,"15min":547.65300000000002},"last":0.69199999999999995},{"interface":"front","average":{"1min":1.3029999999999999,"5min":4.9130000000000003,"15min":2.5619999999999998},"min":{"1min":0.69799999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.5449999999999999,"5min":372.50799999999998,"15min":372.50799999999998},"last":1.002}]},{"osd":2,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.208,"15min":1.1990000000000001},"min":{"1min":0.502,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":5.2400000000000002,"5min":5.2400000000000002,"15min":5.2400000000000002},"last":0.752},{"interface":"front","average":{"1min":1.286,"5min":1.2390000000000001,"15min":1.2609999999999999},"min":{"1min":0.66800000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":6.0490000000000004,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":0.91300000000000003}]},{"osd":3,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.4370000000000001,"15min":1.496},"min":{"1min":0.878,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.0670000000000002,"5min":3.46,"15min":3.46},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3600000000000001,"15min":1.3779999999999999},"min":{"1min":0.70299999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":5.2770000000000001,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":0.98499999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.504,"15min":1.6100000000000001},"min":{"1min":0.70099999999999996,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.165,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.46,"5min":1.625,"15min":1.6559999999999999},"min":{"1min":0.96199999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":2.4279999999999999,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.044}]},{"osd":5,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.2809999999999999,"15min":1.3360000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.3069999999999999,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.6259999999999999,"15min":1.7190000000000001},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.4630000000000001,"5min":5.327,"15min":5.327},"last":1.0609999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.458,"15min":1.4410000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.4900000000000002,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.171},{"interface":"front","average":{"1min":1.5720000000000001,"5min":11.603999999999999,"15min":16.085000000000001},"min":{"1min":1.004,"5min":0.87,"15min":0.87},"max":{"1min":2.7719999999999998,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.157}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1471116,"kb_used_data":422532,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92900724,"statfs":{"total":96636764160,"available":95130341376,"internally_reserved":1073741824,"allocated":432672768,"data_stored":431878824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":20.945,"15min":29.771999999999998},"min":{"1min":0.49199999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":6.8810000000000002,"5min":497.04000000000002,"15min":497.04000000000002},"last":0.92400000000000004},{"interface":"front","average":{"1min":25.385999999999999,"5min":14.656000000000001,"15min":5.6390000000000002},"min":{"1min":0.52200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":513.78899999999999,"5min":993.84500000000003,"15min":993.84500000000003},"last":0.97999999999999998}]},{"osd":2,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.999,"15min":0.93200000000000005},"min":{"1min":0.60099999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.726,"5min":4.585,"15min":4.585},"last":1.512},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.254,"15min":1.22},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":6.4029999999999996,"5min":6.4029999999999996,"15min":6.4029999999999996},"last":1.49}]},{"osd":3,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":40.524000000000001,"5min":9.2829999999999995,"15min":3.968},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":828.31600000000003,"5min":828.31600000000003,"15min":828.31600000000003},"last":0.95799999999999996},{"interface":"front","average":{"1min":44.292000000000002,"5min":10.169,"15min":4.359},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":905.38300000000004,"5min":905.38300000000004,"15min":905.38300000000004},"last":1.022}]},{"osd":4,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":27.831,"5min":6.774,"15min":3.1139999999999999},"min":{"1min":0.79900000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":550.66800000000001,"5min":550.66800000000001,"15min":550.66800000000001},"last":1.345},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.3400000000000001,"15min":1.246},"min":{"1min":0.92100000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":5.9420000000000002,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.4450000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":8.4550000000000001,"15min":11.49},"min":{"1min":0.71199999999999997,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.7200000000000002,"5min":213.63800000000001,"15min":213.63800000000001},"last":1.325},{"interface":"front","average":{"1min":58.497,"5min":12.720000000000001,"15min":5.0330000000000004},"min":{"1min":0.76500000000000001,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":875.44600000000003,"5min":875.44600000000003,"15min":875.44600000000003},"last":1.379}]},{"osd":6,"last update":"Fri Aug 7 15:09:27 2020","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.587,"15min":1.5629999999999999},"min":{"1min":0.81000000000000005,"5min":0.754,"15min":0.754},"max":{"1min":5.7149999999999999,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":1.5489999999999999},{"interface":"front","average":{"1min":11.539,"5min":3.5979999999999999,"15min":2.1059999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":207.137,"5min":207.137,"15min":207.137},"last":1.365}]},{"osd":7,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.782,"5min":1.6879999999999999,"15min":1.639},"min":{"1min":0.94099999999999995,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":6.444,"5min":6.444,"15min":6.444},"last":1.413},{"interface":"front","average":{"1min":1.6819999999999999,"5min":1.6779999999999999,"15min":1.546},"min":{"1min":0.93300000000000005,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":5.9279999999999999,"5min":10.752000000000001,"15min":10.752000000000001},"last":1.4650000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1243132,"kb_used_data":194548,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93128708,"statfs":{"total":96636764160,"available":95363796992,"internally_reserved":1073741824,"allocated":199217152,"data_stored":198423208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":17.058,"15min":6.3369999999999997},"min":{"1min":0.61399999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.1269999999999998,"5min":845.57100000000003,"15min":845.57100000000003},"last":1.978},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.0169999999999999,"15min":0.96099999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":4.8209999999999997,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":1.6160000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.0189999999999999,"15min":0.95799999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.7719999999999998,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":1.726},{"interface":"front","average":{"1min":13.177,"5min":22.48,"15min":29.32},"min":{"1min":0.61399999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":226.911,"5min":606.18499999999995,"15min":606.18499999999995},"last":1.5289999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.0660000000000001,"15min":0.97899999999999998},"min":{"1min":0.54100000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6859999999999999,"5min":3.1779999999999999,"15min":3.1779999999999999},"last":3.141},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.3160000000000001,"15min":1.2170000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.976,"5min":4.21,"15min":4.21},"last":1.702}]},{"osd":4,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.2829999999999999,"15min":1.2050000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":8.3100000000000005,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":2.9049999999999998},{"interface":"front","average":{"1min":1.7709999999999999,"5min":1.446,"15min":1.3200000000000001},"min":{"1min":0.84599999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":8.4190000000000005,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":1.6719999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.9770000000000001,"5min":9.0380000000000003,"15min":3.8300000000000001},"min":{"1min":0.77800000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":7.8929999999999998,"5min":501.267,"15min":501.267},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.2609999999999999,"15min":1.157},"min":{"1min":0.78200000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.0939999999999999,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":2.8679999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":5.7779999999999996,"15min":7.6680000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.0059999999999998,"5min":124.738,"15min":124.738},"last":1.9279999999999999},{"interface":"front","average":{"1min":1.627,"5min":11.811,"15min":16.372},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":4.6879999999999997,"5min":294.94,"15min":294.94},"last":1.5649999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.5589999999999999,"15min":1.577},"min":{"1min":0.95399999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":4.7169999999999996,"5min":6.468,"15min":6.468},"last":2.0209999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.4410000000000001,"15min":1.4039999999999999},"min":{"1min":0.88800000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.1040000000000001,"5min":3.117,"15min":3.117},"last":3.407}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1161884,"kb_used_data":113300,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93209956,"statfs":{"total":96636764160,"available":95446994944,"internally_reserved":1073741824,"allocated":116019200,"data_stored":115225256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.6950000000000001,"5min":1.268,"15min":1.167},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":8.4930000000000003,"5min":8.4930000000000003,"15min":8.4930000000000003},"last":1.123},{"interface":"front","average":{"1min":1.357,"5min":1.113,"15min":1.0209999999999999},"min":{"1min":0.61899999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":3.5,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":1.4530000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.9390000000000001,"5min":1.5469999999999999,"15min":1.2430000000000001},"min":{"1min":0.54400000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":5.6820000000000004,"5min":18.672000000000001,"15min":18.672000000000001},"last":1.8280000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.351,"15min":1.357},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":3.585,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":2.0619999999999998}]},{"osd":3,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.268,"15min":1.198},"min":{"1min":0.72799999999999998,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.8740000000000001,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.8009999999999999},{"interface":"front","average":{"1min":34.170999999999999,"5min":8.0510000000000002,"15min":3.7210000000000001},"min":{"1min":0.68799999999999994,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":501.75599999999997,"5min":501.75599999999997,"15min":501.75599999999997},"last":1.407}]},{"osd":4,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.403,"15min":1.2070000000000001},"min":{"1min":0.65200000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":3.4180000000000001,"5min":19.783000000000001,"15min":19.783000000000001},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.546,"15min":1.6100000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":3.5569999999999999,"5min":3.5569999999999999,"15min":3.5569999999999999},"last":1.095}]},{"osd":5,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.292,"15min":1.272},"min":{"1min":0.83299999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.323,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":1.486},{"interface":"front","average":{"1min":43.396999999999998,"5min":24.298999999999999,"15min":25.116},"min":{"1min":0.99299999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":627.53300000000002,"5min":627.53300000000002,"15min":627.53300000000002},"last":1.6679999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:09:30 2020","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.538,"15min":1.3260000000000001},"min":{"1min":0.85999999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":3.452,"5min":20.035,"15min":20.035},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.498,"5min":9.8979999999999997,"15min":4.1799999999999997},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.976,"5min":751.70000000000005,"15min":751.70000000000005},"last":1.6020000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":54.209000000000003,"5min":12.595000000000001,"15min":5.359},"min":{"1min":0.84999999999999998,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":501.91699999999997,"5min":501.91699999999997,"15min":501.91699999999997},"last":1.288},{"interface":"front","average":{"1min":1.665,"5min":1.919,"15min":1.637},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":3.133,"5min":35.646999999999998,"15min":35.646999999999998},"last":1.637}]}]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1243008,"kb_used_data":194424,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93128832,"statfs":{"total":96636764160,"available":95363923968,"internally_reserved":1073741824,"allocated":199090176,"data_stored":198288736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":1.341,"15min":1.282},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.097,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":1.5389999999999999},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.4530000000000001,"15min":1.575},"min":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.3039999999999998,"5min":13.067,"15min":13.067},"last":1.2709999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":4.5860000000000003,"15min":2.794},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.9910000000000001,"5min":270.68700000000001,"15min":270.68700000000001},"last":2.847},{"interface":"front","average":{"1min":1.4379999999999999,"5min":2.0390000000000001,"15min":2.0209999999999999},"min":{"1min":0.55100000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":7.0940000000000003,"5min":14.349,"15min":14.349},"last":0.76800000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.266,"15min":1.1870000000000001},"min":{"1min":0.71399999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.2639999999999998,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":1.1779999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":2.109,"15min":1.8260000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.661,"5min":51.530000000000001,"15min":51.530000000000001},"last":0.88200000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":10.773999999999999,"15min":4.7220000000000004},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.1360000000000001,"5min":506.21300000000002,"15min":506.21300000000002},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.276,"5min":25.469000000000001,"15min":29.888999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.8940000000000001,"5min":677.88599999999997,"15min":677.88599999999997},"last":1.242}]},{"osd":5,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.5920000000000001,"15min":1.542},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.9710000000000001,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.8009999999999999,"15min":1.7789999999999999},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":3.044,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":1.3200000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.6259999999999999,"15min":1.5629999999999999},"min":{"1min":0.89800000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.9359999999999999,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":1.4279999999999999},{"interface":"front","average":{"1min":21.766999999999999,"5min":5.9390000000000001,"15min":3.3039999999999998},"min":{"1min":0.89900000000000002,"5min":0.876,"15min":0.876},"max":{"1min":387.911,"5min":387.911,"15min":387.911},"last":0.98299999999999998}]},{"osd":7,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.9330000000000001,"15min":2.085},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.0110000000000001,"5min":5.734,"15min":5.734},"last":1.093},{"interface":"front","average":{"1min":1.325,"5min":1.9750000000000001,"15min":1.976},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.169,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":1.2090000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1359000,"kb_used_data":310416,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93012840,"statfs":{"total":96636764160,"available":95245148160,"internally_reserved":1073741824,"allocated":317865984,"data_stored":317064544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":1.0720000000000001,"15min":1.1439999999999999},"min":{"1min":0.52900000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4390000000000001,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.143,"5min":1.0069999999999999,"15min":0.90100000000000002},"min":{"1min":0.498,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":4.907,"5min":6.3970000000000002,"15min":6.3970000000000002},"last":2.665}]},{"osd":1,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":23.940999999999999,"5min":5.8209999999999997,"15min":2.843},"min":{"1min":0.59599999999999997,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":320.67000000000002,"5min":320.67000000000002,"15min":320.67000000000002},"last":1.8109999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":7.5700000000000003,"15min":3.4409999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.784,"5min":689.846,"15min":689.846},"last":1.6719999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.3999999999999999,"15min":1.4339999999999999},"min":{"1min":0.77600000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.0899999999999999,"5min":6.8730000000000002,"15min":6.8730000000000002},"last":2.5489999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.516,"15min":1.5149999999999999},"min":{"1min":0.81699999999999995,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.3159999999999998,"5min":7.1360000000000001,"15min":7.1360000000000001},"last":2.5990000000000002}]},{"osd":3,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":3.681,"15min":2.2320000000000002},"min":{"1min":0.78300000000000003,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.8149999999999999,"5min":247.202,"15min":247.202},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.3720000000000001,"15min":1.2909999999999999},"min":{"1min":0.68500000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":4.931,"5min":6.431,"15min":6.431},"last":0.75800000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":10.57,"15min":4.6079999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.431,"5min":990.71600000000001,"15min":990.71600000000001},"last":1.6020000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.6100000000000001,"15min":1.621},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":3.3519999999999999,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":0.84099999999999997}]},{"osd":6,"last update":"Fri Aug 7 15:09:28 2020","interfaces":[{"interface":"back","average":{"1min":1.6799999999999999,"5min":1.7509999999999999,"15min":1.7769999999999999},"min":{"1min":0.75800000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":5.0149999999999997,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":2.7000000000000002},{"interface":"front","average":{"1min":1.498,"5min":12.069000000000001,"15min":16.762},"min":{"1min":0.83299999999999996,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":3.3530000000000002,"5min":389.14600000000002,"15min":389.14600000000002},"last":1.145}]},{"osd":7,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":3.8300000000000001,"15min":2.2149999999999999},"min":{"1min":0.753,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":4.8710000000000004,"5min":252.798,"15min":252.798},"last":2.7290000000000001},{"interface":"front","average":{"1min":1.262,"5min":10.175000000000001,"15min":4.4889999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.3050000000000002,"5min":501.93000000000001,"15min":501.93000000000001},"last":1.5489999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1280984,"kb_used_data":232400,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93090856,"statfs":{"total":96636764160,"available":95325036544,"internally_reserved":1073741824,"allocated":237977600,"data_stored":237176160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":1.012,"15min":1.0549999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.651,"5min":2.766,"15min":2.766},"last":1.165},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91600000000000004,"15min":0.84199999999999997},"min":{"1min":0.45500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.3340000000000001,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":0.94099999999999995}]},{"osd":1,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.4540000000000002,"5min":10.521000000000001,"15min":4.1260000000000003},"min":{"1min":0.56699999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":29.283000000000001,"5min":787.22900000000004,"15min":787.22900000000004},"last":139.88200000000001},{"interface":"front","average":{"1min":8.0939999999999994,"5min":2.4969999999999999,"15min":1.571},"min":{"1min":0.60999999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":116.548,"5min":116.548,"15min":116.548},"last":1.276}]},{"osd":2,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":5.9340000000000002,"15min":8.1240000000000006},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.78,"5min":168.56899999999999,"15min":168.56899999999999},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.119,"5min":20.361000000000001,"15min":28.873999999999999},"min":{"1min":0.60899999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.665,"5min":676.27099999999996,"15min":676.27099999999996},"last":1.0089999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.6739999999999999,"5min":1.5069999999999999,"15min":1.2470000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":29.129000000000001,"5min":29.129000000000001,"15min":29.129000000000001},"last":0.98899999999999999},{"interface":"front","average":{"1min":3.0219999999999998,"5min":9.9459999999999997,"15min":4.1239999999999997},"min":{"1min":0.70799999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":29.103000000000002,"5min":712.98699999999997,"15min":712.98699999999997},"last":1.534}]},{"osd":4,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":49.854999999999997,"5min":40.755000000000003,"15min":47.512999999999998},"min":{"1min":0.61899999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":971.67999999999995,"5min":971.67999999999995,"15min":971.67999999999995},"last":1.661},{"interface":"front","average":{"1min":1.236,"5min":1.264,"15min":1.292},"min":{"1min":0.64900000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.5960000000000001,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":1.421}]},{"osd":6,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":91.870999999999995,"5min":48.862000000000002,"15min":49.893000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":966.60900000000004,"5min":966.60900000000004,"15min":966.60900000000004},"last":1.45},{"interface":"front","average":{"1min":28.396999999999998,"5min":21.527000000000001,"15min":24.52},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":501.79000000000002,"5min":517.26700000000005,"15min":517.26700000000005},"last":1.5029999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.341,"15min":1.319},"min":{"1min":0.77500000000000002,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.8940000000000001,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.893},{"interface":"front","average":{"1min":27.206,"5min":6.6630000000000003,"15min":3.2650000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":515.09100000000001,"5min":515.09100000000001,"15min":515.09100000000001},"last":1.9239999999999999}]}]}]}} 2020-08-07T15:09:34.005 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:09:34.031 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:09:34.032 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:09:34.032 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:09:34.033 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:09:34.033 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:09:34.033 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:09:34.033 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:09:34.033 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:09:34.034 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:09:34.090 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:34.087+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:34.166 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:34.161+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:34.178 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:34.178 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:34.195 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:09:34.268 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:34.267+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:34.283 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 68 16 82308 82292 9.45362 11.4062 0.00821133 0.0132172 2020-08-07T15:09:34.380 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:34.375+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:34.426 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:34.427 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:34.435 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:09:34.479 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:34.478+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:34.491 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:09:34.495 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:09:34.496 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:09:34.513 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2020-08-07T15:09:33.439329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":645054464,"num_objects":9846,"num_object_clones":0,"num_object_copies":29538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":641,"num_objects_unfound":0,"num_objects_dirty":9846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78742,"num_write_kb":629936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2275,"num_bytes_recovered":148971520,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":135,"acting":135,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":10361580,"kb_used_data":1972908,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":744613140,"statfs":{"total":773094113280,"available":762483855360,"internally_reserved":8589934592,"allocated":2020257792,"data_stored":2013868752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46776320,"num_objects":715,"num_object_clones":0,"num_object_copies":2145,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":641,"num_objects_unfound":0,"num_objects_dirty":715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5710,"num_write_kb":45680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.256508"},"pg_stats":[{"pgid":"1.14","version":"16'2","reported_seq":"65","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199857+0000","last_change":"2020-08-07T15:08:42.533999+0000","last_active":"2020-08-07T15:09:30.199857+0000","last_peered":"2020-08-07T15:09:30.199857+0000","last_clean":"2020-08-07T15:09:30.199857+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:09:30.199857+0000","last_undegraded":"2020-08-07T15:09:30.199857+0000","last_fullsized":"2020-08-07T15:09:30.199857+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:42.533921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:42.533921+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"135","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.206027+0000","last_change":"2020-08-07T15:09:30.205031+0000","last_active":"2020-08-07T15:09:30.206027+0000","last_peered":"2020-08-07T15:09:30.206027+0000","last_clean":"2020-08-07T15:09:30.206027+0000","last_became_active":"2020-08-07T15:09:30.203592+0000","last_became_peered":"2020-08-07T15:09:30.203592+0000","last_unstale":"2020-08-07T15:09:30.206027+0000","last_undegraded":"2020-08-07T15:09:30.206027+0000","last_fullsized":"2020-08-07T15:09:30.206027+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.141010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.141010+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"65","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.200795+0000","last_change":"2020-08-07T15:08:55.078015+0000","last_active":"2020-08-07T15:09:30.200795+0000","last_peered":"2020-08-07T15:09:30.200795+0000","last_clean":"2020-08-07T15:09:30.200795+0000","last_became_active":"2020-08-07T15:08:41.685106+0000","last_became_peered":"2020-08-07T15:08:41.685106+0000","last_unstale":"2020-08-07T15:09:30.200795+0000","last_undegraded":"2020-08-07T15:09:30.200795+0000","last_fullsized":"2020-08-07T15:09:30.200795+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:55.077924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:55.077924+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"178","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.213864+0000","last_change":"2020-08-07T15:09:30.210669+0000","last_active":"2020-08-07T15:09:30.213864+0000","last_peered":"2020-08-07T15:09:30.213864+0000","last_clean":"2020-08-07T15:09:30.213864+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:09:30.213864+0000","last_undegraded":"2020-08-07T15:09:30.213864+0000","last_fullsized":"2020-08-07T15:09:30.213864+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.544324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.544324+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"172","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.200414+0000","last_change":"2020-08-07T15:08:51.599032+0000","last_active":"2020-08-07T15:09:30.200414+0000","last_peered":"2020-08-07T15:09:30.200414+0000","last_clean":"2020-08-07T15:09:30.200414+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:09:30.200414+0000","last_undegraded":"2020-08-07T15:09:30.200414+0000","last_fullsized":"2020-08-07T15:09:30.200414+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:51.598947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.598947+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"131","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.503476+0000","last_change":"2020-08-07T15:08:11.840861+0000","last_active":"2020-08-07T15:09:31.503476+0000","last_peered":"2020-08-07T15:09:31.503476+0000","last_clean":"2020-08-07T15:09:31.503476+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:09:31.503476+0000","last_undegraded":"2020-08-07T15:09:31.503476+0000","last_fullsized":"2020-08-07T15:09:31.503476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:11.840807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:11.840807+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"49","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.284670+0000","last_change":"2020-08-07T15:09:30.283753+0000","last_active":"2020-08-07T15:09:30.284670+0000","last_peered":"2020-08-07T15:09:30.284670+0000","last_clean":"2020-08-07T15:09:30.284670+0000","last_became_active":"2020-08-07T15:09:30.283176+0000","last_became_peered":"2020-08-07T15:09:30.283176+0000","last_unstale":"2020-08-07T15:09:30.284670+0000","last_undegraded":"2020-08-07T15:09:30.284670+0000","last_fullsized":"2020-08-07T15:09:30.284670+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:25.606757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:25.606757+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"136'4608","reported_seq":"4671","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.580740+0000","last_change":"2020-08-07T15:08:46.715869+0000","last_active":"2020-08-07T15:09:31.580740+0000","last_peered":"2020-08-07T15:09:31.580740+0000","last_clean":"2020-08-07T15:09:31.580740+0000","last_became_active":"2020-08-07T15:08:46.708112+0000","last_became_peered":"2020-08-07T15:08:46.708112+0000","last_unstale":"2020-08-07T15:09:31.580740+0000","last_undegraded":"2020-08-07T15:09:31.580740+0000","last_fullsized":"2020-08-07T15:09:31.580740+0000","mapping_epoch":98,"log_start":"135'4600","ondisk_log_start":"135'4600","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":37748736,"num_objects":576,"num_object_clones":0,"num_object_copies":1728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4608,"num_write_kb":36864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"133","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.194839+0000","last_change":"2020-08-07T15:08:40.057705+0000","last_active":"2020-08-07T15:09:30.194839+0000","last_peered":"2020-08-07T15:09:30.194839+0000","last_clean":"2020-08-07T15:09:30.194839+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:09:30.194839+0000","last_undegraded":"2020-08-07T15:09:30.194839+0000","last_fullsized":"2020-08-07T15:09:30.194839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:40.057622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:40.057622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"90","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.200120+0000","last_change":"2020-08-07T15:08:53.756554+0000","last_active":"2020-08-07T15:09:30.200120+0000","last_peered":"2020-08-07T15:09:30.200120+0000","last_clean":"2020-08-07T15:09:30.200120+0000","last_became_active":"2020-08-07T15:08:46.710404+0000","last_became_peered":"2020-08-07T15:08:46.710404+0000","last_unstale":"2020-08-07T15:09:30.200120+0000","last_undegraded":"2020-08-07T15:09:30.200120+0000","last_fullsized":"2020-08-07T15:09:30.200120+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:53.756343+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:53.756343+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"136'5176","reported_seq":"7001","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.717108+0000","last_change":"2020-08-07T15:09:27.180782+0000","last_active":"2020-08-07T15:09:31.717108+0000","last_peered":"2020-08-07T15:09:31.717108+0000","last_clean":"2020-08-07T15:09:31.717108+0000","last_became_active":"2020-08-07T15:09:27.180158+0000","last_became_peered":"2020-08-07T15:09:27.180158+0000","last_unstale":"2020-08-07T15:09:31.717108+0000","last_undegraded":"2020-08-07T15:09:31.717108+0000","last_fullsized":"2020-08-07T15:09:31.717108+0000","mapping_epoch":131,"log_start":"136'5169","ondisk_log_start":"136'5169","created":81,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":42401792,"num_objects":647,"num_object_clones":0,"num_object_copies":1941,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5176,"num_write_kb":41408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":875,"num_bytes_recovered":57344000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"130","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199794+0000","last_change":"2020-08-07T15:08:27.906894+0000","last_active":"2020-08-07T15:09:30.199794+0000","last_peered":"2020-08-07T15:09:30.199794+0000","last_clean":"2020-08-07T15:09:30.199794+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:09:30.199794+0000","last_undegraded":"2020-08-07T15:09:30.199794+0000","last_fullsized":"2020-08-07T15:09:30.199794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:27.906822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:27.906822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"170","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.503477+0000","last_change":"2020-08-07T15:09:30.202363+0000","last_active":"2020-08-07T15:09:31.503477+0000","last_peered":"2020-08-07T15:09:31.503477+0000","last_clean":"2020-08-07T15:09:31.503477+0000","last_became_active":"2020-08-07T15:09:30.201829+0000","last_became_peered":"2020-08-07T15:09:30.201829+0000","last_unstale":"2020-08-07T15:09:31.503477+0000","last_undegraded":"2020-08-07T15:09:31.503477+0000","last_fullsized":"2020-08-07T15:09:31.503477+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.899280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.899280+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"136'4672","reported_seq":"4735","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.621604+0000","last_change":"2020-08-07T15:08:24.549715+0000","last_active":"2020-08-07T15:09:31.621604+0000","last_peered":"2020-08-07T15:09:31.621604+0000","last_clean":"2020-08-07T15:09:31.621604+0000","last_became_active":"2020-08-07T15:08:24.549245+0000","last_became_peered":"2020-08-07T15:08:24.549245+0000","last_unstale":"2020-08-07T15:09:31.621604+0000","last_undegraded":"2020-08-07T15:09:31.621604+0000","last_fullsized":"2020-08-07T15:09:31.621604+0000","mapping_epoch":81,"log_start":"136'4666","ondisk_log_start":"136'4666","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":38273024,"num_objects":584,"num_object_clones":0,"num_object_copies":1752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4672,"num_write_kb":37376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"130","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199214+0000","last_change":"2020-08-07T15:08:13.867355+0000","last_active":"2020-08-07T15:09:30.199214+0000","last_peered":"2020-08-07T15:09:30.199214+0000","last_clean":"2020-08-07T15:09:30.199214+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:09:30.199214+0000","last_undegraded":"2020-08-07T15:09:30.199214+0000","last_fullsized":"2020-08-07T15:09:30.199214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:13.867148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:13.867148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"136","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.503099+0000","last_change":"2020-08-07T15:08:50.686578+0000","last_active":"2020-08-07T15:09:31.503099+0000","last_peered":"2020-08-07T15:09:31.503099+0000","last_clean":"2020-08-07T15:09:31.503099+0000","last_became_active":"2020-08-07T15:08:36.633406+0000","last_became_peered":"2020-08-07T15:08:36.633406+0000","last_unstale":"2020-08-07T15:09:31.503099+0000","last_undegraded":"2020-08-07T15:09:31.503099+0000","last_fullsized":"2020-08-07T15:09:31.503099+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:50.686496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:50.686496+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"136'4536","reported_seq":"4598","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.571269+0000","last_change":"2020-08-07T15:08:24.557398+0000","last_active":"2020-08-07T15:09:31.571269+0000","last_peered":"2020-08-07T15:09:31.571269+0000","last_clean":"2020-08-07T15:09:31.571269+0000","last_became_active":"2020-08-07T15:08:24.557022+0000","last_became_peered":"2020-08-07T15:08:24.557022+0000","last_unstale":"2020-08-07T15:09:31.571269+0000","last_undegraded":"2020-08-07T15:09:31.571269+0000","last_fullsized":"2020-08-07T15:09:31.571269+0000","mapping_epoch":81,"log_start":"136'4531","ondisk_log_start":"136'4531","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":37158912,"num_objects":567,"num_object_clones":0,"num_object_copies":1701,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4536,"num_write_kb":36288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"162","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.197478+0000","last_change":"2020-08-07T15:08:43.036067+0000","last_active":"2020-08-07T15:09:30.197478+0000","last_peered":"2020-08-07T15:09:30.197478+0000","last_clean":"2020-08-07T15:09:30.197478+0000","last_became_active":"2020-08-07T15:08:36.627426+0000","last_became_peered":"2020-08-07T15:08:36.627426+0000","last_unstale":"2020-08-07T15:09:30.197478+0000","last_undegraded":"2020-08-07T15:09:30.197478+0000","last_fullsized":"2020-08-07T15:09:30.197478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.035987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.035987+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"130","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.200827+0000","last_change":"2020-08-07T15:08:26.317430+0000","last_active":"2020-08-07T15:09:30.200827+0000","last_peered":"2020-08-07T15:09:30.200827+0000","last_clean":"2020-08-07T15:09:30.200827+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:09:30.200827+0000","last_undegraded":"2020-08-07T15:09:30.200827+0000","last_fullsized":"2020-08-07T15:09:30.200827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:26.317351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:26.317351+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"137'5027","reported_seq":"5088","reported_epoch":"137","state":"active+clean","last_fresh":"2020-08-07T15:09:32.697664+0000","last_change":"2020-08-07T15:08:24.552570+0000","last_active":"2020-08-07T15:09:32.697664+0000","last_peered":"2020-08-07T15:09:32.697664+0000","last_clean":"2020-08-07T15:09:32.697664+0000","last_became_active":"2020-08-07T15:08:24.552105+0000","last_became_peered":"2020-08-07T15:08:24.552105+0000","last_unstale":"2020-08-07T15:09:32.697664+0000","last_undegraded":"2020-08-07T15:09:32.697664+0000","last_fullsized":"2020-08-07T15:09:32.697664+0000","mapping_epoch":81,"log_start":"137'5023","ondisk_log_start":"137'5023","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41181184,"num_objects":629,"num_object_clones":0,"num_object_copies":1887,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5027,"num_write_kb":40216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"151","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199951+0000","last_change":"2020-08-07T15:09:27.181676+0000","last_active":"2020-08-07T15:09:30.199951+0000","last_peered":"2020-08-07T15:09:30.199951+0000","last_clean":"2020-08-07T15:09:30.199951+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:09:30.199951+0000","last_undegraded":"2020-08-07T15:09:30.199951+0000","last_fullsized":"2020-08-07T15:09:30.199951+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:05.606603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:05.606603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"178","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199259+0000","last_change":"2020-08-07T15:08:42.069637+0000","last_active":"2020-08-07T15:09:30.199259+0000","last_peered":"2020-08-07T15:09:30.199259+0000","last_clean":"2020-08-07T15:09:30.199259+0000","last_became_active":"2020-08-07T15:08:36.628365+0000","last_became_peered":"2020-08-07T15:08:36.628365+0000","last_unstale":"2020-08-07T15:09:30.199259+0000","last_undegraded":"2020-08-07T15:09:30.199259+0000","last_fullsized":"2020-08-07T15:09:30.199259+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:42.069585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:42.069585+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"136'5016","reported_seq":"5075","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.699090+0000","last_change":"2020-08-07T15:08:24.550789+0000","last_active":"2020-08-07T15:09:31.699090+0000","last_peered":"2020-08-07T15:09:31.699090+0000","last_clean":"2020-08-07T15:09:31.699090+0000","last_became_active":"2020-08-07T15:08:24.550222+0000","last_became_peered":"2020-08-07T15:08:24.550222+0000","last_unstale":"2020-08-07T15:09:31.699090+0000","last_undegraded":"2020-08-07T15:09:31.699090+0000","last_fullsized":"2020-08-07T15:09:31.699090+0000","mapping_epoch":81,"log_start":"136'5011","ondisk_log_start":"136'5011","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41091072,"num_objects":627,"num_object_clones":0,"num_object_copies":1881,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5016,"num_write_kb":40128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"130","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.201090+0000","last_change":"2020-08-07T15:08:30.339852+0000","last_active":"2020-08-07T15:09:30.201090+0000","last_peered":"2020-08-07T15:09:30.201090+0000","last_clean":"2020-08-07T15:09:30.201090+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:09:30.201090+0000","last_undegraded":"2020-08-07T15:09:30.201090+0000","last_fullsized":"2020-08-07T15:09:30.201090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:08:30.339785+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"128","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.202603+0000","last_change":"2020-08-07T15:08:38.386792+0000","last_active":"2020-08-07T15:09:30.202603+0000","last_peered":"2020-08-07T15:09:30.202603+0000","last_clean":"2020-08-07T15:09:30.202603+0000","last_became_active":"2020-08-07T15:08:36.618164+0000","last_became_peered":"2020-08-07T15:08:36.618164+0000","last_unstale":"2020-08-07T15:09:30.202603+0000","last_undegraded":"2020-08-07T15:09:30.202603+0000","last_fullsized":"2020-08-07T15:09:30.202603+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:38.386739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:38.386739+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"136'4880","reported_seq":"6034","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.635996+0000","last_change":"2020-08-07T15:09:27.199178+0000","last_active":"2020-08-07T15:09:31.635996+0000","last_peered":"2020-08-07T15:09:31.635996+0000","last_clean":"2020-08-07T15:09:31.635996+0000","last_became_active":"2020-08-07T15:09:27.183115+0000","last_became_peered":"2020-08-07T15:09:27.183115+0000","last_unstale":"2020-08-07T15:09:31.635996+0000","last_undegraded":"2020-08-07T15:09:31.635996+0000","last_fullsized":"2020-08-07T15:09:31.635996+0000","mapping_epoch":131,"log_start":"136'4872","ondisk_log_start":"136'4872","created":81,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":39976960,"num_objects":610,"num_object_clones":0,"num_object_copies":1830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4880,"num_write_kb":39040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":536,"num_bytes_recovered":35127296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"127","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199444+0000","last_change":"2020-08-07T15:08:15.887617+0000","last_active":"2020-08-07T15:09:30.199444+0000","last_peered":"2020-08-07T15:09:30.199444+0000","last_clean":"2020-08-07T15:09:30.199444+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:09:30.199444+0000","last_undegraded":"2020-08-07T15:09:30.199444+0000","last_fullsized":"2020-08-07T15:09:30.199444+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:15.887513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:15.887513+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"204","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.503550+0000","last_change":"2020-08-07T15:09:30.208873+0000","last_active":"2020-08-07T15:09:31.503550+0000","last_peered":"2020-08-07T15:09:31.503550+0000","last_clean":"2020-08-07T15:09:31.503550+0000","last_became_active":"2020-08-07T15:09:30.208345+0000","last_became_peered":"2020-08-07T15:09:30.208345+0000","last_unstale":"2020-08-07T15:09:31.503550+0000","last_undegraded":"2020-08-07T15:09:31.503550+0000","last_fullsized":"2020-08-07T15:09:31.503550+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.381325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.381325+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"136'4707","reported_seq":"4763","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.727732+0000","last_change":"2020-08-07T15:08:24.552962+0000","last_active":"2020-08-07T15:09:31.727732+0000","last_peered":"2020-08-07T15:09:31.727732+0000","last_clean":"2020-08-07T15:09:31.727732+0000","last_became_active":"2020-08-07T15:08:24.552070+0000","last_became_peered":"2020-08-07T15:08:24.552070+0000","last_unstale":"2020-08-07T15:09:31.727732+0000","last_undegraded":"2020-08-07T15:09:31.727732+0000","last_fullsized":"2020-08-07T15:09:31.727732+0000","mapping_epoch":81,"log_start":"136'4698","ondisk_log_start":"136'4698","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":38559744,"num_objects":589,"num_object_clones":0,"num_object_copies":1767,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4707,"num_write_kb":37656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"105","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.200989+0000","last_change":"2020-08-07T15:08:39.409159+0000","last_active":"2020-08-07T15:09:30.200989+0000","last_peered":"2020-08-07T15:09:30.200989+0000","last_clean":"2020-08-07T15:09:30.200989+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:09:30.200989+0000","last_undegraded":"2020-08-07T15:09:30.200989+0000","last_fullsized":"2020-08-07T15:09:30.200989+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:39.409063+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_clean_scrub_stamp":"2020-08-07T15:08:39.409063+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"136'4936","reported_seq":"4999","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.718382+0000","last_change":"2020-08-07T15:08:24.554063+0000","last_active":"2020-08-07T15:09:31.718382+0000","last_peered":"2020-08-07T15:09:31.718382+0000","last_clean":"2020-08-07T15:09:31.718382+0000","last_became_active":"2020-08-07T15:08:24.553668+0000","last_became_peered":"2020-08-07T15:08:24.553668+0000","last_unstale":"2020-08-07T15:09:31.718382+0000","last_undegraded":"2020-08-07T15:09:31.718382+0000","last_fullsized":"2020-08-07T15:09:31.718382+0000","mapping_epoch":81,"log_start":"136'4930","ondisk_log_start":"136'4930","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":40435712,"num_objects":617,"num_object_clones":0,"num_object_copies":1851,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4936,"num_write_kb":39488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"105","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.198725+0000","last_change":"2020-08-07T15:09:09.691838+0000","last_active":"2020-08-07T15:09:30.198725+0000","last_peered":"2020-08-07T15:09:30.198725+0000","last_clean":"2020-08-07T15:09:30.198725+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:09:30.198725+0000","last_undegraded":"2020-08-07T15:09:30.198725+0000","last_fullsized":"2020-08-07T15:09:30.198725+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.691753+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"136'4864","reported_seq":"4927","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.634461+0000","last_change":"2020-08-07T15:08:46.709789+0000","last_active":"2020-08-07T15:09:31.634461+0000","last_peered":"2020-08-07T15:09:31.634461+0000","last_clean":"2020-08-07T15:09:31.634461+0000","last_became_active":"2020-08-07T15:08:46.708674+0000","last_became_peered":"2020-08-07T15:08:46.708674+0000","last_unstale":"2020-08-07T15:09:31.634461+0000","last_undegraded":"2020-08-07T15:09:31.634461+0000","last_fullsized":"2020-08-07T15:09:31.634461+0000","mapping_epoch":98,"log_start":"136'4856","ondisk_log_start":"136'4856","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":39845888,"num_objects":608,"num_object_clones":0,"num_object_copies":1824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4864,"num_write_kb":38912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"169","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199320+0000","last_change":"2020-08-07T15:09:25.984108+0000","last_active":"2020-08-07T15:09:30.199320+0000","last_peered":"2020-08-07T15:09:30.199320+0000","last_clean":"2020-08-07T15:09:30.199320+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:09:30.199320+0000","last_undegraded":"2020-08-07T15:09:30.199320+0000","last_fullsized":"2020-08-07T15:09:30.199320+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:25.984018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:25.984018+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"136'5008","reported_seq":"5091","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.689702+0000","last_change":"2020-08-07T15:08:52.193226+0000","last_active":"2020-08-07T15:09:31.689702+0000","last_peered":"2020-08-07T15:09:31.689702+0000","last_clean":"2020-08-07T15:09:31.689702+0000","last_became_active":"2020-08-07T15:08:52.166336+0000","last_became_peered":"2020-08-07T15:08:52.166336+0000","last_unstale":"2020-08-07T15:09:31.689702+0000","last_undegraded":"2020-08-07T15:09:31.689702+0000","last_fullsized":"2020-08-07T15:09:31.689702+0000","mapping_epoch":104,"log_start":"136'5002","ondisk_log_start":"136'5002","created":81,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41025536,"num_objects":626,"num_object_clones":0,"num_object_copies":1878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5008,"num_write_kb":40064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":40960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"124","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199606+0000","last_change":"2020-08-07T15:08:51.513398+0000","last_active":"2020-08-07T15:09:30.199606+0000","last_peered":"2020-08-07T15:09:30.199606+0000","last_clean":"2020-08-07T15:09:30.199606+0000","last_became_active":"2020-08-07T15:08:41.680448+0000","last_became_peered":"2020-08-07T15:08:41.680448+0000","last_unstale":"2020-08-07T15:09:30.199606+0000","last_undegraded":"2020-08-07T15:09:30.199606+0000","last_fullsized":"2020-08-07T15:09:30.199606+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:51.513319+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.513319+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"136'4856","reported_seq":"4917","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.644606+0000","last_change":"2020-08-07T15:08:24.550496+0000","last_active":"2020-08-07T15:09:31.644606+0000","last_peered":"2020-08-07T15:09:31.644606+0000","last_clean":"2020-08-07T15:09:31.644606+0000","last_became_active":"2020-08-07T15:08:24.549868+0000","last_became_peered":"2020-08-07T15:08:24.549868+0000","last_unstale":"2020-08-07T15:09:31.644606+0000","last_undegraded":"2020-08-07T15:09:31.644606+0000","last_fullsized":"2020-08-07T15:09:31.644606+0000","mapping_epoch":81,"log_start":"136'4847","ondisk_log_start":"136'4847","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":39780352,"num_objects":607,"num_object_clones":0,"num_object_copies":1821,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4856,"num_write_kb":38848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"92","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.206452+0000","last_change":"2020-08-07T15:09:30.205290+0000","last_active":"2020-08-07T15:09:30.206452+0000","last_peered":"2020-08-07T15:09:30.206452+0000","last_clean":"2020-08-07T15:09:30.206452+0000","last_became_active":"2020-08-07T15:09:30.204655+0000","last_became_peered":"2020-08-07T15:09:30.204655+0000","last_unstale":"2020-08-07T15:09:30.206452+0000","last_undegraded":"2020-08-07T15:09:30.206452+0000","last_fullsized":"2020-08-07T15:09:30.206452+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:05.865996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:05.865996+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"135'5128","reported_seq":"6972","reported_epoch":"136","state":"active+remapped+backfill_toofull","last_fresh":"2020-08-07T15:09:31.518434+0000","last_change":"2020-08-07T15:09:28.235048+0000","last_active":"2020-08-07T15:09:31.518434+0000","last_peered":"2020-08-07T15:09:31.518434+0000","last_clean":"2020-08-07T15:09:25.573158+0000","last_became_active":"2020-08-07T15:09:28.212799+0000","last_became_peered":"2020-08-07T15:09:28.212799+0000","last_unstale":"2020-08-07T15:09:31.518434+0000","last_undegraded":"2020-08-07T15:09:31.518434+0000","last_fullsized":"2020-08-07T15:09:31.518434+0000","mapping_epoch":132,"log_start":"135'5121","ondisk_log_start":"135'5121","created":81,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":42008576,"num_objects":641,"num_object_clones":0,"num_object_copies":1923,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":641,"num_objects_unfound":0,"num_objects_dirty":641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5128,"num_write_kb":41024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":862,"num_bytes_recovered":56459264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,6],"avail_no_missing":["5","1","6"],"object_location_counts":[{"shards":"1,5,6","objects":641}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"169","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.503478+0000","last_change":"2020-08-07T15:09:08.399507+0000","last_active":"2020-08-07T15:09:31.503478+0000","last_peered":"2020-08-07T15:09:31.503478+0000","last_clean":"2020-08-07T15:09:31.503478+0000","last_became_active":"2020-08-07T15:08:41.689435+0000","last_became_peered":"2020-08-07T15:08:41.689435+0000","last_unstale":"2020-08-07T15:09:31.503478+0000","last_undegraded":"2020-08-07T15:09:31.503478+0000","last_fullsized":"2020-08-07T15:09:31.503478+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:08.399439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:08.399439+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"137'5480","reported_seq":"5544","reported_epoch":"137","state":"active+clean","last_fresh":"2020-08-07T15:09:32.709539+0000","last_change":"2020-08-07T15:08:24.554472+0000","last_active":"2020-08-07T15:09:32.709539+0000","last_peered":"2020-08-07T15:09:32.709539+0000","last_clean":"2020-08-07T15:09:32.709539+0000","last_became_active":"2020-08-07T15:08:24.554021+0000","last_became_peered":"2020-08-07T15:08:24.554021+0000","last_unstale":"2020-08-07T15:09:32.709539+0000","last_undegraded":"2020-08-07T15:09:32.709539+0000","last_fullsized":"2020-08-07T15:09:32.709539+0000","mapping_epoch":81,"log_start":"137'5472","ondisk_log_start":"137'5472","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44892160,"num_objects":685,"num_object_clones":0,"num_object_copies":2055,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5480,"num_write_kb":43840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"55","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199904+0000","last_change":"2020-08-07T15:09:09.868283+0000","last_active":"2020-08-07T15:09:30.199904+0000","last_peered":"2020-08-07T15:09:30.199904+0000","last_clean":"2020-08-07T15:09:30.199904+0000","last_became_active":"2020-08-07T15:08:46.710059+0000","last_became_peered":"2020-08-07T15:08:46.710059+0000","last_unstale":"2020-08-07T15:09:30.199904+0000","last_undegraded":"2020-08-07T15:09:30.199904+0000","last_fullsized":"2020-08-07T15:09:30.199904+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.868198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.868198+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"136'4984","reported_seq":"5050","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.679946+0000","last_change":"2020-08-07T15:09:26.248248+0000","last_active":"2020-08-07T15:09:31.679946+0000","last_peered":"2020-08-07T15:09:31.679946+0000","last_clean":"2020-08-07T15:09:31.679946+0000","last_became_active":"2020-08-07T15:08:24.552127+0000","last_became_peered":"2020-08-07T15:08:24.552127+0000","last_unstale":"2020-08-07T15:09:31.679946+0000","last_undegraded":"2020-08-07T15:09:31.679946+0000","last_fullsized":"2020-08-07T15:09:31.679946+0000","mapping_epoch":81,"log_start":"136'4974","ondisk_log_start":"136'4974","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"130'4592","last_scrub_stamp":"2020-08-07T15:09:26.248174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:26.248174+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":40828928,"num_objects":623,"num_object_clones":0,"num_object_copies":1869,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4984,"num_write_kb":39872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"137'4864","reported_seq":"4931","reported_epoch":"137","state":"active+clean","last_fresh":"2020-08-07T15:09:32.446888+0000","last_change":"2020-08-07T15:08:46.710643+0000","last_active":"2020-08-07T15:09:32.446888+0000","last_peered":"2020-08-07T15:09:32.446888+0000","last_clean":"2020-08-07T15:09:32.446888+0000","last_became_active":"2020-08-07T15:08:46.710100+0000","last_became_peered":"2020-08-07T15:08:46.710100+0000","last_unstale":"2020-08-07T15:09:32.446888+0000","last_undegraded":"2020-08-07T15:09:32.446888+0000","last_fullsized":"2020-08-07T15:09:32.446888+0000","mapping_epoch":98,"log_start":"137'4856","ondisk_log_start":"137'4856","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":39845888,"num_objects":608,"num_object_clones":0,"num_object_copies":1824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4864,"num_write_kb":38912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"77","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.200940+0000","last_change":"2020-08-07T15:09:08.653773+0000","last_active":"2020-08-07T15:09:30.200940+0000","last_peered":"2020-08-07T15:09:30.200940+0000","last_clean":"2020-08-07T15:09:30.200940+0000","last_became_active":"2020-08-07T15:08:41.677944+0000","last_became_peered":"2020-08-07T15:08:41.677944+0000","last_unstale":"2020-08-07T15:09:30.200940+0000","last_undegraded":"2020-08-07T15:09:30.200940+0000","last_fullsized":"2020-08-07T15:09:30.200940+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:08.653722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:08.653722+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":645054464,"num_objects":9844,"num_object_clones":0,"num_object_copies":29532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":641,"num_objects_unfound":0,"num_objects_dirty":9844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78742,"num_write_kb":629936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2275,"num_bytes_recovered":148971520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2024382464,"data_stored":2024382464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":63,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542185,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1356196,"kb_used_data":307612,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93015644,"statfs":{"total":96636764160,"available":95248019456,"internally_reserved":1073741824,"allocated":314994688,"data_stored":314185744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.95099999999999996,"15min":0.91200000000000003},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.1739999999999999,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.0149999999999999,"15min":0.97999999999999998},"min":{"1min":0.54300000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.47}]},{"osd":1,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.357,"15min":1.3149999999999999},"min":{"1min":0.68700000000000006,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":5.6239999999999997,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":1.105,"5min":1.0720000000000001,"15min":1.0009999999999999},"min":{"1min":0.64100000000000001,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.206,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":1.028}]},{"osd":2,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.272,"15min":1.175},"min":{"1min":0.71999999999999997,"5min":0.628,"15min":0.628},"max":{"1min":5.6619999999999999,"5min":5.6619999999999999,"15min":5.6619999999999999},"last":0.747},{"interface":"front","average":{"1min":1.446,"5min":1.26,"15min":1.165},"min":{"1min":0.66800000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":5.1319999999999997,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":0.86799999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":75.045000000000002,"5min":16.263999999999999,"15min":6.5359999999999996},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":917.38699999999994,"5min":917.38699999999994,"15min":917.38699999999994},"last":1.506},{"interface":"front","average":{"1min":19.670999999999999,"5min":4.9939999999999998,"15min":2.5600000000000001},"min":{"1min":0.80000000000000004,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":333.59199999999998,"5min":333.59199999999998,"15min":333.59199999999998},"last":1.5329999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.411,"15min":1.321},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":4.5250000000000004,"5min":4.835,"15min":4.835},"last":0.76400000000000001},{"interface":"front","average":{"1min":52.259999999999998,"5min":11.52,"15min":4.7140000000000004},"min":{"1min":0.77000000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":866.24099999999999,"5min":866.24099999999999,"15min":866.24099999999999},"last":1.5589999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.2869999999999999,"5min":23.440000000000001,"15min":32.817999999999998},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":6.6509999999999998,"5min":831.88099999999997,"15min":831.88099999999997},"last":1.0089999999999999},{"interface":"front","average":{"1min":2.2519999999999998,"5min":1.8380000000000001,"15min":1.609},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":6.6790000000000003,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":1.1060000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":13.818,"5min":4.0019999999999998,"15min":2.3460000000000001},"min":{"1min":0.82999999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":224.14500000000001,"5min":224.14500000000001,"15min":224.14500000000001},"last":1.9219999999999999},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.6839999999999999,"15min":1.5660000000000001},"min":{"1min":0.90600000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":7.4359999999999999,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":1.1439999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1246260,"kb_used_data":197676,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93125580,"statfs":{"total":96636764160,"available":95360593920,"internally_reserved":1073741824,"allocated":202420224,"data_stored":201626280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.179,"15min":1.2809999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8149999999999999,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":1.024},{"interface":"front","average":{"1min":15.239000000000001,"5min":3.9529999999999998,"15min":2.0840000000000001},"min":{"1min":0.78100000000000003,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":320.31299999999999,"5min":320.31299999999999,"15min":320.31299999999999},"last":0.95699999999999996}]},{"osd":1,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":24.821999999999999,"5min":5.7329999999999997,"15min":2.5489999999999999},"min":{"1min":0.46700000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":547.65300000000002,"5min":547.65300000000002,"15min":547.65300000000002},"last":0.69199999999999995},{"interface":"front","average":{"1min":1.3029999999999999,"5min":4.9130000000000003,"15min":2.5619999999999998},"min":{"1min":0.69799999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.5449999999999999,"5min":372.50799999999998,"15min":372.50799999999998},"last":1.002}]},{"osd":2,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.208,"15min":1.1990000000000001},"min":{"1min":0.502,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":5.2400000000000002,"5min":5.2400000000000002,"15min":5.2400000000000002},"last":0.752},{"interface":"front","average":{"1min":1.286,"5min":1.2390000000000001,"15min":1.2609999999999999},"min":{"1min":0.66800000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":6.0490000000000004,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":0.91300000000000003}]},{"osd":3,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.4370000000000001,"15min":1.496},"min":{"1min":0.878,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.0670000000000002,"5min":3.46,"15min":3.46},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3600000000000001,"15min":1.3779999999999999},"min":{"1min":0.70299999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":5.2770000000000001,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":0.98499999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.504,"15min":1.6100000000000001},"min":{"1min":0.70099999999999996,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.165,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.46,"5min":1.625,"15min":1.6559999999999999},"min":{"1min":0.96199999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":2.4279999999999999,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.044}]},{"osd":5,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.2809999999999999,"15min":1.3360000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.3069999999999999,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.6259999999999999,"15min":1.7190000000000001},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.4630000000000001,"5min":5.327,"15min":5.327},"last":1.0609999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.458,"15min":1.4410000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.4900000000000002,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.171},{"interface":"front","average":{"1min":1.5720000000000001,"5min":11.603999999999999,"15min":16.085000000000001},"min":{"1min":1.004,"5min":0.87,"15min":0.87},"max":{"1min":2.7719999999999998,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.157}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1471116,"kb_used_data":422532,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92900724,"statfs":{"total":96636764160,"available":95130341376,"internally_reserved":1073741824,"allocated":432672768,"data_stored":431878824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":20.945,"15min":29.771999999999998},"min":{"1min":0.49199999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":6.8810000000000002,"5min":497.04000000000002,"15min":497.04000000000002},"last":0.92400000000000004},{"interface":"front","average":{"1min":25.385999999999999,"5min":14.656000000000001,"15min":5.6390000000000002},"min":{"1min":0.52200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":513.78899999999999,"5min":993.84500000000003,"15min":993.84500000000003},"last":0.97999999999999998}]},{"osd":2,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.999,"15min":0.93200000000000005},"min":{"1min":0.60099999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.726,"5min":4.585,"15min":4.585},"last":1.512},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.254,"15min":1.22},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":6.4029999999999996,"5min":6.4029999999999996,"15min":6.4029999999999996},"last":1.49}]},{"osd":3,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":40.524000000000001,"5min":9.2829999999999995,"15min":3.968},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":828.31600000000003,"5min":828.31600000000003,"15min":828.31600000000003},"last":0.95799999999999996},{"interface":"front","average":{"1min":44.292000000000002,"5min":10.169,"15min":4.359},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":905.38300000000004,"5min":905.38300000000004,"15min":905.38300000000004},"last":1.022}]},{"osd":4,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":27.831,"5min":6.774,"15min":3.1139999999999999},"min":{"1min":0.79900000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":550.66800000000001,"5min":550.66800000000001,"15min":550.66800000000001},"last":1.345},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.3400000000000001,"15min":1.246},"min":{"1min":0.92100000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":5.9420000000000002,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.4450000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":8.4550000000000001,"15min":11.49},"min":{"1min":0.71199999999999997,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.7200000000000002,"5min":213.63800000000001,"15min":213.63800000000001},"last":1.325},{"interface":"front","average":{"1min":58.497,"5min":12.720000000000001,"15min":5.0330000000000004},"min":{"1min":0.76500000000000001,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":875.44600000000003,"5min":875.44600000000003,"15min":875.44600000000003},"last":1.379}]},{"osd":6,"last update":"Fri Aug 7 15:09:27 2020","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.587,"15min":1.5629999999999999},"min":{"1min":0.81000000000000005,"5min":0.754,"15min":0.754},"max":{"1min":5.7149999999999999,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":1.5489999999999999},{"interface":"front","average":{"1min":11.539,"5min":3.5979999999999999,"15min":2.1059999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":207.137,"5min":207.137,"15min":207.137},"last":1.365}]},{"osd":7,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.782,"5min":1.6879999999999999,"15min":1.639},"min":{"1min":0.94099999999999995,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":6.444,"5min":6.444,"15min":6.444},"last":1.413},{"interface":"front","average":{"1min":1.6819999999999999,"5min":1.6779999999999999,"15min":1.546},"min":{"1min":0.93300000000000005,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":5.9279999999999999,"5min":10.752000000000001,"15min":10.752000000000001},"last":1.4650000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1243132,"kb_used_data":194548,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93128708,"statfs":{"total":96636764160,"available":95363796992,"internally_reserved":1073741824,"allocated":199217152,"data_stored":198423208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":17.058,"15min":6.3369999999999997},"min":{"1min":0.61399999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.1269999999999998,"5min":845.57100000000003,"15min":845.57100000000003},"last":1.978},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.0169999999999999,"15min":0.96099999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":4.8209999999999997,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":1.6160000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.0189999999999999,"15min":0.95799999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.7719999999999998,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":1.726},{"interface":"front","average":{"1min":13.177,"5min":22.48,"15min":29.32},"min":{"1min":0.61399999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":226.911,"5min":606.18499999999995,"15min":606.18499999999995},"last":1.5289999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.0660000000000001,"15min":0.97899999999999998},"min":{"1min":0.54100000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6859999999999999,"5min":3.1779999999999999,"15min":3.1779999999999999},"last":3.141},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.3160000000000001,"15min":1.2170000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.976,"5min":4.21,"15min":4.21},"last":1.702}]},{"osd":4,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.2829999999999999,"15min":1.2050000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":8.3100000000000005,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":2.9049999999999998},{"interface":"front","average":{"1min":1.7709999999999999,"5min":1.446,"15min":1.3200000000000001},"min":{"1min":0.84599999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":8.4190000000000005,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":1.6719999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.9770000000000001,"5min":9.0380000000000003,"15min":3.8300000000000001},"min":{"1min":0.77800000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":7.8929999999999998,"5min":501.267,"15min":501.267},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.2609999999999999,"15min":1.157},"min":{"1min":0.78200000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.0939999999999999,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":2.8679999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":5.7779999999999996,"15min":7.6680000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.0059999999999998,"5min":124.738,"15min":124.738},"last":1.9279999999999999},{"interface":"front","average":{"1min":1.627,"5min":11.811,"15min":16.372},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":4.6879999999999997,"5min":294.94,"15min":294.94},"last":1.5649999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.5589999999999999,"15min":1.577},"min":{"1min":0.95399999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":4.7169999999999996,"5min":6.468,"15min":6.468},"last":2.0209999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.4410000000000001,"15min":1.4039999999999999},"min":{"1min":0.88800000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.1040000000000001,"5min":3.117,"15min":3.117},"last":3.407}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1161884,"kb_used_data":113300,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93209956,"statfs":{"total":96636764160,"available":95446994944,"internally_reserved":1073741824,"allocated":116019200,"data_stored":115225256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.6950000000000001,"5min":1.268,"15min":1.167},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":8.4930000000000003,"5min":8.4930000000000003,"15min":8.4930000000000003},"last":1.123},{"interface":"front","average":{"1min":1.357,"5min":1.113,"15min":1.0209999999999999},"min":{"1min":0.61899999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":3.5,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":1.4530000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.9390000000000001,"5min":1.5469999999999999,"15min":1.2430000000000001},"min":{"1min":0.54400000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":5.6820000000000004,"5min":18.672000000000001,"15min":18.672000000000001},"last":1.8280000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.351,"15min":1.357},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":3.585,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":2.0619999999999998}]},{"osd":3,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.268,"15min":1.198},"min":{"1min":0.72799999999999998,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.8740000000000001,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.8009999999999999},{"interface":"front","average":{"1min":34.170999999999999,"5min":8.0510000000000002,"15min":3.7210000000000001},"min":{"1min":0.68799999999999994,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":501.75599999999997,"5min":501.75599999999997,"15min":501.75599999999997},"last":1.407}]},{"osd":4,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.403,"15min":1.2070000000000001},"min":{"1min":0.65200000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":3.4180000000000001,"5min":19.783000000000001,"15min":19.783000000000001},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.546,"15min":1.6100000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":3.5569999999999999,"5min":3.5569999999999999,"15min":3.5569999999999999},"last":1.095}]},{"osd":5,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.292,"15min":1.272},"min":{"1min":0.83299999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.323,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":1.486},{"interface":"front","average":{"1min":43.396999999999998,"5min":24.298999999999999,"15min":25.116},"min":{"1min":0.99299999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":627.53300000000002,"5min":627.53300000000002,"15min":627.53300000000002},"last":1.6679999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:09:30 2020","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.538,"15min":1.3260000000000001},"min":{"1min":0.85999999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":3.452,"5min":20.035,"15min":20.035},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.498,"5min":9.8979999999999997,"15min":4.1799999999999997},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.976,"5min":751.70000000000005,"15min":751.70000000000005},"last":1.6020000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":54.209000000000003,"5min":12.595000000000001,"15min":5.359},"min":{"1min":0.84999999999999998,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":501.91699999999997,"5min":501.91699999999997,"15min":501.91699999999997},"last":1.288},{"interface":"front","average":{"1min":1.665,"5min":1.919,"15min":1.637},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":3.133,"5min":35.646999999999998,"15min":35.646999999999998},"last":1.637}]}]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1243008,"kb_used_data":194424,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93128832,"statfs":{"total":96636764160,"available":95363923968,"internally_reserved":1073741824,"allocated":199090176,"data_stored":198288736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":1.341,"15min":1.282},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.097,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":1.5389999999999999},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.4530000000000001,"15min":1.575},"min":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.3039999999999998,"5min":13.067,"15min":13.067},"last":1.2709999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":4.5860000000000003,"15min":2.794},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.9910000000000001,"5min":270.68700000000001,"15min":270.68700000000001},"last":2.847},{"interface":"front","average":{"1min":1.4379999999999999,"5min":2.0390000000000001,"15min":2.0209999999999999},"min":{"1min":0.55100000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":7.0940000000000003,"5min":14.349,"15min":14.349},"last":0.76800000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.266,"15min":1.1870000000000001},"min":{"1min":0.71399999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.2639999999999998,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":1.1779999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":2.109,"15min":1.8260000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.661,"5min":51.530000000000001,"15min":51.530000000000001},"last":0.88200000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":10.773999999999999,"15min":4.7220000000000004},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.1360000000000001,"5min":506.21300000000002,"15min":506.21300000000002},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.276,"5min":25.469000000000001,"15min":29.888999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.8940000000000001,"5min":677.88599999999997,"15min":677.88599999999997},"last":1.242}]},{"osd":5,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.5920000000000001,"15min":1.542},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.9710000000000001,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.8009999999999999,"15min":1.7789999999999999},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":3.044,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":1.3200000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.6259999999999999,"15min":1.5629999999999999},"min":{"1min":0.89800000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.9359999999999999,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":1.4279999999999999},{"interface":"front","average":{"1min":21.766999999999999,"5min":5.9390000000000001,"15min":3.3039999999999998},"min":{"1min":0.89900000000000002,"5min":0.876,"15min":0.876},"max":{"1min":387.911,"5min":387.911,"15min":387.911},"last":0.98299999999999998}]},{"osd":7,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.9330000000000001,"15min":2.085},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.0110000000000001,"5min":5.734,"15min":5.734},"last":1.093},{"interface":"front","average":{"1min":1.325,"5min":1.9750000000000001,"15min":1.976},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.169,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":1.2090000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1359000,"kb_used_data":310416,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93012840,"statfs":{"total":96636764160,"available":95245148160,"internally_reserved":1073741824,"allocated":317865984,"data_stored":317064544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":1.0720000000000001,"15min":1.1439999999999999},"min":{"1min":0.52900000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4390000000000001,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.143,"5min":1.0069999999999999,"15min":0.90100000000000002},"min":{"1min":0.498,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":4.907,"5min":6.3970000000000002,"15min":6.3970000000000002},"last":2.665}]},{"osd":1,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":23.940999999999999,"5min":5.8209999999999997,"15min":2.843},"min":{"1min":0.59599999999999997,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":320.67000000000002,"5min":320.67000000000002,"15min":320.67000000000002},"last":1.8109999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":7.5700000000000003,"15min":3.4409999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.784,"5min":689.846,"15min":689.846},"last":1.6719999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.3999999999999999,"15min":1.4339999999999999},"min":{"1min":0.77600000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.0899999999999999,"5min":6.8730000000000002,"15min":6.8730000000000002},"last":2.5489999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.516,"15min":1.5149999999999999},"min":{"1min":0.81699999999999995,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.3159999999999998,"5min":7.1360000000000001,"15min":7.1360000000000001},"last":2.5990000000000002}]},{"osd":3,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":3.681,"15min":2.2320000000000002},"min":{"1min":0.78300000000000003,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.8149999999999999,"5min":247.202,"15min":247.202},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.3720000000000001,"15min":1.2909999999999999},"min":{"1min":0.68500000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":4.931,"5min":6.431,"15min":6.431},"last":0.75800000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":10.57,"15min":4.6079999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.431,"5min":990.71600000000001,"15min":990.71600000000001},"last":1.6020000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.6100000000000001,"15min":1.621},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":3.3519999999999999,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":0.84099999999999997}]},{"osd":6,"last update":"Fri Aug 7 15:09:28 2020","interfaces":[{"interface":"back","average":{"1min":1.6799999999999999,"5min":1.7509999999999999,"15min":1.7769999999999999},"min":{"1min":0.75800000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":5.0149999999999997,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":2.7000000000000002},{"interface":"front","average":{"1min":1.498,"5min":12.069000000000001,"15min":16.762},"min":{"1min":0.83299999999999996,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":3.3530000000000002,"5min":389.14600000000002,"15min":389.14600000000002},"last":1.145}]},{"osd":7,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":3.8300000000000001,"15min":2.2149999999999999},"min":{"1min":0.753,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":4.8710000000000004,"5min":252.798,"15min":252.798},"last":2.7290000000000001},{"interface":"front","average":{"1min":1.262,"5min":10.175000000000001,"15min":4.4889999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.3050000000000002,"5min":501.93000000000001,"15min":501.93000000000001},"last":1.5489999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1280984,"kb_used_data":232400,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93090856,"statfs":{"total":96636764160,"available":95325036544,"internally_reserved":1073741824,"allocated":237977600,"data_stored":237176160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":1.012,"15min":1.0549999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.651,"5min":2.766,"15min":2.766},"last":1.165},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91600000000000004,"15min":0.84199999999999997},"min":{"1min":0.45500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.3340000000000001,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":0.94099999999999995}]},{"osd":1,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.4540000000000002,"5min":10.521000000000001,"15min":4.1260000000000003},"min":{"1min":0.56699999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":29.283000000000001,"5min":787.22900000000004,"15min":787.22900000000004},"last":139.88200000000001},{"interface":"front","average":{"1min":8.0939999999999994,"5min":2.4969999999999999,"15min":1.571},"min":{"1min":0.60999999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":116.548,"5min":116.548,"15min":116.548},"last":1.276}]},{"osd":2,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":5.9340000000000002,"15min":8.1240000000000006},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.78,"5min":168.56899999999999,"15min":168.56899999999999},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.119,"5min":20.361000000000001,"15min":28.873999999999999},"min":{"1min":0.60899999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.665,"5min":676.27099999999996,"15min":676.27099999999996},"last":1.0089999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.6739999999999999,"5min":1.5069999999999999,"15min":1.2470000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":29.129000000000001,"5min":29.129000000000001,"15min":29.129000000000001},"last":0.98899999999999999},{"interface":"front","average":{"1min":3.0219999999999998,"5min":9.9459999999999997,"15min":4.1239999999999997},"min":{"1min":0.70799999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":29.103000000000002,"5min":712.98699999999997,"15min":712.98699999999997},"last":1.534}]},{"osd":4,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":49.854999999999997,"5min":40.755000000000003,"15min":47.512999999999998},"min":{"1min":0.61899999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":971.67999999999995,"5min":971.67999999999995,"15min":971.67999999999995},"last":1.661},{"interface":"front","average":{"1min":1.236,"5min":1.264,"15min":1.292},"min":{"1min":0.64900000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.5960000000000001,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":1.421}]},{"osd":6,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":91.870999999999995,"5min":48.862000000000002,"15min":49.893000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":966.60900000000004,"5min":966.60900000000004,"15min":966.60900000000004},"last":1.45},{"interface":"front","average":{"1min":28.396999999999998,"5min":21.527000000000001,"15min":24.52},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":501.79000000000002,"5min":517.26700000000005,"15min":517.26700000000005},"last":1.5029999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.341,"15min":1.319},"min":{"1min":0.77500000000000002,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.8940000000000001,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.893},{"interface":"front","average":{"1min":27.206,"5min":6.6630000000000003,"15min":3.2650000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":515.09100000000001,"5min":515.09100000000001,"15min":515.09100000000001},"last":1.9239999999999999}]}]}]}} 2020-08-07T15:09:34.532 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2020-08-07T15:09:34.533 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:09:34.534 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_1 2020-08-07T15:09:34.534 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:09:34.607 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:34.604+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:34.670 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:34.671 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:34.679 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:09:34.725 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:34.720+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:34.787 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:34.785+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:34.894 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:34.889+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:34.916 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:34.916 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:34.935 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:09:35.017 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:34.993+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:35.036 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:09:35.041 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:09:35.059 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2020-08-07T15:09:34.649354+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":645054464,"num_objects":9846,"num_object_clones":0,"num_object_copies":29538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":641,"num_objects_unfound":0,"num_objects_dirty":9846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78742,"num_write_kb":629936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2275,"num_bytes_recovered":148971520,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":135,"acting":135,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":10361580,"kb_used_data":1972908,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":744613140,"statfs":{"total":773094113280,"available":762483855360,"internally_reserved":8589934592,"allocated":2020257792,"data_stored":2013868752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46776320,"num_objects":715,"num_object_clones":0,"num_object_copies":2145,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":641,"num_objects_unfound":0,"num_objects_dirty":715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5710,"num_write_kb":45680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.462114"},"pg_stats":[{"pgid":"1.14","version":"16'2","reported_seq":"65","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199857+0000","last_change":"2020-08-07T15:08:42.533999+0000","last_active":"2020-08-07T15:09:30.199857+0000","last_peered":"2020-08-07T15:09:30.199857+0000","last_clean":"2020-08-07T15:09:30.199857+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:09:30.199857+0000","last_undegraded":"2020-08-07T15:09:30.199857+0000","last_fullsized":"2020-08-07T15:09:30.199857+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:42.533921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:42.533921+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"135","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.206027+0000","last_change":"2020-08-07T15:09:30.205031+0000","last_active":"2020-08-07T15:09:30.206027+0000","last_peered":"2020-08-07T15:09:30.206027+0000","last_clean":"2020-08-07T15:09:30.206027+0000","last_became_active":"2020-08-07T15:09:30.203592+0000","last_became_peered":"2020-08-07T15:09:30.203592+0000","last_unstale":"2020-08-07T15:09:30.206027+0000","last_undegraded":"2020-08-07T15:09:30.206027+0000","last_fullsized":"2020-08-07T15:09:30.206027+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.141010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.141010+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"65","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.200795+0000","last_change":"2020-08-07T15:08:55.078015+0000","last_active":"2020-08-07T15:09:30.200795+0000","last_peered":"2020-08-07T15:09:30.200795+0000","last_clean":"2020-08-07T15:09:30.200795+0000","last_became_active":"2020-08-07T15:08:41.685106+0000","last_became_peered":"2020-08-07T15:08:41.685106+0000","last_unstale":"2020-08-07T15:09:30.200795+0000","last_undegraded":"2020-08-07T15:09:30.200795+0000","last_fullsized":"2020-08-07T15:09:30.200795+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:55.077924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:55.077924+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"178","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.213864+0000","last_change":"2020-08-07T15:09:30.210669+0000","last_active":"2020-08-07T15:09:30.213864+0000","last_peered":"2020-08-07T15:09:30.213864+0000","last_clean":"2020-08-07T15:09:30.213864+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:09:30.213864+0000","last_undegraded":"2020-08-07T15:09:30.213864+0000","last_fullsized":"2020-08-07T15:09:30.213864+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.544324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.544324+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"172","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.200414+0000","last_change":"2020-08-07T15:08:51.599032+0000","last_active":"2020-08-07T15:09:30.200414+0000","last_peered":"2020-08-07T15:09:30.200414+0000","last_clean":"2020-08-07T15:09:30.200414+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:09:30.200414+0000","last_undegraded":"2020-08-07T15:09:30.200414+0000","last_fullsized":"2020-08-07T15:09:30.200414+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:51.598947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.598947+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"131","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.503476+0000","last_change":"2020-08-07T15:08:11.840861+0000","last_active":"2020-08-07T15:09:31.503476+0000","last_peered":"2020-08-07T15:09:31.503476+0000","last_clean":"2020-08-07T15:09:31.503476+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:09:31.503476+0000","last_undegraded":"2020-08-07T15:09:31.503476+0000","last_fullsized":"2020-08-07T15:09:31.503476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:11.840807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:11.840807+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"49","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.284670+0000","last_change":"2020-08-07T15:09:30.283753+0000","last_active":"2020-08-07T15:09:30.284670+0000","last_peered":"2020-08-07T15:09:30.284670+0000","last_clean":"2020-08-07T15:09:30.284670+0000","last_became_active":"2020-08-07T15:09:30.283176+0000","last_became_peered":"2020-08-07T15:09:30.283176+0000","last_unstale":"2020-08-07T15:09:30.284670+0000","last_undegraded":"2020-08-07T15:09:30.284670+0000","last_fullsized":"2020-08-07T15:09:30.284670+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:25.606757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:25.606757+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"136'4608","reported_seq":"4671","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.580740+0000","last_change":"2020-08-07T15:08:46.715869+0000","last_active":"2020-08-07T15:09:31.580740+0000","last_peered":"2020-08-07T15:09:31.580740+0000","last_clean":"2020-08-07T15:09:31.580740+0000","last_became_active":"2020-08-07T15:08:46.708112+0000","last_became_peered":"2020-08-07T15:08:46.708112+0000","last_unstale":"2020-08-07T15:09:31.580740+0000","last_undegraded":"2020-08-07T15:09:31.580740+0000","last_fullsized":"2020-08-07T15:09:31.580740+0000","mapping_epoch":98,"log_start":"135'4600","ondisk_log_start":"135'4600","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":37748736,"num_objects":576,"num_object_clones":0,"num_object_copies":1728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4608,"num_write_kb":36864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"133","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.194839+0000","last_change":"2020-08-07T15:08:40.057705+0000","last_active":"2020-08-07T15:09:30.194839+0000","last_peered":"2020-08-07T15:09:30.194839+0000","last_clean":"2020-08-07T15:09:30.194839+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:09:30.194839+0000","last_undegraded":"2020-08-07T15:09:30.194839+0000","last_fullsized":"2020-08-07T15:09:30.194839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:40.057622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:40.057622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"90","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.200120+0000","last_change":"2020-08-07T15:08:53.756554+0000","last_active":"2020-08-07T15:09:30.200120+0000","last_peered":"2020-08-07T15:09:30.200120+0000","last_clean":"2020-08-07T15:09:30.200120+0000","last_became_active":"2020-08-07T15:08:46.710404+0000","last_became_peered":"2020-08-07T15:08:46.710404+0000","last_unstale":"2020-08-07T15:09:30.200120+0000","last_undegraded":"2020-08-07T15:09:30.200120+0000","last_fullsized":"2020-08-07T15:09:30.200120+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:53.756343+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:53.756343+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"136'5176","reported_seq":"7001","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.717108+0000","last_change":"2020-08-07T15:09:27.180782+0000","last_active":"2020-08-07T15:09:31.717108+0000","last_peered":"2020-08-07T15:09:31.717108+0000","last_clean":"2020-08-07T15:09:31.717108+0000","last_became_active":"2020-08-07T15:09:27.180158+0000","last_became_peered":"2020-08-07T15:09:27.180158+0000","last_unstale":"2020-08-07T15:09:31.717108+0000","last_undegraded":"2020-08-07T15:09:31.717108+0000","last_fullsized":"2020-08-07T15:09:31.717108+0000","mapping_epoch":131,"log_start":"136'5169","ondisk_log_start":"136'5169","created":81,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":42401792,"num_objects":647,"num_object_clones":0,"num_object_copies":1941,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5176,"num_write_kb":41408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":875,"num_bytes_recovered":57344000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"130","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199794+0000","last_change":"2020-08-07T15:08:27.906894+0000","last_active":"2020-08-07T15:09:30.199794+0000","last_peered":"2020-08-07T15:09:30.199794+0000","last_clean":"2020-08-07T15:09:30.199794+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:09:30.199794+0000","last_undegraded":"2020-08-07T15:09:30.199794+0000","last_fullsized":"2020-08-07T15:09:30.199794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:27.906822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:27.906822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"170","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.503477+0000","last_change":"2020-08-07T15:09:30.202363+0000","last_active":"2020-08-07T15:09:31.503477+0000","last_peered":"2020-08-07T15:09:31.503477+0000","last_clean":"2020-08-07T15:09:31.503477+0000","last_became_active":"2020-08-07T15:09:30.201829+0000","last_became_peered":"2020-08-07T15:09:30.201829+0000","last_unstale":"2020-08-07T15:09:31.503477+0000","last_undegraded":"2020-08-07T15:09:31.503477+0000","last_fullsized":"2020-08-07T15:09:31.503477+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.899280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.899280+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"136'4672","reported_seq":"4735","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.621604+0000","last_change":"2020-08-07T15:08:24.549715+0000","last_active":"2020-08-07T15:09:31.621604+0000","last_peered":"2020-08-07T15:09:31.621604+0000","last_clean":"2020-08-07T15:09:31.621604+0000","last_became_active":"2020-08-07T15:08:24.549245+0000","last_became_peered":"2020-08-07T15:08:24.549245+0000","last_unstale":"2020-08-07T15:09:31.621604+0000","last_undegraded":"2020-08-07T15:09:31.621604+0000","last_fullsized":"2020-08-07T15:09:31.621604+0000","mapping_epoch":81,"log_start":"136'4666","ondisk_log_start":"136'4666","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":38273024,"num_objects":584,"num_object_clones":0,"num_object_copies":1752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4672,"num_write_kb":37376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"130","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199214+0000","last_change":"2020-08-07T15:08:13.867355+0000","last_active":"2020-08-07T15:09:30.199214+0000","last_peered":"2020-08-07T15:09:30.199214+0000","last_clean":"2020-08-07T15:09:30.199214+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:09:30.199214+0000","last_undegraded":"2020-08-07T15:09:30.199214+0000","last_fullsized":"2020-08-07T15:09:30.199214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:13.867148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:13.867148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"136","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.503099+0000","last_change":"2020-08-07T15:08:50.686578+0000","last_active":"2020-08-07T15:09:31.503099+0000","last_peered":"2020-08-07T15:09:31.503099+0000","last_clean":"2020-08-07T15:09:31.503099+0000","last_became_active":"2020-08-07T15:08:36.633406+0000","last_became_peered":"2020-08-07T15:08:36.633406+0000","last_unstale":"2020-08-07T15:09:31.503099+0000","last_undegraded":"2020-08-07T15:09:31.503099+0000","last_fullsized":"2020-08-07T15:09:31.503099+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:50.686496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:50.686496+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"136'4536","reported_seq":"4598","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.571269+0000","last_change":"2020-08-07T15:08:24.557398+0000","last_active":"2020-08-07T15:09:31.571269+0000","last_peered":"2020-08-07T15:09:31.571269+0000","last_clean":"2020-08-07T15:09:31.571269+0000","last_became_active":"2020-08-07T15:08:24.557022+0000","last_became_peered":"2020-08-07T15:08:24.557022+0000","last_unstale":"2020-08-07T15:09:31.571269+0000","last_undegraded":"2020-08-07T15:09:31.571269+0000","last_fullsized":"2020-08-07T15:09:31.571269+0000","mapping_epoch":81,"log_start":"136'4531","ondisk_log_start":"136'4531","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":37158912,"num_objects":567,"num_object_clones":0,"num_object_copies":1701,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4536,"num_write_kb":36288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"162","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.197478+0000","last_change":"2020-08-07T15:08:43.036067+0000","last_active":"2020-08-07T15:09:30.197478+0000","last_peered":"2020-08-07T15:09:30.197478+0000","last_clean":"2020-08-07T15:09:30.197478+0000","last_became_active":"2020-08-07T15:08:36.627426+0000","last_became_peered":"2020-08-07T15:08:36.627426+0000","last_unstale":"2020-08-07T15:09:30.197478+0000","last_undegraded":"2020-08-07T15:09:30.197478+0000","last_fullsized":"2020-08-07T15:09:30.197478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.035987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.035987+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"130","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.200827+0000","last_change":"2020-08-07T15:08:26.317430+0000","last_active":"2020-08-07T15:09:30.200827+0000","last_peered":"2020-08-07T15:09:30.200827+0000","last_clean":"2020-08-07T15:09:30.200827+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:09:30.200827+0000","last_undegraded":"2020-08-07T15:09:30.200827+0000","last_fullsized":"2020-08-07T15:09:30.200827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:26.317351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:26.317351+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"137'5027","reported_seq":"5088","reported_epoch":"137","state":"active+clean","last_fresh":"2020-08-07T15:09:32.697664+0000","last_change":"2020-08-07T15:08:24.552570+0000","last_active":"2020-08-07T15:09:32.697664+0000","last_peered":"2020-08-07T15:09:32.697664+0000","last_clean":"2020-08-07T15:09:32.697664+0000","last_became_active":"2020-08-07T15:08:24.552105+0000","last_became_peered":"2020-08-07T15:08:24.552105+0000","last_unstale":"2020-08-07T15:09:32.697664+0000","last_undegraded":"2020-08-07T15:09:32.697664+0000","last_fullsized":"2020-08-07T15:09:32.697664+0000","mapping_epoch":81,"log_start":"137'5023","ondisk_log_start":"137'5023","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41181184,"num_objects":629,"num_object_clones":0,"num_object_copies":1887,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5027,"num_write_kb":40216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"151","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199951+0000","last_change":"2020-08-07T15:09:27.181676+0000","last_active":"2020-08-07T15:09:30.199951+0000","last_peered":"2020-08-07T15:09:30.199951+0000","last_clean":"2020-08-07T15:09:30.199951+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:09:30.199951+0000","last_undegraded":"2020-08-07T15:09:30.199951+0000","last_fullsized":"2020-08-07T15:09:30.199951+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:05.606603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:05.606603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"178","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199259+0000","last_change":"2020-08-07T15:08:42.069637+0000","last_active":"2020-08-07T15:09:30.199259+0000","last_peered":"2020-08-07T15:09:30.199259+0000","last_clean":"2020-08-07T15:09:30.199259+0000","last_became_active":"2020-08-07T15:08:36.628365+0000","last_became_peered":"2020-08-07T15:08:36.628365+0000","last_unstale":"2020-08-07T15:09:30.199259+0000","last_undegraded":"2020-08-07T15:09:30.199259+0000","last_fullsized":"2020-08-07T15:09:30.199259+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:42.069585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:42.069585+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"136'5016","reported_seq":"5075","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.699090+0000","last_change":"2020-08-07T15:08:24.550789+0000","last_active":"2020-08-07T15:09:31.699090+0000","last_peered":"2020-08-07T15:09:31.699090+0000","last_clean":"2020-08-07T15:09:31.699090+0000","last_became_active":"2020-08-07T15:08:24.550222+0000","last_became_peered":"2020-08-07T15:08:24.550222+0000","last_unstale":"2020-08-07T15:09:31.699090+0000","last_undegraded":"2020-08-07T15:09:31.699090+0000","last_fullsized":"2020-08-07T15:09:31.699090+0000","mapping_epoch":81,"log_start":"136'5011","ondisk_log_start":"136'5011","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41091072,"num_objects":627,"num_object_clones":0,"num_object_copies":1881,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5016,"num_write_kb":40128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"130","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.201090+0000","last_change":"2020-08-07T15:08:30.339852+0000","last_active":"2020-08-07T15:09:30.201090+0000","last_peered":"2020-08-07T15:09:30.201090+0000","last_clean":"2020-08-07T15:09:30.201090+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:09:30.201090+0000","last_undegraded":"2020-08-07T15:09:30.201090+0000","last_fullsized":"2020-08-07T15:09:30.201090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:08:30.339785+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"128","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.202603+0000","last_change":"2020-08-07T15:08:38.386792+0000","last_active":"2020-08-07T15:09:30.202603+0000","last_peered":"2020-08-07T15:09:30.202603+0000","last_clean":"2020-08-07T15:09:30.202603+0000","last_became_active":"2020-08-07T15:08:36.618164+0000","last_became_peered":"2020-08-07T15:08:36.618164+0000","last_unstale":"2020-08-07T15:09:30.202603+0000","last_undegraded":"2020-08-07T15:09:30.202603+0000","last_fullsized":"2020-08-07T15:09:30.202603+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:38.386739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:38.386739+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"136'4880","reported_seq":"6034","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.635996+0000","last_change":"2020-08-07T15:09:27.199178+0000","last_active":"2020-08-07T15:09:31.635996+0000","last_peered":"2020-08-07T15:09:31.635996+0000","last_clean":"2020-08-07T15:09:31.635996+0000","last_became_active":"2020-08-07T15:09:27.183115+0000","last_became_peered":"2020-08-07T15:09:27.183115+0000","last_unstale":"2020-08-07T15:09:31.635996+0000","last_undegraded":"2020-08-07T15:09:31.635996+0000","last_fullsized":"2020-08-07T15:09:31.635996+0000","mapping_epoch":131,"log_start":"136'4872","ondisk_log_start":"136'4872","created":81,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":39976960,"num_objects":610,"num_object_clones":0,"num_object_copies":1830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4880,"num_write_kb":39040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":536,"num_bytes_recovered":35127296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"127","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199444+0000","last_change":"2020-08-07T15:08:15.887617+0000","last_active":"2020-08-07T15:09:30.199444+0000","last_peered":"2020-08-07T15:09:30.199444+0000","last_clean":"2020-08-07T15:09:30.199444+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:09:30.199444+0000","last_undegraded":"2020-08-07T15:09:30.199444+0000","last_fullsized":"2020-08-07T15:09:30.199444+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:15.887513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:15.887513+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"204","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.503550+0000","last_change":"2020-08-07T15:09:30.208873+0000","last_active":"2020-08-07T15:09:31.503550+0000","last_peered":"2020-08-07T15:09:31.503550+0000","last_clean":"2020-08-07T15:09:31.503550+0000","last_became_active":"2020-08-07T15:09:30.208345+0000","last_became_peered":"2020-08-07T15:09:30.208345+0000","last_unstale":"2020-08-07T15:09:31.503550+0000","last_undegraded":"2020-08-07T15:09:31.503550+0000","last_fullsized":"2020-08-07T15:09:31.503550+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.381325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.381325+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"136'4707","reported_seq":"4763","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.727732+0000","last_change":"2020-08-07T15:08:24.552962+0000","last_active":"2020-08-07T15:09:31.727732+0000","last_peered":"2020-08-07T15:09:31.727732+0000","last_clean":"2020-08-07T15:09:31.727732+0000","last_became_active":"2020-08-07T15:08:24.552070+0000","last_became_peered":"2020-08-07T15:08:24.552070+0000","last_unstale":"2020-08-07T15:09:31.727732+0000","last_undegraded":"2020-08-07T15:09:31.727732+0000","last_fullsized":"2020-08-07T15:09:31.727732+0000","mapping_epoch":81,"log_start":"136'4698","ondisk_log_start":"136'4698","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":38559744,"num_objects":589,"num_object_clones":0,"num_object_copies":1767,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4707,"num_write_kb":37656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"105","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.200989+0000","last_change":"2020-08-07T15:08:39.409159+0000","last_active":"2020-08-07T15:09:30.200989+0000","last_peered":"2020-08-07T15:09:30.200989+0000","last_clean":"2020-08-07T15:09:30.200989+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:09:30.200989+0000","last_undegraded":"2020-08-07T15:09:30.200989+0000","last_fullsized":"2020-08-07T15:09:30.200989+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:39.409063+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_clean_scrub_stamp":"2020-08-07T15:08:39.409063+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"136'4936","reported_seq":"4999","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.718382+0000","last_change":"2020-08-07T15:08:24.554063+0000","last_active":"2020-08-07T15:09:31.718382+0000","last_peered":"2020-08-07T15:09:31.718382+0000","last_clean":"2020-08-07T15:09:31.718382+0000","last_became_active":"2020-08-07T15:08:24.553668+0000","last_became_peered":"2020-08-07T15:08:24.553668+0000","last_unstale":"2020-08-07T15:09:31.718382+0000","last_undegraded":"2020-08-07T15:09:31.718382+0000","last_fullsized":"2020-08-07T15:09:31.718382+0000","mapping_epoch":81,"log_start":"136'4930","ondisk_log_start":"136'4930","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":40435712,"num_objects":617,"num_object_clones":0,"num_object_copies":1851,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4936,"num_write_kb":39488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"105","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.198725+0000","last_change":"2020-08-07T15:09:09.691838+0000","last_active":"2020-08-07T15:09:30.198725+0000","last_peered":"2020-08-07T15:09:30.198725+0000","last_clean":"2020-08-07T15:09:30.198725+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:09:30.198725+0000","last_undegraded":"2020-08-07T15:09:30.198725+0000","last_fullsized":"2020-08-07T15:09:30.198725+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.691753+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"136'4864","reported_seq":"4927","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.634461+0000","last_change":"2020-08-07T15:08:46.709789+0000","last_active":"2020-08-07T15:09:31.634461+0000","last_peered":"2020-08-07T15:09:31.634461+0000","last_clean":"2020-08-07T15:09:31.634461+0000","last_became_active":"2020-08-07T15:08:46.708674+0000","last_became_peered":"2020-08-07T15:08:46.708674+0000","last_unstale":"2020-08-07T15:09:31.634461+0000","last_undegraded":"2020-08-07T15:09:31.634461+0000","last_fullsized":"2020-08-07T15:09:31.634461+0000","mapping_epoch":98,"log_start":"136'4856","ondisk_log_start":"136'4856","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":39845888,"num_objects":608,"num_object_clones":0,"num_object_copies":1824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4864,"num_write_kb":38912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"169","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199320+0000","last_change":"2020-08-07T15:09:25.984108+0000","last_active":"2020-08-07T15:09:30.199320+0000","last_peered":"2020-08-07T15:09:30.199320+0000","last_clean":"2020-08-07T15:09:30.199320+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:09:30.199320+0000","last_undegraded":"2020-08-07T15:09:30.199320+0000","last_fullsized":"2020-08-07T15:09:30.199320+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:25.984018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:25.984018+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"136'5008","reported_seq":"5091","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.689702+0000","last_change":"2020-08-07T15:08:52.193226+0000","last_active":"2020-08-07T15:09:31.689702+0000","last_peered":"2020-08-07T15:09:31.689702+0000","last_clean":"2020-08-07T15:09:31.689702+0000","last_became_active":"2020-08-07T15:08:52.166336+0000","last_became_peered":"2020-08-07T15:08:52.166336+0000","last_unstale":"2020-08-07T15:09:31.689702+0000","last_undegraded":"2020-08-07T15:09:31.689702+0000","last_fullsized":"2020-08-07T15:09:31.689702+0000","mapping_epoch":104,"log_start":"136'5002","ondisk_log_start":"136'5002","created":81,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41025536,"num_objects":626,"num_object_clones":0,"num_object_copies":1878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5008,"num_write_kb":40064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":40960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"124","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199606+0000","last_change":"2020-08-07T15:08:51.513398+0000","last_active":"2020-08-07T15:09:30.199606+0000","last_peered":"2020-08-07T15:09:30.199606+0000","last_clean":"2020-08-07T15:09:30.199606+0000","last_became_active":"2020-08-07T15:08:41.680448+0000","last_became_peered":"2020-08-07T15:08:41.680448+0000","last_unstale":"2020-08-07T15:09:30.199606+0000","last_undegraded":"2020-08-07T15:09:30.199606+0000","last_fullsized":"2020-08-07T15:09:30.199606+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:51.513319+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.513319+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"136'4856","reported_seq":"4917","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.644606+0000","last_change":"2020-08-07T15:08:24.550496+0000","last_active":"2020-08-07T15:09:31.644606+0000","last_peered":"2020-08-07T15:09:31.644606+0000","last_clean":"2020-08-07T15:09:31.644606+0000","last_became_active":"2020-08-07T15:08:24.549868+0000","last_became_peered":"2020-08-07T15:08:24.549868+0000","last_unstale":"2020-08-07T15:09:31.644606+0000","last_undegraded":"2020-08-07T15:09:31.644606+0000","last_fullsized":"2020-08-07T15:09:31.644606+0000","mapping_epoch":81,"log_start":"136'4847","ondisk_log_start":"136'4847","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":39780352,"num_objects":607,"num_object_clones":0,"num_object_copies":1821,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4856,"num_write_kb":38848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"92","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.206452+0000","last_change":"2020-08-07T15:09:30.205290+0000","last_active":"2020-08-07T15:09:30.206452+0000","last_peered":"2020-08-07T15:09:30.206452+0000","last_clean":"2020-08-07T15:09:30.206452+0000","last_became_active":"2020-08-07T15:09:30.204655+0000","last_became_peered":"2020-08-07T15:09:30.204655+0000","last_unstale":"2020-08-07T15:09:30.206452+0000","last_undegraded":"2020-08-07T15:09:30.206452+0000","last_fullsized":"2020-08-07T15:09:30.206452+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:05.865996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:05.865996+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"135'5128","reported_seq":"6972","reported_epoch":"136","state":"active+remapped+backfill_toofull","last_fresh":"2020-08-07T15:09:31.518434+0000","last_change":"2020-08-07T15:09:28.235048+0000","last_active":"2020-08-07T15:09:31.518434+0000","last_peered":"2020-08-07T15:09:31.518434+0000","last_clean":"2020-08-07T15:09:25.573158+0000","last_became_active":"2020-08-07T15:09:28.212799+0000","last_became_peered":"2020-08-07T15:09:28.212799+0000","last_unstale":"2020-08-07T15:09:31.518434+0000","last_undegraded":"2020-08-07T15:09:31.518434+0000","last_fullsized":"2020-08-07T15:09:31.518434+0000","mapping_epoch":132,"log_start":"135'5121","ondisk_log_start":"135'5121","created":81,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":42008576,"num_objects":641,"num_object_clones":0,"num_object_copies":1923,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":641,"num_objects_unfound":0,"num_objects_dirty":641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5128,"num_write_kb":41024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":862,"num_bytes_recovered":56459264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,6],"avail_no_missing":["5","1","6"],"object_location_counts":[{"shards":"1,5,6","objects":641}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"169","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.503478+0000","last_change":"2020-08-07T15:09:08.399507+0000","last_active":"2020-08-07T15:09:31.503478+0000","last_peered":"2020-08-07T15:09:31.503478+0000","last_clean":"2020-08-07T15:09:31.503478+0000","last_became_active":"2020-08-07T15:08:41.689435+0000","last_became_peered":"2020-08-07T15:08:41.689435+0000","last_unstale":"2020-08-07T15:09:31.503478+0000","last_undegraded":"2020-08-07T15:09:31.503478+0000","last_fullsized":"2020-08-07T15:09:31.503478+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:08.399439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:08.399439+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"137'5480","reported_seq":"5544","reported_epoch":"137","state":"active+clean","last_fresh":"2020-08-07T15:09:32.709539+0000","last_change":"2020-08-07T15:08:24.554472+0000","last_active":"2020-08-07T15:09:32.709539+0000","last_peered":"2020-08-07T15:09:32.709539+0000","last_clean":"2020-08-07T15:09:32.709539+0000","last_became_active":"2020-08-07T15:08:24.554021+0000","last_became_peered":"2020-08-07T15:08:24.554021+0000","last_unstale":"2020-08-07T15:09:32.709539+0000","last_undegraded":"2020-08-07T15:09:32.709539+0000","last_fullsized":"2020-08-07T15:09:32.709539+0000","mapping_epoch":81,"log_start":"137'5472","ondisk_log_start":"137'5472","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44892160,"num_objects":685,"num_object_clones":0,"num_object_copies":2055,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5480,"num_write_kb":43840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"55","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.199904+0000","last_change":"2020-08-07T15:09:09.868283+0000","last_active":"2020-08-07T15:09:30.199904+0000","last_peered":"2020-08-07T15:09:30.199904+0000","last_clean":"2020-08-07T15:09:30.199904+0000","last_became_active":"2020-08-07T15:08:46.710059+0000","last_became_peered":"2020-08-07T15:08:46.710059+0000","last_unstale":"2020-08-07T15:09:30.199904+0000","last_undegraded":"2020-08-07T15:09:30.199904+0000","last_fullsized":"2020-08-07T15:09:30.199904+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.868198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.868198+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"136'4984","reported_seq":"5050","reported_epoch":"136","state":"active+clean","last_fresh":"2020-08-07T15:09:31.679946+0000","last_change":"2020-08-07T15:09:26.248248+0000","last_active":"2020-08-07T15:09:31.679946+0000","last_peered":"2020-08-07T15:09:31.679946+0000","last_clean":"2020-08-07T15:09:31.679946+0000","last_became_active":"2020-08-07T15:08:24.552127+0000","last_became_peered":"2020-08-07T15:08:24.552127+0000","last_unstale":"2020-08-07T15:09:31.679946+0000","last_undegraded":"2020-08-07T15:09:31.679946+0000","last_fullsized":"2020-08-07T15:09:31.679946+0000","mapping_epoch":81,"log_start":"136'4974","ondisk_log_start":"136'4974","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"130'4592","last_scrub_stamp":"2020-08-07T15:09:26.248174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:26.248174+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":40828928,"num_objects":623,"num_object_clones":0,"num_object_copies":1869,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4984,"num_write_kb":39872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"137'4864","reported_seq":"4931","reported_epoch":"137","state":"active+clean","last_fresh":"2020-08-07T15:09:32.446888+0000","last_change":"2020-08-07T15:08:46.710643+0000","last_active":"2020-08-07T15:09:32.446888+0000","last_peered":"2020-08-07T15:09:32.446888+0000","last_clean":"2020-08-07T15:09:32.446888+0000","last_became_active":"2020-08-07T15:08:46.710100+0000","last_became_peered":"2020-08-07T15:08:46.710100+0000","last_unstale":"2020-08-07T15:09:32.446888+0000","last_undegraded":"2020-08-07T15:09:32.446888+0000","last_fullsized":"2020-08-07T15:09:32.446888+0000","mapping_epoch":98,"log_start":"137'4856","ondisk_log_start":"137'4856","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":39845888,"num_objects":608,"num_object_clones":0,"num_object_copies":1824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4864,"num_write_kb":38912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"77","reported_epoch":"135","state":"active+clean","last_fresh":"2020-08-07T15:09:30.200940+0000","last_change":"2020-08-07T15:09:08.653773+0000","last_active":"2020-08-07T15:09:30.200940+0000","last_peered":"2020-08-07T15:09:30.200940+0000","last_clean":"2020-08-07T15:09:30.200940+0000","last_became_active":"2020-08-07T15:08:41.677944+0000","last_became_peered":"2020-08-07T15:08:41.677944+0000","last_unstale":"2020-08-07T15:09:30.200940+0000","last_undegraded":"2020-08-07T15:09:30.200940+0000","last_fullsized":"2020-08-07T15:09:30.200940+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:08.653722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:08.653722+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":645054464,"num_objects":9844,"num_object_clones":0,"num_object_copies":29532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":641,"num_objects_unfound":0,"num_objects_dirty":9844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78742,"num_write_kb":629936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2275,"num_bytes_recovered":148971520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2024382464,"data_stored":2024382464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":63,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542185,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1356196,"kb_used_data":307612,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93015644,"statfs":{"total":96636764160,"available":95248019456,"internally_reserved":1073741824,"allocated":314994688,"data_stored":314185744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.95099999999999996,"15min":0.91200000000000003},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.1739999999999999,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.0149999999999999,"15min":0.97999999999999998},"min":{"1min":0.54300000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.47}]},{"osd":1,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.357,"15min":1.3149999999999999},"min":{"1min":0.68700000000000006,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":5.6239999999999997,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":1.105,"5min":1.0720000000000001,"15min":1.0009999999999999},"min":{"1min":0.64100000000000001,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.206,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":1.028}]},{"osd":2,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.272,"15min":1.175},"min":{"1min":0.71999999999999997,"5min":0.628,"15min":0.628},"max":{"1min":5.6619999999999999,"5min":5.6619999999999999,"15min":5.6619999999999999},"last":0.747},{"interface":"front","average":{"1min":1.446,"5min":1.26,"15min":1.165},"min":{"1min":0.66800000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":5.1319999999999997,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":0.86799999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":75.045000000000002,"5min":16.263999999999999,"15min":6.5359999999999996},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":917.38699999999994,"5min":917.38699999999994,"15min":917.38699999999994},"last":1.506},{"interface":"front","average":{"1min":19.670999999999999,"5min":4.9939999999999998,"15min":2.5600000000000001},"min":{"1min":0.80000000000000004,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":333.59199999999998,"5min":333.59199999999998,"15min":333.59199999999998},"last":1.5329999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.411,"15min":1.321},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":4.5250000000000004,"5min":4.835,"15min":4.835},"last":0.76400000000000001},{"interface":"front","average":{"1min":52.259999999999998,"5min":11.52,"15min":4.7140000000000004},"min":{"1min":0.77000000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":866.24099999999999,"5min":866.24099999999999,"15min":866.24099999999999},"last":1.5589999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.2869999999999999,"5min":23.440000000000001,"15min":32.817999999999998},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":6.6509999999999998,"5min":831.88099999999997,"15min":831.88099999999997},"last":1.0089999999999999},{"interface":"front","average":{"1min":2.2519999999999998,"5min":1.8380000000000001,"15min":1.609},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":6.6790000000000003,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":1.1060000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":13.818,"5min":4.0019999999999998,"15min":2.3460000000000001},"min":{"1min":0.82999999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":224.14500000000001,"5min":224.14500000000001,"15min":224.14500000000001},"last":1.9219999999999999},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.6839999999999999,"15min":1.5660000000000001},"min":{"1min":0.90600000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":7.4359999999999999,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":1.1439999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1246260,"kb_used_data":197676,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93125580,"statfs":{"total":96636764160,"available":95360593920,"internally_reserved":1073741824,"allocated":202420224,"data_stored":201626280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.179,"15min":1.2809999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8149999999999999,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":1.024},{"interface":"front","average":{"1min":15.239000000000001,"5min":3.9529999999999998,"15min":2.0840000000000001},"min":{"1min":0.78100000000000003,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":320.31299999999999,"5min":320.31299999999999,"15min":320.31299999999999},"last":0.95699999999999996}]},{"osd":1,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":24.821999999999999,"5min":5.7329999999999997,"15min":2.5489999999999999},"min":{"1min":0.46700000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":547.65300000000002,"5min":547.65300000000002,"15min":547.65300000000002},"last":0.69199999999999995},{"interface":"front","average":{"1min":1.3029999999999999,"5min":4.9130000000000003,"15min":2.5619999999999998},"min":{"1min":0.69799999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.5449999999999999,"5min":372.50799999999998,"15min":372.50799999999998},"last":1.002}]},{"osd":2,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.208,"15min":1.1990000000000001},"min":{"1min":0.502,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":5.2400000000000002,"5min":5.2400000000000002,"15min":5.2400000000000002},"last":0.752},{"interface":"front","average":{"1min":1.286,"5min":1.2390000000000001,"15min":1.2609999999999999},"min":{"1min":0.66800000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":6.0490000000000004,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":0.91300000000000003}]},{"osd":3,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.4370000000000001,"15min":1.496},"min":{"1min":0.878,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.0670000000000002,"5min":3.46,"15min":3.46},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3600000000000001,"15min":1.3779999999999999},"min":{"1min":0.70299999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":5.2770000000000001,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":0.98499999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.504,"15min":1.6100000000000001},"min":{"1min":0.70099999999999996,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.165,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.46,"5min":1.625,"15min":1.6559999999999999},"min":{"1min":0.96199999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":2.4279999999999999,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.044}]},{"osd":5,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.2809999999999999,"15min":1.3360000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.3069999999999999,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.6259999999999999,"15min":1.7190000000000001},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.4630000000000001,"5min":5.327,"15min":5.327},"last":1.0609999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.458,"15min":1.4410000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.4900000000000002,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.171},{"interface":"front","average":{"1min":1.5720000000000001,"5min":11.603999999999999,"15min":16.085000000000001},"min":{"1min":1.004,"5min":0.87,"15min":0.87},"max":{"1min":2.7719999999999998,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.157}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1471116,"kb_used_data":422532,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92900724,"statfs":{"total":96636764160,"available":95130341376,"internally_reserved":1073741824,"allocated":432672768,"data_stored":431878824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":20.945,"15min":29.771999999999998},"min":{"1min":0.49199999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":6.8810000000000002,"5min":497.04000000000002,"15min":497.04000000000002},"last":0.92400000000000004},{"interface":"front","average":{"1min":25.385999999999999,"5min":14.656000000000001,"15min":5.6390000000000002},"min":{"1min":0.52200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":513.78899999999999,"5min":993.84500000000003,"15min":993.84500000000003},"last":0.97999999999999998}]},{"osd":2,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.999,"15min":0.93200000000000005},"min":{"1min":0.60099999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.726,"5min":4.585,"15min":4.585},"last":1.512},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.254,"15min":1.22},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":6.4029999999999996,"5min":6.4029999999999996,"15min":6.4029999999999996},"last":1.49}]},{"osd":3,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":40.524000000000001,"5min":9.2829999999999995,"15min":3.968},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":828.31600000000003,"5min":828.31600000000003,"15min":828.31600000000003},"last":0.95799999999999996},{"interface":"front","average":{"1min":44.292000000000002,"5min":10.169,"15min":4.359},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":905.38300000000004,"5min":905.38300000000004,"15min":905.38300000000004},"last":1.022}]},{"osd":4,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":27.831,"5min":6.774,"15min":3.1139999999999999},"min":{"1min":0.79900000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":550.66800000000001,"5min":550.66800000000001,"15min":550.66800000000001},"last":1.345},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.3400000000000001,"15min":1.246},"min":{"1min":0.92100000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":5.9420000000000002,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.4450000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":8.4550000000000001,"15min":11.49},"min":{"1min":0.71199999999999997,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.7200000000000002,"5min":213.63800000000001,"15min":213.63800000000001},"last":1.325},{"interface":"front","average":{"1min":58.497,"5min":12.720000000000001,"15min":5.0330000000000004},"min":{"1min":0.76500000000000001,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":875.44600000000003,"5min":875.44600000000003,"15min":875.44600000000003},"last":1.379}]},{"osd":6,"last update":"Fri Aug 7 15:09:27 2020","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.587,"15min":1.5629999999999999},"min":{"1min":0.81000000000000005,"5min":0.754,"15min":0.754},"max":{"1min":5.7149999999999999,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":1.5489999999999999},{"interface":"front","average":{"1min":11.539,"5min":3.5979999999999999,"15min":2.1059999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":207.137,"5min":207.137,"15min":207.137},"last":1.365}]},{"osd":7,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.782,"5min":1.6879999999999999,"15min":1.639},"min":{"1min":0.94099999999999995,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":6.444,"5min":6.444,"15min":6.444},"last":1.413},{"interface":"front","average":{"1min":1.6819999999999999,"5min":1.6779999999999999,"15min":1.546},"min":{"1min":0.93300000000000005,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":5.9279999999999999,"5min":10.752000000000001,"15min":10.752000000000001},"last":1.4650000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1243132,"kb_used_data":194548,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93128708,"statfs":{"total":96636764160,"available":95363796992,"internally_reserved":1073741824,"allocated":199217152,"data_stored":198423208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":17.058,"15min":6.3369999999999997},"min":{"1min":0.61399999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.1269999999999998,"5min":845.57100000000003,"15min":845.57100000000003},"last":1.978},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.0169999999999999,"15min":0.96099999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":4.8209999999999997,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":1.6160000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.0189999999999999,"15min":0.95799999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.7719999999999998,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":1.726},{"interface":"front","average":{"1min":13.177,"5min":22.48,"15min":29.32},"min":{"1min":0.61399999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":226.911,"5min":606.18499999999995,"15min":606.18499999999995},"last":1.5289999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.0660000000000001,"15min":0.97899999999999998},"min":{"1min":0.54100000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6859999999999999,"5min":3.1779999999999999,"15min":3.1779999999999999},"last":3.141},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.3160000000000001,"15min":1.2170000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.976,"5min":4.21,"15min":4.21},"last":1.702}]},{"osd":4,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.2829999999999999,"15min":1.2050000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":8.3100000000000005,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":2.9049999999999998},{"interface":"front","average":{"1min":1.7709999999999999,"5min":1.446,"15min":1.3200000000000001},"min":{"1min":0.84599999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":8.4190000000000005,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":1.6719999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.9770000000000001,"5min":9.0380000000000003,"15min":3.8300000000000001},"min":{"1min":0.77800000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":7.8929999999999998,"5min":501.267,"15min":501.267},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.2609999999999999,"15min":1.157},"min":{"1min":0.78200000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.0939999999999999,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":2.8679999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":5.7779999999999996,"15min":7.6680000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.0059999999999998,"5min":124.738,"15min":124.738},"last":1.9279999999999999},{"interface":"front","average":{"1min":1.627,"5min":11.811,"15min":16.372},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":4.6879999999999997,"5min":294.94,"15min":294.94},"last":1.5649999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.5589999999999999,"15min":1.577},"min":{"1min":0.95399999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":4.7169999999999996,"5min":6.468,"15min":6.468},"last":2.0209999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.4410000000000001,"15min":1.4039999999999999},"min":{"1min":0.88800000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.1040000000000001,"5min":3.117,"15min":3.117},"last":3.407}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1161884,"kb_used_data":113300,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93209956,"statfs":{"total":96636764160,"available":95446994944,"internally_reserved":1073741824,"allocated":116019200,"data_stored":115225256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.6950000000000001,"5min":1.268,"15min":1.167},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":8.4930000000000003,"5min":8.4930000000000003,"15min":8.4930000000000003},"last":1.123},{"interface":"front","average":{"1min":1.357,"5min":1.113,"15min":1.0209999999999999},"min":{"1min":0.61899999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":3.5,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":1.4530000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.9390000000000001,"5min":1.5469999999999999,"15min":1.2430000000000001},"min":{"1min":0.54400000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":5.6820000000000004,"5min":18.672000000000001,"15min":18.672000000000001},"last":1.8280000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.351,"15min":1.357},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":3.585,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":2.0619999999999998}]},{"osd":3,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.268,"15min":1.198},"min":{"1min":0.72799999999999998,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.8740000000000001,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.8009999999999999},{"interface":"front","average":{"1min":34.170999999999999,"5min":8.0510000000000002,"15min":3.7210000000000001},"min":{"1min":0.68799999999999994,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":501.75599999999997,"5min":501.75599999999997,"15min":501.75599999999997},"last":1.407}]},{"osd":4,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.403,"15min":1.2070000000000001},"min":{"1min":0.65200000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":3.4180000000000001,"5min":19.783000000000001,"15min":19.783000000000001},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.546,"15min":1.6100000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":3.5569999999999999,"5min":3.5569999999999999,"15min":3.5569999999999999},"last":1.095}]},{"osd":5,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.292,"15min":1.272},"min":{"1min":0.83299999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.323,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":1.486},{"interface":"front","average":{"1min":43.396999999999998,"5min":24.298999999999999,"15min":25.116},"min":{"1min":0.99299999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":627.53300000000002,"5min":627.53300000000002,"15min":627.53300000000002},"last":1.6679999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:09:30 2020","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.538,"15min":1.3260000000000001},"min":{"1min":0.85999999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":3.452,"5min":20.035,"15min":20.035},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.498,"5min":9.8979999999999997,"15min":4.1799999999999997},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.976,"5min":751.70000000000005,"15min":751.70000000000005},"last":1.6020000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":54.209000000000003,"5min":12.595000000000001,"15min":5.359},"min":{"1min":0.84999999999999998,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":501.91699999999997,"5min":501.91699999999997,"15min":501.91699999999997},"last":1.288},{"interface":"front","average":{"1min":1.665,"5min":1.919,"15min":1.637},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":3.133,"5min":35.646999999999998,"15min":35.646999999999998},"last":1.637}]}]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1243008,"kb_used_data":194424,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93128832,"statfs":{"total":96636764160,"available":95363923968,"internally_reserved":1073741824,"allocated":199090176,"data_stored":198288736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":1.341,"15min":1.282},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.097,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":1.5389999999999999},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.4530000000000001,"15min":1.575},"min":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.3039999999999998,"5min":13.067,"15min":13.067},"last":1.2709999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":4.5860000000000003,"15min":2.794},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.9910000000000001,"5min":270.68700000000001,"15min":270.68700000000001},"last":2.847},{"interface":"front","average":{"1min":1.4379999999999999,"5min":2.0390000000000001,"15min":2.0209999999999999},"min":{"1min":0.55100000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":7.0940000000000003,"5min":14.349,"15min":14.349},"last":0.76800000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.266,"15min":1.1870000000000001},"min":{"1min":0.71399999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.2639999999999998,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":1.1779999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":2.109,"15min":1.8260000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.661,"5min":51.530000000000001,"15min":51.530000000000001},"last":0.88200000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":10.773999999999999,"15min":4.7220000000000004},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.1360000000000001,"5min":506.21300000000002,"15min":506.21300000000002},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.276,"5min":25.469000000000001,"15min":29.888999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.8940000000000001,"5min":677.88599999999997,"15min":677.88599999999997},"last":1.242}]},{"osd":5,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.5920000000000001,"15min":1.542},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.9710000000000001,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.8009999999999999,"15min":1.7789999999999999},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":3.044,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":1.3200000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.6259999999999999,"15min":1.5629999999999999},"min":{"1min":0.89800000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.9359999999999999,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":1.4279999999999999},{"interface":"front","average":{"1min":21.766999999999999,"5min":5.9390000000000001,"15min":3.3039999999999998},"min":{"1min":0.89900000000000002,"5min":0.876,"15min":0.876},"max":{"1min":387.911,"5min":387.911,"15min":387.911},"last":0.98299999999999998}]},{"osd":7,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.9330000000000001,"15min":2.085},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.0110000000000001,"5min":5.734,"15min":5.734},"last":1.093},{"interface":"front","average":{"1min":1.325,"5min":1.9750000000000001,"15min":1.976},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.169,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":1.2090000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1359000,"kb_used_data":310416,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93012840,"statfs":{"total":96636764160,"available":95245148160,"internally_reserved":1073741824,"allocated":317865984,"data_stored":317064544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":1.0720000000000001,"15min":1.1439999999999999},"min":{"1min":0.52900000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4390000000000001,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.143,"5min":1.0069999999999999,"15min":0.90100000000000002},"min":{"1min":0.498,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":4.907,"5min":6.3970000000000002,"15min":6.3970000000000002},"last":2.665}]},{"osd":1,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":23.940999999999999,"5min":5.8209999999999997,"15min":2.843},"min":{"1min":0.59599999999999997,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":320.67000000000002,"5min":320.67000000000002,"15min":320.67000000000002},"last":1.8109999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":7.5700000000000003,"15min":3.4409999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.784,"5min":689.846,"15min":689.846},"last":1.6719999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.3999999999999999,"15min":1.4339999999999999},"min":{"1min":0.77600000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.0899999999999999,"5min":6.8730000000000002,"15min":6.8730000000000002},"last":2.5489999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.516,"15min":1.5149999999999999},"min":{"1min":0.81699999999999995,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.3159999999999998,"5min":7.1360000000000001,"15min":7.1360000000000001},"last":2.5990000000000002}]},{"osd":3,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":3.681,"15min":2.2320000000000002},"min":{"1min":0.78300000000000003,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.8149999999999999,"5min":247.202,"15min":247.202},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.3720000000000001,"15min":1.2909999999999999},"min":{"1min":0.68500000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":4.931,"5min":6.431,"15min":6.431},"last":0.75800000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":10.57,"15min":4.6079999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.431,"5min":990.71600000000001,"15min":990.71600000000001},"last":1.6020000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.6100000000000001,"15min":1.621},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":3.3519999999999999,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":0.84099999999999997}]},{"osd":6,"last update":"Fri Aug 7 15:09:28 2020","interfaces":[{"interface":"back","average":{"1min":1.6799999999999999,"5min":1.7509999999999999,"15min":1.7769999999999999},"min":{"1min":0.75800000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":5.0149999999999997,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":2.7000000000000002},{"interface":"front","average":{"1min":1.498,"5min":12.069000000000001,"15min":16.762},"min":{"1min":0.83299999999999996,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":3.3530000000000002,"5min":389.14600000000002,"15min":389.14600000000002},"last":1.145}]},{"osd":7,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":3.8300000000000001,"15min":2.2149999999999999},"min":{"1min":0.753,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":4.8710000000000004,"5min":252.798,"15min":252.798},"last":2.7290000000000001},{"interface":"front","average":{"1min":1.262,"5min":10.175000000000001,"15min":4.4889999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.3050000000000002,"5min":501.93000000000001,"15min":501.93000000000001},"last":1.5489999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1280984,"kb_used_data":232400,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93090856,"statfs":{"total":96636764160,"available":95325036544,"internally_reserved":1073741824,"allocated":237977600,"data_stored":237176160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":1.012,"15min":1.0549999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.651,"5min":2.766,"15min":2.766},"last":1.165},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91600000000000004,"15min":0.84199999999999997},"min":{"1min":0.45500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.3340000000000001,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":0.94099999999999995}]},{"osd":1,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.4540000000000002,"5min":10.521000000000001,"15min":4.1260000000000003},"min":{"1min":0.56699999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":29.283000000000001,"5min":787.22900000000004,"15min":787.22900000000004},"last":139.88200000000001},{"interface":"front","average":{"1min":8.0939999999999994,"5min":2.4969999999999999,"15min":1.571},"min":{"1min":0.60999999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":116.548,"5min":116.548,"15min":116.548},"last":1.276}]},{"osd":2,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":5.9340000000000002,"15min":8.1240000000000006},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.78,"5min":168.56899999999999,"15min":168.56899999999999},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.119,"5min":20.361000000000001,"15min":28.873999999999999},"min":{"1min":0.60899999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.665,"5min":676.27099999999996,"15min":676.27099999999996},"last":1.0089999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.6739999999999999,"5min":1.5069999999999999,"15min":1.2470000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":29.129000000000001,"5min":29.129000000000001,"15min":29.129000000000001},"last":0.98899999999999999},{"interface":"front","average":{"1min":3.0219999999999998,"5min":9.9459999999999997,"15min":4.1239999999999997},"min":{"1min":0.70799999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":29.103000000000002,"5min":712.98699999999997,"15min":712.98699999999997},"last":1.534}]},{"osd":4,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":49.854999999999997,"5min":40.755000000000003,"15min":47.512999999999998},"min":{"1min":0.61899999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":971.67999999999995,"5min":971.67999999999995,"15min":971.67999999999995},"last":1.661},{"interface":"front","average":{"1min":1.236,"5min":1.264,"15min":1.292},"min":{"1min":0.64900000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.5960000000000001,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":1.421}]},{"osd":6,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":91.870999999999995,"5min":48.862000000000002,"15min":49.893000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":966.60900000000004,"5min":966.60900000000004,"15min":966.60900000000004},"last":1.45},{"interface":"front","average":{"1min":28.396999999999998,"5min":21.527000000000001,"15min":24.52},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":501.79000000000002,"5min":517.26700000000005,"15min":517.26700000000005},"last":1.5029999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.341,"15min":1.319},"min":{"1min":0.77500000000000002,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.8940000000000001,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.893},{"interface":"front","average":{"1min":27.206,"5min":6.6630000000000003,"15min":3.2650000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":515.09100000000001,"5min":515.09100000000001,"15min":515.09100000000001},"last":1.9239999999999999}]}]}]}} 2020-08-07T15:09:35.064 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_1 pgp_num 16 2020-08-07T15:09:35.113 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:35.110+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:35.144 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:35.145 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:35.151 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:09:35.235 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:35.198+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:35.281 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 69 16 83745 83729 9.4793 11.2266 0.00430112 0.0131812 2020-08-07T15:09:35.305 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:35.299+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:35.379 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:35.380 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:35.393 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:09:35.405 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:35.405+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:35.467 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:09:35.510 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:35.505+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:35.569 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:35.569 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:35.583 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:09:35.626 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:35.623+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:35.718 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:35.711+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:35.802 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:35.802 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:35.810 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:09:35.821 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:35.820+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:35.923 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:35.921+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:35.986 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:35.986 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:36.004 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:09:36.038 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:36.028+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:36.061 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:09:36.142 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:36.135+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:36.218 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:36.219 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:36.231 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:09:36.268 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:09:36.274 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:36.272+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:36.280 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 70 16 85395 85379 9.52798 12.8906 0.0599202 0.0131133 2020-08-07T15:09:36.318 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:09:36.350 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:09:36.360 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:36.357+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:36.398 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:09:36.430 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:09:36.450 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:36.450 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:36.462 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:09:36.463 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:36.460+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:36.480 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:09:36.481 INFO:teuthology.orchestra.run.smithi074.stderr:set pool 4 pgp_num to 16 2020-08-07T15:09:36.514 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:09:36.542 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:09:36.543 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:09:36.543 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:09:36.543 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:09:36.543 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:09:36.543 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:09:36.544 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:09:36.544 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:09:36.544 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:09:36.630 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:36.595+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:36.683 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:36.667+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:36.692 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:09:36.692 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:09:36.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:09:36.699 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:09:36.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.700 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42440 4.9 4:9b3fd4e3:::benchmark_data_smithi074_44586_object5304:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:09:36.700 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:09:36.700 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.448048501999999, 2020-08-07T15:09:36.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9384075030000001, 2020-08-07T15:09:36.700 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:36.701 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:36.701 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:36.701 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:36.701 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:36.701 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42440 2020-08-07T15:09:36.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.702 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:36.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:36.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:09:36.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:36.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:36.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:09:36.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6295000000000002e-05 2020-08-07T15:09:36.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:36.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237116+0000", 2020-08-07T15:09:36.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0390000000000001e-06 2020-08-07T15:09:36.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:36.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237122+0000", 2020-08-07T15:09:36.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8340000000000001e-05 2020-08-07T15:09:36.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:36.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237140+0000", 2020-08-07T15:09:36.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.045e-06 2020-08-07T15:09:36.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:36.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237145+0000", 2020-08-07T15:09:36.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044548920000000002 2020-08-07T15:09:36.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:36.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241600+0000", 2020-08-07T15:09:36.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103336 2020-08-07T15:09:36.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:36.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241703+0000", 2020-08-07T15:09:36.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018660399999999999 2020-08-07T15:09:36.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:09:36.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241890+0000", 2020-08-07T15:09:36.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00096796000000000002 2020-08-07T15:09:36.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:36.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.242858+0000", 2020-08-07T15:09:36.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039845000000000002 2020-08-07T15:09:36.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:36.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246843+0000", 2020-08-07T15:09:36.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3499999999999996e-07 2020-08-07T15:09:36.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:36.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246843+0000", 2020-08-07T15:09:36.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7132999999999995e-05 2020-08-07T15:09:36.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:36.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246920+0000", 2020-08-07T15:09:36.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046993410000000001 2020-08-07T15:09:36.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:36.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.251620+0000", 2020-08-07T15:09:36.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040332956000000003 2020-08-07T15:09:36.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:36.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.291953+0000", 2020-08-07T15:09:36.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8390000000000003e-06 2020-08-07T15:09:36.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:36.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.291958+0000", 2020-08-07T15:09:36.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99961357500000003 2020-08-07T15:09:36.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:36.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291572+0000", 2020-08-07T15:09:36.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3913999999999999e-05 2020-08-07T15:09:36.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:36.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291586+0000", 2020-08-07T15:09:36.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88379723099999996 2020-08-07T15:09:36.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:36.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175383+0000", 2020-08-07T15:09:36.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3868000000000004e-05 2020-08-07T15:09:36.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:36.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175467+0000", 2020-08-07T15:09:36.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9384075030000001 2020-08-07T15:09:36.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:36.773 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:36.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:36.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.774 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42489 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:09:36.774 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:09:36.775 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.404748439999999, 2020-08-07T15:09:36.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952608639999999, 2020-08-07T15:09:36.775 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:36.775 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:36.775 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:36.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:36.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:36.776 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42489 2020-08-07T15:09:36.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.778 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:36.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:36.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:09:36.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:36.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:36.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:09:36.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0379e-05 2020-08-07T15:09:36.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:36.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280410+0000", 2020-08-07T15:09:36.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0489999999999998e-06 2020-08-07T15:09:36.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:36.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280417+0000", 2020-08-07T15:09:36.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8281e-05 2020-08-07T15:09:36.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:36.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280435+0000", 2020-08-07T15:09:36.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7510000000000008e-06 2020-08-07T15:09:36.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:36.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280443+0000", 2020-08-07T15:09:36.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018142899999999999 2020-08-07T15:09:36.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:36.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280624+0000", 2020-08-07T15:09:36.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013473599999999999 2020-08-07T15:09:36.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:36.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280759+0000", 2020-08-07T15:09:36.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019964200000000001 2020-08-07T15:09:36.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:09:36.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280959+0000", 2020-08-07T15:09:36.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005588038 2020-08-07T15:09:36.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:36.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286547+0000", 2020-08-07T15:09:36.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.376e-05 2020-08-07T15:09:36.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:36.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286561+0000", 2020-08-07T15:09:36.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4199999999999995e-07 2020-08-07T15:09:36.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:36.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286561+0000", 2020-08-07T15:09:36.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9218e-05 2020-08-07T15:09:36.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:36.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286631+0000", 2020-08-07T15:09:36.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027909199999999999 2020-08-07T15:09:36.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:36.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286910+0000", 2020-08-07T15:09:36.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050904089999999997 2020-08-07T15:09:36.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:36.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292000+0000", 2020-08-07T15:09:36.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9579999999999996e-06 2020-08-07T15:09:36.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:36.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292009+0000", 2020-08-07T15:09:36.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99961402799999999 2020-08-07T15:09:36.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:36.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291623+0000", 2020-08-07T15:09:36.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2490000000000001e-06 2020-08-07T15:09:36.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:36.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291631+0000", 2020-08-07T15:09:36.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88387422199999999 2020-08-07T15:09:36.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:36.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175506+0000", 2020-08-07T15:09:36.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000114881 2020-08-07T15:09:36.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:36.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175620+0000", 2020-08-07T15:09:36.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952608639999999 2020-08-07T15:09:36.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:36.805 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:36.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:36.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.806 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42490 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:09:36.806 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:09:36.806 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.404621272, 2020-08-07T15:09:36.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89528059, 2020-08-07T15:09:36.807 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:36.807 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:36.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:36.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:36.808 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:36.808 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42490 2020-08-07T15:09:36.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.810 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:36.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:36.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:09:36.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:36.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:36.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:09:36.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2571999999999997e-05 2020-08-07T15:09:36.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:36.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280539+0000", 2020-08-07T15:09:36.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999998e-06 2020-08-07T15:09:36.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:36.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280545+0000", 2020-08-07T15:09:36.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.134e-05 2020-08-07T15:09:36.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:36.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280556+0000", 2020-08-07T15:09:36.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.746e-06 2020-08-07T15:09:36.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:36.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280561+0000", 2020-08-07T15:09:36.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00089258999999999996 2020-08-07T15:09:36.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:36.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281453+0000", 2020-08-07T15:09:36.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5641000000000001e-05 2020-08-07T15:09:36.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:36.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281549+0000", 2020-08-07T15:09:36.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020555200000000001 2020-08-07T15:09:36.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:09:36.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281755+0000", 2020-08-07T15:09:36.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048878230000000003 2020-08-07T15:09:36.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:36.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286642+0000", 2020-08-07T15:09:36.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0954e-05 2020-08-07T15:09:36.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:36.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286693+0000", 2020-08-07T15:09:36.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0699999999999997e-07 2020-08-07T15:09:36.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:36.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286694+0000", 2020-08-07T15:09:36.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3697999999999998e-05 2020-08-07T15:09:36.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:36.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286768+0000", 2020-08-07T15:09:36.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017185900000000001 2020-08-07T15:09:36.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:36.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286940+0000", 2020-08-07T15:09:36.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051207819999999999 2020-08-07T15:09:36.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:36.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292060+0000", 2020-08-07T15:09:36.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8000000000000004e-06 2020-08-07T15:09:36.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:36.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292069+0000", 2020-08-07T15:09:36.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99959520599999996 2020-08-07T15:09:36.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:36.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291664+0000", 2020-08-07T15:09:36.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0160000000000008e-06 2020-08-07T15:09:36.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:36.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291672+0000", 2020-08-07T15:09:36.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88399374500000005 2020-08-07T15:09:36.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:36.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175666+0000", 2020-08-07T15:09:36.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101239 2020-08-07T15:09:36.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:36.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175767+0000", 2020-08-07T15:09:36.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89528059 2020-08-07T15:09:36.857 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:36.857 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:36.857 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:36.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.859 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42491 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:09:36.859 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:09:36.859 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.404514337000002, 2020-08-07T15:09:36.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895291936, 2020-08-07T15:09:36.859 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:36.860 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:36.860 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:36.860 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:36.860 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:36.861 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42491 2020-08-07T15:09:36.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.861 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:36.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:36.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:09:36.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:36.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:36.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:09:36.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0402000000000001e-05 2020-08-07T15:09:36.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:36.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280624+0000", 2020-08-07T15:09:36.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0340000000000001e-06 2020-08-07T15:09:36.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:36.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280627+0000", 2020-08-07T15:09:36.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1886999999999999e-05 2020-08-07T15:09:36.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:36.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280649+0000", 2020-08-07T15:09:36.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5430000000000004e-06 2020-08-07T15:09:36.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:36.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280654+0000", 2020-08-07T15:09:36.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032800960000000001 2020-08-07T15:09:36.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:36.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.283934+0000", 2020-08-07T15:09:36.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011264699999999999 2020-08-07T15:09:36.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:36.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284046+0000", 2020-08-07T15:09:36.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019987299999999999 2020-08-07T15:09:36.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:09:36.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284246+0000", 2020-08-07T15:09:36.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025610960000000001 2020-08-07T15:09:36.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:36.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286807+0000", 2020-08-07T15:09:36.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1409999999999999e-06 2020-08-07T15:09:36.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:36.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286814+0000", 2020-08-07T15:09:36.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0499999999999999e-07 2020-08-07T15:09:36.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:36.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286815+0000", 2020-08-07T15:09:36.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8538000000000003e-05 2020-08-07T15:09:36.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:36.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286853+0000", 2020-08-07T15:09:36.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112914 2020-08-07T15:09:36.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:36.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286966+0000", 2020-08-07T15:09:36.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051403120000000002 2020-08-07T15:09:36.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:36.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292107+0000", 2020-08-07T15:09:36.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8019999999999992e-06 2020-08-07T15:09:36.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:36.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292114+0000", 2020-08-07T15:09:36.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.999590859 2020-08-07T15:09:36.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:36.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291705+0000", 2020-08-07T15:09:36.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.174e-06 2020-08-07T15:09:36.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:36.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291713+0000", 2020-08-07T15:09:36.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88407676000000002 2020-08-07T15:09:36.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:36.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175790+0000", 2020-08-07T15:09:36.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5452999999999996e-05 2020-08-07T15:09:36.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:36.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175886+0000", 2020-08-07T15:09:36.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895291936 2020-08-07T15:09:36.978 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:36.978 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:36.978 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:36.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.979 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42492 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:09:36.979 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:09:36.979 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.404421286999998, 2020-08-07T15:09:36.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895293774, 2020-08-07T15:09:36.980 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:36.980 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:36.980 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:36.980 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:36.981 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:36.981 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42492 2020-08-07T15:09:36.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.981 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:36.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:36.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:09:36.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:36.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:36.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:09:36.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8272000000000001e-05 2020-08-07T15:09:36.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:36.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280735+0000", 2020-08-07T15:09:36.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2159999999999997e-06 2020-08-07T15:09:36.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:36.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280740+0000", 2020-08-07T15:09:36.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0251000000000001e-05 2020-08-07T15:09:36.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:36.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280750+0000", 2020-08-07T15:09:36.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3139999999999997e-06 2020-08-07T15:09:36.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:36.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280755+0000", 2020-08-07T15:09:36.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039343579999999998 2020-08-07T15:09:36.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:36.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284689+0000", 2020-08-07T15:09:36.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111648 2020-08-07T15:09:36.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:36.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:36.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284801+0000", 2020-08-07T15:09:36.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019563600000000001 2020-08-07T15:09:36.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:36.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:09:37.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284996+0000", 2020-08-07T15:09:37.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018636709999999999 2020-08-07T15:09:37.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:37.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286860+0000", 2020-08-07T15:09:37.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001865802 2020-08-07T15:09:37.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:37.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288726+0000", 2020-08-07T15:09:37.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0399999999999996e-07 2020-08-07T15:09:37.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:37.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288726+0000", 2020-08-07T15:09:37.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6573999999999998e-05 2020-08-07T15:09:37.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:37.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288773+0000", 2020-08-07T15:09:37.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003455283 2020-08-07T15:09:37.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292228+0000", 2020-08-07T15:09:37.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0099999999999995e-06 2020-08-07T15:09:37.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:37.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292236+0000", 2020-08-07T15:09:37.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99950963999999998 2020-08-07T15:09:37.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291746+0000", 2020-08-07T15:09:37.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0779999999999996e-06 2020-08-07T15:09:37.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:37.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291754+0000", 2020-08-07T15:09:37.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.884152508 2020-08-07T15:09:37.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175906+0000", 2020-08-07T15:09:37.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4009000000000002e-05 2020-08-07T15:09:37.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:37.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175980+0000", 2020-08-07T15:09:37.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895293774 2020-08-07T15:09:37.012 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:37.013 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:37.013 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:37.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.014 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42493 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:09:37.014 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:09:37.014 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.404319639999997, 2020-08-07T15:09:37.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952963789999999, 2020-08-07T15:09:37.015 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:37.016 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:37.016 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:37.017 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:37.017 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:37.017 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42493 2020-08-07T15:09:37.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.017 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:37.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:37.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:09:37.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:37.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:37.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:09:37.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9740999999999997e-05 2020-08-07T15:09:37.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:37.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280838+0000", 2020-08-07T15:09:37.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7809999999999996e-06 2020-08-07T15:09:37.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:37.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280843+0000", 2020-08-07T15:09:37.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0648999999999999e-05 2020-08-07T15:09:37.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:37.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280854+0000", 2020-08-07T15:09:37.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3669999999999999e-06 2020-08-07T15:09:37.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:37.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280858+0000", 2020-08-07T15:09:37.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004850867 2020-08-07T15:09:37.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.285709+0000", 2020-08-07T15:09:37.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110037 2020-08-07T15:09:37.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:37.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.285819+0000", 2020-08-07T15:09:37.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020935699999999999 2020-08-07T15:09:37.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:09:37.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286028+0000", 2020-08-07T15:09:37.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020900850000000002 2020-08-07T15:09:37.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:37.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288118+0000", 2020-08-07T15:09:37.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041669649999999999 2020-08-07T15:09:37.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292285+0000", 2020-08-07T15:09:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3820000000000004e-06 2020-08-07T15:09:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292294+0000", 2020-08-07T15:09:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949242900000002 2020-08-07T15:09:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291786+0000", 2020-08-07T15:09:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000003e-06 2020-08-07T15:09:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291794+0000", 2020-08-07T15:09:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88420652700000002 2020-08-07T15:09:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176000+0000", 2020-08-07T15:09:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4634000000000003e-05 2020-08-07T15:09:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176085+0000", 2020-08-07T15:09:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952963789999999 2020-08-07T15:09:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42494 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:09:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:09:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.404217541000001, 2020-08-07T15:09:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952882820000001, 2020-08-07T15:09:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42494 2020-08-07T15:09:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:09:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:09:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0117999999999997e-05 2020-08-07T15:09:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:37.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280941+0000", 2020-08-07T15:09:37.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1240000000000004e-06 2020-08-07T15:09:37.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:37.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280946+0000", 2020-08-07T15:09:37.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0000000000000001e-05 2020-08-07T15:09:37.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:37.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280956+0000", 2020-08-07T15:09:37.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.279e-06 2020-08-07T15:09:37.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:37.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280960+0000", 2020-08-07T15:09:37.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0062727080000000001 2020-08-07T15:09:37.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287233+0000", 2020-08-07T15:09:37.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018362900000000001 2020-08-07T15:09:37.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:37.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287416+0000", 2020-08-07T15:09:37.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029536100000000001 2020-08-07T15:09:37.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:09:37.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287712+0000", 2020-08-07T15:09:37.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046231739999999999 2020-08-07T15:09:37.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292335+0000", 2020-08-07T15:09:37.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3860000000000001e-06 2020-08-07T15:09:37.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:37.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292342+0000", 2020-08-07T15:09:37.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99948501000000001 2020-08-07T15:09:37.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291827+0000", 2020-08-07T15:09:37.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2272999999999999e-05 2020-08-07T15:09:37.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:37.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291840+0000", 2020-08-07T15:09:37.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88426668100000005 2020-08-07T15:09:37.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176106+0000", 2020-08-07T15:09:37.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2539000000000005e-05 2020-08-07T15:09:37.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:37.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176179+0000", 2020-08-07T15:09:37.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952882820000001 2020-08-07T15:09:37.241 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:37.242 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:37.242 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:37.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.242 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42495 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:09:37.243 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:09:37.243 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.404116451, 2020-08-07T15:09:37.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895286867, 2020-08-07T15:09:37.243 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:37.243 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:37.244 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:37.245 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:37.245 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:37.245 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42495 2020-08-07T15:09:37.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.246 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:37.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:37.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:09:37.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:37.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:37.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:09:37.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6690999999999997e-05 2020-08-07T15:09:37.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:37.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281038+0000", 2020-08-07T15:09:37.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7860000000000002e-06 2020-08-07T15:09:37.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:37.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281044+0000", 2020-08-07T15:09:37.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0429e-05 2020-08-07T15:09:37.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:37.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281054+0000", 2020-08-07T15:09:37.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9040000000000002e-06 2020-08-07T15:09:37.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:37.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281058+0000", 2020-08-07T15:09:37.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070941490000000001 2020-08-07T15:09:37.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288153+0000", 2020-08-07T15:09:37.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9006999999999998e-05 2020-08-07T15:09:37.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:37.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288222+0000", 2020-08-07T15:09:37.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012106400000000001 2020-08-07T15:09:37.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:09:37.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288343+0000", 2020-08-07T15:09:37.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004034752 2020-08-07T15:09:37.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292377+0000", 2020-08-07T15:09:37.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.537e-06 2020-08-07T15:09:37.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:37.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292385+0000", 2020-08-07T15:09:37.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949123500000003 2020-08-07T15:09:37.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291876+0000", 2020-08-07T15:09:37.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7729999999999999e-06 2020-08-07T15:09:37.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:37.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291884+0000", 2020-08-07T15:09:37.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88431700899999999 2020-08-07T15:09:37.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176201+0000", 2020-08-07T15:09:37.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7531e-05 2020-08-07T15:09:37.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:37.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176278+0000", 2020-08-07T15:09:37.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895286867 2020-08-07T15:09:37.289 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:37.289 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:37.290 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:37.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.290 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42496 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:09:37.290 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:09:37.291 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.404011541999999, 2020-08-07T15:09:37.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89526942, 2020-08-07T15:09:37.291 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:37.292 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:37.292 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:37.292 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:37.292 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:37.293 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42496 2020-08-07T15:09:37.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.293 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:37.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:37.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:09:37.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:37.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:37.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:09:37.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9227999999999997e-05 2020-08-07T15:09:37.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:37.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281146+0000", 2020-08-07T15:09:37.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1120000000000004e-06 2020-08-07T15:09:37.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:37.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281151+0000", 2020-08-07T15:09:37.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2208999999999999e-05 2020-08-07T15:09:37.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:37.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281163+0000", 2020-08-07T15:09:37.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5469999999999998e-06 2020-08-07T15:09:37.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:37.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281168+0000", 2020-08-07T15:09:37.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0076520829999999996 2020-08-07T15:09:37.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288820+0000", 2020-08-07T15:09:37.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5035000000000003e-05 2020-08-07T15:09:37.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:37.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288885+0000", 2020-08-07T15:09:37.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116507 2020-08-07T15:09:37.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:09:37.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.289001+0000", 2020-08-07T15:09:37.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034121960000000002 2020-08-07T15:09:37.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292413+0000", 2020-08-07T15:09:37.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8109999999999997e-06 2020-08-07T15:09:37.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:37.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292420+0000", 2020-08-07T15:09:37.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949504600000005 2020-08-07T15:09:37.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291915+0000", 2020-08-07T15:09:37.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3999999999999992e-06 2020-08-07T15:09:37.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:37.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291924+0000", 2020-08-07T15:09:37.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88437394599999997 2020-08-07T15:09:37.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176298+0000", 2020-08-07T15:09:37.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8300000000000007e-05 2020-08-07T15:09:37.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:37.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176366+0000", 2020-08-07T15:09:37.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89526942 2020-08-07T15:09:37.311 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:37.311 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:37.311 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:37.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.312 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73081 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:37.312 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:09:37.313 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4585422680000004, 2020-08-07T15:09:37.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0005038420000001, 2020-08-07T15:09:37.314 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:37.314 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:37.314 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:37.314 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:37.315 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:37.315 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73081 2020-08-07T15:09:37.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.315 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:37.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:37.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:09:37.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:37.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:37.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:09:37.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3016000000000003e-05 2020-08-07T15:09:37.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:37.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226609+0000", 2020-08-07T15:09:37.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4479999999999996e-06 2020-08-07T15:09:37.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:37.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226613+0000", 2020-08-07T15:09:37.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0189999999999995e-06 2020-08-07T15:09:37.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:37.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226622+0000", 2020-08-07T15:09:37.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5980000000000001e-06 2020-08-07T15:09:37.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:37.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226626+0000", 2020-08-07T15:09:37.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8088999999999999e-05 2020-08-07T15:09:37.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226684+0000", 2020-08-07T15:09:37.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2162e-05 2020-08-07T15:09:37.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:37.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226696+0000", 2020-08-07T15:09:37.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98675689 2020-08-07T15:09:37.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213453+0000", 2020-08-07T15:09:37.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012916800000000001 2020-08-07T15:09:37.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:37.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213582+0000", 2020-08-07T15:09:37.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015665899999999999 2020-08-07T15:09:37.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:09:37.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213739+0000", 2020-08-07T15:09:37.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049940239999999997 2020-08-07T15:09:37.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:37.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218733+0000", 2020-08-07T15:09:37.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0079774779999999997 2020-08-07T15:09:37.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:37.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226710+0000", 2020-08-07T15:09:37.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.161e-06 2020-08-07T15:09:37.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:37.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226711+0000", 2020-08-07T15:09:37.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0105000000000001e-05 2020-08-07T15:09:37.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:37.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226772+0000", 2020-08-07T15:09:37.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5633999999999997e-05 2020-08-07T15:09:37.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:37.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226817+0000", 2020-08-07T15:09:37.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7379000000000001e-05 2020-08-07T15:09:37.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:37.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226875+0000", 2020-08-07T15:09:37.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8000000000000005e-07 2020-08-07T15:09:37.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:37.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226875+0000", 2020-08-07T15:09:37.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019423199999999999 2020-08-07T15:09:37.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:37.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227070+0000", 2020-08-07T15:09:37.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0005038420000001 2020-08-07T15:09:37.382 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:37.383 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:37.383 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:37.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.383 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73082 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:37.384 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:09:37.384 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4581718039999991, 2020-08-07T15:09:37.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001326693, 2020-08-07T15:09:37.384 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:37.384 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:37.385 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:37.385 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:37.385 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:37.385 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73082 2020-08-07T15:09:37.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.386 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:37.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:37.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:09:37.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:37.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:37.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:09:37.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6928000000000003e-05 2020-08-07T15:09:37.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:37.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226973+0000", 2020-08-07T15:09:37.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3630000000000004e-06 2020-08-07T15:09:37.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:37.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226977+0000", 2020-08-07T15:09:37.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9659999999999996e-06 2020-08-07T15:09:37.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:37.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226985+0000", 2020-08-07T15:09:37.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5800000000000002e-06 2020-08-07T15:09:37.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:37.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226990+0000", 2020-08-07T15:09:37.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9101999999999998e-05 2020-08-07T15:09:37.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227039+0000", 2020-08-07T15:09:37.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9620000000000005e-06 2020-08-07T15:09:37.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:37.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227049+0000", 2020-08-07T15:09:37.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98722513999999995 2020-08-07T15:09:37.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214274+0000", 2020-08-07T15:09:37.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2700999999999994e-05 2020-08-07T15:09:37.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:37.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214347+0000", 2020-08-07T15:09:37.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015346899999999999 2020-08-07T15:09:37.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:09:37.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214500+0000", 2020-08-07T15:09:37.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042450309999999998 2020-08-07T15:09:37.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:37.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218745+0000", 2020-08-07T15:09:37.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008379503 2020-08-07T15:09:37.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:37.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227125+0000", 2020-08-07T15:09:37.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7899999999999998e-07 2020-08-07T15:09:37.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:37.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227126+0000", 2020-08-07T15:09:37.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1610000000000002e-05 2020-08-07T15:09:37.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:37.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227177+0000", 2020-08-07T15:09:37.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489000000000003e-05 2020-08-07T15:09:37.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:37.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227226+0000", 2020-08-07T15:09:37.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00084338600000000001 2020-08-07T15:09:37.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:37.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228069+0000", 2020-08-07T15:09:37.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7e-07 2020-08-07T15:09:37.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:37.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228069+0000", 2020-08-07T15:09:37.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019351399999999999 2020-08-07T15:09:37.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:37.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228263+0000", 2020-08-07T15:09:37.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001326693 2020-08-07T15:09:37.433 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:37.433 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:37.434 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:37.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.435 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73083 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:37.435 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:09:37.436 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4580775450000001, 2020-08-07T15:09:37.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015968209999999, 2020-08-07T15:09:37.437 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:37.437 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:37.438 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:37.438 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:37.438 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:37.438 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73083 2020-08-07T15:09:37.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.439 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:37.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:37.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:09:37.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:37.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:37.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:09:37.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1627000000000001e-05 2020-08-07T15:09:37.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:37.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227062+0000", 2020-08-07T15:09:37.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.253e-06 2020-08-07T15:09:37.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:37.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227065+0000", 2020-08-07T15:09:37.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3168e-05 2020-08-07T15:09:37.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:37.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227079+0000", 2020-08-07T15:09:37.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1549999999999999e-06 2020-08-07T15:09:37.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:37.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227082+0000", 2020-08-07T15:09:37.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5534999999999997e-05 2020-08-07T15:09:37.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227127+0000", 2020-08-07T15:09:37.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6823000000000001e-05 2020-08-07T15:09:37.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:37.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227144+0000", 2020-08-07T15:09:37.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98771770599999997 2020-08-07T15:09:37.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214862+0000", 2020-08-07T15:09:37.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8252999999999999e-05 2020-08-07T15:09:37.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:37.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214930+0000", 2020-08-07T15:09:37.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013646400000000001 2020-08-07T15:09:37.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:09:37.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215066+0000", 2020-08-07T15:09:37.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036907089999999999 2020-08-07T15:09:37.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:37.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218757+0000", 2020-08-07T15:09:37.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0085133450000000003 2020-08-07T15:09:37.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:37.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227270+0000", 2020-08-07T15:09:37.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4899999999999995e-07 2020-08-07T15:09:37.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:37.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227271+0000", 2020-08-07T15:09:37.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4555999999999997e-05 2020-08-07T15:09:37.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:37.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227326+0000", 2020-08-07T15:09:37.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024594800000000001 2020-08-07T15:09:37.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:37.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227572+0000", 2020-08-07T15:09:37.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00086125700000000004 2020-08-07T15:09:37.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:37.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228433+0000", 2020-08-07T15:09:37.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3799999999999998e-07 2020-08-07T15:09:37.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:37.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228433+0000", 2020-08-07T15:09:37.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019413500000000001 2020-08-07T15:09:37.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:37.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228627+0000", 2020-08-07T15:09:37.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015968209999999 2020-08-07T15:09:37.472 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:37.472 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:37.472 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:37.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.473 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73084 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:37.473 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:09:37.473 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4579886060000007, 2020-08-07T15:09:37.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0016919049999999, 2020-08-07T15:09:37.474 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:37.474 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:37.474 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:37.474 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:37.475 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:37.475 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73084 2020-08-07T15:09:37.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.476 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:37.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:37.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:09:37.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:37.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:37.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:09:37.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5565999999999997e-05 2020-08-07T15:09:37.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:37.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227165+0000", 2020-08-07T15:09:37.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2020-08-07T15:09:37.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:37.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227169+0000", 2020-08-07T15:09:37.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6850000000000001e-06 2020-08-07T15:09:37.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:37.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227176+0000", 2020-08-07T15:09:37.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1949999999999999e-06 2020-08-07T15:09:37.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:37.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227179+0000", 2020-08-07T15:09:37.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0318000000000003e-05 2020-08-07T15:09:37.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227220+0000", 2020-08-07T15:09:37.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0349e-05 2020-08-07T15:09:37.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:37.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227230+0000", 2020-08-07T15:09:37.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98832962400000002 2020-08-07T15:09:37.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215560+0000", 2020-08-07T15:09:37.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013913099999999999 2020-08-07T15:09:37.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:37.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215699+0000", 2020-08-07T15:09:37.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000200006 2020-08-07T15:09:37.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:09:37.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215899+0000", 2020-08-07T15:09:37.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028673650000000002 2020-08-07T15:09:37.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:37.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218766+0000", 2020-08-07T15:09:37.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086101779999999996 2020-08-07T15:09:37.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:37.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227376+0000", 2020-08-07T15:09:37.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0999999999999998e-07 2020-08-07T15:09:37.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:37.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227377+0000", 2020-08-07T15:09:37.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6552999999999997e-05 2020-08-07T15:09:37.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:37.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227454+0000", 2020-08-07T15:09:37.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000421717 2020-08-07T15:09:37.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:37.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227875+0000", 2020-08-07T15:09:37.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00081781300000000003 2020-08-07T15:09:37.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:37.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228693+0000", 2020-08-07T15:09:37.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1699999999999997e-07 2020-08-07T15:09:37.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:37.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228694+0000", 2020-08-07T15:09:37.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117483 2020-08-07T15:09:37.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:37.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228811+0000", 2020-08-07T15:09:37.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0016919049999999 2020-08-07T15:09:37.535 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:37.536 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:37.536 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:37.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.536 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73085 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:37.537 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:09:37.537 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4578527910000005, 2020-08-07T15:09:37.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0018015629999999, 2020-08-07T15:09:37.558 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:37.558 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:37.558 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:37.558 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:37.558 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:37.559 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73085 2020-08-07T15:09:37.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.559 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:37.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:37.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:09:37.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:37.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:37.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:09:37.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6570999999999997e-05 2020-08-07T15:09:37.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:37.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227322+0000", 2020-08-07T15:09:37.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5509999999999996e-06 2020-08-07T15:09:37.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:37.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227328+0000", 2020-08-07T15:09:37.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2498e-05 2020-08-07T15:09:37.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:37.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227341+0000", 2020-08-07T15:09:37.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4389999999999999e-06 2020-08-07T15:09:37.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:37.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227346+0000", 2020-08-07T15:09:37.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3711999999999998e-05 2020-08-07T15:09:37.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227390+0000", 2020-08-07T15:09:37.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1868999999999999e-05 2020-08-07T15:09:37.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:37.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227402+0000", 2020-08-07T15:09:37.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98896525099999999 2020-08-07T15:09:37.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216367+0000", 2020-08-07T15:09:37.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012573 2020-08-07T15:09:37.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:37.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216493+0000", 2020-08-07T15:09:37.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020525200000000001 2020-08-07T15:09:37.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:09:37.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216698+0000", 2020-08-07T15:09:37.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037596470000000001 2020-08-07T15:09:37.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:37.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.220458+0000", 2020-08-07T15:09:37.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070227190000000002 2020-08-07T15:09:37.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:37.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227480+0000", 2020-08-07T15:09:37.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6600000000000003e-07 2020-08-07T15:09:37.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:37.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227481+0000", 2020-08-07T15:09:37.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9898000000000003e-05 2020-08-07T15:09:37.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:37.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227521+0000", 2020-08-07T15:09:37.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039130000000000002 2020-08-07T15:09:37.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:37.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227912+0000", 2020-08-07T15:09:37.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00095502399999999996 2020-08-07T15:09:37.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:37.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228867+0000", 2020-08-07T15:09:37.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2020-08-07T15:09:37.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:37.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228867+0000", 2020-08-07T15:09:37.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018933599999999999 2020-08-07T15:09:37.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:37.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229057+0000", 2020-08-07T15:09:37.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0018015629999999 2020-08-07T15:09:37.707 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:37.707 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:37.708 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:37.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.708 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73086 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:37.709 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:09:37.709 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4577202949999997, 2020-08-07T15:09:37.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002364923, 2020-08-07T15:09:37.710 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:37.710 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:37.710 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:37.711 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:37.711 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:37.711 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73086 2020-08-07T15:09:37.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.711 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:37.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:37.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:09:37.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:37.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:37.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:09:37.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000238426 2020-08-07T15:09:37.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:37.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227626+0000", 2020-08-07T15:09:37.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9609999999999999e-06 2020-08-07T15:09:37.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:37.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227633+0000", 2020-08-07T15:09:37.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2727e-05 2020-08-07T15:09:37.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227646+0000", 2020-08-07T15:09:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1739999999999997e-06 2020-08-07T15:09:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227652+0000", 2020-08-07T15:09:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5004000000000001e-05 2020-08-07T15:09:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227737+0000", 2020-08-07T15:09:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2744e-05 2020-08-07T15:09:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227750+0000", 2020-08-07T15:09:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98938776500000003 2020-08-07T15:09:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217138+0000", 2020-08-07T15:09:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012361499999999999 2020-08-07T15:09:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217261+0000", 2020-08-07T15:09:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000212504 2020-08-07T15:09:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:09:37.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217474+0000", 2020-08-07T15:09:37.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002995036 2020-08-07T15:09:37.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:37.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.220469+0000", 2020-08-07T15:09:37.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071283249999999996 2020-08-07T15:09:37.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227597+0000", 2020-08-07T15:09:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0499999999999999e-07 2020-08-07T15:09:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227597+0000", 2020-08-07T15:09:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1872999999999997e-05 2020-08-07T15:09:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227639+0000", 2020-08-07T15:09:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032010200000000001 2020-08-07T15:09:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227959+0000", 2020-08-07T15:09:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015980899999999999 2020-08-07T15:09:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229557+0000", 2020-08-07T15:09:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9299999999999999e-07 2020-08-07T15:09:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229558+0000", 2020-08-07T15:09:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019477900000000001 2020-08-07T15:09:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:37.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229753+0000", 2020-08-07T15:09:37.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002364923 2020-08-07T15:09:37.830 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:37.830 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:37.830 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:37.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.831 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73087 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:37.854 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:09:37.854 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4567791420000002, 2020-08-07T15:09:37.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001896506, 2020-08-07T15:09:37.855 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:37.855 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:37.856 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:37.856 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:37.856 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:37.857 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73087 2020-08-07T15:09:37.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.857 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:37.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:37.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:09:37.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:37.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:37.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:09:37.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7354e-05 2020-08-07T15:09:37.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:37.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228386+0000", 2020-08-07T15:09:37.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8000000000000001e-06 2020-08-07T15:09:37.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:37.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228393+0000", 2020-08-07T15:09:37.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3784999999999999e-05 2020-08-07T15:09:37.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:37.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228407+0000", 2020-08-07T15:09:37.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8090000000000004e-06 2020-08-07T15:09:37.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:37.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228414+0000", 2020-08-07T15:09:37.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6907000000000003e-05 2020-08-07T15:09:37.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228461+0000", 2020-08-07T15:09:37.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8929999999999994e-06 2020-08-07T15:09:37.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:37.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228469+0000", 2020-08-07T15:09:37.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98950057700000005 2020-08-07T15:09:37.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217970+0000", 2020-08-07T15:09:37.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000132182 2020-08-07T15:09:37.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:37.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218102+0000", 2020-08-07T15:09:37.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020073399999999999 2020-08-07T15:09:37.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:09:37.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218303+0000", 2020-08-07T15:09:37.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035951120000000001 2020-08-07T15:09:37.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:37.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.221898+0000", 2020-08-07T15:09:37.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057898660000000003 2020-08-07T15:09:37.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:37.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227688+0000", 2020-08-07T15:09:37.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0200000000000002e-07 2020-08-07T15:09:37.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:37.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227688+0000", 2020-08-07T15:09:37.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1477999999999996e-05 2020-08-07T15:09:37.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:37.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227760+0000", 2020-08-07T15:09:37.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016657480000000001 2020-08-07T15:09:37.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:37.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229426+0000", 2020-08-07T15:09:37.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061290400000000005 2020-08-07T15:09:37.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:37.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230039+0000", 2020-08-07T15:09:37.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2e-07 2020-08-07T15:09:37.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:37.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230039+0000", 2020-08-07T15:09:37.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018643500000000001 2020-08-07T15:09:37.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:37.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230225+0000", 2020-08-07T15:09:37.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001896506 2020-08-07T15:09:37.920 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:37.920 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:37.921 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:37.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.921 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73088 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:37.922 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:09:37.922 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4563884520000006, 2020-08-07T15:09:37.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001627153, 2020-08-07T15:09:37.922 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:37.923 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:37.923 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:37.923 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:37.923 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:37.923 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73088 2020-08-07T15:09:37.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.924 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:37.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:37.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:09:37.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:37.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:37.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:09:37.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7741000000000003e-05 2020-08-07T15:09:37.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:37.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228767+0000", 2020-08-07T15:09:37.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6409999999999999e-06 2020-08-07T15:09:37.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:37.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228773+0000", 2020-08-07T15:09:37.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0874e-05 2020-08-07T15:09:37.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:37.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228784+0000", 2020-08-07T15:09:37.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1529999999999996e-06 2020-08-07T15:09:37.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:37.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228789+0000", 2020-08-07T15:09:37.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3945e-05 2020-08-07T15:09:37.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228833+0000", 2020-08-07T15:09:37.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4020000000000002e-06 2020-08-07T15:09:37.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:37.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228841+0000", 2020-08-07T15:09:37.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98996913900000005 2020-08-07T15:09:37.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:37.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218810+0000", 2020-08-07T15:09:37.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112484 2020-08-07T15:09:37.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:37.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218923+0000", 2020-08-07T15:09:37.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019891 2020-08-07T15:09:37.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:09:37.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.219122+0000", 2020-08-07T15:09:37.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027884099999999998 2020-08-07T15:09:37.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:37.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.221910+0000", 2020-08-07T15:09:37.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058753399999999997 2020-08-07T15:09:37.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:37.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227786+0000", 2020-08-07T15:09:37.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9200000000000002e-07 2020-08-07T15:09:37.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:37.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227786+0000", 2020-08-07T15:09:37.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8572000000000003e-05 2020-08-07T15:09:37.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:37.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227824+0000", 2020-08-07T15:09:37.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001651296 2020-08-07T15:09:37.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:37.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229476+0000", 2020-08-07T15:09:37.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00077620399999999996 2020-08-07T15:09:37.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:37.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230252+0000", 2020-08-07T15:09:37.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1600000000000002e-07 2020-08-07T15:09:37.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:37.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230252+0000", 2020-08-07T15:09:37.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4433999999999997e-05 2020-08-07T15:09:37.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:37.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230347+0000", 2020-08-07T15:09:37.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001627153 2020-08-07T15:09:37.968 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:37.969 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:37.969 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:37.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.969 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73134 4.9 4:9a05b5c5:::benchmark_data_smithi074_44586_object9141:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:37.996 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.268819+0000", 2020-08-07T15:09:37.996 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4162891139999996, 2020-08-07T15:09:37.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.96356853099999995, 2020-08-07T15:09:37.997 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:37.997 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:37.998 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:37.998 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:37.998 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:37.998 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73134 2020-08-07T15:09:37.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:37.999 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:37.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:37.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:37.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268819+0000", 2020-08-07T15:09:37.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:38.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:38.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268819+0000", 2020-08-07T15:09:38.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8215999999999999e-05 2020-08-07T15:09:38.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:38.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268847+0000", 2020-08-07T15:09:38.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0309999999999999e-06 2020-08-07T15:09:38.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:38.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268850+0000", 2020-08-07T15:09:38.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.908e-06 2020-08-07T15:09:38.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:38.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268856+0000", 2020-08-07T15:09:38.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9000000000000002e-06 2020-08-07T15:09:38.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:38.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268859+0000", 2020-08-07T15:09:38.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6261000000000001e-05 2020-08-07T15:09:38.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:38.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268885+0000", 2020-08-07T15:09:38.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3600000000000004e-06 2020-08-07T15:09:38.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:38.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268891+0000", 2020-08-07T15:09:38.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95510594000000004 2020-08-07T15:09:38.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:38.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.223997+0000", 2020-08-07T15:09:38.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106016 2020-08-07T15:09:38.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:38.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.224103+0000", 2020-08-07T15:09:38.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000215558 2020-08-07T15:09:38.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:09:38.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.224318+0000", 2020-08-07T15:09:38.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001939014 2020-08-07T15:09:38.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:38.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226257+0000", 2020-08-07T15:09:38.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0047820340000000001 2020-08-07T15:09:38.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:38.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.231039+0000", 2020-08-07T15:09:38.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8999999999999996e-07 2020-08-07T15:09:38.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:38.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.231040+0000", 2020-08-07T15:09:38.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2513000000000001e-05 2020-08-07T15:09:38.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:38.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.231122+0000", 2020-08-07T15:09:38.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6319999999999997e-05 2020-08-07T15:09:38.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:38.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.231169+0000", 2020-08-07T15:09:38.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001027996 2020-08-07T15:09:38.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:38.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.232197+0000", 2020-08-07T15:09:38.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9700000000000002e-07 2020-08-07T15:09:38.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:38.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.232197+0000", 2020-08-07T15:09:38.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019047700000000001 2020-08-07T15:09:38.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:38.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.232387+0000", 2020-08-07T15:09:38.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.96356853099999995 2020-08-07T15:09:38.136 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:38.136 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:38.136 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:38.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.137 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73135 4.9 4:9a05b5c5:::benchmark_data_smithi074_44586_object9141:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:38.137 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.268948+0000", 2020-08-07T15:09:38.137 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4161596870000004, 2020-08-07T15:09:38.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.96367391999999996, 2020-08-07T15:09:38.167 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:38.167 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:38.168 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:38.168 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:38.168 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:38.168 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73135 2020-08-07T15:09:38.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268948+0000", 2020-08-07T15:09:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268948+0000", 2020-08-07T15:09:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1641000000000003e-05 2020-08-07T15:09:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268990+0000", 2020-08-07T15:09:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9349999999999999e-06 2020-08-07T15:09:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268993+0000", 2020-08-07T15:09:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6440000000000003e-06 2020-08-07T15:09:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269000+0000", 2020-08-07T15:09:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1190000000000001e-06 2020-08-07T15:09:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269003+0000", 2020-08-07T15:09:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6940000000000002e-05 2020-08-07T15:09:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269040+0000", 2020-08-07T15:09:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0988999999999999e-05 2020-08-07T15:09:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269051+0000", 2020-08-07T15:09:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95570502400000001 2020-08-07T15:09:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.224756+0000", 2020-08-07T15:09:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7387999999999999e-05 2020-08-07T15:09:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.224823+0000", 2020-08-07T15:09:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000119159 2020-08-07T15:09:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:09:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.224942+0000", 2020-08-07T15:09:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015389780000000001 2020-08-07T15:09:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226481+0000", 2020-08-07T15:09:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0047295549999999999 2020-08-07T15:09:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.231211+0000", 2020-08-07T15:09:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3500000000000002e-07 2020-08-07T15:09:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.231211+0000", 2020-08-07T15:09:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5779000000000003e-05 2020-08-07T15:09:38.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:38.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.231257+0000", 2020-08-07T15:09:38.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050559900000000002 2020-08-07T15:09:38.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:38.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.231763+0000", 2020-08-07T15:09:38.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067623900000000003 2020-08-07T15:09:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.232439+0000", 2020-08-07T15:09:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2020-08-07T15:09:38.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:38.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.232439+0000", 2020-08-07T15:09:38.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018316599999999999 2020-08-07T15:09:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.232622+0000", 2020-08-07T15:09:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.96367391999999996 2020-08-07T15:09:38.386 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:38.386 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:38.386 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:38.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.387 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73136 4.9 4:9a05b5c5:::benchmark_data_smithi074_44586_object9141:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:38.387 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.269092+0000", 2020-08-07T15:09:38.387 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4160159009999997, 2020-08-07T15:09:38.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.96368705700000001, 2020-08-07T15:09:38.388 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:38.388 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:38.388 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:38.388 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:38.389 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:38.389 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73136 2020-08-07T15:09:38.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.389 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:38.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:38.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269092+0000", 2020-08-07T15:09:38.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:38.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:38.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269092+0000", 2020-08-07T15:09:38.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9740000000000002e-05 2020-08-07T15:09:38.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:38.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269132+0000", 2020-08-07T15:09:38.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.117e-06 2020-08-07T15:09:38.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:38.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269135+0000", 2020-08-07T15:09:38.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0129999999999999e-06 2020-08-07T15:09:38.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:38.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269141+0000", 2020-08-07T15:09:38.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1769999999999998e-06 2020-08-07T15:09:38.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:38.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269144+0000", 2020-08-07T15:09:38.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3272999999999999e-05 2020-08-07T15:09:38.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:38.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269167+0000", 2020-08-07T15:09:38.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8849999999999998e-06 2020-08-07T15:09:38.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:38.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269172+0000", 2020-08-07T15:09:38.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95619174500000004 2020-08-07T15:09:38.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:38.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.225364+0000", 2020-08-07T15:09:38.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013498199999999999 2020-08-07T15:09:38.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:38.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.225499+0000", 2020-08-07T15:09:38.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020722500000000001 2020-08-07T15:09:38.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:09:38.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.225706+0000", 2020-08-07T15:09:38.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011191230000000001 2020-08-07T15:09:38.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:38.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226825+0000", 2020-08-07T15:09:38.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049631320000000003 2020-08-07T15:09:38.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:38.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.231789+0000", 2020-08-07T15:09:38.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6699999999999999e-07 2020-08-07T15:09:38.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:38.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.231789+0000", 2020-08-07T15:09:38.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2058999999999998e-05 2020-08-07T15:09:38.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:38.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.231831+0000", 2020-08-07T15:09:38.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031094400000000001 2020-08-07T15:09:38.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:38.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.232142+0000", 2020-08-07T15:09:38.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051961300000000004 2020-08-07T15:09:38.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:38.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.232661+0000", 2020-08-07T15:09:38.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2020-08-07T15:09:38.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:38.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.232662+0000", 2020-08-07T15:09:38.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117166 2020-08-07T15:09:38.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:38.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:38.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:38.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.232779+0000", 2020-08-07T15:09:38.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.96368705700000001 2020-08-07T15:09:38.540 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:38.540 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:38.540 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:38.540 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:38.541 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:38.541 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:09:38.545 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:09:38.550 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 71 16 87000 86984 9.57038 12.5391 0.0174912 0.0130572 2020-08-07T15:09:38.550 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 72 16 88343 88327 9.58317 10.4922 0.01062 0.0130395 2020-08-07T15:09:38.559 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:38.544+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:38.560 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:09:38.561 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:09:38.718 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:38.675+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:38.810 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:09:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:09:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2020-08-07T15:09:38.811 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:09:38.823 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:09:38.824 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:09:38.824 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:38.821+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:38.871 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:09:38.910 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:09:38.950 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:09:38.951 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:38.944+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:38.984 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:09:39.019 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:09:39.030 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:09:39.031 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:09:39.031 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:09:39.031 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:09:39.032 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:09:39.036 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:39.035+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:39.040 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:09:39.057 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:09:39.133 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:09:39.133 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:09:39.142 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:09:39.142 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:09:39.143 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:09:39.143 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:09:39.143 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:09:39.143 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:09:39.143 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:09:39.175 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:39.173+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:39.255 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:39.248+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:39.272 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:39.272 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:39.279 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:09:39.282 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 73 16 89813 89797 9.6092 11.4844 0.0143028 0.0130032 2020-08-07T15:09:39.367 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:39.357+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:39.460 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:39.458+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:39.506 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:39.507 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:39.507 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:09:39.524 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:09:39.567 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:39.565+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:39.673 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:39.669+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:39.734 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:39.734 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:39.743 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:09:39.787 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:39.782+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:39.890 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:39.886+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:39.950 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:39.950 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:39.963 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:09:40.018 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:40.011+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:40.091 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:40.089+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:40.145 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:40.146 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:40.156 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:09:40.200 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:40.198+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:40.296 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 74 16 91465 91449 9.65375 12.9062 0.00833474 0.0129431 2020-08-07T15:09:40.342 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:40.318+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:40.386 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:40.387 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:40.393 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:09:40.425 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:40.422+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:40.527 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:40.525+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:40.570 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:40.570 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:40.583 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:09:40.633 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:40.630+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:40.742 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:40.733+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:40.799 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:40.799 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:40.813 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:09:40.873 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:40.871+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:40.962 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:40.958+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:40.992 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:40.992 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:41.004 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:09:41.066 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:41.061+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:41.145 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:09:41.204 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:41.192+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:41.227 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:41.228 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:41.243 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:09:41.282 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 75 16 93106 93090 9.69595 12.8203 0.0157419 0.0128867 2020-08-07T15:09:41.283 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:41.278+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:41.369 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:41.366+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:41.374 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:09:41.408 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:09:41.459 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:09:41.460 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:09:41.460 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:09:41.460 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:09:41.461 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:09:41.467 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:09:41.469 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:41.468+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:41.493 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:09:41.507 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:09:41.507 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 0 1 2020-08-07T15:09:41.513 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:09:41.522 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:09:41.591 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:09:41.591 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:41.590+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:41.612 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:09:41.661 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:09:41.663 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:09:41.664 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:09:41.664 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:09:41.665 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:09:41.666 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:09:41.666 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:09:41.666 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:09:41.666 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:09:41.703 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:09:41.708 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:09:41.708 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:09:41.708 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:09:41.708 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:09:41.712 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:09:41.724 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:41.719+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:41.829 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:41.828+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:41.930 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:41.928+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:41.939 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:09:41.940 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:09:41.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:09:41.940 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:09:41.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:41.941 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:30968 4.4 e98/81 ondisk, result = 0)", 2020-08-07T15:09:41.941 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:09:41.941 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 56.198845458999998, 2020-08-07T15:09:41.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96211418999999998, 2020-08-07T15:09:41.942 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:41.942 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:09:41.942 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:41.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:41.942 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:41.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:09:41.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:41.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:41.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:41.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:41.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:09:41.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2387e-05 2020-08-07T15:09:41.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:41.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:41.944 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:41.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729821+0000", 2020-08-07T15:09:41.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9909999999999999e-06 2020-08-07T15:09:41.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:41.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:41.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:41.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729827+0000", 2020-08-07T15:09:41.946 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96185524600000005 2020-08-07T15:09:41.946 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:41.946 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:41.946 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:41.946 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691682+0000", 2020-08-07T15:09:41.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8760000000000005e-06 2020-08-07T15:09:41.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:41.947 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:41.947 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:41.947 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691690+0000", 2020-08-07T15:09:41.948 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9682000000000001e-05 2020-08-07T15:09:41.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:41.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:41.949 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:41.949 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691759+0000", 2020-08-07T15:09:41.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5381000000000001e-05 2020-08-07T15:09:41.949 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:41.950 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:41.950 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:41.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691775+0000", 2020-08-07T15:09:41.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000127627 2020-08-07T15:09:41.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:41.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:41.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:41.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691902+0000", 2020-08-07T15:09:41.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96211418999999998 2020-08-07T15:09:41.978 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:41.978 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:41.978 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:41.979 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:41.979 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:41.979 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42472 4.4 4:2ff82af8:::benchmark_data_smithi074_44586_object5308:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:09:41.979 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:09:41.979 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 41.661382685, 2020-08-07T15:09:41.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89653816399999997, 2020-08-07T15:09:41.985 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:41.985 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:41.985 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:41.985 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:41.986 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:41.986 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42472 2020-08-07T15:09:41.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:41.986 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:41.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:41.987 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:41.987 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:09:41.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:41.987 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:41.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:41.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:41.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:09:41.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9591e-05 2020-08-07T15:09:41.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:41.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:41.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:41.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267301+0000", 2020-08-07T15:09:41.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7490000000000004e-06 2020-08-07T15:09:41.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:41.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:41.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:42.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267306+0000", 2020-08-07T15:09:42.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2850000000000001e-05 2020-08-07T15:09:42.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:42.004 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267319+0000", 2020-08-07T15:09:42.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9100000000000002e-06 2020-08-07T15:09:42.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:42.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267325+0000", 2020-08-07T15:09:42.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6584000000000001e-05 2020-08-07T15:09:42.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267362+0000", 2020-08-07T15:09:42.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9188000000000001e-05 2020-08-07T15:09:42.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:42.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267451+0000", 2020-08-07T15:09:42.008 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000182917 2020-08-07T15:09:42.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.009 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:09:42.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267634+0000", 2020-08-07T15:09:42.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002484708 2020-08-07T15:09:42.009 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.010 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:42.010 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270119+0000", 2020-08-07T15:09:42.010 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0299999999999997e-07 2020-08-07T15:09:42.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.011 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:42.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270119+0000", 2020-08-07T15:09:42.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8251999999999996e-05 2020-08-07T15:09:42.012 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.013 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:42.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270188+0000", 2020-08-07T15:09:42.013 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24443688 2020-08-07T15:09:42.014 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.014 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.014 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.514625+0000", 2020-08-07T15:09:42.014 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.64911260599999998 2020-08-07T15:09:42.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.016 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.016 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163737+0000", 2020-08-07T15:09:42.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2100000000000001e-06 2020-08-07T15:09:42.016 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:42.017 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163738+0000", 2020-08-07T15:09:42.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0816000000000003e-05 2020-08-07T15:09:42.018 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.019 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.019 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:42.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163789+0000", 2020-08-07T15:09:42.019 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89653816399999997 2020-08-07T15:09:42.020 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.020 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:42.020 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.021 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:42472 4.4 e112/81 ondisk, result = 0)", 2020-08-07T15:09:42.021 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.271353+0000", 2020-08-07T15:09:42.021 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 41.657280647999997, 2020-08-07T15:09:42.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89244306299999998, 2020-08-07T15:09:42.022 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:42.022 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:09:42.022 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:42.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:42.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.271353+0000", 2020-08-07T15:09:42.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:42.023 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:42.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.271353+0000", 2020-08-07T15:09:42.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7875999999999999e-05 2020-08-07T15:09:42.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:42.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.271381+0000", 2020-08-07T15:09:42.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4819999999999996e-06 2020-08-07T15:09:42.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:42.026 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.271385+0000", 2020-08-07T15:09:42.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89228763300000002 2020-08-07T15:09:42.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:42.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163673+0000", 2020-08-07T15:09:42.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9949999999999996e-06 2020-08-07T15:09:42.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:42.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163678+0000", 2020-08-07T15:09:42.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5485000000000003e-05 2020-08-07T15:09:42.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163714+0000", 2020-08-07T15:09:42.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4785e-05 2020-08-07T15:09:42.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.046 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.046 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:42.047 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163728+0000", 2020-08-07T15:09:42.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7806999999999996e-05 2020-08-07T15:09:42.049 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.049 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.050 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:42.050 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163796+0000", 2020-08-07T15:09:42.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89244306299999998 2020-08-07T15:09:42.050 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.050 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:42.051 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.051 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.051 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72688 4.4 4:295d3f6e:::benchmark_data_smithi074_44586_object9085:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e129)", 2020-08-07T15:09:42.055 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:09:42.055 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.531800497999999, 2020-08-07T15:09:42.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7869782759999999, 2020-08-07T15:09:42.056 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:42.056 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:42.056 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:42.074 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:42.074 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:42.075 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72688 2020-08-07T15:09:42.075 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.075 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:42.075 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.076 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:42.076 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:09:42.076 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:42.076 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.077 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.077 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:42.077 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:09:42.077 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8721999999999999e-05 2020-08-07T15:09:42.077 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.086 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.086 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:42.086 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396882+0000", 2020-08-07T15:09:42.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2639999999999999e-06 2020-08-07T15:09:42.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:42.087 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396887+0000", 2020-08-07T15:09:42.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0767999999999999e-05 2020-08-07T15:09:42.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:42.088 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396898+0000", 2020-08-07T15:09:42.089 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1860000000000002e-06 2020-08-07T15:09:42.089 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.089 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.089 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:42.089 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396902+0000", 2020-08-07T15:09:42.094 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0037638649999999999 2020-08-07T15:09:42.094 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.095 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.095 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.095 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400666+0000", 2020-08-07T15:09:42.095 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000118077 2020-08-07T15:09:42.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.096 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.096 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:42.096 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400784+0000", 2020-08-07T15:09:42.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000136728 2020-08-07T15:09:42.096 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.097 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.097 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:09:42.097 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400921+0000", 2020-08-07T15:09:42.097 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001595443 2020-08-07T15:09:42.097 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.098 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.098 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:42.098 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.402516+0000", 2020-08-07T15:09:42.098 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.014817907 2020-08-07T15:09:42.098 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.099 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.099 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.108 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417334+0000", 2020-08-07T15:09:42.108 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.75e-06 2020-08-07T15:09:42.108 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.109 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.109 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:42.109 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417336+0000", 2020-08-07T15:09:42.109 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8066999999999996e-05 2020-08-07T15:09:42.110 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.110 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.110 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:42.111 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417424+0000", 2020-08-07T15:09:42.111 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002210573 2020-08-07T15:09:42.111 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.111 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.112 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.419635+0000", 2020-08-07T15:09:42.113 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1379000000000001e-05 2020-08-07T15:09:42.113 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.113 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.114 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.419646+0000", 2020-08-07T15:09:42.114 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.004643409 2020-08-07T15:09:42.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.116 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.117 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424289+0000", 2020-08-07T15:09:42.117 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7563000000000001e-05 2020-08-07T15:09:42.118 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.118 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.118 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.119 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424307+0000", 2020-08-07T15:09:42.119 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75943629400000001 2020-08-07T15:09:42.119 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.119 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.120 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.120 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183743+0000", 2020-08-07T15:09:42.120 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8281000000000006e-05 2020-08-07T15:09:42.120 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.121 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.121 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:42.121 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183812+0000", 2020-08-07T15:09:42.121 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7869782759999999 2020-08-07T15:09:42.121 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.123 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:42.123 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.123 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.123 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.124 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72985 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:09:42.124 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:09:42.124 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.315814678999999, 2020-08-07T15:09:42.125 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869466839999999, 2020-08-07T15:09:42.125 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:42.125 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:42.125 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:42.126 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:42.126 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:42.126 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72985 2020-08-07T15:09:42.126 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.126 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:42.127 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.127 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:42.127 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:09:42.127 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:42.128 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.128 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.128 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:42.128 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:09:42.129 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8003999999999997e-05 2020-08-07T15:09:42.140 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.238 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.238 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:42.238 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612857+0000", 2020-08-07T15:09:42.239 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.246e-06 2020-08-07T15:09:42.239 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.239 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.239 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:42.239 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612863+0000", 2020-08-07T15:09:42.240 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5146e-05 2020-08-07T15:09:42.240 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.240 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.240 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:42.240 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612878+0000", 2020-08-07T15:09:42.241 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1940000000000003e-06 2020-08-07T15:09:42.241 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.241 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.241 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:42.241 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612885+0000", 2020-08-07T15:09:42.242 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0349999999999997e-05 2020-08-07T15:09:42.242 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.242 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.242 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.242 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612935+0000", 2020-08-07T15:09:42.243 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1029999999999996e-06 2020-08-07T15:09:42.243 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.243 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.243 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.243 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612942+0000", 2020-08-07T15:09:42.244 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81140097700000002 2020-08-07T15:09:42.244 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.244 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.244 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.255 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424343+0000", 2020-08-07T15:09:42.255 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1799999999999996e-06 2020-08-07T15:09:42.255 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.255 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.256 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.266 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424351+0000", 2020-08-07T15:09:42.267 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75948665299999996 2020-08-07T15:09:42.267 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.267 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.268 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.268 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183838+0000", 2020-08-07T15:09:42.268 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000179187 2020-08-07T15:09:42.268 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.268 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.269 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:42.269 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.184017+0000", 2020-08-07T15:09:42.269 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024691600000000002 2020-08-07T15:09:42.269 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.269 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.270 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:09:42.270 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.184264+0000", 2020-08-07T15:09:42.270 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006362721 2020-08-07T15:09:42.270 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.270 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:42.271 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190627+0000", 2020-08-07T15:09:42.271 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7400000000000001e-06 2020-08-07T15:09:42.271 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.271 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:42.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190629+0000", 2020-08-07T15:09:42.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1305999999999999e-05 2020-08-07T15:09:42.273 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.273 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:42.273 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190690+0000", 2020-08-07T15:09:42.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0088507159999999998 2020-08-07T15:09:42.273 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.274 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.274 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.274 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199541+0000", 2020-08-07T15:09:42.274 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015864699999999999 2020-08-07T15:09:42.274 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.275 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.275 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.275 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199699+0000", 2020-08-07T15:09:42.275 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.23e-07 2020-08-07T15:09:42.276 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.276 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.276 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:42.276 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199700+0000", 2020-08-07T15:09:42.276 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5775e-05 2020-08-07T15:09:42.277 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.277 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.277 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:42.277 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199766+0000", 2020-08-07T15:09:42.277 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869466839999999 2020-08-07T15:09:42.278 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.278 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:42.278 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.278 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.278 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.279 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72986 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:09:42.279 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:09:42.279 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.315704097000001, 2020-08-07T15:09:42.279 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.586946441, 2020-08-07T15:09:42.279 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:42.281 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:42.281 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:42.281 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:42.281 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:42.281 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72986 2020-08-07T15:09:42.282 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.282 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:42.282 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.282 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:42.283 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:09:42.283 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:42.283 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.283 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.283 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:42.284 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:09:42.284 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9861999999999997e-05 2020-08-07T15:09:42.284 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.284 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.284 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:42.285 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612990+0000", 2020-08-07T15:09:42.285 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4350000000000001e-06 2020-08-07T15:09:42.285 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.285 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.285 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:42.286 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612993+0000", 2020-08-07T15:09:42.286 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9529999999999999e-06 2020-08-07T15:09:42.286 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.286 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.286 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:42.287 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613001+0000", 2020-08-07T15:09:42.287 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4309999999999998e-06 2020-08-07T15:09:42.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.302 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:42.303 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613004+0000", 2020-08-07T15:09:42.303 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5284000000000001e-05 2020-08-07T15:09:42.303 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.303 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.304 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.304 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613030+0000", 2020-08-07T15:09:42.304 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2400000000000001e-06 2020-08-07T15:09:42.304 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.304 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.305 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.305 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613034+0000", 2020-08-07T15:09:42.305 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81134136199999995 2020-08-07T15:09:42.305 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.305 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.305 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.306 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424375+0000", 2020-08-07T15:09:42.306 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3520000000000001e-06 2020-08-07T15:09:42.306 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.306 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.306 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.307 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424383+0000", 2020-08-07T15:09:42.307 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76064437799999995 2020-08-07T15:09:42.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.307 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.307 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185027+0000", 2020-08-07T15:09:42.308 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000119719 2020-08-07T15:09:42.308 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.308 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:42.309 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185147+0000", 2020-08-07T15:09:42.309 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016829400000000001 2020-08-07T15:09:42.309 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.309 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.309 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:09:42.330 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185315+0000", 2020-08-07T15:09:42.330 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0053821199999999998 2020-08-07T15:09:42.331 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.331 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.332 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:42.332 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190697+0000", 2020-08-07T15:09:42.333 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6899999999999996e-07 2020-08-07T15:09:42.334 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.334 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.335 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:42.335 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190698+0000", 2020-08-07T15:09:42.335 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2072000000000001e-05 2020-08-07T15:09:42.335 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.336 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:42.336 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190730+0000", 2020-08-07T15:09:42.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0088501440000000008 2020-08-07T15:09:42.336 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.336 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.336 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.337 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199580+0000", 2020-08-07T15:09:42.337 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00023199900000000001 2020-08-07T15:09:42.337 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.337 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.337 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.346 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199812+0000", 2020-08-07T15:09:42.346 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6300000000000005e-07 2020-08-07T15:09:42.349 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.350 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.350 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:42.350 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199813+0000", 2020-08-07T15:09:42.350 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3363999999999998e-05 2020-08-07T15:09:42.351 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.351 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.351 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:42.351 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199876+0000", 2020-08-07T15:09:42.351 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.586946441 2020-08-07T15:09:42.352 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.352 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:42.352 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.352 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.352 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.352 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72987 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:09:42.353 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:09:42.353 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.315608342000001, 2020-08-07T15:09:42.353 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869598709999999, 2020-08-07T15:09:42.353 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:42.353 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:42.354 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:42.354 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:42.354 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:42.354 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72987 2020-08-07T15:09:42.355 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.355 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:42.355 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.355 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:42.355 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:09:42.356 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:42.356 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.366 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:42.367 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:09:42.367 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1851999999999999e-05 2020-08-07T15:09:42.370 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.371 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.371 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:42.371 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613057+0000", 2020-08-07T15:09:42.371 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0429999999999999e-06 2020-08-07T15:09:42.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.372 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:42.372 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613060+0000", 2020-08-07T15:09:42.372 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1060000000000002e-06 2020-08-07T15:09:42.372 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.373 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.373 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:42.373 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613067+0000", 2020-08-07T15:09:42.373 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8040000000000002e-06 2020-08-07T15:09:42.373 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.374 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.374 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:42.374 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613070+0000", 2020-08-07T15:09:42.374 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6857000000000001e-05 2020-08-07T15:09:42.374 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.375 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.375 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613097+0000", 2020-08-07T15:09:42.375 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8770000000000001e-06 2020-08-07T15:09:42.375 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.376 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.376 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613102+0000", 2020-08-07T15:09:42.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81131066399999996 2020-08-07T15:09:42.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424413+0000", 2020-08-07T15:09:42.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4599999999999997e-06 2020-08-07T15:09:42.377 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424420+0000", 2020-08-07T15:09:42.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76132809700000004 2020-08-07T15:09:42.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185748+0000", 2020-08-07T15:09:42.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.4264000000000005e-05 2020-08-07T15:09:42.379 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:42.380 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185832+0000", 2020-08-07T15:09:42.380 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112803 2020-08-07T15:09:42.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:09:42.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185945+0000", 2020-08-07T15:09:42.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0089954130000000007 2020-08-07T15:09:42.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:42.382 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194941+0000", 2020-08-07T15:09:42.382 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.091e-06 2020-08-07T15:09:42.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.382 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.382 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:42.383 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194943+0000", 2020-08-07T15:09:42.383 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7423999999999996e-05 2020-08-07T15:09:42.383 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.383 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.383 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:42.384 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195020+0000", 2020-08-07T15:09:42.384 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045973300000000002 2020-08-07T15:09:42.384 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.384 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.384 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.384 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199618+0000", 2020-08-07T15:09:42.385 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000302009 2020-08-07T15:09:42.385 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.385 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.385 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.385 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199920+0000", 2020-08-07T15:09:42.386 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7999999999999996e-07 2020-08-07T15:09:42.386 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.386 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.386 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:42.386 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199920+0000", 2020-08-07T15:09:42.387 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5296999999999995e-05 2020-08-07T15:09:42.387 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.387 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.387 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:42.387 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199985+0000", 2020-08-07T15:09:42.387 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869598709999999 2020-08-07T15:09:42.388 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.388 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:42.388 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.388 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.388 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.389 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72988 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:09:42.389 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:09:42.389 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.315543762000001, 2020-08-07T15:09:42.389 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587111143, 2020-08-07T15:09:42.389 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:42.390 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:42.390 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:42.390 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:42.390 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:42.390 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72988 2020-08-07T15:09:42.391 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.391 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:42.391 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.391 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:42.391 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:09:42.392 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:42.392 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.392 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.392 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:42.392 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:09:42.393 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6563e-05 2020-08-07T15:09:42.393 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.393 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.393 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:42.393 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613117+0000", 2020-08-07T15:09:42.394 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7180000000000001e-06 2020-08-07T15:09:42.394 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.394 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.394 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:42.394 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613119+0000", 2020-08-07T15:09:42.394 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1789999999999999e-06 2020-08-07T15:09:42.395 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.395 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.395 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:42.395 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613124+0000", 2020-08-07T15:09:42.395 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3539999999999998e-06 2020-08-07T15:09:42.396 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.396 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.396 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:42.396 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613127+0000", 2020-08-07T15:09:42.396 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3697e-05 2020-08-07T15:09:42.397 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.397 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.397 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.397 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613151+0000", 2020-08-07T15:09:42.397 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7559999999999999e-06 2020-08-07T15:09:42.397 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.398 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.398 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.398 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613155+0000", 2020-08-07T15:09:42.398 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81128859600000003 2020-08-07T15:09:42.398 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.399 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.399 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424444+0000", 2020-08-07T15:09:42.399 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9619999999999996e-06 2020-08-07T15:09:42.399 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.400 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.400 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424451+0000", 2020-08-07T15:09:42.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76286712899999998 2020-08-07T15:09:42.400 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.401 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.401 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.401 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187318+0000", 2020-08-07T15:09:42.401 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011616300000000001 2020-08-07T15:09:42.401 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.401 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.402 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:42.402 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187434+0000", 2020-08-07T15:09:42.402 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019347400000000001 2020-08-07T15:09:42.402 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.402 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.403 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:09:42.403 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187628+0000", 2020-08-07T15:09:42.403 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074049090000000003 2020-08-07T15:09:42.403 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.403 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.404 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:42.404 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195033+0000", 2020-08-07T15:09:42.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1090000000000001e-06 2020-08-07T15:09:42.404 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.404 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.404 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:42.405 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195034+0000", 2020-08-07T15:09:42.405 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8566999999999997e-05 2020-08-07T15:09:42.405 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.405 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:42.406 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195092+0000", 2020-08-07T15:09:42.406 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004570457 2020-08-07T15:09:42.406 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.406 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.406 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.407 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199663+0000", 2020-08-07T15:09:42.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00046757899999999998 2020-08-07T15:09:42.407 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.407 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.407 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.408 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200130+0000", 2020-08-07T15:09:42.408 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6700000000000002e-07 2020-08-07T15:09:42.408 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.408 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.408 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:42.409 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200131+0000", 2020-08-07T15:09:42.409 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9963999999999996e-05 2020-08-07T15:09:42.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.409 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.409 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:42.409 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200201+0000", 2020-08-07T15:09:42.410 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587111143 2020-08-07T15:09:42.410 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.410 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:42.410 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.410 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.411 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.411 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72989 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:09:42.411 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:09:42.411 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.315487374, 2020-08-07T15:09:42.411 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5872506399999999, 2020-08-07T15:09:42.412 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:42.412 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:42.412 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:42.412 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:42.412 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:42.413 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72989 2020-08-07T15:09:42.413 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.413 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:42.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.413 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:42.413 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:09:42.414 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:42.414 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.414 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.414 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:42.414 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:09:42.415 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6081999999999999e-05 2020-08-07T15:09:42.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.415 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:42.415 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613172+0000", 2020-08-07T15:09:42.416 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6570000000000001e-06 2020-08-07T15:09:42.416 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.416 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.417 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:42.417 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613175+0000", 2020-08-07T15:09:42.417 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7740000000000001e-06 2020-08-07T15:09:42.418 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.418 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.418 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:42.418 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613181+0000", 2020-08-07T15:09:42.418 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2759999999999999e-06 2020-08-07T15:09:42.419 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.419 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.419 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:42.419 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613183+0000", 2020-08-07T15:09:42.420 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3892000000000001e-05 2020-08-07T15:09:42.420 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.420 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.420 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.420 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613207+0000", 2020-08-07T15:09:42.420 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9870000000000001e-06 2020-08-07T15:09:42.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.421 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.421 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.421 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613211+0000", 2020-08-07T15:09:42.421 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81126230700000002 2020-08-07T15:09:42.422 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.422 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.422 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424473+0000", 2020-08-07T15:09:42.422 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1460000000000001e-06 2020-08-07T15:09:42.423 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.423 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.423 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424480+0000", 2020-08-07T15:09:42.423 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76413073099999995 2020-08-07T15:09:42.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.424 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.424 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.424 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188610+0000", 2020-08-07T15:09:42.424 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011278899999999999 2020-08-07T15:09:42.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.425 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:42.425 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188723+0000", 2020-08-07T15:09:42.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000213422 2020-08-07T15:09:42.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.426 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.426 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:09:42.426 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188936+0000", 2020-08-07T15:09:42.426 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0061672929999999999 2020-08-07T15:09:42.426 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.427 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:42.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195104+0000", 2020-08-07T15:09:42.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8199999999999999e-07 2020-08-07T15:09:42.427 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.427 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:42.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195105+0000", 2020-08-07T15:09:42.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2056000000000004e-05 2020-08-07T15:09:42.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.429 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:42.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195167+0000", 2020-08-07T15:09:42.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0051009769999999996 2020-08-07T15:09:42.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.430 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.430 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200268+0000", 2020-08-07T15:09:42.430 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9428000000000001e-05 2020-08-07T15:09:42.430 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.430 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.431 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.431 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200327+0000", 2020-08-07T15:09:42.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9100000000000003e-07 2020-08-07T15:09:42.431 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.432 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:42.432 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200328+0000", 2020-08-07T15:09:42.432 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9250000000000003e-05 2020-08-07T15:09:42.432 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.432 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.433 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:42.433 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200397+0000", 2020-08-07T15:09:42.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5872506399999999 2020-08-07T15:09:42.433 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.433 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:42.434 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.434 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.434 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.434 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72990 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:09:42.434 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:09:42.435 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.315430081999999, 2020-08-07T15:09:42.435 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587335105, 2020-08-07T15:09:42.435 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:42.435 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:42.435 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:42.435 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:42.436 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:42.436 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72990 2020-08-07T15:09:42.436 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.436 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:42.436 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.437 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:42.437 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:09:42.438 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:42.438 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.438 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.438 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:42.438 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:09:42.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5276999999999999e-05 2020-08-07T15:09:42.439 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.439 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.439 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:42.439 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613229+0000", 2020-08-07T15:09:42.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7219999999999999e-06 2020-08-07T15:09:42.440 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.440 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:42.440 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613232+0000", 2020-08-07T15:09:42.440 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1170000000000003e-06 2020-08-07T15:09:42.441 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.441 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.441 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:42.441 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613237+0000", 2020-08-07T15:09:42.441 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2309999999999999e-06 2020-08-07T15:09:42.442 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.442 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.442 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:42.442 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613239+0000", 2020-08-07T15:09:42.442 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2469000000000001e-05 2020-08-07T15:09:42.443 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.443 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.443 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.443 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613261+0000", 2020-08-07T15:09:42.443 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9280000000000001e-06 2020-08-07T15:09:42.444 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.444 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613266+0000", 2020-08-07T15:09:42.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81123414599999999 2020-08-07T15:09:42.444 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.445 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.445 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.445 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424501+0000", 2020-08-07T15:09:42.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5300000000000002e-06 2020-08-07T15:09:42.445 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.446 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.446 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.446 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424507+0000", 2020-08-07T15:09:42.446 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76488660200000003 2020-08-07T15:09:42.446 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.447 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.447 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189394+0000", 2020-08-07T15:09:42.447 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6878000000000003e-05 2020-08-07T15:09:42.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:42.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189491+0000", 2020-08-07T15:09:42.448 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000204477 2020-08-07T15:09:42.448 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.449 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:09:42.449 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189695+0000", 2020-08-07T15:09:42.449 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0065180940000000003 2020-08-07T15:09:42.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.449 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.450 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:42.450 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196213+0000", 2020-08-07T15:09:42.450 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4920000000000001e-06 2020-08-07T15:09:42.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.451 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:42.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196215+0000", 2020-08-07T15:09:42.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9950000000000006e-05 2020-08-07T15:09:42.451 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.451 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.451 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:42.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196285+0000", 2020-08-07T15:09:42.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004163674 2020-08-07T15:09:42.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.453 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200448+0000", 2020-08-07T15:09:42.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6919999999999999e-05 2020-08-07T15:09:42.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.453 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.454 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200485+0000", 2020-08-07T15:09:42.454 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1699999999999998e-07 2020-08-07T15:09:42.454 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.454 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.454 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:42.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200486+0000", 2020-08-07T15:09:42.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3081000000000002e-05 2020-08-07T15:09:42.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:42.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200539+0000", 2020-08-07T15:09:42.456 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587335105 2020-08-07T15:09:42.456 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.456 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:42.456 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.457 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72991 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:09:42.457 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:09:42.457 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.315374556999998, 2020-08-07T15:09:42.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5876210340000001, 2020-08-07T15:09:42.458 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:42.458 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:42.458 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:42.458 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:42.458 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:42.459 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72991 2020-08-07T15:09:42.459 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.459 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:42.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:42.459 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:09:42.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:42.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.460 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.460 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:42.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:09:42.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5763999999999999e-05 2020-08-07T15:09:42.461 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.461 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.461 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:42.461 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613285+0000", 2020-08-07T15:09:42.462 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6479999999999999e-06 2020-08-07T15:09:42.462 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.462 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.462 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:42.462 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613288+0000", 2020-08-07T15:09:42.463 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1909999999999999e-06 2020-08-07T15:09:42.463 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.463 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.463 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:42.465 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613293+0000", 2020-08-07T15:09:42.466 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2560000000000001e-06 2020-08-07T15:09:42.466 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.466 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.466 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:42.467 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613295+0000", 2020-08-07T15:09:42.467 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2561e-05 2020-08-07T15:09:42.467 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.468 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.468 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.468 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613318+0000", 2020-08-07T15:09:42.469 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6349999999999999e-06 2020-08-07T15:09:42.469 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.469 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.469 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.469 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613321+0000", 2020-08-07T15:09:42.470 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81120592400000002 2020-08-07T15:09:42.470 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.470 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.471 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.471 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424527+0000", 2020-08-07T15:09:42.471 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8540000000000002e-06 2020-08-07T15:09:42.471 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.471 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.472 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.472 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424533+0000", 2020-08-07T15:09:42.472 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76623392800000001 2020-08-07T15:09:42.472 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.472 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.472 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.473 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190767+0000", 2020-08-07T15:09:42.473 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3990000000000001e-05 2020-08-07T15:09:42.473 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.474 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.474 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:42.474 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190841+0000", 2020-08-07T15:09:42.474 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011444599999999999 2020-08-07T15:09:42.474 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.475 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.475 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:09:42.475 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190955+0000", 2020-08-07T15:09:42.475 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0053397499999999999 2020-08-07T15:09:42.475 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.476 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.476 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:42.476 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196295+0000", 2020-08-07T15:09:42.476 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006e-06 2020-08-07T15:09:42.476 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.476 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.477 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:42.477 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196296+0000", 2020-08-07T15:09:42.477 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8940999999999997e-05 2020-08-07T15:09:42.477 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.477 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.478 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:42.478 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196355+0000", 2020-08-07T15:09:42.478 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042276700000000002 2020-08-07T15:09:42.478 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.478 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.479 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.479 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200583+0000", 2020-08-07T15:09:42.479 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00026035700000000001 2020-08-07T15:09:42.479 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.479 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.479 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.480 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200843+0000", 2020-08-07T15:09:42.480 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6700000000000003e-07 2020-08-07T15:09:42.480 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.480 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:42.481 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200844+0000", 2020-08-07T15:09:42.481 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6545999999999999e-05 2020-08-07T15:09:42.481 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.481 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.481 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:42.482 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200880+0000", 2020-08-07T15:09:42.482 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5876210340000001 2020-08-07T15:09:42.482 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.482 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:42.482 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.483 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.483 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.483 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72992 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:09:42.484 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:09:42.484 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.314287078, 2020-08-07T15:09:42.484 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5866241210000001, 2020-08-07T15:09:42.484 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:42.484 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:42.485 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:42.485 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:42.485 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:42.485 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72992 2020-08-07T15:09:42.486 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.486 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:42.487 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.487 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:42.487 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:09:42.487 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:42.487 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.487 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:42.488 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:09:42.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7543e-05 2020-08-07T15:09:42.488 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.489 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:42.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614374+0000", 2020-08-07T15:09:42.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9740000000000004e-06 2020-08-07T15:09:42.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.489 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.490 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:42.490 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614378+0000", 2020-08-07T15:09:42.490 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2660000000000007e-06 2020-08-07T15:09:42.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.491 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.491 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:42.491 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614387+0000", 2020-08-07T15:09:42.491 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2950000000000001e-06 2020-08-07T15:09:42.492 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.492 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.492 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:42.492 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614393+0000", 2020-08-07T15:09:42.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7279000000000001e-05 2020-08-07T15:09:42.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.493 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.493 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614420+0000", 2020-08-07T15:09:42.494 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9040000000000002e-06 2020-08-07T15:09:42.494 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.494 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.494 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614424+0000", 2020-08-07T15:09:42.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81012815699999996 2020-08-07T15:09:42.495 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.495 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.495 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.495 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424552+0000", 2020-08-07T15:09:42.496 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2879999999999998e-06 2020-08-07T15:09:42.496 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.496 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.496 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.496 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424558+0000", 2020-08-07T15:09:42.496 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76678665400000001 2020-08-07T15:09:42.497 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.497 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.497 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.497 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191345+0000", 2020-08-07T15:09:42.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010819600000000001 2020-08-07T15:09:42.498 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.498 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.498 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:42.498 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191453+0000", 2020-08-07T15:09:42.498 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000170829 2020-08-07T15:09:42.499 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.499 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.499 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:09:42.499 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191624+0000", 2020-08-07T15:09:42.499 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0047386679999999997 2020-08-07T15:09:42.500 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.500 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.500 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:42.500 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196363+0000", 2020-08-07T15:09:42.500 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6000000000000003e-07 2020-08-07T15:09:42.500 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.501 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.501 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:42.501 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196363+0000", 2020-08-07T15:09:42.501 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3421000000000002e-05 2020-08-07T15:09:42.501 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.502 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.502 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:42.502 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196417+0000", 2020-08-07T15:09:42.502 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042225539999999999 2020-08-07T15:09:42.502 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.503 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.503 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.503 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200639+0000", 2020-08-07T15:09:42.503 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00028031399999999999 2020-08-07T15:09:42.503 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.504 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.504 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.504 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200920+0000", 2020-08-07T15:09:42.504 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0299999999999999e-07 2020-08-07T15:09:42.504 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.504 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.505 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:42.505 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200920+0000", 2020-08-07T15:09:42.505 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0516000000000002e-05 2020-08-07T15:09:42.506 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.507 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.507 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:42.507 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200971+0000", 2020-08-07T15:09:42.507 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5866241210000001 2020-08-07T15:09:42.508 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.508 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:42.508 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.508 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.508 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.509 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73017 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:09:42.509 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:09:42.509 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.258778825, 2020-08-07T15:09:42.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.531181608, 2020-08-07T15:09:42.509 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:42.510 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:42.510 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:42.510 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:42.510 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:42.510 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73017 2020-08-07T15:09:42.511 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.511 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:42.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:42.511 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:09:42.511 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:42.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.512 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.512 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:42.512 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:09:42.512 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3018e-05 2020-08-07T15:09:42.513 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.513 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.513 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:42.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669898+0000", 2020-08-07T15:09:42.513 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5340000000000002e-06 2020-08-07T15:09:42.514 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.514 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:42.514 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669905+0000", 2020-08-07T15:09:42.514 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3307e-05 2020-08-07T15:09:42.515 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.515 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.515 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:42.515 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669919+0000", 2020-08-07T15:09:42.515 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7729999999999999e-06 2020-08-07T15:09:42.515 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.516 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.516 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:42.516 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669927+0000", 2020-08-07T15:09:42.516 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0361000000000001e-05 2020-08-07T15:09:42.516 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.517 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.517 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.517 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669977+0000", 2020-08-07T15:09:42.517 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0049999999999993e-06 2020-08-07T15:09:42.517 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.518 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.518 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669986+0000", 2020-08-07T15:09:42.518 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75459235199999997 2020-08-07T15:09:42.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.519 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.519 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424578+0000", 2020-08-07T15:09:42.519 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.925e-06 2020-08-07T15:09:42.519 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.519 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.520 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424584+0000", 2020-08-07T15:09:42.520 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76741703900000002 2020-08-07T15:09:42.520 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.520 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.521 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.521 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192001+0000", 2020-08-07T15:09:42.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000181196 2020-08-07T15:09:42.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.521 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.541 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:42.541 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192182+0000", 2020-08-07T15:09:42.542 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00021149299999999999 2020-08-07T15:09:42.542 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.542 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.542 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:09:42.542 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192394+0000", 2020-08-07T15:09:42.543 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004031242 2020-08-07T15:09:42.543 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.543 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.543 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:42.543 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196425+0000", 2020-08-07T15:09:42.544 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7800000000000001e-07 2020-08-07T15:09:42.544 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.544 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.544 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:42.544 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196426+0000", 2020-08-07T15:09:42.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5244999999999997e-05 2020-08-07T15:09:42.545 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.545 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:42.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196481+0000", 2020-08-07T15:09:42.546 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041989360000000003 2020-08-07T15:09:42.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.546 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.546 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200680+0000", 2020-08-07T15:09:42.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00032415299999999998 2020-08-07T15:09:42.548 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.548 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.548 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201004+0000", 2020-08-07T15:09:42.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9299999999999999e-07 2020-08-07T15:09:42.549 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.549 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.549 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:42.549 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201005+0000", 2020-08-07T15:09:42.549 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1857999999999999e-05 2020-08-07T15:09:42.550 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.550 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.550 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:42.550 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201037+0000", 2020-08-07T15:09:42.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.531181608 2020-08-07T15:09:42.551 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.551 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:42.551 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.551 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.552 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73018 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:09:42.552 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:09:42.552 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.257811019999998, 2020-08-07T15:09:42.552 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.530301616, 2020-08-07T15:09:42.553 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:42.553 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:42.553 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:42.553 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:42.554 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:42.554 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73018 2020-08-07T15:09:42.554 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.554 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:42.554 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.555 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:42.555 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:09:42.555 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:42.555 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.555 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.555 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:42.556 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:09:42.556 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9747000000000002e-05 2020-08-07T15:09:42.556 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:42.557 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670852+0000", 2020-08-07T15:09:42.557 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9430000000000002e-06 2020-08-07T15:09:42.557 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.557 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.557 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:42.558 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670856+0000", 2020-08-07T15:09:42.558 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0649999999999999e-06 2020-08-07T15:09:42.558 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.559 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.559 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:42.559 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670864+0000", 2020-08-07T15:09:42.559 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.924e-06 2020-08-07T15:09:42.560 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.560 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.560 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:42.560 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670868+0000", 2020-08-07T15:09:42.560 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6205000000000003e-05 2020-08-07T15:09:42.561 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.561 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.561 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.561 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670905+0000", 2020-08-07T15:09:42.561 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3289999999999996e-06 2020-08-07T15:09:42.561 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.562 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.562 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.562 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670912+0000", 2020-08-07T15:09:42.562 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75370200200000004 2020-08-07T15:09:42.562 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.563 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.573 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.573 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424614+0000", 2020-08-07T15:09:42.573 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5150000000000003e-06 2020-08-07T15:09:42.573 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.574 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.574 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.574 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424620+0000", 2020-08-07T15:09:42.574 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.768200153 2020-08-07T15:09:42.574 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.575 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.575 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192821+0000", 2020-08-07T15:09:42.575 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000111764 2020-08-07T15:09:42.575 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.576 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.576 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:42.576 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192932+0000", 2020-08-07T15:09:42.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017542400000000001 2020-08-07T15:09:42.576 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.577 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.577 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:09:42.577 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193108+0000", 2020-08-07T15:09:42.577 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004100851 2020-08-07T15:09:42.577 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.578 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.588 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:42.588 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197209+0000", 2020-08-07T15:09:42.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0100000000000001e-06 2020-08-07T15:09:42.589 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.608 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.608 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:42.608 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197210+0000", 2020-08-07T15:09:42.608 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9672999999999995e-05 2020-08-07T15:09:42.609 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.609 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.609 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:42.609 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197279+0000", 2020-08-07T15:09:42.610 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034275109999999998 2020-08-07T15:09:42.610 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.610 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.610 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.610 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200707+0000", 2020-08-07T15:09:42.610 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00036853100000000001 2020-08-07T15:09:42.611 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.611 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.611 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.611 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201075+0000", 2020-08-07T15:09:42.611 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8599999999999998e-07 2020-08-07T15:09:42.612 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.612 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.612 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:42.612 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201076+0000", 2020-08-07T15:09:42.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8483000000000003e-05 2020-08-07T15:09:42.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.613 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:42.613 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201124+0000", 2020-08-07T15:09:42.613 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.530301616 2020-08-07T15:09:42.625 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.625 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:42.625 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.626 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.626 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.626 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73019 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:09:42.626 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:09:42.626 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.257163369000001, 2020-08-07T15:09:42.627 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.529718224, 2020-08-07T15:09:42.627 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:42.642 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:42.643 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:42.643 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:42.643 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:42.643 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73019 2020-08-07T15:09:42.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.644 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:42.644 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.644 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:42.644 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:09:42.645 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:42.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:42.645 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:09:42.666 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3309000000000002e-05 2020-08-07T15:09:42.666 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.666 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.667 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:42.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671504+0000", 2020-08-07T15:09:42.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6619999999999998e-06 2020-08-07T15:09:42.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:42.668 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671507+0000", 2020-08-07T15:09:42.668 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2279999999999998e-06 2020-08-07T15:09:42.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:42.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671516+0000", 2020-08-07T15:09:42.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4860000000000002e-06 2020-08-07T15:09:42.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.675 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:42.675 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671519+0000", 2020-08-07T15:09:42.675 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0564000000000001e-05 2020-08-07T15:09:42.676 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.676 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.676 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.676 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671550+0000", 2020-08-07T15:09:42.676 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1029999999999998e-06 2020-08-07T15:09:42.677 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.677 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671556+0000", 2020-08-07T15:09:42.677 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75308759199999997 2020-08-07T15:09:42.678 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.678 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.678 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.678 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424643+0000", 2020-08-07T15:09:42.678 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9179999999999997e-06 2020-08-07T15:09:42.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.679 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424650+0000", 2020-08-07T15:09:42.679 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76890373599999995 2020-08-07T15:09:42.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.680 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.680 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193554+0000", 2020-08-07T15:09:42.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012345600000000001 2020-08-07T15:09:42.681 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.681 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:42.681 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193677+0000", 2020-08-07T15:09:42.681 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013528300000000001 2020-08-07T15:09:42.681 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.682 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.682 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:09:42.683 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193813+0000", 2020-08-07T15:09:42.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034777710000000002 2020-08-07T15:09:42.683 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.684 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.684 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:42.684 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197291+0000", 2020-08-07T15:09:42.684 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8000000000000004e-07 2020-08-07T15:09:42.684 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.685 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.685 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:42.685 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197291+0000", 2020-08-07T15:09:42.685 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9176999999999999e-05 2020-08-07T15:09:42.685 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.686 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.686 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:42.686 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197351+0000", 2020-08-07T15:09:42.686 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033931009999999999 2020-08-07T15:09:42.686 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.687 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.687 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.687 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200744+0000", 2020-08-07T15:09:42.687 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00041308000000000002 2020-08-07T15:09:42.687 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.687 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.688 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.688 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201157+0000", 2020-08-07T15:09:42.688 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8999999999999998e-07 2020-08-07T15:09:42.688 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.688 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.689 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:42.689 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201157+0000", 2020-08-07T15:09:42.689 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1588000000000003e-05 2020-08-07T15:09:42.689 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.689 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.690 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:42.690 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201189+0000", 2020-08-07T15:09:42.690 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.529718224 2020-08-07T15:09:42.690 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.690 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:42.691 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.691 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.691 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.691 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73020 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:09:42.691 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:09:42.691 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.188825509000001, 2020-08-07T15:09:42.692 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.461468373, 2020-08-07T15:09:42.692 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:42.692 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:42.692 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:42.692 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:42.693 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:42.693 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73020 2020-08-07T15:09:42.693 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.693 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:42.693 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.694 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:42.694 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:09:42.694 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:42.694 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.694 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.695 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:42.695 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:09:42.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.777e-05 2020-08-07T15:09:42.695 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.696 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:42.696 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739836+0000", 2020-08-07T15:09:42.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0180000000000003e-06 2020-08-07T15:09:42.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.696 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:42.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739840+0000", 2020-08-07T15:09:42.697 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6930000000000002e-06 2020-08-07T15:09:42.697 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:42.698 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739850+0000", 2020-08-07T15:09:42.698 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3619999999999999e-06 2020-08-07T15:09:42.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.698 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.698 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:42.699 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739854+0000", 2020-08-07T15:09:42.699 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7432999999999999e-05 2020-08-07T15:09:42.699 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.699 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.699 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.700 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739892+0000", 2020-08-07T15:09:42.700 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1149999999999996e-06 2020-08-07T15:09:42.700 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.700 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.700 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739899+0000", 2020-08-07T15:09:42.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68477883299999998 2020-08-07T15:09:42.701 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.701 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.701 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424677+0000", 2020-08-07T15:09:42.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7710000000000001e-06 2020-08-07T15:09:42.702 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.702 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.702 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.702 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424684+0000", 2020-08-07T15:09:42.703 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76949970700000003 2020-08-07T15:09:42.703 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.703 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.704 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194184+0000", 2020-08-07T15:09:42.704 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3957999999999997e-05 2020-08-07T15:09:42.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.704 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:42.704 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194278+0000", 2020-08-07T15:09:42.705 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017489599999999999 2020-08-07T15:09:42.705 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.705 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.705 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:09:42.705 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194453+0000", 2020-08-07T15:09:42.706 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002908908 2020-08-07T15:09:42.706 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.707 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:42.707 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197362+0000", 2020-08-07T15:09:42.707 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7299999999999995e-07 2020-08-07T15:09:42.707 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.708 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:42.708 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197362+0000", 2020-08-07T15:09:42.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6848000000000002e-05 2020-08-07T15:09:42.708 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.709 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.709 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:42.709 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197419+0000", 2020-08-07T15:09:42.709 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033597079999999999 2020-08-07T15:09:42.709 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.710 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.711 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.711 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200779+0000", 2020-08-07T15:09:42.711 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00044797899999999999 2020-08-07T15:09:42.711 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.712 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.712 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201227+0000", 2020-08-07T15:09:42.712 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7100000000000002e-07 2020-08-07T15:09:42.712 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.712 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.713 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:42.713 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201227+0000", 2020-08-07T15:09:42.713 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9230000000000001e-05 2020-08-07T15:09:42.713 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.713 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.714 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:42.714 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201277+0000", 2020-08-07T15:09:42.714 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.461468373 2020-08-07T15:09:42.714 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.714 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:42.715 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.715 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.715 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.715 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73021 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:09:42.715 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:09:42.716 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.125791902, 2020-08-07T15:09:42.716 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.398505079, 2020-08-07T15:09:42.716 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:42.716 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:42.716 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:42.717 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:42.717 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:42.717 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73021 2020-08-07T15:09:42.717 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.717 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:42.718 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.718 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:42.718 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:09:42.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:42.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.718 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:42.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:09:42.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7429000000000001e-05 2020-08-07T15:09:42.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:42.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802869+0000", 2020-08-07T15:09:42.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7840000000000001e-06 2020-08-07T15:09:42.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:42.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802873+0000", 2020-08-07T15:09:42.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0119999999999994e-06 2020-08-07T15:09:42.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.721 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.722 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:42.722 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802882+0000", 2020-08-07T15:09:42.722 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.04e-06 2020-08-07T15:09:42.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.722 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:42.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802887+0000", 2020-08-07T15:09:42.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7076000000000002e-05 2020-08-07T15:09:42.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.723 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.724 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802924+0000", 2020-08-07T15:09:42.724 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9280000000000004e-06 2020-08-07T15:09:42.724 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.724 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.724 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.725 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802931+0000", 2020-08-07T15:09:42.725 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.62178051000000001 2020-08-07T15:09:42.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.726 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.726 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424712+0000", 2020-08-07T15:09:42.726 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4749999999999996e-06 2020-08-07T15:09:42.726 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.726 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.726 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.727 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424719+0000", 2020-08-07T15:09:42.727 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77047786799999995 2020-08-07T15:09:42.727 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.727 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.727 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.728 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195197+0000", 2020-08-07T15:09:42.728 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000101909 2020-08-07T15:09:42.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:42.729 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195299+0000", 2020-08-07T15:09:42.730 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000248247 2020-08-07T15:09:42.730 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.730 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.730 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:09:42.730 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195547+0000", 2020-08-07T15:09:42.730 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033458699999999999 2020-08-07T15:09:42.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.731 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:42.731 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198893+0000", 2020-08-07T15:09:42.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0419999999999999e-06 2020-08-07T15:09:42.732 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.732 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:42.732 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198894+0000", 2020-08-07T15:09:42.732 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9354999999999997e-05 2020-08-07T15:09:42.733 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.733 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:42.733 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198953+0000", 2020-08-07T15:09:42.733 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001861096 2020-08-07T15:09:42.734 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.734 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.734 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.735 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200815+0000", 2020-08-07T15:09:42.735 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00049697299999999995 2020-08-07T15:09:42.735 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.736 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.736 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.736 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201311+0000", 2020-08-07T15:09:42.736 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7099999999999998e-07 2020-08-07T15:09:42.737 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.737 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.737 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:42.737 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201312+0000", 2020-08-07T15:09:42.737 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5194000000000002e-05 2020-08-07T15:09:42.737 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.738 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.738 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:42.738 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201347+0000", 2020-08-07T15:09:42.738 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.398505079 2020-08-07T15:09:42.738 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.739 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:42.739 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.739 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.739 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.739 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73022 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:09:42.740 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:09:42.754 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 16.044619847, 2020-08-07T15:09:42.754 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3238320669999999, 2020-08-07T15:09:42.754 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:42.754 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:42.755 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:42.755 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:42.755 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:42.755 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73022 2020-08-07T15:09:42.755 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.756 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:42.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:42.756 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:09:42.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:42.757 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.757 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:42.757 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:09:42.757 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6769999999999999e-05 2020-08-07T15:09:42.758 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.758 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.758 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:42.758 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884041+0000", 2020-08-07T15:09:42.758 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.703e-06 2020-08-07T15:09:42.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.759 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:42.759 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884044+0000", 2020-08-07T15:09:42.759 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5949999999999999e-06 2020-08-07T15:09:42.760 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.760 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.760 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:42.760 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884053+0000", 2020-08-07T15:09:42.760 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000003e-06 2020-08-07T15:09:42.760 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.761 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:42.761 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884058+0000", 2020-08-07T15:09:42.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7281999999999999e-05 2020-08-07T15:09:42.761 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884105+0000", 2020-08-07T15:09:42.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.044e-05 2020-08-07T15:09:42.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884116+0000", 2020-08-07T15:09:42.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.54062884300000003 2020-08-07T15:09:42.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424745+0000", 2020-08-07T15:09:42.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3149999999999998e-06 2020-08-07T15:09:42.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.779 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.779 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424754+0000", 2020-08-07T15:09:42.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77175732200000002 2020-08-07T15:09:42.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.780 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.780 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196511+0000", 2020-08-07T15:09:42.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010110799999999999 2020-08-07T15:09:42.780 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.781 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:42.781 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196612+0000", 2020-08-07T15:09:42.781 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017543100000000001 2020-08-07T15:09:42.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.802 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:09:42.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196788+0000", 2020-08-07T15:09:42.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032134569999999999 2020-08-07T15:09:42.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:42.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200001+0000", 2020-08-07T15:09:42.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.071e-06 2020-08-07T15:09:42.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:42.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200002+0000", 2020-08-07T15:09:42.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2527999999999995e-05 2020-08-07T15:09:42.806 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:42.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200065+0000", 2020-08-07T15:09:42.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001327433 2020-08-07T15:09:42.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201392+0000", 2020-08-07T15:09:42.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063854030000000004 2020-08-07T15:09:42.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.808 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.809 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207778+0000", 2020-08-07T15:09:42.809 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.285e-06 2020-08-07T15:09:42.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.809 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:42.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207779+0000", 2020-08-07T15:09:42.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7070999999999995e-05 2020-08-07T15:09:42.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:42.828 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207846+0000", 2020-08-07T15:09:42.828 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3238320669999999 2020-08-07T15:09:42.828 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.828 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:42.829 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.829 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.829 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.829 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73023 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:09:42.829 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:09:42.830 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 15.931458206, 2020-08-07T15:09:42.830 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2113701729999999, 2020-08-07T15:09:42.830 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:42.830 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:42.830 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:42.831 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:42.831 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:42.831 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73023 2020-08-07T15:09:42.831 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.831 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:42.831 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.832 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:42.832 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:09:42.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:42.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:42.833 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:09:42.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4105e-05 2020-08-07T15:09:42.833 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.865 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:42.866 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997210+0000", 2020-08-07T15:09:42.866 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1089999999999997e-06 2020-08-07T15:09:42.866 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.866 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.866 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:42.867 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997215+0000", 2020-08-07T15:09:42.867 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7040000000000006e-06 2020-08-07T15:09:42.867 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.867 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.867 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:42.868 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997224+0000", 2020-08-07T15:09:42.868 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4439999999999999e-06 2020-08-07T15:09:42.868 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.886 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.886 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:42.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997230+0000", 2020-08-07T15:09:42.887 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5977999999999999e-05 2020-08-07T15:09:42.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.887 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997276+0000", 2020-08-07T15:09:42.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2660000000000007e-06 2020-08-07T15:09:42.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997285+0000", 2020-08-07T15:09:42.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.42749189599999998 2020-08-07T15:09:42.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424777+0000", 2020-08-07T15:09:42.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8940000000000004e-06 2020-08-07T15:09:42.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.906 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.906 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424784+0000", 2020-08-07T15:09:42.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77268199699999995 2020-08-07T15:09:42.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197466+0000", 2020-08-07T15:09:42.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011277699999999999 2020-08-07T15:09:42.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.908 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:42.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197579+0000", 2020-08-07T15:09:42.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018970599999999999 2020-08-07T15:09:42.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:09:42.926 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197768+0000", 2020-08-07T15:09:42.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043168529999999998 2020-08-07T15:09:42.926 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:42.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202085+0000", 2020-08-07T15:09:42.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.186e-06 2020-08-07T15:09:42.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:42.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202086+0000", 2020-08-07T15:09:42.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8975999999999999e-05 2020-08-07T15:09:42.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:42.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202145+0000", 2020-08-07T15:09:42.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00035331499999999999 2020-08-07T15:09:42.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202499+0000", 2020-08-07T15:09:42.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0059703159999999998 2020-08-07T15:09:42.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:42.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208469+0000", 2020-08-07T15:09:42.931 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7400000000000002e-07 2020-08-07T15:09:42.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:42.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208470+0000", 2020-08-07T15:09:42.932 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5877000000000006e-05 2020-08-07T15:09:42.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.932 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.932 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:42.933 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208546+0000", 2020-08-07T15:09:42.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2113701729999999 2020-08-07T15:09:42.933 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.933 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:42.954 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:42.954 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.954 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.955 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73024 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e131)", 2020-08-07T15:09:42.955 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:26.502031+0000", 2020-08-07T15:09:42.955 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 15.426603059, 2020-08-07T15:09:42.955 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.70660062599999995, 2020-08-07T15:09:42.955 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:42.956 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:42.956 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:42.956 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:42.956 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:42.956 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73024 2020-08-07T15:09:42.957 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.957 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:42.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.957 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:42.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502031+0000", 2020-08-07T15:09:42.958 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:42.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:42.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502031+0000", 2020-08-07T15:09:42.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0858999999999997e-05 2020-08-07T15:09:42.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:42.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502062+0000", 2020-08-07T15:09:42.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1159999999999998e-06 2020-08-07T15:09:42.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:42.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502067+0000", 2020-08-07T15:09:42.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1810999999999999e-05 2020-08-07T15:09:42.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:42.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502078+0000", 2020-08-07T15:09:42.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2500000000000006e-06 2020-08-07T15:09:42.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.983 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:42.983 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502087+0000", 2020-08-07T15:09:42.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3461999999999996e-05 2020-08-07T15:09:42.983 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.983 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.984 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:42.984 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502150+0000", 2020-08-07T15:09:42.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3108999999999999e-05 2020-08-07T15:09:42.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:42.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:42.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502163+0000", 2020-08-07T15:09:42.985 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69604141200000003 2020-08-07T15:09:42.985 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:42.985 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.002 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:43.002 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198205+0000", 2020-08-07T15:09:43.002 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013494700000000001 2020-08-07T15:09:43.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:43.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198340+0000", 2020-08-07T15:09:43.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000169696 2020-08-07T15:09:43.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:09:43.004 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198509+0000", 2020-08-07T15:09:43.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0036466960000000001 2020-08-07T15:09:43.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:43.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202156+0000", 2020-08-07T15:09:43.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.16e-07 2020-08-07T15:09:43.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:43.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202157+0000", 2020-08-07T15:09:43.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5290000000000002e-05 2020-08-07T15:09:43.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:43.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202202+0000", 2020-08-07T15:09:43.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00061786700000000003 2020-08-07T15:09:43.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:43.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202820+0000", 2020-08-07T15:09:43.008 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0057728759999999997 2020-08-07T15:09:43.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.009 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.009 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:43.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208593+0000", 2020-08-07T15:09:43.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4000000000000001e-07 2020-08-07T15:09:43.009 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.030 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.030 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:43.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208594+0000", 2020-08-07T15:09:43.030 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7679000000000003e-05 2020-08-07T15:09:43.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:43.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208631+0000", 2020-08-07T15:09:43.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.70660062599999995 2020-08-07T15:09:43.032 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:43.032 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:43.032 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:43.032 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:43.032 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:43.033 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:09:43.052 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:09:43.052 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:09:43.053 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.0 to 1 (10000) 2020-08-07T15:09:43.053 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 1 1 2020-08-07T15:09:43.063 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 76 16 94598 94582 9.72173 11.6562 0.00488289 0.0128532 2020-08-07T15:09:43.063 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:43.033+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:43.144 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:43.136+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:43.240 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:43.238+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:43.286 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 77 15 96341 96326 9.77241 13.625 0.00447374 0.0127873 2020-08-07T15:09:43.295 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:09:43.308 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:09:43.309 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.309 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:96351 4.c 4:3dcc62b4:::benchmark_data_smithi074_44586_object12043:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e143)", 2020-08-07T15:09:43.309 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:43.288018+0000", 2020-08-07T15:09:43.309 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0026130089999999999, 2020-08-07T15:09:43.310 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026474939999999998, 2020-08-07T15:09:43.310 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:43.310 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:09:43.310 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:43.319 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:43.319 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:43.319 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 96351 2020-08-07T15:09:43.320 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.320 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:43.320 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.320 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:43.321 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.288018+0000", 2020-08-07T15:09:43.321 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:43.321 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.321 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.321 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:43.322 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.288018+0000", 2020-08-07T15:09:43.322 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5076000000000001e-05 2020-08-07T15:09:43.330 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.330 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.330 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:43.334 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.288053+0000", 2020-08-07T15:09:43.335 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2729999999999996e-06 2020-08-07T15:09:43.335 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.335 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:43.335 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.288057+0000", 2020-08-07T15:09:43.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2279999999999998e-06 2020-08-07T15:09:43.336 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.336 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.336 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:43.337 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.288065+0000", 2020-08-07T15:09:43.337 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0459999999999999e-06 2020-08-07T15:09:43.367 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.367 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.368 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:43.368 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.288069+0000", 2020-08-07T15:09:43.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5161999999999995e-05 2020-08-07T15:09:43.368 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.368 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.369 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:43.369 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.288154+0000", 2020-08-07T15:09:43.369 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000218846 2020-08-07T15:09:43.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.370 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:43.370 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.288373+0000", 2020-08-07T15:09:43.370 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000191638 2020-08-07T15:09:43.370 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.371 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.371 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:09:43.371 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.288565+0000", 2020-08-07T15:09:43.371 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00054726899999999997 2020-08-07T15:09:43.371 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:43.372 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:43.372 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:43.372 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.372 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:96352 4.c 4:3dcc62b4:::benchmark_data_smithi074_44586_object12043:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e143)", 2020-08-07T15:09:43.373 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:43.288377+0000", 2020-08-07T15:09:43.373 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0022532189999999999, 2020-08-07T15:09:43.373 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023937250000000002, 2020-08-07T15:09:43.373 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:43.373 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:09:43.374 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:43.374 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:43.374 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:43.374 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 96352 2020-08-07T15:09:43.375 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.375 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:43.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.376 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:43.376 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.288377+0000", 2020-08-07T15:09:43.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:43.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:43.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.288377+0000", 2020-08-07T15:09:43.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0117999999999997e-05 2020-08-07T15:09:43.377 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:43.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.288428+0000", 2020-08-07T15:09:43.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2470000000000002e-06 2020-08-07T15:09:43.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:43.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.288435+0000", 2020-08-07T15:09:43.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2576e-05 2020-08-07T15:09:43.379 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:43.388 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.288447+0000", 2020-08-07T15:09:43.389 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7019999999999998e-06 2020-08-07T15:09:43.389 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.389 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.389 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:43.389 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.288453+0000", 2020-08-07T15:09:43.390 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001332562 2020-08-07T15:09:43.390 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.390 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.390 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:43.390 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.289786+0000", 2020-08-07T15:09:43.391 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8185999999999998e-05 2020-08-07T15:09:43.391 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.391 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.392 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:43.393 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.289884+0000", 2020-08-07T15:09:43.393 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017604099999999999 2020-08-07T15:09:43.393 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.394 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.394 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:09:43.394 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.290060+0000", 2020-08-07T15:09:43.409 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001682432 2020-08-07T15:09:43.411 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:43.411 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:43.411 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:43.412 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.412 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.413 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:96349 4.c 4:3dcc62b4:::benchmark_data_smithi074_44586_object12043:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e143)", 2020-08-07T15:09:43.413 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:43.285092+0000", 2020-08-07T15:09:43.413 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0055383469999999999, 2020-08-07T15:09:43.413 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0057703920000000001, 2020-08-07T15:09:43.413 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:43.414 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:43.414 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:43.415 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:43.415 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:43.415 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 96349 2020-08-07T15:09:43.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.415 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:43.416 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.416 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:43.416 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.285092+0000", 2020-08-07T15:09:43.416 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:43.416 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.417 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.417 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:43.417 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.285092+0000", 2020-08-07T15:09:43.417 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3584999999999998e-05 2020-08-07T15:09:43.417 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.418 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.418 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:43.418 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.285126+0000", 2020-08-07T15:09:43.418 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0860000000000004e-06 2020-08-07T15:09:43.419 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.419 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.419 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:43.419 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.285130+0000", 2020-08-07T15:09:43.419 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2400000000000007e-06 2020-08-07T15:09:43.420 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.420 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.421 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:43.421 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.285138+0000", 2020-08-07T15:09:43.421 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8369999999999999e-06 2020-08-07T15:09:43.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.421 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.422 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:43.422 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.285142+0000", 2020-08-07T15:09:43.422 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001006121 2020-08-07T15:09:43.422 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.423 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:43.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.286148+0000", 2020-08-07T15:09:43.423 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6267999999999997e-05 2020-08-07T15:09:43.423 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.423 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.424 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:43.424 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.286204+0000", 2020-08-07T15:09:43.424 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000115686 2020-08-07T15:09:43.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.424 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:09:43.425 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.286320+0000", 2020-08-07T15:09:43.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001724705 2020-08-07T15:09:43.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.426 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.426 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:43.426 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.288045+0000", 2020-08-07T15:09:43.426 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9599999999999998e-07 2020-08-07T15:09:43.426 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.427 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:43.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.288046+0000", 2020-08-07T15:09:43.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4774999999999997e-05 2020-08-07T15:09:43.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:43.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.288090+0000", 2020-08-07T15:09:43.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002998199 2020-08-07T15:09:43.428 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:43.429 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:43.429 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:43.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.430 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:96350 4.c 4:3dcc62b4:::benchmark_data_smithi074_44586_object12043:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e143)", 2020-08-07T15:09:43.430 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:43.285165+0000", 2020-08-07T15:09:43.430 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0054660819999999997, 2020-08-07T15:09:43.430 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0058060890000000004, 2020-08-07T15:09:43.430 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:43.430 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:43.431 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:43.432 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:43.432 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:43.432 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 96350 2020-08-07T15:09:43.433 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.433 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:43.433 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.433 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:43.433 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.285165+0000", 2020-08-07T15:09:43.434 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:43.434 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.434 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.434 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:43.434 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.285165+0000", 2020-08-07T15:09:43.434 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8218e-05 2020-08-07T15:09:43.435 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.435 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.435 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:43.435 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.285193+0000", 2020-08-07T15:09:43.436 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9620000000000002e-06 2020-08-07T15:09:43.443 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.443 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.444 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:43.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.285196+0000", 2020-08-07T15:09:43.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7329999999999997e-06 2020-08-07T15:09:43.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.460 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.461 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:43.461 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.285202+0000", 2020-08-07T15:09:43.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.458e-06 2020-08-07T15:09:43.461 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.461 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.462 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:43.462 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.285205+0000", 2020-08-07T15:09:43.462 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001358869 2020-08-07T15:09:43.462 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.480 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:43.481 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.286564+0000", 2020-08-07T15:09:43.481 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9188999999999999e-05 2020-08-07T15:09:43.481 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.481 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.482 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:43.482 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.286623+0000", 2020-08-07T15:09:43.482 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013772099999999999 2020-08-07T15:09:43.482 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.483 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.483 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:09:43.483 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.286761+0000", 2020-08-07T15:09:43.483 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013362719999999999 2020-08-07T15:09:43.484 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.491 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.491 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:43.506 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.288097+0000", 2020-08-07T15:09:43.506 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8799999999999998e-07 2020-08-07T15:09:43.507 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.507 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.508 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:43.508 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.288097+0000", 2020-08-07T15:09:43.508 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3720999999999998e-05 2020-08-07T15:09:43.509 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.509 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.509 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:43.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:43.288131+0000", 2020-08-07T15:09:43.511 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0029665310000000001 2020-08-07T15:09:43.511 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:43.512 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:43.512 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:43.512 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:43.513 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:09:43.513 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 4 2020-08-07T15:09:43.514 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:09:43.516 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:09:43.517 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:43.515+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:43.616 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:43.615+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:43.692 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:09:43.692 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:09:43.693 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:09:43.693 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:09:43.693 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:09:43.712 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:09:43.715 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:09:43.757 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:43.756+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:43.827 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:43.822+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:43.925 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:43.923+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:43.935 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:09:43.935 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:09:43.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:09:43.936 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:09:43.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.954 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106240 3.7 3:ebdd8242:::benchmark_data_smithi074_35447_object13279:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:09:43.954 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:09:43.955 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 116.574474389, 2020-08-07T15:09:43.955 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.90079396, 2020-08-07T15:09:43.955 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:43.955 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:43.955 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:43.956 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:09:43.956 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:43.956 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106240 2020-08-07T15:09:43.956 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.962 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:43.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:43.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:09:43.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:43.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:43.964 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:09:43.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2522000000000003e-05 2020-08-07T15:09:43.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:43.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:43.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:44.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355118+0000", 2020-08-07T15:09:44.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6159999999999998e-06 2020-08-07T15:09:44.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:44.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355125+0000", 2020-08-07T15:09:44.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2699e-05 2020-08-07T15:09:44.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:44.022 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355138+0000", 2020-08-07T15:09:44.022 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2310000000000001e-06 2020-08-07T15:09:44.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.023 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:44.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355144+0000", 2020-08-07T15:09:44.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9855000000000003e-05 2020-08-07T15:09:44.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:44.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355194+0000", 2020-08-07T15:09:44.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112713 2020-08-07T15:09:44.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:44.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355306+0000", 2020-08-07T15:09:44.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019794600000000001 2020-08-07T15:09:44.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.030 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.030 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:09:44.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355504+0000", 2020-08-07T15:09:44.047 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0048346029999999998 2020-08-07T15:09:44.048 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.048 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.049 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:44.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.360339+0000", 2020-08-07T15:09:44.058 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00033646400000000002 2020-08-07T15:09:44.058 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.058 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.059 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:44.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.360675+0000", 2020-08-07T15:09:44.059 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89508481200000001 2020-08-07T15:09:44.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:44.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255760+0000", 2020-08-07T15:09:44.065 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5659999999999999e-06 2020-08-07T15:09:44.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.066 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:44.066 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255762+0000", 2020-08-07T15:09:44.066 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6595000000000003e-05 2020-08-07T15:09:44.067 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.067 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.067 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:44.068 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255818+0000", 2020-08-07T15:09:44.068 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1338000000000003e-05 2020-08-07T15:09:44.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.069 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.069 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:44.069 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255860+0000", 2020-08-07T15:09:44.082 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.90079396 2020-08-07T15:09:44.082 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.083 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:44.083 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.084 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.085 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106240 3.7 e59/20 ondisk, result = 0)", 2020-08-07T15:09:44.085 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:09:44.086 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 116.47807843699999, 2020-08-07T15:09:44.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80440215100000001, 2020-08-07T15:09:44.087 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:44.087 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:09:44.087 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:44.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:44.088 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:09:44.089 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:44.089 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.089 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.090 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:44.097 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:09:44.098 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0052999999999999e-05 2020-08-07T15:09:44.098 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.098 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.098 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:44.099 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451482+0000", 2020-08-07T15:09:44.099 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3229999999999999e-06 2020-08-07T15:09:44.099 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.099 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.099 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:44.100 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451486+0000", 2020-08-07T15:09:44.100 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80420304799999998 2020-08-07T15:09:44.100 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.100 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.100 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:44.101 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255689+0000", 2020-08-07T15:09:44.101 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7759999999999997e-06 2020-08-07T15:09:44.104 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.105 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.105 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:44.105 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255694+0000", 2020-08-07T15:09:44.109 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5828999999999998e-05 2020-08-07T15:09:44.110 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.110 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.110 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:44.110 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255740+0000", 2020-08-07T15:09:44.111 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1466e-05 2020-08-07T15:09:44.111 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.111 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.111 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:44.111 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255751+0000", 2020-08-07T15:09:44.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112656 2020-08-07T15:09:44.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.117 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:44.118 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255864+0000", 2020-08-07T15:09:44.118 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80440215100000001 2020-08-07T15:09:44.118 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.118 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:44.118 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.119 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.119 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.119 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37057 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:44.119 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:09:44.119 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 49.685969727, 2020-08-07T15:09:44.121 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007248691, 2020-08-07T15:09:44.121 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:44.121 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:44.128 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:44.129 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:44.129 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:44.129 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37057 2020-08-07T15:09:44.129 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.129 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:44.130 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.130 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:44.130 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:09:44.130 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:44.130 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.131 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.131 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:44.131 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:09:44.131 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3661e-05 2020-08-07T15:09:44.131 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.132 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.132 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:44.142 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243624+0000", 2020-08-07T15:09:44.142 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7519999999999996e-06 2020-08-07T15:09:44.142 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.142 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.143 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:44.143 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243632+0000", 2020-08-07T15:09:44.143 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6392000000000002e-05 2020-08-07T15:09:44.143 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.143 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.144 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:44.144 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243648+0000", 2020-08-07T15:09:44.144 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6659999999999997e-06 2020-08-07T15:09:44.144 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.144 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.145 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:44.145 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243657+0000", 2020-08-07T15:09:44.145 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0070484609999999998 2020-08-07T15:09:44.145 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.148 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.148 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:44.149 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250705+0000", 2020-08-07T15:09:44.153 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0083e-05 2020-08-07T15:09:44.154 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.154 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.154 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:44.155 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250715+0000", 2020-08-07T15:09:44.155 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000086695 2020-08-07T15:09:44.155 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.155 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.155 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:44.156 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.250802+0000", 2020-08-07T15:09:44.156 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6980999999999999e-05 2020-08-07T15:09:44.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.156 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:44.157 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.250819+0000", 2020-08-07T15:09:44.157 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007248691 2020-08-07T15:09:44.157 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.157 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:44.163 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.164 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37058 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:44.164 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:09:44.164 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 49.685827992999997, 2020-08-07T15:09:44.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007368357, 2020-08-07T15:09:44.165 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:44.165 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:44.165 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:44.165 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:44.166 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:44.166 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37058 2020-08-07T15:09:44.166 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.166 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:44.166 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.167 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:44.167 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:09:44.167 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:44.167 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.167 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.167 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:44.168 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:09:44.168 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8838999999999998e-05 2020-08-07T15:09:44.168 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.206 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.206 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:44.206 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243761+0000", 2020-08-07T15:09:44.207 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2079999999999999e-06 2020-08-07T15:09:44.207 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.207 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.207 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:44.208 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243766+0000", 2020-08-07T15:09:44.208 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0567999999999999e-05 2020-08-07T15:09:44.208 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.208 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.208 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:44.208 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243777+0000", 2020-08-07T15:09:44.209 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7210000000000002e-06 2020-08-07T15:09:44.209 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.209 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.214 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:44.215 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243782+0000", 2020-08-07T15:09:44.215 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069508749999999996 2020-08-07T15:09:44.215 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.215 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.216 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:44.216 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250732+0000", 2020-08-07T15:09:44.216 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7899999999999999e-06 2020-08-07T15:09:44.216 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.216 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.217 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:44.217 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250737+0000", 2020-08-07T15:09:44.217 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003272409999999 2020-08-07T15:09:44.217 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.217 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.224 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:44.224 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251064+0000", 2020-08-07T15:09:44.225 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6115000000000001e-05 2020-08-07T15:09:44.225 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.225 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.226 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:44.226 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251081+0000", 2020-08-07T15:09:44.226 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007368357 2020-08-07T15:09:44.226 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.226 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:44.227 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.227 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.227 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.227 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37059 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:44.228 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:09:44.235 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 49.685584943999999, 2020-08-07T15:09:44.236 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0071655319999999, 2020-08-07T15:09:44.236 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:44.236 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:44.236 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:44.237 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:44.246 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:44.246 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37059 2020-08-07T15:09:44.257 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.257 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:44.257 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.257 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:44.268 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:09:44.269 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:44.269 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.269 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.270 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:44.270 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:09:44.271 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6533000000000002e-05 2020-08-07T15:09:44.271 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:44.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244012+0000", 2020-08-07T15:09:44.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3860000000000001e-06 2020-08-07T15:09:44.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.273 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:44.273 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244019+0000", 2020-08-07T15:09:44.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6982000000000001e-05 2020-08-07T15:09:44.273 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.274 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.274 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:44.274 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244036+0000", 2020-08-07T15:09:44.274 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0310000000000001e-06 2020-08-07T15:09:44.274 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.283 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.284 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:44.284 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244043+0000", 2020-08-07T15:09:44.284 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0067169259999999998 2020-08-07T15:09:44.284 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.285 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.285 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:44.285 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250760+0000", 2020-08-07T15:09:44.285 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4499999999999998e-06 2020-08-07T15:09:44.285 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.286 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.286 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:44.286 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250768+0000", 2020-08-07T15:09:44.286 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000341691 2020-08-07T15:09:44.286 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.287 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:44.287 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251109+0000", 2020-08-07T15:09:44.287 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1533e-05 2020-08-07T15:09:44.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.288 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:44.292 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251121+0000", 2020-08-07T15:09:44.292 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0071655319999999 2020-08-07T15:09:44.293 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.293 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:44.295 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.296 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.296 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.296 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37060 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:44.296 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:09:44.296 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 49.685461007000001, 2020-08-07T15:09:44.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070762280000001, 2020-08-07T15:09:44.300 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:44.300 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:44.300 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:44.300 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:44.300 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:44.301 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37060 2020-08-07T15:09:44.301 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.301 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:44.301 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.301 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:44.302 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:09:44.302 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:44.302 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.308 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:44.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:09:44.308 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5739000000000003e-05 2020-08-07T15:09:44.309 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.317 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.317 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:44.318 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244135+0000", 2020-08-07T15:09:44.318 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7740000000000001e-06 2020-08-07T15:09:44.318 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.318 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.318 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:44.319 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244141+0000", 2020-08-07T15:09:44.319 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3817e-05 2020-08-07T15:09:44.319 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.321 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.328 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:44.338 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244155+0000", 2020-08-07T15:09:44.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3829999999999997e-06 2020-08-07T15:09:44.339 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.339 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.339 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:44.340 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244161+0000", 2020-08-07T15:09:44.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0066303669999999999 2020-08-07T15:09:44.340 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.340 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:44.341 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250791+0000", 2020-08-07T15:09:44.341 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.392e-06 2020-08-07T15:09:44.341 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.348 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:44.349 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250798+0000", 2020-08-07T15:09:44.349 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003485860000001 2020-08-07T15:09:44.349 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.349 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.350 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:44.350 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251146+0000", 2020-08-07T15:09:44.362 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1700000000000003e-06 2020-08-07T15:09:44.362 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.362 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.362 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:44.363 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251155+0000", 2020-08-07T15:09:44.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070762280000001 2020-08-07T15:09:44.364 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.364 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:44.364 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.373 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.373 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.373 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37061 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:44.374 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:09:44.374 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 49.685346379999999, 2020-08-07T15:09:44.375 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070006920000001, 2020-08-07T15:09:44.375 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:44.375 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:44.375 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:44.375 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:44.376 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:44.376 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37061 2020-08-07T15:09:44.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.376 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:44.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:44.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:09:44.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:44.377 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.385 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:44.396 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:09:44.396 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5453e-05 2020-08-07T15:09:44.396 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.396 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.397 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:44.397 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244239+0000", 2020-08-07T15:09:44.397 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3179999999999997e-06 2020-08-07T15:09:44.397 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.397 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.398 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:44.398 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244245+0000", 2020-08-07T15:09:44.399 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2138e-05 2020-08-07T15:09:44.399 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.400 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:44.400 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244257+0000", 2020-08-07T15:09:44.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.69e-06 2020-08-07T15:09:44.400 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.401 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.401 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:44.401 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244261+0000", 2020-08-07T15:09:44.401 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0065598940000000001 2020-08-07T15:09:44.401 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.420 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.421 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:44.421 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250821+0000", 2020-08-07T15:09:44.421 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4799999999999998e-06 2020-08-07T15:09:44.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:44.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250828+0000", 2020-08-07T15:09:44.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003567310000001 2020-08-07T15:09:44.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.434 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:44.434 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251185+0000", 2020-08-07T15:09:44.435 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9879999999999999e-06 2020-08-07T15:09:44.435 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.435 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.435 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:44.435 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251195+0000", 2020-08-07T15:09:44.436 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070006920000001 2020-08-07T15:09:44.436 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.436 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:44.436 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.436 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.437 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.437 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37062 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:44.437 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:09:44.437 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 49.685246161999999, 2020-08-07T15:09:44.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006933662, 2020-08-07T15:09:44.445 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:44.445 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:44.446 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:44.446 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:44.446 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:44.446 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37062 2020-08-07T15:09:44.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.447 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:44.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.447 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:44.447 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:09:44.448 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:44.448 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:44.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:09:44.448 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4041e-05 2020-08-07T15:09:44.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.458 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.458 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:44.458 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244348+0000", 2020-08-07T15:09:44.459 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2079999999999997e-06 2020-08-07T15:09:44.459 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:44.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244354+0000", 2020-08-07T15:09:44.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1483e-05 2020-08-07T15:09:44.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.460 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.461 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:44.461 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244366+0000", 2020-08-07T15:09:44.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7989999999999999e-06 2020-08-07T15:09:44.461 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.461 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.462 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:44.462 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244372+0000", 2020-08-07T15:09:44.462 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0064783699999999998 2020-08-07T15:09:44.462 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.462 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.463 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:44.463 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250850+0000", 2020-08-07T15:09:44.463 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4729999999999997e-06 2020-08-07T15:09:44.463 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.463 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.464 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:44.464 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250856+0000", 2020-08-07T15:09:44.464 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003630240000001 2020-08-07T15:09:44.464 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.464 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.465 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:44.465 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251219+0000", 2020-08-07T15:09:44.465 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2640000000000008e-06 2020-08-07T15:09:44.465 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.465 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.474 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:44.474 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251228+0000", 2020-08-07T15:09:44.474 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006933662 2020-08-07T15:09:44.474 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.474 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:44.475 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.475 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.475 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.475 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37063 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:44.475 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:09:44.476 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 49.685133827999998, 2020-08-07T15:09:44.476 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006858051, 2020-08-07T15:09:44.476 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:44.481 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:44.481 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:44.482 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:44.482 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:44.482 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37063 2020-08-07T15:09:44.486 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.486 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:44.486 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.487 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:44.487 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:09:44.492 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:44.492 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.492 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.492 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:44.492 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:09:44.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5043000000000001e-05 2020-08-07T15:09:44.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.493 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:44.493 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244461+0000", 2020-08-07T15:09:44.494 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4600000000000002e-06 2020-08-07T15:09:44.494 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.494 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:44.494 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244467+0000", 2020-08-07T15:09:44.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1435e-05 2020-08-07T15:09:44.495 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.495 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.495 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:44.495 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244478+0000", 2020-08-07T15:09:44.496 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0190000000000003e-06 2020-08-07T15:09:44.502 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.503 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.503 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:44.503 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244484+0000", 2020-08-07T15:09:44.503 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006392895 2020-08-07T15:09:44.504 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.504 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.504 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:44.504 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250877+0000", 2020-08-07T15:09:44.504 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4280000000000001e-06 2020-08-07T15:09:44.505 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.505 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.505 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:44.505 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250884+0000", 2020-08-07T15:09:44.505 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000370414 2020-08-07T15:09:44.506 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.506 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.506 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:44.535 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251254+0000", 2020-08-07T15:09:44.535 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0356999999999999e-05 2020-08-07T15:09:44.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.536 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.554 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:44.555 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251264+0000", 2020-08-07T15:09:44.573 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006858051 2020-08-07T15:09:44.574 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.574 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:44.574 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.575 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.575 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37064 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:44.575 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:09:44.575 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 49.685021695000003, 2020-08-07T15:09:44.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0067798189999999, 2020-08-07T15:09:44.576 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:44.576 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:44.576 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:44.577 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:44.577 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:44.577 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37064 2020-08-07T15:09:44.577 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.577 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:44.586 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.587 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:44.587 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:09:44.587 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:44.587 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.587 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.588 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:44.588 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:09:44.588 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4007e-05 2020-08-07T15:09:44.588 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.589 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:44.589 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244573+0000", 2020-08-07T15:09:44.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.831e-06 2020-08-07T15:09:44.589 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.599 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:44.599 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244577+0000", 2020-08-07T15:09:44.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2269000000000001e-05 2020-08-07T15:09:44.599 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.600 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.600 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:44.600 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244590+0000", 2020-08-07T15:09:44.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7149999999999997e-06 2020-08-07T15:09:44.600 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.601 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:44.601 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244594+0000", 2020-08-07T15:09:44.601 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063105670000000004 2020-08-07T15:09:44.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.602 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.602 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:44.602 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250905+0000", 2020-08-07T15:09:44.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1970000000000001e-06 2020-08-07T15:09:44.602 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.603 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:44.631 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250911+0000", 2020-08-07T15:09:44.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003783020000001 2020-08-07T15:09:44.632 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.633 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.633 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:44.633 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251289+0000", 2020-08-07T15:09:44.633 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9309999999999997e-06 2020-08-07T15:09:44.633 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.636 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.637 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:44.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251298+0000", 2020-08-07T15:09:44.642 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0067798189999999 2020-08-07T15:09:44.643 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.643 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:44.643 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.644 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.644 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37065 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:44.644 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:09:44.644 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 49.684911476000003, 2020-08-07T15:09:44.644 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006708545, 2020-08-07T15:09:44.645 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:44.645 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:44.645 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:44.645 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:44.645 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:44.646 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37065 2020-08-07T15:09:44.646 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.646 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:44.646 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.646 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:44.647 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:09:44.647 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:44.647 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.647 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.647 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:44.660 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:09:44.661 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2750000000000001e-05 2020-08-07T15:09:44.661 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.661 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.661 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:44.661 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244681+0000", 2020-08-07T15:09:44.662 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0950000000000003e-06 2020-08-07T15:09:44.662 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.662 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.662 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:44.662 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244687+0000", 2020-08-07T15:09:44.663 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2485e-05 2020-08-07T15:09:44.663 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.663 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.663 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:44.663 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244699+0000", 2020-08-07T15:09:44.664 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8800000000000002e-06 2020-08-07T15:09:44.664 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.664 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:44.664 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244706+0000", 2020-08-07T15:09:44.665 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0062281569999999998 2020-08-07T15:09:44.665 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.665 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.665 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:44.665 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250934+0000", 2020-08-07T15:09:44.674 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1719999999999998e-06 2020-08-07T15:09:44.674 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.674 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:44.677 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250939+0000", 2020-08-07T15:09:44.677 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003861810000001 2020-08-07T15:09:44.682 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.682 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.682 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:44.682 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251325+0000", 2020-08-07T15:09:44.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1824999999999999e-05 2020-08-07T15:09:44.683 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.683 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.683 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:44.683 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251337+0000", 2020-08-07T15:09:44.684 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006708545 2020-08-07T15:09:44.684 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.684 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:44.684 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.684 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.685 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.685 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37066 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:44.685 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:09:44.685 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 49.680242505999999, 2020-08-07T15:09:44.685 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002077205, 2020-08-07T15:09:44.697 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:44.698 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:44.698 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:44.698 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:44.698 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:44.705 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37066 2020-08-07T15:09:44.705 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.705 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:44.705 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.706 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:44.706 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:09:44.706 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:44.706 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.706 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.707 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:44.707 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:09:44.707 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4654000000000002e-05 2020-08-07T15:09:44.707 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.708 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:44.708 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249352+0000", 2020-08-07T15:09:44.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6210000000000004e-06 2020-08-07T15:09:44.708 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.709 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:44.722 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249360+0000", 2020-08-07T15:09:44.722 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1895999999999998e-05 2020-08-07T15:09:44.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.723 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:44.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249382+0000", 2020-08-07T15:09:44.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1479999999999995e-06 2020-08-07T15:09:44.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.724 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.724 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:44.724 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249391+0000", 2020-08-07T15:09:44.733 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015731510000000001 2020-08-07T15:09:44.735 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.735 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.735 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:44.742 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250964+0000", 2020-08-07T15:09:44.742 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5400000000000001e-06 2020-08-07T15:09:44.742 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.746 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:44.746 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250971+0000", 2020-08-07T15:09:44.747 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000392379 2020-08-07T15:09:44.747 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.754 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.754 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:44.754 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251363+0000", 2020-08-07T15:09:44.754 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1816e-05 2020-08-07T15:09:44.754 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.755 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:44.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251375+0000", 2020-08-07T15:09:44.755 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002077205 2020-08-07T15:09:44.755 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.756 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:44.756 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.756 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37067 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:44.757 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:09:44.757 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 49.680101811999997, 2020-08-07T15:09:44.757 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0019761119999999, 2020-08-07T15:09:44.757 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:44.757 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:44.765 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:44.765 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:44.766 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:44.766 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37067 2020-08-07T15:09:44.766 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.766 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:44.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:44.767 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:09:44.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:44.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.768 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:44.768 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:09:44.768 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5961999999999997e-05 2020-08-07T15:09:44.768 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.768 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.768 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:44.769 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249494+0000", 2020-08-07T15:09:44.769 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3210000000000004e-06 2020-08-07T15:09:44.769 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.769 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.769 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:44.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249500+0000", 2020-08-07T15:09:44.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1012000000000001e-05 2020-08-07T15:09:44.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:44.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249511+0000", 2020-08-07T15:09:44.785 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.809e-06 2020-08-07T15:09:44.785 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:44.786 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249515+0000", 2020-08-07T15:09:44.786 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0014777670000000001 2020-08-07T15:09:44.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.786 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:44.787 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250993+0000", 2020-08-07T15:09:44.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9499999999999998e-06 2020-08-07T15:09:44.787 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.787 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:44.788 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250999+0000", 2020-08-07T15:09:44.788 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0004032519999999 2020-08-07T15:09:44.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.788 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:44.789 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251402+0000", 2020-08-07T15:09:44.789 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2038999999999999e-05 2020-08-07T15:09:44.789 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.789 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.789 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:44.802 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251414+0000", 2020-08-07T15:09:44.802 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0019761119999999 2020-08-07T15:09:44.803 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.803 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:44.803 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.804 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37068 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:44.804 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:09:44.805 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 49.677857240999998, 2020-08-07T15:09:44.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99977072, 2020-08-07T15:09:44.805 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:44.805 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:44.805 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:44.806 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:44.806 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:44.893 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37068 2020-08-07T15:09:44.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.914 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:44.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:44.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:09:44.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:44.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.918 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:44.919 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:09:44.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1369000000000003e-05 2020-08-07T15:09:44.919 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.919 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:44.920 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251724+0000", 2020-08-07T15:09:44.920 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.19e-06 2020-08-07T15:09:44.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:44.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251731+0000", 2020-08-07T15:09:44.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1368e-05 2020-08-07T15:09:44.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:44.947 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251742+0000", 2020-08-07T15:09:44.948 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.682e-06 2020-08-07T15:09:44.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.949 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:44.949 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251748+0000", 2020-08-07T15:09:44.950 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5275000000000003e-05 2020-08-07T15:09:44.950 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.950 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:44.951 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251793+0000", 2020-08-07T15:09:44.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5000000000000005e-06 2020-08-07T15:09:44.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:44.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251802+0000", 2020-08-07T15:09:44.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99963953500000002 2020-08-07T15:09:44.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:44.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251442+0000", 2020-08-07T15:09:44.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1800999999999999e-05 2020-08-07T15:09:44.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:44.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251454+0000", 2020-08-07T15:09:44.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99977072 2020-08-07T15:09:44.974 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.974 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:44.975 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:44.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.976 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37069 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:44.976 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:09:44.976 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 49.676753443000003, 2020-08-07T15:09:44.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99870607899999997, 2020-08-07T15:09:44.976 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:44.977 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:44.977 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:44.977 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:44.977 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:44.977 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37069 2020-08-07T15:09:44.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.978 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:44.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.978 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:44.978 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:09:44.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:44.979 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.990 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:44.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:09:44.991 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3755999999999999e-05 2020-08-07T15:09:44.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:44.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252831+0000", 2020-08-07T15:09:44.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7919999999999998e-06 2020-08-07T15:09:44.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:44.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252836+0000", 2020-08-07T15:09:44.993 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1409999999999999e-05 2020-08-07T15:09:44.993 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.993 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.993 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:44.993 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252848+0000", 2020-08-07T15:09:44.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6380000000000001e-06 2020-08-07T15:09:44.994 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.994 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.994 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:44.994 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252853+0000", 2020-08-07T15:09:44.995 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3244e-05 2020-08-07T15:09:44.995 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:44.995 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:44.995 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:45.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252907+0000", 2020-08-07T15:09:45.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1470000000000007e-06 2020-08-07T15:09:45.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:45.004 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252916+0000", 2020-08-07T15:09:45.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99856497499999997 2020-08-07T15:09:45.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:45.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251481+0000", 2020-08-07T15:09:45.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2116999999999999e-05 2020-08-07T15:09:45.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:45.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251493+0000", 2020-08-07T15:09:45.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99870607899999997 2020-08-07T15:09:45.006 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:45.007 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:45.007 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:45.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.007 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37070 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:09:45.008 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:09:45.008 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 49.676649179999998, 2020-08-07T15:09:45.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99863946699999995, 2020-08-07T15:09:45.009 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:45.009 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:09:45.009 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:45.009 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:45.010 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:45.025 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37070 2020-08-07T15:09:45.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.026 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:45.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:45.026 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:09:45.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:45.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:45.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:09:45.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9984000000000001e-05 2020-08-07T15:09:45.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:45.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252941+0000", 2020-08-07T15:09:45.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7010000000000004e-06 2020-08-07T15:09:45.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:45.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252946+0000", 2020-08-07T15:09:45.030 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1144999999999999e-05 2020-08-07T15:09:45.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:45.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252957+0000", 2020-08-07T15:09:45.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6920000000000001e-06 2020-08-07T15:09:45.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:45.038 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252962+0000", 2020-08-07T15:09:45.038 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6739000000000002e-05 2020-08-07T15:09:45.039 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.039 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.039 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:45.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252998+0000", 2020-08-07T15:09:45.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.503e-06 2020-08-07T15:09:45.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:45.040 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.253006+0000", 2020-08-07T15:09:45.047 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99851375499999995 2020-08-07T15:09:45.047 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.048 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.048 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:45.057 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251520+0000", 2020-08-07T15:09:45.058 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0947999999999999e-05 2020-08-07T15:09:45.058 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.058 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.066 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:45.067 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251530+0000", 2020-08-07T15:09:45.068 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99863946699999995 2020-08-07T15:09:45.074 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:45.074 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:45.074 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:45.074 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.075 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.087 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42505 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:09:45.087 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:09:45.087 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 43.627308650000003, 2020-08-07T15:09:45.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0024736009999999, 2020-08-07T15:09:45.088 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:45.088 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:45.088 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:45.088 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:45.088 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:45.089 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42505 2020-08-07T15:09:45.089 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.089 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:45.090 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.090 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:45.090 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:09:45.094 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:45.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.095 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.095 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:45.095 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:09:45.095 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011052600000000001 2020-08-07T15:09:45.096 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.101 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.101 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:45.101 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302342+0000", 2020-08-07T15:09:45.105 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.781e-06 2020-08-07T15:09:45.106 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.106 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.106 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:45.106 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302352+0000", 2020-08-07T15:09:45.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7646e-05 2020-08-07T15:09:45.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.113 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:45.120 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302369+0000", 2020-08-07T15:09:45.131 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3629999999999994e-06 2020-08-07T15:09:45.132 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.132 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.132 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:45.133 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302378+0000", 2020-08-07T15:09:45.133 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8007000000000003e-05 2020-08-07T15:09:45.134 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.134 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.134 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:45.134 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302426+0000", 2020-08-07T15:09:45.135 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5159999999999994e-06 2020-08-07T15:09:45.135 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.135 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.136 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:45.136 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302434+0000", 2020-08-07T15:09:45.136 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99151562699999995 2020-08-07T15:09:45.137 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.137 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.137 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:45.137 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.293950+0000", 2020-08-07T15:09:45.146 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020138099999999999 2020-08-07T15:09:45.147 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.159 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.160 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:45.161 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294151+0000", 2020-08-07T15:09:45.161 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024761 2020-08-07T15:09:45.161 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.161 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.162 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:09:45.162 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294399+0000", 2020-08-07T15:09:45.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0027714369999999999 2020-08-07T15:09:45.164 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.165 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:45.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297170+0000", 2020-08-07T15:09:45.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.234e-06 2020-08-07T15:09:45.165 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.165 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.166 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:45.166 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297172+0000", 2020-08-07T15:09:45.166 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5384999999999998e-05 2020-08-07T15:09:45.166 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.166 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.167 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:45.168 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297237+0000", 2020-08-07T15:09:45.168 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0073576980000000002 2020-08-07T15:09:45.168 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.168 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.169 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:45.169 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304595+0000", 2020-08-07T15:09:45.169 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1006000000000001e-05 2020-08-07T15:09:45.169 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.169 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.170 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:45.170 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304626+0000", 2020-08-07T15:09:45.170 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6000000000000003e-07 2020-08-07T15:09:45.170 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.171 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.171 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:45.171 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304626+0000", 2020-08-07T15:09:45.172 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8724e-05 2020-08-07T15:09:45.172 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.172 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.173 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:45.173 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304705+0000", 2020-08-07T15:09:45.173 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0024736009999999 2020-08-07T15:09:45.174 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:45.175 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:45.175 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:45.175 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.175 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.177 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42506 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:09:45.177 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:09:45.178 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 43.627117437000003, 2020-08-07T15:09:45.178 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002465226, 2020-08-07T15:09:45.178 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:45.178 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:45.180 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:45.180 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:45.180 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:45.181 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42506 2020-08-07T15:09:45.186 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.186 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:45.186 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.187 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:45.187 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:09:45.187 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:45.187 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.187 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.188 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:45.188 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:09:45.188 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8780999999999999e-05 2020-08-07T15:09:45.188 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.188 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.189 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:45.189 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302472+0000", 2020-08-07T15:09:45.189 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3740000000000003e-06 2020-08-07T15:09:45.189 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.189 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.190 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:45.190 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302478+0000", 2020-08-07T15:09:45.190 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1155000000000001e-05 2020-08-07T15:09:45.191 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.262 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.262 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:45.263 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302489+0000", 2020-08-07T15:09:45.263 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7559999999999996e-06 2020-08-07T15:09:45.263 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.263 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.263 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:45.264 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302495+0000", 2020-08-07T15:09:45.264 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4128000000000003e-05 2020-08-07T15:09:45.264 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.264 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.265 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:45.265 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302529+0000", 2020-08-07T15:09:45.266 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.516e-06 2020-08-07T15:09:45.266 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.266 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.266 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:45.266 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302535+0000", 2020-08-07T15:09:45.267 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99240899299999996 2020-08-07T15:09:45.267 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.267 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.267 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:45.267 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294944+0000", 2020-08-07T15:09:45.267 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000115806 2020-08-07T15:09:45.268 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.268 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.268 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:45.268 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295060+0000", 2020-08-07T15:09:45.269 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000205002 2020-08-07T15:09:45.269 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.269 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:09:45.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295265+0000", 2020-08-07T15:09:45.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043414400000000002 2020-08-07T15:09:45.273 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.274 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.274 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:45.274 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299607+0000", 2020-08-07T15:09:45.274 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2229999999999998e-06 2020-08-07T15:09:45.274 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.275 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.275 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:45.276 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299612+0000", 2020-08-07T15:09:45.276 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5797000000000002e-05 2020-08-07T15:09:45.276 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.276 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.276 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:45.277 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299658+0000", 2020-08-07T15:09:45.277 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005095044 2020-08-07T15:09:45.277 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.277 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.277 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:45.278 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304753+0000", 2020-08-07T15:09:45.278 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3121999999999996e-05 2020-08-07T15:09:45.278 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.278 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.278 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:45.279 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304816+0000", 2020-08-07T15:09:45.279 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5700000000000001e-07 2020-08-07T15:09:45.279 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.279 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.279 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:45.280 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304817+0000", 2020-08-07T15:09:45.280 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1231999999999998e-05 2020-08-07T15:09:45.280 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.280 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.280 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:45.280 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304888+0000", 2020-08-07T15:09:45.281 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002465226 2020-08-07T15:09:45.281 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:45.282 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:45.297 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:45.298 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.298 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.298 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42507 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:09:45.298 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:09:45.298 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 43.627006667000003, 2020-08-07T15:09:45.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002567097, 2020-08-07T15:09:45.299 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:45.299 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:45.299 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:45.299 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:45.300 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:45.300 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42507 2020-08-07T15:09:45.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.307 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:45.308 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:45.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:09:45.308 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:45.308 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.309 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.309 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:45.310 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:09:45.310 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7679000000000002e-05 2020-08-07T15:09:45.310 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.310 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.310 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:45.312 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302581+0000", 2020-08-07T15:09:45.312 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2299999999999999e-06 2020-08-07T15:09:45.313 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.313 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.313 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:45.313 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302586+0000", 2020-08-07T15:09:45.313 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0886999999999999e-05 2020-08-07T15:09:45.314 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.314 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.314 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:45.315 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302597+0000", 2020-08-07T15:09:45.315 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4129999999999997e-06 2020-08-07T15:09:45.316 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.316 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.316 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:45.316 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302603+0000", 2020-08-07T15:09:45.316 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3963000000000001e-05 2020-08-07T15:09:45.317 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.317 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.317 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:45.317 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302627+0000", 2020-08-07T15:09:45.317 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.33e-06 2020-08-07T15:09:45.318 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.318 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.318 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:45.326 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302631+0000", 2020-08-07T15:09:45.327 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99319860900000001 2020-08-07T15:09:45.327 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.327 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.327 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:45.327 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295830+0000", 2020-08-07T15:09:45.328 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000116012 2020-08-07T15:09:45.328 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.328 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.328 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:45.328 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295946+0000", 2020-08-07T15:09:45.329 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000230242 2020-08-07T15:09:45.329 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.329 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.329 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:09:45.329 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296176+0000", 2020-08-07T15:09:45.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003494455 2020-08-07T15:09:45.336 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.336 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.336 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:45.337 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299670+0000", 2020-08-07T15:09:45.337 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0299999999999998e-07 2020-08-07T15:09:45.337 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.337 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.339 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:45.339 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299671+0000", 2020-08-07T15:09:45.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7298000000000001e-05 2020-08-07T15:09:45.340 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.340 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:45.341 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299708+0000", 2020-08-07T15:09:45.341 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0052628839999999998 2020-08-07T15:09:45.341 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.341 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:45.342 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304971+0000", 2020-08-07T15:09:45.342 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3754e-05 2020-08-07T15:09:45.344 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.345 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.345 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:45.346 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305035+0000", 2020-08-07T15:09:45.346 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9500000000000001e-07 2020-08-07T15:09:45.346 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.346 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.347 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:45.347 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305036+0000", 2020-08-07T15:09:45.347 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4843000000000002e-05 2020-08-07T15:09:45.347 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.348 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:45.348 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305101+0000", 2020-08-07T15:09:45.348 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002567097 2020-08-07T15:09:45.348 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:45.348 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:45.348 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:45.349 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.349 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.349 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42508 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:09:45.349 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:09:45.349 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 43.616481647999997, 2020-08-07T15:09:45.350 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99221695600000004, 2020-08-07T15:09:45.350 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:09:45.350 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:45.351 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:09:45.351 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:09:45.351 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:45.363 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42508 2020-08-07T15:09:45.363 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.363 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:09:45.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:09:45.364 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:09:45.364 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:09:45.364 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.365 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.368 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:09:45.368 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:09:45.369 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3692e-05 2020-08-07T15:09:45.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.370 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.370 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:09:45.371 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313112+0000", 2020-08-07T15:09:45.371 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7399999999999998e-06 2020-08-07T15:09:45.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.371 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.371 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:09:45.372 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313119+0000", 2020-08-07T15:09:45.372 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5153999999999999e-05 2020-08-07T15:09:45.372 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.372 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:09:45.373 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313134+0000", 2020-08-07T15:09:45.373 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3039999999999999e-06 2020-08-07T15:09:45.373 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.373 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.373 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:09:45.374 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313141+0000", 2020-08-07T15:09:45.374 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6516000000000001e-05 2020-08-07T15:09:45.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.382 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.382 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:45.382 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313178+0000", 2020-08-07T15:09:45.382 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0949999999999998e-06 2020-08-07T15:09:45.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.383 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.383 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:09:45.383 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313185+0000", 2020-08-07T15:09:45.383 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.98342834999999995 2020-08-07T15:09:45.384 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.384 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.384 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:09:45.384 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296613+0000", 2020-08-07T15:09:45.384 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2230999999999993e-05 2020-08-07T15:09:45.384 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.385 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.385 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:09:45.385 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296686+0000", 2020-08-07T15:09:45.385 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013121300000000001 2020-08-07T15:09:45.397 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.398 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.398 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:09:45.398 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296817+0000", 2020-08-07T15:09:45.398 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044228050000000001 2020-08-07T15:09:45.399 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.399 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:09:45.399 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301240+0000", 2020-08-07T15:09:45.399 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0500000000000002e-07 2020-08-07T15:09:45.400 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.400 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:45.400 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301241+0000", 2020-08-07T15:09:45.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2434000000000006e-05 2020-08-07T15:09:45.401 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.401 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.401 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:09:45.401 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301303+0000", 2020-08-07T15:09:45.401 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038622919999999998 2020-08-07T15:09:45.402 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.402 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.402 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:45.402 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305165+0000", 2020-08-07T15:09:45.402 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8012000000000001e-05 2020-08-07T15:09:45.403 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.403 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.403 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:45.403 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305203+0000", 2020-08-07T15:09:45.403 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5299999999999999e-07 2020-08-07T15:09:45.404 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.404 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.404 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:09:45.404 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305204+0000", 2020-08-07T15:09:45.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1760000000000004e-05 2020-08-07T15:09:45.405 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:45.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:45.405 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:09:45.405 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305275+0000", 2020-08-07T15:09:45.406 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99221695600000004 2020-08-07T15:09:45.406 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:45.406 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:45.406 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:45.407 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:45.407 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:45.407 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:09:45.408 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:09:45.429 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:09:45.436 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:09:45.436 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:09:45.436 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:09:45.437 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:09:45.437 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:09:45.437 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:09:45.437 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:09:45.438 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:09:45.438 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:09:45.438 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:09:45.438 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:09:45.438 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:09:45.439 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:09:45.439 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:09:45.439 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:09:45.439 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 78 16 97879 97863 9.80106 12.0078 0.0112414 0.0127504 2020-08-07T15:09:45.440 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 79 16 99629 99613 9.85004 13.6719 0.028918 0.0126854 2020-08-07T15:09:45.440 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.1 to 1 (10000) 2020-08-07T15:09:45.440 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 2 1 2020-08-07T15:09:45.441 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:45.407+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:45.467 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:09:45.516 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:45.515+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:45.614 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:45.613+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:45.640 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:45.641 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:45.651 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:09:45.718 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:45.716+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:45.817 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:45.816+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:45.886 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:45.886 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:45.898 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:09:45.933 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:45.928+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:46.045 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:46.031+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:46.118 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:46.118 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:46.133 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:09:46.146 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:46.137+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:46.250 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:46.237+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:46.282 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:2020-08-07T15:09:46.279543+0000 min lat: 0.00103494 max lat: 2.05564 avg lat: 0.0126396 2020-08-07T15:09:46.283 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-08-07T15:09:46.283 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 80 16 101262 101246 9.88638 12.7578 0.00328916 0.0126396 2020-08-07T15:09:46.342 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:46.339+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:46.368 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:46.368 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:46.378 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:09:46.470 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:46.468+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:46.548 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:46.546+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:46.568 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:09:46.576 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.2 to 1 (10000) 2020-08-07T15:09:46.607 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 3 1 2020-08-07T15:09:46.608 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:46.608 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:46.629 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:09:46.683 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:46.669+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:46.775 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:46.773+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:46.860 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:46.860 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:46.861 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:09:46.876 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:46.874+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:46.980 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:46.976+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:47.003 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:47.004 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:47.010 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:09:47.080 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:47.079+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:47.161 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:47.162 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:47.180 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:09:47.224 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:47.219+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:47.289 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 81 16 102658 102642 9.89896 10.9062 0.00617969 0.0126225 2020-08-07T15:09:47.292 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:47.292+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:47.395 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:47.395 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:47.395 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:47.392+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:47.407 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:09:47.441 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:09:47.505 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:47.502+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:47.579 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.3 to 1 (10000) 2020-08-07T15:09:47.603 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 4 1 2020-08-07T15:09:47.636 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:47.636 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:47.637 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:09:47.651 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:47.637+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:47.715 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:09:47.715 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:09:47.751 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:47.742+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:47.751 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:09:47.794 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:09:47.834 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:09:47.863 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:47.853+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:47.880 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:09:47.890 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:47.890 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:47.912 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:09:47.913 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:09:47.951 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:09:47.951 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:09:47.952 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:09:47.952 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:09:47.952 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:09:47.953 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:09:47.953 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:09:47.953 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:09:47.954 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:09:47.961 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:47.958+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:48.061 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:48.059+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:48.160 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:48.158+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:48.182 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:09:48.182 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:09:48.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:09:48.183 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:09:48.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.183 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:9976 3.e 3:7fb2e2e5:::benchmark_data_smithi074_35447_object1246:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:09:48.183 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:09:48.183 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 185.92655632200001, 2020-08-07T15:09:48.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28420182999999999, 2020-08-07T15:09:48.184 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:48.184 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:48.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:48.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:48.185 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:48.185 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 9976 2020-08-07T15:09:48.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.207 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:48.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:48.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:09:48.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:48.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:48.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237198+0000", 2020-08-07T15:09:48.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7113e-05 2020-08-07T15:09:48.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:48.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237255+0000", 2020-08-07T15:09:48.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5869999999999998e-06 2020-08-07T15:09:48.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:48.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237262+0000", 2020-08-07T15:09:48.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4467999999999999e-05 2020-08-07T15:09:48.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:48.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237276+0000", 2020-08-07T15:09:48.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0749999999999997e-06 2020-08-07T15:09:48.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:48.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.237281+0000", 2020-08-07T15:09:48.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013795109999999999 2020-08-07T15:09:48.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:48.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238661+0000", 2020-08-07T15:09:48.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100309 2020-08-07T15:09:48.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:48.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238761+0000", 2020-08-07T15:09:48.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000141784 2020-08-07T15:09:48.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:09:48.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.238903+0000", 2020-08-07T15:09:48.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00070626599999999999 2020-08-07T15:09:48.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:48.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239609+0000", 2020-08-07T15:09:48.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3499999999999996e-07 2020-08-07T15:09:48.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:48.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239610+0000", 2020-08-07T15:09:48.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0846000000000006e-05 2020-08-07T15:09:48.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:48.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.239680+0000", 2020-08-07T15:09:48.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022037080000000001 2020-08-07T15:09:48.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:48.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.261718+0000", 2020-08-07T15:09:48.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.25961784500000001 2020-08-07T15:09:48.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:48.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521335+0000", 2020-08-07T15:09:48.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2839999999999999e-06 2020-08-07T15:09:48.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:48.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521337+0000", 2020-08-07T15:09:48.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3126999999999994e-05 2020-08-07T15:09:48.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:48.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:42.521400+0000", 2020-08-07T15:09:48.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28420182999999999 2020-08-07T15:09:48.307 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:48.308 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:48.308 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:48.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.308 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15576 3.e 3:727724a5:::benchmark_data_smithi074_35447_object1946:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:09:48.309 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:09:48.309 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 181.49711935900001, 2020-08-07T15:09:48.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999, 2020-08-07T15:09:48.309 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:48.310 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:48.311 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:48.311 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:48.311 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:48.311 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15576 2020-08-07T15:09:48.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.312 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:48.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:48.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:09:48.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:48.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:48.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:09:48.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7156e-05 2020-08-07T15:09:48.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:48.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666682+0000", 2020-08-07T15:09:48.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0209999999999999e-06 2020-08-07T15:09:48.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:48.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666687+0000", 2020-08-07T15:09:48.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0730999999999999e-05 2020-08-07T15:09:48.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:48.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666698+0000", 2020-08-07T15:09:48.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2020-08-07T15:09:48.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:48.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666702+0000", 2020-08-07T15:09:48.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023978459999999999 2020-08-07T15:09:48.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:48.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669100+0000", 2020-08-07T15:09:48.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9664000000000001e-05 2020-08-07T15:09:48.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:48.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669190+0000", 2020-08-07T15:09:48.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015957100000000001 2020-08-07T15:09:48.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:09:48.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669350+0000", 2020-08-07T15:09:48.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016470149999999999 2020-08-07T15:09:48.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:48.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:09:48.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1299999999999999e-07 2020-08-07T15:09:48.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:48.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:09:48.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5269000000000002e-05 2020-08-07T15:09:48.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:48.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.671022+0000", 2020-08-07T15:09:48.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031414400000000001 2020-08-07T15:09:48.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:48.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.674163+0000", 2020-08-07T15:09:48.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45641417400000001 2020-08-07T15:09:48.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:48.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:09:48.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0399999999999995e-07 2020-08-07T15:09:48.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:48.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:09:48.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3343999999999997e-05 2020-08-07T15:09:48.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:48.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130622+0000", 2020-08-07T15:09:48.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999 2020-08-07T15:09:48.409 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:48.409 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:48.410 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:48.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.416 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:41431 3.d 3:b5c8a160:::benchmark_data_smithi074_35447_object5178:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:09:48.416 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.791939+0000", 2020-08-07T15:09:48.416 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 165.37181483000001, 2020-08-07T15:09:48.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28436496, 2020-08-07T15:09:48.416 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:48.417 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:48.417 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:48.417 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:48.417 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:48.418 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 41431 2020-08-07T15:09:48.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.418 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:48.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:48.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791939+0000", 2020-08-07T15:09:48.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:48.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:48.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791939+0000", 2020-08-07T15:09:48.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6304999999999998e-05 2020-08-07T15:09:48.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:48.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791986+0000", 2020-08-07T15:09:48.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6339999999999998e-06 2020-08-07T15:09:48.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:48.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.791991+0000", 2020-08-07T15:09:48.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.026e-05 2020-08-07T15:09:48.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:48.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792002+0000", 2020-08-07T15:09:48.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6240000000000001e-06 2020-08-07T15:09:48.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:48.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792006+0000", 2020-08-07T15:09:48.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015600220000000001 2020-08-07T15:09:48.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:48.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.793566+0000", 2020-08-07T15:09:48.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3628999999999994e-05 2020-08-07T15:09:48.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:48.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.793660+0000", 2020-08-07T15:09:48.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000168197 2020-08-07T15:09:48.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:09:48.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.793828+0000", 2020-08-07T15:09:48.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001273251 2020-08-07T15:09:48.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:48.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795101+0000", 2020-08-07T15:09:48.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5899999999999998e-07 2020-08-07T15:09:48.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:48.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795102+0000", 2020-08-07T15:09:48.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4904000000000001e-05 2020-08-07T15:09:48.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:48.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795127+0000", 2020-08-07T15:09:48.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00670237 2020-08-07T15:09:48.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:48.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.801829+0000", 2020-08-07T15:09:48.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27442024999999998 2020-08-07T15:09:48.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:48.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076249+0000", 2020-08-07T15:09:48.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999997e-07 2020-08-07T15:09:48.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:48.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076250+0000", 2020-08-07T15:09:48.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4369000000000001e-05 2020-08-07T15:09:48.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:48.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.076304+0000", 2020-08-07T15:09:48.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28436496 2020-08-07T15:09:48.446 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:48.446 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:48.446 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:48.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.447 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:41432 3.d 3:b5c8a160:::benchmark_data_smithi074_35447_object5178:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:09:48.447 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:09:48.447 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 165.37146413400001, 2020-08-07T15:09:48.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.37720701699999998, 2020-08-07T15:09:48.448 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:48.448 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:48.448 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:48.448 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:48.448 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:48.449 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 41432 2020-08-07T15:09:48.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.449 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:48.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:48.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:09:48.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:48.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:48.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:09:48.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4787999999999999e-05 2020-08-07T15:09:48.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:48.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792335+0000", 2020-08-07T15:09:48.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5999999999999997e-06 2020-08-07T15:09:48.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:48.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792341+0000", 2020-08-07T15:09:48.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0998e-05 2020-08-07T15:09:48.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:48.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792352+0000", 2020-08-07T15:09:48.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2989999999999996e-06 2020-08-07T15:09:48.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:48.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792357+0000", 2020-08-07T15:09:48.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001852752 2020-08-07T15:09:48.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:48.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794210+0000", 2020-08-07T15:09:48.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5771000000000002e-05 2020-08-07T15:09:48.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:48.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794305+0000", 2020-08-07T15:09:48.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012776399999999999 2020-08-07T15:09:48.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:09:48.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794433+0000", 2020-08-07T15:09:48.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00069673599999999999 2020-08-07T15:09:48.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:48.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795130+0000", 2020-08-07T15:09:48.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1500000000000001e-07 2020-08-07T15:09:48.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:48.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795130+0000", 2020-08-07T15:09:48.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7319e-05 2020-08-07T15:09:48.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:48.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795167+0000", 2020-08-07T15:09:48.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27910026700000001 2020-08-07T15:09:48.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:48.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074268+0000", 2020-08-07T15:09:48.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.095150660999999997 2020-08-07T15:09:48.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:48.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169418+0000", 2020-08-07T15:09:48.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.423e-06 2020-08-07T15:09:48.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:48.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169420+0000", 2020-08-07T15:09:48.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7423999999999996e-05 2020-08-07T15:09:48.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:48.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169497+0000", 2020-08-07T15:09:48.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.37720701699999998 2020-08-07T15:09:48.467 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:48.467 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:48.467 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:48.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.468 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:42104 3.d 3:bcc35439:::benchmark_data_smithi074_35447_object5262:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:09:48.468 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:09:48.468 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 164.924831963, 2020-08-07T15:09:48.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28812091400000001, 2020-08-07T15:09:48.468 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:48.469 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:48.469 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:48.469 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:48.469 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:48.470 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42104 2020-08-07T15:09:48.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.470 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:48.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:48.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:09:48.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:48.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:48.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:09:48.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9734000000000002e-05 2020-08-07T15:09:48.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:48.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238972+0000", 2020-08-07T15:09:48.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6910000000000002e-06 2020-08-07T15:09:48.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:48.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238978+0000", 2020-08-07T15:09:48.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1864e-05 2020-08-07T15:09:48.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:48.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238990+0000", 2020-08-07T15:09:48.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.737e-06 2020-08-07T15:09:48.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:48.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238996+0000", 2020-08-07T15:09:48.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00283943 2020-08-07T15:09:48.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:48.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.241836+0000", 2020-08-07T15:09:48.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4712999999999999e-05 2020-08-07T15:09:48.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:48.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.241930+0000", 2020-08-07T15:09:48.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015328499999999999 2020-08-07T15:09:48.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:09:48.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.242084+0000", 2020-08-07T15:09:48.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016104400000000001 2020-08-07T15:09:48.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:48.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243694+0000", 2020-08-07T15:09:48.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1399999999999997e-07 2020-08-07T15:09:48.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:48.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243695+0000", 2020-08-07T15:09:48.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7800999999999998e-05 2020-08-07T15:09:48.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:48.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243753+0000", 2020-08-07T15:09:48.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090405661999999998 2020-08-07T15:09:48.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:48.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.334158+0000", 2020-08-07T15:09:48.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19283929299999999 2020-08-07T15:09:48.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:48.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.526997+0000", 2020-08-07T15:09:48.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.012e-06 2020-08-07T15:09:48.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:48.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.526998+0000", 2020-08-07T15:09:48.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4737999999999998e-05 2020-08-07T15:09:48.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:48.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.527043+0000", 2020-08-07T15:09:48.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28812091400000001 2020-08-07T15:09:48.549 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:48.549 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:48.550 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:48.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.551 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:86712 3.8 e52/20 3:14925daa:::benchmark_data_smithi074_35447_object10838:head v 52'5488, mlcod=52'5479)", 2020-08-07T15:09:48.552 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.338631+0000", 2020-08-07T15:09:48.552 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 133.825122865, 2020-08-07T15:09:48.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.310229594, 2020-08-07T15:09:48.554 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:48.554 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:48.554 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:48.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:48.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338631+0000", 2020-08-07T15:09:48.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:48.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:48.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338631+0000", 2020-08-07T15:09:48.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6180000000000001e-05 2020-08-07T15:09:48.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:48.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338688+0000", 2020-08-07T15:09:48.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4949999999999996e-06 2020-08-07T15:09:48.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:48.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338694+0000", 2020-08-07T15:09:48.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30438292099999997 2020-08-07T15:09:48.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:48.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643077+0000", 2020-08-07T15:09:48.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9389999999999996e-06 2020-08-07T15:09:48.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:48.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643082+0000", 2020-08-07T15:09:48.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5096999999999999e-05 2020-08-07T15:09:48.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:48.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643127+0000", 2020-08-07T15:09:48.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4323000000000001e-05 2020-08-07T15:09:48.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:48.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643151+0000", 2020-08-07T15:09:48.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055847300000000004 2020-08-07T15:09:48.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:48.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.648736+0000", 2020-08-07T15:09:48.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012490900000000001 2020-08-07T15:09:48.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:48.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.648861+0000", 2020-08-07T15:09:48.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.310229594 2020-08-07T15:09:48.588 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:48.588 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:48.588 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:48.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.589 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86792 3.f 3:fc5386ee:::benchmark_data_smithi074_35447_object10848:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:09:48.589 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.395402+0000", 2020-08-07T15:09:48.589 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 133.768352755, 2020-08-07T15:09:48.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30259831999999998, 2020-08-07T15:09:48.590 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:48.590 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:48.590 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:48.590 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:48.591 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:48.598 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86792 2020-08-07T15:09:48.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.599 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:48.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:48.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395402+0000", 2020-08-07T15:09:48.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:48.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:48.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395402+0000", 2020-08-07T15:09:48.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2262999999999998e-05 2020-08-07T15:09:48.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:48.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395444+0000", 2020-08-07T15:09:48.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1800000000000004e-06 2020-08-07T15:09:48.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:48.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395449+0000", 2020-08-07T15:09:48.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0738e-05 2020-08-07T15:09:48.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:48.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395460+0000", 2020-08-07T15:09:48.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7219999999999999e-06 2020-08-07T15:09:48.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:48.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395464+0000", 2020-08-07T15:09:48.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0063169089999999999 2020-08-07T15:09:48.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:48.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.401781+0000", 2020-08-07T15:09:48.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010702200000000001 2020-08-07T15:09:48.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:48.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.401888+0000", 2020-08-07T15:09:48.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019171999999999999 2020-08-07T15:09:48.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:09:48.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.402080+0000", 2020-08-07T15:09:48.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042866839999999998 2020-08-07T15:09:48.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:48.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.406367+0000", 2020-08-07T15:09:48.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000001e-07 2020-08-07T15:09:48.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:48.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.406367+0000", 2020-08-07T15:09:48.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4720000000000004e-05 2020-08-07T15:09:48.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:48.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.406432+0000", 2020-08-07T15:09:48.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050405365000000001 2020-08-07T15:09:48.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:48.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.456837+0000", 2020-08-07T15:09:48.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24111702700000001 2020-08-07T15:09:48.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:48.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.697954+0000", 2020-08-07T15:09:48.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.22e-06 2020-08-07T15:09:48.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:48.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.697955+0000", 2020-08-07T15:09:48.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4336999999999999e-05 2020-08-07T15:09:48.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:48.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.698000+0000", 2020-08-07T15:09:48.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30259831999999998 2020-08-07T15:09:48.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:48.735 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:48.735 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:48.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.767 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:104104 3.8 e58/20 3:11c276ea:::benchmark_data_smithi074_35447_object13012:head v 58'6496, mlcod=58'6487)", 2020-08-07T15:09:48.768 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:09:48.768 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 122.305268613, 2020-08-07T15:09:48.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33178497099999998, 2020-08-07T15:09:48.769 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:48.778 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:48.778 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:48.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:48.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:09:48.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:48.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:48.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:09:48.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9961999999999999e-05 2020-08-07T15:09:48.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:48.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858516+0000", 2020-08-07T15:09:48.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3200000000000001e-06 2020-08-07T15:09:48.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:48.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858520+0000", 2020-08-07T15:09:48.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32756790000000002 2020-08-07T15:09:48.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:48.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186088+0000", 2020-08-07T15:09:48.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0260000000000004e-06 2020-08-07T15:09:48.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:48.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186094+0000", 2020-08-07T15:09:48.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7919000000000001e-05 2020-08-07T15:09:48.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:48.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186132+0000", 2020-08-07T15:09:48.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4968e-05 2020-08-07T15:09:48.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:48.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186147+0000", 2020-08-07T15:09:48.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040671520000000001 2020-08-07T15:09:48.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:48.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.190214+0000", 2020-08-07T15:09:48.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6724000000000001e-05 2020-08-07T15:09:48.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:48.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.190271+0000", 2020-08-07T15:09:48.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33178497099999998 2020-08-07T15:09:48.830 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:48.830 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:48.830 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:48.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.831 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106248 3.f 3:f950fd9d:::benchmark_data_smithi074_35447_object13280:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:09:48.831 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:09:48.832 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 120.803589819, 2020-08-07T15:09:48.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002, 2020-08-07T15:09:48.832 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:48.838 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:48.838 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:48.839 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:48.839 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:48.839 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106248 2020-08-07T15:09:48.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.839 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:48.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:48.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:09:48.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:48.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:48.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:09:48.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7184e-05 2020-08-07T15:09:48.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:48.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360222+0000", 2020-08-07T15:09:48.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0906e-05 2020-08-07T15:09:48.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:48.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360233+0000", 2020-08-07T15:09:48.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2972999999999999e-05 2020-08-07T15:09:48.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:48.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360246+0000", 2020-08-07T15:09:48.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0114e-05 2020-08-07T15:09:48.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:48.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360256+0000", 2020-08-07T15:09:48.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1303999999999997e-05 2020-08-07T15:09:48.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:48.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360287+0000", 2020-08-07T15:09:48.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3798000000000001e-05 2020-08-07T15:09:48.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:48.910 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:09:48.914 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 82 16 104246 104230 9.92953 12.4062 0.0065882 0.0125824 2020-08-07T15:09:48.915 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.4 to 1 (10000) 2020-08-07T15:09:48.915 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 5 1 2020-08-07T15:09:48.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360351+0000", 2020-08-07T15:09:48.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111308 2020-08-07T15:09:48.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:09:48.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360462+0000", 2020-08-07T15:09:48.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003480674 2020-08-07T15:09:48.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:48.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:09:48.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0900000000000001e-07 2020-08-07T15:09:48.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:48.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:09:48.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8899999999999994e-05 2020-08-07T15:09:48.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:48.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.364012+0000", 2020-08-07T15:09:48.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35291637199999998 2020-08-07T15:09:48.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:48.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716929+0000", 2020-08-07T15:09:48.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.076244233999999994 2020-08-07T15:09:48.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:48.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793173+0000", 2020-08-07T15:09:48.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6109999999999999e-06 2020-08-07T15:09:48.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:48.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:48.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793175+0000", 2020-08-07T15:09:48.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7492999999999999e-05 2020-08-07T15:09:48.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:48.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:49.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793242+0000", 2020-08-07T15:09:49.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002 2020-08-07T15:09:49.019 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:49.019 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:49.019 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:49.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.020 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106248 3.f e59/20 ondisk, result = 0)", 2020-08-07T15:09:49.020 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.365323+0000", 2020-08-07T15:09:49.020 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 120.798431263, 2020-08-07T15:09:49.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35160734999999999, 2020-08-07T15:09:49.020 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:49.021 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:49.021 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:49.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:49.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365323+0000", 2020-08-07T15:09:49.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:49.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:49.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365323+0000", 2020-08-07T15:09:49.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4981000000000003e-05 2020-08-07T15:09:49.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:49.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365358+0000", 2020-08-07T15:09:49.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1090000000000003e-06 2020-08-07T15:09:49.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:49.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365364+0000", 2020-08-07T15:09:49.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35151899399999997 2020-08-07T15:09:49.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:49.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716883+0000", 2020-08-07T15:09:49.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0090000000000001e-06 2020-08-07T15:09:49.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:49.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716887+0000", 2020-08-07T15:09:49.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9238999999999999e-05 2020-08-07T15:09:49.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:49.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716916+0000", 2020-08-07T15:09:49.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0390000000000001e-06 2020-08-07T15:09:49.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:49.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716922+0000", 2020-08-07T15:09:49.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9789999999999993e-06 2020-08-07T15:09:49.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:49.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716930+0000", 2020-08-07T15:09:49.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35160734999999999 2020-08-07T15:09:49.072 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:49.072 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:49.072 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:49.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.073 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106248 3.f e59/20 ondisk, result = 0)", 2020-08-07T15:09:49.073 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:09:49.080 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 120.798326534, 2020-08-07T15:09:49.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001, 2020-08-07T15:09:49.081 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:49.081 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:49.082 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:49.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:49.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:09:49.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:49.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:49.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:09:49.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7518999999999999e-05 2020-08-07T15:09:49.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:49.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365445+0000", 2020-08-07T15:09:49.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-06 2020-08-07T15:09:49.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:49.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365449+0000", 2020-08-07T15:09:49.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42764147600000002 2020-08-07T15:09:49.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:49.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793091+0000", 2020-08-07T15:09:49.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6500000000000002e-06 2020-08-07T15:09:49.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:49.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793099+0000", 2020-08-07T15:09:49.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3664e-05 2020-08-07T15:09:49.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:49.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793153+0000", 2020-08-07T15:09:49.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1078e-05 2020-08-07T15:09:49.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:49.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793164+0000", 2020-08-07T15:09:49.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3404000000000003e-05 2020-08-07T15:09:49.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:49.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793248+0000", 2020-08-07T15:09:49.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001 2020-08-07T15:09:49.120 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:49.120 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:49.120 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:49.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.121 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106344 3.e 3:7abbcafa:::benchmark_data_smithi074_35447_object13292:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:09:49.122 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:48.059687+0000", 2020-08-07T15:09:49.122 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 120.10406743199999, 2020-08-07T15:09:49.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28541866100000002, 2020-08-07T15:09:49.123 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:49.123 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:49.123 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:49.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:49.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:49.124 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106344 2020-08-07T15:09:49.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.125 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:49.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:49.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059687+0000", 2020-08-07T15:09:49.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:49.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:49.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059687+0000", 2020-08-07T15:09:49.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5778999999999998e-05 2020-08-07T15:09:49.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:49.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059713+0000", 2020-08-07T15:09:49.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.422e-06 2020-08-07T15:09:49.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:49.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059716+0000", 2020-08-07T15:09:49.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7679999999999997e-06 2020-08-07T15:09:49.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:49.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059725+0000", 2020-08-07T15:09:49.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2020-08-07T15:09:49.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:49.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059729+0000", 2020-08-07T15:09:49.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6205000000000003e-05 2020-08-07T15:09:49.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:49.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059765+0000", 2020-08-07T15:09:49.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3615999999999995e-05 2020-08-07T15:09:49.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:49.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059829+0000", 2020-08-07T15:09:49.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000118903 2020-08-07T15:09:49.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:09:49.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059948+0000", 2020-08-07T15:09:49.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00070666299999999995 2020-08-07T15:09:49.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:49.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.060654+0000", 2020-08-07T15:09:49.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5199999999999995e-07 2020-08-07T15:09:49.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:49.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.060655+0000", 2020-08-07T15:09:49.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2092999999999998e-05 2020-08-07T15:09:49.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:49.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.060697+0000", 2020-08-07T15:09:49.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00038454500000000003 2020-08-07T15:09:49.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:49.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.061082+0000", 2020-08-07T15:09:49.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.283939936 2020-08-07T15:09:49.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:49.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.345022+0000", 2020-08-07T15:09:49.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.288e-06 2020-08-07T15:09:49.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:49.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.345023+0000", 2020-08-07T15:09:49.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2468999999999999e-05 2020-08-07T15:09:49.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:49.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.345105+0000", 2020-08-07T15:09:49.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28541866100000002 2020-08-07T15:09:49.264 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:49.264 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:49.264 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:49.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.265 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:107952 3.d e61/35)", 2020-08-07T15:09:49.266 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:09:49.266 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 117.837396826, 2020-08-07T15:09:49.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001, 2020-08-07T15:09:49.269 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:49.269 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:49.270 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:49.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:49.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:09:49.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:49.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:49.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:09:49.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8826000000000003e-05 2020-08-07T15:09:49.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:49.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326406+0000", 2020-08-07T15:09:49.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9199999999999998e-06 2020-08-07T15:09:49.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:49.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326413+0000", 2020-08-07T15:09:49.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75082853500000002 2020-08-07T15:09:49.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:49.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077242+0000", 2020-08-07T15:09:49.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1479999999999999e-06 2020-08-07T15:09:49.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:49.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077246+0000", 2020-08-07T15:09:49.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3331999999999998e-05 2020-08-07T15:09:49.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:49.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077289+0000", 2020-08-07T15:09:49.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8977000000000001e-05 2020-08-07T15:09:49.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:49.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077308+0000", 2020-08-07T15:09:49.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001 2020-08-07T15:09:49.312 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:49.313 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:49.314 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:49.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.315 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31696 4.0 4:0335eed9:::benchmark_data_smithi074_44586_object3961:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e100)", 2020-08-07T15:09:49.315 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:09:49.316 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.059489266999996, 2020-08-07T15:09:49.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237, 2020-08-07T15:09:49.316 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:49.317 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:49.317 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:49.317 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:49.318 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:49.318 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31696 2020-08-07T15:09:49.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.319 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:49.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:49.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:09:49.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:49.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:49.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:09:49.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7654e-05 2020-08-07T15:09:49.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:49.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104293+0000", 2020-08-07T15:09:49.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.061e-06 2020-08-07T15:09:49.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:49.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104296+0000", 2020-08-07T15:09:49.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0859999999999998e-06 2020-08-07T15:09:49.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:49.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104302+0000", 2020-08-07T15:09:49.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8030000000000001e-06 2020-08-07T15:09:49.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:49.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104305+0000", 2020-08-07T15:09:49.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037634769999999999 2020-08-07T15:09:49.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:49.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108068+0000", 2020-08-07T15:09:49.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8814999999999998e-05 2020-08-07T15:09:49.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:49.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108137+0000", 2020-08-07T15:09:49.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111455 2020-08-07T15:09:49.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:09:49.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108248+0000", 2020-08-07T15:09:49.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028756820000000001 2020-08-07T15:09:49.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:49.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111124+0000", 2020-08-07T15:09:49.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061054499999999999 2020-08-07T15:09:49.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:49.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111735+0000", 2020-08-07T15:09:49.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.423966167 2020-08-07T15:09:49.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:49.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535701+0000", 2020-08-07T15:09:49.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4929999999999999e-06 2020-08-07T15:09:49.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:49.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535702+0000", 2020-08-07T15:09:49.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2571999999999997e-05 2020-08-07T15:09:49.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:49.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535755+0000", 2020-08-07T15:09:49.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1427000000000003e-05 2020-08-07T15:09:49.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:49.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535786+0000", 2020-08-07T15:09:49.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237 2020-08-07T15:09:49.408 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:49.408 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:49.409 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:49.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.409 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:31696 4.0 e100/81 ondisk, result = 0)", 2020-08-07T15:09:49.409 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:09:49.410 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.049874709000001, 2020-08-07T15:09:49.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954, 2020-08-07T15:09:49.410 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:49.410 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:49.415 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:49.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:49.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:09:49.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:49.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:49.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:09:49.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1185000000000001e-05 2020-08-07T15:09:49.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:49.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113901+0000", 2020-08-07T15:09:49.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2930000000000002e-06 2020-08-07T15:09:49.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:49.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113905+0000", 2020-08-07T15:09:49.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42168640400000001 2020-08-07T15:09:49.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:49.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535591+0000", 2020-08-07T15:09:49.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2820000000000002e-06 2020-08-07T15:09:49.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:49.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535598+0000", 2020-08-07T15:09:49.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6578999999999999e-05 2020-08-07T15:09:49.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:49.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535684+0000", 2020-08-07T15:09:49.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8449999999999992e-06 2020-08-07T15:09:49.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:49.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535693+0000", 2020-08-07T15:09:49.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7366000000000003e-05 2020-08-07T15:09:49.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:49.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535791+0000", 2020-08-07T15:09:49.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954 2020-08-07T15:09:49.512 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:49.512 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:49.512 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:49.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.513 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:37024 4.0 e106/81)", 2020-08-07T15:09:49.513 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:09:49.513 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 53.935085815000001, 2020-08-07T15:09:49.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999, 2020-08-07T15:09:49.531 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:49.531 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:49.532 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:49.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:49.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:09:49.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:49.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:49.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:09:49.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4926000000000001e-05 2020-08-07T15:09:49.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:49.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228683+0000", 2020-08-07T15:09:49.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1280000000000002e-06 2020-08-07T15:09:49.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:49.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228686+0000", 2020-08-07T15:09:49.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69740012100000004 2020-08-07T15:09:49.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:49.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926086+0000", 2020-08-07T15:09:49.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000004e-06 2020-08-07T15:09:49.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:49.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926090+0000", 2020-08-07T15:09:49.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113526 2020-08-07T15:09:49.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:49.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926204+0000", 2020-08-07T15:09:49.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8824000000000001e-05 2020-08-07T15:09:49.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:49.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926222+0000", 2020-08-07T15:09:49.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999 2020-08-07T15:09:49.593 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:49.593 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:49.593 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:49.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.611 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42503 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2615, mlcod=112'2607)", 2020-08-07T15:09:49.611 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:09:49.611 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.867513899000002, 2020-08-07T15:09:49.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998, 2020-08-07T15:09:49.611 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:49.612 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:49.612 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:49.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:49.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:09:49.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:49.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:49.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:09:49.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270999999999998e-05 2020-08-07T15:09:49.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:49.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296292+0000", 2020-08-07T15:09:49.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4729999999999997e-06 2020-08-07T15:09:49.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:49.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296298+0000", 2020-08-07T15:09:49.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.448795049 2020-08-07T15:09:49.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:49.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745093+0000", 2020-08-07T15:09:49.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0140000000000001e-06 2020-08-07T15:09:49.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:49.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745100+0000", 2020-08-07T15:09:49.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2596999999999998e-05 2020-08-07T15:09:49.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:49.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745153+0000", 2020-08-07T15:09:49.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1662000000000003e-05 2020-08-07T15:09:49.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:49.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745184+0000", 2020-08-07T15:09:49.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093174800000000004 2020-08-07T15:09:49.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:49.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746116+0000", 2020-08-07T15:09:49.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0926999999999994e-05 2020-08-07T15:09:49.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:49.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746187+0000", 2020-08-07T15:09:49.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998 2020-08-07T15:09:49.756 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:49.756 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:49.757 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:49.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.758 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42504 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2616, mlcod=113'2609)", 2020-08-07T15:09:49.758 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:09:49.759 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.864177005999998, 2020-08-07T15:09:49.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998, 2020-08-07T15:09:49.759 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:49.760 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:49.760 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:49.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:49.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:09:49.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:49.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:49.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:09:49.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2154000000000002e-05 2020-08-07T15:09:49.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:49.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299629+0000", 2020-08-07T15:09:49.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1929999999999998e-06 2020-08-07T15:09:49.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:49.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299636+0000", 2020-08-07T15:09:49.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44766315400000001 2020-08-07T15:09:49.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:49.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747299+0000", 2020-08-07T15:09:49.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5889999999999999e-06 2020-08-07T15:09:49.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:49.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747305+0000", 2020-08-07T15:09:49.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3094999999999999e-05 2020-08-07T15:09:49.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:49.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747358+0000", 2020-08-07T15:09:49.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3283e-05 2020-08-07T15:09:49.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:49.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747392+0000", 2020-08-07T15:09:49.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015319509999999999 2020-08-07T15:09:49.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:49.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748924+0000", 2020-08-07T15:09:49.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5235999999999996e-05 2020-08-07T15:09:49.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:49.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748999+0000", 2020-08-07T15:09:49.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998 2020-08-07T15:09:49.831 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:49.832 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:49.832 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:49.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.850 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42519 4.8 e113/81 4:13071de5:::benchmark_data_smithi074_44586_object5314:head v 113'2623, mlcod=113'2617)", 2020-08-07T15:09:49.851 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.762041+0000", 2020-08-07T15:09:49.851 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.401712946000004, 2020-08-07T15:09:49.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.34605073400000003, 2020-08-07T15:09:49.852 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:49.852 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:49.852 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:49.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:49.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.762041+0000", 2020-08-07T15:09:49.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:49.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:49.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.762041+0000", 2020-08-07T15:09:49.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9067000000000002e-05 2020-08-07T15:09:49.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:49.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.762100+0000", 2020-08-07T15:09:49.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3529999999999997e-06 2020-08-07T15:09:49.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:49.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.762108+0000", 2020-08-07T15:09:49.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.34498205900000001 2020-08-07T15:09:49.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:49.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.107090+0000", 2020-08-07T15:09:49.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7420000000000003e-06 2020-08-07T15:09:49.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:49.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.107096+0000", 2020-08-07T15:09:49.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1074000000000004e-05 2020-08-07T15:09:49.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:49.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.107167+0000", 2020-08-07T15:09:49.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4423999999999999e-05 2020-08-07T15:09:49.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:49.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.107191+0000", 2020-08-07T15:09:49.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00084080199999999998 2020-08-07T15:09:49.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:49.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.108032+0000", 2020-08-07T15:09:49.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0213000000000001e-05 2020-08-07T15:09:49.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:49.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.108092+0000", 2020-08-07T15:09:49.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.34605073400000003 2020-08-07T15:09:49.894 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:49.894 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:49.894 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:49.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.895 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42520 4.8 e113/81 4:13071de5:::benchmark_data_smithi074_44586_object5314:head v 113'2624, mlcod=113'2618)", 2020-08-07T15:09:49.895 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:09:49.895 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.398232796999999, 2020-08-07T15:09:49.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001, 2020-08-07T15:09:49.896 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:49.896 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:49.896 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:49.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:49.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:09:49.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:49.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:49.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:09:49.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3967999999999995e-05 2020-08-07T15:09:49.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:49.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765585+0000", 2020-08-07T15:09:49.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0949999999999998e-06 2020-08-07T15:09:49.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:49.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765593+0000", 2020-08-07T15:09:49.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52703464200000005 2020-08-07T15:09:49.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:49.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292627+0000", 2020-08-07T15:09:49.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6929999999999996e-06 2020-08-07T15:09:49.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:49.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292636+0000", 2020-08-07T15:09:49.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9292000000000001e-05 2020-08-07T15:09:49.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:49.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292695+0000", 2020-08-07T15:09:49.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5395999999999999e-05 2020-08-07T15:09:49.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:49.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292721+0000", 2020-08-07T15:09:49.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00090361399999999996 2020-08-07T15:09:49.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:49.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293624+0000", 2020-08-07T15:09:49.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1761000000000002e-05 2020-08-07T15:09:49.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:49.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:49.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:49.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293666+0000", 2020-08-07T15:09:49.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001 2020-08-07T15:09:49.940 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:49.940 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:49.940 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:49.941 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:49.941 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:49.941 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:09:49.960 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:48.909+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:49.971 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:09:49.973 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:09:50.015 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 83 16 105589 105573 9.9363 10.4922 0.0111023 0.0125761 2020-08-07T15:09:50.016 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:49.972+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:50.017 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:09:50.112 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:50.100+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:50.180 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:50.179+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:50.263 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:09:50.298 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:09:50.299 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:09:50.299 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:09:50.301 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 84 16 107133 107117 9.9616 12.0625 0.0067986 0.0125421 2020-08-07T15:09:50.301 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:09:50.301 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:50.299+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:50.320 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:09:50.367 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:09:50.401 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:09:50.407 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:50.400+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:50.444 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:09:50.492 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:09:50.506 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:50.505+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:50.534 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:09:50.574 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:50.575 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:50.579 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:09:50.580 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:09:50.608 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.5 to 1 (10000) 2020-08-07T15:09:50.609 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:50.608+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:50.610 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 6 1 2020-08-07T15:09:50.618 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:09:50.619 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:09:50.620 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:09:50.620 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:09:50.620 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:09:50.620 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:09:50.620 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:09:50.621 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:09:50.621 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:09:50.759 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:50.758+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:50.806 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:50.806 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:50.818 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:09:50.860 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:50.859+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:50.967 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:50.966+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:51.062 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:51.063 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:51.077 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:51.076+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:51.080 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:09:51.179 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:51.177+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:51.289 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 85 16 108615 108599 9.98061 11.5781 0.0076333 0.01252 2020-08-07T15:09:51.289 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:51.287+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:51.304 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:51.305 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:51.318 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:09:51.392 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:51.391+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:51.502 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:51.503 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:51.503 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:51.502+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:51.515 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:09:51.604 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:51.602+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:51.609 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.6 to 1 (10000) 2020-08-07T15:09:51.630 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 7 1 2020-08-07T15:09:51.720 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:51.717+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:51.762 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:51.763 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:51.779 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:09:51.814 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:51.811+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:51.916 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:51.912+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:52.002 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:52.003 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:52.012 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:09:52.019 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:09:52.047 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:09:52.046+0000 7f43f0d10700 0 --2- 172.21.15.74:0/2003390760 >> [v2:172.21.15.74:3301/0,v1:172.21.15.74:6790/0] conn(0x7f43ec136cd0 0x7f43ec1419d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2020-08-07T15:09:52.048 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:52.046+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:52.118 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:52.116+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:52.220 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:52.218+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:52.263 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:52.263 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:52.279 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:09:52.280 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 86 16 110118 110102 10.0011 11.7422 0.00485545 0.012493 2020-08-07T15:09:52.322 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:52.319+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:52.421 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:52.420+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:52.466 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:52.466 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:52.477 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:09:52.558 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:52.529+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:52.628 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:52.623+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:52.640 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:09:52.671 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:52.672 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:52.684 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:09:52.726 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:52.724+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:52.828 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:52.824+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:52.870 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:09:52.922 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:09:52.951 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:52.951 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:52.952 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:09:52.953 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:52.942+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:52.998 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:09:52.999 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:09:53.032 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:09:53.049 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:53.046+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:53.076 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:09:53.110 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:09:53.130 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:09:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:09:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:111366 4.f e150/81 4:f5cac1c3:::benchmark_data_smithi074_44586_object13920:head v 150'6862, mlcod=150'6856)", 2020-08-07T15:09:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:53.125587+0000", 2020-08-07T15:09:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00077360500000000002, 2020-08-07T15:09:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00079746199999999998, 2020-08-07T15:09:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:53.125587+0000", 2020-08-07T15:09:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:53.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:53.125587+0000", 2020-08-07T15:09:53.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9241999999999999e-05 2020-08-07T15:09:53.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:53.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:53.125616+0000", 2020-08-07T15:09:53.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8920000000000002e-06 2020-08-07T15:09:53.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:53.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:53.125620+0000", 2020-08-07T15:09:53.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7279000000000001e-05 2020-08-07T15:09:53.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:53.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:53.125647+0000", 2020-08-07T15:09:53.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5200000000000002e-06 2020-08-07T15:09:53.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:53.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:53.125651+0000", 2020-08-07T15:09:53.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2487999999999998e-05 2020-08-07T15:09:53.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:53.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:53.125683+0000", 2020-08-07T15:09:53.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.927e-05 2020-08-07T15:09:53.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:53.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:53.125702+0000", 2020-08-07T15:09:53.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000115691 2020-08-07T15:09:53.176 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:53.177 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:53.177 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:53.177 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:53.177 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:09:53.177 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 1 2020-08-07T15:09:53.178 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:09:53.180 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:09:53.180 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:09:53.181 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:09:53.181 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:09:53.182 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:09:53.182 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:09:53.183 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:09:53.183 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:09:53.183 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:09:53.183 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:53.179+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:53.184 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:09:53.210 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:09:53.211 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.7 to 1 (10000) 2020-08-07T15:09:53.232 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2020-08-07T15:09:53.233 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:09:53.292 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 87 16 111572 111556 10.0167 11.3594 0.0192377 0.0124744 2020-08-07T15:09:53.293 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:53.290+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:53.375 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:09:53.375 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:09:53.376 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:09:53.376 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:09:53.376 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:09:53.405 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:09:53.406 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:53.404+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:53.535 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:53.504+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:53.611 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:53.607+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:53.615 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:09:53.624 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:09:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:09:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:09:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:108376 3.3 e62/20 ondisk, result = 0)", 2020-08-07T15:09:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:09:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 122.92634060499999, 2020-08-07T15:09:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62926872599999994, 2020-08-07T15:09:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:09:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:09:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8491999999999998e-05 2020-08-07T15:09:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683061+0000", 2020-08-07T15:09:53.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9099999999999998e-06 2020-08-07T15:09:53.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683065+0000", 2020-08-07T15:09:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62906670600000003 2020-08-07T15:09:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:53.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312131+0000", 2020-08-07T15:09:53.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2147000000000001e-05 2020-08-07T15:09:53.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312143+0000", 2020-08-07T15:09:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5152e-05 2020-08-07T15:09:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312239+0000", 2020-08-07T15:09:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5089999999999993e-06 2020-08-07T15:09:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:53.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312247+0000", 2020-08-07T15:09:53.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3810000000000001e-05 2020-08-07T15:09:53.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:53.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312311+0000", 2020-08-07T15:09:53.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62926872599999994 2020-08-07T15:09:53.640 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:53.640 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:53.640 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:53.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.641 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "MOSDPGPushReply(3.1 62/60 [PushReplyOp(3:8329f37e:::benchmark_data_smithi074_35447_object11915:head)])", 2020-08-07T15:09:53.641 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:09:53.641 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 122.750982589, 2020-08-07T15:09:53.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996, 2020-08-07T15:09:53.641 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:53.642 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:53.642 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:53.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:53.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:09:53.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:53.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:53.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:09:53.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5372000000000001e-05 2020-08-07T15:09:53.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:53.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858416+0000", 2020-08-07T15:09:53.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6899999999999998e-06 2020-08-07T15:09:53.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:53.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858419+0000", 2020-08-07T15:09:53.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63907904299999996 2020-08-07T15:09:53.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:53.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497498+0000", 2020-08-07T15:09:53.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3332999999999999e-05 2020-08-07T15:09:53.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:53.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497512+0000", 2020-08-07T15:09:53.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000450875 2020-08-07T15:09:53.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:53.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497962+0000", 2020-08-07T15:09:53.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000267779 2020-08-07T15:09:53.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:53.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.498230+0000", 2020-08-07T15:09:53.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996 2020-08-07T15:09:53.652 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:53.652 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:53.652 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:53.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.653 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:145792 3.1 3:881e0420:::benchmark_data_smithi074_35447_object16183:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:09:53.653 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:09:53.653 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 98.942576708000004, 2020-08-07T15:09:53.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997, 2020-08-07T15:09:53.654 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:53.654 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:53.654 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:53.657 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:09:53.657 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:09:53.657 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 145792 2020-08-07T15:09:53.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.658 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:53.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:53.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:09:53.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:53.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:53.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:09:53.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7489000000000001e-05 2020-08-07T15:09:53.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:53.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666824+0000", 2020-08-07T15:09:53.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0169999999999998e-06 2020-08-07T15:09:53.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:53.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666828+0000", 2020-08-07T15:09:53.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9829999999999996e-06 2020-08-07T15:09:53.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:53.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666836+0000", 2020-08-07T15:09:53.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.481e-06 2020-08-07T15:09:53.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:53.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666840+0000", 2020-08-07T15:09:53.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5299000000000001e-05 2020-08-07T15:09:53.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:53.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666885+0000", 2020-08-07T15:09:53.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014456500000000001 2020-08-07T15:09:53.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:53.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667030+0000", 2020-08-07T15:09:53.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017677799999999999 2020-08-07T15:09:53.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,5", 2020-08-07T15:09:53.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667207+0000", 2020-08-07T15:09:53.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004149649 2020-08-07T15:09:53.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:53.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671356+0000", 2020-08-07T15:09:53.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019339649999999999 2020-08-07T15:09:53.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:53.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673290+0000", 2020-08-07T15:09:53.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73346620500000004 2020-08-07T15:09:53.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:53.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406756+0000", 2020-08-07T15:09:53.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4700000000000001e-06 2020-08-07T15:09:53.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:53.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406759+0000", 2020-08-07T15:09:53.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6258999999999998e-05 2020-08-07T15:09:53.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:53.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406815+0000", 2020-08-07T15:09:53.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0126000000000002e-05 2020-08-07T15:09:53.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:53.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406855+0000", 2020-08-07T15:09:53.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997 2020-08-07T15:09:53.681 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:53.681 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:53.682 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:53.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.682 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:145792 3.1 e75/67 ondisk, result = 0)", 2020-08-07T15:09:53.682 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:09:53.683 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 98.93819182, 2020-08-07T15:09:53.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003, 2020-08-07T15:09:53.683 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:53.683 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:53.683 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:53.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:53.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:09:53.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:53.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:53.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:09:53.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2830999999999998e-05 2020-08-07T15:09:53.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:53.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671224+0000", 2020-08-07T15:09:53.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:09:53.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:53.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671229+0000", 2020-08-07T15:09:53.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.735414864 2020-08-07T15:09:53.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:53.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406644+0000", 2020-08-07T15:09:53.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1250000000000004e-06 2020-08-07T15:09:53.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:53.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406651+0000", 2020-08-07T15:09:53.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6567e-05 2020-08-07T15:09:53.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:53.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406737+0000", 2020-08-07T15:09:53.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0054000000000001e-05 2020-08-07T15:09:53.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:53.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406747+0000", 2020-08-07T15:09:53.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011391999999999999 2020-08-07T15:09:53.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:53.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406861+0000", 2020-08-07T15:09:53.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003 2020-08-07T15:09:53.726 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:53.728 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:53.728 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:53.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.734 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30921 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:09:53.734 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:09:53.734 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.903443445999997, 2020-08-07T15:09:53.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001, 2020-08-07T15:09:53.735 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:53.735 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:53.735 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:53.735 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:53.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:53.751 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30921 2020-08-07T15:09:53.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.752 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:53.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:53.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:09:53.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:53.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:53.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:09:53.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6954999999999998e-05 2020-08-07T15:09:53.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:53.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706016+0000", 2020-08-07T15:09:53.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0229999999999992e-06 2020-08-07T15:09:53.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:53.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706024+0000", 2020-08-07T15:09:53.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.432e-05 2020-08-07T15:09:53.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:53.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706039+0000", 2020-08-07T15:09:53.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8490000000000006e-06 2020-08-07T15:09:53.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:53.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706046+0000", 2020-08-07T15:09:53.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6837999999999999e-05 2020-08-07T15:09:53.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:53.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706113+0000", 2020-08-07T15:09:53.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3553000000000001e-05 2020-08-07T15:09:53.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:53.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706127+0000", 2020-08-07T15:09:53.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77277201699999998 2020-08-07T15:09:53.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:53.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.478899+0000", 2020-08-07T15:09:53.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093978599999999996 2020-08-07T15:09:53.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:09:53.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479839+0000", 2020-08-07T15:09:53.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010376029 2020-08-07T15:09:53.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:53.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490215+0000", 2020-08-07T15:09:53.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000190657 2020-08-07T15:09:53.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:53.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490405+0000", 2020-08-07T15:09:53.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001 2020-08-07T15:09:53.793 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:53.794 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:53.794 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:53.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.794 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30922 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:09:53.795 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:09:53.795 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.903283282999993, 2020-08-07T15:09:53.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247, 2020-08-07T15:09:53.795 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:53.795 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:53.796 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:53.796 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:53.796 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:53.796 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30922 2020-08-07T15:09:53.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.796 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:53.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:53.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:09:53.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:53.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:53.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:09:53.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9178999999999997e-05 2020-08-07T15:09:53.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:53.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706159+0000", 2020-08-07T15:09:53.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4999999999999999e-06 2020-08-07T15:09:53.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:53.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706164+0000", 2020-08-07T15:09:53.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1792e-05 2020-08-07T15:09:53.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:53.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706176+0000", 2020-08-07T15:09:53.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.139e-06 2020-08-07T15:09:53.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706182+0000", 2020-08-07T15:09:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039724900000000002 2020-08-07T15:09:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706579+0000", 2020-08-07T15:09:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1191e-05 2020-08-07T15:09:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:53.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706591+0000", 2020-08-07T15:09:53.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329101300000003 2020-08-07T15:09:53.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:53.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479882+0000", 2020-08-07T15:09:53.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9565000000000001e-05 2020-08-07T15:09:53.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:09:53.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479971+0000", 2020-08-07T15:09:53.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010471263 2020-08-07T15:09:53.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:53.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490442+0000", 2020-08-07T15:09:53.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015035600000000001 2020-08-07T15:09:53.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:53.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490593+0000", 2020-08-07T15:09:53.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247 2020-08-07T15:09:53.823 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:53.833 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:53.833 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:53.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.834 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30923 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:09:53.834 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:09:53.834 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.902808222000004, 2020-08-07T15:09:53.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998, 2020-08-07T15:09:53.835 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:53.835 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:53.835 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:53.835 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:53.835 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:53.836 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30923 2020-08-07T15:09:53.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.836 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:53.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:53.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:09:53.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:53.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:53.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:09:53.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5531000000000001e-05 2020-08-07T15:09:53.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:53.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706630+0000", 2020-08-07T15:09:53.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7689999999999999e-06 2020-08-07T15:09:53.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:53.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706637+0000", 2020-08-07T15:09:53.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3577999999999999e-05 2020-08-07T15:09:53.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:53.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706650+0000", 2020-08-07T15:09:53.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1439999999999997e-06 2020-08-07T15:09:53.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:53.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706658+0000", 2020-08-07T15:09:53.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1663999999999999e-05 2020-08-07T15:09:53.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:53.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706709+0000", 2020-08-07T15:09:53.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0149000000000001e-05 2020-08-07T15:09:53.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:53.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706719+0000", 2020-08-07T15:09:53.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329694900000001 2020-08-07T15:09:53.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:53.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480016+0000", 2020-08-07T15:09:53.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7998999999999997e-05 2020-08-07T15:09:53.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:09:53.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480084+0000", 2020-08-07T15:09:53.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010547333000000001 2020-08-07T15:09:53.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:53.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490632+0000", 2020-08-07T15:09:53.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7439e-05 2020-08-07T15:09:53.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:53.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490729+0000", 2020-08-07T15:09:53.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998 2020-08-07T15:09:53.884 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:53.884 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:53.884 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:53.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.885 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30924 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:09:53.885 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:09:53.885 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.902680950000004, 2020-08-07T15:09:53.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995, 2020-08-07T15:09:53.886 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:53.886 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:53.886 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:53.886 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:53.886 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:53.887 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30924 2020-08-07T15:09:53.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.887 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:53.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:53.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:09:53.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:53.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:53.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:09:53.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9698999999999999e-05 2020-08-07T15:09:53.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:53.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706762+0000", 2020-08-07T15:09:53.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8080000000000001e-06 2020-08-07T15:09:53.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:53.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706767+0000", 2020-08-07T15:09:53.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1069e-05 2020-08-07T15:09:53.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:53.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706778+0000", 2020-08-07T15:09:53.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6969999999999998e-06 2020-08-07T15:09:53.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:53.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706784+0000", 2020-08-07T15:09:53.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8792999999999998e-05 2020-08-07T15:09:53.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:53.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706843+0000", 2020-08-07T15:09:53.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5449999999999998e-06 2020-08-07T15:09:53.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:53.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706850+0000", 2020-08-07T15:09:53.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.773278354 2020-08-07T15:09:53.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:53.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480129+0000", 2020-08-07T15:09:53.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1577999999999997e-05 2020-08-07T15:09:53.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:09:53.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480210+0000", 2020-08-07T15:09:53.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010541915000000001 2020-08-07T15:09:53.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:53.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490752+0000", 2020-08-07T15:09:53.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103937 2020-08-07T15:09:53.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:53.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490856+0000", 2020-08-07T15:09:53.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995 2020-08-07T15:09:53.951 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:53.951 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:53.951 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:53.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.952 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30925 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:09:53.952 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:09:53.952 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.902557178999999, 2020-08-07T15:09:53.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995, 2020-08-07T15:09:53.953 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:53.953 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:53.953 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:53.953 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:53.955 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:53.955 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30925 2020-08-07T15:09:53.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.955 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:53.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:53.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:09:53.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:53.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:53.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:09:53.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4664999999999999e-05 2020-08-07T15:09:53.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:53.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706880+0000", 2020-08-07T15:09:53.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7479999999999999e-06 2020-08-07T15:09:53.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:53.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706886+0000", 2020-08-07T15:09:53.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1838000000000001e-05 2020-08-07T15:09:53.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:53.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706898+0000", 2020-08-07T15:09:53.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0089999999999998e-06 2020-08-07T15:09:53.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:53.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706903+0000", 2020-08-07T15:09:53.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3559e-05 2020-08-07T15:09:53.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:53.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706956+0000", 2020-08-07T15:09:53.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0309000000000001e-05 2020-08-07T15:09:53.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:53.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706967+0000", 2020-08-07T15:09:53.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77328290499999996 2020-08-07T15:09:53.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:53.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480250+0000", 2020-08-07T15:09:53.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8680999999999997e-05 2020-08-07T15:09:53.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:09:53.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480298+0000", 2020-08-07T15:09:53.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01059272 2020-08-07T15:09:53.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:53.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490891+0000", 2020-08-07T15:09:53.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014580300000000001 2020-08-07T15:09:53.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:53.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491037+0000", 2020-08-07T15:09:53.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995 2020-08-07T15:09:53.981 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:53.981 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:53.981 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:53.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.982 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30926 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:09:53.982 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:09:53.982 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.902440310000003, 2020-08-07T15:09:53.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005, 2020-08-07T15:09:53.983 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:53.983 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:53.983 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:53.984 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:53.984 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:53.984 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30926 2020-08-07T15:09:53.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.985 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:53.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:53.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:09:53.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:53.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:53.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:09:53.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5223000000000003e-05 2020-08-07T15:09:53.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:53.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706998+0000", 2020-08-07T15:09:53.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4809999999999997e-06 2020-08-07T15:09:53.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:53.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707003+0000", 2020-08-07T15:09:53.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1663e-05 2020-08-07T15:09:53.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:53.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707015+0000", 2020-08-07T15:09:53.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9729999999999997e-06 2020-08-07T15:09:53.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:53.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707020+0000", 2020-08-07T15:09:53.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2565999999999999e-05 2020-08-07T15:09:53.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:53.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707062+0000", 2020-08-07T15:09:53.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8944e-05 2020-08-07T15:09:53.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:53.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707081+0000", 2020-08-07T15:09:53.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77325885000000005 2020-08-07T15:09:53.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:53.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480340+0000", 2020-08-07T15:09:53.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0321000000000006e-05 2020-08-07T15:09:53.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:09:53.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480420+0000", 2020-08-07T15:09:53.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010655958 2020-08-07T15:09:53.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:53.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491076+0000", 2020-08-07T15:09:53.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9784999999999995e-05 2020-08-07T15:09:53.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:53.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:53.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491166+0000", 2020-08-07T15:09:53.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005 2020-08-07T15:09:53.999 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:53.999 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:53.999 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:53.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:53.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.000 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30927 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:09:54.000 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:09:54.001 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.902322491000007, 2020-08-07T15:09:54.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999, 2020-08-07T15:09:54.001 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:54.001 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:54.002 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:54.002 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:54.003 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:54.003 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30927 2020-08-07T15:09:54.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.003 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:54.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:54.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:09:54.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:54.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:54.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:09:54.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9861999999999997e-05 2020-08-07T15:09:54.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:54.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707120+0000", 2020-08-07T15:09:54.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8969999999999996e-06 2020-08-07T15:09:54.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:54.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707126+0000", 2020-08-07T15:09:54.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1248e-05 2020-08-07T15:09:54.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:54.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707137+0000", 2020-08-07T15:09:54.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.955e-06 2020-08-07T15:09:54.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:54.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707142+0000", 2020-08-07T15:09:54.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0643999999999998e-05 2020-08-07T15:09:54.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:54.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707193+0000", 2020-08-07T15:09:54.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1983e-05 2020-08-07T15:09:54.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:54.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707205+0000", 2020-08-07T15:09:54.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77324914 2020-08-07T15:09:54.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:54.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480454+0000", 2020-08-07T15:09:54.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4119999999999998e-05 2020-08-07T15:09:54.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:09:54.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480498+0000", 2020-08-07T15:09:54.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010709707000000001 2020-08-07T15:09:54.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:54.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491208+0000", 2020-08-07T15:09:54.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014433800000000001 2020-08-07T15:09:54.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:54.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491352+0000", 2020-08-07T15:09:54.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999 2020-08-07T15:09:54.068 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:54.068 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:54.068 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:54.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.069 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30928 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:09:54.069 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:09:54.087 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.902198134000002, 2020-08-07T15:09:54.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998, 2020-08-07T15:09:54.088 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:54.088 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:54.088 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:54.088 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:54.089 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:54.089 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30928 2020-08-07T15:09:54.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.089 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:54.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:54.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:09:54.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:54.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:54.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:09:54.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6582000000000001e-05 2020-08-07T15:09:54.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:54.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707241+0000", 2020-08-07T15:09:54.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9190000000000004e-06 2020-08-07T15:09:54.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:54.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707247+0000", 2020-08-07T15:09:54.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000188822 2020-08-07T15:09:54.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:54.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707436+0000", 2020-08-07T15:09:54.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2940000000000002e-06 2020-08-07T15:09:54.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707442+0000", 2020-08-07T15:09:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7800000000000003e-05 2020-08-07T15:09:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707490+0000", 2020-08-07T15:09:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2020-08-07T15:09:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707501+0000", 2020-08-07T15:09:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77303498800000003 2020-08-07T15:09:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:54.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480536+0000", 2020-08-07T15:09:54.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7723000000000001e-05 2020-08-07T15:09:54.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:09:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480614+0000", 2020-08-07T15:09:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010812183 2020-08-07T15:09:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491426+0000", 2020-08-07T15:09:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5502999999999999e-05 2020-08-07T15:09:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491502+0000", 2020-08-07T15:09:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998 2020-08-07T15:09:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31009 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:09:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:09:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.878241828, 2020-08-07T15:09:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002, 2020-08-07T15:09:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31009 2020-08-07T15:09:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:09:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:09:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2429000000000002e-05 2020-08-07T15:09:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731193+0000", 2020-08-07T15:09:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9760000000000003e-06 2020-08-07T15:09:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731198+0000", 2020-08-07T15:09:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3597e-05 2020-08-07T15:09:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731212+0000", 2020-08-07T15:09:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1709999999999996e-06 2020-08-07T15:09:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731219+0000", 2020-08-07T15:09:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5402999999999997e-05 2020-08-07T15:09:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731285+0000", 2020-08-07T15:09:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3495e-05 2020-08-07T15:09:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731298+0000", 2020-08-07T15:09:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74935423599999995 2020-08-07T15:09:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480652+0000", 2020-08-07T15:09:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000137806 2020-08-07T15:09:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480790+0000", 2020-08-07T15:09:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022272399999999999 2020-08-07T15:09:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:09:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481013+0000", 2020-08-07T15:09:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003223381 2020-08-07T15:09:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484236+0000", 2020-08-07T15:09:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1370000000000001e-06 2020-08-07T15:09:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:54.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484237+0000", 2020-08-07T15:09:54.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6288e-05 2020-08-07T15:09:54.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:54.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484294+0000", 2020-08-07T15:09:54.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.007260402 2020-08-07T15:09:54.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:54.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491554+0000", 2020-08-07T15:09:54.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7358000000000003e-05 2020-08-07T15:09:54.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:54.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491601+0000", 2020-08-07T15:09:54.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1100000000000004e-07 2020-08-07T15:09:54.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:54.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491602+0000", 2020-08-07T15:09:54.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013436500000000001 2020-08-07T15:09:54.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:54.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491737+0000", 2020-08-07T15:09:54.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002 2020-08-07T15:09:54.174 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:54.174 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:54.174 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:54.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31010 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:09:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:09:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.878115874000002, 2020-08-07T15:09:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001, 2020-08-07T15:09:54.176 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:54.176 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:54.176 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:54.176 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:54.176 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:54.177 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31010 2020-08-07T15:09:54.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.177 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:54.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:54.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:09:54.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:54.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:54.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:09:54.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2398000000000002e-05 2020-08-07T15:09:54.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:54.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731319+0000", 2020-08-07T15:09:54.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7100000000000004e-06 2020-08-07T15:09:54.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:54.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731325+0000", 2020-08-07T15:09:54.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4328000000000001e-05 2020-08-07T15:09:54.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:54.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731339+0000", 2020-08-07T15:09:54.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3310000000000003e-06 2020-08-07T15:09:54.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:54.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731345+0000", 2020-08-07T15:09:54.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8952999999999998e-05 2020-08-07T15:09:54.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:54.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731384+0000", 2020-08-07T15:09:54.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0036e-05 2020-08-07T15:09:54.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:54.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731394+0000", 2020-08-07T15:09:54.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75027681300000004 2020-08-07T15:09:54.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:54.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481670+0000", 2020-08-07T15:09:54.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016555899999999999 2020-08-07T15:09:54.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:54.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481836+0000", 2020-08-07T15:09:54.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028760999999999999 2020-08-07T15:09:54.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:09:54.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482124+0000", 2020-08-07T15:09:54.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038342680000000001 2020-08-07T15:09:54.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:54.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485958+0000", 2020-08-07T15:09:54.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2559999999999999e-06 2020-08-07T15:09:54.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:54.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485959+0000", 2020-08-07T15:09:54.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3758999999999995e-05 2020-08-07T15:09:54.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:54.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486043+0000", 2020-08-07T15:09:54.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057590419999999998 2020-08-07T15:09:54.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:54.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491802+0000", 2020-08-07T15:09:54.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5757000000000001e-05 2020-08-07T15:09:54.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:54.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:09:54.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9299999999999997e-07 2020-08-07T15:09:54.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:54.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:09:54.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012496999999999999 2020-08-07T15:09:54.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:54.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491993+0000", 2020-08-07T15:09:54.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001 2020-08-07T15:09:54.205 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:54.205 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:54.206 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:54.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.219 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31011 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:09:54.220 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:09:54.220 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.877999669000005, 2020-08-07T15:09:54.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997, 2020-08-07T15:09:54.220 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:54.220 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:54.221 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:54.221 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:54.221 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:54.222 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31011 2020-08-07T15:09:54.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.222 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:54.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:54.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:09:54.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:54.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:54.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:09:54.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9539999999999997e-05 2020-08-07T15:09:54.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:54.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731433+0000", 2020-08-07T15:09:54.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9209999999999997e-06 2020-08-07T15:09:54.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:54.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731439+0000", 2020-08-07T15:09:54.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4246999999999999e-05 2020-08-07T15:09:54.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:54.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731453+0000", 2020-08-07T15:09:54.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8130000000000003e-06 2020-08-07T15:09:54.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:54.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731458+0000", 2020-08-07T15:09:54.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6668000000000001e-05 2020-08-07T15:09:54.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:54.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731504+0000", 2020-08-07T15:09:54.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1112e-05 2020-08-07T15:09:54.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:54.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731515+0000", 2020-08-07T15:09:54.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75121332100000004 2020-08-07T15:09:54.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:54.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482729+0000", 2020-08-07T15:09:54.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1810000000000004e-05 2020-08-07T15:09:54.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:54.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482811+0000", 2020-08-07T15:09:54.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018001399999999999 2020-08-07T15:09:54.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:09:54.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482991+0000", 2020-08-07T15:09:54.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030701140000000001 2020-08-07T15:09:54.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:54.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486061+0000", 2020-08-07T15:09:54.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4499999999999996e-07 2020-08-07T15:09:54.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:54.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486062+0000", 2020-08-07T15:09:54.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5366999999999998e-05 2020-08-07T15:09:54.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:54.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486147+0000", 2020-08-07T15:09:54.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056872779999999996 2020-08-07T15:09:54.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:54.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491834+0000", 2020-08-07T15:09:54.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024645100000000002 2020-08-07T15:09:54.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:54.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:09:54.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1200000000000003e-07 2020-08-07T15:09:54.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:54.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:09:54.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1451000000000002e-05 2020-08-07T15:09:54.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:54.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492173+0000", 2020-08-07T15:09:54.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997 2020-08-07T15:09:54.311 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:54.311 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:54.311 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:54.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.326 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31012 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:09:54.327 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:09:54.327 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.877345550000001, 2020-08-07T15:09:54.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003, 2020-08-07T15:09:54.327 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:54.327 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:54.328 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:54.328 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:54.328 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:54.328 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31012 2020-08-07T15:09:54.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.329 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:54.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:54.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:09:54.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:54.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:54.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:09:54.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6344999999999997e-05 2020-08-07T15:09:54.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:54.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732094+0000", 2020-08-07T15:09:54.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2920000000000001e-06 2020-08-07T15:09:54.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:54.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732100+0000", 2020-08-07T15:09:54.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.189e-05 2020-08-07T15:09:54.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:54.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732112+0000", 2020-08-07T15:09:54.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3720000000000001e-06 2020-08-07T15:09:54.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:54.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732118+0000", 2020-08-07T15:09:54.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2514000000000004e-05 2020-08-07T15:09:54.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:54.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732191+0000", 2020-08-07T15:09:54.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4610000000000001e-05 2020-08-07T15:09:54.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:54.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732205+0000", 2020-08-07T15:09:54.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75132974100000005 2020-08-07T15:09:54.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:54.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483535+0000", 2020-08-07T15:09:54.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9185999999999996e-05 2020-08-07T15:09:54.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:54.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483624+0000", 2020-08-07T15:09:54.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015683 2020-08-07T15:09:54.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:09:54.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483781+0000", 2020-08-07T15:09:54.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046202889999999996 2020-08-07T15:09:54.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:54.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488401+0000", 2020-08-07T15:09:54.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6700000000000002e-07 2020-08-07T15:09:54.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:54.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488402+0000", 2020-08-07T15:09:54.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1487000000000001e-05 2020-08-07T15:09:54.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:54.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488484+0000", 2020-08-07T15:09:54.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035520830000000002 2020-08-07T15:09:54.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:54.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492036+0000", 2020-08-07T15:09:54.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029266600000000002 2020-08-07T15:09:54.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:54.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492328+0000", 2020-08-07T15:09:54.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5099999999999998e-07 2020-08-07T15:09:54.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:54.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492329+0000", 2020-08-07T15:09:54.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000201299 2020-08-07T15:09:54.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:54.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492531+0000", 2020-08-07T15:09:54.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003 2020-08-07T15:09:54.437 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:54.437 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:54.437 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:54.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.438 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31013 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:09:54.438 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:09:54.444 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.877133919000002, 2020-08-07T15:09:54.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996, 2020-08-07T15:09:54.444 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:54.444 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:54.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:54.445 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:54.445 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:54.445 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31013 2020-08-07T15:09:54.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.446 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:54.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:54.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:09:54.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:54.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:54.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:09:54.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8949000000000002e-05 2020-08-07T15:09:54.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:54.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732298+0000", 2020-08-07T15:09:54.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1399999999999997e-06 2020-08-07T15:09:54.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:54.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732304+0000", 2020-08-07T15:09:54.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.154e-05 2020-08-07T15:09:54.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:54.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732315+0000", 2020-08-07T15:09:54.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1319999999999999e-06 2020-08-07T15:09:54.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:54.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732322+0000", 2020-08-07T15:09:54.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7431e-05 2020-08-07T15:09:54.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:54.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732369+0000", 2020-08-07T15:09:54.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0621999999999999e-05 2020-08-07T15:09:54.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:54.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732380+0000", 2020-08-07T15:09:54.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75196845999999995 2020-08-07T15:09:54.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:54.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484348+0000", 2020-08-07T15:09:54.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000123008 2020-08-07T15:09:54.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:54.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484471+0000", 2020-08-07T15:09:54.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017572700000000001 2020-08-07T15:09:54.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:09:54.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484647+0000", 2020-08-07T15:09:54.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038557650000000001 2020-08-07T15:09:54.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:54.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:09:54.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2020-08-07T15:09:54.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:54.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:09:54.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0714e-05 2020-08-07T15:09:54.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:54.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488584+0000", 2020-08-07T15:09:54.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036298319999999999 2020-08-07T15:09:54.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:54.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492214+0000", 2020-08-07T15:09:54.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039530099999999997 2020-08-07T15:09:54.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:54.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492609+0000", 2020-08-07T15:09:54.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5300000000000002e-07 2020-08-07T15:09:54.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:54.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492610+0000", 2020-08-07T15:09:54.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197321 2020-08-07T15:09:54.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:54.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492807+0000", 2020-08-07T15:09:54.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996 2020-08-07T15:09:54.530 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:54.530 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:54.530 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:54.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.531 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31014 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:09:54.531 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:09:54.531 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.875751774999998, 2020-08-07T15:09:54.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998, 2020-08-07T15:09:54.532 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:54.532 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:54.532 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:54.532 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:54.533 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:54.533 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31014 2020-08-07T15:09:54.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.533 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:54.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:54.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:09:54.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:54.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:54.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:09:54.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3743999999999999e-05 2020-08-07T15:09:54.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:54.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733685+0000", 2020-08-07T15:09:54.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1720000000000007e-06 2020-08-07T15:09:54.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:54.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733693+0000", 2020-08-07T15:09:54.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3674e-05 2020-08-07T15:09:54.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:54.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733707+0000", 2020-08-07T15:09:54.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9249999999999998e-06 2020-08-07T15:09:54.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:54.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733714+0000", 2020-08-07T15:09:54.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8338000000000003e-05 2020-08-07T15:09:54.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:54.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733772+0000", 2020-08-07T15:09:54.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3019e-05 2020-08-07T15:09:54.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:54.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733785+0000", 2020-08-07T15:09:54.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75134856299999997 2020-08-07T15:09:54.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:54.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485133+0000", 2020-08-07T15:09:54.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8739999999999995e-05 2020-08-07T15:09:54.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:54.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485212+0000", 2020-08-07T15:09:54.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151244 2020-08-07T15:09:54.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:09:54.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485363+0000", 2020-08-07T15:09:54.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032929090000000001 2020-08-07T15:09:54.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:54.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488656+0000", 2020-08-07T15:09:54.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.065e-06 2020-08-07T15:09:54.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:54.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488657+0000", 2020-08-07T15:09:54.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1123000000000003e-05 2020-08-07T15:09:54.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:54.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488729+0000", 2020-08-07T15:09:54.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035508330000000002 2020-08-07T15:09:54.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:54.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492279+0000", 2020-08-07T15:09:54.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00059376000000000003 2020-08-07T15:09:54.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:54.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492873+0000", 2020-08-07T15:09:54.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5700000000000002e-07 2020-08-07T15:09:54.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:54.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492874+0000", 2020-08-07T15:09:54.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019264 2020-08-07T15:09:54.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:54.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493066+0000", 2020-08-07T15:09:54.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998 2020-08-07T15:09:54.604 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:54.604 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:54.604 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:54.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.605 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31015 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:09:54.605 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:09:54.605 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.875625808999999, 2020-08-07T15:09:54.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002, 2020-08-07T15:09:54.606 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:54.615 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:54.615 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:54.615 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:54.616 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:54.616 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31015 2020-08-07T15:09:54.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.616 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:54.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:54.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:09:54.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:54.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:54.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:09:54.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3707000000000001e-05 2020-08-07T15:09:54.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:54.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733791+0000", 2020-08-07T15:09:54.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8120000000000001e-06 2020-08-07T15:09:54.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:54.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733794+0000", 2020-08-07T15:09:54.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0728e-05 2020-08-07T15:09:54.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:54.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733805+0000", 2020-08-07T15:09:54.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8279999999999999e-06 2020-08-07T15:09:54.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:54.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733811+0000", 2020-08-07T15:09:54.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2955999999999999e-05 2020-08-07T15:09:54.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:54.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733864+0000", 2020-08-07T15:09:54.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1389000000000001e-05 2020-08-07T15:09:54.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:54.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733875+0000", 2020-08-07T15:09:54.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75232711100000005 2020-08-07T15:09:54.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:54.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486202+0000", 2020-08-07T15:09:54.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027503400000000001 2020-08-07T15:09:54.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:54.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486478+0000", 2020-08-07T15:09:54.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033010400000000001 2020-08-07T15:09:54.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:09:54.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486808+0000", 2020-08-07T15:09:54.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019382609999999999 2020-08-07T15:09:54.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:54.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488746+0000", 2020-08-07T15:09:54.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9400000000000004e-07 2020-08-07T15:09:54.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:54.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488747+0000", 2020-08-07T15:09:54.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4324000000000002e-05 2020-08-07T15:09:54.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:54.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488821+0000", 2020-08-07T15:09:54.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043745640000000001 2020-08-07T15:09:54.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:54.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493196+0000", 2020-08-07T15:09:54.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030381000000000001 2020-08-07T15:09:54.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:54.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493499+0000", 2020-08-07T15:09:54.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4e-07 2020-08-07T15:09:54.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:54.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493500+0000", 2020-08-07T15:09:54.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017499 2020-08-07T15:09:54.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:54.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493675+0000", 2020-08-07T15:09:54.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002 2020-08-07T15:09:54.635 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:54.635 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:54.636 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:54.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.636 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31016 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:09:54.637 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:09:54.638 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.875029834000003, 2020-08-07T15:09:54.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971, 2020-08-07T15:09:54.638 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:54.638 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:54.639 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:54.639 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:54.639 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:54.639 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31016 2020-08-07T15:09:54.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.640 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:54.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:54.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:09:54.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:54.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:54.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:09:54.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0924000000000002e-05 2020-08-07T15:09:54.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:54.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734404+0000", 2020-08-07T15:09:54.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2020-08-07T15:09:54.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:54.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734410+0000", 2020-08-07T15:09:54.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3459999999999994e-06 2020-08-07T15:09:54.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:54.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734418+0000", 2020-08-07T15:09:54.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9640000000000002e-06 2020-08-07T15:09:54.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:54.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734423+0000", 2020-08-07T15:09:54.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7253999999999999e-05 2020-08-07T15:09:54.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:54.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734470+0000", 2020-08-07T15:09:54.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0249999999999997e-06 2020-08-07T15:09:54.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:54.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734477+0000", 2020-08-07T15:09:54.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75294592900000001 2020-08-07T15:09:54.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:54.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487423+0000", 2020-08-07T15:09:54.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031604300000000001 2020-08-07T15:09:54.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:54.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487739+0000", 2020-08-07T15:09:54.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000171149 2020-08-07T15:09:54.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:09:54.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487910+0000", 2020-08-07T15:09:54.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024207769999999998 2020-08-07T15:09:54.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:54.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490331+0000", 2020-08-07T15:09:54.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1650000000000001e-06 2020-08-07T15:09:54.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:54.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490332+0000", 2020-08-07T15:09:54.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9796999999999996e-05 2020-08-07T15:09:54.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:54.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490402+0000", 2020-08-07T15:09:54.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029928849999999998 2020-08-07T15:09:54.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:54.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493395+0000", 2020-08-07T15:09:54.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034903199999999999 2020-08-07T15:09:54.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:54.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493744+0000", 2020-08-07T15:09:54.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8899999999999999e-07 2020-08-07T15:09:54.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:54.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493745+0000", 2020-08-07T15:09:54.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197362 2020-08-07T15:09:54.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:54.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493942+0000", 2020-08-07T15:09:54.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971 2020-08-07T15:09:54.690 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:54.690 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:54.690 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:54.690 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:54.690 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:54.691 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:09:54.696 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:09:54.697 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2020-08-07T15:09:53.188025+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":897515520,"num_objects":13697,"num_object_clones":0,"num_object_copies":41091,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":885,"num_objects_unfound":0,"num_objects_dirty":13697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109560,"num_write_kb":876480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2275,"num_bytes_recovered":148971520,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":135,"acting":135,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":11045636,"kb_used_data":2656596,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":743929084,"statfs":{"total":773094113280,"available":761783382016,"internally_reserved":8589934592,"allocated":2720354304,"data_stored":2716687488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,6,5,4,3,0,2,1,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":60162048,"num_objects":918,"num_object_clones":0,"num_object_copies":2754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":59,"num_objects_unfound":0,"num_objects_dirty":918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7344,"num_write_kb":58752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.536318"},"pg_stats":[{"pgid":"1.14","version":"16'2","reported_seq":"79","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.601707+0000","last_change":"2020-08-07T15:08:42.533999+0000","last_active":"2020-08-07T15:09:50.601707+0000","last_peered":"2020-08-07T15:09:50.601707+0000","last_clean":"2020-08-07T15:09:50.601707+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:09:50.601707+0000","last_undegraded":"2020-08-07T15:09:50.601707+0000","last_fullsized":"2020-08-07T15:09:50.601707+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:42.533921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:42.533921+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"149","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.605402+0000","last_change":"2020-08-07T15:09:30.205031+0000","last_active":"2020-08-07T15:09:50.605402+0000","last_peered":"2020-08-07T15:09:50.605402+0000","last_clean":"2020-08-07T15:09:50.605402+0000","last_became_active":"2020-08-07T15:09:30.203592+0000","last_became_peered":"2020-08-07T15:09:30.203592+0000","last_unstale":"2020-08-07T15:09:50.605402+0000","last_undegraded":"2020-08-07T15:09:50.605402+0000","last_fullsized":"2020-08-07T15:09:50.605402+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.141010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.141010+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"79","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.594732+0000","last_change":"2020-08-07T15:08:55.078015+0000","last_active":"2020-08-07T15:09:50.594732+0000","last_peered":"2020-08-07T15:09:50.594732+0000","last_clean":"2020-08-07T15:09:50.594732+0000","last_became_active":"2020-08-07T15:08:41.685106+0000","last_became_peered":"2020-08-07T15:08:41.685106+0000","last_unstale":"2020-08-07T15:09:50.594732+0000","last_undegraded":"2020-08-07T15:09:50.594732+0000","last_fullsized":"2020-08-07T15:09:50.594732+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:55.077924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:55.077924+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"192","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.594713+0000","last_change":"2020-08-07T15:09:30.210669+0000","last_active":"2020-08-07T15:09:50.594713+0000","last_peered":"2020-08-07T15:09:50.594713+0000","last_clean":"2020-08-07T15:09:50.594713+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:09:50.594713+0000","last_undegraded":"2020-08-07T15:09:50.594713+0000","last_fullsized":"2020-08-07T15:09:50.594713+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.544324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.544324+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"186","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.596897+0000","last_change":"2020-08-07T15:08:51.599032+0000","last_active":"2020-08-07T15:09:50.596897+0000","last_peered":"2020-08-07T15:09:50.596897+0000","last_clean":"2020-08-07T15:09:50.596897+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:09:50.596897+0000","last_undegraded":"2020-08-07T15:09:50.596897+0000","last_fullsized":"2020-08-07T15:09:50.596897+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:51.598947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.598947+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"147","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.598537+0000","last_change":"2020-08-07T15:09:41.284321+0000","last_active":"2020-08-07T15:09:50.598537+0000","last_peered":"2020-08-07T15:09:50.598537+0000","last_clean":"2020-08-07T15:09:50.598537+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:09:50.598537+0000","last_undegraded":"2020-08-07T15:09:50.598537+0000","last_fullsized":"2020-08-07T15:09:50.598537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:41.284237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:41.284237+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"63","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.615733+0000","last_change":"2020-08-07T15:09:30.283753+0000","last_active":"2020-08-07T15:09:50.615733+0000","last_peered":"2020-08-07T15:09:50.615733+0000","last_clean":"2020-08-07T15:09:50.615733+0000","last_became_active":"2020-08-07T15:09:30.283176+0000","last_became_peered":"2020-08-07T15:09:30.283176+0000","last_unstale":"2020-08-07T15:09:50.615733+0000","last_undegraded":"2020-08-07T15:09:50.615733+0000","last_fullsized":"2020-08-07T15:09:50.615733+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:25.606757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:25.606757+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"150'6600","reported_seq":"6679","reported_epoch":"150","state":"active+clean+scrubbing","last_fresh":"2020-08-07T15:09:51.690001+0000","last_change":"2020-08-07T15:09:51.601187+0000","last_active":"2020-08-07T15:09:51.690001+0000","last_peered":"2020-08-07T15:09:51.690001+0000","last_clean":"2020-08-07T15:09:51.690001+0000","last_became_active":"2020-08-07T15:08:46.708112+0000","last_became_peered":"2020-08-07T15:08:46.708112+0000","last_unstale":"2020-08-07T15:09:51.690001+0000","last_undegraded":"2020-08-07T15:09:51.690001+0000","last_fullsized":"2020-08-07T15:09:51.690001+0000","mapping_epoch":98,"log_start":"150'6597","ondisk_log_start":"150'6597","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":54067200,"num_objects":825,"num_object_clones":0,"num_object_copies":2475,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6600,"num_write_kb":52800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"147","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.605637+0000","last_change":"2020-08-07T15:08:40.057705+0000","last_active":"2020-08-07T15:09:50.605637+0000","last_peered":"2020-08-07T15:09:50.605637+0000","last_clean":"2020-08-07T15:09:50.605637+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:09:50.605637+0000","last_undegraded":"2020-08-07T15:09:50.605637+0000","last_fullsized":"2020-08-07T15:09:50.605637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:40.057622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:40.057622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"104","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.601329+0000","last_change":"2020-08-07T15:08:53.756554+0000","last_active":"2020-08-07T15:09:50.601329+0000","last_peered":"2020-08-07T15:09:50.601329+0000","last_clean":"2020-08-07T15:09:50.601329+0000","last_became_active":"2020-08-07T15:08:46.710404+0000","last_became_peered":"2020-08-07T15:08:46.710404+0000","last_unstale":"2020-08-07T15:09:50.601329+0000","last_undegraded":"2020-08-07T15:09:50.601329+0000","last_fullsized":"2020-08-07T15:09:50.601329+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:53.756343+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:53.756343+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"150'7232","reported_seq":"9077","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.642016+0000","last_change":"2020-08-07T15:09:35.835839+0000","last_active":"2020-08-07T15:09:51.642016+0000","last_peered":"2020-08-07T15:09:51.642016+0000","last_clean":"2020-08-07T15:09:51.642016+0000","last_became_active":"2020-08-07T15:09:27.180158+0000","last_became_peered":"2020-08-07T15:09:27.180158+0000","last_unstale":"2020-08-07T15:09:51.642016+0000","last_undegraded":"2020-08-07T15:09:51.642016+0000","last_fullsized":"2020-08-07T15:09:51.642016+0000","mapping_epoch":131,"log_start":"150'7226","ondisk_log_start":"150'7226","created":81,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'5536","last_scrub_stamp":"2020-08-07T15:09:35.835763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:35.835763+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":59244544,"num_objects":904,"num_object_clones":0,"num_object_copies":2712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7232,"num_write_kb":57856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":875,"num_bytes_recovered":57344000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"144","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.600155+0000","last_change":"2020-08-07T15:08:27.906894+0000","last_active":"2020-08-07T15:09:50.600155+0000","last_peered":"2020-08-07T15:09:50.600155+0000","last_clean":"2020-08-07T15:09:50.600155+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:09:50.600155+0000","last_undegraded":"2020-08-07T15:09:50.600155+0000","last_fullsized":"2020-08-07T15:09:50.600155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:27.906822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:27.906822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"183","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.598507+0000","last_change":"2020-08-07T15:09:30.202363+0000","last_active":"2020-08-07T15:09:50.598507+0000","last_peered":"2020-08-07T15:09:50.598507+0000","last_clean":"2020-08-07T15:09:50.598507+0000","last_became_active":"2020-08-07T15:09:30.201829+0000","last_became_peered":"2020-08-07T15:09:30.201829+0000","last_unstale":"2020-08-07T15:09:50.598507+0000","last_undegraded":"2020-08-07T15:09:50.598507+0000","last_fullsized":"2020-08-07T15:09:50.598507+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.899280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.899280+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"150'6704","reported_seq":"6780","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.720791+0000","last_change":"2020-08-07T15:08:24.549715+0000","last_active":"2020-08-07T15:09:51.720791+0000","last_peered":"2020-08-07T15:09:51.720791+0000","last_clean":"2020-08-07T15:09:51.720791+0000","last_became_active":"2020-08-07T15:08:24.549245+0000","last_became_peered":"2020-08-07T15:08:24.549245+0000","last_unstale":"2020-08-07T15:09:51.720791+0000","last_undegraded":"2020-08-07T15:09:51.720791+0000","last_fullsized":"2020-08-07T15:09:51.720791+0000","mapping_epoch":81,"log_start":"149'6696","ondisk_log_start":"149'6696","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":54919168,"num_objects":838,"num_object_clones":0,"num_object_copies":2514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6704,"num_write_kb":53632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"147","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:51.036281+0000","last_change":"2020-08-07T15:09:51.036281+0000","last_active":"2020-08-07T15:09:51.036281+0000","last_peered":"2020-08-07T15:09:51.036281+0000","last_clean":"2020-08-07T15:09:51.036281+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:09:51.036281+0000","last_undegraded":"2020-08-07T15:09:51.036281+0000","last_fullsized":"2020-08-07T15:09:51.036281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:51.036224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:51.036224+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"149","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.598302+0000","last_change":"2020-08-07T15:08:50.686578+0000","last_active":"2020-08-07T15:09:50.598302+0000","last_peered":"2020-08-07T15:09:50.598302+0000","last_clean":"2020-08-07T15:09:50.598302+0000","last_became_active":"2020-08-07T15:08:36.633406+0000","last_became_peered":"2020-08-07T15:08:36.633406+0000","last_unstale":"2020-08-07T15:09:50.598302+0000","last_undegraded":"2020-08-07T15:09:50.598302+0000","last_fullsized":"2020-08-07T15:09:50.598302+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:50.686496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:50.686496+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"150'6416","reported_seq":"6493","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.680246+0000","last_change":"2020-08-07T15:08:24.557398+0000","last_active":"2020-08-07T15:09:51.680246+0000","last_peered":"2020-08-07T15:09:51.680246+0000","last_clean":"2020-08-07T15:09:51.680246+0000","last_became_active":"2020-08-07T15:08:24.557022+0000","last_became_peered":"2020-08-07T15:08:24.557022+0000","last_unstale":"2020-08-07T15:09:51.680246+0000","last_undegraded":"2020-08-07T15:09:51.680246+0000","last_fullsized":"2020-08-07T15:09:51.680246+0000","mapping_epoch":81,"log_start":"149'6408","ondisk_log_start":"149'6408","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":52559872,"num_objects":802,"num_object_clones":0,"num_object_copies":2406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6416,"num_write_kb":51328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"176","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.600698+0000","last_change":"2020-08-07T15:08:43.036067+0000","last_active":"2020-08-07T15:09:50.600698+0000","last_peered":"2020-08-07T15:09:50.600698+0000","last_clean":"2020-08-07T15:09:50.600698+0000","last_became_active":"2020-08-07T15:08:36.627426+0000","last_became_peered":"2020-08-07T15:08:36.627426+0000","last_unstale":"2020-08-07T15:09:50.600698+0000","last_undegraded":"2020-08-07T15:09:50.600698+0000","last_fullsized":"2020-08-07T15:09:50.600698+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.035987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.035987+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"144","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.594986+0000","last_change":"2020-08-07T15:08:26.317430+0000","last_active":"2020-08-07T15:09:50.594986+0000","last_peered":"2020-08-07T15:09:50.594986+0000","last_clean":"2020-08-07T15:09:50.594986+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:09:50.594986+0000","last_undegraded":"2020-08-07T15:09:50.594986+0000","last_fullsized":"2020-08-07T15:09:50.594986+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:26.317351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:26.317351+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"150'6856","reported_seq":"6933","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:52.642332+0000","last_change":"2020-08-07T15:08:24.552570+0000","last_active":"2020-08-07T15:09:52.642332+0000","last_peered":"2020-08-07T15:09:52.642332+0000","last_clean":"2020-08-07T15:09:52.642332+0000","last_became_active":"2020-08-07T15:08:24.552105+0000","last_became_peered":"2020-08-07T15:08:24.552105+0000","last_unstale":"2020-08-07T15:09:52.642332+0000","last_undegraded":"2020-08-07T15:09:52.642332+0000","last_fullsized":"2020-08-07T15:09:52.642332+0000","mapping_epoch":81,"log_start":"150'6847","ondisk_log_start":"150'6847","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56164352,"num_objects":857,"num_object_clones":0,"num_object_copies":2571,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6856,"num_write_kb":54848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"165","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.601364+0000","last_change":"2020-08-07T15:09:27.181676+0000","last_active":"2020-08-07T15:09:50.601364+0000","last_peered":"2020-08-07T15:09:50.601364+0000","last_clean":"2020-08-07T15:09:50.601364+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:09:50.601364+0000","last_undegraded":"2020-08-07T15:09:50.601364+0000","last_fullsized":"2020-08-07T15:09:50.601364+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:05.606603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:05.606603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"192","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.599787+0000","last_change":"2020-08-07T15:08:42.069637+0000","last_active":"2020-08-07T15:09:50.599787+0000","last_peered":"2020-08-07T15:09:50.599787+0000","last_clean":"2020-08-07T15:09:50.599787+0000","last_became_active":"2020-08-07T15:08:36.628365+0000","last_became_peered":"2020-08-07T15:08:36.628365+0000","last_unstale":"2020-08-07T15:09:50.599787+0000","last_undegraded":"2020-08-07T15:09:50.599787+0000","last_fullsized":"2020-08-07T15:09:50.599787+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:42.069585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:42.069585+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"149'6904","reported_seq":"6984","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:51.261343+0000","last_change":"2020-08-07T15:09:37.517415+0000","last_active":"2020-08-07T15:09:51.261343+0000","last_peered":"2020-08-07T15:09:51.261343+0000","last_clean":"2020-08-07T15:09:51.261343+0000","last_became_active":"2020-08-07T15:08:24.550222+0000","last_became_peered":"2020-08-07T15:08:24.550222+0000","last_unstale":"2020-08-07T15:09:51.261343+0000","last_undegraded":"2020-08-07T15:09:51.261343+0000","last_fullsized":"2020-08-07T15:09:51.261343+0000","mapping_epoch":81,"log_start":"149'6896","ondisk_log_start":"149'6896","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"141'5600","last_scrub_stamp":"2020-08-07T15:09:37.517334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:37.517334+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56557568,"num_objects":863,"num_object_clones":0,"num_object_copies":2589,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6904,"num_write_kb":55232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"144","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.594884+0000","last_change":"2020-08-07T15:08:30.339852+0000","last_active":"2020-08-07T15:09:50.594884+0000","last_peered":"2020-08-07T15:09:50.594884+0000","last_clean":"2020-08-07T15:09:50.594884+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:09:50.594884+0000","last_undegraded":"2020-08-07T15:09:50.594884+0000","last_fullsized":"2020-08-07T15:09:50.594884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:08:30.339785+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"142","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.595208+0000","last_change":"2020-08-07T15:08:38.386792+0000","last_active":"2020-08-07T15:09:50.595208+0000","last_peered":"2020-08-07T15:09:50.595208+0000","last_clean":"2020-08-07T15:09:50.595208+0000","last_became_active":"2020-08-07T15:08:36.618164+0000","last_became_peered":"2020-08-07T15:08:36.618164+0000","last_unstale":"2020-08-07T15:09:50.595208+0000","last_undegraded":"2020-08-07T15:09:50.595208+0000","last_fullsized":"2020-08-07T15:09:50.595208+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:38.386739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:38.386739+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"150'6704","reported_seq":"7879","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.669793+0000","last_change":"2020-08-07T15:09:36.911393+0000","last_active":"2020-08-07T15:09:51.669793+0000","last_peered":"2020-08-07T15:09:51.669793+0000","last_clean":"2020-08-07T15:09:51.669793+0000","last_became_active":"2020-08-07T15:09:27.183115+0000","last_became_peered":"2020-08-07T15:09:27.183115+0000","last_unstale":"2020-08-07T15:09:51.669793+0000","last_undegraded":"2020-08-07T15:09:51.669793+0000","last_fullsized":"2020-08-07T15:09:51.669793+0000","mapping_epoch":131,"log_start":"150'6696","ondisk_log_start":"150'6696","created":81,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'5280","last_scrub_stamp":"2020-08-07T15:09:36.911321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:36.911321+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":54919168,"num_objects":838,"num_object_clones":0,"num_object_copies":2514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6704,"num_write_kb":53632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":536,"num_bytes_recovered":35127296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"144","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.600554+0000","last_change":"2020-08-07T15:09:37.017383+0000","last_active":"2020-08-07T15:09:50.600554+0000","last_peered":"2020-08-07T15:09:50.600554+0000","last_clean":"2020-08-07T15:09:50.600554+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:09:50.600554+0000","last_undegraded":"2020-08-07T15:09:50.600554+0000","last_fullsized":"2020-08-07T15:09:50.600554+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:37.017307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:37.017307+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"217","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.598646+0000","last_change":"2020-08-07T15:09:30.208873+0000","last_active":"2020-08-07T15:09:50.598646+0000","last_peered":"2020-08-07T15:09:50.598646+0000","last_clean":"2020-08-07T15:09:50.598646+0000","last_became_active":"2020-08-07T15:09:30.208345+0000","last_became_peered":"2020-08-07T15:09:30.208345+0000","last_unstale":"2020-08-07T15:09:50.598646+0000","last_undegraded":"2020-08-07T15:09:50.598646+0000","last_fullsized":"2020-08-07T15:09:50.598646+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.381325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.381325+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"150'6760","reported_seq":"6837","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.712731+0000","last_change":"2020-08-07T15:08:24.552962+0000","last_active":"2020-08-07T15:09:51.712731+0000","last_peered":"2020-08-07T15:09:51.712731+0000","last_clean":"2020-08-07T15:09:51.712731+0000","last_became_active":"2020-08-07T15:08:24.552070+0000","last_became_peered":"2020-08-07T15:08:24.552070+0000","last_unstale":"2020-08-07T15:09:51.712731+0000","last_undegraded":"2020-08-07T15:09:51.712731+0000","last_fullsized":"2020-08-07T15:09:51.712731+0000","mapping_epoch":81,"log_start":"149'6752","ondisk_log_start":"149'6752","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":55377920,"num_objects":845,"num_object_clones":0,"num_object_copies":2535,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6760,"num_write_kb":54080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"119","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.594865+0000","last_change":"2020-08-07T15:08:39.409159+0000","last_active":"2020-08-07T15:09:50.594865+0000","last_peered":"2020-08-07T15:09:50.594865+0000","last_clean":"2020-08-07T15:09:50.594865+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:09:50.594865+0000","last_undegraded":"2020-08-07T15:09:50.594865+0000","last_fullsized":"2020-08-07T15:09:50.594865+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:39.409063+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_clean_scrub_stamp":"2020-08-07T15:08:39.409063+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"150'6752","reported_seq":"6832","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.655457+0000","last_change":"2020-08-07T15:09:40.697977+0000","last_active":"2020-08-07T15:09:51.655457+0000","last_peered":"2020-08-07T15:09:51.655457+0000","last_clean":"2020-08-07T15:09:51.655457+0000","last_became_active":"2020-08-07T15:08:24.553668+0000","last_became_peered":"2020-08-07T15:08:24.553668+0000","last_unstale":"2020-08-07T15:09:51.655457+0000","last_undegraded":"2020-08-07T15:09:51.655457+0000","last_fullsized":"2020-08-07T15:09:51.655457+0000","mapping_epoch":81,"log_start":"149'6744","ondisk_log_start":"149'6744","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"142'5792","last_scrub_stamp":"2020-08-07T15:09:40.697906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:40.697906+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":55312384,"num_objects":844,"num_object_clones":0,"num_object_copies":2532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6752,"num_write_kb":54016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"119","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.596947+0000","last_change":"2020-08-07T15:09:09.691838+0000","last_active":"2020-08-07T15:09:50.596947+0000","last_peered":"2020-08-07T15:09:50.596947+0000","last_clean":"2020-08-07T15:09:50.596947+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:09:50.596947+0000","last_undegraded":"2020-08-07T15:09:50.596947+0000","last_fullsized":"2020-08-07T15:09:50.596947+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.691753+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"150'6728","reported_seq":"6808","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.639617+0000","last_change":"2020-08-07T15:09:42.152452+0000","last_active":"2020-08-07T15:09:51.639617+0000","last_peered":"2020-08-07T15:09:51.639617+0000","last_clean":"2020-08-07T15:09:51.639617+0000","last_became_active":"2020-08-07T15:08:46.708674+0000","last_became_peered":"2020-08-07T15:08:46.708674+0000","last_unstale":"2020-08-07T15:09:51.639617+0000","last_undegraded":"2020-08-07T15:09:51.639617+0000","last_fullsized":"2020-08-07T15:09:51.639617+0000","mapping_epoch":98,"log_start":"149'6720","ondisk_log_start":"149'6720","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"142'5792","last_scrub_stamp":"2020-08-07T15:09:42.152379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:42.152379+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":55115776,"num_objects":841,"num_object_clones":0,"num_object_copies":2523,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6728,"num_write_kb":53824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"183","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.599770+0000","last_change":"2020-08-07T15:09:25.984108+0000","last_active":"2020-08-07T15:09:50.599770+0000","last_peered":"2020-08-07T15:09:50.599770+0000","last_clean":"2020-08-07T15:09:50.599770+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:09:50.599770+0000","last_undegraded":"2020-08-07T15:09:50.599770+0000","last_fullsized":"2020-08-07T15:09:50.599770+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:25.984018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:25.984018+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"150'6872","reported_seq":"6973","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.676987+0000","last_change":"2020-08-07T15:09:38.739730+0000","last_active":"2020-08-07T15:09:51.676987+0000","last_peered":"2020-08-07T15:09:51.676987+0000","last_clean":"2020-08-07T15:09:51.676987+0000","last_became_active":"2020-08-07T15:08:52.166336+0000","last_became_peered":"2020-08-07T15:08:52.166336+0000","last_unstale":"2020-08-07T15:09:51.676987+0000","last_undegraded":"2020-08-07T15:09:51.676987+0000","last_fullsized":"2020-08-07T15:09:51.676987+0000","mapping_epoch":104,"log_start":"150'6868","ondisk_log_start":"150'6868","created":81,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"141'5712","last_scrub_stamp":"2020-08-07T15:09:38.739630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:38.739630+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56295424,"num_objects":859,"num_object_clones":0,"num_object_copies":2577,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6872,"num_write_kb":54976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":40960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"138","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.599966+0000","last_change":"2020-08-07T15:08:51.513398+0000","last_active":"2020-08-07T15:09:50.599966+0000","last_peered":"2020-08-07T15:09:50.599966+0000","last_clean":"2020-08-07T15:09:50.599966+0000","last_became_active":"2020-08-07T15:08:41.680448+0000","last_became_peered":"2020-08-07T15:08:41.680448+0000","last_unstale":"2020-08-07T15:09:50.599966+0000","last_undegraded":"2020-08-07T15:09:50.599966+0000","last_fullsized":"2020-08-07T15:09:50.599966+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:51.513319+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.513319+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"150'6712","reported_seq":"6792","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.709182+0000","last_change":"2020-08-07T15:09:50.869480+0000","last_active":"2020-08-07T15:09:51.709182+0000","last_peered":"2020-08-07T15:09:51.709182+0000","last_clean":"2020-08-07T15:09:51.709182+0000","last_became_active":"2020-08-07T15:08:24.549868+0000","last_became_peered":"2020-08-07T15:08:24.549868+0000","last_unstale":"2020-08-07T15:09:51.709182+0000","last_undegraded":"2020-08-07T15:09:51.709182+0000","last_fullsized":"2020-08-07T15:09:51.709182+0000","mapping_epoch":81,"log_start":"149'6704","ondisk_log_start":"149'6704","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'6599","last_scrub_stamp":"2020-08-07T15:09:50.869400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:50.869400+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":54984704,"num_objects":839,"num_object_clones":0,"num_object_copies":2517,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6712,"num_write_kb":53696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"109","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.615815+0000","last_change":"2020-08-07T15:09:36.687393+0000","last_active":"2020-08-07T15:09:50.615815+0000","last_peered":"2020-08-07T15:09:50.615815+0000","last_clean":"2020-08-07T15:09:50.615815+0000","last_became_active":"2020-08-07T15:09:30.204655+0000","last_became_peered":"2020-08-07T15:09:30.204655+0000","last_unstale":"2020-08-07T15:09:50.615815+0000","last_undegraded":"2020-08-07T15:09:50.615815+0000","last_fullsized":"2020-08-07T15:09:50.615815+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:36.687314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:36.687314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"150'7080","reported_seq":"8938","reported_epoch":"150","state":"active+remapped+backfill_toofull","last_fresh":"2020-08-07T15:09:51.703178+0000","last_change":"2020-08-07T15:09:28.235048+0000","last_active":"2020-08-07T15:09:51.703178+0000","last_peered":"2020-08-07T15:09:51.703178+0000","last_clean":"2020-08-07T15:09:25.573158+0000","last_became_active":"2020-08-07T15:09:28.212799+0000","last_became_peered":"2020-08-07T15:09:28.212799+0000","last_unstale":"2020-08-07T15:09:51.703178+0000","last_undegraded":"2020-08-07T15:09:51.703178+0000","last_fullsized":"2020-08-07T15:09:51.703178+0000","mapping_epoch":132,"log_start":"150'7074","ondisk_log_start":"150'7074","created":81,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57999360,"num_objects":885,"num_object_clones":0,"num_object_copies":2655,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":885,"num_objects_unfound":0,"num_objects_dirty":885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7080,"num_write_kb":56640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":862,"num_bytes_recovered":56459264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,6],"avail_no_missing":["5","1","6"],"object_location_counts":[{"shards":"1,5,6","objects":885}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"182","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.598424+0000","last_change":"2020-08-07T15:09:08.399507+0000","last_active":"2020-08-07T15:09:50.598424+0000","last_peered":"2020-08-07T15:09:50.598424+0000","last_clean":"2020-08-07T15:09:50.598424+0000","last_became_active":"2020-08-07T15:08:41.689435+0000","last_became_peered":"2020-08-07T15:08:41.689435+0000","last_unstale":"2020-08-07T15:09:50.598424+0000","last_undegraded":"2020-08-07T15:09:50.598424+0000","last_fullsized":"2020-08-07T15:09:50.598424+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:08.399439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:08.399439+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"150'7368","reported_seq":"7447","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:52.708850+0000","last_change":"2020-08-07T15:09:52.486813+0000","last_active":"2020-08-07T15:09:52.708850+0000","last_peered":"2020-08-07T15:09:52.708850+0000","last_clean":"2020-08-07T15:09:52.708850+0000","last_became_active":"2020-08-07T15:08:24.554021+0000","last_became_peered":"2020-08-07T15:08:24.554021+0000","last_unstale":"2020-08-07T15:09:52.708850+0000","last_undegraded":"2020-08-07T15:09:52.708850+0000","last_fullsized":"2020-08-07T15:09:52.708850+0000","mapping_epoch":81,"log_start":"150'7363","ondisk_log_start":"150'7363","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'7328","last_scrub_stamp":"2020-08-07T15:09:52.486757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:52.486757+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60358656,"num_objects":921,"num_object_clones":0,"num_object_copies":2763,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7368,"num_write_kb":58944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"69","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.600279+0000","last_change":"2020-08-07T15:09:09.868283+0000","last_active":"2020-08-07T15:09:50.600279+0000","last_peered":"2020-08-07T15:09:50.600279+0000","last_clean":"2020-08-07T15:09:50.600279+0000","last_became_active":"2020-08-07T15:08:46.710059+0000","last_became_peered":"2020-08-07T15:08:46.710059+0000","last_unstale":"2020-08-07T15:09:50.600279+0000","last_undegraded":"2020-08-07T15:09:50.600279+0000","last_fullsized":"2020-08-07T15:09:50.600279+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.868198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.868198+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"150'7032","reported_seq":"7111","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.726534+0000","last_change":"2020-08-07T15:09:26.248248+0000","last_active":"2020-08-07T15:09:51.726534+0000","last_peered":"2020-08-07T15:09:51.726534+0000","last_clean":"2020-08-07T15:09:51.726534+0000","last_became_active":"2020-08-07T15:08:24.552127+0000","last_became_peered":"2020-08-07T15:08:24.552127+0000","last_unstale":"2020-08-07T15:09:51.726534+0000","last_undegraded":"2020-08-07T15:09:51.726534+0000","last_fullsized":"2020-08-07T15:09:51.726534+0000","mapping_epoch":81,"log_start":"150'7027","ondisk_log_start":"150'7027","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"130'4592","last_scrub_stamp":"2020-08-07T15:09:26.248174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:26.248174+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57606144,"num_objects":879,"num_object_clones":0,"num_object_copies":2637,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7032,"num_write_kb":56256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"150'6840","reported_seq":"6920","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:52.473304+0000","last_change":"2020-08-07T15:08:46.710643+0000","last_active":"2020-08-07T15:09:52.473304+0000","last_peered":"2020-08-07T15:09:52.473304+0000","last_clean":"2020-08-07T15:09:52.473304+0000","last_became_active":"2020-08-07T15:08:46.710100+0000","last_became_peered":"2020-08-07T15:08:46.710100+0000","last_unstale":"2020-08-07T15:09:52.473304+0000","last_undegraded":"2020-08-07T15:09:52.473304+0000","last_fullsized":"2020-08-07T15:09:52.473304+0000","mapping_epoch":98,"log_start":"150'6834","ondisk_log_start":"150'6834","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56033280,"num_objects":855,"num_object_clones":0,"num_object_copies":2565,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6840,"num_write_kb":54720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"91","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.594836+0000","last_change":"2020-08-07T15:09:08.653773+0000","last_active":"2020-08-07T15:09:50.594836+0000","last_peered":"2020-08-07T15:09:50.594836+0000","last_clean":"2020-08-07T15:09:50.594836+0000","last_became_active":"2020-08-07T15:08:41.677944+0000","last_became_peered":"2020-08-07T15:08:41.677944+0000","last_unstale":"2020-08-07T15:09:50.594836+0000","last_undegraded":"2020-08-07T15:09:50.594836+0000","last_fullsized":"2020-08-07T15:09:50.594836+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:08.653722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:08.653722+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":897515520,"num_objects":13695,"num_object_clones":0,"num_object_copies":41085,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":885,"num_objects_unfound":0,"num_objects_dirty":13695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109560,"num_write_kb":876480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2275,"num_bytes_recovered":148971520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2726510592,"data_stored":2726510592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":63,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542189,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1436264,"kb_used_data":387680,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92935576,"statfs":{"total":96636764160,"available":95166029824,"internally_reserved":1073741824,"allocated":396984320,"data_stored":396521283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.95099999999999996,"15min":0.91200000000000003},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.1739999999999999,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":1.04},{"interface":"front","average":{"1min":1.095,"5min":1.0149999999999999,"15min":0.97999999999999998},"min":{"1min":0.54300000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.0049999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.357,"15min":1.3149999999999999},"min":{"1min":0.68700000000000006,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":5.6239999999999997,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.105,"5min":1.0720000000000001,"15min":1.0009999999999999},"min":{"1min":0.64100000000000001,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.206,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":4.0060000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.272,"15min":1.175},"min":{"1min":0.71999999999999997,"5min":0.628,"15min":0.628},"max":{"1min":5.6619999999999999,"5min":5.6619999999999999,"15min":5.6619999999999999},"last":1.127},{"interface":"front","average":{"1min":1.446,"5min":1.26,"15min":1.165},"min":{"1min":0.66800000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":5.1319999999999997,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":1.3109999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":75.045000000000002,"5min":16.263999999999999,"15min":6.5359999999999996},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":917.38699999999994,"5min":917.38699999999994,"15min":917.38699999999994},"last":0.90400000000000003},{"interface":"front","average":{"1min":19.670999999999999,"5min":4.9939999999999998,"15min":2.5600000000000001},"min":{"1min":0.80000000000000004,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":333.59199999999998,"5min":333.59199999999998,"15min":333.59199999999998},"last":0.96199999999999997}]},{"osd":4,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.411,"15min":1.321},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":4.5250000000000004,"5min":4.835,"15min":4.835},"last":1.2929999999999999},{"interface":"front","average":{"1min":52.259999999999998,"5min":11.52,"15min":4.7140000000000004},"min":{"1min":0.77000000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":866.24099999999999,"5min":866.24099999999999,"15min":866.24099999999999},"last":0.93500000000000005}]},{"osd":5,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.2869999999999999,"5min":23.440000000000001,"15min":32.817999999999998},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":6.6509999999999998,"5min":831.88099999999997,"15min":831.88099999999997},"last":1.1000000000000001},{"interface":"front","average":{"1min":2.2519999999999998,"5min":1.8380000000000001,"15min":1.609},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":6.6790000000000003,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":1.2529999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":13.818,"5min":4.0019999999999998,"15min":2.3460000000000001},"min":{"1min":0.82999999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":224.14500000000001,"5min":224.14500000000001,"15min":224.14500000000001},"last":2.0590000000000002},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.6839999999999999,"15min":1.5660000000000001},"min":{"1min":0.90600000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":7.4359999999999999,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":1.6419999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1324492,"kb_used_data":275548,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93047348,"statfs":{"total":96636764160,"available":95280484352,"internally_reserved":1073741824,"allocated":282161152,"data_stored":281705611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.179,"15min":1.2809999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8149999999999999,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":1.3160000000000001},{"interface":"front","average":{"1min":15.239000000000001,"5min":3.9529999999999998,"15min":2.0840000000000001},"min":{"1min":0.78100000000000003,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":320.31299999999999,"5min":320.31299999999999,"15min":320.31299999999999},"last":1.238}]},{"osd":1,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":24.821999999999999,"5min":5.7329999999999997,"15min":2.5489999999999999},"min":{"1min":0.46700000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":547.65300000000002,"5min":547.65300000000002,"15min":547.65300000000002},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.3029999999999999,"5min":4.9130000000000003,"15min":2.5619999999999998},"min":{"1min":0.69799999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.5449999999999999,"5min":372.50799999999998,"15min":372.50799999999998},"last":1.7849999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.208,"15min":1.1990000000000001},"min":{"1min":0.502,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":5.2400000000000002,"5min":5.2400000000000002,"15min":5.2400000000000002},"last":0.874},{"interface":"front","average":{"1min":1.286,"5min":1.2390000000000001,"15min":1.2609999999999999},"min":{"1min":0.66800000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":6.0490000000000004,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":1.292}]},{"osd":3,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.4370000000000001,"15min":1.496},"min":{"1min":0.878,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.0670000000000002,"5min":3.46,"15min":3.46},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3600000000000001,"15min":1.3779999999999999},"min":{"1min":0.70299999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":5.2770000000000001,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":1.2669999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.504,"15min":1.6100000000000001},"min":{"1min":0.70099999999999996,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.165,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":1.365},{"interface":"front","average":{"1min":1.46,"5min":1.625,"15min":1.6559999999999999},"min":{"1min":0.96199999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":2.4279999999999999,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.891}]},{"osd":5,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.2809999999999999,"15min":1.3360000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.3069999999999999,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.6259999999999999,"15min":1.7190000000000001},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.4630000000000001,"5min":5.327,"15min":5.327},"last":1.8640000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.458,"15min":1.4410000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.4900000000000002,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.833},{"interface":"front","average":{"1min":1.5720000000000001,"5min":11.603999999999999,"15min":16.085000000000001},"min":{"1min":1.004,"5min":0.87,"15min":0.87},"max":{"1min":2.7719999999999998,"5min":266.26999999999998,"15min":266.26999999999998},"last":2.3330000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1593252,"kb_used_data":544668,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92778588,"statfs":{"total":96636764160,"available":95005274112,"internally_reserved":1073741824,"allocated":557740032,"data_stored":557284491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,5,4,1,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":20.945,"15min":29.771999999999998},"min":{"1min":0.49199999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":6.8810000000000002,"5min":497.04000000000002,"15min":497.04000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":25.385999999999999,"5min":14.656000000000001,"15min":5.6390000000000002},"min":{"1min":0.52200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":513.78899999999999,"5min":993.84500000000003,"15min":993.84500000000003},"last":1.133}]},{"osd":2,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.999,"15min":0.93200000000000005},"min":{"1min":0.60099999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.726,"5min":4.585,"15min":4.585},"last":0.57199999999999995},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.254,"15min":1.22},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":6.4029999999999996,"5min":6.4029999999999996,"15min":6.4029999999999996},"last":1.2130000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":40.524000000000001,"5min":9.2829999999999995,"15min":3.968},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":828.31600000000003,"5min":828.31600000000003,"15min":828.31600000000003},"last":1.0840000000000001},{"interface":"front","average":{"1min":44.292000000000002,"5min":10.169,"15min":4.359},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":905.38300000000004,"5min":905.38300000000004,"15min":905.38300000000004},"last":1.113}]},{"osd":4,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":27.831,"5min":6.774,"15min":3.1139999999999999},"min":{"1min":0.79900000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":550.66800000000001,"5min":550.66800000000001,"15min":550.66800000000001},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.3400000000000001,"15min":1.246},"min":{"1min":0.92100000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":5.9420000000000002,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.1759999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":8.4550000000000001,"15min":11.49},"min":{"1min":0.71199999999999997,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.7200000000000002,"5min":213.63800000000001,"15min":213.63800000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":58.497,"5min":12.720000000000001,"15min":5.0330000000000004},"min":{"1min":0.76500000000000001,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":875.44600000000003,"5min":875.44600000000003,"15min":875.44600000000003},"last":4.6319999999999997}]},{"osd":6,"last update":"Fri Aug 7 15:09:27 2020","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.587,"15min":1.5629999999999999},"min":{"1min":0.81000000000000005,"5min":0.754,"15min":0.754},"max":{"1min":5.7149999999999999,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":1.7030000000000001},{"interface":"front","average":{"1min":11.539,"5min":3.5979999999999999,"15min":2.1059999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":207.137,"5min":207.137,"15min":207.137},"last":1.3029999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.782,"5min":1.6879999999999999,"15min":1.639},"min":{"1min":0.94099999999999995,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":6.444,"5min":6.444,"15min":6.444},"last":2.5139999999999998},{"interface":"front","average":{"1min":1.6819999999999999,"5min":1.6779999999999999,"15min":1.546},"min":{"1min":0.93300000000000005,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":5.9279999999999999,"5min":10.752000000000001,"15min":10.752000000000001},"last":1.1599999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1322708,"kb_used_data":274124,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93049132,"statfs":{"total":96636764160,"available":95282311168,"internally_reserved":1073741824,"allocated":280702976,"data_stored":280247435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":17.058,"15min":6.3369999999999997},"min":{"1min":0.61399999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.1269999999999998,"5min":845.57100000000003,"15min":845.57100000000003},"last":4.5629999999999997},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.0169999999999999,"15min":0.96099999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":4.8209999999999997,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":0.62}]},{"osd":2,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.0189999999999999,"15min":0.95799999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.7719999999999998,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":13.177,"5min":22.48,"15min":29.32},"min":{"1min":0.61399999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":226.911,"5min":606.18499999999995,"15min":606.18499999999995},"last":0.64600000000000002}]},{"osd":3,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.0660000000000001,"15min":0.97899999999999998},"min":{"1min":0.54100000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6859999999999999,"5min":3.1779999999999999,"15min":3.1779999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.3160000000000001,"15min":1.2170000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.976,"5min":4.21,"15min":4.21},"last":0.76300000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.2829999999999999,"15min":1.2050000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":8.3100000000000005,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.7709999999999999,"5min":1.446,"15min":1.3200000000000001},"min":{"1min":0.84599999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":8.4190000000000005,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":0.88900000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.9770000000000001,"5min":9.0380000000000003,"15min":3.8300000000000001},"min":{"1min":0.77800000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":7.8929999999999998,"5min":501.267,"15min":501.267},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.212,"5min":1.2609999999999999,"15min":1.157},"min":{"1min":0.78200000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.0939999999999999,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":0.91500000000000004}]},{"osd":6,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":5.7779999999999996,"15min":7.6680000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.0059999999999998,"5min":124.738,"15min":124.738},"last":0.872},{"interface":"front","average":{"1min":1.627,"5min":11.811,"15min":16.372},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":4.6879999999999997,"5min":294.94,"15min":294.94},"last":1.1659999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.5589999999999999,"15min":1.577},"min":{"1min":0.95399999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":4.7169999999999996,"5min":6.468,"15min":6.468},"last":1.605},{"interface":"front","average":{"1min":1.363,"5min":1.4410000000000001,"15min":1.4039999999999999},"min":{"1min":0.88800000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.1040000000000001,"5min":3.117,"15min":3.117},"last":0.94499999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1207984,"kb_used_data":159400,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93163856,"statfs":{"total":96636764160,"available":95399788544,"internally_reserved":1073741824,"allocated":163225600,"data_stored":162762563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.6950000000000001,"5min":1.268,"15min":1.167},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":8.4930000000000003,"5min":8.4930000000000003,"15min":8.4930000000000003},"last":3.706},{"interface":"front","average":{"1min":1.357,"5min":1.113,"15min":1.0209999999999999},"min":{"1min":0.61899999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":3.5,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.88400000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.9390000000000001,"5min":1.5469999999999999,"15min":1.2430000000000001},"min":{"1min":0.54400000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":5.6820000000000004,"5min":18.672000000000001,"15min":18.672000000000001},"last":2.5339999999999998},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.351,"15min":1.357},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":3.585,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":3.7970000000000002}]},{"osd":3,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.268,"15min":1.198},"min":{"1min":0.72799999999999998,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.8740000000000001,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":3.1970000000000001},{"interface":"front","average":{"1min":34.170999999999999,"5min":8.0510000000000002,"15min":3.7210000000000001},"min":{"1min":0.68799999999999994,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":501.75599999999997,"5min":501.75599999999997,"15min":501.75599999999997},"last":2.5609999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.403,"15min":1.2070000000000001},"min":{"1min":0.65200000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":3.4180000000000001,"5min":19.783000000000001,"15min":19.783000000000001},"last":1.234},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.546,"15min":1.6100000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":3.5569999999999999,"5min":3.5569999999999999,"15min":3.5569999999999999},"last":3.7389999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.292,"15min":1.272},"min":{"1min":0.83299999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.323,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":3.1160000000000001},{"interface":"front","average":{"1min":43.396999999999998,"5min":24.298999999999999,"15min":25.116},"min":{"1min":0.99299999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":627.53300000000002,"5min":627.53300000000002,"15min":627.53300000000002},"last":3.081}]},{"osd":6,"last update":"Fri Aug 7 15:09:30 2020","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.538,"15min":1.3260000000000001},"min":{"1min":0.85999999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":3.452,"5min":20.035,"15min":20.035},"last":1.526},{"interface":"front","average":{"1min":1.498,"5min":9.8979999999999997,"15min":4.1799999999999997},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.976,"5min":751.70000000000005,"15min":751.70000000000005},"last":3.1480000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":54.209000000000003,"5min":12.595000000000001,"15min":5.359},"min":{"1min":0.84999999999999998,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":501.91699999999997,"5min":501.91699999999997,"15min":501.91699999999997},"last":3.7709999999999999},{"interface":"front","average":{"1min":1.665,"5min":1.919,"15min":1.637},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":3.133,"5min":35.646999999999998,"15min":35.646999999999998},"last":3.1739999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1303512,"kb_used_data":254920,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93068328,"statfs":{"total":96636764160,"available":95301967872,"internally_reserved":1073741824,"allocated":261038080,"data_stored":260575043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":1.341,"15min":1.282},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.097,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.4530000000000001,"15min":1.575},"min":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.3039999999999998,"5min":13.067,"15min":13.067},"last":0.89900000000000002}]},{"osd":1,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":4.5860000000000003,"15min":2.794},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.9910000000000001,"5min":270.68700000000001,"15min":270.68700000000001},"last":2.3090000000000002},{"interface":"front","average":{"1min":1.4379999999999999,"5min":2.0390000000000001,"15min":2.0209999999999999},"min":{"1min":0.55100000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":7.0940000000000003,"5min":14.349,"15min":14.349},"last":1.855}]},{"osd":2,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.266,"15min":1.1870000000000001},"min":{"1min":0.71399999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.2639999999999998,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":1.4470000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":2.109,"15min":1.8260000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.661,"5min":51.530000000000001,"15min":51.530000000000001},"last":1.1739999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":10.773999999999999,"15min":4.7220000000000004},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.1360000000000001,"5min":506.21300000000002,"15min":506.21300000000002},"last":2.0150000000000001},{"interface":"front","average":{"1min":1.276,"5min":25.469000000000001,"15min":29.888999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.8940000000000001,"5min":677.88599999999997,"15min":677.88599999999997},"last":3.1480000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.5920000000000001,"15min":1.542},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.9710000000000001,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":3.1869999999999998},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.8009999999999999,"15min":1.7789999999999999},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":3.044,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":3.214}]},{"osd":6,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.6259999999999999,"15min":1.5629999999999999},"min":{"1min":0.89800000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.9359999999999999,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":1.8009999999999999},{"interface":"front","average":{"1min":21.766999999999999,"5min":5.9390000000000001,"15min":3.3039999999999998},"min":{"1min":0.89900000000000002,"5min":0.876,"15min":0.876},"max":{"1min":387.911,"5min":387.911,"15min":387.911},"last":1.2130000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.9330000000000001,"15min":2.085},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.0110000000000001,"5min":5.734,"15min":5.734},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.9750000000000001,"15min":1.976},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.169,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":2.347}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1482524,"kb_used_data":433940,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92889316,"statfs":{"total":96636764160,"available":95118659584,"internally_reserved":1073741824,"allocated":444354560,"data_stored":443899019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":1.0720000000000001,"15min":1.1439999999999999},"min":{"1min":0.52900000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4390000000000001,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":0.55100000000000005},{"interface":"front","average":{"1min":1.143,"5min":1.0069999999999999,"15min":0.90100000000000002},"min":{"1min":0.498,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":4.907,"5min":6.3970000000000002,"15min":6.3970000000000002},"last":0.81299999999999994}]},{"osd":1,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":23.940999999999999,"5min":5.8209999999999997,"15min":2.843},"min":{"1min":0.59599999999999997,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":320.67000000000002,"5min":320.67000000000002,"15min":320.67000000000002},"last":1.413},{"interface":"front","average":{"1min":1.1359999999999999,"5min":7.5700000000000003,"15min":3.4409999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.784,"5min":689.846,"15min":689.846},"last":0.58799999999999997}]},{"osd":2,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.3999999999999999,"15min":1.4339999999999999},"min":{"1min":0.77600000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.0899999999999999,"5min":6.8730000000000002,"15min":6.8730000000000002},"last":2.0219999999999998},{"interface":"front","average":{"1min":1.258,"5min":1.516,"15min":1.5149999999999999},"min":{"1min":0.81699999999999995,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.3159999999999998,"5min":7.1360000000000001,"15min":7.1360000000000001},"last":2.048}]},{"osd":3,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":3.681,"15min":2.2320000000000002},"min":{"1min":0.78300000000000003,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.8149999999999999,"5min":247.202,"15min":247.202},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.3720000000000001,"15min":1.2909999999999999},"min":{"1min":0.68500000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":4.931,"5min":6.431,"15min":6.431},"last":0.99399999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":10.57,"15min":4.6079999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.431,"5min":990.71600000000001,"15min":990.71600000000001},"last":0.878},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.6100000000000001,"15min":1.621},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":3.3519999999999999,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":1.0649999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:09:28 2020","interfaces":[{"interface":"back","average":{"1min":1.6799999999999999,"5min":1.7509999999999999,"15min":1.7769999999999999},"min":{"1min":0.75800000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":5.0149999999999997,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.498,"5min":12.069000000000001,"15min":16.762},"min":{"1min":0.83299999999999996,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":3.3530000000000002,"5min":389.14600000000002,"15min":389.14600000000002},"last":0.93700000000000006}]},{"osd":7,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":3.8300000000000001,"15min":2.2149999999999999},"min":{"1min":0.753,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":4.8710000000000004,"5min":252.798,"15min":252.798},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.262,"5min":10.175000000000001,"15min":4.4889999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.3050000000000002,"5min":501.93000000000001,"15min":501.93000000000001},"last":0.95499999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1374900,"kb_used_data":326316,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92996940,"statfs":{"total":96636764160,"available":95228866560,"internally_reserved":1073741824,"allocated":334147584,"data_stored":333692043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":1.012,"15min":1.0549999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.651,"5min":2.766,"15min":2.766},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91600000000000004,"15min":0.84199999999999997},"min":{"1min":0.45500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.3340000000000001,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":0.86599999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.4540000000000002,"5min":10.521000000000001,"15min":4.1260000000000003},"min":{"1min":0.56699999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":29.283000000000001,"5min":787.22900000000004,"15min":787.22900000000004},"last":1.0089999999999999},{"interface":"front","average":{"1min":8.0939999999999994,"5min":2.4969999999999999,"15min":1.571},"min":{"1min":0.60999999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":116.548,"5min":116.548,"15min":116.548},"last":0.75800000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":5.9340000000000002,"15min":8.1240000000000006},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.78,"5min":168.56899999999999,"15min":168.56899999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.119,"5min":20.361000000000001,"15min":28.873999999999999},"min":{"1min":0.60899999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.665,"5min":676.27099999999996,"15min":676.27099999999996},"last":0.82899999999999996}]},{"osd":3,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.6739999999999999,"5min":1.5069999999999999,"15min":1.2470000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":29.129000000000001,"5min":29.129000000000001,"15min":29.129000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":3.0219999999999998,"5min":9.9459999999999997,"15min":4.1239999999999997},"min":{"1min":0.70799999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":29.103000000000002,"5min":712.98699999999997,"15min":712.98699999999997},"last":0.97399999999999998}]},{"osd":4,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":49.854999999999997,"5min":40.755000000000003,"15min":47.512999999999998},"min":{"1min":0.61899999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":971.67999999999995,"5min":971.67999999999995,"15min":971.67999999999995},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.236,"5min":1.264,"15min":1.292},"min":{"1min":0.64900000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.5960000000000001,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":0.95199999999999996}]},{"osd":6,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":91.870999999999995,"5min":48.862000000000002,"15min":49.893000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":966.60900000000004,"5min":966.60900000000004,"15min":966.60900000000004},"last":1.03},{"interface":"front","average":{"1min":28.396999999999998,"5min":21.527000000000001,"15min":24.52},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":501.79000000000002,"5min":517.26700000000005,"15min":517.26700000000005},"last":0.88100000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.341,"15min":1.319},"min":{"1min":0.77500000000000002,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.8940000000000001,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.109},{"interface":"front","average":{"1min":27.206,"5min":6.6630000000000003,"15min":3.2650000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":515.09100000000001,"5min":515.09100000000001,"15min":515.09100000000001},"last":0.995}]}]}]}} 2020-08-07T15:09:54.700 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:09:54.703 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:09:54.704 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 88 16 113119 113103 10.0402 12.0859 0.00620025 0.0124458 2020-08-07T15:09:54.705 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:54.690+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:54.705 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:09:54.802 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:54.798+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:54.901 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:09:54.903 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:09:54.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.903 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:113975 4.9 4:97c71423:::benchmark_data_smithi074_44586_object14246:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e151)", 2020-08-07T15:09:54.903 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:54.795206+0000", 2020-08-07T15:09:54.903 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.10293904, 2020-08-07T15:09:54.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.10297246, 2020-08-07T15:09:54.904 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:54.904 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:54.904 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:54.904 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:54.905 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:54.905 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 113975 2020-08-07T15:09:54.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.905 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:54.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:54.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.795206+0000", 2020-08-07T15:09:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.795206+0000", 2020-08-07T15:09:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1748e-05 2020-08-07T15:09:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.795258+0000", 2020-08-07T15:09:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2580000000000001e-06 2020-08-07T15:09:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:54.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.795264+0000", 2020-08-07T15:09:54.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1583000000000001e-05 2020-08-07T15:09:54.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:54.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.795276+0000", 2020-08-07T15:09:54.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6760000000000004e-06 2020-08-07T15:09:54.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:54.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.795282+0000", 2020-08-07T15:09:54.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0077085770000000003 2020-08-07T15:09:54.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:54.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.802990+0000", 2020-08-07T15:09:54.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3863000000000006e-05 2020-08-07T15:09:54.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:54.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.803084+0000", 2020-08-07T15:09:54.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000183344 2020-08-07T15:09:54.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:09:54.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.803267+0000", 2020-08-07T15:09:54.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057331780000000002 2020-08-07T15:09:54.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:54.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:54.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:55.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.809001+0000", 2020-08-07T15:09:55.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002952835 2020-08-07T15:09:55.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:55.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.811953+0000", 2020-08-07T15:09:55.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3000000000000001e-07 2020-08-07T15:09:55.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:55.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.811954+0000", 2020-08-07T15:09:55.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0646000000000001e-05 2020-08-07T15:09:55.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:55.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.812025+0000", 2020-08-07T15:09:55.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015484783 2020-08-07T15:09:55.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:55.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.827509+0000", 2020-08-07T15:09:55.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032303021000000001 2020-08-07T15:09:55.004 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:55.004 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:55.004 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:55.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.005 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:114145 4.7 e151/81 4:e56c90ce:::benchmark_data_smithi074_44586_object14268:head v 151'7169, mlcod=151'7166)", 2020-08-07T15:09:55.005 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:54.896911+0000", 2020-08-07T15:09:55.006 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0012344179999999999, 2020-08-07T15:09:55.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013628410000000001, 2020-08-07T15:09:55.006 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:55.007 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:55.007 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:55.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:55.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.896911+0000", 2020-08-07T15:09:55.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:55.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:55.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.896911+0000", 2020-08-07T15:09:55.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2957999999999999e-05 2020-08-07T15:09:55.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:55.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.896944+0000", 2020-08-07T15:09:55.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1479999999999997e-06 2020-08-07T15:09:55.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:55.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.896949+0000", 2020-08-07T15:09:55.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8501000000000003e-05 2020-08-07T15:09:55.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:55.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.896988+0000", 2020-08-07T15:09:55.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3469999999999998e-06 2020-08-07T15:09:55.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:55.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.896993+0000", 2020-08-07T15:09:55.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3164000000000001e-05 2020-08-07T15:09:55.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:55.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.897046+0000", 2020-08-07T15:09:55.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2518e-05 2020-08-07T15:09:55.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:55.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.897069+0000", 2020-08-07T15:09:55.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015763600000000001 2020-08-07T15:09:55.042 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:55.042 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:55.042 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:55.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.043 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:114146 4.7 e151/81 4:e56c90ce:::benchmark_data_smithi074_44586_object14268:head v 151'7170, mlcod=151'7166)", 2020-08-07T15:09:55.044 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:54.897784+0000", 2020-08-07T15:09:55.044 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00036183599999999998, 2020-08-07T15:09:55.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00053027399999999996, 2020-08-07T15:09:55.045 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:55.045 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:09:55.046 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:55.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:55.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.897784+0000", 2020-08-07T15:09:55.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:55.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:55.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.897784+0000", 2020-08-07T15:09:55.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0507999999999998e-05 2020-08-07T15:09:55.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:55.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.897834+0000", 2020-08-07T15:09:55.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1879999999999999e-06 2020-08-07T15:09:55.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:55.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.897840+0000", 2020-08-07T15:09:55.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8785e-05 2020-08-07T15:09:55.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:55.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.897899+0000", 2020-08-07T15:09:55.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2020-08-07T15:09:55.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:55.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.897905+0000", 2020-08-07T15:09:55.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4842999999999998e-05 2020-08-07T15:09:55.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:55.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.897940+0000", 2020-08-07T15:09:55.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1475e-05 2020-08-07T15:09:55.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:55.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.897961+0000", 2020-08-07T15:09:55.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017757 2020-08-07T15:09:55.077 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:55.078 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:55.078 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:55.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.079 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:113976 4.9 4:97c71423:::benchmark_data_smithi074_44586_object14246:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e151)", 2020-08-07T15:09:55.079 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:54.797544+0000", 2020-08-07T15:09:55.079 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.100601752, 2020-08-07T15:09:55.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.100810304, 2020-08-07T15:09:55.080 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:55.080 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:55.080 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:55.080 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:55.081 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:55.081 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 113976 2020-08-07T15:09:55.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.081 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:55.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:55.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.797544+0000", 2020-08-07T15:09:55.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:55.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:55.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.797544+0000", 2020-08-07T15:09:55.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2340999999999999e-05 2020-08-07T15:09:55.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:55.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.797596+0000", 2020-08-07T15:09:55.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0689999999999998e-06 2020-08-07T15:09:55.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:55.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.797602+0000", 2020-08-07T15:09:55.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1528e-05 2020-08-07T15:09:55.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:55.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.797614+0000", 2020-08-07T15:09:55.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5289999999999999e-06 2020-08-07T15:09:55.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:55.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.797619+0000", 2020-08-07T15:09:55.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006129654 2020-08-07T15:09:55.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:55.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.803749+0000", 2020-08-07T15:09:55.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102414 2020-08-07T15:09:55.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:55.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.803851+0000", 2020-08-07T15:09:55.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011446399999999999 2020-08-07T15:09:55.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:09:55.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.803966+0000", 2020-08-07T15:09:55.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050453900000000003 2020-08-07T15:09:55.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:55.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.809011+0000", 2020-08-07T15:09:55.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031157609999999999 2020-08-07T15:09:55.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:55.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.812127+0000", 2020-08-07T15:09:55.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1200000000000002e-07 2020-08-07T15:09:55.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:55.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.812128+0000", 2020-08-07T15:09:55.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9418999999999994e-05 2020-08-07T15:09:55.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:55.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.812197+0000", 2020-08-07T15:09:55.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015352387 2020-08-07T15:09:55.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:55.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.827549+0000", 2020-08-07T15:09:55.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030005667999999999 2020-08-07T15:09:55.100 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:55.100 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:55.101 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:55.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.132 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:113888 4.8 4:1e6eb9e7:::benchmark_data_smithi074_44586_object14235:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e151)", 2020-08-07T15:09:55.132 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:54.748673+0000", 2020-08-07T15:09:55.132 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.149472511, 2020-08-07T15:09:55.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14974220799999999, 2020-08-07T15:09:55.133 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:55.133 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:55.134 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:55.134 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:55.134 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:55.134 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 113888 2020-08-07T15:09:55.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.135 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:55.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:55.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.748673+0000", 2020-08-07T15:09:55.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:55.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:55.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.748673+0000", 2020-08-07T15:09:55.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8824e-05 2020-08-07T15:09:55.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:55.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.748702+0000", 2020-08-07T15:09:55.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1429999999999998e-06 2020-08-07T15:09:55.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:55.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.748705+0000", 2020-08-07T15:09:55.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1009999999999997e-06 2020-08-07T15:09:55.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:55.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.748711+0000", 2020-08-07T15:09:55.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.971e-06 2020-08-07T15:09:55.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:55.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.748714+0000", 2020-08-07T15:09:55.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027846160000000002 2020-08-07T15:09:55.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:55.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.751499+0000", 2020-08-07T15:09:55.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106035 2020-08-07T15:09:55.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:55.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.751605+0000", 2020-08-07T15:09:55.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027429499999999998 2020-08-07T15:09:55.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6,7", 2020-08-07T15:09:55.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.751879+0000", 2020-08-07T15:09:55.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046543679999999999 2020-08-07T15:09:55.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:55.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.756533+0000", 2020-08-07T15:09:55.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6700000000000003e-07 2020-08-07T15:09:55.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:55.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.756534+0000", 2020-08-07T15:09:55.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6595000000000003e-05 2020-08-07T15:09:55.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:55.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.756590+0000", 2020-08-07T15:09:55.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040964133 2020-08-07T15:09:55.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:55.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.797555+0000", 2020-08-07T15:09:55.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048881648 2020-08-07T15:09:55.184 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:55.184 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:55.184 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:55.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.185 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:114015 4.1 4:834f2b89:::benchmark_data_smithi074_44586_object14251:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e151)", 2020-08-07T15:09:55.186 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:54.822353+0000", 2020-08-07T15:09:55.186 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.075792468000000002, 2020-08-07T15:09:55.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.076134907000000002, 2020-08-07T15:09:55.187 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:55.187 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:55.188 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:55.188 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:55.188 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:55.189 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 114015 2020-08-07T15:09:55.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.189 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:55.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:55.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.822353+0000", 2020-08-07T15:09:55.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:55.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:55.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.822353+0000", 2020-08-07T15:09:55.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8056999999999997e-05 2020-08-07T15:09:55.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:55.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.822401+0000", 2020-08-07T15:09:55.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0119999999999996e-06 2020-08-07T15:09:55.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:55.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.822406+0000", 2020-08-07T15:09:55.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.041e-05 2020-08-07T15:09:55.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:55.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.822416+0000", 2020-08-07T15:09:55.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2729999999999996e-06 2020-08-07T15:09:55.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:55.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.822421+0000", 2020-08-07T15:09:55.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038178019999999999 2020-08-07T15:09:55.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:55.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.826239+0000", 2020-08-07T15:09:55.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7700000000000004e-05 2020-08-07T15:09:55.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:55.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.826326+0000", 2020-08-07T15:09:55.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014314800000000001 2020-08-07T15:09:55.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:09:55.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.826469+0000", 2020-08-07T15:09:55.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026675620000000001 2020-08-07T15:09:55.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:55.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.829137+0000", 2020-08-07T15:09:55.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8400000000000002e-07 2020-08-07T15:09:55.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:55.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.829138+0000", 2020-08-07T15:09:55.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5810999999999997e-05 2020-08-07T15:09:55.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:55.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.829204+0000", 2020-08-07T15:09:55.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056303280000000004 2020-08-07T15:09:55.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:55.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.834834+0000", 2020-08-07T15:09:55.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012481087 2020-08-07T15:09:55.227 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:55.228 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:55.229 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:55.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.229 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:114016 4.1 4:834f2b89:::benchmark_data_smithi074_44586_object14251:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e151)", 2020-08-07T15:09:55.229 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:54.822452+0000", 2020-08-07T15:09:55.230 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.075693861000000001, 2020-08-07T15:09:55.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.076126863000000003, 2020-08-07T15:09:55.230 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:55.230 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:55.230 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:55.231 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:55.231 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:55.231 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 114016 2020-08-07T15:09:55.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.231 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:55.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:55.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.822452+0000", 2020-08-07T15:09:55.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:55.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:55.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.822452+0000", 2020-08-07T15:09:55.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8968000000000003e-05 2020-08-07T15:09:55.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:55.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.822501+0000", 2020-08-07T15:09:55.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0239999999999997e-06 2020-08-07T15:09:55.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:55.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.822506+0000", 2020-08-07T15:09:55.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0329999999999999e-05 2020-08-07T15:09:55.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:55.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.822516+0000", 2020-08-07T15:09:55.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.126e-06 2020-08-07T15:09:55.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:55.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.822520+0000", 2020-08-07T15:09:55.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044811319999999997 2020-08-07T15:09:55.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:55.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.827001+0000", 2020-08-07T15:09:55.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4256e-05 2020-08-07T15:09:55.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:55.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.827095+0000", 2020-08-07T15:09:55.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012394899999999999 2020-08-07T15:09:55.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:09:55.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.827219+0000", 2020-08-07T15:09:55.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046696760000000002 2020-08-07T15:09:55.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:55.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.831889+0000", 2020-08-07T15:09:55.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7100000000000011e-07 2020-08-07T15:09:55.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:55.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.831890+0000", 2020-08-07T15:09:55.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2506e-05 2020-08-07T15:09:55.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:55.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.831963+0000", 2020-08-07T15:09:55.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029075189999999999 2020-08-07T15:09:55.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:55.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:54.834870+0000", 2020-08-07T15:09:55.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012418457000000001 2020-08-07T15:09:55.341 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:55.341 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:55.342 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:55.342 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:55.342 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:09:55.342 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 7 2020-08-07T15:09:55.351 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:09:55.352 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:09:54.900+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:09:55.352 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:09:55.353 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:09:55.358 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:09:55.359 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2020-08-07T15:09:54.652969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":897515520,"num_objects":13697,"num_object_clones":0,"num_object_copies":41091,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":885,"num_objects_unfound":0,"num_objects_dirty":13697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109560,"num_write_kb":876480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2275,"num_bytes_recovered":148971520,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":135,"acting":135,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":11045636,"kb_used_data":2656596,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":743929084,"statfs":{"total":773094113280,"available":761783382016,"internally_reserved":8589934592,"allocated":2720354304,"data_stored":2716687488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,6,5,4,3,0,2,1,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":60162048,"num_objects":918,"num_object_clones":0,"num_object_copies":2754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":59,"num_objects_unfound":0,"num_objects_dirty":918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7344,"num_write_kb":58752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.061014"},"pg_stats":[{"pgid":"1.14","version":"16'2","reported_seq":"79","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.601707+0000","last_change":"2020-08-07T15:08:42.533999+0000","last_active":"2020-08-07T15:09:50.601707+0000","last_peered":"2020-08-07T15:09:50.601707+0000","last_clean":"2020-08-07T15:09:50.601707+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:09:50.601707+0000","last_undegraded":"2020-08-07T15:09:50.601707+0000","last_fullsized":"2020-08-07T15:09:50.601707+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:42.533921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:42.533921+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"149","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.605402+0000","last_change":"2020-08-07T15:09:30.205031+0000","last_active":"2020-08-07T15:09:50.605402+0000","last_peered":"2020-08-07T15:09:50.605402+0000","last_clean":"2020-08-07T15:09:50.605402+0000","last_became_active":"2020-08-07T15:09:30.203592+0000","last_became_peered":"2020-08-07T15:09:30.203592+0000","last_unstale":"2020-08-07T15:09:50.605402+0000","last_undegraded":"2020-08-07T15:09:50.605402+0000","last_fullsized":"2020-08-07T15:09:50.605402+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.141010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.141010+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"79","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.594732+0000","last_change":"2020-08-07T15:08:55.078015+0000","last_active":"2020-08-07T15:09:50.594732+0000","last_peered":"2020-08-07T15:09:50.594732+0000","last_clean":"2020-08-07T15:09:50.594732+0000","last_became_active":"2020-08-07T15:08:41.685106+0000","last_became_peered":"2020-08-07T15:08:41.685106+0000","last_unstale":"2020-08-07T15:09:50.594732+0000","last_undegraded":"2020-08-07T15:09:50.594732+0000","last_fullsized":"2020-08-07T15:09:50.594732+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:55.077924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:55.077924+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"192","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.594713+0000","last_change":"2020-08-07T15:09:30.210669+0000","last_active":"2020-08-07T15:09:50.594713+0000","last_peered":"2020-08-07T15:09:50.594713+0000","last_clean":"2020-08-07T15:09:50.594713+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:09:50.594713+0000","last_undegraded":"2020-08-07T15:09:50.594713+0000","last_fullsized":"2020-08-07T15:09:50.594713+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.544324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.544324+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"186","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.596897+0000","last_change":"2020-08-07T15:08:51.599032+0000","last_active":"2020-08-07T15:09:50.596897+0000","last_peered":"2020-08-07T15:09:50.596897+0000","last_clean":"2020-08-07T15:09:50.596897+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:09:50.596897+0000","last_undegraded":"2020-08-07T15:09:50.596897+0000","last_fullsized":"2020-08-07T15:09:50.596897+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:51.598947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.598947+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"147","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.598537+0000","last_change":"2020-08-07T15:09:41.284321+0000","last_active":"2020-08-07T15:09:50.598537+0000","last_peered":"2020-08-07T15:09:50.598537+0000","last_clean":"2020-08-07T15:09:50.598537+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:09:50.598537+0000","last_undegraded":"2020-08-07T15:09:50.598537+0000","last_fullsized":"2020-08-07T15:09:50.598537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:41.284237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:41.284237+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"63","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.615733+0000","last_change":"2020-08-07T15:09:30.283753+0000","last_active":"2020-08-07T15:09:50.615733+0000","last_peered":"2020-08-07T15:09:50.615733+0000","last_clean":"2020-08-07T15:09:50.615733+0000","last_became_active":"2020-08-07T15:09:30.283176+0000","last_became_peered":"2020-08-07T15:09:30.283176+0000","last_unstale":"2020-08-07T15:09:50.615733+0000","last_undegraded":"2020-08-07T15:09:50.615733+0000","last_fullsized":"2020-08-07T15:09:50.615733+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:25.606757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:25.606757+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"150'6600","reported_seq":"6679","reported_epoch":"150","state":"active+clean+scrubbing","last_fresh":"2020-08-07T15:09:51.690001+0000","last_change":"2020-08-07T15:09:51.601187+0000","last_active":"2020-08-07T15:09:51.690001+0000","last_peered":"2020-08-07T15:09:51.690001+0000","last_clean":"2020-08-07T15:09:51.690001+0000","last_became_active":"2020-08-07T15:08:46.708112+0000","last_became_peered":"2020-08-07T15:08:46.708112+0000","last_unstale":"2020-08-07T15:09:51.690001+0000","last_undegraded":"2020-08-07T15:09:51.690001+0000","last_fullsized":"2020-08-07T15:09:51.690001+0000","mapping_epoch":98,"log_start":"150'6597","ondisk_log_start":"150'6597","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":54067200,"num_objects":825,"num_object_clones":0,"num_object_copies":2475,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6600,"num_write_kb":52800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"147","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.605637+0000","last_change":"2020-08-07T15:08:40.057705+0000","last_active":"2020-08-07T15:09:50.605637+0000","last_peered":"2020-08-07T15:09:50.605637+0000","last_clean":"2020-08-07T15:09:50.605637+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:09:50.605637+0000","last_undegraded":"2020-08-07T15:09:50.605637+0000","last_fullsized":"2020-08-07T15:09:50.605637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:40.057622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:40.057622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"104","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.601329+0000","last_change":"2020-08-07T15:08:53.756554+0000","last_active":"2020-08-07T15:09:50.601329+0000","last_peered":"2020-08-07T15:09:50.601329+0000","last_clean":"2020-08-07T15:09:50.601329+0000","last_became_active":"2020-08-07T15:08:46.710404+0000","last_became_peered":"2020-08-07T15:08:46.710404+0000","last_unstale":"2020-08-07T15:09:50.601329+0000","last_undegraded":"2020-08-07T15:09:50.601329+0000","last_fullsized":"2020-08-07T15:09:50.601329+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:53.756343+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:53.756343+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"150'7232","reported_seq":"9077","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.642016+0000","last_change":"2020-08-07T15:09:35.835839+0000","last_active":"2020-08-07T15:09:51.642016+0000","last_peered":"2020-08-07T15:09:51.642016+0000","last_clean":"2020-08-07T15:09:51.642016+0000","last_became_active":"2020-08-07T15:09:27.180158+0000","last_became_peered":"2020-08-07T15:09:27.180158+0000","last_unstale":"2020-08-07T15:09:51.642016+0000","last_undegraded":"2020-08-07T15:09:51.642016+0000","last_fullsized":"2020-08-07T15:09:51.642016+0000","mapping_epoch":131,"log_start":"150'7226","ondisk_log_start":"150'7226","created":81,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'5536","last_scrub_stamp":"2020-08-07T15:09:35.835763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:35.835763+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":59244544,"num_objects":904,"num_object_clones":0,"num_object_copies":2712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7232,"num_write_kb":57856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":875,"num_bytes_recovered":57344000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"144","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.600155+0000","last_change":"2020-08-07T15:08:27.906894+0000","last_active":"2020-08-07T15:09:50.600155+0000","last_peered":"2020-08-07T15:09:50.600155+0000","last_clean":"2020-08-07T15:09:50.600155+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:09:50.600155+0000","last_undegraded":"2020-08-07T15:09:50.600155+0000","last_fullsized":"2020-08-07T15:09:50.600155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:27.906822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:27.906822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"183","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.598507+0000","last_change":"2020-08-07T15:09:30.202363+0000","last_active":"2020-08-07T15:09:50.598507+0000","last_peered":"2020-08-07T15:09:50.598507+0000","last_clean":"2020-08-07T15:09:50.598507+0000","last_became_active":"2020-08-07T15:09:30.201829+0000","last_became_peered":"2020-08-07T15:09:30.201829+0000","last_unstale":"2020-08-07T15:09:50.598507+0000","last_undegraded":"2020-08-07T15:09:50.598507+0000","last_fullsized":"2020-08-07T15:09:50.598507+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.899280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.899280+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"150'6704","reported_seq":"6780","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.720791+0000","last_change":"2020-08-07T15:08:24.549715+0000","last_active":"2020-08-07T15:09:51.720791+0000","last_peered":"2020-08-07T15:09:51.720791+0000","last_clean":"2020-08-07T15:09:51.720791+0000","last_became_active":"2020-08-07T15:08:24.549245+0000","last_became_peered":"2020-08-07T15:08:24.549245+0000","last_unstale":"2020-08-07T15:09:51.720791+0000","last_undegraded":"2020-08-07T15:09:51.720791+0000","last_fullsized":"2020-08-07T15:09:51.720791+0000","mapping_epoch":81,"log_start":"149'6696","ondisk_log_start":"149'6696","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":54919168,"num_objects":838,"num_object_clones":0,"num_object_copies":2514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6704,"num_write_kb":53632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"147","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:51.036281+0000","last_change":"2020-08-07T15:09:51.036281+0000","last_active":"2020-08-07T15:09:51.036281+0000","last_peered":"2020-08-07T15:09:51.036281+0000","last_clean":"2020-08-07T15:09:51.036281+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:09:51.036281+0000","last_undegraded":"2020-08-07T15:09:51.036281+0000","last_fullsized":"2020-08-07T15:09:51.036281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:51.036224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:51.036224+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"149","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.598302+0000","last_change":"2020-08-07T15:08:50.686578+0000","last_active":"2020-08-07T15:09:50.598302+0000","last_peered":"2020-08-07T15:09:50.598302+0000","last_clean":"2020-08-07T15:09:50.598302+0000","last_became_active":"2020-08-07T15:08:36.633406+0000","last_became_peered":"2020-08-07T15:08:36.633406+0000","last_unstale":"2020-08-07T15:09:50.598302+0000","last_undegraded":"2020-08-07T15:09:50.598302+0000","last_fullsized":"2020-08-07T15:09:50.598302+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:50.686496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:50.686496+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"150'6416","reported_seq":"6493","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.680246+0000","last_change":"2020-08-07T15:08:24.557398+0000","last_active":"2020-08-07T15:09:51.680246+0000","last_peered":"2020-08-07T15:09:51.680246+0000","last_clean":"2020-08-07T15:09:51.680246+0000","last_became_active":"2020-08-07T15:08:24.557022+0000","last_became_peered":"2020-08-07T15:08:24.557022+0000","last_unstale":"2020-08-07T15:09:51.680246+0000","last_undegraded":"2020-08-07T15:09:51.680246+0000","last_fullsized":"2020-08-07T15:09:51.680246+0000","mapping_epoch":81,"log_start":"149'6408","ondisk_log_start":"149'6408","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":52559872,"num_objects":802,"num_object_clones":0,"num_object_copies":2406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6416,"num_write_kb":51328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"176","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.600698+0000","last_change":"2020-08-07T15:08:43.036067+0000","last_active":"2020-08-07T15:09:50.600698+0000","last_peered":"2020-08-07T15:09:50.600698+0000","last_clean":"2020-08-07T15:09:50.600698+0000","last_became_active":"2020-08-07T15:08:36.627426+0000","last_became_peered":"2020-08-07T15:08:36.627426+0000","last_unstale":"2020-08-07T15:09:50.600698+0000","last_undegraded":"2020-08-07T15:09:50.600698+0000","last_fullsized":"2020-08-07T15:09:50.600698+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.035987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.035987+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"144","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.594986+0000","last_change":"2020-08-07T15:08:26.317430+0000","last_active":"2020-08-07T15:09:50.594986+0000","last_peered":"2020-08-07T15:09:50.594986+0000","last_clean":"2020-08-07T15:09:50.594986+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:09:50.594986+0000","last_undegraded":"2020-08-07T15:09:50.594986+0000","last_fullsized":"2020-08-07T15:09:50.594986+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:26.317351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:26.317351+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"150'6856","reported_seq":"6933","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:52.642332+0000","last_change":"2020-08-07T15:08:24.552570+0000","last_active":"2020-08-07T15:09:52.642332+0000","last_peered":"2020-08-07T15:09:52.642332+0000","last_clean":"2020-08-07T15:09:52.642332+0000","last_became_active":"2020-08-07T15:08:24.552105+0000","last_became_peered":"2020-08-07T15:08:24.552105+0000","last_unstale":"2020-08-07T15:09:52.642332+0000","last_undegraded":"2020-08-07T15:09:52.642332+0000","last_fullsized":"2020-08-07T15:09:52.642332+0000","mapping_epoch":81,"log_start":"150'6847","ondisk_log_start":"150'6847","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56164352,"num_objects":857,"num_object_clones":0,"num_object_copies":2571,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6856,"num_write_kb":54848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"165","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.601364+0000","last_change":"2020-08-07T15:09:27.181676+0000","last_active":"2020-08-07T15:09:50.601364+0000","last_peered":"2020-08-07T15:09:50.601364+0000","last_clean":"2020-08-07T15:09:50.601364+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:09:50.601364+0000","last_undegraded":"2020-08-07T15:09:50.601364+0000","last_fullsized":"2020-08-07T15:09:50.601364+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:05.606603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:05.606603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"192","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.599787+0000","last_change":"2020-08-07T15:08:42.069637+0000","last_active":"2020-08-07T15:09:50.599787+0000","last_peered":"2020-08-07T15:09:50.599787+0000","last_clean":"2020-08-07T15:09:50.599787+0000","last_became_active":"2020-08-07T15:08:36.628365+0000","last_became_peered":"2020-08-07T15:08:36.628365+0000","last_unstale":"2020-08-07T15:09:50.599787+0000","last_undegraded":"2020-08-07T15:09:50.599787+0000","last_fullsized":"2020-08-07T15:09:50.599787+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:42.069585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:42.069585+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"149'6904","reported_seq":"6984","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:51.261343+0000","last_change":"2020-08-07T15:09:37.517415+0000","last_active":"2020-08-07T15:09:51.261343+0000","last_peered":"2020-08-07T15:09:51.261343+0000","last_clean":"2020-08-07T15:09:51.261343+0000","last_became_active":"2020-08-07T15:08:24.550222+0000","last_became_peered":"2020-08-07T15:08:24.550222+0000","last_unstale":"2020-08-07T15:09:51.261343+0000","last_undegraded":"2020-08-07T15:09:51.261343+0000","last_fullsized":"2020-08-07T15:09:51.261343+0000","mapping_epoch":81,"log_start":"149'6896","ondisk_log_start":"149'6896","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"141'5600","last_scrub_stamp":"2020-08-07T15:09:37.517334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:37.517334+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56557568,"num_objects":863,"num_object_clones":0,"num_object_copies":2589,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6904,"num_write_kb":55232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"144","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.594884+0000","last_change":"2020-08-07T15:08:30.339852+0000","last_active":"2020-08-07T15:09:50.594884+0000","last_peered":"2020-08-07T15:09:50.594884+0000","last_clean":"2020-08-07T15:09:50.594884+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:09:50.594884+0000","last_undegraded":"2020-08-07T15:09:50.594884+0000","last_fullsized":"2020-08-07T15:09:50.594884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:08:30.339785+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"142","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.595208+0000","last_change":"2020-08-07T15:08:38.386792+0000","last_active":"2020-08-07T15:09:50.595208+0000","last_peered":"2020-08-07T15:09:50.595208+0000","last_clean":"2020-08-07T15:09:50.595208+0000","last_became_active":"2020-08-07T15:08:36.618164+0000","last_became_peered":"2020-08-07T15:08:36.618164+0000","last_unstale":"2020-08-07T15:09:50.595208+0000","last_undegraded":"2020-08-07T15:09:50.595208+0000","last_fullsized":"2020-08-07T15:09:50.595208+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:38.386739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:38.386739+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"150'6704","reported_seq":"7879","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.669793+0000","last_change":"2020-08-07T15:09:36.911393+0000","last_active":"2020-08-07T15:09:51.669793+0000","last_peered":"2020-08-07T15:09:51.669793+0000","last_clean":"2020-08-07T15:09:51.669793+0000","last_became_active":"2020-08-07T15:09:27.183115+0000","last_became_peered":"2020-08-07T15:09:27.183115+0000","last_unstale":"2020-08-07T15:09:51.669793+0000","last_undegraded":"2020-08-07T15:09:51.669793+0000","last_fullsized":"2020-08-07T15:09:51.669793+0000","mapping_epoch":131,"log_start":"150'6696","ondisk_log_start":"150'6696","created":81,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'5280","last_scrub_stamp":"2020-08-07T15:09:36.911321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:36.911321+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":54919168,"num_objects":838,"num_object_clones":0,"num_object_copies":2514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6704,"num_write_kb":53632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":536,"num_bytes_recovered":35127296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"144","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.600554+0000","last_change":"2020-08-07T15:09:37.017383+0000","last_active":"2020-08-07T15:09:50.600554+0000","last_peered":"2020-08-07T15:09:50.600554+0000","last_clean":"2020-08-07T15:09:50.600554+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:09:50.600554+0000","last_undegraded":"2020-08-07T15:09:50.600554+0000","last_fullsized":"2020-08-07T15:09:50.600554+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:37.017307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:37.017307+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"217","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.598646+0000","last_change":"2020-08-07T15:09:30.208873+0000","last_active":"2020-08-07T15:09:50.598646+0000","last_peered":"2020-08-07T15:09:50.598646+0000","last_clean":"2020-08-07T15:09:50.598646+0000","last_became_active":"2020-08-07T15:09:30.208345+0000","last_became_peered":"2020-08-07T15:09:30.208345+0000","last_unstale":"2020-08-07T15:09:50.598646+0000","last_undegraded":"2020-08-07T15:09:50.598646+0000","last_fullsized":"2020-08-07T15:09:50.598646+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.381325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.381325+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"150'6760","reported_seq":"6837","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.712731+0000","last_change":"2020-08-07T15:08:24.552962+0000","last_active":"2020-08-07T15:09:51.712731+0000","last_peered":"2020-08-07T15:09:51.712731+0000","last_clean":"2020-08-07T15:09:51.712731+0000","last_became_active":"2020-08-07T15:08:24.552070+0000","last_became_peered":"2020-08-07T15:08:24.552070+0000","last_unstale":"2020-08-07T15:09:51.712731+0000","last_undegraded":"2020-08-07T15:09:51.712731+0000","last_fullsized":"2020-08-07T15:09:51.712731+0000","mapping_epoch":81,"log_start":"149'6752","ondisk_log_start":"149'6752","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":55377920,"num_objects":845,"num_object_clones":0,"num_object_copies":2535,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6760,"num_write_kb":54080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"119","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.594865+0000","last_change":"2020-08-07T15:08:39.409159+0000","last_active":"2020-08-07T15:09:50.594865+0000","last_peered":"2020-08-07T15:09:50.594865+0000","last_clean":"2020-08-07T15:09:50.594865+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:09:50.594865+0000","last_undegraded":"2020-08-07T15:09:50.594865+0000","last_fullsized":"2020-08-07T15:09:50.594865+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:39.409063+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_clean_scrub_stamp":"2020-08-07T15:08:39.409063+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"150'6752","reported_seq":"6832","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.655457+0000","last_change":"2020-08-07T15:09:40.697977+0000","last_active":"2020-08-07T15:09:51.655457+0000","last_peered":"2020-08-07T15:09:51.655457+0000","last_clean":"2020-08-07T15:09:51.655457+0000","last_became_active":"2020-08-07T15:08:24.553668+0000","last_became_peered":"2020-08-07T15:08:24.553668+0000","last_unstale":"2020-08-07T15:09:51.655457+0000","last_undegraded":"2020-08-07T15:09:51.655457+0000","last_fullsized":"2020-08-07T15:09:51.655457+0000","mapping_epoch":81,"log_start":"149'6744","ondisk_log_start":"149'6744","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"142'5792","last_scrub_stamp":"2020-08-07T15:09:40.697906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:40.697906+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":55312384,"num_objects":844,"num_object_clones":0,"num_object_copies":2532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6752,"num_write_kb":54016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"119","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.596947+0000","last_change":"2020-08-07T15:09:09.691838+0000","last_active":"2020-08-07T15:09:50.596947+0000","last_peered":"2020-08-07T15:09:50.596947+0000","last_clean":"2020-08-07T15:09:50.596947+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:09:50.596947+0000","last_undegraded":"2020-08-07T15:09:50.596947+0000","last_fullsized":"2020-08-07T15:09:50.596947+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.691753+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"150'6728","reported_seq":"6808","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.639617+0000","last_change":"2020-08-07T15:09:42.152452+0000","last_active":"2020-08-07T15:09:51.639617+0000","last_peered":"2020-08-07T15:09:51.639617+0000","last_clean":"2020-08-07T15:09:51.639617+0000","last_became_active":"2020-08-07T15:08:46.708674+0000","last_became_peered":"2020-08-07T15:08:46.708674+0000","last_unstale":"2020-08-07T15:09:51.639617+0000","last_undegraded":"2020-08-07T15:09:51.639617+0000","last_fullsized":"2020-08-07T15:09:51.639617+0000","mapping_epoch":98,"log_start":"149'6720","ondisk_log_start":"149'6720","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"142'5792","last_scrub_stamp":"2020-08-07T15:09:42.152379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:42.152379+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":55115776,"num_objects":841,"num_object_clones":0,"num_object_copies":2523,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6728,"num_write_kb":53824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"183","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.599770+0000","last_change":"2020-08-07T15:09:25.984108+0000","last_active":"2020-08-07T15:09:50.599770+0000","last_peered":"2020-08-07T15:09:50.599770+0000","last_clean":"2020-08-07T15:09:50.599770+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:09:50.599770+0000","last_undegraded":"2020-08-07T15:09:50.599770+0000","last_fullsized":"2020-08-07T15:09:50.599770+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:25.984018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:25.984018+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"150'6872","reported_seq":"6973","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.676987+0000","last_change":"2020-08-07T15:09:38.739730+0000","last_active":"2020-08-07T15:09:51.676987+0000","last_peered":"2020-08-07T15:09:51.676987+0000","last_clean":"2020-08-07T15:09:51.676987+0000","last_became_active":"2020-08-07T15:08:52.166336+0000","last_became_peered":"2020-08-07T15:08:52.166336+0000","last_unstale":"2020-08-07T15:09:51.676987+0000","last_undegraded":"2020-08-07T15:09:51.676987+0000","last_fullsized":"2020-08-07T15:09:51.676987+0000","mapping_epoch":104,"log_start":"150'6868","ondisk_log_start":"150'6868","created":81,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"141'5712","last_scrub_stamp":"2020-08-07T15:09:38.739630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:38.739630+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56295424,"num_objects":859,"num_object_clones":0,"num_object_copies":2577,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6872,"num_write_kb":54976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":40960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"138","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.599966+0000","last_change":"2020-08-07T15:08:51.513398+0000","last_active":"2020-08-07T15:09:50.599966+0000","last_peered":"2020-08-07T15:09:50.599966+0000","last_clean":"2020-08-07T15:09:50.599966+0000","last_became_active":"2020-08-07T15:08:41.680448+0000","last_became_peered":"2020-08-07T15:08:41.680448+0000","last_unstale":"2020-08-07T15:09:50.599966+0000","last_undegraded":"2020-08-07T15:09:50.599966+0000","last_fullsized":"2020-08-07T15:09:50.599966+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:51.513319+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.513319+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"150'6712","reported_seq":"6792","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.709182+0000","last_change":"2020-08-07T15:09:50.869480+0000","last_active":"2020-08-07T15:09:51.709182+0000","last_peered":"2020-08-07T15:09:51.709182+0000","last_clean":"2020-08-07T15:09:51.709182+0000","last_became_active":"2020-08-07T15:08:24.549868+0000","last_became_peered":"2020-08-07T15:08:24.549868+0000","last_unstale":"2020-08-07T15:09:51.709182+0000","last_undegraded":"2020-08-07T15:09:51.709182+0000","last_fullsized":"2020-08-07T15:09:51.709182+0000","mapping_epoch":81,"log_start":"149'6704","ondisk_log_start":"149'6704","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'6599","last_scrub_stamp":"2020-08-07T15:09:50.869400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:50.869400+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":54984704,"num_objects":839,"num_object_clones":0,"num_object_copies":2517,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6712,"num_write_kb":53696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"109","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.615815+0000","last_change":"2020-08-07T15:09:36.687393+0000","last_active":"2020-08-07T15:09:50.615815+0000","last_peered":"2020-08-07T15:09:50.615815+0000","last_clean":"2020-08-07T15:09:50.615815+0000","last_became_active":"2020-08-07T15:09:30.204655+0000","last_became_peered":"2020-08-07T15:09:30.204655+0000","last_unstale":"2020-08-07T15:09:50.615815+0000","last_undegraded":"2020-08-07T15:09:50.615815+0000","last_fullsized":"2020-08-07T15:09:50.615815+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:36.687314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:36.687314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"150'7080","reported_seq":"8938","reported_epoch":"150","state":"active+remapped+backfill_toofull","last_fresh":"2020-08-07T15:09:51.703178+0000","last_change":"2020-08-07T15:09:28.235048+0000","last_active":"2020-08-07T15:09:51.703178+0000","last_peered":"2020-08-07T15:09:51.703178+0000","last_clean":"2020-08-07T15:09:25.573158+0000","last_became_active":"2020-08-07T15:09:28.212799+0000","last_became_peered":"2020-08-07T15:09:28.212799+0000","last_unstale":"2020-08-07T15:09:51.703178+0000","last_undegraded":"2020-08-07T15:09:51.703178+0000","last_fullsized":"2020-08-07T15:09:51.703178+0000","mapping_epoch":132,"log_start":"150'7074","ondisk_log_start":"150'7074","created":81,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57999360,"num_objects":885,"num_object_clones":0,"num_object_copies":2655,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":885,"num_objects_unfound":0,"num_objects_dirty":885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7080,"num_write_kb":56640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":862,"num_bytes_recovered":56459264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,6],"avail_no_missing":["5","1","6"],"object_location_counts":[{"shards":"1,5,6","objects":885}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"182","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.598424+0000","last_change":"2020-08-07T15:09:08.399507+0000","last_active":"2020-08-07T15:09:50.598424+0000","last_peered":"2020-08-07T15:09:50.598424+0000","last_clean":"2020-08-07T15:09:50.598424+0000","last_became_active":"2020-08-07T15:08:41.689435+0000","last_became_peered":"2020-08-07T15:08:41.689435+0000","last_unstale":"2020-08-07T15:09:50.598424+0000","last_undegraded":"2020-08-07T15:09:50.598424+0000","last_fullsized":"2020-08-07T15:09:50.598424+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:08.399439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:08.399439+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"150'7368","reported_seq":"7447","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:52.708850+0000","last_change":"2020-08-07T15:09:52.486813+0000","last_active":"2020-08-07T15:09:52.708850+0000","last_peered":"2020-08-07T15:09:52.708850+0000","last_clean":"2020-08-07T15:09:52.708850+0000","last_became_active":"2020-08-07T15:08:24.554021+0000","last_became_peered":"2020-08-07T15:08:24.554021+0000","last_unstale":"2020-08-07T15:09:52.708850+0000","last_undegraded":"2020-08-07T15:09:52.708850+0000","last_fullsized":"2020-08-07T15:09:52.708850+0000","mapping_epoch":81,"log_start":"150'7363","ondisk_log_start":"150'7363","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'7328","last_scrub_stamp":"2020-08-07T15:09:52.486757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:52.486757+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60358656,"num_objects":921,"num_object_clones":0,"num_object_copies":2763,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7368,"num_write_kb":58944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"69","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.600279+0000","last_change":"2020-08-07T15:09:09.868283+0000","last_active":"2020-08-07T15:09:50.600279+0000","last_peered":"2020-08-07T15:09:50.600279+0000","last_clean":"2020-08-07T15:09:50.600279+0000","last_became_active":"2020-08-07T15:08:46.710059+0000","last_became_peered":"2020-08-07T15:08:46.710059+0000","last_unstale":"2020-08-07T15:09:50.600279+0000","last_undegraded":"2020-08-07T15:09:50.600279+0000","last_fullsized":"2020-08-07T15:09:50.600279+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.868198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.868198+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"150'7032","reported_seq":"7111","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.726534+0000","last_change":"2020-08-07T15:09:26.248248+0000","last_active":"2020-08-07T15:09:51.726534+0000","last_peered":"2020-08-07T15:09:51.726534+0000","last_clean":"2020-08-07T15:09:51.726534+0000","last_became_active":"2020-08-07T15:08:24.552127+0000","last_became_peered":"2020-08-07T15:08:24.552127+0000","last_unstale":"2020-08-07T15:09:51.726534+0000","last_undegraded":"2020-08-07T15:09:51.726534+0000","last_fullsized":"2020-08-07T15:09:51.726534+0000","mapping_epoch":81,"log_start":"150'7027","ondisk_log_start":"150'7027","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"130'4592","last_scrub_stamp":"2020-08-07T15:09:26.248174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:26.248174+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57606144,"num_objects":879,"num_object_clones":0,"num_object_copies":2637,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7032,"num_write_kb":56256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"150'6840","reported_seq":"6920","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:52.473304+0000","last_change":"2020-08-07T15:08:46.710643+0000","last_active":"2020-08-07T15:09:52.473304+0000","last_peered":"2020-08-07T15:09:52.473304+0000","last_clean":"2020-08-07T15:09:52.473304+0000","last_became_active":"2020-08-07T15:08:46.710100+0000","last_became_peered":"2020-08-07T15:08:46.710100+0000","last_unstale":"2020-08-07T15:09:52.473304+0000","last_undegraded":"2020-08-07T15:09:52.473304+0000","last_fullsized":"2020-08-07T15:09:52.473304+0000","mapping_epoch":98,"log_start":"150'6834","ondisk_log_start":"150'6834","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56033280,"num_objects":855,"num_object_clones":0,"num_object_copies":2565,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6840,"num_write_kb":54720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"91","reported_epoch":"149","state":"active+clean","last_fresh":"2020-08-07T15:09:50.594836+0000","last_change":"2020-08-07T15:09:08.653773+0000","last_active":"2020-08-07T15:09:50.594836+0000","last_peered":"2020-08-07T15:09:50.594836+0000","last_clean":"2020-08-07T15:09:50.594836+0000","last_became_active":"2020-08-07T15:08:41.677944+0000","last_became_peered":"2020-08-07T15:08:41.677944+0000","last_unstale":"2020-08-07T15:09:50.594836+0000","last_undegraded":"2020-08-07T15:09:50.594836+0000","last_fullsized":"2020-08-07T15:09:50.594836+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:08.653722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:08.653722+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":897515520,"num_objects":13695,"num_object_clones":0,"num_object_copies":41085,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":885,"num_objects_unfound":0,"num_objects_dirty":13695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109560,"num_write_kb":876480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2275,"num_bytes_recovered":148971520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2726510592,"data_stored":2726510592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":63,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542189,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1436264,"kb_used_data":387680,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92935576,"statfs":{"total":96636764160,"available":95166029824,"internally_reserved":1073741824,"allocated":396984320,"data_stored":396521283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.95099999999999996,"15min":0.91200000000000003},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.1739999999999999,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":1.04},{"interface":"front","average":{"1min":1.095,"5min":1.0149999999999999,"15min":0.97999999999999998},"min":{"1min":0.54300000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.0049999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.357,"15min":1.3149999999999999},"min":{"1min":0.68700000000000006,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":5.6239999999999997,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.105,"5min":1.0720000000000001,"15min":1.0009999999999999},"min":{"1min":0.64100000000000001,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.206,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":4.0060000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.272,"15min":1.175},"min":{"1min":0.71999999999999997,"5min":0.628,"15min":0.628},"max":{"1min":5.6619999999999999,"5min":5.6619999999999999,"15min":5.6619999999999999},"last":1.127},{"interface":"front","average":{"1min":1.446,"5min":1.26,"15min":1.165},"min":{"1min":0.66800000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":5.1319999999999997,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":1.3109999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":75.045000000000002,"5min":16.263999999999999,"15min":6.5359999999999996},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":917.38699999999994,"5min":917.38699999999994,"15min":917.38699999999994},"last":0.90400000000000003},{"interface":"front","average":{"1min":19.670999999999999,"5min":4.9939999999999998,"15min":2.5600000000000001},"min":{"1min":0.80000000000000004,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":333.59199999999998,"5min":333.59199999999998,"15min":333.59199999999998},"last":0.96199999999999997}]},{"osd":4,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.411,"15min":1.321},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":4.5250000000000004,"5min":4.835,"15min":4.835},"last":1.2929999999999999},{"interface":"front","average":{"1min":52.259999999999998,"5min":11.52,"15min":4.7140000000000004},"min":{"1min":0.77000000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":866.24099999999999,"5min":866.24099999999999,"15min":866.24099999999999},"last":0.93500000000000005}]},{"osd":5,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.2869999999999999,"5min":23.440000000000001,"15min":32.817999999999998},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":6.6509999999999998,"5min":831.88099999999997,"15min":831.88099999999997},"last":1.1000000000000001},{"interface":"front","average":{"1min":2.2519999999999998,"5min":1.8380000000000001,"15min":1.609},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":6.6790000000000003,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":1.2529999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":13.818,"5min":4.0019999999999998,"15min":2.3460000000000001},"min":{"1min":0.82999999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":224.14500000000001,"5min":224.14500000000001,"15min":224.14500000000001},"last":2.0590000000000002},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.6839999999999999,"15min":1.5660000000000001},"min":{"1min":0.90600000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":7.4359999999999999,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":1.6419999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1324492,"kb_used_data":275548,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93047348,"statfs":{"total":96636764160,"available":95280484352,"internally_reserved":1073741824,"allocated":282161152,"data_stored":281705611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.179,"15min":1.2809999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8149999999999999,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":1.3160000000000001},{"interface":"front","average":{"1min":15.239000000000001,"5min":3.9529999999999998,"15min":2.0840000000000001},"min":{"1min":0.78100000000000003,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":320.31299999999999,"5min":320.31299999999999,"15min":320.31299999999999},"last":1.238}]},{"osd":1,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":24.821999999999999,"5min":5.7329999999999997,"15min":2.5489999999999999},"min":{"1min":0.46700000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":547.65300000000002,"5min":547.65300000000002,"15min":547.65300000000002},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.3029999999999999,"5min":4.9130000000000003,"15min":2.5619999999999998},"min":{"1min":0.69799999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.5449999999999999,"5min":372.50799999999998,"15min":372.50799999999998},"last":1.7849999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.208,"15min":1.1990000000000001},"min":{"1min":0.502,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":5.2400000000000002,"5min":5.2400000000000002,"15min":5.2400000000000002},"last":0.874},{"interface":"front","average":{"1min":1.286,"5min":1.2390000000000001,"15min":1.2609999999999999},"min":{"1min":0.66800000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":6.0490000000000004,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":1.292}]},{"osd":3,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.4370000000000001,"15min":1.496},"min":{"1min":0.878,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.0670000000000002,"5min":3.46,"15min":3.46},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3600000000000001,"15min":1.3779999999999999},"min":{"1min":0.70299999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":5.2770000000000001,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":1.2669999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.504,"15min":1.6100000000000001},"min":{"1min":0.70099999999999996,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.165,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":1.365},{"interface":"front","average":{"1min":1.46,"5min":1.625,"15min":1.6559999999999999},"min":{"1min":0.96199999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":2.4279999999999999,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.891}]},{"osd":5,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.2809999999999999,"15min":1.3360000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.3069999999999999,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.6259999999999999,"15min":1.7190000000000001},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.4630000000000001,"5min":5.327,"15min":5.327},"last":1.8640000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.458,"15min":1.4410000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.4900000000000002,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.833},{"interface":"front","average":{"1min":1.5720000000000001,"5min":11.603999999999999,"15min":16.085000000000001},"min":{"1min":1.004,"5min":0.87,"15min":0.87},"max":{"1min":2.7719999999999998,"5min":266.26999999999998,"15min":266.26999999999998},"last":2.3330000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1593252,"kb_used_data":544668,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92778588,"statfs":{"total":96636764160,"available":95005274112,"internally_reserved":1073741824,"allocated":557740032,"data_stored":557284491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,5,4,1,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":20.945,"15min":29.771999999999998},"min":{"1min":0.49199999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":6.8810000000000002,"5min":497.04000000000002,"15min":497.04000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":25.385999999999999,"5min":14.656000000000001,"15min":5.6390000000000002},"min":{"1min":0.52200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":513.78899999999999,"5min":993.84500000000003,"15min":993.84500000000003},"last":1.133}]},{"osd":2,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.999,"15min":0.93200000000000005},"min":{"1min":0.60099999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.726,"5min":4.585,"15min":4.585},"last":0.57199999999999995},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.254,"15min":1.22},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":6.4029999999999996,"5min":6.4029999999999996,"15min":6.4029999999999996},"last":1.2130000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":40.524000000000001,"5min":9.2829999999999995,"15min":3.968},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":828.31600000000003,"5min":828.31600000000003,"15min":828.31600000000003},"last":1.0840000000000001},{"interface":"front","average":{"1min":44.292000000000002,"5min":10.169,"15min":4.359},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":905.38300000000004,"5min":905.38300000000004,"15min":905.38300000000004},"last":1.113}]},{"osd":4,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":27.831,"5min":6.774,"15min":3.1139999999999999},"min":{"1min":0.79900000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":550.66800000000001,"5min":550.66800000000001,"15min":550.66800000000001},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.3400000000000001,"15min":1.246},"min":{"1min":0.92100000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":5.9420000000000002,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.1759999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":8.4550000000000001,"15min":11.49},"min":{"1min":0.71199999999999997,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.7200000000000002,"5min":213.63800000000001,"15min":213.63800000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":58.497,"5min":12.720000000000001,"15min":5.0330000000000004},"min":{"1min":0.76500000000000001,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":875.44600000000003,"5min":875.44600000000003,"15min":875.44600000000003},"last":4.6319999999999997}]},{"osd":6,"last update":"Fri Aug 7 15:09:27 2020","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.587,"15min":1.5629999999999999},"min":{"1min":0.81000000000000005,"5min":0.754,"15min":0.754},"max":{"1min":5.7149999999999999,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":1.7030000000000001},{"interface":"front","average":{"1min":11.539,"5min":3.5979999999999999,"15min":2.1059999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":207.137,"5min":207.137,"15min":207.137},"last":1.3029999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.782,"5min":1.6879999999999999,"15min":1.639},"min":{"1min":0.94099999999999995,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":6.444,"5min":6.444,"15min":6.444},"last":2.5139999999999998},{"interface":"front","average":{"1min":1.6819999999999999,"5min":1.6779999999999999,"15min":1.546},"min":{"1min":0.93300000000000005,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":5.9279999999999999,"5min":10.752000000000001,"15min":10.752000000000001},"last":1.1599999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1322708,"kb_used_data":274124,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93049132,"statfs":{"total":96636764160,"available":95282311168,"internally_reserved":1073741824,"allocated":280702976,"data_stored":280247435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":17.058,"15min":6.3369999999999997},"min":{"1min":0.61399999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.1269999999999998,"5min":845.57100000000003,"15min":845.57100000000003},"last":4.5629999999999997},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.0169999999999999,"15min":0.96099999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":4.8209999999999997,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":0.62}]},{"osd":2,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.0189999999999999,"15min":0.95799999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.7719999999999998,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":13.177,"5min":22.48,"15min":29.32},"min":{"1min":0.61399999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":226.911,"5min":606.18499999999995,"15min":606.18499999999995},"last":0.64600000000000002}]},{"osd":3,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.0660000000000001,"15min":0.97899999999999998},"min":{"1min":0.54100000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6859999999999999,"5min":3.1779999999999999,"15min":3.1779999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.3160000000000001,"15min":1.2170000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.976,"5min":4.21,"15min":4.21},"last":0.76300000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.2829999999999999,"15min":1.2050000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":8.3100000000000005,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.7709999999999999,"5min":1.446,"15min":1.3200000000000001},"min":{"1min":0.84599999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":8.4190000000000005,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":0.88900000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.9770000000000001,"5min":9.0380000000000003,"15min":3.8300000000000001},"min":{"1min":0.77800000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":7.8929999999999998,"5min":501.267,"15min":501.267},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.212,"5min":1.2609999999999999,"15min":1.157},"min":{"1min":0.78200000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.0939999999999999,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":0.91500000000000004}]},{"osd":6,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":5.7779999999999996,"15min":7.6680000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.0059999999999998,"5min":124.738,"15min":124.738},"last":0.872},{"interface":"front","average":{"1min":1.627,"5min":11.811,"15min":16.372},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":4.6879999999999997,"5min":294.94,"15min":294.94},"last":1.1659999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.5589999999999999,"15min":1.577},"min":{"1min":0.95399999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":4.7169999999999996,"5min":6.468,"15min":6.468},"last":1.605},{"interface":"front","average":{"1min":1.363,"5min":1.4410000000000001,"15min":1.4039999999999999},"min":{"1min":0.88800000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.1040000000000001,"5min":3.117,"15min":3.117},"last":0.94499999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1207984,"kb_used_data":159400,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93163856,"statfs":{"total":96636764160,"available":95399788544,"internally_reserved":1073741824,"allocated":163225600,"data_stored":162762563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.6950000000000001,"5min":1.268,"15min":1.167},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":8.4930000000000003,"5min":8.4930000000000003,"15min":8.4930000000000003},"last":3.706},{"interface":"front","average":{"1min":1.357,"5min":1.113,"15min":1.0209999999999999},"min":{"1min":0.61899999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":3.5,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.88400000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.9390000000000001,"5min":1.5469999999999999,"15min":1.2430000000000001},"min":{"1min":0.54400000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":5.6820000000000004,"5min":18.672000000000001,"15min":18.672000000000001},"last":2.5339999999999998},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.351,"15min":1.357},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":3.585,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":3.7970000000000002}]},{"osd":3,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.268,"15min":1.198},"min":{"1min":0.72799999999999998,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.8740000000000001,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":3.1970000000000001},{"interface":"front","average":{"1min":34.170999999999999,"5min":8.0510000000000002,"15min":3.7210000000000001},"min":{"1min":0.68799999999999994,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":501.75599999999997,"5min":501.75599999999997,"15min":501.75599999999997},"last":2.5609999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.403,"15min":1.2070000000000001},"min":{"1min":0.65200000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":3.4180000000000001,"5min":19.783000000000001,"15min":19.783000000000001},"last":1.234},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.546,"15min":1.6100000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":3.5569999999999999,"5min":3.5569999999999999,"15min":3.5569999999999999},"last":3.7389999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.292,"15min":1.272},"min":{"1min":0.83299999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.323,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":3.1160000000000001},{"interface":"front","average":{"1min":43.396999999999998,"5min":24.298999999999999,"15min":25.116},"min":{"1min":0.99299999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":627.53300000000002,"5min":627.53300000000002,"15min":627.53300000000002},"last":3.081}]},{"osd":6,"last update":"Fri Aug 7 15:09:30 2020","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.538,"15min":1.3260000000000001},"min":{"1min":0.85999999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":3.452,"5min":20.035,"15min":20.035},"last":1.526},{"interface":"front","average":{"1min":1.498,"5min":9.8979999999999997,"15min":4.1799999999999997},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.976,"5min":751.70000000000005,"15min":751.70000000000005},"last":3.1480000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":54.209000000000003,"5min":12.595000000000001,"15min":5.359},"min":{"1min":0.84999999999999998,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":501.91699999999997,"5min":501.91699999999997,"15min":501.91699999999997},"last":3.7709999999999999},{"interface":"front","average":{"1min":1.665,"5min":1.919,"15min":1.637},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":3.133,"5min":35.646999999999998,"15min":35.646999999999998},"last":3.1739999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1303512,"kb_used_data":254920,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93068328,"statfs":{"total":96636764160,"available":95301967872,"internally_reserved":1073741824,"allocated":261038080,"data_stored":260575043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":1.341,"15min":1.282},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.097,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.4530000000000001,"15min":1.575},"min":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.3039999999999998,"5min":13.067,"15min":13.067},"last":0.89900000000000002}]},{"osd":1,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":4.5860000000000003,"15min":2.794},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.9910000000000001,"5min":270.68700000000001,"15min":270.68700000000001},"last":2.3090000000000002},{"interface":"front","average":{"1min":1.4379999999999999,"5min":2.0390000000000001,"15min":2.0209999999999999},"min":{"1min":0.55100000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":7.0940000000000003,"5min":14.349,"15min":14.349},"last":1.855}]},{"osd":2,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.266,"15min":1.1870000000000001},"min":{"1min":0.71399999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.2639999999999998,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":1.4470000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":2.109,"15min":1.8260000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.661,"5min":51.530000000000001,"15min":51.530000000000001},"last":1.1739999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":10.773999999999999,"15min":4.7220000000000004},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.1360000000000001,"5min":506.21300000000002,"15min":506.21300000000002},"last":2.0150000000000001},{"interface":"front","average":{"1min":1.276,"5min":25.469000000000001,"15min":29.888999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.8940000000000001,"5min":677.88599999999997,"15min":677.88599999999997},"last":3.1480000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.5920000000000001,"15min":1.542},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.9710000000000001,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":3.1869999999999998},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.8009999999999999,"15min":1.7789999999999999},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":3.044,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":3.214}]},{"osd":6,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.6259999999999999,"15min":1.5629999999999999},"min":{"1min":0.89800000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.9359999999999999,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":1.8009999999999999},{"interface":"front","average":{"1min":21.766999999999999,"5min":5.9390000000000001,"15min":3.3039999999999998},"min":{"1min":0.89900000000000002,"5min":0.876,"15min":0.876},"max":{"1min":387.911,"5min":387.911,"15min":387.911},"last":1.2130000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.9330000000000001,"15min":2.085},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.0110000000000001,"5min":5.734,"15min":5.734},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.9750000000000001,"15min":1.976},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.169,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":2.347}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1482524,"kb_used_data":433940,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92889316,"statfs":{"total":96636764160,"available":95118659584,"internally_reserved":1073741824,"allocated":444354560,"data_stored":443899019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":1.0720000000000001,"15min":1.1439999999999999},"min":{"1min":0.52900000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4390000000000001,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":0.55100000000000005},{"interface":"front","average":{"1min":1.143,"5min":1.0069999999999999,"15min":0.90100000000000002},"min":{"1min":0.498,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":4.907,"5min":6.3970000000000002,"15min":6.3970000000000002},"last":0.81299999999999994}]},{"osd":1,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":23.940999999999999,"5min":5.8209999999999997,"15min":2.843},"min":{"1min":0.59599999999999997,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":320.67000000000002,"5min":320.67000000000002,"15min":320.67000000000002},"last":1.413},{"interface":"front","average":{"1min":1.1359999999999999,"5min":7.5700000000000003,"15min":3.4409999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.784,"5min":689.846,"15min":689.846},"last":0.58799999999999997}]},{"osd":2,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.3999999999999999,"15min":1.4339999999999999},"min":{"1min":0.77600000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.0899999999999999,"5min":6.8730000000000002,"15min":6.8730000000000002},"last":2.0219999999999998},{"interface":"front","average":{"1min":1.258,"5min":1.516,"15min":1.5149999999999999},"min":{"1min":0.81699999999999995,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.3159999999999998,"5min":7.1360000000000001,"15min":7.1360000000000001},"last":2.048}]},{"osd":3,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":3.681,"15min":2.2320000000000002},"min":{"1min":0.78300000000000003,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.8149999999999999,"5min":247.202,"15min":247.202},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.3720000000000001,"15min":1.2909999999999999},"min":{"1min":0.68500000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":4.931,"5min":6.431,"15min":6.431},"last":0.99399999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":10.57,"15min":4.6079999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.431,"5min":990.71600000000001,"15min":990.71600000000001},"last":0.878},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.6100000000000001,"15min":1.621},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":3.3519999999999999,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":1.0649999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:09:28 2020","interfaces":[{"interface":"back","average":{"1min":1.6799999999999999,"5min":1.7509999999999999,"15min":1.7769999999999999},"min":{"1min":0.75800000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":5.0149999999999997,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.498,"5min":12.069000000000001,"15min":16.762},"min":{"1min":0.83299999999999996,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":3.3530000000000002,"5min":389.14600000000002,"15min":389.14600000000002},"last":0.93700000000000006}]},{"osd":7,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":3.8300000000000001,"15min":2.2149999999999999},"min":{"1min":0.753,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":4.8710000000000004,"5min":252.798,"15min":252.798},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.262,"5min":10.175000000000001,"15min":4.4889999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.3050000000000002,"5min":501.93000000000001,"15min":501.93000000000001},"last":0.95499999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1374900,"kb_used_data":326316,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92996940,"statfs":{"total":96636764160,"available":95228866560,"internally_reserved":1073741824,"allocated":334147584,"data_stored":333692043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":1.012,"15min":1.0549999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.651,"5min":2.766,"15min":2.766},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91600000000000004,"15min":0.84199999999999997},"min":{"1min":0.45500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.3340000000000001,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":0.86599999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.4540000000000002,"5min":10.521000000000001,"15min":4.1260000000000003},"min":{"1min":0.56699999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":29.283000000000001,"5min":787.22900000000004,"15min":787.22900000000004},"last":1.0089999999999999},{"interface":"front","average":{"1min":8.0939999999999994,"5min":2.4969999999999999,"15min":1.571},"min":{"1min":0.60999999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":116.548,"5min":116.548,"15min":116.548},"last":0.75800000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":5.9340000000000002,"15min":8.1240000000000006},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.78,"5min":168.56899999999999,"15min":168.56899999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.119,"5min":20.361000000000001,"15min":28.873999999999999},"min":{"1min":0.60899999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.665,"5min":676.27099999999996,"15min":676.27099999999996},"last":0.82899999999999996}]},{"osd":3,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.6739999999999999,"5min":1.5069999999999999,"15min":1.2470000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":29.129000000000001,"5min":29.129000000000001,"15min":29.129000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":3.0219999999999998,"5min":9.9459999999999997,"15min":4.1239999999999997},"min":{"1min":0.70799999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":29.103000000000002,"5min":712.98699999999997,"15min":712.98699999999997},"last":0.97399999999999998}]},{"osd":4,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":49.854999999999997,"5min":40.755000000000003,"15min":47.512999999999998},"min":{"1min":0.61899999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":971.67999999999995,"5min":971.67999999999995,"15min":971.67999999999995},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.236,"5min":1.264,"15min":1.292},"min":{"1min":0.64900000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.5960000000000001,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":0.95199999999999996}]},{"osd":6,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":91.870999999999995,"5min":48.862000000000002,"15min":49.893000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":966.60900000000004,"5min":966.60900000000004,"15min":966.60900000000004},"last":1.03},{"interface":"front","average":{"1min":28.396999999999998,"5min":21.527000000000001,"15min":24.52},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":501.79000000000002,"5min":517.26700000000005,"15min":517.26700000000005},"last":0.88100000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.341,"15min":1.319},"min":{"1min":0.77500000000000002,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.8940000000000001,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.109},{"interface":"front","average":{"1min":27.206,"5min":6.6630000000000003,"15min":3.2650000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":515.09100000000001,"5min":515.09100000000001,"15min":515.09100000000001},"last":0.995}]}]}]}} 2020-08-07T15:09:55.361 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:09:55.365 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2020-08-07T15:09:55.365 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 89 16 114766 114750 10.0719 12.8672 0.00831756 0.0124057 2020-08-07T15:09:55.366 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:55.351+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:55.367 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:09:55.367 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:09:55.368 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump -f json-pretty 2020-08-07T15:09:55.462 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:55.457+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:55.584 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:09:55.584 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:09:55.595 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:09:55.595 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:09:55.599 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:09:55.601 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:55.582+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:55.602 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:09:55.637 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:09:55.682 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:09:55.689 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:55.686+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:55.749 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:09:55.793 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:09:55.794 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:55.791+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:55.848 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:09:55.849 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:09:55.850 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:09:55.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:09:55.850 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:09:55.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.850 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42440 4.9 4:9b3fd4e3:::benchmark_data_smithi074_44586_object5304:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:09:55.851 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:09:55.851 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.572865262000001, 2020-08-07T15:09:55.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9384075030000001, 2020-08-07T15:09:55.851 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:55.851 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:55.852 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:55.852 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:55.852 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:55.852 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42440 2020-08-07T15:09:55.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.853 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:55.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:55.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:09:55.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:55.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:55.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:09:55.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6295000000000002e-05 2020-08-07T15:09:55.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:55.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237116+0000", 2020-08-07T15:09:55.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0390000000000001e-06 2020-08-07T15:09:55.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:55.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237122+0000", 2020-08-07T15:09:55.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8340000000000001e-05 2020-08-07T15:09:55.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:55.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237140+0000", 2020-08-07T15:09:55.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.045e-06 2020-08-07T15:09:55.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:55.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237145+0000", 2020-08-07T15:09:55.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044548920000000002 2020-08-07T15:09:55.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:55.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241600+0000", 2020-08-07T15:09:55.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103336 2020-08-07T15:09:55.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:55.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241703+0000", 2020-08-07T15:09:55.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018660399999999999 2020-08-07T15:09:55.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:09:55.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241890+0000", 2020-08-07T15:09:55.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00096796000000000002 2020-08-07T15:09:55.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:55.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.242858+0000", 2020-08-07T15:09:55.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039845000000000002 2020-08-07T15:09:55.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:55.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246843+0000", 2020-08-07T15:09:55.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3499999999999996e-07 2020-08-07T15:09:55.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:55.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246843+0000", 2020-08-07T15:09:55.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7132999999999995e-05 2020-08-07T15:09:55.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:55.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246920+0000", 2020-08-07T15:09:55.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046993410000000001 2020-08-07T15:09:55.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:55.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.251620+0000", 2020-08-07T15:09:55.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040332956000000003 2020-08-07T15:09:55.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:55.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.291953+0000", 2020-08-07T15:09:55.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8390000000000003e-06 2020-08-07T15:09:55.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:55.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.291958+0000", 2020-08-07T15:09:55.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99961357500000003 2020-08-07T15:09:55.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:55.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291572+0000", 2020-08-07T15:09:55.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3913999999999999e-05 2020-08-07T15:09:55.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:55.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291586+0000", 2020-08-07T15:09:55.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88379723099999996 2020-08-07T15:09:55.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:55.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175383+0000", 2020-08-07T15:09:55.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3868000000000004e-05 2020-08-07T15:09:55.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:55.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175467+0000", 2020-08-07T15:09:55.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9384075030000001 2020-08-07T15:09:55.917 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:55.922 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:55.926 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:55.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.926 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42489 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:09:55.927 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:09:55.927 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.5295652, 2020-08-07T15:09:55.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952608639999999, 2020-08-07T15:09:55.928 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:55.928 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:55.932 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:55.932 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:55.932 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:55.935 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42489 2020-08-07T15:09:55.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.936 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:55.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:55.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:09:55.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:55.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:55.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:09:55.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0379e-05 2020-08-07T15:09:55.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:55.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:55.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:56.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280410+0000", 2020-08-07T15:09:56.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0489999999999998e-06 2020-08-07T15:09:56.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:56.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280417+0000", 2020-08-07T15:09:56.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8281e-05 2020-08-07T15:09:56.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:56.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280435+0000", 2020-08-07T15:09:56.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7510000000000008e-06 2020-08-07T15:09:56.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:56.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280443+0000", 2020-08-07T15:09:56.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018142899999999999 2020-08-07T15:09:56.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280624+0000", 2020-08-07T15:09:56.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013473599999999999 2020-08-07T15:09:56.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:56.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280759+0000", 2020-08-07T15:09:56.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019964200000000001 2020-08-07T15:09:56.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:09:56.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280959+0000", 2020-08-07T15:09:56.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005588038 2020-08-07T15:09:56.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:56.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286547+0000", 2020-08-07T15:09:56.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.376e-05 2020-08-07T15:09:56.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:56.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286561+0000", 2020-08-07T15:09:56.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4199999999999995e-07 2020-08-07T15:09:56.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:56.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286561+0000", 2020-08-07T15:09:56.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9218e-05 2020-08-07T15:09:56.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:56.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286631+0000", 2020-08-07T15:09:56.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027909199999999999 2020-08-07T15:09:56.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:56.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286910+0000", 2020-08-07T15:09:56.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050904089999999997 2020-08-07T15:09:56.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292000+0000", 2020-08-07T15:09:56.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9579999999999996e-06 2020-08-07T15:09:56.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:56.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292009+0000", 2020-08-07T15:09:56.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99961402799999999 2020-08-07T15:09:56.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291623+0000", 2020-08-07T15:09:56.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2490000000000001e-06 2020-08-07T15:09:56.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:56.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291631+0000", 2020-08-07T15:09:56.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88387422199999999 2020-08-07T15:09:56.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175506+0000", 2020-08-07T15:09:56.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000114881 2020-08-07T15:09:56.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:56.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175620+0000", 2020-08-07T15:09:56.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952608639999999 2020-08-07T15:09:56.186 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:56.186 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:56.200 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:56.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.201 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42490 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:09:56.201 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:09:56.201 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.529438032000002, 2020-08-07T15:09:56.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89528059, 2020-08-07T15:09:56.235 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:56.238 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:56.238 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:56.239 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:56.239 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:56.239 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42490 2020-08-07T15:09:56.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.239 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:56.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:56.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:09:56.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:56.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:56.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:09:56.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2571999999999997e-05 2020-08-07T15:09:56.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:56.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280539+0000", 2020-08-07T15:09:56.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999998e-06 2020-08-07T15:09:56.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:56.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280545+0000", 2020-08-07T15:09:56.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.134e-05 2020-08-07T15:09:56.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:56.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280556+0000", 2020-08-07T15:09:56.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.746e-06 2020-08-07T15:09:56.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:56.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280561+0000", 2020-08-07T15:09:56.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00089258999999999996 2020-08-07T15:09:56.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281453+0000", 2020-08-07T15:09:56.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5641000000000001e-05 2020-08-07T15:09:56.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:56.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281549+0000", 2020-08-07T15:09:56.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020555200000000001 2020-08-07T15:09:56.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:09:56.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281755+0000", 2020-08-07T15:09:56.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048878230000000003 2020-08-07T15:09:56.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:56.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286642+0000", 2020-08-07T15:09:56.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0954e-05 2020-08-07T15:09:56.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:56.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286693+0000", 2020-08-07T15:09:56.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0699999999999997e-07 2020-08-07T15:09:56.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:56.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286694+0000", 2020-08-07T15:09:56.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3697999999999998e-05 2020-08-07T15:09:56.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:56.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286768+0000", 2020-08-07T15:09:56.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017185900000000001 2020-08-07T15:09:56.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:56.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286940+0000", 2020-08-07T15:09:56.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051207819999999999 2020-08-07T15:09:56.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292060+0000", 2020-08-07T15:09:56.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8000000000000004e-06 2020-08-07T15:09:56.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:56.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292069+0000", 2020-08-07T15:09:56.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99959520599999996 2020-08-07T15:09:56.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291664+0000", 2020-08-07T15:09:56.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0160000000000008e-06 2020-08-07T15:09:56.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:56.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291672+0000", 2020-08-07T15:09:56.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88399374500000005 2020-08-07T15:09:56.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175666+0000", 2020-08-07T15:09:56.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101239 2020-08-07T15:09:56.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:56.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175767+0000", 2020-08-07T15:09:56.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89528059 2020-08-07T15:09:56.324 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:56.324 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:56.324 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:56.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.325 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42491 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:09:56.325 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:09:56.325 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.529331096999996, 2020-08-07T15:09:56.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895291936, 2020-08-07T15:09:56.325 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:56.326 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:56.326 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:56.326 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:56.326 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:56.326 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42491 2020-08-07T15:09:56.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.327 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:56.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:56.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:09:56.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:56.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:56.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:09:56.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0402000000000001e-05 2020-08-07T15:09:56.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:56.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280624+0000", 2020-08-07T15:09:56.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0340000000000001e-06 2020-08-07T15:09:56.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:56.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280627+0000", 2020-08-07T15:09:56.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1886999999999999e-05 2020-08-07T15:09:56.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:56.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280649+0000", 2020-08-07T15:09:56.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5430000000000004e-06 2020-08-07T15:09:56.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:56.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280654+0000", 2020-08-07T15:09:56.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032800960000000001 2020-08-07T15:09:56.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.283934+0000", 2020-08-07T15:09:56.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011264699999999999 2020-08-07T15:09:56.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:56.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284046+0000", 2020-08-07T15:09:56.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019987299999999999 2020-08-07T15:09:56.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:09:56.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284246+0000", 2020-08-07T15:09:56.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025610960000000001 2020-08-07T15:09:56.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:56.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286807+0000", 2020-08-07T15:09:56.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1409999999999999e-06 2020-08-07T15:09:56.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:56.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286814+0000", 2020-08-07T15:09:56.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0499999999999999e-07 2020-08-07T15:09:56.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:56.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286815+0000", 2020-08-07T15:09:56.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8538000000000003e-05 2020-08-07T15:09:56.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:56.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286853+0000", 2020-08-07T15:09:56.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112914 2020-08-07T15:09:56.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:56.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286966+0000", 2020-08-07T15:09:56.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051403120000000002 2020-08-07T15:09:56.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292107+0000", 2020-08-07T15:09:56.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8019999999999992e-06 2020-08-07T15:09:56.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:56.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292114+0000", 2020-08-07T15:09:56.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.999590859 2020-08-07T15:09:56.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291705+0000", 2020-08-07T15:09:56.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.174e-06 2020-08-07T15:09:56.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:56.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291713+0000", 2020-08-07T15:09:56.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88407676000000002 2020-08-07T15:09:56.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175790+0000", 2020-08-07T15:09:56.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5452999999999996e-05 2020-08-07T15:09:56.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:56.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175886+0000", 2020-08-07T15:09:56.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895291936 2020-08-07T15:09:56.441 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:56.446 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:56.447 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:56.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.447 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42492 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:09:56.448 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:09:56.448 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.529238047, 2020-08-07T15:09:56.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895293774, 2020-08-07T15:09:56.448 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:56.448 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:56.449 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:56.449 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:56.449 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:56.468 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42492 2020-08-07T15:09:56.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.469 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:56.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:56.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:09:56.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:56.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:56.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:09:56.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8272000000000001e-05 2020-08-07T15:09:56.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:56.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280735+0000", 2020-08-07T15:09:56.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2159999999999997e-06 2020-08-07T15:09:56.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:56.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280740+0000", 2020-08-07T15:09:56.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0251000000000001e-05 2020-08-07T15:09:56.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:56.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280750+0000", 2020-08-07T15:09:56.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3139999999999997e-06 2020-08-07T15:09:56.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:56.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280755+0000", 2020-08-07T15:09:56.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039343579999999998 2020-08-07T15:09:56.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284689+0000", 2020-08-07T15:09:56.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111648 2020-08-07T15:09:56.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:56.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284801+0000", 2020-08-07T15:09:56.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019563600000000001 2020-08-07T15:09:56.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:09:56.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284996+0000", 2020-08-07T15:09:56.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018636709999999999 2020-08-07T15:09:56.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:56.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286860+0000", 2020-08-07T15:09:56.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001865802 2020-08-07T15:09:56.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:56.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288726+0000", 2020-08-07T15:09:56.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0399999999999996e-07 2020-08-07T15:09:56.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:56.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288726+0000", 2020-08-07T15:09:56.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6573999999999998e-05 2020-08-07T15:09:56.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:56.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288773+0000", 2020-08-07T15:09:56.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003455283 2020-08-07T15:09:56.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292228+0000", 2020-08-07T15:09:56.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0099999999999995e-06 2020-08-07T15:09:56.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:56.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292236+0000", 2020-08-07T15:09:56.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99950963999999998 2020-08-07T15:09:56.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291746+0000", 2020-08-07T15:09:56.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0779999999999996e-06 2020-08-07T15:09:56.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:56.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291754+0000", 2020-08-07T15:09:56.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.884152508 2020-08-07T15:09:56.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175906+0000", 2020-08-07T15:09:56.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4009000000000002e-05 2020-08-07T15:09:56.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:56.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175980+0000", 2020-08-07T15:09:56.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895293774 2020-08-07T15:09:56.514 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:56.514 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:56.514 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:56.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.515 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42493 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:09:56.515 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:09:56.515 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.529136399999999, 2020-08-07T15:09:56.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952963789999999, 2020-08-07T15:09:56.516 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:56.516 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:56.526 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:56.526 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:56.527 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:56.527 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42493 2020-08-07T15:09:56.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.527 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:56.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:56.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:09:56.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:56.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:56.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:09:56.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9740999999999997e-05 2020-08-07T15:09:56.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:56.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280838+0000", 2020-08-07T15:09:56.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7809999999999996e-06 2020-08-07T15:09:56.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:56.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280843+0000", 2020-08-07T15:09:56.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0648999999999999e-05 2020-08-07T15:09:56.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:56.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280854+0000", 2020-08-07T15:09:56.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3669999999999999e-06 2020-08-07T15:09:56.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:56.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280858+0000", 2020-08-07T15:09:56.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004850867 2020-08-07T15:09:56.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.285709+0000", 2020-08-07T15:09:56.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110037 2020-08-07T15:09:56.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:56.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.285819+0000", 2020-08-07T15:09:56.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020935699999999999 2020-08-07T15:09:56.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:09:56.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286028+0000", 2020-08-07T15:09:56.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020900850000000002 2020-08-07T15:09:56.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:56.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288118+0000", 2020-08-07T15:09:56.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041669649999999999 2020-08-07T15:09:56.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292285+0000", 2020-08-07T15:09:56.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3820000000000004e-06 2020-08-07T15:09:56.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:56.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292294+0000", 2020-08-07T15:09:56.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949242900000002 2020-08-07T15:09:56.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291786+0000", 2020-08-07T15:09:56.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000003e-06 2020-08-07T15:09:56.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:56.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291794+0000", 2020-08-07T15:09:56.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88420652700000002 2020-08-07T15:09:56.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176000+0000", 2020-08-07T15:09:56.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4634000000000003e-05 2020-08-07T15:09:56.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:56.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176085+0000", 2020-08-07T15:09:56.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952963789999999 2020-08-07T15:09:56.554 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:56.555 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:56.555 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:56.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.555 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42494 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:09:56.556 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:09:56.556 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.529034301000003, 2020-08-07T15:09:56.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952882820000001, 2020-08-07T15:09:56.556 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:56.556 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:56.556 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:56.557 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:56.557 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:56.557 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42494 2020-08-07T15:09:56.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.557 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:56.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:56.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:09:56.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:56.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:56.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:09:56.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0117999999999997e-05 2020-08-07T15:09:56.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:56.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280941+0000", 2020-08-07T15:09:56.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1240000000000004e-06 2020-08-07T15:09:56.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:56.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280946+0000", 2020-08-07T15:09:56.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0000000000000001e-05 2020-08-07T15:09:56.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:56.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280956+0000", 2020-08-07T15:09:56.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.279e-06 2020-08-07T15:09:56.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:56.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280960+0000", 2020-08-07T15:09:56.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0062727080000000001 2020-08-07T15:09:56.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287233+0000", 2020-08-07T15:09:56.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018362900000000001 2020-08-07T15:09:56.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:56.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287416+0000", 2020-08-07T15:09:56.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029536100000000001 2020-08-07T15:09:56.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:09:56.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287712+0000", 2020-08-07T15:09:56.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046231739999999999 2020-08-07T15:09:56.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292335+0000", 2020-08-07T15:09:56.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3860000000000001e-06 2020-08-07T15:09:56.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:56.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292342+0000", 2020-08-07T15:09:56.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99948501000000001 2020-08-07T15:09:56.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291827+0000", 2020-08-07T15:09:56.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2272999999999999e-05 2020-08-07T15:09:56.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:56.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291840+0000", 2020-08-07T15:09:56.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88426668100000005 2020-08-07T15:09:56.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176106+0000", 2020-08-07T15:09:56.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2539000000000005e-05 2020-08-07T15:09:56.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:56.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176179+0000", 2020-08-07T15:09:56.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952882820000001 2020-08-07T15:09:56.670 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:56.670 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:56.670 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:56.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.671 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42495 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:09:56.671 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:09:56.671 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.528933211000002, 2020-08-07T15:09:56.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895286867, 2020-08-07T15:09:56.672 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:56.672 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:56.672 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:56.672 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:56.673 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:56.673 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42495 2020-08-07T15:09:56.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.673 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:56.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:09:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:56.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:09:56.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6690999999999997e-05 2020-08-07T15:09:56.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:56.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281038+0000", 2020-08-07T15:09:56.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7860000000000002e-06 2020-08-07T15:09:56.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:56.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281044+0000", 2020-08-07T15:09:56.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0429e-05 2020-08-07T15:09:56.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:56.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281054+0000", 2020-08-07T15:09:56.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9040000000000002e-06 2020-08-07T15:09:56.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:56.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281058+0000", 2020-08-07T15:09:56.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070941490000000001 2020-08-07T15:09:56.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288153+0000", 2020-08-07T15:09:56.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9006999999999998e-05 2020-08-07T15:09:56.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:56.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288222+0000", 2020-08-07T15:09:56.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012106400000000001 2020-08-07T15:09:56.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:09:56.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288343+0000", 2020-08-07T15:09:56.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004034752 2020-08-07T15:09:56.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292377+0000", 2020-08-07T15:09:56.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.537e-06 2020-08-07T15:09:56.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:56.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292385+0000", 2020-08-07T15:09:56.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949123500000003 2020-08-07T15:09:56.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291876+0000", 2020-08-07T15:09:56.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7729999999999999e-06 2020-08-07T15:09:56.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:56.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291884+0000", 2020-08-07T15:09:56.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88431700899999999 2020-08-07T15:09:56.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176201+0000", 2020-08-07T15:09:56.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7531e-05 2020-08-07T15:09:56.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:56.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176278+0000", 2020-08-07T15:09:56.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895286867 2020-08-07T15:09:56.718 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:56.718 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:56.718 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:56.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.719 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42496 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:09:56.719 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:09:56.719 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.528828302000001, 2020-08-07T15:09:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89526942, 2020-08-07T15:09:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:09:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42496 2020-08-07T15:09:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:09:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:09:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9227999999999997e-05 2020-08-07T15:09:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281146+0000", 2020-08-07T15:09:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1120000000000004e-06 2020-08-07T15:09:56.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:56.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281151+0000", 2020-08-07T15:09:56.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2208999999999999e-05 2020-08-07T15:09:56.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:56.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281163+0000", 2020-08-07T15:09:56.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5469999999999998e-06 2020-08-07T15:09:56.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:56.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281168+0000", 2020-08-07T15:09:56.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0076520829999999996 2020-08-07T15:09:56.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288820+0000", 2020-08-07T15:09:56.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5035000000000003e-05 2020-08-07T15:09:56.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:56.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288885+0000", 2020-08-07T15:09:56.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116507 2020-08-07T15:09:56.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:09:56.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.289001+0000", 2020-08-07T15:09:56.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034121960000000002 2020-08-07T15:09:56.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292413+0000", 2020-08-07T15:09:56.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8109999999999997e-06 2020-08-07T15:09:56.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:56.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292420+0000", 2020-08-07T15:09:56.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949504600000005 2020-08-07T15:09:56.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291915+0000", 2020-08-07T15:09:56.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3999999999999992e-06 2020-08-07T15:09:56.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:56.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291924+0000", 2020-08-07T15:09:56.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88437394599999997 2020-08-07T15:09:56.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176298+0000", 2020-08-07T15:09:56.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8300000000000007e-05 2020-08-07T15:09:56.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:56.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176366+0000", 2020-08-07T15:09:56.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89526942 2020-08-07T15:09:56.746 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:56.746 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:56.746 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:56.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.747 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73081 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:56.747 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:09:56.747 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.583359028, 2020-08-07T15:09:56.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0005038420000001, 2020-08-07T15:09:56.748 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:56.748 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:56.766 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:56.766 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:56.767 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:56.767 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73081 2020-08-07T15:09:56.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.767 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:56.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:56.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:09:56.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:56.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:56.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:09:56.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3016000000000003e-05 2020-08-07T15:09:56.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:56.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226609+0000", 2020-08-07T15:09:56.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4479999999999996e-06 2020-08-07T15:09:56.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:56.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226613+0000", 2020-08-07T15:09:56.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0189999999999995e-06 2020-08-07T15:09:56.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:56.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226622+0000", 2020-08-07T15:09:56.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5980000000000001e-06 2020-08-07T15:09:56.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:56.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226626+0000", 2020-08-07T15:09:56.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8088999999999999e-05 2020-08-07T15:09:56.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226684+0000", 2020-08-07T15:09:56.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2162e-05 2020-08-07T15:09:56.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226696+0000", 2020-08-07T15:09:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98675689 2020-08-07T15:09:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213453+0000", 2020-08-07T15:09:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012916800000000001 2020-08-07T15:09:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213582+0000", 2020-08-07T15:09:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015665899999999999 2020-08-07T15:09:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:09:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213739+0000", 2020-08-07T15:09:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049940239999999997 2020-08-07T15:09:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218733+0000", 2020-08-07T15:09:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0079774779999999997 2020-08-07T15:09:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:56.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226710+0000", 2020-08-07T15:09:56.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.161e-06 2020-08-07T15:09:56.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:56.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226711+0000", 2020-08-07T15:09:56.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0105000000000001e-05 2020-08-07T15:09:56.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:56.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226772+0000", 2020-08-07T15:09:56.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5633999999999997e-05 2020-08-07T15:09:56.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:56.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226817+0000", 2020-08-07T15:09:56.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7379000000000001e-05 2020-08-07T15:09:56.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:56.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226875+0000", 2020-08-07T15:09:56.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8000000000000005e-07 2020-08-07T15:09:56.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:56.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226875+0000", 2020-08-07T15:09:56.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019423199999999999 2020-08-07T15:09:56.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:56.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227070+0000", 2020-08-07T15:09:56.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0005038420000001 2020-08-07T15:09:56.814 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:56.814 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:56.815 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:56.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.815 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73082 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:56.816 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:09:56.816 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.582988564000001, 2020-08-07T15:09:56.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001326693, 2020-08-07T15:09:56.816 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:56.817 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:56.817 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:56.817 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:56.817 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:56.818 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73082 2020-08-07T15:09:56.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.818 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:56.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:56.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:09:56.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:56.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:56.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:09:56.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6928000000000003e-05 2020-08-07T15:09:56.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:56.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226973+0000", 2020-08-07T15:09:56.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3630000000000004e-06 2020-08-07T15:09:56.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:56.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226977+0000", 2020-08-07T15:09:56.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9659999999999996e-06 2020-08-07T15:09:56.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:56.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226985+0000", 2020-08-07T15:09:56.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5800000000000002e-06 2020-08-07T15:09:56.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:56.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226990+0000", 2020-08-07T15:09:56.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9101999999999998e-05 2020-08-07T15:09:56.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227039+0000", 2020-08-07T15:09:56.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9620000000000005e-06 2020-08-07T15:09:56.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:56.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227049+0000", 2020-08-07T15:09:56.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98722513999999995 2020-08-07T15:09:56.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214274+0000", 2020-08-07T15:09:56.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2700999999999994e-05 2020-08-07T15:09:56.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:56.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214347+0000", 2020-08-07T15:09:56.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015346899999999999 2020-08-07T15:09:56.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:09:56.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214500+0000", 2020-08-07T15:09:56.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042450309999999998 2020-08-07T15:09:56.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:56.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218745+0000", 2020-08-07T15:09:56.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008379503 2020-08-07T15:09:56.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:56.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227125+0000", 2020-08-07T15:09:56.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7899999999999998e-07 2020-08-07T15:09:56.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:56.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227126+0000", 2020-08-07T15:09:56.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1610000000000002e-05 2020-08-07T15:09:56.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:56.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227177+0000", 2020-08-07T15:09:56.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489000000000003e-05 2020-08-07T15:09:56.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:56.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227226+0000", 2020-08-07T15:09:56.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00084338600000000001 2020-08-07T15:09:56.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:56.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228069+0000", 2020-08-07T15:09:56.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7e-07 2020-08-07T15:09:56.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:56.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228069+0000", 2020-08-07T15:09:56.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019351399999999999 2020-08-07T15:09:56.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:56.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228263+0000", 2020-08-07T15:09:56.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001326693 2020-08-07T15:09:56.843 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:56.843 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:56.844 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:56.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.844 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73083 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:56.844 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:09:56.845 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.582894305, 2020-08-07T15:09:56.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015968209999999, 2020-08-07T15:09:56.845 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:56.845 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:56.845 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:56.846 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:56.846 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:56.846 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73083 2020-08-07T15:09:56.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.846 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:56.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:56.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:09:56.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:56.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:56.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:09:56.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1627000000000001e-05 2020-08-07T15:09:56.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:56.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227062+0000", 2020-08-07T15:09:56.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.253e-06 2020-08-07T15:09:56.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:56.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227065+0000", 2020-08-07T15:09:56.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3168e-05 2020-08-07T15:09:56.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:56.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227079+0000", 2020-08-07T15:09:56.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1549999999999999e-06 2020-08-07T15:09:56.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:56.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227082+0000", 2020-08-07T15:09:56.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5534999999999997e-05 2020-08-07T15:09:56.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227127+0000", 2020-08-07T15:09:56.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6823000000000001e-05 2020-08-07T15:09:56.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227144+0000", 2020-08-07T15:09:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98771770599999997 2020-08-07T15:09:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214862+0000", 2020-08-07T15:09:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8252999999999999e-05 2020-08-07T15:09:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:56.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214930+0000", 2020-08-07T15:09:56.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013646400000000001 2020-08-07T15:09:56.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:09:56.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215066+0000", 2020-08-07T15:09:56.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036907089999999999 2020-08-07T15:09:56.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:56.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218757+0000", 2020-08-07T15:09:56.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0085133450000000003 2020-08-07T15:09:56.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:56.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227270+0000", 2020-08-07T15:09:56.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4899999999999995e-07 2020-08-07T15:09:56.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:56.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227271+0000", 2020-08-07T15:09:56.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4555999999999997e-05 2020-08-07T15:09:56.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:56.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227326+0000", 2020-08-07T15:09:56.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024594800000000001 2020-08-07T15:09:56.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:56.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227572+0000", 2020-08-07T15:09:56.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00086125700000000004 2020-08-07T15:09:56.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:56.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228433+0000", 2020-08-07T15:09:56.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3799999999999998e-07 2020-08-07T15:09:56.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:56.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228433+0000", 2020-08-07T15:09:56.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019413500000000001 2020-08-07T15:09:56.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:56.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228627+0000", 2020-08-07T15:09:56.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015968209999999 2020-08-07T15:09:56.881 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:56.881 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:56.882 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:56.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.882 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73084 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:56.883 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:09:56.885 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.582805365999999, 2020-08-07T15:09:56.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0016919049999999, 2020-08-07T15:09:56.885 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:56.885 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:56.885 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:56.896 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:56.896 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:56.896 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73084 2020-08-07T15:09:56.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.896 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:56.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:56.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:09:56.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:56.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:56.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:09:56.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5565999999999997e-05 2020-08-07T15:09:56.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:56.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227165+0000", 2020-08-07T15:09:56.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2020-08-07T15:09:56.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227169+0000", 2020-08-07T15:09:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6850000000000001e-06 2020-08-07T15:09:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227176+0000", 2020-08-07T15:09:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1949999999999999e-06 2020-08-07T15:09:56.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:56.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227179+0000", 2020-08-07T15:09:56.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0318000000000003e-05 2020-08-07T15:09:56.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227220+0000", 2020-08-07T15:09:56.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0349e-05 2020-08-07T15:09:56.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:56.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227230+0000", 2020-08-07T15:09:56.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98832962400000002 2020-08-07T15:09:56.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215560+0000", 2020-08-07T15:09:56.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013913099999999999 2020-08-07T15:09:56.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:56.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215699+0000", 2020-08-07T15:09:56.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000200006 2020-08-07T15:09:56.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:09:56.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215899+0000", 2020-08-07T15:09:56.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028673650000000002 2020-08-07T15:09:56.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:56.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218766+0000", 2020-08-07T15:09:56.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086101779999999996 2020-08-07T15:09:56.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:56.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227376+0000", 2020-08-07T15:09:56.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0999999999999998e-07 2020-08-07T15:09:56.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:56.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227377+0000", 2020-08-07T15:09:56.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6552999999999997e-05 2020-08-07T15:09:56.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:56.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227454+0000", 2020-08-07T15:09:56.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000421717 2020-08-07T15:09:56.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:56.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227875+0000", 2020-08-07T15:09:56.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00081781300000000003 2020-08-07T15:09:56.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:56.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228693+0000", 2020-08-07T15:09:56.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1699999999999997e-07 2020-08-07T15:09:56.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:56.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228694+0000", 2020-08-07T15:09:56.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117483 2020-08-07T15:09:56.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:56.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228811+0000", 2020-08-07T15:09:56.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0016919049999999 2020-08-07T15:09:56.977 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:56.979 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:56.979 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:56.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.980 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73085 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:56.980 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:09:56.980 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.582669550999999, 2020-08-07T15:09:56.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0018015629999999, 2020-08-07T15:09:56.981 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:56.981 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:56.981 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:56.982 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:56.982 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:56.982 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73085 2020-08-07T15:09:56.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.983 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:56.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:56.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:09:56.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:56.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:56.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:09:56.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6570999999999997e-05 2020-08-07T15:09:56.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:56.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227322+0000", 2020-08-07T15:09:56.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5509999999999996e-06 2020-08-07T15:09:56.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:56.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227328+0000", 2020-08-07T15:09:56.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2498e-05 2020-08-07T15:09:56.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:56.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227341+0000", 2020-08-07T15:09:56.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4389999999999999e-06 2020-08-07T15:09:56.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:56.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227346+0000", 2020-08-07T15:09:56.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3711999999999998e-05 2020-08-07T15:09:56.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227390+0000", 2020-08-07T15:09:56.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1868999999999999e-05 2020-08-07T15:09:56.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:56.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227402+0000", 2020-08-07T15:09:56.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98896525099999999 2020-08-07T15:09:56.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:56.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216367+0000", 2020-08-07T15:09:56.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012573 2020-08-07T15:09:56.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:56.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216493+0000", 2020-08-07T15:09:56.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020525200000000001 2020-08-07T15:09:56.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:56.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:56.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:09:56.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216698+0000", 2020-08-07T15:09:57.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037596470000000001 2020-08-07T15:09:57.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:57.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.220458+0000", 2020-08-07T15:09:57.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070227190000000002 2020-08-07T15:09:57.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:57.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227480+0000", 2020-08-07T15:09:57.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6600000000000003e-07 2020-08-07T15:09:57.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:57.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227481+0000", 2020-08-07T15:09:57.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9898000000000003e-05 2020-08-07T15:09:57.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:57.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227521+0000", 2020-08-07T15:09:57.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039130000000000002 2020-08-07T15:09:57.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:57.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227912+0000", 2020-08-07T15:09:57.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00095502399999999996 2020-08-07T15:09:57.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:57.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228867+0000", 2020-08-07T15:09:57.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2020-08-07T15:09:57.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:57.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228867+0000", 2020-08-07T15:09:57.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018933599999999999 2020-08-07T15:09:57.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:57.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229057+0000", 2020-08-07T15:09:57.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0018015629999999 2020-08-07T15:09:57.056 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:57.059 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:57.064 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:57.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.065 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73086 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:57.065 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:09:57.065 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.582537055, 2020-08-07T15:09:57.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002364923, 2020-08-07T15:09:57.066 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:57.087 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:57.087 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:57.087 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:57.087 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:57.088 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73086 2020-08-07T15:09:57.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.088 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:57.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:57.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:09:57.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:57.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:57.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:09:57.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000238426 2020-08-07T15:09:57.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:57.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227626+0000", 2020-08-07T15:09:57.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9609999999999999e-06 2020-08-07T15:09:57.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:57.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227633+0000", 2020-08-07T15:09:57.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2727e-05 2020-08-07T15:09:57.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:57.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227646+0000", 2020-08-07T15:09:57.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1739999999999997e-06 2020-08-07T15:09:57.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:57.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227652+0000", 2020-08-07T15:09:57.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5004000000000001e-05 2020-08-07T15:09:57.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:57.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227737+0000", 2020-08-07T15:09:57.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2744e-05 2020-08-07T15:09:57.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:57.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227750+0000", 2020-08-07T15:09:57.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98938776500000003 2020-08-07T15:09:57.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:57.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217138+0000", 2020-08-07T15:09:57.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012361499999999999 2020-08-07T15:09:57.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:57.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217261+0000", 2020-08-07T15:09:57.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000212504 2020-08-07T15:09:57.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:09:57.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217474+0000", 2020-08-07T15:09:57.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002995036 2020-08-07T15:09:57.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:57.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.220469+0000", 2020-08-07T15:09:57.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071283249999999996 2020-08-07T15:09:57.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227597+0000", 2020-08-07T15:09:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0499999999999999e-07 2020-08-07T15:09:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:57.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227597+0000", 2020-08-07T15:09:57.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1872999999999997e-05 2020-08-07T15:09:57.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227639+0000", 2020-08-07T15:09:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032010200000000001 2020-08-07T15:09:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227959+0000", 2020-08-07T15:09:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015980899999999999 2020-08-07T15:09:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229557+0000", 2020-08-07T15:09:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9299999999999999e-07 2020-08-07T15:09:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229558+0000", 2020-08-07T15:09:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019477900000000001 2020-08-07T15:09:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229753+0000", 2020-08-07T15:09:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002364923 2020-08-07T15:09:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73087 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:09:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.581595902, 2020-08-07T15:09:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001896506, 2020-08-07T15:09:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73087 2020-08-07T15:09:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:09:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:09:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7354e-05 2020-08-07T15:09:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228386+0000", 2020-08-07T15:09:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8000000000000001e-06 2020-08-07T15:09:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228393+0000", 2020-08-07T15:09:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3784999999999999e-05 2020-08-07T15:09:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228407+0000", 2020-08-07T15:09:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8090000000000004e-06 2020-08-07T15:09:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228414+0000", 2020-08-07T15:09:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6907000000000003e-05 2020-08-07T15:09:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:57.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228461+0000", 2020-08-07T15:09:57.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8929999999999994e-06 2020-08-07T15:09:57.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:57.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228469+0000", 2020-08-07T15:09:57.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98950057700000005 2020-08-07T15:09:57.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:57.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217970+0000", 2020-08-07T15:09:57.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000132182 2020-08-07T15:09:57.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:57.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218102+0000", 2020-08-07T15:09:57.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020073399999999999 2020-08-07T15:09:57.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:09:57.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218303+0000", 2020-08-07T15:09:57.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035951120000000001 2020-08-07T15:09:57.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:57.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.221898+0000", 2020-08-07T15:09:57.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057898660000000003 2020-08-07T15:09:57.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:57.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227688+0000", 2020-08-07T15:09:57.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0200000000000002e-07 2020-08-07T15:09:57.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:57.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227688+0000", 2020-08-07T15:09:57.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1477999999999996e-05 2020-08-07T15:09:57.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:57.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227760+0000", 2020-08-07T15:09:57.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016657480000000001 2020-08-07T15:09:57.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:57.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229426+0000", 2020-08-07T15:09:57.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061290400000000005 2020-08-07T15:09:57.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:57.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230039+0000", 2020-08-07T15:09:57.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2e-07 2020-08-07T15:09:57.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:57.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230039+0000", 2020-08-07T15:09:57.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018643500000000001 2020-08-07T15:09:57.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:57.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230225+0000", 2020-08-07T15:09:57.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001896506 2020-08-07T15:09:57.325 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:57.325 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:57.326 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:57.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.327 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73088 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:57.327 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:09:57.327 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.581205212, 2020-08-07T15:09:57.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001627153, 2020-08-07T15:09:57.337 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:57.338 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:57.338 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:57.338 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:57.338 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:57.339 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73088 2020-08-07T15:09:57.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.339 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:57.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:57.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:09:57.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:57.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:57.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:09:57.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7741000000000003e-05 2020-08-07T15:09:57.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:57.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228767+0000", 2020-08-07T15:09:57.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6409999999999999e-06 2020-08-07T15:09:57.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:57.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228773+0000", 2020-08-07T15:09:57.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0874e-05 2020-08-07T15:09:57.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:57.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228784+0000", 2020-08-07T15:09:57.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1529999999999996e-06 2020-08-07T15:09:57.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:57.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228789+0000", 2020-08-07T15:09:57.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3945e-05 2020-08-07T15:09:57.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:57.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228833+0000", 2020-08-07T15:09:57.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4020000000000002e-06 2020-08-07T15:09:57.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:57.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228841+0000", 2020-08-07T15:09:57.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98996913900000005 2020-08-07T15:09:57.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:57.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218810+0000", 2020-08-07T15:09:57.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112484 2020-08-07T15:09:57.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:57.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218923+0000", 2020-08-07T15:09:57.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019891 2020-08-07T15:09:57.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:09:57.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.219122+0000", 2020-08-07T15:09:57.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027884099999999998 2020-08-07T15:09:57.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:57.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.221910+0000", 2020-08-07T15:09:57.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058753399999999997 2020-08-07T15:09:57.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:57.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227786+0000", 2020-08-07T15:09:57.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9200000000000002e-07 2020-08-07T15:09:57.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:57.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227786+0000", 2020-08-07T15:09:57.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8572000000000003e-05 2020-08-07T15:09:57.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:57.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227824+0000", 2020-08-07T15:09:57.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001651296 2020-08-07T15:09:57.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:57.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229476+0000", 2020-08-07T15:09:57.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00077620399999999996 2020-08-07T15:09:57.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:57.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230252+0000", 2020-08-07T15:09:57.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1600000000000002e-07 2020-08-07T15:09:57.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:57.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230252+0000", 2020-08-07T15:09:57.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4433999999999997e-05 2020-08-07T15:09:57.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:57.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230347+0000", 2020-08-07T15:09:57.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001627153 2020-08-07T15:09:57.383 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:57.383 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:57.384 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:57.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.384 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73134 4.9 4:9a05b5c5:::benchmark_data_smithi074_44586_object9141:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:57.384 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.268819+0000", 2020-08-07T15:09:57.385 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.541105873999999, 2020-08-07T15:09:57.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.96356853099999995, 2020-08-07T15:09:57.385 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:57.385 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:57.385 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:57.386 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:57.386 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:57.386 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73134 2020-08-07T15:09:57.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.387 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:57.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:57.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268819+0000", 2020-08-07T15:09:57.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:57.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:57.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268819+0000", 2020-08-07T15:09:57.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8215999999999999e-05 2020-08-07T15:09:57.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:57.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268847+0000", 2020-08-07T15:09:57.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0309999999999999e-06 2020-08-07T15:09:57.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:57.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268850+0000", 2020-08-07T15:09:57.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.908e-06 2020-08-07T15:09:57.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:57.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268856+0000", 2020-08-07T15:09:57.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9000000000000002e-06 2020-08-07T15:09:57.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268859+0000", 2020-08-07T15:09:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6261000000000001e-05 2020-08-07T15:09:57.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:57.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268885+0000", 2020-08-07T15:09:57.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3600000000000004e-06 2020-08-07T15:09:57.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:57.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268891+0000", 2020-08-07T15:09:57.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95510594000000004 2020-08-07T15:09:57.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:57.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.223997+0000", 2020-08-07T15:09:57.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106016 2020-08-07T15:09:57.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:57.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.224103+0000", 2020-08-07T15:09:57.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000215558 2020-08-07T15:09:57.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:09:57.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.224318+0000", 2020-08-07T15:09:57.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001939014 2020-08-07T15:09:57.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:57.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226257+0000", 2020-08-07T15:09:57.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0047820340000000001 2020-08-07T15:09:57.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:57.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.231039+0000", 2020-08-07T15:09:57.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8999999999999996e-07 2020-08-07T15:09:57.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:57.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.231040+0000", 2020-08-07T15:09:57.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2513000000000001e-05 2020-08-07T15:09:57.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:57.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.231122+0000", 2020-08-07T15:09:57.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6319999999999997e-05 2020-08-07T15:09:57.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:57.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.231169+0000", 2020-08-07T15:09:57.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001027996 2020-08-07T15:09:57.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:57.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.232197+0000", 2020-08-07T15:09:57.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9700000000000002e-07 2020-08-07T15:09:57.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:57.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.232197+0000", 2020-08-07T15:09:57.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019047700000000001 2020-08-07T15:09:57.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:57.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.232387+0000", 2020-08-07T15:09:57.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.96356853099999995 2020-08-07T15:09:57.421 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:57.421 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:57.421 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73135 4.9 4:9a05b5c5:::benchmark_data_smithi074_44586_object9141:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.268948+0000", 2020-08-07T15:09:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.540976446999998, 2020-08-07T15:09:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.96367391999999996, 2020-08-07T15:09:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73135 2020-08-07T15:09:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.461 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:57.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:57.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268948+0000", 2020-08-07T15:09:57.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:57.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:57.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268948+0000", 2020-08-07T15:09:57.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1641000000000003e-05 2020-08-07T15:09:57.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:57.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268990+0000", 2020-08-07T15:09:57.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9349999999999999e-06 2020-08-07T15:09:57.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:57.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.268993+0000", 2020-08-07T15:09:57.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6440000000000003e-06 2020-08-07T15:09:57.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269000+0000", 2020-08-07T15:09:57.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1190000000000001e-06 2020-08-07T15:09:57.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:57.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269003+0000", 2020-08-07T15:09:57.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6940000000000002e-05 2020-08-07T15:09:57.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:57.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269040+0000", 2020-08-07T15:09:57.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0988999999999999e-05 2020-08-07T15:09:57.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:57.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269051+0000", 2020-08-07T15:09:57.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95570502400000001 2020-08-07T15:09:57.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:57.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.224756+0000", 2020-08-07T15:09:57.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7387999999999999e-05 2020-08-07T15:09:57.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:57.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.224823+0000", 2020-08-07T15:09:57.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000119159 2020-08-07T15:09:57.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:09:57.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.224942+0000", 2020-08-07T15:09:57.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015389780000000001 2020-08-07T15:09:57.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:57.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226481+0000", 2020-08-07T15:09:57.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0047295549999999999 2020-08-07T15:09:57.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:57.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.231211+0000", 2020-08-07T15:09:57.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3500000000000002e-07 2020-08-07T15:09:57.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:57.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.231211+0000", 2020-08-07T15:09:57.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5779000000000003e-05 2020-08-07T15:09:57.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:57.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.231257+0000", 2020-08-07T15:09:57.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050559900000000002 2020-08-07T15:09:57.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:57.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.231763+0000", 2020-08-07T15:09:57.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067623900000000003 2020-08-07T15:09:57.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:57.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.232439+0000", 2020-08-07T15:09:57.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2020-08-07T15:09:57.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:57.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.232439+0000", 2020-08-07T15:09:57.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018316599999999999 2020-08-07T15:09:57.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:57.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.232622+0000", 2020-08-07T15:09:57.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.96367391999999996 2020-08-07T15:09:57.728 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:57.729 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:57.729 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:57.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.729 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73136 4.9 4:9a05b5c5:::benchmark_data_smithi074_44586_object9141:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:09:57.730 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.269092+0000", 2020-08-07T15:09:57.730 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.540832661, 2020-08-07T15:09:57.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.96368705700000001, 2020-08-07T15:09:57.737 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:09:57.738 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:09:57.738 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:09:57.754 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:09:57.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:09:57.755 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73136 2020-08-07T15:09:57.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.755 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:09:57.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:09:57.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269092+0000", 2020-08-07T15:09:57.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:09:57.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:09:57.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269092+0000", 2020-08-07T15:09:57.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9740000000000002e-05 2020-08-07T15:09:57.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:09:57.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269132+0000", 2020-08-07T15:09:57.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.117e-06 2020-08-07T15:09:57.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:09:57.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269135+0000", 2020-08-07T15:09:57.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0129999999999999e-06 2020-08-07T15:09:57.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:09:57.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269141+0000", 2020-08-07T15:09:57.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1769999999999998e-06 2020-08-07T15:09:57.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:09:57.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269144+0000", 2020-08-07T15:09:57.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3272999999999999e-05 2020-08-07T15:09:57.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:57.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269167+0000", 2020-08-07T15:09:57.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8849999999999998e-06 2020-08-07T15:09:57.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:09:57.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.269172+0000", 2020-08-07T15:09:57.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95619174500000004 2020-08-07T15:09:57.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:09:57.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.225364+0000", 2020-08-07T15:09:57.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013498199999999999 2020-08-07T15:09:57.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:09:57.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.225499+0000", 2020-08-07T15:09:57.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020722500000000001 2020-08-07T15:09:57.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:09:57.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.225706+0000", 2020-08-07T15:09:57.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011191230000000001 2020-08-07T15:09:57.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:09:57.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226825+0000", 2020-08-07T15:09:57.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049631320000000003 2020-08-07T15:09:57.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:57.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.231789+0000", 2020-08-07T15:09:57.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6699999999999999e-07 2020-08-07T15:09:57.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:09:57.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.231789+0000", 2020-08-07T15:09:57.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2058999999999998e-05 2020-08-07T15:09:57.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:09:57.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.231831+0000", 2020-08-07T15:09:57.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031094400000000001 2020-08-07T15:09:57.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:57.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.232142+0000", 2020-08-07T15:09:57.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051961300000000004 2020-08-07T15:09:57.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:09:57.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.232661+0000", 2020-08-07T15:09:57.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2020-08-07T15:09:57.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:09:57.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.232662+0000", 2020-08-07T15:09:57.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117166 2020-08-07T15:09:57.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:09:57.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:09:57.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:09:57.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.232779+0000", 2020-08-07T15:09:57.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.96368705700000001 2020-08-07T15:09:57.811 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:57.811 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:57.811 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:57.811 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:09:57.811 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:09:57.812 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:09:57.812 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:09:57.819 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:09:57.819 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:09:57.819 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:09:57.820 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:09:57.820 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:09:57.820 INFO:teuthology.orchestra.run.smithi074.stdout: "epoch": 151, 2020-08-07T15:09:57.820 INFO:teuthology.orchestra.run.smithi074.stdout: "fsid": "a01d3e82-8d24-4288-9e42-576ab3ff4957", 2020-08-07T15:09:57.821 INFO:teuthology.orchestra.run.smithi074.stdout: "created": "2020-08-07T15:06:07.342912+0000", 2020-08-07T15:09:57.821 INFO:teuthology.orchestra.run.smithi074.stdout: "modified": "2020-08-07T15:09:53.183608+0000", 2020-08-07T15:09:57.821 INFO:teuthology.orchestra.run.smithi074.stdout: "last_up_change": "2020-08-07T15:06:17.674292+0000", 2020-08-07T15:09:57.821 INFO:teuthology.orchestra.run.smithi074.stdout: "last_in_change": "2020-08-07T15:06:17.674292+0000", 2020-08-07T15:09:57.821 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-08-07T15:09:57.822 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_num": 5799936, 2020-08-07T15:09:57.822 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_set": [ 2020-08-07T15:09:57.822 INFO:teuthology.orchestra.run.smithi074.stdout: "pglog_hardlimit", 2020-08-07T15:09:57.823 INFO:teuthology.orchestra.run.smithi074.stdout: "purged_snapdirs", 2020-08-07T15:09:57.823 INFO:teuthology.orchestra.run.smithi074.stdout: "recovery_deletes", 2020-08-07T15:09:57.823 INFO:teuthology.orchestra.run.smithi074.stdout: "sortbitwise" 2020-08-07T15:09:57.823 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:09:57.823 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_version": 5, 2020-08-07T15:09:57.823 INFO:teuthology.orchestra.run.smithi074.stdout: "full_ratio": 0.94999998807907104, 2020-08-07T15:09:57.824 INFO:teuthology.orchestra.run.smithi074.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-08-07T15:09:57.824 INFO:teuthology.orchestra.run.smithi074.stdout: "nearfull_ratio": 0.85000002384185791, 2020-08-07T15:09:57.824 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_snapshot": "", 2020-08-07T15:09:57.825 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_max": 4, 2020-08-07T15:09:57.825 INFO:teuthology.orchestra.run.smithi074.stdout: "max_osd": 8, 2020-08-07T15:09:57.826 INFO:teuthology.orchestra.run.smithi074.stdout: "require_min_compat_client": "luminous", 2020-08-07T15:09:57.826 INFO:teuthology.orchestra.run.smithi074.stdout: "min_compat_client": "luminous", 2020-08-07T15:09:57.826 INFO:teuthology.orchestra.run.smithi074.stdout: "require_osd_release": "pacific", 2020-08-07T15:09:57.826 INFO:teuthology.orchestra.run.smithi074.stdout: "pools": [ 2020-08-07T15:09:57.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:57.827 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 1, 2020-08-07T15:09:57.827 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "device_health_metrics", 2020-08-07T15:09:57.827 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:06:10.648876+0000", 2020-08-07T15:09:57.828 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:09:57.828 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:09:57.829 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:09:57.829 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:09:57.829 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:09:57.829 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:09:57.830 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:09:57.830 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:09:57.830 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 21, 2020-08-07T15:09:57.830 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 21, 2020-08-07T15:09:57.830 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 21, 2020-08-07T15:09:57.831 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 21, 2020-08-07T15:09:57.831 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 21, 2020-08-07T15:09:57.831 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:09:57.831 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:09:57.831 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:09:57.831 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:09:57.832 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:09:57.832 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:09:57.832 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:09:57.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:57.832 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "91", 2020-08-07T15:09:57.833 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:09:57.833 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:09:57.833 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "89", 2020-08-07T15:09:57.833 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:09:57.833 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:09:57.834 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:09:57.834 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:09:57.834 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:09:57.834 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:09:57.834 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:09:57.835 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:09:57.835 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:09:57.835 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:09:57.835 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:09:57.835 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:09:57.836 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:09:57.836 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:09:57.836 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:09:57.836 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:09:57.836 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:09:57.837 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:09:57.837 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:09:57.837 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:09:57.837 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:09:57.837 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:09:57.840 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:09:57.840 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:57.840 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:09:57.841 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:09:57.841 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:09:57.841 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:09:57.841 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:09:57.841 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:09:57.842 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:09:57.842 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:09:57.842 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:09:57.842 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:09:57.842 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:09:57.843 INFO:teuthology.orchestra.run.smithi074.stdout: "options": { 2020-08-07T15:09:57.843 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_min": 1 2020-08-07T15:09:57.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:57.843 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:09:57.843 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_devicehealth": {} 2020-08-07T15:09:57.844 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:57.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:57.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:57.844 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 2, 2020-08-07T15:09:57.844 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "rbd", 2020-08-07T15:09:57.845 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:06:21.474648+0000", 2020-08-07T15:09:57.856 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:09:57.857 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:09:57.857 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:09:57.857 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:09:57.857 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:09:57.857 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:09:57.858 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:09:57.858 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:09:57.858 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 8, 2020-08-07T15:09:57.858 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 8, 2020-08-07T15:09:57.858 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 8, 2020-08-07T15:09:57.860 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 8, 2020-08-07T15:09:57.864 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 8, 2020-08-07T15:09:57.864 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:09:57.864 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:09:57.865 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:09:57.865 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:09:57.865 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:09:57.865 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:09:57.865 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:09:57.866 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:57.866 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "19", 2020-08-07T15:09:57.866 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:09:57.866 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:09:57.866 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "0", 2020-08-07T15:09:57.867 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:09:57.867 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:09:57.867 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:09:57.867 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:09:57.867 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:09:57.868 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:09:57.868 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:09:57.868 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:09:57.868 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:09:57.868 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:09:57.869 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:09:57.869 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:09:57.869 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:09:57.870 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:09:57.870 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:09:57.870 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:09:57.870 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:09:57.870 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:09:57.871 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:09:57.871 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:09:57.871 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:09:57.871 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:09:57.871 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:09:57.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:57.872 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:09:57.872 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:09:57.872 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:09:57.872 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:09:57.873 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:09:57.873 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:09:57.910 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:09:57.911 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:09:57.911 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:09:57.911 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:09:57.911 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:09:57.912 INFO:teuthology.orchestra.run.smithi074.stdout: "options": {}, 2020-08-07T15:09:57.912 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:09:57.912 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd": {} 2020-08-07T15:09:57.912 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:57.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:57.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:57.913 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 4, 2020-08-07T15:09:57.913 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "unique_pool_1", 2020-08-07T15:09:57.913 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:08:22.604810+0000", 2020-08-07T15:09:57.913 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:09:57.929 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:09:57.930 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:09:57.930 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:09:57.930 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:09:57.931 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:09:57.931 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:09:57.931 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:09:57.931 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 16, 2020-08-07T15:09:57.931 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 16, 2020-08-07T15:09:57.932 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 16, 2020-08-07T15:09:57.932 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 16, 2020-08-07T15:09:57.932 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 16, 2020-08-07T15:09:57.932 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:09:57.934 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:09:57.934 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:09:57.934 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:09:57.934 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:09:57.935 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:09:57.935 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:09:57.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:57.935 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "140", 2020-08-07T15:09:57.935 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:09:57.936 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:09:57.936 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "0", 2020-08-07T15:09:57.936 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:09:57.936 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:09:57.936 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:09:57.937 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:09:57.937 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:09:57.937 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:09:57.937 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:09:57.937 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:09:57.938 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:09:57.938 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:09:57.938 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:09:57.938 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:09:57.938 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:09:57.938 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:09:57.939 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:09:57.939 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:09:57.940 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:09:57.940 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:09:57.940 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:09:57.940 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:09:57.941 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:09:57.941 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:09:57.941 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:09:57.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:57.942 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:09:57.943 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:09:57.943 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:09:57.944 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:09:57.944 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:09:57.944 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:09:57.945 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:09:57.946 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:09:57.947 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:09:57.947 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:09:57.947 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:09:57.947 INFO:teuthology.orchestra.run.smithi074.stdout: "options": {}, 2020-08-07T15:09:57.947 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:09:57.948 INFO:teuthology.orchestra.run.smithi074.stdout: "rados": {} 2020-08-07T15:09:57.948 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:57.948 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:57.948 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:09:57.948 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:09:57.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:57.949 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 0, 2020-08-07T15:09:57.949 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "fc1ebb23-3ef4-4fc0-b395-382e385885a3", 2020-08-07T15:09:57.949 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:09:57.956 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:09:57.963 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:09:57.963 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0, 2020-08-07T15:09:57.963 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:09:57.964 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:09:57.964 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:09:57.964 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 133, 2020-08-07T15:09:57.964 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:09:57.964 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:09:57.965 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:09:57.965 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:57.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:57.965 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:57.966 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6801", 2020-08-07T15:09:57.966 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:09:57.966 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:57.966 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:57.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:57.966 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:09:57.967 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:57.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:57.967 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:57.967 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6802", 2020-08-07T15:09:57.967 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:09:57.968 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:57.968 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:57.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:57.968 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:09:57.968 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:57.969 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:57.969 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:57.969 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6804", 2020-08-07T15:09:57.969 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:09:57.969 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:57.972 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:57.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:57.973 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:09:57.973 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:57.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:57.974 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:57.974 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6803", 2020-08-07T15:09:57.974 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:09:57.974 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:57.975 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:57.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:57.975 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6801/31577", 2020-08-07T15:09:57.976 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6802/31577", 2020-08-07T15:09:57.976 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6804/31577", 2020-08-07T15:09:57.976 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6803/31577", 2020-08-07T15:09:57.976 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:09:57.976 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:09:57.977 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:09:57.977 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:57.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:57.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:57.977 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 1, 2020-08-07T15:09:57.978 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "1fbe530a-63ce-4dc8-82ab-18793d6ebc0b", 2020-08-07T15:09:57.978 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:09:57.978 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:09:57.978 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:09:57.980 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0.7196044921875, 2020-08-07T15:09:57.980 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:09:57.980 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:09:57.981 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:09:57.981 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 134, 2020-08-07T15:09:57.981 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:09:57.981 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:09:57.981 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:09:57.982 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:57.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:57.982 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:57.982 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6809", 2020-08-07T15:09:57.982 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:09:57.982 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:57.983 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:57.983 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:57.983 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:09:57.984 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:57.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:57.985 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:57.985 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6810", 2020-08-07T15:09:57.985 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:09:57.985 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:57.986 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:57.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:57.986 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:09:57.986 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:57.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:57.987 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:57.987 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6812", 2020-08-07T15:09:57.987 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:09:57.987 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:57.987 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:57.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:57.988 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:09:57.988 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:57.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:57.988 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:57.989 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6811", 2020-08-07T15:09:58.014 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:09:58.015 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.018 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.018 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.018 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6809/31578", 2020-08-07T15:09:58.019 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6810/31578", 2020-08-07T15:09:58.019 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6812/31578", 2020-08-07T15:09:58.019 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6811/31578", 2020-08-07T15:09:58.031 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:09:58.031 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:09:58.031 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:09:58.032 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.032 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 2, 2020-08-07T15:09:58.032 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "d6fe4338-3e11-4f73-9cbb-e1b046ce286a", 2020-08-07T15:09:58.033 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:09:58.033 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:09:58.033 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:09:58.034 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:09:58.034 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:09:58.035 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:09:58.035 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:09:58.035 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 131, 2020-08-07T15:09:58.063 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:09:58.064 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:09:58.064 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:09:58.065 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:58.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.065 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:58.065 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6805", 2020-08-07T15:09:58.065 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:09:58.066 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.066 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.066 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.066 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:09:58.066 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:58.067 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.067 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:58.067 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6806", 2020-08-07T15:09:58.067 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:09:58.067 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.068 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.068 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:09:58.068 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:58.068 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.068 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:58.069 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6808", 2020-08-07T15:09:58.069 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:09:58.069 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.069 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.070 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.070 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:09:58.071 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:58.071 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.071 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:58.071 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6807", 2020-08-07T15:09:58.071 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:09:58.072 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.072 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.072 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.072 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6805/31576", 2020-08-07T15:09:58.072 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6806/31576", 2020-08-07T15:09:58.073 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6808/31576", 2020-08-07T15:09:58.073 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6807/31576", 2020-08-07T15:09:58.073 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:09:58.073 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:09:58.073 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:09:58.074 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.074 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.074 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.074 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 3, 2020-08-07T15:09:58.074 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9", 2020-08-07T15:09:58.074 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:09:58.075 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:09:58.075 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:09:58.075 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:09:58.075 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:09:58.075 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:09:58.076 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:09:58.076 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 134, 2020-08-07T15:09:58.076 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:09:58.076 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:09:58.076 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:09:58.077 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:58.077 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.077 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:58.077 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6813", 2020-08-07T15:09:58.078 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:09:58.078 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.078 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.078 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.078 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:09:58.079 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:58.079 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.079 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:58.079 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6814", 2020-08-07T15:09:58.079 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:09:58.080 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.080 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.080 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.080 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:09:58.080 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:58.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.086 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:58.086 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6816", 2020-08-07T15:09:58.086 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:09:58.086 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.087 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.087 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:09:58.087 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:58.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.088 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:58.088 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6815", 2020-08-07T15:09:58.088 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:09:58.088 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.088 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.089 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.089 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6813/31579", 2020-08-07T15:09:58.089 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6814/31579", 2020-08-07T15:09:58.100 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6816/31579", 2020-08-07T15:09:58.101 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6815/31579", 2020-08-07T15:09:58.101 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:09:58.101 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:09:58.103 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:09:58.104 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.105 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.105 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.105 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 4, 2020-08-07T15:09:58.105 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "01aadc13-29c5-4ddd-9bb9-de24b2b20412", 2020-08-07T15:09:58.105 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:09:58.106 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:09:58.106 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:09:58.106 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0, 2020-08-07T15:09:58.106 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:09:58.106 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:09:58.107 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:09:58.107 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 100, 2020-08-07T15:09:58.107 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:09:58.107 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:09:58.107 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:09:58.108 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:58.108 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.108 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:58.108 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6800", 2020-08-07T15:09:58.108 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:09:58.108 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.109 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.109 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.109 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:09:58.109 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:58.110 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.110 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:58.110 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6801", 2020-08-07T15:09:58.110 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:09:58.110 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.113 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.113 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.113 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:09:58.113 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:58.134 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.134 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:58.135 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6803", 2020-08-07T15:09:58.135 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:09:58.135 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.135 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.135 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.136 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:09:58.136 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:58.136 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.150 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:58.150 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6802", 2020-08-07T15:09:58.150 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:09:58.150 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.150 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.151 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6800/31545", 2020-08-07T15:09:58.151 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6801/31545", 2020-08-07T15:09:58.151 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6803/31545", 2020-08-07T15:09:58.151 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6802/31545", 2020-08-07T15:09:58.152 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:09:58.152 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:09:58.152 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:09:58.152 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.181 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.187 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 5, 2020-08-07T15:09:58.188 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "f5556a82-73b4-47b1-8e29-1d2fe031d8fd", 2020-08-07T15:09:58.188 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:09:58.188 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:09:58.188 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:09:58.200 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:09:58.201 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:09:58.201 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:09:58.201 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:09:58.202 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 134, 2020-08-07T15:09:58.202 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:09:58.202 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:09:58.204 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:09:58.205 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:58.205 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.205 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:58.205 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6804", 2020-08-07T15:09:58.205 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:09:58.206 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.206 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.206 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.206 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:09:58.206 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:58.207 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.207 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:58.207 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6805", 2020-08-07T15:09:58.207 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:09:58.207 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.208 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.208 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.208 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:09:58.208 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:58.208 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.209 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:58.209 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6807", 2020-08-07T15:09:58.209 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:09:58.214 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.214 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.215 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.215 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:09:58.215 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:58.215 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.215 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:58.216 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6806", 2020-08-07T15:09:58.216 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:09:58.216 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.216 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.216 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.272 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6804/31547", 2020-08-07T15:09:58.272 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6805/31547", 2020-08-07T15:09:58.272 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6807/31547", 2020-08-07T15:09:58.273 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6806/31547", 2020-08-07T15:09:58.273 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:09:58.275 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:09:58.275 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:09:58.278 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.278 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.279 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.279 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 6, 2020-08-07T15:09:58.281 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "9b98b337-e1cc-43cd-9e9b-f21ae3f20899", 2020-08-07T15:09:58.281 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:09:58.281 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:09:58.281 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:09:58.282 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:09:58.282 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:09:58.282 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:09:58.282 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 14, 2020-08-07T15:09:58.283 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 134, 2020-08-07T15:09:58.283 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:09:58.283 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:09:58.284 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:09:58.284 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:58.284 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.284 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:58.284 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6808", 2020-08-07T15:09:58.285 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:09:58.285 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.285 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.285 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.318 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:09:58.318 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:58.319 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.319 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:58.334 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6809", 2020-08-07T15:09:58.335 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:09:58.335 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.335 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.335 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.336 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:09:58.336 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:58.336 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.336 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:58.336 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6811", 2020-08-07T15:09:58.337 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:09:58.337 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.337 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.337 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.337 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:09:58.363 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:58.363 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.363 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:58.364 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6810", 2020-08-07T15:09:58.364 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:09:58.364 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.364 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.364 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.365 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6808/31546", 2020-08-07T15:09:58.365 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6809/31546", 2020-08-07T15:09:58.365 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6811/31546", 2020-08-07T15:09:58.365 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6810/31546", 2020-08-07T15:09:58.365 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:09:58.366 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:09:58.366 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:09:58.366 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.366 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.367 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 7, 2020-08-07T15:09:58.367 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "777662d8-bd34-48e8-83e1-24ce9e61f14b", 2020-08-07T15:09:58.367 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:09:58.367 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:09:58.367 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:09:58.368 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:09:58.368 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:09:58.368 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:09:58.368 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:09:58.368 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 130, 2020-08-07T15:09:58.368 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:09:58.369 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:09:58.369 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:09:58.369 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:58.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.386 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:58.387 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6812", 2020-08-07T15:09:58.387 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:09:58.387 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.387 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.387 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.388 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:09:58.388 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:58.388 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.388 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:58.388 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6813", 2020-08-07T15:09:58.388 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:09:58.389 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.389 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.389 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.389 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:09:58.390 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:58.390 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.391 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:58.391 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6815", 2020-08-07T15:09:58.391 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:09:58.391 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.391 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.392 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.392 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:09:58.392 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:09:58.392 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.393 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:09:58.393 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6814", 2020-08-07T15:09:58.393 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:09:58.394 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.394 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.394 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.395 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6812/31570", 2020-08-07T15:09:58.395 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6813/31570", 2020-08-07T15:09:58.395 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6815/31570", 2020-08-07T15:09:58.395 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6814/31570", 2020-08-07T15:09:58.395 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:09:58.396 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:09:58.396 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:09:58.396 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.396 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.396 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:09:58.396 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_xinfo": [ 2020-08-07T15:09:58.397 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.397 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 0, 2020-08-07T15:09:58.397 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:09:58.397 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:09:58.398 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:09:58.398 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:09:58.398 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:09:58.398 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:09:17.304053+0000", 2020-08-07T15:09:58.399 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:09:58.399 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.399 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 1, 2020-08-07T15:09:58.399 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:09:58.399 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:09:58.400 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:09:58.400 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:09:58.400 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:09:58.400 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:09:15.906155+0000", 2020-08-07T15:09:58.400 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:09:58.401 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.401 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.401 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 2, 2020-08-07T15:09:58.401 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:09:58.401 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:09:58.402 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:09:58.402 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:09:58.402 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:09:58.402 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:09:16.320055+0000", 2020-08-07T15:09:58.402 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:09:58.403 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.403 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.403 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 3, 2020-08-07T15:09:58.403 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:09:58.403 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:09:58.404 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:09:58.404 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:09:58.404 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:09:58.404 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:09:18.275504+0000", 2020-08-07T15:09:58.404 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:09:58.404 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.405 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 4, 2020-08-07T15:09:58.405 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:09:58.405 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:09:58.405 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:09:58.406 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:09:58.406 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:09:58.406 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:09:17.167699+0000", 2020-08-07T15:09:58.406 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:09:58.406 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.407 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.407 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 5, 2020-08-07T15:09:58.407 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:09:58.407 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:09:58.408 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:09:58.408 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:09:58.408 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:09:58.408 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:09:17.478690+0000", 2020-08-07T15:09:58.409 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:09:58.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.409 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.409 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 6, 2020-08-07T15:09:58.409 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:09:58.410 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:09:58.410 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:09:58.411 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:09:58.411 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:09:58.412 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:09:17.257278+0000", 2020-08-07T15:09:58.412 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:09:58.412 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.413 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 7, 2020-08-07T15:09:58.413 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:09:58.413 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:09:58.413 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:09:58.414 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:09:58.414 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:09:58.414 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:09:16.183897+0000", 2020-08-07T15:09:58.414 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:09:58.414 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.415 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:09:58.415 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_upmap": [], 2020-08-07T15:09:58.415 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_upmap_items": [ 2020-08-07T15:09:58.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.415 INFO:teuthology.orchestra.run.smithi074.stdout: "pgid": "4.9", 2020-08-07T15:09:58.416 INFO:teuthology.orchestra.run.smithi074.stdout: "mappings": [ 2020-08-07T15:09:58.416 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.416 INFO:teuthology.orchestra.run.smithi074.stdout: "from": 4, 2020-08-07T15:09:58.416 INFO:teuthology.orchestra.run.smithi074.stdout: "to": 5 2020-08-07T15:09:58.416 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.416 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.417 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.417 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:09:58.417 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_temp": [ 2020-08-07T15:09:58.417 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:09:58.417 INFO:teuthology.orchestra.run.smithi074.stdout: "pgid": "4.9", 2020-08-07T15:09:58.425 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:09:58.425 INFO:teuthology.orchestra.run.smithi074.stdout: 5, 2020-08-07T15:09:58.425 INFO:teuthology.orchestra.run.smithi074.stdout: 1, 2020-08-07T15:09:58.425 INFO:teuthology.orchestra.run.smithi074.stdout: 6 2020-08-07T15:09:58.426 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:09:58.426 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.426 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:09:58.426 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_temp": [], 2020-08-07T15:09:58.426 INFO:teuthology.orchestra.run.smithi074.stdout: "blacklist": {}, 2020-08-07T15:09:58.426 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profiles": { 2020-08-07T15:09:58.427 INFO:teuthology.orchestra.run.smithi074.stdout: "default": { 2020-08-07T15:09:58.427 INFO:teuthology.orchestra.run.smithi074.stdout: "crush-failure-domain": "osd", 2020-08-07T15:09:58.427 INFO:teuthology.orchestra.run.smithi074.stdout: "k": "2", 2020-08-07T15:09:58.427 INFO:teuthology.orchestra.run.smithi074.stdout: "m": "1", 2020-08-07T15:09:58.427 INFO:teuthology.orchestra.run.smithi074.stdout: "plugin": "jerasure", 2020-08-07T15:09:58.428 INFO:teuthology.orchestra.run.smithi074.stdout: "ruleset-failure-domain": "osd", 2020-08-07T15:09:58.428 INFO:teuthology.orchestra.run.smithi074.stdout: "technique": "reed_sol_van" 2020-08-07T15:09:58.428 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:09:58.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:09:58.428 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps_queue": [], 2020-08-07T15:09:58.429 INFO:teuthology.orchestra.run.smithi074.stdout: "new_removed_snaps": [], 2020-08-07T15:09:58.429 INFO:teuthology.orchestra.run.smithi074.stdout: "new_purged_snaps": [], 2020-08-07T15:09:58.429 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_node_flags": {}, 2020-08-07T15:09:58.429 INFO:teuthology.orchestra.run.smithi074.stdout: "device_class_flags": {} 2020-08-07T15:09:58.429 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:09:58.434 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:09:58.434 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:09:58.434 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:09:58.434 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:09:58.435 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:09:58.435 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:09:58.435 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:09:58.435 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:09:58.436 INFO:tasks.thrashosds.thrasher:j is {'epoch': 151, 'fsid': 'a01d3e82-8d24-4288-9e42-576ab3ff4957', 'created': '2020-08-07T15:06:07.342912+0000', 'modified': '2020-08-07T15:09:53.183608+0000', 'last_up_change': '2020-08-07T15:06:17.674292+0000', 'last_in_change': '2020-08-07T15:06:17.674292+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-08-07T15:06:10.648876+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 21, 'pg_placement_num': 21, 'pg_placement_num_target': 21, 'pg_num_target': 21, 'pg_num_pending': 21, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '89', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-08-07T15:06:21.474648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'unique_pool_1', 'create_time': '2020-08-07T15:08:22.604810+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': 'fc1ebb23-3ef4-4fc0-b395-382e385885a3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 133, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6801', 'nonce': 31577}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6802', 'nonce': 31577}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6804', 'nonce': 31577}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6803', 'nonce': 31577}]}, 'public_addr': '172.21.15.74:6801/31577', 'cluster_addr': '172.21.15.74:6802/31577', 'heartbeat_back_addr': '172.21.15.74:6804/31577', 'heartbeat_front_addr': '172.21.15.74:6803/31577', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '1fbe530a-63ce-4dc8-82ab-18793d6ebc0b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.7196044921875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 134, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6809', 'nonce': 31578}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6810', 'nonce': 31578}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6812', 'nonce': 31578}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6811', 'nonce': 31578}]}, 'public_addr': '172.21.15.74:6809/31578', 'cluster_addr': '172.21.15.74:6810/31578', 'heartbeat_back_addr': '172.21.15.74:6812/31578', 'heartbeat_front_addr': '172.21.15.74:6811/31578', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd6fe4338-3e11-4f73-9cbb-e1b046ce286a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 131, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6805', 'nonce': 31576}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6806', 'nonce': 31576}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6808', 'nonce': 31576}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6807', 'nonce': 31576}]}, 'public_addr': '172.21.15.74:6805/31576', 'cluster_addr': '172.21.15.74:6806/31576', 'heartbeat_back_addr': '172.21.15.74:6808/31576', 'heartbeat_front_addr': '172.21.15.74:6807/31576', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 134, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6813', 'nonce': 31579}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6814', 'nonce': 31579}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6816', 'nonce': 31579}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6815', 'nonce': 31579}]}, 'public_addr': '172.21.15.74:6813/31579', 'cluster_addr': '172.21.15.74:6814/31579', 'heartbeat_back_addr': '172.21.15.74:6816/31579', 'heartbeat_front_addr': '172.21.15.74:6815/31579', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '01aadc13-29c5-4ddd-9bb9-de24b2b20412', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 100, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6800', 'nonce': 31545}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6801', 'nonce': 31545}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6803', 'nonce': 31545}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6802', 'nonce': 31545}]}, 'public_addr': '172.21.15.96:6800/31545', 'cluster_addr': '172.21.15.96:6801/31545', 'heartbeat_back_addr': '172.21.15.96:6803/31545', 'heartbeat_front_addr': '172.21.15.96:6802/31545', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'f5556a82-73b4-47b1-8e29-1d2fe031d8fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 134, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6804', 'nonce': 31547}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6805', 'nonce': 31547}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6807', 'nonce': 31547}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6806', 'nonce': 31547}]}, 'public_addr': '172.21.15.96:6804/31547', 'cluster_addr': '172.21.15.96:6805/31547', 'heartbeat_back_addr': '172.21.15.96:6807/31547', 'heartbeat_front_addr': '172.21.15.96:6806/31547', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9b98b337-e1cc-43cd-9e9b-f21ae3f20899', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 134, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6808', 'nonce': 31546}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6809', 'nonce': 31546}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6811', 'nonce': 31546}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6810', 'nonce': 31546}]}, 'public_addr': '172.21.15.96:6808/31546', 'cluster_addr': '172.21.15.96:6809/31546', 'heartbeat_back_addr': '172.21.15.96:6811/31546', 'heartbeat_front_addr': '172.21.15.96:6810/31546', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '777662d8-bd34-48e8-83e1-24ce9e61f14b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 130, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6812', 'nonce': 31570}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6813', 'nonce': 31570}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6815', 'nonce': 31570}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6814', 'nonce': 31570}]}, 'public_addr': '172.21.15.96:6812/31570', 'cluster_addr': '172.21.15.96:6813/31570', 'heartbeat_back_addr': '172.21.15.96:6815/31570', 'heartbeat_front_addr': '172.21.15.96:6814/31570', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:09:17.304053+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:09:15.906155+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:09:16.320055+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:09:18.275504+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:09:17.167699+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:09:17.478690+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:09:17.257278+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:09:16.183897+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '4.9', 'mappings': [{'from': 4, 'to': 5}]}], 'pg_temp': [{'pgid': '4.9', 'osds': [5, 1, 6]}], 'primary_temp': [], 'blacklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}} 2020-08-07T15:09:58.437 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:09:58.438 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 90 8 116233 116225 10.0881 11.5234 0.0149614 0.0123864 2020-08-07T15:09:58.438 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Total time run: 90.6032 2020-08-07T15:09:58.438 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Total writes made: 116233 2020-08-07T15:09:58.439 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Write size: 8192 2020-08-07T15:09:58.439 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Object size: 65536 2020-08-07T15:09:58.439 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Bandwidth (MB/sec): 10.0225 2020-08-07T15:09:58.439 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Stddev Bandwidth: 4.32919 2020-08-07T15:09:58.439 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Max bandwidth (MB/sec): 14.3438 2020-08-07T15:09:58.440 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Min bandwidth (MB/sec): 0 2020-08-07T15:09:58.440 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Average IOPS: 1282 2020-08-07T15:09:58.440 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Stddev IOPS: 554.143 2020-08-07T15:09:58.440 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Max IOPS: 1836 2020-08-07T15:09:58.440 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Min IOPS: 0 2020-08-07T15:09:58.441 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Average Latency(s): 0.0124058 2020-08-07T15:09:58.441 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Stddev Latency(s): 0.0561149 2020-08-07T15:09:58.441 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Max latency(s): 2.05564 2020-08-07T15:09:58.441 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Min latency(s): 0.00103494 2020-08-07T15:09:58.442 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Cleaning up (deleting benchmark objects) 2020-08-07T15:09:58.442 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:09:58.443 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:57.814+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:58.445 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:58.443+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:58.546 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:58.545+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:58.654 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:58.655 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:58.655 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:09:58.645+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:09:58.671 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:09:58.759 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:58.758+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:58.899 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:58.871+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:58.931 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:58.931 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:58.933 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:09:58.973 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:09:58.974 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:09:58.975 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:09:58.973+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:09:59.004 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2020-08-07T15:09:58.653710+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":939663360,"num_objects":14341,"num_object_clones":0,"num_object_copies":43023,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":936,"num_objects_unfound":0,"num_objects_dirty":14341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116424,"num_write_kb":929864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2275,"num_bytes_recovered":148971520,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":135,"acting":135,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":11215980,"kb_used_data":2827180,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":743758740,"statfs":{"total":773094113280,"available":761608949760,"internally_reserved":8589934592,"allocated":2895032320,"data_stored":2891260552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,4,2,7,0,0,1,6],"upper_bound":512},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":102309888,"num_objects":1562,"num_object_clones":0,"num_object_copies":4686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":110,"num_objects_unfound":0,"num_objects_dirty":1562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14208,"num_write_kb":112136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.050721"},"pg_stats":[{"pgid":"1.14","version":"16'2","reported_seq":"80","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.610064+0000","last_change":"2020-08-07T15:08:42.533999+0000","last_active":"2020-08-07T15:09:51.610064+0000","last_peered":"2020-08-07T15:09:51.610064+0000","last_clean":"2020-08-07T15:09:51.610064+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:09:51.610064+0000","last_undegraded":"2020-08-07T15:09:51.610064+0000","last_fullsized":"2020-08-07T15:09:51.610064+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:42.533921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:42.533921+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"150","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.620691+0000","last_change":"2020-08-07T15:09:30.205031+0000","last_active":"2020-08-07T15:09:51.620691+0000","last_peered":"2020-08-07T15:09:51.620691+0000","last_clean":"2020-08-07T15:09:51.620691+0000","last_became_active":"2020-08-07T15:09:30.203592+0000","last_became_peered":"2020-08-07T15:09:30.203592+0000","last_unstale":"2020-08-07T15:09:51.620691+0000","last_undegraded":"2020-08-07T15:09:51.620691+0000","last_fullsized":"2020-08-07T15:09:51.620691+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.141010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.141010+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"81","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:53.191948+0000","last_change":"2020-08-07T15:08:55.078015+0000","last_active":"2020-08-07T15:09:53.191948+0000","last_peered":"2020-08-07T15:09:53.191948+0000","last_clean":"2020-08-07T15:09:53.191948+0000","last_became_active":"2020-08-07T15:08:41.685106+0000","last_became_peered":"2020-08-07T15:08:41.685106+0000","last_unstale":"2020-08-07T15:09:53.191948+0000","last_undegraded":"2020-08-07T15:09:53.191948+0000","last_fullsized":"2020-08-07T15:09:53.191948+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:55.077924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:55.077924+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"194","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:53.191910+0000","last_change":"2020-08-07T15:09:30.210669+0000","last_active":"2020-08-07T15:09:53.191910+0000","last_peered":"2020-08-07T15:09:53.191910+0000","last_clean":"2020-08-07T15:09:53.191910+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:09:53.191910+0000","last_undegraded":"2020-08-07T15:09:53.191910+0000","last_fullsized":"2020-08-07T15:09:53.191910+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.544324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.544324+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"188","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:53.190801+0000","last_change":"2020-08-07T15:08:51.599032+0000","last_active":"2020-08-07T15:09:53.190801+0000","last_peered":"2020-08-07T15:09:53.190801+0000","last_clean":"2020-08-07T15:09:53.190801+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:09:53.190801+0000","last_undegraded":"2020-08-07T15:09:53.190801+0000","last_fullsized":"2020-08-07T15:09:53.190801+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:51.598947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.598947+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"149","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:53.194208+0000","last_change":"2020-08-07T15:09:41.284321+0000","last_active":"2020-08-07T15:09:53.194208+0000","last_peered":"2020-08-07T15:09:53.194208+0000","last_clean":"2020-08-07T15:09:53.194208+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:09:53.194208+0000","last_undegraded":"2020-08-07T15:09:53.194208+0000","last_fullsized":"2020-08-07T15:09:53.194208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:41.284237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:41.284237+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"64","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.616306+0000","last_change":"2020-08-07T15:09:30.283753+0000","last_active":"2020-08-07T15:09:51.616306+0000","last_peered":"2020-08-07T15:09:51.616306+0000","last_clean":"2020-08-07T15:09:51.616306+0000","last_became_active":"2020-08-07T15:09:30.283176+0000","last_became_peered":"2020-08-07T15:09:30.283176+0000","last_unstale":"2020-08-07T15:09:51.616306+0000","last_undegraded":"2020-08-07T15:09:51.616306+0000","last_fullsized":"2020-08-07T15:09:51.616306+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:25.606757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:25.606757+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"151'7024","reported_seq":"7106","reported_epoch":"152","state":"active+clean","last_fresh":"2020-08-07T15:09:56.644185+0000","last_change":"2020-08-07T15:09:52.137739+0000","last_active":"2020-08-07T15:09:56.644185+0000","last_peered":"2020-08-07T15:09:56.644185+0000","last_clean":"2020-08-07T15:09:56.644185+0000","last_became_active":"2020-08-07T15:08:46.708112+0000","last_became_peered":"2020-08-07T15:08:46.708112+0000","last_unstale":"2020-08-07T15:09:56.644185+0000","last_undegraded":"2020-08-07T15:09:56.644185+0000","last_fullsized":"2020-08-07T15:09:56.644185+0000","mapping_epoch":98,"log_start":"151'7016","ondisk_log_start":"151'7016","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'6643","last_scrub_stamp":"2020-08-07T15:09:52.137671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:52.137671+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57540608,"num_objects":878,"num_object_clones":0,"num_object_copies":2634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7024,"num_write_kb":56192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"148","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.620862+0000","last_change":"2020-08-07T15:08:40.057705+0000","last_active":"2020-08-07T15:09:51.620862+0000","last_peered":"2020-08-07T15:09:51.620862+0000","last_clean":"2020-08-07T15:09:51.620862+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:09:51.620862+0000","last_undegraded":"2020-08-07T15:09:51.620862+0000","last_fullsized":"2020-08-07T15:09:51.620862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:40.057622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:40.057622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"105","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.610172+0000","last_change":"2020-08-07T15:08:53.756554+0000","last_active":"2020-08-07T15:09:51.610172+0000","last_peered":"2020-08-07T15:09:51.610172+0000","last_clean":"2020-08-07T15:09:51.610172+0000","last_became_active":"2020-08-07T15:08:46.710404+0000","last_became_peered":"2020-08-07T15:08:46.710404+0000","last_unstale":"2020-08-07T15:09:51.610172+0000","last_undegraded":"2020-08-07T15:09:51.610172+0000","last_fullsized":"2020-08-07T15:09:51.610172+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:53.756343+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:53.756343+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"151'7696","reported_seq":"9543","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:56.275758+0000","last_change":"2020-08-07T15:09:35.835839+0000","last_active":"2020-08-07T15:09:56.275758+0000","last_peered":"2020-08-07T15:09:56.275758+0000","last_clean":"2020-08-07T15:09:56.275758+0000","last_became_active":"2020-08-07T15:09:27.180158+0000","last_became_peered":"2020-08-07T15:09:27.180158+0000","last_unstale":"2020-08-07T15:09:56.275758+0000","last_undegraded":"2020-08-07T15:09:56.275758+0000","last_fullsized":"2020-08-07T15:09:56.275758+0000","mapping_epoch":131,"log_start":"151'7690","ondisk_log_start":"151'7690","created":81,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'5536","last_scrub_stamp":"2020-08-07T15:09:35.835763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:35.835763+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63045632,"num_objects":962,"num_object_clones":0,"num_object_copies":2886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7696,"num_write_kb":61568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":875,"num_bytes_recovered":57344000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"148","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:54.908198+0000","last_change":"2020-08-07T15:09:54.908283+0000","last_active":"2020-08-07T15:09:54.908198+0000","last_peered":"2020-08-07T15:09:54.908198+0000","last_clean":"2020-08-07T15:09:54.908198+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:09:54.908198+0000","last_undegraded":"2020-08-07T15:09:54.908198+0000","last_fullsized":"2020-08-07T15:09:54.908198+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:54.908197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:54.908197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"185","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:53.194349+0000","last_change":"2020-08-07T15:09:30.202363+0000","last_active":"2020-08-07T15:09:53.194349+0000","last_peered":"2020-08-07T15:09:53.194349+0000","last_clean":"2020-08-07T15:09:53.194349+0000","last_became_active":"2020-08-07T15:09:30.201829+0000","last_became_peered":"2020-08-07T15:09:30.201829+0000","last_unstale":"2020-08-07T15:09:53.194349+0000","last_undegraded":"2020-08-07T15:09:53.194349+0000","last_fullsized":"2020-08-07T15:09:53.194349+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.899280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.899280+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"151'7065","reported_seq":"7147","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:56.278176+0000","last_change":"2020-08-07T15:09:53.825334+0000","last_active":"2020-08-07T15:09:56.278176+0000","last_peered":"2020-08-07T15:09:56.278176+0000","last_clean":"2020-08-07T15:09:56.278176+0000","last_became_active":"2020-08-07T15:08:24.549245+0000","last_became_peered":"2020-08-07T15:08:24.549245+0000","last_unstale":"2020-08-07T15:09:56.278176+0000","last_undegraded":"2020-08-07T15:09:56.278176+0000","last_fullsized":"2020-08-07T15:09:56.278176+0000","mapping_epoch":81,"log_start":"151'7062","ondisk_log_start":"151'7062","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'6872","last_scrub_stamp":"2020-08-07T15:09:53.825253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:53.825253+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57876480,"num_objects":884,"num_object_clones":0,"num_object_copies":2652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7065,"num_write_kb":56520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"148","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.613554+0000","last_change":"2020-08-07T15:09:51.036281+0000","last_active":"2020-08-07T15:09:51.613554+0000","last_peered":"2020-08-07T15:09:51.613554+0000","last_clean":"2020-08-07T15:09:51.613554+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:09:51.613554+0000","last_undegraded":"2020-08-07T15:09:51.613554+0000","last_fullsized":"2020-08-07T15:09:51.613554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:51.036224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:51.036224+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"151","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:53.193880+0000","last_change":"2020-08-07T15:08:50.686578+0000","last_active":"2020-08-07T15:09:53.193880+0000","last_peered":"2020-08-07T15:09:53.193880+0000","last_clean":"2020-08-07T15:09:53.193880+0000","last_became_active":"2020-08-07T15:08:36.633406+0000","last_became_peered":"2020-08-07T15:08:36.633406+0000","last_unstale":"2020-08-07T15:09:53.193880+0000","last_undegraded":"2020-08-07T15:09:53.193880+0000","last_fullsized":"2020-08-07T15:09:53.193880+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:50.686496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:50.686496+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"151'6944","reported_seq":"7022","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:56.236534+0000","last_change":"2020-08-07T15:08:24.557398+0000","last_active":"2020-08-07T15:09:56.236534+0000","last_peered":"2020-08-07T15:09:56.236534+0000","last_clean":"2020-08-07T15:09:56.236534+0000","last_became_active":"2020-08-07T15:08:24.557022+0000","last_became_peered":"2020-08-07T15:08:24.557022+0000","last_unstale":"2020-08-07T15:09:56.236534+0000","last_undegraded":"2020-08-07T15:09:56.236534+0000","last_fullsized":"2020-08-07T15:09:56.236534+0000","mapping_epoch":81,"log_start":"151'6938","ondisk_log_start":"151'6938","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56885248,"num_objects":868,"num_object_clones":0,"num_object_copies":2604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6944,"num_write_kb":55552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"180","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:55.879786+0000","last_change":"2020-08-07T15:09:55.879867+0000","last_active":"2020-08-07T15:09:55.879786+0000","last_peered":"2020-08-07T15:09:55.879786+0000","last_clean":"2020-08-07T15:09:55.879786+0000","last_became_active":"2020-08-07T15:08:36.627426+0000","last_became_peered":"2020-08-07T15:08:36.627426+0000","last_unstale":"2020-08-07T15:09:55.879786+0000","last_undegraded":"2020-08-07T15:09:55.879786+0000","last_fullsized":"2020-08-07T15:09:55.879786+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:55.879785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:55.879785+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"146","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:53.192392+0000","last_change":"2020-08-07T15:08:26.317430+0000","last_active":"2020-08-07T15:09:53.192392+0000","last_peered":"2020-08-07T15:09:53.192392+0000","last_clean":"2020-08-07T15:09:53.192392+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:09:53.192392+0000","last_undegraded":"2020-08-07T15:09:53.192392+0000","last_fullsized":"2020-08-07T15:09:53.192392+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:26.317351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:26.317351+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"152'7279","reported_seq":"7357","reported_epoch":"152","state":"active+clean","last_fresh":"2020-08-07T15:09:57.715957+0000","last_change":"2020-08-07T15:08:24.552570+0000","last_active":"2020-08-07T15:09:57.715957+0000","last_peered":"2020-08-07T15:09:57.715957+0000","last_clean":"2020-08-07T15:09:57.715957+0000","last_became_active":"2020-08-07T15:08:24.552105+0000","last_became_peered":"2020-08-07T15:08:24.552105+0000","last_unstale":"2020-08-07T15:09:57.715957+0000","last_undegraded":"2020-08-07T15:09:57.715957+0000","last_fullsized":"2020-08-07T15:09:57.715957+0000","mapping_epoch":81,"log_start":"152'7276","ondisk_log_start":"152'7276","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":54984704,"num_objects":839,"num_object_clones":0,"num_object_copies":2517,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7279,"num_write_kb":57728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"166","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.610016+0000","last_change":"2020-08-07T15:09:27.181676+0000","last_active":"2020-08-07T15:09:51.610016+0000","last_peered":"2020-08-07T15:09:51.610016+0000","last_clean":"2020-08-07T15:09:51.610016+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:09:51.610016+0000","last_undegraded":"2020-08-07T15:09:51.610016+0000","last_fullsized":"2020-08-07T15:09:51.610016+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:05.606603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:05.606603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"193","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.613810+0000","last_change":"2020-08-07T15:08:42.069637+0000","last_active":"2020-08-07T15:09:51.613810+0000","last_peered":"2020-08-07T15:09:51.613810+0000","last_clean":"2020-08-07T15:09:51.613810+0000","last_became_active":"2020-08-07T15:08:36.628365+0000","last_became_peered":"2020-08-07T15:08:36.628365+0000","last_unstale":"2020-08-07T15:09:51.613810+0000","last_undegraded":"2020-08-07T15:09:51.613810+0000","last_fullsized":"2020-08-07T15:09:51.613810+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:42.069585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:42.069585+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"151'7312","reported_seq":"7393","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:56.179777+0000","last_change":"2020-08-07T15:09:37.517415+0000","last_active":"2020-08-07T15:09:56.179777+0000","last_peered":"2020-08-07T15:09:56.179777+0000","last_clean":"2020-08-07T15:09:56.179777+0000","last_became_active":"2020-08-07T15:08:24.550222+0000","last_became_peered":"2020-08-07T15:08:24.550222+0000","last_unstale":"2020-08-07T15:09:56.179777+0000","last_undegraded":"2020-08-07T15:09:56.179777+0000","last_fullsized":"2020-08-07T15:09:56.179777+0000","mapping_epoch":81,"log_start":"151'7305","ondisk_log_start":"151'7305","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"141'5600","last_scrub_stamp":"2020-08-07T15:09:37.517334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:37.517334+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":59899904,"num_objects":914,"num_object_clones":0,"num_object_copies":2742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7312,"num_write_kb":58496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"149","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:53.192360+0000","last_change":"2020-08-07T15:09:52.637867+0000","last_active":"2020-08-07T15:09:53.192360+0000","last_peered":"2020-08-07T15:09:53.192360+0000","last_clean":"2020-08-07T15:09:53.192360+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:09:53.192360+0000","last_undegraded":"2020-08-07T15:09:53.192360+0000","last_fullsized":"2020-08-07T15:09:53.192360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:52.637791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:09:52.637791+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"144","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:53.192032+0000","last_change":"2020-08-07T15:08:38.386792+0000","last_active":"2020-08-07T15:09:53.192032+0000","last_peered":"2020-08-07T15:09:53.192032+0000","last_clean":"2020-08-07T15:09:53.192032+0000","last_became_active":"2020-08-07T15:08:36.618164+0000","last_became_peered":"2020-08-07T15:08:36.618164+0000","last_unstale":"2020-08-07T15:09:53.192032+0000","last_undegraded":"2020-08-07T15:09:53.192032+0000","last_fullsized":"2020-08-07T15:09:53.192032+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:38.386739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:38.386739+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"151'7120","reported_seq":"8296","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:56.150832+0000","last_change":"2020-08-07T15:09:36.911393+0000","last_active":"2020-08-07T15:09:56.150832+0000","last_peered":"2020-08-07T15:09:56.150832+0000","last_clean":"2020-08-07T15:09:56.150832+0000","last_became_active":"2020-08-07T15:09:27.183115+0000","last_became_peered":"2020-08-07T15:09:27.183115+0000","last_unstale":"2020-08-07T15:09:56.150832+0000","last_undegraded":"2020-08-07T15:09:56.150832+0000","last_fullsized":"2020-08-07T15:09:56.150832+0000","mapping_epoch":131,"log_start":"151'7115","ondisk_log_start":"151'7115","created":81,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'5280","last_scrub_stamp":"2020-08-07T15:09:36.911321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:36.911321+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":58327040,"num_objects":890,"num_object_clones":0,"num_object_copies":2670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7120,"num_write_kb":56960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":536,"num_bytes_recovered":35127296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"145","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.613748+0000","last_change":"2020-08-07T15:09:37.017383+0000","last_active":"2020-08-07T15:09:51.613748+0000","last_peered":"2020-08-07T15:09:51.613748+0000","last_clean":"2020-08-07T15:09:51.613748+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:09:51.613748+0000","last_undegraded":"2020-08-07T15:09:51.613748+0000","last_fullsized":"2020-08-07T15:09:51.613748+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:37.017307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:37.017307+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"219","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:53.194315+0000","last_change":"2020-08-07T15:09:30.208873+0000","last_active":"2020-08-07T15:09:53.194315+0000","last_peered":"2020-08-07T15:09:53.194315+0000","last_clean":"2020-08-07T15:09:53.194315+0000","last_became_active":"2020-08-07T15:09:30.208345+0000","last_became_peered":"2020-08-07T15:09:30.208345+0000","last_unstale":"2020-08-07T15:09:53.194315+0000","last_undegraded":"2020-08-07T15:09:53.194315+0000","last_fullsized":"2020-08-07T15:09:53.194315+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.381325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.381325+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"151'7232","reported_seq":"7313","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:56.278797+0000","last_change":"2020-08-07T15:09:54.750369+0000","last_active":"2020-08-07T15:09:56.278797+0000","last_peered":"2020-08-07T15:09:56.278797+0000","last_clean":"2020-08-07T15:09:56.278797+0000","last_became_active":"2020-08-07T15:08:24.552070+0000","last_became_peered":"2020-08-07T15:08:24.552070+0000","last_unstale":"2020-08-07T15:09:56.278797+0000","last_undegraded":"2020-08-07T15:09:56.278797+0000","last_fullsized":"2020-08-07T15:09:56.278797+0000","mapping_epoch":81,"log_start":"151'7226","ondisk_log_start":"151'7226","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"151'7048","last_scrub_stamp":"2020-08-07T15:09:54.750287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:54.750287+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":59244544,"num_objects":904,"num_object_clones":0,"num_object_copies":2712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7232,"num_write_kb":57856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"121","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:53.192352+0000","last_change":"2020-08-07T15:08:39.409159+0000","last_active":"2020-08-07T15:09:53.192352+0000","last_peered":"2020-08-07T15:09:53.192352+0000","last_clean":"2020-08-07T15:09:53.192352+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:09:53.192352+0000","last_undegraded":"2020-08-07T15:09:53.192352+0000","last_fullsized":"2020-08-07T15:09:53.192352+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:39.409063+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:07:36.143353+0000","last_clean_scrub_stamp":"2020-08-07T15:08:39.409063+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"151'7192","reported_seq":"7273","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:56.105924+0000","last_change":"2020-08-07T15:09:40.697977+0000","last_active":"2020-08-07T15:09:56.105924+0000","last_peered":"2020-08-07T15:09:56.105924+0000","last_clean":"2020-08-07T15:09:56.105924+0000","last_became_active":"2020-08-07T15:08:24.553668+0000","last_became_peered":"2020-08-07T15:08:24.553668+0000","last_unstale":"2020-08-07T15:09:56.105924+0000","last_undegraded":"2020-08-07T15:09:56.105924+0000","last_fullsized":"2020-08-07T15:09:56.105924+0000","mapping_epoch":81,"log_start":"151'7186","ondisk_log_start":"151'7186","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"142'5792","last_scrub_stamp":"2020-08-07T15:09:40.697906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:40.697906+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":58916864,"num_objects":899,"num_object_clones":0,"num_object_copies":2697,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7192,"num_write_kb":57536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"121","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:53.190940+0000","last_change":"2020-08-07T15:09:09.691838+0000","last_active":"2020-08-07T15:09:53.190940+0000","last_peered":"2020-08-07T15:09:53.190940+0000","last_clean":"2020-08-07T15:09:53.190940+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:09:53.190940+0000","last_undegraded":"2020-08-07T15:09:53.190940+0000","last_fullsized":"2020-08-07T15:09:53.190940+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.691753+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"151'7184","reported_seq":"7265","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:56.206137+0000","last_change":"2020-08-07T15:09:42.152452+0000","last_active":"2020-08-07T15:09:56.206137+0000","last_peered":"2020-08-07T15:09:56.206137+0000","last_clean":"2020-08-07T15:09:56.206137+0000","last_became_active":"2020-08-07T15:08:46.708674+0000","last_became_peered":"2020-08-07T15:08:46.708674+0000","last_unstale":"2020-08-07T15:09:56.206137+0000","last_undegraded":"2020-08-07T15:09:56.206137+0000","last_fullsized":"2020-08-07T15:09:56.206137+0000","mapping_epoch":98,"log_start":"151'7179","ondisk_log_start":"151'7179","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"142'5792","last_scrub_stamp":"2020-08-07T15:09:42.152379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:42.152379+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":58851328,"num_objects":898,"num_object_clones":0,"num_object_copies":2694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7184,"num_write_kb":57472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"184","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.613350+0000","last_change":"2020-08-07T15:09:25.984108+0000","last_active":"2020-08-07T15:09:51.613350+0000","last_peered":"2020-08-07T15:09:51.613350+0000","last_clean":"2020-08-07T15:09:51.613350+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:09:51.613350+0000","last_undegraded":"2020-08-07T15:09:51.613350+0000","last_fullsized":"2020-08-07T15:09:51.613350+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:25.984018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:25.984018+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"151'7320","reported_seq":"7423","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:56.083499+0000","last_change":"2020-08-07T15:09:38.739730+0000","last_active":"2020-08-07T15:09:56.083499+0000","last_peered":"2020-08-07T15:09:56.083499+0000","last_clean":"2020-08-07T15:09:56.083499+0000","last_became_active":"2020-08-07T15:08:52.166336+0000","last_became_peered":"2020-08-07T15:08:52.166336+0000","last_unstale":"2020-08-07T15:09:56.083499+0000","last_undegraded":"2020-08-07T15:09:56.083499+0000","last_fullsized":"2020-08-07T15:09:56.083499+0000","mapping_epoch":104,"log_start":"151'7312","ondisk_log_start":"151'7312","created":81,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"141'5712","last_scrub_stamp":"2020-08-07T15:09:38.739630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:38.739630+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":59965440,"num_objects":915,"num_object_clones":0,"num_object_copies":2745,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7320,"num_write_kb":58560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":40960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"139","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.613514+0000","last_change":"2020-08-07T15:08:51.513398+0000","last_active":"2020-08-07T15:09:51.613514+0000","last_peered":"2020-08-07T15:09:51.613514+0000","last_clean":"2020-08-07T15:09:51.613514+0000","last_became_active":"2020-08-07T15:08:41.680448+0000","last_became_peered":"2020-08-07T15:08:41.680448+0000","last_unstale":"2020-08-07T15:09:51.613514+0000","last_undegraded":"2020-08-07T15:09:51.613514+0000","last_fullsized":"2020-08-07T15:09:51.613514+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:51.513319+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.513319+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"151'7104","reported_seq":"7186","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:56.259041+0000","last_change":"2020-08-07T15:09:50.869480+0000","last_active":"2020-08-07T15:09:56.259041+0000","last_peered":"2020-08-07T15:09:56.259041+0000","last_clean":"2020-08-07T15:09:56.259041+0000","last_became_active":"2020-08-07T15:08:24.549868+0000","last_became_peered":"2020-08-07T15:08:24.549868+0000","last_unstale":"2020-08-07T15:09:56.259041+0000","last_undegraded":"2020-08-07T15:09:56.259041+0000","last_fullsized":"2020-08-07T15:09:56.259041+0000","mapping_epoch":81,"log_start":"151'7098","ondisk_log_start":"151'7098","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"149'6599","last_scrub_stamp":"2020-08-07T15:09:50.869400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:50.869400+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":58195968,"num_objects":888,"num_object_clones":0,"num_object_copies":2664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7104,"num_write_kb":56832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"110","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.616417+0000","last_change":"2020-08-07T15:09:36.687393+0000","last_active":"2020-08-07T15:09:51.616417+0000","last_peered":"2020-08-07T15:09:51.616417+0000","last_clean":"2020-08-07T15:09:51.616417+0000","last_became_active":"2020-08-07T15:09:30.204655+0000","last_became_peered":"2020-08-07T15:09:30.204655+0000","last_unstale":"2020-08-07T15:09:51.616417+0000","last_undegraded":"2020-08-07T15:09:51.616417+0000","last_fullsized":"2020-08-07T15:09:51.616417+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:36.687314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:36.687314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"151'7488","reported_seq":"9347","reported_epoch":"151","state":"active+remapped+backfill_toofull","last_fresh":"2020-08-07T15:09:56.268873+0000","last_change":"2020-08-07T15:09:28.235048+0000","last_active":"2020-08-07T15:09:56.268873+0000","last_peered":"2020-08-07T15:09:56.268873+0000","last_clean":"2020-08-07T15:09:25.573158+0000","last_became_active":"2020-08-07T15:09:28.212799+0000","last_became_peered":"2020-08-07T15:09:28.212799+0000","last_unstale":"2020-08-07T15:09:56.268873+0000","last_undegraded":"2020-08-07T15:09:56.268873+0000","last_fullsized":"2020-08-07T15:09:56.268873+0000","mapping_epoch":132,"log_start":"151'7482","ondisk_log_start":"151'7482","created":81,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61341696,"num_objects":936,"num_object_clones":0,"num_object_copies":2808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":936,"num_objects_unfound":0,"num_objects_dirty":936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7488,"num_write_kb":59904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":862,"num_bytes_recovered":56459264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,6],"avail_no_missing":["5","1","6"],"object_location_counts":[{"shards":"1,5,6","objects":936}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"184","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:53.194354+0000","last_change":"2020-08-07T15:09:08.399507+0000","last_active":"2020-08-07T15:09:53.194354+0000","last_peered":"2020-08-07T15:09:53.194354+0000","last_clean":"2020-08-07T15:09:53.194354+0000","last_became_active":"2020-08-07T15:08:41.689435+0000","last_became_peered":"2020-08-07T15:08:41.689435+0000","last_unstale":"2020-08-07T15:09:53.194354+0000","last_undegraded":"2020-08-07T15:09:53.194354+0000","last_fullsized":"2020-08-07T15:09:53.194354+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:08.399439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:08.399439+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"152'7716","reported_seq":"7795","reported_epoch":"152","state":"active+clean","last_fresh":"2020-08-07T15:09:57.723290+0000","last_change":"2020-08-07T15:09:52.486813+0000","last_active":"2020-08-07T15:09:57.723290+0000","last_peered":"2020-08-07T15:09:57.723290+0000","last_clean":"2020-08-07T15:09:57.723290+0000","last_became_active":"2020-08-07T15:08:24.554021+0000","last_became_peered":"2020-08-07T15:08:24.554021+0000","last_unstale":"2020-08-07T15:09:57.723290+0000","last_undegraded":"2020-08-07T15:09:57.723290+0000","last_fullsized":"2020-08-07T15:09:57.723290+0000","mapping_epoch":81,"log_start":"152'7712","ondisk_log_start":"152'7712","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'7328","last_scrub_stamp":"2020-08-07T15:09:52.486757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:52.486757+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":58195968,"num_objects":888,"num_object_clones":0,"num_object_copies":2664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7716,"num_write_kb":61184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"70","reported_epoch":"150","state":"active+clean","last_fresh":"2020-08-07T15:09:51.613470+0000","last_change":"2020-08-07T15:09:09.868283+0000","last_active":"2020-08-07T15:09:51.613470+0000","last_peered":"2020-08-07T15:09:51.613470+0000","last_clean":"2020-08-07T15:09:51.613470+0000","last_became_active":"2020-08-07T15:08:46.710059+0000","last_became_peered":"2020-08-07T15:08:46.710059+0000","last_unstale":"2020-08-07T15:09:51.613470+0000","last_undegraded":"2020-08-07T15:09:51.613470+0000","last_fullsized":"2020-08-07T15:09:51.613470+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.868198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.868198+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"151'7512","reported_seq":"7592","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:56.220073+0000","last_change":"2020-08-07T15:09:26.248248+0000","last_active":"2020-08-07T15:09:56.220073+0000","last_peered":"2020-08-07T15:09:56.220073+0000","last_clean":"2020-08-07T15:09:56.220073+0000","last_became_active":"2020-08-07T15:08:24.552127+0000","last_became_peered":"2020-08-07T15:08:24.552127+0000","last_unstale":"2020-08-07T15:09:56.220073+0000","last_undegraded":"2020-08-07T15:09:56.220073+0000","last_fullsized":"2020-08-07T15:09:56.220073+0000","mapping_epoch":81,"log_start":"151'7505","ondisk_log_start":"151'7505","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"130'4592","last_scrub_stamp":"2020-08-07T15:09:26.248174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:09:26.248174+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61538304,"num_objects":939,"num_object_clones":0,"num_object_copies":2817,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7512,"num_write_kb":60096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"152'7236","reported_seq":"7318","reported_epoch":"152","state":"active+clean","last_fresh":"2020-08-07T15:09:57.720688+0000","last_change":"2020-08-07T15:08:46.710643+0000","last_active":"2020-08-07T15:09:57.720688+0000","last_peered":"2020-08-07T15:09:57.720688+0000","last_clean":"2020-08-07T15:09:57.720688+0000","last_became_active":"2020-08-07T15:08:46.710100+0000","last_became_peered":"2020-08-07T15:08:46.710100+0000","last_unstale":"2020-08-07T15:09:57.720688+0000","last_undegraded":"2020-08-07T15:09:57.720688+0000","last_fullsized":"2020-08-07T15:09:57.720688+0000","mapping_epoch":98,"log_start":"152'7233","ondisk_log_start":"152'7233","created":81,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:23.520041+0000","last_clean_scrub_stamp":"2020-08-07T15:08:23.520041+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":54853632,"num_objects":837,"num_object_clones":0,"num_object_copies":2511,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7236,"num_write_kb":57408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"93","reported_epoch":"151","state":"active+clean","last_fresh":"2020-08-07T15:09:53.192279+0000","last_change":"2020-08-07T15:09:08.653773+0000","last_active":"2020-08-07T15:09:53.192279+0000","last_peered":"2020-08-07T15:09:53.192279+0000","last_clean":"2020-08-07T15:09:53.192279+0000","last_became_active":"2020-08-07T15:08:41.677944+0000","last_became_peered":"2020-08-07T15:08:41.677944+0000","last_unstale":"2020-08-07T15:09:53.192279+0000","last_undegraded":"2020-08-07T15:09:53.192279+0000","last_fullsized":"2020-08-07T15:09:53.192279+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:08.653722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:08.653722+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":939663360,"num_objects":14339,"num_object_clones":0,"num_object_copies":43017,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":936,"num_objects_unfound":0,"num_objects_dirty":14339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116424,"num_write_kb":929864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2275,"num_bytes_recovered":148971520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2862309376,"data_stored":2862309376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89,"ondisk_log_size":89,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":63,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542190,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1456968,"kb_used_data":408256,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92914872,"statfs":{"total":96636764160,"available":95144828928,"internally_reserved":1073741824,"allocated":418054144,"data_stored":417576107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,3,1,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.95099999999999996,"15min":0.91200000000000003},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.1739999999999999,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":2.1379999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.0149999999999999,"15min":0.97999999999999998},"min":{"1min":0.54300000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.8899999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.357,"15min":1.3149999999999999},"min":{"1min":0.68700000000000006,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":5.6239999999999997,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":1.181},{"interface":"front","average":{"1min":1.105,"5min":1.0720000000000001,"15min":1.0009999999999999},"min":{"1min":0.64100000000000001,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.206,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":1.724}]},{"osd":2,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.272,"15min":1.175},"min":{"1min":0.71999999999999997,"5min":0.628,"15min":0.628},"max":{"1min":5.6619999999999999,"5min":5.6619999999999999,"15min":5.6619999999999999},"last":1.762},{"interface":"front","average":{"1min":1.446,"5min":1.26,"15min":1.165},"min":{"1min":0.66800000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":5.1319999999999997,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":3.6619999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":75.045000000000002,"5min":16.263999999999999,"15min":6.5359999999999996},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":917.38699999999994,"5min":917.38699999999994,"15min":917.38699999999994},"last":2.9710000000000001},{"interface":"front","average":{"1min":19.670999999999999,"5min":4.9939999999999998,"15min":2.5600000000000001},"min":{"1min":0.80000000000000004,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":333.59199999999998,"5min":333.59199999999998,"15min":333.59199999999998},"last":2.3359999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.411,"15min":1.321},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":4.5250000000000004,"5min":4.835,"15min":4.835},"last":3.9750000000000001},{"interface":"front","average":{"1min":52.259999999999998,"5min":11.52,"15min":4.7140000000000004},"min":{"1min":0.77000000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":866.24099999999999,"5min":866.24099999999999,"15min":866.24099999999999},"last":3.0329999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.2869999999999999,"5min":23.440000000000001,"15min":32.817999999999998},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":6.6509999999999998,"5min":831.88099999999997,"15min":831.88099999999997},"last":2.1760000000000002},{"interface":"front","average":{"1min":2.2519999999999998,"5min":1.8380000000000001,"15min":1.609},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":6.6790000000000003,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":2.2160000000000002}]},{"osd":7,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":13.818,"5min":4.0019999999999998,"15min":2.3460000000000001},"min":{"1min":0.82999999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":224.14500000000001,"5min":224.14500000000001,"15min":224.14500000000001},"last":3.0049999999999999},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.6839999999999999,"15min":1.5660000000000001},"min":{"1min":0.90600000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":7.4359999999999999,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":2.2440000000000002}]}]},{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1341556,"kb_used_data":292972,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93030284,"statfs":{"total":96636764160,"available":95263010816,"internally_reserved":1073741824,"allocated":300003328,"data_stored":299532795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.179,"15min":1.2809999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8149999999999999,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":1.2589999999999999},{"interface":"front","average":{"1min":15.239000000000001,"5min":3.9529999999999998,"15min":2.0840000000000001},"min":{"1min":0.78100000000000003,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":320.31299999999999,"5min":320.31299999999999,"15min":320.31299999999999},"last":0.83299999999999996}]},{"osd":1,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":24.821999999999999,"5min":5.7329999999999997,"15min":2.5489999999999999},"min":{"1min":0.46700000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":547.65300000000002,"5min":547.65300000000002,"15min":547.65300000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":1.3029999999999999,"5min":4.9130000000000003,"15min":2.5619999999999998},"min":{"1min":0.69799999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.5449999999999999,"5min":372.50799999999998,"15min":372.50799999999998},"last":1.179}]},{"osd":2,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.208,"15min":1.1990000000000001},"min":{"1min":0.502,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":5.2400000000000002,"5min":5.2400000000000002,"15min":5.2400000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":1.286,"5min":1.2390000000000001,"15min":1.2609999999999999},"min":{"1min":0.66800000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":6.0490000000000004,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":1.1990000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.4370000000000001,"15min":1.496},"min":{"1min":0.878,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.0670000000000002,"5min":3.46,"15min":3.46},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3600000000000001,"15min":1.3779999999999999},"min":{"1min":0.70299999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":5.2770000000000001,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":1.236}]},{"osd":4,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.504,"15min":1.6100000000000001},"min":{"1min":0.70099999999999996,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.165,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.46,"5min":1.625,"15min":1.6559999999999999},"min":{"1min":0.96199999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":2.4279999999999999,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.395}]},{"osd":5,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.2809999999999999,"15min":1.3360000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.3069999999999999,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.6259999999999999,"15min":1.7190000000000001},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.4630000000000001,"5min":5.327,"15min":5.327},"last":1.657}]},{"osd":6,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.458,"15min":1.4410000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.4900000000000002,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.704},{"interface":"front","average":{"1min":1.5720000000000001,"5min":11.603999999999999,"15min":16.085000000000001},"min":{"1min":1.004,"5min":0.87,"15min":0.87},"max":{"1min":2.7719999999999998,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.728}]}]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1631220,"kb_used_data":582636,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92740620,"statfs":{"total":96636764160,"available":94966394880,"internally_reserved":1073741824,"allocated":596619264,"data_stored":596148731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,1,0,0,0,1,6],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":20.945,"15min":29.771999999999998},"min":{"1min":0.49199999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":6.8810000000000002,"5min":497.04000000000002,"15min":497.04000000000002},"last":0.998},{"interface":"front","average":{"1min":25.385999999999999,"5min":14.656000000000001,"15min":5.6390000000000002},"min":{"1min":0.52200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":513.78899999999999,"5min":993.84500000000003,"15min":993.84500000000003},"last":0.94299999999999995}]},{"osd":2,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.999,"15min":0.93200000000000005},"min":{"1min":0.60099999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.726,"5min":4.585,"15min":4.585},"last":1.444},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.254,"15min":1.22},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":6.4029999999999996,"5min":6.4029999999999996,"15min":6.4029999999999996},"last":1.137}]},{"osd":3,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":40.524000000000001,"5min":9.2829999999999995,"15min":3.968},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":828.31600000000003,"5min":828.31600000000003,"15min":828.31600000000003},"last":2.3879999999999999},{"interface":"front","average":{"1min":44.292000000000002,"5min":10.169,"15min":4.359},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":905.38300000000004,"5min":905.38300000000004,"15min":905.38300000000004},"last":2.5659999999999998}]},{"osd":4,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":27.831,"5min":6.774,"15min":3.1139999999999999},"min":{"1min":0.79900000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":550.66800000000001,"5min":550.66800000000001,"15min":550.66800000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.3400000000000001,"15min":1.246},"min":{"1min":0.92100000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":5.9420000000000002,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.508}]},{"osd":5,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":8.4550000000000001,"15min":11.49},"min":{"1min":0.71199999999999997,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.7200000000000002,"5min":213.63800000000001,"15min":213.63800000000001},"last":1.228},{"interface":"front","average":{"1min":58.497,"5min":12.720000000000001,"15min":5.0330000000000004},"min":{"1min":0.76500000000000001,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":875.44600000000003,"5min":875.44600000000003,"15min":875.44600000000003},"last":1.4810000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:09:27 2020","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.587,"15min":1.5629999999999999},"min":{"1min":0.81000000000000005,"5min":0.754,"15min":0.754},"max":{"1min":5.7149999999999999,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":1.1020000000000001},{"interface":"front","average":{"1min":11.539,"5min":3.5979999999999999,"15min":2.1059999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":207.137,"5min":207.137,"15min":207.137},"last":1.2969999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.782,"5min":1.6879999999999999,"15min":1.639},"min":{"1min":0.94099999999999995,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":6.444,"5min":6.444,"15min":6.444},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.6819999999999999,"5min":1.6779999999999999,"15min":1.546},"min":{"1min":0.93300000000000005,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":5.9279999999999999,"5min":10.752000000000001,"15min":10.752000000000001},"last":1.343}]}]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1341572,"kb_used_data":292988,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93030268,"statfs":{"total":96636764160,"available":95262994432,"internally_reserved":1073741824,"allocated":300019712,"data_stored":299549179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":17.058,"15min":6.3369999999999997},"min":{"1min":0.61399999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.1269999999999998,"5min":845.57100000000003,"15min":845.57100000000003},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.0169999999999999,"15min":0.96099999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":4.8209999999999997,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":0.80900000000000005}]},{"osd":2,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.0189999999999999,"15min":0.95799999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.7719999999999998,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":1.173},{"interface":"front","average":{"1min":13.177,"5min":22.48,"15min":29.32},"min":{"1min":0.61399999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":226.911,"5min":606.18499999999995,"15min":606.18499999999995},"last":1.5600000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.0660000000000001,"15min":0.97899999999999998},"min":{"1min":0.54100000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6859999999999999,"5min":3.1779999999999999,"15min":3.1779999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.3160000000000001,"15min":1.2170000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.976,"5min":4.21,"15min":4.21},"last":0.76300000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.2829999999999999,"15min":1.2050000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":8.3100000000000005,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":1.702},{"interface":"front","average":{"1min":1.7709999999999999,"5min":1.446,"15min":1.3200000000000001},"min":{"1min":0.84599999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":8.4190000000000005,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":1.208}]},{"osd":5,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.9770000000000001,"5min":9.0380000000000003,"15min":3.8300000000000001},"min":{"1min":0.77800000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":7.8929999999999998,"5min":501.267,"15min":501.267},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.2609999999999999,"15min":1.157},"min":{"1min":0.78200000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.0939999999999999,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":1.764}]},{"osd":6,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":5.7779999999999996,"15min":7.6680000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.0059999999999998,"5min":124.738,"15min":124.738},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.627,"5min":11.811,"15min":16.372},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":4.6879999999999997,"5min":294.94,"15min":294.94},"last":1.262}]},{"osd":7,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.5589999999999999,"15min":1.577},"min":{"1min":0.95399999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":4.7169999999999996,"5min":6.468,"15min":6.468},"last":1.544},{"interface":"front","average":{"1min":1.363,"5min":1.4410000000000001,"15min":1.4039999999999999},"min":{"1min":0.88800000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.1040000000000001,"5min":3.117,"15min":3.117},"last":1.7969999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1219324,"kb_used_data":170740,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93152516,"statfs":{"total":96636764160,"available":95388176384,"internally_reserved":1073741824,"allocated":174837760,"data_stored":174367227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.6950000000000001,"5min":1.268,"15min":1.167},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":8.4930000000000003,"5min":8.4930000000000003,"15min":8.4930000000000003},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.357,"5min":1.113,"15min":1.0209999999999999},"min":{"1min":0.61899999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":3.5,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":5.194}]},{"osd":1,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.9390000000000001,"5min":1.5469999999999999,"15min":1.2430000000000001},"min":{"1min":0.54400000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":5.6820000000000004,"5min":18.672000000000001,"15min":18.672000000000001},"last":3.766},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.351,"15min":1.357},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":3.585,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":0.72599999999999998}]},{"osd":3,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.268,"15min":1.198},"min":{"1min":0.72799999999999998,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.8740000000000001,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":34.170999999999999,"5min":8.0510000000000002,"15min":3.7210000000000001},"min":{"1min":0.68799999999999994,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":501.75599999999997,"5min":501.75599999999997,"15min":501.75599999999997},"last":0.51400000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.403,"15min":1.2070000000000001},"min":{"1min":0.65200000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":3.4180000000000001,"5min":19.783000000000001,"15min":19.783000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.546,"15min":1.6100000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":3.5569999999999999,"5min":3.5569999999999999,"15min":3.5569999999999999},"last":0.76900000000000002}]},{"osd":5,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.292,"15min":1.272},"min":{"1min":0.83299999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.323,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":43.396999999999998,"5min":24.298999999999999,"15min":25.116},"min":{"1min":0.99299999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":627.53300000000002,"5min":627.53300000000002,"15min":627.53300000000002},"last":0.83699999999999997}]},{"osd":6,"last update":"Fri Aug 7 15:09:30 2020","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.538,"15min":1.3260000000000001},"min":{"1min":0.85999999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":3.452,"5min":20.035,"15min":20.035},"last":0.749},{"interface":"front","average":{"1min":1.498,"5min":9.8979999999999997,"15min":4.1799999999999997},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.976,"5min":751.70000000000005,"15min":751.70000000000005},"last":0.95499999999999996}]},{"osd":7,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":54.209000000000003,"5min":12.595000000000001,"15min":5.359},"min":{"1min":0.84999999999999998,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":501.91699999999997,"5min":501.91699999999997,"15min":501.91699999999997},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.665,"5min":1.919,"15min":1.637},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":3.133,"5min":35.646999999999998,"15min":35.646999999999998},"last":0.97399999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1318012,"kb_used_data":269428,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93053828,"statfs":{"total":96636764160,"available":95287119872,"internally_reserved":1073741824,"allocated":275894272,"data_stored":275423739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":1.341,"15min":1.282},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.097,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.4530000000000001,"15min":1.575},"min":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.3039999999999998,"5min":13.067,"15min":13.067},"last":0.92200000000000004}]},{"osd":1,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":4.5860000000000003,"15min":2.794},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.9910000000000001,"5min":270.68700000000001,"15min":270.68700000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.4379999999999999,"5min":2.0390000000000001,"15min":2.0209999999999999},"min":{"1min":0.55100000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":7.0940000000000003,"5min":14.349,"15min":14.349},"last":1.2270000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.266,"15min":1.1870000000000001},"min":{"1min":0.71399999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.2639999999999998,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.96899999999999997,"5min":2.109,"15min":1.8260000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.661,"5min":51.530000000000001,"15min":51.530000000000001},"last":1.014}]},{"osd":4,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":10.773999999999999,"15min":4.7220000000000004},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.1360000000000001,"5min":506.21300000000002,"15min":506.21300000000002},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.276,"5min":25.469000000000001,"15min":29.888999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.8940000000000001,"5min":677.88599999999997,"15min":677.88599999999997},"last":1.4410000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.5920000000000001,"15min":1.542},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.9710000000000001,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.8009999999999999,"15min":1.7789999999999999},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":3.044,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":1.4730000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.6259999999999999,"15min":1.5629999999999999},"min":{"1min":0.89800000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.9359999999999999,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":1.411},{"interface":"front","average":{"1min":21.766999999999999,"5min":5.9390000000000001,"15min":3.3039999999999998},"min":{"1min":0.89900000000000002,"5min":0.876,"15min":0.876},"max":{"1min":387.911,"5min":387.911,"15min":387.911},"last":1.508}]},{"osd":7,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.9330000000000001,"15min":2.085},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.0110000000000001,"5min":5.734,"15min":5.734},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.325,"5min":1.9750000000000001,"15min":1.976},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.169,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":0.84499999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1512188,"kb_used_data":463604,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92859652,"statfs":{"total":96636764160,"available":95088283648,"internally_reserved":1073741824,"allocated":474730496,"data_stored":474259963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":1.0720000000000001,"15min":1.1439999999999999},"min":{"1min":0.52900000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4390000000000001,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":0.621},{"interface":"front","average":{"1min":1.143,"5min":1.0069999999999999,"15min":0.90100000000000002},"min":{"1min":0.498,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":4.907,"5min":6.3970000000000002,"15min":6.3970000000000002},"last":0.54600000000000004}]},{"osd":1,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":23.940999999999999,"5min":5.8209999999999997,"15min":2.843},"min":{"1min":0.59599999999999997,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":320.67000000000002,"5min":320.67000000000002,"15min":320.67000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":1.1359999999999999,"5min":7.5700000000000003,"15min":3.4409999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.784,"5min":689.846,"15min":689.846},"last":0.90900000000000003}]},{"osd":2,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.3999999999999999,"15min":1.4339999999999999},"min":{"1min":0.77600000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.0899999999999999,"5min":6.8730000000000002,"15min":6.8730000000000002},"last":3.4830000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.516,"15min":1.5149999999999999},"min":{"1min":0.81699999999999995,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.3159999999999998,"5min":7.1360000000000001,"15min":7.1360000000000001},"last":3.5659999999999998}]},{"osd":3,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":3.681,"15min":2.2320000000000002},"min":{"1min":0.78300000000000003,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.8149999999999999,"5min":247.202,"15min":247.202},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.3720000000000001,"15min":1.2909999999999999},"min":{"1min":0.68500000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":4.931,"5min":6.431,"15min":6.431},"last":0.96199999999999997}]},{"osd":5,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":10.57,"15min":4.6079999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.431,"5min":990.71600000000001,"15min":990.71600000000001},"last":2.2559999999999998},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.6100000000000001,"15min":1.621},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":3.3519999999999999,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":1.556}]},{"osd":6,"last update":"Fri Aug 7 15:09:28 2020","interfaces":[{"interface":"back","average":{"1min":1.6799999999999999,"5min":1.7509999999999999,"15min":1.7769999999999999},"min":{"1min":0.75800000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":5.0149999999999997,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.498,"5min":12.069000000000001,"15min":16.762},"min":{"1min":0.83299999999999996,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":3.3530000000000002,"5min":389.14600000000002,"15min":389.14600000000002},"last":1.6040000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":3.8300000000000001,"15min":2.2149999999999999},"min":{"1min":0.753,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":4.8710000000000004,"5min":252.798,"15min":252.798},"last":1.7270000000000001},{"interface":"front","average":{"1min":1.262,"5min":10.175000000000001,"15min":4.4889999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.3050000000000002,"5min":501.93000000000001,"15min":501.93000000000001},"last":2.0270000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1395140,"kb_used_data":346556,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92976700,"statfs":{"total":96636764160,"available":95208140800,"internally_reserved":1073741824,"allocated":354873344,"data_stored":354402811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":1.012,"15min":1.0549999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.651,"5min":2.766,"15min":2.766},"last":1.466},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91600000000000004,"15min":0.84199999999999997},"min":{"1min":0.45500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.3340000000000001,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":0.54200000000000004}]},{"osd":1,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.4540000000000002,"5min":10.521000000000001,"15min":4.1260000000000003},"min":{"1min":0.56699999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":29.283000000000001,"5min":787.22900000000004,"15min":787.22900000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":8.0939999999999994,"5min":2.4969999999999999,"15min":1.571},"min":{"1min":0.60999999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":116.548,"5min":116.548,"15min":116.548},"last":1.2470000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":5.9340000000000002,"15min":8.1240000000000006},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.78,"5min":168.56899999999999,"15min":168.56899999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.119,"5min":20.361000000000001,"15min":28.873999999999999},"min":{"1min":0.60899999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.665,"5min":676.27099999999996,"15min":676.27099999999996},"last":0.71699999999999997}]},{"osd":3,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.6739999999999999,"5min":1.5069999999999999,"15min":1.2470000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":29.129000000000001,"5min":29.129000000000001,"15min":29.129000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":3.0219999999999998,"5min":9.9459999999999997,"15min":4.1239999999999997},"min":{"1min":0.70799999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":29.103000000000002,"5min":712.98699999999997,"15min":712.98699999999997},"last":1.113}]},{"osd":4,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":49.854999999999997,"5min":40.755000000000003,"15min":47.512999999999998},"min":{"1min":0.61899999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":971.67999999999995,"5min":971.67999999999995,"15min":971.67999999999995},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.264,"15min":1.292},"min":{"1min":0.64900000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.5960000000000001,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":0.79800000000000004}]},{"osd":6,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":91.870999999999995,"5min":48.862000000000002,"15min":49.893000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":966.60900000000004,"5min":966.60900000000004,"15min":966.60900000000004},"last":0.85099999999999998},{"interface":"front","average":{"1min":28.396999999999998,"5min":21.527000000000001,"15min":24.52},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":501.79000000000002,"5min":517.26700000000005,"15min":517.26700000000005},"last":1.2130000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.341,"15min":1.319},"min":{"1min":0.77500000000000002,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.8940000000000001,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":0.997},{"interface":"front","average":{"1min":27.206,"5min":6.6630000000000003,"15min":3.2650000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":515.09100000000001,"5min":515.09100000000001,"15min":515.09100000000001},"last":0.96099999999999997}]}]}]}} 2020-08-07T15:09:59.019 INFO:tasks.thrashosds.thrasher:Setting 4.3 to [1, 5, 6, 0, 3, 4] 2020-08-07T15:09:59.020 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '4.3', '1', '5', '6', '0', '3', '4'] 2020-08-07T15:09:59.020 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pg-upmap-items 4.3 1 5 6 0 3 4 2020-08-07T15:09:59.085 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:59.077+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:59.187 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:59.181+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:59.188 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:59.189 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:59.203 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:09:59.303 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:59.290+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:09:59.389 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:59.387+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:59.457 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:59.457 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:59.470 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:09:59.490 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:09:59.488+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:09:59.591 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:09:59.589+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:09:59.695 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:59.691+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:59.733 INFO:teuthology.orchestra.run.smithi074.stderr:set 4.3 pg_upmap_items mapping to [1->5,6->0,3->4] 2020-08-07T15:09:59.734 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:09:59.734 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:09:59.739 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:09:59.837 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:09:59.824+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:09:59.901 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:09:59.892+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:09:59.999 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:09:59.993+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:00.007 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:00.007 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:00.019 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:10:00.102 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:00.096+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:00.204 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:00.200+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:00.265 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:00.265 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:00.279 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:10:00.331 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:00.319+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:00.421 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:00.409+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:00.442 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:10:00.453 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:10:00.521 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:00.511+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:00.559 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:00.559 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:00.580 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:10:00.633 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:00.619+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:00.725 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:00.721+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:00.791 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:10:00.828 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:00.824+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:00.875 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:10:00.919 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:00.922 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:10:00.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:10:00.923 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:10:00.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:00.923 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:10240 3.6 3:665d71ff:::benchmark_data_smithi074_35447_object1279:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e24)", 2020-08-07T15:10:00.929 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:10:00.930 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 198.51984286199999, 2020-08-07T15:10:00.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25191569699999999, 2020-08-07T15:10:00.930 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:00.930 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:00.931 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:00.931 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:10:00.931 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:00.931 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 10240 2020-08-07T15:10:00.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:00.932 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:00.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:00.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:00.940 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:10:00.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:00.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:00.940 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:00.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:00.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393219+0000", 2020-08-07T15:10:00.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5463000000000002e-05 2020-08-07T15:10:00.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:00.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:00.942 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:00.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393264+0000", 2020-08-07T15:10:00.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9219999999999999e-06 2020-08-07T15:10:00.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:00.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:00.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:00.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393268+0000", 2020-08-07T15:10:00.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6149999999999997e-06 2020-08-07T15:10:00.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:00.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:00.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:01.004 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393277+0000", 2020-08-07T15:10:01.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2039999999999996e-06 2020-08-07T15:10:01.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:01.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.393282+0000", 2020-08-07T15:10:01.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001225243 2020-08-07T15:10:01.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:01.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394507+0000", 2020-08-07T15:10:01.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7108999999999995e-05 2020-08-07T15:10:01.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:01.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394605+0000", 2020-08-07T15:10:01.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017490800000000001 2020-08-07T15:10:01.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:10:01.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.394779+0000", 2020-08-07T15:10:01.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032608699999999999 2020-08-07T15:10:01.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.037 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:01.038 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.398040+0000", 2020-08-07T15:10:01.038 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013015259999999999 2020-08-07T15:10:01.038 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.038 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:01.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.399342+0000", 2020-08-07T15:10:01.040 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24566893400000001 2020-08-07T15:10:01.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.041 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:01.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645011+0000", 2020-08-07T15:10:01.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.217e-06 2020-08-07T15:10:01.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.042 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:01.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645012+0000", 2020-08-07T15:10:01.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0803000000000006e-05 2020-08-07T15:10:01.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.044 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:01.044 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645083+0000", 2020-08-07T15:10:01.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1882999999999998e-05 2020-08-07T15:10:01.045 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.045 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:01.045 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:42.645135+0000", 2020-08-07T15:10:01.046 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25191569699999999 2020-08-07T15:10:01.046 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.046 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:01.048 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.048 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.048 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.048 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:71136 3.6 e43/26 3:64cb7acb:::benchmark_data_smithi074_35447_object8891:head v 43'4744, mlcod=43'4739)", 2020-08-07T15:10:01.049 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:10:01.049 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 156.59444827600001, 2020-08-07T15:10:01.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26135482999999998, 2020-08-07T15:10:01.051 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:01.051 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:01.051 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:01.059 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.059 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:01.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:10:01.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:01.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:01.069 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:10:01.069 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3702999999999997e-05 2020-08-07T15:10:01.070 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.070 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.070 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:01.070 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318667+0000", 2020-08-07T15:10:01.070 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3119999999999999e-06 2020-08-07T15:10:01.071 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.071 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.071 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:01.071 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318674+0000", 2020-08-07T15:10:01.072 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.255173292 2020-08-07T15:10:01.084 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.084 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:01.084 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.573847+0000", 2020-08-07T15:10:01.085 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1140000000000002e-06 2020-08-07T15:10:01.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.085 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:01.086 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.573853+0000", 2020-08-07T15:10:01.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00057603799999999996 2020-08-07T15:10:01.086 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.086 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:01.087 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.574429+0000", 2020-08-07T15:10:01.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4776e-05 2020-08-07T15:10:01.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:01.088 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.574444+0000", 2020-08-07T15:10:01.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054577080000000004 2020-08-07T15:10:01.089 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.089 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.089 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:01.096 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.579901+0000", 2020-08-07T15:10:01.097 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6886999999999998e-05 2020-08-07T15:10:01.097 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:01.117 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.579968+0000", 2020-08-07T15:10:01.117 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26135482999999998 2020-08-07T15:10:01.118 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.118 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:01.124 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.125 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.125 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.125 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:85696 3.f e51/20 3:f99fcea4:::benchmark_data_smithi074_35447_object10711:head v 51'5688, mlcod=51'5679)", 2020-08-07T15:10:01.125 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:10:01.126 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 147.260456863, 2020-08-07T15:10:01.126 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49655717300000002, 2020-08-07T15:10:01.133 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:01.133 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:01.133 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:01.134 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.134 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:01.134 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:10:01.134 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:01.135 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.135 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.135 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:01.135 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:10:01.136 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9543000000000002e-05 2020-08-07T15:10:01.136 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.136 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.136 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:01.136 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652665+0000", 2020-08-07T15:10:01.137 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8700000000000003e-06 2020-08-07T15:10:01.137 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.145 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.158 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:01.158 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652671+0000", 2020-08-07T15:10:01.158 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49579595799999998 2020-08-07T15:10:01.159 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.159 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.160 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:01.160 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148467+0000", 2020-08-07T15:10:01.160 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2509999999999997e-06 2020-08-07T15:10:01.161 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.162 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.162 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:01.163 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148475+0000", 2020-08-07T15:10:01.163 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5603999999999998e-05 2020-08-07T15:10:01.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.163 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:01.164 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148530+0000", 2020-08-07T15:10:01.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5486999999999998e-05 2020-08-07T15:10:01.164 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.165 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:01.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148556+0000", 2020-08-07T15:10:01.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00055666600000000004 2020-08-07T15:10:01.166 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.166 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.166 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:01.178 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.149112+0000", 2020-08-07T15:10:01.178 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9793999999999998e-05 2020-08-07T15:10:01.178 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.178 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.179 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:01.179 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.149162+0000", 2020-08-07T15:10:01.179 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49655717300000002 2020-08-07T15:10:01.179 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.180 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:01.180 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.180 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.180 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:86792 3.f e52/20 3:fc5386ee:::benchmark_data_smithi074_35447_object10848:head v 52'5752, mlcod=52'5743)", 2020-08-07T15:10:01.181 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:10:01.181 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 146.51061108799999, 2020-08-07T15:10:01.181 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.295270005, 2020-08-07T15:10:01.181 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:01.181 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:01.185 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:01.185 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.194 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:01.194 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:10:01.194 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:01.195 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.195 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.195 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:01.196 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:10:01.196 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9712000000000001e-05 2020-08-07T15:10:01.196 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.196 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.197 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:01.197 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402500+0000", 2020-08-07T15:10:01.197 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6789999999999999e-06 2020-08-07T15:10:01.197 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.199 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.199 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:01.205 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402507+0000", 2020-08-07T15:10:01.205 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29317074999999998 2020-08-07T15:10:01.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.205 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.206 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:01.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695678+0000", 2020-08-07T15:10:01.209 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8120000000000002e-06 2020-08-07T15:10:01.209 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.210 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.210 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:01.210 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695685+0000", 2020-08-07T15:10:01.210 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1270999999999998e-05 2020-08-07T15:10:01.211 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.211 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.211 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:01.211 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695726+0000", 2020-08-07T15:10:01.212 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2739e-05 2020-08-07T15:10:01.212 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.212 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.212 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:01.213 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695749+0000", 2020-08-07T15:10:01.215 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019333899999999999 2020-08-07T15:10:01.216 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.216 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.216 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:01.216 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.697682+0000", 2020-08-07T15:10:01.217 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8652000000000002e-05 2020-08-07T15:10:01.217 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.218 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.218 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:01.218 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.697721+0000", 2020-08-07T15:10:01.218 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.295270005 2020-08-07T15:10:01.219 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.219 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:01.219 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.219 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.220 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.220 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:87088 3.6 e52/26 3:6b0320df:::benchmark_data_smithi074_35447_object10885:head v 52'5912, mlcod=52'5902)", 2020-08-07T15:10:01.220 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:10:01.220 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 145.946377848, 2020-08-07T15:10:01.221 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29522189799999998, 2020-08-07T15:10:01.221 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:01.221 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:01.221 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:01.221 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.222 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:01.222 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:10:01.222 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:01.222 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.223 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.223 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:01.223 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:10:01.223 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0751000000000002e-05 2020-08-07T15:10:01.223 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.224 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.224 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:01.224 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966735+0000", 2020-08-07T15:10:01.226 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3080000000000004e-06 2020-08-07T15:10:01.226 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.226 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.226 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:01.227 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966741+0000", 2020-08-07T15:10:01.227 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29365034699999998 2020-08-07T15:10:01.227 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.227 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.228 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:01.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260391+0000", 2020-08-07T15:10:01.229 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2720000000000003e-06 2020-08-07T15:10:01.230 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.230 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.230 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:01.231 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260398+0000", 2020-08-07T15:10:01.231 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8819000000000005e-05 2020-08-07T15:10:01.231 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.231 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.232 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:01.241 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260476+0000", 2020-08-07T15:10:01.241 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4958999999999999e-05 2020-08-07T15:10:01.242 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.242 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.242 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:01.243 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260501+0000", 2020-08-07T15:10:01.243 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001353641 2020-08-07T15:10:01.243 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.243 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.244 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:01.244 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.261855+0000", 2020-08-07T15:10:01.244 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0800999999999997e-05 2020-08-07T15:10:01.244 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.245 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.245 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:01.245 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.261906+0000", 2020-08-07T15:10:01.245 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29522189799999998 2020-08-07T15:10:01.245 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.249 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:01.250 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.250 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.251 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.252 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:93920 3.6 e56/26 3:631393d3:::benchmark_data_smithi074_35447_object11739:head v 56'6352, mlcod=55'6342)", 2020-08-07T15:10:01.252 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:10:01.252 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 141.586262417, 2020-08-07T15:10:01.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39765404300000001, 2020-08-07T15:10:01.253 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:01.253 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:01.253 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:01.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.254 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:01.254 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:10:01.254 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:01.254 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.255 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.255 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:01.255 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:10:01.255 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3035999999999998e-05 2020-08-07T15:10:01.255 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.256 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.256 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:01.256 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326852+0000", 2020-08-07T15:10:01.256 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5379999999999999e-06 2020-08-07T15:10:01.257 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.257 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.257 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:01.257 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326859+0000", 2020-08-07T15:10:01.257 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.393917568 2020-08-07T15:10:01.260 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.261 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.261 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:01.261 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.720777+0000", 2020-08-07T15:10:01.261 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2430000000000005e-06 2020-08-07T15:10:01.262 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.262 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.262 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:01.262 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.720785+0000", 2020-08-07T15:10:01.263 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00065658700000000003 2020-08-07T15:10:01.266 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.266 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.266 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:01.266 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.721441+0000", 2020-08-07T15:10:01.267 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8585000000000002e-05 2020-08-07T15:10:01.267 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.267 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.268 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:01.268 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.721470+0000", 2020-08-07T15:10:01.268 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00293349 2020-08-07T15:10:01.268 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.268 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.269 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:01.269 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.724403+0000", 2020-08-07T15:10:01.271 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9996e-05 2020-08-07T15:10:01.271 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:01.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.724453+0000", 2020-08-07T15:10:01.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39765404300000001 2020-08-07T15:10:01.273 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.273 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:01.273 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.273 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.274 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.274 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:116064 3.6 e64/26 3:6cc03ffa:::benchmark_data_smithi074_35447_object14507:head v 64'7632, mlcod=64'7623)", 2020-08-07T15:10:01.274 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:10:01.274 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 124.550301014, 2020-08-07T15:10:01.275 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25728607799999997, 2020-08-07T15:10:01.275 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:01.275 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:01.275 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:01.276 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.276 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:01.276 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:10:01.276 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:01.277 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.277 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.277 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:01.277 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:10:01.278 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6042000000000003e-05 2020-08-07T15:10:01.278 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.278 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.278 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:01.279 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362817+0000", 2020-08-07T15:10:01.279 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9399999999999999e-06 2020-08-07T15:10:01.279 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.279 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.279 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:01.280 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362823+0000", 2020-08-07T15:10:01.280 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.254635583 2020-08-07T15:10:01.281 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.281 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.281 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:01.282 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617458+0000", 2020-08-07T15:10:01.282 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5100000000000004e-06 2020-08-07T15:10:01.282 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.282 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.282 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:01.283 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617465+0000", 2020-08-07T15:10:01.283 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4439999999999999e-05 2020-08-07T15:10:01.283 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.283 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.284 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:01.284 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617539+0000", 2020-08-07T15:10:01.284 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3527e-05 2020-08-07T15:10:01.285 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.285 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.286 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:01.286 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617563+0000", 2020-08-07T15:10:01.286 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024301069999999999 2020-08-07T15:10:01.286 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.287 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:01.287 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.619993+0000", 2020-08-07T15:10:01.287 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3928999999999998e-05 2020-08-07T15:10:01.288 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.288 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.288 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:01.288 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.620047+0000", 2020-08-07T15:10:01.289 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25728607799999997 2020-08-07T15:10:01.289 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.289 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:01.289 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.290 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.290 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.290 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:145801 3.6 e75/26 3:69e4cb22:::benchmark_data_smithi074_35447_object16192:head v 75'9405, mlcod=75'9401)", 2020-08-07T15:10:01.290 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:10:01.291 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 106.24130572200001, 2020-08-07T15:10:01.291 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639, 2020-08-07T15:10:01.292 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:01.292 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:01.292 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:01.292 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.293 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:01.293 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:10:01.293 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:01.293 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.294 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.294 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:01.294 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:10:01.294 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6511000000000002e-05 2020-08-07T15:10:01.295 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.297 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.297 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:01.298 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671803+0000", 2020-08-07T15:10:01.298 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7190000000000004e-06 2020-08-07T15:10:01.298 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.298 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.298 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:01.299 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671809+0000", 2020-08-07T15:10:01.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74210928300000001 2020-08-07T15:10:01.299 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.306 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.317 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:01.317 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413919+0000", 2020-08-07T15:10:01.318 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1759999999999999e-05 2020-08-07T15:10:01.318 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.318 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.319 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:01.319 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413930+0000", 2020-08-07T15:10:01.319 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000108601 2020-08-07T15:10:01.319 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.319 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.320 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:01.320 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414039+0000", 2020-08-07T15:10:01.320 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9948999999999999e-05 2020-08-07T15:10:01.320 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.330 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.330 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:01.331 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414089+0000", 2020-08-07T15:10:01.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00079656499999999995 2020-08-07T15:10:01.331 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.336 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.337 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:01.337 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414885+0000", 2020-08-07T15:10:01.337 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2250999999999998e-05 2020-08-07T15:10:01.337 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.338 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.338 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:01.338 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414938+0000", 2020-08-07T15:10:01.338 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639 2020-08-07T15:10:01.339 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.339 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:01.339 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.339 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.340 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:9112 4.6 e86/81 4:6004be0b:::benchmark_data_smithi074_44586_object1138:head v 86'544, mlcod=86'533)", 2020-08-07T15:10:01.340 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:32.296890+0000", 2020-08-07T15:10:01.340 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 88.616172313000007, 2020-08-07T15:10:01.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26263388100000001, 2020-08-07T15:10:01.341 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:01.341 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:01.341 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:01.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.341 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:01.342 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296890+0000", 2020-08-07T15:10:01.342 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:01.342 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.342 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.342 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:01.343 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296890+0000", 2020-08-07T15:10:01.343 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9907999999999997e-05 2020-08-07T15:10:01.343 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.343 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.343 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:01.344 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296939+0000", 2020-08-07T15:10:01.344 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7630000000000003e-06 2020-08-07T15:10:01.344 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.346 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.347 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:01.347 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296946+0000", 2020-08-07T15:10:01.349 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26058471900000002 2020-08-07T15:10:01.350 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.350 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.350 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:01.350 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557531+0000", 2020-08-07T15:10:01.351 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5270000000000004e-06 2020-08-07T15:10:01.351 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.351 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.351 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:01.351 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557537+0000", 2020-08-07T15:10:01.352 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2843000000000003e-05 2020-08-07T15:10:01.352 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.352 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.352 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:01.353 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557580+0000", 2020-08-07T15:10:01.353 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3833999999999999e-05 2020-08-07T15:10:01.353 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.353 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.353 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:01.355 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557604+0000", 2020-08-07T15:10:01.355 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001875196 2020-08-07T15:10:01.355 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.355 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.356 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:01.356 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.559479+0000", 2020-08-07T15:10:01.356 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4091000000000002e-05 2020-08-07T15:10:01.356 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.357 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.357 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:01.357 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.559523+0000", 2020-08-07T15:10:01.357 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26263388100000001 2020-08-07T15:10:01.357 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.358 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:01.358 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.358 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.358 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.358 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:13784 4.6 e88/81 4:66f64b92:::benchmark_data_smithi074_44586_object1722:head v 88'832, mlcod=88'827)", 2020-08-07T15:10:01.359 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:35.137315+0000", 2020-08-07T15:10:01.359 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 85.775746580000003, 2020-08-07T15:10:01.359 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32459016899999998, 2020-08-07T15:10:01.359 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:01.359 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:01.360 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:01.360 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.360 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:01.360 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137315+0000", 2020-08-07T15:10:01.360 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:01.361 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.361 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.361 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:01.361 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137315+0000", 2020-08-07T15:10:01.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0362999999999998e-05 2020-08-07T15:10:01.363 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.363 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.363 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:01.364 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137376+0000", 2020-08-07T15:10:01.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7020000000000004e-06 2020-08-07T15:10:01.365 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.365 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.365 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:01.366 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137382+0000", 2020-08-07T15:10:01.366 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32172605500000001 2020-08-07T15:10:01.366 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.366 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:01.367 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459108+0000", 2020-08-07T15:10:01.367 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7710000000000003e-06 2020-08-07T15:10:01.367 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.367 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.367 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:01.368 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459114+0000", 2020-08-07T15:10:01.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4297e-05 2020-08-07T15:10:01.368 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.368 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.368 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:01.369 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459158+0000", 2020-08-07T15:10:01.369 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5058999999999998e-05 2020-08-07T15:10:01.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.369 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:01.371 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459183+0000", 2020-08-07T15:10:01.371 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026624740000000002 2020-08-07T15:10:01.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.372 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:01.372 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.461846+0000", 2020-08-07T15:10:01.373 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9447999999999997e-05 2020-08-07T15:10:01.373 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.374 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.374 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:01.374 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.461905+0000", 2020-08-07T15:10:01.375 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32459016899999998 2020-08-07T15:10:01.375 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.375 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:01.376 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.377 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:31608 4.7 e100/81 4:e0ebdc25:::benchmark_data_smithi074_44586_object3950:head v 100'2136, mlcod=100'2126)", 2020-08-07T15:10:01.377 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:47.845848+0000", 2020-08-07T15:10:01.377 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 73.067213719999998, 2020-08-07T15:10:01.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27603951399999999, 2020-08-07T15:10:01.379 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:01.379 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:01.379 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:01.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:01.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:47.845848+0000", 2020-08-07T15:10:01.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:01.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:01.382 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:47.845848+0000", 2020-08-07T15:10:01.382 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3656999999999998e-05 2020-08-07T15:10:01.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.382 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.382 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:01.383 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:47.845902+0000", 2020-08-07T15:10:01.383 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4050000000000003e-06 2020-08-07T15:10:01.383 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.383 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.383 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:01.384 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:47.845909+0000", 2020-08-07T15:10:01.384 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27431144000000002 2020-08-07T15:10:01.384 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.384 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.384 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:01.385 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.120221+0000", 2020-08-07T15:10:01.385 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3640000000000004e-06 2020-08-07T15:10:01.385 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.385 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.385 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:01.386 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.120227+0000", 2020-08-07T15:10:01.386 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1541e-05 2020-08-07T15:10:01.386 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.386 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.386 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:01.386 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.120279+0000", 2020-08-07T15:10:01.387 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.372e-05 2020-08-07T15:10:01.387 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.387 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.387 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:01.387 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.120302+0000", 2020-08-07T15:10:01.388 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015294460000000001 2020-08-07T15:10:01.388 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.388 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.389 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:01.389 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.121832+0000", 2020-08-07T15:10:01.389 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5940999999999999e-05 2020-08-07T15:10:01.389 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.389 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.390 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:01.390 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.121888+0000", 2020-08-07T15:10:01.390 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27603951399999999 2020-08-07T15:10:01.390 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.390 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:01.390 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.391 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.411 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.411 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:31832 4.b e100/81 4:d005fdc5:::benchmark_data_smithi074_44586_object3978:head v 100'1872, mlcod=100'1863)", 2020-08-07T15:10:01.411 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:48.177094+0000", 2020-08-07T15:10:01.412 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 72.735967455999997, 2020-08-07T15:10:01.412 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26337036000000003, 2020-08-07T15:10:01.412 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:01.412 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:01.412 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:01.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.413 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:01.413 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.177094+0000", 2020-08-07T15:10:01.413 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:01.413 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.430 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.430 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:01.431 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.177094+0000", 2020-08-07T15:10:01.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3760999999999997e-05 2020-08-07T15:10:01.431 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.431 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:01.432 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.177128+0000", 2020-08-07T15:10:01.432 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1549999999999998e-06 2020-08-07T15:10:01.432 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.432 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.432 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:01.433 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.177133+0000", 2020-08-07T15:10:01.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.262460639 2020-08-07T15:10:01.438 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.439 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.439 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:01.439 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.439594+0000", 2020-08-07T15:10:01.440 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0943e-05 2020-08-07T15:10:01.440 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.451 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:01.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.439605+0000", 2020-08-07T15:10:01.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013696100000000001 2020-08-07T15:10:01.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:01.453 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.439742+0000", 2020-08-07T15:10:01.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5649000000000003e-05 2020-08-07T15:10:01.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.467 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:01.467 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.439787+0000", 2020-08-07T15:10:01.477 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00063599200000000005 2020-08-07T15:10:01.477 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.477 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.479 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:01.479 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.440423+0000", 2020-08-07T15:10:01.480 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1260000000000001e-05 2020-08-07T15:10:01.480 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.481 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:01.481 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.440465+0000", 2020-08-07T15:10:01.483 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26337036000000003 2020-08-07T15:10:01.484 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.484 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:01.485 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.485 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.486 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.487 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:42280 4.6 e112/81 4:6fc8ffaf:::benchmark_data_smithi074_44586_object5284:head v 112'2464, mlcod=112'2455)", 2020-08-07T15:10:01.487 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.113698+0000", 2020-08-07T15:10:01.487 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 60.799364062999999, 2020-08-07T15:10:01.487 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43055294, 2020-08-07T15:10:01.488 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:01.488 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:01.488 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:01.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.489 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:01.490 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.113698+0000", 2020-08-07T15:10:01.490 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:01.510 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:01.523 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.113698+0000", 2020-08-07T15:10:01.523 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2302000000000003e-05 2020-08-07T15:10:01.524 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.524 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.524 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:01.525 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.113740+0000", 2020-08-07T15:10:01.525 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3669999999999999e-06 2020-08-07T15:10:01.525 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.526 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.526 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:01.526 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.113744+0000", 2020-08-07T15:10:01.527 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.42958231000000002 2020-08-07T15:10:01.527 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.528 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:01.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.543327+0000", 2020-08-07T15:10:01.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2373e-05 2020-08-07T15:10:01.529 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.530 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.531 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:01.531 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.543339+0000", 2020-08-07T15:10:01.531 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014704099999999999 2020-08-07T15:10:01.531 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.532 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.532 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:01.532 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.543486+0000", 2020-08-07T15:10:01.533 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4834999999999999e-05 2020-08-07T15:10:01.533 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.533 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.533 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:01.534 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.543511+0000", 2020-08-07T15:10:01.534 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00069499500000000001 2020-08-07T15:10:01.534 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.534 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:01.535 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.544206+0000", 2020-08-07T15:10:01.535 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4716999999999999e-05 2020-08-07T15:10:01.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.536 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:01.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.544251+0000", 2020-08-07T15:10:01.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43055294 2020-08-07T15:10:01.537 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.538 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:01.538 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.549 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.550 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.550 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:60280 4.3 e121/81 4:c4915982:::benchmark_data_smithi074_44586_object7534:head v 121'3528, mlcod=121'3519)", 2020-08-07T15:10:01.551 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:13.849777+0000", 2020-08-07T15:10:01.551 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 47.063285321000002, 2020-08-07T15:10:01.552 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25718668300000003, 2020-08-07T15:10:01.552 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:01.552 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:01.553 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:01.553 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.553 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:01.555 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:13.849777+0000", 2020-08-07T15:10:01.555 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:01.556 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:01.557 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:13.849777+0000", 2020-08-07T15:10:01.558 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2033999999999997e-05 2020-08-07T15:10:01.558 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.558 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.559 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:01.559 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:13.849829+0000", 2020-08-07T15:10:01.559 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5230000000000001e-06 2020-08-07T15:10:01.560 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.560 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.560 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:01.560 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:13.849835+0000", 2020-08-07T15:10:01.561 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.256119861 2020-08-07T15:10:01.561 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.561 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.564 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:01.565 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.105955+0000", 2020-08-07T15:10:01.565 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.13e-06 2020-08-07T15:10:01.565 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.566 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.579 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:01.580 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.105960+0000", 2020-08-07T15:10:01.580 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8306000000000001e-05 2020-08-07T15:10:01.580 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.581 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.581 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:01.581 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.106008+0000", 2020-08-07T15:10:01.582 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4477e-05 2020-08-07T15:10:01.582 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.582 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.583 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:01.583 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.106033+0000", 2020-08-07T15:10:01.583 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00089500699999999996 2020-08-07T15:10:01.583 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.584 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.584 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:01.584 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.106928+0000", 2020-08-07T15:10:01.585 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5345000000000001e-05 2020-08-07T15:10:01.585 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.609 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:01.609 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.106963+0000", 2020-08-07T15:10:01.610 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25718668300000003 2020-08-07T15:10:01.637 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.641 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:01.646 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.646 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.646 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.647 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:61072 4.b e121/81 4:da2d02c2:::benchmark_data_smithi074_44586_object7633:head v 121'3760, mlcod=121'3755)", 2020-08-07T15:10:01.647 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:14.333829+0000", 2020-08-07T15:10:01.647 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 46.579233117999998, 2020-08-07T15:10:01.648 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25622643699999997, 2020-08-07T15:10:01.648 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:01.653 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:01.653 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:01.653 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.654 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:01.654 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.333829+0000", 2020-08-07T15:10:01.654 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:01.663 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.664 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:01.665 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.333829+0000", 2020-08-07T15:10:01.665 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8536000000000003e-05 2020-08-07T15:10:01.665 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.666 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.666 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:01.666 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.333887+0000", 2020-08-07T15:10:01.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1959999999999996e-06 2020-08-07T15:10:01.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:01.677 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.333893+0000", 2020-08-07T15:10:01.678 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25306922599999998 2020-08-07T15:10:01.678 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.678 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.679 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:01.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.586963+0000", 2020-08-07T15:10:01.679 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6869999999999997e-06 2020-08-07T15:10:01.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.680 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:01.681 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.586969+0000", 2020-08-07T15:10:01.681 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017141300000000001 2020-08-07T15:10:01.681 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.682 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.682 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:01.682 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.587141+0000", 2020-08-07T15:10:01.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5887000000000001e-05 2020-08-07T15:10:01.687 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.687 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.687 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:01.687 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.587167+0000", 2020-08-07T15:10:01.688 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028273999999999999 2020-08-07T15:10:01.688 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.688 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.688 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:01.689 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.589994+0000", 2020-08-07T15:10:01.689 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1092000000000004e-05 2020-08-07T15:10:01.689 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.690 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.690 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:01.690 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.590055+0000", 2020-08-07T15:10:01.691 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25622643699999997 2020-08-07T15:10:01.691 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.691 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:01.692 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.693 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:72952 4.b e130/81 4:dee63f23:::benchmark_data_smithi074_44586_object9118:head v 130'4592, mlcod=130'4584)", 2020-08-07T15:10:01.693 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.589220+0000", 2020-08-07T15:10:01.693 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 35.323841667000003, 2020-08-07T15:10:01.694 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40730996899999999, 2020-08-07T15:10:01.694 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:01.695 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:01.695 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:01.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.696 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:01.696 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.589220+0000", 2020-08-07T15:10:01.697 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:01.697 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:01.698 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.589220+0000", 2020-08-07T15:10:01.698 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9273000000000001e-05 2020-08-07T15:10:01.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.698 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.698 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:01.699 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.589269+0000", 2020-08-07T15:10:01.700 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.417e-06 2020-08-07T15:10:01.700 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.700 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:01.700 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.589275+0000", 2020-08-07T15:10:01.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40641516700000002 2020-08-07T15:10:01.701 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.701 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:01.701 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.995690+0000", 2020-08-07T15:10:01.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7780000000000002e-06 2020-08-07T15:10:01.702 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:01.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.995697+0000", 2020-08-07T15:10:01.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3010999999999999e-05 2020-08-07T15:10:01.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.721 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:01.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.995740+0000", 2020-08-07T15:10:01.726 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1878999999999998e-05 2020-08-07T15:10:01.726 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.726 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.727 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:01.727 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.995762+0000", 2020-08-07T15:10:01.727 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00069480500000000005 2020-08-07T15:10:01.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:01.729 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.996456+0000", 2020-08-07T15:10:01.729 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3639000000000004e-05 2020-08-07T15:10:01.729 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.730 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.730 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:01.731 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.996530+0000", 2020-08-07T15:10:01.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40730996899999999 2020-08-07T15:10:01.731 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.732 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:01.732 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.732 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.733 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:85007 4.6 e139/81 4:6aadd387:::benchmark_data_smithi074_44586_object10625:head v 139'4991, mlcod=139'4981)", 2020-08-07T15:10:01.734 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:36.051294+0000", 2020-08-07T15:10:01.734 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 24.861768099999999, 2020-08-07T15:10:01.734 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28119156699999998, 2020-08-07T15:10:01.735 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:01.735 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:01.735 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:01.735 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.735 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:01.736 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.051294+0000", 2020-08-07T15:10:01.736 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:01.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.736 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.736 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:01.737 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.051294+0000", 2020-08-07T15:10:01.737 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8449000000000003e-05 2020-08-07T15:10:01.737 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.737 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.737 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:01.738 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.051342+0000", 2020-08-07T15:10:01.738 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9329999999999998e-06 2020-08-07T15:10:01.738 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.739 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.739 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:01.739 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.051348+0000", 2020-08-07T15:10:01.740 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28014436300000001 2020-08-07T15:10:01.740 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.740 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.740 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:01.740 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.331492+0000", 2020-08-07T15:10:01.741 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0309999999999998e-06 2020-08-07T15:10:01.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.742 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.742 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:01.742 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.331500+0000", 2020-08-07T15:10:01.742 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6211999999999997e-05 2020-08-07T15:10:01.743 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.743 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:01.744 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.331547+0000", 2020-08-07T15:10:01.744 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6117000000000001e-05 2020-08-07T15:10:01.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:01.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.331573+0000", 2020-08-07T15:10:01.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00085618800000000002 2020-08-07T15:10:01.745 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.745 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:01.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.332429+0000", 2020-08-07T15:10:01.746 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6273999999999997e-05 2020-08-07T15:10:01.746 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.746 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:01.746 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.332485+0000", 2020-08-07T15:10:01.747 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28119156699999998 2020-08-07T15:10:01.747 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.747 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:01.747 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.747 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.748 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.748 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:85008 4.6 e139/81 4:6aadd387:::benchmark_data_smithi074_44586_object10625:head v 139'4992, mlcod=139'4981)", 2020-08-07T15:10:01.748 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:36.051972+0000", 2020-08-07T15:10:01.748 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 24.861089423999999, 2020-08-07T15:10:01.749 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28367819599999999, 2020-08-07T15:10:01.749 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:01.749 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:01.749 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:01.749 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.750 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:01.750 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.051972+0000", 2020-08-07T15:10:01.751 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:01.751 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.752 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:01.752 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.051972+0000", 2020-08-07T15:10:01.752 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1882999999999998e-05 2020-08-07T15:10:01.760 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.761 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:01.761 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.052024+0000", 2020-08-07T15:10:01.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3609999999999997e-06 2020-08-07T15:10:01.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.762 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.762 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:01.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.052031+0000", 2020-08-07T15:10:01.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28155142999999999 2020-08-07T15:10:01.763 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.763 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:01.763 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.333582+0000", 2020-08-07T15:10:01.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8130000000000001e-06 2020-08-07T15:10:01.764 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.764 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:01.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.333588+0000", 2020-08-07T15:10:01.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4005000000000003e-05 2020-08-07T15:10:01.764 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.765 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:01.765 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.333632+0000", 2020-08-07T15:10:01.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2682999999999999e-05 2020-08-07T15:10:01.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.769 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.769 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:01.769 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.333655+0000", 2020-08-07T15:10:01.770 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019567569999999999 2020-08-07T15:10:01.770 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.770 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.770 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:01.770 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.335611+0000", 2020-08-07T15:10:01.771 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9264000000000002e-05 2020-08-07T15:10:01.771 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.771 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.771 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:01.771 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.335651+0000", 2020-08-07T15:10:01.772 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28367819599999999 2020-08-07T15:10:01.772 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.772 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:01.773 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.774 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.774 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.774 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:88160 4.6 e141/81 4:6eb8ba54:::benchmark_data_smithi074_44586_object11019:head v 141'5144, mlcod=141'5136)", 2020-08-07T15:10:01.774 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:37.976442+0000", 2020-08-07T15:10:01.774 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 22.936619456999999, 2020-08-07T15:10:01.775 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45938089599999998, 2020-08-07T15:10:01.775 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:01.775 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:01.775 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:01.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:01.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:37.976442+0000", 2020-08-07T15:10:01.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:01.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:01.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:37.976442+0000", 2020-08-07T15:10:01.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2018999999999998e-05 2020-08-07T15:10:01.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:01.779 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:37.976494+0000", 2020-08-07T15:10:01.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2230000000000004e-06 2020-08-07T15:10:01.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.780 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:01.780 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:37.976501+0000", 2020-08-07T15:10:01.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45734871399999999 2020-08-07T15:10:01.783 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:01.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:38.433849+0000", 2020-08-07T15:10:01.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7040000000000002e-06 2020-08-07T15:10:01.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:01.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:38.433854+0000", 2020-08-07T15:10:01.786 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.371e-05 2020-08-07T15:10:01.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.786 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:01.786 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:38.433898+0000", 2020-08-07T15:10:01.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6178e-05 2020-08-07T15:10:01.787 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.787 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:01.797 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:38.433914+0000", 2020-08-07T15:10:01.798 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0018471399999999999 2020-08-07T15:10:01.798 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.798 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.798 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:01.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:38.435761+0000", 2020-08-07T15:10:01.799 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2207999999999998e-05 2020-08-07T15:10:01.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:01.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:38.435823+0000", 2020-08-07T15:10:01.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45938089599999998 2020-08-07T15:10:01.800 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.800 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:01.800 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.801 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.801 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:104104 4.0 e146/131 4:0ffb764a:::benchmark_data_smithi074_44586_object13012:head v 146'6960, mlcod=146'6953)", 2020-08-07T15:10:01.801 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:10:01.801 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 12.714758946, 2020-08-07T15:10:01.802 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75443331599999996, 2020-08-07T15:10:01.802 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:01.802 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:01.802 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:01.802 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:01.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:10:01.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:01.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:01.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:10:01.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1743000000000002e-05 2020-08-07T15:10:01.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:01.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198355+0000", 2020-08-07T15:10:01.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1249999999999998e-06 2020-08-07T15:10:01.806 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:01.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198361+0000", 2020-08-07T15:10:01.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74719179599999996 2020-08-07T15:10:01.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:01.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945553+0000", 2020-08-07T15:10:01.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6800000000000001e-06 2020-08-07T15:10:01.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.808 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:01.809 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945557+0000", 2020-08-07T15:10:01.809 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012101500000000001 2020-08-07T15:10:01.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.810 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:01.810 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945678+0000", 2020-08-07T15:10:01.810 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6621000000000001e-05 2020-08-07T15:10:01.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:01.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945705+0000", 2020-08-07T15:10:01.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069879540000000002 2020-08-07T15:10:01.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:01.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.952693+0000", 2020-08-07T15:10:01.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3381999999999999e-05 2020-08-07T15:10:01.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:01.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:01.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:01.826 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.952736+0000", 2020-08-07T15:10:01.826 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75443331599999996 2020-08-07T15:10:01.826 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.827 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:01.827 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.827 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:01.827 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:01.827 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:01.831 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:10:01.831 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:10:01.832 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:10:01.832 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:10:01.832 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:10:01.832 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:10:01.832 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:10:01.833 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:10:01.833 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:10:01.833 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:10:01.833 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:10:01.833 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:10:01.834 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:10:01.834 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:10:01.834 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:10:01.835 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:01.828+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:01.835 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:10:01.940 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:01.938+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:01.999 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:01.999 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:10:01.999 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.000 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:125973 4.2 4:4db3dd70:::benchmark_data_smithi074_44586_object9737:head [delete] snapc 0=[] ondisk+write+known_if_redirected e154)", 2020-08-07T15:10:02.000 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:01.845815+0000", 2020-08-07T15:10:02.000 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.15090076699999999, 2020-08-07T15:10:02.000 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.15092451300000001, 2020-08-07T15:10:02.000 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:02.001 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:02.001 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:02.001 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:02.001 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:02.014 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 125973 2020-08-07T15:10:02.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.032 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:02.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:02.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:01.845815+0000", 2020-08-07T15:10:02.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:02.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:02.033 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:01.845815+0000", 2020-08-07T15:10:02.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6280999999999999e-05 2020-08-07T15:10:02.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:02.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:01.845831+0000", 2020-08-07T15:10:02.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4450000000000001e-06 2020-08-07T15:10:02.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:02.037 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:01.845835+0000", 2020-08-07T15:10:02.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0450000000000003e-06 2020-08-07T15:10:02.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.037 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:02.037 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:01.845842+0000", 2020-08-07T15:10:02.038 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2289999999999999e-06 2020-08-07T15:10:02.038 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.038 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:02.038 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:01.845845+0000", 2020-08-07T15:10:02.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5497000000000003e-05 2020-08-07T15:10:02.047 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.048 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:02.048 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:01.845880+0000", 2020-08-07T15:10:02.048 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7585999999999997e-05 2020-08-07T15:10:02.048 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.049 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.049 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:02.049 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:01.845978+0000", 2020-08-07T15:10:02.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3583000000000002e-05 2020-08-07T15:10:02.049 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.054 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:10:02.054 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:01.846071+0000", 2020-08-07T15:10:02.054 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00073563100000000005 2020-08-07T15:10:02.054 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.055 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:02.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:01.846807+0000", 2020-08-07T15:10:02.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1900000000000001e-07 2020-08-07T15:10:02.055 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:02.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:01.846808+0000", 2020-08-07T15:10:02.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6257000000000002e-05 2020-08-07T15:10:02.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:02.057 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:01.846844+0000", 2020-08-07T15:10:02.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0095044080000000006 2020-08-07T15:10:02.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.062 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:02.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:01.856349+0000", 2020-08-07T15:10:02.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.010533781000000001 2020-08-07T15:10:02.063 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:02.068 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:02.069 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:02.069 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:02.070 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:10:02.081 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 1 2020-08-07T15:10:02.082 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:02.084 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:02.082+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:02.085 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:10:02.195 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:02.189+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:02.293 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:02.291+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:02.362 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:02.363 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:10:02.363 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:10:02.363 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:10:02.364 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:02.415 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:10:02.441 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:02.429+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:02.538 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:02.533+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:02.655 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:02.642+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:02.656 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:02.657 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:10:02.657 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:10:02.657 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:10:02.657 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.674 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:15576 3.e e27/20 3:727724a5:::benchmark_data_smithi074_35447_object1946:head v 27'1032, mlcod=27'1023)", 2020-08-07T15:10:02.675 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:10:02.675 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 195.98176845099999, 2020-08-07T15:10:02.675 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.46091973000000003, 2020-08-07T15:10:02.675 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:02.675 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:02.676 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:02.676 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.676 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:02.676 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:10:02.676 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:02.677 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:02.677 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:10:02.677 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9592000000000001e-05 2020-08-07T15:10:02.678 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.678 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.678 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:02.678 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669542+0000", 2020-08-07T15:10:02.678 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.113e-06 2020-08-07T15:10:02.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.679 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:02.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669547+0000", 2020-08-07T15:10:02.679 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45151617799999999 2020-08-07T15:10:02.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.680 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:02.680 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121064+0000", 2020-08-07T15:10:02.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.377e-06 2020-08-07T15:10:02.681 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.681 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:02.694 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121067+0000", 2020-08-07T15:10:02.694 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9195e-05 2020-08-07T15:10:02.695 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.706 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:02.707 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121106+0000", 2020-08-07T15:10:02.707 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7509000000000001e-05 2020-08-07T15:10:02.707 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.707 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:02.708 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121124+0000", 2020-08-07T15:10:02.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0091783100000000003 2020-08-07T15:10:02.708 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.708 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:02.708 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.130302+0000", 2020-08-07T15:10:02.709 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000100456 2020-08-07T15:10:02.709 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.709 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.709 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:02.709 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.130402+0000", 2020-08-07T15:10:02.710 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.46091973000000003 2020-08-07T15:10:02.710 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:02.710 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:02.710 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:02.710 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.711 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:24264 3.8 e28/20 3:1d3fdbda:::benchmark_data_smithi074_35447_object3032:head v 28'1520, mlcod=28'1515)", 2020-08-07T15:10:02.711 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:10:02.711 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 190.26813841399999, 2020-08-07T15:10:02.711 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41856254199999998, 2020-08-07T15:10:02.712 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:02.712 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:02.712 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:02.712 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.712 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:02.713 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:10:02.713 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:02.713 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.713 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.713 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:02.722 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:10:02.722 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1348999999999998e-05 2020-08-07T15:10:02.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.723 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:02.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383174+0000", 2020-08-07T15:10:02.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6579999999999996e-06 2020-08-07T15:10:02.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.724 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.724 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:02.724 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383181+0000", 2020-08-07T15:10:02.724 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41194352699999998 2020-08-07T15:10:02.724 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.725 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:02.725 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.795124+0000", 2020-08-07T15:10:02.725 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5729999999999999e-06 2020-08-07T15:10:02.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.734 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.734 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:02.734 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.795128+0000", 2020-08-07T15:10:02.735 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002170384 2020-08-07T15:10:02.735 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.735 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.735 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:02.735 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.797298+0000", 2020-08-07T15:10:02.736 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5412e-05 2020-08-07T15:10:02.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.736 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.736 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:02.736 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.797324+0000", 2020-08-07T15:10:02.737 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042943360000000002 2020-08-07T15:10:02.737 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.737 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.743 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:02.743 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.801618+0000", 2020-08-07T15:10:02.743 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7302999999999997e-05 2020-08-07T15:10:02.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.748 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.749 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:02.749 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.801675+0000", 2020-08-07T15:10:02.749 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41856254199999998 2020-08-07T15:10:02.749 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:02.749 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:02.750 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:02.750 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.750 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.750 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:93920 3.6 e56/26 ondisk, result = 0)", 2020-08-07T15:10:02.750 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:10:02.751 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 143.32077515099999, 2020-08-07T15:10:02.751 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.397576347, 2020-08-07T15:10:02.754 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:02.754 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:02.754 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:02.755 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:02.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:10:02.755 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:02.755 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:02.756 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:10:02.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2571999999999999e-05 2020-08-07T15:10:02.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.757 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:02.757 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330509+0000", 2020-08-07T15:10:02.757 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2059999999999998e-06 2020-08-07T15:10:02.757 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.771 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.771 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:02.771 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330514+0000", 2020-08-07T15:10:02.771 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39740134599999999 2020-08-07T15:10:02.772 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.772 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.772 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:02.772 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727915+0000", 2020-08-07T15:10:02.772 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.677e-06 2020-08-07T15:10:02.773 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.773 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.773 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:02.773 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727921+0000", 2020-08-07T15:10:02.773 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5431e-05 2020-08-07T15:10:02.774 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.774 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.774 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:02.774 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727956+0000", 2020-08-07T15:10:02.775 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5790000000000004e-06 2020-08-07T15:10:02.775 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.775 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:02.775 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727965+0000", 2020-08-07T15:10:02.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7535999999999997e-05 2020-08-07T15:10:02.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:02.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.728052+0000", 2020-08-07T15:10:02.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.397576347 2020-08-07T15:10:02.777 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:02.777 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:02.777 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:02.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.778 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106104 3.0 3:0d7a2a89:::benchmark_data_smithi074_35447_object13262:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:10:02.778 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.252296+0000", 2020-08-07T15:10:02.778 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 135.39895526199999, 2020-08-07T15:10:02.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.44182183200000003, 2020-08-07T15:10:02.778 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:02.779 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:02.779 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:02.779 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:10:02.779 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:02.779 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106104 2020-08-07T15:10:02.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.782 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:02.782 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.782 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:02.782 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252296+0000", 2020-08-07T15:10:02.783 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:02.783 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:02.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252296+0000", 2020-08-07T15:10:02.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4728000000000001e-05 2020-08-07T15:10:02.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:02.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252321+0000", 2020-08-07T15:10:02.785 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9529999999999999e-06 2020-08-07T15:10:02.785 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:02.786 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252324+0000", 2020-08-07T15:10:02.786 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5759999999999996e-06 2020-08-07T15:10:02.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.786 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:02.787 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252329+0000", 2020-08-07T15:10:02.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.582e-06 2020-08-07T15:10:02.787 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.787 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:02.788 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252332+0000", 2020-08-07T15:10:02.788 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012958519999999999 2020-08-07T15:10:02.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.788 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:02.789 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.253628+0000", 2020-08-07T15:10:02.789 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7871e-05 2020-08-07T15:10:02.789 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.789 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.789 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:02.790 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.253705+0000", 2020-08-07T15:10:02.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017801199999999999 2020-08-07T15:10:02.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.790 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:10:02.791 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.253883+0000", 2020-08-07T15:10:02.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001869051 2020-08-07T15:10:02.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:02.798 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255752+0000", 2020-08-07T15:10:02.798 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3e-07 2020-08-07T15:10:02.798 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:02.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255753+0000", 2020-08-07T15:10:02.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5321000000000001e-05 2020-08-07T15:10:02.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:02.800 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255808+0000", 2020-08-07T15:10:02.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.091917211999999998 2020-08-07T15:10:02.801 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.801 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.801 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:02.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.347726+0000", 2020-08-07T15:10:02.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.34634211500000001 2020-08-07T15:10:02.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.817 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:02.818 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694068+0000", 2020-08-07T15:10:02.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3030000000000001e-06 2020-08-07T15:10:02.818 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.818 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:02.819 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694069+0000", 2020-08-07T15:10:02.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8625999999999998e-05 2020-08-07T15:10:02.819 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:02.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694118+0000", 2020-08-07T15:10:02.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.44182183200000003 2020-08-07T15:10:02.820 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:02.820 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:02.821 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:02.821 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.821 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106104 3.0 e58/20 ondisk, result = 0)", 2020-08-07T15:10:02.821 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.257196+0000", 2020-08-07T15:10:02.822 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 135.39405475500001, 2020-08-07T15:10:02.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43692689000000001, 2020-08-07T15:10:02.822 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:02.822 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:02.822 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:02.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:02.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257196+0000", 2020-08-07T15:10:02.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:02.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:02.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257196+0000", 2020-08-07T15:10:02.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2021999999999999e-05 2020-08-07T15:10:02.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:02.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257228+0000", 2020-08-07T15:10:02.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9549999999999997e-06 2020-08-07T15:10:02.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.832 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:02.832 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257234+0000", 2020-08-07T15:10:02.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43671786400000001 2020-08-07T15:10:02.833 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:02.833 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693952+0000", 2020-08-07T15:10:02.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7690000000000004e-06 2020-08-07T15:10:02.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:02.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693957+0000", 2020-08-07T15:10:02.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3294000000000006e-05 2020-08-07T15:10:02.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:02.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694040+0000", 2020-08-07T15:10:02.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3436e-05 2020-08-07T15:10:02.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.836 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:02.836 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694054+0000", 2020-08-07T15:10:02.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9549999999999996e-05 2020-08-07T15:10:02.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:02.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694123+0000", 2020-08-07T15:10:02.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43692689000000001 2020-08-07T15:10:02.854 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:02.854 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:02.854 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:02.854 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.855 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106144 3.6 3:629055a5:::benchmark_data_smithi074_35447_object13267:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:10:02.855 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.280142+0000", 2020-08-07T15:10:02.855 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 135.37110929299999, 2020-08-07T15:10:02.855 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.57968657199999996, 2020-08-07T15:10:02.856 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:02.856 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:02.856 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:02.856 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:10:02.856 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:02.856 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106144 2020-08-07T15:10:02.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.857 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:02.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:02.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280142+0000", 2020-08-07T15:10:02.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:02.858 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.859 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.859 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:02.859 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280142+0000", 2020-08-07T15:10:02.859 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.905e-05 2020-08-07T15:10:02.860 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.860 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:02.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280191+0000", 2020-08-07T15:10:02.860 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6729999999999997e-06 2020-08-07T15:10:02.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.861 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:02.861 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280197+0000", 2020-08-07T15:10:02.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0747999999999999e-05 2020-08-07T15:10:02.862 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.862 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.862 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:02.862 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280207+0000", 2020-08-07T15:10:02.862 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7700000000000001e-06 2020-08-07T15:10:02.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:02.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280212+0000", 2020-08-07T15:10:02.863 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00060223399999999999 2020-08-07T15:10:02.882 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.882 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.882 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:02.882 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280814+0000", 2020-08-07T15:10:02.883 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7831000000000006e-05 2020-08-07T15:10:02.883 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.883 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.883 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:02.883 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280912+0000", 2020-08-07T15:10:02.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000137119 2020-08-07T15:10:02.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.884 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.884 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:10:02.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.281049+0000", 2020-08-07T15:10:02.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023794369999999999 2020-08-07T15:10:02.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:02.885 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283429+0000", 2020-08-07T15:10:02.898 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.173e-06 2020-08-07T15:10:02.899 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:02.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283430+0000", 2020-08-07T15:10:02.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5530000000000004e-05 2020-08-07T15:10:02.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:02.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283495+0000", 2020-08-07T15:10:02.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5649000000000003e-05 2020-08-07T15:10:02.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:02.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283541+0000", 2020-08-07T15:10:02.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.57624206300000003 2020-08-07T15:10:02.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:02.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859783+0000", 2020-08-07T15:10:02.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.421e-06 2020-08-07T15:10:02.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:02.936 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859785+0000", 2020-08-07T15:10:02.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3874e-05 2020-08-07T15:10:02.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:02.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859828+0000", 2020-08-07T15:10:02.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.57968657199999996 2020-08-07T15:10:02.937 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:02.938 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:02.938 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:02.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.939 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106144 3.6 e59/26 ondisk, result = 0)", 2020-08-07T15:10:02.939 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.344536+0000", 2020-08-07T15:10:02.939 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 135.306714951, 2020-08-07T15:10:02.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51529786099999997, 2020-08-07T15:10:02.940 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:02.940 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:02.940 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:02.940 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.940 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:02.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344536+0000", 2020-08-07T15:10:02.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:02.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:02.942 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344536+0000", 2020-08-07T15:10:02.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7266999999999999e-05 2020-08-07T15:10:02.942 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.942 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:02.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344553+0000", 2020-08-07T15:10:02.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1070000000000001e-06 2020-08-07T15:10:02.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:02.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344557+0000", 2020-08-07T15:10:02.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.515143201 2020-08-07T15:10:02.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:02.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859700+0000", 2020-08-07T15:10:02.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.271e-05 2020-08-07T15:10:02.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:02.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859712+0000", 2020-08-07T15:10:02.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5436999999999998e-05 2020-08-07T15:10:02.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:02.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859768+0000", 2020-08-07T15:10:02.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1540000000000002e-06 2020-08-07T15:10:02.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:02.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859776+0000", 2020-08-07T15:10:02.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7985000000000001e-05 2020-08-07T15:10:02.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:02.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859834+0000", 2020-08-07T15:10:02.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51529786099999997 2020-08-07T15:10:02.975 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:02.975 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:02.975 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:02.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:02.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:02.976 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108352 3.8 3:1aac8373:::benchmark_data_smithi074_35447_object13543:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e62)", 2020-08-07T15:10:02.976 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.642093+0000", 2020-08-07T15:10:02.976 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 132.009158262, 2020-08-07T15:10:02.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39397409900000002, 2020-08-07T15:10:02.977 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:02.977 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:02.977 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:02.977 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:10:03.006 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:03.006 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108352 2020-08-07T15:10:03.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.007 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:03.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:03.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642093+0000", 2020-08-07T15:10:03.008 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:03.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:03.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642093+0000", 2020-08-07T15:10:03.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9475000000000002e-05 2020-08-07T15:10:03.009 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.009 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.010 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:03.010 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642122+0000", 2020-08-07T15:10:03.010 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0439999999999997e-06 2020-08-07T15:10:03.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.011 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:03.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642126+0000", 2020-08-07T15:10:03.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9689999999999994e-06 2020-08-07T15:10:03.012 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.012 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:03.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642134+0000", 2020-08-07T15:10:03.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1219999999999997e-06 2020-08-07T15:10:03.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.013 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:03.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642138+0000", 2020-08-07T15:10:03.014 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011580399999999999 2020-08-07T15:10:03.014 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.014 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:03.014 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.643296+0000", 2020-08-07T15:10:03.015 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3791999999999999e-05 2020-08-07T15:10:03.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.015 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.015 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:03.015 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.643390+0000", 2020-08-07T15:10:03.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011882700000000001 2020-08-07T15:10:03.016 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.016 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4,7", 2020-08-07T15:10:03.017 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.643509+0000", 2020-08-07T15:10:03.017 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034736799999999998 2020-08-07T15:10:03.017 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:03.018 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.646983+0000", 2020-08-07T15:10:03.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0077748490000000003 2020-08-07T15:10:03.018 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.018 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.018 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:03.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.654758+0000", 2020-08-07T15:10:03.019 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000005e-07 2020-08-07T15:10:03.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.019 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.019 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:03.020 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.654758+0000", 2020-08-07T15:10:03.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4258000000000002e-05 2020-08-07T15:10:03.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:03.022 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.654802+0000", 2020-08-07T15:10:03.022 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11166224399999999 2020-08-07T15:10:03.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:03.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.766465+0000", 2020-08-07T15:10:03.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26946740899999999 2020-08-07T15:10:03.023 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:03.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.035932+0000", 2020-08-07T15:10:03.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5829999999999999e-06 2020-08-07T15:10:03.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:03.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.035934+0000", 2020-08-07T15:10:03.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000133306 2020-08-07T15:10:03.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:03.026 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.036067+0000", 2020-08-07T15:10:03.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39397409900000002 2020-08-07T15:10:03.027 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.028 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:03.028 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.028 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145767 3.6 3:62fe5854:::benchmark_data_smithi074_35447_object16158:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:10:03.029 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.654765+0000", 2020-08-07T15:10:03.029 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 107.996486076, 2020-08-07T15:10:03.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.62108287699999998, 2020-08-07T15:10:03.029 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:03.030 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:03.030 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:03.030 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:10:03.030 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:03.032 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145767 2020-08-07T15:10:03.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.033 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:03.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:03.033 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654765+0000", 2020-08-07T15:10:03.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:03.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.034 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.034 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:03.034 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654765+0000", 2020-08-07T15:10:03.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1977999999999998e-05 2020-08-07T15:10:03.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:03.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654797+0000", 2020-08-07T15:10:03.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6799999999999998e-06 2020-08-07T15:10:03.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.037 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:03.037 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654803+0000", 2020-08-07T15:10:03.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1263e-05 2020-08-07T15:10:03.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:03.038 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654814+0000", 2020-08-07T15:10:03.038 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2129999999999999e-06 2020-08-07T15:10:03.038 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.039 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.039 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:03.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654819+0000", 2020-08-07T15:10:03.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0324999999999997e-05 2020-08-07T15:10:03.039 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:03.040 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654869+0000", 2020-08-07T15:10:03.040 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015424199999999999 2020-08-07T15:10:03.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.041 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:03.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.655024+0000", 2020-08-07T15:10:03.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000163573 2020-08-07T15:10:03.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:10:03.044 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.655187+0000", 2020-08-07T15:10:03.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001341645 2020-08-07T15:10:03.045 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.045 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:03.045 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.656529+0000", 2020-08-07T15:10:03.046 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9599999999999998e-07 2020-08-07T15:10:03.046 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.046 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.046 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:03.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.656530+0000", 2020-08-07T15:10:03.047 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4381e-05 2020-08-07T15:10:03.047 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.047 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:03.047 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.656584+0000", 2020-08-07T15:10:03.048 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0057298979999999998 2020-08-07T15:10:03.048 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.048 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.048 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:03.048 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.662314+0000", 2020-08-07T15:10:03.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.61347776300000001 2020-08-07T15:10:03.049 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.049 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.049 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:03.050 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275792+0000", 2020-08-07T15:10:03.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9740000000000001e-06 2020-08-07T15:10:03.050 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.050 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.050 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:03.051 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275794+0000", 2020-08-07T15:10:03.051 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4146e-05 2020-08-07T15:10:03.051 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.051 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:03.052 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275848+0000", 2020-08-07T15:10:03.052 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.62108287699999998 2020-08-07T15:10:03.052 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.052 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:03.052 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.053 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.053 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145801 3.6 3:69e4cb22:::benchmark_data_smithi074_35447_object16192:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:10:03.053 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671123+0000", 2020-08-07T15:10:03.054 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 107.980128201, 2020-08-07T15:10:03.054 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74411769999999999, 2020-08-07T15:10:03.054 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:03.055 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:03.055 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:03.055 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:10:03.055 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:03.055 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145801 2020-08-07T15:10:03.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.056 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:03.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:03.057 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671123+0000", 2020-08-07T15:10:03.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:03.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.058 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.058 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:03.058 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671123+0000", 2020-08-07T15:10:03.058 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3691e-05 2020-08-07T15:10:03.059 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.059 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:03.060 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671157+0000", 2020-08-07T15:10:03.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0249999999999999e-06 2020-08-07T15:10:03.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.061 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:03.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671163+0000", 2020-08-07T15:10:03.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2339e-05 2020-08-07T15:10:03.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.062 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.062 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:03.062 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671175+0000", 2020-08-07T15:10:03.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5249999999999996e-06 2020-08-07T15:10:03.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.063 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:03.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671180+0000", 2020-08-07T15:10:03.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6826000000000002e-05 2020-08-07T15:10:03.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.064 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:03.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671227+0000", 2020-08-07T15:10:03.065 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014152699999999999 2020-08-07T15:10:03.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.065 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:03.066 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671369+0000", 2020-08-07T15:10:03.066 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8021000000000002e-05 2020-08-07T15:10:03.067 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.067 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.067 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:10:03.067 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671467+0000", 2020-08-07T15:10:03.067 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0016293760000000001 2020-08-07T15:10:03.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.068 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.068 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:03.068 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673096+0000", 2020-08-07T15:10:03.068 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.048e-06 2020-08-07T15:10:03.069 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.069 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.069 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:03.069 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673097+0000", 2020-08-07T15:10:03.070 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8841999999999998e-05 2020-08-07T15:10:03.070 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.070 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.070 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:03.070 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673146+0000", 2020-08-07T15:10:03.071 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002829935 2020-08-07T15:10:03.071 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.071 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.071 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:03.071 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675976+0000", 2020-08-07T15:10:03.072 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73917870399999996 2020-08-07T15:10:03.072 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.072 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.072 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:03.072 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.415155+0000", 2020-08-07T15:10:03.073 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.193e-06 2020-08-07T15:10:03.073 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.073 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.073 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:03.074 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.415157+0000", 2020-08-07T15:10:03.074 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3647999999999996e-05 2020-08-07T15:10:03.074 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.074 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.074 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:03.075 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.415241+0000", 2020-08-07T15:10:03.075 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74411769999999999 2020-08-07T15:10:03.075 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.075 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:03.075 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.076 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.076 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.076 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145803 3.0 3:0c939450:::benchmark_data_smithi074_35447_object16194:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:10:03.077 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671678+0000", 2020-08-07T15:10:03.077 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 107.979573406, 2020-08-07T15:10:03.078 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.882157211, 2020-08-07T15:10:03.078 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:03.078 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:03.078 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:03.079 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:10:03.079 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:03.079 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145803 2020-08-07T15:10:03.079 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.079 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:03.080 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.080 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:03.080 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671678+0000", 2020-08-07T15:10:03.080 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:03.080 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.081 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.081 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:03.081 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671678+0000", 2020-08-07T15:10:03.081 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4097999999999998e-05 2020-08-07T15:10:03.081 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.082 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:03.082 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671712+0000", 2020-08-07T15:10:03.082 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2439999999999998e-06 2020-08-07T15:10:03.082 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.083 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.083 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:03.083 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671718+0000", 2020-08-07T15:10:03.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2629999999999999e-05 2020-08-07T15:10:03.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.084 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:03.084 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671731+0000", 2020-08-07T15:10:03.084 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9589999999999998e-06 2020-08-07T15:10:03.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.085 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:03.085 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671738+0000", 2020-08-07T15:10:03.085 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2495000000000004e-05 2020-08-07T15:10:03.086 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.086 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.086 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:03.086 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671800+0000", 2020-08-07T15:10:03.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013640700000000001 2020-08-07T15:10:03.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:03.088 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671937+0000", 2020-08-07T15:10:03.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013672200000000001 2020-08-07T15:10:03.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:10:03.089 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.672073+0000", 2020-08-07T15:10:03.089 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001179982 2020-08-07T15:10:03.089 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.089 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.090 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:03.090 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673253+0000", 2020-08-07T15:10:03.090 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1900000000000001e-07 2020-08-07T15:10:03.090 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.090 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.091 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:03.091 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673254+0000", 2020-08-07T15:10:03.091 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2104000000000001e-05 2020-08-07T15:10:03.091 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.091 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.092 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:03.092 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673306+0000", 2020-08-07T15:10:03.092 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002264369 2020-08-07T15:10:03.092 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.092 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.093 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:03.094 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675571+0000", 2020-08-07T15:10:03.094 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.87820796899999998 2020-08-07T15:10:03.094 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.094 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.094 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:03.095 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553778+0000", 2020-08-07T15:10:03.095 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3629999999999999e-06 2020-08-07T15:10:03.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.095 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.095 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:03.096 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553780+0000", 2020-08-07T15:10:03.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5050000000000003e-05 2020-08-07T15:10:03.096 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.096 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.096 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:03.097 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553835+0000", 2020-08-07T15:10:03.097 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.882157211 2020-08-07T15:10:03.097 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.097 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:03.097 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.098 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.098 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.098 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145805 3.8 3:13b8e273:::benchmark_data_smithi074_35447_object16196:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:10:03.098 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671847+0000", 2020-08-07T15:10:03.099 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 107.979404086, 2020-08-07T15:10:03.099 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.340134453, 2020-08-07T15:10:03.099 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:03.099 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:03.099 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:03.100 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:10:03.100 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:03.100 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145805 2020-08-07T15:10:03.100 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.100 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:03.101 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.101 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:03.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671847+0000", 2020-08-07T15:10:03.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:03.113 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.113 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:03.113 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671847+0000", 2020-08-07T15:10:03.114 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1139999999999997e-05 2020-08-07T15:10:03.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.114 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:03.114 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671878+0000", 2020-08-07T15:10:03.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.784e-06 2020-08-07T15:10:03.115 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.115 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.115 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:03.124 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671884+0000", 2020-08-07T15:10:03.125 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2802999999999999e-05 2020-08-07T15:10:03.125 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.125 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.125 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:03.126 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671897+0000", 2020-08-07T15:10:03.126 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9909999999999999e-06 2020-08-07T15:10:03.126 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.126 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.126 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:03.147 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671903+0000", 2020-08-07T15:10:03.149 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011861 2020-08-07T15:10:03.149 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.149 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.150 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:03.150 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.672021+0000", 2020-08-07T15:10:03.150 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017702399999999999 2020-08-07T15:10:03.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.151 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:03.151 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.672198+0000", 2020-08-07T15:10:03.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020546400000000001 2020-08-07T15:10:03.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.153 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4,7", 2020-08-07T15:10:03.186 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.672404+0000", 2020-08-07T15:10:03.186 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024385019999999999 2020-08-07T15:10:03.187 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.187 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.206 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:03.206 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674842+0000", 2020-08-07T15:10:03.206 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00028594999999999998 2020-08-07T15:10:03.207 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.207 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.207 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:03.207 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675128+0000", 2020-08-07T15:10:03.207 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8100000000000003e-07 2020-08-07T15:10:03.208 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.208 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.208 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:03.208 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675129+0000", 2020-08-07T15:10:03.208 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7973999999999997e-05 2020-08-07T15:10:03.209 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.209 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.209 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:03.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675197+0000", 2020-08-07T15:10:03.209 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032171220000000002 2020-08-07T15:10:03.247 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.248 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.248 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:03.248 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678414+0000", 2020-08-07T15:10:03.249 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33343769299999998 2020-08-07T15:10:03.249 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.249 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.250 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:03.298 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011852+0000", 2020-08-07T15:10:03.298 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3480000000000001e-06 2020-08-07T15:10:03.298 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.299 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.299 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:03.299 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011853+0000", 2020-08-07T15:10:03.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000128467 2020-08-07T15:10:03.299 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.300 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.300 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:03.300 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011981+0000", 2020-08-07T15:10:03.300 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.340134453 2020-08-07T15:10:03.300 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.301 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:03.301 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.301 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.301 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.301 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145806 3.8 3:16669ddb:::benchmark_data_smithi074_35447_object16197:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:10:03.311 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.673992+0000", 2020-08-07T15:10:03.311 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 107.97725955999999, 2020-08-07T15:10:03.311 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40687891900000001, 2020-08-07T15:10:03.311 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:03.311 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:03.312 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:03.312 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:10:03.312 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:03.312 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145806 2020-08-07T15:10:03.312 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.313 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:03.313 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.313 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:03.313 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673992+0000", 2020-08-07T15:10:03.313 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:03.318 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.318 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.318 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:03.318 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673992+0000", 2020-08-07T15:10:03.319 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5429000000000002e-05 2020-08-07T15:10:03.319 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.319 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.319 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:03.319 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674027+0000", 2020-08-07T15:10:03.320 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3199999999999996e-06 2020-08-07T15:10:03.320 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.324 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.324 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:03.324 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674033+0000", 2020-08-07T15:10:03.325 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2835e-05 2020-08-07T15:10:03.325 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.325 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.325 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:03.325 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674046+0000", 2020-08-07T15:10:03.326 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9030000000000001e-06 2020-08-07T15:10:03.326 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.326 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.326 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:03.326 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674052+0000", 2020-08-07T15:10:03.327 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4181000000000003e-05 2020-08-07T15:10:03.327 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.327 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.327 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:03.327 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674096+0000", 2020-08-07T15:10:03.328 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00022310599999999999 2020-08-07T15:10:03.328 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.328 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.328 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:03.328 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674319+0000", 2020-08-07T15:10:03.329 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012681300000000001 2020-08-07T15:10:03.329 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.329 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.329 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4,7", 2020-08-07T15:10:03.329 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674446+0000", 2020-08-07T15:10:03.330 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002882754 2020-08-07T15:10:03.330 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.331 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.331 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:03.331 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.677329+0000", 2020-08-07T15:10:03.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9466e-05 2020-08-07T15:10:03.331 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.332 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.332 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:03.332 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.677418+0000", 2020-08-07T15:10:03.332 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0299999999999998e-07 2020-08-07T15:10:03.332 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.333 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.333 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:03.333 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.677419+0000", 2020-08-07T15:10:03.333 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5274000000000001e-05 2020-08-07T15:10:03.333 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.334 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.334 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:03.335 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.677464+0000", 2020-08-07T15:10:03.335 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40011127099999999 2020-08-07T15:10:03.335 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.335 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:03.336 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.077576+0000", 2020-08-07T15:10:03.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0031778370000000002 2020-08-07T15:10:03.336 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.336 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.336 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:03.337 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080754+0000", 2020-08-07T15:10:03.337 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.92e-07 2020-08-07T15:10:03.337 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.337 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.337 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:03.338 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080754+0000", 2020-08-07T15:10:03.338 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000116135 2020-08-07T15:10:03.338 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.338 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.338 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:03.339 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080870+0000", 2020-08-07T15:10:03.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40687891900000001 2020-08-07T15:10:03.339 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.339 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:03.339 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.339 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.340 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:145767 3.6 e75/26 ondisk, result = 0)", 2020-08-07T15:10:03.340 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.674108+0000", 2020-08-07T15:10:03.340 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 107.977143136, 2020-08-07T15:10:03.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.60174556899999998, 2020-08-07T15:10:03.341 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:03.341 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:03.341 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:03.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.341 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:03.342 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674108+0000", 2020-08-07T15:10:03.342 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:03.342 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.342 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.342 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:03.343 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674108+0000", 2020-08-07T15:10:03.343 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0558000000000001e-05 2020-08-07T15:10:03.343 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.343 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.343 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:03.344 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674139+0000", 2020-08-07T15:10:03.344 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6620000000000002e-06 2020-08-07T15:10:03.344 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.345 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.345 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:03.346 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674144+0000", 2020-08-07T15:10:03.346 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.60156723899999998 2020-08-07T15:10:03.346 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.346 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.346 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:03.347 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275711+0000", 2020-08-07T15:10:03.347 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4070000000000004e-06 2020-08-07T15:10:03.347 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.347 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:03.348 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275719+0000", 2020-08-07T15:10:03.348 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3205999999999998e-05 2020-08-07T15:10:03.348 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.348 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.348 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:03.349 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275772+0000", 2020-08-07T15:10:03.350 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0624e-05 2020-08-07T15:10:03.350 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.350 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.351 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:03.351 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275783+0000", 2020-08-07T15:10:03.351 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0872999999999997e-05 2020-08-07T15:10:03.352 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.352 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.352 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:03.352 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275854+0000", 2020-08-07T15:10:03.352 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.60174556899999998 2020-08-07T15:10:03.353 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.353 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:03.353 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.353 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.354 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.354 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:145803 3.0 e75/20 ondisk, result = 0)", 2020-08-07T15:10:03.354 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.674488+0000", 2020-08-07T15:10:03.354 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 107.976763154, 2020-08-07T15:10:03.354 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.87935229699999995, 2020-08-07T15:10:03.355 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:03.356 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:03.356 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:03.357 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.357 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:03.357 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674488+0000", 2020-08-07T15:10:03.357 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:03.358 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.358 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.358 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:03.358 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674488+0000", 2020-08-07T15:10:03.358 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3062000000000003e-05 2020-08-07T15:10:03.359 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.359 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.359 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:03.359 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674521+0000", 2020-08-07T15:10:03.360 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4911e-05 2020-08-07T15:10:03.360 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.360 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.360 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:03.360 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674536+0000", 2020-08-07T15:10:03.361 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.87914350600000002 2020-08-07T15:10:03.361 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.361 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.361 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:03.361 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553679+0000", 2020-08-07T15:10:03.367 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5950000000000001e-06 2020-08-07T15:10:03.367 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.367 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:03.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553687+0000", 2020-08-07T15:10:03.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2565999999999996e-05 2020-08-07T15:10:03.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:03.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553760+0000", 2020-08-07T15:10:03.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0417999999999999e-05 2020-08-07T15:10:03.379 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:03.380 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553770+0000", 2020-08-07T15:10:03.380 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0239000000000003e-05 2020-08-07T15:10:03.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:03.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553840+0000", 2020-08-07T15:10:03.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.87935229699999995 2020-08-07T15:10:03.381 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.381 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:03.381 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.398 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.399 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:145805 3.8 e75/60 ondisk, result = 0)", 2020-08-07T15:10:03.399 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.676843+0000", 2020-08-07T15:10:03.399 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 107.974407684, 2020-08-07T15:10:03.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.335144041, 2020-08-07T15:10:03.400 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:03.400 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:03.400 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:03.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.401 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:03.401 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.676843+0000", 2020-08-07T15:10:03.401 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:03.401 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.403 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.403 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:03.403 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.676843+0000", 2020-08-07T15:10:03.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7853e-05 2020-08-07T15:10:03.404 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.404 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.404 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:03.404 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.676861+0000", 2020-08-07T15:10:03.405 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7359999999999999e-06 2020-08-07T15:10:03.405 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.405 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:03.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.676865+0000", 2020-08-07T15:10:03.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33484658099999998 2020-08-07T15:10:03.431 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.431 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:03.431 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011712+0000", 2020-08-07T15:10:03.432 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1613e-05 2020-08-07T15:10:03.432 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.432 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.432 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:03.432 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011723+0000", 2020-08-07T15:10:03.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011038 2020-08-07T15:10:03.439 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.439 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.440 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:03.440 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011834+0000", 2020-08-07T15:10:03.440 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0097e-05 2020-08-07T15:10:03.440 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.441 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:03.441 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011844+0000", 2020-08-07T15:10:03.441 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000143781 2020-08-07T15:10:03.441 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.441 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.686 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:03.686 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011987+0000", 2020-08-07T15:10:03.686 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.335144041 2020-08-07T15:10:03.687 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.687 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:03.687 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.687 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.687 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.688 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:145806 3.8 e75/60 ondisk, result = 0)", 2020-08-07T15:10:03.688 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.678157+0000", 2020-08-07T15:10:03.688 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 107.973093632, 2020-08-07T15:10:03.688 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.402718152, 2020-08-07T15:10:03.688 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:03.689 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:03.689 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:03.689 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.689 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:03.689 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678157+0000", 2020-08-07T15:10:03.690 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:03.690 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.690 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.690 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:03.690 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678157+0000", 2020-08-07T15:10:03.691 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8496e-05 2020-08-07T15:10:03.691 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.691 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.691 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:03.691 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678176+0000", 2020-08-07T15:10:03.691 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.676e-06 2020-08-07T15:10:03.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.692 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:03.692 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678180+0000", 2020-08-07T15:10:03.692 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40250966999999999 2020-08-07T15:10:03.693 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.693 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.693 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:03.693 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080689+0000", 2020-08-07T15:10:03.712 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7110000000000003e-06 2020-08-07T15:10:03.713 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.713 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.713 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:03.713 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080694+0000", 2020-08-07T15:10:03.714 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4122999999999998e-05 2020-08-07T15:10:03.714 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.714 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.714 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:03.714 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080738+0000", 2020-08-07T15:10:03.715 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5809999999999999e-06 2020-08-07T15:10:03.715 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.715 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.715 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:03.715 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080746+0000", 2020-08-07T15:10:03.716 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012989499999999999 2020-08-07T15:10:03.716 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.730 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.731 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:03.731 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080876+0000", 2020-08-07T15:10:03.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.402718152 2020-08-07T15:10:03.732 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.732 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:03.732 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.733 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.733 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42288 4.2 4:4cfde355:::benchmark_data_smithi074_44586_object5285:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:10:03.757 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.115516+0000", 2020-08-07T15:10:03.758 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 62.535735533999997, 2020-08-07T15:10:03.760 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.53902283500000003, 2020-08-07T15:10:03.761 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:03.761 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:03.762 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:03.762 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:03.779 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:03.789 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42288 2020-08-07T15:10:03.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.790 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:03.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:03.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.115516+0000", 2020-08-07T15:10:03.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:03.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:03.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.115516+0000", 2020-08-07T15:10:03.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5785999999999997e-05 2020-08-07T15:10:03.801 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.801 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.801 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:03.802 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.115571+0000", 2020-08-07T15:10:03.802 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3130000000000004e-06 2020-08-07T15:10:03.802 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.802 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.802 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:03.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.115578+0000", 2020-08-07T15:10:03.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2514000000000001e-05 2020-08-07T15:10:03.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:03.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.115590+0000", 2020-08-07T15:10:03.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7130000000000002e-06 2020-08-07T15:10:03.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.827 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:03.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.115596+0000", 2020-08-07T15:10:03.828 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0036423369999999998 2020-08-07T15:10:03.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.829 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:03.829 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.119238+0000", 2020-08-07T15:10:03.829 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7732999999999995e-05 2020-08-07T15:10:03.830 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.830 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.830 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:03.831 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.119326+0000", 2020-08-07T15:10:03.831 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000102612 2020-08-07T15:10:03.831 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.832 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:10:03.832 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.119429+0000", 2020-08-07T15:10:03.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001842621 2020-08-07T15:10:03.833 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:03.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.121271+0000", 2020-08-07T15:10:03.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00096829399999999997 2020-08-07T15:10:03.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.836 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:03.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.122239+0000", 2020-08-07T15:10:03.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.53219874700000003 2020-08-07T15:10:03.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.838 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:03.838 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.654438+0000", 2020-08-07T15:10:03.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7290000000000001e-06 2020-08-07T15:10:03.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.840 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:03.840 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.654440+0000", 2020-08-07T15:10:03.840 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0956999999999999e-05 2020-08-07T15:10:03.841 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:03.842 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.654501+0000", 2020-08-07T15:10:03.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7478999999999998e-05 2020-08-07T15:10:03.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.843 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:03.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.654538+0000", 2020-08-07T15:10:03.844 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.53902283500000003 2020-08-07T15:10:03.844 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.844 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:03.844 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.845 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:66488 4.f 4:fdf15c0e:::benchmark_data_smithi074_44586_object8310:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e126)", 2020-08-07T15:10:03.846 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:20.213948+0000", 2020-08-07T15:10:03.846 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 42.437302825000003, 2020-08-07T15:10:03.846 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.34743775999999998, 2020-08-07T15:10:03.846 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:03.847 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:03.847 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:03.847 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:03.847 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:03.858 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 66488 2020-08-07T15:10:03.858 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.859 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:03.859 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:03.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.213948+0000", 2020-08-07T15:10:03.860 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:03.860 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.861 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:03.861 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.213948+0000", 2020-08-07T15:10:03.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0775999999999997e-05 2020-08-07T15:10:03.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.862 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.862 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:03.862 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.213989+0000", 2020-08-07T15:10:03.862 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5179999999999998e-06 2020-08-07T15:10:03.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:03.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.213994+0000", 2020-08-07T15:10:03.863 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7399999999999993e-06 2020-08-07T15:10:03.864 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.868 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.868 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:03.869 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.214002+0000", 2020-08-07T15:10:03.869 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5640000000000001e-06 2020-08-07T15:10:03.869 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.869 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.870 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:03.870 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.214006+0000", 2020-08-07T15:10:03.870 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0035647840000000001 2020-08-07T15:10:03.870 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.871 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:03.871 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.217571+0000", 2020-08-07T15:10:03.871 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000155446 2020-08-07T15:10:03.871 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.872 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.872 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:03.872 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.217726+0000", 2020-08-07T15:10:03.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00026276899999999999 2020-08-07T15:10:03.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.882 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.883 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4", 2020-08-07T15:10:03.883 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.217989+0000", 2020-08-07T15:10:03.883 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001388161 2020-08-07T15:10:03.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.884 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.884 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:03.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.219377+0000", 2020-08-07T15:10:03.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9100000000000002e-07 2020-08-07T15:10:03.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:03.885 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.219378+0000", 2020-08-07T15:10:03.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7027999999999998e-05 2020-08-07T15:10:03.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.886 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:03.887 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.219425+0000", 2020-08-07T15:10:03.887 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054840820000000004 2020-08-07T15:10:03.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:03.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.224909+0000", 2020-08-07T15:10:03.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33640654599999997 2020-08-07T15:10:03.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.890 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:03.890 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.561316+0000", 2020-08-07T15:10:03.890 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.127e-06 2020-08-07T15:10:03.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.891 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.891 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:03.891 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.561317+0000", 2020-08-07T15:10:03.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9327999999999998e-05 2020-08-07T15:10:03.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.894 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:03.894 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.561386+0000", 2020-08-07T15:10:03.894 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.34743775999999998 2020-08-07T15:10:03.894 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.895 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:03.895 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.895 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.895 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.895 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72952 4.b 4:dee63f23:::benchmark_data_smithi074_44586_object9118:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:03.896 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.585664+0000", 2020-08-07T15:10:03.896 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 37.065587116000003, 2020-08-07T15:10:03.896 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.411271632, 2020-08-07T15:10:03.896 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:03.896 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:03.897 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:03.897 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:03.898 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:03.898 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72952 2020-08-07T15:10:03.898 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.899 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:03.899 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.899 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:03.900 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.585664+0000", 2020-08-07T15:10:03.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:03.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.900 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:03.901 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.585664+0000", 2020-08-07T15:10:03.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0828999999999997e-05 2020-08-07T15:10:03.901 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.901 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.901 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:03.902 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.585715+0000", 2020-08-07T15:10:03.902 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1019999999999996e-06 2020-08-07T15:10:03.902 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.902 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.902 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:03.903 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.585720+0000", 2020-08-07T15:10:03.903 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4185000000000001e-05 2020-08-07T15:10:03.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.904 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:03.904 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.585734+0000", 2020-08-07T15:10:03.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5009999999999998e-06 2020-08-07T15:10:03.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:03.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.585742+0000", 2020-08-07T15:10:03.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0027632669999999998 2020-08-07T15:10:03.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.906 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.906 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:03.906 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.588505+0000", 2020-08-07T15:10:03.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4177000000000003e-05 2020-08-07T15:10:03.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.908 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:03.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.588599+0000", 2020-08-07T15:10:03.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013466500000000001 2020-08-07T15:10:03.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.910 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:10:03.910 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.588734+0000", 2020-08-07T15:10:03.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001231181 2020-08-07T15:10:03.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:03.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.589965+0000", 2020-08-07T15:10:03.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0083227890000000006 2020-08-07T15:10:03.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:03.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.598288+0000", 2020-08-07T15:10:03.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39851268299999998 2020-08-07T15:10:03.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:03.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.996800+0000", 2020-08-07T15:10:03.914 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.821e-06 2020-08-07T15:10:03.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:03.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.996802+0000", 2020-08-07T15:10:03.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8633000000000004e-05 2020-08-07T15:10:03.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:03.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.996881+0000", 2020-08-07T15:10:03.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4799000000000002e-05 2020-08-07T15:10:03.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:03.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:03.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:03.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.996936+0000", 2020-08-07T15:10:03.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.411271632 2020-08-07T15:10:03.917 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.918 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:03.918 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.919 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:03.919 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:03.929 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:03.934 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:10:03.935 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:10:03.938 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:10:03.939 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:03.932+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:04.038 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:04.036+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:04.139 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:04.136+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:04.143 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:04.143 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:10:04.143 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:10:04.143 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:04.153 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:10:04.154 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:10:04.180 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:10:04.248 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:10:04.249 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:04.247+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:04.307 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:10:04.351 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:04.349+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:04.360 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:10:04.416 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:10:04.431 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:04.432 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:04.446 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:10:04.457 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:04.455+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:04.472 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:10:04.527 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:10:04.528 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:10:04.528 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:10:04.528 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:10:04.528 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:10:04.529 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:10:04.529 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:10:04.529 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:10:04.529 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:10:04.562 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:04.560+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:04.676 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:04.674+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:04.691 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:04.691 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:04.704 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:10:04.741 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:10:04.742 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 0 1 2020-08-07T15:10:04.783 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:04.779+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:04.889 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:04.885+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:04.981 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:04.982 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:04.989 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:04.987+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:04.995 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:10:05.102 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:05.100+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:05.189 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:05.189 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:05.190 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:10:05.207 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:10:05.208 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:05.205+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:05.308 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:05.307+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:05.335 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:05.336 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:05.352 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:10:05.411 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:05.410+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:05.496 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:05.496 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:05.497 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:10:05.539 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:05.536+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:05.624 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:05.623+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:05.657 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:05.673 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:05.675 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Removed 14530 objects 2020-08-07T15:10:05.675 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Clean up completed and total clean up time :8.37202 2020-08-07T15:10:05.683 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:10:05.763 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:05.760+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:05.833 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:05.834 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:05.834 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:05.832+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:05.853 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:10:05.986 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:05.962+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:06.010 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:06.016 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:06.082 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:10:06.104 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:06.103+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:06.203 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.0 to 1 (10000) 2020-08-07T15:10:06.213 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:06.210+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:06.215 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 1 1 2020-08-07T15:10:06.226 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:06.229 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:06.235 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:10:06.326 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:06.324+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:06.399 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:06.399 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:06.410 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:10:06.420 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:06.413+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:06.520 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:06.517+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:06.562 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:10:06.579 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:06.579 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:06.606 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:10:06.622 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:06.620+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:06.705 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:10:06.733 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:10:06.734 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:06.723+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:06.746 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:06.747 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:06.770 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:10:06.772 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:10:06.782 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:10:06.807 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:10:06.828 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:06.826+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:06.834 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:10:06.859 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:10:06.882 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:10:06.894 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:10:06.895 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:10:06.895 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:10:06.896 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:10:06.896 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:10:06.896 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:10:06.897 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:10:06.897 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:10:06.929 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:06.928+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:06.931 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:10:06.932 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:10:06.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:10:06.933 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:10:06.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:06.933 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86695 3.a 3:5058ba7c:::benchmark_data_smithi074_35447_object10836:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:10:06.942 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:10:06.942 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 152.60311575599999, 2020-08-07T15:10:06.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31490022699999998, 2020-08-07T15:10:06.943 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:06.943 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:06.944 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:06.944 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:10:06.944 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:06.945 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86695 2020-08-07T15:10:06.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:06.945 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:06.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:06.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:06.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:10:06.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:06.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:06.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:06.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:06.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:10:06.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4560000000000005e-05 2020-08-07T15:10:06.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:06.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:06.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:06.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325392+0000", 2020-08-07T15:10:06.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3180000000000003e-06 2020-08-07T15:10:06.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:06.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:06.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:06.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325398+0000", 2020-08-07T15:10:06.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.258e-05 2020-08-07T15:10:06.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:06.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:06.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:06.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325411+0000", 2020-08-07T15:10:06.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6550000000000001e-06 2020-08-07T15:10:06.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:06.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:06.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:07.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325417+0000", 2020-08-07T15:10:07.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024282879999999998 2020-08-07T15:10:07.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:07.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.327845+0000", 2020-08-07T15:10:07.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1203000000000001e-05 2020-08-07T15:10:07.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:07.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.327936+0000", 2020-08-07T15:10:07.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000186505 2020-08-07T15:10:07.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:10:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328123+0000", 2020-08-07T15:10:07.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020062890000000001 2020-08-07T15:10:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:07.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.330129+0000", 2020-08-07T15:10:07.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121513 2020-08-07T15:10:07.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:07.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.330250+0000", 2020-08-07T15:10:07.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30989746899999998 2020-08-07T15:10:07.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:07.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640148+0000", 2020-08-07T15:10:07.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2139999999999999e-06 2020-08-07T15:10:07.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:07.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640149+0000", 2020-08-07T15:10:07.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8081999999999998e-05 2020-08-07T15:10:07.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:07.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640197+0000", 2020-08-07T15:10:07.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0550999999999999e-05 2020-08-07T15:10:07.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:07.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640228+0000", 2020-08-07T15:10:07.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31490022699999998 2020-08-07T15:10:07.114 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:07.114 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:07.114 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:07.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.115 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86696 3.a 3:5058ba7c:::benchmark_data_smithi074_35447_object10836:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:10:07.115 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:10:07.115 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 152.60299107700001, 2020-08-07T15:10:07.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.347347556, 2020-08-07T15:10:07.116 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:07.116 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:07.116 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:07.116 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:10:07.117 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:07.117 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86696 2020-08-07T15:10:07.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.117 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:07.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:07.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:10:07.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:07.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:07.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:10:07.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0436999999999998e-05 2020-08-07T15:10:07.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:07.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325503+0000", 2020-08-07T15:10:07.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4659999999999998e-06 2020-08-07T15:10:07.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:07.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325508+0000", 2020-08-07T15:10:07.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1177e-05 2020-08-07T15:10:07.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:07.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325519+0000", 2020-08-07T15:10:07.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7809999999999996e-06 2020-08-07T15:10:07.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:07.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325524+0000", 2020-08-07T15:10:07.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002989219 2020-08-07T15:10:07.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:07.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328513+0000", 2020-08-07T15:10:07.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0649000000000006e-05 2020-08-07T15:10:07.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:07.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328604+0000", 2020-08-07T15:10:07.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001652 2020-08-07T15:10:07.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:10:07.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328769+0000", 2020-08-07T15:10:07.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023470639999999998 2020-08-07T15:10:07.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:07.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331116+0000", 2020-08-07T15:10:07.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6099999999999999e-07 2020-08-07T15:10:07.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:07.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331117+0000", 2020-08-07T15:10:07.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9150000000000001e-05 2020-08-07T15:10:07.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:07.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331176+0000", 2020-08-07T15:10:07.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065402581000000001 2020-08-07T15:10:07.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:07.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.396579+0000", 2020-08-07T15:10:07.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27618104599999999 2020-08-07T15:10:07.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:07.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672760+0000", 2020-08-07T15:10:07.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2020-08-07T15:10:07.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:07.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672761+0000", 2020-08-07T15:10:07.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9211000000000001e-05 2020-08-07T15:10:07.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:07.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672800+0000", 2020-08-07T15:10:07.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.347347556 2020-08-07T15:10:07.298 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:07.298 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:07.321 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:07.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.322 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:86695 3.a e52/20 ondisk, result = 0)", 2020-08-07T15:10:07.322 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:10:07.323 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 152.59503437999999, 2020-08-07T15:10:07.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.306823501, 2020-08-07T15:10:07.323 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:07.323 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:07.323 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:07.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:07.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:10:07.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:07.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:07.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:10:07.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.048e-05 2020-08-07T15:10:07.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:07.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333439+0000", 2020-08-07T15:10:07.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9990000000000002e-06 2020-08-07T15:10:07.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:07.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333443+0000", 2020-08-07T15:10:07.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30664195100000002 2020-08-07T15:10:07.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:07.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640085+0000", 2020-08-07T15:10:07.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8559999999999998e-06 2020-08-07T15:10:07.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:07.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640090+0000", 2020-08-07T15:10:07.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5166000000000001e-05 2020-08-07T15:10:07.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:07.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640135+0000", 2020-08-07T15:10:07.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4160000000000001e-06 2020-08-07T15:10:07.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:07.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640142+0000", 2020-08-07T15:10:07.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0632999999999998e-05 2020-08-07T15:10:07.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:07.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640232+0000", 2020-08-07T15:10:07.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.306823501 2020-08-07T15:10:07.359 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:07.360 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:07.360 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:07.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.360 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106232 3.a 3:5599d7e6:::benchmark_data_smithi074_35447_object13278:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:10:07.361 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:10:07.361 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 139.58283613, 2020-08-07T15:10:07.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52578228699999996, 2020-08-07T15:10:07.393 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:07.398 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:07.399 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:07.399 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:10:07.399 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:07.399 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106232 2020-08-07T15:10:07.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.400 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:07.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:07.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:10:07.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:07.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:07.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:10:07.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4214999999999997e-05 2020-08-07T15:10:07.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:07.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345641+0000", 2020-08-07T15:10:07.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9279999999999993e-06 2020-08-07T15:10:07.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:07.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345649+0000", 2020-08-07T15:10:07.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4391e-05 2020-08-07T15:10:07.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:07.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345664+0000", 2020-08-07T15:10:07.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7170000000000002e-06 2020-08-07T15:10:07.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:07.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345670+0000", 2020-08-07T15:10:07.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.459e-05 2020-08-07T15:10:07.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:07.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345695+0000", 2020-08-07T15:10:07.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6070999999999999e-05 2020-08-07T15:10:07.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:07.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345751+0000", 2020-08-07T15:10:07.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010069300000000001 2020-08-07T15:10:07.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:10:07.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345852+0000", 2020-08-07T15:10:07.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013512299999999999 2020-08-07T15:10:07.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:07.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.347203+0000", 2020-08-07T15:10:07.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0844000000000001e-05 2020-08-07T15:10:07.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:07.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.347274+0000", 2020-08-07T15:10:07.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52399471600000003 2020-08-07T15:10:07.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:07.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871269+0000", 2020-08-07T15:10:07.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.311e-06 2020-08-07T15:10:07.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:07.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871270+0000", 2020-08-07T15:10:07.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3509000000000004e-05 2020-08-07T15:10:07.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:07.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871343+0000", 2020-08-07T15:10:07.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6072000000000002e-05 2020-08-07T15:10:07.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:07.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871389+0000", 2020-08-07T15:10:07.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52578228699999996 2020-08-07T15:10:07.727 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:07.727 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:07.727 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:07.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.728 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106232 3.a e59/20 ondisk, result = 0)", 2020-08-07T15:10:07.728 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:10:07.728 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 139.579789955, 2020-08-07T15:10:07.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52274359299999995, 2020-08-07T15:10:07.776 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:07.777 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:07.777 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:07.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:07.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:10:07.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:07.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:07.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:10:07.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5011000000000001e-05 2020-08-07T15:10:07.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:07.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348668+0000", 2020-08-07T15:10:07.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1959999999999998e-06 2020-08-07T15:10:07.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:07.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348671+0000", 2020-08-07T15:10:07.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52253456499999995 2020-08-07T15:10:07.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:07.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871205+0000", 2020-08-07T15:10:07.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0230000000000003e-06 2020-08-07T15:10:07.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:07.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871209+0000", 2020-08-07T15:10:07.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0708999999999999e-05 2020-08-07T15:10:07.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:07.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871250+0000", 2020-08-07T15:10:07.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0158e-05 2020-08-07T15:10:07.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:07.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871260+0000", 2020-08-07T15:10:07.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013693099999999999 2020-08-07T15:10:07.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:07.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871397+0000", 2020-08-07T15:10:07.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52274359299999995 2020-08-07T15:10:07.868 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:07.868 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:07.868 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:07.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.880 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:145807 3.c e75/20 3:39741db6:::benchmark_data_smithi074_35447_object16198:head v 75'9594, mlcod=75'9590)", 2020-08-07T15:10:07.880 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.674086+0000", 2020-08-07T15:10:07.880 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 112.254357738, 2020-08-07T15:10:07.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40268798500000003, 2020-08-07T15:10:07.881 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:07.881 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:07.881 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:07.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:07.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674086+0000", 2020-08-07T15:10:07.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:07.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:07.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674086+0000", 2020-08-07T15:10:07.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3321000000000001e-05 2020-08-07T15:10:07.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:07.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674129+0000", 2020-08-07T15:10:07.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1919999999999999e-06 2020-08-07T15:10:07.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:07.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674136+0000", 2020-08-07T15:10:07.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40193062899999998 2020-08-07T15:10:07.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:07.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076067+0000", 2020-08-07T15:10:07.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3579e-05 2020-08-07T15:10:07.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:07.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076080+0000", 2020-08-07T15:10:07.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3733999999999998e-05 2020-08-07T15:10:07.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:07.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076124+0000", 2020-08-07T15:10:07.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0145e-05 2020-08-07T15:10:07.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:07.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076154+0000", 2020-08-07T15:10:07.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00055768600000000005 2020-08-07T15:10:07.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:07.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076712+0000", 2020-08-07T15:10:07.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1699000000000007e-05 2020-08-07T15:10:07.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:07.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076774+0000", 2020-08-07T15:10:07.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40268798500000003 2020-08-07T15:10:07.981 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:07.981 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:07.983 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:07.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.984 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:144 4.5 4:aed0a56a:::benchmark_data_smithi074_44586_object17:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:10:07.984 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.396584+0000", 2020-08-07T15:10:07.984 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 100.53185949900001, 2020-08-07T15:10:07.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45809661000000002, 2020-08-07T15:10:07.985 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:07.985 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:07.985 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:07.985 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:07.985 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:07.986 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 144 2020-08-07T15:10:07.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.986 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:07.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:07.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396584+0000", 2020-08-07T15:10:07.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:07.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:07.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396584+0000", 2020-08-07T15:10:07.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4195e-05 2020-08-07T15:10:07.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:07.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396628+0000", 2020-08-07T15:10:07.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.157e-06 2020-08-07T15:10:07.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:07.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:07.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:07.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396631+0000", 2020-08-07T15:10:08.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.832e-06 2020-08-07T15:10:08.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:08.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396638+0000", 2020-08-07T15:10:08.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.745e-06 2020-08-07T15:10:08.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:08.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396641+0000", 2020-08-07T15:10:08.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001659368 2020-08-07T15:10:08.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:08.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.398300+0000", 2020-08-07T15:10:08.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2652000000000007e-05 2020-08-07T15:10:08.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:08.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.398393+0000", 2020-08-07T15:10:08.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016763699999999999 2020-08-07T15:10:08.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,7", 2020-08-07T15:10:08.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.398560+0000", 2020-08-07T15:10:08.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023917719999999999 2020-08-07T15:10:08.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:08.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.400952+0000", 2020-08-07T15:10:08.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999997e-07 2020-08-07T15:10:08.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:08.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.400953+0000", 2020-08-07T15:10:08.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9373999999999998e-05 2020-08-07T15:10:08.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:08.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.401012+0000", 2020-08-07T15:10:08.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003107865 2020-08-07T15:10:08.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:08.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.404120+0000", 2020-08-07T15:10:08.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45049484299999998 2020-08-07T15:10:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854615+0000", 2020-08-07T15:10:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.035e-06 2020-08-07T15:10:08.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854616+0000", 2020-08-07T15:10:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4249000000000001e-05 2020-08-07T15:10:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:08.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854680+0000", 2020-08-07T15:10:08.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45809661000000002 2020-08-07T15:10:08.120 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.120 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:08.120 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.121 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:30975 4.5 e98/81 ondisk, result = 0)", 2020-08-07T15:10:08.121 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:45.734259+0000", 2020-08-07T15:10:08.121 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 81.194184402000005, 2020-08-07T15:10:08.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46200962800000001, 2020-08-07T15:10:08.122 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:08.122 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:08.147 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:08.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:08.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.734259+0000", 2020-08-07T15:10:08.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:08.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:08.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.734259+0000", 2020-08-07T15:10:08.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9501000000000001e-05 2020-08-07T15:10:08.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:08.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.734288+0000", 2020-08-07T15:10:08.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5990000000000004e-06 2020-08-07T15:10:08.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:08.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.734293+0000", 2020-08-07T15:10:08.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46181377000000001 2020-08-07T15:10:08.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:08.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.196107+0000", 2020-08-07T15:10:08.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6169999999999998e-06 2020-08-07T15:10:08.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:08.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.196112+0000", 2020-08-07T15:10:08.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5908999999999997e-05 2020-08-07T15:10:08.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:08.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.196158+0000", 2020-08-07T15:10:08.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1104000000000001e-05 2020-08-07T15:10:08.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:08.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.196169+0000", 2020-08-07T15:10:08.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9128000000000004e-05 2020-08-07T15:10:08.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:08.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.196268+0000", 2020-08-07T15:10:08.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46200962800000001 2020-08-07T15:10:08.259 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.259 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:08.259 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.260 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:30976 4.5 e98/81 ondisk, result = 0)", 2020-08-07T15:10:08.260 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:45.736493+0000", 2020-08-07T15:10:08.260 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 81.191950230000003, 2020-08-07T15:10:08.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54677504399999999, 2020-08-07T15:10:08.261 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:08.261 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:08.261 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:08.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:08.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736493+0000", 2020-08-07T15:10:08.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:08.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:08.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736493+0000", 2020-08-07T15:10:08.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.012e-05 2020-08-07T15:10:08.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:08.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736523+0000", 2020-08-07T15:10:08.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6669999999999998e-06 2020-08-07T15:10:08.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:08.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736527+0000", 2020-08-07T15:10:08.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54657333200000002 2020-08-07T15:10:08.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:08.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283100+0000", 2020-08-07T15:10:08.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.549e-06 2020-08-07T15:10:08.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:08.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283105+0000", 2020-08-07T15:10:08.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1853000000000006e-05 2020-08-07T15:10:08.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:08.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283167+0000", 2020-08-07T15:10:08.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015e-05 2020-08-07T15:10:08.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:08.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283177+0000", 2020-08-07T15:10:08.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1508000000000005e-05 2020-08-07T15:10:08.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:08.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283268+0000", 2020-08-07T15:10:08.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54677504399999999 2020-08-07T15:10:08.381 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.382 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:08.382 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.385 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31528 4.5 4:a883779b:::benchmark_data_smithi074_44586_object3940:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e100)", 2020-08-07T15:10:08.402 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:47.795320+0000", 2020-08-07T15:10:08.402 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.133123634, 2020-08-07T15:10:08.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33015256900000001, 2020-08-07T15:10:08.403 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:08.403 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:08.403 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:08.403 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:08.404 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:08.404 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31528 2020-08-07T15:10:08.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.406 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:08.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:08.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.795320+0000", 2020-08-07T15:10:08.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:08.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:08.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.795320+0000", 2020-08-07T15:10:08.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9305000000000003e-05 2020-08-07T15:10:08.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:08.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.795369+0000", 2020-08-07T15:10:08.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1659999999999999e-06 2020-08-07T15:10:08.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:08.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.795375+0000", 2020-08-07T15:10:08.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2284e-05 2020-08-07T15:10:08.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:08.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.795387+0000", 2020-08-07T15:10:08.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.789e-06 2020-08-07T15:10:08.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:08.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.795393+0000", 2020-08-07T15:10:08.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025794009999999998 2020-08-07T15:10:08.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:08.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.797973+0000", 2020-08-07T15:10:08.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011622599999999999 2020-08-07T15:10:08.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:08.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.798089+0000", 2020-08-07T15:10:08.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021524900000000001 2020-08-07T15:10:08.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,7", 2020-08-07T15:10:08.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.798304+0000", 2020-08-07T15:10:08.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042865830000000001 2020-08-07T15:10:08.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:08.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.802591+0000", 2020-08-07T15:10:08.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0088220250000000007 2020-08-07T15:10:08.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:08.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.811413+0000", 2020-08-07T15:10:08.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31390628199999998 2020-08-07T15:10:08.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:08.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125319+0000", 2020-08-07T15:10:08.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.037e-06 2020-08-07T15:10:08.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:08.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125320+0000", 2020-08-07T15:10:08.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0878000000000003e-05 2020-08-07T15:10:08.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:08.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125371+0000", 2020-08-07T15:10:08.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101344 2020-08-07T15:10:08.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:08.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125472+0000", 2020-08-07T15:10:08.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33015256900000001 2020-08-07T15:10:08.421 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.421 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:08.421 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.432 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:31536 4.c e100/99 4:326df2a4:::benchmark_data_smithi074_44586_object3941:head v 100'2008, mlcod=100'1999)", 2020-08-07T15:10:08.432 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:47.804640+0000", 2020-08-07T15:10:08.432 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.123803061999993, 2020-08-07T15:10:08.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32071613199999999, 2020-08-07T15:10:08.433 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:08.433 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:08.433 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:08.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:08.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.804640+0000", 2020-08-07T15:10:08.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:08.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:08.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.804640+0000", 2020-08-07T15:10:08.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9385999999999998e-05 2020-08-07T15:10:08.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:08.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.804690+0000", 2020-08-07T15:10:08.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2319999999999998e-06 2020-08-07T15:10:08.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:08.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.804696+0000", 2020-08-07T15:10:08.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.319733934 2020-08-07T15:10:08.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:08.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.124430+0000", 2020-08-07T15:10:08.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1089999999999997e-06 2020-08-07T15:10:08.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:08.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.124435+0000", 2020-08-07T15:10:08.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9981999999999998e-05 2020-08-07T15:10:08.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:08.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.124475+0000", 2020-08-07T15:10:08.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0684e-05 2020-08-07T15:10:08.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:08.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.124506+0000", 2020-08-07T15:10:08.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00079925600000000003 2020-08-07T15:10:08.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:08.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125305+0000", 2020-08-07T15:10:08.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1548999999999997e-05 2020-08-07T15:10:08.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:08.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125356+0000", 2020-08-07T15:10:08.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32071613199999999 2020-08-07T15:10:08.444 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.444 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:08.444 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.445 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:31528 4.5 e100/81 ondisk, result = 0)", 2020-08-07T15:10:08.445 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:47.812815+0000", 2020-08-07T15:10:08.445 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.115628134999994, 2020-08-07T15:10:08.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.312664363, 2020-08-07T15:10:08.445 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:08.446 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:08.447 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:08.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:08.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.812815+0000", 2020-08-07T15:10:08.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:08.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:08.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.812815+0000", 2020-08-07T15:10:08.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2224999999999997e-05 2020-08-07T15:10:08.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:08.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.812847+0000", 2020-08-07T15:10:08.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9460000000000003e-06 2020-08-07T15:10:08.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:08.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.812853+0000", 2020-08-07T15:10:08.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31240245100000003 2020-08-07T15:10:08.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:08.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125256+0000", 2020-08-07T15:10:08.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7509999999999997e-06 2020-08-07T15:10:08.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:08.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125262+0000", 2020-08-07T15:10:08.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9220000000000001e-05 2020-08-07T15:10:08.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:08.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125301+0000", 2020-08-07T15:10:08.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3700000000000001e-06 2020-08-07T15:10:08.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:08.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125310+0000", 2020-08-07T15:10:08.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001694 2020-08-07T15:10:08.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:08.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125479+0000", 2020-08-07T15:10:08.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.312664363 2020-08-07T15:10:08.456 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.457 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:08.457 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.458 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:31816 4.1 e100/98 4:8a2ebf91:::benchmark_data_smithi074_44586_object3976:head v 100'1872, mlcod=100'1866)", 2020-08-07T15:10:08.458 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.169272+0000", 2020-08-07T15:10:08.458 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.759171022999993, 2020-08-07T15:10:08.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38326060099999998, 2020-08-07T15:10:08.459 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:08.459 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:08.459 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:08.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:08.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.169272+0000", 2020-08-07T15:10:08.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:08.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:08.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.169272+0000", 2020-08-07T15:10:08.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.93e-05 2020-08-07T15:10:08.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:08.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.169312+0000", 2020-08-07T15:10:08.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0230000000000003e-06 2020-08-07T15:10:08.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:08.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.169316+0000", 2020-08-07T15:10:08.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38224064899999999 2020-08-07T15:10:08.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:08.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.551556+0000", 2020-08-07T15:10:08.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.673e-06 2020-08-07T15:10:08.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:08.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.551561+0000", 2020-08-07T15:10:08.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0315999999999996e-05 2020-08-07T15:10:08.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:08.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.551631+0000", 2020-08-07T15:10:08.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6877999999999999e-05 2020-08-07T15:10:08.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:08.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.551658+0000", 2020-08-07T15:10:08.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00081916599999999997 2020-08-07T15:10:08.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:08.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.552477+0000", 2020-08-07T15:10:08.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5596000000000001e-05 2020-08-07T15:10:08.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:08.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.552533+0000", 2020-08-07T15:10:08.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38326060099999998 2020-08-07T15:10:08.468 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.469 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:08.469 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.470 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:37048 4.5 4:a403bd70:::benchmark_data_smithi074_44586_object4630:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e106)", 2020-08-07T15:10:08.470 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:54.237330+0000", 2020-08-07T15:10:08.470 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 72.691113153000003, 2020-08-07T15:10:08.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.474059283, 2020-08-07T15:10:08.472 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:08.472 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:08.472 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:08.472 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:08.473 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:08.473 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 37048 2020-08-07T15:10:08.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.473 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:08.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:08.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237330+0000", 2020-08-07T15:10:08.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:08.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:08.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237330+0000", 2020-08-07T15:10:08.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0713999999999998e-05 2020-08-07T15:10:08.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:08.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237361+0000", 2020-08-07T15:10:08.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.756e-06 2020-08-07T15:10:08.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:08.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237364+0000", 2020-08-07T15:10:08.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.694e-06 2020-08-07T15:10:08.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:08.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237369+0000", 2020-08-07T15:10:08.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3709999999999998e-06 2020-08-07T15:10:08.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:08.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237372+0000", 2020-08-07T15:10:08.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015644700000000001 2020-08-07T15:10:08.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:08.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.238936+0000", 2020-08-07T15:10:08.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9940999999999998e-05 2020-08-07T15:10:08.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:08.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.239026+0000", 2020-08-07T15:10:08.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017423800000000001 2020-08-07T15:10:08.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,7", 2020-08-07T15:10:08.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.239200+0000", 2020-08-07T15:10:08.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041267530000000004 2020-08-07T15:10:08.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:08.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.243327+0000", 2020-08-07T15:10:08.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7700000000000004e-07 2020-08-07T15:10:08.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:08.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.243328+0000", 2020-08-07T15:10:08.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1211999999999997e-05 2020-08-07T15:10:08.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:08.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.243379+0000", 2020-08-07T15:10:08.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075155150000000004 2020-08-07T15:10:08.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:08.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.250894+0000", 2020-08-07T15:10:08.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.460450313 2020-08-07T15:10:08.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:08.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.711345+0000", 2020-08-07T15:10:08.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1990000000000001e-06 2020-08-07T15:10:08.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:08.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.711346+0000", 2020-08-07T15:10:08.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3529999999999998e-05 2020-08-07T15:10:08.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:08.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.711389+0000", 2020-08-07T15:10:08.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.474059283 2020-08-07T15:10:08.490 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.490 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:08.490 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.491 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:46032 4.c e117/104 4:3eb0d548:::benchmark_data_smithi074_44586_object5753:head v 117'2968, mlcod=117'2958)", 2020-08-07T15:10:08.491 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:05.163259+0000", 2020-08-07T15:10:08.491 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.765183806000003, 2020-08-07T15:10:08.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31522351999999998, 2020-08-07T15:10:08.491 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:08.492 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:08.493 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:08.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:08.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.163259+0000", 2020-08-07T15:10:08.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:08.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:08.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.163259+0000", 2020-08-07T15:10:08.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9008000000000002e-05 2020-08-07T15:10:08.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:08.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.163318+0000", 2020-08-07T15:10:08.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2689999999999996e-06 2020-08-07T15:10:08.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:08.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.163325+0000", 2020-08-07T15:10:08.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.312753646 2020-08-07T15:10:08.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:08.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.476078+0000", 2020-08-07T15:10:08.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4379999999999999e-06 2020-08-07T15:10:08.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:08.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.476082+0000", 2020-08-07T15:10:08.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2913999999999998e-05 2020-08-07T15:10:08.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:08.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.476175+0000", 2020-08-07T15:10:08.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5939e-05 2020-08-07T15:10:08.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:08.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.476201+0000", 2020-08-07T15:10:08.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022305070000000001 2020-08-07T15:10:08.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:08.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.478431+0000", 2020-08-07T15:10:08.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1799000000000003e-05 2020-08-07T15:10:08.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:08.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.478483+0000", 2020-08-07T15:10:08.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31522351999999998 2020-08-07T15:10:08.540 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.540 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:08.540 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.541 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:60960 4.e 4:73b21c89:::benchmark_data_smithi074_44586_object7619:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e121)", 2020-08-07T15:10:08.541 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:14.254254+0000", 2020-08-07T15:10:08.541 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 52.674189284999997, 2020-08-07T15:10:08.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.29219231800000001, 2020-08-07T15:10:08.542 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:08.542 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:08.542 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:08.542 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:08.543 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:08.543 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 60960 2020-08-07T15:10:08.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.543 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:08.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:08.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.254254+0000", 2020-08-07T15:10:08.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:08.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:08.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.254254+0000", 2020-08-07T15:10:08.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9551e-05 2020-08-07T15:10:08.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:08.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.254304+0000", 2020-08-07T15:10:08.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7420000000000003e-06 2020-08-07T15:10:08.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:08.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.254309+0000", 2020-08-07T15:10:08.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0723000000000001e-05 2020-08-07T15:10:08.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:08.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.254320+0000", 2020-08-07T15:10:08.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1070000000000004e-06 2020-08-07T15:10:08.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:08.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.254325+0000", 2020-08-07T15:10:08.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00072346499999999996 2020-08-07T15:10:08.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:08.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.255049+0000", 2020-08-07T15:10:08.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010703400000000001 2020-08-07T15:10:08.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:08.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.255156+0000", 2020-08-07T15:10:08.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000179338 2020-08-07T15:10:08.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:10:08.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.255335+0000", 2020-08-07T15:10:08.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041513380000000001 2020-08-07T15:10:08.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:08.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.259486+0000", 2020-08-07T15:10:08.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9700000000000002e-07 2020-08-07T15:10:08.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:08.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.259487+0000", 2020-08-07T15:10:08.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3769999999999994e-05 2020-08-07T15:10:08.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:08.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.259550+0000", 2020-08-07T15:10:08.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.069610947000000006 2020-08-07T15:10:08.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:08.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.329161+0000", 2020-08-07T15:10:08.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.217207706 2020-08-07T15:10:08.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:08.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.546369+0000", 2020-08-07T15:10:08.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.612e-06 2020-08-07T15:10:08.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:08.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.546371+0000", 2020-08-07T15:10:08.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5587999999999995e-05 2020-08-07T15:10:08.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:08.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.546446+0000", 2020-08-07T15:10:08.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.29219231800000001 2020-08-07T15:10:08.580 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.580 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:08.580 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.581 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:60960 4.e e121/98 ondisk, result = 0)", 2020-08-07T15:10:08.581 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:14.257727+0000", 2020-08-07T15:10:08.581 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 52.670716614, 2020-08-07T15:10:08.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28872502300000003, 2020-08-07T15:10:08.582 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:08.582 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:08.582 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:08.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:08.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.257727+0000", 2020-08-07T15:10:08.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:08.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:08.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.257727+0000", 2020-08-07T15:10:08.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5155e-05 2020-08-07T15:10:08.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:08.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.257742+0000", 2020-08-07T15:10:08.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4930000000000001e-06 2020-08-07T15:10:08.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:08.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.257744+0000", 2020-08-07T15:10:08.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28856093399999999 2020-08-07T15:10:08.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:08.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.546305+0000", 2020-08-07T15:10:08.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.045e-06 2020-08-07T15:10:08.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:08.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.546310+0000", 2020-08-07T15:10:08.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0689999999999998e-05 2020-08-07T15:10:08.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:08.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.546351+0000", 2020-08-07T15:10:08.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3400000000000004e-06 2020-08-07T15:10:08.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:08.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.546360+0000", 2020-08-07T15:10:08.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1366000000000006e-05 2020-08-07T15:10:08.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:08.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.546452+0000", 2020-08-07T15:10:08.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28872502300000003 2020-08-07T15:10:08.593 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.593 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:08.594 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.594 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:66144 4.c e126/104 4:3c80a167:::benchmark_data_smithi074_44586_object8267:head v 126'4280, mlcod=126'4274)", 2020-08-07T15:10:08.594 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:20.004507+0000", 2020-08-07T15:10:08.595 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.923936130000001, 2020-08-07T15:10:08.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30587809900000001, 2020-08-07T15:10:08.595 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:08.595 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:08.595 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:08.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:08.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.004507+0000", 2020-08-07T15:10:08.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:08.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:08.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.004507+0000", 2020-08-07T15:10:08.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9455e-05 2020-08-07T15:10:08.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:08.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.004557+0000", 2020-08-07T15:10:08.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2020-08-07T15:10:08.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:08.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.004563+0000", 2020-08-07T15:10:08.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30196788899999999 2020-08-07T15:10:08.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:08.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.306531+0000", 2020-08-07T15:10:08.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.259e-06 2020-08-07T15:10:08.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:08.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.306536+0000", 2020-08-07T15:10:08.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1167000000000001e-05 2020-08-07T15:10:08.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:08.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.306578+0000", 2020-08-07T15:10:08.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4280000000000001e-05 2020-08-07T15:10:08.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:08.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.306602+0000", 2020-08-07T15:10:08.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037278799999999998 2020-08-07T15:10:08.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:08.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.310330+0000", 2020-08-07T15:10:08.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5454000000000001e-05 2020-08-07T15:10:08.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:08.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.310385+0000", 2020-08-07T15:10:08.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30587809900000001 2020-08-07T15:10:08.625 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.633 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:08.633 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.634 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:105544 4.c e147/104 4:36abba5f:::benchmark_data_smithi074_44586_object13192:head v 147'6592, mlcod=147'6582)", 2020-08-07T15:10:08.635 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:49.257184+0000", 2020-08-07T15:10:08.635 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.671258975000001, 2020-08-07T15:10:08.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42401245100000001, 2020-08-07T15:10:08.636 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:08.636 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:08.636 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:08.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:08.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:49.257184+0000", 2020-08-07T15:10:08.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:08.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:08.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:49.257184+0000", 2020-08-07T15:10:08.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9108e-05 2020-08-07T15:10:08.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:08.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:49.257213+0000", 2020-08-07T15:10:08.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.703e-06 2020-08-07T15:10:08.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:08.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:49.257217+0000", 2020-08-07T15:10:08.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42086945100000001 2020-08-07T15:10:08.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:08.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:49.678087+0000", 2020-08-07T15:10:08.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4559999999999999e-06 2020-08-07T15:10:08.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:08.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:49.678092+0000", 2020-08-07T15:10:08.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4265999999999999e-05 2020-08-07T15:10:08.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:08.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:49.678136+0000", 2020-08-07T15:10:08.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2603999999999999e-05 2020-08-07T15:10:08.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:08.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:49.678159+0000", 2020-08-07T15:10:08.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029859919999999998 2020-08-07T15:10:08.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:08.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:49.681145+0000", 2020-08-07T15:10:08.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1870999999999999e-05 2020-08-07T15:10:08.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:08.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:49.681197+0000", 2020-08-07T15:10:08.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42401245100000001 2020-08-07T15:10:08.687 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.693 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:08.694 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.694 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:128869 4.e 4:713a1178:::benchmark_data_smithi074_44586_object12633:head [delete] snapc 0=[] ondisk+write+known_if_redirected e155)", 2020-08-07T15:10:08.696 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:03.724293+0000", 2020-08-07T15:10:08.696 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.2041502249999998, 2020-08-07T15:10:08.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31921112899999998, 2020-08-07T15:10:08.697 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:08.697 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:08.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:08.698 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:08.699 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:08.699 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128869 2020-08-07T15:10:08.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.699 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:08.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:08.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.724293+0000", 2020-08-07T15:10:08.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:08.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:08.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.724293+0000", 2020-08-07T15:10:08.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8221000000000001e-05 2020-08-07T15:10:08.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:08.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.724311+0000", 2020-08-07T15:10:08.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1969999999999998e-06 2020-08-07T15:10:08.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:08.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.724315+0000", 2020-08-07T15:10:08.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7659999999999993e-06 2020-08-07T15:10:08.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:08.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.724325+0000", 2020-08-07T15:10:08.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6439999999999999e-06 2020-08-07T15:10:08.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:08.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.724330+0000", 2020-08-07T15:10:08.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7154000000000003e-05 2020-08-07T15:10:08.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:08.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.724377+0000", 2020-08-07T15:10:08.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8620000000000001e-05 2020-08-07T15:10:08.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:08.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.724476+0000", 2020-08-07T15:10:08.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010416099999999999 2020-08-07T15:10:08.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:10:08.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.724580+0000", 2020-08-07T15:10:08.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011972249999999999 2020-08-07T15:10:08.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:08.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.725777+0000", 2020-08-07T15:10:08.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000400216 2020-08-07T15:10:08.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:08.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.726177+0000", 2020-08-07T15:10:08.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31718939200000001 2020-08-07T15:10:08.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:08.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.043367+0000", 2020-08-07T15:10:08.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7260000000000001e-06 2020-08-07T15:10:08.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:08.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.043368+0000", 2020-08-07T15:10:08.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1083999999999998e-05 2020-08-07T15:10:08.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:08.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.043449+0000", 2020-08-07T15:10:08.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4722999999999998e-05 2020-08-07T15:10:08.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:08.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:08.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:08.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.043504+0000", 2020-08-07T15:10:08.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31921112899999998 2020-08-07T15:10:08.806 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.806 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:08.806 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.806 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:08.806 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:08.807 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:10:08.808 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:10:08.810 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.1 to 1 (10000) 2020-08-07T15:10:08.810 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:08.807+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:08.811 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:10:08.812 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 2 1 2020-08-07T15:10:08.902 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:10:08.911 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:08.909+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:08.994 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:10:08.994 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:10:08.994 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2020-08-07T15:10:08.994 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:10:08.999 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:10:09.017 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:09.015+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:09.082 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:10:09.105 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:10:09.126 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:09.121+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:09.130 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:10:09.154 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:10:09.161 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:10:09.161 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:10:09.162 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:10:09.162 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:10:09.162 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:10:09.177 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:10:09.185 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:10:09.203 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:10:09.221 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.2 to 1 (10000) 2020-08-07T15:10:09.221 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:10:09.232 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:10:09.232 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:09.228+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:09.238 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 3 1 2020-08-07T15:10:09.255 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:10:09.256 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:10:09.256 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:10:09.257 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:10:09.257 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:10:09.257 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:10:09.258 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:10:09.259 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:10:09.259 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:10:09.304 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:09.305 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:09.317 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:10:09.362 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:09.357+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:09.446 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:09.443+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:09.463 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:09.463 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:09.473 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:10:09.559 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:09.558+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:09.632 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:09.633 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:09.645 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:10:09.661 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:09.652+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:09.755 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:09.754+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:09.806 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:09.807 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:09.816 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:10:09.866 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:09.860+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:09.936 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:09.937 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:09.950 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:10:09.958 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:09.957+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:10.063 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:10.059+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:10.123 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:10.123 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:10.124 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:10:10.163 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:10.162+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:10.223 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.3 to 1 (10000) 2020-08-07T15:10:10.235 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 4 1 2020-08-07T15:10:10.290 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:10.291 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:10.291 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:10.286+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:10.318 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:10:10.372 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:10.371+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:10.475 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:10.472+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:10.484 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:10.485 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:10.522 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:10:10.615 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:10.605+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:10.675 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:10.673+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:10.691 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:10.691 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:10.692 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:10:10.850 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:10.847+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:10.878 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:10.879 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:10.888 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:10:10.954 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:10.944+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:11.039 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:11.028+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:11.084 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:11.084 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:11.087 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:10:11.129 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:11.126+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:11.232 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.4 to 1 (10000) 2020-08-07T15:10:11.272 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:10:11.277 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:10:11.303 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 5 1 2020-08-07T15:10:11.303 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:11.317 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:11.321 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:10:11.322 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:11.234+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:11.375 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:11.374+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:11.497 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:11.488+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:11.529 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:10:11.547 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:11.547 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:11.557 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:10:11.560 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:10:11.587 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:10:11.610 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:10:11.643 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:11.615+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:11.644 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_1 2020-08-07T15:10:11.663 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool rm unique_pool_1 unique_pool_1 --yes-i-really-really-mean-it 2020-08-07T15:10:11.664 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:10:11.665 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:10:11.691 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:10:11.691 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:10:11.691 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2020-08-07T15:10:11.692 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:10:11.692 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:10:11.702 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:11.696+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:11.702 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:10:11.724 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:10:11.724 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:10:11.725 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:10:11.725 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:10:11.725 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:10:11.725 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:10:11.732 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:10:11.732 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:10:11.732 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:10:11.802 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:11.800+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:11.862 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:10:11.862 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:10:11.862 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:10:11.863 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:10:11.863 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:10:11.875 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:10:11.906 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:11.901+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:12.019 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:12.005+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:12.055 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:10:12.056 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:10:12.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:10:12.056 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:10:12.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.057 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:108376 3.3 e62/20 ondisk, result = 0)", 2020-08-07T15:10:12.057 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:10:12.057 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 141.355137665, 2020-08-07T15:10:12.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62926872599999994, 2020-08-07T15:10:12.080 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:12.080 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:12.080 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:12.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:12.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:10:12.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:12.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:12.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:10:12.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8491999999999998e-05 2020-08-07T15:10:12.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:12.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683061+0000", 2020-08-07T15:10:12.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9099999999999998e-06 2020-08-07T15:10:12.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:12.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683065+0000", 2020-08-07T15:10:12.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62906670600000003 2020-08-07T15:10:12.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:12.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312131+0000", 2020-08-07T15:10:12.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2147000000000001e-05 2020-08-07T15:10:12.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:12.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312143+0000", 2020-08-07T15:10:12.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5152e-05 2020-08-07T15:10:12.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:12.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312239+0000", 2020-08-07T15:10:12.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5089999999999993e-06 2020-08-07T15:10:12.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:12.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312247+0000", 2020-08-07T15:10:12.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3810000000000001e-05 2020-08-07T15:10:12.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:12.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312311+0000", 2020-08-07T15:10:12.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62926872599999994 2020-08-07T15:10:12.148 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:12.148 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:12.148 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:12.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "MOSDPGPushReply(3.1 62/60 [PushReplyOp(3:8329f37e:::benchmark_data_smithi074_35447_object11915:head)])", 2020-08-07T15:10:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:10:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 141.17977964900001, 2020-08-07T15:10:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996, 2020-08-07T15:10:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:10:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:10:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5372000000000001e-05 2020-08-07T15:10:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:12.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858416+0000", 2020-08-07T15:10:12.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6899999999999998e-06 2020-08-07T15:10:12.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:12.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858419+0000", 2020-08-07T15:10:12.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63907904299999996 2020-08-07T15:10:12.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:12.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497498+0000", 2020-08-07T15:10:12.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3332999999999999e-05 2020-08-07T15:10:12.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:12.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497512+0000", 2020-08-07T15:10:12.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000450875 2020-08-07T15:10:12.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:12.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497962+0000", 2020-08-07T15:10:12.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000267779 2020-08-07T15:10:12.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:12.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.498230+0000", 2020-08-07T15:10:12.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996 2020-08-07T15:10:12.411 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:12.411 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:12.411 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:12.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.412 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:145792 3.1 3:881e0420:::benchmark_data_smithi074_35447_object16183:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:10:12.412 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:10:12.416 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 117.371373768, 2020-08-07T15:10:12.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997, 2020-08-07T15:10:12.417 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:12.417 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:12.437 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:12.437 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:10:12.437 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:12.460 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 145792 2020-08-07T15:10:12.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.461 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:12.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:12.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:10:12.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:12.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:12.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:10:12.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7489000000000001e-05 2020-08-07T15:10:12.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:12.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666824+0000", 2020-08-07T15:10:12.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0169999999999998e-06 2020-08-07T15:10:12.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:12.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666828+0000", 2020-08-07T15:10:12.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9829999999999996e-06 2020-08-07T15:10:12.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:12.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666836+0000", 2020-08-07T15:10:12.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.481e-06 2020-08-07T15:10:12.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:12.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666840+0000", 2020-08-07T15:10:12.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5299000000000001e-05 2020-08-07T15:10:12.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:12.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666885+0000", 2020-08-07T15:10:12.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014456500000000001 2020-08-07T15:10:12.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:12.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667030+0000", 2020-08-07T15:10:12.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017677799999999999 2020-08-07T15:10:12.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,5", 2020-08-07T15:10:12.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667207+0000", 2020-08-07T15:10:12.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004149649 2020-08-07T15:10:12.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:12.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671356+0000", 2020-08-07T15:10:12.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019339649999999999 2020-08-07T15:10:12.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:12.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673290+0000", 2020-08-07T15:10:12.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73346620500000004 2020-08-07T15:10:12.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:12.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406756+0000", 2020-08-07T15:10:12.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4700000000000001e-06 2020-08-07T15:10:12.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:12.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406759+0000", 2020-08-07T15:10:12.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6258999999999998e-05 2020-08-07T15:10:12.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:12.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406815+0000", 2020-08-07T15:10:12.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0126000000000002e-05 2020-08-07T15:10:12.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:12.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406855+0000", 2020-08-07T15:10:12.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997 2020-08-07T15:10:12.617 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:12.617 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:12.617 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:12.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.632 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:145792 3.1 e75/67 ondisk, result = 0)", 2020-08-07T15:10:12.632 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:10:12.632 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 117.36698887999999, 2020-08-07T15:10:12.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003, 2020-08-07T15:10:12.633 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:12.633 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:12.633 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:12.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:12.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:10:12.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:12.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:12.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:10:12.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2830999999999998e-05 2020-08-07T15:10:12.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:12.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671224+0000", 2020-08-07T15:10:12.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:10:12.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:12.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671229+0000", 2020-08-07T15:10:12.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.735414864 2020-08-07T15:10:12.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:12.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406644+0000", 2020-08-07T15:10:12.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1250000000000004e-06 2020-08-07T15:10:12.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:12.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406651+0000", 2020-08-07T15:10:12.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6567e-05 2020-08-07T15:10:12.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:12.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406737+0000", 2020-08-07T15:10:12.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0054000000000001e-05 2020-08-07T15:10:12.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:12.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406747+0000", 2020-08-07T15:10:12.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011391999999999999 2020-08-07T15:10:12.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:12.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406861+0000", 2020-08-07T15:10:12.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003 2020-08-07T15:10:12.776 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:12.776 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:12.776 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:12.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.777 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30921 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:10:12.777 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:10:12.777 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.332240506000005, 2020-08-07T15:10:12.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001, 2020-08-07T15:10:12.790 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:12.790 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:12.790 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:12.790 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:12.791 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:12.791 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30921 2020-08-07T15:10:12.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.791 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:12.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:12.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:10:12.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:12.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:12.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:10:12.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6954999999999998e-05 2020-08-07T15:10:12.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:12.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706016+0000", 2020-08-07T15:10:12.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0229999999999992e-06 2020-08-07T15:10:12.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:12.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706024+0000", 2020-08-07T15:10:12.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.432e-05 2020-08-07T15:10:12.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:12.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706039+0000", 2020-08-07T15:10:12.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8490000000000006e-06 2020-08-07T15:10:12.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:12.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706046+0000", 2020-08-07T15:10:12.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6837999999999999e-05 2020-08-07T15:10:12.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:12.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706113+0000", 2020-08-07T15:10:12.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3553000000000001e-05 2020-08-07T15:10:12.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:12.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706127+0000", 2020-08-07T15:10:12.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77277201699999998 2020-08-07T15:10:12.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:12.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.478899+0000", 2020-08-07T15:10:12.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093978599999999996 2020-08-07T15:10:12.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:10:12.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479839+0000", 2020-08-07T15:10:12.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010376029 2020-08-07T15:10:12.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:12.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490215+0000", 2020-08-07T15:10:12.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000190657 2020-08-07T15:10:12.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:12.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490405+0000", 2020-08-07T15:10:12.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001 2020-08-07T15:10:12.841 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:12.841 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:12.841 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:12.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.842 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30922 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:10:12.842 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:10:12.842 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.332080343000001, 2020-08-07T15:10:12.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247, 2020-08-07T15:10:12.843 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:12.843 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:12.843 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:12.858 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:12.858 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:12.858 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30922 2020-08-07T15:10:12.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.875 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:12.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:12.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:10:12.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:12.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:12.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:10:12.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9178999999999997e-05 2020-08-07T15:10:12.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:12.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706159+0000", 2020-08-07T15:10:12.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4999999999999999e-06 2020-08-07T15:10:12.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:12.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706164+0000", 2020-08-07T15:10:12.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1792e-05 2020-08-07T15:10:12.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:12.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706176+0000", 2020-08-07T15:10:12.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.139e-06 2020-08-07T15:10:12.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:12.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706182+0000", 2020-08-07T15:10:12.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039724900000000002 2020-08-07T15:10:12.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:12.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706579+0000", 2020-08-07T15:10:12.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1191e-05 2020-08-07T15:10:12.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:12.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706591+0000", 2020-08-07T15:10:12.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329101300000003 2020-08-07T15:10:12.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:12.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479882+0000", 2020-08-07T15:10:12.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9565000000000001e-05 2020-08-07T15:10:12.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:12.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:12.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:10:12.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479971+0000", 2020-08-07T15:10:12.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010471263 2020-08-07T15:10:12.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490442+0000", 2020-08-07T15:10:13.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015035600000000001 2020-08-07T15:10:13.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:13.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490593+0000", 2020-08-07T15:10:13.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247 2020-08-07T15:10:13.009 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.009 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:13.009 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.034 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30923 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:10:13.035 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:10:13.035 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.331605281999998, 2020-08-07T15:10:13.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998, 2020-08-07T15:10:13.035 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:13.035 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:13.036 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:13.036 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:13.036 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:13.036 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30923 2020-08-07T15:10:13.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.036 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:13.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:13.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:10:13.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:13.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:13.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:10:13.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5531000000000001e-05 2020-08-07T15:10:13.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:13.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706630+0000", 2020-08-07T15:10:13.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7689999999999999e-06 2020-08-07T15:10:13.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:13.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706637+0000", 2020-08-07T15:10:13.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3577999999999999e-05 2020-08-07T15:10:13.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:13.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706650+0000", 2020-08-07T15:10:13.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1439999999999997e-06 2020-08-07T15:10:13.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:13.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706658+0000", 2020-08-07T15:10:13.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1663999999999999e-05 2020-08-07T15:10:13.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706709+0000", 2020-08-07T15:10:13.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0149000000000001e-05 2020-08-07T15:10:13.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:13.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706719+0000", 2020-08-07T15:10:13.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329694900000001 2020-08-07T15:10:13.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480016+0000", 2020-08-07T15:10:13.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7998999999999997e-05 2020-08-07T15:10:13.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:10:13.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480084+0000", 2020-08-07T15:10:13.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010547333000000001 2020-08-07T15:10:13.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490632+0000", 2020-08-07T15:10:13.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7439e-05 2020-08-07T15:10:13.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:13.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490729+0000", 2020-08-07T15:10:13.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998 2020-08-07T15:10:13.110 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.110 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:13.111 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.111 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30924 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:10:13.111 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:10:13.111 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.331478009999998, 2020-08-07T15:10:13.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995, 2020-08-07T15:10:13.112 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:13.112 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:13.112 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:13.112 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:13.113 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:13.113 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30924 2020-08-07T15:10:13.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.113 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:13.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:13.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:10:13.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:13.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:13.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:10:13.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9698999999999999e-05 2020-08-07T15:10:13.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:13.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706762+0000", 2020-08-07T15:10:13.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8080000000000001e-06 2020-08-07T15:10:13.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:13.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706767+0000", 2020-08-07T15:10:13.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1069e-05 2020-08-07T15:10:13.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:13.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706778+0000", 2020-08-07T15:10:13.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6969999999999998e-06 2020-08-07T15:10:13.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:13.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706784+0000", 2020-08-07T15:10:13.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8792999999999998e-05 2020-08-07T15:10:13.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706843+0000", 2020-08-07T15:10:13.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5449999999999998e-06 2020-08-07T15:10:13.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:13.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706850+0000", 2020-08-07T15:10:13.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.773278354 2020-08-07T15:10:13.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480129+0000", 2020-08-07T15:10:13.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1577999999999997e-05 2020-08-07T15:10:13.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:10:13.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480210+0000", 2020-08-07T15:10:13.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010541915000000001 2020-08-07T15:10:13.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490752+0000", 2020-08-07T15:10:13.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103937 2020-08-07T15:10:13.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:13.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490856+0000", 2020-08-07T15:10:13.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995 2020-08-07T15:10:13.212 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.213 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:13.213 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.214 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30925 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:10:13.214 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:10:13.214 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.331354239000007, 2020-08-07T15:10:13.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995, 2020-08-07T15:10:13.215 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:13.216 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:13.234 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:13.234 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:13.234 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:13.235 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30925 2020-08-07T15:10:13.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.235 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:13.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:13.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:10:13.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:13.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:13.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:10:13.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4664999999999999e-05 2020-08-07T15:10:13.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:13.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706880+0000", 2020-08-07T15:10:13.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7479999999999999e-06 2020-08-07T15:10:13.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:13.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706886+0000", 2020-08-07T15:10:13.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1838000000000001e-05 2020-08-07T15:10:13.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:13.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706898+0000", 2020-08-07T15:10:13.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0089999999999998e-06 2020-08-07T15:10:13.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:13.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706903+0000", 2020-08-07T15:10:13.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3559e-05 2020-08-07T15:10:13.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706956+0000", 2020-08-07T15:10:13.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0309000000000001e-05 2020-08-07T15:10:13.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:13.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706967+0000", 2020-08-07T15:10:13.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77328290499999996 2020-08-07T15:10:13.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480250+0000", 2020-08-07T15:10:13.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8680999999999997e-05 2020-08-07T15:10:13.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:10:13.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480298+0000", 2020-08-07T15:10:13.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01059272 2020-08-07T15:10:13.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490891+0000", 2020-08-07T15:10:13.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014580300000000001 2020-08-07T15:10:13.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:13.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491037+0000", 2020-08-07T15:10:13.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995 2020-08-07T15:10:13.387 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.387 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:13.387 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.388 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30926 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:10:13.388 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:10:13.388 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.331237369999997, 2020-08-07T15:10:13.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005, 2020-08-07T15:10:13.388 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:13.389 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:13.389 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:13.389 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:13.389 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:13.389 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30926 2020-08-07T15:10:13.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.391 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:13.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:13.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:10:13.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:13.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:13.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:10:13.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5223000000000003e-05 2020-08-07T15:10:13.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:13.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706998+0000", 2020-08-07T15:10:13.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4809999999999997e-06 2020-08-07T15:10:13.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:13.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707003+0000", 2020-08-07T15:10:13.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1663e-05 2020-08-07T15:10:13.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:13.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707015+0000", 2020-08-07T15:10:13.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9729999999999997e-06 2020-08-07T15:10:13.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:13.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707020+0000", 2020-08-07T15:10:13.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2565999999999999e-05 2020-08-07T15:10:13.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707062+0000", 2020-08-07T15:10:13.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8944e-05 2020-08-07T15:10:13.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:13.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707081+0000", 2020-08-07T15:10:13.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77325885000000005 2020-08-07T15:10:13.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480340+0000", 2020-08-07T15:10:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0321000000000006e-05 2020-08-07T15:10:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:10:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480420+0000", 2020-08-07T15:10:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010655958 2020-08-07T15:10:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491076+0000", 2020-08-07T15:10:13.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9784999999999995e-05 2020-08-07T15:10:13.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:13.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491166+0000", 2020-08-07T15:10:13.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005 2020-08-07T15:10:13.410 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.411 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:13.411 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.411 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30927 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:10:13.412 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:10:13.412 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.331119551, 2020-08-07T15:10:13.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999, 2020-08-07T15:10:13.412 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:13.412 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:13.412 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:13.413 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:13.414 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:13.414 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30927 2020-08-07T15:10:13.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.414 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:13.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:13.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:10:13.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:13.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:13.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:10:13.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9861999999999997e-05 2020-08-07T15:10:13.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:13.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707120+0000", 2020-08-07T15:10:13.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8969999999999996e-06 2020-08-07T15:10:13.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:13.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707126+0000", 2020-08-07T15:10:13.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1248e-05 2020-08-07T15:10:13.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:13.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707137+0000", 2020-08-07T15:10:13.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.955e-06 2020-08-07T15:10:13.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:13.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707142+0000", 2020-08-07T15:10:13.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0643999999999998e-05 2020-08-07T15:10:13.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707193+0000", 2020-08-07T15:10:13.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1983e-05 2020-08-07T15:10:13.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:13.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707205+0000", 2020-08-07T15:10:13.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77324914 2020-08-07T15:10:13.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480454+0000", 2020-08-07T15:10:13.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4119999999999998e-05 2020-08-07T15:10:13.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:10:13.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480498+0000", 2020-08-07T15:10:13.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010709707000000001 2020-08-07T15:10:13.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491208+0000", 2020-08-07T15:10:13.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014433800000000001 2020-08-07T15:10:13.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:13.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491352+0000", 2020-08-07T15:10:13.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999 2020-08-07T15:10:13.462 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.463 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:13.463 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.464 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30928 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:10:13.464 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:10:13.465 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.330995193999996, 2020-08-07T15:10:13.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998, 2020-08-07T15:10:13.465 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:13.465 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:13.465 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:13.466 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:13.466 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:13.466 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30928 2020-08-07T15:10:13.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.466 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:13.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:13.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:10:13.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:13.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:13.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:10:13.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6582000000000001e-05 2020-08-07T15:10:13.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:13.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707241+0000", 2020-08-07T15:10:13.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9190000000000004e-06 2020-08-07T15:10:13.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:13.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707247+0000", 2020-08-07T15:10:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000188822 2020-08-07T15:10:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:13.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707436+0000", 2020-08-07T15:10:13.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2940000000000002e-06 2020-08-07T15:10:13.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:13.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707442+0000", 2020-08-07T15:10:13.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7800000000000003e-05 2020-08-07T15:10:13.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707490+0000", 2020-08-07T15:10:13.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2020-08-07T15:10:13.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:13.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707501+0000", 2020-08-07T15:10:13.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77303498800000003 2020-08-07T15:10:13.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480536+0000", 2020-08-07T15:10:13.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7723000000000001e-05 2020-08-07T15:10:13.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:10:13.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480614+0000", 2020-08-07T15:10:13.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010812183 2020-08-07T15:10:13.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491426+0000", 2020-08-07T15:10:13.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5502999999999999e-05 2020-08-07T15:10:13.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:13.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491502+0000", 2020-08-07T15:10:13.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998 2020-08-07T15:10:13.485 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.485 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:13.485 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.506 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31009 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:10:13.506 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:10:13.506 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.307038887999994, 2020-08-07T15:10:13.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002, 2020-08-07T15:10:13.507 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:13.507 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:13.507 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:13.507 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:13.507 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:13.508 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31009 2020-08-07T15:10:13.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.508 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:13.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:13.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:10:13.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:13.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:13.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:10:13.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2429000000000002e-05 2020-08-07T15:10:13.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:13.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731193+0000", 2020-08-07T15:10:13.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9760000000000003e-06 2020-08-07T15:10:13.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:13.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731198+0000", 2020-08-07T15:10:13.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3597e-05 2020-08-07T15:10:13.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:13.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731212+0000", 2020-08-07T15:10:13.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1709999999999996e-06 2020-08-07T15:10:13.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:13.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731219+0000", 2020-08-07T15:10:13.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5402999999999997e-05 2020-08-07T15:10:13.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731285+0000", 2020-08-07T15:10:13.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3495e-05 2020-08-07T15:10:13.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:13.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731298+0000", 2020-08-07T15:10:13.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74935423599999995 2020-08-07T15:10:13.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480652+0000", 2020-08-07T15:10:13.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000137806 2020-08-07T15:10:13.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:13.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480790+0000", 2020-08-07T15:10:13.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022272399999999999 2020-08-07T15:10:13.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:10:13.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481013+0000", 2020-08-07T15:10:13.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003223381 2020-08-07T15:10:13.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:13.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484236+0000", 2020-08-07T15:10:13.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1370000000000001e-06 2020-08-07T15:10:13.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:13.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484237+0000", 2020-08-07T15:10:13.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6288e-05 2020-08-07T15:10:13.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:13.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484294+0000", 2020-08-07T15:10:13.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.007260402 2020-08-07T15:10:13.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:13.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491554+0000", 2020-08-07T15:10:13.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7358000000000003e-05 2020-08-07T15:10:13.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:13.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491601+0000", 2020-08-07T15:10:13.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1100000000000004e-07 2020-08-07T15:10:13.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:13.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491602+0000", 2020-08-07T15:10:13.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013436500000000001 2020-08-07T15:10:13.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:13.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491737+0000", 2020-08-07T15:10:13.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002 2020-08-07T15:10:13.632 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.632 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:13.633 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.633 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31010 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:10:13.633 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:10:13.641 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.306912933999996, 2020-08-07T15:10:13.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001, 2020-08-07T15:10:13.641 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31010 2020-08-07T15:10:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:10:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:10:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2398000000000002e-05 2020-08-07T15:10:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731319+0000", 2020-08-07T15:10:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7100000000000004e-06 2020-08-07T15:10:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731325+0000", 2020-08-07T15:10:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4328000000000001e-05 2020-08-07T15:10:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731339+0000", 2020-08-07T15:10:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3310000000000003e-06 2020-08-07T15:10:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731345+0000", 2020-08-07T15:10:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8952999999999998e-05 2020-08-07T15:10:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731384+0000", 2020-08-07T15:10:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0036e-05 2020-08-07T15:10:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731394+0000", 2020-08-07T15:10:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75027681300000004 2020-08-07T15:10:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481670+0000", 2020-08-07T15:10:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016555899999999999 2020-08-07T15:10:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481836+0000", 2020-08-07T15:10:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028760999999999999 2020-08-07T15:10:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:10:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482124+0000", 2020-08-07T15:10:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038342680000000001 2020-08-07T15:10:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485958+0000", 2020-08-07T15:10:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2559999999999999e-06 2020-08-07T15:10:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485959+0000", 2020-08-07T15:10:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3758999999999995e-05 2020-08-07T15:10:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486043+0000", 2020-08-07T15:10:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057590419999999998 2020-08-07T15:10:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491802+0000", 2020-08-07T15:10:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5757000000000001e-05 2020-08-07T15:10:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:10:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9299999999999997e-07 2020-08-07T15:10:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:13.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:10:13.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012496999999999999 2020-08-07T15:10:13.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:13.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491993+0000", 2020-08-07T15:10:13.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001 2020-08-07T15:10:13.663 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31011 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:10:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:10:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.306796728999998, 2020-08-07T15:10:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997, 2020-08-07T15:10:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31011 2020-08-07T15:10:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:10:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:10:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9539999999999997e-05 2020-08-07T15:10:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731433+0000", 2020-08-07T15:10:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9209999999999997e-06 2020-08-07T15:10:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731439+0000", 2020-08-07T15:10:13.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4246999999999999e-05 2020-08-07T15:10:13.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:13.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731453+0000", 2020-08-07T15:10:13.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8130000000000003e-06 2020-08-07T15:10:13.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:13.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731458+0000", 2020-08-07T15:10:13.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6668000000000001e-05 2020-08-07T15:10:13.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731504+0000", 2020-08-07T15:10:13.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1112e-05 2020-08-07T15:10:13.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:13.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731515+0000", 2020-08-07T15:10:13.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75121332100000004 2020-08-07T15:10:13.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482729+0000", 2020-08-07T15:10:13.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1810000000000004e-05 2020-08-07T15:10:13.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:13.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482811+0000", 2020-08-07T15:10:13.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018001399999999999 2020-08-07T15:10:13.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:10:13.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482991+0000", 2020-08-07T15:10:13.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030701140000000001 2020-08-07T15:10:13.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:13.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486061+0000", 2020-08-07T15:10:13.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4499999999999996e-07 2020-08-07T15:10:13.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:13.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486062+0000", 2020-08-07T15:10:13.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5366999999999998e-05 2020-08-07T15:10:13.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:13.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486147+0000", 2020-08-07T15:10:13.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056872779999999996 2020-08-07T15:10:13.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:13.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491834+0000", 2020-08-07T15:10:13.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024645100000000002 2020-08-07T15:10:13.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:13.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:10:13.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1200000000000003e-07 2020-08-07T15:10:13.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:13.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:10:13.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1451000000000002e-05 2020-08-07T15:10:13.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:13.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492173+0000", 2020-08-07T15:10:13.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997 2020-08-07T15:10:13.724 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.724 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:13.724 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.725 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31012 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:10:13.725 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:10:13.725 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.306142609999995, 2020-08-07T15:10:13.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003, 2020-08-07T15:10:13.725 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:13.726 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:13.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:13.727 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:13.727 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:13.727 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31012 2020-08-07T15:10:13.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.727 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:13.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:13.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:10:13.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:13.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:13.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:10:13.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6344999999999997e-05 2020-08-07T15:10:13.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:13.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732094+0000", 2020-08-07T15:10:13.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2920000000000001e-06 2020-08-07T15:10:13.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:13.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732100+0000", 2020-08-07T15:10:13.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.189e-05 2020-08-07T15:10:13.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:13.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732112+0000", 2020-08-07T15:10:13.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3720000000000001e-06 2020-08-07T15:10:13.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:13.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732118+0000", 2020-08-07T15:10:13.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2514000000000004e-05 2020-08-07T15:10:13.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732191+0000", 2020-08-07T15:10:13.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4610000000000001e-05 2020-08-07T15:10:13.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:13.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732205+0000", 2020-08-07T15:10:13.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75132974100000005 2020-08-07T15:10:13.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483535+0000", 2020-08-07T15:10:13.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9185999999999996e-05 2020-08-07T15:10:13.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:13.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483624+0000", 2020-08-07T15:10:13.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015683 2020-08-07T15:10:13.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:10:13.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483781+0000", 2020-08-07T15:10:13.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046202889999999996 2020-08-07T15:10:13.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:13.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488401+0000", 2020-08-07T15:10:13.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6700000000000002e-07 2020-08-07T15:10:13.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:13.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488402+0000", 2020-08-07T15:10:13.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1487000000000001e-05 2020-08-07T15:10:13.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:13.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488484+0000", 2020-08-07T15:10:13.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035520830000000002 2020-08-07T15:10:13.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:13.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492036+0000", 2020-08-07T15:10:13.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029266600000000002 2020-08-07T15:10:13.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:13.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492328+0000", 2020-08-07T15:10:13.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5099999999999998e-07 2020-08-07T15:10:13.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:13.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492329+0000", 2020-08-07T15:10:13.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000201299 2020-08-07T15:10:13.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:13.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492531+0000", 2020-08-07T15:10:13.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003 2020-08-07T15:10:13.761 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.761 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:13.761 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.762 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31013 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:10:13.762 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:10:13.762 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.305930978999996, 2020-08-07T15:10:13.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996, 2020-08-07T15:10:13.763 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:13.763 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:13.763 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:13.763 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:13.764 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:13.764 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31013 2020-08-07T15:10:13.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.764 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:13.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:13.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:10:13.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:13.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:13.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:10:13.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8949000000000002e-05 2020-08-07T15:10:13.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:13.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732298+0000", 2020-08-07T15:10:13.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1399999999999997e-06 2020-08-07T15:10:13.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:13.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732304+0000", 2020-08-07T15:10:13.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.154e-05 2020-08-07T15:10:13.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:13.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732315+0000", 2020-08-07T15:10:13.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1319999999999999e-06 2020-08-07T15:10:13.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:13.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732322+0000", 2020-08-07T15:10:13.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7431e-05 2020-08-07T15:10:13.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732369+0000", 2020-08-07T15:10:13.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0621999999999999e-05 2020-08-07T15:10:13.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:13.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732380+0000", 2020-08-07T15:10:13.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75196845999999995 2020-08-07T15:10:13.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484348+0000", 2020-08-07T15:10:13.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000123008 2020-08-07T15:10:13.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:13.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484471+0000", 2020-08-07T15:10:13.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017572700000000001 2020-08-07T15:10:13.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:10:13.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484647+0000", 2020-08-07T15:10:13.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038557650000000001 2020-08-07T15:10:13.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:13.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:10:13.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2020-08-07T15:10:13.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:13.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:10:13.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0714e-05 2020-08-07T15:10:13.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:13.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488584+0000", 2020-08-07T15:10:13.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036298319999999999 2020-08-07T15:10:13.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:13.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492214+0000", 2020-08-07T15:10:13.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039530099999999997 2020-08-07T15:10:13.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:13.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492609+0000", 2020-08-07T15:10:13.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5300000000000002e-07 2020-08-07T15:10:13.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:13.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492610+0000", 2020-08-07T15:10:13.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197321 2020-08-07T15:10:13.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:13.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492807+0000", 2020-08-07T15:10:13.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996 2020-08-07T15:10:13.784 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.784 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:13.784 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.785 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31014 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:10:13.785 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:10:13.785 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.304548835000006, 2020-08-07T15:10:13.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998, 2020-08-07T15:10:13.786 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:13.786 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:13.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:13.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:13.787 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:13.787 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31014 2020-08-07T15:10:13.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.788 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:13.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:13.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:10:13.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:13.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:13.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:10:13.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3743999999999999e-05 2020-08-07T15:10:13.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:13.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733685+0000", 2020-08-07T15:10:13.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1720000000000007e-06 2020-08-07T15:10:13.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:13.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733693+0000", 2020-08-07T15:10:13.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3674e-05 2020-08-07T15:10:13.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:13.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733707+0000", 2020-08-07T15:10:13.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9249999999999998e-06 2020-08-07T15:10:13.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:13.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733714+0000", 2020-08-07T15:10:13.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8338000000000003e-05 2020-08-07T15:10:13.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733772+0000", 2020-08-07T15:10:13.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3019e-05 2020-08-07T15:10:13.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:13.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733785+0000", 2020-08-07T15:10:13.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75134856299999997 2020-08-07T15:10:13.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485133+0000", 2020-08-07T15:10:13.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8739999999999995e-05 2020-08-07T15:10:13.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:13.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485212+0000", 2020-08-07T15:10:13.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151244 2020-08-07T15:10:13.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:10:13.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485363+0000", 2020-08-07T15:10:13.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032929090000000001 2020-08-07T15:10:13.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:13.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488656+0000", 2020-08-07T15:10:13.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.065e-06 2020-08-07T15:10:13.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:13.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488657+0000", 2020-08-07T15:10:13.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1123000000000003e-05 2020-08-07T15:10:13.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:13.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488729+0000", 2020-08-07T15:10:13.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035508330000000002 2020-08-07T15:10:13.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:13.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492279+0000", 2020-08-07T15:10:13.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00059376000000000003 2020-08-07T15:10:13.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:13.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492873+0000", 2020-08-07T15:10:13.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5700000000000002e-07 2020-08-07T15:10:13.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:13.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492874+0000", 2020-08-07T15:10:13.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019264 2020-08-07T15:10:13.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:13.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493066+0000", 2020-08-07T15:10:13.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998 2020-08-07T15:10:13.807 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.807 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:13.808 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.809 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31015 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:10:13.809 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:10:13.817 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.304422869000007, 2020-08-07T15:10:13.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002, 2020-08-07T15:10:13.818 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:13.818 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:13.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:13.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:13.820 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:13.820 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31015 2020-08-07T15:10:13.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.820 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:13.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:13.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:10:13.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:13.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:13.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:10:13.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3707000000000001e-05 2020-08-07T15:10:13.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:13.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733791+0000", 2020-08-07T15:10:13.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8120000000000001e-06 2020-08-07T15:10:13.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:13.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733794+0000", 2020-08-07T15:10:13.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0728e-05 2020-08-07T15:10:13.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:13.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733805+0000", 2020-08-07T15:10:13.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8279999999999999e-06 2020-08-07T15:10:13.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:13.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733811+0000", 2020-08-07T15:10:13.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2955999999999999e-05 2020-08-07T15:10:13.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733864+0000", 2020-08-07T15:10:13.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1389000000000001e-05 2020-08-07T15:10:13.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:13.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733875+0000", 2020-08-07T15:10:13.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75232711100000005 2020-08-07T15:10:13.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486202+0000", 2020-08-07T15:10:13.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027503400000000001 2020-08-07T15:10:13.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:13.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486478+0000", 2020-08-07T15:10:13.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033010400000000001 2020-08-07T15:10:13.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:10:13.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486808+0000", 2020-08-07T15:10:13.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019382609999999999 2020-08-07T15:10:13.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:13.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488746+0000", 2020-08-07T15:10:13.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9400000000000004e-07 2020-08-07T15:10:13.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:13.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488747+0000", 2020-08-07T15:10:13.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4324000000000002e-05 2020-08-07T15:10:13.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:13.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488821+0000", 2020-08-07T15:10:13.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043745640000000001 2020-08-07T15:10:13.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:13.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493196+0000", 2020-08-07T15:10:13.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030381000000000001 2020-08-07T15:10:13.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:13.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493499+0000", 2020-08-07T15:10:13.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4e-07 2020-08-07T15:10:13.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:13.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493500+0000", 2020-08-07T15:10:13.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017499 2020-08-07T15:10:13.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:13.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493675+0000", 2020-08-07T15:10:13.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002 2020-08-07T15:10:13.846 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.846 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:13.846 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.847 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31016 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:10:13.847 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:10:13.847 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.303826893999997, 2020-08-07T15:10:13.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971, 2020-08-07T15:10:13.848 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:13.848 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:13.848 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:13.848 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:13.848 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:13.849 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31016 2020-08-07T15:10:13.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.849 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:13.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:13.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:10:13.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:10:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0924000000000002e-05 2020-08-07T15:10:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734404+0000", 2020-08-07T15:10:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2020-08-07T15:10:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734410+0000", 2020-08-07T15:10:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3459999999999994e-06 2020-08-07T15:10:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734418+0000", 2020-08-07T15:10:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9640000000000002e-06 2020-08-07T15:10:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734423+0000", 2020-08-07T15:10:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7253999999999999e-05 2020-08-07T15:10:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734470+0000", 2020-08-07T15:10:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0249999999999997e-06 2020-08-07T15:10:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734477+0000", 2020-08-07T15:10:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75294592900000001 2020-08-07T15:10:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487423+0000", 2020-08-07T15:10:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031604300000000001 2020-08-07T15:10:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487739+0000", 2020-08-07T15:10:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000171149 2020-08-07T15:10:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:10:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487910+0000", 2020-08-07T15:10:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024207769999999998 2020-08-07T15:10:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490331+0000", 2020-08-07T15:10:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1650000000000001e-06 2020-08-07T15:10:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490332+0000", 2020-08-07T15:10:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9796999999999996e-05 2020-08-07T15:10:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490402+0000", 2020-08-07T15:10:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029928849999999998 2020-08-07T15:10:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493395+0000", 2020-08-07T15:10:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034903199999999999 2020-08-07T15:10:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493744+0000", 2020-08-07T15:10:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8899999999999999e-07 2020-08-07T15:10:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493745+0000", 2020-08-07T15:10:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197362 2020-08-07T15:10:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:13.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:13.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493942+0000", 2020-08-07T15:10:13.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971 2020-08-07T15:10:13.980 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.980 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:13.980 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.980 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:13.980 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:13.981 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:10:13.981 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:10:13.983 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:10:13.985 INFO:teuthology.orchestra.run.smithi074.stderr:pool 'unique_pool_1' removed 2020-08-07T15:10:13.985 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.5 to 1 (10000) 2020-08-07T15:10:13.987 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:10:13.987 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2020-08-07T15:10:13.988 INFO:tasks.radosbench:Beginning radosbench... 2020-08-07T15:10:13.989 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_2 2020-08-07T15:10:13.990 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create unique_pool_2 16 2020-08-07T15:10:13.991 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 6 1 2020-08-07T15:10:13.992 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:13.982+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:14.088 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:14.087+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:14.123 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:10:14.124 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:10:14.124 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2020-08-07T15:10:14.124 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:10:14.149 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:10:14.191 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:14.191+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:14.219 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:10:14.242 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:10:14.275 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:10:14.278 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:10:14.279 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:10:14.279 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:10:14.279 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:10:14.279 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:10:14.288 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:10:14.302 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:10:14.303 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:14.301+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:14.322 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:10:14.345 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:10:14.371 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:10:14.399 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:10:14.399 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:10:14.400 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:10:14.400 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:10:14.400 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:10:14.400 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:10:14.400 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:10:14.401 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:10:14.401 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:10:14.409 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:14.408+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:14.438 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:14.438 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:14.439 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:10:14.511 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:14.509+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:14.560 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:14.560 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:14.569 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:10:14.615 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:14.614+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:14.723 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:14.715+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:14.743 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:14.743 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:14.744 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:10:14.838 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:14.835+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:14.914 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:14.915 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:14.916 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:10:14.926 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:14.925+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:15.028 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:15.026+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:15.074 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:15.075 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:15.087 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:10:15.141 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:15.138+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:15.214 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:15.215 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:15.218 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:10:15.229 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:15.228+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:15.252 INFO:teuthology.orchestra.run.smithi074.stderr:pool 'unique_pool_2' created 2020-08-07T15:10:15.252 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.6 to 1 (10000) 2020-08-07T15:10:15.269 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool application enable unique_pool_2 rados --yes-i-really-mean-it || true 2020-08-07T15:10:15.271 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 7 1 2020-08-07T15:10:15.335 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:15.334+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:15.394 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:15.395 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:15.395 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:10:15.440 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:15.438+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:15.527 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:15.527 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:15.533 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:10:15.552 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:15.539+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:15.645 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:15.642+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:15.671 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:15.672 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:15.673 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:10:15.745 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:15.743+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:15.791 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:15.792 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:15.804 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:10:15.847 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:15.846+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:15.921 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:15.922 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:15.931 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:10:15.950 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:15.948+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:15.992 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:10:16.080 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:16.054+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:16.080 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:16.080 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:16.081 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:10:16.166 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:16.164+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:16.242 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:16.242 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:16.255 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:10:16.256 INFO:teuthology.orchestra.run.smithi074.stderr:enabled application 'rados' on pool 'unique_pool_2' 2020-08-07T15:10:16.259 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:16.258+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:16.260 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.7 to 1 (10000) 2020-08-07T15:10:16.278 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2020-08-07T15:10:16.279 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:10:16.364 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:16.361+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:16.391 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:16.392 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:16.402 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:10:16.414 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:10:16.482 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:16.465+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:16.533 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:16.534 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:16.540 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:10:16.594 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:16.590+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:16.602 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:10:16.620 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:10:16.626 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:10:16.626 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:10:16.629 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2020-08-07T15:10:16.244280+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":87,"acting":87,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8395732,"kb_used_data":7060,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746578988,"statfs":{"total":773094113280,"available":764496883712,"internally_reserved":8589934592,"allocated":7229440,"data_stored":2829468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-130765,"num_write_kb":-929865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2645,"num_bytes_recovered":-173219840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.588198"},"pg_stats":[{"pgid":"1.14","version":"16'2","reported_seq":"94","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.644555+0000","last_change":"2020-08-07T15:10:10.644555+0000","last_active":"2020-08-07T15:10:10.644555+0000","last_peered":"2020-08-07T15:10:10.644555+0000","last_clean":"2020-08-07T15:10:10.644555+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:10:10.644555+0000","last_undegraded":"2020-08-07T15:10:10.644555+0000","last_fullsized":"2020-08-07T15:10:10.644555+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:10.644460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.644460+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"163","reported_epoch":"160","state":"active+clean","last_fresh":"2020-08-07T15:10:09.315412+0000","last_change":"2020-08-07T15:10:07.295971+0000","last_active":"2020-08-07T15:10:09.315412+0000","last_peered":"2020-08-07T15:10:09.315412+0000","last_clean":"2020-08-07T15:10:09.315412+0000","last_became_active":"2020-08-07T15:09:30.203592+0000","last_became_peered":"2020-08-07T15:09:30.203592+0000","last_unstale":"2020-08-07T15:10:09.315412+0000","last_undegraded":"2020-08-07T15:10:09.315412+0000","last_fullsized":"2020-08-07T15:10:09.315412+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:07.295863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:07.295863+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"91","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.218153+0000","last_change":"2020-08-07T15:08:55.078015+0000","last_active":"2020-08-07T15:10:10.218153+0000","last_peered":"2020-08-07T15:10:10.218153+0000","last_clean":"2020-08-07T15:10:10.218153+0000","last_became_active":"2020-08-07T15:08:41.685106+0000","last_became_peered":"2020-08-07T15:08:41.685106+0000","last_unstale":"2020-08-07T15:10:10.218153+0000","last_undegraded":"2020-08-07T15:10:10.218153+0000","last_fullsized":"2020-08-07T15:10:10.218153+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:55.077924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:55.077924+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"204","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.217677+0000","last_change":"2020-08-07T15:09:30.210669+0000","last_active":"2020-08-07T15:10:10.217677+0000","last_peered":"2020-08-07T15:10:10.217677+0000","last_clean":"2020-08-07T15:10:10.217677+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:10:10.217677+0000","last_undegraded":"2020-08-07T15:10:10.217677+0000","last_fullsized":"2020-08-07T15:10:10.217677+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.544324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.544324+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"198","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.217706+0000","last_change":"2020-08-07T15:08:51.599032+0000","last_active":"2020-08-07T15:10:10.217706+0000","last_peered":"2020-08-07T15:10:10.217706+0000","last_clean":"2020-08-07T15:10:10.217706+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:10:10.217706+0000","last_undegraded":"2020-08-07T15:10:10.217706+0000","last_fullsized":"2020-08-07T15:10:10.217706+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:51.598947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.598947+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"159","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.644530+0000","last_change":"2020-08-07T15:09:41.284321+0000","last_active":"2020-08-07T15:10:10.644530+0000","last_peered":"2020-08-07T15:10:10.644530+0000","last_clean":"2020-08-07T15:10:10.644530+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:10:10.644530+0000","last_undegraded":"2020-08-07T15:10:10.644530+0000","last_fullsized":"2020-08-07T15:10:10.644530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:41.284237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:41.284237+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"74","reported_epoch":"160","state":"active+clean","last_fresh":"2020-08-07T15:10:09.316521+0000","last_change":"2020-08-07T15:09:30.283753+0000","last_active":"2020-08-07T15:10:09.316521+0000","last_peered":"2020-08-07T15:10:09.316521+0000","last_clean":"2020-08-07T15:10:09.316521+0000","last_became_active":"2020-08-07T15:09:30.283176+0000","last_became_peered":"2020-08-07T15:09:30.283176+0000","last_unstale":"2020-08-07T15:10:09.316521+0000","last_undegraded":"2020-08-07T15:10:09.316521+0000","last_fullsized":"2020-08-07T15:10:09.316521+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:25.606757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:25.606757+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"158","reported_epoch":"160","state":"active+clean","last_fresh":"2020-08-07T15:10:09.316063+0000","last_change":"2020-08-07T15:08:40.057705+0000","last_active":"2020-08-07T15:10:09.316063+0000","last_peered":"2020-08-07T15:10:09.316063+0000","last_clean":"2020-08-07T15:10:09.316063+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:10:09.316063+0000","last_undegraded":"2020-08-07T15:10:09.316063+0000","last_fullsized":"2020-08-07T15:10:09.316063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:40.057622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:40.057622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"116","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.268900+0000","last_change":"2020-08-07T15:08:53.756554+0000","last_active":"2020-08-07T15:10:10.268900+0000","last_peered":"2020-08-07T15:10:10.268900+0000","last_clean":"2020-08-07T15:10:10.268900+0000","last_became_active":"2020-08-07T15:08:46.710404+0000","last_became_peered":"2020-08-07T15:08:46.710404+0000","last_unstale":"2020-08-07T15:10:10.268900+0000","last_undegraded":"2020-08-07T15:10:10.268900+0000","last_fullsized":"2020-08-07T15:10:10.268900+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:53.756343+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:53.756343+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"158","reported_epoch":"160","state":"active+clean","last_fresh":"2020-08-07T15:10:09.316308+0000","last_change":"2020-08-07T15:09:54.908283+0000","last_active":"2020-08-07T15:10:09.316308+0000","last_peered":"2020-08-07T15:10:09.316308+0000","last_clean":"2020-08-07T15:10:09.316308+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:10:09.316308+0000","last_undegraded":"2020-08-07T15:10:09.316308+0000","last_fullsized":"2020-08-07T15:10:09.316308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:54.908197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:54.908197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"198","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.644613+0000","last_change":"2020-08-07T15:10:07.302258+0000","last_active":"2020-08-07T15:10:10.644613+0000","last_peered":"2020-08-07T15:10:10.644613+0000","last_clean":"2020-08-07T15:10:10.644613+0000","last_became_active":"2020-08-07T15:09:30.201829+0000","last_became_peered":"2020-08-07T15:09:30.201829+0000","last_unstale":"2020-08-07T15:10:10.644613+0000","last_undegraded":"2020-08-07T15:10:10.644613+0000","last_fullsized":"2020-08-07T15:10:10.644613+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:07.302204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:07.302204+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"158","reported_epoch":"160","state":"active+clean","last_fresh":"2020-08-07T15:10:09.316031+0000","last_change":"2020-08-07T15:09:51.036281+0000","last_active":"2020-08-07T15:10:09.316031+0000","last_peered":"2020-08-07T15:10:09.316031+0000","last_clean":"2020-08-07T15:10:09.316031+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:10:09.316031+0000","last_undegraded":"2020-08-07T15:10:09.316031+0000","last_fullsized":"2020-08-07T15:10:09.316031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:51.036224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:51.036224+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"161","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.644228+0000","last_change":"2020-08-07T15:08:50.686578+0000","last_active":"2020-08-07T15:10:10.644228+0000","last_peered":"2020-08-07T15:10:10.644228+0000","last_clean":"2020-08-07T15:10:10.644228+0000","last_became_active":"2020-08-07T15:08:36.633406+0000","last_became_peered":"2020-08-07T15:08:36.633406+0000","last_unstale":"2020-08-07T15:10:10.644228+0000","last_undegraded":"2020-08-07T15:10:10.644228+0000","last_fullsized":"2020-08-07T15:10:10.644228+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:50.686496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:50.686496+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"190","reported_epoch":"160","state":"active+clean","last_fresh":"2020-08-07T15:10:09.315942+0000","last_change":"2020-08-07T15:09:55.879867+0000","last_active":"2020-08-07T15:10:09.315942+0000","last_peered":"2020-08-07T15:10:09.315942+0000","last_clean":"2020-08-07T15:10:09.315942+0000","last_became_active":"2020-08-07T15:08:36.627426+0000","last_became_peered":"2020-08-07T15:08:36.627426+0000","last_unstale":"2020-08-07T15:10:09.315942+0000","last_undegraded":"2020-08-07T15:10:09.315942+0000","last_fullsized":"2020-08-07T15:10:09.315942+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:55.879785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:55.879785+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"159","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.218565+0000","last_change":"2020-08-07T15:10:08.656952+0000","last_active":"2020-08-07T15:10:10.218565+0000","last_peered":"2020-08-07T15:10:10.218565+0000","last_clean":"2020-08-07T15:10:10.218565+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:10:10.218565+0000","last_undegraded":"2020-08-07T15:10:10.218565+0000","last_fullsized":"2020-08-07T15:10:10.218565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:08.656855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:10:08.656855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"177","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.268682+0000","last_change":"2020-08-07T15:09:27.181676+0000","last_active":"2020-08-07T15:10:10.268682+0000","last_peered":"2020-08-07T15:10:10.268682+0000","last_clean":"2020-08-07T15:10:10.268682+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:10:10.268682+0000","last_undegraded":"2020-08-07T15:10:10.268682+0000","last_fullsized":"2020-08-07T15:10:10.268682+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:05.606603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:05.606603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"203","reported_epoch":"160","state":"active+clean","last_fresh":"2020-08-07T15:10:09.316039+0000","last_change":"2020-08-07T15:08:42.069637+0000","last_active":"2020-08-07T15:10:09.316039+0000","last_peered":"2020-08-07T15:10:09.316039+0000","last_clean":"2020-08-07T15:10:09.316039+0000","last_became_active":"2020-08-07T15:08:36.628365+0000","last_became_peered":"2020-08-07T15:08:36.628365+0000","last_unstale":"2020-08-07T15:10:09.316039+0000","last_undegraded":"2020-08-07T15:10:09.316039+0000","last_fullsized":"2020-08-07T15:10:09.316039+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:42.069585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:42.069585+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"159","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.218370+0000","last_change":"2020-08-07T15:09:52.637867+0000","last_active":"2020-08-07T15:10:10.218370+0000","last_peered":"2020-08-07T15:10:10.218370+0000","last_clean":"2020-08-07T15:10:10.218370+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:10:10.218370+0000","last_undegraded":"2020-08-07T15:10:10.218370+0000","last_fullsized":"2020-08-07T15:10:10.218370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:52.637791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:09:52.637791+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"157","reported_epoch":"162","state":"active+clean","last_fresh":"2020-08-07T15:10:11.720144+0000","last_change":"2020-08-07T15:10:11.720217+0000","last_active":"2020-08-07T15:10:11.720144+0000","last_peered":"2020-08-07T15:10:11.720144+0000","last_clean":"2020-08-07T15:10:11.720144+0000","last_became_active":"2020-08-07T15:08:36.618164+0000","last_became_peered":"2020-08-07T15:08:36.618164+0000","last_unstale":"2020-08-07T15:10:11.720144+0000","last_undegraded":"2020-08-07T15:10:11.720144+0000","last_fullsized":"2020-08-07T15:10:11.720144+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:11.720143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:11.720143+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"155","reported_epoch":"160","state":"active+clean","last_fresh":"2020-08-07T15:10:09.316235+0000","last_change":"2020-08-07T15:09:37.017383+0000","last_active":"2020-08-07T15:10:09.316235+0000","last_peered":"2020-08-07T15:10:09.316235+0000","last_clean":"2020-08-07T15:10:09.316235+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:10:09.316235+0000","last_undegraded":"2020-08-07T15:10:09.316235+0000","last_fullsized":"2020-08-07T15:10:09.316235+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:37.017307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:37.017307+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"229","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.644588+0000","last_change":"2020-08-07T15:09:30.208873+0000","last_active":"2020-08-07T15:10:10.644588+0000","last_peered":"2020-08-07T15:10:10.644588+0000","last_clean":"2020-08-07T15:10:10.644588+0000","last_became_active":"2020-08-07T15:09:30.208345+0000","last_became_peered":"2020-08-07T15:09:30.208345+0000","last_unstale":"2020-08-07T15:10:10.644588+0000","last_undegraded":"2020-08-07T15:10:10.644588+0000","last_fullsized":"2020-08-07T15:10:10.644588+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.381325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.381325+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"134","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.715147+0000","last_change":"2020-08-07T15:10:10.715147+0000","last_active":"2020-08-07T15:10:10.715147+0000","last_peered":"2020-08-07T15:10:10.715147+0000","last_clean":"2020-08-07T15:10:10.715147+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:10:10.715147+0000","last_undegraded":"2020-08-07T15:10:10.715147+0000","last_fullsized":"2020-08-07T15:10:10.715147+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.715051+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"131","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.217821+0000","last_change":"2020-08-07T15:09:09.691838+0000","last_active":"2020-08-07T15:10:10.217821+0000","last_peered":"2020-08-07T15:10:10.217821+0000","last_clean":"2020-08-07T15:10:10.217821+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:10:10.217821+0000","last_undegraded":"2020-08-07T15:10:10.217821+0000","last_fullsized":"2020-08-07T15:10:10.217821+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.691753+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"194","reported_epoch":"160","state":"active+clean","last_fresh":"2020-08-07T15:10:09.315907+0000","last_change":"2020-08-07T15:09:25.984108+0000","last_active":"2020-08-07T15:10:09.315907+0000","last_peered":"2020-08-07T15:10:09.315907+0000","last_clean":"2020-08-07T15:10:09.315907+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:10:09.315907+0000","last_undegraded":"2020-08-07T15:10:09.315907+0000","last_fullsized":"2020-08-07T15:10:09.315907+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:25.984018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:25.984018+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"152","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.962481+0000","last_change":"2020-08-07T15:10:10.962565+0000","last_active":"2020-08-07T15:10:10.962481+0000","last_peered":"2020-08-07T15:10:10.962481+0000","last_clean":"2020-08-07T15:10:10.962481+0000","last_became_active":"2020-08-07T15:08:41.680448+0000","last_became_peered":"2020-08-07T15:08:41.680448+0000","last_unstale":"2020-08-07T15:10:10.962481+0000","last_undegraded":"2020-08-07T15:10:10.962481+0000","last_fullsized":"2020-08-07T15:10:10.962481+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:10:10.962480+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.962480+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"120","reported_epoch":"160","state":"active+clean","last_fresh":"2020-08-07T15:10:09.316583+0000","last_change":"2020-08-07T15:09:36.687393+0000","last_active":"2020-08-07T15:10:09.316583+0000","last_peered":"2020-08-07T15:10:09.316583+0000","last_clean":"2020-08-07T15:10:09.316583+0000","last_became_active":"2020-08-07T15:09:30.204655+0000","last_became_peered":"2020-08-07T15:09:30.204655+0000","last_unstale":"2020-08-07T15:10:09.316583+0000","last_undegraded":"2020-08-07T15:10:09.316583+0000","last_fullsized":"2020-08-07T15:10:09.316583+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:36.687314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:36.687314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"194","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.644374+0000","last_change":"2020-08-07T15:09:08.399507+0000","last_active":"2020-08-07T15:10:10.644374+0000","last_peered":"2020-08-07T15:10:10.644374+0000","last_clean":"2020-08-07T15:10:10.644374+0000","last_became_active":"2020-08-07T15:08:41.689435+0000","last_became_peered":"2020-08-07T15:08:41.689435+0000","last_unstale":"2020-08-07T15:10:10.644374+0000","last_undegraded":"2020-08-07T15:10:10.644374+0000","last_fullsized":"2020-08-07T15:10:10.644374+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:08.399439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:08.399439+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"80","reported_epoch":"160","state":"active+clean","last_fresh":"2020-08-07T15:10:09.316344+0000","last_change":"2020-08-07T15:09:09.868283+0000","last_active":"2020-08-07T15:10:09.316344+0000","last_peered":"2020-08-07T15:10:09.316344+0000","last_clean":"2020-08-07T15:10:09.316344+0000","last_became_active":"2020-08-07T15:08:46.710059+0000","last_became_peered":"2020-08-07T15:08:46.710059+0000","last_unstale":"2020-08-07T15:10:09.316344+0000","last_undegraded":"2020-08-07T15:10:09.316344+0000","last_fullsized":"2020-08-07T15:10:09.316344+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.868198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.868198+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"103","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.218309+0000","last_change":"2020-08-07T15:09:08.653773+0000","last_active":"2020-08-07T15:10:10.218309+0000","last_peered":"2020-08-07T15:10:10.218309+0000","last_clean":"2020-08-07T15:10:10.218309+0000","last_became_active":"2020-08-07T15:08:41.677944+0000","last_became_peered":"2020-08-07T15:08:41.677944+0000","last_unstale":"2020-08-07T15:10:10.218309+0000","last_undegraded":"2020-08-07T15:10:10.218309+0000","last_fullsized":"2020-08-07T15:10:10.218309+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:08.653722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:08.653722+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":63,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542193,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049488,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322352,"statfs":{"total":96636764160,"available":95562088448,"internally_reserved":1073741824,"allocated":925696,"data_stored":360808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.95099999999999996,"15min":0.91200000000000003},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.1739999999999999,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.095,"5min":1.0149999999999999,"15min":0.97999999999999998},"min":{"1min":0.54300000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.2430000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.357,"15min":1.3149999999999999},"min":{"1min":0.68700000000000006,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":5.6239999999999997,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.105,"5min":1.0720000000000001,"15min":1.0009999999999999},"min":{"1min":0.64100000000000001,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.206,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":0.92800000000000005}]},{"osd":2,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.272,"15min":1.175},"min":{"1min":0.71999999999999997,"5min":0.628,"15min":0.628},"max":{"1min":5.6619999999999999,"5min":5.6619999999999999,"15min":5.6619999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.446,"5min":1.26,"15min":1.165},"min":{"1min":0.66800000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":5.1319999999999997,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":1.0309999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":75.045000000000002,"5min":16.263999999999999,"15min":6.5359999999999996},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":917.38699999999994,"5min":917.38699999999994,"15min":917.38699999999994},"last":1.2969999999999999},{"interface":"front","average":{"1min":19.670999999999999,"5min":4.9939999999999998,"15min":2.5600000000000001},"min":{"1min":0.80000000000000004,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":333.59199999999998,"5min":333.59199999999998,"15min":333.59199999999998},"last":1.2729999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.411,"15min":1.321},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":4.5250000000000004,"5min":4.835,"15min":4.835},"last":0.998},{"interface":"front","average":{"1min":52.259999999999998,"5min":11.52,"15min":4.7140000000000004},"min":{"1min":0.77000000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":866.24099999999999,"5min":866.24099999999999,"15min":866.24099999999999},"last":1.339}]},{"osd":5,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.2869999999999999,"5min":23.440000000000001,"15min":32.817999999999998},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":6.6509999999999998,"5min":831.88099999999997,"15min":831.88099999999997},"last":1.1659999999999999},{"interface":"front","average":{"1min":2.2519999999999998,"5min":1.8380000000000001,"15min":1.609},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":6.6790000000000003,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":1.1850000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":13.818,"5min":4.0019999999999998,"15min":2.3460000000000001},"min":{"1min":0.82999999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":224.14500000000001,"5min":224.14500000000001,"15min":224.14500000000001},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.6839999999999999,"15min":1.5660000000000001},"min":{"1min":0.90600000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":7.4359999999999999,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":1.2190000000000001}]}]},{"osd":7,"up_from":13,"seq":55834574898,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049460,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322380,"statfs":{"total":96636764160,"available":95562117120,"internally_reserved":1073741824,"allocated":897024,"data_stored":351306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.179,"15min":1.2809999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8149999999999999,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":15.239000000000001,"5min":3.9529999999999998,"15min":2.0840000000000001},"min":{"1min":0.78100000000000003,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":320.31299999999999,"5min":320.31299999999999,"15min":320.31299999999999},"last":0.79300000000000004}]},{"osd":1,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":24.821999999999999,"5min":5.7329999999999997,"15min":2.5489999999999999},"min":{"1min":0.46700000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":547.65300000000002,"5min":547.65300000000002,"15min":547.65300000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":4.9130000000000003,"15min":2.5619999999999998},"min":{"1min":0.69799999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.5449999999999999,"5min":372.50799999999998,"15min":372.50799999999998},"last":1.0860000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.208,"15min":1.1990000000000001},"min":{"1min":0.502,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":5.2400000000000002,"5min":5.2400000000000002,"15min":5.2400000000000002},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.286,"5min":1.2390000000000001,"15min":1.2609999999999999},"min":{"1min":0.66800000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":6.0490000000000004,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":0.88400000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.4370000000000001,"15min":1.496},"min":{"1min":0.878,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.0670000000000002,"5min":3.46,"15min":3.46},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3600000000000001,"15min":1.3779999999999999},"min":{"1min":0.70299999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":5.2770000000000001,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":1.056}]},{"osd":4,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.504,"15min":1.6100000000000001},"min":{"1min":0.70099999999999996,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.165,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.46,"5min":1.625,"15min":1.6559999999999999},"min":{"1min":0.96199999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":2.4279999999999999,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.1399999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.2809999999999999,"15min":1.3360000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.3069999999999999,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.6259999999999999,"15min":1.7190000000000001},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.4630000000000001,"5min":5.327,"15min":5.327},"last":1.46}]},{"osd":6,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.458,"15min":1.4410000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.4900000000000002,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.351},{"interface":"front","average":{"1min":1.5720000000000001,"5min":11.603999999999999,"15min":16.085000000000001},"min":{"1min":1.004,"5min":0.87,"15min":0.87},"max":{"1min":2.7719999999999998,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.431}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049460,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322380,"statfs":{"total":96636764160,"available":95562117120,"internally_reserved":1073741824,"allocated":897024,"data_stored":351306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":20.945,"15min":29.771999999999998},"min":{"1min":0.49199999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":6.8810000000000002,"5min":497.04000000000002,"15min":497.04000000000002},"last":0.85599999999999998},{"interface":"front","average":{"1min":25.385999999999999,"5min":14.656000000000001,"15min":5.6390000000000002},"min":{"1min":0.52200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":513.78899999999999,"5min":993.84500000000003,"15min":993.84500000000003},"last":0.89700000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.999,"15min":0.93200000000000005},"min":{"1min":0.60099999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.726,"5min":4.585,"15min":4.585},"last":0.879},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.254,"15min":1.22},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":6.4029999999999996,"5min":6.4029999999999996,"15min":6.4029999999999996},"last":0.999}]},{"osd":3,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":40.524000000000001,"5min":9.2829999999999995,"15min":3.968},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":828.31600000000003,"5min":828.31600000000003,"15min":828.31600000000003},"last":0.74399999999999999},{"interface":"front","average":{"1min":44.292000000000002,"5min":10.169,"15min":4.359},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":905.38300000000004,"5min":905.38300000000004,"15min":905.38300000000004},"last":0.77000000000000002}]},{"osd":4,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":27.831,"5min":6.774,"15min":3.1139999999999999},"min":{"1min":0.79900000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":550.66800000000001,"5min":550.66800000000001,"15min":550.66800000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.3400000000000001,"15min":1.246},"min":{"1min":0.92100000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":5.9420000000000002,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.145}]},{"osd":5,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":8.4550000000000001,"15min":11.49},"min":{"1min":0.71199999999999997,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.7200000000000002,"5min":213.63800000000001,"15min":213.63800000000001},"last":1.0800000000000001},{"interface":"front","average":{"1min":58.497,"5min":12.720000000000001,"15min":5.0330000000000004},"min":{"1min":0.76500000000000001,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":875.44600000000003,"5min":875.44600000000003,"15min":875.44600000000003},"last":1.099}]},{"osd":6,"last update":"Fri Aug 7 15:09:27 2020","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.587,"15min":1.5629999999999999},"min":{"1min":0.81000000000000005,"5min":0.754,"15min":0.754},"max":{"1min":5.7149999999999999,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":1.2509999999999999},{"interface":"front","average":{"1min":11.539,"5min":3.5979999999999999,"15min":2.1059999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":207.137,"5min":207.137,"15min":207.137},"last":1.204}]},{"osd":7,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.782,"5min":1.6879999999999999,"15min":1.639},"min":{"1min":0.94099999999999995,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":6.444,"5min":6.444,"15min":6.444},"last":1.232},{"interface":"front","average":{"1min":1.6819999999999999,"5min":1.6779999999999999,"15min":1.546},"min":{"1min":0.93300000000000005,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":5.9279999999999999,"5min":10.752000000000001,"15min":10.752000000000001},"last":1.1659999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049472,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322368,"statfs":{"total":96636764160,"available":95562104832,"internally_reserved":1073741824,"allocated":909312,"data_stored":356065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":17.058,"15min":6.3369999999999997},"min":{"1min":0.61399999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.1269999999999998,"5min":845.57100000000003,"15min":845.57100000000003},"last":0.64400000000000002},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.0169999999999999,"15min":0.96099999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":4.8209999999999997,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":1.153}]},{"osd":2,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.0189999999999999,"15min":0.95799999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.7719999999999998,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":13.177,"5min":22.48,"15min":29.32},"min":{"1min":0.61399999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":226.911,"5min":606.18499999999995,"15min":606.18499999999995},"last":0.71199999999999997}]},{"osd":3,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.0660000000000001,"15min":0.97899999999999998},"min":{"1min":0.54100000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6859999999999999,"5min":3.1779999999999999,"15min":3.1779999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.3160000000000001,"15min":1.2170000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.976,"5min":4.21,"15min":4.21},"last":0.95399999999999996}]},{"osd":4,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.2829999999999999,"15min":1.2050000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":8.3100000000000005,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.7709999999999999,"5min":1.446,"15min":1.3200000000000001},"min":{"1min":0.84599999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":8.4190000000000005,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":1.0149999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.9770000000000001,"5min":9.0380000000000003,"15min":3.8300000000000001},"min":{"1min":0.77800000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":7.8929999999999998,"5min":501.267,"15min":501.267},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.2609999999999999,"15min":1.157},"min":{"1min":0.78200000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.0939999999999999,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":1.177}]},{"osd":6,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":5.7779999999999996,"15min":7.6680000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.0059999999999998,"5min":124.738,"15min":124.738},"last":1.123},{"interface":"front","average":{"1min":1.627,"5min":11.811,"15min":16.372},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":4.6879999999999997,"5min":294.94,"15min":294.94},"last":1.073}]},{"osd":7,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.5589999999999999,"15min":1.577},"min":{"1min":0.95399999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":4.7169999999999996,"5min":6.468,"15min":6.468},"last":1.333},{"interface":"front","average":{"1min":1.363,"5min":1.4410000000000001,"15min":1.4039999999999999},"min":{"1min":0.88800000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.1040000000000001,"5min":3.117,"15min":3.117},"last":1.216}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049460,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322380,"statfs":{"total":96636764160,"available":95562117120,"internally_reserved":1073741824,"allocated":897024,"data_stored":351306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.6950000000000001,"5min":1.268,"15min":1.167},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":8.4930000000000003,"5min":8.4930000000000003,"15min":8.4930000000000003},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.357,"5min":1.113,"15min":1.0209999999999999},"min":{"1min":0.61899999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":3.5,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":1.137}]},{"osd":1,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.9390000000000001,"5min":1.5469999999999999,"15min":1.2430000000000001},"min":{"1min":0.54400000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":5.6820000000000004,"5min":18.672000000000001,"15min":18.672000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.351,"15min":1.357},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":3.585,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":1.407}]},{"osd":3,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.268,"15min":1.198},"min":{"1min":0.72799999999999998,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.8740000000000001,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.0900000000000001},{"interface":"front","average":{"1min":34.170999999999999,"5min":8.0510000000000002,"15min":3.7210000000000001},"min":{"1min":0.68799999999999994,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":501.75599999999997,"5min":501.75599999999997,"15min":501.75599999999997},"last":1.024}]},{"osd":4,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.403,"15min":1.2070000000000001},"min":{"1min":0.65200000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":3.4180000000000001,"5min":19.783000000000001,"15min":19.783000000000001},"last":1.496},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.546,"15min":1.6100000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":3.5569999999999999,"5min":3.5569999999999999,"15min":3.5569999999999999},"last":1.536}]},{"osd":5,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.292,"15min":1.272},"min":{"1min":0.83299999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.323,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":1.5669999999999999},{"interface":"front","average":{"1min":43.396999999999998,"5min":24.298999999999999,"15min":25.116},"min":{"1min":0.99299999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":627.53300000000002,"5min":627.53300000000002,"15min":627.53300000000002},"last":1.48}]},{"osd":6,"last update":"Fri Aug 7 15:09:30 2020","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.538,"15min":1.3260000000000001},"min":{"1min":0.85999999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":3.452,"5min":20.035,"15min":20.035},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.498,"5min":9.8979999999999997,"15min":4.1799999999999997},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.976,"5min":751.70000000000005,"15min":751.70000000000005},"last":1.4319999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":54.209000000000003,"5min":12.595000000000001,"15min":5.359},"min":{"1min":0.84999999999999998,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":501.91699999999997,"5min":501.91699999999997,"15min":501.91699999999997},"last":1.454},{"interface":"front","average":{"1min":1.665,"5min":1.919,"15min":1.637},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":3.133,"5min":35.646999999999998,"15min":35.646999999999998},"last":1.52}]}]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049460,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322380,"statfs":{"total":96636764160,"available":95562117120,"internally_reserved":1073741824,"allocated":897024,"data_stored":351306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":1.341,"15min":1.282},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.097,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.4530000000000001,"15min":1.575},"min":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.3039999999999998,"5min":13.067,"15min":13.067},"last":0.88100000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":4.5860000000000003,"15min":2.794},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.9910000000000001,"5min":270.68700000000001,"15min":270.68700000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.4379999999999999,"5min":2.0390000000000001,"15min":2.0209999999999999},"min":{"1min":0.55100000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":7.0940000000000003,"5min":14.349,"15min":14.349},"last":1.073}]},{"osd":2,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.266,"15min":1.1870000000000001},"min":{"1min":0.71399999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.2639999999999998,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":1.3180000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":2.109,"15min":1.8260000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.661,"5min":51.530000000000001,"15min":51.530000000000001},"last":1.0940000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":10.773999999999999,"15min":4.7220000000000004},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.1360000000000001,"5min":506.21300000000002,"15min":506.21300000000002},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.276,"5min":25.469000000000001,"15min":29.888999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.8940000000000001,"5min":677.88599999999997,"15min":677.88599999999997},"last":1.266}]},{"osd":5,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.5920000000000001,"15min":1.542},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.9710000000000001,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.8009999999999999,"15min":1.7789999999999999},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":3.044,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":1.629}]},{"osd":6,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.6259999999999999,"15min":1.5629999999999999},"min":{"1min":0.89800000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.9359999999999999,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":1.5760000000000001},{"interface":"front","average":{"1min":21.766999999999999,"5min":5.9390000000000001,"15min":3.3039999999999998},"min":{"1min":0.89900000000000002,"5min":0.876,"15min":0.876},"max":{"1min":387.911,"5min":387.911,"15min":387.911},"last":1.2470000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.9330000000000001,"15min":2.085},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.0110000000000001,"5min":5.734,"15min":5.734},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.325,"5min":1.9750000000000001,"15min":1.976},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.169,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":1.224}]}]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049460,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322380,"statfs":{"total":96636764160,"available":95562117120,"internally_reserved":1073741824,"allocated":897024,"data_stored":351306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":1.0720000000000001,"15min":1.1439999999999999},"min":{"1min":0.52900000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4390000000000001,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":1.244},{"interface":"front","average":{"1min":1.143,"5min":1.0069999999999999,"15min":0.90100000000000002},"min":{"1min":0.498,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":4.907,"5min":6.3970000000000002,"15min":6.3970000000000002},"last":0.96799999999999997}]},{"osd":1,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":23.940999999999999,"5min":5.8209999999999997,"15min":2.843},"min":{"1min":0.59599999999999997,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":320.67000000000002,"5min":320.67000000000002,"15min":320.67000000000002},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":7.5700000000000003,"15min":3.4409999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.784,"5min":689.846,"15min":689.846},"last":1.224}]},{"osd":2,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.3999999999999999,"15min":1.4339999999999999},"min":{"1min":0.77600000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.0899999999999999,"5min":6.8730000000000002,"15min":6.8730000000000002},"last":1.133},{"interface":"front","average":{"1min":1.258,"5min":1.516,"15min":1.5149999999999999},"min":{"1min":0.81699999999999995,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.3159999999999998,"5min":7.1360000000000001,"15min":7.1360000000000001},"last":1.177}]},{"osd":3,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":3.681,"15min":2.2320000000000002},"min":{"1min":0.78300000000000003,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.8149999999999999,"5min":247.202,"15min":247.202},"last":1.266},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.3720000000000001,"15min":1.2909999999999999},"min":{"1min":0.68500000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":4.931,"5min":6.431,"15min":6.431},"last":1.29}]},{"osd":5,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":10.57,"15min":4.6079999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.431,"5min":990.71600000000001,"15min":990.71600000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.6100000000000001,"15min":1.621},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":3.3519999999999999,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":0.80900000000000005}]},{"osd":6,"last update":"Fri Aug 7 15:09:28 2020","interfaces":[{"interface":"back","average":{"1min":1.6799999999999999,"5min":1.7509999999999999,"15min":1.7769999999999999},"min":{"1min":0.75800000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":5.0149999999999997,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":1.014},{"interface":"front","average":{"1min":1.498,"5min":12.069000000000001,"15min":16.762},"min":{"1min":0.83299999999999996,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":3.3530000000000002,"5min":389.14600000000002,"15min":389.14600000000002},"last":0.94799999999999995}]},{"osd":7,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":3.8300000000000001,"15min":2.2149999999999999},"min":{"1min":0.753,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":4.8710000000000004,"5min":252.798,"15min":252.798},"last":1.151},{"interface":"front","average":{"1min":1.262,"5min":10.175000000000001,"15min":4.4889999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.3050000000000002,"5min":501.93000000000001,"15min":501.93000000000001},"last":0.99199999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049472,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322368,"statfs":{"total":96636764160,"available":95562104832,"internally_reserved":1073741824,"allocated":909312,"data_stored":356065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":1.012,"15min":1.0549999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.651,"5min":2.766,"15min":2.766},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91600000000000004,"15min":0.84199999999999997},"min":{"1min":0.45500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.3340000000000001,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":0.68700000000000006}]},{"osd":1,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.4540000000000002,"5min":10.521000000000001,"15min":4.1260000000000003},"min":{"1min":0.56699999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":29.283000000000001,"5min":787.22900000000004,"15min":787.22900000000004},"last":1.0469999999999999},{"interface":"front","average":{"1min":8.0939999999999994,"5min":2.4969999999999999,"15min":1.571},"min":{"1min":0.60999999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":116.548,"5min":116.548,"15min":116.548},"last":1.518}]},{"osd":2,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":5.9340000000000002,"15min":8.1240000000000006},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.78,"5min":168.56899999999999,"15min":168.56899999999999},"last":1.085},{"interface":"front","average":{"1min":1.119,"5min":20.361000000000001,"15min":28.873999999999999},"min":{"1min":0.60899999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.665,"5min":676.27099999999996,"15min":676.27099999999996},"last":1.02}]},{"osd":3,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.6739999999999999,"5min":1.5069999999999999,"15min":1.2470000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":29.129000000000001,"5min":29.129000000000001,"15min":29.129000000000001},"last":1.113},{"interface":"front","average":{"1min":3.0219999999999998,"5min":9.9459999999999997,"15min":4.1239999999999997},"min":{"1min":0.70799999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":29.103000000000002,"5min":712.98699999999997,"15min":712.98699999999997},"last":1.486}]},{"osd":4,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":49.854999999999997,"5min":40.755000000000003,"15min":47.512999999999998},"min":{"1min":0.61899999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":971.67999999999995,"5min":971.67999999999995,"15min":971.67999999999995},"last":1.575},{"interface":"front","average":{"1min":1.236,"5min":1.264,"15min":1.292},"min":{"1min":0.64900000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.5960000000000001,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":1.204}]},{"osd":6,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":91.870999999999995,"5min":48.862000000000002,"15min":49.893000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":966.60900000000004,"5min":966.60900000000004,"15min":966.60900000000004},"last":1.1719999999999999},{"interface":"front","average":{"1min":28.396999999999998,"5min":21.527000000000001,"15min":24.52},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":501.79000000000002,"5min":517.26700000000005,"15min":517.26700000000005},"last":1.548}]},{"osd":7,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.341,"15min":1.319},"min":{"1min":0.77500000000000002,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.8940000000000001,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":0.879},{"interface":"front","average":{"1min":27.206,"5min":6.6630000000000003,"15min":3.2650000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":515.09100000000001,"5min":515.09100000000001,"15min":515.09100000000001},"last":1.143}]}]}]}} 2020-08-07T15:10:16.652 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:10:16.653 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:10:16.672 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:16.672 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:10:16.673 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:10:16.673 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:10:16.673 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.673 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:30968 4.4 e98/81 ondisk, result = 0)", 2020-08-07T15:10:16.674 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:10:16.674 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 90.938844720999995, 2020-08-07T15:10:16.674 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96211418999999998, 2020-08-07T15:10:16.674 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:16.675 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:16.675 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:16.675 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.675 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:16.675 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:10:16.676 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:16.676 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.676 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.676 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:16.676 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:10:16.682 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2387e-05 2020-08-07T15:10:16.682 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.684 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.684 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:16.684 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729821+0000", 2020-08-07T15:10:16.684 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9909999999999999e-06 2020-08-07T15:10:16.685 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.685 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.685 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:16.685 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729827+0000", 2020-08-07T15:10:16.685 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96185524600000005 2020-08-07T15:10:16.686 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.686 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.686 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:16.686 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691682+0000", 2020-08-07T15:10:16.686 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8760000000000005e-06 2020-08-07T15:10:16.687 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.687 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.687 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:16.687 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691690+0000", 2020-08-07T15:10:16.687 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9682000000000001e-05 2020-08-07T15:10:16.688 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.688 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.688 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:16.688 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691759+0000", 2020-08-07T15:10:16.688 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5381000000000001e-05 2020-08-07T15:10:16.689 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.689 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.689 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:16.689 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691775+0000", 2020-08-07T15:10:16.689 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000127627 2020-08-07T15:10:16.690 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.690 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.690 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:16.690 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691902+0000", 2020-08-07T15:10:16.690 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96211418999999998 2020-08-07T15:10:16.690 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:16.691 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:16.691 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:16.703 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.704 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42472 4.4 4:2ff82af8:::benchmark_data_smithi074_44586_object5308:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:10:16.720 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:10:16.720 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 76.401381947000004, 2020-08-07T15:10:16.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89653816399999997, 2020-08-07T15:10:16.726 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:16.727 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:16.727 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:16.727 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:16.727 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:16.727 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42472 2020-08-07T15:10:16.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.728 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:16.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:16.728 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:10:16.729 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:16.729 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.729 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.729 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:16.730 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:10:16.730 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9591e-05 2020-08-07T15:10:16.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.731 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:16.731 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267301+0000", 2020-08-07T15:10:16.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7490000000000004e-06 2020-08-07T15:10:16.732 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.732 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:16.732 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267306+0000", 2020-08-07T15:10:16.732 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2850000000000001e-05 2020-08-07T15:10:16.732 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.733 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:16.733 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267319+0000", 2020-08-07T15:10:16.733 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9100000000000002e-06 2020-08-07T15:10:16.734 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.734 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.734 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:16.734 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267325+0000", 2020-08-07T15:10:16.734 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6584000000000001e-05 2020-08-07T15:10:16.735 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.745 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:16.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267362+0000", 2020-08-07T15:10:16.746 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9188000000000001e-05 2020-08-07T15:10:16.746 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.747 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:16.747 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267451+0000", 2020-08-07T15:10:16.747 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000182917 2020-08-07T15:10:16.747 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.747 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.748 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:16.748 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267634+0000", 2020-08-07T15:10:16.748 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002484708 2020-08-07T15:10:16.748 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.748 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.781 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:16.782 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270119+0000", 2020-08-07T15:10:16.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0299999999999997e-07 2020-08-07T15:10:16.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:16.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270119+0000", 2020-08-07T15:10:16.813 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8251999999999996e-05 2020-08-07T15:10:16.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.822 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:16.822 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270188+0000", 2020-08-07T15:10:16.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24443688 2020-08-07T15:10:16.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:16.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.514625+0000", 2020-08-07T15:10:16.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.64911260599999998 2020-08-07T15:10:16.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:16.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163737+0000", 2020-08-07T15:10:16.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2100000000000001e-06 2020-08-07T15:10:16.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:16.826 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163738+0000", 2020-08-07T15:10:16.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0816000000000003e-05 2020-08-07T15:10:16.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:16.828 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163789+0000", 2020-08-07T15:10:16.828 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89653816399999997 2020-08-07T15:10:16.829 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:16.829 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:16.830 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:16.831 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.831 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.831 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:42472 4.4 e112/81 ondisk, result = 0)", 2020-08-07T15:10:16.832 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.271353+0000", 2020-08-07T15:10:16.832 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 76.397279909999995, 2020-08-07T15:10:16.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89244306299999998, 2020-08-07T15:10:16.832 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:16.833 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:16.833 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:16.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:16.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.271353+0000", 2020-08-07T15:10:16.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:16.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:16.836 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.271353+0000", 2020-08-07T15:10:16.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7875999999999999e-05 2020-08-07T15:10:16.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:16.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.271381+0000", 2020-08-07T15:10:16.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4819999999999996e-06 2020-08-07T15:10:16.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.838 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:16.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.271385+0000", 2020-08-07T15:10:16.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89228763300000002 2020-08-07T15:10:16.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.840 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:16.850 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163673+0000", 2020-08-07T15:10:16.851 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9949999999999996e-06 2020-08-07T15:10:16.851 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:16.852 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163678+0000", 2020-08-07T15:10:16.853 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5485000000000003e-05 2020-08-07T15:10:16.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.878 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:16.878 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163714+0000", 2020-08-07T15:10:16.878 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4785e-05 2020-08-07T15:10:16.878 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.879 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:16.879 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163728+0000", 2020-08-07T15:10:16.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7806999999999996e-05 2020-08-07T15:10:16.879 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.880 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:16.880 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163796+0000", 2020-08-07T15:10:16.880 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89244306299999998 2020-08-07T15:10:16.880 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:16.881 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:16.881 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:16.881 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.881 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.881 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72688 4.4 4:295d3f6e:::benchmark_data_smithi074_44586_object9085:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e129)", 2020-08-07T15:10:16.882 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:10:16.882 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 51.27179976, 2020-08-07T15:10:16.883 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7869782759999999, 2020-08-07T15:10:16.883 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:16.883 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:10:16.883 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:16.884 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:16.884 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:16.884 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72688 2020-08-07T15:10:16.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.885 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:16.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:16.885 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:10:16.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:16.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.902 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.903 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:16.903 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:10:16.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8721999999999999e-05 2020-08-07T15:10:16.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:16.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396882+0000", 2020-08-07T15:10:16.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2639999999999999e-06 2020-08-07T15:10:16.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:16.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396887+0000", 2020-08-07T15:10:16.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0767999999999999e-05 2020-08-07T15:10:16.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:16.936 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396898+0000", 2020-08-07T15:10:16.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1860000000000002e-06 2020-08-07T15:10:16.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:16.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396902+0000", 2020-08-07T15:10:16.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0037638649999999999 2020-08-07T15:10:16.937 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.952 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.953 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:16.953 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400666+0000", 2020-08-07T15:10:16.953 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000118077 2020-08-07T15:10:16.953 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.953 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.954 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:16.954 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400784+0000", 2020-08-07T15:10:16.954 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000136728 2020-08-07T15:10:16.954 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.954 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.955 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:10:16.955 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400921+0000", 2020-08-07T15:10:16.955 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001595443 2020-08-07T15:10:16.955 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.955 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.955 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:16.956 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.402516+0000", 2020-08-07T15:10:16.956 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.014817907 2020-08-07T15:10:16.956 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.956 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.956 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:16.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417334+0000", 2020-08-07T15:10:16.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.75e-06 2020-08-07T15:10:16.957 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:16.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:16.957 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:17.001 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417336+0000", 2020-08-07T15:10:17.001 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8066999999999996e-05 2020-08-07T15:10:17.002 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.002 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.002 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:17.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417424+0000", 2020-08-07T15:10:17.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002210573 2020-08-07T15:10:17.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:17.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.419635+0000", 2020-08-07T15:10:17.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1379000000000001e-05 2020-08-07T15:10:17.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.054 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.054 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:17.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.419646+0000", 2020-08-07T15:10:17.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.004643409 2020-08-07T15:10:17.055 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.055 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:17.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424289+0000", 2020-08-07T15:10:17.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7563000000000001e-05 2020-08-07T15:10:17.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:17.057 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424307+0000", 2020-08-07T15:10:17.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75943629400000001 2020-08-07T15:10:17.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:17.094 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183743+0000", 2020-08-07T15:10:17.106 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8281000000000006e-05 2020-08-07T15:10:17.107 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.107 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.107 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:17.126 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183812+0000", 2020-08-07T15:10:17.127 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7869782759999999 2020-08-07T15:10:17.127 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:17.150 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:17.150 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:17.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.151 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72985 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:17.151 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:10:17.152 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 51.055813940999997, 2020-08-07T15:10:17.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869466839999999, 2020-08-07T15:10:17.152 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:17.152 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:17.152 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:17.152 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:17.153 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:17.153 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72985 2020-08-07T15:10:17.153 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.153 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:17.155 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.155 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:17.156 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:10:17.156 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:17.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.156 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:17.157 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:10:17.157 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8003999999999997e-05 2020-08-07T15:10:17.157 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.157 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.157 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:17.158 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612857+0000", 2020-08-07T15:10:17.158 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.246e-06 2020-08-07T15:10:17.158 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.158 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.158 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:17.158 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612863+0000", 2020-08-07T15:10:17.159 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5146e-05 2020-08-07T15:10:17.159 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.159 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.159 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:17.159 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612878+0000", 2020-08-07T15:10:17.160 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1940000000000003e-06 2020-08-07T15:10:17.160 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.190 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.191 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:17.191 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612885+0000", 2020-08-07T15:10:17.191 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0349999999999997e-05 2020-08-07T15:10:17.192 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.195 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.196 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:17.196 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612935+0000", 2020-08-07T15:10:17.197 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1029999999999996e-06 2020-08-07T15:10:17.197 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.197 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.197 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:17.197 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612942+0000", 2020-08-07T15:10:17.198 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81140097700000002 2020-08-07T15:10:17.198 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.198 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.198 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:17.198 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424343+0000", 2020-08-07T15:10:17.199 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1799999999999996e-06 2020-08-07T15:10:17.199 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.199 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.199 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:17.199 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424351+0000", 2020-08-07T15:10:17.200 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75948665299999996 2020-08-07T15:10:17.200 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.200 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.200 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:17.200 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183838+0000", 2020-08-07T15:10:17.201 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000179187 2020-08-07T15:10:17.201 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.201 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.201 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:17.201 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.184017+0000", 2020-08-07T15:10:17.202 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024691600000000002 2020-08-07T15:10:17.202 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.203 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.203 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:17.203 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.184264+0000", 2020-08-07T15:10:17.203 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006362721 2020-08-07T15:10:17.203 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.203 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.204 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:17.204 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190627+0000", 2020-08-07T15:10:17.204 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7400000000000001e-06 2020-08-07T15:10:17.204 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.204 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.205 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:17.205 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190629+0000", 2020-08-07T15:10:17.205 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1305999999999999e-05 2020-08-07T15:10:17.214 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.215 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.215 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:17.215 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190690+0000", 2020-08-07T15:10:17.215 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0088507159999999998 2020-08-07T15:10:17.227 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.227 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.227 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:17.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199541+0000", 2020-08-07T15:10:17.228 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015864699999999999 2020-08-07T15:10:17.228 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.228 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.228 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:17.229 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199699+0000", 2020-08-07T15:10:17.229 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.23e-07 2020-08-07T15:10:17.229 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.229 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.229 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:17.242 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199700+0000", 2020-08-07T15:10:17.243 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5775e-05 2020-08-07T15:10:17.243 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.243 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.243 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:17.243 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199766+0000", 2020-08-07T15:10:17.244 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869466839999999 2020-08-07T15:10:17.244 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:17.244 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:17.244 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:17.244 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.245 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.245 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72986 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:17.245 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:10:17.245 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 51.055703358999999, 2020-08-07T15:10:17.245 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.586946441, 2020-08-07T15:10:17.277 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:17.277 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:17.277 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:17.277 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:17.278 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:17.278 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72986 2020-08-07T15:10:17.278 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.278 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:17.278 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.279 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:17.279 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:10:17.318 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:17.318 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.319 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.319 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:17.323 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:10:17.324 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9861999999999997e-05 2020-08-07T15:10:17.346 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.346 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.346 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:17.347 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612990+0000", 2020-08-07T15:10:17.347 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4350000000000001e-06 2020-08-07T15:10:17.347 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.348 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:17.348 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612993+0000", 2020-08-07T15:10:17.348 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9529999999999999e-06 2020-08-07T15:10:17.348 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.348 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.349 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:17.349 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613001+0000", 2020-08-07T15:10:17.349 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4309999999999998e-06 2020-08-07T15:10:17.349 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.374 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.374 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:17.374 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613004+0000", 2020-08-07T15:10:17.375 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5284000000000001e-05 2020-08-07T15:10:17.375 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.375 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:17.376 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613030+0000", 2020-08-07T15:10:17.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2400000000000001e-06 2020-08-07T15:10:17.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.376 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:17.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613034+0000", 2020-08-07T15:10:17.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81134136199999995 2020-08-07T15:10:17.377 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.408 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:17.409 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424375+0000", 2020-08-07T15:10:17.409 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3520000000000001e-06 2020-08-07T15:10:17.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.410 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.410 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:17.410 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424383+0000", 2020-08-07T15:10:17.411 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76064437799999995 2020-08-07T15:10:17.411 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.412 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.412 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:17.412 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185027+0000", 2020-08-07T15:10:17.413 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000119719 2020-08-07T15:10:17.413 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.436 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.437 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:17.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185147+0000", 2020-08-07T15:10:17.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016829400000000001 2020-08-07T15:10:17.445 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.445 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.446 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:17.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185315+0000", 2020-08-07T15:10:17.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0053821199999999998 2020-08-07T15:10:17.451 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:17.453 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190697+0000", 2020-08-07T15:10:17.454 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6899999999999996e-07 2020-08-07T15:10:17.454 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.454 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:17.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190698+0000", 2020-08-07T15:10:17.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2072000000000001e-05 2020-08-07T15:10:17.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:17.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190730+0000", 2020-08-07T15:10:17.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0088501440000000008 2020-08-07T15:10:17.458 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.458 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:17.459 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199580+0000", 2020-08-07T15:10:17.459 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00023199900000000001 2020-08-07T15:10:17.469 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.469 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.470 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:17.470 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199812+0000", 2020-08-07T15:10:17.470 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6300000000000005e-07 2020-08-07T15:10:17.471 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.471 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.472 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:17.472 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199813+0000", 2020-08-07T15:10:17.472 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3363999999999998e-05 2020-08-07T15:10:17.472 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.473 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.473 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:17.482 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199876+0000", 2020-08-07T15:10:17.482 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.586946441 2020-08-07T15:10:17.483 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:17.483 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:17.483 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:17.484 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.484 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.485 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72987 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:17.485 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:10:17.485 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 51.055607604000002, 2020-08-07T15:10:17.486 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869598709999999, 2020-08-07T15:10:17.487 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:17.487 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:17.487 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:17.488 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:17.488 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:17.508 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72987 2020-08-07T15:10:17.538 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.538 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:17.546 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.547 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:17.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:10:17.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:17.560 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.561 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.561 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:17.561 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:10:17.562 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1851999999999999e-05 2020-08-07T15:10:17.585 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.585 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.585 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:17.586 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613057+0000", 2020-08-07T15:10:17.586 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0429999999999999e-06 2020-08-07T15:10:17.586 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.587 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.587 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:17.587 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613060+0000", 2020-08-07T15:10:17.588 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1060000000000002e-06 2020-08-07T15:10:17.588 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.589 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:17.589 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613067+0000", 2020-08-07T15:10:17.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8040000000000002e-06 2020-08-07T15:10:17.629 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.629 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.630 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:17.630 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613070+0000", 2020-08-07T15:10:17.630 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6857000000000001e-05 2020-08-07T15:10:17.630 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.631 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.631 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:17.631 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613097+0000", 2020-08-07T15:10:17.631 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8770000000000001e-06 2020-08-07T15:10:17.631 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.632 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.632 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:17.632 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613102+0000", 2020-08-07T15:10:17.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81131066399999996 2020-08-07T15:10:17.632 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.633 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.633 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:17.633 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424413+0000", 2020-08-07T15:10:17.633 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4599999999999997e-06 2020-08-07T15:10:17.633 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.647 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.648 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:17.649 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424420+0000", 2020-08-07T15:10:17.649 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76132809700000004 2020-08-07T15:10:17.649 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.649 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.650 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:17.650 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185748+0000", 2020-08-07T15:10:17.650 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.4264000000000005e-05 2020-08-07T15:10:17.650 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.650 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.651 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:17.651 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185832+0000", 2020-08-07T15:10:17.651 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112803 2020-08-07T15:10:17.651 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.651 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.651 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:17.652 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185945+0000", 2020-08-07T15:10:17.652 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0089954130000000007 2020-08-07T15:10:17.652 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.652 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.653 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:17.717 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194941+0000", 2020-08-07T15:10:17.726 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.091e-06 2020-08-07T15:10:17.727 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.727 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.727 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:17.727 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194943+0000", 2020-08-07T15:10:17.727 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7423999999999996e-05 2020-08-07T15:10:17.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:17.728 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195020+0000", 2020-08-07T15:10:17.728 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045973300000000002 2020-08-07T15:10:17.729 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.729 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.729 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:17.729 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199618+0000", 2020-08-07T15:10:17.729 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000302009 2020-08-07T15:10:17.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.779 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:17.780 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199920+0000", 2020-08-07T15:10:17.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7999999999999996e-07 2020-08-07T15:10:17.780 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.780 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:17.781 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199920+0000", 2020-08-07T15:10:17.781 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5296999999999995e-05 2020-08-07T15:10:17.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.786 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:17.786 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199985+0000", 2020-08-07T15:10:17.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869598709999999 2020-08-07T15:10:17.787 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:17.787 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:17.788 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:17.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.789 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72988 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:17.789 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:10:17.789 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 51.055543024000002, 2020-08-07T15:10:17.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587111143, 2020-08-07T15:10:17.790 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:17.790 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:17.791 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:17.793 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:17.793 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:17.793 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72988 2020-08-07T15:10:17.793 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.794 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:17.794 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.794 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:17.795 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:10:17.795 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:17.795 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.802 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.802 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:17.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:10:17.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6563e-05 2020-08-07T15:10:17.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:17.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613117+0000", 2020-08-07T15:10:17.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7180000000000001e-06 2020-08-07T15:10:17.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:17.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613119+0000", 2020-08-07T15:10:17.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1789999999999999e-06 2020-08-07T15:10:17.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:17.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613124+0000", 2020-08-07T15:10:17.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3539999999999998e-06 2020-08-07T15:10:17.806 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:17.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613127+0000", 2020-08-07T15:10:17.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3697e-05 2020-08-07T15:10:17.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:17.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613151+0000", 2020-08-07T15:10:17.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7559999999999999e-06 2020-08-07T15:10:17.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.808 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:17.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613155+0000", 2020-08-07T15:10:17.809 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81128859600000003 2020-08-07T15:10:17.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.809 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:17.809 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424444+0000", 2020-08-07T15:10:17.810 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9619999999999996e-06 2020-08-07T15:10:17.810 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.810 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.810 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:17.810 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424451+0000", 2020-08-07T15:10:17.810 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76286712899999998 2020-08-07T15:10:17.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.811 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:17.811 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187318+0000", 2020-08-07T15:10:17.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011616300000000001 2020-08-07T15:10:17.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:17.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187434+0000", 2020-08-07T15:10:17.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019347400000000001 2020-08-07T15:10:17.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.813 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:17.814 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187628+0000", 2020-08-07T15:10:17.814 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074049090000000003 2020-08-07T15:10:17.814 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.814 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.815 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:17.815 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195033+0000", 2020-08-07T15:10:17.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1090000000000001e-06 2020-08-07T15:10:17.816 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:17.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195034+0000", 2020-08-07T15:10:17.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8566999999999997e-05 2020-08-07T15:10:17.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.818 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.818 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:17.818 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195092+0000", 2020-08-07T15:10:17.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004570457 2020-08-07T15:10:17.819 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:17.819 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199663+0000", 2020-08-07T15:10:17.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00046757899999999998 2020-08-07T15:10:17.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:17.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200130+0000", 2020-08-07T15:10:17.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6700000000000002e-07 2020-08-07T15:10:17.821 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:17.822 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200131+0000", 2020-08-07T15:10:17.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9963999999999996e-05 2020-08-07T15:10:17.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:17.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200201+0000", 2020-08-07T15:10:17.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587111143 2020-08-07T15:10:17.824 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:17.824 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:17.825 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:17.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.826 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72989 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:17.846 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:10:17.846 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 51.055486635999998, 2020-08-07T15:10:17.847 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5872506399999999, 2020-08-07T15:10:17.847 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:17.847 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:17.848 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:17.848 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:17.849 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:17.849 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72989 2020-08-07T15:10:17.849 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.850 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:17.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.850 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:17.851 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:10:17.851 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:17.851 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.851 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:17.852 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:10:17.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6081999999999999e-05 2020-08-07T15:10:17.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:17.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613172+0000", 2020-08-07T15:10:17.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6570000000000001e-06 2020-08-07T15:10:17.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:17.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613175+0000", 2020-08-07T15:10:17.890 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7740000000000001e-06 2020-08-07T15:10:17.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.891 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:17.891 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613181+0000", 2020-08-07T15:10:17.891 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2759999999999999e-06 2020-08-07T15:10:17.892 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.892 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.892 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:17.892 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613183+0000", 2020-08-07T15:10:17.914 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3892000000000001e-05 2020-08-07T15:10:17.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:17.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613207+0000", 2020-08-07T15:10:17.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9870000000000001e-06 2020-08-07T15:10:17.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:17.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613211+0000", 2020-08-07T15:10:17.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81126230700000002 2020-08-07T15:10:17.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:17.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424473+0000", 2020-08-07T15:10:17.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1460000000000001e-06 2020-08-07T15:10:17.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:17.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424480+0000", 2020-08-07T15:10:17.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76413073099999995 2020-08-07T15:10:17.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:17.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188610+0000", 2020-08-07T15:10:17.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011278899999999999 2020-08-07T15:10:17.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:17.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188723+0000", 2020-08-07T15:10:17.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000213422 2020-08-07T15:10:17.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:17.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188936+0000", 2020-08-07T15:10:17.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0061672929999999999 2020-08-07T15:10:17.955 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.955 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.956 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:17.956 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195104+0000", 2020-08-07T15:10:17.956 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8199999999999999e-07 2020-08-07T15:10:17.957 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.957 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:17.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195105+0000", 2020-08-07T15:10:17.958 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2056000000000004e-05 2020-08-07T15:10:17.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:17.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195167+0000", 2020-08-07T15:10:17.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0051009769999999996 2020-08-07T15:10:17.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:17.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200268+0000", 2020-08-07T15:10:17.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9428000000000001e-05 2020-08-07T15:10:17.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:17.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200327+0000", 2020-08-07T15:10:17.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9100000000000003e-07 2020-08-07T15:10:17.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.979 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:17.979 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200328+0000", 2020-08-07T15:10:17.980 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9250000000000003e-05 2020-08-07T15:10:17.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:17.980 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200397+0000", 2020-08-07T15:10:17.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5872506399999999 2020-08-07T15:10:17.981 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:17.981 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:17.981 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:17.982 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.982 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72990 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:17.982 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:10:17.982 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 51.055429343999997, 2020-08-07T15:10:17.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587335105, 2020-08-07T15:10:17.983 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:17.983 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:17.983 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:17.983 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:17.984 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:17.993 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72990 2020-08-07T15:10:17.993 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.993 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:17.994 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:17.994 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:17.994 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:10:17.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:17.994 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:17.995 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:18.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:10:18.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5276999999999999e-05 2020-08-07T15:10:18.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:18.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613229+0000", 2020-08-07T15:10:18.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7219999999999999e-06 2020-08-07T15:10:18.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:18.033 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613232+0000", 2020-08-07T15:10:18.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1170000000000003e-06 2020-08-07T15:10:18.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.062 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:18.082 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613237+0000", 2020-08-07T15:10:18.082 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2309999999999999e-06 2020-08-07T15:10:18.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.083 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.083 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:18.083 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613239+0000", 2020-08-07T15:10:18.084 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2469000000000001e-05 2020-08-07T15:10:18.084 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.084 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:18.084 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613261+0000", 2020-08-07T15:10:18.085 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9280000000000001e-06 2020-08-07T15:10:18.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.116 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:18.116 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613266+0000", 2020-08-07T15:10:18.117 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81123414599999999 2020-08-07T15:10:18.117 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.118 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.118 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:18.118 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424501+0000", 2020-08-07T15:10:18.119 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5300000000000002e-06 2020-08-07T15:10:18.119 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.119 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.120 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:18.120 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424507+0000", 2020-08-07T15:10:18.121 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76488660200000003 2020-08-07T15:10:18.121 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.150 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.150 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:18.151 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189394+0000", 2020-08-07T15:10:18.151 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6878000000000003e-05 2020-08-07T15:10:18.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:18.152 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189491+0000", 2020-08-07T15:10:18.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000204477 2020-08-07T15:10:18.153 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.153 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.153 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:18.161 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189695+0000", 2020-08-07T15:10:18.161 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0065180940000000003 2020-08-07T15:10:18.162 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.162 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.162 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:18.162 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196213+0000", 2020-08-07T15:10:18.163 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4920000000000001e-06 2020-08-07T15:10:18.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.163 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:18.164 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196215+0000", 2020-08-07T15:10:18.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9950000000000006e-05 2020-08-07T15:10:18.164 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.165 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.165 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:18.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196285+0000", 2020-08-07T15:10:18.166 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004163674 2020-08-07T15:10:18.166 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.167 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.167 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:18.167 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200448+0000", 2020-08-07T15:10:18.168 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6919999999999999e-05 2020-08-07T15:10:18.168 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.168 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.169 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:18.169 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200485+0000", 2020-08-07T15:10:18.169 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1699999999999998e-07 2020-08-07T15:10:18.170 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.170 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.170 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:18.171 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200486+0000", 2020-08-07T15:10:18.182 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3081000000000002e-05 2020-08-07T15:10:18.182 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.206 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.206 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:18.207 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200539+0000", 2020-08-07T15:10:18.207 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587335105 2020-08-07T15:10:18.207 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:18.207 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:18.208 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:18.208 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.208 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.209 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72991 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:18.209 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:10:18.209 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 51.055373819000003, 2020-08-07T15:10:18.238 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5876210340000001, 2020-08-07T15:10:18.238 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:18.239 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:18.239 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:18.258 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:18.258 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:18.259 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72991 2020-08-07T15:10:18.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.259 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:18.259 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.260 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:18.260 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:10:18.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:18.260 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.261 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.261 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:18.261 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:10:18.261 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5763999999999999e-05 2020-08-07T15:10:18.275 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.276 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.276 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:18.276 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613285+0000", 2020-08-07T15:10:18.277 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6479999999999999e-06 2020-08-07T15:10:18.277 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.277 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.278 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:18.278 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613288+0000", 2020-08-07T15:10:18.278 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1909999999999999e-06 2020-08-07T15:10:18.279 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.279 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.279 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:18.280 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613293+0000", 2020-08-07T15:10:18.280 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2560000000000001e-06 2020-08-07T15:10:18.280 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.281 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.281 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:18.281 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613295+0000", 2020-08-07T15:10:18.281 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2561e-05 2020-08-07T15:10:18.284 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.284 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.285 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:18.285 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613318+0000", 2020-08-07T15:10:18.285 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6349999999999999e-06 2020-08-07T15:10:18.285 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.286 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.286 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:18.286 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613321+0000", 2020-08-07T15:10:18.287 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81120592400000002 2020-08-07T15:10:18.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.287 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:18.288 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424527+0000", 2020-08-07T15:10:18.288 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8540000000000002e-06 2020-08-07T15:10:18.288 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.289 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.289 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:18.289 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424533+0000", 2020-08-07T15:10:18.290 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76623392800000001 2020-08-07T15:10:18.306 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.306 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.307 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:18.307 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190767+0000", 2020-08-07T15:10:18.307 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3990000000000001e-05 2020-08-07T15:10:18.308 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.308 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:18.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190841+0000", 2020-08-07T15:10:18.308 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011444599999999999 2020-08-07T15:10:18.309 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.309 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.309 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:18.309 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190955+0000", 2020-08-07T15:10:18.318 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0053397499999999999 2020-08-07T15:10:18.328 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.348 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.349 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:18.349 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196295+0000", 2020-08-07T15:10:18.349 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006e-06 2020-08-07T15:10:18.349 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.350 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.350 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:18.350 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196296+0000", 2020-08-07T15:10:18.350 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8940999999999997e-05 2020-08-07T15:10:18.350 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.351 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.351 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:18.351 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196355+0000", 2020-08-07T15:10:18.351 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042276700000000002 2020-08-07T15:10:18.363 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:18.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200583+0000", 2020-08-07T15:10:18.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00026035700000000001 2020-08-07T15:10:18.379 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:18.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200843+0000", 2020-08-07T15:10:18.380 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6700000000000003e-07 2020-08-07T15:10:18.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:18.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200844+0000", 2020-08-07T15:10:18.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6545999999999999e-05 2020-08-07T15:10:18.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.411 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.412 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:18.412 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200880+0000", 2020-08-07T15:10:18.412 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5876210340000001 2020-08-07T15:10:18.413 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:18.413 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:18.413 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:18.437 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.437 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.438 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72992 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:18.438 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:10:18.438 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 51.054286339999997, 2020-08-07T15:10:18.438 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5866241210000001, 2020-08-07T15:10:18.439 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:18.439 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:18.439 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:18.439 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:18.450 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:18.451 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72992 2020-08-07T15:10:18.451 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.451 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:18.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:18.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:10:18.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:18.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.461 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:18.462 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:10:18.462 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7543e-05 2020-08-07T15:10:18.463 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.463 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.463 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:18.464 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614374+0000", 2020-08-07T15:10:18.464 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9740000000000004e-06 2020-08-07T15:10:18.464 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.465 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.465 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:18.465 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614378+0000", 2020-08-07T15:10:18.486 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2660000000000007e-06 2020-08-07T15:10:18.486 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.486 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.487 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:18.487 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614387+0000", 2020-08-07T15:10:18.487 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2950000000000001e-06 2020-08-07T15:10:18.488 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:18.488 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614393+0000", 2020-08-07T15:10:18.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7279000000000001e-05 2020-08-07T15:10:18.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.489 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.489 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:18.502 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614420+0000", 2020-08-07T15:10:18.518 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9040000000000002e-06 2020-08-07T15:10:18.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.519 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.519 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:18.519 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614424+0000", 2020-08-07T15:10:18.520 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81012815699999996 2020-08-07T15:10:18.520 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.520 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.521 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:18.521 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424552+0000", 2020-08-07T15:10:18.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2879999999999998e-06 2020-08-07T15:10:18.534 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:18.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424558+0000", 2020-08-07T15:10:18.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76678665400000001 2020-08-07T15:10:18.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.554 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.555 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:18.555 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191345+0000", 2020-08-07T15:10:18.556 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010819600000000001 2020-08-07T15:10:18.556 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.557 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.557 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:18.573 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191453+0000", 2020-08-07T15:10:18.573 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000170829 2020-08-07T15:10:18.574 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.574 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.574 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:18.575 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191624+0000", 2020-08-07T15:10:18.575 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0047386679999999997 2020-08-07T15:10:18.575 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.576 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.576 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:18.576 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196363+0000", 2020-08-07T15:10:18.577 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6000000000000003e-07 2020-08-07T15:10:18.577 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.602 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.602 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:18.603 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196363+0000", 2020-08-07T15:10:18.603 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3421000000000002e-05 2020-08-07T15:10:18.603 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:18.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196417+0000", 2020-08-07T15:10:18.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042225539999999999 2020-08-07T15:10:18.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.605 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.605 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:18.605 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200639+0000", 2020-08-07T15:10:18.624 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00028031399999999999 2020-08-07T15:10:18.625 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.625 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.642 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:18.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200920+0000", 2020-08-07T15:10:18.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0299999999999999e-07 2020-08-07T15:10:18.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.670 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.670 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:18.670 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200920+0000", 2020-08-07T15:10:18.670 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0516000000000002e-05 2020-08-07T15:10:18.690 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.690 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.691 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:18.691 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200971+0000", 2020-08-07T15:10:18.691 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5866241210000001 2020-08-07T15:10:18.692 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:18.692 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:18.692 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:18.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.693 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73017 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:18.693 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:10:18.693 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 50.998778086999998, 2020-08-07T15:10:18.693 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.531181608, 2020-08-07T15:10:18.716 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:18.716 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:18.717 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:18.717 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:18.718 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:18.745 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73017 2020-08-07T15:10:18.745 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.745 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:18.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.746 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:18.746 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:10:18.746 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:18.746 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.747 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.747 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:18.747 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:10:18.747 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3018e-05 2020-08-07T15:10:18.748 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.762 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.762 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:18.763 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669898+0000", 2020-08-07T15:10:18.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5340000000000002e-06 2020-08-07T15:10:18.763 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.764 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:18.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669905+0000", 2020-08-07T15:10:18.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3307e-05 2020-08-07T15:10:18.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:18.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669919+0000", 2020-08-07T15:10:18.785 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7729999999999999e-06 2020-08-07T15:10:18.789 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.792 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.792 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:18.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669927+0000", 2020-08-07T15:10:18.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0361000000000001e-05 2020-08-07T15:10:18.798 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.798 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.798 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:18.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669977+0000", 2020-08-07T15:10:18.799 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0049999999999993e-06 2020-08-07T15:10:18.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:18.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669986+0000", 2020-08-07T15:10:18.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75459235199999997 2020-08-07T15:10:18.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:18.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424578+0000", 2020-08-07T15:10:18.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.925e-06 2020-08-07T15:10:18.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.808 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.809 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:18.809 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424584+0000", 2020-08-07T15:10:18.809 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76741703900000002 2020-08-07T15:10:18.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:18.846 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192001+0000", 2020-08-07T15:10:18.846 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000181196 2020-08-07T15:10:18.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.846 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.847 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:18.847 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192182+0000", 2020-08-07T15:10:18.847 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00021149299999999999 2020-08-07T15:10:18.847 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.847 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.847 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:18.848 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192394+0000", 2020-08-07T15:10:18.848 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004031242 2020-08-07T15:10:18.848 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.848 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.848 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:18.849 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196425+0000", 2020-08-07T15:10:18.849 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7800000000000001e-07 2020-08-07T15:10:18.849 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.849 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.849 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:18.894 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196426+0000", 2020-08-07T15:10:18.895 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5244999999999997e-05 2020-08-07T15:10:18.895 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.896 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:18.896 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196481+0000", 2020-08-07T15:10:18.897 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041989360000000003 2020-08-07T15:10:18.897 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.897 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.922 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:18.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200680+0000", 2020-08-07T15:10:18.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00032415299999999998 2020-08-07T15:10:18.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:18.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201004+0000", 2020-08-07T15:10:18.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9299999999999999e-07 2020-08-07T15:10:18.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:18.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201005+0000", 2020-08-07T15:10:18.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1857999999999999e-05 2020-08-07T15:10:18.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:18.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201037+0000", 2020-08-07T15:10:18.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.531181608 2020-08-07T15:10:18.930 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:18.930 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:18.947 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:18.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.957 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73018 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:18.958 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:10:18.962 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 50.997810282000003, 2020-08-07T15:10:18.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.530301616, 2020-08-07T15:10:18.962 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:18.963 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:18.963 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:18.963 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:18.963 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:18.963 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73018 2020-08-07T15:10:18.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.964 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:18.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:18.964 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:10:18.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:18.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:18.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:10:18.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9747000000000002e-05 2020-08-07T15:10:18.987 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:18.987 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:18.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670852+0000", 2020-08-07T15:10:18.998 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9430000000000002e-06 2020-08-07T15:10:18.998 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:18.999 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.000 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:19.016 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670856+0000", 2020-08-07T15:10:19.017 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0649999999999999e-06 2020-08-07T15:10:19.017 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:19.043 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670864+0000", 2020-08-07T15:10:19.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.924e-06 2020-08-07T15:10:19.049 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.050 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.050 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:19.050 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670868+0000", 2020-08-07T15:10:19.051 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6205000000000003e-05 2020-08-07T15:10:19.051 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.052 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:19.075 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670905+0000", 2020-08-07T15:10:19.075 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3289999999999996e-06 2020-08-07T15:10:19.076 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.076 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.076 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:19.076 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670912+0000", 2020-08-07T15:10:19.077 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75370200200000004 2020-08-07T15:10:19.077 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.077 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.077 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:19.077 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424614+0000", 2020-08-07T15:10:19.106 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5150000000000003e-06 2020-08-07T15:10:19.106 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.106 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.107 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:19.107 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424620+0000", 2020-08-07T15:10:19.107 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.768200153 2020-08-07T15:10:19.107 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.107 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.108 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:19.108 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192821+0000", 2020-08-07T15:10:19.108 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000111764 2020-08-07T15:10:19.108 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.125 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.125 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:19.126 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192932+0000", 2020-08-07T15:10:19.126 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017542400000000001 2020-08-07T15:10:19.126 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.126 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.127 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:19.127 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193108+0000", 2020-08-07T15:10:19.127 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004100851 2020-08-07T15:10:19.127 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.127 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.127 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:19.128 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197209+0000", 2020-08-07T15:10:19.128 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0100000000000001e-06 2020-08-07T15:10:19.128 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.128 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.128 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:19.129 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197210+0000", 2020-08-07T15:10:19.129 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9672999999999995e-05 2020-08-07T15:10:19.129 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.129 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.129 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:19.151 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197279+0000", 2020-08-07T15:10:19.151 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034275109999999998 2020-08-07T15:10:19.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:19.152 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200707+0000", 2020-08-07T15:10:19.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00036853100000000001 2020-08-07T15:10:19.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.153 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:19.153 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201075+0000", 2020-08-07T15:10:19.153 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8599999999999998e-07 2020-08-07T15:10:19.153 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.153 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.169 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:19.169 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201076+0000", 2020-08-07T15:10:19.170 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8483000000000003e-05 2020-08-07T15:10:19.171 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.172 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.172 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:19.172 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201124+0000", 2020-08-07T15:10:19.172 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.530301616 2020-08-07T15:10:19.174 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:19.177 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:19.177 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:19.177 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.177 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.177 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73019 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:19.178 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:10:19.178 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 50.997162631000002, 2020-08-07T15:10:19.178 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.529718224, 2020-08-07T15:10:19.178 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:19.178 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:19.179 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:19.187 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:19.187 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:19.187 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73019 2020-08-07T15:10:19.204 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.204 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:19.205 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.205 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:19.205 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:10:19.205 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:19.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.206 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.206 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:19.206 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:10:19.206 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3309000000000002e-05 2020-08-07T15:10:19.206 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.207 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.207 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:19.207 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671504+0000", 2020-08-07T15:10:19.207 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6619999999999998e-06 2020-08-07T15:10:19.207 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.208 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.208 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:19.208 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671507+0000", 2020-08-07T15:10:19.208 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2279999999999998e-06 2020-08-07T15:10:19.208 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.209 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.209 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:19.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671516+0000", 2020-08-07T15:10:19.209 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4860000000000002e-06 2020-08-07T15:10:19.209 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.236 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.236 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:19.237 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671519+0000", 2020-08-07T15:10:19.237 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0564000000000001e-05 2020-08-07T15:10:19.242 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.247 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.247 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:19.248 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671550+0000", 2020-08-07T15:10:19.248 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1029999999999998e-06 2020-08-07T15:10:19.248 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.248 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.248 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:19.249 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671556+0000", 2020-08-07T15:10:19.249 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75308759199999997 2020-08-07T15:10:19.249 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.249 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.249 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:19.255 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424643+0000", 2020-08-07T15:10:19.256 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9179999999999997e-06 2020-08-07T15:10:19.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.259 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.262 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:19.263 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424650+0000", 2020-08-07T15:10:19.263 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76890373599999995 2020-08-07T15:10:19.263 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.263 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.263 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:19.264 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193554+0000", 2020-08-07T15:10:19.264 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012345600000000001 2020-08-07T15:10:19.264 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.264 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.264 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:19.264 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193677+0000", 2020-08-07T15:10:19.265 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013528300000000001 2020-08-07T15:10:19.265 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.265 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.277 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:19.278 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193813+0000", 2020-08-07T15:10:19.278 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034777710000000002 2020-08-07T15:10:19.279 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.279 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.280 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:19.280 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197291+0000", 2020-08-07T15:10:19.280 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8000000000000004e-07 2020-08-07T15:10:19.280 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.305 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.305 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:19.307 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197291+0000", 2020-08-07T15:10:19.307 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9176999999999999e-05 2020-08-07T15:10:19.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.307 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:19.347 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197351+0000", 2020-08-07T15:10:19.348 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033931009999999999 2020-08-07T15:10:19.348 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.348 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.349 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:19.349 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200744+0000", 2020-08-07T15:10:19.349 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00041308000000000002 2020-08-07T15:10:19.374 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.375 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:19.376 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201157+0000", 2020-08-07T15:10:19.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8999999999999998e-07 2020-08-07T15:10:19.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:19.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201157+0000", 2020-08-07T15:10:19.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1588000000000003e-05 2020-08-07T15:10:19.379 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:19.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201189+0000", 2020-08-07T15:10:19.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.529718224 2020-08-07T15:10:19.380 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:19.380 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:19.380 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:19.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.381 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73020 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:19.381 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:10:19.381 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 50.928824771000002, 2020-08-07T15:10:19.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.461468373, 2020-08-07T15:10:19.381 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:19.386 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:19.394 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:19.394 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:19.394 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:19.394 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73020 2020-08-07T15:10:19.395 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.395 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:19.395 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.395 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:19.395 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:10:19.396 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:19.396 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.396 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.396 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:19.396 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:10:19.397 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.777e-05 2020-08-07T15:10:19.399 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.399 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:19.399 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739836+0000", 2020-08-07T15:10:19.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0180000000000003e-06 2020-08-07T15:10:19.400 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.400 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:19.400 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739840+0000", 2020-08-07T15:10:19.401 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6930000000000002e-06 2020-08-07T15:10:19.401 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.401 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.401 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:19.401 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739850+0000", 2020-08-07T15:10:19.402 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3619999999999999e-06 2020-08-07T15:10:19.402 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.402 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.402 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:19.402 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739854+0000", 2020-08-07T15:10:19.421 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7432999999999999e-05 2020-08-07T15:10:19.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.422 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:19.422 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739892+0000", 2020-08-07T15:10:19.422 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1149999999999996e-06 2020-08-07T15:10:19.422 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.423 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.423 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:19.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739899+0000", 2020-08-07T15:10:19.423 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68477883299999998 2020-08-07T15:10:19.423 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.424 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.424 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:19.424 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424677+0000", 2020-08-07T15:10:19.424 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7710000000000001e-06 2020-08-07T15:10:19.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.425 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:19.425 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424684+0000", 2020-08-07T15:10:19.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76949970700000003 2020-08-07T15:10:19.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.432 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.432 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:19.432 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194184+0000", 2020-08-07T15:10:19.432 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3957999999999997e-05 2020-08-07T15:10:19.432 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.433 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.433 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:19.433 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194278+0000", 2020-08-07T15:10:19.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017489599999999999 2020-08-07T15:10:19.433 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.434 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.434 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:19.434 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194453+0000", 2020-08-07T15:10:19.434 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002908908 2020-08-07T15:10:19.434 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.435 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.435 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:19.435 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197362+0000", 2020-08-07T15:10:19.435 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7299999999999995e-07 2020-08-07T15:10:19.435 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.436 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.436 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:19.436 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197362+0000", 2020-08-07T15:10:19.436 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6848000000000002e-05 2020-08-07T15:10:19.436 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.436 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.437 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:19.437 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197419+0000", 2020-08-07T15:10:19.437 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033597079999999999 2020-08-07T15:10:19.437 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.482 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.482 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:19.484 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200779+0000", 2020-08-07T15:10:19.486 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00044797899999999999 2020-08-07T15:10:19.486 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.486 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.487 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:19.487 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201227+0000", 2020-08-07T15:10:19.487 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7100000000000002e-07 2020-08-07T15:10:19.488 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:19.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201227+0000", 2020-08-07T15:10:19.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9230000000000001e-05 2020-08-07T15:10:19.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.513 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.513 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:19.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201277+0000", 2020-08-07T15:10:19.513 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.461468373 2020-08-07T15:10:19.514 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:19.514 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:19.514 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:19.514 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.515 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73021 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:19.515 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:10:19.516 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 50.865791164000001, 2020-08-07T15:10:19.517 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.398505079, 2020-08-07T15:10:19.517 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:19.553 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:19.553 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:19.553 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:19.554 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:19.554 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73021 2020-08-07T15:10:19.554 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.554 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:19.555 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.555 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:19.555 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:10:19.556 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:19.556 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.557 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:19.599 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:10:19.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7429000000000001e-05 2020-08-07T15:10:19.600 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.601 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:19.601 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802869+0000", 2020-08-07T15:10:19.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7840000000000001e-06 2020-08-07T15:10:19.602 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.623 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.630 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:19.630 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802873+0000", 2020-08-07T15:10:19.630 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0119999999999994e-06 2020-08-07T15:10:19.630 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.631 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.631 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:19.631 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802882+0000", 2020-08-07T15:10:19.631 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.04e-06 2020-08-07T15:10:19.631 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.632 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.632 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:19.632 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802887+0000", 2020-08-07T15:10:19.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7076000000000002e-05 2020-08-07T15:10:19.632 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.633 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.633 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:19.633 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802924+0000", 2020-08-07T15:10:19.633 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9280000000000004e-06 2020-08-07T15:10:19.635 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.635 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.642 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:19.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802931+0000", 2020-08-07T15:10:19.643 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.62178051000000001 2020-08-07T15:10:19.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.643 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.643 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:19.643 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424712+0000", 2020-08-07T15:10:19.644 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4749999999999996e-06 2020-08-07T15:10:19.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.647 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:19.647 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424719+0000", 2020-08-07T15:10:19.647 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77047786799999995 2020-08-07T15:10:19.647 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.648 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.648 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:19.648 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195197+0000", 2020-08-07T15:10:19.648 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000101909 2020-08-07T15:10:19.648 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.649 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.649 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:19.649 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195299+0000", 2020-08-07T15:10:19.649 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000248247 2020-08-07T15:10:19.651 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.652 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.652 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:19.652 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195547+0000", 2020-08-07T15:10:19.652 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033458699999999999 2020-08-07T15:10:19.652 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.653 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.653 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:19.653 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198893+0000", 2020-08-07T15:10:19.653 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0419999999999999e-06 2020-08-07T15:10:19.653 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.653 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.654 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:19.659 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198894+0000", 2020-08-07T15:10:19.660 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9354999999999997e-05 2020-08-07T15:10:19.660 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.660 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.660 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:19.660 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198953+0000", 2020-08-07T15:10:19.661 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001861096 2020-08-07T15:10:19.661 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.665 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.666 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:19.666 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200815+0000", 2020-08-07T15:10:19.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00049697299999999995 2020-08-07T15:10:19.676 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.676 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:19.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201311+0000", 2020-08-07T15:10:19.679 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7099999999999998e-07 2020-08-07T15:10:19.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.680 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:19.680 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201312+0000", 2020-08-07T15:10:19.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5194000000000002e-05 2020-08-07T15:10:19.681 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.681 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:19.681 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201347+0000", 2020-08-07T15:10:19.681 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.398505079 2020-08-07T15:10:19.682 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:19.682 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:19.682 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:19.682 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.682 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.683 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73022 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:19.683 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:10:19.683 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 50.784619108999998, 2020-08-07T15:10:19.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3238320669999999, 2020-08-07T15:10:19.683 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:19.683 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:19.684 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:19.684 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:19.684 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:19.684 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73022 2020-08-07T15:10:19.684 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.685 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:19.685 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.685 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:19.685 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:10:19.685 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:19.702 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.702 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.703 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:19.703 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:10:19.703 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6769999999999999e-05 2020-08-07T15:10:19.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.704 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:19.705 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884041+0000", 2020-08-07T15:10:19.705 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.703e-06 2020-08-07T15:10:19.705 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.740 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.740 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:19.740 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884044+0000", 2020-08-07T15:10:19.758 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5949999999999999e-06 2020-08-07T15:10:19.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.759 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:19.759 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884053+0000", 2020-08-07T15:10:19.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000003e-06 2020-08-07T15:10:19.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:19.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884058+0000", 2020-08-07T15:10:19.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7281999999999999e-05 2020-08-07T15:10:19.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.779 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:19.779 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884105+0000", 2020-08-07T15:10:19.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.044e-05 2020-08-07T15:10:19.780 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.780 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:19.780 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884116+0000", 2020-08-07T15:10:19.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.54062884300000003 2020-08-07T15:10:19.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.797 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:19.797 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424745+0000", 2020-08-07T15:10:19.797 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3149999999999998e-06 2020-08-07T15:10:19.797 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.801 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.801 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:19.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424754+0000", 2020-08-07T15:10:19.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77175732200000002 2020-08-07T15:10:19.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:19.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196511+0000", 2020-08-07T15:10:19.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010110799999999999 2020-08-07T15:10:19.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:19.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196612+0000", 2020-08-07T15:10:19.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017543100000000001 2020-08-07T15:10:19.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:19.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196788+0000", 2020-08-07T15:10:19.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032134569999999999 2020-08-07T15:10:19.806 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:19.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200001+0000", 2020-08-07T15:10:19.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.071e-06 2020-08-07T15:10:19.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:19.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200002+0000", 2020-08-07T15:10:19.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2527999999999995e-05 2020-08-07T15:10:19.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.808 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:19.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200065+0000", 2020-08-07T15:10:19.809 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001327433 2020-08-07T15:10:19.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.818 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:19.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201392+0000", 2020-08-07T15:10:19.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063854030000000004 2020-08-07T15:10:19.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:19.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207778+0000", 2020-08-07T15:10:19.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.285e-06 2020-08-07T15:10:19.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:19.826 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207779+0000", 2020-08-07T15:10:19.826 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7070999999999995e-05 2020-08-07T15:10:19.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.827 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:19.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207846+0000", 2020-08-07T15:10:19.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3238320669999999 2020-08-07T15:10:19.827 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:19.842 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:19.843 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:19.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.843 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73023 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:19.843 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:10:19.844 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 50.671457468, 2020-08-07T15:10:19.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2113701729999999, 2020-08-07T15:10:19.858 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:19.858 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:19.858 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:19.859 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:19.859 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:19.859 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73023 2020-08-07T15:10:19.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.859 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:19.886 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:19.887 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:10:19.887 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:19.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:19.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:10:19.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4105e-05 2020-08-07T15:10:19.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.919 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:19.920 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997210+0000", 2020-08-07T15:10:19.920 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1089999999999997e-06 2020-08-07T15:10:19.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:19.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997215+0000", 2020-08-07T15:10:19.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7040000000000006e-06 2020-08-07T15:10:19.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.922 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:19.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997224+0000", 2020-08-07T15:10:19.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4439999999999999e-06 2020-08-07T15:10:19.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:19.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997230+0000", 2020-08-07T15:10:19.931 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5977999999999999e-05 2020-08-07T15:10:19.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:19.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997276+0000", 2020-08-07T15:10:19.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2660000000000007e-06 2020-08-07T15:10:19.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:19.947 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997285+0000", 2020-08-07T15:10:19.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.42749189599999998 2020-08-07T15:10:19.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.956 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.956 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:19.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424777+0000", 2020-08-07T15:10:19.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8940000000000004e-06 2020-08-07T15:10:19.957 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:19.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424784+0000", 2020-08-07T15:10:19.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77268199699999995 2020-08-07T15:10:19.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:19.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197466+0000", 2020-08-07T15:10:19.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011277699999999999 2020-08-07T15:10:19.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:19.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197579+0000", 2020-08-07T15:10:19.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018970599999999999 2020-08-07T15:10:19.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:19.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197768+0000", 2020-08-07T15:10:19.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043168529999999998 2020-08-07T15:10:19.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:19.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:19.975 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:19.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202085+0000", 2020-08-07T15:10:19.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.186e-06 2020-08-07T15:10:19.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:20.014 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202086+0000", 2020-08-07T15:10:20.015 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8975999999999999e-05 2020-08-07T15:10:20.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.015 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.016 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:20.016 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202145+0000", 2020-08-07T15:10:20.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00035331499999999999 2020-08-07T15:10:20.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.020 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.020 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:20.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202499+0000", 2020-08-07T15:10:20.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0059703159999999998 2020-08-07T15:10:20.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:20.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208469+0000", 2020-08-07T15:10:20.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7400000000000002e-07 2020-08-07T15:10:20.023 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:20.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208470+0000", 2020-08-07T15:10:20.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5877000000000006e-05 2020-08-07T15:10:20.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:20.026 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208546+0000", 2020-08-07T15:10:20.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2113701729999999 2020-08-07T15:10:20.027 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:20.027 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:20.027 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:20.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.028 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73024 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e131)", 2020-08-07T15:10:20.028 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:26.502031+0000", 2020-08-07T15:10:20.029 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 50.166602320999999, 2020-08-07T15:10:20.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.70660062599999995, 2020-08-07T15:10:20.033 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:20.033 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:20.034 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:20.034 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:20.034 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:20.035 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73024 2020-08-07T15:10:20.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.035 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:20.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:20.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502031+0000", 2020-08-07T15:10:20.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:20.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:20.037 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502031+0000", 2020-08-07T15:10:20.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0858999999999997e-05 2020-08-07T15:10:20.038 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.038 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:20.038 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502062+0000", 2020-08-07T15:10:20.038 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1159999999999998e-06 2020-08-07T15:10:20.039 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.039 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.039 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:20.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502067+0000", 2020-08-07T15:10:20.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1810999999999999e-05 2020-08-07T15:10:20.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:20.040 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502078+0000", 2020-08-07T15:10:20.040 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2500000000000006e-06 2020-08-07T15:10:20.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.041 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:20.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502087+0000", 2020-08-07T15:10:20.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3461999999999996e-05 2020-08-07T15:10:20.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.042 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:20.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502150+0000", 2020-08-07T15:10:20.042 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3108999999999999e-05 2020-08-07T15:10:20.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.044 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:20.044 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502163+0000", 2020-08-07T15:10:20.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69604141200000003 2020-08-07T15:10:20.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.045 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:20.045 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198205+0000", 2020-08-07T15:10:20.045 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013494700000000001 2020-08-07T15:10:20.045 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.046 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.046 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:20.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198340+0000", 2020-08-07T15:10:20.046 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000169696 2020-08-07T15:10:20.046 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.047 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:20.047 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198509+0000", 2020-08-07T15:10:20.047 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0036466960000000001 2020-08-07T15:10:20.048 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.048 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.048 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:20.049 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202156+0000", 2020-08-07T15:10:20.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.16e-07 2020-08-07T15:10:20.049 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.049 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.049 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:20.050 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202157+0000", 2020-08-07T15:10:20.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5290000000000002e-05 2020-08-07T15:10:20.050 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.050 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.051 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:20.051 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202202+0000", 2020-08-07T15:10:20.051 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00061786700000000003 2020-08-07T15:10:20.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.052 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.052 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:20.052 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202820+0000", 2020-08-07T15:10:20.053 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0057728759999999997 2020-08-07T15:10:20.053 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.053 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:20.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208593+0000", 2020-08-07T15:10:20.054 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4000000000000001e-07 2020-08-07T15:10:20.062 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.062 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.062 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:20.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208594+0000", 2020-08-07T15:10:20.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7679000000000003e-05 2020-08-07T15:10:20.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.063 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:20.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208631+0000", 2020-08-07T15:10:20.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.70660062599999995 2020-08-07T15:10:20.064 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:20.066 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:20.067 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:20.067 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:20.067 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:20.067 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:20.075 INFO:teuthology.orchestra.run.smithi074:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 8192 --object-size 65536 -p unique_pool_2 bench 90 write' 2020-08-07T15:10:20.102 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:16.671+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:20.103 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:10:20.103 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:10:20.103 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:10:20.103 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:10:20.104 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:10:20.104 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:10:20.105 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:10:20.105 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:10:20.123 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:10:20.123 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:10:20.123 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:10:20.124 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:10:20.124 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:10:20.124 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:10:20.125 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:10:20.133 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2020-08-07T15:10:16.657218+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":87,"acting":87,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8395732,"kb_used_data":7060,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746578988,"statfs":{"total":773094113280,"available":764496883712,"internally_reserved":8589934592,"allocated":7229440,"data_stored":2829468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-130765,"num_write_kb":-929865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2645,"num_bytes_recovered":-173219840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.434983"},"pg_stats":[{"pgid":"1.14","version":"16'2","reported_seq":"94","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.644555+0000","last_change":"2020-08-07T15:10:10.644555+0000","last_active":"2020-08-07T15:10:10.644555+0000","last_peered":"2020-08-07T15:10:10.644555+0000","last_clean":"2020-08-07T15:10:10.644555+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:10:10.644555+0000","last_undegraded":"2020-08-07T15:10:10.644555+0000","last_fullsized":"2020-08-07T15:10:10.644555+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:10.644460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.644460+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"163","reported_epoch":"160","state":"active+clean","last_fresh":"2020-08-07T15:10:09.315412+0000","last_change":"2020-08-07T15:10:07.295971+0000","last_active":"2020-08-07T15:10:09.315412+0000","last_peered":"2020-08-07T15:10:09.315412+0000","last_clean":"2020-08-07T15:10:09.315412+0000","last_became_active":"2020-08-07T15:09:30.203592+0000","last_became_peered":"2020-08-07T15:09:30.203592+0000","last_unstale":"2020-08-07T15:10:09.315412+0000","last_undegraded":"2020-08-07T15:10:09.315412+0000","last_fullsized":"2020-08-07T15:10:09.315412+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:07.295863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:07.295863+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"91","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.218153+0000","last_change":"2020-08-07T15:08:55.078015+0000","last_active":"2020-08-07T15:10:10.218153+0000","last_peered":"2020-08-07T15:10:10.218153+0000","last_clean":"2020-08-07T15:10:10.218153+0000","last_became_active":"2020-08-07T15:08:41.685106+0000","last_became_peered":"2020-08-07T15:08:41.685106+0000","last_unstale":"2020-08-07T15:10:10.218153+0000","last_undegraded":"2020-08-07T15:10:10.218153+0000","last_fullsized":"2020-08-07T15:10:10.218153+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:55.077924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:55.077924+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"204","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.217677+0000","last_change":"2020-08-07T15:09:30.210669+0000","last_active":"2020-08-07T15:10:10.217677+0000","last_peered":"2020-08-07T15:10:10.217677+0000","last_clean":"2020-08-07T15:10:10.217677+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:10:10.217677+0000","last_undegraded":"2020-08-07T15:10:10.217677+0000","last_fullsized":"2020-08-07T15:10:10.217677+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.544324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.544324+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"198","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.217706+0000","last_change":"2020-08-07T15:08:51.599032+0000","last_active":"2020-08-07T15:10:10.217706+0000","last_peered":"2020-08-07T15:10:10.217706+0000","last_clean":"2020-08-07T15:10:10.217706+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:10:10.217706+0000","last_undegraded":"2020-08-07T15:10:10.217706+0000","last_fullsized":"2020-08-07T15:10:10.217706+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:51.598947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.598947+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"159","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.644530+0000","last_change":"2020-08-07T15:09:41.284321+0000","last_active":"2020-08-07T15:10:10.644530+0000","last_peered":"2020-08-07T15:10:10.644530+0000","last_clean":"2020-08-07T15:10:10.644530+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:10:10.644530+0000","last_undegraded":"2020-08-07T15:10:10.644530+0000","last_fullsized":"2020-08-07T15:10:10.644530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:41.284237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:41.284237+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"74","reported_epoch":"160","state":"active+clean","last_fresh":"2020-08-07T15:10:09.316521+0000","last_change":"2020-08-07T15:09:30.283753+0000","last_active":"2020-08-07T15:10:09.316521+0000","last_peered":"2020-08-07T15:10:09.316521+0000","last_clean":"2020-08-07T15:10:09.316521+0000","last_became_active":"2020-08-07T15:09:30.283176+0000","last_became_peered":"2020-08-07T15:09:30.283176+0000","last_unstale":"2020-08-07T15:10:09.316521+0000","last_undegraded":"2020-08-07T15:10:09.316521+0000","last_fullsized":"2020-08-07T15:10:09.316521+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:25.606757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:25.606757+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"158","reported_epoch":"160","state":"active+clean","last_fresh":"2020-08-07T15:10:09.316063+0000","last_change":"2020-08-07T15:08:40.057705+0000","last_active":"2020-08-07T15:10:09.316063+0000","last_peered":"2020-08-07T15:10:09.316063+0000","last_clean":"2020-08-07T15:10:09.316063+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:10:09.316063+0000","last_undegraded":"2020-08-07T15:10:09.316063+0000","last_fullsized":"2020-08-07T15:10:09.316063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:40.057622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:40.057622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"116","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.268900+0000","last_change":"2020-08-07T15:08:53.756554+0000","last_active":"2020-08-07T15:10:10.268900+0000","last_peered":"2020-08-07T15:10:10.268900+0000","last_clean":"2020-08-07T15:10:10.268900+0000","last_became_active":"2020-08-07T15:08:46.710404+0000","last_became_peered":"2020-08-07T15:08:46.710404+0000","last_unstale":"2020-08-07T15:10:10.268900+0000","last_undegraded":"2020-08-07T15:10:10.268900+0000","last_fullsized":"2020-08-07T15:10:10.268900+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:53.756343+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:53.756343+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"158","reported_epoch":"160","state":"active+clean","last_fresh":"2020-08-07T15:10:09.316308+0000","last_change":"2020-08-07T15:09:54.908283+0000","last_active":"2020-08-07T15:10:09.316308+0000","last_peered":"2020-08-07T15:10:09.316308+0000","last_clean":"2020-08-07T15:10:09.316308+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:10:09.316308+0000","last_undegraded":"2020-08-07T15:10:09.316308+0000","last_fullsized":"2020-08-07T15:10:09.316308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:54.908197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:54.908197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"198","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.644613+0000","last_change":"2020-08-07T15:10:07.302258+0000","last_active":"2020-08-07T15:10:10.644613+0000","last_peered":"2020-08-07T15:10:10.644613+0000","last_clean":"2020-08-07T15:10:10.644613+0000","last_became_active":"2020-08-07T15:09:30.201829+0000","last_became_peered":"2020-08-07T15:09:30.201829+0000","last_unstale":"2020-08-07T15:10:10.644613+0000","last_undegraded":"2020-08-07T15:10:10.644613+0000","last_fullsized":"2020-08-07T15:10:10.644613+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:07.302204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:07.302204+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"158","reported_epoch":"160","state":"active+clean","last_fresh":"2020-08-07T15:10:09.316031+0000","last_change":"2020-08-07T15:09:51.036281+0000","last_active":"2020-08-07T15:10:09.316031+0000","last_peered":"2020-08-07T15:10:09.316031+0000","last_clean":"2020-08-07T15:10:09.316031+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:10:09.316031+0000","last_undegraded":"2020-08-07T15:10:09.316031+0000","last_fullsized":"2020-08-07T15:10:09.316031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:51.036224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:51.036224+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"161","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.644228+0000","last_change":"2020-08-07T15:08:50.686578+0000","last_active":"2020-08-07T15:10:10.644228+0000","last_peered":"2020-08-07T15:10:10.644228+0000","last_clean":"2020-08-07T15:10:10.644228+0000","last_became_active":"2020-08-07T15:08:36.633406+0000","last_became_peered":"2020-08-07T15:08:36.633406+0000","last_unstale":"2020-08-07T15:10:10.644228+0000","last_undegraded":"2020-08-07T15:10:10.644228+0000","last_fullsized":"2020-08-07T15:10:10.644228+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:50.686496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:50.686496+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"190","reported_epoch":"160","state":"active+clean","last_fresh":"2020-08-07T15:10:09.315942+0000","last_change":"2020-08-07T15:09:55.879867+0000","last_active":"2020-08-07T15:10:09.315942+0000","last_peered":"2020-08-07T15:10:09.315942+0000","last_clean":"2020-08-07T15:10:09.315942+0000","last_became_active":"2020-08-07T15:08:36.627426+0000","last_became_peered":"2020-08-07T15:08:36.627426+0000","last_unstale":"2020-08-07T15:10:09.315942+0000","last_undegraded":"2020-08-07T15:10:09.315942+0000","last_fullsized":"2020-08-07T15:10:09.315942+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:55.879785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:55.879785+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"159","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.218565+0000","last_change":"2020-08-07T15:10:08.656952+0000","last_active":"2020-08-07T15:10:10.218565+0000","last_peered":"2020-08-07T15:10:10.218565+0000","last_clean":"2020-08-07T15:10:10.218565+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:10:10.218565+0000","last_undegraded":"2020-08-07T15:10:10.218565+0000","last_fullsized":"2020-08-07T15:10:10.218565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:08.656855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:10:08.656855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"177","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.268682+0000","last_change":"2020-08-07T15:09:27.181676+0000","last_active":"2020-08-07T15:10:10.268682+0000","last_peered":"2020-08-07T15:10:10.268682+0000","last_clean":"2020-08-07T15:10:10.268682+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:10:10.268682+0000","last_undegraded":"2020-08-07T15:10:10.268682+0000","last_fullsized":"2020-08-07T15:10:10.268682+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:05.606603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:05.606603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"203","reported_epoch":"160","state":"active+clean","last_fresh":"2020-08-07T15:10:09.316039+0000","last_change":"2020-08-07T15:08:42.069637+0000","last_active":"2020-08-07T15:10:09.316039+0000","last_peered":"2020-08-07T15:10:09.316039+0000","last_clean":"2020-08-07T15:10:09.316039+0000","last_became_active":"2020-08-07T15:08:36.628365+0000","last_became_peered":"2020-08-07T15:08:36.628365+0000","last_unstale":"2020-08-07T15:10:09.316039+0000","last_undegraded":"2020-08-07T15:10:09.316039+0000","last_fullsized":"2020-08-07T15:10:09.316039+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:42.069585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:42.069585+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"159","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.218370+0000","last_change":"2020-08-07T15:09:52.637867+0000","last_active":"2020-08-07T15:10:10.218370+0000","last_peered":"2020-08-07T15:10:10.218370+0000","last_clean":"2020-08-07T15:10:10.218370+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:10:10.218370+0000","last_undegraded":"2020-08-07T15:10:10.218370+0000","last_fullsized":"2020-08-07T15:10:10.218370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:52.637791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:09:52.637791+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"157","reported_epoch":"162","state":"active+clean","last_fresh":"2020-08-07T15:10:11.720144+0000","last_change":"2020-08-07T15:10:11.720217+0000","last_active":"2020-08-07T15:10:11.720144+0000","last_peered":"2020-08-07T15:10:11.720144+0000","last_clean":"2020-08-07T15:10:11.720144+0000","last_became_active":"2020-08-07T15:08:36.618164+0000","last_became_peered":"2020-08-07T15:08:36.618164+0000","last_unstale":"2020-08-07T15:10:11.720144+0000","last_undegraded":"2020-08-07T15:10:11.720144+0000","last_fullsized":"2020-08-07T15:10:11.720144+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:11.720143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:11.720143+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"155","reported_epoch":"160","state":"active+clean","last_fresh":"2020-08-07T15:10:09.316235+0000","last_change":"2020-08-07T15:09:37.017383+0000","last_active":"2020-08-07T15:10:09.316235+0000","last_peered":"2020-08-07T15:10:09.316235+0000","last_clean":"2020-08-07T15:10:09.316235+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:10:09.316235+0000","last_undegraded":"2020-08-07T15:10:09.316235+0000","last_fullsized":"2020-08-07T15:10:09.316235+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:37.017307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:37.017307+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"229","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.644588+0000","last_change":"2020-08-07T15:09:30.208873+0000","last_active":"2020-08-07T15:10:10.644588+0000","last_peered":"2020-08-07T15:10:10.644588+0000","last_clean":"2020-08-07T15:10:10.644588+0000","last_became_active":"2020-08-07T15:09:30.208345+0000","last_became_peered":"2020-08-07T15:09:30.208345+0000","last_unstale":"2020-08-07T15:10:10.644588+0000","last_undegraded":"2020-08-07T15:10:10.644588+0000","last_fullsized":"2020-08-07T15:10:10.644588+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.381325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.381325+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"134","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.715147+0000","last_change":"2020-08-07T15:10:10.715147+0000","last_active":"2020-08-07T15:10:10.715147+0000","last_peered":"2020-08-07T15:10:10.715147+0000","last_clean":"2020-08-07T15:10:10.715147+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:10:10.715147+0000","last_undegraded":"2020-08-07T15:10:10.715147+0000","last_fullsized":"2020-08-07T15:10:10.715147+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.715051+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"131","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.217821+0000","last_change":"2020-08-07T15:09:09.691838+0000","last_active":"2020-08-07T15:10:10.217821+0000","last_peered":"2020-08-07T15:10:10.217821+0000","last_clean":"2020-08-07T15:10:10.217821+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:10:10.217821+0000","last_undegraded":"2020-08-07T15:10:10.217821+0000","last_fullsized":"2020-08-07T15:10:10.217821+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.691753+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"194","reported_epoch":"160","state":"active+clean","last_fresh":"2020-08-07T15:10:09.315907+0000","last_change":"2020-08-07T15:09:25.984108+0000","last_active":"2020-08-07T15:10:09.315907+0000","last_peered":"2020-08-07T15:10:09.315907+0000","last_clean":"2020-08-07T15:10:09.315907+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:10:09.315907+0000","last_undegraded":"2020-08-07T15:10:09.315907+0000","last_fullsized":"2020-08-07T15:10:09.315907+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:25.984018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:25.984018+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"152","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.962481+0000","last_change":"2020-08-07T15:10:10.962565+0000","last_active":"2020-08-07T15:10:10.962481+0000","last_peered":"2020-08-07T15:10:10.962481+0000","last_clean":"2020-08-07T15:10:10.962481+0000","last_became_active":"2020-08-07T15:08:41.680448+0000","last_became_peered":"2020-08-07T15:08:41.680448+0000","last_unstale":"2020-08-07T15:10:10.962481+0000","last_undegraded":"2020-08-07T15:10:10.962481+0000","last_fullsized":"2020-08-07T15:10:10.962481+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:10:10.962480+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.962480+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"120","reported_epoch":"160","state":"active+clean","last_fresh":"2020-08-07T15:10:09.316583+0000","last_change":"2020-08-07T15:09:36.687393+0000","last_active":"2020-08-07T15:10:09.316583+0000","last_peered":"2020-08-07T15:10:09.316583+0000","last_clean":"2020-08-07T15:10:09.316583+0000","last_became_active":"2020-08-07T15:09:30.204655+0000","last_became_peered":"2020-08-07T15:09:30.204655+0000","last_unstale":"2020-08-07T15:10:09.316583+0000","last_undegraded":"2020-08-07T15:10:09.316583+0000","last_fullsized":"2020-08-07T15:10:09.316583+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:36.687314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:36.687314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"194","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.644374+0000","last_change":"2020-08-07T15:09:08.399507+0000","last_active":"2020-08-07T15:10:10.644374+0000","last_peered":"2020-08-07T15:10:10.644374+0000","last_clean":"2020-08-07T15:10:10.644374+0000","last_became_active":"2020-08-07T15:08:41.689435+0000","last_became_peered":"2020-08-07T15:08:41.689435+0000","last_unstale":"2020-08-07T15:10:10.644374+0000","last_undegraded":"2020-08-07T15:10:10.644374+0000","last_fullsized":"2020-08-07T15:10:10.644374+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:08.399439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:08.399439+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"80","reported_epoch":"160","state":"active+clean","last_fresh":"2020-08-07T15:10:09.316344+0000","last_change":"2020-08-07T15:09:09.868283+0000","last_active":"2020-08-07T15:10:09.316344+0000","last_peered":"2020-08-07T15:10:09.316344+0000","last_clean":"2020-08-07T15:10:09.316344+0000","last_became_active":"2020-08-07T15:08:46.710059+0000","last_became_peered":"2020-08-07T15:08:46.710059+0000","last_unstale":"2020-08-07T15:10:09.316344+0000","last_undegraded":"2020-08-07T15:10:09.316344+0000","last_fullsized":"2020-08-07T15:10:09.316344+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.868198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.868198+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":"0","reported_epoch":"0","state":"unknown","last_fresh":"2020-08-07T15:10:15.236926+0000","last_change":"2020-08-07T15:10:15.236926+0000","last_active":"2020-08-07T15:10:15.236926+0000","last_peered":"2020-08-07T15:10:15.236926+0000","last_clean":"2020-08-07T15:10:15.236926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2020-08-07T15:10:15.236926+0000","last_undegraded":"2020-08-07T15:10:15.236926+0000","last_fullsized":"2020-08-07T15:10:15.236926+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"103","reported_epoch":"161","state":"active+clean","last_fresh":"2020-08-07T15:10:10.218309+0000","last_change":"2020-08-07T15:09:08.653773+0000","last_active":"2020-08-07T15:10:10.218309+0000","last_peered":"2020-08-07T15:10:10.218309+0000","last_clean":"2020-08-07T15:10:10.218309+0000","last_became_active":"2020-08-07T15:08:41.677944+0000","last_became_peered":"2020-08-07T15:08:41.677944+0000","last_unstale":"2020-08-07T15:10:10.218309+0000","last_undegraded":"2020-08-07T15:10:10.218309+0000","last_fullsized":"2020-08-07T15:10:10.218309+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:08.653722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:08.653722+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":63,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542193,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049488,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322352,"statfs":{"total":96636764160,"available":95562088448,"internally_reserved":1073741824,"allocated":925696,"data_stored":360808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.95099999999999996,"15min":0.91200000000000003},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.1739999999999999,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.095,"5min":1.0149999999999999,"15min":0.97999999999999998},"min":{"1min":0.54300000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.2430000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.357,"15min":1.3149999999999999},"min":{"1min":0.68700000000000006,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":5.6239999999999997,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.105,"5min":1.0720000000000001,"15min":1.0009999999999999},"min":{"1min":0.64100000000000001,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.206,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":0.92800000000000005}]},{"osd":2,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.272,"15min":1.175},"min":{"1min":0.71999999999999997,"5min":0.628,"15min":0.628},"max":{"1min":5.6619999999999999,"5min":5.6619999999999999,"15min":5.6619999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.446,"5min":1.26,"15min":1.165},"min":{"1min":0.66800000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":5.1319999999999997,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":1.0309999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":75.045000000000002,"5min":16.263999999999999,"15min":6.5359999999999996},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":917.38699999999994,"5min":917.38699999999994,"15min":917.38699999999994},"last":1.2969999999999999},{"interface":"front","average":{"1min":19.670999999999999,"5min":4.9939999999999998,"15min":2.5600000000000001},"min":{"1min":0.80000000000000004,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":333.59199999999998,"5min":333.59199999999998,"15min":333.59199999999998},"last":1.2729999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.411,"15min":1.321},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":4.5250000000000004,"5min":4.835,"15min":4.835},"last":0.998},{"interface":"front","average":{"1min":52.259999999999998,"5min":11.52,"15min":4.7140000000000004},"min":{"1min":0.77000000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":866.24099999999999,"5min":866.24099999999999,"15min":866.24099999999999},"last":1.339}]},{"osd":5,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.2869999999999999,"5min":23.440000000000001,"15min":32.817999999999998},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":6.6509999999999998,"5min":831.88099999999997,"15min":831.88099999999997},"last":1.1659999999999999},{"interface":"front","average":{"1min":2.2519999999999998,"5min":1.8380000000000001,"15min":1.609},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":6.6790000000000003,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":1.1850000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":13.818,"5min":4.0019999999999998,"15min":2.3460000000000001},"min":{"1min":0.82999999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":224.14500000000001,"5min":224.14500000000001,"15min":224.14500000000001},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.6839999999999999,"15min":1.5660000000000001},"min":{"1min":0.90600000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":7.4359999999999999,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":1.2190000000000001}]}]},{"osd":7,"up_from":13,"seq":55834574898,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049460,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322380,"statfs":{"total":96636764160,"available":95562117120,"internally_reserved":1073741824,"allocated":897024,"data_stored":351306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.179,"15min":1.2809999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8149999999999999,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":15.239000000000001,"5min":3.9529999999999998,"15min":2.0840000000000001},"min":{"1min":0.78100000000000003,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":320.31299999999999,"5min":320.31299999999999,"15min":320.31299999999999},"last":0.79300000000000004}]},{"osd":1,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":24.821999999999999,"5min":5.7329999999999997,"15min":2.5489999999999999},"min":{"1min":0.46700000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":547.65300000000002,"5min":547.65300000000002,"15min":547.65300000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":4.9130000000000003,"15min":2.5619999999999998},"min":{"1min":0.69799999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.5449999999999999,"5min":372.50799999999998,"15min":372.50799999999998},"last":1.0860000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.208,"15min":1.1990000000000001},"min":{"1min":0.502,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":5.2400000000000002,"5min":5.2400000000000002,"15min":5.2400000000000002},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.286,"5min":1.2390000000000001,"15min":1.2609999999999999},"min":{"1min":0.66800000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":6.0490000000000004,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":0.88400000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.4370000000000001,"15min":1.496},"min":{"1min":0.878,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.0670000000000002,"5min":3.46,"15min":3.46},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3600000000000001,"15min":1.3779999999999999},"min":{"1min":0.70299999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":5.2770000000000001,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":1.056}]},{"osd":4,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.504,"15min":1.6100000000000001},"min":{"1min":0.70099999999999996,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.165,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.46,"5min":1.625,"15min":1.6559999999999999},"min":{"1min":0.96199999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":2.4279999999999999,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.1399999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.2809999999999999,"15min":1.3360000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.3069999999999999,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.6259999999999999,"15min":1.7190000000000001},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.4630000000000001,"5min":5.327,"15min":5.327},"last":1.46}]},{"osd":6,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.458,"15min":1.4410000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.4900000000000002,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.351},{"interface":"front","average":{"1min":1.5720000000000001,"5min":11.603999999999999,"15min":16.085000000000001},"min":{"1min":1.004,"5min":0.87,"15min":0.87},"max":{"1min":2.7719999999999998,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.431}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049460,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322380,"statfs":{"total":96636764160,"available":95562117120,"internally_reserved":1073741824,"allocated":897024,"data_stored":351306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":20.945,"15min":29.771999999999998},"min":{"1min":0.49199999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":6.8810000000000002,"5min":497.04000000000002,"15min":497.04000000000002},"last":0.85599999999999998},{"interface":"front","average":{"1min":25.385999999999999,"5min":14.656000000000001,"15min":5.6390000000000002},"min":{"1min":0.52200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":513.78899999999999,"5min":993.84500000000003,"15min":993.84500000000003},"last":0.89700000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.999,"15min":0.93200000000000005},"min":{"1min":0.60099999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.726,"5min":4.585,"15min":4.585},"last":0.879},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.254,"15min":1.22},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":6.4029999999999996,"5min":6.4029999999999996,"15min":6.4029999999999996},"last":0.999}]},{"osd":3,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":40.524000000000001,"5min":9.2829999999999995,"15min":3.968},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":828.31600000000003,"5min":828.31600000000003,"15min":828.31600000000003},"last":0.74399999999999999},{"interface":"front","average":{"1min":44.292000000000002,"5min":10.169,"15min":4.359},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":905.38300000000004,"5min":905.38300000000004,"15min":905.38300000000004},"last":0.77000000000000002}]},{"osd":4,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":27.831,"5min":6.774,"15min":3.1139999999999999},"min":{"1min":0.79900000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":550.66800000000001,"5min":550.66800000000001,"15min":550.66800000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.3400000000000001,"15min":1.246},"min":{"1min":0.92100000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":5.9420000000000002,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.145}]},{"osd":5,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":8.4550000000000001,"15min":11.49},"min":{"1min":0.71199999999999997,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.7200000000000002,"5min":213.63800000000001,"15min":213.63800000000001},"last":1.0800000000000001},{"interface":"front","average":{"1min":58.497,"5min":12.720000000000001,"15min":5.0330000000000004},"min":{"1min":0.76500000000000001,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":875.44600000000003,"5min":875.44600000000003,"15min":875.44600000000003},"last":1.099}]},{"osd":6,"last update":"Fri Aug 7 15:09:27 2020","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.587,"15min":1.5629999999999999},"min":{"1min":0.81000000000000005,"5min":0.754,"15min":0.754},"max":{"1min":5.7149999999999999,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":1.2509999999999999},{"interface":"front","average":{"1min":11.539,"5min":3.5979999999999999,"15min":2.1059999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":207.137,"5min":207.137,"15min":207.137},"last":1.204}]},{"osd":7,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.782,"5min":1.6879999999999999,"15min":1.639},"min":{"1min":0.94099999999999995,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":6.444,"5min":6.444,"15min":6.444},"last":1.232},{"interface":"front","average":{"1min":1.6819999999999999,"5min":1.6779999999999999,"15min":1.546},"min":{"1min":0.93300000000000005,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":5.9279999999999999,"5min":10.752000000000001,"15min":10.752000000000001},"last":1.1659999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049472,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322368,"statfs":{"total":96636764160,"available":95562104832,"internally_reserved":1073741824,"allocated":909312,"data_stored":356065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":17.058,"15min":6.3369999999999997},"min":{"1min":0.61399999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.1269999999999998,"5min":845.57100000000003,"15min":845.57100000000003},"last":0.64400000000000002},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.0169999999999999,"15min":0.96099999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":4.8209999999999997,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":1.153}]},{"osd":2,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.0189999999999999,"15min":0.95799999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.7719999999999998,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":13.177,"5min":22.48,"15min":29.32},"min":{"1min":0.61399999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":226.911,"5min":606.18499999999995,"15min":606.18499999999995},"last":0.71199999999999997}]},{"osd":3,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.0660000000000001,"15min":0.97899999999999998},"min":{"1min":0.54100000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6859999999999999,"5min":3.1779999999999999,"15min":3.1779999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.3160000000000001,"15min":1.2170000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.976,"5min":4.21,"15min":4.21},"last":0.95399999999999996}]},{"osd":4,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.2829999999999999,"15min":1.2050000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":8.3100000000000005,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.7709999999999999,"5min":1.446,"15min":1.3200000000000001},"min":{"1min":0.84599999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":8.4190000000000005,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":1.0149999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.9770000000000001,"5min":9.0380000000000003,"15min":3.8300000000000001},"min":{"1min":0.77800000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":7.8929999999999998,"5min":501.267,"15min":501.267},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.2609999999999999,"15min":1.157},"min":{"1min":0.78200000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.0939999999999999,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":1.177}]},{"osd":6,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":5.7779999999999996,"15min":7.6680000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.0059999999999998,"5min":124.738,"15min":124.738},"last":1.123},{"interface":"front","average":{"1min":1.627,"5min":11.811,"15min":16.372},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":4.6879999999999997,"5min":294.94,"15min":294.94},"last":1.073}]},{"osd":7,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.5589999999999999,"15min":1.577},"min":{"1min":0.95399999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":4.7169999999999996,"5min":6.468,"15min":6.468},"last":1.333},{"interface":"front","average":{"1min":1.363,"5min":1.4410000000000001,"15min":1.4039999999999999},"min":{"1min":0.88800000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.1040000000000001,"5min":3.117,"15min":3.117},"last":1.216}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049460,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322380,"statfs":{"total":96636764160,"available":95562117120,"internally_reserved":1073741824,"allocated":897024,"data_stored":351306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.6950000000000001,"5min":1.268,"15min":1.167},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":8.4930000000000003,"5min":8.4930000000000003,"15min":8.4930000000000003},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.357,"5min":1.113,"15min":1.0209999999999999},"min":{"1min":0.61899999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":3.5,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":1.137}]},{"osd":1,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.9390000000000001,"5min":1.5469999999999999,"15min":1.2430000000000001},"min":{"1min":0.54400000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":5.6820000000000004,"5min":18.672000000000001,"15min":18.672000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.351,"15min":1.357},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":3.585,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":1.407}]},{"osd":3,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.268,"15min":1.198},"min":{"1min":0.72799999999999998,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.8740000000000001,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.0900000000000001},{"interface":"front","average":{"1min":34.170999999999999,"5min":8.0510000000000002,"15min":3.7210000000000001},"min":{"1min":0.68799999999999994,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":501.75599999999997,"5min":501.75599999999997,"15min":501.75599999999997},"last":1.024}]},{"osd":4,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.403,"15min":1.2070000000000001},"min":{"1min":0.65200000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":3.4180000000000001,"5min":19.783000000000001,"15min":19.783000000000001},"last":1.496},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.546,"15min":1.6100000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":3.5569999999999999,"5min":3.5569999999999999,"15min":3.5569999999999999},"last":1.536}]},{"osd":5,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.292,"15min":1.272},"min":{"1min":0.83299999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.323,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":1.5669999999999999},{"interface":"front","average":{"1min":43.396999999999998,"5min":24.298999999999999,"15min":25.116},"min":{"1min":0.99299999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":627.53300000000002,"5min":627.53300000000002,"15min":627.53300000000002},"last":1.48}]},{"osd":6,"last update":"Fri Aug 7 15:09:30 2020","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.538,"15min":1.3260000000000001},"min":{"1min":0.85999999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":3.452,"5min":20.035,"15min":20.035},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.498,"5min":9.8979999999999997,"15min":4.1799999999999997},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.976,"5min":751.70000000000005,"15min":751.70000000000005},"last":1.4319999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":54.209000000000003,"5min":12.595000000000001,"15min":5.359},"min":{"1min":0.84999999999999998,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":501.91699999999997,"5min":501.91699999999997,"15min":501.91699999999997},"last":1.454},{"interface":"front","average":{"1min":1.665,"5min":1.919,"15min":1.637},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":3.133,"5min":35.646999999999998,"15min":35.646999999999998},"last":1.52}]}]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049460,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322380,"statfs":{"total":96636764160,"available":95562117120,"internally_reserved":1073741824,"allocated":897024,"data_stored":351306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":1.341,"15min":1.282},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.097,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.4530000000000001,"15min":1.575},"min":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.3039999999999998,"5min":13.067,"15min":13.067},"last":0.88100000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":4.5860000000000003,"15min":2.794},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.9910000000000001,"5min":270.68700000000001,"15min":270.68700000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.4379999999999999,"5min":2.0390000000000001,"15min":2.0209999999999999},"min":{"1min":0.55100000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":7.0940000000000003,"5min":14.349,"15min":14.349},"last":1.073}]},{"osd":2,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.266,"15min":1.1870000000000001},"min":{"1min":0.71399999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.2639999999999998,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":1.3180000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":2.109,"15min":1.8260000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.661,"5min":51.530000000000001,"15min":51.530000000000001},"last":1.0940000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":10.773999999999999,"15min":4.7220000000000004},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.1360000000000001,"5min":506.21300000000002,"15min":506.21300000000002},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.276,"5min":25.469000000000001,"15min":29.888999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.8940000000000001,"5min":677.88599999999997,"15min":677.88599999999997},"last":1.266}]},{"osd":5,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.5920000000000001,"15min":1.542},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.9710000000000001,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.8009999999999999,"15min":1.7789999999999999},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":3.044,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":1.629}]},{"osd":6,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.6259999999999999,"15min":1.5629999999999999},"min":{"1min":0.89800000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.9359999999999999,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":1.5760000000000001},{"interface":"front","average":{"1min":21.766999999999999,"5min":5.9390000000000001,"15min":3.3039999999999998},"min":{"1min":0.89900000000000002,"5min":0.876,"15min":0.876},"max":{"1min":387.911,"5min":387.911,"15min":387.911},"last":1.2470000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.9330000000000001,"15min":2.085},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.0110000000000001,"5min":5.734,"15min":5.734},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.325,"5min":1.9750000000000001,"15min":1.976},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.169,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":1.224}]}]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049460,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322380,"statfs":{"total":96636764160,"available":95562117120,"internally_reserved":1073741824,"allocated":897024,"data_stored":351306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":1.0720000000000001,"15min":1.1439999999999999},"min":{"1min":0.52900000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4390000000000001,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":1.244},{"interface":"front","average":{"1min":1.143,"5min":1.0069999999999999,"15min":0.90100000000000002},"min":{"1min":0.498,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":4.907,"5min":6.3970000000000002,"15min":6.3970000000000002},"last":0.96799999999999997}]},{"osd":1,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":23.940999999999999,"5min":5.8209999999999997,"15min":2.843},"min":{"1min":0.59599999999999997,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":320.67000000000002,"5min":320.67000000000002,"15min":320.67000000000002},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":7.5700000000000003,"15min":3.4409999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.784,"5min":689.846,"15min":689.846},"last":1.224}]},{"osd":2,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.3999999999999999,"15min":1.4339999999999999},"min":{"1min":0.77600000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.0899999999999999,"5min":6.8730000000000002,"15min":6.8730000000000002},"last":1.133},{"interface":"front","average":{"1min":1.258,"5min":1.516,"15min":1.5149999999999999},"min":{"1min":0.81699999999999995,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.3159999999999998,"5min":7.1360000000000001,"15min":7.1360000000000001},"last":1.177}]},{"osd":3,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":3.681,"15min":2.2320000000000002},"min":{"1min":0.78300000000000003,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.8149999999999999,"5min":247.202,"15min":247.202},"last":1.266},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.3720000000000001,"15min":1.2909999999999999},"min":{"1min":0.68500000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":4.931,"5min":6.431,"15min":6.431},"last":1.29}]},{"osd":5,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":10.57,"15min":4.6079999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.431,"5min":990.71600000000001,"15min":990.71600000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.6100000000000001,"15min":1.621},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":3.3519999999999999,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":0.80900000000000005}]},{"osd":6,"last update":"Fri Aug 7 15:09:28 2020","interfaces":[{"interface":"back","average":{"1min":1.6799999999999999,"5min":1.7509999999999999,"15min":1.7769999999999999},"min":{"1min":0.75800000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":5.0149999999999997,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":1.014},{"interface":"front","average":{"1min":1.498,"5min":12.069000000000001,"15min":16.762},"min":{"1min":0.83299999999999996,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":3.3530000000000002,"5min":389.14600000000002,"15min":389.14600000000002},"last":0.94799999999999995}]},{"osd":7,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":3.8300000000000001,"15min":2.2149999999999999},"min":{"1min":0.753,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":4.8710000000000004,"5min":252.798,"15min":252.798},"last":1.151},{"interface":"front","average":{"1min":1.262,"5min":10.175000000000001,"15min":4.4889999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.3050000000000002,"5min":501.93000000000001,"15min":501.93000000000001},"last":0.99199999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049472,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322368,"statfs":{"total":96636764160,"available":95562104832,"internally_reserved":1073741824,"allocated":909312,"data_stored":356065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":1.012,"15min":1.0549999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.651,"5min":2.766,"15min":2.766},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91600000000000004,"15min":0.84199999999999997},"min":{"1min":0.45500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.3340000000000001,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":0.68700000000000006}]},{"osd":1,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.4540000000000002,"5min":10.521000000000001,"15min":4.1260000000000003},"min":{"1min":0.56699999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":29.283000000000001,"5min":787.22900000000004,"15min":787.22900000000004},"last":1.0469999999999999},{"interface":"front","average":{"1min":8.0939999999999994,"5min":2.4969999999999999,"15min":1.571},"min":{"1min":0.60999999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":116.548,"5min":116.548,"15min":116.548},"last":1.518}]},{"osd":2,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":5.9340000000000002,"15min":8.1240000000000006},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.78,"5min":168.56899999999999,"15min":168.56899999999999},"last":1.085},{"interface":"front","average":{"1min":1.119,"5min":20.361000000000001,"15min":28.873999999999999},"min":{"1min":0.60899999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.665,"5min":676.27099999999996,"15min":676.27099999999996},"last":1.02}]},{"osd":3,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.6739999999999999,"5min":1.5069999999999999,"15min":1.2470000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":29.129000000000001,"5min":29.129000000000001,"15min":29.129000000000001},"last":1.113},{"interface":"front","average":{"1min":3.0219999999999998,"5min":9.9459999999999997,"15min":4.1239999999999997},"min":{"1min":0.70799999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":29.103000000000002,"5min":712.98699999999997,"15min":712.98699999999997},"last":1.486}]},{"osd":4,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":49.854999999999997,"5min":40.755000000000003,"15min":47.512999999999998},"min":{"1min":0.61899999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":971.67999999999995,"5min":971.67999999999995,"15min":971.67999999999995},"last":1.575},{"interface":"front","average":{"1min":1.236,"5min":1.264,"15min":1.292},"min":{"1min":0.64900000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.5960000000000001,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":1.204}]},{"osd":6,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":91.870999999999995,"5min":48.862000000000002,"15min":49.893000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":966.60900000000004,"5min":966.60900000000004,"15min":966.60900000000004},"last":1.1719999999999999},{"interface":"front","average":{"1min":28.396999999999998,"5min":21.527000000000001,"15min":24.52},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":501.79000000000002,"5min":517.26700000000005,"15min":517.26700000000005},"last":1.548}]},{"osd":7,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.341,"15min":1.319},"min":{"1min":0.77500000000000002,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.8940000000000001,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":0.879},{"interface":"front","average":{"1min":27.206,"5min":6.6630000000000003,"15min":3.2650000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":515.09100000000001,"5min":515.09100000000001,"15min":515.09100000000001},"last":1.143}]}]}]}} 2020-08-07T15:10:20.135 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:10:20.145 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early status --format=json 2020-08-07T15:10:20.146 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:10:20.146 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:20.075+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:20.164 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2020-08-07T15:10:20.165 INFO:teuthology.orchestra.run:waiting for 3000 2020-08-07T15:10:20.179 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:20.178+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:20.214 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:hints = 1 2020-08-07T15:10:20.214 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Maintaining 16 concurrent writes of 8192 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2020-08-07T15:10:20.215 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Object prefix: benchmark_data_smithi074_53697 2020-08-07T15:10:20.215 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-08-07T15:10:20.215 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 0 0 0 0 0 0 - 0 2020-08-07T15:10:20.287 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:20.278+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:20.319 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:20.319 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:10:20.320 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:10:20.320 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:20.339 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:10:20.380 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:20.378+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:20.487 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:20.480+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:20.586 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:20.587 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:10:20.587 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:10:20.587 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:10:20.587 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:20.589 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:20.586+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:20.597 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:10:20.689 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:20.687+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:20.701 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:10:20.702 INFO:teuthology.orchestra.run.smithi074.stdout:{"fsid":"a01d3e82-8d24-4288-9e42-576ab3ff4957","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub,nodeep-scrub flag(s) set","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":253,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":168,"num_osds":8,"num_up_osds":8,"osd_up_since":1596812777,"num_in_osds":8,"osd_in_since":1596812777,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":45}],"num_pgs":45,"num_pools":3,"num_objects":2,"data_bytes":0,"bytes_used":8597577728,"bytes_avail":764496535552,"bytes_total":773094113280},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2020-08-07T15:07:18.623632+0000","services":{}},"progress_events":{}} 2020-08-07T15:10:20.724 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2020-08-07T15:10:20.724 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:10:20.790 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:20.788+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:20.837 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:20.837 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:10:20.838 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:10:20.838 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:10:20.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.838 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106240 3.7 3:ebdd8242:::benchmark_data_smithi074_35447_object13279:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:10:20.839 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:10:20.839 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 153.47816234699999, 2020-08-07T15:10:20.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.90079396, 2020-08-07T15:10:20.839 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:20.840 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:20.840 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:20.840 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:10:20.840 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:20.840 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106240 2020-08-07T15:10:20.841 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.841 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:20.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:20.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:10:20.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:20.864 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:20.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:10:20.865 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2522000000000003e-05 2020-08-07T15:10:20.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.865 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:20.865 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355118+0000", 2020-08-07T15:10:20.866 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6159999999999998e-06 2020-08-07T15:10:20.866 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.874 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:20.875 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355125+0000", 2020-08-07T15:10:20.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2699e-05 2020-08-07T15:10:20.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.876 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:20.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355138+0000", 2020-08-07T15:10:20.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2310000000000001e-06 2020-08-07T15:10:20.876 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.877 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.877 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:20.877 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355144+0000", 2020-08-07T15:10:20.877 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9855000000000003e-05 2020-08-07T15:10:20.877 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.878 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.878 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:20.878 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355194+0000", 2020-08-07T15:10:20.878 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112713 2020-08-07T15:10:20.879 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.879 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:20.879 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355306+0000", 2020-08-07T15:10:20.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019794600000000001 2020-08-07T15:10:20.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.892 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:10:20.892 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355504+0000", 2020-08-07T15:10:20.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0048346029999999998 2020-08-07T15:10:20.892 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.906 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:20.906 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.360339+0000", 2020-08-07T15:10:20.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00033646400000000002 2020-08-07T15:10:20.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:20.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.360675+0000", 2020-08-07T15:10:20.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89508481200000001 2020-08-07T15:10:20.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:20.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255760+0000", 2020-08-07T15:10:20.918 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5659999999999999e-06 2020-08-07T15:10:20.918 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.919 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:20.919 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255762+0000", 2020-08-07T15:10:20.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6595000000000003e-05 2020-08-07T15:10:20.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:20.920 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255818+0000", 2020-08-07T15:10:20.920 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1338000000000003e-05 2020-08-07T15:10:20.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:20.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255860+0000", 2020-08-07T15:10:20.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.90079396 2020-08-07T15:10:20.927 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:20.927 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:20.927 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:20.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.928 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106240 3.7 e59/20 ondisk, result = 0)", 2020-08-07T15:10:20.928 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:10:20.928 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 153.381766395, 2020-08-07T15:10:20.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80440215100000001, 2020-08-07T15:10:20.929 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:20.929 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:20.929 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:20.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:20.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:10:20.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:20.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:20.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:10:20.931 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0052999999999999e-05 2020-08-07T15:10:20.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:20.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451482+0000", 2020-08-07T15:10:20.932 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3229999999999999e-06 2020-08-07T15:10:20.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.933 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.933 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:20.933 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451486+0000", 2020-08-07T15:10:20.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80420304799999998 2020-08-07T15:10:20.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.934 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.934 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:20.934 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255689+0000", 2020-08-07T15:10:20.934 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7759999999999997e-06 2020-08-07T15:10:20.934 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:20.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255694+0000", 2020-08-07T15:10:20.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5828999999999998e-05 2020-08-07T15:10:20.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:20.942 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255740+0000", 2020-08-07T15:10:20.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1466e-05 2020-08-07T15:10:20.942 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:20.951 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255751+0000", 2020-08-07T15:10:20.952 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112656 2020-08-07T15:10:20.953 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.953 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.953 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:20.953 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255864+0000", 2020-08-07T15:10:20.953 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80440215100000001 2020-08-07T15:10:20.954 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:20.954 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:20.954 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:20.954 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.954 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.955 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37057 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:10:20.955 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:10:20.955 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 86.589657685000006, 2020-08-07T15:10:20.955 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007248691, 2020-08-07T15:10:20.955 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:20.956 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:10:20.956 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:20.956 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:20.956 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:20.956 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37057 2020-08-07T15:10:20.957 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.957 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:20.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.957 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:20.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:10:20.958 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:20.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:20.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:10:20.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3661e-05 2020-08-07T15:10:20.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:20.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243624+0000", 2020-08-07T15:10:20.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7519999999999996e-06 2020-08-07T15:10:20.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:20.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243632+0000", 2020-08-07T15:10:20.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6392000000000002e-05 2020-08-07T15:10:20.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:20.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243648+0000", 2020-08-07T15:10:20.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6659999999999997e-06 2020-08-07T15:10:20.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.962 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:20.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243657+0000", 2020-08-07T15:10:20.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0070484609999999998 2020-08-07T15:10:20.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:20.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250705+0000", 2020-08-07T15:10:20.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0083e-05 2020-08-07T15:10:20.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:20.964 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250715+0000", 2020-08-07T15:10:20.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000086695 2020-08-07T15:10:20.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:20.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.250802+0000", 2020-08-07T15:10:20.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6980999999999999e-05 2020-08-07T15:10:20.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:20.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.250819+0000", 2020-08-07T15:10:20.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007248691 2020-08-07T15:10:20.968 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:20.968 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:20.969 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:20.969 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.969 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.969 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37058 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:10:20.969 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:10:20.969 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 86.589515950999996, 2020-08-07T15:10:20.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007368357, 2020-08-07T15:10:20.976 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:20.976 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:10:20.976 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:20.977 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:20.977 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:20.977 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37058 2020-08-07T15:10:20.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.977 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:20.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.978 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:20.978 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:10:20.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:20.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.979 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.979 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:20.979 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:10:20.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8838999999999998e-05 2020-08-07T15:10:20.979 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:20.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243761+0000", 2020-08-07T15:10:20.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2079999999999999e-06 2020-08-07T15:10:20.990 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.990 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.990 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:20.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243766+0000", 2020-08-07T15:10:20.991 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0567999999999999e-05 2020-08-07T15:10:20.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:20.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243777+0000", 2020-08-07T15:10:20.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7210000000000002e-06 2020-08-07T15:10:20.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:20.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:20.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:20.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243782+0000", 2020-08-07T15:10:21.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069508749999999996 2020-08-07T15:10:21.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250732+0000", 2020-08-07T15:10:21.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7899999999999999e-06 2020-08-07T15:10:21.012 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.013 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:21.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250737+0000", 2020-08-07T15:10:21.013 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003272409999999 2020-08-07T15:10:21.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.015 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.015 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.015 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251064+0000", 2020-08-07T15:10:21.015 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6115000000000001e-05 2020-08-07T15:10:21.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.016 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:21.016 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251081+0000", 2020-08-07T15:10:21.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007368357 2020-08-07T15:10:21.016 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.017 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:21.017 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.017 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.017 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37059 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:10:21.031 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:10:21.031 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 86.589272902000005, 2020-08-07T15:10:21.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0071655319999999, 2020-08-07T15:10:21.033 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:21.033 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:10:21.041 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:21.041 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:21.043 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:21.043 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37059 2020-08-07T15:10:21.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.054 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:21.054 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.054 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:21.070 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:10:21.071 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:21.071 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.071 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.114 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:21.114 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:10:21.114 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6533000000000002e-05 2020-08-07T15:10:21.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.115 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.115 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:21.118 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244012+0000", 2020-08-07T15:10:21.118 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3860000000000001e-06 2020-08-07T15:10:21.118 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.118 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.119 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:21.119 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244019+0000", 2020-08-07T15:10:21.119 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6982000000000001e-05 2020-08-07T15:10:21.119 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.119 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.120 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:21.120 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244036+0000", 2020-08-07T15:10:21.120 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0310000000000001e-06 2020-08-07T15:10:21.120 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.120 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.121 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:21.121 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244043+0000", 2020-08-07T15:10:21.121 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0067169259999999998 2020-08-07T15:10:21.121 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.121 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.122 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.122 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250760+0000", 2020-08-07T15:10:21.122 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4499999999999998e-06 2020-08-07T15:10:21.122 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.122 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.122 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:21.123 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250768+0000", 2020-08-07T15:10:21.123 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000341691 2020-08-07T15:10:21.123 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.123 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.123 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.124 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251109+0000", 2020-08-07T15:10:21.124 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1533e-05 2020-08-07T15:10:21.124 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.124 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.124 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:21.125 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251121+0000", 2020-08-07T15:10:21.125 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0071655319999999 2020-08-07T15:10:21.125 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.125 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:21.125 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.126 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.126 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.126 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37060 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:10:21.154 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:10:21.154 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 86.589148965000007, 2020-08-07T15:10:21.155 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070762280000001, 2020-08-07T15:10:21.155 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:21.155 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:10:21.155 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:21.155 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:21.156 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:21.156 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37060 2020-08-07T15:10:21.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.156 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:21.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.156 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:21.157 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:10:21.157 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:21.157 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.157 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.157 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:21.202 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:10:21.202 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5739000000000003e-05 2020-08-07T15:10:21.202 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.203 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.203 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:21.203 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244135+0000", 2020-08-07T15:10:21.203 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7740000000000001e-06 2020-08-07T15:10:21.203 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.204 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.204 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:21.204 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244141+0000", 2020-08-07T15:10:21.204 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3817e-05 2020-08-07T15:10:21.204 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.205 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.205 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:21.205 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244155+0000", 2020-08-07T15:10:21.205 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3829999999999997e-06 2020-08-07T15:10:21.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.254 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.254 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:21.254 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244161+0000", 2020-08-07T15:10:21.255 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0066303669999999999 2020-08-07T15:10:21.255 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.255 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.256 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.256 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250791+0000", 2020-08-07T15:10:21.256 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.392e-06 2020-08-07T15:10:21.256 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.256 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.257 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:21.257 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250798+0000", 2020-08-07T15:10:21.257 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003485860000001 2020-08-07T15:10:21.257 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.257 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.310 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.310 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251146+0000", 2020-08-07T15:10:21.310 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1700000000000003e-06 2020-08-07T15:10:21.310 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.311 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.311 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:21.311 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251155+0000", 2020-08-07T15:10:21.311 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070762280000001 2020-08-07T15:10:21.311 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.312 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:21.312 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.312 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.312 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.312 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37061 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:10:21.312 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:10:21.313 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 86.589034338000005, 2020-08-07T15:10:21.313 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070006920000001, 2020-08-07T15:10:21.313 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:21.313 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:10:21.327 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:21.328 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:21.328 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:21.332 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37061 2020-08-07T15:10:21.333 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.333 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:21.333 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.340 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:21.340 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:10:21.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:21.341 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.341 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:21.341 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:10:21.341 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5453e-05 2020-08-07T15:10:21.354 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.355 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.355 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:21.355 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244239+0000", 2020-08-07T15:10:21.355 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3179999999999997e-06 2020-08-07T15:10:21.355 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.356 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.362 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:21.366 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244245+0000", 2020-08-07T15:10:21.366 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2138e-05 2020-08-07T15:10:21.366 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.367 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:21.367 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244257+0000", 2020-08-07T15:10:21.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.69e-06 2020-08-07T15:10:21.368 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.368 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.368 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:21.368 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244261+0000", 2020-08-07T15:10:21.369 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0065598940000000001 2020-08-07T15:10:21.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.369 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.369 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250821+0000", 2020-08-07T15:10:21.393 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4799999999999998e-06 2020-08-07T15:10:21.420 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.420 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.420 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:21.420 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250828+0000", 2020-08-07T15:10:21.420 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003567310000001 2020-08-07T15:10:21.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.421 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.421 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.421 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251185+0000", 2020-08-07T15:10:21.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9879999999999999e-06 2020-08-07T15:10:21.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.430 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:21.430 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251195+0000", 2020-08-07T15:10:21.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070006920000001 2020-08-07T15:10:21.431 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.435 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:21.436 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.436 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.436 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.437 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37062 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:10:21.437 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:10:21.437 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 86.588934120000005, 2020-08-07T15:10:21.437 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006933662, 2020-08-07T15:10:21.437 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:21.438 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:10:21.438 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:21.438 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:21.438 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:21.438 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37062 2020-08-07T15:10:21.439 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.439 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:21.439 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.439 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:21.439 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:10:21.440 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:21.440 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.440 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:21.440 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:10:21.440 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4041e-05 2020-08-07T15:10:21.441 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.441 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:21.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244348+0000", 2020-08-07T15:10:21.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2079999999999997e-06 2020-08-07T15:10:21.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:21.456 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244354+0000", 2020-08-07T15:10:21.456 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1483e-05 2020-08-07T15:10:21.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:21.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244366+0000", 2020-08-07T15:10:21.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7989999999999999e-06 2020-08-07T15:10:21.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.486 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.487 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:21.487 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244372+0000", 2020-08-07T15:10:21.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0064783699999999998 2020-08-07T15:10:21.490 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.490 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.490 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.490 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250850+0000", 2020-08-07T15:10:21.490 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4729999999999997e-06 2020-08-07T15:10:21.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.491 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.491 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:21.491 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250856+0000", 2020-08-07T15:10:21.491 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003630240000001 2020-08-07T15:10:21.492 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.501 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.501 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.501 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251219+0000", 2020-08-07T15:10:21.501 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2640000000000008e-06 2020-08-07T15:10:21.502 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.502 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.502 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:21.502 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251228+0000", 2020-08-07T15:10:21.502 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006933662 2020-08-07T15:10:21.503 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.503 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:21.503 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.503 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.503 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.504 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37063 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:10:21.504 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:10:21.504 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 86.588821785999997, 2020-08-07T15:10:21.504 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006858051, 2020-08-07T15:10:21.504 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:21.505 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:10:21.505 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:21.505 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:21.505 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:21.505 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37063 2020-08-07T15:10:21.508 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.509 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:21.509 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.509 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:21.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:10:21.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:21.510 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.510 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:21.510 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:10:21.510 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5043000000000001e-05 2020-08-07T15:10:21.511 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:21.511 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244461+0000", 2020-08-07T15:10:21.511 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4600000000000002e-06 2020-08-07T15:10:21.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.512 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.512 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:21.512 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244467+0000", 2020-08-07T15:10:21.512 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1435e-05 2020-08-07T15:10:21.513 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.513 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.513 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:21.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244478+0000", 2020-08-07T15:10:21.513 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0190000000000003e-06 2020-08-07T15:10:21.514 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.514 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:21.514 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244484+0000", 2020-08-07T15:10:21.514 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006392895 2020-08-07T15:10:21.515 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.515 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.515 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.515 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250877+0000", 2020-08-07T15:10:21.515 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4280000000000001e-06 2020-08-07T15:10:21.516 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.516 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.516 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:21.516 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250884+0000", 2020-08-07T15:10:21.516 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000370414 2020-08-07T15:10:21.517 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.517 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.517 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.517 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251254+0000", 2020-08-07T15:10:21.517 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0356999999999999e-05 2020-08-07T15:10:21.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.518 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:21.528 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251264+0000", 2020-08-07T15:10:21.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006858051 2020-08-07T15:10:21.528 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.529 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:21.529 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.529 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.529 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.529 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37064 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:10:21.530 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:10:21.530 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 86.588709652999995, 2020-08-07T15:10:21.530 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0067798189999999, 2020-08-07T15:10:21.530 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:21.531 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:10:21.531 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:21.531 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:21.531 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:21.531 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37064 2020-08-07T15:10:21.531 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.532 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:21.532 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.532 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:21.532 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:10:21.532 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:21.533 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.533 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.533 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:21.533 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:10:21.533 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4007e-05 2020-08-07T15:10:21.534 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.534 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:21.535 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244573+0000", 2020-08-07T15:10:21.535 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.831e-06 2020-08-07T15:10:21.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:21.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244577+0000", 2020-08-07T15:10:21.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2269000000000001e-05 2020-08-07T15:10:21.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.536 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.537 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:21.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244590+0000", 2020-08-07T15:10:21.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7149999999999997e-06 2020-08-07T15:10:21.537 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.537 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.537 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:21.538 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244594+0000", 2020-08-07T15:10:21.538 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063105670000000004 2020-08-07T15:10:21.538 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.538 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.538 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.539 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250905+0000", 2020-08-07T15:10:21.539 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1970000000000001e-06 2020-08-07T15:10:21.539 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.539 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.539 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:21.540 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250911+0000", 2020-08-07T15:10:21.540 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003783020000001 2020-08-07T15:10:21.540 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.540 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.540 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.541 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251289+0000", 2020-08-07T15:10:21.541 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9309999999999997e-06 2020-08-07T15:10:21.541 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.541 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:21.542 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251298+0000", 2020-08-07T15:10:21.542 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0067798189999999 2020-08-07T15:10:21.542 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.542 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:21.542 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.543 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.543 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.543 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37065 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:10:21.543 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:10:21.543 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 86.588599434000002, 2020-08-07T15:10:21.543 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006708545, 2020-08-07T15:10:21.544 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:21.544 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:10:21.544 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:21.544 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:21.544 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:21.545 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37065 2020-08-07T15:10:21.545 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.545 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:21.545 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:21.546 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:10:21.546 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:21.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.546 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.546 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:21.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:10:21.547 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2750000000000001e-05 2020-08-07T15:10:21.547 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.547 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:21.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244681+0000", 2020-08-07T15:10:21.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0950000000000003e-06 2020-08-07T15:10:21.548 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.548 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:21.548 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244687+0000", 2020-08-07T15:10:21.549 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2485e-05 2020-08-07T15:10:21.549 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.549 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.549 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:21.550 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244699+0000", 2020-08-07T15:10:21.550 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8800000000000002e-06 2020-08-07T15:10:21.550 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.550 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.552 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:21.552 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244706+0000", 2020-08-07T15:10:21.552 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0062281569999999998 2020-08-07T15:10:21.552 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.553 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.553 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.553 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250934+0000", 2020-08-07T15:10:21.553 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1719999999999998e-06 2020-08-07T15:10:21.553 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.554 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.554 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:21.554 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250939+0000", 2020-08-07T15:10:21.554 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003861810000001 2020-08-07T15:10:21.554 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.554 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.555 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.555 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251325+0000", 2020-08-07T15:10:21.555 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1824999999999999e-05 2020-08-07T15:10:21.555 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.555 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:21.556 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251337+0000", 2020-08-07T15:10:21.556 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006708545 2020-08-07T15:10:21.556 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.556 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:21.557 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.557 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.557 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.557 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37066 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:10:21.558 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:10:21.558 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 86.583930464000005, 2020-08-07T15:10:21.558 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002077205, 2020-08-07T15:10:21.558 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:21.559 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:10:21.559 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:21.559 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:21.559 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:21.559 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37066 2020-08-07T15:10:21.559 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.560 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:21.560 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.560 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:21.560 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:10:21.560 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:21.561 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.561 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.561 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:21.561 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:10:21.561 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4654000000000002e-05 2020-08-07T15:10:21.562 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.562 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.562 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:21.562 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249352+0000", 2020-08-07T15:10:21.562 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6210000000000004e-06 2020-08-07T15:10:21.563 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.563 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.563 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:21.563 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249360+0000", 2020-08-07T15:10:21.563 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1895999999999998e-05 2020-08-07T15:10:21.563 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.564 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:21.564 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249382+0000", 2020-08-07T15:10:21.564 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1479999999999995e-06 2020-08-07T15:10:21.564 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.565 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.565 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:21.565 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249391+0000", 2020-08-07T15:10:21.569 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015731510000000001 2020-08-07T15:10:21.570 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.570 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.570 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.571 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250964+0000", 2020-08-07T15:10:21.571 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5400000000000001e-06 2020-08-07T15:10:21.571 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.571 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.571 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:21.572 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250971+0000", 2020-08-07T15:10:21.572 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000392379 2020-08-07T15:10:21.572 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.572 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.573 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.573 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251363+0000", 2020-08-07T15:10:21.573 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1816e-05 2020-08-07T15:10:21.573 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.573 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.574 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:21.574 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251375+0000", 2020-08-07T15:10:21.574 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002077205 2020-08-07T15:10:21.574 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.574 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:21.575 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.575 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.575 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37067 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:10:21.575 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:10:21.576 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 86.583789769999996, 2020-08-07T15:10:21.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0019761119999999, 2020-08-07T15:10:21.576 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:21.576 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:10:21.577 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:21.577 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:21.577 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:21.578 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37067 2020-08-07T15:10:21.578 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.578 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:21.578 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.578 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:21.579 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:10:21.579 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:21.579 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.579 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.579 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:21.580 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:10:21.580 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5961999999999997e-05 2020-08-07T15:10:21.581 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.582 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.582 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:21.582 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249494+0000", 2020-08-07T15:10:21.582 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3210000000000004e-06 2020-08-07T15:10:21.582 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.583 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.583 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:21.583 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249500+0000", 2020-08-07T15:10:21.583 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1012000000000001e-05 2020-08-07T15:10:21.583 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.584 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.584 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:21.584 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249511+0000", 2020-08-07T15:10:21.584 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.809e-06 2020-08-07T15:10:21.585 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.585 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.598 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:21.598 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249515+0000", 2020-08-07T15:10:21.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0014777670000000001 2020-08-07T15:10:21.599 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.599 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.599 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.600 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250993+0000", 2020-08-07T15:10:21.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9499999999999998e-06 2020-08-07T15:10:21.600 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.600 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.600 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:21.601 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250999+0000", 2020-08-07T15:10:21.601 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0004032519999999 2020-08-07T15:10:21.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.601 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.602 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251402+0000", 2020-08-07T15:10:21.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2038999999999999e-05 2020-08-07T15:10:21.602 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.602 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.602 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:21.603 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251414+0000", 2020-08-07T15:10:21.603 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0019761119999999 2020-08-07T15:10:21.603 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.603 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:21.603 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.604 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37068 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:10:21.604 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:10:21.604 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 86.581545199000004, 2020-08-07T15:10:21.605 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99977072, 2020-08-07T15:10:21.605 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:21.605 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:10:21.605 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:21.605 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:21.619 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:21.619 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37068 2020-08-07T15:10:21.620 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.620 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:21.620 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.620 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:21.620 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:10:21.621 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:21.621 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.621 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.621 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:21.621 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:10:21.622 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1369000000000003e-05 2020-08-07T15:10:21.622 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.622 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.622 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:21.622 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251724+0000", 2020-08-07T15:10:21.623 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.19e-06 2020-08-07T15:10:21.623 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.623 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.623 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:21.623 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251731+0000", 2020-08-07T15:10:21.623 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1368e-05 2020-08-07T15:10:21.624 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.624 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.624 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:21.624 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251742+0000", 2020-08-07T15:10:21.624 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.682e-06 2020-08-07T15:10:21.625 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.625 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.625 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:21.625 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251748+0000", 2020-08-07T15:10:21.625 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5275000000000003e-05 2020-08-07T15:10:21.628 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.628 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.628 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.629 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251793+0000", 2020-08-07T15:10:21.629 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5000000000000005e-06 2020-08-07T15:10:21.629 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.629 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.629 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:21.630 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251802+0000", 2020-08-07T15:10:21.630 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99963953500000002 2020-08-07T15:10:21.630 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.630 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.630 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.630 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251442+0000", 2020-08-07T15:10:21.631 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1800999999999999e-05 2020-08-07T15:10:21.631 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.631 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.631 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:21.631 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251454+0000", 2020-08-07T15:10:21.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99977072 2020-08-07T15:10:21.632 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.632 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:21.632 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.632 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.633 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.633 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37069 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:10:21.633 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:10:21.633 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 86.580441401000002, 2020-08-07T15:10:21.633 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99870607899999997, 2020-08-07T15:10:21.634 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:21.634 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:10:21.642 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:21.643 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:21.643 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:21.643 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37069 2020-08-07T15:10:21.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.643 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:21.650 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.650 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:21.650 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:10:21.650 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:21.650 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.651 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.651 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:21.651 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:10:21.651 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3755999999999999e-05 2020-08-07T15:10:21.651 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.652 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.652 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:21.652 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252831+0000", 2020-08-07T15:10:21.652 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7919999999999998e-06 2020-08-07T15:10:21.652 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.653 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.653 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:21.653 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252836+0000", 2020-08-07T15:10:21.653 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1409999999999999e-05 2020-08-07T15:10:21.653 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.653 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.654 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:21.654 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252848+0000", 2020-08-07T15:10:21.654 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6380000000000001e-06 2020-08-07T15:10:21.654 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.654 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.655 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:21.655 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252853+0000", 2020-08-07T15:10:21.655 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3244e-05 2020-08-07T15:10:21.666 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.667 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252907+0000", 2020-08-07T15:10:21.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1470000000000007e-06 2020-08-07T15:10:21.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:21.668 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252916+0000", 2020-08-07T15:10:21.668 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99856497499999997 2020-08-07T15:10:21.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251481+0000", 2020-08-07T15:10:21.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2116999999999999e-05 2020-08-07T15:10:21.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.670 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.670 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:21.670 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251493+0000", 2020-08-07T15:10:21.670 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99870607899999997 2020-08-07T15:10:21.671 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.671 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:21.671 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.671 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.671 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.672 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37070 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:10:21.672 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:10:21.672 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 86.580337138000004, 2020-08-07T15:10:21.672 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99863946699999995, 2020-08-07T15:10:21.672 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:21.673 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:10:21.673 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:21.673 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:21.673 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:21.673 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37070 2020-08-07T15:10:21.674 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.674 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:21.674 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.674 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:21.674 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:10:21.674 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:21.675 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.675 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.675 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:21.675 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:10:21.675 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9984000000000001e-05 2020-08-07T15:10:21.676 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.676 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.676 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:21.677 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252941+0000", 2020-08-07T15:10:21.677 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7010000000000004e-06 2020-08-07T15:10:21.677 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:21.678 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252946+0000", 2020-08-07T15:10:21.678 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1144999999999999e-05 2020-08-07T15:10:21.678 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.678 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.678 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:21.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252957+0000", 2020-08-07T15:10:21.679 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6920000000000001e-06 2020-08-07T15:10:21.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.679 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:21.680 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252962+0000", 2020-08-07T15:10:21.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6739000000000002e-05 2020-08-07T15:10:21.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.680 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.681 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252998+0000", 2020-08-07T15:10:21.681 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.503e-06 2020-08-07T15:10:21.681 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.681 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:21.681 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.253006+0000", 2020-08-07T15:10:21.682 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99851375499999995 2020-08-07T15:10:21.682 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.682 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.682 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.682 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251520+0000", 2020-08-07T15:10:21.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0947999999999999e-05 2020-08-07T15:10:21.699 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.699 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.699 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:21.700 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251530+0000", 2020-08-07T15:10:21.700 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99863946699999995 2020-08-07T15:10:21.700 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.700 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:21.701 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.701 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.701 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42505 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:10:21.701 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:10:21.702 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 80.530996607999995, 2020-08-07T15:10:21.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0024736009999999, 2020-08-07T15:10:21.702 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:21.702 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:21.702 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:21.703 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:21.703 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:21.703 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42505 2020-08-07T15:10:21.703 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.703 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:21.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.704 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:21.704 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:10:21.704 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:21.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.705 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:21.705 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:10:21.705 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011052600000000001 2020-08-07T15:10:21.705 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.714 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.714 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:21.715 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302342+0000", 2020-08-07T15:10:21.715 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.781e-06 2020-08-07T15:10:21.715 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.715 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.715 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:21.716 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302352+0000", 2020-08-07T15:10:21.716 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7646e-05 2020-08-07T15:10:21.716 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.716 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.716 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:21.717 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302369+0000", 2020-08-07T15:10:21.717 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3629999999999994e-06 2020-08-07T15:10:21.717 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.717 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.717 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:21.718 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302378+0000", 2020-08-07T15:10:21.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8007000000000003e-05 2020-08-07T15:10:21.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.718 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.718 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302426+0000", 2020-08-07T15:10:21.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5159999999999994e-06 2020-08-07T15:10:21.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:21.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302434+0000", 2020-08-07T15:10:21.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99151562699999995 2020-08-07T15:10:21.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.293950+0000", 2020-08-07T15:10:21.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020138099999999999 2020-08-07T15:10:21.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.721 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:21.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294151+0000", 2020-08-07T15:10:21.722 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024761 2020-08-07T15:10:21.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.727 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:10:21.728 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294399+0000", 2020-08-07T15:10:21.728 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0027714369999999999 2020-08-07T15:10:21.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.729 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:21.729 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297170+0000", 2020-08-07T15:10:21.729 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.234e-06 2020-08-07T15:10:21.729 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.729 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.730 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:21.730 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297172+0000", 2020-08-07T15:10:21.730 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5384999999999998e-05 2020-08-07T15:10:21.730 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.730 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.731 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:21.731 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297237+0000", 2020-08-07T15:10:21.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0073576980000000002 2020-08-07T15:10:21.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:21.732 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304595+0000", 2020-08-07T15:10:21.732 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1006000000000001e-05 2020-08-07T15:10:21.746 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.746 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:21.746 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304626+0000", 2020-08-07T15:10:21.747 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6000000000000003e-07 2020-08-07T15:10:21.747 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.747 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.747 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:21.747 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304626+0000", 2020-08-07T15:10:21.748 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8724e-05 2020-08-07T15:10:21.748 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.748 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.748 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:21.748 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304705+0000", 2020-08-07T15:10:21.749 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0024736009999999 2020-08-07T15:10:21.749 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.749 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:21.749 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.749 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.750 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.750 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42506 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:10:21.750 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:10:21.750 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 80.530805395000002, 2020-08-07T15:10:21.750 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002465226, 2020-08-07T15:10:21.751 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:21.751 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:21.751 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:21.751 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:21.751 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:21.752 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42506 2020-08-07T15:10:21.752 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.752 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:21.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.752 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:21.753 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:10:21.753 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:21.753 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.753 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.753 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:21.753 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:10:21.754 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8780999999999999e-05 2020-08-07T15:10:21.754 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.754 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.754 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:21.754 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302472+0000", 2020-08-07T15:10:21.755 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3740000000000003e-06 2020-08-07T15:10:21.755 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.755 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:21.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302478+0000", 2020-08-07T15:10:21.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1155000000000001e-05 2020-08-07T15:10:21.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:21.756 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302489+0000", 2020-08-07T15:10:21.757 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7559999999999996e-06 2020-08-07T15:10:21.757 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.757 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:21.757 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302495+0000", 2020-08-07T15:10:21.768 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4128000000000003e-05 2020-08-07T15:10:21.769 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.769 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.769 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.769 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302529+0000", 2020-08-07T15:10:21.770 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.516e-06 2020-08-07T15:10:21.770 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.770 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.770 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:21.770 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302535+0000", 2020-08-07T15:10:21.771 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99240899299999996 2020-08-07T15:10:21.771 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.771 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.771 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.771 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294944+0000", 2020-08-07T15:10:21.772 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000115806 2020-08-07T15:10:21.772 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.772 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.772 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:21.772 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295060+0000", 2020-08-07T15:10:21.773 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000205002 2020-08-07T15:10:21.773 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.773 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.773 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:10:21.773 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295265+0000", 2020-08-07T15:10:21.774 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043414400000000002 2020-08-07T15:10:21.774 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.774 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.774 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:21.775 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299607+0000", 2020-08-07T15:10:21.775 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2229999999999998e-06 2020-08-07T15:10:21.775 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.775 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:21.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299612+0000", 2020-08-07T15:10:21.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5797000000000002e-05 2020-08-07T15:10:21.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:21.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299658+0000", 2020-08-07T15:10:21.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005095044 2020-08-07T15:10:21.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:21.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304753+0000", 2020-08-07T15:10:21.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3121999999999996e-05 2020-08-07T15:10:21.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:21.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304816+0000", 2020-08-07T15:10:21.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5700000000000001e-07 2020-08-07T15:10:21.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:21.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304817+0000", 2020-08-07T15:10:21.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1231999999999998e-05 2020-08-07T15:10:21.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:21.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304888+0000", 2020-08-07T15:10:21.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002465226 2020-08-07T15:10:21.806 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.807 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:21.807 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.807 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42507 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:10:21.808 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:10:21.808 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 80.530694624999995, 2020-08-07T15:10:21.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002567097, 2020-08-07T15:10:21.808 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:21.808 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:21.809 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:21.809 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:21.809 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:21.809 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42507 2020-08-07T15:10:21.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.829 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:21.830 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.830 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:21.830 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:10:21.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:21.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:21.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:10:21.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7679000000000002e-05 2020-08-07T15:10:21.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.838 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:21.838 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302581+0000", 2020-08-07T15:10:21.838 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2299999999999999e-06 2020-08-07T15:10:21.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.839 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:21.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302586+0000", 2020-08-07T15:10:21.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0886999999999999e-05 2020-08-07T15:10:21.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.840 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:21.840 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302597+0000", 2020-08-07T15:10:21.840 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4129999999999997e-06 2020-08-07T15:10:21.841 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.842 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:21.842 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302603+0000", 2020-08-07T15:10:21.842 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3963000000000001e-05 2020-08-07T15:10:21.842 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.842 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.843 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302627+0000", 2020-08-07T15:10:21.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.33e-06 2020-08-07T15:10:21.854 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.854 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.855 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:21.855 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302631+0000", 2020-08-07T15:10:21.855 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99319860900000001 2020-08-07T15:10:21.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.856 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.856 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295830+0000", 2020-08-07T15:10:21.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000116012 2020-08-07T15:10:21.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.856 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:21.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295946+0000", 2020-08-07T15:10:21.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000230242 2020-08-07T15:10:21.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.868 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:10:21.869 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296176+0000", 2020-08-07T15:10:21.869 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003494455 2020-08-07T15:10:21.869 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.869 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.870 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:21.870 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299670+0000", 2020-08-07T15:10:21.870 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0299999999999998e-07 2020-08-07T15:10:21.871 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.871 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:21.871 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299671+0000", 2020-08-07T15:10:21.871 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7298000000000001e-05 2020-08-07T15:10:21.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.872 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.872 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:21.872 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299708+0000", 2020-08-07T15:10:21.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0052628839999999998 2020-08-07T15:10:21.873 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.873 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:21.873 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304971+0000", 2020-08-07T15:10:21.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3754e-05 2020-08-07T15:10:21.879 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.879 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:21.879 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305035+0000", 2020-08-07T15:10:21.880 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9500000000000001e-07 2020-08-07T15:10:21.880 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.880 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:21.880 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305036+0000", 2020-08-07T15:10:21.881 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4843000000000002e-05 2020-08-07T15:10:21.881 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.881 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.881 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:21.881 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305101+0000", 2020-08-07T15:10:21.895 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002567097 2020-08-07T15:10:21.895 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.895 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:21.896 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.901 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.901 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42508 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:10:21.903 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:10:21.903 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 80.520169605999996, 2020-08-07T15:10:21.903 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99221695600000004, 2020-08-07T15:10:21.903 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:21.904 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:21.904 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:21.904 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:21.904 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:21.904 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42508 2020-08-07T15:10:21.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.905 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:21.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:21.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:10:21.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:21.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.906 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.906 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:21.906 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:10:21.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3692e-05 2020-08-07T15:10:21.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:21.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313112+0000", 2020-08-07T15:10:21.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7399999999999998e-06 2020-08-07T15:10:21.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.908 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:21.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313119+0000", 2020-08-07T15:10:21.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5153999999999999e-05 2020-08-07T15:10:21.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:21.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313134+0000", 2020-08-07T15:10:21.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3039999999999999e-06 2020-08-07T15:10:21.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.910 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:21.910 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313141+0000", 2020-08-07T15:10:21.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6516000000000001e-05 2020-08-07T15:10:21.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313178+0000", 2020-08-07T15:10:21.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0949999999999998e-06 2020-08-07T15:10:21.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:21.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313185+0000", 2020-08-07T15:10:21.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.98342834999999995 2020-08-07T15:10:21.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:21.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296613+0000", 2020-08-07T15:10:21.914 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2230999999999993e-05 2020-08-07T15:10:21.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:21.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296686+0000", 2020-08-07T15:10:21.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013121300000000001 2020-08-07T15:10:21.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:10:21.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296817+0000", 2020-08-07T15:10:21.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044228050000000001 2020-08-07T15:10:21.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:21.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301240+0000", 2020-08-07T15:10:21.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0500000000000002e-07 2020-08-07T15:10:21.925 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.926 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:21.926 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301241+0000", 2020-08-07T15:10:21.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2434000000000006e-05 2020-08-07T15:10:21.926 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:21.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301303+0000", 2020-08-07T15:10:21.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038622919999999998 2020-08-07T15:10:21.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:21.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305165+0000", 2020-08-07T15:10:21.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8012000000000001e-05 2020-08-07T15:10:21.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:21.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305203+0000", 2020-08-07T15:10:21.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5299999999999999e-07 2020-08-07T15:10:21.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:21.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305204+0000", 2020-08-07T15:10:21.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1760000000000004e-05 2020-08-07T15:10:21.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:21.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:21.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:21.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305275+0000", 2020-08-07T15:10:21.931 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99221695600000004 2020-08-07T15:10:21.932 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.932 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:21.932 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.932 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:21.932 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:21.932 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:21.933 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 1 15 1682 1667 13.0211 13.0234 0.00617334 0.00947367 2020-08-07T15:10:21.934 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:10:21.935 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2020-08-07T15:10:20.658035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":135,"acting":135,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":93,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8396072,"kb_used_data":7400,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746578648,"statfs":{"total":773094113280,"available":764496535552,"internally_reserved":8589934592,"allocated":7577600,"data_stored":2957724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.413755"},"pg_stats":[{"pgid":"1.14","version":"16'2","reported_seq":"97","reported_epoch":"164","state":"active+clean","last_fresh":"2020-08-07T15:10:15.308204+0000","last_change":"2020-08-07T15:10:10.644555+0000","last_active":"2020-08-07T15:10:15.308204+0000","last_peered":"2020-08-07T15:10:15.308204+0000","last_clean":"2020-08-07T15:10:15.308204+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:10:15.308204+0000","last_undegraded":"2020-08-07T15:10:15.308204+0000","last_fullsized":"2020-08-07T15:10:15.308204+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:10.644460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.644460+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"166","reported_epoch":"164","state":"active+clean","last_fresh":"2020-08-07T15:10:15.246664+0000","last_change":"2020-08-07T15:10:07.295971+0000","last_active":"2020-08-07T15:10:15.246664+0000","last_peered":"2020-08-07T15:10:15.246664+0000","last_clean":"2020-08-07T15:10:15.246664+0000","last_became_active":"2020-08-07T15:09:30.203592+0000","last_became_peered":"2020-08-07T15:09:30.203592+0000","last_unstale":"2020-08-07T15:10:15.246664+0000","last_undegraded":"2020-08-07T15:10:15.246664+0000","last_fullsized":"2020-08-07T15:10:15.246664+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:07.295863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:07.295863+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"94","reported_epoch":"164","state":"active+clean","last_fresh":"2020-08-07T15:10:15.244582+0000","last_change":"2020-08-07T15:08:55.078015+0000","last_active":"2020-08-07T15:10:15.244582+0000","last_peered":"2020-08-07T15:10:15.244582+0000","last_clean":"2020-08-07T15:10:15.244582+0000","last_became_active":"2020-08-07T15:08:41.685106+0000","last_became_peered":"2020-08-07T15:08:41.685106+0000","last_unstale":"2020-08-07T15:10:15.244582+0000","last_undegraded":"2020-08-07T15:10:15.244582+0000","last_fullsized":"2020-08-07T15:10:15.244582+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:55.077924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:55.077924+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"207","reported_epoch":"164","state":"active+clean","last_fresh":"2020-08-07T15:10:15.244381+0000","last_change":"2020-08-07T15:09:30.210669+0000","last_active":"2020-08-07T15:10:15.244381+0000","last_peered":"2020-08-07T15:10:15.244381+0000","last_clean":"2020-08-07T15:10:15.244381+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:10:15.244381+0000","last_undegraded":"2020-08-07T15:10:15.244381+0000","last_fullsized":"2020-08-07T15:10:15.244381+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.544324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.544324+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"201","reported_epoch":"164","state":"active+clean","last_fresh":"2020-08-07T15:10:15.244181+0000","last_change":"2020-08-07T15:08:51.599032+0000","last_active":"2020-08-07T15:10:15.244181+0000","last_peered":"2020-08-07T15:10:15.244181+0000","last_clean":"2020-08-07T15:10:15.244181+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:10:15.244181+0000","last_undegraded":"2020-08-07T15:10:15.244181+0000","last_fullsized":"2020-08-07T15:10:15.244181+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:51.598947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.598947+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":"9","reported_epoch":"165","state":"active+clean","last_fresh":"2020-08-07T15:10:16.261122+0000","last_change":"2020-08-07T15:10:16.261122+0000","last_active":"2020-08-07T15:10:16.261122+0000","last_peered":"2020-08-07T15:10:16.261122+0000","last_clean":"2020-08-07T15:10:16.261122+0000","last_became_active":"2020-08-07T15:10:16.260690+0000","last_became_peered":"2020-08-07T15:10:16.260690+0000","last_unstale":"2020-08-07T15:10:16.261122+0000","last_undegraded":"2020-08-07T15:10:16.261122+0000","last_fullsized":"2020-08-07T15:10:16.261122+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"162","reported_epoch":"165","state":"active+clean","last_fresh":"2020-08-07T15:10:16.249198+0000","last_change":"2020-08-07T15:09:41.284321+0000","last_active":"2020-08-07T15:10:16.249198+0000","last_peered":"2020-08-07T15:10:16.249198+0000","last_clean":"2020-08-07T15:10:16.249198+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:10:16.249198+0000","last_undegraded":"2020-08-07T15:10:16.249198+0000","last_fullsized":"2020-08-07T15:10:16.249198+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:41.284237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:41.284237+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"77","reported_epoch":"164","state":"active+clean","last_fresh":"2020-08-07T15:10:15.303934+0000","last_change":"2020-08-07T15:09:30.283753+0000","last_active":"2020-08-07T15:10:15.303934+0000","last_peered":"2020-08-07T15:10:15.303934+0000","last_clean":"2020-08-07T15:10:15.303934+0000","last_became_active":"2020-08-07T15:09:30.283176+0000","last_became_peered":"2020-08-07T15:09:30.283176+0000","last_unstale":"2020-08-07T15:10:15.303934+0000","last_undegraded":"2020-08-07T15:10:15.303934+0000","last_fullsized":"2020-08-07T15:10:15.303934+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:25.606757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:25.606757+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":"9","reported_epoch":"165","state":"active+clean","last_fresh":"2020-08-07T15:10:16.259807+0000","last_change":"2020-08-07T15:10:16.259807+0000","last_active":"2020-08-07T15:10:16.259807+0000","last_peered":"2020-08-07T15:10:16.259807+0000","last_clean":"2020-08-07T15:10:16.259807+0000","last_became_active":"2020-08-07T15:10:16.259365+0000","last_became_peered":"2020-08-07T15:10:16.259365+0000","last_unstale":"2020-08-07T15:10:16.259807+0000","last_undegraded":"2020-08-07T15:10:16.259807+0000","last_fullsized":"2020-08-07T15:10:16.259807+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"161","reported_epoch":"164","state":"active+clean","last_fresh":"2020-08-07T15:10:15.246875+0000","last_change":"2020-08-07T15:08:40.057705+0000","last_active":"2020-08-07T15:10:15.246875+0000","last_peered":"2020-08-07T15:10:15.246875+0000","last_clean":"2020-08-07T15:10:15.246875+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:10:15.246875+0000","last_undegraded":"2020-08-07T15:10:15.246875+0000","last_fullsized":"2020-08-07T15:10:15.246875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:40.057622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:40.057622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"119","reported_epoch":"164","state":"active+clean","last_fresh":"2020-08-07T15:10:15.308290+0000","last_change":"2020-08-07T15:08:53.756554+0000","last_active":"2020-08-07T15:10:15.308290+0000","last_peered":"2020-08-07T15:10:15.308290+0000","last_clean":"2020-08-07T15:10:15.308290+0000","last_became_active":"2020-08-07T15:08:46.710404+0000","last_became_peered":"2020-08-07T15:08:46.710404+0000","last_unstale":"2020-08-07T15:10:15.308290+0000","last_undegraded":"2020-08-07T15:10:15.308290+0000","last_fullsized":"2020-08-07T15:10:15.308290+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:53.756343+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:53.756343+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":"10","reported_epoch":"165","state":"active+clean","last_fresh":"2020-08-07T15:10:16.449777+0000","last_change":"2020-08-07T15:10:16.448360+0000","last_active":"2020-08-07T15:10:16.449777+0000","last_peered":"2020-08-07T15:10:16.449777+0000","last_clean":"2020-08-07T15:10:16.449777+0000","last_became_active":"2020-08-07T15:10:16.447826+0000","last_became_peered":"2020-08-07T15:10:16.447826+0000","last_unstale":"2020-08-07T15:10:16.449777+0000","last_undegraded":"2020-08-07T15:10:16.449777+0000","last_fullsized":"2020-08-07T15:10:16.449777+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"162","reported_epoch":"164","state":"active+clean","last_fresh":"2020-08-07T15:10:15.306243+0000","last_change":"2020-08-07T15:09:54.908283+0000","last_active":"2020-08-07T15:10:15.306243+0000","last_peered":"2020-08-07T15:10:15.306243+0000","last_clean":"2020-08-07T15:10:15.306243+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:10:15.306243+0000","last_undegraded":"2020-08-07T15:10:15.306243+0000","last_fullsized":"2020-08-07T15:10:15.306243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:54.908197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:54.908197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"201","reported_epoch":"165","state":"active+clean","last_fresh":"2020-08-07T15:10:16.249222+0000","last_change":"2020-08-07T15:10:07.302258+0000","last_active":"2020-08-07T15:10:16.249222+0000","last_peered":"2020-08-07T15:10:16.249222+0000","last_clean":"2020-08-07T15:10:16.249222+0000","last_became_active":"2020-08-07T15:09:30.201829+0000","last_became_peered":"2020-08-07T15:09:30.201829+0000","last_unstale":"2020-08-07T15:10:16.249222+0000","last_undegraded":"2020-08-07T15:10:16.249222+0000","last_fullsized":"2020-08-07T15:10:16.249222+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:07.302204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:07.302204+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":"9","reported_epoch":"165","state":"active+clean","last_fresh":"2020-08-07T15:10:16.447819+0000","last_change":"2020-08-07T15:10:16.447819+0000","last_active":"2020-08-07T15:10:16.447819+0000","last_peered":"2020-08-07T15:10:16.447819+0000","last_clean":"2020-08-07T15:10:16.447819+0000","last_became_active":"2020-08-07T15:10:16.447312+0000","last_became_peered":"2020-08-07T15:10:16.447312+0000","last_unstale":"2020-08-07T15:10:16.447819+0000","last_undegraded":"2020-08-07T15:10:16.447819+0000","last_fullsized":"2020-08-07T15:10:16.447819+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"162","reported_epoch":"164","state":"active+clean","last_fresh":"2020-08-07T15:10:15.305811+0000","last_change":"2020-08-07T15:09:51.036281+0000","last_active":"2020-08-07T15:10:15.305811+0000","last_peered":"2020-08-07T15:10:15.305811+0000","last_clean":"2020-08-07T15:10:15.305811+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:10:15.305811+0000","last_undegraded":"2020-08-07T15:10:15.305811+0000","last_fullsized":"2020-08-07T15:10:15.305811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:51.036224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:51.036224+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"164","reported_epoch":"165","state":"active+clean","last_fresh":"2020-08-07T15:10:16.249358+0000","last_change":"2020-08-07T15:08:50.686578+0000","last_active":"2020-08-07T15:10:16.249358+0000","last_peered":"2020-08-07T15:10:16.249358+0000","last_clean":"2020-08-07T15:10:16.249358+0000","last_became_active":"2020-08-07T15:08:36.633406+0000","last_became_peered":"2020-08-07T15:08:36.633406+0000","last_unstale":"2020-08-07T15:10:16.249358+0000","last_undegraded":"2020-08-07T15:10:16.249358+0000","last_fullsized":"2020-08-07T15:10:16.249358+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:50.686496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:50.686496+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":"9","reported_epoch":"165","state":"active+clean","last_fresh":"2020-08-07T15:10:16.255436+0000","last_change":"2020-08-07T15:10:16.255436+0000","last_active":"2020-08-07T15:10:16.255436+0000","last_peered":"2020-08-07T15:10:16.255436+0000","last_clean":"2020-08-07T15:10:16.255436+0000","last_became_active":"2020-08-07T15:10:16.254967+0000","last_became_peered":"2020-08-07T15:10:16.254967+0000","last_unstale":"2020-08-07T15:10:16.255436+0000","last_undegraded":"2020-08-07T15:10:16.255436+0000","last_fullsized":"2020-08-07T15:10:16.255436+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"194","reported_epoch":"164","state":"active+clean","last_fresh":"2020-08-07T15:10:15.305773+0000","last_change":"2020-08-07T15:09:55.879867+0000","last_active":"2020-08-07T15:10:15.305773+0000","last_peered":"2020-08-07T15:10:15.305773+0000","last_clean":"2020-08-07T15:10:15.305773+0000","last_became_active":"2020-08-07T15:08:36.627426+0000","last_became_peered":"2020-08-07T15:08:36.627426+0000","last_unstale":"2020-08-07T15:10:15.305773+0000","last_undegraded":"2020-08-07T15:10:15.305773+0000","last_fullsized":"2020-08-07T15:10:15.305773+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:55.879785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:55.879785+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"162","reported_epoch":"164","state":"active+clean","last_fresh":"2020-08-07T15:10:15.244644+0000","last_change":"2020-08-07T15:10:08.656952+0000","last_active":"2020-08-07T15:10:15.244644+0000","last_peered":"2020-08-07T15:10:15.244644+0000","last_clean":"2020-08-07T15:10:15.244644+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:10:15.244644+0000","last_undegraded":"2020-08-07T15:10:15.244644+0000","last_fullsized":"2020-08-07T15:10:15.244644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:08.656855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:10:08.656855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":"9","reported_epoch":"165","state":"active+clean","last_fresh":"2020-08-07T15:10:16.263356+0000","last_change":"2020-08-07T15:10:16.263356+0000","last_active":"2020-08-07T15:10:16.263356+0000","last_peered":"2020-08-07T15:10:16.263356+0000","last_clean":"2020-08-07T15:10:16.263356+0000","last_became_active":"2020-08-07T15:10:16.262971+0000","last_became_peered":"2020-08-07T15:10:16.262971+0000","last_unstale":"2020-08-07T15:10:16.263356+0000","last_undegraded":"2020-08-07T15:10:16.263356+0000","last_fullsized":"2020-08-07T15:10:16.263356+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"180","reported_epoch":"164","state":"active+clean","last_fresh":"2020-08-07T15:10:15.307956+0000","last_change":"2020-08-07T15:09:27.181676+0000","last_active":"2020-08-07T15:10:15.307956+0000","last_peered":"2020-08-07T15:10:15.307956+0000","last_clean":"2020-08-07T15:10:15.307956+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:10:15.307956+0000","last_undegraded":"2020-08-07T15:10:15.307956+0000","last_fullsized":"2020-08-07T15:10:15.307956+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:05.606603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:05.606603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"210","reported_epoch":"164","state":"active+clean","last_fresh":"2020-08-07T15:10:15.305840+0000","last_change":"2020-08-07T15:10:11.912614+0000","last_active":"2020-08-07T15:10:15.305840+0000","last_peered":"2020-08-07T15:10:15.305840+0000","last_clean":"2020-08-07T15:10:15.305840+0000","last_became_active":"2020-08-07T15:08:36.628365+0000","last_became_peered":"2020-08-07T15:08:36.628365+0000","last_unstale":"2020-08-07T15:10:15.305840+0000","last_undegraded":"2020-08-07T15:10:15.305840+0000","last_fullsized":"2020-08-07T15:10:15.305840+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:11.912550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:11.912550+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":"9","reported_epoch":"165","state":"active+clean","last_fresh":"2020-08-07T15:10:16.447870+0000","last_change":"2020-08-07T15:10:16.447870+0000","last_active":"2020-08-07T15:10:16.447870+0000","last_peered":"2020-08-07T15:10:16.447870+0000","last_clean":"2020-08-07T15:10:16.447870+0000","last_became_active":"2020-08-07T15:10:16.447478+0000","last_became_peered":"2020-08-07T15:10:16.447478+0000","last_unstale":"2020-08-07T15:10:16.447870+0000","last_undegraded":"2020-08-07T15:10:16.447870+0000","last_fullsized":"2020-08-07T15:10:16.447870+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"162","reported_epoch":"164","state":"active+clean","last_fresh":"2020-08-07T15:10:15.244776+0000","last_change":"2020-08-07T15:09:52.637867+0000","last_active":"2020-08-07T15:10:15.244776+0000","last_peered":"2020-08-07T15:10:15.244776+0000","last_clean":"2020-08-07T15:10:15.244776+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:10:15.244776+0000","last_undegraded":"2020-08-07T15:10:15.244776+0000","last_fullsized":"2020-08-07T15:10:15.244776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:52.637791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:09:52.637791+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"160","reported_epoch":"164","state":"active+clean","last_fresh":"2020-08-07T15:10:15.244476+0000","last_change":"2020-08-07T15:10:11.720217+0000","last_active":"2020-08-07T15:10:15.244476+0000","last_peered":"2020-08-07T15:10:15.244476+0000","last_clean":"2020-08-07T15:10:15.244476+0000","last_became_active":"2020-08-07T15:08:36.618164+0000","last_became_peered":"2020-08-07T15:08:36.618164+0000","last_unstale":"2020-08-07T15:10:15.244476+0000","last_undegraded":"2020-08-07T15:10:15.244476+0000","last_fullsized":"2020-08-07T15:10:15.244476+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:11.720143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:11.720143+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":"10","reported_epoch":"165","state":"active+clean","last_fresh":"2020-08-07T15:10:16.444006+0000","last_change":"2020-08-07T15:10:16.443153+0000","last_active":"2020-08-07T15:10:16.444006+0000","last_peered":"2020-08-07T15:10:16.444006+0000","last_clean":"2020-08-07T15:10:16.444006+0000","last_became_active":"2020-08-07T15:10:16.442674+0000","last_became_peered":"2020-08-07T15:10:16.442674+0000","last_unstale":"2020-08-07T15:10:16.444006+0000","last_undegraded":"2020-08-07T15:10:16.444006+0000","last_fullsized":"2020-08-07T15:10:16.444006+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"159","reported_epoch":"164","state":"active+clean","last_fresh":"2020-08-07T15:10:15.306168+0000","last_change":"2020-08-07T15:09:37.017383+0000","last_active":"2020-08-07T15:10:15.306168+0000","last_peered":"2020-08-07T15:10:15.306168+0000","last_clean":"2020-08-07T15:10:15.306168+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:10:15.306168+0000","last_undegraded":"2020-08-07T15:10:15.306168+0000","last_fullsized":"2020-08-07T15:10:15.306168+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:37.017307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:37.017307+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"232","reported_epoch":"165","state":"active+clean","last_fresh":"2020-08-07T15:10:16.249267+0000","last_change":"2020-08-07T15:09:30.208873+0000","last_active":"2020-08-07T15:10:16.249267+0000","last_peered":"2020-08-07T15:10:16.249267+0000","last_clean":"2020-08-07T15:10:16.249267+0000","last_became_active":"2020-08-07T15:09:30.208345+0000","last_became_peered":"2020-08-07T15:09:30.208345+0000","last_unstale":"2020-08-07T15:10:16.249267+0000","last_undegraded":"2020-08-07T15:10:16.249267+0000","last_fullsized":"2020-08-07T15:10:16.249267+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.381325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.381325+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":"9","reported_epoch":"165","state":"active+clean","last_fresh":"2020-08-07T15:10:16.448334+0000","last_change":"2020-08-07T15:10:16.448334+0000","last_active":"2020-08-07T15:10:16.448334+0000","last_peered":"2020-08-07T15:10:16.448334+0000","last_clean":"2020-08-07T15:10:16.448334+0000","last_became_active":"2020-08-07T15:10:16.447541+0000","last_became_peered":"2020-08-07T15:10:16.447541+0000","last_unstale":"2020-08-07T15:10:16.448334+0000","last_undegraded":"2020-08-07T15:10:16.448334+0000","last_fullsized":"2020-08-07T15:10:16.448334+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"137","reported_epoch":"164","state":"active+clean","last_fresh":"2020-08-07T15:10:15.244675+0000","last_change":"2020-08-07T15:10:10.715147+0000","last_active":"2020-08-07T15:10:15.244675+0000","last_peered":"2020-08-07T15:10:15.244675+0000","last_clean":"2020-08-07T15:10:15.244675+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:10:15.244675+0000","last_undegraded":"2020-08-07T15:10:15.244675+0000","last_fullsized":"2020-08-07T15:10:15.244675+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.715051+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":"9","reported_epoch":"165","state":"active+clean","last_fresh":"2020-08-07T15:10:16.448023+0000","last_change":"2020-08-07T15:10:16.448023+0000","last_active":"2020-08-07T15:10:16.448023+0000","last_peered":"2020-08-07T15:10:16.448023+0000","last_clean":"2020-08-07T15:10:16.448023+0000","last_became_active":"2020-08-07T15:10:16.447526+0000","last_became_peered":"2020-08-07T15:10:16.447526+0000","last_unstale":"2020-08-07T15:10:16.448023+0000","last_undegraded":"2020-08-07T15:10:16.448023+0000","last_fullsized":"2020-08-07T15:10:16.448023+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"134","reported_epoch":"164","state":"active+clean","last_fresh":"2020-08-07T15:10:15.244842+0000","last_change":"2020-08-07T15:09:09.691838+0000","last_active":"2020-08-07T15:10:15.244842+0000","last_peered":"2020-08-07T15:10:15.244842+0000","last_clean":"2020-08-07T15:10:15.244842+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:10:15.244842+0000","last_undegraded":"2020-08-07T15:10:15.244842+0000","last_fullsized":"2020-08-07T15:10:15.244842+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.691753+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":"9","reported_epoch":"165","state":"active+clean","last_fresh":"2020-08-07T15:10:16.263289+0000","last_change":"2020-08-07T15:10:16.263289+0000","last_active":"2020-08-07T15:10:16.263289+0000","last_peered":"2020-08-07T15:10:16.263289+0000","last_clean":"2020-08-07T15:10:16.263289+0000","last_became_active":"2020-08-07T15:10:16.262869+0000","last_became_peered":"2020-08-07T15:10:16.262869+0000","last_unstale":"2020-08-07T15:10:16.263289+0000","last_undegraded":"2020-08-07T15:10:16.263289+0000","last_fullsized":"2020-08-07T15:10:16.263289+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"198","reported_epoch":"164","state":"active+clean","last_fresh":"2020-08-07T15:10:15.305964+0000","last_change":"2020-08-07T15:09:25.984108+0000","last_active":"2020-08-07T15:10:15.305964+0000","last_peered":"2020-08-07T15:10:15.305964+0000","last_clean":"2020-08-07T15:10:15.305964+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:10:15.305964+0000","last_undegraded":"2020-08-07T15:10:15.305964+0000","last_fullsized":"2020-08-07T15:10:15.305964+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:25.984018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:25.984018+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":"9","reported_epoch":"165","state":"active+clean","last_fresh":"2020-08-07T15:10:16.256000+0000","last_change":"2020-08-07T15:10:16.256000+0000","last_active":"2020-08-07T15:10:16.256000+0000","last_peered":"2020-08-07T15:10:16.256000+0000","last_clean":"2020-08-07T15:10:16.256000+0000","last_became_active":"2020-08-07T15:10:16.255356+0000","last_became_peered":"2020-08-07T15:10:16.255356+0000","last_unstale":"2020-08-07T15:10:16.256000+0000","last_undegraded":"2020-08-07T15:10:16.256000+0000","last_fullsized":"2020-08-07T15:10:16.256000+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"156","reported_epoch":"164","state":"active+clean","last_fresh":"2020-08-07T15:10:15.306150+0000","last_change":"2020-08-07T15:10:10.962565+0000","last_active":"2020-08-07T15:10:15.306150+0000","last_peered":"2020-08-07T15:10:15.306150+0000","last_clean":"2020-08-07T15:10:15.306150+0000","last_became_active":"2020-08-07T15:08:41.680448+0000","last_became_peered":"2020-08-07T15:08:41.680448+0000","last_unstale":"2020-08-07T15:10:15.306150+0000","last_undegraded":"2020-08-07T15:10:15.306150+0000","last_fullsized":"2020-08-07T15:10:15.306150+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:10:10.962480+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.962480+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":"9","reported_epoch":"165","state":"active+clean","last_fresh":"2020-08-07T15:10:16.444776+0000","last_change":"2020-08-07T15:10:16.444776+0000","last_active":"2020-08-07T15:10:16.444776+0000","last_peered":"2020-08-07T15:10:16.444776+0000","last_clean":"2020-08-07T15:10:16.444776+0000","last_became_active":"2020-08-07T15:10:16.444316+0000","last_became_peered":"2020-08-07T15:10:16.444316+0000","last_unstale":"2020-08-07T15:10:16.444776+0000","last_undegraded":"2020-08-07T15:10:16.444776+0000","last_fullsized":"2020-08-07T15:10:16.444776+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"123","reported_epoch":"164","state":"active+clean","last_fresh":"2020-08-07T15:10:15.303974+0000","last_change":"2020-08-07T15:09:36.687393+0000","last_active":"2020-08-07T15:10:15.303974+0000","last_peered":"2020-08-07T15:10:15.303974+0000","last_clean":"2020-08-07T15:10:15.303974+0000","last_became_active":"2020-08-07T15:09:30.204655+0000","last_became_peered":"2020-08-07T15:09:30.204655+0000","last_unstale":"2020-08-07T15:10:15.303974+0000","last_undegraded":"2020-08-07T15:10:15.303974+0000","last_fullsized":"2020-08-07T15:10:15.303974+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:36.687314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:36.687314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":"9","reported_epoch":"165","state":"active+clean","last_fresh":"2020-08-07T15:10:16.444970+0000","last_change":"2020-08-07T15:10:16.444970+0000","last_active":"2020-08-07T15:10:16.444970+0000","last_peered":"2020-08-07T15:10:16.444970+0000","last_clean":"2020-08-07T15:10:16.444970+0000","last_became_active":"2020-08-07T15:10:16.444294+0000","last_became_peered":"2020-08-07T15:10:16.444294+0000","last_unstale":"2020-08-07T15:10:16.444970+0000","last_undegraded":"2020-08-07T15:10:16.444970+0000","last_fullsized":"2020-08-07T15:10:16.444970+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"197","reported_epoch":"165","state":"active+clean","last_fresh":"2020-08-07T15:10:16.249127+0000","last_change":"2020-08-07T15:09:08.399507+0000","last_active":"2020-08-07T15:10:16.249127+0000","last_peered":"2020-08-07T15:10:16.249127+0000","last_clean":"2020-08-07T15:10:16.249127+0000","last_became_active":"2020-08-07T15:08:41.689435+0000","last_became_peered":"2020-08-07T15:08:41.689435+0000","last_unstale":"2020-08-07T15:10:16.249127+0000","last_undegraded":"2020-08-07T15:10:16.249127+0000","last_fullsized":"2020-08-07T15:10:16.249127+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:08.399439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:08.399439+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":"9","reported_epoch":"165","state":"active+clean","last_fresh":"2020-08-07T15:10:16.256758+0000","last_change":"2020-08-07T15:10:16.256758+0000","last_active":"2020-08-07T15:10:16.256758+0000","last_peered":"2020-08-07T15:10:16.256758+0000","last_clean":"2020-08-07T15:10:16.256758+0000","last_became_active":"2020-08-07T15:10:16.256419+0000","last_became_peered":"2020-08-07T15:10:16.256419+0000","last_unstale":"2020-08-07T15:10:16.256758+0000","last_undegraded":"2020-08-07T15:10:16.256758+0000","last_fullsized":"2020-08-07T15:10:16.256758+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"84","reported_epoch":"164","state":"active+clean","last_fresh":"2020-08-07T15:10:15.306158+0000","last_change":"2020-08-07T15:09:09.868283+0000","last_active":"2020-08-07T15:10:15.306158+0000","last_peered":"2020-08-07T15:10:15.306158+0000","last_clean":"2020-08-07T15:10:15.306158+0000","last_became_active":"2020-08-07T15:08:46.710059+0000","last_became_peered":"2020-08-07T15:08:46.710059+0000","last_unstale":"2020-08-07T15:10:15.306158+0000","last_undegraded":"2020-08-07T15:10:15.306158+0000","last_fullsized":"2020-08-07T15:10:15.306158+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.868198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.868198+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":"9","reported_epoch":"165","state":"active+clean","last_fresh":"2020-08-07T15:10:16.253662+0000","last_change":"2020-08-07T15:10:16.253662+0000","last_active":"2020-08-07T15:10:16.253662+0000","last_peered":"2020-08-07T15:10:16.253662+0000","last_clean":"2020-08-07T15:10:16.253662+0000","last_became_active":"2020-08-07T15:10:16.253105+0000","last_became_peered":"2020-08-07T15:10:16.253105+0000","last_unstale":"2020-08-07T15:10:16.253662+0000","last_undegraded":"2020-08-07T15:10:16.253662+0000","last_fullsized":"2020-08-07T15:10:16.253662+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"106","reported_epoch":"164","state":"active+clean","last_fresh":"2020-08-07T15:10:15.244579+0000","last_change":"2020-08-07T15:09:08.653773+0000","last_active":"2020-08-07T15:10:15.244579+0000","last_peered":"2020-08-07T15:10:15.244579+0000","last_clean":"2020-08-07T15:10:15.244579+0000","last_became_active":"2020-08-07T15:08:41.677944+0000","last_became_peered":"2020-08-07T15:08:41.677944+0000","last_unstale":"2020-08-07T15:10:15.244579+0000","last_undegraded":"2020-08-07T15:10:15.244579+0000","last_fullsized":"2020-08-07T15:10:15.244579+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:08.653722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:08.653722+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":63,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542194,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049512,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322328,"statfs":{"total":96636764160,"available":95562063872,"internally_reserved":1073741824,"allocated":950272,"data_stored":370995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.95099999999999996,"15min":0.91200000000000003},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.1739999999999999,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.095,"5min":1.0149999999999999,"15min":0.97999999999999998},"min":{"1min":0.54300000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":0.67400000000000004}]},{"osd":1,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.357,"15min":1.3149999999999999},"min":{"1min":0.68700000000000006,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":5.6239999999999997,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":1.214},{"interface":"front","average":{"1min":1.105,"5min":1.0720000000000001,"15min":1.0009999999999999},"min":{"1min":0.64100000000000001,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.206,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":1.02}]},{"osd":2,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.272,"15min":1.175},"min":{"1min":0.71999999999999997,"5min":0.628,"15min":0.628},"max":{"1min":5.6619999999999999,"5min":5.6619999999999999,"15min":5.6619999999999999},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.446,"5min":1.26,"15min":1.165},"min":{"1min":0.66800000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":5.1319999999999997,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":1.0489999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":75.045000000000002,"5min":16.263999999999999,"15min":6.5359999999999996},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":917.38699999999994,"5min":917.38699999999994,"15min":917.38699999999994},"last":1.0880000000000001},{"interface":"front","average":{"1min":19.670999999999999,"5min":4.9939999999999998,"15min":2.5600000000000001},"min":{"1min":0.80000000000000004,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":333.59199999999998,"5min":333.59199999999998,"15min":333.59199999999998},"last":1.1850000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.411,"15min":1.321},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":4.5250000000000004,"5min":4.835,"15min":4.835},"last":1.153},{"interface":"front","average":{"1min":52.259999999999998,"5min":11.52,"15min":4.7140000000000004},"min":{"1min":0.77000000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":866.24099999999999,"5min":866.24099999999999,"15min":866.24099999999999},"last":1.242}]},{"osd":5,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.2869999999999999,"5min":23.440000000000001,"15min":32.817999999999998},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":6.6509999999999998,"5min":831.88099999999997,"15min":831.88099999999997},"last":1.2689999999999999},{"interface":"front","average":{"1min":2.2519999999999998,"5min":1.8380000000000001,"15min":1.609},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":6.6790000000000003,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":1.399}]},{"osd":7,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":13.818,"5min":4.0019999999999998,"15min":2.3460000000000001},"min":{"1min":0.82999999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":224.14500000000001,"5min":224.14500000000001,"15min":224.14500000000001},"last":1.371},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.6839999999999999,"15min":1.5660000000000001},"min":{"1min":0.90600000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":7.4359999999999999,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":1.421}]}]},{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049500,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322340,"statfs":{"total":96636764160,"available":95562076160,"internally_reserved":1073741824,"allocated":937984,"data_stored":365877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.179,"15min":1.2809999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8149999999999999,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":15.239000000000001,"5min":3.9529999999999998,"15min":2.0840000000000001},"min":{"1min":0.78100000000000003,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":320.31299999999999,"5min":320.31299999999999,"15min":320.31299999999999},"last":0.89800000000000002}]},{"osd":1,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":24.821999999999999,"5min":5.7329999999999997,"15min":2.5489999999999999},"min":{"1min":0.46700000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":547.65300000000002,"5min":547.65300000000002,"15min":547.65300000000002},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":4.9130000000000003,"15min":2.5619999999999998},"min":{"1min":0.69799999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.5449999999999999,"5min":372.50799999999998,"15min":372.50799999999998},"last":0.66600000000000004}]},{"osd":2,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.208,"15min":1.1990000000000001},"min":{"1min":0.502,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":5.2400000000000002,"5min":5.2400000000000002,"15min":5.2400000000000002},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.2390000000000001,"15min":1.2609999999999999},"min":{"1min":0.66800000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":6.0490000000000004,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":0.93300000000000005}]},{"osd":3,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.4370000000000001,"15min":1.496},"min":{"1min":0.878,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.0670000000000002,"5min":3.46,"15min":3.46},"last":1.335},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3600000000000001,"15min":1.3779999999999999},"min":{"1min":0.70299999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":5.2770000000000001,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":0.98499999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.504,"15min":1.6100000000000001},"min":{"1min":0.70099999999999996,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.165,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.46,"5min":1.625,"15min":1.6559999999999999},"min":{"1min":0.96199999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":2.4279999999999999,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.355}]},{"osd":5,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.2809999999999999,"15min":1.3360000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.3069999999999999,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":1.381},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.6259999999999999,"15min":1.7190000000000001},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.4630000000000001,"5min":5.327,"15min":5.327},"last":1.3140000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.458,"15min":1.4410000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.4900000000000002,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.403},{"interface":"front","average":{"1min":1.5720000000000001,"5min":11.603999999999999,"15min":16.085000000000001},"min":{"1min":1.004,"5min":0.87,"15min":0.87},"max":{"1min":2.7719999999999998,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.4319999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049512,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322328,"statfs":{"total":96636764160,"available":95562063872,"internally_reserved":1073741824,"allocated":950272,"data_stored":370995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":20.945,"15min":29.771999999999998},"min":{"1min":0.49199999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":6.8810000000000002,"5min":497.04000000000002,"15min":497.04000000000002},"last":1.004},{"interface":"front","average":{"1min":25.385999999999999,"5min":14.656000000000001,"15min":5.6390000000000002},"min":{"1min":0.52200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":513.78899999999999,"5min":993.84500000000003,"15min":993.84500000000003},"last":1.04}]},{"osd":2,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.999,"15min":0.93200000000000005},"min":{"1min":0.60099999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.726,"5min":4.585,"15min":4.585},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.254,"15min":1.22},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":6.4029999999999996,"5min":6.4029999999999996,"15min":6.4029999999999996},"last":0.65500000000000003}]},{"osd":3,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":40.524000000000001,"5min":9.2829999999999995,"15min":3.968},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":828.31600000000003,"5min":828.31600000000003,"15min":828.31600000000003},"last":0.81699999999999995},{"interface":"front","average":{"1min":44.292000000000002,"5min":10.169,"15min":4.359},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":905.38300000000004,"5min":905.38300000000004,"15min":905.38300000000004},"last":0.88}]},{"osd":4,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":27.831,"5min":6.774,"15min":3.1139999999999999},"min":{"1min":0.79900000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":550.66800000000001,"5min":550.66800000000001,"15min":550.66800000000001},"last":872.58500000000004},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.3400000000000001,"15min":1.246},"min":{"1min":0.92100000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":5.9420000000000002,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.504}]},{"osd":5,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":8.4550000000000001,"15min":11.49},"min":{"1min":0.71199999999999997,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.7200000000000002,"5min":213.63800000000001,"15min":213.63800000000001},"last":1.0800000000000001},{"interface":"front","average":{"1min":58.497,"5min":12.720000000000001,"15min":5.0330000000000004},"min":{"1min":0.76500000000000001,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":875.44600000000003,"5min":875.44600000000003,"15min":875.44600000000003},"last":1.099}]},{"osd":6,"last update":"Fri Aug 7 15:09:27 2020","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.587,"15min":1.5629999999999999},"min":{"1min":0.81000000000000005,"5min":0.754,"15min":0.754},"max":{"1min":5.7149999999999999,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":11.539,"5min":3.5979999999999999,"15min":2.1059999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":207.137,"5min":207.137,"15min":207.137},"last":1.331}]},{"osd":7,"last update":"Fri Aug 7 15:09:19 2020","interfaces":[{"interface":"back","average":{"1min":1.782,"5min":1.6879999999999999,"15min":1.639},"min":{"1min":0.94099999999999995,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":6.444,"5min":6.444,"15min":6.444},"last":1.202},{"interface":"front","average":{"1min":1.6819999999999999,"5min":1.6779999999999999,"15min":1.546},"min":{"1min":0.93300000000000005,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":5.9279999999999999,"5min":10.752000000000001,"15min":10.752000000000001},"last":1.3740000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049512,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322328,"statfs":{"total":96636764160,"available":95562063872,"internally_reserved":1073741824,"allocated":950272,"data_stored":370995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":17.058,"15min":6.3369999999999997},"min":{"1min":0.61399999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.1269999999999998,"5min":845.57100000000003,"15min":845.57100000000003},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.0169999999999999,"15min":0.96099999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":4.8209999999999997,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":0.751}]},{"osd":2,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.0189999999999999,"15min":0.95799999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.7719999999999998,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":13.177,"5min":22.48,"15min":29.32},"min":{"1min":0.61399999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":226.911,"5min":606.18499999999995,"15min":606.18499999999995},"last":1.1279999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.0660000000000001,"15min":0.97899999999999998},"min":{"1min":0.54100000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6859999999999999,"5min":3.1779999999999999,"15min":3.1779999999999999},"last":1.258},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.3160000000000001,"15min":1.2170000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.976,"5min":4.21,"15min":4.21},"last":1.415}]},{"osd":4,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.2829999999999999,"15min":1.2050000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":8.3100000000000005,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.7709999999999999,"5min":1.446,"15min":1.3200000000000001},"min":{"1min":0.84599999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":8.4190000000000005,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":1.331}]},{"osd":5,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.9770000000000001,"5min":9.0380000000000003,"15min":3.8300000000000001},"min":{"1min":0.77800000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":7.8929999999999998,"5min":501.267,"15min":501.267},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.2609999999999999,"15min":1.157},"min":{"1min":0.78200000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.0939999999999999,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":1.2}]},{"osd":6,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":5.7779999999999996,"15min":7.6680000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.0059999999999998,"5min":124.738,"15min":124.738},"last":1.369},{"interface":"front","average":{"1min":1.627,"5min":11.811,"15min":16.372},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":4.6879999999999997,"5min":294.94,"15min":294.94},"last":1.389}]},{"osd":7,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.5589999999999999,"15min":1.577},"min":{"1min":0.95399999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":4.7169999999999996,"5min":6.468,"15min":6.468},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.4410000000000001,"15min":1.4039999999999999},"min":{"1min":0.88800000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.1040000000000001,"5min":3.117,"15min":3.117},"last":1.4530000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049512,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322328,"statfs":{"total":96636764160,"available":95562063872,"internally_reserved":1073741824,"allocated":950272,"data_stored":370995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.6950000000000001,"5min":1.268,"15min":1.167},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":8.4930000000000003,"5min":8.4930000000000003,"15min":8.4930000000000003},"last":0.58199999999999996},{"interface":"front","average":{"1min":1.357,"5min":1.113,"15min":1.0209999999999999},"min":{"1min":0.61899999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":3.5,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.79300000000000004}]},{"osd":1,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.9390000000000001,"5min":1.5469999999999999,"15min":1.2430000000000001},"min":{"1min":0.54400000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":5.6820000000000004,"5min":18.672000000000001,"15min":18.672000000000001},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.351,"15min":1.357},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":3.585,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":1.1830000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.268,"15min":1.198},"min":{"1min":0.72799999999999998,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.8740000000000001,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.026},{"interface":"front","average":{"1min":34.170999999999999,"5min":8.0510000000000002,"15min":3.7210000000000001},"min":{"1min":0.68799999999999994,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":501.75599999999997,"5min":501.75599999999997,"15min":501.75599999999997},"last":1.3029999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.403,"15min":1.2070000000000001},"min":{"1min":0.65200000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":3.4180000000000001,"5min":19.783000000000001,"15min":19.783000000000001},"last":1.278},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.546,"15min":1.6100000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":3.5569999999999999,"5min":3.5569999999999999,"15min":3.5569999999999999},"last":1.3300000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.292,"15min":1.272},"min":{"1min":0.83299999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.323,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":1.2529999999999999},{"interface":"front","average":{"1min":43.396999999999998,"5min":24.298999999999999,"15min":25.116},"min":{"1min":0.99299999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":627.53300000000002,"5min":627.53300000000002,"15min":627.53300000000002},"last":1.518}]},{"osd":6,"last update":"Fri Aug 7 15:09:30 2020","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.538,"15min":1.3260000000000001},"min":{"1min":0.85999999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":3.452,"5min":20.035,"15min":20.035},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.498,"5min":9.8979999999999997,"15min":4.1799999999999997},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.976,"5min":751.70000000000005,"15min":751.70000000000005},"last":1.5009999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":54.209000000000003,"5min":12.595000000000001,"15min":5.359},"min":{"1min":0.84999999999999998,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":501.91699999999997,"5min":501.91699999999997,"15min":501.91699999999997},"last":1.218},{"interface":"front","average":{"1min":1.665,"5min":1.919,"15min":1.637},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":3.133,"5min":35.646999999999998,"15min":35.646999999999998},"last":1.349}]}]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049512,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322328,"statfs":{"total":96636764160,"available":95562063872,"internally_reserved":1073741824,"allocated":950272,"data_stored":370995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":1.341,"15min":1.282},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.097,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.4530000000000001,"15min":1.575},"min":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.3039999999999998,"5min":13.067,"15min":13.067},"last":0.88100000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":4.5860000000000003,"15min":2.794},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.9910000000000001,"5min":270.68700000000001,"15min":270.68700000000001},"last":1.302},{"interface":"front","average":{"1min":1.4379999999999999,"5min":2.0390000000000001,"15min":2.0209999999999999},"min":{"1min":0.55100000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":7.0940000000000003,"5min":14.349,"15min":14.349},"last":0.999}]},{"osd":2,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.266,"15min":1.1870000000000001},"min":{"1min":0.71399999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.2639999999999998,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.96899999999999997,"5min":2.109,"15min":1.8260000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.661,"5min":51.530000000000001,"15min":51.530000000000001},"last":1.0409999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":10.773999999999999,"15min":4.7220000000000004},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.1360000000000001,"5min":506.21300000000002,"15min":506.21300000000002},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.276,"5min":25.469000000000001,"15min":29.888999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.8940000000000001,"5min":677.88599999999997,"15min":677.88599999999997},"last":1.1970000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.5920000000000001,"15min":1.542},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.9710000000000001,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.8009999999999999,"15min":1.7789999999999999},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":3.044,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":1.53}]},{"osd":6,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.6259999999999999,"15min":1.5629999999999999},"min":{"1min":0.89800000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.9359999999999999,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":1.488},{"interface":"front","average":{"1min":21.766999999999999,"5min":5.9390000000000001,"15min":3.3039999999999998},"min":{"1min":0.89900000000000002,"5min":0.876,"15min":0.876},"max":{"1min":387.911,"5min":387.911,"15min":387.911},"last":1.3520000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.9330000000000001,"15min":2.085},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.0110000000000001,"5min":5.734,"15min":5.734},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.325,"5min":1.9750000000000001,"15min":1.976},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.169,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":1.26}]}]},{"osd":4,"up_from":13,"seq":55834574898,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049500,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322340,"statfs":{"total":96636764160,"available":95562076160,"internally_reserved":1073741824,"allocated":937984,"data_stored":365877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":1.0720000000000001,"15min":1.1439999999999999},"min":{"1min":0.52900000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4390000000000001,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":399.73700000000002},{"interface":"front","average":{"1min":1.143,"5min":1.0069999999999999,"15min":0.90100000000000002},"min":{"1min":0.498,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":4.907,"5min":6.3970000000000002,"15min":6.3970000000000002},"last":0.91000000000000003}]},{"osd":1,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":23.940999999999999,"5min":5.8209999999999997,"15min":2.843},"min":{"1min":0.59599999999999997,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":320.67000000000002,"5min":320.67000000000002,"15min":320.67000000000002},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.1359999999999999,"5min":7.5700000000000003,"15min":3.4409999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.784,"5min":689.846,"15min":689.846},"last":1.256}]},{"osd":2,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.3999999999999999,"15min":1.4339999999999999},"min":{"1min":0.77600000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.0899999999999999,"5min":6.8730000000000002,"15min":6.8730000000000002},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.516,"15min":1.5149999999999999},"min":{"1min":0.81699999999999995,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.3159999999999998,"5min":7.1360000000000001,"15min":7.1360000000000001},"last":1.0249999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":3.681,"15min":2.2320000000000002},"min":{"1min":0.78300000000000003,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.8149999999999999,"5min":247.202,"15min":247.202},"last":1.286},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.3720000000000001,"15min":1.2909999999999999},"min":{"1min":0.68500000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":4.931,"5min":6.431,"15min":6.431},"last":1.3100000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":10.57,"15min":4.6079999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.431,"5min":990.71600000000001,"15min":990.71600000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.6100000000000001,"15min":1.621},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":3.3519999999999999,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":0.873}]},{"osd":6,"last update":"Fri Aug 7 15:09:28 2020","interfaces":[{"interface":"back","average":{"1min":1.6799999999999999,"5min":1.7509999999999999,"15min":1.7769999999999999},"min":{"1min":0.75800000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":5.0149999999999997,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.498,"5min":12.069000000000001,"15min":16.762},"min":{"1min":0.83299999999999996,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":3.3530000000000002,"5min":389.14600000000002,"15min":389.14600000000002},"last":1.226}]},{"osd":7,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":3.8300000000000001,"15min":2.2149999999999999},"min":{"1min":0.753,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":4.8710000000000004,"5min":252.798,"15min":252.798},"last":1.044},{"interface":"front","average":{"1min":1.262,"5min":10.175000000000001,"15min":4.4889999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.3050000000000002,"5min":501.93000000000001,"15min":501.93000000000001},"last":0.76800000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574898,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049512,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322328,"statfs":{"total":96636764160,"available":95562063872,"internally_reserved":1073741824,"allocated":950272,"data_stored":370995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":1.012,"15min":1.0549999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.651,"5min":2.766,"15min":2.766},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91600000000000004,"15min":0.84199999999999997},"min":{"1min":0.45500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.3340000000000001,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.159}]},{"osd":1,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.4540000000000002,"5min":10.521000000000001,"15min":4.1260000000000003},"min":{"1min":0.56699999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":29.283000000000001,"5min":787.22900000000004,"15min":787.22900000000004},"last":0.98099999999999998},{"interface":"front","average":{"1min":8.0939999999999994,"5min":2.4969999999999999,"15min":1.571},"min":{"1min":0.60999999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":116.548,"5min":116.548,"15min":116.548},"last":1.454}]},{"osd":2,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":5.9340000000000002,"15min":8.1240000000000006},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.78,"5min":168.56899999999999,"15min":168.56899999999999},"last":1.135},{"interface":"front","average":{"1min":1.119,"5min":20.361000000000001,"15min":28.873999999999999},"min":{"1min":0.60899999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.665,"5min":676.27099999999996,"15min":676.27099999999996},"last":1.208}]},{"osd":3,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.6739999999999999,"5min":1.5069999999999999,"15min":1.2470000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":29.129000000000001,"5min":29.129000000000001,"15min":29.129000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":3.0219999999999998,"5min":9.9459999999999997,"15min":4.1239999999999997},"min":{"1min":0.70799999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":29.103000000000002,"5min":712.98699999999997,"15min":712.98699999999997},"last":1.5129999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":49.854999999999997,"5min":40.755000000000003,"15min":47.512999999999998},"min":{"1min":0.61899999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":971.67999999999995,"5min":971.67999999999995,"15min":971.67999999999995},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.264,"15min":1.292},"min":{"1min":0.64900000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.5960000000000001,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":0.79700000000000004}]},{"osd":6,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":91.870999999999995,"5min":48.862000000000002,"15min":49.893000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":966.60900000000004,"5min":966.60900000000004,"15min":966.60900000000004},"last":1.1100000000000001},{"interface":"front","average":{"1min":28.396999999999998,"5min":21.527000000000001,"15min":24.52},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":501.79000000000002,"5min":517.26700000000005,"15min":517.26700000000005},"last":1.486}]},{"osd":7,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.341,"15min":1.319},"min":{"1min":0.77500000000000002,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.8940000000000001,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":27.206,"5min":6.6630000000000003,"15min":3.2650000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":515.09100000000001,"5min":515.09100000000001,"15min":515.09100000000001},"last":1.2430000000000001}]}]}]}} 2020-08-07T15:10:21.943 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:10:21.947 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:21.565+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:21.948 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:10:21.973 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:21.971+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:22.077 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:22.076+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:22.125 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:10:22.131 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:10:22.131 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:10:22.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.132 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:3117 5.5 e168/164 5:a3c2c276:::benchmark_data_smithi074_53697_object389:head v 168'245, mlcod=168'240)", 2020-08-07T15:10:22.132 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:22.119033+0000", 2020-08-07T15:10:22.132 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.003870755, 2020-08-07T15:10:22.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038958930000000001, 2020-08-07T15:10:22.133 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:22.133 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:22.133 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:22.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:22.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.119033+0000", 2020-08-07T15:10:22.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:22.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:22.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.119033+0000", 2020-08-07T15:10:22.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8736000000000001e-05 2020-08-07T15:10:22.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:22.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.119061+0000", 2020-08-07T15:10:22.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0690000000000003e-06 2020-08-07T15:10:22.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:22.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.119066+0000", 2020-08-07T15:10:22.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0074e-05 2020-08-07T15:10:22.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:22.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.119096+0000", 2020-08-07T15:10:22.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8259999999999998e-06 2020-08-07T15:10:22.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:22.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.119101+0000", 2020-08-07T15:10:22.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00083079900000000001 2020-08-07T15:10:22.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:22.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.119932+0000", 2020-08-07T15:10:22.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1393999999999999e-05 2020-08-07T15:10:22.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:22.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.119954+0000", 2020-08-07T15:10:22.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00092089799999999996 2020-08-07T15:10:22.164 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:22.165 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:22.165 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:22.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.165 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:3118 5.5 e168/164 5:a3c2c276:::benchmark_data_smithi074_53697_object389:head v 168'246, mlcod=168'240)", 2020-08-07T15:10:22.181 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:22.119694+0000", 2020-08-07T15:10:22.189 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.003209483, 2020-08-07T15:10:22.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003277889, 2020-08-07T15:10:22.190 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:22.190 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:22.191 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:22.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:22.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.119694+0000", 2020-08-07T15:10:22.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:22.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:22.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.119694+0000", 2020-08-07T15:10:22.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3891999999999999e-05 2020-08-07T15:10:22.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:22.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.119738+0000", 2020-08-07T15:10:22.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1570000000000002e-06 2020-08-07T15:10:22.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:22.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.119742+0000", 2020-08-07T15:10:22.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5812e-05 2020-08-07T15:10:22.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:22.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.119758+0000", 2020-08-07T15:10:22.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9589999999999997e-06 2020-08-07T15:10:22.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:22.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.119762+0000", 2020-08-07T15:10:22.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049066099999999998 2020-08-07T15:10:22.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:22.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.120252+0000", 2020-08-07T15:10:22.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2365e-05 2020-08-07T15:10:22.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:22.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.120275+0000", 2020-08-07T15:10:22.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00058084599999999999 2020-08-07T15:10:22.251 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:22.252 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:22.252 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:22.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.252 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:3119 5.5 e168/164 5:a3c2c276:::benchmark_data_smithi074_53697_object389:head v 168'247, mlcod=168'240)", 2020-08-07T15:10:22.253 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:22.120211+0000", 2020-08-07T15:10:22.254 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.002692037, 2020-08-07T15:10:22.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027957910000000002, 2020-08-07T15:10:22.255 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:22.255 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:22.255 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:22.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:22.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.120211+0000", 2020-08-07T15:10:22.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:22.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:22.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.120211+0000", 2020-08-07T15:10:22.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4968999999999997e-05 2020-08-07T15:10:22.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:22.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.120246+0000", 2020-08-07T15:10:22.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3229999999999999e-06 2020-08-07T15:10:22.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:22.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.120251+0000", 2020-08-07T15:10:22.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5840000000000001e-05 2020-08-07T15:10:22.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:22.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.120267+0000", 2020-08-07T15:10:22.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1880000000000004e-06 2020-08-07T15:10:22.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:22.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.120271+0000", 2020-08-07T15:10:22.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000266333 2020-08-07T15:10:22.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:22.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.120537+0000", 2020-08-07T15:10:22.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9610999999999999e-05 2020-08-07T15:10:22.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:22.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.120557+0000", 2020-08-07T15:10:22.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034526400000000002 2020-08-07T15:10:22.266 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:22.267 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:22.268 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:22.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.268 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:3120 5.5 e168/164 5:a3c2c276:::benchmark_data_smithi074_53697_object389:head v 168'248, mlcod=168'240)", 2020-08-07T15:10:22.268 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:22.121117+0000", 2020-08-07T15:10:22.269 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0017868210000000001, 2020-08-07T15:10:22.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019266280000000001, 2020-08-07T15:10:22.269 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:22.270 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:22.270 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:22.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:22.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.121117+0000", 2020-08-07T15:10:22.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:22.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:22.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.121117+0000", 2020-08-07T15:10:22.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5870000000000001e-05 2020-08-07T15:10:22.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:22.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.121143+0000", 2020-08-07T15:10:22.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6260000000000001e-06 2020-08-07T15:10:22.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:22.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.121146+0000", 2020-08-07T15:10:22.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3660999999999999e-05 2020-08-07T15:10:22.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:22.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.121170+0000", 2020-08-07T15:10:22.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9000000000000003e-06 2020-08-07T15:10:22.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:22.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.121176+0000", 2020-08-07T15:10:22.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5148999999999999e-05 2020-08-07T15:10:22.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:22.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.121201+0000", 2020-08-07T15:10:22.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.523e-05 2020-08-07T15:10:22.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:22.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.121216+0000", 2020-08-07T15:10:22.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9436000000000001e-05 2020-08-07T15:10:22.279 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:22.279 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:22.279 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:22.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.280 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:3121 5.1 e168/164 5:8a059730:::benchmark_data_smithi074_53697_object390:head v 168'209, mlcod=168'207)", 2020-08-07T15:10:22.280 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:22.122163+0000", 2020-08-07T15:10:22.280 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00074028600000000005, 2020-08-07T15:10:22.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00092129099999999997, 2020-08-07T15:10:22.280 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:22.281 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:22.281 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:22.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:22.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.122163+0000", 2020-08-07T15:10:22.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:22.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:22.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.122163+0000", 2020-08-07T15:10:22.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2532e-05 2020-08-07T15:10:22.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:22.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.122196+0000", 2020-08-07T15:10:22.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1949999999999996e-06 2020-08-07T15:10:22.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:22.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.122200+0000", 2020-08-07T15:10:22.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5781000000000002e-05 2020-08-07T15:10:22.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:22.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.122236+0000", 2020-08-07T15:10:22.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0579999999999999e-06 2020-08-07T15:10:22.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:22.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.122240+0000", 2020-08-07T15:10:22.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8311e-05 2020-08-07T15:10:22.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:22.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.122278+0000", 2020-08-07T15:10:22.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6558e-05 2020-08-07T15:10:22.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:22.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.122295+0000", 2020-08-07T15:10:22.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000131435 2020-08-07T15:10:22.291 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:22.292 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:22.292 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:22.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.293 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:3115 5.5 e168/164 5:a3c2c276:::benchmark_data_smithi074_53697_object389:head v 168'243, mlcod=168'239)", 2020-08-07T15:10:22.293 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:22.117342+0000", 2020-08-07T15:10:22.293 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0055609639999999998, 2020-08-07T15:10:22.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057760529999999997, 2020-08-07T15:10:22.294 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:22.294 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:22.304 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:22.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:22.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.117342+0000", 2020-08-07T15:10:22.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:22.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:22.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.117342+0000", 2020-08-07T15:10:22.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1162999999999998e-05 2020-08-07T15:10:22.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:22.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.117394+0000", 2020-08-07T15:10:22.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1859999999999997e-06 2020-08-07T15:10:22.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:22.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.117400+0000", 2020-08-07T15:10:22.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4742000000000001e-05 2020-08-07T15:10:22.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:22.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.117425+0000", 2020-08-07T15:10:22.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4310000000000002e-06 2020-08-07T15:10:22.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:22.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.117430+0000", 2020-08-07T15:10:22.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3239000000000003e-05 2020-08-07T15:10:22.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:22.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.117483+0000", 2020-08-07T15:10:22.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4848e-05 2020-08-07T15:10:22.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:22.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.117508+0000", 2020-08-07T15:10:22.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016560899999999999 2020-08-07T15:10:22.313 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:22.314 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:22.314 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:22.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.315 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:3116 5.5 e168/164 5:a3c2c276:::benchmark_data_smithi074_53697_object389:head v 168'244, mlcod=168'239)", 2020-08-07T15:10:22.315 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:22.117524+0000", 2020-08-07T15:10:22.316 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0053797769999999996, 2020-08-07T15:10:22.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056279779999999996, 2020-08-07T15:10:22.318 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:22.318 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:22.318 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:22.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:22.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.117524+0000", 2020-08-07T15:10:22.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:22.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:22.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.117524+0000", 2020-08-07T15:10:22.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1683e-05 2020-08-07T15:10:22.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:22.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.117575+0000", 2020-08-07T15:10:22.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8749999999999997e-06 2020-08-07T15:10:22.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:22.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.117581+0000", 2020-08-07T15:10:22.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3730000000000001e-05 2020-08-07T15:10:22.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:22.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.117605+0000", 2020-08-07T15:10:22.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7199999999999997e-06 2020-08-07T15:10:22.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:22.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.117610+0000", 2020-08-07T15:10:22.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020302100000000002 2020-08-07T15:10:22.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:22.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.119640+0000", 2020-08-07T15:10:22.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1319999999999999e-05 2020-08-07T15:10:22.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:22.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:22.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:22.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:22.119661+0000", 2020-08-07T15:10:22.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002137538 2020-08-07T15:10:22.364 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:22.364 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:22.364 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:22.364 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:22.364 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:10:22.365 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 7 2020-08-07T15:10:22.365 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:10:22.365 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:10:22.380 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 2 16 3260 3244 12.6702 12.3203 0.0118721 0.00978704 2020-08-07T15:10:22.381 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:22.379+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:22.383 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:10:22.414 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:10:22.437 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:10:22.479 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:10:22.493 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:22.487+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:22.514 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:10:22.562 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:10:22.586 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:22.586 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:22.596 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:10:22.597 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:10:22.627 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:22.594+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:22.630 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:10:22.667 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:10:22.668 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:10:22.669 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:10:22.669 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:10:22.669 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:10:22.669 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:10:22.670 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:10:22.670 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:10:22.670 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:10:22.722 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:22.720+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:22.802 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:22.824 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:22.847 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:22.802+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:22.848 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:10:22.916 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:22.910+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:23.022 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:23.019+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:23.024 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:23.024 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:23.043 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:10:23.130 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:23.126+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:23.214 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 3 16 4803 4787 12.4648 12.0547 0.00514697 0.00999995 2020-08-07T15:10:23.230 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:23.227+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:23.237 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:23.237 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:23.245 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:10:23.303 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:10:23.329 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:23.328+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:23.426 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:23.426 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:23.436 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:23.431+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:23.437 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:10:23.536 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:23.534+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:23.637 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:23.636+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:23.647 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:23.647 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:23.658 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:10:23.742 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:23.736+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:23.829 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:23.829 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:23.841 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:10:23.842 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:23.840+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:23.947 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:23.942+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:24.005 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:24.006 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:24.006 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:10:24.047 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:24.046+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:24.149 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:24.147+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:24.172 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:24.173 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:24.183 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:10:24.210 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 4 16 6338 6322 12.3464 11.9922 0.00786787 0.0100834 2020-08-07T15:10:24.284 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:24.277+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:24.371 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:24.369+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:24.405 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:24.405 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:24.432 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:10:24.476 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:24.474+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:24.580 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:24.576+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:24.659 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:24.659 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:24.660 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:10:24.680 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:24.679+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:24.684 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:10:24.783 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:24.781+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:24.861 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:24.861 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:24.879 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:10:24.883 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:24.882+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:24.948 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:10:24.954 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:10:24.987 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:10:24.987 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:24.986+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:25.030 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:10:25.074 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:10:25.075 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:25.075 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:10:25.076 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:10:25.076 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:10:25.076 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:25.093 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:10:25.107 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:25.091+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:25.108 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:10:25.204 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:25.193+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:25.210 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 5 16 7910 7894 12.3332 12.2812 0.00844874 0.0100918 2020-08-07T15:10:25.302 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:25.296+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:25.325 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:25.325 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:10:25.326 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:10:25.326 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:10:25.326 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.326 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:71136 3.6 e43/26 3:64cb7acb:::benchmark_data_smithi074_35447_object8891:head v 43'4744, mlcod=43'4739)", 2020-08-07T15:10:25.326 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:10:25.327 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 181.00152719299999, 2020-08-07T15:10:25.327 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26135482999999998, 2020-08-07T15:10:25.327 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:25.327 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:25.327 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:25.328 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.328 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:25.328 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:10:25.328 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:25.328 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.329 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.329 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:25.336 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:10:25.367 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3702999999999997e-05 2020-08-07T15:10:25.367 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.367 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.368 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:25.368 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318667+0000", 2020-08-07T15:10:25.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3119999999999999e-06 2020-08-07T15:10:25.368 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.368 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.369 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:25.369 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318674+0000", 2020-08-07T15:10:25.369 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.255173292 2020-08-07T15:10:25.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.370 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:25.370 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.573847+0000", 2020-08-07T15:10:25.387 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1140000000000002e-06 2020-08-07T15:10:25.388 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.388 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.388 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:25.389 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.573853+0000", 2020-08-07T15:10:25.389 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00057603799999999996 2020-08-07T15:10:25.389 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.389 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.389 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:25.390 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.574429+0000", 2020-08-07T15:10:25.390 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4776e-05 2020-08-07T15:10:25.390 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.390 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.390 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:25.391 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.574444+0000", 2020-08-07T15:10:25.391 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054577080000000004 2020-08-07T15:10:25.391 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.391 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.391 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:25.392 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.579901+0000", 2020-08-07T15:10:25.392 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6886999999999998e-05 2020-08-07T15:10:25.392 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.392 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.392 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:25.393 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.579968+0000", 2020-08-07T15:10:25.393 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26135482999999998 2020-08-07T15:10:25.393 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.394 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:25.394 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.394 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.394 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.394 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:85696 3.f e51/20 3:f99fcea4:::benchmark_data_smithi074_35447_object10711:head v 51'5688, mlcod=51'5679)", 2020-08-07T15:10:25.395 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:10:25.395 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 171.66753578000001, 2020-08-07T15:10:25.395 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49655717300000002, 2020-08-07T15:10:25.395 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:25.410 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:25.411 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:25.411 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.411 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:25.411 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:10:25.412 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:25.412 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.412 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.412 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:25.412 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:10:25.413 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9543000000000002e-05 2020-08-07T15:10:25.413 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.413 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:25.413 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652665+0000", 2020-08-07T15:10:25.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8700000000000003e-06 2020-08-07T15:10:25.435 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.435 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.438 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:25.438 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652671+0000", 2020-08-07T15:10:25.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49579595799999998 2020-08-07T15:10:25.442 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.444 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:25.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148467+0000", 2020-08-07T15:10:25.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2509999999999997e-06 2020-08-07T15:10:25.445 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.445 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.445 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:25.445 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148475+0000", 2020-08-07T15:10:25.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5603999999999998e-05 2020-08-07T15:10:25.446 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.446 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.446 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:25.446 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148530+0000", 2020-08-07T15:10:25.446 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5486999999999998e-05 2020-08-07T15:10:25.446 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.447 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:25.447 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148556+0000", 2020-08-07T15:10:25.447 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00055666600000000004 2020-08-07T15:10:25.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:25.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.149112+0000", 2020-08-07T15:10:25.448 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9793999999999998e-05 2020-08-07T15:10:25.471 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.473 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.479 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:25.479 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.149162+0000", 2020-08-07T15:10:25.479 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49655717300000002 2020-08-07T15:10:25.479 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.479 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:25.480 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.480 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.480 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:86792 3.f e52/20 3:fc5386ee:::benchmark_data_smithi074_35447_object10848:head v 52'5752, mlcod=52'5743)", 2020-08-07T15:10:25.480 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:10:25.481 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 170.917690005, 2020-08-07T15:10:25.481 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.295270005, 2020-08-07T15:10:25.482 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:25.500 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:25.501 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:25.501 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.501 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:25.501 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:10:25.516 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:25.516 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.516 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.516 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:25.517 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:10:25.517 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9712000000000001e-05 2020-08-07T15:10:25.517 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.517 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.517 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:25.518 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402500+0000", 2020-08-07T15:10:25.518 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6789999999999999e-06 2020-08-07T15:10:25.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.518 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:25.519 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402507+0000", 2020-08-07T15:10:25.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29317074999999998 2020-08-07T15:10:25.545 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.546 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.546 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:25.546 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695678+0000", 2020-08-07T15:10:25.546 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8120000000000002e-06 2020-08-07T15:10:25.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.547 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:25.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695685+0000", 2020-08-07T15:10:25.547 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1270999999999998e-05 2020-08-07T15:10:25.547 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.548 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:25.548 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695726+0000", 2020-08-07T15:10:25.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2739e-05 2020-08-07T15:10:25.548 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.549 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.549 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:25.549 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695749+0000", 2020-08-07T15:10:25.558 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019333899999999999 2020-08-07T15:10:25.558 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.559 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.559 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:25.559 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.697682+0000", 2020-08-07T15:10:25.559 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8652000000000002e-05 2020-08-07T15:10:25.560 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.560 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.560 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:25.561 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.697721+0000", 2020-08-07T15:10:25.565 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.295270005 2020-08-07T15:10:25.565 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.565 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:25.565 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.566 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.566 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.566 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:87088 3.6 e52/26 3:6b0320df:::benchmark_data_smithi074_35447_object10885:head v 52'5912, mlcod=52'5902)", 2020-08-07T15:10:25.566 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:10:25.566 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 170.353456765, 2020-08-07T15:10:25.567 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29522189799999998, 2020-08-07T15:10:25.567 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:25.567 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:25.567 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:25.567 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.568 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:25.568 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:10:25.568 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:25.568 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.568 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.568 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:25.569 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:10:25.569 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0751000000000002e-05 2020-08-07T15:10:25.569 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.569 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.570 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:25.571 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966735+0000", 2020-08-07T15:10:25.571 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3080000000000004e-06 2020-08-07T15:10:25.571 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.572 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.573 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:25.573 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966741+0000", 2020-08-07T15:10:25.574 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29365034699999998 2020-08-07T15:10:25.574 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.574 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.574 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:25.575 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260391+0000", 2020-08-07T15:10:25.575 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2720000000000003e-06 2020-08-07T15:10:25.576 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.576 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.576 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:25.576 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260398+0000", 2020-08-07T15:10:25.577 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8819000000000005e-05 2020-08-07T15:10:25.577 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.577 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.577 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:25.577 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260476+0000", 2020-08-07T15:10:25.578 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4958999999999999e-05 2020-08-07T15:10:25.578 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.578 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.578 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:25.578 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260501+0000", 2020-08-07T15:10:25.579 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001353641 2020-08-07T15:10:25.579 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.579 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.579 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:25.579 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.261855+0000", 2020-08-07T15:10:25.580 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0800999999999997e-05 2020-08-07T15:10:25.580 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.581 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.581 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:25.582 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.261906+0000", 2020-08-07T15:10:25.582 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29522189799999998 2020-08-07T15:10:25.583 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.583 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:25.583 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.596 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.596 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.597 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:93920 3.6 e56/26 3:631393d3:::benchmark_data_smithi074_35447_object11739:head v 56'6352, mlcod=55'6342)", 2020-08-07T15:10:25.597 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:10:25.597 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 165.99334133400001, 2020-08-07T15:10:25.597 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39765404300000001, 2020-08-07T15:10:25.597 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:25.598 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:25.598 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:25.598 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.598 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:25.598 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:10:25.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:25.599 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.599 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.599 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:25.599 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:10:25.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3035999999999998e-05 2020-08-07T15:10:25.600 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.600 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.612 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:25.613 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326852+0000", 2020-08-07T15:10:25.613 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5379999999999999e-06 2020-08-07T15:10:25.615 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.615 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.616 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:25.616 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326859+0000", 2020-08-07T15:10:25.616 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.393917568 2020-08-07T15:10:25.616 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.616 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.617 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:25.617 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.720777+0000", 2020-08-07T15:10:25.617 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2430000000000005e-06 2020-08-07T15:10:25.617 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.617 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.618 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:25.618 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.720785+0000", 2020-08-07T15:10:25.618 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00065658700000000003 2020-08-07T15:10:25.618 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.618 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.618 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:25.619 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.721441+0000", 2020-08-07T15:10:25.619 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8585000000000002e-05 2020-08-07T15:10:25.619 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.619 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.619 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:25.620 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.721470+0000", 2020-08-07T15:10:25.620 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00293349 2020-08-07T15:10:25.620 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.620 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.620 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:25.632 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.724403+0000", 2020-08-07T15:10:25.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9996e-05 2020-08-07T15:10:25.632 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.633 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.633 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:25.633 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.724453+0000", 2020-08-07T15:10:25.633 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39765404300000001 2020-08-07T15:10:25.633 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.634 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:25.634 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.636 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.637 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.637 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:116064 3.6 e64/26 3:6cc03ffa:::benchmark_data_smithi074_35447_object14507:head v 64'7632, mlcod=64'7623)", 2020-08-07T15:10:25.637 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:10:25.637 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 148.95737993099999, 2020-08-07T15:10:25.637 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25728607799999997, 2020-08-07T15:10:25.638 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:25.638 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:25.638 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:25.638 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.638 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:25.639 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:10:25.639 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:25.639 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.639 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.639 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:25.640 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:10:25.640 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6042000000000003e-05 2020-08-07T15:10:25.640 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.640 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.640 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:25.641 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362817+0000", 2020-08-07T15:10:25.641 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9399999999999999e-06 2020-08-07T15:10:25.641 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.641 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.641 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:25.658 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362823+0000", 2020-08-07T15:10:25.658 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.254635583 2020-08-07T15:10:25.659 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.659 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.659 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:25.659 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617458+0000", 2020-08-07T15:10:25.660 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5100000000000004e-06 2020-08-07T15:10:25.660 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.660 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.660 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:25.661 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617465+0000", 2020-08-07T15:10:25.661 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4439999999999999e-05 2020-08-07T15:10:25.661 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.661 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.661 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:25.662 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617539+0000", 2020-08-07T15:10:25.662 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3527e-05 2020-08-07T15:10:25.662 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.662 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.662 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:25.663 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617563+0000", 2020-08-07T15:10:25.663 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024301069999999999 2020-08-07T15:10:25.663 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.663 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.663 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:25.663 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.619993+0000", 2020-08-07T15:10:25.664 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3928999999999998e-05 2020-08-07T15:10:25.664 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.686 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.686 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:25.686 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.620047+0000", 2020-08-07T15:10:25.687 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25728607799999997 2020-08-07T15:10:25.687 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.687 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:25.687 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.687 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.688 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.688 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:145801 3.6 e75/26 3:69e4cb22:::benchmark_data_smithi074_35447_object16192:head v 75'9405, mlcod=75'9401)", 2020-08-07T15:10:25.689 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:10:25.689 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 130.648384639, 2020-08-07T15:10:25.689 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639, 2020-08-07T15:10:25.689 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:25.694 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:25.695 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:25.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.695 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:25.695 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:10:25.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:25.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.696 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:25.696 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:10:25.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6511000000000002e-05 2020-08-07T15:10:25.697 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:25.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671803+0000", 2020-08-07T15:10:25.705 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7190000000000004e-06 2020-08-07T15:10:25.706 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.706 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.706 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:25.707 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671809+0000", 2020-08-07T15:10:25.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74210928300000001 2020-08-07T15:10:25.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:25.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413919+0000", 2020-08-07T15:10:25.724 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1759999999999999e-05 2020-08-07T15:10:25.724 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.725 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:25.725 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413930+0000", 2020-08-07T15:10:25.725 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000108601 2020-08-07T15:10:25.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.726 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.726 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:25.726 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414039+0000", 2020-08-07T15:10:25.726 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9948999999999999e-05 2020-08-07T15:10:25.726 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.727 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.727 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:25.727 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414089+0000", 2020-08-07T15:10:25.727 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00079656499999999995 2020-08-07T15:10:25.727 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:25.728 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414885+0000", 2020-08-07T15:10:25.728 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2250999999999998e-05 2020-08-07T15:10:25.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.729 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:25.729 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414938+0000", 2020-08-07T15:10:25.729 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639 2020-08-07T15:10:25.729 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.729 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:25.744 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.762 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:9112 4.6 e86/81 4:6004be0b:::benchmark_data_smithi074_44586_object1138:head v 86'544, mlcod=86'533)", 2020-08-07T15:10:25.762 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:32.296890+0000", 2020-08-07T15:10:25.763 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 113.02325123, 2020-08-07T15:10:25.769 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26263388100000001, 2020-08-07T15:10:25.769 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:25.769 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:25.769 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:25.770 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.770 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:25.770 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296890+0000", 2020-08-07T15:10:25.770 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:25.770 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.771 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.771 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:25.771 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296890+0000", 2020-08-07T15:10:25.771 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9907999999999997e-05 2020-08-07T15:10:25.771 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.772 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.772 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:25.772 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296939+0000", 2020-08-07T15:10:25.772 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7630000000000003e-06 2020-08-07T15:10:25.772 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.772 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.773 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:25.773 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296946+0000", 2020-08-07T15:10:25.773 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26058471900000002 2020-08-07T15:10:25.773 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.790 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:25.790 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557531+0000", 2020-08-07T15:10:25.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5270000000000004e-06 2020-08-07T15:10:25.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:25.791 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557537+0000", 2020-08-07T15:10:25.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2843000000000003e-05 2020-08-07T15:10:25.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.796 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:25.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557580+0000", 2020-08-07T15:10:25.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3833999999999999e-05 2020-08-07T15:10:25.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:25.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557604+0000", 2020-08-07T15:10:25.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001875196 2020-08-07T15:10:25.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.815 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:25.815 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.559479+0000", 2020-08-07T15:10:25.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4091000000000002e-05 2020-08-07T15:10:25.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:25.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.559523+0000", 2020-08-07T15:10:25.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26263388100000001 2020-08-07T15:10:25.818 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.818 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:25.818 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.818 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.819 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:13784 4.6 e88/81 4:66f64b92:::benchmark_data_smithi074_44586_object1722:head v 88'832, mlcod=88'827)", 2020-08-07T15:10:25.820 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:35.137315+0000", 2020-08-07T15:10:25.820 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 110.182825497, 2020-08-07T15:10:25.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32459016899999998, 2020-08-07T15:10:25.821 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:25.821 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:25.821 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:25.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.822 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:25.822 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137315+0000", 2020-08-07T15:10:25.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:25.833 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:25.833 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137315+0000", 2020-08-07T15:10:25.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0362999999999998e-05 2020-08-07T15:10:25.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.836 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:25.836 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137376+0000", 2020-08-07T15:10:25.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7020000000000004e-06 2020-08-07T15:10:25.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:25.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137382+0000", 2020-08-07T15:10:25.846 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32172605500000001 2020-08-07T15:10:25.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.846 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.847 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:25.847 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459108+0000", 2020-08-07T15:10:25.847 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7710000000000003e-06 2020-08-07T15:10:25.847 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.847 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.848 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:25.849 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459114+0000", 2020-08-07T15:10:25.849 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4297e-05 2020-08-07T15:10:25.849 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.849 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.849 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:25.850 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459158+0000", 2020-08-07T15:10:25.850 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5058999999999998e-05 2020-08-07T15:10:25.850 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.851 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:25.851 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459183+0000", 2020-08-07T15:10:25.851 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026624740000000002 2020-08-07T15:10:25.851 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.851 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:25.852 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.461846+0000", 2020-08-07T15:10:25.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9447999999999997e-05 2020-08-07T15:10:25.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:25.853 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.461905+0000", 2020-08-07T15:10:25.853 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32459016899999998 2020-08-07T15:10:25.853 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.853 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:25.854 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.854 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.854 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.854 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:31608 4.7 e100/81 4:e0ebdc25:::benchmark_data_smithi074_44586_object3950:head v 100'2136, mlcod=100'2126)", 2020-08-07T15:10:25.854 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:47.845848+0000", 2020-08-07T15:10:25.855 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 97.474292637000005, 2020-08-07T15:10:25.855 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27603951399999999, 2020-08-07T15:10:25.855 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:25.855 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:25.855 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:25.856 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.856 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:25.856 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:47.845848+0000", 2020-08-07T15:10:25.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:25.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:25.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:47.845848+0000", 2020-08-07T15:10:25.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3656999999999998e-05 2020-08-07T15:10:25.858 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.858 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:25.858 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:47.845902+0000", 2020-08-07T15:10:25.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4050000000000003e-06 2020-08-07T15:10:25.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.859 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.861 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:25.861 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:47.845909+0000", 2020-08-07T15:10:25.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27431144000000002 2020-08-07T15:10:25.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.862 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.862 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:25.862 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.120221+0000", 2020-08-07T15:10:25.862 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3640000000000004e-06 2020-08-07T15:10:25.862 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:25.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.120227+0000", 2020-08-07T15:10:25.863 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1541e-05 2020-08-07T15:10:25.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:25.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.120279+0000", 2020-08-07T15:10:25.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.372e-05 2020-08-07T15:10:25.864 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.865 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:25.865 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.120302+0000", 2020-08-07T15:10:25.865 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015294460000000001 2020-08-07T15:10:25.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.866 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:25.866 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.121832+0000", 2020-08-07T15:10:25.866 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5940999999999999e-05 2020-08-07T15:10:25.866 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.866 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.867 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:25.867 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.121888+0000", 2020-08-07T15:10:25.867 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27603951399999999 2020-08-07T15:10:25.867 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.867 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:25.868 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.868 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.868 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.868 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:31832 4.b e100/81 4:d005fdc5:::benchmark_data_smithi074_44586_object3978:head v 100'1872, mlcod=100'1863)", 2020-08-07T15:10:25.868 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:48.177094+0000", 2020-08-07T15:10:25.869 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 97.143046373000004, 2020-08-07T15:10:25.869 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26337036000000003, 2020-08-07T15:10:25.869 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:25.870 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:25.870 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:25.870 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.870 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:25.882 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.177094+0000", 2020-08-07T15:10:25.883 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:25.883 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.883 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.884 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:25.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.177094+0000", 2020-08-07T15:10:25.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3760999999999997e-05 2020-08-07T15:10:25.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:25.885 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.177128+0000", 2020-08-07T15:10:25.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1549999999999998e-06 2020-08-07T15:10:25.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.886 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.886 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:25.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.177133+0000", 2020-08-07T15:10:25.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.262460639 2020-08-07T15:10:25.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:25.887 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.439594+0000", 2020-08-07T15:10:25.887 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0943e-05 2020-08-07T15:10:25.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:25.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.439605+0000", 2020-08-07T15:10:25.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013696100000000001 2020-08-07T15:10:25.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:25.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.439742+0000", 2020-08-07T15:10:25.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5649000000000003e-05 2020-08-07T15:10:25.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.890 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:25.890 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.439787+0000", 2020-08-07T15:10:25.890 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00063599200000000005 2020-08-07T15:10:25.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.891 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.891 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:25.892 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.440423+0000", 2020-08-07T15:10:25.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1260000000000001e-05 2020-08-07T15:10:25.892 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.892 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.892 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:25.893 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.440465+0000", 2020-08-07T15:10:25.893 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26337036000000003 2020-08-07T15:10:25.893 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.893 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:25.893 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.894 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.894 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.894 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:42280 4.6 e112/81 4:6fc8ffaf:::benchmark_data_smithi074_44586_object5284:head v 112'2464, mlcod=112'2455)", 2020-08-07T15:10:25.894 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.113698+0000", 2020-08-07T15:10:25.894 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 85.206442980000006, 2020-08-07T15:10:25.894 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43055294, 2020-08-07T15:10:25.895 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:25.895 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:25.895 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:25.895 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.895 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:25.896 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.113698+0000", 2020-08-07T15:10:25.896 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:25.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.897 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.897 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:25.897 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.113698+0000", 2020-08-07T15:10:25.897 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2302000000000003e-05 2020-08-07T15:10:25.897 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.898 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.898 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:25.898 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.113740+0000", 2020-08-07T15:10:25.898 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3669999999999999e-06 2020-08-07T15:10:25.899 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.899 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.899 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:25.899 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.113744+0000", 2020-08-07T15:10:25.899 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.42958231000000002 2020-08-07T15:10:25.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.900 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:25.900 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.543327+0000", 2020-08-07T15:10:25.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2373e-05 2020-08-07T15:10:25.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.901 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.903 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:25.904 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.543339+0000", 2020-08-07T15:10:25.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014704099999999999 2020-08-07T15:10:25.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.904 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:25.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.543486+0000", 2020-08-07T15:10:25.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4834999999999999e-05 2020-08-07T15:10:25.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:25.906 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.543511+0000", 2020-08-07T15:10:25.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00069499500000000001 2020-08-07T15:10:25.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.906 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.906 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:25.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.544206+0000", 2020-08-07T15:10:25.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4716999999999999e-05 2020-08-07T15:10:25.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:25.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.544251+0000", 2020-08-07T15:10:25.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43055294 2020-08-07T15:10:25.908 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.908 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:25.908 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.909 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:60280 4.3 e121/81 4:c4915982:::benchmark_data_smithi074_44586_object7534:head v 121'3528, mlcod=121'3519)", 2020-08-07T15:10:25.909 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:13.849777+0000", 2020-08-07T15:10:25.909 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 71.470364238000002, 2020-08-07T15:10:25.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25718668300000003, 2020-08-07T15:10:25.910 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:25.910 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:25.910 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:25.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:25.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:13.849777+0000", 2020-08-07T15:10:25.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:25.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:25.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:13.849777+0000", 2020-08-07T15:10:25.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2033999999999997e-05 2020-08-07T15:10:25.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:25.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:13.849829+0000", 2020-08-07T15:10:25.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5230000000000001e-06 2020-08-07T15:10:25.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:25.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:13.849835+0000", 2020-08-07T15:10:25.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.256119861 2020-08-07T15:10:25.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:25.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.105955+0000", 2020-08-07T15:10:25.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.13e-06 2020-08-07T15:10:25.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:25.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.105960+0000", 2020-08-07T15:10:25.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8306000000000001e-05 2020-08-07T15:10:25.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:25.918 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.106008+0000", 2020-08-07T15:10:25.918 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4477e-05 2020-08-07T15:10:25.918 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.919 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:25.919 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.106033+0000", 2020-08-07T15:10:25.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00089500699999999996 2020-08-07T15:10:25.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:25.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.106928+0000", 2020-08-07T15:10:25.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5345000000000001e-05 2020-08-07T15:10:25.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.922 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:25.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:14.106963+0000", 2020-08-07T15:10:25.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25718668300000003 2020-08-07T15:10:25.923 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.923 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:25.923 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.924 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:72952 4.b e130/81 4:dee63f23:::benchmark_data_smithi074_44586_object9118:head v 130'4592, mlcod=130'4584)", 2020-08-07T15:10:25.924 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.589220+0000", 2020-08-07T15:10:25.925 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 59.730920584000003, 2020-08-07T15:10:25.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40730996899999999, 2020-08-07T15:10:25.927 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:25.927 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:25.927 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:25.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:25.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.589220+0000", 2020-08-07T15:10:25.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:25.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:25.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.589220+0000", 2020-08-07T15:10:25.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9273000000000001e-05 2020-08-07T15:10:25.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:25.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.589269+0000", 2020-08-07T15:10:25.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.417e-06 2020-08-07T15:10:25.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:25.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.589275+0000", 2020-08-07T15:10:25.931 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40641516700000002 2020-08-07T15:10:25.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:25.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.995690+0000", 2020-08-07T15:10:25.932 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7780000000000002e-06 2020-08-07T15:10:25.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.932 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.932 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:25.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.995697+0000", 2020-08-07T15:10:25.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3010999999999999e-05 2020-08-07T15:10:25.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.933 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.933 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:25.933 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.995740+0000", 2020-08-07T15:10:25.934 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1878999999999998e-05 2020-08-07T15:10:25.934 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.934 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:25.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.995762+0000", 2020-08-07T15:10:25.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00069480500000000005 2020-08-07T15:10:25.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:25.936 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.996456+0000", 2020-08-07T15:10:25.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3639000000000004e-05 2020-08-07T15:10:25.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:25.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.996530+0000", 2020-08-07T15:10:25.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40730996899999999 2020-08-07T15:10:25.937 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.937 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:25.938 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.938 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:85007 4.6 e139/81 4:6aadd387:::benchmark_data_smithi074_44586_object10625:head v 139'4991, mlcod=139'4981)", 2020-08-07T15:10:25.938 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:36.051294+0000", 2020-08-07T15:10:25.939 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 49.268847016999999, 2020-08-07T15:10:25.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28119156699999998, 2020-08-07T15:10:25.939 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:25.939 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:25.939 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:25.940 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.940 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:25.940 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.051294+0000", 2020-08-07T15:10:25.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:25.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:25.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.051294+0000", 2020-08-07T15:10:25.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8449000000000003e-05 2020-08-07T15:10:25.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:25.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.051342+0000", 2020-08-07T15:10:25.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9329999999999998e-06 2020-08-07T15:10:25.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:25.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.051348+0000", 2020-08-07T15:10:25.946 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28014436300000001 2020-08-07T15:10:25.946 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.946 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.946 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:25.946 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.331492+0000", 2020-08-07T15:10:25.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0309999999999998e-06 2020-08-07T15:10:25.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.947 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.947 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:25.947 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.331500+0000", 2020-08-07T15:10:25.948 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6211999999999997e-05 2020-08-07T15:10:25.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.948 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:25.948 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.331547+0000", 2020-08-07T15:10:25.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6117000000000001e-05 2020-08-07T15:10:25.949 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.949 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:25.949 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.331573+0000", 2020-08-07T15:10:25.950 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00085618800000000002 2020-08-07T15:10:25.950 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.950 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.950 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:25.950 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.332429+0000", 2020-08-07T15:10:25.950 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6273999999999997e-05 2020-08-07T15:10:25.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:25.951 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.332485+0000", 2020-08-07T15:10:25.952 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28119156699999998 2020-08-07T15:10:25.952 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.952 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:25.952 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.952 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.953 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.953 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:85008 4.6 e139/81 4:6aadd387:::benchmark_data_smithi074_44586_object10625:head v 139'4992, mlcod=139'4981)", 2020-08-07T15:10:25.953 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:36.051972+0000", 2020-08-07T15:10:25.954 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 49.268168340999999, 2020-08-07T15:10:25.954 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28367819599999999, 2020-08-07T15:10:25.954 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:25.954 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:25.954 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:25.955 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.955 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:25.955 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.051972+0000", 2020-08-07T15:10:25.956 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:25.956 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.956 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.956 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:25.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.051972+0000", 2020-08-07T15:10:25.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1882999999999998e-05 2020-08-07T15:10:25.957 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:25.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.052024+0000", 2020-08-07T15:10:25.958 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3609999999999997e-06 2020-08-07T15:10:25.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:25.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.052031+0000", 2020-08-07T15:10:25.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28155142999999999 2020-08-07T15:10:25.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:25.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.333582+0000", 2020-08-07T15:10:25.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8130000000000001e-06 2020-08-07T15:10:25.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:25.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.333588+0000", 2020-08-07T15:10:25.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4005000000000003e-05 2020-08-07T15:10:25.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.962 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:25.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.333632+0000", 2020-08-07T15:10:25.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2682999999999999e-05 2020-08-07T15:10:25.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:25.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.333655+0000", 2020-08-07T15:10:25.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019567569999999999 2020-08-07T15:10:25.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:25.964 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.335611+0000", 2020-08-07T15:10:25.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9264000000000002e-05 2020-08-07T15:10:25.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:25.966 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.335651+0000", 2020-08-07T15:10:25.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28367819599999999 2020-08-07T15:10:25.966 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.966 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:25.967 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.967 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:88160 4.6 e141/81 4:6eb8ba54:::benchmark_data_smithi074_44586_object11019:head v 141'5144, mlcod=141'5136)", 2020-08-07T15:10:25.967 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:37.976442+0000", 2020-08-07T15:10:25.968 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 47.343698373999999, 2020-08-07T15:10:25.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45938089599999998, 2020-08-07T15:10:25.968 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:25.968 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:25.968 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:25.969 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.969 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:25.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:37.976442+0000", 2020-08-07T15:10:25.969 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:25.969 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.970 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:25.970 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:37.976442+0000", 2020-08-07T15:10:25.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2018999999999998e-05 2020-08-07T15:10:25.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:25.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:37.976494+0000", 2020-08-07T15:10:25.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2230000000000004e-06 2020-08-07T15:10:25.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:25.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:37.976501+0000", 2020-08-07T15:10:25.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45734871399999999 2020-08-07T15:10:25.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:25.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:38.433849+0000", 2020-08-07T15:10:25.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7040000000000002e-06 2020-08-07T15:10:25.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:25.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:38.433854+0000", 2020-08-07T15:10:25.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.371e-05 2020-08-07T15:10:25.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.975 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:25.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:38.433898+0000", 2020-08-07T15:10:25.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6178e-05 2020-08-07T15:10:25.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:25.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:38.433914+0000", 2020-08-07T15:10:25.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0018471399999999999 2020-08-07T15:10:25.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:25.978 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:38.435761+0000", 2020-08-07T15:10:25.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2207999999999998e-05 2020-08-07T15:10:25.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.978 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:25.978 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:38.435823+0000", 2020-08-07T15:10:25.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45938089599999998 2020-08-07T15:10:25.979 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.979 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:25.979 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.979 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.980 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:104104 4.0 e146/131 4:0ffb764a:::benchmark_data_smithi074_44586_object13012:head v 146'6960, mlcod=146'6953)", 2020-08-07T15:10:25.980 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:10:25.980 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 37.121837863000003, 2020-08-07T15:10:25.980 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75443331599999996, 2020-08-07T15:10:25.981 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:25.981 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:25.981 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:25.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.981 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:25.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:10:25.982 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:25.982 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.983 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:25.984 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:10:25.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1743000000000002e-05 2020-08-07T15:10:25.985 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.985 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:25.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198355+0000", 2020-08-07T15:10:25.985 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1249999999999998e-06 2020-08-07T15:10:25.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.986 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:25.987 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198361+0000", 2020-08-07T15:10:25.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74719179599999996 2020-08-07T15:10:25.987 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:25.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945553+0000", 2020-08-07T15:10:25.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6800000000000001e-06 2020-08-07T15:10:25.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:25.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945557+0000", 2020-08-07T15:10:25.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012101500000000001 2020-08-07T15:10:25.990 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.990 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.990 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:25.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945678+0000", 2020-08-07T15:10:25.991 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6621000000000001e-05 2020-08-07T15:10:25.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:25.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945705+0000", 2020-08-07T15:10:25.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069879540000000002 2020-08-07T15:10:25.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:25.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.952693+0000", 2020-08-07T15:10:25.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3381999999999999e-05 2020-08-07T15:10:25.993 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:25.993 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:25.993 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:25.993 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.952736+0000", 2020-08-07T15:10:25.993 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75443331599999996 2020-08-07T15:10:25.994 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:25.994 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.002 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.003 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:128781 4.3 e155/81 4:c7b7232c:::benchmark_data_smithi074_44586_object12545:head v 155'7824, mlcod=154'7821)", 2020-08-07T15:10:26.003 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:03.440866+0000", 2020-08-07T15:10:26.003 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 21.879274957, 2020-08-07T15:10:26.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26739538699999998, 2020-08-07T15:10:26.004 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:26.004 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:26.004 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:26.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:26.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.440866+0000", 2020-08-07T15:10:26.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:26.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:26.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.440866+0000", 2020-08-07T15:10:26.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5170000000000001e-05 2020-08-07T15:10:26.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:26.026 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.440891+0000", 2020-08-07T15:10:26.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3590000000000001e-06 2020-08-07T15:10:26.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:26.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.440895+0000", 2020-08-07T15:10:26.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26665756499999999 2020-08-07T15:10:26.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:26.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.707553+0000", 2020-08-07T15:10:26.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9789999999999996e-06 2020-08-07T15:10:26.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:26.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.707560+0000", 2020-08-07T15:10:26.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9085999999999998e-05 2020-08-07T15:10:26.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.039 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:26.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.707599+0000", 2020-08-07T15:10:26.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8845999999999999e-05 2020-08-07T15:10:26.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:26.040 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.707618+0000", 2020-08-07T15:10:26.040 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000590597 2020-08-07T15:10:26.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.041 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:26.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.708208+0000", 2020-08-07T15:10:26.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2784999999999997e-05 2020-08-07T15:10:26.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.050 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.051 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:26.051 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.708261+0000", 2020-08-07T15:10:26.051 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26739538699999998 2020-08-07T15:10:26.051 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.051 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.052 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.052 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.052 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:128871 4.3 e155/81 4:cc9116ea:::benchmark_data_smithi074_44586_object12635:head v 155'7827, mlcod=155'7824)", 2020-08-07T15:10:26.052 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:03.727430+0000", 2020-08-07T15:10:26.053 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 21.592710922999999, 2020-08-07T15:10:26.053 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29142945999999997, 2020-08-07T15:10:26.054 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:26.054 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:26.054 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:26.054 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.054 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:26.054 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.727430+0000", 2020-08-07T15:10:26.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:26.055 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.055 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:26.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.727430+0000", 2020-08-07T15:10:26.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4294e-05 2020-08-07T15:10:26.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:26.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.727474+0000", 2020-08-07T15:10:26.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6970000000000001e-06 2020-08-07T15:10:26.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.058 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:26.058 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.727479+0000", 2020-08-07T15:10:26.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.290211159 2020-08-07T15:10:26.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:26.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:04.017690+0000", 2020-08-07T15:10:26.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5390000000000007e-06 2020-08-07T15:10:26.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.061 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:26.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:04.017699+0000", 2020-08-07T15:10:26.062 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0505000000000003e-05 2020-08-07T15:10:26.062 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.062 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.062 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:26.062 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:04.017779+0000", 2020-08-07T15:10:26.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0101000000000001e-05 2020-08-07T15:10:26.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.063 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:26.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:04.017819+0000", 2020-08-07T15:10:26.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00095239000000000001 2020-08-07T15:10:26.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.064 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:26.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:04.018772+0000", 2020-08-07T15:10:26.065 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7775000000000006e-05 2020-08-07T15:10:26.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.065 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:26.065 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:04.018859+0000", 2020-08-07T15:10:26.065 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29142945999999997 2020-08-07T15:10:26.066 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.066 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.066 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.066 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.066 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.067 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:26.067 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:10:26.073 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:10:26.074 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:10:26.074 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:10:26.074 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:10:26.075 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:10:26.076 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2020-08-07T15:10:24.658674+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":22216704,"num_objects":341,"num_object_clones":0,"num_object_copies":1023,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2712,"num_write_kb":21696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":135,"acting":135,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8433668,"kb_used_data":44996,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746541052,"statfs":{"total":773094113280,"available":764458037248,"internally_reserved":8589934592,"allocated":46075904,"data_stored":41258458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,3,5,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22216704,"num_objects":339,"num_object_clones":0,"num_object_copies":1017,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2712,"num_write_kb":21696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001083"},"pg_stats":[{"pgid":"1.14","version":"16'2","reported_seq":"100","reported_epoch":"167","state":"active+clean","last_fresh":"2020-08-07T15:10:18.519232+0000","last_change":"2020-08-07T15:10:10.644555+0000","last_active":"2020-08-07T15:10:18.519232+0000","last_peered":"2020-08-07T15:10:18.519232+0000","last_clean":"2020-08-07T15:10:18.519232+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:10:18.519232+0000","last_undegraded":"2020-08-07T15:10:18.519232+0000","last_fullsized":"2020-08-07T15:10:18.519232+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:10.644460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.644460+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"169","reported_epoch":"167","state":"active+clean","last_fresh":"2020-08-07T15:10:18.518757+0000","last_change":"2020-08-07T15:10:07.295971+0000","last_active":"2020-08-07T15:10:18.518757+0000","last_peered":"2020-08-07T15:10:18.518757+0000","last_clean":"2020-08-07T15:10:18.518757+0000","last_became_active":"2020-08-07T15:09:30.203592+0000","last_became_peered":"2020-08-07T15:09:30.203592+0000","last_unstale":"2020-08-07T15:10:18.518757+0000","last_undegraded":"2020-08-07T15:10:18.518757+0000","last_fullsized":"2020-08-07T15:10:18.518757+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:07.295863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:07.295863+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"97","reported_epoch":"167","state":"active+clean","last_fresh":"2020-08-07T15:10:18.269480+0000","last_change":"2020-08-07T15:08:55.078015+0000","last_active":"2020-08-07T15:10:18.269480+0000","last_peered":"2020-08-07T15:10:18.269480+0000","last_clean":"2020-08-07T15:10:18.269480+0000","last_became_active":"2020-08-07T15:08:41.685106+0000","last_became_peered":"2020-08-07T15:08:41.685106+0000","last_unstale":"2020-08-07T15:10:18.269480+0000","last_undegraded":"2020-08-07T15:10:18.269480+0000","last_fullsized":"2020-08-07T15:10:18.269480+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:55.077924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:55.077924+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"210","reported_epoch":"167","state":"active+clean","last_fresh":"2020-08-07T15:10:18.269476+0000","last_change":"2020-08-07T15:09:30.210669+0000","last_active":"2020-08-07T15:10:18.269476+0000","last_peered":"2020-08-07T15:10:18.269476+0000","last_clean":"2020-08-07T15:10:18.269476+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:10:18.269476+0000","last_undegraded":"2020-08-07T15:10:18.269476+0000","last_fullsized":"2020-08-07T15:10:18.269476+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:43.544324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:43.544324+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"204","reported_epoch":"167","state":"active+clean","last_fresh":"2020-08-07T15:10:18.269479+0000","last_change":"2020-08-07T15:08:51.599032+0000","last_active":"2020-08-07T15:10:18.269479+0000","last_peered":"2020-08-07T15:10:18.269479+0000","last_clean":"2020-08-07T15:10:18.269479+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:10:18.269479+0000","last_undegraded":"2020-08-07T15:10:18.269479+0000","last_fullsized":"2020-08-07T15:10:18.269479+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:51.598947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:51.598947+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.0","version":"168'128","reported_seq":"138","reported_epoch":"168","state":"active+clean","last_fresh":"2020-08-07T15:10:21.484921+0000","last_change":"2020-08-07T15:10:16.261122+0000","last_active":"2020-08-07T15:10:21.484921+0000","last_peered":"2020-08-07T15:10:21.484921+0000","last_clean":"2020-08-07T15:10:21.484921+0000","last_became_active":"2020-08-07T15:10:16.260690+0000","last_became_peered":"2020-08-07T15:10:16.260690+0000","last_unstale":"2020-08-07T15:10:21.484921+0000","last_undegraded":"2020-08-07T15:10:21.484921+0000","last_fullsized":"2020-08-07T15:10:21.484921+0000","mapping_epoch":164,"log_start":"168'121","ondisk_log_start":"168'121","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1048576,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"165","reported_epoch":"168","state":"active+clean","last_fresh":"2020-08-07T15:10:19.275778+0000","last_change":"2020-08-07T15:09:41.284321+0000","last_active":"2020-08-07T15:10:19.275778+0000","last_peered":"2020-08-07T15:10:19.275778+0000","last_clean":"2020-08-07T15:10:19.275778+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:10:19.275778+0000","last_undegraded":"2020-08-07T15:10:19.275778+0000","last_fullsized":"2020-08-07T15:10:19.275778+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:41.284237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:41.284237+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"80","reported_epoch":"167","state":"active+clean","last_fresh":"2020-08-07T15:10:18.519683+0000","last_change":"2020-08-07T15:09:30.283753+0000","last_active":"2020-08-07T15:10:18.519683+0000","last_peered":"2020-08-07T15:10:18.519683+0000","last_clean":"2020-08-07T15:10:18.519683+0000","last_became_active":"2020-08-07T15:09:30.283176+0000","last_became_peered":"2020-08-07T15:09:30.283176+0000","last_unstale":"2020-08-07T15:10:18.519683+0000","last_undegraded":"2020-08-07T15:10:18.519683+0000","last_fullsized":"2020-08-07T15:10:18.519683+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:25.606757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:25.606757+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"168'176","reported_seq":"187","reported_epoch":"168","state":"active+clean","last_fresh":"2020-08-07T15:10:21.686901+0000","last_change":"2020-08-07T15:10:16.259807+0000","last_active":"2020-08-07T15:10:21.686901+0000","last_peered":"2020-08-07T15:10:21.686901+0000","last_clean":"2020-08-07T15:10:21.686901+0000","last_became_active":"2020-08-07T15:10:16.259365+0000","last_became_peered":"2020-08-07T15:10:16.259365+0000","last_unstale":"2020-08-07T15:10:21.686901+0000","last_undegraded":"2020-08-07T15:10:21.686901+0000","last_fullsized":"2020-08-07T15:10:21.686901+0000","mapping_epoch":164,"log_start":"168'171","ondisk_log_start":"168'171","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"164","reported_epoch":"167","state":"active+clean","last_fresh":"2020-08-07T15:10:18.518680+0000","last_change":"2020-08-07T15:08:40.057705+0000","last_active":"2020-08-07T15:10:18.518680+0000","last_peered":"2020-08-07T15:10:18.518680+0000","last_clean":"2020-08-07T15:10:18.518680+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:10:18.518680+0000","last_undegraded":"2020-08-07T15:10:18.518680+0000","last_fullsized":"2020-08-07T15:10:18.518680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:08:40.057622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:08:40.057622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"122","reported_epoch":"167","state":"active+clean","last_fresh":"2020-08-07T15:10:18.519397+0000","last_change":"2020-08-07T15:08:53.756554+0000","last_active":"2020-08-07T15:10:18.519397+0000","last_peered":"2020-08-07T15:10:18.519397+0000","last_clean":"2020-08-07T15:10:18.519397+0000","last_became_active":"2020-08-07T15:08:46.710404+0000","last_became_peered":"2020-08-07T15:08:46.710404+0000","last_unstale":"2020-08-07T15:10:18.519397+0000","last_undegraded":"2020-08-07T15:10:18.519397+0000","last_fullsized":"2020-08-07T15:10:18.519397+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:08:53.756343+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:08:53.756343+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"169'176","reported_seq":"189","reported_epoch":"169","state":"active+clean","last_fresh":"2020-08-07T15:10:22.624736+0000","last_change":"2020-08-07T15:10:16.448360+0000","last_active":"2020-08-07T15:10:22.624736+0000","last_peered":"2020-08-07T15:10:22.624736+0000","last_clean":"2020-08-07T15:10:22.624736+0000","last_became_active":"2020-08-07T15:10:16.447826+0000","last_became_peered":"2020-08-07T15:10:16.447826+0000","last_unstale":"2020-08-07T15:10:22.624736+0000","last_undegraded":"2020-08-07T15:10:22.624736+0000","last_fullsized":"2020-08-07T15:10:22.624736+0000","mapping_epoch":164,"log_start":"169'169","ondisk_log_start":"169'169","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1441792,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"165","reported_epoch":"167","state":"active+clean","last_fresh":"2020-08-07T15:10:18.519614+0000","last_change":"2020-08-07T15:09:54.908283+0000","last_active":"2020-08-07T15:10:18.519614+0000","last_peered":"2020-08-07T15:10:18.519614+0000","last_clean":"2020-08-07T15:10:18.519614+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:10:18.519614+0000","last_undegraded":"2020-08-07T15:10:18.519614+0000","last_fullsized":"2020-08-07T15:10:18.519614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:54.908197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:54.908197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"204","reported_epoch":"168","state":"active+clean","last_fresh":"2020-08-07T15:10:19.275423+0000","last_change":"2020-08-07T15:10:07.302258+0000","last_active":"2020-08-07T15:10:19.275423+0000","last_peered":"2020-08-07T15:10:19.275423+0000","last_clean":"2020-08-07T15:10:19.275423+0000","last_became_active":"2020-08-07T15:09:30.201829+0000","last_became_peered":"2020-08-07T15:09:30.201829+0000","last_unstale":"2020-08-07T15:10:19.275423+0000","last_undegraded":"2020-08-07T15:10:19.275423+0000","last_fullsized":"2020-08-07T15:10:19.275423+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:07.302204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:07.302204+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"168'184","reported_seq":"196","reported_epoch":"168","state":"active+clean","last_fresh":"2020-08-07T15:10:21.598998+0000","last_change":"2020-08-07T15:10:16.447819+0000","last_active":"2020-08-07T15:10:21.598998+0000","last_peered":"2020-08-07T15:10:21.598998+0000","last_clean":"2020-08-07T15:10:21.598998+0000","last_became_active":"2020-08-07T15:10:16.447312+0000","last_became_peered":"2020-08-07T15:10:16.447312+0000","last_unstale":"2020-08-07T15:10:21.598998+0000","last_undegraded":"2020-08-07T15:10:21.598998+0000","last_fullsized":"2020-08-07T15:10:21.598998+0000","mapping_epoch":164,"log_start":"168'177","ondisk_log_start":"168'177","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1507328,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":184,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"165","reported_epoch":"167","state":"active+clean","last_fresh":"2020-08-07T15:10:18.518898+0000","last_change":"2020-08-07T15:09:51.036281+0000","last_active":"2020-08-07T15:10:18.518898+0000","last_peered":"2020-08-07T15:10:18.518898+0000","last_clean":"2020-08-07T15:10:18.518898+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:10:18.518898+0000","last_undegraded":"2020-08-07T15:10:18.518898+0000","last_fullsized":"2020-08-07T15:10:18.518898+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:51.036224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:51.036224+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"167","reported_epoch":"168","state":"active+clean","last_fresh":"2020-08-07T15:10:19.275143+0000","last_change":"2020-08-07T15:08:50.686578+0000","last_active":"2020-08-07T15:10:19.275143+0000","last_peered":"2020-08-07T15:10:19.275143+0000","last_clean":"2020-08-07T15:10:19.275143+0000","last_became_active":"2020-08-07T15:08:36.633406+0000","last_became_peered":"2020-08-07T15:08:36.633406+0000","last_unstale":"2020-08-07T15:10:19.275143+0000","last_undegraded":"2020-08-07T15:10:19.275143+0000","last_fullsized":"2020-08-07T15:10:19.275143+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:08:50.686496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:08:50.686496+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"168'152","reported_seq":"163","reported_epoch":"168","state":"active+clean","last_fresh":"2020-08-07T15:10:21.709798+0000","last_change":"2020-08-07T15:10:16.255436+0000","last_active":"2020-08-07T15:10:21.709798+0000","last_peered":"2020-08-07T15:10:21.709798+0000","last_clean":"2020-08-07T15:10:21.709798+0000","last_became_active":"2020-08-07T15:10:16.254967+0000","last_became_peered":"2020-08-07T15:10:16.254967+0000","last_unstale":"2020-08-07T15:10:21.709798+0000","last_undegraded":"2020-08-07T15:10:21.709798+0000","last_fullsized":"2020-08-07T15:10:21.709798+0000","mapping_epoch":164,"log_start":"168'144","ondisk_log_start":"168'144","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"197","reported_epoch":"167","state":"active+clean","last_fresh":"2020-08-07T15:10:18.518796+0000","last_change":"2020-08-07T15:09:55.879867+0000","last_active":"2020-08-07T15:10:18.518796+0000","last_peered":"2020-08-07T15:10:18.518796+0000","last_clean":"2020-08-07T15:10:18.518796+0000","last_became_active":"2020-08-07T15:08:36.627426+0000","last_became_peered":"2020-08-07T15:08:36.627426+0000","last_unstale":"2020-08-07T15:10:18.518796+0000","last_undegraded":"2020-08-07T15:10:18.518796+0000","last_fullsized":"2020-08-07T15:10:18.518796+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:55.879785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:55.879785+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"165","reported_epoch":"167","state":"active+clean","last_fresh":"2020-08-07T15:10:18.269816+0000","last_change":"2020-08-07T15:10:08.656952+0000","last_active":"2020-08-07T15:10:18.269816+0000","last_peered":"2020-08-07T15:10:18.269816+0000","last_clean":"2020-08-07T15:10:18.269816+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:10:18.269816+0000","last_undegraded":"2020-08-07T15:10:18.269816+0000","last_fullsized":"2020-08-07T15:10:18.269816+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:08.656855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:10:08.656855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"168'136","reported_seq":"148","reported_epoch":"168","state":"active+clean","last_fresh":"2020-08-07T15:10:21.597598+0000","last_change":"2020-08-07T15:10:16.263356+0000","last_active":"2020-08-07T15:10:21.597598+0000","last_peered":"2020-08-07T15:10:21.597598+0000","last_clean":"2020-08-07T15:10:21.597598+0000","last_became_active":"2020-08-07T15:10:16.262971+0000","last_became_peered":"2020-08-07T15:10:16.262971+0000","last_unstale":"2020-08-07T15:10:21.597598+0000","last_undegraded":"2020-08-07T15:10:21.597598+0000","last_fullsized":"2020-08-07T15:10:21.597598+0000","mapping_epoch":164,"log_start":"168'131","ondisk_log_start":"168'131","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":1088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"183","reported_epoch":"167","state":"active+clean","last_fresh":"2020-08-07T15:10:18.519169+0000","last_change":"2020-08-07T15:09:27.181676+0000","last_active":"2020-08-07T15:10:18.519169+0000","last_peered":"2020-08-07T15:10:18.519169+0000","last_clean":"2020-08-07T15:10:18.519169+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:10:18.519169+0000","last_undegraded":"2020-08-07T15:10:18.519169+0000","last_fullsized":"2020-08-07T15:10:18.519169+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:05.606603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:05.606603+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"213","reported_epoch":"167","state":"active+clean","last_fresh":"2020-08-07T15:10:18.518914+0000","last_change":"2020-08-07T15:10:11.912614+0000","last_active":"2020-08-07T15:10:18.518914+0000","last_peered":"2020-08-07T15:10:18.518914+0000","last_clean":"2020-08-07T15:10:18.518914+0000","last_became_active":"2020-08-07T15:08:36.628365+0000","last_became_peered":"2020-08-07T15:08:36.628365+0000","last_unstale":"2020-08-07T15:10:18.518914+0000","last_undegraded":"2020-08-07T15:10:18.518914+0000","last_fullsized":"2020-08-07T15:10:18.518914+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:11.912550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:11.912550+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.5","version":"168'192","reported_seq":"202","reported_epoch":"168","state":"active+clean","last_fresh":"2020-08-07T15:10:21.725404+0000","last_change":"2020-08-07T15:10:16.447870+0000","last_active":"2020-08-07T15:10:21.725404+0000","last_peered":"2020-08-07T15:10:21.725404+0000","last_clean":"2020-08-07T15:10:21.725404+0000","last_became_active":"2020-08-07T15:10:16.447478+0000","last_became_peered":"2020-08-07T15:10:16.447478+0000","last_unstale":"2020-08-07T15:10:21.725404+0000","last_undegraded":"2020-08-07T15:10:21.725404+0000","last_fullsized":"2020-08-07T15:10:21.725404+0000","mapping_epoch":164,"log_start":"168'187","ondisk_log_start":"168'187","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1572864,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":192,"num_write_kb":1536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"165","reported_epoch":"167","state":"active+clean","last_fresh":"2020-08-07T15:10:18.269679+0000","last_change":"2020-08-07T15:09:52.637867+0000","last_active":"2020-08-07T15:10:18.269679+0000","last_peered":"2020-08-07T15:10:18.269679+0000","last_clean":"2020-08-07T15:10:18.269679+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:10:18.269679+0000","last_undegraded":"2020-08-07T15:10:18.269679+0000","last_fullsized":"2020-08-07T15:10:18.269679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:52.637791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:09:52.637791+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"163","reported_epoch":"167","state":"active+clean","last_fresh":"2020-08-07T15:10:18.269536+0000","last_change":"2020-08-07T15:10:11.720217+0000","last_active":"2020-08-07T15:10:18.269536+0000","last_peered":"2020-08-07T15:10:18.269536+0000","last_clean":"2020-08-07T15:10:18.269536+0000","last_became_active":"2020-08-07T15:08:36.618164+0000","last_became_peered":"2020-08-07T15:08:36.618164+0000","last_unstale":"2020-08-07T15:10:18.269536+0000","last_undegraded":"2020-08-07T15:10:18.269536+0000","last_fullsized":"2020-08-07T15:10:18.269536+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:11.720143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:11.720143+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"169'248","reported_seq":"260","reported_epoch":"169","state":"active+clean","last_fresh":"2020-08-07T15:10:22.638066+0000","last_change":"2020-08-07T15:10:16.443153+0000","last_active":"2020-08-07T15:10:22.638066+0000","last_peered":"2020-08-07T15:10:22.638066+0000","last_clean":"2020-08-07T15:10:22.638066+0000","last_became_active":"2020-08-07T15:10:16.442674+0000","last_became_peered":"2020-08-07T15:10:16.442674+0000","last_unstale":"2020-08-07T15:10:22.638066+0000","last_undegraded":"2020-08-07T15:10:22.638066+0000","last_fullsized":"2020-08-07T15:10:22.638066+0000","mapping_epoch":164,"log_start":"169'240","ondisk_log_start":"169'240","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":248,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"162","reported_epoch":"167","state":"active+clean","last_fresh":"2020-08-07T15:10:18.519108+0000","last_change":"2020-08-07T15:09:37.017383+0000","last_active":"2020-08-07T15:10:18.519108+0000","last_peered":"2020-08-07T15:10:18.519108+0000","last_clean":"2020-08-07T15:10:18.519108+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:10:18.519108+0000","last_undegraded":"2020-08-07T15:10:18.519108+0000","last_fullsized":"2020-08-07T15:10:18.519108+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:37.017307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:37.017307+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"235","reported_epoch":"168","state":"active+clean","last_fresh":"2020-08-07T15:10:19.276002+0000","last_change":"2020-08-07T15:09:30.208873+0000","last_active":"2020-08-07T15:10:19.276002+0000","last_peered":"2020-08-07T15:10:19.276002+0000","last_clean":"2020-08-07T15:10:19.276002+0000","last_became_active":"2020-08-07T15:09:30.208345+0000","last_became_peered":"2020-08-07T15:09:30.208345+0000","last_unstale":"2020-08-07T15:10:19.276002+0000","last_undegraded":"2020-08-07T15:10:19.276002+0000","last_fullsized":"2020-08-07T15:10:19.276002+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.381325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.381325+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.e","version":"168'112","reported_seq":"124","reported_epoch":"168","state":"active+clean","last_fresh":"2020-08-07T15:10:21.658855+0000","last_change":"2020-08-07T15:10:16.448334+0000","last_active":"2020-08-07T15:10:21.658855+0000","last_peered":"2020-08-07T15:10:21.658855+0000","last_clean":"2020-08-07T15:10:21.658855+0000","last_became_active":"2020-08-07T15:10:16.447541+0000","last_became_peered":"2020-08-07T15:10:16.447541+0000","last_unstale":"2020-08-07T15:10:21.658855+0000","last_undegraded":"2020-08-07T15:10:21.658855+0000","last_fullsized":"2020-08-07T15:10:21.658855+0000","mapping_epoch":164,"log_start":"168'104","ondisk_log_start":"168'104","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":917504,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"140","reported_epoch":"167","state":"active+clean","last_fresh":"2020-08-07T15:10:18.269506+0000","last_change":"2020-08-07T15:10:10.715147+0000","last_active":"2020-08-07T15:10:18.269506+0000","last_peered":"2020-08-07T15:10:18.269506+0000","last_clean":"2020-08-07T15:10:18.269506+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:10:18.269506+0000","last_undegraded":"2020-08-07T15:10:18.269506+0000","last_fullsized":"2020-08-07T15:10:18.269506+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.715051+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"168'216","reported_seq":"228","reported_epoch":"168","state":"active+clean","last_fresh":"2020-08-07T15:10:21.720095+0000","last_change":"2020-08-07T15:10:16.448023+0000","last_active":"2020-08-07T15:10:21.720095+0000","last_peered":"2020-08-07T15:10:21.720095+0000","last_clean":"2020-08-07T15:10:21.720095+0000","last_became_active":"2020-08-07T15:10:16.447526+0000","last_became_peered":"2020-08-07T15:10:16.447526+0000","last_unstale":"2020-08-07T15:10:21.720095+0000","last_undegraded":"2020-08-07T15:10:21.720095+0000","last_fullsized":"2020-08-07T15:10:21.720095+0000","mapping_epoch":164,"log_start":"168'210","ondisk_log_start":"168'210","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1769472,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":216,"num_write_kb":1728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"137","reported_epoch":"167","state":"active+clean","last_fresh":"2020-08-07T15:10:18.269479+0000","last_change":"2020-08-07T15:09:09.691838+0000","last_active":"2020-08-07T15:10:18.269479+0000","last_peered":"2020-08-07T15:10:18.269479+0000","last_clean":"2020-08-07T15:10:18.269479+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:10:18.269479+0000","last_undegraded":"2020-08-07T15:10:18.269479+0000","last_fullsized":"2020-08-07T15:10:18.269479+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.691753+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.d","version":"168'152","reported_seq":"164","reported_epoch":"168","state":"active+clean","last_fresh":"2020-08-07T15:10:21.569432+0000","last_change":"2020-08-07T15:10:16.263289+0000","last_active":"2020-08-07T15:10:21.569432+0000","last_peered":"2020-08-07T15:10:21.569432+0000","last_clean":"2020-08-07T15:10:21.569432+0000","last_became_active":"2020-08-07T15:10:16.262869+0000","last_became_peered":"2020-08-07T15:10:16.262869+0000","last_unstale":"2020-08-07T15:10:21.569432+0000","last_undegraded":"2020-08-07T15:10:21.569432+0000","last_fullsized":"2020-08-07T15:10:21.569432+0000","mapping_epoch":164,"log_start":"168'145","ondisk_log_start":"168'145","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"201","reported_epoch":"167","state":"active+clean","last_fresh":"2020-08-07T15:10:18.518908+0000","last_change":"2020-08-07T15:09:25.984108+0000","last_active":"2020-08-07T15:10:18.518908+0000","last_peered":"2020-08-07T15:10:18.518908+0000","last_clean":"2020-08-07T15:10:18.518908+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:10:18.518908+0000","last_undegraded":"2020-08-07T15:10:18.518908+0000","last_fullsized":"2020-08-07T15:10:18.518908+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:25.984018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:25.984018+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"168'136","reported_seq":"146","reported_epoch":"168","state":"active+clean","last_fresh":"2020-08-07T15:10:21.628536+0000","last_change":"2020-08-07T15:10:16.256000+0000","last_active":"2020-08-07T15:10:21.628536+0000","last_peered":"2020-08-07T15:10:21.628536+0000","last_clean":"2020-08-07T15:10:21.628536+0000","last_became_active":"2020-08-07T15:10:16.255356+0000","last_became_peered":"2020-08-07T15:10:16.255356+0000","last_unstale":"2020-08-07T15:10:21.628536+0000","last_undegraded":"2020-08-07T15:10:21.628536+0000","last_fullsized":"2020-08-07T15:10:21.628536+0000","mapping_epoch":164,"log_start":"168'127","ondisk_log_start":"168'127","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":1088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"159","reported_epoch":"167","state":"active+clean","last_fresh":"2020-08-07T15:10:18.519609+0000","last_change":"2020-08-07T15:10:10.962565+0000","last_active":"2020-08-07T15:10:18.519609+0000","last_peered":"2020-08-07T15:10:18.519609+0000","last_clean":"2020-08-07T15:10:18.519609+0000","last_became_active":"2020-08-07T15:08:41.680448+0000","last_became_peered":"2020-08-07T15:08:41.680448+0000","last_unstale":"2020-08-07T15:10:18.519609+0000","last_undegraded":"2020-08-07T15:10:18.519609+0000","last_fullsized":"2020-08-07T15:10:18.519609+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:10:10.962480+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.962480+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"168'208","reported_seq":"219","reported_epoch":"168","state":"active+clean","last_fresh":"2020-08-07T15:10:21.698594+0000","last_change":"2020-08-07T15:10:16.444776+0000","last_active":"2020-08-07T15:10:21.698594+0000","last_peered":"2020-08-07T15:10:21.698594+0000","last_clean":"2020-08-07T15:10:21.698594+0000","last_became_active":"2020-08-07T15:10:16.444316+0000","last_became_peered":"2020-08-07T15:10:16.444316+0000","last_unstale":"2020-08-07T15:10:21.698594+0000","last_undegraded":"2020-08-07T15:10:21.698594+0000","last_fullsized":"2020-08-07T15:10:21.698594+0000","mapping_epoch":164,"log_start":"168'203","ondisk_log_start":"168'203","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1703936,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":208,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"126","reported_epoch":"167","state":"active+clean","last_fresh":"2020-08-07T15:10:18.519797+0000","last_change":"2020-08-07T15:09:36.687393+0000","last_active":"2020-08-07T15:10:18.519797+0000","last_peered":"2020-08-07T15:10:18.519797+0000","last_clean":"2020-08-07T15:10:18.519797+0000","last_became_active":"2020-08-07T15:09:30.204655+0000","last_became_peered":"2020-08-07T15:09:30.204655+0000","last_unstale":"2020-08-07T15:10:18.519797+0000","last_undegraded":"2020-08-07T15:10:18.519797+0000","last_fullsized":"2020-08-07T15:10:18.519797+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:36.687314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:36.687314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"168'120","reported_seq":"131","reported_epoch":"168","state":"active+clean","last_fresh":"2020-08-07T15:10:21.670630+0000","last_change":"2020-08-07T15:10:16.444970+0000","last_active":"2020-08-07T15:10:21.670630+0000","last_peered":"2020-08-07T15:10:21.670630+0000","last_clean":"2020-08-07T15:10:21.670630+0000","last_became_active":"2020-08-07T15:10:16.444294+0000","last_became_peered":"2020-08-07T15:10:16.444294+0000","last_unstale":"2020-08-07T15:10:21.670630+0000","last_undegraded":"2020-08-07T15:10:21.670630+0000","last_fullsized":"2020-08-07T15:10:21.670630+0000","mapping_epoch":164,"log_start":"168'117","ondisk_log_start":"168'117","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"200","reported_epoch":"168","state":"active+clean","last_fresh":"2020-08-07T15:10:19.275728+0000","last_change":"2020-08-07T15:09:08.399507+0000","last_active":"2020-08-07T15:10:19.275728+0000","last_peered":"2020-08-07T15:10:19.275728+0000","last_clean":"2020-08-07T15:10:19.275728+0000","last_became_active":"2020-08-07T15:08:41.689435+0000","last_became_peered":"2020-08-07T15:08:41.689435+0000","last_unstale":"2020-08-07T15:10:19.275728+0000","last_undegraded":"2020-08-07T15:10:19.275728+0000","last_fullsized":"2020-08-07T15:10:19.275728+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:08.399439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:08.399439+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"168'152","reported_seq":"163","reported_epoch":"168","state":"active+clean","last_fresh":"2020-08-07T15:10:21.669208+0000","last_change":"2020-08-07T15:10:16.256758+0000","last_active":"2020-08-07T15:10:21.669208+0000","last_peered":"2020-08-07T15:10:21.669208+0000","last_clean":"2020-08-07T15:10:21.669208+0000","last_became_active":"2020-08-07T15:10:16.256419+0000","last_became_peered":"2020-08-07T15:10:16.256419+0000","last_unstale":"2020-08-07T15:10:21.669208+0000","last_undegraded":"2020-08-07T15:10:21.669208+0000","last_fullsized":"2020-08-07T15:10:21.669208+0000","mapping_epoch":164,"log_start":"168'146","ondisk_log_start":"168'146","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":152,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"87","reported_epoch":"167","state":"active+clean","last_fresh":"2020-08-07T15:10:18.519674+0000","last_change":"2020-08-07T15:09:09.868283+0000","last_active":"2020-08-07T15:10:18.519674+0000","last_peered":"2020-08-07T15:10:18.519674+0000","last_clean":"2020-08-07T15:10:18.519674+0000","last_became_active":"2020-08-07T15:08:46.710059+0000","last_became_peered":"2020-08-07T15:08:46.710059+0000","last_unstale":"2020-08-07T15:10:18.519674+0000","last_undegraded":"2020-08-07T15:10:18.519674+0000","last_fullsized":"2020-08-07T15:10:18.519674+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.868198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.868198+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"168'224","reported_seq":"232","reported_epoch":"168","state":"active+clean","last_fresh":"2020-08-07T15:10:21.717105+0000","last_change":"2020-08-07T15:10:16.253662+0000","last_active":"2020-08-07T15:10:21.717105+0000","last_peered":"2020-08-07T15:10:21.717105+0000","last_clean":"2020-08-07T15:10:21.717105+0000","last_became_active":"2020-08-07T15:10:16.253105+0000","last_became_peered":"2020-08-07T15:10:16.253105+0000","last_unstale":"2020-08-07T15:10:21.717105+0000","last_undegraded":"2020-08-07T15:10:21.717105+0000","last_fullsized":"2020-08-07T15:10:21.717105+0000","mapping_epoch":164,"log_start":"168'214","ondisk_log_start":"168'214","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":224,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"109","reported_epoch":"167","state":"active+clean","last_fresh":"2020-08-07T15:10:18.269754+0000","last_change":"2020-08-07T15:09:08.653773+0000","last_active":"2020-08-07T15:10:18.269754+0000","last_peered":"2020-08-07T15:10:18.269754+0000","last_clean":"2020-08-07T15:10:18.269754+0000","last_became_active":"2020-08-07T15:08:41.677944+0000","last_became_peered":"2020-08-07T15:08:41.677944+0000","last_unstale":"2020-08-07T15:10:18.269754+0000","last_undegraded":"2020-08-07T15:10:18.269754+0000","last_fullsized":"2020-08-07T15:10:18.269754+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:08.653722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:08.653722+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":22216704,"num_objects":339,"num_object_clones":0,"num_object_copies":1017,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2712,"num_write_kb":21696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65413120,"data_stored":65413120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106,"ondisk_log_size":106,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":63,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542195,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1056344,"kb_used_data":7760,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93315496,"statfs":{"total":96636764160,"available":95555067904,"internally_reserved":1073741824,"allocated":7946240,"data_stored":7337439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.95099999999999996,"15min":0.91200000000000003},"min":{"1min":0.437,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.1739999999999999,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.0149999999999999,"15min":0.97999999999999998},"min":{"1min":0.54300000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":0.73699999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.357,"15min":1.3149999999999999},"min":{"1min":0.68700000000000006,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":5.6239999999999997,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":1.623},{"interface":"front","average":{"1min":1.105,"5min":1.0720000000000001,"15min":1.0009999999999999},"min":{"1min":0.64100000000000001,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.206,"5min":3.4620000000000002,"15min":3.4620000000000002},"last":1.292}]},{"osd":2,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.272,"15min":1.175},"min":{"1min":0.71999999999999997,"5min":0.628,"15min":0.628},"max":{"1min":5.6619999999999999,"5min":5.6619999999999999,"15min":5.6619999999999999},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.446,"5min":1.26,"15min":1.165},"min":{"1min":0.66800000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":5.1319999999999997,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":75.045000000000002,"5min":16.263999999999999,"15min":6.5359999999999996},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":917.38699999999994,"5min":917.38699999999994,"15min":917.38699999999994},"last":1.0760000000000001},{"interface":"front","average":{"1min":19.670999999999999,"5min":4.9939999999999998,"15min":2.5600000000000001},"min":{"1min":0.80000000000000004,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":333.59199999999998,"5min":333.59199999999998,"15min":333.59199999999998},"last":0.92300000000000004}]},{"osd":4,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.411,"15min":1.321},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":4.5250000000000004,"5min":4.835,"15min":4.835},"last":0.88100000000000001},{"interface":"front","average":{"1min":52.259999999999998,"5min":11.52,"15min":4.7140000000000004},"min":{"1min":0.77000000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":866.24099999999999,"5min":866.24099999999999,"15min":866.24099999999999},"last":0.77300000000000002}]},{"osd":5,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.2869999999999999,"5min":23.440000000000001,"15min":32.817999999999998},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":6.6509999999999998,"5min":831.88099999999997,"15min":831.88099999999997},"last":1.3200000000000001},{"interface":"front","average":{"1min":2.2519999999999998,"5min":1.8380000000000001,"15min":1.609},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":6.6790000000000003,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":1.407}]},{"osd":7,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":13.818,"5min":4.0019999999999998,"15min":2.3460000000000001},"min":{"1min":0.82999999999999996,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":224.14500000000001,"5min":224.14500000000001,"15min":224.14500000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.9770000000000001,"5min":1.6839999999999999,"15min":1.5660000000000001},"min":{"1min":0.90600000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":7.4359999999999999,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":1.5920000000000001}]}]},{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1052932,"kb_used_data":4348,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93318908,"statfs":{"total":96636764160,"available":95558561792,"internally_reserved":1073741824,"allocated":4452352,"data_stored":3851117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,5,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.179,"15min":1.2809999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.8149999999999999,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":15.239000000000001,"5min":3.9529999999999998,"15min":2.0840000000000001},"min":{"1min":0.78100000000000003,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":320.31299999999999,"5min":320.31299999999999,"15min":320.31299999999999},"last":0.67500000000000004}]},{"osd":1,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":24.821999999999999,"5min":5.7329999999999997,"15min":2.5489999999999999},"min":{"1min":0.46700000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":547.65300000000002,"5min":547.65300000000002,"15min":547.65300000000002},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":4.9130000000000003,"15min":2.5619999999999998},"min":{"1min":0.69799999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.5449999999999999,"5min":372.50799999999998,"15min":372.50799999999998},"last":0.57399999999999995}]},{"osd":2,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.208,"15min":1.1990000000000001},"min":{"1min":0.502,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":5.2400000000000002,"5min":5.2400000000000002,"15min":5.2400000000000002},"last":2.7429999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.2390000000000001,"15min":1.2609999999999999},"min":{"1min":0.66800000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":6.0490000000000004,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":1.21}]},{"osd":3,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.4370000000000001,"15min":1.496},"min":{"1min":0.878,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.0670000000000002,"5min":3.46,"15min":3.46},"last":0.72299999999999998},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3600000000000001,"15min":1.3779999999999999},"min":{"1min":0.70299999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":5.2770000000000001,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":1.8959999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.504,"15min":1.6100000000000001},"min":{"1min":0.70099999999999996,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.165,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.46,"5min":1.625,"15min":1.6559999999999999},"min":{"1min":0.96199999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":2.4279999999999999,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.841}]},{"osd":5,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.2809999999999999,"15min":1.3360000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.3069999999999999,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":0.75},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.6259999999999999,"15min":1.7190000000000001},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.4630000000000001,"5min":5.327,"15min":5.327},"last":1.0429999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:09:21 2020","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.458,"15min":1.4410000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.4900000000000002,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.075},{"interface":"front","average":{"1min":1.5720000000000001,"5min":11.603999999999999,"15min":16.085000000000001},"min":{"1min":1.004,"5min":0.87,"15min":0.87},"max":{"1min":2.7719999999999998,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.1859999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1055172,"kb_used_data":6588,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93316668,"statfs":{"total":96636764160,"available":95556268032,"internally_reserved":1073741824,"allocated":6746112,"data_stored":6144877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":13.122,"5min":16.731999999999999,"15min":28.367999999999999},"min":{"1min":0.47199999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":211.63399999999999,"5min":497.04000000000002,"15min":497.04000000000002},"last":1.321},{"interface":"front","average":{"1min":1.645,"5min":14.757999999999999,"15min":5.673},"min":{"1min":0.57399999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":8.8849999999999998,"5min":993.84500000000003,"15min":993.84500000000003},"last":1.4079999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":54.984000000000002,"5min":11.816000000000001,"15min":4.5380000000000003},"min":{"1min":0.57199999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":959.36900000000003,"5min":959.36900000000003,"15min":959.36900000000003},"last":1.351},{"interface":"front","average":{"1min":1.72,"5min":1.357,"15min":1.2549999999999999},"min":{"1min":0.53600000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":5.4530000000000003,"5min":6.4029999999999996,"15min":6.4029999999999996},"last":0.58399999999999996}]},{"osd":3,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":9.2420000000000009,"15min":3.9540000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.3879999999999999,"5min":828.31600000000003,"15min":828.31600000000003},"last":2.0710000000000002},{"interface":"front","average":{"1min":1.2130000000000001,"5min":10.119999999999999,"15min":4.3419999999999996},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.5659999999999998,"5min":905.38300000000004,"15min":905.38300000000004},"last":2.101}]},{"osd":4,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":49.530999999999999,"5min":16.422999999999998,"15min":6.3300000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":872.58500000000004,"5min":872.58500000000004,"15min":872.58500000000004},"last":2.1829999999999998},{"interface":"front","average":{"1min":2.2080000000000002,"5min":1.5409999999999999,"15min":1.3129999999999999},"min":{"1min":0.75800000000000001,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":12.640000000000001,"5min":12.640000000000001,"15min":12.640000000000001},"last":1.4359999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":6.0919999999999996,"15min":10.702},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.056,"5min":213.63800000000001,"15min":213.63800000000001},"last":1.383},{"interface":"front","average":{"1min":2.4630000000000001,"5min":12.974,"15min":5.1180000000000003},"min":{"1min":0.93400000000000005,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":12.677,"5min":875.44600000000003,"15min":875.44600000000003},"last":2.1539999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:09:27 2020","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.587,"15min":1.5629999999999999},"min":{"1min":0.81000000000000005,"5min":0.754,"15min":0.754},"max":{"1min":5.7149999999999999,"5min":5.7149999999999999,"15min":5.7149999999999999},"last":1.256},{"interface":"front","average":{"1min":11.539,"5min":3.5979999999999999,"15min":2.1059999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":207.137,"5min":207.137,"15min":207.137},"last":2.3980000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":2.0910000000000002,"5min":1.7829999999999999,"15min":1.671},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":5.4960000000000004,"5min":6.444,"15min":6.444},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.655,"15min":1.538},"min":{"1min":0.78700000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.0219999999999998,"5min":10.752000000000001,"15min":10.752000000000001},"last":2.3690000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1052868,"kb_used_data":4284,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93318972,"statfs":{"total":96636764160,"available":95558627328,"internally_reserved":1073741824,"allocated":4386816,"data_stored":3785581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":17.058,"15min":6.3369999999999997},"min":{"1min":0.61399999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.1269999999999998,"5min":845.57100000000003,"15min":845.57100000000003},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.0169999999999999,"15min":0.96099999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":4.8209999999999997,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":0.98199999999999998}]},{"osd":2,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.0189999999999999,"15min":0.95799999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.7719999999999998,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":13.177,"5min":22.48,"15min":29.32},"min":{"1min":0.61399999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":226.911,"5min":606.18499999999995,"15min":606.18499999999995},"last":0.72799999999999998}]},{"osd":3,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.0660000000000001,"15min":0.97899999999999998},"min":{"1min":0.54100000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6859999999999999,"5min":3.1779999999999999,"15min":3.1779999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.3160000000000001,"15min":1.2170000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.976,"5min":4.21,"15min":4.21},"last":1.018}]},{"osd":4,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.2829999999999999,"15min":1.2050000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":8.3100000000000005,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.7709999999999999,"5min":1.446,"15min":1.3200000000000001},"min":{"1min":0.84599999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":8.4190000000000005,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":1.042}]},{"osd":5,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.9770000000000001,"5min":9.0380000000000003,"15min":3.8300000000000001},"min":{"1min":0.77800000000000002,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":7.8929999999999998,"5min":501.267,"15min":501.267},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.2609999999999999,"15min":1.157},"min":{"1min":0.78200000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.0939999999999999,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":1.254}]},{"osd":6,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":5.7779999999999996,"15min":7.6680000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.0059999999999998,"5min":124.738,"15min":124.738},"last":1.151},{"interface":"front","average":{"1min":1.627,"5min":11.811,"15min":16.372},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":4.6879999999999997,"5min":294.94,"15min":294.94},"last":1.302}]},{"osd":7,"last update":"Fri Aug 7 15:09:25 2020","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.5589999999999999,"15min":1.577},"min":{"1min":0.95399999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":4.7169999999999996,"5min":6.468,"15min":6.468},"last":1.22},{"interface":"front","average":{"1min":1.363,"5min":1.4410000000000001,"15min":1.4039999999999999},"min":{"1min":0.88800000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.1040000000000001,"5min":3.117,"15min":3.117},"last":1.177}]}]},{"osd":2,"up_from":13,"seq":55834574899,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1056460,"kb_used_data":7876,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93315380,"statfs":{"total":96636764160,"available":95554949120,"internally_reserved":1073741824,"allocated":8065024,"data_stored":7463789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.6950000000000001,"5min":1.268,"15min":1.167},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":8.4930000000000003,"5min":8.4930000000000003,"15min":8.4930000000000003},"last":0.59899999999999998},{"interface":"front","average":{"1min":1.357,"5min":1.113,"15min":1.0209999999999999},"min":{"1min":0.61899999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":3.5,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.97799999999999998}]},{"osd":1,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.9390000000000001,"5min":1.5469999999999999,"15min":1.2430000000000001},"min":{"1min":0.54400000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":5.6820000000000004,"5min":18.672000000000001,"15min":18.672000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.351,"15min":1.357},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":3.585,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":1.163}]},{"osd":3,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.268,"15min":1.198},"min":{"1min":0.72799999999999998,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.8740000000000001,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":1.087},{"interface":"front","average":{"1min":34.170999999999999,"5min":8.0510000000000002,"15min":3.7210000000000001},"min":{"1min":0.68799999999999994,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":501.75599999999997,"5min":501.75599999999997,"15min":501.75599999999997},"last":0.70299999999999996}]},{"osd":4,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.403,"15min":1.2070000000000001},"min":{"1min":0.65200000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":3.4180000000000001,"5min":19.783000000000001,"15min":19.783000000000001},"last":1.23},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.546,"15min":1.6100000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":3.5569999999999999,"5min":3.5569999999999999,"15min":3.5569999999999999},"last":1.2589999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.292,"15min":1.272},"min":{"1min":0.83299999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.323,"5min":2.7730000000000001,"15min":2.7730000000000001},"last":1.2909999999999999},{"interface":"front","average":{"1min":43.396999999999998,"5min":24.298999999999999,"15min":25.116},"min":{"1min":0.99299999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":627.53300000000002,"5min":627.53300000000002,"15min":627.53300000000002},"last":1.339}]},{"osd":6,"last update":"Fri Aug 7 15:09:30 2020","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.538,"15min":1.3260000000000001},"min":{"1min":0.85999999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":3.452,"5min":20.035,"15min":20.035},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.498,"5min":9.8979999999999997,"15min":4.1799999999999997},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.976,"5min":751.70000000000005,"15min":751.70000000000005},"last":1.365}]},{"osd":7,"last update":"Fri Aug 7 15:09:20 2020","interfaces":[{"interface":"back","average":{"1min":54.209000000000003,"5min":12.595000000000001,"15min":5.359},"min":{"1min":0.84999999999999998,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":501.91699999999997,"5min":501.91699999999997,"15min":501.91699999999997},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.665,"5min":1.919,"15min":1.637},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":3.133,"5min":35.646999999999998,"15min":35.646999999999998},"last":1.3109999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1052932,"kb_used_data":4348,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93318908,"statfs":{"total":96636764160,"available":95558561792,"internally_reserved":1073741824,"allocated":4452352,"data_stored":3851117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,3,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":1.341,"15min":1.282},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.097,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.4530000000000001,"15min":1.575},"min":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":2.3039999999999998,"5min":13.067,"15min":13.067},"last":0.67800000000000005}]},{"osd":1,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":4.5860000000000003,"15min":2.794},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.9910000000000001,"5min":270.68700000000001,"15min":270.68700000000001},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.4379999999999999,"5min":2.0390000000000001,"15min":2.0209999999999999},"min":{"1min":0.55100000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":7.0940000000000003,"5min":14.349,"15min":14.349},"last":1.825}]},{"osd":2,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.266,"15min":1.1870000000000001},"min":{"1min":0.71399999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.2639999999999998,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":1.5269999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":2.109,"15min":1.8260000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.661,"5min":51.530000000000001,"15min":51.530000000000001},"last":2.9420000000000002}]},{"osd":4,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":10.773999999999999,"15min":4.7220000000000004},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.1360000000000001,"5min":506.21300000000002,"15min":506.21300000000002},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.276,"5min":25.469000000000001,"15min":29.888999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":2.8940000000000001,"5min":677.88599999999997,"15min":677.88599999999997},"last":1.4730000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.5920000000000001,"15min":1.542},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.9710000000000001,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.728},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.8009999999999999,"15min":1.7789999999999999},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":3.044,"5min":5.4329999999999998,"15min":5.4329999999999998},"last":1.7709999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.6259999999999999,"15min":1.5629999999999999},"min":{"1min":0.89800000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.9359999999999999,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":1.679},{"interface":"front","average":{"1min":21.766999999999999,"5min":5.9390000000000001,"15min":3.3039999999999998},"min":{"1min":0.89900000000000002,"5min":0.876,"15min":0.876},"max":{"1min":387.911,"5min":387.911,"15min":387.911},"last":1.9079999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:09:24 2020","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.9330000000000001,"15min":2.085},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.0110000000000001,"5min":5.734,"15min":5.734},"last":1.865},{"interface":"front","average":{"1min":1.325,"5min":1.9750000000000001,"15min":1.976},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.169,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":1.218}]}]},{"osd":4,"up_from":13,"seq":55834574899,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1053452,"kb_used_data":4868,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93318388,"statfs":{"total":96636764160,"available":95558029312,"internally_reserved":1073741824,"allocated":4984832,"data_stored":4383597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":1.0720000000000001,"15min":1.1439999999999999},"min":{"1min":0.52900000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4390000000000001,"5min":4.9480000000000004,"15min":4.9480000000000004},"last":1.175},{"interface":"front","average":{"1min":1.143,"5min":1.0069999999999999,"15min":0.90100000000000002},"min":{"1min":0.498,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":4.907,"5min":6.3970000000000002,"15min":6.3970000000000002},"last":0.71099999999999997}]},{"osd":1,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":23.940999999999999,"5min":5.8209999999999997,"15min":2.843},"min":{"1min":0.59599999999999997,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":320.67000000000002,"5min":320.67000000000002,"15min":320.67000000000002},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.1359999999999999,"5min":7.5700000000000003,"15min":3.4409999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.784,"5min":689.846,"15min":689.846},"last":1.268}]},{"osd":2,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.3999999999999999,"15min":1.4339999999999999},"min":{"1min":0.77600000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.0899999999999999,"5min":6.8730000000000002,"15min":6.8730000000000002},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.258,"5min":1.516,"15min":1.5149999999999999},"min":{"1min":0.81699999999999995,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.3159999999999998,"5min":7.1360000000000001,"15min":7.1360000000000001},"last":0.98499999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":3.681,"15min":2.2320000000000002},"min":{"1min":0.78300000000000003,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.8149999999999999,"5min":247.202,"15min":247.202},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.3720000000000001,"15min":1.2909999999999999},"min":{"1min":0.68500000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":4.931,"5min":6.431,"15min":6.431},"last":1.244}]},{"osd":5,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":10.57,"15min":4.6079999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.431,"5min":990.71600000000001,"15min":990.71600000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.6100000000000001,"15min":1.621},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":3.3519999999999999,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":1.3360000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:09:28 2020","interfaces":[{"interface":"back","average":{"1min":1.6799999999999999,"5min":1.7509999999999999,"15min":1.7769999999999999},"min":{"1min":0.75800000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":5.0149999999999997,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":1.083},{"interface":"front","average":{"1min":1.498,"5min":12.069000000000001,"15min":16.762},"min":{"1min":0.83299999999999996,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":3.3530000000000002,"5min":389.14600000000002,"15min":389.14600000000002},"last":1.143}]},{"osd":7,"last update":"Fri Aug 7 15:09:22 2020","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":3.8300000000000001,"15min":2.2149999999999999},"min":{"1min":0.753,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":4.8710000000000004,"5min":252.798,"15min":252.798},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.262,"5min":10.175000000000001,"15min":4.4889999999999999},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.3050000000000002,"5min":501.93000000000001,"15min":501.93000000000001},"last":1.01}]}]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1053508,"kb_used_data":4924,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93318332,"statfs":{"total":96636764160,"available":95557971968,"internally_reserved":1073741824,"allocated":5042176,"data_stored":4440941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":1.012,"15min":1.0549999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.651,"5min":2.766,"15min":2.766},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91600000000000004,"15min":0.84199999999999997},"min":{"1min":0.45500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.3340000000000001,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":0.98199999999999998}]},{"osd":1,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.4540000000000002,"5min":10.521000000000001,"15min":4.1260000000000003},"min":{"1min":0.56699999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":29.283000000000001,"5min":787.22900000000004,"15min":787.22900000000004},"last":1.1950000000000001},{"interface":"front","average":{"1min":8.0939999999999994,"5min":2.4969999999999999,"15min":1.571},"min":{"1min":0.60999999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":116.548,"5min":116.548,"15min":116.548},"last":0.91800000000000004}]},{"osd":2,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":5.9340000000000002,"15min":8.1240000000000006},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.78,"5min":168.56899999999999,"15min":168.56899999999999},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.119,"5min":20.361000000000001,"15min":28.873999999999999},"min":{"1min":0.60899999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.665,"5min":676.27099999999996,"15min":676.27099999999996},"last":0.85799999999999998}]},{"osd":3,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":2.6739999999999999,"5min":1.5069999999999999,"15min":1.2470000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":29.129000000000001,"5min":29.129000000000001,"15min":29.129000000000001},"last":1.1299999999999999},{"interface":"front","average":{"1min":3.0219999999999998,"5min":9.9459999999999997,"15min":4.1239999999999997},"min":{"1min":0.70799999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":29.103000000000002,"5min":712.98699999999997,"15min":712.98699999999997},"last":1.0049999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":49.854999999999997,"5min":40.755000000000003,"15min":47.512999999999998},"min":{"1min":0.61899999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":971.67999999999995,"5min":971.67999999999995,"15min":971.67999999999995},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.236,"5min":1.264,"15min":1.292},"min":{"1min":0.64900000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.5960000000000001,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":1.1559999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":91.870999999999995,"5min":48.862000000000002,"15min":49.893000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":966.60900000000004,"5min":966.60900000000004,"15min":966.60900000000004},"last":1.2749999999999999},{"interface":"front","average":{"1min":28.396999999999998,"5min":21.527000000000001,"15min":24.52},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":501.79000000000002,"5min":517.26700000000005,"15min":517.26700000000005},"last":1.488}]},{"osd":7,"last update":"Fri Aug 7 15:09:26 2020","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.341,"15min":1.319},"min":{"1min":0.77500000000000002,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.8940000000000001,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.226},{"interface":"front","average":{"1min":27.206,"5min":6.6630000000000003,"15min":3.2650000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":515.09100000000001,"5min":515.09100000000001,"15min":515.09100000000001},"last":1.4099999999999999}]}]}]}} 2020-08-07T15:10:26.077 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:10:26.083 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2020-08-07T15:10:26.083 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:10:26.083 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:10:26.084 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:10:26.084 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:10:26.084 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:10:26.084 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:10:26.084 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:10:26.085 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:10:26.086 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:26.068+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:26.087 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:10:26.087 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump -f json-pretty 2020-08-07T15:10:26.172 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:26.171+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:26.211 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 6 16 9203 9187 11.9611 10.1016 0.00797386 0.010426 2020-08-07T15:10:26.294 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:26.272+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:26.327 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:26.328 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:10:26.328 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.328 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5099.0:9356 5.5 e170/164)", 2020-08-07T15:10:26.328 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:26.324617+0000", 2020-08-07T15:10:26.328 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.00040266800000000001, 2020-08-07T15:10:26.329 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00042403200000000002, 2020-08-07T15:10:26.329 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:26.329 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:10:26.329 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:26.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.341 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:26.341 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.324617+0000", 2020-08-07T15:10:26.342 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:26.342 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.342 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.342 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:26.342 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.324617+0000", 2020-08-07T15:10:26.343 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2777999999999997e-05 2020-08-07T15:10:26.343 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.343 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.343 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:26.343 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.324650+0000", 2020-08-07T15:10:26.344 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.207e-06 2020-08-07T15:10:26.344 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.344 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.344 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:26.344 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.324656+0000", 2020-08-07T15:10:26.345 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0969000000000004e-05 2020-08-07T15:10:26.345 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.345 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.345 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:26.345 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.324727+0000", 2020-08-07T15:10:26.349 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4309999999999999e-06 2020-08-07T15:10:26.350 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.350 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.351 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:26.351 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.324733+0000", 2020-08-07T15:10:26.351 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000116385 2020-08-07T15:10:26.351 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.351 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.352 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.352 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.352 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.352 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:9356 5.5 5:a217cb35:::benchmark_data_smithi074_53697_object1169:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e170)", 2020-08-07T15:10:26.352 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:26.314103+0000", 2020-08-07T15:10:26.353 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.010916352000000001, 2020-08-07T15:10:26.353 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.011009863, 2020-08-07T15:10:26.353 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:26.353 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:10:26.353 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:26.354 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:26.354 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:26.354 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 9356 2020-08-07T15:10:26.361 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.362 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:26.362 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.362 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:26.362 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314103+0000", 2020-08-07T15:10:26.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:26.363 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.363 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.363 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:26.363 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314103+0000", 2020-08-07T15:10:26.364 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7432000000000003e-05 2020-08-07T15:10:26.364 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:26.364 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314151+0000", 2020-08-07T15:10:26.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7189999999999998e-06 2020-08-07T15:10:26.365 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.365 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.365 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:26.365 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314156+0000", 2020-08-07T15:10:26.366 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1725999999999999e-05 2020-08-07T15:10:26.367 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.367 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.367 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:26.367 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314168+0000", 2020-08-07T15:10:26.367 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8040000000000004e-06 2020-08-07T15:10:26.368 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.368 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.368 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:26.368 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314175+0000", 2020-08-07T15:10:26.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015221049999999999 2020-08-07T15:10:26.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.369 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:26.369 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.315697+0000", 2020-08-07T15:10:26.369 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000116249 2020-08-07T15:10:26.370 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.370 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.370 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:26.370 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.315813+0000", 2020-08-07T15:10:26.370 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020119800000000001 2020-08-07T15:10:26.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.371 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.371 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:10:26.371 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.316015+0000", 2020-08-07T15:10:26.371 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0031674110000000002 2020-08-07T15:10:26.372 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.372 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:26.372 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.319182+0000", 2020-08-07T15:10:26.372 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001453684 2020-08-07T15:10:26.373 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.373 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.373 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:26.373 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.320636+0000", 2020-08-07T15:10:26.373 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0065323279999999996 2020-08-07T15:10:26.376 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.376 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.376 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.377 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.377 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:9357 5.5 5:a217cb35:::benchmark_data_smithi074_53697_object1169:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e170)", 2020-08-07T15:10:26.377 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:26.314223+0000", 2020-08-07T15:10:26.378 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.010796747000000001, 2020-08-07T15:10:26.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.011004441, 2020-08-07T15:10:26.378 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:26.378 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:10:26.378 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:26.379 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:26.379 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:26.379 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 9357 2020-08-07T15:10:26.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.381 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:26.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:26.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314223+0000", 2020-08-07T15:10:26.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:26.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.382 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.382 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:26.382 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314223+0000", 2020-08-07T15:10:26.382 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0268000000000001e-05 2020-08-07T15:10:26.383 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.383 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.383 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:26.383 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314273+0000", 2020-08-07T15:10:26.383 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0839999999999997e-06 2020-08-07T15:10:26.384 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.384 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.384 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:26.384 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314279+0000", 2020-08-07T15:10:26.384 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1528999999999999e-05 2020-08-07T15:10:26.386 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.386 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.387 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:26.387 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314291+0000", 2020-08-07T15:10:26.387 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1599999999999997e-06 2020-08-07T15:10:26.387 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.388 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.388 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:26.389 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314296+0000", 2020-08-07T15:10:26.389 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002169266 2020-08-07T15:10:26.389 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.389 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.389 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:26.390 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.316465+0000", 2020-08-07T15:10:26.390 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0682999999999999e-05 2020-08-07T15:10:26.391 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.391 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.391 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:26.392 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.316556+0000", 2020-08-07T15:10:26.392 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016317400000000001 2020-08-07T15:10:26.393 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.393 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.393 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:10:26.393 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.316719+0000", 2020-08-07T15:10:26.394 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003539141 2020-08-07T15:10:26.395 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.395 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.396 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:26.397 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.320258+0000", 2020-08-07T15:10:26.397 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00038517500000000002 2020-08-07T15:10:26.397 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.397 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.397 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:26.398 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.320643+0000", 2020-08-07T15:10:26.398 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0064204800000000001 2020-08-07T15:10:26.398 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.398 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.398 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.399 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.399 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:9358 5.5 5:a217cb35:::benchmark_data_smithi074_53697_object1169:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e170)", 2020-08-07T15:10:26.399 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:26.314332+0000", 2020-08-07T15:10:26.400 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.010687231, 2020-08-07T15:10:26.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.011006607999999999, 2020-08-07T15:10:26.400 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:26.400 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:10:26.400 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:26.403 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:26.404 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:26.404 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 9358 2020-08-07T15:10:26.408 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.409 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:26.409 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.409 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:26.410 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314332+0000", 2020-08-07T15:10:26.410 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:26.410 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.411 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.411 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:26.411 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314332+0000", 2020-08-07T15:10:26.411 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8955999999999997e-05 2020-08-07T15:10:26.412 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.412 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.412 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:26.413 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314381+0000", 2020-08-07T15:10:26.413 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2830000000000001e-06 2020-08-07T15:10:26.414 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.414 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.415 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:26.415 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314387+0000", 2020-08-07T15:10:26.415 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0922e-05 2020-08-07T15:10:26.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.422 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:26.422 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314398+0000", 2020-08-07T15:10:26.423 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0200000000000002e-06 2020-08-07T15:10:26.423 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.423 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.423 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:26.424 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314403+0000", 2020-08-07T15:10:26.424 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026891660000000002 2020-08-07T15:10:26.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.425 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:26.426 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.317092+0000", 2020-08-07T15:10:26.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000102097 2020-08-07T15:10:26.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.429 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:26.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.317194+0000", 2020-08-07T15:10:26.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00071375300000000002 2020-08-07T15:10:26.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.432 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.432 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:10:26.432 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.317908+0000", 2020-08-07T15:10:26.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023919649999999998 2020-08-07T15:10:26.433 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.433 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.434 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:26.434 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.320300+0000", 2020-08-07T15:10:26.435 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00068766499999999996 2020-08-07T15:10:26.435 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.436 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.436 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:26.436 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.320987+0000", 2020-08-07T15:10:26.437 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0066548270000000003 2020-08-07T15:10:26.439 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.439 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.439 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.439 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.440 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:9359 5.5 5:a217cb35:::benchmark_data_smithi074_53697_object1169:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e170)", 2020-08-07T15:10:26.440 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:26.314438+0000", 2020-08-07T15:10:26.440 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.010581798999999999, 2020-08-07T15:10:26.440 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.011020482, 2020-08-07T15:10:26.441 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:26.441 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:10:26.441 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:26.441 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:26.442 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:26.443 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 9359 2020-08-07T15:10:26.443 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.443 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:26.443 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.444 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:26.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314438+0000", 2020-08-07T15:10:26.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:26.444 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.445 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:26.445 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314438+0000", 2020-08-07T15:10:26.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0247000000000002e-05 2020-08-07T15:10:26.446 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.446 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.446 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:26.446 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314488+0000", 2020-08-07T15:10:26.447 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5460000000000001e-05 2020-08-07T15:10:26.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:26.449 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314504+0000", 2020-08-07T15:10:26.449 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5747e-05 2020-08-07T15:10:26.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.450 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:26.450 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314519+0000", 2020-08-07T15:10:26.450 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1299999999999998e-06 2020-08-07T15:10:26.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.451 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.451 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:26.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314525+0000", 2020-08-07T15:10:26.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038409910000000002 2020-08-07T15:10:26.451 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.451 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:26.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.318366+0000", 2020-08-07T15:10:26.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.658e-05 2020-08-07T15:10:26.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.454 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.454 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:26.454 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.318463+0000", 2020-08-07T15:10:26.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000171857 2020-08-07T15:10:26.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:10:26.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.318635+0000", 2020-08-07T15:10:26.456 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001982251 2020-08-07T15:10:26.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:26.456 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.320617+0000", 2020-08-07T15:10:26.456 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00040920399999999998 2020-08-07T15:10:26.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:26.458 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.321026+0000", 2020-08-07T15:10:26.458 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0065884669999999998 2020-08-07T15:10:26.458 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.458 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.459 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.459 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.459 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:9360 5.5 5:a217cb35:::benchmark_data_smithi074_53697_object1169:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e170)", 2020-08-07T15:10:26.459 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:26.314563+0000", 2020-08-07T15:10:26.460 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.010456929, 2020-08-07T15:10:26.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.011031367, 2020-08-07T15:10:26.460 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:26.460 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:10:26.460 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:26.461 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:26.461 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:26.461 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 9360 2020-08-07T15:10:26.461 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.461 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:26.462 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.462 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:26.462 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314563+0000", 2020-08-07T15:10:26.462 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:26.462 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.463 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.463 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:26.463 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314563+0000", 2020-08-07T15:10:26.463 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7509000000000002e-05 2020-08-07T15:10:26.463 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.464 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.464 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:26.464 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314610+0000", 2020-08-07T15:10:26.464 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4190000000000001e-06 2020-08-07T15:10:26.464 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.465 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.465 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:26.465 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314616+0000", 2020-08-07T15:10:26.465 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1219e-05 2020-08-07T15:10:26.465 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.467 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.467 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:26.467 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314627+0000", 2020-08-07T15:10:26.467 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.028e-06 2020-08-07T15:10:26.467 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.468 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.468 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:26.468 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.314632+0000", 2020-08-07T15:10:26.468 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045875569999999999 2020-08-07T15:10:26.468 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.469 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.469 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:26.469 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.319219+0000", 2020-08-07T15:10:26.469 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8706e-05 2020-08-07T15:10:26.469 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.470 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.470 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:26.470 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.319318+0000", 2020-08-07T15:10:26.471 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017829900000000001 2020-08-07T15:10:26.472 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.473 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.473 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:10:26.473 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.319496+0000", 2020-08-07T15:10:26.473 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011944659999999999 2020-08-07T15:10:26.474 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.474 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.474 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:26.474 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.320691+0000", 2020-08-07T15:10:26.474 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00035262100000000001 2020-08-07T15:10:26.475 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.475 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.475 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:26.475 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.321044+0000", 2020-08-07T15:10:26.475 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0064808239999999996 2020-08-07T15:10:26.476 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.476 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.476 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.477 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.477 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.477 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:9364 5.c 5:366fbe43:::benchmark_data_smithi074_53697_object1170:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e170)", 2020-08-07T15:10:26.478 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:26.315301+0000", 2020-08-07T15:10:26.478 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0097184260000000005, 2020-08-07T15:10:26.478 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.010409165999999999, 2020-08-07T15:10:26.478 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:26.478 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:10:26.479 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:26.479 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:26.479 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:26.479 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 9364 2020-08-07T15:10:26.479 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.480 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:26.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.480 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:26.480 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.315301+0000", 2020-08-07T15:10:26.480 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:26.481 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.481 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.481 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:26.481 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.315301+0000", 2020-08-07T15:10:26.482 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2426999999999998e-05 2020-08-07T15:10:26.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.491 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.492 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:26.492 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.315354+0000", 2020-08-07T15:10:26.492 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6279999999999996e-06 2020-08-07T15:10:26.492 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.492 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:26.493 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.315361+0000", 2020-08-07T15:10:26.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3294e-05 2020-08-07T15:10:26.494 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.494 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:26.494 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.315375+0000", 2020-08-07T15:10:26.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4470000000000003e-06 2020-08-07T15:10:26.495 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.495 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.495 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:26.495 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.315381+0000", 2020-08-07T15:10:26.496 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013710490000000001 2020-08-07T15:10:26.496 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.496 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.496 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:26.496 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.316752+0000", 2020-08-07T15:10:26.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.4858999999999995e-05 2020-08-07T15:10:26.497 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.497 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.497 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:26.497 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.316837+0000", 2020-08-07T15:10:26.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014866200000000001 2020-08-07T15:10:26.498 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.498 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.498 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:10:26.498 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.316986+0000", 2020-08-07T15:10:26.498 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0029900439999999999 2020-08-07T15:10:26.499 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.499 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.499 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:26.499 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.319976+0000", 2020-08-07T15:10:26.499 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000605836 2020-08-07T15:10:26.500 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.500 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.500 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:26.500 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:26.320581+0000", 2020-08-07T15:10:26.500 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0052802459999999997 2020-08-07T15:10:26.501 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.501 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.503 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.504 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.504 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:10:26.504 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 7 2020-08-07T15:10:26.504 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:26.506 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:26.505+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:26.506 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:10:26.606 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:10:26.607 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:26.607 INFO:teuthology.orchestra.run.smithi074.stdout: "epoch": 170, 2020-08-07T15:10:26.607 INFO:teuthology.orchestra.run.smithi074.stdout: "fsid": "a01d3e82-8d24-4288-9e42-576ab3ff4957", 2020-08-07T15:10:26.608 INFO:teuthology.orchestra.run.smithi074.stdout: "created": "2020-08-07T15:06:07.342912+0000", 2020-08-07T15:10:26.608 INFO:teuthology.orchestra.run.smithi074.stdout: "modified": "2020-08-07T15:10:23.288360+0000", 2020-08-07T15:10:26.608 INFO:teuthology.orchestra.run.smithi074.stdout: "last_up_change": "2020-08-07T15:06:17.674292+0000", 2020-08-07T15:10:26.621 INFO:teuthology.orchestra.run.smithi074.stdout: "last_in_change": "2020-08-07T15:06:17.674292+0000", 2020-08-07T15:10:26.622 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-08-07T15:10:26.622 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_num": 5804032, 2020-08-07T15:10:26.622 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_set": [ 2020-08-07T15:10:26.622 INFO:teuthology.orchestra.run.smithi074.stdout: "nodeep-scrub", 2020-08-07T15:10:26.623 INFO:teuthology.orchestra.run.smithi074.stdout: "pglog_hardlimit", 2020-08-07T15:10:26.623 INFO:teuthology.orchestra.run.smithi074.stdout: "purged_snapdirs", 2020-08-07T15:10:26.623 INFO:teuthology.orchestra.run.smithi074.stdout: "recovery_deletes", 2020-08-07T15:10:26.623 INFO:teuthology.orchestra.run.smithi074.stdout: "sortbitwise" 2020-08-07T15:10:26.624 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:10:26.624 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_version": 5, 2020-08-07T15:10:26.624 INFO:teuthology.orchestra.run.smithi074.stdout: "full_ratio": 0.94999998807907104, 2020-08-07T15:10:26.624 INFO:teuthology.orchestra.run.smithi074.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-08-07T15:10:26.625 INFO:teuthology.orchestra.run.smithi074.stdout: "nearfull_ratio": 0.85000002384185791, 2020-08-07T15:10:26.625 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_snapshot": "", 2020-08-07T15:10:26.625 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_max": 5, 2020-08-07T15:10:26.625 INFO:teuthology.orchestra.run.smithi074.stdout: "max_osd": 8, 2020-08-07T15:10:26.625 INFO:teuthology.orchestra.run.smithi074.stdout: "require_min_compat_client": "luminous", 2020-08-07T15:10:26.626 INFO:teuthology.orchestra.run.smithi074.stdout: "min_compat_client": "jewel", 2020-08-07T15:10:26.626 INFO:teuthology.orchestra.run.smithi074.stdout: "require_osd_release": "pacific", 2020-08-07T15:10:26.626 INFO:teuthology.orchestra.run.smithi074.stdout: "pools": [ 2020-08-07T15:10:26.626 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.626 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 1, 2020-08-07T15:10:26.627 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "device_health_metrics", 2020-08-07T15:10:26.627 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:06:10.648876+0000", 2020-08-07T15:10:26.627 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:10:26.627 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:10:26.627 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:10:26.628 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:10:26.628 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:10:26.628 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:10:26.628 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:10:26.628 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:10:26.628 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 21, 2020-08-07T15:10:26.629 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 21, 2020-08-07T15:10:26.629 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 21, 2020-08-07T15:10:26.629 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 21, 2020-08-07T15:10:26.629 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 21, 2020-08-07T15:10:26.629 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:10:26.630 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:10:26.630 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:10:26.630 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:10:26.630 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:10:26.630 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:10:26.631 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:10:26.631 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.631 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "91", 2020-08-07T15:10:26.631 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:10:26.631 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:10:26.632 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "89", 2020-08-07T15:10:26.632 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:10:26.632 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:10:26.632 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:10:26.632 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:10:26.632 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:10:26.633 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:10:26.633 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:10:26.633 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:10:26.633 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:10:26.633 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:10:26.634 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:10:26.634 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:10:26.634 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:10:26.634 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:10:26.634 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:10:26.635 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:10:26.635 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:10:26.635 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:10:26.635 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:10:26.635 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:10:26.636 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:10:26.636 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:10:26.636 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:10:26.636 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.636 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:10:26.636 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:10:26.637 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:10:26.637 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:10:26.637 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:10:26.637 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:10:26.638 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:10:26.638 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:10:26.639 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:10:26.639 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:10:26.639 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:10:26.639 INFO:teuthology.orchestra.run.smithi074.stdout: "options": { 2020-08-07T15:10:26.639 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_min": 1 2020-08-07T15:10:26.640 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.640 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:10:26.640 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_devicehealth": {} 2020-08-07T15:10:26.640 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.640 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.641 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.641 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 2, 2020-08-07T15:10:26.641 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "rbd", 2020-08-07T15:10:26.641 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:06:21.474648+0000", 2020-08-07T15:10:26.641 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:10:26.641 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:10:26.642 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:10:26.642 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:10:26.642 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:10:26.642 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:10:26.642 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:10:26.643 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:10:26.643 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 8, 2020-08-07T15:10:26.643 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 8, 2020-08-07T15:10:26.643 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 8, 2020-08-07T15:10:26.643 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 8, 2020-08-07T15:10:26.644 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 8, 2020-08-07T15:10:26.644 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:10:26.644 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:10:26.644 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:10:26.644 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:10:26.644 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:10:26.645 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:10:26.645 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:10:26.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.645 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "19", 2020-08-07T15:10:26.645 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:10:26.646 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:10:26.646 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "0", 2020-08-07T15:10:26.646 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:10:26.646 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:10:26.646 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:10:26.647 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:10:26.647 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:10:26.647 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:10:26.647 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:10:26.647 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:10:26.648 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:10:26.648 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:10:26.648 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:10:26.648 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:10:26.648 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:10:26.648 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:10:26.649 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:10:26.649 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:10:26.649 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:10:26.649 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:10:26.649 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:10:26.650 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:10:26.650 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:10:26.650 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:10:26.650 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:10:26.650 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.651 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:10:26.651 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:10:26.651 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:10:26.651 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:10:26.651 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:10:26.652 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:10:26.652 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:10:26.652 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:10:26.652 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:10:26.652 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:10:26.652 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:10:26.653 INFO:teuthology.orchestra.run.smithi074.stdout: "options": {}, 2020-08-07T15:10:26.653 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:10:26.653 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd": {} 2020-08-07T15:10:26.653 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.653 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.654 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.654 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 5, 2020-08-07T15:10:26.654 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "unique_pool_2", 2020-08-07T15:10:26.654 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:10:14.379166+0000", 2020-08-07T15:10:26.654 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:10:26.655 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:10:26.655 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:10:26.655 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:10:26.655 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:10:26.655 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:10:26.655 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:10:26.656 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:10:26.656 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 16, 2020-08-07T15:10:26.656 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 16, 2020-08-07T15:10:26.656 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 16, 2020-08-07T15:10:26.656 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 16, 2020-08-07T15:10:26.657 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 16, 2020-08-07T15:10:26.657 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:10:26.657 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:10:26.657 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:10:26.657 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:10:26.658 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:10:26.658 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:10:26.658 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:10:26.658 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.658 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "166", 2020-08-07T15:10:26.658 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:10:26.670 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:10:26.671 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "0", 2020-08-07T15:10:26.671 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:10:26.671 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:10:26.671 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:10:26.672 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:10:26.672 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:10:26.672 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:10:26.672 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:10:26.672 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:10:26.672 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:10:26.673 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:10:26.673 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:10:26.673 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:10:26.673 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:10:26.674 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:10:26.674 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:10:26.674 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:10:26.674 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:10:26.674 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:10:26.674 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:10:26.675 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:10:26.675 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:10:26.675 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:10:26.675 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:10:26.675 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.676 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:10:26.676 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:10:26.676 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:10:26.676 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:10:26.676 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:10:26.677 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:10:26.677 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:10:26.677 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:10:26.695 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:10:26.700 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:10:26.700 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:10:26.701 INFO:teuthology.orchestra.run.smithi074.stdout: "options": {}, 2020-08-07T15:10:26.701 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:10:26.701 INFO:teuthology.orchestra.run.smithi074.stdout: "rados": {} 2020-08-07T15:10:26.701 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.701 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.702 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:10:26.702 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:10:26.702 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.702 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 0, 2020-08-07T15:10:26.702 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "fc1ebb23-3ef4-4fc0-b395-382e385885a3", 2020-08-07T15:10:26.703 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:10:26.703 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:10:26.703 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:10:26.703 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0, 2020-08-07T15:10:26.703 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:10:26.704 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:10:26.704 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:10:26.704 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 133, 2020-08-07T15:10:26.704 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:10:26.705 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:10:26.705 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:10:26.705 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.705 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.705 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.706 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6801", 2020-08-07T15:10:26.706 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:10:26.706 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.706 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.707 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.707 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:10:26.707 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.707 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.708 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6802", 2020-08-07T15:10:26.708 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:10:26.708 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.708 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.708 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.708 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:10:26.709 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.709 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.709 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.709 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6804", 2020-08-07T15:10:26.709 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:10:26.710 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.710 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.710 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.711 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:10:26.711 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.711 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.712 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6803", 2020-08-07T15:10:26.712 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:10:26.712 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.712 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.712 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.713 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6801/31577", 2020-08-07T15:10:26.713 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6802/31577", 2020-08-07T15:10:26.713 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6804/31577", 2020-08-07T15:10:26.713 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6803/31577", 2020-08-07T15:10:26.713 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:10:26.714 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:10:26.714 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:10:26.714 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.714 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.714 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.715 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 1, 2020-08-07T15:10:26.715 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "1fbe530a-63ce-4dc8-82ab-18793d6ebc0b", 2020-08-07T15:10:26.715 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:10:26.715 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:10:26.715 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:10:26.716 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0.7196044921875, 2020-08-07T15:10:26.716 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:10:26.716 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:10:26.717 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:10:26.717 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 164, 2020-08-07T15:10:26.717 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:10:26.717 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:10:26.717 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:10:26.718 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.718 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.718 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.719 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6809", 2020-08-07T15:10:26.719 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:10:26.719 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.719 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.720 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:10:26.720 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.722 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.723 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6810", 2020-08-07T15:10:26.723 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:10:26.723 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.723 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.724 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:10:26.724 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.725 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.726 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6812", 2020-08-07T15:10:26.726 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:10:26.726 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.727 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.727 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.727 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:10:26.728 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.728 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.729 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6811", 2020-08-07T15:10:26.729 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:10:26.729 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.731 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.732 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.732 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6809/31578", 2020-08-07T15:10:26.732 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6810/31578", 2020-08-07T15:10:26.732 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6812/31578", 2020-08-07T15:10:26.733 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6811/31578", 2020-08-07T15:10:26.733 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:10:26.733 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:10:26.733 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:10:26.733 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.734 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.734 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.745 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 2, 2020-08-07T15:10:26.745 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "d6fe4338-3e11-4f73-9cbb-e1b046ce286a", 2020-08-07T15:10:26.745 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:10:26.746 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:10:26.746 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:10:26.746 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:10:26.746 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:10:26.746 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:10:26.747 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:10:26.747 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 164, 2020-08-07T15:10:26.747 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:10:26.747 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:10:26.747 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:10:26.747 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.748 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.748 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.748 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6805", 2020-08-07T15:10:26.748 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:10:26.748 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.749 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.751 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.751 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:10:26.751 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.753 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.766 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6806", 2020-08-07T15:10:26.766 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:10:26.767 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.767 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.767 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:10:26.767 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.768 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.768 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.768 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6808", 2020-08-07T15:10:26.768 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:10:26.768 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.769 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.769 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.769 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:10:26.769 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.770 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.770 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.770 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6807", 2020-08-07T15:10:26.770 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:10:26.771 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.771 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.771 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.771 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6805/31576", 2020-08-07T15:10:26.771 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6806/31576", 2020-08-07T15:10:26.771 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6808/31576", 2020-08-07T15:10:26.772 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6807/31576", 2020-08-07T15:10:26.772 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:10:26.772 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:10:26.772 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:10:26.772 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.773 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.773 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.773 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 3, 2020-08-07T15:10:26.773 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9", 2020-08-07T15:10:26.773 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:10:26.779 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:10:26.779 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:10:26.779 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:10:26.780 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:10:26.780 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:10:26.780 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:10:26.787 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 164, 2020-08-07T15:10:26.787 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:10:26.787 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:10:26.788 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:10:26.788 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.788 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.788 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6813", 2020-08-07T15:10:26.789 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:10:26.789 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.789 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.789 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.789 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:10:26.790 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.790 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.790 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6814", 2020-08-07T15:10:26.790 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:10:26.791 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.791 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.791 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:10:26.791 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.792 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.792 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6816", 2020-08-07T15:10:26.792 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:10:26.792 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.792 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.793 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.793 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:10:26.793 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.793 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.793 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.806 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6815", 2020-08-07T15:10:26.806 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:10:26.806 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.807 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.807 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6813/31579", 2020-08-07T15:10:26.807 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6814/31579", 2020-08-07T15:10:26.808 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6816/31579", 2020-08-07T15:10:26.808 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6815/31579", 2020-08-07T15:10:26.808 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:10:26.808 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:10:26.808 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:10:26.809 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.809 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 4, 2020-08-07T15:10:26.809 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "01aadc13-29c5-4ddd-9bb9-de24b2b20412", 2020-08-07T15:10:26.810 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:10:26.810 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:10:26.810 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:10:26.810 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0, 2020-08-07T15:10:26.819 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:10:26.819 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:10:26.820 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:10:26.820 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 100, 2020-08-07T15:10:26.824 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:10:26.824 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:10:26.825 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:10:26.825 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.826 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.826 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6800", 2020-08-07T15:10:26.827 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:10:26.827 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.827 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.828 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:10:26.828 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.829 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.829 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.829 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6801", 2020-08-07T15:10:26.829 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:10:26.830 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.830 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.831 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.831 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:10:26.832 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.832 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.833 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6803", 2020-08-07T15:10:26.833 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:10:26.833 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.835 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.836 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:10:26.836 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.837 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.837 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6802", 2020-08-07T15:10:26.837 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:10:26.838 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.839 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.839 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6800/31545", 2020-08-07T15:10:26.840 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6801/31545", 2020-08-07T15:10:26.840 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6803/31545", 2020-08-07T15:10:26.840 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6802/31545", 2020-08-07T15:10:26.840 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:10:26.841 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:10:26.841 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:10:26.842 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.842 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.843 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 5, 2020-08-07T15:10:26.843 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "f5556a82-73b4-47b1-8e29-1d2fe031d8fd", 2020-08-07T15:10:26.844 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:10:26.844 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:10:26.846 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:10:26.847 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:10:26.847 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:10:26.847 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:10:26.848 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:10:26.848 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 164, 2020-08-07T15:10:26.848 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:10:26.849 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:10:26.849 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:10:26.849 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.850 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.850 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6804", 2020-08-07T15:10:26.851 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:10:26.851 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.851 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.873 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:10:26.874 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.875 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.875 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6805", 2020-08-07T15:10:26.876 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:10:26.876 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.877 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.877 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.902 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:10:26.902 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.903 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.903 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6807", 2020-08-07T15:10:26.903 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:10:26.903 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.904 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.904 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:10:26.907 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.908 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.908 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6806", 2020-08-07T15:10:26.909 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:10:26.909 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.909 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.918 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.918 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6804/31547", 2020-08-07T15:10:26.919 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6805/31547", 2020-08-07T15:10:26.919 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6807/31547", 2020-08-07T15:10:26.919 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6806/31547", 2020-08-07T15:10:26.919 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:10:26.919 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:10:26.920 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:10:26.920 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.920 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 6, 2020-08-07T15:10:26.921 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "9b98b337-e1cc-43cd-9e9b-f21ae3f20899", 2020-08-07T15:10:26.921 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:10:26.921 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:10:26.921 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:10:26.921 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:10:26.924 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:10:26.924 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:10:26.924 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 14, 2020-08-07T15:10:26.924 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 164, 2020-08-07T15:10:26.924 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:10:26.925 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:10:26.925 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:10:26.925 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.925 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.926 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6808", 2020-08-07T15:10:26.926 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:10:26.926 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.926 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.927 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:10:26.927 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.927 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.927 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6809", 2020-08-07T15:10:26.928 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:10:26.928 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.928 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.928 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:10:26.929 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.929 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.929 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6811", 2020-08-07T15:10:26.929 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:10:26.930 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.930 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.931 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:10:26.931 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.931 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.932 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6810", 2020-08-07T15:10:26.932 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:10:26.932 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.932 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.933 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6808/31546", 2020-08-07T15:10:26.933 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6809/31546", 2020-08-07T15:10:26.933 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6811/31546", 2020-08-07T15:10:26.935 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6810/31546", 2020-08-07T15:10:26.935 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:10:26.935 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:10:26.935 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:10:26.936 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.936 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 7, 2020-08-07T15:10:26.936 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "777662d8-bd34-48e8-83e1-24ce9e61f14b", 2020-08-07T15:10:26.937 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:10:26.937 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:10:26.937 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:10:26.937 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:10:26.937 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:10:26.938 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:10:26.938 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:10:26.938 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 164, 2020-08-07T15:10:26.938 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:10:26.938 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:10:26.938 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:10:26.939 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.939 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.939 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6812", 2020-08-07T15:10:26.939 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:10:26.940 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.940 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.940 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:10:26.940 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.941 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.941 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6813", 2020-08-07T15:10:26.941 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:10:26.941 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.941 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.942 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.942 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:10:26.942 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.942 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.943 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6815", 2020-08-07T15:10:26.943 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:10:26.943 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.943 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.944 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:10:26.944 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:10:26.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.944 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:10:26.945 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6814", 2020-08-07T15:10:26.946 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:10:26.946 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.946 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.946 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.947 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6812/31570", 2020-08-07T15:10:26.947 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6813/31570", 2020-08-07T15:10:26.947 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6815/31570", 2020-08-07T15:10:26.947 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6814/31570", 2020-08-07T15:10:26.948 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:10:26.948 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:10:26.948 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:10:26.948 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:26.948 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.948 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:10:26.949 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_xinfo": [ 2020-08-07T15:10:26.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.949 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 0, 2020-08-07T15:10:26.949 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:10:26.949 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:10:26.950 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:10:26.950 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:10:26.959 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:10:26.959 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:10:17.548342+0000", 2020-08-07T15:10:26.960 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:10:26.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.960 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 1, 2020-08-07T15:10:26.960 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:10:26.961 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:10:26.961 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:10:26.961 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:10:26.961 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:10:26.961 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:10:16.729479+0000", 2020-08-07T15:10:26.962 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:10:26.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.962 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 2, 2020-08-07T15:10:26.962 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:10:26.963 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:10:26.963 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:10:26.963 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:10:26.963 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:10:26.963 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:10:17.020686+0000", 2020-08-07T15:10:26.964 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:10:26.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.964 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 3, 2020-08-07T15:10:26.964 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:10:26.964 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:10:26.965 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:10:26.965 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:10:26.965 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:10:26.965 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:10:18.633656+0000", 2020-08-07T15:10:26.965 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:10:26.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.970 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 4, 2020-08-07T15:10:26.970 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:10:26.971 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:10:26.971 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:10:26.971 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:10:26.971 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:10:26.971 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:10:17.840360+0000", 2020-08-07T15:10:26.972 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:10:26.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.972 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 5, 2020-08-07T15:10:26.972 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:10:26.973 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:10:26.973 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:10:26.973 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:10:26.973 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:10:26.973 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:10:17.893395+0000", 2020-08-07T15:10:26.974 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:10:26.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.975 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 6, 2020-08-07T15:10:26.975 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:10:26.975 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:10:26.975 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:10:26.976 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:10:26.976 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:10:26.976 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:10:17.639810+0000", 2020-08-07T15:10:26.976 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:10:26.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:26.977 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 7, 2020-08-07T15:10:26.977 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:10:26.977 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:10:26.977 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:10:26.978 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:10:26.978 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:10:26.978 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:10:16.305143+0000", 2020-08-07T15:10:26.978 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:10:26.978 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.979 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:10:26.979 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_upmap": [], 2020-08-07T15:10:26.979 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_upmap_items": [], 2020-08-07T15:10:26.979 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_temp": [], 2020-08-07T15:10:26.979 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_temp": [], 2020-08-07T15:10:26.979 INFO:teuthology.orchestra.run.smithi074.stdout: "blacklist": {}, 2020-08-07T15:10:26.980 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profiles": { 2020-08-07T15:10:26.980 INFO:teuthology.orchestra.run.smithi074.stdout: "default": { 2020-08-07T15:10:26.980 INFO:teuthology.orchestra.run.smithi074.stdout: "crush-failure-domain": "osd", 2020-08-07T15:10:26.980 INFO:teuthology.orchestra.run.smithi074.stdout: "k": "2", 2020-08-07T15:10:26.980 INFO:teuthology.orchestra.run.smithi074.stdout: "m": "1", 2020-08-07T15:10:26.981 INFO:teuthology.orchestra.run.smithi074.stdout: "plugin": "jerasure", 2020-08-07T15:10:26.981 INFO:teuthology.orchestra.run.smithi074.stdout: "ruleset-failure-domain": "osd", 2020-08-07T15:10:26.981 INFO:teuthology.orchestra.run.smithi074.stdout: "technique": "reed_sol_van" 2020-08-07T15:10:26.981 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:26.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:26.982 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps_queue": [], 2020-08-07T15:10:26.982 INFO:teuthology.orchestra.run.smithi074.stdout: "new_removed_snaps": [], 2020-08-07T15:10:26.982 INFO:teuthology.orchestra.run.smithi074.stdout: "new_purged_snaps": [], 2020-08-07T15:10:26.982 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_node_flags": {}, 2020-08-07T15:10:26.983 INFO:teuthology.orchestra.run.smithi074.stdout: "device_class_flags": {} 2020-08-07T15:10:26.983 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:26.985 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:26.986 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:10:26.986 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:10:26.986 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:10:26.987 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:26.987 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:10:26.988 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:26.983+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:26.989 INFO:tasks.thrashosds.thrasher:j is {'epoch': 170, 'fsid': 'a01d3e82-8d24-4288-9e42-576ab3ff4957', 'created': '2020-08-07T15:06:07.342912+0000', 'modified': '2020-08-07T15:10:23.288360+0000', 'last_up_change': '2020-08-07T15:06:17.674292+0000', 'last_in_change': '2020-08-07T15:06:17.674292+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 5, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-08-07T15:06:10.648876+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 21, 'pg_placement_num': 21, 'pg_placement_num_target': 21, 'pg_num_target': 21, 'pg_num_pending': 21, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '89', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-08-07T15:06:21.474648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'unique_pool_2', 'create_time': '2020-08-07T15:10:14.379166+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '166', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': 'fc1ebb23-3ef4-4fc0-b395-382e385885a3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 133, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6801', 'nonce': 31577}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6802', 'nonce': 31577}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6804', 'nonce': 31577}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6803', 'nonce': 31577}]}, 'public_addr': '172.21.15.74:6801/31577', 'cluster_addr': '172.21.15.74:6802/31577', 'heartbeat_back_addr': '172.21.15.74:6804/31577', 'heartbeat_front_addr': '172.21.15.74:6803/31577', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '1fbe530a-63ce-4dc8-82ab-18793d6ebc0b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.7196044921875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 164, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6809', 'nonce': 31578}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6810', 'nonce': 31578}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6812', 'nonce': 31578}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6811', 'nonce': 31578}]}, 'public_addr': '172.21.15.74:6809/31578', 'cluster_addr': '172.21.15.74:6810/31578', 'heartbeat_back_addr': '172.21.15.74:6812/31578', 'heartbeat_front_addr': '172.21.15.74:6811/31578', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd6fe4338-3e11-4f73-9cbb-e1b046ce286a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 164, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6805', 'nonce': 31576}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6806', 'nonce': 31576}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6808', 'nonce': 31576}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6807', 'nonce': 31576}]}, 'public_addr': '172.21.15.74:6805/31576', 'cluster_addr': '172.21.15.74:6806/31576', 'heartbeat_back_addr': '172.21.15.74:6808/31576', 'heartbeat_front_addr': '172.21.15.74:6807/31576', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 164, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6813', 'nonce': 31579}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6814', 'nonce': 31579}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6816', 'nonce': 31579}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6815', 'nonce': 31579}]}, 'public_addr': '172.21.15.74:6813/31579', 'cluster_addr': '172.21.15.74:6814/31579', 'heartbeat_back_addr': '172.21.15.74:6816/31579', 'heartbeat_front_addr': '172.21.15.74:6815/31579', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '01aadc13-29c5-4ddd-9bb9-de24b2b20412', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 100, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6800', 'nonce': 31545}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6801', 'nonce': 31545}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6803', 'nonce': 31545}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6802', 'nonce': 31545}]}, 'public_addr': '172.21.15.96:6800/31545', 'cluster_addr': '172.21.15.96:6801/31545', 'heartbeat_back_addr': '172.21.15.96:6803/31545', 'heartbeat_front_addr': '172.21.15.96:6802/31545', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'f5556a82-73b4-47b1-8e29-1d2fe031d8fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 164, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6804', 'nonce': 31547}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6805', 'nonce': 31547}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6807', 'nonce': 31547}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6806', 'nonce': 31547}]}, 'public_addr': '172.21.15.96:6804/31547', 'cluster_addr': '172.21.15.96:6805/31547', 'heartbeat_back_addr': '172.21.15.96:6807/31547', 'heartbeat_front_addr': '172.21.15.96:6806/31547', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9b98b337-e1cc-43cd-9e9b-f21ae3f20899', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 164, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6808', 'nonce': 31546}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6809', 'nonce': 31546}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6811', 'nonce': 31546}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6810', 'nonce': 31546}]}, 'public_addr': '172.21.15.96:6808/31546', 'cluster_addr': '172.21.15.96:6809/31546', 'heartbeat_back_addr': '172.21.15.96:6811/31546', 'heartbeat_front_addr': '172.21.15.96:6810/31546', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '777662d8-bd34-48e8-83e1-24ce9e61f14b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 164, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6812', 'nonce': 31570}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6813', 'nonce': 31570}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6815', 'nonce': 31570}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6814', 'nonce': 31570}]}, 'public_addr': '172.21.15.96:6812/31570', 'cluster_addr': '172.21.15.96:6813/31570', 'heartbeat_back_addr': '172.21.15.96:6815/31570', 'heartbeat_front_addr': '172.21.15.96:6814/31570', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:10:17.548342+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:10:16.729479+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:10:17.020686+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:10:18.633656+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:10:17.840360+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:10:17.893395+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:10:17.639810+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:10:16.305143+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blacklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}} 2020-08-07T15:10:26.990 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2020-08-07T15:10:27.093 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:27.086+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:27.188 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:27.187+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:27.202 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:10:27.207 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:27.208 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:10:27.208 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:10:27.208 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:10:27.208 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.209 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:15576 3.e e27/20 3:727724a5:::benchmark_data_smithi074_35447_object1946:head v 27'1032, mlcod=27'1023)", 2020-08-07T15:10:27.209 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:10:27.209 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 220.53475489300001, 2020-08-07T15:10:27.210 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.46091973000000003, 2020-08-07T15:10:27.210 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:27.210 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:27.210 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:27.210 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.211 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:27.211 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:10:27.211 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:27.211 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.216 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.216 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:27.216 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:10:27.217 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9592000000000001e-05 2020-08-07T15:10:27.217 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.220 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.220 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:27.220 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669542+0000", 2020-08-07T15:10:27.221 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.113e-06 2020-08-07T15:10:27.221 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.222 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.222 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:27.222 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669547+0000", 2020-08-07T15:10:27.222 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45151617799999999 2020-08-07T15:10:27.223 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.226 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.226 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:27.226 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121064+0000", 2020-08-07T15:10:27.226 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.377e-06 2020-08-07T15:10:27.227 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.228 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.228 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:27.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121067+0000", 2020-08-07T15:10:27.228 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9195e-05 2020-08-07T15:10:27.229 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.229 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.230 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:27.230 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121106+0000", 2020-08-07T15:10:27.230 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7509000000000001e-05 2020-08-07T15:10:27.231 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.231 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.236 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:27.236 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121124+0000", 2020-08-07T15:10:27.236 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0091783100000000003 2020-08-07T15:10:27.237 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.237 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.238 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:27.238 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.130302+0000", 2020-08-07T15:10:27.238 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000100456 2020-08-07T15:10:27.239 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.239 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.239 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:27.239 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.130402+0000", 2020-08-07T15:10:27.239 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.46091973000000003 2020-08-07T15:10:27.240 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.240 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:27.240 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.240 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.240 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.241 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:24264 3.8 e28/20 3:1d3fdbda:::benchmark_data_smithi074_35447_object3032:head v 28'1520, mlcod=28'1515)", 2020-08-07T15:10:27.241 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:10:27.241 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 214.82112485600001, 2020-08-07T15:10:27.241 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41856254199999998, 2020-08-07T15:10:27.241 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:27.242 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:27.242 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:27.242 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.242 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:27.242 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:10:27.242 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:27.243 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.243 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.243 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:27.243 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:10:27.243 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1348999999999998e-05 2020-08-07T15:10:27.244 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.244 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.244 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:27.244 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383174+0000", 2020-08-07T15:10:27.244 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6579999999999996e-06 2020-08-07T15:10:27.245 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.245 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.245 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:27.252 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383181+0000", 2020-08-07T15:10:27.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41194352699999998 2020-08-07T15:10:27.253 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.253 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:27.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.795124+0000", 2020-08-07T15:10:27.253 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5729999999999999e-06 2020-08-07T15:10:27.258 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.258 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.258 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:27.259 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.795128+0000", 2020-08-07T15:10:27.259 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002170384 2020-08-07T15:10:27.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.260 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.260 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:27.260 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.797298+0000", 2020-08-07T15:10:27.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5412e-05 2020-08-07T15:10:27.263 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.263 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.264 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:27.264 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.797324+0000", 2020-08-07T15:10:27.265 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042943360000000002 2020-08-07T15:10:27.265 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.265 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.266 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:27.266 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.801618+0000", 2020-08-07T15:10:27.266 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7302999999999997e-05 2020-08-07T15:10:27.267 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.268 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.268 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:27.269 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.801675+0000", 2020-08-07T15:10:27.269 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41856254199999998 2020-08-07T15:10:27.269 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.270 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:27.270 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.270 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.271 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.271 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:93920 3.6 e56/26 ondisk, result = 0)", 2020-08-07T15:10:27.272 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:10:27.272 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 167.87376159300001, 2020-08-07T15:10:27.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.397576347, 2020-08-07T15:10:27.273 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:27.274 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:27.274 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:27.274 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.275 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:27.275 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:10:27.275 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:27.276 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.276 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.276 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:27.277 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:10:27.277 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2571999999999999e-05 2020-08-07T15:10:27.277 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.285 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.285 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:27.285 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330509+0000", 2020-08-07T15:10:27.285 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2059999999999998e-06 2020-08-07T15:10:27.286 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.286 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.286 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:27.286 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330514+0000", 2020-08-07T15:10:27.286 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39740134599999999 2020-08-07T15:10:27.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.287 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:27.287 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727915+0000", 2020-08-07T15:10:27.287 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.677e-06 2020-08-07T15:10:27.288 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.288 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.288 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:27.288 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727921+0000", 2020-08-07T15:10:27.288 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5431e-05 2020-08-07T15:10:27.289 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.289 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.289 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:27.289 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727956+0000", 2020-08-07T15:10:27.289 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5790000000000004e-06 2020-08-07T15:10:27.290 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.291 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.291 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:27.291 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727965+0000", 2020-08-07T15:10:27.292 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7535999999999997e-05 2020-08-07T15:10:27.292 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.292 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.292 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:27.292 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.728052+0000", 2020-08-07T15:10:27.292 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.397576347 2020-08-07T15:10:27.293 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.293 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:27.293 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.293 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.293 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.294 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106104 3.0 3:0d7a2a89:::benchmark_data_smithi074_35447_object13262:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:10:27.294 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.252296+0000", 2020-08-07T15:10:27.294 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 159.95194170400001, 2020-08-07T15:10:27.294 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.44182183200000003, 2020-08-07T15:10:27.294 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:27.295 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:27.295 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:27.295 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:10:27.295 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:27.295 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106104 2020-08-07T15:10:27.296 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.296 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:27.296 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.296 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:27.296 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252296+0000", 2020-08-07T15:10:27.297 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:27.297 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.297 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.297 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:27.310 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252296+0000", 2020-08-07T15:10:27.310 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4728000000000001e-05 2020-08-07T15:10:27.310 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.311 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.311 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:27.312 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252321+0000", 2020-08-07T15:10:27.312 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9529999999999999e-06 2020-08-07T15:10:27.312 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.312 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.313 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:27.313 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252324+0000", 2020-08-07T15:10:27.313 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5759999999999996e-06 2020-08-07T15:10:27.313 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.314 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.314 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:27.314 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252329+0000", 2020-08-07T15:10:27.314 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.582e-06 2020-08-07T15:10:27.314 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.315 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.315 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:27.315 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252332+0000", 2020-08-07T15:10:27.315 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012958519999999999 2020-08-07T15:10:27.315 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.315 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.316 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:27.316 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.253628+0000", 2020-08-07T15:10:27.316 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7871e-05 2020-08-07T15:10:27.316 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.316 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.317 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:27.317 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.253705+0000", 2020-08-07T15:10:27.317 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017801199999999999 2020-08-07T15:10:27.317 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.317 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.318 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:10:27.318 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.253883+0000", 2020-08-07T15:10:27.318 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001869051 2020-08-07T15:10:27.318 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.318 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.319 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:27.319 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255752+0000", 2020-08-07T15:10:27.319 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3e-07 2020-08-07T15:10:27.320 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.320 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.320 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:27.320 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255753+0000", 2020-08-07T15:10:27.321 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5321000000000001e-05 2020-08-07T15:10:27.321 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.321 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.321 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:27.321 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255808+0000", 2020-08-07T15:10:27.322 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.091917211999999998 2020-08-07T15:10:27.322 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.322 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.322 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:27.322 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.347726+0000", 2020-08-07T15:10:27.323 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.34634211500000001 2020-08-07T15:10:27.323 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.323 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.323 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:27.323 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694068+0000", 2020-08-07T15:10:27.324 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3030000000000001e-06 2020-08-07T15:10:27.324 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.324 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.324 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:27.324 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694069+0000", 2020-08-07T15:10:27.324 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8625999999999998e-05 2020-08-07T15:10:27.325 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.325 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.325 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:27.325 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694118+0000", 2020-08-07T15:10:27.325 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.44182183200000003 2020-08-07T15:10:27.326 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.326 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:27.326 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.326 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.326 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.327 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106104 3.0 e58/20 ondisk, result = 0)", 2020-08-07T15:10:27.329 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.257196+0000", 2020-08-07T15:10:27.329 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 159.947041197, 2020-08-07T15:10:27.329 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43692689000000001, 2020-08-07T15:10:27.330 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:27.330 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:27.330 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:27.331 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.331 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:27.331 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257196+0000", 2020-08-07T15:10:27.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:27.331 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.333 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.333 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:27.335 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257196+0000", 2020-08-07T15:10:27.335 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2021999999999999e-05 2020-08-07T15:10:27.335 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.336 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:27.336 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257228+0000", 2020-08-07T15:10:27.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9549999999999997e-06 2020-08-07T15:10:27.336 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.337 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.337 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:27.337 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257234+0000", 2020-08-07T15:10:27.337 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43671786400000001 2020-08-07T15:10:27.337 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.338 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.338 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:27.338 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693952+0000", 2020-08-07T15:10:27.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7690000000000004e-06 2020-08-07T15:10:27.339 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.339 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.339 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:27.339 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693957+0000", 2020-08-07T15:10:27.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3294000000000006e-05 2020-08-07T15:10:27.340 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.341 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:27.341 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694040+0000", 2020-08-07T15:10:27.341 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3436e-05 2020-08-07T15:10:27.341 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.342 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:27.342 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694054+0000", 2020-08-07T15:10:27.342 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9549999999999996e-05 2020-08-07T15:10:27.342 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.342 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.343 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:27.343 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694123+0000", 2020-08-07T15:10:27.343 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43692689000000001 2020-08-07T15:10:27.343 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.343 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:27.344 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.344 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.344 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.344 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106144 3.6 3:629055a5:::benchmark_data_smithi074_35447_object13267:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:10:27.344 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.280142+0000", 2020-08-07T15:10:27.345 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 159.92409573500001, 2020-08-07T15:10:27.345 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.57968657199999996, 2020-08-07T15:10:27.345 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:27.345 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:27.345 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:27.346 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:10:27.346 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:27.346 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106144 2020-08-07T15:10:27.346 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.346 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:27.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.347 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:27.347 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280142+0000", 2020-08-07T15:10:27.347 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:27.351 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.351 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.352 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:27.352 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280142+0000", 2020-08-07T15:10:27.352 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.905e-05 2020-08-07T15:10:27.353 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.353 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.354 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:27.354 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280191+0000", 2020-08-07T15:10:27.355 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6729999999999997e-06 2020-08-07T15:10:27.355 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.355 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.356 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:27.356 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280197+0000", 2020-08-07T15:10:27.356 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0747999999999999e-05 2020-08-07T15:10:27.357 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.357 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.357 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:27.359 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280207+0000", 2020-08-07T15:10:27.359 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7700000000000001e-06 2020-08-07T15:10:27.360 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.360 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.360 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:27.361 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280212+0000", 2020-08-07T15:10:27.361 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00060223399999999999 2020-08-07T15:10:27.362 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.362 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.362 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:27.363 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280814+0000", 2020-08-07T15:10:27.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7831000000000006e-05 2020-08-07T15:10:27.364 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:27.364 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280912+0000", 2020-08-07T15:10:27.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000137119 2020-08-07T15:10:27.365 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.365 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.365 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:10:27.365 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.281049+0000", 2020-08-07T15:10:27.366 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023794369999999999 2020-08-07T15:10:27.366 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.366 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:27.366 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283429+0000", 2020-08-07T15:10:27.367 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.173e-06 2020-08-07T15:10:27.367 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.367 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.368 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:27.368 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283430+0000", 2020-08-07T15:10:27.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5530000000000004e-05 2020-08-07T15:10:27.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.370 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:27.370 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283495+0000", 2020-08-07T15:10:27.370 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5649000000000003e-05 2020-08-07T15:10:27.370 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.370 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.371 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:27.371 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283541+0000", 2020-08-07T15:10:27.371 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.57624206300000003 2020-08-07T15:10:27.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:27.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859783+0000", 2020-08-07T15:10:27.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.421e-06 2020-08-07T15:10:27.379 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:27.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859785+0000", 2020-08-07T15:10:27.380 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3874e-05 2020-08-07T15:10:27.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:27.380 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859828+0000", 2020-08-07T15:10:27.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.57968657199999996 2020-08-07T15:10:27.381 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.381 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:27.381 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.383 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.383 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106144 3.6 e59/26 ondisk, result = 0)", 2020-08-07T15:10:27.383 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.344536+0000", 2020-08-07T15:10:27.383 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 159.85970139299999, 2020-08-07T15:10:27.383 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51529786099999997, 2020-08-07T15:10:27.384 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:27.384 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:27.384 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:27.384 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.384 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:27.385 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344536+0000", 2020-08-07T15:10:27.385 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:27.385 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.385 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.386 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:27.386 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344536+0000", 2020-08-07T15:10:27.386 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7266999999999999e-05 2020-08-07T15:10:27.386 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.386 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.386 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:27.387 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344553+0000", 2020-08-07T15:10:27.387 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1070000000000001e-06 2020-08-07T15:10:27.400 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.401 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.401 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:27.401 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344557+0000", 2020-08-07T15:10:27.401 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.515143201 2020-08-07T15:10:27.402 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.402 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.402 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:27.402 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859700+0000", 2020-08-07T15:10:27.402 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.271e-05 2020-08-07T15:10:27.403 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.403 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.403 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:27.408 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859712+0000", 2020-08-07T15:10:27.408 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5436999999999998e-05 2020-08-07T15:10:27.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.411 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.411 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:27.412 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859768+0000", 2020-08-07T15:10:27.412 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1540000000000002e-06 2020-08-07T15:10:27.412 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.412 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.413 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:27.425 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859776+0000", 2020-08-07T15:10:27.426 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7985000000000001e-05 2020-08-07T15:10:27.426 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.426 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.426 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:27.426 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859834+0000", 2020-08-07T15:10:27.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51529786099999997 2020-08-07T15:10:27.427 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.427 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:27.427 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.427 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.428 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108352 3.8 3:1aac8373:::benchmark_data_smithi074_35447_object13543:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e62)", 2020-08-07T15:10:27.428 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.642093+0000", 2020-08-07T15:10:27.428 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 156.56214470399999, 2020-08-07T15:10:27.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39397409900000002, 2020-08-07T15:10:27.429 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:27.429 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:27.429 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:27.429 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:10:27.429 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:27.430 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108352 2020-08-07T15:10:27.430 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.431 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:27.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.431 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:27.431 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642093+0000", 2020-08-07T15:10:27.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:27.432 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.432 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.432 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:27.432 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642093+0000", 2020-08-07T15:10:27.432 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9475000000000002e-05 2020-08-07T15:10:27.433 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.433 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.433 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:27.433 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642122+0000", 2020-08-07T15:10:27.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0439999999999997e-06 2020-08-07T15:10:27.434 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.434 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.437 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:27.437 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642126+0000", 2020-08-07T15:10:27.437 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9689999999999994e-06 2020-08-07T15:10:27.438 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.438 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.438 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:27.438 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642134+0000", 2020-08-07T15:10:27.438 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1219999999999997e-06 2020-08-07T15:10:27.439 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.439 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.439 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:27.439 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642138+0000", 2020-08-07T15:10:27.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011580399999999999 2020-08-07T15:10:27.440 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.440 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:27.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.643296+0000", 2020-08-07T15:10:27.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3791999999999999e-05 2020-08-07T15:10:27.444 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.445 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:27.445 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.643390+0000", 2020-08-07T15:10:27.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011882700000000001 2020-08-07T15:10:27.445 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.445 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.446 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4,7", 2020-08-07T15:10:27.446 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.643509+0000", 2020-08-07T15:10:27.446 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034736799999999998 2020-08-07T15:10:27.446 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.447 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:27.447 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.646983+0000", 2020-08-07T15:10:27.447 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0077748490000000003 2020-08-07T15:10:27.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.449 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.449 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:27.449 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.654758+0000", 2020-08-07T15:10:27.450 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000005e-07 2020-08-07T15:10:27.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:27.453 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.654758+0000", 2020-08-07T15:10:27.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4258000000000002e-05 2020-08-07T15:10:27.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:27.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.654802+0000", 2020-08-07T15:10:27.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11166224399999999 2020-08-07T15:10:27.458 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.458 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.458 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:27.458 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.766465+0000", 2020-08-07T15:10:27.459 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26946740899999999 2020-08-07T15:10:27.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.460 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.460 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:27.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.035932+0000", 2020-08-07T15:10:27.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5829999999999999e-06 2020-08-07T15:10:27.461 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.461 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.461 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:27.463 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.035934+0000", 2020-08-07T15:10:27.463 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000133306 2020-08-07T15:10:27.463 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.463 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.464 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:27.464 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.036067+0000", 2020-08-07T15:10:27.464 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39397409900000002 2020-08-07T15:10:27.464 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.465 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:27.465 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.465 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.465 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.466 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145767 3.6 3:62fe5854:::benchmark_data_smithi074_35447_object16158:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:10:27.467 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.654765+0000", 2020-08-07T15:10:27.467 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 132.54947251799999, 2020-08-07T15:10:27.467 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.62108287699999998, 2020-08-07T15:10:27.467 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:27.468 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:27.468 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:27.468 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:10:27.468 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:27.469 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145767 2020-08-07T15:10:27.469 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.469 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:27.469 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.469 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:27.470 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654765+0000", 2020-08-07T15:10:27.470 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:27.470 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.470 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.471 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:27.471 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654765+0000", 2020-08-07T15:10:27.471 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1977999999999998e-05 2020-08-07T15:10:27.471 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.471 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.472 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:27.472 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654797+0000", 2020-08-07T15:10:27.472 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6799999999999998e-06 2020-08-07T15:10:27.473 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.473 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.473 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:27.473 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654803+0000", 2020-08-07T15:10:27.473 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1263e-05 2020-08-07T15:10:27.474 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.474 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.474 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:27.474 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654814+0000", 2020-08-07T15:10:27.475 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2129999999999999e-06 2020-08-07T15:10:27.475 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.475 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.475 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:27.476 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654819+0000", 2020-08-07T15:10:27.476 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0324999999999997e-05 2020-08-07T15:10:27.476 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.476 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.476 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:27.477 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654869+0000", 2020-08-07T15:10:27.477 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015424199999999999 2020-08-07T15:10:27.477 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.477 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.478 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:27.479 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.655024+0000", 2020-08-07T15:10:27.479 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000163573 2020-08-07T15:10:27.479 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.479 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.479 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:10:27.480 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.655187+0000", 2020-08-07T15:10:27.480 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001341645 2020-08-07T15:10:27.480 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.481 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:27.481 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.656529+0000", 2020-08-07T15:10:27.481 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9599999999999998e-07 2020-08-07T15:10:27.482 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.482 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.482 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:27.483 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.656530+0000", 2020-08-07T15:10:27.483 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4381e-05 2020-08-07T15:10:27.483 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.483 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.484 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:27.484 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.656584+0000", 2020-08-07T15:10:27.484 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0057298979999999998 2020-08-07T15:10:27.484 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.485 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.485 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:27.485 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.662314+0000", 2020-08-07T15:10:27.485 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.61347776300000001 2020-08-07T15:10:27.486 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.486 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.486 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:27.487 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275792+0000", 2020-08-07T15:10:27.487 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9740000000000001e-06 2020-08-07T15:10:27.487 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.487 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.487 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:27.488 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275794+0000", 2020-08-07T15:10:27.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4146e-05 2020-08-07T15:10:27.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.492 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.492 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:27.492 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275848+0000", 2020-08-07T15:10:27.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.62108287699999998 2020-08-07T15:10:27.493 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.493 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:27.493 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.494 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.494 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145801 3.6 3:69e4cb22:::benchmark_data_smithi074_35447_object16192:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:10:27.495 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671123+0000", 2020-08-07T15:10:27.495 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 132.533114643, 2020-08-07T15:10:27.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74411769999999999, 2020-08-07T15:10:27.495 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:27.495 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:27.496 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:27.496 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:10:27.496 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:27.496 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145801 2020-08-07T15:10:27.497 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.497 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:27.497 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.497 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:27.498 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671123+0000", 2020-08-07T15:10:27.498 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:27.498 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.498 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.499 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:27.499 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671123+0000", 2020-08-07T15:10:27.499 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3691e-05 2020-08-07T15:10:27.500 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.500 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.500 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:27.500 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671157+0000", 2020-08-07T15:10:27.500 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0249999999999999e-06 2020-08-07T15:10:27.501 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.501 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.501 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:27.507 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671163+0000", 2020-08-07T15:10:27.508 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2339e-05 2020-08-07T15:10:27.508 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.508 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.508 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:27.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671175+0000", 2020-08-07T15:10:27.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5249999999999996e-06 2020-08-07T15:10:27.509 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.509 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.509 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:27.523 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671180+0000", 2020-08-07T15:10:27.524 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6826000000000002e-05 2020-08-07T15:10:27.524 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.524 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.524 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:27.524 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671227+0000", 2020-08-07T15:10:27.525 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014152699999999999 2020-08-07T15:10:27.525 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.525 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.525 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:27.525 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671369+0000", 2020-08-07T15:10:27.526 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8021000000000002e-05 2020-08-07T15:10:27.526 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.526 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.526 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:10:27.526 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671467+0000", 2020-08-07T15:10:27.527 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0016293760000000001 2020-08-07T15:10:27.527 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.527 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.527 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:27.527 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673096+0000", 2020-08-07T15:10:27.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.048e-06 2020-08-07T15:10:27.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.528 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:27.528 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673097+0000", 2020-08-07T15:10:27.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8841999999999998e-05 2020-08-07T15:10:27.529 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.529 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.531 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:27.531 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673146+0000", 2020-08-07T15:10:27.531 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002829935 2020-08-07T15:10:27.531 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.531 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.532 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:27.532 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675976+0000", 2020-08-07T15:10:27.532 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73917870399999996 2020-08-07T15:10:27.532 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.532 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.533 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:27.533 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.415155+0000", 2020-08-07T15:10:27.535 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.193e-06 2020-08-07T15:10:27.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:27.535 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.415157+0000", 2020-08-07T15:10:27.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3647999999999996e-05 2020-08-07T15:10:27.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.536 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:27.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.415241+0000", 2020-08-07T15:10:27.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74411769999999999 2020-08-07T15:10:27.537 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.537 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:27.537 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.537 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.538 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.538 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145803 3.0 3:0c939450:::benchmark_data_smithi074_35447_object16194:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:10:27.538 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671678+0000", 2020-08-07T15:10:27.538 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 132.53255984800001, 2020-08-07T15:10:27.538 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.882157211, 2020-08-07T15:10:27.539 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:27.539 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:27.539 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:27.539 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:10:27.539 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:27.540 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145803 2020-08-07T15:10:27.540 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.540 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:27.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.542 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:27.542 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671678+0000", 2020-08-07T15:10:27.542 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:27.542 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.542 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.543 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:27.543 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671678+0000", 2020-08-07T15:10:27.543 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4097999999999998e-05 2020-08-07T15:10:27.543 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.543 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.544 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:27.544 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671712+0000", 2020-08-07T15:10:27.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2439999999999998e-06 2020-08-07T15:10:27.545 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.545 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:27.546 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671718+0000", 2020-08-07T15:10:27.546 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2629999999999999e-05 2020-08-07T15:10:27.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.546 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.546 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:27.546 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671731+0000", 2020-08-07T15:10:27.547 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9589999999999998e-06 2020-08-07T15:10:27.547 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.547 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:27.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671738+0000", 2020-08-07T15:10:27.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2495000000000004e-05 2020-08-07T15:10:27.548 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.548 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:27.548 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671800+0000", 2020-08-07T15:10:27.549 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013640700000000001 2020-08-07T15:10:27.550 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.550 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.550 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:27.550 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671937+0000", 2020-08-07T15:10:27.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013672200000000001 2020-08-07T15:10:27.551 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.552 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:10:27.553 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.672073+0000", 2020-08-07T15:10:27.553 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001179982 2020-08-07T15:10:27.553 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.553 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.553 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:27.554 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673253+0000", 2020-08-07T15:10:27.554 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1900000000000001e-07 2020-08-07T15:10:27.554 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.555 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.555 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:27.555 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673254+0000", 2020-08-07T15:10:27.556 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2104000000000001e-05 2020-08-07T15:10:27.556 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:27.556 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673306+0000", 2020-08-07T15:10:27.557 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002264369 2020-08-07T15:10:27.557 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.557 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.557 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:27.557 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675571+0000", 2020-08-07T15:10:27.558 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.87820796899999998 2020-08-07T15:10:27.558 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.558 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.558 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:27.567 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553778+0000", 2020-08-07T15:10:27.568 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3629999999999999e-06 2020-08-07T15:10:27.568 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.575 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:27.575 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553780+0000", 2020-08-07T15:10:27.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5050000000000003e-05 2020-08-07T15:10:27.576 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.576 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.576 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:27.576 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553835+0000", 2020-08-07T15:10:27.577 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.882157211 2020-08-07T15:10:27.577 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.577 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:27.584 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.584 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.585 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.585 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145805 3.8 3:13b8e273:::benchmark_data_smithi074_35447_object16196:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:10:27.585 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671847+0000", 2020-08-07T15:10:27.585 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 132.53239052800001, 2020-08-07T15:10:27.586 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.340134453, 2020-08-07T15:10:27.586 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:27.586 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:27.586 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:27.586 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:10:27.587 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:27.587 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145805 2020-08-07T15:10:27.587 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.587 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:27.587 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.588 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:27.588 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671847+0000", 2020-08-07T15:10:27.588 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:27.588 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.589 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:27.589 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671847+0000", 2020-08-07T15:10:27.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1139999999999997e-05 2020-08-07T15:10:27.589 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.592 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:27.592 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671878+0000", 2020-08-07T15:10:27.592 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.784e-06 2020-08-07T15:10:27.593 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.593 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.593 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:27.593 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671884+0000", 2020-08-07T15:10:27.594 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2802999999999999e-05 2020-08-07T15:10:27.594 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.594 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.594 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:27.595 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671897+0000", 2020-08-07T15:10:27.595 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9909999999999999e-06 2020-08-07T15:10:27.602 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.602 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.602 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:27.603 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671903+0000", 2020-08-07T15:10:27.603 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011861 2020-08-07T15:10:27.603 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.603 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:27.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.672021+0000", 2020-08-07T15:10:27.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017702399999999999 2020-08-07T15:10:27.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:27.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.672198+0000", 2020-08-07T15:10:27.605 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020546400000000001 2020-08-07T15:10:27.605 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.605 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.605 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4,7", 2020-08-07T15:10:27.607 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.672404+0000", 2020-08-07T15:10:27.608 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024385019999999999 2020-08-07T15:10:27.608 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.608 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.609 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:27.609 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674842+0000", 2020-08-07T15:10:27.609 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00028594999999999998 2020-08-07T15:10:27.610 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.610 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.610 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:27.610 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675128+0000", 2020-08-07T15:10:27.610 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8100000000000003e-07 2020-08-07T15:10:27.611 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.611 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.611 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:27.611 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675129+0000", 2020-08-07T15:10:27.611 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7973999999999997e-05 2020-08-07T15:10:27.612 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.612 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.613 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:27.613 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675197+0000", 2020-08-07T15:10:27.613 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032171220000000002 2020-08-07T15:10:27.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.614 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:27.614 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678414+0000", 2020-08-07T15:10:27.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33343769299999998 2020-08-07T15:10:27.614 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.614 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.615 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:27.615 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011852+0000", 2020-08-07T15:10:27.615 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3480000000000001e-06 2020-08-07T15:10:27.615 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.615 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.616 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:27.616 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011853+0000", 2020-08-07T15:10:27.616 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000128467 2020-08-07T15:10:27.616 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.616 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.617 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:27.617 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011981+0000", 2020-08-07T15:10:27.617 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.340134453 2020-08-07T15:10:27.617 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.617 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:27.620 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.620 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.621 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.621 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145806 3.8 3:16669ddb:::benchmark_data_smithi074_35447_object16197:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:10:27.625 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.673992+0000", 2020-08-07T15:10:27.625 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 132.53024600200001, 2020-08-07T15:10:27.625 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40687891900000001, 2020-08-07T15:10:27.626 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:27.627 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:27.627 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:27.627 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:10:27.627 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:27.628 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145806 2020-08-07T15:10:27.628 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.628 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:27.628 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.628 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:27.653 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673992+0000", 2020-08-07T15:10:27.653 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:27.653 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.654 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.654 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:27.654 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673992+0000", 2020-08-07T15:10:27.654 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5429000000000002e-05 2020-08-07T15:10:27.655 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.655 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.655 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:27.655 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674027+0000", 2020-08-07T15:10:27.655 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3199999999999996e-06 2020-08-07T15:10:27.656 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.656 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.656 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:27.656 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674033+0000", 2020-08-07T15:10:27.656 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2835e-05 2020-08-07T15:10:27.657 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.657 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.657 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:27.657 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674046+0000", 2020-08-07T15:10:27.657 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9030000000000001e-06 2020-08-07T15:10:27.664 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.665 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:27.665 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674052+0000", 2020-08-07T15:10:27.665 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4181000000000003e-05 2020-08-07T15:10:27.665 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.665 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.666 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:27.666 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674096+0000", 2020-08-07T15:10:27.666 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00022310599999999999 2020-08-07T15:10:27.666 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.666 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.667 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:27.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674319+0000", 2020-08-07T15:10:27.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012681300000000001 2020-08-07T15:10:27.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4,7", 2020-08-07T15:10:27.668 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674446+0000", 2020-08-07T15:10:27.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002882754 2020-08-07T15:10:27.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:27.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.677329+0000", 2020-08-07T15:10:27.670 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9466e-05 2020-08-07T15:10:27.670 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.670 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.670 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:27.670 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.677418+0000", 2020-08-07T15:10:27.671 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0299999999999998e-07 2020-08-07T15:10:27.671 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.671 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.671 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:27.671 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.677419+0000", 2020-08-07T15:10:27.672 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5274000000000001e-05 2020-08-07T15:10:27.672 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.672 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.672 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:27.672 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.677464+0000", 2020-08-07T15:10:27.673 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40011127099999999 2020-08-07T15:10:27.673 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.673 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.673 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:27.673 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.077576+0000", 2020-08-07T15:10:27.674 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0031778370000000002 2020-08-07T15:10:27.674 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.674 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.675 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:27.675 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080754+0000", 2020-08-07T15:10:27.675 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.92e-07 2020-08-07T15:10:27.676 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.676 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.676 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:27.676 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080754+0000", 2020-08-07T15:10:27.676 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000116135 2020-08-07T15:10:27.677 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:27.677 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080870+0000", 2020-08-07T15:10:27.677 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40687891900000001 2020-08-07T15:10:27.678 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.679 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:27.679 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.680 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:145767 3.6 e75/26 ondisk, result = 0)", 2020-08-07T15:10:27.680 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.674108+0000", 2020-08-07T15:10:27.680 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 132.53012957799999, 2020-08-07T15:10:27.681 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.60174556899999998, 2020-08-07T15:10:27.681 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:27.681 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:27.681 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:27.681 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.682 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:27.682 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674108+0000", 2020-08-07T15:10:27.682 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:27.682 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.683 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.683 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:27.683 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674108+0000", 2020-08-07T15:10:27.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0558000000000001e-05 2020-08-07T15:10:27.684 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.684 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.684 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:27.684 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674139+0000", 2020-08-07T15:10:27.685 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6620000000000002e-06 2020-08-07T15:10:27.685 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.685 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.685 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:27.685 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674144+0000", 2020-08-07T15:10:27.686 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.60156723899999998 2020-08-07T15:10:27.686 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.686 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.686 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:27.686 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275711+0000", 2020-08-07T15:10:27.687 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4070000000000004e-06 2020-08-07T15:10:27.687 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.687 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.688 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:27.688 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275719+0000", 2020-08-07T15:10:27.689 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3205999999999998e-05 2020-08-07T15:10:27.690 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.690 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.691 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:27.691 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275772+0000", 2020-08-07T15:10:27.692 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0624e-05 2020-08-07T15:10:27.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.693 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:27.693 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275783+0000", 2020-08-07T15:10:27.694 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0872999999999997e-05 2020-08-07T15:10:27.694 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.694 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.695 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:27.695 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275854+0000", 2020-08-07T15:10:27.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.60174556899999998 2020-08-07T15:10:27.696 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.696 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:27.696 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.697 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.698 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.698 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:145803 3.0 e75/20 ondisk, result = 0)", 2020-08-07T15:10:27.699 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.674488+0000", 2020-08-07T15:10:27.701 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 132.52974959599999, 2020-08-07T15:10:27.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.87935229699999995, 2020-08-07T15:10:27.702 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:27.702 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:27.703 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:27.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.704 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:27.704 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674488+0000", 2020-08-07T15:10:27.705 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:27.705 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.706 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.706 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:27.717 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674488+0000", 2020-08-07T15:10:27.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3062000000000003e-05 2020-08-07T15:10:27.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:27.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674521+0000", 2020-08-07T15:10:27.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4911e-05 2020-08-07T15:10:27.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.734 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:27.734 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674536+0000", 2020-08-07T15:10:27.734 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.87914350600000002 2020-08-07T15:10:27.735 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.735 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.736 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:27.736 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553679+0000", 2020-08-07T15:10:27.736 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5950000000000001e-06 2020-08-07T15:10:27.737 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.737 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.746 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:27.746 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553687+0000", 2020-08-07T15:10:27.747 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2565999999999996e-05 2020-08-07T15:10:27.747 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.747 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.748 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:27.748 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553760+0000", 2020-08-07T15:10:27.748 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0417999999999999e-05 2020-08-07T15:10:27.749 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.749 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.750 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:27.750 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553770+0000", 2020-08-07T15:10:27.751 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0239000000000003e-05 2020-08-07T15:10:27.751 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.751 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.771 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:27.772 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553840+0000", 2020-08-07T15:10:27.772 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.87935229699999995 2020-08-07T15:10:27.784 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.785 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:27.786 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.787 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:145805 3.8 e75/60 ondisk, result = 0)", 2020-08-07T15:10:27.787 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.676843+0000", 2020-08-07T15:10:27.788 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 132.52739412599999, 2020-08-07T15:10:27.788 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.335144041, 2020-08-07T15:10:27.788 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:27.789 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:27.789 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:27.789 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.795 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:27.795 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.676843+0000", 2020-08-07T15:10:27.796 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:27.796 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.797 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:27.797 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.676843+0000", 2020-08-07T15:10:27.798 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7853e-05 2020-08-07T15:10:27.798 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.798 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:27.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.676861+0000", 2020-08-07T15:10:27.799 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7359999999999999e-06 2020-08-07T15:10:27.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:27.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.676865+0000", 2020-08-07T15:10:27.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33484658099999998 2020-08-07T15:10:27.801 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.827 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:27.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011712+0000", 2020-08-07T15:10:27.828 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1613e-05 2020-08-07T15:10:27.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:27.828 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011723+0000", 2020-08-07T15:10:27.829 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011038 2020-08-07T15:10:27.829 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.829 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.829 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:27.829 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011834+0000", 2020-08-07T15:10:27.830 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0097e-05 2020-08-07T15:10:27.830 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.830 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.830 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:27.830 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011844+0000", 2020-08-07T15:10:27.831 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000143781 2020-08-07T15:10:27.831 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.831 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.831 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:27.831 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011987+0000", 2020-08-07T15:10:27.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.335144041 2020-08-07T15:10:27.832 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.832 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:27.832 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.833 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:145806 3.8 e75/60 ondisk, result = 0)", 2020-08-07T15:10:27.833 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.678157+0000", 2020-08-07T15:10:27.833 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 132.52608007399999, 2020-08-07T15:10:27.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.402718152, 2020-08-07T15:10:27.839 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:27.839 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:27.840 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:27.840 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.840 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:27.840 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678157+0000", 2020-08-07T15:10:27.841 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:27.841 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:27.841 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678157+0000", 2020-08-07T15:10:27.842 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8496e-05 2020-08-07T15:10:27.851 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:27.852 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678176+0000", 2020-08-07T15:10:27.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.676e-06 2020-08-07T15:10:27.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.853 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:27.853 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678180+0000", 2020-08-07T15:10:27.853 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40250966999999999 2020-08-07T15:10:27.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.855 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:27.855 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080689+0000", 2020-08-07T15:10:27.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7110000000000003e-06 2020-08-07T15:10:27.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.868 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:27.868 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080694+0000", 2020-08-07T15:10:27.868 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4122999999999998e-05 2020-08-07T15:10:27.868 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.869 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.869 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:27.869 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080738+0000", 2020-08-07T15:10:27.869 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5809999999999999e-06 2020-08-07T15:10:27.869 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.886 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:27.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080746+0000", 2020-08-07T15:10:27.887 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012989499999999999 2020-08-07T15:10:27.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:27.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:27.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:27.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080876+0000", 2020-08-07T15:10:27.985 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.402718152 2020-08-07T15:10:27.985 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:27.985 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:28.000 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.000 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.000 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.000 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42288 4.2 4:4cfde355:::benchmark_data_smithi074_44586_object5285:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:10:28.001 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.115516+0000", 2020-08-07T15:10:28.001 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 87.088721976000002, 2020-08-07T15:10:28.001 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.53902283500000003, 2020-08-07T15:10:28.001 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:28.001 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:28.019 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:28.020 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:28.020 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:28.020 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42288 2020-08-07T15:10:28.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.020 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:28.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:28.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.115516+0000", 2020-08-07T15:10:28.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:28.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:28.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.115516+0000", 2020-08-07T15:10:28.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5785999999999997e-05 2020-08-07T15:10:28.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:28.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.115571+0000", 2020-08-07T15:10:28.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3130000000000004e-06 2020-08-07T15:10:28.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.030 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:28.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.115578+0000", 2020-08-07T15:10:28.030 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2514000000000001e-05 2020-08-07T15:10:28.030 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.030 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:28.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.115590+0000", 2020-08-07T15:10:28.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7130000000000002e-06 2020-08-07T15:10:28.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:28.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.115596+0000", 2020-08-07T15:10:28.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0036423369999999998 2020-08-07T15:10:28.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:28.033 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.119238+0000", 2020-08-07T15:10:28.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7732999999999995e-05 2020-08-07T15:10:28.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.034 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:28.034 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.119326+0000", 2020-08-07T15:10:28.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000102612 2020-08-07T15:10:28.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:10:28.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.119429+0000", 2020-08-07T15:10:28.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001842621 2020-08-07T15:10:28.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.038 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:28.038 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.121271+0000", 2020-08-07T15:10:28.038 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00096829399999999997 2020-08-07T15:10:28.039 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.039 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.039 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:28.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.122239+0000", 2020-08-07T15:10:28.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.53219874700000003 2020-08-07T15:10:28.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:28.040 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.654438+0000", 2020-08-07T15:10:28.040 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7290000000000001e-06 2020-08-07T15:10:28.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.041 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:28.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.654440+0000", 2020-08-07T15:10:28.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0956999999999999e-05 2020-08-07T15:10:28.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.042 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:28.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.654501+0000", 2020-08-07T15:10:28.042 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7478999999999998e-05 2020-08-07T15:10:28.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:28.043 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.654538+0000", 2020-08-07T15:10:28.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.53902283500000003 2020-08-07T15:10:28.044 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.044 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:28.044 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.045 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.045 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:66488 4.f 4:fdf15c0e:::benchmark_data_smithi074_44586_object8310:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e126)", 2020-08-07T15:10:28.045 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:20.213948+0000", 2020-08-07T15:10:28.045 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 66.990289266999994, 2020-08-07T15:10:28.045 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.34743775999999998, 2020-08-07T15:10:28.046 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:28.046 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:28.046 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:28.047 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:28.048 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:28.048 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 66488 2020-08-07T15:10:28.049 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.049 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:28.059 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.059 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:28.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.213948+0000", 2020-08-07T15:10:28.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:28.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:28.060 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.213948+0000", 2020-08-07T15:10:28.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0775999999999997e-05 2020-08-07T15:10:28.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.061 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:28.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.213989+0000", 2020-08-07T15:10:28.062 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5179999999999998e-06 2020-08-07T15:10:28.062 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.062 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.062 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:28.062 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.213994+0000", 2020-08-07T15:10:28.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7399999999999993e-06 2020-08-07T15:10:28.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.063 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:28.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.214002+0000", 2020-08-07T15:10:28.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5640000000000001e-06 2020-08-07T15:10:28.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.064 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:28.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.214006+0000", 2020-08-07T15:10:28.065 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0035647840000000001 2020-08-07T15:10:28.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.065 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:28.065 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.217571+0000", 2020-08-07T15:10:28.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000155446 2020-08-07T15:10:28.086 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.086 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:28.087 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.217726+0000", 2020-08-07T15:10:28.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00026276899999999999 2020-08-07T15:10:28.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4", 2020-08-07T15:10:28.088 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.217989+0000", 2020-08-07T15:10:28.099 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001388161 2020-08-07T15:10:28.099 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.100 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.100 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:28.100 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.219377+0000", 2020-08-07T15:10:28.100 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9100000000000002e-07 2020-08-07T15:10:28.100 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.101 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.101 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:28.101 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.219378+0000", 2020-08-07T15:10:28.101 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7027999999999998e-05 2020-08-07T15:10:28.101 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.102 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.102 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:28.102 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.219425+0000", 2020-08-07T15:10:28.102 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054840820000000004 2020-08-07T15:10:28.102 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.103 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.103 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:28.103 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.224909+0000", 2020-08-07T15:10:28.103 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33640654599999997 2020-08-07T15:10:28.103 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.104 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.104 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:28.104 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.561316+0000", 2020-08-07T15:10:28.104 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.127e-06 2020-08-07T15:10:28.104 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.105 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.105 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:28.105 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.561317+0000", 2020-08-07T15:10:28.105 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9327999999999998e-05 2020-08-07T15:10:28.105 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.124 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.125 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:28.125 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.561386+0000", 2020-08-07T15:10:28.125 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.34743775999999998 2020-08-07T15:10:28.126 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.126 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:28.126 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.126 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.126 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.127 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72952 4.b 4:dee63f23:::benchmark_data_smithi074_44586_object9118:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:28.127 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.585664+0000", 2020-08-07T15:10:28.127 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 61.618573558000001, 2020-08-07T15:10:28.127 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.411271632, 2020-08-07T15:10:28.128 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:28.128 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:28.128 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:28.128 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:28.128 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:28.129 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72952 2020-08-07T15:10:28.129 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.129 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:28.129 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.141 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:28.141 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.585664+0000", 2020-08-07T15:10:28.142 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:28.142 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.142 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.142 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:28.143 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.585664+0000", 2020-08-07T15:10:28.143 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0828999999999997e-05 2020-08-07T15:10:28.143 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.143 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.144 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:28.144 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.585715+0000", 2020-08-07T15:10:28.146 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1019999999999996e-06 2020-08-07T15:10:28.146 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.146 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.147 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:28.147 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.585720+0000", 2020-08-07T15:10:28.147 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4185000000000001e-05 2020-08-07T15:10:28.147 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.147 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.148 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:28.148 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.585734+0000", 2020-08-07T15:10:28.148 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5009999999999998e-06 2020-08-07T15:10:28.148 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.148 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.149 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:28.149 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.585742+0000", 2020-08-07T15:10:28.149 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0027632669999999998 2020-08-07T15:10:28.149 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.150 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.151 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:28.151 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.588505+0000", 2020-08-07T15:10:28.151 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4177000000000003e-05 2020-08-07T15:10:28.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:28.152 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.588599+0000", 2020-08-07T15:10:28.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013466500000000001 2020-08-07T15:10:28.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.153 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.153 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:10:28.154 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.588734+0000", 2020-08-07T15:10:28.154 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001231181 2020-08-07T15:10:28.154 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.154 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.154 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:28.155 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.589965+0000", 2020-08-07T15:10:28.155 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0083227890000000006 2020-08-07T15:10:28.155 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.156 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:28.156 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.598288+0000", 2020-08-07T15:10:28.156 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39851268299999998 2020-08-07T15:10:28.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.157 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.157 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:28.157 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.996800+0000", 2020-08-07T15:10:28.158 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.821e-06 2020-08-07T15:10:28.159 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.159 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.159 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:28.159 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.996802+0000", 2020-08-07T15:10:28.160 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8633000000000004e-05 2020-08-07T15:10:28.160 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.160 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.160 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:28.161 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.996881+0000", 2020-08-07T15:10:28.161 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4799000000000002e-05 2020-08-07T15:10:28.161 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.161 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.162 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:28.162 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.996936+0000", 2020-08-07T15:10:28.162 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.411271632 2020-08-07T15:10:28.162 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.163 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:28.163 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.163 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.163 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:28.163 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:28.165 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:10:28.167 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 7 15 10870 10855 12.1138 13.0312 0.0082044 0.010289 2020-08-07T15:10:28.167 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:10:28.180 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:28.171+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:28.210 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 8 16 12502 12486 12.1923 12.7422 0.00939654 0.0102416 2020-08-07T15:10:28.270 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:28.269+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:28.375 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:28.376 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:10:28.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.376 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:12726 5.6 5:6b6cbba3:::benchmark_data_smithi074_53697_object1590:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e171)", 2020-08-07T15:10:28.377 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:28.346798+0000", 2020-08-07T15:10:28.377 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.017495441, 2020-08-07T15:10:28.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.017525223999999999, 2020-08-07T15:10:28.377 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:28.379 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:28.379 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:28.380 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:28.386 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:28.386 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 12726 2020-08-07T15:10:28.386 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.386 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:28.387 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.387 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:28.387 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.346798+0000", 2020-08-07T15:10:28.387 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:28.387 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.388 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.388 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:28.388 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.346798+0000", 2020-08-07T15:10:28.388 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0998999999999998e-05 2020-08-07T15:10:28.389 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.389 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.389 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:28.406 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.346839+0000", 2020-08-07T15:10:28.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1760000000000003e-06 2020-08-07T15:10:28.407 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.407 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.407 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:28.408 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.346843+0000", 2020-08-07T15:10:28.408 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7959999999999996e-06 2020-08-07T15:10:28.408 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.408 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.408 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:28.409 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.346851+0000", 2020-08-07T15:10:28.409 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7840000000000003e-06 2020-08-07T15:10:28.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.409 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.418 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:28.418 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.346856+0000", 2020-08-07T15:10:28.419 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00083527600000000003 2020-08-07T15:10:28.419 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.419 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.419 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:28.419 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.347691+0000", 2020-08-07T15:10:28.420 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4124999999999998e-05 2020-08-07T15:10:28.420 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.420 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.420 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:28.421 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.347785+0000", 2020-08-07T15:10:28.421 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000150827 2020-08-07T15:10:28.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.466 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.466 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,7", 2020-08-07T15:10:28.467 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.347936+0000", 2020-08-07T15:10:28.467 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003003618 2020-08-07T15:10:28.467 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.467 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.487 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:28.487 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.350940+0000", 2020-08-07T15:10:28.487 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4299999999999998e-07 2020-08-07T15:10:28.488 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:28.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.350940+0000", 2020-08-07T15:10:28.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3077000000000002e-05 2020-08-07T15:10:28.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:28.511 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.350973+0000", 2020-08-07T15:10:28.511 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005405422 2020-08-07T15:10:28.511 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.512 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.512 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:28.512 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.356379+0000", 2020-08-07T15:10:28.513 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0095805430000000004 2020-08-07T15:10:28.513 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.513 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:28.513 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.514 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.514 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:12727 5.6 5:6b6cbba3:::benchmark_data_smithi074_53697_object1590:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e171)", 2020-08-07T15:10:28.515 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:28.346887+0000", 2020-08-07T15:10:28.515 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.017406187, 2020-08-07T15:10:28.515 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.017564298999999998, 2020-08-07T15:10:28.515 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:28.516 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:28.516 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:28.516 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:28.517 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:28.518 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 12727 2020-08-07T15:10:28.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.518 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:28.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.519 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:28.519 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.346887+0000", 2020-08-07T15:10:28.519 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:28.519 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.520 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:28.520 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.346887+0000", 2020-08-07T15:10:28.520 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6592999999999998e-05 2020-08-07T15:10:28.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.521 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.521 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:28.521 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.346944+0000", 2020-08-07T15:10:28.522 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4500000000000003e-06 2020-08-07T15:10:28.522 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.522 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.522 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:28.522 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.346950+0000", 2020-08-07T15:10:28.523 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2498e-05 2020-08-07T15:10:28.523 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.523 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.523 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:28.524 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.346962+0000", 2020-08-07T15:10:28.524 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.989e-06 2020-08-07T15:10:28.524 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.524 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.525 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:28.525 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.346967+0000", 2020-08-07T15:10:28.525 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002842121 2020-08-07T15:10:28.525 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.534 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.543 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:28.544 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.349809+0000", 2020-08-07T15:10:28.547 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4991000000000001e-05 2020-08-07T15:10:28.548 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.548 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:28.548 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.349884+0000", 2020-08-07T15:10:28.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000149467 2020-08-07T15:10:28.549 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.549 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.549 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,7", 2020-08-07T15:10:28.549 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.350034+0000", 2020-08-07T15:10:28.550 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00094678600000000003 2020-08-07T15:10:28.550 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.550 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.550 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:28.550 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.350980+0000", 2020-08-07T15:10:28.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1500000000000004e-07 2020-08-07T15:10:28.551 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.551 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.551 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:28.552 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.350981+0000", 2020-08-07T15:10:28.552 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4017999999999997e-05 2020-08-07T15:10:28.552 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.553 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:28.567 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.351035+0000", 2020-08-07T15:10:28.567 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041475280000000002 2020-08-07T15:10:28.567 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.567 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:28.568 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.568 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.568 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.568 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:12761 5.a 5:5bad90a1:::benchmark_data_smithi074_53697_object1595:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e171)", 2020-08-07T15:10:28.569 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:28.362334+0000", 2020-08-07T15:10:28.569 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0019596539999999999, 2020-08-07T15:10:28.569 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0022285220000000001, 2020-08-07T15:10:28.569 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:28.570 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:10:28.570 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:28.570 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:28.571 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:28.571 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 12761 2020-08-07T15:10:28.571 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.571 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:28.572 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.572 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:28.572 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.362334+0000", 2020-08-07T15:10:28.572 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:28.572 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.573 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.573 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:28.573 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.362334+0000", 2020-08-07T15:10:28.573 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9431e-05 2020-08-07T15:10:28.580 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.582 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.582 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:28.582 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.362363+0000", 2020-08-07T15:10:28.583 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8279999999999999e-06 2020-08-07T15:10:28.583 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.583 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.583 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:28.584 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.362369+0000", 2020-08-07T15:10:28.584 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2074e-05 2020-08-07T15:10:28.584 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.584 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.585 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:28.585 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.362381+0000", 2020-08-07T15:10:28.585 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6659999999999996e-06 2020-08-07T15:10:28.585 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.585 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.586 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:28.597 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.362387+0000", 2020-08-07T15:10:28.598 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5698999999999997e-05 2020-08-07T15:10:28.598 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.598 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.599 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:28.600 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.362433+0000", 2020-08-07T15:10:28.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014764499999999999 2020-08-07T15:10:28.600 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.600 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.601 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:28.601 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.362580+0000", 2020-08-07T15:10:28.601 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019974000000000001 2020-08-07T15:10:28.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.602 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.602 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4", 2020-08-07T15:10:28.602 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.362780+0000", 2020-08-07T15:10:28.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000446083 2020-08-07T15:10:28.603 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.603 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:28.603 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.603 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.604 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:12762 5.a 5:5bad90a1:::benchmark_data_smithi074_53697_object1595:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e171)", 2020-08-07T15:10:28.604 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:28.362432+0000", 2020-08-07T15:10:28.604 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0018613460000000001, 2020-08-07T15:10:28.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0022462350000000001, 2020-08-07T15:10:28.605 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:28.605 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:10:28.605 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:28.606 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:28.606 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:28.607 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 12762 2020-08-07T15:10:28.607 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.607 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:28.608 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.608 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:28.608 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.362432+0000", 2020-08-07T15:10:28.608 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:28.609 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.609 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.609 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:28.609 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.362432+0000", 2020-08-07T15:10:28.609 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7363e-05 2020-08-07T15:10:28.610 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.611 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.611 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:28.611 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.362480+0000", 2020-08-07T15:10:28.611 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0379999999999999e-06 2020-08-07T15:10:28.611 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.612 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.612 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:28.612 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.362485+0000", 2020-08-07T15:10:28.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9049999999999992e-06 2020-08-07T15:10:28.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.613 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:28.613 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.362495+0000", 2020-08-07T15:10:28.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2150000000000003e-06 2020-08-07T15:10:28.614 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.615 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.616 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:28.616 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.362499+0000", 2020-08-07T15:10:28.616 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00075318100000000001 2020-08-07T15:10:28.617 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.617 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.618 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:28.618 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.363252+0000", 2020-08-07T15:10:28.618 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4921999999999999e-05 2020-08-07T15:10:28.618 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.619 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.619 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:28.619 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.363317+0000", 2020-08-07T15:10:28.619 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000135193 2020-08-07T15:10:28.620 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.620 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.620 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4", 2020-08-07T15:10:28.621 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.363452+0000", 2020-08-07T15:10:28.621 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010198169999999999 2020-08-07T15:10:28.621 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.622 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:28.622 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.622 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.623 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.624 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:12763 5.a 5:5bad90a1:::benchmark_data_smithi074_53697_object1595:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e171)", 2020-08-07T15:10:28.624 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:28.362998+0000", 2020-08-07T15:10:28.624 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.001295228, 2020-08-07T15:10:28.625 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017697979999999999, 2020-08-07T15:10:28.625 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:28.625 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:10:28.625 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:28.626 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:28.626 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:28.626 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 12763 2020-08-07T15:10:28.626 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.627 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:28.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.627 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:28.627 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.362998+0000", 2020-08-07T15:10:28.628 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:28.628 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.628 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.628 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:28.629 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.362998+0000", 2020-08-07T15:10:28.629 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0268000000000001e-05 2020-08-07T15:10:28.630 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.630 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.630 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:28.631 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.363049+0000", 2020-08-07T15:10:28.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0800000000000002e-06 2020-08-07T15:10:28.632 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.632 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.632 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:28.633 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.363055+0000", 2020-08-07T15:10:28.633 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1752e-05 2020-08-07T15:10:28.633 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.633 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.641 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:28.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.363067+0000", 2020-08-07T15:10:28.642 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6189999999999999e-06 2020-08-07T15:10:28.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.643 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.643 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:28.644 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.363072+0000", 2020-08-07T15:10:28.644 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00073360400000000001 2020-08-07T15:10:28.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.644 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.644 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:28.645 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.363806+0000", 2020-08-07T15:10:28.645 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8470000000000001e-05 2020-08-07T15:10:28.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.659 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:28.660 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.363864+0000", 2020-08-07T15:10:28.660 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010114499999999999 2020-08-07T15:10:28.660 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.660 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.660 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4", 2020-08-07T15:10:28.661 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.363965+0000", 2020-08-07T15:10:28.661 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00096693799999999998 2020-08-07T15:10:28.661 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.661 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:28.661 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.662 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.662 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.662 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:12764 5.a 5:5bad90a1:::benchmark_data_smithi074_53697_object1595:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e171)", 2020-08-07T15:10:28.663 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:28.363108+0000", 2020-08-07T15:10:28.663 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0011853499999999999, 2020-08-07T15:10:28.663 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017434620000000001, 2020-08-07T15:10:28.671 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:28.671 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:10:28.671 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:28.672 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:28.672 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:28.672 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 12764 2020-08-07T15:10:28.673 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.673 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:28.673 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.674 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:28.674 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.363108+0000", 2020-08-07T15:10:28.674 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:28.675 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.675 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.675 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:28.676 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.363108+0000", 2020-08-07T15:10:28.676 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7357000000000001e-05 2020-08-07T15:10:28.676 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.676 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:28.677 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.363156+0000", 2020-08-07T15:10:28.677 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7300000000000002e-06 2020-08-07T15:10:28.678 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.678 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.678 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:28.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.363161+0000", 2020-08-07T15:10:28.679 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1092e-05 2020-08-07T15:10:28.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.679 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:28.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.363173+0000", 2020-08-07T15:10:28.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3809999999999998e-06 2020-08-07T15:10:28.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:28.681 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.363178+0000", 2020-08-07T15:10:28.681 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001237812 2020-08-07T15:10:28.681 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.682 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.682 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:28.682 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.364416+0000", 2020-08-07T15:10:28.682 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011325899999999999 2020-08-07T15:10:28.682 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.683 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.683 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:28.683 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.364529+0000", 2020-08-07T15:10:28.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018800699999999999 2020-08-07T15:10:28.683 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.684 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.684 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4", 2020-08-07T15:10:28.684 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.364717+0000", 2020-08-07T15:10:28.684 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0016086379999999999 2020-08-07T15:10:28.684 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.685 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:28.685 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.685 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.685 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.685 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:12384 5.8 5:1a543af4:::benchmark_data_smithi074_53697_object1547:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e171)", 2020-08-07T15:10:28.686 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:28.148845+0000", 2020-08-07T15:10:28.686 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.21544856400000001, 2020-08-07T15:10:28.686 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.21608981799999999, 2020-08-07T15:10:28.686 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:28.686 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:28.687 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:28.687 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:28.687 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:28.687 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 12384 2020-08-07T15:10:28.687 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.688 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:28.688 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.689 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:28.689 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.148845+0000", 2020-08-07T15:10:28.689 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:28.689 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.689 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.690 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:28.690 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.148845+0000", 2020-08-07T15:10:28.690 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8226000000000002e-05 2020-08-07T15:10:28.690 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.690 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.690 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:28.691 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.148893+0000", 2020-08-07T15:10:28.691 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7819999999999999e-06 2020-08-07T15:10:28.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.692 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:28.692 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.148899+0000", 2020-08-07T15:10:28.692 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1189000000000001e-05 2020-08-07T15:10:28.693 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.693 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.693 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:28.693 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.148910+0000", 2020-08-07T15:10:28.693 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6110000000000001e-06 2020-08-07T15:10:28.694 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.694 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.694 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:28.694 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.148916+0000", 2020-08-07T15:10:28.694 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001704243 2020-08-07T15:10:28.695 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.695 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:28.695 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.150620+0000", 2020-08-07T15:10:28.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000107972 2020-08-07T15:10:28.695 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.696 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:28.696 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.150728+0000", 2020-08-07T15:10:28.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016947199999999999 2020-08-07T15:10:28.697 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.698 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.698 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:10:28.698 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.150898+0000", 2020-08-07T15:10:28.707 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0031457970000000001 2020-08-07T15:10:28.707 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.707 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:28.707 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.154043+0000", 2020-08-07T15:10:28.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7e-07 2020-08-07T15:10:28.708 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:28.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.154044+0000", 2020-08-07T15:10:28.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5819000000000002e-05 2020-08-07T15:10:28.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.721 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.722 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:28.722 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.154100+0000", 2020-08-07T15:10:28.751 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.027033673000000001 2020-08-07T15:10:28.751 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.752 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:28.752 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.181133+0000", 2020-08-07T15:10:28.753 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.032288254000000002 2020-08-07T15:10:28.753 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.753 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:28.758 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.758 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.758 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.759 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:12728 5.6 5:6b6cbba3:::benchmark_data_smithi074_53697_object1590:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e171)", 2020-08-07T15:10:28.759 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:28.347980+0000", 2020-08-07T15:10:28.759 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.016313807, 2020-08-07T15:10:28.760 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.017013799, 2020-08-07T15:10:28.760 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:28.761 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:28.761 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:28.767 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:28.767 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:28.792 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 12728 2020-08-07T15:10:28.793 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.793 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:28.794 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.794 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:28.794 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.347980+0000", 2020-08-07T15:10:28.795 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:28.795 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.796 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:28.796 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.347980+0000", 2020-08-07T15:10:28.797 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8616999999999999e-05 2020-08-07T15:10:28.797 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.797 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:28.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.348028+0000", 2020-08-07T15:10:28.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2060000000000004e-06 2020-08-07T15:10:28.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.809 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:28.810 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.348035+0000", 2020-08-07T15:10:28.810 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2334000000000001e-05 2020-08-07T15:10:28.810 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.810 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.810 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:28.811 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.348047+0000", 2020-08-07T15:10:28.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.46e-06 2020-08-07T15:10:28.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.811 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:28.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.348053+0000", 2020-08-07T15:10:28.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023284830000000001 2020-08-07T15:10:28.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:28.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.350382+0000", 2020-08-07T15:10:28.813 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2676999999999994e-05 2020-08-07T15:10:28.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.815 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:28.815 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.350475+0000", 2020-08-07T15:10:28.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013877200000000001 2020-08-07T15:10:28.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,7", 2020-08-07T15:10:28.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.350613+0000", 2020-08-07T15:10:28.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00081101100000000005 2020-08-07T15:10:28.816 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.830 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:28.831 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.351424+0000", 2020-08-07T15:10:28.831 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0699999999999996e-07 2020-08-07T15:10:28.850 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.851 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:28.851 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.351425+0000", 2020-08-07T15:10:28.851 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0670000000000002e-05 2020-08-07T15:10:28.851 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.851 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:28.852 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.351466+0000", 2020-08-07T15:10:28.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034861369999999998 2020-08-07T15:10:28.852 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.852 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:28.853 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.856 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:12765 5.a 5.8509b5da (undecoded) ondisk+write+known_if_redirected e171)", 2020-08-07T15:10:28.857 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:28.364749+0000", 2020-08-07T15:10:28.857 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4294967295.9995446, 2020-08-07T15:10:28.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00029732800000000002, 2020-08-07T15:10:28.857 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:28.858 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:10:28.858 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:28.858 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:28.858 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:28.858 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 12765 2020-08-07T15:10:28.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.859 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:28.878 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.878 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:28.878 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.364749+0000", 2020-08-07T15:10:28.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:28.879 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.879 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:28.879 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.364749+0000", 2020-08-07T15:10:28.880 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4703999999999998e-05 2020-08-07T15:10:28.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.891 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.892 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:28.892 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.364794+0000", 2020-08-07T15:10:28.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2780000000000001e-06 2020-08-07T15:10:28.892 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.892 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.893 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:28.893 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.364799+0000", 2020-08-07T15:10:28.893 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.594e-06 2020-08-07T15:10:28.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:28.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.364809+0000", 2020-08-07T15:10:28.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8210000000000001e-06 2020-08-07T15:10:28.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:28.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.364814+0000", 2020-08-07T15:10:28.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4397e-05 2020-08-07T15:10:28.916 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.916 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:28.916 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.917 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5099.0:12761 5.a e171/164)", 2020-08-07T15:10:28.935 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:28.364784+0000", 2020-08-07T15:10:28.936 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4294967295.9995093, 2020-08-07T15:10:28.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00029667199999999999, 2020-08-07T15:10:28.936 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:28.936 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:10:28.937 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:28.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:28.938 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.364784+0000", 2020-08-07T15:10:28.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:28.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.938 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:28.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.364784+0000", 2020-08-07T15:10:28.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4211000000000001e-05 2020-08-07T15:10:28.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:28.940 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.364829+0000", 2020-08-07T15:10:28.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5959999999999998e-06 2020-08-07T15:10:28.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.940 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.940 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:28.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.364836+0000", 2020-08-07T15:10:28.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9814e-05 2020-08-07T15:10:28.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.950 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:28.950 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.364856+0000", 2020-08-07T15:10:28.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8750000000000002e-06 2020-08-07T15:10:28.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:28.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:28.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:28.951 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:28.364860+0000", 2020-08-07T15:10:28.952 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5495999999999997e-05 2020-08-07T15:10:28.952 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.952 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:28.952 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.952 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:28.952 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:10:28.953 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 10 2020-08-07T15:10:28.953 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:28.962 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:28.374+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:28.963 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:10:28.963 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:10:28.964 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:10:28.964 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:10:28.964 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:10:28.964 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:10:28.982 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:10:28.983 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:10:28.983 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:10:28.984 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:10:28.984 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:10:28.984 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:10:28.984 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:10:28.984 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:10:28.985 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:10:28.985 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:10:28.985 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:28.953+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:28.987 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:10:29.074 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:29.062+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:29.166 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:29.164+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:29.197 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:29.198 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:29.205 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:10:29.211 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 9 16 14255 14239 12.3592 13.6953 0.00378799 0.0101082 2020-08-07T15:10:29.274 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:29.272+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:29.376 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:29.373+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:29.406 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:29.407 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:29.421 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:10:29.487 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:29.483+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:29.583 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:29.582+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:29.630 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:29.631 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:29.632 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:10:29.722 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:29.719+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:29.787 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:29.787+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:29.847 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:29.847 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:29.860 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:10:29.905 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:29.900+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:30.002 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:29.994+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:30.077 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:30.077 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:30.091 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:10:30.132 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:30.131+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:30.172 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:10:30.210 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 10 16 15962 15946 12.4567 13.3359 0.00213975 0.0100026 2020-08-07T15:10:30.211 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:30.209+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:30.264 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:30.264 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:30.285 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:10:30.313 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:30.310+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:30.414 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:30.411+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:30.470 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:30.470 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:30.483 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:10:30.520 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:30.518+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:30.626 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:30.619+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:30.660 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:30.661 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:30.668 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:10:30.724 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:30.721+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:30.831 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:30.829+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:30.857 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:30.858 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:30.871 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:10:30.935 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:30.931+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:30.990 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:10:31.042 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:31.036+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:31.090 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:31.090 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:31.099 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:10:31.142 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:31.136+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:31.206 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:10:31.215 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 11 16 17692 17676 12.5529 13.5156 0.0132403 0.00994496 2020-08-07T15:10:31.246 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:10:31.247 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:31.246+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:31.278 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:31.278 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:31.295 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:10:31.296 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:10:31.316 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:10:31.357 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:31.352+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:31.362 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:10:31.363 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:10:31.407 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:10:31.443 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:10:31.453 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:31.451+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:31.483 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:10:31.487 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:10:31.487 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:10:31.487 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:10:31.487 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:10:31.488 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:10:31.488 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:10:31.488 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:10:31.488 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:10:31.522 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:10:31.523 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:10:31.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:31.523 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:18208 5.3 5:c4b8f0f2:::benchmark_data_smithi074_53697_object2275:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e172)", 2020-08-07T15:10:31.523 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:31.506708+0000", 2020-08-07T15:10:31.523 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.01239663, 2020-08-07T15:10:31.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012426825000000001, 2020-08-07T15:10:31.524 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:31.524 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:31.524 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:31.524 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:10:31.525 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:31.525 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 18208 2020-08-07T15:10:31.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:31.525 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:31.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:31.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:31.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:31.506708+0000", 2020-08-07T15:10:31.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:31.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:31.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:31.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:31.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:31.506708+0000", 2020-08-07T15:10:31.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4579999999999998e-05 2020-08-07T15:10:31.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:31.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:31.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:31.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:31.506742+0000", 2020-08-07T15:10:31.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8759999999999998e-06 2020-08-07T15:10:31.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:31.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:31.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:31.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:31.506746+0000", 2020-08-07T15:10:31.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4010000000000003e-06 2020-08-07T15:10:31.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:31.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:31.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:31.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:31.506756+0000", 2020-08-07T15:10:31.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6460000000000004e-06 2020-08-07T15:10:31.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:31.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:31.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:31.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:31.506762+0000", 2020-08-07T15:10:31.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019618560000000001 2020-08-07T15:10:31.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:31.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:31.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:31.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:31.508724+0000", 2020-08-07T15:10:31.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6595000000000006e-05 2020-08-07T15:10:31.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:31.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:31.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:31.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:31.508821+0000", 2020-08-07T15:10:31.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102013 2020-08-07T15:10:31.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:31.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:31.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,5", 2020-08-07T15:10:31.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:31.508923+0000", 2020-08-07T15:10:31.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035762490000000001 2020-08-07T15:10:31.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:31.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:31.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:31.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:31.512499+0000", 2020-08-07T15:10:31.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3600000000000003e-07 2020-08-07T15:10:31.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:31.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:31.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:31.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:31.512500+0000", 2020-08-07T15:10:31.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5665000000000001e-05 2020-08-07T15:10:31.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:31.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:31.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:31.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:31.512575+0000", 2020-08-07T15:10:31.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005867517 2020-08-07T15:10:31.554 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:31.554 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:31.554 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:31.555 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:31.555 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:10:31.555 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 1 2020-08-07T15:10:31.555 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:10:31.583 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:10:31.584 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:31.582+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:31.698 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:31.697+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:31.769 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:10:31.769 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:10:31.769 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:10:31.770 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:10:31.770 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:10:31.778 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:10:31.849 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:31.848+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:31.919 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:31.917+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:31.958 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:10:31.958 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:10:31.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:10:31.959 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:10:31.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:31.959 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86695 3.a 3:5058ba7c:::benchmark_data_smithi074_35447_object10836:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:10:31.959 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:10:31.959 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 177.627728932, 2020-08-07T15:10:31.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31490022699999998, 2020-08-07T15:10:31.960 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:31.961 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:31.961 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:31.961 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:10:31.961 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:31.970 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86695 2020-08-07T15:10:31.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:31.970 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:31.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:31.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:31.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:10:31.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:31.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:31.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:31.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:31.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325328+0000", 2020-08-07T15:10:31.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4560000000000005e-05 2020-08-07T15:10:31.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:31.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:31.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:31.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325392+0000", 2020-08-07T15:10:31.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3180000000000003e-06 2020-08-07T15:10:31.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:31.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:31.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:31.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325398+0000", 2020-08-07T15:10:31.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.258e-05 2020-08-07T15:10:31.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:31.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:31.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:31.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325411+0000", 2020-08-07T15:10:31.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6550000000000001e-06 2020-08-07T15:10:31.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:31.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:31.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:31.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325417+0000", 2020-08-07T15:10:31.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024282879999999998 2020-08-07T15:10:31.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:31.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:31.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:31.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.327845+0000", 2020-08-07T15:10:31.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1203000000000001e-05 2020-08-07T15:10:31.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:31.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:31.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:31.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.327936+0000", 2020-08-07T15:10:32.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000186505 2020-08-07T15:10:32.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:10:32.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328123+0000", 2020-08-07T15:10:32.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020062890000000001 2020-08-07T15:10:32.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:32.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.330129+0000", 2020-08-07T15:10:32.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121513 2020-08-07T15:10:32.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:32.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.330250+0000", 2020-08-07T15:10:32.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30989746899999998 2020-08-07T15:10:32.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:32.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640148+0000", 2020-08-07T15:10:32.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2139999999999999e-06 2020-08-07T15:10:32.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:32.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640149+0000", 2020-08-07T15:10:32.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8081999999999998e-05 2020-08-07T15:10:32.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:32.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640197+0000", 2020-08-07T15:10:32.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0550999999999999e-05 2020-08-07T15:10:32.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:32.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640228+0000", 2020-08-07T15:10:32.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31490022699999998 2020-08-07T15:10:32.026 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.027 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:32.027 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.027 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86696 3.a 3:5058ba7c:::benchmark_data_smithi074_35447_object10836:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:10:32.028 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:10:32.028 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 177.62760425299999, 2020-08-07T15:10:32.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.347347556, 2020-08-07T15:10:32.028 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:32.028 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:32.028 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:32.029 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:10:32.029 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:32.029 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86696 2020-08-07T15:10:32.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.031 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:32.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:32.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:10:32.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:32.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:32.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325452+0000", 2020-08-07T15:10:32.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0436999999999998e-05 2020-08-07T15:10:32.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:32.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325503+0000", 2020-08-07T15:10:32.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4659999999999998e-06 2020-08-07T15:10:32.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:32.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325508+0000", 2020-08-07T15:10:32.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1177e-05 2020-08-07T15:10:32.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:32.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325519+0000", 2020-08-07T15:10:32.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7809999999999996e-06 2020-08-07T15:10:32.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:32.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.325524+0000", 2020-08-07T15:10:32.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002989219 2020-08-07T15:10:32.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:32.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328513+0000", 2020-08-07T15:10:32.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0649000000000006e-05 2020-08-07T15:10:32.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:32.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328604+0000", 2020-08-07T15:10:32.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001652 2020-08-07T15:10:32.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:10:32.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.328769+0000", 2020-08-07T15:10:32.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023470639999999998 2020-08-07T15:10:32.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:32.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331116+0000", 2020-08-07T15:10:32.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6099999999999999e-07 2020-08-07T15:10:32.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:32.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331117+0000", 2020-08-07T15:10:32.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9150000000000001e-05 2020-08-07T15:10:32.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:32.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.331176+0000", 2020-08-07T15:10:32.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065402581000000001 2020-08-07T15:10:32.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:32.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.396579+0000", 2020-08-07T15:10:32.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27618104599999999 2020-08-07T15:10:32.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:32.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672760+0000", 2020-08-07T15:10:32.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2020-08-07T15:10:32.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:32.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672761+0000", 2020-08-07T15:10:32.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9211000000000001e-05 2020-08-07T15:10:32.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:32.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.672800+0000", 2020-08-07T15:10:32.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.347347556 2020-08-07T15:10:32.080 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.080 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:32.080 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.081 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:86695 3.a e52/20 ondisk, result = 0)", 2020-08-07T15:10:32.081 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:10:32.081 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 177.61964755599999, 2020-08-07T15:10:32.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.306823501, 2020-08-07T15:10:32.107 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:32.107 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:32.108 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:32.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:32.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:10:32.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:32.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:32.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333409+0000", 2020-08-07T15:10:32.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.048e-05 2020-08-07T15:10:32.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:32.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333439+0000", 2020-08-07T15:10:32.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9990000000000002e-06 2020-08-07T15:10:32.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:32.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.333443+0000", 2020-08-07T15:10:32.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30664195100000002 2020-08-07T15:10:32.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:32.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640085+0000", 2020-08-07T15:10:32.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8559999999999998e-06 2020-08-07T15:10:32.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:32.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640090+0000", 2020-08-07T15:10:32.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5166000000000001e-05 2020-08-07T15:10:32.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:32.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640135+0000", 2020-08-07T15:10:32.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4160000000000001e-06 2020-08-07T15:10:32.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:32.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640142+0000", 2020-08-07T15:10:32.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0632999999999998e-05 2020-08-07T15:10:32.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:32.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.640232+0000", 2020-08-07T15:10:32.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.306823501 2020-08-07T15:10:32.129 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.129 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:32.129 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.130 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106232 3.a 3:5599d7e6:::benchmark_data_smithi074_35447_object13278:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:10:32.130 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:10:32.130 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 164.60744930600001, 2020-08-07T15:10:32.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52578228699999996, 2020-08-07T15:10:32.130 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:32.131 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:32.131 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:32.131 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:10:32.131 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:32.131 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106232 2020-08-07T15:10:32.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.132 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:32.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:32.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:10:32.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:32.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:32.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:10:32.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4214999999999997e-05 2020-08-07T15:10:32.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:32.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345641+0000", 2020-08-07T15:10:32.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9279999999999993e-06 2020-08-07T15:10:32.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:32.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345649+0000", 2020-08-07T15:10:32.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4391e-05 2020-08-07T15:10:32.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:32.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345664+0000", 2020-08-07T15:10:32.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7170000000000002e-06 2020-08-07T15:10:32.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:32.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345670+0000", 2020-08-07T15:10:32.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.459e-05 2020-08-07T15:10:32.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:32.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345695+0000", 2020-08-07T15:10:32.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6070999999999999e-05 2020-08-07T15:10:32.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:32.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345751+0000", 2020-08-07T15:10:32.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010069300000000001 2020-08-07T15:10:32.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:10:32.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345852+0000", 2020-08-07T15:10:32.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013512299999999999 2020-08-07T15:10:32.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:32.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.347203+0000", 2020-08-07T15:10:32.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0844000000000001e-05 2020-08-07T15:10:32.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:32.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.347274+0000", 2020-08-07T15:10:32.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52399471600000003 2020-08-07T15:10:32.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:32.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871269+0000", 2020-08-07T15:10:32.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.311e-06 2020-08-07T15:10:32.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:32.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871270+0000", 2020-08-07T15:10:32.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3509000000000004e-05 2020-08-07T15:10:32.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:32.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871343+0000", 2020-08-07T15:10:32.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6072000000000002e-05 2020-08-07T15:10:32.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:32.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871389+0000", 2020-08-07T15:10:32.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52578228699999996 2020-08-07T15:10:32.148 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.148 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:32.149 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.150 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106232 3.a e59/20 ondisk, result = 0)", 2020-08-07T15:10:32.150 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:10:32.150 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 164.604403131, 2020-08-07T15:10:32.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52274359299999995, 2020-08-07T15:10:32.150 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:32.151 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:32.151 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:32.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:32.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:10:32.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:32.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:32.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:10:32.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5011000000000001e-05 2020-08-07T15:10:32.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:32.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348668+0000", 2020-08-07T15:10:32.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1959999999999998e-06 2020-08-07T15:10:32.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:32.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348671+0000", 2020-08-07T15:10:32.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52253456499999995 2020-08-07T15:10:32.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:32.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871205+0000", 2020-08-07T15:10:32.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0230000000000003e-06 2020-08-07T15:10:32.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:32.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871209+0000", 2020-08-07T15:10:32.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0708999999999999e-05 2020-08-07T15:10:32.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:32.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871250+0000", 2020-08-07T15:10:32.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0158e-05 2020-08-07T15:10:32.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:32.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871260+0000", 2020-08-07T15:10:32.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013693099999999999 2020-08-07T15:10:32.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:32.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871397+0000", 2020-08-07T15:10:32.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52274359299999995 2020-08-07T15:10:32.161 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.162 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:32.162 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.162 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:145807 3.c e75/20 3:39741db6:::benchmark_data_smithi074_35447_object16198:head v 75'9594, mlcod=75'9590)", 2020-08-07T15:10:32.163 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.674086+0000", 2020-08-07T15:10:32.163 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 137.27897091400001, 2020-08-07T15:10:32.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40268798500000003, 2020-08-07T15:10:32.163 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:32.163 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:32.163 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:32.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:32.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674086+0000", 2020-08-07T15:10:32.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:32.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:32.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674086+0000", 2020-08-07T15:10:32.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3321000000000001e-05 2020-08-07T15:10:32.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:32.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674129+0000", 2020-08-07T15:10:32.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1919999999999999e-06 2020-08-07T15:10:32.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:32.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.674136+0000", 2020-08-07T15:10:32.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40193062899999998 2020-08-07T15:10:32.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:32.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076067+0000", 2020-08-07T15:10:32.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3579e-05 2020-08-07T15:10:32.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:32.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076080+0000", 2020-08-07T15:10:32.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3733999999999998e-05 2020-08-07T15:10:32.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:32.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076124+0000", 2020-08-07T15:10:32.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0145e-05 2020-08-07T15:10:32.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:32.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076154+0000", 2020-08-07T15:10:32.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00055768600000000005 2020-08-07T15:10:32.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:32.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076712+0000", 2020-08-07T15:10:32.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1699000000000007e-05 2020-08-07T15:10:32.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:32.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.076774+0000", 2020-08-07T15:10:32.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40268798500000003 2020-08-07T15:10:32.222 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.222 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:32.223 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.223 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:144 4.5 4:aed0a56a:::benchmark_data_smithi074_44586_object17:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e83)", 2020-08-07T15:10:32.223 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:26.396584+0000", 2020-08-07T15:10:32.224 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 125.55647267499999, 2020-08-07T15:10:32.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45809661000000002, 2020-08-07T15:10:32.225 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:32.225 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:32.225 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:32.225 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:32.226 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:32.226 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 144 2020-08-07T15:10:32.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.226 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:32.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:32.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396584+0000", 2020-08-07T15:10:32.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:32.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:32.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396584+0000", 2020-08-07T15:10:32.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4195e-05 2020-08-07T15:10:32.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:32.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396628+0000", 2020-08-07T15:10:32.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.157e-06 2020-08-07T15:10:32.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:32.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396631+0000", 2020-08-07T15:10:32.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.832e-06 2020-08-07T15:10:32.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:32.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396638+0000", 2020-08-07T15:10:32.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.745e-06 2020-08-07T15:10:32.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:32.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.396641+0000", 2020-08-07T15:10:32.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001659368 2020-08-07T15:10:32.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:32.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.398300+0000", 2020-08-07T15:10:32.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2652000000000007e-05 2020-08-07T15:10:32.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:32.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.398393+0000", 2020-08-07T15:10:32.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016763699999999999 2020-08-07T15:10:32.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,7", 2020-08-07T15:10:32.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.398560+0000", 2020-08-07T15:10:32.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023917719999999999 2020-08-07T15:10:32.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:32.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.400952+0000", 2020-08-07T15:10:32.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999997e-07 2020-08-07T15:10:32.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:32.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.400953+0000", 2020-08-07T15:10:32.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9373999999999998e-05 2020-08-07T15:10:32.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:32.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.401012+0000", 2020-08-07T15:10:32.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003107865 2020-08-07T15:10:32.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:32.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.404120+0000", 2020-08-07T15:10:32.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45049484299999998 2020-08-07T15:10:32.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:32.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854615+0000", 2020-08-07T15:10:32.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.035e-06 2020-08-07T15:10:32.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:32.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854616+0000", 2020-08-07T15:10:32.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4249000000000001e-05 2020-08-07T15:10:32.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:32.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:26.854680+0000", 2020-08-07T15:10:32.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45809661000000002 2020-08-07T15:10:32.243 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.243 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:32.243 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.244 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:30975 4.5 e98/81 ondisk, result = 0)", 2020-08-07T15:10:32.244 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:45.734259+0000", 2020-08-07T15:10:32.244 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 106.21879757799999, 2020-08-07T15:10:32.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46200962800000001, 2020-08-07T15:10:32.244 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:32.245 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:32.245 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:32.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:32.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.734259+0000", 2020-08-07T15:10:32.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:32.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:32.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.734259+0000", 2020-08-07T15:10:32.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9501000000000001e-05 2020-08-07T15:10:32.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:32.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.734288+0000", 2020-08-07T15:10:32.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5990000000000004e-06 2020-08-07T15:10:32.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:32.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.734293+0000", 2020-08-07T15:10:32.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46181377000000001 2020-08-07T15:10:32.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:32.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.196107+0000", 2020-08-07T15:10:32.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6169999999999998e-06 2020-08-07T15:10:32.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:32.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.196112+0000", 2020-08-07T15:10:32.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5908999999999997e-05 2020-08-07T15:10:32.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:32.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.196158+0000", 2020-08-07T15:10:32.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1104000000000001e-05 2020-08-07T15:10:32.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:32.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.196169+0000", 2020-08-07T15:10:32.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9128000000000004e-05 2020-08-07T15:10:32.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:32.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.196268+0000", 2020-08-07T15:10:32.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46200962800000001 2020-08-07T15:10:32.253 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.255 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:32.255 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.255 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:30976 4.5 e98/81 ondisk, result = 0)", 2020-08-07T15:10:32.256 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:45.736493+0000", 2020-08-07T15:10:32.256 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 106.21656340600001, 2020-08-07T15:10:32.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54677504399999999, 2020-08-07T15:10:32.257 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:32.257 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:32.257 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:32.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:32.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736493+0000", 2020-08-07T15:10:32.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:32.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:32.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736493+0000", 2020-08-07T15:10:32.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.012e-05 2020-08-07T15:10:32.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:32.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736523+0000", 2020-08-07T15:10:32.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6669999999999998e-06 2020-08-07T15:10:32.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:32.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736527+0000", 2020-08-07T15:10:32.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54657333200000002 2020-08-07T15:10:32.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:32.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283100+0000", 2020-08-07T15:10:32.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.549e-06 2020-08-07T15:10:32.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:32.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283105+0000", 2020-08-07T15:10:32.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1853000000000006e-05 2020-08-07T15:10:32.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:32.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283167+0000", 2020-08-07T15:10:32.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015e-05 2020-08-07T15:10:32.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:32.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283177+0000", 2020-08-07T15:10:32.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1508000000000005e-05 2020-08-07T15:10:32.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:32.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283268+0000", 2020-08-07T15:10:32.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54677504399999999 2020-08-07T15:10:32.276 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.276 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:32.276 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.277 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31528 4.5 4:a883779b:::benchmark_data_smithi074_44586_object3940:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e100)", 2020-08-07T15:10:32.277 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:47.795320+0000", 2020-08-07T15:10:32.277 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 104.15773681, 2020-08-07T15:10:32.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33015256900000001, 2020-08-07T15:10:32.278 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:32.278 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:32.278 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:32.278 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:32.279 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:32.279 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31528 2020-08-07T15:10:32.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.279 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:32.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:32.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.795320+0000", 2020-08-07T15:10:32.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:32.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:32.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.795320+0000", 2020-08-07T15:10:32.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9305000000000003e-05 2020-08-07T15:10:32.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:32.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.795369+0000", 2020-08-07T15:10:32.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1659999999999999e-06 2020-08-07T15:10:32.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:32.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.795375+0000", 2020-08-07T15:10:32.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2284e-05 2020-08-07T15:10:32.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:32.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.795387+0000", 2020-08-07T15:10:32.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.789e-06 2020-08-07T15:10:32.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:32.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.795393+0000", 2020-08-07T15:10:32.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025794009999999998 2020-08-07T15:10:32.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:32.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.797973+0000", 2020-08-07T15:10:32.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011622599999999999 2020-08-07T15:10:32.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:32.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.798089+0000", 2020-08-07T15:10:32.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021524900000000001 2020-08-07T15:10:32.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,7", 2020-08-07T15:10:32.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.798304+0000", 2020-08-07T15:10:32.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042865830000000001 2020-08-07T15:10:32.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:32.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.802591+0000", 2020-08-07T15:10:32.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0088220250000000007 2020-08-07T15:10:32.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:32.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.811413+0000", 2020-08-07T15:10:32.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31390628199999998 2020-08-07T15:10:32.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:32.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125319+0000", 2020-08-07T15:10:32.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.037e-06 2020-08-07T15:10:32.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:32.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125320+0000", 2020-08-07T15:10:32.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0878000000000003e-05 2020-08-07T15:10:32.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:32.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125371+0000", 2020-08-07T15:10:32.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101344 2020-08-07T15:10:32.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:32.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125472+0000", 2020-08-07T15:10:32.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33015256900000001 2020-08-07T15:10:32.312 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.312 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:32.312 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.313 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:31536 4.c e100/99 4:326df2a4:::benchmark_data_smithi074_44586_object3941:head v 100'2008, mlcod=100'1999)", 2020-08-07T15:10:32.313 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:47.804640+0000", 2020-08-07T15:10:32.313 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 104.148416238, 2020-08-07T15:10:32.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32071613199999999, 2020-08-07T15:10:32.314 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:32.314 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:32.314 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:32.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:32.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.804640+0000", 2020-08-07T15:10:32.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:32.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:32.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.804640+0000", 2020-08-07T15:10:32.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9385999999999998e-05 2020-08-07T15:10:32.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:32.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.804690+0000", 2020-08-07T15:10:32.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2319999999999998e-06 2020-08-07T15:10:32.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:32.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.804696+0000", 2020-08-07T15:10:32.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.319733934 2020-08-07T15:10:32.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:32.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.124430+0000", 2020-08-07T15:10:32.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1089999999999997e-06 2020-08-07T15:10:32.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:32.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.124435+0000", 2020-08-07T15:10:32.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9981999999999998e-05 2020-08-07T15:10:32.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:32.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.124475+0000", 2020-08-07T15:10:32.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0684e-05 2020-08-07T15:10:32.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:32.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.124506+0000", 2020-08-07T15:10:32.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00079925600000000003 2020-08-07T15:10:32.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:32.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125305+0000", 2020-08-07T15:10:32.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1548999999999997e-05 2020-08-07T15:10:32.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:32.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125356+0000", 2020-08-07T15:10:32.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32071613199999999 2020-08-07T15:10:32.359 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.359 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:32.360 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.360 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:31528 4.5 e100/81 ondisk, result = 0)", 2020-08-07T15:10:32.360 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:47.812815+0000", 2020-08-07T15:10:32.361 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 104.140241311, 2020-08-07T15:10:32.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.312664363, 2020-08-07T15:10:32.361 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:32.361 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:32.361 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:32.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:32.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.812815+0000", 2020-08-07T15:10:32.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:32.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:32.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.812815+0000", 2020-08-07T15:10:32.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2224999999999997e-05 2020-08-07T15:10:32.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:32.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.812847+0000", 2020-08-07T15:10:32.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9460000000000003e-06 2020-08-07T15:10:32.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:32.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.812853+0000", 2020-08-07T15:10:32.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31240245100000003 2020-08-07T15:10:32.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:32.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125256+0000", 2020-08-07T15:10:32.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7509999999999997e-06 2020-08-07T15:10:32.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:32.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125262+0000", 2020-08-07T15:10:32.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9220000000000001e-05 2020-08-07T15:10:32.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:32.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125301+0000", 2020-08-07T15:10:32.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3700000000000001e-06 2020-08-07T15:10:32.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:32.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125310+0000", 2020-08-07T15:10:32.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001694 2020-08-07T15:10:32.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:32.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.125479+0000", 2020-08-07T15:10:32.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.312664363 2020-08-07T15:10:32.395 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.396 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:32.396 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.396 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:31816 4.1 e100/98 4:8a2ebf91:::benchmark_data_smithi074_44586_object3976:head v 100'1872, mlcod=100'1866)", 2020-08-07T15:10:32.397 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.169272+0000", 2020-08-07T15:10:32.428 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 103.783784199, 2020-08-07T15:10:32.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38326060099999998, 2020-08-07T15:10:32.428 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:32.429 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:32.429 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:32.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:32.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.169272+0000", 2020-08-07T15:10:32.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:32.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:32.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.169272+0000", 2020-08-07T15:10:32.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.93e-05 2020-08-07T15:10:32.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:32.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.169312+0000", 2020-08-07T15:10:32.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0230000000000003e-06 2020-08-07T15:10:32.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:32.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.169316+0000", 2020-08-07T15:10:32.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38224064899999999 2020-08-07T15:10:32.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:32.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.551556+0000", 2020-08-07T15:10:32.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.673e-06 2020-08-07T15:10:32.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:32.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.551561+0000", 2020-08-07T15:10:32.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0315999999999996e-05 2020-08-07T15:10:32.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:32.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.551631+0000", 2020-08-07T15:10:32.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6877999999999999e-05 2020-08-07T15:10:32.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:32.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.551658+0000", 2020-08-07T15:10:32.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00081916599999999997 2020-08-07T15:10:32.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:32.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.552477+0000", 2020-08-07T15:10:32.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5596000000000001e-05 2020-08-07T15:10:32.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:32.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.552533+0000", 2020-08-07T15:10:32.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38326060099999998 2020-08-07T15:10:32.444 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.445 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:32.445 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.445 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:37048 4.5 4:a403bd70:::benchmark_data_smithi074_44586_object4630:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e106)", 2020-08-07T15:10:32.446 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:54.237330+0000", 2020-08-07T15:10:32.446 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 97.715726329000006, 2020-08-07T15:10:32.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.474059283, 2020-08-07T15:10:32.446 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:32.448 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:32.448 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:32.448 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:32.449 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:32.449 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 37048 2020-08-07T15:10:32.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.449 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:32.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:32.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237330+0000", 2020-08-07T15:10:32.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:32.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:32.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237330+0000", 2020-08-07T15:10:32.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0713999999999998e-05 2020-08-07T15:10:32.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:32.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237361+0000", 2020-08-07T15:10:32.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.756e-06 2020-08-07T15:10:32.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:32.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237364+0000", 2020-08-07T15:10:32.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.694e-06 2020-08-07T15:10:32.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:32.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237369+0000", 2020-08-07T15:10:32.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3709999999999998e-06 2020-08-07T15:10:32.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:32.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237372+0000", 2020-08-07T15:10:32.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015644700000000001 2020-08-07T15:10:32.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:32.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.238936+0000", 2020-08-07T15:10:32.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9940999999999998e-05 2020-08-07T15:10:32.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:32.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.239026+0000", 2020-08-07T15:10:32.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017423800000000001 2020-08-07T15:10:32.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,7", 2020-08-07T15:10:32.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.239200+0000", 2020-08-07T15:10:32.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041267530000000004 2020-08-07T15:10:32.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:32.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.243327+0000", 2020-08-07T15:10:32.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7700000000000004e-07 2020-08-07T15:10:32.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:32.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.243328+0000", 2020-08-07T15:10:32.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1211999999999997e-05 2020-08-07T15:10:32.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:32.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.243379+0000", 2020-08-07T15:10:32.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075155150000000004 2020-08-07T15:10:32.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:32.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.250894+0000", 2020-08-07T15:10:32.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.460450313 2020-08-07T15:10:32.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:32.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.711345+0000", 2020-08-07T15:10:32.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1990000000000001e-06 2020-08-07T15:10:32.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:32.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.711346+0000", 2020-08-07T15:10:32.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3529999999999998e-05 2020-08-07T15:10:32.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:32.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.711389+0000", 2020-08-07T15:10:32.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.474059283 2020-08-07T15:10:32.477 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.478 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:32.478 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.478 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:46032 4.c e117/104 4:3eb0d548:::benchmark_data_smithi074_44586_object5753:head v 117'2968, mlcod=117'2958)", 2020-08-07T15:10:32.479 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:05.163259+0000", 2020-08-07T15:10:32.479 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.789796981999999, 2020-08-07T15:10:32.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31522351999999998, 2020-08-07T15:10:32.479 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:32.479 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:32.480 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:32.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:32.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.163259+0000", 2020-08-07T15:10:32.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:32.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:32.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.163259+0000", 2020-08-07T15:10:32.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9008000000000002e-05 2020-08-07T15:10:32.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:32.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.163318+0000", 2020-08-07T15:10:32.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2689999999999996e-06 2020-08-07T15:10:32.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:32.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.163325+0000", 2020-08-07T15:10:32.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.312753646 2020-08-07T15:10:32.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:32.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.476078+0000", 2020-08-07T15:10:32.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4379999999999999e-06 2020-08-07T15:10:32.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:32.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.476082+0000", 2020-08-07T15:10:32.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2913999999999998e-05 2020-08-07T15:10:32.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:32.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.476175+0000", 2020-08-07T15:10:32.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5939e-05 2020-08-07T15:10:32.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:32.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.476201+0000", 2020-08-07T15:10:32.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022305070000000001 2020-08-07T15:10:32.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:32.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.478431+0000", 2020-08-07T15:10:32.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1799000000000003e-05 2020-08-07T15:10:32.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:32.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:05.478483+0000", 2020-08-07T15:10:32.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31522351999999998 2020-08-07T15:10:32.500 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.500 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:32.501 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.502 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:60960 4.e 4:73b21c89:::benchmark_data_smithi074_44586_object7619:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e121)", 2020-08-07T15:10:32.502 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:14.254254+0000", 2020-08-07T15:10:32.502 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.698802461, 2020-08-07T15:10:32.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.29219231800000001, 2020-08-07T15:10:32.502 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:32.503 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:32.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:32.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:32.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:32.503 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 60960 2020-08-07T15:10:32.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.504 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:32.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:32.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.254254+0000", 2020-08-07T15:10:32.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:32.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:32.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.254254+0000", 2020-08-07T15:10:32.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9551e-05 2020-08-07T15:10:32.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:32.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.254304+0000", 2020-08-07T15:10:32.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7420000000000003e-06 2020-08-07T15:10:32.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:32.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.254309+0000", 2020-08-07T15:10:32.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0723000000000001e-05 2020-08-07T15:10:32.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:32.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.254320+0000", 2020-08-07T15:10:32.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1070000000000004e-06 2020-08-07T15:10:32.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:32.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.254325+0000", 2020-08-07T15:10:32.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00072346499999999996 2020-08-07T15:10:32.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:32.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.255049+0000", 2020-08-07T15:10:32.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010703400000000001 2020-08-07T15:10:32.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:32.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.255156+0000", 2020-08-07T15:10:32.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000179338 2020-08-07T15:10:32.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:10:32.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.255335+0000", 2020-08-07T15:10:32.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041513380000000001 2020-08-07T15:10:32.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:32.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.259486+0000", 2020-08-07T15:10:32.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9700000000000002e-07 2020-08-07T15:10:32.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:32.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.259487+0000", 2020-08-07T15:10:32.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3769999999999994e-05 2020-08-07T15:10:32.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:32.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.259550+0000", 2020-08-07T15:10:32.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.069610947000000006 2020-08-07T15:10:32.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:32.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.329161+0000", 2020-08-07T15:10:32.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.217207706 2020-08-07T15:10:32.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:32.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.546369+0000", 2020-08-07T15:10:32.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.612e-06 2020-08-07T15:10:32.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:32.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.546371+0000", 2020-08-07T15:10:32.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5587999999999995e-05 2020-08-07T15:10:32.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:32.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.546446+0000", 2020-08-07T15:10:32.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.29219231800000001 2020-08-07T15:10:32.522 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.522 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:32.522 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.523 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:60960 4.e e121/98 ondisk, result = 0)", 2020-08-07T15:10:32.523 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:14.257727+0000", 2020-08-07T15:10:32.523 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.695329790000002, 2020-08-07T15:10:32.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28872502300000003, 2020-08-07T15:10:32.523 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:32.524 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:32.524 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:32.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:32.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.257727+0000", 2020-08-07T15:10:32.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:32.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:32.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.257727+0000", 2020-08-07T15:10:32.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5155e-05 2020-08-07T15:10:32.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:32.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.257742+0000", 2020-08-07T15:10:32.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4930000000000001e-06 2020-08-07T15:10:32.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:32.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.257744+0000", 2020-08-07T15:10:32.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28856093399999999 2020-08-07T15:10:32.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:32.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.546305+0000", 2020-08-07T15:10:32.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.045e-06 2020-08-07T15:10:32.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:32.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.546310+0000", 2020-08-07T15:10:32.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0689999999999998e-05 2020-08-07T15:10:32.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:32.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.546351+0000", 2020-08-07T15:10:32.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3400000000000004e-06 2020-08-07T15:10:32.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:32.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.546360+0000", 2020-08-07T15:10:32.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1366000000000006e-05 2020-08-07T15:10:32.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:32.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:14.546452+0000", 2020-08-07T15:10:32.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28872502300000003 2020-08-07T15:10:32.535 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.535 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:32.536 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.536 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:66144 4.c e126/104 4:3c80a167:::benchmark_data_smithi074_44586_object8267:head v 126'4280, mlcod=126'4274)", 2020-08-07T15:10:32.541 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:20.004507+0000", 2020-08-07T15:10:32.543 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 71.948549306000004, 2020-08-07T15:10:32.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30587809900000001, 2020-08-07T15:10:32.544 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:32.544 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:32.544 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:32.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:32.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.004507+0000", 2020-08-07T15:10:32.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:32.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:32.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.004507+0000", 2020-08-07T15:10:32.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9455e-05 2020-08-07T15:10:32.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:32.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.004557+0000", 2020-08-07T15:10:32.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2020-08-07T15:10:32.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:32.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.004563+0000", 2020-08-07T15:10:32.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30196788899999999 2020-08-07T15:10:32.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:32.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.306531+0000", 2020-08-07T15:10:32.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.259e-06 2020-08-07T15:10:32.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:32.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.306536+0000", 2020-08-07T15:10:32.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1167000000000001e-05 2020-08-07T15:10:32.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:32.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.306578+0000", 2020-08-07T15:10:32.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4280000000000001e-05 2020-08-07T15:10:32.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:32.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.306602+0000", 2020-08-07T15:10:32.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037278799999999998 2020-08-07T15:10:32.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:32.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.310330+0000", 2020-08-07T15:10:32.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5454000000000001e-05 2020-08-07T15:10:32.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:32.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:20.310385+0000", 2020-08-07T15:10:32.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30587809900000001 2020-08-07T15:10:32.555 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.555 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:32.557 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.557 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:105544 4.c e147/104 4:36abba5f:::benchmark_data_smithi074_44586_object13192:head v 147'6592, mlcod=147'6582)", 2020-08-07T15:10:32.558 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:49.257184+0000", 2020-08-07T15:10:32.558 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.695872151000003, 2020-08-07T15:10:32.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42401245100000001, 2020-08-07T15:10:32.558 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:32.558 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:32.559 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:32.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:32.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:49.257184+0000", 2020-08-07T15:10:32.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:32.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:32.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:49.257184+0000", 2020-08-07T15:10:32.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9108e-05 2020-08-07T15:10:32.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:32.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:49.257213+0000", 2020-08-07T15:10:32.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.703e-06 2020-08-07T15:10:32.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:32.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:49.257217+0000", 2020-08-07T15:10:32.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42086945100000001 2020-08-07T15:10:32.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:32.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:49.678087+0000", 2020-08-07T15:10:32.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4559999999999999e-06 2020-08-07T15:10:32.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:32.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:49.678092+0000", 2020-08-07T15:10:32.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4265999999999999e-05 2020-08-07T15:10:32.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:32.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:49.678136+0000", 2020-08-07T15:10:32.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2603999999999999e-05 2020-08-07T15:10:32.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:32.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:49.678159+0000", 2020-08-07T15:10:32.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029859919999999998 2020-08-07T15:10:32.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:32.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:49.681145+0000", 2020-08-07T15:10:32.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1870999999999999e-05 2020-08-07T15:10:32.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:32.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:49.681197+0000", 2020-08-07T15:10:32.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42401245100000001 2020-08-07T15:10:32.570 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.570 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:32.570 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.571 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:128869 4.e 4:713a1178:::benchmark_data_smithi074_44586_object12633:head [delete] snapc 0=[] ondisk+write+known_if_redirected e155)", 2020-08-07T15:10:32.571 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:03.724293+0000", 2020-08-07T15:10:32.571 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.228763400999998, 2020-08-07T15:10:32.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31921112899999998, 2020-08-07T15:10:32.571 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:32.572 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:32.572 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:32.572 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:32.572 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:32.572 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128869 2020-08-07T15:10:32.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.573 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:32.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:32.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.724293+0000", 2020-08-07T15:10:32.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:32.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:32.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.724293+0000", 2020-08-07T15:10:32.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8221000000000001e-05 2020-08-07T15:10:32.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:32.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.724311+0000", 2020-08-07T15:10:32.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1969999999999998e-06 2020-08-07T15:10:32.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:32.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.724315+0000", 2020-08-07T15:10:32.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7659999999999993e-06 2020-08-07T15:10:32.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:32.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.724325+0000", 2020-08-07T15:10:32.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6439999999999999e-06 2020-08-07T15:10:32.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:32.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.724330+0000", 2020-08-07T15:10:32.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7154000000000003e-05 2020-08-07T15:10:32.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:32.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.724377+0000", 2020-08-07T15:10:32.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8620000000000001e-05 2020-08-07T15:10:32.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:32.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.724476+0000", 2020-08-07T15:10:32.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010416099999999999 2020-08-07T15:10:32.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:10:32.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.724580+0000", 2020-08-07T15:10:32.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011972249999999999 2020-08-07T15:10:32.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:32.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.725777+0000", 2020-08-07T15:10:32.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000400216 2020-08-07T15:10:32.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:32.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.726177+0000", 2020-08-07T15:10:32.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31718939200000001 2020-08-07T15:10:32.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:32.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.043367+0000", 2020-08-07T15:10:32.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7260000000000001e-06 2020-08-07T15:10:32.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:32.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.043368+0000", 2020-08-07T15:10:32.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1083999999999998e-05 2020-08-07T15:10:32.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:32.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.043449+0000", 2020-08-07T15:10:32.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4722999999999998e-05 2020-08-07T15:10:32.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:32.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.043504+0000", 2020-08-07T15:10:32.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31921112899999998 2020-08-07T15:10:32.588 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.589 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:32.589 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.589 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.589 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:32.589 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:10:32.590 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:10:32.590 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 0 1 2020-08-07T15:10:32.592 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 12 16 19479 19463 12.6701 13.9609 0.00539483 0.00985683 2020-08-07T15:10:32.593 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:32.590+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:32.594 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:10:32.696 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:32.694+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:32.799 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:32.795+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:32.813 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:10:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:10:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:20500 5.d e173/164 5:b20ae120:::benchmark_data_smithi074_53697_object2562:head v 173'1276, mlcod=173'1271)", 2020-08-07T15:10:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:32.810253+0000", 2020-08-07T15:10:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0021247470000000002, 2020-08-07T15:10:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021499259999999999, 2020-08-07T15:10:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.810253+0000", 2020-08-07T15:10:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.810253+0000", 2020-08-07T15:10:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5321000000000001e-05 2020-08-07T15:10:32.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.810309+0000", 2020-08-07T15:10:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1799999999999999e-06 2020-08-07T15:10:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.810316+0000", 2020-08-07T15:10:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7759999999999998e-05 2020-08-07T15:10:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.810354+0000", 2020-08-07T15:10:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0499999999999999e-06 2020-08-07T15:10:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.810359+0000", 2020-08-07T15:10:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6718999999999999e-05 2020-08-07T15:10:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.810395+0000", 2020-08-07T15:10:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3074e-05 2020-08-07T15:10:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:32.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.810418+0000", 2020-08-07T15:10:32.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016510400000000001 2020-08-07T15:10:32.866 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.866 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:32.874 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.878 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:20501 5.d e173/164 5:b20ae120:::benchmark_data_smithi074_53697_object2562:head v 173'1277, mlcod=173'1271)", 2020-08-07T15:10:32.878 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:32.811299+0000", 2020-08-07T15:10:32.879 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0010793479999999999, 2020-08-07T15:10:32.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011788910000000001, 2020-08-07T15:10:32.879 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:32.879 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:32.879 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:32.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:32.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.811299+0000", 2020-08-07T15:10:32.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:32.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:32.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.811299+0000", 2020-08-07T15:10:32.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1665000000000002e-05 2020-08-07T15:10:32.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:32.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.811340+0000", 2020-08-07T15:10:32.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3440000000000003e-06 2020-08-07T15:10:32.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:32.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.811345+0000", 2020-08-07T15:10:32.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5197000000000002e-05 2020-08-07T15:10:32.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:32.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.811380+0000", 2020-08-07T15:10:32.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3619999999999997e-06 2020-08-07T15:10:32.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:32.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.811385+0000", 2020-08-07T15:10:32.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4440999999999998e-05 2020-08-07T15:10:32.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:32.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.811420+0000", 2020-08-07T15:10:32.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4703999999999999e-05 2020-08-07T15:10:32.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:32.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.811444+0000", 2020-08-07T15:10:32.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014571300000000001 2020-08-07T15:10:32.936 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.936 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:32.936 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.936 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:20488 5.4 5:2c350347:::benchmark_data_smithi074_53697_object2560:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e173)", 2020-08-07T15:10:32.937 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:32.803041+0000", 2020-08-07T15:10:32.937 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.009337026, 2020-08-07T15:10:32.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0095112200000000008, 2020-08-07T15:10:32.937 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:32.938 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:32.938 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:32.938 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:10:32.938 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:32.938 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20488 2020-08-07T15:10:32.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.939 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:32.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:32.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.803041+0000", 2020-08-07T15:10:32.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:32.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:32.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.803041+0000", 2020-08-07T15:10:32.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00066547700000000004 2020-08-07T15:10:32.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:32.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.803707+0000", 2020-08-07T15:10:32.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0249999999999999e-06 2020-08-07T15:10:32.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:32.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.803713+0000", 2020-08-07T15:10:32.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.112e-05 2020-08-07T15:10:32.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:32.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.803724+0000", 2020-08-07T15:10:32.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5509999999999999e-06 2020-08-07T15:10:32.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:32.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.803729+0000", 2020-08-07T15:10:32.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00094590199999999998 2020-08-07T15:10:32.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:32.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.804675+0000", 2020-08-07T15:10:32.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4451999999999996e-05 2020-08-07T15:10:32.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:32.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.804770+0000", 2020-08-07T15:10:32.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017497199999999999 2020-08-07T15:10:32.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,5", 2020-08-07T15:10:32.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.804945+0000", 2020-08-07T15:10:32.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017816799999999999 2020-08-07T15:10:32.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:32.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.806726+0000", 2020-08-07T15:10:32.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999999e-07 2020-08-07T15:10:32.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:32.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.806727+0000", 2020-08-07T15:10:32.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2289e-05 2020-08-07T15:10:32.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:32.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.806779+0000", 2020-08-07T15:10:32.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00086472299999999997 2020-08-07T15:10:32.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:32.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:32.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:32.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.807644+0000", 2020-08-07T15:10:32.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046026269999999998 2020-08-07T15:10:32.990 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.990 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:32.990 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.991 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:32.991 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:10:32.991 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 3 2020-08-07T15:10:32.995 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:10:33.000 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:10:33.001 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:32.996+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:33.145 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:33.130+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:33.214 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 13 15 21180 21165 12.7183 13.2969 0.00679038 0.00981613 2020-08-07T15:10:33.224 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:10:33.225 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:10:33.225 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:10:33.226 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:10:33.226 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:10:33.244 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:33.232+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:33.245 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:10:33.346 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:33.340+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:33.362 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.0 to 1 (10000) 2020-08-07T15:10:33.375 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 1 1 2020-08-07T15:10:33.388 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:10:33.420 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:10:33.420 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:10:33.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:10:33.421 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:10:33.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.421 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15576 3.e 3:727724a5:::benchmark_data_smithi074_35447_object1946:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:10:33.422 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:10:33.422 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 226.75084558200001, 2020-08-07T15:10:33.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999, 2020-08-07T15:10:33.422 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:33.422 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:33.423 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:33.423 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:10:33.423 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:33.423 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15576 2020-08-07T15:10:33.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.446 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:33.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:33.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:10:33.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:33.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:33.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:10:33.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7156e-05 2020-08-07T15:10:33.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:33.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666682+0000", 2020-08-07T15:10:33.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0209999999999999e-06 2020-08-07T15:10:33.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:33.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666687+0000", 2020-08-07T15:10:33.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0730999999999999e-05 2020-08-07T15:10:33.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:33.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666698+0000", 2020-08-07T15:10:33.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2020-08-07T15:10:33.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:33.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666702+0000", 2020-08-07T15:10:33.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023978459999999999 2020-08-07T15:10:33.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:33.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669100+0000", 2020-08-07T15:10:33.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9664000000000001e-05 2020-08-07T15:10:33.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:33.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669190+0000", 2020-08-07T15:10:33.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015957100000000001 2020-08-07T15:10:33.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:10:33.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669350+0000", 2020-08-07T15:10:33.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016470149999999999 2020-08-07T15:10:33.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:33.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:10:33.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1299999999999999e-07 2020-08-07T15:10:33.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:33.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:10:33.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5269000000000002e-05 2020-08-07T15:10:33.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:33.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.671022+0000", 2020-08-07T15:10:33.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031414400000000001 2020-08-07T15:10:33.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:33.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.674163+0000", 2020-08-07T15:10:33.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45641417400000001 2020-08-07T15:10:33.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:33.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:10:33.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0399999999999995e-07 2020-08-07T15:10:33.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:33.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:10:33.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3343999999999997e-05 2020-08-07T15:10:33.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:33.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130622+0000", 2020-08-07T15:10:33.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999 2020-08-07T15:10:33.486 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.486 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:33.487 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.487 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:41432 3.d 3:b5c8a160:::benchmark_data_smithi074_35447_object5178:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:10:33.488 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:10:33.488 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 210.62519035700001, 2020-08-07T15:10:33.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.37720701699999998, 2020-08-07T15:10:33.489 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:33.489 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:33.489 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:33.489 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:10:33.490 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:33.490 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 41432 2020-08-07T15:10:33.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.490 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:33.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:33.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:10:33.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:33.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:33.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:10:33.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4787999999999999e-05 2020-08-07T15:10:33.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:33.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792335+0000", 2020-08-07T15:10:33.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5999999999999997e-06 2020-08-07T15:10:33.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:33.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792341+0000", 2020-08-07T15:10:33.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0998e-05 2020-08-07T15:10:33.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:33.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792352+0000", 2020-08-07T15:10:33.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2989999999999996e-06 2020-08-07T15:10:33.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:33.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792357+0000", 2020-08-07T15:10:33.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001852752 2020-08-07T15:10:33.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:33.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794210+0000", 2020-08-07T15:10:33.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5771000000000002e-05 2020-08-07T15:10:33.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:33.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794305+0000", 2020-08-07T15:10:33.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012776399999999999 2020-08-07T15:10:33.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:10:33.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794433+0000", 2020-08-07T15:10:33.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00069673599999999999 2020-08-07T15:10:33.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:33.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795130+0000", 2020-08-07T15:10:33.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1500000000000001e-07 2020-08-07T15:10:33.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:33.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795130+0000", 2020-08-07T15:10:33.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7319e-05 2020-08-07T15:10:33.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:33.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795167+0000", 2020-08-07T15:10:33.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27910026700000001 2020-08-07T15:10:33.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:33.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074268+0000", 2020-08-07T15:10:33.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.095150660999999997 2020-08-07T15:10:33.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:33.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169418+0000", 2020-08-07T15:10:33.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.423e-06 2020-08-07T15:10:33.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:33.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169420+0000", 2020-08-07T15:10:33.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7423999999999996e-05 2020-08-07T15:10:33.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:33.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169497+0000", 2020-08-07T15:10:33.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.37720701699999998 2020-08-07T15:10:33.510 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.510 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:33.510 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.511 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:42104 3.d 3:bcc35439:::benchmark_data_smithi074_35447_object5262:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:10:33.511 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:10:33.511 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 210.178558186, 2020-08-07T15:10:33.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28812091400000001, 2020-08-07T15:10:33.512 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:33.512 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:33.512 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:33.512 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:10:33.512 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:33.513 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42104 2020-08-07T15:10:33.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.513 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:33.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:33.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:10:33.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:33.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:33.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238922+0000", 2020-08-07T15:10:33.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9734000000000002e-05 2020-08-07T15:10:33.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:33.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238972+0000", 2020-08-07T15:10:33.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6910000000000002e-06 2020-08-07T15:10:33.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:33.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238978+0000", 2020-08-07T15:10:33.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1864e-05 2020-08-07T15:10:33.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:33.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238990+0000", 2020-08-07T15:10:33.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.737e-06 2020-08-07T15:10:33.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:33.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.238996+0000", 2020-08-07T15:10:33.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00283943 2020-08-07T15:10:33.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:33.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.241836+0000", 2020-08-07T15:10:33.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4712999999999999e-05 2020-08-07T15:10:33.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:33.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.241930+0000", 2020-08-07T15:10:33.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015328499999999999 2020-08-07T15:10:33.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:10:33.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.242084+0000", 2020-08-07T15:10:33.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016104400000000001 2020-08-07T15:10:33.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:33.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243694+0000", 2020-08-07T15:10:33.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1399999999999997e-07 2020-08-07T15:10:33.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:33.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243695+0000", 2020-08-07T15:10:33.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7800999999999998e-05 2020-08-07T15:10:33.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:33.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.243753+0000", 2020-08-07T15:10:33.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090405661999999998 2020-08-07T15:10:33.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:33.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.334158+0000", 2020-08-07T15:10:33.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19283929299999999 2020-08-07T15:10:33.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.526997+0000", 2020-08-07T15:10:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.012e-06 2020-08-07T15:10:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:33.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.526998+0000", 2020-08-07T15:10:33.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4737999999999998e-05 2020-08-07T15:10:33.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:33.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.527043+0000", 2020-08-07T15:10:33.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28812091400000001 2020-08-07T15:10:33.534 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.534 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:33.535 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.535 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:86712 3.8 e52/20 3:14925daa:::benchmark_data_smithi074_35447_object10838:head v 52'5488, mlcod=52'5479)", 2020-08-07T15:10:33.535 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.338631+0000", 2020-08-07T15:10:33.536 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 179.078849088, 2020-08-07T15:10:33.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.310229594, 2020-08-07T15:10:33.536 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:33.536 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:33.537 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:33.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:33.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338631+0000", 2020-08-07T15:10:33.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:33.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:33.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338631+0000", 2020-08-07T15:10:33.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6180000000000001e-05 2020-08-07T15:10:33.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:33.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338688+0000", 2020-08-07T15:10:33.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4949999999999996e-06 2020-08-07T15:10:33.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:33.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338694+0000", 2020-08-07T15:10:33.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30438292099999997 2020-08-07T15:10:33.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:33.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643077+0000", 2020-08-07T15:10:33.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9389999999999996e-06 2020-08-07T15:10:33.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:33.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643082+0000", 2020-08-07T15:10:33.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5096999999999999e-05 2020-08-07T15:10:33.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:33.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643127+0000", 2020-08-07T15:10:33.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4323000000000001e-05 2020-08-07T15:10:33.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:33.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643151+0000", 2020-08-07T15:10:33.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055847300000000004 2020-08-07T15:10:33.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:33.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.648736+0000", 2020-08-07T15:10:33.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012490900000000001 2020-08-07T15:10:33.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:33.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.648861+0000", 2020-08-07T15:10:33.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.310229594 2020-08-07T15:10:33.556 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.556 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:33.556 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.556 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86792 3.f 3:fc5386ee:::benchmark_data_smithi074_35447_object10848:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:10:33.557 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.395402+0000", 2020-08-07T15:10:33.557 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 179.022078978, 2020-08-07T15:10:33.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30259831999999998, 2020-08-07T15:10:33.557 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:33.571 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:33.572 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:33.572 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:10:33.572 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:33.572 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86792 2020-08-07T15:10:33.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.573 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:33.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:33.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395402+0000", 2020-08-07T15:10:33.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:33.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:33.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395402+0000", 2020-08-07T15:10:33.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2262999999999998e-05 2020-08-07T15:10:33.582 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:10:33.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:33.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395444+0000", 2020-08-07T15:10:33.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1800000000000004e-06 2020-08-07T15:10:33.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:33.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395449+0000", 2020-08-07T15:10:33.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0738e-05 2020-08-07T15:10:33.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:33.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395460+0000", 2020-08-07T15:10:33.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7219999999999999e-06 2020-08-07T15:10:33.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:33.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395464+0000", 2020-08-07T15:10:33.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0063169089999999999 2020-08-07T15:10:33.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:33.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.401781+0000", 2020-08-07T15:10:33.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010702200000000001 2020-08-07T15:10:33.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:33.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.401888+0000", 2020-08-07T15:10:33.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019171999999999999 2020-08-07T15:10:33.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:10:33.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.402080+0000", 2020-08-07T15:10:33.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042866839999999998 2020-08-07T15:10:33.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:33.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.406367+0000", 2020-08-07T15:10:33.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000001e-07 2020-08-07T15:10:33.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:33.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.406367+0000", 2020-08-07T15:10:33.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4720000000000004e-05 2020-08-07T15:10:33.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:33.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.406432+0000", 2020-08-07T15:10:33.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050405365000000001 2020-08-07T15:10:33.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:33.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.456837+0000", 2020-08-07T15:10:33.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24111702700000001 2020-08-07T15:10:33.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:33.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.697954+0000", 2020-08-07T15:10:33.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.22e-06 2020-08-07T15:10:33.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:33.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.697955+0000", 2020-08-07T15:10:33.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4336999999999999e-05 2020-08-07T15:10:33.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:33.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.698000+0000", 2020-08-07T15:10:33.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30259831999999998 2020-08-07T15:10:33.605 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.606 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:33.606 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.608 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:104104 3.8 e58/20 3:11c276ea:::benchmark_data_smithi074_35447_object13012:head v 58'6496, mlcod=58'6487)", 2020-08-07T15:10:33.608 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:10:33.608 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 167.55899483600001, 2020-08-07T15:10:33.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33178497099999998, 2020-08-07T15:10:33.609 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:33.609 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:33.609 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:33.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:33.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:10:33.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:33.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:33.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:10:33.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9961999999999999e-05 2020-08-07T15:10:33.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:33.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858516+0000", 2020-08-07T15:10:33.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3200000000000001e-06 2020-08-07T15:10:33.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:33.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858520+0000", 2020-08-07T15:10:33.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32756790000000002 2020-08-07T15:10:33.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:33.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186088+0000", 2020-08-07T15:10:33.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0260000000000004e-06 2020-08-07T15:10:33.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:33.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186094+0000", 2020-08-07T15:10:33.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7919000000000001e-05 2020-08-07T15:10:33.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:33.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186132+0000", 2020-08-07T15:10:33.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4968e-05 2020-08-07T15:10:33.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:33.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186147+0000", 2020-08-07T15:10:33.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040671520000000001 2020-08-07T15:10:33.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:33.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.190214+0000", 2020-08-07T15:10:33.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6724000000000001e-05 2020-08-07T15:10:33.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:33.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.190271+0000", 2020-08-07T15:10:33.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33178497099999998 2020-08-07T15:10:33.621 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.621 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:33.623 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.623 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106248 3.f 3:f950fd9d:::benchmark_data_smithi074_35447_object13280:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:10:33.624 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:10:33.624 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 166.057316042, 2020-08-07T15:10:33.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002, 2020-08-07T15:10:33.624 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:33.624 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:33.625 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:33.625 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:10:33.625 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:33.625 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106248 2020-08-07T15:10:33.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.626 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:33.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:33.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:10:33.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:33.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:33.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:10:33.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7184e-05 2020-08-07T15:10:33.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:33.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360222+0000", 2020-08-07T15:10:33.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0906e-05 2020-08-07T15:10:33.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:33.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360233+0000", 2020-08-07T15:10:33.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2972999999999999e-05 2020-08-07T15:10:33.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:33.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360246+0000", 2020-08-07T15:10:33.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0114e-05 2020-08-07T15:10:33.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:33.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360256+0000", 2020-08-07T15:10:33.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1303999999999997e-05 2020-08-07T15:10:33.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:33.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360287+0000", 2020-08-07T15:10:33.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3798000000000001e-05 2020-08-07T15:10:33.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:33.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360351+0000", 2020-08-07T15:10:33.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111308 2020-08-07T15:10:33.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:10:33.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360462+0000", 2020-08-07T15:10:33.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003480674 2020-08-07T15:10:33.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:33.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:10:33.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0900000000000001e-07 2020-08-07T15:10:33.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:33.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:10:33.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8899999999999994e-05 2020-08-07T15:10:33.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:33.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.364012+0000", 2020-08-07T15:10:33.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35291637199999998 2020-08-07T15:10:33.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:33.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716929+0000", 2020-08-07T15:10:33.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.076244233999999994 2020-08-07T15:10:33.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:33.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793173+0000", 2020-08-07T15:10:33.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6109999999999999e-06 2020-08-07T15:10:33.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:33.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793175+0000", 2020-08-07T15:10:33.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7492999999999999e-05 2020-08-07T15:10:33.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:33.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793242+0000", 2020-08-07T15:10:33.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002 2020-08-07T15:10:33.654 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.654 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:33.654 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.655 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106248 3.f e59/20 ondisk, result = 0)", 2020-08-07T15:10:33.655 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.365323+0000", 2020-08-07T15:10:33.655 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 166.052157486, 2020-08-07T15:10:33.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35160734999999999, 2020-08-07T15:10:33.655 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:33.655 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:33.656 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:33.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:33.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365323+0000", 2020-08-07T15:10:33.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:33.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:33.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365323+0000", 2020-08-07T15:10:33.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4981000000000003e-05 2020-08-07T15:10:33.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:33.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365358+0000", 2020-08-07T15:10:33.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1090000000000003e-06 2020-08-07T15:10:33.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:33.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365364+0000", 2020-08-07T15:10:33.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35151899399999997 2020-08-07T15:10:33.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:33.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716883+0000", 2020-08-07T15:10:33.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0090000000000001e-06 2020-08-07T15:10:33.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:33.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716887+0000", 2020-08-07T15:10:33.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9238999999999999e-05 2020-08-07T15:10:33.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:33.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716916+0000", 2020-08-07T15:10:33.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0390000000000001e-06 2020-08-07T15:10:33.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:33.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716922+0000", 2020-08-07T15:10:33.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9789999999999993e-06 2020-08-07T15:10:33.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:33.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716930+0000", 2020-08-07T15:10:33.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35160734999999999 2020-08-07T15:10:33.709 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.709 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:33.709 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.720 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106248 3.f e59/20 ondisk, result = 0)", 2020-08-07T15:10:33.720 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:10:33.720 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 166.05205275700001, 2020-08-07T15:10:33.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001, 2020-08-07T15:10:33.721 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:33.721 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:33.721 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:33.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:33.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:10:33.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:33.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:33.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:10:33.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7518999999999999e-05 2020-08-07T15:10:33.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:33.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365445+0000", 2020-08-07T15:10:33.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-06 2020-08-07T15:10:33.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:33.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365449+0000", 2020-08-07T15:10:33.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42764147600000002 2020-08-07T15:10:33.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:33.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793091+0000", 2020-08-07T15:10:33.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6500000000000002e-06 2020-08-07T15:10:33.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:33.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793099+0000", 2020-08-07T15:10:33.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3664e-05 2020-08-07T15:10:33.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:33.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793153+0000", 2020-08-07T15:10:33.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1078e-05 2020-08-07T15:10:33.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:33.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793164+0000", 2020-08-07T15:10:33.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3404000000000003e-05 2020-08-07T15:10:33.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:33.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793248+0000", 2020-08-07T15:10:33.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001 2020-08-07T15:10:33.731 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.731 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:33.732 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.733 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106344 3.e 3:7abbcafa:::benchmark_data_smithi074_35447_object13292:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:10:33.745 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:48.059687+0000", 2020-08-07T15:10:33.745 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 165.35779365499999, 2020-08-07T15:10:33.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28541866100000002, 2020-08-07T15:10:33.745 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:33.746 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:33.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:33.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:10:33.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:33.749 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106344 2020-08-07T15:10:33.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.750 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:33.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:33.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059687+0000", 2020-08-07T15:10:33.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:33.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:33.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059687+0000", 2020-08-07T15:10:33.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5778999999999998e-05 2020-08-07T15:10:33.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:33.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059713+0000", 2020-08-07T15:10:33.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.422e-06 2020-08-07T15:10:33.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:33.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059716+0000", 2020-08-07T15:10:33.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7679999999999997e-06 2020-08-07T15:10:33.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:33.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059725+0000", 2020-08-07T15:10:33.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2020-08-07T15:10:33.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:33.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059729+0000", 2020-08-07T15:10:33.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6205000000000003e-05 2020-08-07T15:10:33.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:33.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059765+0000", 2020-08-07T15:10:33.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3615999999999995e-05 2020-08-07T15:10:33.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:33.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059829+0000", 2020-08-07T15:10:33.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000118903 2020-08-07T15:10:33.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:10:33.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.059948+0000", 2020-08-07T15:10:33.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00070666299999999995 2020-08-07T15:10:33.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:33.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.060654+0000", 2020-08-07T15:10:33.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5199999999999995e-07 2020-08-07T15:10:33.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:33.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.060655+0000", 2020-08-07T15:10:33.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2092999999999998e-05 2020-08-07T15:10:33.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:33.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.060697+0000", 2020-08-07T15:10:33.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00038454500000000003 2020-08-07T15:10:33.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:33.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.061082+0000", 2020-08-07T15:10:33.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.283939936 2020-08-07T15:10:33.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:33.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.345022+0000", 2020-08-07T15:10:33.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.288e-06 2020-08-07T15:10:33.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:33.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.345023+0000", 2020-08-07T15:10:33.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2468999999999999e-05 2020-08-07T15:10:33.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:33.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:48.345105+0000", 2020-08-07T15:10:33.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.28541866100000002 2020-08-07T15:10:33.767 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.768 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:33.768 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.769 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:107952 3.d e61/35)", 2020-08-07T15:10:33.769 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:10:33.770 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 163.091123049, 2020-08-07T15:10:33.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001, 2020-08-07T15:10:33.770 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:33.770 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:33.771 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:33.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:33.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:10:33.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:33.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:33.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:10:33.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8826000000000003e-05 2020-08-07T15:10:33.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:33.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326406+0000", 2020-08-07T15:10:33.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9199999999999998e-06 2020-08-07T15:10:33.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:33.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326413+0000", 2020-08-07T15:10:33.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75082853500000002 2020-08-07T15:10:33.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:33.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077242+0000", 2020-08-07T15:10:33.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1479999999999999e-06 2020-08-07T15:10:33.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:33.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077246+0000", 2020-08-07T15:10:33.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3331999999999998e-05 2020-08-07T15:10:33.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:33.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077289+0000", 2020-08-07T15:10:33.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8977000000000001e-05 2020-08-07T15:10:33.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:33.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077308+0000", 2020-08-07T15:10:33.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001 2020-08-07T15:10:33.783 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.783 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:33.784 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.784 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31696 4.0 4:0335eed9:::benchmark_data_smithi074_44586_object3961:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e100)", 2020-08-07T15:10:33.784 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:10:33.785 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 105.31321549, 2020-08-07T15:10:33.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237, 2020-08-07T15:10:33.785 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:33.785 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:33.785 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:33.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:33.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:33.786 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31696 2020-08-07T15:10:33.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.787 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:33.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:33.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:10:33.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:33.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:33.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:10:33.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7654e-05 2020-08-07T15:10:33.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:33.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104293+0000", 2020-08-07T15:10:33.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.061e-06 2020-08-07T15:10:33.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:33.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104296+0000", 2020-08-07T15:10:33.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0859999999999998e-06 2020-08-07T15:10:33.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:33.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104302+0000", 2020-08-07T15:10:33.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8030000000000001e-06 2020-08-07T15:10:33.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:33.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104305+0000", 2020-08-07T15:10:33.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037634769999999999 2020-08-07T15:10:33.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:33.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108068+0000", 2020-08-07T15:10:33.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8814999999999998e-05 2020-08-07T15:10:33.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:33.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108137+0000", 2020-08-07T15:10:33.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111455 2020-08-07T15:10:33.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:10:33.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108248+0000", 2020-08-07T15:10:33.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028756820000000001 2020-08-07T15:10:33.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:33.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111124+0000", 2020-08-07T15:10:33.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061054499999999999 2020-08-07T15:10:33.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:33.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111735+0000", 2020-08-07T15:10:33.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.423966167 2020-08-07T15:10:33.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:33.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535701+0000", 2020-08-07T15:10:33.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4929999999999999e-06 2020-08-07T15:10:33.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:33.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535702+0000", 2020-08-07T15:10:33.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2571999999999997e-05 2020-08-07T15:10:33.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:33.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535755+0000", 2020-08-07T15:10:33.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1427000000000003e-05 2020-08-07T15:10:33.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:33.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535786+0000", 2020-08-07T15:10:33.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237 2020-08-07T15:10:33.804 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.805 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:33.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.805 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:31696 4.0 e100/81 ondisk, result = 0)", 2020-08-07T15:10:33.806 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:10:33.806 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 105.30360093199999, 2020-08-07T15:10:33.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954, 2020-08-07T15:10:33.806 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:33.806 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:33.807 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:33.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:33.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:10:33.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:33.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:33.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:10:33.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1185000000000001e-05 2020-08-07T15:10:33.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:33.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113901+0000", 2020-08-07T15:10:33.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2930000000000002e-06 2020-08-07T15:10:33.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:33.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113905+0000", 2020-08-07T15:10:33.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42168640400000001 2020-08-07T15:10:33.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:33.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535591+0000", 2020-08-07T15:10:33.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2820000000000002e-06 2020-08-07T15:10:33.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:33.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535598+0000", 2020-08-07T15:10:33.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6578999999999999e-05 2020-08-07T15:10:33.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:33.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535684+0000", 2020-08-07T15:10:33.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8449999999999992e-06 2020-08-07T15:10:33.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:33.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535693+0000", 2020-08-07T15:10:33.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7366000000000003e-05 2020-08-07T15:10:33.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:33.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535791+0000", 2020-08-07T15:10:33.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954 2020-08-07T15:10:33.826 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.826 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:33.827 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.827 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:37024 4.0 e106/81)", 2020-08-07T15:10:33.827 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:10:33.827 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 99.188812037999995, 2020-08-07T15:10:33.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999, 2020-08-07T15:10:33.828 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:33.828 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:33.828 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:33.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:33.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:10:33.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:33.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:33.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:10:33.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4926000000000001e-05 2020-08-07T15:10:33.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:33.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228683+0000", 2020-08-07T15:10:33.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1280000000000002e-06 2020-08-07T15:10:33.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:33.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228686+0000", 2020-08-07T15:10:33.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69740012100000004 2020-08-07T15:10:33.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:33.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926086+0000", 2020-08-07T15:10:33.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000004e-06 2020-08-07T15:10:33.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:33.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926090+0000", 2020-08-07T15:10:33.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113526 2020-08-07T15:10:33.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:33.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926204+0000", 2020-08-07T15:10:33.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8824000000000001e-05 2020-08-07T15:10:33.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:33.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926222+0000", 2020-08-07T15:10:33.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999 2020-08-07T15:10:33.884 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.884 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:33.884 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.885 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42503 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2615, mlcod=112'2607)", 2020-08-07T15:10:33.885 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:10:33.885 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.121240122000003, 2020-08-07T15:10:33.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998, 2020-08-07T15:10:33.887 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:33.887 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:33.888 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:33.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:33.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:10:33.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:33.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:33.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:10:33.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270999999999998e-05 2020-08-07T15:10:33.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:33.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296292+0000", 2020-08-07T15:10:33.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4729999999999997e-06 2020-08-07T15:10:33.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:33.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296298+0000", 2020-08-07T15:10:33.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.448795049 2020-08-07T15:10:33.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:33.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745093+0000", 2020-08-07T15:10:33.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0140000000000001e-06 2020-08-07T15:10:33.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:33.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745100+0000", 2020-08-07T15:10:33.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2596999999999998e-05 2020-08-07T15:10:33.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:33.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745153+0000", 2020-08-07T15:10:33.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1662000000000003e-05 2020-08-07T15:10:33.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:33.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745184+0000", 2020-08-07T15:10:33.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093174800000000004 2020-08-07T15:10:33.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:33.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746116+0000", 2020-08-07T15:10:33.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0926999999999994e-05 2020-08-07T15:10:33.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:33.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746187+0000", 2020-08-07T15:10:33.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998 2020-08-07T15:10:33.900 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.900 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:33.900 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.901 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42504 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2616, mlcod=113'2609)", 2020-08-07T15:10:33.901 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:10:33.901 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.117903229000007, 2020-08-07T15:10:33.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998, 2020-08-07T15:10:33.901 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:33.902 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:33.902 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:33.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:33.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:10:33.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:33.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:33.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:10:33.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2154000000000002e-05 2020-08-07T15:10:33.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:33.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299629+0000", 2020-08-07T15:10:33.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1929999999999998e-06 2020-08-07T15:10:33.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:33.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299636+0000", 2020-08-07T15:10:33.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44766315400000001 2020-08-07T15:10:33.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:33.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747299+0000", 2020-08-07T15:10:33.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5889999999999999e-06 2020-08-07T15:10:33.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:33.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747305+0000", 2020-08-07T15:10:33.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3094999999999999e-05 2020-08-07T15:10:33.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747358+0000", 2020-08-07T15:10:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3283e-05 2020-08-07T15:10:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747392+0000", 2020-08-07T15:10:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015319509999999999 2020-08-07T15:10:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:33.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748924+0000", 2020-08-07T15:10:33.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5235999999999996e-05 2020-08-07T15:10:33.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:33.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748999+0000", 2020-08-07T15:10:33.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998 2020-08-07T15:10:33.913 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.914 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:33.914 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.915 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42519 4.8 e113/81 4:13071de5:::benchmark_data_smithi074_44586_object5314:head v 113'2623, mlcod=113'2617)", 2020-08-07T15:10:33.915 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.762041+0000", 2020-08-07T15:10:33.916 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 92.655439169000005, 2020-08-07T15:10:33.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.34605073400000003, 2020-08-07T15:10:33.916 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:33.916 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:33.916 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:33.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:33.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.762041+0000", 2020-08-07T15:10:33.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:33.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:33.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.762041+0000", 2020-08-07T15:10:33.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9067000000000002e-05 2020-08-07T15:10:33.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:33.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.762100+0000", 2020-08-07T15:10:33.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3529999999999997e-06 2020-08-07T15:10:33.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:33.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.762108+0000", 2020-08-07T15:10:33.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.34498205900000001 2020-08-07T15:10:33.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:33.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.107090+0000", 2020-08-07T15:10:33.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7420000000000003e-06 2020-08-07T15:10:33.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:33.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.107096+0000", 2020-08-07T15:10:33.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1074000000000004e-05 2020-08-07T15:10:33.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:33.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.107167+0000", 2020-08-07T15:10:33.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4423999999999999e-05 2020-08-07T15:10:33.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.107191+0000", 2020-08-07T15:10:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00084080199999999998 2020-08-07T15:10:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.108032+0000", 2020-08-07T15:10:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0213000000000001e-05 2020-08-07T15:10:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.108092+0000", 2020-08-07T15:10:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.34605073400000003 2020-08-07T15:10:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42520 4.8 e113/81 4:13071de5:::benchmark_data_smithi074_44586_object5314:head v 113'2624, mlcod=113'2618)", 2020-08-07T15:10:33.942 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:10:33.942 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 92.651959020000007, 2020-08-07T15:10:33.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001, 2020-08-07T15:10:33.942 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:33.942 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:33.943 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:33.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:33.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:10:33.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:33.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:33.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:10:33.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3967999999999995e-05 2020-08-07T15:10:33.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:33.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765585+0000", 2020-08-07T15:10:33.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0949999999999998e-06 2020-08-07T15:10:33.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:33.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765593+0000", 2020-08-07T15:10:33.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52703464200000005 2020-08-07T15:10:33.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:33.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292627+0000", 2020-08-07T15:10:33.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6929999999999996e-06 2020-08-07T15:10:33.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:33.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292636+0000", 2020-08-07T15:10:33.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9292000000000001e-05 2020-08-07T15:10:33.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:33.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292695+0000", 2020-08-07T15:10:33.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5395999999999999e-05 2020-08-07T15:10:33.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:33.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292721+0000", 2020-08-07T15:10:33.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00090361399999999996 2020-08-07T15:10:33.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:33.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293624+0000", 2020-08-07T15:10:33.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1761000000000002e-05 2020-08-07T15:10:33.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:33.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293666+0000", 2020-08-07T15:10:33.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001 2020-08-07T15:10:33.965 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.965 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:33.965 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:33.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.970 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:19776 5.4 5:2bb0c0d9:::benchmark_data_smithi074_53697_object2471:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e173)", 2020-08-07T15:10:33.971 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:10:33.971 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.0177663349999999, 2020-08-07T15:10:33.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.402061844, 2020-08-07T15:10:33.971 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:33.972 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:33.972 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:33.972 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:10:33.972 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:33.972 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 19776 2020-08-07T15:10:33.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.973 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:33.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:33.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:10:33.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:33.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:33.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:10:33.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-05 2020-08-07T15:10:33.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:33.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399752+0000", 2020-08-07T15:10:33.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8799999999999999e-06 2020-08-07T15:10:33.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:33.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399757+0000", 2020-08-07T15:10:33.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7690000000000008e-06 2020-08-07T15:10:33.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:33.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399767+0000", 2020-08-07T15:10:33.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-06 2020-08-07T15:10:33.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:33.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399771+0000", 2020-08-07T15:10:33.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003255109 2020-08-07T15:10:33.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:33.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403026+0000", 2020-08-07T15:10:33.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101044 2020-08-07T15:10:33.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:33.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403127+0000", 2020-08-07T15:10:33.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000193158 2020-08-07T15:10:33.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,5", 2020-08-07T15:10:33.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403320+0000", 2020-08-07T15:10:33.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017185530000000001 2020-08-07T15:10:33.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:33.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405038+0000", 2020-08-07T15:10:33.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6300000000000004e-07 2020-08-07T15:10:33.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:33.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405039+0000", 2020-08-07T15:10:33.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012590600000000001 2020-08-07T15:10:33.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:33.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405165+0000", 2020-08-07T15:10:33.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109757 2020-08-07T15:10:33.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:33.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405275+0000", 2020-08-07T15:10:33.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39642174699999999 2020-08-07T15:10:33.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:33.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:33.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801697+0000", 2020-08-07T15:10:33.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2799999999999996e-07 2020-08-07T15:10:33.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:33.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:34.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:34.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801697+0000", 2020-08-07T15:10:34.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8740999999999997e-05 2020-08-07T15:10:34.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:34.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:34.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:34.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801776+0000", 2020-08-07T15:10:34.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.402061844 2020-08-07T15:10:34.001 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:34.001 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:34.014 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:34.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:34.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:34.015 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:19776 5.4 e173/164 ondisk, result = 0)", 2020-08-07T15:10:34.015 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:10:34.016 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.0120180320000001, 2020-08-07T15:10:34.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39631765200000002, 2020-08-07T15:10:34.016 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:34.017 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:34.017 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:34.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:34.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:34.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:10:34.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:34.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:34.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:34.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:34.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:10:34.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9332e-05 2020-08-07T15:10:34.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:34.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:34.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:34.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405492+0000", 2020-08-07T15:10:34.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8900000000000004e-06 2020-08-07T15:10:34.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:34.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:34.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:34.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405498+0000", 2020-08-07T15:10:34.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39566664200000001 2020-08-07T15:10:34.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:34.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:34.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:34.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801164+0000", 2020-08-07T15:10:34.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7440000000000001e-06 2020-08-07T15:10:34.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:34.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:34.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:34.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801168+0000", 2020-08-07T15:10:34.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050828199999999998 2020-08-07T15:10:34.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:34.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:34.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:34.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801676+0000", 2020-08-07T15:10:34.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1011e-05 2020-08-07T15:10:34.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:34.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:34.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:34.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801687+0000", 2020-08-07T15:10:34.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2750999999999993e-05 2020-08-07T15:10:34.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:34.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:34.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:34.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801780+0000", 2020-08-07T15:10:34.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39631765200000002 2020-08-07T15:10:34.078 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:34.078 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:34.078 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:34.078 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:34.078 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:34.079 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:10:34.079 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:33.581+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:34.080 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:10:34.084 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:34.083+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:34.194 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:34.192+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:34.210 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 14 16 22655 22639 12.6323 11.5156 0.0168833 0.00988495 2020-08-07T15:10:34.264 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:34.265 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:10:34.265 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.265 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5099.0:22718 5.e e174/164 5:700b64a9:::benchmark_data_smithi074_53697_object2839:head v 174'1446, mlcod=174'1439)", 2020-08-07T15:10:34.265 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:34.249360+0000", 2020-08-07T15:10:34.265 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.006984072, 2020-08-07T15:10:34.266 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0070109289999999999, 2020-08-07T15:10:34.266 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:34.267 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:34.267 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:34.271 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:34.271 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:34.249360+0000", 2020-08-07T15:10:34.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:34.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:34.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:34.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:34.249360+0000", 2020-08-07T15:10:34.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7875e-05 2020-08-07T15:10:34.273 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:34.273 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:34.286 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:34.249388+0000", 2020-08-07T15:10:34.287 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2080000000000002e-06 2020-08-07T15:10:34.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:34.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.287 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:34.287 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:34.249392+0000", 2020-08-07T15:10:34.288 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8979000000000002e-05 2020-08-07T15:10:34.288 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:34.288 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.288 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:34.288 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:34.249421+0000", 2020-08-07T15:10:34.289 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.439e-06 2020-08-07T15:10:34.289 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:34.289 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.289 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:34.291 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:34.249424+0000", 2020-08-07T15:10:34.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019001739999999999 2020-08-07T15:10:34.299 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:34.300 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.300 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:34.305 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:34.251324+0000", 2020-08-07T15:10:34.306 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2708999999999999e-05 2020-08-07T15:10:34.306 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:34.306 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.306 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:34.306 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:34.251347+0000", 2020-08-07T15:10:34.307 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001987384 2020-08-07T15:10:34.307 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:34.307 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:34.307 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:34.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:34.308 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.308 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5099.0:22719 5.e e174/164 5:700b64a9:::benchmark_data_smithi074_53697_object2839:head v 174'1447, mlcod=174'1440)", 2020-08-07T15:10:34.309 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:34.251908+0000", 2020-08-07T15:10:34.309 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0044360190000000002, 2020-08-07T15:10:34.309 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045545239999999999, 2020-08-07T15:10:34.309 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:34.310 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:34.310 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:34.310 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.311 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:34.311 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:34.251908+0000", 2020-08-07T15:10:34.311 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:34.311 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:34.311 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.312 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:34.312 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:34.251908+0000", 2020-08-07T15:10:34.312 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1804000000000002e-05 2020-08-07T15:10:34.312 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:34.312 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.313 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:34.313 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:34.251950+0000", 2020-08-07T15:10:34.313 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6329999999999999e-06 2020-08-07T15:10:34.313 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:34.313 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.314 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:34.314 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:34.251956+0000", 2020-08-07T15:10:34.314 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7361999999999998e-05 2020-08-07T15:10:34.314 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:34.314 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.315 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:34.315 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:34.252004+0000", 2020-08-07T15:10:34.315 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3359999999999997e-06 2020-08-07T15:10:34.315 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:34.315 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.315 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:34.316 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:34.252008+0000", 2020-08-07T15:10:34.317 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001561118 2020-08-07T15:10:34.317 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:34.317 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.317 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:34.317 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:34.253569+0000", 2020-08-07T15:10:34.318 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.016e-05 2020-08-07T15:10:34.318 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:34.318 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.318 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:34.318 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:34.253589+0000", 2020-08-07T15:10:34.319 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0016814130000000001 2020-08-07T15:10:34.319 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:34.320 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:34.320 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:34.320 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:34.320 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.320 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5099.0:22720 5.e e174/164 5:700b64a9:::benchmark_data_smithi074_53697_object2839:head v 174'1448, mlcod=174'1440)", 2020-08-07T15:10:34.321 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:34.252365+0000", 2020-08-07T15:10:34.321 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0039791039999999998, 2020-08-07T15:10:34.321 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041742539999999996, 2020-08-07T15:10:34.321 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:34.321 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:34.322 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:34.322 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.322 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:34.323 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:34.252365+0000", 2020-08-07T15:10:34.323 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:34.323 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:34.323 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.323 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:34.339 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:34.252365+0000", 2020-08-07T15:10:34.342 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5615999999999997e-05 2020-08-07T15:10:34.342 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:34.342 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.342 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:34.343 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:34.252420+0000", 2020-08-07T15:10:34.343 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2330000000000003e-06 2020-08-07T15:10:34.343 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:34.343 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.343 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:34.344 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:34.252427+0000", 2020-08-07T15:10:34.344 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4524999999999997e-05 2020-08-07T15:10:34.344 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:34.344 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.344 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:34.345 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:34.252481+0000", 2020-08-07T15:10:34.345 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3450000000000002e-06 2020-08-07T15:10:34.345 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:34.345 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.345 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:34.371 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:34.252488+0000", 2020-08-07T15:10:34.371 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015006119999999999 2020-08-07T15:10:34.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:34.371 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:34.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:34.253988+0000", 2020-08-07T15:10:34.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0908e-05 2020-08-07T15:10:34.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:34.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:34.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:34.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:34.254009+0000", 2020-08-07T15:10:34.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001644239 2020-08-07T15:10:34.379 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:34.379 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:34.379 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:34.379 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:34.379 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:10:34.380 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 3 2020-08-07T15:10:34.380 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:34.384 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:10:34.384 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:10:34.385 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:10:34.385 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:10:34.385 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.1 to 1 (10000) 2020-08-07T15:10:34.387 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:10:34.387 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:34.380+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:34.395 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 2 1 2020-08-07T15:10:34.406 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:10:34.452 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:10:34.496 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:34.482+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:34.496 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:10:34.547 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:10:34.590 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:34.584+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:34.591 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:10:34.599 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:10:34.599 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:10:34.599 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:10:34.600 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:10:34.600 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:10:34.600 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:10:34.600 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:10:34.600 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:10:34.601 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:34.601 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:34.614 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:10:34.705 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:34.702+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:34.798 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:34.785+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:34.842 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:34.843 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:34.859 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:10:34.888 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:34.886+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:34.991 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:34.989+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:35.092 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:35.092 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:35.093 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:35.091+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:35.107 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:10:35.201 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:35.200+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:35.227 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 15 16 24235 24219 12.613 12.3438 0.00621518 0.0099048 2020-08-07T15:10:35.278 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:35.279 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:35.294 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:10:35.303 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:35.301+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:35.403 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.2 to 1 (10000) 2020-08-07T15:10:35.404 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:35.402+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:35.433 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 3 1 2020-08-07T15:10:35.478 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:35.479 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:35.495 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:10:35.537 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:35.536+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:35.622 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:35.617+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:35.725 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:35.721+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:35.726 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:35.726 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:35.736 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:10:35.826 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:35.822+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:35.930 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:35.925+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:35.961 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:35.961 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:35.972 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:10:36.031 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:36.030+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:36.137 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:36.133+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:36.170 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:36.171 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:36.184 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:10:36.210 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 16 16 25872 25856 12.6239 12.7891 0.0187663 0.00989151 2020-08-07T15:10:36.243 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:36.235+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:36.340 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:36.338+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:36.356 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:36.356 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:36.395 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:10:36.412 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:10:36.415 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.3 to 1 (10000) 2020-08-07T15:10:36.447 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 4 1 2020-08-07T15:10:36.487 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:36.461+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:36.564 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:36.562+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:36.606 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:36.607 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:36.615 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:10:36.619 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:10:36.709 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:36.703+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:36.777 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:36.771+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:36.878 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:36.874+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:36.886 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:36.887 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:36.899 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:10:36.938 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:10:36.977 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:36.974+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:36.987 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:10:37.013 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:10:37.052 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:10:37.082 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:37.082 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:37.083 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:37.081+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:37.085 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:10:37.090 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:10:37.131 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:10:37.159 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:10:37.210 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:37.191+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:37.211 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:10:37.211 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:10:37.211 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:10:37.211 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:10:37.212 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:10:37.212 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:10:37.221 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:10:37.223 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:10:37.224 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:10:37.225 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 17 16 27343 27327 12.5573 11.4922 0.00623384 0.00994147 2020-08-07T15:10:37.301 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:37.298+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:37.315 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:10:37.316 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:10:37.316 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:10:37.316 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:10:37.316 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:10:37.345 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:10:37.405 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:37.401+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:37.433 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.4 to 1 (10000) 2020-08-07T15:10:37.434 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:10:37.434 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 5 1 2020-08-07T15:10:37.510 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:37.503+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:37.590 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:10:37.591 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:10:37.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:10:37.591 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:10:37.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.592 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:108376 3.3 e62/20 ondisk, result = 0)", 2020-08-07T15:10:37.592 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:10:37.592 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 166.88779777900001, 2020-08-07T15:10:37.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62926872599999994, 2020-08-07T15:10:37.593 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:37.593 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:37.593 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:37.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:37.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:10:37.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:37.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:37.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:10:37.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8491999999999998e-05 2020-08-07T15:10:37.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:37.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683061+0000", 2020-08-07T15:10:37.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9099999999999998e-06 2020-08-07T15:10:37.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:37.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683065+0000", 2020-08-07T15:10:37.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62906670600000003 2020-08-07T15:10:37.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:37.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312131+0000", 2020-08-07T15:10:37.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2147000000000001e-05 2020-08-07T15:10:37.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:37.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312143+0000", 2020-08-07T15:10:37.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5152e-05 2020-08-07T15:10:37.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:37.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312239+0000", 2020-08-07T15:10:37.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5089999999999993e-06 2020-08-07T15:10:37.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:37.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312247+0000", 2020-08-07T15:10:37.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3810000000000001e-05 2020-08-07T15:10:37.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:37.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312311+0000", 2020-08-07T15:10:37.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62926872599999994 2020-08-07T15:10:37.668 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:37.668 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:37.668 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:37.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.671 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "MOSDPGPushReply(3.1 62/60 [PushReplyOp(3:8329f37e:::benchmark_data_smithi074_35447_object11915:head)])", 2020-08-07T15:10:37.671 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:10:37.671 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 166.71243976299999, 2020-08-07T15:10:37.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996, 2020-08-07T15:10:37.672 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:37.672 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:37.672 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:37.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:37.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:10:37.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:37.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:37.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:10:37.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5372000000000001e-05 2020-08-07T15:10:37.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:37.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858416+0000", 2020-08-07T15:10:37.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6899999999999998e-06 2020-08-07T15:10:37.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:37.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858419+0000", 2020-08-07T15:10:37.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63907904299999996 2020-08-07T15:10:37.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:37.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497498+0000", 2020-08-07T15:10:37.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3332999999999999e-05 2020-08-07T15:10:37.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:37.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497512+0000", 2020-08-07T15:10:37.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000450875 2020-08-07T15:10:37.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:37.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497962+0000", 2020-08-07T15:10:37.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000267779 2020-08-07T15:10:37.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:37.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.498230+0000", 2020-08-07T15:10:37.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996 2020-08-07T15:10:37.687 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:37.687 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:37.687 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:37.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.688 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:145792 3.1 3:881e0420:::benchmark_data_smithi074_35447_object16183:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:10:37.688 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:10:37.688 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 142.90403388199999, 2020-08-07T15:10:37.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997, 2020-08-07T15:10:37.689 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:37.689 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:37.689 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:37.689 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:10:37.690 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:37.691 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 145792 2020-08-07T15:10:37.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.691 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:37.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:37.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:10:37.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:37.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:37.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:10:37.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7489000000000001e-05 2020-08-07T15:10:37.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:37.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666824+0000", 2020-08-07T15:10:37.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0169999999999998e-06 2020-08-07T15:10:37.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:37.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666828+0000", 2020-08-07T15:10:37.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9829999999999996e-06 2020-08-07T15:10:37.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:37.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666836+0000", 2020-08-07T15:10:37.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.481e-06 2020-08-07T15:10:37.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:37.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666840+0000", 2020-08-07T15:10:37.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5299000000000001e-05 2020-08-07T15:10:37.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:37.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666885+0000", 2020-08-07T15:10:37.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014456500000000001 2020-08-07T15:10:37.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:37.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667030+0000", 2020-08-07T15:10:37.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017677799999999999 2020-08-07T15:10:37.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,5", 2020-08-07T15:10:37.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667207+0000", 2020-08-07T15:10:37.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004149649 2020-08-07T15:10:37.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:37.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671356+0000", 2020-08-07T15:10:37.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019339649999999999 2020-08-07T15:10:37.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:37.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673290+0000", 2020-08-07T15:10:37.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73346620500000004 2020-08-07T15:10:37.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:37.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406756+0000", 2020-08-07T15:10:37.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4700000000000001e-06 2020-08-07T15:10:37.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:37.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406759+0000", 2020-08-07T15:10:37.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6258999999999998e-05 2020-08-07T15:10:37.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:37.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406815+0000", 2020-08-07T15:10:37.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0126000000000002e-05 2020-08-07T15:10:37.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:37.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406855+0000", 2020-08-07T15:10:37.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997 2020-08-07T15:10:37.712 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:37.712 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:37.712 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:37.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.713 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:145792 3.1 e75/67 ondisk, result = 0)", 2020-08-07T15:10:37.713 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:10:37.713 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 142.89964899399999, 2020-08-07T15:10:37.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003, 2020-08-07T15:10:37.714 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:37.714 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:37.714 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:37.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:37.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:10:37.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:37.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:37.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:10:37.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2830999999999998e-05 2020-08-07T15:10:37.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:37.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671224+0000", 2020-08-07T15:10:37.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:10:37.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:37.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671229+0000", 2020-08-07T15:10:37.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.735414864 2020-08-07T15:10:37.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:37.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406644+0000", 2020-08-07T15:10:37.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1250000000000004e-06 2020-08-07T15:10:37.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:37.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406651+0000", 2020-08-07T15:10:37.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6567e-05 2020-08-07T15:10:37.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:37.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406737+0000", 2020-08-07T15:10:37.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0054000000000001e-05 2020-08-07T15:10:37.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:37.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406747+0000", 2020-08-07T15:10:37.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011391999999999999 2020-08-07T15:10:37.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:37.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406861+0000", 2020-08-07T15:10:37.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003 2020-08-07T15:10:37.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:37.734 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:37.735 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:37.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.735 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30921 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:10:37.735 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:10:37.736 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.86490062, 2020-08-07T15:10:37.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001, 2020-08-07T15:10:37.736 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:37.736 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:37.736 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:37.741 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:37.741 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:37.742 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30921 2020-08-07T15:10:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:10:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:10:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6954999999999998e-05 2020-08-07T15:10:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706016+0000", 2020-08-07T15:10:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0229999999999992e-06 2020-08-07T15:10:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706024+0000", 2020-08-07T15:10:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.432e-05 2020-08-07T15:10:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706039+0000", 2020-08-07T15:10:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8490000000000006e-06 2020-08-07T15:10:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706046+0000", 2020-08-07T15:10:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6837999999999999e-05 2020-08-07T15:10:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706113+0000", 2020-08-07T15:10:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3553000000000001e-05 2020-08-07T15:10:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706127+0000", 2020-08-07T15:10:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77277201699999998 2020-08-07T15:10:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.478899+0000", 2020-08-07T15:10:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093978599999999996 2020-08-07T15:10:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:10:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479839+0000", 2020-08-07T15:10:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010376029 2020-08-07T15:10:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490215+0000", 2020-08-07T15:10:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000190657 2020-08-07T15:10:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490405+0000", 2020-08-07T15:10:37.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001 2020-08-07T15:10:37.774 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:37.774 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:37.775 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30922 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:10:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:10:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.864740457, 2020-08-07T15:10:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247, 2020-08-07T15:10:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30922 2020-08-07T15:10:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:10:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:10:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9178999999999997e-05 2020-08-07T15:10:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706159+0000", 2020-08-07T15:10:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4999999999999999e-06 2020-08-07T15:10:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706164+0000", 2020-08-07T15:10:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1792e-05 2020-08-07T15:10:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706176+0000", 2020-08-07T15:10:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.139e-06 2020-08-07T15:10:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706182+0000", 2020-08-07T15:10:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039724900000000002 2020-08-07T15:10:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706579+0000", 2020-08-07T15:10:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1191e-05 2020-08-07T15:10:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706591+0000", 2020-08-07T15:10:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329101300000003 2020-08-07T15:10:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479882+0000", 2020-08-07T15:10:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9565000000000001e-05 2020-08-07T15:10:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:10:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479971+0000", 2020-08-07T15:10:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010471263 2020-08-07T15:10:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490442+0000", 2020-08-07T15:10:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015035600000000001 2020-08-07T15:10:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490593+0000", 2020-08-07T15:10:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247 2020-08-07T15:10:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30923 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:10:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:10:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.86426539599999, 2020-08-07T15:10:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998, 2020-08-07T15:10:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30923 2020-08-07T15:10:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:10:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:37.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:10:37.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5531000000000001e-05 2020-08-07T15:10:37.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:37.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706630+0000", 2020-08-07T15:10:37.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7689999999999999e-06 2020-08-07T15:10:37.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:37.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706637+0000", 2020-08-07T15:10:37.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3577999999999999e-05 2020-08-07T15:10:37.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:37.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706650+0000", 2020-08-07T15:10:37.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1439999999999997e-06 2020-08-07T15:10:37.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:37.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706658+0000", 2020-08-07T15:10:37.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1663999999999999e-05 2020-08-07T15:10:37.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:37.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706709+0000", 2020-08-07T15:10:37.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0149000000000001e-05 2020-08-07T15:10:37.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:37.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706719+0000", 2020-08-07T15:10:37.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329694900000001 2020-08-07T15:10:37.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:37.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480016+0000", 2020-08-07T15:10:37.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7998999999999997e-05 2020-08-07T15:10:37.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:10:37.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480084+0000", 2020-08-07T15:10:37.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010547333000000001 2020-08-07T15:10:37.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:37.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490632+0000", 2020-08-07T15:10:37.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7439e-05 2020-08-07T15:10:37.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:37.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490729+0000", 2020-08-07T15:10:37.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998 2020-08-07T15:10:37.842 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:37.843 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:37.844 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:37.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.844 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30924 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:10:37.844 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:10:37.845 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.86413812399999, 2020-08-07T15:10:37.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995, 2020-08-07T15:10:37.847 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:37.847 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:37.847 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:37.848 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:37.848 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:37.848 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30924 2020-08-07T15:10:37.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.848 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:37.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:37.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:10:37.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:37.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:37.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:10:37.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9698999999999999e-05 2020-08-07T15:10:37.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:37.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706762+0000", 2020-08-07T15:10:37.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8080000000000001e-06 2020-08-07T15:10:37.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:37.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706767+0000", 2020-08-07T15:10:37.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1069e-05 2020-08-07T15:10:37.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:37.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706778+0000", 2020-08-07T15:10:37.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6969999999999998e-06 2020-08-07T15:10:37.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:37.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706784+0000", 2020-08-07T15:10:37.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8792999999999998e-05 2020-08-07T15:10:37.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:37.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706843+0000", 2020-08-07T15:10:37.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5449999999999998e-06 2020-08-07T15:10:37.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:37.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706850+0000", 2020-08-07T15:10:37.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.773278354 2020-08-07T15:10:37.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:37.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480129+0000", 2020-08-07T15:10:37.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1577999999999997e-05 2020-08-07T15:10:37.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:10:37.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480210+0000", 2020-08-07T15:10:37.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010541915000000001 2020-08-07T15:10:37.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:37.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490752+0000", 2020-08-07T15:10:37.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103937 2020-08-07T15:10:37.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:37.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490856+0000", 2020-08-07T15:10:37.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995 2020-08-07T15:10:37.864 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:37.865 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:37.865 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:37.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.926 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30925 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:10:37.926 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:10:37.926 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.864014353, 2020-08-07T15:10:37.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995, 2020-08-07T15:10:37.929 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:37.929 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:37.929 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:37.929 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:37.930 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:37.930 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30925 2020-08-07T15:10:37.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.930 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:37.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:37.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:10:37.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:37.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:37.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:10:37.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4664999999999999e-05 2020-08-07T15:10:37.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:37.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706880+0000", 2020-08-07T15:10:37.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7479999999999999e-06 2020-08-07T15:10:37.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:37.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706886+0000", 2020-08-07T15:10:37.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1838000000000001e-05 2020-08-07T15:10:37.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:37.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706898+0000", 2020-08-07T15:10:37.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0089999999999998e-06 2020-08-07T15:10:37.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:37.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706903+0000", 2020-08-07T15:10:37.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3559e-05 2020-08-07T15:10:37.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:37.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706956+0000", 2020-08-07T15:10:37.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0309000000000001e-05 2020-08-07T15:10:37.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:37.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706967+0000", 2020-08-07T15:10:37.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77328290499999996 2020-08-07T15:10:37.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:37.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480250+0000", 2020-08-07T15:10:37.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8680999999999997e-05 2020-08-07T15:10:37.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:10:37.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480298+0000", 2020-08-07T15:10:37.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01059272 2020-08-07T15:10:37.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:37.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490891+0000", 2020-08-07T15:10:37.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014580300000000001 2020-08-07T15:10:37.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:37.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491037+0000", 2020-08-07T15:10:37.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995 2020-08-07T15:10:37.958 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:37.958 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:37.958 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:37.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.959 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30926 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:10:37.959 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:10:37.959 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.86389748400001, 2020-08-07T15:10:37.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005, 2020-08-07T15:10:37.960 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:37.960 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:37.960 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:37.960 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:37.961 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:37.961 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30926 2020-08-07T15:10:37.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.969 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:37.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:37.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:10:37.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:37.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:37.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:10:37.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5223000000000003e-05 2020-08-07T15:10:37.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:37.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706998+0000", 2020-08-07T15:10:37.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4809999999999997e-06 2020-08-07T15:10:37.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:37.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707003+0000", 2020-08-07T15:10:37.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1663e-05 2020-08-07T15:10:37.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:37.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707015+0000", 2020-08-07T15:10:37.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9729999999999997e-06 2020-08-07T15:10:37.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:37.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707020+0000", 2020-08-07T15:10:37.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2565999999999999e-05 2020-08-07T15:10:37.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:37.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707062+0000", 2020-08-07T15:10:37.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8944e-05 2020-08-07T15:10:37.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:37.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707081+0000", 2020-08-07T15:10:37.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77325885000000005 2020-08-07T15:10:37.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:37.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480340+0000", 2020-08-07T15:10:37.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0321000000000006e-05 2020-08-07T15:10:37.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:37.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:10:37.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480420+0000", 2020-08-07T15:10:37.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010655958 2020-08-07T15:10:37.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:37.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491076+0000", 2020-08-07T15:10:38.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9784999999999995e-05 2020-08-07T15:10:38.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:38.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491166+0000", 2020-08-07T15:10:38.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005 2020-08-07T15:10:38.010 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.011 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:38.011 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.015 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30927 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:10:38.015 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:10:38.016 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.863779665, 2020-08-07T15:10:38.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999, 2020-08-07T15:10:38.016 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:38.016 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:38.016 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:38.017 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:38.017 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:38.017 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30927 2020-08-07T15:10:38.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.017 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:38.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:38.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:10:38.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:38.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:38.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:10:38.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9861999999999997e-05 2020-08-07T15:10:38.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:38.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707120+0000", 2020-08-07T15:10:38.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8969999999999996e-06 2020-08-07T15:10:38.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:38.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707126+0000", 2020-08-07T15:10:38.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1248e-05 2020-08-07T15:10:38.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:38.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707137+0000", 2020-08-07T15:10:38.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.955e-06 2020-08-07T15:10:38.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:38.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707142+0000", 2020-08-07T15:10:38.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0643999999999998e-05 2020-08-07T15:10:38.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707193+0000", 2020-08-07T15:10:38.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1983e-05 2020-08-07T15:10:38.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:38.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707205+0000", 2020-08-07T15:10:38.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77324914 2020-08-07T15:10:38.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480454+0000", 2020-08-07T15:10:38.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4119999999999998e-05 2020-08-07T15:10:38.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:10:38.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480498+0000", 2020-08-07T15:10:38.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010709707000000001 2020-08-07T15:10:38.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491208+0000", 2020-08-07T15:10:38.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014433800000000001 2020-08-07T15:10:38.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:38.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491352+0000", 2020-08-07T15:10:38.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999 2020-08-07T15:10:38.045 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.064 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:38.064 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.065 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30928 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:10:38.065 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:10:38.065 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.86365530800001, 2020-08-07T15:10:38.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998, 2020-08-07T15:10:38.066 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:38.066 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:38.066 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:38.066 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:38.066 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:38.067 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30928 2020-08-07T15:10:38.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.068 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:38.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:38.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:10:38.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:38.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:38.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:10:38.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6582000000000001e-05 2020-08-07T15:10:38.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:38.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707241+0000", 2020-08-07T15:10:38.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9190000000000004e-06 2020-08-07T15:10:38.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:38.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707247+0000", 2020-08-07T15:10:38.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000188822 2020-08-07T15:10:38.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:38.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707436+0000", 2020-08-07T15:10:38.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2940000000000002e-06 2020-08-07T15:10:38.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:38.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707442+0000", 2020-08-07T15:10:38.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7800000000000003e-05 2020-08-07T15:10:38.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707490+0000", 2020-08-07T15:10:38.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2020-08-07T15:10:38.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:38.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707501+0000", 2020-08-07T15:10:38.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77303498800000003 2020-08-07T15:10:38.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480536+0000", 2020-08-07T15:10:38.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7723000000000001e-05 2020-08-07T15:10:38.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:10:38.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480614+0000", 2020-08-07T15:10:38.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010812183 2020-08-07T15:10:38.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491426+0000", 2020-08-07T15:10:38.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5502999999999999e-05 2020-08-07T15:10:38.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:38.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491502+0000", 2020-08-07T15:10:38.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998 2020-08-07T15:10:38.096 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.096 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:38.097 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.097 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31009 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:10:38.097 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:10:38.098 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.839699002, 2020-08-07T15:10:38.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002, 2020-08-07T15:10:38.098 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:38.099 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:38.099 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:38.100 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:38.100 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:38.102 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31009 2020-08-07T15:10:38.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.102 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:38.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:38.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:10:38.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:38.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:38.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:10:38.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2429000000000002e-05 2020-08-07T15:10:38.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:38.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731193+0000", 2020-08-07T15:10:38.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9760000000000003e-06 2020-08-07T15:10:38.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:38.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731198+0000", 2020-08-07T15:10:38.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3597e-05 2020-08-07T15:10:38.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:38.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731212+0000", 2020-08-07T15:10:38.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1709999999999996e-06 2020-08-07T15:10:38.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:38.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731219+0000", 2020-08-07T15:10:38.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5402999999999997e-05 2020-08-07T15:10:38.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731285+0000", 2020-08-07T15:10:38.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3495e-05 2020-08-07T15:10:38.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:38.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731298+0000", 2020-08-07T15:10:38.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74935423599999995 2020-08-07T15:10:38.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480652+0000", 2020-08-07T15:10:38.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000137806 2020-08-07T15:10:38.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:38.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480790+0000", 2020-08-07T15:10:38.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022272399999999999 2020-08-07T15:10:38.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:10:38.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481013+0000", 2020-08-07T15:10:38.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003223381 2020-08-07T15:10:38.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:38.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484236+0000", 2020-08-07T15:10:38.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1370000000000001e-06 2020-08-07T15:10:38.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:38.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484237+0000", 2020-08-07T15:10:38.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6288e-05 2020-08-07T15:10:38.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:38.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484294+0000", 2020-08-07T15:10:38.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.007260402 2020-08-07T15:10:38.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:38.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491554+0000", 2020-08-07T15:10:38.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7358000000000003e-05 2020-08-07T15:10:38.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:38.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491601+0000", 2020-08-07T15:10:38.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1100000000000004e-07 2020-08-07T15:10:38.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:38.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491602+0000", 2020-08-07T15:10:38.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013436500000000001 2020-08-07T15:10:38.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:38.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491737+0000", 2020-08-07T15:10:38.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002 2020-08-07T15:10:38.128 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.128 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:38.129 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.130 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31010 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:10:38.130 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:10:38.131 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.83957304800001, 2020-08-07T15:10:38.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001, 2020-08-07T15:10:38.131 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:38.131 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:38.132 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:38.132 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:38.132 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:38.133 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31010 2020-08-07T15:10:38.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.133 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:38.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:38.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:10:38.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:38.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:38.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:10:38.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2398000000000002e-05 2020-08-07T15:10:38.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:38.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731319+0000", 2020-08-07T15:10:38.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7100000000000004e-06 2020-08-07T15:10:38.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:38.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731325+0000", 2020-08-07T15:10:38.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4328000000000001e-05 2020-08-07T15:10:38.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:38.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731339+0000", 2020-08-07T15:10:38.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3310000000000003e-06 2020-08-07T15:10:38.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:38.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731345+0000", 2020-08-07T15:10:38.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8952999999999998e-05 2020-08-07T15:10:38.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731384+0000", 2020-08-07T15:10:38.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0036e-05 2020-08-07T15:10:38.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:38.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731394+0000", 2020-08-07T15:10:38.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75027681300000004 2020-08-07T15:10:38.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481670+0000", 2020-08-07T15:10:38.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016555899999999999 2020-08-07T15:10:38.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:38.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481836+0000", 2020-08-07T15:10:38.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028760999999999999 2020-08-07T15:10:38.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:10:38.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482124+0000", 2020-08-07T15:10:38.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038342680000000001 2020-08-07T15:10:38.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:38.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485958+0000", 2020-08-07T15:10:38.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2559999999999999e-06 2020-08-07T15:10:38.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:38.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485959+0000", 2020-08-07T15:10:38.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3758999999999995e-05 2020-08-07T15:10:38.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:38.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486043+0000", 2020-08-07T15:10:38.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057590419999999998 2020-08-07T15:10:38.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:38.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491802+0000", 2020-08-07T15:10:38.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5757000000000001e-05 2020-08-07T15:10:38.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:38.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:10:38.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9299999999999997e-07 2020-08-07T15:10:38.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:38.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:10:38.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012496999999999999 2020-08-07T15:10:38.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:38.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491993+0000", 2020-08-07T15:10:38.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001 2020-08-07T15:10:38.154 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.154 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:38.154 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.155 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31011 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:10:38.155 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:10:38.155 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.83945684299999, 2020-08-07T15:10:38.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997, 2020-08-07T15:10:38.156 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:38.156 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:38.167 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:38.169 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:38.170 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:38.170 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31011 2020-08-07T15:10:38.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.171 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:38.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:38.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:10:38.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:38.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:38.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:10:38.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9539999999999997e-05 2020-08-07T15:10:38.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:38.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731433+0000", 2020-08-07T15:10:38.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9209999999999997e-06 2020-08-07T15:10:38.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:38.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731439+0000", 2020-08-07T15:10:38.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4246999999999999e-05 2020-08-07T15:10:38.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:38.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731453+0000", 2020-08-07T15:10:38.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8130000000000003e-06 2020-08-07T15:10:38.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:38.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731458+0000", 2020-08-07T15:10:38.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6668000000000001e-05 2020-08-07T15:10:38.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731504+0000", 2020-08-07T15:10:38.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1112e-05 2020-08-07T15:10:38.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:38.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731515+0000", 2020-08-07T15:10:38.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75121332100000004 2020-08-07T15:10:38.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482729+0000", 2020-08-07T15:10:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1810000000000004e-05 2020-08-07T15:10:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482811+0000", 2020-08-07T15:10:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018001399999999999 2020-08-07T15:10:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:10:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482991+0000", 2020-08-07T15:10:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030701140000000001 2020-08-07T15:10:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486061+0000", 2020-08-07T15:10:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4499999999999996e-07 2020-08-07T15:10:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486062+0000", 2020-08-07T15:10:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5366999999999998e-05 2020-08-07T15:10:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486147+0000", 2020-08-07T15:10:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056872779999999996 2020-08-07T15:10:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491834+0000", 2020-08-07T15:10:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024645100000000002 2020-08-07T15:10:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:10:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1200000000000003e-07 2020-08-07T15:10:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:10:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1451000000000002e-05 2020-08-07T15:10:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492173+0000", 2020-08-07T15:10:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997 2020-08-07T15:10:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31012 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:10:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:10:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.838802724, 2020-08-07T15:10:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003, 2020-08-07T15:10:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31012 2020-08-07T15:10:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:10:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:10:38.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6344999999999997e-05 2020-08-07T15:10:38.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:38.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732094+0000", 2020-08-07T15:10:38.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2920000000000001e-06 2020-08-07T15:10:38.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:38.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732100+0000", 2020-08-07T15:10:38.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.189e-05 2020-08-07T15:10:38.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:38.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732112+0000", 2020-08-07T15:10:38.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3720000000000001e-06 2020-08-07T15:10:38.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:38.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732118+0000", 2020-08-07T15:10:38.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2514000000000004e-05 2020-08-07T15:10:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732191+0000", 2020-08-07T15:10:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4610000000000001e-05 2020-08-07T15:10:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732205+0000", 2020-08-07T15:10:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75132974100000005 2020-08-07T15:10:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483535+0000", 2020-08-07T15:10:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9185999999999996e-05 2020-08-07T15:10:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483624+0000", 2020-08-07T15:10:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015683 2020-08-07T15:10:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:10:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483781+0000", 2020-08-07T15:10:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046202889999999996 2020-08-07T15:10:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488401+0000", 2020-08-07T15:10:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6700000000000002e-07 2020-08-07T15:10:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488402+0000", 2020-08-07T15:10:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1487000000000001e-05 2020-08-07T15:10:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488484+0000", 2020-08-07T15:10:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035520830000000002 2020-08-07T15:10:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:38.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492036+0000", 2020-08-07T15:10:38.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029266600000000002 2020-08-07T15:10:38.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492328+0000", 2020-08-07T15:10:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5099999999999998e-07 2020-08-07T15:10:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:38.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492329+0000", 2020-08-07T15:10:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000201299 2020-08-07T15:10:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492531+0000", 2020-08-07T15:10:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003 2020-08-07T15:10:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31013 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:10:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:10:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.83859109300001, 2020-08-07T15:10:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996, 2020-08-07T15:10:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:38.224 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:38.224 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:38.224 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31013 2020-08-07T15:10:38.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.224 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:10:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:10:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8949000000000002e-05 2020-08-07T15:10:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732298+0000", 2020-08-07T15:10:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1399999999999997e-06 2020-08-07T15:10:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732304+0000", 2020-08-07T15:10:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.154e-05 2020-08-07T15:10:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:38.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732315+0000", 2020-08-07T15:10:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1319999999999999e-06 2020-08-07T15:10:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:38.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732322+0000", 2020-08-07T15:10:38.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7431e-05 2020-08-07T15:10:38.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732369+0000", 2020-08-07T15:10:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0621999999999999e-05 2020-08-07T15:10:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:38.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732380+0000", 2020-08-07T15:10:38.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75196845999999995 2020-08-07T15:10:38.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484348+0000", 2020-08-07T15:10:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000123008 2020-08-07T15:10:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484471+0000", 2020-08-07T15:10:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017572700000000001 2020-08-07T15:10:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:10:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484647+0000", 2020-08-07T15:10:38.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038557650000000001 2020-08-07T15:10:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:38.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:10:38.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2020-08-07T15:10:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:10:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0714e-05 2020-08-07T15:10:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488584+0000", 2020-08-07T15:10:38.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036298319999999999 2020-08-07T15:10:38.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:38.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492214+0000", 2020-08-07T15:10:38.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039530099999999997 2020-08-07T15:10:38.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:38.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492609+0000", 2020-08-07T15:10:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5300000000000002e-07 2020-08-07T15:10:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492610+0000", 2020-08-07T15:10:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197321 2020-08-07T15:10:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492807+0000", 2020-08-07T15:10:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996 2020-08-07T15:10:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31014 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:10:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:10:38.266 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.837208949, 2020-08-07T15:10:38.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998, 2020-08-07T15:10:38.266 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:38.266 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:38.266 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31014 2020-08-07T15:10:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:10:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:10:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3743999999999999e-05 2020-08-07T15:10:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733685+0000", 2020-08-07T15:10:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1720000000000007e-06 2020-08-07T15:10:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733693+0000", 2020-08-07T15:10:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3674e-05 2020-08-07T15:10:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733707+0000", 2020-08-07T15:10:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9249999999999998e-06 2020-08-07T15:10:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733714+0000", 2020-08-07T15:10:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8338000000000003e-05 2020-08-07T15:10:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733772+0000", 2020-08-07T15:10:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3019e-05 2020-08-07T15:10:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733785+0000", 2020-08-07T15:10:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75134856299999997 2020-08-07T15:10:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485133+0000", 2020-08-07T15:10:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8739999999999995e-05 2020-08-07T15:10:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485212+0000", 2020-08-07T15:10:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151244 2020-08-07T15:10:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:10:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485363+0000", 2020-08-07T15:10:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032929090000000001 2020-08-07T15:10:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488656+0000", 2020-08-07T15:10:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.065e-06 2020-08-07T15:10:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488657+0000", 2020-08-07T15:10:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1123000000000003e-05 2020-08-07T15:10:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488729+0000", 2020-08-07T15:10:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035508330000000002 2020-08-07T15:10:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492279+0000", 2020-08-07T15:10:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00059376000000000003 2020-08-07T15:10:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492873+0000", 2020-08-07T15:10:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5700000000000002e-07 2020-08-07T15:10:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492874+0000", 2020-08-07T15:10:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019264 2020-08-07T15:10:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493066+0000", 2020-08-07T15:10:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998 2020-08-07T15:10:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31015 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:10:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:10:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.837082983, 2020-08-07T15:10:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002, 2020-08-07T15:10:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:38.346 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31015 2020-08-07T15:10:38.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.346 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:10:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:38.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:10:38.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3707000000000001e-05 2020-08-07T15:10:38.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:38.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733791+0000", 2020-08-07T15:10:38.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8120000000000001e-06 2020-08-07T15:10:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733794+0000", 2020-08-07T15:10:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0728e-05 2020-08-07T15:10:38.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:38.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733805+0000", 2020-08-07T15:10:38.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8279999999999999e-06 2020-08-07T15:10:38.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:38.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733811+0000", 2020-08-07T15:10:38.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2955999999999999e-05 2020-08-07T15:10:38.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733864+0000", 2020-08-07T15:10:38.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1389000000000001e-05 2020-08-07T15:10:38.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:38.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733875+0000", 2020-08-07T15:10:38.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75232711100000005 2020-08-07T15:10:38.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486202+0000", 2020-08-07T15:10:38.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027503400000000001 2020-08-07T15:10:38.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:38.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486478+0000", 2020-08-07T15:10:38.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033010400000000001 2020-08-07T15:10:38.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:10:38.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486808+0000", 2020-08-07T15:10:38.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019382609999999999 2020-08-07T15:10:38.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:38.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488746+0000", 2020-08-07T15:10:38.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9400000000000004e-07 2020-08-07T15:10:38.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:38.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488747+0000", 2020-08-07T15:10:38.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4324000000000002e-05 2020-08-07T15:10:38.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:38.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488821+0000", 2020-08-07T15:10:38.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043745640000000001 2020-08-07T15:10:38.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:38.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493196+0000", 2020-08-07T15:10:38.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030381000000000001 2020-08-07T15:10:38.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:38.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493499+0000", 2020-08-07T15:10:38.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4e-07 2020-08-07T15:10:38.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:38.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493500+0000", 2020-08-07T15:10:38.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017499 2020-08-07T15:10:38.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:38.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493675+0000", 2020-08-07T15:10:38.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002 2020-08-07T15:10:38.467 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.468 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:38.468 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.468 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31016 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:10:38.469 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:10:38.469 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.83648700800001, 2020-08-07T15:10:38.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971, 2020-08-07T15:10:38.469 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:38.470 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:38.470 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:38.470 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:38.482 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:38.483 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31016 2020-08-07T15:10:38.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.509 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:38.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:38.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:10:38.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:38.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:38.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:10:38.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0924000000000002e-05 2020-08-07T15:10:38.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:38.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734404+0000", 2020-08-07T15:10:38.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2020-08-07T15:10:38.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:38.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734410+0000", 2020-08-07T15:10:38.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3459999999999994e-06 2020-08-07T15:10:38.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:38.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734418+0000", 2020-08-07T15:10:38.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9640000000000002e-06 2020-08-07T15:10:38.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:38.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734423+0000", 2020-08-07T15:10:38.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7253999999999999e-05 2020-08-07T15:10:38.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734470+0000", 2020-08-07T15:10:38.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0249999999999997e-06 2020-08-07T15:10:38.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:38.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734477+0000", 2020-08-07T15:10:38.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75294592900000001 2020-08-07T15:10:38.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487423+0000", 2020-08-07T15:10:38.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031604300000000001 2020-08-07T15:10:38.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:38.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487739+0000", 2020-08-07T15:10:38.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000171149 2020-08-07T15:10:38.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:10:38.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487910+0000", 2020-08-07T15:10:38.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024207769999999998 2020-08-07T15:10:38.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:38.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490331+0000", 2020-08-07T15:10:38.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1650000000000001e-06 2020-08-07T15:10:38.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:38.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490332+0000", 2020-08-07T15:10:38.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9796999999999996e-05 2020-08-07T15:10:38.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:38.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490402+0000", 2020-08-07T15:10:38.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029928849999999998 2020-08-07T15:10:38.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:38.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493395+0000", 2020-08-07T15:10:38.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034903199999999999 2020-08-07T15:10:38.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:38.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493744+0000", 2020-08-07T15:10:38.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8899999999999999e-07 2020-08-07T15:10:38.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:38.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493745+0000", 2020-08-07T15:10:38.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197362 2020-08-07T15:10:38.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:38.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493942+0000", 2020-08-07T15:10:38.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971 2020-08-07T15:10:38.594 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.594 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:38.595 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.595 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.595 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:38.595 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:10:38.596 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:10:38.598 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 18 16 28995 28979 12.5766 12.9062 0.00578592 0.00992843 2020-08-07T15:10:38.599 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.5 to 1 (10000) 2020-08-07T15:10:38.599 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 6 1 2020-08-07T15:10:38.600 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:38.596+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:38.707 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:38.705+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:38.779 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:10:38.779 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:10:38.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.780 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:29919 5.e 5:7d698a78:::benchmark_data_smithi074_53697_object3739:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-08-07T15:10:38.780 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:38.770498+0000", 2020-08-07T15:10:38.780 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0072382790000000002, 2020-08-07T15:10:38.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0072701830000000004, 2020-08-07T15:10:38.781 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:38.781 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:38.781 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:38.781 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:10:38.781 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:38.782 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 29919 2020-08-07T15:10:38.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.782 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:38.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:38.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.770498+0000", 2020-08-07T15:10:38.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:38.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:38.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.770498+0000", 2020-08-07T15:10:38.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4407999999999998e-05 2020-08-07T15:10:38.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:38.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.770552+0000", 2020-08-07T15:10:38.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4810000000000003e-06 2020-08-07T15:10:38.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.770559+0000", 2020-08-07T15:10:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1460999999999999e-05 2020-08-07T15:10:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.770570+0000", 2020-08-07T15:10:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2020-08-07T15:10:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.770577+0000", 2020-08-07T15:10:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9842999999999998e-05 2020-08-07T15:10:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.770617+0000", 2020-08-07T15:10:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3387999999999994e-05 2020-08-07T15:10:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.770710+0000", 2020-08-07T15:10:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013978300000000001 2020-08-07T15:10:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:10:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.770850+0000", 2020-08-07T15:10:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022614699999999998 2020-08-07T15:10:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.773111+0000", 2020-08-07T15:10:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3300000000000001e-07 2020-08-07T15:10:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.773112+0000", 2020-08-07T15:10:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4349000000000003e-05 2020-08-07T15:10:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.773186+0000", 2020-08-07T15:10:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038972249999999998 2020-08-07T15:10:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.777084+0000", 2020-08-07T15:10:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0065857290000000002 2020-08-07T15:10:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:29920 5.e 5:7d698a78:::benchmark_data_smithi074_53697_object3739:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e179)", 2020-08-07T15:10:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:38.770627+0000", 2020-08-07T15:10:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0071092890000000004, 2020-08-07T15:10:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.007283599, 2020-08-07T15:10:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:10:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 29920 2020-08-07T15:10:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.770627+0000", 2020-08-07T15:10:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.770627+0000", 2020-08-07T15:10:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0379e-05 2020-08-07T15:10:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:38.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.770677+0000", 2020-08-07T15:10:38.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4280000000000004e-06 2020-08-07T15:10:38.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:38.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.770683+0000", 2020-08-07T15:10:38.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1221e-05 2020-08-07T15:10:38.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:38.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.770694+0000", 2020-08-07T15:10:38.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.203e-06 2020-08-07T15:10:38.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.770699+0000", 2020-08-07T15:10:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00053696500000000003 2020-08-07T15:10:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.771236+0000", 2020-08-07T15:10:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0611000000000004e-05 2020-08-07T15:10:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.771327+0000", 2020-08-07T15:10:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017803299999999999 2020-08-07T15:10:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:10:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.771505+0000", 2020-08-07T15:10:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00169141 2020-08-07T15:10:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.773196+0000", 2020-08-07T15:10:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.13e-07 2020-08-07T15:10:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.773197+0000", 2020-08-07T15:10:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1496000000000003e-05 2020-08-07T15:10:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.773248+0000", 2020-08-07T15:10:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026212589999999999 2020-08-07T15:10:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:29925 5.2 e179/164 5:49bea0b5:::benchmark_data_smithi074_53697_object3740:head v 179'1917, mlcod=179'1911)", 2020-08-07T15:10:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:38.775183+0000", 2020-08-07T15:10:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0025531680000000002, 2020-08-07T15:10:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028360239999999999, 2020-08-07T15:10:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.775183+0000", 2020-08-07T15:10:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.775183+0000", 2020-08-07T15:10:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2212999999999995e-05 2020-08-07T15:10:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.775245+0000", 2020-08-07T15:10:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2020000000000001e-06 2020-08-07T15:10:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.775251+0000", 2020-08-07T15:10:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00089792099999999998 2020-08-07T15:10:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.776149+0000", 2020-08-07T15:10:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2679999999999996e-06 2020-08-07T15:10:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.776154+0000", 2020-08-07T15:10:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034309299999999999 2020-08-07T15:10:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.776497+0000", 2020-08-07T15:10:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3221999999999999e-05 2020-08-07T15:10:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.776520+0000", 2020-08-07T15:10:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001336919 2020-08-07T15:10:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.887 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:29926 5.2 e179/164 5:49bea0b5:::benchmark_data_smithi074_53697_object3740:head v 179'1918, mlcod=179'1911)", 2020-08-07T15:10:38.887 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:38.775281+0000", 2020-08-07T15:10:38.887 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0024550430000000001, 2020-08-07T15:10:38.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028270270000000002, 2020-08-07T15:10:38.888 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:38.888 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:38.893 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:38.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:38.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.775281+0000", 2020-08-07T15:10:38.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:38.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:38.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.775281+0000", 2020-08-07T15:10:38.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7630999999999998e-05 2020-08-07T15:10:38.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:38.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.775329+0000", 2020-08-07T15:10:38.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2080000000000002e-06 2020-08-07T15:10:38.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:38.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.775333+0000", 2020-08-07T15:10:38.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00082880900000000001 2020-08-07T15:10:38.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:38.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.776162+0000", 2020-08-07T15:10:38.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7310000000000002e-06 2020-08-07T15:10:38.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:38.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.776164+0000", 2020-08-07T15:10:38.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001189559 2020-08-07T15:10:38.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.777354+0000", 2020-08-07T15:10:38.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.236e-05 2020-08-07T15:10:38.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:38.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.777366+0000", 2020-08-07T15:10:38.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020852980000000002 2020-08-07T15:10:38.905 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.919 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:38.919 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.920 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:29927 5.2 e179/164 5:49bea0b5:::benchmark_data_smithi074_53697_object3740:head v 179'1919, mlcod=179'1911)", 2020-08-07T15:10:38.920 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:38.775349+0000", 2020-08-07T15:10:38.920 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0023870369999999998, 2020-08-07T15:10:38.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027986629999999998, 2020-08-07T15:10:38.920 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:38.921 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:38.921 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:38.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:38.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.775349+0000", 2020-08-07T15:10:38.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:38.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:38.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.775349+0000", 2020-08-07T15:10:38.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6665999999999998e-05 2020-08-07T15:10:38.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:38.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.775386+0000", 2020-08-07T15:10:38.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3360000000000002e-06 2020-08-07T15:10:38.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:38.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.775391+0000", 2020-08-07T15:10:38.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00077994299999999996 2020-08-07T15:10:38.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:38.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.776171+0000", 2020-08-07T15:10:38.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0090000000000002e-06 2020-08-07T15:10:38.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:38.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.776173+0000", 2020-08-07T15:10:38.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014213299999999999 2020-08-07T15:10:38.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.777594+0000", 2020-08-07T15:10:38.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9731000000000001e-05 2020-08-07T15:10:38.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:38.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.777614+0000", 2020-08-07T15:10:38.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002265015 2020-08-07T15:10:38.936 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.936 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:38.936 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.937 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:29928 5.2 e179/164 5:49bea0b5:::benchmark_data_smithi074_53697_object3740:head v 179'1920, mlcod=179'1911)", 2020-08-07T15:10:38.937 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:38.776194+0000", 2020-08-07T15:10:38.937 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0015423500000000001, 2020-08-07T15:10:38.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019958200000000001, 2020-08-07T15:10:38.938 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:38.938 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:38.938 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:38.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:38.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.776194+0000", 2020-08-07T15:10:38.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:38.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:38.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.776194+0000", 2020-08-07T15:10:38.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7325999999999999e-05 2020-08-07T15:10:38.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:38.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.776221+0000", 2020-08-07T15:10:38.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1230000000000002e-06 2020-08-07T15:10:38.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.776225+0000", 2020-08-07T15:10:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9400000000000002e-05 2020-08-07T15:10:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:38.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.776265+0000", 2020-08-07T15:10:38.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2020-08-07T15:10:38.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:38.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.776269+0000", 2020-08-07T15:10:38.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017257520000000001 2020-08-07T15:10:38.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:38.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.777995+0000", 2020-08-07T15:10:38.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0426000000000001e-05 2020-08-07T15:10:38.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:38.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:38.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:38.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:38.778015+0000", 2020-08-07T15:10:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001821526 2020-08-07T15:10:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:39.000 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:39.004 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:39.004 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:10:39.004 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 6 2020-08-07T15:10:39.005 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:10:39.007 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:10:39.008 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:39.006+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:39.109 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:39.106+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:39.191 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:10:39.192 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:10:39.194 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:10:39.194 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:10:39.194 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:10:39.206 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:10:39.212 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 19 16 30699 30683 12.6153 13.3125 0.00365486 0.0098983 2020-08-07T15:10:39.227 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:39.212+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:39.239 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:10:39.320 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:39.318+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:39.426 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:39.419+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:39.436 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:10:39.437 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.6 to 1 (10000) 2020-08-07T15:10:39.439 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:10:39.441 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:10:39.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:10:39.441 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:10:39.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.442 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42440 4.9 4:9b3fd4e3:::benchmark_data_smithi074_44586_object5304:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:10:39.442 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:10:39.442 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 99.197700155999996, 2020-08-07T15:10:39.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9384075030000001, 2020-08-07T15:10:39.443 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:39.443 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:39.443 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:39.443 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:39.443 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:39.444 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42440 2020-08-07T15:10:39.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.462 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:39.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:39.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:10:39.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:39.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:39.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:10:39.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6295000000000002e-05 2020-08-07T15:10:39.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:39.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237116+0000", 2020-08-07T15:10:39.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0390000000000001e-06 2020-08-07T15:10:39.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:39.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237122+0000", 2020-08-07T15:10:39.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8340000000000001e-05 2020-08-07T15:10:39.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:39.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237140+0000", 2020-08-07T15:10:39.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.045e-06 2020-08-07T15:10:39.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:39.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237145+0000", 2020-08-07T15:10:39.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044548920000000002 2020-08-07T15:10:39.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241600+0000", 2020-08-07T15:10:39.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103336 2020-08-07T15:10:39.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:39.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241703+0000", 2020-08-07T15:10:39.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018660399999999999 2020-08-07T15:10:39.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:10:39.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241890+0000", 2020-08-07T15:10:39.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00096796000000000002 2020-08-07T15:10:39.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:39.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.242858+0000", 2020-08-07T15:10:39.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039845000000000002 2020-08-07T15:10:39.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:39.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246843+0000", 2020-08-07T15:10:39.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3499999999999996e-07 2020-08-07T15:10:39.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:39.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246843+0000", 2020-08-07T15:10:39.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7132999999999995e-05 2020-08-07T15:10:39.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:39.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246920+0000", 2020-08-07T15:10:39.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046993410000000001 2020-08-07T15:10:39.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:39.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.251620+0000", 2020-08-07T15:10:39.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040332956000000003 2020-08-07T15:10:39.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.291953+0000", 2020-08-07T15:10:39.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8390000000000003e-06 2020-08-07T15:10:39.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:39.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.291958+0000", 2020-08-07T15:10:39.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99961357500000003 2020-08-07T15:10:39.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291572+0000", 2020-08-07T15:10:39.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3913999999999999e-05 2020-08-07T15:10:39.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:39.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291586+0000", 2020-08-07T15:10:39.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88379723099999996 2020-08-07T15:10:39.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175383+0000", 2020-08-07T15:10:39.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3868000000000004e-05 2020-08-07T15:10:39.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:39.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175467+0000", 2020-08-07T15:10:39.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9384075030000001 2020-08-07T15:10:39.541 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:39.541 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:39.541 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:39.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.542 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42489 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:10:39.542 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:10:39.542 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 99.154400093999996, 2020-08-07T15:10:39.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952608639999999, 2020-08-07T15:10:39.543 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:39.543 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:39.543 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:39.543 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:39.543 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:39.544 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42489 2020-08-07T15:10:39.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.544 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:39.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:39.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:10:39.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:39.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:39.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:10:39.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0379e-05 2020-08-07T15:10:39.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:39.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280410+0000", 2020-08-07T15:10:39.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0489999999999998e-06 2020-08-07T15:10:39.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:39.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280417+0000", 2020-08-07T15:10:39.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8281e-05 2020-08-07T15:10:39.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:39.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280435+0000", 2020-08-07T15:10:39.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7510000000000008e-06 2020-08-07T15:10:39.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:39.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280443+0000", 2020-08-07T15:10:39.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018142899999999999 2020-08-07T15:10:39.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280624+0000", 2020-08-07T15:10:39.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013473599999999999 2020-08-07T15:10:39.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:39.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280759+0000", 2020-08-07T15:10:39.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019964200000000001 2020-08-07T15:10:39.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:10:39.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280959+0000", 2020-08-07T15:10:39.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005588038 2020-08-07T15:10:39.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:39.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286547+0000", 2020-08-07T15:10:39.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.376e-05 2020-08-07T15:10:39.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:39.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286561+0000", 2020-08-07T15:10:39.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4199999999999995e-07 2020-08-07T15:10:39.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:39.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286561+0000", 2020-08-07T15:10:39.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9218e-05 2020-08-07T15:10:39.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:39.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286631+0000", 2020-08-07T15:10:39.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027909199999999999 2020-08-07T15:10:39.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:39.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286910+0000", 2020-08-07T15:10:39.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050904089999999997 2020-08-07T15:10:39.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292000+0000", 2020-08-07T15:10:39.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9579999999999996e-06 2020-08-07T15:10:39.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:39.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292009+0000", 2020-08-07T15:10:39.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99961402799999999 2020-08-07T15:10:39.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291623+0000", 2020-08-07T15:10:39.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2490000000000001e-06 2020-08-07T15:10:39.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:39.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291631+0000", 2020-08-07T15:10:39.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88387422199999999 2020-08-07T15:10:39.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175506+0000", 2020-08-07T15:10:39.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000114881 2020-08-07T15:10:39.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:39.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175620+0000", 2020-08-07T15:10:39.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952608639999999 2020-08-07T15:10:39.564 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:39.565 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:39.565 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:39.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.567 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42490 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:10:39.567 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:10:39.567 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 99.154272926000004, 2020-08-07T15:10:39.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89528059, 2020-08-07T15:10:39.568 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:39.568 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:39.568 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:39.568 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:39.569 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:39.569 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42490 2020-08-07T15:10:39.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.569 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:39.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:39.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:10:39.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:39.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:39.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:10:39.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2571999999999997e-05 2020-08-07T15:10:39.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:39.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280539+0000", 2020-08-07T15:10:39.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999998e-06 2020-08-07T15:10:39.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:39.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280545+0000", 2020-08-07T15:10:39.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.134e-05 2020-08-07T15:10:39.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:39.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280556+0000", 2020-08-07T15:10:39.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.746e-06 2020-08-07T15:10:39.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:39.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280561+0000", 2020-08-07T15:10:39.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00089258999999999996 2020-08-07T15:10:39.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281453+0000", 2020-08-07T15:10:39.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5641000000000001e-05 2020-08-07T15:10:39.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:39.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281549+0000", 2020-08-07T15:10:39.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020555200000000001 2020-08-07T15:10:39.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:10:39.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281755+0000", 2020-08-07T15:10:39.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048878230000000003 2020-08-07T15:10:39.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:39.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286642+0000", 2020-08-07T15:10:39.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0954e-05 2020-08-07T15:10:39.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:39.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286693+0000", 2020-08-07T15:10:39.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0699999999999997e-07 2020-08-07T15:10:39.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:39.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286694+0000", 2020-08-07T15:10:39.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3697999999999998e-05 2020-08-07T15:10:39.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:39.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286768+0000", 2020-08-07T15:10:39.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017185900000000001 2020-08-07T15:10:39.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:39.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286940+0000", 2020-08-07T15:10:39.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051207819999999999 2020-08-07T15:10:39.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292060+0000", 2020-08-07T15:10:39.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8000000000000004e-06 2020-08-07T15:10:39.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:39.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292069+0000", 2020-08-07T15:10:39.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99959520599999996 2020-08-07T15:10:39.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291664+0000", 2020-08-07T15:10:39.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0160000000000008e-06 2020-08-07T15:10:39.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:39.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291672+0000", 2020-08-07T15:10:39.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88399374500000005 2020-08-07T15:10:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175666+0000", 2020-08-07T15:10:39.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101239 2020-08-07T15:10:39.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:39.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175767+0000", 2020-08-07T15:10:39.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89528059 2020-08-07T15:10:39.602 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:39.602 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:39.602 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:39.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.603 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42491 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:10:39.603 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:10:39.603 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 99.154165990999999, 2020-08-07T15:10:39.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895291936, 2020-08-07T15:10:39.605 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:39.605 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:39.605 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:39.605 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:39.606 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:39.607 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42491 2020-08-07T15:10:39.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.608 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:39.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:39.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:10:39.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:39.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:39.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:10:39.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0402000000000001e-05 2020-08-07T15:10:39.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:39.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280624+0000", 2020-08-07T15:10:39.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0340000000000001e-06 2020-08-07T15:10:39.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:39.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280627+0000", 2020-08-07T15:10:39.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1886999999999999e-05 2020-08-07T15:10:39.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:39.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280649+0000", 2020-08-07T15:10:39.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5430000000000004e-06 2020-08-07T15:10:39.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:39.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280654+0000", 2020-08-07T15:10:39.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032800960000000001 2020-08-07T15:10:39.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.283934+0000", 2020-08-07T15:10:39.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011264699999999999 2020-08-07T15:10:39.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:39.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284046+0000", 2020-08-07T15:10:39.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019987299999999999 2020-08-07T15:10:39.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:10:39.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284246+0000", 2020-08-07T15:10:39.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025610960000000001 2020-08-07T15:10:39.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:39.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286807+0000", 2020-08-07T15:10:39.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1409999999999999e-06 2020-08-07T15:10:39.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:39.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286814+0000", 2020-08-07T15:10:39.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0499999999999999e-07 2020-08-07T15:10:39.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:39.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286815+0000", 2020-08-07T15:10:39.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8538000000000003e-05 2020-08-07T15:10:39.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:39.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286853+0000", 2020-08-07T15:10:39.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112914 2020-08-07T15:10:39.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:39.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286966+0000", 2020-08-07T15:10:39.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051403120000000002 2020-08-07T15:10:39.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292107+0000", 2020-08-07T15:10:39.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8019999999999992e-06 2020-08-07T15:10:39.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:39.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292114+0000", 2020-08-07T15:10:39.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.999590859 2020-08-07T15:10:39.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291705+0000", 2020-08-07T15:10:39.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.174e-06 2020-08-07T15:10:39.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:39.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291713+0000", 2020-08-07T15:10:39.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88407676000000002 2020-08-07T15:10:39.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175790+0000", 2020-08-07T15:10:39.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5452999999999996e-05 2020-08-07T15:10:39.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:39.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175886+0000", 2020-08-07T15:10:39.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895291936 2020-08-07T15:10:39.642 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:39.642 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:39.642 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:39.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.643 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42492 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:10:39.643 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:10:39.643 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 99.154072940999995, 2020-08-07T15:10:39.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895293774, 2020-08-07T15:10:39.644 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:39.644 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:39.644 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:39.644 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:39.644 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:39.645 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42492 2020-08-07T15:10:39.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.645 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:39.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:39.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:10:39.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:39.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:39.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:10:39.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8272000000000001e-05 2020-08-07T15:10:39.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:39.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280735+0000", 2020-08-07T15:10:39.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2159999999999997e-06 2020-08-07T15:10:39.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:39.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280740+0000", 2020-08-07T15:10:39.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0251000000000001e-05 2020-08-07T15:10:39.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:39.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280750+0000", 2020-08-07T15:10:39.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3139999999999997e-06 2020-08-07T15:10:39.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:39.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280755+0000", 2020-08-07T15:10:39.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039343579999999998 2020-08-07T15:10:39.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284689+0000", 2020-08-07T15:10:39.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111648 2020-08-07T15:10:39.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:39.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284801+0000", 2020-08-07T15:10:39.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019563600000000001 2020-08-07T15:10:39.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:10:39.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284996+0000", 2020-08-07T15:10:39.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018636709999999999 2020-08-07T15:10:39.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:39.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286860+0000", 2020-08-07T15:10:39.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001865802 2020-08-07T15:10:39.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:39.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288726+0000", 2020-08-07T15:10:39.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0399999999999996e-07 2020-08-07T15:10:39.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:39.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288726+0000", 2020-08-07T15:10:39.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6573999999999998e-05 2020-08-07T15:10:39.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:39.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288773+0000", 2020-08-07T15:10:39.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003455283 2020-08-07T15:10:39.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292228+0000", 2020-08-07T15:10:39.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0099999999999995e-06 2020-08-07T15:10:39.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:39.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292236+0000", 2020-08-07T15:10:39.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99950963999999998 2020-08-07T15:10:39.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291746+0000", 2020-08-07T15:10:39.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0779999999999996e-06 2020-08-07T15:10:39.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:39.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291754+0000", 2020-08-07T15:10:39.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.884152508 2020-08-07T15:10:39.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175906+0000", 2020-08-07T15:10:39.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4009000000000002e-05 2020-08-07T15:10:39.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:39.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175980+0000", 2020-08-07T15:10:39.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895293774 2020-08-07T15:10:39.678 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:39.678 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:39.678 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:39.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.679 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42493 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:10:39.679 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:10:39.679 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 99.153971294000002, 2020-08-07T15:10:39.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952963789999999, 2020-08-07T15:10:39.680 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:39.680 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:39.680 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:39.680 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:39.681 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:39.681 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42493 2020-08-07T15:10:39.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.681 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:39.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:39.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:10:39.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:39.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:39.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:10:39.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9740999999999997e-05 2020-08-07T15:10:39.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:39.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280838+0000", 2020-08-07T15:10:39.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7809999999999996e-06 2020-08-07T15:10:39.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:39.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280843+0000", 2020-08-07T15:10:39.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0648999999999999e-05 2020-08-07T15:10:39.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:39.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280854+0000", 2020-08-07T15:10:39.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3669999999999999e-06 2020-08-07T15:10:39.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:39.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280858+0000", 2020-08-07T15:10:39.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004850867 2020-08-07T15:10:39.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.285709+0000", 2020-08-07T15:10:39.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110037 2020-08-07T15:10:39.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:39.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.285819+0000", 2020-08-07T15:10:39.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020935699999999999 2020-08-07T15:10:39.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:10:39.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286028+0000", 2020-08-07T15:10:39.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020900850000000002 2020-08-07T15:10:39.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:39.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288118+0000", 2020-08-07T15:10:39.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041669649999999999 2020-08-07T15:10:39.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292285+0000", 2020-08-07T15:10:39.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3820000000000004e-06 2020-08-07T15:10:39.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:39.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292294+0000", 2020-08-07T15:10:39.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949242900000002 2020-08-07T15:10:39.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291786+0000", 2020-08-07T15:10:39.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000003e-06 2020-08-07T15:10:39.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:39.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291794+0000", 2020-08-07T15:10:39.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88420652700000002 2020-08-07T15:10:39.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176000+0000", 2020-08-07T15:10:39.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4634000000000003e-05 2020-08-07T15:10:39.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:39.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176085+0000", 2020-08-07T15:10:39.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952963789999999 2020-08-07T15:10:39.698 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:39.699 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:39.701 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:39.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.702 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42494 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:10:39.702 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:10:39.702 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 99.153869194999999, 2020-08-07T15:10:39.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952882820000001, 2020-08-07T15:10:39.703 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:39.703 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:39.715 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:39.715 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:39.715 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:39.716 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42494 2020-08-07T15:10:39.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.716 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:39.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:39.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:10:39.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:39.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:39.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:10:39.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0117999999999997e-05 2020-08-07T15:10:39.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:39.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280941+0000", 2020-08-07T15:10:39.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1240000000000004e-06 2020-08-07T15:10:39.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:39.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280946+0000", 2020-08-07T15:10:39.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0000000000000001e-05 2020-08-07T15:10:39.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:39.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280956+0000", 2020-08-07T15:10:39.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.279e-06 2020-08-07T15:10:39.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:39.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280960+0000", 2020-08-07T15:10:39.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0062727080000000001 2020-08-07T15:10:39.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287233+0000", 2020-08-07T15:10:39.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018362900000000001 2020-08-07T15:10:39.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:39.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287416+0000", 2020-08-07T15:10:39.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029536100000000001 2020-08-07T15:10:39.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:10:39.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287712+0000", 2020-08-07T15:10:39.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046231739999999999 2020-08-07T15:10:39.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292335+0000", 2020-08-07T15:10:39.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3860000000000001e-06 2020-08-07T15:10:39.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:39.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292342+0000", 2020-08-07T15:10:39.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99948501000000001 2020-08-07T15:10:39.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291827+0000", 2020-08-07T15:10:39.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2272999999999999e-05 2020-08-07T15:10:39.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:39.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291840+0000", 2020-08-07T15:10:39.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88426668100000005 2020-08-07T15:10:39.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176106+0000", 2020-08-07T15:10:39.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2539000000000005e-05 2020-08-07T15:10:39.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:39.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176179+0000", 2020-08-07T15:10:39.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952882820000001 2020-08-07T15:10:39.756 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:39.756 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:39.756 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:39.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.756 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42495 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:10:39.757 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:10:39.757 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 99.153768104999997, 2020-08-07T15:10:39.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895286867, 2020-08-07T15:10:39.757 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:39.757 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:39.758 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:39.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:39.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:39.759 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42495 2020-08-07T15:10:39.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.759 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:39.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:39.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:10:39.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:39.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:39.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:10:39.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6690999999999997e-05 2020-08-07T15:10:39.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:39.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281038+0000", 2020-08-07T15:10:39.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7860000000000002e-06 2020-08-07T15:10:39.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:39.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281044+0000", 2020-08-07T15:10:39.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0429e-05 2020-08-07T15:10:39.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:39.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281054+0000", 2020-08-07T15:10:39.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9040000000000002e-06 2020-08-07T15:10:39.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:39.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281058+0000", 2020-08-07T15:10:39.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070941490000000001 2020-08-07T15:10:39.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288153+0000", 2020-08-07T15:10:39.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9006999999999998e-05 2020-08-07T15:10:39.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:39.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288222+0000", 2020-08-07T15:10:39.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012106400000000001 2020-08-07T15:10:39.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:10:39.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288343+0000", 2020-08-07T15:10:39.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004034752 2020-08-07T15:10:39.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292377+0000", 2020-08-07T15:10:39.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.537e-06 2020-08-07T15:10:39.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:39.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292385+0000", 2020-08-07T15:10:39.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949123500000003 2020-08-07T15:10:39.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291876+0000", 2020-08-07T15:10:39.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7729999999999999e-06 2020-08-07T15:10:39.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:39.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291884+0000", 2020-08-07T15:10:39.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88431700899999999 2020-08-07T15:10:39.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176201+0000", 2020-08-07T15:10:39.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7531e-05 2020-08-07T15:10:39.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:39.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176278+0000", 2020-08-07T15:10:39.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895286867 2020-08-07T15:10:39.780 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:39.781 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:39.781 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:39.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.782 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42496 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:10:39.782 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:10:39.782 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 99.153663195999997, 2020-08-07T15:10:39.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89526942, 2020-08-07T15:10:39.783 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:39.783 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:39.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:39.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:39.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:39.783 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42496 2020-08-07T15:10:39.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.784 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:39.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:39.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:10:39.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:39.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:39.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:10:39.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9227999999999997e-05 2020-08-07T15:10:39.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:39.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281146+0000", 2020-08-07T15:10:39.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1120000000000004e-06 2020-08-07T15:10:39.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:39.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281151+0000", 2020-08-07T15:10:39.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2208999999999999e-05 2020-08-07T15:10:39.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:39.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281163+0000", 2020-08-07T15:10:39.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5469999999999998e-06 2020-08-07T15:10:39.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:39.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281168+0000", 2020-08-07T15:10:39.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0076520829999999996 2020-08-07T15:10:39.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288820+0000", 2020-08-07T15:10:39.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5035000000000003e-05 2020-08-07T15:10:39.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:39.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288885+0000", 2020-08-07T15:10:39.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116507 2020-08-07T15:10:39.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:10:39.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.289001+0000", 2020-08-07T15:10:39.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034121960000000002 2020-08-07T15:10:39.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292413+0000", 2020-08-07T15:10:39.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8109999999999997e-06 2020-08-07T15:10:39.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:39.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292420+0000", 2020-08-07T15:10:39.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949504600000005 2020-08-07T15:10:39.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291915+0000", 2020-08-07T15:10:39.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3999999999999992e-06 2020-08-07T15:10:39.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:39.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291924+0000", 2020-08-07T15:10:39.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88437394599999997 2020-08-07T15:10:39.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176298+0000", 2020-08-07T15:10:39.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8300000000000007e-05 2020-08-07T15:10:39.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:39.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176366+0000", 2020-08-07T15:10:39.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89526942 2020-08-07T15:10:39.859 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:39.860 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:39.860 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:39.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.861 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73081 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:10:39.861 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:10:39.861 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 72.208193922000007, 2020-08-07T15:10:39.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0005038420000001, 2020-08-07T15:10:39.862 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:39.862 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:39.862 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:39.862 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:39.863 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:39.863 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73081 2020-08-07T15:10:39.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.863 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:39.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:39.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:10:39.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:39.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:39.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:10:39.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3016000000000003e-05 2020-08-07T15:10:39.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:39.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226609+0000", 2020-08-07T15:10:39.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4479999999999996e-06 2020-08-07T15:10:39.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:39.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226613+0000", 2020-08-07T15:10:39.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0189999999999995e-06 2020-08-07T15:10:39.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:39.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226622+0000", 2020-08-07T15:10:39.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5980000000000001e-06 2020-08-07T15:10:39.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:39.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226626+0000", 2020-08-07T15:10:39.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8088999999999999e-05 2020-08-07T15:10:39.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226684+0000", 2020-08-07T15:10:39.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2162e-05 2020-08-07T15:10:39.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:39.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226696+0000", 2020-08-07T15:10:39.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98675689 2020-08-07T15:10:39.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213453+0000", 2020-08-07T15:10:39.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012916800000000001 2020-08-07T15:10:39.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:39.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213582+0000", 2020-08-07T15:10:39.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015665899999999999 2020-08-07T15:10:39.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:10:39.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213739+0000", 2020-08-07T15:10:39.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049940239999999997 2020-08-07T15:10:39.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:39.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218733+0000", 2020-08-07T15:10:39.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0079774779999999997 2020-08-07T15:10:39.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:39.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226710+0000", 2020-08-07T15:10:39.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.161e-06 2020-08-07T15:10:39.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:39.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226711+0000", 2020-08-07T15:10:39.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0105000000000001e-05 2020-08-07T15:10:39.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:39.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226772+0000", 2020-08-07T15:10:39.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5633999999999997e-05 2020-08-07T15:10:39.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:39.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226817+0000", 2020-08-07T15:10:39.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7379000000000001e-05 2020-08-07T15:10:39.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:39.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226875+0000", 2020-08-07T15:10:39.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8000000000000005e-07 2020-08-07T15:10:39.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:39.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226875+0000", 2020-08-07T15:10:39.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019423199999999999 2020-08-07T15:10:39.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:39.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227070+0000", 2020-08-07T15:10:39.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0005038420000001 2020-08-07T15:10:39.947 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:39.947 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:39.947 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:39.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.948 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73082 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:10:39.949 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:10:39.950 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 72.207823457999993, 2020-08-07T15:10:39.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001326693, 2020-08-07T15:10:39.950 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:39.951 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:39.951 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:39.951 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:39.951 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:39.952 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73082 2020-08-07T15:10:39.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.952 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:39.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:39.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:10:39.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:39.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:39.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:10:39.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6928000000000003e-05 2020-08-07T15:10:39.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:39.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226973+0000", 2020-08-07T15:10:39.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3630000000000004e-06 2020-08-07T15:10:39.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:39.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226977+0000", 2020-08-07T15:10:39.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9659999999999996e-06 2020-08-07T15:10:39.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:39.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226985+0000", 2020-08-07T15:10:39.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5800000000000002e-06 2020-08-07T15:10:39.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:39.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226990+0000", 2020-08-07T15:10:39.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9101999999999998e-05 2020-08-07T15:10:39.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227039+0000", 2020-08-07T15:10:39.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9620000000000005e-06 2020-08-07T15:10:39.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:39.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227049+0000", 2020-08-07T15:10:39.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98722513999999995 2020-08-07T15:10:39.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214274+0000", 2020-08-07T15:10:39.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2700999999999994e-05 2020-08-07T15:10:39.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:39.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214347+0000", 2020-08-07T15:10:39.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015346899999999999 2020-08-07T15:10:39.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:10:39.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214500+0000", 2020-08-07T15:10:39.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042450309999999998 2020-08-07T15:10:39.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:39.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218745+0000", 2020-08-07T15:10:39.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008379503 2020-08-07T15:10:39.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:39.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227125+0000", 2020-08-07T15:10:39.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7899999999999998e-07 2020-08-07T15:10:39.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:39.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227126+0000", 2020-08-07T15:10:39.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1610000000000002e-05 2020-08-07T15:10:39.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:39.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227177+0000", 2020-08-07T15:10:39.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489000000000003e-05 2020-08-07T15:10:39.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:39.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227226+0000", 2020-08-07T15:10:39.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00084338600000000001 2020-08-07T15:10:39.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:39.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228069+0000", 2020-08-07T15:10:39.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7e-07 2020-08-07T15:10:39.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:39.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228069+0000", 2020-08-07T15:10:39.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019351399999999999 2020-08-07T15:10:39.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:39.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228263+0000", 2020-08-07T15:10:39.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001326693 2020-08-07T15:10:39.975 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:39.975 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:39.975 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:39.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.976 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73083 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:10:39.976 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:10:39.977 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 72.207729198999999, 2020-08-07T15:10:39.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015968209999999, 2020-08-07T15:10:39.977 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:39.977 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:39.978 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:39.978 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:39.978 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:39.985 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73083 2020-08-07T15:10:39.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.986 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:39.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:10:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:10:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1627000000000001e-05 2020-08-07T15:10:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227062+0000", 2020-08-07T15:10:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.253e-06 2020-08-07T15:10:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227065+0000", 2020-08-07T15:10:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3168e-05 2020-08-07T15:10:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227079+0000", 2020-08-07T15:10:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1549999999999999e-06 2020-08-07T15:10:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227082+0000", 2020-08-07T15:10:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5534999999999997e-05 2020-08-07T15:10:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227127+0000", 2020-08-07T15:10:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6823000000000001e-05 2020-08-07T15:10:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227144+0000", 2020-08-07T15:10:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98771770599999997 2020-08-07T15:10:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214862+0000", 2020-08-07T15:10:39.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8252999999999999e-05 2020-08-07T15:10:39.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:39.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:40.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214930+0000", 2020-08-07T15:10:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013646400000000001 2020-08-07T15:10:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:10:40.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215066+0000", 2020-08-07T15:10:40.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036907089999999999 2020-08-07T15:10:40.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:40.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218757+0000", 2020-08-07T15:10:40.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0085133450000000003 2020-08-07T15:10:40.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:40.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227270+0000", 2020-08-07T15:10:40.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4899999999999995e-07 2020-08-07T15:10:40.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:40.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227271+0000", 2020-08-07T15:10:40.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4555999999999997e-05 2020-08-07T15:10:40.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:40.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227326+0000", 2020-08-07T15:10:40.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024594800000000001 2020-08-07T15:10:40.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:40.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227572+0000", 2020-08-07T15:10:40.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00086125700000000004 2020-08-07T15:10:40.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:40.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228433+0000", 2020-08-07T15:10:40.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3799999999999998e-07 2020-08-07T15:10:40.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:40.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228433+0000", 2020-08-07T15:10:40.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019413500000000001 2020-08-07T15:10:40.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:40.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228627+0000", 2020-08-07T15:10:40.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015968209999999 2020-08-07T15:10:40.035 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:40.036 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:40.036 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:40.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.036 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73084 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:10:40.037 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:10:40.037 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 72.207640260000005, 2020-08-07T15:10:40.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0016919049999999, 2020-08-07T15:10:40.037 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:40.057 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:40.057 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:40.058 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:40.058 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:40.058 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73084 2020-08-07T15:10:40.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.058 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:40.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:40.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:10:40.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:40.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:40.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:10:40.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5565999999999997e-05 2020-08-07T15:10:40.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:40.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227165+0000", 2020-08-07T15:10:40.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2020-08-07T15:10:40.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:40.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227169+0000", 2020-08-07T15:10:40.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6850000000000001e-06 2020-08-07T15:10:40.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:40.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227176+0000", 2020-08-07T15:10:40.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1949999999999999e-06 2020-08-07T15:10:40.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:40.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227179+0000", 2020-08-07T15:10:40.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0318000000000003e-05 2020-08-07T15:10:40.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:40.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227220+0000", 2020-08-07T15:10:40.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0349e-05 2020-08-07T15:10:40.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:40.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227230+0000", 2020-08-07T15:10:40.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98832962400000002 2020-08-07T15:10:40.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:40.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215560+0000", 2020-08-07T15:10:40.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013913099999999999 2020-08-07T15:10:40.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:40.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215699+0000", 2020-08-07T15:10:40.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000200006 2020-08-07T15:10:40.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:10:40.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215899+0000", 2020-08-07T15:10:40.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028673650000000002 2020-08-07T15:10:40.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:40.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218766+0000", 2020-08-07T15:10:40.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086101779999999996 2020-08-07T15:10:40.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:40.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227376+0000", 2020-08-07T15:10:40.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0999999999999998e-07 2020-08-07T15:10:40.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:40.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227377+0000", 2020-08-07T15:10:40.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6552999999999997e-05 2020-08-07T15:10:40.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:40.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227454+0000", 2020-08-07T15:10:40.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000421717 2020-08-07T15:10:40.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:40.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227875+0000", 2020-08-07T15:10:40.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00081781300000000003 2020-08-07T15:10:40.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:40.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228693+0000", 2020-08-07T15:10:40.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1699999999999997e-07 2020-08-07T15:10:40.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:40.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228694+0000", 2020-08-07T15:10:40.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117483 2020-08-07T15:10:40.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:40.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228811+0000", 2020-08-07T15:10:40.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0016919049999999 2020-08-07T15:10:40.110 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:40.110 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:40.111 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:40.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.111 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73085 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:10:40.112 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:10:40.112 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 72.207504444999998, 2020-08-07T15:10:40.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0018015629999999, 2020-08-07T15:10:40.121 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:40.121 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:40.121 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:40.122 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:40.122 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:40.130 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73085 2020-08-07T15:10:40.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.131 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:40.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:40.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:10:40.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:40.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:40.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:10:40.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6570999999999997e-05 2020-08-07T15:10:40.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:40.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227322+0000", 2020-08-07T15:10:40.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5509999999999996e-06 2020-08-07T15:10:40.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:40.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227328+0000", 2020-08-07T15:10:40.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2498e-05 2020-08-07T15:10:40.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:40.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227341+0000", 2020-08-07T15:10:40.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4389999999999999e-06 2020-08-07T15:10:40.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:40.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227346+0000", 2020-08-07T15:10:40.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3711999999999998e-05 2020-08-07T15:10:40.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:40.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227390+0000", 2020-08-07T15:10:40.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1868999999999999e-05 2020-08-07T15:10:40.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:40.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227402+0000", 2020-08-07T15:10:40.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98896525099999999 2020-08-07T15:10:40.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:40.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216367+0000", 2020-08-07T15:10:40.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012573 2020-08-07T15:10:40.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:40.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216493+0000", 2020-08-07T15:10:40.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020525200000000001 2020-08-07T15:10:40.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:10:40.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216698+0000", 2020-08-07T15:10:40.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037596470000000001 2020-08-07T15:10:40.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:40.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.220458+0000", 2020-08-07T15:10:40.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070227190000000002 2020-08-07T15:10:40.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:40.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227480+0000", 2020-08-07T15:10:40.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6600000000000003e-07 2020-08-07T15:10:40.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:40.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227481+0000", 2020-08-07T15:10:40.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9898000000000003e-05 2020-08-07T15:10:40.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:40.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227521+0000", 2020-08-07T15:10:40.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039130000000000002 2020-08-07T15:10:40.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:40.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227912+0000", 2020-08-07T15:10:40.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00095502399999999996 2020-08-07T15:10:40.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:40.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228867+0000", 2020-08-07T15:10:40.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2020-08-07T15:10:40.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:40.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228867+0000", 2020-08-07T15:10:40.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018933599999999999 2020-08-07T15:10:40.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:40.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229057+0000", 2020-08-07T15:10:40.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0018015629999999 2020-08-07T15:10:40.171 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:40.171 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:40.171 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:40.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.172 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73086 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:10:40.172 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:10:40.172 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 72.207371949000006, 2020-08-07T15:10:40.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002364923, 2020-08-07T15:10:40.173 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:40.173 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:40.173 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:40.173 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:40.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:40.174 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73086 2020-08-07T15:10:40.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.174 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:40.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:40.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:10:40.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:40.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:40.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:10:40.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000238426 2020-08-07T15:10:40.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:40.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227626+0000", 2020-08-07T15:10:40.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9609999999999999e-06 2020-08-07T15:10:40.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:40.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227633+0000", 2020-08-07T15:10:40.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2727e-05 2020-08-07T15:10:40.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:40.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227646+0000", 2020-08-07T15:10:40.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1739999999999997e-06 2020-08-07T15:10:40.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:40.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227652+0000", 2020-08-07T15:10:40.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5004000000000001e-05 2020-08-07T15:10:40.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:40.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227737+0000", 2020-08-07T15:10:40.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2744e-05 2020-08-07T15:10:40.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:40.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227750+0000", 2020-08-07T15:10:40.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98938776500000003 2020-08-07T15:10:40.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:40.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217138+0000", 2020-08-07T15:10:40.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012361499999999999 2020-08-07T15:10:40.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:40.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217261+0000", 2020-08-07T15:10:40.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000212504 2020-08-07T15:10:40.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:10:40.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217474+0000", 2020-08-07T15:10:40.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002995036 2020-08-07T15:10:40.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:40.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.220469+0000", 2020-08-07T15:10:40.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071283249999999996 2020-08-07T15:10:40.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:40.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227597+0000", 2020-08-07T15:10:40.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0499999999999999e-07 2020-08-07T15:10:40.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:40.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227597+0000", 2020-08-07T15:10:40.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1872999999999997e-05 2020-08-07T15:10:40.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:40.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227639+0000", 2020-08-07T15:10:40.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032010200000000001 2020-08-07T15:10:40.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:40.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227959+0000", 2020-08-07T15:10:40.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015980899999999999 2020-08-07T15:10:40.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:40.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229557+0000", 2020-08-07T15:10:40.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9299999999999999e-07 2020-08-07T15:10:40.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:40.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229558+0000", 2020-08-07T15:10:40.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019477900000000001 2020-08-07T15:10:40.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:40.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229753+0000", 2020-08-07T15:10:40.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002364923 2020-08-07T15:10:40.250 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:40.251 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:40.251 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:40.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.252 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73087 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:10:40.252 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:10:40.252 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 72.206430796000006, 2020-08-07T15:10:40.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001896506, 2020-08-07T15:10:40.253 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:40.253 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:40.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:40.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:40.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:40.254 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73087 2020-08-07T15:10:40.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.254 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:40.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:40.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:10:40.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:40.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:40.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:10:40.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7354e-05 2020-08-07T15:10:40.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:40.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228386+0000", 2020-08-07T15:10:40.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8000000000000001e-06 2020-08-07T15:10:40.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:40.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228393+0000", 2020-08-07T15:10:40.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3784999999999999e-05 2020-08-07T15:10:40.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:40.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228407+0000", 2020-08-07T15:10:40.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8090000000000004e-06 2020-08-07T15:10:40.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:40.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228414+0000", 2020-08-07T15:10:40.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6907000000000003e-05 2020-08-07T15:10:40.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:40.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228461+0000", 2020-08-07T15:10:40.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8929999999999994e-06 2020-08-07T15:10:40.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:40.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228469+0000", 2020-08-07T15:10:40.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98950057700000005 2020-08-07T15:10:40.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:40.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217970+0000", 2020-08-07T15:10:40.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000132182 2020-08-07T15:10:40.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:40.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218102+0000", 2020-08-07T15:10:40.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020073399999999999 2020-08-07T15:10:40.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:10:40.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218303+0000", 2020-08-07T15:10:40.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035951120000000001 2020-08-07T15:10:40.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:40.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.221898+0000", 2020-08-07T15:10:40.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057898660000000003 2020-08-07T15:10:40.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:40.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227688+0000", 2020-08-07T15:10:40.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0200000000000002e-07 2020-08-07T15:10:40.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:40.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227688+0000", 2020-08-07T15:10:40.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1477999999999996e-05 2020-08-07T15:10:40.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:40.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227760+0000", 2020-08-07T15:10:40.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016657480000000001 2020-08-07T15:10:40.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:40.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229426+0000", 2020-08-07T15:10:40.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061290400000000005 2020-08-07T15:10:40.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:40.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230039+0000", 2020-08-07T15:10:40.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2e-07 2020-08-07T15:10:40.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:40.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230039+0000", 2020-08-07T15:10:40.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018643500000000001 2020-08-07T15:10:40.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:40.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230225+0000", 2020-08-07T15:10:40.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001896506 2020-08-07T15:10:40.282 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:40.282 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:40.282 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:40.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.290 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73088 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:10:40.290 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:10:40.291 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 72.206040106000003, 2020-08-07T15:10:40.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001627153, 2020-08-07T15:10:40.291 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:40.297 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:40.297 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:40.298 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:40.298 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:40.298 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73088 2020-08-07T15:10:40.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.299 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:40.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:40.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:10:40.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:40.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:40.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:10:40.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7741000000000003e-05 2020-08-07T15:10:40.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:40.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228767+0000", 2020-08-07T15:10:40.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6409999999999999e-06 2020-08-07T15:10:40.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:40.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228773+0000", 2020-08-07T15:10:40.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0874e-05 2020-08-07T15:10:40.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:40.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228784+0000", 2020-08-07T15:10:40.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1529999999999996e-06 2020-08-07T15:10:40.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:40.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228789+0000", 2020-08-07T15:10:40.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3945e-05 2020-08-07T15:10:40.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:40.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228833+0000", 2020-08-07T15:10:40.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4020000000000002e-06 2020-08-07T15:10:40.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:40.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228841+0000", 2020-08-07T15:10:40.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98996913900000005 2020-08-07T15:10:40.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:40.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218810+0000", 2020-08-07T15:10:40.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112484 2020-08-07T15:10:40.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:40.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218923+0000", 2020-08-07T15:10:40.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019891 2020-08-07T15:10:40.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:10:40.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.219122+0000", 2020-08-07T15:10:40.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027884099999999998 2020-08-07T15:10:40.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:40.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.221910+0000", 2020-08-07T15:10:40.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058753399999999997 2020-08-07T15:10:40.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:40.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227786+0000", 2020-08-07T15:10:40.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9200000000000002e-07 2020-08-07T15:10:40.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:40.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227786+0000", 2020-08-07T15:10:40.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8572000000000003e-05 2020-08-07T15:10:40.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:40.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227824+0000", 2020-08-07T15:10:40.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001651296 2020-08-07T15:10:40.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:40.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229476+0000", 2020-08-07T15:10:40.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00077620399999999996 2020-08-07T15:10:40.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:40.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230252+0000", 2020-08-07T15:10:40.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1600000000000002e-07 2020-08-07T15:10:40.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:40.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230252+0000", 2020-08-07T15:10:40.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4433999999999997e-05 2020-08-07T15:10:40.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:40.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230347+0000", 2020-08-07T15:10:40.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001627153 2020-08-07T15:10:40.329 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:40.329 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:40.329 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:40.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.330 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:128738 4.9 4.370c6199 (undecoded) ondisk+write+known_if_redirected e155)", 2020-08-07T15:10:40.330 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:03.204387+0000", 2020-08-07T15:10:40.331 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.230372453000001, 2020-08-07T15:10:40.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99077720899999999, 2020-08-07T15:10:40.331 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:40.331 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:40.332 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:40.332 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:40.332 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:40.332 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128738 2020-08-07T15:10:40.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.333 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:40.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:40.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204387+0000", 2020-08-07T15:10:40.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:40.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:40.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204387+0000", 2020-08-07T15:10:40.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4561999999999999e-05 2020-08-07T15:10:40.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:40.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204422+0000", 2020-08-07T15:10:40.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7270000000000001e-06 2020-08-07T15:10:40.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:40.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204428+0000", 2020-08-07T15:10:40.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3451e-05 2020-08-07T15:10:40.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:40.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204442+0000", 2020-08-07T15:10:40.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1589999999999998e-06 2020-08-07T15:10:40.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:40.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204448+0000", 2020-08-07T15:10:40.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3755999999999999e-05 2020-08-07T15:10:40.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:40.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204492+0000", 2020-08-07T15:10:40.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2450000000000001e-06 2020-08-07T15:10:40.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:40.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204499+0000", 2020-08-07T15:10:40.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99065224600000001 2020-08-07T15:10:40.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:40.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195151+0000", 2020-08-07T15:10:40.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3063e-05 2020-08-07T15:10:40.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:40.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195164+0000", 2020-08-07T15:10:40.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99077720899999999 2020-08-07T15:10:40.362 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:40.362 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:40.363 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:40.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.363 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:128742 4.9 4.69b70ab9 (undecoded) ondisk+write+known_if_redirected e155)", 2020-08-07T15:10:40.363 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:03.205224+0000", 2020-08-07T15:10:40.364 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.229535818999999, 2020-08-07T15:10:40.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.989964548, 2020-08-07T15:10:40.364 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:40.364 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:40.365 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:40.365 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:40.365 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:40.365 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128742 2020-08-07T15:10:40.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.374 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:40.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:40.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205224+0000", 2020-08-07T15:10:40.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:40.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:40.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205224+0000", 2020-08-07T15:10:40.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1018e-05 2020-08-07T15:10:40.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:40.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205255+0000", 2020-08-07T15:10:40.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5840000000000002e-06 2020-08-07T15:10:40.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:40.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205260+0000", 2020-08-07T15:10:40.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2388999999999999e-05 2020-08-07T15:10:40.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:40.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205273+0000", 2020-08-07T15:10:40.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3799999999999999e-06 2020-08-07T15:10:40.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:40.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205279+0000", 2020-08-07T15:10:40.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6684999999999998e-05 2020-08-07T15:10:40.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:40.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205326+0000", 2020-08-07T15:10:40.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6200000000000004e-06 2020-08-07T15:10:40.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:40.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205332+0000", 2020-08-07T15:10:40.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98984944600000002 2020-08-07T15:10:40.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:40.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195181+0000", 2020-08-07T15:10:40.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4259999999999997e-06 2020-08-07T15:10:40.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:40.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195188+0000", 2020-08-07T15:10:40.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.989964548 2020-08-07T15:10:40.413 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:40.413 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:40.414 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:40.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.414 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:128743 4.9 4.1824cba9 (undecoded) ondisk+write+known_if_redirected e155)", 2020-08-07T15:10:40.415 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:03.205500+0000", 2020-08-07T15:10:40.415 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.229259956999996, 2020-08-07T15:10:40.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98971761999999996, 2020-08-07T15:10:40.415 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:40.415 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:40.416 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:40.416 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:40.416 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:40.417 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128743 2020-08-07T15:10:40.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.417 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:40.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:40.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205500+0000", 2020-08-07T15:10:40.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:40.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:40.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205500+0000", 2020-08-07T15:10:40.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6441000000000001e-05 2020-08-07T15:10:40.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:40.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205516+0000", 2020-08-07T15:10:40.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4429999999999999e-06 2020-08-07T15:10:40.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:40.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205520+0000", 2020-08-07T15:10:40.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4590000000000003e-06 2020-08-07T15:10:40.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:40.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205526+0000", 2020-08-07T15:10:40.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0429999999999999e-06 2020-08-07T15:10:40.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:40.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205529+0000", 2020-08-07T15:10:40.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1313000000000003e-05 2020-08-07T15:10:40.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:40.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205560+0000", 2020-08-07T15:10:40.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.934e-06 2020-08-07T15:10:40.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:10:40.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205567+0000", 2020-08-07T15:10:40.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98964017599999998 2020-08-07T15:10:40.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:40.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195208+0000", 2020-08-07T15:10:40.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8109999999999997e-06 2020-08-07T15:10:40.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:40.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:40.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:40.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195217+0000", 2020-08-07T15:10:40.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98971761999999996 2020-08-07T15:10:40.441 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:40.441 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:40.441 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:40.443 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:40.443 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:40.444 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:10:40.450 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:10:40.450 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:10:40.451 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:10:40.451 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:10:40.451 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:10:40.452 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:10:40.452 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:10:40.452 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:10:40.453 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 7 1 2020-08-07T15:10:40.454 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:10:40.454 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:10:40.454 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:10:40.454 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:10:40.455 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:10:40.455 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:10:40.455 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:10:40.455 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:10:40.469 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:2020-08-07T15:10:40.210814+0000 min lat: 0.00100396 max lat: 0.396108 avg lat: 0.00988605 2020-08-07T15:10:40.469 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-08-07T15:10:40.470 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 20 16 32370 32354 12.6372 13.0547 0.00678468 0.00988605 2020-08-07T15:10:40.475 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:10:40.475 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:40.446+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:40.477 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:10:40.564 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:40.562+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:40.658 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:40.651+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:40.663 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:40.663 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:40.679 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:10:40.758 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:40.753+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:40.833 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:40.854 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:40.863 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:10:40.864 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:40.861+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:40.972 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:40.966+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:41.075 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:41.070+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:41.085 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:41.086 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:41.106 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:10:41.178 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:41.173+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:41.215 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 21 16 33909 33893 12.608 12.0234 0.00739357 0.00989231 2020-08-07T15:10:41.291 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:41.281+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:41.310 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:41.311 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:41.320 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:10:41.389 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:41.384+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:41.454 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.7 to 1 (10000) 2020-08-07T15:10:41.468 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2020-08-07T15:10:41.469 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:10:41.501 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:41.494+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:41.519 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:41.527 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:41.537 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:10:41.625 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:41.623+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:41.700 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:41.697+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:41.744 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:41.747 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:41.755 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:10:41.799 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:41.798+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:41.901 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:41.899+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:41.942 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:10:41.943 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:10:41.971 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:41.971 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:41.972 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2020-08-07T15:10:41.446655+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":220069888,"num_objects":3361,"num_object_clones":0,"num_object_copies":10083,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26864,"num_write_kb":214912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":135,"acting":135,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9027276,"kb_used_data":638604,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":745947444,"statfs":{"total":773094113280,"available":763850182656,"internally_reserved":8589934592,"allocated":653930496,"data_stored":648598794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8388608,"num_objects":128,"num_object_clones":0,"num_object_copies":384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1024,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.037763"},"pg_stats":[{"pgid":"1.14","version":"16'2","reported_seq":"109","reported_epoch":"176","state":"active+clean","last_fresh":"2020-08-07T15:10:35.402334+0000","last_change":"2020-08-07T15:10:10.644555+0000","last_active":"2020-08-07T15:10:35.402334+0000","last_peered":"2020-08-07T15:10:35.402334+0000","last_clean":"2020-08-07T15:10:35.402334+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:10:35.402334+0000","last_undegraded":"2020-08-07T15:10:35.402334+0000","last_fullsized":"2020-08-07T15:10:35.402334+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:10.644460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.644460+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"178","reported_epoch":"176","state":"active+clean","last_fresh":"2020-08-07T15:10:35.400930+0000","last_change":"2020-08-07T15:10:07.295971+0000","last_active":"2020-08-07T15:10:35.400930+0000","last_peered":"2020-08-07T15:10:35.400930+0000","last_clean":"2020-08-07T15:10:35.400930+0000","last_became_active":"2020-08-07T15:09:30.203592+0000","last_became_peered":"2020-08-07T15:09:30.203592+0000","last_unstale":"2020-08-07T15:10:35.400930+0000","last_undegraded":"2020-08-07T15:10:35.400930+0000","last_fullsized":"2020-08-07T15:10:35.400930+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:07.295863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:07.295863+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"109","reported_epoch":"176","state":"active+clean","last_fresh":"2020-08-07T15:10:35.391420+0000","last_change":"2020-08-07T15:10:28.734269+0000","last_active":"2020-08-07T15:10:35.391420+0000","last_peered":"2020-08-07T15:10:35.391420+0000","last_clean":"2020-08-07T15:10:35.391420+0000","last_became_active":"2020-08-07T15:08:41.685106+0000","last_became_peered":"2020-08-07T15:08:41.685106+0000","last_unstale":"2020-08-07T15:10:35.391420+0000","last_undegraded":"2020-08-07T15:10:35.391420+0000","last_fullsized":"2020-08-07T15:10:35.391420+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:28.734193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:28.734193+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"222","reported_epoch":"176","state":"active+clean","last_fresh":"2020-08-07T15:10:35.391421+0000","last_change":"2020-08-07T15:10:24.690028+0000","last_active":"2020-08-07T15:10:35.391421+0000","last_peered":"2020-08-07T15:10:35.391421+0000","last_clean":"2020-08-07T15:10:35.391421+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:10:35.391421+0000","last_undegraded":"2020-08-07T15:10:35.391421+0000","last_fullsized":"2020-08-07T15:10:35.391421+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:24.689950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:24.689950+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"216","reported_epoch":"176","state":"active+clean","last_fresh":"2020-08-07T15:10:35.391411+0000","last_change":"2020-08-07T15:10:26.631030+0000","last_active":"2020-08-07T15:10:35.391411+0000","last_peered":"2020-08-07T15:10:35.391411+0000","last_clean":"2020-08-07T15:10:35.391411+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:10:35.391411+0000","last_undegraded":"2020-08-07T15:10:35.391411+0000","last_fullsized":"2020-08-07T15:10:35.391411+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:26.630945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:26.630945+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.0","version":"177'1448","reported_seq":"1469","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.535058+0000","last_change":"2020-08-07T15:10:16.261122+0000","last_active":"2020-08-07T15:10:36.535058+0000","last_peered":"2020-08-07T15:10:36.535058+0000","last_clean":"2020-08-07T15:10:36.535058+0000","last_became_active":"2020-08-07T15:10:16.260690+0000","last_became_peered":"2020-08-07T15:10:16.260690+0000","last_unstale":"2020-08-07T15:10:36.535058+0000","last_undegraded":"2020-08-07T15:10:36.535058+0000","last_fullsized":"2020-08-07T15:10:36.535058+0000","mapping_epoch":164,"log_start":"177'1441","ondisk_log_start":"177'1441","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11862016,"num_objects":181,"num_object_clones":0,"num_object_copies":543,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1448,"num_write_kb":11584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"174","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.401143+0000","last_change":"2020-08-07T15:09:41.284321+0000","last_active":"2020-08-07T15:10:36.401143+0000","last_peered":"2020-08-07T15:10:36.401143+0000","last_clean":"2020-08-07T15:10:36.401143+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:10:36.401143+0000","last_undegraded":"2020-08-07T15:10:36.401143+0000","last_fullsized":"2020-08-07T15:10:36.401143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:41.284237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:41.284237+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"89","reported_epoch":"176","state":"active+clean","last_fresh":"2020-08-07T15:10:35.425969+0000","last_change":"2020-08-07T15:09:30.283753+0000","last_active":"2020-08-07T15:10:35.425969+0000","last_peered":"2020-08-07T15:10:35.425969+0000","last_clean":"2020-08-07T15:10:35.425969+0000","last_became_active":"2020-08-07T15:09:30.283176+0000","last_became_peered":"2020-08-07T15:09:30.283176+0000","last_unstale":"2020-08-07T15:10:35.425969+0000","last_undegraded":"2020-08-07T15:10:35.425969+0000","last_fullsized":"2020-08-07T15:10:35.425969+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:25.606757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:25.606757+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"177'1848","reported_seq":"1869","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.668092+0000","last_change":"2020-08-07T15:10:16.259807+0000","last_active":"2020-08-07T15:10:36.668092+0000","last_peered":"2020-08-07T15:10:36.668092+0000","last_clean":"2020-08-07T15:10:36.668092+0000","last_became_active":"2020-08-07T15:10:16.259365+0000","last_became_peered":"2020-08-07T15:10:16.259365+0000","last_unstale":"2020-08-07T15:10:36.668092+0000","last_undegraded":"2020-08-07T15:10:36.668092+0000","last_fullsized":"2020-08-07T15:10:36.668092+0000","mapping_epoch":164,"log_start":"177'1840","ondisk_log_start":"177'1840","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":15138816,"num_objects":231,"num_object_clones":0,"num_object_copies":693,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1848,"num_write_kb":14784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"176","reported_epoch":"176","state":"active+clean","last_fresh":"2020-08-07T15:10:35.400854+0000","last_change":"2020-08-07T15:10:25.463712+0000","last_active":"2020-08-07T15:10:35.400854+0000","last_peered":"2020-08-07T15:10:35.400854+0000","last_clean":"2020-08-07T15:10:35.400854+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:10:35.400854+0000","last_undegraded":"2020-08-07T15:10:35.400854+0000","last_fullsized":"2020-08-07T15:10:35.400854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:25.463628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:10:25.463628+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"134","reported_epoch":"176","state":"active+clean","last_fresh":"2020-08-07T15:10:35.402482+0000","last_change":"2020-08-07T15:10:27.642227+0000","last_active":"2020-08-07T15:10:35.402482+0000","last_peered":"2020-08-07T15:10:35.402482+0000","last_clean":"2020-08-07T15:10:35.402482+0000","last_became_active":"2020-08-07T15:08:46.710404+0000","last_became_peered":"2020-08-07T15:08:46.710404+0000","last_unstale":"2020-08-07T15:10:35.402482+0000","last_undegraded":"2020-08-07T15:10:35.402482+0000","last_fullsized":"2020-08-07T15:10:35.402482+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:10:27.642141+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:10:27.642141+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"178'1808","reported_seq":"1830","reported_epoch":"178","state":"active+clean","last_fresh":"2020-08-07T15:10:37.620364+0000","last_change":"2020-08-07T15:10:16.448360+0000","last_active":"2020-08-07T15:10:37.620364+0000","last_peered":"2020-08-07T15:10:37.620364+0000","last_clean":"2020-08-07T15:10:37.620364+0000","last_became_active":"2020-08-07T15:10:16.447826+0000","last_became_peered":"2020-08-07T15:10:16.447826+0000","last_unstale":"2020-08-07T15:10:37.620364+0000","last_undegraded":"2020-08-07T15:10:37.620364+0000","last_fullsized":"2020-08-07T15:10:37.620364+0000","mapping_epoch":164,"log_start":"178'1800","ondisk_log_start":"178'1800","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":14811136,"num_objects":226,"num_object_clones":0,"num_object_copies":678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1808,"num_write_kb":14464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"174","reported_epoch":"176","state":"active+clean","last_fresh":"2020-08-07T15:10:35.435803+0000","last_change":"2020-08-07T15:09:54.908283+0000","last_active":"2020-08-07T15:10:35.435803+0000","last_peered":"2020-08-07T15:10:35.435803+0000","last_clean":"2020-08-07T15:10:35.435803+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:10:35.435803+0000","last_undegraded":"2020-08-07T15:10:35.435803+0000","last_fullsized":"2020-08-07T15:10:35.435803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:54.908197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:54.908197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"213","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.400956+0000","last_change":"2020-08-07T15:10:07.302258+0000","last_active":"2020-08-07T15:10:36.400956+0000","last_peered":"2020-08-07T15:10:36.400956+0000","last_clean":"2020-08-07T15:10:36.400956+0000","last_became_active":"2020-08-07T15:09:30.201829+0000","last_became_peered":"2020-08-07T15:09:30.201829+0000","last_unstale":"2020-08-07T15:10:36.400956+0000","last_undegraded":"2020-08-07T15:10:36.400956+0000","last_fullsized":"2020-08-07T15:10:36.400956+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:07.302204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:07.302204+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"177'1840","reported_seq":"1860","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.722822+0000","last_change":"2020-08-07T15:10:16.447819+0000","last_active":"2020-08-07T15:10:36.722822+0000","last_peered":"2020-08-07T15:10:36.722822+0000","last_clean":"2020-08-07T15:10:36.722822+0000","last_became_active":"2020-08-07T15:10:16.447312+0000","last_became_peered":"2020-08-07T15:10:16.447312+0000","last_unstale":"2020-08-07T15:10:36.722822+0000","last_undegraded":"2020-08-07T15:10:36.722822+0000","last_fullsized":"2020-08-07T15:10:36.722822+0000","mapping_epoch":164,"log_start":"177'1832","ondisk_log_start":"177'1832","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":15073280,"num_objects":230,"num_object_clones":0,"num_object_copies":690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1840,"num_write_kb":14720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"174","reported_epoch":"176","state":"active+clean","last_fresh":"2020-08-07T15:10:35.435354+0000","last_change":"2020-08-07T15:09:51.036281+0000","last_active":"2020-08-07T15:10:35.435354+0000","last_peered":"2020-08-07T15:10:35.435354+0000","last_clean":"2020-08-07T15:10:35.435354+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:10:35.435354+0000","last_undegraded":"2020-08-07T15:10:35.435354+0000","last_fullsized":"2020-08-07T15:10:35.435354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:51.036224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:51.036224+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"179","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.400704+0000","last_change":"2020-08-07T15:10:23.400748+0000","last_active":"2020-08-07T15:10:36.400704+0000","last_peered":"2020-08-07T15:10:36.400704+0000","last_clean":"2020-08-07T15:10:36.400704+0000","last_became_active":"2020-08-07T15:08:36.633406+0000","last_became_peered":"2020-08-07T15:08:36.633406+0000","last_unstale":"2020-08-07T15:10:36.400704+0000","last_undegraded":"2020-08-07T15:10:36.400704+0000","last_fullsized":"2020-08-07T15:10:36.400704+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:23.400672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:23.400672+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"177'1496","reported_seq":"1516","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.678454+0000","last_change":"2020-08-07T15:10:16.255436+0000","last_active":"2020-08-07T15:10:36.678454+0000","last_peered":"2020-08-07T15:10:36.678454+0000","last_clean":"2020-08-07T15:10:36.678454+0000","last_became_active":"2020-08-07T15:10:16.254967+0000","last_became_peered":"2020-08-07T15:10:16.254967+0000","last_unstale":"2020-08-07T15:10:36.678454+0000","last_undegraded":"2020-08-07T15:10:36.678454+0000","last_fullsized":"2020-08-07T15:10:36.678454+0000","mapping_epoch":164,"log_start":"177'1490","ondisk_log_start":"177'1490","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12255232,"num_objects":187,"num_object_clones":0,"num_object_copies":561,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1496,"num_write_kb":11968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"206","reported_epoch":"176","state":"active+clean","last_fresh":"2020-08-07T15:10:35.435309+0000","last_change":"2020-08-07T15:09:55.879867+0000","last_active":"2020-08-07T15:10:35.435309+0000","last_peered":"2020-08-07T15:10:35.435309+0000","last_clean":"2020-08-07T15:10:35.435309+0000","last_became_active":"2020-08-07T15:08:36.627426+0000","last_became_peered":"2020-08-07T15:08:36.627426+0000","last_unstale":"2020-08-07T15:10:35.435309+0000","last_undegraded":"2020-08-07T15:10:35.435309+0000","last_fullsized":"2020-08-07T15:10:35.435309+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:55.879785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:55.879785+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"174","reported_epoch":"176","state":"active+clean","last_fresh":"2020-08-07T15:10:35.391658+0000","last_change":"2020-08-07T15:10:08.656952+0000","last_active":"2020-08-07T15:10:35.391658+0000","last_peered":"2020-08-07T15:10:35.391658+0000","last_clean":"2020-08-07T15:10:35.391658+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:10:35.391658+0000","last_undegraded":"2020-08-07T15:10:35.391658+0000","last_fullsized":"2020-08-07T15:10:35.391658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:08.656855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:10:08.656855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"177'1648","reported_seq":"1669","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.599716+0000","last_change":"2020-08-07T15:10:16.263356+0000","last_active":"2020-08-07T15:10:36.599716+0000","last_peered":"2020-08-07T15:10:36.599716+0000","last_clean":"2020-08-07T15:10:36.599716+0000","last_became_active":"2020-08-07T15:10:16.262971+0000","last_became_peered":"2020-08-07T15:10:16.262971+0000","last_unstale":"2020-08-07T15:10:36.599716+0000","last_undegraded":"2020-08-07T15:10:36.599716+0000","last_fullsized":"2020-08-07T15:10:36.599716+0000","mapping_epoch":164,"log_start":"177'1643","ondisk_log_start":"177'1643","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13500416,"num_objects":206,"num_object_clones":0,"num_object_copies":618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1648,"num_write_kb":13184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"195","reported_epoch":"176","state":"active+clean","last_fresh":"2020-08-07T15:10:35.402351+0000","last_change":"2020-08-07T15:10:24.576099+0000","last_active":"2020-08-07T15:10:35.402351+0000","last_peered":"2020-08-07T15:10:35.402351+0000","last_clean":"2020-08-07T15:10:35.402351+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:10:35.402351+0000","last_undegraded":"2020-08-07T15:10:35.402351+0000","last_fullsized":"2020-08-07T15:10:35.402351+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:24.576023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:10:24.576023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"222","reported_epoch":"176","state":"active+clean","last_fresh":"2020-08-07T15:10:35.435469+0000","last_change":"2020-08-07T15:10:11.912614+0000","last_active":"2020-08-07T15:10:35.435469+0000","last_peered":"2020-08-07T15:10:35.435469+0000","last_clean":"2020-08-07T15:10:35.435469+0000","last_became_active":"2020-08-07T15:08:36.628365+0000","last_became_peered":"2020-08-07T15:08:36.628365+0000","last_unstale":"2020-08-07T15:10:35.435469+0000","last_undegraded":"2020-08-07T15:10:35.435469+0000","last_fullsized":"2020-08-07T15:10:35.435469+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:11.912550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:11.912550+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.5","version":"177'1598","reported_seq":"1616","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.733238+0000","last_change":"2020-08-07T15:10:16.447870+0000","last_active":"2020-08-07T15:10:36.733238+0000","last_peered":"2020-08-07T15:10:36.733238+0000","last_clean":"2020-08-07T15:10:36.733238+0000","last_became_active":"2020-08-07T15:10:16.447478+0000","last_became_peered":"2020-08-07T15:10:16.447478+0000","last_unstale":"2020-08-07T15:10:36.733238+0000","last_undegraded":"2020-08-07T15:10:36.733238+0000","last_fullsized":"2020-08-07T15:10:36.733238+0000","mapping_epoch":164,"log_start":"177'1593","ondisk_log_start":"177'1593","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13090816,"num_objects":200,"num_object_clones":0,"num_object_copies":600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1598,"num_write_kb":12784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"174","reported_epoch":"176","state":"active+clean","last_fresh":"2020-08-07T15:10:35.391637+0000","last_change":"2020-08-07T15:09:52.637867+0000","last_active":"2020-08-07T15:10:35.391637+0000","last_peered":"2020-08-07T15:10:35.391637+0000","last_clean":"2020-08-07T15:10:35.391637+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:10:35.391637+0000","last_undegraded":"2020-08-07T15:10:35.391637+0000","last_fullsized":"2020-08-07T15:10:35.391637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:52.637791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:09:52.637791+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"172","reported_epoch":"176","state":"active+clean","last_fresh":"2020-08-07T15:10:35.391432+0000","last_change":"2020-08-07T15:10:11.720217+0000","last_active":"2020-08-07T15:10:35.391432+0000","last_peered":"2020-08-07T15:10:35.391432+0000","last_clean":"2020-08-07T15:10:35.391432+0000","last_became_active":"2020-08-07T15:08:36.618164+0000","last_became_peered":"2020-08-07T15:08:36.618164+0000","last_unstale":"2020-08-07T15:10:35.391432+0000","last_undegraded":"2020-08-07T15:10:35.391432+0000","last_fullsized":"2020-08-07T15:10:35.391432+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:11.720143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:11.720143+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"178'1800","reported_seq":"1822","reported_epoch":"178","state":"active+clean","last_fresh":"2020-08-07T15:10:37.652956+0000","last_change":"2020-08-07T15:10:16.443153+0000","last_active":"2020-08-07T15:10:37.652956+0000","last_peered":"2020-08-07T15:10:37.652956+0000","last_clean":"2020-08-07T15:10:37.652956+0000","last_became_active":"2020-08-07T15:10:16.442674+0000","last_became_peered":"2020-08-07T15:10:16.442674+0000","last_unstale":"2020-08-07T15:10:37.652956+0000","last_undegraded":"2020-08-07T15:10:37.652956+0000","last_fullsized":"2020-08-07T15:10:37.652956+0000","mapping_epoch":164,"log_start":"178'1792","ondisk_log_start":"178'1792","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":14745600,"num_objects":225,"num_object_clones":0,"num_object_copies":675,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1800,"num_write_kb":14400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"171","reported_epoch":"176","state":"active+clean","last_fresh":"2020-08-07T15:10:35.435503+0000","last_change":"2020-08-07T15:09:37.017383+0000","last_active":"2020-08-07T15:10:35.435503+0000","last_peered":"2020-08-07T15:10:35.435503+0000","last_clean":"2020-08-07T15:10:35.435503+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:10:35.435503+0000","last_undegraded":"2020-08-07T15:10:35.435503+0000","last_fullsized":"2020-08-07T15:10:35.435503+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:37.017307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:37.017307+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"244","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.401366+0000","last_change":"2020-08-07T15:09:30.208873+0000","last_active":"2020-08-07T15:10:36.401366+0000","last_peered":"2020-08-07T15:10:36.401366+0000","last_clean":"2020-08-07T15:10:36.401366+0000","last_became_active":"2020-08-07T15:09:30.208345+0000","last_became_peered":"2020-08-07T15:09:30.208345+0000","last_unstale":"2020-08-07T15:10:36.401366+0000","last_undegraded":"2020-08-07T15:10:36.401366+0000","last_fullsized":"2020-08-07T15:10:36.401366+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.381325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.381325+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.e","version":"177'1688","reported_seq":"1709","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.583678+0000","last_change":"2020-08-07T15:10:16.448334+0000","last_active":"2020-08-07T15:10:36.583678+0000","last_peered":"2020-08-07T15:10:36.583678+0000","last_clean":"2020-08-07T15:10:36.583678+0000","last_became_active":"2020-08-07T15:10:16.447541+0000","last_became_peered":"2020-08-07T15:10:16.447541+0000","last_unstale":"2020-08-07T15:10:36.583678+0000","last_undegraded":"2020-08-07T15:10:36.583678+0000","last_fullsized":"2020-08-07T15:10:36.583678+0000","mapping_epoch":164,"log_start":"177'1681","ondisk_log_start":"177'1681","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13828096,"num_objects":211,"num_object_clones":0,"num_object_copies":633,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1688,"num_write_kb":13504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"149","reported_epoch":"176","state":"active+clean","last_fresh":"2020-08-07T15:10:35.391785+0000","last_change":"2020-08-07T15:10:10.715147+0000","last_active":"2020-08-07T15:10:35.391785+0000","last_peered":"2020-08-07T15:10:35.391785+0000","last_clean":"2020-08-07T15:10:35.391785+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:10:35.391785+0000","last_undegraded":"2020-08-07T15:10:35.391785+0000","last_fullsized":"2020-08-07T15:10:35.391785+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.715051+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"177'1960","reported_seq":"1981","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.666048+0000","last_change":"2020-08-07T15:10:16.448023+0000","last_active":"2020-08-07T15:10:36.666048+0000","last_peered":"2020-08-07T15:10:36.666048+0000","last_clean":"2020-08-07T15:10:36.666048+0000","last_became_active":"2020-08-07T15:10:16.447526+0000","last_became_peered":"2020-08-07T15:10:16.447526+0000","last_unstale":"2020-08-07T15:10:36.666048+0000","last_undegraded":"2020-08-07T15:10:36.666048+0000","last_fullsized":"2020-08-07T15:10:36.666048+0000","mapping_epoch":164,"log_start":"177'1953","ondisk_log_start":"177'1953","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16056320,"num_objects":245,"num_object_clones":0,"num_object_copies":735,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1960,"num_write_kb":15680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"146","reported_epoch":"176","state":"active+clean","last_fresh":"2020-08-07T15:10:35.391432+0000","last_change":"2020-08-07T15:09:09.691838+0000","last_active":"2020-08-07T15:10:35.391432+0000","last_peered":"2020-08-07T15:10:35.391432+0000","last_clean":"2020-08-07T15:10:35.391432+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:10:35.391432+0000","last_undegraded":"2020-08-07T15:10:35.391432+0000","last_fullsized":"2020-08-07T15:10:35.391432+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.691753+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.d","version":"177'1616","reported_seq":"1636","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.707687+0000","last_change":"2020-08-07T15:10:16.263289+0000","last_active":"2020-08-07T15:10:36.707687+0000","last_peered":"2020-08-07T15:10:36.707687+0000","last_clean":"2020-08-07T15:10:36.707687+0000","last_became_active":"2020-08-07T15:10:16.262869+0000","last_became_peered":"2020-08-07T15:10:16.262869+0000","last_unstale":"2020-08-07T15:10:36.707687+0000","last_undegraded":"2020-08-07T15:10:36.707687+0000","last_fullsized":"2020-08-07T15:10:36.707687+0000","mapping_epoch":164,"log_start":"177'1608","ondisk_log_start":"177'1608","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13238272,"num_objects":202,"num_object_clones":0,"num_object_copies":606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1616,"num_write_kb":12928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"213","reported_epoch":"176","state":"active+clean","last_fresh":"2020-08-07T15:10:35.435431+0000","last_change":"2020-08-07T15:10:31.122527+0000","last_active":"2020-08-07T15:10:35.435431+0000","last_peered":"2020-08-07T15:10:35.435431+0000","last_clean":"2020-08-07T15:10:35.435431+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:10:35.435431+0000","last_undegraded":"2020-08-07T15:10:35.435431+0000","last_fullsized":"2020-08-07T15:10:35.435431+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:31.122435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:31.122435+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"177'1664","reported_seq":"1685","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.721978+0000","last_change":"2020-08-07T15:10:16.256000+0000","last_active":"2020-08-07T15:10:36.721978+0000","last_peered":"2020-08-07T15:10:36.721978+0000","last_clean":"2020-08-07T15:10:36.721978+0000","last_became_active":"2020-08-07T15:10:16.255356+0000","last_became_peered":"2020-08-07T15:10:16.255356+0000","last_unstale":"2020-08-07T15:10:36.721978+0000","last_undegraded":"2020-08-07T15:10:36.721978+0000","last_fullsized":"2020-08-07T15:10:36.721978+0000","mapping_epoch":164,"log_start":"177'1659","ondisk_log_start":"177'1659","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13631488,"num_objects":208,"num_object_clones":0,"num_object_copies":624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1664,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"168","reported_epoch":"176","state":"active+clean","last_fresh":"2020-08-07T15:10:35.435699+0000","last_change":"2020-08-07T15:10:10.962565+0000","last_active":"2020-08-07T15:10:35.435699+0000","last_peered":"2020-08-07T15:10:35.435699+0000","last_clean":"2020-08-07T15:10:35.435699+0000","last_became_active":"2020-08-07T15:08:41.680448+0000","last_became_peered":"2020-08-07T15:08:41.680448+0000","last_unstale":"2020-08-07T15:10:35.435699+0000","last_undegraded":"2020-08-07T15:10:35.435699+0000","last_fullsized":"2020-08-07T15:10:35.435699+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:10:10.962480+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.962480+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"177'1648","reported_seq":"1669","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.690710+0000","last_change":"2020-08-07T15:10:16.444776+0000","last_active":"2020-08-07T15:10:36.690710+0000","last_peered":"2020-08-07T15:10:36.690710+0000","last_clean":"2020-08-07T15:10:36.690710+0000","last_became_active":"2020-08-07T15:10:16.444316+0000","last_became_peered":"2020-08-07T15:10:16.444316+0000","last_unstale":"2020-08-07T15:10:36.690710+0000","last_undegraded":"2020-08-07T15:10:36.690710+0000","last_fullsized":"2020-08-07T15:10:36.690710+0000","mapping_epoch":164,"log_start":"177'1638","ondisk_log_start":"177'1638","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13500416,"num_objects":206,"num_object_clones":0,"num_object_copies":618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1648,"num_write_kb":13184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"135","reported_epoch":"176","state":"active+clean","last_fresh":"2020-08-07T15:10:35.426131+0000","last_change":"2020-08-07T15:09:36.687393+0000","last_active":"2020-08-07T15:10:35.426131+0000","last_peered":"2020-08-07T15:10:35.426131+0000","last_clean":"2020-08-07T15:10:35.426131+0000","last_became_active":"2020-08-07T15:09:30.204655+0000","last_became_peered":"2020-08-07T15:09:30.204655+0000","last_unstale":"2020-08-07T15:10:35.426131+0000","last_undegraded":"2020-08-07T15:10:35.426131+0000","last_fullsized":"2020-08-07T15:10:35.426131+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:36.687314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:36.687314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"177'1442","reported_seq":"1460","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.691657+0000","last_change":"2020-08-07T15:10:16.444970+0000","last_active":"2020-08-07T15:10:36.691657+0000","last_peered":"2020-08-07T15:10:36.691657+0000","last_clean":"2020-08-07T15:10:36.691657+0000","last_became_active":"2020-08-07T15:10:16.444294+0000","last_became_peered":"2020-08-07T15:10:16.444294+0000","last_unstale":"2020-08-07T15:10:36.691657+0000","last_undegraded":"2020-08-07T15:10:36.691657+0000","last_fullsized":"2020-08-07T15:10:36.691657+0000","mapping_epoch":164,"log_start":"177'1438","ondisk_log_start":"177'1438","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11812864,"num_objects":181,"num_object_clones":0,"num_object_copies":543,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1442,"num_write_kb":11536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"209","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.401336+0000","last_change":"2020-08-07T15:09:08.399507+0000","last_active":"2020-08-07T15:10:36.401336+0000","last_peered":"2020-08-07T15:10:36.401336+0000","last_clean":"2020-08-07T15:10:36.401336+0000","last_became_active":"2020-08-07T15:08:41.689435+0000","last_became_peered":"2020-08-07T15:08:41.689435+0000","last_unstale":"2020-08-07T15:10:36.401336+0000","last_undegraded":"2020-08-07T15:10:36.401336+0000","last_fullsized":"2020-08-07T15:10:36.401336+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:08.399439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:08.399439+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"177'1576","reported_seq":"1597","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.504748+0000","last_change":"2020-08-07T15:10:16.256758+0000","last_active":"2020-08-07T15:10:36.504748+0000","last_peered":"2020-08-07T15:10:36.504748+0000","last_clean":"2020-08-07T15:10:36.504748+0000","last_became_active":"2020-08-07T15:10:16.256419+0000","last_became_peered":"2020-08-07T15:10:16.256419+0000","last_unstale":"2020-08-07T15:10:36.504748+0000","last_undegraded":"2020-08-07T15:10:36.504748+0000","last_fullsized":"2020-08-07T15:10:36.504748+0000","mapping_epoch":164,"log_start":"176'1568","ondisk_log_start":"176'1568","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12910592,"num_objects":197,"num_object_clones":0,"num_object_copies":591,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1576,"num_write_kb":12608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"99","reported_epoch":"176","state":"active+clean","last_fresh":"2020-08-07T15:10:35.435541+0000","last_change":"2020-08-07T15:10:29.055136+0000","last_active":"2020-08-07T15:10:35.435541+0000","last_peered":"2020-08-07T15:10:35.435541+0000","last_clean":"2020-08-07T15:10:35.435541+0000","last_became_active":"2020-08-07T15:08:46.710059+0000","last_became_peered":"2020-08-07T15:08:46.710059+0000","last_unstale":"2020-08-07T15:10:35.435541+0000","last_undegraded":"2020-08-07T15:10:35.435541+0000","last_fullsized":"2020-08-07T15:10:35.435541+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:29.055048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:29.055048+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"177'1784","reported_seq":"1805","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.704387+0000","last_change":"2020-08-07T15:10:16.253662+0000","last_active":"2020-08-07T15:10:36.704387+0000","last_peered":"2020-08-07T15:10:36.704387+0000","last_clean":"2020-08-07T15:10:36.704387+0000","last_became_active":"2020-08-07T15:10:16.253105+0000","last_became_peered":"2020-08-07T15:10:16.253105+0000","last_unstale":"2020-08-07T15:10:36.704387+0000","last_undegraded":"2020-08-07T15:10:36.704387+0000","last_fullsized":"2020-08-07T15:10:36.704387+0000","mapping_epoch":164,"log_start":"177'1775","ondisk_log_start":"177'1775","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":14614528,"num_objects":223,"num_object_clones":0,"num_object_copies":669,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1784,"num_write_kb":14272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"118","reported_epoch":"176","state":"active+clean","last_fresh":"2020-08-07T15:10:35.391817+0000","last_change":"2020-08-07T15:09:08.653773+0000","last_active":"2020-08-07T15:10:35.391817+0000","last_peered":"2020-08-07T15:10:35.391817+0000","last_clean":"2020-08-07T15:10:35.391817+0000","last_became_active":"2020-08-07T15:08:41.677944+0000","last_became_peered":"2020-08-07T15:08:41.677944+0000","last_unstale":"2020-08-07T15:10:35.391817+0000","last_undegraded":"2020-08-07T15:10:35.391817+0000","last_fullsized":"2020-08-07T15:10:35.391817+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:08.653722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:08.653722+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":220069888,"num_objects":3359,"num_object_clones":0,"num_object_copies":10077,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26864,"num_write_kb":214912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":658620416,"data_stored":658620416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":63,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542198,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1106488,"kb_used_data":57904,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93265352,"statfs":{"total":96636764160,"available":95503720448,"internally_reserved":1073741824,"allocated":59293696,"data_stored":58624399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":43.911000000000001,"5min":9.5549999999999997,"15min":3.7799999999999998},"min":{"1min":0.50800000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":856.07100000000003,"5min":856.07100000000003,"15min":856.07100000000003},"last":0.42199999999999999},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.026,"15min":0.98399999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.8899999999999999,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":0.79400000000000004}]},{"osd":1,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.335,"15min":1.3080000000000001},"min":{"1min":0.64200000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.9199999999999999,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":2.395},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.141,"15min":1.024},"min":{"1min":0.61799999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":4.0060000000000002,"5min":4.0060000000000002,"15min":4.0060000000000002},"last":2.4950000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.304,"15min":1.1859999999999999},"min":{"1min":0.747,"5min":0.628,"15min":0.628},"max":{"1min":2.0249999999999999,"5min":5.6619999999999999,"15min":5.6619999999999999},"last":2.4359999999999999},{"interface":"front","average":{"1min":1.512,"5min":1.3380000000000001,"15min":1.1910000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.6619999999999999,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":0.54300000000000004}]},{"osd":3,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":16.18,"15min":6.508},"min":{"1min":0.64700000000000002,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.9710000000000001,"5min":917.38699999999994,"15min":917.38699999999994},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.196,"5min":4.9649999999999999,"15min":2.5499999999999998},"min":{"1min":0.77200000000000002,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.3359999999999999,"5min":333.59199999999998,"15min":333.59199999999998},"last":0.84799999999999998}]},{"osd":4,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.464,"15min":1.3380000000000001},"min":{"1min":0.63400000000000001,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":3.9750000000000001,"5min":4.835,"15min":4.835},"last":0.65400000000000003},{"interface":"front","average":{"1min":1.306,"5min":11.518000000000001,"15min":4.7140000000000004},"min":{"1min":0.69699999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":3.0329999999999999,"5min":866.24099999999999,"15min":866.24099999999999},"last":0.92500000000000004}]},{"osd":5,"last update":"Fri Aug 7 15:10:27 2020","interfaces":[{"interface":"back","average":{"1min":39.496000000000002,"5min":23.838000000000001,"15min":32.950000000000003},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":678.76099999999997,"5min":831.88099999999997,"15min":831.88099999999997},"last":2.468},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.823,"15min":1.6040000000000001},"min":{"1min":0.84999999999999998,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.4060000000000001,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":2.5219999999999998}]},{"osd":7,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":3.9929999999999999,"15min":2.343},"min":{"1min":0.89900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":3.0049999999999999,"5min":224.14500000000001,"15min":224.14500000000001},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.6899999999999999,"15min":1.5680000000000001},"min":{"1min":1.113,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.2919999999999998,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":3.097}]}]},{"osd":7,"up_from":13,"seq":55834574903,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1113388,"kb_used_data":64804,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93258452,"statfs":{"total":96636764160,"available":95496654848,"internally_reserved":1073741824,"allocated":66359296,"data_stored":65697557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1459999999999999,"15min":1.27},"min":{"1min":0.58099999999999996,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":3.2090000000000001,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":30.457000000000001,"5min":9.8149999999999995,"15min":4.0380000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":529.88699999999994,"5min":529.88699999999994,"15min":529.88699999999994},"last":0.63600000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":5.758,"15min":2.5579999999999998},"min":{"1min":0.47699999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.7290000000000001,"5min":547.65300000000002,"15min":547.65300000000002},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.4159999999999999,"5min":4.9189999999999996,"15min":2.5640000000000001},"min":{"1min":0.57399999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":5.4960000000000004,"5min":372.50799999999998,"15min":372.50799999999998},"last":1.101}]},{"osd":2,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.262,"15min":1.2170000000000001},"min":{"1min":0.56799999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":3.4060000000000001,"5min":5.2400000000000002,"15min":5.2400000000000002},"last":6.968},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2170000000000001,"15min":1.2529999999999999},"min":{"1min":0.61899999999999999,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.4569999999999999,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":1.468}]},{"osd":3,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":42.097999999999999,"5min":9.5510000000000002,"15min":4.2009999999999996},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":698.19899999999996,"5min":698.19899999999996,"15min":698.19899999999996},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.3160000000000001,"15min":1.3640000000000001},"min":{"1min":0.80500000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.8959999999999999,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":0.92100000000000004}]},{"osd":4,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.3979999999999999,"15min":1.5740000000000001},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.885,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.353,"5min":1.5620000000000001,"15min":1.635},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":2.0699999999999998,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":0.83399999999999996}]},{"osd":5,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.2589999999999999,"15min":1.329},"min":{"1min":0.66500000000000004,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":3.0009999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.749},{"interface":"front","average":{"1min":1.298,"5min":1.532,"15min":1.6879999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.9419999999999999,"5min":5.327,"15min":5.327},"last":1.03}]},{"osd":6,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.466,"15min":1.444},"min":{"1min":0.93400000000000005,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":3.2229999999999999,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":0.998},{"interface":"front","average":{"1min":1.4970000000000001,"5min":8.2379999999999995,"15min":14.962999999999999},"min":{"1min":0.88700000000000001,"5min":0.87,"15min":0.87},"max":{"1min":2.3330000000000002,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.0129999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1116204,"kb_used_data":67620,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93255636,"statfs":{"total":96636764160,"available":95493771264,"internally_reserved":1073741824,"allocated":69242880,"data_stored":68581141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":13.122,"5min":16.731999999999999,"15min":28.367999999999999},"min":{"1min":0.47199999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":211.63399999999999,"5min":497.04000000000002,"15min":497.04000000000002},"last":1.179},{"interface":"front","average":{"1min":1.645,"5min":14.757999999999999,"15min":5.673},"min":{"1min":0.57399999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":8.8849999999999998,"5min":993.84500000000003,"15min":993.84500000000003},"last":1.6020000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":54.984000000000002,"5min":11.816000000000001,"15min":4.5380000000000003},"min":{"1min":0.57199999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":959.36900000000003,"5min":959.36900000000003,"15min":959.36900000000003},"last":1.6279999999999999},{"interface":"front","average":{"1min":1.72,"5min":1.357,"15min":1.2549999999999999},"min":{"1min":0.53600000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":5.4530000000000003,"5min":6.4029999999999996,"15min":6.4029999999999996},"last":1.407}]},{"osd":3,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":9.2420000000000009,"15min":3.9540000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.3879999999999999,"5min":828.31600000000003,"15min":828.31600000000003},"last":1.054},{"interface":"front","average":{"1min":1.2130000000000001,"5min":10.119999999999999,"15min":4.3419999999999996},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.5659999999999998,"5min":905.38300000000004,"15min":905.38300000000004},"last":1.069}]},{"osd":4,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":49.530999999999999,"5min":16.422999999999998,"15min":6.3300000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":872.58500000000004,"5min":872.58500000000004,"15min":872.58500000000004},"last":1.032},{"interface":"front","average":{"1min":2.2080000000000002,"5min":1.5409999999999999,"15min":1.3129999999999999},"min":{"1min":0.75800000000000001,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":12.640000000000001,"5min":12.640000000000001,"15min":12.640000000000001},"last":1.3759999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":6.0919999999999996,"15min":10.702},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.056,"5min":213.63800000000001,"15min":213.63800000000001},"last":1.4319999999999999},{"interface":"front","average":{"1min":2.4630000000000001,"5min":12.974,"15min":5.1180000000000003},"min":{"1min":0.93400000000000005,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":12.677,"5min":875.44600000000003,"15min":875.44600000000003},"last":1.4710000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:10:32 2020","interfaces":[{"interface":"back","average":{"1min":2.9340000000000002,"5min":1.863,"15min":1.655},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":22.402000000000001,"5min":22.402000000000001,"15min":22.402000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":1.3160000000000001,"5min":3.589,"15min":2.1030000000000002},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.3980000000000001,"5min":207.137,"15min":207.137},"last":1.524}]},{"osd":7,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":2.0910000000000002,"5min":1.7829999999999999,"15min":1.671},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":5.4960000000000004,"5min":6.444,"15min":6.444},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.655,"15min":1.538},"min":{"1min":0.78700000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.0219999999999998,"5min":10.752000000000001,"15min":10.752000000000001},"last":1.5569999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574903,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1151436,"kb_used_data":102852,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93220404,"statfs":{"total":96636764160,"available":95457693696,"internally_reserved":1073741824,"allocated":105320448,"data_stored":104658709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":24.291,"5min":21.719999999999999,"15min":7.891},"min":{"1min":0.52700000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":432.899,"5min":845.57100000000003,"15min":845.57100000000003},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.752,"5min":1.181,"15min":1.0149999999999999},"min":{"1min":0.62,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":9.3650000000000002,"5min":9.3650000000000002,"15min":9.3650000000000002},"last":0.95299999999999996}]},{"osd":2,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.0249999999999999,"15min":0.95999999999999996},"min":{"1min":0.52200000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.972,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.296,"5min":16.190999999999999,"15min":27.222999999999999},"min":{"1min":0.55900000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.8519999999999999,"5min":606.18499999999995,"15min":606.18499999999995},"last":1.5589999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":1.7809999999999999,"5min":1.2350000000000001,"15min":1.0349999999999999},"min":{"1min":0.54300000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":9.3360000000000003,"5min":9.3360000000000003,"15min":9.3360000000000003},"last":2.298},{"interface":"front","average":{"1min":1.677,"5min":1.4179999999999999,"15min":1.2509999999999999},"min":{"1min":0.63600000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":11.19,"5min":11.19,"15min":11.19},"last":1.917}]},{"osd":4,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.355,"15min":1.2290000000000001},"min":{"1min":0.68300000000000005,"5min":0.63,"15min":0.63},"max":{"1min":3.9729999999999999,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":1.7589999999999999},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.4510000000000001,"15min":1.3220000000000001},"min":{"1min":0.83899999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.012,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":1.887}]},{"osd":5,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":9.0800000000000001,"15min":3.8439999999999999},"min":{"1min":0.75600000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":3.883,"5min":501.267,"15min":501.267},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.669,"5min":1.3740000000000001,"15min":1.1950000000000001},"min":{"1min":0.752,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":4.0140000000000002,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":1.9590000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":4.3310000000000004,"15min":7.1859999999999999},"min":{"1min":0.84999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.1400000000000001,"5min":124.738,"15min":124.738},"last":1.853},{"interface":"front","average":{"1min":1.611,"5min":8.4019999999999992,"15min":15.236000000000001},"min":{"1min":0.95899999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":3.9119999999999999,"5min":294.94,"15min":294.94},"last":1.593}]},{"osd":7,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":1.647,"5min":1.571,"15min":1.581},"min":{"1min":0.98899999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":3.9369999999999998,"5min":6.468,"15min":6.468},"last":2.0169999999999999},{"interface":"front","average":{"1min":46.985999999999997,"5min":10.561,"15min":4.444},"min":{"1min":0.88200000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":859.82399999999996,"5min":859.82399999999996,"15min":859.82399999999996},"last":1.615}]}]},{"osd":2,"up_from":13,"seq":55834574902,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1124984,"kb_used_data":76400,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93246856,"statfs":{"total":96636764160,"available":95484780544,"internally_reserved":1073741824,"allocated":78233600,"data_stored":77564303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:10:21 2020","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.268,"15min":1.167},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":3.706,"5min":8.4930000000000003,"15min":8.4930000000000003},"last":0.498},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.1910000000000001,"15min":1.0469999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":5.194,"5min":5.194,"15min":5.194},"last":0.65300000000000002}]},{"osd":1,"last update":"Fri Aug 7 15:10:21 2020","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.6970000000000001,"15min":1.2929999999999999},"min":{"1min":0.52000000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":5.5919999999999996,"5min":18.672000000000001,"15min":18.672000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.264,"5min":1.3320000000000001,"15min":1.3500000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":3.7970000000000002,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":1.0700000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:10:21 2020","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.2949999999999999,"15min":1.2070000000000001},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":3.1970000000000001,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":1.796,"5min":8.0990000000000002,"15min":3.7370000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":7.9349999999999996,"5min":501.75599999999997,"15min":501.75599999999997},"last":0.61299999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:10:21 2020","interfaces":[{"interface":"back","average":{"1min":1.6379999999999999,"5min":1.508,"15min":1.242},"min":{"1min":0.64500000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":3.9409999999999998,"5min":19.783000000000001,"15min":19.783000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.6020000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":3.7919999999999998,"5min":3.7919999999999998,"15min":3.7919999999999998},"last":0.92000000000000004}]},{"osd":5,"last update":"Fri Aug 7 15:10:21 2020","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.3520000000000001,"15min":1.292},"min":{"1min":0.80900000000000005,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":5.2220000000000004,"5min":5.2220000000000004,"15min":5.2220000000000004},"last":0.82799999999999996},{"interface":"front","average":{"1min":30.856000000000002,"5min":25.364999999999998,"15min":25.471},"min":{"1min":0.83699999999999997,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":584.69000000000005,"5min":627.53300000000002,"15min":627.53300000000002},"last":0.88500000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:10:32 2020","interfaces":[{"interface":"back","average":{"1min":1.8,"5min":1.6539999999999999,"15min":1.365},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":4.1639999999999997,"5min":20.035,"15min":20.035},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.6459999999999999,"5min":9.9629999999999992,"15min":4.2009999999999996},"min":{"1min":0.95499999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":5.3019999999999996,"5min":751.70000000000005,"15min":751.70000000000005},"last":0.98999999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:10:21 2020","interfaces":[{"interface":"back","average":{"1min":1.6140000000000001,"5min":12.569000000000001,"15min":5.351},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":3.7709999999999999,"5min":501.91699999999997,"15min":501.91699999999997},"last":1.768},{"interface":"front","average":{"1min":1.6639999999999999,"5min":1.9530000000000001,"15min":1.6479999999999999},"min":{"1min":0.92900000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":5.7220000000000004,"5min":35.646999999999998,"15min":35.646999999999998},"last":1.2010000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1088568,"kb_used_data":39984,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93283272,"statfs":{"total":96636764160,"available":95522070528,"internally_reserved":1073741824,"allocated":40943616,"data_stored":40274319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.3,"15min":1.268},"min":{"1min":0.498,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":3.617,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.369,"15min":1.5469999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":3.6520000000000001,"5min":13.067,"15min":13.067},"last":1.405}]},{"osd":1,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":4.4779999999999998,"15min":2.758},"min":{"1min":0.59099999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.847,"5min":270.68700000000001,"15min":270.68700000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.109,"5min":1.8580000000000001,"15min":1.96},"min":{"1min":0.54100000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.0550000000000002,"5min":14.349,"15min":14.349},"last":2.0680000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.3169999999999999,"15min":1.204},"min":{"1min":0.56999999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.6930000000000001,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":1.512},{"interface":"front","average":{"1min":1.2629999999999999,"5min":2.024,"15min":1.798},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.9420000000000002,"5min":51.530000000000001,"15min":51.530000000000001},"last":2.0899999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":35.152000000000001,"5min":17.465,"15min":6.9530000000000003},"min":{"1min":0.64100000000000001,"5min":0.62,"15min":0.62},"max":{"1min":646.26700000000005,"5min":646.26700000000005,"15min":646.26700000000005},"last":1.143},{"interface":"front","average":{"1min":1.5269999999999999,"5min":19.353999999999999,"15min":27.850999999999999},"min":{"1min":0.69699999999999995,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":3.911,"5min":677.88599999999997,"15min":677.88599999999997},"last":1.343}]},{"osd":5,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":23.408000000000001,"5min":5.9699999999999998,"15min":3.0019999999999998},"min":{"1min":0.90000000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":403.88400000000001,"5min":403.88400000000001,"15min":403.88400000000001},"last":2.2040000000000002},{"interface":"front","average":{"1min":2.4209999999999998,"5min":1.931,"15min":1.823},"min":{"1min":1.0289999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":8.9120000000000008,"5min":8.9120000000000008,"15min":8.9120000000000008},"last":2.2330000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.6930000000000001,"5min":1.659,"15min":1.5740000000000001},"min":{"1min":0.79100000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":3.9500000000000002,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":1.379},{"interface":"front","average":{"1min":39.466999999999999,"5min":13.435,"15min":5.8029999999999999},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":688.15999999999997,"5min":688.15999999999997,"15min":688.15999999999997},"last":2.0089999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.7809999999999999,"15min":2.0350000000000001},"min":{"1min":0.91000000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.2410000000000001,"5min":5.734,"15min":5.734},"last":2.044},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.857,"15min":1.9370000000000001},"min":{"1min":0.84499999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.4140000000000001,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":1.1699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574902,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1158072,"kb_used_data":109488,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93213768,"statfs":{"total":96636764160,"available":95450898432,"internally_reserved":1073741824,"allocated":112115712,"data_stored":111446415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:10:24 2020","interfaces":[{"interface":"back","average":{"1min":26.023,"5min":6.04,"15min":2.7999999999999998},"min":{"1min":0.48299999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":399.73700000000002,"5min":399.73700000000002,"15min":399.73700000000002},"last":0.70099999999999996},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":0.91500000000000004},"min":{"1min":0.54600000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":2.665,"5min":6.3970000000000002,"15min":6.3970000000000002},"last":0.54300000000000004}]},{"osd":1,"last update":"Fri Aug 7 15:10:24 2020","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":5.7629999999999999,"15min":2.823},"min":{"1min":0.52900000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.8109999999999999,"5min":320.67000000000002,"15min":320.67000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.177,"5min":7.5300000000000002,"15min":3.4279999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.7200000000000002,"5min":689.846,"15min":689.846},"last":0.89600000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:10:24 2020","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.4019999999999999,"15min":1.4350000000000001},"min":{"1min":0.76600000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":3.4830000000000001,"5min":6.8730000000000002,"15min":6.8730000000000002},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.556,"5min":1.524,"15min":1.518},"min":{"1min":0.88600000000000001,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":3.5659999999999998,"5min":7.1360000000000001,"15min":7.1360000000000001},"last":1.0580000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:10:24 2020","interfaces":[{"interface":"back","average":{"1min":43.743000000000002,"5min":12.128,"15min":5.048},"min":{"1min":0.82399999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":638.92100000000005,"5min":638.92100000000005,"15min":638.92100000000005},"last":1.282},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.367,"15min":1.2889999999999999},"min":{"1min":0.75800000000000001,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.1200000000000001,"5min":6.431,"15min":6.431},"last":1.319}]},{"osd":5,"last update":"Fri Aug 7 15:10:24 2020","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":10.446,"15min":4.5670000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.2559999999999998,"5min":990.71600000000001,"15min":990.71600000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":1.204,"5min":1.526,"15min":1.593},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.669,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":1.3580000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:10:32 2020","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.661,"15min":1.7470000000000001},"min":{"1min":0.73399999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.7000000000000002,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":0.87},{"interface":"front","average":{"1min":1.282,"5min":8.5039999999999996,"15min":15.574},"min":{"1min":0.93700000000000006,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.0870000000000002,"5min":389.14600000000002,"15min":389.14600000000002},"last":1.391}]},{"osd":7,"last update":"Fri Aug 7 15:10:24 2020","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":3.8079999999999998,"15min":2.2080000000000002},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.7290000000000001,"5min":252.798,"15min":252.798},"last":1.0249999999999999},{"interface":"front","average":{"1min":21.542000000000002,"5min":14.154,"15min":5.8150000000000004},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":326.327,"5min":501.93000000000001,"15min":501.93000000000001},"last":0.95699999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574902,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1168136,"kb_used_data":119552,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93203704,"statfs":{"total":96636764160,"available":95440592896,"internally_reserved":1073741824,"allocated":122421248,"data_stored":121751951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.0289999999999999,"15min":1.0609999999999999},"min":{"1min":0.64000000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":3.7349999999999999,"5min":3.7349999999999999,"15min":3.7349999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94399999999999995,"15min":0.85099999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":2.0449999999999999,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.45}]},{"osd":1,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":8.5239999999999991,"5min":12.039999999999999,"15min":4.6319999999999997},"min":{"1min":0.56799999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":139.88200000000001,"5min":787.22900000000004,"15min":787.22900000000004},"last":2.052},{"interface":"front","average":{"1min":1.448,"5min":2.5649999999999999,"15min":1.593},"min":{"1min":0.59899999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":4.3700000000000001,"5min":116.548,"15min":116.548},"last":1.2569999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":4.3600000000000003,"15min":7.5999999999999996},"min":{"1min":0.77300000000000002,"5min":0.499,"15min":0.499},"max":{"1min":3.8109999999999999,"5min":168.56899999999999,"15min":168.56899999999999},"last":1.9850000000000001},{"interface":"front","average":{"1min":1.165,"5min":13.967000000000001,"15min":26.742999999999999},"min":{"1min":0.66200000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.1280000000000001,"5min":676.27099999999996,"15min":676.27099999999996},"last":1.6930000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.589,"15min":1.2749999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":4.8120000000000003,"5min":29.129000000000001,"15min":29.129000000000001},"last":2.5430000000000001},{"interface":"front","average":{"1min":1.5660000000000001,"5min":10.016999999999999,"15min":4.1470000000000002},"min":{"1min":0.76400000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":6.4189999999999996,"5min":712.98699999999997,"15min":712.98699999999997},"last":1.548}]},{"osd":4,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":10.253,"5min":32.627000000000002,"15min":44.804000000000002},"min":{"1min":0.60499999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":171.43700000000001,"5min":971.67999999999995,"15min":971.67999999999995},"last":1.52},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.2410000000000001,"15min":1.284},"min":{"1min":0.61899999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.411,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":2.0859999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":12.24,"5min":41.228000000000002,"15min":47.347999999999999},"min":{"1min":0.85099999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":208.25299999999999,"5min":966.60900000000004,"15min":966.60900000000004},"last":2.1160000000000001},{"interface":"front","average":{"1min":1.5049999999999999,"5min":16.625,"15min":22.885999999999999},"min":{"1min":0.88100000000000001,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":4.4039999999999999,"5min":517.26700000000005,"15min":517.26700000000005},"last":1.603}]},{"osd":7,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.329,"15min":1.3149999999999999},"min":{"1min":0.79200000000000004,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.0830000000000002,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.427,"5min":6.6349999999999998,"15min":3.2559999999999998},"min":{"1min":0.89600000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.7269999999999999,"5min":515.09100000000001,"15min":515.09100000000001},"last":2.1539999999999999}]}]}]}} 2020-08-07T15:10:41.994 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:10:41.996 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:10:42.008 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:42.006+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:42.112 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:42.111+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:42.198 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:42.198 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:42.212 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:10:42.214 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 22 16 35511 35495 12.6037 12.5156 0.0101402 0.00991116 2020-08-07T15:10:42.215 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:42.214+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:42.319 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:42.317+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:42.408 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:42.409 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:42.425 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:10:42.447 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:42.421+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:42.460 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:10:42.487 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:10:42.489 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2020-08-07T15:10:42.189284+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":277815296,"num_objects":4242,"num_object_clones":0,"num_object_copies":12726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33913,"num_write_kb":271304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":135,"acting":135,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9200476,"kb_used_data":811804,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":745774244,"statfs":{"total":773094113280,"available":763672825856,"internally_reserved":8589934592,"allocated":831287296,"data_stored":825698566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":57745408,"num_objects":881,"num_object_clones":0,"num_object_copies":2643,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7049,"num_write_kb":56392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.779905"},"pg_stats":[{"pgid":"1.14","version":"16'2","reported_seq":"114","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.457457+0000","last_change":"2020-08-07T15:10:10.644555+0000","last_active":"2020-08-07T15:10:40.457457+0000","last_peered":"2020-08-07T15:10:40.457457+0000","last_clean":"2020-08-07T15:10:40.457457+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:10:40.457457+0000","last_undegraded":"2020-08-07T15:10:40.457457+0000","last_fullsized":"2020-08-07T15:10:40.457457+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:10.644460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.644460+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"183","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.460503+0000","last_change":"2020-08-07T15:10:07.295971+0000","last_active":"2020-08-07T15:10:40.460503+0000","last_peered":"2020-08-07T15:10:40.460503+0000","last_clean":"2020-08-07T15:10:40.460503+0000","last_became_active":"2020-08-07T15:09:30.203592+0000","last_became_peered":"2020-08-07T15:09:30.203592+0000","last_unstale":"2020-08-07T15:10:40.460503+0000","last_undegraded":"2020-08-07T15:10:40.460503+0000","last_fullsized":"2020-08-07T15:10:40.460503+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:07.295863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:07.295863+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"114","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.452969+0000","last_change":"2020-08-07T15:10:28.734269+0000","last_active":"2020-08-07T15:10:40.452969+0000","last_peered":"2020-08-07T15:10:40.452969+0000","last_clean":"2020-08-07T15:10:40.452969+0000","last_became_active":"2020-08-07T15:08:41.685106+0000","last_became_peered":"2020-08-07T15:08:41.685106+0000","last_unstale":"2020-08-07T15:10:40.452969+0000","last_undegraded":"2020-08-07T15:10:40.452969+0000","last_fullsized":"2020-08-07T15:10:40.452969+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:28.734193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:28.734193+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"227","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.452853+0000","last_change":"2020-08-07T15:10:24.690028+0000","last_active":"2020-08-07T15:10:40.452853+0000","last_peered":"2020-08-07T15:10:40.452853+0000","last_clean":"2020-08-07T15:10:40.452853+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:10:40.452853+0000","last_undegraded":"2020-08-07T15:10:40.452853+0000","last_fullsized":"2020-08-07T15:10:40.452853+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:24.689950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:24.689950+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"221","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.452679+0000","last_change":"2020-08-07T15:10:26.631030+0000","last_active":"2020-08-07T15:10:40.452679+0000","last_peered":"2020-08-07T15:10:40.452679+0000","last_clean":"2020-08-07T15:10:40.452679+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:10:40.452679+0000","last_undegraded":"2020-08-07T15:10:40.452679+0000","last_fullsized":"2020-08-07T15:10:40.452679+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:26.630945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:26.630945+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.0","version":"182'1912","reported_seq":"1938","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.514198+0000","last_change":"2020-08-07T15:10:16.261122+0000","last_active":"2020-08-07T15:10:41.514198+0000","last_peered":"2020-08-07T15:10:41.514198+0000","last_clean":"2020-08-07T15:10:41.514198+0000","last_became_active":"2020-08-07T15:10:16.260690+0000","last_became_peered":"2020-08-07T15:10:16.260690+0000","last_unstale":"2020-08-07T15:10:41.514198+0000","last_undegraded":"2020-08-07T15:10:41.514198+0000","last_fullsized":"2020-08-07T15:10:41.514198+0000","mapping_epoch":164,"log_start":"182'1906","ondisk_log_start":"182'1906","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":15663104,"num_objects":239,"num_object_clones":0,"num_object_copies":717,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1912,"num_write_kb":15296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"174","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.401143+0000","last_change":"2020-08-07T15:09:41.284321+0000","last_active":"2020-08-07T15:10:36.401143+0000","last_peered":"2020-08-07T15:10:36.401143+0000","last_clean":"2020-08-07T15:10:36.401143+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:10:36.401143+0000","last_undegraded":"2020-08-07T15:10:36.401143+0000","last_fullsized":"2020-08-07T15:10:36.401143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:41.284237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:41.284237+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"94","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.454638+0000","last_change":"2020-08-07T15:09:30.283753+0000","last_active":"2020-08-07T15:10:40.454638+0000","last_peered":"2020-08-07T15:10:40.454638+0000","last_clean":"2020-08-07T15:10:40.454638+0000","last_became_active":"2020-08-07T15:09:30.283176+0000","last_became_peered":"2020-08-07T15:09:30.283176+0000","last_unstale":"2020-08-07T15:10:40.454638+0000","last_undegraded":"2020-08-07T15:10:40.454638+0000","last_fullsized":"2020-08-07T15:10:40.454638+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:25.606757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:25.606757+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"182'2336","reported_seq":"2362","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.717515+0000","last_change":"2020-08-07T15:10:16.259807+0000","last_active":"2020-08-07T15:10:41.717515+0000","last_peered":"2020-08-07T15:10:41.717515+0000","last_clean":"2020-08-07T15:10:41.717515+0000","last_became_active":"2020-08-07T15:10:16.259365+0000","last_became_peered":"2020-08-07T15:10:16.259365+0000","last_unstale":"2020-08-07T15:10:41.717515+0000","last_undegraded":"2020-08-07T15:10:41.717515+0000","last_fullsized":"2020-08-07T15:10:41.717515+0000","mapping_epoch":164,"log_start":"182'2330","ondisk_log_start":"182'2330","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19136512,"num_objects":292,"num_object_clones":0,"num_object_copies":876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2336,"num_write_kb":18688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"181","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.460503+0000","last_change":"2020-08-07T15:10:25.463712+0000","last_active":"2020-08-07T15:10:40.460503+0000","last_peered":"2020-08-07T15:10:40.460503+0000","last_clean":"2020-08-07T15:10:40.460503+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:10:40.460503+0000","last_undegraded":"2020-08-07T15:10:40.460503+0000","last_fullsized":"2020-08-07T15:10:40.460503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:25.463628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:10:25.463628+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"139","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.457575+0000","last_change":"2020-08-07T15:10:27.642227+0000","last_active":"2020-08-07T15:10:40.457575+0000","last_peered":"2020-08-07T15:10:40.457575+0000","last_clean":"2020-08-07T15:10:40.457575+0000","last_became_active":"2020-08-07T15:08:46.710404+0000","last_became_peered":"2020-08-07T15:08:46.710404+0000","last_unstale":"2020-08-07T15:10:40.457575+0000","last_undegraded":"2020-08-07T15:10:40.457575+0000","last_fullsized":"2020-08-07T15:10:40.457575+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:10:27.642141+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:10:27.642141+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"178'1808","reported_seq":"1830","reported_epoch":"178","state":"active+clean","last_fresh":"2020-08-07T15:10:37.620364+0000","last_change":"2020-08-07T15:10:16.448360+0000","last_active":"2020-08-07T15:10:37.620364+0000","last_peered":"2020-08-07T15:10:37.620364+0000","last_clean":"2020-08-07T15:10:37.620364+0000","last_became_active":"2020-08-07T15:10:16.447826+0000","last_became_peered":"2020-08-07T15:10:16.447826+0000","last_unstale":"2020-08-07T15:10:37.620364+0000","last_undegraded":"2020-08-07T15:10:37.620364+0000","last_fullsized":"2020-08-07T15:10:37.620364+0000","mapping_epoch":164,"log_start":"178'1800","ondisk_log_start":"178'1800","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":14811136,"num_objects":226,"num_object_clones":0,"num_object_copies":678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1808,"num_write_kb":14464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"179","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.459304+0000","last_change":"2020-08-07T15:09:54.908283+0000","last_active":"2020-08-07T15:10:40.459304+0000","last_peered":"2020-08-07T15:10:40.459304+0000","last_clean":"2020-08-07T15:10:40.459304+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:10:40.459304+0000","last_undegraded":"2020-08-07T15:10:40.459304+0000","last_fullsized":"2020-08-07T15:10:40.459304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:54.908197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:54.908197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"213","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.400956+0000","last_change":"2020-08-07T15:10:07.302258+0000","last_active":"2020-08-07T15:10:36.400956+0000","last_peered":"2020-08-07T15:10:36.400956+0000","last_clean":"2020-08-07T15:10:36.400956+0000","last_became_active":"2020-08-07T15:09:30.201829+0000","last_became_peered":"2020-08-07T15:09:30.201829+0000","last_unstale":"2020-08-07T15:10:36.400956+0000","last_undegraded":"2020-08-07T15:10:36.400956+0000","last_fullsized":"2020-08-07T15:10:36.400956+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:07.302204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:07.302204+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"182'2352","reported_seq":"2376","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.708822+0000","last_change":"2020-08-07T15:10:16.447819+0000","last_active":"2020-08-07T15:10:41.708822+0000","last_peered":"2020-08-07T15:10:41.708822+0000","last_clean":"2020-08-07T15:10:41.708822+0000","last_became_active":"2020-08-07T15:10:16.447312+0000","last_became_peered":"2020-08-07T15:10:16.447312+0000","last_unstale":"2020-08-07T15:10:41.708822+0000","last_undegraded":"2020-08-07T15:10:41.708822+0000","last_fullsized":"2020-08-07T15:10:41.708822+0000","mapping_epoch":164,"log_start":"182'2344","ondisk_log_start":"182'2344","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19267584,"num_objects":294,"num_object_clones":0,"num_object_copies":882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2352,"num_write_kb":18816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"179","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.459154+0000","last_change":"2020-08-07T15:09:51.036281+0000","last_active":"2020-08-07T15:10:40.459154+0000","last_peered":"2020-08-07T15:10:40.459154+0000","last_clean":"2020-08-07T15:10:40.459154+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:10:40.459154+0000","last_undegraded":"2020-08-07T15:10:40.459154+0000","last_fullsized":"2020-08-07T15:10:40.459154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:51.036224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:51.036224+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"179","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.400704+0000","last_change":"2020-08-07T15:10:23.400748+0000","last_active":"2020-08-07T15:10:36.400704+0000","last_peered":"2020-08-07T15:10:36.400704+0000","last_clean":"2020-08-07T15:10:36.400704+0000","last_became_active":"2020-08-07T15:08:36.633406+0000","last_became_peered":"2020-08-07T15:08:36.633406+0000","last_unstale":"2020-08-07T15:10:36.400704+0000","last_undegraded":"2020-08-07T15:10:36.400704+0000","last_fullsized":"2020-08-07T15:10:36.400704+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:23.400672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:23.400672+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"182'2000","reported_seq":"2026","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.624472+0000","last_change":"2020-08-07T15:10:16.255436+0000","last_active":"2020-08-07T15:10:41.624472+0000","last_peered":"2020-08-07T15:10:41.624472+0000","last_clean":"2020-08-07T15:10:41.624472+0000","last_became_active":"2020-08-07T15:10:16.254967+0000","last_became_peered":"2020-08-07T15:10:16.254967+0000","last_unstale":"2020-08-07T15:10:41.624472+0000","last_undegraded":"2020-08-07T15:10:41.624472+0000","last_fullsized":"2020-08-07T15:10:41.624472+0000","mapping_epoch":164,"log_start":"182'1992","ondisk_log_start":"182'1992","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16384000,"num_objects":250,"num_object_clones":0,"num_object_copies":750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2000,"num_write_kb":16000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"211","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.459075+0000","last_change":"2020-08-07T15:09:55.879867+0000","last_active":"2020-08-07T15:10:40.459075+0000","last_peered":"2020-08-07T15:10:40.459075+0000","last_clean":"2020-08-07T15:10:40.459075+0000","last_became_active":"2020-08-07T15:08:36.627426+0000","last_became_peered":"2020-08-07T15:08:36.627426+0000","last_unstale":"2020-08-07T15:10:40.459075+0000","last_undegraded":"2020-08-07T15:10:40.459075+0000","last_fullsized":"2020-08-07T15:10:40.459075+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:55.879785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:55.879785+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"179","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.453047+0000","last_change":"2020-08-07T15:10:08.656952+0000","last_active":"2020-08-07T15:10:40.453047+0000","last_peered":"2020-08-07T15:10:40.453047+0000","last_clean":"2020-08-07T15:10:40.453047+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:10:40.453047+0000","last_undegraded":"2020-08-07T15:10:40.453047+0000","last_fullsized":"2020-08-07T15:10:40.453047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:08.656855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:10:08.656855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"182'2216","reported_seq":"2242","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.607555+0000","last_change":"2020-08-07T15:10:16.263356+0000","last_active":"2020-08-07T15:10:41.607555+0000","last_peered":"2020-08-07T15:10:41.607555+0000","last_clean":"2020-08-07T15:10:41.607555+0000","last_became_active":"2020-08-07T15:10:16.262971+0000","last_became_peered":"2020-08-07T15:10:16.262971+0000","last_unstale":"2020-08-07T15:10:41.607555+0000","last_undegraded":"2020-08-07T15:10:41.607555+0000","last_fullsized":"2020-08-07T15:10:41.607555+0000","mapping_epoch":164,"log_start":"182'2206","ondisk_log_start":"182'2206","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18153472,"num_objects":277,"num_object_clones":0,"num_object_copies":831,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2216,"num_write_kb":17728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"200","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.457332+0000","last_change":"2020-08-07T15:10:24.576099+0000","last_active":"2020-08-07T15:10:40.457332+0000","last_peered":"2020-08-07T15:10:40.457332+0000","last_clean":"2020-08-07T15:10:40.457332+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:10:40.457332+0000","last_undegraded":"2020-08-07T15:10:40.457332+0000","last_fullsized":"2020-08-07T15:10:40.457332+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:24.576023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:10:24.576023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"227","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.459082+0000","last_change":"2020-08-07T15:10:11.912614+0000","last_active":"2020-08-07T15:10:40.459082+0000","last_peered":"2020-08-07T15:10:40.459082+0000","last_clean":"2020-08-07T15:10:40.459082+0000","last_became_active":"2020-08-07T15:08:36.628365+0000","last_became_peered":"2020-08-07T15:08:36.628365+0000","last_unstale":"2020-08-07T15:10:40.459082+0000","last_undegraded":"2020-08-07T15:10:40.459082+0000","last_fullsized":"2020-08-07T15:10:40.459082+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:11.912550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:11.912550+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.5","version":"182'2008","reported_seq":"2034","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.558443+0000","last_change":"2020-08-07T15:10:16.447870+0000","last_active":"2020-08-07T15:10:41.558443+0000","last_peered":"2020-08-07T15:10:41.558443+0000","last_clean":"2020-08-07T15:10:41.558443+0000","last_became_active":"2020-08-07T15:10:16.447478+0000","last_became_peered":"2020-08-07T15:10:16.447478+0000","last_unstale":"2020-08-07T15:10:41.558443+0000","last_undegraded":"2020-08-07T15:10:41.558443+0000","last_fullsized":"2020-08-07T15:10:41.558443+0000","mapping_epoch":164,"log_start":"182'2002","ondisk_log_start":"182'2002","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16449536,"num_objects":251,"num_object_clones":0,"num_object_copies":753,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2008,"num_write_kb":16064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"179","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.453200+0000","last_change":"2020-08-07T15:09:52.637867+0000","last_active":"2020-08-07T15:10:40.453200+0000","last_peered":"2020-08-07T15:10:40.453200+0000","last_clean":"2020-08-07T15:10:40.453200+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:10:40.453200+0000","last_undegraded":"2020-08-07T15:10:40.453200+0000","last_fullsized":"2020-08-07T15:10:40.453200+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:52.637791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:09:52.637791+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"177","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.452968+0000","last_change":"2020-08-07T15:10:11.720217+0000","last_active":"2020-08-07T15:10:40.452968+0000","last_peered":"2020-08-07T15:10:40.452968+0000","last_clean":"2020-08-07T15:10:40.452968+0000","last_became_active":"2020-08-07T15:08:36.618164+0000","last_became_peered":"2020-08-07T15:08:36.618164+0000","last_unstale":"2020-08-07T15:10:40.452968+0000","last_undegraded":"2020-08-07T15:10:40.452968+0000","last_fullsized":"2020-08-07T15:10:40.452968+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:11.720143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:11.720143+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"178'1800","reported_seq":"1822","reported_epoch":"178","state":"active+clean","last_fresh":"2020-08-07T15:10:37.652956+0000","last_change":"2020-08-07T15:10:16.443153+0000","last_active":"2020-08-07T15:10:37.652956+0000","last_peered":"2020-08-07T15:10:37.652956+0000","last_clean":"2020-08-07T15:10:37.652956+0000","last_became_active":"2020-08-07T15:10:16.442674+0000","last_became_peered":"2020-08-07T15:10:16.442674+0000","last_unstale":"2020-08-07T15:10:37.652956+0000","last_undegraded":"2020-08-07T15:10:37.652956+0000","last_fullsized":"2020-08-07T15:10:37.652956+0000","mapping_epoch":164,"log_start":"178'1792","ondisk_log_start":"178'1792","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":14745600,"num_objects":225,"num_object_clones":0,"num_object_copies":675,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1800,"num_write_kb":14400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"176","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.459314+0000","last_change":"2020-08-07T15:09:37.017383+0000","last_active":"2020-08-07T15:10:40.459314+0000","last_peered":"2020-08-07T15:10:40.459314+0000","last_clean":"2020-08-07T15:10:40.459314+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:10:40.459314+0000","last_undegraded":"2020-08-07T15:10:40.459314+0000","last_fullsized":"2020-08-07T15:10:40.459314+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:37.017307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:37.017307+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"244","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.401366+0000","last_change":"2020-08-07T15:09:30.208873+0000","last_active":"2020-08-07T15:10:36.401366+0000","last_peered":"2020-08-07T15:10:36.401366+0000","last_clean":"2020-08-07T15:10:36.401366+0000","last_became_active":"2020-08-07T15:09:30.208345+0000","last_became_peered":"2020-08-07T15:09:30.208345+0000","last_unstale":"2020-08-07T15:10:36.401366+0000","last_undegraded":"2020-08-07T15:10:36.401366+0000","last_fullsized":"2020-08-07T15:10:36.401366+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.381325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.381325+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.e","version":"182'2112","reported_seq":"2137","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.704631+0000","last_change":"2020-08-07T15:10:16.448334+0000","last_active":"2020-08-07T15:10:41.704631+0000","last_peered":"2020-08-07T15:10:41.704631+0000","last_clean":"2020-08-07T15:10:41.704631+0000","last_became_active":"2020-08-07T15:10:16.447541+0000","last_became_peered":"2020-08-07T15:10:16.447541+0000","last_unstale":"2020-08-07T15:10:41.704631+0000","last_undegraded":"2020-08-07T15:10:41.704631+0000","last_fullsized":"2020-08-07T15:10:41.704631+0000","mapping_epoch":164,"log_start":"182'2107","ondisk_log_start":"182'2107","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":17301504,"num_objects":264,"num_object_clones":0,"num_object_copies":792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2112,"num_write_kb":16896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"154","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.453139+0000","last_change":"2020-08-07T15:10:10.715147+0000","last_active":"2020-08-07T15:10:40.453139+0000","last_peered":"2020-08-07T15:10:40.453139+0000","last_clean":"2020-08-07T15:10:40.453139+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:10:40.453139+0000","last_undegraded":"2020-08-07T15:10:40.453139+0000","last_fullsized":"2020-08-07T15:10:40.453139+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.715051+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"182'2488","reported_seq":"2512","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.637571+0000","last_change":"2020-08-07T15:10:16.448023+0000","last_active":"2020-08-07T15:10:41.637571+0000","last_peered":"2020-08-07T15:10:41.637571+0000","last_clean":"2020-08-07T15:10:41.637571+0000","last_became_active":"2020-08-07T15:10:16.447526+0000","last_became_peered":"2020-08-07T15:10:16.447526+0000","last_unstale":"2020-08-07T15:10:41.637571+0000","last_undegraded":"2020-08-07T15:10:41.637571+0000","last_fullsized":"2020-08-07T15:10:41.637571+0000","mapping_epoch":164,"log_start":"182'2482","ondisk_log_start":"182'2482","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20381696,"num_objects":311,"num_object_clones":0,"num_object_copies":933,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2488,"num_write_kb":19904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"151","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.452822+0000","last_change":"2020-08-07T15:09:09.691838+0000","last_active":"2020-08-07T15:10:40.452822+0000","last_peered":"2020-08-07T15:10:40.452822+0000","last_clean":"2020-08-07T15:10:40.452822+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:10:40.452822+0000","last_undegraded":"2020-08-07T15:10:40.452822+0000","last_fullsized":"2020-08-07T15:10:40.452822+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.691753+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.d","version":"182'2216","reported_seq":"2242","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.561687+0000","last_change":"2020-08-07T15:10:16.263289+0000","last_active":"2020-08-07T15:10:41.561687+0000","last_peered":"2020-08-07T15:10:41.561687+0000","last_clean":"2020-08-07T15:10:41.561687+0000","last_became_active":"2020-08-07T15:10:16.262869+0000","last_became_peered":"2020-08-07T15:10:16.262869+0000","last_unstale":"2020-08-07T15:10:41.561687+0000","last_undegraded":"2020-08-07T15:10:41.561687+0000","last_fullsized":"2020-08-07T15:10:41.561687+0000","mapping_epoch":164,"log_start":"182'2208","ondisk_log_start":"182'2208","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18153472,"num_objects":277,"num_object_clones":0,"num_object_copies":831,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2216,"num_write_kb":17728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"218","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.459158+0000","last_change":"2020-08-07T15:10:31.122527+0000","last_active":"2020-08-07T15:10:40.459158+0000","last_peered":"2020-08-07T15:10:40.459158+0000","last_clean":"2020-08-07T15:10:40.459158+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:10:40.459158+0000","last_undegraded":"2020-08-07T15:10:40.459158+0000","last_fullsized":"2020-08-07T15:10:40.459158+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:31.122435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:31.122435+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"182'2192","reported_seq":"2218","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.691962+0000","last_change":"2020-08-07T15:10:16.256000+0000","last_active":"2020-08-07T15:10:41.691962+0000","last_peered":"2020-08-07T15:10:41.691962+0000","last_clean":"2020-08-07T15:10:41.691962+0000","last_became_active":"2020-08-07T15:10:16.255356+0000","last_became_peered":"2020-08-07T15:10:16.255356+0000","last_unstale":"2020-08-07T15:10:41.691962+0000","last_undegraded":"2020-08-07T15:10:41.691962+0000","last_fullsized":"2020-08-07T15:10:41.691962+0000","mapping_epoch":164,"log_start":"182'2185","ondisk_log_start":"182'2185","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":17956864,"num_objects":274,"num_object_clones":0,"num_object_copies":822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2192,"num_write_kb":17536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"173","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.459322+0000","last_change":"2020-08-07T15:10:10.962565+0000","last_active":"2020-08-07T15:10:40.459322+0000","last_peered":"2020-08-07T15:10:40.459322+0000","last_clean":"2020-08-07T15:10:40.459322+0000","last_became_active":"2020-08-07T15:08:41.680448+0000","last_became_peered":"2020-08-07T15:08:41.680448+0000","last_unstale":"2020-08-07T15:10:40.459322+0000","last_undegraded":"2020-08-07T15:10:40.459322+0000","last_fullsized":"2020-08-07T15:10:40.459322+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:10:10.962480+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.962480+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"182'2193","reported_seq":"2215","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.725314+0000","last_change":"2020-08-07T15:10:16.444776+0000","last_active":"2020-08-07T15:10:41.725314+0000","last_peered":"2020-08-07T15:10:41.725314+0000","last_clean":"2020-08-07T15:10:41.725314+0000","last_became_active":"2020-08-07T15:10:16.444316+0000","last_became_peered":"2020-08-07T15:10:16.444316+0000","last_unstale":"2020-08-07T15:10:41.725314+0000","last_undegraded":"2020-08-07T15:10:41.725314+0000","last_fullsized":"2020-08-07T15:10:41.725314+0000","mapping_epoch":164,"log_start":"182'2186","ondisk_log_start":"182'2186","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":17965056,"num_objects":275,"num_object_clones":0,"num_object_copies":825,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2193,"num_write_kb":17544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"140","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.454781+0000","last_change":"2020-08-07T15:09:36.687393+0000","last_active":"2020-08-07T15:10:40.454781+0000","last_peered":"2020-08-07T15:10:40.454781+0000","last_clean":"2020-08-07T15:10:40.454781+0000","last_became_active":"2020-08-07T15:09:30.204655+0000","last_became_peered":"2020-08-07T15:09:30.204655+0000","last_unstale":"2020-08-07T15:10:40.454781+0000","last_undegraded":"2020-08-07T15:10:40.454781+0000","last_fullsized":"2020-08-07T15:10:40.454781+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:36.687314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:36.687314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"182'1944","reported_seq":"1965","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.725310+0000","last_change":"2020-08-07T15:10:16.444970+0000","last_active":"2020-08-07T15:10:41.725310+0000","last_peered":"2020-08-07T15:10:41.725310+0000","last_clean":"2020-08-07T15:10:41.725310+0000","last_became_active":"2020-08-07T15:10:16.444294+0000","last_became_peered":"2020-08-07T15:10:16.444294+0000","last_unstale":"2020-08-07T15:10:41.725310+0000","last_undegraded":"2020-08-07T15:10:41.725310+0000","last_fullsized":"2020-08-07T15:10:41.725310+0000","mapping_epoch":164,"log_start":"182'1936","ondisk_log_start":"182'1936","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":15925248,"num_objects":243,"num_object_clones":0,"num_object_copies":729,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1944,"num_write_kb":15552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"209","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.401336+0000","last_change":"2020-08-07T15:09:08.399507+0000","last_active":"2020-08-07T15:10:36.401336+0000","last_peered":"2020-08-07T15:10:36.401336+0000","last_clean":"2020-08-07T15:10:36.401336+0000","last_became_active":"2020-08-07T15:08:41.689435+0000","last_became_peered":"2020-08-07T15:08:41.689435+0000","last_unstale":"2020-08-07T15:10:36.401336+0000","last_undegraded":"2020-08-07T15:10:36.401336+0000","last_fullsized":"2020-08-07T15:10:36.401336+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:08.399439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:08.399439+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"182'2048","reported_seq":"2074","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.710979+0000","last_change":"2020-08-07T15:10:16.256758+0000","last_active":"2020-08-07T15:10:41.710979+0000","last_peered":"2020-08-07T15:10:41.710979+0000","last_clean":"2020-08-07T15:10:41.710979+0000","last_became_active":"2020-08-07T15:10:16.256419+0000","last_became_peered":"2020-08-07T15:10:16.256419+0000","last_unstale":"2020-08-07T15:10:41.710979+0000","last_undegraded":"2020-08-07T15:10:41.710979+0000","last_fullsized":"2020-08-07T15:10:41.710979+0000","mapping_epoch":164,"log_start":"182'2041","ondisk_log_start":"182'2041","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777216,"num_objects":256,"num_object_clones":0,"num_object_copies":768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2048,"num_write_kb":16384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"104","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.459290+0000","last_change":"2020-08-07T15:10:29.055136+0000","last_active":"2020-08-07T15:10:40.459290+0000","last_peered":"2020-08-07T15:10:40.459290+0000","last_clean":"2020-08-07T15:10:40.459290+0000","last_became_active":"2020-08-07T15:08:46.710059+0000","last_became_peered":"2020-08-07T15:08:46.710059+0000","last_unstale":"2020-08-07T15:10:40.459290+0000","last_undegraded":"2020-08-07T15:10:40.459290+0000","last_fullsized":"2020-08-07T15:10:40.459290+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:29.055048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:29.055048+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"182'2288","reported_seq":"2314","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.689743+0000","last_change":"2020-08-07T15:10:16.253662+0000","last_active":"2020-08-07T15:10:41.689743+0000","last_peered":"2020-08-07T15:10:41.689743+0000","last_clean":"2020-08-07T15:10:41.689743+0000","last_became_active":"2020-08-07T15:10:16.253105+0000","last_became_peered":"2020-08-07T15:10:16.253105+0000","last_unstale":"2020-08-07T15:10:41.689743+0000","last_undegraded":"2020-08-07T15:10:41.689743+0000","last_fullsized":"2020-08-07T15:10:41.689743+0000","mapping_epoch":164,"log_start":"182'2278","ondisk_log_start":"182'2278","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18743296,"num_objects":286,"num_object_clones":0,"num_object_copies":858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2288,"num_write_kb":18304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"126","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.452966+0000","last_change":"2020-08-07T15:10:37.601162+0000","last_active":"2020-08-07T15:10:40.452966+0000","last_peered":"2020-08-07T15:10:40.452966+0000","last_clean":"2020-08-07T15:10:40.452966+0000","last_became_active":"2020-08-07T15:08:41.677944+0000","last_became_peered":"2020-08-07T15:08:41.677944+0000","last_unstale":"2020-08-07T15:10:40.452966+0000","last_undegraded":"2020-08-07T15:10:40.452966+0000","last_fullsized":"2020-08-07T15:10:40.452966+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:37.601099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:37.601099+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":277815296,"num_objects":4240,"num_object_clones":0,"num_object_copies":12720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33913,"num_write_kb":271304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":839819264,"data_stored":839819264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":63,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542198,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1106488,"kb_used_data":57904,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93265352,"statfs":{"total":96636764160,"available":95503720448,"internally_reserved":1073741824,"allocated":59293696,"data_stored":58624399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":43.911000000000001,"5min":9.5549999999999997,"15min":3.7799999999999998},"min":{"1min":0.50800000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":856.07100000000003,"5min":856.07100000000003,"15min":856.07100000000003},"last":0.42199999999999999},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.026,"15min":0.98399999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.8899999999999999,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":0.79400000000000004}]},{"osd":1,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.335,"15min":1.3080000000000001},"min":{"1min":0.64200000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.9199999999999999,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":2.395},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.141,"15min":1.024},"min":{"1min":0.61799999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":4.0060000000000002,"5min":4.0060000000000002,"15min":4.0060000000000002},"last":2.4950000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.304,"15min":1.1859999999999999},"min":{"1min":0.747,"5min":0.628,"15min":0.628},"max":{"1min":2.0249999999999999,"5min":5.6619999999999999,"15min":5.6619999999999999},"last":2.4359999999999999},{"interface":"front","average":{"1min":1.512,"5min":1.3380000000000001,"15min":1.1910000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.6619999999999999,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":0.54300000000000004}]},{"osd":3,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":16.18,"15min":6.508},"min":{"1min":0.64700000000000002,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.9710000000000001,"5min":917.38699999999994,"15min":917.38699999999994},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.196,"5min":4.9649999999999999,"15min":2.5499999999999998},"min":{"1min":0.77200000000000002,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.3359999999999999,"5min":333.59199999999998,"15min":333.59199999999998},"last":0.84799999999999998}]},{"osd":4,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.464,"15min":1.3380000000000001},"min":{"1min":0.63400000000000001,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":3.9750000000000001,"5min":4.835,"15min":4.835},"last":0.65400000000000003},{"interface":"front","average":{"1min":1.306,"5min":11.518000000000001,"15min":4.7140000000000004},"min":{"1min":0.69699999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":3.0329999999999999,"5min":866.24099999999999,"15min":866.24099999999999},"last":0.92500000000000004}]},{"osd":5,"last update":"Fri Aug 7 15:10:27 2020","interfaces":[{"interface":"back","average":{"1min":39.496000000000002,"5min":23.838000000000001,"15min":32.950000000000003},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":678.76099999999997,"5min":831.88099999999997,"15min":831.88099999999997},"last":2.468},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.823,"15min":1.6040000000000001},"min":{"1min":0.84999999999999998,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.4060000000000001,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":2.5219999999999998}]},{"osd":7,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":3.9929999999999999,"15min":2.343},"min":{"1min":0.89900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":3.0049999999999999,"5min":224.14500000000001,"15min":224.14500000000001},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.6899999999999999,"15min":1.5680000000000001},"min":{"1min":1.113,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.2919999999999998,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":3.097}]}]},{"osd":7,"up_from":13,"seq":55834574904,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1134112,"kb_used_data":85528,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93237728,"statfs":{"total":96636764160,"available":95475433472,"internally_reserved":1073741824,"allocated":87580672,"data_stored":86881135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1459999999999999,"15min":1.27},"min":{"1min":0.58099999999999996,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":3.2090000000000001,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":1.3160000000000001},{"interface":"front","average":{"1min":30.457000000000001,"5min":9.8149999999999995,"15min":4.0380000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":529.88699999999994,"5min":529.88699999999994,"15min":529.88699999999994},"last":0.92400000000000004}]},{"osd":1,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":5.758,"15min":2.5579999999999998},"min":{"1min":0.47699999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.7290000000000001,"5min":547.65300000000002,"15min":547.65300000000002},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.4159999999999999,"5min":4.9189999999999996,"15min":2.5640000000000001},"min":{"1min":0.57399999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":5.4960000000000004,"5min":372.50799999999998,"15min":372.50799999999998},"last":2.3199999999999998}]},{"osd":2,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.262,"15min":1.2170000000000001},"min":{"1min":0.56799999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":3.4060000000000001,"5min":5.2400000000000002,"15min":5.2400000000000002},"last":1.365},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2170000000000001,"15min":1.2529999999999999},"min":{"1min":0.61899999999999999,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.4569999999999999,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":1.1799999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":42.097999999999999,"5min":9.5510000000000002,"15min":4.2009999999999996},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":698.19899999999996,"5min":698.19899999999996,"15min":698.19899999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.3160000000000001,"15min":1.3640000000000001},"min":{"1min":0.80500000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.8959999999999999,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":4.1779999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.3979999999999999,"15min":1.5740000000000001},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.885,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":1.446},{"interface":"front","average":{"1min":1.353,"5min":1.5620000000000001,"15min":1.635},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":2.0699999999999998,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.3460000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.2589999999999999,"15min":1.329},"min":{"1min":0.66500000000000004,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":3.0009999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.87},{"interface":"front","average":{"1min":1.298,"5min":1.532,"15min":1.6879999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.9419999999999999,"5min":5.327,"15min":5.327},"last":1.2529999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.466,"15min":1.444},"min":{"1min":0.93400000000000005,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":3.2229999999999999,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":8.2379999999999995,"15min":14.962999999999999},"min":{"1min":0.88700000000000001,"5min":0.87,"15min":0.87},"max":{"1min":2.3330000000000002,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.6839999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1136296,"kb_used_data":87712,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93235544,"statfs":{"total":96636764160,"available":95473197056,"internally_reserved":1073741824,"allocated":89817088,"data_stored":89117551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":13.122,"5min":16.731999999999999,"15min":28.367999999999999},"min":{"1min":0.47199999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":211.63399999999999,"5min":497.04000000000002,"15min":497.04000000000002},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.645,"5min":14.757999999999999,"15min":5.673},"min":{"1min":0.57399999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":8.8849999999999998,"5min":993.84500000000003,"15min":993.84500000000003},"last":0.84999999999999998}]},{"osd":2,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":54.984000000000002,"5min":11.816000000000001,"15min":4.5380000000000003},"min":{"1min":0.57199999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":959.36900000000003,"5min":959.36900000000003,"15min":959.36900000000003},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.72,"5min":1.357,"15min":1.2549999999999999},"min":{"1min":0.53600000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":5.4530000000000003,"5min":6.4029999999999996,"15min":6.4029999999999996},"last":1.6479999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":9.2420000000000009,"15min":3.9540000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.3879999999999999,"5min":828.31600000000003,"15min":828.31600000000003},"last":1.165},{"interface":"front","average":{"1min":1.2130000000000001,"5min":10.119999999999999,"15min":4.3419999999999996},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.5659999999999998,"5min":905.38300000000004,"15min":905.38300000000004},"last":1.2030000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":49.530999999999999,"5min":16.422999999999998,"15min":6.3300000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":872.58500000000004,"5min":872.58500000000004,"15min":872.58500000000004},"last":1.127},{"interface":"front","average":{"1min":2.2080000000000002,"5min":1.5409999999999999,"15min":1.3129999999999999},"min":{"1min":0.75800000000000001,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":12.640000000000001,"5min":12.640000000000001,"15min":12.640000000000001},"last":1.2470000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":6.0919999999999996,"15min":10.702},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.056,"5min":213.63800000000001,"15min":213.63800000000001},"last":1.2849999999999999},{"interface":"front","average":{"1min":2.4630000000000001,"5min":12.974,"15min":5.1180000000000003},"min":{"1min":0.93400000000000005,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":12.677,"5min":875.44600000000003,"15min":875.44600000000003},"last":1.5660000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:10:32 2020","interfaces":[{"interface":"back","average":{"1min":2.9340000000000002,"5min":1.863,"15min":1.655},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":22.402000000000001,"5min":22.402000000000001,"15min":22.402000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.3160000000000001,"5min":3.589,"15min":2.1030000000000002},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.3980000000000001,"5min":207.137,"15min":207.137},"last":1.5349999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":2.0910000000000002,"5min":1.7829999999999999,"15min":1.671},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":5.4960000000000004,"5min":6.444,"15min":6.444},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.655,"15min":1.538},"min":{"1min":0.78700000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.0219999999999998,"5min":10.752000000000001,"15min":10.752000000000001},"last":1.7330000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1184608,"kb_used_data":136024,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93187232,"statfs":{"total":96636764160,"available":95423725568,"internally_reserved":1073741824,"allocated":139288576,"data_stored":138589039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":24.291,"5min":21.719999999999999,"15min":7.891},"min":{"1min":0.52700000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":432.899,"5min":845.57100000000003,"15min":845.57100000000003},"last":0.51500000000000001},{"interface":"front","average":{"1min":1.752,"5min":1.181,"15min":1.0149999999999999},"min":{"1min":0.62,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":9.3650000000000002,"5min":9.3650000000000002,"15min":9.3650000000000002},"last":1.127}]},{"osd":2,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.0249999999999999,"15min":0.95999999999999996},"min":{"1min":0.52200000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.972,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.296,"5min":16.190999999999999,"15min":27.222999999999999},"min":{"1min":0.55900000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.8519999999999999,"5min":606.18499999999995,"15min":606.18499999999995},"last":1.194}]},{"osd":3,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":1.7809999999999999,"5min":1.2350000000000001,"15min":1.0349999999999999},"min":{"1min":0.54300000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":9.3360000000000003,"5min":9.3360000000000003,"15min":9.3360000000000003},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.677,"5min":1.4179999999999999,"15min":1.2509999999999999},"min":{"1min":0.63600000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":11.19,"5min":11.19,"15min":11.19},"last":0.77500000000000002}]},{"osd":4,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.355,"15min":1.2290000000000001},"min":{"1min":0.68300000000000005,"5min":0.63,"15min":0.63},"max":{"1min":3.9729999999999999,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.4510000000000001,"15min":1.3220000000000001},"min":{"1min":0.83899999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.012,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":0.89000000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":9.0800000000000001,"15min":3.8439999999999999},"min":{"1min":0.75600000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":3.883,"5min":501.267,"15min":501.267},"last":3.5579999999999998},{"interface":"front","average":{"1min":1.669,"5min":1.3740000000000001,"15min":1.1950000000000001},"min":{"1min":0.752,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":4.0140000000000002,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":3.5790000000000002}]},{"osd":6,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":4.3310000000000004,"15min":7.1859999999999999},"min":{"1min":0.84999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.1400000000000001,"5min":124.738,"15min":124.738},"last":3.1899999999999999},{"interface":"front","average":{"1min":1.611,"5min":8.4019999999999992,"15min":15.236000000000001},"min":{"1min":0.95899999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":3.9119999999999999,"5min":294.94,"15min":294.94},"last":5.1799999999999997}]},{"osd":7,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":1.647,"5min":1.571,"15min":1.581},"min":{"1min":0.98899999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":3.9369999999999998,"5min":6.468,"15min":6.468},"last":4.0010000000000003},{"interface":"front","average":{"1min":46.985999999999997,"5min":10.561,"15min":4.444},"min":{"1min":0.88200000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":859.82399999999996,"5min":859.82399999999996,"15min":859.82399999999996},"last":1.3799999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1150076,"kb_used_data":101492,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93221764,"statfs":{"total":96636764160,"available":95459086336,"internally_reserved":1073741824,"allocated":103927808,"data_stored":103220713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:10:21 2020","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.268,"15min":1.167},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":3.706,"5min":8.4930000000000003,"15min":8.4930000000000003},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.1910000000000001,"15min":1.0469999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":5.194,"5min":5.194,"15min":5.194},"last":0.46800000000000003}]},{"osd":1,"last update":"Fri Aug 7 15:10:21 2020","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.6970000000000001,"15min":1.2929999999999999},"min":{"1min":0.52000000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":5.5919999999999996,"5min":18.672000000000001,"15min":18.672000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":1.264,"5min":1.3320000000000001,"15min":1.3500000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":3.7970000000000002,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":0.70499999999999996}]},{"osd":3,"last update":"Fri Aug 7 15:10:21 2020","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.2949999999999999,"15min":1.2070000000000001},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":3.1970000000000001,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.796,"5min":8.0990000000000002,"15min":3.7370000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":7.9349999999999996,"5min":501.75599999999997,"15min":501.75599999999997},"last":0.85399999999999998}]},{"osd":4,"last update":"Fri Aug 7 15:10:21 2020","interfaces":[{"interface":"back","average":{"1min":1.6379999999999999,"5min":1.508,"15min":1.242},"min":{"1min":0.64500000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":3.9409999999999998,"5min":19.783000000000001,"15min":19.783000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.6020000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":3.7919999999999998,"5min":3.7919999999999998,"15min":3.7919999999999998},"last":0.92000000000000004}]},{"osd":5,"last update":"Fri Aug 7 15:10:21 2020","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.3520000000000001,"15min":1.292},"min":{"1min":0.80900000000000005,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":5.2220000000000004,"5min":5.2220000000000004,"15min":5.2220000000000004},"last":1.1259999999999999},{"interface":"front","average":{"1min":30.856000000000002,"5min":25.364999999999998,"15min":25.471},"min":{"1min":0.83699999999999997,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":584.69000000000005,"5min":627.53300000000002,"15min":627.53300000000002},"last":0.73199999999999998}]},{"osd":6,"last update":"Fri Aug 7 15:10:32 2020","interfaces":[{"interface":"back","average":{"1min":1.8,"5min":1.6539999999999999,"15min":1.365},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":4.1639999999999997,"5min":20.035,"15min":20.035},"last":1.502},{"interface":"front","average":{"1min":1.6459999999999999,"5min":9.9629999999999992,"15min":4.2009999999999996},"min":{"1min":0.95499999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":5.3019999999999996,"5min":751.70000000000005,"15min":751.70000000000005},"last":0.80200000000000005}]},{"osd":7,"last update":"Fri Aug 7 15:10:21 2020","interfaces":[{"interface":"back","average":{"1min":1.6140000000000001,"5min":12.569000000000001,"15min":5.351},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":3.7709999999999999,"5min":501.91699999999997,"15min":501.91699999999997},"last":1.702},{"interface":"front","average":{"1min":1.6639999999999999,"5min":1.9530000000000001,"15min":1.6479999999999999},"min":{"1min":0.92900000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":5.7220000000000004,"5min":35.646999999999998,"15min":35.646999999999998},"last":1.5389999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574903,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1099764,"kb_used_data":51180,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93272076,"statfs":{"total":96636764160,"available":95510605824,"internally_reserved":1073741824,"allocated":52408320,"data_stored":51701225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.3,"15min":1.268},"min":{"1min":0.498,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":3.617,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.369,"15min":1.5469999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":3.6520000000000001,"5min":13.067,"15min":13.067},"last":1.488}]},{"osd":1,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":4.4779999999999998,"15min":2.758},"min":{"1min":0.59099999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.847,"5min":270.68700000000001,"15min":270.68700000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":1.109,"5min":1.8580000000000001,"15min":1.96},"min":{"1min":0.54100000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.0550000000000002,"5min":14.349,"15min":14.349},"last":1.2629999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.3169999999999999,"15min":1.204},"min":{"1min":0.56999999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.6930000000000001,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":2.024,"15min":1.798},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.9420000000000002,"5min":51.530000000000001,"15min":51.530000000000001},"last":1.0980000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":35.152000000000001,"5min":17.465,"15min":6.9530000000000003},"min":{"1min":0.64100000000000001,"5min":0.62,"15min":0.62},"max":{"1min":646.26700000000005,"5min":646.26700000000005,"15min":646.26700000000005},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.5269999999999999,"5min":19.353999999999999,"15min":27.850999999999999},"min":{"1min":0.69699999999999995,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":3.911,"5min":677.88599999999997,"15min":677.88599999999997},"last":1.3440000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":23.408000000000001,"5min":5.9699999999999998,"15min":3.0019999999999998},"min":{"1min":0.90000000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":403.88400000000001,"5min":403.88400000000001,"15min":403.88400000000001},"last":1.4630000000000001},{"interface":"front","average":{"1min":2.4209999999999998,"5min":1.931,"15min":1.823},"min":{"1min":1.0289999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":8.9120000000000008,"5min":8.9120000000000008,"15min":8.9120000000000008},"last":1.5169999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.6930000000000001,"5min":1.659,"15min":1.5740000000000001},"min":{"1min":0.79100000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":3.9500000000000002,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":1.381},{"interface":"front","average":{"1min":39.466999999999999,"5min":13.435,"15min":5.8029999999999999},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":688.15999999999997,"5min":688.15999999999997,"15min":688.15999999999997},"last":0.83099999999999996}]},{"osd":7,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.7809999999999999,"15min":2.0350000000000001},"min":{"1min":0.91000000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.2410000000000001,"5min":5.734,"15min":5.734},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.857,"15min":1.9370000000000001},"min":{"1min":0.84499999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.4140000000000001,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":0.91800000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574903,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1183872,"kb_used_data":135288,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93187968,"statfs":{"total":96636764160,"available":95424479232,"internally_reserved":1073741824,"allocated":138534912,"data_stored":137835375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:10:24 2020","interfaces":[{"interface":"back","average":{"1min":26.023,"5min":6.04,"15min":2.7999999999999998},"min":{"1min":0.48299999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":399.73700000000002,"5min":399.73700000000002,"15min":399.73700000000002},"last":1.087},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":0.91500000000000004},"min":{"1min":0.54600000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":2.665,"5min":6.3970000000000002,"15min":6.3970000000000002},"last":0.58299999999999996}]},{"osd":1,"last update":"Fri Aug 7 15:10:24 2020","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":5.7629999999999999,"15min":2.823},"min":{"1min":0.52900000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.8109999999999999,"5min":320.67000000000002,"15min":320.67000000000002},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.177,"5min":7.5300000000000002,"15min":3.4279999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.7200000000000002,"5min":689.846,"15min":689.846},"last":1.2070000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:10:24 2020","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.4019999999999999,"15min":1.4350000000000001},"min":{"1min":0.76600000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":3.4830000000000001,"5min":6.8730000000000002,"15min":6.8730000000000002},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.556,"5min":1.524,"15min":1.518},"min":{"1min":0.88600000000000001,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":3.5659999999999998,"5min":7.1360000000000001,"15min":7.1360000000000001},"last":0.999}]},{"osd":3,"last update":"Fri Aug 7 15:10:24 2020","interfaces":[{"interface":"back","average":{"1min":43.743000000000002,"5min":12.128,"15min":5.048},"min":{"1min":0.82399999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":638.92100000000005,"5min":638.92100000000005,"15min":638.92100000000005},"last":1.238},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.367,"15min":1.2889999999999999},"min":{"1min":0.75800000000000001,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.1200000000000001,"5min":6.431,"15min":6.431},"last":1.264}]},{"osd":5,"last update":"Fri Aug 7 15:10:24 2020","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":10.446,"15min":4.5670000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.2559999999999998,"5min":990.71600000000001,"15min":990.71600000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":1.204,"5min":1.526,"15min":1.593},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.669,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":1.5780000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:10:32 2020","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.661,"15min":1.7470000000000001},"min":{"1min":0.73399999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.7000000000000002,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":0.87},{"interface":"front","average":{"1min":1.282,"5min":8.5039999999999996,"15min":15.574},"min":{"1min":0.93700000000000006,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.0870000000000002,"5min":389.14600000000002,"15min":389.14600000000002},"last":1.391}]},{"osd":7,"last update":"Fri Aug 7 15:10:24 2020","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":3.8079999999999998,"15min":2.2080000000000002},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.7290000000000001,"5min":252.798,"15min":252.798},"last":0.92000000000000004},{"interface":"front","average":{"1min":21.542000000000002,"5min":14.154,"15min":5.8150000000000004},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":326.327,"5min":501.93000000000001,"15min":501.93000000000001},"last":1.145}]}]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1205260,"kb_used_data":156676,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93166580,"statfs":{"total":96636764160,"available":95402577920,"internally_reserved":1073741824,"allocated":160436224,"data_stored":159729129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.0289999999999999,"15min":1.0609999999999999},"min":{"1min":0.64000000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":3.7349999999999999,"5min":3.7349999999999999,"15min":3.7349999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94399999999999995,"15min":0.85099999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":2.0449999999999999,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.375}]},{"osd":1,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":8.5239999999999991,"5min":12.039999999999999,"15min":4.6319999999999997},"min":{"1min":0.56799999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":139.88200000000001,"5min":787.22900000000004,"15min":787.22900000000004},"last":1.587},{"interface":"front","average":{"1min":1.448,"5min":2.5649999999999999,"15min":1.593},"min":{"1min":0.59899999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":4.3700000000000001,"5min":116.548,"15min":116.548},"last":1.0580000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":4.3600000000000003,"15min":7.5999999999999996},"min":{"1min":0.77300000000000002,"5min":0.499,"15min":0.499},"max":{"1min":3.8109999999999999,"5min":168.56899999999999,"15min":168.56899999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":1.165,"5min":13.967000000000001,"15min":26.742999999999999},"min":{"1min":0.66200000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.1280000000000001,"5min":676.27099999999996,"15min":676.27099999999996},"last":0.52200000000000002}]},{"osd":3,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.589,"15min":1.2749999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":4.8120000000000003,"5min":29.129000000000001,"15min":29.129000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.5660000000000001,"5min":10.016999999999999,"15min":4.1470000000000002},"min":{"1min":0.76400000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":6.4189999999999996,"5min":712.98699999999997,"15min":712.98699999999997},"last":1.073}]},{"osd":4,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":10.253,"5min":32.627000000000002,"15min":44.804000000000002},"min":{"1min":0.60499999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":171.43700000000001,"5min":971.67999999999995,"15min":971.67999999999995},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.2410000000000001,"15min":1.284},"min":{"1min":0.61899999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.411,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":0.745}]},{"osd":6,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":12.24,"5min":41.228000000000002,"15min":47.347999999999999},"min":{"1min":0.85099999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":208.25299999999999,"5min":966.60900000000004,"15min":966.60900000000004},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.5049999999999999,"5min":16.625,"15min":22.885999999999999},"min":{"1min":0.88100000000000001,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":4.4039999999999999,"5min":517.26700000000005,"15min":517.26700000000005},"last":1.1160000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.329,"15min":1.3149999999999999},"min":{"1min":0.79200000000000004,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.0830000000000002,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.427,"5min":6.6349999999999998,"15min":3.2559999999999998},"min":{"1min":0.89600000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.7269999999999999,"5min":515.09100000000001,"15min":515.09100000000001},"last":1.085}]}]}]}} 2020-08-07T15:10:42.498 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:10:42.501 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2020-08-07T15:10:42.518 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:10:42.518 INFO:tasks.thrashosds.thrasher:Growing pool device_health_metrics 2020-08-07T15:10:42.519 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:10:42.529 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:10:42.531 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:42.530+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:42.653 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:42.632+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:42.660 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:42.660 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:42.694 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:10:42.726 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:10:42.740 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:42.737+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:42.778 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:10:42.827 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:10:42.854 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:42.852+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:42.881 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:10:42.918 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:42.918 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:10:42.919 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:10:42.919 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:10:42.919 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:42.926 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:10:42.928 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:10:42.955 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:42.953+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:42.970 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:10:43.019 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:10:43.059 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:43.057+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:43.071 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:10:43.072 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:10:43.072 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:10:43.072 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:10:43.072 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:10:43.072 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:10:43.073 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:10:43.073 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:10:43.073 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:10:43.084 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:10:43.085 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:10:43.105 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2020-08-07T15:10:42.661961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":277815296,"num_objects":4242,"num_object_clones":0,"num_object_copies":12726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33913,"num_write_kb":271304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":135,"acting":135,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9200476,"kb_used_data":811804,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":745774244,"statfs":{"total":773094113280,"available":763672825856,"internally_reserved":8589934592,"allocated":831287296,"data_stored":825698566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":57745408,"num_objects":881,"num_object_clones":0,"num_object_copies":2643,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7049,"num_write_kb":56392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000825"},"pg_stats":[{"pgid":"1.14","version":"16'2","reported_seq":"114","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.457457+0000","last_change":"2020-08-07T15:10:10.644555+0000","last_active":"2020-08-07T15:10:40.457457+0000","last_peered":"2020-08-07T15:10:40.457457+0000","last_clean":"2020-08-07T15:10:40.457457+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:10:40.457457+0000","last_undegraded":"2020-08-07T15:10:40.457457+0000","last_fullsized":"2020-08-07T15:10:40.457457+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:10.644460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.644460+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"183","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.460503+0000","last_change":"2020-08-07T15:10:07.295971+0000","last_active":"2020-08-07T15:10:40.460503+0000","last_peered":"2020-08-07T15:10:40.460503+0000","last_clean":"2020-08-07T15:10:40.460503+0000","last_became_active":"2020-08-07T15:09:30.203592+0000","last_became_peered":"2020-08-07T15:09:30.203592+0000","last_unstale":"2020-08-07T15:10:40.460503+0000","last_undegraded":"2020-08-07T15:10:40.460503+0000","last_fullsized":"2020-08-07T15:10:40.460503+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:07.295863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:07.295863+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"114","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.452969+0000","last_change":"2020-08-07T15:10:28.734269+0000","last_active":"2020-08-07T15:10:40.452969+0000","last_peered":"2020-08-07T15:10:40.452969+0000","last_clean":"2020-08-07T15:10:40.452969+0000","last_became_active":"2020-08-07T15:08:41.685106+0000","last_became_peered":"2020-08-07T15:08:41.685106+0000","last_unstale":"2020-08-07T15:10:40.452969+0000","last_undegraded":"2020-08-07T15:10:40.452969+0000","last_fullsized":"2020-08-07T15:10:40.452969+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:28.734193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:28.734193+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"227","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.452853+0000","last_change":"2020-08-07T15:10:24.690028+0000","last_active":"2020-08-07T15:10:40.452853+0000","last_peered":"2020-08-07T15:10:40.452853+0000","last_clean":"2020-08-07T15:10:40.452853+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:10:40.452853+0000","last_undegraded":"2020-08-07T15:10:40.452853+0000","last_fullsized":"2020-08-07T15:10:40.452853+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:24.689950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:24.689950+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"221","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.452679+0000","last_change":"2020-08-07T15:10:26.631030+0000","last_active":"2020-08-07T15:10:40.452679+0000","last_peered":"2020-08-07T15:10:40.452679+0000","last_clean":"2020-08-07T15:10:40.452679+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:10:40.452679+0000","last_undegraded":"2020-08-07T15:10:40.452679+0000","last_fullsized":"2020-08-07T15:10:40.452679+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:26.630945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:26.630945+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.0","version":"182'1912","reported_seq":"1938","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.514198+0000","last_change":"2020-08-07T15:10:16.261122+0000","last_active":"2020-08-07T15:10:41.514198+0000","last_peered":"2020-08-07T15:10:41.514198+0000","last_clean":"2020-08-07T15:10:41.514198+0000","last_became_active":"2020-08-07T15:10:16.260690+0000","last_became_peered":"2020-08-07T15:10:16.260690+0000","last_unstale":"2020-08-07T15:10:41.514198+0000","last_undegraded":"2020-08-07T15:10:41.514198+0000","last_fullsized":"2020-08-07T15:10:41.514198+0000","mapping_epoch":164,"log_start":"182'1906","ondisk_log_start":"182'1906","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":15663104,"num_objects":239,"num_object_clones":0,"num_object_copies":717,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1912,"num_write_kb":15296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"174","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.401143+0000","last_change":"2020-08-07T15:09:41.284321+0000","last_active":"2020-08-07T15:10:36.401143+0000","last_peered":"2020-08-07T15:10:36.401143+0000","last_clean":"2020-08-07T15:10:36.401143+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:10:36.401143+0000","last_undegraded":"2020-08-07T15:10:36.401143+0000","last_fullsized":"2020-08-07T15:10:36.401143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:41.284237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:41.284237+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"94","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.454638+0000","last_change":"2020-08-07T15:09:30.283753+0000","last_active":"2020-08-07T15:10:40.454638+0000","last_peered":"2020-08-07T15:10:40.454638+0000","last_clean":"2020-08-07T15:10:40.454638+0000","last_became_active":"2020-08-07T15:09:30.283176+0000","last_became_peered":"2020-08-07T15:09:30.283176+0000","last_unstale":"2020-08-07T15:10:40.454638+0000","last_undegraded":"2020-08-07T15:10:40.454638+0000","last_fullsized":"2020-08-07T15:10:40.454638+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:25.606757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:25.606757+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"182'2336","reported_seq":"2362","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.717515+0000","last_change":"2020-08-07T15:10:16.259807+0000","last_active":"2020-08-07T15:10:41.717515+0000","last_peered":"2020-08-07T15:10:41.717515+0000","last_clean":"2020-08-07T15:10:41.717515+0000","last_became_active":"2020-08-07T15:10:16.259365+0000","last_became_peered":"2020-08-07T15:10:16.259365+0000","last_unstale":"2020-08-07T15:10:41.717515+0000","last_undegraded":"2020-08-07T15:10:41.717515+0000","last_fullsized":"2020-08-07T15:10:41.717515+0000","mapping_epoch":164,"log_start":"182'2330","ondisk_log_start":"182'2330","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19136512,"num_objects":292,"num_object_clones":0,"num_object_copies":876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2336,"num_write_kb":18688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"181","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.460503+0000","last_change":"2020-08-07T15:10:25.463712+0000","last_active":"2020-08-07T15:10:40.460503+0000","last_peered":"2020-08-07T15:10:40.460503+0000","last_clean":"2020-08-07T15:10:40.460503+0000","last_became_active":"2020-08-07T15:06:23.073633+0000","last_became_peered":"2020-08-07T15:06:23.073633+0000","last_unstale":"2020-08-07T15:10:40.460503+0000","last_undegraded":"2020-08-07T15:10:40.460503+0000","last_fullsized":"2020-08-07T15:10:40.460503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:25.463628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:10:25.463628+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"139","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.457575+0000","last_change":"2020-08-07T15:10:27.642227+0000","last_active":"2020-08-07T15:10:40.457575+0000","last_peered":"2020-08-07T15:10:40.457575+0000","last_clean":"2020-08-07T15:10:40.457575+0000","last_became_active":"2020-08-07T15:08:46.710404+0000","last_became_peered":"2020-08-07T15:08:46.710404+0000","last_unstale":"2020-08-07T15:10:40.457575+0000","last_undegraded":"2020-08-07T15:10:40.457575+0000","last_fullsized":"2020-08-07T15:10:40.457575+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:10:27.642141+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:10:27.642141+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"178'1808","reported_seq":"1830","reported_epoch":"178","state":"active+clean","last_fresh":"2020-08-07T15:10:37.620364+0000","last_change":"2020-08-07T15:10:16.448360+0000","last_active":"2020-08-07T15:10:37.620364+0000","last_peered":"2020-08-07T15:10:37.620364+0000","last_clean":"2020-08-07T15:10:37.620364+0000","last_became_active":"2020-08-07T15:10:16.447826+0000","last_became_peered":"2020-08-07T15:10:16.447826+0000","last_unstale":"2020-08-07T15:10:37.620364+0000","last_undegraded":"2020-08-07T15:10:37.620364+0000","last_fullsized":"2020-08-07T15:10:37.620364+0000","mapping_epoch":164,"log_start":"178'1800","ondisk_log_start":"178'1800","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":14811136,"num_objects":226,"num_object_clones":0,"num_object_copies":678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1808,"num_write_kb":14464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"179","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.459304+0000","last_change":"2020-08-07T15:09:54.908283+0000","last_active":"2020-08-07T15:10:40.459304+0000","last_peered":"2020-08-07T15:10:40.459304+0000","last_clean":"2020-08-07T15:10:40.459304+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:10:40.459304+0000","last_undegraded":"2020-08-07T15:10:40.459304+0000","last_fullsized":"2020-08-07T15:10:40.459304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:54.908197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:54.908197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"213","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.400956+0000","last_change":"2020-08-07T15:10:07.302258+0000","last_active":"2020-08-07T15:10:36.400956+0000","last_peered":"2020-08-07T15:10:36.400956+0000","last_clean":"2020-08-07T15:10:36.400956+0000","last_became_active":"2020-08-07T15:09:30.201829+0000","last_became_peered":"2020-08-07T15:09:30.201829+0000","last_unstale":"2020-08-07T15:10:36.400956+0000","last_undegraded":"2020-08-07T15:10:36.400956+0000","last_fullsized":"2020-08-07T15:10:36.400956+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:07.302204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:07.302204+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"182'2352","reported_seq":"2376","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.708822+0000","last_change":"2020-08-07T15:10:16.447819+0000","last_active":"2020-08-07T15:10:41.708822+0000","last_peered":"2020-08-07T15:10:41.708822+0000","last_clean":"2020-08-07T15:10:41.708822+0000","last_became_active":"2020-08-07T15:10:16.447312+0000","last_became_peered":"2020-08-07T15:10:16.447312+0000","last_unstale":"2020-08-07T15:10:41.708822+0000","last_undegraded":"2020-08-07T15:10:41.708822+0000","last_fullsized":"2020-08-07T15:10:41.708822+0000","mapping_epoch":164,"log_start":"182'2344","ondisk_log_start":"182'2344","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19267584,"num_objects":294,"num_object_clones":0,"num_object_copies":882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2352,"num_write_kb":18816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"179","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.459154+0000","last_change":"2020-08-07T15:09:51.036281+0000","last_active":"2020-08-07T15:10:40.459154+0000","last_peered":"2020-08-07T15:10:40.459154+0000","last_clean":"2020-08-07T15:10:40.459154+0000","last_became_active":"2020-08-07T15:06:23.223755+0000","last_became_peered":"2020-08-07T15:06:23.223755+0000","last_unstale":"2020-08-07T15:10:40.459154+0000","last_undegraded":"2020-08-07T15:10:40.459154+0000","last_fullsized":"2020-08-07T15:10:40.459154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:51.036224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:51.036224+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"179","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.400704+0000","last_change":"2020-08-07T15:10:23.400748+0000","last_active":"2020-08-07T15:10:36.400704+0000","last_peered":"2020-08-07T15:10:36.400704+0000","last_clean":"2020-08-07T15:10:36.400704+0000","last_became_active":"2020-08-07T15:08:36.633406+0000","last_became_peered":"2020-08-07T15:08:36.633406+0000","last_unstale":"2020-08-07T15:10:36.400704+0000","last_undegraded":"2020-08-07T15:10:36.400704+0000","last_fullsized":"2020-08-07T15:10:36.400704+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:23.400672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:23.400672+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"182'2000","reported_seq":"2026","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.624472+0000","last_change":"2020-08-07T15:10:16.255436+0000","last_active":"2020-08-07T15:10:41.624472+0000","last_peered":"2020-08-07T15:10:41.624472+0000","last_clean":"2020-08-07T15:10:41.624472+0000","last_became_active":"2020-08-07T15:10:16.254967+0000","last_became_peered":"2020-08-07T15:10:16.254967+0000","last_unstale":"2020-08-07T15:10:41.624472+0000","last_undegraded":"2020-08-07T15:10:41.624472+0000","last_fullsized":"2020-08-07T15:10:41.624472+0000","mapping_epoch":164,"log_start":"182'1992","ondisk_log_start":"182'1992","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16384000,"num_objects":250,"num_object_clones":0,"num_object_copies":750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2000,"num_write_kb":16000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"211","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.459075+0000","last_change":"2020-08-07T15:09:55.879867+0000","last_active":"2020-08-07T15:10:40.459075+0000","last_peered":"2020-08-07T15:10:40.459075+0000","last_clean":"2020-08-07T15:10:40.459075+0000","last_became_active":"2020-08-07T15:08:36.627426+0000","last_became_peered":"2020-08-07T15:08:36.627426+0000","last_unstale":"2020-08-07T15:10:40.459075+0000","last_undegraded":"2020-08-07T15:10:40.459075+0000","last_fullsized":"2020-08-07T15:10:40.459075+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:55.879785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:55.879785+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"179","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.453047+0000","last_change":"2020-08-07T15:10:08.656952+0000","last_active":"2020-08-07T15:10:40.453047+0000","last_peered":"2020-08-07T15:10:40.453047+0000","last_clean":"2020-08-07T15:10:40.453047+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:10:40.453047+0000","last_undegraded":"2020-08-07T15:10:40.453047+0000","last_fullsized":"2020-08-07T15:10:40.453047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:08.656855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:10:08.656855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"182'2216","reported_seq":"2242","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.607555+0000","last_change":"2020-08-07T15:10:16.263356+0000","last_active":"2020-08-07T15:10:41.607555+0000","last_peered":"2020-08-07T15:10:41.607555+0000","last_clean":"2020-08-07T15:10:41.607555+0000","last_became_active":"2020-08-07T15:10:16.262971+0000","last_became_peered":"2020-08-07T15:10:16.262971+0000","last_unstale":"2020-08-07T15:10:41.607555+0000","last_undegraded":"2020-08-07T15:10:41.607555+0000","last_fullsized":"2020-08-07T15:10:41.607555+0000","mapping_epoch":164,"log_start":"182'2206","ondisk_log_start":"182'2206","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18153472,"num_objects":277,"num_object_clones":0,"num_object_copies":831,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2216,"num_write_kb":17728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"200","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.457332+0000","last_change":"2020-08-07T15:10:24.576099+0000","last_active":"2020-08-07T15:10:40.457332+0000","last_peered":"2020-08-07T15:10:40.457332+0000","last_clean":"2020-08-07T15:10:40.457332+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:10:40.457332+0000","last_undegraded":"2020-08-07T15:10:40.457332+0000","last_fullsized":"2020-08-07T15:10:40.457332+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:24.576023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:10:24.576023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"227","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.459082+0000","last_change":"2020-08-07T15:10:11.912614+0000","last_active":"2020-08-07T15:10:40.459082+0000","last_peered":"2020-08-07T15:10:40.459082+0000","last_clean":"2020-08-07T15:10:40.459082+0000","last_became_active":"2020-08-07T15:08:36.628365+0000","last_became_peered":"2020-08-07T15:08:36.628365+0000","last_unstale":"2020-08-07T15:10:40.459082+0000","last_undegraded":"2020-08-07T15:10:40.459082+0000","last_fullsized":"2020-08-07T15:10:40.459082+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:11.912550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:11.912550+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.5","version":"182'2008","reported_seq":"2034","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.558443+0000","last_change":"2020-08-07T15:10:16.447870+0000","last_active":"2020-08-07T15:10:41.558443+0000","last_peered":"2020-08-07T15:10:41.558443+0000","last_clean":"2020-08-07T15:10:41.558443+0000","last_became_active":"2020-08-07T15:10:16.447478+0000","last_became_peered":"2020-08-07T15:10:16.447478+0000","last_unstale":"2020-08-07T15:10:41.558443+0000","last_undegraded":"2020-08-07T15:10:41.558443+0000","last_fullsized":"2020-08-07T15:10:41.558443+0000","mapping_epoch":164,"log_start":"182'2002","ondisk_log_start":"182'2002","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16449536,"num_objects":251,"num_object_clones":0,"num_object_copies":753,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2008,"num_write_kb":16064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"179","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.453200+0000","last_change":"2020-08-07T15:09:52.637867+0000","last_active":"2020-08-07T15:10:40.453200+0000","last_peered":"2020-08-07T15:10:40.453200+0000","last_clean":"2020-08-07T15:10:40.453200+0000","last_became_active":"2020-08-07T15:06:23.072995+0000","last_became_peered":"2020-08-07T15:06:23.072995+0000","last_unstale":"2020-08-07T15:10:40.453200+0000","last_undegraded":"2020-08-07T15:10:40.453200+0000","last_fullsized":"2020-08-07T15:10:40.453200+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:52.637791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:09:52.637791+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"177","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.452968+0000","last_change":"2020-08-07T15:10:11.720217+0000","last_active":"2020-08-07T15:10:40.452968+0000","last_peered":"2020-08-07T15:10:40.452968+0000","last_clean":"2020-08-07T15:10:40.452968+0000","last_became_active":"2020-08-07T15:08:36.618164+0000","last_became_peered":"2020-08-07T15:08:36.618164+0000","last_unstale":"2020-08-07T15:10:40.452968+0000","last_undegraded":"2020-08-07T15:10:40.452968+0000","last_fullsized":"2020-08-07T15:10:40.452968+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:11.720143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:11.720143+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"178'1800","reported_seq":"1822","reported_epoch":"178","state":"active+clean","last_fresh":"2020-08-07T15:10:37.652956+0000","last_change":"2020-08-07T15:10:16.443153+0000","last_active":"2020-08-07T15:10:37.652956+0000","last_peered":"2020-08-07T15:10:37.652956+0000","last_clean":"2020-08-07T15:10:37.652956+0000","last_became_active":"2020-08-07T15:10:16.442674+0000","last_became_peered":"2020-08-07T15:10:16.442674+0000","last_unstale":"2020-08-07T15:10:37.652956+0000","last_undegraded":"2020-08-07T15:10:37.652956+0000","last_fullsized":"2020-08-07T15:10:37.652956+0000","mapping_epoch":164,"log_start":"178'1792","ondisk_log_start":"178'1792","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":14745600,"num_objects":225,"num_object_clones":0,"num_object_copies":675,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1800,"num_write_kb":14400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"176","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.459314+0000","last_change":"2020-08-07T15:09:37.017383+0000","last_active":"2020-08-07T15:10:40.459314+0000","last_peered":"2020-08-07T15:10:40.459314+0000","last_clean":"2020-08-07T15:10:40.459314+0000","last_became_active":"2020-08-07T15:06:46.198562+0000","last_became_peered":"2020-08-07T15:06:46.198562+0000","last_unstale":"2020-08-07T15:10:40.459314+0000","last_undegraded":"2020-08-07T15:10:40.459314+0000","last_fullsized":"2020-08-07T15:10:40.459314+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:37.017307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:37.017307+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"244","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.401366+0000","last_change":"2020-08-07T15:09:30.208873+0000","last_active":"2020-08-07T15:10:36.401366+0000","last_peered":"2020-08-07T15:10:36.401366+0000","last_clean":"2020-08-07T15:10:36.401366+0000","last_became_active":"2020-08-07T15:09:30.208345+0000","last_became_peered":"2020-08-07T15:09:30.208345+0000","last_unstale":"2020-08-07T15:10:36.401366+0000","last_undegraded":"2020-08-07T15:10:36.401366+0000","last_fullsized":"2020-08-07T15:10:36.401366+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.381325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.381325+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.e","version":"182'2112","reported_seq":"2137","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.704631+0000","last_change":"2020-08-07T15:10:16.448334+0000","last_active":"2020-08-07T15:10:41.704631+0000","last_peered":"2020-08-07T15:10:41.704631+0000","last_clean":"2020-08-07T15:10:41.704631+0000","last_became_active":"2020-08-07T15:10:16.447541+0000","last_became_peered":"2020-08-07T15:10:16.447541+0000","last_unstale":"2020-08-07T15:10:41.704631+0000","last_undegraded":"2020-08-07T15:10:41.704631+0000","last_fullsized":"2020-08-07T15:10:41.704631+0000","mapping_epoch":164,"log_start":"182'2107","ondisk_log_start":"182'2107","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":17301504,"num_objects":264,"num_object_clones":0,"num_object_copies":792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2112,"num_write_kb":16896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"154","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.453139+0000","last_change":"2020-08-07T15:10:10.715147+0000","last_active":"2020-08-07T15:10:40.453139+0000","last_peered":"2020-08-07T15:10:40.453139+0000","last_clean":"2020-08-07T15:10:40.453139+0000","last_became_active":"2020-08-07T15:07:33.166373+0000","last_became_peered":"2020-08-07T15:07:33.166373+0000","last_unstale":"2020-08-07T15:10:40.453139+0000","last_undegraded":"2020-08-07T15:10:40.453139+0000","last_fullsized":"2020-08-07T15:10:40.453139+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.715051+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"182'2488","reported_seq":"2512","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.637571+0000","last_change":"2020-08-07T15:10:16.448023+0000","last_active":"2020-08-07T15:10:41.637571+0000","last_peered":"2020-08-07T15:10:41.637571+0000","last_clean":"2020-08-07T15:10:41.637571+0000","last_became_active":"2020-08-07T15:10:16.447526+0000","last_became_peered":"2020-08-07T15:10:16.447526+0000","last_unstale":"2020-08-07T15:10:41.637571+0000","last_undegraded":"2020-08-07T15:10:41.637571+0000","last_fullsized":"2020-08-07T15:10:41.637571+0000","mapping_epoch":164,"log_start":"182'2482","ondisk_log_start":"182'2482","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":20381696,"num_objects":311,"num_object_clones":0,"num_object_copies":933,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2488,"num_write_kb":19904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"151","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.452822+0000","last_change":"2020-08-07T15:09:09.691838+0000","last_active":"2020-08-07T15:10:40.452822+0000","last_peered":"2020-08-07T15:10:40.452822+0000","last_clean":"2020-08-07T15:10:40.452822+0000","last_became_active":"2020-08-07T15:07:33.165407+0000","last_became_peered":"2020-08-07T15:07:33.165407+0000","last_unstale":"2020-08-07T15:10:40.452822+0000","last_undegraded":"2020-08-07T15:10:40.452822+0000","last_fullsized":"2020-08-07T15:10:40.452822+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:09:09.691753+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.d","version":"182'2216","reported_seq":"2242","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.561687+0000","last_change":"2020-08-07T15:10:16.263289+0000","last_active":"2020-08-07T15:10:41.561687+0000","last_peered":"2020-08-07T15:10:41.561687+0000","last_clean":"2020-08-07T15:10:41.561687+0000","last_became_active":"2020-08-07T15:10:16.262869+0000","last_became_peered":"2020-08-07T15:10:16.262869+0000","last_unstale":"2020-08-07T15:10:41.561687+0000","last_undegraded":"2020-08-07T15:10:41.561687+0000","last_fullsized":"2020-08-07T15:10:41.561687+0000","mapping_epoch":164,"log_start":"182'2208","ondisk_log_start":"182'2208","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18153472,"num_objects":277,"num_object_clones":0,"num_object_copies":831,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2216,"num_write_kb":17728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"218","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.459158+0000","last_change":"2020-08-07T15:10:31.122527+0000","last_active":"2020-08-07T15:10:40.459158+0000","last_peered":"2020-08-07T15:10:40.459158+0000","last_clean":"2020-08-07T15:10:40.459158+0000","last_became_active":"2020-08-07T15:07:33.160417+0000","last_became_peered":"2020-08-07T15:07:33.160417+0000","last_unstale":"2020-08-07T15:10:40.459158+0000","last_undegraded":"2020-08-07T15:10:40.459158+0000","last_fullsized":"2020-08-07T15:10:40.459158+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:31.122435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:31.122435+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"182'2192","reported_seq":"2218","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.691962+0000","last_change":"2020-08-07T15:10:16.256000+0000","last_active":"2020-08-07T15:10:41.691962+0000","last_peered":"2020-08-07T15:10:41.691962+0000","last_clean":"2020-08-07T15:10:41.691962+0000","last_became_active":"2020-08-07T15:10:16.255356+0000","last_became_peered":"2020-08-07T15:10:16.255356+0000","last_unstale":"2020-08-07T15:10:41.691962+0000","last_undegraded":"2020-08-07T15:10:41.691962+0000","last_fullsized":"2020-08-07T15:10:41.691962+0000","mapping_epoch":164,"log_start":"182'2185","ondisk_log_start":"182'2185","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":17956864,"num_objects":274,"num_object_clones":0,"num_object_copies":822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2192,"num_write_kb":17536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"173","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.459322+0000","last_change":"2020-08-07T15:10:10.962565+0000","last_active":"2020-08-07T15:10:40.459322+0000","last_peered":"2020-08-07T15:10:40.459322+0000","last_clean":"2020-08-07T15:10:40.459322+0000","last_became_active":"2020-08-07T15:08:41.680448+0000","last_became_peered":"2020-08-07T15:08:41.680448+0000","last_unstale":"2020-08-07T15:10:40.459322+0000","last_undegraded":"2020-08-07T15:10:40.459322+0000","last_fullsized":"2020-08-07T15:10:40.459322+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:10:10.962480+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.962480+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"182'2193","reported_seq":"2215","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.725314+0000","last_change":"2020-08-07T15:10:16.444776+0000","last_active":"2020-08-07T15:10:41.725314+0000","last_peered":"2020-08-07T15:10:41.725314+0000","last_clean":"2020-08-07T15:10:41.725314+0000","last_became_active":"2020-08-07T15:10:16.444316+0000","last_became_peered":"2020-08-07T15:10:16.444316+0000","last_unstale":"2020-08-07T15:10:41.725314+0000","last_undegraded":"2020-08-07T15:10:41.725314+0000","last_fullsized":"2020-08-07T15:10:41.725314+0000","mapping_epoch":164,"log_start":"182'2186","ondisk_log_start":"182'2186","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":17965056,"num_objects":275,"num_object_clones":0,"num_object_copies":825,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2193,"num_write_kb":17544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"140","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.454781+0000","last_change":"2020-08-07T15:09:36.687393+0000","last_active":"2020-08-07T15:10:40.454781+0000","last_peered":"2020-08-07T15:10:40.454781+0000","last_clean":"2020-08-07T15:10:40.454781+0000","last_became_active":"2020-08-07T15:09:30.204655+0000","last_became_peered":"2020-08-07T15:09:30.204655+0000","last_unstale":"2020-08-07T15:10:40.454781+0000","last_undegraded":"2020-08-07T15:10:40.454781+0000","last_fullsized":"2020-08-07T15:10:40.454781+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:36.687314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:36.687314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"182'1944","reported_seq":"1965","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.725310+0000","last_change":"2020-08-07T15:10:16.444970+0000","last_active":"2020-08-07T15:10:41.725310+0000","last_peered":"2020-08-07T15:10:41.725310+0000","last_clean":"2020-08-07T15:10:41.725310+0000","last_became_active":"2020-08-07T15:10:16.444294+0000","last_became_peered":"2020-08-07T15:10:16.444294+0000","last_unstale":"2020-08-07T15:10:41.725310+0000","last_undegraded":"2020-08-07T15:10:41.725310+0000","last_fullsized":"2020-08-07T15:10:41.725310+0000","mapping_epoch":164,"log_start":"182'1936","ondisk_log_start":"182'1936","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":15925248,"num_objects":243,"num_object_clones":0,"num_object_copies":729,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1944,"num_write_kb":15552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"209","reported_epoch":"177","state":"active+clean","last_fresh":"2020-08-07T15:10:36.401336+0000","last_change":"2020-08-07T15:09:08.399507+0000","last_active":"2020-08-07T15:10:36.401336+0000","last_peered":"2020-08-07T15:10:36.401336+0000","last_clean":"2020-08-07T15:10:36.401336+0000","last_became_active":"2020-08-07T15:08:41.689435+0000","last_became_peered":"2020-08-07T15:08:41.689435+0000","last_unstale":"2020-08-07T15:10:36.401336+0000","last_undegraded":"2020-08-07T15:10:36.401336+0000","last_fullsized":"2020-08-07T15:10:36.401336+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:08.399439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:08.399439+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"182'2048","reported_seq":"2074","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.710979+0000","last_change":"2020-08-07T15:10:16.256758+0000","last_active":"2020-08-07T15:10:41.710979+0000","last_peered":"2020-08-07T15:10:41.710979+0000","last_clean":"2020-08-07T15:10:41.710979+0000","last_became_active":"2020-08-07T15:10:16.256419+0000","last_became_peered":"2020-08-07T15:10:16.256419+0000","last_unstale":"2020-08-07T15:10:41.710979+0000","last_undegraded":"2020-08-07T15:10:41.710979+0000","last_fullsized":"2020-08-07T15:10:41.710979+0000","mapping_epoch":164,"log_start":"182'2041","ondisk_log_start":"182'2041","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16777216,"num_objects":256,"num_object_clones":0,"num_object_copies":768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2048,"num_write_kb":16384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"104","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.459290+0000","last_change":"2020-08-07T15:10:29.055136+0000","last_active":"2020-08-07T15:10:40.459290+0000","last_peered":"2020-08-07T15:10:40.459290+0000","last_clean":"2020-08-07T15:10:40.459290+0000","last_became_active":"2020-08-07T15:08:46.710059+0000","last_became_peered":"2020-08-07T15:08:46.710059+0000","last_unstale":"2020-08-07T15:10:40.459290+0000","last_undegraded":"2020-08-07T15:10:40.459290+0000","last_fullsized":"2020-08-07T15:10:40.459290+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:29.055048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:29.055048+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"182'2288","reported_seq":"2314","reported_epoch":"182","state":"active+clean","last_fresh":"2020-08-07T15:10:41.689743+0000","last_change":"2020-08-07T15:10:16.253662+0000","last_active":"2020-08-07T15:10:41.689743+0000","last_peered":"2020-08-07T15:10:41.689743+0000","last_clean":"2020-08-07T15:10:41.689743+0000","last_became_active":"2020-08-07T15:10:16.253105+0000","last_became_peered":"2020-08-07T15:10:16.253105+0000","last_unstale":"2020-08-07T15:10:41.689743+0000","last_undegraded":"2020-08-07T15:10:41.689743+0000","last_fullsized":"2020-08-07T15:10:41.689743+0000","mapping_epoch":164,"log_start":"182'2278","ondisk_log_start":"182'2278","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":18743296,"num_objects":286,"num_object_clones":0,"num_object_copies":858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2288,"num_write_kb":18304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"126","reported_epoch":"181","state":"active+clean","last_fresh":"2020-08-07T15:10:40.452966+0000","last_change":"2020-08-07T15:10:37.601162+0000","last_active":"2020-08-07T15:10:40.452966+0000","last_peered":"2020-08-07T15:10:40.452966+0000","last_clean":"2020-08-07T15:10:40.452966+0000","last_became_active":"2020-08-07T15:08:41.677944+0000","last_became_peered":"2020-08-07T15:08:41.677944+0000","last_unstale":"2020-08-07T15:10:40.452966+0000","last_undegraded":"2020-08-07T15:10:40.452966+0000","last_fullsized":"2020-08-07T15:10:40.452966+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:37.601099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:37.601099+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":277815296,"num_objects":4240,"num_object_clones":0,"num_object_copies":12720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33913,"num_write_kb":271304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":839819264,"data_stored":839819264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":63,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542198,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1106488,"kb_used_data":57904,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93265352,"statfs":{"total":96636764160,"available":95503720448,"internally_reserved":1073741824,"allocated":59293696,"data_stored":58624399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":43.911000000000001,"5min":9.5549999999999997,"15min":3.7799999999999998},"min":{"1min":0.50800000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":856.07100000000003,"5min":856.07100000000003,"15min":856.07100000000003},"last":0.42199999999999999},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.026,"15min":0.98399999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.8899999999999999,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":0.79400000000000004}]},{"osd":1,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.335,"15min":1.3080000000000001},"min":{"1min":0.64200000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.9199999999999999,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":2.395},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.141,"15min":1.024},"min":{"1min":0.61799999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":4.0060000000000002,"5min":4.0060000000000002,"15min":4.0060000000000002},"last":2.4950000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.304,"15min":1.1859999999999999},"min":{"1min":0.747,"5min":0.628,"15min":0.628},"max":{"1min":2.0249999999999999,"5min":5.6619999999999999,"15min":5.6619999999999999},"last":2.4359999999999999},{"interface":"front","average":{"1min":1.512,"5min":1.3380000000000001,"15min":1.1910000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.6619999999999999,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":0.54300000000000004}]},{"osd":3,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":16.18,"15min":6.508},"min":{"1min":0.64700000000000002,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.9710000000000001,"5min":917.38699999999994,"15min":917.38699999999994},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.196,"5min":4.9649999999999999,"15min":2.5499999999999998},"min":{"1min":0.77200000000000002,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.3359999999999999,"5min":333.59199999999998,"15min":333.59199999999998},"last":0.84799999999999998}]},{"osd":4,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.464,"15min":1.3380000000000001},"min":{"1min":0.63400000000000001,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":3.9750000000000001,"5min":4.835,"15min":4.835},"last":0.65400000000000003},{"interface":"front","average":{"1min":1.306,"5min":11.518000000000001,"15min":4.7140000000000004},"min":{"1min":0.69699999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":3.0329999999999999,"5min":866.24099999999999,"15min":866.24099999999999},"last":0.92500000000000004}]},{"osd":5,"last update":"Fri Aug 7 15:10:27 2020","interfaces":[{"interface":"back","average":{"1min":39.496000000000002,"5min":23.838000000000001,"15min":32.950000000000003},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":678.76099999999997,"5min":831.88099999999997,"15min":831.88099999999997},"last":2.468},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.823,"15min":1.6040000000000001},"min":{"1min":0.84999999999999998,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.4060000000000001,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":2.5219999999999998}]},{"osd":7,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":3.9929999999999999,"15min":2.343},"min":{"1min":0.89900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":3.0049999999999999,"5min":224.14500000000001,"15min":224.14500000000001},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.6899999999999999,"15min":1.5680000000000001},"min":{"1min":1.113,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.2919999999999998,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":3.097}]}]},{"osd":7,"up_from":13,"seq":55834574904,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1134112,"kb_used_data":85528,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93237728,"statfs":{"total":96636764160,"available":95475433472,"internally_reserved":1073741824,"allocated":87580672,"data_stored":86881135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1459999999999999,"15min":1.27},"min":{"1min":0.58099999999999996,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":3.2090000000000001,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":1.3160000000000001},{"interface":"front","average":{"1min":30.457000000000001,"5min":9.8149999999999995,"15min":4.0380000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":529.88699999999994,"5min":529.88699999999994,"15min":529.88699999999994},"last":0.92400000000000004}]},{"osd":1,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":5.758,"15min":2.5579999999999998},"min":{"1min":0.47699999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.7290000000000001,"5min":547.65300000000002,"15min":547.65300000000002},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.4159999999999999,"5min":4.9189999999999996,"15min":2.5640000000000001},"min":{"1min":0.57399999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":5.4960000000000004,"5min":372.50799999999998,"15min":372.50799999999998},"last":2.3199999999999998}]},{"osd":2,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.262,"15min":1.2170000000000001},"min":{"1min":0.56799999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":3.4060000000000001,"5min":5.2400000000000002,"15min":5.2400000000000002},"last":1.365},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2170000000000001,"15min":1.2529999999999999},"min":{"1min":0.61899999999999999,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.4569999999999999,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":1.1799999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":42.097999999999999,"5min":9.5510000000000002,"15min":4.2009999999999996},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":698.19899999999996,"5min":698.19899999999996,"15min":698.19899999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.3160000000000001,"15min":1.3640000000000001},"min":{"1min":0.80500000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.8959999999999999,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":4.1779999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.3979999999999999,"15min":1.5740000000000001},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.885,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":1.446},{"interface":"front","average":{"1min":1.353,"5min":1.5620000000000001,"15min":1.635},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":2.0699999999999998,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.3460000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.2589999999999999,"15min":1.329},"min":{"1min":0.66500000000000004,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":3.0009999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.87},{"interface":"front","average":{"1min":1.298,"5min":1.532,"15min":1.6879999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.9419999999999999,"5min":5.327,"15min":5.327},"last":1.2529999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.466,"15min":1.444},"min":{"1min":0.93400000000000005,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":3.2229999999999999,"5min":3.3809999999999998,"15min":3.3809999999999998},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":8.2379999999999995,"15min":14.962999999999999},"min":{"1min":0.88700000000000001,"5min":0.87,"15min":0.87},"max":{"1min":2.3330000000000002,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.6839999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1136296,"kb_used_data":87712,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93235544,"statfs":{"total":96636764160,"available":95473197056,"internally_reserved":1073741824,"allocated":89817088,"data_stored":89117551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":13.122,"5min":16.731999999999999,"15min":28.367999999999999},"min":{"1min":0.47199999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":211.63399999999999,"5min":497.04000000000002,"15min":497.04000000000002},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.645,"5min":14.757999999999999,"15min":5.673},"min":{"1min":0.57399999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":8.8849999999999998,"5min":993.84500000000003,"15min":993.84500000000003},"last":0.84999999999999998}]},{"osd":2,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":54.984000000000002,"5min":11.816000000000001,"15min":4.5380000000000003},"min":{"1min":0.57199999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":959.36900000000003,"5min":959.36900000000003,"15min":959.36900000000003},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.72,"5min":1.357,"15min":1.2549999999999999},"min":{"1min":0.53600000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":5.4530000000000003,"5min":6.4029999999999996,"15min":6.4029999999999996},"last":1.6479999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":9.2420000000000009,"15min":3.9540000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.3879999999999999,"5min":828.31600000000003,"15min":828.31600000000003},"last":1.165},{"interface":"front","average":{"1min":1.2130000000000001,"5min":10.119999999999999,"15min":4.3419999999999996},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.5659999999999998,"5min":905.38300000000004,"15min":905.38300000000004},"last":1.2030000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":49.530999999999999,"5min":16.422999999999998,"15min":6.3300000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":872.58500000000004,"5min":872.58500000000004,"15min":872.58500000000004},"last":1.127},{"interface":"front","average":{"1min":2.2080000000000002,"5min":1.5409999999999999,"15min":1.3129999999999999},"min":{"1min":0.75800000000000001,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":12.640000000000001,"5min":12.640000000000001,"15min":12.640000000000001},"last":1.2470000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":6.0919999999999996,"15min":10.702},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.056,"5min":213.63800000000001,"15min":213.63800000000001},"last":1.2849999999999999},{"interface":"front","average":{"1min":2.4630000000000001,"5min":12.974,"15min":5.1180000000000003},"min":{"1min":0.93400000000000005,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":12.677,"5min":875.44600000000003,"15min":875.44600000000003},"last":1.5660000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:10:32 2020","interfaces":[{"interface":"back","average":{"1min":2.9340000000000002,"5min":1.863,"15min":1.655},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":22.402000000000001,"5min":22.402000000000001,"15min":22.402000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.3160000000000001,"5min":3.589,"15min":2.1030000000000002},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.3980000000000001,"5min":207.137,"15min":207.137},"last":1.5349999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":2.0910000000000002,"5min":1.7829999999999999,"15min":1.671},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":5.4960000000000004,"5min":6.444,"15min":6.444},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.655,"15min":1.538},"min":{"1min":0.78700000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.0219999999999998,"5min":10.752000000000001,"15min":10.752000000000001},"last":1.7330000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1184608,"kb_used_data":136024,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93187232,"statfs":{"total":96636764160,"available":95423725568,"internally_reserved":1073741824,"allocated":139288576,"data_stored":138589039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":24.291,"5min":21.719999999999999,"15min":7.891},"min":{"1min":0.52700000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":432.899,"5min":845.57100000000003,"15min":845.57100000000003},"last":0.51500000000000001},{"interface":"front","average":{"1min":1.752,"5min":1.181,"15min":1.0149999999999999},"min":{"1min":0.62,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":9.3650000000000002,"5min":9.3650000000000002,"15min":9.3650000000000002},"last":1.127}]},{"osd":2,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.0249999999999999,"15min":0.95999999999999996},"min":{"1min":0.52200000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.972,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.296,"5min":16.190999999999999,"15min":27.222999999999999},"min":{"1min":0.55900000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.8519999999999999,"5min":606.18499999999995,"15min":606.18499999999995},"last":1.194}]},{"osd":3,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":1.7809999999999999,"5min":1.2350000000000001,"15min":1.0349999999999999},"min":{"1min":0.54300000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":9.3360000000000003,"5min":9.3360000000000003,"15min":9.3360000000000003},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.677,"5min":1.4179999999999999,"15min":1.2509999999999999},"min":{"1min":0.63600000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":11.19,"5min":11.19,"15min":11.19},"last":0.77500000000000002}]},{"osd":4,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.355,"15min":1.2290000000000001},"min":{"1min":0.68300000000000005,"5min":0.63,"15min":0.63},"max":{"1min":3.9729999999999999,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.4510000000000001,"15min":1.3220000000000001},"min":{"1min":0.83899999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.012,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":0.89000000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":9.0800000000000001,"15min":3.8439999999999999},"min":{"1min":0.75600000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":3.883,"5min":501.267,"15min":501.267},"last":3.5579999999999998},{"interface":"front","average":{"1min":1.669,"5min":1.3740000000000001,"15min":1.1950000000000001},"min":{"1min":0.752,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":4.0140000000000002,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":3.5790000000000002}]},{"osd":6,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":4.3310000000000004,"15min":7.1859999999999999},"min":{"1min":0.84999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.1400000000000001,"5min":124.738,"15min":124.738},"last":3.1899999999999999},{"interface":"front","average":{"1min":1.611,"5min":8.4019999999999992,"15min":15.236000000000001},"min":{"1min":0.95899999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":3.9119999999999999,"5min":294.94,"15min":294.94},"last":5.1799999999999997}]},{"osd":7,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":1.647,"5min":1.571,"15min":1.581},"min":{"1min":0.98899999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":3.9369999999999998,"5min":6.468,"15min":6.468},"last":4.0010000000000003},{"interface":"front","average":{"1min":46.985999999999997,"5min":10.561,"15min":4.444},"min":{"1min":0.88200000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":859.82399999999996,"5min":859.82399999999996,"15min":859.82399999999996},"last":1.3799999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1150076,"kb_used_data":101492,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93221764,"statfs":{"total":96636764160,"available":95459086336,"internally_reserved":1073741824,"allocated":103927808,"data_stored":103220713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:10:21 2020","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.268,"15min":1.167},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":3.706,"5min":8.4930000000000003,"15min":8.4930000000000003},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.1910000000000001,"15min":1.0469999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":5.194,"5min":5.194,"15min":5.194},"last":0.46800000000000003}]},{"osd":1,"last update":"Fri Aug 7 15:10:21 2020","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.6970000000000001,"15min":1.2929999999999999},"min":{"1min":0.52000000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":5.5919999999999996,"5min":18.672000000000001,"15min":18.672000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":1.264,"5min":1.3320000000000001,"15min":1.3500000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":3.7970000000000002,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":0.70499999999999996}]},{"osd":3,"last update":"Fri Aug 7 15:10:21 2020","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.2949999999999999,"15min":1.2070000000000001},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":3.1970000000000001,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.796,"5min":8.0990000000000002,"15min":3.7370000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":7.9349999999999996,"5min":501.75599999999997,"15min":501.75599999999997},"last":0.85399999999999998}]},{"osd":4,"last update":"Fri Aug 7 15:10:21 2020","interfaces":[{"interface":"back","average":{"1min":1.6379999999999999,"5min":1.508,"15min":1.242},"min":{"1min":0.64500000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":3.9409999999999998,"5min":19.783000000000001,"15min":19.783000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.6020000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":3.7919999999999998,"5min":3.7919999999999998,"15min":3.7919999999999998},"last":0.92000000000000004}]},{"osd":5,"last update":"Fri Aug 7 15:10:21 2020","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.3520000000000001,"15min":1.292},"min":{"1min":0.80900000000000005,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":5.2220000000000004,"5min":5.2220000000000004,"15min":5.2220000000000004},"last":1.1259999999999999},{"interface":"front","average":{"1min":30.856000000000002,"5min":25.364999999999998,"15min":25.471},"min":{"1min":0.83699999999999997,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":584.69000000000005,"5min":627.53300000000002,"15min":627.53300000000002},"last":0.73199999999999998}]},{"osd":6,"last update":"Fri Aug 7 15:10:32 2020","interfaces":[{"interface":"back","average":{"1min":1.8,"5min":1.6539999999999999,"15min":1.365},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":4.1639999999999997,"5min":20.035,"15min":20.035},"last":1.502},{"interface":"front","average":{"1min":1.6459999999999999,"5min":9.9629999999999992,"15min":4.2009999999999996},"min":{"1min":0.95499999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":5.3019999999999996,"5min":751.70000000000005,"15min":751.70000000000005},"last":0.80200000000000005}]},{"osd":7,"last update":"Fri Aug 7 15:10:21 2020","interfaces":[{"interface":"back","average":{"1min":1.6140000000000001,"5min":12.569000000000001,"15min":5.351},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":3.7709999999999999,"5min":501.91699999999997,"15min":501.91699999999997},"last":1.702},{"interface":"front","average":{"1min":1.6639999999999999,"5min":1.9530000000000001,"15min":1.6479999999999999},"min":{"1min":0.92900000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":5.7220000000000004,"5min":35.646999999999998,"15min":35.646999999999998},"last":1.5389999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574903,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1099764,"kb_used_data":51180,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93272076,"statfs":{"total":96636764160,"available":95510605824,"internally_reserved":1073741824,"allocated":52408320,"data_stored":51701225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.3,"15min":1.268},"min":{"1min":0.498,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":3.617,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.369,"15min":1.5469999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":3.6520000000000001,"5min":13.067,"15min":13.067},"last":1.488}]},{"osd":1,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":4.4779999999999998,"15min":2.758},"min":{"1min":0.59099999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.847,"5min":270.68700000000001,"15min":270.68700000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":1.109,"5min":1.8580000000000001,"15min":1.96},"min":{"1min":0.54100000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.0550000000000002,"5min":14.349,"15min":14.349},"last":1.2629999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.3169999999999999,"15min":1.204},"min":{"1min":0.56999999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.6930000000000001,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":2.024,"15min":1.798},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.9420000000000002,"5min":51.530000000000001,"15min":51.530000000000001},"last":1.0980000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":35.152000000000001,"5min":17.465,"15min":6.9530000000000003},"min":{"1min":0.64100000000000001,"5min":0.62,"15min":0.62},"max":{"1min":646.26700000000005,"5min":646.26700000000005,"15min":646.26700000000005},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.5269999999999999,"5min":19.353999999999999,"15min":27.850999999999999},"min":{"1min":0.69699999999999995,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":3.911,"5min":677.88599999999997,"15min":677.88599999999997},"last":1.3440000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":23.408000000000001,"5min":5.9699999999999998,"15min":3.0019999999999998},"min":{"1min":0.90000000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":403.88400000000001,"5min":403.88400000000001,"15min":403.88400000000001},"last":1.4630000000000001},{"interface":"front","average":{"1min":2.4209999999999998,"5min":1.931,"15min":1.823},"min":{"1min":1.0289999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":8.9120000000000008,"5min":8.9120000000000008,"15min":8.9120000000000008},"last":1.5169999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.6930000000000001,"5min":1.659,"15min":1.5740000000000001},"min":{"1min":0.79100000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":3.9500000000000002,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":1.381},{"interface":"front","average":{"1min":39.466999999999999,"5min":13.435,"15min":5.8029999999999999},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":688.15999999999997,"5min":688.15999999999997,"15min":688.15999999999997},"last":0.83099999999999996}]},{"osd":7,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.7809999999999999,"15min":2.0350000000000001},"min":{"1min":0.91000000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.2410000000000001,"5min":5.734,"15min":5.734},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.857,"15min":1.9370000000000001},"min":{"1min":0.84499999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.4140000000000001,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":0.91800000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574903,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1183872,"kb_used_data":135288,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93187968,"statfs":{"total":96636764160,"available":95424479232,"internally_reserved":1073741824,"allocated":138534912,"data_stored":137835375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:10:24 2020","interfaces":[{"interface":"back","average":{"1min":26.023,"5min":6.04,"15min":2.7999999999999998},"min":{"1min":0.48299999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":399.73700000000002,"5min":399.73700000000002,"15min":399.73700000000002},"last":1.087},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":0.91500000000000004},"min":{"1min":0.54600000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":2.665,"5min":6.3970000000000002,"15min":6.3970000000000002},"last":0.58299999999999996}]},{"osd":1,"last update":"Fri Aug 7 15:10:24 2020","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":5.7629999999999999,"15min":2.823},"min":{"1min":0.52900000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.8109999999999999,"5min":320.67000000000002,"15min":320.67000000000002},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.177,"5min":7.5300000000000002,"15min":3.4279999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.7200000000000002,"5min":689.846,"15min":689.846},"last":1.2070000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:10:24 2020","interfaces":[{"interface":"back","average":{"1min":1.462,"5min":1.4019999999999999,"15min":1.4350000000000001},"min":{"1min":0.76600000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":3.4830000000000001,"5min":6.8730000000000002,"15min":6.8730000000000002},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.556,"5min":1.524,"15min":1.518},"min":{"1min":0.88600000000000001,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":3.5659999999999998,"5min":7.1360000000000001,"15min":7.1360000000000001},"last":0.999}]},{"osd":3,"last update":"Fri Aug 7 15:10:24 2020","interfaces":[{"interface":"back","average":{"1min":43.743000000000002,"5min":12.128,"15min":5.048},"min":{"1min":0.82399999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":638.92100000000005,"5min":638.92100000000005,"15min":638.92100000000005},"last":1.238},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.367,"15min":1.2889999999999999},"min":{"1min":0.75800000000000001,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.1200000000000001,"5min":6.431,"15min":6.431},"last":1.264}]},{"osd":5,"last update":"Fri Aug 7 15:10:24 2020","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":10.446,"15min":4.5670000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.2559999999999998,"5min":990.71600000000001,"15min":990.71600000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":1.204,"5min":1.526,"15min":1.593},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.669,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":1.5780000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:10:32 2020","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.661,"15min":1.7470000000000001},"min":{"1min":0.73399999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.7000000000000002,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":0.87},{"interface":"front","average":{"1min":1.282,"5min":8.5039999999999996,"15min":15.574},"min":{"1min":0.93700000000000006,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.0870000000000002,"5min":389.14600000000002,"15min":389.14600000000002},"last":1.391}]},{"osd":7,"last update":"Fri Aug 7 15:10:24 2020","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":3.8079999999999998,"15min":2.2080000000000002},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.7290000000000001,"5min":252.798,"15min":252.798},"last":0.92000000000000004},{"interface":"front","average":{"1min":21.542000000000002,"5min":14.154,"15min":5.8150000000000004},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":326.327,"5min":501.93000000000001,"15min":501.93000000000001},"last":1.145}]}]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1205260,"kb_used_data":156676,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93166580,"statfs":{"total":96636764160,"available":95402577920,"internally_reserved":1073741824,"allocated":160436224,"data_stored":159729129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.0289999999999999,"15min":1.0609999999999999},"min":{"1min":0.64000000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":3.7349999999999999,"5min":3.7349999999999999,"15min":3.7349999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94399999999999995,"15min":0.85099999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":2.0449999999999999,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.375}]},{"osd":1,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":8.5239999999999991,"5min":12.039999999999999,"15min":4.6319999999999997},"min":{"1min":0.56799999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":139.88200000000001,"5min":787.22900000000004,"15min":787.22900000000004},"last":1.587},{"interface":"front","average":{"1min":1.448,"5min":2.5649999999999999,"15min":1.593},"min":{"1min":0.59899999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":4.3700000000000001,"5min":116.548,"15min":116.548},"last":1.0580000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":4.3600000000000003,"15min":7.5999999999999996},"min":{"1min":0.77300000000000002,"5min":0.499,"15min":0.499},"max":{"1min":3.8109999999999999,"5min":168.56899999999999,"15min":168.56899999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":1.165,"5min":13.967000000000001,"15min":26.742999999999999},"min":{"1min":0.66200000000000003,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.1280000000000001,"5min":676.27099999999996,"15min":676.27099999999996},"last":0.52200000000000002}]},{"osd":3,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.589,"15min":1.2749999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":4.8120000000000003,"5min":29.129000000000001,"15min":29.129000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.5660000000000001,"5min":10.016999999999999,"15min":4.1470000000000002},"min":{"1min":0.76400000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":6.4189999999999996,"5min":712.98699999999997,"15min":712.98699999999997},"last":1.073}]},{"osd":4,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":10.253,"5min":32.627000000000002,"15min":44.804000000000002},"min":{"1min":0.60499999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":171.43700000000001,"5min":971.67999999999995,"15min":971.67999999999995},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.2410000000000001,"15min":1.284},"min":{"1min":0.61899999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.411,"5min":3.5960000000000001,"15min":3.5960000000000001},"last":0.745}]},{"osd":6,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":12.24,"5min":41.228000000000002,"15min":47.347999999999999},"min":{"1min":0.85099999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":208.25299999999999,"5min":966.60900000000004,"15min":966.60900000000004},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.5049999999999999,"5min":16.625,"15min":22.885999999999999},"min":{"1min":0.88100000000000001,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":4.4039999999999999,"5min":517.26700000000005,"15min":517.26700000000005},"last":1.1160000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.329,"15min":1.3149999999999999},"min":{"1min":0.79200000000000004,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.0830000000000002,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.427,"5min":6.6349999999999998,"15min":3.2559999999999998},"min":{"1min":0.89600000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.7269999999999999,"5min":515.09100000000001,"15min":515.09100000000001},"last":1.085}]}]}]}} 2020-08-07T15:10:43.128 INFO:tasks.ceph.ceph_manager.ceph:increase pool size by 10 2020-08-07T15:10:43.129 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set device_health_metrics pg_num 31 2020-08-07T15:10:43.188 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:43.189 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:10:43.189 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:10:43.189 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:10:43.195 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.198 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:15576 3.e e27/20 3:727724a5:::benchmark_data_smithi074_35447_object1946:head v 27'1032, mlcod=27'1023)", 2020-08-07T15:10:43.198 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:10:43.199 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 236.51395437299999, 2020-08-07T15:10:43.199 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.46091973000000003, 2020-08-07T15:10:43.199 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:43.199 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:43.199 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:43.200 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.200 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:43.200 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:10:43.200 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:43.200 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.201 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.201 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:43.218 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669483+0000", 2020-08-07T15:10:43.218 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9592000000000001e-05 2020-08-07T15:10:43.218 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.219 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.219 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:43.219 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669542+0000", 2020-08-07T15:10:43.219 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.113e-06 2020-08-07T15:10:43.219 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.220 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.220 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:43.220 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:46.669547+0000", 2020-08-07T15:10:43.220 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45151617799999999 2020-08-07T15:10:43.220 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.221 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.221 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:43.221 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121064+0000", 2020-08-07T15:10:43.221 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.377e-06 2020-08-07T15:10:43.221 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.238 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.238 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:43.238 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121067+0000", 2020-08-07T15:10:43.239 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9195e-05 2020-08-07T15:10:43.239 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.239 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.239 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:43.239 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121106+0000", 2020-08-07T15:10:43.240 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7509000000000001e-05 2020-08-07T15:10:43.240 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.262 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.263 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:43.263 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.121124+0000", 2020-08-07T15:10:43.264 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0091783100000000003 2020-08-07T15:10:43.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.273 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:43.274 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.130302+0000", 2020-08-07T15:10:43.274 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000100456 2020-08-07T15:10:43.274 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.275 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.275 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:43.275 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:47.130402+0000", 2020-08-07T15:10:43.276 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.46091973000000003 2020-08-07T15:10:43.276 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.276 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:43.277 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.277 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.281 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.281 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:24264 3.8 e28/20 3:1d3fdbda:::benchmark_data_smithi074_35447_object3032:head v 28'1520, mlcod=28'1515)", 2020-08-07T15:10:43.282 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:10:43.282 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 230.80032433599999, 2020-08-07T15:10:43.282 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41856254199999998, 2020-08-07T15:10:43.282 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:43.282 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:43.283 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:43.283 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.283 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:43.283 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:10:43.283 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:43.284 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.284 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.284 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:43.284 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383113+0000", 2020-08-07T15:10:43.284 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1348999999999998e-05 2020-08-07T15:10:43.284 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.285 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.285 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:43.285 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383174+0000", 2020-08-07T15:10:43.294 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6579999999999996e-06 2020-08-07T15:10:43.294 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.294 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.294 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:43.303 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.383181+0000", 2020-08-07T15:10:43.303 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41194352699999998 2020-08-07T15:10:43.303 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.304 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.304 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:43.304 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.795124+0000", 2020-08-07T15:10:43.305 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5729999999999999e-06 2020-08-07T15:10:43.305 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.316 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.316 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:43.317 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.795128+0000", 2020-08-07T15:10:43.317 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002170384 2020-08-07T15:10:43.318 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.318 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.318 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:43.319 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.797298+0000", 2020-08-07T15:10:43.319 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5412e-05 2020-08-07T15:10:43.319 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.320 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.320 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:43.320 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.797324+0000", 2020-08-07T15:10:43.321 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042943360000000002 2020-08-07T15:10:43.321 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.321 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.322 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:43.323 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.801618+0000", 2020-08-07T15:10:43.323 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7302999999999997e-05 2020-08-07T15:10:43.324 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.324 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.325 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:43.325 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:06:52.801675+0000", 2020-08-07T15:10:43.325 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41856254199999998 2020-08-07T15:10:43.326 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.326 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:43.326 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.327 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.327 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.327 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:93920 3.6 e56/26 ondisk, result = 0)", 2020-08-07T15:10:43.328 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:10:43.328 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 183.85296107299999, 2020-08-07T15:10:43.328 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.397576347, 2020-08-07T15:10:43.329 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:43.329 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:43.330 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:43.330 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.330 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:43.330 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:10:43.330 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:43.331 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.331 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.331 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:43.331 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330476+0000", 2020-08-07T15:10:43.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2571999999999999e-05 2020-08-07T15:10:43.332 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.332 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.332 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:43.333 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330509+0000", 2020-08-07T15:10:43.333 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2059999999999998e-06 2020-08-07T15:10:43.333 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.333 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.333 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:43.334 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.330514+0000", 2020-08-07T15:10:43.334 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39740134599999999 2020-08-07T15:10:43.334 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.334 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.334 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:43.335 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727915+0000", 2020-08-07T15:10:43.335 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.677e-06 2020-08-07T15:10:43.335 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.335 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:43.336 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727921+0000", 2020-08-07T15:10:43.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5431e-05 2020-08-07T15:10:43.336 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.336 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.336 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:43.337 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727956+0000", 2020-08-07T15:10:43.337 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5790000000000004e-06 2020-08-07T15:10:43.337 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.337 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.337 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:43.338 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.727965+0000", 2020-08-07T15:10:43.338 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7535999999999997e-05 2020-08-07T15:10:43.338 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.339 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.339 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:43.339 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.728052+0000", 2020-08-07T15:10:43.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.397576347 2020-08-07T15:10:43.339 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.340 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:43.340 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.340 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.340 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106104 3.0 3:0d7a2a89:::benchmark_data_smithi074_35447_object13262:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:10:43.341 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.252296+0000", 2020-08-07T15:10:43.341 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 175.93114118400001, 2020-08-07T15:10:43.341 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.44182183200000003, 2020-08-07T15:10:43.341 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:43.341 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:43.342 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:43.342 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:10:43.342 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:43.342 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106104 2020-08-07T15:10:43.342 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.343 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:43.343 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.344 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:43.344 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252296+0000", 2020-08-07T15:10:43.344 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:43.344 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.344 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.345 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:43.345 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252296+0000", 2020-08-07T15:10:43.345 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4728000000000001e-05 2020-08-07T15:10:43.345 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.345 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.346 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:43.346 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252321+0000", 2020-08-07T15:10:43.346 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9529999999999999e-06 2020-08-07T15:10:43.346 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.346 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.347 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:43.347 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252324+0000", 2020-08-07T15:10:43.347 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5759999999999996e-06 2020-08-07T15:10:43.347 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.347 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:43.348 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252329+0000", 2020-08-07T15:10:43.348 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.582e-06 2020-08-07T15:10:43.348 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.348 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.348 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:43.349 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.252332+0000", 2020-08-07T15:10:43.349 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012958519999999999 2020-08-07T15:10:43.349 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.349 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.349 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:43.360 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.253628+0000", 2020-08-07T15:10:43.360 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7871e-05 2020-08-07T15:10:43.361 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.361 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.361 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:43.361 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.253705+0000", 2020-08-07T15:10:43.361 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017801199999999999 2020-08-07T15:10:43.362 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.362 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.362 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:10:43.362 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.253883+0000", 2020-08-07T15:10:43.362 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001869051 2020-08-07T15:10:43.362 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.363 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.363 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:43.363 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255752+0000", 2020-08-07T15:10:43.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3e-07 2020-08-07T15:10:43.363 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:43.364 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255753+0000", 2020-08-07T15:10:43.364 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5321000000000001e-05 2020-08-07T15:10:43.364 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.365 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.365 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:43.365 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.255808+0000", 2020-08-07T15:10:43.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.091917211999999998 2020-08-07T15:10:43.365 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.376 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:43.376 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.347726+0000", 2020-08-07T15:10:43.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.34634211500000001 2020-08-07T15:10:43.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:43.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694068+0000", 2020-08-07T15:10:43.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3030000000000001e-06 2020-08-07T15:10:43.377 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:43.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694069+0000", 2020-08-07T15:10:43.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8625999999999998e-05 2020-08-07T15:10:43.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:43.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694118+0000", 2020-08-07T15:10:43.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.44182183200000003 2020-08-07T15:10:43.379 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.380 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:43.380 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.380 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106104 3.0 e58/20 ondisk, result = 0)", 2020-08-07T15:10:43.380 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.257196+0000", 2020-08-07T15:10:43.381 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 175.92624067700001, 2020-08-07T15:10:43.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43692689000000001, 2020-08-07T15:10:43.381 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:43.381 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:43.381 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:43.382 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.382 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:43.382 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257196+0000", 2020-08-07T15:10:43.382 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:43.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.383 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.383 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:43.384 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257196+0000", 2020-08-07T15:10:43.384 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2021999999999999e-05 2020-08-07T15:10:43.384 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.384 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.384 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:43.385 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257228+0000", 2020-08-07T15:10:43.385 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9549999999999997e-06 2020-08-07T15:10:43.385 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.385 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.385 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:43.385 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.257234+0000", 2020-08-07T15:10:43.386 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43671786400000001 2020-08-07T15:10:43.386 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.386 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.386 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:43.386 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693952+0000", 2020-08-07T15:10:43.387 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7690000000000004e-06 2020-08-07T15:10:43.387 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.387 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.387 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:43.387 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.693957+0000", 2020-08-07T15:10:43.388 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3294000000000006e-05 2020-08-07T15:10:43.388 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.388 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.388 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:43.388 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694040+0000", 2020-08-07T15:10:43.389 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3436e-05 2020-08-07T15:10:43.389 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.389 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.389 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:43.389 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694054+0000", 2020-08-07T15:10:43.389 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9549999999999996e-05 2020-08-07T15:10:43.390 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.390 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.390 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:43.390 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.694123+0000", 2020-08-07T15:10:43.390 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43692689000000001 2020-08-07T15:10:43.391 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.391 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:43.392 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.392 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.392 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.393 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106144 3.6 3:629055a5:::benchmark_data_smithi074_35447_object13267:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e58)", 2020-08-07T15:10:43.394 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.280142+0000", 2020-08-07T15:10:43.394 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 175.90329521500001, 2020-08-07T15:10:43.394 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.57968657199999996, 2020-08-07T15:10:43.394 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:43.395 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:43.395 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:43.395 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:10:43.395 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:43.395 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106144 2020-08-07T15:10:43.396 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.396 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:43.396 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.396 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:43.396 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280142+0000", 2020-08-07T15:10:43.397 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:43.397 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.397 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.397 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:43.397 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280142+0000", 2020-08-07T15:10:43.397 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.905e-05 2020-08-07T15:10:43.398 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.398 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.398 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:43.398 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280191+0000", 2020-08-07T15:10:43.398 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6729999999999997e-06 2020-08-07T15:10:43.399 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.399 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:43.399 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280197+0000", 2020-08-07T15:10:43.399 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0747999999999999e-05 2020-08-07T15:10:43.400 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.400 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:43.400 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280207+0000", 2020-08-07T15:10:43.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7700000000000001e-06 2020-08-07T15:10:43.401 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.401 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.401 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:43.401 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280212+0000", 2020-08-07T15:10:43.401 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00060223399999999999 2020-08-07T15:10:43.401 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.402 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.402 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:43.402 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280814+0000", 2020-08-07T15:10:43.402 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7831000000000006e-05 2020-08-07T15:10:43.402 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.403 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.403 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:43.403 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.280912+0000", 2020-08-07T15:10:43.403 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000137119 2020-08-07T15:10:43.403 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.404 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.404 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:10:43.405 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.281049+0000", 2020-08-07T15:10:43.405 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023794369999999999 2020-08-07T15:10:43.405 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.405 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:43.406 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283429+0000", 2020-08-07T15:10:43.406 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.173e-06 2020-08-07T15:10:43.406 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.406 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.406 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:43.407 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283430+0000", 2020-08-07T15:10:43.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5530000000000004e-05 2020-08-07T15:10:43.407 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.407 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.407 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:43.408 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283495+0000", 2020-08-07T15:10:43.408 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5649000000000003e-05 2020-08-07T15:10:43.408 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.408 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.408 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:43.409 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.283541+0000", 2020-08-07T15:10:43.409 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.57624206300000003 2020-08-07T15:10:43.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.409 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.409 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:43.410 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859783+0000", 2020-08-07T15:10:43.410 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.421e-06 2020-08-07T15:10:43.410 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.410 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.410 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:43.410 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859785+0000", 2020-08-07T15:10:43.411 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3874e-05 2020-08-07T15:10:43.411 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.411 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.411 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:43.411 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859828+0000", 2020-08-07T15:10:43.412 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.57968657199999996 2020-08-07T15:10:43.412 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.412 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:43.412 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.412 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.413 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106144 3.6 e59/26 ondisk, result = 0)", 2020-08-07T15:10:43.413 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.344536+0000", 2020-08-07T15:10:43.413 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 175.838900873, 2020-08-07T15:10:43.413 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51529786099999997, 2020-08-07T15:10:43.413 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:43.414 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:43.414 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:43.414 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.415 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:43.415 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344536+0000", 2020-08-07T15:10:43.415 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:43.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.416 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.417 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:43.417 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344536+0000", 2020-08-07T15:10:43.417 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7266999999999999e-05 2020-08-07T15:10:43.418 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.418 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.418 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:43.418 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344553+0000", 2020-08-07T15:10:43.419 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1070000000000001e-06 2020-08-07T15:10:43.419 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.419 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.419 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:43.419 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.344557+0000", 2020-08-07T15:10:43.420 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.515143201 2020-08-07T15:10:43.420 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.420 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.420 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:43.421 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859700+0000", 2020-08-07T15:10:43.422 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.271e-05 2020-08-07T15:10:43.422 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.422 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:43.422 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859712+0000", 2020-08-07T15:10:43.422 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5436999999999998e-05 2020-08-07T15:10:43.423 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.423 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.424 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:43.424 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859768+0000", 2020-08-07T15:10:43.424 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1540000000000002e-06 2020-08-07T15:10:43.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.424 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:43.425 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859776+0000", 2020-08-07T15:10:43.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7985000000000001e-05 2020-08-07T15:10:43.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.429 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:43.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.859834+0000", 2020-08-07T15:10:43.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.51529786099999997 2020-08-07T15:10:43.429 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.430 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:43.431 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.431 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.432 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:108352 3.8 3:1aac8373:::benchmark_data_smithi074_35447_object13543:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e62)", 2020-08-07T15:10:43.432 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:50.642093+0000", 2020-08-07T15:10:43.432 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 172.541344184, 2020-08-07T15:10:43.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39397409900000002, 2020-08-07T15:10:43.433 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:43.433 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:43.434 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:43.434 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:10:43.435 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:43.435 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 108352 2020-08-07T15:10:43.442 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.443 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:43.443 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.443 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:43.443 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642093+0000", 2020-08-07T15:10:43.443 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:43.444 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.444 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:43.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642093+0000", 2020-08-07T15:10:43.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9475000000000002e-05 2020-08-07T15:10:43.444 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.445 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.445 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:43.447 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642122+0000", 2020-08-07T15:10:43.447 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0439999999999997e-06 2020-08-07T15:10:43.458 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.458 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.458 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:43.459 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642126+0000", 2020-08-07T15:10:43.465 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9689999999999994e-06 2020-08-07T15:10:43.465 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.465 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.465 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:43.465 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642134+0000", 2020-08-07T15:10:43.470 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1219999999999997e-06 2020-08-07T15:10:43.470 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.471 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.471 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:43.471 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.642138+0000", 2020-08-07T15:10:43.471 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011580399999999999 2020-08-07T15:10:43.490 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.491 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.491 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:43.491 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.643296+0000", 2020-08-07T15:10:43.491 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3791999999999999e-05 2020-08-07T15:10:43.492 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.492 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.492 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:43.492 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.643390+0000", 2020-08-07T15:10:43.492 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011882700000000001 2020-08-07T15:10:43.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.493 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4,7", 2020-08-07T15:10:43.493 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.643509+0000", 2020-08-07T15:10:43.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034736799999999998 2020-08-07T15:10:43.494 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.494 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:43.494 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.646983+0000", 2020-08-07T15:10:43.494 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0077748490000000003 2020-08-07T15:10:43.495 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.495 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.495 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:43.495 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.654758+0000", 2020-08-07T15:10:43.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000005e-07 2020-08-07T15:10:43.496 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.496 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.496 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:43.496 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.654758+0000", 2020-08-07T15:10:43.496 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4258000000000002e-05 2020-08-07T15:10:43.496 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.497 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.497 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:43.497 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.654802+0000", 2020-08-07T15:10:43.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11166224399999999 2020-08-07T15:10:43.497 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.507 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.514 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:43.514 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:50.766465+0000", 2020-08-07T15:10:43.515 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26946740899999999 2020-08-07T15:10:43.515 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.515 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.515 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:43.515 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.035932+0000", 2020-08-07T15:10:43.516 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5829999999999999e-06 2020-08-07T15:10:43.516 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.516 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.516 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:43.516 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.035934+0000", 2020-08-07T15:10:43.517 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000133306 2020-08-07T15:10:43.517 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.517 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.517 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:43.517 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:51.036067+0000", 2020-08-07T15:10:43.518 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39397409900000002 2020-08-07T15:10:43.518 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.518 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:43.518 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.519 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.519 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145767 3.6 3:62fe5854:::benchmark_data_smithi074_35447_object16158:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:10:43.519 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.654765+0000", 2020-08-07T15:10:43.519 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 148.52867199799999, 2020-08-07T15:10:43.519 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.62108287699999998, 2020-08-07T15:10:43.520 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:43.520 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:43.520 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:43.520 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:10:43.520 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:43.521 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145767 2020-08-07T15:10:43.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.521 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:43.521 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.521 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:43.522 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654765+0000", 2020-08-07T15:10:43.522 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:43.522 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.523 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.523 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:43.523 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654765+0000", 2020-08-07T15:10:43.524 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1977999999999998e-05 2020-08-07T15:10:43.534 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.534 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:43.535 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654797+0000", 2020-08-07T15:10:43.535 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6799999999999998e-06 2020-08-07T15:10:43.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:43.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654803+0000", 2020-08-07T15:10:43.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1263e-05 2020-08-07T15:10:43.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.536 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:43.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654814+0000", 2020-08-07T15:10:43.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2129999999999999e-06 2020-08-07T15:10:43.537 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.537 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.537 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:43.539 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654819+0000", 2020-08-07T15:10:43.539 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0324999999999997e-05 2020-08-07T15:10:43.540 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.540 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.540 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:43.540 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.654869+0000", 2020-08-07T15:10:43.540 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015424199999999999 2020-08-07T15:10:43.541 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.541 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:43.541 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.655024+0000", 2020-08-07T15:10:43.541 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000163573 2020-08-07T15:10:43.542 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.542 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.542 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:10:43.542 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.655187+0000", 2020-08-07T15:10:43.542 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001341645 2020-08-07T15:10:43.543 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.543 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.543 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:43.543 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.656529+0000", 2020-08-07T15:10:43.543 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9599999999999998e-07 2020-08-07T15:10:43.544 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.553 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.553 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:43.553 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.656530+0000", 2020-08-07T15:10:43.553 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4381e-05 2020-08-07T15:10:43.554 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.554 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.554 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:43.556 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.656584+0000", 2020-08-07T15:10:43.557 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0057298979999999998 2020-08-07T15:10:43.557 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.557 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.557 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:43.558 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.662314+0000", 2020-08-07T15:10:43.558 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.61347776300000001 2020-08-07T15:10:43.558 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.558 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.558 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:43.559 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275792+0000", 2020-08-07T15:10:43.559 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9740000000000001e-06 2020-08-07T15:10:43.559 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.559 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.559 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:43.559 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275794+0000", 2020-08-07T15:10:43.560 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4146e-05 2020-08-07T15:10:43.560 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.560 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.560 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:43.560 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275848+0000", 2020-08-07T15:10:43.561 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.62108287699999998 2020-08-07T15:10:43.561 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.561 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:43.561 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.561 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.562 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.562 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145801 3.6 3:69e4cb22:::benchmark_data_smithi074_35447_object16192:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:10:43.562 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671123+0000", 2020-08-07T15:10:43.562 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 148.51231412300001, 2020-08-07T15:10:43.563 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74411769999999999, 2020-08-07T15:10:43.563 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:43.563 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:43.563 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:43.563 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:10:43.563 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:43.564 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145801 2020-08-07T15:10:43.564 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.564 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:43.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.564 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:43.565 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671123+0000", 2020-08-07T15:10:43.565 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:43.566 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.566 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.566 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:43.566 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671123+0000", 2020-08-07T15:10:43.566 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3691e-05 2020-08-07T15:10:43.567 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.567 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.567 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:43.567 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671157+0000", 2020-08-07T15:10:43.567 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0249999999999999e-06 2020-08-07T15:10:43.568 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.568 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.568 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:43.568 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671163+0000", 2020-08-07T15:10:43.569 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2339e-05 2020-08-07T15:10:43.569 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.570 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.570 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:43.570 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671175+0000", 2020-08-07T15:10:43.570 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5249999999999996e-06 2020-08-07T15:10:43.570 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.570 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.571 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:43.571 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671180+0000", 2020-08-07T15:10:43.571 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6826000000000002e-05 2020-08-07T15:10:43.571 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.571 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.572 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:43.572 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671227+0000", 2020-08-07T15:10:43.572 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014152699999999999 2020-08-07T15:10:43.572 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.572 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.573 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:43.573 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671369+0000", 2020-08-07T15:10:43.573 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8021000000000002e-05 2020-08-07T15:10:43.574 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.574 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.574 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:10:43.575 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671467+0000", 2020-08-07T15:10:43.575 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0016293760000000001 2020-08-07T15:10:43.575 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.576 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.576 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:43.578 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673096+0000", 2020-08-07T15:10:43.579 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.048e-06 2020-08-07T15:10:43.579 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.579 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.580 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:43.580 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673097+0000", 2020-08-07T15:10:43.580 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8841999999999998e-05 2020-08-07T15:10:43.581 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.581 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.581 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:43.582 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673146+0000", 2020-08-07T15:10:43.582 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002829935 2020-08-07T15:10:43.582 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.583 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.583 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:43.583 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675976+0000", 2020-08-07T15:10:43.584 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73917870399999996 2020-08-07T15:10:43.584 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.584 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.585 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:43.585 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.415155+0000", 2020-08-07T15:10:43.585 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.193e-06 2020-08-07T15:10:43.586 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.586 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.587 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:43.587 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.415157+0000", 2020-08-07T15:10:43.587 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3647999999999996e-05 2020-08-07T15:10:43.597 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.597 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.597 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:43.598 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.415241+0000", 2020-08-07T15:10:43.598 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74411769999999999 2020-08-07T15:10:43.598 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.598 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:43.598 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.599 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.599 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.599 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145803 3.0 3:0c939450:::benchmark_data_smithi074_35447_object16194:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:10:43.599 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671678+0000", 2020-08-07T15:10:43.599 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 148.51175932800001, 2020-08-07T15:10:43.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.882157211, 2020-08-07T15:10:43.600 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:43.600 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:43.600 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:43.600 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:10:43.601 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:43.601 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145803 2020-08-07T15:10:43.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.601 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:43.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.602 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:43.602 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671678+0000", 2020-08-07T15:10:43.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:43.602 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.602 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.603 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:43.603 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671678+0000", 2020-08-07T15:10:43.603 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4097999999999998e-05 2020-08-07T15:10:43.603 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:43.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671712+0000", 2020-08-07T15:10:43.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2439999999999998e-06 2020-08-07T15:10:43.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:43.605 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671718+0000", 2020-08-07T15:10:43.605 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2629999999999999e-05 2020-08-07T15:10:43.605 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.605 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.605 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:43.606 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671731+0000", 2020-08-07T15:10:43.606 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9589999999999998e-06 2020-08-07T15:10:43.606 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.606 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.606 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:43.607 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671738+0000", 2020-08-07T15:10:43.607 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2495000000000004e-05 2020-08-07T15:10:43.607 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.607 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.607 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:43.608 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671800+0000", 2020-08-07T15:10:43.608 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013640700000000001 2020-08-07T15:10:43.608 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.608 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.608 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:43.609 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671937+0000", 2020-08-07T15:10:43.609 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013672200000000001 2020-08-07T15:10:43.609 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.609 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.609 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:10:43.610 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.672073+0000", 2020-08-07T15:10:43.610 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001179982 2020-08-07T15:10:43.610 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.611 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.611 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:43.611 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673253+0000", 2020-08-07T15:10:43.611 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1900000000000001e-07 2020-08-07T15:10:43.611 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.612 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.612 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:43.612 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673254+0000", 2020-08-07T15:10:43.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2104000000000001e-05 2020-08-07T15:10:43.612 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.613 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:43.613 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673306+0000", 2020-08-07T15:10:43.613 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002264369 2020-08-07T15:10:43.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.614 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.614 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:43.614 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675571+0000", 2020-08-07T15:10:43.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.87820796899999998 2020-08-07T15:10:43.614 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.614 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.615 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:43.615 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553778+0000", 2020-08-07T15:10:43.615 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3629999999999999e-06 2020-08-07T15:10:43.615 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.615 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.616 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:43.616 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553780+0000", 2020-08-07T15:10:43.616 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5050000000000003e-05 2020-08-07T15:10:43.616 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.616 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.617 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:43.617 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553835+0000", 2020-08-07T15:10:43.617 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.882157211 2020-08-07T15:10:43.617 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.617 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:43.618 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.618 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.618 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.618 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145805 3.8 3:13b8e273:::benchmark_data_smithi074_35447_object16196:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:10:43.618 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671847+0000", 2020-08-07T15:10:43.618 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 148.51159000800001, 2020-08-07T15:10:43.619 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.340134453, 2020-08-07T15:10:43.619 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:43.619 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:43.619 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:43.619 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:10:43.620 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:43.620 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145805 2020-08-07T15:10:43.621 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.621 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:43.621 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.621 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:43.621 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671847+0000", 2020-08-07T15:10:43.622 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:43.622 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.622 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.622 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:43.622 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671847+0000", 2020-08-07T15:10:43.623 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1139999999999997e-05 2020-08-07T15:10:43.623 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.623 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.623 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:43.623 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671878+0000", 2020-08-07T15:10:43.623 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.784e-06 2020-08-07T15:10:43.624 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.624 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.624 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:43.624 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671884+0000", 2020-08-07T15:10:43.624 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2802999999999999e-05 2020-08-07T15:10:43.625 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.625 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.625 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:43.625 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671897+0000", 2020-08-07T15:10:43.625 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9909999999999999e-06 2020-08-07T15:10:43.626 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.626 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.626 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:43.626 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671903+0000", 2020-08-07T15:10:43.626 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011861 2020-08-07T15:10:43.626 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.627 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:43.627 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.672021+0000", 2020-08-07T15:10:43.627 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017702399999999999 2020-08-07T15:10:43.627 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.628 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.628 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:43.628 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.672198+0000", 2020-08-07T15:10:43.628 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020546400000000001 2020-08-07T15:10:43.628 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.629 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.629 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4,7", 2020-08-07T15:10:43.629 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.672404+0000", 2020-08-07T15:10:43.629 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024385019999999999 2020-08-07T15:10:43.629 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.630 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.630 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:43.630 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674842+0000", 2020-08-07T15:10:43.630 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00028594999999999998 2020-08-07T15:10:43.630 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.631 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.631 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:43.631 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675128+0000", 2020-08-07T15:10:43.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8100000000000003e-07 2020-08-07T15:10:43.632 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.632 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.632 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:43.632 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675129+0000", 2020-08-07T15:10:43.633 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7973999999999997e-05 2020-08-07T15:10:43.633 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.633 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.633 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:43.633 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.675197+0000", 2020-08-07T15:10:43.634 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032171220000000002 2020-08-07T15:10:43.634 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.634 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.634 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:43.634 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678414+0000", 2020-08-07T15:10:43.635 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33343769299999998 2020-08-07T15:10:43.635 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.635 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.635 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:43.635 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011852+0000", 2020-08-07T15:10:43.636 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3480000000000001e-06 2020-08-07T15:10:43.636 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.636 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.636 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:43.636 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011853+0000", 2020-08-07T15:10:43.636 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000128467 2020-08-07T15:10:43.637 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.637 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.637 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:43.637 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011981+0000", 2020-08-07T15:10:43.637 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.340134453 2020-08-07T15:10:43.638 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.638 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:43.638 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.639 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.639 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.639 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:145806 3.8 3:16669ddb:::benchmark_data_smithi074_35447_object16197:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:10:43.639 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.673992+0000", 2020-08-07T15:10:43.639 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 148.50944548199999, 2020-08-07T15:10:43.640 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40687891900000001, 2020-08-07T15:10:43.640 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:43.640 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:43.640 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:43.640 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:10:43.641 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:43.641 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 145806 2020-08-07T15:10:43.641 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.642 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:43.642 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.642 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:43.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673992+0000", 2020-08-07T15:10:43.643 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:43.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.643 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.643 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:43.643 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.673992+0000", 2020-08-07T15:10:43.644 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5429000000000002e-05 2020-08-07T15:10:43.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.644 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.644 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:43.644 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674027+0000", 2020-08-07T15:10:43.645 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3199999999999996e-06 2020-08-07T15:10:43.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:43.645 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674033+0000", 2020-08-07T15:10:43.645 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2835e-05 2020-08-07T15:10:43.646 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.646 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.646 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:43.646 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674046+0000", 2020-08-07T15:10:43.646 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9030000000000001e-06 2020-08-07T15:10:43.647 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.647 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.647 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:43.647 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674052+0000", 2020-08-07T15:10:43.647 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4181000000000003e-05 2020-08-07T15:10:43.648 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.648 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.648 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:43.648 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674096+0000", 2020-08-07T15:10:43.648 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00022310599999999999 2020-08-07T15:10:43.649 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.649 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.649 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:43.649 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674319+0000", 2020-08-07T15:10:43.649 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012681300000000001 2020-08-07T15:10:43.650 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.650 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.650 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4,7", 2020-08-07T15:10:43.650 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674446+0000", 2020-08-07T15:10:43.650 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002882754 2020-08-07T15:10:43.650 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.651 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.651 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:43.651 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.677329+0000", 2020-08-07T15:10:43.651 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9466e-05 2020-08-07T15:10:43.651 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.652 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.652 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:43.653 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.677418+0000", 2020-08-07T15:10:43.653 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0299999999999998e-07 2020-08-07T15:10:43.653 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.653 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.653 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:43.654 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.677419+0000", 2020-08-07T15:10:43.654 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5274000000000001e-05 2020-08-07T15:10:43.654 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.654 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.655 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:43.655 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.677464+0000", 2020-08-07T15:10:43.655 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40011127099999999 2020-08-07T15:10:43.656 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.665 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:43.665 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.077576+0000", 2020-08-07T15:10:43.666 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0031778370000000002 2020-08-07T15:10:43.666 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.666 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.667 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:43.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080754+0000", 2020-08-07T15:10:43.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.92e-07 2020-08-07T15:10:43.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:43.678 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080754+0000", 2020-08-07T15:10:43.679 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000116135 2020-08-07T15:10:43.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.680 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:43.680 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080870+0000", 2020-08-07T15:10:43.681 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40687891900000001 2020-08-07T15:10:43.681 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.682 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:43.682 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.683 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.683 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.684 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:145767 3.6 e75/26 ondisk, result = 0)", 2020-08-07T15:10:43.684 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.674108+0000", 2020-08-07T15:10:43.684 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 148.50932905799999, 2020-08-07T15:10:43.685 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.60174556899999998, 2020-08-07T15:10:43.686 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:43.686 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:43.687 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:43.687 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.687 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:43.688 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674108+0000", 2020-08-07T15:10:43.688 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:43.688 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.689 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.689 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:43.690 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674108+0000", 2020-08-07T15:10:43.690 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0558000000000001e-05 2020-08-07T15:10:43.690 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.691 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.691 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:43.691 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674139+0000", 2020-08-07T15:10:43.692 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6620000000000002e-06 2020-08-07T15:10:43.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.693 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.693 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:43.694 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674144+0000", 2020-08-07T15:10:43.694 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.60156723899999998 2020-08-07T15:10:43.694 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.694 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.694 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:43.695 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275711+0000", 2020-08-07T15:10:43.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4070000000000004e-06 2020-08-07T15:10:43.695 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.695 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:43.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275719+0000", 2020-08-07T15:10:43.698 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3205999999999998e-05 2020-08-07T15:10:43.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.699 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.699 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:43.699 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275772+0000", 2020-08-07T15:10:43.699 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0624e-05 2020-08-07T15:10:43.699 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.699 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.700 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:43.700 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275783+0000", 2020-08-07T15:10:43.700 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0872999999999997e-05 2020-08-07T15:10:43.700 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.701 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:43.701 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.275854+0000", 2020-08-07T15:10:43.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.60174556899999998 2020-08-07T15:10:43.701 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.701 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:43.702 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.702 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.702 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.702 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:145803 3.0 e75/20 ondisk, result = 0)", 2020-08-07T15:10:43.702 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.674488+0000", 2020-08-07T15:10:43.703 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 148.50894907599999, 2020-08-07T15:10:43.703 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.87935229699999995, 2020-08-07T15:10:43.703 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:43.703 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:43.703 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:43.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.704 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:43.704 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674488+0000", 2020-08-07T15:10:43.704 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:43.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.705 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:43.705 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674488+0000", 2020-08-07T15:10:43.705 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3062000000000003e-05 2020-08-07T15:10:43.705 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.705 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.706 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:43.706 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674521+0000", 2020-08-07T15:10:43.706 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4911e-05 2020-08-07T15:10:43.706 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.707 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:43.709 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.674536+0000", 2020-08-07T15:10:43.709 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.87914350600000002 2020-08-07T15:10:43.709 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.709 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.710 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:43.710 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553679+0000", 2020-08-07T15:10:43.710 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5950000000000001e-06 2020-08-07T15:10:43.710 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.710 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.711 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:43.711 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553687+0000", 2020-08-07T15:10:43.711 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2565999999999996e-05 2020-08-07T15:10:43.711 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.712 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:43.712 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553760+0000", 2020-08-07T15:10:43.712 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0417999999999999e-05 2020-08-07T15:10:43.712 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.712 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.713 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:43.713 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553770+0000", 2020-08-07T15:10:43.713 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0239000000000003e-05 2020-08-07T15:10:43.713 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.713 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.714 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:43.714 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.553840+0000", 2020-08-07T15:10:43.714 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.87935229699999995 2020-08-07T15:10:43.714 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.714 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:43.715 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.715 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.715 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.715 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:145805 3.8 e75/60 ondisk, result = 0)", 2020-08-07T15:10:43.715 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.676843+0000", 2020-08-07T15:10:43.716 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 148.506593606, 2020-08-07T15:10:43.716 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.335144041, 2020-08-07T15:10:43.716 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:43.716 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:43.716 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:43.716 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.717 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:43.717 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.676843+0000", 2020-08-07T15:10:43.717 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:43.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.718 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.718 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:43.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.676843+0000", 2020-08-07T15:10:43.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7853e-05 2020-08-07T15:10:43.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:43.725 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.676861+0000", 2020-08-07T15:10:43.725 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7359999999999999e-06 2020-08-07T15:10:43.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.726 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.726 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:43.726 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.676865+0000", 2020-08-07T15:10:43.726 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33484658099999998 2020-08-07T15:10:43.727 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.727 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.727 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:43.727 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011712+0000", 2020-08-07T15:10:43.727 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1613e-05 2020-08-07T15:10:43.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:43.728 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011723+0000", 2020-08-07T15:10:43.728 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011038 2020-08-07T15:10:43.729 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.729 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.729 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:43.729 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011834+0000", 2020-08-07T15:10:43.730 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0097e-05 2020-08-07T15:10:43.730 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.730 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.730 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:43.730 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011844+0000", 2020-08-07T15:10:43.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000143781 2020-08-07T15:10:43.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.731 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:43.731 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.011987+0000", 2020-08-07T15:10:43.732 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.335144041 2020-08-07T15:10:43.732 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.733 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:43.733 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.734 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.734 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.734 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:145806 3.8 e75/60 ondisk, result = 0)", 2020-08-07T15:10:43.734 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.678157+0000", 2020-08-07T15:10:43.734 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 148.505279554, 2020-08-07T15:10:43.735 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.402718152, 2020-08-07T15:10:43.735 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:43.735 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:43.735 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:43.735 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.736 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:43.736 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678157+0000", 2020-08-07T15:10:43.736 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:43.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.737 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.738 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:43.739 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678157+0000", 2020-08-07T15:10:43.739 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8496e-05 2020-08-07T15:10:43.739 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.741 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.741 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:43.741 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678176+0000", 2020-08-07T15:10:43.741 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.676e-06 2020-08-07T15:10:43.742 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.742 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.742 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:43.742 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.678180+0000", 2020-08-07T15:10:43.742 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40250966999999999 2020-08-07T15:10:43.751 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.752 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:43.752 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080689+0000", 2020-08-07T15:10:43.752 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7110000000000003e-06 2020-08-07T15:10:43.752 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.753 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.753 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:43.760 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080694+0000", 2020-08-07T15:10:43.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4122999999999998e-05 2020-08-07T15:10:43.761 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.761 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:43.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080738+0000", 2020-08-07T15:10:43.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5809999999999999e-06 2020-08-07T15:10:43.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.762 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.764 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:43.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080746+0000", 2020-08-07T15:10:43.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012989499999999999 2020-08-07T15:10:43.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.766 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:43.766 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.080876+0000", 2020-08-07T15:10:43.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.402718152 2020-08-07T15:10:43.766 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.766 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:43.766 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.767 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42288 4.2 4:4cfde355:::benchmark_data_smithi074_44586_object5285:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:10:43.767 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.115516+0000", 2020-08-07T15:10:43.767 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 103.06792145599999, 2020-08-07T15:10:43.768 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.53902283500000003, 2020-08-07T15:10:43.768 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:43.768 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:43.768 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:43.768 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:43.769 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:43.769 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42288 2020-08-07T15:10:43.769 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.769 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:43.769 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.770 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:43.770 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.115516+0000", 2020-08-07T15:10:43.770 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:43.771 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.771 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.771 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:43.771 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.115516+0000", 2020-08-07T15:10:43.772 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5785999999999997e-05 2020-08-07T15:10:43.774 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.775 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:43.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.115571+0000", 2020-08-07T15:10:43.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3130000000000004e-06 2020-08-07T15:10:43.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:43.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.115578+0000", 2020-08-07T15:10:43.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2514000000000001e-05 2020-08-07T15:10:43.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:43.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.115590+0000", 2020-08-07T15:10:43.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7130000000000002e-06 2020-08-07T15:10:43.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.779 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:43.779 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.115596+0000", 2020-08-07T15:10:43.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0036423369999999998 2020-08-07T15:10:43.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.780 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:43.780 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.119238+0000", 2020-08-07T15:10:43.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7732999999999995e-05 2020-08-07T15:10:43.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.781 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:43.782 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.119326+0000", 2020-08-07T15:10:43.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000102612 2020-08-07T15:10:43.782 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:10:43.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.119429+0000", 2020-08-07T15:10:43.783 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001842621 2020-08-07T15:10:43.783 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:43.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.121271+0000", 2020-08-07T15:10:43.785 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00096829399999999997 2020-08-07T15:10:43.785 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:43.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.122239+0000", 2020-08-07T15:10:43.786 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.53219874700000003 2020-08-07T15:10:43.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.786 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:43.786 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.654438+0000", 2020-08-07T15:10:43.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7290000000000001e-06 2020-08-07T15:10:43.787 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.787 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:43.788 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.654440+0000", 2020-08-07T15:10:43.788 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0956999999999999e-05 2020-08-07T15:10:43.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.789 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:43.789 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.654501+0000", 2020-08-07T15:10:43.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7478999999999998e-05 2020-08-07T15:10:43.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.790 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:43.790 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.654538+0000", 2020-08-07T15:10:43.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.53902283500000003 2020-08-07T15:10:43.791 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.791 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:43.791 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.792 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.792 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:66488 4.f 4:fdf15c0e:::benchmark_data_smithi074_44586_object8310:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e126)", 2020-08-07T15:10:43.792 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:20.213948+0000", 2020-08-07T15:10:43.792 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 82.969488747, 2020-08-07T15:10:43.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.34743775999999998, 2020-08-07T15:10:43.793 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:43.793 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:43.793 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:43.793 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:43.793 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:43.795 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 66488 2020-08-07T15:10:43.795 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.795 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:43.795 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.795 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:43.796 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.213948+0000", 2020-08-07T15:10:43.796 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:43.796 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.796 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:43.797 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.213948+0000", 2020-08-07T15:10:43.797 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0775999999999997e-05 2020-08-07T15:10:43.797 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.797 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.797 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:43.798 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.213989+0000", 2020-08-07T15:10:43.798 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5179999999999998e-06 2020-08-07T15:10:43.798 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.798 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.798 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:43.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.213994+0000", 2020-08-07T15:10:43.799 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7399999999999993e-06 2020-08-07T15:10:43.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:43.800 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.214002+0000", 2020-08-07T15:10:43.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5640000000000001e-06 2020-08-07T15:10:43.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.801 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.801 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:43.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.214006+0000", 2020-08-07T15:10:43.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0035647840000000001 2020-08-07T15:10:43.801 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.802 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.802 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:43.802 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.217571+0000", 2020-08-07T15:10:43.802 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000155446 2020-08-07T15:10:43.802 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:43.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.217726+0000", 2020-08-07T15:10:43.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00026276899999999999 2020-08-07T15:10:43.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4", 2020-08-07T15:10:43.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.217989+0000", 2020-08-07T15:10:43.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001388161 2020-08-07T15:10:43.806 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:43.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.219377+0000", 2020-08-07T15:10:43.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9100000000000002e-07 2020-08-07T15:10:43.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:43.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.219378+0000", 2020-08-07T15:10:43.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7027999999999998e-05 2020-08-07T15:10:43.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.808 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:43.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.219425+0000", 2020-08-07T15:10:43.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054840820000000004 2020-08-07T15:10:43.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.810 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.810 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:43.810 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.224909+0000", 2020-08-07T15:10:43.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33640654599999997 2020-08-07T15:10:43.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.811 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:43.811 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.561316+0000", 2020-08-07T15:10:43.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.127e-06 2020-08-07T15:10:43.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:43.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.561317+0000", 2020-08-07T15:10:43.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9327999999999998e-05 2020-08-07T15:10:43.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.813 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:43.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:20.561386+0000", 2020-08-07T15:10:43.813 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.34743775999999998 2020-08-07T15:10:43.814 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.814 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:43.814 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.814 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.814 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.815 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72952 4.b 4:dee63f23:::benchmark_data_smithi074_44586_object9118:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:43.815 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.585664+0000", 2020-08-07T15:10:43.815 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 77.597773038, 2020-08-07T15:10:43.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.411271632, 2020-08-07T15:10:43.816 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:43.817 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:43.833 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:43.833 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:43.833 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:43.834 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72952 2020-08-07T15:10:43.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.834 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:43.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:43.836 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.585664+0000", 2020-08-07T15:10:43.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:43.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.853 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:43.853 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.585664+0000", 2020-08-07T15:10:43.853 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0828999999999997e-05 2020-08-07T15:10:43.854 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.854 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.854 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:43.854 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.585715+0000", 2020-08-07T15:10:43.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1019999999999996e-06 2020-08-07T15:10:43.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.855 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:43.855 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.585720+0000", 2020-08-07T15:10:43.855 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4185000000000001e-05 2020-08-07T15:10:43.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.856 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.856 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:43.856 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.585734+0000", 2020-08-07T15:10:43.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5009999999999998e-06 2020-08-07T15:10:43.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:43.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.585742+0000", 2020-08-07T15:10:43.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0027632669999999998 2020-08-07T15:10:43.870 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.871 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:43.872 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.588505+0000", 2020-08-07T15:10:43.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4177000000000003e-05 2020-08-07T15:10:43.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.873 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:43.873 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.588599+0000", 2020-08-07T15:10:43.874 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013466500000000001 2020-08-07T15:10:43.874 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.874 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:10:43.875 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.588734+0000", 2020-08-07T15:10:43.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001231181 2020-08-07T15:10:43.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:43.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.589965+0000", 2020-08-07T15:10:43.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0083227890000000006 2020-08-07T15:10:43.876 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.877 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:43.877 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.598288+0000", 2020-08-07T15:10:43.877 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39851268299999998 2020-08-07T15:10:43.878 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.878 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.878 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:43.878 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.996800+0000", 2020-08-07T15:10:43.878 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.821e-06 2020-08-07T15:10:43.879 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.879 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:43.879 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.996802+0000", 2020-08-07T15:10:43.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8633000000000004e-05 2020-08-07T15:10:43.880 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.880 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:43.880 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.996881+0000", 2020-08-07T15:10:43.880 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4799000000000002e-05 2020-08-07T15:10:43.882 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:43.883 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:43.883 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:43.883 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.996936+0000", 2020-08-07T15:10:43.883 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.411271632 2020-08-07T15:10:43.883 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.883 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:43.884 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.884 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:43.884 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:43.884 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:43.885 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:43.184+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:43.888 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:10:43.888 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 23 16 37072 37056 12.5859 12.1953 0.00720478 0.00992337 2020-08-07T15:10:43.889 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:10:43.890 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:43.884+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:43.993 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:43.986+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:44.094 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:44.092+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:44.115 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:44.116 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:10:44.116 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.116 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:38016 5.8 5:19f06fec:::benchmark_data_smithi074_53697_object4751:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e184)", 2020-08-07T15:10:44.116 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:43.998518+0000", 2020-08-07T15:10:44.117 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.112016034, 2020-08-07T15:10:44.119 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.11204604999999999, 2020-08-07T15:10:44.119 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:44.120 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "delayed", 2020-08-07T15:10:44.120 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:44.120 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:44.120 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:44.120 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 38016 2020-08-07T15:10:44.121 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.121 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:44.121 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.121 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:44.122 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.998518+0000", 2020-08-07T15:10:44.122 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:44.122 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.122 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.122 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:44.123 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.998518+0000", 2020-08-07T15:10:44.123 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6067999999999998e-05 2020-08-07T15:10:44.128 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.128 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.128 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:44.128 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.998544+0000", 2020-08-07T15:10:44.128 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5619999999999997e-06 2020-08-07T15:10:44.130 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.130 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.130 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:44.130 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.998549+0000", 2020-08-07T15:10:44.131 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3830000000000003e-06 2020-08-07T15:10:44.131 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.131 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.131 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:44.131 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.998556+0000", 2020-08-07T15:10:44.131 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2710000000000001e-06 2020-08-07T15:10:44.132 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.132 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.132 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:44.132 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.998560+0000", 2020-08-07T15:10:44.132 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2907999999999997e-05 2020-08-07T15:10:44.133 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.133 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.133 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:44.133 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.998592+0000", 2020-08-07T15:10:44.133 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9379999999999998e-05 2020-08-07T15:10:44.140 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.141 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.141 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for ondisk", 2020-08-07T15:10:44.141 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.998642+0000", 2020-08-07T15:10:44.141 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012357199999999999 2020-08-07T15:10:44.142 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.142 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:44.142 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.142 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.142 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.143 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:38408 5.a 5:59136dd4:::benchmark_data_smithi074_53697_object4800:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e184)", 2020-08-07T15:10:44.143 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:43.998905+0000", 2020-08-07T15:10:44.143 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.111629643, 2020-08-07T15:10:44.143 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.111759278, 2020-08-07T15:10:44.143 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:44.143 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:44.144 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:44.144 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:44.144 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:44.144 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 38408 2020-08-07T15:10:44.144 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.145 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:44.145 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.146 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:44.146 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.998905+0000", 2020-08-07T15:10:44.148 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:44.148 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.148 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.148 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:44.149 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.998905+0000", 2020-08-07T15:10:44.149 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6078999999999997e-05 2020-08-07T15:10:44.149 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.149 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.149 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:44.150 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.998951+0000", 2020-08-07T15:10:44.150 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9370000000000003e-06 2020-08-07T15:10:44.150 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.150 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.150 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:44.151 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.998956+0000", 2020-08-07T15:10:44.151 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0011e-05 2020-08-07T15:10:44.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.151 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:44.152 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.998966+0000", 2020-08-07T15:10:44.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.481e-06 2020-08-07T15:10:44.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:44.153 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.998970+0000", 2020-08-07T15:10:44.153 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000162139 2020-08-07T15:10:44.153 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.153 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.153 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:44.154 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.999132+0000", 2020-08-07T15:10:44.154 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6292999999999994e-05 2020-08-07T15:10:44.154 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.155 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.155 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:44.155 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.999229+0000", 2020-08-07T15:10:44.155 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020242599999999999 2020-08-07T15:10:44.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.156 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4", 2020-08-07T15:10:44.157 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.999431+0000", 2020-08-07T15:10:44.157 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001610467 2020-08-07T15:10:44.160 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.160 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.160 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:44.160 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:44.001042+0000", 2020-08-07T15:10:44.161 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1200000000000001e-06 2020-08-07T15:10:44.161 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.161 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.161 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:44.161 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:44.001043+0000", 2020-08-07T15:10:44.162 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8198e-05 2020-08-07T15:10:44.162 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.162 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.162 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:44.162 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:44.001111+0000", 2020-08-07T15:10:44.162 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0022061509999999999 2020-08-07T15:10:44.163 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.163 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:44.163 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.163 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.164 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:38016 5.8 5:19f06fec:::benchmark_data_smithi074_53697_object4751:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e184)", 2020-08-07T15:10:44.164 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:43.762628+0000", 2020-08-07T15:10:44.164 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.34790634500000001, 2020-08-07T15:10:44.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.34814085900000002, 2020-08-07T15:10:44.164 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:44.165 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:10:44.165 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:44.165 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:44.165 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:44.165 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 38016 2020-08-07T15:10:44.167 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.167 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:44.167 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.170 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:44.171 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.762628+0000", 2020-08-07T15:10:44.171 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:44.171 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.171 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.171 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:44.172 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.762628+0000", 2020-08-07T15:10:44.172 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8640999999999999e-05 2020-08-07T15:10:44.172 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.172 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.172 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:44.173 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.762657+0000", 2020-08-07T15:10:44.180 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9589999999999998e-06 2020-08-07T15:10:44.189 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.189 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.190 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:44.190 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.762664+0000", 2020-08-07T15:10:44.191 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9909999999999999e-06 2020-08-07T15:10:44.192 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.192 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.192 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:44.192 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.762670+0000", 2020-08-07T15:10:44.193 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8480000000000001e-06 2020-08-07T15:10:44.193 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.193 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.193 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:44.193 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.762672+0000", 2020-08-07T15:10:44.194 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0022267400000000001 2020-08-07T15:10:44.194 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.194 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.194 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:44.194 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.764899+0000", 2020-08-07T15:10:44.194 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6581000000000002e-05 2020-08-07T15:10:44.198 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.198 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.198 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:44.198 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.764996+0000", 2020-08-07T15:10:44.199 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016199399999999999 2020-08-07T15:10:44.199 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.199 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.199 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:10:44.199 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.765158+0000", 2020-08-07T15:10:44.200 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002598941 2020-08-07T15:10:44.200 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.200 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.200 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:44.200 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.767757+0000", 2020-08-07T15:10:44.201 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00087624899999999995 2020-08-07T15:10:44.201 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.201 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.201 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:44.201 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:43.768633+0000", 2020-08-07T15:10:44.210 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0060049439999999999 2020-08-07T15:10:44.210 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.211 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:44.211 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.211 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.211 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.211 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:38016 5.8 5:19f06fec:::benchmark_data_smithi074_53697_object4751:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e184)", 2020-08-07T15:10:44.212 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:44.025438+0000", 2020-08-07T15:10:44.212 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.085096382999999998, 2020-08-07T15:10:44.212 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.085435659999999997, 2020-08-07T15:10:44.212 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:44.212 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "delayed", 2020-08-07T15:10:44.213 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:44.213 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:44.213 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:44.213 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 38016 2020-08-07T15:10:44.214 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.214 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:44.214 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.214 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:44.214 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:44.025438+0000", 2020-08-07T15:10:44.215 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:44.215 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.215 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.216 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:44.216 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:44.025438+0000", 2020-08-07T15:10:44.225 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1637e-05 2020-08-07T15:10:44.226 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.226 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.226 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:44.226 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:44.025490+0000", 2020-08-07T15:10:44.227 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4019999999999996e-06 2020-08-07T15:10:44.227 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.227 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.227 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:44.227 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:44.025497+0000", 2020-08-07T15:10:44.228 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2101e-05 2020-08-07T15:10:44.228 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.228 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.228 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:44.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:44.025509+0000", 2020-08-07T15:10:44.229 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7600000000000002e-06 2020-08-07T15:10:44.229 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.229 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.229 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:44.229 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:44.025517+0000", 2020-08-07T15:10:44.230 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9466000000000003e-05 2020-08-07T15:10:44.230 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.231 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.231 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:44.231 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:44.025576+0000", 2020-08-07T15:10:44.231 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1099000000000005e-05 2020-08-07T15:10:44.232 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.232 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.233 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for ondisk", 2020-08-07T15:10:44.233 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:44.025647+0000", 2020-08-07T15:10:44.233 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020946499999999999 2020-08-07T15:10:44.233 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.233 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:44.234 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.234 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.234 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:10:44.234 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 4 2020-08-07T15:10:44.234 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:44.239 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 24 16 38735 38719 12.6028 12.9922 0.00341528 0.00990869 2020-08-07T15:10:44.240 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:44.234+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:44.240 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:10:44.342 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:44.337+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:44.456 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:44.445+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:44.459 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:44.459 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:10:44.459 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:10:44.459 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:10:44.459 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:44.485 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:10:44.486 INFO:teuthology.orchestra.run.smithi074.stderr:set pool 1 pg_num to 31 2020-08-07T15:10:44.578 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:44.574+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:44.656 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:44.653+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:44.695 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:44.696 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:10:44.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:10:44.696 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:10:44.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.697 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:30968 4.4 e98/81 ondisk, result = 0)", 2020-08-07T15:10:44.697 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:10:44.697 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 118.960721795, 2020-08-07T15:10:44.697 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96211418999999998, 2020-08-07T15:10:44.700 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:44.701 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:44.701 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:44.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.701 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:44.701 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:10:44.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:44.702 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.702 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.703 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:44.704 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:10:44.705 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2387e-05 2020-08-07T15:10:44.705 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.705 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.705 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:44.705 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729821+0000", 2020-08-07T15:10:44.706 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9909999999999999e-06 2020-08-07T15:10:44.706 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.706 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.706 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:44.706 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729827+0000", 2020-08-07T15:10:44.707 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96185524600000005 2020-08-07T15:10:44.707 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.707 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:44.707 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691682+0000", 2020-08-07T15:10:44.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8760000000000005e-06 2020-08-07T15:10:44.708 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.710 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:44.710 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691690+0000", 2020-08-07T15:10:44.710 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9682000000000001e-05 2020-08-07T15:10:44.710 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.710 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.711 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:44.711 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691759+0000", 2020-08-07T15:10:44.711 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5381000000000001e-05 2020-08-07T15:10:44.711 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.712 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:44.712 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691775+0000", 2020-08-07T15:10:44.712 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000127627 2020-08-07T15:10:44.712 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.712 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.713 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:44.713 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691902+0000", 2020-08-07T15:10:44.713 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96211418999999998 2020-08-07T15:10:44.714 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.714 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:44.714 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.714 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.714 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.715 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42472 4.4 4:2ff82af8:::benchmark_data_smithi074_44586_object5308:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:10:44.715 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:10:44.715 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 104.42325902100001, 2020-08-07T15:10:44.715 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89653816399999997, 2020-08-07T15:10:44.715 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:44.716 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:44.716 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:44.716 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:44.716 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:44.717 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42472 2020-08-07T15:10:44.717 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.717 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:44.717 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.717 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:44.718 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:10:44.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:44.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.718 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.718 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:44.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:10:44.722 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9591e-05 2020-08-07T15:10:44.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.722 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:44.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267301+0000", 2020-08-07T15:10:44.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7490000000000004e-06 2020-08-07T15:10:44.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:44.728 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267306+0000", 2020-08-07T15:10:44.728 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2850000000000001e-05 2020-08-07T15:10:44.729 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.730 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.731 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:44.731 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267319+0000", 2020-08-07T15:10:44.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9100000000000002e-06 2020-08-07T15:10:44.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:44.732 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267325+0000", 2020-08-07T15:10:44.732 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6584000000000001e-05 2020-08-07T15:10:44.733 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.733 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:44.733 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267362+0000", 2020-08-07T15:10:44.734 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9188000000000001e-05 2020-08-07T15:10:44.734 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.734 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.734 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:44.734 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267451+0000", 2020-08-07T15:10:44.735 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000182917 2020-08-07T15:10:44.735 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.735 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.735 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:44.735 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267634+0000", 2020-08-07T15:10:44.735 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002484708 2020-08-07T15:10:44.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.736 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.736 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:44.736 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270119+0000", 2020-08-07T15:10:44.736 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0299999999999997e-07 2020-08-07T15:10:44.737 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.737 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.738 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:44.738 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270119+0000", 2020-08-07T15:10:44.738 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8251999999999996e-05 2020-08-07T15:10:44.738 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.738 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.739 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:44.739 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270188+0000", 2020-08-07T15:10:44.739 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24443688 2020-08-07T15:10:44.739 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.739 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.740 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:44.741 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.514625+0000", 2020-08-07T15:10:44.741 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.64911260599999998 2020-08-07T15:10:44.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.741 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.742 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:44.743 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163737+0000", 2020-08-07T15:10:44.744 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2100000000000001e-06 2020-08-07T15:10:44.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:44.744 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163738+0000", 2020-08-07T15:10:44.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0816000000000003e-05 2020-08-07T15:10:44.745 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.745 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:44.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163789+0000", 2020-08-07T15:10:44.746 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89653816399999997 2020-08-07T15:10:44.746 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.746 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:44.746 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.746 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.747 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.747 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:42472 4.4 e112/81 ondisk, result = 0)", 2020-08-07T15:10:44.747 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.271353+0000", 2020-08-07T15:10:44.747 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 104.419156984, 2020-08-07T15:10:44.747 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89244306299999998, 2020-08-07T15:10:44.748 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:44.748 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:44.748 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:44.749 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.749 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:44.749 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.271353+0000", 2020-08-07T15:10:44.749 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:44.749 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.750 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.750 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:44.750 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.271353+0000", 2020-08-07T15:10:44.750 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7875999999999999e-05 2020-08-07T15:10:44.751 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.752 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:44.752 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.271381+0000", 2020-08-07T15:10:44.752 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4819999999999996e-06 2020-08-07T15:10:44.753 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.753 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.753 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:44.754 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.271385+0000", 2020-08-07T15:10:44.754 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89228763300000002 2020-08-07T15:10:44.754 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.754 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.754 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:44.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163673+0000", 2020-08-07T15:10:44.755 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9949999999999996e-06 2020-08-07T15:10:44.755 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.755 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:44.756 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163678+0000", 2020-08-07T15:10:44.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5485000000000003e-05 2020-08-07T15:10:44.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:44.757 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163714+0000", 2020-08-07T15:10:44.757 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4785e-05 2020-08-07T15:10:44.757 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.757 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:44.758 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163728+0000", 2020-08-07T15:10:44.758 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7806999999999996e-05 2020-08-07T15:10:44.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.759 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:44.759 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163796+0000", 2020-08-07T15:10:44.759 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89244306299999998 2020-08-07T15:10:44.760 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.760 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:44.760 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.760 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.760 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.761 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72688 4.4 4:295d3f6e:::benchmark_data_smithi074_44586_object9085:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e129)", 2020-08-07T15:10:44.762 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:10:44.762 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 79.293676833999996, 2020-08-07T15:10:44.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7869782759999999, 2020-08-07T15:10:44.763 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:44.763 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:10:44.763 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:44.764 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:44.764 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:44.764 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72688 2020-08-07T15:10:44.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.765 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:44.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.765 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:44.765 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:10:44.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:44.766 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.766 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:44.766 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:10:44.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8721999999999999e-05 2020-08-07T15:10:44.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:44.767 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396882+0000", 2020-08-07T15:10:44.768 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2639999999999999e-06 2020-08-07T15:10:44.768 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.768 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.768 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:44.768 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396887+0000", 2020-08-07T15:10:44.769 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0767999999999999e-05 2020-08-07T15:10:44.769 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.769 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.769 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:44.769 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396898+0000", 2020-08-07T15:10:44.770 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1860000000000002e-06 2020-08-07T15:10:44.770 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.770 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.770 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:44.771 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396902+0000", 2020-08-07T15:10:44.772 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0037638649999999999 2020-08-07T15:10:44.773 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.773 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.773 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:44.774 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400666+0000", 2020-08-07T15:10:44.774 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000118077 2020-08-07T15:10:44.774 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.774 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.775 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:44.775 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400784+0000", 2020-08-07T15:10:44.775 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000136728 2020-08-07T15:10:44.775 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:10:44.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400921+0000", 2020-08-07T15:10:44.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001595443 2020-08-07T15:10:44.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:44.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.402516+0000", 2020-08-07T15:10:44.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.014817907 2020-08-07T15:10:44.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:44.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417334+0000", 2020-08-07T15:10:44.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.75e-06 2020-08-07T15:10:44.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.779 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:44.779 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417336+0000", 2020-08-07T15:10:44.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8066999999999996e-05 2020-08-07T15:10:44.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.780 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:44.780 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417424+0000", 2020-08-07T15:10:44.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002210573 2020-08-07T15:10:44.780 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.780 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:44.781 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.419635+0000", 2020-08-07T15:10:44.781 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1379000000000001e-05 2020-08-07T15:10:44.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:44.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.419646+0000", 2020-08-07T15:10:44.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.004643409 2020-08-07T15:10:44.785 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.786 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:44.786 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424289+0000", 2020-08-07T15:10:44.786 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7563000000000001e-05 2020-08-07T15:10:44.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:44.787 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424307+0000", 2020-08-07T15:10:44.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75943629400000001 2020-08-07T15:10:44.787 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.787 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.788 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:44.788 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183743+0000", 2020-08-07T15:10:44.788 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8281000000000006e-05 2020-08-07T15:10:44.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.789 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:44.789 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183812+0000", 2020-08-07T15:10:44.789 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7869782759999999 2020-08-07T15:10:44.789 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.789 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:44.790 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.790 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72985 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:44.790 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:10:44.790 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 79.077691014999999, 2020-08-07T15:10:44.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869466839999999, 2020-08-07T15:10:44.791 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:44.791 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:44.791 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:44.791 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:44.792 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:44.792 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72985 2020-08-07T15:10:44.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.792 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:44.792 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.794 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:44.795 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:10:44.795 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:44.795 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.796 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:44.796 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:10:44.796 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8003999999999997e-05 2020-08-07T15:10:44.796 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.797 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.797 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:44.797 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612857+0000", 2020-08-07T15:10:44.797 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.246e-06 2020-08-07T15:10:44.797 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.798 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.798 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:44.798 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612863+0000", 2020-08-07T15:10:44.798 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5146e-05 2020-08-07T15:10:44.798 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:44.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612878+0000", 2020-08-07T15:10:44.799 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1940000000000003e-06 2020-08-07T15:10:44.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:44.800 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612885+0000", 2020-08-07T15:10:44.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0349999999999997e-05 2020-08-07T15:10:44.801 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.801 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.801 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:44.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612935+0000", 2020-08-07T15:10:44.802 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1029999999999996e-06 2020-08-07T15:10:44.802 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.802 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.802 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:44.802 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612942+0000", 2020-08-07T15:10:44.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81140097700000002 2020-08-07T15:10:44.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:44.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424343+0000", 2020-08-07T15:10:44.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1799999999999996e-06 2020-08-07T15:10:44.806 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:44.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424351+0000", 2020-08-07T15:10:44.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75948665299999996 2020-08-07T15:10:44.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:44.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183838+0000", 2020-08-07T15:10:44.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000179187 2020-08-07T15:10:44.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.808 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:44.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.184017+0000", 2020-08-07T15:10:44.809 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024691600000000002 2020-08-07T15:10:44.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.809 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:44.809 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.184264+0000", 2020-08-07T15:10:44.810 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006362721 2020-08-07T15:10:44.810 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.810 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.810 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:44.810 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190627+0000", 2020-08-07T15:10:44.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7400000000000001e-06 2020-08-07T15:10:44.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.811 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:44.811 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190629+0000", 2020-08-07T15:10:44.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1305999999999999e-05 2020-08-07T15:10:44.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:44.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190690+0000", 2020-08-07T15:10:44.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0088507159999999998 2020-08-07T15:10:44.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.813 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:44.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199541+0000", 2020-08-07T15:10:44.813 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015864699999999999 2020-08-07T15:10:44.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:44.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199699+0000", 2020-08-07T15:10:44.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.23e-07 2020-08-07T15:10:44.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.817 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:44.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199700+0000", 2020-08-07T15:10:44.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5775e-05 2020-08-07T15:10:44.818 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.818 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.818 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:44.818 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199766+0000", 2020-08-07T15:10:44.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869466839999999 2020-08-07T15:10:44.819 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.819 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:44.819 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.819 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.820 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72986 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:44.820 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:10:44.820 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 79.077580432999994, 2020-08-07T15:10:44.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.586946441, 2020-08-07T15:10:44.821 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:44.821 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:44.821 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:44.821 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:44.822 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:44.822 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72986 2020-08-07T15:10:44.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.822 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:44.822 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:44.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:10:44.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:44.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:44.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:10:44.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9861999999999997e-05 2020-08-07T15:10:44.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:44.828 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612990+0000", 2020-08-07T15:10:44.828 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4350000000000001e-06 2020-08-07T15:10:44.829 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.829 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.830 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:44.830 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612993+0000", 2020-08-07T15:10:44.830 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9529999999999999e-06 2020-08-07T15:10:44.831 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.831 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.831 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:44.831 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613001+0000", 2020-08-07T15:10:44.831 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4309999999999998e-06 2020-08-07T15:10:44.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.832 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:44.832 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613004+0000", 2020-08-07T15:10:44.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5284000000000001e-05 2020-08-07T15:10:44.833 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:44.833 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613030+0000", 2020-08-07T15:10:44.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2400000000000001e-06 2020-08-07T15:10:44.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:44.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613034+0000", 2020-08-07T15:10:44.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81134136199999995 2020-08-07T15:10:44.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:44.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424375+0000", 2020-08-07T15:10:44.838 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3520000000000001e-06 2020-08-07T15:10:44.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.838 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:44.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424383+0000", 2020-08-07T15:10:44.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76064437799999995 2020-08-07T15:10:44.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.839 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:44.840 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185027+0000", 2020-08-07T15:10:44.840 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000119719 2020-08-07T15:10:44.840 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.840 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:44.841 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185147+0000", 2020-08-07T15:10:44.841 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016829400000000001 2020-08-07T15:10:44.841 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.842 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:44.842 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185315+0000", 2020-08-07T15:10:44.842 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0053821199999999998 2020-08-07T15:10:44.842 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.842 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.843 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:44.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190697+0000", 2020-08-07T15:10:44.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6899999999999996e-07 2020-08-07T15:10:44.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.844 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:44.844 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190698+0000", 2020-08-07T15:10:44.844 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2072000000000001e-05 2020-08-07T15:10:44.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.851 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:44.851 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190730+0000", 2020-08-07T15:10:44.851 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0088501440000000008 2020-08-07T15:10:44.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:44.852 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199580+0000", 2020-08-07T15:10:44.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00023199900000000001 2020-08-07T15:10:44.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.853 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:44.853 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199812+0000", 2020-08-07T15:10:44.855 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6300000000000005e-07 2020-08-07T15:10:44.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.856 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.856 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:44.856 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199813+0000", 2020-08-07T15:10:44.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3363999999999998e-05 2020-08-07T15:10:44.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:44.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199876+0000", 2020-08-07T15:10:44.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.586946441 2020-08-07T15:10:44.858 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.858 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:44.858 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.858 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.859 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72987 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:44.859 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:10:44.859 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 79.077484678000005, 2020-08-07T15:10:44.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869598709999999, 2020-08-07T15:10:44.861 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:44.862 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:44.862 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:44.862 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:44.862 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:44.862 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72987 2020-08-07T15:10:44.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.863 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:44.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:44.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:10:44.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:44.864 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.865 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:44.865 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:10:44.866 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1851999999999999e-05 2020-08-07T15:10:44.866 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.866 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.867 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:44.867 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613057+0000", 2020-08-07T15:10:44.867 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0429999999999999e-06 2020-08-07T15:10:44.868 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.868 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.868 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:44.869 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613060+0000", 2020-08-07T15:10:44.869 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1060000000000002e-06 2020-08-07T15:10:44.870 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.874 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:44.874 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613067+0000", 2020-08-07T15:10:44.874 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8040000000000002e-06 2020-08-07T15:10:44.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.876 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:44.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613070+0000", 2020-08-07T15:10:44.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6857000000000001e-05 2020-08-07T15:10:44.877 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.877 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.877 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:44.878 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613097+0000", 2020-08-07T15:10:44.878 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8770000000000001e-06 2020-08-07T15:10:44.878 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:44.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613102+0000", 2020-08-07T15:10:44.890 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81131066399999996 2020-08-07T15:10:44.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.890 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:44.890 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424413+0000", 2020-08-07T15:10:44.891 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4599999999999997e-06 2020-08-07T15:10:44.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.891 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.891 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:44.891 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424420+0000", 2020-08-07T15:10:44.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76132809700000004 2020-08-07T15:10:44.892 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.892 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.892 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:44.892 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185748+0000", 2020-08-07T15:10:44.893 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.4264000000000005e-05 2020-08-07T15:10:44.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.893 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:44.893 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185832+0000", 2020-08-07T15:10:44.894 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112803 2020-08-07T15:10:44.894 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.894 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.894 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:44.894 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185945+0000", 2020-08-07T15:10:44.895 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0089954130000000007 2020-08-07T15:10:44.895 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.899 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.900 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:44.900 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194941+0000", 2020-08-07T15:10:44.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.091e-06 2020-08-07T15:10:44.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.901 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:44.901 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194943+0000", 2020-08-07T15:10:44.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7423999999999996e-05 2020-08-07T15:10:44.901 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.902 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.902 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:44.902 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195020+0000", 2020-08-07T15:10:44.903 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045973300000000002 2020-08-07T15:10:44.903 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.903 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:44.903 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199618+0000", 2020-08-07T15:10:44.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000302009 2020-08-07T15:10:44.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.904 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:44.904 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199920+0000", 2020-08-07T15:10:44.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7999999999999996e-07 2020-08-07T15:10:44.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:44.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199920+0000", 2020-08-07T15:10:44.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5296999999999995e-05 2020-08-07T15:10:44.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.906 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.906 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:44.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199985+0000", 2020-08-07T15:10:44.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869598709999999 2020-08-07T15:10:44.909 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.909 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:44.910 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.910 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72988 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:44.910 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:10:44.911 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 79.077420098000005, 2020-08-07T15:10:44.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587111143, 2020-08-07T15:10:44.911 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:44.911 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:44.911 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:44.912 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:44.912 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:44.912 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72988 2020-08-07T15:10:44.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.912 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:44.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:44.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:10:44.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:44.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:44.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:10:44.914 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6563e-05 2020-08-07T15:10:44.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:44.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613117+0000", 2020-08-07T15:10:44.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7180000000000001e-06 2020-08-07T15:10:44.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:44.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613119+0000", 2020-08-07T15:10:44.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1789999999999999e-06 2020-08-07T15:10:44.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:44.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613124+0000", 2020-08-07T15:10:44.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3539999999999998e-06 2020-08-07T15:10:44.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:44.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613127+0000", 2020-08-07T15:10:44.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3697e-05 2020-08-07T15:10:44.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.922 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:44.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613151+0000", 2020-08-07T15:10:44.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7559999999999999e-06 2020-08-07T15:10:44.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:44.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613155+0000", 2020-08-07T15:10:44.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81128859600000003 2020-08-07T15:10:44.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:44.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424444+0000", 2020-08-07T15:10:44.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9619999999999996e-06 2020-08-07T15:10:44.925 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.926 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:44.926 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424451+0000", 2020-08-07T15:10:44.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76286712899999998 2020-08-07T15:10:44.926 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:44.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187318+0000", 2020-08-07T15:10:44.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011616300000000001 2020-08-07T15:10:44.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:44.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187434+0000", 2020-08-07T15:10:44.932 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019347400000000001 2020-08-07T15:10:44.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.932 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.932 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:44.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187628+0000", 2020-08-07T15:10:44.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074049090000000003 2020-08-07T15:10:44.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.933 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.933 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:44.933 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195033+0000", 2020-08-07T15:10:44.934 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1090000000000001e-06 2020-08-07T15:10:44.934 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.934 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.934 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:44.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195034+0000", 2020-08-07T15:10:44.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8566999999999997e-05 2020-08-07T15:10:44.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:44.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195092+0000", 2020-08-07T15:10:44.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004570457 2020-08-07T15:10:44.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:44.936 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199663+0000", 2020-08-07T15:10:44.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00046757899999999998 2020-08-07T15:10:44.937 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:44.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200130+0000", 2020-08-07T15:10:44.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6700000000000002e-07 2020-08-07T15:10:44.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.938 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:44.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200131+0000", 2020-08-07T15:10:44.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9963999999999996e-05 2020-08-07T15:10:44.942 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.942 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:44.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200201+0000", 2020-08-07T15:10:44.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587111143 2020-08-07T15:10:44.943 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.943 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:44.943 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.944 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72989 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:44.944 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:10:44.944 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 79.07736371, 2020-08-07T15:10:44.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5872506399999999, 2020-08-07T15:10:44.945 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:44.945 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:44.945 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:44.945 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:44.946 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:44.946 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72989 2020-08-07T15:10:44.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.947 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:44.947 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.948 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:44.948 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:10:44.948 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:44.949 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:44.952 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:10:44.952 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6081999999999999e-05 2020-08-07T15:10:44.952 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.953 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.953 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:44.953 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613172+0000", 2020-08-07T15:10:44.954 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6570000000000001e-06 2020-08-07T15:10:44.954 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.954 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.955 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:44.955 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613175+0000", 2020-08-07T15:10:44.955 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7740000000000001e-06 2020-08-07T15:10:44.956 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.956 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.957 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:44.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613181+0000", 2020-08-07T15:10:44.958 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2759999999999999e-06 2020-08-07T15:10:44.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:44.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613183+0000", 2020-08-07T15:10:44.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3892000000000001e-05 2020-08-07T15:10:44.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:44.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613207+0000", 2020-08-07T15:10:44.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9870000000000001e-06 2020-08-07T15:10:44.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:44.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613211+0000", 2020-08-07T15:10:44.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81126230700000002 2020-08-07T15:10:44.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:44.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424473+0000", 2020-08-07T15:10:44.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1460000000000001e-06 2020-08-07T15:10:44.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.969 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.970 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:44.970 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424480+0000", 2020-08-07T15:10:44.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76413073099999995 2020-08-07T15:10:44.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.975 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:44.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188610+0000", 2020-08-07T15:10:44.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011278899999999999 2020-08-07T15:10:44.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:44.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188723+0000", 2020-08-07T15:10:44.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000213422 2020-08-07T15:10:44.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.982 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:44.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188936+0000", 2020-08-07T15:10:44.982 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0061672929999999999 2020-08-07T15:10:44.985 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.985 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.986 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:44.986 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195104+0000", 2020-08-07T15:10:44.986 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8199999999999999e-07 2020-08-07T15:10:44.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.987 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:44.987 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195105+0000", 2020-08-07T15:10:44.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2056000000000004e-05 2020-08-07T15:10:44.987 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:44.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195167+0000", 2020-08-07T15:10:44.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0051009769999999996 2020-08-07T15:10:44.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:44.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200268+0000", 2020-08-07T15:10:44.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9428000000000001e-05 2020-08-07T15:10:44.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.993 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:44.993 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200327+0000", 2020-08-07T15:10:44.993 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9100000000000003e-07 2020-08-07T15:10:44.993 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.995 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.996 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:44.996 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200328+0000", 2020-08-07T15:10:44.996 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9250000000000003e-05 2020-08-07T15:10:44.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.997 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.997 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:44.997 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200397+0000", 2020-08-07T15:10:44.997 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5872506399999999 2020-08-07T15:10:44.997 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.998 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:44.998 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:44.998 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:44.998 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:44.998 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72990 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:44.999 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:10:44.999 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 79.077306418000006, 2020-08-07T15:10:44.999 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587335105, 2020-08-07T15:10:44.999 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:44.999 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:45.000 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:45.000 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:45.000 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:45.001 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72990 2020-08-07T15:10:45.001 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.001 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:45.001 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.002 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:45.002 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:10:45.002 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:45.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:45.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:10:45.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5276999999999999e-05 2020-08-07T15:10:45.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:45.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613229+0000", 2020-08-07T15:10:45.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7219999999999999e-06 2020-08-07T15:10:45.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:45.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613232+0000", 2020-08-07T15:10:45.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1170000000000003e-06 2020-08-07T15:10:45.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:45.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613237+0000", 2020-08-07T15:10:45.008 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2309999999999999e-06 2020-08-07T15:10:45.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:45.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613239+0000", 2020-08-07T15:10:45.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2469000000000001e-05 2020-08-07T15:10:45.009 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.009 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.009 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.010 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613261+0000", 2020-08-07T15:10:45.010 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9280000000000001e-06 2020-08-07T15:10:45.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.011 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:45.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613266+0000", 2020-08-07T15:10:45.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81123414599999999 2020-08-07T15:10:45.012 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.012 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424501+0000", 2020-08-07T15:10:45.013 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5300000000000002e-06 2020-08-07T15:10:45.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.014 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:45.014 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424507+0000", 2020-08-07T15:10:45.014 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76488660200000003 2020-08-07T15:10:45.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.015 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.015 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.015 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189394+0000", 2020-08-07T15:10:45.015 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6878000000000003e-05 2020-08-07T15:10:45.016 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:45.017 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189491+0000", 2020-08-07T15:10:45.017 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000204477 2020-08-07T15:10:45.018 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.018 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.018 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:45.018 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189695+0000", 2020-08-07T15:10:45.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0065180940000000003 2020-08-07T15:10:45.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.019 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.019 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:45.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196213+0000", 2020-08-07T15:10:45.019 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4920000000000001e-06 2020-08-07T15:10:45.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:45.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196215+0000", 2020-08-07T15:10:45.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9950000000000006e-05 2020-08-07T15:10:45.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:45.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196285+0000", 2020-08-07T15:10:45.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004163674 2020-08-07T15:10:45.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:45.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200448+0000", 2020-08-07T15:10:45.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6919999999999999e-05 2020-08-07T15:10:45.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:45.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200485+0000", 2020-08-07T15:10:45.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1699999999999998e-07 2020-08-07T15:10:45.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.038 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:45.038 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200486+0000", 2020-08-07T15:10:45.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3081000000000002e-05 2020-08-07T15:10:45.039 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.039 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.041 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:45.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200539+0000", 2020-08-07T15:10:45.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587335105 2020-08-07T15:10:45.041 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:45.042 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:45.042 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:45.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.042 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.042 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72991 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:45.043 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:10:45.043 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 79.077250892999999, 2020-08-07T15:10:45.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5876210340000001, 2020-08-07T15:10:45.043 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:45.043 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:45.044 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:45.044 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:45.044 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:45.044 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72991 2020-08-07T15:10:45.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.045 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:45.045 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:45.045 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:10:45.045 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:45.046 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.049 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.049 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:45.050 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:10:45.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5763999999999999e-05 2020-08-07T15:10:45.050 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.050 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.050 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:45.051 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613285+0000", 2020-08-07T15:10:45.051 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6479999999999999e-06 2020-08-07T15:10:45.051 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.051 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:45.052 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613288+0000", 2020-08-07T15:10:45.052 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1909999999999999e-06 2020-08-07T15:10:45.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.052 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.052 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:45.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613293+0000", 2020-08-07T15:10:45.053 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2560000000000001e-06 2020-08-07T15:10:45.053 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.054 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:45.054 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613295+0000", 2020-08-07T15:10:45.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2561e-05 2020-08-07T15:10:45.055 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.057 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613318+0000", 2020-08-07T15:10:45.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6349999999999999e-06 2020-08-07T15:10:45.058 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.058 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.059 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:45.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613321+0000", 2020-08-07T15:10:45.059 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81120592400000002 2020-08-07T15:10:45.059 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424527+0000", 2020-08-07T15:10:45.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8540000000000002e-06 2020-08-07T15:10:45.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.062 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.062 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:45.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424533+0000", 2020-08-07T15:10:45.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76623392800000001 2020-08-07T15:10:45.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190767+0000", 2020-08-07T15:10:45.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3990000000000001e-05 2020-08-07T15:10:45.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.065 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:45.065 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190841+0000", 2020-08-07T15:10:45.066 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011444599999999999 2020-08-07T15:10:45.066 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.066 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.066 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:45.066 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190955+0000", 2020-08-07T15:10:45.067 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0053397499999999999 2020-08-07T15:10:45.067 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.068 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.068 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:45.068 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196295+0000", 2020-08-07T15:10:45.069 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006e-06 2020-08-07T15:10:45.069 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.069 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.069 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:45.070 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196296+0000", 2020-08-07T15:10:45.070 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8940999999999997e-05 2020-08-07T15:10:45.070 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.070 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.071 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:45.071 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196355+0000", 2020-08-07T15:10:45.071 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042276700000000002 2020-08-07T15:10:45.071 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.071 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.072 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:45.072 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200583+0000", 2020-08-07T15:10:45.072 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00026035700000000001 2020-08-07T15:10:45.073 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.074 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.074 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:45.074 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200843+0000", 2020-08-07T15:10:45.075 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6700000000000003e-07 2020-08-07T15:10:45.075 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.075 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.076 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:45.076 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200844+0000", 2020-08-07T15:10:45.076 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6545999999999999e-05 2020-08-07T15:10:45.077 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.077 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.077 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:45.077 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200880+0000", 2020-08-07T15:10:45.078 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5876210340000001 2020-08-07T15:10:45.078 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:45.078 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:45.079 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:45.079 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.079 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.079 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72992 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:45.080 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:10:45.080 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 79.076163414000007, 2020-08-07T15:10:45.080 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5866241210000001, 2020-08-07T15:10:45.080 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:45.080 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:45.081 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:45.081 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:45.081 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:45.081 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72992 2020-08-07T15:10:45.081 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.082 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:45.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.082 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:45.082 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:10:45.082 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:45.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.083 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.083 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:45.083 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:10:45.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7543e-05 2020-08-07T15:10:45.084 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.084 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:45.084 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614374+0000", 2020-08-07T15:10:45.085 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9740000000000004e-06 2020-08-07T15:10:45.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.085 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:45.086 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614378+0000", 2020-08-07T15:10:45.099 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2660000000000007e-06 2020-08-07T15:10:45.099 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.099 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.100 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:45.100 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614387+0000", 2020-08-07T15:10:45.101 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2950000000000001e-06 2020-08-07T15:10:45.101 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.102 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.102 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:45.102 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614393+0000", 2020-08-07T15:10:45.102 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7279000000000001e-05 2020-08-07T15:10:45.103 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.103 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.103 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.103 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614420+0000", 2020-08-07T15:10:45.103 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9040000000000002e-06 2020-08-07T15:10:45.104 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.104 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.104 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:45.104 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614424+0000", 2020-08-07T15:10:45.104 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81012815699999996 2020-08-07T15:10:45.105 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.106 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.106 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.106 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424552+0000", 2020-08-07T15:10:45.106 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2879999999999998e-06 2020-08-07T15:10:45.107 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.107 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.107 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:45.107 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424558+0000", 2020-08-07T15:10:45.108 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76678665400000001 2020-08-07T15:10:45.108 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.108 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.108 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.108 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191345+0000", 2020-08-07T15:10:45.109 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010819600000000001 2020-08-07T15:10:45.109 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.109 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.109 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:45.109 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191453+0000", 2020-08-07T15:10:45.110 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000170829 2020-08-07T15:10:45.110 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.110 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.110 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:45.110 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191624+0000", 2020-08-07T15:10:45.111 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0047386679999999997 2020-08-07T15:10:45.111 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.111 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.111 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:45.111 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196363+0000", 2020-08-07T15:10:45.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6000000000000003e-07 2020-08-07T15:10:45.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.112 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:45.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196363+0000", 2020-08-07T15:10:45.113 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3421000000000002e-05 2020-08-07T15:10:45.113 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.113 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:45.114 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196417+0000", 2020-08-07T15:10:45.114 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042225539999999999 2020-08-07T15:10:45.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.114 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:45.115 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200639+0000", 2020-08-07T15:10:45.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00028031399999999999 2020-08-07T15:10:45.116 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.117 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:45.117 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200920+0000", 2020-08-07T15:10:45.117 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0299999999999999e-07 2020-08-07T15:10:45.118 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.118 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.118 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:45.118 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200920+0000", 2020-08-07T15:10:45.118 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0516000000000002e-05 2020-08-07T15:10:45.119 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.119 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.119 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:45.119 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200971+0000", 2020-08-07T15:10:45.119 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5866241210000001 2020-08-07T15:10:45.120 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:45.120 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:45.120 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:45.120 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.120 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.121 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73017 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:45.121 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:10:45.121 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 79.020655160999993, 2020-08-07T15:10:45.121 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.531181608, 2020-08-07T15:10:45.121 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:45.122 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:45.122 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:45.122 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:45.122 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:45.123 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73017 2020-08-07T15:10:45.123 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.123 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:45.123 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.123 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:45.124 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:10:45.124 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:45.124 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.124 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.124 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:45.125 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:10:45.125 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3018e-05 2020-08-07T15:10:45.125 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.125 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.126 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:45.127 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669898+0000", 2020-08-07T15:10:45.127 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5340000000000002e-06 2020-08-07T15:10:45.128 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.128 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.129 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:45.129 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669905+0000", 2020-08-07T15:10:45.129 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3307e-05 2020-08-07T15:10:45.130 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.130 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.130 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:45.131 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669919+0000", 2020-08-07T15:10:45.131 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7729999999999999e-06 2020-08-07T15:10:45.131 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.132 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.132 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:45.132 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669927+0000", 2020-08-07T15:10:45.132 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0361000000000001e-05 2020-08-07T15:10:45.133 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.133 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.133 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.134 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669977+0000", 2020-08-07T15:10:45.134 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0049999999999993e-06 2020-08-07T15:10:45.134 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.135 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.135 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:45.135 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669986+0000", 2020-08-07T15:10:45.136 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75459235199999997 2020-08-07T15:10:45.136 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.137 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.137 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.138 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424578+0000", 2020-08-07T15:10:45.138 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.925e-06 2020-08-07T15:10:45.138 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.139 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.139 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:45.140 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424584+0000", 2020-08-07T15:10:45.140 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76741703900000002 2020-08-07T15:10:45.140 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.141 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.141 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.141 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192001+0000", 2020-08-07T15:10:45.142 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000181196 2020-08-07T15:10:45.143 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.144 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.144 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:45.146 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192182+0000", 2020-08-07T15:10:45.146 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00021149299999999999 2020-08-07T15:10:45.146 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.146 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.147 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:45.147 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192394+0000", 2020-08-07T15:10:45.148 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004031242 2020-08-07T15:10:45.148 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.148 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.148 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:45.149 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196425+0000", 2020-08-07T15:10:45.149 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7800000000000001e-07 2020-08-07T15:10:45.149 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.149 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.150 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:45.150 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196426+0000", 2020-08-07T15:10:45.150 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5244999999999997e-05 2020-08-07T15:10:45.150 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.150 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.151 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:45.151 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196481+0000", 2020-08-07T15:10:45.151 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041989360000000003 2020-08-07T15:10:45.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:45.152 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200680+0000", 2020-08-07T15:10:45.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00032415299999999998 2020-08-07T15:10:45.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.153 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:45.153 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201004+0000", 2020-08-07T15:10:45.153 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9299999999999999e-07 2020-08-07T15:10:45.153 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.153 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.154 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:45.154 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201005+0000", 2020-08-07T15:10:45.154 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1857999999999999e-05 2020-08-07T15:10:45.154 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.155 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.155 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:45.155 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201037+0000", 2020-08-07T15:10:45.155 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.531181608 2020-08-07T15:10:45.155 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:45.156 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:45.156 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:45.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.156 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73018 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:45.157 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:10:45.157 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 79.019687356000006, 2020-08-07T15:10:45.157 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.530301616, 2020-08-07T15:10:45.157 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:45.157 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:45.159 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:45.159 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:45.159 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:45.159 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73018 2020-08-07T15:10:45.160 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.160 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:45.160 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.160 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:45.160 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:10:45.161 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:45.161 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.161 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.161 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:45.161 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:10:45.162 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9747000000000002e-05 2020-08-07T15:10:45.162 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.162 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.162 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:45.163 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670852+0000", 2020-08-07T15:10:45.163 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9430000000000002e-06 2020-08-07T15:10:45.164 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:45.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670856+0000", 2020-08-07T15:10:45.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0649999999999999e-06 2020-08-07T15:10:45.165 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.165 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.165 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:45.166 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670864+0000", 2020-08-07T15:10:45.178 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.924e-06 2020-08-07T15:10:45.178 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.179 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.179 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:45.180 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670868+0000", 2020-08-07T15:10:45.180 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6205000000000003e-05 2020-08-07T15:10:45.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.180 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.180 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.181 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670905+0000", 2020-08-07T15:10:45.181 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3289999999999996e-06 2020-08-07T15:10:45.181 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.181 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:45.182 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670912+0000", 2020-08-07T15:10:45.182 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75370200200000004 2020-08-07T15:10:45.182 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.183 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.183 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424614+0000", 2020-08-07T15:10:45.183 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5150000000000003e-06 2020-08-07T15:10:45.183 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.183 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.184 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:45.184 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424620+0000", 2020-08-07T15:10:45.184 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.768200153 2020-08-07T15:10:45.184 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.185 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.185 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.202 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192821+0000", 2020-08-07T15:10:45.202 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000111764 2020-08-07T15:10:45.202 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.203 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.203 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:45.203 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192932+0000", 2020-08-07T15:10:45.203 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017542400000000001 2020-08-07T15:10:45.204 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.204 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.204 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:45.205 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193108+0000", 2020-08-07T15:10:45.205 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004100851 2020-08-07T15:10:45.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.206 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.206 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:45.206 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197209+0000", 2020-08-07T15:10:45.206 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0100000000000001e-06 2020-08-07T15:10:45.207 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.207 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.207 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:45.207 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197210+0000", 2020-08-07T15:10:45.208 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9672999999999995e-05 2020-08-07T15:10:45.208 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.208 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.211 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:45.211 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197279+0000", 2020-08-07T15:10:45.212 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034275109999999998 2020-08-07T15:10:45.212 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.212 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.213 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:45.218 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200707+0000", 2020-08-07T15:10:45.219 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00036853100000000001 2020-08-07T15:10:45.219 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.222 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.222 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:45.223 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201075+0000", 2020-08-07T15:10:45.223 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8599999999999998e-07 2020-08-07T15:10:45.223 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.224 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.224 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:45.224 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201076+0000", 2020-08-07T15:10:45.225 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8483000000000003e-05 2020-08-07T15:10:45.225 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.225 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.227 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:45.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201124+0000", 2020-08-07T15:10:45.229 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.530301616 2020-08-07T15:10:45.229 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:45.230 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:45.230 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:45.230 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.231 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.231 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73019 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:45.232 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:10:45.232 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 79.019039704999997, 2020-08-07T15:10:45.232 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.529718224, 2020-08-07T15:10:45.233 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:45.233 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:45.233 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:45.242 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:45.242 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:45.242 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73019 2020-08-07T15:10:45.242 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.243 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:45.243 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.243 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:45.243 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:10:45.243 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:45.244 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.244 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.244 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:45.246 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:10:45.247 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3309000000000002e-05 2020-08-07T15:10:45.247 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.247 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.248 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:45.248 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671504+0000", 2020-08-07T15:10:45.249 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6619999999999998e-06 2020-08-07T15:10:45.249 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.250 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.250 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:45.250 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671507+0000", 2020-08-07T15:10:45.250 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2279999999999998e-06 2020-08-07T15:10:45.250 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.251 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.251 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:45.251 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671516+0000", 2020-08-07T15:10:45.251 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4860000000000002e-06 2020-08-07T15:10:45.251 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.252 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.252 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:45.252 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671519+0000", 2020-08-07T15:10:45.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0564000000000001e-05 2020-08-07T15:10:45.252 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.253 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671550+0000", 2020-08-07T15:10:45.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1029999999999998e-06 2020-08-07T15:10:45.261 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.261 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.261 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:45.261 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671556+0000", 2020-08-07T15:10:45.262 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75308759199999997 2020-08-07T15:10:45.262 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.262 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.262 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.262 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424643+0000", 2020-08-07T15:10:45.263 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9179999999999997e-06 2020-08-07T15:10:45.263 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.263 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.263 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:45.263 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424650+0000", 2020-08-07T15:10:45.264 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76890373599999995 2020-08-07T15:10:45.264 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.264 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.264 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.264 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193554+0000", 2020-08-07T15:10:45.264 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012345600000000001 2020-08-07T15:10:45.265 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.265 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.265 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:45.265 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193677+0000", 2020-08-07T15:10:45.265 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013528300000000001 2020-08-07T15:10:45.266 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.266 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.266 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:45.266 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193813+0000", 2020-08-07T15:10:45.266 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034777710000000002 2020-08-07T15:10:45.267 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.267 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.267 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:45.267 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197291+0000", 2020-08-07T15:10:45.267 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8000000000000004e-07 2020-08-07T15:10:45.268 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.268 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.268 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:45.268 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197291+0000", 2020-08-07T15:10:45.268 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9176999999999999e-05 2020-08-07T15:10:45.269 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.269 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.269 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:45.269 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197351+0000", 2020-08-07T15:10:45.269 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033931009999999999 2020-08-07T15:10:45.269 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.270 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.270 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:45.270 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200744+0000", 2020-08-07T15:10:45.270 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00041308000000000002 2020-08-07T15:10:45.270 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.271 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:45.271 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201157+0000", 2020-08-07T15:10:45.271 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8999999999999998e-07 2020-08-07T15:10:45.271 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:45.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201157+0000", 2020-08-07T15:10:45.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1588000000000003e-05 2020-08-07T15:10:45.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.273 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:45.273 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201189+0000", 2020-08-07T15:10:45.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.529718224 2020-08-07T15:10:45.273 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:45.274 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:45.274 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:45.274 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.274 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.274 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73020 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:45.275 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:10:45.275 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 78.950701844999998, 2020-08-07T15:10:45.275 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.461468373, 2020-08-07T15:10:45.275 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:45.275 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:45.276 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:45.276 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:45.276 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:45.276 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73020 2020-08-07T15:10:45.276 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.276 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:45.277 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.277 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:45.277 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:10:45.277 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:45.277 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.278 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.278 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:45.278 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:10:45.278 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.777e-05 2020-08-07T15:10:45.278 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.279 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.279 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:45.279 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739836+0000", 2020-08-07T15:10:45.279 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0180000000000003e-06 2020-08-07T15:10:45.279 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.280 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.280 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:45.280 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739840+0000", 2020-08-07T15:10:45.280 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6930000000000002e-06 2020-08-07T15:10:45.280 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.280 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.281 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:45.281 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739850+0000", 2020-08-07T15:10:45.281 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3619999999999999e-06 2020-08-07T15:10:45.281 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.281 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.282 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:45.282 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739854+0000", 2020-08-07T15:10:45.282 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7432999999999999e-05 2020-08-07T15:10:45.283 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.283 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.283 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.283 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739892+0000", 2020-08-07T15:10:45.283 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1149999999999996e-06 2020-08-07T15:10:45.284 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.284 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.284 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:45.284 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739899+0000", 2020-08-07T15:10:45.284 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68477883299999998 2020-08-07T15:10:45.285 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.285 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.285 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.285 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424677+0000", 2020-08-07T15:10:45.285 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7710000000000001e-06 2020-08-07T15:10:45.286 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.286 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.286 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:45.286 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424684+0000", 2020-08-07T15:10:45.287 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76949970700000003 2020-08-07T15:10:45.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.287 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.287 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194184+0000", 2020-08-07T15:10:45.288 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3957999999999997e-05 2020-08-07T15:10:45.288 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.288 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.288 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:45.288 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194278+0000", 2020-08-07T15:10:45.289 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017489599999999999 2020-08-07T15:10:45.289 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.294 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.294 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:45.295 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194453+0000", 2020-08-07T15:10:45.295 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002908908 2020-08-07T15:10:45.295 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.295 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.295 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:45.295 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197362+0000", 2020-08-07T15:10:45.296 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7299999999999995e-07 2020-08-07T15:10:45.296 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.296 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.296 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:45.307 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197362+0000", 2020-08-07T15:10:45.308 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6848000000000002e-05 2020-08-07T15:10:45.308 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.308 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:45.309 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197419+0000", 2020-08-07T15:10:45.309 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033597079999999999 2020-08-07T15:10:45.309 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.319 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.319 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:45.319 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200779+0000", 2020-08-07T15:10:45.319 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00044797899999999999 2020-08-07T15:10:45.320 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.320 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.320 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:45.320 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201227+0000", 2020-08-07T15:10:45.320 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7100000000000002e-07 2020-08-07T15:10:45.321 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.321 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.321 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:45.321 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201227+0000", 2020-08-07T15:10:45.321 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9230000000000001e-05 2020-08-07T15:10:45.322 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.322 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.322 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:45.322 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201277+0000", 2020-08-07T15:10:45.323 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.461468373 2020-08-07T15:10:45.323 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:45.323 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:45.323 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:45.323 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.324 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.324 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73021 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:45.324 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:10:45.324 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 78.887668238000003, 2020-08-07T15:10:45.324 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.398505079, 2020-08-07T15:10:45.325 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:45.325 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:45.325 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:45.325 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:45.325 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:45.326 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73021 2020-08-07T15:10:45.326 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.326 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:45.331 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.331 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:45.332 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:10:45.332 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:45.332 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.332 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.332 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:45.332 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:10:45.333 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7429000000000001e-05 2020-08-07T15:10:45.333 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.333 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.333 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:45.334 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802869+0000", 2020-08-07T15:10:45.334 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7840000000000001e-06 2020-08-07T15:10:45.334 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.334 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.334 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:45.334 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802873+0000", 2020-08-07T15:10:45.335 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0119999999999994e-06 2020-08-07T15:10:45.335 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.335 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:45.335 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802882+0000", 2020-08-07T15:10:45.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.04e-06 2020-08-07T15:10:45.336 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.336 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.336 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:45.336 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802887+0000", 2020-08-07T15:10:45.337 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7076000000000002e-05 2020-08-07T15:10:45.337 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.337 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.337 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.337 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802924+0000", 2020-08-07T15:10:45.338 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9280000000000004e-06 2020-08-07T15:10:45.338 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.338 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.338 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:45.338 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802931+0000", 2020-08-07T15:10:45.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.62178051000000001 2020-08-07T15:10:45.339 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.339 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.339 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.339 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424712+0000", 2020-08-07T15:10:45.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4749999999999996e-06 2020-08-07T15:10:45.340 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.340 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:45.340 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424719+0000", 2020-08-07T15:10:45.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77047786799999995 2020-08-07T15:10:45.341 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.341 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.341 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195197+0000", 2020-08-07T15:10:45.341 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000101909 2020-08-07T15:10:45.342 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.342 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.342 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:45.342 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195299+0000", 2020-08-07T15:10:45.342 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000248247 2020-08-07T15:10:45.343 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.343 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.343 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:45.343 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195547+0000", 2020-08-07T15:10:45.343 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033458699999999999 2020-08-07T15:10:45.343 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.344 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.344 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:45.344 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198893+0000", 2020-08-07T15:10:45.344 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0419999999999999e-06 2020-08-07T15:10:45.344 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.345 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.345 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:45.345 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198894+0000", 2020-08-07T15:10:45.345 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9354999999999997e-05 2020-08-07T15:10:45.345 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.346 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.346 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:45.346 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198953+0000", 2020-08-07T15:10:45.347 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001861096 2020-08-07T15:10:45.347 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.347 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:45.347 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200815+0000", 2020-08-07T15:10:45.348 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00049697299999999995 2020-08-07T15:10:45.348 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.348 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.348 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:45.348 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201311+0000", 2020-08-07T15:10:45.349 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7099999999999998e-07 2020-08-07T15:10:45.349 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.351 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.351 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:45.351 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201312+0000", 2020-08-07T15:10:45.352 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5194000000000002e-05 2020-08-07T15:10:45.352 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.352 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.353 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:45.353 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201347+0000", 2020-08-07T15:10:45.353 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.398505079 2020-08-07T15:10:45.353 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:45.353 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:45.354 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:45.354 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.354 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.354 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73022 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:45.354 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:10:45.366 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 78.806496182999993, 2020-08-07T15:10:45.366 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3238320669999999, 2020-08-07T15:10:45.366 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:45.366 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:45.367 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:45.367 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:45.367 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:45.367 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73022 2020-08-07T15:10:45.367 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.368 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:45.368 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.368 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:45.368 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:10:45.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:45.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.369 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:45.369 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:10:45.369 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6769999999999999e-05 2020-08-07T15:10:45.370 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.370 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.370 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:45.370 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884041+0000", 2020-08-07T15:10:45.370 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.703e-06 2020-08-07T15:10:45.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.371 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.371 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:45.371 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884044+0000", 2020-08-07T15:10:45.371 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5949999999999999e-06 2020-08-07T15:10:45.372 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.372 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:45.372 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884053+0000", 2020-08-07T15:10:45.372 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000003e-06 2020-08-07T15:10:45.372 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.373 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.373 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:45.373 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884058+0000", 2020-08-07T15:10:45.373 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7281999999999999e-05 2020-08-07T15:10:45.392 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.393 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.393 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.394 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884105+0000", 2020-08-07T15:10:45.394 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.044e-05 2020-08-07T15:10:45.394 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.395 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.395 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:45.395 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884116+0000", 2020-08-07T15:10:45.395 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.54062884300000003 2020-08-07T15:10:45.395 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.396 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.396 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.396 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424745+0000", 2020-08-07T15:10:45.396 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3149999999999998e-06 2020-08-07T15:10:45.396 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.397 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.397 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:45.397 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424754+0000", 2020-08-07T15:10:45.397 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77175732200000002 2020-08-07T15:10:45.397 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.398 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.398 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.398 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196511+0000", 2020-08-07T15:10:45.398 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010110799999999999 2020-08-07T15:10:45.398 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.399 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:45.399 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196612+0000", 2020-08-07T15:10:45.399 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017543100000000001 2020-08-07T15:10:45.399 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.400 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:45.400 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196788+0000", 2020-08-07T15:10:45.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032134569999999999 2020-08-07T15:10:45.400 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.401 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:45.401 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200001+0000", 2020-08-07T15:10:45.401 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.071e-06 2020-08-07T15:10:45.401 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.402 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.402 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:45.402 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200002+0000", 2020-08-07T15:10:45.403 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2527999999999995e-05 2020-08-07T15:10:45.403 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.403 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.403 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:45.403 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200065+0000", 2020-08-07T15:10:45.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001327433 2020-08-07T15:10:45.404 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.404 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.404 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:45.404 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201392+0000", 2020-08-07T15:10:45.405 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063854030000000004 2020-08-07T15:10:45.405 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.405 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:45.405 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207778+0000", 2020-08-07T15:10:45.416 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.285e-06 2020-08-07T15:10:45.416 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.416 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.416 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:45.416 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207779+0000", 2020-08-07T15:10:45.417 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7070999999999995e-05 2020-08-07T15:10:45.417 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.417 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.417 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:45.417 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207846+0000", 2020-08-07T15:10:45.418 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3238320669999999 2020-08-07T15:10:45.418 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:45.418 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:45.418 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:45.419 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.419 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.419 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73023 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:10:45.419 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:10:45.420 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 78.693334542000002, 2020-08-07T15:10:45.420 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2113701729999999, 2020-08-07T15:10:45.420 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:45.420 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:45.420 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:45.421 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:45.421 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:45.421 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73023 2020-08-07T15:10:45.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.421 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:45.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.422 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:45.422 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:10:45.422 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:45.422 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.423 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.423 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:45.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:10:45.423 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4105e-05 2020-08-07T15:10:45.423 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.423 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.424 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:45.424 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997210+0000", 2020-08-07T15:10:45.424 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1089999999999997e-06 2020-08-07T15:10:45.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.424 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:45.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997215+0000", 2020-08-07T15:10:45.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7040000000000006e-06 2020-08-07T15:10:45.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:45.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997224+0000", 2020-08-07T15:10:45.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4439999999999999e-06 2020-08-07T15:10:45.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.429 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:45.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997230+0000", 2020-08-07T15:10:45.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5977999999999999e-05 2020-08-07T15:10:45.430 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.430 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.430 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.430 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997276+0000", 2020-08-07T15:10:45.430 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2660000000000007e-06 2020-08-07T15:10:45.430 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.431 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:45.431 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997285+0000", 2020-08-07T15:10:45.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.42749189599999998 2020-08-07T15:10:45.431 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.432 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.432 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.432 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424777+0000", 2020-08-07T15:10:45.432 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8940000000000004e-06 2020-08-07T15:10:45.432 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.433 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.433 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:45.433 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424784+0000", 2020-08-07T15:10:45.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77268199699999995 2020-08-07T15:10:45.433 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.434 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.434 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.434 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197466+0000", 2020-08-07T15:10:45.434 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011277699999999999 2020-08-07T15:10:45.435 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.435 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.435 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:45.435 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197579+0000", 2020-08-07T15:10:45.435 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018970599999999999 2020-08-07T15:10:45.436 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.436 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.436 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:45.436 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197768+0000", 2020-08-07T15:10:45.436 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043168529999999998 2020-08-07T15:10:45.436 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.437 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.437 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:45.437 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202085+0000", 2020-08-07T15:10:45.437 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.186e-06 2020-08-07T15:10:45.437 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.438 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.438 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:45.438 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202086+0000", 2020-08-07T15:10:45.438 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8975999999999999e-05 2020-08-07T15:10:45.438 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.439 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.439 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:45.439 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202145+0000", 2020-08-07T15:10:45.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00035331499999999999 2020-08-07T15:10:45.439 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.440 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:45.440 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202499+0000", 2020-08-07T15:10:45.440 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0059703159999999998 2020-08-07T15:10:45.440 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.441 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.441 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:45.441 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208469+0000", 2020-08-07T15:10:45.441 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7400000000000002e-07 2020-08-07T15:10:45.441 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.442 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.442 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:45.442 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208470+0000", 2020-08-07T15:10:45.442 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5877000000000006e-05 2020-08-07T15:10:45.442 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.442 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.443 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:45.443 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208546+0000", 2020-08-07T15:10:45.443 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2113701729999999 2020-08-07T15:10:45.443 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:45.443 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:45.444 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:45.444 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.444 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73024 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e131)", 2020-08-07T15:10:45.444 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:26.502031+0000", 2020-08-07T15:10:45.445 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 78.188479395000002, 2020-08-07T15:10:45.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.70660062599999995, 2020-08-07T15:10:45.445 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:45.445 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:45.446 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:45.446 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:10:45.446 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:45.447 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73024 2020-08-07T15:10:45.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.447 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:45.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.447 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:45.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502031+0000", 2020-08-07T15:10:45.448 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:45.448 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:45.449 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502031+0000", 2020-08-07T15:10:45.449 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0858999999999997e-05 2020-08-07T15:10:45.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.449 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.450 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:45.450 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502062+0000", 2020-08-07T15:10:45.450 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1159999999999998e-06 2020-08-07T15:10:45.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.451 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.451 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:45.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502067+0000", 2020-08-07T15:10:45.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1810999999999999e-05 2020-08-07T15:10:45.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:45.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502078+0000", 2020-08-07T15:10:45.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2500000000000006e-06 2020-08-07T15:10:45.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.453 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:45.453 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502087+0000", 2020-08-07T15:10:45.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3461999999999996e-05 2020-08-07T15:10:45.465 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.466 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.466 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.466 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502150+0000", 2020-08-07T15:10:45.466 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3108999999999999e-05 2020-08-07T15:10:45.467 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.467 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.467 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:10:45.467 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502163+0000", 2020-08-07T15:10:45.468 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69604141200000003 2020-08-07T15:10:45.468 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.468 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.468 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:45.468 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198205+0000", 2020-08-07T15:10:45.469 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013494700000000001 2020-08-07T15:10:45.469 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.469 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.469 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:45.477 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198340+0000", 2020-08-07T15:10:45.478 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000169696 2020-08-07T15:10:45.478 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.478 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.478 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:10:45.479 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198509+0000", 2020-08-07T15:10:45.479 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0036466960000000001 2020-08-07T15:10:45.479 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.479 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.479 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:45.480 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202156+0000", 2020-08-07T15:10:45.480 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.16e-07 2020-08-07T15:10:45.480 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.480 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:45.481 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202157+0000", 2020-08-07T15:10:45.481 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5290000000000002e-05 2020-08-07T15:10:45.481 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.481 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.481 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:45.499 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202202+0000", 2020-08-07T15:10:45.499 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00061786700000000003 2020-08-07T15:10:45.499 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.500 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.500 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:45.500 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202820+0000", 2020-08-07T15:10:45.500 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0057728759999999997 2020-08-07T15:10:45.501 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.501 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.501 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:45.501 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208593+0000", 2020-08-07T15:10:45.501 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4000000000000001e-07 2020-08-07T15:10:45.507 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.508 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.508 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:10:45.526 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208594+0000", 2020-08-07T15:10:45.526 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7679000000000003e-05 2020-08-07T15:10:45.526 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:45.526 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:45.527 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:45.527 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208631+0000", 2020-08-07T15:10:45.527 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.70660062599999995 2020-08-07T15:10:45.527 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:45.527 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:45.528 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:45.528 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:45.528 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:45.528 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:45.529 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:10:45.531 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:10:45.532 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 25 16 40323 40307 12.5949 12.4062 0.00379143 0.00991141 2020-08-07T15:10:45.533 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:45.529+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:45.632 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:45.629+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:45.733 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:45.731+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:45.748 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:45.750 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:10:45.750 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:10:45.750 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:45.763 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:10:45.779 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:10:45.818 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:10:45.842 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:45.840+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:45.891 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:10:45.955 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:45.955 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:10:45.955 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:10:45.956 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:10:45.956 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:45.956 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:45.953+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:45.977 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:10:46.056 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:10:46.057 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:46.055+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:46.125 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:10:46.142 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:10:46.159 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:46.158+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:46.190 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:10:46.207 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:46.207 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:46.216 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 26 16 41876 41860 12.5767 12.1328 0.013457 0.00993462 2020-08-07T15:10:46.218 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:10:46.229 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:10:46.267 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:10:46.268 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:46.266+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:46.271 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:10:46.271 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:10:46.271 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:10:46.271 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:10:46.272 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:10:46.272 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:10:46.272 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:10:46.272 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:10:46.370 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:46.366+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:46.392 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:46.392 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:46.404 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:10:46.472 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:46.468+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:46.491 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:10:46.572 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:46.570+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:46.627 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:46.627 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:46.640 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:10:46.673 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:46.671+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:46.775 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:46.773+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:46.816 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:46.817 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:46.828 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:10:46.875 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:46.873+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:46.976 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:46.974+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:47.056 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:47.056 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:47.069 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:10:47.079 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:47.075+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:47.186 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:47.182+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:47.214 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 27 16 43508 43492 12.5831 12.75 0.0313388 0.00992678 2020-08-07T15:10:47.274 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:47.274 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:47.291 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:10:47.292 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:47.283+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:47.394 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:47.389+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:47.465 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:47.466 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:47.476 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:10:47.500 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:47.492+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:47.597 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:47.595+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:47.667 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:47.668 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:47.680 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:10:47.727 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:47.724+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:47.798 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:47.796+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:47.851 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:47.851 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:47.864 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:10:47.898 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:47.897+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:48.000 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:47.998+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:48.071 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:48.071 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:48.082 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:10:48.132 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:48.131+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:48.207 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:48.202+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:48.223 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 28 16 45059 45043 12.5664 12.1172 0.00491105 0.00993668 2020-08-07T15:10:48.332 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:10:48.335 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:48.335 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:48.336 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:10:48.340 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:48.336+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:48.440 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:48.434+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:48.508 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:10:48.523 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:48.524 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:48.535 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:10:48.566 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:10:48.578 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:48.574+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:48.610 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:10:48.646 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:48.641+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:48.647 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:10:48.682 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:10:48.718 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:10:48.747 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:48.745+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:48.767 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:48.768 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:48.768 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:10:48.775 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:10:48.802 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:10:48.850 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:10:48.851 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:48.849+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:48.852 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:10:48.852 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:10:48.852 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:10:48.852 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:10:48.852 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:10:48.853 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:10:48.853 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:10:48.853 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:10:48.954 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:48.953+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:48.980 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:48.980 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:10:48.981 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:10:48.981 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:48.991 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:10:49.057 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:49.055+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:49.162 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:49.163 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:10:49.163 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:10:49.163 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:10:49.163 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:49.164 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:49.161+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:49.177 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:10:49.214 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 29 16 46638 46622 12.5584 12.3359 0.0463239 0.00994577 2020-08-07T15:10:49.264 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:49.261+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:49.368 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:49.364+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:49.376 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:49.377 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:10:49.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:10:49.377 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:10:49.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.377 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:71136 3.6 e43/26 3:64cb7acb:::benchmark_data_smithi074_35447_object8891:head v 43'4744, mlcod=43'4739)", 2020-08-07T15:10:49.383 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:10:49.383 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 205.05105371799999, 2020-08-07T15:10:49.384 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26135482999999998, 2020-08-07T15:10:49.384 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:49.384 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:49.384 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:49.384 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.385 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:49.385 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:10:49.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:49.400 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.406 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:49.406 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318614+0000", 2020-08-07T15:10:49.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3702999999999997e-05 2020-08-07T15:10:49.407 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.408 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.408 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:49.408 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318667+0000", 2020-08-07T15:10:49.408 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3119999999999999e-06 2020-08-07T15:10:49.408 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.409 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.409 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:49.409 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.318674+0000", 2020-08-07T15:10:49.409 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.255173292 2020-08-07T15:10:49.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.416 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.416 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:49.417 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.573847+0000", 2020-08-07T15:10:49.423 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1140000000000002e-06 2020-08-07T15:10:49.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.424 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.424 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:49.424 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.573853+0000", 2020-08-07T15:10:49.424 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00057603799999999996 2020-08-07T15:10:49.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.425 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:49.425 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.574429+0000", 2020-08-07T15:10:49.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4776e-05 2020-08-07T15:10:49.426 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.426 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.436 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:49.437 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.574444+0000", 2020-08-07T15:10:49.437 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054577080000000004 2020-08-07T15:10:49.437 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.438 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.438 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:49.438 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.579901+0000", 2020-08-07T15:10:49.442 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6886999999999998e-05 2020-08-07T15:10:49.443 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.443 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.443 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:49.443 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:24.579968+0000", 2020-08-07T15:10:49.446 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26135482999999998 2020-08-07T15:10:49.446 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.446 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:49.450 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.450 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:85696 3.f e51/20 3:f99fcea4:::benchmark_data_smithi074_35447_object10711:head v 51'5688, mlcod=51'5679)", 2020-08-07T15:10:49.451 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:10:49.451 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 195.71706230500001, 2020-08-07T15:10:49.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49655717300000002, 2020-08-07T15:10:49.451 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:49.451 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:49.452 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:49.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:49.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:10:49.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:49.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.454 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:49.454 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652605+0000", 2020-08-07T15:10:49.454 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9543000000000002e-05 2020-08-07T15:10:49.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:49.456 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652665+0000", 2020-08-07T15:10:49.458 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8700000000000003e-06 2020-08-07T15:10:49.458 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.458 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:49.459 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:33.652671+0000", 2020-08-07T15:10:49.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49579595799999998 2020-08-07T15:10:49.462 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.462 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.462 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:49.463 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148467+0000", 2020-08-07T15:10:49.463 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2509999999999997e-06 2020-08-07T15:10:49.463 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.464 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.464 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:49.465 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148475+0000", 2020-08-07T15:10:49.465 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5603999999999998e-05 2020-08-07T15:10:49.465 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.468 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.469 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:49.469 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148530+0000", 2020-08-07T15:10:49.470 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5486999999999998e-05 2020-08-07T15:10:49.470 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.478 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.479 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:49.479 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.148556+0000", 2020-08-07T15:10:49.480 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00055666600000000004 2020-08-07T15:10:49.480 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.481 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:49.481 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.149112+0000", 2020-08-07T15:10:49.481 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9793999999999998e-05 2020-08-07T15:10:49.483 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.483 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.484 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:49.484 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.149162+0000", 2020-08-07T15:10:49.484 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.49655717300000002 2020-08-07T15:10:49.484 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.485 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:49.485 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.485 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.485 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.485 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:86792 3.f e52/20 3:fc5386ee:::benchmark_data_smithi074_35447_object10848:head v 52'5752, mlcod=52'5743)", 2020-08-07T15:10:49.486 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:10:49.486 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 194.96721653, 2020-08-07T15:10:49.486 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.295270005, 2020-08-07T15:10:49.486 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:49.487 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:49.487 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:49.487 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.487 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:49.487 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:10:49.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:49.488 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:49.488 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402451+0000", 2020-08-07T15:10:49.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9712000000000001e-05 2020-08-07T15:10:49.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.489 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.489 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:49.490 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402500+0000", 2020-08-07T15:10:49.490 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6789999999999999e-06 2020-08-07T15:10:49.490 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.490 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.491 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:49.491 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.402507+0000", 2020-08-07T15:10:49.491 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29317074999999998 2020-08-07T15:10:49.492 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.492 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:49.493 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695678+0000", 2020-08-07T15:10:49.494 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8120000000000002e-06 2020-08-07T15:10:49.494 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.495 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:49.495 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695685+0000", 2020-08-07T15:10:49.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1270999999999998e-05 2020-08-07T15:10:49.496 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.496 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.497 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:49.498 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695726+0000", 2020-08-07T15:10:49.498 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2739e-05 2020-08-07T15:10:49.499 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.499 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.499 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:49.500 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.695749+0000", 2020-08-07T15:10:49.500 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019333899999999999 2020-08-07T15:10:49.501 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.501 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.501 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:49.502 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.697682+0000", 2020-08-07T15:10:49.503 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8652000000000002e-05 2020-08-07T15:10:49.503 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.503 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.504 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:49.504 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.697721+0000", 2020-08-07T15:10:49.505 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.295270005 2020-08-07T15:10:49.505 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.505 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:49.506 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.506 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.506 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.507 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:87088 3.6 e52/26 3:6b0320df:::benchmark_data_smithi074_35447_object10885:head v 52'5912, mlcod=52'5902)", 2020-08-07T15:10:49.507 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:10:49.507 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 194.40298329000001, 2020-08-07T15:10:49.508 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29522189799999998, 2020-08-07T15:10:49.508 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:49.508 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:49.509 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:49.509 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.510 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:49.510 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:10:49.510 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:49.511 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:49.512 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966684+0000", 2020-08-07T15:10:49.512 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0751000000000002e-05 2020-08-07T15:10:49.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.513 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.513 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:49.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966735+0000", 2020-08-07T15:10:49.514 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3080000000000004e-06 2020-08-07T15:10:49.514 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.515 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.515 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:49.516 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:34.966741+0000", 2020-08-07T15:10:49.516 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29365034699999998 2020-08-07T15:10:49.516 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.516 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.517 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:49.517 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260391+0000", 2020-08-07T15:10:49.518 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2720000000000003e-06 2020-08-07T15:10:49.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.519 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.519 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:49.520 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260398+0000", 2020-08-07T15:10:49.520 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8819000000000005e-05 2020-08-07T15:10:49.520 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.521 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.521 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:49.521 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260476+0000", 2020-08-07T15:10:49.522 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4958999999999999e-05 2020-08-07T15:10:49.522 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.522 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.523 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:49.523 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.260501+0000", 2020-08-07T15:10:49.523 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001353641 2020-08-07T15:10:49.524 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.527 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.527 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:49.527 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.261855+0000", 2020-08-07T15:10:49.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0800999999999997e-05 2020-08-07T15:10:49.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.529 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:49.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:35.261906+0000", 2020-08-07T15:10:49.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29522189799999998 2020-08-07T15:10:49.530 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.530 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:49.530 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.531 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.531 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.531 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:93920 3.6 e56/26 3:631393d3:::benchmark_data_smithi074_35447_object11739:head v 56'6352, mlcod=55'6342)", 2020-08-07T15:10:49.532 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:10:49.532 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 190.04286785900001, 2020-08-07T15:10:49.532 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39765404300000001, 2020-08-07T15:10:49.533 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:49.534 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:49.535 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:49.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:49.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:10:49.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:49.537 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.537 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.538 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:49.538 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326799+0000", 2020-08-07T15:10:49.538 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3035999999999998e-05 2020-08-07T15:10:49.539 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.540 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.540 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:49.540 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326852+0000", 2020-08-07T15:10:49.541 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5379999999999999e-06 2020-08-07T15:10:49.541 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.542 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:49.542 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.326859+0000", 2020-08-07T15:10:49.542 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.393917568 2020-08-07T15:10:49.542 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.543 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.543 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:49.543 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.720777+0000", 2020-08-07T15:10:49.544 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2430000000000005e-06 2020-08-07T15:10:49.544 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.544 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:49.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.720785+0000", 2020-08-07T15:10:49.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00065658700000000003 2020-08-07T15:10:49.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.546 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.547 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:49.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.721441+0000", 2020-08-07T15:10:49.547 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8585000000000002e-05 2020-08-07T15:10:49.548 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.548 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:49.550 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.721470+0000", 2020-08-07T15:10:49.550 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00293349 2020-08-07T15:10:49.550 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.551 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.551 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:49.551 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.724403+0000", 2020-08-07T15:10:49.552 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9996e-05 2020-08-07T15:10:49.552 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.553 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:49.553 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:39.724453+0000", 2020-08-07T15:10:49.554 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.39765404300000001 2020-08-07T15:10:49.554 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.554 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:49.555 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.555 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.556 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:116064 3.6 e64/26 3:6cc03ffa:::benchmark_data_smithi074_35447_object14507:head v 64'7632, mlcod=64'7623)", 2020-08-07T15:10:49.556 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:10:49.557 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 173.006906456, 2020-08-07T15:10:49.557 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25728607799999997, 2020-08-07T15:10:49.557 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:49.558 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:49.558 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:49.558 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.559 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:49.559 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:10:49.559 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:49.560 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.560 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.560 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:49.561 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362761+0000", 2020-08-07T15:10:49.561 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6042000000000003e-05 2020-08-07T15:10:49.561 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.562 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.562 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:49.562 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362817+0000", 2020-08-07T15:10:49.563 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9399999999999999e-06 2020-08-07T15:10:49.563 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.564 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:49.564 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.362823+0000", 2020-08-07T15:10:49.564 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.254635583 2020-08-07T15:10:49.565 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.565 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.565 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:49.566 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617458+0000", 2020-08-07T15:10:49.566 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5100000000000004e-06 2020-08-07T15:10:49.566 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.567 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.567 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:49.567 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617465+0000", 2020-08-07T15:10:49.568 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4439999999999999e-05 2020-08-07T15:10:49.568 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.568 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.568 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:49.568 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617539+0000", 2020-08-07T15:10:49.569 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3527e-05 2020-08-07T15:10:49.569 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.569 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.570 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:49.570 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.617563+0000", 2020-08-07T15:10:49.570 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024301069999999999 2020-08-07T15:10:49.571 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.571 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.572 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:49.572 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.619993+0000", 2020-08-07T15:10:49.573 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3928999999999998e-05 2020-08-07T15:10:49.573 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.574 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.574 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:49.574 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:56.620047+0000", 2020-08-07T15:10:49.584 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25728607799999997 2020-08-07T15:10:49.584 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.585 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:49.585 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.585 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.585 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.585 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:145801 3.6 e75/26 3:69e4cb22:::benchmark_data_smithi074_35447_object16192:head v 75'9405, mlcod=75'9401)", 2020-08-07T15:10:49.586 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:10:49.586 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 154.697911164, 2020-08-07T15:10:49.586 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639, 2020-08-07T15:10:49.586 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:49.586 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:49.587 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:49.587 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.587 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:49.587 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:10:49.587 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:49.587 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.588 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:49.588 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:10:49.588 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6511000000000002e-05 2020-08-07T15:10:49.589 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.589 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:49.589 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671803+0000", 2020-08-07T15:10:49.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7190000000000004e-06 2020-08-07T15:10:49.590 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.590 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.590 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:49.590 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671809+0000", 2020-08-07T15:10:49.590 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74210928300000001 2020-08-07T15:10:49.591 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.591 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.591 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:49.591 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413919+0000", 2020-08-07T15:10:49.591 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1759999999999999e-05 2020-08-07T15:10:49.591 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.592 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.592 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:49.592 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413930+0000", 2020-08-07T15:10:49.592 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000108601 2020-08-07T15:10:49.592 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.593 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.593 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:49.593 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414039+0000", 2020-08-07T15:10:49.593 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9948999999999999e-05 2020-08-07T15:10:49.593 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.594 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.594 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:49.594 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414089+0000", 2020-08-07T15:10:49.594 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00079656499999999995 2020-08-07T15:10:49.594 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.595 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.595 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:49.595 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414885+0000", 2020-08-07T15:10:49.595 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2250999999999998e-05 2020-08-07T15:10:49.595 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.596 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.596 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:49.596 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414938+0000", 2020-08-07T15:10:49.598 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639 2020-08-07T15:10:49.598 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.598 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:49.598 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.598 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.599 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.599 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:9112 4.6 e86/81 4:6004be0b:::benchmark_data_smithi074_44586_object1138:head v 86'544, mlcod=86'533)", 2020-08-07T15:10:49.599 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:32.296890+0000", 2020-08-07T15:10:49.599 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 137.072777755, 2020-08-07T15:10:49.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26263388100000001, 2020-08-07T15:10:49.600 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:49.600 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:49.600 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:49.600 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.601 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:49.601 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296890+0000", 2020-08-07T15:10:49.601 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:49.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.602 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:49.602 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296890+0000", 2020-08-07T15:10:49.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9907999999999997e-05 2020-08-07T15:10:49.602 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.602 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.603 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:49.603 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296939+0000", 2020-08-07T15:10:49.603 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7630000000000003e-06 2020-08-07T15:10:49.603 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:49.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.296946+0000", 2020-08-07T15:10:49.607 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26058471900000002 2020-08-07T15:10:49.607 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.607 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.608 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:49.608 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557531+0000", 2020-08-07T15:10:49.608 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5270000000000004e-06 2020-08-07T15:10:49.609 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.609 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.609 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:49.609 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557537+0000", 2020-08-07T15:10:49.609 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2843000000000003e-05 2020-08-07T15:10:49.610 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.610 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.610 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:49.610 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557580+0000", 2020-08-07T15:10:49.610 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3833999999999999e-05 2020-08-07T15:10:49.611 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.611 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.611 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:49.611 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.557604+0000", 2020-08-07T15:10:49.611 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001875196 2020-08-07T15:10:49.612 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.612 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.612 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:49.612 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.559479+0000", 2020-08-07T15:10:49.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4091000000000002e-05 2020-08-07T15:10:49.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.613 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:49.613 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:32.559523+0000", 2020-08-07T15:10:49.613 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26263388100000001 2020-08-07T15:10:49.614 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.614 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:49.614 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.614 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.614 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.614 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:13784 4.6 e88/81 4:66f64b92:::benchmark_data_smithi074_44586_object1722:head v 88'832, mlcod=88'827)", 2020-08-07T15:10:49.615 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:35.137315+0000", 2020-08-07T15:10:49.615 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 134.23235202199999, 2020-08-07T15:10:49.615 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32459016899999998, 2020-08-07T15:10:49.615 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:49.615 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:49.616 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:49.616 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.616 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:49.616 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137315+0000", 2020-08-07T15:10:49.616 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:49.617 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.617 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.617 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:49.617 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137315+0000", 2020-08-07T15:10:49.617 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0362999999999998e-05 2020-08-07T15:10:49.618 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.618 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.618 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:49.618 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137376+0000", 2020-08-07T15:10:49.618 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7020000000000004e-06 2020-08-07T15:10:49.619 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.619 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.619 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:49.619 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.137382+0000", 2020-08-07T15:10:49.620 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32172605500000001 2020-08-07T15:10:49.620 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.620 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.620 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:49.620 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459108+0000", 2020-08-07T15:10:49.621 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7710000000000003e-06 2020-08-07T15:10:49.621 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.621 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.621 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:49.621 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459114+0000", 2020-08-07T15:10:49.622 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4297e-05 2020-08-07T15:10:49.622 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.622 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.622 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:49.622 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459158+0000", 2020-08-07T15:10:49.623 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5058999999999998e-05 2020-08-07T15:10:49.623 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.623 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.623 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:49.623 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.459183+0000", 2020-08-07T15:10:49.624 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026624740000000002 2020-08-07T15:10:49.624 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.624 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.624 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:49.625 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.461846+0000", 2020-08-07T15:10:49.625 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9447999999999997e-05 2020-08-07T15:10:49.626 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.626 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.626 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:49.628 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:35.461905+0000", 2020-08-07T15:10:49.628 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32459016899999998 2020-08-07T15:10:49.628 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.628 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:49.628 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.629 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.629 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.629 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:31608 4.7 e100/81 4:e0ebdc25:::benchmark_data_smithi074_44586_object3950:head v 100'2136, mlcod=100'2126)", 2020-08-07T15:10:49.630 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:47.845848+0000", 2020-08-07T15:10:49.630 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 121.523819162, 2020-08-07T15:10:49.630 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27603951399999999, 2020-08-07T15:10:49.630 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:49.630 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:49.631 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:49.631 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.631 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:49.631 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:47.845848+0000", 2020-08-07T15:10:49.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:49.632 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.632 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.632 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:49.632 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:47.845848+0000", 2020-08-07T15:10:49.633 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3656999999999998e-05 2020-08-07T15:10:49.633 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.633 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.633 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:49.633 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:47.845902+0000", 2020-08-07T15:10:49.634 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4050000000000003e-06 2020-08-07T15:10:49.634 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.634 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.634 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:49.634 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:47.845909+0000", 2020-08-07T15:10:49.635 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27431144000000002 2020-08-07T15:10:49.635 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.635 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.635 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:49.635 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.120221+0000", 2020-08-07T15:10:49.636 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3640000000000004e-06 2020-08-07T15:10:49.636 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.636 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.636 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:49.636 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.120227+0000", 2020-08-07T15:10:49.636 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1541e-05 2020-08-07T15:10:49.637 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.637 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.637 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:49.637 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.120279+0000", 2020-08-07T15:10:49.637 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.372e-05 2020-08-07T15:10:49.638 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.639 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.639 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:49.639 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.120302+0000", 2020-08-07T15:10:49.639 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015294460000000001 2020-08-07T15:10:49.640 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.640 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.640 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:49.641 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.121832+0000", 2020-08-07T15:10:49.641 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5940999999999999e-05 2020-08-07T15:10:49.641 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.641 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.641 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:49.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.121888+0000", 2020-08-07T15:10:49.642 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27603951399999999 2020-08-07T15:10:49.642 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.642 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:49.642 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.645 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:31832 4.b e100/81 4:d005fdc5:::benchmark_data_smithi074_44586_object3978:head v 100'1872, mlcod=100'1863)", 2020-08-07T15:10:49.646 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:48.177094+0000", 2020-08-07T15:10:49.646 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 121.19257289799999, 2020-08-07T15:10:49.646 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26337036000000003, 2020-08-07T15:10:49.647 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:49.647 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:49.648 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:49.649 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.651 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:49.651 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.177094+0000", 2020-08-07T15:10:49.652 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:49.652 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.652 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.652 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:49.652 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.177094+0000", 2020-08-07T15:10:49.653 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3760999999999997e-05 2020-08-07T15:10:49.653 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.654 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.654 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:49.655 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.177128+0000", 2020-08-07T15:10:49.655 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1549999999999998e-06 2020-08-07T15:10:49.656 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.656 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.656 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:49.656 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.177133+0000", 2020-08-07T15:10:49.656 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.262460639 2020-08-07T15:10:49.657 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.657 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.657 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:49.657 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.439594+0000", 2020-08-07T15:10:49.657 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0943e-05 2020-08-07T15:10:49.658 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.658 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.658 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:49.658 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.439605+0000", 2020-08-07T15:10:49.658 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013696100000000001 2020-08-07T15:10:49.659 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.659 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.659 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:49.659 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.439742+0000", 2020-08-07T15:10:49.660 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5649000000000003e-05 2020-08-07T15:10:49.660 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.660 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.660 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:49.661 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.439787+0000", 2020-08-07T15:10:49.661 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00063599200000000005 2020-08-07T15:10:49.661 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.661 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.661 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:49.661 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.440423+0000", 2020-08-07T15:10:49.662 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1260000000000001e-05 2020-08-07T15:10:49.662 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.662 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.662 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:49.662 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:48.440465+0000", 2020-08-07T15:10:49.663 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26337036000000003 2020-08-07T15:10:49.663 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.663 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:49.663 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.663 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.664 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:42280 4.6 e112/81 4:6fc8ffaf:::benchmark_data_smithi074_44586_object5284:head v 112'2464, mlcod=112'2455)", 2020-08-07T15:10:49.664 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.113698+0000", 2020-08-07T15:10:49.664 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 109.255969505, 2020-08-07T15:10:49.664 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43055294, 2020-08-07T15:10:49.665 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:49.665 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:49.665 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:49.665 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.665 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:49.666 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.113698+0000", 2020-08-07T15:10:49.666 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:49.666 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.666 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.666 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:49.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.113698+0000", 2020-08-07T15:10:49.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2302000000000003e-05 2020-08-07T15:10:49.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.667 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:49.668 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.113740+0000", 2020-08-07T15:10:49.668 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3669999999999999e-06 2020-08-07T15:10:49.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:49.668 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.113744+0000", 2020-08-07T15:10:49.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.42958231000000002 2020-08-07T15:10:49.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:49.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.543327+0000", 2020-08-07T15:10:49.670 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2373e-05 2020-08-07T15:10:49.670 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.670 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.670 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:49.670 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.543339+0000", 2020-08-07T15:10:49.671 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014704099999999999 2020-08-07T15:10:49.678 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.678 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.679 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:49.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.543486+0000", 2020-08-07T15:10:49.679 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4834999999999999e-05 2020-08-07T15:10:49.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.680 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:49.680 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.543511+0000", 2020-08-07T15:10:49.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00069499500000000001 2020-08-07T15:10:49.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.698 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.699 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:49.699 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.544206+0000", 2020-08-07T15:10:49.699 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4716999999999999e-05 2020-08-07T15:10:49.699 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.700 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:49.700 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.544251+0000", 2020-08-07T15:10:49.700 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.43055294 2020-08-07T15:10:49.700 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.701 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:49.701 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.701 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.701 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:72952 4.b e130/81 4:dee63f23:::benchmark_data_smithi074_44586_object9118:head v 130'4592, mlcod=130'4584)", 2020-08-07T15:10:49.702 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.589220+0000", 2020-08-07T15:10:49.702 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 83.780447108999994, 2020-08-07T15:10:49.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40730996899999999, 2020-08-07T15:10:49.702 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:49.702 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:49.703 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:49.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.703 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:49.703 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.589220+0000", 2020-08-07T15:10:49.704 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:49.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.704 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:49.704 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.589220+0000", 2020-08-07T15:10:49.705 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9273000000000001e-05 2020-08-07T15:10:49.705 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.705 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.705 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:49.705 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.589269+0000", 2020-08-07T15:10:49.706 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.417e-06 2020-08-07T15:10:49.706 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.706 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.706 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:49.706 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.589275+0000", 2020-08-07T15:10:49.707 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40641516700000002 2020-08-07T15:10:49.707 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.707 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:49.707 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.995690+0000", 2020-08-07T15:10:49.707 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7780000000000002e-06 2020-08-07T15:10:49.708 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.708 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:49.708 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.995697+0000", 2020-08-07T15:10:49.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3010999999999999e-05 2020-08-07T15:10:49.709 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.709 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.709 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:49.709 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.995740+0000", 2020-08-07T15:10:49.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1878999999999998e-05 2020-08-07T15:10:49.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.723 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:49.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.995762+0000", 2020-08-07T15:10:49.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00069480500000000005 2020-08-07T15:10:49.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.724 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:49.727 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.996456+0000", 2020-08-07T15:10:49.728 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3639000000000004e-05 2020-08-07T15:10:49.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:49.729 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.996530+0000", 2020-08-07T15:10:49.729 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40730996899999999 2020-08-07T15:10:49.729 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.729 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:49.729 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.730 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.730 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.730 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:85007 4.6 e139/81 4:6aadd387:::benchmark_data_smithi074_44586_object10625:head v 139'4991, mlcod=139'4981)", 2020-08-07T15:10:49.730 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:36.051294+0000", 2020-08-07T15:10:49.730 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 73.318373542000003, 2020-08-07T15:10:49.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28119156699999998, 2020-08-07T15:10:49.731 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:49.731 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:49.731 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:49.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:49.732 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.051294+0000", 2020-08-07T15:10:49.732 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:49.732 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.732 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.733 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:49.733 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.051294+0000", 2020-08-07T15:10:49.733 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8449000000000003e-05 2020-08-07T15:10:49.733 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.747 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.750 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:49.750 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.051342+0000", 2020-08-07T15:10:49.751 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9329999999999998e-06 2020-08-07T15:10:49.752 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.753 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:49.753 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.051348+0000", 2020-08-07T15:10:49.753 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28014436300000001 2020-08-07T15:10:49.753 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:49.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.331492+0000", 2020-08-07T15:10:49.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0309999999999998e-06 2020-08-07T15:10:49.764 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.765 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:49.765 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.331500+0000", 2020-08-07T15:10:49.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6211999999999997e-05 2020-08-07T15:10:49.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.766 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:49.766 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.331547+0000", 2020-08-07T15:10:49.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6117000000000001e-05 2020-08-07T15:10:49.766 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:49.768 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.331573+0000", 2020-08-07T15:10:49.769 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00085618800000000002 2020-08-07T15:10:49.770 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.770 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.770 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:49.771 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.332429+0000", 2020-08-07T15:10:49.771 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6273999999999997e-05 2020-08-07T15:10:49.771 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.771 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.771 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:49.772 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.332485+0000", 2020-08-07T15:10:49.772 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28119156699999998 2020-08-07T15:10:49.772 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.783 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:49.795 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.796 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.796 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:85008 4.6 e139/81 4:6aadd387:::benchmark_data_smithi074_44586_object10625:head v 139'4992, mlcod=139'4981)", 2020-08-07T15:10:49.796 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:36.051972+0000", 2020-08-07T15:10:49.796 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 73.317694865999997, 2020-08-07T15:10:49.797 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28367819599999999, 2020-08-07T15:10:49.797 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:49.797 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:49.797 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:49.797 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.798 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:49.798 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.051972+0000", 2020-08-07T15:10:49.798 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:49.798 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.802 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:49.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.051972+0000", 2020-08-07T15:10:49.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1882999999999998e-05 2020-08-07T15:10:49.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:49.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.052024+0000", 2020-08-07T15:10:49.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3609999999999997e-06 2020-08-07T15:10:49.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:49.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.052031+0000", 2020-08-07T15:10:49.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28155142999999999 2020-08-07T15:10:49.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.813 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:49.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.333582+0000", 2020-08-07T15:10:49.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8130000000000001e-06 2020-08-07T15:10:49.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.815 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:49.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.333588+0000", 2020-08-07T15:10:49.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4005000000000003e-05 2020-08-07T15:10:49.816 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:49.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.333632+0000", 2020-08-07T15:10:49.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2682999999999999e-05 2020-08-07T15:10:49.819 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:49.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.333655+0000", 2020-08-07T15:10:49.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019567569999999999 2020-08-07T15:10:49.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:49.821 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.335611+0000", 2020-08-07T15:10:49.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9264000000000002e-05 2020-08-07T15:10:49.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.822 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.822 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:49.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:36.335651+0000", 2020-08-07T15:10:49.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28367819599999999 2020-08-07T15:10:49.823 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.823 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:49.824 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.824 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:88160 4.6 e141/81 4:6eb8ba54:::benchmark_data_smithi074_44586_object11019:head v 141'5144, mlcod=141'5136)", 2020-08-07T15:10:49.825 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:37.976442+0000", 2020-08-07T15:10:49.825 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 71.393224899000003, 2020-08-07T15:10:49.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45938089599999998, 2020-08-07T15:10:49.825 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:49.879 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:49.879 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:49.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.880 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:49.880 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:37.976442+0000", 2020-08-07T15:10:49.880 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:49.880 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.881 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:49.881 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:37.976442+0000", 2020-08-07T15:10:49.881 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2018999999999998e-05 2020-08-07T15:10:49.881 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.881 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:49.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:37.976494+0000", 2020-08-07T15:10:49.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2230000000000004e-06 2020-08-07T15:10:49.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.922 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:49.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:37.976501+0000", 2020-08-07T15:10:49.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45734871399999999 2020-08-07T15:10:49.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:49.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:38.433849+0000", 2020-08-07T15:10:49.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7040000000000002e-06 2020-08-07T15:10:49.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:49.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:38.433854+0000", 2020-08-07T15:10:49.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.371e-05 2020-08-07T15:10:49.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:49.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:38.433898+0000", 2020-08-07T15:10:49.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6178e-05 2020-08-07T15:10:49.925 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.926 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:49.926 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:38.433914+0000", 2020-08-07T15:10:49.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0018471399999999999 2020-08-07T15:10:49.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:49.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:38.435761+0000", 2020-08-07T15:10:49.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2207999999999998e-05 2020-08-07T15:10:49.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:49.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:38.435823+0000", 2020-08-07T15:10:49.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.45938089599999998 2020-08-07T15:10:49.928 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.929 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:49.929 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:49.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.929 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:104104 4.0 e146/131 4:0ffb764a:::benchmark_data_smithi074_44586_object13012:head v 146'6960, mlcod=146'6953)", 2020-08-07T15:10:49.930 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:10:49.930 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 61.171364388000001, 2020-08-07T15:10:49.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75443331599999996, 2020-08-07T15:10:49.930 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:49.930 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:49.931 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:49.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:49.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:10:49.931 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:49.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.970 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:49.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:10:49.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1743000000000002e-05 2020-08-07T15:10:49.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:49.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198355+0000", 2020-08-07T15:10:49.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1249999999999998e-06 2020-08-07T15:10:49.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:49.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198361+0000", 2020-08-07T15:10:49.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74719179599999996 2020-08-07T15:10:49.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:49.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:49.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:50.001 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945553+0000", 2020-08-07T15:10:50.001 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6800000000000001e-06 2020-08-07T15:10:50.001 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.002 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.002 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:50.002 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945557+0000", 2020-08-07T15:10:50.002 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012101500000000001 2020-08-07T15:10:50.002 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:50.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945678+0000", 2020-08-07T15:10:50.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6621000000000001e-05 2020-08-07T15:10:50.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:50.004 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945705+0000", 2020-08-07T15:10:50.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069879540000000002 2020-08-07T15:10:50.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:50.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.952693+0000", 2020-08-07T15:10:50.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3381999999999999e-05 2020-08-07T15:10:50.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.030 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:50.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.952736+0000", 2020-08-07T15:10:50.030 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75443331599999996 2020-08-07T15:10:50.031 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.031 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:50.031 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.032 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:128781 4.3 e155/81 4:c7b7232c:::benchmark_data_smithi074_44586_object12545:head v 155'7824, mlcod=154'7821)", 2020-08-07T15:10:50.032 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:03.440866+0000", 2020-08-07T15:10:50.032 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 45.928801481999997, 2020-08-07T15:10:50.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26739538699999998, 2020-08-07T15:10:50.032 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:50.033 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:50.033 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:50.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:50.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.440866+0000", 2020-08-07T15:10:50.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:50.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.042 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:50.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.440866+0000", 2020-08-07T15:10:50.042 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5170000000000001e-05 2020-08-07T15:10:50.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:50.043 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.440891+0000", 2020-08-07T15:10:50.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3590000000000001e-06 2020-08-07T15:10:50.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.044 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:50.044 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.440895+0000", 2020-08-07T15:10:50.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26665756499999999 2020-08-07T15:10:50.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.045 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:50.045 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.707553+0000", 2020-08-07T15:10:50.045 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9789999999999996e-06 2020-08-07T15:10:50.045 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.046 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.046 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:50.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.707560+0000", 2020-08-07T15:10:50.047 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9085999999999998e-05 2020-08-07T15:10:50.047 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.047 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:50.047 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.707599+0000", 2020-08-07T15:10:50.048 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8845999999999999e-05 2020-08-07T15:10:50.048 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.048 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.048 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:50.048 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.707618+0000", 2020-08-07T15:10:50.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000590597 2020-08-07T15:10:50.049 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.049 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.049 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:50.049 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.708208+0000", 2020-08-07T15:10:50.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2784999999999997e-05 2020-08-07T15:10:50.050 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.050 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.050 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:50.050 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.708261+0000", 2020-08-07T15:10:50.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26739538699999998 2020-08-07T15:10:50.051 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.051 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:50.051 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.051 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.052 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:128871 4.3 e155/81 4:cc9116ea:::benchmark_data_smithi074_44586_object12635:head v 155'7827, mlcod=155'7824)", 2020-08-07T15:10:50.052 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:03.727430+0000", 2020-08-07T15:10:50.052 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 45.642237448000003, 2020-08-07T15:10:50.052 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29142945999999997, 2020-08-07T15:10:50.052 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:50.053 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:50.053 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:50.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.053 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:50.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.727430+0000", 2020-08-07T15:10:50.054 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:50.054 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.054 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.054 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:50.054 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.727430+0000", 2020-08-07T15:10:50.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4294e-05 2020-08-07T15:10:50.055 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.055 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:50.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.727474+0000", 2020-08-07T15:10:50.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6970000000000001e-06 2020-08-07T15:10:50.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:50.062 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:03.727479+0000", 2020-08-07T15:10:50.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.290211159 2020-08-07T15:10:50.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.063 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:50.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:04.017690+0000", 2020-08-07T15:10:50.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5390000000000007e-06 2020-08-07T15:10:50.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.064 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:50.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:04.017699+0000", 2020-08-07T15:10:50.066 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0505000000000003e-05 2020-08-07T15:10:50.066 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.066 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.067 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:50.067 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:04.017779+0000", 2020-08-07T15:10:50.067 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0101000000000001e-05 2020-08-07T15:10:50.067 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.067 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.068 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:50.068 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:04.017819+0000", 2020-08-07T15:10:50.068 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00095239000000000001 2020-08-07T15:10:50.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.068 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.068 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:50.069 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:04.018772+0000", 2020-08-07T15:10:50.069 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7775000000000006e-05 2020-08-07T15:10:50.069 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.069 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.070 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:50.070 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:04.018859+0000", 2020-08-07T15:10:50.070 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29142945999999997 2020-08-07T15:10:50.070 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.071 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:50.071 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.071 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.071 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.071 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5099.0:27760 5.c e178/164 5:3f3d4696:::benchmark_data_smithi074_53697_object3469:head v 178'2016, mlcod=177'2006)", 2020-08-07T15:10:50.072 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:37.453863+0000", 2020-08-07T15:10:50.072 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.915804483000001, 2020-08-07T15:10:50.072 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30220765999999999, 2020-08-07T15:10:50.072 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:50.072 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:50.073 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:50.081 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.081 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:50.081 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:37.453863+0000", 2020-08-07T15:10:50.082 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:50.082 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.082 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:50.082 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:37.453863+0000", 2020-08-07T15:10:50.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2908000000000002e-05 2020-08-07T15:10:50.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.089 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.089 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:50.090 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:37.453916+0000", 2020-08-07T15:10:50.093 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2330000000000003e-06 2020-08-07T15:10:50.094 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.094 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.094 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:50.094 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:37.453922+0000", 2020-08-07T15:10:50.095 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29103976300000001 2020-08-07T15:10:50.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.095 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.095 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:50.095 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:37.744962+0000", 2020-08-07T15:10:50.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0860000000000001e-06 2020-08-07T15:10:50.096 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.096 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.096 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:50.096 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:37.744967+0000", 2020-08-07T15:10:50.097 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8820999999999998e-05 2020-08-07T15:10:50.097 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.097 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.097 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:50.097 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:37.745026+0000", 2020-08-07T15:10:50.109 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4017999999999999e-05 2020-08-07T15:10:50.109 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.110 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.110 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:50.110 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:37.745050+0000", 2020-08-07T15:10:50.110 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.010975202 2020-08-07T15:10:50.110 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.111 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.111 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:50.111 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:37.756025+0000", 2020-08-07T15:10:50.111 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5629e-05 2020-08-07T15:10:50.111 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.112 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:10:50.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:37.756070+0000", 2020-08-07T15:10:50.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30220765999999999 2020-08-07T15:10:50.112 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.112 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:50.113 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.113 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.113 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:50.113 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:50.118 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:10:50.119 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:10:50.119 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:10:50.119 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2020-08-07T15:10:50.120 INFO:teuthology.orchestra.run:waiting for 300 2020-08-07T15:10:50.121 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:10:50.127 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:10:50.128 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:10:50.117+0000 7f23d768f700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31569) UID: 0 2020-08-07T15:10:50.172 INFO:tasks.ceph.osd.1:Stopped 2020-08-07T15:10:50.172 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.1 2020-08-07T15:10:50.173 INFO:tasks.thrashosds.thrasher:remote for osd 1 is ubuntu@smithi074.front.sepia.ceph.com 2020-08-07T15:10:50.173 INFO:teuthology.orchestra.run.smithi074:> type ceph-objectstore-tool 2020-08-07T15:10:50.213 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 30 16 48115 48099 12.5244 11.5391 0.0030294 0.00995309 2020-08-07T15:10:50.233 INFO:teuthology.orchestra.run.smithi074.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2020-08-07T15:10:50.234 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-objectstore-tool --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --op list-pgs 2020-08-07T15:10:50.235 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:50.232+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:50.310 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:50.311 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:10:50.311 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.311 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:48084 5.8 5:12f3d078:::benchmark_data_smithi074_53697_object6010:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e190)", 2020-08-07T15:10:50.311 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:50.170933+0000", 2020-08-07T15:10:50.312 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.12845432000000001, 2020-08-07T15:10:50.312 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.128489512, 2020-08-07T15:10:50.312 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:50.312 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:50.312 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:50.330 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:50.330 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:50.331 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48084 2020-08-07T15:10:50.331 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.331 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:50.331 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.331 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:50.332 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.170933+0000", 2020-08-07T15:10:50.332 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:50.332 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.332 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.333 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:50.333 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.170933+0000", 2020-08-07T15:10:50.333 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9635000000000002e-05 2020-08-07T15:10:50.333 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.356 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.356 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:50.356 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.170982+0000", 2020-08-07T15:10:50.357 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5390000000000004e-06 2020-08-07T15:10:50.357 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.357 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.357 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:50.357 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.170989+0000", 2020-08-07T15:10:50.358 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3498e-05 2020-08-07T15:10:50.358 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.358 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.358 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:50.358 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.171002+0000", 2020-08-07T15:10:50.359 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0380000000000002e-06 2020-08-07T15:10:50.370 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.370 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.370 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:50.370 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.171009+0000", 2020-08-07T15:10:50.371 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0348000000000001e-05 2020-08-07T15:10:50.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.371 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.371 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:50.371 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.171040+0000", 2020-08-07T15:10:50.372 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8814999999999998e-05 2020-08-07T15:10:50.372 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.372 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:50.372 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.171109+0000", 2020-08-07T15:10:50.373 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000119684 2020-08-07T15:10:50.373 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.373 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.373 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:10:50.373 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.171228+0000", 2020-08-07T15:10:50.375 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045871790000000003 2020-08-07T15:10:50.375 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.375 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:50.375 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.175815+0000", 2020-08-07T15:10:50.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.145e-06 2020-08-07T15:10:50.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.376 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:50.376 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.175817+0000", 2020-08-07T15:10:50.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8727999999999997e-05 2020-08-07T15:10:50.377 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:50.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.175885+0000", 2020-08-07T15:10:50.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000948905 2020-08-07T15:10:50.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:50.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.176834+0000", 2020-08-07T15:10:50.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0059015140000000001 2020-08-07T15:10:50.379 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.379 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:50.379 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.386 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.386 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.386 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:48085 5.8 5:12f3d078:::benchmark_data_smithi074_53697_object6010:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e190)", 2020-08-07T15:10:50.387 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:50.173512+0000", 2020-08-07T15:10:50.387 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.12587487999999999, 2020-08-07T15:10:50.387 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.12601722800000001, 2020-08-07T15:10:50.387 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:50.387 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:50.388 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:50.388 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:50.388 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:50.388 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48085 2020-08-07T15:10:50.388 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.389 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:50.389 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.389 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:50.389 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.173512+0000", 2020-08-07T15:10:50.389 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:50.402 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.402 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.402 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:50.403 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.173512+0000", 2020-08-07T15:10:50.403 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8087000000000001e-05 2020-08-07T15:10:50.403 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.403 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.403 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:50.404 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.173540+0000", 2020-08-07T15:10:50.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0709999999999996e-06 2020-08-07T15:10:50.404 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.404 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.404 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:50.405 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.173544+0000", 2020-08-07T15:10:50.405 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0040000000000005e-06 2020-08-07T15:10:50.405 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.405 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:50.426 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.173553+0000", 2020-08-07T15:10:50.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.109e-06 2020-08-07T15:10:50.427 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.427 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:50.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.173557+0000", 2020-08-07T15:10:50.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.341e-05 2020-08-07T15:10:50.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:50.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.173591+0000", 2020-08-07T15:10:50.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2204000000000002e-05 2020-08-07T15:10:50.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.429 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:50.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.173663+0000", 2020-08-07T15:10:50.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011445700000000001 2020-08-07T15:10:50.430 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.430 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.430 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:10:50.430 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.173778+0000", 2020-08-07T15:10:50.430 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024424490000000002 2020-08-07T15:10:50.431 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.431 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:50.431 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.176220+0000", 2020-08-07T15:10:50.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1009999999999999e-06 2020-08-07T15:10:50.432 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.432 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.432 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:50.432 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.176221+0000", 2020-08-07T15:10:50.432 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3848000000000003e-05 2020-08-07T15:10:50.433 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.433 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.433 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:50.446 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.176285+0000", 2020-08-07T15:10:50.446 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002796321 2020-08-07T15:10:50.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.447 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:50.447 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.179081+0000", 2020-08-07T15:10:50.447 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0055690610000000002 2020-08-07T15:10:50.448 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.448 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:50.448 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.448 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.452 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:48086 5.8 5:12f3d078:::benchmark_data_smithi074_53697_object6010:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e190)", 2020-08-07T15:10:50.452 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:50.176122+0000", 2020-08-07T15:10:50.453 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.12326551099999999, 2020-08-07T15:10:50.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.123518847, 2020-08-07T15:10:50.453 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:50.453 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:50.453 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:50.454 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:50.454 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:50.454 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48086 2020-08-07T15:10:50.454 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.454 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:50.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:50.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.176122+0000", 2020-08-07T15:10:50.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:50.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:50.456 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.176122+0000", 2020-08-07T15:10:50.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3406999999999999e-05 2020-08-07T15:10:50.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:50.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.176175+0000", 2020-08-07T15:10:50.458 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5189999999999998e-06 2020-08-07T15:10:50.458 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.458 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.458 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:50.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.176181+0000", 2020-08-07T15:10:50.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.239e-05 2020-08-07T15:10:50.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.460 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.461 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:50.461 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.176194+0000", 2020-08-07T15:10:50.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.297e-06 2020-08-07T15:10:50.461 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.461 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.462 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:50.462 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.176200+0000", 2020-08-07T15:10:50.462 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010597799999999999 2020-08-07T15:10:50.462 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.462 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.463 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:50.463 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.176306+0000", 2020-08-07T15:10:50.463 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8689e-05 2020-08-07T15:10:50.463 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.464 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.464 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:50.464 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.176365+0000", 2020-08-07T15:10:50.464 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000103318 2020-08-07T15:10:50.464 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.465 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.480 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:10:50.480 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.176468+0000", 2020-08-07T15:10:50.480 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0050863339999999996 2020-08-07T15:10:50.481 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.481 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.481 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:50.481 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.181554+0000", 2020-08-07T15:10:50.481 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5999999999999991e-07 2020-08-07T15:10:50.482 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.482 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.482 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:50.482 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.181555+0000", 2020-08-07T15:10:50.482 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9789999999999994e-05 2020-08-07T15:10:50.483 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.483 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.483 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:50.494 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.181625+0000", 2020-08-07T15:10:50.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00025359500000000001 2020-08-07T15:10:50.495 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.495 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.495 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:50.495 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.181879+0000", 2020-08-07T15:10:50.496 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0057572769999999999 2020-08-07T15:10:50.496 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.496 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:50.496 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.496 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.496 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.497 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:48087 5.8 5:12f3d078:::benchmark_data_smithi074_53697_object6010:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e190)", 2020-08-07T15:10:50.497 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:50.176513+0000", 2020-08-07T15:10:50.497 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.122874071, 2020-08-07T15:10:50.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.123245777, 2020-08-07T15:10:50.503 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:50.504 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:50.504 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:50.504 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:50.504 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:50.504 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48087 2020-08-07T15:10:50.505 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.505 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:50.505 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.505 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:50.506 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.176513+0000", 2020-08-07T15:10:50.506 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:50.506 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.506 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.506 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:50.507 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.176513+0000", 2020-08-07T15:10:50.507 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7426000000000003e-05 2020-08-07T15:10:50.507 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.507 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.507 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:50.508 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.176560+0000", 2020-08-07T15:10:50.508 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6600000000000003e-06 2020-08-07T15:10:50.508 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.508 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.508 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:50.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.176565+0000", 2020-08-07T15:10:50.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7489999999999998e-06 2020-08-07T15:10:50.509 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.509 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.509 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:50.510 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.176573+0000", 2020-08-07T15:10:50.510 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0500000000000002e-06 2020-08-07T15:10:50.510 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.510 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:50.510 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.176577+0000", 2020-08-07T15:10:50.511 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00029189500000000002 2020-08-07T15:10:50.511 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:50.511 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.176869+0000", 2020-08-07T15:10:50.512 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8369000000000006e-05 2020-08-07T15:10:50.517 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.517 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.517 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:50.518 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.176967+0000", 2020-08-07T15:10:50.518 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017304899999999999 2020-08-07T15:10:50.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.518 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:10:50.519 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.177140+0000", 2020-08-07T15:10:50.519 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0048392859999999999 2020-08-07T15:10:50.519 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.519 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:50.520 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.181979+0000", 2020-08-07T15:10:50.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.85e-07 2020-08-07T15:10:50.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.521 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.521 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:50.521 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.181980+0000", 2020-08-07T15:10:50.522 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8534000000000001e-05 2020-08-07T15:10:50.522 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.522 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.522 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:50.522 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.182019+0000", 2020-08-07T15:10:50.523 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028888030000000001 2020-08-07T15:10:50.523 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.523 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.523 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:50.523 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.184908+0000", 2020-08-07T15:10:50.524 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0083946060000000006 2020-08-07T15:10:50.524 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.524 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:50.524 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.524 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.524 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.525 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:48088 5.8 5:12f3d078:::benchmark_data_smithi074_53697_object6010:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e190)", 2020-08-07T15:10:50.525 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:50.181904+0000", 2020-08-07T15:10:50.525 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.117482843, 2020-08-07T15:10:50.525 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.117958168, 2020-08-07T15:10:50.525 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:50.526 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:50.526 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:50.526 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:50.526 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:50.526 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48088 2020-08-07T15:10:50.527 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.527 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:50.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.528 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:50.528 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.181904+0000", 2020-08-07T15:10:50.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:50.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.529 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.529 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:50.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.181904+0000", 2020-08-07T15:10:50.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5945000000000001e-05 2020-08-07T15:10:50.529 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.530 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.530 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:50.530 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.181960+0000", 2020-08-07T15:10:50.530 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1130000000000003e-06 2020-08-07T15:10:50.530 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.531 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.531 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:50.531 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.181967+0000", 2020-08-07T15:10:50.531 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3760999999999999e-05 2020-08-07T15:10:50.531 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.532 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.532 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:50.532 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.181981+0000", 2020-08-07T15:10:50.532 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3600000000000001e-06 2020-08-07T15:10:50.532 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.533 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.533 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:50.533 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.181987+0000", 2020-08-07T15:10:50.533 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2646999999999999e-05 2020-08-07T15:10:50.533 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.534 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.534 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:50.534 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.182040+0000", 2020-08-07T15:10:50.534 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2627999999999996e-05 2020-08-07T15:10:50.534 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:50.535 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.182113+0000", 2020-08-07T15:10:50.535 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000144432 2020-08-07T15:10:50.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.536 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:10:50.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.182257+0000", 2020-08-07T15:10:50.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024411939999999998 2020-08-07T15:10:50.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.537 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.537 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:50.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.184698+0000", 2020-08-07T15:10:50.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.209e-06 2020-08-07T15:10:50.537 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.538 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.539 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:50.539 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.184699+0000", 2020-08-07T15:10:50.539 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9321999999999998e-05 2020-08-07T15:10:50.539 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.539 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.540 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:50.540 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.184769+0000", 2020-08-07T15:10:50.540 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002864611 2020-08-07T15:10:50.540 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.540 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:50.541 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.541 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.541 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:48073 5.8 5:1eab3d78:::benchmark_data_smithi074_53697_object6009:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e190)", 2020-08-07T15:10:50.541 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:50.143286+0000", 2020-08-07T15:10:50.542 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.156100607, 2020-08-07T15:10:50.542 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.156692575, 2020-08-07T15:10:50.542 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:50.542 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:50.544 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:50.544 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:50.544 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:50.544 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48073 2020-08-07T15:10:50.544 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.545 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:50.545 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:50.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.143286+0000", 2020-08-07T15:10:50.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:50.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.546 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.546 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:50.546 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.143286+0000", 2020-08-07T15:10:50.546 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.83e-05 2020-08-07T15:10:50.547 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.547 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:50.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.143315+0000", 2020-08-07T15:10:50.547 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7430000000000002e-06 2020-08-07T15:10:50.548 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.548 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:50.551 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.143319+0000", 2020-08-07T15:10:50.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1980000000000007e-06 2020-08-07T15:10:50.551 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.551 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.551 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:50.552 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.143329+0000", 2020-08-07T15:10:50.552 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.507e-06 2020-08-07T15:10:50.552 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.553 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:50.553 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.143334+0000", 2020-08-07T15:10:50.553 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3572999999999998e-05 2020-08-07T15:10:50.553 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.554 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.554 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:50.554 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.143378+0000", 2020-08-07T15:10:50.554 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012077400000000001 2020-08-07T15:10:50.554 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.555 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.555 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:50.555 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.143499+0000", 2020-08-07T15:10:50.555 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000127918 2020-08-07T15:10:50.555 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:10:50.556 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.143626+0000", 2020-08-07T15:10:50.556 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0014024440000000001 2020-08-07T15:10:50.557 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.557 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.557 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:50.557 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.145029+0000", 2020-08-07T15:10:50.557 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2589999999999999e-06 2020-08-07T15:10:50.557 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.558 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.558 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:50.558 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.145030+0000", 2020-08-07T15:10:50.558 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4709000000000002e-05 2020-08-07T15:10:50.558 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.559 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.559 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:50.559 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.145075+0000", 2020-08-07T15:10:50.559 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038096800000000002 2020-08-07T15:10:50.559 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.560 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.560 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:50.560 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.148885+0000", 2020-08-07T15:10:50.560 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0055981049999999999 2020-08-07T15:10:50.560 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.584 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:50.585 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.585 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.585 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.609 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:48074 5.8 5:1eab3d78:::benchmark_data_smithi074_53697_object6009:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e190)", 2020-08-07T15:10:50.610 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:50.145251+0000", 2020-08-07T15:10:50.610 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.15413605499999999, 2020-08-07T15:10:50.610 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.15482869299999999, 2020-08-07T15:10:50.611 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:50.611 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:50.611 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:50.611 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:50.611 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:50.612 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48074 2020-08-07T15:10:50.612 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.612 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:50.612 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.612 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:50.613 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.145251+0000", 2020-08-07T15:10:50.613 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:50.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.613 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:50.614 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.145251+0000", 2020-08-07T15:10:50.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6985e-05 2020-08-07T15:10:50.614 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.614 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.614 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:50.615 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.145278+0000", 2020-08-07T15:10:50.615 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3390000000000003e-06 2020-08-07T15:10:50.615 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.615 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.615 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:50.616 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.145282+0000", 2020-08-07T15:10:50.616 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1970000000000002e-06 2020-08-07T15:10:50.616 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.616 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.616 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:50.616 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.145292+0000", 2020-08-07T15:10:50.617 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3249999999999998e-06 2020-08-07T15:10:50.617 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.617 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.617 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:50.617 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.145297+0000", 2020-08-07T15:10:50.618 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2138999999999998e-05 2020-08-07T15:10:50.618 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.618 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.619 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:50.620 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.145329+0000", 2020-08-07T15:10:50.620 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9241999999999999e-05 2020-08-07T15:10:50.620 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.620 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.620 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:50.621 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.145398+0000", 2020-08-07T15:10:50.621 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013041699999999999 2020-08-07T15:10:50.621 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.621 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.621 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:10:50.622 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.145529+0000", 2020-08-07T15:10:50.622 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002505097 2020-08-07T15:10:50.622 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.622 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.624 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:50.624 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.148034+0000", 2020-08-07T15:10:50.624 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1370000000000001e-06 2020-08-07T15:10:50.624 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.624 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.625 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:50.625 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.148035+0000", 2020-08-07T15:10:50.625 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5618999999999996e-05 2020-08-07T15:10:50.625 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.625 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.626 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:50.626 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.148100+0000", 2020-08-07T15:10:50.626 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00081050700000000002 2020-08-07T15:10:50.626 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.626 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.627 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:50.627 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.148911+0000", 2020-08-07T15:10:50.627 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0036600040000000001 2020-08-07T15:10:50.627 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.627 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:50.628 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.628 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.628 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.628 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:48075 5.8 5:1eab3d78:::benchmark_data_smithi074_53697_object6009:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e190)", 2020-08-07T15:10:50.628 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:50.145979+0000", 2020-08-07T15:10:50.629 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.15340827400000001, 2020-08-07T15:10:50.629 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.154206549, 2020-08-07T15:10:50.629 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:50.629 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:50.629 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:50.630 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:50.630 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:50.631 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48075 2020-08-07T15:10:50.631 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.631 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:50.631 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.631 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:50.632 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.145979+0000", 2020-08-07T15:10:50.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:50.634 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.634 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.634 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:50.635 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.145979+0000", 2020-08-07T15:10:50.635 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8909000000000001e-05 2020-08-07T15:10:50.635 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.636 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.636 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:50.637 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.146038+0000", 2020-08-07T15:10:50.637 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5809999999999999e-06 2020-08-07T15:10:50.637 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.637 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.638 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:50.638 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.146045+0000", 2020-08-07T15:10:50.638 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4731e-05 2020-08-07T15:10:50.638 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.638 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.639 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:50.639 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.146060+0000", 2020-08-07T15:10:50.639 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4460000000000004e-06 2020-08-07T15:10:50.639 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.650 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.650 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:50.650 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.146067+0000", 2020-08-07T15:10:50.651 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0782999999999999e-05 2020-08-07T15:10:50.654 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.654 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.655 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:50.655 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.146108+0000", 2020-08-07T15:10:50.655 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9060999999999995e-05 2020-08-07T15:10:50.655 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.655 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.656 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:50.656 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.146177+0000", 2020-08-07T15:10:50.656 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000116826 2020-08-07T15:10:50.656 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.656 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.656 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:10:50.657 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.146294+0000", 2020-08-07T15:10:50.657 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0046157409999999996 2020-08-07T15:10:50.657 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.657 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.657 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:50.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.150910+0000", 2020-08-07T15:10:50.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.173e-06 2020-08-07T15:10:50.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:50.668 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.150911+0000", 2020-08-07T15:10:50.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9245999999999994e-05 2020-08-07T15:10:50.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:50.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.150980+0000", 2020-08-07T15:10:50.670 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000304357 2020-08-07T15:10:50.670 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.670 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.670 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:50.670 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.151285+0000", 2020-08-07T15:10:50.671 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0053058539999999996 2020-08-07T15:10:50.671 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.671 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:50.671 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.671 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.672 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.672 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:48076 5.8 5:1eab3d78:::benchmark_data_smithi074_53697_object6009:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e190)", 2020-08-07T15:10:50.672 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:50.148302+0000", 2020-08-07T15:10:50.672 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.151085002, 2020-08-07T15:10:50.672 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.15198877199999999, 2020-08-07T15:10:50.673 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:50.673 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:50.673 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:50.673 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:50.673 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:50.687 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48076 2020-08-07T15:10:50.687 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.687 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:50.689 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.689 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:50.689 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.148302+0000", 2020-08-07T15:10:50.689 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:50.689 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.690 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.690 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:50.691 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.148302+0000", 2020-08-07T15:10:50.691 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0068000000000002e-05 2020-08-07T15:10:50.691 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.691 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.692 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:50.692 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.148352+0000", 2020-08-07T15:10:50.692 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3400000000000003e-06 2020-08-07T15:10:50.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.693 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:50.693 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.148358+0000", 2020-08-07T15:10:50.693 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2669e-05 2020-08-07T15:10:50.693 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.694 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.694 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:50.694 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.148371+0000", 2020-08-07T15:10:50.694 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7209999999999997e-06 2020-08-07T15:10:50.695 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.695 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:50.695 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.148378+0000", 2020-08-07T15:10:50.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6239000000000003e-05 2020-08-07T15:10:50.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.696 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:50.696 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.148414+0000", 2020-08-07T15:10:50.697 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2518e-05 2020-08-07T15:10:50.697 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:50.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.148477+0000", 2020-08-07T15:10:50.698 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000107891 2020-08-07T15:10:50.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.699 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.699 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:10:50.699 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.148584+0000", 2020-08-07T15:10:50.699 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0051127869999999997 2020-08-07T15:10:50.699 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.700 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:50.700 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.153697+0000", 2020-08-07T15:10:50.700 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.237e-06 2020-08-07T15:10:50.700 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.701 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:50.701 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.153699+0000", 2020-08-07T15:10:50.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2556999999999999e-05 2020-08-07T15:10:50.701 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.702 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.702 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:50.702 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.153741+0000", 2020-08-07T15:10:50.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00095882300000000003 2020-08-07T15:10:50.703 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.703 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:50.703 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.154700+0000", 2020-08-07T15:10:50.703 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063978500000000001 2020-08-07T15:10:50.703 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.704 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:50.713 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.713 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.714 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.714 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:48077 5.8 5:1eab3d78:::benchmark_data_smithi074_53697_object6009:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e190)", 2020-08-07T15:10:50.714 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:50.151189+0000", 2020-08-07T15:10:50.714 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.14819850700000001, 2020-08-07T15:10:50.714 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.149207437, 2020-08-07T15:10:50.715 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:50.715 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:50.715 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:50.715 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:50.716 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:50.716 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48077 2020-08-07T15:10:50.716 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.716 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:50.716 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.717 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:50.717 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.151189+0000", 2020-08-07T15:10:50.717 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:50.717 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.717 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.734 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:50.736 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.151189+0000", 2020-08-07T15:10:50.736 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8365999999999999e-05 2020-08-07T15:10:50.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.736 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.737 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:50.737 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.151217+0000", 2020-08-07T15:10:50.737 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.121e-06 2020-08-07T15:10:50.737 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.745 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:50.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.151221+0000", 2020-08-07T15:10:50.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9789999999999993e-06 2020-08-07T15:10:50.746 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.746 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:50.746 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.151229+0000", 2020-08-07T15:10:50.746 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1930000000000003e-06 2020-08-07T15:10:50.747 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.747 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.747 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:50.747 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.151233+0000", 2020-08-07T15:10:50.747 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7053999999999998e-05 2020-08-07T15:10:50.748 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.748 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.748 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:50.748 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.151310+0000", 2020-08-07T15:10:50.748 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000102021 2020-08-07T15:10:50.749 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.749 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.749 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:50.749 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.151412+0000", 2020-08-07T15:10:50.749 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000181507 2020-08-07T15:10:50.763 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.764 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:10:50.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.151594+0000", 2020-08-07T15:10:50.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074583849999999997 2020-08-07T15:10:50.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.765 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:50.765 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.159052+0000", 2020-08-07T15:10:50.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7090000000000001e-06 2020-08-07T15:10:50.766 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.766 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:50.766 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.159054+0000", 2020-08-07T15:10:50.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6131999999999995e-05 2020-08-07T15:10:50.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:50.767 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.159130+0000", 2020-08-07T15:10:50.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000411226 2020-08-07T15:10:50.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.768 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.768 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:50.768 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.159541+0000", 2020-08-07T15:10:50.768 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0083526929999999996 2020-08-07T15:10:50.768 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.769 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:50.769 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.769 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.769 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.769 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:48078 5.8 5:1eab3d78:::benchmark_data_smithi074_53697_object6009:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e190)", 2020-08-07T15:10:50.770 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:50.153927+0000", 2020-08-07T15:10:50.771 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.14545999900000001, 2020-08-07T15:10:50.771 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14661131699999999, 2020-08-07T15:10:50.771 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:50.771 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:50.772 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:50.772 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:50.772 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:50.773 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48078 2020-08-07T15:10:50.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.776 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:50.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:50.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.153927+0000", 2020-08-07T15:10:50.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:50.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:50.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.153927+0000", 2020-08-07T15:10:50.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4045999999999999e-05 2020-08-07T15:10:50.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:50.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.153961+0000", 2020-08-07T15:10:50.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9859999999999996e-06 2020-08-07T15:10:50.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.779 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:50.779 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.153965+0000", 2020-08-07T15:10:50.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7940000000000003e-06 2020-08-07T15:10:50.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.782 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.782 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:50.782 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.153973+0000", 2020-08-07T15:10:50.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0130000000000004e-06 2020-08-07T15:10:50.783 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.790 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:50.815 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.153977+0000", 2020-08-07T15:10:50.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2276e-05 2020-08-07T15:10:50.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.817 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:50.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.154019+0000", 2020-08-07T15:10:50.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000102512 2020-08-07T15:10:50.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:50.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.154122+0000", 2020-08-07T15:10:50.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000180378 2020-08-07T15:10:50.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:10:50.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.154302+0000", 2020-08-07T15:10:50.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0051166370000000003 2020-08-07T15:10:50.854 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.854 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.878 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:50.878 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.159419+0000", 2020-08-07T15:10:50.878 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0699999999999999e-06 2020-08-07T15:10:50.879 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.879 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:50.879 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.159420+0000", 2020-08-07T15:10:50.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5499000000000004e-05 2020-08-07T15:10:50.880 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.880 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:50.880 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.159485+0000", 2020-08-07T15:10:50.880 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0066186810000000004 2020-08-07T15:10:50.881 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.881 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.881 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:50.881 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.166104+0000", 2020-08-07T15:10:50.881 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.012176892 2020-08-07T15:10:50.896 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.896 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:50.896 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.897 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.897 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.897 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:48079 5.8 5:1eab3d78:::benchmark_data_smithi074_53697_object6009:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e190)", 2020-08-07T15:10:50.897 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:50.159387+0000", 2020-08-07T15:10:50.901 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.14000051999999999, 2020-08-07T15:10:50.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.141264537, 2020-08-07T15:10:50.901 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:50.913 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:50.914 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:50.914 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:50.914 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:50.914 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48079 2020-08-07T15:10:50.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.915 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:50.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:50.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.159387+0000", 2020-08-07T15:10:50.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:50.918 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.918 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:50.918 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.159387+0000", 2020-08-07T15:10:50.918 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5224e-05 2020-08-07T15:10:50.919 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.919 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:50.919 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.159422+0000", 2020-08-07T15:10:50.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3440000000000003e-06 2020-08-07T15:10:50.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:50.920 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.159426+0000", 2020-08-07T15:10:50.920 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1926000000000001e-05 2020-08-07T15:10:50.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:50.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.159438+0000", 2020-08-07T15:10:50.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0000000000000004e-06 2020-08-07T15:10:50.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.922 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:50.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.159443+0000", 2020-08-07T15:10:50.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000123764 2020-08-07T15:10:50.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:50.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.159567+0000", 2020-08-07T15:10:50.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000106525 2020-08-07T15:10:50.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.934 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:50.934 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.159673+0000", 2020-08-07T15:10:50.934 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000181581 2020-08-07T15:10:50.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:10:50.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.159855+0000", 2020-08-07T15:10:50.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004876394 2020-08-07T15:10:50.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:50.938 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.164731+0000", 2020-08-07T15:10:50.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6020000000000001e-06 2020-08-07T15:10:50.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:50.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.164733+0000", 2020-08-07T15:10:50.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6448e-05 2020-08-07T15:10:50.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.944 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:50.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.164779+0000", 2020-08-07T15:10:50.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001752158 2020-08-07T15:10:50.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:50.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.166531+0000", 2020-08-07T15:10:50.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.007144966 2020-08-07T15:10:50.945 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.949 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:50.950 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.950 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.950 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.950 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:48080 5.8 5:1eab3d78:::benchmark_data_smithi074_53697_object6009:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e190)", 2020-08-07T15:10:50.951 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:50.159671+0000", 2020-08-07T15:10:50.951 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.13971651600000001, 2020-08-07T15:10:50.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14109835500000001, 2020-08-07T15:10:50.958 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:50.959 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:50.959 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:50.960 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:50.960 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:50.960 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48080 2020-08-07T15:10:50.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.961 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:50.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:50.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.159671+0000", 2020-08-07T15:10:50.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:50.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.962 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:50.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.159671+0000", 2020-08-07T15:10:50.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7183e-05 2020-08-07T15:10:50.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:50.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.159698+0000", 2020-08-07T15:10:50.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8700000000000002e-06 2020-08-07T15:10:50.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:50.964 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.159702+0000", 2020-08-07T15:10:50.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1250000000000004e-06 2020-08-07T15:10:50.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:50.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.159709+0000", 2020-08-07T15:10:50.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9009999999999996e-06 2020-08-07T15:10:50.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:50.980 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.159713+0000", 2020-08-07T15:10:50.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00053857899999999997 2020-08-07T15:10:50.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.981 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:50.981 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.160251+0000", 2020-08-07T15:10:50.982 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9612999999999996e-05 2020-08-07T15:10:50.982 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.982 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:50.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.160351+0000", 2020-08-07T15:10:50.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017383399999999999 2020-08-07T15:10:50.983 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.983 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.983 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:10:50.983 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.160525+0000", 2020-08-07T15:10:50.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054799439999999996 2020-08-07T15:10:50.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.993 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.993 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:50.993 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.166005+0000", 2020-08-07T15:10:50.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3829999999999999e-06 2020-08-07T15:10:50.994 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.994 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.994 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:50.994 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.166006+0000", 2020-08-07T15:10:50.995 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4199000000000001e-05 2020-08-07T15:10:50.995 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.996 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.996 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:50.996 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.166060+0000", 2020-08-07T15:10:50.996 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0047054189999999997 2020-08-07T15:10:50.997 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.997 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.997 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:50.997 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.170766+0000", 2020-08-07T15:10:50.997 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.01109505 2020-08-07T15:10:50.998 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.998 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:50.998 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:50.998 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:50.998 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:50.999 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:48081 5.8 5:12f3d078:::benchmark_data_smithi074_53697_object6010:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e190)", 2020-08-07T15:10:51.000 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:50.165059+0000", 2020-08-07T15:10:51.000 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.13432756800000001, 2020-08-07T15:10:51.000 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13582295599999999, 2020-08-07T15:10:51.001 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:51.001 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:51.023 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:51.023 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:51.023 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:51.024 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48081 2020-08-07T15:10:51.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.024 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:51.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:51.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.165059+0000", 2020-08-07T15:10:51.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:51.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.042 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:51.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.165059+0000", 2020-08-07T15:10:51.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8663000000000003e-05 2020-08-07T15:10:51.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.044 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:51.045 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.165128+0000", 2020-08-07T15:10:51.045 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4690000000000002e-06 2020-08-07T15:10:51.045 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.046 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.046 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:51.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.165135+0000", 2020-08-07T15:10:51.046 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7369999999999995e-06 2020-08-07T15:10:51.047 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.047 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:51.047 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.165143+0000", 2020-08-07T15:10:51.051 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6789999999999996e-06 2020-08-07T15:10:51.051 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.051 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:51.052 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.165148+0000", 2020-08-07T15:10:51.052 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2459999999999997e-05 2020-08-07T15:10:51.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.052 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.052 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:51.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.165190+0000", 2020-08-07T15:10:51.053 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016681700000000001 2020-08-07T15:10:51.053 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:51.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.165357+0000", 2020-08-07T15:10:51.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00022022799999999999 2020-08-07T15:10:51.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.058 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:10:51.058 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.165578+0000", 2020-08-07T15:10:51.058 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023440819999999999 2020-08-07T15:10:51.058 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.059 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.059 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:51.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.167922+0000", 2020-08-07T15:10:51.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2109999999999999e-06 2020-08-07T15:10:51.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.061 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:51.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.167923+0000", 2020-08-07T15:10:51.062 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3566999999999995e-05 2020-08-07T15:10:51.062 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.062 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.062 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:51.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.167996+0000", 2020-08-07T15:10:51.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002823168 2020-08-07T15:10:51.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.063 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:51.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.170820+0000", 2020-08-07T15:10:51.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0057600810000000002 2020-08-07T15:10:51.084 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:51.084 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:51.084 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:51.084 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.085 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:48082 5.8 5:12f3d078:::benchmark_data_smithi074_53697_object6010:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e190)", 2020-08-07T15:10:51.085 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:50.166505+0000", 2020-08-07T15:10:51.085 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.132882202, 2020-08-07T15:10:51.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13450126300000001, 2020-08-07T15:10:51.086 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:51.086 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:51.086 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:51.086 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:51.087 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:51.087 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48082 2020-08-07T15:10:51.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.087 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:51.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:51.098 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.166505+0000", 2020-08-07T15:10:51.098 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:51.098 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.099 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.099 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:51.100 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.166505+0000", 2020-08-07T15:10:51.102 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8767999999999998e-05 2020-08-07T15:10:51.102 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.102 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.103 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:51.103 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.166554+0000", 2020-08-07T15:10:51.103 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.917e-06 2020-08-07T15:10:51.103 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.103 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.104 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:51.104 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.166561+0000", 2020-08-07T15:10:51.104 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2308e-05 2020-08-07T15:10:51.104 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.104 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.105 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:51.105 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.166573+0000", 2020-08-07T15:10:51.105 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7350000000000001e-06 2020-08-07T15:10:51.105 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.113 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.114 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:51.114 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.166579+0000", 2020-08-07T15:10:51.114 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6273000000000002e-05 2020-08-07T15:10:51.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.115 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:51.115 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.166625+0000", 2020-08-07T15:10:51.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010927199999999999 2020-08-07T15:10:51.115 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.115 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.116 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:51.116 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.166734+0000", 2020-08-07T15:10:51.116 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018375800000000001 2020-08-07T15:10:51.116 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.116 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:10:51.117 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.166918+0000", 2020-08-07T15:10:51.117 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0037306959999999999 2020-08-07T15:10:51.117 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.117 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.130 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:51.130 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.170649+0000", 2020-08-07T15:10:51.130 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.08e-06 2020-08-07T15:10:51.131 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.131 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.132 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:51.132 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.170650+0000", 2020-08-07T15:10:51.132 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3365999999999997e-05 2020-08-07T15:10:51.133 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.134 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.135 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:51.136 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.170703+0000", 2020-08-07T15:10:51.136 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0027014080000000002 2020-08-07T15:10:51.136 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.136 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.136 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:51.137 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.173404+0000", 2020-08-07T15:10:51.137 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0068995810000000001 2020-08-07T15:10:51.137 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:51.138 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:51.142 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:51.144 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.146 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.146 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:48083 5.8 5:12f3d078:::benchmark_data_smithi074_53697_object6010:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e190)", 2020-08-07T15:10:51.147 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:50.168221+0000", 2020-08-07T15:10:51.147 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.13116636400000001, 2020-08-07T15:10:51.147 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13289325799999999, 2020-08-07T15:10:51.148 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:51.148 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:51.148 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:51.148 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:51.148 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:51.149 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48083 2020-08-07T15:10:51.149 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.149 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:51.149 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.150 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:51.150 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.168221+0000", 2020-08-07T15:10:51.150 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:51.150 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.150 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.151 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:51.151 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.168221+0000", 2020-08-07T15:10:51.163 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.325e-05 2020-08-07T15:10:51.164 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:51.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.168274+0000", 2020-08-07T15:10:51.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9640000000000003e-06 2020-08-07T15:10:51.165 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.165 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.165 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:51.166 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.168282+0000", 2020-08-07T15:10:51.166 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5837000000000001e-05 2020-08-07T15:10:51.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.181 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.181 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:51.182 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.168298+0000", 2020-08-07T15:10:51.182 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6499999999999996e-06 2020-08-07T15:10:51.182 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:51.183 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.168305+0000", 2020-08-07T15:10:51.183 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0102000000000003e-05 2020-08-07T15:10:51.196 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.196 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.196 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:51.197 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.168345+0000", 2020-08-07T15:10:51.197 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3238999999999994e-05 2020-08-07T15:10:51.197 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.198 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.199 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:51.199 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.168429+0000", 2020-08-07T15:10:51.199 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019204000000000001 2020-08-07T15:10:51.200 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.200 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.204 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:10:51.204 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.168621+0000", 2020-08-07T15:10:51.205 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0046249999999999998 2020-08-07T15:10:51.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.205 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.217 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:10:51.218 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.173246+0000", 2020-08-07T15:10:51.218 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0759999999999999e-06 2020-08-07T15:10:51.218 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.220 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.220 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:51.220 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.173247+0000", 2020-08-07T15:10:51.220 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7809000000000001e-05 2020-08-07T15:10:51.221 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.221 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.221 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:10:51.221 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.173315+0000", 2020-08-07T15:10:51.221 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026143070000000002 2020-08-07T15:10:51.222 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:51.222 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:51.222 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:51.222 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:50.175929+0000", 2020-08-07T15:10:51.222 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0077082740000000002 2020-08-07T15:10:51.223 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:51.223 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:51.223 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:51.223 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:51.223 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:10:51.224 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 16 2020-08-07T15:10:51.224 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:51.234 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:10:51.235 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:10:51.244 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:50.735+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:51.244 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:51.235+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:51.244 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 31 16 48156 48140 12.1307 0.320312 0.269827 0.00996905 2020-08-07T15:10:51.340 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:51.336+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:51.374 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:51.375 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:10:51.375 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:10:51.377 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:10:51.377 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:51.385 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:10:51.394 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:10:51.420 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:10:51.441 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:51.439+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:51.449 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:10:51.473 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:10:51.498 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:10:51.519 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:51.519 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:51.530 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:10:51.531 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:10:51.545 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:51.542+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:51.570 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:10:51.572 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:10:51.572 INFO:teuthology.orchestra.run.smithi074.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-08-07T15:10:51.572 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: problem getting command descriptions from osd.1 2020-08-07T15:10:51.572 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:10:51.573 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:10:51.574 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:10:51.574 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:10:51.575 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:10:51.577 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:10:51.578 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-08-07T15:10:51.579 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-08-07T15:10:51.650 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:51.645+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:51.749 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:51.745+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:51.760 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:51.760 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:51.773 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:10:51.850 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:51.847+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:51.950 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:51.948+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:51.989 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:51.989 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:52.000 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:10:52.057 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:52.053+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:52.137 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:10:52.175 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:52.175 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:52.175 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:52.171+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:52.185 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:10:52.212 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 32 15 49148 49133 11.9941 7.75781 0.0216307 0.010411 2020-08-07T15:10:52.263 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:52.261+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:52.376 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:52.368+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:52.377 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:52.377 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:52.385 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:10:52.479 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:52.476+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:52.578 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:52.579 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:52.580 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:52.577+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:52.581 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:10:52.683 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:52.682+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:52.753 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:52.754 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:52.766 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:10:52.807 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:52.806+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:52.886 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:52.884+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:52.975 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:52.976 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:53.001 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:10:53.001 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:52.998+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:53.102 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:53.100+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:53.174 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:53.174 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:53.186 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:10:53.214 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 33 16 50295 50279 11.9018 8.95312 0.00671132 0.0104932 2020-08-07T15:10:53.238 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:53.237+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:53.315 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:53.313+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:53.376 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:53.376 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:53.387 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:10:53.417 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:53.414+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:53.519 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:53.518+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:53.553 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:10:53.576 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:53.576 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:53.580 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:10:53.584 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:10:53.653 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:53.646+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:53.723 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:53.721+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:53.797 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:53.797 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:53.798 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:10:53.807 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:10:53.832 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:53.821+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:53.833 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:10:53.871 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:10:53.913 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:10:53.928 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:53.922+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:53.956 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:10:53.988 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:53.988 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:53.997 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:10:53.999 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:10:54.039 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:10:54.040 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:10:54.040 INFO:teuthology.orchestra.run.smithi074.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-08-07T15:10:54.040 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: problem getting command descriptions from osd.1 2020-08-07T15:10:54.056 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:10:54.056 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:10:54.057 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:10:54.059 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:10:54.059 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:10:54.060 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:10:54.060 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-08-07T15:10:54.060 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-08-07T15:10:54.061 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:54.050+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:54.138 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:54.133+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:54.194 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:10:54.195 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:10:54.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.195 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:51699 5.d e194/164 5:b151db1c:::benchmark_data_smithi074_53697_object6462:head v 194'3259, mlcod=194'3254)", 2020-08-07T15:10:54.195 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:54.184986+0000", 2020-08-07T15:10:54.196 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0055261090000000004, 2020-08-07T15:10:54.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055523079999999997, 2020-08-07T15:10:54.196 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:54.196 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:54.196 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:54.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:54.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.184986+0000", 2020-08-07T15:10:54.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:54.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:54.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.184986+0000", 2020-08-07T15:10:54.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029478799999999998 2020-08-07T15:10:54.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:54.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.185281+0000", 2020-08-07T15:10:54.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7399999999999998e-06 2020-08-07T15:10:54.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:54.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.185288+0000", 2020-08-07T15:10:54.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7444999999999999e-05 2020-08-07T15:10:54.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:54.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.185315+0000", 2020-08-07T15:10:54.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9039999999999997e-06 2020-08-07T15:10:54.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:54.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.185321+0000", 2020-08-07T15:10:54.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000257953 2020-08-07T15:10:54.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:54.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.185579+0000", 2020-08-07T15:10:54.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4837e-05 2020-08-07T15:10:54.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:54.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.185604+0000", 2020-08-07T15:10:54.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061766700000000002 2020-08-07T15:10:54.236 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:54.236 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:54.236 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:54.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.237 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:51700 5.d e194/164 5:b151db1c:::benchmark_data_smithi074_53697_object6462:head v 194'3260, mlcod=194'3254)", 2020-08-07T15:10:54.237 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:54.186011+0000", 2020-08-07T15:10:54.237 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.004501255, 2020-08-07T15:10:54.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046112710000000001, 2020-08-07T15:10:54.250 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:54.250 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:54.250 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:54.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:54.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.186011+0000", 2020-08-07T15:10:54.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:54.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:54.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.186011+0000", 2020-08-07T15:10:54.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1885999999999998e-05 2020-08-07T15:10:54.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:54.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.186063+0000", 2020-08-07T15:10:54.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1749000000000001e-05 2020-08-07T15:10:54.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:54.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.186075+0000", 2020-08-07T15:10:54.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4232e-05 2020-08-07T15:10:54.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:54.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.186109+0000", 2020-08-07T15:10:54.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6060000000000002e-06 2020-08-07T15:10:54.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:54.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.186114+0000", 2020-08-07T15:10:54.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010502599999999999 2020-08-07T15:10:54.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:54.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.186219+0000", 2020-08-07T15:10:54.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4110000000000001e-05 2020-08-07T15:10:54.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:54.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.186244+0000", 2020-08-07T15:10:54.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000232609 2020-08-07T15:10:54.316 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:54.317 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:54.317 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:54.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.322 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:51701 5.d e194/164 5:b151db1c:::benchmark_data_smithi074_53697_object6462:head v 194'3261, mlcod=194'3255)", 2020-08-07T15:10:54.322 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:54.187020+0000", 2020-08-07T15:10:54.323 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0034926169999999999, 2020-08-07T15:10:54.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003674643, 2020-08-07T15:10:54.324 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:54.324 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:54.324 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:54.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:54.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.187020+0000", 2020-08-07T15:10:54.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:54.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:54.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.187020+0000", 2020-08-07T15:10:54.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5652000000000002e-05 2020-08-07T15:10:54.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:54.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.187075+0000", 2020-08-07T15:10:54.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6340000000000003e-06 2020-08-07T15:10:54.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:54.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.187082+0000", 2020-08-07T15:10:54.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0505999999999999e-05 2020-08-07T15:10:54.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:54.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.187112+0000", 2020-08-07T15:10:54.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5780000000000005e-06 2020-08-07T15:10:54.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:54.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.187122+0000", 2020-08-07T15:10:54.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00057426199999999997 2020-08-07T15:10:54.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:54.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.187696+0000", 2020-08-07T15:10:54.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4805999999999999e-05 2020-08-07T15:10:54.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:54.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.187721+0000", 2020-08-07T15:10:54.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00070143799999999997 2020-08-07T15:10:54.361 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:54.361 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:54.361 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:54.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.362 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:51702 5.d e194/164 5:b151db1c:::benchmark_data_smithi074_53697_object6462:head v 194'3262, mlcod=194'3255)", 2020-08-07T15:10:54.362 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:54.187467+0000", 2020-08-07T15:10:54.363 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.003045514, 2020-08-07T15:10:54.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033041329999999999, 2020-08-07T15:10:54.363 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:54.363 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:54.364 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:54.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:54.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.187467+0000", 2020-08-07T15:10:54.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:54.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:54.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.187467+0000", 2020-08-07T15:10:54.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1690000000000001e-05 2020-08-07T15:10:54.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:54.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.187518+0000", 2020-08-07T15:10:54.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0430000000000004e-06 2020-08-07T15:10:54.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:54.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.187524+0000", 2020-08-07T15:10:54.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4612000000000001e-05 2020-08-07T15:10:54.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:54.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.187549+0000", 2020-08-07T15:10:54.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7690000000000004e-06 2020-08-07T15:10:54.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:54.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.187554+0000", 2020-08-07T15:10:54.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002436485 2020-08-07T15:10:54.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:54.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.189990+0000", 2020-08-07T15:10:54.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1407999999999999e-05 2020-08-07T15:10:54.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:54.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.190012+0000", 2020-08-07T15:10:54.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025450070000000002 2020-08-07T15:10:54.381 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:54.381 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:54.393 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:54.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.394 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:51703 5.d e194/164 5:b151db1c:::benchmark_data_smithi074_53697_object6462:head v 194'3263, mlcod=194'3255)", 2020-08-07T15:10:54.394 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:54.188313+0000", 2020-08-07T15:10:54.394 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0021991329999999998, 2020-08-07T15:10:54.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002529627, 2020-08-07T15:10:54.404 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:54.404 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:54.405 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:54.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:54.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.188313+0000", 2020-08-07T15:10:54.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:54.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:54.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.188313+0000", 2020-08-07T15:10:54.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3433000000000002e-05 2020-08-07T15:10:54.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:54.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.188367+0000", 2020-08-07T15:10:54.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.528e-06 2020-08-07T15:10:54.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:54.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.188373+0000", 2020-08-07T15:10:54.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7676999999999999e-05 2020-08-07T15:10:54.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:54.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.188401+0000", 2020-08-07T15:10:54.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2920000000000003e-06 2020-08-07T15:10:54.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:54.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.188406+0000", 2020-08-07T15:10:54.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001873977 2020-08-07T15:10:54.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:54.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.190280+0000", 2020-08-07T15:10:54.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0152999999999998e-05 2020-08-07T15:10:54.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:54.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.190300+0000", 2020-08-07T15:10:54.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019870600000000001 2020-08-07T15:10:54.445 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:54.446 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:54.446 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:54.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.447 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:51704 5.d e194/164 5:b151db1c:::benchmark_data_smithi074_53697_object6462:head v 194'3264, mlcod=194'3255)", 2020-08-07T15:10:54.447 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:54.189107+0000", 2020-08-07T15:10:54.448 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.001405739, 2020-08-07T15:10:54.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018014750000000001, 2020-08-07T15:10:54.448 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:54.448 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:54.449 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:54.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:54.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.189107+0000", 2020-08-07T15:10:54.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:54.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:54.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.189107+0000", 2020-08-07T15:10:54.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4202e-05 2020-08-07T15:10:54.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:54.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.189161+0000", 2020-08-07T15:10:54.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5089999999999999e-06 2020-08-07T15:10:54.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:54.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.189167+0000", 2020-08-07T15:10:54.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6639999999999999e-05 2020-08-07T15:10:54.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:54.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.189194+0000", 2020-08-07T15:10:54.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2719999999999997e-06 2020-08-07T15:10:54.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:54.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.189199+0000", 2020-08-07T15:10:54.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001506117 2020-08-07T15:10:54.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:54.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.190705+0000", 2020-08-07T15:10:54.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2347000000000001e-05 2020-08-07T15:10:54.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:54.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.190728+0000", 2020-08-07T15:10:54.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001621087 2020-08-07T15:10:54.504 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:54.504 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:54.504 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:54.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.505 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:51705 5.6 e194/164 5:651e0ba8:::benchmark_data_smithi074_53697_object6463:head v 194'3185, mlcod=194'3183)", 2020-08-07T15:10:54.505 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:54.190504+0000", 2020-08-07T15:10:54.505 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.4950000000000008e-06, 2020-08-07T15:10:54.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00047295399999999999, 2020-08-07T15:10:54.506 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:54.506 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:54.506 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:54.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:54.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.190504+0000", 2020-08-07T15:10:54.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:54.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:54.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.190504+0000", 2020-08-07T15:10:54.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0087999999999999e-05 2020-08-07T15:10:54.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:54.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.190554+0000", 2020-08-07T15:10:54.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5359999999999998e-06 2020-08-07T15:10:54.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:54.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.190560+0000", 2020-08-07T15:10:54.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6074000000000001e-05 2020-08-07T15:10:54.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:54.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.190586+0000", 2020-08-07T15:10:54.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7169999999999996e-06 2020-08-07T15:10:54.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:54.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.190592+0000", 2020-08-07T15:10:54.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6786999999999998e-05 2020-08-07T15:10:54.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:54.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.190639+0000", 2020-08-07T15:10:54.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4034000000000001e-05 2020-08-07T15:10:54.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:54.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.190663+0000", 2020-08-07T15:10:54.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015923599999999999 2020-08-07T15:10:54.515 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:54.516 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:54.516 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:54.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.516 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:51536 5.9 5:912e560e:::benchmark_data_smithi074_53697_object6441:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e194)", 2020-08-07T15:10:54.517 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:54.085583+0000", 2020-08-07T15:10:54.517 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.10492946, 2020-08-07T15:10:54.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.10547217, 2020-08-07T15:10:54.526 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:54.527 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:54.527 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:54.527 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:10:54.527 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:54.527 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 51536 2020-08-07T15:10:54.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.528 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:54.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:54.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.085583+0000", 2020-08-07T15:10:54.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:54.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:54.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.085583+0000", 2020-08-07T15:10:54.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6051999999999999e-05 2020-08-07T15:10:54.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:54.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.085629+0000", 2020-08-07T15:10:54.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.789e-06 2020-08-07T15:10:54.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:54.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.085635+0000", 2020-08-07T15:10:54.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.116e-05 2020-08-07T15:10:54.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:54.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.085646+0000", 2020-08-07T15:10:54.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9579999999999998e-06 2020-08-07T15:10:54.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:54.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.085651+0000", 2020-08-07T15:10:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002807042 2020-08-07T15:10:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.088458+0000", 2020-08-07T15:10:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3380000000000004e-05 2020-08-07T15:10:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.088551+0000", 2020-08-07T15:10:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000173401 2020-08-07T15:10:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:10:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.088725+0000", 2020-08-07T15:10:54.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055530529999999996 2020-08-07T15:10:54.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:54.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.094278+0000", 2020-08-07T15:10:54.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0389999999999999e-06 2020-08-07T15:10:54.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:54.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.094279+0000", 2020-08-07T15:10:54.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0723999999999998e-05 2020-08-07T15:10:54.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:54.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.094319+0000", 2020-08-07T15:10:54.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000728047 2020-08-07T15:10:54.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:54.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:54.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:54.095047+0000", 2020-08-07T15:10:54.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0094646450000000007 2020-08-07T15:10:54.555 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:54.556 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:54.556 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:54.556 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:54.556 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:10:54.556 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 8 2020-08-07T15:10:54.557 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:10:54.558 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 34 16 51739 51723 11.8835 11.2812 0.00556326 0.0105059 2020-08-07T15:10:54.559 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:10:54.560 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:54.557+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:54.662 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:54.658+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:54.763 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:10:54.764 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:10:54.764 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:10:54.764 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:10:54.765 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:10:54.765 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:54.762+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:54.779 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:10:54.870 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:54.867+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:54.972 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:10:54.968+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:10:54.996 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:10:54.997 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:10:54.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:10:54.997 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:10:54.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:54.998 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15576 3.e 3:727724a5:::benchmark_data_smithi074_35447_object1946:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:10:54.998 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:10:54.999 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 248.322400773, 2020-08-07T15:10:54.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999, 2020-08-07T15:10:54.999 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:55.000 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:55.000 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:55.000 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:10:55.001 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:55.001 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15576 2020-08-07T15:10:55.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.027 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:55.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:55.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:10:55.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:55.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:55.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:10:55.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7156e-05 2020-08-07T15:10:55.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:55.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666682+0000", 2020-08-07T15:10:55.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0209999999999999e-06 2020-08-07T15:10:55.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:55.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666687+0000", 2020-08-07T15:10:55.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0730999999999999e-05 2020-08-07T15:10:55.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:55.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666698+0000", 2020-08-07T15:10:55.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2020-08-07T15:10:55.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:55.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666702+0000", 2020-08-07T15:10:55.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023978459999999999 2020-08-07T15:10:55.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:55.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669100+0000", 2020-08-07T15:10:55.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9664000000000001e-05 2020-08-07T15:10:55.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:55.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669190+0000", 2020-08-07T15:10:55.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015957100000000001 2020-08-07T15:10:55.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:10:55.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669350+0000", 2020-08-07T15:10:55.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016470149999999999 2020-08-07T15:10:55.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:55.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:10:55.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1299999999999999e-07 2020-08-07T15:10:55.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:55.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:10:55.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5269000000000002e-05 2020-08-07T15:10:55.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:55.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.671022+0000", 2020-08-07T15:10:55.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031414400000000001 2020-08-07T15:10:55.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:55.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.674163+0000", 2020-08-07T15:10:55.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45641417400000001 2020-08-07T15:10:55.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:55.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:10:55.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0399999999999995e-07 2020-08-07T15:10:55.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:55.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:10:55.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3343999999999997e-05 2020-08-07T15:10:55.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:55.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130622+0000", 2020-08-07T15:10:55.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999 2020-08-07T15:10:55.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:55.198 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:55.198 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:55.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.199 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:41432 3.d 3:b5c8a160:::benchmark_data_smithi074_35447_object5178:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:10:55.199 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:10:55.199 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 232.196745548, 2020-08-07T15:10:55.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.37720701699999998, 2020-08-07T15:10:55.215 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:55.215 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:55.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:55.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:10:55.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:55.217 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 41432 2020-08-07T15:10:55.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.217 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:55.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:55.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:10:55.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:55.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:55.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:10:55.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4787999999999999e-05 2020-08-07T15:10:55.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:55.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792335+0000", 2020-08-07T15:10:55.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5999999999999997e-06 2020-08-07T15:10:55.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:55.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792341+0000", 2020-08-07T15:10:55.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0998e-05 2020-08-07T15:10:55.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:55.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792352+0000", 2020-08-07T15:10:55.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2989999999999996e-06 2020-08-07T15:10:55.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:55.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792357+0000", 2020-08-07T15:10:55.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001852752 2020-08-07T15:10:55.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:55.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794210+0000", 2020-08-07T15:10:55.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5771000000000002e-05 2020-08-07T15:10:55.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:55.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794305+0000", 2020-08-07T15:10:55.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012776399999999999 2020-08-07T15:10:55.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:10:55.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794433+0000", 2020-08-07T15:10:55.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00069673599999999999 2020-08-07T15:10:55.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:55.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795130+0000", 2020-08-07T15:10:55.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1500000000000001e-07 2020-08-07T15:10:55.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:55.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795130+0000", 2020-08-07T15:10:55.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7319e-05 2020-08-07T15:10:55.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:55.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795167+0000", 2020-08-07T15:10:55.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27910026700000001 2020-08-07T15:10:55.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:55.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074268+0000", 2020-08-07T15:10:55.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.095150660999999997 2020-08-07T15:10:55.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:55.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169418+0000", 2020-08-07T15:10:55.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.423e-06 2020-08-07T15:10:55.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:55.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169420+0000", 2020-08-07T15:10:55.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7423999999999996e-05 2020-08-07T15:10:55.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:55.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169497+0000", 2020-08-07T15:10:55.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.37720701699999998 2020-08-07T15:10:55.377 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:55.377 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:55.390 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:55.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.391 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:86712 3.8 e52/20 3:14925daa:::benchmark_data_smithi074_35447_object10838:head v 52'5488, mlcod=52'5479)", 2020-08-07T15:10:55.391 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.338631+0000", 2020-08-07T15:10:55.392 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 200.65040427900001, 2020-08-07T15:10:55.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.310229594, 2020-08-07T15:10:55.392 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:55.393 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:55.393 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:55.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:55.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338631+0000", 2020-08-07T15:10:55.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:55.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:55.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338631+0000", 2020-08-07T15:10:55.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6180000000000001e-05 2020-08-07T15:10:55.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:55.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338688+0000", 2020-08-07T15:10:55.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4949999999999996e-06 2020-08-07T15:10:55.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:55.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.338694+0000", 2020-08-07T15:10:55.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30438292099999997 2020-08-07T15:10:55.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:55.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643077+0000", 2020-08-07T15:10:55.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9389999999999996e-06 2020-08-07T15:10:55.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:55.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643082+0000", 2020-08-07T15:10:55.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5096999999999999e-05 2020-08-07T15:10:55.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:55.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643127+0000", 2020-08-07T15:10:55.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4323000000000001e-05 2020-08-07T15:10:55.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:55.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.643151+0000", 2020-08-07T15:10:55.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055847300000000004 2020-08-07T15:10:55.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:55.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.648736+0000", 2020-08-07T15:10:55.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012490900000000001 2020-08-07T15:10:55.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:55.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.648861+0000", 2020-08-07T15:10:55.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.310229594 2020-08-07T15:10:55.516 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:55.516 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:55.516 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:55.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.517 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:86792 3.f 3:fc5386ee:::benchmark_data_smithi074_35447_object10848:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e52)", 2020-08-07T15:10:55.534 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:34.395402+0000", 2020-08-07T15:10:55.535 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 200.59363416900001, 2020-08-07T15:10:55.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30259831999999998, 2020-08-07T15:10:55.535 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:55.536 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:55.536 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:55.536 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:10:55.537 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:55.537 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 86792 2020-08-07T15:10:55.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.558 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:55.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:55.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395402+0000", 2020-08-07T15:10:55.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:55.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:55.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395402+0000", 2020-08-07T15:10:55.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2262999999999998e-05 2020-08-07T15:10:55.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:55.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395444+0000", 2020-08-07T15:10:55.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1800000000000004e-06 2020-08-07T15:10:55.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:55.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395449+0000", 2020-08-07T15:10:55.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0738e-05 2020-08-07T15:10:55.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:55.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395460+0000", 2020-08-07T15:10:55.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7219999999999999e-06 2020-08-07T15:10:55.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:55.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.395464+0000", 2020-08-07T15:10:55.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0063169089999999999 2020-08-07T15:10:55.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:55.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.401781+0000", 2020-08-07T15:10:55.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010702200000000001 2020-08-07T15:10:55.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:55.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.401888+0000", 2020-08-07T15:10:55.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019171999999999999 2020-08-07T15:10:55.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:10:55.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.402080+0000", 2020-08-07T15:10:55.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042866839999999998 2020-08-07T15:10:55.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:55.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.406367+0000", 2020-08-07T15:10:55.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000001e-07 2020-08-07T15:10:55.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:55.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.406367+0000", 2020-08-07T15:10:55.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4720000000000004e-05 2020-08-07T15:10:55.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:55.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.406432+0000", 2020-08-07T15:10:55.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050405365000000001 2020-08-07T15:10:55.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:55.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.456837+0000", 2020-08-07T15:10:55.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.24111702700000001 2020-08-07T15:10:55.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:55.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.697954+0000", 2020-08-07T15:10:55.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.22e-06 2020-08-07T15:10:55.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:55.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.697955+0000", 2020-08-07T15:10:55.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4336999999999999e-05 2020-08-07T15:10:55.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:55.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:34.698000+0000", 2020-08-07T15:10:55.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30259831999999998 2020-08-07T15:10:55.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:55.734 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:55.735 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:55.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.735 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:104104 3.8 e58/20 3:11c276ea:::benchmark_data_smithi074_35447_object13012:head v 58'6496, mlcod=58'6487)", 2020-08-07T15:10:55.735 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:10:55.736 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 189.130550027, 2020-08-07T15:10:55.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33178497099999998, 2020-08-07T15:10:55.736 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:55.737 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:55.737 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:55.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:55.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:10:55.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:55.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:55.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:10:55.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9961999999999999e-05 2020-08-07T15:10:55.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:55.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858516+0000", 2020-08-07T15:10:55.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3200000000000001e-06 2020-08-07T15:10:55.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:55.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858520+0000", 2020-08-07T15:10:55.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32756790000000002 2020-08-07T15:10:55.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:55.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186088+0000", 2020-08-07T15:10:55.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0260000000000004e-06 2020-08-07T15:10:55.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:55.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186094+0000", 2020-08-07T15:10:55.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7919000000000001e-05 2020-08-07T15:10:55.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:55.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186132+0000", 2020-08-07T15:10:55.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4968e-05 2020-08-07T15:10:55.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:55.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186147+0000", 2020-08-07T15:10:55.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040671520000000001 2020-08-07T15:10:55.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:55.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.190214+0000", 2020-08-07T15:10:55.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6724000000000001e-05 2020-08-07T15:10:55.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:55.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.190271+0000", 2020-08-07T15:10:55.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33178497099999998 2020-08-07T15:10:55.749 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:55.750 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:55.750 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:55.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.750 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106248 3.f 3:f950fd9d:::benchmark_data_smithi074_35447_object13280:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:10:55.751 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:10:55.751 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 187.62887123300001, 2020-08-07T15:10:55.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002, 2020-08-07T15:10:55.751 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:55.751 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:55.752 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:55.752 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:10:55.752 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:10:55.752 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106248 2020-08-07T15:10:55.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.754 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:55.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:55.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:10:55.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:55.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:55.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:10:55.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7184e-05 2020-08-07T15:10:55.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:55.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360222+0000", 2020-08-07T15:10:55.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0906e-05 2020-08-07T15:10:55.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:55.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360233+0000", 2020-08-07T15:10:55.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2972999999999999e-05 2020-08-07T15:10:55.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:55.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360246+0000", 2020-08-07T15:10:55.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0114e-05 2020-08-07T15:10:55.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:55.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360256+0000", 2020-08-07T15:10:55.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1303999999999997e-05 2020-08-07T15:10:55.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:55.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360287+0000", 2020-08-07T15:10:55.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3798000000000001e-05 2020-08-07T15:10:55.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:55.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360351+0000", 2020-08-07T15:10:55.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111308 2020-08-07T15:10:55.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:10:55.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360462+0000", 2020-08-07T15:10:55.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003480674 2020-08-07T15:10:55.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:55.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:10:55.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0900000000000001e-07 2020-08-07T15:10:55.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:55.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:10:55.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8899999999999994e-05 2020-08-07T15:10:55.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:55.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.364012+0000", 2020-08-07T15:10:55.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35291637199999998 2020-08-07T15:10:55.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:55.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716929+0000", 2020-08-07T15:10:55.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.076244233999999994 2020-08-07T15:10:55.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:55.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793173+0000", 2020-08-07T15:10:55.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6109999999999999e-06 2020-08-07T15:10:55.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:55.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793175+0000", 2020-08-07T15:10:55.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7492999999999999e-05 2020-08-07T15:10:55.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:55.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793242+0000", 2020-08-07T15:10:55.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002 2020-08-07T15:10:55.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:55.773 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:55.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:55.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.776 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106248 3.f e59/20 ondisk, result = 0)", 2020-08-07T15:10:55.776 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.365323+0000", 2020-08-07T15:10:55.776 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 187.62371267699999, 2020-08-07T15:10:55.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35160734999999999, 2020-08-07T15:10:55.777 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:55.777 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:55.777 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:55.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:55.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365323+0000", 2020-08-07T15:10:55.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:55.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:55.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365323+0000", 2020-08-07T15:10:55.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4981000000000003e-05 2020-08-07T15:10:55.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:55.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365358+0000", 2020-08-07T15:10:55.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1090000000000003e-06 2020-08-07T15:10:55.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:55.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365364+0000", 2020-08-07T15:10:55.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35151899399999997 2020-08-07T15:10:55.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:55.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716883+0000", 2020-08-07T15:10:55.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0090000000000001e-06 2020-08-07T15:10:55.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:55.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716887+0000", 2020-08-07T15:10:55.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9238999999999999e-05 2020-08-07T15:10:55.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:55.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716916+0000", 2020-08-07T15:10:55.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0390000000000001e-06 2020-08-07T15:10:55.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:55.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716922+0000", 2020-08-07T15:10:55.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9789999999999993e-06 2020-08-07T15:10:55.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:55.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716930+0000", 2020-08-07T15:10:55.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35160734999999999 2020-08-07T15:10:55.791 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:55.791 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:55.791 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:55.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.792 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106248 3.f e59/20 ondisk, result = 0)", 2020-08-07T15:10:55.792 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:10:55.792 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 187.623607948, 2020-08-07T15:10:55.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001, 2020-08-07T15:10:55.793 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:55.793 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:55.793 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:55.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:55.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:10:55.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:55.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:55.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:10:55.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7518999999999999e-05 2020-08-07T15:10:55.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:55.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365445+0000", 2020-08-07T15:10:55.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-06 2020-08-07T15:10:55.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:55.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365449+0000", 2020-08-07T15:10:55.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42764147600000002 2020-08-07T15:10:55.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:55.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793091+0000", 2020-08-07T15:10:55.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6500000000000002e-06 2020-08-07T15:10:55.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:55.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793099+0000", 2020-08-07T15:10:55.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3664e-05 2020-08-07T15:10:55.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:55.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793153+0000", 2020-08-07T15:10:55.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1078e-05 2020-08-07T15:10:55.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:55.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793164+0000", 2020-08-07T15:10:55.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3404000000000003e-05 2020-08-07T15:10:55.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:55.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793248+0000", 2020-08-07T15:10:55.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001 2020-08-07T15:10:55.844 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:55.844 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:55.845 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:55.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.846 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:107952 3.d e61/35)", 2020-08-07T15:10:55.846 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:10:55.847 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 184.66267823999999, 2020-08-07T15:10:55.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001, 2020-08-07T15:10:55.847 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:55.848 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:55.848 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:55.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:55.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:10:55.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:55.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:55.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:10:55.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8826000000000003e-05 2020-08-07T15:10:55.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:55.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326406+0000", 2020-08-07T15:10:55.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9199999999999998e-06 2020-08-07T15:10:55.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:55.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326413+0000", 2020-08-07T15:10:55.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75082853500000002 2020-08-07T15:10:55.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:55.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077242+0000", 2020-08-07T15:10:55.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1479999999999999e-06 2020-08-07T15:10:55.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:55.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077246+0000", 2020-08-07T15:10:55.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3331999999999998e-05 2020-08-07T15:10:55.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:55.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077289+0000", 2020-08-07T15:10:55.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8977000000000001e-05 2020-08-07T15:10:55.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:55.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077308+0000", 2020-08-07T15:10:55.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001 2020-08-07T15:10:55.896 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:55.896 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:55.896 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:55.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.899 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31696 4.0 4:0335eed9:::benchmark_data_smithi074_44586_object3961:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e100)", 2020-08-07T15:10:55.899 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:10:55.899 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 126.88477068100001, 2020-08-07T15:10:55.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237, 2020-08-07T15:10:55.900 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:55.900 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:55.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:55.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:10:55.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:10:55.901 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31696 2020-08-07T15:10:55.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.901 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:55.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:55.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:10:55.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:55.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:55.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:10:55.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7654e-05 2020-08-07T15:10:55.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:55.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104293+0000", 2020-08-07T15:10:55.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.061e-06 2020-08-07T15:10:55.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:55.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104296+0000", 2020-08-07T15:10:55.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0859999999999998e-06 2020-08-07T15:10:55.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:55.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104302+0000", 2020-08-07T15:10:55.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8030000000000001e-06 2020-08-07T15:10:55.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:55.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104305+0000", 2020-08-07T15:10:55.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037634769999999999 2020-08-07T15:10:55.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:55.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108068+0000", 2020-08-07T15:10:55.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8814999999999998e-05 2020-08-07T15:10:55.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:55.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108137+0000", 2020-08-07T15:10:55.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111455 2020-08-07T15:10:55.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:10:55.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108248+0000", 2020-08-07T15:10:55.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028756820000000001 2020-08-07T15:10:55.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:55.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111124+0000", 2020-08-07T15:10:55.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061054499999999999 2020-08-07T15:10:55.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:55.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111735+0000", 2020-08-07T15:10:55.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.423966167 2020-08-07T15:10:55.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:55.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535701+0000", 2020-08-07T15:10:55.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4929999999999999e-06 2020-08-07T15:10:55.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:55.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535702+0000", 2020-08-07T15:10:55.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2571999999999997e-05 2020-08-07T15:10:55.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:55.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535755+0000", 2020-08-07T15:10:55.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1427000000000003e-05 2020-08-07T15:10:55.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:55.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535786+0000", 2020-08-07T15:10:55.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237 2020-08-07T15:10:55.925 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:55.925 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:55.925 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:55.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.926 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:31696 4.0 e100/81 ondisk, result = 0)", 2020-08-07T15:10:55.926 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:10:55.926 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 126.875156123, 2020-08-07T15:10:55.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954, 2020-08-07T15:10:55.927 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:55.927 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:55.928 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:55.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:55.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:10:55.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:55.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:55.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:10:55.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1185000000000001e-05 2020-08-07T15:10:55.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:55.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113901+0000", 2020-08-07T15:10:55.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2930000000000002e-06 2020-08-07T15:10:55.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:55.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113905+0000", 2020-08-07T15:10:55.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42168640400000001 2020-08-07T15:10:55.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:55.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535591+0000", 2020-08-07T15:10:55.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2820000000000002e-06 2020-08-07T15:10:55.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:55.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535598+0000", 2020-08-07T15:10:55.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6578999999999999e-05 2020-08-07T15:10:55.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:55.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535684+0000", 2020-08-07T15:10:55.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8449999999999992e-06 2020-08-07T15:10:55.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:55.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535693+0000", 2020-08-07T15:10:55.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7366000000000003e-05 2020-08-07T15:10:55.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:55.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535791+0000", 2020-08-07T15:10:55.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954 2020-08-07T15:10:55.939 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:55.939 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:55.939 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:55.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.940 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:37024 4.0 e106/81)", 2020-08-07T15:10:55.941 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:10:55.941 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 120.760367229, 2020-08-07T15:10:55.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999, 2020-08-07T15:10:55.943 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:55.944 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:10:55.944 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:55.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:55.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:10:55.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:55.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:55.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:10:55.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4926000000000001e-05 2020-08-07T15:10:55.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:55.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228683+0000", 2020-08-07T15:10:55.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1280000000000002e-06 2020-08-07T15:10:55.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:55.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228686+0000", 2020-08-07T15:10:55.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69740012100000004 2020-08-07T15:10:55.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:55.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:55.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:55.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926086+0000", 2020-08-07T15:10:55.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000004e-06 2020-08-07T15:10:56.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:56.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926090+0000", 2020-08-07T15:10:56.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113526 2020-08-07T15:10:56.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:56.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926204+0000", 2020-08-07T15:10:56.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8824000000000001e-05 2020-08-07T15:10:56.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:56.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926222+0000", 2020-08-07T15:10:56.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999 2020-08-07T15:10:56.025 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:56.025 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:56.025 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:56.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.039 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42503 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2615, mlcod=112'2607)", 2020-08-07T15:10:56.039 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:10:56.039 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 114.692795313, 2020-08-07T15:10:56.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998, 2020-08-07T15:10:56.040 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:56.040 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:56.041 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:56.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:56.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:10:56.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:56.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:56.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:10:56.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270999999999998e-05 2020-08-07T15:10:56.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:56.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296292+0000", 2020-08-07T15:10:56.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4729999999999997e-06 2020-08-07T15:10:56.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:56.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296298+0000", 2020-08-07T15:10:56.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.448795049 2020-08-07T15:10:56.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:56.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745093+0000", 2020-08-07T15:10:56.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0140000000000001e-06 2020-08-07T15:10:56.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:56.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745100+0000", 2020-08-07T15:10:56.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2596999999999998e-05 2020-08-07T15:10:56.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:56.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745153+0000", 2020-08-07T15:10:56.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1662000000000003e-05 2020-08-07T15:10:56.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:56.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745184+0000", 2020-08-07T15:10:56.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093174800000000004 2020-08-07T15:10:56.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:56.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746116+0000", 2020-08-07T15:10:56.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0926999999999994e-05 2020-08-07T15:10:56.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:56.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746187+0000", 2020-08-07T15:10:56.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998 2020-08-07T15:10:56.139 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:56.140 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:56.140 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:56.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.140 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42504 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2616, mlcod=113'2609)", 2020-08-07T15:10:56.141 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:10:56.141 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 114.68945841999999, 2020-08-07T15:10:56.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998, 2020-08-07T15:10:56.159 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:56.159 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:56.160 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:56.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:56.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:10:56.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:56.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:56.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:10:56.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2154000000000002e-05 2020-08-07T15:10:56.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:56.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299629+0000", 2020-08-07T15:10:56.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1929999999999998e-06 2020-08-07T15:10:56.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:56.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299636+0000", 2020-08-07T15:10:56.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44766315400000001 2020-08-07T15:10:56.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:56.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747299+0000", 2020-08-07T15:10:56.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5889999999999999e-06 2020-08-07T15:10:56.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:56.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747305+0000", 2020-08-07T15:10:56.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3094999999999999e-05 2020-08-07T15:10:56.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:56.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747358+0000", 2020-08-07T15:10:56.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3283e-05 2020-08-07T15:10:56.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:56.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747392+0000", 2020-08-07T15:10:56.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015319509999999999 2020-08-07T15:10:56.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:56.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748924+0000", 2020-08-07T15:10:56.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5235999999999996e-05 2020-08-07T15:10:56.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:56.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748999+0000", 2020-08-07T15:10:56.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998 2020-08-07T15:10:56.275 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:56.276 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:56.276 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:56.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.277 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42519 4.8 e113/81 4:13071de5:::benchmark_data_smithi074_44586_object5314:head v 113'2623, mlcod=113'2617)", 2020-08-07T15:10:56.277 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.762041+0000", 2020-08-07T15:10:56.282 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 114.22699436000001, 2020-08-07T15:10:56.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.34605073400000003, 2020-08-07T15:10:56.283 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:56.284 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:56.284 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:56.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:56.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.762041+0000", 2020-08-07T15:10:56.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:56.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:56.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.762041+0000", 2020-08-07T15:10:56.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9067000000000002e-05 2020-08-07T15:10:56.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:56.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.762100+0000", 2020-08-07T15:10:56.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3529999999999997e-06 2020-08-07T15:10:56.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:56.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.762108+0000", 2020-08-07T15:10:56.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.34498205900000001 2020-08-07T15:10:56.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:56.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.107090+0000", 2020-08-07T15:10:56.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7420000000000003e-06 2020-08-07T15:10:56.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:56.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.107096+0000", 2020-08-07T15:10:56.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1074000000000004e-05 2020-08-07T15:10:56.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:56.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.107167+0000", 2020-08-07T15:10:56.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4423999999999999e-05 2020-08-07T15:10:56.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:56.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.107191+0000", 2020-08-07T15:10:56.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00084080199999999998 2020-08-07T15:10:56.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:56.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.108032+0000", 2020-08-07T15:10:56.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0213000000000001e-05 2020-08-07T15:10:56.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:56.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.108092+0000", 2020-08-07T15:10:56.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.34605073400000003 2020-08-07T15:10:56.308 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:56.308 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:56.308 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:56.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.309 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42520 4.8 e113/81 4:13071de5:::benchmark_data_smithi074_44586_object5314:head v 113'2624, mlcod=113'2618)", 2020-08-07T15:10:56.309 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:10:56.309 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 114.22351421099999, 2020-08-07T15:10:56.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001, 2020-08-07T15:10:56.309 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:56.316 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:56.317 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:56.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:56.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:10:56.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:56.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:56.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:10:56.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3967999999999995e-05 2020-08-07T15:10:56.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:56.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765585+0000", 2020-08-07T15:10:56.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0949999999999998e-06 2020-08-07T15:10:56.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:56.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765593+0000", 2020-08-07T15:10:56.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52703464200000005 2020-08-07T15:10:56.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:56.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292627+0000", 2020-08-07T15:10:56.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6929999999999996e-06 2020-08-07T15:10:56.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:56.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292636+0000", 2020-08-07T15:10:56.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9292000000000001e-05 2020-08-07T15:10:56.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:56.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292695+0000", 2020-08-07T15:10:56.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5395999999999999e-05 2020-08-07T15:10:56.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:56.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292721+0000", 2020-08-07T15:10:56.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00090361399999999996 2020-08-07T15:10:56.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:56.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293624+0000", 2020-08-07T15:10:56.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1761000000000002e-05 2020-08-07T15:10:56.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:56.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293666+0000", 2020-08-07T15:10:56.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001 2020-08-07T15:10:56.369 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:56.369 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:56.369 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:56.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.370 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:19776 5.4 5:2bb0c0d9:::benchmark_data_smithi074_53697_object2471:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e173)", 2020-08-07T15:10:56.370 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:10:56.371 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.589321525999999, 2020-08-07T15:10:56.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.402061844, 2020-08-07T15:10:56.372 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:56.372 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:56.373 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:56.373 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:10:56.373 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:56.374 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 19776 2020-08-07T15:10:56.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.374 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:56.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:56.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:10:56.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:56.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:56.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:10:56.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-05 2020-08-07T15:10:56.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:56.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399752+0000", 2020-08-07T15:10:56.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8799999999999999e-06 2020-08-07T15:10:56.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:56.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399757+0000", 2020-08-07T15:10:56.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7690000000000008e-06 2020-08-07T15:10:56.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:56.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399767+0000", 2020-08-07T15:10:56.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-06 2020-08-07T15:10:56.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:56.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399771+0000", 2020-08-07T15:10:56.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003255109 2020-08-07T15:10:56.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:56.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403026+0000", 2020-08-07T15:10:56.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101044 2020-08-07T15:10:56.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:56.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403127+0000", 2020-08-07T15:10:56.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000193158 2020-08-07T15:10:56.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,5", 2020-08-07T15:10:56.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403320+0000", 2020-08-07T15:10:56.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017185530000000001 2020-08-07T15:10:56.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:56.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405038+0000", 2020-08-07T15:10:56.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6300000000000004e-07 2020-08-07T15:10:56.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:10:56.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405039+0000", 2020-08-07T15:10:56.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012590600000000001 2020-08-07T15:10:56.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:56.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405165+0000", 2020-08-07T15:10:56.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109757 2020-08-07T15:10:56.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:56.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405275+0000", 2020-08-07T15:10:56.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39642174699999999 2020-08-07T15:10:56.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:56.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801697+0000", 2020-08-07T15:10:56.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2799999999999996e-07 2020-08-07T15:10:56.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:56.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801697+0000", 2020-08-07T15:10:56.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8740999999999997e-05 2020-08-07T15:10:56.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:56.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801776+0000", 2020-08-07T15:10:56.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.402061844 2020-08-07T15:10:56.396 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:56.396 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:56.396 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:56.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.399 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:19776 5.4 e173/164 ondisk, result = 0)", 2020-08-07T15:10:56.399 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:10:56.399 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.583573222999998, 2020-08-07T15:10:56.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39631765200000002, 2020-08-07T15:10:56.400 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:56.400 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:56.400 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:56.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:56.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:10:56.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:56.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:56.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:10:56.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9332e-05 2020-08-07T15:10:56.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:56.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405492+0000", 2020-08-07T15:10:56.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8900000000000004e-06 2020-08-07T15:10:56.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:56.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405498+0000", 2020-08-07T15:10:56.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39566664200000001 2020-08-07T15:10:56.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:56.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801164+0000", 2020-08-07T15:10:56.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7440000000000001e-06 2020-08-07T15:10:56.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:56.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801168+0000", 2020-08-07T15:10:56.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050828199999999998 2020-08-07T15:10:56.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:56.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801676+0000", 2020-08-07T15:10:56.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1011e-05 2020-08-07T15:10:56.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:56.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801687+0000", 2020-08-07T15:10:56.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2750999999999993e-05 2020-08-07T15:10:56.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:56.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801780+0000", 2020-08-07T15:10:56.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39631765200000002 2020-08-07T15:10:56.438 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:56.439 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:56.439 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:56.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.440 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:38568 5.4 5:27bd1ca8:::benchmark_data_smithi074_53697_object4820:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e184)", 2020-08-07T15:10:56.441 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:44.100898+0000", 2020-08-07T15:10:56.442 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.888138028, 2020-08-07T15:10:56.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31617937600000001, 2020-08-07T15:10:56.443 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:56.443 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:10:56.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:10:56.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:10:56.445 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:56.446 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 38568 2020-08-07T15:10:56.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.447 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:56.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:56.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.100898+0000", 2020-08-07T15:10:56.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:56.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:56.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.100898+0000", 2020-08-07T15:10:56.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6347999999999999e-05 2020-08-07T15:10:56.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:56.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.100924+0000", 2020-08-07T15:10:56.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.779e-06 2020-08-07T15:10:56.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:56.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.100927+0000", 2020-08-07T15:10:56.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5209999999999999e-06 2020-08-07T15:10:56.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:56.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.100933+0000", 2020-08-07T15:10:56.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5829999999999998e-06 2020-08-07T15:10:56.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:56.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.100937+0000", 2020-08-07T15:10:56.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037280479999999999 2020-08-07T15:10:56.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:56.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.104665+0000", 2020-08-07T15:10:56.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6570000000000005e-05 2020-08-07T15:10:56.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:56.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.104761+0000", 2020-08-07T15:10:56.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016373099999999999 2020-08-07T15:10:56.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,5", 2020-08-07T15:10:56.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.104925+0000", 2020-08-07T15:10:56.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001714641 2020-08-07T15:10:56.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:56.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.106640+0000", 2020-08-07T15:10:56.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015086769999999999 2020-08-07T15:10:56.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:10:56.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.108149+0000", 2020-08-07T15:10:56.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30880820399999998 2020-08-07T15:10:56.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:56.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.416957+0000", 2020-08-07T15:10:56.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6400000000000001e-07 2020-08-07T15:10:56.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:10:56.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.416958+0000", 2020-08-07T15:10:56.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6707999999999999e-05 2020-08-07T15:10:56.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:10:56.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.417024+0000", 2020-08-07T15:10:56.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2802e-05 2020-08-07T15:10:56.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:56.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.417077+0000", 2020-08-07T15:10:56.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31617937600000001 2020-08-07T15:10:56.597 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:56.597 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:56.597 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:56.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.617 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:38568 5.4 e184/164 ondisk, result = 0)", 2020-08-07T15:10:56.617 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:44.111379+0000", 2020-08-07T15:10:56.617 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.877656626, 2020-08-07T15:10:56.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.305702486, 2020-08-07T15:10:56.629 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:10:56.629 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:10:56.630 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:10:56.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:10:56.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.111379+0000", 2020-08-07T15:10:56.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:10:56.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:10:56.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.111379+0000", 2020-08-07T15:10:56.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4854999999999999e-05 2020-08-07T15:10:56.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:10:56.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.111394+0000", 2020-08-07T15:10:56.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3259999999999998e-06 2020-08-07T15:10:56.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:10:56.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.111396+0000", 2020-08-07T15:10:56.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30438256200000002 2020-08-07T15:10:56.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:10:56.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.415779+0000", 2020-08-07T15:10:56.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1490000000000002e-06 2020-08-07T15:10:56.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:10:56.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.415784+0000", 2020-08-07T15:10:56.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011559160000000001 2020-08-07T15:10:56.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:10:56.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.416940+0000", 2020-08-07T15:10:56.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2919999999999996e-06 2020-08-07T15:10:56.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:10:56.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.416948+0000", 2020-08-07T15:10:56.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013338600000000001 2020-08-07T15:10:56.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:10:56.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:10:56.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:10:56.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.417082+0000", 2020-08-07T15:10:56.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.305702486 2020-08-07T15:10:56.673 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:10:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:10:56.674 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:10:56.675 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:10:56.705 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:10:56.708 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 35 16 53235 53219 11.8779 11.6875 0.00589843 0.0105155 2020-08-07T15:10:56.708 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 36 16 54606 54590 11.8454 10.7109 0.0049257 0.0105463 2020-08-07T15:10:56.709 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:56.675+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:56.786 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:10:56.789 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:56.786+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:56.891 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:56.889+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:56.991 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:10:56.992 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:10:56.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:56.992 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:55575 5.5 5:a67c8d83:::benchmark_data_smithi074_53697_object6946:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e194)", 2020-08-07T15:10:56.992 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:56.974278+0000", 2020-08-07T15:10:56.992 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.009749799, 2020-08-07T15:10:56.993 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0097780950000000005, 2020-08-07T15:10:56.993 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:56.993 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:10:56.993 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:56.993 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:57.018 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:57.018 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 55575 2020-08-07T15:10:57.018 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.019 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:57.019 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.019 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:57.020 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.974278+0000", 2020-08-07T15:10:57.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:57.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:57.080 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.974278+0000", 2020-08-07T15:10:57.081 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0575000000000003e-05 2020-08-07T15:10:57.081 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.102 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:57.103 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.974329+0000", 2020-08-07T15:10:57.103 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7490000000000004e-06 2020-08-07T15:10:57.103 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.104 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.104 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:57.130 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.974334+0000", 2020-08-07T15:10:57.130 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1583000000000001e-05 2020-08-07T15:10:57.144 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.144 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.145 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:57.145 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.974346+0000", 2020-08-07T15:10:57.145 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7759999999999997e-06 2020-08-07T15:10:57.145 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.146 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.146 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:57.146 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.974351+0000", 2020-08-07T15:10:57.146 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0020354209999999999 2020-08-07T15:10:57.147 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.175 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.176 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:57.176 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.976386+0000", 2020-08-07T15:10:57.176 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3949000000000005e-05 2020-08-07T15:10:57.198 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.210 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.210 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:57.211 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.976470+0000", 2020-08-07T15:10:57.211 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000107268 2020-08-07T15:10:57.212 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.212 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.213 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4", 2020-08-07T15:10:57.213 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.976577+0000", 2020-08-07T15:10:57.213 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0056721929999999999 2020-08-07T15:10:57.230 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.230 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.231 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:10:57.231 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.982249+0000", 2020-08-07T15:10:57.231 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0079715140000000007 2020-08-07T15:10:57.276 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:57.276 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:57.276 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:57.276 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.277 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.277 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:55576 5.5 5:a67c8d83:::benchmark_data_smithi074_53697_object6946:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e194)", 2020-08-07T15:10:57.277 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:56.975367+0000", 2020-08-07T15:10:57.277 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0086609640000000002, 2020-08-07T15:10:57.277 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0087954390000000004, 2020-08-07T15:10:57.286 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:57.286 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:10:57.287 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:10:57.287 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:10:57.287 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:10:57.287 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 55576 2020-08-07T15:10:57.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.288 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:57.288 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.288 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:57.288 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.975367+0000", 2020-08-07T15:10:57.288 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:57.288 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.289 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.291 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:57.291 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.975367+0000", 2020-08-07T15:10:57.292 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0748999999999998e-05 2020-08-07T15:10:57.292 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.292 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.292 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:57.292 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.975418+0000", 2020-08-07T15:10:57.293 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1190000000000002e-06 2020-08-07T15:10:57.293 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.293 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.293 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:57.293 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.975424+0000", 2020-08-07T15:10:57.294 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2823000000000001e-05 2020-08-07T15:10:57.294 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.294 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.294 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:57.294 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.975436+0000", 2020-08-07T15:10:57.295 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9950000000000002e-06 2020-08-07T15:10:57.295 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.295 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.295 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:57.295 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.975442+0000", 2020-08-07T15:10:57.296 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042317869999999999 2020-08-07T15:10:57.296 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.296 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.297 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:57.297 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.979674+0000", 2020-08-07T15:10:57.297 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010711 2020-08-07T15:10:57.298 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.298 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.298 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:57.299 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.979781+0000", 2020-08-07T15:10:57.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001025633 2020-08-07T15:10:57.299 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.300 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.300 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4", 2020-08-07T15:10:57.301 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.980807+0000", 2020-08-07T15:10:57.314 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0054402160000000003 2020-08-07T15:10:57.315 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:57.315 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:57.316 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:57.316 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.316 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.317 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5099.0:55579 5.8 e194/191 5:19b3be9f:::benchmark_data_smithi074_53697_object6947:head v 194'3411, mlcod=194'3407)", 2020-08-07T15:10:57.326 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:56.978112+0000", 2020-08-07T15:10:57.326 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0059158769999999999, 2020-08-07T15:10:57.327 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0061412189999999998, 2020-08-07T15:10:57.327 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:57.327 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:57.328 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:57.329 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.330 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:57.331 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.978112+0000", 2020-08-07T15:10:57.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:57.332 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.332 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.332 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:57.333 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.978112+0000", 2020-08-07T15:10:57.333 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9555999999999998e-05 2020-08-07T15:10:57.333 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.334 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.334 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:57.335 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.978161+0000", 2020-08-07T15:10:57.335 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0509999999999996e-06 2020-08-07T15:10:57.336 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.352 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.352 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:57.358 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.978166+0000", 2020-08-07T15:10:57.358 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001637828 2020-08-07T15:10:57.359 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.359 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.359 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:57.360 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.979804+0000", 2020-08-07T15:10:57.360 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.746e-06 2020-08-07T15:10:57.360 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.361 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:57.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.979809+0000", 2020-08-07T15:10:57.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000270272 2020-08-07T15:10:57.434 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.435 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.435 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:57.435 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.980079+0000", 2020-08-07T15:10:57.435 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4593999999999999e-05 2020-08-07T15:10:57.436 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.436 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.436 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:57.436 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.980104+0000", 2020-08-07T15:10:57.437 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019920469999999998 2020-08-07T15:10:57.437 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:57.437 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:57.466 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:57.466 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.467 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.467 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5099.0:55580 5.8 e194/191 5:19b3be9f:::benchmark_data_smithi074_53697_object6947:head v 194'3412, mlcod=194'3407)", 2020-08-07T15:10:57.467 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:56.979729+0000", 2020-08-07T15:10:57.468 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.004298975, 2020-08-07T15:10:57.468 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045925030000000004, 2020-08-07T15:10:57.468 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:57.469 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:57.469 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:57.469 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.469 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:57.490 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.979729+0000", 2020-08-07T15:10:57.491 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:57.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.491 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.492 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:57.492 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.979729+0000", 2020-08-07T15:10:57.492 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4431000000000002e-05 2020-08-07T15:10:57.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.493 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:57.510 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.979773+0000", 2020-08-07T15:10:57.510 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5409999999999997e-06 2020-08-07T15:10:57.511 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.512 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:57.512 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.979780+0000", 2020-08-07T15:10:57.512 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00038025199999999998 2020-08-07T15:10:57.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.513 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.513 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:57.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.980160+0000", 2020-08-07T15:10:57.522 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9200000000000001e-06 2020-08-07T15:10:57.522 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.523 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.523 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:57.527 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.980166+0000", 2020-08-07T15:10:57.527 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024155100000000001 2020-08-07T15:10:57.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.528 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:57.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.980407+0000", 2020-08-07T15:10:57.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8082999999999999e-05 2020-08-07T15:10:57.529 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.539 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.540 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:57.540 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.980436+0000", 2020-08-07T15:10:57.541 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00070677800000000003 2020-08-07T15:10:57.541 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:57.541 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:57.548 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:57.549 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.549 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.550 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5099.0:55576 5.5 e194/191)", 2020-08-07T15:10:57.593 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:56.981873+0000", 2020-08-07T15:10:57.593 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0021545100000000001, 2020-08-07T15:10:57.611 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0025166810000000002, 2020-08-07T15:10:57.611 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:57.612 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:10:57.612 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:57.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.628 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:57.628 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.981873+0000", 2020-08-07T15:10:57.628 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:57.629 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.637 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.637 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:57.637 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.981873+0000", 2020-08-07T15:10:57.637 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3259999999999997e-05 2020-08-07T15:10:57.640 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.640 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.640 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:57.641 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.981906+0000", 2020-08-07T15:10:57.641 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9519999999999999e-06 2020-08-07T15:10:57.641 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.641 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.641 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:57.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.981912+0000", 2020-08-07T15:10:57.642 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000122376 2020-08-07T15:10:57.642 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.642 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.642 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:57.662 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.982035+0000", 2020-08-07T15:10:57.662 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1830000000000002e-06 2020-08-07T15:10:57.662 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.663 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.663 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:57.663 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.982040+0000", 2020-08-07T15:10:57.663 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016677099999999999 2020-08-07T15:10:57.663 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:57.664 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:57.664 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:57.664 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.708 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5099.0:55581 5.8 e194/191 5:19b3be9f:::benchmark_data_smithi074_53697_object6947:head v 194'3413, mlcod=194'3407)", 2020-08-07T15:10:57.708 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:56.980071+0000", 2020-08-07T15:10:57.709 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0039568219999999996, 2020-08-07T15:10:57.716 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043753630000000002, 2020-08-07T15:10:57.716 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:57.716 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:57.717 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:57.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.753 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:57.753 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.980071+0000", 2020-08-07T15:10:57.754 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:57.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.764 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:57.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.980071+0000", 2020-08-07T15:10:57.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3808999999999999e-05 2020-08-07T15:10:57.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.765 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:57.765 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.980125+0000", 2020-08-07T15:10:57.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0790000000000003e-06 2020-08-07T15:10:57.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.780 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:57.780 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.980132+0000", 2020-08-07T15:10:57.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026078059999999998 2020-08-07T15:10:57.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.790 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:57.791 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.982740+0000", 2020-08-07T15:10:57.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7219999999999996e-06 2020-08-07T15:10:57.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:57.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.982745+0000", 2020-08-07T15:10:57.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1060999999999999e-05 2020-08-07T15:10:57.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:57.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.982786+0000", 2020-08-07T15:10:57.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5780000000000001e-05 2020-08-07T15:10:57.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:57.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.982812+0000", 2020-08-07T15:10:57.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002741257 2020-08-07T15:10:57.806 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:57.806 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:57.806 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:57.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.807 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5099.0:55408 5.3 e194/164 5:c4cea0b2:::benchmark_data_smithi074_53697_object6925:head v 194'3464, mlcod=194'3456)", 2020-08-07T15:10:57.807 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:56.853503+0000", 2020-08-07T15:10:57.807 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.130524428, 2020-08-07T15:10:57.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13100431800000001, 2020-08-07T15:10:57.808 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:57.808 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:57.808 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:57.808 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.809 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:57.809 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.853503+0000", 2020-08-07T15:10:57.809 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:57.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:57.819 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.853503+0000", 2020-08-07T15:10:57.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1678000000000002e-05 2020-08-07T15:10:57.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:57.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.853555+0000", 2020-08-07T15:10:57.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6259999999999997e-06 2020-08-07T15:10:57.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:57.828 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.853562+0000", 2020-08-07T15:10:57.829 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.129334169 2020-08-07T15:10:57.829 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.829 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.829 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:57.830 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.982896+0000", 2020-08-07T15:10:57.830 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7480000000000001e-06 2020-08-07T15:10:57.830 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.847 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.847 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:57.848 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.982901+0000", 2020-08-07T15:10:57.848 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012886989999999999 2020-08-07T15:10:57.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.855 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:57.856 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.984189+0000", 2020-08-07T15:10:57.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9954000000000002e-05 2020-08-07T15:10:57.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.856 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.856 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:57.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.984219+0000", 2020-08-07T15:10:57.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13071587400000001 2020-08-07T15:10:57.857 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:57.857 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:57.857 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:57.858 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.858 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5099.0:55582 5.8 e194/191)", 2020-08-07T15:10:57.858 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:56.982650+0000", 2020-08-07T15:10:57.858 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.001377542, 2020-08-07T15:10:57.859 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019153060000000001, 2020-08-07T15:10:57.859 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:57.859 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:10:57.859 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:57.859 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:57.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.982650+0000", 2020-08-07T15:10:57.860 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:57.860 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.860 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.861 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:57.861 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.982650+0000", 2020-08-07T15:10:57.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4277000000000002e-05 2020-08-07T15:10:57.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.881 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:57.881 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.982704+0000", 2020-08-07T15:10:57.881 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7170000000000002e-06 2020-08-07T15:10:57.882 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.882 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.882 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:57.882 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.982711+0000", 2020-08-07T15:10:57.882 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000693528 2020-08-07T15:10:57.883 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.883 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.883 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:57.883 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.983405+0000", 2020-08-07T15:10:57.883 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2110000000000003e-06 2020-08-07T15:10:57.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.884 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.884 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:57.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.983411+0000", 2020-08-07T15:10:57.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00076073299999999998 2020-08-07T15:10:57.885 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:57.885 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:57.885 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:57.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.886 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5099.0:55585 5.2 e194/164 5:48c62345:::benchmark_data_smithi074_53697_object6948:head v 194'3729, mlcod=194'3727)", 2020-08-07T15:10:57.886 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:56.982815+0000", 2020-08-07T15:10:57.886 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.001212259, 2020-08-07T15:10:57.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001800975, 2020-08-07T15:10:57.886 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:57.887 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:10:57.887 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:57.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:57.887 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.982815+0000", 2020-08-07T15:10:57.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:57.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:57.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.982815+0000", 2020-08-07T15:10:57.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9289000000000002e-05 2020-08-07T15:10:57.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:57.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.982865+0000", 2020-08-07T15:10:57.890 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8649999999999998e-06 2020-08-07T15:10:57.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.901 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:57.902 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.982871+0000", 2020-08-07T15:10:57.902 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00068097399999999999 2020-08-07T15:10:57.902 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.903 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:57.903 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.983552+0000", 2020-08-07T15:10:57.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.437e-06 2020-08-07T15:10:57.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:57.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.983556+0000", 2020-08-07T15:10:57.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0318000000000002e-05 2020-08-07T15:10:57.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:10:57.920 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.983606+0000", 2020-08-07T15:10:57.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0482000000000001e-05 2020-08-07T15:10:57.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:10:57.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.983627+0000", 2020-08-07T15:10:57.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00081136499999999998 2020-08-07T15:10:57.928 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:57.928 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:57.928 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:57.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.929 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5099.0:55583 5.8 e194/191)", 2020-08-07T15:10:57.930 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:56.983320+0000", 2020-08-07T15:10:57.954 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.00070747699999999998, 2020-08-07T15:10:57.956 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013461549999999999, 2020-08-07T15:10:57.956 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:57.956 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:10:57.957 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:57.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.957 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:57.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.983320+0000", 2020-08-07T15:10:57.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:57.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:57.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.983320+0000", 2020-08-07T15:10:57.958 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0895e-05 2020-08-07T15:10:57.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:57.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.983371+0000", 2020-08-07T15:10:57.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8310000000000003e-06 2020-08-07T15:10:57.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:57.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.983378+0000", 2020-08-07T15:10:57.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00063048300000000004 2020-08-07T15:10:57.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:57.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:57.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:57.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.984008+0000", 2020-08-07T15:10:57.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4240000000000001e-06 2020-08-07T15:10:58.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:58.052 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:58.053 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:58.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.984014+0000", 2020-08-07T15:10:58.053 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00069363300000000003 2020-08-07T15:10:58.053 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:58.054 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:58.054 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:58.054 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:58.054 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:58.054 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5099.0:55586 5.2 e194/164)", 2020-08-07T15:10:58.055 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:10:56.983471+0000", 2020-08-07T15:10:58.055 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.00055659199999999996, 2020-08-07T15:10:58.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001230936, 2020-08-07T15:10:58.055 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:10:58.055 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:10:58.056 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:10:58.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:58.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:10:58.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.983471+0000", 2020-08-07T15:10:58.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:10:58.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:58.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:58.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:10:58.057 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.983471+0000", 2020-08-07T15:10:58.084 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0144e-05 2020-08-07T15:10:58.084 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:58.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:58.085 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:10:58.085 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.983521+0000", 2020-08-07T15:10:58.102 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0480000000000004e-06 2020-08-07T15:10:58.103 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:58.103 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:58.104 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:10:58.104 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.983527+0000", 2020-08-07T15:10:58.104 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00062929900000000005 2020-08-07T15:10:58.105 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:58.110 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:58.110 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:10:58.111 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.984157+0000", 2020-08-07T15:10:58.111 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9189999999999998e-06 2020-08-07T15:10:58.111 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:10:58.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:10:58.112 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:10:58.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:10:56.984162+0000", 2020-08-07T15:10:58.113 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00069041000000000003 2020-08-07T15:10:58.113 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:58.113 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:10:58.130 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:58.131 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:10:58.131 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:10:58.133 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 11 2020-08-07T15:10:58.133 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:10:58.142 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:56.990+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:58.143 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:10:58.145 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:10:58.145 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:10:58.146 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:10:58.146 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:10:58.148 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:10:58.148 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:10:58.148 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:10:58.149 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 37 16 55909 55893 11.8004 10.1797 0.0197869 0.0105849 2020-08-07T15:10:58.149 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:10:58.150 INFO:teuthology.orchestra.run.smithi074.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-08-07T15:10:58.151 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: problem getting command descriptions from osd.1 2020-08-07T15:10:58.151 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:10:58.183 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:10:58.184 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:10:58.184 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:10:58.185 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:10:58.185 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:10:58.185 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-08-07T15:10:58.186 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-08-07T15:10:58.187 INFO:teuthology.orchestra.run.smithi074.stdout:5.c 2020-08-07T15:10:58.187 INFO:teuthology.orchestra.run.smithi074.stdout:5.8 2020-08-07T15:10:58.200 INFO:teuthology.orchestra.run.smithi074.stdout:5.7 2020-08-07T15:10:58.200 INFO:teuthology.orchestra.run.smithi074.stdout:5.5 2020-08-07T15:10:58.200 INFO:teuthology.orchestra.run.smithi074.stdout:5.0 2020-08-07T15:10:58.201 INFO:teuthology.orchestra.run.smithi074.stdout:2.6 2020-08-07T15:10:58.201 INFO:teuthology.orchestra.run.smithi074.stdout:2.4 2020-08-07T15:10:58.201 INFO:teuthology.orchestra.run.smithi074.stdout:2.2 2020-08-07T15:10:58.202 INFO:teuthology.orchestra.run.smithi074.stdout:2.0 2020-08-07T15:10:58.203 INFO:teuthology.orchestra.run.smithi074.stdout:1.d 2020-08-07T15:10:58.203 INFO:teuthology.orchestra.run.smithi074.stdout:1.b 2020-08-07T15:10:58.203 INFO:teuthology.orchestra.run.smithi074.stdout:1.1c 2020-08-07T15:10:58.204 INFO:teuthology.orchestra.run.smithi074.stdout:1.1d 2020-08-07T15:10:58.204 INFO:teuthology.orchestra.run.smithi074.stdout:1.1b 2020-08-07T15:10:58.204 INFO:teuthology.orchestra.run.smithi074.stdout:1.1a 2020-08-07T15:10:58.204 INFO:teuthology.orchestra.run.smithi074.stdout:1.16 2020-08-07T15:10:58.222 INFO:teuthology.orchestra.run.smithi074.stdout:1.3 2020-08-07T15:10:58.222 INFO:teuthology.orchestra.run.smithi074.stdout:1.15 2020-08-07T15:10:58.223 INFO:teuthology.orchestra.run.smithi074.stdout:1.13 2020-08-07T15:10:58.223 INFO:teuthology.orchestra.run.smithi074.stdout:1.12 2020-08-07T15:10:58.224 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:58.133+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:58.239 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:10:58.239 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 38 16 57426 57410 11.8017 11.8516 0.011092 0.0105853 2020-08-07T15:10:58.271 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:58.266+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:58.352 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:58.351+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:58.430 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:58.430 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:58.437 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:10:58.479 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:58.478+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:58.486 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-objectstore-tool --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --op export --pgid 2.2 --file /var/log/ceph/exp.2.2.1 2020-08-07T15:10:58.581 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:58.580+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:58.648 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:58.648 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:58.657 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:10:58.688 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:58.687+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:58.807 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:58.806+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:58.824 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:58.824 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:58.841 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:10:58.908 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:58.907+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:59.017 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:59.013+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:59.026 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:59.026 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:59.058 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:10:59.122 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:10:59.114+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:10:59.214 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 39 16 58995 58979 11.8134 12.2578 0.00811861 0.010576 2020-08-07T15:10:59.222 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:10:59.218+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:10:59.277 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:59.278 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:59.289 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:10:59.331 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:59.330+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:59.433 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:10:59.431+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:10:59.490 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:59.490 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:59.498 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:10:59.550 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:10:59.546+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:10:59.640 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:59.641 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:59.641 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:59.637+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:59.652 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:10:59.740 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:59.738+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:59.834 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:59.835 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:10:59.843 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:10:59.844 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:10:59.839+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:10:59.945 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:10:59.940+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:10:59.995 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:10:59.995 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:00.004 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:11:00.044 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:00.042+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:00.148 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:00.146+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:00.174 INFO:teuthology.orchestra.run.smithi074.stderr:Exporting 2.2 info 2.2( empty local-lis/les=17/18 n=0 ec=17/17 lis/c=17/17 les/c/f=18/18/0 sis=17) 2020-08-07T15:11:00.189 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:11:00.190 INFO:teuthology.orchestra.run.smithi074.stderr:Export successful 2020-08-07T15:11:00.208 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:00.208 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:00.219 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:11:00.220 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:2020-08-07T15:11:00.213558+0000 min lat: 0.00100396 max lat: 1.487 avg lat: 0.0106501 2020-08-07T15:11:00.220 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-08-07T15:11:00.220 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 40 16 60042 60026 11.7225 8.17969 0.0048542 0.0106501 2020-08-07T15:11:00.244 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:11:00.247 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:00.246+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:00.350 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:00.348+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:00.377 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:11:00.391 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:00.391 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:00.403 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:11:00.409 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:11:00.446 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:11:00.455 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:00.453+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:00.490 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:11:00.538 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:11:00.555 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:00.553+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:00.575 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:11:00.607 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:11:00.608 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:11:00.608 INFO:teuthology.orchestra.run.smithi074.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-08-07T15:11:00.608 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: problem getting command descriptions from osd.1 2020-08-07T15:11:00.609 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:11:00.609 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:11:00.609 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:11:00.609 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:11:00.609 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:11:00.610 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:11:00.611 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:11:00.611 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:11:00.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.612 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:60550 5.f 5:f98859b6:::benchmark_data_smithi074_53697_object7568:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e196)", 2020-08-07T15:11:00.612 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:00.596641+0000", 2020-08-07T15:11:00.612 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.011923081, 2020-08-07T15:11:00.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011952417, 2020-08-07T15:11:00.612 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:00.613 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:00.613 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:00.624 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:11:00.624 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:00.625 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 60550 2020-08-07T15:11:00.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.625 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:00.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:00.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.596641+0000", 2020-08-07T15:11:00.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:00.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:00.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.596641+0000", 2020-08-07T15:11:00.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8590000000000001e-05 2020-08-07T15:11:00.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:00.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.596689+0000", 2020-08-07T15:11:00.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8830000000000003e-06 2020-08-07T15:11:00.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:00.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.596695+0000", 2020-08-07T15:11:00.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3427e-05 2020-08-07T15:11:00.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:00.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.596709+0000", 2020-08-07T15:11:00.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7680000000000003e-06 2020-08-07T15:11:00.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:00.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.596716+0000", 2020-08-07T15:11:00.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017873660000000001 2020-08-07T15:11:00.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:00.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.598503+0000", 2020-08-07T15:11:00.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2664999999999995e-05 2020-08-07T15:11:00.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:00.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.598596+0000", 2020-08-07T15:11:00.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016875899999999999 2020-08-07T15:11:00.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:11:00.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.598764+0000", 2020-08-07T15:11:00.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0047278830000000004 2020-08-07T15:11:00.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:00.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.603492+0000", 2020-08-07T15:11:00.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1399999999999997e-07 2020-08-07T15:11:00.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:00.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.603493+0000", 2020-08-07T15:11:00.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2145999999999998e-05 2020-08-07T15:11:00.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:00.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.603545+0000", 2020-08-07T15:11:00.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012231550000000001 2020-08-07T15:11:00.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:00.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.604768+0000", 2020-08-07T15:11:00.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003793556 2020-08-07T15:11:00.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:00.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.608562+0000", 2020-08-07T15:11:00.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999999e-07 2020-08-07T15:11:00.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:00.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.608562+0000", 2020-08-07T15:11:00.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6343000000000001e-05 2020-08-07T15:11:00.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:00.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.608598+0000", 2020-08-07T15:11:00.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011957491000000001 2020-08-07T15:11:00.679 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:00.679 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:00.679 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:00.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.680 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:60551 5.f 5:f98859b6:::benchmark_data_smithi074_53697_object7568:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e196)", 2020-08-07T15:11:00.680 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:00.597254+0000", 2020-08-07T15:11:00.680 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.011309961, 2020-08-07T15:11:00.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011509943, 2020-08-07T15:11:00.681 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:00.681 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:00.681 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:00.681 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:11:00.682 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:00.682 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 60551 2020-08-07T15:11:00.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.682 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:00.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:00.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.597254+0000", 2020-08-07T15:11:00.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:00.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:00.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.597254+0000", 2020-08-07T15:11:00.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1996999999999997e-05 2020-08-07T15:11:00.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:00.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.597306+0000", 2020-08-07T15:11:00.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5440000000000001e-06 2020-08-07T15:11:00.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:00.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.597314+0000", 2020-08-07T15:11:00.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3276e-05 2020-08-07T15:11:00.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:00.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.597327+0000", 2020-08-07T15:11:00.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7229999999999998e-06 2020-08-07T15:11:00.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:00.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.597334+0000", 2020-08-07T15:11:00.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001864849 2020-08-07T15:11:00.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:00.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.599198+0000", 2020-08-07T15:11:00.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8097999999999995e-05 2020-08-07T15:11:00.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:00.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.599296+0000", 2020-08-07T15:11:00.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018750200000000001 2020-08-07T15:11:00.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:11:00.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.599484+0000", 2020-08-07T15:11:00.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040681329999999998 2020-08-07T15:11:00.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:00.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.603552+0000", 2020-08-07T15:11:00.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.13e-07 2020-08-07T15:11:00.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:00.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.603553+0000", 2020-08-07T15:11:00.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1628000000000001e-05 2020-08-07T15:11:00.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:00.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.603604+0000", 2020-08-07T15:11:00.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030409370000000001 2020-08-07T15:11:00.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:00.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.606645+0000", 2020-08-07T15:11:00.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0093912000000000006 2020-08-07T15:11:00.699 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:00.699 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:00.699 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:00.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.700 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:60552 5.f 5:f98859b6:::benchmark_data_smithi074_53697_object7568:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e196)", 2020-08-07T15:11:00.700 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:00.598533+0000", 2020-08-07T15:11:00.700 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.010030697999999999, 2020-08-07T15:11:00.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01029793, 2020-08-07T15:11:00.701 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:00.701 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:00.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:00.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:11:00.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:00.726 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 60552 2020-08-07T15:11:00.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.727 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:00.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:00.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.598533+0000", 2020-08-07T15:11:00.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:00.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:00.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.598533+0000", 2020-08-07T15:11:00.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8208999999999999e-05 2020-08-07T15:11:00.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:00.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.598581+0000", 2020-08-07T15:11:00.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5409999999999997e-06 2020-08-07T15:11:00.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:00.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.598588+0000", 2020-08-07T15:11:00.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4456e-05 2020-08-07T15:11:00.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:00.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.598602+0000", 2020-08-07T15:11:00.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4069999999999998e-06 2020-08-07T15:11:00.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:00.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.598609+0000", 2020-08-07T15:11:00.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016225510000000001 2020-08-07T15:11:00.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:00.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.600231+0000", 2020-08-07T15:11:00.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010697299999999999 2020-08-07T15:11:00.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:00.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.600338+0000", 2020-08-07T15:11:00.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00036656599999999997 2020-08-07T15:11:00.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:11:00.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.600705+0000", 2020-08-07T15:11:00.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054875039999999998 2020-08-07T15:11:00.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:00.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.606192+0000", 2020-08-07T15:11:00.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5099999999999998e-07 2020-08-07T15:11:00.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:00.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.606193+0000", 2020-08-07T15:11:00.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5009e-05 2020-08-07T15:11:00.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:00.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.606258+0000", 2020-08-07T15:11:00.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00088479499999999996 2020-08-07T15:11:00.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:00.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.607143+0000", 2020-08-07T15:11:00.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086099620000000005 2020-08-07T15:11:00.751 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:00.751 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:00.751 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:00.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.752 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:60556 5.3 e196/164 5:c270528c:::benchmark_data_smithi074_53697_object7569:head v 196'3836, mlcod=196'3831)", 2020-08-07T15:11:00.752 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:00.606539+0000", 2020-08-07T15:11:00.753 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0020251570000000001, 2020-08-07T15:11:00.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023601360000000001, 2020-08-07T15:11:00.753 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:00.753 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:11:00.753 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:00.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:00.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.606539+0000", 2020-08-07T15:11:00.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:00.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:00.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.606539+0000", 2020-08-07T15:11:00.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9950000000000001e-05 2020-08-07T15:11:00.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:00.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.606589+0000", 2020-08-07T15:11:00.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2279999999999998e-06 2020-08-07T15:11:00.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:00.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.606594+0000", 2020-08-07T15:11:00.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018017900000000001 2020-08-07T15:11:00.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:00.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.606774+0000", 2020-08-07T15:11:00.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1070000000000001e-06 2020-08-07T15:11:00.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:00.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.606777+0000", 2020-08-07T15:11:00.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032332900000000002 2020-08-07T15:11:00.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:00.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.607101+0000", 2020-08-07T15:11:00.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2086000000000002e-05 2020-08-07T15:11:00.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:00.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.607123+0000", 2020-08-07T15:11:00.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00058387900000000004 2020-08-07T15:11:00.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:00.774 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:00.774 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:00.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.776 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:60557 5.3 e196/164 5:c270528c:::benchmark_data_smithi074_53697_object7569:head v 196'3837, mlcod=196'3831)", 2020-08-07T15:11:00.776 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:00.606677+0000", 2020-08-07T15:11:00.776 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0018866989999999999, 2020-08-07T15:11:00.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022888499999999998, 2020-08-07T15:11:00.777 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:00.777 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:11:00.777 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:00.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:00.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.606677+0000", 2020-08-07T15:11:00.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:00.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:00.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.606677+0000", 2020-08-07T15:11:00.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9789999999999997e-05 2020-08-07T15:11:00.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:00.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.606717+0000", 2020-08-07T15:11:00.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5230000000000003e-06 2020-08-07T15:11:00.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:00.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.606723+0000", 2020-08-07T15:11:00.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00038238500000000001 2020-08-07T15:11:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.607105+0000", 2020-08-07T15:11:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1450000000000001e-06 2020-08-07T15:11:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.607109+0000", 2020-08-07T15:11:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026119199999999999 2020-08-07T15:11:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:00.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.607370+0000", 2020-08-07T15:11:00.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0492999999999998e-05 2020-08-07T15:11:00.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:00.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.607391+0000", 2020-08-07T15:11:00.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00071352799999999995 2020-08-07T15:11:00.810 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:00.811 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:00.811 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:00.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.812 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:60558 5.3 e196/164 5:c270528c:::benchmark_data_smithi074_53697_object7569:head v 196'3838, mlcod=196'3831)", 2020-08-07T15:11:00.812 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:00.607236+0000", 2020-08-07T15:11:00.812 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.001327812, 2020-08-07T15:11:00.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017737829999999999, 2020-08-07T15:11:00.812 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:00.813 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:11:00.813 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:00.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:00.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.607236+0000", 2020-08-07T15:11:00.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:00.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:00.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.607236+0000", 2020-08-07T15:11:00.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7731000000000002e-05 2020-08-07T15:11:00.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:00.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.607274+0000", 2020-08-07T15:11:00.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1479999999999997e-06 2020-08-07T15:11:00.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:00.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.607279+0000", 2020-08-07T15:11:00.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9953000000000002e-05 2020-08-07T15:11:00.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:00.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.607319+0000", 2020-08-07T15:11:00.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1900000000000003e-06 2020-08-07T15:11:00.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:00.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.607324+0000", 2020-08-07T15:11:00.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000336101 2020-08-07T15:11:00.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:00.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.607660+0000", 2020-08-07T15:11:00.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9470000000000002e-05 2020-08-07T15:11:00.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:00.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:00.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:00.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:00.607680+0000", 2020-08-07T15:11:00.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00044359299999999999 2020-08-07T15:11:00.838 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:00.838 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:00.839 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:00.839 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:00.839 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:11:00.839 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 6 2020-08-07T15:11:00.839 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:11:00.840 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-08-07T15:11:00.840 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-08-07T15:11:00.860 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:11:00.861 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:00.840+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:00.862 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-objectstore-tool --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --force --op remove --pgid 2.2 2020-08-07T15:11:00.950 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:00.942+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:01.030 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:11:01.030 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:11:01.030 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:11:01.031 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:11:01.031 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:11:01.044 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:11:01.045 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:01.043+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:01.146 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:01.144+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:01.214 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 41 16 61329 61313 11.6818 10.0547 0.0229278 0.0106938 2020-08-07T15:11:01.252 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:01.247+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:01.267 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:11:01.268 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:11:01.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:11:01.268 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:11:01.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.269 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42440 4.9 4:9b3fd4e3:::benchmark_data_smithi074_44586_object5304:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:11:01.269 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:11:01.269 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 121.02305283, 2020-08-07T15:11:01.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9384075030000001, 2020-08-07T15:11:01.281 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:01.281 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:01.284 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:01.284 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:01.284 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:01.284 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42440 2020-08-07T15:11:01.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.285 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:01.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:01.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:11:01.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:01.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:01.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:11:01.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6295000000000002e-05 2020-08-07T15:11:01.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:01.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237116+0000", 2020-08-07T15:11:01.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0390000000000001e-06 2020-08-07T15:11:01.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:01.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237122+0000", 2020-08-07T15:11:01.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8340000000000001e-05 2020-08-07T15:11:01.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:01.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237140+0000", 2020-08-07T15:11:01.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.045e-06 2020-08-07T15:11:01.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:01.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237145+0000", 2020-08-07T15:11:01.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044548920000000002 2020-08-07T15:11:01.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:01.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241600+0000", 2020-08-07T15:11:01.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103336 2020-08-07T15:11:01.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:01.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241703+0000", 2020-08-07T15:11:01.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018660399999999999 2020-08-07T15:11:01.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:11:01.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241890+0000", 2020-08-07T15:11:01.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00096796000000000002 2020-08-07T15:11:01.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:01.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.242858+0000", 2020-08-07T15:11:01.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039845000000000002 2020-08-07T15:11:01.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:01.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246843+0000", 2020-08-07T15:11:01.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3499999999999996e-07 2020-08-07T15:11:01.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:01.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246843+0000", 2020-08-07T15:11:01.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7132999999999995e-05 2020-08-07T15:11:01.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:01.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246920+0000", 2020-08-07T15:11:01.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046993410000000001 2020-08-07T15:11:01.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:01.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.251620+0000", 2020-08-07T15:11:01.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040332956000000003 2020-08-07T15:11:01.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:01.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.291953+0000", 2020-08-07T15:11:01.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8390000000000003e-06 2020-08-07T15:11:01.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:01.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.291958+0000", 2020-08-07T15:11:01.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99961357500000003 2020-08-07T15:11:01.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:01.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291572+0000", 2020-08-07T15:11:01.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3913999999999999e-05 2020-08-07T15:11:01.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:01.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291586+0000", 2020-08-07T15:11:01.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88379723099999996 2020-08-07T15:11:01.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:01.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175383+0000", 2020-08-07T15:11:01.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3868000000000004e-05 2020-08-07T15:11:01.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:01.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175467+0000", 2020-08-07T15:11:01.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9384075030000001 2020-08-07T15:11:01.438 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:01.438 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:01.438 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:01.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.439 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42489 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:11:01.439 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:11:01.439 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 120.979752768, 2020-08-07T15:11:01.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952608639999999, 2020-08-07T15:11:01.439 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:01.440 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:01.440 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:01.440 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:01.440 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:01.440 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42489 2020-08-07T15:11:01.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.441 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:01.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:01.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:11:01.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:01.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:01.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:11:01.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0379e-05 2020-08-07T15:11:01.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:01.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280410+0000", 2020-08-07T15:11:01.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0489999999999998e-06 2020-08-07T15:11:01.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:01.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280417+0000", 2020-08-07T15:11:01.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8281e-05 2020-08-07T15:11:01.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:01.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280435+0000", 2020-08-07T15:11:01.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7510000000000008e-06 2020-08-07T15:11:01.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:01.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280443+0000", 2020-08-07T15:11:01.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018142899999999999 2020-08-07T15:11:01.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:01.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280624+0000", 2020-08-07T15:11:01.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013473599999999999 2020-08-07T15:11:01.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:01.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280759+0000", 2020-08-07T15:11:01.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019964200000000001 2020-08-07T15:11:01.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:11:01.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280959+0000", 2020-08-07T15:11:01.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005588038 2020-08-07T15:11:01.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:01.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286547+0000", 2020-08-07T15:11:01.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.376e-05 2020-08-07T15:11:01.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:01.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286561+0000", 2020-08-07T15:11:01.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4199999999999995e-07 2020-08-07T15:11:01.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:01.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286561+0000", 2020-08-07T15:11:01.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9218e-05 2020-08-07T15:11:01.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:01.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286631+0000", 2020-08-07T15:11:01.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027909199999999999 2020-08-07T15:11:01.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:01.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286910+0000", 2020-08-07T15:11:01.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050904089999999997 2020-08-07T15:11:01.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:01.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292000+0000", 2020-08-07T15:11:01.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9579999999999996e-06 2020-08-07T15:11:01.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292009+0000", 2020-08-07T15:11:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99961402799999999 2020-08-07T15:11:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:01.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291623+0000", 2020-08-07T15:11:01.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2490000000000001e-06 2020-08-07T15:11:01.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:01.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291631+0000", 2020-08-07T15:11:01.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88387422199999999 2020-08-07T15:11:01.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:01.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175506+0000", 2020-08-07T15:11:01.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000114881 2020-08-07T15:11:01.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:01.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175620+0000", 2020-08-07T15:11:01.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952608639999999 2020-08-07T15:11:01.631 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:01.631 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:01.631 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:01.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.633 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42490 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:11:01.633 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:11:01.648 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 120.97962560000001, 2020-08-07T15:11:01.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89528059, 2020-08-07T15:11:01.649 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:01.667 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:01.667 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:01.668 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:01.668 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:01.668 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42490 2020-08-07T15:11:01.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.668 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:01.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:01.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:11:01.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:01.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:01.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:11:01.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2571999999999997e-05 2020-08-07T15:11:01.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:01.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280539+0000", 2020-08-07T15:11:01.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999998e-06 2020-08-07T15:11:01.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:01.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280545+0000", 2020-08-07T15:11:01.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.134e-05 2020-08-07T15:11:01.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:01.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280556+0000", 2020-08-07T15:11:01.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.746e-06 2020-08-07T15:11:01.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:01.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280561+0000", 2020-08-07T15:11:01.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00089258999999999996 2020-08-07T15:11:01.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:01.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281453+0000", 2020-08-07T15:11:01.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5641000000000001e-05 2020-08-07T15:11:01.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:01.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281549+0000", 2020-08-07T15:11:01.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020555200000000001 2020-08-07T15:11:01.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:11:01.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281755+0000", 2020-08-07T15:11:01.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048878230000000003 2020-08-07T15:11:01.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:01.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286642+0000", 2020-08-07T15:11:01.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0954e-05 2020-08-07T15:11:01.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:01.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286693+0000", 2020-08-07T15:11:01.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0699999999999997e-07 2020-08-07T15:11:01.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:01.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286694+0000", 2020-08-07T15:11:01.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3697999999999998e-05 2020-08-07T15:11:01.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:01.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286768+0000", 2020-08-07T15:11:01.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017185900000000001 2020-08-07T15:11:01.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:01.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286940+0000", 2020-08-07T15:11:01.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051207819999999999 2020-08-07T15:11:01.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:01.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292060+0000", 2020-08-07T15:11:01.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8000000000000004e-06 2020-08-07T15:11:01.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:01.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292069+0000", 2020-08-07T15:11:01.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99959520599999996 2020-08-07T15:11:01.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:01.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291664+0000", 2020-08-07T15:11:01.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0160000000000008e-06 2020-08-07T15:11:01.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:01.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291672+0000", 2020-08-07T15:11:01.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88399374500000005 2020-08-07T15:11:01.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:01.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175666+0000", 2020-08-07T15:11:01.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101239 2020-08-07T15:11:01.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:01.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175767+0000", 2020-08-07T15:11:01.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89528059 2020-08-07T15:11:01.843 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:01.843 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:01.843 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:01.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.844 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42491 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:11:01.844 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:11:01.844 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 120.979518665, 2020-08-07T15:11:01.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895291936, 2020-08-07T15:11:01.845 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:01.845 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:01.845 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:01.845 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:01.846 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:01.846 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42491 2020-08-07T15:11:01.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.847 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:01.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:01.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:11:01.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:01.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:01.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:11:01.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0402000000000001e-05 2020-08-07T15:11:01.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:01.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280624+0000", 2020-08-07T15:11:01.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0340000000000001e-06 2020-08-07T15:11:01.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:01.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280627+0000", 2020-08-07T15:11:01.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1886999999999999e-05 2020-08-07T15:11:01.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:01.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280649+0000", 2020-08-07T15:11:01.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5430000000000004e-06 2020-08-07T15:11:01.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:01.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280654+0000", 2020-08-07T15:11:01.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032800960000000001 2020-08-07T15:11:01.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:01.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.283934+0000", 2020-08-07T15:11:01.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011264699999999999 2020-08-07T15:11:01.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:01.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284046+0000", 2020-08-07T15:11:01.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019987299999999999 2020-08-07T15:11:01.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:11:01.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284246+0000", 2020-08-07T15:11:01.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025610960000000001 2020-08-07T15:11:01.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:01.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286807+0000", 2020-08-07T15:11:01.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1409999999999999e-06 2020-08-07T15:11:01.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:01.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286814+0000", 2020-08-07T15:11:01.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0499999999999999e-07 2020-08-07T15:11:01.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:01.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286815+0000", 2020-08-07T15:11:01.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8538000000000003e-05 2020-08-07T15:11:01.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:01.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286853+0000", 2020-08-07T15:11:01.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112914 2020-08-07T15:11:01.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:01.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286966+0000", 2020-08-07T15:11:01.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051403120000000002 2020-08-07T15:11:01.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:01.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292107+0000", 2020-08-07T15:11:01.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8019999999999992e-06 2020-08-07T15:11:01.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:01.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292114+0000", 2020-08-07T15:11:01.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.999590859 2020-08-07T15:11:01.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:01.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291705+0000", 2020-08-07T15:11:01.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.174e-06 2020-08-07T15:11:01.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:01.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291713+0000", 2020-08-07T15:11:01.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88407676000000002 2020-08-07T15:11:01.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:01.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175790+0000", 2020-08-07T15:11:01.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5452999999999996e-05 2020-08-07T15:11:01.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:01.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175886+0000", 2020-08-07T15:11:01.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895291936 2020-08-07T15:11:01.882 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:01.882 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:01.882 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:01.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.883 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42492 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:11:01.883 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:11:01.883 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 120.979425615, 2020-08-07T15:11:01.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895293774, 2020-08-07T15:11:01.884 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:01.884 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:01.892 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:01.892 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:01.892 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:01.893 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42492 2020-08-07T15:11:01.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.893 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:01.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:01.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:11:01.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:01.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:01.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:11:01.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8272000000000001e-05 2020-08-07T15:11:01.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:01.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280735+0000", 2020-08-07T15:11:01.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2159999999999997e-06 2020-08-07T15:11:01.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:01.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280740+0000", 2020-08-07T15:11:01.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0251000000000001e-05 2020-08-07T15:11:01.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:01.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280750+0000", 2020-08-07T15:11:01.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3139999999999997e-06 2020-08-07T15:11:01.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:01.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280755+0000", 2020-08-07T15:11:01.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039343579999999998 2020-08-07T15:11:01.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:01.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284689+0000", 2020-08-07T15:11:01.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111648 2020-08-07T15:11:01.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:01.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284801+0000", 2020-08-07T15:11:01.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019563600000000001 2020-08-07T15:11:01.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:11:01.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284996+0000", 2020-08-07T15:11:01.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018636709999999999 2020-08-07T15:11:01.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:01.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286860+0000", 2020-08-07T15:11:01.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001865802 2020-08-07T15:11:01.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:01.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288726+0000", 2020-08-07T15:11:01.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0399999999999996e-07 2020-08-07T15:11:01.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:01.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288726+0000", 2020-08-07T15:11:01.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6573999999999998e-05 2020-08-07T15:11:01.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:01.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288773+0000", 2020-08-07T15:11:01.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003455283 2020-08-07T15:11:01.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:01.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292228+0000", 2020-08-07T15:11:01.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0099999999999995e-06 2020-08-07T15:11:01.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:01.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292236+0000", 2020-08-07T15:11:01.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99950963999999998 2020-08-07T15:11:01.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:01.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291746+0000", 2020-08-07T15:11:01.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0779999999999996e-06 2020-08-07T15:11:01.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:01.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291754+0000", 2020-08-07T15:11:01.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.884152508 2020-08-07T15:11:01.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:01.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:01.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:01.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175906+0000", 2020-08-07T15:11:01.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4009000000000002e-05 2020-08-07T15:11:01.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:02.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175980+0000", 2020-08-07T15:11:02.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895293774 2020-08-07T15:11:02.005 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:02.005 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:02.006 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:02.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.006 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42493 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:11:02.007 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:11:02.007 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 120.979323968, 2020-08-07T15:11:02.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952963789999999, 2020-08-07T15:11:02.007 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:02.007 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:02.008 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:02.008 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:02.008 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:02.008 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42493 2020-08-07T15:11:02.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.009 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:02.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:02.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:11:02.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:02.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:02.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:11:02.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9740999999999997e-05 2020-08-07T15:11:02.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:02.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280838+0000", 2020-08-07T15:11:02.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7809999999999996e-06 2020-08-07T15:11:02.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:02.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280843+0000", 2020-08-07T15:11:02.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0648999999999999e-05 2020-08-07T15:11:02.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:02.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280854+0000", 2020-08-07T15:11:02.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3669999999999999e-06 2020-08-07T15:11:02.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:02.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280858+0000", 2020-08-07T15:11:02.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004850867 2020-08-07T15:11:02.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.285709+0000", 2020-08-07T15:11:02.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110037 2020-08-07T15:11:02.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:02.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.285819+0000", 2020-08-07T15:11:02.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020935699999999999 2020-08-07T15:11:02.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:11:02.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286028+0000", 2020-08-07T15:11:02.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020900850000000002 2020-08-07T15:11:02.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:02.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288118+0000", 2020-08-07T15:11:02.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041669649999999999 2020-08-07T15:11:02.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292285+0000", 2020-08-07T15:11:02.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3820000000000004e-06 2020-08-07T15:11:02.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:02.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292294+0000", 2020-08-07T15:11:02.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949242900000002 2020-08-07T15:11:02.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291786+0000", 2020-08-07T15:11:02.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000003e-06 2020-08-07T15:11:02.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:02.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291794+0000", 2020-08-07T15:11:02.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88420652700000002 2020-08-07T15:11:02.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176000+0000", 2020-08-07T15:11:02.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4634000000000003e-05 2020-08-07T15:11:02.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:02.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176085+0000", 2020-08-07T15:11:02.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952963789999999 2020-08-07T15:11:02.135 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:02.135 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:02.136 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:02.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.136 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42494 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:11:02.136 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:11:02.137 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 120.979221869, 2020-08-07T15:11:02.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952882820000001, 2020-08-07T15:11:02.137 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:02.137 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:02.137 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:02.153 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:02.153 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:02.153 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42494 2020-08-07T15:11:02.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.154 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:02.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:02.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:11:02.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:02.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:02.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:11:02.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0117999999999997e-05 2020-08-07T15:11:02.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:02.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280941+0000", 2020-08-07T15:11:02.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1240000000000004e-06 2020-08-07T15:11:02.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:02.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280946+0000", 2020-08-07T15:11:02.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0000000000000001e-05 2020-08-07T15:11:02.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:02.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280956+0000", 2020-08-07T15:11:02.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.279e-06 2020-08-07T15:11:02.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:02.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280960+0000", 2020-08-07T15:11:02.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0062727080000000001 2020-08-07T15:11:02.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287233+0000", 2020-08-07T15:11:02.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018362900000000001 2020-08-07T15:11:02.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:02.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287416+0000", 2020-08-07T15:11:02.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029536100000000001 2020-08-07T15:11:02.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:11:02.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287712+0000", 2020-08-07T15:11:02.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046231739999999999 2020-08-07T15:11:02.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292335+0000", 2020-08-07T15:11:02.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3860000000000001e-06 2020-08-07T15:11:02.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:02.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292342+0000", 2020-08-07T15:11:02.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99948501000000001 2020-08-07T15:11:02.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291827+0000", 2020-08-07T15:11:02.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2272999999999999e-05 2020-08-07T15:11:02.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:02.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291840+0000", 2020-08-07T15:11:02.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88426668100000005 2020-08-07T15:11:02.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176106+0000", 2020-08-07T15:11:02.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2539000000000005e-05 2020-08-07T15:11:02.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:02.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176179+0000", 2020-08-07T15:11:02.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952882820000001 2020-08-07T15:11:02.240 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:02.240 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:02.240 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:02.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.241 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42495 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:11:02.241 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:11:02.241 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 120.979120779, 2020-08-07T15:11:02.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895286867, 2020-08-07T15:11:02.266 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:02.266 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:02.267 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:02.267 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:02.267 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:02.267 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42495 2020-08-07T15:11:02.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.268 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:02.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:02.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:11:02.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:02.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:02.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:11:02.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6690999999999997e-05 2020-08-07T15:11:02.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:02.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281038+0000", 2020-08-07T15:11:02.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7860000000000002e-06 2020-08-07T15:11:02.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:02.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281044+0000", 2020-08-07T15:11:02.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0429e-05 2020-08-07T15:11:02.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:02.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281054+0000", 2020-08-07T15:11:02.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9040000000000002e-06 2020-08-07T15:11:02.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:02.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281058+0000", 2020-08-07T15:11:02.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070941490000000001 2020-08-07T15:11:02.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288153+0000", 2020-08-07T15:11:02.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9006999999999998e-05 2020-08-07T15:11:02.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:02.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288222+0000", 2020-08-07T15:11:02.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012106400000000001 2020-08-07T15:11:02.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:11:02.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288343+0000", 2020-08-07T15:11:02.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004034752 2020-08-07T15:11:02.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292377+0000", 2020-08-07T15:11:02.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.537e-06 2020-08-07T15:11:02.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:02.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292385+0000", 2020-08-07T15:11:02.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949123500000003 2020-08-07T15:11:02.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291876+0000", 2020-08-07T15:11:02.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7729999999999999e-06 2020-08-07T15:11:02.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:02.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291884+0000", 2020-08-07T15:11:02.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88431700899999999 2020-08-07T15:11:02.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176201+0000", 2020-08-07T15:11:02.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7531e-05 2020-08-07T15:11:02.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:02.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176278+0000", 2020-08-07T15:11:02.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895286867 2020-08-07T15:11:02.401 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:02.401 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:02.401 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:02.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.402 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42496 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:11:02.402 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:11:02.403 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 120.97901587, 2020-08-07T15:11:02.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89526942, 2020-08-07T15:11:02.403 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:02.403 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:02.403 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:02.404 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:02.404 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:02.404 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42496 2020-08-07T15:11:02.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.404 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:02.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:02.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:11:02.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:02.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:02.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:11:02.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9227999999999997e-05 2020-08-07T15:11:02.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:02.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281146+0000", 2020-08-07T15:11:02.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1120000000000004e-06 2020-08-07T15:11:02.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:02.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281151+0000", 2020-08-07T15:11:02.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2208999999999999e-05 2020-08-07T15:11:02.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:02.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281163+0000", 2020-08-07T15:11:02.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5469999999999998e-06 2020-08-07T15:11:02.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:02.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281168+0000", 2020-08-07T15:11:02.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0076520829999999996 2020-08-07T15:11:02.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288820+0000", 2020-08-07T15:11:02.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5035000000000003e-05 2020-08-07T15:11:02.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:02.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288885+0000", 2020-08-07T15:11:02.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116507 2020-08-07T15:11:02.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:11:02.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.289001+0000", 2020-08-07T15:11:02.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034121960000000002 2020-08-07T15:11:02.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292413+0000", 2020-08-07T15:11:02.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8109999999999997e-06 2020-08-07T15:11:02.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:02.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292420+0000", 2020-08-07T15:11:02.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949504600000005 2020-08-07T15:11:02.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291915+0000", 2020-08-07T15:11:02.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3999999999999992e-06 2020-08-07T15:11:02.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:02.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291924+0000", 2020-08-07T15:11:02.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88437394599999997 2020-08-07T15:11:02.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176298+0000", 2020-08-07T15:11:02.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8300000000000007e-05 2020-08-07T15:11:02.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:02.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176366+0000", 2020-08-07T15:11:02.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89526942 2020-08-07T15:11:02.436 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:02.437 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:02.437 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:02.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.437 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73081 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:11:02.438 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:11:02.438 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 94.033546595999994, 2020-08-07T15:11:02.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0005038420000001, 2020-08-07T15:11:02.438 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:02.439 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:02.439 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:02.439 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:02.439 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:02.439 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73081 2020-08-07T15:11:02.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.440 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:02.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:02.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:11:02.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:02.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:02.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:11:02.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3016000000000003e-05 2020-08-07T15:11:02.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:02.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226609+0000", 2020-08-07T15:11:02.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4479999999999996e-06 2020-08-07T15:11:02.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:02.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226613+0000", 2020-08-07T15:11:02.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0189999999999995e-06 2020-08-07T15:11:02.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:02.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226622+0000", 2020-08-07T15:11:02.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5980000000000001e-06 2020-08-07T15:11:02.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:02.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226626+0000", 2020-08-07T15:11:02.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8088999999999999e-05 2020-08-07T15:11:02.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226684+0000", 2020-08-07T15:11:02.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2162e-05 2020-08-07T15:11:02.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:02.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226696+0000", 2020-08-07T15:11:02.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98675689 2020-08-07T15:11:02.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213453+0000", 2020-08-07T15:11:02.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012916800000000001 2020-08-07T15:11:02.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:02.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213582+0000", 2020-08-07T15:11:02.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015665899999999999 2020-08-07T15:11:02.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:11:02.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213739+0000", 2020-08-07T15:11:02.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049940239999999997 2020-08-07T15:11:02.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:02.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218733+0000", 2020-08-07T15:11:02.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0079774779999999997 2020-08-07T15:11:02.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:02.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226710+0000", 2020-08-07T15:11:02.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.161e-06 2020-08-07T15:11:02.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:02.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226711+0000", 2020-08-07T15:11:02.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0105000000000001e-05 2020-08-07T15:11:02.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:02.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226772+0000", 2020-08-07T15:11:02.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5633999999999997e-05 2020-08-07T15:11:02.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:02.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226817+0000", 2020-08-07T15:11:02.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7379000000000001e-05 2020-08-07T15:11:02.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:02.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226875+0000", 2020-08-07T15:11:02.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8000000000000005e-07 2020-08-07T15:11:02.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:02.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226875+0000", 2020-08-07T15:11:02.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019423199999999999 2020-08-07T15:11:02.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:02.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227070+0000", 2020-08-07T15:11:02.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0005038420000001 2020-08-07T15:11:02.505 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:02.505 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:02.505 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:02.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.508 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73082 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:11:02.508 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:11:02.508 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 94.033176131999994, 2020-08-07T15:11:02.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001326693, 2020-08-07T15:11:02.520 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:02.520 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:02.521 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:02.521 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:02.521 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:02.521 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73082 2020-08-07T15:11:02.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.522 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:02.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:02.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:11:02.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:02.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:02.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:11:02.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6928000000000003e-05 2020-08-07T15:11:02.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:02.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226973+0000", 2020-08-07T15:11:02.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3630000000000004e-06 2020-08-07T15:11:02.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:02.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226977+0000", 2020-08-07T15:11:02.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9659999999999996e-06 2020-08-07T15:11:02.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:02.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226985+0000", 2020-08-07T15:11:02.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5800000000000002e-06 2020-08-07T15:11:02.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:02.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226990+0000", 2020-08-07T15:11:02.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9101999999999998e-05 2020-08-07T15:11:02.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227039+0000", 2020-08-07T15:11:02.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9620000000000005e-06 2020-08-07T15:11:02.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:02.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227049+0000", 2020-08-07T15:11:02.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98722513999999995 2020-08-07T15:11:02.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214274+0000", 2020-08-07T15:11:02.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2700999999999994e-05 2020-08-07T15:11:02.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:02.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214347+0000", 2020-08-07T15:11:02.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015346899999999999 2020-08-07T15:11:02.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:11:02.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214500+0000", 2020-08-07T15:11:02.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042450309999999998 2020-08-07T15:11:02.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:02.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218745+0000", 2020-08-07T15:11:02.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008379503 2020-08-07T15:11:02.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:02.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227125+0000", 2020-08-07T15:11:02.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7899999999999998e-07 2020-08-07T15:11:02.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:02.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227126+0000", 2020-08-07T15:11:02.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1610000000000002e-05 2020-08-07T15:11:02.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:02.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227177+0000", 2020-08-07T15:11:02.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489000000000003e-05 2020-08-07T15:11:02.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:02.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227226+0000", 2020-08-07T15:11:02.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00084338600000000001 2020-08-07T15:11:02.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:02.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228069+0000", 2020-08-07T15:11:02.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7e-07 2020-08-07T15:11:02.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:02.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228069+0000", 2020-08-07T15:11:02.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019351399999999999 2020-08-07T15:11:02.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:02.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228263+0000", 2020-08-07T15:11:02.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001326693 2020-08-07T15:11:02.600 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:02.600 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:02.600 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:02.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.601 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73083 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:11:02.602 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:11:02.602 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 94.033081873, 2020-08-07T15:11:02.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015968209999999, 2020-08-07T15:11:02.602 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:02.603 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:02.603 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:02.603 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:02.604 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:02.604 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73083 2020-08-07T15:11:02.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.605 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:02.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:02.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:11:02.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:02.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:02.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:11:02.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1627000000000001e-05 2020-08-07T15:11:02.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:02.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227062+0000", 2020-08-07T15:11:02.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.253e-06 2020-08-07T15:11:02.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:02.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227065+0000", 2020-08-07T15:11:02.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3168e-05 2020-08-07T15:11:02.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:02.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227079+0000", 2020-08-07T15:11:02.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1549999999999999e-06 2020-08-07T15:11:02.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:02.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227082+0000", 2020-08-07T15:11:02.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5534999999999997e-05 2020-08-07T15:11:02.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227127+0000", 2020-08-07T15:11:02.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6823000000000001e-05 2020-08-07T15:11:02.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:02.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227144+0000", 2020-08-07T15:11:02.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98771770599999997 2020-08-07T15:11:02.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214862+0000", 2020-08-07T15:11:02.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8252999999999999e-05 2020-08-07T15:11:02.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:02.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214930+0000", 2020-08-07T15:11:02.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013646400000000001 2020-08-07T15:11:02.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:11:02.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215066+0000", 2020-08-07T15:11:02.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036907089999999999 2020-08-07T15:11:02.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:02.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218757+0000", 2020-08-07T15:11:02.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0085133450000000003 2020-08-07T15:11:02.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:02.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227270+0000", 2020-08-07T15:11:02.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4899999999999995e-07 2020-08-07T15:11:02.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:02.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227271+0000", 2020-08-07T15:11:02.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4555999999999997e-05 2020-08-07T15:11:02.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:02.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227326+0000", 2020-08-07T15:11:02.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024594800000000001 2020-08-07T15:11:02.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:02.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227572+0000", 2020-08-07T15:11:02.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00086125700000000004 2020-08-07T15:11:02.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:02.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228433+0000", 2020-08-07T15:11:02.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3799999999999998e-07 2020-08-07T15:11:02.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:02.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228433+0000", 2020-08-07T15:11:02.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019413500000000001 2020-08-07T15:11:02.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:02.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228627+0000", 2020-08-07T15:11:02.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015968209999999 2020-08-07T15:11:02.659 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:02.660 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:02.660 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:02.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.660 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73084 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:11:02.661 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:11:02.661 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 94.032992934000006, 2020-08-07T15:11:02.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0016919049999999, 2020-08-07T15:11:02.661 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:02.661 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:02.662 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:02.662 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:02.662 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:02.662 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73084 2020-08-07T15:11:02.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.663 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:02.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:02.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:11:02.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:02.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:02.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:11:02.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5565999999999997e-05 2020-08-07T15:11:02.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:02.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227165+0000", 2020-08-07T15:11:02.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2020-08-07T15:11:02.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:02.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227169+0000", 2020-08-07T15:11:02.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6850000000000001e-06 2020-08-07T15:11:02.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:02.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227176+0000", 2020-08-07T15:11:02.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1949999999999999e-06 2020-08-07T15:11:02.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:02.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227179+0000", 2020-08-07T15:11:02.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0318000000000003e-05 2020-08-07T15:11:02.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227220+0000", 2020-08-07T15:11:02.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0349e-05 2020-08-07T15:11:02.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:02.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227230+0000", 2020-08-07T15:11:02.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98832962400000002 2020-08-07T15:11:02.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215560+0000", 2020-08-07T15:11:02.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013913099999999999 2020-08-07T15:11:02.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:02.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215699+0000", 2020-08-07T15:11:02.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000200006 2020-08-07T15:11:02.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:11:02.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215899+0000", 2020-08-07T15:11:02.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028673650000000002 2020-08-07T15:11:02.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:02.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218766+0000", 2020-08-07T15:11:02.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086101779999999996 2020-08-07T15:11:02.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:02.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227376+0000", 2020-08-07T15:11:02.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0999999999999998e-07 2020-08-07T15:11:02.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:02.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227377+0000", 2020-08-07T15:11:02.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6552999999999997e-05 2020-08-07T15:11:02.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:02.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227454+0000", 2020-08-07T15:11:02.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000421717 2020-08-07T15:11:02.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:02.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227875+0000", 2020-08-07T15:11:02.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00081781300000000003 2020-08-07T15:11:02.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:02.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228693+0000", 2020-08-07T15:11:02.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1699999999999997e-07 2020-08-07T15:11:02.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:02.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228694+0000", 2020-08-07T15:11:02.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117483 2020-08-07T15:11:02.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:02.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228811+0000", 2020-08-07T15:11:02.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0016919049999999 2020-08-07T15:11:02.696 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:02.696 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:02.697 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:02.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.697 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73085 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:11:02.698 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:11:02.698 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 94.032857118999999, 2020-08-07T15:11:02.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0018015629999999, 2020-08-07T15:11:02.698 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:02.698 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:02.699 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:02.699 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:02.699 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:02.699 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73085 2020-08-07T15:11:02.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.700 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:02.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:02.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:11:02.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:02.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:02.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:11:02.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6570999999999997e-05 2020-08-07T15:11:02.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:02.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227322+0000", 2020-08-07T15:11:02.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5509999999999996e-06 2020-08-07T15:11:02.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:02.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227328+0000", 2020-08-07T15:11:02.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2498e-05 2020-08-07T15:11:02.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:02.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227341+0000", 2020-08-07T15:11:02.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4389999999999999e-06 2020-08-07T15:11:02.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:02.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227346+0000", 2020-08-07T15:11:02.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3711999999999998e-05 2020-08-07T15:11:02.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227390+0000", 2020-08-07T15:11:02.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1868999999999999e-05 2020-08-07T15:11:02.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:02.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227402+0000", 2020-08-07T15:11:02.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98896525099999999 2020-08-07T15:11:02.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216367+0000", 2020-08-07T15:11:02.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012573 2020-08-07T15:11:02.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:02.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216493+0000", 2020-08-07T15:11:02.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020525200000000001 2020-08-07T15:11:02.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:11:02.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216698+0000", 2020-08-07T15:11:02.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037596470000000001 2020-08-07T15:11:02.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:02.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.220458+0000", 2020-08-07T15:11:02.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070227190000000002 2020-08-07T15:11:02.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:02.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227480+0000", 2020-08-07T15:11:02.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6600000000000003e-07 2020-08-07T15:11:02.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:02.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227481+0000", 2020-08-07T15:11:02.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9898000000000003e-05 2020-08-07T15:11:02.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:02.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227521+0000", 2020-08-07T15:11:02.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039130000000000002 2020-08-07T15:11:02.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:02.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227912+0000", 2020-08-07T15:11:02.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00095502399999999996 2020-08-07T15:11:02.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:02.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228867+0000", 2020-08-07T15:11:02.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2020-08-07T15:11:02.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:02.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228867+0000", 2020-08-07T15:11:02.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018933599999999999 2020-08-07T15:11:02.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:02.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229057+0000", 2020-08-07T15:11:02.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0018015629999999 2020-08-07T15:11:02.722 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:02.722 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:02.723 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:02.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.723 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73086 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:11:02.723 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:11:02.724 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 94.032724622999993, 2020-08-07T15:11:02.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002364923, 2020-08-07T15:11:02.724 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:02.724 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:02.724 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:02.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:02.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:02.725 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73086 2020-08-07T15:11:02.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.725 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:02.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:02.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:11:02.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:02.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:02.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:11:02.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000238426 2020-08-07T15:11:02.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:02.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227626+0000", 2020-08-07T15:11:02.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9609999999999999e-06 2020-08-07T15:11:02.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:02.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227633+0000", 2020-08-07T15:11:02.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2727e-05 2020-08-07T15:11:02.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:02.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227646+0000", 2020-08-07T15:11:02.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1739999999999997e-06 2020-08-07T15:11:02.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:02.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227652+0000", 2020-08-07T15:11:02.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5004000000000001e-05 2020-08-07T15:11:02.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227737+0000", 2020-08-07T15:11:02.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2744e-05 2020-08-07T15:11:02.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:02.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227750+0000", 2020-08-07T15:11:02.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98938776500000003 2020-08-07T15:11:02.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217138+0000", 2020-08-07T15:11:02.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012361499999999999 2020-08-07T15:11:02.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:02.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217261+0000", 2020-08-07T15:11:02.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000212504 2020-08-07T15:11:02.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:11:02.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217474+0000", 2020-08-07T15:11:02.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002995036 2020-08-07T15:11:02.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:02.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.220469+0000", 2020-08-07T15:11:02.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071283249999999996 2020-08-07T15:11:02.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227597+0000", 2020-08-07T15:11:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0499999999999999e-07 2020-08-07T15:11:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227597+0000", 2020-08-07T15:11:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1872999999999997e-05 2020-08-07T15:11:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227639+0000", 2020-08-07T15:11:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032010200000000001 2020-08-07T15:11:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:02.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227959+0000", 2020-08-07T15:11:02.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015980899999999999 2020-08-07T15:11:02.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229557+0000", 2020-08-07T15:11:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9299999999999999e-07 2020-08-07T15:11:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229558+0000", 2020-08-07T15:11:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019477900000000001 2020-08-07T15:11:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:02.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229753+0000", 2020-08-07T15:11:02.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002364923 2020-08-07T15:11:02.757 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:02.758 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:02.758 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:02.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.759 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73087 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:11:02.759 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:11:02.759 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 94.031783469999993, 2020-08-07T15:11:02.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001896506, 2020-08-07T15:11:02.760 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:02.760 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:02.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:02.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:02.761 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:02.761 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73087 2020-08-07T15:11:02.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.761 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:02.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:02.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:11:02.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:02.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:02.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:11:02.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7354e-05 2020-08-07T15:11:02.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228386+0000", 2020-08-07T15:11:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8000000000000001e-06 2020-08-07T15:11:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228393+0000", 2020-08-07T15:11:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3784999999999999e-05 2020-08-07T15:11:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228407+0000", 2020-08-07T15:11:02.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8090000000000004e-06 2020-08-07T15:11:02.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:02.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228414+0000", 2020-08-07T15:11:02.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6907000000000003e-05 2020-08-07T15:11:02.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228461+0000", 2020-08-07T15:11:02.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8929999999999994e-06 2020-08-07T15:11:02.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228469+0000", 2020-08-07T15:11:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98950057700000005 2020-08-07T15:11:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217970+0000", 2020-08-07T15:11:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000132182 2020-08-07T15:11:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218102+0000", 2020-08-07T15:11:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020073399999999999 2020-08-07T15:11:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:11:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218303+0000", 2020-08-07T15:11:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035951120000000001 2020-08-07T15:11:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.221898+0000", 2020-08-07T15:11:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057898660000000003 2020-08-07T15:11:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227688+0000", 2020-08-07T15:11:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0200000000000002e-07 2020-08-07T15:11:02.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:02.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227688+0000", 2020-08-07T15:11:02.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1477999999999996e-05 2020-08-07T15:11:02.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:02.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227760+0000", 2020-08-07T15:11:02.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016657480000000001 2020-08-07T15:11:02.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:02.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229426+0000", 2020-08-07T15:11:02.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061290400000000005 2020-08-07T15:11:02.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230039+0000", 2020-08-07T15:11:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2e-07 2020-08-07T15:11:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230039+0000", 2020-08-07T15:11:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018643500000000001 2020-08-07T15:11:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:02.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230225+0000", 2020-08-07T15:11:02.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001896506 2020-08-07T15:11:02.838 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:02.838 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:02.838 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:02.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.839 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73088 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:11:02.839 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:11:02.839 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 94.031392780000004, 2020-08-07T15:11:02.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001627153, 2020-08-07T15:11:02.862 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:02.862 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:02.862 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:02.863 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:02.863 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:02.863 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73088 2020-08-07T15:11:02.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.863 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:02.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:02.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:11:02.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:02.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:02.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:11:02.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7741000000000003e-05 2020-08-07T15:11:02.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228767+0000", 2020-08-07T15:11:02.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6409999999999999e-06 2020-08-07T15:11:02.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:02.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228773+0000", 2020-08-07T15:11:02.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0874e-05 2020-08-07T15:11:02.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:02.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228784+0000", 2020-08-07T15:11:02.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1529999999999996e-06 2020-08-07T15:11:02.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228789+0000", 2020-08-07T15:11:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3945e-05 2020-08-07T15:11:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228833+0000", 2020-08-07T15:11:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4020000000000002e-06 2020-08-07T15:11:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:02.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228841+0000", 2020-08-07T15:11:02.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98996913900000005 2020-08-07T15:11:02.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218810+0000", 2020-08-07T15:11:02.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112484 2020-08-07T15:11:02.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:02.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218923+0000", 2020-08-07T15:11:02.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019891 2020-08-07T15:11:02.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:11:02.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.219122+0000", 2020-08-07T15:11:02.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027884099999999998 2020-08-07T15:11:02.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:02.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.221910+0000", 2020-08-07T15:11:02.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058753399999999997 2020-08-07T15:11:02.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:02.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227786+0000", 2020-08-07T15:11:02.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9200000000000002e-07 2020-08-07T15:11:02.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:02.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227786+0000", 2020-08-07T15:11:02.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8572000000000003e-05 2020-08-07T15:11:02.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227824+0000", 2020-08-07T15:11:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001651296 2020-08-07T15:11:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229476+0000", 2020-08-07T15:11:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00077620399999999996 2020-08-07T15:11:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:02.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230252+0000", 2020-08-07T15:11:02.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1600000000000002e-07 2020-08-07T15:11:02.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:02.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230252+0000", 2020-08-07T15:11:02.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4433999999999997e-05 2020-08-07T15:11:02.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:02.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230347+0000", 2020-08-07T15:11:02.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001627153 2020-08-07T15:11:02.944 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:02.944 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:02.944 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:02.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.945 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:128738 4.9 4.370c6199 (undecoded) ondisk+write+known_if_redirected e155)", 2020-08-07T15:11:02.945 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:03.204387+0000", 2020-08-07T15:11:02.945 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.055725127000002, 2020-08-07T15:11:02.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99077720899999999, 2020-08-07T15:11:02.962 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:02.963 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:02.963 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:02.963 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:02.963 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:02.963 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128738 2020-08-07T15:11:02.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.964 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:02.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:02.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204387+0000", 2020-08-07T15:11:02.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:02.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:02.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204387+0000", 2020-08-07T15:11:02.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4561999999999999e-05 2020-08-07T15:11:02.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:02.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204422+0000", 2020-08-07T15:11:02.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7270000000000001e-06 2020-08-07T15:11:02.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:02.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204428+0000", 2020-08-07T15:11:02.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3451e-05 2020-08-07T15:11:02.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:02.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204442+0000", 2020-08-07T15:11:02.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1589999999999998e-06 2020-08-07T15:11:02.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:02.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204448+0000", 2020-08-07T15:11:02.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3755999999999999e-05 2020-08-07T15:11:02.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:02.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204492+0000", 2020-08-07T15:11:02.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2450000000000001e-06 2020-08-07T15:11:02.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:02.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:02.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:03.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204499+0000", 2020-08-07T15:11:03.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99065224600000001 2020-08-07T15:11:03.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:03.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:03.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:03.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195151+0000", 2020-08-07T15:11:03.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3063e-05 2020-08-07T15:11:03.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:03.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:03.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:03.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195164+0000", 2020-08-07T15:11:03.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99077720899999999 2020-08-07T15:11:03.022 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:03.022 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:03.023 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:03.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:03.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:03.023 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:128742 4.9 4.69b70ab9 (undecoded) ondisk+write+known_if_redirected e155)", 2020-08-07T15:11:03.023 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:03.205224+0000", 2020-08-07T15:11:03.024 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.054888493, 2020-08-07T15:11:03.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.989964548, 2020-08-07T15:11:03.024 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:03.024 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:03.024 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:03.025 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:03.025 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:03.025 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128742 2020-08-07T15:11:03.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:03.025 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:03.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:03.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:03.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205224+0000", 2020-08-07T15:11:03.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:03.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:03.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:03.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:03.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205224+0000", 2020-08-07T15:11:03.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1018e-05 2020-08-07T15:11:03.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:03.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:03.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:03.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205255+0000", 2020-08-07T15:11:03.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5840000000000002e-06 2020-08-07T15:11:03.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:03.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:03.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:03.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205260+0000", 2020-08-07T15:11:03.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2388999999999999e-05 2020-08-07T15:11:03.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:03.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:03.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:03.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205273+0000", 2020-08-07T15:11:03.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3799999999999999e-06 2020-08-07T15:11:03.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:03.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:03.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:03.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205279+0000", 2020-08-07T15:11:03.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6684999999999998e-05 2020-08-07T15:11:03.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:03.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:03.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:03.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205326+0000", 2020-08-07T15:11:03.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6200000000000004e-06 2020-08-07T15:11:03.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:03.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:03.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:03.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205332+0000", 2020-08-07T15:11:03.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98984944600000002 2020-08-07T15:11:03.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:03.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:03.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:03.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195181+0000", 2020-08-07T15:11:03.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4259999999999997e-06 2020-08-07T15:11:03.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:03.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:03.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:03.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195188+0000", 2020-08-07T15:11:03.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.989964548 2020-08-07T15:11:03.115 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:03.115 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:03.115 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:03.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:03.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:03.116 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:128743 4.9 4.1824cba9 (undecoded) ondisk+write+known_if_redirected e155)", 2020-08-07T15:11:03.116 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:03.205500+0000", 2020-08-07T15:11:03.116 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.054612630999998, 2020-08-07T15:11:03.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98971761999999996, 2020-08-07T15:11:03.117 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:03.117 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:03.117 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:03.126 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:03.127 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:03.127 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128743 2020-08-07T15:11:03.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:03.128 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:03.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:03.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:03.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205500+0000", 2020-08-07T15:11:03.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:03.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:03.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:03.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:03.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205500+0000", 2020-08-07T15:11:03.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6441000000000001e-05 2020-08-07T15:11:03.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:03.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:03.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:03.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205516+0000", 2020-08-07T15:11:03.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4429999999999999e-06 2020-08-07T15:11:03.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:03.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:03.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:03.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205520+0000", 2020-08-07T15:11:03.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4590000000000003e-06 2020-08-07T15:11:03.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:03.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:03.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:03.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205526+0000", 2020-08-07T15:11:03.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0429999999999999e-06 2020-08-07T15:11:03.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:03.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:03.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:03.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205529+0000", 2020-08-07T15:11:03.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1313000000000003e-05 2020-08-07T15:11:03.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:03.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:03.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:03.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205560+0000", 2020-08-07T15:11:03.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.934e-06 2020-08-07T15:11:03.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:03.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:03.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:03.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205567+0000", 2020-08-07T15:11:03.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98964017599999998 2020-08-07T15:11:03.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:03.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:03.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:03.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195208+0000", 2020-08-07T15:11:03.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8109999999999997e-06 2020-08-07T15:11:03.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:03.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:03.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:03.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195217+0000", 2020-08-07T15:11:03.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98971761999999996 2020-08-07T15:11:03.146 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:03.146 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:03.146 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:03.146 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:03.147 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:03.147 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:11:03.148 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:11:03.194 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:11:03.194 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 42 16 63059 63043 11.7255 13.5156 0.0112946 0.0106557 2020-08-07T15:11:03.195 INFO:teuthology.orchestra.run.smithi074.stdout: marking collection for removal 2020-08-07T15:11:03.195 INFO:teuthology.orchestra.run.smithi074.stdout:setting '_remove' omap key 2020-08-07T15:11:03.195 INFO:teuthology.orchestra.run.smithi074.stdout:finish_remove_pgs 2.2_head removing 2.2 2020-08-07T15:11:03.195 INFO:teuthology.orchestra.run.smithi074.stdout:Remove successful 2020-08-07T15:11:03.196 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:11:03.197 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:03.147+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:03.223 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 43 16 64670 64654 11.7455 12.5859 0.00515701 0.010631 2020-08-07T15:11:03.265 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.2.2.1 2020-08-07T15:11:03.278 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:03.272+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:03.354 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:03.352+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:03.392 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:11:03.432 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:11:03.474 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:03.453+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:03.487 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:11:03.516 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:11:03.552 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:11:03.558 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:03.554+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:03.592 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:11:03.629 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:11:03.630 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:11:03.630 INFO:teuthology.orchestra.run.smithi074.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-08-07T15:11:03.630 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: problem getting command descriptions from osd.1 2020-08-07T15:11:03.630 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:11:03.630 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:11:03.631 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:11:03.631 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:11:03.631 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:11:03.631 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:11:03.635 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-08-07T15:11:03.636 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-08-07T15:11:03.661 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:03.658+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:03.767 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:03.764+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:03.868 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:03.867+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:03.970 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:03.968+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:04.071 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:04.069+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:04.177 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:04.176+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:04.222 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 44 16 66162 66146 11.7434 11.6562 0.00765994 0.0106388 2020-08-07T15:11:04.286 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:04.278+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:04.382 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:04.379+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:04.491 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:04.481+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:04.586 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:04.585+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:04.690 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:04.686+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:04.795 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:04.788+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:04.894 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:04.890+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:05.000 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:04.999+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:05.111 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:05.110+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:05.195 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:11:05.215 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 45 16 67424 67408 11.7016 9.85938 0.0116507 0.0106777 2020-08-07T15:11:05.216 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:05.213+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:05.234 INFO:teuthology.orchestra.run.smithi074.stdout:get_pg_num_history pg_num_history pg_num_history(e190 pg_nums {1={89=21,186=31},2={17=8},4={81=16},5={164=16}} deleted_pools 163,4) 2020-08-07T15:11:05.235 INFO:teuthology.orchestra.run.smithi074.stdout:Importing pgid 2.2 2020-08-07T15:11:05.236 INFO:teuthology.orchestra.run.smithi074.stdout:write_pg epoch 150 info 2.2( empty local-lis/les=17/18 n=0 ec=17/17 lis/c=17/17 les/c/f=18/18/0 sis=17) 2020-08-07T15:11:05.237 INFO:teuthology.orchestra.run.smithi074.stdout:Import successful 2020-08-07T15:11:05.326 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:05.312+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:05.415 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:05.413+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:05.430 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:05.430 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:05.518 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:05.517+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:05.619 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:05.618+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:05.646 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:11:05.648 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:11:05.776 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:05.723+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:05.778 INFO:teuthology.orchestra.run.smithi074:> sudo rm -f /var/log/ceph/exp.2.2.1 2020-08-07T15:11:05.813 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:11:05.820 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json 2020-08-07T15:11:05.852 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:11:05.853 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:05.850+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:05.890 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:11:05.921 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:11:05.948 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:05.941+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:05.958 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:11:05.999 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:11:06.041 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:11:06.042 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:11:06.043 INFO:teuthology.orchestra.run.smithi074.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-08-07T15:11:06.043 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: problem getting command descriptions from osd.1 2020-08-07T15:11:06.043 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:11:06.043 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:11:06.044 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:11:06.044 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:11:06.044 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:11:06.044 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:11:06.045 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-08-07T15:11:06.059 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-08-07T15:11:06.068 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:06.044+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:06.149 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:06.146+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:06.214 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 46 16 68669 68653 11.6586 9.72656 0.00952169 0.0107066 2020-08-07T15:11:06.240 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:11:06.241 INFO:teuthology.orchestra.run.smithi074.stdout:{"epoch":197,"fsid":"a01d3e82-8d24-4288-9e42-576ab3ff4957","created":"2020-08-07T15:06:07.342912+0000","modified":"2020-08-07T15:11:01.676915+0000","last_up_change":"2020-08-07T15:10:50.524683+0000","last_in_change":"2020-08-07T15:06:17.674292+0000","flags":"noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5806080,"flags_set":["nodeep-scrub","noscrub","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-08-07T15:06:10.648876+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":31,"pg_placement_num":31,"pg_placement_num_target":31,"pg_num_target":31,"pg_num_pending":31,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"186","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-08-07T15:06:21.474648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"unique_pool_2","create_time":"2020-08-07T15:10:14.379166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}}}],"osds":[{"osd":0,"uuid":"fc1ebb23-3ef4-4fc0-b395-382e385885a3","up":1,"in":1,"weight":1,"primary_affinity":0,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6801","nonce":31577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6802","nonce":31577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6804","nonce":31577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6803","nonce":31577}]},"public_addr":"172.21.15.74:6801/31577","cluster_addr":"172.21.15.74:6802/31577","heartbeat_back_addr":"172.21.15.74:6804/31577","heartbeat_front_addr":"172.21.15.74:6803/31577","state":["exists","up"]},{"osd":1,"uuid":"1fbe530a-63ce-4dc8-82ab-18793d6ebc0b","up":0,"in":1,"weight":1,"primary_affinity":0.7196044921875,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":191,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6809","nonce":31578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6810","nonce":31578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6812","nonce":31578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6811","nonce":31578}]},"public_addr":"172.21.15.74:6809/31578","cluster_addr":"172.21.15.74:6810/31578","heartbeat_back_addr":"172.21.15.74:6812/31578","heartbeat_front_addr":"172.21.15.74:6811/31578","state":["exists"]},{"osd":2,"uuid":"d6fe4338-3e11-4f73-9cbb-e1b046ce286a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6805","nonce":31576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6806","nonce":31576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6808","nonce":31576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6807","nonce":31576}]},"public_addr":"172.21.15.74:6805/31576","cluster_addr":"172.21.15.74:6806/31576","heartbeat_back_addr":"172.21.15.74:6808/31576","heartbeat_front_addr":"172.21.15.74:6807/31576","state":["exists","up"]},{"osd":3,"uuid":"bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6813","nonce":31579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6814","nonce":31579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6816","nonce":31579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.74:6815","nonce":31579}]},"public_addr":"172.21.15.74:6813/31579","cluster_addr":"172.21.15.74:6814/31579","heartbeat_back_addr":"172.21.15.74:6816/31579","heartbeat_front_addr":"172.21.15.74:6815/31579","state":["exists","up"]},{"osd":4,"uuid":"01aadc13-29c5-4ddd-9bb9-de24b2b20412","up":1,"in":1,"weight":1,"primary_affinity":0,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":31545}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6801","nonce":31545}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6803","nonce":31545}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":31545}]},"public_addr":"172.21.15.96:6800/31545","cluster_addr":"172.21.15.96:6801/31545","heartbeat_back_addr":"172.21.15.96:6803/31545","heartbeat_front_addr":"172.21.15.96:6802/31545","state":["exists","up"]},{"osd":5,"uuid":"f5556a82-73b4-47b1-8e29-1d2fe031d8fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":31547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6805","nonce":31547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6807","nonce":31547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":31547}]},"public_addr":"172.21.15.96:6804/31547","cluster_addr":"172.21.15.96:6805/31547","heartbeat_back_addr":"172.21.15.96:6807/31547","heartbeat_front_addr":"172.21.15.96:6806/31547","state":["exists","up"]},{"osd":6,"uuid":"9b98b337-e1cc-43cd-9e9b-f21ae3f20899","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":31546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6809","nonce":31546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6811","nonce":31546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":31546}]},"public_addr":"172.21.15.96:6808/31546","cluster_addr":"172.21.15.96:6809/31546","heartbeat_back_addr":"172.21.15.96:6811/31546","heartbeat_front_addr":"172.21.15.96:6810/31546","state":["exists","up"]},{"osd":7,"uuid":"777662d8-bd34-48e8-83e1-24ce9e61f14b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":192,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":31570}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6813","nonce":31570}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6815","nonce":31570}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":31570}]},"public_addr":"172.21.15.96:6812/31570","cluster_addr":"172.21.15.96:6813/31570","heartbeat_back_addr":"172.21.15.96:6815/31570","heartbeat_front_addr":"172.21.15.96:6814/31570","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:10:17.548342+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:10:16.729479+0000","dead_epoch":191},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:10:17.020686+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:10:18.633656+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:10:17.840360+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:10:17.893395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:10:17.639810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292837744639,"old_weight":0,"last_purged_snaps_scrub":"2020-08-07T15:10:16.305143+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[{"pgid":"1.15","osds":[7,5,2]},{"pgid":"1.16","osds":[3,0,6]},{"pgid":"1.1a","osds":[7,4,2]},{"pgid":"1.1c","osds":[7,6,0]},{"pgid":"1.1d","osds":[3,4,7]}],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2020-08-07T15:11:06.258 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:06.248+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:06.259 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-08-07T15:06:10.648876+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 31, 'pg_placement_num': 31, 'pg_placement_num_target': 31, 'pg_num_target': 31, 'pg_num_pending': 31, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '188', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '186', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-08-07T15:06:21.474648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'unique_pool_2', 'create_time': '2020-08-07T15:10:14.379166+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '166', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}] 2020-08-07T15:11:06.260 INFO:teuthology.orchestra.run.smithi074:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --op apply-layout-settings --pool device_health_metrics 2020-08-07T15:11:06.271 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:11:06.372 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:06.358+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:06.462 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:06.461+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:06.575 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:06.562+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:06.667 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:06.665+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:06.784 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:06.781+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:06.884 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:06.882+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:06.985 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:06.984+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:07.088 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:07.084+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:07.220 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:07.214+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:07.220 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 47 16 70197 70181 11.6645 11.9375 0.004166 0.0107116 2020-08-07T15:11:07.320 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:07.318+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:07.426 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:07.422+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:07.525 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:07.522+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:07.630 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:07.629+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:07.918 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:07.911+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:08.018 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:08.014+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:08.063 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:11:08.116 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:08.114+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:08.176 INFO:teuthology.orchestra.run.smithi074.stderr:Nothing to do for non-filestore backend 2020-08-07T15:11:08.219 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 48 16 71735 71719 11.6718 12.0156 0.00657434 0.0107049 2020-08-07T15:11:08.225 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:08.218+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:08.267 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:11:08.298 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:11:08.306 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:11:08.325 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:08.319+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:08.346 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:11:08.385 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:11:08.430 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:11:08.431 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:08.429+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:08.464 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:11:08.518 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:11:08.518 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:11:08.518 INFO:teuthology.orchestra.run.smithi074.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-08-07T15:11:08.519 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: problem getting command descriptions from osd.1 2020-08-07T15:11:08.519 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:11:08.519 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:11:08.527 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:11:08.527 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:11:08.527 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:11:08.527 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:11:08.528 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-08-07T15:11:08.528 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-08-07T15:11:08.553 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:08.552+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:08.656 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:08.651+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:08.699 INFO:teuthology.orchestra.run.smithi074:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --op apply-layout-settings --pool rbd 2020-08-07T15:11:08.757 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:08.755+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:08.885 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:08.857+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:08.960 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:08.958+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:09.064 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:09.062+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:09.166 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:09.164+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:09.216 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 49 16 73307 73291 11.6842 12.2812 0.00705919 0.0106946 2020-08-07T15:11:09.268 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:09.266+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:09.283 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:11:09.368 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:09.367+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:09.468 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:09.468+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:09.570 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:09.569+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:09.671 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:09.670+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:09.771 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:09.770+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:09.872 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:09.871+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:09.987 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:09.974+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:10.081 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:10.076+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:10.178 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:10.177+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:10.215 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 50 16 74860 74844 11.6932 12.1328 0.0626414 0.0106836 2020-08-07T15:11:10.284 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:10.283+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:10.395 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:10.394+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:10.498 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:10.498+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:10.532 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:11:10.602 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:10.600+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:10.641 INFO:teuthology.orchestra.run.smithi074.stderr:Nothing to do for non-filestore backend 2020-08-07T15:11:10.703 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:10.701+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:10.742 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:11:10.782 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:11:10.803 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:10.802+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:10.812 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:11:10.871 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:11:10.897 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:11:10.903 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:10.903+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:10.932 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:11:10.959 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:11:10.960 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:11:10.960 INFO:teuthology.orchestra.run.smithi074.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-08-07T15:11:10.960 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: problem getting command descriptions from osd.1 2020-08-07T15:11:10.960 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:11:10.960 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:11:10.961 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:11:10.961 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:11:10.961 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:11:10.961 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:11:10.963 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-08-07T15:11:10.964 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-08-07T15:11:11.029 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:11.028+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:11.130 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:11.130+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:11.161 INFO:teuthology.orchestra.run.smithi074:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --op apply-layout-settings --pool unique_pool_2 2020-08-07T15:11:11.215 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 51 16 76468 76452 11.7102 12.5625 0.00608045 0.0106701 2020-08-07T15:11:11.236 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:11.231+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:11.302 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:11:11.334 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:11.334+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:11.452 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:11.435+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:11.538 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:11.537+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:11.638 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:11.638+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:11.764 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:11.763+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:11.878 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:11.866+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:11.974 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:11.967+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:12.071 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:12.067+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:12.179 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:12.171+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:12.209 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:11:12.223 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 52 16 78105 78089 11.7309 12.7891 0.00927872 0.0106503 2020-08-07T15:11:12.278 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:12.274+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:12.377 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:12.376+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:12.477 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:12.475+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:12.601 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:12.599+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:12.724 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:12.719+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:12.823 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:12.822+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:12.929 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:12.923+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:12.969 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:11:13.029 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:13.025+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:13.070 INFO:teuthology.orchestra.run.smithi074.stderr:Nothing to do for non-filestore backend 2020-08-07T15:11:13.131 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:13.128+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:13.177 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:11:13.211 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:11:13.218 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 53 16 79680 79664 11.7417 12.3047 0.00222574 0.0106411 2020-08-07T15:11:13.233 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:13.229+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:13.246 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:11:13.284 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:11:13.318 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:11:13.338 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:13.333+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:13.354 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:11:13.387 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:11:13.387 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:11:13.387 INFO:teuthology.orchestra.run.smithi074.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-08-07T15:11:13.388 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: problem getting command descriptions from osd.1 2020-08-07T15:11:13.388 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:11:13.388 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:11:13.388 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:11:13.388 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:11:13.389 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:11:13.389 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:11:13.389 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-08-07T15:11:13.389 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-08-07T15:11:13.441 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:13.437+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:13.550 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:13.543+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:13.645 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:13.643+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:13.746 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:13.745+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:13.854 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:13.852+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:13.955 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:13.954+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:14.057 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:14.056+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:14.158 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:14.156+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:14.218 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 54 15 81328 81313 11.7629 12.8828 0.00888233 0.0106181 2020-08-07T15:11:14.230 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:11:14.283 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:14.279+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:14.359 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:14.359+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:14.462 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:14.459+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:14.565 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:14.563+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:14.617 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:14.617 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:14.666 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:14.664+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:14.738 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:11:14.767 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:14.766+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:14.870 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:14.867+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:14.974 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:14.970+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:15.081 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:15.072+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:15.179 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:15.178+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:15.215 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 55 16 83024 83008 11.7897 13.2422 0.0104568 0.0105968 2020-08-07T15:11:15.285 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:15.280+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:15.333 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:11:15.392 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:11:15.431 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:15.392+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:15.494 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:15.493+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:15.571 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:11:15.595 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:15.594+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:15.604 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:11:15.631 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:11:15.662 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:11:15.698 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:11:15.698 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:15.697+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:15.701 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:15.702 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:15.723 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:11:15.734 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:11:15.767 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:11:15.767 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:11:15.768 INFO:teuthology.orchestra.run.smithi074.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-08-07T15:11:15.768 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: problem getting command descriptions from osd.1 2020-08-07T15:11:15.768 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:11:15.768 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:11:15.768 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:11:15.769 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:11:15.769 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:11:15.769 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:11:15.775 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-08-07T15:11:15.775 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-08-07T15:11:15.819 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:15.816+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:15.900 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:15.898+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:16.005 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:16.002+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:16.058 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:11:16.058 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:11:16.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.059 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:84205 5.9 5:933e5a80:::benchmark_data_smithi074_53697_object10525:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e202)", 2020-08-07T15:11:16.059 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.031441+0000", 2020-08-07T15:11:16.059 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.023574649999999999, 2020-08-07T15:11:16.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.02360435, 2020-08-07T15:11:16.060 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:16.060 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:16.060 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:16.060 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:11:16.060 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:16.061 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 84205 2020-08-07T15:11:16.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.061 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:16.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:16.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.031441+0000", 2020-08-07T15:11:16.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:16.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:16.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.031441+0000", 2020-08-07T15:11:16.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9118e-05 2020-08-07T15:11:16.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:16.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.031490+0000", 2020-08-07T15:11:16.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4670000000000001e-06 2020-08-07T15:11:16.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:16.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.031496+0000", 2020-08-07T15:11:16.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2571999999999999e-05 2020-08-07T15:11:16.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:16.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.031509+0000", 2020-08-07T15:11:16.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1859999999999997e-06 2020-08-07T15:11:16.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:16.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.031515+0000", 2020-08-07T15:11:16.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015233289999999999 2020-08-07T15:11:16.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:16.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.033038+0000", 2020-08-07T15:11:16.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0797000000000006e-05 2020-08-07T15:11:16.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:16.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.033119+0000", 2020-08-07T15:11:16.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5686999999999994e-05 2020-08-07T15:11:16.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:11:16.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.033215+0000", 2020-08-07T15:11:16.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017535928999999999 2020-08-07T15:11:16.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:16.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.050751+0000", 2020-08-07T15:11:16.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.85e-07 2020-08-07T15:11:16.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:16.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.050751+0000", 2020-08-07T15:11:16.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9703000000000001e-05 2020-08-07T15:11:16.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:16.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.050811+0000", 2020-08-07T15:11:16.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00066587000000000005 2020-08-07T15:11:16.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:16.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.051477+0000", 2020-08-07T15:11:16.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.020036443000000001 2020-08-07T15:11:16.080 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:16.081 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:16.081 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:16.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.081 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:84206 5.9 5:933e5a80:::benchmark_data_smithi074_53697_object10525:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e202)", 2020-08-07T15:11:16.082 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.031553+0000", 2020-08-07T15:11:16.082 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.023462426000000002, 2020-08-07T15:11:16.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023620173000000001, 2020-08-07T15:11:16.082 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:16.082 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:16.083 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:16.083 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:11:16.093 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:16.094 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 84206 2020-08-07T15:11:16.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.094 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:16.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:16.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.031553+0000", 2020-08-07T15:11:16.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:16.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:16.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.031553+0000", 2020-08-07T15:11:16.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0862999999999997e-05 2020-08-07T15:11:16.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:16.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.031604+0000", 2020-08-07T15:11:16.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3369999999999999e-06 2020-08-07T15:11:16.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:16.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.031609+0000", 2020-08-07T15:11:16.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0406e-05 2020-08-07T15:11:16.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:16.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.031619+0000", 2020-08-07T15:11:16.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7099999999999998e-06 2020-08-07T15:11:16.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:16.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.031624+0000", 2020-08-07T15:11:16.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00186579 2020-08-07T15:11:16.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:16.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.033490+0000", 2020-08-07T15:11:16.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6386999999999998e-05 2020-08-07T15:11:16.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:16.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.033576+0000", 2020-08-07T15:11:16.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014741099999999999 2020-08-07T15:11:16.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:11:16.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.033724+0000", 2020-08-07T15:11:16.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017180312999999999 2020-08-07T15:11:16.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:16.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.050904+0000", 2020-08-07T15:11:16.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7700000000000003e-07 2020-08-07T15:11:16.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:16.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.050905+0000", 2020-08-07T15:11:16.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0093000000000002e-05 2020-08-07T15:11:16.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:16.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.050965+0000", 2020-08-07T15:11:16.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00054721399999999997 2020-08-07T15:11:16.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:16.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.051512+0000", 2020-08-07T15:11:16.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019959401000000002 2020-08-07T15:11:16.166 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:16.166 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:16.167 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:16.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.167 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:84207 5.9 5:933e5a80:::benchmark_data_smithi074_53697_object10525:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e202)", 2020-08-07T15:11:16.168 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.031662+0000", 2020-08-07T15:11:16.168 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.023353071, 2020-08-07T15:11:16.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023625119999999999, 2020-08-07T15:11:16.169 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:16.170 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:16.170 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:16.170 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:11:16.171 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:16.171 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 84207 2020-08-07T15:11:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.031662+0000", 2020-08-07T15:11:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.031662+0000", 2020-08-07T15:11:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8004000000000003e-05 2020-08-07T15:11:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.031710+0000", 2020-08-07T15:11:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3489999999999999e-06 2020-08-07T15:11:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.031716+0000", 2020-08-07T15:11:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0049e-05 2020-08-07T15:11:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:16.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.031726+0000", 2020-08-07T15:11:16.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7779999999999998e-06 2020-08-07T15:11:16.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:16.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.031730+0000", 2020-08-07T15:11:16.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024516780000000001 2020-08-07T15:11:16.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:16.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.034182+0000", 2020-08-07T15:11:16.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6317000000000003e-05 2020-08-07T15:11:16.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:16.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.034238+0000", 2020-08-07T15:11:16.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113035 2020-08-07T15:11:16.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:11:16.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.034351+0000", 2020-08-07T15:11:16.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016744677999999999 2020-08-07T15:11:16.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:16.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.051096+0000", 2020-08-07T15:11:16.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5700000000000002e-07 2020-08-07T15:11:16.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:16.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.051097+0000", 2020-08-07T15:11:16.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7049999999999998e-05 2020-08-07T15:11:16.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:16.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.051154+0000", 2020-08-07T15:11:16.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00078304399999999999 2020-08-07T15:11:16.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:16.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.051937+0000", 2020-08-07T15:11:16.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.020274739 2020-08-07T15:11:16.218 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:16.218 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:16.218 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:16.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.219 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:84208 5.9 5:933e5a80:::benchmark_data_smithi074_53697_object10525:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e202)", 2020-08-07T15:11:16.219 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.034341+0000", 2020-08-07T15:11:16.219 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.02067393, 2020-08-07T15:11:16.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021031943000000001, 2020-08-07T15:11:16.220 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:16.220 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:16.220 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:16.220 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:11:16.221 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:16.221 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 84208 2020-08-07T15:11:16.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.222 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:16.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:16.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.034341+0000", 2020-08-07T15:11:16.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:16.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:16.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.034341+0000", 2020-08-07T15:11:16.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0033e-05 2020-08-07T15:11:16.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:16.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.034391+0000", 2020-08-07T15:11:16.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2020-08-07T15:11:16.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:16.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.034398+0000", 2020-08-07T15:11:16.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1110999999999999e-05 2020-08-07T15:11:16.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:16.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.034409+0000", 2020-08-07T15:11:16.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7289999999999997e-06 2020-08-07T15:11:16.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:16.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.034415+0000", 2020-08-07T15:11:16.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015311504 2020-08-07T15:11:16.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:16.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.049726+0000", 2020-08-07T15:11:16.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9840000000000006e-05 2020-08-07T15:11:16.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:16.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.049826+0000", 2020-08-07T15:11:16.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000171273 2020-08-07T15:11:16.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:11:16.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.049997+0000", 2020-08-07T15:11:16.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020408430000000001 2020-08-07T15:11:16.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:16.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.052038+0000", 2020-08-07T15:11:16.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0160000000000001e-06 2020-08-07T15:11:16.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:16.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.052039+0000", 2020-08-07T15:11:16.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4523000000000005e-05 2020-08-07T15:11:16.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:16.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.052114+0000", 2020-08-07T15:11:16.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010745500000000001 2020-08-07T15:11:16.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:16.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.053188+0000", 2020-08-07T15:11:16.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018846910000000001 2020-08-07T15:11:16.280 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:16.281 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:16.281 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:16.281 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:16.288 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:11:16.288 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 4 2020-08-07T15:11:16.289 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:11:16.291 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 56 16 84516 84500 11.7873 11.6562 0.00656602 0.0106005 2020-08-07T15:11:16.291 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:16.289+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:16.292 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:11:16.396 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:16.394+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:16.495 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:16.494+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:16.610 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:16.608+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:16.698 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:11:16.698 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:11:16.699 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:11:16.699 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:11:16.700 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:11:16.706 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:11:16.711 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:16.710+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:16.812 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:16.810+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:16.917 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:16.915+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:16.938 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:11:16.938 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:11:16.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:11:16.939 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:11:16.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.941 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15576 3.e 3:727724a5:::benchmark_data_smithi074_35447_object1946:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:11:16.941 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:11:16.941 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 270.26381402700002, 2020-08-07T15:11:16.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999, 2020-08-07T15:11:16.948 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:16.948 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:16.948 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:16.948 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:11:16.948 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:11:16.949 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15576 2020-08-07T15:11:16.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.949 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:16.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:16.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:11:16.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:16.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:16.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:11:16.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7156e-05 2020-08-07T15:11:16.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:16.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666682+0000", 2020-08-07T15:11:16.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0209999999999999e-06 2020-08-07T15:11:16.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:16.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666687+0000", 2020-08-07T15:11:16.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0730999999999999e-05 2020-08-07T15:11:16.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:16.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666698+0000", 2020-08-07T15:11:16.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2020-08-07T15:11:16.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:16.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666702+0000", 2020-08-07T15:11:16.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023978459999999999 2020-08-07T15:11:16.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:16.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669100+0000", 2020-08-07T15:11:16.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9664000000000001e-05 2020-08-07T15:11:16.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:16.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669190+0000", 2020-08-07T15:11:16.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015957100000000001 2020-08-07T15:11:16.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:11:16.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669350+0000", 2020-08-07T15:11:16.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016470149999999999 2020-08-07T15:11:16.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:16.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:11:16.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1299999999999999e-07 2020-08-07T15:11:16.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:16.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:11:16.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5269000000000002e-05 2020-08-07T15:11:16.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:16.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.671022+0000", 2020-08-07T15:11:16.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031414400000000001 2020-08-07T15:11:16.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:16.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.674163+0000", 2020-08-07T15:11:16.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45641417400000001 2020-08-07T15:11:16.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:16.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:11:16.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0399999999999995e-07 2020-08-07T15:11:16.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:16.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:11:16.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3343999999999997e-05 2020-08-07T15:11:16.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:16.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130622+0000", 2020-08-07T15:11:16.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999 2020-08-07T15:11:16.986 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:16.986 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:16.986 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:16.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.987 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:41432 3.d 3:b5c8a160:::benchmark_data_smithi074_35447_object5178:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e32)", 2020-08-07T15:11:16.987 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:11:16.988 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 254.13815880199999, 2020-08-07T15:11:16.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.37720701699999998, 2020-08-07T15:11:16.988 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:16.988 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:16.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:16.989 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:11:16.989 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:11:16.989 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 41432 2020-08-07T15:11:16.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.989 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:16.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:16.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:11:16.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:16.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:16.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792290+0000", 2020-08-07T15:11:16.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4787999999999999e-05 2020-08-07T15:11:16.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:16.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792335+0000", 2020-08-07T15:11:16.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5999999999999997e-06 2020-08-07T15:11:16.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:16.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792341+0000", 2020-08-07T15:11:16.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0998e-05 2020-08-07T15:11:16.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:16.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792352+0000", 2020-08-07T15:11:16.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2989999999999996e-06 2020-08-07T15:11:16.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:16.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.792357+0000", 2020-08-07T15:11:16.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001852752 2020-08-07T15:11:16.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:16.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:16.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:16.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794210+0000", 2020-08-07T15:11:16.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5771000000000002e-05 2020-08-07T15:11:17.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:17.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794305+0000", 2020-08-07T15:11:17.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012776399999999999 2020-08-07T15:11:17.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:11:17.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.794433+0000", 2020-08-07T15:11:17.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00069673599999999999 2020-08-07T15:11:17.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:17.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795130+0000", 2020-08-07T15:11:17.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1500000000000001e-07 2020-08-07T15:11:17.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:17.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795130+0000", 2020-08-07T15:11:17.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7319e-05 2020-08-07T15:11:17.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:17.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:02.795167+0000", 2020-08-07T15:11:17.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27910026700000001 2020-08-07T15:11:17.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:17.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.074268+0000", 2020-08-07T15:11:17.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.095150660999999997 2020-08-07T15:11:17.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:17.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169418+0000", 2020-08-07T15:11:17.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.423e-06 2020-08-07T15:11:17.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:17.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169420+0000", 2020-08-07T15:11:17.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7423999999999996e-05 2020-08-07T15:11:17.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:17.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:03.169497+0000", 2020-08-07T15:11:17.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.37720701699999998 2020-08-07T15:11:17.016 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.016 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:17.016 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.017 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:104104 3.8 e58/20 3:11c276ea:::benchmark_data_smithi074_35447_object13012:head v 58'6496, mlcod=58'6487)", 2020-08-07T15:11:17.017 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:11:17.017 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 211.071963281, 2020-08-07T15:11:17.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33178497099999998, 2020-08-07T15:11:17.017 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:17.018 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:17.018 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:17.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:17.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:11:17.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:17.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:17.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858486+0000", 2020-08-07T15:11:17.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9961999999999999e-05 2020-08-07T15:11:17.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:17.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858516+0000", 2020-08-07T15:11:17.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3200000000000001e-06 2020-08-07T15:11:17.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:17.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:45.858520+0000", 2020-08-07T15:11:17.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32756790000000002 2020-08-07T15:11:17.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:17.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186088+0000", 2020-08-07T15:11:17.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0260000000000004e-06 2020-08-07T15:11:17.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:17.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186094+0000", 2020-08-07T15:11:17.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7919000000000001e-05 2020-08-07T15:11:17.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:17.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186132+0000", 2020-08-07T15:11:17.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4968e-05 2020-08-07T15:11:17.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:17.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.186147+0000", 2020-08-07T15:11:17.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040671520000000001 2020-08-07T15:11:17.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:17.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.190214+0000", 2020-08-07T15:11:17.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6724000000000001e-05 2020-08-07T15:11:17.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:17.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:46.190271+0000", 2020-08-07T15:11:17.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33178497099999998 2020-08-07T15:11:17.041 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.041 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:17.041 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.042 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106248 3.f 3:f950fd9d:::benchmark_data_smithi074_35447_object13280:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:11:17.042 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:11:17.042 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 209.57028448700001, 2020-08-07T15:11:17.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002, 2020-08-07T15:11:17.043 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:17.043 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:17.043 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:17.051 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:11:17.051 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:11:17.051 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106248 2020-08-07T15:11:17.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.052 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:17.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:17.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:11:17.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:17.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:17.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:11:17.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7184e-05 2020-08-07T15:11:17.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:17.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360222+0000", 2020-08-07T15:11:17.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0906e-05 2020-08-07T15:11:17.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:17.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360233+0000", 2020-08-07T15:11:17.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2972999999999999e-05 2020-08-07T15:11:17.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:17.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360246+0000", 2020-08-07T15:11:17.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0114e-05 2020-08-07T15:11:17.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:17.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360256+0000", 2020-08-07T15:11:17.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1303999999999997e-05 2020-08-07T15:11:17.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:17.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360287+0000", 2020-08-07T15:11:17.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3798000000000001e-05 2020-08-07T15:11:17.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:17.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360351+0000", 2020-08-07T15:11:17.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111308 2020-08-07T15:11:17.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:11:17.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360462+0000", 2020-08-07T15:11:17.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003480674 2020-08-07T15:11:17.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:17.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:11:17.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0900000000000001e-07 2020-08-07T15:11:17.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:17.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:11:17.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8899999999999994e-05 2020-08-07T15:11:17.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:17.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.364012+0000", 2020-08-07T15:11:17.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35291637199999998 2020-08-07T15:11:17.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:17.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716929+0000", 2020-08-07T15:11:17.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.076244233999999994 2020-08-07T15:11:17.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:17.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793173+0000", 2020-08-07T15:11:17.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6109999999999999e-06 2020-08-07T15:11:17.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:17.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793175+0000", 2020-08-07T15:11:17.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7492999999999999e-05 2020-08-07T15:11:17.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:17.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793242+0000", 2020-08-07T15:11:17.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002 2020-08-07T15:11:17.105 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.105 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:17.105 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.106 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106248 3.f e59/20 ondisk, result = 0)", 2020-08-07T15:11:17.106 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.365323+0000", 2020-08-07T15:11:17.106 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 209.56512593100001, 2020-08-07T15:11:17.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35160734999999999, 2020-08-07T15:11:17.107 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:17.107 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:11:17.107 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:17.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:17.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365323+0000", 2020-08-07T15:11:17.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:17.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:17.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365323+0000", 2020-08-07T15:11:17.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4981000000000003e-05 2020-08-07T15:11:17.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:17.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365358+0000", 2020-08-07T15:11:17.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1090000000000003e-06 2020-08-07T15:11:17.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:17.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365364+0000", 2020-08-07T15:11:17.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35151899399999997 2020-08-07T15:11:17.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:17.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716883+0000", 2020-08-07T15:11:17.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0090000000000001e-06 2020-08-07T15:11:17.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:17.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716887+0000", 2020-08-07T15:11:17.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9238999999999999e-05 2020-08-07T15:11:17.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:17.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716916+0000", 2020-08-07T15:11:17.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0390000000000001e-06 2020-08-07T15:11:17.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:17.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716922+0000", 2020-08-07T15:11:17.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9789999999999993e-06 2020-08-07T15:11:17.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:17.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716930+0000", 2020-08-07T15:11:17.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35160734999999999 2020-08-07T15:11:17.120 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.120 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:17.120 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.120 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106248 3.f e59/20 ondisk, result = 0)", 2020-08-07T15:11:17.121 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:11:17.121 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 209.565021202, 2020-08-07T15:11:17.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001, 2020-08-07T15:11:17.121 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:17.123 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:11:17.123 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:17.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:17.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:11:17.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:17.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:17.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:11:17.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7518999999999999e-05 2020-08-07T15:11:17.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:17.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365445+0000", 2020-08-07T15:11:17.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-06 2020-08-07T15:11:17.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:17.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365449+0000", 2020-08-07T15:11:17.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42764147600000002 2020-08-07T15:11:17.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:17.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793091+0000", 2020-08-07T15:11:17.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6500000000000002e-06 2020-08-07T15:11:17.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:17.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793099+0000", 2020-08-07T15:11:17.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3664e-05 2020-08-07T15:11:17.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:17.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793153+0000", 2020-08-07T15:11:17.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1078e-05 2020-08-07T15:11:17.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:17.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793164+0000", 2020-08-07T15:11:17.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3404000000000003e-05 2020-08-07T15:11:17.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:17.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793248+0000", 2020-08-07T15:11:17.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001 2020-08-07T15:11:17.137 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.137 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:17.137 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.138 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:107952 3.d e61/35)", 2020-08-07T15:11:17.139 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:11:17.139 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 206.60409149399999, 2020-08-07T15:11:17.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001, 2020-08-07T15:11:17.140 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:17.140 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:17.141 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:17.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:17.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:11:17.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:17.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:17.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:11:17.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8826000000000003e-05 2020-08-07T15:11:17.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:17.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326406+0000", 2020-08-07T15:11:17.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9199999999999998e-06 2020-08-07T15:11:17.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:17.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326413+0000", 2020-08-07T15:11:17.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75082853500000002 2020-08-07T15:11:17.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:17.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077242+0000", 2020-08-07T15:11:17.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1479999999999999e-06 2020-08-07T15:11:17.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:17.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077246+0000", 2020-08-07T15:11:17.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3331999999999998e-05 2020-08-07T15:11:17.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:17.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077289+0000", 2020-08-07T15:11:17.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8977000000000001e-05 2020-08-07T15:11:17.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:17.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077308+0000", 2020-08-07T15:11:17.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001 2020-08-07T15:11:17.195 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.195 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:17.195 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.196 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31696 4.0 4:0335eed9:::benchmark_data_smithi074_44586_object3961:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e100)", 2020-08-07T15:11:17.196 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:11:17.196 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 148.82618393499999, 2020-08-07T15:11:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237, 2020-08-07T15:11:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31696 2020-08-07T15:11:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:17.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:17.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:11:17.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:17.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:17.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:11:17.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7654e-05 2020-08-07T15:11:17.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:17.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104293+0000", 2020-08-07T15:11:17.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.061e-06 2020-08-07T15:11:17.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:17.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104296+0000", 2020-08-07T15:11:17.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0859999999999998e-06 2020-08-07T15:11:17.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:17.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104302+0000", 2020-08-07T15:11:17.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8030000000000001e-06 2020-08-07T15:11:17.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:17.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104305+0000", 2020-08-07T15:11:17.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037634769999999999 2020-08-07T15:11:17.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:17.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108068+0000", 2020-08-07T15:11:17.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8814999999999998e-05 2020-08-07T15:11:17.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:17.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108137+0000", 2020-08-07T15:11:17.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111455 2020-08-07T15:11:17.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:11:17.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108248+0000", 2020-08-07T15:11:17.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028756820000000001 2020-08-07T15:11:17.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:17.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111124+0000", 2020-08-07T15:11:17.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061054499999999999 2020-08-07T15:11:17.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:17.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111735+0000", 2020-08-07T15:11:17.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.423966167 2020-08-07T15:11:17.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:17.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535701+0000", 2020-08-07T15:11:17.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4929999999999999e-06 2020-08-07T15:11:17.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:17.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535702+0000", 2020-08-07T15:11:17.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2571999999999997e-05 2020-08-07T15:11:17.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:17.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535755+0000", 2020-08-07T15:11:17.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1427000000000003e-05 2020-08-07T15:11:17.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:17.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535786+0000", 2020-08-07T15:11:17.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237 2020-08-07T15:11:17.249 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.249 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:17.250 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.250 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:31696 4.0 e100/81 ondisk, result = 0)", 2020-08-07T15:11:17.251 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:11:17.251 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 148.81656937700001, 2020-08-07T15:11:17.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954, 2020-08-07T15:11:17.252 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:17.252 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:11:17.252 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:17.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:17.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:11:17.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:17.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:17.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:11:17.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1185000000000001e-05 2020-08-07T15:11:17.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:17.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113901+0000", 2020-08-07T15:11:17.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2930000000000002e-06 2020-08-07T15:11:17.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:17.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113905+0000", 2020-08-07T15:11:17.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42168640400000001 2020-08-07T15:11:17.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:17.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535591+0000", 2020-08-07T15:11:17.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2820000000000002e-06 2020-08-07T15:11:17.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:17.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535598+0000", 2020-08-07T15:11:17.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6578999999999999e-05 2020-08-07T15:11:17.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:17.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535684+0000", 2020-08-07T15:11:17.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8449999999999992e-06 2020-08-07T15:11:17.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:17.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535693+0000", 2020-08-07T15:11:17.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7366000000000003e-05 2020-08-07T15:11:17.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:17.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535791+0000", 2020-08-07T15:11:17.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954 2020-08-07T15:11:17.279 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.280 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:17.280 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.281 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:37024 4.0 e106/81)", 2020-08-07T15:11:17.281 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:11:17.281 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 142.70178048299999, 2020-08-07T15:11:17.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999, 2020-08-07T15:11:17.281 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:17.283 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:17.283 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:17.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:17.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:11:17.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:17.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:17.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:11:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4926000000000001e-05 2020-08-07T15:11:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228683+0000", 2020-08-07T15:11:17.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1280000000000002e-06 2020-08-07T15:11:17.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:17.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228686+0000", 2020-08-07T15:11:17.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69740012100000004 2020-08-07T15:11:17.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:17.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926086+0000", 2020-08-07T15:11:17.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000004e-06 2020-08-07T15:11:17.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:17.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926090+0000", 2020-08-07T15:11:17.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113526 2020-08-07T15:11:17.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:17.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926204+0000", 2020-08-07T15:11:17.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8824000000000001e-05 2020-08-07T15:11:17.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:17.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926222+0000", 2020-08-07T15:11:17.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999 2020-08-07T15:11:17.291 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.291 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:17.291 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.292 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42503 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2615, mlcod=112'2607)", 2020-08-07T15:11:17.292 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:11:17.292 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 136.634208567, 2020-08-07T15:11:17.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998, 2020-08-07T15:11:17.293 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:17.293 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:17.293 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:17.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:17.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:11:17.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:17.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:17.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:11:17.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270999999999998e-05 2020-08-07T15:11:17.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:17.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296292+0000", 2020-08-07T15:11:17.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4729999999999997e-06 2020-08-07T15:11:17.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:17.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296298+0000", 2020-08-07T15:11:17.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.448795049 2020-08-07T15:11:17.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:17.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745093+0000", 2020-08-07T15:11:17.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0140000000000001e-06 2020-08-07T15:11:17.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:17.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745100+0000", 2020-08-07T15:11:17.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2596999999999998e-05 2020-08-07T15:11:17.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:17.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745153+0000", 2020-08-07T15:11:17.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1662000000000003e-05 2020-08-07T15:11:17.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:17.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745184+0000", 2020-08-07T15:11:17.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093174800000000004 2020-08-07T15:11:17.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:17.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746116+0000", 2020-08-07T15:11:17.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0926999999999994e-05 2020-08-07T15:11:17.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:17.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746187+0000", 2020-08-07T15:11:17.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998 2020-08-07T15:11:17.333 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.333 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:17.341 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.342 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42504 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2616, mlcod=113'2609)", 2020-08-07T15:11:17.342 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:11:17.343 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 136.63087167399999, 2020-08-07T15:11:17.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998, 2020-08-07T15:11:17.344 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:17.344 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:17.345 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:17.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:17.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:11:17.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:17.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:17.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:11:17.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2154000000000002e-05 2020-08-07T15:11:17.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:17.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299629+0000", 2020-08-07T15:11:17.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1929999999999998e-06 2020-08-07T15:11:17.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:17.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299636+0000", 2020-08-07T15:11:17.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44766315400000001 2020-08-07T15:11:17.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:17.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747299+0000", 2020-08-07T15:11:17.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5889999999999999e-06 2020-08-07T15:11:17.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:17.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747305+0000", 2020-08-07T15:11:17.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3094999999999999e-05 2020-08-07T15:11:17.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:17.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747358+0000", 2020-08-07T15:11:17.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3283e-05 2020-08-07T15:11:17.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:17.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747392+0000", 2020-08-07T15:11:17.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015319509999999999 2020-08-07T15:11:17.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:17.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748924+0000", 2020-08-07T15:11:17.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5235999999999996e-05 2020-08-07T15:11:17.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:17.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748999+0000", 2020-08-07T15:11:17.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998 2020-08-07T15:11:17.369 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.369 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:17.370 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.374 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42519 4.8 e113/81 4:13071de5:::benchmark_data_smithi074_44586_object5314:head v 113'2623, mlcod=113'2617)", 2020-08-07T15:11:17.374 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.762041+0000", 2020-08-07T15:11:17.374 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 136.16840761399999, 2020-08-07T15:11:17.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.34605073400000003, 2020-08-07T15:11:17.375 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:17.375 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:17.376 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:17.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:17.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.762041+0000", 2020-08-07T15:11:17.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:17.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:17.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.762041+0000", 2020-08-07T15:11:17.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9067000000000002e-05 2020-08-07T15:11:17.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:17.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.762100+0000", 2020-08-07T15:11:17.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3529999999999997e-06 2020-08-07T15:11:17.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:17.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.762108+0000", 2020-08-07T15:11:17.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.34498205900000001 2020-08-07T15:11:17.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:17.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.107090+0000", 2020-08-07T15:11:17.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7420000000000003e-06 2020-08-07T15:11:17.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:17.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.107096+0000", 2020-08-07T15:11:17.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1074000000000004e-05 2020-08-07T15:11:17.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:17.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.107167+0000", 2020-08-07T15:11:17.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4423999999999999e-05 2020-08-07T15:11:17.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:17.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.107191+0000", 2020-08-07T15:11:17.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00084080199999999998 2020-08-07T15:11:17.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:17.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.108032+0000", 2020-08-07T15:11:17.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0213000000000001e-05 2020-08-07T15:11:17.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:17.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.108092+0000", 2020-08-07T15:11:17.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.34605073400000003 2020-08-07T15:11:17.408 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.408 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:17.408 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.412 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42520 4.8 e113/81 4:13071de5:::benchmark_data_smithi074_44586_object5314:head v 113'2624, mlcod=113'2618)", 2020-08-07T15:11:17.412 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:11:17.412 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 136.16492746500001, 2020-08-07T15:11:17.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001, 2020-08-07T15:11:17.413 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:17.413 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:17.413 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:17.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:17.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:11:17.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:17.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:17.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:11:17.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3967999999999995e-05 2020-08-07T15:11:17.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:17.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765585+0000", 2020-08-07T15:11:17.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0949999999999998e-06 2020-08-07T15:11:17.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:17.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765593+0000", 2020-08-07T15:11:17.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52703464200000005 2020-08-07T15:11:17.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:17.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292627+0000", 2020-08-07T15:11:17.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6929999999999996e-06 2020-08-07T15:11:17.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:17.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292636+0000", 2020-08-07T15:11:17.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9292000000000001e-05 2020-08-07T15:11:17.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:17.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292695+0000", 2020-08-07T15:11:17.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5395999999999999e-05 2020-08-07T15:11:17.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:17.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292721+0000", 2020-08-07T15:11:17.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00090361399999999996 2020-08-07T15:11:17.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:17.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293624+0000", 2020-08-07T15:11:17.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1761000000000002e-05 2020-08-07T15:11:17.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:17.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293666+0000", 2020-08-07T15:11:17.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001 2020-08-07T15:11:17.557 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.571 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:17.572 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.584 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:19776 5.4 5:2bb0c0d9:::benchmark_data_smithi074_53697_object2471:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e173)", 2020-08-07T15:11:17.584 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:11:17.587 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 44.530734780000003, 2020-08-07T15:11:17.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.402061844, 2020-08-07T15:11:17.587 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:17.588 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:17.588 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:17.588 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:11:17.588 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:17.588 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 19776 2020-08-07T15:11:17.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.589 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:17.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:17.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:11:17.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:17.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:17.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:11:17.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-05 2020-08-07T15:11:17.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:17.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399752+0000", 2020-08-07T15:11:17.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8799999999999999e-06 2020-08-07T15:11:17.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:17.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399757+0000", 2020-08-07T15:11:17.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7690000000000008e-06 2020-08-07T15:11:17.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:17.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399767+0000", 2020-08-07T15:11:17.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-06 2020-08-07T15:11:17.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:17.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399771+0000", 2020-08-07T15:11:17.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003255109 2020-08-07T15:11:17.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:17.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403026+0000", 2020-08-07T15:11:17.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101044 2020-08-07T15:11:17.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:17.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403127+0000", 2020-08-07T15:11:17.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000193158 2020-08-07T15:11:17.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,5", 2020-08-07T15:11:17.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403320+0000", 2020-08-07T15:11:17.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017185530000000001 2020-08-07T15:11:17.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:17.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405038+0000", 2020-08-07T15:11:17.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6300000000000004e-07 2020-08-07T15:11:17.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:17.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405039+0000", 2020-08-07T15:11:17.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012590600000000001 2020-08-07T15:11:17.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:17.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405165+0000", 2020-08-07T15:11:17.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109757 2020-08-07T15:11:17.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:17.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405275+0000", 2020-08-07T15:11:17.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39642174699999999 2020-08-07T15:11:17.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:17.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801697+0000", 2020-08-07T15:11:17.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2799999999999996e-07 2020-08-07T15:11:17.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:17.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801697+0000", 2020-08-07T15:11:17.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8740999999999997e-05 2020-08-07T15:11:17.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:17.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801776+0000", 2020-08-07T15:11:17.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.402061844 2020-08-07T15:11:17.694 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.694 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:17.695 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.695 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:19776 5.4 e173/164 ondisk, result = 0)", 2020-08-07T15:11:17.696 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:11:17.696 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 44.524986476999999, 2020-08-07T15:11:17.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39631765200000002, 2020-08-07T15:11:17.696 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:17.696 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:11:17.696 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:17.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:17.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:11:17.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:17.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:17.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:11:17.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9332e-05 2020-08-07T15:11:17.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:17.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405492+0000", 2020-08-07T15:11:17.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8900000000000004e-06 2020-08-07T15:11:17.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:17.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405498+0000", 2020-08-07T15:11:17.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39566664200000001 2020-08-07T15:11:17.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:17.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801164+0000", 2020-08-07T15:11:17.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7440000000000001e-06 2020-08-07T15:11:17.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:17.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801168+0000", 2020-08-07T15:11:17.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050828199999999998 2020-08-07T15:11:17.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:17.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801676+0000", 2020-08-07T15:11:17.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1011e-05 2020-08-07T15:11:17.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:17.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801687+0000", 2020-08-07T15:11:17.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2750999999999993e-05 2020-08-07T15:11:17.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:17.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801780+0000", 2020-08-07T15:11:17.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39631765200000002 2020-08-07T15:11:17.753 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.753 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:17.754 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.754 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:38568 5.4 5:27bd1ca8:::benchmark_data_smithi074_53697_object4820:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e184)", 2020-08-07T15:11:17.754 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:44.100898+0000", 2020-08-07T15:11:17.755 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.829551281999997, 2020-08-07T15:11:17.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31617937600000001, 2020-08-07T15:11:17.755 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:17.755 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:17.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:17.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:11:17.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:17.756 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 38568 2020-08-07T15:11:17.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.756 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:17.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:17.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.100898+0000", 2020-08-07T15:11:17.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:17.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:17.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.100898+0000", 2020-08-07T15:11:17.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6347999999999999e-05 2020-08-07T15:11:17.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:17.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.100924+0000", 2020-08-07T15:11:17.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.779e-06 2020-08-07T15:11:17.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:17.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.100927+0000", 2020-08-07T15:11:17.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5209999999999999e-06 2020-08-07T15:11:17.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:17.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.100933+0000", 2020-08-07T15:11:17.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5829999999999998e-06 2020-08-07T15:11:17.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:17.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.100937+0000", 2020-08-07T15:11:17.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037280479999999999 2020-08-07T15:11:17.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:17.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.104665+0000", 2020-08-07T15:11:17.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6570000000000005e-05 2020-08-07T15:11:17.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:17.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.104761+0000", 2020-08-07T15:11:17.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016373099999999999 2020-08-07T15:11:17.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,5", 2020-08-07T15:11:17.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.104925+0000", 2020-08-07T15:11:17.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001714641 2020-08-07T15:11:17.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:17.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.106640+0000", 2020-08-07T15:11:17.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015086769999999999 2020-08-07T15:11:17.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:17.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.108149+0000", 2020-08-07T15:11:17.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.30880820399999998 2020-08-07T15:11:17.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:17.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.416957+0000", 2020-08-07T15:11:17.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6400000000000001e-07 2020-08-07T15:11:17.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:17.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.416958+0000", 2020-08-07T15:11:17.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6707999999999999e-05 2020-08-07T15:11:17.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:17.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.417024+0000", 2020-08-07T15:11:17.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2802e-05 2020-08-07T15:11:17.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:17.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:44.417077+0000", 2020-08-07T15:11:17.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31617937600000001 2020-08-07T15:11:17.832 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.833 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:17.837 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.838 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:68968 5.4 5:25575389:::benchmark_data_smithi074_53697_object8620:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e198)", 2020-08-07T15:11:17.838 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:06.447296+0000", 2020-08-07T15:11:17.838 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.483152780999999, 2020-08-07T15:11:17.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33959526099999998, 2020-08-07T15:11:17.839 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:17.839 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:17.839 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:17.839 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:11:17.840 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:17.840 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 68968 2020-08-07T15:11:17.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.840 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:17.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:17.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:06.447296+0000", 2020-08-07T15:11:17.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:17.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:17.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:06.447296+0000", 2020-08-07T15:11:17.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7803999999999998e-05 2020-08-07T15:11:17.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:17.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:06.447344+0000", 2020-08-07T15:11:17.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3240000000000002e-06 2020-08-07T15:11:17.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:17.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:06.447349+0000", 2020-08-07T15:11:17.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0536999999999999e-05 2020-08-07T15:11:17.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:17.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:06.447360+0000", 2020-08-07T15:11:17.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7030000000000003e-06 2020-08-07T15:11:17.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:17.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:06.447366+0000", 2020-08-07T15:11:17.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012812749999999999 2020-08-07T15:11:17.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:17.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:06.448647+0000", 2020-08-07T15:11:17.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5853000000000005e-05 2020-08-07T15:11:17.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:17.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:06.448743+0000", 2020-08-07T15:11:17.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022243100000000001 2020-08-07T15:11:17.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,5", 2020-08-07T15:11:17.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:06.448965+0000", 2020-08-07T15:11:17.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001222435 2020-08-07T15:11:17.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:17.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:06.450188+0000", 2020-08-07T15:11:17.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.063e-06 2020-08-07T15:11:17.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:17.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:06.450189+0000", 2020-08-07T15:11:17.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1961999999999999e-05 2020-08-07T15:11:17.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:17.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:06.450271+0000", 2020-08-07T15:11:17.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023218499999999999 2020-08-07T15:11:17.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:17.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:06.450503+0000", 2020-08-07T15:11:17.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33631804900000001 2020-08-07T15:11:17.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:17.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:06.786821+0000", 2020-08-07T15:11:17.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1100000000000005e-07 2020-08-07T15:11:17.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:17.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:06.786822+0000", 2020-08-07T15:11:17.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9828999999999998e-05 2020-08-07T15:11:17.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:17.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:06.786891+0000", 2020-08-07T15:11:17.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33959526099999998 2020-08-07T15:11:17.935 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.953 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:17.957 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:17.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.958 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:68968 5.4 e198/164 ondisk, result = 0)", 2020-08-07T15:11:17.962 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:06.450803+0000", 2020-08-07T15:11:17.962 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.479645469999999, 2020-08-07T15:11:17.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33609480899999999, 2020-08-07T15:11:17.963 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:17.963 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:11:17.963 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:17.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:17.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:06.450803+0000", 2020-08-07T15:11:17.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:17.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:17.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:06.450803+0000", 2020-08-07T15:11:17.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1349000000000001e-05 2020-08-07T15:11:17.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:17.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:06.450835+0000", 2020-08-07T15:11:17.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7559999999999996e-06 2020-08-07T15:11:17.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:17.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:06.450841+0000", 2020-08-07T15:11:17.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33592522800000002 2020-08-07T15:11:17.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:17.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:06.786766+0000", 2020-08-07T15:11:17.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.354e-06 2020-08-07T15:11:17.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:17.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:17.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:17.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:06.786769+0000", 2020-08-07T15:11:18.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4365e-05 2020-08-07T15:11:18.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:18.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:18.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:18.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:06.786804+0000", 2020-08-07T15:11:18.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5949999999999999e-06 2020-08-07T15:11:18.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:18.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:18.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:18.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:06.786812+0000", 2020-08-07T15:11:18.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6162000000000006e-05 2020-08-07T15:11:18.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:18.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:18.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:18.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:06.786898+0000", 2020-08-07T15:11:18.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.33609480899999999 2020-08-07T15:11:18.014 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:18.014 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:18.015 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:18.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:18.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:18.015 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:77920 5.9 5:922793ba:::benchmark_data_smithi074_53697_object9739:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e200)", 2020-08-07T15:11:18.015 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:12.100962+0000", 2020-08-07T15:11:18.016 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8294866619999999, 2020-08-07T15:11:18.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32626295300000002, 2020-08-07T15:11:18.016 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:18.016 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:18.016 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:18.017 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:11:18.017 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:18.017 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 77920 2020-08-07T15:11:18.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:18.017 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:18.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:18.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:18.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:12.100962+0000", 2020-08-07T15:11:18.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:18.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:18.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:18.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:18.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:12.100962+0000", 2020-08-07T15:11:18.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8789000000000003e-05 2020-08-07T15:11:18.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:18.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:18.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:18.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:12.101011+0000", 2020-08-07T15:11:18.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5820000000000001e-06 2020-08-07T15:11:18.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:18.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:18.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:18.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:12.101017+0000", 2020-08-07T15:11:18.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0814e-05 2020-08-07T15:11:18.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:18.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:18.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:18.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:12.101027+0000", 2020-08-07T15:11:18.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6330000000000002e-06 2020-08-07T15:11:18.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:18.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:18.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:18.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:12.101036+0000", 2020-08-07T15:11:18.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041611649999999997 2020-08-07T15:11:18.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:18.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:18.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:18.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:12.105197+0000", 2020-08-07T15:11:18.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104362 2020-08-07T15:11:18.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:18.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:18.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:18.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:12.105302+0000", 2020-08-07T15:11:18.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020629000000000001 2020-08-07T15:11:18.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:18.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:18.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:11:18.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:12.105508+0000", 2020-08-07T15:11:18.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044300989999999998 2020-08-07T15:11:18.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:18.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:18.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:18.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:12.109938+0000", 2020-08-07T15:11:18.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8599999999999997e-07 2020-08-07T15:11:18.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:18.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:18.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:18.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:12.109939+0000", 2020-08-07T15:11:18.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5481000000000005e-05 2020-08-07T15:11:18.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:18.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:18.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:18.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:12.110004+0000", 2020-08-07T15:11:18.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012138749999999999 2020-08-07T15:11:18.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:18.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:18.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:18.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:12.111218+0000", 2020-08-07T15:11:18.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.31594077700000001 2020-08-07T15:11:18.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:18.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:18.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:18.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:12.427159+0000", 2020-08-07T15:11:18.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.632e-06 2020-08-07T15:11:18.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:18.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:18.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:18.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:12.427161+0000", 2020-08-07T15:11:18.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4667999999999998e-05 2020-08-07T15:11:18.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:18.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:18.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:18.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:12.427225+0000", 2020-08-07T15:11:18.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32626295300000002 2020-08-07T15:11:18.078 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:18.078 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:18.078 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:18.079 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:18.086 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:18.086 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:11:18.087 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:11:18.088 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:11:18.108 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 57 15 85943 85928 11.7762 11.1562 0.113868 0.0105913 2020-08-07T15:11:18.109 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:18.087+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:18.111 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:11:18.222 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 58 16 87390 87374 11.768 11.2969 0.00893165 0.0106167 2020-08-07T15:11:18.306 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:18.305+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:18.321 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:11:18.322 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:11:18.322 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.322 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:87496 5.5 5:aa720834:::benchmark_data_smithi074_53697_object10936:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e203)", 2020-08-07T15:11:18.322 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:18.288488+0000", 2020-08-07T15:11:18.323 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.028342181000000001, 2020-08-07T15:11:18.323 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.028373270999999999, 2020-08-07T15:11:18.323 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:18.323 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:11:18.323 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:18.324 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:18.324 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:18.324 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 87496 2020-08-07T15:11:18.324 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.324 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:18.325 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.325 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:18.325 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.288488+0000", 2020-08-07T15:11:18.325 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:18.325 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.330 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.330 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:18.331 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.288488+0000", 2020-08-07T15:11:18.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5717999999999997e-05 2020-08-07T15:11:18.342 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.343 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.343 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:18.343 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.288544+0000", 2020-08-07T15:11:18.343 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.092e-06 2020-08-07T15:11:18.344 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.344 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.344 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:18.344 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.288551+0000", 2020-08-07T15:11:18.344 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4032000000000001e-05 2020-08-07T15:11:18.345 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.345 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.345 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:18.345 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.288565+0000", 2020-08-07T15:11:18.345 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7360000000000004e-06 2020-08-07T15:11:18.346 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.346 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.346 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:18.347 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.288572+0000", 2020-08-07T15:11:18.347 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00244779 2020-08-07T15:11:18.347 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.347 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:18.347 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.291020+0000", 2020-08-07T15:11:18.348 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000110623 2020-08-07T15:11:18.348 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.348 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.348 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:18.349 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.291130+0000", 2020-08-07T15:11:18.349 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000223391 2020-08-07T15:11:18.349 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.349 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.349 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4", 2020-08-07T15:11:18.370 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.291354+0000", 2020-08-07T15:11:18.370 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003234378 2020-08-07T15:11:18.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.372 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:18.372 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.294588+0000", 2020-08-07T15:11:18.373 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0060997600000000001 2020-08-07T15:11:18.373 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:18.373 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:18.373 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:18.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.377 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5099.0:87534 5.8 e203/191 5:1838e406:::benchmark_data_smithi074_53697_object10941:head v 203'5486, mlcod=203'5477)", 2020-08-07T15:11:18.377 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:18.315018+0000", 2020-08-07T15:11:18.377 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0018122220000000001, 2020-08-07T15:11:18.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019538020000000001, 2020-08-07T15:11:18.378 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:18.378 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:11:18.378 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:18.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:18.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.315018+0000", 2020-08-07T15:11:18.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:18.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:18.380 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.315018+0000", 2020-08-07T15:11:18.380 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1187000000000003e-05 2020-08-07T15:11:18.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:18.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.315069+0000", 2020-08-07T15:11:18.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.285e-06 2020-08-07T15:11:18.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.382 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.382 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:18.382 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.315076+0000", 2020-08-07T15:11:18.383 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6709999999999997e-05 2020-08-07T15:11:18.383 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.383 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.383 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:18.383 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.315132+0000", 2020-08-07T15:11:18.384 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8019999999999997e-06 2020-08-07T15:11:18.384 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.384 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.384 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:18.384 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.315138+0000", 2020-08-07T15:11:18.385 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011191700000000001 2020-08-07T15:11:18.385 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.385 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.385 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:18.385 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.315250+0000", 2020-08-07T15:11:18.386 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5562999999999999e-05 2020-08-07T15:11:18.386 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.386 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.386 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:18.386 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.315276+0000", 2020-08-07T15:11:18.387 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000257464 2020-08-07T15:11:18.387 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:18.387 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:18.387 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:18.387 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.388 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.388 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5099.0:87535 5.8 e203/191 5:1838e406:::benchmark_data_smithi074_53697_object10941:head v 203'5487, mlcod=203'5478)", 2020-08-07T15:11:18.388 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:18.316301+0000", 2020-08-07T15:11:18.388 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.00052893600000000003, 2020-08-07T15:11:18.389 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00075654900000000002, 2020-08-07T15:11:18.389 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:18.389 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:11:18.389 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:18.390 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.390 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:18.390 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.316301+0000", 2020-08-07T15:11:18.390 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:18.391 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.391 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.391 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:18.391 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.316301+0000", 2020-08-07T15:11:18.391 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8103999999999999e-05 2020-08-07T15:11:18.392 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.392 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.392 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:18.392 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.316350+0000", 2020-08-07T15:11:18.392 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6769999999999998e-06 2020-08-07T15:11:18.393 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.393 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.393 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:18.393 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.316356+0000", 2020-08-07T15:11:18.393 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000105481 2020-08-07T15:11:18.394 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.394 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.394 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:18.394 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.316462+0000", 2020-08-07T15:11:18.394 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4e-06 2020-08-07T15:11:18.395 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.395 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.395 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:18.395 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.316467+0000", 2020-08-07T15:11:18.395 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9354000000000003e-05 2020-08-07T15:11:18.396 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.396 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.396 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:18.396 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.316506+0000", 2020-08-07T15:11:18.397 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4443e-05 2020-08-07T15:11:18.397 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.397 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.397 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:18.399 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.316531+0000", 2020-08-07T15:11:18.399 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000229459 2020-08-07T15:11:18.399 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:18.399 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:18.400 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:18.400 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.400 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5099.0:87536 5.8 e203/191 5:1838e406:::benchmark_data_smithi074_53697_object10941:head v 203'5488, mlcod=203'5478)", 2020-08-07T15:11:18.400 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:18.316859+0000", 2020-08-07T15:11:18.401 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4294967295.9999709, 2020-08-07T15:11:18.401 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00027681100000000002, 2020-08-07T15:11:18.401 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:18.401 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:11:18.401 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:18.402 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.402 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:18.402 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.316859+0000", 2020-08-07T15:11:18.402 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:18.402 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.403 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.403 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:18.403 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.316859+0000", 2020-08-07T15:11:18.403 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0491999999999997e-05 2020-08-07T15:11:18.403 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.404 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.404 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:18.404 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.316910+0000", 2020-08-07T15:11:18.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4640000000000003e-06 2020-08-07T15:11:18.404 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.405 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:18.405 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.316916+0000", 2020-08-07T15:11:18.406 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0143999999999998e-05 2020-08-07T15:11:18.406 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.406 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.406 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:18.407 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.316986+0000", 2020-08-07T15:11:18.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8229999999999997e-06 2020-08-07T15:11:18.408 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.408 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.408 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:18.408 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.316990+0000", 2020-08-07T15:11:18.408 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7283999999999997e-05 2020-08-07T15:11:18.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.409 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.409 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:18.410 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.317028+0000", 2020-08-07T15:11:18.411 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4238000000000001e-05 2020-08-07T15:11:18.411 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:18.411 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:18.411 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:18.411 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:18.317052+0000", 2020-08-07T15:11:18.412 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019244499999999999 2020-08-07T15:11:18.412 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:18.412 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:18.412 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:18.412 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:18.413 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:11:18.413 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 4 2020-08-07T15:11:18.413 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:11:18.415 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:18.323+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:18.416 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:11:18.416 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:11:18.416 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:18.413+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:18.417 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:11:18.423 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:11:18.462 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:11:18.498 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:11:18.538 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:11:18.579 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:11:18.579 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:11:18.579 INFO:teuthology.orchestra.run.smithi074.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-08-07T15:11:18.580 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: problem getting command descriptions from osd.1 2020-08-07T15:11:18.580 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:11:18.580 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:11:18.580 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:11:18.581 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:11:18.581 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:11:18.581 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:11:18.581 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-08-07T15:11:18.582 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-08-07T15:11:18.590 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3, 4, 5, 6, 7] 2020-08-07T15:11:18.590 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:11:18.598 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2020-08-07T15:11:18.599 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight-by-utilization 110 0.05 2020-08-07T15:11:18.614 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:18.615 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:18.626 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:18.601+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:18.671 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:11:18.672 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:18.626+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:18.733 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:18.728+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:18.850 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:18.851 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:18.860 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:11:18.954 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:18.952+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:19.052 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:19.048+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:19.073 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:19.074 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:19.081 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:11:19.090 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:19.086+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:19.150 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:19.148+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:19.215 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 59 16 88874 88858 11.765 11.5938 0.00828697 0.0106175 2020-08-07T15:11:19.264 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:19.264 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:19.275 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:11:19.349 INFO:teuthology.orchestra.run.smithi074.stdout:moved 2 / 165 (1.21212%) 2020-08-07T15:11:19.350 INFO:teuthology.orchestra.run.smithi074.stdout:avg 23.5714 2020-08-07T15:11:19.350 INFO:teuthology.orchestra.run.smithi074.stdout:stddev 4.04313 -> 4.04313 (expected baseline 4.49489) 2020-08-07T15:11:19.350 INFO:teuthology.orchestra.run.smithi074.stdout:min osd.2 with 16 -> 16 pgs (0.678788 -> 0.678788 * mean) 2020-08-07T15:11:19.350 INFO:teuthology.orchestra.run.smithi074.stdout:max osd.5 with 24 -> 22 pgs (1.01818 -> 0.933333 * mean) 2020-08-07T15:11:19.351 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:11:19.351 INFO:teuthology.orchestra.run.smithi074.stdout:oload 110 2020-08-07T15:11:19.351 INFO:teuthology.orchestra.run.smithi074.stdout:max_change 0.05 2020-08-07T15:11:19.352 INFO:teuthology.orchestra.run.smithi074.stdout:max_change_osds 4 2020-08-07T15:11:19.352 INFO:teuthology.orchestra.run.smithi074.stdout:average_utilization 0.0137 2020-08-07T15:11:19.352 INFO:teuthology.orchestra.run.smithi074.stdout:overload_utilization 0.0150 2020-08-07T15:11:19.352 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5 weight 1.0000 -> 0.9500 2020-08-07T15:11:19.353 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:11:19.366 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:19.364+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:19.373 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2020-08-07T15:11:19.374 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight-by-utilization 110 1.0 2020-08-07T15:11:19.446 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:19.447 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:19.457 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:19.455+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:19.472 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:11:19.500 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:19.498+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:19.591 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:19.582+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:19.689 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:19.690 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:19.691 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:11:19.693 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:19.687+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:19.797 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:19.795+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:19.852 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:19.853 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:19.862 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:11:19.906 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:19.905+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:19.979 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:19.979 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:19.988 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:11:20.034 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:20.033+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:20.117 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:20.109+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:20.211 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:20.211+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:20.215 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:2020-08-07T15:11:20.215087+0000 min lat: 0.000974592 max lat: 1.487 avg lat: 0.0106213 2020-08-07T15:11:20.216 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-08-07T15:11:20.216 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 60 16 89423 89407 11.6404 4.28906 0.214218 0.0106213 2020-08-07T15:11:20.312 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:20.311+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:20.350 INFO:teuthology.orchestra.run.smithi074.stdout:moved 1 / 165 (0.606061%) 2020-08-07T15:11:20.351 INFO:teuthology.orchestra.run.smithi074.stdout:avg 23.5714 2020-08-07T15:11:20.351 INFO:teuthology.orchestra.run.smithi074.stdout:stddev 4.04313 -> 4.04313 (expected baseline 4.49489) 2020-08-07T15:11:20.351 INFO:teuthology.orchestra.run.smithi074.stdout:min osd.2 with 16 -> 16 pgs (0.678788 -> 0.678788 * mean) 2020-08-07T15:11:20.351 INFO:teuthology.orchestra.run.smithi074.stdout:max osd.7 with 25 -> 25 pgs (1.06061 -> 1.06061 * mean) 2020-08-07T15:11:20.351 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:11:20.352 INFO:teuthology.orchestra.run.smithi074.stdout:oload 110 2020-08-07T15:11:20.352 INFO:teuthology.orchestra.run.smithi074.stdout:max_change 1 2020-08-07T15:11:20.352 INFO:teuthology.orchestra.run.smithi074.stdout:max_change_osds 4 2020-08-07T15:11:20.352 INFO:teuthology.orchestra.run.smithi074.stdout:average_utilization 0.0137 2020-08-07T15:11:20.352 INFO:teuthology.orchestra.run.smithi074.stdout:overload_utilization 0.0150 2020-08-07T15:11:20.353 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5 weight 0.9500 -> 0.8567 2020-08-07T15:11:20.369 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2020-08-07T15:11:20.370 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd test-reweight-by-utilization 1000 0.05 2020-08-07T15:11:20.414 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:20.413+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:20.514 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:20.513+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:20.583 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:11:20.615 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:20.614+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:20.623 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:20.623 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:20.633 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:11:20.696 INFO:teuthology.orchestra.run.smithi074.stdout:moved 0 / 165 (0%) 2020-08-07T15:11:20.696 INFO:teuthology.orchestra.run.smithi074.stdout:avg 23.5714 2020-08-07T15:11:20.697 INFO:teuthology.orchestra.run.smithi074.stdout:stddev 4.04313 -> 4.04313 (expected baseline 4.49489) 2020-08-07T15:11:20.697 INFO:teuthology.orchestra.run.smithi074.stdout:min osd.2 with 16 -> 16 pgs (0.678788 -> 0.678788 * mean) 2020-08-07T15:11:20.697 INFO:teuthology.orchestra.run.smithi074.stdout:max osd.7 with 25 -> 25 pgs (1.06061 -> 1.06061 * mean) 2020-08-07T15:11:20.697 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:11:20.698 INFO:teuthology.orchestra.run.smithi074.stdout:oload 1000 2020-08-07T15:11:20.698 INFO:teuthology.orchestra.run.smithi074.stdout:max_change 0.05 2020-08-07T15:11:20.698 INFO:teuthology.orchestra.run.smithi074.stdout:max_change_osds 4 2020-08-07T15:11:20.698 INFO:teuthology.orchestra.run.smithi074.stdout:average_utilization 0.0137 2020-08-07T15:11:20.698 INFO:teuthology.orchestra.run.smithi074.stdout:overload_utilization 0.1367 2020-08-07T15:11:20.699 INFO:teuthology.orchestra.run.smithi074.stderr:no change 2020-08-07T15:11:20.708 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2020-08-07T15:11:20.709 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd test-reweight-by-utilization 110 0.05 2020-08-07T15:11:20.720 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:20.719+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:20.729 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:11:20.759 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:11:20.790 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:11:20.794 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:20.795 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:20.803 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:11:20.812 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:11:20.841 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:20.827+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:20.843 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:11:20.861 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:11:20.885 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:11:20.885 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:11:20.886 INFO:teuthology.orchestra.run.smithi074.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-08-07T15:11:20.886 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: problem getting command descriptions from osd.1 2020-08-07T15:11:20.886 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:11:20.886 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:11:20.887 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:11:20.887 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:11:20.887 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:11:20.887 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:11:20.888 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-08-07T15:11:20.889 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-08-07T15:11:20.932 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:20.927+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:20.969 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:11:20.970 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:11:20.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:11:20.971 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:11:20.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:20.971 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:108376 3.3 e62/20 ondisk, result = 0)", 2020-08-07T15:11:20.971 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:11:20.971 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 210.28182352600001, 2020-08-07T15:11:20.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62926872599999994, 2020-08-07T15:11:20.972 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:20.972 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:11:20.972 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:20.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:20.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:20.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:11:20.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:20.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:20.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:20.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:20.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683042+0000", 2020-08-07T15:11:20.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8491999999999998e-05 2020-08-07T15:11:20.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:20.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:20.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:20.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683061+0000", 2020-08-07T15:11:20.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9099999999999998e-06 2020-08-07T15:11:20.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:20.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:20.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:20.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.683065+0000", 2020-08-07T15:11:20.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62906670600000003 2020-08-07T15:11:20.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:20.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:20.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:20.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312131+0000", 2020-08-07T15:11:20.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2147000000000001e-05 2020-08-07T15:11:20.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:20.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:20.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:20.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312143+0000", 2020-08-07T15:11:20.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5152e-05 2020-08-07T15:11:20.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:20.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:20.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312239+0000", 2020-08-07T15:11:21.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5089999999999993e-06 2020-08-07T15:11:21.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:21.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312247+0000", 2020-08-07T15:11:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3810000000000001e-05 2020-08-07T15:11:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.312311+0000", 2020-08-07T15:11:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.62926872599999994 2020-08-07T15:11:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "MOSDPGPushReply(3.1 62/60 [PushReplyOp(3:8329f37e:::benchmark_data_smithi074_35447_object11915:head)])", 2020-08-07T15:11:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:11:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 210.10646550999999, 2020-08-07T15:11:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996, 2020-08-07T15:11:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:11:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:11:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5372000000000001e-05 2020-08-07T15:11:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858416+0000", 2020-08-07T15:11:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6899999999999998e-06 2020-08-07T15:11:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858419+0000", 2020-08-07T15:11:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63907904299999996 2020-08-07T15:11:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497498+0000", 2020-08-07T15:11:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3332999999999999e-05 2020-08-07T15:11:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497512+0000", 2020-08-07T15:11:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000450875 2020-08-07T15:11:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497962+0000", 2020-08-07T15:11:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000267779 2020-08-07T15:11:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.498230+0000", 2020-08-07T15:11:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996 2020-08-07T15:11:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:145792 3.1 3:881e0420:::benchmark_data_smithi074_35447_object16183:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:11:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:11:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 186.29805962899999, 2020-08-07T15:11:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997, 2020-08-07T15:11:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:11:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:11:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 145792 2020-08-07T15:11:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:11:21.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:21.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:21.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:11:21.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7489000000000001e-05 2020-08-07T15:11:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666824+0000", 2020-08-07T15:11:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0169999999999998e-06 2020-08-07T15:11:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666828+0000", 2020-08-07T15:11:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9829999999999996e-06 2020-08-07T15:11:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666836+0000", 2020-08-07T15:11:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.481e-06 2020-08-07T15:11:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666840+0000", 2020-08-07T15:11:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5299000000000001e-05 2020-08-07T15:11:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666885+0000", 2020-08-07T15:11:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014456500000000001 2020-08-07T15:11:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667030+0000", 2020-08-07T15:11:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017677799999999999 2020-08-07T15:11:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,5", 2020-08-07T15:11:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667207+0000", 2020-08-07T15:11:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004149649 2020-08-07T15:11:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671356+0000", 2020-08-07T15:11:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019339649999999999 2020-08-07T15:11:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673290+0000", 2020-08-07T15:11:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73346620500000004 2020-08-07T15:11:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406756+0000", 2020-08-07T15:11:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4700000000000001e-06 2020-08-07T15:11:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406759+0000", 2020-08-07T15:11:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6258999999999998e-05 2020-08-07T15:11:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406815+0000", 2020-08-07T15:11:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0126000000000002e-05 2020-08-07T15:11:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406855+0000", 2020-08-07T15:11:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997 2020-08-07T15:11:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:145792 3.1 e75/67 ondisk, result = 0)", 2020-08-07T15:11:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:11:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 186.29367474099999, 2020-08-07T15:11:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003, 2020-08-07T15:11:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:11:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:11:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:11:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2830999999999998e-05 2020-08-07T15:11:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671224+0000", 2020-08-07T15:11:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:11:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:21.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671229+0000", 2020-08-07T15:11:21.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.735414864 2020-08-07T15:11:21.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:21.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406644+0000", 2020-08-07T15:11:21.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1250000000000004e-06 2020-08-07T15:11:21.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:21.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406651+0000", 2020-08-07T15:11:21.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6567e-05 2020-08-07T15:11:21.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406737+0000", 2020-08-07T15:11:21.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0054000000000001e-05 2020-08-07T15:11:21.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:21.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406747+0000", 2020-08-07T15:11:21.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011391999999999999 2020-08-07T15:11:21.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:21.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406861+0000", 2020-08-07T15:11:21.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003 2020-08-07T15:11:21.152 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.152 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:21.152 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.157 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30921 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:11:21.158 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:11:21.166 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 154.25892636699999, 2020-08-07T15:11:21.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001, 2020-08-07T15:11:21.166 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:21.166 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:21.167 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:21.167 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:21.167 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:21.167 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30921 2020-08-07T15:11:21.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.168 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:21.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:21.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:11:21.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:21.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:21.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:11:21.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6954999999999998e-05 2020-08-07T15:11:21.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:21.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706016+0000", 2020-08-07T15:11:21.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0229999999999992e-06 2020-08-07T15:11:21.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:21.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706024+0000", 2020-08-07T15:11:21.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.432e-05 2020-08-07T15:11:21.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:21.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706039+0000", 2020-08-07T15:11:21.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8490000000000006e-06 2020-08-07T15:11:21.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:21.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706046+0000", 2020-08-07T15:11:21.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6837999999999999e-05 2020-08-07T15:11:21.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706113+0000", 2020-08-07T15:11:21.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3553000000000001e-05 2020-08-07T15:11:21.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:21.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706127+0000", 2020-08-07T15:11:21.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77277201699999998 2020-08-07T15:11:21.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.478899+0000", 2020-08-07T15:11:21.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093978599999999996 2020-08-07T15:11:21.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:11:21.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479839+0000", 2020-08-07T15:11:21.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010376029 2020-08-07T15:11:21.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490215+0000", 2020-08-07T15:11:21.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000190657 2020-08-07T15:11:21.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:21.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490405+0000", 2020-08-07T15:11:21.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001 2020-08-07T15:11:21.182 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.183 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:21.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.183 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30922 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:11:21.184 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:11:21.184 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 154.25876620400001, 2020-08-07T15:11:21.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247, 2020-08-07T15:11:21.184 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:21.184 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:21.185 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:21.185 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:21.185 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:21.185 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30922 2020-08-07T15:11:21.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.186 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:21.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:21.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:11:21.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:21.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:21.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:11:21.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9178999999999997e-05 2020-08-07T15:11:21.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:21.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706159+0000", 2020-08-07T15:11:21.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4999999999999999e-06 2020-08-07T15:11:21.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:21.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706164+0000", 2020-08-07T15:11:21.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1792e-05 2020-08-07T15:11:21.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:21.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706176+0000", 2020-08-07T15:11:21.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.139e-06 2020-08-07T15:11:21.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:21.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706182+0000", 2020-08-07T15:11:21.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039724900000000002 2020-08-07T15:11:21.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706579+0000", 2020-08-07T15:11:21.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1191e-05 2020-08-07T15:11:21.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:21.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706591+0000", 2020-08-07T15:11:21.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329101300000003 2020-08-07T15:11:21.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479882+0000", 2020-08-07T15:11:21.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9565000000000001e-05 2020-08-07T15:11:21.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:11:21.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479971+0000", 2020-08-07T15:11:21.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010471263 2020-08-07T15:11:21.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490442+0000", 2020-08-07T15:11:21.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015035600000000001 2020-08-07T15:11:21.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:21.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490593+0000", 2020-08-07T15:11:21.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247 2020-08-07T15:11:21.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.202 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:21.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.203 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30923 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:11:21.203 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:11:21.203 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 154.25829114300001, 2020-08-07T15:11:21.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998, 2020-08-07T15:11:21.204 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:21.204 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:21.204 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:21.204 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:21.205 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:21.205 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30923 2020-08-07T15:11:21.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.206 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:21.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:21.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:11:21.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:21.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:21.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:11:21.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5531000000000001e-05 2020-08-07T15:11:21.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:21.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706630+0000", 2020-08-07T15:11:21.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7689999999999999e-06 2020-08-07T15:11:21.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:21.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706637+0000", 2020-08-07T15:11:21.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3577999999999999e-05 2020-08-07T15:11:21.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:21.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706650+0000", 2020-08-07T15:11:21.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1439999999999997e-06 2020-08-07T15:11:21.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:21.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706658+0000", 2020-08-07T15:11:21.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1663999999999999e-05 2020-08-07T15:11:21.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706709+0000", 2020-08-07T15:11:21.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0149000000000001e-05 2020-08-07T15:11:21.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:21.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706719+0000", 2020-08-07T15:11:21.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329694900000001 2020-08-07T15:11:21.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480016+0000", 2020-08-07T15:11:21.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7998999999999997e-05 2020-08-07T15:11:21.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:11:21.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480084+0000", 2020-08-07T15:11:21.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010547333000000001 2020-08-07T15:11:21.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490632+0000", 2020-08-07T15:11:21.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7439e-05 2020-08-07T15:11:21.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:21.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490729+0000", 2020-08-07T15:11:21.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998 2020-08-07T15:11:21.223 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.223 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:21.224 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.224 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30924 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:11:21.224 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:11:21.225 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 154.25816387099999, 2020-08-07T15:11:21.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995, 2020-08-07T15:11:21.225 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:21.225 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:21.226 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:21.226 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:21.226 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:21.226 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30924 2020-08-07T15:11:21.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.227 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:21.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:21.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:11:21.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:21.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:21.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:11:21.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9698999999999999e-05 2020-08-07T15:11:21.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:21.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706762+0000", 2020-08-07T15:11:21.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8080000000000001e-06 2020-08-07T15:11:21.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:21.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706767+0000", 2020-08-07T15:11:21.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1069e-05 2020-08-07T15:11:21.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:21.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706778+0000", 2020-08-07T15:11:21.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6969999999999998e-06 2020-08-07T15:11:21.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:21.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706784+0000", 2020-08-07T15:11:21.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8792999999999998e-05 2020-08-07T15:11:21.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706843+0000", 2020-08-07T15:11:21.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5449999999999998e-06 2020-08-07T15:11:21.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:21.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706850+0000", 2020-08-07T15:11:21.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.773278354 2020-08-07T15:11:21.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480129+0000", 2020-08-07T15:11:21.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1577999999999997e-05 2020-08-07T15:11:21.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:11:21.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480210+0000", 2020-08-07T15:11:21.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010541915000000001 2020-08-07T15:11:21.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490752+0000", 2020-08-07T15:11:21.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103937 2020-08-07T15:11:21.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:21.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490856+0000", 2020-08-07T15:11:21.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995 2020-08-07T15:11:21.252 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.252 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:21.252 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30925 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:11:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:11:21.253 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 154.25804009999999, 2020-08-07T15:11:21.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995, 2020-08-07T15:11:21.254 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:21.254 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:21.254 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:21.254 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:21.255 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:21.255 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30925 2020-08-07T15:11:21.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.255 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:21.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:21.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:11:21.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:21.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:21.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:11:21.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4664999999999999e-05 2020-08-07T15:11:21.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:21.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706880+0000", 2020-08-07T15:11:21.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7479999999999999e-06 2020-08-07T15:11:21.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:21.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706886+0000", 2020-08-07T15:11:21.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1838000000000001e-05 2020-08-07T15:11:21.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:21.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706898+0000", 2020-08-07T15:11:21.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0089999999999998e-06 2020-08-07T15:11:21.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:21.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706903+0000", 2020-08-07T15:11:21.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3559e-05 2020-08-07T15:11:21.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706956+0000", 2020-08-07T15:11:21.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0309000000000001e-05 2020-08-07T15:11:21.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:21.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706967+0000", 2020-08-07T15:11:21.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77328290499999996 2020-08-07T15:11:21.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480250+0000", 2020-08-07T15:11:21.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8680999999999997e-05 2020-08-07T15:11:21.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:11:21.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480298+0000", 2020-08-07T15:11:21.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01059272 2020-08-07T15:11:21.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490891+0000", 2020-08-07T15:11:21.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014580300000000001 2020-08-07T15:11:21.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:21.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491037+0000", 2020-08-07T15:11:21.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995 2020-08-07T15:11:21.284 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.285 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:21.285 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.286 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30926 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:11:21.286 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:11:21.286 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 154.25792323100001, 2020-08-07T15:11:21.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005, 2020-08-07T15:11:21.289 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:21.289 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:21.291 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:21.291 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:21.291 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:21.292 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30926 2020-08-07T15:11:21.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.292 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:21.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:21.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:11:21.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:21.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:21.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:11:21.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5223000000000003e-05 2020-08-07T15:11:21.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706998+0000", 2020-08-07T15:11:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4809999999999997e-06 2020-08-07T15:11:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707003+0000", 2020-08-07T15:11:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1663e-05 2020-08-07T15:11:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707015+0000", 2020-08-07T15:11:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9729999999999997e-06 2020-08-07T15:11:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707020+0000", 2020-08-07T15:11:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2565999999999999e-05 2020-08-07T15:11:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707062+0000", 2020-08-07T15:11:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8944e-05 2020-08-07T15:11:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707081+0000", 2020-08-07T15:11:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77325885000000005 2020-08-07T15:11:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480340+0000", 2020-08-07T15:11:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0321000000000006e-05 2020-08-07T15:11:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:11:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480420+0000", 2020-08-07T15:11:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010655958 2020-08-07T15:11:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491076+0000", 2020-08-07T15:11:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9784999999999995e-05 2020-08-07T15:11:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491166+0000", 2020-08-07T15:11:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005 2020-08-07T15:11:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30927 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:11:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:11:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 154.25780541200001, 2020-08-07T15:11:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999, 2020-08-07T15:11:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30927 2020-08-07T15:11:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:11:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:11:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9861999999999997e-05 2020-08-07T15:11:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707120+0000", 2020-08-07T15:11:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8969999999999996e-06 2020-08-07T15:11:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707126+0000", 2020-08-07T15:11:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1248e-05 2020-08-07T15:11:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707137+0000", 2020-08-07T15:11:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.955e-06 2020-08-07T15:11:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707142+0000", 2020-08-07T15:11:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0643999999999998e-05 2020-08-07T15:11:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707193+0000", 2020-08-07T15:11:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1983e-05 2020-08-07T15:11:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707205+0000", 2020-08-07T15:11:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77324914 2020-08-07T15:11:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480454+0000", 2020-08-07T15:11:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4119999999999998e-05 2020-08-07T15:11:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:11:21.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480498+0000", 2020-08-07T15:11:21.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010709707000000001 2020-08-07T15:11:21.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491208+0000", 2020-08-07T15:11:21.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014433800000000001 2020-08-07T15:11:21.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491352+0000", 2020-08-07T15:11:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999 2020-08-07T15:11:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30928 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:11:21.432 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:11:21.432 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 154.25768105500001, 2020-08-07T15:11:21.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998, 2020-08-07T15:11:21.432 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:21.432 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:21.433 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:21.433 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:21.433 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:21.433 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30928 2020-08-07T15:11:21.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.434 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:21.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:21.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:11:21.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:21.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:21.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:11:21.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6582000000000001e-05 2020-08-07T15:11:21.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:21.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707241+0000", 2020-08-07T15:11:21.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9190000000000004e-06 2020-08-07T15:11:21.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:21.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707247+0000", 2020-08-07T15:11:21.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000188822 2020-08-07T15:11:21.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:21.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707436+0000", 2020-08-07T15:11:21.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2940000000000002e-06 2020-08-07T15:11:21.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:21.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707442+0000", 2020-08-07T15:11:21.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7800000000000003e-05 2020-08-07T15:11:21.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707490+0000", 2020-08-07T15:11:21.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2020-08-07T15:11:21.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:21.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707501+0000", 2020-08-07T15:11:21.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77303498800000003 2020-08-07T15:11:21.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480536+0000", 2020-08-07T15:11:21.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7723000000000001e-05 2020-08-07T15:11:21.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:11:21.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480614+0000", 2020-08-07T15:11:21.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010812183 2020-08-07T15:11:21.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491426+0000", 2020-08-07T15:11:21.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5502999999999999e-05 2020-08-07T15:11:21.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:21.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491502+0000", 2020-08-07T15:11:21.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998 2020-08-07T15:11:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31009 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:11:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:11:21.498 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 154.233724749, 2020-08-07T15:11:21.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002, 2020-08-07T15:11:21.498 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:21.498 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:21.499 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:21.499 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:21.499 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:21.499 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31009 2020-08-07T15:11:21.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.500 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:21.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:21.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:11:21.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:21.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:21.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:11:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2429000000000002e-05 2020-08-07T15:11:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731193+0000", 2020-08-07T15:11:21.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9760000000000003e-06 2020-08-07T15:11:21.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:21.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731198+0000", 2020-08-07T15:11:21.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3597e-05 2020-08-07T15:11:21.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:21.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731212+0000", 2020-08-07T15:11:21.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1709999999999996e-06 2020-08-07T15:11:21.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:21.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731219+0000", 2020-08-07T15:11:21.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5402999999999997e-05 2020-08-07T15:11:21.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731285+0000", 2020-08-07T15:11:21.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3495e-05 2020-08-07T15:11:21.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:21.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731298+0000", 2020-08-07T15:11:21.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74935423599999995 2020-08-07T15:11:21.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480652+0000", 2020-08-07T15:11:21.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000137806 2020-08-07T15:11:21.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:21.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480790+0000", 2020-08-07T15:11:21.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022272399999999999 2020-08-07T15:11:21.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:11:21.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481013+0000", 2020-08-07T15:11:21.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003223381 2020-08-07T15:11:21.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:21.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484236+0000", 2020-08-07T15:11:21.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1370000000000001e-06 2020-08-07T15:11:21.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:21.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484237+0000", 2020-08-07T15:11:21.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6288e-05 2020-08-07T15:11:21.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:21.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484294+0000", 2020-08-07T15:11:21.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.007260402 2020-08-07T15:11:21.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:21.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491554+0000", 2020-08-07T15:11:21.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7358000000000003e-05 2020-08-07T15:11:21.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:21.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491601+0000", 2020-08-07T15:11:21.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1100000000000004e-07 2020-08-07T15:11:21.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:21.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491602+0000", 2020-08-07T15:11:21.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013436500000000001 2020-08-07T15:11:21.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:21.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491737+0000", 2020-08-07T15:11:21.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002 2020-08-07T15:11:21.604 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.604 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:21.604 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.605 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31010 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:11:21.605 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:11:21.616 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 154.23359879500001, 2020-08-07T15:11:21.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001, 2020-08-07T15:11:21.617 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:21.617 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:21.617 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:21.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:21.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:21.618 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31010 2020-08-07T15:11:21.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.618 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:21.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:21.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:11:21.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:21.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:21.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:11:21.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2398000000000002e-05 2020-08-07T15:11:21.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:21.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731319+0000", 2020-08-07T15:11:21.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7100000000000004e-06 2020-08-07T15:11:21.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:21.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731325+0000", 2020-08-07T15:11:21.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4328000000000001e-05 2020-08-07T15:11:21.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:21.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731339+0000", 2020-08-07T15:11:21.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3310000000000003e-06 2020-08-07T15:11:21.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:21.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731345+0000", 2020-08-07T15:11:21.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8952999999999998e-05 2020-08-07T15:11:21.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731384+0000", 2020-08-07T15:11:21.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0036e-05 2020-08-07T15:11:21.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:21.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731394+0000", 2020-08-07T15:11:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75027681300000004 2020-08-07T15:11:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481670+0000", 2020-08-07T15:11:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016555899999999999 2020-08-07T15:11:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481836+0000", 2020-08-07T15:11:21.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028760999999999999 2020-08-07T15:11:21.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:11:21.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482124+0000", 2020-08-07T15:11:21.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038342680000000001 2020-08-07T15:11:21.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:21.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485958+0000", 2020-08-07T15:11:21.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2559999999999999e-06 2020-08-07T15:11:21.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:21.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485959+0000", 2020-08-07T15:11:21.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3758999999999995e-05 2020-08-07T15:11:21.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:21.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486043+0000", 2020-08-07T15:11:21.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057590419999999998 2020-08-07T15:11:21.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:21.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491802+0000", 2020-08-07T15:11:21.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5757000000000001e-05 2020-08-07T15:11:21.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:21.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:11:21.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9299999999999997e-07 2020-08-07T15:11:21.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:21.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:11:21.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012496999999999999 2020-08-07T15:11:21.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:21.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491993+0000", 2020-08-07T15:11:21.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001 2020-08-07T15:11:21.737 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.738 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:21.738 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.739 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31011 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:11:21.739 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:11:21.739 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 154.23348258999999, 2020-08-07T15:11:21.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997, 2020-08-07T15:11:21.740 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:21.747 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:21.747 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:21.748 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:21.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:21.756 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31011 2020-08-07T15:11:21.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.756 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:21.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:21.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:11:21.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:21.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:21.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:11:21.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9539999999999997e-05 2020-08-07T15:11:21.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:21.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731433+0000", 2020-08-07T15:11:21.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9209999999999997e-06 2020-08-07T15:11:21.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:21.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731439+0000", 2020-08-07T15:11:21.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4246999999999999e-05 2020-08-07T15:11:21.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:21.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731453+0000", 2020-08-07T15:11:21.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8130000000000003e-06 2020-08-07T15:11:21.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:21.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731458+0000", 2020-08-07T15:11:21.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6668000000000001e-05 2020-08-07T15:11:21.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731504+0000", 2020-08-07T15:11:21.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1112e-05 2020-08-07T15:11:21.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:21.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731515+0000", 2020-08-07T15:11:21.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75121332100000004 2020-08-07T15:11:21.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482729+0000", 2020-08-07T15:11:21.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1810000000000004e-05 2020-08-07T15:11:21.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:21.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482811+0000", 2020-08-07T15:11:21.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018001399999999999 2020-08-07T15:11:21.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:11:21.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482991+0000", 2020-08-07T15:11:21.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030701140000000001 2020-08-07T15:11:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486061+0000", 2020-08-07T15:11:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4499999999999996e-07 2020-08-07T15:11:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486062+0000", 2020-08-07T15:11:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5366999999999998e-05 2020-08-07T15:11:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486147+0000", 2020-08-07T15:11:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056872779999999996 2020-08-07T15:11:21.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491834+0000", 2020-08-07T15:11:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024645100000000002 2020-08-07T15:11:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:11:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1200000000000003e-07 2020-08-07T15:11:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:11:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1451000000000002e-05 2020-08-07T15:11:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492173+0000", 2020-08-07T15:11:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997 2020-08-07T15:11:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31012 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:11:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:11:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 154.232828471, 2020-08-07T15:11:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003, 2020-08-07T15:11:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31012 2020-08-07T15:11:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:11:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:11:21.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6344999999999997e-05 2020-08-07T15:11:21.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:21.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732094+0000", 2020-08-07T15:11:21.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2920000000000001e-06 2020-08-07T15:11:21.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:21.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732100+0000", 2020-08-07T15:11:21.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.189e-05 2020-08-07T15:11:21.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:21.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732112+0000", 2020-08-07T15:11:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3720000000000001e-06 2020-08-07T15:11:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:21.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732118+0000", 2020-08-07T15:11:21.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2514000000000004e-05 2020-08-07T15:11:21.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732191+0000", 2020-08-07T15:11:21.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4610000000000001e-05 2020-08-07T15:11:21.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:21.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732205+0000", 2020-08-07T15:11:21.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75132974100000005 2020-08-07T15:11:21.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:21.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483535+0000", 2020-08-07T15:11:21.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9185999999999996e-05 2020-08-07T15:11:21.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:21.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483624+0000", 2020-08-07T15:11:21.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015683 2020-08-07T15:11:21.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:11:21.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483781+0000", 2020-08-07T15:11:21.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046202889999999996 2020-08-07T15:11:21.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:21.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:21.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:21.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488401+0000", 2020-08-07T15:11:21.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6700000000000002e-07 2020-08-07T15:11:21.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:22.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488402+0000", 2020-08-07T15:11:22.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1487000000000001e-05 2020-08-07T15:11:22.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:22.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488484+0000", 2020-08-07T15:11:22.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035520830000000002 2020-08-07T15:11:22.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:22.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492036+0000", 2020-08-07T15:11:22.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029266600000000002 2020-08-07T15:11:22.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:22.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492328+0000", 2020-08-07T15:11:22.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5099999999999998e-07 2020-08-07T15:11:22.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:22.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492329+0000", 2020-08-07T15:11:22.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000201299 2020-08-07T15:11:22.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:22.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492531+0000", 2020-08-07T15:11:22.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003 2020-08-07T15:11:22.032 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:22.032 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:22.033 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:22.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.033 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31013 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:11:22.033 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:11:22.034 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 154.23261683999999, 2020-08-07T15:11:22.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996, 2020-08-07T15:11:22.034 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:22.034 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:22.034 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:22.035 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:22.035 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:22.035 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31013 2020-08-07T15:11:22.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.035 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:22.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:22.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:11:22.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:22.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:22.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:11:22.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8949000000000002e-05 2020-08-07T15:11:22.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:22.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732298+0000", 2020-08-07T15:11:22.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1399999999999997e-06 2020-08-07T15:11:22.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:22.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732304+0000", 2020-08-07T15:11:22.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.154e-05 2020-08-07T15:11:22.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:22.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732315+0000", 2020-08-07T15:11:22.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1319999999999999e-06 2020-08-07T15:11:22.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:22.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732322+0000", 2020-08-07T15:11:22.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7431e-05 2020-08-07T15:11:22.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:22.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732369+0000", 2020-08-07T15:11:22.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0621999999999999e-05 2020-08-07T15:11:22.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:22.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732380+0000", 2020-08-07T15:11:22.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75196845999999995 2020-08-07T15:11:22.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:22.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484348+0000", 2020-08-07T15:11:22.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000123008 2020-08-07T15:11:22.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:22.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484471+0000", 2020-08-07T15:11:22.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017572700000000001 2020-08-07T15:11:22.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:11:22.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484647+0000", 2020-08-07T15:11:22.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038557650000000001 2020-08-07T15:11:22.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:22.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:11:22.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2020-08-07T15:11:22.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:22.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:11:22.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0714e-05 2020-08-07T15:11:22.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:22.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488584+0000", 2020-08-07T15:11:22.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036298319999999999 2020-08-07T15:11:22.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:22.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492214+0000", 2020-08-07T15:11:22.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039530099999999997 2020-08-07T15:11:22.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:22.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492609+0000", 2020-08-07T15:11:22.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5300000000000002e-07 2020-08-07T15:11:22.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:22.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492610+0000", 2020-08-07T15:11:22.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197321 2020-08-07T15:11:22.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:22.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492807+0000", 2020-08-07T15:11:22.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996 2020-08-07T15:11:22.071 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:22.075 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:22.076 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:22.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.076 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31014 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:11:22.076 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:11:22.077 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 154.231234696, 2020-08-07T15:11:22.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998, 2020-08-07T15:11:22.077 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:22.077 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:22.077 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:22.078 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:22.099 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:22.100 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31014 2020-08-07T15:11:22.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.108 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:22.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:22.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:11:22.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:22.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:22.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:11:22.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3743999999999999e-05 2020-08-07T15:11:22.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:22.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733685+0000", 2020-08-07T15:11:22.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1720000000000007e-06 2020-08-07T15:11:22.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:22.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733693+0000", 2020-08-07T15:11:22.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3674e-05 2020-08-07T15:11:22.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:22.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733707+0000", 2020-08-07T15:11:22.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9249999999999998e-06 2020-08-07T15:11:22.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:22.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733714+0000", 2020-08-07T15:11:22.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8338000000000003e-05 2020-08-07T15:11:22.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:22.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733772+0000", 2020-08-07T15:11:22.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3019e-05 2020-08-07T15:11:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733785+0000", 2020-08-07T15:11:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75134856299999997 2020-08-07T15:11:22.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:22.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485133+0000", 2020-08-07T15:11:22.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8739999999999995e-05 2020-08-07T15:11:22.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:22.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485212+0000", 2020-08-07T15:11:22.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151244 2020-08-07T15:11:22.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:11:22.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485363+0000", 2020-08-07T15:11:22.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032929090000000001 2020-08-07T15:11:22.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:22.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488656+0000", 2020-08-07T15:11:22.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.065e-06 2020-08-07T15:11:22.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:22.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488657+0000", 2020-08-07T15:11:22.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1123000000000003e-05 2020-08-07T15:11:22.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:22.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488729+0000", 2020-08-07T15:11:22.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035508330000000002 2020-08-07T15:11:22.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:22.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492279+0000", 2020-08-07T15:11:22.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00059376000000000003 2020-08-07T15:11:22.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:22.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492873+0000", 2020-08-07T15:11:22.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5700000000000002e-07 2020-08-07T15:11:22.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:22.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492874+0000", 2020-08-07T15:11:22.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019264 2020-08-07T15:11:22.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:22.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493066+0000", 2020-08-07T15:11:22.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998 2020-08-07T15:11:22.260 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:22.260 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:22.260 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:22.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.261 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31015 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:11:22.261 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:11:22.277 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 154.23110872999999, 2020-08-07T15:11:22.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002, 2020-08-07T15:11:22.278 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:22.278 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:22.278 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:22.278 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:22.279 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:22.279 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31015 2020-08-07T15:11:22.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.279 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:22.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:22.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:11:22.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:22.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:22.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:11:22.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3707000000000001e-05 2020-08-07T15:11:22.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:22.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733791+0000", 2020-08-07T15:11:22.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8120000000000001e-06 2020-08-07T15:11:22.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:22.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733794+0000", 2020-08-07T15:11:22.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0728e-05 2020-08-07T15:11:22.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:22.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733805+0000", 2020-08-07T15:11:22.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8279999999999999e-06 2020-08-07T15:11:22.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:22.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733811+0000", 2020-08-07T15:11:22.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2955999999999999e-05 2020-08-07T15:11:22.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:22.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733864+0000", 2020-08-07T15:11:22.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1389000000000001e-05 2020-08-07T15:11:22.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:22.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733875+0000", 2020-08-07T15:11:22.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75232711100000005 2020-08-07T15:11:22.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:22.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486202+0000", 2020-08-07T15:11:22.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027503400000000001 2020-08-07T15:11:22.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:22.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486478+0000", 2020-08-07T15:11:22.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033010400000000001 2020-08-07T15:11:22.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:11:22.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486808+0000", 2020-08-07T15:11:22.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019382609999999999 2020-08-07T15:11:22.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:22.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488746+0000", 2020-08-07T15:11:22.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9400000000000004e-07 2020-08-07T15:11:22.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:22.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488747+0000", 2020-08-07T15:11:22.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4324000000000002e-05 2020-08-07T15:11:22.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:22.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488821+0000", 2020-08-07T15:11:22.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043745640000000001 2020-08-07T15:11:22.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:22.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493196+0000", 2020-08-07T15:11:22.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030381000000000001 2020-08-07T15:11:22.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:22.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493499+0000", 2020-08-07T15:11:22.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4e-07 2020-08-07T15:11:22.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:22.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493500+0000", 2020-08-07T15:11:22.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017499 2020-08-07T15:11:22.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:22.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493675+0000", 2020-08-07T15:11:22.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002 2020-08-07T15:11:22.472 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:22.472 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:22.472 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:22.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.473 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31016 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:11:22.473 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:11:22.473 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 154.23051275500001, 2020-08-07T15:11:22.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971, 2020-08-07T15:11:22.476 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:22.476 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:22.476 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:22.477 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:22.477 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:22.477 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31016 2020-08-07T15:11:22.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.495 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:22.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:22.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:11:22.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:22.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:22.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:11:22.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0924000000000002e-05 2020-08-07T15:11:22.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:22.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734404+0000", 2020-08-07T15:11:22.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2020-08-07T15:11:22.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:22.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734410+0000", 2020-08-07T15:11:22.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3459999999999994e-06 2020-08-07T15:11:22.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:22.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734418+0000", 2020-08-07T15:11:22.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9640000000000002e-06 2020-08-07T15:11:22.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:22.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734423+0000", 2020-08-07T15:11:22.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7253999999999999e-05 2020-08-07T15:11:22.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:22.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734470+0000", 2020-08-07T15:11:22.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0249999999999997e-06 2020-08-07T15:11:22.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:22.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734477+0000", 2020-08-07T15:11:22.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75294592900000001 2020-08-07T15:11:22.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:22.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487423+0000", 2020-08-07T15:11:22.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031604300000000001 2020-08-07T15:11:22.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:22.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487739+0000", 2020-08-07T15:11:22.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000171149 2020-08-07T15:11:22.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:11:22.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487910+0000", 2020-08-07T15:11:22.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024207769999999998 2020-08-07T15:11:22.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:22.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490331+0000", 2020-08-07T15:11:22.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1650000000000001e-06 2020-08-07T15:11:22.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:22.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490332+0000", 2020-08-07T15:11:22.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9796999999999996e-05 2020-08-07T15:11:22.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:22.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490402+0000", 2020-08-07T15:11:22.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029928849999999998 2020-08-07T15:11:22.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:22.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493395+0000", 2020-08-07T15:11:22.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034903199999999999 2020-08-07T15:11:22.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:22.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493744+0000", 2020-08-07T15:11:22.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8899999999999999e-07 2020-08-07T15:11:22.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:22.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493745+0000", 2020-08-07T15:11:22.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197362 2020-08-07T15:11:22.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:22.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493942+0000", 2020-08-07T15:11:22.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971 2020-08-07T15:11:22.569 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:22.569 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:22.569 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:22.569 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:22.569 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:22.574 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:11:22.576 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:11:22.578 INFO:tasks.ceph.mgr.y.smithi074.stderr:2020-08-07T15:11:21.001+0000 7fc8841ec700 -1 mgr.server reply reply (1) Operation not permitted no change 2020-08-07T15:11:22.579 INFO:teuthology.orchestra.run.smithi074.stdout:moved 1 / 165 (0.606061%) 2020-08-07T15:11:22.579 INFO:teuthology.orchestra.run.smithi074.stdout:avg 23.5714 2020-08-07T15:11:22.579 INFO:teuthology.orchestra.run.smithi074.stdout:stddev 4.04313 -> 4.15024 (expected baseline 4.49489) 2020-08-07T15:11:22.579 INFO:teuthology.orchestra.run.smithi074.stdout:min osd.2 with 16 -> 16 pgs (0.678788 -> 0.678788 * mean) 2020-08-07T15:11:22.580 INFO:teuthology.orchestra.run.smithi074.stdout:max osd.7 with 25 -> 25 pgs (1.06061 -> 1.06061 * mean) 2020-08-07T15:11:22.580 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:11:22.580 INFO:teuthology.orchestra.run.smithi074.stdout:oload 110 2020-08-07T15:11:22.580 INFO:teuthology.orchestra.run.smithi074.stdout:max_change 0.05 2020-08-07T15:11:22.580 INFO:teuthology.orchestra.run.smithi074.stdout:max_change_osds 4 2020-08-07T15:11:22.581 INFO:teuthology.orchestra.run.smithi074.stdout:average_utilization 0.0137 2020-08-07T15:11:22.581 INFO:teuthology.orchestra.run.smithi074.stdout:overload_utilization 0.0150 2020-08-07T15:11:22.581 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5 weight 0.8567 -> 0.8067 2020-08-07T15:11:22.581 INFO:teuthology.orchestra.run.smithi074.stderr:no change 2020-08-07T15:11:22.584 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2020-08-07T15:11:22.585 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd test-reweight-by-utilization 1000 0.05 2020-08-07T15:11:22.585 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 61 16 89424 89408 11.4497 0.0078125 0.629951 0.0106282 2020-08-07T15:11:22.586 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 62 16 89696 89680 11.2993 2.125 0.455663 0.01095 2020-08-07T15:11:22.586 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:22.575+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:22.587 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:11:22.724 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:22.721+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:22.786 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:11:22.787 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:11:22.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.787 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:90384 5.e 5:76ac0adc:::benchmark_data_smithi074_53697_object11297:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e208)", 2020-08-07T15:11:22.787 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:22.717096+0000", 2020-08-07T15:11:22.805 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.067983245999999997, 2020-08-07T15:11:22.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.068005525999999997, 2020-08-07T15:11:22.806 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:22.806 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:22.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:22.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:11:22.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:22.807 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 90384 2020-08-07T15:11:22.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.807 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:22.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:22.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.717096+0000", 2020-08-07T15:11:22.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:22.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:22.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.717096+0000", 2020-08-07T15:11:22.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7222999999999998e-05 2020-08-07T15:11:22.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:22.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.717143+0000", 2020-08-07T15:11:22.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8860000000000001e-06 2020-08-07T15:11:22.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.717149+0000", 2020-08-07T15:11:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1243999999999999e-05 2020-08-07T15:11:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.717160+0000", 2020-08-07T15:11:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1889999999999998e-06 2020-08-07T15:11:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.717166+0000", 2020-08-07T15:11:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014897140000000001 2020-08-07T15:11:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.718655+0000", 2020-08-07T15:11:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010166900000000001 2020-08-07T15:11:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.718757+0000", 2020-08-07T15:11:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000163349 2020-08-07T15:11:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:11:22.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.718920+0000", 2020-08-07T15:11:22.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014901280000000001 2020-08-07T15:11:22.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:22.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.720410+0000", 2020-08-07T15:11:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8100000000000001e-07 2020-08-07T15:11:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.720411+0000", 2020-08-07T15:11:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8904000000000003e-05 2020-08-07T15:11:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.720480+0000", 2020-08-07T15:11:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033841869999999999 2020-08-07T15:11:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:90472 5.d e208/164 5:b24c7ec3:::benchmark_data_smithi074_53697_object11308:head v 208'5704, mlcod=208'5697)", 2020-08-07T15:11:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:22.782811+0000", 2020-08-07T15:11:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0022678500000000001, 2020-08-07T15:11:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023587209999999998, 2020-08-07T15:11:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:11:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:23.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:23.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.782811+0000", 2020-08-07T15:11:23.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:23.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:23.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:23.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:23.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.782811+0000", 2020-08-07T15:11:23.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6242999999999997e-05 2020-08-07T15:11:23.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:23.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:23.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:23.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.782868+0000", 2020-08-07T15:11:23.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4060000000000002e-06 2020-08-07T15:11:23.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:23.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:23.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:23.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.782874+0000", 2020-08-07T15:11:23.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7957e-05 2020-08-07T15:11:23.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:23.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:23.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:23.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.782902+0000", 2020-08-07T15:11:23.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4410000000000001e-06 2020-08-07T15:11:23.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:23.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:23.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:23.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.782907+0000", 2020-08-07T15:11:23.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0991e-05 2020-08-07T15:11:23.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:23.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:23.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:23.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.782958+0000", 2020-08-07T15:11:23.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0377000000000002e-05 2020-08-07T15:11:23.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:23.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:23.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:23.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.782979+0000", 2020-08-07T15:11:23.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000167415 2020-08-07T15:11:23.084 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:23.084 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:23.084 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:23.085 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:23.085 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:11:23.085 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 2 2020-08-07T15:11:23.085 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:11:23.112 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:11:23.115 INFO:teuthology.orchestra.run.smithi074.stdout:moved 0 / 165 (0%) 2020-08-07T15:11:23.116 INFO:teuthology.orchestra.run.smithi074.stdout:avg 23.5714 2020-08-07T15:11:23.116 INFO:teuthology.orchestra.run.smithi074.stdout:stddev 4.04313 -> 4.04313 (expected baseline 4.49489) 2020-08-07T15:11:23.116 INFO:teuthology.orchestra.run.smithi074.stdout:min osd.2 with 16 -> 16 pgs (0.678788 -> 0.678788 * mean) 2020-08-07T15:11:23.116 INFO:teuthology.orchestra.run.smithi074.stdout:max osd.7 with 25 -> 25 pgs (1.06061 -> 1.06061 * mean) 2020-08-07T15:11:23.117 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:11:23.117 INFO:teuthology.orchestra.run.smithi074.stdout:oload 1000 2020-08-07T15:11:23.117 INFO:teuthology.orchestra.run.smithi074.stdout:max_change 0.05 2020-08-07T15:11:23.117 INFO:teuthology.orchestra.run.smithi074.stdout:max_change_osds 4 2020-08-07T15:11:23.117 INFO:teuthology.orchestra.run.smithi074.stdout:average_utilization 0.0138 2020-08-07T15:11:23.118 INFO:teuthology.orchestra.run.smithi074.stdout:overload_utilization 0.1381 2020-08-07T15:11:23.118 INFO:teuthology.orchestra.run.smithi074.stderr:no change 2020-08-07T15:11:23.118 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:23.111+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:23.119 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:11:23.217 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 63 16 91088 91072 11.2925 10.875 0.00442163 0.0110644 2020-08-07T15:11:23.219 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:23.214+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:23.223 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:11:23.322 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:11:23.323 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:11:23.323 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:11:23.323 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:11:23.323 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:11:23.324 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:23.321+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:23.331 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:11:23.371 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:11:23.401 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:11:23.415 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:11:23.456 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:23.453+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:23.498 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:11:23.531 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:23.531 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:23.531 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:11:23.541 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:11:23.558 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:23.554+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:23.566 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:11:23.602 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:11:23.603 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:11:23.603 INFO:teuthology.orchestra.run.smithi074.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-08-07T15:11:23.603 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: problem getting command descriptions from osd.1 2020-08-07T15:11:23.604 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:11:23.604 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:11:23.604 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:11:23.604 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:11:23.604 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:11:23.614 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:11:23.615 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-08-07T15:11:23.620 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-08-07T15:11:23.659 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:23.657+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:23.780 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:23.781 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:23.781 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:11:23.822 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:23.820+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:23.890 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:23.889+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:23.990 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:23.991 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:23.991 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:23.989+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:24.005 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:11:24.110 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:24.103+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:24.212 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:24.205+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:24.215 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 64 16 92453 92437 11.2827 10.6641 0.00647875 0.0110739 2020-08-07T15:11:24.229 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:24.229 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:24.264 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:11:24.316 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:24.309+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:24.416 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:24.412+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:24.627 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:24.628 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:24.628 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:11:24.629 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:24.620+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:24.723 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:24.718+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:24.822 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:24.823 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:24.823 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:24.822+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:24.833 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:11:24.941 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:24.940+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:25.052 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:25.052 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:25.053 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:25.049+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:25.057 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:11:25.164 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:25.163+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:25.217 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 65 16 93897 93881 11.2827 11.2812 0.00598759 0.0110724 2020-08-07T15:11:25.243 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:25.247 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:25.248 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:11:25.251 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:11:25.286 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:25.285+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:25.396 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:25.386+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:25.471 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:25.471 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:25.484 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:11:25.501 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:25.500+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:25.606 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:25.602+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:25.621 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:11:25.690 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:25.690 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:25.701 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:11:25.713 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:25.712+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:25.821 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:25.812+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:25.842 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:11:25.891 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:11:25.892 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:11:25.892 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:11:25.892 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:11:25.893 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:11:25.909 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:11:25.916 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:25.914+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:25.935 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:11:25.967 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:11:26.019 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:11:26.019 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:26.018+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:26.050 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:11:26.082 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:11:26.083 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:11:26.083 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:11:26.083 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:11:26.083 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:11:26.084 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:11:26.085 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:11:26.085 INFO:teuthology.orchestra.run.smithi074.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-08-07T15:11:26.085 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: problem getting command descriptions from osd.1 2020-08-07T15:11:26.095 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:11:26.095 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:11:26.095 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:11:26.095 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:11:26.096 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:11:26.096 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:11:26.097 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-08-07T15:11:26.098 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:11:26.098 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-08-07T15:11:26.125 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:26.123+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:26.215 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 66 15 95403 95388 11.2901 11.7734 0.00747996 0.011067 2020-08-07T15:11:26.227 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:26.225+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:26.313 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:11:26.313 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:11:26.314 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:11:26.314 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:11:26.314 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.314 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:30968 4.4 e98/81 ondisk, result = 0)", 2020-08-07T15:11:26.315 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:11:26.315 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 160.577365826, 2020-08-07T15:11:26.317 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96211418999999998, 2020-08-07T15:11:26.317 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:26.318 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:11:26.318 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:26.318 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.318 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:26.319 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:11:26.319 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:26.319 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.319 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.319 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:26.320 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:11:26.320 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2387e-05 2020-08-07T15:11:26.332 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.333 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.333 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:26.333 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729821+0000", 2020-08-07T15:11:26.333 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9909999999999999e-06 2020-08-07T15:11:26.333 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.334 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.334 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:26.334 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729827+0000", 2020-08-07T15:11:26.334 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96185524600000005 2020-08-07T15:11:26.334 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.335 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:26.336 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691682+0000", 2020-08-07T15:11:26.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8760000000000005e-06 2020-08-07T15:11:26.336 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.337 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.337 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:26.337 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691690+0000", 2020-08-07T15:11:26.338 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9682000000000001e-05 2020-08-07T15:11:26.338 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.338 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.338 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:26.338 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691759+0000", 2020-08-07T15:11:26.338 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5381000000000001e-05 2020-08-07T15:11:26.339 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.339 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.339 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:26.339 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691775+0000", 2020-08-07T15:11:26.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000127627 2020-08-07T15:11:26.340 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.340 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:26.340 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691902+0000", 2020-08-07T15:11:26.347 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96211418999999998 2020-08-07T15:11:26.348 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:26.348 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:26.348 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:26.348 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.348 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.349 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42472 4.4 4:2ff82af8:::benchmark_data_smithi074_44586_object5308:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:11:26.349 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:11:26.349 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 146.039903052, 2020-08-07T15:11:26.349 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89653816399999997, 2020-08-07T15:11:26.349 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:26.350 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:26.350 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:26.350 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:26.350 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:26.350 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42472 2020-08-07T15:11:26.351 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.351 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:26.351 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.351 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:26.351 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:11:26.352 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:26.352 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.352 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.352 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:26.365 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:11:26.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9591e-05 2020-08-07T15:11:26.366 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.366 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:26.366 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267301+0000", 2020-08-07T15:11:26.366 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7490000000000004e-06 2020-08-07T15:11:26.367 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.367 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.367 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:26.367 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267306+0000", 2020-08-07T15:11:26.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2850000000000001e-05 2020-08-07T15:11:26.379 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:26.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267319+0000", 2020-08-07T15:11:26.380 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9100000000000002e-06 2020-08-07T15:11:26.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:26.380 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267325+0000", 2020-08-07T15:11:26.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6584000000000001e-05 2020-08-07T15:11:26.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:26.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267362+0000", 2020-08-07T15:11:26.382 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9188000000000001e-05 2020-08-07T15:11:26.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.382 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.382 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:26.382 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267451+0000", 2020-08-07T15:11:26.382 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000182917 2020-08-07T15:11:26.389 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.389 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.389 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:11:26.389 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267634+0000", 2020-08-07T15:11:26.390 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002484708 2020-08-07T15:11:26.390 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.390 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.390 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:26.391 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270119+0000", 2020-08-07T15:11:26.391 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0299999999999997e-07 2020-08-07T15:11:26.391 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.391 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.391 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:26.391 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270119+0000", 2020-08-07T15:11:26.392 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8251999999999996e-05 2020-08-07T15:11:26.392 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.396 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.396 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:26.396 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270188+0000", 2020-08-07T15:11:26.396 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24443688 2020-08-07T15:11:26.398 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.399 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:26.399 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.514625+0000", 2020-08-07T15:11:26.399 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.64911260599999998 2020-08-07T15:11:26.400 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.400 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:26.400 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163737+0000", 2020-08-07T15:11:26.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2100000000000001e-06 2020-08-07T15:11:26.401 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.401 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.401 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:26.401 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163738+0000", 2020-08-07T15:11:26.401 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0816000000000003e-05 2020-08-07T15:11:26.406 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.406 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.406 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:26.407 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163789+0000", 2020-08-07T15:11:26.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89653816399999997 2020-08-07T15:11:26.407 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:26.407 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:26.407 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:26.408 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.408 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.408 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:42472 4.4 e112/81 ondisk, result = 0)", 2020-08-07T15:11:26.408 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.271353+0000", 2020-08-07T15:11:26.408 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 146.035801015, 2020-08-07T15:11:26.409 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89244306299999998, 2020-08-07T15:11:26.409 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:26.409 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:11:26.409 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:26.409 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.410 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:26.410 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.271353+0000", 2020-08-07T15:11:26.410 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:26.410 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.410 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.410 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:26.411 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.271353+0000", 2020-08-07T15:11:26.411 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7875999999999999e-05 2020-08-07T15:11:26.411 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.411 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.411 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:26.412 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.271381+0000", 2020-08-07T15:11:26.412 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4819999999999996e-06 2020-08-07T15:11:26.412 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.412 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.412 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:26.413 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.271385+0000", 2020-08-07T15:11:26.413 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89228763300000002 2020-08-07T15:11:26.413 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.413 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:26.414 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163673+0000", 2020-08-07T15:11:26.414 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9949999999999996e-06 2020-08-07T15:11:26.414 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.414 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.449 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:26.449 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163678+0000", 2020-08-07T15:11:26.449 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5485000000000003e-05 2020-08-07T15:11:26.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.460 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:26.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163714+0000", 2020-08-07T15:11:26.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4785e-05 2020-08-07T15:11:26.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.461 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.461 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:26.461 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163728+0000", 2020-08-07T15:11:26.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7806999999999996e-05 2020-08-07T15:11:26.461 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.462 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.462 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:26.462 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163796+0000", 2020-08-07T15:11:26.462 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89244306299999998 2020-08-07T15:11:26.462 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:26.463 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:26.463 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:26.463 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.463 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.463 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72688 4.4 4:295d3f6e:::benchmark_data_smithi074_44586_object9085:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e129)", 2020-08-07T15:11:26.463 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:11:26.464 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 120.910320865, 2020-08-07T15:11:26.464 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7869782759999999, 2020-08-07T15:11:26.464 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:26.464 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:11:26.464 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:26.465 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:26.467 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:26.468 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72688 2020-08-07T15:11:26.468 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.473 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:26.473 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.474 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:26.474 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:11:26.474 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:26.474 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.474 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.475 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:26.475 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:11:26.475 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8721999999999999e-05 2020-08-07T15:11:26.475 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.475 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.476 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:26.518 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396882+0000", 2020-08-07T15:11:26.518 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2639999999999999e-06 2020-08-07T15:11:26.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.519 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.519 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:26.519 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396887+0000", 2020-08-07T15:11:26.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0767999999999999e-05 2020-08-07T15:11:26.537 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.537 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.537 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:26.563 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396898+0000", 2020-08-07T15:11:26.563 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1860000000000002e-06 2020-08-07T15:11:26.564 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.564 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:26.565 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396902+0000", 2020-08-07T15:11:26.565 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0037638649999999999 2020-08-07T15:11:26.566 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.567 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.567 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:26.568 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400666+0000", 2020-08-07T15:11:26.568 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000118077 2020-08-07T15:11:26.569 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.569 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.569 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:26.570 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400784+0000", 2020-08-07T15:11:26.580 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000136728 2020-08-07T15:11:26.580 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.580 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.581 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:11:26.581 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400921+0000", 2020-08-07T15:11:26.582 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001595443 2020-08-07T15:11:26.582 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.583 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.583 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:26.583 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.402516+0000", 2020-08-07T15:11:26.584 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.014817907 2020-08-07T15:11:26.584 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.584 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.585 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:26.585 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417334+0000", 2020-08-07T15:11:26.595 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.75e-06 2020-08-07T15:11:26.596 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.596 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.597 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:26.597 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417336+0000", 2020-08-07T15:11:26.597 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8066999999999996e-05 2020-08-07T15:11:26.599 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.599 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.599 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:26.600 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417424+0000", 2020-08-07T15:11:26.601 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002210573 2020-08-07T15:11:26.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.602 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:26.602 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.419635+0000", 2020-08-07T15:11:26.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1379000000000001e-05 2020-08-07T15:11:26.603 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:26.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.419646+0000", 2020-08-07T15:11:26.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.004643409 2020-08-07T15:11:26.605 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.605 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.605 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:26.606 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424289+0000", 2020-08-07T15:11:26.606 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7563000000000001e-05 2020-08-07T15:11:26.606 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.607 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.607 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:26.607 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424307+0000", 2020-08-07T15:11:26.608 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75943629400000001 2020-08-07T15:11:26.610 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.610 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.610 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:26.612 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183743+0000", 2020-08-07T15:11:26.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8281000000000006e-05 2020-08-07T15:11:26.612 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.613 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:26.613 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183812+0000", 2020-08-07T15:11:26.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7869782759999999 2020-08-07T15:11:26.643 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:26.643 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:26.644 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:26.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.645 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72985 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:11:26.663 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:11:26.664 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 120.69433504600001, 2020-08-07T15:11:26.665 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869466839999999, 2020-08-07T15:11:26.665 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:26.665 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:26.666 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:26.666 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:26.667 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:26.667 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72985 2020-08-07T15:11:26.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.668 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:26.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:26.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:11:26.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:26.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.675 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.676 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:26.676 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:11:26.676 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8003999999999997e-05 2020-08-07T15:11:26.677 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:26.677 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612857+0000", 2020-08-07T15:11:26.678 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.246e-06 2020-08-07T15:11:26.678 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.681 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:26.682 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612863+0000", 2020-08-07T15:11:26.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5146e-05 2020-08-07T15:11:26.683 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.683 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.684 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:26.685 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612878+0000", 2020-08-07T15:11:26.685 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1940000000000003e-06 2020-08-07T15:11:26.685 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.686 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.686 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:26.686 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612885+0000", 2020-08-07T15:11:26.687 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0349999999999997e-05 2020-08-07T15:11:26.687 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.687 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.688 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:26.688 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612935+0000", 2020-08-07T15:11:26.688 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1029999999999996e-06 2020-08-07T15:11:26.689 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.689 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.690 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:26.691 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612942+0000", 2020-08-07T15:11:26.691 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81140097700000002 2020-08-07T15:11:26.691 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.691 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.692 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:26.692 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424343+0000", 2020-08-07T15:11:26.693 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1799999999999996e-06 2020-08-07T15:11:26.693 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.694 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.694 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:26.695 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424351+0000", 2020-08-07T15:11:26.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75948665299999996 2020-08-07T15:11:26.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:26.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183838+0000", 2020-08-07T15:11:26.698 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000179187 2020-08-07T15:11:26.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.698 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.699 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:26.699 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.184017+0000", 2020-08-07T15:11:26.700 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024691600000000002 2020-08-07T15:11:26.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.705 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.706 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:11:26.706 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.184264+0000", 2020-08-07T15:11:26.706 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006362721 2020-08-07T15:11:26.707 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.707 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:26.708 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190627+0000", 2020-08-07T15:11:26.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7400000000000001e-06 2020-08-07T15:11:26.708 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.709 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.709 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:26.709 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190629+0000", 2020-08-07T15:11:26.710 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1305999999999999e-05 2020-08-07T15:11:26.710 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.711 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:26.711 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190690+0000", 2020-08-07T15:11:26.712 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0088507159999999998 2020-08-07T15:11:26.714 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.715 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.715 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:26.715 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199541+0000", 2020-08-07T15:11:26.716 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015864699999999999 2020-08-07T15:11:26.717 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.717 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.717 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:26.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199699+0000", 2020-08-07T15:11:26.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.23e-07 2020-08-07T15:11:26.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:26.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199700+0000", 2020-08-07T15:11:26.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5775e-05 2020-08-07T15:11:26.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.722 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:26.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199766+0000", 2020-08-07T15:11:26.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869466839999999 2020-08-07T15:11:26.723 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:26.724 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:26.735 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:26.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.736 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.736 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72986 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:11:26.737 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:11:26.737 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 120.694224464, 2020-08-07T15:11:26.742 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.586946441, 2020-08-07T15:11:26.742 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:26.742 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:26.743 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:26.743 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:26.743 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:26.743 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72986 2020-08-07T15:11:26.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.744 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:26.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:26.744 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:11:26.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:26.757 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.757 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:26.757 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:11:26.758 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9861999999999997e-05 2020-08-07T15:11:26.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.760 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.760 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:26.760 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612990+0000", 2020-08-07T15:11:26.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4350000000000001e-06 2020-08-07T15:11:26.761 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.762 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:26.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612993+0000", 2020-08-07T15:11:26.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9529999999999999e-06 2020-08-07T15:11:26.763 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.763 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.764 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:26.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613001+0000", 2020-08-07T15:11:26.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4309999999999998e-06 2020-08-07T15:11:26.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.766 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:26.766 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613004+0000", 2020-08-07T15:11:26.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5284000000000001e-05 2020-08-07T15:11:26.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:26.768 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613030+0000", 2020-08-07T15:11:26.768 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2400000000000001e-06 2020-08-07T15:11:26.768 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.769 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.770 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:26.771 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613034+0000", 2020-08-07T15:11:26.771 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81134136199999995 2020-08-07T15:11:26.771 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.772 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.772 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:26.772 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424375+0000", 2020-08-07T15:11:26.772 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3520000000000001e-06 2020-08-07T15:11:26.773 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.773 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.774 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:26.774 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424383+0000", 2020-08-07T15:11:26.775 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76064437799999995 2020-08-07T15:11:26.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:26.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185027+0000", 2020-08-07T15:11:26.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000119719 2020-08-07T15:11:26.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:26.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185147+0000", 2020-08-07T15:11:26.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016829400000000001 2020-08-07T15:11:26.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.782 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:11:26.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185315+0000", 2020-08-07T15:11:26.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0053821199999999998 2020-08-07T15:11:26.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:26.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190697+0000", 2020-08-07T15:11:26.786 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6899999999999996e-07 2020-08-07T15:11:26.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:26.787 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190698+0000", 2020-08-07T15:11:26.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2072000000000001e-05 2020-08-07T15:11:26.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.788 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:26.789 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190730+0000", 2020-08-07T15:11:26.789 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0088501440000000008 2020-08-07T15:11:26.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.792 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:26.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199580+0000", 2020-08-07T15:11:26.793 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00023199900000000001 2020-08-07T15:11:26.793 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.793 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.794 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:26.794 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199812+0000", 2020-08-07T15:11:26.794 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6300000000000005e-07 2020-08-07T15:11:26.795 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.795 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.795 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:26.796 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199813+0000", 2020-08-07T15:11:26.796 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3363999999999998e-05 2020-08-07T15:11:26.796 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.797 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:26.797 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199876+0000", 2020-08-07T15:11:26.797 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.586946441 2020-08-07T15:11:26.798 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:26.798 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:26.798 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:26.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.799 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72987 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:11:26.800 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:11:26.800 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 120.694128709, 2020-08-07T15:11:26.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869598709999999, 2020-08-07T15:11:26.802 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:26.803 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:26.803 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:26.803 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:26.803 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:26.804 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72987 2020-08-07T15:11:26.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.804 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:26.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:26.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:11:26.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:26.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:26.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:11:26.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1851999999999999e-05 2020-08-07T15:11:26.806 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:26.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613057+0000", 2020-08-07T15:11:26.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0429999999999999e-06 2020-08-07T15:11:26.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:26.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613060+0000", 2020-08-07T15:11:26.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1060000000000002e-06 2020-08-07T15:11:26.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.808 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.809 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:26.809 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613067+0000", 2020-08-07T15:11:26.809 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8040000000000002e-06 2020-08-07T15:11:26.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.809 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:26.810 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613070+0000", 2020-08-07T15:11:26.810 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6857000000000001e-05 2020-08-07T15:11:26.810 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.810 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.810 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:26.811 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613097+0000", 2020-08-07T15:11:26.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8770000000000001e-06 2020-08-07T15:11:26.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.811 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:26.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613102+0000", 2020-08-07T15:11:26.814 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81131066399999996 2020-08-07T15:11:26.814 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.815 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:26.815 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424413+0000", 2020-08-07T15:11:26.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4599999999999997e-06 2020-08-07T15:11:26.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:26.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424420+0000", 2020-08-07T15:11:26.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76132809700000004 2020-08-07T15:11:26.816 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.817 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:26.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185748+0000", 2020-08-07T15:11:26.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.4264000000000005e-05 2020-08-07T15:11:26.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.818 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.818 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:26.818 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185832+0000", 2020-08-07T15:11:26.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112803 2020-08-07T15:11:26.818 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:11:26.819 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185945+0000", 2020-08-07T15:11:26.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0089954130000000007 2020-08-07T15:11:26.819 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:26.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194941+0000", 2020-08-07T15:11:26.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.091e-06 2020-08-07T15:11:26.821 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:26.821 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194943+0000", 2020-08-07T15:11:26.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7423999999999996e-05 2020-08-07T15:11:26.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.822 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:26.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195020+0000", 2020-08-07T15:11:26.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045973300000000002 2020-08-07T15:11:26.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:26.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199618+0000", 2020-08-07T15:11:26.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000302009 2020-08-07T15:11:26.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:26.826 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199920+0000", 2020-08-07T15:11:26.826 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7999999999999996e-07 2020-08-07T15:11:26.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.827 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:26.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199920+0000", 2020-08-07T15:11:26.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5296999999999995e-05 2020-08-07T15:11:26.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:26.828 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199985+0000", 2020-08-07T15:11:26.828 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869598709999999 2020-08-07T15:11:26.829 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:26.829 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:26.829 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:26.829 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.829 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.854 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72988 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:11:26.855 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:11:26.855 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 120.694064129, 2020-08-07T15:11:26.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587111143, 2020-08-07T15:11:26.862 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:26.862 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:26.862 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:26.862 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:26.862 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:26.863 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72988 2020-08-07T15:11:26.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.894 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:26.894 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.894 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:26.894 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:11:26.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:26.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.906 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.906 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:26.906 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:11:26.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6563e-05 2020-08-07T15:11:26.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:26.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613117+0000", 2020-08-07T15:11:26.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7180000000000001e-06 2020-08-07T15:11:26.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.908 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:26.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613119+0000", 2020-08-07T15:11:26.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1789999999999999e-06 2020-08-07T15:11:26.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:26.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613124+0000", 2020-08-07T15:11:26.920 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3539999999999998e-06 2020-08-07T15:11:26.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:26.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613127+0000", 2020-08-07T15:11:26.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3697e-05 2020-08-07T15:11:26.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.922 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:26.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613151+0000", 2020-08-07T15:11:26.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7559999999999999e-06 2020-08-07T15:11:26.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:26.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613155+0000", 2020-08-07T15:11:26.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81128859600000003 2020-08-07T15:11:26.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:26.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424444+0000", 2020-08-07T15:11:26.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9619999999999996e-06 2020-08-07T15:11:26.925 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:26.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424451+0000", 2020-08-07T15:11:26.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76286712899999998 2020-08-07T15:11:26.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:26.942 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187318+0000", 2020-08-07T15:11:26.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011616300000000001 2020-08-07T15:11:26.942 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.942 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:26.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187434+0000", 2020-08-07T15:11:26.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019347400000000001 2020-08-07T15:11:26.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:11:26.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187628+0000", 2020-08-07T15:11:26.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074049090000000003 2020-08-07T15:11:26.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.944 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:26.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195033+0000", 2020-08-07T15:11:26.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1090000000000001e-06 2020-08-07T15:11:26.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.962 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:26.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195034+0000", 2020-08-07T15:11:26.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8566999999999997e-05 2020-08-07T15:11:26.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:26.964 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195092+0000", 2020-08-07T15:11:26.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004570457 2020-08-07T15:11:26.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:26.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199663+0000", 2020-08-07T15:11:26.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00046757899999999998 2020-08-07T15:11:26.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:26.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200130+0000", 2020-08-07T15:11:26.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6700000000000002e-07 2020-08-07T15:11:26.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.969 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:26.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200131+0000", 2020-08-07T15:11:26.969 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9963999999999996e-05 2020-08-07T15:11:26.969 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.969 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.986 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:26.986 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200201+0000", 2020-08-07T15:11:26.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587111143 2020-08-07T15:11:26.987 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:26.987 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:26.988 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:26.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.989 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72989 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:11:26.989 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:11:26.989 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 120.69400774099999, 2020-08-07T15:11:26.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5872506399999999, 2020-08-07T15:11:26.992 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:26.993 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:26.993 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:26.993 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:26.994 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:26.994 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72989 2020-08-07T15:11:26.994 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.995 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:26.995 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.995 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:26.996 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:11:26.996 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:26.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:26.997 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:26.997 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:27.018 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:11:27.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6081999999999999e-05 2020-08-07T15:11:27.018 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.018 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.019 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:27.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613172+0000", 2020-08-07T15:11:27.019 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6570000000000001e-06 2020-08-07T15:11:27.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.020 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.020 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:27.020 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613175+0000", 2020-08-07T15:11:27.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7740000000000001e-06 2020-08-07T15:11:27.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:27.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613181+0000", 2020-08-07T15:11:27.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2759999999999999e-06 2020-08-07T15:11:27.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:27.022 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613183+0000", 2020-08-07T15:11:27.022 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3892000000000001e-05 2020-08-07T15:11:27.023 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.023 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613207+0000", 2020-08-07T15:11:27.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9870000000000001e-06 2020-08-07T15:11:27.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:27.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613211+0000", 2020-08-07T15:11:27.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81126230700000002 2020-08-07T15:11:27.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424473+0000", 2020-08-07T15:11:27.042 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1460000000000001e-06 2020-08-07T15:11:27.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:27.044 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424480+0000", 2020-08-07T15:11:27.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76413073099999995 2020-08-07T15:11:27.045 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.045 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.062 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188610+0000", 2020-08-07T15:11:27.062 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011278899999999999 2020-08-07T15:11:27.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.063 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:27.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188723+0000", 2020-08-07T15:11:27.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000213422 2020-08-07T15:11:27.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.064 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:11:27.066 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188936+0000", 2020-08-07T15:11:27.067 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0061672929999999999 2020-08-07T15:11:27.067 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.067 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.067 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:27.068 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195104+0000", 2020-08-07T15:11:27.068 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8199999999999999e-07 2020-08-07T15:11:27.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.068 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.068 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:27.069 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195105+0000", 2020-08-07T15:11:27.069 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2056000000000004e-05 2020-08-07T15:11:27.069 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.069 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.069 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:27.073 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195167+0000", 2020-08-07T15:11:27.073 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0051009769999999996 2020-08-07T15:11:27.076 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.076 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.077 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:27.077 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200268+0000", 2020-08-07T15:11:27.077 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9428000000000001e-05 2020-08-07T15:11:27.077 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.077 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.078 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:27.078 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200327+0000", 2020-08-07T15:11:27.078 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9100000000000003e-07 2020-08-07T15:11:27.078 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.079 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.079 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:27.079 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200328+0000", 2020-08-07T15:11:27.079 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9250000000000003e-05 2020-08-07T15:11:27.079 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.080 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.080 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:27.080 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200397+0000", 2020-08-07T15:11:27.080 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5872506399999999 2020-08-07T15:11:27.080 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:27.080 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:27.081 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:27.081 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.086 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.087 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72990 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:11:27.087 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:11:27.087 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 120.693950449, 2020-08-07T15:11:27.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587335105, 2020-08-07T15:11:27.088 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:27.088 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:27.089 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:27.089 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:27.089 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:27.090 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72990 2020-08-07T15:11:27.090 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.090 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:27.091 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.091 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:27.091 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:11:27.092 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:27.092 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.093 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.093 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:27.094 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:11:27.094 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5276999999999999e-05 2020-08-07T15:11:27.094 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.095 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.095 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:27.095 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613229+0000", 2020-08-07T15:11:27.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7219999999999999e-06 2020-08-07T15:11:27.096 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.096 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.097 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:27.097 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613232+0000", 2020-08-07T15:11:27.097 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1170000000000003e-06 2020-08-07T15:11:27.098 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.098 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.098 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:27.099 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613237+0000", 2020-08-07T15:11:27.099 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2309999999999999e-06 2020-08-07T15:11:27.099 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.100 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.100 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:27.100 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613239+0000", 2020-08-07T15:11:27.101 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2469000000000001e-05 2020-08-07T15:11:27.101 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.102 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.102 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.102 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613261+0000", 2020-08-07T15:11:27.103 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9280000000000001e-06 2020-08-07T15:11:27.104 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.104 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.104 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:27.105 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613266+0000", 2020-08-07T15:11:27.105 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81123414599999999 2020-08-07T15:11:27.105 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.106 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.106 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.106 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424501+0000", 2020-08-07T15:11:27.107 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5300000000000002e-06 2020-08-07T15:11:27.107 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.107 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.108 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:27.108 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424507+0000", 2020-08-07T15:11:27.108 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76488660200000003 2020-08-07T15:11:27.109 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.109 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.109 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.110 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189394+0000", 2020-08-07T15:11:27.110 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6878000000000003e-05 2020-08-07T15:11:27.110 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.111 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.111 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:27.111 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189491+0000", 2020-08-07T15:11:27.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000204477 2020-08-07T15:11:27.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:11:27.113 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189695+0000", 2020-08-07T15:11:27.114 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0065180940000000003 2020-08-07T15:11:27.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.115 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.115 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:27.116 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196213+0000", 2020-08-07T15:11:27.116 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4920000000000001e-06 2020-08-07T15:11:27.116 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.117 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:27.117 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196215+0000", 2020-08-07T15:11:27.118 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9950000000000006e-05 2020-08-07T15:11:27.118 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.118 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.119 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:27.119 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196285+0000", 2020-08-07T15:11:27.120 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004163674 2020-08-07T15:11:27.120 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.120 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.121 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:27.121 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200448+0000", 2020-08-07T15:11:27.121 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6919999999999999e-05 2020-08-07T15:11:27.133 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.134 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.134 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:27.134 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200485+0000", 2020-08-07T15:11:27.134 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1699999999999998e-07 2020-08-07T15:11:27.135 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.147 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.147 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:27.147 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200486+0000", 2020-08-07T15:11:27.148 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3081000000000002e-05 2020-08-07T15:11:27.148 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.148 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.149 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:27.149 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200539+0000", 2020-08-07T15:11:27.149 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587335105 2020-08-07T15:11:27.157 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:27.157 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:27.162 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:27.162 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.162 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.163 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72991 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:11:27.163 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:11:27.163 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 120.69389492400001, 2020-08-07T15:11:27.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5876210340000001, 2020-08-07T15:11:27.164 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:27.164 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:27.165 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:27.165 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:27.176 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:27.177 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72991 2020-08-07T15:11:27.177 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.178 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:27.178 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.179 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:27.179 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:11:27.180 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:27.181 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.181 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.181 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:27.182 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:11:27.182 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5763999999999999e-05 2020-08-07T15:11:27.182 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.183 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.183 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:27.183 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613285+0000", 2020-08-07T15:11:27.192 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6479999999999999e-06 2020-08-07T15:11:27.192 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.192 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.192 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:27.193 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613288+0000", 2020-08-07T15:11:27.193 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1909999999999999e-06 2020-08-07T15:11:27.193 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.193 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.193 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:27.194 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613293+0000", 2020-08-07T15:11:27.194 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2560000000000001e-06 2020-08-07T15:11:27.194 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.194 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.194 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:27.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613295+0000", 2020-08-07T15:11:27.210 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2561e-05 2020-08-07T15:11:27.210 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.210 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.210 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.211 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613318+0000", 2020-08-07T15:11:27.211 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6349999999999999e-06 2020-08-07T15:11:27.211 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.211 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.212 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:27.219 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613321+0000", 2020-08-07T15:11:27.221 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81120592400000002 2020-08-07T15:11:27.222 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.222 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.222 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.222 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424527+0000", 2020-08-07T15:11:27.222 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8540000000000002e-06 2020-08-07T15:11:27.223 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.223 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.223 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:27.223 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424533+0000", 2020-08-07T15:11:27.224 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76623392800000001 2020-08-07T15:11:27.224 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.224 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.224 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.224 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190767+0000", 2020-08-07T15:11:27.225 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3990000000000001e-05 2020-08-07T15:11:27.225 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.225 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.225 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:27.225 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190841+0000", 2020-08-07T15:11:27.229 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011444599999999999 2020-08-07T15:11:27.229 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.229 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.229 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:11:27.230 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190955+0000", 2020-08-07T15:11:27.230 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0053397499999999999 2020-08-07T15:11:27.236 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.236 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.237 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:27.237 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196295+0000", 2020-08-07T15:11:27.237 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006e-06 2020-08-07T15:11:27.237 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.237 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.247 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:27.247 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196296+0000", 2020-08-07T15:11:27.247 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8940999999999997e-05 2020-08-07T15:11:27.247 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.248 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.248 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:27.248 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196355+0000", 2020-08-07T15:11:27.248 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042276700000000002 2020-08-07T15:11:27.248 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.249 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.249 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:27.249 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200583+0000", 2020-08-07T15:11:27.250 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00026035700000000001 2020-08-07T15:11:27.250 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.250 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.250 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:27.251 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200843+0000", 2020-08-07T15:11:27.251 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6700000000000003e-07 2020-08-07T15:11:27.251 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.251 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.251 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:27.252 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200844+0000", 2020-08-07T15:11:27.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6545999999999999e-05 2020-08-07T15:11:27.252 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.252 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.252 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:27.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200880+0000", 2020-08-07T15:11:27.253 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5876210340000001 2020-08-07T15:11:27.253 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:27.253 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:27.254 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:27.254 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.254 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.254 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72992 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:11:27.255 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:11:27.255 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 120.692807445, 2020-08-07T15:11:27.256 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5866241210000001, 2020-08-07T15:11:27.256 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:27.257 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:27.257 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:27.258 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:27.258 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:27.259 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72992 2020-08-07T15:11:27.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.259 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:27.259 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.260 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:27.260 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:11:27.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:27.261 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.261 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.261 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:27.262 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:11:27.262 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7543e-05 2020-08-07T15:11:27.262 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.263 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.263 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:27.263 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614374+0000", 2020-08-07T15:11:27.263 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9740000000000004e-06 2020-08-07T15:11:27.264 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.264 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.264 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:27.264 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614378+0000", 2020-08-07T15:11:27.265 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2660000000000007e-06 2020-08-07T15:11:27.265 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.265 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.265 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:27.266 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614387+0000", 2020-08-07T15:11:27.275 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2950000000000001e-06 2020-08-07T15:11:27.275 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.275 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.283 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:27.284 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614393+0000", 2020-08-07T15:11:27.287 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7279000000000001e-05 2020-08-07T15:11:27.288 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.288 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.288 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.289 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614420+0000", 2020-08-07T15:11:27.289 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9040000000000002e-06 2020-08-07T15:11:27.289 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.290 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.290 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:27.290 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614424+0000", 2020-08-07T15:11:27.290 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81012815699999996 2020-08-07T15:11:27.291 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.291 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.292 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.292 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424552+0000", 2020-08-07T15:11:27.292 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2879999999999998e-06 2020-08-07T15:11:27.293 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.293 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.293 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:27.294 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424558+0000", 2020-08-07T15:11:27.294 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76678665400000001 2020-08-07T15:11:27.294 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.295 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.296 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.296 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191345+0000", 2020-08-07T15:11:27.296 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010819600000000001 2020-08-07T15:11:27.297 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.297 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.297 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:27.298 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191453+0000", 2020-08-07T15:11:27.298 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000170829 2020-08-07T15:11:27.338 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.339 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.339 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:11:27.340 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191624+0000", 2020-08-07T15:11:27.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0047386679999999997 2020-08-07T15:11:27.340 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.341 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:27.341 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196363+0000", 2020-08-07T15:11:27.341 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6000000000000003e-07 2020-08-07T15:11:27.368 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.368 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.368 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:27.368 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196363+0000", 2020-08-07T15:11:27.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3421000000000002e-05 2020-08-07T15:11:27.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.369 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:27.369 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196417+0000", 2020-08-07T15:11:27.369 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042225539999999999 2020-08-07T15:11:27.370 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.370 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.370 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:27.370 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200639+0000", 2020-08-07T15:11:27.370 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00028031399999999999 2020-08-07T15:11:27.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.371 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.371 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:27.371 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200920+0000", 2020-08-07T15:11:27.371 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0299999999999999e-07 2020-08-07T15:11:27.372 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.372 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:27.372 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200920+0000", 2020-08-07T15:11:27.372 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0516000000000002e-05 2020-08-07T15:11:27.373 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.373 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.373 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:27.373 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200971+0000", 2020-08-07T15:11:27.373 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5866241210000001 2020-08-07T15:11:27.421 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:27.421 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:27.421 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:27.422 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.422 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73017 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:11:27.422 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:11:27.423 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 120.637299192, 2020-08-07T15:11:27.423 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.531181608, 2020-08-07T15:11:27.423 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:27.423 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:27.423 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:27.424 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:27.424 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:27.424 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73017 2020-08-07T15:11:27.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.424 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:27.425 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:27.425 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:11:27.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:27.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.426 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.426 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:27.426 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:11:27.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3018e-05 2020-08-07T15:11:27.427 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.427 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:27.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669898+0000", 2020-08-07T15:11:27.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5340000000000002e-06 2020-08-07T15:11:27.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:27.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669905+0000", 2020-08-07T15:11:27.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3307e-05 2020-08-07T15:11:27.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.429 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:27.430 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669919+0000", 2020-08-07T15:11:27.430 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7729999999999999e-06 2020-08-07T15:11:27.431 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.432 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:27.432 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669927+0000", 2020-08-07T15:11:27.432 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0361000000000001e-05 2020-08-07T15:11:27.432 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.432 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.433 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.433 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669977+0000", 2020-08-07T15:11:27.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0049999999999993e-06 2020-08-07T15:11:27.433 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.434 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.434 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:27.434 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669986+0000", 2020-08-07T15:11:27.435 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75459235199999997 2020-08-07T15:11:27.435 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.435 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.435 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.436 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424578+0000", 2020-08-07T15:11:27.436 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.925e-06 2020-08-07T15:11:27.436 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.436 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.436 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:27.437 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424584+0000", 2020-08-07T15:11:27.437 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76741703900000002 2020-08-07T15:11:27.438 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.438 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.438 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.438 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192001+0000", 2020-08-07T15:11:27.438 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000181196 2020-08-07T15:11:27.439 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.439 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.439 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:27.439 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192182+0000", 2020-08-07T15:11:27.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00021149299999999999 2020-08-07T15:11:27.440 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.441 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:11:27.441 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192394+0000", 2020-08-07T15:11:27.447 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004031242 2020-08-07T15:11:27.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:27.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196425+0000", 2020-08-07T15:11:27.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7800000000000001e-07 2020-08-07T15:11:27.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.453 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:27.453 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196426+0000", 2020-08-07T15:11:27.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5244999999999997e-05 2020-08-07T15:11:27.454 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.454 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.454 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:27.454 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196481+0000", 2020-08-07T15:11:27.454 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041989360000000003 2020-08-07T15:11:27.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:27.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200680+0000", 2020-08-07T15:11:27.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00032415299999999998 2020-08-07T15:11:27.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:27.456 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201004+0000", 2020-08-07T15:11:27.456 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9299999999999999e-07 2020-08-07T15:11:27.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:27.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201005+0000", 2020-08-07T15:11:27.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1857999999999999e-05 2020-08-07T15:11:27.480 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.481 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:27.481 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201037+0000", 2020-08-07T15:11:27.481 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.531181608 2020-08-07T15:11:27.481 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:27.482 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:27.482 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:27.482 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.483 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.484 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73018 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:11:27.484 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:11:27.484 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 120.636331387, 2020-08-07T15:11:27.484 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.530301616, 2020-08-07T15:11:27.484 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:27.485 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:27.485 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:27.485 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:27.485 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:27.485 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73018 2020-08-07T15:11:27.486 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.486 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:27.486 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.486 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:27.486 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:11:27.487 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:27.487 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.487 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.487 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:27.487 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:11:27.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9747000000000002e-05 2020-08-07T15:11:27.488 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:27.488 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670852+0000", 2020-08-07T15:11:27.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9430000000000002e-06 2020-08-07T15:11:27.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.489 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.489 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:27.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670856+0000", 2020-08-07T15:11:27.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0649999999999999e-06 2020-08-07T15:11:27.497 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.498 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.498 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:27.498 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670864+0000", 2020-08-07T15:11:27.498 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.924e-06 2020-08-07T15:11:27.499 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.499 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.499 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:27.499 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670868+0000", 2020-08-07T15:11:27.499 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6205000000000003e-05 2020-08-07T15:11:27.500 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.500 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.500 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.500 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670905+0000", 2020-08-07T15:11:27.500 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3289999999999996e-06 2020-08-07T15:11:27.501 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.501 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.501 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:27.501 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670912+0000", 2020-08-07T15:11:27.501 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75370200200000004 2020-08-07T15:11:27.502 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.502 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.502 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.502 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424614+0000", 2020-08-07T15:11:27.502 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5150000000000003e-06 2020-08-07T15:11:27.503 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.503 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.503 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:27.504 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424620+0000", 2020-08-07T15:11:27.504 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.768200153 2020-08-07T15:11:27.504 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.504 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.504 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.504 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192821+0000", 2020-08-07T15:11:27.505 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000111764 2020-08-07T15:11:27.505 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.505 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.505 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:27.506 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192932+0000", 2020-08-07T15:11:27.506 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017542400000000001 2020-08-07T15:11:27.506 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.507 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.507 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:11:27.507 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193108+0000", 2020-08-07T15:11:27.507 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004100851 2020-08-07T15:11:27.507 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.508 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.508 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:27.508 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197209+0000", 2020-08-07T15:11:27.508 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0100000000000001e-06 2020-08-07T15:11:27.508 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.509 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.509 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:27.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197210+0000", 2020-08-07T15:11:27.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9672999999999995e-05 2020-08-07T15:11:27.509 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.510 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:27.510 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197279+0000", 2020-08-07T15:11:27.512 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034275109999999998 2020-08-07T15:11:27.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.512 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.512 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:27.512 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200707+0000", 2020-08-07T15:11:27.513 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00036853100000000001 2020-08-07T15:11:27.513 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.513 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.513 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:27.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201075+0000", 2020-08-07T15:11:27.514 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8599999999999998e-07 2020-08-07T15:11:27.514 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.514 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:27.514 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201076+0000", 2020-08-07T15:11:27.515 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8483000000000003e-05 2020-08-07T15:11:27.515 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.516 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.516 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:27.516 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201124+0000", 2020-08-07T15:11:27.516 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.530301616 2020-08-07T15:11:27.517 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:27.517 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:27.518 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:27.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.519 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73019 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:11:27.520 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:11:27.520 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 120.635683736, 2020-08-07T15:11:27.520 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.529718224, 2020-08-07T15:11:27.520 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:27.521 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:27.521 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:27.521 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:27.521 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:27.521 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73019 2020-08-07T15:11:27.522 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.522 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:27.522 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.522 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:27.522 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:11:27.522 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:27.523 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.523 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.523 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:27.523 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:11:27.524 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3309000000000002e-05 2020-08-07T15:11:27.524 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.524 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.524 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:27.524 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671504+0000", 2020-08-07T15:11:27.525 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6619999999999998e-06 2020-08-07T15:11:27.525 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.525 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.525 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:27.525 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671507+0000", 2020-08-07T15:11:27.526 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2279999999999998e-06 2020-08-07T15:11:27.526 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.526 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.526 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:27.526 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671516+0000", 2020-08-07T15:11:27.527 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4860000000000002e-06 2020-08-07T15:11:27.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.529 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.529 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:27.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671519+0000", 2020-08-07T15:11:27.530 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0564000000000001e-05 2020-08-07T15:11:27.530 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.530 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.531 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.532 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671550+0000", 2020-08-07T15:11:27.532 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1029999999999998e-06 2020-08-07T15:11:27.533 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.533 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.534 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:27.534 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671556+0000", 2020-08-07T15:11:27.534 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75308759199999997 2020-08-07T15:11:27.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424643+0000", 2020-08-07T15:11:27.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9179999999999997e-06 2020-08-07T15:11:27.537 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.537 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.537 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:27.538 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424650+0000", 2020-08-07T15:11:27.539 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76890373599999995 2020-08-07T15:11:27.539 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.540 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.540 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.540 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193554+0000", 2020-08-07T15:11:27.541 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012345600000000001 2020-08-07T15:11:27.541 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.542 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.542 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:27.543 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193677+0000", 2020-08-07T15:11:27.543 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013528300000000001 2020-08-07T15:11:27.544 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.544 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.544 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:11:27.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193813+0000", 2020-08-07T15:11:27.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034777710000000002 2020-08-07T15:11:27.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.546 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.547 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:27.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197291+0000", 2020-08-07T15:11:27.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8000000000000004e-07 2020-08-07T15:11:27.550 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.551 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.551 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:27.551 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197291+0000", 2020-08-07T15:11:27.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9176999999999999e-05 2020-08-07T15:11:27.552 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.552 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:27.553 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197351+0000", 2020-08-07T15:11:27.553 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033931009999999999 2020-08-07T15:11:27.553 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.553 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.554 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:27.554 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200744+0000", 2020-08-07T15:11:27.554 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00041308000000000002 2020-08-07T15:11:27.554 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.554 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.555 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:27.555 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201157+0000", 2020-08-07T15:11:27.555 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8999999999999998e-07 2020-08-07T15:11:27.555 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.555 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:27.556 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201157+0000", 2020-08-07T15:11:27.556 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1588000000000003e-05 2020-08-07T15:11:27.556 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.557 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:27.557 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201189+0000", 2020-08-07T15:11:27.557 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.529718224 2020-08-07T15:11:27.557 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:27.558 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:27.558 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:27.558 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.558 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.558 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73020 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:11:27.559 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:11:27.559 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 120.567345876, 2020-08-07T15:11:27.559 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.461468373, 2020-08-07T15:11:27.559 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:27.559 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:27.560 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:27.560 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:27.561 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:27.561 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73020 2020-08-07T15:11:27.561 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.561 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:27.562 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.562 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:27.562 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:11:27.562 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:27.562 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.563 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.563 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:27.563 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:11:27.563 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.777e-05 2020-08-07T15:11:27.563 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.564 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:27.564 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739836+0000", 2020-08-07T15:11:27.564 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0180000000000003e-06 2020-08-07T15:11:27.564 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.565 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.565 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:27.565 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739840+0000", 2020-08-07T15:11:27.565 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6930000000000002e-06 2020-08-07T15:11:27.566 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.566 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.566 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:27.567 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739850+0000", 2020-08-07T15:11:27.567 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3619999999999999e-06 2020-08-07T15:11:27.567 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.567 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.567 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:27.568 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739854+0000", 2020-08-07T15:11:27.568 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7432999999999999e-05 2020-08-07T15:11:27.568 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.568 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.568 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.569 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739892+0000", 2020-08-07T15:11:27.570 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1149999999999996e-06 2020-08-07T15:11:27.570 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.570 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.571 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:27.572 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739899+0000", 2020-08-07T15:11:27.572 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68477883299999998 2020-08-07T15:11:27.572 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.573 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.573 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.573 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424677+0000", 2020-08-07T15:11:27.574 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7710000000000001e-06 2020-08-07T15:11:27.575 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.576 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:27.576 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424684+0000", 2020-08-07T15:11:27.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76949970700000003 2020-08-07T15:11:27.577 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.577 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.577 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.578 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194184+0000", 2020-08-07T15:11:27.578 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3957999999999997e-05 2020-08-07T15:11:27.579 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.579 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.579 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:27.580 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194278+0000", 2020-08-07T15:11:27.580 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017489599999999999 2020-08-07T15:11:27.580 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.581 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.582 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:11:27.582 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194453+0000", 2020-08-07T15:11:27.582 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002908908 2020-08-07T15:11:27.583 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.583 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.583 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:27.584 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197362+0000", 2020-08-07T15:11:27.584 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7299999999999995e-07 2020-08-07T15:11:27.587 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.588 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:27.589 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197362+0000", 2020-08-07T15:11:27.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6848000000000002e-05 2020-08-07T15:11:27.589 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.590 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.590 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:27.591 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197419+0000", 2020-08-07T15:11:27.591 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033597079999999999 2020-08-07T15:11:27.591 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.592 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.593 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:27.593 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200779+0000", 2020-08-07T15:11:27.593 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00044797899999999999 2020-08-07T15:11:27.594 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.594 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.597 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:27.597 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201227+0000", 2020-08-07T15:11:27.598 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7100000000000002e-07 2020-08-07T15:11:27.598 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.598 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.599 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:27.599 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201227+0000", 2020-08-07T15:11:27.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9230000000000001e-05 2020-08-07T15:11:27.600 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.600 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.601 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:27.601 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201277+0000", 2020-08-07T15:11:27.601 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.461468373 2020-08-07T15:11:27.602 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:27.610 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:27.611 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:27.611 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.611 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.611 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73021 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:11:27.612 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:11:27.612 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 120.504312269, 2020-08-07T15:11:27.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.398505079, 2020-08-07T15:11:27.612 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:27.612 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:27.613 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:27.613 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:27.613 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:27.613 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73021 2020-08-07T15:11:27.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.615 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:27.616 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.616 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:27.616 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:11:27.616 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:27.617 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.617 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.617 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:27.617 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:11:27.618 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7429000000000001e-05 2020-08-07T15:11:27.618 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.618 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.618 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:27.618 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802869+0000", 2020-08-07T15:11:27.619 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7840000000000001e-06 2020-08-07T15:11:27.619 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.619 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.619 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:27.619 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802873+0000", 2020-08-07T15:11:27.620 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0119999999999994e-06 2020-08-07T15:11:27.635 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.635 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.636 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:27.636 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802882+0000", 2020-08-07T15:11:27.636 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.04e-06 2020-08-07T15:11:27.636 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.636 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.637 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:27.637 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802887+0000", 2020-08-07T15:11:27.637 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7076000000000002e-05 2020-08-07T15:11:27.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.646 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802924+0000", 2020-08-07T15:11:27.646 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9280000000000004e-06 2020-08-07T15:11:27.654 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.654 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.654 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:27.655 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802931+0000", 2020-08-07T15:11:27.655 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.62178051000000001 2020-08-07T15:11:27.655 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.656 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.656 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.656 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424712+0000", 2020-08-07T15:11:27.656 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4749999999999996e-06 2020-08-07T15:11:27.656 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.657 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.657 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:27.657 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424719+0000", 2020-08-07T15:11:27.657 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77047786799999995 2020-08-07T15:11:27.657 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.658 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.658 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.658 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195197+0000", 2020-08-07T15:11:27.658 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000101909 2020-08-07T15:11:27.658 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.658 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.659 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:27.678 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195299+0000", 2020-08-07T15:11:27.678 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000248247 2020-08-07T15:11:27.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.679 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:11:27.680 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195547+0000", 2020-08-07T15:11:27.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033458699999999999 2020-08-07T15:11:27.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.681 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:27.681 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198893+0000", 2020-08-07T15:11:27.694 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0419999999999999e-06 2020-08-07T15:11:27.694 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.694 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.694 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:27.694 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198894+0000", 2020-08-07T15:11:27.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9354999999999997e-05 2020-08-07T15:11:27.695 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.695 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:27.695 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198953+0000", 2020-08-07T15:11:27.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001861096 2020-08-07T15:11:27.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.696 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:27.696 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200815+0000", 2020-08-07T15:11:27.697 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00049697299999999995 2020-08-07T15:11:27.697 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:27.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201311+0000", 2020-08-07T15:11:27.698 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7099999999999998e-07 2020-08-07T15:11:27.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.698 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.698 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:27.698 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201312+0000", 2020-08-07T15:11:27.699 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5194000000000002e-05 2020-08-07T15:11:27.699 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.699 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.699 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:27.700 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201347+0000", 2020-08-07T15:11:27.700 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.398505079 2020-08-07T15:11:27.700 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:27.700 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:27.700 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:27.700 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.701 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73022 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:11:27.701 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:11:27.701 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 120.423140214, 2020-08-07T15:11:27.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3238320669999999, 2020-08-07T15:11:27.702 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:27.702 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:27.702 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:27.702 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:27.702 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:27.703 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73022 2020-08-07T15:11:27.703 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.703 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:27.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.703 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:27.704 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:11:27.704 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:27.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.704 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:27.705 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:11:27.705 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6769999999999999e-05 2020-08-07T15:11:27.705 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.705 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.716 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:27.717 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884041+0000", 2020-08-07T15:11:27.717 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.703e-06 2020-08-07T15:11:27.717 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.722 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:27.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884044+0000", 2020-08-07T15:11:27.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5949999999999999e-06 2020-08-07T15:11:27.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.723 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:27.724 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884053+0000", 2020-08-07T15:11:27.724 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000003e-06 2020-08-07T15:11:27.724 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.724 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.724 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:27.725 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884058+0000", 2020-08-07T15:11:27.725 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7281999999999999e-05 2020-08-07T15:11:27.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.742 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.743 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884105+0000", 2020-08-07T15:11:27.743 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.044e-05 2020-08-07T15:11:27.743 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.743 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:27.744 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884116+0000", 2020-08-07T15:11:27.744 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.54062884300000003 2020-08-07T15:11:27.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.745 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424745+0000", 2020-08-07T15:11:27.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3149999999999998e-06 2020-08-07T15:11:27.745 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.768 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:27.768 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424754+0000", 2020-08-07T15:11:27.769 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77175732200000002 2020-08-07T15:11:27.772 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.772 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.772 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.772 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196511+0000", 2020-08-07T15:11:27.772 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010110799999999999 2020-08-07T15:11:27.773 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.773 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.773 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:27.773 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196612+0000", 2020-08-07T15:11:27.774 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017543100000000001 2020-08-07T15:11:27.774 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.774 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.774 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:11:27.775 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196788+0000", 2020-08-07T15:11:27.775 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032134569999999999 2020-08-07T15:11:27.775 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.775 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:27.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200001+0000", 2020-08-07T15:11:27.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.071e-06 2020-08-07T15:11:27.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:27.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200002+0000", 2020-08-07T15:11:27.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2527999999999995e-05 2020-08-07T15:11:27.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:27.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200065+0000", 2020-08-07T15:11:27.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001327433 2020-08-07T15:11:27.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.779 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:27.780 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201392+0000", 2020-08-07T15:11:27.781 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063854030000000004 2020-08-07T15:11:27.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.781 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:27.781 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207778+0000", 2020-08-07T15:11:27.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.285e-06 2020-08-07T15:11:27.782 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.782 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.782 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:27.782 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207779+0000", 2020-08-07T15:11:27.783 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7070999999999995e-05 2020-08-07T15:11:27.783 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:27.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207846+0000", 2020-08-07T15:11:27.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3238320669999999 2020-08-07T15:11:27.784 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:27.784 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:27.784 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:27.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.785 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73023 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:11:27.785 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:11:27.785 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 120.309978573, 2020-08-07T15:11:27.786 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2113701729999999, 2020-08-07T15:11:27.786 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:27.786 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:27.787 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:27.787 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:27.787 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:27.787 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73023 2020-08-07T15:11:27.787 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.788 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:27.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.788 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:27.788 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:11:27.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:27.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:27.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:11:27.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4105e-05 2020-08-07T15:11:27.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.792 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.793 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:27.793 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997210+0000", 2020-08-07T15:11:27.793 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1089999999999997e-06 2020-08-07T15:11:27.793 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.794 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.794 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:27.794 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997215+0000", 2020-08-07T15:11:27.794 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7040000000000006e-06 2020-08-07T15:11:27.795 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.795 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.795 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:27.795 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997224+0000", 2020-08-07T15:11:27.795 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4439999999999999e-06 2020-08-07T15:11:27.796 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.796 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:27.796 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997230+0000", 2020-08-07T15:11:27.796 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5977999999999999e-05 2020-08-07T15:11:27.798 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.798 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.798 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997276+0000", 2020-08-07T15:11:27.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2660000000000007e-06 2020-08-07T15:11:27.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:27.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997285+0000", 2020-08-07T15:11:27.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.42749189599999998 2020-08-07T15:11:27.802 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.802 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.802 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.802 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424777+0000", 2020-08-07T15:11:27.802 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8940000000000004e-06 2020-08-07T15:11:27.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:27.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424784+0000", 2020-08-07T15:11:27.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77268199699999995 2020-08-07T15:11:27.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197466+0000", 2020-08-07T15:11:27.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011277699999999999 2020-08-07T15:11:27.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.818 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:27.818 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197579+0000", 2020-08-07T15:11:27.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018970599999999999 2020-08-07T15:11:27.818 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.818 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:11:27.819 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197768+0000", 2020-08-07T15:11:27.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043168529999999998 2020-08-07T15:11:27.819 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:27.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202085+0000", 2020-08-07T15:11:27.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.186e-06 2020-08-07T15:11:27.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:27.821 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202086+0000", 2020-08-07T15:11:27.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8975999999999999e-05 2020-08-07T15:11:27.821 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.822 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:27.822 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202145+0000", 2020-08-07T15:11:27.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00035331499999999999 2020-08-07T15:11:27.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:27.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202499+0000", 2020-08-07T15:11:27.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0059703159999999998 2020-08-07T15:11:27.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:27.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208469+0000", 2020-08-07T15:11:27.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7400000000000002e-07 2020-08-07T15:11:27.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:27.826 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208470+0000", 2020-08-07T15:11:27.826 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5877000000000006e-05 2020-08-07T15:11:27.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.827 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:27.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208546+0000", 2020-08-07T15:11:27.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2113701729999999 2020-08-07T15:11:27.827 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:27.827 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:27.828 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:27.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.828 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73024 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e131)", 2020-08-07T15:11:27.828 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:26.502031+0000", 2020-08-07T15:11:27.829 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 119.80512342599999, 2020-08-07T15:11:27.829 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.70660062599999995, 2020-08-07T15:11:27.829 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:27.829 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:27.829 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:27.830 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:27.830 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:27.830 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73024 2020-08-07T15:11:27.830 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.831 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:27.831 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.831 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:27.831 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502031+0000", 2020-08-07T15:11:27.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:27.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.832 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:27.832 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502031+0000", 2020-08-07T15:11:27.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0858999999999997e-05 2020-08-07T15:11:27.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:27.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502062+0000", 2020-08-07T15:11:27.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1159999999999998e-06 2020-08-07T15:11:27.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:27.842 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502067+0000", 2020-08-07T15:11:27.842 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1810999999999999e-05 2020-08-07T15:11:27.842 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.842 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.842 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:27.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502078+0000", 2020-08-07T15:11:27.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2500000000000006e-06 2020-08-07T15:11:27.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:27.845 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502087+0000", 2020-08-07T15:11:27.845 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3461999999999996e-05 2020-08-07T15:11:27.845 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.846 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.846 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502150+0000", 2020-08-07T15:11:27.846 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3108999999999999e-05 2020-08-07T15:11:27.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.846 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.847 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:27.847 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.502163+0000", 2020-08-07T15:11:27.847 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69604141200000003 2020-08-07T15:11:27.847 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.847 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.848 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:27.848 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198205+0000", 2020-08-07T15:11:27.848 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013494700000000001 2020-08-07T15:11:27.848 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.848 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.848 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:27.849 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198340+0000", 2020-08-07T15:11:27.849 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000169696 2020-08-07T15:11:27.849 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.849 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.850 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:11:27.850 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198509+0000", 2020-08-07T15:11:27.850 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0036466960000000001 2020-08-07T15:11:27.850 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.851 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:27.851 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202156+0000", 2020-08-07T15:11:27.851 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.16e-07 2020-08-07T15:11:27.851 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.851 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:27.852 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202157+0000", 2020-08-07T15:11:27.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5290000000000002e-05 2020-08-07T15:11:27.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:27.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202202+0000", 2020-08-07T15:11:27.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00061786700000000003 2020-08-07T15:11:27.858 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.858 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:27.858 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202820+0000", 2020-08-07T15:11:27.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0057728759999999997 2020-08-07T15:11:27.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.859 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.859 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:27.859 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208593+0000", 2020-08-07T15:11:27.859 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4000000000000001e-07 2020-08-07T15:11:27.860 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.860 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:27.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208594+0000", 2020-08-07T15:11:27.860 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7679000000000003e-05 2020-08-07T15:11:27.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:27.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:27.861 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:27.861 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208631+0000", 2020-08-07T15:11:27.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.70660062599999995 2020-08-07T15:11:27.862 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:27.862 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:27.862 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:27.862 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:27.862 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:27.863 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:11:27.867 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:11:27.867 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:11:27.868 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 67 16 96928 96912 11.2993 11.9062 0.00699292 0.0110566 2020-08-07T15:11:27.868 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:27.863+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:27.975 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:27.974+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:28.090 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:28.088+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:28.110 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:11:28.121 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3, 4, 5, 6, 7] 2020-08-07T15:11:28.121 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:11:28.122 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_2 2020-08-07T15:11:28.122 INFO:tasks.ceph.ceph_manager.ceph:contract_pool unique_pool_2 by 10 min 8 2020-08-07T15:11:28.122 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:11:28.213 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:28.211+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:28.216 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 68 16 98571 98555 11.3218 12.8359 0.00401355 0.0110369 2020-08-07T15:11:28.292 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:28.290+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:28.397 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:28.396+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:28.506 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:28.499+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:28.610 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:28.601+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:28.707 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:28.701+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:28.815 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:28.813+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:28.922 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:28.914+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:29.021 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:29.018+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:29.130 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:29.123+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:29.217 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 69 16 100021 100005 11.3219 11.3281 0.00951881 0.0110323 2020-08-07T15:11:29.240 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:29.228+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:29.339 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:29.337+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:29.445 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:29.443+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:29.551 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:29.549+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:29.659 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:29.657+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:29.766 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:29.761+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:29.864 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:29.862+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:29.875 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:11:29.965 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:29.963+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:30.069 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:30.067+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:30.175 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:30.169+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:30.217 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 70 16 101386 101370 11.3125 10.6641 0.0100775 0.0110459 2020-08-07T15:11:30.282 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:30.270+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:30.399 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:30.386+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:30.506 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:30.493+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:30.596 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:30.594+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:30.701 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:30.699+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:30.819 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:30.817+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:30.955 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:30.953+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:31.065 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:31.063+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:31.166 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:31.165+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:31.230 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 71 16 102870 102854 11.3165 11.5938 0.0112908 0.0110416 2020-08-07T15:11:31.272 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:31.270+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:31.379 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:31.377+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:31.488 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:31.486+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:31.595 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:31.589+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:31.697 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:31.696+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:31.799 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:31.798+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:31.905 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:31.900+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:32.007 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:32.006+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:32.115 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:32.112+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:32.217 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 72 16 103400 103384 11.2168 4.14062 0.608766 0.0110453 2020-08-07T15:11:32.225 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:32.216+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:32.319 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:32.317+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:32.424 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:32.423+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:32.438 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:11:32.439 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:11:32.439 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:11:32.439 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:11:32.481 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:11:32.526 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:11:32.534 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:32.525+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:32.569 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:11:32.570 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:11:32.603 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:11:32.650 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:11:32.651 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:32.649+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:32.681 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:11:32.708 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2020-08-07T15:11:32.201736+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":840892416,"num_objects":12834,"num_object_clones":0,"num_object_copies":38502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3247,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":12834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102648,"num_write_kb":821184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1498,"num_bytes_recovered":98172928,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":149,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":171,"ondisk_log_size":171,"up":146,"acting":152,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":10799980,"kb_used_data":2411308,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":744174740,"statfs":{"total":773094113280,"available":762034933760,"internally_reserved":8589934592,"allocated":2469179392,"data_stored":2461993204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,7,1,0,2,9,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":105512960,"num_objects":1610,"num_object_clones":0,"num_object_copies":4830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":452,"num_objects_misplaced":-659,"num_objects_unfound":0,"num_objects_dirty":1610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12880,"num_write_kb":103040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1486,"num_bytes_recovered":97386496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.816993"},"pg_stats":[{"pgid":"1.1e","version":"16'2","reported_seq":"60","reported_epoch":"211","state":"active+clean","last_fresh":"2020-08-07T15:11:27.247463+0000","last_change":"2020-08-07T15:10:51.560726+0000","last_active":"2020-08-07T15:11:27.247463+0000","last_peered":"2020-08-07T15:11:27.247463+0000","last_clean":"2020-08-07T15:11:27.247463+0000","last_became_active":"2020-08-07T15:10:51.559894+0000","last_became_peered":"2020-08-07T15:10:51.559894+0000","last_unstale":"2020-08-07T15:11:27.247463+0000","last_undegraded":"2020-08-07T15:11:27.247463+0000","last_fullsized":"2020-08-07T15:11:27.247463+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:29.055048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:29.055048+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1d","version":"75'3","reported_seq":"106","reported_epoch":"211","state":"active+clean+remapped","last_fresh":"2020-08-07T15:11:27.247224+0000","last_change":"2020-08-07T15:10:52.203763+0000","last_active":"2020-08-07T15:11:27.247224+0000","last_peered":"2020-08-07T15:11:27.247224+0000","last_clean":"2020-08-07T15:11:27.247224+0000","last_became_active":"2020-08-07T15:10:52.203050+0000","last_became_peered":"2020-08-07T15:10:52.203050+0000","last_unstale":"2020-08-07T15:11:27.247224+0000","last_undegraded":"2020-08-07T15:11:27.247224+0000","last_fullsized":"2020-08-07T15:11:27.247224+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:10:10.962480+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.962480+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":149,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,4],"acting":[3,4,7],"avail_no_missing":["3","4","7"],"object_location_counts":[{"shards":"3,4,7","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1c","version":"16'2","reported_seq":"138","reported_epoch":"211","state":"active+clean+remapped","last_fresh":"2020-08-07T15:11:27.215596+0000","last_change":"2020-08-07T15:10:52.413792+0000","last_active":"2020-08-07T15:11:27.215596+0000","last_peered":"2020-08-07T15:11:27.215596+0000","last_clean":"2020-08-07T15:11:27.215596+0000","last_became_active":"2020-08-07T15:10:52.413159+0000","last_became_peered":"2020-08-07T15:10:52.413159+0000","last_unstale":"2020-08-07T15:11:27.215596+0000","last_undegraded":"2020-08-07T15:11:27.215596+0000","last_fullsized":"2020-08-07T15:11:27.215596+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:36.687314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:36.687314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6,0],"avail_no_missing":["7","0","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1b","version":"16'2","reported_seq":"184","reported_epoch":"211","state":"active+clean","last_fresh":"2020-08-07T15:11:27.204977+0000","last_change":"2020-08-07T15:10:57.797029+0000","last_active":"2020-08-07T15:11:27.204977+0000","last_peered":"2020-08-07T15:11:27.204977+0000","last_clean":"2020-08-07T15:11:27.204977+0000","last_became_active":"2020-08-07T15:10:51.557392+0000","last_became_peered":"2020-08-07T15:10:51.557392+0000","last_unstale":"2020-08-07T15:11:27.204977+0000","last_undegraded":"2020-08-07T15:11:27.204977+0000","last_fullsized":"2020-08-07T15:11:27.204977+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:57.796946+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:57.796946+0000","last_clean_scrub_stamp":"2020-08-07T15:10:57.796946+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1a","version":"16'2","reported_seq":"38","reported_epoch":"211","state":"active+clean+remapped","last_fresh":"2020-08-07T15:11:27.215620+0000","last_change":"2020-08-07T15:11:11.780113+0000","last_active":"2020-08-07T15:11:27.215620+0000","last_peered":"2020-08-07T15:11:27.215620+0000","last_clean":"2020-08-07T15:11:27.215620+0000","last_became_active":"2020-08-07T15:10:52.339672+0000","last_became_peered":"2020-08-07T15:10:52.339672+0000","last_unstale":"2020-08-07T15:11:27.215620+0000","last_undegraded":"2020-08-07T15:11:27.215620+0000","last_fullsized":"2020-08-07T15:11:27.215620+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:11.779828+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:11:11.779828+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4,2],"avail_no_missing":["7","2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.19","version":"16'2","reported_seq":"106","reported_epoch":"211","state":"active+clean","last_fresh":"2020-08-07T15:11:27.246686+0000","last_change":"2020-08-07T15:11:08.710172+0000","last_active":"2020-08-07T15:11:27.246686+0000","last_peered":"2020-08-07T15:11:27.246686+0000","last_clean":"2020-08-07T15:11:27.246686+0000","last_became_active":"2020-08-07T15:10:51.559752+0000","last_became_peered":"2020-08-07T15:10:51.559752+0000","last_unstale":"2020-08-07T15:11:27.246686+0000","last_undegraded":"2020-08-07T15:11:27.246686+0000","last_fullsized":"2020-08-07T15:11:27.246686+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:08.710086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:08.710086+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.18","version":"16'2","reported_seq":"209","reported_epoch":"211","state":"active+clean","last_fresh":"2020-08-07T15:11:27.204929+0000","last_change":"2020-08-07T15:11:06.955336+0000","last_active":"2020-08-07T15:11:27.204929+0000","last_peered":"2020-08-07T15:11:27.204929+0000","last_clean":"2020-08-07T15:11:27.204929+0000","last_became_active":"2020-08-07T15:10:51.557665+0000","last_became_peered":"2020-08-07T15:10:51.557665+0000","last_unstale":"2020-08-07T15:11:27.204929+0000","last_undegraded":"2020-08-07T15:11:27.204929+0000","last_fullsized":"2020-08-07T15:11:27.204929+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:06.955262+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:06.955262+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.17","version":"16'2","reported_seq":"37","reported_epoch":"211","state":"active+clean","last_fresh":"2020-08-07T15:11:27.205512+0000","last_change":"2020-08-07T15:11:08.916219+0000","last_active":"2020-08-07T15:11:27.205512+0000","last_peered":"2020-08-07T15:11:27.205512+0000","last_clean":"2020-08-07T15:11:27.205512+0000","last_became_active":"2020-08-07T15:10:51.561769+0000","last_became_peered":"2020-08-07T15:10:51.561769+0000","last_unstale":"2020-08-07T15:11:27.205512+0000","last_undegraded":"2020-08-07T15:11:27.205512+0000","last_fullsized":"2020-08-07T15:11:27.205512+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:08.916169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:08.916169+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.16","version":"16'2","reported_seq":"183","reported_epoch":"211","state":"active+clean+remapped","last_fresh":"2020-08-07T15:11:27.247589+0000","last_change":"2020-08-07T15:10:52.208583+0000","last_active":"2020-08-07T15:11:27.247589+0000","last_peered":"2020-08-07T15:11:27.247589+0000","last_clean":"2020-08-07T15:11:27.247589+0000","last_became_active":"2020-08-07T15:10:52.207790+0000","last_became_peered":"2020-08-07T15:10:52.207790+0000","last_unstale":"2020-08-07T15:11:27.247589+0000","last_undegraded":"2020-08-07T15:11:27.247589+0000","last_fullsized":"2020-08-07T15:11:27.247589+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:07.302204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:07.302204+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0,6],"avail_no_missing":["3","0","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.15","version":"75'3","reported_seq":"38","reported_epoch":"211","state":"active+clean+remapped","last_fresh":"2020-08-07T15:11:27.215978+0000","last_change":"2020-08-07T15:11:20.562885+0000","last_active":"2020-08-07T15:11:27.215978+0000","last_peered":"2020-08-07T15:11:27.215978+0000","last_clean":"2020-08-07T15:11:27.215978+0000","last_became_active":"2020-08-07T15:10:52.207820+0000","last_became_peered":"2020-08-07T15:10:52.207820+0000","last_unstale":"2020-08-07T15:11:27.215978+0000","last_undegraded":"2020-08-07T15:11:27.215978+0000","last_fullsized":"2020-08-07T15:11:27.215978+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:11:20.562710+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:11:20.562710+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5,2],"avail_no_missing":["7","2","5"],"object_location_counts":[{"shards":"2,5,7","objects":1}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.14","version":"16'2","reported_seq":"144","reported_epoch":"211","state":"active+clean","last_fresh":"2020-08-07T15:11:27.211689+0000","last_change":"2020-08-07T15:10:10.644555+0000","last_active":"2020-08-07T15:11:27.211689+0000","last_peered":"2020-08-07T15:11:27.211689+0000","last_clean":"2020-08-07T15:11:27.211689+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:11:27.211689+0000","last_undegraded":"2020-08-07T15:11:27.211689+0000","last_fullsized":"2020-08-07T15:11:27.211689+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:10.644460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.644460+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"145","reported_epoch":"211","state":"active+undersized","last_fresh":"2020-08-07T15:11:27.246787+0000","last_change":"2020-08-07T15:10:51.552550+0000","last_active":"2020-08-07T15:11:27.246787+0000","last_peered":"2020-08-07T15:11:27.246787+0000","last_clean":"2020-08-07T15:09:25.422675+0000","last_became_active":"2020-08-07T15:10:51.552550+0000","last_became_peered":"2020-08-07T15:10:51.552550+0000","last_unstale":"2020-08-07T15:11:27.246787+0000","last_undegraded":"2020-08-07T15:11:27.246787+0000","last_fullsized":"2020-08-07T15:10:51.544042+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:07.295863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:07.295863+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":["3","4"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"150","reported_epoch":"211","state":"active+undersized","last_fresh":"2020-08-07T15:11:27.204990+0000","last_change":"2020-08-07T15:10:51.559962+0000","last_active":"2020-08-07T15:11:27.204990+0000","last_peered":"2020-08-07T15:11:27.204990+0000","last_clean":"2020-08-07T15:10:49.526673+0000","last_became_active":"2020-08-07T15:10:51.559962+0000","last_became_peered":"2020-08-07T15:10:51.559962+0000","last_unstale":"2020-08-07T15:11:27.204990+0000","last_undegraded":"2020-08-07T15:11:27.204990+0000","last_fullsized":"2020-08-07T15:10:51.549931+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:28.734193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:28.734193+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"257","reported_epoch":"211","state":"active+clean","last_fresh":"2020-08-07T15:11:27.204918+0000","last_change":"2020-08-07T15:10:24.690028+0000","last_active":"2020-08-07T15:11:27.204918+0000","last_peered":"2020-08-07T15:11:27.204918+0000","last_clean":"2020-08-07T15:11:27.204918+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:11:27.204918+0000","last_undegraded":"2020-08-07T15:11:27.204918+0000","last_fullsized":"2020-08-07T15:11:27.204918+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:24.689950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:24.689950+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"251","reported_epoch":"211","state":"active+clean","last_fresh":"2020-08-07T15:11:27.205056+0000","last_change":"2020-08-07T15:10:26.631030+0000","last_active":"2020-08-07T15:11:27.205056+0000","last_peered":"2020-08-07T15:11:27.205056+0000","last_clean":"2020-08-07T15:11:27.205056+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:11:27.205056+0000","last_undegraded":"2020-08-07T15:11:27.205056+0000","last_fullsized":"2020-08-07T15:11:27.205056+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:26.630945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:26.630945+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.0","version":"212'6536","reported_seq":"6582","reported_epoch":"212","state":"active+undersized+degraded","last_fresh":"2020-08-07T15:11:31.494712+0000","last_change":"2020-08-07T15:10:51.552627+0000","last_active":"2020-08-07T15:11:31.494712+0000","last_peered":"2020-08-07T15:11:31.494712+0000","last_clean":"2020-08-07T15:10:50.162105+0000","last_became_active":"2020-08-07T15:10:51.552627+0000","last_became_peered":"2020-08-07T15:10:51.552627+0000","last_unstale":"2020-08-07T15:11:31.494712+0000","last_undegraded":"2020-08-07T15:10:51.544354+0000","last_fullsized":"2020-08-07T15:10:51.544200+0000","mapping_epoch":191,"log_start":"212'6532","ondisk_log_start":"212'6532","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53542912,"num_objects":817,"num_object_clones":0,"num_object_copies":2451,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":817,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6536,"num_write_kb":52288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":["3","4"],"object_location_counts":[{"shards":"3,4","objects":817}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"210","reported_epoch":"210","state":"active+clean","last_fresh":"2020-08-07T15:11:26.407260+0000","last_change":"2020-08-07T15:10:57.384596+0000","last_active":"2020-08-07T15:11:26.407260+0000","last_peered":"2020-08-07T15:11:26.407260+0000","last_clean":"2020-08-07T15:11:26.407260+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:11:26.407260+0000","last_undegraded":"2020-08-07T15:11:26.407260+0000","last_fullsized":"2020-08-07T15:11:26.407260+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:57.384515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:10:57.384515+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"127","reported_epoch":"211","state":"active+clean","last_fresh":"2020-08-07T15:11:27.247008+0000","last_change":"2020-08-07T15:10:52.653421+0000","last_active":"2020-08-07T15:11:27.247008+0000","last_peered":"2020-08-07T15:11:27.247008+0000","last_clean":"2020-08-07T15:11:27.247008+0000","last_became_active":"2020-08-07T15:09:30.283176+0000","last_became_peered":"2020-08-07T15:09:30.283176+0000","last_unstale":"2020-08-07T15:11:27.247008+0000","last_undegraded":"2020-08-07T15:11:27.247008+0000","last_fullsized":"2020-08-07T15:11:27.247008+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:52.653356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:52.653356+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"212'6440","reported_seq":"6496","reported_epoch":"212","state":"active+clean","last_fresh":"2020-08-07T15:11:31.457477+0000","last_change":"2020-08-07T15:10:16.259807+0000","last_active":"2020-08-07T15:11:31.457477+0000","last_peered":"2020-08-07T15:11:31.457477+0000","last_clean":"2020-08-07T15:11:31.457477+0000","last_became_active":"2020-08-07T15:10:16.259365+0000","last_became_peered":"2020-08-07T15:10:16.259365+0000","last_unstale":"2020-08-07T15:11:31.457477+0000","last_undegraded":"2020-08-07T15:11:31.457477+0000","last_fullsized":"2020-08-07T15:11:31.457477+0000","mapping_epoch":164,"log_start":"211'6432","ondisk_log_start":"211'6432","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":52756480,"num_objects":805,"num_object_clones":0,"num_object_copies":2415,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6440,"num_write_kb":51520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"28","reported_epoch":"210","state":"active+undersized","last_fresh":"2020-08-07T15:11:26.407108+0000","last_change":"2020-08-07T15:10:51.558681+0000","last_active":"2020-08-07T15:11:26.407108+0000","last_peered":"2020-08-07T15:11:26.407108+0000","last_clean":"2020-08-07T15:06:22.062496+0000","last_became_active":"2020-08-07T15:10:51.558681+0000","last_became_peered":"2020-08-07T15:10:51.558681+0000","last_unstale":"2020-08-07T15:11:26.407108+0000","last_undegraded":"2020-08-07T15:11:26.407108+0000","last_fullsized":"2020-08-07T15:10:51.552830+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:25.463628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:10:25.463628+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"181","reported_epoch":"211","state":"active+clean","last_fresh":"2020-08-07T15:11:27.211986+0000","last_change":"2020-08-07T15:10:52.485772+0000","last_active":"2020-08-07T15:11:27.211986+0000","last_peered":"2020-08-07T15:11:27.211986+0000","last_clean":"2020-08-07T15:11:27.211986+0000","last_became_active":"2020-08-07T15:10:46.501793+0000","last_became_peered":"2020-08-07T15:10:46.501793+0000","last_unstale":"2020-08-07T15:11:27.211986+0000","last_undegraded":"2020-08-07T15:11:27.211986+0000","last_fullsized":"2020-08-07T15:11:27.211986+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:10:52.485722+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:10:52.485722+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"211'6384","reported_seq":"6437","reported_epoch":"211","state":"active+clean","last_fresh":"2020-08-07T15:11:27.696375+0000","last_change":"2020-08-07T15:10:16.448360+0000","last_active":"2020-08-07T15:11:27.696375+0000","last_peered":"2020-08-07T15:11:27.696375+0000","last_clean":"2020-08-07T15:11:27.696375+0000","last_became_active":"2020-08-07T15:10:16.447826+0000","last_became_peered":"2020-08-07T15:10:16.447826+0000","last_unstale":"2020-08-07T15:11:27.696375+0000","last_undegraded":"2020-08-07T15:11:27.696375+0000","last_fullsized":"2020-08-07T15:11:27.696375+0000","mapping_epoch":164,"log_start":"211'6378","ondisk_log_start":"211'6378","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":52297728,"num_objects":798,"num_object_clones":0,"num_object_copies":2394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6384,"num_write_kb":51072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"209","reported_epoch":"211","state":"active+clean","last_fresh":"2020-08-07T15:11:27.216208+0000","last_change":"2020-08-07T15:09:54.908283+0000","last_active":"2020-08-07T15:11:27.216208+0000","last_peered":"2020-08-07T15:11:27.216208+0000","last_clean":"2020-08-07T15:11:27.216208+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:11:27.216208+0000","last_undegraded":"2020-08-07T15:11:27.216208+0000","last_fullsized":"2020-08-07T15:11:27.216208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:54.908197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:54.908197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"258","reported_epoch":"210","state":"active+clean","last_fresh":"2020-08-07T15:11:26.407105+0000","last_change":"2020-08-07T15:10:53.292841+0000","last_active":"2020-08-07T15:11:26.407105+0000","last_peered":"2020-08-07T15:11:26.407105+0000","last_clean":"2020-08-07T15:11:26.407105+0000","last_became_active":"2020-08-07T15:10:46.516484+0000","last_became_peered":"2020-08-07T15:10:46.516484+0000","last_unstale":"2020-08-07T15:11:26.407105+0000","last_undegraded":"2020-08-07T15:11:26.407105+0000","last_fullsized":"2020-08-07T15:11:26.407105+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:53.292768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:53.292768+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"211'6704","reported_seq":"8341","reported_epoch":"212","state":"remapped+peering","last_fresh":"2020-08-07T15:11:31.407652+0000","last_change":"2020-08-07T15:11:31.408033+0000","last_active":"2020-08-07T15:11:31.407652+0000","last_peered":"2020-08-07T15:11:31.405012+0000","last_clean":"2020-08-07T15:10:50.103316+0000","last_became_active":"2020-08-07T15:11:22.227564+0000","last_became_peered":"2020-08-07T15:11:22.227564+0000","last_unstale":"2020-08-07T15:11:31.407652+0000","last_undegraded":"2020-08-07T15:11:31.407652+0000","last_fullsized":"2020-08-07T15:11:31.407652+0000","mapping_epoch":212,"log_start":"211'6698","ondisk_log_start":"211'6698","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":54919168,"num_objects":838,"num_object_clones":0,"num_object_copies":2514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6704,"num_write_kb":53632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":786,"num_bytes_recovered":51511296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"215","reported_epoch":"211","state":"active+undersized","last_fresh":"2020-08-07T15:11:27.215490+0000","last_change":"2020-08-07T15:10:51.559204+0000","last_active":"2020-08-07T15:11:27.215490+0000","last_peered":"2020-08-07T15:11:27.215490+0000","last_clean":"2020-08-07T15:10:49.542912+0000","last_became_active":"2020-08-07T15:10:51.559204+0000","last_became_peered":"2020-08-07T15:10:51.559204+0000","last_unstale":"2020-08-07T15:11:27.215490+0000","last_undegraded":"2020-08-07T15:11:27.215490+0000","last_fullsized":"2020-08-07T15:10:51.546815+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:51.036224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:51.036224+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":["7","0"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"218","reported_epoch":"210","state":"active+undersized","last_fresh":"2020-08-07T15:11:26.406570+0000","last_change":"2020-08-07T15:10:51.557999+0000","last_active":"2020-08-07T15:11:26.406570+0000","last_peered":"2020-08-07T15:11:26.406570+0000","last_clean":"2020-08-07T15:10:49.532877+0000","last_became_active":"2020-08-07T15:10:51.557999+0000","last_became_peered":"2020-08-07T15:10:51.557999+0000","last_unstale":"2020-08-07T15:11:26.406570+0000","last_undegraded":"2020-08-07T15:11:26.406570+0000","last_fullsized":"2020-08-07T15:10:51.549893+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:23.400672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:23.400672+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"212'6368","reported_seq":"6423","reported_epoch":"212","state":"active+clean","last_fresh":"2020-08-07T15:11:31.591880+0000","last_change":"2020-08-07T15:10:16.255436+0000","last_active":"2020-08-07T15:11:31.591880+0000","last_peered":"2020-08-07T15:11:31.591880+0000","last_clean":"2020-08-07T15:11:31.591880+0000","last_became_active":"2020-08-07T15:10:16.254967+0000","last_became_peered":"2020-08-07T15:10:16.254967+0000","last_unstale":"2020-08-07T15:11:31.591880+0000","last_undegraded":"2020-08-07T15:11:31.591880+0000","last_fullsized":"2020-08-07T15:11:31.591880+0000","mapping_epoch":164,"log_start":"212'6363","ondisk_log_start":"212'6363","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":52166656,"num_objects":796,"num_object_clones":0,"num_object_copies":2388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6368,"num_write_kb":50944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"241","reported_epoch":"211","state":"active+clean","last_fresh":"2020-08-07T15:11:27.215381+0000","last_change":"2020-08-07T15:09:55.879867+0000","last_active":"2020-08-07T15:11:27.215381+0000","last_peered":"2020-08-07T15:11:27.215381+0000","last_clean":"2020-08-07T15:11:27.215381+0000","last_became_active":"2020-08-07T15:08:36.627426+0000","last_became_peered":"2020-08-07T15:08:36.627426+0000","last_unstale":"2020-08-07T15:11:27.215381+0000","last_undegraded":"2020-08-07T15:11:27.215381+0000","last_fullsized":"2020-08-07T15:11:27.215381+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:55.879785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:55.879785+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"209","reported_epoch":"211","state":"active+clean","last_fresh":"2020-08-07T15:11:27.205691+0000","last_change":"2020-08-07T15:10:08.656952+0000","last_active":"2020-08-07T15:11:27.205691+0000","last_peered":"2020-08-07T15:11:27.205691+0000","last_clean":"2020-08-07T15:11:27.205691+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:11:27.205691+0000","last_undegraded":"2020-08-07T15:11:27.205691+0000","last_fullsized":"2020-08-07T15:11:27.205691+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:08.656855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:10:08.656855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"212'6200","reported_seq":"6254","reported_epoch":"212","state":"active+clean","last_fresh":"2020-08-07T15:11:31.599246+0000","last_change":"2020-08-07T15:10:16.263356+0000","last_active":"2020-08-07T15:11:31.599246+0000","last_peered":"2020-08-07T15:11:31.599246+0000","last_clean":"2020-08-07T15:11:31.599246+0000","last_became_active":"2020-08-07T15:10:16.262971+0000","last_became_peered":"2020-08-07T15:10:16.262971+0000","last_unstale":"2020-08-07T15:11:31.599246+0000","last_undegraded":"2020-08-07T15:11:31.599246+0000","last_fullsized":"2020-08-07T15:11:31.599246+0000","mapping_epoch":164,"log_start":"211'6192","ondisk_log_start":"211'6192","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50790400,"num_objects":775,"num_object_clones":0,"num_object_copies":2325,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6200,"num_write_kb":49600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"230","reported_epoch":"211","state":"active+clean","last_fresh":"2020-08-07T15:11:27.211805+0000","last_change":"2020-08-07T15:10:24.576099+0000","last_active":"2020-08-07T15:11:27.211805+0000","last_peered":"2020-08-07T15:11:27.211805+0000","last_clean":"2020-08-07T15:11:27.211805+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:11:27.211805+0000","last_undegraded":"2020-08-07T15:11:27.211805+0000","last_fullsized":"2020-08-07T15:11:27.211805+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:24.576023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:10:24.576023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"257","reported_epoch":"211","state":"active+clean","last_fresh":"2020-08-07T15:11:27.215805+0000","last_change":"2020-08-07T15:10:11.912614+0000","last_active":"2020-08-07T15:11:27.215805+0000","last_peered":"2020-08-07T15:11:27.215805+0000","last_clean":"2020-08-07T15:11:27.215805+0000","last_became_active":"2020-08-07T15:08:36.628365+0000","last_became_peered":"2020-08-07T15:08:36.628365+0000","last_unstale":"2020-08-07T15:11:27.215805+0000","last_undegraded":"2020-08-07T15:11:27.215805+0000","last_fullsized":"2020-08-07T15:11:27.215805+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:11.912550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:11.912550+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.5","version":"212'6096","reported_seq":"6154","reported_epoch":"212","state":"active+undersized+degraded","last_fresh":"2020-08-07T15:11:31.591108+0000","last_change":"2020-08-07T15:10:51.555872+0000","last_active":"2020-08-07T15:11:31.591108+0000","last_peered":"2020-08-07T15:11:31.591108+0000","last_clean":"2020-08-07T15:10:50.059205+0000","last_became_active":"2020-08-07T15:10:51.555872+0000","last_became_peered":"2020-08-07T15:10:51.555872+0000","last_unstale":"2020-08-07T15:11:31.591108+0000","last_undegraded":"2020-08-07T15:10:51.546355+0000","last_fullsized":"2020-08-07T15:10:51.546219+0000","mapping_epoch":191,"log_start":"212'6091","ondisk_log_start":"212'6091","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49938432,"num_objects":762,"num_object_clones":0,"num_object_copies":2286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":762,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6096,"num_write_kb":48768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[{"shards":"0,4","objects":762}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"215","reported_epoch":"211","state":"active+undersized","last_fresh":"2020-08-07T15:11:27.205285+0000","last_change":"2020-08-07T15:10:51.560479+0000","last_active":"2020-08-07T15:11:27.205285+0000","last_peered":"2020-08-07T15:11:27.205285+0000","last_clean":"2020-08-07T15:10:49.526877+0000","last_became_active":"2020-08-07T15:10:51.560479+0000","last_became_peered":"2020-08-07T15:10:51.560479+0000","last_unstale":"2020-08-07T15:11:27.205285+0000","last_undegraded":"2020-08-07T15:11:27.205285+0000","last_fullsized":"2020-08-07T15:10:51.550565+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:52.637791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:09:52.637791+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"207","reported_epoch":"211","state":"active+clean","last_fresh":"2020-08-07T15:11:27.205113+0000","last_change":"2020-08-07T15:10:11.720217+0000","last_active":"2020-08-07T15:11:27.205113+0000","last_peered":"2020-08-07T15:11:27.205113+0000","last_clean":"2020-08-07T15:11:27.205113+0000","last_became_active":"2020-08-07T15:08:36.618164+0000","last_became_peered":"2020-08-07T15:08:36.618164+0000","last_unstale":"2020-08-07T15:11:27.205113+0000","last_undegraded":"2020-08-07T15:11:27.205113+0000","last_fullsized":"2020-08-07T15:11:27.205113+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:11.720143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:11.720143+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"211'6288","reported_seq":"6343","reported_epoch":"211","state":"active+clean","last_fresh":"2020-08-07T15:11:27.602136+0000","last_change":"2020-08-07T15:10:16.443153+0000","last_active":"2020-08-07T15:11:27.602136+0000","last_peered":"2020-08-07T15:11:27.602136+0000","last_clean":"2020-08-07T15:11:27.602136+0000","last_became_active":"2020-08-07T15:10:16.442674+0000","last_became_peered":"2020-08-07T15:10:16.442674+0000","last_unstale":"2020-08-07T15:11:27.602136+0000","last_undegraded":"2020-08-07T15:11:27.602136+0000","last_fullsized":"2020-08-07T15:11:27.602136+0000","mapping_epoch":164,"log_start":"211'6279","ondisk_log_start":"211'6279","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":51511296,"num_objects":786,"num_object_clones":0,"num_object_copies":2358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6288,"num_write_kb":50304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"212","reported_epoch":"211","state":"active+undersized","last_fresh":"2020-08-07T15:11:27.215821+0000","last_change":"2020-08-07T15:10:51.561524+0000","last_active":"2020-08-07T15:11:27.215821+0000","last_peered":"2020-08-07T15:11:27.215821+0000","last_clean":"2020-08-07T15:10:49.540461+0000","last_became_active":"2020-08-07T15:10:51.561524+0000","last_became_peered":"2020-08-07T15:10:51.561524+0000","last_unstale":"2020-08-07T15:11:27.215821+0000","last_undegraded":"2020-08-07T15:11:27.215821+0000","last_fullsized":"2020-08-07T15:10:51.549397+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:37.017307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:37.017307+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":["7","0"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"292","reported_epoch":"210","state":"active+clean","last_fresh":"2020-08-07T15:11:26.407317+0000","last_change":"2020-08-07T15:10:50.296729+0000","last_active":"2020-08-07T15:11:26.407317+0000","last_peered":"2020-08-07T15:11:26.407317+0000","last_clean":"2020-08-07T15:11:26.407317+0000","last_became_active":"2020-08-07T15:10:46.496544+0000","last_became_peered":"2020-08-07T15:10:46.496544+0000","last_unstale":"2020-08-07T15:11:26.407317+0000","last_undegraded":"2020-08-07T15:11:26.407317+0000","last_fullsized":"2020-08-07T15:11:26.407317+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:50.296646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:50.296646+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.e","version":"212'6368","reported_seq":"6424","reported_epoch":"212","state":"active+clean","last_fresh":"2020-08-07T15:11:31.420855+0000","last_change":"2020-08-07T15:10:16.448334+0000","last_active":"2020-08-07T15:11:31.420855+0000","last_peered":"2020-08-07T15:11:31.420855+0000","last_clean":"2020-08-07T15:11:31.420855+0000","last_became_active":"2020-08-07T15:10:16.447541+0000","last_became_peered":"2020-08-07T15:10:16.447541+0000","last_unstale":"2020-08-07T15:11:31.420855+0000","last_undegraded":"2020-08-07T15:11:31.420855+0000","last_fullsized":"2020-08-07T15:11:31.420855+0000","mapping_epoch":164,"log_start":"211'6360","ondisk_log_start":"211'6360","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":52166656,"num_objects":796,"num_object_clones":0,"num_object_copies":2388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6368,"num_write_kb":50944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"196","reported_epoch":"211","state":"active+clean","last_fresh":"2020-08-07T15:11:27.205173+0000","last_change":"2020-08-07T15:10:51.644951+0000","last_active":"2020-08-07T15:11:27.205173+0000","last_peered":"2020-08-07T15:11:27.205173+0000","last_clean":"2020-08-07T15:11:27.205173+0000","last_became_active":"2020-08-07T15:10:46.498230+0000","last_became_peered":"2020-08-07T15:10:46.498230+0000","last_unstale":"2020-08-07T15:11:27.205173+0000","last_undegraded":"2020-08-07T15:11:27.205173+0000","last_fullsized":"2020-08-07T15:11:27.205173+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:51.644875+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:10:51.644875+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"212'6872","reported_seq":"6926","reported_epoch":"212","state":"active+undersized+degraded","last_fresh":"2020-08-07T15:11:31.546389+0000","last_change":"2020-08-07T15:10:51.550117+0000","last_active":"2020-08-07T15:11:31.546389+0000","last_peered":"2020-08-07T15:11:31.546389+0000","last_clean":"2020-08-07T15:10:50.051777+0000","last_became_active":"2020-08-07T15:10:51.550117+0000","last_became_peered":"2020-08-07T15:10:51.550117+0000","last_unstale":"2020-08-07T15:11:31.546389+0000","last_undegraded":"2020-08-07T15:10:51.543706+0000","last_fullsized":"2020-08-07T15:10:51.543546+0000","mapping_epoch":191,"log_start":"212'6867","ondisk_log_start":"212'6867","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56295424,"num_objects":859,"num_object_clones":0,"num_object_copies":2577,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":859,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6872,"num_write_kb":54976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[{"shards":"0,4","objects":859}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"33","reported_epoch":"211","state":"active+clean","last_fresh":"2020-08-07T15:11:27.211256+0000","last_change":"2020-08-07T15:11:23.233126+0000","last_active":"2020-08-07T15:11:27.211256+0000","last_peered":"2020-08-07T15:11:27.211256+0000","last_clean":"2020-08-07T15:11:27.211256+0000","last_became_active":"2020-08-07T15:11:23.232406+0000","last_became_peered":"2020-08-07T15:11:23.232406+0000","last_unstale":"2020-08-07T15:11:27.211256+0000","last_undegraded":"2020-08-07T15:11:27.211256+0000","last_fullsized":"2020-08-07T15:11:27.211256+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:55.673165+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:10:55.673165+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.d","version":"212'6480","reported_seq":"6536","reported_epoch":"212","state":"active+clean","last_fresh":"2020-08-07T15:11:31.564832+0000","last_change":"2020-08-07T15:10:16.263289+0000","last_active":"2020-08-07T15:11:31.564832+0000","last_peered":"2020-08-07T15:11:31.564832+0000","last_clean":"2020-08-07T15:11:31.564832+0000","last_became_active":"2020-08-07T15:10:16.262869+0000","last_became_peered":"2020-08-07T15:10:16.262869+0000","last_unstale":"2020-08-07T15:11:31.564832+0000","last_undegraded":"2020-08-07T15:11:31.564832+0000","last_fullsized":"2020-08-07T15:11:31.564832+0000","mapping_epoch":164,"log_start":"212'6469","ondisk_log_start":"212'6469","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53084160,"num_objects":810,"num_object_clones":0,"num_object_copies":2430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6480,"num_write_kb":51840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"260","reported_epoch":"211","state":"active+clean","last_fresh":"2020-08-07T15:11:27.215650+0000","last_change":"2020-08-07T15:10:58.029660+0000","last_active":"2020-08-07T15:11:27.215650+0000","last_peered":"2020-08-07T15:11:27.215650+0000","last_clean":"2020-08-07T15:11:27.215650+0000","last_became_active":"2020-08-07T15:10:46.499635+0000","last_became_peered":"2020-08-07T15:10:46.499635+0000","last_unstale":"2020-08-07T15:11:27.215650+0000","last_undegraded":"2020-08-07T15:11:27.215650+0000","last_fullsized":"2020-08-07T15:11:27.215650+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:58.029609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:58.029609+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"212'6504","reported_seq":"6559","reported_epoch":"212","state":"active+clean","last_fresh":"2020-08-07T15:11:31.603860+0000","last_change":"2020-08-07T15:10:16.256000+0000","last_active":"2020-08-07T15:11:31.603860+0000","last_peered":"2020-08-07T15:11:31.603860+0000","last_clean":"2020-08-07T15:11:31.603860+0000","last_became_active":"2020-08-07T15:10:16.255356+0000","last_became_peered":"2020-08-07T15:10:16.255356+0000","last_unstale":"2020-08-07T15:11:31.603860+0000","last_undegraded":"2020-08-07T15:11:31.603860+0000","last_fullsized":"2020-08-07T15:11:31.603860+0000","mapping_epoch":164,"log_start":"212'6498","ondisk_log_start":"212'6498","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53280768,"num_objects":813,"num_object_clones":0,"num_object_copies":2439,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6504,"num_write_kb":52032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"218","reported_epoch":"211","state":"active+undersized","last_fresh":"2020-08-07T15:11:27.215991+0000","last_change":"2020-08-07T15:10:51.553978+0000","last_active":"2020-08-07T15:11:27.215991+0000","last_peered":"2020-08-07T15:11:27.215991+0000","last_clean":"2020-08-07T15:10:49.540110+0000","last_became_active":"2020-08-07T15:10:51.553978+0000","last_became_peered":"2020-08-07T15:10:51.553978+0000","last_unstale":"2020-08-07T15:11:27.215991+0000","last_undegraded":"2020-08-07T15:11:27.215991+0000","last_fullsized":"2020-08-07T15:10:51.546352+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:10:10.962480+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.962480+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"212'6472","reported_seq":"6518","reported_epoch":"212","state":"active+undersized+degraded","last_fresh":"2020-08-07T15:11:31.486342+0000","last_change":"2020-08-07T15:10:51.550411+0000","last_active":"2020-08-07T15:11:31.486342+0000","last_peered":"2020-08-07T15:11:31.486342+0000","last_clean":"2020-08-07T15:10:50.163110+0000","last_became_active":"2020-08-07T15:10:51.550411+0000","last_became_peered":"2020-08-07T15:10:51.550411+0000","last_unstale":"2020-08-07T15:11:31.486342+0000","last_undegraded":"2020-08-07T15:10:51.545024+0000","last_fullsized":"2020-08-07T15:10:51.544891+0000","mapping_epoch":191,"log_start":"212'6467","ondisk_log_start":"212'6467","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53018624,"num_objects":809,"num_object_clones":0,"num_object_copies":2427,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":809,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6472,"num_write_kb":51776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":809}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"182","reported_epoch":"211","state":"active+clean","last_fresh":"2020-08-07T15:11:27.247087+0000","last_change":"2020-08-07T15:10:50.654783+0000","last_active":"2020-08-07T15:11:27.247087+0000","last_peered":"2020-08-07T15:11:27.247087+0000","last_clean":"2020-08-07T15:11:27.247087+0000","last_became_active":"2020-08-07T15:10:46.498051+0000","last_became_peered":"2020-08-07T15:10:46.498051+0000","last_unstale":"2020-08-07T15:11:27.247087+0000","last_undegraded":"2020-08-07T15:11:27.247087+0000","last_fullsized":"2020-08-07T15:11:27.247087+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:50.654689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:50.654689+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"211'6160","reported_seq":"7655","reported_epoch":"212","state":"peering","last_fresh":"2020-08-07T15:11:31.405466+0000","last_change":"2020-08-07T15:11:31.405899+0000","last_active":"2020-08-07T15:11:31.405466+0000","last_peered":"2020-08-07T15:11:31.353071+0000","last_clean":"2020-08-07T15:11:31.353071+0000","last_became_active":"2020-08-07T15:11:21.380498+0000","last_became_peered":"2020-08-07T15:11:21.380498+0000","last_unstale":"2020-08-07T15:11:31.405466+0000","last_undegraded":"2020-08-07T15:11:31.405466+0000","last_fullsized":"2020-08-07T15:11:31.405466+0000","mapping_epoch":212,"log_start":"211'6152","ondisk_log_start":"211'6152","created":164,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50462720,"num_objects":770,"num_object_clones":0,"num_object_copies":2310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6160,"num_write_kb":49280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":712,"num_bytes_recovered":46661632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5,7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"245","reported_epoch":"210","state":"active+clean","last_fresh":"2020-08-07T15:11:26.407258+0000","last_change":"2020-08-07T15:10:40.358431+0000","last_active":"2020-08-07T15:11:26.407258+0000","last_peered":"2020-08-07T15:11:26.407258+0000","last_clean":"2020-08-07T15:11:26.407258+0000","last_became_active":"2020-08-07T15:08:41.689435+0000","last_became_peered":"2020-08-07T15:08:41.689435+0000","last_unstale":"2020-08-07T15:11:26.407258+0000","last_undegraded":"2020-08-07T15:11:26.407258+0000","last_fullsized":"2020-08-07T15:11:26.407258+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:40.358337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:40.358337+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"212'6264","reported_seq":"6322","reported_epoch":"212","state":"active+clean","last_fresh":"2020-08-07T15:11:31.552470+0000","last_change":"2020-08-07T15:11:19.764496+0000","last_active":"2020-08-07T15:11:31.552470+0000","last_peered":"2020-08-07T15:11:31.552470+0000","last_clean":"2020-08-07T15:11:31.552470+0000","last_became_active":"2020-08-07T15:10:16.256419+0000","last_became_peered":"2020-08-07T15:10:16.256419+0000","last_unstale":"2020-08-07T15:11:31.552470+0000","last_undegraded":"2020-08-07T15:11:31.552470+0000","last_fullsized":"2020-08-07T15:11:31.552470+0000","mapping_epoch":164,"log_start":"212'6259","ondisk_log_start":"212'6259","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'5328","last_scrub_stamp":"2020-08-07T15:11:19.764422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:11:19.764422+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":51314688,"num_objects":783,"num_object_clones":0,"num_object_copies":2349,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6264,"num_write_kb":50112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"146","reported_epoch":"211","state":"active+clean","last_fresh":"2020-08-07T15:11:27.215831+0000","last_change":"2020-08-07T15:11:10.853414+0000","last_active":"2020-08-07T15:11:27.215831+0000","last_peered":"2020-08-07T15:11:27.215831+0000","last_clean":"2020-08-07T15:11:27.215831+0000","last_became_active":"2020-08-07T15:10:46.493498+0000","last_became_peered":"2020-08-07T15:10:46.493498+0000","last_unstale":"2020-08-07T15:11:27.215831+0000","last_undegraded":"2020-08-07T15:11:27.215831+0000","last_fullsized":"2020-08-07T15:11:27.215831+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:10.853337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:10.853337+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"212'6512","reported_seq":"6567","reported_epoch":"212","state":"active+clean","last_fresh":"2020-08-07T15:11:31.583109+0000","last_change":"2020-08-07T15:10:16.253662+0000","last_active":"2020-08-07T15:11:31.583109+0000","last_peered":"2020-08-07T15:11:31.583109+0000","last_clean":"2020-08-07T15:11:31.583109+0000","last_became_active":"2020-08-07T15:10:16.253105+0000","last_became_peered":"2020-08-07T15:10:16.253105+0000","last_unstale":"2020-08-07T15:11:31.583109+0000","last_undegraded":"2020-08-07T15:11:31.583109+0000","last_fullsized":"2020-08-07T15:11:31.583109+0000","mapping_epoch":164,"log_start":"212'6506","ondisk_log_start":"212'6506","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53346304,"num_objects":814,"num_object_clones":0,"num_object_copies":2442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6512,"num_write_kb":52096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"171","reported_epoch":"211","state":"active+undersized","last_fresh":"2020-08-07T15:11:27.205649+0000","last_change":"2020-08-07T15:10:51.561336+0000","last_active":"2020-08-07T15:11:27.205649+0000","last_peered":"2020-08-07T15:11:27.205649+0000","last_clean":"2020-08-07T15:10:49.526944+0000","last_became_active":"2020-08-07T15:10:51.561336+0000","last_became_peered":"2020-08-07T15:10:51.561336+0000","last_unstale":"2020-08-07T15:11:27.205649+0000","last_undegraded":"2020-08-07T15:11:27.205649+0000","last_fullsized":"2020-08-07T15:10:51.547335+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:37.601099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:37.601099+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":840892416,"num_objects":12831,"num_object_clones":0,"num_object_copies":38493,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3247,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102648,"num_write_kb":821184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1498,"num_bytes_recovered":98172928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2477858816,"data_stored":2477858816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":43,"acting":44,"num_store_stats":8},{"poolid":1,"num_pg":31,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":149,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":83,"acting":88,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542208,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1248784,"kb_used_data":200200,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93123056,"statfs":{"total":96636764160,"available":95358009344,"internally_reserved":1073741824,"allocated":205004800,"data_stored":204085519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":43.911000000000001,"5min":9.5549999999999997,"15min":3.7799999999999998},"min":{"1min":0.50800000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":856.07100000000003,"5min":856.07100000000003,"15min":856.07100000000003},"last":1.069},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.026,"15min":0.98399999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.8899999999999999,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.135}]},{"osd":1,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.335,"15min":1.3080000000000001},"min":{"1min":0.64200000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.9199999999999999,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.141,"15min":1.024},"min":{"1min":0.61799999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":4.0060000000000002,"5min":4.0060000000000002,"15min":4.0060000000000002},"last":2.4049999999999998}]},{"osd":2,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.304,"15min":1.1859999999999999},"min":{"1min":0.747,"5min":0.628,"15min":0.628},"max":{"1min":2.0249999999999999,"5min":5.6619999999999999,"15min":5.6619999999999999},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.512,"5min":1.3380000000000001,"15min":1.1910000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":3.6619999999999999,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":1.732}]},{"osd":3,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":16.18,"15min":6.508},"min":{"1min":0.64700000000000002,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.9710000000000001,"5min":917.38699999999994,"15min":917.38699999999994},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.196,"5min":4.9649999999999999,"15min":2.5499999999999998},"min":{"1min":0.77200000000000002,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.3359999999999999,"5min":333.59199999999998,"15min":333.59199999999998},"last":1.3680000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.464,"15min":1.3380000000000001},"min":{"1min":0.63400000000000001,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":3.9750000000000001,"5min":4.835,"15min":4.835},"last":1.774},{"interface":"front","average":{"1min":1.306,"5min":11.518000000000001,"15min":4.7140000000000004},"min":{"1min":0.69699999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":3.0329999999999999,"5min":866.24099999999999,"15min":866.24099999999999},"last":1.409}]},{"osd":5,"last update":"Fri Aug 7 15:10:27 2020","interfaces":[{"interface":"back","average":{"1min":39.496000000000002,"5min":23.838000000000001,"15min":32.950000000000003},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":678.76099999999997,"5min":831.88099999999997,"15min":831.88099999999997},"last":1.351},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.823,"15min":1.6040000000000001},"min":{"1min":0.84999999999999998,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.4060000000000001,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":1.4359999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":3.9929999999999999,"15min":2.343},"min":{"1min":0.89900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":3.0049999999999999,"5min":224.14500000000001,"15min":224.14500000000001},"last":3.3319999999999999},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.6899999999999999,"15min":1.5680000000000001},"min":{"1min":1.113,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.2919999999999998,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":2.8599999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1352120,"kb_used_data":303536,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93019720,"statfs":{"total":96636764160,"available":95252193280,"internally_reserved":1073741824,"allocated":310820864,"data_stored":309901583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":42.923000000000002,"5min":9.4640000000000004,"15min":4.0430000000000001},"min":{"1min":0.48999999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":755.10000000000002,"5min":755.10000000000002,"15min":755.10000000000002},"last":1.268},{"interface":"front","average":{"1min":0.96699999999999997,"5min":9.7780000000000005,"15min":4.0259999999999998},"min":{"1min":0.46300000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.718,"5min":529.88699999999994,"15min":529.88699999999994},"last":0.88}]},{"osd":1,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":5.758,"15min":2.5579999999999998},"min":{"1min":0.47699999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.7290000000000001,"5min":547.65300000000002,"15min":547.65300000000002},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.4159999999999999,"5min":4.9189999999999996,"15min":2.5640000000000001},"min":{"1min":0.57399999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":5.4960000000000004,"5min":372.50799999999998,"15min":372.50799999999998},"last":0.95099999999999996}]},{"osd":2,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.3240000000000001,"15min":1.238},"min":{"1min":0.54900000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":6.968,"5min":6.968,"15min":6.968},"last":2.0590000000000002},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1799999999999999,"15min":1.2410000000000001},"min":{"1min":0.58599999999999997,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.7709999999999999,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":1.7849999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":9.5,"15min":4.1840000000000002},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":3.3540000000000001,"5min":698.19899999999996,"15min":698.19899999999996},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.2969999999999999,"15min":1.357},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":4.1779999999999999,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":0.84099999999999997}]},{"osd":4,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.3120000000000001,"15min":1.546},"min":{"1min":0.60999999999999999,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.5449999999999999,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.53,"15min":1.6240000000000001},"min":{"1min":0.71399999999999997,"5min":0.621,"15min":0.621},"max":{"1min":2.9900000000000002,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.3919999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.7130000000000001,"5min":1.329,"15min":1.3520000000000001},"min":{"1min":0.749,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":6.2080000000000002,"5min":6.2080000000000002,"15min":6.2080000000000002},"last":2.0259999999999998},{"interface":"front","average":{"1min":1.7350000000000001,"5min":1.526,"15min":1.6859999999999999},"min":{"1min":0.89000000000000001,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":3.052,"5min":5.327,"15min":5.327},"last":1.5680000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":40.012999999999998,"5min":9.1820000000000004,"15min":4.016},"min":{"1min":0.86399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":695.92899999999997,"5min":695.92899999999997,"15min":695.92899999999997},"last":1.671},{"interface":"front","average":{"1min":1.613,"5min":4.8959999999999999,"15min":13.849},"min":{"1min":0.91300000000000003,"5min":0.87,"15min":0.87},"max":{"1min":3.0270000000000001,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.5389999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1155876,"kb_used_data":107292,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93215964,"statfs":{"total":96636764160,"available":95453147136,"internally_reserved":1073741824,"allocated":109867008,"data_stored":109130383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":13.122,"5min":16.731999999999999,"15min":28.367999999999999},"min":{"1min":0.47199999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":211.63399999999999,"5min":497.04000000000002,"15min":497.04000000000002},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.645,"5min":14.757999999999999,"15min":5.673},"min":{"1min":0.57399999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":8.8849999999999998,"5min":993.84500000000003,"15min":993.84500000000003},"last":1.2929999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":54.984000000000002,"5min":11.816000000000001,"15min":4.5380000000000003},"min":{"1min":0.57199999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":959.36900000000003,"5min":959.36900000000003,"15min":959.36900000000003},"last":1.407},{"interface":"front","average":{"1min":1.72,"5min":1.357,"15min":1.2549999999999999},"min":{"1min":0.53600000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":5.4530000000000003,"5min":6.4029999999999996,"15min":6.4029999999999996},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":9.2420000000000009,"15min":3.9540000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.3879999999999999,"5min":828.31600000000003,"15min":828.31600000000003},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.2130000000000001,"5min":10.119999999999999,"15min":4.3419999999999996},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.5659999999999998,"5min":905.38300000000004,"15min":905.38300000000004},"last":1.071}]},{"osd":4,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":49.530999999999999,"5min":16.422999999999998,"15min":6.3300000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":872.58500000000004,"5min":872.58500000000004,"15min":872.58500000000004},"last":1.137},{"interface":"front","average":{"1min":2.2080000000000002,"5min":1.5409999999999999,"15min":1.3129999999999999},"min":{"1min":0.75800000000000001,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":12.640000000000001,"5min":12.640000000000001,"15min":12.640000000000001},"last":0.64100000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":6.0919999999999996,"15min":10.702},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.056,"5min":213.63800000000001,"15min":213.63800000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":2.4630000000000001,"5min":12.974,"15min":5.1180000000000003},"min":{"1min":0.93400000000000005,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":12.677,"5min":875.44600000000003,"15min":875.44600000000003},"last":1.276}]},{"osd":6,"last update":"Fri Aug 7 15:10:32 2020","interfaces":[{"interface":"back","average":{"1min":2.9340000000000002,"5min":1.863,"15min":1.655},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":22.402000000000001,"5min":22.402000000000001,"15min":22.402000000000001},"last":1.786},{"interface":"front","average":{"1min":1.3160000000000001,"5min":3.589,"15min":2.1030000000000002},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.3980000000000001,"5min":207.137,"15min":207.137},"last":1.105}]},{"osd":7,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":2.0910000000000002,"5min":1.7829999999999999,"15min":1.671},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":5.4960000000000004,"5min":6.444,"15min":6.444},"last":2.04},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.655,"15min":1.538},"min":{"1min":0.78700000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.0219999999999998,"5min":10.752000000000001,"15min":10.752000000000001},"last":1.1679999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1462544,"kb_used_data":413960,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92909296,"statfs":{"total":96636764160,"available":95139119104,"internally_reserved":1073741824,"allocated":423895040,"data_stored":422975759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":24.291,"5min":21.719999999999999,"15min":7.891},"min":{"1min":0.52700000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":432.899,"5min":845.57100000000003,"15min":845.57100000000003},"last":1.96},{"interface":"front","average":{"1min":1.752,"5min":1.181,"15min":1.0149999999999999},"min":{"1min":0.62,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":9.3650000000000002,"5min":9.3650000000000002,"15min":9.3650000000000002},"last":1.363}]},{"osd":2,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.0249999999999999,"15min":0.95999999999999996},"min":{"1min":0.52200000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.972,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":0.505},{"interface":"front","average":{"1min":1.296,"5min":16.190999999999999,"15min":27.222999999999999},"min":{"1min":0.55900000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.8519999999999999,"5min":606.18499999999995,"15min":606.18499999999995},"last":0.42799999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":1.7809999999999999,"5min":1.2350000000000001,"15min":1.0349999999999999},"min":{"1min":0.54300000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":9.3360000000000003,"5min":9.3360000000000003,"15min":9.3360000000000003},"last":1.839},{"interface":"front","average":{"1min":1.677,"5min":1.4179999999999999,"15min":1.2509999999999999},"min":{"1min":0.63600000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":11.19,"5min":11.19,"15min":11.19},"last":1.863}]},{"osd":4,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.355,"15min":1.2290000000000001},"min":{"1min":0.68300000000000005,"5min":0.63,"15min":0.63},"max":{"1min":3.9729999999999999,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":0.876},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.4510000000000001,"15min":1.3220000000000001},"min":{"1min":0.83899999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.012,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":1.603}]},{"osd":5,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":9.0800000000000001,"15min":3.8439999999999999},"min":{"1min":0.75600000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":3.883,"5min":501.267,"15min":501.267},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.669,"5min":1.3740000000000001,"15min":1.1950000000000001},"min":{"1min":0.752,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":4.0140000000000002,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":0.91300000000000003}]},{"osd":6,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":4.3310000000000004,"15min":7.1859999999999999},"min":{"1min":0.84999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.1400000000000001,"5min":124.738,"15min":124.738},"last":1.6859999999999999},{"interface":"front","average":{"1min":1.611,"5min":8.4019999999999992,"15min":15.236000000000001},"min":{"1min":0.95899999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":3.9119999999999999,"5min":294.94,"15min":294.94},"last":1.2689999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":1.647,"5min":1.571,"15min":1.581},"min":{"1min":0.98899999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":3.9369999999999998,"5min":6.468,"15min":6.468},"last":1.0429999999999999},{"interface":"front","average":{"1min":46.985999999999997,"5min":10.561,"15min":4.444},"min":{"1min":0.88200000000000001,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":859.82399999999996,"5min":859.82399999999996,"15min":859.82399999999996},"last":1.2949999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1353564,"kb_used_data":304980,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93018276,"statfs":{"total":96636764160,"available":95250714624,"internally_reserved":1073741824,"allocated":312299520,"data_stored":311372941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,5,1,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:11:22 2020","interfaces":[{"interface":"back","average":{"1min":2.2519999999999998,"5min":1.4950000000000001,"15min":1.2430000000000001},"min":{"1min":0.46000000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":23.699000000000002,"5min":23.699000000000002,"15min":23.699000000000002},"last":0.42699999999999999},{"interface":"front","average":{"1min":2.1059999999999999,"5min":1.417,"15min":1.1220000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":23.792999999999999,"5min":23.792999999999999,"15min":23.792999999999999},"last":0.64000000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:10:21 2020","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.6970000000000001,"15min":1.2929999999999999},"min":{"1min":0.52000000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":5.5919999999999996,"5min":18.672000000000001,"15min":18.672000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.264,"5min":1.3320000000000001,"15min":1.3500000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":3.7970000000000002,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":0.93400000000000005}]},{"osd":3,"last update":"Fri Aug 7 15:11:22 2020","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":1.234,"15min":1.1859999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.681,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":15.957000000000001,"5min":10.978999999999999,"15min":4.6970000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":312.87900000000002,"5min":501.75599999999997,"15min":501.75599999999997},"last":0.47799999999999998}]},{"osd":4,"last update":"Fri Aug 7 15:11:22 2020","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.603,"15min":1.274},"min":{"1min":0.623,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":7.9569999999999999,"5min":19.783000000000001,"15min":19.783000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.8049999999999999,"5min":1.554,"15min":1.613},"min":{"1min":0.80700000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":8.048,"5min":8.048,"15min":8.048},"last":0.871}]},{"osd":5,"last update":"Fri Aug 7 15:11:22 2020","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.367,"15min":1.2969999999999999},"min":{"1min":0.72899999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":4.3639999999999999,"5min":5.2220000000000004,"15min":5.2220000000000004},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.2989999999999999,"5min":20.52,"15min":23.856000000000002},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":3.7559999999999998,"5min":627.53300000000002,"15min":627.53300000000002},"last":0.67200000000000004}]},{"osd":6,"last update":"Fri Aug 7 15:10:32 2020","interfaces":[{"interface":"back","average":{"1min":1.8,"5min":1.6539999999999999,"15min":1.365},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":4.1639999999999997,"5min":20.035,"15min":20.035},"last":821.91899999999998},{"interface":"front","average":{"1min":1.6459999999999999,"5min":9.9629999999999992,"15min":4.2009999999999996},"min":{"1min":0.95499999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":5.3019999999999996,"5min":751.70000000000005,"15min":751.70000000000005},"last":0.89600000000000002}]},{"osd":7,"last update":"Fri Aug 7 15:11:22 2020","interfaces":[{"interface":"back","average":{"1min":1.899,"5min":12.601000000000001,"15min":5.3609999999999998},"min":{"1min":0.88500000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":7.9900000000000002,"5min":501.91699999999997,"15min":501.91699999999997},"last":0.91500000000000004},{"interface":"front","average":{"1min":2.044,"5min":2.0630000000000002,"15min":1.6850000000000001},"min":{"1min":0.94199999999999995,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":8.0779999999999994,"5min":35.646999999999998,"15min":35.646999999999998},"last":1.0029999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1203868,"kb_used_data":155284,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93167972,"statfs":{"total":96636764160,"available":95404003328,"internally_reserved":1073741824,"allocated":159010816,"data_stored":158084237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.2689999999999999,"15min":1.258},"min":{"1min":0.51100000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":3.2149999999999999,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.296,"15min":1.5229999999999999},"min":{"1min":0.53900000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":4.0190000000000001,"5min":13.067,"15min":13.067},"last":0.42099999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":4.4779999999999998,"15min":2.758},"min":{"1min":0.59099999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.847,"5min":270.68700000000001,"15min":270.68700000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":1.109,"5min":1.8580000000000001,"15min":1.96},"min":{"1min":0.54100000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.0550000000000002,"5min":14.349,"15min":14.349},"last":0.56699999999999995}]},{"osd":2,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.3380000000000001,"15min":1.2110000000000001},"min":{"1min":0.65300000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.6389999999999998,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.9810000000000001,"15min":1.7829999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":6.7889999999999997,"5min":51.530000000000001,"15min":51.530000000000001},"last":0.64500000000000002}]},{"osd":4,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":17.366,"15min":6.9199999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":3.1600000000000001,"5min":646.26700000000005,"15min":646.26700000000005},"last":56.688000000000002},{"interface":"front","average":{"1min":43.484000000000002,"5min":21.631,"15min":28.609999999999999},"min":{"1min":0.77100000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":844.10500000000002,"5min":844.10500000000002,"15min":844.10500000000002},"last":0.91700000000000004}]},{"osd":5,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":5.9560000000000004,"15min":2.9969999999999999},"min":{"1min":0.79300000000000004,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.2370000000000001,"5min":403.88400000000001,"15min":403.88400000000001},"last":1.145},{"interface":"front","average":{"1min":1.591,"5min":1.895,"15min":1.8109999999999999},"min":{"1min":0.97399999999999998,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":3.9790000000000001,"5min":8.9120000000000008,"15min":8.9120000000000008},"last":1.1930000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.653,"15min":1.5720000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":4.1600000000000001,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":1.169},{"interface":"front","average":{"1min":1.944,"5min":13.427,"15min":5.7999999999999998},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":6.9980000000000002,"5min":688.15999999999997,"15min":688.15999999999997},"last":1.2969999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.655,"5min":1.6799999999999999,"15min":2.0009999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":6.8209999999999997,"5min":6.8209999999999997,"15min":6.8209999999999997},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.77,"15min":1.9079999999999999},"min":{"1min":0.77000000000000002,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":3.1499999999999999,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":1.0269999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1511352,"kb_used_data":462768,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92860488,"statfs":{"total":96636764160,"available":95089139712,"internally_reserved":1073741824,"allocated":473874432,"data_stored":472955151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:11:25 2020","interfaces":[{"interface":"back","average":{"1min":20.597999999999999,"5min":9.9239999999999995,"15min":4.0949999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":356.91800000000001,"5min":399.73700000000002,"15min":399.73700000000002},"last":1.151},{"interface":"front","average":{"1min":0.84299999999999997,"5min":1.046,"15min":0.91400000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.972,"5min":6.3970000000000002,"15min":6.3970000000000002},"last":0.76000000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:10:24 2020","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":5.7629999999999999,"15min":2.823},"min":{"1min":0.52900000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.8109999999999999,"5min":320.67000000000002,"15min":320.67000000000002},"last":1.085},{"interface":"front","average":{"1min":1.177,"5min":7.5300000000000002,"15min":3.4279999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.7200000000000002,"5min":689.846,"15min":689.846},"last":0.66900000000000004}]},{"osd":2,"last update":"Fri Aug 7 15:11:25 2020","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.3680000000000001,"15min":1.4239999999999999},"min":{"1min":0.67300000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.786,"5min":6.8730000000000002,"15min":6.8730000000000002},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.446,"5min":1.51,"15min":1.5129999999999999},"min":{"1min":0.85899999999999999,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.8140000000000001,"5min":7.1360000000000001,"15min":7.1360000000000001},"last":1.52}]},{"osd":3,"last update":"Fri Aug 7 15:11:25 2020","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":12.045,"15min":5.0199999999999996},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.6579999999999999,"5min":638.92100000000005,"15min":638.92100000000005},"last":1.129},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.3480000000000001,"15min":1.2829999999999999},"min":{"1min":0.70899999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.0289999999999999,"5min":6.431,"15min":6.431},"last":2.0379999999999998}]},{"osd":5,"last update":"Fri Aug 7 15:11:25 2020","interfaces":[{"interface":"back","average":{"1min":41.594000000000001,"5min":18.439,"15min":7.2309999999999999},"min":{"1min":0.60499999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":725.98599999999999,"5min":990.71600000000001,"15min":990.71600000000001},"last":4.7960000000000003},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.4259999999999999,"15min":1.5600000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.7310000000000001,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":1.105}]},{"osd":6,"last update":"Fri Aug 7 15:10:32 2020","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.661,"15min":1.7470000000000001},"min":{"1min":0.73399999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.7000000000000002,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.282,"5min":8.5039999999999996,"15min":15.574},"min":{"1min":0.93700000000000006,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.0870000000000002,"5min":389.14600000000002,"15min":389.14600000000002},"last":2.3719999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:11:25 2020","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":3.782,"15min":2.1989999999999998},"min":{"1min":0.78300000000000003,"5min":0.62,"15min":0.62},"max":{"1min":2.5310000000000001,"5min":252.798,"15min":252.798},"last":3.1400000000000001},{"interface":"front","average":{"1min":1.4370000000000001,"5min":14.112,"15min":5.8010000000000002},"min":{"1min":0.78100000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":3.0369999999999999,"5min":501.93000000000001,"15min":501.93000000000001},"last":2.3239999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1511872,"kb_used_data":463288,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92859968,"statfs":{"total":96636764160,"available":95088607232,"internally_reserved":1073741824,"allocated":474406912,"data_stored":473487631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:11:26 2020","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.996,"15min":1.05},"min":{"1min":0.54400000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.5289999999999999,"5min":3.7349999999999999,"15min":3.7349999999999999},"last":43.292999999999999},{"interface":"front","average":{"1min":15.863,"5min":3.956,"15min":1.855},"min":{"1min":0.44,"5min":0.438,"15min":0.438},"max":{"1min":329.11399999999998,"5min":329.11399999999998,"15min":329.11399999999998},"last":42.933}]},{"osd":1,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":8.5239999999999991,"5min":12.039999999999999,"15min":4.6319999999999997},"min":{"1min":0.56799999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":139.88200000000001,"5min":787.22900000000004,"15min":787.22900000000004},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.448,"5min":2.5649999999999999,"15min":1.593},"min":{"1min":0.59899999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":4.3700000000000001,"5min":116.548,"15min":116.548},"last":1.5900000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:11:26 2020","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":2.714,"15min":7.0510000000000002},"min":{"1min":0.52100000000000002,"5min":0.499,"15min":0.499},"max":{"1min":2.109,"5min":168.56899999999999,"15min":168.56899999999999},"last":43.923000000000002},{"interface":"front","average":{"1min":1.3939999999999999,"5min":7.6200000000000001,"15min":24.626999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":3.5609999999999999,"5min":676.27099999999996,"15min":676.27099999999996},"last":43.076999999999998}]},{"osd":3,"last update":"Fri Aug 7 15:11:26 2020","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.6559999999999999,"15min":1.2969999999999999},"min":{"1min":0.71499999999999997,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":5.1539999999999999,"5min":29.129000000000001,"15min":29.129000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.1140000000000001,"5min":9.9969999999999999,"15min":4.141},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.149,"5min":712.98699999999997,"15min":712.98699999999997},"last":0.81599999999999995}]},{"osd":4,"last update":"Fri Aug 7 15:11:26 2020","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":22.683,"15min":41.488999999999997},"min":{"1min":0.54800000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.5899999999999999,"5min":971.67999999999995,"15min":971.67999999999995},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.272,"15min":1.294},"min":{"1min":0.63700000000000001,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":5.4900000000000002,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":0.73599999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:11:26 2020","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":31.462,"15min":44.093000000000004},"min":{"1min":0.85099999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":5.4480000000000004,"5min":966.60900000000004,"15min":966.60900000000004},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":11.677,"15min":21.236999999999998},"min":{"1min":0.871,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.4710000000000001,"5min":517.26700000000005,"15min":517.26700000000005},"last":0.92900000000000005}]},{"osd":7,"last update":"Fri Aug 7 15:11:26 2020","interfaces":[{"interface":"back","average":{"1min":1.6799999999999999,"5min":1.403,"15min":1.3400000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":5.1559999999999997,"5min":5.1559999999999997,"15min":5.1559999999999997},"last":0.85399999999999998},{"interface":"front","average":{"1min":32.561999999999998,"5min":12.834,"15min":5.3220000000000001},"min":{"1min":0.63700000000000001,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":501.64299999999997,"5min":515.09100000000001,"15min":515.09100000000001},"last":0.97599999999999998}]}]}]}} 2020-08-07T15:11:32.719 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:11:32.746 INFO:tasks.ceph.ceph_manager.ceph:would drop below min_pgs, proj 6, currently 16 2020-08-07T15:11:32.747 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:11:32.748 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:11:32.748 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:11:32.748 INFO:teuthology.orchestra.run.smithi074.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-08-07T15:11:32.749 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: problem getting command descriptions from osd.1 2020-08-07T15:11:32.749 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:11:32.749 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:11:32.749 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:11:32.749 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:11:32.801 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:11:32.802 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:11:32.803 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-08-07T15:11:32.803 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-08-07T15:11:32.827 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:32.822+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:32.928 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:32.926+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:33.060 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:33.057+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:33.177 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:33.161+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:33.218 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 73 16 103938 103922 11.1207 4.20312 0.0755047 0.0112258 2020-08-07T15:11:33.267 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:33.266+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:33.368 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:33.367+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:33.426 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:11:33.478 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:33.469+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:33.574 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:33.573+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:33.697 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:33.689+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:33.802 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:33.800+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:33.903 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:33.901+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:34.008 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:34.005+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:34.110 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:34.106+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:34.216 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:34.206+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:34.217 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 74 16 105473 105457 11.1325 11.9922 0.00355196 0.011223 2020-08-07T15:11:34.322 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:34.321+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:34.435 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:34.434+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:34.540 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:34.538+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:34.660 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:34.659+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:34.772 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:34.760+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:34.806 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:11:34.866 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:34.860+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:34.967 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:34.966+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:35.079 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:35.072+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:35.175 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:35.173+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:35.216 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 75 16 107104 107088 11.1539 12.7422 0.008012 0.0112017 2020-08-07T15:11:35.284 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:35.275+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:35.394 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:35.382+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:35.445 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:11:35.502 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:35.484+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:35.590 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:35.588+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:35.711 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:35.689+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:35.794 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:35.793+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:35.899 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:35.897+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:36.000 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:35.999+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:36.103 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:36.100+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:36.150 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:36.151 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:36.173 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:11:36.193 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:11:36.205 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:36.204+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:36.216 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 76 16 108738 108722 11.1751 12.7656 0.0133981 0.0111802 2020-08-07T15:11:36.217 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:11:36.266 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:11:36.315 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:11:36.315 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:36.313+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:36.352 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:11:36.393 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:11:36.419 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:36.418+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:36.443 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:11:36.443 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:11:36.443 INFO:teuthology.orchestra.run.smithi074.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-08-07T15:11:36.444 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: problem getting command descriptions from osd.1 2020-08-07T15:11:36.444 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:11:36.444 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:11:36.444 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:11:36.446 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:11:36.446 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:11:36.446 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:11:36.447 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-08-07T15:11:36.447 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-08-07T15:11:36.514 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:11:36.514 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:11:36.514 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:11:36.514 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:11:36.515 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.515 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106240 3.7 3:ebdd8242:::benchmark_data_smithi074_35447_object13279:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:11:36.515 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:11:36.515 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 229.155367463, 2020-08-07T15:11:36.515 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.90079396, 2020-08-07T15:11:36.516 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:36.516 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:36.516 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:36.516 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:11:36.516 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:11:36.517 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106240 2020-08-07T15:11:36.517 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.517 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:36.517 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.517 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:36.518 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:11:36.518 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:36.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.521 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.521 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:36.521 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:11:36.522 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2522000000000003e-05 2020-08-07T15:11:36.522 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.522 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.522 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:36.522 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355118+0000", 2020-08-07T15:11:36.523 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6159999999999998e-06 2020-08-07T15:11:36.523 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.523 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.523 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:36.523 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355125+0000", 2020-08-07T15:11:36.524 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2699e-05 2020-08-07T15:11:36.524 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.524 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.524 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:36.524 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355138+0000", 2020-08-07T15:11:36.525 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2310000000000001e-06 2020-08-07T15:11:36.542 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.543 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.543 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:36.543 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355144+0000", 2020-08-07T15:11:36.543 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9855000000000003e-05 2020-08-07T15:11:36.544 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.544 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:36.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355194+0000", 2020-08-07T15:11:36.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112713 2020-08-07T15:11:36.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.558 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.559 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:36.559 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355306+0000", 2020-08-07T15:11:36.559 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019794600000000001 2020-08-07T15:11:36.559 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.560 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.560 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:11:36.560 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355504+0000", 2020-08-07T15:11:36.560 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0048346029999999998 2020-08-07T15:11:36.560 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.561 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.561 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:36.561 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.360339+0000", 2020-08-07T15:11:36.561 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00033646400000000002 2020-08-07T15:11:36.561 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.589 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:36.590 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.360675+0000", 2020-08-07T15:11:36.593 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89508481200000001 2020-08-07T15:11:36.593 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.593 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.593 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:36.613 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255760+0000", 2020-08-07T15:11:36.613 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5659999999999999e-06 2020-08-07T15:11:36.614 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.614 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.614 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:36.614 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255762+0000", 2020-08-07T15:11:36.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6595000000000003e-05 2020-08-07T15:11:36.615 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.615 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.615 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:36.615 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255818+0000", 2020-08-07T15:11:36.615 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1338000000000003e-05 2020-08-07T15:11:36.616 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.616 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.616 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:36.616 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255860+0000", 2020-08-07T15:11:36.616 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.90079396 2020-08-07T15:11:36.617 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:36.617 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:36.617 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:36.617 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.617 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.617 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106240 3.7 e59/20 ondisk, result = 0)", 2020-08-07T15:11:36.618 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:11:36.618 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 229.05897151100001, 2020-08-07T15:11:36.618 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80440215100000001, 2020-08-07T15:11:36.618 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:36.618 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:11:36.619 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:36.619 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.619 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:36.619 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:11:36.619 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:36.620 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.620 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.620 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:36.620 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:11:36.620 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0052999999999999e-05 2020-08-07T15:11:36.621 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.621 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.621 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:36.621 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451482+0000", 2020-08-07T15:11:36.621 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3229999999999999e-06 2020-08-07T15:11:36.622 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.622 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.622 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:36.622 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451486+0000", 2020-08-07T15:11:36.622 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80420304799999998 2020-08-07T15:11:36.623 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.623 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.623 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:36.623 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255689+0000", 2020-08-07T15:11:36.623 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7759999999999997e-06 2020-08-07T15:11:36.624 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.624 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.642 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:36.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255694+0000", 2020-08-07T15:11:36.642 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5828999999999998e-05 2020-08-07T15:11:36.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.643 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.643 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:36.643 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255740+0000", 2020-08-07T15:11:36.644 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1466e-05 2020-08-07T15:11:36.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.644 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.644 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:36.644 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255751+0000", 2020-08-07T15:11:36.645 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112656 2020-08-07T15:11:36.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:36.645 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255864+0000", 2020-08-07T15:11:36.666 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80440215100000001 2020-08-07T15:11:36.666 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:36.679 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:36.680 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:36.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.680 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37057 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:11:36.681 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:11:36.681 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 162.266862801, 2020-08-07T15:11:36.681 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007248691, 2020-08-07T15:11:36.681 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:36.681 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:11:36.682 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:36.682 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:36.682 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:36.682 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37057 2020-08-07T15:11:36.682 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.683 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:36.683 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.683 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:36.683 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:11:36.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:36.684 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.684 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.684 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:36.684 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:11:36.684 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3661e-05 2020-08-07T15:11:36.684 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.685 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.685 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:36.685 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243624+0000", 2020-08-07T15:11:36.685 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7519999999999996e-06 2020-08-07T15:11:36.694 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.695 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:36.695 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243632+0000", 2020-08-07T15:11:36.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6392000000000002e-05 2020-08-07T15:11:36.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.696 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:36.696 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243648+0000", 2020-08-07T15:11:36.697 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6659999999999997e-06 2020-08-07T15:11:36.697 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:36.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243657+0000", 2020-08-07T15:11:36.697 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0070484609999999998 2020-08-07T15:11:36.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.698 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.698 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:36.698 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250705+0000", 2020-08-07T15:11:36.699 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0083e-05 2020-08-07T15:11:36.699 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.699 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.699 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:36.699 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250715+0000", 2020-08-07T15:11:36.699 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000086695 2020-08-07T15:11:36.700 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.700 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:36.701 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.250802+0000", 2020-08-07T15:11:36.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6980999999999999e-05 2020-08-07T15:11:36.701 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.701 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:36.702 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.250819+0000", 2020-08-07T15:11:36.704 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007248691 2020-08-07T15:11:36.705 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:36.705 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:36.705 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:36.716 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.716 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.716 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37058 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:11:36.716 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:11:36.717 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 162.26672106699999, 2020-08-07T15:11:36.717 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007368357, 2020-08-07T15:11:36.717 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:36.717 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:11:36.717 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:36.718 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:36.718 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:36.718 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37058 2020-08-07T15:11:36.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.718 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:36.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:36.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:11:36.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:36.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.790 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:36.790 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:11:36.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8838999999999998e-05 2020-08-07T15:11:36.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:36.791 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243761+0000", 2020-08-07T15:11:36.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2079999999999999e-06 2020-08-07T15:11:36.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.802 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.802 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:36.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243766+0000", 2020-08-07T15:11:36.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0567999999999999e-05 2020-08-07T15:11:36.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:36.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243777+0000", 2020-08-07T15:11:36.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7210000000000002e-06 2020-08-07T15:11:36.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:36.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243782+0000", 2020-08-07T15:11:36.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069508749999999996 2020-08-07T15:11:36.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.822 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:36.822 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250732+0000", 2020-08-07T15:11:36.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7899999999999999e-06 2020-08-07T15:11:36.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:36.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250737+0000", 2020-08-07T15:11:36.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003272409999999 2020-08-07T15:11:36.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:36.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251064+0000", 2020-08-07T15:11:36.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6115000000000001e-05 2020-08-07T15:11:36.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:36.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251081+0000", 2020-08-07T15:11:36.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007368357 2020-08-07T15:11:36.838 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:36.838 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:36.838 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:36.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.839 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37059 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:11:36.839 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:11:36.839 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 162.26647801799999, 2020-08-07T15:11:36.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0071655319999999, 2020-08-07T15:11:36.839 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:36.840 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:11:36.840 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:36.840 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:36.840 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:36.840 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37059 2020-08-07T15:11:36.841 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.841 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:36.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:36.841 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:11:36.844 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:36.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.844 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:36.845 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:11:36.845 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6533000000000002e-05 2020-08-07T15:11:36.845 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:36.846 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244012+0000", 2020-08-07T15:11:36.846 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3860000000000001e-06 2020-08-07T15:11:36.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.847 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.847 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:36.847 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244019+0000", 2020-08-07T15:11:36.847 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6982000000000001e-05 2020-08-07T15:11:36.847 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.848 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.848 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:36.848 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244036+0000", 2020-08-07T15:11:36.848 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0310000000000001e-06 2020-08-07T15:11:36.848 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.849 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.849 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:36.849 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244043+0000", 2020-08-07T15:11:36.849 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0067169259999999998 2020-08-07T15:11:36.849 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.850 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:36.851 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250760+0000", 2020-08-07T15:11:36.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4499999999999998e-06 2020-08-07T15:11:36.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:36.853 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250768+0000", 2020-08-07T15:11:36.853 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000341691 2020-08-07T15:11:36.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.853 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:36.854 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251109+0000", 2020-08-07T15:11:36.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1533e-05 2020-08-07T15:11:36.854 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.854 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.855 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:36.855 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251121+0000", 2020-08-07T15:11:36.855 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0071655319999999 2020-08-07T15:11:36.855 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:36.855 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:36.856 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:36.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.856 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.856 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37060 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:11:36.856 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:11:36.857 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 162.266354081, 2020-08-07T15:11:36.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070762280000001, 2020-08-07T15:11:36.857 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:36.857 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:11:36.857 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:36.858 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:36.859 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:36.859 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37060 2020-08-07T15:11:36.860 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.860 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:36.860 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.861 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:36.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:11:36.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:36.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:36.887 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:11:36.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5739000000000003e-05 2020-08-07T15:11:36.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:36.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244135+0000", 2020-08-07T15:11:36.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7740000000000001e-06 2020-08-07T15:11:36.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:36.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244141+0000", 2020-08-07T15:11:36.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3817e-05 2020-08-07T15:11:36.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:36.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244155+0000", 2020-08-07T15:11:36.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3829999999999997e-06 2020-08-07T15:11:36.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:36.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244161+0000", 2020-08-07T15:11:36.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0066303669999999999 2020-08-07T15:11:36.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:36.918 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250791+0000", 2020-08-07T15:11:36.918 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.392e-06 2020-08-07T15:11:36.918 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.918 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:36.919 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250798+0000", 2020-08-07T15:11:36.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003485860000001 2020-08-07T15:11:36.919 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.919 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:36.920 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251146+0000", 2020-08-07T15:11:36.920 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1700000000000003e-06 2020-08-07T15:11:36.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:36.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251155+0000", 2020-08-07T15:11:36.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070762280000001 2020-08-07T15:11:36.921 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:36.921 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:36.921 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:36.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.941 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37061 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:11:36.942 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:11:36.942 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 162.26623945399999, 2020-08-07T15:11:36.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070006920000001, 2020-08-07T15:11:36.942 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:36.942 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:11:36.962 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:36.963 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:36.963 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:36.963 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37061 2020-08-07T15:11:36.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.964 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:36.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:36.964 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:11:36.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:36.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:36.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:11:36.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5453e-05 2020-08-07T15:11:36.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.975 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:36.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244239+0000", 2020-08-07T15:11:36.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3179999999999997e-06 2020-08-07T15:11:36.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:36.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244245+0000", 2020-08-07T15:11:36.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2138e-05 2020-08-07T15:11:36.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.982 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:36.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244257+0000", 2020-08-07T15:11:36.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.69e-06 2020-08-07T15:11:36.987 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:36.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244261+0000", 2020-08-07T15:11:36.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0065598940000000001 2020-08-07T15:11:36.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:36.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250821+0000", 2020-08-07T15:11:36.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4799999999999998e-06 2020-08-07T15:11:36.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:36.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250828+0000", 2020-08-07T15:11:36.993 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003567310000001 2020-08-07T15:11:36.993 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.993 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.993 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:36.993 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251185+0000", 2020-08-07T15:11:36.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9879999999999999e-06 2020-08-07T15:11:36.994 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.995 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.995 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:36.995 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251195+0000", 2020-08-07T15:11:36.995 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070006920000001 2020-08-07T15:11:36.995 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:36.996 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:36.996 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:36.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:36.996 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:36.996 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37062 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:11:36.997 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:11:36.997 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 162.26613923599999, 2020-08-07T15:11:36.998 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006933662, 2020-08-07T15:11:36.998 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:36.998 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:11:36.998 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:36.998 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:36.999 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:36.999 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37062 2020-08-07T15:11:37.000 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.000 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:37.000 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.000 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:37.000 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:11:37.001 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:37.001 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.001 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.001 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:37.001 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:11:37.002 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4041e-05 2020-08-07T15:11:37.002 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.002 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.002 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:37.002 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244348+0000", 2020-08-07T15:11:37.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2079999999999997e-06 2020-08-07T15:11:37.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:37.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244354+0000", 2020-08-07T15:11:37.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1483e-05 2020-08-07T15:11:37.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.011 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:37.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244366+0000", 2020-08-07T15:11:37.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7989999999999999e-06 2020-08-07T15:11:37.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:37.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244372+0000", 2020-08-07T15:11:37.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0064783699999999998 2020-08-07T15:11:37.012 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.012 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.013 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:37.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250850+0000", 2020-08-07T15:11:37.013 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4729999999999997e-06 2020-08-07T15:11:37.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:37.014 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250856+0000", 2020-08-07T15:11:37.014 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003630240000001 2020-08-07T15:11:37.014 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.014 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.015 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:37.015 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251219+0000", 2020-08-07T15:11:37.015 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2640000000000008e-06 2020-08-07T15:11:37.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.015 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.015 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:37.016 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251228+0000", 2020-08-07T15:11:37.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006933662 2020-08-07T15:11:37.016 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.016 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:37.017 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.017 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.018 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37063 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:11:37.018 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:11:37.018 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 162.26602690199999, 2020-08-07T15:11:37.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006858051, 2020-08-07T15:11:37.018 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:37.019 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:11:37.019 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:37.020 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:37.021 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:37.021 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37063 2020-08-07T15:11:37.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.021 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:37.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:37.022 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:11:37.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:37.023 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.023 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:37.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:11:37.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5043000000000001e-05 2020-08-07T15:11:37.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:37.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244461+0000", 2020-08-07T15:11:37.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4600000000000002e-06 2020-08-07T15:11:37.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:37.026 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244467+0000", 2020-08-07T15:11:37.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1435e-05 2020-08-07T15:11:37.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:37.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244478+0000", 2020-08-07T15:11:37.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0190000000000003e-06 2020-08-07T15:11:37.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:37.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244484+0000", 2020-08-07T15:11:37.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006392895 2020-08-07T15:11:37.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:37.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250877+0000", 2020-08-07T15:11:37.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4280000000000001e-06 2020-08-07T15:11:37.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.030 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:37.033 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250884+0000", 2020-08-07T15:11:37.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000370414 2020-08-07T15:11:37.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.034 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.034 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:37.034 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251254+0000", 2020-08-07T15:11:37.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0356999999999999e-05 2020-08-07T15:11:37.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:37.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251264+0000", 2020-08-07T15:11:37.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006858051 2020-08-07T15:11:37.036 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.036 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:37.036 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.037 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37064 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:11:37.037 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:11:37.037 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 162.26591476900001, 2020-08-07T15:11:37.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0067798189999999, 2020-08-07T15:11:37.037 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:37.038 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:11:37.039 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:37.054 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:37.054 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:37.054 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37064 2020-08-07T15:11:37.055 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.055 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:37.055 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:37.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:11:37.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:37.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:37.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:11:37.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4007e-05 2020-08-07T15:11:37.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:37.057 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244573+0000", 2020-08-07T15:11:37.069 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.831e-06 2020-08-07T15:11:37.076 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.076 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.076 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:37.076 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244577+0000", 2020-08-07T15:11:37.076 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2269000000000001e-05 2020-08-07T15:11:37.077 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.077 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.077 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:37.077 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244590+0000", 2020-08-07T15:11:37.077 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7149999999999997e-06 2020-08-07T15:11:37.078 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.078 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.078 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:37.078 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244594+0000", 2020-08-07T15:11:37.078 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063105670000000004 2020-08-07T15:11:37.079 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.079 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.079 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:37.079 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250905+0000", 2020-08-07T15:11:37.079 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1970000000000001e-06 2020-08-07T15:11:37.080 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.080 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.080 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:37.080 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250911+0000", 2020-08-07T15:11:37.080 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003783020000001 2020-08-07T15:11:37.081 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.081 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.081 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:37.081 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251289+0000", 2020-08-07T15:11:37.081 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9309999999999997e-06 2020-08-07T15:11:37.082 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.083 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:37.083 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251298+0000", 2020-08-07T15:11:37.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0067798189999999 2020-08-07T15:11:37.083 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.094 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:37.095 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.105 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.108 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.109 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37065 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:11:37.109 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:11:37.109 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 162.26580455000001, 2020-08-07T15:11:37.109 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006708545, 2020-08-07T15:11:37.111 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:37.111 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:11:37.111 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:37.111 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:37.112 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:37.112 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37065 2020-08-07T15:11:37.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.113 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:37.113 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:37.113 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:11:37.114 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:37.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.114 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:37.114 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:11:37.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2750000000000001e-05 2020-08-07T15:11:37.115 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.116 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.116 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:37.117 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244681+0000", 2020-08-07T15:11:37.117 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0950000000000003e-06 2020-08-07T15:11:37.117 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.117 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:37.118 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244687+0000", 2020-08-07T15:11:37.118 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2485e-05 2020-08-07T15:11:37.118 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.118 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.118 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:37.119 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244699+0000", 2020-08-07T15:11:37.119 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8800000000000002e-06 2020-08-07T15:11:37.119 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.119 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.119 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:37.120 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244706+0000", 2020-08-07T15:11:37.120 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0062281569999999998 2020-08-07T15:11:37.120 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.120 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.120 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:37.121 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250934+0000", 2020-08-07T15:11:37.121 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1719999999999998e-06 2020-08-07T15:11:37.121 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.121 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.121 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:37.122 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250939+0000", 2020-08-07T15:11:37.122 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003861810000001 2020-08-07T15:11:37.122 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.122 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.124 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:37.124 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251325+0000", 2020-08-07T15:11:37.124 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1824999999999999e-05 2020-08-07T15:11:37.124 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.124 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.125 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:37.125 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251337+0000", 2020-08-07T15:11:37.125 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006708545 2020-08-07T15:11:37.125 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.126 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:37.126 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.127 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.127 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.128 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37066 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:11:37.128 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:11:37.128 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 162.26113558, 2020-08-07T15:11:37.128 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002077205, 2020-08-07T15:11:37.128 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:37.129 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:11:37.129 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:37.129 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:37.129 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:37.129 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37066 2020-08-07T15:11:37.130 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.130 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:37.130 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.130 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:37.130 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:11:37.131 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:37.131 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.131 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.131 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:37.131 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:11:37.131 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4654000000000002e-05 2020-08-07T15:11:37.132 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.132 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.132 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:37.132 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249352+0000", 2020-08-07T15:11:37.132 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6210000000000004e-06 2020-08-07T15:11:37.133 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.133 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.133 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:37.133 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249360+0000", 2020-08-07T15:11:37.133 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1895999999999998e-05 2020-08-07T15:11:37.136 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.136 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.137 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:37.137 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249382+0000", 2020-08-07T15:11:37.138 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1479999999999995e-06 2020-08-07T15:11:37.138 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.138 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.138 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:37.139 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249391+0000", 2020-08-07T15:11:37.139 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015731510000000001 2020-08-07T15:11:37.139 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.139 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.139 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:37.140 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250964+0000", 2020-08-07T15:11:37.140 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5400000000000001e-06 2020-08-07T15:11:37.140 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.140 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.140 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:37.141 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250971+0000", 2020-08-07T15:11:37.141 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000392379 2020-08-07T15:11:37.141 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.141 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.142 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:37.142 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251363+0000", 2020-08-07T15:11:37.142 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1816e-05 2020-08-07T15:11:37.142 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.142 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.143 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:37.143 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251375+0000", 2020-08-07T15:11:37.143 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002077205 2020-08-07T15:11:37.143 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.143 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:37.144 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.144 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.144 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.145 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37067 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:11:37.145 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:11:37.145 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 162.26099488599999, 2020-08-07T15:11:37.145 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0019761119999999, 2020-08-07T15:11:37.159 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:37.159 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:11:37.164 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:37.164 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:37.164 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:37.164 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37067 2020-08-07T15:11:37.165 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.165 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:37.165 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.165 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:37.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:11:37.166 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:37.166 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.166 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.168 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:37.168 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:11:37.168 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5961999999999997e-05 2020-08-07T15:11:37.168 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.169 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.169 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:37.169 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249494+0000", 2020-08-07T15:11:37.178 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3210000000000004e-06 2020-08-07T15:11:37.178 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.181 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.185 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:37.185 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249500+0000", 2020-08-07T15:11:37.186 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1012000000000001e-05 2020-08-07T15:11:37.186 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.186 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.186 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:37.186 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249511+0000", 2020-08-07T15:11:37.187 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.809e-06 2020-08-07T15:11:37.187 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.187 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.194 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:37.194 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249515+0000", 2020-08-07T15:11:37.194 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0014777670000000001 2020-08-07T15:11:37.194 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.207 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.207 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:37.208 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250993+0000", 2020-08-07T15:11:37.208 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9499999999999998e-06 2020-08-07T15:11:37.216 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.216 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.217 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:37.217 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250999+0000", 2020-08-07T15:11:37.220 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0004032519999999 2020-08-07T15:11:37.221 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.222 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.222 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:37.222 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251402+0000", 2020-08-07T15:11:37.223 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2038999999999999e-05 2020-08-07T15:11:37.223 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.223 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.223 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:37.224 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251414+0000", 2020-08-07T15:11:37.224 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0019761119999999 2020-08-07T15:11:37.224 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.224 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:37.238 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.238 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.238 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.238 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37068 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:11:37.239 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:11:37.239 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 162.25875031499999, 2020-08-07T15:11:37.239 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99977072, 2020-08-07T15:11:37.239 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:37.240 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:11:37.240 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:37.252 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:37.252 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:37.252 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37068 2020-08-07T15:11:37.253 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.253 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:37.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.254 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:37.254 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:11:37.254 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:37.254 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.255 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.255 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:37.255 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:11:37.255 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1369000000000003e-05 2020-08-07T15:11:37.256 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.256 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.256 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:37.256 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251724+0000", 2020-08-07T15:11:37.257 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.19e-06 2020-08-07T15:11:37.257 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.257 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.257 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:37.257 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251731+0000", 2020-08-07T15:11:37.258 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1368e-05 2020-08-07T15:11:37.258 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.258 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.258 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:37.259 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251742+0000", 2020-08-07T15:11:37.259 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.682e-06 2020-08-07T15:11:37.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.259 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.259 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:37.260 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251748+0000", 2020-08-07T15:11:37.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5275000000000003e-05 2020-08-07T15:11:37.260 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.260 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.261 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:37.261 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251793+0000", 2020-08-07T15:11:37.261 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5000000000000005e-06 2020-08-07T15:11:37.261 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.291 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.291 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:37.291 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251802+0000", 2020-08-07T15:11:37.294 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99963953500000002 2020-08-07T15:11:37.295 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.295 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.296 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:37.296 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251442+0000", 2020-08-07T15:11:37.296 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1800999999999999e-05 2020-08-07T15:11:37.297 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.309 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.310 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:37.310 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251454+0000", 2020-08-07T15:11:37.310 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99977072 2020-08-07T15:11:37.311 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.311 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:37.311 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.311 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.311 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.312 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37069 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:11:37.312 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:11:37.312 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 162.25764651700001, 2020-08-07T15:11:37.312 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99870607899999997, 2020-08-07T15:11:37.312 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:37.313 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:11:37.313 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:37.313 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:37.313 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:37.314 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37069 2020-08-07T15:11:37.314 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.314 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:37.314 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.314 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:37.315 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:11:37.315 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:37.315 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.315 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.316 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:37.316 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:11:37.316 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3755999999999999e-05 2020-08-07T15:11:37.316 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.316 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.317 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:37.317 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252831+0000", 2020-08-07T15:11:37.317 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7919999999999998e-06 2020-08-07T15:11:37.317 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.331 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.332 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:37.344 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252836+0000", 2020-08-07T15:11:37.344 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1409999999999999e-05 2020-08-07T15:11:37.345 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.345 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.345 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:37.345 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252848+0000", 2020-08-07T15:11:37.345 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6380000000000001e-06 2020-08-07T15:11:37.346 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.346 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.346 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:37.346 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252853+0000", 2020-08-07T15:11:37.347 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3244e-05 2020-08-07T15:11:37.347 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.347 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:37.347 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252907+0000", 2020-08-07T15:11:37.348 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1470000000000007e-06 2020-08-07T15:11:37.348 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.348 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.348 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:37.348 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252916+0000", 2020-08-07T15:11:37.349 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99856497499999997 2020-08-07T15:11:37.349 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.349 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.363 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:37.364 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251481+0000", 2020-08-07T15:11:37.364 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2116999999999999e-05 2020-08-07T15:11:37.364 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:37.365 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251493+0000", 2020-08-07T15:11:37.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99870607899999997 2020-08-07T15:11:37.365 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.365 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:37.365 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.366 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.366 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37070 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:11:37.366 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:11:37.367 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 162.25754225399999, 2020-08-07T15:11:37.367 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99863946699999995, 2020-08-07T15:11:37.367 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:37.367 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:11:37.367 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:37.368 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:37.368 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:37.368 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37070 2020-08-07T15:11:37.368 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.368 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:37.374 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.374 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:37.375 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:11:37.375 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:37.375 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.375 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:37.376 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:11:37.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9984000000000001e-05 2020-08-07T15:11:37.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.376 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:37.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252941+0000", 2020-08-07T15:11:37.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7010000000000004e-06 2020-08-07T15:11:37.377 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:37.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252946+0000", 2020-08-07T15:11:37.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1144999999999999e-05 2020-08-07T15:11:37.392 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.392 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.392 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:37.392 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252957+0000", 2020-08-07T15:11:37.393 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6920000000000001e-06 2020-08-07T15:11:37.393 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.422 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:37.422 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252962+0000", 2020-08-07T15:11:37.422 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6739000000000002e-05 2020-08-07T15:11:37.423 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.423 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.423 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:37.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252998+0000", 2020-08-07T15:11:37.423 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.503e-06 2020-08-07T15:11:37.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.424 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.424 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:37.424 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.253006+0000", 2020-08-07T15:11:37.424 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99851375499999995 2020-08-07T15:11:37.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.425 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:37.425 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251520+0000", 2020-08-07T15:11:37.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0947999999999999e-05 2020-08-07T15:11:37.426 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.426 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.426 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:37.426 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251530+0000", 2020-08-07T15:11:37.426 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99863946699999995 2020-08-07T15:11:37.427 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.427 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:37.427 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.427 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.427 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.428 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42505 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:11:37.428 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:11:37.428 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 156.20820172399999, 2020-08-07T15:11:37.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0024736009999999, 2020-08-07T15:11:37.428 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:37.429 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:37.429 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:37.429 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:37.429 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:37.429 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42505 2020-08-07T15:11:37.430 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.431 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:37.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.431 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:37.431 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:11:37.437 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:37.438 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.438 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.438 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:37.438 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:11:37.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011052600000000001 2020-08-07T15:11:37.439 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.439 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.439 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:37.439 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302342+0000", 2020-08-07T15:11:37.440 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.781e-06 2020-08-07T15:11:37.440 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.440 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:37.440 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302352+0000", 2020-08-07T15:11:37.440 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7646e-05 2020-08-07T15:11:37.441 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.441 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.441 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:37.441 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302369+0000", 2020-08-07T15:11:37.441 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3629999999999994e-06 2020-08-07T15:11:37.442 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.442 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.443 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:37.443 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302378+0000", 2020-08-07T15:11:37.443 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8007000000000003e-05 2020-08-07T15:11:37.443 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.444 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:37.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302426+0000", 2020-08-07T15:11:37.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5159999999999994e-06 2020-08-07T15:11:37.445 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.445 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.445 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:37.445 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302434+0000", 2020-08-07T15:11:37.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99151562699999995 2020-08-07T15:11:37.446 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.446 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.446 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:37.446 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.293950+0000", 2020-08-07T15:11:37.446 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020138099999999999 2020-08-07T15:11:37.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.447 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:37.447 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294151+0000", 2020-08-07T15:11:37.447 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024761 2020-08-07T15:11:37.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:11:37.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294399+0000", 2020-08-07T15:11:37.448 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0027714369999999999 2020-08-07T15:11:37.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.449 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.451 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:37.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297170+0000", 2020-08-07T15:11:37.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.234e-06 2020-08-07T15:11:37.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:37.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297172+0000", 2020-08-07T15:11:37.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5384999999999998e-05 2020-08-07T15:11:37.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.453 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:37.453 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297237+0000", 2020-08-07T15:11:37.454 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0073576980000000002 2020-08-07T15:11:37.454 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.454 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.454 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:37.454 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304595+0000", 2020-08-07T15:11:37.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1006000000000001e-05 2020-08-07T15:11:37.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:37.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304626+0000", 2020-08-07T15:11:37.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6000000000000003e-07 2020-08-07T15:11:37.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:37.456 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304626+0000", 2020-08-07T15:11:37.456 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8724e-05 2020-08-07T15:11:37.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:37.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304705+0000", 2020-08-07T15:11:37.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0024736009999999 2020-08-07T15:11:37.459 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.460 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:37.460 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.461 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.461 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.461 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42506 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:11:37.461 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:11:37.461 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 156.208010511, 2020-08-07T15:11:37.462 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002465226, 2020-08-07T15:11:37.462 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:37.462 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:37.462 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:37.462 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:37.463 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:37.463 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42506 2020-08-07T15:11:37.463 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.463 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:37.463 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.464 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:37.464 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:11:37.464 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:37.464 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.464 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.465 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:37.465 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:11:37.465 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8780999999999999e-05 2020-08-07T15:11:37.465 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.465 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.481 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:37.482 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302472+0000", 2020-08-07T15:11:37.482 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3740000000000003e-06 2020-08-07T15:11:37.483 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.491 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.491 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:37.491 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302478+0000", 2020-08-07T15:11:37.491 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1155000000000001e-05 2020-08-07T15:11:37.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.492 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.492 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:37.492 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302489+0000", 2020-08-07T15:11:37.492 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7559999999999996e-06 2020-08-07T15:11:37.492 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.493 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:37.493 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302495+0000", 2020-08-07T15:11:37.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4128000000000003e-05 2020-08-07T15:11:37.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.494 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:37.495 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302529+0000", 2020-08-07T15:11:37.496 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.516e-06 2020-08-07T15:11:37.496 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.497 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.497 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:37.497 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302535+0000", 2020-08-07T15:11:37.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99240899299999996 2020-08-07T15:11:37.498 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.498 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.498 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:37.498 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294944+0000", 2020-08-07T15:11:37.498 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000115806 2020-08-07T15:11:37.499 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.499 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.499 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:37.499 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295060+0000", 2020-08-07T15:11:37.500 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000205002 2020-08-07T15:11:37.500 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.500 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.500 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:11:37.501 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295265+0000", 2020-08-07T15:11:37.501 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043414400000000002 2020-08-07T15:11:37.501 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.501 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.501 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:37.502 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299607+0000", 2020-08-07T15:11:37.502 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2229999999999998e-06 2020-08-07T15:11:37.502 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:37.511 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299612+0000", 2020-08-07T15:11:37.511 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5797000000000002e-05 2020-08-07T15:11:37.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.512 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.512 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:37.512 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299658+0000", 2020-08-07T15:11:37.512 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005095044 2020-08-07T15:11:37.513 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.513 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.513 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:37.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304753+0000", 2020-08-07T15:11:37.513 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3121999999999996e-05 2020-08-07T15:11:37.514 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.516 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:37.516 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304816+0000", 2020-08-07T15:11:37.516 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5700000000000001e-07 2020-08-07T15:11:37.516 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.517 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.517 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:37.517 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304817+0000", 2020-08-07T15:11:37.517 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1231999999999998e-05 2020-08-07T15:11:37.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.519 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:37.519 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304888+0000", 2020-08-07T15:11:37.519 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002465226 2020-08-07T15:11:37.519 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.519 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:37.520 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.520 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.520 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.520 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42507 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:11:37.520 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:11:37.521 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 156.20789974100001, 2020-08-07T15:11:37.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002567097, 2020-08-07T15:11:37.521 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:37.521 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:37.521 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:37.522 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:37.522 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:37.522 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42507 2020-08-07T15:11:37.523 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.523 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:37.523 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.523 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:37.523 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:11:37.524 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:37.524 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.524 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.524 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:37.524 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:11:37.525 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7679000000000002e-05 2020-08-07T15:11:37.525 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.525 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.525 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:37.525 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302581+0000", 2020-08-07T15:11:37.526 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2299999999999999e-06 2020-08-07T15:11:37.526 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.526 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.527 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:37.527 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302586+0000", 2020-08-07T15:11:37.527 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0886999999999999e-05 2020-08-07T15:11:37.527 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.527 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.528 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:37.528 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302597+0000", 2020-08-07T15:11:37.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4129999999999997e-06 2020-08-07T15:11:37.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.536 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:37.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302603+0000", 2020-08-07T15:11:37.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3963000000000001e-05 2020-08-07T15:11:37.537 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.537 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.537 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:37.538 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302627+0000", 2020-08-07T15:11:37.538 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.33e-06 2020-08-07T15:11:37.538 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.538 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.538 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:37.539 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302631+0000", 2020-08-07T15:11:37.539 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99319860900000001 2020-08-07T15:11:37.539 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.539 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.539 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:37.540 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295830+0000", 2020-08-07T15:11:37.540 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000116012 2020-08-07T15:11:37.540 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.541 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:37.541 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295946+0000", 2020-08-07T15:11:37.541 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000230242 2020-08-07T15:11:37.542 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.542 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.542 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:11:37.542 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296176+0000", 2020-08-07T15:11:37.542 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003494455 2020-08-07T15:11:37.543 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.543 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.543 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:37.543 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299670+0000", 2020-08-07T15:11:37.543 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0299999999999998e-07 2020-08-07T15:11:37.543 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.544 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.544 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:37.544 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299671+0000", 2020-08-07T15:11:37.544 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7298000000000001e-05 2020-08-07T15:11:37.544 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.545 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:37.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299708+0000", 2020-08-07T15:11:37.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0052628839999999998 2020-08-07T15:11:37.545 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.546 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.546 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:37.546 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304971+0000", 2020-08-07T15:11:37.547 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3754e-05 2020-08-07T15:11:37.547 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.547 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:37.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305035+0000", 2020-08-07T15:11:37.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9500000000000001e-07 2020-08-07T15:11:37.548 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.548 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:37.548 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305036+0000", 2020-08-07T15:11:37.549 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4843000000000002e-05 2020-08-07T15:11:37.549 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.549 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.549 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:37.549 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305101+0000", 2020-08-07T15:11:37.550 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002567097 2020-08-07T15:11:37.551 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.551 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:37.553 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.553 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.553 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.553 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42508 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:11:37.554 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:11:37.554 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 156.19737472200001, 2020-08-07T15:11:37.554 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99221695600000004, 2020-08-07T15:11:37.554 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:37.554 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:37.555 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:37.555 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:11:37.555 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:37.556 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42508 2020-08-07T15:11:37.556 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.557 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:37.558 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.562 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:37.562 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:11:37.562 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:37.562 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.563 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.563 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:37.563 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:11:37.563 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3692e-05 2020-08-07T15:11:37.563 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.564 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:37.564 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313112+0000", 2020-08-07T15:11:37.564 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7399999999999998e-06 2020-08-07T15:11:37.564 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.565 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.565 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:37.565 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313119+0000", 2020-08-07T15:11:37.565 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5153999999999999e-05 2020-08-07T15:11:37.566 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.566 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.566 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:37.566 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313134+0000", 2020-08-07T15:11:37.567 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3039999999999999e-06 2020-08-07T15:11:37.567 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.567 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.567 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:37.567 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313141+0000", 2020-08-07T15:11:37.568 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6516000000000001e-05 2020-08-07T15:11:37.568 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.568 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.568 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:37.568 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313178+0000", 2020-08-07T15:11:37.569 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0949999999999998e-06 2020-08-07T15:11:37.569 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.569 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.569 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:37.569 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313185+0000", 2020-08-07T15:11:37.570 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.98342834999999995 2020-08-07T15:11:37.570 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.570 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.570 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:37.570 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296613+0000", 2020-08-07T15:11:37.571 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2230999999999993e-05 2020-08-07T15:11:37.571 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.571 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.571 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:37.571 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296686+0000", 2020-08-07T15:11:37.572 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013121300000000001 2020-08-07T15:11:37.572 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.572 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.572 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:11:37.572 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296817+0000", 2020-08-07T15:11:37.573 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044228050000000001 2020-08-07T15:11:37.573 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.573 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.573 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:37.573 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301240+0000", 2020-08-07T15:11:37.574 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0500000000000002e-07 2020-08-07T15:11:37.574 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.574 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.575 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:37.584 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301241+0000", 2020-08-07T15:11:37.584 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2434000000000006e-05 2020-08-07T15:11:37.585 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.590 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:37.590 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301303+0000", 2020-08-07T15:11:37.590 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038622919999999998 2020-08-07T15:11:37.590 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.591 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.591 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:37.591 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305165+0000", 2020-08-07T15:11:37.591 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8012000000000001e-05 2020-08-07T15:11:37.591 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.592 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.592 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:37.592 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305203+0000", 2020-08-07T15:11:37.592 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5299999999999999e-07 2020-08-07T15:11:37.592 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.593 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.593 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:37.593 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305204+0000", 2020-08-07T15:11:37.593 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1760000000000004e-05 2020-08-07T15:11:37.593 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:37.594 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:37.594 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:37.594 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305275+0000", 2020-08-07T15:11:37.595 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99221695600000004 2020-08-07T15:11:37.596 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.596 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:37.596 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.596 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:37.596 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:37.597 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:11:37.600 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:11:37.601 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:11:37.601 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 77 16 110340 110324 11.1925 12.5156 0.0112424 0.011164 2020-08-07T15:11:37.602 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:37.596+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:37.750 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:37.749+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:37.762 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3, 4, 5, 6, 7] 2020-08-07T15:11:37.762 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:11:37.763 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump -f json-pretty 2020-08-07T15:11:37.854 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:37.852+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:37.872 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:11:37.873 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:11:37.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:37.873 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:111368 5.c 5:3e1e8367:::benchmark_data_smithi074_53697_object13920:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e215)", 2020-08-07T15:11:37.873 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:37.865606+0000", 2020-08-07T15:11:37.874 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0063963980000000002, 2020-08-07T15:11:37.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064176090000000003, 2020-08-07T15:11:37.874 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:37.874 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:11:37.874 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:37.875 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:11:37.875 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:37.875 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 111368 2020-08-07T15:11:37.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:37.875 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:37.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:37.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:37.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:37.865606+0000", 2020-08-07T15:11:37.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:37.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:37.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:37.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:37.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:37.865606+0000", 2020-08-07T15:11:37.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1619000000000003e-05 2020-08-07T15:11:37.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:37.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:37.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:37.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:37.865648+0000", 2020-08-07T15:11:37.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8909999999999997e-06 2020-08-07T15:11:37.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:37.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:37.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:37.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:37.865652+0000", 2020-08-07T15:11:37.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3519e-05 2020-08-07T15:11:37.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:37.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:37.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:37.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:37.865665+0000", 2020-08-07T15:11:37.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1609999999999996e-06 2020-08-07T15:11:37.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:37.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:37.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:37.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:37.865669+0000", 2020-08-07T15:11:37.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022867410000000001 2020-08-07T15:11:37.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:37.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:37.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:37.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:37.867956+0000", 2020-08-07T15:11:37.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8615000000000006e-05 2020-08-07T15:11:37.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:37.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:37.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:37.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:37.868025+0000", 2020-08-07T15:11:37.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011741 2020-08-07T15:11:37.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:37.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:37.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2020-08-07T15:11:37.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:37.868142+0000", 2020-08-07T15:11:37.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012405389999999999 2020-08-07T15:11:37.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:37.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:37.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:37.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:37.869383+0000", 2020-08-07T15:11:37.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037764949999999999 2020-08-07T15:11:37.941 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:37.941 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:37.958 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:37.959 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:37.959 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:11:37.959 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 1 2020-08-07T15:11:37.959 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:11:37.961 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:11:37.970 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:37.959+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:38.066 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:38.061+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:38.170 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:38.162+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:38.187 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:11:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:11:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:11:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:11:38.188 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:11:38.189 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:11:38.223 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 78 16 111917 111901 11.207 12.3203 0.00536707 0.011149 2020-08-07T15:11:38.271 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:38.264+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:38.371 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:38.369+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:38.374 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:11:38.394 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:11:38.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:11:38.394 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:11:38.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.394 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "MOSDPGPushReply(3.1 62/60 [PushReplyOp(3:8329f37e:::benchmark_data_smithi074_35447_object11915:head)])", 2020-08-07T15:11:38.395 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:11:38.395 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 227.51185187900001, 2020-08-07T15:11:38.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996, 2020-08-07T15:11:38.395 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:38.396 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:38.396 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:38.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:38.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:11:38.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:38.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:38.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:11:38.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5372000000000001e-05 2020-08-07T15:11:38.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:38.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858416+0000", 2020-08-07T15:11:38.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6899999999999998e-06 2020-08-07T15:11:38.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:38.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858419+0000", 2020-08-07T15:11:38.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63907904299999996 2020-08-07T15:11:38.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:38.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497498+0000", 2020-08-07T15:11:38.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3332999999999999e-05 2020-08-07T15:11:38.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:38.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497512+0000", 2020-08-07T15:11:38.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000450875 2020-08-07T15:11:38.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:38.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497962+0000", 2020-08-07T15:11:38.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000267779 2020-08-07T15:11:38.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:38.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.498230+0000", 2020-08-07T15:11:38.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996 2020-08-07T15:11:38.441 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:38.441 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:38.441 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:38.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.442 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:145792 3.1 3:881e0420:::benchmark_data_smithi074_35447_object16183:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:11:38.442 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:11:38.442 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 203.70344599800001, 2020-08-07T15:11:38.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997, 2020-08-07T15:11:38.442 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:38.443 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:38.450 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:38.450 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:11:38.451 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:11:38.451 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 145792 2020-08-07T15:11:38.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.466 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:38.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:38.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:11:38.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:38.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:38.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:11:38.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7489000000000001e-05 2020-08-07T15:11:38.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:38.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666824+0000", 2020-08-07T15:11:38.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0169999999999998e-06 2020-08-07T15:11:38.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:38.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666828+0000", 2020-08-07T15:11:38.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9829999999999996e-06 2020-08-07T15:11:38.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:38.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666836+0000", 2020-08-07T15:11:38.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.481e-06 2020-08-07T15:11:38.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:38.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666840+0000", 2020-08-07T15:11:38.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5299000000000001e-05 2020-08-07T15:11:38.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:38.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666885+0000", 2020-08-07T15:11:38.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014456500000000001 2020-08-07T15:11:38.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:38.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667030+0000", 2020-08-07T15:11:38.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017677799999999999 2020-08-07T15:11:38.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,5", 2020-08-07T15:11:38.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667207+0000", 2020-08-07T15:11:38.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004149649 2020-08-07T15:11:38.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:38.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671356+0000", 2020-08-07T15:11:38.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019339649999999999 2020-08-07T15:11:38.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:38.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673290+0000", 2020-08-07T15:11:38.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73346620500000004 2020-08-07T15:11:38.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:38.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406756+0000", 2020-08-07T15:11:38.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4700000000000001e-06 2020-08-07T15:11:38.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:38.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406759+0000", 2020-08-07T15:11:38.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6258999999999998e-05 2020-08-07T15:11:38.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:38.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406815+0000", 2020-08-07T15:11:38.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0126000000000002e-05 2020-08-07T15:11:38.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:38.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406855+0000", 2020-08-07T15:11:38.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997 2020-08-07T15:11:38.524 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:38.524 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:38.524 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:38.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.525 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:145792 3.1 e75/67 ondisk, result = 0)", 2020-08-07T15:11:38.525 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:11:38.543 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 203.69906111, 2020-08-07T15:11:38.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003, 2020-08-07T15:11:38.544 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:38.544 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:11:38.544 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:38.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:38.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:11:38.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:38.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:38.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:11:38.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2830999999999998e-05 2020-08-07T15:11:38.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:38.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671224+0000", 2020-08-07T15:11:38.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:11:38.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:38.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671229+0000", 2020-08-07T15:11:38.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.735414864 2020-08-07T15:11:38.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:38.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406644+0000", 2020-08-07T15:11:38.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1250000000000004e-06 2020-08-07T15:11:38.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:38.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406651+0000", 2020-08-07T15:11:38.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6567e-05 2020-08-07T15:11:38.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:38.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406737+0000", 2020-08-07T15:11:38.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0054000000000001e-05 2020-08-07T15:11:38.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:38.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406747+0000", 2020-08-07T15:11:38.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011391999999999999 2020-08-07T15:11:38.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:38.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406861+0000", 2020-08-07T15:11:38.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003 2020-08-07T15:11:38.617 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:38.623 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:38.623 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:38.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.624 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30921 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:11:38.625 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:11:38.625 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 171.664312736, 2020-08-07T15:11:38.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001, 2020-08-07T15:11:38.625 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:38.625 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:38.658 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:38.658 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:38.658 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:38.659 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30921 2020-08-07T15:11:38.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.659 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:38.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:38.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:11:38.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:38.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:38.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:11:38.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6954999999999998e-05 2020-08-07T15:11:38.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:38.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706016+0000", 2020-08-07T15:11:38.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0229999999999992e-06 2020-08-07T15:11:38.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:38.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706024+0000", 2020-08-07T15:11:38.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.432e-05 2020-08-07T15:11:38.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:38.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706039+0000", 2020-08-07T15:11:38.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8490000000000006e-06 2020-08-07T15:11:38.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:38.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706046+0000", 2020-08-07T15:11:38.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6837999999999999e-05 2020-08-07T15:11:38.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:38.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706113+0000", 2020-08-07T15:11:38.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3553000000000001e-05 2020-08-07T15:11:38.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:38.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706127+0000", 2020-08-07T15:11:38.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77277201699999998 2020-08-07T15:11:38.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:38.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.478899+0000", 2020-08-07T15:11:38.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093978599999999996 2020-08-07T15:11:38.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:11:38.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479839+0000", 2020-08-07T15:11:38.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010376029 2020-08-07T15:11:38.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:38.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490215+0000", 2020-08-07T15:11:38.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000190657 2020-08-07T15:11:38.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:38.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490405+0000", 2020-08-07T15:11:38.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001 2020-08-07T15:11:38.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:38.730 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:38.731 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:38.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.731 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30922 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:11:38.731 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:11:38.732 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 171.664152573, 2020-08-07T15:11:38.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247, 2020-08-07T15:11:38.745 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:38.746 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:38.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:38.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:38.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:38.747 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30922 2020-08-07T15:11:38.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.747 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:38.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:38.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:11:38.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:38.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:38.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:11:38.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9178999999999997e-05 2020-08-07T15:11:38.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:38.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706159+0000", 2020-08-07T15:11:38.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4999999999999999e-06 2020-08-07T15:11:38.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:38.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706164+0000", 2020-08-07T15:11:38.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1792e-05 2020-08-07T15:11:38.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:38.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706176+0000", 2020-08-07T15:11:38.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.139e-06 2020-08-07T15:11:38.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:38.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706182+0000", 2020-08-07T15:11:38.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039724900000000002 2020-08-07T15:11:38.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:38.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706579+0000", 2020-08-07T15:11:38.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1191e-05 2020-08-07T15:11:38.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:38.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706591+0000", 2020-08-07T15:11:38.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329101300000003 2020-08-07T15:11:38.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:38.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479882+0000", 2020-08-07T15:11:38.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9565000000000001e-05 2020-08-07T15:11:38.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:11:38.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479971+0000", 2020-08-07T15:11:38.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010471263 2020-08-07T15:11:38.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:38.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490442+0000", 2020-08-07T15:11:38.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015035600000000001 2020-08-07T15:11:38.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:38.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490593+0000", 2020-08-07T15:11:38.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247 2020-08-07T15:11:38.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:38.764 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:38.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:38.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.765 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30923 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:11:38.765 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:11:38.766 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 171.66367751199999, 2020-08-07T15:11:38.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998, 2020-08-07T15:11:38.766 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:38.766 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:38.766 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:38.767 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:38.767 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:38.767 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30923 2020-08-07T15:11:38.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.767 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:38.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:38.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:11:38.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:38.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:38.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:11:38.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5531000000000001e-05 2020-08-07T15:11:38.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:38.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706630+0000", 2020-08-07T15:11:38.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7689999999999999e-06 2020-08-07T15:11:38.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:38.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706637+0000", 2020-08-07T15:11:38.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3577999999999999e-05 2020-08-07T15:11:38.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:38.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706650+0000", 2020-08-07T15:11:38.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1439999999999997e-06 2020-08-07T15:11:38.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:38.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706658+0000", 2020-08-07T15:11:38.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1663999999999999e-05 2020-08-07T15:11:38.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:38.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706709+0000", 2020-08-07T15:11:38.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0149000000000001e-05 2020-08-07T15:11:38.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:38.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706719+0000", 2020-08-07T15:11:38.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329694900000001 2020-08-07T15:11:38.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:38.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480016+0000", 2020-08-07T15:11:38.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7998999999999997e-05 2020-08-07T15:11:38.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:11:38.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480084+0000", 2020-08-07T15:11:38.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010547333000000001 2020-08-07T15:11:38.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:38.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490632+0000", 2020-08-07T15:11:38.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7439e-05 2020-08-07T15:11:38.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:38.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490729+0000", 2020-08-07T15:11:38.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998 2020-08-07T15:11:38.781 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:38.781 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:38.781 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:38.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.782 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30924 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:11:38.782 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:11:38.783 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 171.66355024000001, 2020-08-07T15:11:38.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995, 2020-08-07T15:11:38.783 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:38.783 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:38.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:38.784 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:38.784 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:38.784 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30924 2020-08-07T15:11:38.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.784 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:38.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:38.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:11:38.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:38.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:38.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:11:38.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9698999999999999e-05 2020-08-07T15:11:38.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:38.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706762+0000", 2020-08-07T15:11:38.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8080000000000001e-06 2020-08-07T15:11:38.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706767+0000", 2020-08-07T15:11:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1069e-05 2020-08-07T15:11:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:38.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706778+0000", 2020-08-07T15:11:38.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6969999999999998e-06 2020-08-07T15:11:38.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:38.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706784+0000", 2020-08-07T15:11:38.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8792999999999998e-05 2020-08-07T15:11:38.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:38.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706843+0000", 2020-08-07T15:11:38.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5449999999999998e-06 2020-08-07T15:11:38.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:38.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706850+0000", 2020-08-07T15:11:38.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.773278354 2020-08-07T15:11:38.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:38.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480129+0000", 2020-08-07T15:11:38.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1577999999999997e-05 2020-08-07T15:11:38.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:11:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480210+0000", 2020-08-07T15:11:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010541915000000001 2020-08-07T15:11:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490752+0000", 2020-08-07T15:11:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103937 2020-08-07T15:11:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490856+0000", 2020-08-07T15:11:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995 2020-08-07T15:11:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30925 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:11:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:11:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 171.663426469, 2020-08-07T15:11:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995, 2020-08-07T15:11:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30925 2020-08-07T15:11:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:11:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:11:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4664999999999999e-05 2020-08-07T15:11:38.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:38.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706880+0000", 2020-08-07T15:11:38.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7479999999999999e-06 2020-08-07T15:11:38.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:38.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706886+0000", 2020-08-07T15:11:38.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1838000000000001e-05 2020-08-07T15:11:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706898+0000", 2020-08-07T15:11:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0089999999999998e-06 2020-08-07T15:11:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706903+0000", 2020-08-07T15:11:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3559e-05 2020-08-07T15:11:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706956+0000", 2020-08-07T15:11:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0309000000000001e-05 2020-08-07T15:11:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706967+0000", 2020-08-07T15:11:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77328290499999996 2020-08-07T15:11:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480250+0000", 2020-08-07T15:11:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8680999999999997e-05 2020-08-07T15:11:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:11:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480298+0000", 2020-08-07T15:11:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01059272 2020-08-07T15:11:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490891+0000", 2020-08-07T15:11:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014580300000000001 2020-08-07T15:11:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491037+0000", 2020-08-07T15:11:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995 2020-08-07T15:11:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30926 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:11:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:11:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 171.66330959999999, 2020-08-07T15:11:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005, 2020-08-07T15:11:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30926 2020-08-07T15:11:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:11:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:11:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5223000000000003e-05 2020-08-07T15:11:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706998+0000", 2020-08-07T15:11:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4809999999999997e-06 2020-08-07T15:11:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707003+0000", 2020-08-07T15:11:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1663e-05 2020-08-07T15:11:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707015+0000", 2020-08-07T15:11:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9729999999999997e-06 2020-08-07T15:11:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707020+0000", 2020-08-07T15:11:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2565999999999999e-05 2020-08-07T15:11:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707062+0000", 2020-08-07T15:11:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8944e-05 2020-08-07T15:11:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707081+0000", 2020-08-07T15:11:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77325885000000005 2020-08-07T15:11:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480340+0000", 2020-08-07T15:11:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0321000000000006e-05 2020-08-07T15:11:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:11:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480420+0000", 2020-08-07T15:11:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010655958 2020-08-07T15:11:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491076+0000", 2020-08-07T15:11:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9784999999999995e-05 2020-08-07T15:11:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491166+0000", 2020-08-07T15:11:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005 2020-08-07T15:11:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30927 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:11:38.896 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:11:38.897 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 171.66319178099999, 2020-08-07T15:11:38.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999, 2020-08-07T15:11:38.899 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:38.899 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:38.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:38.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:38.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:38.900 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30927 2020-08-07T15:11:38.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.901 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:38.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:38.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:11:38.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:38.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:38.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:11:38.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9861999999999997e-05 2020-08-07T15:11:38.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:38.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707120+0000", 2020-08-07T15:11:38.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8969999999999996e-06 2020-08-07T15:11:38.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:38.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707126+0000", 2020-08-07T15:11:38.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1248e-05 2020-08-07T15:11:38.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:38.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707137+0000", 2020-08-07T15:11:38.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.955e-06 2020-08-07T15:11:38.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:38.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707142+0000", 2020-08-07T15:11:38.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0643999999999998e-05 2020-08-07T15:11:38.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:38.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707193+0000", 2020-08-07T15:11:38.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1983e-05 2020-08-07T15:11:38.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:38.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707205+0000", 2020-08-07T15:11:38.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77324914 2020-08-07T15:11:38.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:38.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480454+0000", 2020-08-07T15:11:38.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4119999999999998e-05 2020-08-07T15:11:38.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:11:38.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480498+0000", 2020-08-07T15:11:38.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010709707000000001 2020-08-07T15:11:38.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:38.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491208+0000", 2020-08-07T15:11:38.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014433800000000001 2020-08-07T15:11:38.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:38.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491352+0000", 2020-08-07T15:11:38.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999 2020-08-07T15:11:38.973 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:38.973 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:38.974 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:38.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.974 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30928 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:11:38.974 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:11:38.975 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 171.66306742399999, 2020-08-07T15:11:38.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998, 2020-08-07T15:11:38.975 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:38.975 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:38.975 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:38.975 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30928 2020-08-07T15:11:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:38.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:11:38.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:38.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:38.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:11:38.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6582000000000001e-05 2020-08-07T15:11:38.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:38.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707241+0000", 2020-08-07T15:11:38.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9190000000000004e-06 2020-08-07T15:11:38.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:38.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707247+0000", 2020-08-07T15:11:38.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000188822 2020-08-07T15:11:38.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:38.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:38.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:38.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707436+0000", 2020-08-07T15:11:39.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2940000000000002e-06 2020-08-07T15:11:39.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:39.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707442+0000", 2020-08-07T15:11:39.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7800000000000003e-05 2020-08-07T15:11:39.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:39.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707490+0000", 2020-08-07T15:11:39.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2020-08-07T15:11:39.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:39.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707501+0000", 2020-08-07T15:11:39.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77303498800000003 2020-08-07T15:11:39.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:39.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480536+0000", 2020-08-07T15:11:39.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7723000000000001e-05 2020-08-07T15:11:39.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:11:39.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480614+0000", 2020-08-07T15:11:39.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010812183 2020-08-07T15:11:39.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:39.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491426+0000", 2020-08-07T15:11:39.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5502999999999999e-05 2020-08-07T15:11:39.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:39.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491502+0000", 2020-08-07T15:11:39.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998 2020-08-07T15:11:39.017 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:39.022 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:39.022 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:39.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.023 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31009 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:11:39.023 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:11:39.023 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 171.63911111799999, 2020-08-07T15:11:39.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002, 2020-08-07T15:11:39.024 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:39.024 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:39.024 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:39.024 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:39.024 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:39.025 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31009 2020-08-07T15:11:39.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.025 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:39.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:39.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:11:39.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:39.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:39.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:11:39.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2429000000000002e-05 2020-08-07T15:11:39.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:39.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731193+0000", 2020-08-07T15:11:39.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9760000000000003e-06 2020-08-07T15:11:39.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:39.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731198+0000", 2020-08-07T15:11:39.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3597e-05 2020-08-07T15:11:39.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:39.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731212+0000", 2020-08-07T15:11:39.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1709999999999996e-06 2020-08-07T15:11:39.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:39.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731219+0000", 2020-08-07T15:11:39.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5402999999999997e-05 2020-08-07T15:11:39.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:39.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731285+0000", 2020-08-07T15:11:39.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3495e-05 2020-08-07T15:11:39.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:39.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731298+0000", 2020-08-07T15:11:39.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74935423599999995 2020-08-07T15:11:39.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:39.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480652+0000", 2020-08-07T15:11:39.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000137806 2020-08-07T15:11:39.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:39.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480790+0000", 2020-08-07T15:11:39.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022272399999999999 2020-08-07T15:11:39.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:11:39.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481013+0000", 2020-08-07T15:11:39.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003223381 2020-08-07T15:11:39.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:39.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484236+0000", 2020-08-07T15:11:39.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1370000000000001e-06 2020-08-07T15:11:39.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:39.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484237+0000", 2020-08-07T15:11:39.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6288e-05 2020-08-07T15:11:39.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:39.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484294+0000", 2020-08-07T15:11:39.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.007260402 2020-08-07T15:11:39.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:39.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491554+0000", 2020-08-07T15:11:39.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7358000000000003e-05 2020-08-07T15:11:39.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:39.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491601+0000", 2020-08-07T15:11:39.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1100000000000004e-07 2020-08-07T15:11:39.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:39.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491602+0000", 2020-08-07T15:11:39.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013436500000000001 2020-08-07T15:11:39.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:39.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491737+0000", 2020-08-07T15:11:39.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002 2020-08-07T15:11:39.056 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:39.056 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:39.057 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:39.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.057 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31010 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:11:39.057 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:11:39.058 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 171.63898516399999, 2020-08-07T15:11:39.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001, 2020-08-07T15:11:39.058 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:39.058 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:39.058 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:39.058 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:39.060 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:39.061 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31010 2020-08-07T15:11:39.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.062 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:39.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:39.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:11:39.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:39.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:39.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:11:39.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2398000000000002e-05 2020-08-07T15:11:39.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:39.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731319+0000", 2020-08-07T15:11:39.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7100000000000004e-06 2020-08-07T15:11:39.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:39.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731325+0000", 2020-08-07T15:11:39.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4328000000000001e-05 2020-08-07T15:11:39.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:39.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731339+0000", 2020-08-07T15:11:39.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3310000000000003e-06 2020-08-07T15:11:39.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:39.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731345+0000", 2020-08-07T15:11:39.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8952999999999998e-05 2020-08-07T15:11:39.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:39.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731384+0000", 2020-08-07T15:11:39.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0036e-05 2020-08-07T15:11:39.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:39.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731394+0000", 2020-08-07T15:11:39.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75027681300000004 2020-08-07T15:11:39.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:39.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481670+0000", 2020-08-07T15:11:39.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016555899999999999 2020-08-07T15:11:39.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:39.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481836+0000", 2020-08-07T15:11:39.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028760999999999999 2020-08-07T15:11:39.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:11:39.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482124+0000", 2020-08-07T15:11:39.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038342680000000001 2020-08-07T15:11:39.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:39.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485958+0000", 2020-08-07T15:11:39.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2559999999999999e-06 2020-08-07T15:11:39.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:39.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485959+0000", 2020-08-07T15:11:39.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3758999999999995e-05 2020-08-07T15:11:39.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:39.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486043+0000", 2020-08-07T15:11:39.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057590419999999998 2020-08-07T15:11:39.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:39.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491802+0000", 2020-08-07T15:11:39.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5757000000000001e-05 2020-08-07T15:11:39.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:39.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:11:39.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9299999999999997e-07 2020-08-07T15:11:39.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:39.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:11:39.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012496999999999999 2020-08-07T15:11:39.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:39.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491993+0000", 2020-08-07T15:11:39.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001 2020-08-07T15:11:39.141 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:39.141 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:39.142 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:39.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.142 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31011 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:11:39.144 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:11:39.144 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 171.63886895900001, 2020-08-07T15:11:39.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997, 2020-08-07T15:11:39.144 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:39.145 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:39.145 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:39.145 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:39.145 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:39.146 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31011 2020-08-07T15:11:39.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.146 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:39.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:39.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:11:39.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:39.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:39.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:11:39.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9539999999999997e-05 2020-08-07T15:11:39.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:39.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731433+0000", 2020-08-07T15:11:39.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9209999999999997e-06 2020-08-07T15:11:39.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:39.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731439+0000", 2020-08-07T15:11:39.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4246999999999999e-05 2020-08-07T15:11:39.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:39.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731453+0000", 2020-08-07T15:11:39.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8130000000000003e-06 2020-08-07T15:11:39.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:39.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731458+0000", 2020-08-07T15:11:39.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6668000000000001e-05 2020-08-07T15:11:39.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:39.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731504+0000", 2020-08-07T15:11:39.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1112e-05 2020-08-07T15:11:39.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:39.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731515+0000", 2020-08-07T15:11:39.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75121332100000004 2020-08-07T15:11:39.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:39.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482729+0000", 2020-08-07T15:11:39.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1810000000000004e-05 2020-08-07T15:11:39.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:39.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482811+0000", 2020-08-07T15:11:39.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018001399999999999 2020-08-07T15:11:39.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:11:39.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482991+0000", 2020-08-07T15:11:39.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030701140000000001 2020-08-07T15:11:39.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:39.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486061+0000", 2020-08-07T15:11:39.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4499999999999996e-07 2020-08-07T15:11:39.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:39.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486062+0000", 2020-08-07T15:11:39.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5366999999999998e-05 2020-08-07T15:11:39.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:39.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486147+0000", 2020-08-07T15:11:39.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056872779999999996 2020-08-07T15:11:39.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:39.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491834+0000", 2020-08-07T15:11:39.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024645100000000002 2020-08-07T15:11:39.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:39.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:11:39.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1200000000000003e-07 2020-08-07T15:11:39.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:39.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:11:39.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1451000000000002e-05 2020-08-07T15:11:39.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:39.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492173+0000", 2020-08-07T15:11:39.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997 2020-08-07T15:11:39.175 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:39.176 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:39.176 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:39.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.203 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31012 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:11:39.203 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:11:39.203 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 171.63821483999999, 2020-08-07T15:11:39.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003, 2020-08-07T15:11:39.204 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:39.204 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:39.204 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:39.205 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:39.205 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:39.205 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31012 2020-08-07T15:11:39.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.205 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:39.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:39.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:11:39.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:39.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:39.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:11:39.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6344999999999997e-05 2020-08-07T15:11:39.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:39.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732094+0000", 2020-08-07T15:11:39.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2920000000000001e-06 2020-08-07T15:11:39.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:39.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732100+0000", 2020-08-07T15:11:39.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.189e-05 2020-08-07T15:11:39.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:39.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732112+0000", 2020-08-07T15:11:39.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3720000000000001e-06 2020-08-07T15:11:39.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:39.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732118+0000", 2020-08-07T15:11:39.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2514000000000004e-05 2020-08-07T15:11:39.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:39.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732191+0000", 2020-08-07T15:11:39.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4610000000000001e-05 2020-08-07T15:11:39.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:39.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732205+0000", 2020-08-07T15:11:39.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75132974100000005 2020-08-07T15:11:39.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:39.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483535+0000", 2020-08-07T15:11:39.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9185999999999996e-05 2020-08-07T15:11:39.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:39.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483624+0000", 2020-08-07T15:11:39.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015683 2020-08-07T15:11:39.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:11:39.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483781+0000", 2020-08-07T15:11:39.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046202889999999996 2020-08-07T15:11:39.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:39.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488401+0000", 2020-08-07T15:11:39.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6700000000000002e-07 2020-08-07T15:11:39.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:39.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488402+0000", 2020-08-07T15:11:39.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1487000000000001e-05 2020-08-07T15:11:39.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:39.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488484+0000", 2020-08-07T15:11:39.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035520830000000002 2020-08-07T15:11:39.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492036+0000", 2020-08-07T15:11:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029266600000000002 2020-08-07T15:11:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492328+0000", 2020-08-07T15:11:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5099999999999998e-07 2020-08-07T15:11:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:39.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492329+0000", 2020-08-07T15:11:39.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000201299 2020-08-07T15:11:39.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:39.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492531+0000", 2020-08-07T15:11:39.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003 2020-08-07T15:11:39.268 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:39.268 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:39.268 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:39.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31013 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:11:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:11:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 171.638003209, 2020-08-07T15:11:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996, 2020-08-07T15:11:39.287 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:39.287 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:39.287 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:39.288 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:39.288 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:39.288 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31013 2020-08-07T15:11:39.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.289 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:39.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:39.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:11:39.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:39.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:39.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:11:39.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8949000000000002e-05 2020-08-07T15:11:39.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:39.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732298+0000", 2020-08-07T15:11:39.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1399999999999997e-06 2020-08-07T15:11:39.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:39.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732304+0000", 2020-08-07T15:11:39.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.154e-05 2020-08-07T15:11:39.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:39.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732315+0000", 2020-08-07T15:11:39.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1319999999999999e-06 2020-08-07T15:11:39.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:39.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732322+0000", 2020-08-07T15:11:39.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7431e-05 2020-08-07T15:11:39.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:39.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732369+0000", 2020-08-07T15:11:39.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0621999999999999e-05 2020-08-07T15:11:39.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:39.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732380+0000", 2020-08-07T15:11:39.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75196845999999995 2020-08-07T15:11:39.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:39.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484348+0000", 2020-08-07T15:11:39.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000123008 2020-08-07T15:11:39.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:39.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484471+0000", 2020-08-07T15:11:39.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017572700000000001 2020-08-07T15:11:39.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:11:39.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484647+0000", 2020-08-07T15:11:39.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038557650000000001 2020-08-07T15:11:39.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:39.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:11:39.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2020-08-07T15:11:39.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:39.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:11:39.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0714e-05 2020-08-07T15:11:39.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:39.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488584+0000", 2020-08-07T15:11:39.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036298319999999999 2020-08-07T15:11:39.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:39.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492214+0000", 2020-08-07T15:11:39.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039530099999999997 2020-08-07T15:11:39.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:39.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492609+0000", 2020-08-07T15:11:39.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5300000000000002e-07 2020-08-07T15:11:39.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:39.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492610+0000", 2020-08-07T15:11:39.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197321 2020-08-07T15:11:39.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:39.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492807+0000", 2020-08-07T15:11:39.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996 2020-08-07T15:11:39.382 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:39.383 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:39.383 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:39.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.384 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31014 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:11:39.384 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:11:39.385 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 171.63662106500001, 2020-08-07T15:11:39.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998, 2020-08-07T15:11:39.385 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:39.385 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:39.386 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:39.386 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:39.386 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:39.386 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31014 2020-08-07T15:11:39.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.387 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:39.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:39.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:11:39.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:39.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:39.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:11:39.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3743999999999999e-05 2020-08-07T15:11:39.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:39.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733685+0000", 2020-08-07T15:11:39.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1720000000000007e-06 2020-08-07T15:11:39.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:39.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733693+0000", 2020-08-07T15:11:39.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3674e-05 2020-08-07T15:11:39.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:39.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733707+0000", 2020-08-07T15:11:39.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9249999999999998e-06 2020-08-07T15:11:39.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:39.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733714+0000", 2020-08-07T15:11:39.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8338000000000003e-05 2020-08-07T15:11:39.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:39.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733772+0000", 2020-08-07T15:11:39.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3019e-05 2020-08-07T15:11:39.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:39.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733785+0000", 2020-08-07T15:11:39.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75134856299999997 2020-08-07T15:11:39.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:39.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485133+0000", 2020-08-07T15:11:39.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8739999999999995e-05 2020-08-07T15:11:39.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:39.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485212+0000", 2020-08-07T15:11:39.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151244 2020-08-07T15:11:39.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:11:39.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485363+0000", 2020-08-07T15:11:39.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032929090000000001 2020-08-07T15:11:39.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:39.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488656+0000", 2020-08-07T15:11:39.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.065e-06 2020-08-07T15:11:39.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:39.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488657+0000", 2020-08-07T15:11:39.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1123000000000003e-05 2020-08-07T15:11:39.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:39.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488729+0000", 2020-08-07T15:11:39.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035508330000000002 2020-08-07T15:11:39.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:39.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492279+0000", 2020-08-07T15:11:39.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00059376000000000003 2020-08-07T15:11:39.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:39.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492873+0000", 2020-08-07T15:11:39.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5700000000000002e-07 2020-08-07T15:11:39.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:39.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492874+0000", 2020-08-07T15:11:39.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019264 2020-08-07T15:11:39.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:39.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493066+0000", 2020-08-07T15:11:39.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998 2020-08-07T15:11:39.437 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:39.437 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:39.447 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:39.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.448 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31015 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:11:39.448 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:11:39.448 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 171.636495099, 2020-08-07T15:11:39.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002, 2020-08-07T15:11:39.448 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:39.449 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:39.449 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:39.449 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:39.449 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:39.449 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31015 2020-08-07T15:11:39.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.450 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:39.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:39.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:11:39.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:39.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:39.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:11:39.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3707000000000001e-05 2020-08-07T15:11:39.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:39.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733791+0000", 2020-08-07T15:11:39.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8120000000000001e-06 2020-08-07T15:11:39.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:39.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733794+0000", 2020-08-07T15:11:39.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0728e-05 2020-08-07T15:11:39.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:39.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733805+0000", 2020-08-07T15:11:39.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8279999999999999e-06 2020-08-07T15:11:39.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:39.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733811+0000", 2020-08-07T15:11:39.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2955999999999999e-05 2020-08-07T15:11:39.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:39.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733864+0000", 2020-08-07T15:11:39.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1389000000000001e-05 2020-08-07T15:11:39.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:39.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733875+0000", 2020-08-07T15:11:39.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75232711100000005 2020-08-07T15:11:39.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:39.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486202+0000", 2020-08-07T15:11:39.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027503400000000001 2020-08-07T15:11:39.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:39.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486478+0000", 2020-08-07T15:11:39.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033010400000000001 2020-08-07T15:11:39.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:11:39.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486808+0000", 2020-08-07T15:11:39.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019382609999999999 2020-08-07T15:11:39.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:39.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488746+0000", 2020-08-07T15:11:39.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9400000000000004e-07 2020-08-07T15:11:39.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:39.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488747+0000", 2020-08-07T15:11:39.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4324000000000002e-05 2020-08-07T15:11:39.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:39.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488821+0000", 2020-08-07T15:11:39.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043745640000000001 2020-08-07T15:11:39.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:39.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493196+0000", 2020-08-07T15:11:39.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030381000000000001 2020-08-07T15:11:39.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:39.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493499+0000", 2020-08-07T15:11:39.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4e-07 2020-08-07T15:11:39.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:39.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493500+0000", 2020-08-07T15:11:39.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017499 2020-08-07T15:11:39.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:39.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493675+0000", 2020-08-07T15:11:39.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002 2020-08-07T15:11:39.534 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:39.546 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:39.546 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:39.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.547 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31016 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:11:39.547 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:11:39.548 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 171.63589912399999, 2020-08-07T15:11:39.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971, 2020-08-07T15:11:39.548 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:39.548 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:39.549 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:39.549 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:39.549 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:39.549 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31016 2020-08-07T15:11:39.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.556 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:39.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:39.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:11:39.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:39.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:39.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:11:39.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0924000000000002e-05 2020-08-07T15:11:39.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:39.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734404+0000", 2020-08-07T15:11:39.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2020-08-07T15:11:39.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:39.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734410+0000", 2020-08-07T15:11:39.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3459999999999994e-06 2020-08-07T15:11:39.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:39.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734418+0000", 2020-08-07T15:11:39.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9640000000000002e-06 2020-08-07T15:11:39.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:39.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734423+0000", 2020-08-07T15:11:39.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7253999999999999e-05 2020-08-07T15:11:39.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:39.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734470+0000", 2020-08-07T15:11:39.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0249999999999997e-06 2020-08-07T15:11:39.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:11:39.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734477+0000", 2020-08-07T15:11:39.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75294592900000001 2020-08-07T15:11:39.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:39.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487423+0000", 2020-08-07T15:11:39.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031604300000000001 2020-08-07T15:11:39.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:39.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487739+0000", 2020-08-07T15:11:39.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000171149 2020-08-07T15:11:39.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:11:39.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487910+0000", 2020-08-07T15:11:39.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024207769999999998 2020-08-07T15:11:39.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:39.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490331+0000", 2020-08-07T15:11:39.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1650000000000001e-06 2020-08-07T15:11:39.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:39.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490332+0000", 2020-08-07T15:11:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9796999999999996e-05 2020-08-07T15:11:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490402+0000", 2020-08-07T15:11:39.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029928849999999998 2020-08-07T15:11:39.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:39.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493395+0000", 2020-08-07T15:11:39.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034903199999999999 2020-08-07T15:11:39.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:39.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493744+0000", 2020-08-07T15:11:39.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8899999999999999e-07 2020-08-07T15:11:39.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:39.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493745+0000", 2020-08-07T15:11:39.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197362 2020-08-07T15:11:39.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:39.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493942+0000", 2020-08-07T15:11:39.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971 2020-08-07T15:11:39.627 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:39.627 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:39.628 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:39.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.628 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:89632 5.f e207/164 5:fd260020:::benchmark_data_smithi074_53697_object11203:head v 207'5744, mlcod=207'5734)", 2020-08-07T15:11:39.631 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:21.551762+0000", 2020-08-07T15:11:39.631 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.818490267000001, 2020-08-07T15:11:39.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67527211499999995, 2020-08-07T15:11:39.631 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:39.632 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:39.639 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:39.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:39.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551762+0000", 2020-08-07T15:11:39.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:39.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:39.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551762+0000", 2020-08-07T15:11:39.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8560000000000003e-05 2020-08-07T15:11:39.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:39.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551810+0000", 2020-08-07T15:11:39.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9430000000000002e-06 2020-08-07T15:11:39.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:39.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551814+0000", 2020-08-07T15:11:39.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67230272000000002 2020-08-07T15:11:39.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:39.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.224117+0000", 2020-08-07T15:11:39.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7120000000000003e-06 2020-08-07T15:11:39.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:39.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.224124+0000", 2020-08-07T15:11:39.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001719307 2020-08-07T15:11:39.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:39.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.225843+0000", 2020-08-07T15:11:39.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3902999999999997e-05 2020-08-07T15:11:39.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:39.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.225877+0000", 2020-08-07T15:11:39.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001110279 2020-08-07T15:11:39.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:39.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.226987+0000", 2020-08-07T15:11:39.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6690999999999997e-05 2020-08-07T15:11:39.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:39.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.227034+0000", 2020-08-07T15:11:39.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67527211499999995 2020-08-07T15:11:39.679 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:39.679 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:39.679 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:39.679 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:39.680 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:39.680 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:11:39.680 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:11:39.681 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:11:39.689 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 79 16 113419 113403 11.2136 11.7344 0.0106668 0.0111428 2020-08-07T15:11:39.703 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:11:39.704 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:39.681+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:39.800 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:39.799+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:39.898 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:39.894+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:39.906 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:11:39.907 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:11:39.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.907 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:114487 5.e 5:776ce125:::benchmark_data_smithi074_53697_object14310:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e215)", 2020-08-07T15:11:39.907 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:39.884405+0000", 2020-08-07T15:11:39.908 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.016783580999999999, 2020-08-07T15:11:39.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016804021999999998, 2020-08-07T15:11:39.908 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:39.910 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:39.910 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:39.910 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:11:39.910 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:39.911 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 114487 2020-08-07T15:11:39.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.911 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:39.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:39.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.884405+0000", 2020-08-07T15:11:39.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:39.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:39.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.884405+0000", 2020-08-07T15:11:39.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0139999999999998e-05 2020-08-07T15:11:39.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:39.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.884455+0000", 2020-08-07T15:11:39.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2780000000000001e-06 2020-08-07T15:11:39.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:39.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.884461+0000", 2020-08-07T15:11:39.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1316e-05 2020-08-07T15:11:39.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:39.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.884472+0000", 2020-08-07T15:11:39.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4320000000000001e-06 2020-08-07T15:11:39.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:39.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.884476+0000", 2020-08-07T15:11:39.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001811944 2020-08-07T15:11:39.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:39.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.886288+0000", 2020-08-07T15:11:39.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3293000000000003e-05 2020-08-07T15:11:39.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.886382+0000", 2020-08-07T15:11:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016658600000000001 2020-08-07T15:11:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:11:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.886548+0000", 2020-08-07T15:11:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014095620000000001 2020-08-07T15:11:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.887958+0000", 2020-08-07T15:11:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2020-08-07T15:11:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:39.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:39.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:39.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.887958+0000", 2020-08-07T15:11:39.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1332000000000002e-05 2020-08-07T15:11:39.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:40.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:40.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:40.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.888020+0000", 2020-08-07T15:11:40.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010668482 2020-08-07T15:11:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.898688+0000", 2020-08-07T15:11:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014282922 2020-08-07T15:11:40.016 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:40.017 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:40.018 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:40.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:40.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:40.018 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:114488 5.e 5:776ce125:::benchmark_data_smithi074_53697_object14310:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e215)", 2020-08-07T15:11:40.019 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:39.885120+0000", 2020-08-07T15:11:40.019 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.016069155000000002, 2020-08-07T15:11:40.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016156631000000001, 2020-08-07T15:11:40.019 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:40.020 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:40.020 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:40.020 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:11:40.020 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:40.020 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 114488 2020-08-07T15:11:40.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:40.021 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:40.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:40.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:40.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.885120+0000", 2020-08-07T15:11:40.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:40.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:40.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:40.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:40.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.885120+0000", 2020-08-07T15:11:40.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7338e-05 2020-08-07T15:11:40.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:40.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:40.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:40.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.885167+0000", 2020-08-07T15:11:40.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9200000000000001e-06 2020-08-07T15:11:40.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:40.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:40.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:40.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.885173+0000", 2020-08-07T15:11:40.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2021000000000001e-05 2020-08-07T15:11:40.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:40.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:40.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:40.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.885185+0000", 2020-08-07T15:11:40.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4280000000000004e-06 2020-08-07T15:11:40.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:40.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:40.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:40.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.885190+0000", 2020-08-07T15:11:40.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018217299999999999 2020-08-07T15:11:40.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:40.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:40.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:40.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.887012+0000", 2020-08-07T15:11:40.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0325000000000001e-05 2020-08-07T15:11:40.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:40.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:40.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:40.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.887092+0000", 2020-08-07T15:11:40.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000108216 2020-08-07T15:11:40.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:40.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:40.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:11:40.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.887201+0000", 2020-08-07T15:11:40.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001317892 2020-08-07T15:11:40.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:40.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:40.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:40.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.888519+0000", 2020-08-07T15:11:40.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1299999999999999e-07 2020-08-07T15:11:40.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:40.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:40.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:40.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.888519+0000", 2020-08-07T15:11:40.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4774e-05 2020-08-07T15:11:40.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:40.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:40.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:40.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.888584+0000", 2020-08-07T15:11:40.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010300136 2020-08-07T15:11:40.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:40.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:40.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:40.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.898884+0000", 2020-08-07T15:11:40.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013764593 2020-08-07T15:11:40.080 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:40.080 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:40.081 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:40.081 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:40.081 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:11:40.081 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 2 2020-08-07T15:11:40.091 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:11:40.120 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:11:40.121 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:11:40.121 INFO:teuthology.orchestra.run.smithi074.stdout: "epoch": 215, 2020-08-07T15:11:40.121 INFO:teuthology.orchestra.run.smithi074.stdout: "fsid": "a01d3e82-8d24-4288-9e42-576ab3ff4957", 2020-08-07T15:11:40.121 INFO:teuthology.orchestra.run.smithi074.stdout: "created": "2020-08-07T15:06:07.342912+0000", 2020-08-07T15:11:40.122 INFO:teuthology.orchestra.run.smithi074.stdout: "modified": "2020-08-07T15:11:37.200782+0000", 2020-08-07T15:11:40.122 INFO:teuthology.orchestra.run.smithi074.stdout: "last_up_change": "2020-08-07T15:10:50.524683+0000", 2020-08-07T15:11:40.122 INFO:teuthology.orchestra.run.smithi074.stdout: "last_in_change": "2020-08-07T15:06:17.674292+0000", 2020-08-07T15:11:40.122 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-08-07T15:11:40.122 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_num": 5804032, 2020-08-07T15:11:40.123 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_set": [ 2020-08-07T15:11:40.123 INFO:teuthology.orchestra.run.smithi074.stdout: "nodeep-scrub", 2020-08-07T15:11:40.123 INFO:teuthology.orchestra.run.smithi074.stdout: "pglog_hardlimit", 2020-08-07T15:11:40.123 INFO:teuthology.orchestra.run.smithi074.stdout: "purged_snapdirs", 2020-08-07T15:11:40.123 INFO:teuthology.orchestra.run.smithi074.stdout: "recovery_deletes", 2020-08-07T15:11:40.124 INFO:teuthology.orchestra.run.smithi074.stdout: "sortbitwise" 2020-08-07T15:11:40.124 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:11:40.124 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_version": 5, 2020-08-07T15:11:40.124 INFO:teuthology.orchestra.run.smithi074.stdout: "full_ratio": 0.94999998807907104, 2020-08-07T15:11:40.124 INFO:teuthology.orchestra.run.smithi074.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-08-07T15:11:40.125 INFO:teuthology.orchestra.run.smithi074.stdout: "nearfull_ratio": 0.85000002384185791, 2020-08-07T15:11:40.125 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_snapshot": "", 2020-08-07T15:11:40.125 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_max": 5, 2020-08-07T15:11:40.125 INFO:teuthology.orchestra.run.smithi074.stdout: "max_osd": 8, 2020-08-07T15:11:40.125 INFO:teuthology.orchestra.run.smithi074.stdout: "require_min_compat_client": "luminous", 2020-08-07T15:11:40.152 INFO:teuthology.orchestra.run.smithi074.stdout: "min_compat_client": "jewel", 2020-08-07T15:11:40.153 INFO:teuthology.orchestra.run.smithi074.stdout: "require_osd_release": "pacific", 2020-08-07T15:11:40.154 INFO:teuthology.orchestra.run.smithi074.stdout: "pools": [ 2020-08-07T15:11:40.154 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.154 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 1, 2020-08-07T15:11:40.154 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "device_health_metrics", 2020-08-07T15:11:40.154 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:06:10.648876+0000", 2020-08-07T15:11:40.161 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:11:40.162 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:11:40.162 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:11:40.163 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:11:40.163 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:11:40.163 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:11:40.163 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:11:40.164 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:11:40.164 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 31, 2020-08-07T15:11:40.164 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 31, 2020-08-07T15:11:40.164 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 31, 2020-08-07T15:11:40.164 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 31, 2020-08-07T15:11:40.164 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 31, 2020-08-07T15:11:40.165 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:11:40.165 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:11:40.165 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:11:40.165 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:11:40.165 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:11:40.173 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:11:40.173 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:11:40.174 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.174 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "188", 2020-08-07T15:11:40.174 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:11:40.174 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:11:40.174 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "186", 2020-08-07T15:11:40.175 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:11:40.175 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:11:40.175 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:11:40.175 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:11:40.175 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:11:40.176 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:11:40.176 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:11:40.176 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:11:40.176 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:11:40.176 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:11:40.177 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:11:40.177 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:11:40.177 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:11:40.177 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:11:40.177 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:11:40.195 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:11:40.195 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:11:40.196 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:11:40.196 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:11:40.196 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:11:40.196 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:11:40.197 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:11:40.197 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:11:40.197 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.197 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:11:40.201 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:11:40.202 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:11:40.202 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:11:40.202 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:11:40.202 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:11:40.202 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:11:40.203 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:11:40.203 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:11:40.203 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:11:40.203 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:11:40.203 INFO:teuthology.orchestra.run.smithi074.stdout: "options": { 2020-08-07T15:11:40.210 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_min": 1 2020-08-07T15:11:40.210 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.210 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:11:40.215 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_devicehealth": {} 2020-08-07T15:11:40.216 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.216 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.216 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.217 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 2, 2020-08-07T15:11:40.217 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "rbd", 2020-08-07T15:11:40.217 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:06:21.474648+0000", 2020-08-07T15:11:40.217 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:11:40.217 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:11:40.218 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:11:40.218 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:11:40.236 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:11:40.236 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:11:40.236 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:11:40.236 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:11:40.237 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 8, 2020-08-07T15:11:40.237 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 8, 2020-08-07T15:11:40.237 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 8, 2020-08-07T15:11:40.237 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 8, 2020-08-07T15:11:40.237 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 8, 2020-08-07T15:11:40.238 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:11:40.238 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:11:40.238 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:11:40.238 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:11:40.238 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:11:40.239 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:11:40.239 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:11:40.239 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.239 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "19", 2020-08-07T15:11:40.239 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:11:40.240 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:11:40.240 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "0", 2020-08-07T15:11:40.240 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:11:40.240 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:11:40.240 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:11:40.240 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:11:40.241 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:11:40.241 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:11:40.241 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:11:40.241 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:11:40.263 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:11:40.264 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:11:40.264 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:11:40.264 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:11:40.264 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:11:40.265 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:11:40.265 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:11:40.265 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:11:40.265 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:11:40.267 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:11:40.268 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:11:40.268 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:11:40.268 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:11:40.268 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:11:40.268 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:11:40.269 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.269 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:11:40.269 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:11:40.279 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:11:40.279 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:11:40.285 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:11:40.286 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:11:40.286 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:11:40.286 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:11:40.287 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:11:40.297 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:11:40.297 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:11:40.297 INFO:teuthology.orchestra.run.smithi074.stdout: "options": {}, 2020-08-07T15:11:40.302 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:11:40.302 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd": {} 2020-08-07T15:11:40.303 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.303 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.303 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.303 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 5, 2020-08-07T15:11:40.303 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "unique_pool_2", 2020-08-07T15:11:40.304 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:10:14.379166+0000", 2020-08-07T15:11:40.304 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:11:40.304 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:11:40.304 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:11:40.304 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:11:40.305 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:11:40.305 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:11:40.305 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:11:40.305 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:11:40.305 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 16, 2020-08-07T15:11:40.314 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 16, 2020-08-07T15:11:40.314 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 16, 2020-08-07T15:11:40.314 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 16, 2020-08-07T15:11:40.315 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 16, 2020-08-07T15:11:40.315 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:11:40.315 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:11:40.315 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:11:40.315 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:11:40.316 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:11:40.316 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:11:40.316 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:11:40.317 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.317 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "166", 2020-08-07T15:11:40.317 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:11:40.317 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:11:40.317 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "0", 2020-08-07T15:11:40.317 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:11:40.318 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:11:40.318 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:11:40.321 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:11:40.322 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:11:40.322 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:11:40.323 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:11:40.323 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:11:40.323 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:11:40.323 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:11:40.324 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:11:40.324 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:11:40.324 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:11:40.324 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:11:40.324 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:11:40.324 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:11:40.325 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:11:40.325 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:11:40.325 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:11:40.325 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:11:40.325 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:11:40.326 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:11:40.326 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:11:40.326 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.326 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:11:40.326 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:11:40.333 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:11:40.334 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:11:40.334 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:11:40.334 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:11:40.334 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:11:40.335 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:11:40.335 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:11:40.335 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:11:40.335 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:11:40.335 INFO:teuthology.orchestra.run.smithi074.stdout: "options": {}, 2020-08-07T15:11:40.336 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:11:40.336 INFO:teuthology.orchestra.run.smithi074.stdout: "rados": {} 2020-08-07T15:11:40.336 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.336 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.336 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:11:40.336 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:11:40.337 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.337 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 0, 2020-08-07T15:11:40.337 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "fc1ebb23-3ef4-4fc0-b395-382e385885a3", 2020-08-07T15:11:40.337 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:11:40.369 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:11:40.383 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:11:40.383 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0, 2020-08-07T15:11:40.383 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:11:40.383 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:11:40.384 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:11:40.384 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 212, 2020-08-07T15:11:40.384 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:11:40.384 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:11:40.384 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:11:40.385 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.385 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.385 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.385 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6801", 2020-08-07T15:11:40.385 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:11:40.386 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.386 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.386 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.386 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:11:40.386 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.387 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.387 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.387 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6802", 2020-08-07T15:11:40.387 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:11:40.387 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.388 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.388 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.388 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:11:40.388 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.388 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.389 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.389 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6804", 2020-08-07T15:11:40.389 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:11:40.389 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.389 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.406 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.422 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:11:40.422 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.422 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.423 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6803", 2020-08-07T15:11:40.423 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:11:40.423 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.423 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.423 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.424 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6801/31577", 2020-08-07T15:11:40.424 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6802/31577", 2020-08-07T15:11:40.424 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6804/31577", 2020-08-07T15:11:40.424 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6803/31577", 2020-08-07T15:11:40.424 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:11:40.425 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:11:40.425 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:11:40.425 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.425 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.438 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 1, 2020-08-07T15:11:40.438 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "1fbe530a-63ce-4dc8-82ab-18793d6ebc0b", 2020-08-07T15:11:40.438 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 0, 2020-08-07T15:11:40.439 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:11:40.439 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:11:40.439 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0.7196044921875, 2020-08-07T15:11:40.439 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:11:40.439 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:11:40.440 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:11:40.440 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 190, 2020-08-07T15:11:40.440 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 191, 2020-08-07T15:11:40.440 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:11:40.440 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:11:40.441 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.441 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.441 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.441 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6809", 2020-08-07T15:11:40.441 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:11:40.452 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.452 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.453 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:11:40.454 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.454 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.454 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.454 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6810", 2020-08-07T15:11:40.454 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:11:40.455 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.455 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.455 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:11:40.455 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.456 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.456 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6812", 2020-08-07T15:11:40.456 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:11:40.456 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.457 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.457 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:11:40.457 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.468 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.468 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6811", 2020-08-07T15:11:40.468 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31578 2020-08-07T15:11:40.469 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.469 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.469 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.469 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6809/31578", 2020-08-07T15:11:40.469 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6810/31578", 2020-08-07T15:11:40.470 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6812/31578", 2020-08-07T15:11:40.470 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6811/31578", 2020-08-07T15:11:40.470 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:11:40.470 INFO:teuthology.orchestra.run.smithi074.stdout: "exists" 2020-08-07T15:11:40.470 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.471 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.471 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.471 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 2, 2020-08-07T15:11:40.471 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "d6fe4338-3e11-4f73-9cbb-e1b046ce286a", 2020-08-07T15:11:40.471 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:11:40.472 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:11:40.472 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:11:40.472 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:11:40.472 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:11:40.472 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:11:40.473 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:11:40.477 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 212, 2020-08-07T15:11:40.477 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:11:40.477 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:11:40.477 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:11:40.477 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.481 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.481 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6805", 2020-08-07T15:11:40.482 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:11:40.482 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.482 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.482 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.482 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:11:40.483 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.483 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.483 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.483 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6806", 2020-08-07T15:11:40.483 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:11:40.484 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.488 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.489 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:11:40.491 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.499 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.500 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.500 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6808", 2020-08-07T15:11:40.500 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:11:40.500 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.501 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.501 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.501 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:11:40.501 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.501 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.502 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.502 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6807", 2020-08-07T15:11:40.502 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:11:40.502 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.502 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.503 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.503 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6805/31576", 2020-08-07T15:11:40.503 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6806/31576", 2020-08-07T15:11:40.503 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6808/31576", 2020-08-07T15:11:40.503 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6807/31576", 2020-08-07T15:11:40.504 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:11:40.511 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:11:40.511 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:11:40.511 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.511 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.512 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.512 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 3, 2020-08-07T15:11:40.512 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9", 2020-08-07T15:11:40.512 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:11:40.512 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:11:40.513 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:11:40.513 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:11:40.513 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:11:40.513 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:11:40.513 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:11:40.514 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 192, 2020-08-07T15:11:40.514 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:11:40.514 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:11:40.514 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:11:40.514 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.515 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.515 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6813", 2020-08-07T15:11:40.515 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:11:40.515 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.515 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.521 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:11:40.522 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.522 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.522 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.522 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6814", 2020-08-07T15:11:40.522 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:11:40.523 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.533 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.533 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.533 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:11:40.533 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.533 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.534 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.534 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6816", 2020-08-07T15:11:40.534 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:11:40.534 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.534 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.535 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:11:40.535 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.535 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.536 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6815", 2020-08-07T15:11:40.536 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:11:40.536 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.536 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.536 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6813/31579", 2020-08-07T15:11:40.537 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6814/31579", 2020-08-07T15:11:40.537 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6816/31579", 2020-08-07T15:11:40.537 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6815/31579", 2020-08-07T15:11:40.537 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:11:40.546 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:11:40.546 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:11:40.546 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.547 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.547 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 4, 2020-08-07T15:11:40.547 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "01aadc13-29c5-4ddd-9bb9-de24b2b20412", 2020-08-07T15:11:40.547 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:11:40.548 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:11:40.548 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:11:40.548 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0, 2020-08-07T15:11:40.548 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:11:40.548 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:11:40.549 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:11:40.549 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 191, 2020-08-07T15:11:40.549 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:11:40.549 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:11:40.549 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:11:40.563 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.564 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.564 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6800", 2020-08-07T15:11:40.564 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:11:40.565 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.565 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.565 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.565 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:11:40.565 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.566 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.566 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.566 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6801", 2020-08-07T15:11:40.566 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:11:40.566 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.567 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.567 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.567 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:11:40.567 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.567 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.568 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.568 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6803", 2020-08-07T15:11:40.568 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:11:40.568 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.568 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.568 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.569 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:11:40.569 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.569 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.569 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.569 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6802", 2020-08-07T15:11:40.570 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:11:40.570 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.570 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.570 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.570 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6800/31545", 2020-08-07T15:11:40.571 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6801/31545", 2020-08-07T15:11:40.571 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6803/31545", 2020-08-07T15:11:40.571 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6802/31545", 2020-08-07T15:11:40.571 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:11:40.571 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:11:40.572 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:11:40.572 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.572 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.572 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.572 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 5, 2020-08-07T15:11:40.573 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "f5556a82-73b4-47b1-8e29-1d2fe031d8fd", 2020-08-07T15:11:40.573 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:11:40.573 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:11:40.573 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 0.8567047119140625, 2020-08-07T15:11:40.582 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:11:40.582 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:11:40.583 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:11:40.583 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:11:40.583 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 191, 2020-08-07T15:11:40.583 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:11:40.584 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:11:40.584 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:11:40.584 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.584 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.584 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.584 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6804", 2020-08-07T15:11:40.585 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:11:40.588 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.588 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.589 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.589 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:11:40.589 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.589 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.590 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6805", 2020-08-07T15:11:40.590 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:11:40.590 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.590 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.590 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.591 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:11:40.591 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.591 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.591 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.591 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6807", 2020-08-07T15:11:40.592 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:11:40.592 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.593 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.593 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.594 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:11:40.596 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.597 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.597 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.597 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6806", 2020-08-07T15:11:40.597 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:11:40.598 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.598 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.598 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.598 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6804/31547", 2020-08-07T15:11:40.598 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6805/31547", 2020-08-07T15:11:40.598 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6807/31547", 2020-08-07T15:11:40.599 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6806/31547", 2020-08-07T15:11:40.599 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:11:40.599 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:11:40.599 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:11:40.600 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.600 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.601 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 6, 2020-08-07T15:11:40.611 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "9b98b337-e1cc-43cd-9e9b-f21ae3f20899", 2020-08-07T15:11:40.612 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:11:40.612 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:11:40.612 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:11:40.612 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:11:40.612 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:11:40.613 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:11:40.613 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 14, 2020-08-07T15:11:40.613 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 191, 2020-08-07T15:11:40.613 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:11:40.613 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:11:40.614 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:11:40.614 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.614 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.619 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.619 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6808", 2020-08-07T15:11:40.619 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:11:40.620 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.620 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.620 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.620 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:11:40.620 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.638 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.639 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.641 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6809", 2020-08-07T15:11:40.641 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:11:40.641 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.641 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.642 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.642 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:11:40.642 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.642 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.642 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.643 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6811", 2020-08-07T15:11:40.643 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:11:40.645 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.645 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.645 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:11:40.645 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.646 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.646 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.646 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6810", 2020-08-07T15:11:40.646 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:11:40.647 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.647 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.647 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.647 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6808/31546", 2020-08-07T15:11:40.647 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6809/31546", 2020-08-07T15:11:40.648 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6811/31546", 2020-08-07T15:11:40.648 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6810/31546", 2020-08-07T15:11:40.648 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:11:40.648 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:11:40.648 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:11:40.649 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.649 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.649 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.649 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 7, 2020-08-07T15:11:40.649 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "777662d8-bd34-48e8-83e1-24ce9e61f14b", 2020-08-07T15:11:40.650 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:11:40.651 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:11:40.651 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:11:40.651 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:11:40.651 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:11:40.651 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:11:40.652 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:11:40.652 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 192, 2020-08-07T15:11:40.652 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:11:40.657 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:11:40.657 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:11:40.658 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.658 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.658 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.658 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6812", 2020-08-07T15:11:40.658 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:11:40.659 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.659 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.659 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.659 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:11:40.660 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.660 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.660 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.660 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6813", 2020-08-07T15:11:40.660 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:11:40.661 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.661 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.661 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.661 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:11:40.661 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.662 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.662 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.665 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6815", 2020-08-07T15:11:40.666 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:11:40.666 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.666 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.666 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.667 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:11:40.667 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:11:40.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.667 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:11:40.667 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6814", 2020-08-07T15:11:40.668 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:11:40.668 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.668 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.668 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6812/31570", 2020-08-07T15:11:40.669 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6813/31570", 2020-08-07T15:11:40.669 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6815/31570", 2020-08-07T15:11:40.669 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6814/31570", 2020-08-07T15:11:40.669 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:11:40.669 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:11:40.672 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:11:40.672 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.673 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.673 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:11:40.673 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_xinfo": [ 2020-08-07T15:11:40.673 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.673 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 0, 2020-08-07T15:11:40.674 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:11:40.674 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:11:40.674 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:11:40.674 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:11:40.674 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:11:40.675 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:11:18.244439+0000", 2020-08-07T15:11:40.675 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:11:40.675 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.675 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.675 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 1, 2020-08-07T15:11:40.676 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:11:40.676 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:11:40.676 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:11:40.676 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:11:40.676 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:11:40.677 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:10:16.729479+0000", 2020-08-07T15:11:40.677 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 191 2020-08-07T15:11:40.677 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.677 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 2, 2020-08-07T15:11:40.678 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:11:40.678 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:11:40.678 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:11:40.678 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:11:40.679 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:11:40.679 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:11:17.155508+0000", 2020-08-07T15:11:40.679 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:11:40.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.680 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 3, 2020-08-07T15:11:40.680 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:11:40.680 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:11:40.680 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:11:40.680 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:11:40.681 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:11:40.681 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:11:19.498993+0000", 2020-08-07T15:11:40.681 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:11:40.681 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.681 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.682 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 4, 2020-08-07T15:11:40.682 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:11:40.683 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:11:40.683 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:11:40.683 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:11:40.683 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:11:40.683 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:11:18.591484+0000", 2020-08-07T15:11:40.684 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:11:40.684 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.684 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.684 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 5, 2020-08-07T15:11:40.684 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:11:40.685 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:11:40.685 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:11:40.685 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:11:40.685 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:11:40.685 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:11:18.086077+0000", 2020-08-07T15:11:40.686 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:11:40.686 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.686 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.686 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 6, 2020-08-07T15:11:40.686 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:11:40.686 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:11:40.687 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:11:40.687 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:11:40.687 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:11:40.687 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:11:17.762417+0000", 2020-08-07T15:11:40.687 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:11:40.688 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.688 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.688 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 7, 2020-08-07T15:11:40.688 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:11:40.688 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:11:40.689 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:11:40.689 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:11:40.689 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:11:40.689 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:11:17.114874+0000", 2020-08-07T15:11:40.689 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:11:40.690 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.690 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:11:40.690 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_upmap": [], 2020-08-07T15:11:40.690 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_upmap_items": [], 2020-08-07T15:11:40.690 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_temp": [ 2020-08-07T15:11:40.691 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.691 INFO:teuthology.orchestra.run.smithi074.stdout: "pgid": "1.15", 2020-08-07T15:11:40.691 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:11:40.691 INFO:teuthology.orchestra.run.smithi074.stdout: 7, 2020-08-07T15:11:40.691 INFO:teuthology.orchestra.run.smithi074.stdout: 5, 2020-08-07T15:11:40.691 INFO:teuthology.orchestra.run.smithi074.stdout: 2 2020-08-07T15:11:40.692 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.692 INFO:teuthology.orchestra.run.smithi074.stdout: "pgid": "1.16", 2020-08-07T15:11:40.692 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:11:40.693 INFO:teuthology.orchestra.run.smithi074.stdout: 3, 2020-08-07T15:11:40.693 INFO:teuthology.orchestra.run.smithi074.stdout: 0, 2020-08-07T15:11:40.693 INFO:teuthology.orchestra.run.smithi074.stdout: 6 2020-08-07T15:11:40.693 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.693 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.693 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.694 INFO:teuthology.orchestra.run.smithi074.stdout: "pgid": "1.1a", 2020-08-07T15:11:40.694 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:11:40.694 INFO:teuthology.orchestra.run.smithi074.stdout: 7, 2020-08-07T15:11:40.694 INFO:teuthology.orchestra.run.smithi074.stdout: 4, 2020-08-07T15:11:40.694 INFO:teuthology.orchestra.run.smithi074.stdout: 2 2020-08-07T15:11:40.695 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.695 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.695 INFO:teuthology.orchestra.run.smithi074.stdout: "pgid": "1.1c", 2020-08-07T15:11:40.695 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:11:40.696 INFO:teuthology.orchestra.run.smithi074.stdout: 7, 2020-08-07T15:11:40.696 INFO:teuthology.orchestra.run.smithi074.stdout: 6, 2020-08-07T15:11:40.696 INFO:teuthology.orchestra.run.smithi074.stdout: 0 2020-08-07T15:11:40.696 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.697 INFO:teuthology.orchestra.run.smithi074.stdout: "pgid": "1.1d", 2020-08-07T15:11:40.697 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:11:40.697 INFO:teuthology.orchestra.run.smithi074.stdout: 3, 2020-08-07T15:11:40.697 INFO:teuthology.orchestra.run.smithi074.stdout: 4, 2020-08-07T15:11:40.698 INFO:teuthology.orchestra.run.smithi074.stdout: 7 2020-08-07T15:11:40.698 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.698 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:40.698 INFO:teuthology.orchestra.run.smithi074.stdout: "pgid": "5.7", 2020-08-07T15:11:40.698 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:11:40.699 INFO:teuthology.orchestra.run.smithi074.stdout: 0, 2020-08-07T15:11:40.699 INFO:teuthology.orchestra.run.smithi074.stdout: 4, 2020-08-07T15:11:40.699 INFO:teuthology.orchestra.run.smithi074.stdout: 5 2020-08-07T15:11:40.699 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:40.699 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.700 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:11:40.700 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_temp": [], 2020-08-07T15:11:40.700 INFO:teuthology.orchestra.run.smithi074.stdout: "blacklist": {}, 2020-08-07T15:11:40.700 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profiles": { 2020-08-07T15:11:40.700 INFO:teuthology.orchestra.run.smithi074.stdout: "default": { 2020-08-07T15:11:40.700 INFO:teuthology.orchestra.run.smithi074.stdout: "crush-failure-domain": "osd", 2020-08-07T15:11:40.701 INFO:teuthology.orchestra.run.smithi074.stdout: "k": "2", 2020-08-07T15:11:40.701 INFO:teuthology.orchestra.run.smithi074.stdout: "m": "1", 2020-08-07T15:11:40.701 INFO:teuthology.orchestra.run.smithi074.stdout: "plugin": "jerasure", 2020-08-07T15:11:40.701 INFO:teuthology.orchestra.run.smithi074.stdout: "ruleset-failure-domain": "osd", 2020-08-07T15:11:40.701 INFO:teuthology.orchestra.run.smithi074.stdout: "technique": "reed_sol_van" 2020-08-07T15:11:40.702 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:40.702 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:40.702 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps_queue": [], 2020-08-07T15:11:40.702 INFO:teuthology.orchestra.run.smithi074.stdout: "new_removed_snaps": [], 2020-08-07T15:11:40.702 INFO:teuthology.orchestra.run.smithi074.stdout: "new_purged_snaps": [], 2020-08-07T15:11:40.703 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_node_flags": {}, 2020-08-07T15:11:40.703 INFO:teuthology.orchestra.run.smithi074.stdout: "device_class_flags": {} 2020-08-07T15:11:40.703 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:11:40.703 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:11:40.703 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:11:40.704 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:11:40.704 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:11:40.704 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:40.090+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:40.705 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:11:40.706 INFO:tasks.thrashosds.thrasher:j is {'epoch': 215, 'fsid': 'a01d3e82-8d24-4288-9e42-576ab3ff4957', 'created': '2020-08-07T15:06:07.342912+0000', 'modified': '2020-08-07T15:11:37.200782+0000', 'last_up_change': '2020-08-07T15:10:50.524683+0000', 'last_in_change': '2020-08-07T15:06:17.674292+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 5, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-08-07T15:06:10.648876+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 31, 'pg_placement_num': 31, 'pg_placement_num_target': 31, 'pg_num_target': 31, 'pg_num_pending': 31, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '188', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '186', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-08-07T15:06:21.474648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'unique_pool_2', 'create_time': '2020-08-07T15:10:14.379166+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '166', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': 'fc1ebb23-3ef4-4fc0-b395-382e385885a3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 212, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6801', 'nonce': 31577}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6802', 'nonce': 31577}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6804', 'nonce': 31577}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6803', 'nonce': 31577}]}, 'public_addr': '172.21.15.74:6801/31577', 'cluster_addr': '172.21.15.74:6802/31577', 'heartbeat_back_addr': '172.21.15.74:6804/31577', 'heartbeat_front_addr': '172.21.15.74:6803/31577', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '1fbe530a-63ce-4dc8-82ab-18793d6ebc0b', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 0.7196044921875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 190, 'down_at': 191, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6809', 'nonce': 31578}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6810', 'nonce': 31578}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6812', 'nonce': 31578}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6811', 'nonce': 31578}]}, 'public_addr': '172.21.15.74:6809/31578', 'cluster_addr': '172.21.15.74:6810/31578', 'heartbeat_back_addr': '172.21.15.74:6812/31578', 'heartbeat_front_addr': '172.21.15.74:6811/31578', 'state': ['exists']}, {'osd': 2, 'uuid': 'd6fe4338-3e11-4f73-9cbb-e1b046ce286a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 212, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6805', 'nonce': 31576}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6806', 'nonce': 31576}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6808', 'nonce': 31576}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6807', 'nonce': 31576}]}, 'public_addr': '172.21.15.74:6805/31576', 'cluster_addr': '172.21.15.74:6806/31576', 'heartbeat_back_addr': '172.21.15.74:6808/31576', 'heartbeat_front_addr': '172.21.15.74:6807/31576', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 192, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6813', 'nonce': 31579}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6814', 'nonce': 31579}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6816', 'nonce': 31579}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6815', 'nonce': 31579}]}, 'public_addr': '172.21.15.74:6813/31579', 'cluster_addr': '172.21.15.74:6814/31579', 'heartbeat_back_addr': '172.21.15.74:6816/31579', 'heartbeat_front_addr': '172.21.15.74:6815/31579', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '01aadc13-29c5-4ddd-9bb9-de24b2b20412', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 191, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6800', 'nonce': 31545}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6801', 'nonce': 31545}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6803', 'nonce': 31545}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6802', 'nonce': 31545}]}, 'public_addr': '172.21.15.96:6800/31545', 'cluster_addr': '172.21.15.96:6801/31545', 'heartbeat_back_addr': '172.21.15.96:6803/31545', 'heartbeat_front_addr': '172.21.15.96:6802/31545', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'f5556a82-73b4-47b1-8e29-1d2fe031d8fd', 'up': 1, 'in': 1, 'weight': 0.8567047119140625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 191, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6804', 'nonce': 31547}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6805', 'nonce': 31547}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6807', 'nonce': 31547}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6806', 'nonce': 31547}]}, 'public_addr': '172.21.15.96:6804/31547', 'cluster_addr': '172.21.15.96:6805/31547', 'heartbeat_back_addr': '172.21.15.96:6807/31547', 'heartbeat_front_addr': '172.21.15.96:6806/31547', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9b98b337-e1cc-43cd-9e9b-f21ae3f20899', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 191, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6808', 'nonce': 31546}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6809', 'nonce': 31546}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6811', 'nonce': 31546}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6810', 'nonce': 31546}]}, 'public_addr': '172.21.15.96:6808/31546', 'cluster_addr': '172.21.15.96:6809/31546', 'heartbeat_back_addr': '172.21.15.96:6811/31546', 'heartbeat_front_addr': '172.21.15.96:6810/31546', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '777662d8-bd34-48e8-83e1-24ce9e61f14b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 192, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6812', 'nonce': 31570}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6813', 'nonce': 31570}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6815', 'nonce': 31570}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6814', 'nonce': 31570}]}, 'public_addr': '172.21.15.96:6812/31570', 'cluster_addr': '172.21.15.96:6813/31570', 'heartbeat_back_addr': '172.21.15.96:6815/31570', 'heartbeat_front_addr': '172.21.15.96:6814/31570', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:11:18.244439+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:10:16.729479+0000', 'dead_epoch': 191}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:11:17.155508+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:11:19.498993+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:11:18.591484+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:11:18.086077+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:11:17.762417+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:11:17.114874+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [{'pgid': '1.15', 'osds': [7, 5, 2]}, {'pgid': '1.16', 'osds': [3, 0, 6]}, {'pgid': '1.1a', 'osds': [7, 4, 2]}, {'pgid': '1.1c', 'osds': [7, 6, 0]}, {'pgid': '1.1d', 'osds': [3, 4, 7]}, {'pgid': '5.7', 'osds': [0, 4, 5]}], 'primary_temp': [], 'blacklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}} 2020-08-07T15:11:40.707 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:11:40.728 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:11:40.728 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:11:40.728 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:11:40.741 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:2020-08-07T15:11:40.216823+0000 min lat: 0.000974592 max lat: 1.93766 avg lat: 0.0111343 2020-08-07T15:11:40.741 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-08-07T15:11:40.741 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 80 16 114940 114924 11.222 11.8828 0.00929389 0.0111343 2020-08-07T15:11:40.741 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:11:40.745 INFO:teuthology.orchestra.run.smithi074.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-08-07T15:11:40.745 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: problem getting command descriptions from osd.1 2020-08-07T15:11:40.745 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:11:40.745 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:11:40.745 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:11:40.746 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:11:40.746 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:11:40.746 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:11:40.746 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-08-07T15:11:40.747 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-08-07T15:11:40.768 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:11:40.769 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:40.707+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:40.835 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:40.828+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:40.924 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:40.913+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:40.938 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:40.938 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:40.959 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:11:41.034 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:41.026+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:41.140 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:41.131+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:41.166 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:41.166 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:41.187 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:11:41.229 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 81 16 116560 116544 11.2397 12.6562 0.00607078 0.0111179 2020-08-07T15:11:41.235 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:41.231+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:41.248 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:11:41.272 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2020-08-07T15:11:40.673108+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":898613248,"num_objects":13715,"num_object_clones":0,"num_object_copies":41145,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3443,"num_objects_misplaced":898,"num_objects_unfound":0,"num_objects_dirty":13715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109694,"num_write_kb":877552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1498,"num_bytes_recovered":98172928,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":149,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":186,"ondisk_log_size":186,"up":146,"acting":152,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":10942260,"kb_used_data":2553588,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":744032460,"statfs":{"total":773094113280,"available":761889239040,"internally_reserved":8589934592,"allocated":2614874112,"data_stored":2607541026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":51363840,"num_objects":784,"num_object_clones":0,"num_object_copies":2352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":196,"num_objects_misplaced":896,"num_objects_unfound":0,"num_objects_dirty":784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6270,"num_write_kb":50160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.263064"},"pg_stats":[{"pgid":"1.1e","version":"16'2","reported_seq":"62","reported_epoch":"213","state":"active+clean","last_fresh":"2020-08-07T15:11:32.269160+0000","last_change":"2020-08-07T15:10:51.560726+0000","last_active":"2020-08-07T15:11:32.269160+0000","last_peered":"2020-08-07T15:11:32.269160+0000","last_clean":"2020-08-07T15:11:32.269160+0000","last_became_active":"2020-08-07T15:10:51.559894+0000","last_became_peered":"2020-08-07T15:10:51.559894+0000","last_unstale":"2020-08-07T15:11:32.269160+0000","last_undegraded":"2020-08-07T15:11:32.269160+0000","last_fullsized":"2020-08-07T15:11:32.269160+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:29.055048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:29.055048+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1d","version":"75'3","reported_seq":"108","reported_epoch":"213","state":"active+clean+remapped","last_fresh":"2020-08-07T15:11:32.265858+0000","last_change":"2020-08-07T15:10:52.203763+0000","last_active":"2020-08-07T15:11:32.265858+0000","last_peered":"2020-08-07T15:11:32.265858+0000","last_clean":"2020-08-07T15:11:32.265858+0000","last_became_active":"2020-08-07T15:10:52.203050+0000","last_became_peered":"2020-08-07T15:10:52.203050+0000","last_unstale":"2020-08-07T15:11:32.265858+0000","last_undegraded":"2020-08-07T15:11:32.265858+0000","last_fullsized":"2020-08-07T15:11:32.265858+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:10:10.962480+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.962480+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":149,"num_omap_keys":1,"num_objects_repaired":0},"up":[3,4],"acting":[3,4,7],"avail_no_missing":["3","4","7"],"object_location_counts":[{"shards":"3,4,7","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1c","version":"16'2","reported_seq":"140","reported_epoch":"213","state":"active+clean+remapped","last_fresh":"2020-08-07T15:11:32.204165+0000","last_change":"2020-08-07T15:10:52.413792+0000","last_active":"2020-08-07T15:11:32.204165+0000","last_peered":"2020-08-07T15:11:32.204165+0000","last_clean":"2020-08-07T15:11:32.204165+0000","last_became_active":"2020-08-07T15:10:52.413159+0000","last_became_peered":"2020-08-07T15:10:52.413159+0000","last_unstale":"2020-08-07T15:11:32.204165+0000","last_undegraded":"2020-08-07T15:11:32.204165+0000","last_fullsized":"2020-08-07T15:11:32.204165+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:36.687314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:36.687314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6,0],"avail_no_missing":["7","0","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1b","version":"16'2","reported_seq":"186","reported_epoch":"213","state":"active+clean","last_fresh":"2020-08-07T15:11:32.205530+0000","last_change":"2020-08-07T15:10:57.797029+0000","last_active":"2020-08-07T15:11:32.205530+0000","last_peered":"2020-08-07T15:11:32.205530+0000","last_clean":"2020-08-07T15:11:32.205530+0000","last_became_active":"2020-08-07T15:10:51.557392+0000","last_became_peered":"2020-08-07T15:10:51.557392+0000","last_unstale":"2020-08-07T15:11:32.205530+0000","last_undegraded":"2020-08-07T15:11:32.205530+0000","last_fullsized":"2020-08-07T15:11:32.205530+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:57.796946+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:57.796946+0000","last_clean_scrub_stamp":"2020-08-07T15:10:57.796946+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1a","version":"16'2","reported_seq":"40","reported_epoch":"213","state":"active+clean+remapped","last_fresh":"2020-08-07T15:11:32.204081+0000","last_change":"2020-08-07T15:11:11.780113+0000","last_active":"2020-08-07T15:11:32.204081+0000","last_peered":"2020-08-07T15:11:32.204081+0000","last_clean":"2020-08-07T15:11:32.204081+0000","last_became_active":"2020-08-07T15:10:52.339672+0000","last_became_peered":"2020-08-07T15:10:52.339672+0000","last_unstale":"2020-08-07T15:11:32.204081+0000","last_undegraded":"2020-08-07T15:11:32.204081+0000","last_fullsized":"2020-08-07T15:11:32.204081+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:11.779828+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:11:11.779828+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4,2],"avail_no_missing":["7","2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.19","version":"16'2","reported_seq":"108","reported_epoch":"213","state":"active+clean","last_fresh":"2020-08-07T15:11:32.268863+0000","last_change":"2020-08-07T15:11:08.710172+0000","last_active":"2020-08-07T15:11:32.268863+0000","last_peered":"2020-08-07T15:11:32.268863+0000","last_clean":"2020-08-07T15:11:32.268863+0000","last_became_active":"2020-08-07T15:10:51.559752+0000","last_became_peered":"2020-08-07T15:10:51.559752+0000","last_unstale":"2020-08-07T15:11:32.268863+0000","last_undegraded":"2020-08-07T15:11:32.268863+0000","last_fullsized":"2020-08-07T15:11:32.268863+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:08.710086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:08.710086+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.18","version":"16'2","reported_seq":"211","reported_epoch":"213","state":"active+clean","last_fresh":"2020-08-07T15:11:32.205147+0000","last_change":"2020-08-07T15:11:06.955336+0000","last_active":"2020-08-07T15:11:32.205147+0000","last_peered":"2020-08-07T15:11:32.205147+0000","last_clean":"2020-08-07T15:11:32.205147+0000","last_became_active":"2020-08-07T15:10:51.557665+0000","last_became_peered":"2020-08-07T15:10:51.557665+0000","last_unstale":"2020-08-07T15:11:32.205147+0000","last_undegraded":"2020-08-07T15:11:32.205147+0000","last_fullsized":"2020-08-07T15:11:32.205147+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:06.955262+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:06.955262+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.17","version":"16'2","reported_seq":"39","reported_epoch":"213","state":"active+clean","last_fresh":"2020-08-07T15:11:32.205849+0000","last_change":"2020-08-07T15:11:08.916219+0000","last_active":"2020-08-07T15:11:32.205849+0000","last_peered":"2020-08-07T15:11:32.205849+0000","last_clean":"2020-08-07T15:11:32.205849+0000","last_became_active":"2020-08-07T15:10:51.561769+0000","last_became_peered":"2020-08-07T15:10:51.561769+0000","last_unstale":"2020-08-07T15:11:32.205849+0000","last_undegraded":"2020-08-07T15:11:32.205849+0000","last_fullsized":"2020-08-07T15:11:32.205849+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:08.916169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:08.916169+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.16","version":"16'2","reported_seq":"185","reported_epoch":"213","state":"active+clean+remapped","last_fresh":"2020-08-07T15:11:32.265883+0000","last_change":"2020-08-07T15:10:52.208583+0000","last_active":"2020-08-07T15:11:32.265883+0000","last_peered":"2020-08-07T15:11:32.265883+0000","last_clean":"2020-08-07T15:11:32.265883+0000","last_became_active":"2020-08-07T15:10:52.207790+0000","last_became_peered":"2020-08-07T15:10:52.207790+0000","last_unstale":"2020-08-07T15:11:32.265883+0000","last_undegraded":"2020-08-07T15:11:32.265883+0000","last_fullsized":"2020-08-07T15:11:32.265883+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:07.302204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:07.302204+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0,6],"avail_no_missing":["3","0","6"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.15","version":"75'3","reported_seq":"40","reported_epoch":"213","state":"active+clean+remapped","last_fresh":"2020-08-07T15:11:32.204357+0000","last_change":"2020-08-07T15:11:20.562885+0000","last_active":"2020-08-07T15:11:32.204357+0000","last_peered":"2020-08-07T15:11:32.204357+0000","last_clean":"2020-08-07T15:11:32.204357+0000","last_became_active":"2020-08-07T15:10:52.207820+0000","last_became_peered":"2020-08-07T15:10:52.207820+0000","last_unstale":"2020-08-07T15:11:32.204357+0000","last_undegraded":"2020-08-07T15:11:32.204357+0000","last_fullsized":"2020-08-07T15:11:32.204357+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:11:20.562710+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:11:20.562710+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5,2],"avail_no_missing":["7","2","5"],"object_location_counts":[{"shards":"2,5,7","objects":1}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.14","version":"16'2","reported_seq":"146","reported_epoch":"213","state":"active+clean","last_fresh":"2020-08-07T15:11:32.207031+0000","last_change":"2020-08-07T15:10:10.644555+0000","last_active":"2020-08-07T15:11:32.207031+0000","last_peered":"2020-08-07T15:11:32.207031+0000","last_clean":"2020-08-07T15:11:32.207031+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:11:32.207031+0000","last_undegraded":"2020-08-07T15:11:32.207031+0000","last_fullsized":"2020-08-07T15:11:32.207031+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:10.644460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.644460+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"147","reported_epoch":"213","state":"active+undersized","last_fresh":"2020-08-07T15:11:32.268931+0000","last_change":"2020-08-07T15:10:51.552550+0000","last_active":"2020-08-07T15:11:32.268931+0000","last_peered":"2020-08-07T15:11:32.268931+0000","last_clean":"2020-08-07T15:09:25.422675+0000","last_became_active":"2020-08-07T15:10:51.552550+0000","last_became_peered":"2020-08-07T15:10:51.552550+0000","last_unstale":"2020-08-07T15:11:32.268931+0000","last_undegraded":"2020-08-07T15:11:32.268931+0000","last_fullsized":"2020-08-07T15:10:51.544042+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:07.295863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:07.295863+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":["3","4"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"152","reported_epoch":"213","state":"active+undersized","last_fresh":"2020-08-07T15:11:32.205477+0000","last_change":"2020-08-07T15:10:51.559962+0000","last_active":"2020-08-07T15:11:32.205477+0000","last_peered":"2020-08-07T15:11:32.205477+0000","last_clean":"2020-08-07T15:10:49.526673+0000","last_became_active":"2020-08-07T15:10:51.559962+0000","last_became_peered":"2020-08-07T15:10:51.559962+0000","last_unstale":"2020-08-07T15:11:32.205477+0000","last_undegraded":"2020-08-07T15:11:32.205477+0000","last_fullsized":"2020-08-07T15:10:51.549931+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:28.734193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:28.734193+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"259","reported_epoch":"213","state":"active+clean","last_fresh":"2020-08-07T15:11:32.205377+0000","last_change":"2020-08-07T15:10:24.690028+0000","last_active":"2020-08-07T15:11:32.205377+0000","last_peered":"2020-08-07T15:11:32.205377+0000","last_clean":"2020-08-07T15:11:32.205377+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:11:32.205377+0000","last_undegraded":"2020-08-07T15:11:32.205377+0000","last_fullsized":"2020-08-07T15:11:32.205377+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:24.689950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:24.689950+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"253","reported_epoch":"213","state":"active+clean","last_fresh":"2020-08-07T15:11:32.205345+0000","last_change":"2020-08-07T15:10:26.631030+0000","last_active":"2020-08-07T15:11:32.205345+0000","last_peered":"2020-08-07T15:11:32.205345+0000","last_clean":"2020-08-07T15:11:32.205345+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:11:32.205345+0000","last_undegraded":"2020-08-07T15:11:32.205345+0000","last_fullsized":"2020-08-07T15:11:32.205345+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:26.630945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:26.630945+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.0","version":"214'6880","reported_seq":"6924","reported_epoch":"214","state":"active+undersized+degraded","last_fresh":"2020-08-07T15:11:36.747195+0000","last_change":"2020-08-07T15:10:51.552627+0000","last_active":"2020-08-07T15:11:36.747195+0000","last_peered":"2020-08-07T15:11:36.747195+0000","last_clean":"2020-08-07T15:10:50.162105+0000","last_became_active":"2020-08-07T15:10:51.552627+0000","last_became_peered":"2020-08-07T15:10:51.552627+0000","last_unstale":"2020-08-07T15:11:36.747195+0000","last_undegraded":"2020-08-07T15:10:51.544354+0000","last_fullsized":"2020-08-07T15:10:51.544200+0000","mapping_epoch":191,"log_start":"214'6869","ondisk_log_start":"214'6869","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56360960,"num_objects":860,"num_object_clones":0,"num_object_copies":2580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6880,"num_write_kb":55040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":["3","4"],"object_location_counts":[{"shards":"3,4","objects":860}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"214","reported_epoch":"214","state":"active+clean","last_fresh":"2020-08-07T15:11:33.420832+0000","last_change":"2020-08-07T15:10:57.384596+0000","last_active":"2020-08-07T15:11:33.420832+0000","last_peered":"2020-08-07T15:11:33.420832+0000","last_clean":"2020-08-07T15:11:33.420832+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:11:33.420832+0000","last_undegraded":"2020-08-07T15:11:33.420832+0000","last_fullsized":"2020-08-07T15:11:33.420832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:57.384515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:10:57.384515+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"129","reported_epoch":"213","state":"active+clean","last_fresh":"2020-08-07T15:11:32.269116+0000","last_change":"2020-08-07T15:10:52.653421+0000","last_active":"2020-08-07T15:11:32.269116+0000","last_peered":"2020-08-07T15:11:32.269116+0000","last_clean":"2020-08-07T15:11:32.269116+0000","last_became_active":"2020-08-07T15:09:30.283176+0000","last_became_peered":"2020-08-07T15:09:30.283176+0000","last_unstale":"2020-08-07T15:11:32.269116+0000","last_undegraded":"2020-08-07T15:11:32.269116+0000","last_fullsized":"2020-08-07T15:11:32.269116+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:52.653356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:52.653356+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"214'6824","reported_seq":"6882","reported_epoch":"214","state":"active+clean","last_fresh":"2020-08-07T15:11:36.737568+0000","last_change":"2020-08-07T15:10:16.259807+0000","last_active":"2020-08-07T15:11:36.737568+0000","last_peered":"2020-08-07T15:11:36.737568+0000","last_clean":"2020-08-07T15:11:36.737568+0000","last_became_active":"2020-08-07T15:10:16.259365+0000","last_became_peered":"2020-08-07T15:10:16.259365+0000","last_unstale":"2020-08-07T15:11:36.737568+0000","last_undegraded":"2020-08-07T15:11:36.737568+0000","last_fullsized":"2020-08-07T15:11:36.737568+0000","mapping_epoch":164,"log_start":"214'6818","ondisk_log_start":"214'6818","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":55902208,"num_objects":853,"num_object_clones":0,"num_object_copies":2559,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6824,"num_write_kb":54592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"32","reported_epoch":"214","state":"active+undersized","last_fresh":"2020-08-07T15:11:33.420236+0000","last_change":"2020-08-07T15:10:51.558681+0000","last_active":"2020-08-07T15:11:33.420236+0000","last_peered":"2020-08-07T15:11:33.420236+0000","last_clean":"2020-08-07T15:06:22.062496+0000","last_became_active":"2020-08-07T15:10:51.558681+0000","last_became_peered":"2020-08-07T15:10:51.558681+0000","last_unstale":"2020-08-07T15:11:33.420236+0000","last_undegraded":"2020-08-07T15:11:33.420236+0000","last_fullsized":"2020-08-07T15:10:51.552830+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:25.463628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:10:25.463628+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"183","reported_epoch":"213","state":"active+clean","last_fresh":"2020-08-07T15:11:32.206379+0000","last_change":"2020-08-07T15:10:52.485772+0000","last_active":"2020-08-07T15:11:32.206379+0000","last_peered":"2020-08-07T15:11:32.206379+0000","last_clean":"2020-08-07T15:11:32.206379+0000","last_became_active":"2020-08-07T15:10:46.501793+0000","last_became_peered":"2020-08-07T15:10:46.501793+0000","last_unstale":"2020-08-07T15:11:32.206379+0000","last_undegraded":"2020-08-07T15:11:32.206379+0000","last_fullsized":"2020-08-07T15:11:32.206379+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:10:52.485722+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:10:52.485722+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"215'7104","reported_seq":"7159","reported_epoch":"215","state":"active+clean","last_fresh":"2020-08-07T15:11:37.739171+0000","last_change":"2020-08-07T15:10:16.448360+0000","last_active":"2020-08-07T15:11:37.739171+0000","last_peered":"2020-08-07T15:11:37.739171+0000","last_clean":"2020-08-07T15:11:37.739171+0000","last_became_active":"2020-08-07T15:10:16.447826+0000","last_became_peered":"2020-08-07T15:10:16.447826+0000","last_unstale":"2020-08-07T15:11:37.739171+0000","last_undegraded":"2020-08-07T15:11:37.739171+0000","last_fullsized":"2020-08-07T15:11:37.739171+0000","mapping_epoch":164,"log_start":"215'7096","ondisk_log_start":"215'7096","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":58195968,"num_objects":888,"num_object_clones":0,"num_object_copies":2664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7104,"num_write_kb":56832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"211","reported_epoch":"213","state":"active+clean","last_fresh":"2020-08-07T15:11:32.204960+0000","last_change":"2020-08-07T15:09:54.908283+0000","last_active":"2020-08-07T15:11:32.204960+0000","last_peered":"2020-08-07T15:11:32.204960+0000","last_clean":"2020-08-07T15:11:32.204960+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:11:32.204960+0000","last_undegraded":"2020-08-07T15:11:32.204960+0000","last_fullsized":"2020-08-07T15:11:32.204960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:54.908197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:54.908197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"262","reported_epoch":"214","state":"active+clean","last_fresh":"2020-08-07T15:11:33.420408+0000","last_change":"2020-08-07T15:10:53.292841+0000","last_active":"2020-08-07T15:11:33.420408+0000","last_peered":"2020-08-07T15:11:33.420408+0000","last_clean":"2020-08-07T15:11:33.420408+0000","last_became_active":"2020-08-07T15:10:46.516484+0000","last_became_peered":"2020-08-07T15:10:46.516484+0000","last_unstale":"2020-08-07T15:11:33.420408+0000","last_undegraded":"2020-08-07T15:11:33.420408+0000","last_fullsized":"2020-08-07T15:11:33.420408+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:53.292768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:53.292768+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"214'7166","reported_seq":"8808","reported_epoch":"214","state":"active+clean+remapped","last_fresh":"2020-08-07T15:11:36.624927+0000","last_change":"2020-08-07T15:11:32.227230+0000","last_active":"2020-08-07T15:11:36.624927+0000","last_peered":"2020-08-07T15:11:36.624927+0000","last_clean":"2020-08-07T15:11:36.624927+0000","last_became_active":"2020-08-07T15:11:32.212526+0000","last_became_peered":"2020-08-07T15:11:32.212526+0000","last_unstale":"2020-08-07T15:11:36.624927+0000","last_undegraded":"2020-08-07T15:11:36.624927+0000","last_fullsized":"2020-08-07T15:11:36.624927+0000","mapping_epoch":212,"log_start":"214'7160","ondisk_log_start":"214'7160","created":164,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":58703872,"num_objects":896,"num_object_clones":0,"num_object_copies":2688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":896,"num_objects_unfound":0,"num_objects_dirty":896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7166,"num_write_kb":57328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":786,"num_bytes_recovered":51511296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4,5],"avail_no_missing":["0","4","5"],"object_location_counts":[{"shards":"0,4,5","objects":896}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"217","reported_epoch":"213","state":"active+undersized","last_fresh":"2020-08-07T15:11:32.204065+0000","last_change":"2020-08-07T15:10:51.559204+0000","last_active":"2020-08-07T15:11:32.204065+0000","last_peered":"2020-08-07T15:11:32.204065+0000","last_clean":"2020-08-07T15:10:49.542912+0000","last_became_active":"2020-08-07T15:10:51.559204+0000","last_became_peered":"2020-08-07T15:10:51.559204+0000","last_unstale":"2020-08-07T15:11:32.204065+0000","last_undegraded":"2020-08-07T15:11:32.204065+0000","last_fullsized":"2020-08-07T15:10:51.546815+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:51.036224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:51.036224+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":["7","0"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"222","reported_epoch":"214","state":"active+undersized","last_fresh":"2020-08-07T15:11:33.419888+0000","last_change":"2020-08-07T15:10:51.557999+0000","last_active":"2020-08-07T15:11:33.419888+0000","last_peered":"2020-08-07T15:11:33.419888+0000","last_clean":"2020-08-07T15:10:49.532877+0000","last_became_active":"2020-08-07T15:10:51.557999+0000","last_became_peered":"2020-08-07T15:10:51.557999+0000","last_unstale":"2020-08-07T15:11:33.419888+0000","last_undegraded":"2020-08-07T15:11:33.419888+0000","last_fullsized":"2020-08-07T15:10:51.549893+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:23.400672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:23.400672+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"214'6664","reported_seq":"6722","reported_epoch":"214","state":"active+clean","last_fresh":"2020-08-07T15:11:36.582808+0000","last_change":"2020-08-07T15:10:16.255436+0000","last_active":"2020-08-07T15:11:36.582808+0000","last_peered":"2020-08-07T15:11:36.582808+0000","last_clean":"2020-08-07T15:11:36.582808+0000","last_became_active":"2020-08-07T15:10:16.254967+0000","last_became_peered":"2020-08-07T15:10:16.254967+0000","last_unstale":"2020-08-07T15:11:36.582808+0000","last_undegraded":"2020-08-07T15:11:36.582808+0000","last_fullsized":"2020-08-07T15:11:36.582808+0000","mapping_epoch":164,"log_start":"214'6656","ondisk_log_start":"214'6656","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":54591488,"num_objects":833,"num_object_clones":0,"num_object_copies":2499,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6664,"num_write_kb":53312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"243","reported_epoch":"213","state":"active+clean","last_fresh":"2020-08-07T15:11:32.203920+0000","last_change":"2020-08-07T15:09:55.879867+0000","last_active":"2020-08-07T15:11:32.203920+0000","last_peered":"2020-08-07T15:11:32.203920+0000","last_clean":"2020-08-07T15:11:32.203920+0000","last_became_active":"2020-08-07T15:08:36.627426+0000","last_became_peered":"2020-08-07T15:08:36.627426+0000","last_unstale":"2020-08-07T15:11:32.203920+0000","last_undegraded":"2020-08-07T15:11:32.203920+0000","last_fullsized":"2020-08-07T15:11:32.203920+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:09:55.879785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:09:55.879785+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"211","reported_epoch":"213","state":"active+clean","last_fresh":"2020-08-07T15:11:32.205831+0000","last_change":"2020-08-07T15:10:08.656952+0000","last_active":"2020-08-07T15:11:32.205831+0000","last_peered":"2020-08-07T15:11:32.205831+0000","last_clean":"2020-08-07T15:11:32.205831+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:11:32.205831+0000","last_undegraded":"2020-08-07T15:11:32.205831+0000","last_fullsized":"2020-08-07T15:11:32.205831+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:08.656855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:10:08.656855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"214'6600","reported_seq":"6658","reported_epoch":"214","state":"active+clean","last_fresh":"2020-08-07T15:11:36.675901+0000","last_change":"2020-08-07T15:10:16.263356+0000","last_active":"2020-08-07T15:11:36.675901+0000","last_peered":"2020-08-07T15:11:36.675901+0000","last_clean":"2020-08-07T15:11:36.675901+0000","last_became_active":"2020-08-07T15:10:16.262971+0000","last_became_peered":"2020-08-07T15:10:16.262971+0000","last_unstale":"2020-08-07T15:11:36.675901+0000","last_undegraded":"2020-08-07T15:11:36.675901+0000","last_fullsized":"2020-08-07T15:11:36.675901+0000","mapping_epoch":164,"log_start":"214'6585","ondisk_log_start":"214'6585","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":54067200,"num_objects":825,"num_object_clones":0,"num_object_copies":2475,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6600,"num_write_kb":52800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"232","reported_epoch":"213","state":"active+clean","last_fresh":"2020-08-07T15:11:32.206289+0000","last_change":"2020-08-07T15:10:24.576099+0000","last_active":"2020-08-07T15:11:32.206289+0000","last_peered":"2020-08-07T15:11:32.206289+0000","last_clean":"2020-08-07T15:11:32.206289+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:11:32.206289+0000","last_undegraded":"2020-08-07T15:11:32.206289+0000","last_fullsized":"2020-08-07T15:11:32.206289+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:24.576023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:10:24.576023+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"259","reported_epoch":"213","state":"active+clean","last_fresh":"2020-08-07T15:11:32.204290+0000","last_change":"2020-08-07T15:10:11.912614+0000","last_active":"2020-08-07T15:11:32.204290+0000","last_peered":"2020-08-07T15:11:32.204290+0000","last_clean":"2020-08-07T15:11:32.204290+0000","last_became_active":"2020-08-07T15:08:36.628365+0000","last_became_peered":"2020-08-07T15:08:36.628365+0000","last_unstale":"2020-08-07T15:11:32.204290+0000","last_undegraded":"2020-08-07T15:11:32.204290+0000","last_fullsized":"2020-08-07T15:11:32.204290+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:11.912550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:11.912550+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.5","version":"214'6536","reported_seq":"6596","reported_epoch":"214","state":"active+undersized+degraded","last_fresh":"2020-08-07T15:11:36.651384+0000","last_change":"2020-08-07T15:10:51.555872+0000","last_active":"2020-08-07T15:11:36.651384+0000","last_peered":"2020-08-07T15:11:36.651384+0000","last_clean":"2020-08-07T15:10:50.059205+0000","last_became_active":"2020-08-07T15:10:51.555872+0000","last_became_peered":"2020-08-07T15:10:51.555872+0000","last_unstale":"2020-08-07T15:11:36.651384+0000","last_undegraded":"2020-08-07T15:10:51.546355+0000","last_fullsized":"2020-08-07T15:10:51.546219+0000","mapping_epoch":191,"log_start":"214'6531","ondisk_log_start":"214'6531","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53542912,"num_objects":817,"num_object_clones":0,"num_object_copies":2451,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":817,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6536,"num_write_kb":52288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[{"shards":"0,4","objects":817}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"217","reported_epoch":"213","state":"active+undersized","last_fresh":"2020-08-07T15:11:32.205789+0000","last_change":"2020-08-07T15:10:51.560479+0000","last_active":"2020-08-07T15:11:32.205789+0000","last_peered":"2020-08-07T15:11:32.205789+0000","last_clean":"2020-08-07T15:10:49.526877+0000","last_became_active":"2020-08-07T15:10:51.560479+0000","last_became_peered":"2020-08-07T15:10:51.560479+0000","last_unstale":"2020-08-07T15:11:32.205789+0000","last_undegraded":"2020-08-07T15:11:32.205789+0000","last_fullsized":"2020-08-07T15:10:51.550565+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:52.637791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:09:52.637791+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"209","reported_epoch":"213","state":"active+clean","last_fresh":"2020-08-07T15:11:32.205501+0000","last_change":"2020-08-07T15:10:11.720217+0000","last_active":"2020-08-07T15:11:32.205501+0000","last_peered":"2020-08-07T15:11:32.205501+0000","last_clean":"2020-08-07T15:11:32.205501+0000","last_became_active":"2020-08-07T15:08:36.618164+0000","last_became_peered":"2020-08-07T15:08:36.618164+0000","last_unstale":"2020-08-07T15:11:32.205501+0000","last_undegraded":"2020-08-07T15:11:32.205501+0000","last_fullsized":"2020-08-07T15:11:32.205501+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:11.720143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:11.720143+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"215'7144","reported_seq":"7203","reported_epoch":"215","state":"active+clean","last_fresh":"2020-08-07T15:11:37.631924+0000","last_change":"2020-08-07T15:10:16.443153+0000","last_active":"2020-08-07T15:11:37.631924+0000","last_peered":"2020-08-07T15:11:37.631924+0000","last_clean":"2020-08-07T15:11:37.631924+0000","last_became_active":"2020-08-07T15:10:16.442674+0000","last_became_peered":"2020-08-07T15:10:16.442674+0000","last_unstale":"2020-08-07T15:11:37.631924+0000","last_undegraded":"2020-08-07T15:11:37.631924+0000","last_fullsized":"2020-08-07T15:11:37.631924+0000","mapping_epoch":164,"log_start":"215'7136","ondisk_log_start":"215'7136","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":58523648,"num_objects":893,"num_object_clones":0,"num_object_copies":2679,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7144,"num_write_kb":57152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"214","reported_epoch":"213","state":"active+undersized","last_fresh":"2020-08-07T15:11:32.204457+0000","last_change":"2020-08-07T15:10:51.561524+0000","last_active":"2020-08-07T15:11:32.204457+0000","last_peered":"2020-08-07T15:11:32.204457+0000","last_clean":"2020-08-07T15:10:49.540461+0000","last_became_active":"2020-08-07T15:10:51.561524+0000","last_became_peered":"2020-08-07T15:10:51.561524+0000","last_unstale":"2020-08-07T15:11:32.204457+0000","last_undegraded":"2020-08-07T15:11:32.204457+0000","last_fullsized":"2020-08-07T15:10:51.549397+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:09:37.017307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:09:37.017307+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":["7","0"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"296","reported_epoch":"214","state":"active+clean","last_fresh":"2020-08-07T15:11:33.420964+0000","last_change":"2020-08-07T15:10:50.296729+0000","last_active":"2020-08-07T15:11:33.420964+0000","last_peered":"2020-08-07T15:11:33.420964+0000","last_clean":"2020-08-07T15:11:33.420964+0000","last_became_active":"2020-08-07T15:10:46.496544+0000","last_became_peered":"2020-08-07T15:10:46.496544+0000","last_unstale":"2020-08-07T15:11:33.420964+0000","last_undegraded":"2020-08-07T15:11:33.420964+0000","last_fullsized":"2020-08-07T15:11:33.420964+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:50.296646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:50.296646+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.e","version":"214'6752","reported_seq":"6810","reported_epoch":"214","state":"active+clean","last_fresh":"2020-08-07T15:11:36.705211+0000","last_change":"2020-08-07T15:10:16.448334+0000","last_active":"2020-08-07T15:11:36.705211+0000","last_peered":"2020-08-07T15:11:36.705211+0000","last_clean":"2020-08-07T15:11:36.705211+0000","last_became_active":"2020-08-07T15:10:16.447541+0000","last_became_peered":"2020-08-07T15:10:16.447541+0000","last_unstale":"2020-08-07T15:11:36.705211+0000","last_undegraded":"2020-08-07T15:11:36.705211+0000","last_fullsized":"2020-08-07T15:11:36.705211+0000","mapping_epoch":164,"log_start":"214'6744","ondisk_log_start":"214'6744","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":55312384,"num_objects":844,"num_object_clones":0,"num_object_copies":2532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6752,"num_write_kb":54016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"198","reported_epoch":"213","state":"active+clean","last_fresh":"2020-08-07T15:11:32.205461+0000","last_change":"2020-08-07T15:10:51.644951+0000","last_active":"2020-08-07T15:11:32.205461+0000","last_peered":"2020-08-07T15:11:32.205461+0000","last_clean":"2020-08-07T15:11:32.205461+0000","last_became_active":"2020-08-07T15:10:46.498230+0000","last_became_peered":"2020-08-07T15:10:46.498230+0000","last_unstale":"2020-08-07T15:11:32.205461+0000","last_undegraded":"2020-08-07T15:11:32.205461+0000","last_fullsized":"2020-08-07T15:11:32.205461+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:51.644875+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:10:51.644875+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"214'7248","reported_seq":"7304","reported_epoch":"214","state":"active+undersized+degraded","last_fresh":"2020-08-07T15:11:36.533929+0000","last_change":"2020-08-07T15:10:51.550117+0000","last_active":"2020-08-07T15:11:36.533929+0000","last_peered":"2020-08-07T15:11:36.533929+0000","last_clean":"2020-08-07T15:10:50.051777+0000","last_became_active":"2020-08-07T15:10:51.550117+0000","last_became_peered":"2020-08-07T15:10:51.550117+0000","last_unstale":"2020-08-07T15:11:36.533929+0000","last_undegraded":"2020-08-07T15:10:51.543706+0000","last_fullsized":"2020-08-07T15:10:51.543546+0000","mapping_epoch":191,"log_start":"214'7243","ondisk_log_start":"214'7243","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":59375616,"num_objects":906,"num_object_clones":0,"num_object_copies":2718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":906,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7248,"num_write_kb":57984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[{"shards":"0,4","objects":906}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"35","reported_epoch":"213","state":"active+clean","last_fresh":"2020-08-07T15:11:32.205650+0000","last_change":"2020-08-07T15:11:23.233126+0000","last_active":"2020-08-07T15:11:32.205650+0000","last_peered":"2020-08-07T15:11:32.205650+0000","last_clean":"2020-08-07T15:11:32.205650+0000","last_became_active":"2020-08-07T15:11:23.232406+0000","last_became_peered":"2020-08-07T15:11:23.232406+0000","last_unstale":"2020-08-07T15:11:32.205650+0000","last_undegraded":"2020-08-07T15:11:32.205650+0000","last_fullsized":"2020-08-07T15:11:32.205650+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:55.673165+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:10:55.673165+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.d","version":"214'6808","reported_seq":"6865","reported_epoch":"214","state":"active+clean","last_fresh":"2020-08-07T15:11:36.695297+0000","last_change":"2020-08-07T15:10:16.263289+0000","last_active":"2020-08-07T15:11:36.695297+0000","last_peered":"2020-08-07T15:11:36.695297+0000","last_clean":"2020-08-07T15:11:36.695297+0000","last_became_active":"2020-08-07T15:10:16.262869+0000","last_became_peered":"2020-08-07T15:10:16.262869+0000","last_unstale":"2020-08-07T15:11:36.695297+0000","last_undegraded":"2020-08-07T15:11:36.695297+0000","last_fullsized":"2020-08-07T15:11:36.695297+0000","mapping_epoch":164,"log_start":"214'6805","ondisk_log_start":"214'6805","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":55771136,"num_objects":851,"num_object_clones":0,"num_object_copies":2553,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6808,"num_write_kb":54464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"262","reported_epoch":"213","state":"active+clean","last_fresh":"2020-08-07T15:11:32.204440+0000","last_change":"2020-08-07T15:10:58.029660+0000","last_active":"2020-08-07T15:11:32.204440+0000","last_peered":"2020-08-07T15:11:32.204440+0000","last_clean":"2020-08-07T15:11:32.204440+0000","last_became_active":"2020-08-07T15:10:46.499635+0000","last_became_peered":"2020-08-07T15:10:46.499635+0000","last_unstale":"2020-08-07T15:11:32.204440+0000","last_undegraded":"2020-08-07T15:11:32.204440+0000","last_fullsized":"2020-08-07T15:11:32.204440+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:58.029609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:58.029609+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"214'7040","reported_seq":"7097","reported_epoch":"214","state":"active+clean","last_fresh":"2020-08-07T15:11:36.664480+0000","last_change":"2020-08-07T15:10:16.256000+0000","last_active":"2020-08-07T15:11:36.664480+0000","last_peered":"2020-08-07T15:11:36.664480+0000","last_clean":"2020-08-07T15:11:36.664480+0000","last_became_active":"2020-08-07T15:10:16.255356+0000","last_became_peered":"2020-08-07T15:10:16.255356+0000","last_unstale":"2020-08-07T15:11:36.664480+0000","last_undegraded":"2020-08-07T15:11:36.664480+0000","last_fullsized":"2020-08-07T15:11:36.664480+0000","mapping_epoch":164,"log_start":"214'7034","ondisk_log_start":"214'7034","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57671680,"num_objects":880,"num_object_clones":0,"num_object_copies":2640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7040,"num_write_kb":56320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"220","reported_epoch":"213","state":"active+undersized","last_fresh":"2020-08-07T15:11:32.204727+0000","last_change":"2020-08-07T15:10:51.553978+0000","last_active":"2020-08-07T15:11:32.204727+0000","last_peered":"2020-08-07T15:11:32.204727+0000","last_clean":"2020-08-07T15:10:49.540110+0000","last_became_active":"2020-08-07T15:10:51.553978+0000","last_became_peered":"2020-08-07T15:10:51.553978+0000","last_unstale":"2020-08-07T15:11:32.204727+0000","last_undegraded":"2020-08-07T15:11:32.204727+0000","last_fullsized":"2020-08-07T15:10:51.546352+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:10:10.962480+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.962480+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"214'6880","reported_seq":"6928","reported_epoch":"214","state":"active+undersized+degraded","last_fresh":"2020-08-07T15:11:36.561963+0000","last_change":"2020-08-07T15:10:51.550411+0000","last_active":"2020-08-07T15:11:36.561963+0000","last_peered":"2020-08-07T15:11:36.561963+0000","last_clean":"2020-08-07T15:10:50.163110+0000","last_became_active":"2020-08-07T15:10:51.550411+0000","last_became_peered":"2020-08-07T15:10:51.550411+0000","last_unstale":"2020-08-07T15:11:36.561963+0000","last_undegraded":"2020-08-07T15:10:51.545024+0000","last_fullsized":"2020-08-07T15:10:51.544891+0000","mapping_epoch":191,"log_start":"214'6873","ondisk_log_start":"214'6873","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56360960,"num_objects":860,"num_object_clones":0,"num_object_copies":2580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6880,"num_write_kb":55040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":860}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"184","reported_epoch":"213","state":"active+clean","last_fresh":"2020-08-07T15:11:32.269258+0000","last_change":"2020-08-07T15:10:50.654783+0000","last_active":"2020-08-07T15:11:32.269258+0000","last_peered":"2020-08-07T15:11:32.269258+0000","last_clean":"2020-08-07T15:11:32.269258+0000","last_became_active":"2020-08-07T15:10:46.498051+0000","last_became_peered":"2020-08-07T15:10:46.498051+0000","last_unstale":"2020-08-07T15:11:32.269258+0000","last_undegraded":"2020-08-07T15:11:32.269258+0000","last_fullsized":"2020-08-07T15:11:32.269258+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:50.654689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:50.654689+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"214'6544","reported_seq":"8048","reported_epoch":"214","state":"active+clean","last_fresh":"2020-08-07T15:11:36.721145+0000","last_change":"2020-08-07T15:11:32.212911+0000","last_active":"2020-08-07T15:11:36.721145+0000","last_peered":"2020-08-07T15:11:36.721145+0000","last_clean":"2020-08-07T15:11:36.721145+0000","last_became_active":"2020-08-07T15:11:32.212339+0000","last_became_peered":"2020-08-07T15:11:32.212339+0000","last_unstale":"2020-08-07T15:11:36.721145+0000","last_undegraded":"2020-08-07T15:11:36.721145+0000","last_fullsized":"2020-08-07T15:11:36.721145+0000","mapping_epoch":212,"log_start":"214'6536","ondisk_log_start":"214'6536","created":164,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53608448,"num_objects":818,"num_object_clones":0,"num_object_copies":2454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6544,"num_write_kb":52352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":712,"num_bytes_recovered":46661632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"249","reported_epoch":"214","state":"active+clean","last_fresh":"2020-08-07T15:11:33.420819+0000","last_change":"2020-08-07T15:10:40.358431+0000","last_active":"2020-08-07T15:11:33.420819+0000","last_peered":"2020-08-07T15:11:33.420819+0000","last_clean":"2020-08-07T15:11:33.420819+0000","last_became_active":"2020-08-07T15:08:41.689435+0000","last_became_peered":"2020-08-07T15:08:41.689435+0000","last_unstale":"2020-08-07T15:11:33.420819+0000","last_undegraded":"2020-08-07T15:11:33.420819+0000","last_fullsized":"2020-08-07T15:11:33.420819+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:40.358337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:40.358337+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"214'6648","reported_seq":"6708","reported_epoch":"214","state":"active+clean","last_fresh":"2020-08-07T15:11:36.685892+0000","last_change":"2020-08-07T15:11:19.764496+0000","last_active":"2020-08-07T15:11:36.685892+0000","last_peered":"2020-08-07T15:11:36.685892+0000","last_clean":"2020-08-07T15:11:36.685892+0000","last_became_active":"2020-08-07T15:10:16.256419+0000","last_became_peered":"2020-08-07T15:10:16.256419+0000","last_unstale":"2020-08-07T15:11:36.685892+0000","last_undegraded":"2020-08-07T15:11:36.685892+0000","last_fullsized":"2020-08-07T15:11:36.685892+0000","mapping_epoch":164,"log_start":"214'6640","ondisk_log_start":"214'6640","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'5328","last_scrub_stamp":"2020-08-07T15:11:19.764422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:11:19.764422+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":54460416,"num_objects":831,"num_object_clones":0,"num_object_copies":2493,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6648,"num_write_kb":53184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"148","reported_epoch":"213","state":"active+clean","last_fresh":"2020-08-07T15:11:32.204627+0000","last_change":"2020-08-07T15:11:10.853414+0000","last_active":"2020-08-07T15:11:32.204627+0000","last_peered":"2020-08-07T15:11:32.204627+0000","last_clean":"2020-08-07T15:11:32.204627+0000","last_became_active":"2020-08-07T15:10:46.493498+0000","last_became_peered":"2020-08-07T15:10:46.493498+0000","last_unstale":"2020-08-07T15:11:32.204627+0000","last_undegraded":"2020-08-07T15:11:32.204627+0000","last_fullsized":"2020-08-07T15:11:32.204627+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:10.853337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:10.853337+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"214'6856","reported_seq":"6914","reported_epoch":"214","state":"active+clean","last_fresh":"2020-08-07T15:11:36.679830+0000","last_change":"2020-08-07T15:10:16.253662+0000","last_active":"2020-08-07T15:11:36.679830+0000","last_peered":"2020-08-07T15:11:36.679830+0000","last_clean":"2020-08-07T15:11:36.679830+0000","last_became_active":"2020-08-07T15:10:16.253105+0000","last_became_peered":"2020-08-07T15:10:16.253105+0000","last_unstale":"2020-08-07T15:11:36.679830+0000","last_undegraded":"2020-08-07T15:11:36.679830+0000","last_fullsized":"2020-08-07T15:11:36.679830+0000","mapping_epoch":164,"log_start":"214'6848","ondisk_log_start":"214'6848","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:10:15.236926+0000","last_clean_scrub_stamp":"2020-08-07T15:10:15.236926+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56164352,"num_objects":857,"num_object_clones":0,"num_object_copies":2571,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6856,"num_write_kb":54848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"173","reported_epoch":"213","state":"active+undersized","last_fresh":"2020-08-07T15:11:32.205853+0000","last_change":"2020-08-07T15:10:51.561336+0000","last_active":"2020-08-07T15:11:32.205853+0000","last_peered":"2020-08-07T15:11:32.205853+0000","last_clean":"2020-08-07T15:10:49.526944+0000","last_became_active":"2020-08-07T15:10:51.561336+0000","last_became_peered":"2020-08-07T15:10:51.561336+0000","last_unstale":"2020-08-07T15:11:32.205853+0000","last_undegraded":"2020-08-07T15:11:32.205853+0000","last_fullsized":"2020-08-07T15:10:51.547335+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:37.601099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:37.601099+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":898613248,"num_objects":13712,"num_object_clones":0,"num_object_copies":41136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3443,"num_objects_misplaced":896,"num_objects_unfound":0,"num_objects_dirty":13712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109694,"num_write_kb":877552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1498,"num_bytes_recovered":98172928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2620366848,"data_stored":2620366848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":43,"acting":44,"num_store_stats":8},{"poolid":1,"num_pg":31,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":149,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":83,"acting":88,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542210,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1275584,"kb_used_data":227000,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93096256,"statfs":{"total":96636764160,"available":95330566144,"internally_reserved":1073741824,"allocated":232448000,"data_stored":231499333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:11:28 2020","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":9.548,"15min":3.778},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.1970000000000001,"5min":856.07100000000003,"15min":856.07100000000003},"last":0.65500000000000003},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0389999999999999,"15min":0.98799999999999999},"min":{"1min":0.53400000000000003,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":3.4159999999999999,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":0.63500000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.335,"15min":1.3080000000000001},"min":{"1min":0.64200000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.9199999999999999,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.141,"15min":1.024},"min":{"1min":0.61799999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":4.0060000000000002,"5min":4.0060000000000002,"15min":4.0060000000000002},"last":2.4049999999999998}]},{"osd":2,"last update":"Fri Aug 7 15:11:28 2020","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.292,"15min":1.1819999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.4359999999999999,"5min":5.6619999999999999,"15min":5.6619999999999999},"last":1.071},{"interface":"front","average":{"1min":1.105,"5min":1.3360000000000001,"15min":1.1899999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.109,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":0.98999999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:11:28 2020","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":16.053000000000001,"15min":6.4649999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.5409999999999999,"5min":917.38699999999994,"15min":917.38699999999994},"last":1.149},{"interface":"front","average":{"1min":1.1299999999999999,"5min":4.9219999999999997,"15min":2.536},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.4750000000000001,"5min":333.59199999999998,"15min":333.59199999999998},"last":1.3859999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:11:28 2020","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.427,"15min":1.3260000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.1469999999999998,"5min":4.835,"15min":4.835},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.125,"5min":11.481,"15min":4.7009999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.4300000000000002,"5min":866.24099999999999,"15min":866.24099999999999},"last":1.4259999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:11:28 2020","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":16.597000000000001,"15min":30.536999999999999},"min":{"1min":0.70599999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.5870000000000002,"5min":831.88099999999997,"15min":831.88099999999997},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.8049999999999999,"15min":1.5980000000000001},"min":{"1min":0.93700000000000006,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.6200000000000001,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":1.524}]},{"osd":7,"last update":"Fri Aug 7 15:11:28 2020","interfaces":[{"interface":"back","average":{"1min":1.609,"5min":4.0119999999999996,"15min":2.3490000000000002},"min":{"1min":0.873,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":5.4800000000000004,"5min":224.14500000000001,"15min":224.14500000000001},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.716,"5min":1.732,"15min":1.5820000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":5.5309999999999997,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":1.111}]}]},{"osd":7,"up_from":13,"seq":55834574915,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1373812,"kb_used_data":325228,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92998028,"statfs":{"total":96636764160,"available":95229980672,"internally_reserved":1073741824,"allocated":333033472,"data_stored":332092173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":42.923000000000002,"5min":9.4640000000000004,"15min":4.0430000000000001},"min":{"1min":0.48999999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":755.10000000000002,"5min":755.10000000000002,"15min":755.10000000000002},"last":0.622},{"interface":"front","average":{"1min":0.96699999999999997,"5min":9.7780000000000005,"15min":4.0259999999999998},"min":{"1min":0.46300000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.718,"5min":529.88699999999994,"15min":529.88699999999994},"last":1.2949999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":5.758,"15min":2.5579999999999998},"min":{"1min":0.47699999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.7290000000000001,"5min":547.65300000000002,"15min":547.65300000000002},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.4159999999999999,"5min":4.9189999999999996,"15min":2.5640000000000001},"min":{"1min":0.57399999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":5.4960000000000004,"5min":372.50799999999998,"15min":372.50799999999998},"last":0.95099999999999996}]},{"osd":2,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.3240000000000001,"15min":1.238},"min":{"1min":0.54900000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":6.968,"5min":6.968,"15min":6.968},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1799999999999999,"15min":1.2410000000000001},"min":{"1min":0.58599999999999997,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.7709999999999999,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":1.268}]},{"osd":3,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":9.5,"15min":4.1840000000000002},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":3.3540000000000001,"5min":698.19899999999996,"15min":698.19899999999996},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.2969999999999999,"15min":1.357},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":4.1779999999999999,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":1.3959999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.3120000000000001,"15min":1.546},"min":{"1min":0.60999999999999999,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.5449999999999999,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.53,"15min":1.6240000000000001},"min":{"1min":0.71399999999999997,"5min":0.621,"15min":0.621},"max":{"1min":2.9900000000000002,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.2450000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.7130000000000001,"5min":1.329,"15min":1.3520000000000001},"min":{"1min":0.749,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":6.2080000000000002,"5min":6.2080000000000002,"15min":6.2080000000000002},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.7350000000000001,"5min":1.526,"15min":1.6859999999999999},"min":{"1min":0.89000000000000001,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":3.052,"5min":5.327,"15min":5.327},"last":0.93200000000000005}]},{"osd":6,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":40.012999999999998,"5min":9.1820000000000004,"15min":4.016},"min":{"1min":0.86399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":695.92899999999997,"5min":695.92899999999997,"15min":695.92899999999997},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.613,"5min":4.8959999999999999,"15min":13.849},"min":{"1min":0.91300000000000003,"5min":0.87,"15min":0.87},"max":{"1min":3.0270000000000001,"5min":266.26999999999998,"15min":266.26999999999998},"last":0.89500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1155876,"kb_used_data":107292,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93215964,"statfs":{"total":96636764160,"available":95453147136,"internally_reserved":1073741824,"allocated":109867008,"data_stored":109130383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":13.122,"5min":16.731999999999999,"15min":28.367999999999999},"min":{"1min":0.47199999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":211.63399999999999,"5min":497.04000000000002,"15min":497.04000000000002},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.645,"5min":14.757999999999999,"15min":5.673},"min":{"1min":0.57399999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":8.8849999999999998,"5min":993.84500000000003,"15min":993.84500000000003},"last":1.2929999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":54.984000000000002,"5min":11.816000000000001,"15min":4.5380000000000003},"min":{"1min":0.57199999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":959.36900000000003,"5min":959.36900000000003,"15min":959.36900000000003},"last":1.407},{"interface":"front","average":{"1min":1.72,"5min":1.357,"15min":1.2549999999999999},"min":{"1min":0.53600000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":5.4530000000000003,"5min":6.4029999999999996,"15min":6.4029999999999996},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":9.2420000000000009,"15min":3.9540000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.3879999999999999,"5min":828.31600000000003,"15min":828.31600000000003},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.2130000000000001,"5min":10.119999999999999,"15min":4.3419999999999996},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.5659999999999998,"5min":905.38300000000004,"15min":905.38300000000004},"last":1.071}]},{"osd":4,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":49.530999999999999,"5min":16.422999999999998,"15min":6.3300000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":872.58500000000004,"5min":872.58500000000004,"15min":872.58500000000004},"last":1.137},{"interface":"front","average":{"1min":2.2080000000000002,"5min":1.5409999999999999,"15min":1.3129999999999999},"min":{"1min":0.75800000000000001,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":12.640000000000001,"5min":12.640000000000001,"15min":12.640000000000001},"last":0.64100000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":6.0919999999999996,"15min":10.702},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.056,"5min":213.63800000000001,"15min":213.63800000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":2.4630000000000001,"5min":12.974,"15min":5.1180000000000003},"min":{"1min":0.93400000000000005,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":12.677,"5min":875.44600000000003,"15min":875.44600000000003},"last":1.276}]},{"osd":6,"last update":"Fri Aug 7 15:10:32 2020","interfaces":[{"interface":"back","average":{"1min":2.9340000000000002,"5min":1.863,"15min":1.655},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":22.402000000000001,"5min":22.402000000000001,"15min":22.402000000000001},"last":1.786},{"interface":"front","average":{"1min":1.3160000000000001,"5min":3.589,"15min":2.1030000000000002},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.3980000000000001,"5min":207.137,"15min":207.137},"last":1.105}]},{"osd":7,"last update":"Fri Aug 7 15:10:19 2020","interfaces":[{"interface":"back","average":{"1min":2.0910000000000002,"5min":1.7829999999999999,"15min":1.671},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":5.4960000000000004,"5min":6.444,"15min":6.444},"last":2.04},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.655,"15min":1.538},"min":{"1min":0.78700000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.0219999999999998,"5min":10.752000000000001,"15min":10.752000000000001},"last":1.1679999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574915,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1485876,"kb_used_data":437292,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92885964,"statfs":{"total":96636764160,"available":95115227136,"internally_reserved":1073741824,"allocated":447787008,"data_stored":446845709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":24.291,"5min":21.719999999999999,"15min":7.891},"min":{"1min":0.52700000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":432.899,"5min":845.57100000000003,"15min":845.57100000000003},"last":1.96},{"interface":"front","average":{"1min":1.752,"5min":1.181,"15min":1.0149999999999999},"min":{"1min":0.62,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":9.3650000000000002,"5min":9.3650000000000002,"15min":9.3650000000000002},"last":1.363}]},{"osd":2,"last update":"Fri Aug 7 15:11:32 2020","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":1.0069999999999999,"15min":0.95399999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.5289999999999999,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.5629999999999999,"5min":9.9559999999999995,"15min":25.145},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":4.8689999999999998,"5min":606.18499999999995,"15min":606.18499999999995},"last":0.89600000000000002}]},{"osd":3,"last update":"Fri Aug 7 15:11:32 2020","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.264,"15min":1.0449999999999999},"min":{"1min":0.54900000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.298,"5min":9.3360000000000003,"15min":9.3360000000000003},"last":0.749},{"interface":"front","average":{"1min":1.121,"5min":1.409,"15min":1.248},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.9330000000000001,"5min":11.19,"15min":11.19},"last":1.0529999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:11:32 2020","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.321,"15min":1.2170000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.7589999999999999,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":1.083},{"interface":"front","average":{"1min":1.198,"5min":1.4390000000000001,"15min":1.3180000000000001},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.887,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":1.466}]},{"osd":5,"last update":"Fri Aug 7 15:11:32 2020","interfaces":[{"interface":"back","average":{"1min":28.428000000000001,"5min":14.52,"15min":5.6580000000000004},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":539.31100000000004,"5min":539.31100000000004,"15min":539.31100000000004},"last":1.421},{"interface":"front","average":{"1min":1.444,"5min":1.4419999999999999,"15min":1.218},"min":{"1min":0.77600000000000002,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":3.5790000000000002,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":1.1379999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:11:32 2020","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":2.8889999999999998,"15min":6.7050000000000001},"min":{"1min":0.84099999999999997,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":3.1899999999999999,"5min":124.738,"15min":124.738},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.5940000000000001,"5min":4.9909999999999997,"15min":14.099},"min":{"1min":0.81799999999999995,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":5.1799999999999997,"5min":294.94,"15min":294.94},"last":1.633}]},{"osd":7,"last update":"Fri Aug 7 15:11:32 2020","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.5629999999999999,"15min":1.5780000000000001},"min":{"1min":0.90600000000000003,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":4.0010000000000003,"5min":6.468,"15min":6.468},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.492,"5min":10.583,"15min":4.4509999999999996},"min":{"1min":0.90600000000000003,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":3.1749999999999998,"5min":859.82399999999996,"15min":859.82399999999996},"last":1.6679999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574915,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1370420,"kb_used_data":321836,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93001420,"statfs":{"total":96636764160,"available":95233454080,"internally_reserved":1073741824,"allocated":329560064,"data_stored":328618765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:11:22 2020","interfaces":[{"interface":"back","average":{"1min":2.2519999999999998,"5min":1.4950000000000001,"15min":1.2430000000000001},"min":{"1min":0.46000000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":23.699000000000002,"5min":23.699000000000002,"15min":23.699000000000002},"last":0.42899999999999999},{"interface":"front","average":{"1min":2.1059999999999999,"5min":1.417,"15min":1.1220000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":23.792999999999999,"5min":23.792999999999999,"15min":23.792999999999999},"last":0.66800000000000004}]},{"osd":1,"last update":"Fri Aug 7 15:10:21 2020","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.6970000000000001,"15min":1.2929999999999999},"min":{"1min":0.52000000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":5.5919999999999996,"5min":18.672000000000001,"15min":18.672000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.264,"5min":1.3320000000000001,"15min":1.3500000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":3.7970000000000002,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":0.93400000000000005}]},{"osd":3,"last update":"Fri Aug 7 15:11:22 2020","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":1.234,"15min":1.1859999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.681,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":15.957000000000001,"5min":10.978999999999999,"15min":4.6970000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":312.87900000000002,"5min":501.75599999999997,"15min":501.75599999999997},"last":1.4650000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:11:22 2020","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.603,"15min":1.274},"min":{"1min":0.623,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":7.9569999999999999,"5min":19.783000000000001,"15min":19.783000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.8049999999999999,"5min":1.554,"15min":1.613},"min":{"1min":0.80700000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":8.048,"5min":8.048,"15min":8.048},"last":1.242}]},{"osd":5,"last update":"Fri Aug 7 15:11:22 2020","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.367,"15min":1.2969999999999999},"min":{"1min":0.72899999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":4.3639999999999999,"5min":5.2220000000000004,"15min":5.2220000000000004},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.2989999999999999,"5min":20.52,"15min":23.856000000000002},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":3.7559999999999998,"5min":627.53300000000002,"15min":627.53300000000002},"last":1.5}]},{"osd":6,"last update":"Fri Aug 7 15:11:34 2020","interfaces":[{"interface":"back","average":{"1min":42.643000000000001,"5min":9.9380000000000006,"15min":4.1260000000000003},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":821.91899999999998,"5min":821.91899999999998,"15min":821.91899999999998},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.2190000000000001,"5min":9.9429999999999996,"15min":4.1950000000000003},"min":{"1min":0.80200000000000005,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.73,"5min":751.70000000000005,"15min":751.70000000000005},"last":0.95399999999999996}]},{"osd":7,"last update":"Fri Aug 7 15:11:22 2020","interfaces":[{"interface":"back","average":{"1min":1.899,"5min":12.601000000000001,"15min":5.3609999999999998},"min":{"1min":0.88500000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":7.9900000000000002,"5min":501.91699999999997,"15min":501.91699999999997},"last":1.401},{"interface":"front","average":{"1min":2.044,"5min":2.0630000000000002,"15min":1.6850000000000001},"min":{"1min":0.94199999999999995,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":8.0779999999999994,"5min":35.646999999999998,"15min":35.646999999999998},"last":1.7729999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1211980,"kb_used_data":163396,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93159860,"statfs":{"total":96636764160,"available":95395696640,"internally_reserved":1073741824,"allocated":167317504,"data_stored":166376205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.2689999999999999,"15min":1.258},"min":{"1min":0.51100000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":3.2149999999999999,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.296,"15min":1.5229999999999999},"min":{"1min":0.53900000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":4.0190000000000001,"5min":13.067,"15min":13.067},"last":1.3640000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":4.4779999999999998,"15min":2.758},"min":{"1min":0.59099999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.847,"5min":270.68700000000001,"15min":270.68700000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":1.109,"5min":1.8580000000000001,"15min":1.96},"min":{"1min":0.54100000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.0550000000000002,"5min":14.349,"15min":14.349},"last":0.56699999999999995}]},{"osd":2,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.3380000000000001,"15min":1.2110000000000001},"min":{"1min":0.65300000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.6389999999999998,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":1.036},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.9810000000000001,"15min":1.7829999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":6.7889999999999997,"5min":51.530000000000001,"15min":51.530000000000001},"last":1.298}]},{"osd":4,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":17.366,"15min":6.9199999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":3.1600000000000001,"5min":646.26700000000005,"15min":646.26700000000005},"last":0.80800000000000005},{"interface":"front","average":{"1min":43.484000000000002,"5min":21.631,"15min":28.609999999999999},"min":{"1min":0.77100000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":844.10500000000002,"5min":844.10500000000002,"15min":844.10500000000002},"last":1.48}]},{"osd":5,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":5.9560000000000004,"15min":2.9969999999999999},"min":{"1min":0.79300000000000004,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.2370000000000001,"5min":403.88400000000001,"15min":403.88400000000001},"last":244.08099999999999},{"interface":"front","average":{"1min":1.591,"5min":1.895,"15min":1.8109999999999999},"min":{"1min":0.97399999999999998,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":3.9790000000000001,"5min":8.9120000000000008,"15min":8.9120000000000008},"last":1.9430000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.653,"15min":1.5720000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":4.1600000000000001,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":1.978},{"interface":"front","average":{"1min":1.944,"5min":13.427,"15min":5.7999999999999998},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":6.9980000000000002,"5min":688.15999999999997,"15min":688.15999999999997},"last":2.0390000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.655,"5min":1.6799999999999999,"15min":2.0009999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":6.8209999999999997,"5min":6.8209999999999997,"15min":6.8209999999999997},"last":2.0110000000000001},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.77,"15min":1.9079999999999999},"min":{"1min":0.77000000000000002,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":3.1499999999999999,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":1.6619999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1540284,"kb_used_data":491700,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92831556,"statfs":{"total":96636764160,"available":95059513344,"internally_reserved":1073741824,"allocated":503500800,"data_stored":502559501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:11:25 2020","interfaces":[{"interface":"back","average":{"1min":20.597999999999999,"5min":9.9239999999999995,"15min":4.0949999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":356.91800000000001,"5min":399.73700000000002,"15min":399.73700000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.84299999999999997,"5min":1.046,"15min":0.91400000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.972,"5min":6.3970000000000002,"15min":6.3970000000000002},"last":1.087}]},{"osd":1,"last update":"Fri Aug 7 15:10:24 2020","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":5.7629999999999999,"15min":2.823},"min":{"1min":0.52900000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.8109999999999999,"5min":320.67000000000002,"15min":320.67000000000002},"last":1.085},{"interface":"front","average":{"1min":1.177,"5min":7.5300000000000002,"15min":3.4279999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.7200000000000002,"5min":689.846,"15min":689.846},"last":0.66900000000000004}]},{"osd":2,"last update":"Fri Aug 7 15:11:25 2020","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.3680000000000001,"15min":1.4239999999999999},"min":{"1min":0.67300000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.786,"5min":6.8730000000000002,"15min":6.8730000000000002},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.51,"15min":1.5129999999999999},"min":{"1min":0.85899999999999999,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.8140000000000001,"5min":7.1360000000000001,"15min":7.1360000000000001},"last":1.171}]},{"osd":3,"last update":"Fri Aug 7 15:11:25 2020","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":12.045,"15min":5.0199999999999996},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.6579999999999999,"5min":638.92100000000005,"15min":638.92100000000005},"last":0.59099999999999997},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.3480000000000001,"15min":1.2829999999999999},"min":{"1min":0.70899999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.0289999999999999,"5min":6.431,"15min":6.431},"last":1.5680000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:11:25 2020","interfaces":[{"interface":"back","average":{"1min":41.594000000000001,"5min":18.439,"15min":7.2309999999999999},"min":{"1min":0.60499999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":725.98599999999999,"5min":990.71600000000001,"15min":990.71600000000001},"last":1.115},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.4259999999999999,"15min":1.5600000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.7310000000000001,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":0.88}]},{"osd":6,"last update":"Fri Aug 7 15:11:34 2020","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.5800000000000001,"15min":1.72},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":5.4459999999999997,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.256,"5min":4.9329999999999998,"15min":14.382999999999999},"min":{"1min":0.71399999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.3719999999999999,"5min":389.14600000000002,"15min":389.14600000000002},"last":0.73799999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:11:25 2020","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":3.782,"15min":2.1989999999999998},"min":{"1min":0.78300000000000003,"5min":0.62,"15min":0.62},"max":{"1min":2.5310000000000001,"5min":252.798,"15min":252.798},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.4370000000000001,"5min":14.112,"15min":5.8010000000000002},"min":{"1min":0.78100000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":3.0369999999999999,"5min":501.93000000000001,"15min":501.93000000000001},"last":1.1950000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1528428,"kb_used_data":479844,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92843412,"statfs":{"total":96636764160,"available":95071653888,"internally_reserved":1073741824,"allocated":491360256,"data_stored":490418957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:11:26 2020","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.996,"15min":1.05},"min":{"1min":0.54400000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.5289999999999999,"5min":3.7349999999999999,"15min":3.7349999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":15.863,"5min":3.956,"15min":1.855},"min":{"1min":0.44,"5min":0.438,"15min":0.438},"max":{"1min":329.11399999999998,"5min":329.11399999999998,"15min":329.11399999999998},"last":0.82799999999999996}]},{"osd":1,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":8.5239999999999991,"5min":12.039999999999999,"15min":4.6319999999999997},"min":{"1min":0.56799999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":139.88200000000001,"5min":787.22900000000004,"15min":787.22900000000004},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.448,"5min":2.5649999999999999,"15min":1.593},"min":{"1min":0.59899999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":4.3700000000000001,"5min":116.548,"15min":116.548},"last":1.5900000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:11:26 2020","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":2.714,"15min":7.0510000000000002},"min":{"1min":0.52100000000000002,"5min":0.499,"15min":0.499},"max":{"1min":2.109,"5min":168.56899999999999,"15min":168.56899999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.3939999999999999,"5min":7.6200000000000001,"15min":24.626999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":3.5609999999999999,"5min":676.27099999999996,"15min":676.27099999999996},"last":0.86899999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:11:26 2020","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.6559999999999999,"15min":1.2969999999999999},"min":{"1min":0.71499999999999997,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":5.1539999999999999,"5min":29.129000000000001,"15min":29.129000000000001},"last":2.1629999999999998},{"interface":"front","average":{"1min":1.1140000000000001,"5min":9.9969999999999999,"15min":4.141},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.149,"5min":712.98699999999997,"15min":712.98699999999997},"last":0.80200000000000005}]},{"osd":4,"last update":"Fri Aug 7 15:11:26 2020","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":22.683,"15min":41.488999999999997},"min":{"1min":0.54800000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.5899999999999999,"5min":971.67999999999995,"15min":971.67999999999995},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.272,"15min":1.294},"min":{"1min":0.63700000000000001,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":5.4900000000000002,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":1.3169999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:11:26 2020","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":31.462,"15min":44.093000000000004},"min":{"1min":0.85099999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":5.4480000000000004,"5min":966.60900000000004,"15min":966.60900000000004},"last":2.2010000000000001},{"interface":"front","average":{"1min":1.2789999999999999,"5min":11.677,"15min":21.236999999999998},"min":{"1min":0.871,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.4710000000000001,"5min":517.26700000000005,"15min":517.26700000000005},"last":0.90900000000000003}]},{"osd":7,"last update":"Fri Aug 7 15:11:26 2020","interfaces":[{"interface":"back","average":{"1min":1.6799999999999999,"5min":1.403,"15min":1.3400000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":5.1559999999999997,"5min":5.1559999999999997,"15min":5.1559999999999997},"last":1.4259999999999999},{"interface":"front","average":{"1min":32.561999999999998,"5min":12.834,"15min":5.3220000000000001},"min":{"1min":0.63700000000000001,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":501.64299999999997,"5min":515.09100000000001,"15min":515.09100000000001},"last":2.2290000000000001}]}]}]}} 2020-08-07T15:11:41.292 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:11:41.312 INFO:tasks.thrashosds.thrasher:Setting 5.2 to [2, 4, 6] 2020-08-07T15:11:41.313 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '5.2', '2', '4', '6'] 2020-08-07T15:11:41.313 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pg-upmap 5.2 2 4 6 2020-08-07T15:11:41.370 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:41.367+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:41.415 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:41.416 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:41.427 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:11:41.447 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:41.435+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:41.538 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:41.536+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:41.602 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:41.602 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:41.615 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:11:41.672 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:41.668+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:41.739 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:41.737+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:41.814 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:41.815 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:41.831 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:11:41.882 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:41.871+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:41.950 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:41.943+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:42.046 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:42.043+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:42.088 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:42.089 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:42.108 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:11:42.151 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:42.149+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:42.218 INFO:teuthology.orchestra.run.smithi074.stderr:set 5.2 pg_upmap mapping to [2,4,6] 2020-08-07T15:11:42.219 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 82 16 118150 118134 11.2541 12.4219 0.0131681 0.0111031 2020-08-07T15:11:42.251 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:42.249+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:42.323 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:42.323 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:42.343 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:11:42.355 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:42.350+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:42.454 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:42.452+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:42.521 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:42.522 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:42.532 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:11:42.586 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:42.581+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:42.657 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:42.658 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:42.672 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:42.662+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:42.686 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:11:42.773 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:11:42.774 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:11:42.775 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:42.773+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:42.859 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:42.860 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:42.869 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:11:42.884 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:42.883+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:42.988 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:11:42.988 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:42.987+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:43.008 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:11:43.034 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:11:43.074 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:11:43.075 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:11:43.075 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:11:43.075 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:11:43.075 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.076 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:145801 3.6 e75/26 3:69e4cb22:::benchmark_data_smithi074_35447_object16192:head v 75'9405, mlcod=75'9401)", 2020-08-07T15:11:43.076 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:11:43.076 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 208.365384025, 2020-08-07T15:11:43.076 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639, 2020-08-07T15:11:43.077 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:43.077 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:43.077 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:43.097 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.098 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:43.098 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:11:43.099 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:43.103 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.112 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:43.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:11:43.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6511000000000002e-05 2020-08-07T15:11:43.113 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.113 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:43.113 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671803+0000", 2020-08-07T15:11:43.132 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7190000000000004e-06 2020-08-07T15:11:43.138 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.153 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.153 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:43.155 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671809+0000", 2020-08-07T15:11:43.155 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74210928300000001 2020-08-07T15:11:43.155 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.172 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.172 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:43.173 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413919+0000", 2020-08-07T15:11:43.173 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1759999999999999e-05 2020-08-07T15:11:43.173 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.174 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.174 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:43.174 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413930+0000", 2020-08-07T15:11:43.174 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000108601 2020-08-07T15:11:43.175 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.175 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.175 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:43.175 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414039+0000", 2020-08-07T15:11:43.175 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9948999999999999e-05 2020-08-07T15:11:43.176 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.176 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.176 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:43.176 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414089+0000", 2020-08-07T15:11:43.177 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00079656499999999995 2020-08-07T15:11:43.177 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.177 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.177 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:43.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414885+0000", 2020-08-07T15:11:43.209 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2250999999999998e-05 2020-08-07T15:11:43.210 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.210 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.210 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:43.211 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414938+0000", 2020-08-07T15:11:43.211 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639 2020-08-07T15:11:43.211 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:43.211 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:43.212 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:43.252 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.260 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:104104 4.0 e146/131 4:0ffb764a:::benchmark_data_smithi074_44586_object13012:head v 146'6960, mlcod=146'6953)", 2020-08-07T15:11:43.260 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:11:43.272 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 114.83883724899999, 2020-08-07T15:11:43.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75443331599999996, 2020-08-07T15:11:43.273 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:43.273 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:43.295 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:43.298 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.300 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:43.302 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:11:43.302 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:43.302 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.309 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.310 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:43.314 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:11:43.314 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1743000000000002e-05 2020-08-07T15:11:43.314 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.315 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.315 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:43.315 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198355+0000", 2020-08-07T15:11:43.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1249999999999998e-06 2020-08-07T15:11:43.332 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.332 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.332 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:43.333 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198361+0000", 2020-08-07T15:11:43.333 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74719179599999996 2020-08-07T15:11:43.333 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.333 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:43.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945553+0000", 2020-08-07T15:11:43.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6800000000000001e-06 2020-08-07T15:11:43.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.396 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.396 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:43.397 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945557+0000", 2020-08-07T15:11:43.397 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012101500000000001 2020-08-07T15:11:43.397 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.397 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.412 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:43.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945678+0000", 2020-08-07T15:11:43.423 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6621000000000001e-05 2020-08-07T15:11:43.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.424 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.424 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:43.424 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945705+0000", 2020-08-07T15:11:43.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069879540000000002 2020-08-07T15:11:43.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.425 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:43.426 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.952693+0000", 2020-08-07T15:11:43.426 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3381999999999999e-05 2020-08-07T15:11:43.426 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.426 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:43.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.952736+0000", 2020-08-07T15:11:43.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75443331599999996 2020-08-07T15:11:43.427 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:43.427 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:43.428 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:43.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.428 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89641 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:11:43.429 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.550933+0000", 2020-08-07T15:11:43.429 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 21.486206828, 2020-08-07T15:11:43.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69051440900000005, 2020-08-07T15:11:43.429 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:43.462 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:43.463 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:43.463 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:43.463 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:43.463 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89641 2020-08-07T15:11:43.464 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.464 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:43.464 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.464 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:43.464 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550933+0000", 2020-08-07T15:11:43.465 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:43.465 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.465 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.465 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:43.471 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550933+0000", 2020-08-07T15:11:43.472 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6520000000000002e-05 2020-08-07T15:11:43.472 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.472 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.472 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:43.473 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550980+0000", 2020-08-07T15:11:43.473 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2410000000000003e-06 2020-08-07T15:11:43.473 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.473 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.473 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:43.474 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550985+0000", 2020-08-07T15:11:43.474 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2707e-05 2020-08-07T15:11:43.474 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.474 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.474 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:43.475 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550998+0000", 2020-08-07T15:11:43.475 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8629999999999996e-06 2020-08-07T15:11:43.475 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.475 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.476 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:43.476 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551004+0000", 2020-08-07T15:11:43.476 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4987000000000001e-05 2020-08-07T15:11:43.477 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.477 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.477 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:43.477 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551059+0000", 2020-08-07T15:11:43.477 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1718e-05 2020-08-07T15:11:43.478 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.478 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.478 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:43.478 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551070+0000", 2020-08-07T15:11:43.479 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67720431000000003 2020-08-07T15:11:43.483 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.483 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.483 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:43.484 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.228275+0000", 2020-08-07T15:11:43.484 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000731965 2020-08-07T15:11:43.484 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.484 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.485 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:43.485 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229007+0000", 2020-08-07T15:11:43.522 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00025396199999999999 2020-08-07T15:11:43.534 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:43.535 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229261+0000", 2020-08-07T15:11:43.535 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002511189 2020-08-07T15:11:43.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.536 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:43.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231772+0000", 2020-08-07T15:11:43.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3399999999999997e-07 2020-08-07T15:11:43.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.537 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.537 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:43.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231773+0000", 2020-08-07T15:11:43.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0274e-05 2020-08-07T15:11:43.537 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.545 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.546 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:43.546 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231823+0000", 2020-08-07T15:11:43.546 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0094095440000000006 2020-08-07T15:11:43.547 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.547 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:43.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241233+0000", 2020-08-07T15:11:43.547 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4011999999999998e-05 2020-08-07T15:11:43.548 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.557 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:43.557 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241277+0000", 2020-08-07T15:11:43.557 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7199999999999999e-07 2020-08-07T15:11:43.567 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.567 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.568 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:43.568 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241277+0000", 2020-08-07T15:11:43.568 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017061100000000001 2020-08-07T15:11:43.568 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.568 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.569 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:43.569 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241448+0000", 2020-08-07T15:11:43.569 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69051440900000005 2020-08-07T15:11:43.569 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:43.569 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:43.572 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:43.573 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.573 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.573 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89642 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:11:43.573 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.551807+0000", 2020-08-07T15:11:43.586 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 21.485333275999999, 2020-08-07T15:11:43.587 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68991613100000004, 2020-08-07T15:11:43.587 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:43.587 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:43.587 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:43.587 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:43.588 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:43.588 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89642 2020-08-07T15:11:43.588 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.588 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:43.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.588 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:43.589 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551807+0000", 2020-08-07T15:11:43.594 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:43.594 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.595 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.595 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:43.595 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551807+0000", 2020-08-07T15:11:43.595 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7049000000000001e-05 2020-08-07T15:11:43.596 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.596 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.596 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:43.596 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551834+0000", 2020-08-07T15:11:43.596 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9249999999999997e-06 2020-08-07T15:11:43.597 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.597 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.597 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:43.597 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551838+0000", 2020-08-07T15:11:43.597 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9160000000000001e-06 2020-08-07T15:11:43.598 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.598 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.598 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:43.598 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551846+0000", 2020-08-07T15:11:43.598 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3510000000000002e-06 2020-08-07T15:11:43.599 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.599 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.599 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:43.599 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551849+0000", 2020-08-07T15:11:43.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9276000000000001e-05 2020-08-07T15:11:43.600 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.600 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.601 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:43.601 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551898+0000", 2020-08-07T15:11:43.601 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3095000000000001e-05 2020-08-07T15:11:43.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.603 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:43.603 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551911+0000", 2020-08-07T15:11:43.603 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67789159799999998 2020-08-07T15:11:43.603 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:43.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229803+0000", 2020-08-07T15:11:43.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8069000000000001e-05 2020-08-07T15:11:43.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.605 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:43.605 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229881+0000", 2020-08-07T15:11:43.605 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000152917 2020-08-07T15:11:43.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.622 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:43.622 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230034+0000", 2020-08-07T15:11:43.622 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044936760000000003 2020-08-07T15:11:43.622 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.623 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.624 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:43.624 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234528+0000", 2020-08-07T15:11:43.624 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8999999999999995e-07 2020-08-07T15:11:43.624 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.625 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.625 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:43.625 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234529+0000", 2020-08-07T15:11:43.625 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7741000000000002e-05 2020-08-07T15:11:43.625 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.626 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.626 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:43.626 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234586+0000", 2020-08-07T15:11:43.626 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069143959999999997 2020-08-07T15:11:43.626 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.627 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:43.627 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241501+0000", 2020-08-07T15:11:43.627 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1433000000000001e-05 2020-08-07T15:11:43.627 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.636 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:43.636 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241542+0000", 2020-08-07T15:11:43.636 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7800000000000001e-07 2020-08-07T15:11:43.636 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.637 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.637 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:43.637 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241543+0000", 2020-08-07T15:11:43.637 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000180321 2020-08-07T15:11:43.637 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.638 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.638 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:43.638 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241723+0000", 2020-08-07T15:11:43.639 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68991613100000004 2020-08-07T15:11:43.639 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:43.639 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:43.639 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:43.673 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.673 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.673 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89643 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:11:43.673 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.552181+0000", 2020-08-07T15:11:43.674 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 21.484959281999998, 2020-08-07T15:11:43.674 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68980872400000004, 2020-08-07T15:11:43.674 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:43.674 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:43.674 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:43.675 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:43.675 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:43.675 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89643 2020-08-07T15:11:43.675 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.675 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:43.676 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.676 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:43.676 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552181+0000", 2020-08-07T15:11:43.676 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:43.676 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.676 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:43.677 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552181+0000", 2020-08-07T15:11:43.677 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0926999999999997e-05 2020-08-07T15:11:43.677 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.727 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:43.737 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552212+0000", 2020-08-07T15:11:43.740 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0512e-05 2020-08-07T15:11:43.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.741 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.741 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:43.741 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552222+0000", 2020-08-07T15:11:43.741 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9379999999999992e-06 2020-08-07T15:11:43.742 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.742 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.742 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:43.742 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552230+0000", 2020-08-07T15:11:43.742 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0260000000000001e-06 2020-08-07T15:11:43.743 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.743 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:43.743 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552234+0000", 2020-08-07T15:11:43.743 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6109000000000002e-05 2020-08-07T15:11:43.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:43.744 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552280+0000", 2020-08-07T15:11:43.744 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5318000000000001e-05 2020-08-07T15:11:43.745 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.745 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:43.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552296+0000", 2020-08-07T15:11:43.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67811269100000005 2020-08-07T15:11:43.747 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.747 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.748 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:43.748 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230408+0000", 2020-08-07T15:11:43.748 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000125518 2020-08-07T15:11:43.748 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.755 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:43.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230534+0000", 2020-08-07T15:11:43.755 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024637999999999998 2020-08-07T15:11:43.755 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:43.756 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230780+0000", 2020-08-07T15:11:43.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0066398869999999997 2020-08-07T15:11:43.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.757 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:43.757 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237420+0000", 2020-08-07T15:11:43.757 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3500000000000005e-07 2020-08-07T15:11:43.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.760 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:43.760 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237421+0000", 2020-08-07T15:11:43.760 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4005000000000003e-05 2020-08-07T15:11:43.761 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.762 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:43.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237465+0000", 2020-08-07T15:11:43.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043133529999999998 2020-08-07T15:11:43.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.763 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:43.763 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241778+0000", 2020-08-07T15:11:43.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0592e-05 2020-08-07T15:11:43.763 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.764 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:43.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241819+0000", 2020-08-07T15:11:43.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4999999999999998e-07 2020-08-07T15:11:43.764 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.765 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:43.765 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241819+0000", 2020-08-07T15:11:43.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017018299999999999 2020-08-07T15:11:43.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.766 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:43.766 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241990+0000", 2020-08-07T15:11:43.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68980872400000004 2020-08-07T15:11:43.766 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:43.766 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:43.767 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:43.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.767 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89644 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:11:43.767 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.552384+0000", 2020-08-07T15:11:43.768 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 21.484756248, 2020-08-07T15:11:43.768 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68997479299999998, 2020-08-07T15:11:43.768 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:43.768 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:43.768 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:43.769 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:43.769 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:43.769 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89644 2020-08-07T15:11:43.770 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.770 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:43.771 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.771 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:43.771 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552384+0000", 2020-08-07T15:11:43.771 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:43.772 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.772 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.772 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:43.772 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552384+0000", 2020-08-07T15:11:43.772 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9238999999999999e-05 2020-08-07T15:11:43.772 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.773 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.773 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:43.773 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552413+0000", 2020-08-07T15:11:43.773 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3340000000000003e-06 2020-08-07T15:11:43.773 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.774 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.774 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:43.774 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552417+0000", 2020-08-07T15:11:43.774 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3119999999999994e-06 2020-08-07T15:11:43.774 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.775 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:43.775 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552426+0000", 2020-08-07T15:11:43.775 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4139999999999996e-06 2020-08-07T15:11:43.775 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:43.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552430+0000", 2020-08-07T15:11:43.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6613999999999999e-05 2020-08-07T15:11:43.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:43.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:11:43.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7280000000000007e-06 2020-08-07T15:11:43.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:43.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552477+0000", 2020-08-07T15:11:43.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67873990399999995 2020-08-07T15:11:43.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.779 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:43.779 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231216+0000", 2020-08-07T15:11:43.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3629999999999998e-05 2020-08-07T15:11:43.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.790 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:43.791 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231290+0000", 2020-08-07T15:11:43.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000132108 2020-08-07T15:11:43.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:43.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231422+0000", 2020-08-07T15:11:43.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006049652 2020-08-07T15:11:43.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.798 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.798 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:43.798 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237472+0000", 2020-08-07T15:11:43.799 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000005e-07 2020-08-07T15:11:43.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:43.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237472+0000", 2020-08-07T15:11:43.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2848000000000001e-05 2020-08-07T15:11:43.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:43.800 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237505+0000", 2020-08-07T15:11:43.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045352179999999997 2020-08-07T15:11:43.801 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.801 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.801 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:43.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242040+0000", 2020-08-07T15:11:43.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012843699999999999 2020-08-07T15:11:43.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:43.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242169+0000", 2020-08-07T15:11:43.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9999999999999998e-07 2020-08-07T15:11:43.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.813 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:43.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242169+0000", 2020-08-07T15:11:43.813 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000189354 2020-08-07T15:11:43.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.814 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:43.814 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242359+0000", 2020-08-07T15:11:43.814 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68997479299999998 2020-08-07T15:11:43.827 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:43.828 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:43.828 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:43.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.834 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89645 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:11:43.835 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:11:43.835 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 21.484673466, 2020-08-07T15:11:43.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69011194600000003, 2020-08-07T15:11:43.835 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:43.835 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:43.836 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:43.836 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:43.836 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:43.849 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89645 2020-08-07T15:11:43.849 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.849 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:43.851 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.851 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:43.851 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:11:43.851 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:43.851 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:43.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:11:43.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6750999999999998e-05 2020-08-07T15:11:43.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.865 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:43.866 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552493+0000", 2020-08-07T15:11:43.866 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0350000000000002e-06 2020-08-07T15:11:43.882 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.890 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:43.890 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552496+0000", 2020-08-07T15:11:43.891 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3029999999999996e-06 2020-08-07T15:11:43.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.891 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.891 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:43.910 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552503+0000", 2020-08-07T15:11:43.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7580000000000001e-06 2020-08-07T15:11:43.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:43.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552506+0000", 2020-08-07T15:11:43.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4868000000000003e-05 2020-08-07T15:11:43.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:43.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552570+0000", 2020-08-07T15:11:43.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9649999999999997e-06 2020-08-07T15:11:43.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:43.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552579+0000", 2020-08-07T15:11:43.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67928357699999997 2020-08-07T15:11:43.926 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:43.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231863+0000", 2020-08-07T15:11:43.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000115696 2020-08-07T15:11:43.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:43.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231979+0000", 2020-08-07T15:11:43.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020456599999999999 2020-08-07T15:11:43.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:43.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232183+0000", 2020-08-07T15:11:43.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005327923 2020-08-07T15:11:43.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:43.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237511+0000", 2020-08-07T15:11:43.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8999999999999998e-07 2020-08-07T15:11:43.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.962 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:43.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237511+0000", 2020-08-07T15:11:43.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6019000000000002e-05 2020-08-07T15:11:43.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:43.964 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237547+0000", 2020-08-07T15:11:43.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045340980000000003 2020-08-07T15:11:43.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:43.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242082+0000", 2020-08-07T15:11:43.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00032620499999999998 2020-08-07T15:11:43.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:43.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242408+0000", 2020-08-07T15:11:43.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3500000000000002e-07 2020-08-07T15:11:43.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.975 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:43.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242408+0000", 2020-08-07T15:11:43.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000170457 2020-08-07T15:11:43.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:43.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242579+0000", 2020-08-07T15:11:43.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69011194600000003 2020-08-07T15:11:43.976 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:43.976 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:43.977 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:43.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.977 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89646 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:11:43.978 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.557244+0000", 2020-08-07T15:11:43.978 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 21.479895694, 2020-08-07T15:11:43.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68559352299999998, 2020-08-07T15:11:43.978 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:43.979 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:43.979 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:43.979 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:43.979 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:43.980 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89646 2020-08-07T15:11:43.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.980 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:43.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:43.981 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557244+0000", 2020-08-07T15:11:43.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:43.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.981 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:43.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557244+0000", 2020-08-07T15:11:43.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9699000000000001e-05 2020-08-07T15:11:43.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:43.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557274+0000", 2020-08-07T15:11:43.986 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1269999999999996e-06 2020-08-07T15:11:43.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.987 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:43.987 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557278+0000", 2020-08-07T15:11:43.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0099999999999995e-06 2020-08-07T15:11:43.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:43.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557286+0000", 2020-08-07T15:11:43.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7439999999999998e-06 2020-08-07T15:11:43.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:43.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557291+0000", 2020-08-07T15:11:43.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5927999999999999e-05 2020-08-07T15:11:43.990 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.990 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.990 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:43.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557327+0000", 2020-08-07T15:11:43.991 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5069999999999997e-06 2020-08-07T15:11:43.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:43.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557333+0000", 2020-08-07T15:11:43.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67527419200000005 2020-08-07T15:11:43.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:43.993 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232608+0000", 2020-08-07T15:11:43.993 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8699e-05 2020-08-07T15:11:43.993 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.993 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.993 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:43.994 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232686+0000", 2020-08-07T15:11:43.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018185099999999999 2020-08-07T15:11:43.995 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.995 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.995 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:43.995 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232868+0000", 2020-08-07T15:11:43.996 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0075819360000000001 2020-08-07T15:11:43.997 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.997 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.997 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:43.998 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240450+0000", 2020-08-07T15:11:43.998 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2799999999999995e-07 2020-08-07T15:11:43.998 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.998 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.998 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:43.999 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240451+0000", 2020-08-07T15:11:43.999 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3751000000000002e-05 2020-08-07T15:11:43.999 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:43.999 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:43.999 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:44.000 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240495+0000", 2020-08-07T15:11:44.000 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001628064 2020-08-07T15:11:44.000 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.000 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.000 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:44.001 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242123+0000", 2020-08-07T15:11:44.001 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00050955599999999998 2020-08-07T15:11:44.001 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.001 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.001 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:44.002 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242632+0000", 2020-08-07T15:11:44.002 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2900000000000004e-07 2020-08-07T15:11:44.002 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.002 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.002 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:44.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242633+0000", 2020-08-07T15:11:44.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000205202 2020-08-07T15:11:44.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:44.004 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242838+0000", 2020-08-07T15:11:44.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68559352299999998 2020-08-07T15:11:44.004 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:44.004 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:44.004 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:44.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.005 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89647 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:11:44.005 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.557315+0000", 2020-08-07T15:11:44.005 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 21.479825371, 2020-08-07T15:11:44.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68581815800000001, 2020-08-07T15:11:44.007 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:44.007 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:44.007 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:44.007 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:44.007 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:44.008 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89647 2020-08-07T15:11:44.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.008 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:44.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:44.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557315+0000", 2020-08-07T15:11:44.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:44.009 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.009 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.010 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:44.010 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557315+0000", 2020-08-07T15:11:44.010 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6639999999999999e-05 2020-08-07T15:11:44.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.011 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:44.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557341+0000", 2020-08-07T15:11:44.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7989999999999998e-06 2020-08-07T15:11:44.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:44.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557344+0000", 2020-08-07T15:11:44.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6960000000000002e-06 2020-08-07T15:11:44.012 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.012 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:44.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557350+0000", 2020-08-07T15:11:44.013 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6089999999999999e-06 2020-08-07T15:11:44.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.013 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:44.014 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557352+0000", 2020-08-07T15:11:44.014 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1837e-05 2020-08-07T15:11:44.014 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.014 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:44.015 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557384+0000", 2020-08-07T15:11:44.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8099999999999998e-06 2020-08-07T15:11:44.016 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.016 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:44.017 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557392+0000", 2020-08-07T15:11:44.017 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67581117300000004 2020-08-07T15:11:44.017 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:44.018 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233203+0000", 2020-08-07T15:11:44.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112316 2020-08-07T15:11:44.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.019 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.019 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:44.020 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233316+0000", 2020-08-07T15:11:44.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000221766 2020-08-07T15:11:44.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.020 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.020 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:44.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233537+0000", 2020-08-07T15:11:44.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069695019999999998 2020-08-07T15:11:44.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:44.022 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240507+0000", 2020-08-07T15:11:44.022 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3999999999999997e-07 2020-08-07T15:11:44.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.023 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:44.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240507+0000", 2020-08-07T15:11:44.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8844000000000003e-05 2020-08-07T15:11:44.023 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.062 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.062 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:44.062 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240546+0000", 2020-08-07T15:11:44.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023473399999999998 2020-08-07T15:11:44.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.064 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:44.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242893+0000", 2020-08-07T15:11:44.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5410000000000001e-05 2020-08-07T15:11:44.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.065 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:44.065 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242939+0000", 2020-08-07T15:11:44.065 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8000000000000001e-07 2020-08-07T15:11:44.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.098 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.099 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:44.099 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242939+0000", 2020-08-07T15:11:44.099 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019369600000000001 2020-08-07T15:11:44.099 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.100 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.100 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:44.100 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243133+0000", 2020-08-07T15:11:44.100 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68581815800000001 2020-08-07T15:11:44.100 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:44.101 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:44.101 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:44.101 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.101 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.101 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89648 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:11:44.125 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.557372+0000", 2020-08-07T15:11:44.126 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 21.479768249999999, 2020-08-07T15:11:44.126 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68606040000000001, 2020-08-07T15:11:44.126 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:44.126 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:44.127 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:44.127 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:44.127 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:44.127 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89648 2020-08-07T15:11:44.127 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.128 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:44.128 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.128 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:44.128 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557372+0000", 2020-08-07T15:11:44.128 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:44.129 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.135 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.145 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:44.145 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557372+0000", 2020-08-07T15:11:44.145 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5534e-05 2020-08-07T15:11:44.145 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.146 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.146 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:44.146 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557397+0000", 2020-08-07T15:11:44.146 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.599e-06 2020-08-07T15:11:44.146 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.147 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.147 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:44.147 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557400+0000", 2020-08-07T15:11:44.147 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.288e-06 2020-08-07T15:11:44.147 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.148 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.148 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:44.148 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557405+0000", 2020-08-07T15:11:44.148 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.142e-06 2020-08-07T15:11:44.148 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.149 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.149 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:44.149 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557407+0000", 2020-08-07T15:11:44.149 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0954999999999998e-05 2020-08-07T15:11:44.149 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.202 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.202 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:44.202 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557428+0000", 2020-08-07T15:11:44.203 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7529999999999998e-06 2020-08-07T15:11:44.203 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.203 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.203 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:44.203 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557434+0000", 2020-08-07T15:11:44.204 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67655471099999998 2020-08-07T15:11:44.204 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.204 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.204 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:44.204 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233989+0000", 2020-08-07T15:11:44.204 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8301999999999997e-05 2020-08-07T15:11:44.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.205 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.205 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:44.205 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234067+0000", 2020-08-07T15:11:44.205 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000129908 2020-08-07T15:11:44.216 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.217 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.218 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:44.218 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234197+0000", 2020-08-07T15:11:44.221 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063551249999999997 2020-08-07T15:11:44.231 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.232 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.232 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:44.232 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240552+0000", 2020-08-07T15:11:44.232 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9400000000000001e-07 2020-08-07T15:11:44.233 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.233 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.233 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:44.233 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240552+0000", 2020-08-07T15:11:44.233 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2787999999999998e-05 2020-08-07T15:11:44.234 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.234 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.234 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:44.258 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240585+0000", 2020-08-07T15:11:44.258 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002605652 2020-08-07T15:11:44.258 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.258 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.259 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:44.259 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243191+0000", 2020-08-07T15:11:44.259 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8690000000000002e-05 2020-08-07T15:11:44.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.259 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.260 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:44.260 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243250+0000", 2020-08-07T15:11:44.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4700000000000001e-07 2020-08-07T15:11:44.260 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.260 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.261 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:44.261 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243250+0000", 2020-08-07T15:11:44.261 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000181912 2020-08-07T15:11:44.261 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.261 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.278 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:44.279 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243432+0000", 2020-08-07T15:11:44.279 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68606040000000001 2020-08-07T15:11:44.279 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:44.280 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:44.281 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:44.281 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.281 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.289 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89689 5.7 5:e83ee43b:::benchmark_data_smithi074_53697_object11211:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:11:44.289 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.585493+0000", 2020-08-07T15:11:44.289 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 21.451647432000001, 2020-08-07T15:11:44.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66549445299999999, 2020-08-07T15:11:44.300 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:44.300 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:44.300 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:44.300 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:44.300 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:44.301 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89689 2020-08-07T15:11:44.301 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.301 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:44.301 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.301 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:44.302 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585493+0000", 2020-08-07T15:11:44.302 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:44.302 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.302 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.302 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:44.303 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585493+0000", 2020-08-07T15:11:44.303 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7746000000000002e-05 2020-08-07T15:11:44.303 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.303 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.303 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:44.304 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585520+0000", 2020-08-07T15:11:44.304 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1099999999999996e-06 2020-08-07T15:11:44.304 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.304 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.304 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:44.305 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585525+0000", 2020-08-07T15:11:44.305 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5359999999999998e-06 2020-08-07T15:11:44.305 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.305 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.305 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:44.339 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585534+0000", 2020-08-07T15:11:44.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5040000000000002e-06 2020-08-07T15:11:44.340 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.342 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.342 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:44.342 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585540+0000", 2020-08-07T15:11:44.342 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3548999999999998e-05 2020-08-07T15:11:44.342 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.343 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.343 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:44.343 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585593+0000", 2020-08-07T15:11:44.343 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2014000000000001e-05 2020-08-07T15:11:44.343 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.344 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.344 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:44.344 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585605+0000", 2020-08-07T15:11:44.344 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.64901847700000004 2020-08-07T15:11:44.344 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.345 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.345 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:44.345 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234624+0000", 2020-08-07T15:11:44.345 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000176329 2020-08-07T15:11:44.345 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.363 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.363 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:44.364 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234800+0000", 2020-08-07T15:11:44.364 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00025457300000000002 2020-08-07T15:11:44.364 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:44.365 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.235055+0000", 2020-08-07T15:11:44.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0055364669999999998 2020-08-07T15:11:44.365 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.365 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.365 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:44.371 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240591+0000", 2020-08-07T15:11:44.371 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3700000000000001e-07 2020-08-07T15:11:44.372 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.373 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.373 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:44.374 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240591+0000", 2020-08-07T15:11:44.374 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8479999999999997e-05 2020-08-07T15:11:44.374 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.374 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.374 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:44.375 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240640+0000", 2020-08-07T15:11:44.375 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00285564 2020-08-07T15:11:44.375 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.375 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:44.376 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243496+0000", 2020-08-07T15:11:44.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0073477400000000002 2020-08-07T15:11:44.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.376 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:44.376 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.250843+0000", 2020-08-07T15:11:44.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7300000000000005e-07 2020-08-07T15:11:44.377 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:44.385 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.250844+0000", 2020-08-07T15:11:44.385 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014307799999999999 2020-08-07T15:11:44.385 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.385 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.386 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:44.386 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.250987+0000", 2020-08-07T15:11:44.386 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66549445299999999 2020-08-07T15:11:44.386 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:44.386 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:44.387 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:44.387 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.387 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.387 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103152 5.7 5:e865d79a:::benchmark_data_smithi074_53697_object12893:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e211)", 2020-08-07T15:11:44.387 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.399967+0000", 2020-08-07T15:11:44.388 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.637173542999999, 2020-08-07T15:11:44.388 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81348256399999996, 2020-08-07T15:11:44.388 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:44.388 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:11:44.388 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:44.389 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:44.389 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:44.389 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103152 2020-08-07T15:11:44.389 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.389 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:44.424 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.424 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:44.425 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.399967+0000", 2020-08-07T15:11:44.477 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:44.477 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.478 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.478 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:44.478 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.399967+0000", 2020-08-07T15:11:44.478 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017528899999999999 2020-08-07T15:11:44.478 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.479 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.479 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:44.486 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400142+0000", 2020-08-07T15:11:44.487 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8239999999999996e-06 2020-08-07T15:11:44.487 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.487 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.487 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:44.488 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400148+0000", 2020-08-07T15:11:44.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0251000000000001e-05 2020-08-07T15:11:44.488 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:44.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400158+0000", 2020-08-07T15:11:44.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1579999999999998e-06 2020-08-07T15:11:44.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.489 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.489 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:44.493 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400162+0000", 2020-08-07T15:11:44.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00078404699999999996 2020-08-07T15:11:44.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.493 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.494 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:44.494 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400946+0000", 2020-08-07T15:11:44.494 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010605800000000001 2020-08-07T15:11:44.495 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.495 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.495 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:44.495 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.401052+0000", 2020-08-07T15:11:44.496 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020987499999999999 2020-08-07T15:11:44.496 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.496 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.496 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:44.497 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.401262+0000", 2020-08-07T15:11:44.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023707820000000001 2020-08-07T15:11:44.498 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.498 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:44.521 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.403633+0000", 2020-08-07T15:11:44.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.023e-06 2020-08-07T15:11:44.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.521 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.522 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:44.522 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.403634+0000", 2020-08-07T15:11:44.522 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3560999999999995e-05 2020-08-07T15:11:44.522 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.522 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.523 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:44.523 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.403717+0000", 2020-08-07T15:11:44.523 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011669689999999999 2020-08-07T15:11:44.523 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.523 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.524 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:44.524 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.404884+0000", 2020-08-07T15:11:44.524 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033332850000000001 2020-08-07T15:11:44.524 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.524 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.525 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:44.525 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.408218+0000", 2020-08-07T15:11:44.525 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0268000000000001e-05 2020-08-07T15:11:44.525 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.525 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.526 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:44.526 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.408228+0000", 2020-08-07T15:11:44.526 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80508180500000004 2020-08-07T15:11:44.526 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.527 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.527 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:44.527 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213310+0000", 2020-08-07T15:11:44.527 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000139369 2020-08-07T15:11:44.527 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.528 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:44.528 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213449+0000", 2020-08-07T15:11:44.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81348256399999996 2020-08-07T15:11:44.528 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:44.529 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:44.529 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:44.529 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.529 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.529 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103265 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:11:44.530 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478593+0000", 2020-08-07T15:11:44.530 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.558547487, 2020-08-07T15:11:44.530 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74922190499999997, 2020-08-07T15:11:44.530 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:44.531 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:44.531 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:44.531 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:44.531 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:44.532 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103265 2020-08-07T15:11:44.532 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.532 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:44.538 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.538 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:44.539 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478593+0000", 2020-08-07T15:11:44.539 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:44.539 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.539 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.539 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:44.540 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478593+0000", 2020-08-07T15:11:44.540 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9968000000000001e-05 2020-08-07T15:11:44.540 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.540 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.540 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:44.541 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478633+0000", 2020-08-07T15:11:44.541 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1420000000000003e-06 2020-08-07T15:11:44.541 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.541 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:44.542 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478637+0000", 2020-08-07T15:11:44.542 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2919999999999996e-06 2020-08-07T15:11:44.543 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.543 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.543 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:44.543 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478645+0000", 2020-08-07T15:11:44.544 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3699999999999997e-06 2020-08-07T15:11:44.544 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.544 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.544 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:44.544 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478649+0000", 2020-08-07T15:11:44.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1440000000000002e-05 2020-08-07T15:11:44.545 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.545 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:44.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478701+0000", 2020-08-07T15:11:44.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0831e-05 2020-08-07T15:11:44.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.546 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.546 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:44.546 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478712+0000", 2020-08-07T15:11:44.546 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73476696699999999 2020-08-07T15:11:44.547 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.547 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:44.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213479+0000", 2020-08-07T15:11:44.547 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000167026 2020-08-07T15:11:44.548 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.548 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:44.548 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213646+0000", 2020-08-07T15:11:44.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000179557 2020-08-07T15:11:44.549 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.549 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.549 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:44.550 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213825+0000", 2020-08-07T15:11:44.550 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041300360000000001 2020-08-07T15:11:44.550 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.550 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.550 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:44.551 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217955+0000", 2020-08-07T15:11:44.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6840000000000001e-06 2020-08-07T15:11:44.551 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.559 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.559 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:44.560 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217957+0000", 2020-08-07T15:11:44.560 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4758e-05 2020-08-07T15:11:44.560 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.570 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.570 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:44.571 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218012+0000", 2020-08-07T15:11:44.571 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0095519990000000003 2020-08-07T15:11:44.571 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.571 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.571 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:44.572 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227564+0000", 2020-08-07T15:11:44.572 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9956e-05 2020-08-07T15:11:44.572 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.572 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.572 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:44.573 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227624+0000", 2020-08-07T15:11:44.573 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3790000000000001e-06 2020-08-07T15:11:44.573 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.573 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.573 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:44.576 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227625+0000", 2020-08-07T15:11:44.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0001895 2020-08-07T15:11:44.576 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.576 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.577 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:44.577 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227815+0000", 2020-08-07T15:11:44.577 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74922190499999997 2020-08-07T15:11:44.578 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:44.578 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:44.578 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:44.578 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.578 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.579 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103266 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:11:44.579 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478693+0000", 2020-08-07T15:11:44.579 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.558447146000001, 2020-08-07T15:11:44.579 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74942137399999997, 2020-08-07T15:11:44.579 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:44.580 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:44.580 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:44.580 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:44.580 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:44.580 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103266 2020-08-07T15:11:44.581 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.581 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:44.581 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.581 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:44.582 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478693+0000", 2020-08-07T15:11:44.582 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:44.582 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.582 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.582 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:44.583 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478693+0000", 2020-08-07T15:11:44.583 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5658000000000002e-05 2020-08-07T15:11:44.583 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.583 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.583 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:44.584 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478739+0000", 2020-08-07T15:11:44.584 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.113e-06 2020-08-07T15:11:44.584 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.584 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.584 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:44.585 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478744+0000", 2020-08-07T15:11:44.585 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9690000000000006e-06 2020-08-07T15:11:44.585 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.585 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.585 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:44.586 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478754+0000", 2020-08-07T15:11:44.586 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1849999999999997e-06 2020-08-07T15:11:44.586 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.586 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.586 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:44.587 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478758+0000", 2020-08-07T15:11:44.587 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7024999999999998e-05 2020-08-07T15:11:44.587 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.587 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.587 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:44.587 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478805+0000", 2020-08-07T15:11:44.588 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1357e-05 2020-08-07T15:11:44.588 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.588 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:44.589 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478816+0000", 2020-08-07T15:11:44.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73554277499999998 2020-08-07T15:11:44.589 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.589 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:44.590 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.214359+0000", 2020-08-07T15:11:44.590 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000129145 2020-08-07T15:11:44.590 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.590 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.590 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:44.590 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.214488+0000", 2020-08-07T15:11:44.591 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00027129999999999998 2020-08-07T15:11:44.591 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.591 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.591 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:44.591 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.214760+0000", 2020-08-07T15:11:44.592 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032631079999999998 2020-08-07T15:11:44.592 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.592 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.592 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:44.592 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218023+0000", 2020-08-07T15:11:44.593 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1000000000000001e-06 2020-08-07T15:11:44.593 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.593 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.593 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:44.593 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218024+0000", 2020-08-07T15:11:44.594 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9085000000000006e-05 2020-08-07T15:11:44.594 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.594 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.594 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:44.594 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218103+0000", 2020-08-07T15:11:44.595 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.009779088 2020-08-07T15:11:44.596 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.596 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.596 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:44.597 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227882+0000", 2020-08-07T15:11:44.597 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1989999999999994e-05 2020-08-07T15:11:44.597 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.597 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.598 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:44.598 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227944+0000", 2020-08-07T15:11:44.598 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8299999999999996e-07 2020-08-07T15:11:44.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.601 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:44.602 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227945+0000", 2020-08-07T15:11:44.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000169693 2020-08-07T15:11:44.602 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.602 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.602 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:44.603 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228114+0000", 2020-08-07T15:11:44.603 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74942137399999997 2020-08-07T15:11:44.603 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:44.603 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:44.603 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:44.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.604 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103267 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:11:44.604 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478794+0000", 2020-08-07T15:11:44.604 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.558345658, 2020-08-07T15:11:44.605 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74960417300000004, 2020-08-07T15:11:44.617 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:44.618 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:44.618 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:44.618 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:44.618 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:44.618 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103267 2020-08-07T15:11:44.619 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.619 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:44.619 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.619 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:44.620 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478794+0000", 2020-08-07T15:11:44.627 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:44.628 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.628 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.628 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:44.628 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478794+0000", 2020-08-07T15:11:44.628 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6646e-05 2020-08-07T15:11:44.629 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.629 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.639 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:44.639 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478841+0000", 2020-08-07T15:11:44.639 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:11:44.640 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.640 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.640 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:44.640 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478846+0000", 2020-08-07T15:11:44.640 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0171e-05 2020-08-07T15:11:44.641 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.641 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.641 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:44.641 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478856+0000", 2020-08-07T15:11:44.641 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5929999999999999e-06 2020-08-07T15:11:44.642 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.642 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.642 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:44.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478861+0000", 2020-08-07T15:11:44.642 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2032e-05 2020-08-07T15:11:44.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.643 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.643 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:44.643 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478903+0000", 2020-08-07T15:11:44.643 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0312000000000001e-05 2020-08-07T15:11:44.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.644 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.644 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:44.644 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478913+0000", 2020-08-07T15:11:44.644 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73634238600000002 2020-08-07T15:11:44.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:44.645 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.215256+0000", 2020-08-07T15:11:44.645 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000124933 2020-08-07T15:11:44.665 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.666 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.666 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:44.666 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.215380+0000", 2020-08-07T15:11:44.666 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024624 2020-08-07T15:11:44.666 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.667 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:44.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.215627+0000", 2020-08-07T15:11:44.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0040879540000000004 2020-08-07T15:11:44.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:44.668 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219715+0000", 2020-08-07T15:11:44.668 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9230000000000001e-06 2020-08-07T15:11:44.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:44.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219717+0000", 2020-08-07T15:11:44.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8881999999999995e-05 2020-08-07T15:11:44.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.680 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:44.680 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219785+0000", 2020-08-07T15:11:44.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0083925119999999995 2020-08-07T15:11:44.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:44.681 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228178+0000", 2020-08-07T15:11:44.681 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2254000000000004e-05 2020-08-07T15:11:44.681 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.681 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.683 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:44.683 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228240+0000", 2020-08-07T15:11:44.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4499999999999996e-07 2020-08-07T15:11:44.684 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.684 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.684 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:44.685 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228241+0000", 2020-08-07T15:11:44.685 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000157656 2020-08-07T15:11:44.685 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.685 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.685 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:44.686 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228399+0000", 2020-08-07T15:11:44.686 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74960417300000004 2020-08-07T15:11:44.693 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:44.694 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:44.694 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:44.694 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.695 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103268 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:11:44.695 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478898+0000", 2020-08-07T15:11:44.695 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.558241941, 2020-08-07T15:11:44.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74981275700000005, 2020-08-07T15:11:44.696 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:44.696 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:44.696 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:44.696 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:44.696 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:44.697 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103268 2020-08-07T15:11:44.697 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.697 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:44.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:44.698 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478898+0000", 2020-08-07T15:11:44.707 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:44.708 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.708 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:44.708 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478898+0000", 2020-08-07T15:11:44.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2558999999999997e-05 2020-08-07T15:11:44.709 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.709 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.709 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:44.709 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478941+0000", 2020-08-07T15:11:44.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6469999999999997e-06 2020-08-07T15:11:44.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.718 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.718 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:44.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478945+0000", 2020-08-07T15:11:44.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3109999999999995e-06 2020-08-07T15:11:44.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:44.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478955+0000", 2020-08-07T15:11:44.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.799e-06 2020-08-07T15:11:44.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:44.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478958+0000", 2020-08-07T15:11:44.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5682000000000002e-05 2020-08-07T15:11:44.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.721 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:44.727 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:11:44.727 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3260000000000002e-06 2020-08-07T15:11:44.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.734 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.735 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:44.735 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479004+0000", 2020-08-07T15:11:44.735 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73703730499999998 2020-08-07T15:11:44.735 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.736 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.736 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:44.746 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216041+0000", 2020-08-07T15:11:44.747 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4445999999999998e-05 2020-08-07T15:11:44.747 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.747 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.747 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:44.747 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216115+0000", 2020-08-07T15:11:44.748 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013977 2020-08-07T15:11:44.748 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.748 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.748 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:44.748 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216255+0000", 2020-08-07T15:11:44.749 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0051267359999999998 2020-08-07T15:11:44.749 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.749 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.764 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:44.782 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221382+0000", 2020-08-07T15:11:44.783 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0310000000000001e-06 2020-08-07T15:11:44.783 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:44.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221384+0000", 2020-08-07T15:11:44.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8097999999999998e-05 2020-08-07T15:11:44.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:44.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221452+0000", 2020-08-07T15:11:44.785 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0070259249999999997 2020-08-07T15:11:44.785 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:44.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228478+0000", 2020-08-07T15:11:44.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7479999999999999e-05 2020-08-07T15:11:44.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:44.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228535+0000", 2020-08-07T15:11:44.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4399999999999999e-07 2020-08-07T15:11:44.821 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:44.821 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228536+0000", 2020-08-07T15:11:44.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017489799999999999 2020-08-07T15:11:44.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.822 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.822 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:44.822 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228711+0000", 2020-08-07T15:11:44.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74981275700000005 2020-08-07T15:11:44.823 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:44.823 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:44.823 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:44.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.824 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103269 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:11:44.824 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:11:44.824 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.558145832999999, 2020-08-07T15:11:44.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74992452799999998, 2020-08-07T15:11:44.825 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:44.825 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:44.825 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:44.825 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:44.825 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:44.869 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103269 2020-08-07T15:11:44.869 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.869 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:44.870 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.870 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:44.870 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:11:44.870 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:44.870 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.871 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:44.871 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:11:44.871 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5529999999999999e-05 2020-08-07T15:11:44.871 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.872 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.872 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:44.872 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479040+0000", 2020-08-07T15:11:44.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6550000000000003e-06 2020-08-07T15:11:44.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.873 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:44.873 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479044+0000", 2020-08-07T15:11:44.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4285e-05 2020-08-07T15:11:44.873 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.876 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:44.877 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479059+0000", 2020-08-07T15:11:44.877 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2050000000000001e-06 2020-08-07T15:11:44.877 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.877 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.878 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:44.878 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479064+0000", 2020-08-07T15:11:44.878 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0952999999999999e-05 2020-08-07T15:11:44.878 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.878 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.879 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:44.879 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479105+0000", 2020-08-07T15:11:44.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3559999999999998e-06 2020-08-07T15:11:44.879 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:44.890 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479111+0000", 2020-08-07T15:11:44.890 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.737508104 2020-08-07T15:11:44.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.890 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:44.891 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216619+0000", 2020-08-07T15:11:44.891 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000126367 2020-08-07T15:11:44.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.891 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.891 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:44.892 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216746+0000", 2020-08-07T15:11:44.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014069899999999999 2020-08-07T15:11:44.892 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.892 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.892 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:44.893 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216886+0000", 2020-08-07T15:11:44.893 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045753160000000003 2020-08-07T15:11:44.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.893 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:44.895 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221462+0000", 2020-08-07T15:11:44.895 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.142e-06 2020-08-07T15:11:44.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.896 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:44.896 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221463+0000", 2020-08-07T15:11:44.896 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7718e-05 2020-08-07T15:11:44.897 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.897 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.897 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:44.897 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221501+0000", 2020-08-07T15:11:44.897 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0072733210000000001 2020-08-07T15:11:44.898 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.898 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.898 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:44.898 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228774+0000", 2020-08-07T15:11:44.898 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3828e-05 2020-08-07T15:11:44.899 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.899 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.899 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:44.899 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228828+0000", 2020-08-07T15:11:44.899 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9800000000000003e-07 2020-08-07T15:11:44.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.900 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:44.900 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228828+0000", 2020-08-07T15:11:44.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0451000000000005e-05 2020-08-07T15:11:44.901 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.901 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.901 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:44.901 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228919+0000", 2020-08-07T15:11:44.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74992452799999998 2020-08-07T15:11:44.902 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:44.902 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:44.902 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:44.903 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.903 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103270 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:11:44.903 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.479168+0000", 2020-08-07T15:11:44.903 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.557971796, 2020-08-07T15:11:44.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75043371699999994, 2020-08-07T15:11:44.904 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:44.904 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:44.904 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:44.904 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:44.905 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:44.905 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103270 2020-08-07T15:11:44.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.905 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:44.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.906 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:44.906 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479168+0000", 2020-08-07T15:11:44.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:44.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.906 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:44.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479168+0000", 2020-08-07T15:11:44.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1557000000000002e-05 2020-08-07T15:11:44.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.908 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:44.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479210+0000", 2020-08-07T15:11:44.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9840000000000001e-06 2020-08-07T15:11:44.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:44.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479215+0000", 2020-08-07T15:11:44.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9539999999999999e-06 2020-08-07T15:11:44.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:44.940 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479225+0000", 2020-08-07T15:11:44.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.942e-06 2020-08-07T15:11:44.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.940 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.940 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:44.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479231+0000", 2020-08-07T15:11:44.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2493000000000002e-05 2020-08-07T15:11:44.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:44.951 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479263+0000", 2020-08-07T15:11:44.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6640000000000003e-06 2020-08-07T15:11:44.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.952 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:44.952 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479269+0000", 2020-08-07T15:11:44.952 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73802486499999997 2020-08-07T15:11:44.952 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.952 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.953 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:44.953 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217294+0000", 2020-08-07T15:11:44.953 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8640000000000006e-05 2020-08-07T15:11:44.953 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:44.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217372+0000", 2020-08-07T15:11:44.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000140338 2020-08-07T15:11:44.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.962 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:44.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217513+0000", 2020-08-07T15:11:44.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039952119999999997 2020-08-07T15:11:44.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:44.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221508+0000", 2020-08-07T15:11:44.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8899999999999999e-07 2020-08-07T15:11:44.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:44.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221509+0000", 2020-08-07T15:11:44.969 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8108999999999997e-05 2020-08-07T15:11:44.969 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.969 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.970 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:44.970 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221547+0000", 2020-08-07T15:11:44.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074232029999999997 2020-08-07T15:11:44.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:44.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228970+0000", 2020-08-07T15:11:44.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00046710099999999999 2020-08-07T15:11:44.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:44.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229437+0000", 2020-08-07T15:11:44.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5400000000000001e-07 2020-08-07T15:11:44.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:44.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229438+0000", 2020-08-07T15:11:44.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016441200000000001 2020-08-07T15:11:44.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:44.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229602+0000", 2020-08-07T15:11:44.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75043371699999994 2020-08-07T15:11:44.978 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:44.978 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:44.979 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:44.979 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.979 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.980 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103271 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:11:44.980 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.479356+0000", 2020-08-07T15:11:44.981 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.557783828, 2020-08-07T15:11:44.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75047653800000003, 2020-08-07T15:11:44.981 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:44.982 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:44.983 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:44.983 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:44.984 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:44.984 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103271 2020-08-07T15:11:44.985 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.985 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:44.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.986 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:44.987 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479356+0000", 2020-08-07T15:11:44.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:44.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:44.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479356+0000", 2020-08-07T15:11:44.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2416000000000002e-05 2020-08-07T15:11:44.993 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.993 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.994 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:44.994 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479399+0000", 2020-08-07T15:11:44.995 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0819999999999998e-06 2020-08-07T15:11:44.995 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.996 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.996 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:44.996 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479404+0000", 2020-08-07T15:11:44.997 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2486999999999999e-05 2020-08-07T15:11:44.997 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:44.998 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:44.998 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:45.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479416+0000", 2020-08-07T15:11:45.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3810000000000001e-06 2020-08-07T15:11:45.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:45.022 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479421+0000", 2020-08-07T15:11:45.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2311999999999998e-05 2020-08-07T15:11:45.023 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.023 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:45.034 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479453+0000", 2020-08-07T15:11:45.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7690000000000002e-06 2020-08-07T15:11:45.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:45.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479459+0000", 2020-08-07T15:11:45.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73869566499999995 2020-08-07T15:11:45.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:45.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218154+0000", 2020-08-07T15:11:45.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000124535 2020-08-07T15:11:45.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.037 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:45.037 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218279+0000", 2020-08-07T15:11:45.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000223441 2020-08-07T15:11:45.054 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.054 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.078 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:45.079 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218502+0000", 2020-08-07T15:11:45.079 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003054761 2020-08-07T15:11:45.079 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.079 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.079 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:45.080 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221557+0000", 2020-08-07T15:11:45.080 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0449999999999999e-06 2020-08-07T15:11:45.080 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.080 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.080 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:45.081 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221558+0000", 2020-08-07T15:11:45.081 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3655e-05 2020-08-07T15:11:45.081 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.081 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.083 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:45.084 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221622+0000", 2020-08-07T15:11:45.084 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.007661434 2020-08-07T15:11:45.084 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.085 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:45.085 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229283+0000", 2020-08-07T15:11:45.085 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00036023099999999997 2020-08-07T15:11:45.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.086 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.086 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:45.086 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229644+0000", 2020-08-07T15:11:45.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.63e-07 2020-08-07T15:11:45.086 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:45.087 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229644+0000", 2020-08-07T15:11:45.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018886099999999999 2020-08-07T15:11:45.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:45.088 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229833+0000", 2020-08-07T15:11:45.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75047653800000003 2020-08-07T15:11:45.088 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:45.088 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:45.100 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:45.101 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.101 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.109 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103272 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:11:45.110 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.481658+0000", 2020-08-07T15:11:45.110 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.555482404999999, 2020-08-07T15:11:45.110 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74920410900000001, 2020-08-07T15:11:45.128 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:45.140 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:45.140 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:45.141 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:45.141 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:45.141 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103272 2020-08-07T15:11:45.141 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.141 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:45.142 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.142 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:45.142 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481658+0000", 2020-08-07T15:11:45.142 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:45.142 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.143 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.143 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:45.143 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481658+0000", 2020-08-07T15:11:45.143 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8787e-05 2020-08-07T15:11:45.143 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.144 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.144 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:45.144 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481697+0000", 2020-08-07T15:11:45.144 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0559999999999998e-06 2020-08-07T15:11:45.144 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.145 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.145 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:45.146 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481701+0000", 2020-08-07T15:11:45.146 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1329999999999999e-06 2020-08-07T15:11:45.158 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.158 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.159 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:45.159 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481709+0000", 2020-08-07T15:11:45.159 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.189e-06 2020-08-07T15:11:45.170 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.171 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.171 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:45.171 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481713+0000", 2020-08-07T15:11:45.171 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6025999999999997e-05 2020-08-07T15:11:45.174 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.174 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.174 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:45.175 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481749+0000", 2020-08-07T15:11:45.175 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5619999999999998e-06 2020-08-07T15:11:45.175 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.175 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.175 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:45.176 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481756+0000", 2020-08-07T15:11:45.176 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73729155000000002 2020-08-07T15:11:45.176 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.176 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.176 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:45.177 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219048+0000", 2020-08-07T15:11:45.177 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000120387 2020-08-07T15:11:45.177 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.198 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.198 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:45.199 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219168+0000", 2020-08-07T15:11:45.199 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017865599999999999 2020-08-07T15:11:45.199 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.199 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.199 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:45.200 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219347+0000", 2020-08-07T15:11:45.200 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039049169999999999 2020-08-07T15:11:45.200 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.212 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.212 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:45.213 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.223252+0000", 2020-08-07T15:11:45.213 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.412e-06 2020-08-07T15:11:45.213 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.213 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.214 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:45.214 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.223253+0000", 2020-08-07T15:11:45.214 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7521999999999997e-05 2020-08-07T15:11:45.214 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.214 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.215 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:45.215 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.223301+0000", 2020-08-07T15:11:45.215 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00603478 2020-08-07T15:11:45.215 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.215 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.216 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:45.216 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229336+0000", 2020-08-07T15:11:45.216 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013309579999999999 2020-08-07T15:11:45.216 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.216 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.216 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:45.217 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.230667+0000", 2020-08-07T15:11:45.217 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7400000000000003e-07 2020-08-07T15:11:45.217 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.217 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.217 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:45.218 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.230667+0000", 2020-08-07T15:11:45.218 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019450000000000001 2020-08-07T15:11:45.218 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:45.219 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:45.219 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:45.219 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.230862+0000", 2020-08-07T15:11:45.219 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74920410900000001 2020-08-07T15:11:45.219 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:45.233 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:45.233 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:45.233 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:45.234 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:45.234 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:11:45.235 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:11:45.235 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:11:45.284 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:11:45.285 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:11:45.285 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:11:45.307 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:11:45.307 INFO:teuthology.orchestra.run.smithi074.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-08-07T15:11:45.307 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: problem getting command descriptions from osd.1 2020-08-07T15:11:45.308 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:11:45.308 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:11:45.308 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:11:45.308 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:11:45.308 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:11:45.309 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:11:45.309 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-08-07T15:11:45.309 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-08-07T15:11:45.310 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 83 16 118448 118432 11.1465 2.32812 0.733116 0.011098 2020-08-07T15:11:45.310 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 84 16 118448 118432 11.0138 0 - 0.011098 2020-08-07T15:11:45.310 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 85 16 119842 119826 11.0123 5.44531 0.00524526 0.0113451 2020-08-07T15:11:45.311 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:11:45.314 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:45.245+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:45.361 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:45.359+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:45.466 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:45.463+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:45.569 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:45.566+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:45.672 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:45.671+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:45.792 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:45.791+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:45.895 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:45.894+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:46.017 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:46.016+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:46.126 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:46.125+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:46.222 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 86 16 121538 121522 11.0384 13.25 0.0335753 0.0113191 2020-08-07T15:11:46.227 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:46.226+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:46.329 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:46.328+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:46.459 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:46.446+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:46.551 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:46.549+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:46.654 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:46.651+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:46.753 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:46.753+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:46.861 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:46.860+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:46.992 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:46.984+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:47.095 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:47.094+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:47.212 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:47.211+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:47.235 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3, 4, 5, 6, 7] 2020-08-07T15:11:47.235 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:11:47.236 INFO:tasks.thrashosds.thrasher:inject_pause on 7 2020-08-07T15:11:47.236 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2020-08-07T15:11:47.237 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2020-08-07T15:11:47.261 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2020-08-07T15:11:47.265 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 87 16 123133 123117 11.0547 12.4609 0.00850871 0.0113029 2020-08-07T15:11:47.311 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:11:47.316 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:11:47.331 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:11:47.332 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:11:47.332 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:47.332 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:123128 5.a 5:5ef3b8d8:::benchmark_data_smithi074_53697_object15390:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e219)", 2020-08-07T15:11:47.333 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:47.214131+0000", 2020-08-07T15:11:47.333 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.098827781000000003, 2020-08-07T15:11:47.333 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.098858857999999994, 2020-08-07T15:11:47.334 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:47.334 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:47.334 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:47.334 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:47.334 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:47.335 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 123128 2020-08-07T15:11:47.335 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:47.335 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:47.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:47.335 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:47.336 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:47.214131+0000", 2020-08-07T15:11:47.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:47.336 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:47.336 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:47.336 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:47.337 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:47.214131+0000", 2020-08-07T15:11:47.337 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9787000000000001e-05 2020-08-07T15:11:47.337 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:47.346 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:47.346 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:47.347 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:47.214160+0000", 2020-08-07T15:11:47.347 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9489999999999998e-06 2020-08-07T15:11:47.347 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:47.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:47.347 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:47.348 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:47.214164+0000", 2020-08-07T15:11:47.348 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5460000000000003e-06 2020-08-07T15:11:47.348 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:47.354 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:47.354 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:47.354 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:47.214172+0000", 2020-08-07T15:11:47.354 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3069999999999996e-06 2020-08-07T15:11:47.355 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:47.363 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:47.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:47.364 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:47.214176+0000", 2020-08-07T15:11:47.364 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00051378299999999997 2020-08-07T15:11:47.366 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:47.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:47.367 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:47.369 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:47.214690+0000", 2020-08-07T15:11:47.369 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8752999999999998e-05 2020-08-07T15:11:47.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:47.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:47.372 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:47.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:47.214749+0000", 2020-08-07T15:11:47.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010538 2020-08-07T15:11:47.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:47.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:47.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4", 2020-08-07T15:11:47.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:47.214854+0000", 2020-08-07T15:11:47.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044224160000000002 2020-08-07T15:11:47.379 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:47.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:47.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:47.380 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:47.219276+0000", 2020-08-07T15:11:47.380 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6499999999999999e-07 2020-08-07T15:11:47.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:47.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:47.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:47.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:47.219277+0000", 2020-08-07T15:11:47.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2524999999999998e-05 2020-08-07T15:11:47.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:47.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:47.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:47.382 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:47.219309+0000", 2020-08-07T15:11:47.382 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0047069219999999997 2020-08-07T15:11:47.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:47.382 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:47.382 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:47.383 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:47.224016+0000", 2020-08-07T15:11:47.383 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0098858330000000001 2020-08-07T15:11:47.388 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:47.388 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:47.388 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:47.389 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:47.389 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:47.389 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5099.0:123276 5.4 e219/164 5:21bee938:::benchmark_data_smithi074_53697_object15409:head v 219'7580, mlcod=219'7575)", 2020-08-07T15:11:47.389 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:47.312484+0000", 2020-08-07T15:11:47.389 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.00047439200000000002, 2020-08-07T15:11:47.390 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00062010699999999995, 2020-08-07T15:11:47.390 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:47.390 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:11:47.390 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:47.390 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:47.413 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:47.413 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:47.312484+0000", 2020-08-07T15:11:47.413 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:47.420 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:47.420 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:47.420 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:47.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:47.312484+0000", 2020-08-07T15:11:47.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5674000000000003e-05 2020-08-07T15:11:47.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:47.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:47.436 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:47.437 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:47.312540+0000", 2020-08-07T15:11:47.437 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.737e-06 2020-08-07T15:11:47.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:47.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:47.447 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:47.447 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:47.312546+0000", 2020-08-07T15:11:47.447 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8666e-05 2020-08-07T15:11:47.448 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:47.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:47.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:47.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:47.312575+0000", 2020-08-07T15:11:47.448 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5039999999999996e-06 2020-08-07T15:11:47.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:47.449 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:47.449 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:47.449 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:47.312582+0000", 2020-08-07T15:11:47.449 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5339000000000002e-05 2020-08-07T15:11:47.454 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:47.454 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:47.454 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:47.454 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:47.312618+0000", 2020-08-07T15:11:47.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1118999999999999e-05 2020-08-07T15:11:47.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:47.465 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:47.465 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:47.465 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:47.312639+0000", 2020-08-07T15:11:47.470 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015503899999999999 2020-08-07T15:11:47.471 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:47.471 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:47.471 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:47.471 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:47.472 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:11:47.472 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 2 2020-08-07T15:11:47.472 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:11:47.481 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:11:47.491 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:47.315+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:47.492 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:11:47.492 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "" 2020-08-07T15:11:47.492 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:11:47.494 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:47.481+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:47.594 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:47.593+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:47.677 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:11:47.678 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:11:47.678 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:11:47.678 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:11:47.678 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:11:47.755 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:11:47.755 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:47.753+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:47.756 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:11:47.804 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:11:47.862 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:11:47.863 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:47.862+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:47.914 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:11:47.946 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:47.947 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:47.947 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:11:47.992 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:11:47.993 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:11:48.026 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:47.990+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:48.027 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:11:48.028 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:11:48.028 INFO:teuthology.orchestra.run.smithi074.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-08-07T15:11:48.028 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: problem getting command descriptions from osd.1 2020-08-07T15:11:48.028 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:11:48.028 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:11:48.029 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:11:48.029 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:11:48.029 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:11:48.029 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:11:48.051 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-08-07T15:11:48.051 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-08-07T15:11:48.101 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:48.098+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:48.190 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:48.191 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:48.209 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:11:48.265 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 88 16 124482 124466 11.0488 10.5391 0.00407273 0.0113087 2020-08-07T15:11:48.279 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:11:48.280 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:48.208+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:48.315 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:48.314+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:48.419 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:48.417+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:48.445 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:48.446 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:48.463 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:11:48.534 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:48.523+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:48.605 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:48.606 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:48.616 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:11:48.629 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:48.627+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:48.732 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:48.729+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:48.812 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:48.812 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:48.822 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:11:48.833 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:48.832+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:48.938 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:48.932+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:49.000 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:49.000 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:49.028 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:11:49.044 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:49.038+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:49.145 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:49.139+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:49.220 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 89 16 125633 125617 11.0257 8.99219 0.012114 0.0113316 2020-08-07T15:11:49.245 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:49.246 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:49.248 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:11:49.249 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:49.245+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:49.350 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:49.348+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:49.452 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:49.448+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:49.455 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:49.456 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:49.470 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:11:49.601 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:49.565+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:49.682 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:49.668+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:49.702 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:49.703 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:49.711 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:11:49.816 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:49.808+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:49.874 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:49.869+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:49.916 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:49.917 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:49.931 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:11:49.972 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:49.971+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:50.054 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:11:50.074 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:50.072+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:50.170 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:50.171 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:50.182 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:50.172+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:50.183 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:11:50.218 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 90 7 126918 126911 11.0155 10.1094 0.0217434 0.0113431 2020-08-07T15:11:50.278 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:50.273+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:50.281 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:11:50.370 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:50.371 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:50.379 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:11:50.380 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:50.374+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:50.476 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:50.475+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:50.518 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:50.518 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:50.535 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:11:50.594 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:50.590+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:50.677 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:50.678 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:50.686 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:50.681+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:50.687 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:11:50.786 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:50.785+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:50.856 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:50.856 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:50.865 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:11:50.891 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:50.889+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:50.994 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:50.993+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:51.005 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:51.006 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:51.012 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:11:51.096 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:51.095+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:51.158 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:51.158 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:51.159 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:11:51.175 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:11:51.193 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:11:51.220 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:11:51.228 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Total time run: 90.7818 2020-08-07T15:11:51.228 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Total writes made: 126918 2020-08-07T15:11:51.237 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Write size: 8192 2020-08-07T15:11:51.237 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Object size: 65536 2020-08-07T15:11:51.237 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Bandwidth (MB/sec): 10.9223 2020-08-07T15:11:51.238 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Stddev Bandwidth: 3.11328 2020-08-07T15:11:51.238 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Max bandwidth (MB/sec): 13.9609 2020-08-07T15:11:51.238 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Min bandwidth (MB/sec): 0 2020-08-07T15:11:51.238 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Average IOPS: 1398 2020-08-07T15:11:51.239 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Stddev IOPS: 398.5 2020-08-07T15:11:51.239 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Max IOPS: 1787 2020-08-07T15:11:51.239 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Min IOPS: 0 2020-08-07T15:11:51.239 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Average Latency(s): 0.0113523 2020-08-07T15:11:51.239 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Stddev Latency(s): 0.0390823 2020-08-07T15:11:51.240 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Max latency(s): 2.06045 2020-08-07T15:11:51.240 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Min latency(s): 0.000974592 2020-08-07T15:11:51.240 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:51.214+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:51.255 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:11:51.256 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Cleaning up (deleting benchmark objects) 2020-08-07T15:11:51.290 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:11:51.291 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:11:51.319 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:51.316+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:51.327 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:11:51.344 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:11:51.345 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:11:51.345 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:11:51.345 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:11:51.345 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:11:51.356 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:11:51.357 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:11:51.357 INFO:teuthology.orchestra.run.smithi074.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-08-07T15:11:51.357 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: problem getting command descriptions from osd.1 2020-08-07T15:11:51.357 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:11:51.359 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:11:51.359 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:11:51.359 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:11:51.359 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:11:51.359 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:11:51.360 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-08-07T15:11:51.360 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-08-07T15:11:51.362 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:11:51.442 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:51.441+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:51.533 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:51.530+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:51.589 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:11:51.593 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:11:51.593 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:11:51.594 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:11:51.594 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.594 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:145801 3.6 e75/26 3:69e4cb22:::benchmark_data_smithi074_35447_object16192:head v 75'9405, mlcod=75'9401)", 2020-08-07T15:11:51.594 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:11:51.595 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 216.90768773799999, 2020-08-07T15:11:51.595 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639, 2020-08-07T15:11:51.595 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:51.595 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:51.595 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:51.596 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.596 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:51.596 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:11:51.596 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:51.596 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.597 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.597 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:51.597 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:11:51.597 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6511000000000002e-05 2020-08-07T15:11:51.597 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.598 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.598 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:51.599 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671803+0000", 2020-08-07T15:11:51.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7190000000000004e-06 2020-08-07T15:11:51.599 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.599 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.599 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:51.600 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671809+0000", 2020-08-07T15:11:51.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74210928300000001 2020-08-07T15:11:51.600 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.600 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.612 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:51.612 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413919+0000", 2020-08-07T15:11:51.613 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1759999999999999e-05 2020-08-07T15:11:51.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.613 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:51.614 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413930+0000", 2020-08-07T15:11:51.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000108601 2020-08-07T15:11:51.614 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.614 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.614 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:51.615 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414039+0000", 2020-08-07T15:11:51.615 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9948999999999999e-05 2020-08-07T15:11:51.615 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.615 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.615 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:51.615 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414089+0000", 2020-08-07T15:11:51.616 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00079656499999999995 2020-08-07T15:11:51.616 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.634 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.634 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:51.634 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414885+0000", 2020-08-07T15:11:51.634 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2250999999999998e-05 2020-08-07T15:11:51.649 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.650 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.650 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:51.650 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414938+0000", 2020-08-07T15:11:51.650 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639 2020-08-07T15:11:51.651 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:51.651 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:51.651 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:51.651 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.651 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.652 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:104104 4.0 e146/131 4:0ffb764a:::benchmark_data_smithi074_44586_object13012:head v 146'6960, mlcod=146'6953)", 2020-08-07T15:11:51.652 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:11:51.652 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 123.381140962, 2020-08-07T15:11:51.652 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75443331599999996, 2020-08-07T15:11:51.652 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:51.653 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:51.653 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:51.653 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.653 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:51.653 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:11:51.663 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:51.663 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.664 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:51.664 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:11:51.664 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1743000000000002e-05 2020-08-07T15:11:51.665 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.665 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.665 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:51.665 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198355+0000", 2020-08-07T15:11:51.665 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1249999999999998e-06 2020-08-07T15:11:51.666 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.666 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.666 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:51.666 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198361+0000", 2020-08-07T15:11:51.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74719179599999996 2020-08-07T15:11:51.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.667 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:51.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945553+0000", 2020-08-07T15:11:51.668 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6800000000000001e-06 2020-08-07T15:11:51.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:51.668 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945557+0000", 2020-08-07T15:11:51.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012101500000000001 2020-08-07T15:11:51.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:51.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945678+0000", 2020-08-07T15:11:51.693 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6621000000000001e-05 2020-08-07T15:11:51.693 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.693 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.693 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:51.693 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945705+0000", 2020-08-07T15:11:51.694 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069879540000000002 2020-08-07T15:11:51.694 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.694 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.694 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:51.694 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.952693+0000", 2020-08-07T15:11:51.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3381999999999999e-05 2020-08-07T15:11:51.695 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.695 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:51.695 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.952736+0000", 2020-08-07T15:11:51.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75443331599999996 2020-08-07T15:11:51.696 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:51.696 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:51.696 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:51.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.697 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89641 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:11:51.697 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.550933+0000", 2020-08-07T15:11:51.697 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 30.028510540999999, 2020-08-07T15:11:51.697 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69051440900000005, 2020-08-07T15:11:51.715 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:51.716 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:51.716 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:51.716 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:51.716 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:51.716 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89641 2020-08-07T15:11:51.717 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.717 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:51.717 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.717 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:51.717 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550933+0000", 2020-08-07T15:11:51.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:51.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.718 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.718 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:51.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550933+0000", 2020-08-07T15:11:51.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6520000000000002e-05 2020-08-07T15:11:51.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:51.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550980+0000", 2020-08-07T15:11:51.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2410000000000003e-06 2020-08-07T15:11:51.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:51.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550985+0000", 2020-08-07T15:11:51.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2707e-05 2020-08-07T15:11:51.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.721 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:51.722 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550998+0000", 2020-08-07T15:11:51.722 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8629999999999996e-06 2020-08-07T15:11:51.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.722 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:51.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551004+0000", 2020-08-07T15:11:51.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4987000000000001e-05 2020-08-07T15:11:51.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.723 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:51.724 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551059+0000", 2020-08-07T15:11:51.724 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1718e-05 2020-08-07T15:11:51.724 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.724 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.724 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:51.724 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551070+0000", 2020-08-07T15:11:51.725 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67720431000000003 2020-08-07T15:11:51.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.725 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:51.725 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.228275+0000", 2020-08-07T15:11:51.727 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000731965 2020-08-07T15:11:51.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:51.728 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229007+0000", 2020-08-07T15:11:51.728 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00025396199999999999 2020-08-07T15:11:51.729 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.729 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.729 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:51.729 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229261+0000", 2020-08-07T15:11:51.730 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002511189 2020-08-07T15:11:51.730 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.730 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.730 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:51.730 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231772+0000", 2020-08-07T15:11:51.730 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3399999999999997e-07 2020-08-07T15:11:51.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.731 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:51.731 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231773+0000", 2020-08-07T15:11:51.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0274e-05 2020-08-07T15:11:51.732 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.732 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:51.732 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231823+0000", 2020-08-07T15:11:51.732 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0094095440000000006 2020-08-07T15:11:51.733 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.733 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:51.733 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241233+0000", 2020-08-07T15:11:51.733 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4011999999999998e-05 2020-08-07T15:11:51.735 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.735 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.735 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:51.735 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241277+0000", 2020-08-07T15:11:51.736 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7199999999999999e-07 2020-08-07T15:11:51.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.736 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.736 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:51.736 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241277+0000", 2020-08-07T15:11:51.737 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017061100000000001 2020-08-07T15:11:51.737 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.737 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.737 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:51.737 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241448+0000", 2020-08-07T15:11:51.738 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69051440900000005 2020-08-07T15:11:51.738 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:51.738 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:51.738 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:51.738 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.738 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.739 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89642 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:11:51.739 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.551807+0000", 2020-08-07T15:11:51.739 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 30.027636989000001, 2020-08-07T15:11:51.739 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68991613100000004, 2020-08-07T15:11:51.739 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:51.740 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:51.740 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:51.740 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:51.740 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:51.740 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89642 2020-08-07T15:11:51.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.741 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:51.741 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.741 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:51.741 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551807+0000", 2020-08-07T15:11:51.742 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:51.742 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.742 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.742 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:51.742 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551807+0000", 2020-08-07T15:11:51.743 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7049000000000001e-05 2020-08-07T15:11:51.743 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.743 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:51.743 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551834+0000", 2020-08-07T15:11:51.743 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9249999999999997e-06 2020-08-07T15:11:51.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:51.744 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551838+0000", 2020-08-07T15:11:51.744 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9160000000000001e-06 2020-08-07T15:11:51.745 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.745 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:51.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551846+0000", 2020-08-07T15:11:51.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3510000000000002e-06 2020-08-07T15:11:51.746 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.746 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:51.747 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551849+0000", 2020-08-07T15:11:51.747 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9276000000000001e-05 2020-08-07T15:11:51.747 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.747 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.747 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:51.748 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551898+0000", 2020-08-07T15:11:51.749 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3095000000000001e-05 2020-08-07T15:11:51.749 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.749 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.749 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:51.750 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551911+0000", 2020-08-07T15:11:51.750 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67789159799999998 2020-08-07T15:11:51.750 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.750 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.751 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:51.751 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229803+0000", 2020-08-07T15:11:51.751 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8069000000000001e-05 2020-08-07T15:11:51.751 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.751 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.752 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:51.752 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229881+0000", 2020-08-07T15:11:51.752 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000152917 2020-08-07T15:11:51.752 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.753 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:51.753 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230034+0000", 2020-08-07T15:11:51.753 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044936760000000003 2020-08-07T15:11:51.753 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.753 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.754 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:51.754 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234528+0000", 2020-08-07T15:11:51.754 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8999999999999995e-07 2020-08-07T15:11:51.754 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.754 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:51.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234529+0000", 2020-08-07T15:11:51.755 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7741000000000002e-05 2020-08-07T15:11:51.755 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:51.756 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234586+0000", 2020-08-07T15:11:51.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069143959999999997 2020-08-07T15:11:51.757 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.758 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:51.758 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241501+0000", 2020-08-07T15:11:51.758 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1433000000000001e-05 2020-08-07T15:11:51.758 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.758 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.759 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:51.759 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241542+0000", 2020-08-07T15:11:51.759 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7800000000000001e-07 2020-08-07T15:11:51.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.760 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:51.760 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241543+0000", 2020-08-07T15:11:51.760 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000180321 2020-08-07T15:11:51.760 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.761 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:51.761 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241723+0000", 2020-08-07T15:11:51.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68991613100000004 2020-08-07T15:11:51.761 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:51.761 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:51.762 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:51.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.762 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.762 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89643 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:11:51.762 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.552181+0000", 2020-08-07T15:11:51.763 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 30.027262995000001, 2020-08-07T15:11:51.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68980872400000004, 2020-08-07T15:11:51.782 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:51.782 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:51.782 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:51.783 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:51.783 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:51.783 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89643 2020-08-07T15:11:51.783 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.783 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:51.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:51.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552181+0000", 2020-08-07T15:11:51.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:51.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:51.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552181+0000", 2020-08-07T15:11:51.785 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0926999999999997e-05 2020-08-07T15:11:51.785 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.786 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:51.786 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552212+0000", 2020-08-07T15:11:51.786 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0512e-05 2020-08-07T15:11:51.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:51.787 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552222+0000", 2020-08-07T15:11:51.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9379999999999992e-06 2020-08-07T15:11:51.787 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.787 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.788 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:51.788 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552230+0000", 2020-08-07T15:11:51.788 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0260000000000001e-06 2020-08-07T15:11:51.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.789 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:51.789 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552234+0000", 2020-08-07T15:11:51.789 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6109000000000002e-05 2020-08-07T15:11:51.789 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:51.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552280+0000", 2020-08-07T15:11:51.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5318000000000001e-05 2020-08-07T15:11:51.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:51.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552296+0000", 2020-08-07T15:11:51.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67811269100000005 2020-08-07T15:11:51.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.808 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:51.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230408+0000", 2020-08-07T15:11:51.809 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000125518 2020-08-07T15:11:51.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.810 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:51.811 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230534+0000", 2020-08-07T15:11:51.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024637999999999998 2020-08-07T15:11:51.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.811 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:51.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230780+0000", 2020-08-07T15:11:51.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0066398869999999997 2020-08-07T15:11:51.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:51.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237420+0000", 2020-08-07T15:11:51.813 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3500000000000005e-07 2020-08-07T15:11:51.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.813 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:51.814 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237421+0000", 2020-08-07T15:11:51.814 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4005000000000003e-05 2020-08-07T15:11:51.814 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.814 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.814 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:51.815 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237465+0000", 2020-08-07T15:11:51.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043133529999999998 2020-08-07T15:11:51.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.815 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:51.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241778+0000", 2020-08-07T15:11:51.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0592e-05 2020-08-07T15:11:51.816 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:51.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241819+0000", 2020-08-07T15:11:51.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4999999999999998e-07 2020-08-07T15:11:51.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.817 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:51.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241819+0000", 2020-08-07T15:11:51.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017018299999999999 2020-08-07T15:11:51.818 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.818 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.818 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:51.818 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241990+0000", 2020-08-07T15:11:51.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68980872400000004 2020-08-07T15:11:51.819 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:51.819 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:51.819 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:51.819 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.820 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89644 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:11:51.820 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.552384+0000", 2020-08-07T15:11:51.820 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 30.027059960999999, 2020-08-07T15:11:51.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68997479299999998, 2020-08-07T15:11:51.821 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:51.821 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:51.822 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:51.822 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:51.822 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:51.822 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89644 2020-08-07T15:11:51.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.823 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:51.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:51.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552384+0000", 2020-08-07T15:11:51.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:51.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:51.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552384+0000", 2020-08-07T15:11:51.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9238999999999999e-05 2020-08-07T15:11:51.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:51.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552413+0000", 2020-08-07T15:11:51.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3340000000000003e-06 2020-08-07T15:11:51.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:51.831 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552417+0000", 2020-08-07T15:11:51.831 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3119999999999994e-06 2020-08-07T15:11:51.831 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.832 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:51.832 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552426+0000", 2020-08-07T15:11:51.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4139999999999996e-06 2020-08-07T15:11:51.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:51.833 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552430+0000", 2020-08-07T15:11:51.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6613999999999999e-05 2020-08-07T15:11:51.833 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:51.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:11:51.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7280000000000007e-06 2020-08-07T15:11:51.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:51.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552477+0000", 2020-08-07T15:11:51.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67873990399999995 2020-08-07T15:11:51.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.836 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:51.836 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231216+0000", 2020-08-07T15:11:51.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3629999999999998e-05 2020-08-07T15:11:51.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:51.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231290+0000", 2020-08-07T15:11:51.838 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000132108 2020-08-07T15:11:51.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.838 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:51.838 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231422+0000", 2020-08-07T15:11:51.838 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006049652 2020-08-07T15:11:51.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.839 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:51.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237472+0000", 2020-08-07T15:11:51.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000005e-07 2020-08-07T15:11:51.840 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.840 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.840 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:51.840 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237472+0000", 2020-08-07T15:11:51.840 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2848000000000001e-05 2020-08-07T15:11:51.841 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:51.841 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237505+0000", 2020-08-07T15:11:51.841 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045352179999999997 2020-08-07T15:11:51.842 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.842 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.842 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:51.842 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242040+0000", 2020-08-07T15:11:51.842 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012843699999999999 2020-08-07T15:11:51.842 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.843 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:51.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242169+0000", 2020-08-07T15:11:51.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9999999999999998e-07 2020-08-07T15:11:51.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.853 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:51.853 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242169+0000", 2020-08-07T15:11:51.853 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000189354 2020-08-07T15:11:51.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.854 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.854 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:51.854 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242359+0000", 2020-08-07T15:11:51.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68997479299999998 2020-08-07T15:11:51.899 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:51.900 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:51.900 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:51.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.900 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89645 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:11:51.901 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:11:51.901 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 30.026977178999999, 2020-08-07T15:11:51.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69011194600000003, 2020-08-07T15:11:51.901 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:51.901 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:51.903 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:51.903 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:51.903 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:51.903 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89645 2020-08-07T15:11:51.903 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.904 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:51.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.904 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:51.904 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:11:51.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:51.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:51.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:11:51.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6750999999999998e-05 2020-08-07T15:11:51.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.922 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:51.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552493+0000", 2020-08-07T15:11:51.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0350000000000002e-06 2020-08-07T15:11:51.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:51.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552496+0000", 2020-08-07T15:11:51.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3029999999999996e-06 2020-08-07T15:11:51.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:51.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552503+0000", 2020-08-07T15:11:51.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7580000000000001e-06 2020-08-07T15:11:51.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:51.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552506+0000", 2020-08-07T15:11:51.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4868000000000003e-05 2020-08-07T15:11:51.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:51.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552570+0000", 2020-08-07T15:11:51.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9649999999999997e-06 2020-08-07T15:11:51.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:51.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552579+0000", 2020-08-07T15:11:51.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67928357699999997 2020-08-07T15:11:51.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:51.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231863+0000", 2020-08-07T15:11:51.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000115696 2020-08-07T15:11:51.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.940 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.940 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:51.940 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231979+0000", 2020-08-07T15:11:51.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020456599999999999 2020-08-07T15:11:51.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:51.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232183+0000", 2020-08-07T15:11:51.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005327923 2020-08-07T15:11:51.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.944 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:51.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237511+0000", 2020-08-07T15:11:51.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8999999999999998e-07 2020-08-07T15:11:51.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:51.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237511+0000", 2020-08-07T15:11:51.946 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6019000000000002e-05 2020-08-07T15:11:51.946 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.946 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.946 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:51.946 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237547+0000", 2020-08-07T15:11:51.946 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045340980000000003 2020-08-07T15:11:51.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.947 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.947 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:51.947 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242082+0000", 2020-08-07T15:11:51.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00032620499999999998 2020-08-07T15:11:51.949 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.949 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:51.949 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242408+0000", 2020-08-07T15:11:51.950 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3500000000000002e-07 2020-08-07T15:11:51.950 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.950 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:51.951 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242408+0000", 2020-08-07T15:11:51.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000170457 2020-08-07T15:11:51.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.952 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:51.952 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242579+0000", 2020-08-07T15:11:51.952 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69011194600000003 2020-08-07T15:11:51.952 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:51.952 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:51.953 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:51.953 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.953 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.953 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89646 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:11:51.954 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.557244+0000", 2020-08-07T15:11:51.954 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 30.022199406999999, 2020-08-07T15:11:51.954 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68559352299999998, 2020-08-07T15:11:51.954 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:51.955 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:51.955 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:51.955 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:51.955 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:51.955 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89646 2020-08-07T15:11:51.956 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.956 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:51.956 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.956 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:51.956 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557244+0000", 2020-08-07T15:11:51.956 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:51.957 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.957 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:51.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557244+0000", 2020-08-07T15:11:51.958 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9699000000000001e-05 2020-08-07T15:11:51.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:51.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557274+0000", 2020-08-07T15:11:51.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1269999999999996e-06 2020-08-07T15:11:51.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:51.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557278+0000", 2020-08-07T15:11:51.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0099999999999995e-06 2020-08-07T15:11:51.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:51.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557286+0000", 2020-08-07T15:11:51.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7439999999999998e-06 2020-08-07T15:11:51.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.962 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:51.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557291+0000", 2020-08-07T15:11:51.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5927999999999999e-05 2020-08-07T15:11:51.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:51.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557327+0000", 2020-08-07T15:11:51.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5069999999999997e-06 2020-08-07T15:11:51.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:51.964 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557333+0000", 2020-08-07T15:11:51.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67527419200000005 2020-08-07T15:11:51.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:51.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232608+0000", 2020-08-07T15:11:51.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8699e-05 2020-08-07T15:11:51.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:51.966 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232686+0000", 2020-08-07T15:11:51.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018185099999999999 2020-08-07T15:11:51.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:51.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232868+0000", 2020-08-07T15:11:51.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0075819360000000001 2020-08-07T15:11:51.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:51.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240450+0000", 2020-08-07T15:11:51.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2799999999999995e-07 2020-08-07T15:11:51.969 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.969 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.970 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:51.970 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240451+0000", 2020-08-07T15:11:51.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3751000000000002e-05 2020-08-07T15:11:51.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:51.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240495+0000", 2020-08-07T15:11:51.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001628064 2020-08-07T15:11:51.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:51.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242123+0000", 2020-08-07T15:11:51.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00050955599999999998 2020-08-07T15:11:51.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:51.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242632+0000", 2020-08-07T15:11:51.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2900000000000004e-07 2020-08-07T15:11:51.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:51.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242633+0000", 2020-08-07T15:11:51.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000205202 2020-08-07T15:11:51.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.975 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:51.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242838+0000", 2020-08-07T15:11:51.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68559352299999998 2020-08-07T15:11:51.975 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:51.976 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:51.976 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:51.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.976 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89647 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:11:51.977 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.557315+0000", 2020-08-07T15:11:51.977 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 30.022129083999999, 2020-08-07T15:11:51.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68581815800000001, 2020-08-07T15:11:51.977 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:51.977 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:51.978 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:51.978 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:51.978 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:51.978 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89647 2020-08-07T15:11:51.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.979 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:51.979 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.979 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:51.979 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557315+0000", 2020-08-07T15:11:51.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:51.979 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.983 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.983 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:51.983 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557315+0000", 2020-08-07T15:11:51.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6639999999999999e-05 2020-08-07T15:11:51.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.984 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:51.984 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557341+0000", 2020-08-07T15:11:51.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7989999999999998e-06 2020-08-07T15:11:51.985 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.985 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:51.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557344+0000", 2020-08-07T15:11:51.985 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6960000000000002e-06 2020-08-07T15:11:51.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.986 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:51.987 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557350+0000", 2020-08-07T15:11:51.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6089999999999999e-06 2020-08-07T15:11:51.987 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:51.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557352+0000", 2020-08-07T15:11:51.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1837e-05 2020-08-07T15:11:51.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:51.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557384+0000", 2020-08-07T15:11:51.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8099999999999998e-06 2020-08-07T15:11:51.990 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:51.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557392+0000", 2020-08-07T15:11:51.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67581117300000004 2020-08-07T15:11:51.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:51.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:51.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:51.999 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233203+0000", 2020-08-07T15:11:52.000 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112316 2020-08-07T15:11:52.000 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.000 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.000 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:52.000 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233316+0000", 2020-08-07T15:11:52.001 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000221766 2020-08-07T15:11:52.009 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.009 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.009 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:52.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233537+0000", 2020-08-07T15:11:52.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069695019999999998 2020-08-07T15:11:52.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.010 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:52.010 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240507+0000", 2020-08-07T15:11:52.010 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3999999999999997e-07 2020-08-07T15:11:52.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.011 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:52.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240507+0000", 2020-08-07T15:11:52.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8844000000000003e-05 2020-08-07T15:11:52.018 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.020 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.020 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:52.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240546+0000", 2020-08-07T15:11:52.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023473399999999998 2020-08-07T15:11:52.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:52.022 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242893+0000", 2020-08-07T15:11:52.042 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5410000000000001e-05 2020-08-07T15:11:52.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.042 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:52.043 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242939+0000", 2020-08-07T15:11:52.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8000000000000001e-07 2020-08-07T15:11:52.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:52.044 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242939+0000", 2020-08-07T15:11:52.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019369600000000001 2020-08-07T15:11:52.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.044 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:52.045 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243133+0000", 2020-08-07T15:11:52.045 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68581815800000001 2020-08-07T15:11:52.068 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:52.068 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:52.068 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:52.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.069 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.069 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89648 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:11:52.069 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.557372+0000", 2020-08-07T15:11:52.069 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 30.022071962999998, 2020-08-07T15:11:52.069 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68606040000000001, 2020-08-07T15:11:52.070 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:52.070 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:52.070 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:52.070 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:52.070 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:52.071 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89648 2020-08-07T15:11:52.071 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.071 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:52.071 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.071 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:52.071 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557372+0000", 2020-08-07T15:11:52.072 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:52.072 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.072 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.072 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:52.072 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557372+0000", 2020-08-07T15:11:52.073 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5534e-05 2020-08-07T15:11:52.073 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.073 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.073 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:52.073 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557397+0000", 2020-08-07T15:11:52.102 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.599e-06 2020-08-07T15:11:52.103 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.103 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.109 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:52.110 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557400+0000", 2020-08-07T15:11:52.110 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.288e-06 2020-08-07T15:11:52.111 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.111 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.111 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:52.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557405+0000", 2020-08-07T15:11:52.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.142e-06 2020-08-07T15:11:52.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:52.113 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557407+0000", 2020-08-07T15:11:52.113 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0954999999999998e-05 2020-08-07T15:11:52.127 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.127 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.128 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:52.128 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557428+0000", 2020-08-07T15:11:52.128 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7529999999999998e-06 2020-08-07T15:11:52.129 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.129 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.129 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:52.130 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557434+0000", 2020-08-07T15:11:52.142 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67655471099999998 2020-08-07T15:11:52.142 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.142 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.142 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:52.143 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233989+0000", 2020-08-07T15:11:52.143 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8301999999999997e-05 2020-08-07T15:11:52.143 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.143 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.143 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:52.144 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234067+0000", 2020-08-07T15:11:52.144 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000129908 2020-08-07T15:11:52.144 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.144 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.144 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:52.145 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234197+0000", 2020-08-07T15:11:52.145 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063551249999999997 2020-08-07T15:11:52.145 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.145 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.145 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:52.163 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240552+0000", 2020-08-07T15:11:52.163 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9400000000000001e-07 2020-08-07T15:11:52.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:52.164 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240552+0000", 2020-08-07T15:11:52.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2787999999999998e-05 2020-08-07T15:11:52.164 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.165 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:52.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240585+0000", 2020-08-07T15:11:52.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002605652 2020-08-07T15:11:52.170 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.170 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.170 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:52.171 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243191+0000", 2020-08-07T15:11:52.171 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8690000000000002e-05 2020-08-07T15:11:52.171 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.171 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.171 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:52.172 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243250+0000", 2020-08-07T15:11:52.172 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4700000000000001e-07 2020-08-07T15:11:52.172 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.172 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.172 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:52.182 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243250+0000", 2020-08-07T15:11:52.182 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000181912 2020-08-07T15:11:52.182 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.183 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:52.183 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243432+0000", 2020-08-07T15:11:52.184 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68606040000000001 2020-08-07T15:11:52.184 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:52.184 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:52.184 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:52.184 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.185 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.185 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89689 5.7 5:e83ee43b:::benchmark_data_smithi074_53697_object11211:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:11:52.185 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.585493+0000", 2020-08-07T15:11:52.185 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 29.993951145, 2020-08-07T15:11:52.185 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66549445299999999, 2020-08-07T15:11:52.186 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:52.186 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:52.186 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:52.186 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:52.186 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:52.187 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89689 2020-08-07T15:11:52.193 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.193 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:52.193 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.194 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:52.194 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585493+0000", 2020-08-07T15:11:52.195 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:52.195 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.195 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.195 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:52.195 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585493+0000", 2020-08-07T15:11:52.196 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7746000000000002e-05 2020-08-07T15:11:52.196 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.196 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.196 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:52.197 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585520+0000", 2020-08-07T15:11:52.197 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1099999999999996e-06 2020-08-07T15:11:52.197 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.197 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.199 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:52.199 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585525+0000", 2020-08-07T15:11:52.203 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5359999999999998e-06 2020-08-07T15:11:52.203 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.204 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.204 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:52.205 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585534+0000", 2020-08-07T15:11:52.206 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5040000000000002e-06 2020-08-07T15:11:52.207 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.207 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.208 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:52.208 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585540+0000", 2020-08-07T15:11:52.208 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3548999999999998e-05 2020-08-07T15:11:52.208 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.208 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.209 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:52.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585593+0000", 2020-08-07T15:11:52.212 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2014000000000001e-05 2020-08-07T15:11:52.213 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.213 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.213 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:52.213 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585605+0000", 2020-08-07T15:11:52.214 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.64901847700000004 2020-08-07T15:11:52.214 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.214 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.214 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:52.221 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234624+0000", 2020-08-07T15:11:52.221 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000176329 2020-08-07T15:11:52.221 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.221 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.222 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:52.222 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234800+0000", 2020-08-07T15:11:52.222 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00025457300000000002 2020-08-07T15:11:52.222 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.223 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.223 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:52.223 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.235055+0000", 2020-08-07T15:11:52.223 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0055364669999999998 2020-08-07T15:11:52.223 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.224 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.224 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:52.224 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240591+0000", 2020-08-07T15:11:52.224 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3700000000000001e-07 2020-08-07T15:11:52.224 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.225 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.225 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:52.225 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240591+0000", 2020-08-07T15:11:52.230 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8479999999999997e-05 2020-08-07T15:11:52.230 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.230 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.231 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:52.231 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240640+0000", 2020-08-07T15:11:52.231 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00285564 2020-08-07T15:11:52.232 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.232 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.232 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:52.232 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243496+0000", 2020-08-07T15:11:52.232 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0073477400000000002 2020-08-07T15:11:52.233 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.233 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.233 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:52.233 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.250843+0000", 2020-08-07T15:11:52.233 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7300000000000005e-07 2020-08-07T15:11:52.234 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.234 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.234 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:52.270 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.250844+0000", 2020-08-07T15:11:52.270 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014307799999999999 2020-08-07T15:11:52.271 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.271 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:52.271 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.250987+0000", 2020-08-07T15:11:52.271 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66549445299999999 2020-08-07T15:11:52.272 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:52.272 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:52.272 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:52.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.273 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103152 5.7 5:e865d79a:::benchmark_data_smithi074_53697_object12893:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e211)", 2020-08-07T15:11:52.273 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.399967+0000", 2020-08-07T15:11:52.273 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 20.179477255999998, 2020-08-07T15:11:52.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81348256399999996, 2020-08-07T15:11:52.287 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:52.288 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:11:52.288 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:52.288 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:52.305 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:52.306 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103152 2020-08-07T15:11:52.306 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.310 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:52.310 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.327 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:52.327 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.399967+0000", 2020-08-07T15:11:52.327 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:52.328 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.328 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.328 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:52.328 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.399967+0000", 2020-08-07T15:11:52.328 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017528899999999999 2020-08-07T15:11:52.329 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.329 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.351 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:52.351 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400142+0000", 2020-08-07T15:11:52.352 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8239999999999996e-06 2020-08-07T15:11:52.352 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.352 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.362 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:52.362 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400148+0000", 2020-08-07T15:11:52.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0251000000000001e-05 2020-08-07T15:11:52.363 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.363 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.363 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:52.363 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400158+0000", 2020-08-07T15:11:52.364 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1579999999999998e-06 2020-08-07T15:11:52.364 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:52.364 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400162+0000", 2020-08-07T15:11:52.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00078404699999999996 2020-08-07T15:11:52.379 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:52.380 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400946+0000", 2020-08-07T15:11:52.380 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010605800000000001 2020-08-07T15:11:52.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:52.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.401052+0000", 2020-08-07T15:11:52.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020987499999999999 2020-08-07T15:11:52.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.387 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.387 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:52.388 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.401262+0000", 2020-08-07T15:11:52.388 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023707820000000001 2020-08-07T15:11:52.388 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.388 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.389 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:52.389 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.403633+0000", 2020-08-07T15:11:52.389 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.023e-06 2020-08-07T15:11:52.389 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.389 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.390 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:52.390 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.403634+0000", 2020-08-07T15:11:52.390 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3560999999999995e-05 2020-08-07T15:11:52.390 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.390 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.392 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:52.392 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.403717+0000", 2020-08-07T15:11:52.392 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011669689999999999 2020-08-07T15:11:52.392 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.392 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.393 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:52.393 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.404884+0000", 2020-08-07T15:11:52.393 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033332850000000001 2020-08-07T15:11:52.393 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.401 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.402 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:52.402 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.408218+0000", 2020-08-07T15:11:52.402 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0268000000000001e-05 2020-08-07T15:11:52.403 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.403 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.403 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:52.404 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.408228+0000", 2020-08-07T15:11:52.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80508180500000004 2020-08-07T15:11:52.404 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.404 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.404 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:52.405 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213310+0000", 2020-08-07T15:11:52.405 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000139369 2020-08-07T15:11:52.405 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.405 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:52.406 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213449+0000", 2020-08-07T15:11:52.406 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81348256399999996 2020-08-07T15:11:52.406 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:52.408 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:52.408 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:52.408 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.408 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.408 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103265 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:11:52.409 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478593+0000", 2020-08-07T15:11:52.409 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 20.100851200000001, 2020-08-07T15:11:52.409 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74922190499999997, 2020-08-07T15:11:52.409 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:52.409 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:52.410 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:52.410 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:52.410 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:52.410 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103265 2020-08-07T15:11:52.410 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.411 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:52.411 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.411 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:52.411 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478593+0000", 2020-08-07T15:11:52.411 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:52.412 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.412 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.412 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:52.412 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478593+0000", 2020-08-07T15:11:52.412 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9968000000000001e-05 2020-08-07T15:11:52.412 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.413 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:52.413 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478633+0000", 2020-08-07T15:11:52.413 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1420000000000003e-06 2020-08-07T15:11:52.413 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.422 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:52.422 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478637+0000", 2020-08-07T15:11:52.423 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2919999999999996e-06 2020-08-07T15:11:52.423 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.423 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.423 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:52.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478645+0000", 2020-08-07T15:11:52.424 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3699999999999997e-06 2020-08-07T15:11:52.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.424 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.424 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:52.424 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478649+0000", 2020-08-07T15:11:52.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1440000000000002e-05 2020-08-07T15:11:52.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.426 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.426 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:52.426 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478701+0000", 2020-08-07T15:11:52.426 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0831e-05 2020-08-07T15:11:52.426 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.427 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:52.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478712+0000", 2020-08-07T15:11:52.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73476696699999999 2020-08-07T15:11:52.427 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.427 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:52.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213479+0000", 2020-08-07T15:11:52.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000167026 2020-08-07T15:11:52.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.429 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:52.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213646+0000", 2020-08-07T15:11:52.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000179557 2020-08-07T15:11:52.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:52.456 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213825+0000", 2020-08-07T15:11:52.456 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041300360000000001 2020-08-07T15:11:52.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:52.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217955+0000", 2020-08-07T15:11:52.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6840000000000001e-06 2020-08-07T15:11:52.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.460 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.460 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:52.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217957+0000", 2020-08-07T15:11:52.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4758e-05 2020-08-07T15:11:52.461 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.461 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.461 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:52.461 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218012+0000", 2020-08-07T15:11:52.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0095519990000000003 2020-08-07T15:11:52.472 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.473 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.473 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:52.473 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227564+0000", 2020-08-07T15:11:52.474 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9956e-05 2020-08-07T15:11:52.474 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.474 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.474 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:52.475 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227624+0000", 2020-08-07T15:11:52.475 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3790000000000001e-06 2020-08-07T15:11:52.475 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.475 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.475 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:52.476 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227625+0000", 2020-08-07T15:11:52.476 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0001895 2020-08-07T15:11:52.476 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.476 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.476 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:52.476 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227815+0000", 2020-08-07T15:11:52.477 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74922190499999997 2020-08-07T15:11:52.477 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:52.477 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:52.477 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:52.478 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.478 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.485 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103266 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:11:52.485 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478693+0000", 2020-08-07T15:11:52.485 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 20.100750859000001, 2020-08-07T15:11:52.485 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74942137399999997, 2020-08-07T15:11:52.486 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:52.486 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:52.486 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:52.486 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:52.486 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:52.487 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103266 2020-08-07T15:11:52.487 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.487 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:52.487 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.487 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:52.488 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478693+0000", 2020-08-07T15:11:52.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:52.488 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:52.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478693+0000", 2020-08-07T15:11:52.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5658000000000002e-05 2020-08-07T15:11:52.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.489 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.489 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:52.490 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478739+0000", 2020-08-07T15:11:52.490 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.113e-06 2020-08-07T15:11:52.490 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.490 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.490 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:52.491 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478744+0000", 2020-08-07T15:11:52.491 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9690000000000006e-06 2020-08-07T15:11:52.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.491 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.491 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:52.491 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478754+0000", 2020-08-07T15:11:52.492 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1849999999999997e-06 2020-08-07T15:11:52.492 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.492 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.492 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:52.492 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478758+0000", 2020-08-07T15:11:52.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7024999999999998e-05 2020-08-07T15:11:52.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.493 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:52.493 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478805+0000", 2020-08-07T15:11:52.504 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1357e-05 2020-08-07T15:11:52.505 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.506 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.508 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:52.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478816+0000", 2020-08-07T15:11:52.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73554277499999998 2020-08-07T15:11:52.509 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.509 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.510 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:52.510 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.214359+0000", 2020-08-07T15:11:52.510 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000129145 2020-08-07T15:11:52.510 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:52.511 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.214488+0000", 2020-08-07T15:11:52.511 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00027129999999999998 2020-08-07T15:11:52.511 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.512 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:52.512 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.214760+0000", 2020-08-07T15:11:52.512 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032631079999999998 2020-08-07T15:11:52.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.512 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.513 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:52.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218023+0000", 2020-08-07T15:11:52.513 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1000000000000001e-06 2020-08-07T15:11:52.513 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.518 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:52.519 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218024+0000", 2020-08-07T15:11:52.519 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9085000000000006e-05 2020-08-07T15:11:52.519 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.519 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:52.520 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218103+0000", 2020-08-07T15:11:52.520 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.009779088 2020-08-07T15:11:52.520 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.520 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:52.521 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227882+0000", 2020-08-07T15:11:52.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1989999999999994e-05 2020-08-07T15:11:52.526 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.527 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.527 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:52.527 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227944+0000", 2020-08-07T15:11:52.527 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8299999999999996e-07 2020-08-07T15:11:52.527 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.528 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:52.528 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227945+0000", 2020-08-07T15:11:52.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000169693 2020-08-07T15:11:52.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.529 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.529 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:52.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228114+0000", 2020-08-07T15:11:52.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74942137399999997 2020-08-07T15:11:52.529 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:52.536 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:52.537 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:52.537 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.545 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.546 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103267 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:11:52.546 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478794+0000", 2020-08-07T15:11:52.600 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 20.100649370999999, 2020-08-07T15:11:52.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74960417300000004, 2020-08-07T15:11:52.600 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:52.601 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:52.601 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:52.601 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:52.601 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:52.606 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103267 2020-08-07T15:11:52.606 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.606 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:52.607 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.607 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:52.607 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478794+0000", 2020-08-07T15:11:52.607 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:52.607 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.607 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.614 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:52.614 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478794+0000", 2020-08-07T15:11:52.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6646e-05 2020-08-07T15:11:52.614 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.614 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.615 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:52.615 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478841+0000", 2020-08-07T15:11:52.615 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:11:52.615 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.615 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.616 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:52.616 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478846+0000", 2020-08-07T15:11:52.616 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0171e-05 2020-08-07T15:11:52.616 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.616 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.617 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:52.617 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478856+0000", 2020-08-07T15:11:52.617 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5929999999999999e-06 2020-08-07T15:11:52.617 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.617 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.618 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:52.618 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478861+0000", 2020-08-07T15:11:52.618 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2032e-05 2020-08-07T15:11:52.618 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.618 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.618 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:52.619 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478903+0000", 2020-08-07T15:11:52.619 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0312000000000001e-05 2020-08-07T15:11:52.619 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.619 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.619 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:52.620 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478913+0000", 2020-08-07T15:11:52.620 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73634238600000002 2020-08-07T15:11:52.620 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.620 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.620 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:52.621 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.215256+0000", 2020-08-07T15:11:52.621 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000124933 2020-08-07T15:11:52.621 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.621 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.621 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:52.639 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.215380+0000", 2020-08-07T15:11:52.639 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024624 2020-08-07T15:11:52.639 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.639 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.640 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:52.640 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.215627+0000", 2020-08-07T15:11:52.671 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0040879540000000004 2020-08-07T15:11:52.671 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.671 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.672 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:52.672 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219715+0000", 2020-08-07T15:11:52.673 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9230000000000001e-06 2020-08-07T15:11:52.673 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.681 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:52.681 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219717+0000", 2020-08-07T15:11:52.682 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8881999999999995e-05 2020-08-07T15:11:52.682 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.682 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.682 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:52.682 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219785+0000", 2020-08-07T15:11:52.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0083925119999999995 2020-08-07T15:11:52.683 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.683 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.683 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:52.683 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228178+0000", 2020-08-07T15:11:52.684 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2254000000000004e-05 2020-08-07T15:11:52.684 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.684 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.685 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:52.685 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228240+0000", 2020-08-07T15:11:52.685 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4499999999999996e-07 2020-08-07T15:11:52.685 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.686 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.686 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:52.686 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228241+0000", 2020-08-07T15:11:52.686 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000157656 2020-08-07T15:11:52.686 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.687 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.687 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:52.687 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228399+0000", 2020-08-07T15:11:52.687 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74960417300000004 2020-08-07T15:11:52.688 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:52.688 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:52.688 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:52.688 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.706 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.706 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103268 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:11:52.707 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478898+0000", 2020-08-07T15:11:52.707 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 20.100545654000001, 2020-08-07T15:11:52.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74981275700000005, 2020-08-07T15:11:52.708 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:52.708 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:52.709 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:52.709 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:52.709 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:52.710 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103268 2020-08-07T15:11:52.710 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.710 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:52.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.711 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:52.711 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478898+0000", 2020-08-07T15:11:52.712 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:52.712 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.712 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.713 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:52.713 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478898+0000", 2020-08-07T15:11:52.724 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2558999999999997e-05 2020-08-07T15:11:52.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.725 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:52.742 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478941+0000", 2020-08-07T15:11:52.743 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6469999999999997e-06 2020-08-07T15:11:52.743 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:52.744 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478945+0000", 2020-08-07T15:11:52.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3109999999999995e-06 2020-08-07T15:11:52.745 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.753 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:52.754 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478955+0000", 2020-08-07T15:11:52.754 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.799e-06 2020-08-07T15:11:52.754 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.755 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:52.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478958+0000", 2020-08-07T15:11:52.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5682000000000002e-05 2020-08-07T15:11:52.757 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.758 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.758 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:52.758 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:11:52.759 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3260000000000002e-06 2020-08-07T15:11:52.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.760 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:52.761 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479004+0000", 2020-08-07T15:11:52.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73703730499999998 2020-08-07T15:11:52.761 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.762 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:52.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216041+0000", 2020-08-07T15:11:52.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4445999999999998e-05 2020-08-07T15:11:52.763 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.763 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:52.763 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216115+0000", 2020-08-07T15:11:52.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013977 2020-08-07T15:11:52.764 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.764 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:52.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216255+0000", 2020-08-07T15:11:52.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0051267359999999998 2020-08-07T15:11:52.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.765 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:52.765 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221382+0000", 2020-08-07T15:11:52.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0310000000000001e-06 2020-08-07T15:11:52.766 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.766 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:52.766 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221384+0000", 2020-08-07T15:11:52.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8097999999999998e-05 2020-08-07T15:11:52.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:52.767 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221452+0000", 2020-08-07T15:11:52.768 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0070259249999999997 2020-08-07T15:11:52.768 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.768 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.768 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:52.768 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228478+0000", 2020-08-07T15:11:52.769 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7479999999999999e-05 2020-08-07T15:11:52.769 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.769 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.769 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:52.770 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228535+0000", 2020-08-07T15:11:52.770 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4399999999999999e-07 2020-08-07T15:11:52.770 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.770 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.771 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:52.771 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228536+0000", 2020-08-07T15:11:52.771 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017489799999999999 2020-08-07T15:11:52.772 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.772 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.772 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:52.772 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228711+0000", 2020-08-07T15:11:52.773 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74981275700000005 2020-08-07T15:11:52.773 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:52.774 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:52.774 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:52.774 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.774 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.775 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103269 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:11:52.776 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:11:52.776 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 20.100449546, 2020-08-07T15:11:52.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74992452799999998, 2020-08-07T15:11:52.776 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:52.777 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:52.777 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:52.782 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:52.782 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:52.782 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103269 2020-08-07T15:11:52.782 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.782 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:52.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:52.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:11:52.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:52.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.792 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.792 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:52.793 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:11:52.793 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5529999999999999e-05 2020-08-07T15:11:52.793 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.794 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.794 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:52.794 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479040+0000", 2020-08-07T15:11:52.795 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6550000000000003e-06 2020-08-07T15:11:52.795 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.795 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.796 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:52.796 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479044+0000", 2020-08-07T15:11:52.796 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4285e-05 2020-08-07T15:11:52.797 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.797 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.797 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:52.814 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479059+0000", 2020-08-07T15:11:52.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2050000000000001e-06 2020-08-07T15:11:52.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:52.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479064+0000", 2020-08-07T15:11:52.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0952999999999999e-05 2020-08-07T15:11:52.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:52.822 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479105+0000", 2020-08-07T15:11:52.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3559999999999998e-06 2020-08-07T15:11:52.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:52.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479111+0000", 2020-08-07T15:11:52.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.737508104 2020-08-07T15:11:52.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:52.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216619+0000", 2020-08-07T15:11:52.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000126367 2020-08-07T15:11:52.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:52.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216746+0000", 2020-08-07T15:11:52.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014069899999999999 2020-08-07T15:11:52.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:52.832 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216886+0000", 2020-08-07T15:11:52.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045753160000000003 2020-08-07T15:11:52.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:52.833 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221462+0000", 2020-08-07T15:11:52.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.142e-06 2020-08-07T15:11:52.833 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:52.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221463+0000", 2020-08-07T15:11:52.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7718e-05 2020-08-07T15:11:52.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:52.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221501+0000", 2020-08-07T15:11:52.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0072733210000000001 2020-08-07T15:11:52.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.836 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:52.836 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228774+0000", 2020-08-07T15:11:52.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3828e-05 2020-08-07T15:11:52.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:52.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228828+0000", 2020-08-07T15:11:52.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9800000000000003e-07 2020-08-07T15:11:52.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.838 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:52.852 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228828+0000", 2020-08-07T15:11:52.853 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0451000000000005e-05 2020-08-07T15:11:52.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.853 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:52.854 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228919+0000", 2020-08-07T15:11:52.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74992452799999998 2020-08-07T15:11:52.854 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:52.854 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:52.854 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:52.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.855 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103270 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:11:52.855 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.479168+0000", 2020-08-07T15:11:52.856 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 20.100275508999999, 2020-08-07T15:11:52.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75043371699999994, 2020-08-07T15:11:52.856 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:52.856 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:52.856 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:52.857 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:52.857 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:52.857 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103270 2020-08-07T15:11:52.858 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.858 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:52.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.859 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:52.859 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479168+0000", 2020-08-07T15:11:52.859 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:52.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.859 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:52.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479168+0000", 2020-08-07T15:11:52.860 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1557000000000002e-05 2020-08-07T15:11:52.860 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.860 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.861 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:52.861 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479210+0000", 2020-08-07T15:11:52.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9840000000000001e-06 2020-08-07T15:11:52.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.862 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:52.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479215+0000", 2020-08-07T15:11:52.863 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9539999999999999e-06 2020-08-07T15:11:52.864 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:52.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479225+0000", 2020-08-07T15:11:52.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.942e-06 2020-08-07T15:11:52.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.865 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:52.865 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479231+0000", 2020-08-07T15:11:52.866 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2493000000000002e-05 2020-08-07T15:11:52.866 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.866 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.866 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:52.866 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479263+0000", 2020-08-07T15:11:52.867 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6640000000000003e-06 2020-08-07T15:11:52.867 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.867 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.867 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:52.867 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479269+0000", 2020-08-07T15:11:52.868 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73802486499999997 2020-08-07T15:11:52.868 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.868 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.868 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:52.869 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217294+0000", 2020-08-07T15:11:52.869 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8640000000000006e-05 2020-08-07T15:11:52.869 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.869 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.869 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:52.870 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217372+0000", 2020-08-07T15:11:52.870 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000140338 2020-08-07T15:11:52.870 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.870 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.870 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:52.871 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217513+0000", 2020-08-07T15:11:52.871 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039952119999999997 2020-08-07T15:11:52.871 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.872 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:52.872 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221508+0000", 2020-08-07T15:11:52.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8899999999999999e-07 2020-08-07T15:11:52.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.873 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:52.873 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221509+0000", 2020-08-07T15:11:52.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8108999999999997e-05 2020-08-07T15:11:52.874 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.874 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.874 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:52.875 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221547+0000", 2020-08-07T15:11:52.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074232029999999997 2020-08-07T15:11:52.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:52.885 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228970+0000", 2020-08-07T15:11:52.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00046710099999999999 2020-08-07T15:11:52.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.886 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.886 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:52.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229437+0000", 2020-08-07T15:11:52.887 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5400000000000001e-07 2020-08-07T15:11:52.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:52.887 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229438+0000", 2020-08-07T15:11:52.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016441200000000001 2020-08-07T15:11:52.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:52.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229602+0000", 2020-08-07T15:11:52.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75043371699999994 2020-08-07T15:11:52.889 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:52.889 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:52.890 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:52.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.891 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103271 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:11:52.891 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.479356+0000", 2020-08-07T15:11:52.891 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 20.100087541000001, 2020-08-07T15:11:52.891 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75047653800000003, 2020-08-07T15:11:52.891 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:52.892 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:52.892 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:52.892 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:52.892 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:52.892 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103271 2020-08-07T15:11:52.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.893 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:52.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.893 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:52.893 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479356+0000", 2020-08-07T15:11:52.894 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:52.894 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.894 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.894 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:52.895 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479356+0000", 2020-08-07T15:11:52.895 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2416000000000002e-05 2020-08-07T15:11:52.895 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.895 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.895 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:52.896 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479399+0000", 2020-08-07T15:11:52.896 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0819999999999998e-06 2020-08-07T15:11:52.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.896 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:52.897 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479404+0000", 2020-08-07T15:11:52.897 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2486999999999999e-05 2020-08-07T15:11:52.897 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.897 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.897 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:52.898 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479416+0000", 2020-08-07T15:11:52.898 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3810000000000001e-06 2020-08-07T15:11:52.898 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.910 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:52.910 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479421+0000", 2020-08-07T15:11:52.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2311999999999998e-05 2020-08-07T15:11:52.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:52.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479453+0000", 2020-08-07T15:11:52.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7690000000000002e-06 2020-08-07T15:11:52.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:52.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479459+0000", 2020-08-07T15:11:52.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73869566499999995 2020-08-07T15:11:52.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:52.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218154+0000", 2020-08-07T15:11:52.914 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000124535 2020-08-07T15:11:52.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:52.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218279+0000", 2020-08-07T15:11:52.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000223441 2020-08-07T15:11:52.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:52.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218502+0000", 2020-08-07T15:11:52.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003054761 2020-08-07T15:11:52.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:52.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221557+0000", 2020-08-07T15:11:52.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0449999999999999e-06 2020-08-07T15:11:52.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:52.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221558+0000", 2020-08-07T15:11:52.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3655e-05 2020-08-07T15:11:52.937 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:52.938 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221622+0000", 2020-08-07T15:11:52.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.007661434 2020-08-07T15:11:52.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:52.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229283+0000", 2020-08-07T15:11:52.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00036023099999999997 2020-08-07T15:11:52.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.940 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:52.940 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229644+0000", 2020-08-07T15:11:52.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.63e-07 2020-08-07T15:11:52.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.940 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:52.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229644+0000", 2020-08-07T15:11:52.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018886099999999999 2020-08-07T15:11:52.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.942 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:52.942 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229833+0000", 2020-08-07T15:11:52.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75047653800000003 2020-08-07T15:11:52.942 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:52.942 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:52.943 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:52.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.943 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103272 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:11:52.943 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.481658+0000", 2020-08-07T15:11:52.944 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 20.097786117999998, 2020-08-07T15:11:52.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74920410900000001, 2020-08-07T15:11:52.944 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:52.944 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:52.944 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:52.945 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:52.945 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:52.945 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103272 2020-08-07T15:11:52.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.947 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:52.947 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:52.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481658+0000", 2020-08-07T15:11:52.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:52.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:52.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481658+0000", 2020-08-07T15:11:52.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8787e-05 2020-08-07T15:11:52.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.970 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:52.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481697+0000", 2020-08-07T15:11:52.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0559999999999998e-06 2020-08-07T15:11:52.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:52.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481701+0000", 2020-08-07T15:11:52.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1329999999999999e-06 2020-08-07T15:11:52.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:52.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481709+0000", 2020-08-07T15:11:52.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.189e-06 2020-08-07T15:11:52.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:52.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481713+0000", 2020-08-07T15:11:52.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6025999999999997e-05 2020-08-07T15:11:52.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.975 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:52.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481749+0000", 2020-08-07T15:11:52.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5619999999999998e-06 2020-08-07T15:11:52.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:11:52.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481756+0000", 2020-08-07T15:11:52.986 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73729155000000002 2020-08-07T15:11:52.987 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.987 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:52.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219048+0000", 2020-08-07T15:11:52.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000120387 2020-08-07T15:11:52.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:52.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219168+0000", 2020-08-07T15:11:52.996 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017865599999999999 2020-08-07T15:11:52.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.996 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.997 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:52.997 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219347+0000", 2020-08-07T15:11:52.997 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039049169999999999 2020-08-07T15:11:52.998 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:52.998 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:52.998 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:53.010 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.223252+0000", 2020-08-07T15:11:53.010 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.412e-06 2020-08-07T15:11:53.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:53.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:53.011 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:53.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.223253+0000", 2020-08-07T15:11:53.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7521999999999997e-05 2020-08-07T15:11:53.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:53.012 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:53.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:11:53.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.223301+0000", 2020-08-07T15:11:53.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00603478 2020-08-07T15:11:53.012 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:53.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:53.013 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:53.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229336+0000", 2020-08-07T15:11:53.013 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013309579999999999 2020-08-07T15:11:53.014 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:53.014 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:53.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:53.014 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.230667+0000", 2020-08-07T15:11:53.014 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7400000000000003e-07 2020-08-07T15:11:53.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:53.015 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:53.015 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:11:53.016 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.230667+0000", 2020-08-07T15:11:53.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019450000000000001 2020-08-07T15:11:53.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:53.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:53.023 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:11:53.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.230862+0000", 2020-08-07T15:11:53.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74920410900000001 2020-08-07T15:11:53.024 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:53.024 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:53.024 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:53.024 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:53.024 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:53.025 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:11:53.025 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3, 4, 5, 6, 7] 2020-08-07T15:11:53.026 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:11:53.026 INFO:tasks.thrashosds.thrasher:inject_pause on 2 2020-08-07T15:11:53.026 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2020-08-07T15:11:53.026 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2020-08-07T15:11:53.027 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2020-08-07T15:11:53.029 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:11:53.029 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:53.025+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:53.127 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:53.126+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:53.229 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:53.229+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:53.301 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:11:53.355 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:53.355+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:53.362 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:11:53.494 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:53.458+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:53.562 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:53.560+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:53.660 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:53.659+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:53.766 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:53.760+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:53.866 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:53.861+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:53.972 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:53.963+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:54.076 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:54.064+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:54.167 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:54.166+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:54.270 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:54.267+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:54.369 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:54.368+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:54.471 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:54.470+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:54.579 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:54.578+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:54.680 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:54.679+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:54.780 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:54.779+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:54.886 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:54.885+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:54.971 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:11:54.972 INFO:teuthology.orchestra.run.smithi074.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2020-08-07T15:11:54.972 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:11:54.972 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:11:54.972 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:11:54.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:54.973 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:134414 5.8 5:1186785c:::benchmark_data_smithi074_53697_object7493:head [delete] snapc 0=[] ondisk+write+known_if_redirected e221)", 2020-08-07T15:11:54.973 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:54.838098+0000", 2020-08-07T15:11:54.973 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.015106636999999999, 2020-08-07T15:11:54.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.015131567, 2020-08-07T15:11:54.985 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:54.985 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:11:54.985 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:54.986 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:54.986 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:54.986 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 134414 2020-08-07T15:11:54.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:54.987 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:54.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:54.987 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:54.987 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.838098+0000", 2020-08-07T15:11:54.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:54.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:54.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:54.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:54.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.838098+0000", 2020-08-07T15:11:54.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.647e-05 2020-08-07T15:11:54.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:54.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:54.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:54.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.838115+0000", 2020-08-07T15:11:54.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7469999999999999e-06 2020-08-07T15:11:54.990 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:54.990 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:54.990 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:54.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.838118+0000", 2020-08-07T15:11:54.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5070000000000003e-06 2020-08-07T15:11:54.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:54.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:54.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:54.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.838126+0000", 2020-08-07T15:11:54.991 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.507e-06 2020-08-07T15:11:54.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:54.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:54.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:54.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.838131+0000", 2020-08-07T15:11:54.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9326000000000003e-05 2020-08-07T15:11:54.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:54.993 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:54.993 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:54.993 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.838171+0000", 2020-08-07T15:11:54.993 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000154012 2020-08-07T15:11:54.993 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:55.017 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.838325+0000", 2020-08-07T15:11:55.017 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00029001 2020-08-07T15:11:55.017 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.018 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0", 2020-08-07T15:11:55.018 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.838615+0000", 2020-08-07T15:11:55.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024880229999999998 2020-08-07T15:11:55.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.020 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.020 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:55.020 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.841103+0000", 2020-08-07T15:11:55.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0030046019999999999 2020-08-07T15:11:55.021 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:55.021 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:55.021 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:55.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.022 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:134417 5.6 5:69949208:::benchmark_data_smithi074_53697_object7496:head [delete] snapc 0=[] ondisk+write+known_if_redirected e221)", 2020-08-07T15:11:55.022 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:54.840698+0000", 2020-08-07T15:11:55.022 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.012507236999999999, 2020-08-07T15:11:55.022 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.012639918, 2020-08-07T15:11:55.023 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:55.023 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:11:55.023 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:55.023 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:55.023 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:55.023 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 134417 2020-08-07T15:11:55.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.024 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:55.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:55.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.840698+0000", 2020-08-07T15:11:55.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:55.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:55.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.840698+0000", 2020-08-07T15:11:55.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5456000000000002e-05 2020-08-07T15:11:55.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:55.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.840723+0000", 2020-08-07T15:11:55.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0229999999999998e-06 2020-08-07T15:11:55.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:55.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.840729+0000", 2020-08-07T15:11:55.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0294e-05 2020-08-07T15:11:55.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:55.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.840739+0000", 2020-08-07T15:11:55.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7459999999999997e-06 2020-08-07T15:11:55.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:55.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.840745+0000", 2020-08-07T15:11:55.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6804999999999997e-05 2020-08-07T15:11:55.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:55.034 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.840782+0000", 2020-08-07T15:11:55.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014568599999999999 2020-08-07T15:11:55.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.034 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:55.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.840928+0000", 2020-08-07T15:11:55.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013550300000000001 2020-08-07T15:11:55.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,7", 2020-08-07T15:11:55.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.841063+0000", 2020-08-07T15:11:55.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0022310070000000001 2020-08-07T15:11:55.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:55.037 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.843294+0000", 2020-08-07T15:11:55.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010456510000000001 2020-08-07T15:11:55.038 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.038 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:55.038 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.844340+0000", 2020-08-07T15:11:55.038 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003642171 2020-08-07T15:11:55.039 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:55.039 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:55.039 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:55.039 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.039 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.040 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:134428 5.d 5:ba196795:::benchmark_data_smithi074_53697_object7507:head [delete] snapc 0=[] ondisk+write+known_if_redirected e221)", 2020-08-07T15:11:55.040 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:54.848396+0000", 2020-08-07T15:11:55.040 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0048090679999999997, 2020-08-07T15:11:55.040 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0050317529999999999, 2020-08-07T15:11:55.040 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:55.041 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:11:55.041 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:55.042 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:55.042 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:55.042 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 134428 2020-08-07T15:11:55.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.042 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:55.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:55.043 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.848396+0000", 2020-08-07T15:11:55.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:55.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.044 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:55.044 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.848396+0000", 2020-08-07T15:11:55.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2291e-05 2020-08-07T15:11:55.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:55.045 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.848428+0000", 2020-08-07T15:11:55.045 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6359999999999997e-06 2020-08-07T15:11:55.045 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.046 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.046 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:55.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.848435+0000", 2020-08-07T15:11:55.046 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2802e-05 2020-08-07T15:11:55.047 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.047 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:55.047 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.848447+0000", 2020-08-07T15:11:55.048 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1279999999999996e-06 2020-08-07T15:11:55.049 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.049 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.049 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:55.049 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.848454+0000", 2020-08-07T15:11:55.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4441999999999998e-05 2020-08-07T15:11:55.050 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.050 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.050 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:55.050 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.848508+0000", 2020-08-07T15:11:55.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000162721 2020-08-07T15:11:55.051 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.051 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:55.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.848671+0000", 2020-08-07T15:11:55.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015882799999999999 2020-08-07T15:11:55.055 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,7", 2020-08-07T15:11:55.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.848830+0000", 2020-08-07T15:11:55.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001626937 2020-08-07T15:11:55.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:55.057 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.850456+0000", 2020-08-07T15:11:55.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00052650400000000001 2020-08-07T15:11:55.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.058 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.058 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:55.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.850983+0000", 2020-08-07T15:11:55.059 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002587289 2020-08-07T15:11:55.059 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:55.059 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:55.059 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:55.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.060 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:134429 5.8 5:10c511ee:::benchmark_data_smithi074_53697_object7508:head [delete] snapc 0=[] ondisk+write+known_if_redirected e221)", 2020-08-07T15:11:55.060 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:54.849351+0000", 2020-08-07T15:11:55.060 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0038535710000000001, 2020-08-07T15:11:55.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041554790000000001, 2020-08-07T15:11:55.061 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:55.061 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:11:55.061 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:55.062 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:55.062 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:55.062 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 134429 2020-08-07T15:11:55.062 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.062 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:55.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.063 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:55.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.849351+0000", 2020-08-07T15:11:55.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:55.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:55.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.849351+0000", 2020-08-07T15:11:55.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7260000000000001e-05 2020-08-07T15:11:55.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.073 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.073 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:55.074 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.849368+0000", 2020-08-07T15:11:55.074 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4240000000000003e-06 2020-08-07T15:11:55.074 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.074 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.075 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:55.075 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.849373+0000", 2020-08-07T15:11:55.075 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9480000000000008e-06 2020-08-07T15:11:55.075 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.075 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.076 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:55.076 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.849381+0000", 2020-08-07T15:11:55.076 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1099999999999996e-06 2020-08-07T15:11:55.076 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.080 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.080 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:55.080 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.849385+0000", 2020-08-07T15:11:55.080 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1740999999999998e-05 2020-08-07T15:11:55.081 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.081 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.081 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:55.081 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.849437+0000", 2020-08-07T15:11:55.082 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000164914 2020-08-07T15:11:55.082 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.082 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:55.082 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.849602+0000", 2020-08-07T15:11:55.082 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019909499999999999 2020-08-07T15:11:55.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.083 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.083 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0", 2020-08-07T15:11:55.083 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.849801+0000", 2020-08-07T15:11:55.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001734862 2020-08-07T15:11:55.084 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.084 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:55.084 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.851536+0000", 2020-08-07T15:11:55.084 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0021843539999999999 2020-08-07T15:11:55.085 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:55.085 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:55.085 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:55.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.086 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:134402 5.a 5:56920744:::benchmark_data_smithi074_53697_object7481:head [delete] snapc 0=[] ondisk+write+known_if_redirected e221)", 2020-08-07T15:11:55.086 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:54.831510+0000", 2020-08-07T15:11:55.086 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.021695030000000001, 2020-08-07T15:11:55.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.022087967, 2020-08-07T15:11:55.086 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:55.087 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:11:55.087 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:55.087 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:55.087 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:55.087 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 134402 2020-08-07T15:11:55.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.088 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:55.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:55.088 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.831510+0000", 2020-08-07T15:11:55.089 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:55.089 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.089 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.089 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:55.089 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.831510+0000", 2020-08-07T15:11:55.090 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1696999999999998e-05 2020-08-07T15:11:55.105 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.105 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.106 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:55.106 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.831541+0000", 2020-08-07T15:11:55.106 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2999999999999998e-06 2020-08-07T15:11:55.106 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.107 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.107 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:55.107 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.831548+0000", 2020-08-07T15:11:55.107 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2999e-05 2020-08-07T15:11:55.107 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.108 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.108 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:55.108 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.831561+0000", 2020-08-07T15:11:55.108 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.716e-06 2020-08-07T15:11:55.108 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.109 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.109 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:55.109 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.831566+0000", 2020-08-07T15:11:55.109 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00033520000000000002 2020-08-07T15:11:55.109 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.126 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.126 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:55.126 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.831902+0000", 2020-08-07T15:11:55.127 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015489599999999999 2020-08-07T15:11:55.127 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.127 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.127 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:55.127 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.832057+0000", 2020-08-07T15:11:55.128 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016318300000000001 2020-08-07T15:11:55.128 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.128 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.128 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4", 2020-08-07T15:11:55.128 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.832220+0000", 2020-08-07T15:11:55.128 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032667970000000001 2020-08-07T15:11:55.129 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.129 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.129 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:55.129 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.835487+0000", 2020-08-07T15:11:55.129 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023745979999999999 2020-08-07T15:11:55.130 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.130 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.130 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:55.130 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.837861+0000", 2020-08-07T15:11:55.130 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063513859999999997 2020-08-07T15:11:55.131 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:55.131 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:55.131 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:55.131 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.131 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.132 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:134413 5.6 5:603840d6:::benchmark_data_smithi074_53697_object7492:head [delete] snapc 0=[] ondisk+write+known_if_redirected e221)", 2020-08-07T15:11:55.132 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:54.837316+0000", 2020-08-07T15:11:55.132 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.015888308, 2020-08-07T15:11:55.132 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.016359364000000001, 2020-08-07T15:11:55.132 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:55.133 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:11:55.133 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:55.133 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:55.133 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:55.144 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 134413 2020-08-07T15:11:55.144 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.145 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:55.145 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.145 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:55.145 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.837316+0000", 2020-08-07T15:11:55.146 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:55.146 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.146 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.146 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:55.146 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.837316+0000", 2020-08-07T15:11:55.147 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.955e-05 2020-08-07T15:11:55.147 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.147 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.147 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:55.147 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.837346+0000", 2020-08-07T15:11:55.147 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7610000000000004e-06 2020-08-07T15:11:55.148 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.148 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.148 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:55.148 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.837352+0000", 2020-08-07T15:11:55.148 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1375e-05 2020-08-07T15:11:55.149 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.149 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.149 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:55.149 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.837363+0000", 2020-08-07T15:11:55.149 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7320000000000003e-06 2020-08-07T15:11:55.150 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.150 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.150 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:55.150 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.837369+0000", 2020-08-07T15:11:55.150 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0634999999999999e-05 2020-08-07T15:11:55.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.151 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:55.151 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.837419+0000", 2020-08-07T15:11:55.151 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000102391 2020-08-07T15:11:55.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:55.152 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.837522+0000", 2020-08-07T15:11:55.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000110331 2020-08-07T15:11:55.153 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.154 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.155 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,7", 2020-08-07T15:11:55.155 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.837632+0000", 2020-08-07T15:11:55.155 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012654280000000001 2020-08-07T15:11:55.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.156 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:55.156 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.838898+0000", 2020-08-07T15:11:55.156 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00048879499999999998 2020-08-07T15:11:55.157 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:55.157 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:55.157 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:55.157 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:54.839386+0000", 2020-08-07T15:11:55.157 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002069998 2020-08-07T15:11:55.158 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:55.158 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:55.158 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:55.158 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:55.158 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:11:55.159 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 6 2020-08-07T15:11:55.159 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:11:55.159 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:11:55.161 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:11:55.162 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:11:55.162 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:11:55.162 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:11:55.163 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:11:55.164 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:55.160+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:55.201 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:11:55.246 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:11:55.247 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:11:55.247 INFO:teuthology.orchestra.run.smithi074.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-08-07T15:11:55.248 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: problem getting command descriptions from osd.1 2020-08-07T15:11:55.248 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:11:55.248 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:11:55.248 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:11:55.248 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:11:55.249 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:11:55.249 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:11:55.252 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-08-07T15:11:55.253 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-08-07T15:11:55.266 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:55.264+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:55.289 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:11:55.367 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:55.368 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:55.369 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:55.366+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:55.378 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:11:55.480 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:55.477+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:55.575 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:55.576 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:55.576 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:55.574+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:55.607 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:11:55.708 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:55.698+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:55.780 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:55.779+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:55.854 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:55.855 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:55.871 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:11:55.929 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:55.911+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:55.988 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:55.985+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:56.088 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:56.089 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:56.089 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:56.088+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:56.112 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:11:56.190 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:56.189+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:56.302 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:56.291+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:56.353 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:56.354 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:56.371 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:11:56.470 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:56.395+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:56.493 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:56.492+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:56.577 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:56.578 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:56.585 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:11:56.596 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:56.595+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:56.698 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:56.697+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:56.731 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:56.732 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:56.732 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:11:56.806 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:56.802+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:56.893 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:56.894 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:56.902 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:11:56.915 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:56.914+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:57.019 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:57.018+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:57.032 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:57.032 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:57.041 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:11:57.121 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:57.119+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:57.160 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:57.161 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:57.169 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:11:57.224 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:57.221+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:57.254 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:11:57.329 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:57.327+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:57.332 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:11:57.340 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:57.341 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:57.350 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:11:57.431 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:57.430+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:57.533 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:57.532+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:57.593 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:57.593 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:57.605 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:11:57.651 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:57.645+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:57.746 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:57.743+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:57.858 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:57.858 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:57.868 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:11:57.960 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:57.953+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:58.054 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:11:58.052+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:11:58.110 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:58.111 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:58.120 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:11:58.266 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:58.263+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:58.326 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:11:58.354 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:58.355 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:58.375 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:11:58.424 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:58.418+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:58.425 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:58.418+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:58.425 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:11:58.418+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:11:58.456 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:11:58.486 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:11:58.484+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:11:58.494 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:11:58.535 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:11:58.578 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:11:58.587 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:58.586+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:58.607 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:11:58.608 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:11:58.619 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:11:58.620 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:11:58.662 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:11:58.708 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:58.705+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:58.708 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:11:58.708 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:11:58.709 INFO:teuthology.orchestra.run.smithi074.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-08-07T15:11:58.709 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: problem getting command descriptions from osd.1 2020-08-07T15:11:58.709 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:11:58.709 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:11:58.710 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:11:58.710 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:11:58.710 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:11:58.710 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:11:58.711 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-08-07T15:11:58.711 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-08-07T15:11:58.793 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:58.790+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:11:58.842 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:11:58.842 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:11:58.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:11:58.843 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:11:58.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.843 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15576 3.e 3:727724a5:::benchmark_data_smithi074_35447_object1946:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:11:58.843 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:11:58.843 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 312.17155797499998, 2020-08-07T15:11:58.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999, 2020-08-07T15:11:58.855 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:58.855 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:58.857 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:58.857 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:11:58.857 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:11:58.858 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15576 2020-08-07T15:11:58.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.858 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:58.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:58.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:11:58.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:58.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:58.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:11:58.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7156e-05 2020-08-07T15:11:58.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:58.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666682+0000", 2020-08-07T15:11:58.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0209999999999999e-06 2020-08-07T15:11:58.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:58.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666687+0000", 2020-08-07T15:11:58.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0730999999999999e-05 2020-08-07T15:11:58.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:58.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666698+0000", 2020-08-07T15:11:58.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2020-08-07T15:11:58.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:58.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666702+0000", 2020-08-07T15:11:58.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023978459999999999 2020-08-07T15:11:58.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:58.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669100+0000", 2020-08-07T15:11:58.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9664000000000001e-05 2020-08-07T15:11:58.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:58.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669190+0000", 2020-08-07T15:11:58.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015957100000000001 2020-08-07T15:11:58.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:11:58.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669350+0000", 2020-08-07T15:11:58.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016470149999999999 2020-08-07T15:11:58.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:58.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:11:58.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1299999999999999e-07 2020-08-07T15:11:58.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:58.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:11:58.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5269000000000002e-05 2020-08-07T15:11:58.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:58.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.671022+0000", 2020-08-07T15:11:58.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031414400000000001 2020-08-07T15:11:58.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:58.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.674163+0000", 2020-08-07T15:11:58.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45641417400000001 2020-08-07T15:11:58.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:58.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:11:58.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0399999999999995e-07 2020-08-07T15:11:58.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:58.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:11:58.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3343999999999997e-05 2020-08-07T15:11:58.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:58.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130622+0000", 2020-08-07T15:11:58.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999 2020-08-07T15:11:58.893 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:58.894 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:58.894 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:58.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.895 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106248 3.f 3:f950fd9d:::benchmark_data_smithi074_35447_object13280:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:11:58.896 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:11:58.896 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 251.478028435, 2020-08-07T15:11:58.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002, 2020-08-07T15:11:58.899 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:58.900 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:58.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:58.901 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:11:58.901 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:11:58.901 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106248 2020-08-07T15:11:58.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.902 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:58.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:58.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:11:58.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:58.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:58.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:11:58.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7184e-05 2020-08-07T15:11:58.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:58.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360222+0000", 2020-08-07T15:11:58.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0906e-05 2020-08-07T15:11:58.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:58.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360233+0000", 2020-08-07T15:11:58.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2972999999999999e-05 2020-08-07T15:11:58.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:58.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360246+0000", 2020-08-07T15:11:58.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0114e-05 2020-08-07T15:11:58.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:58.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360256+0000", 2020-08-07T15:11:58.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1303999999999997e-05 2020-08-07T15:11:58.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:58.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360287+0000", 2020-08-07T15:11:58.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3798000000000001e-05 2020-08-07T15:11:58.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:58.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360351+0000", 2020-08-07T15:11:58.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111308 2020-08-07T15:11:58.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:11:58.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360462+0000", 2020-08-07T15:11:58.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003480674 2020-08-07T15:11:58.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:58.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:11:58.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0900000000000001e-07 2020-08-07T15:11:58.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:58.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:11:58.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8899999999999994e-05 2020-08-07T15:11:58.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:58.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.364012+0000", 2020-08-07T15:11:58.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35291637199999998 2020-08-07T15:11:58.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:58.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716929+0000", 2020-08-07T15:11:58.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.076244233999999994 2020-08-07T15:11:58.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:58.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793173+0000", 2020-08-07T15:11:58.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6109999999999999e-06 2020-08-07T15:11:58.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:58.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793175+0000", 2020-08-07T15:11:58.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7492999999999999e-05 2020-08-07T15:11:58.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:58.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793242+0000", 2020-08-07T15:11:58.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002 2020-08-07T15:11:58.925 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:58.925 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:58.926 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:58.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.926 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106248 3.f e59/20 ondisk, result = 0)", 2020-08-07T15:11:58.927 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:11:58.927 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 251.47276514999999, 2020-08-07T15:11:58.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001, 2020-08-07T15:11:58.927 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:58.927 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:11:58.928 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:58.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:58.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:11:58.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:58.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:58.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:11:58.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7518999999999999e-05 2020-08-07T15:11:58.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:58.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365445+0000", 2020-08-07T15:11:58.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-06 2020-08-07T15:11:58.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:58.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365449+0000", 2020-08-07T15:11:58.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42764147600000002 2020-08-07T15:11:58.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:58.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793091+0000", 2020-08-07T15:11:58.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6500000000000002e-06 2020-08-07T15:11:58.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:58.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793099+0000", 2020-08-07T15:11:58.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3664e-05 2020-08-07T15:11:58.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:58.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793153+0000", 2020-08-07T15:11:58.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1078e-05 2020-08-07T15:11:58.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:58.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793164+0000", 2020-08-07T15:11:58.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3404000000000003e-05 2020-08-07T15:11:58.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:58.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793248+0000", 2020-08-07T15:11:58.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001 2020-08-07T15:11:58.956 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:58.957 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:58.960 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:58.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.960 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:107952 3.d e61/35)", 2020-08-07T15:11:58.961 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:11:58.961 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 248.51183544200001, 2020-08-07T15:11:58.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001, 2020-08-07T15:11:58.961 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:58.961 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:58.962 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:58.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:58.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:11:58.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:58.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:58.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:11:58.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8826000000000003e-05 2020-08-07T15:11:58.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:58.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326406+0000", 2020-08-07T15:11:58.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9199999999999998e-06 2020-08-07T15:11:58.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:58.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326413+0000", 2020-08-07T15:11:58.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75082853500000002 2020-08-07T15:11:58.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:58.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077242+0000", 2020-08-07T15:11:58.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1479999999999999e-06 2020-08-07T15:11:58.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:58.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077246+0000", 2020-08-07T15:11:58.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3331999999999998e-05 2020-08-07T15:11:58.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:58.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077289+0000", 2020-08-07T15:11:58.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8977000000000001e-05 2020-08-07T15:11:58.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:58.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077308+0000", 2020-08-07T15:11:58.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001 2020-08-07T15:11:58.973 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:58.973 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:58.973 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:58.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.975 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31696 4.0 4:0335eed9:::benchmark_data_smithi074_44586_object3961:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e100)", 2020-08-07T15:11:58.975 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:11:58.975 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 190.73392788300001, 2020-08-07T15:11:58.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237, 2020-08-07T15:11:58.975 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:58.976 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:58.976 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:58.976 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:11:58.976 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:11:58.977 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31696 2020-08-07T15:11:58.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.977 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:58.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:58.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:11:58.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:58.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:58.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:11:58.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7654e-05 2020-08-07T15:11:58.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:58.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104293+0000", 2020-08-07T15:11:58.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.061e-06 2020-08-07T15:11:58.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:58.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104296+0000", 2020-08-07T15:11:58.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0859999999999998e-06 2020-08-07T15:11:58.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:58.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104302+0000", 2020-08-07T15:11:58.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8030000000000001e-06 2020-08-07T15:11:58.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:58.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:58.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:58.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104305+0000", 2020-08-07T15:11:59.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037634769999999999 2020-08-07T15:11:59.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:59.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108068+0000", 2020-08-07T15:11:59.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8814999999999998e-05 2020-08-07T15:11:59.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:59.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108137+0000", 2020-08-07T15:11:59.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111455 2020-08-07T15:11:59.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:11:59.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108248+0000", 2020-08-07T15:11:59.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028756820000000001 2020-08-07T15:11:59.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:59.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111124+0000", 2020-08-07T15:11:59.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061054499999999999 2020-08-07T15:11:59.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:59.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111735+0000", 2020-08-07T15:11:59.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.423966167 2020-08-07T15:11:59.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:59.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535701+0000", 2020-08-07T15:11:59.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4929999999999999e-06 2020-08-07T15:11:59.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:59.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535702+0000", 2020-08-07T15:11:59.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2571999999999997e-05 2020-08-07T15:11:59.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:59.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535755+0000", 2020-08-07T15:11:59.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1427000000000003e-05 2020-08-07T15:11:59.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:59.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535786+0000", 2020-08-07T15:11:59.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237 2020-08-07T15:11:59.051 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.051 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:59.051 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.053 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:31696 4.0 e100/81 ondisk, result = 0)", 2020-08-07T15:11:59.053 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:11:59.053 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 190.724313325, 2020-08-07T15:11:59.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954, 2020-08-07T15:11:59.054 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:59.054 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:11:59.055 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:59.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:59.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:11:59.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:59.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:59.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:11:59.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1185000000000001e-05 2020-08-07T15:11:59.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:59.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113901+0000", 2020-08-07T15:11:59.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2930000000000002e-06 2020-08-07T15:11:59.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:59.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113905+0000", 2020-08-07T15:11:59.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42168640400000001 2020-08-07T15:11:59.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:59.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535591+0000", 2020-08-07T15:11:59.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2820000000000002e-06 2020-08-07T15:11:59.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:59.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535598+0000", 2020-08-07T15:11:59.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6578999999999999e-05 2020-08-07T15:11:59.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:59.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535684+0000", 2020-08-07T15:11:59.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8449999999999992e-06 2020-08-07T15:11:59.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:59.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535693+0000", 2020-08-07T15:11:59.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7366000000000003e-05 2020-08-07T15:11:59.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:59.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535791+0000", 2020-08-07T15:11:59.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954 2020-08-07T15:11:59.073 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.073 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:59.073 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.074 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:37024 4.0 e106/81)", 2020-08-07T15:11:59.075 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:11:59.075 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 184.60952443100001, 2020-08-07T15:11:59.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999, 2020-08-07T15:11:59.076 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:59.076 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:11:59.077 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:59.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:59.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:11:59.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:59.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:59.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:11:59.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4926000000000001e-05 2020-08-07T15:11:59.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:59.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228683+0000", 2020-08-07T15:11:59.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1280000000000002e-06 2020-08-07T15:11:59.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:59.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228686+0000", 2020-08-07T15:11:59.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69740012100000004 2020-08-07T15:11:59.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:59.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926086+0000", 2020-08-07T15:11:59.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000004e-06 2020-08-07T15:11:59.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:59.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926090+0000", 2020-08-07T15:11:59.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113526 2020-08-07T15:11:59.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:59.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926204+0000", 2020-08-07T15:11:59.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8824000000000001e-05 2020-08-07T15:11:59.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:59.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926222+0000", 2020-08-07T15:11:59.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999 2020-08-07T15:11:59.092 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.092 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:59.093 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.094 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42503 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2615, mlcod=112'2607)", 2020-08-07T15:11:59.095 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:11:59.095 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 178.54195251499999, 2020-08-07T15:11:59.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998, 2020-08-07T15:11:59.095 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:59.096 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:59.096 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:59.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:59.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:11:59.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:59.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:59.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:11:59.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270999999999998e-05 2020-08-07T15:11:59.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:59.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296292+0000", 2020-08-07T15:11:59.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4729999999999997e-06 2020-08-07T15:11:59.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:59.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296298+0000", 2020-08-07T15:11:59.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.448795049 2020-08-07T15:11:59.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:59.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745093+0000", 2020-08-07T15:11:59.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0140000000000001e-06 2020-08-07T15:11:59.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:59.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745100+0000", 2020-08-07T15:11:59.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2596999999999998e-05 2020-08-07T15:11:59.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:59.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745153+0000", 2020-08-07T15:11:59.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1662000000000003e-05 2020-08-07T15:11:59.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:59.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745184+0000", 2020-08-07T15:11:59.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093174800000000004 2020-08-07T15:11:59.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:59.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746116+0000", 2020-08-07T15:11:59.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0926999999999994e-05 2020-08-07T15:11:59.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:59.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746187+0000", 2020-08-07T15:11:59.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998 2020-08-07T15:11:59.119 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.119 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:59.119 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.121 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42504 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2616, mlcod=113'2609)", 2020-08-07T15:11:59.121 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:11:59.122 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 178.53861562200001, 2020-08-07T15:11:59.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998, 2020-08-07T15:11:59.122 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:59.123 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:59.123 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:59.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:59.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:11:59.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:59.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:59.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:11:59.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2154000000000002e-05 2020-08-07T15:11:59.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:59.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299629+0000", 2020-08-07T15:11:59.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1929999999999998e-06 2020-08-07T15:11:59.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:59.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299636+0000", 2020-08-07T15:11:59.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44766315400000001 2020-08-07T15:11:59.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:59.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747299+0000", 2020-08-07T15:11:59.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5889999999999999e-06 2020-08-07T15:11:59.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:59.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747305+0000", 2020-08-07T15:11:59.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3094999999999999e-05 2020-08-07T15:11:59.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:59.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747358+0000", 2020-08-07T15:11:59.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3283e-05 2020-08-07T15:11:59.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:59.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747392+0000", 2020-08-07T15:11:59.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015319509999999999 2020-08-07T15:11:59.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:59.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748924+0000", 2020-08-07T15:11:59.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5235999999999996e-05 2020-08-07T15:11:59.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:59.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748999+0000", 2020-08-07T15:11:59.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998 2020-08-07T15:11:59.153 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.153 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:59.153 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.154 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42520 4.8 e113/81 4:13071de5:::benchmark_data_smithi074_44586_object5314:head v 113'2624, mlcod=113'2618)", 2020-08-07T15:11:59.154 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:11:59.154 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 178.07267141299999, 2020-08-07T15:11:59.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001, 2020-08-07T15:11:59.155 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:59.155 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:59.155 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:59.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:59.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:11:59.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:59.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:59.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:11:59.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3967999999999995e-05 2020-08-07T15:11:59.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:59.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765585+0000", 2020-08-07T15:11:59.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0949999999999998e-06 2020-08-07T15:11:59.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:59.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765593+0000", 2020-08-07T15:11:59.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52703464200000005 2020-08-07T15:11:59.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:59.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292627+0000", 2020-08-07T15:11:59.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6929999999999996e-06 2020-08-07T15:11:59.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:59.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292636+0000", 2020-08-07T15:11:59.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9292000000000001e-05 2020-08-07T15:11:59.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:59.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292695+0000", 2020-08-07T15:11:59.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5395999999999999e-05 2020-08-07T15:11:59.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:59.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292721+0000", 2020-08-07T15:11:59.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00090361399999999996 2020-08-07T15:11:59.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:59.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293624+0000", 2020-08-07T15:11:59.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1761000000000002e-05 2020-08-07T15:11:59.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:59.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293666+0000", 2020-08-07T15:11:59.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001 2020-08-07T15:11:59.168 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.168 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:59.168 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.170 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:19776 5.4 5:2bb0c0d9:::benchmark_data_smithi074_53697_object2471:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e173)", 2020-08-07T15:11:59.171 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:11:59.171 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.438478728000007, 2020-08-07T15:11:59.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.402061844, 2020-08-07T15:11:59.172 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:59.172 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:59.172 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:59.172 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:11:59.172 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:59.173 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 19776 2020-08-07T15:11:59.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.173 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:59.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:59.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:11:59.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:59.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:59.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:11:59.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-05 2020-08-07T15:11:59.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:59.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399752+0000", 2020-08-07T15:11:59.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8799999999999999e-06 2020-08-07T15:11:59.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:59.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399757+0000", 2020-08-07T15:11:59.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7690000000000008e-06 2020-08-07T15:11:59.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:59.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399767+0000", 2020-08-07T15:11:59.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-06 2020-08-07T15:11:59.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:59.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399771+0000", 2020-08-07T15:11:59.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003255109 2020-08-07T15:11:59.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:59.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403026+0000", 2020-08-07T15:11:59.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101044 2020-08-07T15:11:59.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:59.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403127+0000", 2020-08-07T15:11:59.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000193158 2020-08-07T15:11:59.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,5", 2020-08-07T15:11:59.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403320+0000", 2020-08-07T15:11:59.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017185530000000001 2020-08-07T15:11:59.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:59.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405038+0000", 2020-08-07T15:11:59.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6300000000000004e-07 2020-08-07T15:11:59.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:59.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405039+0000", 2020-08-07T15:11:59.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012590600000000001 2020-08-07T15:11:59.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:59.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405165+0000", 2020-08-07T15:11:59.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109757 2020-08-07T15:11:59.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:59.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405275+0000", 2020-08-07T15:11:59.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39642174699999999 2020-08-07T15:11:59.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:59.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801697+0000", 2020-08-07T15:11:59.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2799999999999996e-07 2020-08-07T15:11:59.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:59.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801697+0000", 2020-08-07T15:11:59.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8740999999999997e-05 2020-08-07T15:11:59.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:59.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801776+0000", 2020-08-07T15:11:59.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.402061844 2020-08-07T15:11:59.191 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.191 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:59.192 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.192 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:19776 5.4 e173/164 ondisk, result = 0)", 2020-08-07T15:11:59.192 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:11:59.193 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.432730425000003, 2020-08-07T15:11:59.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39631765200000002, 2020-08-07T15:11:59.193 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:59.193 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:11:59.194 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:59.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:59.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:11:59.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:59.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:59.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:11:59.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9332e-05 2020-08-07T15:11:59.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:59.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405492+0000", 2020-08-07T15:11:59.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8900000000000004e-06 2020-08-07T15:11:59.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:59.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405498+0000", 2020-08-07T15:11:59.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39566664200000001 2020-08-07T15:11:59.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:59.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801164+0000", 2020-08-07T15:11:59.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7440000000000001e-06 2020-08-07T15:11:59.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:59.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801168+0000", 2020-08-07T15:11:59.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050828199999999998 2020-08-07T15:11:59.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:59.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801676+0000", 2020-08-07T15:11:59.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1011e-05 2020-08-07T15:11:59.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:59.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801687+0000", 2020-08-07T15:11:59.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2750999999999993e-05 2020-08-07T15:11:59.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:59.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801780+0000", 2020-08-07T15:11:59.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39631765200000002 2020-08-07T15:11:59.204 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.204 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:59.204 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.205 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:85551 5.9 5:9d69b7cc:::benchmark_data_smithi074_53697_object10693:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e202)", 2020-08-07T15:11:59.205 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.807277+0000", 2020-08-07T15:11:59.205 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.030916357999999, 2020-08-07T15:11:59.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39614286500000001, 2020-08-07T15:11:59.206 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:59.206 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:59.206 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:59.206 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:11:59.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:59.207 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 85551 2020-08-07T15:11:59.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.207 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:59.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:59.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807277+0000", 2020-08-07T15:11:59.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:59.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:59.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807277+0000", 2020-08-07T15:11:59.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8867999999999999e-05 2020-08-07T15:11:59.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:59.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807305+0000", 2020-08-07T15:11:59.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.775e-06 2020-08-07T15:11:59.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:59.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807310+0000", 2020-08-07T15:11:59.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2747e-05 2020-08-07T15:11:59.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:59.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807323+0000", 2020-08-07T15:11:59.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2669999999999997e-06 2020-08-07T15:11:59.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:59.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807328+0000", 2020-08-07T15:11:59.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001048513 2020-08-07T15:11:59.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:59.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.808377+0000", 2020-08-07T15:11:59.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010425399999999999 2020-08-07T15:11:59.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:59.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.808481+0000", 2020-08-07T15:11:59.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013761900000000001 2020-08-07T15:11:59.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:11:59.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.808619+0000", 2020-08-07T15:11:59.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042907500000000003 2020-08-07T15:11:59.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:59.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.812909+0000", 2020-08-07T15:11:59.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.077e-06 2020-08-07T15:11:59.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:59.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.812910+0000", 2020-08-07T15:11:59.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8186000000000001e-05 2020-08-07T15:11:59.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:59.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.812979+0000", 2020-08-07T15:11:59.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00081526099999999996 2020-08-07T15:11:59.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:59.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813794+0000", 2020-08-07T15:11:59.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38955804599999999 2020-08-07T15:11:59.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:59.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203352+0000", 2020-08-07T15:11:59.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5239999999999999e-06 2020-08-07T15:11:59.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:59.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203353+0000", 2020-08-07T15:11:59.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5977999999999997e-05 2020-08-07T15:11:59.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:59.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203419+0000", 2020-08-07T15:11:59.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39614286500000001 2020-08-07T15:11:59.234 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.234 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:59.235 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.236 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:85552 5.9 5:9d69b7cc:::benchmark_data_smithi074_53697_object10693:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e202)", 2020-08-07T15:11:59.236 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:11:59.236 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.030675633999998, 2020-08-07T15:11:59.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54429374399999997, 2020-08-07T15:11:59.237 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:59.238 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:59.239 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:59.239 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:11:59.240 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:59.240 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 85552 2020-08-07T15:11:59.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.242 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:59.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:59.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:11:59.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:59.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:59.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:11:59.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2691000000000001e-05 2020-08-07T15:11:59.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:59.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807570+0000", 2020-08-07T15:11:59.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0619999999999997e-06 2020-08-07T15:11:59.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:59.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807576+0000", 2020-08-07T15:11:59.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1558e-05 2020-08-07T15:11:59.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:59.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807588+0000", 2020-08-07T15:11:59.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4600000000000002e-06 2020-08-07T15:11:59.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:59.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807593+0000", 2020-08-07T15:11:59.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022126279999999999 2020-08-07T15:11:59.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:59.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.809806+0000", 2020-08-07T15:11:59.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104868 2020-08-07T15:11:59.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:59.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.809911+0000", 2020-08-07T15:11:59.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000186019 2020-08-07T15:11:59.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:11:59.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.810097+0000", 2020-08-07T15:11:59.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037966739999999999 2020-08-07T15:11:59.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:59.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813893+0000", 2020-08-07T15:11:59.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1310000000000001e-06 2020-08-07T15:11:59.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:59.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813894+0000", 2020-08-07T15:11:59.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4704999999999999e-05 2020-08-07T15:11:59.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:59.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813949+0000", 2020-08-07T15:11:59.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015487280000000001 2020-08-07T15:11:59.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:59.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.815498+0000", 2020-08-07T15:11:59.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53621442900000005 2020-08-07T15:11:59.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:59.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351712+0000", 2020-08-07T15:11:59.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8199999999999998e-07 2020-08-07T15:11:59.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:59.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351713+0000", 2020-08-07T15:11:59.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7909000000000001e-05 2020-08-07T15:11:59.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:59.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351811+0000", 2020-08-07T15:11:59.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54429374399999997 2020-08-07T15:11:59.283 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.283 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:59.283 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.284 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:85551 5.9 e202/164 ondisk, result = 0)", 2020-08-07T15:11:59.285 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.813683+0000", 2020-08-07T15:11:59.285 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.024510331000002, 2020-08-07T15:11:59.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38973959000000002, 2020-08-07T15:11:59.286 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:59.287 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:11:59.287 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:59.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:59.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813683+0000", 2020-08-07T15:11:59.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:59.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:59.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813683+0000", 2020-08-07T15:11:59.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9434999999999999e-05 2020-08-07T15:11:59.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:59.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813712+0000", 2020-08-07T15:11:59.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6759999999999998e-06 2020-08-07T15:11:59.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:59.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813717+0000", 2020-08-07T15:11:59.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.389373209 2020-08-07T15:11:59.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:59.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203090+0000", 2020-08-07T15:11:59.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0859999999999998e-06 2020-08-07T15:11:59.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:59.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203096+0000", 2020-08-07T15:11:59.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024188899999999999 2020-08-07T15:11:59.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:59.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203338+0000", 2020-08-07T15:11:59.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2490000000000001e-06 2020-08-07T15:11:59.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:59.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203346+0000", 2020-08-07T15:11:59.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6045999999999997e-05 2020-08-07T15:11:59.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:59.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203422+0000", 2020-08-07T15:11:59.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38973959000000002 2020-08-07T15:11:59.307 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.307 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:59.308 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.309 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:85552 5.9 e202/164 ondisk, result = 0)", 2020-08-07T15:11:59.309 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:11:59.310 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.024449232000002, 2020-08-07T15:11:59.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53813836800000003, 2020-08-07T15:11:59.311 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:59.312 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:11:59.312 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:59.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:59.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:11:59.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:59.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:59.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:11:59.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8833e-05 2020-08-07T15:11:59.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:59.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813772+0000", 2020-08-07T15:11:59.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8459999999999999e-06 2020-08-07T15:11:59.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:59.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813777+0000", 2020-08-07T15:11:59.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53732224900000003 2020-08-07T15:11:59.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:59.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351100+0000", 2020-08-07T15:11:59.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1869999999999996e-06 2020-08-07T15:11:59.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:59.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351105+0000", 2020-08-07T15:11:59.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00058858100000000002 2020-08-07T15:11:59.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:59.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351693+0000", 2020-08-07T15:11:59.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4229999999999994e-06 2020-08-07T15:11:59.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:59.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351703+0000", 2020-08-07T15:11:59.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000179249 2020-08-07T15:11:59.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:59.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351882+0000", 2020-08-07T15:11:59.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53813836800000003 2020-08-07T15:11:59.344 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.344 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:59.345 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.346 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:85552 5.9 5:9d69b7cc:::benchmark_data_smithi074_53697_object10693:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e202)", 2020-08-07T15:11:59.347 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:11:59.347 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.019701108, 2020-08-07T15:11:59.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53326610600000002, 2020-08-07T15:11:59.349 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:59.349 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:11:59.350 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:59.350 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:11:59.350 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:59.351 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 85552 2020-08-07T15:11:59.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.352 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:59.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:59.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:11:59.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:59.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:59.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:11:59.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5001e-05 2020-08-07T15:11:59.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:59.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818517+0000", 2020-08-07T15:11:59.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5000000000000002e-06 2020-08-07T15:11:59.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:59.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818519+0000", 2020-08-07T15:11:59.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0699999999999997e-06 2020-08-07T15:11:59.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:59.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818524+0000", 2020-08-07T15:11:59.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2910000000000002e-06 2020-08-07T15:11:59.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:59.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818527+0000", 2020-08-07T15:11:59.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030912499999999999 2020-08-07T15:11:59.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:59.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818836+0000", 2020-08-07T15:11:59.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2908000000000002e-05 2020-08-07T15:11:59.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for ondisk", 2020-08-07T15:11:59.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818889+0000", 2020-08-07T15:11:59.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53286921099999995 2020-08-07T15:11:59.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:59.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351758+0000", 2020-08-07T15:11:59.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53326610600000002 2020-08-07T15:11:59.365 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.365 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:59.365 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.368 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:89464 5.9 5:94c69ba5:::benchmark_data_smithi074_53697_object11182:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:11:59.368 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:11:59.368 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.421861153999998, 2020-08-07T15:11:59.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67692916199999997, 2020-08-07T15:11:59.369 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:59.369 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:59.369 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:59.371 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:11:59.371 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:59.371 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 89464 2020-08-07T15:11:59.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.372 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:59.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:59.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:11:59.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:59.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:59.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:11:59.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0074000000000002e-05 2020-08-07T15:11:59.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:59.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416382+0000", 2020-08-07T15:11:59.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5000000000000001e-06 2020-08-07T15:11:59.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:59.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416386+0000", 2020-08-07T15:11:59.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2447e-05 2020-08-07T15:11:59.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:59.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416399+0000", 2020-08-07T15:11:59.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1279999999999996e-06 2020-08-07T15:11:59.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:59.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416405+0000", 2020-08-07T15:11:59.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016641805999999999 2020-08-07T15:11:59.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:59.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433047+0000", 2020-08-07T15:11:59.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107995 2020-08-07T15:11:59.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:59.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433155+0000", 2020-08-07T15:11:59.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018818599999999999 2020-08-07T15:11:59.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:11:59.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433343+0000", 2020-08-07T15:11:59.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039911679999999998 2020-08-07T15:11:59.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:59.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437334+0000", 2020-08-07T15:11:59.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2020-08-07T15:11:59.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:59.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437335+0000", 2020-08-07T15:11:59.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6351000000000002e-05 2020-08-07T15:11:59.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:59.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437401+0000", 2020-08-07T15:11:59.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001267949 2020-08-07T15:11:59.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:59.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438669+0000", 2020-08-07T15:11:59.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65454035600000005 2020-08-07T15:11:59.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:59.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093209+0000", 2020-08-07T15:11:59.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7859999999999999e-06 2020-08-07T15:11:59.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:59.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093211+0000", 2020-08-07T15:11:59.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9665e-05 2020-08-07T15:11:59.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:59.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093261+0000", 2020-08-07T15:11:59.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67692916199999997 2020-08-07T15:11:59.399 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.399 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:59.400 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.401 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:89464 5.9 e207/164 ondisk, result = 0)", 2020-08-07T15:11:59.401 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:11:59.401 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.399231391999997, 2020-08-07T15:11:59.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65430647200000003, 2020-08-07T15:11:59.402 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:59.414 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:11:59.415 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:59.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:59.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:11:59.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:59.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:59.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:11:59.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6529000000000002e-05 2020-08-07T15:11:59.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:59.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438998+0000", 2020-08-07T15:11:59.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.567e-06 2020-08-07T15:11:59.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:59.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.439005+0000", 2020-08-07T15:11:59.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65408475099999996 2020-08-07T15:11:59.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:59.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093089+0000", 2020-08-07T15:11:59.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6880000000000002e-06 2020-08-07T15:11:59.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:59.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093096+0000", 2020-08-07T15:11:59.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4980000000000002e-05 2020-08-07T15:11:59.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:59.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093191+0000", 2020-08-07T15:11:59.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3090000000000001e-06 2020-08-07T15:11:59.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:59.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093200+0000", 2020-08-07T15:11:59.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7648e-05 2020-08-07T15:11:59.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:59.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093268+0000", 2020-08-07T15:11:59.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65430647200000003 2020-08-07T15:11:59.433 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.433 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:59.433 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.435 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:114080 5.4 5:24adcd54:::benchmark_data_smithi074_53697_object14259:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e215)", 2020-08-07T15:11:59.435 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:11:59.435 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.238523935, 2020-08-07T15:11:59.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.57290862300000001, 2020-08-07T15:11:59.436 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:11:59.436 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:11:59.436 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:11:59.437 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:11:59.437 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:59.437 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 114080 2020-08-07T15:11:59.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.438 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:11:59.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:11:59.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:11:59.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:11:59.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:11:59.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:11:59.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9238e-05 2020-08-07T15:11:59.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:11:59.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599698+0000", 2020-08-07T15:11:59.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6260000000000003e-06 2020-08-07T15:11:59.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:11:59.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599703+0000", 2020-08-07T15:11:59.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5577999999999999e-05 2020-08-07T15:11:59.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:11:59.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599718+0000", 2020-08-07T15:11:59.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9869999999999999e-06 2020-08-07T15:11:59.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:11:59.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599726+0000", 2020-08-07T15:11:59.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00077929599999999996 2020-08-07T15:11:59.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:11:59.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600506+0000", 2020-08-07T15:11:59.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1717000000000003e-05 2020-08-07T15:11:59.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:11:59.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600597+0000", 2020-08-07T15:11:59.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011828999999999999 2020-08-07T15:11:59.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,5", 2020-08-07T15:11:59.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600716+0000", 2020-08-07T15:11:59.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020861479999999999 2020-08-07T15:11:59.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:11:59.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602802+0000", 2020-08-07T15:11:59.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4600000000000005e-07 2020-08-07T15:11:59.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:11:59.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602802+0000", 2020-08-07T15:11:59.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9165e-05 2020-08-07T15:11:59.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:11:59.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602862+0000", 2020-08-07T15:11:59.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0093485080000000002 2020-08-07T15:11:59.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:59.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.612210+0000", 2020-08-07T15:11:59.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.56030197699999995 2020-08-07T15:11:59.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:59.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172512+0000", 2020-08-07T15:11:59.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8800000000000001e-07 2020-08-07T15:11:59.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:11:59.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172513+0000", 2020-08-07T15:11:59.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4659000000000006e-05 2020-08-07T15:11:59.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:11:59.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:11:59.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:11:59.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172578+0000", 2020-08-07T15:11:59.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.57290862300000001 2020-08-07T15:11:59.459 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.459 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:59.459 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.459 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:11:59.459 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:11:59.460 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:11:59.462 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:11:59.659 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:59.648+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:59.659 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:11:59.648+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:11:59.674 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:11:59.663+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:11:59.733 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:11:59.733 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:11:59.758 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:59.772 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:142240 5.7 5:ef62e09b:::benchmark_data_smithi074_53697_object15319:head [delete] snapc 0=[] ondisk+write+known_if_redirected e225)", 2020-08-07T15:11:59.772 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:59.707826+0000", 2020-08-07T15:11:59.773 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.019133230000000001, 2020-08-07T15:11:59.773 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.019150809000000001, 2020-08-07T15:11:59.773 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:11:59.773 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:11:59.773 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:11:59.774 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:11:59.774 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:11:59.774 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 142240 2020-08-07T15:11:59.774 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:59.775 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:11:59.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:59.775 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:11:59.775 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:59.707826+0000", 2020-08-07T15:11:59.775 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:11:59.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:59.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:59.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:11:59.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:59.707826+0000", 2020-08-07T15:11:59.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3232999999999999e-05 2020-08-07T15:11:59.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:59.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:59.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:11:59.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:59.707849+0000", 2020-08-07T15:11:59.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9129999999999996e-06 2020-08-07T15:11:59.782 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:59.782 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:59.782 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:11:59.782 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:59.707853+0000", 2020-08-07T15:11:59.783 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6430000000000007e-06 2020-08-07T15:11:59.783 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:59.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:59.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:11:59.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:59.707863+0000", 2020-08-07T15:11:59.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3240000000000002e-06 2020-08-07T15:11:59.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:59.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:59.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:11:59.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:59.707868+0000", 2020-08-07T15:11:59.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8902e-05 2020-08-07T15:11:59.785 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:59.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:59.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:11:59.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:59.707917+0000", 2020-08-07T15:11:59.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016290099999999999 2020-08-07T15:11:59.806 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:59.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:59.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:11:59.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:59.708080+0000", 2020-08-07T15:11:59.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017722199999999999 2020-08-07T15:11:59.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:59.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:59.836 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:11:59.836 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:59.708257+0000", 2020-08-07T15:11:59.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012151779999999999 2020-08-07T15:11:59.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:59.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:59.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:11:59.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:59.709472+0000", 2020-08-07T15:11:59.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000760211 2020-08-07T15:11:59.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:11:59.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:11:59.838 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:11:59.838 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:59.710232+0000", 2020-08-07T15:11:59.838 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024065269999999999 2020-08-07T15:11:59.840 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:59.840 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:11:59.841 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:59.841 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:11:59.841 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:11:59.841 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 1 2020-08-07T15:11:59.841 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:11:59.843 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:11:59.842+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:11:59.844 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:11:59.956 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:11:59.954+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:00.063 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:00.063 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:12:00.064 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:12:00.064 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:12:00.064 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:00.065 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:00.061+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:00.090 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:12:00.165 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3, 4, 5, 6, 7] 2020-08-07T15:12:00.166 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:12:00.166 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2020-08-07T15:12:00.167 INFO:tasks.ceph.osd.1:Restarting daemon 2020-08-07T15:12:00.167 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2020-08-07T15:12:00.169 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:00.163+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:00.184 INFO:tasks.ceph.osd.1:Started 2020-08-07T15:12:00.185 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:12:00.258 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:00.258 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:12:00.259 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:12:00.259 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:12:00.264 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.265 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:145801 3.6 e75/26 3:69e4cb22:::benchmark_data_smithi074_35447_object16192:head v 75'9405, mlcod=75'9401)", 2020-08-07T15:12:00.265 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:12:00.265 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 225.580490551, 2020-08-07T15:12:00.266 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639, 2020-08-07T15:12:00.266 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:00.267 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:00.268 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:00.269 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.269 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:00.269 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:12:00.269 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:00.270 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.271 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:00.271 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:12:00.271 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6511000000000002e-05 2020-08-07T15:12:00.271 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:00.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671803+0000", 2020-08-07T15:12:00.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7190000000000004e-06 2020-08-07T15:12:00.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.273 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:00.273 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671809+0000", 2020-08-07T15:12:00.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74210928300000001 2020-08-07T15:12:00.273 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.274 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.274 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:00.274 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413919+0000", 2020-08-07T15:12:00.274 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1759999999999999e-05 2020-08-07T15:12:00.275 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.275 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.275 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:00.275 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413930+0000", 2020-08-07T15:12:00.275 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000108601 2020-08-07T15:12:00.276 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.276 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.276 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:00.276 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414039+0000", 2020-08-07T15:12:00.277 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9948999999999999e-05 2020-08-07T15:12:00.277 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.277 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.278 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:00.278 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414089+0000", 2020-08-07T15:12:00.278 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00079656499999999995 2020-08-07T15:12:00.278 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.279 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.279 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:00.279 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414885+0000", 2020-08-07T15:12:00.279 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2250999999999998e-05 2020-08-07T15:12:00.279 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.280 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.280 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:00.280 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414938+0000", 2020-08-07T15:12:00.281 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639 2020-08-07T15:12:00.281 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:00.281 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:00.281 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:00.282 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.282 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.282 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:104104 4.0 e146/131 4:0ffb764a:::benchmark_data_smithi074_44586_object13012:head v 146'6960, mlcod=146'6953)", 2020-08-07T15:12:00.282 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:12:00.283 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 132.05394377499999, 2020-08-07T15:12:00.283 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75443331599999996, 2020-08-07T15:12:00.283 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:00.292 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:00.292 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:00.292 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.292 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:00.307 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:12:00.307 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:00.308 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.308 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:00.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:12:00.309 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1743000000000002e-05 2020-08-07T15:12:00.309 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.309 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.309 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:00.317 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198355+0000", 2020-08-07T15:12:00.317 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1249999999999998e-06 2020-08-07T15:12:00.317 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.318 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.318 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:00.318 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198361+0000", 2020-08-07T15:12:00.318 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74719179599999996 2020-08-07T15:12:00.318 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.319 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.319 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:00.319 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945553+0000", 2020-08-07T15:12:00.319 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6800000000000001e-06 2020-08-07T15:12:00.320 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.320 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.320 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:00.320 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945557+0000", 2020-08-07T15:12:00.320 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012101500000000001 2020-08-07T15:12:00.321 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.321 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.321 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:00.321 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945678+0000", 2020-08-07T15:12:00.321 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6621000000000001e-05 2020-08-07T15:12:00.343 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.351 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.351 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:00.351 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945705+0000", 2020-08-07T15:12:00.352 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069879540000000002 2020-08-07T15:12:00.352 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.361 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.362 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:00.362 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.952693+0000", 2020-08-07T15:12:00.362 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3381999999999999e-05 2020-08-07T15:12:00.362 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.382 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.394 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:00.395 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.952736+0000", 2020-08-07T15:12:00.396 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75443331599999996 2020-08-07T15:12:00.419 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:00.428 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:00.428 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:00.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.429 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89641 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:00.429 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.550933+0000", 2020-08-07T15:12:00.430 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 38.701313354, 2020-08-07T15:12:00.430 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69051440900000005, 2020-08-07T15:12:00.430 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:00.459 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:00.459 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:00.460 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:00.460 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:00.460 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89641 2020-08-07T15:12:00.461 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.461 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:00.461 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.461 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:00.493 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550933+0000", 2020-08-07T15:12:00.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:00.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.508 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.509 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:00.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550933+0000", 2020-08-07T15:12:00.510 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6520000000000002e-05 2020-08-07T15:12:00.510 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:00.528 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550980+0000", 2020-08-07T15:12:00.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2410000000000003e-06 2020-08-07T15:12:00.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.534 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.534 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:00.534 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550985+0000", 2020-08-07T15:12:00.535 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2707e-05 2020-08-07T15:12:00.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:00.535 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550998+0000", 2020-08-07T15:12:00.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8629999999999996e-06 2020-08-07T15:12:00.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.536 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:00.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551004+0000", 2020-08-07T15:12:00.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4987000000000001e-05 2020-08-07T15:12:00.537 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.537 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.537 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:00.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551059+0000", 2020-08-07T15:12:00.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1718e-05 2020-08-07T15:12:00.538 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.538 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.538 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:00.538 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551070+0000", 2020-08-07T15:12:00.538 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67720431000000003 2020-08-07T15:12:00.539 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.546 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.546 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:00.546 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.228275+0000", 2020-08-07T15:12:00.547 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000731965 2020-08-07T15:12:00.547 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.547 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:00.548 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229007+0000", 2020-08-07T15:12:00.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00025396199999999999 2020-08-07T15:12:00.548 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.548 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:00.549 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229261+0000", 2020-08-07T15:12:00.549 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002511189 2020-08-07T15:12:00.549 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.562 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.563 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:00.563 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231772+0000", 2020-08-07T15:12:00.563 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3399999999999997e-07 2020-08-07T15:12:00.563 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.564 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:00.564 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231773+0000", 2020-08-07T15:12:00.564 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0274e-05 2020-08-07T15:12:00.564 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.565 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.565 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:00.565 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231823+0000", 2020-08-07T15:12:00.565 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0094095440000000006 2020-08-07T15:12:00.565 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.574 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.574 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:00.575 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241233+0000", 2020-08-07T15:12:00.575 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4011999999999998e-05 2020-08-07T15:12:00.575 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.575 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:00.576 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241277+0000", 2020-08-07T15:12:00.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7199999999999999e-07 2020-08-07T15:12:00.576 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.576 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.576 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:00.582 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241277+0000", 2020-08-07T15:12:00.582 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017061100000000001 2020-08-07T15:12:00.582 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.582 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.583 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:00.583 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241448+0000", 2020-08-07T15:12:00.583 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69051440900000005 2020-08-07T15:12:00.583 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:00.583 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:00.584 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:00.584 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.584 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.584 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89642 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:00.585 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.551807+0000", 2020-08-07T15:12:00.585 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 38.700439801999998, 2020-08-07T15:12:00.585 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68991613100000004, 2020-08-07T15:12:00.591 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:00.591 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:00.591 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:00.592 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:00.592 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:00.592 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89642 2020-08-07T15:12:00.593 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.593 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:00.608 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.608 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:00.614 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551807+0000", 2020-08-07T15:12:00.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:00.615 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.615 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.615 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:00.616 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551807+0000", 2020-08-07T15:12:00.616 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7049000000000001e-05 2020-08-07T15:12:00.616 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.617 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.617 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:00.617 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551834+0000", 2020-08-07T15:12:00.634 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9249999999999997e-06 2020-08-07T15:12:00.635 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.635 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.635 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:00.635 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551838+0000", 2020-08-07T15:12:00.635 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9160000000000001e-06 2020-08-07T15:12:00.636 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.636 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.636 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:00.636 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551846+0000", 2020-08-07T15:12:00.636 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3510000000000002e-06 2020-08-07T15:12:00.637 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.637 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.637 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:00.637 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551849+0000", 2020-08-07T15:12:00.637 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9276000000000001e-05 2020-08-07T15:12:00.659 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.659 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.660 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:00.660 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551898+0000", 2020-08-07T15:12:00.660 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3095000000000001e-05 2020-08-07T15:12:00.661 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.661 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.661 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:00.673 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551911+0000", 2020-08-07T15:12:00.674 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67789159799999998 2020-08-07T15:12:00.674 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.674 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.674 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:00.675 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229803+0000", 2020-08-07T15:12:00.675 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8069000000000001e-05 2020-08-07T15:12:00.675 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.675 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.675 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:00.675 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229881+0000", 2020-08-07T15:12:00.676 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000152917 2020-08-07T15:12:00.676 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.676 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.676 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:00.676 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230034+0000", 2020-08-07T15:12:00.677 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044936760000000003 2020-08-07T15:12:00.677 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:00.677 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234528+0000", 2020-08-07T15:12:00.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8999999999999995e-07 2020-08-07T15:12:00.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:00.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234529+0000", 2020-08-07T15:12:00.698 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7741000000000002e-05 2020-08-07T15:12:00.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.698 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.699 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:00.699 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234586+0000", 2020-08-07T15:12:00.699 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069143959999999997 2020-08-07T15:12:00.700 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.700 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:00.701 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241501+0000", 2020-08-07T15:12:00.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1433000000000001e-05 2020-08-07T15:12:00.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.722 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:00.722 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241542+0000", 2020-08-07T15:12:00.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7800000000000001e-07 2020-08-07T15:12:00.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.723 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:00.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241543+0000", 2020-08-07T15:12:00.724 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000180321 2020-08-07T15:12:00.724 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.724 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.740 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:00.740 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241723+0000", 2020-08-07T15:12:00.741 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68991613100000004 2020-08-07T15:12:00.741 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:00.741 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:00.741 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:00.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.742 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.742 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89643 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:00.742 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.552181+0000", 2020-08-07T15:12:00.742 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 38.700065807999998, 2020-08-07T15:12:00.742 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68980872400000004, 2020-08-07T15:12:00.743 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:00.743 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:00.743 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:00.744 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:00.744 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:00.744 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89643 2020-08-07T15:12:00.745 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.745 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:00.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.745 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:00.779 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552181+0000", 2020-08-07T15:12:00.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:00.794 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.794 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.795 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:00.795 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552181+0000", 2020-08-07T15:12:00.795 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0926999999999997e-05 2020-08-07T15:12:00.795 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:00.809 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552212+0000", 2020-08-07T15:12:00.809 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0512e-05 2020-08-07T15:12:00.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.810 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:00.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552222+0000", 2020-08-07T15:12:00.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9379999999999992e-06 2020-08-07T15:12:00.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:00.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552230+0000", 2020-08-07T15:12:00.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0260000000000001e-06 2020-08-07T15:12:00.860 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.861 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:00.861 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552234+0000", 2020-08-07T15:12:00.862 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6109000000000002e-05 2020-08-07T15:12:00.862 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.862 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:00.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552280+0000", 2020-08-07T15:12:00.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5318000000000001e-05 2020-08-07T15:12:00.864 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.865 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:00.865 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552296+0000", 2020-08-07T15:12:00.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67811269100000005 2020-08-07T15:12:00.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:00.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230408+0000", 2020-08-07T15:12:00.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000125518 2020-08-07T15:12:00.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.908 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:00.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230534+0000", 2020-08-07T15:12:00.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024637999999999998 2020-08-07T15:12:00.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:00.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230780+0000", 2020-08-07T15:12:00.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0066398869999999997 2020-08-07T15:12:00.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:00.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237420+0000", 2020-08-07T15:12:00.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3500000000000005e-07 2020-08-07T15:12:00.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:00.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237421+0000", 2020-08-07T15:12:00.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4005000000000003e-05 2020-08-07T15:12:00.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:00.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237465+0000", 2020-08-07T15:12:00.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043133529999999998 2020-08-07T15:12:00.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:00.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241778+0000", 2020-08-07T15:12:00.969 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0592e-05 2020-08-07T15:12:00.969 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.969 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.969 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:00.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241819+0000", 2020-08-07T15:12:00.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4999999999999998e-07 2020-08-07T15:12:00.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.970 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:00.970 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241819+0000", 2020-08-07T15:12:00.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017018299999999999 2020-08-07T15:12:00.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:00.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241990+0000", 2020-08-07T15:12:00.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68980872400000004 2020-08-07T15:12:00.972 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:00.972 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:00.972 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:00.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.973 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89644 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:00.973 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.552384+0000", 2020-08-07T15:12:00.973 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 38.699862774000003, 2020-08-07T15:12:00.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68997479299999998, 2020-08-07T15:12:00.974 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:00.974 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:00.974 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:00.975 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:00.975 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:00.975 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89644 2020-08-07T15:12:00.982 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.983 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:00.983 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.983 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:00.983 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552384+0000", 2020-08-07T15:12:00.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:00.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.984 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:00.984 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552384+0000", 2020-08-07T15:12:00.985 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9238999999999999e-05 2020-08-07T15:12:00.985 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.985 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:00.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552413+0000", 2020-08-07T15:12:00.986 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3340000000000003e-06 2020-08-07T15:12:00.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.986 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:00.986 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552417+0000", 2020-08-07T15:12:00.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3119999999999994e-06 2020-08-07T15:12:00.987 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.987 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:00.987 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552426+0000", 2020-08-07T15:12:00.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4139999999999996e-06 2020-08-07T15:12:00.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:00.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552430+0000", 2020-08-07T15:12:00.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6613999999999999e-05 2020-08-07T15:12:00.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:00.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:12:00.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7280000000000007e-06 2020-08-07T15:12:00.990 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.990 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.990 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:00.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552477+0000", 2020-08-07T15:12:00.991 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67873990399999995 2020-08-07T15:12:00.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:00.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231216+0000", 2020-08-07T15:12:00.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3629999999999998e-05 2020-08-07T15:12:00.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:00.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231290+0000", 2020-08-07T15:12:00.993 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000132108 2020-08-07T15:12:00.993 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.993 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.993 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:00.994 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231422+0000", 2020-08-07T15:12:00.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006049652 2020-08-07T15:12:00.994 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.994 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.995 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:00.995 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237472+0000", 2020-08-07T15:12:00.995 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000005e-07 2020-08-07T15:12:00.995 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.995 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.996 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:00.996 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237472+0000", 2020-08-07T15:12:00.996 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2848000000000001e-05 2020-08-07T15:12:00.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.996 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.996 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:00.997 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237505+0000", 2020-08-07T15:12:00.997 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045352179999999997 2020-08-07T15:12:00.997 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.997 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.997 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:00.998 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242040+0000", 2020-08-07T15:12:00.998 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012843699999999999 2020-08-07T15:12:00.998 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.998 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.998 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:00.999 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242169+0000", 2020-08-07T15:12:00.999 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9999999999999998e-07 2020-08-07T15:12:00.999 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:00.999 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:00.999 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:01.000 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242169+0000", 2020-08-07T15:12:01.000 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000189354 2020-08-07T15:12:01.000 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.000 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.000 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:01.000 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242359+0000", 2020-08-07T15:12:01.001 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68997479299999998 2020-08-07T15:12:01.001 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:01.001 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:01.001 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:01.001 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.002 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.002 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89645 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:01.002 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:12:01.002 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 38.699779992000003, 2020-08-07T15:12:01.002 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69011194600000003, 2020-08-07T15:12:01.003 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:01.003 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:01.003 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:01.003 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:01.004 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:01.004 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89645 2020-08-07T15:12:01.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.004 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:01.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:01.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:12:01.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:01.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:01.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:12:01.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6750999999999998e-05 2020-08-07T15:12:01.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:01.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552493+0000", 2020-08-07T15:12:01.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0350000000000002e-06 2020-08-07T15:12:01.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:01.015 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552496+0000", 2020-08-07T15:12:01.015 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3029999999999996e-06 2020-08-07T15:12:01.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:01.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552503+0000", 2020-08-07T15:12:01.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7580000000000001e-06 2020-08-07T15:12:01.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:01.026 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552506+0000", 2020-08-07T15:12:01.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4868000000000003e-05 2020-08-07T15:12:01.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:01.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552570+0000", 2020-08-07T15:12:01.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9649999999999997e-06 2020-08-07T15:12:01.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:01.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552579+0000", 2020-08-07T15:12:01.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67928357699999997 2020-08-07T15:12:01.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:01.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231863+0000", 2020-08-07T15:12:01.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000115696 2020-08-07T15:12:01.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.067 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.067 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:01.067 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231979+0000", 2020-08-07T15:12:01.068 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020456599999999999 2020-08-07T15:12:01.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.068 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.068 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:01.068 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232183+0000", 2020-08-07T15:12:01.082 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005327923 2020-08-07T15:12:01.082 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.083 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:01.083 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237511+0000", 2020-08-07T15:12:01.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8999999999999998e-07 2020-08-07T15:12:01.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.083 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:01.084 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237511+0000", 2020-08-07T15:12:01.085 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6019000000000002e-05 2020-08-07T15:12:01.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.085 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:01.085 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237547+0000", 2020-08-07T15:12:01.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045340980000000003 2020-08-07T15:12:01.086 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.086 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.086 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:01.086 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242082+0000", 2020-08-07T15:12:01.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00032620499999999998 2020-08-07T15:12:01.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:01.087 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242408+0000", 2020-08-07T15:12:01.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3500000000000002e-07 2020-08-07T15:12:01.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:01.105 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242408+0000", 2020-08-07T15:12:01.105 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000170457 2020-08-07T15:12:01.105 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.106 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.106 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:01.106 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242579+0000", 2020-08-07T15:12:01.106 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69011194600000003 2020-08-07T15:12:01.106 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:01.107 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:01.107 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:01.107 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.107 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.107 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89646 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:01.108 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.557244+0000", 2020-08-07T15:12:01.108 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 38.695002219999999, 2020-08-07T15:12:01.108 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68559352299999998, 2020-08-07T15:12:01.108 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:01.108 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:01.109 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:01.109 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:01.109 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:01.109 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89646 2020-08-07T15:12:01.109 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.119 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:01.137 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.138 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:01.138 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557244+0000", 2020-08-07T15:12:01.138 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:01.139 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.139 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.139 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:01.140 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557244+0000", 2020-08-07T15:12:01.140 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9699000000000001e-05 2020-08-07T15:12:01.140 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.172 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.172 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:01.173 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557274+0000", 2020-08-07T15:12:01.173 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1269999999999996e-06 2020-08-07T15:12:01.174 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.174 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.174 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:01.175 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557278+0000", 2020-08-07T15:12:01.175 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0099999999999995e-06 2020-08-07T15:12:01.175 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.176 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.176 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:01.176 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557286+0000", 2020-08-07T15:12:01.182 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7439999999999998e-06 2020-08-07T15:12:01.199 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.199 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.199 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:01.200 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557291+0000", 2020-08-07T15:12:01.200 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5927999999999999e-05 2020-08-07T15:12:01.200 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.200 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.201 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:01.201 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557327+0000", 2020-08-07T15:12:01.201 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5069999999999997e-06 2020-08-07T15:12:01.201 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.201 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.218 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:01.219 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557333+0000", 2020-08-07T15:12:01.219 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67527419200000005 2020-08-07T15:12:01.219 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.227 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.227 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:01.227 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232608+0000", 2020-08-07T15:12:01.228 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8699e-05 2020-08-07T15:12:01.228 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.228 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.228 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:01.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232686+0000", 2020-08-07T15:12:01.229 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018185099999999999 2020-08-07T15:12:01.229 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.229 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.229 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:01.229 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232868+0000", 2020-08-07T15:12:01.230 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0075819360000000001 2020-08-07T15:12:01.230 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.230 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.230 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:01.256 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240450+0000", 2020-08-07T15:12:01.270 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2799999999999995e-07 2020-08-07T15:12:01.270 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.271 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:01.271 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240451+0000", 2020-08-07T15:12:01.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3751000000000002e-05 2020-08-07T15:12:01.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:01.273 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240495+0000", 2020-08-07T15:12:01.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001628064 2020-08-07T15:12:01.274 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.274 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.274 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:01.275 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242123+0000", 2020-08-07T15:12:01.275 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00050955599999999998 2020-08-07T15:12:01.275 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.276 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.276 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:01.276 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242632+0000", 2020-08-07T15:12:01.297 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2900000000000004e-07 2020-08-07T15:12:01.297 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.298 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.298 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:01.299 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242633+0000", 2020-08-07T15:12:01.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000205202 2020-08-07T15:12:01.300 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.300 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.301 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:01.301 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242838+0000", 2020-08-07T15:12:01.301 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68559352299999998 2020-08-07T15:12:01.332 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:01.338 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:01.339 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:01.339 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.339 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.340 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89647 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:01.340 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.557315+0000", 2020-08-07T15:12:01.350 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 38.694931896999996, 2020-08-07T15:12:01.350 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68581815800000001, 2020-08-07T15:12:01.351 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:01.351 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:01.351 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:01.352 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:01.352 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:01.352 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89647 2020-08-07T15:12:01.353 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.354 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:01.354 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.354 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:01.355 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557315+0000", 2020-08-07T15:12:01.355 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:01.355 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.356 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.356 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:01.356 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557315+0000", 2020-08-07T15:12:01.357 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6639999999999999e-05 2020-08-07T15:12:01.357 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.357 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.368 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:01.369 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557341+0000", 2020-08-07T15:12:01.369 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7989999999999998e-06 2020-08-07T15:12:01.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.370 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:01.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557344+0000", 2020-08-07T15:12:01.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6960000000000002e-06 2020-08-07T15:12:01.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.382 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.382 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:01.382 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557350+0000", 2020-08-07T15:12:01.382 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6089999999999999e-06 2020-08-07T15:12:01.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.390 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.391 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:01.391 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557352+0000", 2020-08-07T15:12:01.392 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1837e-05 2020-08-07T15:12:01.392 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.392 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.392 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:01.392 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557384+0000", 2020-08-07T15:12:01.393 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8099999999999998e-06 2020-08-07T15:12:01.393 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.393 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.393 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:01.394 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557392+0000", 2020-08-07T15:12:01.394 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67581117300000004 2020-08-07T15:12:01.395 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.395 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.395 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:01.410 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233203+0000", 2020-08-07T15:12:01.410 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112316 2020-08-07T15:12:01.410 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.410 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.411 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:01.411 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233316+0000", 2020-08-07T15:12:01.411 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000221766 2020-08-07T15:12:01.411 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.411 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.412 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:01.412 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233537+0000", 2020-08-07T15:12:01.412 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069695019999999998 2020-08-07T15:12:01.412 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.412 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.413 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:01.413 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240507+0000", 2020-08-07T15:12:01.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3999999999999997e-07 2020-08-07T15:12:01.427 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:01.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240507+0000", 2020-08-07T15:12:01.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8844000000000003e-05 2020-08-07T15:12:01.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.429 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:01.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240546+0000", 2020-08-07T15:12:01.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023473399999999998 2020-08-07T15:12:01.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.430 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.430 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:01.430 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242893+0000", 2020-08-07T15:12:01.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5410000000000001e-05 2020-08-07T15:12:01.431 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.432 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:01.432 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242939+0000", 2020-08-07T15:12:01.432 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8000000000000001e-07 2020-08-07T15:12:01.433 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.433 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.433 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:01.434 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242939+0000", 2020-08-07T15:12:01.434 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019369600000000001 2020-08-07T15:12:01.434 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.435 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.435 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:01.435 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243133+0000", 2020-08-07T15:12:01.436 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68581815800000001 2020-08-07T15:12:01.436 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:01.436 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:01.447 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:01.448 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.448 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89648 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:01.449 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.557372+0000", 2020-08-07T15:12:01.449 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 38.694874775999999, 2020-08-07T15:12:01.449 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68606040000000001, 2020-08-07T15:12:01.450 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:01.450 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:01.462 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:01.463 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:01.463 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:01.464 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89648 2020-08-07T15:12:01.464 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.464 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:01.465 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.465 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:01.465 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557372+0000", 2020-08-07T15:12:01.466 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:01.466 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.467 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.467 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:01.467 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557372+0000", 2020-08-07T15:12:01.468 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5534e-05 2020-08-07T15:12:01.468 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.468 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.469 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:01.469 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557397+0000", 2020-08-07T15:12:01.469 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.599e-06 2020-08-07T15:12:01.470 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.470 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.471 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:01.471 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557400+0000", 2020-08-07T15:12:01.471 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.288e-06 2020-08-07T15:12:01.472 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.472 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.472 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:01.473 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557405+0000", 2020-08-07T15:12:01.473 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.142e-06 2020-08-07T15:12:01.473 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.474 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.474 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:01.474 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557407+0000", 2020-08-07T15:12:01.475 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0954999999999998e-05 2020-08-07T15:12:01.475 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.475 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.475 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:01.475 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557428+0000", 2020-08-07T15:12:01.476 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7529999999999998e-06 2020-08-07T15:12:01.476 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.476 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.476 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:01.476 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557434+0000", 2020-08-07T15:12:01.477 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67655471099999998 2020-08-07T15:12:01.477 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.477 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.477 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:01.477 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233989+0000", 2020-08-07T15:12:01.477 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8301999999999997e-05 2020-08-07T15:12:01.478 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.478 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.478 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:01.478 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234067+0000", 2020-08-07T15:12:01.478 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000129908 2020-08-07T15:12:01.479 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.479 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.479 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:01.479 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234197+0000", 2020-08-07T15:12:01.479 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063551249999999997 2020-08-07T15:12:01.480 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.480 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:01.480 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240552+0000", 2020-08-07T15:12:01.480 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9400000000000001e-07 2020-08-07T15:12:01.481 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.481 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.481 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:01.481 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240552+0000", 2020-08-07T15:12:01.481 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2787999999999998e-05 2020-08-07T15:12:01.482 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.482 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.482 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:01.482 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240585+0000", 2020-08-07T15:12:01.482 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002605652 2020-08-07T15:12:01.483 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.483 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.483 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:01.483 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243191+0000", 2020-08-07T15:12:01.483 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8690000000000002e-05 2020-08-07T15:12:01.484 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.484 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.484 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:01.484 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243250+0000", 2020-08-07T15:12:01.485 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4700000000000001e-07 2020-08-07T15:12:01.485 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.485 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.486 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:01.486 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243250+0000", 2020-08-07T15:12:01.486 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000181912 2020-08-07T15:12:01.488 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:01.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243432+0000", 2020-08-07T15:12:01.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68606040000000001 2020-08-07T15:12:01.489 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:01.489 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:01.490 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:01.490 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.490 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.490 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89689 5.7 5:e83ee43b:::benchmark_data_smithi074_53697_object11211:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:01.490 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.585493+0000", 2020-08-07T15:12:01.491 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 38.666753958000001, 2020-08-07T15:12:01.491 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66549445299999999, 2020-08-07T15:12:01.491 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:01.491 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:01.491 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:01.492 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:01.492 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:01.492 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89689 2020-08-07T15:12:01.492 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.492 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:01.493 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:01.493 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585493+0000", 2020-08-07T15:12:01.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:01.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.494 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:01.494 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585493+0000", 2020-08-07T15:12:01.494 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7746000000000002e-05 2020-08-07T15:12:01.494 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.495 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:01.495 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585520+0000", 2020-08-07T15:12:01.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1099999999999996e-06 2020-08-07T15:12:01.495 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.495 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.496 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:01.496 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585525+0000", 2020-08-07T15:12:01.496 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5359999999999998e-06 2020-08-07T15:12:01.497 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.497 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.498 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:01.498 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585534+0000", 2020-08-07T15:12:01.498 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5040000000000002e-06 2020-08-07T15:12:01.502 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.502 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.503 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:01.503 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585540+0000", 2020-08-07T15:12:01.504 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3548999999999998e-05 2020-08-07T15:12:01.504 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.504 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.505 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:01.505 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585593+0000", 2020-08-07T15:12:01.506 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2014000000000001e-05 2020-08-07T15:12:01.506 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.515 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:01.542 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585605+0000", 2020-08-07T15:12:01.543 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.64901847700000004 2020-08-07T15:12:01.543 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.544 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.544 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:01.544 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234624+0000", 2020-08-07T15:12:01.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000176329 2020-08-07T15:12:01.545 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.545 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.562 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:01.562 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234800+0000", 2020-08-07T15:12:01.563 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00025457300000000002 2020-08-07T15:12:01.563 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.564 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:01.564 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.235055+0000", 2020-08-07T15:12:01.564 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0055364669999999998 2020-08-07T15:12:01.564 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.565 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.565 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:01.565 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240591+0000", 2020-08-07T15:12:01.565 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3700000000000001e-07 2020-08-07T15:12:01.565 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.566 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.566 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:01.566 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240591+0000", 2020-08-07T15:12:01.566 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8479999999999997e-05 2020-08-07T15:12:01.566 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.567 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.567 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:01.567 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240640+0000", 2020-08-07T15:12:01.567 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00285564 2020-08-07T15:12:01.590 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.590 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.590 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:01.591 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243496+0000", 2020-08-07T15:12:01.591 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0073477400000000002 2020-08-07T15:12:01.591 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.591 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.591 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:01.592 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.250843+0000", 2020-08-07T15:12:01.592 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7300000000000005e-07 2020-08-07T15:12:01.592 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.592 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.592 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:01.592 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.250844+0000", 2020-08-07T15:12:01.593 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014307799999999999 2020-08-07T15:12:01.593 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.593 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.593 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:01.603 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.250987+0000", 2020-08-07T15:12:01.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66549445299999999 2020-08-07T15:12:01.604 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:01.604 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:01.605 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:01.606 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.606 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.607 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103152 5.7 5:e865d79a:::benchmark_data_smithi074_53697_object12893:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e211)", 2020-08-07T15:12:01.607 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.399967+0000", 2020-08-07T15:12:01.607 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 28.852280068999999, 2020-08-07T15:12:01.608 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81348256399999996, 2020-08-07T15:12:01.608 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:01.608 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:01.608 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:01.608 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:01.609 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:01.609 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103152 2020-08-07T15:12:01.609 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.609 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:01.609 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.610 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:01.610 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.399967+0000", 2020-08-07T15:12:01.610 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:01.610 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.610 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.611 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:01.611 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.399967+0000", 2020-08-07T15:12:01.611 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017528899999999999 2020-08-07T15:12:01.611 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.611 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.611 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:01.612 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400142+0000", 2020-08-07T15:12:01.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8239999999999996e-06 2020-08-07T15:12:01.612 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.612 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.612 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:01.613 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400148+0000", 2020-08-07T15:12:01.613 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0251000000000001e-05 2020-08-07T15:12:01.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.613 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:01.643 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400158+0000", 2020-08-07T15:12:01.643 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1579999999999998e-06 2020-08-07T15:12:01.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.643 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.644 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:01.644 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400162+0000", 2020-08-07T15:12:01.644 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00078404699999999996 2020-08-07T15:12:01.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.644 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:01.645 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400946+0000", 2020-08-07T15:12:01.645 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010605800000000001 2020-08-07T15:12:01.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.646 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:01.646 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.401052+0000", 2020-08-07T15:12:01.646 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020987499999999999 2020-08-07T15:12:01.646 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.646 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.647 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:01.647 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.401262+0000", 2020-08-07T15:12:01.647 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023707820000000001 2020-08-07T15:12:01.648 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.649 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.649 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:01.649 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.403633+0000", 2020-08-07T15:12:01.649 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.023e-06 2020-08-07T15:12:01.649 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.650 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.650 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:01.650 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.403634+0000", 2020-08-07T15:12:01.650 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3560999999999995e-05 2020-08-07T15:12:01.651 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.651 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.651 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:01.651 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.403717+0000", 2020-08-07T15:12:01.684 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011669689999999999 2020-08-07T15:12:01.685 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.685 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.696 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:01.696 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.404884+0000", 2020-08-07T15:12:01.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033332850000000001 2020-08-07T15:12:01.697 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:01.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.408218+0000", 2020-08-07T15:12:01.697 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0268000000000001e-05 2020-08-07T15:12:01.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.698 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.710 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:01.710 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.408228+0000", 2020-08-07T15:12:01.710 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80508180500000004 2020-08-07T15:12:01.711 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.711 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:01.712 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213310+0000", 2020-08-07T15:12:01.712 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000139369 2020-08-07T15:12:01.712 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.713 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.713 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:01.713 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213449+0000", 2020-08-07T15:12:01.713 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81348256399999996 2020-08-07T15:12:01.714 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:01.714 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:01.714 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:01.715 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.715 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.715 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103265 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:12:01.716 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478593+0000", 2020-08-07T15:12:01.716 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 28.773654013000002, 2020-08-07T15:12:01.716 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74922190499999997, 2020-08-07T15:12:01.717 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:01.717 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:01.725 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:01.725 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:01.725 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:01.726 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103265 2020-08-07T15:12:01.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.732 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:01.732 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:01.732 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478593+0000", 2020-08-07T15:12:01.733 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:01.733 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.733 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:01.733 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478593+0000", 2020-08-07T15:12:01.734 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9968000000000001e-05 2020-08-07T15:12:01.734 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.734 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.734 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:01.735 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478633+0000", 2020-08-07T15:12:01.735 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1420000000000003e-06 2020-08-07T15:12:01.735 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.735 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.751 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:01.751 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478637+0000", 2020-08-07T15:12:01.751 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2919999999999996e-06 2020-08-07T15:12:01.752 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.752 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:01.752 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478645+0000", 2020-08-07T15:12:01.755 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3699999999999997e-06 2020-08-07T15:12:01.755 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.755 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:01.756 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478649+0000", 2020-08-07T15:12:01.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1440000000000002e-05 2020-08-07T15:12:01.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.757 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:01.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478701+0000", 2020-08-07T15:12:01.785 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0831e-05 2020-08-07T15:12:01.785 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:01.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478712+0000", 2020-08-07T15:12:01.786 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73476696699999999 2020-08-07T15:12:01.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.786 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:01.786 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213479+0000", 2020-08-07T15:12:01.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000167026 2020-08-07T15:12:01.787 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.787 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:01.787 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213646+0000", 2020-08-07T15:12:01.788 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000179557 2020-08-07T15:12:01.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.788 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:01.788 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213825+0000", 2020-08-07T15:12:01.789 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041300360000000001 2020-08-07T15:12:01.789 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.789 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.789 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:01.789 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217955+0000", 2020-08-07T15:12:01.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6840000000000001e-06 2020-08-07T15:12:01.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:01.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217957+0000", 2020-08-07T15:12:01.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4758e-05 2020-08-07T15:12:01.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.838 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:01.838 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218012+0000", 2020-08-07T15:12:01.838 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0095519990000000003 2020-08-07T15:12:01.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.839 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:01.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227564+0000", 2020-08-07T15:12:01.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9956e-05 2020-08-07T15:12:01.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.840 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.840 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:01.840 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227624+0000", 2020-08-07T15:12:01.840 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3790000000000001e-06 2020-08-07T15:12:01.840 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:01.841 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227625+0000", 2020-08-07T15:12:01.841 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0001895 2020-08-07T15:12:01.841 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.842 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.842 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:01.842 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227815+0000", 2020-08-07T15:12:01.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74922190499999997 2020-08-07T15:12:01.843 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:01.843 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:01.843 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:01.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.844 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103266 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:12:01.845 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478693+0000", 2020-08-07T15:12:01.845 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 28.773553671999998, 2020-08-07T15:12:01.845 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74942137399999997, 2020-08-07T15:12:01.845 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:01.855 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:01.856 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:01.856 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:01.856 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:01.856 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103266 2020-08-07T15:12:01.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.857 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:01.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:01.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478693+0000", 2020-08-07T15:12:01.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:01.858 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.858 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:01.858 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478693+0000", 2020-08-07T15:12:01.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5658000000000002e-05 2020-08-07T15:12:01.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.859 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.859 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:01.893 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478739+0000", 2020-08-07T15:12:01.893 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.113e-06 2020-08-07T15:12:01.894 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.894 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.894 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:01.894 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478744+0000", 2020-08-07T15:12:01.895 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9690000000000006e-06 2020-08-07T15:12:01.895 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.895 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.895 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:01.895 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478754+0000", 2020-08-07T15:12:01.896 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1849999999999997e-06 2020-08-07T15:12:01.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.896 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:01.896 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478758+0000", 2020-08-07T15:12:01.897 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7024999999999998e-05 2020-08-07T15:12:01.897 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.897 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.897 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:01.897 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478805+0000", 2020-08-07T15:12:01.902 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1357e-05 2020-08-07T15:12:01.902 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.902 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.902 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:01.903 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478816+0000", 2020-08-07T15:12:01.903 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73554277499999998 2020-08-07T15:12:01.903 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.903 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:01.904 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.214359+0000", 2020-08-07T15:12:01.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000129145 2020-08-07T15:12:01.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.904 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:01.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.214488+0000", 2020-08-07T15:12:01.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00027129999999999998 2020-08-07T15:12:01.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:01.906 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.214760+0000", 2020-08-07T15:12:01.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032631079999999998 2020-08-07T15:12:01.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:01.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218023+0000", 2020-08-07T15:12:01.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1000000000000001e-06 2020-08-07T15:12:01.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.908 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:01.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218024+0000", 2020-08-07T15:12:01.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9085000000000006e-05 2020-08-07T15:12:01.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:01.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218103+0000", 2020-08-07T15:12:01.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.009779088 2020-08-07T15:12:01.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.910 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:01.910 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227882+0000", 2020-08-07T15:12:01.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1989999999999994e-05 2020-08-07T15:12:01.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:01.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227944+0000", 2020-08-07T15:12:01.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8299999999999996e-07 2020-08-07T15:12:01.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:01.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227945+0000", 2020-08-07T15:12:01.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000169693 2020-08-07T15:12:01.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:01.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228114+0000", 2020-08-07T15:12:01.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74942137399999997 2020-08-07T15:12:01.917 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:01.918 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:01.918 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:01.918 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.918 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103267 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:12:01.919 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478794+0000", 2020-08-07T15:12:01.919 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 28.773452184, 2020-08-07T15:12:01.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74960417300000004, 2020-08-07T15:12:01.919 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:01.919 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:01.920 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:01.920 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:01.920 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:01.920 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103267 2020-08-07T15:12:01.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.921 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:01.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:01.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478794+0000", 2020-08-07T15:12:01.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:01.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.922 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:01.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478794+0000", 2020-08-07T15:12:01.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6646e-05 2020-08-07T15:12:01.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:01.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478841+0000", 2020-08-07T15:12:01.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:12:01.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:01.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478846+0000", 2020-08-07T15:12:01.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0171e-05 2020-08-07T15:12:01.925 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:01.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478856+0000", 2020-08-07T15:12:01.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5929999999999999e-06 2020-08-07T15:12:01.926 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.926 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:01.926 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478861+0000", 2020-08-07T15:12:01.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2032e-05 2020-08-07T15:12:01.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:01.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478903+0000", 2020-08-07T15:12:01.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0312000000000001e-05 2020-08-07T15:12:01.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:01.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478913+0000", 2020-08-07T15:12:01.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73634238600000002 2020-08-07T15:12:01.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:01.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.215256+0000", 2020-08-07T15:12:01.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000124933 2020-08-07T15:12:01.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:01.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.215380+0000", 2020-08-07T15:12:01.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024624 2020-08-07T15:12:01.937 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.938 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:01.938 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.215627+0000", 2020-08-07T15:12:01.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0040879540000000004 2020-08-07T15:12:01.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:01.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219715+0000", 2020-08-07T15:12:01.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9230000000000001e-06 2020-08-07T15:12:01.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:01.940 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219717+0000", 2020-08-07T15:12:01.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8881999999999995e-05 2020-08-07T15:12:01.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.940 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.940 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:01.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219785+0000", 2020-08-07T15:12:01.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0083925119999999995 2020-08-07T15:12:01.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:01.942 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228178+0000", 2020-08-07T15:12:01.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2254000000000004e-05 2020-08-07T15:12:01.942 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.942 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:01.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228240+0000", 2020-08-07T15:12:01.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4499999999999996e-07 2020-08-07T15:12:01.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:01.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228241+0000", 2020-08-07T15:12:01.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000157656 2020-08-07T15:12:01.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:01.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228399+0000", 2020-08-07T15:12:01.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74960417300000004 2020-08-07T15:12:01.945 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:01.946 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:01.946 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:01.946 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.946 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.946 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103268 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:12:01.947 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478898+0000", 2020-08-07T15:12:01.947 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 28.773348467000002, 2020-08-07T15:12:01.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74981275700000005, 2020-08-07T15:12:01.947 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:01.947 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:01.948 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:01.948 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:01.948 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:01.948 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103268 2020-08-07T15:12:01.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.949 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:01.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.949 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:01.949 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478898+0000", 2020-08-07T15:12:01.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:01.950 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.950 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.950 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:01.950 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478898+0000", 2020-08-07T15:12:01.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2558999999999997e-05 2020-08-07T15:12:01.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:01.952 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478941+0000", 2020-08-07T15:12:01.952 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6469999999999997e-06 2020-08-07T15:12:01.952 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.952 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.952 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:01.953 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478945+0000", 2020-08-07T15:12:01.953 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3109999999999995e-06 2020-08-07T15:12:01.953 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.953 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.953 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:01.954 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478955+0000", 2020-08-07T15:12:01.954 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.799e-06 2020-08-07T15:12:01.954 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.954 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.955 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:01.955 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478958+0000", 2020-08-07T15:12:01.955 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5682000000000002e-05 2020-08-07T15:12:01.956 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.956 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.956 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:01.956 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:12:01.956 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3260000000000002e-06 2020-08-07T15:12:01.957 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.957 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:01.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479004+0000", 2020-08-07T15:12:01.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73703730499999998 2020-08-07T15:12:01.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:01.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216041+0000", 2020-08-07T15:12:01.958 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4445999999999998e-05 2020-08-07T15:12:01.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:01.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216115+0000", 2020-08-07T15:12:01.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013977 2020-08-07T15:12:01.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:01.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216255+0000", 2020-08-07T15:12:01.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0051267359999999998 2020-08-07T15:12:01.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:01.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221382+0000", 2020-08-07T15:12:01.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0310000000000001e-06 2020-08-07T15:12:01.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.962 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:01.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221384+0000", 2020-08-07T15:12:01.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8097999999999998e-05 2020-08-07T15:12:01.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:01.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221452+0000", 2020-08-07T15:12:01.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0070259249999999997 2020-08-07T15:12:01.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:01.964 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228478+0000", 2020-08-07T15:12:01.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7479999999999999e-05 2020-08-07T15:12:01.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:01.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228535+0000", 2020-08-07T15:12:01.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4399999999999999e-07 2020-08-07T15:12:01.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:01.966 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228536+0000", 2020-08-07T15:12:01.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017489799999999999 2020-08-07T15:12:01.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:01.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228711+0000", 2020-08-07T15:12:01.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74981275700000005 2020-08-07T15:12:01.968 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:01.968 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:01.968 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:01.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.968 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103269 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:12:01.969 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:12:01.969 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 28.773252359000001, 2020-08-07T15:12:01.969 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74992452799999998, 2020-08-07T15:12:01.969 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:01.969 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:01.970 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:01.970 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:01.970 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:01.970 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103269 2020-08-07T15:12:01.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.971 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:01.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:01.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:12:01.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:01.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:01.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:12:01.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5529999999999999e-05 2020-08-07T15:12:01.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:01.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479040+0000", 2020-08-07T15:12:01.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6550000000000003e-06 2020-08-07T15:12:01.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:01.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479044+0000", 2020-08-07T15:12:01.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4285e-05 2020-08-07T15:12:01.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.975 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:01.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479059+0000", 2020-08-07T15:12:01.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2050000000000001e-06 2020-08-07T15:12:01.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:01.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479064+0000", 2020-08-07T15:12:01.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0952999999999999e-05 2020-08-07T15:12:01.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.978 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:01.978 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479105+0000", 2020-08-07T15:12:01.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3559999999999998e-06 2020-08-07T15:12:01.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.979 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:01.979 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479111+0000", 2020-08-07T15:12:01.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.737508104 2020-08-07T15:12:01.979 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.979 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:01.980 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216619+0000", 2020-08-07T15:12:01.980 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000126367 2020-08-07T15:12:01.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.981 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:01.981 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216746+0000", 2020-08-07T15:12:01.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014069899999999999 2020-08-07T15:12:01.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.982 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:01.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216886+0000", 2020-08-07T15:12:01.982 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045753160000000003 2020-08-07T15:12:01.982 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.983 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.983 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:01.983 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221462+0000", 2020-08-07T15:12:01.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.142e-06 2020-08-07T15:12:01.983 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.984 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:01.984 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221463+0000", 2020-08-07T15:12:01.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7718e-05 2020-08-07T15:12:01.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.985 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:01.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221501+0000", 2020-08-07T15:12:01.985 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0072733210000000001 2020-08-07T15:12:01.985 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.986 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:01.986 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228774+0000", 2020-08-07T15:12:01.986 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3828e-05 2020-08-07T15:12:01.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.987 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:01.987 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228828+0000", 2020-08-07T15:12:01.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9800000000000003e-07 2020-08-07T15:12:01.987 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:01.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228828+0000", 2020-08-07T15:12:01.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0451000000000005e-05 2020-08-07T15:12:01.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:01.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:01.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:01.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228919+0000", 2020-08-07T15:12:01.999 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74992452799999998 2020-08-07T15:12:01.999 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:01.999 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:02.000 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:02.000 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.000 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.000 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103270 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:12:02.000 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.479168+0000", 2020-08-07T15:12:02.001 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 28.773078322, 2020-08-07T15:12:02.001 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75043371699999994, 2020-08-07T15:12:02.001 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:02.001 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:02.001 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:02.002 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:02.002 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:02.002 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103270 2020-08-07T15:12:02.002 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.003 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:02.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:02.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479168+0000", 2020-08-07T15:12:02.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:02.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:02.004 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479168+0000", 2020-08-07T15:12:02.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1557000000000002e-05 2020-08-07T15:12:02.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:02.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479210+0000", 2020-08-07T15:12:02.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9840000000000001e-06 2020-08-07T15:12:02.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:02.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479215+0000", 2020-08-07T15:12:02.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9539999999999999e-06 2020-08-07T15:12:02.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:02.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479225+0000", 2020-08-07T15:12:02.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.942e-06 2020-08-07T15:12:02.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:02.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479231+0000", 2020-08-07T15:12:02.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2493000000000002e-05 2020-08-07T15:12:02.009 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.009 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.010 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:02.010 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479263+0000", 2020-08-07T15:12:02.010 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6640000000000003e-06 2020-08-07T15:12:02.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.011 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:02.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479269+0000", 2020-08-07T15:12:02.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73802486499999997 2020-08-07T15:12:02.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.013 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:02.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217294+0000", 2020-08-07T15:12:02.013 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8640000000000006e-05 2020-08-07T15:12:02.014 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.014 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:02.014 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217372+0000", 2020-08-07T15:12:02.014 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000140338 2020-08-07T15:12:02.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.015 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.015 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:02.015 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217513+0000", 2020-08-07T15:12:02.015 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039952119999999997 2020-08-07T15:12:02.016 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.016 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:02.016 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221508+0000", 2020-08-07T15:12:02.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8899999999999999e-07 2020-08-07T15:12:02.017 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:02.017 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221509+0000", 2020-08-07T15:12:02.017 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8108999999999997e-05 2020-08-07T15:12:02.018 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.018 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.018 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:02.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221547+0000", 2020-08-07T15:12:02.019 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074232029999999997 2020-08-07T15:12:02.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.020 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.020 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:02.020 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228970+0000", 2020-08-07T15:12:02.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00046710099999999999 2020-08-07T15:12:02.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:02.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229437+0000", 2020-08-07T15:12:02.022 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5400000000000001e-07 2020-08-07T15:12:02.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:02.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229438+0000", 2020-08-07T15:12:02.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016441200000000001 2020-08-07T15:12:02.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:02.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229602+0000", 2020-08-07T15:12:02.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75043371699999994 2020-08-07T15:12:02.028 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:02.028 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:02.028 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:02.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.028 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103271 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:12:02.029 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.479356+0000", 2020-08-07T15:12:02.029 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 28.772890354000001, 2020-08-07T15:12:02.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75047653800000003, 2020-08-07T15:12:02.029 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:02.030 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:02.030 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:02.030 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:02.043 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:02.044 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103271 2020-08-07T15:12:02.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.075 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:02.076 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.076 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:02.091 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479356+0000", 2020-08-07T15:12:02.091 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:02.092 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.092 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.092 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:02.092 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479356+0000", 2020-08-07T15:12:02.093 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2416000000000002e-05 2020-08-07T15:12:02.093 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.093 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.093 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:02.110 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479399+0000", 2020-08-07T15:12:02.110 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0819999999999998e-06 2020-08-07T15:12:02.111 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.113 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:02.114 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479404+0000", 2020-08-07T15:12:02.114 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2486999999999999e-05 2020-08-07T15:12:02.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.115 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:02.115 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479416+0000", 2020-08-07T15:12:02.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3810000000000001e-06 2020-08-07T15:12:02.115 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.115 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.116 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:02.116 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479421+0000", 2020-08-07T15:12:02.131 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2311999999999998e-05 2020-08-07T15:12:02.132 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.132 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.132 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:02.133 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479453+0000", 2020-08-07T15:12:02.133 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7690000000000002e-06 2020-08-07T15:12:02.143 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.143 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.143 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:02.144 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479459+0000", 2020-08-07T15:12:02.144 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73869566499999995 2020-08-07T15:12:02.144 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.145 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.145 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:02.145 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218154+0000", 2020-08-07T15:12:02.145 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000124535 2020-08-07T15:12:02.160 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.160 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.161 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:02.161 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218279+0000", 2020-08-07T15:12:02.161 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000223441 2020-08-07T15:12:02.161 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.161 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.162 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:02.162 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218502+0000", 2020-08-07T15:12:02.162 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003054761 2020-08-07T15:12:02.162 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.162 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.163 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:02.163 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221557+0000", 2020-08-07T15:12:02.163 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0449999999999999e-06 2020-08-07T15:12:02.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.163 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:02.164 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221558+0000", 2020-08-07T15:12:02.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3655e-05 2020-08-07T15:12:02.164 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.165 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:02.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221622+0000", 2020-08-07T15:12:02.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.007661434 2020-08-07T15:12:02.181 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:02.182 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229283+0000", 2020-08-07T15:12:02.183 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00036023099999999997 2020-08-07T15:12:02.183 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.183 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.184 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:02.184 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229644+0000", 2020-08-07T15:12:02.184 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.63e-07 2020-08-07T15:12:02.185 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.185 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.185 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:02.196 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229644+0000", 2020-08-07T15:12:02.196 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018886099999999999 2020-08-07T15:12:02.197 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.197 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.197 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:02.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229833+0000", 2020-08-07T15:12:02.210 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75047653800000003 2020-08-07T15:12:02.211 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:02.211 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:02.212 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:02.212 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.212 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.213 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103272 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:12:02.214 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.481658+0000", 2020-08-07T15:12:02.215 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 28.770588930999999, 2020-08-07T15:12:02.215 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74920410900000001, 2020-08-07T15:12:02.215 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:02.216 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:02.216 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:02.216 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:02.217 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:02.217 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103272 2020-08-07T15:12:02.217 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.234 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:02.234 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.235 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:02.235 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481658+0000", 2020-08-07T15:12:02.235 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:02.236 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.236 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.236 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:02.237 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481658+0000", 2020-08-07T15:12:02.237 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8787e-05 2020-08-07T15:12:02.237 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.238 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.238 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:02.238 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481697+0000", 2020-08-07T15:12:02.239 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0559999999999998e-06 2020-08-07T15:12:02.239 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.239 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.240 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:02.240 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481701+0000", 2020-08-07T15:12:02.240 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1329999999999999e-06 2020-08-07T15:12:02.241 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.241 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.253 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:02.254 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481709+0000", 2020-08-07T15:12:02.262 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.189e-06 2020-08-07T15:12:02.263 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.263 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.263 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:02.264 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481713+0000", 2020-08-07T15:12:02.264 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6025999999999997e-05 2020-08-07T15:12:02.264 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.265 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.265 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:02.265 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481749+0000", 2020-08-07T15:12:02.271 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5619999999999998e-06 2020-08-07T15:12:02.276 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.277 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.290 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:02.290 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481756+0000", 2020-08-07T15:12:02.291 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73729155000000002 2020-08-07T15:12:02.291 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.291 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.292 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:02.292 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219048+0000", 2020-08-07T15:12:02.292 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000120387 2020-08-07T15:12:02.293 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.293 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.293 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:02.294 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219168+0000", 2020-08-07T15:12:02.294 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017865599999999999 2020-08-07T15:12:02.295 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.295 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.295 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:02.296 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219347+0000", 2020-08-07T15:12:02.296 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039049169999999999 2020-08-07T15:12:02.334 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.335 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:02.335 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.223252+0000", 2020-08-07T15:12:02.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.412e-06 2020-08-07T15:12:02.336 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.336 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.337 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:02.337 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.223253+0000", 2020-08-07T15:12:02.337 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7521999999999997e-05 2020-08-07T15:12:02.338 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.338 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.338 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:02.339 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.223301+0000", 2020-08-07T15:12:02.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00603478 2020-08-07T15:12:02.339 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.340 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:02.341 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229336+0000", 2020-08-07T15:12:02.341 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013309579999999999 2020-08-07T15:12:02.341 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.350 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.350 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:02.350 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.230667+0000", 2020-08-07T15:12:02.351 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7400000000000003e-07 2020-08-07T15:12:02.351 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.351 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.351 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:02.351 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.230667+0000", 2020-08-07T15:12:02.352 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019450000000000001 2020-08-07T15:12:02.352 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:02.352 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:02.352 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:02.352 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.230862+0000", 2020-08-07T15:12:02.353 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74920410900000001 2020-08-07T15:12:02.353 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:02.353 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:02.353 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:02.353 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:02.355 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:02.355 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:02.356 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:12:02.357 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:12:02.379 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Removed 15865 objects 2020-08-07T15:12:02.380 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Clean up completed and total clean up time :9.01231 2020-08-07T15:12:02.380 INFO:teuthology.orchestra.run.smithi074.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2020-08-07T15:12:02.380 DEBUG:teuthology.orchestra.run:got remote process result: 22 2020-08-07T15:12:02.381 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2020-08-07T15:12:02.381 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:01.350+0000 7f9369d2aec0 -1 Falling back to public interface 2020-08-07T15:12:02.381 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:02.356+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:02.382 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:12:02.466 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:02.456+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:02.563 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:02.560+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:02.563 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:02.564 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:12:02.564 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:12:02.564 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:02.572 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:12:02.586 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:12:02.610 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:12:02.637 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:12:02.666 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:12:02.667 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:02.665+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:02.698 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:12:02.714 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:12:02.718 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:02.719 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:02.728 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:12:02.739 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:12:02.740 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:12:02.740 INFO:teuthology.orchestra.run.smithi074.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-08-07T15:12:02.740 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: problem getting command descriptions from osd.1 2020-08-07T15:12:02.740 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:12:02.741 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:12:02.741 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:12:02.741 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:12:02.741 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:12:02.741 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:12:02.751 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-08-07T15:12:02.752 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-08-07T15:12:02.777 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:02.773+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:02.878 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:02.876+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:02.897 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:02.897 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:02.916 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:12:02.988 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:02.986+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:03.053 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:03.054 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:03.067 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:12:03.125 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:03.108+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:03.203 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:03.195+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:03.250 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:03.250 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:03.259 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:12:03.305 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:03.304+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:03.351 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:12:03.397 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:03.403 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:03.407 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:03.406+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:03.409 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:12:03.534 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:03.522+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:03.560 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:03.560 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:03.570 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:12:03.625 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:03.624+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:03.727 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:03.726+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:03.728 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:03.728 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:03.747 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:12:03.828 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:03.827+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:03.871 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:03.872 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:03.885 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:12:03.958 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:03.953+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:04.023 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:04.024 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:04.032 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:12:04.043 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:04.038+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:04.146 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:04.138+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:04.191 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:04.191 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:04.197 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:12:04.243 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:04.242+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:04.344 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:04.343+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:04.366 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:04.367 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:04.368 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:12:04.450 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:04.446+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:04.488 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:04.489 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:04.496 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:12:04.548 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:04.547+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:04.614 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:04.614 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:04.623 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:12:04.672 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:04.670+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:04.743 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:04.743 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:04.752 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:04.751+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:04.752 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:12:04.761 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:12:04.837 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:04.836+0000 7f9369d2aec0 -1 osd.1 190 log_to_monitors {default=true} 2020-08-07T15:12:04.855 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:04.854+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:04.918 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:04.918 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:04.933 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:12:04.957 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:12:04.958 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:12:04.959 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:04.958+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:04.996 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:12:05.007 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:12:05.034 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:12:05.058 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:12:05.069 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:05.064+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:05.082 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:12:05.083 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:12:05.083 INFO:teuthology.orchestra.run.smithi074.stderr:Error ENXIO: problem getting command descriptions from osd.1 2020-08-07T15:12:05.083 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: problem getting command descriptions from osd.1 2020-08-07T15:12:05.083 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:12:05.084 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:12:05.084 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:12:05.084 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:12:05.084 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:12:05.107 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:12:05.108 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:05.108 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:05.110 DEBUG:teuthology.orchestra.run:got remote process result: 6 2020-08-07T15:12:05.110 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2020-08-07T15:12:05.111 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:12:05.210 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:05.202+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:05.283 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:05.266+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:05.284 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:05.284 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:12:05.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:12:05.284 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:12:05.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.285 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15576 3.e 3:727724a5:::benchmark_data_smithi074_35447_object1946:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:12:05.285 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:12:05.285 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 318.60669806300001, 2020-08-07T15:12:05.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999, 2020-08-07T15:12:05.286 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:05.286 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:05.286 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:05.286 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:12:05.286 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:12:05.287 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15576 2020-08-07T15:12:05.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.287 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:05.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:05.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:12:05.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:05.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:05.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:12:05.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7156e-05 2020-08-07T15:12:05.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:05.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666682+0000", 2020-08-07T15:12:05.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0209999999999999e-06 2020-08-07T15:12:05.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:05.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666687+0000", 2020-08-07T15:12:05.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0730999999999999e-05 2020-08-07T15:12:05.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:05.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666698+0000", 2020-08-07T15:12:05.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2020-08-07T15:12:05.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:05.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666702+0000", 2020-08-07T15:12:05.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023978459999999999 2020-08-07T15:12:05.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:05.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669100+0000", 2020-08-07T15:12:05.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9664000000000001e-05 2020-08-07T15:12:05.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:05.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669190+0000", 2020-08-07T15:12:05.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015957100000000001 2020-08-07T15:12:05.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:12:05.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669350+0000", 2020-08-07T15:12:05.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016470149999999999 2020-08-07T15:12:05.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:05.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:12:05.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1299999999999999e-07 2020-08-07T15:12:05.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:05.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:12:05.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5269000000000002e-05 2020-08-07T15:12:05.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:05.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.671022+0000", 2020-08-07T15:12:05.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031414400000000001 2020-08-07T15:12:05.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:05.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.674163+0000", 2020-08-07T15:12:05.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45641417400000001 2020-08-07T15:12:05.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:05.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:12:05.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0399999999999995e-07 2020-08-07T15:12:05.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:05.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:12:05.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3343999999999997e-05 2020-08-07T15:12:05.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:05.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130622+0000", 2020-08-07T15:12:05.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999 2020-08-07T15:12:05.430 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:05.430 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:05.431 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:05.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.431 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106248 3.f 3:f950fd9d:::benchmark_data_smithi074_35447_object13280:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:12:05.431 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:12:05.432 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 257.91316852300002, 2020-08-07T15:12:05.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002, 2020-08-07T15:12:05.432 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:05.432 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:05.433 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:05.433 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:12:05.433 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:12:05.433 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106248 2020-08-07T15:12:05.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.435 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:05.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:05.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:12:05.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:05.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:05.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:12:05.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7184e-05 2020-08-07T15:12:05.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:05.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360222+0000", 2020-08-07T15:12:05.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0906e-05 2020-08-07T15:12:05.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:05.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360233+0000", 2020-08-07T15:12:05.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2972999999999999e-05 2020-08-07T15:12:05.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:05.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360246+0000", 2020-08-07T15:12:05.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0114e-05 2020-08-07T15:12:05.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:05.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360256+0000", 2020-08-07T15:12:05.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1303999999999997e-05 2020-08-07T15:12:05.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:05.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360287+0000", 2020-08-07T15:12:05.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3798000000000001e-05 2020-08-07T15:12:05.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:05.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360351+0000", 2020-08-07T15:12:05.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111308 2020-08-07T15:12:05.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:12:05.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360462+0000", 2020-08-07T15:12:05.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003480674 2020-08-07T15:12:05.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:05.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:12:05.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0900000000000001e-07 2020-08-07T15:12:05.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:05.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:12:05.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8899999999999994e-05 2020-08-07T15:12:05.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:05.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.364012+0000", 2020-08-07T15:12:05.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35291637199999998 2020-08-07T15:12:05.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:05.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716929+0000", 2020-08-07T15:12:05.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.076244233999999994 2020-08-07T15:12:05.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:05.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793173+0000", 2020-08-07T15:12:05.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6109999999999999e-06 2020-08-07T15:12:05.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:05.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793175+0000", 2020-08-07T15:12:05.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7492999999999999e-05 2020-08-07T15:12:05.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:05.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793242+0000", 2020-08-07T15:12:05.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002 2020-08-07T15:12:05.541 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:05.541 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:05.541 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:05.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.542 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106248 3.f e59/20 ondisk, result = 0)", 2020-08-07T15:12:05.542 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:12:05.542 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 257.90790523800001, 2020-08-07T15:12:05.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001, 2020-08-07T15:12:05.543 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:05.543 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:05.543 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:05.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:05.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:12:05.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:05.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:05.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:12:05.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7518999999999999e-05 2020-08-07T15:12:05.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:05.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365445+0000", 2020-08-07T15:12:05.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-06 2020-08-07T15:12:05.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:05.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365449+0000", 2020-08-07T15:12:05.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42764147600000002 2020-08-07T15:12:05.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:05.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793091+0000", 2020-08-07T15:12:05.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6500000000000002e-06 2020-08-07T15:12:05.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:05.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793099+0000", 2020-08-07T15:12:05.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3664e-05 2020-08-07T15:12:05.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:05.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793153+0000", 2020-08-07T15:12:05.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1078e-05 2020-08-07T15:12:05.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:05.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793164+0000", 2020-08-07T15:12:05.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3404000000000003e-05 2020-08-07T15:12:05.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:05.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793248+0000", 2020-08-07T15:12:05.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001 2020-08-07T15:12:05.563 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:05.566 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:05.567 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:05.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.567 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:107952 3.d e61/35)", 2020-08-07T15:12:05.568 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:12:05.568 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 254.94697553, 2020-08-07T15:12:05.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001, 2020-08-07T15:12:05.569 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:05.570 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:05.570 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:05.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:05.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:12:05.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:05.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:05.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:12:05.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8826000000000003e-05 2020-08-07T15:12:05.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:05.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326406+0000", 2020-08-07T15:12:05.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9199999999999998e-06 2020-08-07T15:12:05.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:05.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326413+0000", 2020-08-07T15:12:05.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75082853500000002 2020-08-07T15:12:05.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:05.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077242+0000", 2020-08-07T15:12:05.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1479999999999999e-06 2020-08-07T15:12:05.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:05.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077246+0000", 2020-08-07T15:12:05.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3331999999999998e-05 2020-08-07T15:12:05.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:05.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077289+0000", 2020-08-07T15:12:05.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8977000000000001e-05 2020-08-07T15:12:05.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:05.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077308+0000", 2020-08-07T15:12:05.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001 2020-08-07T15:12:05.581 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:05.582 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:05.582 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:05.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.582 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31696 4.0 4:0335eed9:::benchmark_data_smithi074_44586_object3961:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e100)", 2020-08-07T15:12:05.586 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:12:05.587 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 197.169067971, 2020-08-07T15:12:05.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237, 2020-08-07T15:12:05.590 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:05.590 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:05.590 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:05.590 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:05.591 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:05.591 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31696 2020-08-07T15:12:05.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.592 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:05.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:05.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:12:05.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:05.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:05.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:12:05.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7654e-05 2020-08-07T15:12:05.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104293+0000", 2020-08-07T15:12:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.061e-06 2020-08-07T15:12:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104296+0000", 2020-08-07T15:12:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0859999999999998e-06 2020-08-07T15:12:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104302+0000", 2020-08-07T15:12:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8030000000000001e-06 2020-08-07T15:12:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104305+0000", 2020-08-07T15:12:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037634769999999999 2020-08-07T15:12:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108068+0000", 2020-08-07T15:12:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8814999999999998e-05 2020-08-07T15:12:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108137+0000", 2020-08-07T15:12:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111455 2020-08-07T15:12:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:12:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108248+0000", 2020-08-07T15:12:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028756820000000001 2020-08-07T15:12:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111124+0000", 2020-08-07T15:12:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061054499999999999 2020-08-07T15:12:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111735+0000", 2020-08-07T15:12:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.423966167 2020-08-07T15:12:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535701+0000", 2020-08-07T15:12:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4929999999999999e-06 2020-08-07T15:12:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535702+0000", 2020-08-07T15:12:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2571999999999997e-05 2020-08-07T15:12:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535755+0000", 2020-08-07T15:12:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1427000000000003e-05 2020-08-07T15:12:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535786+0000", 2020-08-07T15:12:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237 2020-08-07T15:12:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:31696 4.0 e100/81 ondisk, result = 0)", 2020-08-07T15:12:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:12:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 197.15945341299999, 2020-08-07T15:12:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954, 2020-08-07T15:12:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:12:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:12:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1185000000000001e-05 2020-08-07T15:12:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113901+0000", 2020-08-07T15:12:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2930000000000002e-06 2020-08-07T15:12:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113905+0000", 2020-08-07T15:12:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42168640400000001 2020-08-07T15:12:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535591+0000", 2020-08-07T15:12:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2820000000000002e-06 2020-08-07T15:12:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535598+0000", 2020-08-07T15:12:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6578999999999999e-05 2020-08-07T15:12:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535684+0000", 2020-08-07T15:12:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8449999999999992e-06 2020-08-07T15:12:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535693+0000", 2020-08-07T15:12:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7366000000000003e-05 2020-08-07T15:12:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535791+0000", 2020-08-07T15:12:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954 2020-08-07T15:12:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:37024 4.0 e106/81)", 2020-08-07T15:12:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:12:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 191.04466451900001, 2020-08-07T15:12:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999, 2020-08-07T15:12:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:12:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:12:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4926000000000001e-05 2020-08-07T15:12:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228683+0000", 2020-08-07T15:12:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1280000000000002e-06 2020-08-07T15:12:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228686+0000", 2020-08-07T15:12:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69740012100000004 2020-08-07T15:12:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926086+0000", 2020-08-07T15:12:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000004e-06 2020-08-07T15:12:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926090+0000", 2020-08-07T15:12:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113526 2020-08-07T15:12:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926204+0000", 2020-08-07T15:12:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8824000000000001e-05 2020-08-07T15:12:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926222+0000", 2020-08-07T15:12:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999 2020-08-07T15:12:05.707 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:05.707 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:05.708 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:05.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.708 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42503 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2615, mlcod=112'2607)", 2020-08-07T15:12:05.709 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:12:05.709 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 184.97709260299999, 2020-08-07T15:12:05.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998, 2020-08-07T15:12:05.709 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:05.710 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:05.710 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:05.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:05.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:12:05.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:05.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:05.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:12:05.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270999999999998e-05 2020-08-07T15:12:05.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:05.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296292+0000", 2020-08-07T15:12:05.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4729999999999997e-06 2020-08-07T15:12:05.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:05.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296298+0000", 2020-08-07T15:12:05.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.448795049 2020-08-07T15:12:05.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:05.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745093+0000", 2020-08-07T15:12:05.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0140000000000001e-06 2020-08-07T15:12:05.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:05.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745100+0000", 2020-08-07T15:12:05.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2596999999999998e-05 2020-08-07T15:12:05.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:05.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745153+0000", 2020-08-07T15:12:05.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1662000000000003e-05 2020-08-07T15:12:05.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:05.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745184+0000", 2020-08-07T15:12:05.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093174800000000004 2020-08-07T15:12:05.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:05.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746116+0000", 2020-08-07T15:12:05.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0926999999999994e-05 2020-08-07T15:12:05.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:05.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746187+0000", 2020-08-07T15:12:05.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998 2020-08-07T15:12:05.745 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:05.745 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:05.745 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:05.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.746 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42504 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2616, mlcod=113'2609)", 2020-08-07T15:12:05.746 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:12:05.746 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 184.97375571000001, 2020-08-07T15:12:05.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998, 2020-08-07T15:12:05.747 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:05.747 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:05.747 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:05.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:05.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:12:05.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:05.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:05.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:12:05.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2154000000000002e-05 2020-08-07T15:12:05.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:05.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299629+0000", 2020-08-07T15:12:05.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1929999999999998e-06 2020-08-07T15:12:05.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:05.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299636+0000", 2020-08-07T15:12:05.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44766315400000001 2020-08-07T15:12:05.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:05.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747299+0000", 2020-08-07T15:12:05.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5889999999999999e-06 2020-08-07T15:12:05.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:05.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747305+0000", 2020-08-07T15:12:05.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3094999999999999e-05 2020-08-07T15:12:05.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:05.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747358+0000", 2020-08-07T15:12:05.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3283e-05 2020-08-07T15:12:05.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:05.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747392+0000", 2020-08-07T15:12:05.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015319509999999999 2020-08-07T15:12:05.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:05.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748924+0000", 2020-08-07T15:12:05.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5235999999999996e-05 2020-08-07T15:12:05.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:05.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748999+0000", 2020-08-07T15:12:05.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998 2020-08-07T15:12:05.777 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:05.778 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:05.778 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:05.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.779 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42520 4.8 e113/81 4:13071de5:::benchmark_data_smithi074_44586_object5314:head v 113'2624, mlcod=113'2618)", 2020-08-07T15:12:05.779 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:12:05.779 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 184.50781150099999, 2020-08-07T15:12:05.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001, 2020-08-07T15:12:05.779 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:05.780 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:05.780 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:05.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:05.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:12:05.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:05.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:05.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:12:05.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3967999999999995e-05 2020-08-07T15:12:05.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:05.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765585+0000", 2020-08-07T15:12:05.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0949999999999998e-06 2020-08-07T15:12:05.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:05.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765593+0000", 2020-08-07T15:12:05.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52703464200000005 2020-08-07T15:12:05.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:05.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292627+0000", 2020-08-07T15:12:05.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6929999999999996e-06 2020-08-07T15:12:05.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:05.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292636+0000", 2020-08-07T15:12:05.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9292000000000001e-05 2020-08-07T15:12:05.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:05.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292695+0000", 2020-08-07T15:12:05.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5395999999999999e-05 2020-08-07T15:12:05.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:05.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292721+0000", 2020-08-07T15:12:05.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00090361399999999996 2020-08-07T15:12:05.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:05.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293624+0000", 2020-08-07T15:12:05.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1761000000000002e-05 2020-08-07T15:12:05.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:05.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293666+0000", 2020-08-07T15:12:05.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001 2020-08-07T15:12:05.827 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:05.827 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:05.828 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:05.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.828 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:19776 5.4 5:2bb0c0d9:::benchmark_data_smithi074_53697_object2471:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e173)", 2020-08-07T15:12:05.828 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:12:05.829 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 92.873618816000004, 2020-08-07T15:12:05.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.402061844, 2020-08-07T15:12:05.829 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:05.829 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:05.829 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:05.831 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:05.831 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:05.831 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 19776 2020-08-07T15:12:05.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.832 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:05.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:05.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:12:05.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:05.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:05.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:12:05.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-05 2020-08-07T15:12:05.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:05.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399752+0000", 2020-08-07T15:12:05.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8799999999999999e-06 2020-08-07T15:12:05.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:05.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399757+0000", 2020-08-07T15:12:05.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7690000000000008e-06 2020-08-07T15:12:05.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:05.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399767+0000", 2020-08-07T15:12:05.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-06 2020-08-07T15:12:05.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:05.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399771+0000", 2020-08-07T15:12:05.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003255109 2020-08-07T15:12:05.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:05.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403026+0000", 2020-08-07T15:12:05.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101044 2020-08-07T15:12:05.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:05.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403127+0000", 2020-08-07T15:12:05.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000193158 2020-08-07T15:12:05.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,5", 2020-08-07T15:12:05.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403320+0000", 2020-08-07T15:12:05.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017185530000000001 2020-08-07T15:12:05.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:05.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405038+0000", 2020-08-07T15:12:05.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6300000000000004e-07 2020-08-07T15:12:05.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:05.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405039+0000", 2020-08-07T15:12:05.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012590600000000001 2020-08-07T15:12:05.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:05.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405165+0000", 2020-08-07T15:12:05.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109757 2020-08-07T15:12:05.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:05.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405275+0000", 2020-08-07T15:12:05.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39642174699999999 2020-08-07T15:12:05.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:05.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801697+0000", 2020-08-07T15:12:05.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2799999999999996e-07 2020-08-07T15:12:05.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:05.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801697+0000", 2020-08-07T15:12:05.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8740999999999997e-05 2020-08-07T15:12:05.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:05.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801776+0000", 2020-08-07T15:12:05.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.402061844 2020-08-07T15:12:05.859 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:05.860 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:05.860 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:05.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.861 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:19776 5.4 e173/164 ondisk, result = 0)", 2020-08-07T15:12:05.861 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:12:05.861 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 92.867870513, 2020-08-07T15:12:05.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39631765200000002, 2020-08-07T15:12:05.861 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:05.862 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:05.862 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:05.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:05.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:12:05.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:05.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:05.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:12:05.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9332e-05 2020-08-07T15:12:05.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:05.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405492+0000", 2020-08-07T15:12:05.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8900000000000004e-06 2020-08-07T15:12:05.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:05.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405498+0000", 2020-08-07T15:12:05.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39566664200000001 2020-08-07T15:12:05.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:05.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801164+0000", 2020-08-07T15:12:05.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7440000000000001e-06 2020-08-07T15:12:05.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:05.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801168+0000", 2020-08-07T15:12:05.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050828199999999998 2020-08-07T15:12:05.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:05.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801676+0000", 2020-08-07T15:12:05.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1011e-05 2020-08-07T15:12:05.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:05.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801687+0000", 2020-08-07T15:12:05.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2750999999999993e-05 2020-08-07T15:12:05.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:05.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801780+0000", 2020-08-07T15:12:05.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39631765200000002 2020-08-07T15:12:05.884 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:05.885 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:05.885 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:05.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.885 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:85551 5.9 5:9d69b7cc:::benchmark_data_smithi074_53697_object10693:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e202)", 2020-08-07T15:12:05.886 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.807277+0000", 2020-08-07T15:12:05.886 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.466056446000003, 2020-08-07T15:12:05.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39614286500000001, 2020-08-07T15:12:05.886 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:05.886 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:05.887 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:05.887 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:05.887 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:05.887 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 85551 2020-08-07T15:12:05.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.888 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:05.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:05.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807277+0000", 2020-08-07T15:12:05.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:05.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:05.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807277+0000", 2020-08-07T15:12:05.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8867999999999999e-05 2020-08-07T15:12:05.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:05.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807305+0000", 2020-08-07T15:12:05.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.775e-06 2020-08-07T15:12:05.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:05.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807310+0000", 2020-08-07T15:12:05.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2747e-05 2020-08-07T15:12:05.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:05.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807323+0000", 2020-08-07T15:12:05.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2669999999999997e-06 2020-08-07T15:12:05.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:05.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807328+0000", 2020-08-07T15:12:05.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001048513 2020-08-07T15:12:05.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:05.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.808377+0000", 2020-08-07T15:12:05.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010425399999999999 2020-08-07T15:12:05.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:05.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.808481+0000", 2020-08-07T15:12:05.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013761900000000001 2020-08-07T15:12:05.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:12:05.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.808619+0000", 2020-08-07T15:12:05.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042907500000000003 2020-08-07T15:12:05.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:05.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.812909+0000", 2020-08-07T15:12:05.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.077e-06 2020-08-07T15:12:05.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:05.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.812910+0000", 2020-08-07T15:12:05.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8186000000000001e-05 2020-08-07T15:12:05.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:05.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.812979+0000", 2020-08-07T15:12:05.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00081526099999999996 2020-08-07T15:12:05.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:05.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813794+0000", 2020-08-07T15:12:05.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38955804599999999 2020-08-07T15:12:05.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:05.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203352+0000", 2020-08-07T15:12:05.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5239999999999999e-06 2020-08-07T15:12:05.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:05.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203353+0000", 2020-08-07T15:12:05.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5977999999999997e-05 2020-08-07T15:12:05.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:05.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203419+0000", 2020-08-07T15:12:05.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39614286500000001 2020-08-07T15:12:05.934 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:05.935 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:05.935 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:05.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.935 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:85552 5.9 5:9d69b7cc:::benchmark_data_smithi074_53697_object10693:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e202)", 2020-08-07T15:12:05.935 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:12:05.936 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.465815722000002, 2020-08-07T15:12:05.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54429374399999997, 2020-08-07T15:12:05.970 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:05.970 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:05.970 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:05.971 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:05.982 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:05.982 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 85552 2020-08-07T15:12:05.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.983 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:05.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:05.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:12:05.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:05.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:05.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:12:05.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2691000000000001e-05 2020-08-07T15:12:05.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:05.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:05.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:05.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807570+0000", 2020-08-07T15:12:06.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0619999999999997e-06 2020-08-07T15:12:06.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:06.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807576+0000", 2020-08-07T15:12:06.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1558e-05 2020-08-07T15:12:06.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:06.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807588+0000", 2020-08-07T15:12:06.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4600000000000002e-06 2020-08-07T15:12:06.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:06.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807593+0000", 2020-08-07T15:12:06.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022126279999999999 2020-08-07T15:12:06.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:06.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.809806+0000", 2020-08-07T15:12:06.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104868 2020-08-07T15:12:06.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:06.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.809911+0000", 2020-08-07T15:12:06.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000186019 2020-08-07T15:12:06.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:12:06.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.810097+0000", 2020-08-07T15:12:06.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037966739999999999 2020-08-07T15:12:06.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:06.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813893+0000", 2020-08-07T15:12:06.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1310000000000001e-06 2020-08-07T15:12:06.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:06.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813894+0000", 2020-08-07T15:12:06.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4704999999999999e-05 2020-08-07T15:12:06.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:06.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813949+0000", 2020-08-07T15:12:06.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015487280000000001 2020-08-07T15:12:06.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:06.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.815498+0000", 2020-08-07T15:12:06.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53621442900000005 2020-08-07T15:12:06.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:06.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351712+0000", 2020-08-07T15:12:06.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8199999999999998e-07 2020-08-07T15:12:06.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:06.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351713+0000", 2020-08-07T15:12:06.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7909000000000001e-05 2020-08-07T15:12:06.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:06.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351811+0000", 2020-08-07T15:12:06.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54429374399999997 2020-08-07T15:12:06.083 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:06.083 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:06.083 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:06.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.084 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:85551 5.9 e202/164 ondisk, result = 0)", 2020-08-07T15:12:06.084 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.813683+0000", 2020-08-07T15:12:06.084 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.459650418999999, 2020-08-07T15:12:06.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38973959000000002, 2020-08-07T15:12:06.125 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:06.125 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:06.125 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:06.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:06.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813683+0000", 2020-08-07T15:12:06.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:06.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:06.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813683+0000", 2020-08-07T15:12:06.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9434999999999999e-05 2020-08-07T15:12:06.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:06.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813712+0000", 2020-08-07T15:12:06.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6759999999999998e-06 2020-08-07T15:12:06.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:06.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813717+0000", 2020-08-07T15:12:06.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.389373209 2020-08-07T15:12:06.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:06.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203090+0000", 2020-08-07T15:12:06.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0859999999999998e-06 2020-08-07T15:12:06.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:06.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203096+0000", 2020-08-07T15:12:06.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024188899999999999 2020-08-07T15:12:06.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:06.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203338+0000", 2020-08-07T15:12:06.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2490000000000001e-06 2020-08-07T15:12:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203346+0000", 2020-08-07T15:12:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6045999999999997e-05 2020-08-07T15:12:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203422+0000", 2020-08-07T15:12:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38973959000000002 2020-08-07T15:12:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:85552 5.9 e202/164 ondisk, result = 0)", 2020-08-07T15:12:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:12:06.144 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.459589319999999, 2020-08-07T15:12:06.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53813836800000003, 2020-08-07T15:12:06.144 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:06.145 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:06.145 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:06.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:06.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:12:06.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:06.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:06.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:12:06.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8833e-05 2020-08-07T15:12:06.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:06.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813772+0000", 2020-08-07T15:12:06.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8459999999999999e-06 2020-08-07T15:12:06.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:06.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813777+0000", 2020-08-07T15:12:06.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53732224900000003 2020-08-07T15:12:06.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:06.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351100+0000", 2020-08-07T15:12:06.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1869999999999996e-06 2020-08-07T15:12:06.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:06.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351105+0000", 2020-08-07T15:12:06.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00058858100000000002 2020-08-07T15:12:06.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:06.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351693+0000", 2020-08-07T15:12:06.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4229999999999994e-06 2020-08-07T15:12:06.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:06.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351703+0000", 2020-08-07T15:12:06.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000179249 2020-08-07T15:12:06.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:06.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351882+0000", 2020-08-07T15:12:06.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53813836800000003 2020-08-07T15:12:06.168 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:06.168 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:06.169 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:06.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.169 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:85552 5.9 5:9d69b7cc:::benchmark_data_smithi074_53697_object10693:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e202)", 2020-08-07T15:12:06.169 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:12:06.170 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.454841195999997, 2020-08-07T15:12:06.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53326610600000002, 2020-08-07T15:12:06.173 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:06.174 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:12:06.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:06.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:06.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:06.174 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 85552 2020-08-07T15:12:06.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.175 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:06.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:06.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:12:06.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:06.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:06.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:12:06.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5001e-05 2020-08-07T15:12:06.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:06.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818517+0000", 2020-08-07T15:12:06.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5000000000000002e-06 2020-08-07T15:12:06.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:06.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818519+0000", 2020-08-07T15:12:06.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0699999999999997e-06 2020-08-07T15:12:06.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:06.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818524+0000", 2020-08-07T15:12:06.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2910000000000002e-06 2020-08-07T15:12:06.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:06.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818527+0000", 2020-08-07T15:12:06.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030912499999999999 2020-08-07T15:12:06.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:06.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818836+0000", 2020-08-07T15:12:06.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2908000000000002e-05 2020-08-07T15:12:06.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for ondisk", 2020-08-07T15:12:06.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818889+0000", 2020-08-07T15:12:06.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53286921099999995 2020-08-07T15:12:06.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:06.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351758+0000", 2020-08-07T15:12:06.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53326610600000002 2020-08-07T15:12:06.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:06.197 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:06.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:06.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.198 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:89464 5.9 5:94c69ba5:::benchmark_data_smithi074_53697_object11182:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:06.198 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:12:06.198 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 43.857001242000003, 2020-08-07T15:12:06.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67692916199999997, 2020-08-07T15:12:06.198 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:06.199 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:06.199 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:06.199 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:06.199 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:06.199 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 89464 2020-08-07T15:12:06.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.200 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:06.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:06.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:12:06.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:06.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:06.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:12:06.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0074000000000002e-05 2020-08-07T15:12:06.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:06.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416382+0000", 2020-08-07T15:12:06.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5000000000000001e-06 2020-08-07T15:12:06.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:06.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416386+0000", 2020-08-07T15:12:06.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2447e-05 2020-08-07T15:12:06.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:06.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416399+0000", 2020-08-07T15:12:06.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1279999999999996e-06 2020-08-07T15:12:06.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:06.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416405+0000", 2020-08-07T15:12:06.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016641805999999999 2020-08-07T15:12:06.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:06.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433047+0000", 2020-08-07T15:12:06.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107995 2020-08-07T15:12:06.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:06.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433155+0000", 2020-08-07T15:12:06.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018818599999999999 2020-08-07T15:12:06.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:12:06.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433343+0000", 2020-08-07T15:12:06.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039911679999999998 2020-08-07T15:12:06.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:06.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437334+0000", 2020-08-07T15:12:06.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2020-08-07T15:12:06.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:06.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437335+0000", 2020-08-07T15:12:06.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6351000000000002e-05 2020-08-07T15:12:06.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:06.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437401+0000", 2020-08-07T15:12:06.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001267949 2020-08-07T15:12:06.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:06.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438669+0000", 2020-08-07T15:12:06.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65454035600000005 2020-08-07T15:12:06.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:06.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093209+0000", 2020-08-07T15:12:06.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7859999999999999e-06 2020-08-07T15:12:06.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:06.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093211+0000", 2020-08-07T15:12:06.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9665e-05 2020-08-07T15:12:06.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:06.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093261+0000", 2020-08-07T15:12:06.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67692916199999997 2020-08-07T15:12:06.245 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:06.245 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:06.246 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:06.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.247 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:89464 5.9 e207/164 ondisk, result = 0)", 2020-08-07T15:12:06.247 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:12:06.247 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 43.834371480000001, 2020-08-07T15:12:06.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65430647200000003, 2020-08-07T15:12:06.248 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:06.248 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:06.249 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:06.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:06.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:12:06.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:06.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:06.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:12:06.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6529000000000002e-05 2020-08-07T15:12:06.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:06.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438998+0000", 2020-08-07T15:12:06.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.567e-06 2020-08-07T15:12:06.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:06.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.439005+0000", 2020-08-07T15:12:06.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65408475099999996 2020-08-07T15:12:06.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:06.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093089+0000", 2020-08-07T15:12:06.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6880000000000002e-06 2020-08-07T15:12:06.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:06.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093096+0000", 2020-08-07T15:12:06.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4980000000000002e-05 2020-08-07T15:12:06.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:06.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093191+0000", 2020-08-07T15:12:06.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3090000000000001e-06 2020-08-07T15:12:06.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:06.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093200+0000", 2020-08-07T15:12:06.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7648e-05 2020-08-07T15:12:06.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:06.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093268+0000", 2020-08-07T15:12:06.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65430647200000003 2020-08-07T15:12:06.266 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:06.267 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:06.267 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:06.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.268 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:114080 5.4 5:24adcd54:::benchmark_data_smithi074_53697_object14259:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e215)", 2020-08-07T15:12:06.269 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:12:06.269 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.673664023000001, 2020-08-07T15:12:06.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.57290862300000001, 2020-08-07T15:12:06.270 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:06.270 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:06.271 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:06.272 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:06.273 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:06.273 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 114080 2020-08-07T15:12:06.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.275 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:06.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:06.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:12:06.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:06.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:06.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:12:06.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9238e-05 2020-08-07T15:12:06.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:06.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599698+0000", 2020-08-07T15:12:06.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6260000000000003e-06 2020-08-07T15:12:06.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:06.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599703+0000", 2020-08-07T15:12:06.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5577999999999999e-05 2020-08-07T15:12:06.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:06.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599718+0000", 2020-08-07T15:12:06.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9869999999999999e-06 2020-08-07T15:12:06.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:06.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599726+0000", 2020-08-07T15:12:06.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00077929599999999996 2020-08-07T15:12:06.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:06.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600506+0000", 2020-08-07T15:12:06.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1717000000000003e-05 2020-08-07T15:12:06.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:06.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600597+0000", 2020-08-07T15:12:06.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011828999999999999 2020-08-07T15:12:06.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,5", 2020-08-07T15:12:06.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600716+0000", 2020-08-07T15:12:06.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020861479999999999 2020-08-07T15:12:06.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:06.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602802+0000", 2020-08-07T15:12:06.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4600000000000005e-07 2020-08-07T15:12:06.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:06.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602802+0000", 2020-08-07T15:12:06.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9165e-05 2020-08-07T15:12:06.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:06.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602862+0000", 2020-08-07T15:12:06.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0093485080000000002 2020-08-07T15:12:06.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:06.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.612210+0000", 2020-08-07T15:12:06.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.56030197699999995 2020-08-07T15:12:06.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:06.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172512+0000", 2020-08-07T15:12:06.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8800000000000001e-07 2020-08-07T15:12:06.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:06.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172513+0000", 2020-08-07T15:12:06.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4659000000000006e-05 2020-08-07T15:12:06.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:06.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:06.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:06.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172578+0000", 2020-08-07T15:12:06.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.57290862300000001 2020-08-07T15:12:06.300 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:06.300 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:06.300 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:06.301 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:06.301 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:06.301 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:06.302 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:12:06.307 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:12:06.322 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_2 2020-08-07T15:12:06.325 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool rm unique_pool_2 unique_pool_2 --yes-i-really-really-mean-it 2020-08-07T15:12:06.326 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:06.317+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:06.419 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:06.418+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:06.498 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:06.499 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:12:06.499 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:12:06.499 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:06.504 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:12:06.523 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:06.522+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:06.625 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:06.623+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:06.667 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:06.668 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:12:06.668 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:12:06.668 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:12:06.668 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:06.679 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:12:06.726 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:06.723+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:06.803 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:06.804 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:12:06.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:12:06.804 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:12:06.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.805 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:145801 3.6 e75/26 3:69e4cb22:::benchmark_data_smithi074_35447_object16192:head v 75'9405, mlcod=75'9401)", 2020-08-07T15:12:06.805 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:12:06.805 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 232.12477960699999, 2020-08-07T15:12:06.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639, 2020-08-07T15:12:06.823 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:06.823 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:06.824 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:06.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:06.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:12:06.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:06.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:06.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:12:06.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6511000000000002e-05 2020-08-07T15:12:06.833 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:06.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671803+0000", 2020-08-07T15:12:06.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7190000000000004e-06 2020-08-07T15:12:06.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:06.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671809+0000", 2020-08-07T15:12:06.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74210928300000001 2020-08-07T15:12:06.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:06.836 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413919+0000", 2020-08-07T15:12:06.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1759999999999999e-05 2020-08-07T15:12:06.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.836 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:06.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413930+0000", 2020-08-07T15:12:06.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000108601 2020-08-07T15:12:06.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:06.847 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414039+0000", 2020-08-07T15:12:06.848 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9948999999999999e-05 2020-08-07T15:12:06.848 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.848 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.848 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:06.849 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414089+0000", 2020-08-07T15:12:06.849 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00079656499999999995 2020-08-07T15:12:06.849 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.849 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.850 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:06.850 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414885+0000", 2020-08-07T15:12:06.850 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2250999999999998e-05 2020-08-07T15:12:06.850 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.851 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:06.856 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414938+0000", 2020-08-07T15:12:06.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639 2020-08-07T15:12:06.856 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:06.857 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:06.857 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:06.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.857 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:104104 4.0 e146/131 4:0ffb764a:::benchmark_data_smithi074_44586_object13012:head v 146'6960, mlcod=146'6953)", 2020-08-07T15:12:06.858 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:12:06.858 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 138.59823283099999, 2020-08-07T15:12:06.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75443331599999996, 2020-08-07T15:12:06.858 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:06.858 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:06.859 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:06.859 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.859 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:06.859 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:12:06.859 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:06.860 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.860 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:06.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:12:06.860 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1743000000000002e-05 2020-08-07T15:12:06.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.861 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:06.861 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198355+0000", 2020-08-07T15:12:06.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1249999999999998e-06 2020-08-07T15:12:06.862 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.862 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.862 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:06.862 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198361+0000", 2020-08-07T15:12:06.862 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74719179599999996 2020-08-07T15:12:06.862 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:06.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945553+0000", 2020-08-07T15:12:06.863 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6800000000000001e-06 2020-08-07T15:12:06.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:06.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945557+0000", 2020-08-07T15:12:06.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012101500000000001 2020-08-07T15:12:06.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.865 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:06.866 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945678+0000", 2020-08-07T15:12:06.866 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6621000000000001e-05 2020-08-07T15:12:06.867 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.867 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.867 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:06.867 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945705+0000", 2020-08-07T15:12:06.867 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069879540000000002 2020-08-07T15:12:06.868 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.868 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.868 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:06.868 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.952693+0000", 2020-08-07T15:12:06.868 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3381999999999999e-05 2020-08-07T15:12:06.869 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.869 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.869 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:06.869 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.952736+0000", 2020-08-07T15:12:06.870 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75443331599999996 2020-08-07T15:12:06.870 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:06.870 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:06.871 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:06.871 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.871 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89641 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:06.872 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.550933+0000", 2020-08-07T15:12:06.872 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 45.245602409999996, 2020-08-07T15:12:06.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69051440900000005, 2020-08-07T15:12:06.872 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:06.872 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:06.873 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:06.873 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:06.873 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:06.873 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89641 2020-08-07T15:12:06.873 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.874 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:06.874 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.874 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:06.874 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550933+0000", 2020-08-07T15:12:06.874 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:06.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:06.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550933+0000", 2020-08-07T15:12:06.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6520000000000002e-05 2020-08-07T15:12:06.876 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.876 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:06.877 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550980+0000", 2020-08-07T15:12:06.877 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2410000000000003e-06 2020-08-07T15:12:06.877 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.877 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.877 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:06.878 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550985+0000", 2020-08-07T15:12:06.878 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2707e-05 2020-08-07T15:12:06.878 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.878 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.878 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:06.879 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550998+0000", 2020-08-07T15:12:06.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8629999999999996e-06 2020-08-07T15:12:06.879 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.879 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:06.879 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551004+0000", 2020-08-07T15:12:06.880 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4987000000000001e-05 2020-08-07T15:12:06.880 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.880 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:06.880 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551059+0000", 2020-08-07T15:12:06.881 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1718e-05 2020-08-07T15:12:06.881 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.881 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.881 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:06.881 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551070+0000", 2020-08-07T15:12:06.882 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67720431000000003 2020-08-07T15:12:06.882 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.882 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.882 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:06.882 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.228275+0000", 2020-08-07T15:12:06.899 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000731965 2020-08-07T15:12:06.899 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.899 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.899 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:06.900 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229007+0000", 2020-08-07T15:12:06.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00025396199999999999 2020-08-07T15:12:06.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.900 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:06.901 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229261+0000", 2020-08-07T15:12:06.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002511189 2020-08-07T15:12:06.901 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.901 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.901 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:06.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231772+0000", 2020-08-07T15:12:06.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3399999999999997e-07 2020-08-07T15:12:06.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.922 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:06.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231773+0000", 2020-08-07T15:12:06.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0274e-05 2020-08-07T15:12:06.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:06.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231823+0000", 2020-08-07T15:12:06.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0094095440000000006 2020-08-07T15:12:06.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:06.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241233+0000", 2020-08-07T15:12:06.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4011999999999998e-05 2020-08-07T15:12:06.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:06.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241277+0000", 2020-08-07T15:12:06.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7199999999999999e-07 2020-08-07T15:12:06.925 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:06.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241277+0000", 2020-08-07T15:12:06.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017061100000000001 2020-08-07T15:12:06.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:06.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241448+0000", 2020-08-07T15:12:06.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69051440900000005 2020-08-07T15:12:06.929 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:06.929 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:06.929 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:06.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.930 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89642 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:06.930 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.551807+0000", 2020-08-07T15:12:06.930 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 45.244728858000002, 2020-08-07T15:12:06.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68991613100000004, 2020-08-07T15:12:06.930 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:06.931 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:06.932 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:06.932 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:06.933 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:06.933 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89642 2020-08-07T15:12:06.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.933 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:06.934 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.934 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:06.934 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551807+0000", 2020-08-07T15:12:06.934 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:06.934 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:06.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551807+0000", 2020-08-07T15:12:06.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7049000000000001e-05 2020-08-07T15:12:06.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:06.936 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551834+0000", 2020-08-07T15:12:06.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9249999999999997e-06 2020-08-07T15:12:06.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:06.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551838+0000", 2020-08-07T15:12:06.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9160000000000001e-06 2020-08-07T15:12:06.937 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.938 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:06.938 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551846+0000", 2020-08-07T15:12:06.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3510000000000002e-06 2020-08-07T15:12:06.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:06.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551849+0000", 2020-08-07T15:12:06.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9276000000000001e-05 2020-08-07T15:12:06.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.940 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:06.940 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551898+0000", 2020-08-07T15:12:06.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3095000000000001e-05 2020-08-07T15:12:06.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.940 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.940 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:06.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551911+0000", 2020-08-07T15:12:06.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67789159799999998 2020-08-07T15:12:06.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.942 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:06.942 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229803+0000", 2020-08-07T15:12:06.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8069000000000001e-05 2020-08-07T15:12:06.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:06.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229881+0000", 2020-08-07T15:12:06.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000152917 2020-08-07T15:12:06.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.944 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:06.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230034+0000", 2020-08-07T15:12:06.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044936760000000003 2020-08-07T15:12:06.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:06.946 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234528+0000", 2020-08-07T15:12:06.946 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8999999999999995e-07 2020-08-07T15:12:06.946 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:06.964 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234529+0000", 2020-08-07T15:12:06.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7741000000000002e-05 2020-08-07T15:12:06.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:06.966 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234586+0000", 2020-08-07T15:12:06.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069143959999999997 2020-08-07T15:12:06.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:06.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241501+0000", 2020-08-07T15:12:06.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1433000000000001e-05 2020-08-07T15:12:06.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:06.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241542+0000", 2020-08-07T15:12:06.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7800000000000001e-07 2020-08-07T15:12:06.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.969 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:06.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241543+0000", 2020-08-07T15:12:06.969 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000180321 2020-08-07T15:12:06.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:06.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241723+0000", 2020-08-07T15:12:06.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68991613100000004 2020-08-07T15:12:06.986 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:06.987 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:06.987 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:06.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:06.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:06.990 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89643 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:06.990 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.552181+0000", 2020-08-07T15:12:06.991 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 45.244354864000002, 2020-08-07T15:12:06.991 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68980872400000004, 2020-08-07T15:12:06.991 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:06.991 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:06.991 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:06.992 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:07.004 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:07.004 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89643 2020-08-07T15:12:07.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.004 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:07.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:07.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552181+0000", 2020-08-07T15:12:07.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:07.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:07.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552181+0000", 2020-08-07T15:12:07.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0926999999999997e-05 2020-08-07T15:12:07.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:07.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552212+0000", 2020-08-07T15:12:07.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0512e-05 2020-08-07T15:12:07.012 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.012 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:07.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552222+0000", 2020-08-07T15:12:07.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9379999999999992e-06 2020-08-07T15:12:07.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:07.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552230+0000", 2020-08-07T15:12:07.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0260000000000001e-06 2020-08-07T15:12:07.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:07.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552234+0000", 2020-08-07T15:12:07.030 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6109000000000002e-05 2020-08-07T15:12:07.030 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.030 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552280+0000", 2020-08-07T15:12:07.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5318000000000001e-05 2020-08-07T15:12:07.049 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.049 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.049 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:07.049 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552296+0000", 2020-08-07T15:12:07.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67811269100000005 2020-08-07T15:12:07.058 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.058 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.059 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.065 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230408+0000", 2020-08-07T15:12:07.065 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000125518 2020-08-07T15:12:07.066 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.066 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.074 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:07.074 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230534+0000", 2020-08-07T15:12:07.075 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024637999999999998 2020-08-07T15:12:07.076 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.086 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:07.087 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230780+0000", 2020-08-07T15:12:07.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0066398869999999997 2020-08-07T15:12:07.090 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.091 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.091 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:07.091 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237420+0000", 2020-08-07T15:12:07.092 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3500000000000005e-07 2020-08-07T15:12:07.092 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.093 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.093 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:07.093 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237421+0000", 2020-08-07T15:12:07.106 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4005000000000003e-05 2020-08-07T15:12:07.107 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.108 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.112 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:07.118 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237465+0000", 2020-08-07T15:12:07.119 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043133529999999998 2020-08-07T15:12:07.119 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.119 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.120 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:07.120 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241778+0000", 2020-08-07T15:12:07.120 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0592e-05 2020-08-07T15:12:07.124 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.126 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.126 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:07.127 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241819+0000", 2020-08-07T15:12:07.127 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4999999999999998e-07 2020-08-07T15:12:07.127 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.128 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.140 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:07.140 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241819+0000", 2020-08-07T15:12:07.140 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017018299999999999 2020-08-07T15:12:07.141 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.141 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.141 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:07.143 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241990+0000", 2020-08-07T15:12:07.143 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68980872400000004 2020-08-07T15:12:07.150 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:07.151 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:07.151 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:07.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.152 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89644 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:07.161 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.552384+0000", 2020-08-07T15:12:07.161 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 45.24415183, 2020-08-07T15:12:07.162 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68997479299999998, 2020-08-07T15:12:07.162 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:07.166 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:07.166 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:07.180 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:07.180 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:07.181 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89644 2020-08-07T15:12:07.188 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.189 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:07.189 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.189 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:07.190 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552384+0000", 2020-08-07T15:12:07.210 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:07.211 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.211 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.212 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:07.212 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552384+0000", 2020-08-07T15:12:07.212 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9238999999999999e-05 2020-08-07T15:12:07.213 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.213 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.213 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:07.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552413+0000", 2020-08-07T15:12:07.228 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3340000000000003e-06 2020-08-07T15:12:07.229 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.229 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.229 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:07.235 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552417+0000", 2020-08-07T15:12:07.235 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3119999999999994e-06 2020-08-07T15:12:07.242 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.243 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.243 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:07.243 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552426+0000", 2020-08-07T15:12:07.244 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4139999999999996e-06 2020-08-07T15:12:07.244 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.244 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.250 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:07.251 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552430+0000", 2020-08-07T15:12:07.257 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6613999999999999e-05 2020-08-07T15:12:07.257 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.257 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.265 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.265 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:12:07.265 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7280000000000007e-06 2020-08-07T15:12:07.270 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.270 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:07.271 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552477+0000", 2020-08-07T15:12:07.271 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67873990399999995 2020-08-07T15:12:07.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.273 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231216+0000", 2020-08-07T15:12:07.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3629999999999998e-05 2020-08-07T15:12:07.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.288 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:07.288 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231290+0000", 2020-08-07T15:12:07.288 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000132108 2020-08-07T15:12:07.289 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.289 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.295 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:07.295 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231422+0000", 2020-08-07T15:12:07.295 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006049652 2020-08-07T15:12:07.296 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.296 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.296 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:07.306 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237472+0000", 2020-08-07T15:12:07.306 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000005e-07 2020-08-07T15:12:07.306 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.306 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.312 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:07.313 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237472+0000", 2020-08-07T15:12:07.313 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2848000000000001e-05 2020-08-07T15:12:07.313 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.318 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.318 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:07.318 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237505+0000", 2020-08-07T15:12:07.318 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045352179999999997 2020-08-07T15:12:07.319 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.319 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.319 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:07.319 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242040+0000", 2020-08-07T15:12:07.319 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012843699999999999 2020-08-07T15:12:07.320 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.320 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.320 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:07.320 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242169+0000", 2020-08-07T15:12:07.320 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9999999999999998e-07 2020-08-07T15:12:07.321 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.321 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.321 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:07.330 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242169+0000", 2020-08-07T15:12:07.330 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000189354 2020-08-07T15:12:07.330 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.331 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.331 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:07.331 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242359+0000", 2020-08-07T15:12:07.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68997479299999998 2020-08-07T15:12:07.331 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:07.334 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:07.334 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:07.335 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.335 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89645 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:07.335 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:12:07.336 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 45.244069048, 2020-08-07T15:12:07.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69011194600000003, 2020-08-07T15:12:07.336 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:07.337 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:07.337 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:07.337 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:07.351 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:07.351 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89645 2020-08-07T15:12:07.352 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.352 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:07.357 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.358 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:07.358 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:12:07.358 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:07.362 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.362 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.362 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:07.362 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:12:07.370 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6750999999999998e-05 2020-08-07T15:12:07.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.371 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.371 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:07.372 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552493+0000", 2020-08-07T15:12:07.372 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0350000000000002e-06 2020-08-07T15:12:07.373 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.373 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.374 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:07.374 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552496+0000", 2020-08-07T15:12:07.375 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3029999999999996e-06 2020-08-07T15:12:07.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:07.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552503+0000", 2020-08-07T15:12:07.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7580000000000001e-06 2020-08-07T15:12:07.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:07.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552506+0000", 2020-08-07T15:12:07.412 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4868000000000003e-05 2020-08-07T15:12:07.412 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.413 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.414 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552570+0000", 2020-08-07T15:12:07.414 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9649999999999997e-06 2020-08-07T15:12:07.414 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.415 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:07.415 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552579+0000", 2020-08-07T15:12:07.416 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67928357699999997 2020-08-07T15:12:07.416 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.417 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.417 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.421 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231863+0000", 2020-08-07T15:12:07.421 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000115696 2020-08-07T15:12:07.422 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.423 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.423 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:07.424 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231979+0000", 2020-08-07T15:12:07.424 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020456599999999999 2020-08-07T15:12:07.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.425 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.426 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:07.426 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232183+0000", 2020-08-07T15:12:07.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005327923 2020-08-07T15:12:07.427 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.427 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:07.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237511+0000", 2020-08-07T15:12:07.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8999999999999998e-07 2020-08-07T15:12:07.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.429 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:07.433 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237511+0000", 2020-08-07T15:12:07.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6019000000000002e-05 2020-08-07T15:12:07.433 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.433 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.434 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:07.434 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237547+0000", 2020-08-07T15:12:07.434 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045340980000000003 2020-08-07T15:12:07.434 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.434 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.435 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:07.435 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242082+0000", 2020-08-07T15:12:07.435 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00032620499999999998 2020-08-07T15:12:07.435 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.435 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.436 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:07.436 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242408+0000", 2020-08-07T15:12:07.436 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3500000000000002e-07 2020-08-07T15:12:07.436 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.436 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.437 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:07.437 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242408+0000", 2020-08-07T15:12:07.437 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000170457 2020-08-07T15:12:07.437 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.437 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.438 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:07.438 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242579+0000", 2020-08-07T15:12:07.438 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69011194600000003 2020-08-07T15:12:07.438 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:07.438 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:07.439 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:07.439 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.439 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.439 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89646 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:07.439 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.557244+0000", 2020-08-07T15:12:07.440 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 45.239291276000003, 2020-08-07T15:12:07.440 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68559352299999998, 2020-08-07T15:12:07.440 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:07.440 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:07.440 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:07.441 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:07.441 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:07.441 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89646 2020-08-07T15:12:07.441 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.441 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:07.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.444 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:07.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557244+0000", 2020-08-07T15:12:07.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:07.445 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.445 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.445 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:07.446 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557244+0000", 2020-08-07T15:12:07.446 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9699000000000001e-05 2020-08-07T15:12:07.446 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.446 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.446 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:07.446 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557274+0000", 2020-08-07T15:12:07.447 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1269999999999996e-06 2020-08-07T15:12:07.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.447 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:07.447 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557278+0000", 2020-08-07T15:12:07.448 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0099999999999995e-06 2020-08-07T15:12:07.448 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:07.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557286+0000", 2020-08-07T15:12:07.449 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7439999999999998e-06 2020-08-07T15:12:07.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.449 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.449 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:07.449 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557291+0000", 2020-08-07T15:12:07.450 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5927999999999999e-05 2020-08-07T15:12:07.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.450 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.450 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557327+0000", 2020-08-07T15:12:07.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5069999999999997e-06 2020-08-07T15:12:07.451 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.451 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.451 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:07.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557333+0000", 2020-08-07T15:12:07.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67527419200000005 2020-08-07T15:12:07.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232608+0000", 2020-08-07T15:12:07.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8699e-05 2020-08-07T15:12:07.458 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:07.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232686+0000", 2020-08-07T15:12:07.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018185099999999999 2020-08-07T15:12:07.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.461 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.461 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:07.462 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232868+0000", 2020-08-07T15:12:07.462 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0075819360000000001 2020-08-07T15:12:07.484 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.485 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.485 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:07.485 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240450+0000", 2020-08-07T15:12:07.486 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2799999999999995e-07 2020-08-07T15:12:07.486 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.486 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.487 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:07.487 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240451+0000", 2020-08-07T15:12:07.487 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3751000000000002e-05 2020-08-07T15:12:07.488 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:07.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240495+0000", 2020-08-07T15:12:07.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001628064 2020-08-07T15:12:07.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.489 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.490 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:07.490 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242123+0000", 2020-08-07T15:12:07.490 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00050955599999999998 2020-08-07T15:12:07.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.494 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:07.494 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242632+0000", 2020-08-07T15:12:07.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2900000000000004e-07 2020-08-07T15:12:07.495 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.495 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.496 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:07.496 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242633+0000", 2020-08-07T15:12:07.496 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000205202 2020-08-07T15:12:07.496 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.497 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.497 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:07.497 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242838+0000", 2020-08-07T15:12:07.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68559352299999998 2020-08-07T15:12:07.498 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:07.499 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:07.499 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:07.499 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.500 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.500 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89647 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:07.500 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.557315+0000", 2020-08-07T15:12:07.501 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 45.239220953, 2020-08-07T15:12:07.501 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68581815800000001, 2020-08-07T15:12:07.501 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:07.504 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:07.505 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:07.505 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:07.505 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:07.506 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89647 2020-08-07T15:12:07.506 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.506 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:07.507 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.507 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:07.507 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557315+0000", 2020-08-07T15:12:07.507 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:07.507 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.508 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.508 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:07.508 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557315+0000", 2020-08-07T15:12:07.508 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6639999999999999e-05 2020-08-07T15:12:07.508 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.509 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.509 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:07.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557341+0000", 2020-08-07T15:12:07.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7989999999999998e-06 2020-08-07T15:12:07.509 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.510 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:07.511 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557344+0000", 2020-08-07T15:12:07.511 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6960000000000002e-06 2020-08-07T15:12:07.511 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:07.512 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557350+0000", 2020-08-07T15:12:07.512 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6089999999999999e-06 2020-08-07T15:12:07.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.513 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.513 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:07.515 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557352+0000", 2020-08-07T15:12:07.516 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1837e-05 2020-08-07T15:12:07.516 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.516 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.516 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.516 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557384+0000", 2020-08-07T15:12:07.517 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8099999999999998e-06 2020-08-07T15:12:07.517 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.517 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.517 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:07.517 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557392+0000", 2020-08-07T15:12:07.518 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67581117300000004 2020-08-07T15:12:07.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.518 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.518 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233203+0000", 2020-08-07T15:12:07.519 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112316 2020-08-07T15:12:07.519 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.519 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.519 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:07.519 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233316+0000", 2020-08-07T15:12:07.520 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000221766 2020-08-07T15:12:07.520 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.520 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:07.520 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233537+0000", 2020-08-07T15:12:07.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069695019999999998 2020-08-07T15:12:07.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.521 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.521 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:07.521 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240507+0000", 2020-08-07T15:12:07.526 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3999999999999997e-07 2020-08-07T15:12:07.527 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.527 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.527 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:07.527 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240507+0000", 2020-08-07T15:12:07.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8844000000000003e-05 2020-08-07T15:12:07.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.528 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:07.528 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240546+0000", 2020-08-07T15:12:07.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023473399999999998 2020-08-07T15:12:07.529 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.529 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.529 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:07.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242893+0000", 2020-08-07T15:12:07.530 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5410000000000001e-05 2020-08-07T15:12:07.530 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.530 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.530 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:07.530 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242939+0000", 2020-08-07T15:12:07.531 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8000000000000001e-07 2020-08-07T15:12:07.531 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.531 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.531 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:07.531 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242939+0000", 2020-08-07T15:12:07.532 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019369600000000001 2020-08-07T15:12:07.532 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.532 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.533 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:07.533 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243133+0000", 2020-08-07T15:12:07.534 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68581815800000001 2020-08-07T15:12:07.534 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:07.534 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:07.534 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:07.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.535 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89648 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:07.535 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.557372+0000", 2020-08-07T15:12:07.535 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 45.239163832000003, 2020-08-07T15:12:07.539 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68606040000000001, 2020-08-07T15:12:07.540 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:07.540 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:07.540 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:07.540 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:07.541 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:07.541 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89648 2020-08-07T15:12:07.541 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.541 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:07.542 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.542 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:07.542 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557372+0000", 2020-08-07T15:12:07.542 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:07.542 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.543 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.543 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:07.543 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557372+0000", 2020-08-07T15:12:07.543 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5534e-05 2020-08-07T15:12:07.543 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.544 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.544 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:07.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557397+0000", 2020-08-07T15:12:07.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.599e-06 2020-08-07T15:12:07.545 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.545 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:07.546 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557400+0000", 2020-08-07T15:12:07.546 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.288e-06 2020-08-07T15:12:07.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.546 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.546 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:07.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557405+0000", 2020-08-07T15:12:07.547 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.142e-06 2020-08-07T15:12:07.549 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.549 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.549 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:07.550 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557407+0000", 2020-08-07T15:12:07.550 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0954999999999998e-05 2020-08-07T15:12:07.550 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.550 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.550 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.551 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557428+0000", 2020-08-07T15:12:07.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7529999999999998e-06 2020-08-07T15:12:07.551 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.551 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.551 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:07.552 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557434+0000", 2020-08-07T15:12:07.552 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67655471099999998 2020-08-07T15:12:07.552 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.552 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.552 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233989+0000", 2020-08-07T15:12:07.553 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8301999999999997e-05 2020-08-07T15:12:07.553 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.553 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.553 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:07.554 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234067+0000", 2020-08-07T15:12:07.554 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000129908 2020-08-07T15:12:07.554 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.554 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.555 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:07.555 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234197+0000", 2020-08-07T15:12:07.555 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063551249999999997 2020-08-07T15:12:07.556 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:07.556 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240552+0000", 2020-08-07T15:12:07.557 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9400000000000001e-07 2020-08-07T15:12:07.557 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.557 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.557 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:07.558 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240552+0000", 2020-08-07T15:12:07.560 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2787999999999998e-05 2020-08-07T15:12:07.560 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.560 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.561 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:07.561 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240585+0000", 2020-08-07T15:12:07.561 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002605652 2020-08-07T15:12:07.561 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.561 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.562 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:07.562 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243191+0000", 2020-08-07T15:12:07.562 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8690000000000002e-05 2020-08-07T15:12:07.562 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.562 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.563 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:07.563 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243250+0000", 2020-08-07T15:12:07.563 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4700000000000001e-07 2020-08-07T15:12:07.563 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.563 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.564 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:07.564 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243250+0000", 2020-08-07T15:12:07.564 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000181912 2020-08-07T15:12:07.564 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.565 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:07.565 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243432+0000", 2020-08-07T15:12:07.566 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68606040000000001 2020-08-07T15:12:07.566 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:07.566 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:07.566 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:07.566 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.567 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.567 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89689 5.7 5:e83ee43b:::benchmark_data_smithi074_53697_object11211:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:07.567 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.585493+0000", 2020-08-07T15:12:07.567 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 45.211043013999998, 2020-08-07T15:12:07.567 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66549445299999999, 2020-08-07T15:12:07.568 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:07.568 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:07.568 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:07.571 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:07.571 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:07.572 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89689 2020-08-07T15:12:07.572 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.572 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:07.573 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.573 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:07.573 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585493+0000", 2020-08-07T15:12:07.574 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:07.574 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.575 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:07.575 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585493+0000", 2020-08-07T15:12:07.578 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7746000000000002e-05 2020-08-07T15:12:07.578 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.578 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.579 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:07.580 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585520+0000", 2020-08-07T15:12:07.580 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1099999999999996e-06 2020-08-07T15:12:07.584 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.585 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.585 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:07.585 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585525+0000", 2020-08-07T15:12:07.586 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5359999999999998e-06 2020-08-07T15:12:07.586 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.587 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.587 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:07.588 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585534+0000", 2020-08-07T15:12:07.588 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5040000000000002e-06 2020-08-07T15:12:07.588 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.589 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:07.590 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585540+0000", 2020-08-07T15:12:07.590 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3548999999999998e-05 2020-08-07T15:12:07.591 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.591 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.591 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.592 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585593+0000", 2020-08-07T15:12:07.592 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2014000000000001e-05 2020-08-07T15:12:07.592 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.593 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.593 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:07.596 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585605+0000", 2020-08-07T15:12:07.596 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.64901847700000004 2020-08-07T15:12:07.598 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.598 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.598 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.599 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234624+0000", 2020-08-07T15:12:07.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000176329 2020-08-07T15:12:07.600 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.601 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:07.601 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234800+0000", 2020-08-07T15:12:07.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00025457300000000002 2020-08-07T15:12:07.602 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.603 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:07.603 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.235055+0000", 2020-08-07T15:12:07.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0055364669999999998 2020-08-07T15:12:07.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.618 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:07.619 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240591+0000", 2020-08-07T15:12:07.619 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3700000000000001e-07 2020-08-07T15:12:07.620 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.620 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.620 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:07.621 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240591+0000", 2020-08-07T15:12:07.621 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8479999999999997e-05 2020-08-07T15:12:07.621 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.622 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.622 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:07.622 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240640+0000", 2020-08-07T15:12:07.623 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00285564 2020-08-07T15:12:07.623 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.623 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.624 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:07.624 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243496+0000", 2020-08-07T15:12:07.624 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0073477400000000002 2020-08-07T15:12:07.625 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.627 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:07.628 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.250843+0000", 2020-08-07T15:12:07.628 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7300000000000005e-07 2020-08-07T15:12:07.629 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.629 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.629 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:07.630 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.250844+0000", 2020-08-07T15:12:07.630 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014307799999999999 2020-08-07T15:12:07.631 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.631 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.631 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:07.632 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.250987+0000", 2020-08-07T15:12:07.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66549445299999999 2020-08-07T15:12:07.639 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:07.639 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:07.640 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:07.640 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.640 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.641 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103152 5.7 5:e865d79a:::benchmark_data_smithi074_53697_object12893:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e211)", 2020-08-07T15:12:07.641 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.399967+0000", 2020-08-07T15:12:07.641 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 35.396569124999999, 2020-08-07T15:12:07.642 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81348256399999996, 2020-08-07T15:12:07.642 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:07.642 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:07.643 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:07.643 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:07.643 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:07.644 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103152 2020-08-07T15:12:07.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.644 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:07.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.647 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:07.648 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.399967+0000", 2020-08-07T15:12:07.648 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:07.650 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.650 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.650 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:07.651 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.399967+0000", 2020-08-07T15:12:07.651 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017528899999999999 2020-08-07T15:12:07.651 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.652 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.652 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:07.652 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400142+0000", 2020-08-07T15:12:07.653 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8239999999999996e-06 2020-08-07T15:12:07.653 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.653 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.654 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:07.654 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400148+0000", 2020-08-07T15:12:07.654 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0251000000000001e-05 2020-08-07T15:12:07.655 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.655 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.656 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:07.656 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400158+0000", 2020-08-07T15:12:07.662 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1579999999999998e-06 2020-08-07T15:12:07.662 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.663 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.663 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:07.663 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400162+0000", 2020-08-07T15:12:07.663 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00078404699999999996 2020-08-07T15:12:07.664 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.665 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.665 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400946+0000", 2020-08-07T15:12:07.665 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010605800000000001 2020-08-07T15:12:07.666 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.666 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.666 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:07.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.401052+0000", 2020-08-07T15:12:07.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020987499999999999 2020-08-07T15:12:07.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:07.668 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.401262+0000", 2020-08-07T15:12:07.671 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023707820000000001 2020-08-07T15:12:07.672 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.672 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.672 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:07.673 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.403633+0000", 2020-08-07T15:12:07.673 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.023e-06 2020-08-07T15:12:07.674 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.675 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.675 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:07.675 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.403634+0000", 2020-08-07T15:12:07.676 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3560999999999995e-05 2020-08-07T15:12:07.676 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.676 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:07.677 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.403717+0000", 2020-08-07T15:12:07.678 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011669689999999999 2020-08-07T15:12:07.678 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.679 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:07.682 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.404884+0000", 2020-08-07T15:12:07.682 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033332850000000001 2020-08-07T15:12:07.683 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.683 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.684 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.684 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.408218+0000", 2020-08-07T15:12:07.684 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0268000000000001e-05 2020-08-07T15:12:07.685 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.685 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.685 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:07.687 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.408228+0000", 2020-08-07T15:12:07.687 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80508180500000004 2020-08-07T15:12:07.687 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.688 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.688 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.688 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213310+0000", 2020-08-07T15:12:07.689 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000139369 2020-08-07T15:12:07.689 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.690 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.690 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:07.694 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213449+0000", 2020-08-07T15:12:07.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81348256399999996 2020-08-07T15:12:07.695 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:07.695 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:07.696 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:07.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.697 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103265 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:12:07.697 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478593+0000", 2020-08-07T15:12:07.697 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 35.317943069000002, 2020-08-07T15:12:07.698 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74922190499999997, 2020-08-07T15:12:07.698 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:07.698 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:07.699 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:07.699 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:07.699 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:07.700 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103265 2020-08-07T15:12:07.700 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.700 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:07.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.701 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:07.706 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478593+0000", 2020-08-07T15:12:07.706 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:07.707 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.708 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:07.708 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478593+0000", 2020-08-07T15:12:07.709 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9968000000000001e-05 2020-08-07T15:12:07.709 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.709 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.710 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:07.710 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478633+0000", 2020-08-07T15:12:07.710 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1420000000000003e-06 2020-08-07T15:12:07.711 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.711 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:07.712 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478637+0000", 2020-08-07T15:12:07.712 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2919999999999996e-06 2020-08-07T15:12:07.712 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.713 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.717 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:07.718 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478645+0000", 2020-08-07T15:12:07.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3699999999999997e-06 2020-08-07T15:12:07.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:07.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478649+0000", 2020-08-07T15:12:07.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1440000000000002e-05 2020-08-07T15:12:07.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478701+0000", 2020-08-07T15:12:07.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0831e-05 2020-08-07T15:12:07.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.721 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:07.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478712+0000", 2020-08-07T15:12:07.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73476696699999999 2020-08-07T15:12:07.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.722 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.722 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213479+0000", 2020-08-07T15:12:07.722 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000167026 2020-08-07T15:12:07.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.723 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:07.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213646+0000", 2020-08-07T15:12:07.727 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000179557 2020-08-07T15:12:07.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:07.728 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213825+0000", 2020-08-07T15:12:07.728 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041300360000000001 2020-08-07T15:12:07.729 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.729 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.729 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:07.729 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217955+0000", 2020-08-07T15:12:07.730 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6840000000000001e-06 2020-08-07T15:12:07.730 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.730 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.730 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:07.730 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217957+0000", 2020-08-07T15:12:07.730 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4758e-05 2020-08-07T15:12:07.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.731 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:07.731 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218012+0000", 2020-08-07T15:12:07.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0095519990000000003 2020-08-07T15:12:07.732 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.732 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:07.732 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227564+0000", 2020-08-07T15:12:07.732 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9956e-05 2020-08-07T15:12:07.733 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.733 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:07.733 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227624+0000", 2020-08-07T15:12:07.733 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3790000000000001e-06 2020-08-07T15:12:07.734 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.734 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.734 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:07.734 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227625+0000", 2020-08-07T15:12:07.734 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0001895 2020-08-07T15:12:07.735 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.735 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.735 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:07.735 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227815+0000", 2020-08-07T15:12:07.735 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74922190499999997 2020-08-07T15:12:07.738 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:07.738 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:07.739 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:07.739 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.739 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.739 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103266 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:12:07.740 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478693+0000", 2020-08-07T15:12:07.740 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 35.317842728000002, 2020-08-07T15:12:07.740 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74942137399999997, 2020-08-07T15:12:07.740 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:07.740 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:07.740 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:07.741 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:07.741 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:07.741 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103266 2020-08-07T15:12:07.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.741 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:07.742 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.742 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:07.742 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478693+0000", 2020-08-07T15:12:07.742 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:07.742 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.743 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:07.743 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478693+0000", 2020-08-07T15:12:07.743 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5658000000000002e-05 2020-08-07T15:12:07.743 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:07.744 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478739+0000", 2020-08-07T15:12:07.744 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.113e-06 2020-08-07T15:12:07.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.745 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:07.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478744+0000", 2020-08-07T15:12:07.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9690000000000006e-06 2020-08-07T15:12:07.745 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.746 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:07.746 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478754+0000", 2020-08-07T15:12:07.746 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1849999999999997e-06 2020-08-07T15:12:07.749 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.749 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.749 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:07.749 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478758+0000", 2020-08-07T15:12:07.750 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7024999999999998e-05 2020-08-07T15:12:07.750 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.750 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.750 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.750 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478805+0000", 2020-08-07T15:12:07.751 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1357e-05 2020-08-07T15:12:07.751 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.751 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.751 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:07.751 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478816+0000", 2020-08-07T15:12:07.752 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73554277499999998 2020-08-07T15:12:07.752 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.752 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.752 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.214359+0000", 2020-08-07T15:12:07.752 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000129145 2020-08-07T15:12:07.753 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.753 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.753 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:07.753 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.214488+0000", 2020-08-07T15:12:07.753 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00027129999999999998 2020-08-07T15:12:07.754 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.754 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.754 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:07.754 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.214760+0000", 2020-08-07T15:12:07.754 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032631079999999998 2020-08-07T15:12:07.755 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.755 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:07.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218023+0000", 2020-08-07T15:12:07.755 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1000000000000001e-06 2020-08-07T15:12:07.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:07.756 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218024+0000", 2020-08-07T15:12:07.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9085000000000006e-05 2020-08-07T15:12:07.757 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.760 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:07.771 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218103+0000", 2020-08-07T15:12:07.771 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.009779088 2020-08-07T15:12:07.771 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.771 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.772 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:07.772 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227882+0000", 2020-08-07T15:12:07.772 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1989999999999994e-05 2020-08-07T15:12:07.772 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.773 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.773 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:07.773 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227944+0000", 2020-08-07T15:12:07.773 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8299999999999996e-07 2020-08-07T15:12:07.774 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.774 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.774 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:07.774 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227945+0000", 2020-08-07T15:12:07.774 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000169693 2020-08-07T15:12:07.775 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.775 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:07.775 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228114+0000", 2020-08-07T15:12:07.775 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74942137399999997 2020-08-07T15:12:07.776 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:07.776 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:07.776 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:07.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.777 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103267 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:12:07.777 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478794+0000", 2020-08-07T15:12:07.777 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 35.317741239999997, 2020-08-07T15:12:07.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74960417300000004, 2020-08-07T15:12:07.778 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:07.779 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:07.781 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:07.781 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:07.781 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:07.782 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103267 2020-08-07T15:12:07.782 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.782 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:07.782 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:07.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478794+0000", 2020-08-07T15:12:07.785 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:07.785 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:07.786 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478794+0000", 2020-08-07T15:12:07.786 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6646e-05 2020-08-07T15:12:07.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:07.787 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478841+0000", 2020-08-07T15:12:07.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:12:07.787 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.793 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.794 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:07.794 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478846+0000", 2020-08-07T15:12:07.794 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0171e-05 2020-08-07T15:12:07.794 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.795 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.795 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:07.795 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478856+0000", 2020-08-07T15:12:07.795 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5929999999999999e-06 2020-08-07T15:12:07.795 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.796 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:07.796 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478861+0000", 2020-08-07T15:12:07.796 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2032e-05 2020-08-07T15:12:07.796 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.797 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.797 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.797 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478903+0000", 2020-08-07T15:12:07.797 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0312000000000001e-05 2020-08-07T15:12:07.797 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.798 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.798 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:07.798 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478913+0000", 2020-08-07T15:12:07.799 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73634238600000002 2020-08-07T15:12:07.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.215256+0000", 2020-08-07T15:12:07.799 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000124933 2020-08-07T15:12:07.801 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.801 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.801 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:07.802 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.215380+0000", 2020-08-07T15:12:07.802 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024624 2020-08-07T15:12:07.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:07.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.215627+0000", 2020-08-07T15:12:07.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0040879540000000004 2020-08-07T15:12:07.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:07.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219715+0000", 2020-08-07T15:12:07.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9230000000000001e-06 2020-08-07T15:12:07.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:07.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219717+0000", 2020-08-07T15:12:07.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8881999999999995e-05 2020-08-07T15:12:07.806 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:07.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219785+0000", 2020-08-07T15:12:07.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0083925119999999995 2020-08-07T15:12:07.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:07.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228178+0000", 2020-08-07T15:12:07.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2254000000000004e-05 2020-08-07T15:12:07.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.808 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:07.809 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228240+0000", 2020-08-07T15:12:07.809 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4499999999999996e-07 2020-08-07T15:12:07.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.809 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:07.821 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228241+0000", 2020-08-07T15:12:07.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000157656 2020-08-07T15:12:07.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.822 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.822 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:07.822 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228399+0000", 2020-08-07T15:12:07.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74960417300000004 2020-08-07T15:12:07.823 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:07.823 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:07.823 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:07.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.824 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103268 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:12:07.825 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478898+0000", 2020-08-07T15:12:07.825 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 35.317637523000002, 2020-08-07T15:12:07.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74981275700000005, 2020-08-07T15:12:07.825 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:07.825 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:07.826 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:07.826 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:07.826 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:07.826 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103268 2020-08-07T15:12:07.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.827 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:07.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.827 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:07.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478898+0000", 2020-08-07T15:12:07.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:07.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:07.828 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478898+0000", 2020-08-07T15:12:07.829 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2558999999999997e-05 2020-08-07T15:12:07.829 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.829 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.831 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:07.831 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478941+0000", 2020-08-07T15:12:07.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6469999999999997e-06 2020-08-07T15:12:07.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.832 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:07.832 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478945+0000", 2020-08-07T15:12:07.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3109999999999995e-06 2020-08-07T15:12:07.833 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:07.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478955+0000", 2020-08-07T15:12:07.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.799e-06 2020-08-07T15:12:07.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.836 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:07.836 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478958+0000", 2020-08-07T15:12:07.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5682000000000002e-05 2020-08-07T15:12:07.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:12:07.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3260000000000002e-06 2020-08-07T15:12:07.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.838 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:07.838 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479004+0000", 2020-08-07T15:12:07.838 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73703730499999998 2020-08-07T15:12:07.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.839 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216041+0000", 2020-08-07T15:12:07.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4445999999999998e-05 2020-08-07T15:12:07.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.840 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.844 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:07.844 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216115+0000", 2020-08-07T15:12:07.845 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013977 2020-08-07T15:12:07.845 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:07.845 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216255+0000", 2020-08-07T15:12:07.846 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0051267359999999998 2020-08-07T15:12:07.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.846 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.846 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:07.846 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221382+0000", 2020-08-07T15:12:07.847 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0310000000000001e-06 2020-08-07T15:12:07.847 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.847 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.847 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:07.847 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221384+0000", 2020-08-07T15:12:07.848 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8097999999999998e-05 2020-08-07T15:12:07.848 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.848 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.848 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:07.848 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221452+0000", 2020-08-07T15:12:07.849 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0070259249999999997 2020-08-07T15:12:07.849 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.849 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.849 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:07.849 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228478+0000", 2020-08-07T15:12:07.874 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7479999999999999e-05 2020-08-07T15:12:07.874 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.874 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:07.875 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228535+0000", 2020-08-07T15:12:07.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4399999999999999e-07 2020-08-07T15:12:07.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.876 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:07.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228536+0000", 2020-08-07T15:12:07.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017489799999999999 2020-08-07T15:12:07.876 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.876 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:07.877 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228711+0000", 2020-08-07T15:12:07.877 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74981275700000005 2020-08-07T15:12:07.877 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:07.877 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:07.877 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:07.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.891 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.891 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103269 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:12:07.891 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:12:07.891 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 35.317541415000001, 2020-08-07T15:12:07.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74992452799999998, 2020-08-07T15:12:07.892 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:07.892 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:07.892 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:07.892 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:07.893 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:07.893 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103269 2020-08-07T15:12:07.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.893 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:07.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:07.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:12:07.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:07.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:07.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:12:07.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5529999999999999e-05 2020-08-07T15:12:07.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:07.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479040+0000", 2020-08-07T15:12:07.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6550000000000003e-06 2020-08-07T15:12:07.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:07.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479044+0000", 2020-08-07T15:12:07.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4285e-05 2020-08-07T15:12:07.925 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:07.952 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479059+0000", 2020-08-07T15:12:07.953 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2050000000000001e-06 2020-08-07T15:12:07.953 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:07.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479064+0000", 2020-08-07T15:12:07.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0952999999999999e-05 2020-08-07T15:12:07.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479105+0000", 2020-08-07T15:12:07.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3559999999999998e-06 2020-08-07T15:12:07.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:07.966 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479111+0000", 2020-08-07T15:12:07.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.737508104 2020-08-07T15:12:07.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:07.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216619+0000", 2020-08-07T15:12:07.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000126367 2020-08-07T15:12:07.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:07.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216746+0000", 2020-08-07T15:12:07.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014069899999999999 2020-08-07T15:12:07.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:07.969 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:07.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216886+0000", 2020-08-07T15:12:07.969 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045753160000000003 2020-08-07T15:12:07.969 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:07.969 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:08.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221462+0000", 2020-08-07T15:12:08.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.142e-06 2020-08-07T15:12:08.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:08.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221463+0000", 2020-08-07T15:12:08.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7718e-05 2020-08-07T15:12:08.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:08.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221501+0000", 2020-08-07T15:12:08.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0072733210000000001 2020-08-07T15:12:08.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:08.026 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228774+0000", 2020-08-07T15:12:08.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3828e-05 2020-08-07T15:12:08.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:08.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228828+0000", 2020-08-07T15:12:08.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9800000000000003e-07 2020-08-07T15:12:08.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:08.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228828+0000", 2020-08-07T15:12:08.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0451000000000005e-05 2020-08-07T15:12:08.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:08.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228919+0000", 2020-08-07T15:12:08.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74992452799999998 2020-08-07T15:12:08.029 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:08.029 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:08.075 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:08.082 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.083 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103270 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:12:08.083 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.479168+0000", 2020-08-07T15:12:08.083 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 35.317367378, 2020-08-07T15:12:08.084 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75043371699999994, 2020-08-07T15:12:08.084 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:08.084 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:08.085 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:08.085 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:08.121 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:08.122 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103270 2020-08-07T15:12:08.122 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.122 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:08.123 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.123 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:08.123 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479168+0000", 2020-08-07T15:12:08.124 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:08.124 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.124 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.125 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:08.125 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479168+0000", 2020-08-07T15:12:08.125 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1557000000000002e-05 2020-08-07T15:12:08.149 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.149 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.150 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:08.150 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479210+0000", 2020-08-07T15:12:08.150 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9840000000000001e-06 2020-08-07T15:12:08.150 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.150 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.151 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:08.151 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479215+0000", 2020-08-07T15:12:08.151 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9539999999999999e-06 2020-08-07T15:12:08.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.173 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:08.184 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479225+0000", 2020-08-07T15:12:08.185 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.942e-06 2020-08-07T15:12:08.185 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.185 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.194 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:08.205 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479231+0000", 2020-08-07T15:12:08.205 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2493000000000002e-05 2020-08-07T15:12:08.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.205 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.206 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:08.206 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479263+0000", 2020-08-07T15:12:08.207 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6640000000000003e-06 2020-08-07T15:12:08.207 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.207 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.207 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:08.207 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479269+0000", 2020-08-07T15:12:08.208 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73802486499999997 2020-08-07T15:12:08.208 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.208 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.208 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:08.208 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217294+0000", 2020-08-07T15:12:08.209 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8640000000000006e-05 2020-08-07T15:12:08.209 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.209 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.209 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:08.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217372+0000", 2020-08-07T15:12:08.210 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000140338 2020-08-07T15:12:08.210 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.210 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.210 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:08.210 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217513+0000", 2020-08-07T15:12:08.211 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039952119999999997 2020-08-07T15:12:08.211 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.211 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.211 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:08.211 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221508+0000", 2020-08-07T15:12:08.212 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8899999999999999e-07 2020-08-07T15:12:08.212 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.212 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.212 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:08.212 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221509+0000", 2020-08-07T15:12:08.212 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8108999999999997e-05 2020-08-07T15:12:08.213 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.213 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.213 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:08.213 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221547+0000", 2020-08-07T15:12:08.213 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074232029999999997 2020-08-07T15:12:08.228 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.229 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.229 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:08.229 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228970+0000", 2020-08-07T15:12:08.230 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00046710099999999999 2020-08-07T15:12:08.230 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.230 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.234 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:08.234 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229437+0000", 2020-08-07T15:12:08.234 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5400000000000001e-07 2020-08-07T15:12:08.235 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.235 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.235 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:08.235 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229438+0000", 2020-08-07T15:12:08.235 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016441200000000001 2020-08-07T15:12:08.236 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.236 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.236 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:08.236 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229602+0000", 2020-08-07T15:12:08.236 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75043371699999994 2020-08-07T15:12:08.237 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:08.237 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:08.237 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:08.237 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.237 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.242 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103271 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:12:08.242 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.479356+0000", 2020-08-07T15:12:08.242 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 35.317179410000001, 2020-08-07T15:12:08.242 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75047653800000003, 2020-08-07T15:12:08.243 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:08.243 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:08.243 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:08.243 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:08.243 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:08.244 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103271 2020-08-07T15:12:08.244 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.244 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:08.244 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.244 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:08.244 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479356+0000", 2020-08-07T15:12:08.245 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:08.245 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.245 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.245 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:08.245 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479356+0000", 2020-08-07T15:12:08.246 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2416000000000002e-05 2020-08-07T15:12:08.246 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.246 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.246 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:08.247 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479399+0000", 2020-08-07T15:12:08.247 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0819999999999998e-06 2020-08-07T15:12:08.247 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.247 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.248 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:08.248 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479404+0000", 2020-08-07T15:12:08.249 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2486999999999999e-05 2020-08-07T15:12:08.250 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.250 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.250 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:08.250 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479416+0000", 2020-08-07T15:12:08.251 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3810000000000001e-06 2020-08-07T15:12:08.251 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.251 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.251 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:08.251 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479421+0000", 2020-08-07T15:12:08.251 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2311999999999998e-05 2020-08-07T15:12:08.252 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.252 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.252 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:08.252 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479453+0000", 2020-08-07T15:12:08.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7690000000000002e-06 2020-08-07T15:12:08.253 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.253 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:08.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479459+0000", 2020-08-07T15:12:08.253 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73869566499999995 2020-08-07T15:12:08.254 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.254 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.254 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:08.254 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218154+0000", 2020-08-07T15:12:08.254 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000124535 2020-08-07T15:12:08.255 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.255 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.255 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:08.255 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218279+0000", 2020-08-07T15:12:08.255 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000223441 2020-08-07T15:12:08.256 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.256 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.256 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:08.256 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218502+0000", 2020-08-07T15:12:08.256 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003054761 2020-08-07T15:12:08.257 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.257 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.257 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:08.257 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221557+0000", 2020-08-07T15:12:08.257 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0449999999999999e-06 2020-08-07T15:12:08.258 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.258 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.258 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:08.258 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221558+0000", 2020-08-07T15:12:08.258 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3655e-05 2020-08-07T15:12:08.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.259 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.260 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:08.260 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221622+0000", 2020-08-07T15:12:08.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.007661434 2020-08-07T15:12:08.261 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.261 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.261 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:08.261 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229283+0000", 2020-08-07T15:12:08.261 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00036023099999999997 2020-08-07T15:12:08.262 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.262 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.262 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:08.262 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229644+0000", 2020-08-07T15:12:08.262 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.63e-07 2020-08-07T15:12:08.263 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.263 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.263 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:08.264 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229644+0000", 2020-08-07T15:12:08.264 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018886099999999999 2020-08-07T15:12:08.264 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.265 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.265 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:08.265 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229833+0000", 2020-08-07T15:12:08.266 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75047653800000003 2020-08-07T15:12:08.266 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:08.266 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:08.267 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:08.267 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.267 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.268 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103272 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:12:08.268 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.481658+0000", 2020-08-07T15:12:08.268 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 35.314877987000003, 2020-08-07T15:12:08.269 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74920410900000001, 2020-08-07T15:12:08.278 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:08.279 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:08.279 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:08.280 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:08.289 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:08.290 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103272 2020-08-07T15:12:08.290 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.290 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:08.291 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.300 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:08.300 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481658+0000", 2020-08-07T15:12:08.301 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:08.301 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.301 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.301 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:08.302 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481658+0000", 2020-08-07T15:12:08.302 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8787e-05 2020-08-07T15:12:08.302 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.302 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.303 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:08.303 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481697+0000", 2020-08-07T15:12:08.303 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0559999999999998e-06 2020-08-07T15:12:08.303 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.303 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.304 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:08.304 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481701+0000", 2020-08-07T15:12:08.304 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1329999999999999e-06 2020-08-07T15:12:08.304 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.304 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.305 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:08.305 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481709+0000", 2020-08-07T15:12:08.305 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.189e-06 2020-08-07T15:12:08.305 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.317 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.319 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:08.319 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481713+0000", 2020-08-07T15:12:08.319 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6025999999999997e-05 2020-08-07T15:12:08.319 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.320 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.320 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:08.320 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481749+0000", 2020-08-07T15:12:08.320 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5619999999999998e-06 2020-08-07T15:12:08.321 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.346 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.346 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:08.347 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481756+0000", 2020-08-07T15:12:08.347 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73729155000000002 2020-08-07T15:12:08.347 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.348 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:08.348 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219048+0000", 2020-08-07T15:12:08.348 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000120387 2020-08-07T15:12:08.348 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.348 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.372 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:08.375 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219168+0000", 2020-08-07T15:12:08.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017865599999999999 2020-08-07T15:12:08.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.390 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:08.391 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219347+0000", 2020-08-07T15:12:08.391 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039049169999999999 2020-08-07T15:12:08.391 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.391 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.404 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:08.404 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.223252+0000", 2020-08-07T15:12:08.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.412e-06 2020-08-07T15:12:08.404 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.405 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:08.405 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.223253+0000", 2020-08-07T15:12:08.405 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7521999999999997e-05 2020-08-07T15:12:08.405 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.406 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.406 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:08.406 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.223301+0000", 2020-08-07T15:12:08.406 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00603478 2020-08-07T15:12:08.407 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.407 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.407 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:08.407 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229336+0000", 2020-08-07T15:12:08.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013309579999999999 2020-08-07T15:12:08.408 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.408 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.408 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:08.408 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.230667+0000", 2020-08-07T15:12:08.408 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7400000000000003e-07 2020-08-07T15:12:08.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.409 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.409 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:08.409 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.230667+0000", 2020-08-07T15:12:08.409 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019450000000000001 2020-08-07T15:12:08.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:08.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:08.460 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:08.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.230862+0000", 2020-08-07T15:12:08.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74920410900000001 2020-08-07T15:12:08.460 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:08.461 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:08.461 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:08.461 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:08.461 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:08.461 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:08.464 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:12:08.465 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:12:08.482 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:12:08.482 INFO:teuthology.orchestra.run.smithi074.stderr:pool 'unique_pool_2' removed 2020-08-07T15:12:08.483 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2020-08-07T15:12:08.484 INFO:tasks.radosbench:Beginning radosbench... 2020-08-07T15:12:08.484 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_3 2020-08-07T15:12:08.485 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create unique_pool_3 16 2020-08-07T15:12:08.487 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:08.464+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:08.488 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:12:08.567 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:08.566+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:08.655 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:08.656 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:12:08.656 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:12:08.656 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:08.662 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:08.662 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:12:08.662 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:12:08.663 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:08.671 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:12:08.672 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:08.669+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:08.694 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:12:08.728 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:12:08.754 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:12:08.772 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:08.770+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:08.778 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:12:08.802 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:12:08.819 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:08.819 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:08.895 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:12:08.896 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:12:08.906 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:12:08.906 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:12:08.906 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:12:08.906 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:12:08.907 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:12:08.907 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:12:08.908 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:12:08.908 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:12:08.908 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:12:08.908 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:12:08.940 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:08.934+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:09.023 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:09.021+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:09.058 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:09.059 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:09.067 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:12:09.125 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:09.122+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:09.186 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:09.186 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:09.198 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:12:09.244 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:09.238+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:09.330 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:09.330+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:09.356 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:09.356 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:09.368 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:12:09.397 INFO:teuthology.orchestra.run.smithi074.stderr:pool 'unique_pool_3' created 2020-08-07T15:12:09.408 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool application enable unique_pool_3 rados --yes-i-really-mean-it || true 2020-08-07T15:12:09.439 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:09.436+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:09.522 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:09.522 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:09.531 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:12:09.546 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:09.545+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:09.660 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:09.647+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:09.684 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:09.685 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:09.699 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:12:09.787 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:09.783+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:09.854 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:09.853+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:09.862 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:09.862 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:09.864 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:12:09.956 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:09.955+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:10.021 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:10.021 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:10.032 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:12:10.090 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:10.088+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:10.160 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:10.161 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:10.169 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:12:10.176 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:10.175+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:10.277 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:10.276+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:10.328 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:10.328 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:10.344 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:12:10.381 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:10.380+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:10.404 INFO:teuthology.orchestra.run.smithi074.stderr:enabled application 'rados' on pool 'unique_pool_3' 2020-08-07T15:12:10.481 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:10.481+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:10.489 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:12:10.507 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:10.508 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:10.521 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:12:10.583 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:10.581+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:10.646 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:10.646 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:10.653 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:12:10.700 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:10.693+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:10.785 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:10.784+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:10.823 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:10.823 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:10.825 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:12:10.894 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:10.886+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:10.914 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:12:10.943 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:10.943 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:10.955 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:12:10.998 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:10.996+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:11.060 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:12:11.087 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:12:11.091 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:11.089+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:11.110 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:12:11.118 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:11.118 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:11.126 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:12:11.133 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:12:11.157 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:12:11.187 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:12:11.201 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:11.190+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:11.213 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:12:11.231 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:12:11.231 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:12:11.232 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:12:11.232 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:12:11.232 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:12:11.232 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:12:11.232 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:12:11.233 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:12:11.233 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:12:11.245 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:11.246 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:12:11.246 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:12:11.246 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:12:11.253 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:11.262 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:12:11.301 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:11.299+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:11.382 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:11.382 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:12:11.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:12:11.383 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:12:11.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.383 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15576 3.e 3:727724a5:::benchmark_data_smithi074_35447_object1946:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:12:11.386 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:12:11.386 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 324.71325992499999, 2020-08-07T15:12:11.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999, 2020-08-07T15:12:11.387 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:11.388 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:11.389 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:11.389 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:12:11.431 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:12:11.432 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15576 2020-08-07T15:12:11.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.432 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:11.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:11.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:12:11.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:11.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:11.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:12:11.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7156e-05 2020-08-07T15:12:11.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:11.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666682+0000", 2020-08-07T15:12:11.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0209999999999999e-06 2020-08-07T15:12:11.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:11.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666687+0000", 2020-08-07T15:12:11.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0730999999999999e-05 2020-08-07T15:12:11.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:11.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666698+0000", 2020-08-07T15:12:11.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2020-08-07T15:12:11.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:11.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666702+0000", 2020-08-07T15:12:11.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023978459999999999 2020-08-07T15:12:11.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:11.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669100+0000", 2020-08-07T15:12:11.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9664000000000001e-05 2020-08-07T15:12:11.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:11.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669190+0000", 2020-08-07T15:12:11.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015957100000000001 2020-08-07T15:12:11.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:12:11.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669350+0000", 2020-08-07T15:12:11.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016470149999999999 2020-08-07T15:12:11.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:11.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:12:11.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1299999999999999e-07 2020-08-07T15:12:11.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:11.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:12:11.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5269000000000002e-05 2020-08-07T15:12:11.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:11.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.671022+0000", 2020-08-07T15:12:11.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031414400000000001 2020-08-07T15:12:11.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:11.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.674163+0000", 2020-08-07T15:12:11.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45641417400000001 2020-08-07T15:12:11.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:11.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:12:11.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0399999999999995e-07 2020-08-07T15:12:11.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:11.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:12:11.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3343999999999997e-05 2020-08-07T15:12:11.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:11.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130622+0000", 2020-08-07T15:12:11.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999 2020-08-07T15:12:11.469 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.481 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:11.482 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.482 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106248 3.f 3:f950fd9d:::benchmark_data_smithi074_35447_object13280:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:12:11.483 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:12:11.483 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 264.019730385, 2020-08-07T15:12:11.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002, 2020-08-07T15:12:11.483 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:11.484 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:11.484 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:11.484 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:12:11.484 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:12:11.484 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106248 2020-08-07T15:12:11.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.485 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:11.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:11.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:12:11.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:11.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:11.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:12:11.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7184e-05 2020-08-07T15:12:11.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:11.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360222+0000", 2020-08-07T15:12:11.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0906e-05 2020-08-07T15:12:11.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:11.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360233+0000", 2020-08-07T15:12:11.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2972999999999999e-05 2020-08-07T15:12:11.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:11.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360246+0000", 2020-08-07T15:12:11.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0114e-05 2020-08-07T15:12:11.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:11.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360256+0000", 2020-08-07T15:12:11.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1303999999999997e-05 2020-08-07T15:12:11.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:11.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360287+0000", 2020-08-07T15:12:11.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3798000000000001e-05 2020-08-07T15:12:11.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:11.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360351+0000", 2020-08-07T15:12:11.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111308 2020-08-07T15:12:11.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:12:11.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360462+0000", 2020-08-07T15:12:11.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003480674 2020-08-07T15:12:11.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:11.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:12:11.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0900000000000001e-07 2020-08-07T15:12:11.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:11.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:12:11.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8899999999999994e-05 2020-08-07T15:12:11.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:11.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.364012+0000", 2020-08-07T15:12:11.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35291637199999998 2020-08-07T15:12:11.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:11.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716929+0000", 2020-08-07T15:12:11.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.076244233999999994 2020-08-07T15:12:11.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:11.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793173+0000", 2020-08-07T15:12:11.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6109999999999999e-06 2020-08-07T15:12:11.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:11.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793175+0000", 2020-08-07T15:12:11.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7492999999999999e-05 2020-08-07T15:12:11.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:11.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793242+0000", 2020-08-07T15:12:11.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002 2020-08-07T15:12:11.532 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.532 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:11.533 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.533 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106248 3.f e59/20 ondisk, result = 0)", 2020-08-07T15:12:11.533 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:12:11.534 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 264.01446709999999, 2020-08-07T15:12:11.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001, 2020-08-07T15:12:11.534 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:11.534 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:11.534 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:11.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:11.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:12:11.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:11.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:11.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:12:11.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7518999999999999e-05 2020-08-07T15:12:11.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:11.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365445+0000", 2020-08-07T15:12:11.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-06 2020-08-07T15:12:11.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:11.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365449+0000", 2020-08-07T15:12:11.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42764147600000002 2020-08-07T15:12:11.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:11.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793091+0000", 2020-08-07T15:12:11.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6500000000000002e-06 2020-08-07T15:12:11.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:11.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793099+0000", 2020-08-07T15:12:11.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3664e-05 2020-08-07T15:12:11.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:11.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793153+0000", 2020-08-07T15:12:11.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1078e-05 2020-08-07T15:12:11.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:11.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793164+0000", 2020-08-07T15:12:11.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3404000000000003e-05 2020-08-07T15:12:11.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:11.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793248+0000", 2020-08-07T15:12:11.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001 2020-08-07T15:12:11.547 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.547 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:11.547 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.548 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:107952 3.d e61/35)", 2020-08-07T15:12:11.548 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:12:11.548 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 261.05353739200001, 2020-08-07T15:12:11.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001, 2020-08-07T15:12:11.548 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:11.549 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:11.549 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:11.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:11.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:12:11.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:11.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:11.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:12:11.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8826000000000003e-05 2020-08-07T15:12:11.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:11.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326406+0000", 2020-08-07T15:12:11.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9199999999999998e-06 2020-08-07T15:12:11.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:11.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326413+0000", 2020-08-07T15:12:11.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75082853500000002 2020-08-07T15:12:11.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:11.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077242+0000", 2020-08-07T15:12:11.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1479999999999999e-06 2020-08-07T15:12:11.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:11.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077246+0000", 2020-08-07T15:12:11.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3331999999999998e-05 2020-08-07T15:12:11.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:11.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077289+0000", 2020-08-07T15:12:11.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8977000000000001e-05 2020-08-07T15:12:11.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:11.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077308+0000", 2020-08-07T15:12:11.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001 2020-08-07T15:12:11.559 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.559 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:11.560 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.560 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31696 4.0 4:0335eed9:::benchmark_data_smithi074_44586_object3961:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e100)", 2020-08-07T15:12:11.560 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:12:11.561 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 203.27562983300001, 2020-08-07T15:12:11.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237, 2020-08-07T15:12:11.561 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:11.561 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:11.561 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:11.562 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:11.562 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:11.562 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31696 2020-08-07T15:12:11.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.562 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:11.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:11.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:12:11.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:11.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:11.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:12:11.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7654e-05 2020-08-07T15:12:11.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:11.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104293+0000", 2020-08-07T15:12:11.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.061e-06 2020-08-07T15:12:11.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:11.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104296+0000", 2020-08-07T15:12:11.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0859999999999998e-06 2020-08-07T15:12:11.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:11.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104302+0000", 2020-08-07T15:12:11.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8030000000000001e-06 2020-08-07T15:12:11.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:11.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104305+0000", 2020-08-07T15:12:11.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037634769999999999 2020-08-07T15:12:11.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:11.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108068+0000", 2020-08-07T15:12:11.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8814999999999998e-05 2020-08-07T15:12:11.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:11.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108137+0000", 2020-08-07T15:12:11.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111455 2020-08-07T15:12:11.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:12:11.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108248+0000", 2020-08-07T15:12:11.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028756820000000001 2020-08-07T15:12:11.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:11.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111124+0000", 2020-08-07T15:12:11.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061054499999999999 2020-08-07T15:12:11.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:11.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111735+0000", 2020-08-07T15:12:11.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.423966167 2020-08-07T15:12:11.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:11.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535701+0000", 2020-08-07T15:12:11.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4929999999999999e-06 2020-08-07T15:12:11.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:11.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535702+0000", 2020-08-07T15:12:11.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2571999999999997e-05 2020-08-07T15:12:11.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:11.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535755+0000", 2020-08-07T15:12:11.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1427000000000003e-05 2020-08-07T15:12:11.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:11.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535786+0000", 2020-08-07T15:12:11.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237 2020-08-07T15:12:11.583 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.584 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:11.584 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.585 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:31696 4.0 e100/81 ondisk, result = 0)", 2020-08-07T15:12:11.585 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:12:11.586 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 203.266015275, 2020-08-07T15:12:11.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954, 2020-08-07T15:12:11.586 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:11.586 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:11.586 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:11.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:11.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:12:11.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:11.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:11.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:12:11.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1185000000000001e-05 2020-08-07T15:12:11.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:11.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113901+0000", 2020-08-07T15:12:11.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2930000000000002e-06 2020-08-07T15:12:11.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:11.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113905+0000", 2020-08-07T15:12:11.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42168640400000001 2020-08-07T15:12:11.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:11.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535591+0000", 2020-08-07T15:12:11.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2820000000000002e-06 2020-08-07T15:12:11.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:11.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535598+0000", 2020-08-07T15:12:11.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6578999999999999e-05 2020-08-07T15:12:11.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:11.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535684+0000", 2020-08-07T15:12:11.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8449999999999992e-06 2020-08-07T15:12:11.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:11.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535693+0000", 2020-08-07T15:12:11.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7366000000000003e-05 2020-08-07T15:12:11.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:11.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535791+0000", 2020-08-07T15:12:11.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954 2020-08-07T15:12:11.614 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.614 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:11.614 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.615 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:37024 4.0 e106/81)", 2020-08-07T15:12:11.615 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:12:11.615 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 197.15122638099999, 2020-08-07T15:12:11.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999, 2020-08-07T15:12:11.616 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:11.616 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:11.616 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:11.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:11.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:12:11.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:11.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:11.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:12:11.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4926000000000001e-05 2020-08-07T15:12:11.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:11.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228683+0000", 2020-08-07T15:12:11.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1280000000000002e-06 2020-08-07T15:12:11.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:11.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228686+0000", 2020-08-07T15:12:11.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69740012100000004 2020-08-07T15:12:11.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:11.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926086+0000", 2020-08-07T15:12:11.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000004e-06 2020-08-07T15:12:11.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:11.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926090+0000", 2020-08-07T15:12:11.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113526 2020-08-07T15:12:11.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:11.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926204+0000", 2020-08-07T15:12:11.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8824000000000001e-05 2020-08-07T15:12:11.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:11.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926222+0000", 2020-08-07T15:12:11.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999 2020-08-07T15:12:11.637 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.637 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:11.637 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.638 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42503 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2615, mlcod=112'2607)", 2020-08-07T15:12:11.638 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:12:11.638 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 191.083654465, 2020-08-07T15:12:11.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998, 2020-08-07T15:12:11.648 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:11.648 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:11.648 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:11.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:11.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:12:11.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:11.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:11.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:12:11.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270999999999998e-05 2020-08-07T15:12:11.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:11.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296292+0000", 2020-08-07T15:12:11.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4729999999999997e-06 2020-08-07T15:12:11.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:11.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296298+0000", 2020-08-07T15:12:11.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.448795049 2020-08-07T15:12:11.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:11.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745093+0000", 2020-08-07T15:12:11.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0140000000000001e-06 2020-08-07T15:12:11.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:11.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745100+0000", 2020-08-07T15:12:11.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2596999999999998e-05 2020-08-07T15:12:11.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:11.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745153+0000", 2020-08-07T15:12:11.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1662000000000003e-05 2020-08-07T15:12:11.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:11.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745184+0000", 2020-08-07T15:12:11.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093174800000000004 2020-08-07T15:12:11.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:11.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746116+0000", 2020-08-07T15:12:11.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0926999999999994e-05 2020-08-07T15:12:11.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:11.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746187+0000", 2020-08-07T15:12:11.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998 2020-08-07T15:12:11.672 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.672 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:11.672 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.673 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42504 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2616, mlcod=113'2609)", 2020-08-07T15:12:11.673 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:12:11.674 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 191.08031757200001, 2020-08-07T15:12:11.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998, 2020-08-07T15:12:11.674 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:11.674 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:11.674 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:11.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:11.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:12:11.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:11.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:11.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:12:11.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2154000000000002e-05 2020-08-07T15:12:11.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:11.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299629+0000", 2020-08-07T15:12:11.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1929999999999998e-06 2020-08-07T15:12:11.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:11.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299636+0000", 2020-08-07T15:12:11.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44766315400000001 2020-08-07T15:12:11.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:11.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747299+0000", 2020-08-07T15:12:11.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5889999999999999e-06 2020-08-07T15:12:11.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:11.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747305+0000", 2020-08-07T15:12:11.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3094999999999999e-05 2020-08-07T15:12:11.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:11.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747358+0000", 2020-08-07T15:12:11.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3283e-05 2020-08-07T15:12:11.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:11.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747392+0000", 2020-08-07T15:12:11.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015319509999999999 2020-08-07T15:12:11.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:11.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748924+0000", 2020-08-07T15:12:11.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5235999999999996e-05 2020-08-07T15:12:11.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:11.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748999+0000", 2020-08-07T15:12:11.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998 2020-08-07T15:12:11.689 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.689 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:11.689 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.690 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42520 4.8 e113/81 4:13071de5:::benchmark_data_smithi074_44586_object5314:head v 113'2624, mlcod=113'2618)", 2020-08-07T15:12:11.690 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:12:11.691 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 190.614373363, 2020-08-07T15:12:11.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001, 2020-08-07T15:12:11.691 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:11.691 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:11.692 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:11.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:11.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:12:11.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:11.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:11.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:12:11.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3967999999999995e-05 2020-08-07T15:12:11.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:11.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765585+0000", 2020-08-07T15:12:11.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0949999999999998e-06 2020-08-07T15:12:11.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:11.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765593+0000", 2020-08-07T15:12:11.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52703464200000005 2020-08-07T15:12:11.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:11.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292627+0000", 2020-08-07T15:12:11.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6929999999999996e-06 2020-08-07T15:12:11.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:11.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292636+0000", 2020-08-07T15:12:11.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9292000000000001e-05 2020-08-07T15:12:11.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:11.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292695+0000", 2020-08-07T15:12:11.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5395999999999999e-05 2020-08-07T15:12:11.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:11.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292721+0000", 2020-08-07T15:12:11.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00090361399999999996 2020-08-07T15:12:11.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:11.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293624+0000", 2020-08-07T15:12:11.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1761000000000002e-05 2020-08-07T15:12:11.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:11.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293666+0000", 2020-08-07T15:12:11.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001 2020-08-07T15:12:11.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.734 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:11.735 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.735 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:19776 5.4 5:2bb0c0d9:::benchmark_data_smithi074_53697_object2471:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e173)", 2020-08-07T15:12:11.735 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:12:11.736 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 98.980180677999996, 2020-08-07T15:12:11.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.402061844, 2020-08-07T15:12:11.736 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:11.736 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:11.736 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:11.737 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:11.737 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:11.737 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 19776 2020-08-07T15:12:11.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.737 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:11.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:11.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:12:11.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:11.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:11.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:12:11.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-05 2020-08-07T15:12:11.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:11.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399752+0000", 2020-08-07T15:12:11.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8799999999999999e-06 2020-08-07T15:12:11.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:11.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399757+0000", 2020-08-07T15:12:11.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7690000000000008e-06 2020-08-07T15:12:11.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:11.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399767+0000", 2020-08-07T15:12:11.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-06 2020-08-07T15:12:11.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:11.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399771+0000", 2020-08-07T15:12:11.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003255109 2020-08-07T15:12:11.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:11.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403026+0000", 2020-08-07T15:12:11.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101044 2020-08-07T15:12:11.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:11.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403127+0000", 2020-08-07T15:12:11.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000193158 2020-08-07T15:12:11.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,5", 2020-08-07T15:12:11.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403320+0000", 2020-08-07T15:12:11.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017185530000000001 2020-08-07T15:12:11.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:11.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405038+0000", 2020-08-07T15:12:11.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6300000000000004e-07 2020-08-07T15:12:11.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:11.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405039+0000", 2020-08-07T15:12:11.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012590600000000001 2020-08-07T15:12:11.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:11.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405165+0000", 2020-08-07T15:12:11.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109757 2020-08-07T15:12:11.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:11.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405275+0000", 2020-08-07T15:12:11.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39642174699999999 2020-08-07T15:12:11.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:11.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801697+0000", 2020-08-07T15:12:11.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2799999999999996e-07 2020-08-07T15:12:11.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:11.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801697+0000", 2020-08-07T15:12:11.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8740999999999997e-05 2020-08-07T15:12:11.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:11.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801776+0000", 2020-08-07T15:12:11.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.402061844 2020-08-07T15:12:11.845 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.845 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:11.845 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.846 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:19776 5.4 e173/164 ondisk, result = 0)", 2020-08-07T15:12:11.846 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:12:11.847 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 98.974432375000006, 2020-08-07T15:12:11.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39631765200000002, 2020-08-07T15:12:11.847 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:11.847 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:11.848 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:11.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:11.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:12:11.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:11.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:11.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:12:11.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9332e-05 2020-08-07T15:12:11.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:11.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405492+0000", 2020-08-07T15:12:11.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8900000000000004e-06 2020-08-07T15:12:11.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:11.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405498+0000", 2020-08-07T15:12:11.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39566664200000001 2020-08-07T15:12:11.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:11.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801164+0000", 2020-08-07T15:12:11.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7440000000000001e-06 2020-08-07T15:12:11.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:11.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801168+0000", 2020-08-07T15:12:11.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050828199999999998 2020-08-07T15:12:11.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:11.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801676+0000", 2020-08-07T15:12:11.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1011e-05 2020-08-07T15:12:11.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:11.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801687+0000", 2020-08-07T15:12:11.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2750999999999993e-05 2020-08-07T15:12:11.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:11.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801780+0000", 2020-08-07T15:12:11.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39631765200000002 2020-08-07T15:12:11.866 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.866 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:11.866 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.867 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:85551 5.9 5:9d69b7cc:::benchmark_data_smithi074_53697_object10693:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e202)", 2020-08-07T15:12:11.867 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.807277+0000", 2020-08-07T15:12:11.868 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.572618308000003, 2020-08-07T15:12:11.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39614286500000001, 2020-08-07T15:12:11.868 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:11.868 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:11.868 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:11.869 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:11.869 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:11.869 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 85551 2020-08-07T15:12:11.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.869 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:11.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:11.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807277+0000", 2020-08-07T15:12:11.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:11.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:11.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807277+0000", 2020-08-07T15:12:11.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8867999999999999e-05 2020-08-07T15:12:11.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:11.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807305+0000", 2020-08-07T15:12:11.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.775e-06 2020-08-07T15:12:11.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:11.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807310+0000", 2020-08-07T15:12:11.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2747e-05 2020-08-07T15:12:11.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:11.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807323+0000", 2020-08-07T15:12:11.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2669999999999997e-06 2020-08-07T15:12:11.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:11.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807328+0000", 2020-08-07T15:12:11.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001048513 2020-08-07T15:12:11.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:11.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.808377+0000", 2020-08-07T15:12:11.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010425399999999999 2020-08-07T15:12:11.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:11.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.808481+0000", 2020-08-07T15:12:11.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013761900000000001 2020-08-07T15:12:11.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:12:11.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.808619+0000", 2020-08-07T15:12:11.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042907500000000003 2020-08-07T15:12:11.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:11.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.812909+0000", 2020-08-07T15:12:11.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.077e-06 2020-08-07T15:12:11.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:11.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.812910+0000", 2020-08-07T15:12:11.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8186000000000001e-05 2020-08-07T15:12:11.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:11.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.812979+0000", 2020-08-07T15:12:11.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00081526099999999996 2020-08-07T15:12:11.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:11.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813794+0000", 2020-08-07T15:12:11.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38955804599999999 2020-08-07T15:12:11.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:11.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203352+0000", 2020-08-07T15:12:11.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5239999999999999e-06 2020-08-07T15:12:11.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:11.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203353+0000", 2020-08-07T15:12:11.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5977999999999997e-05 2020-08-07T15:12:11.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:11.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203419+0000", 2020-08-07T15:12:11.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39614286500000001 2020-08-07T15:12:11.911 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.911 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:11.912 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.912 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:85552 5.9 5:9d69b7cc:::benchmark_data_smithi074_53697_object10693:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e202)", 2020-08-07T15:12:11.912 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:12:11.913 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.572377584000002, 2020-08-07T15:12:11.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54429374399999997, 2020-08-07T15:12:11.913 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:11.913 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:11.913 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:11.914 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:11.914 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:11.914 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 85552 2020-08-07T15:12:11.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.914 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:11.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:11.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:12:11.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:11.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:11.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:12:11.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2691000000000001e-05 2020-08-07T15:12:11.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:11.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807570+0000", 2020-08-07T15:12:11.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0619999999999997e-06 2020-08-07T15:12:11.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:11.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807576+0000", 2020-08-07T15:12:11.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1558e-05 2020-08-07T15:12:11.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:11.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807588+0000", 2020-08-07T15:12:11.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4600000000000002e-06 2020-08-07T15:12:11.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:11.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807593+0000", 2020-08-07T15:12:11.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022126279999999999 2020-08-07T15:12:11.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:11.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.809806+0000", 2020-08-07T15:12:11.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104868 2020-08-07T15:12:11.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:11.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.809911+0000", 2020-08-07T15:12:11.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000186019 2020-08-07T15:12:11.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:12:11.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.810097+0000", 2020-08-07T15:12:11.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037966739999999999 2020-08-07T15:12:11.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:11.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813893+0000", 2020-08-07T15:12:11.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1310000000000001e-06 2020-08-07T15:12:11.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:11.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813894+0000", 2020-08-07T15:12:11.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4704999999999999e-05 2020-08-07T15:12:11.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:11.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813949+0000", 2020-08-07T15:12:11.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015487280000000001 2020-08-07T15:12:11.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:11.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.815498+0000", 2020-08-07T15:12:11.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53621442900000005 2020-08-07T15:12:11.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:11.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351712+0000", 2020-08-07T15:12:11.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8199999999999998e-07 2020-08-07T15:12:11.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:11.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351713+0000", 2020-08-07T15:12:11.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7909000000000001e-05 2020-08-07T15:12:11.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:11.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351811+0000", 2020-08-07T15:12:11.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54429374399999997 2020-08-07T15:12:11.953 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.953 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:11.959 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.960 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:85551 5.9 e202/164 ondisk, result = 0)", 2020-08-07T15:12:11.960 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.813683+0000", 2020-08-07T15:12:11.961 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.566212280999999, 2020-08-07T15:12:11.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38973959000000002, 2020-08-07T15:12:11.961 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:11.961 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:11.961 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:11.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:11.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813683+0000", 2020-08-07T15:12:11.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:11.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:11.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813683+0000", 2020-08-07T15:12:11.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9434999999999999e-05 2020-08-07T15:12:11.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:11.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813712+0000", 2020-08-07T15:12:11.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6759999999999998e-06 2020-08-07T15:12:11.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:11.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813717+0000", 2020-08-07T15:12:11.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.389373209 2020-08-07T15:12:11.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:11.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203090+0000", 2020-08-07T15:12:11.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0859999999999998e-06 2020-08-07T15:12:11.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:11.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203096+0000", 2020-08-07T15:12:11.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024188899999999999 2020-08-07T15:12:11.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:11.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203338+0000", 2020-08-07T15:12:11.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2490000000000001e-06 2020-08-07T15:12:11.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:11.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203346+0000", 2020-08-07T15:12:11.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6045999999999997e-05 2020-08-07T15:12:11.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:11.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203422+0000", 2020-08-07T15:12:11.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38973959000000002 2020-08-07T15:12:11.971 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.971 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:11.971 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.972 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:85552 5.9 e202/164 ondisk, result = 0)", 2020-08-07T15:12:11.972 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:12:11.972 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.566151181999999, 2020-08-07T15:12:11.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53813836800000003, 2020-08-07T15:12:11.973 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:11.973 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:11.973 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:11.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:11.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:12:11.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:11.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:11.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:12:11.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8833e-05 2020-08-07T15:12:11.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:11.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813772+0000", 2020-08-07T15:12:11.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8459999999999999e-06 2020-08-07T15:12:11.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:11.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813777+0000", 2020-08-07T15:12:11.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53732224900000003 2020-08-07T15:12:11.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:11.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351100+0000", 2020-08-07T15:12:11.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1869999999999996e-06 2020-08-07T15:12:11.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:11.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351105+0000", 2020-08-07T15:12:11.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00058858100000000002 2020-08-07T15:12:11.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:11.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351693+0000", 2020-08-07T15:12:11.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4229999999999994e-06 2020-08-07T15:12:11.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:11.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351703+0000", 2020-08-07T15:12:11.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000179249 2020-08-07T15:12:11.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:11.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351882+0000", 2020-08-07T15:12:11.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53813836800000003 2020-08-07T15:12:11.983 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.983 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:11.984 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:11.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.984 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:85552 5.9 5:9d69b7cc:::benchmark_data_smithi074_53697_object10693:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e202)", 2020-08-07T15:12:11.984 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:12:11.985 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.561403058000003, 2020-08-07T15:12:11.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53326610600000002, 2020-08-07T15:12:11.985 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:11.985 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:12:11.985 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:11.986 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:11.986 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:11.986 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 85552 2020-08-07T15:12:11.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.986 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:11.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:11.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:12:11.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:11.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:11.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:12:11.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5001e-05 2020-08-07T15:12:11.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:11.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818517+0000", 2020-08-07T15:12:11.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5000000000000002e-06 2020-08-07T15:12:11.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:11.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818519+0000", 2020-08-07T15:12:11.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0699999999999997e-06 2020-08-07T15:12:11.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:11.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818524+0000", 2020-08-07T15:12:11.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2910000000000002e-06 2020-08-07T15:12:11.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:11.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818527+0000", 2020-08-07T15:12:11.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030912499999999999 2020-08-07T15:12:11.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:11.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:11.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818836+0000", 2020-08-07T15:12:11.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2908000000000002e-05 2020-08-07T15:12:11.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:11.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for ondisk", 2020-08-07T15:12:12.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818889+0000", 2020-08-07T15:12:12.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53286921099999995 2020-08-07T15:12:12.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:12.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351758+0000", 2020-08-07T15:12:12.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53326610600000002 2020-08-07T15:12:12.022 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:12.022 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:12.022 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:12.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.023 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:89464 5.9 5:94c69ba5:::benchmark_data_smithi074_53697_object11182:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:12.024 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:12:12.025 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 49.963563104000002, 2020-08-07T15:12:12.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67692916199999997, 2020-08-07T15:12:12.025 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:12.025 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:12.026 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:12.026 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:12.026 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:12.026 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 89464 2020-08-07T15:12:12.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.027 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:12.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:12.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:12:12.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:12.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:12.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:12:12.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0074000000000002e-05 2020-08-07T15:12:12.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:12.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416382+0000", 2020-08-07T15:12:12.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5000000000000001e-06 2020-08-07T15:12:12.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:12.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416386+0000", 2020-08-07T15:12:12.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2447e-05 2020-08-07T15:12:12.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:12.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416399+0000", 2020-08-07T15:12:12.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1279999999999996e-06 2020-08-07T15:12:12.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:12.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416405+0000", 2020-08-07T15:12:12.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016641805999999999 2020-08-07T15:12:12.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:12.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433047+0000", 2020-08-07T15:12:12.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107995 2020-08-07T15:12:12.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:12.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433155+0000", 2020-08-07T15:12:12.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018818599999999999 2020-08-07T15:12:12.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:12:12.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433343+0000", 2020-08-07T15:12:12.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039911679999999998 2020-08-07T15:12:12.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:12.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437334+0000", 2020-08-07T15:12:12.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2020-08-07T15:12:12.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:12.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437335+0000", 2020-08-07T15:12:12.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6351000000000002e-05 2020-08-07T15:12:12.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:12.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437401+0000", 2020-08-07T15:12:12.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001267949 2020-08-07T15:12:12.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:12.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438669+0000", 2020-08-07T15:12:12.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65454035600000005 2020-08-07T15:12:12.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:12.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093209+0000", 2020-08-07T15:12:12.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7859999999999999e-06 2020-08-07T15:12:12.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:12.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093211+0000", 2020-08-07T15:12:12.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9665e-05 2020-08-07T15:12:12.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:12.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093261+0000", 2020-08-07T15:12:12.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67692916199999997 2020-08-07T15:12:12.052 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:12.052 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:12.052 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:12.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.053 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:89464 5.9 e207/164 ondisk, result = 0)", 2020-08-07T15:12:12.053 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:12:12.054 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 49.940933342000001, 2020-08-07T15:12:12.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65430647200000003, 2020-08-07T15:12:12.055 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:12.055 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:12.056 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:12.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:12.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:12:12.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:12.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:12.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:12:12.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6529000000000002e-05 2020-08-07T15:12:12.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:12.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438998+0000", 2020-08-07T15:12:12.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.567e-06 2020-08-07T15:12:12.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:12.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.439005+0000", 2020-08-07T15:12:12.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65408475099999996 2020-08-07T15:12:12.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:12.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093089+0000", 2020-08-07T15:12:12.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6880000000000002e-06 2020-08-07T15:12:12.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:12.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093096+0000", 2020-08-07T15:12:12.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4980000000000002e-05 2020-08-07T15:12:12.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:12.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093191+0000", 2020-08-07T15:12:12.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3090000000000001e-06 2020-08-07T15:12:12.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:12.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093200+0000", 2020-08-07T15:12:12.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7648e-05 2020-08-07T15:12:12.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:12.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093268+0000", 2020-08-07T15:12:12.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65430647200000003 2020-08-07T15:12:12.067 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:12.067 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:12.068 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:12.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.068 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:114080 5.4 5:24adcd54:::benchmark_data_smithi074_53697_object14259:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e215)", 2020-08-07T15:12:12.069 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:12:12.069 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.780225885, 2020-08-07T15:12:12.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.57290862300000001, 2020-08-07T15:12:12.069 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:12.069 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:12.070 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:12.070 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:12.070 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:12.070 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 114080 2020-08-07T15:12:12.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.071 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:12.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:12.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:12:12.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:12.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:12.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:12:12.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9238e-05 2020-08-07T15:12:12.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:12.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599698+0000", 2020-08-07T15:12:12.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6260000000000003e-06 2020-08-07T15:12:12.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:12.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599703+0000", 2020-08-07T15:12:12.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5577999999999999e-05 2020-08-07T15:12:12.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:12.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599718+0000", 2020-08-07T15:12:12.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9869999999999999e-06 2020-08-07T15:12:12.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:12.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599726+0000", 2020-08-07T15:12:12.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00077929599999999996 2020-08-07T15:12:12.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:12.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600506+0000", 2020-08-07T15:12:12.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1717000000000003e-05 2020-08-07T15:12:12.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:12.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600597+0000", 2020-08-07T15:12:12.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011828999999999999 2020-08-07T15:12:12.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,5", 2020-08-07T15:12:12.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600716+0000", 2020-08-07T15:12:12.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020861479999999999 2020-08-07T15:12:12.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:12.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602802+0000", 2020-08-07T15:12:12.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4600000000000005e-07 2020-08-07T15:12:12.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:12.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602802+0000", 2020-08-07T15:12:12.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9165e-05 2020-08-07T15:12:12.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:12.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602862+0000", 2020-08-07T15:12:12.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0093485080000000002 2020-08-07T15:12:12.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:12.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.612210+0000", 2020-08-07T15:12:12.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.56030197699999995 2020-08-07T15:12:12.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:12.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172512+0000", 2020-08-07T15:12:12.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8800000000000001e-07 2020-08-07T15:12:12.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:12.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172513+0000", 2020-08-07T15:12:12.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4659000000000006e-05 2020-08-07T15:12:12.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:12.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:12.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:12.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172578+0000", 2020-08-07T15:12:12.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.57290862300000001 2020-08-07T15:12:12.131 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:12.131 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:12.131 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:12.131 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:12.132 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:12.138 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:12.139 INFO:teuthology.orchestra.run.smithi074:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 8192 --object-size 65536 -p unique_pool_3 bench 90 write' 2020-08-07T15:12:12.141 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:12:12.141 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:12.138+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:12.160 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:12:12.186 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2020-08-07T15:12:12.187 INFO:teuthology.orchestra.run:waiting for 3000 2020-08-07T15:12:12.242 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:12.239+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:12.266 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:hints = 1 2020-08-07T15:12:12.343 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Maintaining 16 concurrent writes of 8192 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2020-08-07T15:12:12.344 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Object prefix: benchmark_data_smithi074_63843 2020-08-07T15:12:12.345 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-08-07T15:12:12.345 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 0 0 0 0 0 0 - 0 2020-08-07T15:12:12.348 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:12.348 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:12:12.349 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.353 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:32 6.e 6:742bc132:::benchmark_data_smithi074_63843_object3:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e233)", 2020-08-07T15:12:12.354 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:12.286452+0000", 2020-08-07T15:12:12.354 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.051804436000000002, 2020-08-07T15:12:12.355 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.051824490000000001, 2020-08-07T15:12:12.355 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:12.356 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:12.356 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:12.357 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:12.357 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:12.379 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 32 2020-08-07T15:12:12.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.380 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:12.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:12.406 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.286452+0000", 2020-08-07T15:12:12.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:12.407 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.407 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.407 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:12.408 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.286452+0000", 2020-08-07T15:12:12.408 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7348000000000001e-05 2020-08-07T15:12:12.408 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.408 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.408 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:12.409 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.286500+0000", 2020-08-07T15:12:12.409 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8669999999999999e-06 2020-08-07T15:12:12.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.409 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.409 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:12.410 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.286506+0000", 2020-08-07T15:12:12.410 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0665e-05 2020-08-07T15:12:12.410 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.411 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.411 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:12.411 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.286516+0000", 2020-08-07T15:12:12.411 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3959999999999997e-06 2020-08-07T15:12:12.411 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.412 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.412 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:12.412 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.286522+0000", 2020-08-07T15:12:12.412 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003014671 2020-08-07T15:12:12.412 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.413 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:12.413 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.289536+0000", 2020-08-07T15:12:12.413 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6293999999999997e-05 2020-08-07T15:12:12.413 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.414 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:12.414 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.289633+0000", 2020-08-07T15:12:12.414 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019547499999999999 2020-08-07T15:12:12.414 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.414 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.415 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:12:12.415 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.289828+0000", 2020-08-07T15:12:12.415 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0091574470000000009 2020-08-07T15:12:12.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.416 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:12.416 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.298986+0000", 2020-08-07T15:12:12.417 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1200000000000003e-07 2020-08-07T15:12:12.417 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.417 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.417 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:12.417 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.298986+0000", 2020-08-07T15:12:12.418 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9392999999999999e-05 2020-08-07T15:12:12.418 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.418 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.418 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:12.419 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.299046+0000", 2020-08-07T15:12:12.419 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.020255318000000001 2020-08-07T15:12:12.419 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.419 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.419 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:12.420 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.319301+0000", 2020-08-07T15:12:12.420 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.032848486000000003 2020-08-07T15:12:12.420 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:12.420 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:12.420 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:12.421 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:12.421 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:12:12.421 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 1 2020-08-07T15:12:12.421 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:12.422 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:12:12.423 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:12.350+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:12.493 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:12.490+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:12.552 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:12.551+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:12.607 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:12.607 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:12:12.608 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:12:12.608 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:12:12.608 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:12.620 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:12:12.661 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:12.659+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:12.754 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:12.753+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:12.857 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:12.858 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:12:12.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:12:12.858 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:12:12.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.859 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "MOSDScrubReserve(2.6 GRANT e232)", 2020-08-07T15:12:12.859 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:11.909673+0000", 2020-08-07T15:12:12.859 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.942534011, 2020-08-07T15:12:12.859 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.36313456, 2020-08-07T15:12:12.859 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:12.860 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:12:12.860 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:12.860 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:12.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:11.909673+0000", 2020-08-07T15:12:12.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:12.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.861 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:12.861 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:11.909673+0000", 2020-08-07T15:12:12.862 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0409000000000001e-05 2020-08-07T15:12:12.862 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.862 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.862 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:12.862 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:11.909693+0000", 2020-08-07T15:12:12.862 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8580000000000002e-06 2020-08-07T15:12:12.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:12.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:11.909697+0000", 2020-08-07T15:12:12.863 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.36297494299999999 2020-08-07T15:12:12.864 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:12.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272672+0000", 2020-08-07T15:12:12.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4490000000000001e-06 2020-08-07T15:12:12.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.865 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:12.865 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272676+0000", 2020-08-07T15:12:12.865 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0395999999999994e-05 2020-08-07T15:12:12.882 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.882 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.882 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:12.882 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272757+0000", 2020-08-07T15:12:12.883 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6416e-05 2020-08-07T15:12:12.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:12.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272773+0000", 2020-08-07T15:12:12.890 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4088999999999998e-05 2020-08-07T15:12:12.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.890 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:12.890 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272807+0000", 2020-08-07T15:12:12.891 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.36313456 2020-08-07T15:12:12.891 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:12.891 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:12.891 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:12.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.891 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.892 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:25 6.e 6:742bc132:::benchmark_data_smithi074_63843_object3:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e233)", 2020-08-07T15:12:12.892 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:12.277229+0000", 2020-08-07T15:12:12.892 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.57497814400000002, 2020-08-07T15:12:12.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.021252779999999999, 2020-08-07T15:12:12.892 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:12.893 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:12.893 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:12.893 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:12.893 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:12.893 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 25 2020-08-07T15:12:12.894 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.894 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:12.894 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.894 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:12.894 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.277229+0000", 2020-08-07T15:12:12.895 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:12.895 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.895 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.895 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:12.895 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.277229+0000", 2020-08-07T15:12:12.896 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2781999999999999e-05 2020-08-07T15:12:12.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.896 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:12.902 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.277261+0000", 2020-08-07T15:12:12.902 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1540000000000004e-06 2020-08-07T15:12:12.902 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.902 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.902 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:12.903 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.277265+0000", 2020-08-07T15:12:12.903 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7419999999999997e-06 2020-08-07T15:12:12.903 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.903 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:12.904 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.277273+0000", 2020-08-07T15:12:12.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7119999999999999e-06 2020-08-07T15:12:12.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.904 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:12.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.277278+0000", 2020-08-07T15:12:12.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8285999999999999e-05 2020-08-07T15:12:12.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:12.906 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.277316+0000", 2020-08-07T15:12:12.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012778900000000001 2020-08-07T15:12:12.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.906 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.906 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:12.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.277444+0000", 2020-08-07T15:12:12.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012607100000000001 2020-08-07T15:12:12.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:12:12.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.277570+0000", 2020-08-07T15:12:12.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0016804319999999999 2020-08-07T15:12:12.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:12.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.279250+0000", 2020-08-07T15:12:12.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0037760929999999999 2020-08-07T15:12:12.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.910 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:12.910 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.283027+0000", 2020-08-07T15:12:12.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.015322598 2020-08-07T15:12:12.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:12.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.298349+0000", 2020-08-07T15:12:12.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3850000000000001e-06 2020-08-07T15:12:12.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:12.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.298351+0000", 2020-08-07T15:12:12.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0910999999999999e-05 2020-08-07T15:12:12.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:12.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.298421+0000", 2020-08-07T15:12:12.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9824999999999997e-05 2020-08-07T15:12:12.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:12.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.298481+0000", 2020-08-07T15:12:12.914 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.021252779999999999 2020-08-07T15:12:12.914 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:12.914 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:12.914 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:12.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.915 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:30 6.e 6:742bc132:::benchmark_data_smithi074_63843_object3:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e233)", 2020-08-07T15:12:12.915 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:12.286244+0000", 2020-08-07T15:12:12.915 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.56596221999999996, 2020-08-07T15:12:12.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.032807349, 2020-08-07T15:12:12.916 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:12.916 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:12.916 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:12.916 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:12.917 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:12.917 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 30 2020-08-07T15:12:12.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.917 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:12.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:12.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.286244+0000", 2020-08-07T15:12:12.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:12.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:12.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.286244+0000", 2020-08-07T15:12:12.931 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9431000000000002e-05 2020-08-07T15:12:12.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:12.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.286294+0000", 2020-08-07T15:12:12.932 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2689999999999996e-06 2020-08-07T15:12:12.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.932 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.932 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:12.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.286300+0000", 2020-08-07T15:12:12.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2266000000000001e-05 2020-08-07T15:12:12.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.933 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.933 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:12.933 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.286312+0000", 2020-08-07T15:12:12.934 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7259999999999999e-06 2020-08-07T15:12:12.934 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.934 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.934 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:12.934 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.286318+0000", 2020-08-07T15:12:12.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0020186819999999999 2020-08-07T15:12:12.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:12.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.288337+0000", 2020-08-07T15:12:12.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9305e-05 2020-08-07T15:12:12.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:12.936 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.288426+0000", 2020-08-07T15:12:12.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000152557 2020-08-07T15:12:12.937 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:12:12.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.288579+0000", 2020-08-07T15:12:12.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002311652 2020-08-07T15:12:12.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.938 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:12.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.290890+0000", 2020-08-07T15:12:12.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0080818379999999992 2020-08-07T15:12:12.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:12.940 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.298972+0000", 2020-08-07T15:12:12.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.020002894 2020-08-07T15:12:12.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.940 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.940 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:12.940 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.318975+0000", 2020-08-07T15:12:12.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1710000000000001e-06 2020-08-07T15:12:12.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.946 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:12.946 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.318976+0000", 2020-08-07T15:12:12.946 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4512999999999999e-05 2020-08-07T15:12:12.946 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.946 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.947 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:12.947 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.319021+0000", 2020-08-07T15:12:12.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1044999999999998e-05 2020-08-07T15:12:12.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.947 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.948 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:12.948 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.319052+0000", 2020-08-07T15:12:12.948 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.032807349 2020-08-07T15:12:12.948 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:12.948 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:12.949 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:12.949 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.949 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:31 6.e 6:742bc132:::benchmark_data_smithi074_63843_object3:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e233)", 2020-08-07T15:12:12.949 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:12.286354+0000", 2020-08-07T15:12:12.950 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.56585297499999998, 2020-08-07T15:12:12.950 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.032877000000000003, 2020-08-07T15:12:12.950 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:12.950 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:12.950 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:12.951 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:12.951 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:12.951 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 31 2020-08-07T15:12:12.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.951 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:12.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.952 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:12.952 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.286354+0000", 2020-08-07T15:12:12.952 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:12.952 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.952 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.953 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:12.953 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.286354+0000", 2020-08-07T15:12:12.953 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7985999999999998e-05 2020-08-07T15:12:12.953 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.953 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.954 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:12.954 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.286402+0000", 2020-08-07T15:12:12.954 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.057e-06 2020-08-07T15:12:12.954 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.954 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.955 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:12.955 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.286407+0000", 2020-08-07T15:12:12.955 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9110000000000005e-06 2020-08-07T15:12:12.955 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.955 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.955 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:12.956 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.286417+0000", 2020-08-07T15:12:12.956 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9140000000000001e-06 2020-08-07T15:12:12.956 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.956 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.956 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:12.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.286421+0000", 2020-08-07T15:12:12.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026055789999999998 2020-08-07T15:12:12.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:12.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.289026+0000", 2020-08-07T15:12:12.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.4178000000000006e-05 2020-08-07T15:12:12.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:12.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.289110+0000", 2020-08-07T15:12:12.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013477000000000001 2020-08-07T15:12:12.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:12:12.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.289245+0000", 2020-08-07T15:12:12.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002060264 2020-08-07T15:12:12.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.975 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:12.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.291305+0000", 2020-08-07T15:12:12.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0076736640000000002 2020-08-07T15:12:12.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:12.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.298979+0000", 2020-08-07T15:12:12.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.020126076 2020-08-07T15:12:12.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:12.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.319105+0000", 2020-08-07T15:12:12.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6199999999999996e-07 2020-08-07T15:12:12.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.978 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:12.978 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.319106+0000", 2020-08-07T15:12:12.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0714000000000001e-05 2020-08-07T15:12:12.979 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.979 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.979 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:12.979 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.319177+0000", 2020-08-07T15:12:12.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4024999999999999e-05 2020-08-07T15:12:12.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:12.980 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.319231+0000", 2020-08-07T15:12:12.980 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.032877000000000003 2020-08-07T15:12:12.980 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:12.981 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:12.981 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:12.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.981 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:32 6.e 6:742bc132:::benchmark_data_smithi074_63843_object3:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e233)", 2020-08-07T15:12:12.982 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:12.286452+0000", 2020-08-07T15:12:12.982 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.56575421299999995, 2020-08-07T15:12:12.982 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20969924300000001, 2020-08-07T15:12:12.982 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:12.982 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:12.983 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:12.983 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:12.983 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:12.983 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 32 2020-08-07T15:12:12.983 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.983 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:12.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.984 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:12.984 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.286452+0000", 2020-08-07T15:12:12.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:12.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.985 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:12.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.286452+0000", 2020-08-07T15:12:12.985 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7348000000000001e-05 2020-08-07T15:12:12.985 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.986 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:12.986 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.286500+0000", 2020-08-07T15:12:12.986 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8669999999999999e-06 2020-08-07T15:12:12.987 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.987 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:12.987 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.286506+0000", 2020-08-07T15:12:12.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0665e-05 2020-08-07T15:12:12.987 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:12.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.286516+0000", 2020-08-07T15:12:12.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3959999999999997e-06 2020-08-07T15:12:12.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:12.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.286522+0000", 2020-08-07T15:12:12.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003014671 2020-08-07T15:12:12.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.990 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.990 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:12.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.289536+0000", 2020-08-07T15:12:12.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6293999999999997e-05 2020-08-07T15:12:12.990 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:12.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.289633+0000", 2020-08-07T15:12:12.991 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019547499999999999 2020-08-07T15:12:12.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:12:12.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.289828+0000", 2020-08-07T15:12:12.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0091574470000000009 2020-08-07T15:12:12.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.993 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:12.993 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.298986+0000", 2020-08-07T15:12:12.993 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1200000000000003e-07 2020-08-07T15:12:12.993 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.994 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.994 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:12.994 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.298986+0000", 2020-08-07T15:12:12.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9392999999999999e-05 2020-08-07T15:12:12.994 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.994 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.995 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:12.995 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.299046+0000", 2020-08-07T15:12:12.995 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.020255318000000001 2020-08-07T15:12:12.995 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.995 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.996 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:12.996 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.319301+0000", 2020-08-07T15:12:12.996 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.176778719 2020-08-07T15:12:12.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.996 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.997 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:12.997 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.496080+0000", 2020-08-07T15:12:12.997 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0389999999999999e-06 2020-08-07T15:12:12.997 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.997 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.998 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:12.998 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.496081+0000", 2020-08-07T15:12:12.998 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0999000000000002e-05 2020-08-07T15:12:12.998 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:12.998 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:12.998 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:12.999 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.496152+0000", 2020-08-07T15:12:12.999 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20969924300000001 2020-08-07T15:12:12.999 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:12.999 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:12.999 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.000 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.000 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.000 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:64 6.e 6:71a0dc8c:::benchmark_data_smithi074_63843_object7:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e233)", 2020-08-07T15:12:13.000 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:12.490498+0000", 2020-08-07T15:12:13.000 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.36170865099999999, 2020-08-07T15:12:13.001 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.024026182, 2020-08-07T15:12:13.001 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:13.001 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:13.001 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:13.001 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:13.002 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:13.002 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 64 2020-08-07T15:12:13.002 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.002 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:13.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:13.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.490498+0000", 2020-08-07T15:12:13.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:13.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:13.004 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.490498+0000", 2020-08-07T15:12:13.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3766e-05 2020-08-07T15:12:13.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:13.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.490552+0000", 2020-08-07T15:12:13.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0599999999999996e-06 2020-08-07T15:12:13.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:13.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.490558+0000", 2020-08-07T15:12:13.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1839e-05 2020-08-07T15:12:13.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:13.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.490570+0000", 2020-08-07T15:12:13.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2519999999999999e-06 2020-08-07T15:12:13.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:13.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.490575+0000", 2020-08-07T15:12:13.008 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023779199999999999 2020-08-07T15:12:13.009 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.009 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.009 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:13.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.492953+0000", 2020-08-07T15:12:13.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8951999999999998e-05 2020-08-07T15:12:13.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.010 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:13.010 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.493042+0000", 2020-08-07T15:12:13.010 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012790499999999999 2020-08-07T15:12:13.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.011 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:12:13.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.493170+0000", 2020-08-07T15:12:13.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001588059 2020-08-07T15:12:13.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.012 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:13.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.494758+0000", 2020-08-07T15:12:13.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1999999999999999e-07 2020-08-07T15:12:13.012 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.013 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:13.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.494758+0000", 2020-08-07T15:12:13.013 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0906000000000003e-05 2020-08-07T15:12:13.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.014 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:13.014 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.494819+0000", 2020-08-07T15:12:13.014 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0027100340000000001 2020-08-07T15:12:13.014 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.015 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.015 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:13.015 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.497529+0000", 2020-08-07T15:12:13.015 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.016931885000000001 2020-08-07T15:12:13.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.015 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.016 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:13.016 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.514461+0000", 2020-08-07T15:12:13.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2600000000000001e-07 2020-08-07T15:12:13.016 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:13.017 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.514462+0000", 2020-08-07T15:12:13.017 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2157999999999997e-05 2020-08-07T15:12:13.017 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.018 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:13.018 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.514524+0000", 2020-08-07T15:12:13.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.024026182 2020-08-07T15:12:13.018 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.018 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:13.019 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.019 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.019 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:105 6.e 6:78da53a0:::benchmark_data_smithi074_63843_object13:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e233)", 2020-08-07T15:12:13.019 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:12.508322+0000", 2020-08-07T15:12:13.020 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.34388434800000001, 2020-08-07T15:12:13.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.022811340999999999, 2020-08-07T15:12:13.020 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:13.020 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:13.020 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:13.020 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:13.021 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:13.021 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 105 2020-08-07T15:12:13.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.021 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:13.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:13.022 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.508322+0000", 2020-08-07T15:12:13.022 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:13.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.023 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:13.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.508322+0000", 2020-08-07T15:12:13.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4685999999999999e-05 2020-08-07T15:12:13.023 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:13.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.508367+0000", 2020-08-07T15:12:13.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1460000000000001e-06 2020-08-07T15:12:13.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:13.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.508373+0000", 2020-08-07T15:12:13.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4286e-05 2020-08-07T15:12:13.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:13.026 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.508387+0000", 2020-08-07T15:12:13.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6000000000000003e-06 2020-08-07T15:12:13.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:13.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.508394+0000", 2020-08-07T15:12:13.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2138999999999997e-05 2020-08-07T15:12:13.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:13.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.508446+0000", 2020-08-07T15:12:13.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016982799999999999 2020-08-07T15:12:13.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:13.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.508616+0000", 2020-08-07T15:12:13.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016363200000000001 2020-08-07T15:12:13.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:12:13.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.508780+0000", 2020-08-07T15:12:13.030 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001933555 2020-08-07T15:12:13.030 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.030 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.030 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:13.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.510713+0000", 2020-08-07T15:12:13.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.159e-06 2020-08-07T15:12:13.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:13.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.510714+0000", 2020-08-07T15:12:13.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4391999999999997e-05 2020-08-07T15:12:13.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:13.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.510769+0000", 2020-08-07T15:12:13.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0056893610000000004 2020-08-07T15:12:13.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:13.033 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.516458+0000", 2020-08-07T15:12:13.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.014600617 2020-08-07T15:12:13.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.034 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.034 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:13.034 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.531059+0000", 2020-08-07T15:12:13.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.074e-06 2020-08-07T15:12:13.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:13.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.531060+0000", 2020-08-07T15:12:13.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3866000000000001e-05 2020-08-07T15:12:13.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:13.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.531134+0000", 2020-08-07T15:12:13.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.022811340999999999 2020-08-07T15:12:13.037 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.037 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:13.037 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.038 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:106 6.e 6:78da53a0:::benchmark_data_smithi074_63843_object13:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e233)", 2020-08-07T15:12:13.038 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:12.508422+0000", 2020-08-07T15:12:13.038 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.34378511899999997, 2020-08-07T15:12:13.038 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.022813131, 2020-08-07T15:12:13.038 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:13.038 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:13.039 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:13.039 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:13.039 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:13.039 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106 2020-08-07T15:12:13.039 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.040 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:13.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:13.040 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.508422+0000", 2020-08-07T15:12:13.040 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:13.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.041 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:13.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.508422+0000", 2020-08-07T15:12:13.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6965999999999999e-05 2020-08-07T15:12:13.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.042 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:13.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.508448+0000", 2020-08-07T15:12:13.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.549e-06 2020-08-07T15:12:13.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:13.043 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.508453+0000", 2020-08-07T15:12:13.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6749999999999996e-06 2020-08-07T15:12:13.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.044 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:13.044 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.508460+0000", 2020-08-07T15:12:13.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.689e-06 2020-08-07T15:12:13.045 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.045 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:13.045 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.508462+0000", 2020-08-07T15:12:13.045 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00072704200000000001 2020-08-07T15:12:13.046 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.046 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.046 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:13.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.509189+0000", 2020-08-07T15:12:13.047 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1172000000000001e-05 2020-08-07T15:12:13.047 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.047 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:13.047 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.509281+0000", 2020-08-07T15:12:13.047 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000188092 2020-08-07T15:12:13.048 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.048 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.048 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:12:13.048 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.509469+0000", 2020-08-07T15:12:13.048 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017182429999999999 2020-08-07T15:12:13.049 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.049 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.049 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:13.049 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.511187+0000", 2020-08-07T15:12:13.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.001e-06 2020-08-07T15:12:13.050 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.050 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.050 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:13.050 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.511188+0000", 2020-08-07T15:12:13.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3280000000000004e-05 2020-08-07T15:12:13.051 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.051 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:13.051 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.511251+0000", 2020-08-07T15:12:13.051 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0052475610000000004 2020-08-07T15:12:13.051 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.052 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.052 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:13.052 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.516499+0000", 2020-08-07T15:12:13.052 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.01467326 2020-08-07T15:12:13.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.053 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:13.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.531172+0000", 2020-08-07T15:12:13.053 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0299999999999999e-07 2020-08-07T15:12:13.053 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.054 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.054 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:13.054 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.531173+0000", 2020-08-07T15:12:13.054 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1997999999999998e-05 2020-08-07T15:12:13.054 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.054 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:13.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.531235+0000", 2020-08-07T15:12:13.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.022813131 2020-08-07T15:12:13.055 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.055 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:13.056 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.056 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:107 6.e 6:78da53a0:::benchmark_data_smithi074_63843_object13:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e233)", 2020-08-07T15:12:13.056 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:12.509201+0000", 2020-08-07T15:12:13.057 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.34300598300000001, 2020-08-07T15:12:13.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.022171142000000001, 2020-08-07T15:12:13.057 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:13.057 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:13.057 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:13.058 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:13.058 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:13.059 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 107 2020-08-07T15:12:13.059 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.059 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:13.059 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.059 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:13.060 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.509201+0000", 2020-08-07T15:12:13.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:13.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:13.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.509201+0000", 2020-08-07T15:12:13.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012528929999999999 2020-08-07T15:12:13.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.061 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:13.062 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.510454+0000", 2020-08-07T15:12:13.062 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4009999999999999e-06 2020-08-07T15:12:13.062 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.062 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.062 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:13.062 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.510458+0000", 2020-08-07T15:12:13.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.4579999999999993e-06 2020-08-07T15:12:13.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.063 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:13.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.510466+0000", 2020-08-07T15:12:13.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4460000000000003e-06 2020-08-07T15:12:13.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.064 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:13.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.510471+0000", 2020-08-07T15:12:13.065 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00096509299999999998 2020-08-07T15:12:13.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.065 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:13.065 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.511436+0000", 2020-08-07T15:12:13.066 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000102385 2020-08-07T15:12:13.066 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.066 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.066 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:13.067 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.511538+0000", 2020-08-07T15:12:13.067 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017023200000000001 2020-08-07T15:12:13.067 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.067 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.067 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:12:13.068 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.511709+0000", 2020-08-07T15:12:13.068 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026338889999999999 2020-08-07T15:12:13.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.069 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.069 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:13.069 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.514342+0000", 2020-08-07T15:12:13.069 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0999999999999997e-07 2020-08-07T15:12:13.082 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.082 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:13.083 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.514343+0000", 2020-08-07T15:12:13.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2929999999999995e-05 2020-08-07T15:12:13.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.083 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.083 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:13.084 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.514416+0000", 2020-08-07T15:12:13.084 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024146739999999999 2020-08-07T15:12:13.084 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.084 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:13.085 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.516831+0000", 2020-08-07T15:12:13.085 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.014472479 2020-08-07T15:12:13.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.085 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:13.085 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.531303+0000", 2020-08-07T15:12:13.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3499999999999996e-07 2020-08-07T15:12:13.086 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.086 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.086 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:13.086 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.531304+0000", 2020-08-07T15:12:13.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7817000000000005e-05 2020-08-07T15:12:13.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:13.087 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.531372+0000", 2020-08-07T15:12:13.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.022171142000000001 2020-08-07T15:12:13.088 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.088 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:13.088 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.089 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.089 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:137 6.1 6:898ae748:::benchmark_data_smithi074_63843_object17:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e233)", 2020-08-07T15:12:13.089 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:12.522152+0000", 2020-08-07T15:12:13.089 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.33005426100000002, 2020-08-07T15:12:13.089 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.021804673, 2020-08-07T15:12:13.090 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:13.090 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:13.090 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:13.090 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:13.090 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:13.090 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 137 2020-08-07T15:12:13.091 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.091 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:13.091 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.091 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:13.091 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.522152+0000", 2020-08-07T15:12:13.092 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:13.092 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.092 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.092 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:13.092 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.522152+0000", 2020-08-07T15:12:13.093 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1010000000000001e-05 2020-08-07T15:12:13.093 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.093 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.093 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:13.093 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.522203+0000", 2020-08-07T15:12:13.094 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6479999999999997e-06 2020-08-07T15:12:13.094 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.094 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.094 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:13.094 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.522210+0000", 2020-08-07T15:12:13.094 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3502e-05 2020-08-07T15:12:13.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.095 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.096 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:13.096 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.522224+0000", 2020-08-07T15:12:13.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2750000000000001e-06 2020-08-07T15:12:13.096 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.097 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.097 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:13.097 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.522230+0000", 2020-08-07T15:12:13.097 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2639999999999997e-05 2020-08-07T15:12:13.097 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.098 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.098 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:13.098 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.522282+0000", 2020-08-07T15:12:13.098 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000145628 2020-08-07T15:12:13.098 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.098 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.099 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:13.099 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.522428+0000", 2020-08-07T15:12:13.099 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010282500000000001 2020-08-07T15:12:13.099 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.099 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.100 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:12:13.100 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.522531+0000", 2020-08-07T15:12:13.100 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0067327669999999997 2020-08-07T15:12:13.100 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.100 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.101 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:13.101 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.529264+0000", 2020-08-07T15:12:13.101 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.850000000000001e-07 2020-08-07T15:12:13.101 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.102 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.102 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:13.102 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.529265+0000", 2020-08-07T15:12:13.102 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6240000000000003e-05 2020-08-07T15:12:13.102 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.103 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.104 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:13.104 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.529331+0000", 2020-08-07T15:12:13.104 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002094289 2020-08-07T15:12:13.104 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.104 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.105 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:13.105 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.531425+0000", 2020-08-07T15:12:13.105 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.012451732 2020-08-07T15:12:13.105 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.105 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.106 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:13.106 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.543877+0000", 2020-08-07T15:12:13.106 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8509999999999999e-06 2020-08-07T15:12:13.106 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.106 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.106 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:13.107 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.543879+0000", 2020-08-07T15:12:13.107 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8281000000000005e-05 2020-08-07T15:12:13.107 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.108 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.108 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:13.108 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.543957+0000", 2020-08-07T15:12:13.108 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.021804673 2020-08-07T15:12:13.108 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.109 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:13.109 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.109 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.109 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.110 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:138 6.1 6:898ae748:::benchmark_data_smithi074_63843_object17:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e233)", 2020-08-07T15:12:13.110 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:12.523183+0000", 2020-08-07T15:12:13.110 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.32902400199999998, 2020-08-07T15:12:13.110 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.020887968999999999, 2020-08-07T15:12:13.110 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:13.111 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:13.111 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:13.111 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:13.111 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:13.111 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 138 2020-08-07T15:12:13.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.112 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:13.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.112 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:13.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.523183+0000", 2020-08-07T15:12:13.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:13.113 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.113 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:13.113 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.523183+0000", 2020-08-07T15:12:13.113 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2713999999999997e-05 2020-08-07T15:12:13.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.114 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:13.114 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.523235+0000", 2020-08-07T15:12:13.114 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5710000000000003e-06 2020-08-07T15:12:13.115 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.115 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.115 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:13.115 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.523242+0000", 2020-08-07T15:12:13.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2768e-05 2020-08-07T15:12:13.115 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.116 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.116 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:13.116 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.523255+0000", 2020-08-07T15:12:13.116 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.516e-06 2020-08-07T15:12:13.116 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.117 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.118 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:13.118 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.523261+0000", 2020-08-07T15:12:13.118 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2895000000000002e-05 2020-08-07T15:12:13.118 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.119 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.119 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:13.119 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.523304+0000", 2020-08-07T15:12:13.119 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000109658 2020-08-07T15:12:13.120 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.120 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.120 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:13.120 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.523414+0000", 2020-08-07T15:12:13.120 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017447599999999999 2020-08-07T15:12:13.121 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.121 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.121 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:12:13.121 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.523588+0000", 2020-08-07T15:12:13.121 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0057507160000000003 2020-08-07T15:12:13.122 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.122 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.122 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:13.122 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.529339+0000", 2020-08-07T15:12:13.122 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7400000000000003e-07 2020-08-07T15:12:13.123 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.123 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.123 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:13.123 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.529340+0000", 2020-08-07T15:12:13.123 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6276000000000001e-05 2020-08-07T15:12:13.123 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.124 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.124 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:13.124 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.529396+0000", 2020-08-07T15:12:13.124 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002453972 2020-08-07T15:12:13.124 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.125 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.125 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:13.125 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.531850+0000", 2020-08-07T15:12:13.125 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.012148760999999999 2020-08-07T15:12:13.125 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.126 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.126 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:13.126 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.543999+0000", 2020-08-07T15:12:13.126 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9299999999999998e-07 2020-08-07T15:12:13.126 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.127 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.127 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:13.127 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.543999+0000", 2020-08-07T15:12:13.127 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1579e-05 2020-08-07T15:12:13.127 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.127 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.128 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:13.128 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.544071+0000", 2020-08-07T15:12:13.128 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.020887968999999999 2020-08-07T15:12:13.128 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.128 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:13.129 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.129 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.129 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.129 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:140 6.1 6:898ae748:::benchmark_data_smithi074_63843_object17:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e233)", 2020-08-07T15:12:13.129 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:12.527566+0000", 2020-08-07T15:12:13.130 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.32464106100000001, 2020-08-07T15:12:13.130 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.021214762000000002, 2020-08-07T15:12:13.130 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:13.131 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:13.140 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:13.140 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:13.140 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:13.140 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 140 2020-08-07T15:12:13.140 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.141 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:13.141 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.141 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:13.141 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.527566+0000", 2020-08-07T15:12:13.141 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:13.142 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.142 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.142 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:13.142 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.527566+0000", 2020-08-07T15:12:13.142 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0246000000000003e-05 2020-08-07T15:12:13.143 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.143 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.143 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:13.143 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.527656+0000", 2020-08-07T15:12:13.143 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0502e-05 2020-08-07T15:12:13.144 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.144 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.144 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:13.144 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.527666+0000", 2020-08-07T15:12:13.144 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2464e-05 2020-08-07T15:12:13.145 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.145 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.145 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:13.145 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.527679+0000", 2020-08-07T15:12:13.145 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7020000000000004e-06 2020-08-07T15:12:13.146 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.146 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.146 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:13.146 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.527686+0000", 2020-08-07T15:12:13.146 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000340133 2020-08-07T15:12:13.147 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.147 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.147 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:13.147 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.528026+0000", 2020-08-07T15:12:13.147 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010537 2020-08-07T15:12:13.148 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.148 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.148 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:13.148 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.528131+0000", 2020-08-07T15:12:13.148 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000173742 2020-08-07T15:12:13.149 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.149 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.149 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:12:13.149 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.528305+0000", 2020-08-07T15:12:13.149 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0036325400000000001 2020-08-07T15:12:13.149 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.150 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.150 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:13.150 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.531937+0000", 2020-08-07T15:12:13.150 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3207000000000001e-05 2020-08-07T15:12:13.150 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.151 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:13.151 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.531950+0000", 2020-08-07T15:12:13.151 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.016697498000000002 2020-08-07T15:12:13.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:13.152 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.548648+0000", 2020-08-07T15:12:13.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.279e-06 2020-08-07T15:12:13.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.153 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.153 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:13.153 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.548649+0000", 2020-08-07T15:12:13.153 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0397999999999998e-05 2020-08-07T15:12:13.153 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.153 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.154 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:13.154 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.548730+0000", 2020-08-07T15:12:13.154 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0680999999999998e-05 2020-08-07T15:12:13.154 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.154 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.155 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:13.155 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.548780+0000", 2020-08-07T15:12:13.155 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.021214762000000002 2020-08-07T15:12:13.155 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.155 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:13.156 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.156 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:141 6.1 6:898ae748:::benchmark_data_smithi074_63843_object17:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e233)", 2020-08-07T15:12:13.156 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:12.527729+0000", 2020-08-07T15:12:13.157 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.32447805600000001, 2020-08-07T15:12:13.157 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.021162253999999998, 2020-08-07T15:12:13.157 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:13.157 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:13.157 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:13.157 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:13.158 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:13.158 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 141 2020-08-07T15:12:13.158 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.158 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:13.158 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.159 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:13.159 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.527729+0000", 2020-08-07T15:12:13.159 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:13.159 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.159 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.160 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:13.160 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.527729+0000", 2020-08-07T15:12:13.160 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7327000000000002e-05 2020-08-07T15:12:13.160 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.160 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.160 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:13.161 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.527786+0000", 2020-08-07T15:12:13.161 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3600000000000001e-06 2020-08-07T15:12:13.161 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.161 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.161 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:13.162 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.527792+0000", 2020-08-07T15:12:13.162 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1619e-05 2020-08-07T15:12:13.162 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.163 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.163 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:13.163 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.527804+0000", 2020-08-07T15:12:13.163 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2739999999999998e-06 2020-08-07T15:12:13.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:13.164 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.527809+0000", 2020-08-07T15:12:13.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00085318299999999998 2020-08-07T15:12:13.165 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.165 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.166 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:13.166 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.528662+0000", 2020-08-07T15:12:13.166 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5885999999999996e-05 2020-08-07T15:12:13.166 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.166 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.167 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:13.167 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.528758+0000", 2020-08-07T15:12:13.167 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000171917 2020-08-07T15:12:13.167 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.167 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.168 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:12:13.168 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.528930+0000", 2020-08-07T15:12:13.168 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0030264089999999999 2020-08-07T15:12:13.168 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.168 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.169 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:13.169 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.531957+0000", 2020-08-07T15:12:13.169 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4499999999999997e-07 2020-08-07T15:12:13.169 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.169 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.170 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:13.170 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.531957+0000", 2020-08-07T15:12:13.170 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3168000000000002e-05 2020-08-07T15:12:13.170 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.171 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.171 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:13.171 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.532020+0000", 2020-08-07T15:12:13.171 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00082684200000000005 2020-08-07T15:12:13.172 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.172 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.172 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:13.172 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.532847+0000", 2020-08-07T15:12:13.172 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.015981403000000002 2020-08-07T15:12:13.173 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.174 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.174 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:13.174 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.548828+0000", 2020-08-07T15:12:13.174 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8199999999999999e-07 2020-08-07T15:12:13.174 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.175 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.175 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:13.175 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.548829+0000", 2020-08-07T15:12:13.175 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1739e-05 2020-08-07T15:12:13.175 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.176 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.176 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:13.176 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.548891+0000", 2020-08-07T15:12:13.176 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.021162253999999998 2020-08-07T15:12:13.176 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.177 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:13.177 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.177 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.177 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.178 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:142 6.1 6:898ae748:::benchmark_data_smithi074_63843_object17:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e233)", 2020-08-07T15:12:13.178 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:12.527847+0000", 2020-08-07T15:12:13.178 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.32435935100000002, 2020-08-07T15:12:13.178 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.021202972, 2020-08-07T15:12:13.178 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:13.179 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:13.179 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:13.179 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:13.179 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:13.179 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 142 2020-08-07T15:12:13.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.180 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:13.180 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.180 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:13.180 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.527847+0000", 2020-08-07T15:12:13.181 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:13.181 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.181 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:13.182 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.527847+0000", 2020-08-07T15:12:13.182 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6241000000000005e-05 2020-08-07T15:12:13.182 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:13.183 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.527914+0000", 2020-08-07T15:12:13.198 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7599999999999999e-06 2020-08-07T15:12:13.198 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.198 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.198 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:13.199 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.527919+0000", 2020-08-07T15:12:13.199 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1707e-05 2020-08-07T15:12:13.199 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.199 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.199 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:13.200 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.527931+0000", 2020-08-07T15:12:13.200 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9300000000000002e-06 2020-08-07T15:12:13.200 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.200 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.200 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:13.201 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.527936+0000", 2020-08-07T15:12:13.201 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0014891430000000001 2020-08-07T15:12:13.201 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.201 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.201 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:13.202 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.529425+0000", 2020-08-07T15:12:13.202 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6037000000000002e-05 2020-08-07T15:12:13.202 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.202 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.202 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:13.203 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.529521+0000", 2020-08-07T15:12:13.203 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00023669500000000001 2020-08-07T15:12:13.203 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.203 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.203 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:12:13.203 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.529758+0000", 2020-08-07T15:12:13.204 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0036211250000000002 2020-08-07T15:12:13.204 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.204 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.204 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:13.204 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.533379+0000", 2020-08-07T15:12:13.205 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024803350000000002 2020-08-07T15:12:13.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.205 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.205 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:13.205 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.535859+0000", 2020-08-07T15:12:13.220 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.013078533999999999 2020-08-07T15:12:13.220 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.220 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.221 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:13.221 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.548938+0000", 2020-08-07T15:12:13.221 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6699999999999999e-07 2020-08-07T15:12:13.222 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.222 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.222 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:13.222 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.548938+0000", 2020-08-07T15:12:13.222 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7098e-05 2020-08-07T15:12:13.223 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.223 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.223 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:13.223 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.549005+0000", 2020-08-07T15:12:13.223 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.49e-05 2020-08-07T15:12:13.224 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.224 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.224 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:13.224 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.549050+0000", 2020-08-07T15:12:13.224 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.021202972 2020-08-07T15:12:13.224 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.225 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:13.225 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.225 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.225 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.225 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:143 6.1 6:898ae748:::benchmark_data_smithi074_63843_object17:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e233)", 2020-08-07T15:12:13.226 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:12.527979+0000", 2020-08-07T15:12:13.226 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.32422804900000002, 2020-08-07T15:12:13.226 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.021239179, 2020-08-07T15:12:13.226 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:13.226 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:13.227 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:13.227 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:13.227 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:13.227 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 143 2020-08-07T15:12:13.227 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.228 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:13.228 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.228 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:13.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.527979+0000", 2020-08-07T15:12:13.228 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:13.228 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.229 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.229 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:13.229 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.527979+0000", 2020-08-07T15:12:13.229 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8797e-05 2020-08-07T15:12:13.229 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.231 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.231 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:13.231 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.528037+0000", 2020-08-07T15:12:13.231 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4410000000000001e-06 2020-08-07T15:12:13.231 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.232 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.232 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:13.232 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.528043+0000", 2020-08-07T15:12:13.232 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.414e-05 2020-08-07T15:12:13.232 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.232 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.233 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:13.233 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.528057+0000", 2020-08-07T15:12:13.233 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9649999999999999e-06 2020-08-07T15:12:13.233 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.233 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.234 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:13.234 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.528062+0000", 2020-08-07T15:12:13.234 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0020844460000000002 2020-08-07T15:12:13.234 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.234 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.235 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:13.235 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.530146+0000", 2020-08-07T15:12:13.235 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4588999999999998e-05 2020-08-07T15:12:13.235 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.235 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.236 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:13.236 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.530241+0000", 2020-08-07T15:12:13.236 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017816699999999999 2020-08-07T15:12:13.236 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.236 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.236 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:12:13.237 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.530419+0000", 2020-08-07T15:12:13.237 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0030037380000000002 2020-08-07T15:12:13.237 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.237 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.237 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:13.242 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.533423+0000", 2020-08-07T15:12:13.242 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024442230000000001 2020-08-07T15:12:13.242 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.242 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.243 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:13.243 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.535867+0000", 2020-08-07T15:12:13.243 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.013231244 2020-08-07T15:12:13.243 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.243 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.244 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:13.244 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.549098+0000", 2020-08-07T15:12:13.244 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.75e-07 2020-08-07T15:12:13.244 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.244 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.245 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:13.250 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.549099+0000", 2020-08-07T15:12:13.250 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8918000000000007e-05 2020-08-07T15:12:13.250 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.250 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.251 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:13.251 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.549168+0000", 2020-08-07T15:12:13.251 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9935999999999998e-05 2020-08-07T15:12:13.251 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.251 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.252 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:13.252 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.549218+0000", 2020-08-07T15:12:13.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.021239179 2020-08-07T15:12:13.252 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.252 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:13.253 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.253 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.253 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:144 6.1 6:898ae748:::benchmark_data_smithi074_63843_object17:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e233)", 2020-08-07T15:12:13.253 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:12.528440+0000", 2020-08-07T15:12:13.254 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.323766573, 2020-08-07T15:12:13.254 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.020918448999999999, 2020-08-07T15:12:13.254 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:13.254 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:13.254 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:13.255 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:13.255 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:13.255 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 144 2020-08-07T15:12:13.255 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.255 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:13.256 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.256 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:13.256 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.528440+0000", 2020-08-07T15:12:13.256 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:13.256 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.257 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.257 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:13.257 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.528440+0000", 2020-08-07T15:12:13.257 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8048999999999999e-05 2020-08-07T15:12:13.257 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.257 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.258 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:13.258 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.528498+0000", 2020-08-07T15:12:13.258 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8610000000000003e-06 2020-08-07T15:12:13.258 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.258 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.259 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:13.259 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.528504+0000", 2020-08-07T15:12:13.259 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2139e-05 2020-08-07T15:12:13.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.259 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.260 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:13.260 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.528516+0000", 2020-08-07T15:12:13.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7729999999999996e-06 2020-08-07T15:12:13.260 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.260 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.261 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:13.261 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.528522+0000", 2020-08-07T15:12:13.261 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023700769999999999 2020-08-07T15:12:13.261 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.261 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.262 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:13.262 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.530892+0000", 2020-08-07T15:12:13.262 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1238999999999998e-05 2020-08-07T15:12:13.262 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.262 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.263 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:13.263 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.530983+0000", 2020-08-07T15:12:13.263 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012246499999999999 2020-08-07T15:12:13.263 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.263 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.263 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:12:13.264 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.531106+0000", 2020-08-07T15:12:13.264 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002478535 2020-08-07T15:12:13.264 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.264 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.264 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:13.265 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.533584+0000", 2020-08-07T15:12:13.265 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0022887720000000001 2020-08-07T15:12:13.265 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.265 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.265 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:13.266 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.535873+0000", 2020-08-07T15:12:13.266 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.013383116 2020-08-07T15:12:13.266 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.266 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.266 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:13.267 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.549256+0000", 2020-08-07T15:12:13.267 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6800000000000001e-07 2020-08-07T15:12:13.267 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.267 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.267 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:13.268 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.549257+0000", 2020-08-07T15:12:13.268 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7667000000000002e-05 2020-08-07T15:12:13.268 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.268 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.268 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:13.269 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.549314+0000", 2020-08-07T15:12:13.269 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4288e-05 2020-08-07T15:12:13.269 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.270 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.270 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:13.270 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.549359+0000", 2020-08-07T15:12:13.270 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.020918448999999999 2020-08-07T15:12:13.271 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.271 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:13.271 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.271 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.272 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:384 6.1 6:8b9c4cfb:::benchmark_data_smithi074_63843_object47:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e233)", 2020-08-07T15:12:13.272 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:12.668399+0000", 2020-08-07T15:12:13.272 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.18380766300000001, 2020-08-07T15:12:13.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.064293390000000006, 2020-08-07T15:12:13.273 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:13.273 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:13.273 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:13.273 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:13.273 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:13.274 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 384 2020-08-07T15:12:13.274 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.274 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:13.274 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.274 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:13.275 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.668399+0000", 2020-08-07T15:12:13.275 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:13.275 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.275 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.275 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:13.275 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.668399+0000", 2020-08-07T15:12:13.276 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1681999999999997e-05 2020-08-07T15:12:13.276 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.276 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.276 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:13.276 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.668451+0000", 2020-08-07T15:12:13.277 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2520000000000002e-06 2020-08-07T15:12:13.277 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.277 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.277 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:13.277 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.668458+0000", 2020-08-07T15:12:13.278 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3145e-05 2020-08-07T15:12:13.278 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.278 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.278 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:13.278 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.668471+0000", 2020-08-07T15:12:13.279 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0290000000000005e-06 2020-08-07T15:12:13.279 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.279 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.279 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:13.279 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.668479+0000", 2020-08-07T15:12:13.280 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00048491000000000002 2020-08-07T15:12:13.280 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.280 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.280 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:13.280 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.668964+0000", 2020-08-07T15:12:13.280 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000113453 2020-08-07T15:12:13.281 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.281 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.281 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:13.281 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.669077+0000", 2020-08-07T15:12:13.281 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017818700000000001 2020-08-07T15:12:13.282 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.282 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.282 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:12:13.282 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.669256+0000", 2020-08-07T15:12:13.282 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0021138889999999999 2020-08-07T15:12:13.283 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.283 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.283 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:13.283 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.671370+0000", 2020-08-07T15:12:13.283 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00038495400000000001 2020-08-07T15:12:13.283 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.284 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.284 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:13.284 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.671754+0000", 2020-08-07T15:12:13.284 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.060768681999999997 2020-08-07T15:12:13.284 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.285 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.285 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:13.285 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.732523+0000", 2020-08-07T15:12:13.285 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1200000000000001e-07 2020-08-07T15:12:13.285 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.286 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.286 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:13.286 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.732524+0000", 2020-08-07T15:12:13.286 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3263999999999993e-05 2020-08-07T15:12:13.286 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.287 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:13.287 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.732617+0000", 2020-08-07T15:12:13.287 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5030999999999994e-05 2020-08-07T15:12:13.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.288 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:13.288 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.732692+0000", 2020-08-07T15:12:13.288 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.064293390000000006 2020-08-07T15:12:13.288 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.288 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:13.289 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.289 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.289 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.289 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:384 6.1 e233/230 ondisk, result = 0)", 2020-08-07T15:12:13.289 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:12.672299+0000", 2020-08-07T15:12:13.290 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.179907649, 2020-08-07T15:12:13.290 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.060403042999999997, 2020-08-07T15:12:13.290 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:13.291 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:12:13.291 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:13.291 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.291 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:13.291 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.672299+0000", 2020-08-07T15:12:13.292 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:13.292 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.292 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.292 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:13.292 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.672299+0000", 2020-08-07T15:12:13.292 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7495999999999999e-05 2020-08-07T15:12:13.293 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.293 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.293 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:13.293 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.672326+0000", 2020-08-07T15:12:13.293 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7209999999999999e-06 2020-08-07T15:12:13.294 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.294 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.294 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:13.294 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.672329+0000", 2020-08-07T15:12:13.294 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.059253103000000001 2020-08-07T15:12:13.295 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.295 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.295 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:13.295 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.731582+0000", 2020-08-07T15:12:13.295 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8770000000000004e-06 2020-08-07T15:12:13.296 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.296 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.296 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:13.296 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.731589+0000", 2020-08-07T15:12:13.296 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00091616400000000002 2020-08-07T15:12:13.296 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.297 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.297 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:13.297 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.732505+0000", 2020-08-07T15:12:13.297 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0909999999999999e-06 2020-08-07T15:12:13.297 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.298 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.298 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:13.298 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.732514+0000", 2020-08-07T15:12:13.298 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000187591 2020-08-07T15:12:13.298 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.299 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.299 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:13.299 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.732702+0000", 2020-08-07T15:12:13.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.060403042999999997 2020-08-07T15:12:13.299 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.300 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:13.300 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.300 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.300 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.300 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:512 6.1 6:87452d83:::benchmark_data_smithi074_63843_object63:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e233)", 2020-08-07T15:12:13.301 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:12.742686+0000", 2020-08-07T15:12:13.301 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.109520677, 2020-08-07T15:12:13.301 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.066214734999999997, 2020-08-07T15:12:13.301 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:13.301 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:13.302 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:13.302 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:13.302 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:13.302 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 512 2020-08-07T15:12:13.302 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.302 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:13.303 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.303 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:13.303 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.742686+0000", 2020-08-07T15:12:13.303 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:13.303 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.304 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.305 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:13.305 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.742686+0000", 2020-08-07T15:12:13.305 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9487999999999998e-05 2020-08-07T15:12:13.305 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.306 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.306 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:13.307 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.742735+0000", 2020-08-07T15:12:13.307 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.259e-06 2020-08-07T15:12:13.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.308 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:13.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.742741+0000", 2020-08-07T15:12:13.308 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.188e-05 2020-08-07T15:12:13.309 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.309 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.309 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:13.309 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.742753+0000", 2020-08-07T15:12:13.309 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2119999999999997e-06 2020-08-07T15:12:13.310 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.310 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.310 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:13.310 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.742757+0000", 2020-08-07T15:12:13.310 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0022313189999999998 2020-08-07T15:12:13.310 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.311 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.311 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:13.311 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.744988+0000", 2020-08-07T15:12:13.311 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3819000000000004e-05 2020-08-07T15:12:13.311 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.312 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.312 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:13.312 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.745082+0000", 2020-08-07T15:12:13.312 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000149658 2020-08-07T15:12:13.312 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.313 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.313 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:12:13.313 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.745232+0000", 2020-08-07T15:12:13.313 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032346139999999998 2020-08-07T15:12:13.313 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.314 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.314 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:13.314 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.748466+0000", 2020-08-07T15:12:13.314 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0299999999999999e-07 2020-08-07T15:12:13.314 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.314 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.315 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:13.315 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.748467+0000", 2020-08-07T15:12:13.315 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1995000000000004e-05 2020-08-07T15:12:13.315 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.315 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.316 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:13.316 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.748539+0000", 2020-08-07T15:12:13.316 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00026646299999999999 2020-08-07T15:12:13.316 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.316 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.317 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:13.317 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.748805+0000", 2020-08-07T15:12:13.317 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.060032982999999998 2020-08-07T15:12:13.317 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.317 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.318 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:13.318 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.808838+0000", 2020-08-07T15:12:13.318 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1099999999999995e-07 2020-08-07T15:12:13.318 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.318 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.319 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:13.319 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.808839+0000", 2020-08-07T15:12:13.319 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1830999999999998e-05 2020-08-07T15:12:13.319 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.320 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.320 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:13.320 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.808901+0000", 2020-08-07T15:12:13.321 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.066214734999999997 2020-08-07T15:12:13.321 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.321 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:13.321 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.321 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.322 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.322 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:512 6.1 e233/230 ondisk, result = 0)", 2020-08-07T15:12:13.322 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:12.750405+0000", 2020-08-07T15:12:13.322 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.101802027, 2020-08-07T15:12:13.322 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.058503524000000001, 2020-08-07T15:12:13.323 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:13.323 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:12:13.323 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:13.323 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.323 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:13.324 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.750405+0000", 2020-08-07T15:12:13.324 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:13.324 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.324 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.324 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:13.325 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.750405+0000", 2020-08-07T15:12:13.325 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0446e-05 2020-08-07T15:12:13.325 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.325 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.325 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:13.325 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.750435+0000", 2020-08-07T15:12:13.326 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.865e-06 2020-08-07T15:12:13.326 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.326 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.326 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:13.326 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.750440+0000", 2020-08-07T15:12:13.327 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.057969606999999999 2020-08-07T15:12:13.327 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.327 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.327 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:13.327 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.808410+0000", 2020-08-07T15:12:13.328 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8300000000000003e-06 2020-08-07T15:12:13.328 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.328 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.328 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:13.328 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.808414+0000", 2020-08-07T15:12:13.328 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00041058799999999998 2020-08-07T15:12:13.329 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.329 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.329 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:13.329 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.808825+0000", 2020-08-07T15:12:13.329 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7989999999999996e-06 2020-08-07T15:12:13.346 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.346 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.346 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:13.346 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.808832+0000", 2020-08-07T15:12:13.347 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6389000000000003e-05 2020-08-07T15:12:13.347 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:13.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:13.347 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:13.347 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.808908+0000", 2020-08-07T15:12:13.348 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.058503524000000001 2020-08-07T15:12:13.348 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.348 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:13.348 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.348 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:13.349 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:13.349 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:13.350 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:12:13.351 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:12.855+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:13.352 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 1 16 1433 1417 11.0697 11.0703 0.0105589 0.0111776 2020-08-07T15:12:13.352 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:13.349+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:13.353 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:12:13.454 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:13.451+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:13.556 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:13.553+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:13.619 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:13.620 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:12:13.620 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:12:13.620 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:13.634 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:12:13.640 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:12:13.656 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:13.654+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:13.680 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2020-08-07T15:12:13.681 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:12:13.682 INFO:tasks.thrashosds.thrasher:inject_pause on 5 2020-08-07T15:12:13.682 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2020-08-07T15:12:13.682 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2020-08-07T15:12:13.683 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2020-08-07T15:12:13.684 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:12:13.724 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:12:13.757 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:13.756+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:13.767 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:12:13.807 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:12:13.848 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:13.849 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "" 2020-08-07T15:12:13.849 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:13.856 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:12:13.860 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:13.861 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:13.861 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:13.858+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:13.881 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:12:13.914 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:12:13.955 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:12:13.956 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:12:13.956 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:12:13.956 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:12:13.956 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:12:13.957 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:12:13.957 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:12:13.958 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:12:13.958 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:12:13.999 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:13.994+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:14.072 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:14.064+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:14.106 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:14.107 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:14.116 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:12:14.161 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:12:14.212 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:14.205+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:14.272 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 2 15 3031 3016 11.7805 12.4922 0.00820189 0.0105686 2020-08-07T15:12:14.273 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:14.271+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:14.370 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:14.371 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:14.379 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:14.375+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:14.387 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:12:14.485 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:14.482+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:14.581 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:14.579+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:14.645 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:14.645 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:14.663 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:12:14.686 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:14.681+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:14.791 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:14.789+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:14.877 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:14.877 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:14.881 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:12:14.926 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:14.924+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:15.007 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:14.997+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:15.072 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:15.072 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:15.085 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:12:15.147 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:15.125+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:15.206 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:15.201+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:15.268 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 3 16 4475 4459 11.6112 11.2734 0.00313992 0.0106971 2020-08-07T15:12:15.298 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:15.298 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:15.304 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:15.302+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:15.311 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:12:15.405 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:15.403+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:15.426 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:12:15.506 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:15.505+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:15.542 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:15.542 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:15.554 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:12:15.607 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:15.605+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:15.715 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:15.712+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:15.779 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:15.779 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:15.802 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:12:15.815 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:15.813+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:15.926 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:15.923+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:15.961 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:12:16.010 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:16.011 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:16.019 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:12:16.060 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:16.058+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:16.124 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:16.121+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:16.193 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:12:16.211 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:16.211 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:16.223 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:12:16.228 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:16.226+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:16.230 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:12:16.265 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 4 16 6108 6092 11.8963 12.7578 0.0123381 0.0104748 2020-08-07T15:12:16.274 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:12:16.314 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:12:16.348 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:16.345+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:16.366 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:12:16.384 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:16.385 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:16.396 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:12:16.427 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:12:16.436 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:12:16.449 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:16.447+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:16.480 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:12:16.481 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:12:16.482 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:12:16.482 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:12:16.482 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:12:16.483 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:12:16.483 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:12:16.483 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:12:16.483 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:12:16.554 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:16.552+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:16.597 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:16.597 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:12:16.598 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:12:16.598 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:12:16.598 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:16.615 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:12:16.661 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:16.659+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:16.759 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:16.758+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:16.799 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:16.799 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:12:16.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:12:16.800 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:12:16.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.800 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42440 4.9 4:9b3fd4e3:::benchmark_data_smithi074_44586_object5304:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:12:16.800 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:12:16.801 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 196.554196085, 2020-08-07T15:12:16.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9384075030000001, 2020-08-07T15:12:16.801 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:16.801 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:16.801 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:16.804 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:16.804 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:16.804 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42440 2020-08-07T15:12:16.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.805 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:16.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:16.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:12:16.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:16.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:16.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:12:16.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6295000000000002e-05 2020-08-07T15:12:16.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:16.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237116+0000", 2020-08-07T15:12:16.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0390000000000001e-06 2020-08-07T15:12:16.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:16.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237122+0000", 2020-08-07T15:12:16.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8340000000000001e-05 2020-08-07T15:12:16.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:16.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237140+0000", 2020-08-07T15:12:16.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.045e-06 2020-08-07T15:12:16.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:16.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237145+0000", 2020-08-07T15:12:16.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044548920000000002 2020-08-07T15:12:16.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241600+0000", 2020-08-07T15:12:16.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103336 2020-08-07T15:12:16.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:16.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241703+0000", 2020-08-07T15:12:16.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018660399999999999 2020-08-07T15:12:16.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:12:16.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241890+0000", 2020-08-07T15:12:16.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00096796000000000002 2020-08-07T15:12:16.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:16.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.242858+0000", 2020-08-07T15:12:16.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039845000000000002 2020-08-07T15:12:16.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:16.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246843+0000", 2020-08-07T15:12:16.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3499999999999996e-07 2020-08-07T15:12:16.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:16.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246843+0000", 2020-08-07T15:12:16.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7132999999999995e-05 2020-08-07T15:12:16.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:16.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246920+0000", 2020-08-07T15:12:16.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046993410000000001 2020-08-07T15:12:16.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:16.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.251620+0000", 2020-08-07T15:12:16.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040332956000000003 2020-08-07T15:12:16.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.291953+0000", 2020-08-07T15:12:16.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8390000000000003e-06 2020-08-07T15:12:16.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:16.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.291958+0000", 2020-08-07T15:12:16.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99961357500000003 2020-08-07T15:12:16.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291572+0000", 2020-08-07T15:12:16.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3913999999999999e-05 2020-08-07T15:12:16.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:16.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291586+0000", 2020-08-07T15:12:16.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88379723099999996 2020-08-07T15:12:16.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175383+0000", 2020-08-07T15:12:16.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3868000000000004e-05 2020-08-07T15:12:16.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:16.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175467+0000", 2020-08-07T15:12:16.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9384075030000001 2020-08-07T15:12:16.837 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:16.837 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:16.837 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:16.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.838 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42489 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:12:16.838 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:12:16.838 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 196.51089602299999, 2020-08-07T15:12:16.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952608639999999, 2020-08-07T15:12:16.839 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:16.839 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:16.839 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:16.839 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:16.840 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:16.840 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42489 2020-08-07T15:12:16.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.843 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:16.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:16.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:12:16.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:16.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:16.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:12:16.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0379e-05 2020-08-07T15:12:16.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:16.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280410+0000", 2020-08-07T15:12:16.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0489999999999998e-06 2020-08-07T15:12:16.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:16.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280417+0000", 2020-08-07T15:12:16.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8281e-05 2020-08-07T15:12:16.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:16.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280435+0000", 2020-08-07T15:12:16.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7510000000000008e-06 2020-08-07T15:12:16.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:16.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280443+0000", 2020-08-07T15:12:16.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018142899999999999 2020-08-07T15:12:16.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280624+0000", 2020-08-07T15:12:16.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013473599999999999 2020-08-07T15:12:16.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:16.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280759+0000", 2020-08-07T15:12:16.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019964200000000001 2020-08-07T15:12:16.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:12:16.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280959+0000", 2020-08-07T15:12:16.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005588038 2020-08-07T15:12:16.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:16.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286547+0000", 2020-08-07T15:12:16.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.376e-05 2020-08-07T15:12:16.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:16.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286561+0000", 2020-08-07T15:12:16.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4199999999999995e-07 2020-08-07T15:12:16.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:16.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286561+0000", 2020-08-07T15:12:16.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9218e-05 2020-08-07T15:12:16.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:16.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286631+0000", 2020-08-07T15:12:16.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027909199999999999 2020-08-07T15:12:16.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:16.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286910+0000", 2020-08-07T15:12:16.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050904089999999997 2020-08-07T15:12:16.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292000+0000", 2020-08-07T15:12:16.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9579999999999996e-06 2020-08-07T15:12:16.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:16.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292009+0000", 2020-08-07T15:12:16.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99961402799999999 2020-08-07T15:12:16.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291623+0000", 2020-08-07T15:12:16.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2490000000000001e-06 2020-08-07T15:12:16.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:16.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291631+0000", 2020-08-07T15:12:16.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88387422199999999 2020-08-07T15:12:16.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175506+0000", 2020-08-07T15:12:16.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000114881 2020-08-07T15:12:16.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:16.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175620+0000", 2020-08-07T15:12:16.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952608639999999 2020-08-07T15:12:16.872 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:16.873 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:16.873 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:16.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.873 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42490 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:12:16.874 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:12:16.874 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 196.51076885500001, 2020-08-07T15:12:16.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89528059, 2020-08-07T15:12:16.874 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:16.874 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:16.875 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:16.875 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:16.875 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:16.875 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42490 2020-08-07T15:12:16.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.876 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:16.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:16.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:12:16.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:16.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:16.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:12:16.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2571999999999997e-05 2020-08-07T15:12:16.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:16.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280539+0000", 2020-08-07T15:12:16.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999998e-06 2020-08-07T15:12:16.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:16.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280545+0000", 2020-08-07T15:12:16.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.134e-05 2020-08-07T15:12:16.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:16.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280556+0000", 2020-08-07T15:12:16.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.746e-06 2020-08-07T15:12:16.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:16.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280561+0000", 2020-08-07T15:12:16.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00089258999999999996 2020-08-07T15:12:16.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281453+0000", 2020-08-07T15:12:16.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5641000000000001e-05 2020-08-07T15:12:16.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:16.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281549+0000", 2020-08-07T15:12:16.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020555200000000001 2020-08-07T15:12:16.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:12:16.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281755+0000", 2020-08-07T15:12:16.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048878230000000003 2020-08-07T15:12:16.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:16.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286642+0000", 2020-08-07T15:12:16.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0954e-05 2020-08-07T15:12:16.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:16.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286693+0000", 2020-08-07T15:12:16.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0699999999999997e-07 2020-08-07T15:12:16.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:16.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286694+0000", 2020-08-07T15:12:16.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3697999999999998e-05 2020-08-07T15:12:16.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:16.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286768+0000", 2020-08-07T15:12:16.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017185900000000001 2020-08-07T15:12:16.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:16.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286940+0000", 2020-08-07T15:12:16.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051207819999999999 2020-08-07T15:12:16.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292060+0000", 2020-08-07T15:12:16.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8000000000000004e-06 2020-08-07T15:12:16.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:16.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292069+0000", 2020-08-07T15:12:16.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99959520599999996 2020-08-07T15:12:16.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291664+0000", 2020-08-07T15:12:16.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0160000000000008e-06 2020-08-07T15:12:16.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:16.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291672+0000", 2020-08-07T15:12:16.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88399374500000005 2020-08-07T15:12:16.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175666+0000", 2020-08-07T15:12:16.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101239 2020-08-07T15:12:16.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:16.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175767+0000", 2020-08-07T15:12:16.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89528059 2020-08-07T15:12:16.897 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:16.897 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:16.897 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:16.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.898 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42491 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:12:16.899 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:12:16.899 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 196.51066191999999, 2020-08-07T15:12:16.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895291936, 2020-08-07T15:12:16.899 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:16.899 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:16.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:16.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:16.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:16.900 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42491 2020-08-07T15:12:16.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.901 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:16.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:16.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:12:16.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:16.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:16.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:12:16.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0402000000000001e-05 2020-08-07T15:12:16.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:16.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280624+0000", 2020-08-07T15:12:16.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0340000000000001e-06 2020-08-07T15:12:16.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:16.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280627+0000", 2020-08-07T15:12:16.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1886999999999999e-05 2020-08-07T15:12:16.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:16.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280649+0000", 2020-08-07T15:12:16.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5430000000000004e-06 2020-08-07T15:12:16.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:16.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280654+0000", 2020-08-07T15:12:16.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032800960000000001 2020-08-07T15:12:16.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.283934+0000", 2020-08-07T15:12:16.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011264699999999999 2020-08-07T15:12:16.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:16.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284046+0000", 2020-08-07T15:12:16.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019987299999999999 2020-08-07T15:12:16.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:12:16.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284246+0000", 2020-08-07T15:12:16.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025610960000000001 2020-08-07T15:12:16.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:16.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286807+0000", 2020-08-07T15:12:16.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1409999999999999e-06 2020-08-07T15:12:16.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:16.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286814+0000", 2020-08-07T15:12:16.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0499999999999999e-07 2020-08-07T15:12:16.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:16.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286815+0000", 2020-08-07T15:12:16.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8538000000000003e-05 2020-08-07T15:12:16.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:16.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286853+0000", 2020-08-07T15:12:16.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112914 2020-08-07T15:12:16.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:16.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286966+0000", 2020-08-07T15:12:16.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051403120000000002 2020-08-07T15:12:16.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292107+0000", 2020-08-07T15:12:16.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8019999999999992e-06 2020-08-07T15:12:16.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:16.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292114+0000", 2020-08-07T15:12:16.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.999590859 2020-08-07T15:12:16.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291705+0000", 2020-08-07T15:12:16.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.174e-06 2020-08-07T15:12:16.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:16.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291713+0000", 2020-08-07T15:12:16.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88407676000000002 2020-08-07T15:12:16.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175790+0000", 2020-08-07T15:12:16.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5452999999999996e-05 2020-08-07T15:12:16.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:16.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175886+0000", 2020-08-07T15:12:16.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895291936 2020-08-07T15:12:16.922 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:16.922 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:16.922 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:16.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.922 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42492 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:12:16.923 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:12:16.923 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 196.51056886999999, 2020-08-07T15:12:16.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895293774, 2020-08-07T15:12:16.923 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:16.923 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:16.924 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:16.924 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:16.924 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:16.924 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42492 2020-08-07T15:12:16.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.925 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:16.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:16.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:12:16.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:16.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:16.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:12:16.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8272000000000001e-05 2020-08-07T15:12:16.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:16.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280735+0000", 2020-08-07T15:12:16.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2159999999999997e-06 2020-08-07T15:12:16.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:16.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280740+0000", 2020-08-07T15:12:16.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0251000000000001e-05 2020-08-07T15:12:16.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:16.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280750+0000", 2020-08-07T15:12:16.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3139999999999997e-06 2020-08-07T15:12:16.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:16.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280755+0000", 2020-08-07T15:12:16.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039343579999999998 2020-08-07T15:12:16.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284689+0000", 2020-08-07T15:12:16.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111648 2020-08-07T15:12:16.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:16.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284801+0000", 2020-08-07T15:12:16.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019563600000000001 2020-08-07T15:12:16.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:12:16.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284996+0000", 2020-08-07T15:12:16.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018636709999999999 2020-08-07T15:12:16.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:16.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286860+0000", 2020-08-07T15:12:16.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001865802 2020-08-07T15:12:16.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:16.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288726+0000", 2020-08-07T15:12:16.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0399999999999996e-07 2020-08-07T15:12:16.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:16.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288726+0000", 2020-08-07T15:12:16.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6573999999999998e-05 2020-08-07T15:12:16.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:16.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288773+0000", 2020-08-07T15:12:16.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003455283 2020-08-07T15:12:16.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292228+0000", 2020-08-07T15:12:16.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0099999999999995e-06 2020-08-07T15:12:16.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:16.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292236+0000", 2020-08-07T15:12:16.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99950963999999998 2020-08-07T15:12:16.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291746+0000", 2020-08-07T15:12:16.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0779999999999996e-06 2020-08-07T15:12:16.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:16.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291754+0000", 2020-08-07T15:12:16.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.884152508 2020-08-07T15:12:16.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175906+0000", 2020-08-07T15:12:16.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4009000000000002e-05 2020-08-07T15:12:16.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:16.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175980+0000", 2020-08-07T15:12:16.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895293774 2020-08-07T15:12:16.945 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:16.945 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:16.945 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:16.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.946 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42493 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:12:16.946 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:12:16.946 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 196.51046722300001, 2020-08-07T15:12:16.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952963789999999, 2020-08-07T15:12:16.947 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:16.947 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:16.947 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:16.947 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:16.948 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:16.948 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42493 2020-08-07T15:12:16.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.948 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:16.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:16.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:12:16.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:16.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:16.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:12:16.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9740999999999997e-05 2020-08-07T15:12:16.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:16.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280838+0000", 2020-08-07T15:12:16.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7809999999999996e-06 2020-08-07T15:12:16.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:16.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280843+0000", 2020-08-07T15:12:16.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0648999999999999e-05 2020-08-07T15:12:16.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:16.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280854+0000", 2020-08-07T15:12:16.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3669999999999999e-06 2020-08-07T15:12:16.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:16.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280858+0000", 2020-08-07T15:12:16.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004850867 2020-08-07T15:12:16.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.285709+0000", 2020-08-07T15:12:16.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110037 2020-08-07T15:12:16.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:16.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.285819+0000", 2020-08-07T15:12:16.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020935699999999999 2020-08-07T15:12:16.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:12:16.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286028+0000", 2020-08-07T15:12:16.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020900850000000002 2020-08-07T15:12:16.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:16.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288118+0000", 2020-08-07T15:12:16.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041669649999999999 2020-08-07T15:12:16.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292285+0000", 2020-08-07T15:12:16.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3820000000000004e-06 2020-08-07T15:12:16.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:16.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292294+0000", 2020-08-07T15:12:16.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949242900000002 2020-08-07T15:12:16.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291786+0000", 2020-08-07T15:12:16.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000003e-06 2020-08-07T15:12:16.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:16.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291794+0000", 2020-08-07T15:12:16.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88420652700000002 2020-08-07T15:12:16.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176000+0000", 2020-08-07T15:12:16.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4634000000000003e-05 2020-08-07T15:12:16.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:16.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176085+0000", 2020-08-07T15:12:16.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952963789999999 2020-08-07T15:12:16.972 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:16.972 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:16.972 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:16.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.973 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42494 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:12:16.973 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:12:16.973 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 196.510365124, 2020-08-07T15:12:16.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952882820000001, 2020-08-07T15:12:16.974 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:16.974 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:16.974 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:16.974 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:16.974 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:16.975 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42494 2020-08-07T15:12:16.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.975 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:16.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:16.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:12:16.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:16.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:16.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:12:16.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0117999999999997e-05 2020-08-07T15:12:16.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:16.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280941+0000", 2020-08-07T15:12:16.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1240000000000004e-06 2020-08-07T15:12:16.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:16.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280946+0000", 2020-08-07T15:12:16.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0000000000000001e-05 2020-08-07T15:12:16.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:16.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280956+0000", 2020-08-07T15:12:16.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.279e-06 2020-08-07T15:12:16.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:16.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280960+0000", 2020-08-07T15:12:16.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0062727080000000001 2020-08-07T15:12:16.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287233+0000", 2020-08-07T15:12:16.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018362900000000001 2020-08-07T15:12:16.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:16.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287416+0000", 2020-08-07T15:12:16.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029536100000000001 2020-08-07T15:12:16.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:12:16.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287712+0000", 2020-08-07T15:12:16.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046231739999999999 2020-08-07T15:12:16.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292335+0000", 2020-08-07T15:12:16.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3860000000000001e-06 2020-08-07T15:12:16.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:16.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292342+0000", 2020-08-07T15:12:16.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99948501000000001 2020-08-07T15:12:16.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291827+0000", 2020-08-07T15:12:16.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2272999999999999e-05 2020-08-07T15:12:16.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:16.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291840+0000", 2020-08-07T15:12:16.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88426668100000005 2020-08-07T15:12:16.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:16.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176106+0000", 2020-08-07T15:12:16.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2539000000000005e-05 2020-08-07T15:12:16.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:16.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176179+0000", 2020-08-07T15:12:16.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952882820000001 2020-08-07T15:12:16.992 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:16.992 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:16.992 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:16.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.993 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42495 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:12:16.993 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:12:16.993 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 196.51026403399999, 2020-08-07T15:12:16.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895286867, 2020-08-07T15:12:16.994 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:16.994 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:16.994 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:16.994 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:16.994 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:16.995 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42495 2020-08-07T15:12:16.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.995 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:16.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:16.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:12:16.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:16.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:16.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:12:16.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6690999999999997e-05 2020-08-07T15:12:16.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:16.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:16.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:16.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281038+0000", 2020-08-07T15:12:17.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7860000000000002e-06 2020-08-07T15:12:17.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:17.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281044+0000", 2020-08-07T15:12:17.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0429e-05 2020-08-07T15:12:17.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:17.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281054+0000", 2020-08-07T15:12:17.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9040000000000002e-06 2020-08-07T15:12:17.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:17.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281058+0000", 2020-08-07T15:12:17.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070941490000000001 2020-08-07T15:12:17.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288153+0000", 2020-08-07T15:12:17.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9006999999999998e-05 2020-08-07T15:12:17.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:17.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288222+0000", 2020-08-07T15:12:17.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012106400000000001 2020-08-07T15:12:17.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:12:17.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288343+0000", 2020-08-07T15:12:17.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004034752 2020-08-07T15:12:17.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292377+0000", 2020-08-07T15:12:17.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.537e-06 2020-08-07T15:12:17.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:17.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292385+0000", 2020-08-07T15:12:17.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949123500000003 2020-08-07T15:12:17.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291876+0000", 2020-08-07T15:12:17.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7729999999999999e-06 2020-08-07T15:12:17.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:17.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291884+0000", 2020-08-07T15:12:17.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88431700899999999 2020-08-07T15:12:17.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176201+0000", 2020-08-07T15:12:17.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7531e-05 2020-08-07T15:12:17.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:17.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176278+0000", 2020-08-07T15:12:17.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895286867 2020-08-07T15:12:17.068 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.068 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:17.069 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.100 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42496 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:12:17.100 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:12:17.100 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 196.510159125, 2020-08-07T15:12:17.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89526942, 2020-08-07T15:12:17.100 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:17.101 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:17.101 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:17.101 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:17.101 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:17.101 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42496 2020-08-07T15:12:17.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.118 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:17.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:17.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:12:17.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:17.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:17.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:12:17.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9227999999999997e-05 2020-08-07T15:12:17.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:17.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281146+0000", 2020-08-07T15:12:17.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1120000000000004e-06 2020-08-07T15:12:17.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:17.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281151+0000", 2020-08-07T15:12:17.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2208999999999999e-05 2020-08-07T15:12:17.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:17.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281163+0000", 2020-08-07T15:12:17.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5469999999999998e-06 2020-08-07T15:12:17.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:17.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281168+0000", 2020-08-07T15:12:17.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0076520829999999996 2020-08-07T15:12:17.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288820+0000", 2020-08-07T15:12:17.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5035000000000003e-05 2020-08-07T15:12:17.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:17.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288885+0000", 2020-08-07T15:12:17.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116507 2020-08-07T15:12:17.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:12:17.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.289001+0000", 2020-08-07T15:12:17.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034121960000000002 2020-08-07T15:12:17.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292413+0000", 2020-08-07T15:12:17.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8109999999999997e-06 2020-08-07T15:12:17.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:17.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292420+0000", 2020-08-07T15:12:17.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949504600000005 2020-08-07T15:12:17.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291915+0000", 2020-08-07T15:12:17.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3999999999999992e-06 2020-08-07T15:12:17.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:17.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291924+0000", 2020-08-07T15:12:17.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88437394599999997 2020-08-07T15:12:17.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176298+0000", 2020-08-07T15:12:17.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8300000000000007e-05 2020-08-07T15:12:17.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:17.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176366+0000", 2020-08-07T15:12:17.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89526942 2020-08-07T15:12:17.172 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.172 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:17.172 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.173 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73081 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:12:17.173 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:12:17.173 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 169.564689851, 2020-08-07T15:12:17.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0005038420000001, 2020-08-07T15:12:17.174 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:17.174 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:17.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:17.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:17.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:17.174 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73081 2020-08-07T15:12:17.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.175 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:17.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:17.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:12:17.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:17.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:17.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:12:17.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3016000000000003e-05 2020-08-07T15:12:17.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:17.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226609+0000", 2020-08-07T15:12:17.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4479999999999996e-06 2020-08-07T15:12:17.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:17.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226613+0000", 2020-08-07T15:12:17.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0189999999999995e-06 2020-08-07T15:12:17.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:17.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226622+0000", 2020-08-07T15:12:17.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5980000000000001e-06 2020-08-07T15:12:17.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:17.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226626+0000", 2020-08-07T15:12:17.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8088999999999999e-05 2020-08-07T15:12:17.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226684+0000", 2020-08-07T15:12:17.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2162e-05 2020-08-07T15:12:17.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:17.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226696+0000", 2020-08-07T15:12:17.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98675689 2020-08-07T15:12:17.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213453+0000", 2020-08-07T15:12:17.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012916800000000001 2020-08-07T15:12:17.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213582+0000", 2020-08-07T15:12:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015665899999999999 2020-08-07T15:12:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:12:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213739+0000", 2020-08-07T15:12:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049940239999999997 2020-08-07T15:12:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218733+0000", 2020-08-07T15:12:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0079774779999999997 2020-08-07T15:12:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226710+0000", 2020-08-07T15:12:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.161e-06 2020-08-07T15:12:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226711+0000", 2020-08-07T15:12:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0105000000000001e-05 2020-08-07T15:12:17.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:17.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226772+0000", 2020-08-07T15:12:17.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5633999999999997e-05 2020-08-07T15:12:17.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:17.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226817+0000", 2020-08-07T15:12:17.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7379000000000001e-05 2020-08-07T15:12:17.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:17.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226875+0000", 2020-08-07T15:12:17.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8000000000000005e-07 2020-08-07T15:12:17.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:17.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226875+0000", 2020-08-07T15:12:17.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019423199999999999 2020-08-07T15:12:17.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:17.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227070+0000", 2020-08-07T15:12:17.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0005038420000001 2020-08-07T15:12:17.220 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.220 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:17.221 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.221 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73082 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:12:17.221 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:12:17.222 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 169.56431938700001, 2020-08-07T15:12:17.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001326693, 2020-08-07T15:12:17.222 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:17.222 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:17.222 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:17.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:17.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:17.223 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73082 2020-08-07T15:12:17.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.223 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:17.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:17.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:12:17.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:17.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:17.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:12:17.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6928000000000003e-05 2020-08-07T15:12:17.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:17.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226973+0000", 2020-08-07T15:12:17.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3630000000000004e-06 2020-08-07T15:12:17.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:17.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226977+0000", 2020-08-07T15:12:17.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9659999999999996e-06 2020-08-07T15:12:17.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:17.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226985+0000", 2020-08-07T15:12:17.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5800000000000002e-06 2020-08-07T15:12:17.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:17.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226990+0000", 2020-08-07T15:12:17.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9101999999999998e-05 2020-08-07T15:12:17.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227039+0000", 2020-08-07T15:12:17.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9620000000000005e-06 2020-08-07T15:12:17.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:17.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227049+0000", 2020-08-07T15:12:17.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98722513999999995 2020-08-07T15:12:17.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214274+0000", 2020-08-07T15:12:17.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2700999999999994e-05 2020-08-07T15:12:17.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:17.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214347+0000", 2020-08-07T15:12:17.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015346899999999999 2020-08-07T15:12:17.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:12:17.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214500+0000", 2020-08-07T15:12:17.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042450309999999998 2020-08-07T15:12:17.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:17.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218745+0000", 2020-08-07T15:12:17.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008379503 2020-08-07T15:12:17.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:17.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227125+0000", 2020-08-07T15:12:17.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7899999999999998e-07 2020-08-07T15:12:17.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:17.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227126+0000", 2020-08-07T15:12:17.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1610000000000002e-05 2020-08-07T15:12:17.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:17.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227177+0000", 2020-08-07T15:12:17.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489000000000003e-05 2020-08-07T15:12:17.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:17.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227226+0000", 2020-08-07T15:12:17.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00084338600000000001 2020-08-07T15:12:17.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:17.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228069+0000", 2020-08-07T15:12:17.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7e-07 2020-08-07T15:12:17.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:17.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228069+0000", 2020-08-07T15:12:17.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019351399999999999 2020-08-07T15:12:17.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:17.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228263+0000", 2020-08-07T15:12:17.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001326693 2020-08-07T15:12:17.251 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.251 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:17.251 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.252 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73083 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:12:17.252 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:12:17.252 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 169.564225128, 2020-08-07T15:12:17.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015968209999999, 2020-08-07T15:12:17.253 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:17.253 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:17.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:17.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:17.254 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:17.254 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73083 2020-08-07T15:12:17.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.255 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:17.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:17.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:12:17.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:17.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:17.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:12:17.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1627000000000001e-05 2020-08-07T15:12:17.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:17.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227062+0000", 2020-08-07T15:12:17.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.253e-06 2020-08-07T15:12:17.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:17.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227065+0000", 2020-08-07T15:12:17.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3168e-05 2020-08-07T15:12:17.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:17.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227079+0000", 2020-08-07T15:12:17.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1549999999999999e-06 2020-08-07T15:12:17.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:17.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227082+0000", 2020-08-07T15:12:17.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5534999999999997e-05 2020-08-07T15:12:17.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227127+0000", 2020-08-07T15:12:17.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6823000000000001e-05 2020-08-07T15:12:17.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:17.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227144+0000", 2020-08-07T15:12:17.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98771770599999997 2020-08-07T15:12:17.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214862+0000", 2020-08-07T15:12:17.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8252999999999999e-05 2020-08-07T15:12:17.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:17.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214930+0000", 2020-08-07T15:12:17.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013646400000000001 2020-08-07T15:12:17.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:12:17.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215066+0000", 2020-08-07T15:12:17.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036907089999999999 2020-08-07T15:12:17.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:17.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218757+0000", 2020-08-07T15:12:17.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0085133450000000003 2020-08-07T15:12:17.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:17.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227270+0000", 2020-08-07T15:12:17.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4899999999999995e-07 2020-08-07T15:12:17.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:17.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227271+0000", 2020-08-07T15:12:17.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4555999999999997e-05 2020-08-07T15:12:17.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:17.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227326+0000", 2020-08-07T15:12:17.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024594800000000001 2020-08-07T15:12:17.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:17.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227572+0000", 2020-08-07T15:12:17.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00086125700000000004 2020-08-07T15:12:17.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:17.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228433+0000", 2020-08-07T15:12:17.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3799999999999998e-07 2020-08-07T15:12:17.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228433+0000", 2020-08-07T15:12:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019413500000000001 2020-08-07T15:12:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:17.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228627+0000", 2020-08-07T15:12:17.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015968209999999 2020-08-07T15:12:17.286 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.286 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:17.286 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.287 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73084 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:12:17.288 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:12:17.288 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 169.56413618900001, 2020-08-07T15:12:17.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0016919049999999, 2020-08-07T15:12:17.288 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:17.288 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:17.289 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:17.289 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:17.290 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:17.290 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73084 2020-08-07T15:12:17.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.290 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:17.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:17.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:12:17.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:17.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:17.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:12:17.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5565999999999997e-05 2020-08-07T15:12:17.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:17.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227165+0000", 2020-08-07T15:12:17.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2020-08-07T15:12:17.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:17.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227169+0000", 2020-08-07T15:12:17.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6850000000000001e-06 2020-08-07T15:12:17.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:17.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227176+0000", 2020-08-07T15:12:17.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1949999999999999e-06 2020-08-07T15:12:17.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:17.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227179+0000", 2020-08-07T15:12:17.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0318000000000003e-05 2020-08-07T15:12:17.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227220+0000", 2020-08-07T15:12:17.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0349e-05 2020-08-07T15:12:17.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:17.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227230+0000", 2020-08-07T15:12:17.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98832962400000002 2020-08-07T15:12:17.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215560+0000", 2020-08-07T15:12:17.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013913099999999999 2020-08-07T15:12:17.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:17.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215699+0000", 2020-08-07T15:12:17.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000200006 2020-08-07T15:12:17.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:12:17.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215899+0000", 2020-08-07T15:12:17.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028673650000000002 2020-08-07T15:12:17.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:17.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218766+0000", 2020-08-07T15:12:17.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086101779999999996 2020-08-07T15:12:17.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:17.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227376+0000", 2020-08-07T15:12:17.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0999999999999998e-07 2020-08-07T15:12:17.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:17.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227377+0000", 2020-08-07T15:12:17.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6552999999999997e-05 2020-08-07T15:12:17.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:17.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227454+0000", 2020-08-07T15:12:17.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000421717 2020-08-07T15:12:17.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:17.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227875+0000", 2020-08-07T15:12:17.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00081781300000000003 2020-08-07T15:12:17.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:17.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228693+0000", 2020-08-07T15:12:17.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1699999999999997e-07 2020-08-07T15:12:17.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:17.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228694+0000", 2020-08-07T15:12:17.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117483 2020-08-07T15:12:17.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:17.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228811+0000", 2020-08-07T15:12:17.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0016919049999999 2020-08-07T15:12:17.310 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.310 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:17.310 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.311 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73085 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:12:17.311 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:12:17.311 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 169.56400037399999, 2020-08-07T15:12:17.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0018015629999999, 2020-08-07T15:12:17.312 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:17.312 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:17.312 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:17.312 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:17.312 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:17.313 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73085 2020-08-07T15:12:17.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.313 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:17.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:17.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:12:17.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:17.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:17.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:12:17.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6570999999999997e-05 2020-08-07T15:12:17.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:17.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227322+0000", 2020-08-07T15:12:17.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5509999999999996e-06 2020-08-07T15:12:17.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:17.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227328+0000", 2020-08-07T15:12:17.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2498e-05 2020-08-07T15:12:17.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:17.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227341+0000", 2020-08-07T15:12:17.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4389999999999999e-06 2020-08-07T15:12:17.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:17.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227346+0000", 2020-08-07T15:12:17.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3711999999999998e-05 2020-08-07T15:12:17.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227390+0000", 2020-08-07T15:12:17.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1868999999999999e-05 2020-08-07T15:12:17.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:17.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227402+0000", 2020-08-07T15:12:17.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98896525099999999 2020-08-07T15:12:17.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216367+0000", 2020-08-07T15:12:17.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012573 2020-08-07T15:12:17.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:17.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216493+0000", 2020-08-07T15:12:17.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020525200000000001 2020-08-07T15:12:17.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:12:17.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216698+0000", 2020-08-07T15:12:17.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037596470000000001 2020-08-07T15:12:17.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:17.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.220458+0000", 2020-08-07T15:12:17.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070227190000000002 2020-08-07T15:12:17.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:17.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227480+0000", 2020-08-07T15:12:17.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6600000000000003e-07 2020-08-07T15:12:17.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:17.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227481+0000", 2020-08-07T15:12:17.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9898000000000003e-05 2020-08-07T15:12:17.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:17.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227521+0000", 2020-08-07T15:12:17.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039130000000000002 2020-08-07T15:12:17.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:17.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227912+0000", 2020-08-07T15:12:17.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00095502399999999996 2020-08-07T15:12:17.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:17.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228867+0000", 2020-08-07T15:12:17.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2020-08-07T15:12:17.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:17.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228867+0000", 2020-08-07T15:12:17.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018933599999999999 2020-08-07T15:12:17.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:17.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229057+0000", 2020-08-07T15:12:17.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0018015629999999 2020-08-07T15:12:17.349 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.349 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:17.352 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.352 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73086 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:12:17.353 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:12:17.354 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 169.563867878, 2020-08-07T15:12:17.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002364923, 2020-08-07T15:12:17.354 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:17.357 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:17.357 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:17.357 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:17.358 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:17.361 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73086 2020-08-07T15:12:17.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.362 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:17.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:17.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:12:17.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:17.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:17.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:12:17.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000238426 2020-08-07T15:12:17.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:17.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227626+0000", 2020-08-07T15:12:17.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9609999999999999e-06 2020-08-07T15:12:17.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:17.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227633+0000", 2020-08-07T15:12:17.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2727e-05 2020-08-07T15:12:17.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:17.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227646+0000", 2020-08-07T15:12:17.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1739999999999997e-06 2020-08-07T15:12:17.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:17.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227652+0000", 2020-08-07T15:12:17.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5004000000000001e-05 2020-08-07T15:12:17.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227737+0000", 2020-08-07T15:12:17.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2744e-05 2020-08-07T15:12:17.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:17.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227750+0000", 2020-08-07T15:12:17.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98938776500000003 2020-08-07T15:12:17.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217138+0000", 2020-08-07T15:12:17.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012361499999999999 2020-08-07T15:12:17.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:17.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217261+0000", 2020-08-07T15:12:17.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000212504 2020-08-07T15:12:17.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:12:17.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217474+0000", 2020-08-07T15:12:17.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002995036 2020-08-07T15:12:17.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:17.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.220469+0000", 2020-08-07T15:12:17.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071283249999999996 2020-08-07T15:12:17.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:17.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227597+0000", 2020-08-07T15:12:17.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0499999999999999e-07 2020-08-07T15:12:17.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:17.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227597+0000", 2020-08-07T15:12:17.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1872999999999997e-05 2020-08-07T15:12:17.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:17.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227639+0000", 2020-08-07T15:12:17.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032010200000000001 2020-08-07T15:12:17.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:17.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227959+0000", 2020-08-07T15:12:17.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015980899999999999 2020-08-07T15:12:17.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:17.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229557+0000", 2020-08-07T15:12:17.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9299999999999999e-07 2020-08-07T15:12:17.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:17.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229558+0000", 2020-08-07T15:12:17.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019477900000000001 2020-08-07T15:12:17.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:17.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229753+0000", 2020-08-07T15:12:17.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002364923 2020-08-07T15:12:17.410 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.410 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:17.410 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.412 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73087 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:12:17.412 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:12:17.412 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 169.56292672500001, 2020-08-07T15:12:17.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001896506, 2020-08-07T15:12:17.413 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:17.413 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:17.413 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:17.413 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:17.413 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:17.414 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73087 2020-08-07T15:12:17.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.414 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:17.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:17.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:12:17.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:17.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:17.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:12:17.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7354e-05 2020-08-07T15:12:17.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:17.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228386+0000", 2020-08-07T15:12:17.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8000000000000001e-06 2020-08-07T15:12:17.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:17.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228393+0000", 2020-08-07T15:12:17.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3784999999999999e-05 2020-08-07T15:12:17.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:17.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228407+0000", 2020-08-07T15:12:17.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8090000000000004e-06 2020-08-07T15:12:17.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:17.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228414+0000", 2020-08-07T15:12:17.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6907000000000003e-05 2020-08-07T15:12:17.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228461+0000", 2020-08-07T15:12:17.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8929999999999994e-06 2020-08-07T15:12:17.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:17.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228469+0000", 2020-08-07T15:12:17.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98950057700000005 2020-08-07T15:12:17.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217970+0000", 2020-08-07T15:12:17.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000132182 2020-08-07T15:12:17.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:17.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218102+0000", 2020-08-07T15:12:17.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020073399999999999 2020-08-07T15:12:17.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:12:17.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218303+0000", 2020-08-07T15:12:17.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035951120000000001 2020-08-07T15:12:17.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:17.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.221898+0000", 2020-08-07T15:12:17.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057898660000000003 2020-08-07T15:12:17.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:17.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227688+0000", 2020-08-07T15:12:17.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0200000000000002e-07 2020-08-07T15:12:17.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:17.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227688+0000", 2020-08-07T15:12:17.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1477999999999996e-05 2020-08-07T15:12:17.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:17.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227760+0000", 2020-08-07T15:12:17.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016657480000000001 2020-08-07T15:12:17.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:17.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229426+0000", 2020-08-07T15:12:17.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061290400000000005 2020-08-07T15:12:17.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:17.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230039+0000", 2020-08-07T15:12:17.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2e-07 2020-08-07T15:12:17.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:17.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230039+0000", 2020-08-07T15:12:17.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018643500000000001 2020-08-07T15:12:17.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:17.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230225+0000", 2020-08-07T15:12:17.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001896506 2020-08-07T15:12:17.456 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.456 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:17.456 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.457 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73088 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:12:17.457 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:12:17.457 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 169.56253603499999, 2020-08-07T15:12:17.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001627153, 2020-08-07T15:12:17.457 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:17.458 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:17.458 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:17.458 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:17.458 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:17.458 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73088 2020-08-07T15:12:17.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.459 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:17.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:17.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:12:17.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:17.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:17.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:12:17.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7741000000000003e-05 2020-08-07T15:12:17.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:17.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228767+0000", 2020-08-07T15:12:17.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6409999999999999e-06 2020-08-07T15:12:17.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:17.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228773+0000", 2020-08-07T15:12:17.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0874e-05 2020-08-07T15:12:17.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:17.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228784+0000", 2020-08-07T15:12:17.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1529999999999996e-06 2020-08-07T15:12:17.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:17.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228789+0000", 2020-08-07T15:12:17.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3945e-05 2020-08-07T15:12:17.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228833+0000", 2020-08-07T15:12:17.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4020000000000002e-06 2020-08-07T15:12:17.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:17.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228841+0000", 2020-08-07T15:12:17.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98996913900000005 2020-08-07T15:12:17.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218810+0000", 2020-08-07T15:12:17.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112484 2020-08-07T15:12:17.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:17.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218923+0000", 2020-08-07T15:12:17.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019891 2020-08-07T15:12:17.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:12:17.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.219122+0000", 2020-08-07T15:12:17.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027884099999999998 2020-08-07T15:12:17.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:17.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.221910+0000", 2020-08-07T15:12:17.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058753399999999997 2020-08-07T15:12:17.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:17.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227786+0000", 2020-08-07T15:12:17.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9200000000000002e-07 2020-08-07T15:12:17.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:17.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227786+0000", 2020-08-07T15:12:17.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8572000000000003e-05 2020-08-07T15:12:17.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:17.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227824+0000", 2020-08-07T15:12:17.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001651296 2020-08-07T15:12:17.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:17.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229476+0000", 2020-08-07T15:12:17.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00077620399999999996 2020-08-07T15:12:17.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:17.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230252+0000", 2020-08-07T15:12:17.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1600000000000002e-07 2020-08-07T15:12:17.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:17.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230252+0000", 2020-08-07T15:12:17.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4433999999999997e-05 2020-08-07T15:12:17.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:17.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230347+0000", 2020-08-07T15:12:17.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001627153 2020-08-07T15:12:17.556 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.556 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:17.556 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.557 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:128738 4.9 4.370c6199 (undecoded) ondisk+write+known_if_redirected e155)", 2020-08-07T15:12:17.557 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:03.204387+0000", 2020-08-07T15:12:17.557 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 133.58686838200001, 2020-08-07T15:12:17.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99077720899999999, 2020-08-07T15:12:17.557 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:17.558 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:17.558 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:17.558 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:17.558 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:17.558 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128738 2020-08-07T15:12:17.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.559 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:17.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:17.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204387+0000", 2020-08-07T15:12:17.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:17.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:17.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204387+0000", 2020-08-07T15:12:17.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4561999999999999e-05 2020-08-07T15:12:17.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:17.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204422+0000", 2020-08-07T15:12:17.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7270000000000001e-06 2020-08-07T15:12:17.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:17.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204428+0000", 2020-08-07T15:12:17.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3451e-05 2020-08-07T15:12:17.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:17.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204442+0000", 2020-08-07T15:12:17.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1589999999999998e-06 2020-08-07T15:12:17.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:17.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204448+0000", 2020-08-07T15:12:17.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3755999999999999e-05 2020-08-07T15:12:17.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204492+0000", 2020-08-07T15:12:17.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2450000000000001e-06 2020-08-07T15:12:17.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:17.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204499+0000", 2020-08-07T15:12:17.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99065224600000001 2020-08-07T15:12:17.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195151+0000", 2020-08-07T15:12:17.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3063e-05 2020-08-07T15:12:17.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:17.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195164+0000", 2020-08-07T15:12:17.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99077720899999999 2020-08-07T15:12:17.610 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.610 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:17.611 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.612 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:128742 4.9 4.69b70ab9 (undecoded) ondisk+write+known_if_redirected e155)", 2020-08-07T15:12:17.612 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:03.205224+0000", 2020-08-07T15:12:17.612 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 133.58603174800001, 2020-08-07T15:12:17.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.989964548, 2020-08-07T15:12:17.613 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:17.613 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:17.613 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:17.613 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:17.613 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:17.614 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128742 2020-08-07T15:12:17.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.614 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:17.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:17.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205224+0000", 2020-08-07T15:12:17.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:17.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:17.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205224+0000", 2020-08-07T15:12:17.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1018e-05 2020-08-07T15:12:17.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:17.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205255+0000", 2020-08-07T15:12:17.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5840000000000002e-06 2020-08-07T15:12:17.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:17.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205260+0000", 2020-08-07T15:12:17.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2388999999999999e-05 2020-08-07T15:12:17.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:17.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205273+0000", 2020-08-07T15:12:17.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3799999999999999e-06 2020-08-07T15:12:17.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:17.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205279+0000", 2020-08-07T15:12:17.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6684999999999998e-05 2020-08-07T15:12:17.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205326+0000", 2020-08-07T15:12:17.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6200000000000004e-06 2020-08-07T15:12:17.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:17.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205332+0000", 2020-08-07T15:12:17.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98984944600000002 2020-08-07T15:12:17.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195181+0000", 2020-08-07T15:12:17.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4259999999999997e-06 2020-08-07T15:12:17.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:17.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195188+0000", 2020-08-07T15:12:17.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.989964548 2020-08-07T15:12:17.624 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.624 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:17.624 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.625 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:128743 4.9 4.1824cba9 (undecoded) ondisk+write+known_if_redirected e155)", 2020-08-07T15:12:17.625 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:03.205500+0000", 2020-08-07T15:12:17.625 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 133.58575588599999, 2020-08-07T15:12:17.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98971761999999996, 2020-08-07T15:12:17.626 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:17.626 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:17.626 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:17.626 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:17.627 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:17.627 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128743 2020-08-07T15:12:17.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.627 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:17.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:17.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205500+0000", 2020-08-07T15:12:17.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:17.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:17.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205500+0000", 2020-08-07T15:12:17.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6441000000000001e-05 2020-08-07T15:12:17.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:17.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205516+0000", 2020-08-07T15:12:17.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4429999999999999e-06 2020-08-07T15:12:17.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:17.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205520+0000", 2020-08-07T15:12:17.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4590000000000003e-06 2020-08-07T15:12:17.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:17.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205526+0000", 2020-08-07T15:12:17.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0429999999999999e-06 2020-08-07T15:12:17.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:17.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205529+0000", 2020-08-07T15:12:17.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1313000000000003e-05 2020-08-07T15:12:17.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205560+0000", 2020-08-07T15:12:17.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.934e-06 2020-08-07T15:12:17.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:17.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205567+0000", 2020-08-07T15:12:17.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98964017599999998 2020-08-07T15:12:17.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:17.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195208+0000", 2020-08-07T15:12:17.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8109999999999997e-06 2020-08-07T15:12:17.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:17.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:17.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:17.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195217+0000", 2020-08-07T15:12:17.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98971761999999996 2020-08-07T15:12:17.655 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.655 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:17.655 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.655 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:17.656 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:17.656 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:17.657 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:12:17.659 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 5 16 7717 7701 12.0309 12.5703 0.0107638 0.0103502 2020-08-07T15:12:17.660 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:12:17.866 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:17.863+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:17.882 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:17.882 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:12:17.883 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2020-08-07T15:12:17.883 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:17.895 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:12:17.986 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:17.984+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:18.101 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:18.102 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:12:18.103 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:12:18.103 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:12:18.103 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:18.113 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:12:18.261 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:18.251+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:18.263 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:18.251+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:18.263 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:18.251+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:18.263 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:18.251+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:18.275 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 6 16 9046 9030 11.7561 10.3828 0.00572515 0.0106025 2020-08-07T15:12:18.276 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:18.274+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:18.319 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:18.319 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:12:18.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:12:18.330 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:12:18.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.330 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106232 3.a 3:5599d7e6:::benchmark_data_smithi074_35447_object13278:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:12:18.333 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:12:18.333 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 270.96612248500003, 2020-08-07T15:12:18.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52578228699999996, 2020-08-07T15:12:18.334 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:18.334 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:18.334 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:18.334 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:12:18.334 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:12:18.335 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106232 2020-08-07T15:12:18.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.336 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:18.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:18.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:12:18.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:18.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:18.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:12:18.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4214999999999997e-05 2020-08-07T15:12:18.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:18.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345641+0000", 2020-08-07T15:12:18.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9279999999999993e-06 2020-08-07T15:12:18.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:18.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345649+0000", 2020-08-07T15:12:18.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4391e-05 2020-08-07T15:12:18.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:18.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345664+0000", 2020-08-07T15:12:18.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7170000000000002e-06 2020-08-07T15:12:18.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:18.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345670+0000", 2020-08-07T15:12:18.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.459e-05 2020-08-07T15:12:18.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:18.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345695+0000", 2020-08-07T15:12:18.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6070999999999999e-05 2020-08-07T15:12:18.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:18.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345751+0000", 2020-08-07T15:12:18.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010069300000000001 2020-08-07T15:12:18.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:12:18.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345852+0000", 2020-08-07T15:12:18.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013512299999999999 2020-08-07T15:12:18.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:18.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.347203+0000", 2020-08-07T15:12:18.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0844000000000001e-05 2020-08-07T15:12:18.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:18.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.347274+0000", 2020-08-07T15:12:18.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52399471600000003 2020-08-07T15:12:18.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:18.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871269+0000", 2020-08-07T15:12:18.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.311e-06 2020-08-07T15:12:18.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:18.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871270+0000", 2020-08-07T15:12:18.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3509000000000004e-05 2020-08-07T15:12:18.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:18.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871343+0000", 2020-08-07T15:12:18.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6072000000000002e-05 2020-08-07T15:12:18.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:18.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871389+0000", 2020-08-07T15:12:18.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52578228699999996 2020-08-07T15:12:18.416 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:18.416 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:18.416 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:18.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.417 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106232 3.a e59/20 ondisk, result = 0)", 2020-08-07T15:12:18.417 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:12:18.417 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 270.96307631000002, 2020-08-07T15:12:18.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52274359299999995, 2020-08-07T15:12:18.417 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:18.418 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:18.418 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:18.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:18.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:12:18.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:18.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:18.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:12:18.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5011000000000001e-05 2020-08-07T15:12:18.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:18.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348668+0000", 2020-08-07T15:12:18.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1959999999999998e-06 2020-08-07T15:12:18.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:18.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348671+0000", 2020-08-07T15:12:18.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52253456499999995 2020-08-07T15:12:18.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:18.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871205+0000", 2020-08-07T15:12:18.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0230000000000003e-06 2020-08-07T15:12:18.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:18.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871209+0000", 2020-08-07T15:12:18.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0708999999999999e-05 2020-08-07T15:12:18.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:18.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871250+0000", 2020-08-07T15:12:18.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0158e-05 2020-08-07T15:12:18.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:18.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871260+0000", 2020-08-07T15:12:18.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013693099999999999 2020-08-07T15:12:18.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:18.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871397+0000", 2020-08-07T15:12:18.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52274359299999995 2020-08-07T15:12:18.427 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:18.427 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:18.427 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:18.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.428 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:30976 4.5 e98/81 ondisk, result = 0)", 2020-08-07T15:12:18.428 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:45.736493+0000", 2020-08-07T15:12:18.428 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 212.575236585, 2020-08-07T15:12:18.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54677504399999999, 2020-08-07T15:12:18.429 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:18.429 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:18.429 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:18.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:18.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736493+0000", 2020-08-07T15:12:18.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:18.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:18.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736493+0000", 2020-08-07T15:12:18.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.012e-05 2020-08-07T15:12:18.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:18.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736523+0000", 2020-08-07T15:12:18.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6669999999999998e-06 2020-08-07T15:12:18.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:18.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736527+0000", 2020-08-07T15:12:18.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54657333200000002 2020-08-07T15:12:18.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:18.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283100+0000", 2020-08-07T15:12:18.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.549e-06 2020-08-07T15:12:18.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:18.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283105+0000", 2020-08-07T15:12:18.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1853000000000006e-05 2020-08-07T15:12:18.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:18.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283167+0000", 2020-08-07T15:12:18.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015e-05 2020-08-07T15:12:18.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:18.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283177+0000", 2020-08-07T15:12:18.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1508000000000005e-05 2020-08-07T15:12:18.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:18.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283268+0000", 2020-08-07T15:12:18.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54677504399999999 2020-08-07T15:12:18.459 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:18.459 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:18.459 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:18.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.461 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:37048 4.5 4:a403bd70:::benchmark_data_smithi074_44586_object4630:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e106)", 2020-08-07T15:12:18.461 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:54.237330+0000", 2020-08-07T15:12:18.461 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 204.074399508, 2020-08-07T15:12:18.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.474059283, 2020-08-07T15:12:18.478 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:18.478 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:18.478 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:18.478 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:18.479 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:18.479 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 37048 2020-08-07T15:12:18.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.479 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:18.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:18.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237330+0000", 2020-08-07T15:12:18.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:18.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:18.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237330+0000", 2020-08-07T15:12:18.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0713999999999998e-05 2020-08-07T15:12:18.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:18.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237361+0000", 2020-08-07T15:12:18.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.756e-06 2020-08-07T15:12:18.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:18.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237364+0000", 2020-08-07T15:12:18.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.694e-06 2020-08-07T15:12:18.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:18.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237369+0000", 2020-08-07T15:12:18.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3709999999999998e-06 2020-08-07T15:12:18.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:18.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237372+0000", 2020-08-07T15:12:18.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015644700000000001 2020-08-07T15:12:18.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:18.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.238936+0000", 2020-08-07T15:12:18.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9940999999999998e-05 2020-08-07T15:12:18.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:18.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.239026+0000", 2020-08-07T15:12:18.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017423800000000001 2020-08-07T15:12:18.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,7", 2020-08-07T15:12:18.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.239200+0000", 2020-08-07T15:12:18.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041267530000000004 2020-08-07T15:12:18.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:18.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.243327+0000", 2020-08-07T15:12:18.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7700000000000004e-07 2020-08-07T15:12:18.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:18.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.243328+0000", 2020-08-07T15:12:18.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1211999999999997e-05 2020-08-07T15:12:18.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:18.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.243379+0000", 2020-08-07T15:12:18.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075155150000000004 2020-08-07T15:12:18.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:18.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.250894+0000", 2020-08-07T15:12:18.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.460450313 2020-08-07T15:12:18.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:18.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.711345+0000", 2020-08-07T15:12:18.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1990000000000001e-06 2020-08-07T15:12:18.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:18.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.711346+0000", 2020-08-07T15:12:18.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3529999999999998e-05 2020-08-07T15:12:18.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:18.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.711389+0000", 2020-08-07T15:12:18.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.474059283 2020-08-07T15:12:18.590 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:18.591 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:18.591 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:18.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.591 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118153 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:18.592 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.222760+0000", 2020-08-07T15:12:18.596 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.088969384999999, 2020-08-07T15:12:18.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0520728660000001, 2020-08-07T15:12:18.596 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:18.597 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:18.597 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:18.597 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:18.598 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:18.599 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118153 2020-08-07T15:12:18.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.599 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:18.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:18.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222760+0000", 2020-08-07T15:12:18.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:18.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:18.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222760+0000", 2020-08-07T15:12:18.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1894000000000002e-05 2020-08-07T15:12:18.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:18.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222812+0000", 2020-08-07T15:12:18.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4880000000000001e-06 2020-08-07T15:12:18.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:18.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222820+0000", 2020-08-07T15:12:18.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6744000000000001e-05 2020-08-07T15:12:18.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:18.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222836+0000", 2020-08-07T15:12:18.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5470000000000002e-06 2020-08-07T15:12:18.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:18.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222846+0000", 2020-08-07T15:12:18.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9478e-05 2020-08-07T15:12:18.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:18.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222925+0000", 2020-08-07T15:12:18.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0966e-05 2020-08-07T15:12:18.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2020-08-07T15:12:18.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222936+0000", 2020-08-07T15:12:18.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002974099 2020-08-07T15:12:18.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:18.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.225910+0000", 2020-08-07T15:12:18.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5197e-05 2020-08-07T15:12:18.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:18.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.225926+0000", 2020-08-07T15:12:18.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.02944747 2020-08-07T15:12:18.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:18.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.255373+0000", 2020-08-07T15:12:18.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022597200000000001 2020-08-07T15:12:18.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:18.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.255599+0000", 2020-08-07T15:12:18.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000194415 2020-08-07T15:12:18.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:18.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.255794+0000", 2020-08-07T15:12:18.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0076831360000000001 2020-08-07T15:12:18.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:18.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263477+0000", 2020-08-07T15:12:18.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01099552 2020-08-07T15:12:18.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:18.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274472+0000", 2020-08-07T15:12:18.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2948999999999998e-05 2020-08-07T15:12:18.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:18.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274515+0000", 2020-08-07T15:12:18.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.569999999999999e-07 2020-08-07T15:12:18.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:18.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274516+0000", 2020-08-07T15:12:18.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2750999999999997e-05 2020-08-07T15:12:18.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:18.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274569+0000", 2020-08-07T15:12:18.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4109000000000001e-05 2020-08-07T15:12:18.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:18.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274613+0000", 2020-08-07T15:12:18.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0432999999999997e-05 2020-08-07T15:12:18.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:18.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274653+0000", 2020-08-07T15:12:18.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5900000000000002e-07 2020-08-07T15:12:18.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:18.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274654+0000", 2020-08-07T15:12:18.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017928200000000001 2020-08-07T15:12:18.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:18.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274833+0000", 2020-08-07T15:12:18.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0520728660000001 2020-08-07T15:12:18.722 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:18.723 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:18.723 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:18.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.723 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118154 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:18.724 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.222899+0000", 2020-08-07T15:12:18.724 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.088831083000002, 2020-08-07T15:12:18.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0539106499999999, 2020-08-07T15:12:18.724 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:18.724 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:18.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:18.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:18.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:18.725 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118154 2020-08-07T15:12:18.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.726 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:18.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:18.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222899+0000", 2020-08-07T15:12:18.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:18.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:18.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222899+0000", 2020-08-07T15:12:18.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4440000000000001e-05 2020-08-07T15:12:18.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:18.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222953+0000", 2020-08-07T15:12:18.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4560000000000003e-06 2020-08-07T15:12:18.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:18.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222960+0000", 2020-08-07T15:12:18.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.201e-05 2020-08-07T15:12:18.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:18.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222972+0000", 2020-08-07T15:12:18.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4779999999999996e-06 2020-08-07T15:12:18.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:18.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222979+0000", 2020-08-07T15:12:18.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2364000000000002e-05 2020-08-07T15:12:18.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:18.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223031+0000", 2020-08-07T15:12:18.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7779999999999996e-06 2020-08-07T15:12:18.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223040+0000", 2020-08-07T15:12:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029381720000000002 2020-08-07T15:12:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.225978+0000", 2020-08-07T15:12:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1806e-05 2020-08-07T15:12:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.225990+0000", 2020-08-07T15:12:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0316258110000001 2020-08-07T15:12:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.257616+0000", 2020-08-07T15:12:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000143759 2020-08-07T15:12:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.257760+0000", 2020-08-07T15:12:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000296249 2020-08-07T15:12:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.258056+0000", 2020-08-07T15:12:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054347309999999999 2020-08-07T15:12:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263491+0000", 2020-08-07T15:12:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011408567999999999 2020-08-07T15:12:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274899+0000", 2020-08-07T15:12:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030791400000000001 2020-08-07T15:12:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275207+0000", 2020-08-07T15:12:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6499999999999998e-07 2020-08-07T15:12:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275208+0000", 2020-08-07T15:12:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3047999999999998e-05 2020-08-07T15:12:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275261+0000", 2020-08-07T15:12:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000419706 2020-08-07T15:12:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275681+0000", 2020-08-07T15:12:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091386099999999997 2020-08-07T15:12:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276594+0000", 2020-08-07T15:12:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6000000000000001e-06 2020-08-07T15:12:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276597+0000", 2020-08-07T15:12:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000212134 2020-08-07T15:12:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276809+0000", 2020-08-07T15:12:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0539106499999999 2020-08-07T15:12:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118155 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223018+0000", 2020-08-07T15:12:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.088711273000001, 2020-08-07T15:12:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0540362350000001, 2020-08-07T15:12:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118155 2020-08-07T15:12:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.787 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:18.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:18.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223018+0000", 2020-08-07T15:12:18.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:18.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:18.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223018+0000", 2020-08-07T15:12:18.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8291999999999999e-05 2020-08-07T15:12:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223057+0000", 2020-08-07T15:12:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.805e-06 2020-08-07T15:12:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223060+0000", 2020-08-07T15:12:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0050000000000004e-06 2020-08-07T15:12:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223068+0000", 2020-08-07T15:12:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4846e-05 2020-08-07T15:12:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223083+0000", 2020-08-07T15:12:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8695000000000005e-05 2020-08-07T15:12:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223162+0000", 2020-08-07T15:12:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4919999999999996e-06 2020-08-07T15:12:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:18.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223169+0000", 2020-08-07T15:12:18.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028562219999999998 2020-08-07T15:12:18.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:18.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226026+0000", 2020-08-07T15:12:18.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.027e-05 2020-08-07T15:12:18.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:18.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226036+0000", 2020-08-07T15:12:18.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0327165199999999 2020-08-07T15:12:18.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:18.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.258752+0000", 2020-08-07T15:12:18.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012614899999999999 2020-08-07T15:12:18.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:18.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.258879+0000", 2020-08-07T15:12:18.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000258008 2020-08-07T15:12:18.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:18.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.259137+0000", 2020-08-07T15:12:18.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0067332820000000002 2020-08-07T15:12:18.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:18.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.265870+0000", 2020-08-07T15:12:18.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0090796000000000002 2020-08-07T15:12:18.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:18.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274950+0000", 2020-08-07T15:12:18.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000632601 2020-08-07T15:12:18.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275582+0000", 2020-08-07T15:12:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8999999999999997e-07 2020-08-07T15:12:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275583+0000", 2020-08-07T15:12:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6400999999999998e-05 2020-08-07T15:12:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:18.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275629+0000", 2020-08-07T15:12:18.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014887 2020-08-07T15:12:18.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:18.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275778+0000", 2020-08-07T15:12:18.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010622559999999999 2020-08-07T15:12:18.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:18.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276840+0000", 2020-08-07T15:12:18.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2199999999999999e-07 2020-08-07T15:12:18.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:18.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276841+0000", 2020-08-07T15:12:18.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000214009 2020-08-07T15:12:18.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:18.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277055+0000", 2020-08-07T15:12:18.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0540362350000001 2020-08-07T15:12:18.939 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:18.939 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:18.939 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:18.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.940 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118156 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:18.940 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223123+0000", 2020-08-07T15:12:18.940 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.088606407, 2020-08-07T15:12:18.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0542400009999999, 2020-08-07T15:12:18.941 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:18.941 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:18.941 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:18.941 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:18.960 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:18.960 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118156 2020-08-07T15:12:18.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.960 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:18.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223123+0000", 2020-08-07T15:12:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223123+0000", 2020-08-07T15:12:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2212999999999995e-05 2020-08-07T15:12:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:18.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223195+0000", 2020-08-07T15:12:18.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8709999999999999e-06 2020-08-07T15:12:18.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223202+0000", 2020-08-07T15:12:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3509999999999999e-05 2020-08-07T15:12:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223216+0000", 2020-08-07T15:12:18.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2729999999999997e-06 2020-08-07T15:12:18.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:18.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223223+0000", 2020-08-07T15:12:18.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6728000000000003e-05 2020-08-07T15:12:18.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:18.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223280+0000", 2020-08-07T15:12:18.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1046999999999999e-05 2020-08-07T15:12:18.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:18.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223291+0000", 2020-08-07T15:12:18.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027721030000000002 2020-08-07T15:12:18.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:18.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226063+0000", 2020-08-07T15:12:18.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1809000000000004e-05 2020-08-07T15:12:18.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:18.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226125+0000", 2020-08-07T15:12:18.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0337493870000001 2020-08-07T15:12:18.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:18.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:18.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.259874+0000", 2020-08-07T15:12:19.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000133974 2020-08-07T15:12:19.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:19.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.260008+0000", 2020-08-07T15:12:19.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00064730600000000005 2020-08-07T15:12:19.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:19.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.260655+0000", 2020-08-07T15:12:19.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052233740000000002 2020-08-07T15:12:19.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:19.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.265879+0000", 2020-08-07T15:12:19.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.009433238 2020-08-07T15:12:19.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275312+0000", 2020-08-07T15:12:19.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050955399999999995 2020-08-07T15:12:19.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275822+0000", 2020-08-07T15:12:19.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.58e-07 2020-08-07T15:12:19.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:19.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275822+0000", 2020-08-07T15:12:19.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5469000000000006e-05 2020-08-07T15:12:19.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:19.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275887+0000", 2020-08-07T15:12:19.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000371212 2020-08-07T15:12:19.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276259+0000", 2020-08-07T15:12:19.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00085558499999999996 2020-08-07T15:12:19.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277114+0000", 2020-08-07T15:12:19.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2020-08-07T15:12:19.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:19.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277115+0000", 2020-08-07T15:12:19.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024832000000000001 2020-08-07T15:12:19.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:19.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277363+0000", 2020-08-07T15:12:19.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0542400009999999 2020-08-07T15:12:19.025 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.025 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:19.026 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.026 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118157 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:19.027 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223267+0000", 2020-08-07T15:12:19.027 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.088462172, 2020-08-07T15:12:19.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0543752369999999, 2020-08-07T15:12:19.027 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:19.027 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:19.027 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:19.028 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:19.028 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:19.028 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118157 2020-08-07T15:12:19.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.028 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:19.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:19.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223267+0000", 2020-08-07T15:12:19.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:19.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:19.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223267+0000", 2020-08-07T15:12:19.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9616000000000001e-05 2020-08-07T15:12:19.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:19.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223317+0000", 2020-08-07T15:12:19.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1359999999999996e-06 2020-08-07T15:12:19.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:19.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223322+0000", 2020-08-07T15:12:19.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0529e-05 2020-08-07T15:12:19.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:19.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223333+0000", 2020-08-07T15:12:19.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1059999999999999e-06 2020-08-07T15:12:19.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:19.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223338+0000", 2020-08-07T15:12:19.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8609000000000002e-05 2020-08-07T15:12:19.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223386+0000", 2020-08-07T15:12:19.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3830000000000003e-06 2020-08-07T15:12:19.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:19.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223394+0000", 2020-08-07T15:12:19.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027643580000000002 2020-08-07T15:12:19.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226158+0000", 2020-08-07T15:12:19.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6809999999999993e-06 2020-08-07T15:12:19.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:19.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226168+0000", 2020-08-07T15:12:19.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.035171356 2020-08-07T15:12:19.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.261339+0000", 2020-08-07T15:12:19.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000130514 2020-08-07T15:12:19.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:19.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.261470+0000", 2020-08-07T15:12:19.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025685699999999998 2020-08-07T15:12:19.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:19.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.261727+0000", 2020-08-07T15:12:19.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075810410000000002 2020-08-07T15:12:19.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:19.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269308+0000", 2020-08-07T15:12:19.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064252490000000001 2020-08-07T15:12:19.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275733+0000", 2020-08-07T15:12:19.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020602000000000001 2020-08-07T15:12:19.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275939+0000", 2020-08-07T15:12:19.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999999e-07 2020-08-07T15:12:19.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:19.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275939+0000", 2020-08-07T15:12:19.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7071999999999997e-05 2020-08-07T15:12:19.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:19.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276016+0000", 2020-08-07T15:12:19.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029090700000000003 2020-08-07T15:12:19.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276307+0000", 2020-08-07T15:12:19.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001114929 2020-08-07T15:12:19.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277422+0000", 2020-08-07T15:12:19.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1200000000000003e-07 2020-08-07T15:12:19.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:19.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277423+0000", 2020-08-07T15:12:19.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021982600000000001 2020-08-07T15:12:19.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:19.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277643+0000", 2020-08-07T15:12:19.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0543752369999999 2020-08-07T15:12:19.084 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.084 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:19.084 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.086 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118158 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:19.086 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223375+0000", 2020-08-07T15:12:19.087 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.088354436000003, 2020-08-07T15:12:19.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0545579519999999, 2020-08-07T15:12:19.087 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:19.087 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:19.087 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:19.088 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:19.088 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:19.088 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118158 2020-08-07T15:12:19.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.088 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:19.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:19.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223375+0000", 2020-08-07T15:12:19.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:19.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:19.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223375+0000", 2020-08-07T15:12:19.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7657000000000001e-05 2020-08-07T15:12:19.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:19.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223423+0000", 2020-08-07T15:12:19.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0810000000000001e-06 2020-08-07T15:12:19.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:19.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223428+0000", 2020-08-07T15:12:19.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7839999999999998e-06 2020-08-07T15:12:19.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:19.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223438+0000", 2020-08-07T15:12:19.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.634e-06 2020-08-07T15:12:19.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:19.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223442+0000", 2020-08-07T15:12:19.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0154999999999998e-05 2020-08-07T15:12:19.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223482+0000", 2020-08-07T15:12:19.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9579999999999998e-06 2020-08-07T15:12:19.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:19.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223487+0000", 2020-08-07T15:12:19.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027060999999999999 2020-08-07T15:12:19.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226194+0000", 2020-08-07T15:12:19.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1311999999999999e-05 2020-08-07T15:12:19.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:19.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226205+0000", 2020-08-07T15:12:19.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0361494609999999 2020-08-07T15:12:19.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.262354+0000", 2020-08-07T15:12:19.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9950000000000004e-05 2020-08-07T15:12:19.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:19.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.262454+0000", 2020-08-07T15:12:19.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000244093 2020-08-07T15:12:19.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:19.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.262698+0000", 2020-08-07T15:12:19.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0066176150000000003 2020-08-07T15:12:19.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:19.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269316+0000", 2020-08-07T15:12:19.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006768998 2020-08-07T15:12:19.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276085+0000", 2020-08-07T15:12:19.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9524000000000002e-05 2020-08-07T15:12:19.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276135+0000", 2020-08-07T15:12:19.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6100000000000001e-07 2020-08-07T15:12:19.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:19.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276135+0000", 2020-08-07T15:12:19.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1214e-05 2020-08-07T15:12:19.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:19.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276206+0000", 2020-08-07T15:12:19.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014914999999999999 2020-08-07T15:12:19.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276355+0000", 2020-08-07T15:12:19.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013427770000000001 2020-08-07T15:12:19.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277698+0000", 2020-08-07T15:12:19.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8599999999999998e-07 2020-08-07T15:12:19.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:19.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277699+0000", 2020-08-07T15:12:19.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023444199999999999 2020-08-07T15:12:19.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:19.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277933+0000", 2020-08-07T15:12:19.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0545579519999999 2020-08-07T15:12:19.163 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.165 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:19.165 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.168 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118159 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:19.170 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223476+0000", 2020-08-07T15:12:19.170 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.088253270000003, 2020-08-07T15:12:19.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0547920749999999, 2020-08-07T15:12:19.171 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:19.171 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:19.172 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:19.172 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:19.172 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:19.172 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118159 2020-08-07T15:12:19.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.173 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:19.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:19.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223476+0000", 2020-08-07T15:12:19.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:19.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:19.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223476+0000", 2020-08-07T15:12:19.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6094000000000003e-05 2020-08-07T15:12:19.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:19.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223522+0000", 2020-08-07T15:12:19.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6650000000000002e-06 2020-08-07T15:12:19.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:19.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223527+0000", 2020-08-07T15:12:19.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5629999999999998e-06 2020-08-07T15:12:19.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:19.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223537+0000", 2020-08-07T15:12:19.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6810000000000003e-06 2020-08-07T15:12:19.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:19.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223542+0000", 2020-08-07T15:12:19.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6680000000000001e-05 2020-08-07T15:12:19.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223579+0000", 2020-08-07T15:12:19.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2020-08-07T15:12:19.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:19.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223586+0000", 2020-08-07T15:12:19.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026469250000000001 2020-08-07T15:12:19.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226233+0000", 2020-08-07T15:12:19.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3130000000000004e-06 2020-08-07T15:12:19.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:19.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226242+0000", 2020-08-07T15:12:19.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.037297957 2020-08-07T15:12:19.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263540+0000", 2020-08-07T15:12:19.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4307000000000004e-05 2020-08-07T15:12:19.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:19.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263634+0000", 2020-08-07T15:12:19.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019544300000000001 2020-08-07T15:12:19.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:19.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263830+0000", 2020-08-07T15:12:19.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054939949999999998 2020-08-07T15:12:19.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:19.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269324+0000", 2020-08-07T15:12:19.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070859139999999996 2020-08-07T15:12:19.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276410+0000", 2020-08-07T15:12:19.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0924000000000002e-05 2020-08-07T15:12:19.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276461+0000", 2020-08-07T15:12:19.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4099999999999998e-07 2020-08-07T15:12:19.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:19.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276461+0000", 2020-08-07T15:12:19.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8027000000000004e-05 2020-08-07T15:12:19.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:19.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276539+0000", 2020-08-07T15:12:19.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001458453 2020-08-07T15:12:19.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277998+0000", 2020-08-07T15:12:19.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7283000000000001e-05 2020-08-07T15:12:19.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278045+0000", 2020-08-07T15:12:19.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9100000000000004e-07 2020-08-07T15:12:19.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:19.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278046+0000", 2020-08-07T15:12:19.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022290400000000001 2020-08-07T15:12:19.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:19.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278268+0000", 2020-08-07T15:12:19.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0547920749999999 2020-08-07T15:12:19.221 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.221 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:19.222 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.222 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118160 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:19.223 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223579+0000", 2020-08-07T15:12:19.223 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.088150288999998, 2020-08-07T15:12:19.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0553689980000001, 2020-08-07T15:12:19.223 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:19.223 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:19.224 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:19.224 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:19.224 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:19.225 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118160 2020-08-07T15:12:19.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.225 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:19.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:19.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223579+0000", 2020-08-07T15:12:19.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:19.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:19.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223579+0000", 2020-08-07T15:12:19.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5401000000000001e-05 2020-08-07T15:12:19.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:19.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223625+0000", 2020-08-07T15:12:19.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9069999999999998e-06 2020-08-07T15:12:19.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:19.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223630+0000", 2020-08-07T15:12:19.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4800000000000007e-06 2020-08-07T15:12:19.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:19.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223639+0000", 2020-08-07T15:12:19.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5410000000000002e-06 2020-08-07T15:12:19.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:19.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223644+0000", 2020-08-07T15:12:19.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1476000000000002e-05 2020-08-07T15:12:19.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223675+0000", 2020-08-07T15:12:19.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0240000000000003e-06 2020-08-07T15:12:19.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:19.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223681+0000", 2020-08-07T15:12:19.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025870820000000001 2020-08-07T15:12:19.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226268+0000", 2020-08-07T15:12:19.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0540000000000001e-05 2020-08-07T15:12:19.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:19.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226279+0000", 2020-08-07T15:12:19.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.038209656 2020-08-07T15:12:19.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.264488+0000", 2020-08-07T15:12:19.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029908499999999999 2020-08-07T15:12:19.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:19.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.264788+0000", 2020-08-07T15:12:19.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000216997 2020-08-07T15:12:19.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:19.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.265005+0000", 2020-08-07T15:12:19.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0060308230000000003 2020-08-07T15:12:19.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:19.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271035+0000", 2020-08-07T15:12:19.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0072889499999999998 2020-08-07T15:12:19.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278324+0000", 2020-08-07T15:12:19.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4982999999999999e-05 2020-08-07T15:12:19.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278369+0000", 2020-08-07T15:12:19.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4000000000000002e-07 2020-08-07T15:12:19.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:19.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278370+0000", 2020-08-07T15:12:19.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7656000000000003e-05 2020-08-07T15:12:19.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:19.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278447+0000", 2020-08-07T15:12:19.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000277416 2020-08-07T15:12:19.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278725+0000", 2020-08-07T15:12:19.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2345000000000001e-05 2020-08-07T15:12:19.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278777+0000", 2020-08-07T15:12:19.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8500000000000002e-07 2020-08-07T15:12:19.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:19.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278778+0000", 2020-08-07T15:12:19.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017071100000000001 2020-08-07T15:12:19.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:19.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278948+0000", 2020-08-07T15:12:19.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0553689980000001 2020-08-07T15:12:19.265 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.265 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:19.266 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.266 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118441 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:19.266 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223679+0000", 2020-08-07T15:12:19.267 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.088050273999997, 2020-08-07T15:12:19.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057502779, 2020-08-07T15:12:19.267 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118441 2020-08-07T15:12:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223679+0000", 2020-08-07T15:12:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223679+0000", 2020-08-07T15:12:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5559999999999997e-05 2020-08-07T15:12:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223725+0000", 2020-08-07T15:12:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1250000000000001e-06 2020-08-07T15:12:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223730+0000", 2020-08-07T15:12:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0046e-05 2020-08-07T15:12:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223740+0000", 2020-08-07T15:12:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5979999999999999e-06 2020-08-07T15:12:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223745+0000", 2020-08-07T15:12:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7234999999999998e-05 2020-08-07T15:12:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223782+0000", 2020-08-07T15:12:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7789999999999998e-06 2020-08-07T15:12:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223789+0000", 2020-08-07T15:12:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002516169 2020-08-07T15:12:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226305+0000", 2020-08-07T15:12:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3860000000000007e-06 2020-08-07T15:12:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226313+0000", 2020-08-07T15:12:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0396126269999999 2020-08-07T15:12:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.265926+0000", 2020-08-07T15:12:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00079104700000000002 2020-08-07T15:12:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.266717+0000", 2020-08-07T15:12:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024298599999999999 2020-08-07T15:12:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.266960+0000", 2020-08-07T15:12:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040855509999999998 2020-08-07T15:12:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271045+0000", 2020-08-07T15:12:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0074549170000000001 2020-08-07T15:12:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278500+0000", 2020-08-07T15:12:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8816999999999997e-05 2020-08-07T15:12:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278549+0000", 2020-08-07T15:12:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9000000000000002e-07 2020-08-07T15:12:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:19.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278550+0000", 2020-08-07T15:12:19.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0120999999999997e-05 2020-08-07T15:12:19.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:19.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278600+0000", 2020-08-07T15:12:19.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00092631699999999996 2020-08-07T15:12:19.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279526+0000", 2020-08-07T15:12:19.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014504520000000001 2020-08-07T15:12:19.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280976+0000", 2020-08-07T15:12:19.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6899999999999997e-07 2020-08-07T15:12:19.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:19.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280977+0000", 2020-08-07T15:12:19.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020508700000000001 2020-08-07T15:12:19.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:19.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281182+0000", 2020-08-07T15:12:19.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057502779 2020-08-07T15:12:19.385 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.386 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:19.386 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.387 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118442 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:19.387 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223780+0000", 2020-08-07T15:12:19.388 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.087949404, 2020-08-07T15:12:19.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0575529850000001, 2020-08-07T15:12:19.390 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:19.391 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:19.391 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:19.391 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:19.392 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:19.392 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118442 2020-08-07T15:12:19.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.393 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:19.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:19.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223780+0000", 2020-08-07T15:12:19.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:19.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:19.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223780+0000", 2020-08-07T15:12:19.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0340999999999998e-05 2020-08-07T15:12:19.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:19.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223831+0000", 2020-08-07T15:12:19.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3499999999999996e-06 2020-08-07T15:12:19.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:19.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223836+0000", 2020-08-07T15:12:19.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0414999999999999e-05 2020-08-07T15:12:19.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:19.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223846+0000", 2020-08-07T15:12:19.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9239999999999998e-06 2020-08-07T15:12:19.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:19.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223851+0000", 2020-08-07T15:12:19.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5946000000000003e-05 2020-08-07T15:12:19.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223907+0000", 2020-08-07T15:12:19.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6189999999999996e-06 2020-08-07T15:12:19.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:19.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223914+0000", 2020-08-07T15:12:19.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002424149 2020-08-07T15:12:19.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226338+0000", 2020-08-07T15:12:19.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2830000000000001e-06 2020-08-07T15:12:19.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:19.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226346+0000", 2020-08-07T15:12:19.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0412357219999999 2020-08-07T15:12:19.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.267582+0000", 2020-08-07T15:12:19.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000123924 2020-08-07T15:12:19.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:19.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.267706+0000", 2020-08-07T15:12:19.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000216376 2020-08-07T15:12:19.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:19.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.267922+0000", 2020-08-07T15:12:19.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031340790000000001 2020-08-07T15:12:19.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:19.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271056+0000", 2020-08-07T15:12:19.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.007623276 2020-08-07T15:12:19.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278680+0000", 2020-08-07T15:12:19.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000333853 2020-08-07T15:12:19.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279013+0000", 2020-08-07T15:12:19.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.02e-06 2020-08-07T15:12:19.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:19.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279014+0000", 2020-08-07T15:12:19.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6306999999999999e-05 2020-08-07T15:12:19.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:19.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279101+0000", 2020-08-07T15:12:19.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050739800000000005 2020-08-07T15:12:19.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279608+0000", 2020-08-07T15:12:19.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001617098 2020-08-07T15:12:19.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281225+0000", 2020-08-07T15:12:19.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9700000000000002e-07 2020-08-07T15:12:19.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:19.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281226+0000", 2020-08-07T15:12:19.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107508 2020-08-07T15:12:19.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:19.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281333+0000", 2020-08-07T15:12:19.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0575529850000001 2020-08-07T15:12:19.424 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.424 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:19.424 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.425 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118443 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:19.425 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223889+0000", 2020-08-07T15:12:19.425 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.087840733999997, 2020-08-07T15:12:19.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.05758606, 2020-08-07T15:12:19.426 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:19.426 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:19.426 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:19.426 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:19.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:19.427 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118443 2020-08-07T15:12:19.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.427 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:19.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:19.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223889+0000", 2020-08-07T15:12:19.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:19.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:19.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223889+0000", 2020-08-07T15:12:19.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8148e-05 2020-08-07T15:12:19.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:19.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223937+0000", 2020-08-07T15:12:19.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2789999999999998e-06 2020-08-07T15:12:19.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:19.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223942+0000", 2020-08-07T15:12:19.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0472000000000001e-05 2020-08-07T15:12:19.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:19.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223953+0000", 2020-08-07T15:12:19.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6580000000000001e-06 2020-08-07T15:12:19.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:19.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223957+0000", 2020-08-07T15:12:19.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5981000000000001e-05 2020-08-07T15:12:19.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223983+0000", 2020-08-07T15:12:19.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.288e-06 2020-08-07T15:12:19.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:19.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223989+0000", 2020-08-07T15:12:19.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024051889999999999 2020-08-07T15:12:19.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226394+0000", 2020-08-07T15:12:19.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2439000000000001e-05 2020-08-07T15:12:19.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:19.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226406+0000", 2020-08-07T15:12:19.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0421168519999999 2020-08-07T15:12:19.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.268523+0000", 2020-08-07T15:12:19.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0449000000000002e-05 2020-08-07T15:12:19.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:19.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.268604+0000", 2020-08-07T15:12:19.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014104300000000001 2020-08-07T15:12:19.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:19.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.268745+0000", 2020-08-07T15:12:19.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051249829999999996 2020-08-07T15:12:19.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:19.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.273870+0000", 2020-08-07T15:12:19.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052772840000000001 2020-08-07T15:12:19.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279147+0000", 2020-08-07T15:12:19.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6060000000000003e-05 2020-08-07T15:12:19.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279193+0000", 2020-08-07T15:12:19.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9999999999999997e-07 2020-08-07T15:12:19.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:19.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279194+0000", 2020-08-07T15:12:19.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3566000000000003e-05 2020-08-07T15:12:19.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:19.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279237+0000", 2020-08-07T15:12:19.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011546359999999999 2020-08-07T15:12:19.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280392+0000", 2020-08-07T15:12:19.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00097857799999999991 2020-08-07T15:12:19.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281370+0000", 2020-08-07T15:12:19.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999999e-07 2020-08-07T15:12:19.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:19.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281371+0000", 2020-08-07T15:12:19.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104119 2020-08-07T15:12:19.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:19.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281475+0000", 2020-08-07T15:12:19.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.05758606 2020-08-07T15:12:19.484 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.485 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:19.485 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.486 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118444 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:19.486 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223994+0000", 2020-08-07T15:12:19.487 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.087735359, 2020-08-07T15:12:19.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057634572, 2020-08-07T15:12:19.487 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:19.487 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:19.488 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:19.488 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:19.488 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:19.488 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118444 2020-08-07T15:12:19.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.489 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:19.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:19.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223994+0000", 2020-08-07T15:12:19.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:19.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:19.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223994+0000", 2020-08-07T15:12:19.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1279999999999997e-05 2020-08-07T15:12:19.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:19.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224046+0000", 2020-08-07T15:12:19.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6930000000000004e-06 2020-08-07T15:12:19.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:19.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224051+0000", 2020-08-07T15:12:19.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.095e-05 2020-08-07T15:12:19.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:19.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224062+0000", 2020-08-07T15:12:19.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9529999999999998e-06 2020-08-07T15:12:19.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:19.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224067+0000", 2020-08-07T15:12:19.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000162091 2020-08-07T15:12:19.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224229+0000", 2020-08-07T15:12:19.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.276e-06 2020-08-07T15:12:19.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:19.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224238+0000", 2020-08-07T15:12:19.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022140409999999999 2020-08-07T15:12:19.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226452+0000", 2020-08-07T15:12:19.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5951999999999999e-05 2020-08-07T15:12:19.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:19.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226467+0000", 2020-08-07T15:12:19.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0428868099999999 2020-08-07T15:12:19.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269354+0000", 2020-08-07T15:12:19.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5264000000000003e-05 2020-08-07T15:12:19.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:19.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269430+0000", 2020-08-07T15:12:19.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020438499999999999 2020-08-07T15:12:19.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:19.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269634+0000", 2020-08-07T15:12:19.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042509959999999999 2020-08-07T15:12:19.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:19.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.273885+0000", 2020-08-07T15:12:19.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054290140000000002 2020-08-07T15:12:19.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279314+0000", 2020-08-07T15:12:19.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00038996700000000002 2020-08-07T15:12:19.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279704+0000", 2020-08-07T15:12:19.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5700000000000002e-07 2020-08-07T15:12:19.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:19.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279705+0000", 2020-08-07T15:12:19.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9377999999999997e-05 2020-08-07T15:12:19.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:19.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279794+0000", 2020-08-07T15:12:19.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00064521299999999995 2020-08-07T15:12:19.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280439+0000", 2020-08-07T15:12:19.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001072185 2020-08-07T15:12:19.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281511+0000", 2020-08-07T15:12:19.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7300000000000002e-07 2020-08-07T15:12:19.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:19.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281512+0000", 2020-08-07T15:12:19.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117094 2020-08-07T15:12:19.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:19.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281629+0000", 2020-08-07T15:12:19.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057634572 2020-08-07T15:12:19.527 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.527 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:19.527 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.528 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118445 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:19.528 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.224116+0000", 2020-08-07T15:12:19.528 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.087613417999997, 2020-08-07T15:12:19.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057744163, 2020-08-07T15:12:19.528 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:19.529 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:19.529 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:19.529 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:19.530 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:19.530 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118445 2020-08-07T15:12:19.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.531 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:19.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:19.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224116+0000", 2020-08-07T15:12:19.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:19.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:19.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224116+0000", 2020-08-07T15:12:19.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3542999999999999e-05 2020-08-07T15:12:19.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:19.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224170+0000", 2020-08-07T15:12:19.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.446e-06 2020-08-07T15:12:19.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:19.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224175+0000", 2020-08-07T15:12:19.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1473e-05 2020-08-07T15:12:19.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:19.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224187+0000", 2020-08-07T15:12:19.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9449999999999998e-06 2020-08-07T15:12:19.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:19.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224193+0000", 2020-08-07T15:12:19.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4945999999999997e-05 2020-08-07T15:12:19.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224268+0000", 2020-08-07T15:12:19.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5799999999999992e-06 2020-08-07T15:12:19.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:19.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224276+0000", 2020-08-07T15:12:19.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022246980000000002 2020-08-07T15:12:19.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226501+0000", 2020-08-07T15:12:19.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1359999999999997e-06 2020-08-07T15:12:19.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:19.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226509+0000", 2020-08-07T15:12:19.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.043599741 2020-08-07T15:12:19.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.270109+0000", 2020-08-07T15:12:19.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000129503 2020-08-07T15:12:19.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:19.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.270238+0000", 2020-08-07T15:12:19.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021162500000000001 2020-08-07T15:12:19.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:19.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.270450+0000", 2020-08-07T15:12:19.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036313090000000001 2020-08-07T15:12:19.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:19.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274081+0000", 2020-08-07T15:12:19.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00557165 2020-08-07T15:12:19.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279653+0000", 2020-08-07T15:12:19.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000196658 2020-08-07T15:12:19.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279849+0000", 2020-08-07T15:12:19.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6700000000000003e-07 2020-08-07T15:12:19.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:19.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279850+0000", 2020-08-07T15:12:19.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6522999999999998e-05 2020-08-07T15:12:19.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:19.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279937+0000", 2020-08-07T15:12:19.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00055050400000000005 2020-08-07T15:12:19.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280487+0000", 2020-08-07T15:12:19.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011833760000000001 2020-08-07T15:12:19.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281670+0000", 2020-08-07T15:12:19.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2020-08-07T15:12:19.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:19.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281671+0000", 2020-08-07T15:12:19.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001896 2020-08-07T15:12:19.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:19.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281860+0000", 2020-08-07T15:12:19.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057744163 2020-08-07T15:12:19.582 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.582 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:19.583 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.584 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118446 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:19.584 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.224256+0000", 2020-08-07T15:12:19.584 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.087473633000002, 2020-08-07T15:12:19.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057864804, 2020-08-07T15:12:19.584 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:19.585 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:19.585 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:19.585 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:19.585 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:19.585 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118446 2020-08-07T15:12:19.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.586 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:19.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:19.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224256+0000", 2020-08-07T15:12:19.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:19.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:19.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224256+0000", 2020-08-07T15:12:19.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5838000000000003e-05 2020-08-07T15:12:19.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:19.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224312+0000", 2020-08-07T15:12:19.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.879e-06 2020-08-07T15:12:19.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:19.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224318+0000", 2020-08-07T15:12:19.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3593e-05 2020-08-07T15:12:19.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:19.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224331+0000", 2020-08-07T15:12:19.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6169999999999998e-06 2020-08-07T15:12:19.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:19.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224337+0000", 2020-08-07T15:12:19.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7246000000000001e-05 2020-08-07T15:12:19.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224384+0000", 2020-08-07T15:12:19.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7690000000000008e-06 2020-08-07T15:12:19.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:19.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224394+0000", 2020-08-07T15:12:19.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021394589999999998 2020-08-07T15:12:19.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226533+0000", 2020-08-07T15:12:19.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3849999999999996e-06 2020-08-07T15:12:19.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:19.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226541+0000", 2020-08-07T15:12:19.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0445569109999999 2020-08-07T15:12:19.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271098+0000", 2020-08-07T15:12:19.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012137 2020-08-07T15:12:19.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:19.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271219+0000", 2020-08-07T15:12:19.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013978300000000001 2020-08-07T15:12:19.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:19.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271359+0000", 2020-08-07T15:12:19.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027367699999999999 2020-08-07T15:12:19.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:19.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274096+0000", 2020-08-07T15:12:19.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058973660000000002 2020-08-07T15:12:19.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279993+0000", 2020-08-07T15:12:19.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116291 2020-08-07T15:12:19.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280109+0000", 2020-08-07T15:12:19.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1099999999999995e-07 2020-08-07T15:12:19.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:19.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280110+0000", 2020-08-07T15:12:19.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.564e-05 2020-08-07T15:12:19.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:19.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280196+0000", 2020-08-07T15:12:19.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061969 2020-08-07T15:12:19.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280815+0000", 2020-08-07T15:12:19.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001097612 2020-08-07T15:12:19.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281913+0000", 2020-08-07T15:12:19.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4300000000000003e-07 2020-08-07T15:12:19.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:19.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281913+0000", 2020-08-07T15:12:19.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000207431 2020-08-07T15:12:19.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:19.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282121+0000", 2020-08-07T15:12:19.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057864804 2020-08-07T15:12:19.610 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.611 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:19.611 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.611 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118447 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:19.612 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.224379+0000", 2020-08-07T15:12:19.612 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.087350866999998, 2020-08-07T15:12:19.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0579997480000001, 2020-08-07T15:12:19.612 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:19.612 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:19.613 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:19.613 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:19.613 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:19.613 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118447 2020-08-07T15:12:19.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.614 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:19.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:19.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224379+0000", 2020-08-07T15:12:19.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:19.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:19.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224379+0000", 2020-08-07T15:12:19.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9775999999999999e-05 2020-08-07T15:12:19.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:19.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224429+0000", 2020-08-07T15:12:19.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8500000000000002e-06 2020-08-07T15:12:19.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:19.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224433+0000", 2020-08-07T15:12:19.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7340000000000008e-06 2020-08-07T15:12:19.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:19.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224441+0000", 2020-08-07T15:12:19.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.031e-06 2020-08-07T15:12:19.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:19.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224445+0000", 2020-08-07T15:12:19.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4264000000000003e-05 2020-08-07T15:12:19.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224479+0000", 2020-08-07T15:12:19.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0039999999999999e-06 2020-08-07T15:12:19.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:19.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224487+0000", 2020-08-07T15:12:19.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002241682 2020-08-07T15:12:19.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226729+0000", 2020-08-07T15:12:19.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0087e-05 2020-08-07T15:12:19.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:19.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226739+0000", 2020-08-07T15:12:19.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.045170006 2020-08-07T15:12:19.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271909+0000", 2020-08-07T15:12:19.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012782900000000001 2020-08-07T15:12:19.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:19.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.272037+0000", 2020-08-07T15:12:19.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024079400000000001 2020-08-07T15:12:19.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:19.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.272278+0000", 2020-08-07T15:12:19.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023454399999999999 2020-08-07T15:12:19.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:19.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274623+0000", 2020-08-07T15:12:19.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054268190000000003 2020-08-07T15:12:19.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280050+0000", 2020-08-07T15:12:19.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00048391200000000002 2020-08-07T15:12:19.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280534+0000", 2020-08-07T15:12:19.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5400000000000001e-07 2020-08-07T15:12:19.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:19.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280535+0000", 2020-08-07T15:12:19.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8922e-05 2020-08-07T15:12:19.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:19.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280623+0000", 2020-08-07T15:12:19.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024637999999999998 2020-08-07T15:12:19.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280870+0000", 2020-08-07T15:12:19.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001300142 2020-08-07T15:12:19.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282170+0000", 2020-08-07T15:12:19.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2099999999999997e-07 2020-08-07T15:12:19.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:19.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282170+0000", 2020-08-07T15:12:19.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000208001 2020-08-07T15:12:19.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:19.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282378+0000", 2020-08-07T15:12:19.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0579997480000001 2020-08-07T15:12:19.645 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.645 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:19.646 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.646 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118448 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:19.647 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.224474+0000", 2020-08-07T15:12:19.647 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.087255880999997, 2020-08-07T15:12:19.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0581683310000001, 2020-08-07T15:12:19.647 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:19.670 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:19.671 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:19.671 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:19.671 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:19.671 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118448 2020-08-07T15:12:19.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.672 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:19.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:19.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224474+0000", 2020-08-07T15:12:19.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:19.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:19.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224474+0000", 2020-08-07T15:12:19.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3804000000000002e-05 2020-08-07T15:12:19.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:19.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224528+0000", 2020-08-07T15:12:19.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5140000000000001e-06 2020-08-07T15:12:19.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:19.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224533+0000", 2020-08-07T15:12:19.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3315999999999999e-05 2020-08-07T15:12:19.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:19.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224546+0000", 2020-08-07T15:12:19.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7240000000000003e-06 2020-08-07T15:12:19.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:19.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224553+0000", 2020-08-07T15:12:19.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002226263 2020-08-07T15:12:19.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226779+0000", 2020-08-07T15:12:19.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1362000000000001e-05 2020-08-07T15:12:19.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:19.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226791+0000", 2020-08-07T15:12:19.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.046092652 2020-08-07T15:12:19.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.272883+0000", 2020-08-07T15:12:19.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000115981 2020-08-07T15:12:19.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:19.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.272999+0000", 2020-08-07T15:12:19.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029648899999999999 2020-08-07T15:12:19.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:19.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.273296+0000", 2020-08-07T15:12:19.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001852521 2020-08-07T15:12:19.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:19.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275148+0000", 2020-08-07T15:12:19.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051948309999999996 2020-08-07T15:12:19.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280343+0000", 2020-08-07T15:12:19.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033301799999999998 2020-08-07T15:12:19.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280676+0000", 2020-08-07T15:12:19.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8400000000000004e-07 2020-08-07T15:12:19.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:19.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280677+0000", 2020-08-07T15:12:19.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9652000000000002e-05 2020-08-07T15:12:19.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:19.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280756+0000", 2020-08-07T15:12:19.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017072699999999999 2020-08-07T15:12:19.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280927+0000", 2020-08-07T15:12:19.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015026530000000001 2020-08-07T15:12:19.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:19.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282430+0000", 2020-08-07T15:12:19.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4700000000000001e-07 2020-08-07T15:12:19.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:19.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282430+0000", 2020-08-07T15:12:19.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021169299999999999 2020-08-07T15:12:19.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:19.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282642+0000", 2020-08-07T15:12:19.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0581683310000001 2020-08-07T15:12:19.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.773 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:19.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.773 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:19.774 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:19.775 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:12:19.775 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2020-08-07T15:12:19.776 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:12:19.776 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:12:19.785 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:12:19.785 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 7 16 10560 10544 11.7663 11.8281 0.0215354 0.0105941 2020-08-07T15:12:19.786 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:19.773+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:19.787 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:12:19.914 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:19.900+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:19.947 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:19.947 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:12:19.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.948 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:11657 6.8 6:146122a6:::benchmark_data_smithi074_63843_object1457:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e237)", 2020-08-07T15:12:19.948 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:19.930742+0000", 2020-08-07T15:12:19.948 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0049473490000000002, 2020-08-07T15:12:19.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049669019999999996, 2020-08-07T15:12:19.948 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:19.949 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:19.949 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:19.949 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:12:19.949 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:19.949 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11657 2020-08-07T15:12:19.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.950 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:19.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:19.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.930742+0000", 2020-08-07T15:12:19.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:19.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:19.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.930742+0000", 2020-08-07T15:12:19.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9011999999999999e-05 2020-08-07T15:12:19.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:19.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.930771+0000", 2020-08-07T15:12:19.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3250000000000001e-06 2020-08-07T15:12:19.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:19.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.930775+0000", 2020-08-07T15:12:19.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8019999999999998e-06 2020-08-07T15:12:19.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:19.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.930784+0000", 2020-08-07T15:12:19.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4610000000000001e-06 2020-08-07T15:12:19.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:19.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.930788+0000", 2020-08-07T15:12:19.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1070999999999999e-05 2020-08-07T15:12:19.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:19.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.930839+0000", 2020-08-07T15:12:19.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018027000000000001 2020-08-07T15:12:19.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:19.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.931020+0000", 2020-08-07T15:12:19.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021997499999999999 2020-08-07T15:12:19.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:12:19.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.931239+0000", 2020-08-07T15:12:19.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001571271 2020-08-07T15:12:19.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:19.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.932811+0000", 2020-08-07T15:12:19.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3399999999999997e-07 2020-08-07T15:12:19.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:19.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.932812+0000", 2020-08-07T15:12:19.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4023999999999997e-05 2020-08-07T15:12:19.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:19.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.932856+0000", 2020-08-07T15:12:19.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002114145 2020-08-07T15:12:19.993 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.993 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:19.993 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:19.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.994 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:11658 6.8 6:146122a6:::benchmark_data_smithi074_63843_object1457:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e237)", 2020-08-07T15:12:19.994 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:19.930988+0000", 2020-08-07T15:12:19.995 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0047012770000000002, 2020-08-07T15:12:19.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0047935139999999996, 2020-08-07T15:12:19.995 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:19.995 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:19.996 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:19.996 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:12:19.996 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:19.996 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11658 2020-08-07T15:12:19.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.997 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:19.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:19.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.930988+0000", 2020-08-07T15:12:19.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:19.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:19.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.930988+0000", 2020-08-07T15:12:19.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5995000000000001e-05 2020-08-07T15:12:19.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:19.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:19.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:19.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.931014+0000", 2020-08-07T15:12:20.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1640000000000001e-06 2020-08-07T15:12:20.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:20.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.931017+0000", 2020-08-07T15:12:20.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1959999999999996e-06 2020-08-07T15:12:20.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:20.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.931023+0000", 2020-08-07T15:12:20.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9840000000000001e-06 2020-08-07T15:12:20.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:20.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.931026+0000", 2020-08-07T15:12:20.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067099900000000003 2020-08-07T15:12:20.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:20.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.931697+0000", 2020-08-07T15:12:20.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7944999999999998e-05 2020-08-07T15:12:20.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:20.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.931795+0000", 2020-08-07T15:12:20.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019168000000000001 2020-08-07T15:12:20.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:12:20.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.931987+0000", 2020-08-07T15:12:20.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003483444 2020-08-07T15:12:20.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:20.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.935470+0000", 2020-08-07T15:12:20.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1699999999999998e-07 2020-08-07T15:12:20.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:20.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.935471+0000", 2020-08-07T15:12:20.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0779999999999999e-05 2020-08-07T15:12:20.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:20.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.935511+0000", 2020-08-07T15:12:20.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045238040000000002 2020-08-07T15:12:20.014 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:20.015 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:20.015 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:20.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.016 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:11659 6.8 6:146122a6:::benchmark_data_smithi074_63843_object1457:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e237)", 2020-08-07T15:12:20.016 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:19.931137+0000", 2020-08-07T15:12:20.016 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0045521989999999998, 2020-08-07T15:12:20.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0047019870000000004, 2020-08-07T15:12:20.017 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:20.017 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:12:20.017 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:20.017 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:12:20.018 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:20.018 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11659 2020-08-07T15:12:20.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.018 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:20.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:20.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.931137+0000", 2020-08-07T15:12:20.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:20.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:20.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.931137+0000", 2020-08-07T15:12:20.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5554e-05 2020-08-07T15:12:20.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:20.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.931162+0000", 2020-08-07T15:12:20.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9239999999999999e-06 2020-08-07T15:12:20.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:20.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.931165+0000", 2020-08-07T15:12:20.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5440000000000004e-06 2020-08-07T15:12:20.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:20.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.931172+0000", 2020-08-07T15:12:20.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5859999999999999e-06 2020-08-07T15:12:20.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:20.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.931174+0000", 2020-08-07T15:12:20.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011863990000000001 2020-08-07T15:12:20.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:20.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.932361+0000", 2020-08-07T15:12:20.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0268e-05 2020-08-07T15:12:20.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:20.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.932421+0000", 2020-08-07T15:12:20.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011875300000000001 2020-08-07T15:12:20.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:12:20.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.932540+0000", 2020-08-07T15:12:20.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001403028 2020-08-07T15:12:20.027 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:20.027 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:20.027 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:20.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.028 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:11660 6.8 6:146122a6:::benchmark_data_smithi074_63843_object1457:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e237)", 2020-08-07T15:12:20.028 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:19.933109+0000", 2020-08-07T15:12:20.042 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0025797369999999999, 2020-08-07T15:12:20.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002775685, 2020-08-07T15:12:20.043 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:20.043 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:12:20.043 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:20.044 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:12:20.044 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:20.044 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11660 2020-08-07T15:12:20.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.045 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:20.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:20.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.933109+0000", 2020-08-07T15:12:20.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:20.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:20.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.933109+0000", 2020-08-07T15:12:20.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8351999999999999e-05 2020-08-07T15:12:20.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:20.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.933138+0000", 2020-08-07T15:12:20.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9140000000000001e-06 2020-08-07T15:12:20.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:20.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.933141+0000", 2020-08-07T15:12:20.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9129999999999992e-06 2020-08-07T15:12:20.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:20.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.933150+0000", 2020-08-07T15:12:20.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3159999999999996e-06 2020-08-07T15:12:20.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:20.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.933156+0000", 2020-08-07T15:12:20.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5451999999999999e-05 2020-08-07T15:12:20.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:20.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.933191+0000", 2020-08-07T15:12:20.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7874999999999997e-05 2020-08-07T15:12:20.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:20.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.933249+0000", 2020-08-07T15:12:20.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106152 2020-08-07T15:12:20.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:12:20.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.933355+0000", 2020-08-07T15:12:20.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024597399999999999 2020-08-07T15:12:20.103 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:20.103 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:20.104 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:20.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.105 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:11661 6.8 6:146122a6:::benchmark_data_smithi074_63843_object1457:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e237)", 2020-08-07T15:12:20.105 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:19.934907+0000", 2020-08-07T15:12:20.107 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00078210900000000002, 2020-08-07T15:12:20.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001025094, 2020-08-07T15:12:20.107 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:20.108 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:12:20.108 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:20.108 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:12:20.109 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:20.109 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11661 2020-08-07T15:12:20.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.110 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:20.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:20.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.934907+0000", 2020-08-07T15:12:20.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:20.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:20.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.934907+0000", 2020-08-07T15:12:20.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2238999999999998e-05 2020-08-07T15:12:20.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:20.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.934969+0000", 2020-08-07T15:12:20.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0369999999999995e-06 2020-08-07T15:12:20.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:20.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.934977+0000", 2020-08-07T15:12:20.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5892999999999998e-05 2020-08-07T15:12:20.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:20.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.934993+0000", 2020-08-07T15:12:20.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7559999999999999e-06 2020-08-07T15:12:20.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:20.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.935001+0000", 2020-08-07T15:12:20.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6543999999999999e-05 2020-08-07T15:12:20.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:20.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.935057+0000", 2020-08-07T15:12:20.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3064000000000005e-05 2020-08-07T15:12:20.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:20.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.935120+0000", 2020-08-07T15:12:20.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101562 2020-08-07T15:12:20.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:12:20.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.935222+0000", 2020-08-07T15:12:20.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031509499999999999 2020-08-07T15:12:20.128 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:20.128 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:20.128 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:20.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.129 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:11662 6.8 6:146122a6:::benchmark_data_smithi074_63843_object1457:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e237)", 2020-08-07T15:12:20.129 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:19.935814+0000", 2020-08-07T15:12:20.130 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4294967295.9998751, 2020-08-07T15:12:20.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016585499999999999, 2020-08-07T15:12:20.130 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:20.131 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:20.134 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:20.134 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:12:20.134 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:20.135 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11662 2020-08-07T15:12:20.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.135 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:20.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:20.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.935814+0000", 2020-08-07T15:12:20.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:20.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:20.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.935814+0000", 2020-08-07T15:12:20.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6724999999999999e-05 2020-08-07T15:12:20.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:20.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.935841+0000", 2020-08-07T15:12:20.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0199999999999996e-06 2020-08-07T15:12:20.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:20.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.935845+0000", 2020-08-07T15:12:20.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9160000000000007e-06 2020-08-07T15:12:20.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:20.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.935854+0000", 2020-08-07T15:12:20.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.485e-06 2020-08-07T15:12:20.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:20.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.935859+0000", 2020-08-07T15:12:20.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3368000000000003e-05 2020-08-07T15:12:20.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:20.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.935892+0000", 2020-08-07T15:12:20.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8633999999999994e-05 2020-08-07T15:12:20.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:20.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:20.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:20.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:19.935961+0000", 2020-08-07T15:12:20.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014714799999999999 2020-08-07T15:12:20.173 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:20.173 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:20.173 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:20.175 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:20.175 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:12:20.175 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 6 2020-08-07T15:12:20.175 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:20.176 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:12:20.179 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:12:20.180 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:12:20.180 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:12:20.180 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:20.178+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:20.181 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:12:20.222 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:12:20.270 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 8 16 12144 12128 11.8422 12.375 0.0103494 0.0105218 2020-08-07T15:12:20.271 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:12:20.286 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:20.281+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:20.306 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:12:20.311 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:12:20.313 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:12:20.331 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2020-08-07T15:12:20.244253+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":64110592,"num_objects":982,"num_object_clones":0,"num_object_copies":2946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7826,"num_write_kb":62608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":149,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":181,"ondisk_log_size":181,"up":165,"acting":165,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8575100,"kb_used_data":186420,"kb_used_omap":2,"kb_used_meta":8388605,"kb_avail":746399620,"statfs":{"total":773094113280,"available":764313210880,"internally_reserved":8589934592,"allocated":190894080,"data_stored":182152767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":8589932013},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,11,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":57753600,"num_objects":882,"num_object_clones":0,"num_object_copies":2646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7050,"num_write_kb":56400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.829340"},"pg_stats":[{"pgid":"1.1e","version":"16'2","reported_seq":"85","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:12.262609+0000","last_change":"2020-08-07T15:11:37.457186+0000","last_active":"2020-08-07T15:12:12.262609+0000","last_peered":"2020-08-07T15:12:12.262609+0000","last_clean":"2020-08-07T15:12:12.262609+0000","last_became_active":"2020-08-07T15:10:51.559894+0000","last_became_peered":"2020-08-07T15:10:51.559894+0000","last_unstale":"2020-08-07T15:12:12.262609+0000","last_undegraded":"2020-08-07T15:12:12.262609+0000","last_fullsized":"2020-08-07T15:12:12.262609+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.457111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.457111+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1d","version":"75'3","reported_seq":"98","reported_epoch":"236","state":"active+clean","last_fresh":"2020-08-07T15:12:18.264340+0000","last_change":"2020-08-07T15:12:07.357104+0000","last_active":"2020-08-07T15:12:18.264340+0000","last_peered":"2020-08-07T15:12:18.264340+0000","last_clean":"2020-08-07T15:12:18.264340+0000","last_became_active":"2020-08-07T15:12:07.356691+0000","last_became_peered":"2020-08-07T15:12:07.356691+0000","last_unstale":"2020-08-07T15:12:18.264340+0000","last_undegraded":"2020-08-07T15:12:18.264340+0000","last_fullsized":"2020-08-07T15:12:18.264340+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:10:10.962480+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.962480+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":149,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1c","version":"16'2","reported_seq":"173","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:12.215745+0000","last_change":"2020-08-07T15:12:07.262797+0000","last_active":"2020-08-07T15:12:12.215745+0000","last_peered":"2020-08-07T15:12:12.215745+0000","last_clean":"2020-08-07T15:12:12.215745+0000","last_became_active":"2020-08-07T15:12:07.245731+0000","last_became_peered":"2020-08-07T15:12:07.245731+0000","last_unstale":"2020-08-07T15:12:12.215745+0000","last_undegraded":"2020-08-07T15:12:12.215745+0000","last_fullsized":"2020-08-07T15:12:12.215745+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:39.738160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:39.738160+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1b","version":"16'2","reported_seq":"206","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:12.213961+0000","last_change":"2020-08-07T15:10:57.797029+0000","last_active":"2020-08-07T15:12:12.213961+0000","last_peered":"2020-08-07T15:12:12.213961+0000","last_clean":"2020-08-07T15:12:12.213961+0000","last_became_active":"2020-08-07T15:10:51.557392+0000","last_became_peered":"2020-08-07T15:10:51.557392+0000","last_unstale":"2020-08-07T15:12:12.213961+0000","last_undegraded":"2020-08-07T15:12:12.213961+0000","last_fullsized":"2020-08-07T15:12:12.213961+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:57.796946+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:57.796946+0000","last_clean_scrub_stamp":"2020-08-07T15:10:57.796946+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1a","version":"16'2","reported_seq":"70","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:12.215526+0000","last_change":"2020-08-07T15:12:07.262337+0000","last_active":"2020-08-07T15:12:12.215526+0000","last_peered":"2020-08-07T15:12:12.215526+0000","last_clean":"2020-08-07T15:12:12.215526+0000","last_became_active":"2020-08-07T15:12:07.243542+0000","last_became_peered":"2020-08-07T15:12:07.243542+0000","last_unstale":"2020-08-07T15:12:12.215526+0000","last_undegraded":"2020-08-07T15:12:12.215526+0000","last_fullsized":"2020-08-07T15:12:12.215526+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:11.779828+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:11:11.779828+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.19","version":"16'2","reported_seq":"131","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:15.372999+0000","last_change":"2020-08-07T15:12:15.372999+0000","last_active":"2020-08-07T15:12:15.372999+0000","last_peered":"2020-08-07T15:12:15.372999+0000","last_clean":"2020-08-07T15:12:15.372999+0000","last_became_active":"2020-08-07T15:10:51.559752+0000","last_became_peered":"2020-08-07T15:10:51.559752+0000","last_unstale":"2020-08-07T15:12:15.372999+0000","last_undegraded":"2020-08-07T15:12:15.372999+0000","last_fullsized":"2020-08-07T15:12:15.372999+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:15.372910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:15.372910+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.18","version":"16'2","reported_seq":"231","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:12.213804+0000","last_change":"2020-08-07T15:11:06.955336+0000","last_active":"2020-08-07T15:12:12.213804+0000","last_peered":"2020-08-07T15:12:12.213804+0000","last_clean":"2020-08-07T15:12:12.213804+0000","last_became_active":"2020-08-07T15:10:51.557665+0000","last_became_peered":"2020-08-07T15:10:51.557665+0000","last_unstale":"2020-08-07T15:12:12.213804+0000","last_undegraded":"2020-08-07T15:12:12.213804+0000","last_fullsized":"2020-08-07T15:12:12.213804+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:06.955262+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:06.955262+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.17","version":"16'2","reported_seq":"59","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:12.214311+0000","last_change":"2020-08-07T15:11:08.916219+0000","last_active":"2020-08-07T15:12:12.214311+0000","last_peered":"2020-08-07T15:12:12.214311+0000","last_clean":"2020-08-07T15:12:12.214311+0000","last_became_active":"2020-08-07T15:10:51.561769+0000","last_became_peered":"2020-08-07T15:10:51.561769+0000","last_unstale":"2020-08-07T15:12:12.214311+0000","last_undegraded":"2020-08-07T15:12:12.214311+0000","last_fullsized":"2020-08-07T15:12:12.214311+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:08.916169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:08.916169+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.16","version":"16'2","reported_seq":"39","reported_epoch":"236","state":"active+clean","last_fresh":"2020-08-07T15:12:18.265326+0000","last_change":"2020-08-07T15:12:12.883619+0000","last_active":"2020-08-07T15:12:18.265326+0000","last_peered":"2020-08-07T15:12:18.265326+0000","last_clean":"2020-08-07T15:12:18.265326+0000","last_became_active":"2020-08-07T15:12:07.231877+0000","last_became_peered":"2020-08-07T15:12:07.231877+0000","last_unstale":"2020-08-07T15:12:18.265326+0000","last_undegraded":"2020-08-07T15:12:18.265326+0000","last_fullsized":"2020-08-07T15:12:18.265326+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:12.883558+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:12.883558+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.883558+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.15","version":"75'3","reported_seq":"70","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:12.215800+0000","last_change":"2020-08-07T15:12:07.284341+0000","last_active":"2020-08-07T15:12:12.215800+0000","last_peered":"2020-08-07T15:12:12.215800+0000","last_clean":"2020-08-07T15:12:12.215800+0000","last_became_active":"2020-08-07T15:12:07.276899+0000","last_became_peered":"2020-08-07T15:12:07.276899+0000","last_unstale":"2020-08-07T15:12:12.215800+0000","last_undegraded":"2020-08-07T15:12:12.215800+0000","last_fullsized":"2020-08-07T15:12:12.215800+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:11:20.562710+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:11:20.562710+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.14","version":"16'2","reported_seq":"169","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:12.261470+0000","last_change":"2020-08-07T15:12:09.224231+0000","last_active":"2020-08-07T15:12:12.261470+0000","last_peered":"2020-08-07T15:12:12.261470+0000","last_clean":"2020-08-07T15:12:12.261470+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:12:12.261470+0000","last_undegraded":"2020-08-07T15:12:12.261470+0000","last_fullsized":"2020-08-07T15:12:12.261470+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:09.224158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.224158+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"187","reported_epoch":"237","state":"active+clean","last_fresh":"2020-08-07T15:12:19.322861+0000","last_change":"2020-08-07T15:12:19.577926+0000","last_active":"2020-08-07T15:12:19.322861+0000","last_peered":"2020-08-07T15:12:19.322861+0000","last_clean":"2020-08-07T15:12:19.322861+0000","last_became_active":"2020-08-07T15:12:06.375861+0000","last_became_peered":"2020-08-07T15:12:06.375861+0000","last_unstale":"2020-08-07T15:12:19.322861+0000","last_undegraded":"2020-08-07T15:12:19.322861+0000","last_fullsized":"2020-08-07T15:12:19.322861+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:07.887356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.887356+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"181","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:12.213981+0000","last_change":"2020-08-07T15:12:06.368227+0000","last_active":"2020-08-07T15:12:12.213981+0000","last_peered":"2020-08-07T15:12:12.213981+0000","last_clean":"2020-08-07T15:12:12.213981+0000","last_became_active":"2020-08-07T15:12:06.367362+0000","last_became_peered":"2020-08-07T15:12:06.367362+0000","last_unstale":"2020-08-07T15:12:12.213981+0000","last_undegraded":"2020-08-07T15:12:12.213981+0000","last_fullsized":"2020-08-07T15:12:12.213981+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:28.734193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:28.734193+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"279","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:12.213868+0000","last_change":"2020-08-07T15:10:24.690028+0000","last_active":"2020-08-07T15:12:12.213868+0000","last_peered":"2020-08-07T15:12:12.213868+0000","last_clean":"2020-08-07T15:12:12.213868+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:12:12.213868+0000","last_undegraded":"2020-08-07T15:12:12.213868+0000","last_fullsized":"2020-08-07T15:12:12.213868+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:24.689950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:24.689950+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"276","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:13.231819+0000","last_change":"2020-08-07T15:12:13.231819+0000","last_active":"2020-08-07T15:12:13.231819+0000","last_peered":"2020-08-07T15:12:13.231819+0000","last_clean":"2020-08-07T15:12:13.231819+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:12:13.231819+0000","last_undegraded":"2020-08-07T15:12:13.231819+0000","last_fullsized":"2020-08-07T15:12:13.231819+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:13.231762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.231762+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3","version":"234'392","reported_seq":"404","reported_epoch":"234","state":"active+clean","last_fresh":"2020-08-07T15:12:16.708387+0000","last_change":"2020-08-07T15:12:10.464875+0000","last_active":"2020-08-07T15:12:16.708387+0000","last_peered":"2020-08-07T15:12:16.708387+0000","last_clean":"2020-08-07T15:12:16.708387+0000","last_became_active":"2020-08-07T15:12:10.464346+0000","last_became_peered":"2020-08-07T15:12:10.464346+0000","last_unstale":"2020-08-07T15:12:16.708387+0000","last_undegraded":"2020-08-07T15:12:16.708387+0000","last_fullsized":"2020-08-07T15:12:16.708387+0000","mapping_epoch":230,"log_start":"234'384","ondisk_log_start":"234'384","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3211264,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":392,"num_write_kb":3136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"234","reported_epoch":"234","state":"active+clean","last_fresh":"2020-08-07T15:12:15.419058+0000","last_change":"2020-08-07T15:10:57.384596+0000","last_active":"2020-08-07T15:12:15.419058+0000","last_peered":"2020-08-07T15:12:15.419058+0000","last_clean":"2020-08-07T15:12:15.419058+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:12:15.419058+0000","last_undegraded":"2020-08-07T15:12:15.419058+0000","last_fullsized":"2020-08-07T15:12:15.419058+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:57.384515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:10:57.384515+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"152","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:12.262371+0000","last_change":"2020-08-07T15:12:10.265465+0000","last_active":"2020-08-07T15:12:12.262371+0000","last_peered":"2020-08-07T15:12:12.262371+0000","last_clean":"2020-08-07T15:12:12.262371+0000","last_became_active":"2020-08-07T15:09:30.283176+0000","last_became_peered":"2020-08-07T15:09:30.283176+0000","last_unstale":"2020-08-07T15:12:12.262371+0000","last_undegraded":"2020-08-07T15:12:12.262371+0000","last_fullsized":"2020-08-07T15:12:12.262371+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:10.265395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:10.265395+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"234'456","reported_seq":"468","reported_epoch":"234","state":"active+clean","last_fresh":"2020-08-07T15:12:16.716015+0000","last_change":"2020-08-07T15:12:10.465048+0000","last_active":"2020-08-07T15:12:16.716015+0000","last_peered":"2020-08-07T15:12:16.716015+0000","last_clean":"2020-08-07T15:12:16.716015+0000","last_became_active":"2020-08-07T15:12:10.464493+0000","last_became_peered":"2020-08-07T15:12:10.464493+0000","last_unstale":"2020-08-07T15:12:16.716015+0000","last_undegraded":"2020-08-07T15:12:16.716015+0000","last_fullsized":"2020-08-07T15:12:16.716015+0000","mapping_epoch":230,"log_start":"234'449","ondisk_log_start":"234'449","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":456,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"195","reported_epoch":"237","state":"active+clean","last_fresh":"2020-08-07T15:12:19.323071+0000","last_change":"2020-08-07T15:12:19.561477+0000","last_active":"2020-08-07T15:12:19.323071+0000","last_peered":"2020-08-07T15:12:19.323071+0000","last_clean":"2020-08-07T15:12:19.323071+0000","last_became_active":"2020-08-07T15:12:06.367376+0000","last_became_peered":"2020-08-07T15:12:06.367376+0000","last_unstale":"2020-08-07T15:12:19.323071+0000","last_undegraded":"2020-08-07T15:12:19.323071+0000","last_fullsized":"2020-08-07T15:12:19.323071+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:12.319383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.319383+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"206","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:12.546009+0000","last_change":"2020-08-07T15:12:12.546009+0000","last_active":"2020-08-07T15:12:12.546009+0000","last_peered":"2020-08-07T15:12:12.546009+0000","last_clean":"2020-08-07T15:12:12.546009+0000","last_became_active":"2020-08-07T15:10:46.501793+0000","last_became_peered":"2020-08-07T15:10:46.501793+0000","last_unstale":"2020-08-07T15:12:12.546009+0000","last_undegraded":"2020-08-07T15:12:12.546009+0000","last_fullsized":"2020-08-07T15:12:12.546009+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:12.545945+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.545945+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"237'744","reported_seq":"758","reported_epoch":"237","state":"active+clean","last_fresh":"2020-08-07T15:12:19.836688+0000","last_change":"2020-08-07T15:12:10.464412+0000","last_active":"2020-08-07T15:12:19.836688+0000","last_peered":"2020-08-07T15:12:19.836688+0000","last_clean":"2020-08-07T15:12:19.836688+0000","last_became_active":"2020-08-07T15:12:10.463950+0000","last_became_peered":"2020-08-07T15:12:10.463950+0000","last_unstale":"2020-08-07T15:12:19.836688+0000","last_undegraded":"2020-08-07T15:12:19.836688+0000","last_fullsized":"2020-08-07T15:12:19.836688+0000","mapping_epoch":230,"log_start":"237'738","ondisk_log_start":"237'738","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6094848,"num_objects":93,"num_object_clones":0,"num_object_copies":279,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":744,"num_write_kb":5952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"234","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:12.215955+0000","last_change":"2020-08-07T15:11:55.731391+0000","last_active":"2020-08-07T15:12:12.215955+0000","last_peered":"2020-08-07T15:12:12.215955+0000","last_clean":"2020-08-07T15:12:12.215955+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:12:12.215955+0000","last_undegraded":"2020-08-07T15:12:12.215955+0000","last_fullsized":"2020-08-07T15:12:12.215955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:11:55.731304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:11:55.731304+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"285","reported_epoch":"234","state":"active+clean","last_fresh":"2020-08-07T15:12:15.419062+0000","last_change":"2020-08-07T15:12:14.121460+0000","last_active":"2020-08-07T15:12:15.419062+0000","last_peered":"2020-08-07T15:12:15.419062+0000","last_clean":"2020-08-07T15:12:15.419062+0000","last_became_active":"2020-08-07T15:10:46.516484+0000","last_became_peered":"2020-08-07T15:10:46.516484+0000","last_unstale":"2020-08-07T15:12:15.419062+0000","last_undegraded":"2020-08-07T15:12:15.419062+0000","last_fullsized":"2020-08-07T15:12:15.419062+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_clean_scrub_stamp":"2020-08-07T15:12:14.121397+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"237'816","reported_seq":"828","reported_epoch":"237","state":"active+clean","last_fresh":"2020-08-07T15:12:19.760885+0000","last_change":"2020-08-07T15:12:10.398415+0000","last_active":"2020-08-07T15:12:19.760885+0000","last_peered":"2020-08-07T15:12:19.760885+0000","last_clean":"2020-08-07T15:12:19.760885+0000","last_became_active":"2020-08-07T15:12:10.397834+0000","last_became_peered":"2020-08-07T15:12:10.397834+0000","last_unstale":"2020-08-07T15:12:19.760885+0000","last_undegraded":"2020-08-07T15:12:19.760885+0000","last_fullsized":"2020-08-07T15:12:19.760885+0000","mapping_epoch":230,"log_start":"237'808","ondisk_log_start":"237'808","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6684672,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":816,"num_write_kb":6528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"249","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:12.215618+0000","last_change":"2020-08-07T15:12:08.622779+0000","last_active":"2020-08-07T15:12:12.215618+0000","last_peered":"2020-08-07T15:12:12.215618+0000","last_clean":"2020-08-07T15:12:12.215618+0000","last_became_active":"2020-08-07T15:12:06.367457+0000","last_became_peered":"2020-08-07T15:12:06.367457+0000","last_unstale":"2020-08-07T15:12:12.215618+0000","last_undegraded":"2020-08-07T15:12:12.215618+0000","last_fullsized":"2020-08-07T15:12:12.215618+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:08.622683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:08.622683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"254","reported_epoch":"234","state":"active+clean","last_fresh":"2020-08-07T15:12:15.418901+0000","last_change":"2020-08-07T15:12:10.175824+0000","last_active":"2020-08-07T15:12:15.418901+0000","last_peered":"2020-08-07T15:12:15.418901+0000","last_clean":"2020-08-07T15:12:15.418901+0000","last_became_active":"2020-08-07T15:12:06.366891+0000","last_became_peered":"2020-08-07T15:12:06.366891+0000","last_unstale":"2020-08-07T15:12:15.418901+0000","last_undegraded":"2020-08-07T15:12:15.418901+0000","last_fullsized":"2020-08-07T15:12:15.418901+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:10.175739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:10.175739+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.7","version":"234'516","reported_seq":"521","reported_epoch":"234","state":"active+clean","last_fresh":"2020-08-07T15:12:16.740634+0000","last_change":"2020-08-07T15:12:10.464810+0000","last_active":"2020-08-07T15:12:16.740634+0000","last_peered":"2020-08-07T15:12:16.740634+0000","last_clean":"2020-08-07T15:12:16.740634+0000","last_became_active":"2020-08-07T15:12:10.464050+0000","last_became_peered":"2020-08-07T15:12:10.464050+0000","last_unstale":"2020-08-07T15:12:16.740634+0000","last_undegraded":"2020-08-07T15:12:16.740634+0000","last_fullsized":"2020-08-07T15:12:16.740634+0000","mapping_epoch":230,"log_start":"234'507","ondisk_log_start":"234'507","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4227072,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":516,"num_write_kb":4128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"266","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:12.215513+0000","last_change":"2020-08-07T15:11:40.696919+0000","last_active":"2020-08-07T15:12:12.215513+0000","last_peered":"2020-08-07T15:12:12.215513+0000","last_clean":"2020-08-07T15:12:12.215513+0000","last_became_active":"2020-08-07T15:08:36.627426+0000","last_became_peered":"2020-08-07T15:08:36.627426+0000","last_unstale":"2020-08-07T15:12:12.215513+0000","last_undegraded":"2020-08-07T15:12:12.215513+0000","last_fullsized":"2020-08-07T15:12:12.215513+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:40.696862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:40.696862+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"234","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:12.214248+0000","last_change":"2020-08-07T15:11:40.328534+0000","last_active":"2020-08-07T15:12:12.214248+0000","last_peered":"2020-08-07T15:12:12.214248+0000","last_clean":"2020-08-07T15:12:12.214248+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:12:12.214248+0000","last_undegraded":"2020-08-07T15:12:12.214248+0000","last_fullsized":"2020-08-07T15:12:12.214248+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:11:40.328482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:11:40.328482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"234'432","reported_seq":"444","reported_epoch":"234","state":"active+clean","last_fresh":"2020-08-07T15:12:16.689160+0000","last_change":"2020-08-07T15:12:10.464478+0000","last_active":"2020-08-07T15:12:16.689160+0000","last_peered":"2020-08-07T15:12:16.689160+0000","last_clean":"2020-08-07T15:12:16.689160+0000","last_became_active":"2020-08-07T15:12:10.463997+0000","last_became_peered":"2020-08-07T15:12:10.463997+0000","last_unstale":"2020-08-07T15:12:16.689160+0000","last_undegraded":"2020-08-07T15:12:16.689160+0000","last_fullsized":"2020-08-07T15:12:16.689160+0000","mapping_epoch":230,"log_start":"234'424","ondisk_log_start":"234'424","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3538944,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":432,"num_write_kb":3456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"255","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:12.261487+0000","last_change":"2020-08-07T15:12:10.585667+0000","last_active":"2020-08-07T15:12:12.261487+0000","last_peered":"2020-08-07T15:12:12.261487+0000","last_clean":"2020-08-07T15:12:12.261487+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:12:12.261487+0000","last_undegraded":"2020-08-07T15:12:12.261487+0000","last_fullsized":"2020-08-07T15:12:12.261487+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:10.585580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:10.585580+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"282","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:12.215527+0000","last_change":"2020-08-07T15:12:11.658986+0000","last_active":"2020-08-07T15:12:12.215527+0000","last_peered":"2020-08-07T15:12:12.215527+0000","last_clean":"2020-08-07T15:12:12.215527+0000","last_became_active":"2020-08-07T15:08:36.628365+0000","last_became_peered":"2020-08-07T15:08:36.628365+0000","last_unstale":"2020-08-07T15:12:12.215527+0000","last_undegraded":"2020-08-07T15:12:12.215527+0000","last_fullsized":"2020-08-07T15:12:12.215527+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:11.658923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:11.658923+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"234'480","reported_seq":"489","reported_epoch":"234","state":"active+clean","last_fresh":"2020-08-07T15:12:16.729230+0000","last_change":"2020-08-07T15:12:10.465113+0000","last_active":"2020-08-07T15:12:16.729230+0000","last_peered":"2020-08-07T15:12:16.729230+0000","last_clean":"2020-08-07T15:12:16.729230+0000","last_became_active":"2020-08-07T15:12:10.464582+0000","last_became_peered":"2020-08-07T15:12:10.464582+0000","last_unstale":"2020-08-07T15:12:16.729230+0000","last_undegraded":"2020-08-07T15:12:16.729230+0000","last_fullsized":"2020-08-07T15:12:16.729230+0000","mapping_epoch":230,"log_start":"234'473","ondisk_log_start":"234'473","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":480,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"249","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:12.214313+0000","last_change":"2020-08-07T15:12:07.340958+0000","last_active":"2020-08-07T15:12:12.214313+0000","last_peered":"2020-08-07T15:12:12.214313+0000","last_clean":"2020-08-07T15:12:12.214313+0000","last_became_active":"2020-08-07T15:12:06.366967+0000","last_became_peered":"2020-08-07T15:12:06.366967+0000","last_unstale":"2020-08-07T15:12:12.214313+0000","last_undegraded":"2020-08-07T15:12:12.214313+0000","last_fullsized":"2020-08-07T15:12:12.214313+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:07.340855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.340855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"232","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:12.214018+0000","last_change":"2020-08-07T15:12:09.261178+0000","last_active":"2020-08-07T15:12:12.214018+0000","last_peered":"2020-08-07T15:12:12.214018+0000","last_clean":"2020-08-07T15:12:12.214018+0000","last_became_active":"2020-08-07T15:08:36.618164+0000","last_became_peered":"2020-08-07T15:08:36.618164+0000","last_unstale":"2020-08-07T15:12:12.214018+0000","last_undegraded":"2020-08-07T15:12:12.214018+0000","last_fullsized":"2020-08-07T15:12:12.214018+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:09.261101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.261101+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"234'400","reported_seq":"412","reported_epoch":"234","state":"active+clean","last_fresh":"2020-08-07T15:12:16.740316+0000","last_change":"2020-08-07T15:12:10.466261+0000","last_active":"2020-08-07T15:12:16.740316+0000","last_peered":"2020-08-07T15:12:16.740316+0000","last_clean":"2020-08-07T15:12:16.740316+0000","last_became_active":"2020-08-07T15:12:10.465583+0000","last_became_peered":"2020-08-07T15:12:10.465583+0000","last_unstale":"2020-08-07T15:12:16.740316+0000","last_undegraded":"2020-08-07T15:12:16.740316+0000","last_fullsized":"2020-08-07T15:12:16.740316+0000","mapping_epoch":230,"log_start":"234'392","ondisk_log_start":"234'392","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3276800,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":400,"num_write_kb":3200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"246","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:12.215875+0000","last_change":"2020-08-07T15:12:07.583222+0000","last_active":"2020-08-07T15:12:12.215875+0000","last_peered":"2020-08-07T15:12:12.215875+0000","last_clean":"2020-08-07T15:12:12.215875+0000","last_became_active":"2020-08-07T15:12:06.366915+0000","last_became_peered":"2020-08-07T15:12:06.366915+0000","last_unstale":"2020-08-07T15:12:12.215875+0000","last_undegraded":"2020-08-07T15:12:12.215875+0000","last_fullsized":"2020-08-07T15:12:12.215875+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:07.583152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.583152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"319","reported_epoch":"234","state":"active+clean","last_fresh":"2020-08-07T15:12:15.419149+0000","last_change":"2020-08-07T15:12:15.163743+0000","last_active":"2020-08-07T15:12:15.419149+0000","last_peered":"2020-08-07T15:12:15.419149+0000","last_clean":"2020-08-07T15:12:15.419149+0000","last_became_active":"2020-08-07T15:10:46.496544+0000","last_became_peered":"2020-08-07T15:10:46.496544+0000","last_unstale":"2020-08-07T15:12:15.419149+0000","last_undegraded":"2020-08-07T15:12:15.419149+0000","last_fullsized":"2020-08-07T15:12:15.419149+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:15.163639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:15.163639+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"234'392","reported_seq":"396","reported_epoch":"234","state":"active+clean","last_fresh":"2020-08-07T15:12:16.716272+0000","last_change":"2020-08-07T15:12:10.467447+0000","last_active":"2020-08-07T15:12:16.716272+0000","last_peered":"2020-08-07T15:12:16.716272+0000","last_clean":"2020-08-07T15:12:16.716272+0000","last_became_active":"2020-08-07T15:12:10.467000+0000","last_became_peered":"2020-08-07T15:12:10.467000+0000","last_unstale":"2020-08-07T15:12:16.716272+0000","last_undegraded":"2020-08-07T15:12:16.716272+0000","last_fullsized":"2020-08-07T15:12:16.716272+0000","mapping_epoch":230,"log_start":"234'383","ondisk_log_start":"234'383","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3211264,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":392,"num_write_kb":3136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"218","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:12.214114+0000","last_change":"2020-08-07T15:10:51.644951+0000","last_active":"2020-08-07T15:12:12.214114+0000","last_peered":"2020-08-07T15:12:12.214114+0000","last_clean":"2020-08-07T15:12:12.214114+0000","last_became_active":"2020-08-07T15:10:46.498230+0000","last_became_peered":"2020-08-07T15:10:46.498230+0000","last_unstale":"2020-08-07T15:12:12.214114+0000","last_undegraded":"2020-08-07T15:12:12.214114+0000","last_fullsized":"2020-08-07T15:12:12.214114+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:51.644875+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:10:51.644875+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"234'384","reported_seq":"396","reported_epoch":"234","state":"active+clean","last_fresh":"2020-08-07T15:12:16.601264+0000","last_change":"2020-08-07T15:12:10.465075+0000","last_active":"2020-08-07T15:12:16.601264+0000","last_peered":"2020-08-07T15:12:16.601264+0000","last_clean":"2020-08-07T15:12:16.601264+0000","last_became_active":"2020-08-07T15:12:10.464640+0000","last_became_peered":"2020-08-07T15:12:10.464640+0000","last_unstale":"2020-08-07T15:12:16.601264+0000","last_undegraded":"2020-08-07T15:12:16.601264+0000","last_fullsized":"2020-08-07T15:12:16.601264+0000","mapping_epoch":230,"log_start":"234'379","ondisk_log_start":"234'379","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3145728,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":384,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"58","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:12.262105+0000","last_change":"2020-08-07T15:11:37.490719+0000","last_active":"2020-08-07T15:12:12.262105+0000","last_peered":"2020-08-07T15:12:12.262105+0000","last_clean":"2020-08-07T15:12:12.262105+0000","last_became_active":"2020-08-07T15:11:23.232406+0000","last_became_peered":"2020-08-07T15:11:23.232406+0000","last_unstale":"2020-08-07T15:12:12.262105+0000","last_undegraded":"2020-08-07T15:12:12.262105+0000","last_fullsized":"2020-08-07T15:12:12.262105+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.490655+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.490655+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"237'680","reported_seq":"695","reported_epoch":"237","state":"active+clean","last_fresh":"2020-08-07T15:12:19.726043+0000","last_change":"2020-08-07T15:12:10.465359+0000","last_active":"2020-08-07T15:12:19.726043+0000","last_peered":"2020-08-07T15:12:19.726043+0000","last_clean":"2020-08-07T15:12:19.726043+0000","last_became_active":"2020-08-07T15:12:10.464900+0000","last_became_peered":"2020-08-07T15:12:10.464900+0000","last_unstale":"2020-08-07T15:12:19.726043+0000","last_undegraded":"2020-08-07T15:12:19.726043+0000","last_fullsized":"2020-08-07T15:12:19.726043+0000","mapping_epoch":230,"log_start":"237'675","ondisk_log_start":"237'675","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5570560,"num_objects":85,"num_object_clones":0,"num_object_copies":255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":680,"num_write_kb":5440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"282","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:12.215540+0000","last_change":"2020-08-07T15:10:58.029660+0000","last_active":"2020-08-07T15:12:12.215540+0000","last_peered":"2020-08-07T15:12:12.215540+0000","last_clean":"2020-08-07T15:12:12.215540+0000","last_became_active":"2020-08-07T15:10:46.499635+0000","last_became_peered":"2020-08-07T15:10:46.499635+0000","last_unstale":"2020-08-07T15:12:12.215540+0000","last_undegraded":"2020-08-07T15:12:12.215540+0000","last_fullsized":"2020-08-07T15:12:12.215540+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:58.029609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:58.029609+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"234'368","reported_seq":"380","reported_epoch":"234","state":"active+clean","last_fresh":"2020-08-07T15:12:16.695534+0000","last_change":"2020-08-07T15:12:10.468110+0000","last_active":"2020-08-07T15:12:16.695534+0000","last_peered":"2020-08-07T15:12:16.695534+0000","last_clean":"2020-08-07T15:12:16.695534+0000","last_became_active":"2020-08-07T15:12:10.467697+0000","last_became_peered":"2020-08-07T15:12:10.467697+0000","last_unstale":"2020-08-07T15:12:16.695534+0000","last_undegraded":"2020-08-07T15:12:16.695534+0000","last_fullsized":"2020-08-07T15:12:16.695534+0000","mapping_epoch":230,"log_start":"234'361","ondisk_log_start":"234'361","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3014656,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":368,"num_write_kb":2944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"252","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:13.680022+0000","last_change":"2020-08-07T15:12:13.680022+0000","last_active":"2020-08-07T15:12:13.680022+0000","last_peered":"2020-08-07T15:12:13.680022+0000","last_clean":"2020-08-07T15:12:13.680022+0000","last_became_active":"2020-08-07T15:12:06.380392+0000","last_became_peered":"2020-08-07T15:12:06.380392+0000","last_unstale":"2020-08-07T15:12:13.680022+0000","last_undegraded":"2020-08-07T15:12:13.680022+0000","last_fullsized":"2020-08-07T15:12:13.680022+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:13.679943+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.679943+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"234'480","reported_seq":"492","reported_epoch":"234","state":"active+clean","last_fresh":"2020-08-07T15:12:16.727117+0000","last_change":"2020-08-07T15:12:10.398816+0000","last_active":"2020-08-07T15:12:16.727117+0000","last_peered":"2020-08-07T15:12:16.727117+0000","last_clean":"2020-08-07T15:12:16.727117+0000","last_became_active":"2020-08-07T15:12:10.398332+0000","last_became_peered":"2020-08-07T15:12:10.398332+0000","last_unstale":"2020-08-07T15:12:16.727117+0000","last_undegraded":"2020-08-07T15:12:16.727117+0000","last_fullsized":"2020-08-07T15:12:16.727117+0000","mapping_epoch":230,"log_start":"234'475","ondisk_log_start":"234'475","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":480,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"207","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:14.241339+0000","last_change":"2020-08-07T15:12:14.241339+0000","last_active":"2020-08-07T15:12:14.241339+0000","last_peered":"2020-08-07T15:12:14.241339+0000","last_clean":"2020-08-07T15:12:14.241339+0000","last_became_active":"2020-08-07T15:10:46.498051+0000","last_became_peered":"2020-08-07T15:10:46.498051+0000","last_unstale":"2020-08-07T15:12:14.241339+0000","last_undegraded":"2020-08-07T15:12:14.241339+0000","last_fullsized":"2020-08-07T15:12:14.241339+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:14.241262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:14.241262+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"234'512","reported_seq":"524","reported_epoch":"234","state":"active+clean","last_fresh":"2020-08-07T15:12:16.703474+0000","last_change":"2020-08-07T15:12:10.398015+0000","last_active":"2020-08-07T15:12:16.703474+0000","last_peered":"2020-08-07T15:12:16.703474+0000","last_clean":"2020-08-07T15:12:16.703474+0000","last_became_active":"2020-08-07T15:12:10.397610+0000","last_became_peered":"2020-08-07T15:12:10.397610+0000","last_unstale":"2020-08-07T15:12:16.703474+0000","last_undegraded":"2020-08-07T15:12:16.703474+0000","last_fullsized":"2020-08-07T15:12:16.703474+0000","mapping_epoch":230,"log_start":"234'504","ondisk_log_start":"234'504","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4194304,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":512,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"272","reported_epoch":"234","state":"active+clean","last_fresh":"2020-08-07T15:12:15.419102+0000","last_change":"2020-08-07T15:12:13.176485+0000","last_active":"2020-08-07T15:12:15.419102+0000","last_peered":"2020-08-07T15:12:15.419102+0000","last_clean":"2020-08-07T15:12:15.419102+0000","last_became_active":"2020-08-07T15:08:41.689435+0000","last_became_peered":"2020-08-07T15:08:41.689435+0000","last_unstale":"2020-08-07T15:12:15.419102+0000","last_undegraded":"2020-08-07T15:12:15.419102+0000","last_fullsized":"2020-08-07T15:12:15.419102+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.176383+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"234'392","reported_seq":"402","reported_epoch":"234","state":"active+clean","last_fresh":"2020-08-07T15:12:16.732108+0000","last_change":"2020-08-07T15:12:10.468495+0000","last_active":"2020-08-07T15:12:16.732108+0000","last_peered":"2020-08-07T15:12:16.732108+0000","last_clean":"2020-08-07T15:12:16.732108+0000","last_became_active":"2020-08-07T15:12:10.467915+0000","last_became_peered":"2020-08-07T15:12:10.467915+0000","last_unstale":"2020-08-07T15:12:16.732108+0000","last_undegraded":"2020-08-07T15:12:16.732108+0000","last_fullsized":"2020-08-07T15:12:16.732108+0000","mapping_epoch":230,"log_start":"234'384","ondisk_log_start":"234'384","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3211264,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":392,"num_write_kb":3136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,2],"acting":[7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"168","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:12.215714+0000","last_change":"2020-08-07T15:11:10.853414+0000","last_active":"2020-08-07T15:12:12.215714+0000","last_peered":"2020-08-07T15:12:12.215714+0000","last_clean":"2020-08-07T15:12:12.215714+0000","last_became_active":"2020-08-07T15:10:46.493498+0000","last_became_peered":"2020-08-07T15:10:46.493498+0000","last_unstale":"2020-08-07T15:12:12.215714+0000","last_undegraded":"2020-08-07T15:12:12.215714+0000","last_fullsized":"2020-08-07T15:12:12.215714+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:10.853337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:10.853337+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"234'382","reported_seq":"388","reported_epoch":"234","state":"active+clean","last_fresh":"2020-08-07T15:12:16.717175+0000","last_change":"2020-08-07T15:12:10.465251+0000","last_active":"2020-08-07T15:12:16.717175+0000","last_peered":"2020-08-07T15:12:16.717175+0000","last_clean":"2020-08-07T15:12:16.717175+0000","last_became_active":"2020-08-07T15:12:10.464698+0000","last_became_peered":"2020-08-07T15:12:10.464698+0000","last_unstale":"2020-08-07T15:12:16.717175+0000","last_undegraded":"2020-08-07T15:12:16.717175+0000","last_fullsized":"2020-08-07T15:12:16.717175+0000","mapping_epoch":230,"log_start":"234'375","ondisk_log_start":"234'375","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3129344,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":382,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"202","reported_epoch":"233","state":"active+clean","last_fresh":"2020-08-07T15:12:12.214119+0000","last_change":"2020-08-07T15:12:06.367666+0000","last_active":"2020-08-07T15:12:12.214119+0000","last_peered":"2020-08-07T15:12:12.214119+0000","last_clean":"2020-08-07T15:12:12.214119+0000","last_became_active":"2020-08-07T15:12:06.367081+0000","last_became_peered":"2020-08-07T15:12:06.367081+0000","last_unstale":"2020-08-07T15:12:12.214119+0000","last_undegraded":"2020-08-07T15:12:12.214119+0000","last_fullsized":"2020-08-07T15:12:12.214119+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:37.601099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:37.601099+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":64110592,"num_objects":979,"num_object_clones":0,"num_object_copies":2937,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7826,"num_write_kb":62608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":185393152,"data_stored":185393152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":31,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":149,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":93,"acting":93,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542218,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1067872,"kb_used_data":19288,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93303968,"statfs":{"total":96636764160,"available":95543263232,"internally_reserved":1073741824,"allocated":19750912,"data_stored":18659679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:11:28 2020","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":9.548,"15min":3.778},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.1970000000000001,"5min":856.07100000000003,"15min":856.07100000000003},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0389999999999999,"15min":0.98799999999999999},"min":{"1min":0.53400000000000003,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":3.4159999999999999,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":0.65300000000000002}]},{"osd":1,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.335,"15min":1.3080000000000001},"min":{"1min":0.64200000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.9199999999999999,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.141,"15min":1.024},"min":{"1min":0.61799999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":4.0060000000000002,"5min":4.0060000000000002,"15min":4.0060000000000002},"last":1.1000000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:11:28 2020","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.292,"15min":1.1819999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.4359999999999999,"5min":5.6619999999999999,"15min":5.6619999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.105,"5min":1.3360000000000001,"15min":1.1899999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.109,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":1.141}]},{"osd":3,"last update":"Fri Aug 7 15:11:28 2020","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":16.053000000000001,"15min":6.4649999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.5409999999999999,"5min":917.38699999999994,"15min":917.38699999999994},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.1299999999999999,"5min":4.9219999999999997,"15min":2.536},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.4750000000000001,"5min":333.59199999999998,"15min":333.59199999999998},"last":0.93799999999999994}]},{"osd":4,"last update":"Fri Aug 7 15:11:28 2020","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.427,"15min":1.3260000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.1469999999999998,"5min":4.835,"15min":4.835},"last":1.125},{"interface":"front","average":{"1min":1.125,"5min":11.481,"15min":4.7009999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.4300000000000002,"5min":866.24099999999999,"15min":866.24099999999999},"last":0.88900000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:11:28 2020","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":16.597000000000001,"15min":30.536999999999999},"min":{"1min":0.70599999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.5870000000000002,"5min":831.88099999999997,"15min":831.88099999999997},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.8049999999999999,"15min":1.5980000000000001},"min":{"1min":0.93700000000000006,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.6200000000000001,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":0.91400000000000003}]},{"osd":7,"last update":"Fri Aug 7 15:11:28 2020","interfaces":[{"interface":"back","average":{"1min":1.609,"5min":4.0119999999999996,"15min":2.3490000000000002},"min":{"1min":0.873,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":5.4800000000000004,"5min":224.14500000000001,"15min":224.14500000000001},"last":1.01},{"interface":"front","average":{"1min":1.716,"5min":1.732,"15min":1.5820000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":5.5309999999999997,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":1.0469999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1071648,"kb_used_data":23056,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93300192,"statfs":{"total":96636764160,"available":95539396608,"internally_reserved":1073741824,"allocated":23609344,"data_stored":22518111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":42.923000000000002,"5min":9.4640000000000004,"15min":4.0430000000000001},"min":{"1min":0.48999999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":755.10000000000002,"5min":755.10000000000002,"15min":755.10000000000002},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.96699999999999997,"5min":9.7780000000000005,"15min":4.0259999999999998},"min":{"1min":0.46300000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.718,"5min":529.88699999999994,"15min":529.88699999999994},"last":2.1989999999999998}]},{"osd":1,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":5.758,"15min":2.5579999999999998},"min":{"1min":0.47699999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.7290000000000001,"5min":547.65300000000002,"15min":547.65300000000002},"last":0.997},{"interface":"front","average":{"1min":1.4159999999999999,"5min":4.9189999999999996,"15min":2.5640000000000001},"min":{"1min":0.57399999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":5.4960000000000004,"5min":372.50799999999998,"15min":372.50799999999998},"last":1.0680000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.3240000000000001,"15min":1.238},"min":{"1min":0.54900000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":6.968,"5min":6.968,"15min":6.968},"last":1.044},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1799999999999999,"15min":1.2410000000000001},"min":{"1min":0.58599999999999997,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.7709999999999999,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":2.6579999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":9.5,"15min":4.1840000000000002},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":3.3540000000000001,"5min":698.19899999999996,"15min":698.19899999999996},"last":1.28},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.2969999999999999,"15min":1.357},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":4.1779999999999999,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":2.617}]},{"osd":4,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.3120000000000001,"15min":1.546},"min":{"1min":0.60999999999999999,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.5449999999999999,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":1.73},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.53,"15min":1.6240000000000001},"min":{"1min":0.71399999999999997,"5min":0.621,"15min":0.621},"max":{"1min":2.9900000000000002,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.762}]},{"osd":5,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.7130000000000001,"5min":1.329,"15min":1.3520000000000001},"min":{"1min":0.749,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":6.2080000000000002,"5min":6.2080000000000002,"15min":6.2080000000000002},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.7350000000000001,"5min":1.526,"15min":1.6859999999999999},"min":{"1min":0.89000000000000001,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":3.052,"5min":5.327,"15min":5.327},"last":1.958}]},{"osd":6,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":40.012999999999998,"5min":9.1820000000000004,"15min":4.016},"min":{"1min":0.86399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":695.92899999999997,"5min":695.92899999999997,"15min":695.92899999999997},"last":1.851},{"interface":"front","average":{"1min":1.613,"5min":4.8959999999999999,"15min":13.849},"min":{"1min":0.91300000000000003,"5min":0.87,"15min":0.87},"max":{"1min":3.0270000000000001,"5min":266.26999999999998,"15min":266.26999999999998},"last":2.2869999999999999}]}]},{"osd":1,"up_from":227,"seq":974957576196,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1085604,"kb_used_data":37020,"kb_used_omap":2,"kb_used_meta":1048573,"kb_avail":93286236,"statfs":{"total":96636764160,"available":95525105664,"internally_reserved":1073741824,"allocated":37908480,"data_stored":36805798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":1073739245},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.6399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7589999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1059736,"kb_used_data":11152,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93312104,"statfs":{"total":96636764160,"available":95551594496,"internally_reserved":1073741824,"allocated":11419648,"data_stored":10328415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":24.291,"5min":21.719999999999999,"15min":7.891},"min":{"1min":0.52700000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":432.899,"5min":845.57100000000003,"15min":845.57100000000003},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.752,"5min":1.181,"15min":1.0149999999999999},"min":{"1min":0.62,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":9.3650000000000002,"5min":9.3650000000000002,"15min":9.3650000000000002},"last":0.68999999999999995}]},{"osd":2,"last update":"Fri Aug 7 15:11:32 2020","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":1.0069999999999999,"15min":0.95399999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.5289999999999999,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.5629999999999999,"5min":9.9559999999999995,"15min":25.145},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":4.8689999999999998,"5min":606.18499999999995,"15min":606.18499999999995},"last":1.7070000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:11:32 2020","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.264,"15min":1.0449999999999999},"min":{"1min":0.54900000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.298,"5min":9.3360000000000003,"15min":9.3360000000000003},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.121,"5min":1.409,"15min":1.248},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.9330000000000001,"5min":11.19,"15min":11.19},"last":1.4159999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:11:32 2020","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.321,"15min":1.2170000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.7589999999999999,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":0.996},{"interface":"front","average":{"1min":1.198,"5min":1.4390000000000001,"15min":1.3180000000000001},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.887,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":1.48}]},{"osd":5,"last update":"Fri Aug 7 15:11:32 2020","interfaces":[{"interface":"back","average":{"1min":28.428000000000001,"5min":14.52,"15min":5.6580000000000004},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":539.31100000000004,"5min":539.31100000000004,"15min":539.31100000000004},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.444,"5min":1.4419999999999999,"15min":1.218},"min":{"1min":0.77600000000000002,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":3.5790000000000002,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":1.0229999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:11:32 2020","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":2.8889999999999998,"15min":6.7050000000000001},"min":{"1min":0.84099999999999997,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":3.1899999999999999,"5min":124.738,"15min":124.738},"last":1.45},{"interface":"front","average":{"1min":1.5940000000000001,"5min":4.9909999999999997,"15min":14.099},"min":{"1min":0.81799999999999995,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":5.1799999999999997,"5min":294.94,"15min":294.94},"last":1.371}]},{"osd":7,"last update":"Fri Aug 7 15:11:32 2020","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.5629999999999999,"15min":1.5780000000000001},"min":{"1min":0.90600000000000003,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":4.0010000000000003,"5min":6.468,"15min":6.468},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.492,"5min":10.583,"15min":4.4509999999999996},"min":{"1min":0.90600000000000003,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":3.1749999999999998,"5min":859.82399999999996,"15min":859.82399999999996},"last":1.3380000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574923,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1076256,"kb_used_data":27672,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93295584,"statfs":{"total":96636764160,"available":95534678016,"internally_reserved":1073741824,"allocated":28336128,"data_stored":27244895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:11:22 2020","interfaces":[{"interface":"back","average":{"1min":2.2519999999999998,"5min":1.4950000000000001,"15min":1.2430000000000001},"min":{"1min":0.46000000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":23.699000000000002,"5min":23.699000000000002,"15min":23.699000000000002},"last":1.0860000000000001},{"interface":"front","average":{"1min":2.1059999999999999,"5min":1.417,"15min":1.1220000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":23.792999999999999,"5min":23.792999999999999,"15min":23.792999999999999},"last":2.169}]},{"osd":1,"last update":"Fri Aug 7 15:10:21 2020","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.6970000000000001,"15min":1.2929999999999999},"min":{"1min":0.52000000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":5.5919999999999996,"5min":18.672000000000001,"15min":18.672000000000001},"last":1.7270000000000001},{"interface":"front","average":{"1min":1.264,"5min":1.3320000000000001,"15min":1.3500000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":3.7970000000000002,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":3.2440000000000002}]},{"osd":3,"last update":"Fri Aug 7 15:11:22 2020","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":1.234,"15min":1.1859999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.681,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":3.4689999999999999},{"interface":"front","average":{"1min":15.957000000000001,"5min":10.978999999999999,"15min":4.6970000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":312.87900000000002,"5min":501.75599999999997,"15min":501.75599999999997},"last":3.3980000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:11:22 2020","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.603,"15min":1.274},"min":{"1min":0.623,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":7.9569999999999999,"5min":19.783000000000001,"15min":19.783000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.8049999999999999,"5min":1.554,"15min":1.613},"min":{"1min":0.80700000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":8.048,"5min":8.048,"15min":8.048},"last":1.351}]},{"osd":5,"last update":"Fri Aug 7 15:11:22 2020","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.367,"15min":1.2969999999999999},"min":{"1min":0.72899999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":4.3639999999999999,"5min":5.2220000000000004,"15min":5.2220000000000004},"last":1.385},{"interface":"front","average":{"1min":1.2989999999999999,"5min":20.52,"15min":23.856000000000002},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":3.7559999999999998,"5min":627.53300000000002,"15min":627.53300000000002},"last":1.6870000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:11:34 2020","interfaces":[{"interface":"back","average":{"1min":42.643000000000001,"5min":9.9380000000000006,"15min":4.1260000000000003},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":821.91899999999998,"5min":821.91899999999998,"15min":821.91899999999998},"last":1.615},{"interface":"front","average":{"1min":1.2190000000000001,"5min":9.9429999999999996,"15min":4.1950000000000003},"min":{"1min":0.80200000000000005,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.73,"5min":751.70000000000005,"15min":751.70000000000005},"last":2.0529999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:11:22 2020","interfaces":[{"interface":"back","average":{"1min":1.899,"5min":12.601000000000001,"15min":5.3609999999999998},"min":{"1min":0.88500000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":7.9900000000000002,"5min":501.91699999999997,"15min":501.91699999999997},"last":1.421},{"interface":"front","average":{"1min":2.044,"5min":2.0630000000000002,"15min":1.6850000000000001},"min":{"1min":0.94199999999999995,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":8.0779999999999994,"5min":35.646999999999998,"15min":35.646999999999998},"last":1.647}]}]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1081576,"kb_used_data":32992,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93290264,"statfs":{"total":96636764160,"available":95529230336,"internally_reserved":1073741824,"allocated":33783808,"data_stored":32692575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.2689999999999999,"15min":1.258},"min":{"1min":0.51100000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":3.2149999999999999,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.296,"15min":1.5229999999999999},"min":{"1min":0.53900000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":4.0190000000000001,"5min":13.067,"15min":13.067},"last":1.0900000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":4.4779999999999998,"15min":2.758},"min":{"1min":0.59099999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.847,"5min":270.68700000000001,"15min":270.68700000000001},"last":2.5990000000000002},{"interface":"front","average":{"1min":1.109,"5min":1.8580000000000001,"15min":1.96},"min":{"1min":0.54100000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.0550000000000002,"5min":14.349,"15min":14.349},"last":2.085}]},{"osd":2,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.3380000000000001,"15min":1.2110000000000001},"min":{"1min":0.65300000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.6389999999999998,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":1.984},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.9810000000000001,"15min":1.7829999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":6.7889999999999997,"5min":51.530000000000001,"15min":51.530000000000001},"last":1.9610000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":17.366,"15min":6.9199999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":3.1600000000000001,"5min":646.26700000000005,"15min":646.26700000000005},"last":0.91300000000000003},{"interface":"front","average":{"1min":43.484000000000002,"5min":21.631,"15min":28.609999999999999},"min":{"1min":0.77100000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":844.10500000000002,"5min":844.10500000000002,"15min":844.10500000000002},"last":1.2270000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":5.9560000000000004,"15min":2.9969999999999999},"min":{"1min":0.79300000000000004,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.2370000000000001,"5min":403.88400000000001,"15min":403.88400000000001},"last":1.7070000000000001},{"interface":"front","average":{"1min":1.591,"5min":1.895,"15min":1.8109999999999999},"min":{"1min":0.97399999999999998,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":3.9790000000000001,"5min":8.9120000000000008,"15min":8.9120000000000008},"last":1.744}]},{"osd":6,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.653,"15min":1.5720000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":4.1600000000000001,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.944,"5min":13.427,"15min":5.7999999999999998},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":6.9980000000000002,"5min":688.15999999999997,"15min":688.15999999999997},"last":1.5189999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.655,"5min":1.6799999999999999,"15min":2.0009999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":6.8209999999999997,"5min":6.8209999999999997,"15min":6.8209999999999997},"last":1.05},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.77,"15min":1.9079999999999999},"min":{"1min":0.77000000000000002,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":3.1499999999999999,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":1.776}]}]},{"osd":4,"up_from":13,"seq":55834574922,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1068304,"kb_used_data":19720,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93303536,"statfs":{"total":96636764160,"available":95542820864,"internally_reserved":1073741824,"allocated":20193280,"data_stored":19102047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:11:25 2020","interfaces":[{"interface":"back","average":{"1min":20.597999999999999,"5min":9.9239999999999995,"15min":4.0949999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":356.91800000000001,"5min":399.73700000000002,"15min":399.73700000000002},"last":1.1559999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":1.046,"15min":0.91400000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.972,"5min":6.3970000000000002,"15min":6.3970000000000002},"last":1.6040000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:10:24 2020","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":5.7629999999999999,"15min":2.823},"min":{"1min":0.52900000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.8109999999999999,"5min":320.67000000000002,"15min":320.67000000000002},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.177,"5min":7.5300000000000002,"15min":3.4279999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.7200000000000002,"5min":689.846,"15min":689.846},"last":1.778}]},{"osd":2,"last update":"Fri Aug 7 15:11:25 2020","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.3680000000000001,"15min":1.4239999999999999},"min":{"1min":0.67300000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.786,"5min":6.8730000000000002,"15min":6.8730000000000002},"last":1.6659999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.51,"15min":1.5129999999999999},"min":{"1min":0.85899999999999999,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.8140000000000001,"5min":7.1360000000000001,"15min":7.1360000000000001},"last":1.704}]},{"osd":3,"last update":"Fri Aug 7 15:11:25 2020","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":12.045,"15min":5.0199999999999996},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.6579999999999999,"5min":638.92100000000005,"15min":638.92100000000005},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.3480000000000001,"15min":1.2829999999999999},"min":{"1min":0.70899999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.0289999999999999,"5min":6.431,"15min":6.431},"last":3.2360000000000002}]},{"osd":5,"last update":"Fri Aug 7 15:11:25 2020","interfaces":[{"interface":"back","average":{"1min":41.594000000000001,"5min":18.439,"15min":7.2309999999999999},"min":{"1min":0.60499999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":725.98599999999999,"5min":990.71600000000001,"15min":990.71600000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.4259999999999999,"15min":1.5600000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.7310000000000001,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":1.504}]},{"osd":6,"last update":"Fri Aug 7 15:11:34 2020","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.5800000000000001,"15min":1.72},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":5.4459999999999997,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":1.635},{"interface":"front","average":{"1min":1.256,"5min":4.9329999999999998,"15min":14.382999999999999},"min":{"1min":0.71399999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.3719999999999999,"5min":389.14600000000002,"15min":389.14600000000002},"last":1.526}]},{"osd":7,"last update":"Fri Aug 7 15:11:25 2020","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":3.782,"15min":2.1989999999999998},"min":{"1min":0.78300000000000003,"5min":0.62,"15min":0.62},"max":{"1min":2.5310000000000001,"5min":252.798,"15min":252.798},"last":1.881},{"interface":"front","average":{"1min":1.4370000000000001,"5min":14.112,"15min":5.8010000000000002},"min":{"1min":0.78100000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":3.0369999999999999,"5min":501.93000000000001,"15min":501.93000000000001},"last":1.738}]}]},{"osd":5,"up_from":13,"seq":55834574922,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1064104,"kb_used_data":15520,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93307736,"statfs":{"total":96636764160,"available":95547121664,"internally_reserved":1073741824,"allocated":15892480,"data_stored":14801247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:11:26 2020","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.996,"15min":1.05},"min":{"1min":0.54400000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.5289999999999999,"5min":3.7349999999999999,"15min":3.7349999999999999},"last":1.1990000000000001},{"interface":"front","average":{"1min":15.863,"5min":3.956,"15min":1.855},"min":{"1min":0.44,"5min":0.438,"15min":0.438},"max":{"1min":329.11399999999998,"5min":329.11399999999998,"15min":329.11399999999998},"last":0.58299999999999996}]},{"osd":1,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":8.5239999999999991,"5min":12.039999999999999,"15min":4.6319999999999997},"min":{"1min":0.56799999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":139.88200000000001,"5min":787.22900000000004,"15min":787.22900000000004},"last":4.4039999999999999},{"interface":"front","average":{"1min":1.448,"5min":2.5649999999999999,"15min":1.593},"min":{"1min":0.59899999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":4.3700000000000001,"5min":116.548,"15min":116.548},"last":4.0810000000000004}]},{"osd":2,"last update":"Fri Aug 7 15:11:26 2020","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":2.714,"15min":7.0510000000000002},"min":{"1min":0.52100000000000002,"5min":0.499,"15min":0.499},"max":{"1min":2.109,"5min":168.56899999999999,"15min":168.56899999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.3939999999999999,"5min":7.6200000000000001,"15min":24.626999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":3.5609999999999999,"5min":676.27099999999996,"15min":676.27099999999996},"last":0.61299999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:11:26 2020","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.6559999999999999,"15min":1.2969999999999999},"min":{"1min":0.71499999999999997,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":5.1539999999999999,"5min":29.129000000000001,"15min":29.129000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.1140000000000001,"5min":9.9969999999999999,"15min":4.141},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.149,"5min":712.98699999999997,"15min":712.98699999999997},"last":1.163}]},{"osd":4,"last update":"Fri Aug 7 15:11:26 2020","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":22.683,"15min":41.488999999999997},"min":{"1min":0.54800000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.5899999999999999,"5min":971.67999999999995,"15min":971.67999999999995},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.272,"15min":1.294},"min":{"1min":0.63700000000000001,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":5.4900000000000002,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":1.0920000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:11:26 2020","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":31.462,"15min":44.093000000000004},"min":{"1min":0.85099999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":5.4480000000000004,"5min":966.60900000000004,"15min":966.60900000000004},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.2789999999999999,"5min":11.677,"15min":21.236999999999998},"min":{"1min":0.871,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.4710000000000001,"5min":517.26700000000005,"15min":517.26700000000005},"last":1.304}]},{"osd":7,"last update":"Fri Aug 7 15:11:26 2020","interfaces":[{"interface":"back","average":{"1min":1.6799999999999999,"5min":1.403,"15min":1.3400000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":5.1559999999999997,"5min":5.1559999999999997,"15min":5.1559999999999997},"last":1.1259999999999999},{"interface":"front","average":{"1min":32.561999999999998,"5min":12.834,"15min":5.3220000000000001},"min":{"1min":0.63700000000000001,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":501.64299999999997,"5min":515.09100000000001,"15min":515.09100000000001},"last":1.1379999999999999}]}]}]}} 2020-08-07T15:12:20.350 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:12:20.351 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:12:20.351 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:12:20.351 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:12:20.351 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:12:20.352 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:12:20.352 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:12:20.352 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:12:20.352 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:12:20.353 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:20.353 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:12:20.353 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:12:20.353 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:12:20.353 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:20.355 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:12:20.414 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:20.406+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:20.489 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:20.486+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:20.543 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:20.543 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:20.584 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:12:20.592 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:20.591+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:20.715 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:20.704+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:20.770 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:20.771 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:20.787 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:12:20.811 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:20.809+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:20.914 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:20.909+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:21.015 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:21.013+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:21.026 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:21.026 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:21.034 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:12:21.115 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:21.114+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:21.220 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:21.217+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:21.262 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:21.262 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:21.266 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 9 16 13713 13697 11.888 12.2578 0.015994 0.0105054 2020-08-07T15:12:21.278 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:12:21.321 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:21.318+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:21.423 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:21.421+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:21.505 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:21.507 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:21.527 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:12:21.528 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:21.524+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:21.627 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:21.626+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:21.730 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:21.726+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:21.750 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:21.750 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:21.761 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:12:21.787 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:12:21.848 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:21.845+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:21.939 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:21.938+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:21.965 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:21.965 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:21.976 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:12:22.065 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:22.060+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:22.150 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:22.143+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:22.226 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:22.227 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:22.244 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:12:22.258 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:22.247+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:22.266 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 10 16 15203 15187 11.8632 11.6406 0.00821121 0.0105222 2020-08-07T15:12:22.352 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:22.350+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:22.367 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:12:22.456 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:22.453+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:22.482 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:22.483 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:22.499 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:12:22.557 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:22.555+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:22.618 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:12:22.650 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:12:22.658 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:22.656+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:22.700 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:12:22.700 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:22.701 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:12:22.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:12:22.701 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:12:22.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.707 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:30968 4.4 e98/81 ondisk, result = 0)", 2020-08-07T15:12:22.708 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:12:22.708 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 216.963931516, 2020-08-07T15:12:22.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96211418999999998, 2020-08-07T15:12:22.709 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:22.709 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:12:22.709 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:22.710 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.710 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:22.710 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:12:22.711 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:22.711 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:22.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:12:22.722 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2387e-05 2020-08-07T15:12:22.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.726 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:22.727 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729821+0000", 2020-08-07T15:12:22.727 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9909999999999999e-06 2020-08-07T15:12:22.727 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:22.728 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729827+0000", 2020-08-07T15:12:22.729 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96185524600000005 2020-08-07T15:12:22.729 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.729 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:22.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691682+0000", 2020-08-07T15:12:22.755 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8760000000000005e-06 2020-08-07T15:12:22.757 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.757 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:22.758 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691690+0000", 2020-08-07T15:12:22.758 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9682000000000001e-05 2020-08-07T15:12:22.758 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.759 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:22.759 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691759+0000", 2020-08-07T15:12:22.760 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5381000000000001e-05 2020-08-07T15:12:22.760 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.760 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.761 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:22.761 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691775+0000", 2020-08-07T15:12:22.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000127627 2020-08-07T15:12:22.783 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:22.787 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691902+0000", 2020-08-07T15:12:22.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96211418999999998 2020-08-07T15:12:22.790 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:22.790 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:22.790 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:22.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.797 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42472 4.4 4:2ff82af8:::benchmark_data_smithi074_44586_object5308:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:12:22.797 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:12:22.798 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 202.426468742, 2020-08-07T15:12:22.798 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89653816399999997, 2020-08-07T15:12:22.798 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:22.798 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:22.798 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:22.799 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:22.799 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:22.799 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42472 2020-08-07T15:12:22.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.799 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:22.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:22.800 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:12:22.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:22.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.801 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.801 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:22.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:12:22.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9591e-05 2020-08-07T15:12:22.801 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.814 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.814 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:22.815 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267301+0000", 2020-08-07T15:12:22.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7490000000000004e-06 2020-08-07T15:12:22.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:22.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267306+0000", 2020-08-07T15:12:22.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2850000000000001e-05 2020-08-07T15:12:22.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:22.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267319+0000", 2020-08-07T15:12:22.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9100000000000002e-06 2020-08-07T15:12:22.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:22.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267325+0000", 2020-08-07T15:12:22.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6584000000000001e-05 2020-08-07T15:12:22.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:22.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267362+0000", 2020-08-07T15:12:22.828 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9188000000000001e-05 2020-08-07T15:12:22.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:22.829 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267451+0000", 2020-08-07T15:12:22.829 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000182917 2020-08-07T15:12:22.829 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.829 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.829 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:22.830 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267634+0000", 2020-08-07T15:12:22.830 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002484708 2020-08-07T15:12:22.830 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.830 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.830 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:22.831 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270119+0000", 2020-08-07T15:12:22.831 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0299999999999997e-07 2020-08-07T15:12:22.831 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.831 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.831 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:22.832 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270119+0000", 2020-08-07T15:12:22.838 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8251999999999996e-05 2020-08-07T15:12:22.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.839 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:22.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270188+0000", 2020-08-07T15:12:22.840 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24443688 2020-08-07T15:12:22.840 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.840 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.840 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:22.840 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.514625+0000", 2020-08-07T15:12:22.841 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.64911260599999998 2020-08-07T15:12:22.841 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:22.841 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163737+0000", 2020-08-07T15:12:22.842 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2100000000000001e-06 2020-08-07T15:12:22.842 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.842 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.842 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:22.842 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163738+0000", 2020-08-07T15:12:22.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0816000000000003e-05 2020-08-07T15:12:22.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.855 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:22.855 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163789+0000", 2020-08-07T15:12:22.855 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89653816399999997 2020-08-07T15:12:22.856 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:22.856 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:22.856 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:22.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.857 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:42472 4.4 e112/81 ondisk, result = 0)", 2020-08-07T15:12:22.857 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.271353+0000", 2020-08-07T15:12:22.857 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 202.422366705, 2020-08-07T15:12:22.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89244306299999998, 2020-08-07T15:12:22.858 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:22.858 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:12:22.858 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:22.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.859 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:22.859 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.271353+0000", 2020-08-07T15:12:22.859 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:22.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.859 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:22.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.271353+0000", 2020-08-07T15:12:22.860 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7875999999999999e-05 2020-08-07T15:12:22.860 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.860 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:22.861 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.271381+0000", 2020-08-07T15:12:22.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4819999999999996e-06 2020-08-07T15:12:22.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.882 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:22.898 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.271385+0000", 2020-08-07T15:12:22.898 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89228763300000002 2020-08-07T15:12:22.898 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.899 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.899 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:22.899 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163673+0000", 2020-08-07T15:12:22.899 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9949999999999996e-06 2020-08-07T15:12:22.899 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.900 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:22.900 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163678+0000", 2020-08-07T15:12:22.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5485000000000003e-05 2020-08-07T15:12:22.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.901 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.901 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:22.901 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163714+0000", 2020-08-07T15:12:22.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4785e-05 2020-08-07T15:12:22.901 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:22.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163728+0000", 2020-08-07T15:12:22.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7806999999999996e-05 2020-08-07T15:12:22.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:22.926 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163796+0000", 2020-08-07T15:12:22.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89244306299999998 2020-08-07T15:12:22.926 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:22.927 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:22.927 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:22.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.927 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72688 4.4 4:295d3f6e:::benchmark_data_smithi074_44586_object9085:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e129)", 2020-08-07T15:12:22.928 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:12:22.928 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 177.29688655499999, 2020-08-07T15:12:22.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7869782759999999, 2020-08-07T15:12:22.929 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:22.929 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:22.930 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:22.930 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:22.930 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:22.930 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72688 2020-08-07T15:12:22.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.931 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:22.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:22.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:12:22.931 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:22.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.932 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.932 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:22.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:12:22.932 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8721999999999999e-05 2020-08-07T15:12:22.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.933 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.933 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:22.933 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396882+0000", 2020-08-07T15:12:22.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2639999999999999e-06 2020-08-07T15:12:22.946 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.946 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.947 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:22.947 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396887+0000", 2020-08-07T15:12:22.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0767999999999999e-05 2020-08-07T15:12:22.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.947 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.948 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:22.948 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396898+0000", 2020-08-07T15:12:22.948 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1860000000000002e-06 2020-08-07T15:12:22.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.949 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:22.949 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396902+0000", 2020-08-07T15:12:22.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0037638649999999999 2020-08-07T15:12:22.949 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:22.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400666+0000", 2020-08-07T15:12:22.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000118077 2020-08-07T15:12:22.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:22.978 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400784+0000", 2020-08-07T15:12:22.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000136728 2020-08-07T15:12:22.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.979 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:12:22.979 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400921+0000", 2020-08-07T15:12:22.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001595443 2020-08-07T15:12:22.979 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.979 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:22.980 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.402516+0000", 2020-08-07T15:12:22.980 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.014817907 2020-08-07T15:12:22.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.981 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:22.981 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417334+0000", 2020-08-07T15:12:22.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.75e-06 2020-08-07T15:12:22.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:22.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:22.982 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:22.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417336+0000", 2020-08-07T15:12:22.982 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8066999999999996e-05 2020-08-07T15:12:23.009 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.011 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:23.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417424+0000", 2020-08-07T15:12:23.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002210573 2020-08-07T15:12:23.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.419635+0000", 2020-08-07T15:12:23.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1379000000000001e-05 2020-08-07T15:12:23.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.023 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.419646+0000", 2020-08-07T15:12:23.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.004643409 2020-08-07T15:12:23.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424289+0000", 2020-08-07T15:12:23.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7563000000000001e-05 2020-08-07T15:12:23.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.034 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424307+0000", 2020-08-07T15:12:23.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75943629400000001 2020-08-07T15:12:23.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183743+0000", 2020-08-07T15:12:23.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8281000000000006e-05 2020-08-07T15:12:23.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:23.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183812+0000", 2020-08-07T15:12:23.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7869782759999999 2020-08-07T15:12:23.036 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.036 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:23.037 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.037 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72985 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:23.038 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:12:23.038 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 177.08090073599999, 2020-08-07T15:12:23.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869466839999999, 2020-08-07T15:12:23.039 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:23.040 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:23.053 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:23.053 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:23.054 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:23.054 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72985 2020-08-07T15:12:23.054 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.055 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:23.055 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:23.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:12:23.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:23.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:23.057 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:12:23.059 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8003999999999997e-05 2020-08-07T15:12:23.059 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:23.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612857+0000", 2020-08-07T15:12:23.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.246e-06 2020-08-07T15:12:23.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.065 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:23.065 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612863+0000", 2020-08-07T15:12:23.066 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5146e-05 2020-08-07T15:12:23.066 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.066 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.066 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:23.067 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612878+0000", 2020-08-07T15:12:23.067 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1940000000000003e-06 2020-08-07T15:12:23.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.068 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.068 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:23.068 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612885+0000", 2020-08-07T15:12:23.069 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0349999999999997e-05 2020-08-07T15:12:23.069 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.069 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.069 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.069 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612935+0000", 2020-08-07T15:12:23.070 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1029999999999996e-06 2020-08-07T15:12:23.070 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.070 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.070 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.070 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612942+0000", 2020-08-07T15:12:23.071 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81140097700000002 2020-08-07T15:12:23.071 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.071 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.071 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.071 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424343+0000", 2020-08-07T15:12:23.072 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1799999999999996e-06 2020-08-07T15:12:23.072 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.072 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.072 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.072 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424351+0000", 2020-08-07T15:12:23.073 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75948665299999996 2020-08-07T15:12:23.073 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.073 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.073 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.073 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183838+0000", 2020-08-07T15:12:23.074 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000179187 2020-08-07T15:12:23.074 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.074 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.074 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:23.074 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.184017+0000", 2020-08-07T15:12:23.076 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024691600000000002 2020-08-07T15:12:23.076 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.076 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.076 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:23.077 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.184264+0000", 2020-08-07T15:12:23.077 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006362721 2020-08-07T15:12:23.077 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.078 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.078 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:23.078 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190627+0000", 2020-08-07T15:12:23.078 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7400000000000001e-06 2020-08-07T15:12:23.078 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.079 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.079 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:23.079 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190629+0000", 2020-08-07T15:12:23.079 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1305999999999999e-05 2020-08-07T15:12:23.079 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.080 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.080 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:23.080 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190690+0000", 2020-08-07T15:12:23.080 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0088507159999999998 2020-08-07T15:12:23.080 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.081 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.081 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.081 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199541+0000", 2020-08-07T15:12:23.081 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015864699999999999 2020-08-07T15:12:23.081 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.082 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.082 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199699+0000", 2020-08-07T15:12:23.082 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.23e-07 2020-08-07T15:12:23.082 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.083 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.083 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:23.083 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199700+0000", 2020-08-07T15:12:23.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5775e-05 2020-08-07T15:12:23.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.083 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:23.084 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199766+0000", 2020-08-07T15:12:23.084 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869466839999999 2020-08-07T15:12:23.084 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.084 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:23.085 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.085 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72986 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:23.086 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:12:23.086 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 177.080790154, 2020-08-07T15:12:23.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.586946441, 2020-08-07T15:12:23.087 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:23.087 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:23.087 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:23.087 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:23.087 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:23.088 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72986 2020-08-07T15:12:23.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.088 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:23.089 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.089 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:23.089 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:12:23.090 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:23.090 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.091 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.091 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:23.092 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:12:23.092 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9861999999999997e-05 2020-08-07T15:12:23.093 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.093 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.093 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:23.093 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612990+0000", 2020-08-07T15:12:23.093 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4350000000000001e-06 2020-08-07T15:12:23.094 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.094 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.094 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:23.094 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612993+0000", 2020-08-07T15:12:23.095 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9529999999999999e-06 2020-08-07T15:12:23.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.095 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.095 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:23.095 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613001+0000", 2020-08-07T15:12:23.095 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4309999999999998e-06 2020-08-07T15:12:23.096 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.096 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.096 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:23.096 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613004+0000", 2020-08-07T15:12:23.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5284000000000001e-05 2020-08-07T15:12:23.097 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.097 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.097 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.097 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613030+0000", 2020-08-07T15:12:23.097 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2400000000000001e-06 2020-08-07T15:12:23.099 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.099 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.099 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.099 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613034+0000", 2020-08-07T15:12:23.099 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81134136199999995 2020-08-07T15:12:23.100 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.100 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.100 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.100 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424375+0000", 2020-08-07T15:12:23.100 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3520000000000001e-06 2020-08-07T15:12:23.101 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.101 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.101 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.101 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424383+0000", 2020-08-07T15:12:23.101 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76064437799999995 2020-08-07T15:12:23.102 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.102 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.102 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.102 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185027+0000", 2020-08-07T15:12:23.102 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000119719 2020-08-07T15:12:23.103 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.103 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.103 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:23.103 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185147+0000", 2020-08-07T15:12:23.103 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016829400000000001 2020-08-07T15:12:23.104 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.104 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.104 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:23.104 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185315+0000", 2020-08-07T15:12:23.104 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0053821199999999998 2020-08-07T15:12:23.105 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.105 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.105 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:23.105 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190697+0000", 2020-08-07T15:12:23.105 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6899999999999996e-07 2020-08-07T15:12:23.106 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.106 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.106 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:23.106 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190698+0000", 2020-08-07T15:12:23.106 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2072000000000001e-05 2020-08-07T15:12:23.107 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.107 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.107 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:23.108 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190730+0000", 2020-08-07T15:12:23.111 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0088501440000000008 2020-08-07T15:12:23.111 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.112 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199580+0000", 2020-08-07T15:12:23.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00023199900000000001 2020-08-07T15:12:23.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.113 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.113 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199812+0000", 2020-08-07T15:12:23.113 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6300000000000005e-07 2020-08-07T15:12:23.113 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.114 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:23.114 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199813+0000", 2020-08-07T15:12:23.114 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3363999999999998e-05 2020-08-07T15:12:23.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.115 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.115 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:23.115 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199876+0000", 2020-08-07T15:12:23.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.586946441 2020-08-07T15:12:23.115 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.115 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:23.116 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.116 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.116 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.116 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72987 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:23.117 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:12:23.117 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 177.08069439900001, 2020-08-07T15:12:23.117 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869598709999999, 2020-08-07T15:12:23.117 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:23.117 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:23.129 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:23.129 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:23.129 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:23.130 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72987 2020-08-07T15:12:23.130 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.130 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:23.130 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.131 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:23.131 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:12:23.131 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:23.131 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.131 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.131 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:23.132 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:12:23.132 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1851999999999999e-05 2020-08-07T15:12:23.132 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.132 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.132 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:23.133 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613057+0000", 2020-08-07T15:12:23.133 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0429999999999999e-06 2020-08-07T15:12:23.133 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.133 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.133 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:23.167 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613060+0000", 2020-08-07T15:12:23.167 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1060000000000002e-06 2020-08-07T15:12:23.168 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.168 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.168 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:23.168 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613067+0000", 2020-08-07T15:12:23.168 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8040000000000002e-06 2020-08-07T15:12:23.169 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.169 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.169 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:23.169 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613070+0000", 2020-08-07T15:12:23.170 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6857000000000001e-05 2020-08-07T15:12:23.170 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.170 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.170 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.171 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613097+0000", 2020-08-07T15:12:23.171 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8770000000000001e-06 2020-08-07T15:12:23.171 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.171 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.171 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.171 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613102+0000", 2020-08-07T15:12:23.172 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81131066399999996 2020-08-07T15:12:23.172 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.172 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.172 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.172 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424413+0000", 2020-08-07T15:12:23.173 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4599999999999997e-06 2020-08-07T15:12:23.173 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.173 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.173 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.173 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424420+0000", 2020-08-07T15:12:23.174 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76132809700000004 2020-08-07T15:12:23.174 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.174 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.174 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.174 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185748+0000", 2020-08-07T15:12:23.175 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.4264000000000005e-05 2020-08-07T15:12:23.175 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.175 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.175 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:23.175 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185832+0000", 2020-08-07T15:12:23.176 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112803 2020-08-07T15:12:23.176 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.176 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.179 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:23.179 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185945+0000", 2020-08-07T15:12:23.179 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0089954130000000007 2020-08-07T15:12:23.179 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.180 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.180 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:23.180 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194941+0000", 2020-08-07T15:12:23.180 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.091e-06 2020-08-07T15:12:23.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.181 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.181 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:23.181 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194943+0000", 2020-08-07T15:12:23.181 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7423999999999996e-05 2020-08-07T15:12:23.181 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:23.182 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195020+0000", 2020-08-07T15:12:23.182 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045973300000000002 2020-08-07T15:12:23.182 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.183 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.183 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.183 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199618+0000", 2020-08-07T15:12:23.183 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000302009 2020-08-07T15:12:23.183 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.183 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.184 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.184 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199920+0000", 2020-08-07T15:12:23.184 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7999999999999996e-07 2020-08-07T15:12:23.184 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.184 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.185 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:23.185 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199920+0000", 2020-08-07T15:12:23.185 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5296999999999995e-05 2020-08-07T15:12:23.185 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.185 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.186 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:23.186 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199985+0000", 2020-08-07T15:12:23.186 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869598709999999 2020-08-07T15:12:23.187 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.187 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:23.187 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.187 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.187 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.188 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72988 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:23.188 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:12:23.189 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 177.08062981899999, 2020-08-07T15:12:23.189 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587111143, 2020-08-07T15:12:23.189 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:23.189 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:23.189 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:23.190 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:23.190 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:23.190 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72988 2020-08-07T15:12:23.190 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.190 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:23.191 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.191 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:23.191 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:12:23.191 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:23.191 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.192 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.192 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:23.192 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:12:23.192 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6563e-05 2020-08-07T15:12:23.192 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.192 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.193 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:23.193 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613117+0000", 2020-08-07T15:12:23.193 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7180000000000001e-06 2020-08-07T15:12:23.193 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.193 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.194 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:23.194 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613119+0000", 2020-08-07T15:12:23.194 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1789999999999999e-06 2020-08-07T15:12:23.194 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.194 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.195 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:23.195 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613124+0000", 2020-08-07T15:12:23.195 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3539999999999998e-06 2020-08-07T15:12:23.195 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.195 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.196 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:23.197 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613127+0000", 2020-08-07T15:12:23.197 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3697e-05 2020-08-07T15:12:23.197 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.197 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.197 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.198 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613151+0000", 2020-08-07T15:12:23.199 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7559999999999999e-06 2020-08-07T15:12:23.199 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.199 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.199 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.200 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613155+0000", 2020-08-07T15:12:23.200 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81128859600000003 2020-08-07T15:12:23.200 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.200 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.200 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.201 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424444+0000", 2020-08-07T15:12:23.201 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9619999999999996e-06 2020-08-07T15:12:23.201 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.201 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.201 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.202 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424451+0000", 2020-08-07T15:12:23.202 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76286712899999998 2020-08-07T15:12:23.202 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.202 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.202 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.202 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187318+0000", 2020-08-07T15:12:23.203 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011616300000000001 2020-08-07T15:12:23.203 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.203 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.203 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:23.203 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187434+0000", 2020-08-07T15:12:23.204 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019347400000000001 2020-08-07T15:12:23.204 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.204 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.204 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:23.204 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187628+0000", 2020-08-07T15:12:23.205 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074049090000000003 2020-08-07T15:12:23.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.216 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.216 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:23.216 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195033+0000", 2020-08-07T15:12:23.217 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1090000000000001e-06 2020-08-07T15:12:23.217 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.217 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.217 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:23.218 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195034+0000", 2020-08-07T15:12:23.218 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8566999999999997e-05 2020-08-07T15:12:23.218 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.218 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.218 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:23.219 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195092+0000", 2020-08-07T15:12:23.219 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004570457 2020-08-07T15:12:23.219 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.220 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.220 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.220 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199663+0000", 2020-08-07T15:12:23.221 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00046757899999999998 2020-08-07T15:12:23.221 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.221 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.221 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.222 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200130+0000", 2020-08-07T15:12:23.222 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6700000000000002e-07 2020-08-07T15:12:23.222 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.222 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.222 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:23.223 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200131+0000", 2020-08-07T15:12:23.223 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9963999999999996e-05 2020-08-07T15:12:23.223 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.223 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.223 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:23.223 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200201+0000", 2020-08-07T15:12:23.224 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587111143 2020-08-07T15:12:23.224 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.224 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:23.224 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.224 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.225 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.225 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72989 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:23.225 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:12:23.225 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 177.080573431, 2020-08-07T15:12:23.225 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5872506399999999, 2020-08-07T15:12:23.226 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:23.226 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:23.226 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:23.226 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:23.226 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:23.227 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72989 2020-08-07T15:12:23.227 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.227 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:23.227 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.227 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:23.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:12:23.228 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:23.228 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.228 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.228 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:23.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:12:23.229 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6081999999999999e-05 2020-08-07T15:12:23.229 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.229 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.229 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:23.229 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613172+0000", 2020-08-07T15:12:23.230 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6570000000000001e-06 2020-08-07T15:12:23.230 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.231 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.231 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:23.231 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613175+0000", 2020-08-07T15:12:23.231 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7740000000000001e-06 2020-08-07T15:12:23.231 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.232 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.232 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:23.232 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613181+0000", 2020-08-07T15:12:23.232 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2759999999999999e-06 2020-08-07T15:12:23.232 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.233 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.233 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:23.233 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613183+0000", 2020-08-07T15:12:23.233 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3892000000000001e-05 2020-08-07T15:12:23.233 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.234 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.234 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.234 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613207+0000", 2020-08-07T15:12:23.234 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9870000000000001e-06 2020-08-07T15:12:23.234 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.235 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.235 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.235 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613211+0000", 2020-08-07T15:12:23.235 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81126230700000002 2020-08-07T15:12:23.236 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.237 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.237 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.237 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424473+0000", 2020-08-07T15:12:23.237 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1460000000000001e-06 2020-08-07T15:12:23.237 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.238 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.238 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.238 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424480+0000", 2020-08-07T15:12:23.238 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76413073099999995 2020-08-07T15:12:23.238 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.238 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.239 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.239 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188610+0000", 2020-08-07T15:12:23.239 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011278899999999999 2020-08-07T15:12:23.239 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.239 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.240 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:23.242 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188723+0000", 2020-08-07T15:12:23.242 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000213422 2020-08-07T15:12:23.242 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.242 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.243 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:23.243 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188936+0000", 2020-08-07T15:12:23.243 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0061672929999999999 2020-08-07T15:12:23.243 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.243 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.243 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:23.244 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195104+0000", 2020-08-07T15:12:23.244 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8199999999999999e-07 2020-08-07T15:12:23.244 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.244 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.244 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:23.245 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195105+0000", 2020-08-07T15:12:23.245 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2056000000000004e-05 2020-08-07T15:12:23.245 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.245 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.245 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:23.246 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195167+0000", 2020-08-07T15:12:23.246 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0051009769999999996 2020-08-07T15:12:23.246 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.246 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.246 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.247 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200268+0000", 2020-08-07T15:12:23.247 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9428000000000001e-05 2020-08-07T15:12:23.248 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.248 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.248 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.248 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200327+0000", 2020-08-07T15:12:23.249 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9100000000000003e-07 2020-08-07T15:12:23.249 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.249 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.249 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:23.249 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200328+0000", 2020-08-07T15:12:23.250 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9250000000000003e-05 2020-08-07T15:12:23.250 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.250 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.250 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:23.250 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200397+0000", 2020-08-07T15:12:23.251 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5872506399999999 2020-08-07T15:12:23.251 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.251 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:23.251 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.260 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.260 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72990 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:23.260 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:12:23.260 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 177.080516139, 2020-08-07T15:12:23.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587335105, 2020-08-07T15:12:23.261 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:23.261 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:23.261 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:23.261 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:23.261 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:23.262 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72990 2020-08-07T15:12:23.262 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.262 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:23.262 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.262 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:23.266 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:12:23.267 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:23.267 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.267 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.267 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:23.267 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:12:23.268 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5276999999999999e-05 2020-08-07T15:12:23.268 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.268 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:23.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613229+0000", 2020-08-07T15:12:23.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7219999999999999e-06 2020-08-07T15:12:23.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:23.273 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613232+0000", 2020-08-07T15:12:23.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1170000000000003e-06 2020-08-07T15:12:23.273 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.282 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.283 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:23.283 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613237+0000", 2020-08-07T15:12:23.283 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2309999999999999e-06 2020-08-07T15:12:23.283 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.283 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.284 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:23.284 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613239+0000", 2020-08-07T15:12:23.284 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2469000000000001e-05 2020-08-07T15:12:23.284 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.284 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.285 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.285 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613261+0000", 2020-08-07T15:12:23.294 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9280000000000001e-06 2020-08-07T15:12:23.295 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.305 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.306 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.307 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613266+0000", 2020-08-07T15:12:23.307 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81123414599999999 2020-08-07T15:12:23.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.308 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.309 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424501+0000", 2020-08-07T15:12:23.309 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5300000000000002e-06 2020-08-07T15:12:23.311 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.311 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.311 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.312 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424507+0000", 2020-08-07T15:12:23.312 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76488660200000003 2020-08-07T15:12:23.318 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.319 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.319 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.319 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189394+0000", 2020-08-07T15:12:23.319 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6878000000000003e-05 2020-08-07T15:12:23.320 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.320 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.320 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:23.320 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189491+0000", 2020-08-07T15:12:23.320 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000204477 2020-08-07T15:12:23.321 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.321 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.329 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:23.329 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189695+0000", 2020-08-07T15:12:23.329 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0065180940000000003 2020-08-07T15:12:23.330 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.330 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.330 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:23.330 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196213+0000", 2020-08-07T15:12:23.330 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4920000000000001e-06 2020-08-07T15:12:23.331 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.331 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.331 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:23.331 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196215+0000", 2020-08-07T15:12:23.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9950000000000006e-05 2020-08-07T15:12:23.332 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.332 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.332 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:23.332 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196285+0000", 2020-08-07T15:12:23.332 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004163674 2020-08-07T15:12:23.333 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.333 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.333 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.340 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200448+0000", 2020-08-07T15:12:23.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6919999999999999e-05 2020-08-07T15:12:23.341 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.341 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.342 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200485+0000", 2020-08-07T15:12:23.342 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1699999999999998e-07 2020-08-07T15:12:23.342 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.343 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.343 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:23.343 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200486+0000", 2020-08-07T15:12:23.344 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3081000000000002e-05 2020-08-07T15:12:23.344 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.344 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.345 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:23.345 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200539+0000", 2020-08-07T15:12:23.345 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587335105 2020-08-07T15:12:23.345 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.345 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:23.371 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.372 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.373 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72991 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:23.373 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:12:23.373 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 177.080460614, 2020-08-07T15:12:23.374 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5876210340000001, 2020-08-07T15:12:23.374 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:23.374 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:23.375 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:23.375 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:23.375 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:23.376 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72991 2020-08-07T15:12:23.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.376 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:23.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:23.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:12:23.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:23.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:23.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:12:23.382 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5763999999999999e-05 2020-08-07T15:12:23.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.382 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.382 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:23.383 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613285+0000", 2020-08-07T15:12:23.383 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6479999999999999e-06 2020-08-07T15:12:23.383 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.384 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.384 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:23.384 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613288+0000", 2020-08-07T15:12:23.385 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1909999999999999e-06 2020-08-07T15:12:23.385 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.385 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.386 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:23.386 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613293+0000", 2020-08-07T15:12:23.387 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2560000000000001e-06 2020-08-07T15:12:23.387 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.388 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.399 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:23.400 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613295+0000", 2020-08-07T15:12:23.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2561e-05 2020-08-07T15:12:23.410 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.410 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.411 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.411 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613318+0000", 2020-08-07T15:12:23.411 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6349999999999999e-06 2020-08-07T15:12:23.412 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.412 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.413 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.413 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613321+0000", 2020-08-07T15:12:23.413 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81120592400000002 2020-08-07T15:12:23.427 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424527+0000", 2020-08-07T15:12:23.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8540000000000002e-06 2020-08-07T15:12:23.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.429 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.430 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424533+0000", 2020-08-07T15:12:23.430 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76623392800000001 2020-08-07T15:12:23.430 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.430 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.431 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.431 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190767+0000", 2020-08-07T15:12:23.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3990000000000001e-05 2020-08-07T15:12:23.431 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.432 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.432 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:23.432 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190841+0000", 2020-08-07T15:12:23.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011444599999999999 2020-08-07T15:12:23.433 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.433 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.433 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:23.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190955+0000", 2020-08-07T15:12:23.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0053397499999999999 2020-08-07T15:12:23.444 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.445 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:23.445 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196295+0000", 2020-08-07T15:12:23.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006e-06 2020-08-07T15:12:23.446 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.446 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.446 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:23.446 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196296+0000", 2020-08-07T15:12:23.446 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8940999999999997e-05 2020-08-07T15:12:23.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.447 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:23.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196355+0000", 2020-08-07T15:12:23.448 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042276700000000002 2020-08-07T15:12:23.448 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.449 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.449 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200583+0000", 2020-08-07T15:12:23.449 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00026035700000000001 2020-08-07T15:12:23.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.449 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.450 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.450 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200843+0000", 2020-08-07T15:12:23.450 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6700000000000003e-07 2020-08-07T15:12:23.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.451 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:23.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200844+0000", 2020-08-07T15:12:23.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6545999999999999e-05 2020-08-07T15:12:23.451 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.451 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:23.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200880+0000", 2020-08-07T15:12:23.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5876210340000001 2020-08-07T15:12:23.452 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.452 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:23.453 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.470 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.471 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72992 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:23.471 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:12:23.471 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 177.079373135, 2020-08-07T15:12:23.471 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5866241210000001, 2020-08-07T15:12:23.472 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:23.472 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:23.472 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:23.472 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:23.472 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:23.473 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72992 2020-08-07T15:12:23.473 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.473 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:23.473 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.473 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:23.474 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:12:23.474 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:23.474 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.474 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.474 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:23.475 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:12:23.475 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7543e-05 2020-08-07T15:12:23.475 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.475 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.475 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:23.476 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614374+0000", 2020-08-07T15:12:23.476 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9740000000000004e-06 2020-08-07T15:12:23.476 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.476 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.476 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:23.476 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614378+0000", 2020-08-07T15:12:23.477 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2660000000000007e-06 2020-08-07T15:12:23.477 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.477 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.477 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:23.480 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614387+0000", 2020-08-07T15:12:23.480 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2950000000000001e-06 2020-08-07T15:12:23.481 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.481 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.481 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:23.481 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614393+0000", 2020-08-07T15:12:23.481 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7279000000000001e-05 2020-08-07T15:12:23.482 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.482 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.482 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.482 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614420+0000", 2020-08-07T15:12:23.482 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9040000000000002e-06 2020-08-07T15:12:23.483 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.483 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.483 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.483 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614424+0000", 2020-08-07T15:12:23.483 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81012815699999996 2020-08-07T15:12:23.484 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.484 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.484 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.484 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424552+0000", 2020-08-07T15:12:23.484 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2879999999999998e-06 2020-08-07T15:12:23.485 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.485 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.485 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.485 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424558+0000", 2020-08-07T15:12:23.485 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76678665400000001 2020-08-07T15:12:23.486 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.486 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.486 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.486 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191345+0000", 2020-08-07T15:12:23.486 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010819600000000001 2020-08-07T15:12:23.486 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.487 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.487 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:23.487 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191453+0000", 2020-08-07T15:12:23.487 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000170829 2020-08-07T15:12:23.487 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:23.488 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191624+0000", 2020-08-07T15:12:23.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0047386679999999997 2020-08-07T15:12:23.488 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.489 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.489 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:23.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196363+0000", 2020-08-07T15:12:23.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6000000000000003e-07 2020-08-07T15:12:23.490 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.490 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.490 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:23.491 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196363+0000", 2020-08-07T15:12:23.491 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3421000000000002e-05 2020-08-07T15:12:23.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.491 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.491 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:23.492 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196417+0000", 2020-08-07T15:12:23.492 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042225539999999999 2020-08-07T15:12:23.492 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.492 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.492 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.493 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200639+0000", 2020-08-07T15:12:23.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00028031399999999999 2020-08-07T15:12:23.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.493 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.494 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200920+0000", 2020-08-07T15:12:23.494 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0299999999999999e-07 2020-08-07T15:12:23.494 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.494 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:23.495 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200920+0000", 2020-08-07T15:12:23.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0516000000000002e-05 2020-08-07T15:12:23.495 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.495 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.495 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:23.496 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200971+0000", 2020-08-07T15:12:23.496 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5866241210000001 2020-08-07T15:12:23.496 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.496 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:23.496 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.497 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.497 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.497 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73017 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:23.497 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:12:23.497 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 177.023864882, 2020-08-07T15:12:23.498 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.531181608, 2020-08-07T15:12:23.498 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:23.498 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:23.498 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:23.499 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:23.499 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:23.499 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73017 2020-08-07T15:12:23.499 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.499 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:23.500 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.500 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:23.503 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:12:23.504 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:23.504 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.504 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.504 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:23.504 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:12:23.505 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3018e-05 2020-08-07T15:12:23.505 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.505 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.505 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:23.505 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669898+0000", 2020-08-07T15:12:23.506 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5340000000000002e-06 2020-08-07T15:12:23.506 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.506 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.506 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:23.506 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669905+0000", 2020-08-07T15:12:23.507 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3307e-05 2020-08-07T15:12:23.507 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.507 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.507 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:23.507 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669919+0000", 2020-08-07T15:12:23.508 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7729999999999999e-06 2020-08-07T15:12:23.508 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.508 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.508 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:23.508 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669927+0000", 2020-08-07T15:12:23.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0361000000000001e-05 2020-08-07T15:12:23.509 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.509 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.509 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669977+0000", 2020-08-07T15:12:23.510 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0049999999999993e-06 2020-08-07T15:12:23.510 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.510 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.510 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669986+0000", 2020-08-07T15:12:23.510 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75459235199999997 2020-08-07T15:12:23.511 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.511 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424578+0000", 2020-08-07T15:12:23.511 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.925e-06 2020-08-07T15:12:23.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.512 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.532 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.532 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424584+0000", 2020-08-07T15:12:23.533 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76741703900000002 2020-08-07T15:12:23.533 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.555 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.555 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.555 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192001+0000", 2020-08-07T15:12:23.556 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000181196 2020-08-07T15:12:23.556 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:23.557 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192182+0000", 2020-08-07T15:12:23.557 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00021149299999999999 2020-08-07T15:12:23.557 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.557 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.557 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:23.558 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192394+0000", 2020-08-07T15:12:23.558 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004031242 2020-08-07T15:12:23.558 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.558 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.558 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:23.559 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196425+0000", 2020-08-07T15:12:23.559 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7800000000000001e-07 2020-08-07T15:12:23.559 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.559 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.559 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:23.559 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196426+0000", 2020-08-07T15:12:23.560 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5244999999999997e-05 2020-08-07T15:12:23.560 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.560 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.560 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:23.560 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196481+0000", 2020-08-07T15:12:23.561 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041989360000000003 2020-08-07T15:12:23.561 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.561 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.561 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.561 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200680+0000", 2020-08-07T15:12:23.584 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00032415299999999998 2020-08-07T15:12:23.585 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.585 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.585 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.585 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201004+0000", 2020-08-07T15:12:23.585 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9299999999999999e-07 2020-08-07T15:12:23.586 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.586 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.586 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:23.586 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201005+0000", 2020-08-07T15:12:23.586 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1857999999999999e-05 2020-08-07T15:12:23.587 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.587 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.587 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:23.587 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201037+0000", 2020-08-07T15:12:23.587 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.531181608 2020-08-07T15:12:23.587 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.588 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:23.588 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.588 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.588 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73018 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:23.589 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:12:23.589 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 177.02289707700001, 2020-08-07T15:12:23.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.530301616, 2020-08-07T15:12:23.589 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:23.589 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:23.590 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:23.590 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:23.590 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:23.590 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73018 2020-08-07T15:12:23.590 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.591 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:23.591 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.591 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:23.591 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:12:23.591 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:23.592 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.592 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.592 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:23.592 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:12:23.592 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9747000000000002e-05 2020-08-07T15:12:23.593 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.593 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.593 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:23.593 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670852+0000", 2020-08-07T15:12:23.593 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9430000000000002e-06 2020-08-07T15:12:23.595 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.595 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.595 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:23.595 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670856+0000", 2020-08-07T15:12:23.596 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0649999999999999e-06 2020-08-07T15:12:23.596 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.596 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.596 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:23.596 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670864+0000", 2020-08-07T15:12:23.597 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.924e-06 2020-08-07T15:12:23.597 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.597 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.597 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:23.597 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670868+0000", 2020-08-07T15:12:23.598 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6205000000000003e-05 2020-08-07T15:12:23.598 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.598 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.598 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.598 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670905+0000", 2020-08-07T15:12:23.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3289999999999996e-06 2020-08-07T15:12:23.599 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.599 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.599 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.599 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670912+0000", 2020-08-07T15:12:23.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75370200200000004 2020-08-07T15:12:23.600 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.600 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.600 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.600 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424614+0000", 2020-08-07T15:12:23.601 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5150000000000003e-06 2020-08-07T15:12:23.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.601 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.602 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424620+0000", 2020-08-07T15:12:23.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.768200153 2020-08-07T15:12:23.602 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.602 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.602 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.603 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192821+0000", 2020-08-07T15:12:23.603 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000111764 2020-08-07T15:12:23.603 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.603 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:23.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192932+0000", 2020-08-07T15:12:23.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017542400000000001 2020-08-07T15:12:23.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:23.605 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193108+0000", 2020-08-07T15:12:23.606 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004100851 2020-08-07T15:12:23.606 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.606 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.606 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:23.607 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197209+0000", 2020-08-07T15:12:23.607 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0100000000000001e-06 2020-08-07T15:12:23.607 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.607 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.607 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:23.607 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197210+0000", 2020-08-07T15:12:23.608 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9672999999999995e-05 2020-08-07T15:12:23.608 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.608 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.608 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:23.608 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197279+0000", 2020-08-07T15:12:23.609 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034275109999999998 2020-08-07T15:12:23.609 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.609 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.610 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.610 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200707+0000", 2020-08-07T15:12:23.610 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00036853100000000001 2020-08-07T15:12:23.611 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.611 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.611 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.612 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201075+0000", 2020-08-07T15:12:23.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8599999999999998e-07 2020-08-07T15:12:23.612 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.613 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:23.613 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201076+0000", 2020-08-07T15:12:23.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8483000000000003e-05 2020-08-07T15:12:23.614 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.614 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.615 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:23.617 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201124+0000", 2020-08-07T15:12:23.618 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.530301616 2020-08-07T15:12:23.618 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.618 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:23.618 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.619 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.619 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.620 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73019 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:23.620 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:12:23.620 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 177.022249426, 2020-08-07T15:12:23.620 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.529718224, 2020-08-07T15:12:23.620 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:23.621 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:23.621 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:23.621 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:23.621 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:23.621 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73019 2020-08-07T15:12:23.622 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.622 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:23.622 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.622 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:23.622 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:12:23.623 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:23.623 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.632 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.633 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:23.633 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:12:23.634 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3309000000000002e-05 2020-08-07T15:12:23.634 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.634 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.635 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:23.635 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671504+0000", 2020-08-07T15:12:23.635 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6619999999999998e-06 2020-08-07T15:12:23.635 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.636 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.636 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:23.636 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671507+0000", 2020-08-07T15:12:23.636 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2279999999999998e-06 2020-08-07T15:12:23.637 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.647 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.647 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:23.647 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671516+0000", 2020-08-07T15:12:23.648 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4860000000000002e-06 2020-08-07T15:12:23.666 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.666 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.666 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:23.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671519+0000", 2020-08-07T15:12:23.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0564000000000001e-05 2020-08-07T15:12:23.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.668 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671550+0000", 2020-08-07T15:12:23.668 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1029999999999998e-06 2020-08-07T15:12:23.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671556+0000", 2020-08-07T15:12:23.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75308759199999997 2020-08-07T15:12:23.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.694 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.694 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.695 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424643+0000", 2020-08-07T15:12:23.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9179999999999997e-06 2020-08-07T15:12:23.695 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.695 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.696 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424650+0000", 2020-08-07T15:12:23.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76890373599999995 2020-08-07T15:12:23.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193554+0000", 2020-08-07T15:12:23.697 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012345600000000001 2020-08-07T15:12:23.697 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.698 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.698 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:23.698 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193677+0000", 2020-08-07T15:12:23.699 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013528300000000001 2020-08-07T15:12:23.699 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.699 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.699 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:23.699 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193813+0000", 2020-08-07T15:12:23.700 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034777710000000002 2020-08-07T15:12:23.700 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.700 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:23.701 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197291+0000", 2020-08-07T15:12:23.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8000000000000004e-07 2020-08-07T15:12:23.701 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.702 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:23.702 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197291+0000", 2020-08-07T15:12:23.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9176999999999999e-05 2020-08-07T15:12:23.702 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.703 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:23.703 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197351+0000", 2020-08-07T15:12:23.703 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033931009999999999 2020-08-07T15:12:23.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.705 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.705 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.705 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200744+0000", 2020-08-07T15:12:23.706 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00041308000000000002 2020-08-07T15:12:23.706 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.706 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.706 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.707 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201157+0000", 2020-08-07T15:12:23.707 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8999999999999998e-07 2020-08-07T15:12:23.707 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.707 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:23.708 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201157+0000", 2020-08-07T15:12:23.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1588000000000003e-05 2020-08-07T15:12:23.708 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.708 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:23.709 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201189+0000", 2020-08-07T15:12:23.709 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.529718224 2020-08-07T15:12:23.709 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.709 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:23.710 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.710 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.710 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.710 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73020 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:23.710 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:12:23.711 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 176.95391156599999, 2020-08-07T15:12:23.711 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.461468373, 2020-08-07T15:12:23.711 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:23.711 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:23.711 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:23.711 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:23.712 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:23.712 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73020 2020-08-07T15:12:23.712 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.712 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:23.712 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.713 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:23.713 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:12:23.714 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:23.717 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.717 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.717 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:23.717 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:12:23.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.777e-05 2020-08-07T15:12:23.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.718 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.718 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:23.718 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739836+0000", 2020-08-07T15:12:23.728 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0180000000000003e-06 2020-08-07T15:12:23.729 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.729 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.729 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:23.729 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739840+0000", 2020-08-07T15:12:23.730 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6930000000000002e-06 2020-08-07T15:12:23.730 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.730 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.730 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:23.730 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739850+0000", 2020-08-07T15:12:23.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3619999999999999e-06 2020-08-07T15:12:23.735 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.735 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.736 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:23.736 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739854+0000", 2020-08-07T15:12:23.736 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7432999999999999e-05 2020-08-07T15:12:23.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.736 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.737 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.737 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739892+0000", 2020-08-07T15:12:23.737 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1149999999999996e-06 2020-08-07T15:12:23.737 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.738 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.738 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.738 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739899+0000", 2020-08-07T15:12:23.738 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68477883299999998 2020-08-07T15:12:23.738 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.740 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.740 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.741 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424677+0000", 2020-08-07T15:12:23.741 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7710000000000001e-06 2020-08-07T15:12:23.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.741 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.741 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.742 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424684+0000", 2020-08-07T15:12:23.743 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76949970700000003 2020-08-07T15:12:23.743 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.743 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.743 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194184+0000", 2020-08-07T15:12:23.744 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3957999999999997e-05 2020-08-07T15:12:23.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:23.744 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194278+0000", 2020-08-07T15:12:23.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017489599999999999 2020-08-07T15:12:23.751 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.751 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.752 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:23.752 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194453+0000", 2020-08-07T15:12:23.752 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002908908 2020-08-07T15:12:23.753 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.753 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.753 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:23.754 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197362+0000", 2020-08-07T15:12:23.754 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7299999999999995e-07 2020-08-07T15:12:23.754 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.754 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.754 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:23.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197362+0000", 2020-08-07T15:12:23.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6848000000000002e-05 2020-08-07T15:12:23.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:23.756 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197419+0000", 2020-08-07T15:12:23.757 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033597079999999999 2020-08-07T15:12:23.757 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.757 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.757 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200779+0000", 2020-08-07T15:12:23.758 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00044797899999999999 2020-08-07T15:12:23.758 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.758 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.758 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.758 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201227+0000", 2020-08-07T15:12:23.759 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7100000000000002e-07 2020-08-07T15:12:23.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.759 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:23.759 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201227+0000", 2020-08-07T15:12:23.760 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9230000000000001e-05 2020-08-07T15:12:23.760 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.760 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.768 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:23.768 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201277+0000", 2020-08-07T15:12:23.769 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.461468373 2020-08-07T15:12:23.770 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.770 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:23.770 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.770 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.770 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.771 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73021 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:23.786 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:12:23.787 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 176.89087795899999, 2020-08-07T15:12:23.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.398505079, 2020-08-07T15:12:23.787 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:23.787 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:23.787 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:23.788 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:23.792 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:23.793 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73021 2020-08-07T15:12:23.793 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.794 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:23.794 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.794 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:23.796 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:12:23.796 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:23.797 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.798 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.798 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:23.798 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:12:23.798 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7429000000000001e-05 2020-08-07T15:12:23.798 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:23.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802869+0000", 2020-08-07T15:12:23.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7840000000000001e-06 2020-08-07T15:12:23.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:23.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802873+0000", 2020-08-07T15:12:23.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0119999999999994e-06 2020-08-07T15:12:23.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:23.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802882+0000", 2020-08-07T15:12:23.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.04e-06 2020-08-07T15:12:23.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:23.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802887+0000", 2020-08-07T15:12:23.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7076000000000002e-05 2020-08-07T15:12:23.806 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802924+0000", 2020-08-07T15:12:23.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9280000000000004e-06 2020-08-07T15:12:23.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802931+0000", 2020-08-07T15:12:23.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.62178051000000001 2020-08-07T15:12:23.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.808 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424712+0000", 2020-08-07T15:12:23.809 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4749999999999996e-06 2020-08-07T15:12:23.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.809 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.809 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424719+0000", 2020-08-07T15:12:23.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77047786799999995 2020-08-07T15:12:23.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195197+0000", 2020-08-07T15:12:23.813 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000101909 2020-08-07T15:12:23.814 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.814 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.814 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:23.815 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195299+0000", 2020-08-07T15:12:23.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000248247 2020-08-07T15:12:23.816 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:23.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195547+0000", 2020-08-07T15:12:23.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033458699999999999 2020-08-07T15:12:23.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.817 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:23.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198893+0000", 2020-08-07T15:12:23.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0419999999999999e-06 2020-08-07T15:12:23.818 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.818 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.818 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:23.818 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198894+0000", 2020-08-07T15:12:23.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9354999999999997e-05 2020-08-07T15:12:23.819 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:23.819 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198953+0000", 2020-08-07T15:12:23.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001861096 2020-08-07T15:12:23.819 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200815+0000", 2020-08-07T15:12:23.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00049697299999999995 2020-08-07T15:12:23.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.822 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.822 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201311+0000", 2020-08-07T15:12:23.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7099999999999998e-07 2020-08-07T15:12:23.831 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.831 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.832 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:23.832 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201312+0000", 2020-08-07T15:12:23.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5194000000000002e-05 2020-08-07T15:12:23.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:23.833 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201347+0000", 2020-08-07T15:12:23.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.398505079 2020-08-07T15:12:23.835 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.836 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:23.836 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.836 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73022 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:23.837 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:12:23.837 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 176.809705904, 2020-08-07T15:12:23.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3238320669999999, 2020-08-07T15:12:23.837 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:23.838 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:23.838 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:23.838 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:23.838 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:23.838 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73022 2020-08-07T15:12:23.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.839 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:23.847 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.847 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:23.848 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:12:23.848 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:23.848 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.848 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.849 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:23.855 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:12:23.855 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6769999999999999e-05 2020-08-07T15:12:23.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:23.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884041+0000", 2020-08-07T15:12:23.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.703e-06 2020-08-07T15:12:23.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.858 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:23.858 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884044+0000", 2020-08-07T15:12:23.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5949999999999999e-06 2020-08-07T15:12:23.858 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.859 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.859 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:23.859 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884053+0000", 2020-08-07T15:12:23.859 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000003e-06 2020-08-07T15:12:23.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.860 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:23.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884058+0000", 2020-08-07T15:12:23.868 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7281999999999999e-05 2020-08-07T15:12:23.869 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.869 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.869 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.869 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884105+0000", 2020-08-07T15:12:23.870 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.044e-05 2020-08-07T15:12:23.870 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.870 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.870 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.870 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884116+0000", 2020-08-07T15:12:23.874 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.54062884300000003 2020-08-07T15:12:23.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.875 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424745+0000", 2020-08-07T15:12:23.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3149999999999998e-06 2020-08-07T15:12:23.876 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.876 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424754+0000", 2020-08-07T15:12:23.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77175732200000002 2020-08-07T15:12:23.877 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.877 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.877 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196511+0000", 2020-08-07T15:12:23.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010110799999999999 2020-08-07T15:12:23.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:23.885 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196612+0000", 2020-08-07T15:12:23.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017543100000000001 2020-08-07T15:12:23.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:23.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196788+0000", 2020-08-07T15:12:23.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032134569999999999 2020-08-07T15:12:23.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.890 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:23.890 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200001+0000", 2020-08-07T15:12:23.890 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.071e-06 2020-08-07T15:12:23.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.896 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:23.898 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200002+0000", 2020-08-07T15:12:23.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2527999999999995e-05 2020-08-07T15:12:23.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.906 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.906 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:23.906 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200065+0000", 2020-08-07T15:12:23.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001327433 2020-08-07T15:12:23.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201392+0000", 2020-08-07T15:12:23.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063854030000000004 2020-08-07T15:12:23.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.908 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:23.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207778+0000", 2020-08-07T15:12:23.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.285e-06 2020-08-07T15:12:23.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:23.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207779+0000", 2020-08-07T15:12:23.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7070999999999995e-05 2020-08-07T15:12:23.918 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.918 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:23.919 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207846+0000", 2020-08-07T15:12:23.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3238320669999999 2020-08-07T15:12:23.919 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.919 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:23.926 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:23.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.927 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73023 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:23.927 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:12:23.928 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 176.69654426299999, 2020-08-07T15:12:23.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2113701729999999, 2020-08-07T15:12:23.928 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:23.928 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:23.928 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:23.929 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:23.929 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:23.929 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73023 2020-08-07T15:12:23.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.942 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:23.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.942 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:23.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:12:23.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:23.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:23.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:12:23.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4105e-05 2020-08-07T15:12:23.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.944 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:23.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997210+0000", 2020-08-07T15:12:23.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1089999999999997e-06 2020-08-07T15:12:23.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:23.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997215+0000", 2020-08-07T15:12:23.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7040000000000006e-06 2020-08-07T15:12:23.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:23.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997224+0000", 2020-08-07T15:12:23.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4439999999999999e-06 2020-08-07T15:12:23.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:23.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997230+0000", 2020-08-07T15:12:23.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5977999999999999e-05 2020-08-07T15:12:23.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.978 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.978 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997276+0000", 2020-08-07T15:12:23.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2660000000000007e-06 2020-08-07T15:12:23.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.993 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.993 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997285+0000", 2020-08-07T15:12:23.993 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.42749189599999998 2020-08-07T15:12:23.993 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.994 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.994 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.994 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424777+0000", 2020-08-07T15:12:23.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8940000000000004e-06 2020-08-07T15:12:23.994 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.995 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.995 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:23.995 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424784+0000", 2020-08-07T15:12:23.995 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77268199699999995 2020-08-07T15:12:23.995 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.996 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.996 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:23.996 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197466+0000", 2020-08-07T15:12:23.996 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011277699999999999 2020-08-07T15:12:23.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:23.997 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:23.997 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:23.997 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197579+0000", 2020-08-07T15:12:23.997 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018970599999999999 2020-08-07T15:12:23.997 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:24.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197768+0000", 2020-08-07T15:12:24.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043168529999999998 2020-08-07T15:12:24.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:24.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202085+0000", 2020-08-07T15:12:24.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.186e-06 2020-08-07T15:12:24.009 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.009 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.009 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:24.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202086+0000", 2020-08-07T15:12:24.010 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8975999999999999e-05 2020-08-07T15:12:24.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.010 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:24.010 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202145+0000", 2020-08-07T15:12:24.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00035331499999999999 2020-08-07T15:12:24.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.011 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:24.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202499+0000", 2020-08-07T15:12:24.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0059703159999999998 2020-08-07T15:12:24.012 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.018 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.018 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:24.018 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208469+0000", 2020-08-07T15:12:24.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7400000000000002e-07 2020-08-07T15:12:24.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.019 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.019 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:24.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208470+0000", 2020-08-07T15:12:24.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5877000000000006e-05 2020-08-07T15:12:24.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.020 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.020 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:24.020 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208546+0000", 2020-08-07T15:12:24.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2113701729999999 2020-08-07T15:12:24.021 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:24.021 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:24.021 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:24.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.022 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103336 5.a 5:5e0ebd9f:::benchmark_data_smithi074_53697_object12916:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:12:24.022 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.550524+0000", 2020-08-07T15:12:24.022 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 51.143195757000001, 2020-08-07T15:12:24.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.71324962300000005, 2020-08-07T15:12:24.023 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:24.023 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:24.024 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:24.024 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:24.024 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:24.024 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103336 2020-08-07T15:12:24.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.025 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:24.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:24.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.550524+0000", 2020-08-07T15:12:24.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:24.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:24.026 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.550524+0000", 2020-08-07T15:12:24.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4780000000000002e-05 2020-08-07T15:12:24.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:24.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.550569+0000", 2020-08-07T15:12:24.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9410000000000004e-06 2020-08-07T15:12:24.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:24.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.550575+0000", 2020-08-07T15:12:24.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0842999999999999e-05 2020-08-07T15:12:24.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:24.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.550586+0000", 2020-08-07T15:12:24.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6670000000000001e-06 2020-08-07T15:12:24.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:24.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.550591+0000", 2020-08-07T15:12:24.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010161409999999999 2020-08-07T15:12:24.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:24.033 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.551607+0000", 2020-08-07T15:12:24.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9899e-05 2020-08-07T15:12:24.038 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.058 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.058 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:24.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.551707+0000", 2020-08-07T15:12:24.059 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000122602 2020-08-07T15:12:24.059 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.059 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.059 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4", 2020-08-07T15:12:24.060 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.551830+0000", 2020-08-07T15:12:24.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011245630000000001 2020-08-07T15:12:24.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:24.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.552954+0000", 2020-08-07T15:12:24.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9400000000000004e-07 2020-08-07T15:12:24.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.061 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:24.082 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.552955+0000", 2020-08-07T15:12:24.082 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2538000000000003e-05 2020-08-07T15:12:24.082 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.083 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:24.083 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.553018+0000", 2020-08-07T15:12:24.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034281989999999998 2020-08-07T15:12:24.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.083 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:24.084 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.556446+0000", 2020-08-07T15:12:24.084 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.707284417 2020-08-07T15:12:24.084 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.084 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:24.085 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.263731+0000", 2020-08-07T15:12:24.085 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3399999999999998e-07 2020-08-07T15:12:24.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.092 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:24.093 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.263731+0000", 2020-08-07T15:12:24.093 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2305000000000002e-05 2020-08-07T15:12:24.093 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.093 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.093 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:24.094 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.263774+0000", 2020-08-07T15:12:24.094 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.71324962300000005 2020-08-07T15:12:24.094 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:24.094 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:24.095 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:24.095 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:24.095 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:24.095 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:24.114 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:12:24.115 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:12:24.116 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:12:24.116 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:12:24.116 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:12:24.116 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:12:24.116 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:12:24.117 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:12:24.117 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:12:24.117 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:12:24.117 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:12:24.117 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:12:24.118 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:12:24.118 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:12:24.118 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 11 16 16741 16725 11.8769 12.0156 0.00757718 0.0104821 2020-08-07T15:12:24.119 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:12:24.119 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:24.095+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:24.201 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:24.197+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:24.270 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 12 16 18296 18280 11.8994 12.1484 0.0156115 0.0104989 2020-08-07T15:12:24.312 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:24.305+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:24.313 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:24.313 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:12:24.313 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.313 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:18370 6.b 6:d04ffc29:::benchmark_data_smithi074_63843_object2296:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e240)", 2020-08-07T15:12:24.313 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.305480+0000", 2020-08-07T15:12:24.314 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0022161020000000002, 2020-08-07T15:12:24.314 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0022432849999999998, 2020-08-07T15:12:24.314 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:24.314 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:12:24.314 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:24.315 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:24.315 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:24.315 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18370 2020-08-07T15:12:24.321 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.321 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:24.321 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.322 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:24.322 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.305480+0000", 2020-08-07T15:12:24.322 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:24.322 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.323 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.323 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:24.323 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.305480+0000", 2020-08-07T15:12:24.324 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6019000000000001e-05 2020-08-07T15:12:24.324 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.324 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.324 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:24.324 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.305526+0000", 2020-08-07T15:12:24.325 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0619999999999997e-06 2020-08-07T15:12:24.325 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.332 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.332 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:24.332 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.305532+0000", 2020-08-07T15:12:24.333 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4439e-05 2020-08-07T15:12:24.333 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.333 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.338 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:24.338 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.305547+0000", 2020-08-07T15:12:24.338 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1859999999999997e-06 2020-08-07T15:12:24.339 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.340 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:24.342 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.305553+0000", 2020-08-07T15:12:24.342 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8736999999999999e-05 2020-08-07T15:12:24.343 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.343 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.343 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:24.350 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.305592+0000", 2020-08-07T15:12:24.350 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010577 2020-08-07T15:12:24.350 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.351 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.353 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:24.353 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.305697+0000", 2020-08-07T15:12:24.353 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000202165 2020-08-07T15:12:24.353 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.354 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.354 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:12:24.355 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.305899+0000", 2020-08-07T15:12:24.355 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00041937799999999998 2020-08-07T15:12:24.355 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:24.355 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:24.356 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:24.356 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.356 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.356 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:18371 6.b 6:d04ffc29:::benchmark_data_smithi074_63843_object2296:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e240)", 2020-08-07T15:12:24.357 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.305592+0000", 2020-08-07T15:12:24.357 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0021044449999999999, 2020-08-07T15:12:24.357 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002206286, 2020-08-07T15:12:24.358 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:24.358 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:12:24.358 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:24.358 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:24.359 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:24.359 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18371 2020-08-07T15:12:24.359 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.359 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:24.360 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.360 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:24.363 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.305592+0000", 2020-08-07T15:12:24.364 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:24.364 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:24.365 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.305592+0000", 2020-08-07T15:12:24.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9259999999999999e-05 2020-08-07T15:12:24.365 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.370 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.370 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:24.370 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.305641+0000", 2020-08-07T15:12:24.371 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8449999999999999e-06 2020-08-07T15:12:24.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.371 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.371 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:24.372 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.305647+0000", 2020-08-07T15:12:24.372 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5563e-05 2020-08-07T15:12:24.374 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.374 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.374 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:24.375 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.305662+0000", 2020-08-07T15:12:24.375 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2480000000000004e-06 2020-08-07T15:12:24.375 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.376 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:24.376 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.305668+0000", 2020-08-07T15:12:24.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00065005900000000003 2020-08-07T15:12:24.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:24.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.306318+0000", 2020-08-07T15:12:24.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8533e-05 2020-08-07T15:12:24.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:24.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.306416+0000", 2020-08-07T15:12:24.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000188789 2020-08-07T15:12:24.379 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:12:24.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.306605+0000", 2020-08-07T15:12:24.380 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001013297 2020-08-07T15:12:24.380 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:24.380 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:24.380 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:24.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.381 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:18372 6.b 6:d04ffc29:::benchmark_data_smithi074_63843_object2296:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e240)", 2020-08-07T15:12:24.381 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.305704+0000", 2020-08-07T15:12:24.394 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0019917469999999999, 2020-08-07T15:12:24.394 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0021750760000000002, 2020-08-07T15:12:24.394 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:24.395 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:12:24.395 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:24.395 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:24.395 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:24.396 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18372 2020-08-07T15:12:24.396 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.396 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:24.396 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.397 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:24.397 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.305704+0000", 2020-08-07T15:12:24.397 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:24.397 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.398 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.398 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:24.398 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.305704+0000", 2020-08-07T15:12:24.398 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.846e-05 2020-08-07T15:12:24.398 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.399 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:24.399 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.305753+0000", 2020-08-07T15:12:24.399 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.64e-06 2020-08-07T15:12:24.400 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.400 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:24.400 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.305760+0000", 2020-08-07T15:12:24.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1276e-05 2020-08-07T15:12:24.401 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.401 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.408 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:24.409 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.305771+0000", 2020-08-07T15:12:24.409 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8690000000000003e-06 2020-08-07T15:12:24.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.410 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.410 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:24.410 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.305776+0000", 2020-08-07T15:12:24.410 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001426914 2020-08-07T15:12:24.411 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.411 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.411 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:24.411 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.307203+0000", 2020-08-07T15:12:24.411 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5210999999999996e-05 2020-08-07T15:12:24.412 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.425 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:24.426 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.307268+0000", 2020-08-07T15:12:24.426 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000110203 2020-08-07T15:12:24.426 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.426 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.426 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:12:24.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.307378+0000", 2020-08-07T15:12:24.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001673573 2020-08-07T15:12:24.427 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:24.427 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:24.428 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:24.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.428 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:18369 6.b e240/230)", 2020-08-07T15:12:24.429 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.307175+0000", 2020-08-07T15:12:24.429 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.00052153799999999999, 2020-08-07T15:12:24.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00078317100000000004, 2020-08-07T15:12:24.429 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:24.436 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:12:24.454 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:24.473 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.474 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:24.474 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.307175+0000", 2020-08-07T15:12:24.474 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:24.474 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.474 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.475 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:24.475 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.307175+0000", 2020-08-07T15:12:24.475 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000114147 2020-08-07T15:12:24.475 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.505 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.505 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:24.505 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.307289+0000", 2020-08-07T15:12:24.506 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1929999999999998e-06 2020-08-07T15:12:24.506 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.506 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.506 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:24.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.307295+0000", 2020-08-07T15:12:24.513 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8093000000000003e-05 2020-08-07T15:12:24.514 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.514 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:24.514 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.307333+0000", 2020-08-07T15:12:24.515 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3540000000000002e-06 2020-08-07T15:12:24.515 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.515 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.515 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:24.515 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.307340+0000", 2020-08-07T15:12:24.516 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000165787 2020-08-07T15:12:24.516 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:24.516 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:24.516 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:24.516 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.516 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.517 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:18343 6.c 6:37571fdd:::benchmark_data_smithi074_63843_object2292:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e240)", 2020-08-07T15:12:24.517 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.294334+0000", 2020-08-07T15:12:24.517 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.013362498, 2020-08-07T15:12:24.517 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.013687955, 2020-08-07T15:12:24.517 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:24.518 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:24.518 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:24.518 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:24.518 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:24.518 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18343 2020-08-07T15:12:24.519 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.519 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:24.519 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.519 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:24.519 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.294334+0000", 2020-08-07T15:12:24.520 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:24.520 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.520 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:24.520 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.294334+0000", 2020-08-07T15:12:24.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8721000000000003e-05 2020-08-07T15:12:24.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.521 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.521 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:24.521 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.294382+0000", 2020-08-07T15:12:24.534 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4890000000000003e-06 2020-08-07T15:12:24.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:24.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.294388+0000", 2020-08-07T15:12:24.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0798999999999999e-05 2020-08-07T15:12:24.537 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.537 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.537 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:24.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.294399+0000", 2020-08-07T15:12:24.538 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9420000000000003e-06 2020-08-07T15:12:24.538 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.538 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.538 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:24.539 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.294404+0000", 2020-08-07T15:12:24.539 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001324237 2020-08-07T15:12:24.539 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.540 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.540 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:24.540 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.295728+0000", 2020-08-07T15:12:24.540 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9450999999999994e-05 2020-08-07T15:12:24.541 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.541 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:24.541 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.295827+0000", 2020-08-07T15:12:24.542 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000174988 2020-08-07T15:12:24.542 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.542 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.542 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:12:24.543 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.296002+0000", 2020-08-07T15:12:24.543 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019800590000000002 2020-08-07T15:12:24.543 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.543 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.544 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:24.544 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.297982+0000", 2020-08-07T15:12:24.544 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1900000000000001e-07 2020-08-07T15:12:24.545 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.545 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:24.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.297983+0000", 2020-08-07T15:12:24.546 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3439000000000003e-05 2020-08-07T15:12:24.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.546 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.546 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:24.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.298017+0000", 2020-08-07T15:12:24.547 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0050511710000000001 2020-08-07T15:12:24.547 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.548 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:24.548 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.303068+0000", 2020-08-07T15:12:24.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0087341150000000006 2020-08-07T15:12:24.548 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:24.549 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:24.549 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:24.549 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.550 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.550 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:18344 6.c 6:37571fdd:::benchmark_data_smithi074_63843_object2292:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e240)", 2020-08-07T15:12:24.550 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.294438+0000", 2020-08-07T15:12:24.550 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.013258236, 2020-08-07T15:12:24.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.013701036, 2020-08-07T15:12:24.551 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:24.551 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:24.551 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:24.552 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:24.552 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:24.552 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18344 2020-08-07T15:12:24.552 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.553 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:24.553 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.553 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:24.553 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.294438+0000", 2020-08-07T15:12:24.554 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:24.554 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.554 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.555 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:24.555 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.294438+0000", 2020-08-07T15:12:24.555 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0096999999999998e-05 2020-08-07T15:12:24.555 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:24.556 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.294488+0000", 2020-08-07T15:12:24.556 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3179999999999997e-06 2020-08-07T15:12:24.557 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.570 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.570 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:24.570 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.294493+0000", 2020-08-07T15:12:24.571 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0522e-05 2020-08-07T15:12:24.571 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.571 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.571 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:24.572 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.294504+0000", 2020-08-07T15:12:24.572 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3019999999999996e-06 2020-08-07T15:12:24.572 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.572 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.573 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:24.573 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.294508+0000", 2020-08-07T15:12:24.573 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023805200000000001 2020-08-07T15:12:24.573 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.574 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.574 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:24.574 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.296889+0000", 2020-08-07T15:12:24.575 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000105641 2020-08-07T15:12:24.575 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.575 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:24.576 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.296994+0000", 2020-08-07T15:12:24.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018462 2020-08-07T15:12:24.576 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.576 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.577 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:12:24.577 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.297179+0000", 2020-08-07T15:12:24.577 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0021789159999999999 2020-08-07T15:12:24.590 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.590 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.611 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:24.611 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.299358+0000", 2020-08-07T15:12:24.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.229e-06 2020-08-07T15:12:24.612 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.612 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.613 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:24.613 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.299359+0000", 2020-08-07T15:12:24.629 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3491999999999994e-05 2020-08-07T15:12:24.629 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.629 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.629 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:24.630 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.299433+0000", 2020-08-07T15:12:24.630 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0036857529999999999 2020-08-07T15:12:24.630 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.630 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.631 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:24.631 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.303118+0000", 2020-08-07T15:12:24.631 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0086804099999999995 2020-08-07T15:12:24.632 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:24.632 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:24.632 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:24.632 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.633 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.633 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:18264 6.2 6:4061dd2c:::benchmark_data_smithi074_63843_object2282:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e240)", 2020-08-07T15:12:24.633 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.251261+0000", 2020-08-07T15:12:24.633 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.056435405000000001, 2020-08-07T15:12:24.636 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.056959846000000001, 2020-08-07T15:12:24.636 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:24.637 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:24.637 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:24.637 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:24.637 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:24.638 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18264 2020-08-07T15:12:24.638 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.638 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:24.639 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.639 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:24.639 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.251261+0000", 2020-08-07T15:12:24.639 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:24.640 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.640 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.640 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:24.640 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.251261+0000", 2020-08-07T15:12:24.641 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9987999999999997e-05 2020-08-07T15:12:24.641 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.641 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.641 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:24.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.251301+0000", 2020-08-07T15:12:24.642 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.22e-06 2020-08-07T15:12:24.642 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.642 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.643 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:24.643 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.251306+0000", 2020-08-07T15:12:24.643 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9840000000000001e-06 2020-08-07T15:12:24.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.643 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.644 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:24.644 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.251314+0000", 2020-08-07T15:12:24.644 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4920000000000004e-06 2020-08-07T15:12:24.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:24.648 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.251318+0000", 2020-08-07T15:12:24.648 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002504676 2020-08-07T15:12:24.648 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.649 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.649 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:24.649 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.253823+0000", 2020-08-07T15:12:24.649 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5740000000000002e-05 2020-08-07T15:12:24.650 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.650 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.650 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:24.650 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.253919+0000", 2020-08-07T15:12:24.651 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017218100000000001 2020-08-07T15:12:24.651 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.651 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.651 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:12:24.652 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.254091+0000", 2020-08-07T15:12:24.652 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033054880000000001 2020-08-07T15:12:24.652 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.652 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.653 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:24.653 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.257397+0000", 2020-08-07T15:12:24.653 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.09e-07 2020-08-07T15:12:24.654 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.654 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.654 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:24.654 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.257397+0000", 2020-08-07T15:12:24.655 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3022999999999995e-05 2020-08-07T15:12:24.655 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.655 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.655 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:24.656 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.257470+0000", 2020-08-07T15:12:24.656 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.031864775999999997 2020-08-07T15:12:24.656 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.656 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.657 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:24.657 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.289335+0000", 2020-08-07T15:12:24.658 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.038074377 2020-08-07T15:12:24.658 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:24.658 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:24.658 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:24.659 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.659 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.659 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:18369 6.b 6:d04ffc29:::benchmark_data_smithi074_63843_object2296:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e240)", 2020-08-07T15:12:24.659 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.303882+0000", 2020-08-07T15:12:24.660 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.003814544, 2020-08-07T15:12:24.660 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044368690000000004, 2020-08-07T15:12:24.660 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:24.661 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:12:24.661 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:24.661 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:24.661 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:24.661 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18369 2020-08-07T15:12:24.662 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.662 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:24.662 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.662 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:24.663 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.303882+0000", 2020-08-07T15:12:24.663 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:24.663 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.664 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:24.664 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.303882+0000", 2020-08-07T15:12:24.664 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8010000000000002e-05 2020-08-07T15:12:24.665 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.665 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.665 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:24.665 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.303940+0000", 2020-08-07T15:12:24.666 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7889999999999997e-06 2020-08-07T15:12:24.666 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.666 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.666 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:24.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.303946+0000", 2020-08-07T15:12:24.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3655000000000001e-05 2020-08-07T15:12:24.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:24.668 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.303960+0000", 2020-08-07T15:12:24.668 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3489999999999997e-06 2020-08-07T15:12:24.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:24.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.303966+0000", 2020-08-07T15:12:24.670 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0015999999999997e-05 2020-08-07T15:12:24.670 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.670 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.670 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:24.671 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.304016+0000", 2020-08-07T15:12:24.671 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019501100000000001 2020-08-07T15:12:24.671 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.671 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.672 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:24.672 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.304211+0000", 2020-08-07T15:12:24.672 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00023657999999999999 2020-08-07T15:12:24.672 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.673 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.673 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:12:24.673 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.304448+0000", 2020-08-07T15:12:24.673 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003680104 2020-08-07T15:12:24.674 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.674 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.674 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:24.674 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.308128+0000", 2020-08-07T15:12:24.675 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0655e-05 2020-08-07T15:12:24.675 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:24.675 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:24.675 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:24.676 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.308139+0000", 2020-08-07T15:12:24.676 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042571689999999999 2020-08-07T15:12:24.676 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:24.676 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:24.677 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:24.677 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:24.677 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:12:24.677 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 8 2020-08-07T15:12:24.678 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:24.680 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:12:24.680 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:24.677+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:24.786 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:24.781+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:24.890 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:24.885+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:24.891 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:24.891 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:12:24.892 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:12:24.892 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:12:24.892 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:24.906 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:12:25.019 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:25.007+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:25.091 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:25.089+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:25.147 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:25.147 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:12:25.148 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:12:25.148 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:12:25.148 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.148 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106240 3.7 3:ebdd8242:::benchmark_data_smithi074_35447_object13279:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:12:25.148 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:12:25.149 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 277.78729039699999, 2020-08-07T15:12:25.149 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.90079396, 2020-08-07T15:12:25.149 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:25.149 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:25.172 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:25.173 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:12:25.173 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:12:25.173 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106240 2020-08-07T15:12:25.173 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.173 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:25.174 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.174 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:25.174 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:12:25.174 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:25.174 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.175 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.175 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:25.175 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:12:25.175 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2522000000000003e-05 2020-08-07T15:12:25.175 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.176 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.176 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:25.176 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355118+0000", 2020-08-07T15:12:25.176 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6159999999999998e-06 2020-08-07T15:12:25.176 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.177 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.177 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:25.177 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355125+0000", 2020-08-07T15:12:25.177 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2699e-05 2020-08-07T15:12:25.177 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.189 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.190 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:25.190 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355138+0000", 2020-08-07T15:12:25.190 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2310000000000001e-06 2020-08-07T15:12:25.198 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.198 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.198 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:25.198 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355144+0000", 2020-08-07T15:12:25.218 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9855000000000003e-05 2020-08-07T15:12:25.218 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.219 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.219 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:25.219 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355194+0000", 2020-08-07T15:12:25.220 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112713 2020-08-07T15:12:25.230 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.230 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.230 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:25.230 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355306+0000", 2020-08-07T15:12:25.231 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019794600000000001 2020-08-07T15:12:25.231 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.231 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.231 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:12:25.231 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355504+0000", 2020-08-07T15:12:25.232 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0048346029999999998 2020-08-07T15:12:25.232 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.232 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.232 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:25.232 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.360339+0000", 2020-08-07T15:12:25.233 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00033646400000000002 2020-08-07T15:12:25.233 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.233 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.233 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:25.233 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.360675+0000", 2020-08-07T15:12:25.251 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89508481200000001 2020-08-07T15:12:25.252 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.252 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.252 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:25.252 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255760+0000", 2020-08-07T15:12:25.253 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5659999999999999e-06 2020-08-07T15:12:25.253 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.253 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:25.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255762+0000", 2020-08-07T15:12:25.254 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6595000000000003e-05 2020-08-07T15:12:25.254 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.254 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.254 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:25.254 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255818+0000", 2020-08-07T15:12:25.254 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1338000000000003e-05 2020-08-07T15:12:25.255 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.269 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.269 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:25.269 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255860+0000", 2020-08-07T15:12:25.270 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.90079396 2020-08-07T15:12:25.270 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:25.270 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:25.270 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:25.270 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.271 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.271 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106240 3.7 e59/20 ondisk, result = 0)", 2020-08-07T15:12:25.271 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:12:25.271 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 277.69089444500003, 2020-08-07T15:12:25.271 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80440215100000001, 2020-08-07T15:12:25.272 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:25.272 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:12:25.272 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:25.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:25.273 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:12:25.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:25.273 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.273 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:25.298 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:12:25.298 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0052999999999999e-05 2020-08-07T15:12:25.298 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.299 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.299 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:25.299 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451482+0000", 2020-08-07T15:12:25.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3229999999999999e-06 2020-08-07T15:12:25.299 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.300 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.300 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:25.300 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451486+0000", 2020-08-07T15:12:25.300 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80420304799999998 2020-08-07T15:12:25.300 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.301 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.301 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:25.301 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255689+0000", 2020-08-07T15:12:25.301 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7759999999999997e-06 2020-08-07T15:12:25.301 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.311 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.311 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:25.311 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255694+0000", 2020-08-07T15:12:25.311 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5828999999999998e-05 2020-08-07T15:12:25.311 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.312 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.312 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:25.312 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255740+0000", 2020-08-07T15:12:25.322 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1466e-05 2020-08-07T15:12:25.322 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.323 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.323 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:25.323 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255751+0000", 2020-08-07T15:12:25.324 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112656 2020-08-07T15:12:25.324 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.324 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.324 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:25.324 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255864+0000", 2020-08-07T15:12:25.325 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80440215100000001 2020-08-07T15:12:25.325 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:25.325 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:25.325 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:25.351 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.351 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.352 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37057 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:25.352 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:12:25.352 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 210.89878573499999, 2020-08-07T15:12:25.352 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007248691, 2020-08-07T15:12:25.352 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:25.353 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:25.353 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:25.353 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:25.353 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:25.353 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37057 2020-08-07T15:12:25.354 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.354 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:25.354 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.354 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:25.354 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:12:25.355 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:25.355 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.355 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.355 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:25.355 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:12:25.356 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3661e-05 2020-08-07T15:12:25.356 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.356 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.356 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:25.356 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243624+0000", 2020-08-07T15:12:25.357 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7519999999999996e-06 2020-08-07T15:12:25.357 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.357 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.357 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:25.357 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243632+0000", 2020-08-07T15:12:25.382 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6392000000000002e-05 2020-08-07T15:12:25.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.383 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.383 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:25.384 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243648+0000", 2020-08-07T15:12:25.384 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6659999999999997e-06 2020-08-07T15:12:25.397 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.398 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.398 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:25.399 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243657+0000", 2020-08-07T15:12:25.399 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0070484609999999998 2020-08-07T15:12:25.399 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.400 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:25.414 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250705+0000", 2020-08-07T15:12:25.415 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0083e-05 2020-08-07T15:12:25.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.416 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:25.416 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250715+0000", 2020-08-07T15:12:25.416 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000086695 2020-08-07T15:12:25.417 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.417 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.417 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:25.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.250802+0000", 2020-08-07T15:12:25.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6980999999999999e-05 2020-08-07T15:12:25.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.430 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.430 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:25.450 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.250819+0000", 2020-08-07T15:12:25.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007248691 2020-08-07T15:12:25.451 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:25.451 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:25.451 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:25.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.452 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37058 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:25.452 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:12:25.452 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 210.89864400100001, 2020-08-07T15:12:25.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007368357, 2020-08-07T15:12:25.453 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:25.453 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:25.453 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:25.453 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:25.476 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:25.477 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37058 2020-08-07T15:12:25.477 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.477 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:25.478 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.478 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:25.479 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:12:25.479 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:25.479 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.494 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:25.495 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:12:25.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8838999999999998e-05 2020-08-07T15:12:25.495 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.495 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.495 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:25.496 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243761+0000", 2020-08-07T15:12:25.496 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2079999999999999e-06 2020-08-07T15:12:25.496 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.496 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.496 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:25.497 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243766+0000", 2020-08-07T15:12:25.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0567999999999999e-05 2020-08-07T15:12:25.497 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.497 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.516 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:25.516 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243777+0000", 2020-08-07T15:12:25.517 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7210000000000002e-06 2020-08-07T15:12:25.517 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.518 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:25.519 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243782+0000", 2020-08-07T15:12:25.519 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069508749999999996 2020-08-07T15:12:25.519 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.520 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:25.521 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250732+0000", 2020-08-07T15:12:25.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7899999999999999e-06 2020-08-07T15:12:25.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.523 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.523 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:25.523 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250737+0000", 2020-08-07T15:12:25.524 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003272409999999 2020-08-07T15:12:25.524 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.524 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.525 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:25.526 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251064+0000", 2020-08-07T15:12:25.534 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6115000000000001e-05 2020-08-07T15:12:25.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.540 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.540 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:25.541 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251081+0000", 2020-08-07T15:12:25.541 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007368357 2020-08-07T15:12:25.541 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:25.542 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:25.542 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:25.542 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.543 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.543 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37059 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:25.543 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:12:25.543 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 210.898400952, 2020-08-07T15:12:25.544 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0071655319999999, 2020-08-07T15:12:25.544 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:25.544 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:25.544 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:25.544 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:25.545 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:25.545 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37059 2020-08-07T15:12:25.545 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.545 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:25.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.564 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:25.564 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:12:25.565 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:25.565 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.566 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.566 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:25.567 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:12:25.567 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6533000000000002e-05 2020-08-07T15:12:25.567 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.568 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.568 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:25.569 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244012+0000", 2020-08-07T15:12:25.569 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3860000000000001e-06 2020-08-07T15:12:25.579 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.579 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.580 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:25.580 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244019+0000", 2020-08-07T15:12:25.581 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6982000000000001e-05 2020-08-07T15:12:25.581 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.598 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.598 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:25.599 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244036+0000", 2020-08-07T15:12:25.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0310000000000001e-06 2020-08-07T15:12:25.599 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.599 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.600 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:25.600 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244043+0000", 2020-08-07T15:12:25.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0067169259999999998 2020-08-07T15:12:25.600 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.600 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.601 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:25.601 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250760+0000", 2020-08-07T15:12:25.601 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4499999999999998e-06 2020-08-07T15:12:25.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.603 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:25.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250768+0000", 2020-08-07T15:12:25.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000341691 2020-08-07T15:12:25.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.626 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.626 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:25.627 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251109+0000", 2020-08-07T15:12:25.627 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1533e-05 2020-08-07T15:12:25.627 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.627 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:25.628 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251121+0000", 2020-08-07T15:12:25.628 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0071655319999999 2020-08-07T15:12:25.628 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:25.628 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:25.628 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:25.629 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.629 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.629 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37060 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:25.629 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:12:25.650 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 210.89827701499999, 2020-08-07T15:12:25.651 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070762280000001, 2020-08-07T15:12:25.651 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:25.651 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:25.652 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:25.652 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:25.652 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:25.653 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37060 2020-08-07T15:12:25.653 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.653 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:25.674 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.675 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:25.675 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:12:25.675 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:25.675 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.675 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.676 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:25.676 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:12:25.676 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5739000000000003e-05 2020-08-07T15:12:25.676 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:25.677 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244135+0000", 2020-08-07T15:12:25.677 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7740000000000001e-06 2020-08-07T15:12:25.677 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.692 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:25.692 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244141+0000", 2020-08-07T15:12:25.693 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3817e-05 2020-08-07T15:12:25.693 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.693 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.693 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:25.694 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244155+0000", 2020-08-07T15:12:25.733 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3829999999999997e-06 2020-08-07T15:12:25.734 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.734 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.734 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:25.751 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244161+0000", 2020-08-07T15:12:25.751 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0066303669999999999 2020-08-07T15:12:25.752 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.752 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:25.753 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250791+0000", 2020-08-07T15:12:25.753 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.392e-06 2020-08-07T15:12:25.753 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.753 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.753 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:25.754 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250798+0000", 2020-08-07T15:12:25.754 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003485860000001 2020-08-07T15:12:25.754 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.755 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:25.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251146+0000", 2020-08-07T15:12:25.755 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1700000000000003e-06 2020-08-07T15:12:25.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:25.756 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251155+0000", 2020-08-07T15:12:25.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070762280000001 2020-08-07T15:12:25.795 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:25.795 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:25.795 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:25.796 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.796 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37061 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:25.797 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:12:25.797 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 210.898162388, 2020-08-07T15:12:25.797 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070006920000001, 2020-08-07T15:12:25.797 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:25.798 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:25.798 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:25.798 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:25.799 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:25.799 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37061 2020-08-07T15:12:25.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.799 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:25.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:25.800 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:12:25.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:25.801 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.801 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.801 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:25.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:12:25.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5453e-05 2020-08-07T15:12:25.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:25.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244239+0000", 2020-08-07T15:12:25.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3179999999999997e-06 2020-08-07T15:12:25.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:25.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244245+0000", 2020-08-07T15:12:25.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2138e-05 2020-08-07T15:12:25.806 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:25.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244257+0000", 2020-08-07T15:12:25.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.69e-06 2020-08-07T15:12:25.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.810 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.811 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:25.811 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244261+0000", 2020-08-07T15:12:25.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0065598940000000001 2020-08-07T15:12:25.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:25.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250821+0000", 2020-08-07T15:12:25.813 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4799999999999998e-06 2020-08-07T15:12:25.814 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.814 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:25.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250828+0000", 2020-08-07T15:12:25.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003567310000001 2020-08-07T15:12:25.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.817 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:25.818 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251185+0000", 2020-08-07T15:12:25.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9879999999999999e-06 2020-08-07T15:12:25.818 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.818 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:25.819 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251195+0000", 2020-08-07T15:12:25.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070006920000001 2020-08-07T15:12:25.820 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:25.820 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:25.821 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:25.821 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.822 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37062 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:25.822 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:12:25.822 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 210.89806217, 2020-08-07T15:12:25.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006933662, 2020-08-07T15:12:25.823 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:25.823 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:25.823 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:25.823 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:25.843 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:25.844 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37062 2020-08-07T15:12:25.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.844 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:25.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.844 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:25.845 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:12:25.845 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:25.845 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:25.846 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:12:25.846 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4041e-05 2020-08-07T15:12:25.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.846 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.846 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:25.847 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244348+0000", 2020-08-07T15:12:25.847 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2079999999999997e-06 2020-08-07T15:12:25.847 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.847 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.848 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:25.849 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244354+0000", 2020-08-07T15:12:25.849 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1483e-05 2020-08-07T15:12:25.850 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.850 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:25.851 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244366+0000", 2020-08-07T15:12:25.851 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7989999999999999e-06 2020-08-07T15:12:25.851 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.851 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:25.852 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244372+0000", 2020-08-07T15:12:25.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0064783699999999998 2020-08-07T15:12:25.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.876 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:25.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250850+0000", 2020-08-07T15:12:25.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4729999999999997e-06 2020-08-07T15:12:25.883 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.883 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.883 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:25.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250856+0000", 2020-08-07T15:12:25.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003630240000001 2020-08-07T15:12:25.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.896 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:25.897 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251219+0000", 2020-08-07T15:12:25.897 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2640000000000008e-06 2020-08-07T15:12:25.897 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.898 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.898 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:25.898 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251228+0000", 2020-08-07T15:12:25.898 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006933662 2020-08-07T15:12:25.898 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:25.899 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:25.899 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:25.899 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.899 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.916 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37063 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:25.916 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:12:25.917 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 210.89794983600001, 2020-08-07T15:12:25.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006858051, 2020-08-07T15:12:25.917 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:25.917 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:25.917 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:25.918 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:25.918 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:25.918 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37063 2020-08-07T15:12:25.918 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.919 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:25.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.919 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:25.919 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:12:25.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:25.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:25.920 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:12:25.920 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5043000000000001e-05 2020-08-07T15:12:25.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:25.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244461+0000", 2020-08-07T15:12:25.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4600000000000002e-06 2020-08-07T15:12:25.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:25.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244467+0000", 2020-08-07T15:12:25.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1435e-05 2020-08-07T15:12:25.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:25.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244478+0000", 2020-08-07T15:12:25.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0190000000000003e-06 2020-08-07T15:12:25.925 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:25.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244484+0000", 2020-08-07T15:12:25.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006392895 2020-08-07T15:12:25.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:25.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250877+0000", 2020-08-07T15:12:25.932 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4280000000000001e-06 2020-08-07T15:12:25.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.932 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.933 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:25.933 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250884+0000", 2020-08-07T15:12:25.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000370414 2020-08-07T15:12:25.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.975 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:25.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251254+0000", 2020-08-07T15:12:25.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0356999999999999e-05 2020-08-07T15:12:25.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:25.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:25.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:25.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251264+0000", 2020-08-07T15:12:25.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006858051 2020-08-07T15:12:25.977 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:26.014 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:26.015 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:26.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.015 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.015 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37064 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:26.016 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:12:26.016 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 210.89783770299999, 2020-08-07T15:12:26.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0067798189999999, 2020-08-07T15:12:26.016 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:26.017 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:26.017 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:26.017 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:26.061 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:26.062 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37064 2020-08-07T15:12:26.062 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.065 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:26.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.065 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:26.065 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:12:26.066 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:26.066 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.066 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.066 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:26.066 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:12:26.067 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4007e-05 2020-08-07T15:12:26.067 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.082 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:26.083 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244573+0000", 2020-08-07T15:12:26.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.831e-06 2020-08-07T15:12:26.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.083 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.083 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:26.084 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244577+0000", 2020-08-07T15:12:26.084 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2269000000000001e-05 2020-08-07T15:12:26.084 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.084 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:26.085 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244590+0000", 2020-08-07T15:12:26.085 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7149999999999997e-06 2020-08-07T15:12:26.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.085 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:26.110 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244594+0000", 2020-08-07T15:12:26.110 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063105670000000004 2020-08-07T15:12:26.110 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.110 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.111 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:26.111 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250905+0000", 2020-08-07T15:12:26.118 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1970000000000001e-06 2020-08-07T15:12:26.118 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.119 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.119 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:26.119 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250911+0000", 2020-08-07T15:12:26.120 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003783020000001 2020-08-07T15:12:26.120 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.120 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.120 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:26.121 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251289+0000", 2020-08-07T15:12:26.121 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9309999999999997e-06 2020-08-07T15:12:26.121 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.134 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.134 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:26.134 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251298+0000", 2020-08-07T15:12:26.134 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0067798189999999 2020-08-07T15:12:26.134 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:26.135 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:26.135 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:26.135 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.135 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.135 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37065 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:26.136 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:12:26.136 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 210.897727484, 2020-08-07T15:12:26.136 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006708545, 2020-08-07T15:12:26.136 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:26.136 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:26.137 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:26.137 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:26.137 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:26.137 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37065 2020-08-07T15:12:26.137 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.162 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:26.176 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.177 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:26.177 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:12:26.177 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:26.177 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.178 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.178 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:26.178 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:12:26.178 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2750000000000001e-05 2020-08-07T15:12:26.178 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.179 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.179 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:26.179 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244681+0000", 2020-08-07T15:12:26.179 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0950000000000003e-06 2020-08-07T15:12:26.179 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.180 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.180 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:26.180 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244687+0000", 2020-08-07T15:12:26.180 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2485e-05 2020-08-07T15:12:26.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.181 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.181 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:26.181 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244699+0000", 2020-08-07T15:12:26.181 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8800000000000002e-06 2020-08-07T15:12:26.181 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.215 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.216 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:26.216 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244706+0000", 2020-08-07T15:12:26.216 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0062281569999999998 2020-08-07T15:12:26.216 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.217 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.217 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:26.217 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250934+0000", 2020-08-07T15:12:26.217 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1719999999999998e-06 2020-08-07T15:12:26.217 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.218 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.218 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:26.218 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250939+0000", 2020-08-07T15:12:26.218 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003861810000001 2020-08-07T15:12:26.219 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.219 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.219 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:26.219 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251325+0000", 2020-08-07T15:12:26.219 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1824999999999999e-05 2020-08-07T15:12:26.220 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.220 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.220 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:26.220 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251337+0000", 2020-08-07T15:12:26.220 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006708545 2020-08-07T15:12:26.220 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:26.221 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:26.221 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:26.221 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.221 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.239 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37066 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:26.239 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:12:26.240 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 210.89305851399999, 2020-08-07T15:12:26.240 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002077205, 2020-08-07T15:12:26.240 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:26.240 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:26.240 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:26.241 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:26.241 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:26.247 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37066 2020-08-07T15:12:26.247 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.260 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:26.260 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.260 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:26.261 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:12:26.261 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:26.262 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.264 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.264 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:26.265 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:12:26.265 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4654000000000002e-05 2020-08-07T15:12:26.265 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.265 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.265 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:26.266 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249352+0000", 2020-08-07T15:12:26.266 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6210000000000004e-06 2020-08-07T15:12:26.266 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.266 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.267 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:26.268 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249360+0000", 2020-08-07T15:12:26.268 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1895999999999998e-05 2020-08-07T15:12:26.268 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.268 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.268 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:26.269 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249382+0000", 2020-08-07T15:12:26.269 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1479999999999995e-06 2020-08-07T15:12:26.269 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.269 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.270 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:26.270 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249391+0000", 2020-08-07T15:12:26.291 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015731510000000001 2020-08-07T15:12:26.291 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.291 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.292 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:26.292 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250964+0000", 2020-08-07T15:12:26.292 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5400000000000001e-06 2020-08-07T15:12:26.292 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.292 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.292 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:26.293 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250971+0000", 2020-08-07T15:12:26.293 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000392379 2020-08-07T15:12:26.293 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.293 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.319 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:26.319 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251363+0000", 2020-08-07T15:12:26.319 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1816e-05 2020-08-07T15:12:26.319 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.320 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.320 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:26.320 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251375+0000", 2020-08-07T15:12:26.320 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002077205 2020-08-07T15:12:26.320 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:26.334 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:26.334 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:26.334 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.334 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.335 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37067 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:26.335 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:12:26.335 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 210.89291782000001, 2020-08-07T15:12:26.335 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0019761119999999, 2020-08-07T15:12:26.335 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:26.336 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:26.336 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:26.336 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:26.336 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:26.336 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37067 2020-08-07T15:12:26.337 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.337 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:26.337 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.337 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:26.337 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:12:26.338 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:26.338 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.338 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.338 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:26.338 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:12:26.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5961999999999997e-05 2020-08-07T15:12:26.339 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.339 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.339 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:26.339 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249494+0000", 2020-08-07T15:12:26.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3210000000000004e-06 2020-08-07T15:12:26.340 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.340 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:26.340 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249500+0000", 2020-08-07T15:12:26.341 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1012000000000001e-05 2020-08-07T15:12:26.341 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.341 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:26.341 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249511+0000", 2020-08-07T15:12:26.366 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.809e-06 2020-08-07T15:12:26.366 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.367 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:26.367 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249515+0000", 2020-08-07T15:12:26.367 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0014777670000000001 2020-08-07T15:12:26.367 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.367 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.368 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:26.368 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250993+0000", 2020-08-07T15:12:26.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9499999999999998e-06 2020-08-07T15:12:26.368 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.368 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.369 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:26.369 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250999+0000", 2020-08-07T15:12:26.369 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0004032519999999 2020-08-07T15:12:26.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.390 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:26.390 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251402+0000", 2020-08-07T15:12:26.390 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2038999999999999e-05 2020-08-07T15:12:26.391 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.391 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.391 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:26.391 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251414+0000", 2020-08-07T15:12:26.391 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0019761119999999 2020-08-07T15:12:26.392 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:26.392 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:26.392 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:26.392 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.392 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.393 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37068 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:26.393 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:12:26.393 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 210.890673249, 2020-08-07T15:12:26.393 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99977072, 2020-08-07T15:12:26.393 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:26.403 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:26.404 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:26.404 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:26.404 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:26.404 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37068 2020-08-07T15:12:26.405 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.405 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:26.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.405 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:26.405 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:12:26.406 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:26.406 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.406 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.406 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:26.407 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:12:26.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1369000000000003e-05 2020-08-07T15:12:26.407 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.407 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.407 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:26.408 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251724+0000", 2020-08-07T15:12:26.408 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.19e-06 2020-08-07T15:12:26.408 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.408 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.408 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:26.410 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251731+0000", 2020-08-07T15:12:26.416 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1368e-05 2020-08-07T15:12:26.416 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.417 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.417 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:26.417 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251742+0000", 2020-08-07T15:12:26.418 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.682e-06 2020-08-07T15:12:26.418 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.419 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.420 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:26.420 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251748+0000", 2020-08-07T15:12:26.421 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5275000000000003e-05 2020-08-07T15:12:26.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.421 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.422 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:26.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251793+0000", 2020-08-07T15:12:26.423 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5000000000000005e-06 2020-08-07T15:12:26.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.425 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:26.426 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251802+0000", 2020-08-07T15:12:26.426 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99963953500000002 2020-08-07T15:12:26.427 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.427 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:26.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251442+0000", 2020-08-07T15:12:26.441 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1800999999999999e-05 2020-08-07T15:12:26.441 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.441 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.443 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:26.443 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251454+0000", 2020-08-07T15:12:26.443 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99977072 2020-08-07T15:12:26.443 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:26.443 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:26.444 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:26.446 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.447 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37069 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:26.447 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:12:26.447 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 210.889569451, 2020-08-07T15:12:26.448 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99870607899999997, 2020-08-07T15:12:26.448 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:26.448 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:26.452 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:26.452 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:26.452 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:26.478 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37069 2020-08-07T15:12:26.478 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.479 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:26.479 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.479 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:26.479 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:12:26.480 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:26.480 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.481 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:26.481 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:12:26.481 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3755999999999999e-05 2020-08-07T15:12:26.502 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.502 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.502 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:26.503 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252831+0000", 2020-08-07T15:12:26.503 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7919999999999998e-06 2020-08-07T15:12:26.503 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.504 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.504 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:26.518 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252836+0000", 2020-08-07T15:12:26.519 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1409999999999999e-05 2020-08-07T15:12:26.519 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.520 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:26.520 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252848+0000", 2020-08-07T15:12:26.523 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6380000000000001e-06 2020-08-07T15:12:26.524 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.524 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.524 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:26.525 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252853+0000", 2020-08-07T15:12:26.525 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3244e-05 2020-08-07T15:12:26.533 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.533 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:26.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252907+0000", 2020-08-07T15:12:26.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1470000000000007e-06 2020-08-07T15:12:26.537 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.542 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.546 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:26.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252916+0000", 2020-08-07T15:12:26.547 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99856497499999997 2020-08-07T15:12:26.547 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.548 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:26.548 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251481+0000", 2020-08-07T15:12:26.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2116999999999999e-05 2020-08-07T15:12:26.549 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.549 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.550 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:26.551 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251493+0000", 2020-08-07T15:12:26.558 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99870607899999997 2020-08-07T15:12:26.559 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:26.559 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:26.559 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:26.559 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.559 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.560 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37070 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:26.560 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:12:26.560 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 210.889465188, 2020-08-07T15:12:26.561 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99863946699999995, 2020-08-07T15:12:26.561 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:26.561 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:26.562 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:26.562 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:26.563 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:26.563 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37070 2020-08-07T15:12:26.564 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.564 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:26.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.565 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:26.565 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:12:26.566 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:26.566 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.566 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.567 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:26.568 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:12:26.569 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9984000000000001e-05 2020-08-07T15:12:26.569 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.569 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.570 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:26.570 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252941+0000", 2020-08-07T15:12:26.570 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7010000000000004e-06 2020-08-07T15:12:26.571 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.571 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.572 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:26.572 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252946+0000", 2020-08-07T15:12:26.573 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1144999999999999e-05 2020-08-07T15:12:26.573 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.574 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.574 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:26.575 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252957+0000", 2020-08-07T15:12:26.575 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6920000000000001e-06 2020-08-07T15:12:26.575 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.576 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.576 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:26.576 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252962+0000", 2020-08-07T15:12:26.577 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6739000000000002e-05 2020-08-07T15:12:26.577 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.592 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.592 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:26.593 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252998+0000", 2020-08-07T15:12:26.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.503e-06 2020-08-07T15:12:26.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.614 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.614 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:26.631 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.253006+0000", 2020-08-07T15:12:26.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99851375499999995 2020-08-07T15:12:26.633 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.633 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.633 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:26.634 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251520+0000", 2020-08-07T15:12:26.658 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0947999999999999e-05 2020-08-07T15:12:26.658 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.662 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.663 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:26.663 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251530+0000", 2020-08-07T15:12:26.664 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99863946699999995 2020-08-07T15:12:26.664 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:26.664 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:26.665 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:26.665 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.666 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.667 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42505 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:12:26.667 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:12:26.667 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 204.84012465800001, 2020-08-07T15:12:26.668 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0024736009999999, 2020-08-07T15:12:26.668 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:26.668 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:26.669 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:26.669 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:26.674 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:26.675 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42505 2020-08-07T15:12:26.675 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.676 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:26.676 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.690 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:26.691 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:12:26.699 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:26.699 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.700 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:26.701 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:12:26.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011052600000000001 2020-08-07T15:12:26.701 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.702 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:26.702 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302342+0000", 2020-08-07T15:12:26.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.781e-06 2020-08-07T15:12:26.703 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.703 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:26.708 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302352+0000", 2020-08-07T15:12:26.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7646e-05 2020-08-07T15:12:26.709 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.709 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.710 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:26.710 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302369+0000", 2020-08-07T15:12:26.710 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3629999999999994e-06 2020-08-07T15:12:26.710 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.711 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:26.711 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302378+0000", 2020-08-07T15:12:26.711 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8007000000000003e-05 2020-08-07T15:12:26.711 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.712 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.712 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:26.712 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302426+0000", 2020-08-07T15:12:26.712 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5159999999999994e-06 2020-08-07T15:12:26.713 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.713 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.713 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:26.714 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302434+0000", 2020-08-07T15:12:26.714 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99151562699999995 2020-08-07T15:12:26.714 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.715 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.715 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:26.715 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.293950+0000", 2020-08-07T15:12:26.715 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020138099999999999 2020-08-07T15:12:26.716 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.716 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.716 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:26.716 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294151+0000", 2020-08-07T15:12:26.717 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024761 2020-08-07T15:12:26.717 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.717 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.717 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:12:26.717 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294399+0000", 2020-08-07T15:12:26.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0027714369999999999 2020-08-07T15:12:26.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.718 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.718 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:26.718 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297170+0000", 2020-08-07T15:12:26.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.234e-06 2020-08-07T15:12:26.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:26.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297172+0000", 2020-08-07T15:12:26.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5384999999999998e-05 2020-08-07T15:12:26.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.721 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:26.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297237+0000", 2020-08-07T15:12:26.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0073576980000000002 2020-08-07T15:12:26.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.722 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:26.722 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304595+0000", 2020-08-07T15:12:26.722 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1006000000000001e-05 2020-08-07T15:12:26.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.723 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:26.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304626+0000", 2020-08-07T15:12:26.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6000000000000003e-07 2020-08-07T15:12:26.724 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.724 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.724 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:26.724 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304626+0000", 2020-08-07T15:12:26.724 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8724e-05 2020-08-07T15:12:26.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.726 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:26.726 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304705+0000", 2020-08-07T15:12:26.726 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0024736009999999 2020-08-07T15:12:26.726 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:26.727 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:26.727 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:26.727 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.727 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.727 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42506 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:12:26.728 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:12:26.728 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 204.83993344500001, 2020-08-07T15:12:26.728 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002465226, 2020-08-07T15:12:26.728 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:26.729 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:26.729 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:26.732 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:26.732 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:26.732 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42506 2020-08-07T15:12:26.733 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.733 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:26.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.733 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:26.734 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:12:26.734 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:26.734 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.734 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.734 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:26.735 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:12:26.735 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8780999999999999e-05 2020-08-07T15:12:26.746 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.746 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:26.747 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302472+0000", 2020-08-07T15:12:26.747 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3740000000000003e-06 2020-08-07T15:12:26.747 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.747 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.747 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:26.748 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302478+0000", 2020-08-07T15:12:26.748 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1155000000000001e-05 2020-08-07T15:12:26.748 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.748 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.748 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:26.749 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302489+0000", 2020-08-07T15:12:26.749 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7559999999999996e-06 2020-08-07T15:12:26.749 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.749 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.759 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:26.760 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302495+0000", 2020-08-07T15:12:26.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4128000000000003e-05 2020-08-07T15:12:26.761 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.761 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:26.793 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302529+0000", 2020-08-07T15:12:26.793 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.516e-06 2020-08-07T15:12:26.795 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:26.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302535+0000", 2020-08-07T15:12:26.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99240899299999996 2020-08-07T15:12:26.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:26.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294944+0000", 2020-08-07T15:12:26.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000115806 2020-08-07T15:12:26.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.809 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:26.809 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295060+0000", 2020-08-07T15:12:26.810 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000205002 2020-08-07T15:12:26.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.811 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:12:26.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295265+0000", 2020-08-07T15:12:26.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043414400000000002 2020-08-07T15:12:26.814 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.817 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:26.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299607+0000", 2020-08-07T15:12:26.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2229999999999998e-06 2020-08-07T15:12:26.818 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.818 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:26.819 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299612+0000", 2020-08-07T15:12:26.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5797000000000002e-05 2020-08-07T15:12:26.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:26.822 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299658+0000", 2020-08-07T15:12:26.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005095044 2020-08-07T15:12:26.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:26.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304753+0000", 2020-08-07T15:12:26.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3121999999999996e-05 2020-08-07T15:12:26.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.827 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:26.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304816+0000", 2020-08-07T15:12:26.828 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5700000000000001e-07 2020-08-07T15:12:26.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.829 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:26.829 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304817+0000", 2020-08-07T15:12:26.830 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1231999999999998e-05 2020-08-07T15:12:26.830 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.831 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.831 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:26.832 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304888+0000", 2020-08-07T15:12:26.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002465226 2020-08-07T15:12:26.833 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:26.833 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:26.853 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:26.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.854 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.854 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42507 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:12:26.854 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:12:26.854 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 204.83982267499999, 2020-08-07T15:12:26.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002567097, 2020-08-07T15:12:26.855 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:26.855 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:26.855 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:26.855 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:26.855 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:26.856 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42507 2020-08-07T15:12:26.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.856 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:26.856 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.856 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:26.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:12:26.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:26.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.873 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:26.873 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:12:26.874 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7679000000000002e-05 2020-08-07T15:12:26.874 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.874 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.877 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:26.877 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302581+0000", 2020-08-07T15:12:26.878 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2299999999999999e-06 2020-08-07T15:12:26.878 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.878 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.878 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:26.878 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302586+0000", 2020-08-07T15:12:26.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0886999999999999e-05 2020-08-07T15:12:26.879 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.879 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:26.879 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302597+0000", 2020-08-07T15:12:26.880 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4129999999999997e-06 2020-08-07T15:12:26.880 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.880 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:26.880 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302603+0000", 2020-08-07T15:12:26.881 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3963000000000001e-05 2020-08-07T15:12:26.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:26.885 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302627+0000", 2020-08-07T15:12:26.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.33e-06 2020-08-07T15:12:26.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.886 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.886 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:26.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302631+0000", 2020-08-07T15:12:26.887 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99319860900000001 2020-08-07T15:12:26.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:26.887 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295830+0000", 2020-08-07T15:12:26.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000116012 2020-08-07T15:12:26.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:26.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295946+0000", 2020-08-07T15:12:26.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000230242 2020-08-07T15:12:26.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:12:26.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296176+0000", 2020-08-07T15:12:26.890 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003494455 2020-08-07T15:12:26.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.890 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:26.890 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299670+0000", 2020-08-07T15:12:26.891 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0299999999999998e-07 2020-08-07T15:12:26.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.891 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.891 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:26.892 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299671+0000", 2020-08-07T15:12:26.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7298000000000001e-05 2020-08-07T15:12:26.892 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.893 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:26.893 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299708+0000", 2020-08-07T15:12:26.893 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0052628839999999998 2020-08-07T15:12:26.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.895 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.895 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:26.896 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304971+0000", 2020-08-07T15:12:26.896 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3754e-05 2020-08-07T15:12:26.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.897 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.897 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:26.897 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305035+0000", 2020-08-07T15:12:26.898 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9500000000000001e-07 2020-08-07T15:12:26.898 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.898 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.899 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:26.899 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305036+0000", 2020-08-07T15:12:26.899 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4843000000000002e-05 2020-08-07T15:12:26.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:26.900 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:26.900 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305101+0000", 2020-08-07T15:12:26.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002567097 2020-08-07T15:12:26.901 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:26.901 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:26.908 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:26.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:26.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:27.010 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42508 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:12:27.010 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:12:27.010 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 204.82929765599999, 2020-08-07T15:12:27.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99221695600000004, 2020-08-07T15:12:27.011 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:27.011 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:27.011 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:27.012 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:27.012 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:27.012 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42508 2020-08-07T15:12:27.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:27.013 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:27.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:27.013 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:27.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:12:27.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:27.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:27.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:27.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:27.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:12:27.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3692e-05 2020-08-07T15:12:27.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:27.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:27.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:27.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313112+0000", 2020-08-07T15:12:27.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7399999999999998e-06 2020-08-07T15:12:27.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:27.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:27.041 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:27.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313119+0000", 2020-08-07T15:12:27.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5153999999999999e-05 2020-08-07T15:12:27.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:27.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:27.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:27.043 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313134+0000", 2020-08-07T15:12:27.045 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3039999999999999e-06 2020-08-07T15:12:27.045 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:27.045 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:27.046 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:27.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313141+0000", 2020-08-07T15:12:27.052 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6516000000000001e-05 2020-08-07T15:12:27.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:27.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:27.054 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:27.054 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313178+0000", 2020-08-07T15:12:27.054 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0949999999999998e-06 2020-08-07T15:12:27.055 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:27.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:27.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:27.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313185+0000", 2020-08-07T15:12:27.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.98342834999999995 2020-08-07T15:12:27.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:27.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:27.058 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:27.058 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296613+0000", 2020-08-07T15:12:27.058 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2230999999999993e-05 2020-08-07T15:12:27.059 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:27.059 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:27.059 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:27.060 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296686+0000", 2020-08-07T15:12:27.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013121300000000001 2020-08-07T15:12:27.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:27.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:27.061 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:12:27.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296817+0000", 2020-08-07T15:12:27.062 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044228050000000001 2020-08-07T15:12:27.062 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:27.062 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:27.062 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:27.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301240+0000", 2020-08-07T15:12:27.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0500000000000002e-07 2020-08-07T15:12:27.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:27.064 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:27.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:27.065 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301241+0000", 2020-08-07T15:12:27.065 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2434000000000006e-05 2020-08-07T15:12:27.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:27.066 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:27.066 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:27.067 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301303+0000", 2020-08-07T15:12:27.067 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038622919999999998 2020-08-07T15:12:27.067 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:27.067 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:27.067 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:27.068 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305165+0000", 2020-08-07T15:12:27.068 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8012000000000001e-05 2020-08-07T15:12:27.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:27.068 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:27.068 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:27.068 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305203+0000", 2020-08-07T15:12:27.069 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5299999999999999e-07 2020-08-07T15:12:27.069 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:27.069 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:27.069 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:27.069 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305204+0000", 2020-08-07T15:12:27.070 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1760000000000004e-05 2020-08-07T15:12:27.070 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:27.070 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:27.070 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:27.070 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305275+0000", 2020-08-07T15:12:27.071 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99221695600000004 2020-08-07T15:12:27.071 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:27.071 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:27.071 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:27.071 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:27.072 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:27.072 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:27.072 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2020-08-07T15:12:27.073 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:12:27.073 INFO:tasks.thrashosds.thrasher:Growing pool device_health_metrics 2020-08-07T15:12:27.075 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:12:27.077 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:12:27.078 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:12:27.081 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 13 16 19703 19687 11.8295 10.9922 0.0402433 0.0105601 2020-08-07T15:12:27.081 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 14 16 21336 21320 11.8958 12.7578 0.0106796 0.0104974 2020-08-07T15:12:27.081 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:27.071+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:27.083 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:12:27.175 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:27.173+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:27.266 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:27.266 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:12:27.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.267 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5481.0:22670 6.f e241/230 6:fb7bdf89:::benchmark_data_smithi074_63843_object2833:head v 241'1454, mlcod=240'1447)", 2020-08-07T15:12:27.267 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:27.261280+0000", 2020-08-07T15:12:27.268 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0014919670000000001, 2020-08-07T15:12:27.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001519893, 2020-08-07T15:12:27.268 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:27.269 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:27.269 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:27.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:27.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:27.261280+0000", 2020-08-07T15:12:27.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:27.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:27.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:27.261280+0000", 2020-08-07T15:12:27.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1881000000000002e-05 2020-08-07T15:12:27.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:27.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:27.261312+0000", 2020-08-07T15:12:27.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1389999999999997e-06 2020-08-07T15:12:27.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:27.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:27.261316+0000", 2020-08-07T15:12:27.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1332999999999999e-05 2020-08-07T15:12:27.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:27.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:27.261357+0000", 2020-08-07T15:12:27.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9670000000000003e-06 2020-08-07T15:12:27.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:27.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:27.261361+0000", 2020-08-07T15:12:27.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5188999999999998e-05 2020-08-07T15:12:27.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:27.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:27.261397+0000", 2020-08-07T15:12:27.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1713e-05 2020-08-07T15:12:27.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:27.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:27.261418+0000", 2020-08-07T15:12:27.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013822199999999999 2020-08-07T15:12:27.336 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:27.336 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:27.336 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:27.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.337 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5481.0:22671 6.f e241/230 6:fb7bdf89:::benchmark_data_smithi074_63843_object2833:head v 241'1455, mlcod=240'1447)", 2020-08-07T15:12:27.337 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:27.261879+0000", 2020-08-07T15:12:27.337 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00089344399999999997, 2020-08-07T15:12:27.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001003316, 2020-08-07T15:12:27.338 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:27.338 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:27.338 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:27.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:27.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:27.261879+0000", 2020-08-07T15:12:27.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:27.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:27.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:27.261879+0000", 2020-08-07T15:12:27.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8964000000000002e-05 2020-08-07T15:12:27.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:27.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:27.261918+0000", 2020-08-07T15:12:27.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2830000000000003e-06 2020-08-07T15:12:27.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:27.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:27.261922+0000", 2020-08-07T15:12:27.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0817e-05 2020-08-07T15:12:27.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:27.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:27.261953+0000", 2020-08-07T15:12:27.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1609999999999996e-06 2020-08-07T15:12:27.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:27.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:27.261957+0000", 2020-08-07T15:12:27.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9150999999999999e-05 2020-08-07T15:12:27.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:27.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:27.261986+0000", 2020-08-07T15:12:27.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6725e-05 2020-08-07T15:12:27.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:27.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:27.262003+0000", 2020-08-07T15:12:27.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000124101 2020-08-07T15:12:27.370 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:27.370 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:27.371 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:27.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.372 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5481.0:22672 6.f e241/230)", 2020-08-07T15:12:27.372 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:27.262772+0000", 2020-08-07T15:12:27.372 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.8400000000000001e-07, 2020-08-07T15:12:27.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017471500000000001, 2020-08-07T15:12:27.373 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:27.373 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:27.382 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:27.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:27.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:27.262772+0000", 2020-08-07T15:12:27.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:27.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:27.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:27.262772+0000", 2020-08-07T15:12:27.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4125000000000001e-05 2020-08-07T15:12:27.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:27.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:27.262826+0000", 2020-08-07T15:12:27.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6329999999999999e-06 2020-08-07T15:12:27.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:27.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:27.262833+0000", 2020-08-07T15:12:27.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4023999999999996e-05 2020-08-07T15:12:27.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:27.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:27.262897+0000", 2020-08-07T15:12:27.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0900000000000001e-06 2020-08-07T15:12:27.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:27.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:27.262903+0000", 2020-08-07T15:12:27.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0831999999999998e-05 2020-08-07T15:12:27.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:27.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:27.262944+0000", 2020-08-07T15:12:27.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017170400000000001 2020-08-07T15:12:27.398 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:27.398 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:27.399 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:27.399 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:27.399 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:12:27.399 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 3 2020-08-07T15:12:27.399 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:27.401 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 15 16 22680 22664 11.8025 10.5 0.0152849 0.0105689 2020-08-07T15:12:27.401 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:12:27.419 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:27.399+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:27.505 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:27.504+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:27.602 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:27.603 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:12:27.603 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:12:27.603 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:12:27.604 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:27.614 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:27.609+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:27.616 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:12:27.722 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:12:27.723 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:27.714+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:27.767 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:12:27.811 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:27.811 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:12:27.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:12:27.812 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:12:27.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.812 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "MOSDPGPushReply(3.1 62/60 [PushReplyOp(3:8329f37e:::benchmark_data_smithi074_35447_object11915:head)])", 2020-08-07T15:12:27.812 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:12:27.813 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 276.946854105, 2020-08-07T15:12:27.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996, 2020-08-07T15:12:27.813 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:27.813 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:27.813 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:27.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:27.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:12:27.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:27.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:27.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:12:27.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5372000000000001e-05 2020-08-07T15:12:27.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:27.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858416+0000", 2020-08-07T15:12:27.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6899999999999998e-06 2020-08-07T15:12:27.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:27.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858419+0000", 2020-08-07T15:12:27.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63907904299999996 2020-08-07T15:12:27.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:27.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497498+0000", 2020-08-07T15:12:27.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3332999999999999e-05 2020-08-07T15:12:27.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:27.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497512+0000", 2020-08-07T15:12:27.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000450875 2020-08-07T15:12:27.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:27.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497962+0000", 2020-08-07T15:12:27.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000267779 2020-08-07T15:12:27.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:27.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.498230+0000", 2020-08-07T15:12:27.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996 2020-08-07T15:12:27.869 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:27.869 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:27.869 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:27.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.870 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:145792 3.1 3:881e0420:::benchmark_data_smithi074_35447_object16183:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:12:27.870 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:12:27.870 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 253.138448224, 2020-08-07T15:12:27.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997, 2020-08-07T15:12:27.871 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:27.871 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:27.871 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:27.871 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:12:27.872 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:12:27.872 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 145792 2020-08-07T15:12:27.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.882 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:27.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:27.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:12:27.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:27.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:27.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:12:27.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7489000000000001e-05 2020-08-07T15:12:27.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:27.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666824+0000", 2020-08-07T15:12:27.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0169999999999998e-06 2020-08-07T15:12:27.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:27.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666828+0000", 2020-08-07T15:12:27.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9829999999999996e-06 2020-08-07T15:12:27.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:27.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666836+0000", 2020-08-07T15:12:27.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.481e-06 2020-08-07T15:12:27.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:27.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666840+0000", 2020-08-07T15:12:27.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5299000000000001e-05 2020-08-07T15:12:27.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:27.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666885+0000", 2020-08-07T15:12:27.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014456500000000001 2020-08-07T15:12:27.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:27.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667030+0000", 2020-08-07T15:12:27.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017677799999999999 2020-08-07T15:12:27.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,5", 2020-08-07T15:12:27.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667207+0000", 2020-08-07T15:12:27.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004149649 2020-08-07T15:12:27.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:27.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671356+0000", 2020-08-07T15:12:27.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019339649999999999 2020-08-07T15:12:27.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:27.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673290+0000", 2020-08-07T15:12:27.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73346620500000004 2020-08-07T15:12:27.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:27.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406756+0000", 2020-08-07T15:12:27.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4700000000000001e-06 2020-08-07T15:12:27.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:27.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406759+0000", 2020-08-07T15:12:27.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6258999999999998e-05 2020-08-07T15:12:27.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:27.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406815+0000", 2020-08-07T15:12:27.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0126000000000002e-05 2020-08-07T15:12:27.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:27.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406855+0000", 2020-08-07T15:12:27.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997 2020-08-07T15:12:27.917 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:27.917 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:27.917 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:27.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.918 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:145792 3.1 e75/67 ondisk, result = 0)", 2020-08-07T15:12:27.918 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:12:27.918 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 253.134063336, 2020-08-07T15:12:27.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003, 2020-08-07T15:12:27.919 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:27.919 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:27.919 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:27.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:27.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:12:27.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:27.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:27.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:12:27.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2830999999999998e-05 2020-08-07T15:12:27.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:27.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671224+0000", 2020-08-07T15:12:27.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:12:27.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:27.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671229+0000", 2020-08-07T15:12:27.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.735414864 2020-08-07T15:12:27.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:27.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406644+0000", 2020-08-07T15:12:27.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1250000000000004e-06 2020-08-07T15:12:27.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:27.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406651+0000", 2020-08-07T15:12:27.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6567e-05 2020-08-07T15:12:27.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:27.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406737+0000", 2020-08-07T15:12:27.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0054000000000001e-05 2020-08-07T15:12:27.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:27.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406747+0000", 2020-08-07T15:12:27.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011391999999999999 2020-08-07T15:12:27.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:27.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406861+0000", 2020-08-07T15:12:27.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003 2020-08-07T15:12:27.953 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:27.953 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:27.953 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:27.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.962 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30921 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:12:27.962 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:12:27.962 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 221.09931496199999, 2020-08-07T15:12:27.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001, 2020-08-07T15:12:27.963 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:27.963 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:27.963 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:27.963 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:27.963 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:27.964 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30921 2020-08-07T15:12:27.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.964 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:27.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:27.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:12:27.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:27.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:27.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:12:27.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6954999999999998e-05 2020-08-07T15:12:27.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:27.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706016+0000", 2020-08-07T15:12:27.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0229999999999992e-06 2020-08-07T15:12:27.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:27.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706024+0000", 2020-08-07T15:12:27.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.432e-05 2020-08-07T15:12:27.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:27.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:27.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:28.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706039+0000", 2020-08-07T15:12:28.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8490000000000006e-06 2020-08-07T15:12:28.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:28.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706046+0000", 2020-08-07T15:12:28.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6837999999999999e-05 2020-08-07T15:12:28.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706113+0000", 2020-08-07T15:12:28.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3553000000000001e-05 2020-08-07T15:12:28.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:28.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706127+0000", 2020-08-07T15:12:28.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77277201699999998 2020-08-07T15:12:28.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.478899+0000", 2020-08-07T15:12:28.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093978599999999996 2020-08-07T15:12:28.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:12:28.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479839+0000", 2020-08-07T15:12:28.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010376029 2020-08-07T15:12:28.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490215+0000", 2020-08-07T15:12:28.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000190657 2020-08-07T15:12:28.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:28.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490405+0000", 2020-08-07T15:12:28.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001 2020-08-07T15:12:28.034 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.034 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:28.035 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.035 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30922 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:12:28.035 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:12:28.035 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 221.09915479899999, 2020-08-07T15:12:28.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247, 2020-08-07T15:12:28.036 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:28.036 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:28.036 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:28.036 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:28.037 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:28.037 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30922 2020-08-07T15:12:28.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.037 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:28.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:28.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:12:28.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:28.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:28.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:12:28.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9178999999999997e-05 2020-08-07T15:12:28.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:28.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706159+0000", 2020-08-07T15:12:28.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4999999999999999e-06 2020-08-07T15:12:28.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:28.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706164+0000", 2020-08-07T15:12:28.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1792e-05 2020-08-07T15:12:28.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:28.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706176+0000", 2020-08-07T15:12:28.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.139e-06 2020-08-07T15:12:28.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:28.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706182+0000", 2020-08-07T15:12:28.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039724900000000002 2020-08-07T15:12:28.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706579+0000", 2020-08-07T15:12:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1191e-05 2020-08-07T15:12:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706591+0000", 2020-08-07T15:12:28.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329101300000003 2020-08-07T15:12:28.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479882+0000", 2020-08-07T15:12:28.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9565000000000001e-05 2020-08-07T15:12:28.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:12:28.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479971+0000", 2020-08-07T15:12:28.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010471263 2020-08-07T15:12:28.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490442+0000", 2020-08-07T15:12:28.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015035600000000001 2020-08-07T15:12:28.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:28.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490593+0000", 2020-08-07T15:12:28.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247 2020-08-07T15:12:28.093 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.093 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:28.094 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.095 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30923 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:12:28.095 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:12:28.095 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 221.09867973799999, 2020-08-07T15:12:28.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998, 2020-08-07T15:12:28.096 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:28.096 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:28.096 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:28.097 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:28.114 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:28.115 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30923 2020-08-07T15:12:28.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.115 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:28.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:28.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:12:28.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:28.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:28.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:12:28.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5531000000000001e-05 2020-08-07T15:12:28.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:28.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706630+0000", 2020-08-07T15:12:28.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7689999999999999e-06 2020-08-07T15:12:28.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:28.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706637+0000", 2020-08-07T15:12:28.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3577999999999999e-05 2020-08-07T15:12:28.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:28.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706650+0000", 2020-08-07T15:12:28.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1439999999999997e-06 2020-08-07T15:12:28.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:28.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706658+0000", 2020-08-07T15:12:28.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1663999999999999e-05 2020-08-07T15:12:28.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706709+0000", 2020-08-07T15:12:28.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0149000000000001e-05 2020-08-07T15:12:28.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:28.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706719+0000", 2020-08-07T15:12:28.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329694900000001 2020-08-07T15:12:28.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480016+0000", 2020-08-07T15:12:28.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7998999999999997e-05 2020-08-07T15:12:28.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:12:28.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480084+0000", 2020-08-07T15:12:28.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010547333000000001 2020-08-07T15:12:28.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490632+0000", 2020-08-07T15:12:28.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7439e-05 2020-08-07T15:12:28.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:28.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490729+0000", 2020-08-07T15:12:28.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998 2020-08-07T15:12:28.219 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.219 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:28.220 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.221 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30924 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:12:28.221 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:12:28.221 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 221.098552466, 2020-08-07T15:12:28.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995, 2020-08-07T15:12:28.238 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:28.239 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:28.239 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:28.239 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:28.240 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:28.240 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30924 2020-08-07T15:12:28.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.241 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:28.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:28.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:12:28.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:28.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:28.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:12:28.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9698999999999999e-05 2020-08-07T15:12:28.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:28.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706762+0000", 2020-08-07T15:12:28.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8080000000000001e-06 2020-08-07T15:12:28.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:28.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706767+0000", 2020-08-07T15:12:28.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1069e-05 2020-08-07T15:12:28.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:28.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706778+0000", 2020-08-07T15:12:28.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6969999999999998e-06 2020-08-07T15:12:28.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:28.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706784+0000", 2020-08-07T15:12:28.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8792999999999998e-05 2020-08-07T15:12:28.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706843+0000", 2020-08-07T15:12:28.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5449999999999998e-06 2020-08-07T15:12:28.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:28.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706850+0000", 2020-08-07T15:12:28.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.773278354 2020-08-07T15:12:28.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480129+0000", 2020-08-07T15:12:28.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1577999999999997e-05 2020-08-07T15:12:28.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:12:28.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480210+0000", 2020-08-07T15:12:28.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010541915000000001 2020-08-07T15:12:28.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490752+0000", 2020-08-07T15:12:28.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103937 2020-08-07T15:12:28.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:28.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490856+0000", 2020-08-07T15:12:28.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995 2020-08-07T15:12:28.285 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.285 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:28.285 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.286 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30925 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:12:28.286 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:12:28.286 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 221.098428695, 2020-08-07T15:12:28.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995, 2020-08-07T15:12:28.287 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:28.287 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:28.287 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:28.287 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:28.288 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:28.288 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30925 2020-08-07T15:12:28.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.288 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:28.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:28.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:12:28.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:28.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:28.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:12:28.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4664999999999999e-05 2020-08-07T15:12:28.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:28.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706880+0000", 2020-08-07T15:12:28.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7479999999999999e-06 2020-08-07T15:12:28.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:28.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706886+0000", 2020-08-07T15:12:28.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1838000000000001e-05 2020-08-07T15:12:28.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:28.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706898+0000", 2020-08-07T15:12:28.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0089999999999998e-06 2020-08-07T15:12:28.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:28.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706903+0000", 2020-08-07T15:12:28.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3559e-05 2020-08-07T15:12:28.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706956+0000", 2020-08-07T15:12:28.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0309000000000001e-05 2020-08-07T15:12:28.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:28.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706967+0000", 2020-08-07T15:12:28.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77328290499999996 2020-08-07T15:12:28.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480250+0000", 2020-08-07T15:12:28.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8680999999999997e-05 2020-08-07T15:12:28.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:12:28.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480298+0000", 2020-08-07T15:12:28.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01059272 2020-08-07T15:12:28.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490891+0000", 2020-08-07T15:12:28.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014580300000000001 2020-08-07T15:12:28.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:28.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491037+0000", 2020-08-07T15:12:28.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995 2020-08-07T15:12:28.327 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.327 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:28.327 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.342 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30926 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:12:28.343 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:12:28.343 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 221.09831182600001, 2020-08-07T15:12:28.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005, 2020-08-07T15:12:28.344 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:28.344 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:28.345 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:28.345 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:28.345 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:28.346 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30926 2020-08-07T15:12:28.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.346 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:28.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:28.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:12:28.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:28.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:28.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:12:28.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5223000000000003e-05 2020-08-07T15:12:28.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:28.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706998+0000", 2020-08-07T15:12:28.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4809999999999997e-06 2020-08-07T15:12:28.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:28.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707003+0000", 2020-08-07T15:12:28.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1663e-05 2020-08-07T15:12:28.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:28.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707015+0000", 2020-08-07T15:12:28.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9729999999999997e-06 2020-08-07T15:12:28.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:28.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707020+0000", 2020-08-07T15:12:28.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2565999999999999e-05 2020-08-07T15:12:28.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707062+0000", 2020-08-07T15:12:28.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8944e-05 2020-08-07T15:12:28.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:28.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707081+0000", 2020-08-07T15:12:28.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77325885000000005 2020-08-07T15:12:28.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480340+0000", 2020-08-07T15:12:28.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0321000000000006e-05 2020-08-07T15:12:28.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:12:28.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480420+0000", 2020-08-07T15:12:28.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010655958 2020-08-07T15:12:28.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491076+0000", 2020-08-07T15:12:28.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9784999999999995e-05 2020-08-07T15:12:28.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:28.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491166+0000", 2020-08-07T15:12:28.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005 2020-08-07T15:12:28.400 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.400 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:28.401 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.401 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30927 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:12:28.401 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:12:28.402 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 221.09819400699999, 2020-08-07T15:12:28.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999, 2020-08-07T15:12:28.403 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:28.404 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:28.408 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:28.409 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:28.409 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:28.409 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30927 2020-08-07T15:12:28.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.410 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:28.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:28.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:12:28.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:28.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:28.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:12:28.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9861999999999997e-05 2020-08-07T15:12:28.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:28.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707120+0000", 2020-08-07T15:12:28.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8969999999999996e-06 2020-08-07T15:12:28.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:28.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707126+0000", 2020-08-07T15:12:28.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1248e-05 2020-08-07T15:12:28.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:28.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707137+0000", 2020-08-07T15:12:28.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.955e-06 2020-08-07T15:12:28.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:28.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707142+0000", 2020-08-07T15:12:28.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0643999999999998e-05 2020-08-07T15:12:28.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707193+0000", 2020-08-07T15:12:28.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1983e-05 2020-08-07T15:12:28.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:28.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707205+0000", 2020-08-07T15:12:28.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77324914 2020-08-07T15:12:28.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480454+0000", 2020-08-07T15:12:28.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4119999999999998e-05 2020-08-07T15:12:28.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:12:28.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480498+0000", 2020-08-07T15:12:28.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010709707000000001 2020-08-07T15:12:28.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491208+0000", 2020-08-07T15:12:28.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014433800000000001 2020-08-07T15:12:28.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:28.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491352+0000", 2020-08-07T15:12:28.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999 2020-08-07T15:12:28.435 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.435 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:28.435 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.436 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30928 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:12:28.437 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:12:28.437 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 221.09806965000001, 2020-08-07T15:12:28.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998, 2020-08-07T15:12:28.438 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:28.438 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:28.439 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:28.439 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:28.439 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:28.440 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30928 2020-08-07T15:12:28.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.440 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:28.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:28.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:12:28.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:28.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:28.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:12:28.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6582000000000001e-05 2020-08-07T15:12:28.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:28.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707241+0000", 2020-08-07T15:12:28.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9190000000000004e-06 2020-08-07T15:12:28.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:28.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707247+0000", 2020-08-07T15:12:28.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000188822 2020-08-07T15:12:28.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:28.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707436+0000", 2020-08-07T15:12:28.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2940000000000002e-06 2020-08-07T15:12:28.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:28.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707442+0000", 2020-08-07T15:12:28.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7800000000000003e-05 2020-08-07T15:12:28.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707490+0000", 2020-08-07T15:12:28.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2020-08-07T15:12:28.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:28.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707501+0000", 2020-08-07T15:12:28.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77303498800000003 2020-08-07T15:12:28.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480536+0000", 2020-08-07T15:12:28.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7723000000000001e-05 2020-08-07T15:12:28.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:12:28.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480614+0000", 2020-08-07T15:12:28.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010812183 2020-08-07T15:12:28.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491426+0000", 2020-08-07T15:12:28.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5502999999999999e-05 2020-08-07T15:12:28.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:28.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491502+0000", 2020-08-07T15:12:28.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998 2020-08-07T15:12:28.461 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.485 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:28.485 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.486 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31009 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:12:28.486 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:12:28.486 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 221.07411334400001, 2020-08-07T15:12:28.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002, 2020-08-07T15:12:28.487 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:28.487 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:28.487 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:28.487 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:28.488 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:28.488 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31009 2020-08-07T15:12:28.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.488 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:28.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:28.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:12:28.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:28.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:28.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:12:28.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2429000000000002e-05 2020-08-07T15:12:28.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:28.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731193+0000", 2020-08-07T15:12:28.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9760000000000003e-06 2020-08-07T15:12:28.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:28.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731198+0000", 2020-08-07T15:12:28.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3597e-05 2020-08-07T15:12:28.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:28.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731212+0000", 2020-08-07T15:12:28.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1709999999999996e-06 2020-08-07T15:12:28.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:28.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731219+0000", 2020-08-07T15:12:28.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5402999999999997e-05 2020-08-07T15:12:28.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731285+0000", 2020-08-07T15:12:28.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3495e-05 2020-08-07T15:12:28.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:28.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731298+0000", 2020-08-07T15:12:28.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74935423599999995 2020-08-07T15:12:28.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480652+0000", 2020-08-07T15:12:28.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000137806 2020-08-07T15:12:28.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:28.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480790+0000", 2020-08-07T15:12:28.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022272399999999999 2020-08-07T15:12:28.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:12:28.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481013+0000", 2020-08-07T15:12:28.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003223381 2020-08-07T15:12:28.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:28.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484236+0000", 2020-08-07T15:12:28.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1370000000000001e-06 2020-08-07T15:12:28.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:28.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484237+0000", 2020-08-07T15:12:28.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6288e-05 2020-08-07T15:12:28.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:28.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484294+0000", 2020-08-07T15:12:28.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.007260402 2020-08-07T15:12:28.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:28.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491554+0000", 2020-08-07T15:12:28.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7358000000000003e-05 2020-08-07T15:12:28.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:28.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491601+0000", 2020-08-07T15:12:28.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1100000000000004e-07 2020-08-07T15:12:28.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:28.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491602+0000", 2020-08-07T15:12:28.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013436500000000001 2020-08-07T15:12:28.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:28.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491737+0000", 2020-08-07T15:12:28.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002 2020-08-07T15:12:28.552 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.552 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:28.552 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.561 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31010 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:12:28.562 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:12:28.563 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 221.07398739000001, 2020-08-07T15:12:28.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001, 2020-08-07T15:12:28.563 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:28.563 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:28.563 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:28.564 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:28.564 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:28.564 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31010 2020-08-07T15:12:28.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.564 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:28.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:28.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:12:28.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:28.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:28.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:12:28.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2398000000000002e-05 2020-08-07T15:12:28.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:28.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731319+0000", 2020-08-07T15:12:28.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7100000000000004e-06 2020-08-07T15:12:28.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:28.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731325+0000", 2020-08-07T15:12:28.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4328000000000001e-05 2020-08-07T15:12:28.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:28.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731339+0000", 2020-08-07T15:12:28.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3310000000000003e-06 2020-08-07T15:12:28.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:28.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731345+0000", 2020-08-07T15:12:28.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8952999999999998e-05 2020-08-07T15:12:28.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731384+0000", 2020-08-07T15:12:28.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0036e-05 2020-08-07T15:12:28.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:28.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731394+0000", 2020-08-07T15:12:28.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75027681300000004 2020-08-07T15:12:28.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481670+0000", 2020-08-07T15:12:28.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016555899999999999 2020-08-07T15:12:28.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:28.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481836+0000", 2020-08-07T15:12:28.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028760999999999999 2020-08-07T15:12:28.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:12:28.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482124+0000", 2020-08-07T15:12:28.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038342680000000001 2020-08-07T15:12:28.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:28.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485958+0000", 2020-08-07T15:12:28.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2559999999999999e-06 2020-08-07T15:12:28.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:28.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485959+0000", 2020-08-07T15:12:28.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3758999999999995e-05 2020-08-07T15:12:28.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:28.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486043+0000", 2020-08-07T15:12:28.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057590419999999998 2020-08-07T15:12:28.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:28.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491802+0000", 2020-08-07T15:12:28.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5757000000000001e-05 2020-08-07T15:12:28.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:28.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:12:28.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9299999999999997e-07 2020-08-07T15:12:28.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:28.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:12:28.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012496999999999999 2020-08-07T15:12:28.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:28.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491993+0000", 2020-08-07T15:12:28.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001 2020-08-07T15:12:28.583 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.584 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:28.584 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.584 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31011 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:12:28.585 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:12:28.585 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 221.073871185, 2020-08-07T15:12:28.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997, 2020-08-07T15:12:28.585 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:28.585 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:28.585 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:28.586 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:28.586 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:28.586 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31011 2020-08-07T15:12:28.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.586 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:28.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:28.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:12:28.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:28.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:28.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:12:28.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9539999999999997e-05 2020-08-07T15:12:28.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:28.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731433+0000", 2020-08-07T15:12:28.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9209999999999997e-06 2020-08-07T15:12:28.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:28.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731439+0000", 2020-08-07T15:12:28.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4246999999999999e-05 2020-08-07T15:12:28.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:28.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731453+0000", 2020-08-07T15:12:28.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8130000000000003e-06 2020-08-07T15:12:28.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:28.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731458+0000", 2020-08-07T15:12:28.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6668000000000001e-05 2020-08-07T15:12:28.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731504+0000", 2020-08-07T15:12:28.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1112e-05 2020-08-07T15:12:28.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:28.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731515+0000", 2020-08-07T15:12:28.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75121332100000004 2020-08-07T15:12:28.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482729+0000", 2020-08-07T15:12:28.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1810000000000004e-05 2020-08-07T15:12:28.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:28.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482811+0000", 2020-08-07T15:12:28.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018001399999999999 2020-08-07T15:12:28.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:12:28.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482991+0000", 2020-08-07T15:12:28.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030701140000000001 2020-08-07T15:12:28.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:28.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486061+0000", 2020-08-07T15:12:28.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4499999999999996e-07 2020-08-07T15:12:28.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:28.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486062+0000", 2020-08-07T15:12:28.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5366999999999998e-05 2020-08-07T15:12:28.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:28.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486147+0000", 2020-08-07T15:12:28.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056872779999999996 2020-08-07T15:12:28.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:28.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491834+0000", 2020-08-07T15:12:28.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024645100000000002 2020-08-07T15:12:28.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:28.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:12:28.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1200000000000003e-07 2020-08-07T15:12:28.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:28.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:12:28.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1451000000000002e-05 2020-08-07T15:12:28.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:28.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492173+0000", 2020-08-07T15:12:28.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997 2020-08-07T15:12:28.612 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.612 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:28.612 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.613 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31012 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:12:28.614 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:12:28.614 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 221.07321706600001, 2020-08-07T15:12:28.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003, 2020-08-07T15:12:28.615 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:28.615 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:28.615 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:28.615 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:28.616 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:28.616 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31012 2020-08-07T15:12:28.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.617 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:28.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:28.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:12:28.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:28.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:28.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:12:28.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6344999999999997e-05 2020-08-07T15:12:28.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:28.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732094+0000", 2020-08-07T15:12:28.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2920000000000001e-06 2020-08-07T15:12:28.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:28.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732100+0000", 2020-08-07T15:12:28.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.189e-05 2020-08-07T15:12:28.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:28.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732112+0000", 2020-08-07T15:12:28.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3720000000000001e-06 2020-08-07T15:12:28.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:28.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732118+0000", 2020-08-07T15:12:28.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2514000000000004e-05 2020-08-07T15:12:28.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732191+0000", 2020-08-07T15:12:28.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4610000000000001e-05 2020-08-07T15:12:28.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:28.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732205+0000", 2020-08-07T15:12:28.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75132974100000005 2020-08-07T15:12:28.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483535+0000", 2020-08-07T15:12:28.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9185999999999996e-05 2020-08-07T15:12:28.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:28.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483624+0000", 2020-08-07T15:12:28.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015683 2020-08-07T15:12:28.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:12:28.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483781+0000", 2020-08-07T15:12:28.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046202889999999996 2020-08-07T15:12:28.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:28.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488401+0000", 2020-08-07T15:12:28.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6700000000000002e-07 2020-08-07T15:12:28.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:28.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488402+0000", 2020-08-07T15:12:28.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1487000000000001e-05 2020-08-07T15:12:28.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:28.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488484+0000", 2020-08-07T15:12:28.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035520830000000002 2020-08-07T15:12:28.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:28.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492036+0000", 2020-08-07T15:12:28.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029266600000000002 2020-08-07T15:12:28.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:28.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492328+0000", 2020-08-07T15:12:28.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5099999999999998e-07 2020-08-07T15:12:28.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:28.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492329+0000", 2020-08-07T15:12:28.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000201299 2020-08-07T15:12:28.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:28.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492531+0000", 2020-08-07T15:12:28.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003 2020-08-07T15:12:28.644 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.645 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:28.645 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.647 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31013 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:12:28.647 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:12:28.648 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 221.073005435, 2020-08-07T15:12:28.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996, 2020-08-07T15:12:28.649 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:28.650 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:28.650 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:28.651 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:28.651 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:28.652 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31013 2020-08-07T15:12:28.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.652 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:28.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:28.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:12:28.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:28.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:28.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:12:28.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8949000000000002e-05 2020-08-07T15:12:28.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:28.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732298+0000", 2020-08-07T15:12:28.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1399999999999997e-06 2020-08-07T15:12:28.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:28.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732304+0000", 2020-08-07T15:12:28.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.154e-05 2020-08-07T15:12:28.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:28.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732315+0000", 2020-08-07T15:12:28.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1319999999999999e-06 2020-08-07T15:12:28.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:28.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732322+0000", 2020-08-07T15:12:28.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7431e-05 2020-08-07T15:12:28.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732369+0000", 2020-08-07T15:12:28.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0621999999999999e-05 2020-08-07T15:12:28.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:28.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732380+0000", 2020-08-07T15:12:28.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75196845999999995 2020-08-07T15:12:28.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484348+0000", 2020-08-07T15:12:28.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000123008 2020-08-07T15:12:28.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:28.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484471+0000", 2020-08-07T15:12:28.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017572700000000001 2020-08-07T15:12:28.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:12:28.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484647+0000", 2020-08-07T15:12:28.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038557650000000001 2020-08-07T15:12:28.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:28.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:12:28.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2020-08-07T15:12:28.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:28.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:12:28.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0714e-05 2020-08-07T15:12:28.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:28.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488584+0000", 2020-08-07T15:12:28.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036298319999999999 2020-08-07T15:12:28.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:28.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492214+0000", 2020-08-07T15:12:28.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039530099999999997 2020-08-07T15:12:28.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:28.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492609+0000", 2020-08-07T15:12:28.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5300000000000002e-07 2020-08-07T15:12:28.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:28.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492610+0000", 2020-08-07T15:12:28.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197321 2020-08-07T15:12:28.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:28.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492807+0000", 2020-08-07T15:12:28.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996 2020-08-07T15:12:28.683 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.683 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:28.683 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.684 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31014 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:12:28.684 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:12:28.684 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 221.07162329100001, 2020-08-07T15:12:28.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998, 2020-08-07T15:12:28.685 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:28.685 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:28.685 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:28.685 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:28.686 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:28.686 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31014 2020-08-07T15:12:28.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.688 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:28.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:28.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:12:28.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:28.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:28.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:12:28.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3743999999999999e-05 2020-08-07T15:12:28.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:28.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733685+0000", 2020-08-07T15:12:28.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1720000000000007e-06 2020-08-07T15:12:28.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:28.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733693+0000", 2020-08-07T15:12:28.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3674e-05 2020-08-07T15:12:28.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:28.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733707+0000", 2020-08-07T15:12:28.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9249999999999998e-06 2020-08-07T15:12:28.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:28.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733714+0000", 2020-08-07T15:12:28.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8338000000000003e-05 2020-08-07T15:12:28.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733772+0000", 2020-08-07T15:12:28.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3019e-05 2020-08-07T15:12:28.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:28.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733785+0000", 2020-08-07T15:12:28.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75134856299999997 2020-08-07T15:12:28.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485133+0000", 2020-08-07T15:12:28.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8739999999999995e-05 2020-08-07T15:12:28.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:28.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485212+0000", 2020-08-07T15:12:28.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151244 2020-08-07T15:12:28.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:12:28.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485363+0000", 2020-08-07T15:12:28.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032929090000000001 2020-08-07T15:12:28.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:28.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488656+0000", 2020-08-07T15:12:28.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.065e-06 2020-08-07T15:12:28.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:28.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488657+0000", 2020-08-07T15:12:28.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1123000000000003e-05 2020-08-07T15:12:28.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:28.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488729+0000", 2020-08-07T15:12:28.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035508330000000002 2020-08-07T15:12:28.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:28.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492279+0000", 2020-08-07T15:12:28.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00059376000000000003 2020-08-07T15:12:28.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:28.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492873+0000", 2020-08-07T15:12:28.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5700000000000002e-07 2020-08-07T15:12:28.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:28.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492874+0000", 2020-08-07T15:12:28.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019264 2020-08-07T15:12:28.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:28.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493066+0000", 2020-08-07T15:12:28.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998 2020-08-07T15:12:28.722 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.722 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:28.722 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.724 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31015 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:12:28.724 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:12:28.724 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 221.071497325, 2020-08-07T15:12:28.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002, 2020-08-07T15:12:28.725 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:28.725 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:28.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:28.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:28.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:28.727 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31015 2020-08-07T15:12:28.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.727 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:28.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:28.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:12:28.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:28.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:28.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:12:28.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3707000000000001e-05 2020-08-07T15:12:28.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:28.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733791+0000", 2020-08-07T15:12:28.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8120000000000001e-06 2020-08-07T15:12:28.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:28.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733794+0000", 2020-08-07T15:12:28.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0728e-05 2020-08-07T15:12:28.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:28.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733805+0000", 2020-08-07T15:12:28.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8279999999999999e-06 2020-08-07T15:12:28.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:28.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733811+0000", 2020-08-07T15:12:28.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2955999999999999e-05 2020-08-07T15:12:28.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733864+0000", 2020-08-07T15:12:28.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1389000000000001e-05 2020-08-07T15:12:28.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:28.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733875+0000", 2020-08-07T15:12:28.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75232711100000005 2020-08-07T15:12:28.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486202+0000", 2020-08-07T15:12:28.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027503400000000001 2020-08-07T15:12:28.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:28.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486478+0000", 2020-08-07T15:12:28.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033010400000000001 2020-08-07T15:12:28.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:12:28.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486808+0000", 2020-08-07T15:12:28.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019382609999999999 2020-08-07T15:12:28.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:28.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488746+0000", 2020-08-07T15:12:28.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9400000000000004e-07 2020-08-07T15:12:28.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:28.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488747+0000", 2020-08-07T15:12:28.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4324000000000002e-05 2020-08-07T15:12:28.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:28.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488821+0000", 2020-08-07T15:12:28.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043745640000000001 2020-08-07T15:12:28.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:28.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493196+0000", 2020-08-07T15:12:28.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030381000000000001 2020-08-07T15:12:28.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:28.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493499+0000", 2020-08-07T15:12:28.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4e-07 2020-08-07T15:12:28.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:28.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493500+0000", 2020-08-07T15:12:28.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017499 2020-08-07T15:12:28.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:28.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493675+0000", 2020-08-07T15:12:28.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002 2020-08-07T15:12:28.755 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.756 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:28.756 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.757 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31016 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:12:28.758 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:12:28.758 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 221.07090135000001, 2020-08-07T15:12:28.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971, 2020-08-07T15:12:28.759 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:28.759 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:28.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:28.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:28.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:28.767 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31016 2020-08-07T15:12:28.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.768 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:28.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:28.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:12:28.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:28.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:28.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:12:28.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0924000000000002e-05 2020-08-07T15:12:28.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:28.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734404+0000", 2020-08-07T15:12:28.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2020-08-07T15:12:28.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:28.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734410+0000", 2020-08-07T15:12:28.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3459999999999994e-06 2020-08-07T15:12:28.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:28.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734418+0000", 2020-08-07T15:12:28.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9640000000000002e-06 2020-08-07T15:12:28.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:28.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734423+0000", 2020-08-07T15:12:28.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7253999999999999e-05 2020-08-07T15:12:28.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734470+0000", 2020-08-07T15:12:28.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0249999999999997e-06 2020-08-07T15:12:28.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:28.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734477+0000", 2020-08-07T15:12:28.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75294592900000001 2020-08-07T15:12:28.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487423+0000", 2020-08-07T15:12:28.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031604300000000001 2020-08-07T15:12:28.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:28.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487739+0000", 2020-08-07T15:12:28.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000171149 2020-08-07T15:12:28.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:12:28.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487910+0000", 2020-08-07T15:12:28.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024207769999999998 2020-08-07T15:12:28.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:28.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490331+0000", 2020-08-07T15:12:28.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1650000000000001e-06 2020-08-07T15:12:28.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:28.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490332+0000", 2020-08-07T15:12:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9796999999999996e-05 2020-08-07T15:12:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490402+0000", 2020-08-07T15:12:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029928849999999998 2020-08-07T15:12:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493395+0000", 2020-08-07T15:12:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034903199999999999 2020-08-07T15:12:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493744+0000", 2020-08-07T15:12:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8899999999999999e-07 2020-08-07T15:12:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493745+0000", 2020-08-07T15:12:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197362 2020-08-07T15:12:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493942+0000", 2020-08-07T15:12:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971 2020-08-07T15:12:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.803 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:28.804 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.804 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:89632 5.f e207/164 5:fd260020:::benchmark_data_smithi074_53697_object11203:head v 207'5744, mlcod=207'5734)", 2020-08-07T15:12:28.805 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:21.551762+0000", 2020-08-07T15:12:28.805 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.253492492999996, 2020-08-07T15:12:28.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67527211499999995, 2020-08-07T15:12:28.806 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:28.806 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:28.806 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:28.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:28.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551762+0000", 2020-08-07T15:12:28.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:28.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:28.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551762+0000", 2020-08-07T15:12:28.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8560000000000003e-05 2020-08-07T15:12:28.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:28.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551810+0000", 2020-08-07T15:12:28.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9430000000000002e-06 2020-08-07T15:12:28.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:28.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551814+0000", 2020-08-07T15:12:28.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67230272000000002 2020-08-07T15:12:28.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:28.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.224117+0000", 2020-08-07T15:12:28.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7120000000000003e-06 2020-08-07T15:12:28.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:28.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.224124+0000", 2020-08-07T15:12:28.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001719307 2020-08-07T15:12:28.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:28.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.225843+0000", 2020-08-07T15:12:28.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3902999999999997e-05 2020-08-07T15:12:28.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:28.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.225877+0000", 2020-08-07T15:12:28.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001110279 2020-08-07T15:12:28.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:28.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.226987+0000", 2020-08-07T15:12:28.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6690999999999997e-05 2020-08-07T15:12:28.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:28.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:28.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:28.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.227034+0000", 2020-08-07T15:12:28.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67527211499999995 2020-08-07T15:12:28.833 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.833 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:28.833 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.834 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:28.834 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:28.835 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:28.845 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:12:28.846 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:12:28.846 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:12:28.847 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:12:28.848 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:12:28.848 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:12:28.849 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:12:28.851 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2020-08-07T15:12:27.217052+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":174563328,"num_objects":2667,"num_object_clones":0,"num_object_copies":8001,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21309,"num_write_kb":170472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":149,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":198,"ondisk_log_size":198,"up":165,"acting":165,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8889524,"kb_used_data":500852,"kb_used_omap":2,"kb_used_meta":8388605,"kb_avail":746085196,"statfs":{"total":773094113280,"available":763991240704,"internally_reserved":8589934592,"allocated":512872448,"data_stored":503800881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":8589932013},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[12,3,1,2,2,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":110452736,"num_objects":1685,"num_object_clones":0,"num_object_copies":5055,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13483,"num_write_kb":107864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.535352"},"pg_stats":[{"pgid":"1.1e","version":"16'2","reported_seq":"91","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.227487+0000","last_change":"2020-08-07T15:11:37.457186+0000","last_active":"2020-08-07T15:12:22.227487+0000","last_peered":"2020-08-07T15:12:22.227487+0000","last_clean":"2020-08-07T15:12:22.227487+0000","last_became_active":"2020-08-07T15:10:51.559894+0000","last_became_peered":"2020-08-07T15:10:51.559894+0000","last_unstale":"2020-08-07T15:12:22.227487+0000","last_undegraded":"2020-08-07T15:12:22.227487+0000","last_fullsized":"2020-08-07T15:12:22.227487+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.457111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.457111+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1d","version":"75'3","reported_seq":"101","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.226420+0000","last_change":"2020-08-07T15:12:07.357104+0000","last_active":"2020-08-07T15:12:22.226420+0000","last_peered":"2020-08-07T15:12:22.226420+0000","last_clean":"2020-08-07T15:12:22.226420+0000","last_became_active":"2020-08-07T15:12:07.356691+0000","last_became_peered":"2020-08-07T15:12:07.356691+0000","last_unstale":"2020-08-07T15:12:22.226420+0000","last_undegraded":"2020-08-07T15:12:22.226420+0000","last_fullsized":"2020-08-07T15:12:22.226420+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:10:10.962480+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:10:10.962480+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":149,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1c","version":"16'2","reported_seq":"179","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.227960+0000","last_change":"2020-08-07T15:12:07.262797+0000","last_active":"2020-08-07T15:12:22.227960+0000","last_peered":"2020-08-07T15:12:22.227960+0000","last_clean":"2020-08-07T15:12:22.227960+0000","last_became_active":"2020-08-07T15:12:07.245731+0000","last_became_peered":"2020-08-07T15:12:07.245731+0000","last_unstale":"2020-08-07T15:12:22.227960+0000","last_undegraded":"2020-08-07T15:12:22.227960+0000","last_fullsized":"2020-08-07T15:12:22.227960+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:39.738160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:39.738160+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1b","version":"16'2","reported_seq":"212","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.222085+0000","last_change":"2020-08-07T15:10:57.797029+0000","last_active":"2020-08-07T15:12:22.222085+0000","last_peered":"2020-08-07T15:12:22.222085+0000","last_clean":"2020-08-07T15:12:22.222085+0000","last_became_active":"2020-08-07T15:10:51.557392+0000","last_became_peered":"2020-08-07T15:10:51.557392+0000","last_unstale":"2020-08-07T15:12:22.222085+0000","last_undegraded":"2020-08-07T15:12:22.222085+0000","last_fullsized":"2020-08-07T15:12:22.222085+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:57.796946+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:57.796946+0000","last_clean_scrub_stamp":"2020-08-07T15:10:57.796946+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1a","version":"16'2","reported_seq":"76","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.227854+0000","last_change":"2020-08-07T15:12:07.262337+0000","last_active":"2020-08-07T15:12:22.227854+0000","last_peered":"2020-08-07T15:12:22.227854+0000","last_clean":"2020-08-07T15:12:22.227854+0000","last_became_active":"2020-08-07T15:12:07.243542+0000","last_became_peered":"2020-08-07T15:12:07.243542+0000","last_unstale":"2020-08-07T15:12:22.227854+0000","last_undegraded":"2020-08-07T15:12:22.227854+0000","last_fullsized":"2020-08-07T15:12:22.227854+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:11.779828+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:11:11.779828+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.19","version":"16'2","reported_seq":"137","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.227033+0000","last_change":"2020-08-07T15:12:15.372999+0000","last_active":"2020-08-07T15:12:22.227033+0000","last_peered":"2020-08-07T15:12:22.227033+0000","last_clean":"2020-08-07T15:12:22.227033+0000","last_became_active":"2020-08-07T15:10:51.559752+0000","last_became_peered":"2020-08-07T15:10:51.559752+0000","last_unstale":"2020-08-07T15:12:22.227033+0000","last_undegraded":"2020-08-07T15:12:22.227033+0000","last_fullsized":"2020-08-07T15:12:22.227033+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:15.372910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:15.372910+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.18","version":"16'2","reported_seq":"237","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.222012+0000","last_change":"2020-08-07T15:11:06.955336+0000","last_active":"2020-08-07T15:12:22.222012+0000","last_peered":"2020-08-07T15:12:22.222012+0000","last_clean":"2020-08-07T15:12:22.222012+0000","last_became_active":"2020-08-07T15:10:51.557665+0000","last_became_peered":"2020-08-07T15:10:51.557665+0000","last_unstale":"2020-08-07T15:12:22.222012+0000","last_undegraded":"2020-08-07T15:12:22.222012+0000","last_fullsized":"2020-08-07T15:12:22.222012+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:06.955262+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:06.955262+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.17","version":"16'2","reported_seq":"65","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.222675+0000","last_change":"2020-08-07T15:11:08.916219+0000","last_active":"2020-08-07T15:12:22.222675+0000","last_peered":"2020-08-07T15:12:22.222675+0000","last_clean":"2020-08-07T15:12:22.222675+0000","last_became_active":"2020-08-07T15:10:51.561769+0000","last_became_peered":"2020-08-07T15:10:51.561769+0000","last_unstale":"2020-08-07T15:12:22.222675+0000","last_undegraded":"2020-08-07T15:12:22.222675+0000","last_fullsized":"2020-08-07T15:12:22.222675+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:08.916169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:08.916169+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.16","version":"16'2","reported_seq":"42","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.230331+0000","last_change":"2020-08-07T15:12:12.883619+0000","last_active":"2020-08-07T15:12:22.230331+0000","last_peered":"2020-08-07T15:12:22.230331+0000","last_clean":"2020-08-07T15:12:22.230331+0000","last_became_active":"2020-08-07T15:12:07.231877+0000","last_became_peered":"2020-08-07T15:12:07.231877+0000","last_unstale":"2020-08-07T15:12:22.230331+0000","last_undegraded":"2020-08-07T15:12:22.230331+0000","last_fullsized":"2020-08-07T15:12:22.230331+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:12.883558+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:12.883558+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.883558+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.15","version":"75'3","reported_seq":"76","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.228151+0000","last_change":"2020-08-07T15:12:07.284341+0000","last_active":"2020-08-07T15:12:22.228151+0000","last_peered":"2020-08-07T15:12:22.228151+0000","last_clean":"2020-08-07T15:12:22.228151+0000","last_became_active":"2020-08-07T15:12:07.276899+0000","last_became_peered":"2020-08-07T15:12:07.276899+0000","last_unstale":"2020-08-07T15:12:22.228151+0000","last_undegraded":"2020-08-07T15:12:22.228151+0000","last_fullsized":"2020-08-07T15:12:22.228151+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:11:20.562710+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:11:20.562710+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.14","version":"16'2","reported_seq":"175","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.232748+0000","last_change":"2020-08-07T15:12:09.224231+0000","last_active":"2020-08-07T15:12:22.232748+0000","last_peered":"2020-08-07T15:12:22.232748+0000","last_clean":"2020-08-07T15:12:22.232748+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:12:22.232748+0000","last_undegraded":"2020-08-07T15:12:22.232748+0000","last_fullsized":"2020-08-07T15:12:22.232748+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:09.224158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.224158+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"190","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.226234+0000","last_change":"2020-08-07T15:12:19.577926+0000","last_active":"2020-08-07T15:12:22.226234+0000","last_peered":"2020-08-07T15:12:22.226234+0000","last_clean":"2020-08-07T15:12:22.226234+0000","last_became_active":"2020-08-07T15:12:06.375861+0000","last_became_peered":"2020-08-07T15:12:06.375861+0000","last_unstale":"2020-08-07T15:12:22.226234+0000","last_undegraded":"2020-08-07T15:12:22.226234+0000","last_fullsized":"2020-08-07T15:12:22.226234+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:07.887356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.887356+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"190","reported_epoch":"240","state":"active+clean","last_fresh":"2020-08-07T15:12:26.104526+0000","last_change":"2020-08-07T15:12:26.104598+0000","last_active":"2020-08-07T15:12:26.104526+0000","last_peered":"2020-08-07T15:12:26.104526+0000","last_clean":"2020-08-07T15:12:26.104526+0000","last_became_active":"2020-08-07T15:12:06.367362+0000","last_became_peered":"2020-08-07T15:12:06.367362+0000","last_unstale":"2020-08-07T15:12:26.104526+0000","last_undegraded":"2020-08-07T15:12:26.104526+0000","last_fullsized":"2020-08-07T15:12:26.104526+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:26.104526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:26.104526+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"288","reported_epoch":"240","state":"active+clean","last_fresh":"2020-08-07T15:12:24.105531+0000","last_change":"2020-08-07T15:12:24.105593+0000","last_active":"2020-08-07T15:12:24.105531+0000","last_peered":"2020-08-07T15:12:24.105531+0000","last_clean":"2020-08-07T15:12:24.105531+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:12:24.105531+0000","last_undegraded":"2020-08-07T15:12:24.105531+0000","last_fullsized":"2020-08-07T15:12:24.105531+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:24.105531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:24.105531+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"282","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.222196+0000","last_change":"2020-08-07T15:12:13.231819+0000","last_active":"2020-08-07T15:12:22.222196+0000","last_peered":"2020-08-07T15:12:22.222196+0000","last_clean":"2020-08-07T15:12:22.222196+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:12:22.222196+0000","last_undegraded":"2020-08-07T15:12:22.222196+0000","last_fullsized":"2020-08-07T15:12:22.222196+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:13.231762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.231762+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3","version":"240'1224","reported_seq":"1242","reported_epoch":"240","state":"active+clean","last_fresh":"2020-08-07T15:12:26.398995+0000","last_change":"2020-08-07T15:12:10.464875+0000","last_active":"2020-08-07T15:12:26.398995+0000","last_peered":"2020-08-07T15:12:26.398995+0000","last_clean":"2020-08-07T15:12:26.398995+0000","last_became_active":"2020-08-07T15:12:10.464346+0000","last_became_peered":"2020-08-07T15:12:10.464346+0000","last_unstale":"2020-08-07T15:12:26.398995+0000","last_undegraded":"2020-08-07T15:12:26.398995+0000","last_fullsized":"2020-08-07T15:12:26.398995+0000","mapping_epoch":230,"log_start":"240'1216","ondisk_log_start":"240'1216","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":10027008,"num_objects":153,"num_object_clones":0,"num_object_copies":459,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1224,"num_write_kb":9792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"238","reported_epoch":"238","state":"active+clean","last_fresh":"2020-08-07T15:12:20.264786+0000","last_change":"2020-08-07T15:10:57.384596+0000","last_active":"2020-08-07T15:12:20.264786+0000","last_peered":"2020-08-07T15:12:20.264786+0000","last_clean":"2020-08-07T15:12:20.264786+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:12:20.264786+0000","last_undegraded":"2020-08-07T15:12:20.264786+0000","last_fullsized":"2020-08-07T15:12:20.264786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:10:57.384515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:10:57.384515+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"158","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.227674+0000","last_change":"2020-08-07T15:12:10.265465+0000","last_active":"2020-08-07T15:12:22.227674+0000","last_peered":"2020-08-07T15:12:22.227674+0000","last_clean":"2020-08-07T15:12:22.227674+0000","last_became_active":"2020-08-07T15:09:30.283176+0000","last_became_peered":"2020-08-07T15:09:30.283176+0000","last_unstale":"2020-08-07T15:12:22.227674+0000","last_undegraded":"2020-08-07T15:12:22.227674+0000","last_fullsized":"2020-08-07T15:12:22.227674+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:10.265395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:10.265395+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"240'1312","reported_seq":"1330","reported_epoch":"240","state":"active+clean","last_fresh":"2020-08-07T15:12:26.487719+0000","last_change":"2020-08-07T15:12:10.465048+0000","last_active":"2020-08-07T15:12:26.487719+0000","last_peered":"2020-08-07T15:12:26.487719+0000","last_clean":"2020-08-07T15:12:26.487719+0000","last_became_active":"2020-08-07T15:12:10.464493+0000","last_became_peered":"2020-08-07T15:12:10.464493+0000","last_unstale":"2020-08-07T15:12:26.487719+0000","last_undegraded":"2020-08-07T15:12:26.487719+0000","last_fullsized":"2020-08-07T15:12:26.487719+0000","mapping_epoch":230,"log_start":"240'1304","ondisk_log_start":"240'1304","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":10747904,"num_objects":164,"num_object_clones":0,"num_object_copies":492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1312,"num_write_kb":10496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"198","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.230241+0000","last_change":"2020-08-07T15:12:19.561477+0000","last_active":"2020-08-07T15:12:22.230241+0000","last_peered":"2020-08-07T15:12:22.230241+0000","last_clean":"2020-08-07T15:12:22.230241+0000","last_became_active":"2020-08-07T15:12:06.367376+0000","last_became_peered":"2020-08-07T15:12:06.367376+0000","last_unstale":"2020-08-07T15:12:22.230241+0000","last_undegraded":"2020-08-07T15:12:22.230241+0000","last_fullsized":"2020-08-07T15:12:22.230241+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:12.319383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.319383+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"212","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.232869+0000","last_change":"2020-08-07T15:12:12.546009+0000","last_active":"2020-08-07T15:12:22.232869+0000","last_peered":"2020-08-07T15:12:22.232869+0000","last_clean":"2020-08-07T15:12:22.232869+0000","last_became_active":"2020-08-07T15:10:46.501793+0000","last_became_peered":"2020-08-07T15:10:46.501793+0000","last_unstale":"2020-08-07T15:12:22.232869+0000","last_undegraded":"2020-08-07T15:12:22.232869+0000","last_fullsized":"2020-08-07T15:12:22.232869+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:12.545945+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.545945+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"240'1160","reported_seq":"1177","reported_epoch":"240","state":"active+clean","last_fresh":"2020-08-07T15:12:24.808303+0000","last_change":"2020-08-07T15:12:10.464412+0000","last_active":"2020-08-07T15:12:24.808303+0000","last_peered":"2020-08-07T15:12:24.808303+0000","last_clean":"2020-08-07T15:12:24.808303+0000","last_became_active":"2020-08-07T15:12:10.463950+0000","last_became_peered":"2020-08-07T15:12:10.463950+0000","last_unstale":"2020-08-07T15:12:24.808303+0000","last_undegraded":"2020-08-07T15:12:24.808303+0000","last_fullsized":"2020-08-07T15:12:24.808303+0000","mapping_epoch":230,"log_start":"240'1153","ondisk_log_start":"240'1153","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":9502720,"num_objects":145,"num_object_clones":0,"num_object_copies":435,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1160,"num_write_kb":9280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"240","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.228366+0000","last_change":"2020-08-07T15:11:55.731391+0000","last_active":"2020-08-07T15:12:22.228366+0000","last_peered":"2020-08-07T15:12:22.228366+0000","last_clean":"2020-08-07T15:12:22.228366+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:12:22.228366+0000","last_undegraded":"2020-08-07T15:12:22.228366+0000","last_fullsized":"2020-08-07T15:12:22.228366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:11:55.731304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:11:55.731304+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"289","reported_epoch":"238","state":"active+clean","last_fresh":"2020-08-07T15:12:20.264480+0000","last_change":"2020-08-07T15:12:14.121460+0000","last_active":"2020-08-07T15:12:20.264480+0000","last_peered":"2020-08-07T15:12:20.264480+0000","last_clean":"2020-08-07T15:12:20.264480+0000","last_became_active":"2020-08-07T15:10:46.516484+0000","last_became_peered":"2020-08-07T15:10:46.516484+0000","last_unstale":"2020-08-07T15:12:20.264480+0000","last_undegraded":"2020-08-07T15:12:20.264480+0000","last_fullsized":"2020-08-07T15:12:20.264480+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_clean_scrub_stamp":"2020-08-07T15:12:14.121397+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"240'1208","reported_seq":"1226","reported_epoch":"240","state":"active+clean","last_fresh":"2020-08-07T15:12:24.828360+0000","last_change":"2020-08-07T15:12:10.398415+0000","last_active":"2020-08-07T15:12:24.828360+0000","last_peered":"2020-08-07T15:12:24.828360+0000","last_clean":"2020-08-07T15:12:24.828360+0000","last_became_active":"2020-08-07T15:12:10.397834+0000","last_became_peered":"2020-08-07T15:12:10.397834+0000","last_unstale":"2020-08-07T15:12:24.828360+0000","last_undegraded":"2020-08-07T15:12:24.828360+0000","last_fullsized":"2020-08-07T15:12:24.828360+0000","mapping_epoch":230,"log_start":"240'1199","ondisk_log_start":"240'1199","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":9895936,"num_objects":151,"num_object_clones":0,"num_object_copies":453,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1208,"num_write_kb":9664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"255","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.230105+0000","last_change":"2020-08-07T15:12:08.622779+0000","last_active":"2020-08-07T15:12:22.230105+0000","last_peered":"2020-08-07T15:12:22.230105+0000","last_clean":"2020-08-07T15:12:22.230105+0000","last_became_active":"2020-08-07T15:12:06.367457+0000","last_became_peered":"2020-08-07T15:12:06.367457+0000","last_unstale":"2020-08-07T15:12:22.230105+0000","last_undegraded":"2020-08-07T15:12:22.230105+0000","last_fullsized":"2020-08-07T15:12:22.230105+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:08.622683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:08.622683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"258","reported_epoch":"238","state":"active+clean","last_fresh":"2020-08-07T15:12:20.264094+0000","last_change":"2020-08-07T15:12:10.175824+0000","last_active":"2020-08-07T15:12:20.264094+0000","last_peered":"2020-08-07T15:12:20.264094+0000","last_clean":"2020-08-07T15:12:20.264094+0000","last_became_active":"2020-08-07T15:12:06.366891+0000","last_became_peered":"2020-08-07T15:12:06.366891+0000","last_unstale":"2020-08-07T15:12:20.264094+0000","last_undegraded":"2020-08-07T15:12:20.264094+0000","last_fullsized":"2020-08-07T15:12:20.264094+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:10.175739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:10.175739+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.7","version":"240'1456","reported_seq":"1474","reported_epoch":"240","state":"active+clean","last_fresh":"2020-08-07T15:12:26.462267+0000","last_change":"2020-08-07T15:12:10.464810+0000","last_active":"2020-08-07T15:12:26.462267+0000","last_peered":"2020-08-07T15:12:26.462267+0000","last_clean":"2020-08-07T15:12:26.462267+0000","last_became_active":"2020-08-07T15:12:10.464050+0000","last_became_peered":"2020-08-07T15:12:10.464050+0000","last_unstale":"2020-08-07T15:12:26.462267+0000","last_undegraded":"2020-08-07T15:12:26.462267+0000","last_fullsized":"2020-08-07T15:12:26.462267+0000","mapping_epoch":230,"log_start":"240'1442","ondisk_log_start":"240'1442","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11927552,"num_objects":182,"num_object_clones":0,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1456,"num_write_kb":11648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"272","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.229993+0000","last_change":"2020-08-07T15:11:40.696919+0000","last_active":"2020-08-07T15:12:22.229993+0000","last_peered":"2020-08-07T15:12:22.229993+0000","last_clean":"2020-08-07T15:12:22.229993+0000","last_became_active":"2020-08-07T15:08:36.627426+0000","last_became_peered":"2020-08-07T15:08:36.627426+0000","last_unstale":"2020-08-07T15:12:22.229993+0000","last_undegraded":"2020-08-07T15:12:22.229993+0000","last_fullsized":"2020-08-07T15:12:22.229993+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:40.696862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:40.696862+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"240","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.222323+0000","last_change":"2020-08-07T15:11:40.328534+0000","last_active":"2020-08-07T15:12:22.222323+0000","last_peered":"2020-08-07T15:12:22.222323+0000","last_clean":"2020-08-07T15:12:22.222323+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:12:22.222323+0000","last_undegraded":"2020-08-07T15:12:22.222323+0000","last_fullsized":"2020-08-07T15:12:22.222323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:11:40.328482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:11:40.328482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"240'1352","reported_seq":"1370","reported_epoch":"240","state":"active+clean","last_fresh":"2020-08-07T15:12:26.446271+0000","last_change":"2020-08-07T15:12:10.464478+0000","last_active":"2020-08-07T15:12:26.446271+0000","last_peered":"2020-08-07T15:12:26.446271+0000","last_clean":"2020-08-07T15:12:26.446271+0000","last_became_active":"2020-08-07T15:12:10.463997+0000","last_became_peered":"2020-08-07T15:12:10.463997+0000","last_unstale":"2020-08-07T15:12:26.446271+0000","last_undegraded":"2020-08-07T15:12:26.446271+0000","last_fullsized":"2020-08-07T15:12:26.446271+0000","mapping_epoch":230,"log_start":"240'1348","ondisk_log_start":"240'1348","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11075584,"num_objects":169,"num_object_clones":0,"num_object_copies":507,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1352,"num_write_kb":10816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"261","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.232384+0000","last_change":"2020-08-07T15:12:10.585667+0000","last_active":"2020-08-07T15:12:22.232384+0000","last_peered":"2020-08-07T15:12:22.232384+0000","last_clean":"2020-08-07T15:12:22.232384+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:12:22.232384+0000","last_undegraded":"2020-08-07T15:12:22.232384+0000","last_fullsized":"2020-08-07T15:12:22.232384+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:10.585580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:10.585580+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"288","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.227970+0000","last_change":"2020-08-07T15:12:11.658986+0000","last_active":"2020-08-07T15:12:22.227970+0000","last_peered":"2020-08-07T15:12:22.227970+0000","last_clean":"2020-08-07T15:12:22.227970+0000","last_became_active":"2020-08-07T15:08:36.628365+0000","last_became_peered":"2020-08-07T15:08:36.628365+0000","last_unstale":"2020-08-07T15:12:22.227970+0000","last_undegraded":"2020-08-07T15:12:22.227970+0000","last_fullsized":"2020-08-07T15:12:22.227970+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:11.658923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:11.658923+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"240'1568","reported_seq":"1586","reported_epoch":"240","state":"active+clean","last_fresh":"2020-08-07T15:12:26.423689+0000","last_change":"2020-08-07T15:12:10.465113+0000","last_active":"2020-08-07T15:12:26.423689+0000","last_peered":"2020-08-07T15:12:26.423689+0000","last_clean":"2020-08-07T15:12:26.423689+0000","last_became_active":"2020-08-07T15:12:10.464582+0000","last_became_peered":"2020-08-07T15:12:10.464582+0000","last_unstale":"2020-08-07T15:12:26.423689+0000","last_undegraded":"2020-08-07T15:12:26.423689+0000","last_fullsized":"2020-08-07T15:12:26.423689+0000","mapping_epoch":230,"log_start":"240'1560","ondisk_log_start":"240'1560","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":12845056,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1568,"num_write_kb":12544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"255","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.222534+0000","last_change":"2020-08-07T15:12:07.340958+0000","last_active":"2020-08-07T15:12:22.222534+0000","last_peered":"2020-08-07T15:12:22.222534+0000","last_clean":"2020-08-07T15:12:22.222534+0000","last_became_active":"2020-08-07T15:12:06.366967+0000","last_became_peered":"2020-08-07T15:12:06.366967+0000","last_unstale":"2020-08-07T15:12:22.222534+0000","last_undegraded":"2020-08-07T15:12:22.222534+0000","last_fullsized":"2020-08-07T15:12:22.222534+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:07.340855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.340855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"238","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.222137+0000","last_change":"2020-08-07T15:12:09.261178+0000","last_active":"2020-08-07T15:12:22.222137+0000","last_peered":"2020-08-07T15:12:22.222137+0000","last_clean":"2020-08-07T15:12:22.222137+0000","last_became_active":"2020-08-07T15:08:36.618164+0000","last_became_peered":"2020-08-07T15:08:36.618164+0000","last_unstale":"2020-08-07T15:12:22.222137+0000","last_undegraded":"2020-08-07T15:12:22.222137+0000","last_fullsized":"2020-08-07T15:12:22.222137+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:09.261101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.261101+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"240'1304","reported_seq":"1322","reported_epoch":"240","state":"active+clean","last_fresh":"2020-08-07T15:12:26.678998+0000","last_change":"2020-08-07T15:12:10.466261+0000","last_active":"2020-08-07T15:12:26.678998+0000","last_peered":"2020-08-07T15:12:26.678998+0000","last_clean":"2020-08-07T15:12:26.678998+0000","last_became_active":"2020-08-07T15:12:10.465583+0000","last_became_peered":"2020-08-07T15:12:10.465583+0000","last_unstale":"2020-08-07T15:12:26.678998+0000","last_undegraded":"2020-08-07T15:12:26.678998+0000","last_fullsized":"2020-08-07T15:12:26.678998+0000","mapping_epoch":230,"log_start":"240'1296","ondisk_log_start":"240'1296","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":10682368,"num_objects":163,"num_object_clones":0,"num_object_copies":489,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1304,"num_write_kb":10432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"252","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.228153+0000","last_change":"2020-08-07T15:12:07.583222+0000","last_active":"2020-08-07T15:12:22.228153+0000","last_peered":"2020-08-07T15:12:22.228153+0000","last_clean":"2020-08-07T15:12:22.228153+0000","last_became_active":"2020-08-07T15:12:06.366915+0000","last_became_peered":"2020-08-07T15:12:06.366915+0000","last_unstale":"2020-08-07T15:12:22.228153+0000","last_undegraded":"2020-08-07T15:12:22.228153+0000","last_fullsized":"2020-08-07T15:12:22.228153+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:07.583152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.583152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"323","reported_epoch":"238","state":"active+clean","last_fresh":"2020-08-07T15:12:20.264885+0000","last_change":"2020-08-07T15:12:15.163743+0000","last_active":"2020-08-07T15:12:20.264885+0000","last_peered":"2020-08-07T15:12:20.264885+0000","last_clean":"2020-08-07T15:12:20.264885+0000","last_became_active":"2020-08-07T15:10:46.496544+0000","last_became_peered":"2020-08-07T15:10:46.496544+0000","last_unstale":"2020-08-07T15:12:20.264885+0000","last_undegraded":"2020-08-07T15:12:20.264885+0000","last_fullsized":"2020-08-07T15:12:20.264885+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:15.163639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:15.163639+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"240'1328","reported_seq":"1345","reported_epoch":"240","state":"active+clean","last_fresh":"2020-08-07T15:12:26.699985+0000","last_change":"2020-08-07T15:12:10.467447+0000","last_active":"2020-08-07T15:12:26.699985+0000","last_peered":"2020-08-07T15:12:26.699985+0000","last_clean":"2020-08-07T15:12:26.699985+0000","last_became_active":"2020-08-07T15:12:10.467000+0000","last_became_peered":"2020-08-07T15:12:10.467000+0000","last_unstale":"2020-08-07T15:12:26.699985+0000","last_undegraded":"2020-08-07T15:12:26.699985+0000","last_fullsized":"2020-08-07T15:12:26.699985+0000","mapping_epoch":230,"log_start":"240'1321","ondisk_log_start":"240'1321","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":10878976,"num_objects":166,"num_object_clones":0,"num_object_copies":498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1328,"num_write_kb":10624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"224","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.222448+0000","last_change":"2020-08-07T15:10:51.644951+0000","last_active":"2020-08-07T15:12:22.222448+0000","last_peered":"2020-08-07T15:12:22.222448+0000","last_clean":"2020-08-07T15:12:22.222448+0000","last_became_active":"2020-08-07T15:10:46.498230+0000","last_became_peered":"2020-08-07T15:10:46.498230+0000","last_unstale":"2020-08-07T15:12:22.222448+0000","last_undegraded":"2020-08-07T15:12:22.222448+0000","last_fullsized":"2020-08-07T15:12:22.222448+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:51.644875+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:10:51.644875+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"240'1416","reported_seq":"1433","reported_epoch":"240","state":"active+clean","last_fresh":"2020-08-07T15:12:26.733198+0000","last_change":"2020-08-07T15:12:10.465075+0000","last_active":"2020-08-07T15:12:26.733198+0000","last_peered":"2020-08-07T15:12:26.733198+0000","last_clean":"2020-08-07T15:12:26.733198+0000","last_became_active":"2020-08-07T15:12:10.464640+0000","last_became_peered":"2020-08-07T15:12:10.464640+0000","last_unstale":"2020-08-07T15:12:26.733198+0000","last_undegraded":"2020-08-07T15:12:26.733198+0000","last_fullsized":"2020-08-07T15:12:26.733198+0000","mapping_epoch":230,"log_start":"240'1408","ondisk_log_start":"240'1408","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11599872,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1416,"num_write_kb":11328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"64","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.233343+0000","last_change":"2020-08-07T15:11:37.490719+0000","last_active":"2020-08-07T15:12:22.233343+0000","last_peered":"2020-08-07T15:12:22.233343+0000","last_clean":"2020-08-07T15:12:22.233343+0000","last_became_active":"2020-08-07T15:11:23.232406+0000","last_became_peered":"2020-08-07T15:11:23.232406+0000","last_unstale":"2020-08-07T15:12:22.233343+0000","last_undegraded":"2020-08-07T15:12:22.233343+0000","last_fullsized":"2020-08-07T15:12:22.233343+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.490655+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.490655+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"240'1168","reported_seq":"1186","reported_epoch":"240","state":"active+clean","last_fresh":"2020-08-07T15:12:24.769568+0000","last_change":"2020-08-07T15:12:10.465359+0000","last_active":"2020-08-07T15:12:24.769568+0000","last_peered":"2020-08-07T15:12:24.769568+0000","last_clean":"2020-08-07T15:12:24.769568+0000","last_became_active":"2020-08-07T15:12:10.464900+0000","last_became_peered":"2020-08-07T15:12:10.464900+0000","last_unstale":"2020-08-07T15:12:24.769568+0000","last_undegraded":"2020-08-07T15:12:24.769568+0000","last_fullsized":"2020-08-07T15:12:24.769568+0000","mapping_epoch":230,"log_start":"240'1159","ondisk_log_start":"240'1159","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":9568256,"num_objects":146,"num_object_clones":0,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1168,"num_write_kb":9344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"291","reported_epoch":"240","state":"active+clean","last_fresh":"2020-08-07T15:12:23.886262+0000","last_change":"2020-08-07T15:12:23.886345+0000","last_active":"2020-08-07T15:12:23.886262+0000","last_peered":"2020-08-07T15:12:23.886262+0000","last_clean":"2020-08-07T15:12:23.886262+0000","last_became_active":"2020-08-07T15:10:46.499635+0000","last_became_peered":"2020-08-07T15:10:46.499635+0000","last_unstale":"2020-08-07T15:12:23.886262+0000","last_undegraded":"2020-08-07T15:12:23.886262+0000","last_fullsized":"2020-08-07T15:12:23.886262+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:23.886261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:23.886261+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"240'1304","reported_seq":"1321","reported_epoch":"240","state":"active+clean","last_fresh":"2020-08-07T15:12:26.698895+0000","last_change":"2020-08-07T15:12:10.468110+0000","last_active":"2020-08-07T15:12:26.698895+0000","last_peered":"2020-08-07T15:12:26.698895+0000","last_clean":"2020-08-07T15:12:26.698895+0000","last_became_active":"2020-08-07T15:12:10.467697+0000","last_became_peered":"2020-08-07T15:12:10.467697+0000","last_unstale":"2020-08-07T15:12:26.698895+0000","last_undegraded":"2020-08-07T15:12:26.698895+0000","last_fullsized":"2020-08-07T15:12:26.698895+0000","mapping_epoch":230,"log_start":"240'1297","ondisk_log_start":"240'1297","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":10682368,"num_objects":163,"num_object_clones":0,"num_object_copies":489,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1304,"num_write_kb":10432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"258","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.228262+0000","last_change":"2020-08-07T15:12:13.680022+0000","last_active":"2020-08-07T15:12:22.228262+0000","last_peered":"2020-08-07T15:12:22.228262+0000","last_clean":"2020-08-07T15:12:22.228262+0000","last_became_active":"2020-08-07T15:12:06.380392+0000","last_became_peered":"2020-08-07T15:12:06.380392+0000","last_unstale":"2020-08-07T15:12:22.228262+0000","last_undegraded":"2020-08-07T15:12:22.228262+0000","last_fullsized":"2020-08-07T15:12:22.228262+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:13.679943+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.679943+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"240'1429","reported_seq":"1445","reported_epoch":"240","state":"active+clean","last_fresh":"2020-08-07T15:12:26.757098+0000","last_change":"2020-08-07T15:12:10.398816+0000","last_active":"2020-08-07T15:12:26.757098+0000","last_peered":"2020-08-07T15:12:26.757098+0000","last_clean":"2020-08-07T15:12:26.757098+0000","last_became_active":"2020-08-07T15:12:10.398332+0000","last_became_peered":"2020-08-07T15:12:10.398332+0000","last_unstale":"2020-08-07T15:12:26.757098+0000","last_undegraded":"2020-08-07T15:12:26.757098+0000","last_fullsized":"2020-08-07T15:12:26.757098+0000","mapping_epoch":230,"log_start":"240'1424","ondisk_log_start":"240'1424","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11706368,"num_objects":179,"num_object_clones":0,"num_object_copies":537,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1429,"num_write_kb":11432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"213","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.227780+0000","last_change":"2020-08-07T15:12:14.241339+0000","last_active":"2020-08-07T15:12:22.227780+0000","last_peered":"2020-08-07T15:12:22.227780+0000","last_clean":"2020-08-07T15:12:22.227780+0000","last_became_active":"2020-08-07T15:10:46.498051+0000","last_became_peered":"2020-08-07T15:10:46.498051+0000","last_unstale":"2020-08-07T15:12:22.227780+0000","last_undegraded":"2020-08-07T15:12:22.227780+0000","last_fullsized":"2020-08-07T15:12:22.227780+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:14.241262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:14.241262+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"240'1400","reported_seq":"1403","reported_epoch":"240","state":"active+clean","last_fresh":"2020-08-07T15:12:26.726813+0000","last_change":"2020-08-07T15:12:10.398015+0000","last_active":"2020-08-07T15:12:26.726813+0000","last_peered":"2020-08-07T15:12:26.726813+0000","last_clean":"2020-08-07T15:12:26.726813+0000","last_became_active":"2020-08-07T15:12:10.397610+0000","last_became_peered":"2020-08-07T15:12:10.397610+0000","last_unstale":"2020-08-07T15:12:26.726813+0000","last_undegraded":"2020-08-07T15:12:26.726813+0000","last_fullsized":"2020-08-07T15:12:26.726813+0000","mapping_epoch":230,"log_start":"240'1384","ondisk_log_start":"240'1384","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11468800,"num_objects":175,"num_object_clones":0,"num_object_copies":525,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1400,"num_write_kb":11200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"276","reported_epoch":"238","state":"active+clean","last_fresh":"2020-08-07T15:12:20.264685+0000","last_change":"2020-08-07T15:12:13.176485+0000","last_active":"2020-08-07T15:12:20.264685+0000","last_peered":"2020-08-07T15:12:20.264685+0000","last_clean":"2020-08-07T15:12:20.264685+0000","last_became_active":"2020-08-07T15:08:41.689435+0000","last_became_peered":"2020-08-07T15:08:41.689435+0000","last_unstale":"2020-08-07T15:12:20.264685+0000","last_undegraded":"2020-08-07T15:12:20.264685+0000","last_fullsized":"2020-08-07T15:12:20.264685+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.176383+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"240'1272","reported_seq":"1289","reported_epoch":"240","state":"active+clean","last_fresh":"2020-08-07T15:12:26.731959+0000","last_change":"2020-08-07T15:12:10.468495+0000","last_active":"2020-08-07T15:12:26.731959+0000","last_peered":"2020-08-07T15:12:26.731959+0000","last_clean":"2020-08-07T15:12:26.731959+0000","last_became_active":"2020-08-07T15:12:10.467915+0000","last_became_peered":"2020-08-07T15:12:10.467915+0000","last_unstale":"2020-08-07T15:12:26.731959+0000","last_undegraded":"2020-08-07T15:12:26.731959+0000","last_fullsized":"2020-08-07T15:12:26.731959+0000","mapping_epoch":230,"log_start":"240'1264","ondisk_log_start":"240'1264","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":10420224,"num_objects":159,"num_object_clones":0,"num_object_copies":477,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1272,"num_write_kb":10176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,2],"acting":[7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"174","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.228124+0000","last_change":"2020-08-07T15:11:10.853414+0000","last_active":"2020-08-07T15:12:22.228124+0000","last_peered":"2020-08-07T15:12:22.228124+0000","last_clean":"2020-08-07T15:12:22.228124+0000","last_became_active":"2020-08-07T15:10:46.493498+0000","last_became_peered":"2020-08-07T15:10:46.493498+0000","last_unstale":"2020-08-07T15:12:22.228124+0000","last_undegraded":"2020-08-07T15:12:22.228124+0000","last_fullsized":"2020-08-07T15:12:22.228124+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:10.853337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:10.853337+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"240'1408","reported_seq":"1425","reported_epoch":"240","state":"active+clean","last_fresh":"2020-08-07T15:12:26.739297+0000","last_change":"2020-08-07T15:12:10.465251+0000","last_active":"2020-08-07T15:12:26.739297+0000","last_peered":"2020-08-07T15:12:26.739297+0000","last_clean":"2020-08-07T15:12:26.739297+0000","last_became_active":"2020-08-07T15:12:10.464698+0000","last_became_peered":"2020-08-07T15:12:10.464698+0000","last_unstale":"2020-08-07T15:12:26.739297+0000","last_undegraded":"2020-08-07T15:12:26.739297+0000","last_fullsized":"2020-08-07T15:12:26.739297+0000","mapping_epoch":230,"log_start":"240'1402","ondisk_log_start":"240'1402","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11534336,"num_objects":176,"num_object_clones":0,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1408,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"208","reported_epoch":"239","state":"active+clean","last_fresh":"2020-08-07T15:12:22.222207+0000","last_change":"2020-08-07T15:12:06.367666+0000","last_active":"2020-08-07T15:12:22.222207+0000","last_peered":"2020-08-07T15:12:22.222207+0000","last_clean":"2020-08-07T15:12:22.222207+0000","last_became_active":"2020-08-07T15:12:06.367081+0000","last_became_peered":"2020-08-07T15:12:06.367081+0000","last_unstale":"2020-08-07T15:12:22.222207+0000","last_undegraded":"2020-08-07T15:12:22.222207+0000","last_fullsized":"2020-08-07T15:12:22.222207+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:37.601099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:37.601099+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":174563328,"num_objects":2664,"num_object_clones":0,"num_object_copies":7992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21309,"num_write_kb":170472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":515022848,"data_stored":515022848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":31,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":149,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":93,"acting":93,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542219,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1086800,"kb_used_data":38216,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93285040,"statfs":{"total":96636764160,"available":95523880960,"internally_reserved":1073741824,"allocated":39133184,"data_stored":38012037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:11:28 2020","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":9.548,"15min":3.778},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.1970000000000001,"5min":856.07100000000003,"15min":856.07100000000003},"last":0.45400000000000001},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0389999999999999,"15min":0.98799999999999999},"min":{"1min":0.53400000000000003,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":3.4159999999999999,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":0.52500000000000002}]},{"osd":1,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.335,"15min":1.3080000000000001},"min":{"1min":0.64200000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.9199999999999999,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.141,"15min":1.024},"min":{"1min":0.61799999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":4.0060000000000002,"5min":4.0060000000000002,"15min":4.0060000000000002},"last":0.73399999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:11:28 2020","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.292,"15min":1.1819999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.4359999999999999,"5min":5.6619999999999999,"15min":5.6619999999999999},"last":1.133},{"interface":"front","average":{"1min":1.105,"5min":1.3360000000000001,"15min":1.1899999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.109,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":1.0229999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:11:28 2020","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":16.053000000000001,"15min":6.4649999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.5409999999999999,"5min":917.38699999999994,"15min":917.38699999999994},"last":656.99099999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":4.9219999999999997,"15min":2.536},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.4750000000000001,"5min":333.59199999999998,"15min":333.59199999999998},"last":0.98299999999999998}]},{"osd":4,"last update":"Fri Aug 7 15:11:28 2020","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.427,"15min":1.3260000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.1469999999999998,"5min":4.835,"15min":4.835},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.125,"5min":11.481,"15min":4.7009999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.4300000000000002,"5min":866.24099999999999,"15min":866.24099999999999},"last":1.5569999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:11:28 2020","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":16.597000000000001,"15min":30.536999999999999},"min":{"1min":0.70599999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.5870000000000002,"5min":831.88099999999997,"15min":831.88099999999997},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.8049999999999999,"15min":1.5980000000000001},"min":{"1min":0.93700000000000006,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.6200000000000001,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":1.1000000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:11:28 2020","interfaces":[{"interface":"back","average":{"1min":1.609,"5min":4.0119999999999996,"15min":2.3490000000000002},"min":{"1min":0.873,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":5.4800000000000004,"5min":224.14500000000001,"15min":224.14500000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.716,"5min":1.732,"15min":1.5820000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":5.5309999999999997,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":1.163}]}]},{"osd":7,"up_from":13,"seq":55834574925,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1112760,"kb_used_data":64176,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93259080,"statfs":{"total":96636764160,"available":95497297920,"internally_reserved":1073741824,"allocated":65716224,"data_stored":64579945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":42.923000000000002,"5min":9.4640000000000004,"15min":4.0430000000000001},"min":{"1min":0.48999999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":755.10000000000002,"5min":755.10000000000002,"15min":755.10000000000002},"last":763.39099999999996},{"interface":"front","average":{"1min":0.96699999999999997,"5min":9.7780000000000005,"15min":4.0259999999999998},"min":{"1min":0.46300000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.718,"5min":529.88699999999994,"15min":529.88699999999994},"last":0.89700000000000002}]},{"osd":1,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":5.758,"15min":2.5579999999999998},"min":{"1min":0.47699999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.7290000000000001,"5min":547.65300000000002,"15min":547.65300000000002},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.4159999999999999,"5min":4.9189999999999996,"15min":2.5640000000000001},"min":{"1min":0.57399999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":5.4960000000000004,"5min":372.50799999999998,"15min":372.50799999999998},"last":1.6040000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.3240000000000001,"15min":1.238},"min":{"1min":0.54900000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":6.968,"5min":6.968,"15min":6.968},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1799999999999999,"15min":1.2410000000000001},"min":{"1min":0.58599999999999997,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.7709999999999999,"5min":6.0490000000000004,"15min":6.0490000000000004},"last":0.95099999999999996}]},{"osd":3,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":9.5,"15min":4.1840000000000002},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":3.3540000000000001,"5min":698.19899999999996,"15min":698.19899999999996},"last":1.012},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.2969999999999999,"15min":1.357},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":4.1779999999999999,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":1.1599999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.3120000000000001,"15min":1.546},"min":{"1min":0.60999999999999999,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.5449999999999999,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.53,"15min":1.6240000000000001},"min":{"1min":0.71399999999999997,"5min":0.621,"15min":0.621},"max":{"1min":2.9900000000000002,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.6240000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.7130000000000001,"5min":1.329,"15min":1.3520000000000001},"min":{"1min":0.749,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":6.2080000000000002,"5min":6.2080000000000002,"15min":6.2080000000000002},"last":1.028},{"interface":"front","average":{"1min":1.7350000000000001,"5min":1.526,"15min":1.6859999999999999},"min":{"1min":0.89000000000000001,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":3.052,"5min":5.327,"15min":5.327},"last":1.7470000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":40.012999999999998,"5min":9.1820000000000004,"15min":4.016},"min":{"1min":0.86399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":695.92899999999997,"5min":695.92899999999997,"15min":695.92899999999997},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.613,"5min":4.8959999999999999,"15min":13.849},"min":{"1min":0.91300000000000003,"5min":0.87,"15min":0.87},"max":{"1min":3.0270000000000001,"5min":266.26999999999998,"15min":266.26999999999998},"last":1.7290000000000001}]}]},{"osd":1,"up_from":227,"seq":974957576197,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1097772,"kb_used_data":49188,"kb_used_omap":2,"kb_used_meta":1048573,"kb_avail":93274068,"statfs":{"total":96636764160,"available":95512645632,"internally_reserved":1073741824,"allocated":50368512,"data_stored":49235766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":1073739245},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.385}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.343}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0609999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1089384,"kb_used_data":40800,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93282456,"statfs":{"total":96636764160,"available":95521234944,"internally_reserved":1073741824,"allocated":41779200,"data_stored":40642921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":24.291,"5min":21.719999999999999,"15min":7.891},"min":{"1min":0.52700000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":432.899,"5min":845.57100000000003,"15min":845.57100000000003},"last":2.468},{"interface":"front","average":{"1min":1.752,"5min":1.181,"15min":1.0149999999999999},"min":{"1min":0.62,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":9.3650000000000002,"5min":9.3650000000000002,"15min":9.3650000000000002},"last":2.5550000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:11:32 2020","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":1.0069999999999999,"15min":0.95399999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.5289999999999999,"5min":4.1600000000000001,"15min":4.1600000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.5629999999999999,"5min":9.9559999999999995,"15min":25.145},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":4.8689999999999998,"5min":606.18499999999995,"15min":606.18499999999995},"last":1.377}]},{"osd":3,"last update":"Fri Aug 7 15:11:32 2020","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.264,"15min":1.0449999999999999},"min":{"1min":0.54900000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.298,"5min":9.3360000000000003,"15min":9.3360000000000003},"last":1.9159999999999999},{"interface":"front","average":{"1min":1.121,"5min":1.409,"15min":1.248},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.9330000000000001,"5min":11.19,"15min":11.19},"last":1.1599999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:11:32 2020","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":1.321,"15min":1.2170000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.7589999999999999,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":1.819},{"interface":"front","average":{"1min":1.198,"5min":1.4390000000000001,"15min":1.3180000000000001},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.887,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":0.91300000000000003}]},{"osd":5,"last update":"Fri Aug 7 15:11:32 2020","interfaces":[{"interface":"back","average":{"1min":28.428000000000001,"5min":14.52,"15min":5.6580000000000004},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":539.31100000000004,"5min":539.31100000000004,"15min":539.31100000000004},"last":1.403},{"interface":"front","average":{"1min":1.444,"5min":1.4419999999999999,"15min":1.218},"min":{"1min":0.77600000000000002,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":3.5790000000000002,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":1.8600000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:11:32 2020","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":2.8889999999999998,"15min":6.7050000000000001},"min":{"1min":0.84099999999999997,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":3.1899999999999999,"5min":124.738,"15min":124.738},"last":1.131},{"interface":"front","average":{"1min":1.5940000000000001,"5min":4.9909999999999997,"15min":14.099},"min":{"1min":0.81799999999999995,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":5.1799999999999997,"5min":294.94,"15min":294.94},"last":1.3049999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:11:32 2020","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.5629999999999999,"15min":1.5780000000000001},"min":{"1min":0.90600000000000003,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":4.0010000000000003,"5min":6.468,"15min":6.468},"last":1.355},{"interface":"front","average":{"1min":1.492,"5min":10.583,"15min":4.4509999999999996},"min":{"1min":0.90600000000000003,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":3.1749999999999998,"5min":859.82399999999996,"15min":859.82399999999996},"last":1.7849999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574925,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1134432,"kb_used_data":85848,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93237408,"statfs":{"total":96636764160,"available":95475105792,"internally_reserved":1073741824,"allocated":87908352,"data_stored":86772073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5],"upper_bound":2},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":32.965000000000003,"5min":7.8639999999999999,"15min":3.3660000000000001},"min":{"1min":0.41899999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":501.53800000000001,"5min":501.53800000000001,"15min":501.53800000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":27.341999999999999,"5min":6.6900000000000004,"15min":2.8799999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":501.36000000000001,"5min":501.36000000000001,"15min":501.36000000000001},"last":0.57699999999999996}]},{"osd":1,"last update":"Fri Aug 7 15:10:21 2020","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.6970000000000001,"15min":1.2929999999999999},"min":{"1min":0.52000000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":5.5919999999999996,"5min":18.672000000000001,"15min":18.672000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.264,"5min":1.3320000000000001,"15min":1.3500000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":3.7970000000000002,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":0.59999999999999998}]},{"osd":3,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.2130000000000001,"15min":1.179},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":0.624},{"interface":"front","average":{"1min":23.064,"5min":15.281000000000001,"15min":6.1310000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":412.61500000000001,"5min":501.75599999999997,"15min":501.75599999999997},"last":0.82199999999999995}]},{"osd":4,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.5940000000000001,"15min":1.2709999999999999},"min":{"1min":0.67400000000000004,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.712,"5min":19.783000000000001,"15min":19.783000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.4610000000000001,"15min":1.5820000000000001},"min":{"1min":0.68999999999999995,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8979999999999999,"5min":8.048,"15min":8.048},"last":1.083}]},{"osd":5,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.3460000000000001,"15min":1.29},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.4009999999999998,"5min":5.2220000000000004,"15min":5.2220000000000004},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.4990000000000001,"5min":15.715,"15min":22.254999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":8.0649999999999995,"5min":627.53300000000002,"15min":627.53300000000002},"last":1.7270000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:11:34 2020","interfaces":[{"interface":"back","average":{"1min":42.643000000000001,"5min":9.9380000000000006,"15min":4.1260000000000003},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":821.91899999999998,"5min":821.91899999999998,"15min":821.91899999999998},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.2190000000000001,"5min":9.9429999999999996,"15min":4.1950000000000003},"min":{"1min":0.80200000000000005,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.73,"5min":751.70000000000005,"15min":751.70000000000005},"last":1.004}]},{"osd":7,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":12.522,"15min":5.335},"min":{"1min":0.85599999999999998,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.3519999999999999,"5min":501.91699999999997,"15min":501.91699999999997},"last":1.276},{"interface":"front","average":{"1min":1.7230000000000001,"5min":2.1080000000000001,"15min":1.7},"min":{"1min":0.88500000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":7.2149999999999999,"5min":35.646999999999998,"15min":35.646999999999998},"last":0.997}]}]},{"osd":3,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1158176,"kb_used_data":109592,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93213664,"statfs":{"total":96636764160,"available":95450791936,"internally_reserved":1073741824,"allocated":112222208,"data_stored":111085929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,0,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.2689999999999999,"15min":1.258},"min":{"1min":0.51100000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":3.2149999999999999,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":1.421},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.296,"15min":1.5229999999999999},"min":{"1min":0.53900000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":4.0190000000000001,"5min":13.067,"15min":13.067},"last":1.472}]},{"osd":1,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":4.4779999999999998,"15min":2.758},"min":{"1min":0.59099999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.847,"5min":270.68700000000001,"15min":270.68700000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.109,"5min":1.8580000000000001,"15min":1.96},"min":{"1min":0.54100000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.0550000000000002,"5min":14.349,"15min":14.349},"last":1.504}]},{"osd":2,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.3380000000000001,"15min":1.2110000000000001},"min":{"1min":0.65300000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.6389999999999998,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":1.526},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.9810000000000001,"15min":1.7829999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":6.7889999999999997,"5min":51.530000000000001,"15min":51.530000000000001},"last":1.3919999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":17.366,"15min":6.9199999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":3.1600000000000001,"5min":646.26700000000005,"15min":646.26700000000005},"last":0.71499999999999997},{"interface":"front","average":{"1min":43.484000000000002,"5min":21.631,"15min":28.609999999999999},"min":{"1min":0.77100000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":844.10500000000002,"5min":844.10500000000002,"15min":844.10500000000002},"last":2.1349999999999998}]},{"osd":5,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":5.9560000000000004,"15min":2.9969999999999999},"min":{"1min":0.79300000000000004,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.2370000000000001,"5min":403.88400000000001,"15min":403.88400000000001},"last":1.8600000000000001},{"interface":"front","average":{"1min":1.591,"5min":1.895,"15min":1.8109999999999999},"min":{"1min":0.97399999999999998,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":3.9790000000000001,"5min":8.9120000000000008,"15min":8.9120000000000008},"last":1.889}]},{"osd":6,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.653,"15min":1.5720000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":4.1600000000000001,"5min":5.0389999999999997,"15min":5.0389999999999997},"last":1.244},{"interface":"front","average":{"1min":1.944,"5min":13.427,"15min":5.7999999999999998},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":6.9980000000000002,"5min":688.15999999999997,"15min":688.15999999999997},"last":1.2769999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:11:29 2020","interfaces":[{"interface":"back","average":{"1min":1.655,"5min":1.6799999999999999,"15min":2.0009999999999999},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":6.8209999999999997,"5min":6.8209999999999997,"15min":6.8209999999999997},"last":1.21},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.77,"15min":1.9079999999999999},"min":{"1min":0.77000000000000002,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":3.1499999999999999,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":0.91200000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1107120,"kb_used_data":58536,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93264720,"statfs":{"total":96636764160,"available":95503073280,"internally_reserved":1073741824,"allocated":59940864,"data_stored":58804585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:11:25 2020","interfaces":[{"interface":"back","average":{"1min":20.597999999999999,"5min":9.9239999999999995,"15min":4.0949999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":356.91800000000001,"5min":399.73700000000002,"15min":399.73700000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.84299999999999997,"5min":1.046,"15min":0.91400000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.972,"5min":6.3970000000000002,"15min":6.3970000000000002},"last":0.89300000000000002}]},{"osd":1,"last update":"Fri Aug 7 15:10:24 2020","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":5.7629999999999999,"15min":2.823},"min":{"1min":0.52900000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.8109999999999999,"5min":320.67000000000002,"15min":320.67000000000002},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.177,"5min":7.5300000000000002,"15min":3.4279999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.7200000000000002,"5min":689.846,"15min":689.846},"last":1.3260000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:11:25 2020","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.3680000000000001,"15min":1.4239999999999999},"min":{"1min":0.67300000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.786,"5min":6.8730000000000002,"15min":6.8730000000000002},"last":1.917},{"interface":"front","average":{"1min":1.446,"5min":1.51,"15min":1.5129999999999999},"min":{"1min":0.85899999999999999,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.8140000000000001,"5min":7.1360000000000001,"15min":7.1360000000000001},"last":1.9419999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:11:25 2020","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":12.045,"15min":5.0199999999999996},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.6579999999999999,"5min":638.92100000000005,"15min":638.92100000000005},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.3480000000000001,"15min":1.2829999999999999},"min":{"1min":0.70899999999999996,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.0289999999999999,"5min":6.431,"15min":6.431},"last":1.722}]},{"osd":5,"last update":"Fri Aug 7 15:11:25 2020","interfaces":[{"interface":"back","average":{"1min":41.594000000000001,"5min":18.439,"15min":7.2309999999999999},"min":{"1min":0.60499999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":725.98599999999999,"5min":990.71600000000001,"15min":990.71600000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.4259999999999999,"15min":1.5600000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.7310000000000001,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":0.55700000000000005}]},{"osd":6,"last update":"Fri Aug 7 15:11:34 2020","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.5800000000000001,"15min":1.72},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":5.4459999999999997,"5min":6.9749999999999996,"15min":6.9749999999999996},"last":0.67200000000000004},{"interface":"front","average":{"1min":1.256,"5min":4.9329999999999998,"15min":14.382999999999999},"min":{"1min":0.71399999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.3719999999999999,"5min":389.14600000000002,"15min":389.14600000000002},"last":0.64600000000000002}]},{"osd":7,"last update":"Fri Aug 7 15:11:25 2020","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":3.782,"15min":2.1989999999999998},"min":{"1min":0.78300000000000003,"5min":0.62,"15min":0.62},"max":{"1min":2.5310000000000001,"5min":252.798,"15min":252.798},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.4370000000000001,"5min":14.112,"15min":5.8010000000000002},"min":{"1min":0.78100000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":3.0369999999999999,"5min":501.93000000000001,"15min":501.93000000000001},"last":0.84799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1103080,"kb_used_data":54496,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93268760,"statfs":{"total":96636764160,"available":95507210240,"internally_reserved":1073741824,"allocated":55803904,"data_stored":54667625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:11:26 2020","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.996,"15min":1.05},"min":{"1min":0.54400000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.5289999999999999,"5min":3.7349999999999999,"15min":3.7349999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":15.863,"5min":3.956,"15min":1.855},"min":{"1min":0.44,"5min":0.438,"15min":0.438},"max":{"1min":329.11399999999998,"5min":329.11399999999998,"15min":329.11399999999998},"last":1.0449999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":8.5239999999999991,"5min":12.039999999999999,"15min":4.6319999999999997},"min":{"1min":0.56799999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":139.88200000000001,"5min":787.22900000000004,"15min":787.22900000000004},"last":2.0569999999999999},{"interface":"front","average":{"1min":1.448,"5min":2.5649999999999999,"15min":1.593},"min":{"1min":0.59899999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":4.3700000000000001,"5min":116.548,"15min":116.548},"last":1.673}]},{"osd":2,"last update":"Fri Aug 7 15:11:26 2020","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":2.714,"15min":7.0510000000000002},"min":{"1min":0.52100000000000002,"5min":0.499,"15min":0.499},"max":{"1min":2.109,"5min":168.56899999999999,"15min":168.56899999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":7.6200000000000001,"15min":24.626999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":3.5609999999999999,"5min":676.27099999999996,"15min":676.27099999999996},"last":1.3129999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:11:26 2020","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.6559999999999999,"15min":1.2969999999999999},"min":{"1min":0.71499999999999997,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":5.1539999999999999,"5min":29.129000000000001,"15min":29.129000000000001},"last":2.0249999999999999},{"interface":"front","average":{"1min":1.1140000000000001,"5min":9.9969999999999999,"15min":4.141},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.149,"5min":712.98699999999997,"15min":712.98699999999997},"last":1.502}]},{"osd":4,"last update":"Fri Aug 7 15:11:26 2020","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":22.683,"15min":41.488999999999997},"min":{"1min":0.54800000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.5899999999999999,"5min":971.67999999999995,"15min":971.67999999999995},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.272,"15min":1.294},"min":{"1min":0.63700000000000001,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":5.4900000000000002,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":1.3440000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:11:26 2020","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":31.462,"15min":44.093000000000004},"min":{"1min":0.85099999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":5.4480000000000004,"5min":966.60900000000004,"15min":966.60900000000004},"last":2.7000000000000002},{"interface":"front","average":{"1min":1.2789999999999999,"5min":11.677,"15min":21.236999999999998},"min":{"1min":0.871,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.4710000000000001,"5min":517.26700000000005,"15min":517.26700000000005},"last":1.536}]},{"osd":7,"last update":"Fri Aug 7 15:11:26 2020","interfaces":[{"interface":"back","average":{"1min":1.6799999999999999,"5min":1.403,"15min":1.3400000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":5.1559999999999997,"5min":5.1559999999999997,"15min":5.1559999999999997},"last":2.4700000000000002},{"interface":"front","average":{"1min":32.561999999999998,"5min":12.834,"15min":5.3220000000000001},"min":{"1min":0.63700000000000001,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":501.64299999999997,"5min":515.09100000000001,"15min":515.09100000000001},"last":1.468}]}]}]}} 2020-08-07T15:12:28.854 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:12:28.861 INFO:tasks.ceph.ceph_manager.ceph:increase pool size by 10 2020-08-07T15:12:28.866 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set device_health_metrics pg_num 41 2020-08-07T15:12:28.868 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:12:28.869 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:12:28.869 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:12:28.869 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:12:28.869 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:12:28.870 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:12:28.870 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:12:28.870 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:12:28.870 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 16 16 24217 24201 11.8149 12.0078 0.00985418 0.0105699 2020-08-07T15:12:28.871 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:12:28.871 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:28.835+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:28.872 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:12:28.939 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:28.938+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:29.043 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:29.042+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:29.048 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:29.048 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:29.060 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:12:29.145 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:29.144+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:29.251 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:29.245+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:29.258 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:29.259 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:29.273 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:12:29.274 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 17 16 25881 25865 11.8845 13 0.0236932 0.0105099 2020-08-07T15:12:29.347 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:29.346+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:29.437 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:29.437 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:29.448 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:12:29.449 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:29.447+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:29.549 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:29.548+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:29.650 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:29.649+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:29.653 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:29.653 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:29.666 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:12:29.752 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:29.750+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:29.820 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:29.820 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:29.829 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:12:29.882 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:29.874+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:29.955 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:29.954+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:30.046 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:30.046 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:30.059 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:12:30.060 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:30.056+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:30.168 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:30.166+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:30.257 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:30.258 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:30.268 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 18 16 27346 27330 11.8601 11.4453 0.0119177 0.0105326 2020-08-07T15:12:30.269 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:12:30.270 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:30.267+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:30.338 INFO:teuthology.orchestra.run.smithi074.stderr:set pool 1 pg_num to 41 2020-08-07T15:12:30.395 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:30.394+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:30.466 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:30.467 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:30.470 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:30.469+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:30.478 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:12:30.571 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:30.570+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:30.672 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:30.671+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:30.690 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:30.691 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:30.701 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:12:30.773 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:30.771+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:30.871 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:12:30.873 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:30.873 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:30.877 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:12:30.880 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:30.878+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:30.888 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:12:30.981 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:30.980+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:31.097 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:31.088+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:31.140 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:12:31.152 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:31.153 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:31.164 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:12:31.185 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:12:31.202 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:31.196+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:31.226 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:12:31.263 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:12:31.270 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 19 16 28840 28824 11.8501 11.6719 0.0179883 0.0105355 2020-08-07T15:12:31.308 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:31.302+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:31.311 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:12:31.373 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:12:31.399 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:31.400 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:12:31.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:12:31.400 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:12:31.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.400 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "MOSDScrubReserve(2.6 GRANT e232)", 2020-08-07T15:12:31.401 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:11.909673+0000", 2020-08-07T15:12:31.401 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 19.486002456000001, 2020-08-07T15:12:31.401 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.36313456, 2020-08-07T15:12:31.401 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:31.401 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:12:31.402 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:31.402 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.403 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:31.403 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:11.909673+0000", 2020-08-07T15:12:31.403 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:31.403 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.403 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.406 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:31.406 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:11.909673+0000", 2020-08-07T15:12:31.406 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0409000000000001e-05 2020-08-07T15:12:31.406 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.406 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.407 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:31.407 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:11.909693+0000", 2020-08-07T15:12:31.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8580000000000002e-06 2020-08-07T15:12:31.407 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.407 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.408 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:31.408 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:11.909697+0000", 2020-08-07T15:12:31.408 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.36297494299999999 2020-08-07T15:12:31.408 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.408 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.409 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:31.409 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272672+0000", 2020-08-07T15:12:31.409 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4490000000000001e-06 2020-08-07T15:12:31.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.409 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.410 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:31.410 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272676+0000", 2020-08-07T15:12:31.410 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0395999999999994e-05 2020-08-07T15:12:31.410 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.410 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.411 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:31.411 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272757+0000", 2020-08-07T15:12:31.411 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6416e-05 2020-08-07T15:12:31.411 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.411 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.411 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:31.412 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272773+0000", 2020-08-07T15:12:31.412 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4088999999999998e-05 2020-08-07T15:12:31.413 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.413 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:31.413 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272807+0000", 2020-08-07T15:12:31.414 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.36313456 2020-08-07T15:12:31.414 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.414 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:31.415 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.415 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:7272 6.e 6:74263cb8:::benchmark_data_smithi074_63843_object908:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e234)", 2020-08-07T15:12:31.415 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:16.977784+0000", 2020-08-07T15:12:31.417 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 14.417890708, 2020-08-07T15:12:31.417 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31581518600000003, 2020-08-07T15:12:31.418 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:31.418 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:31.418 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:31.418 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:31.419 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:31.419 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7272 2020-08-07T15:12:31.419 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.419 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:31.419 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.419 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:31.420 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977784+0000", 2020-08-07T15:12:31.420 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:31.420 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.420 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.420 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:31.421 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977784+0000", 2020-08-07T15:12:31.421 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6940999999999998e-05 2020-08-07T15:12:31.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.421 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.421 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:31.422 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977811+0000", 2020-08-07T15:12:31.422 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1870000000000001e-06 2020-08-07T15:12:31.422 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.422 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:31.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977815+0000", 2020-08-07T15:12:31.423 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1249999999999998e-06 2020-08-07T15:12:31.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.424 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.424 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:31.424 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977821+0000", 2020-08-07T15:12:31.424 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8739999999999999e-06 2020-08-07T15:12:31.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.425 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:31.425 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977824+0000", 2020-08-07T15:12:31.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001740655 2020-08-07T15:12:31.426 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.426 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.426 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:31.426 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.979564+0000", 2020-08-07T15:12:31.426 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6272000000000003e-05 2020-08-07T15:12:31.427 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.427 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:31.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.979650+0000", 2020-08-07T15:12:31.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012951600000000001 2020-08-07T15:12:31.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:12:31.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.979780+0000", 2020-08-07T15:12:31.430 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00085408999999999995 2020-08-07T15:12:31.430 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.430 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.432 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:31.433 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.980634+0000", 2020-08-07T15:12:31.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8099999999999999e-07 2020-08-07T15:12:31.434 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.434 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.434 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:31.435 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.980634+0000", 2020-08-07T15:12:31.435 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5664000000000003e-05 2020-08-07T15:12:31.435 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.436 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.436 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:31.436 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.980670+0000", 2020-08-07T15:12:31.437 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.104650663 2020-08-07T15:12:31.437 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.437 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.438 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:31.438 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.085321+0000", 2020-08-07T15:12:31.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20821616000000001 2020-08-07T15:12:31.441 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.441 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.441 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:31.441 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.293537+0000", 2020-08-07T15:12:31.441 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1900000000000001e-07 2020-08-07T15:12:31.442 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.442 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.442 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:31.442 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.293538+0000", 2020-08-07T15:12:31.442 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1939000000000004e-05 2020-08-07T15:12:31.443 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.443 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.443 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:31.443 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.293600+0000", 2020-08-07T15:12:31.443 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31581518600000003 2020-08-07T15:12:31.444 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.444 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:31.444 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.444 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.445 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:7983 6.1 6:83bbbe87:::benchmark_data_smithi074_63843_object997:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e235)", 2020-08-07T15:12:31.445 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:17.448194+0000", 2020-08-07T15:12:31.445 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 13.947480914, 2020-08-07T15:12:31.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25910564899999999, 2020-08-07T15:12:31.445 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:31.446 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:31.446 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:31.446 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:31.446 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:31.446 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7983 2020-08-07T15:12:31.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.447 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:31.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:31.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.448194+0000", 2020-08-07T15:12:31.448 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:31.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.449 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.449 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:31.449 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.448194+0000", 2020-08-07T15:12:31.449 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2516999999999999e-05 2020-08-07T15:12:31.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.450 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:31.450 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.448247+0000", 2020-08-07T15:12:31.450 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8919999999999997e-06 2020-08-07T15:12:31.451 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.451 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.451 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:31.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.448253+0000", 2020-08-07T15:12:31.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2149e-05 2020-08-07T15:12:31.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:31.453 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.448265+0000", 2020-08-07T15:12:31.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6380000000000001e-06 2020-08-07T15:12:31.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.453 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:31.454 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.448270+0000", 2020-08-07T15:12:31.454 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0543e-05 2020-08-07T15:12:31.454 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.454 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.454 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:31.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.448321+0000", 2020-08-07T15:12:31.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010807299999999999 2020-08-07T15:12:31.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:31.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.448429+0000", 2020-08-07T15:12:31.456 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000151008 2020-08-07T15:12:31.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:12:31.456 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.448580+0000", 2020-08-07T15:12:31.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012197849999999999 2020-08-07T15:12:31.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:31.458 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.449800+0000", 2020-08-07T15:12:31.458 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6000000000000004e-07 2020-08-07T15:12:31.458 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.458 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.458 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:31.459 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.449800+0000", 2020-08-07T15:12:31.459 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5127000000000003e-05 2020-08-07T15:12:31.459 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:31.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.449855+0000", 2020-08-07T15:12:31.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019709250000000001 2020-08-07T15:12:31.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.460 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.460 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:31.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.451826+0000", 2020-08-07T15:12:31.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25541297200000002 2020-08-07T15:12:31.461 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.461 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.461 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:31.461 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.707239+0000", 2020-08-07T15:12:31.462 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8400000000000004e-07 2020-08-07T15:12:31.462 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.462 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.462 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:31.462 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.707240+0000", 2020-08-07T15:12:31.463 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9775999999999998e-05 2020-08-07T15:12:31.463 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.463 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.463 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:31.463 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.707300+0000", 2020-08-07T15:12:31.464 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25910564899999999 2020-08-07T15:12:31.464 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.464 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:31.464 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.464 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.465 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.465 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:7984 6.1 6:83bbbe87:::benchmark_data_smithi074_63843_object997:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e235)", 2020-08-07T15:12:31.465 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:17.448389+0000", 2020-08-07T15:12:31.465 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 13.947286517, 2020-08-07T15:12:31.465 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26151915100000001, 2020-08-07T15:12:31.466 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:31.466 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:31.466 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:31.467 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:31.467 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:31.468 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7984 2020-08-07T15:12:31.468 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.468 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:31.468 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.469 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:31.469 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.448389+0000", 2020-08-07T15:12:31.469 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:31.469 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.469 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.470 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:31.470 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.448389+0000", 2020-08-07T15:12:31.470 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0078999999999999e-05 2020-08-07T15:12:31.470 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.470 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.471 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:31.471 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.448439+0000", 2020-08-07T15:12:31.471 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7170000000000008e-06 2020-08-07T15:12:31.471 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.471 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.472 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:31.472 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.448446+0000", 2020-08-07T15:12:31.472 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2069000000000001e-05 2020-08-07T15:12:31.472 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.472 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.473 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:31.473 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.448458+0000", 2020-08-07T15:12:31.473 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4489999999999998e-06 2020-08-07T15:12:31.473 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.473 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.474 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:31.474 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.448464+0000", 2020-08-07T15:12:31.474 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00053153599999999999 2020-08-07T15:12:31.474 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.474 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.474 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:31.475 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.448995+0000", 2020-08-07T15:12:31.475 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000102731 2020-08-07T15:12:31.475 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.475 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.475 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:31.476 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.449098+0000", 2020-08-07T15:12:31.476 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013514700000000001 2020-08-07T15:12:31.476 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.476 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.476 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:12:31.477 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.449233+0000", 2020-08-07T15:12:31.477 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000629399 2020-08-07T15:12:31.477 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.477 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.477 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:31.478 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.449863+0000", 2020-08-07T15:12:31.478 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8100000000000003e-07 2020-08-07T15:12:31.478 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.478 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.478 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:31.478 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.449863+0000", 2020-08-07T15:12:31.479 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6907999999999998e-05 2020-08-07T15:12:31.479 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.479 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.479 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:31.479 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.449920+0000", 2020-08-07T15:12:31.480 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024481630000000002 2020-08-07T15:12:31.480 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.480 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:31.480 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.452368+0000", 2020-08-07T15:12:31.481 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.257502279 2020-08-07T15:12:31.481 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.481 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.481 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:31.481 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.709871+0000", 2020-08-07T15:12:31.482 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2300000000000001e-07 2020-08-07T15:12:31.482 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.482 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.482 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:31.482 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.709871+0000", 2020-08-07T15:12:31.483 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6470000000000001e-05 2020-08-07T15:12:31.483 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.483 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.483 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:31.483 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.709908+0000", 2020-08-07T15:12:31.483 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26151915100000001 2020-08-07T15:12:31.484 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.484 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:31.484 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.484 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.484 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.485 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:7983 6.1 e235/230 ondisk, result = 0)", 2020-08-07T15:12:31.485 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:17.453940+0000", 2020-08-07T15:12:31.485 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 13.941735373, 2020-08-07T15:12:31.485 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25336447699999998, 2020-08-07T15:12:31.485 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:31.486 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:12:31.486 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:31.486 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.486 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:31.486 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.453940+0000", 2020-08-07T15:12:31.487 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:31.487 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.487 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.487 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:31.488 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.453940+0000", 2020-08-07T15:12:31.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9741999999999998e-05 2020-08-07T15:12:31.488 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.489 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:31.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.453959+0000", 2020-08-07T15:12:31.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5909999999999998e-06 2020-08-07T15:12:31.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.489 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.490 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:31.490 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.453962+0000", 2020-08-07T15:12:31.490 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25289571700000002 2020-08-07T15:12:31.490 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.490 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.491 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:31.491 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.706858+0000", 2020-08-07T15:12:31.491 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1939999999999997e-06 2020-08-07T15:12:31.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.491 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.492 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:31.492 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.706863+0000", 2020-08-07T15:12:31.492 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00036133399999999999 2020-08-07T15:12:31.492 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.492 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.492 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:31.493 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.707224+0000", 2020-08-07T15:12:31.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1149999999999996e-06 2020-08-07T15:12:31.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.493 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:31.494 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.707231+0000", 2020-08-07T15:12:31.494 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2784e-05 2020-08-07T15:12:31.494 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.494 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:31.495 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.707304+0000", 2020-08-07T15:12:31.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25336447699999998 2020-08-07T15:12:31.495 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.495 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:31.495 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.496 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.496 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.496 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:8359 6.1 6:844c6d67:::benchmark_data_smithi074_63843_object1044:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e235)", 2020-08-07T15:12:31.497 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:17.720675+0000", 2020-08-07T15:12:31.497 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 13.675000551, 2020-08-07T15:12:31.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.309231859, 2020-08-07T15:12:31.497 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:31.498 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:31.498 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:31.498 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:31.499 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:31.499 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 8359 2020-08-07T15:12:31.499 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.500 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:31.500 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.500 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:31.501 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720675+0000", 2020-08-07T15:12:31.501 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:31.501 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.502 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.502 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:31.503 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720675+0000", 2020-08-07T15:12:31.503 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.884e-05 2020-08-07T15:12:31.503 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.504 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.504 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:31.504 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720723+0000", 2020-08-07T15:12:31.505 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2989999999999996e-06 2020-08-07T15:12:31.505 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.505 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.505 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:31.506 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720729+0000", 2020-08-07T15:12:31.506 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0288999999999999e-05 2020-08-07T15:12:31.506 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.507 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.507 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:31.507 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720739+0000", 2020-08-07T15:12:31.508 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7849999999999999e-06 2020-08-07T15:12:31.508 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.508 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.509 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:31.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720744+0000", 2020-08-07T15:12:31.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001862991 2020-08-07T15:12:31.510 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:31.511 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.722607+0000", 2020-08-07T15:12:31.512 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000100839 2020-08-07T15:12:31.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.512 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.512 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:31.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.722708+0000", 2020-08-07T15:12:31.513 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017526600000000001 2020-08-07T15:12:31.513 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.514 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:12:31.514 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.722883+0000", 2020-08-07T15:12:31.515 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001504619 2020-08-07T15:12:31.515 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.515 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.516 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:31.516 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.724387+0000", 2020-08-07T15:12:31.516 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4499999999999996e-07 2020-08-07T15:12:31.517 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.518 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:31.518 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.724388+0000", 2020-08-07T15:12:31.519 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8369000000000003e-05 2020-08-07T15:12:31.519 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.519 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.519 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:31.521 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.724447+0000", 2020-08-07T15:12:31.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00097551199999999999 2020-08-07T15:12:31.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.522 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.522 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:31.522 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.725422+0000", 2020-08-07T15:12:31.522 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30441636900000002 2020-08-07T15:12:31.522 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.523 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.523 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:31.523 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029838+0000", 2020-08-07T15:12:31.523 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7699999999999992e-07 2020-08-07T15:12:31.523 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.524 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.524 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:31.524 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029839+0000", 2020-08-07T15:12:31.524 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6958999999999994e-05 2020-08-07T15:12:31.524 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.525 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.525 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:31.525 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029906+0000", 2020-08-07T15:12:31.525 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.309231859 2020-08-07T15:12:31.525 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.526 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:31.526 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.526 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.526 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.526 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:8360 6.1 6:844c6d67:::benchmark_data_smithi074_63843_object1044:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e235)", 2020-08-07T15:12:31.527 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:17.721396+0000", 2020-08-07T15:12:31.527 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 13.674278888, 2020-08-07T15:12:31.527 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31055011399999999, 2020-08-07T15:12:31.527 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:31.527 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:31.528 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:31.528 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:31.528 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:31.528 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 8360 2020-08-07T15:12:31.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.529 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:31.529 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.529 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:31.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721396+0000", 2020-08-07T15:12:31.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:31.529 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.531 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.531 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:31.531 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721396+0000", 2020-08-07T15:12:31.531 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8222e-05 2020-08-07T15:12:31.532 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.532 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.532 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:31.532 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721444+0000", 2020-08-07T15:12:31.532 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.297e-06 2020-08-07T15:12:31.533 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.533 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.533 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:31.533 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721451+0000", 2020-08-07T15:12:31.533 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.172e-05 2020-08-07T15:12:31.534 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.534 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.534 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:31.534 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721462+0000", 2020-08-07T15:12:31.534 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7639999999999996e-06 2020-08-07T15:12:31.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:31.535 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721467+0000", 2020-08-07T15:12:31.535 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023051790000000001 2020-08-07T15:12:31.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.536 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:31.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.723772+0000", 2020-08-07T15:12:31.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1051000000000006e-05 2020-08-07T15:12:31.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.563 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.563 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:31.563 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.723863+0000", 2020-08-07T15:12:31.564 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017915699999999999 2020-08-07T15:12:31.564 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.564 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:12:31.564 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.724043+0000", 2020-08-07T15:12:31.564 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001191646 2020-08-07T15:12:31.565 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.565 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.565 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:31.565 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.725234+0000", 2020-08-07T15:12:31.565 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9100000000000002e-07 2020-08-07T15:12:31.566 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.566 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.566 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:31.566 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.725235+0000", 2020-08-07T15:12:31.566 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0677999999999997e-05 2020-08-07T15:12:31.567 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.568 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.568 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:31.568 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.725296+0000", 2020-08-07T15:12:31.568 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001252663 2020-08-07T15:12:31.568 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.569 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.569 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:31.569 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.726548+0000", 2020-08-07T15:12:31.569 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30531503100000001 2020-08-07T15:12:31.569 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.570 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.570 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:31.570 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.031864+0000", 2020-08-07T15:12:31.570 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.057e-06 2020-08-07T15:12:31.570 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.571 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.571 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:31.571 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.031865+0000", 2020-08-07T15:12:31.571 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1757999999999999e-05 2020-08-07T15:12:31.571 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.572 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.572 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:31.572 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.031946+0000", 2020-08-07T15:12:31.572 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31055011399999999 2020-08-07T15:12:31.572 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.572 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:31.573 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.573 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.573 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.573 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:8359 6.1 e235/230 ondisk, result = 0)", 2020-08-07T15:12:31.574 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:17.726764+0000", 2020-08-07T15:12:31.574 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 13.668910872, 2020-08-07T15:12:31.574 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30314883199999998, 2020-08-07T15:12:31.574 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:31.575 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:12:31.575 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:31.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.575 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:31.575 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.726764+0000", 2020-08-07T15:12:31.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:31.576 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.576 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.576 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:31.576 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.726764+0000", 2020-08-07T15:12:31.577 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1386000000000001e-05 2020-08-07T15:12:31.577 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.577 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.577 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:31.577 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.726796+0000", 2020-08-07T15:12:31.577 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7989999999999999e-06 2020-08-07T15:12:31.578 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.578 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.578 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:31.578 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.726801+0000", 2020-08-07T15:12:31.584 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30298438700000002 2020-08-07T15:12:31.585 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.585 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.586 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:31.586 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029786+0000", 2020-08-07T15:12:31.586 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5040000000000002e-06 2020-08-07T15:12:31.586 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.587 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.602 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:31.602 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029791+0000", 2020-08-07T15:12:31.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0737000000000001e-05 2020-08-07T15:12:31.602 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:31.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029822+0000", 2020-08-07T15:12:31.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9359999999999999e-06 2020-08-07T15:12:31.605 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.605 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.605 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:31.605 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029830+0000", 2020-08-07T15:12:31.605 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3083000000000004e-05 2020-08-07T15:12:31.606 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.606 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.606 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:31.606 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029913+0000", 2020-08-07T15:12:31.606 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30314883199999998 2020-08-07T15:12:31.607 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.607 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:31.607 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.607 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.607 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.608 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:8432 6.e 6:7fd18db4:::benchmark_data_smithi074_63843_object1053:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e235)", 2020-08-07T15:12:31.608 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:17.761348+0000", 2020-08-07T15:12:31.608 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 13.634326720000001, 2020-08-07T15:12:31.608 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.216116636, 2020-08-07T15:12:31.608 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:31.609 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:31.609 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:31.609 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:31.609 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:31.609 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 8432 2020-08-07T15:12:31.609 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.610 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:31.610 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.610 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:31.610 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.761348+0000", 2020-08-07T15:12:31.610 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:31.611 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.611 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.611 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:31.611 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.761348+0000", 2020-08-07T15:12:31.611 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9784999999999999e-05 2020-08-07T15:12:31.612 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.612 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.612 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:31.612 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.761398+0000", 2020-08-07T15:12:31.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1510000000000003e-06 2020-08-07T15:12:31.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.613 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:31.613 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.761403+0000", 2020-08-07T15:12:31.613 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0467e-05 2020-08-07T15:12:31.614 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.614 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.614 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:31.614 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.761414+0000", 2020-08-07T15:12:31.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.967e-06 2020-08-07T15:12:31.614 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.615 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.615 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:31.615 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.761419+0000", 2020-08-07T15:12:31.616 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0027405929999999999 2020-08-07T15:12:31.617 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.618 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.618 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:31.618 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.764159+0000", 2020-08-07T15:12:31.618 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8010000000000005e-05 2020-08-07T15:12:31.619 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.619 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.619 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:31.619 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.764257+0000", 2020-08-07T15:12:31.619 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000162404 2020-08-07T15:12:31.620 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.620 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.621 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:12:31.621 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.764420+0000", 2020-08-07T15:12:31.621 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0031968280000000001 2020-08-07T15:12:31.622 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.622 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.622 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:31.622 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.767617+0000", 2020-08-07T15:12:31.622 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0900000000000001e-07 2020-08-07T15:12:31.623 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.623 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.624 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:31.624 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.767617+0000", 2020-08-07T15:12:31.624 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7429999999999997e-05 2020-08-07T15:12:31.626 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.627 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:31.628 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.767675+0000", 2020-08-07T15:12:31.628 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013540640000000001 2020-08-07T15:12:31.628 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.628 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.629 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:31.629 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.769029+0000", 2020-08-07T15:12:31.629 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.208354435 2020-08-07T15:12:31.630 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.630 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.630 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:31.631 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.977383+0000", 2020-08-07T15:12:31.631 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6599999999999994e-07 2020-08-07T15:12:31.631 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.631 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.632 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:31.632 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.977384+0000", 2020-08-07T15:12:31.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0927000000000006e-05 2020-08-07T15:12:31.632 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.632 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.633 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:31.633 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.977465+0000", 2020-08-07T15:12:31.633 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.216116636 2020-08-07T15:12:31.634 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.634 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:31.634 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.635 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.635 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.635 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:10224 6.e 6:77f719e6:::benchmark_data_smithi074_63843_object1277:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e236)", 2020-08-07T15:12:31.635 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:19.038704+0000", 2020-08-07T15:12:31.635 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 12.356971570000001, 2020-08-07T15:12:31.637 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32955690900000001, 2020-08-07T15:12:31.637 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:31.638 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:31.638 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:31.639 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:31.639 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:31.639 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 10224 2020-08-07T15:12:31.639 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.639 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:31.640 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.640 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:31.640 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038704+0000", 2020-08-07T15:12:31.640 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:31.640 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.642 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.642 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:31.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038704+0000", 2020-08-07T15:12:31.642 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6672000000000001e-05 2020-08-07T15:12:31.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.643 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.643 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:31.643 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038730+0000", 2020-08-07T15:12:31.643 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.27e-06 2020-08-07T15:12:31.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.644 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.644 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:31.645 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038733+0000", 2020-08-07T15:12:31.645 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4030000000000004e-06 2020-08-07T15:12:31.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.646 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.648 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:31.648 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038740+0000", 2020-08-07T15:12:31.649 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.712e-06 2020-08-07T15:12:31.649 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.649 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.650 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:31.650 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038743+0000", 2020-08-07T15:12:31.650 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0025767279999999999 2020-08-07T15:12:31.650 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.651 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.651 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:31.651 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.041319+0000", 2020-08-07T15:12:31.652 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4727e-05 2020-08-07T15:12:31.652 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.652 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.652 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:31.653 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.041374+0000", 2020-08-07T15:12:31.653 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9403999999999999e-05 2020-08-07T15:12:31.653 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.653 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.653 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:12:31.654 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.041473+0000", 2020-08-07T15:12:31.654 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010041290000000001 2020-08-07T15:12:31.654 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.654 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.654 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:31.655 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.042478+0000", 2020-08-07T15:12:31.656 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0100000000000003e-07 2020-08-07T15:12:31.656 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.656 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.656 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:31.656 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.042478+0000", 2020-08-07T15:12:31.657 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7923999999999995e-05 2020-08-07T15:12:31.657 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.657 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.657 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:31.657 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.042546+0000", 2020-08-07T15:12:31.658 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0046169310000000003 2020-08-07T15:12:31.662 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.663 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.663 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:31.663 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.047163+0000", 2020-08-07T15:12:31.663 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.321023999 2020-08-07T15:12:31.664 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.664 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:31.664 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368187+0000", 2020-08-07T15:12:31.664 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.048e-06 2020-08-07T15:12:31.665 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.665 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.667 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:31.668 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368188+0000", 2020-08-07T15:12:31.668 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2061e-05 2020-08-07T15:12:31.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.684 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:31.684 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368260+0000", 2020-08-07T15:12:31.685 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32955690900000001 2020-08-07T15:12:31.685 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.685 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:31.685 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.685 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.686 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.686 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:10224 6.e e236/230 ondisk, result = 0)", 2020-08-07T15:12:31.686 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:19.047929+0000", 2020-08-07T15:12:31.686 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 12.34774597, 2020-08-07T15:12:31.686 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32041415400000001, 2020-08-07T15:12:31.687 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:31.687 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:12:31.687 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:31.687 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.687 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:31.688 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.047929+0000", 2020-08-07T15:12:31.688 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:31.688 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.688 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.688 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:31.689 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.047929+0000", 2020-08-07T15:12:31.689 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9706999999999999e-05 2020-08-07T15:12:31.689 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.691 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.691 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:31.692 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.047959+0000", 2020-08-07T15:12:31.692 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9100000000000004e-06 2020-08-07T15:12:31.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.692 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:31.693 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.047964+0000", 2020-08-07T15:12:31.693 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31873185399999998 2020-08-07T15:12:31.693 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.693 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.693 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:31.694 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.366696+0000", 2020-08-07T15:12:31.694 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2970000000000003e-06 2020-08-07T15:12:31.694 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.694 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.694 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:31.695 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.366701+0000", 2020-08-07T15:12:31.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001472522 2020-08-07T15:12:31.695 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.695 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:31.696 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368173+0000", 2020-08-07T15:12:31.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5949999999999998e-06 2020-08-07T15:12:31.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.696 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:31.696 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368179+0000", 2020-08-07T15:12:31.697 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016426900000000001 2020-08-07T15:12:31.697 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:31.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368343+0000", 2020-08-07T15:12:31.698 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32041415400000001 2020-08-07T15:12:31.698 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.698 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:31.698 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.699 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.699 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:11399 6.4 6:24f306ff:::benchmark_data_smithi074_63843_object1424:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e237)", 2020-08-07T15:12:31.699 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:19.754048+0000", 2020-08-07T15:12:31.699 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.641627282, 2020-08-07T15:12:31.699 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.261007094, 2020-08-07T15:12:31.700 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:31.700 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:31.701 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:31.701 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:31.701 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:31.704 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 11399 2020-08-07T15:12:31.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.704 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:31.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.705 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:31.705 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.754048+0000", 2020-08-07T15:12:31.705 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:31.705 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.705 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.706 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:31.706 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.754048+0000", 2020-08-07T15:12:31.706 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1152000000000001e-05 2020-08-07T15:12:31.706 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.706 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.707 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:31.707 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.754099+0000", 2020-08-07T15:12:31.707 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4060000000000002e-06 2020-08-07T15:12:31.707 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.707 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:31.708 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.754105+0000", 2020-08-07T15:12:31.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0482000000000001e-05 2020-08-07T15:12:31.708 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.708 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:31.709 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.754116+0000", 2020-08-07T15:12:31.709 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3080000000000004e-06 2020-08-07T15:12:31.709 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.709 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.709 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:31.710 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.754122+0000", 2020-08-07T15:12:31.710 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001128581 2020-08-07T15:12:31.710 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.711 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:31.711 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.755251+0000", 2020-08-07T15:12:31.711 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1481000000000003e-05 2020-08-07T15:12:31.711 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.712 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.712 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:31.712 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.755312+0000", 2020-08-07T15:12:31.712 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000104462 2020-08-07T15:12:31.712 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.713 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.713 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,5", 2020-08-07T15:12:31.713 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.755417+0000", 2020-08-07T15:12:31.713 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043585439999999998 2020-08-07T15:12:31.713 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.726 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.727 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:31.727 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.759775+0000", 2020-08-07T15:12:31.727 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.08e-06 2020-08-07T15:12:31.727 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:31.728 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.759776+0000", 2020-08-07T15:12:31.728 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7015999999999997e-05 2020-08-07T15:12:31.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.729 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:31.729 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.759843+0000", 2020-08-07T15:12:31.729 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.023840996999999999 2020-08-07T15:12:31.729 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.762 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:31.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.783684+0000", 2020-08-07T15:12:31.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.23130477699999999 2020-08-07T15:12:31.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.763 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:31.763 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:20.014989+0000", 2020-08-07T15:12:31.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3379999999999999e-06 2020-08-07T15:12:31.764 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.764 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:31.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:20.014990+0000", 2020-08-07T15:12:31.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4469999999999998e-05 2020-08-07T15:12:31.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.765 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:31.765 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:20.015055+0000", 2020-08-07T15:12:31.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.261007094 2020-08-07T15:12:31.766 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.766 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:31.766 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.766 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.767 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:11400 6.4 6:24f306ff:::benchmark_data_smithi074_63843_object1424:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e237)", 2020-08-07T15:12:31.767 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:19.754156+0000", 2020-08-07T15:12:31.767 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 11.641519169, 2020-08-07T15:12:31.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.261511936, 2020-08-07T15:12:31.768 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:31.768 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:31.768 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:31.768 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:31.768 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:31.769 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 11400 2020-08-07T15:12:31.769 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.769 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:31.769 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.769 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:31.770 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.754156+0000", 2020-08-07T15:12:31.770 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:31.770 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.770 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.770 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:31.771 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.754156+0000", 2020-08-07T15:12:31.771 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9839000000000002e-05 2020-08-07T15:12:31.771 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.771 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:31.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.754206+0000", 2020-08-07T15:12:31.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4650000000000002e-06 2020-08-07T15:12:31.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.779 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:31.779 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.754211+0000", 2020-08-07T15:12:31.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2354000000000001e-05 2020-08-07T15:12:31.780 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.780 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:31.780 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.754224+0000", 2020-08-07T15:12:31.781 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2510000000000002e-06 2020-08-07T15:12:31.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.781 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:31.781 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.754229+0000", 2020-08-07T15:12:31.789 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0016718530000000001 2020-08-07T15:12:31.789 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:31.791 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.755901+0000", 2020-08-07T15:12:31.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7055000000000002e-05 2020-08-07T15:12:31.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.792 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.792 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:31.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.755958+0000", 2020-08-07T15:12:31.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000150039 2020-08-07T15:12:31.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.793 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.793 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,5", 2020-08-07T15:12:31.793 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.756108+0000", 2020-08-07T15:12:31.793 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038638320000000002 2020-08-07T15:12:31.802 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:31.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.759972+0000", 2020-08-07T15:12:31.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.063e-06 2020-08-07T15:12:31.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.810 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.810 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:31.810 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.759973+0000", 2020-08-07T15:12:31.810 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2852999999999998e-05 2020-08-07T15:12:31.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.811 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:31.811 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.760016+0000", 2020-08-07T15:12:31.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.029835351 2020-08-07T15:12:31.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:31.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.789851+0000", 2020-08-07T15:12:31.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22575097099999999 2020-08-07T15:12:31.814 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.831 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.831 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:31.832 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:20.015602+0000", 2020-08-07T15:12:31.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5199999999999995e-07 2020-08-07T15:12:31.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.832 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:31.833 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:20.015603+0000", 2020-08-07T15:12:31.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5158000000000002e-05 2020-08-07T15:12:31.833 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:31.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:20.015668+0000", 2020-08-07T15:12:31.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.261511936 2020-08-07T15:12:31.834 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.834 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:31.834 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.835 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:17215 6.e 6:7a7ac88c:::benchmark_data_smithi074_63843_object2151:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e240)", 2020-08-07T15:12:31.835 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:23.659016+0000", 2020-08-07T15:12:31.835 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 7.7366592909999996, 2020-08-07T15:12:31.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.215546025, 2020-08-07T15:12:31.836 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:31.836 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:31.836 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:31.836 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:31.836 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:31.837 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 17215 2020-08-07T15:12:31.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.837 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:31.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:31.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.659016+0000", 2020-08-07T15:12:31.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:31.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.839 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:31.840 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.659016+0000", 2020-08-07T15:12:31.840 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7922e-05 2020-08-07T15:12:31.840 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:31.842 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.659064+0000", 2020-08-07T15:12:31.842 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2570000000000004e-06 2020-08-07T15:12:31.842 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.842 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.842 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:31.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.659070+0000", 2020-08-07T15:12:31.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2492e-05 2020-08-07T15:12:31.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.843 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:31.844 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.659082+0000", 2020-08-07T15:12:31.844 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0099999999999998e-06 2020-08-07T15:12:31.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.844 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:31.844 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.659089+0000", 2020-08-07T15:12:31.845 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0018134030000000001 2020-08-07T15:12:31.845 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:31.845 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.660903+0000", 2020-08-07T15:12:31.846 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7571999999999993e-05 2020-08-07T15:12:31.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.846 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.846 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:31.846 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.661000+0000", 2020-08-07T15:12:31.847 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017270799999999999 2020-08-07T15:12:31.847 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.847 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.847 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:12:31.847 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.661173+0000", 2020-08-07T15:12:31.848 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0020130840000000001 2020-08-07T15:12:31.848 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.848 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.848 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:31.861 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.663186+0000", 2020-08-07T15:12:31.862 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8700000000000001e-07 2020-08-07T15:12:31.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:31.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.663187+0000", 2020-08-07T15:12:31.863 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7921000000000003e-05 2020-08-07T15:12:31.864 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:31.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.663245+0000", 2020-08-07T15:12:31.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13881302200000001 2020-08-07T15:12:31.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.865 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:31.865 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.802058+0000", 2020-08-07T15:12:31.865 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.072442866999999994 2020-08-07T15:12:31.866 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.866 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.866 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:31.866 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.874500+0000", 2020-08-07T15:12:31.866 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0330000000000001e-06 2020-08-07T15:12:31.867 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.867 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.867 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:31.867 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.874501+0000", 2020-08-07T15:12:31.867 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0346999999999997e-05 2020-08-07T15:12:31.868 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.868 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.868 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:31.868 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.874562+0000", 2020-08-07T15:12:31.868 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.215546025 2020-08-07T15:12:31.869 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.869 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:31.869 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.869 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.869 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.870 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:17216 6.e 6:7a7ac88c:::benchmark_data_smithi074_63843_object2151:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e240)", 2020-08-07T15:12:31.885 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:23.659125+0000", 2020-08-07T15:12:31.885 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 7.7365498610000003, 2020-08-07T15:12:31.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22101887000000001, 2020-08-07T15:12:31.886 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:31.886 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:31.886 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:31.886 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:31.886 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:31.887 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 17216 2020-08-07T15:12:31.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.887 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:31.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:31.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.659125+0000", 2020-08-07T15:12:31.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:31.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:31.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.659125+0000", 2020-08-07T15:12:31.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3327e-05 2020-08-07T15:12:31.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:31.906 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.659179+0000", 2020-08-07T15:12:31.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.259e-06 2020-08-07T15:12:31.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:31.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.659184+0000", 2020-08-07T15:12:31.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1090999999999999e-05 2020-08-07T15:12:31.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.908 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:31.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.659195+0000", 2020-08-07T15:12:31.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7480000000000001e-06 2020-08-07T15:12:31.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:31.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.659200+0000", 2020-08-07T15:12:31.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002822144 2020-08-07T15:12:31.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.910 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:31.910 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.662022+0000", 2020-08-07T15:12:31.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010464800000000001 2020-08-07T15:12:31.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:31.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.662126+0000", 2020-08-07T15:12:31.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019341599999999999 2020-08-07T15:12:31.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:12:31.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.662320+0000", 2020-08-07T15:12:31.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00093403799999999999 2020-08-07T15:12:31.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:31.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.663254+0000", 2020-08-07T15:12:31.914 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3900000000000005e-07 2020-08-07T15:12:31.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:31.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.663254+0000", 2020-08-07T15:12:31.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0612000000000003e-05 2020-08-07T15:12:31.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:31.942 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.663305+0000", 2020-08-07T15:12:31.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.13926165400000001 2020-08-07T15:12:31.942 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:31.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.802567+0000", 2020-08-07T15:12:31.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.077504864000000007 2020-08-07T15:12:31.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:31.946 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.880072+0000", 2020-08-07T15:12:31.946 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2929999999999999e-06 2020-08-07T15:12:31.946 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.946 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:31.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.880073+0000", 2020-08-07T15:12:31.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1236999999999996e-05 2020-08-07T15:12:31.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.962 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:31.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:23.880144+0000", 2020-08-07T15:12:31.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22101887000000001 2020-08-07T15:12:31.962 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.962 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:31.963 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:31.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.963 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:18480 6.1 6:8a2c1569:::benchmark_data_smithi074_63843_object2309:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e240)", 2020-08-07T15:12:31.963 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.367850+0000", 2020-08-07T15:12:31.964 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 7.0278255600000001, 2020-08-07T15:12:31.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41222172800000001, 2020-08-07T15:12:31.964 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:31.964 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:31.964 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:31.965 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:31.965 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:31.970 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18480 2020-08-07T15:12:31.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.970 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:31.983 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.984 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:31.984 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367850+0000", 2020-08-07T15:12:31.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:31.985 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.985 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:31.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367850+0000", 2020-08-07T15:12:31.985 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6174999999999999e-05 2020-08-07T15:12:31.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.986 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:31.986 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367876+0000", 2020-08-07T15:12:31.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.061e-06 2020-08-07T15:12:31.987 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.987 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:31.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367879+0000", 2020-08-07T15:12:31.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7899999999999996e-06 2020-08-07T15:12:31.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:31.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:31.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:31.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367885+0000", 2020-08-07T15:12:31.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6589999999999999e-06 2020-08-07T15:12:32.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:32.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367887+0000", 2020-08-07T15:12:32.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001651275 2020-08-07T15:12:32.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:32.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.369538+0000", 2020-08-07T15:12:32.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7794000000000006e-05 2020-08-07T15:12:32.009 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.009 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.010 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:32.010 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.369636+0000", 2020-08-07T15:12:32.010 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017274 2020-08-07T15:12:32.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.011 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:12:32.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.369809+0000", 2020-08-07T15:12:32.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010733769999999999 2020-08-07T15:12:32.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.012 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:32.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.370882+0000", 2020-08-07T15:12:32.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0100000000000003e-07 2020-08-07T15:12:32.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.013 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:32.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.370883+0000", 2020-08-07T15:12:32.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6001000000000001e-05 2020-08-07T15:12:32.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:32.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.370939+0000", 2020-08-07T15:12:32.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074658490000000001 2020-08-07T15:12:32.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:32.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.378405+0000", 2020-08-07T15:12:32.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40160148899999998 2020-08-07T15:12:32.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:32.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.780007+0000", 2020-08-07T15:12:32.030 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9700000000000005e-07 2020-08-07T15:12:32.030 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.030 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:32.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.780008+0000", 2020-08-07T15:12:32.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3720000000000007e-05 2020-08-07T15:12:32.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:32.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.780071+0000", 2020-08-07T15:12:32.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41222172800000001 2020-08-07T15:12:32.033 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:32.033 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:32.033 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:32.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.050 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.051 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:18480 6.1 e240/230 ondisk, result = 0)", 2020-08-07T15:12:32.051 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.375696+0000", 2020-08-07T15:12:32.051 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 7.0199789570000002, 2020-08-07T15:12:32.051 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.404380817, 2020-08-07T15:12:32.052 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:32.052 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:12:32.052 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:32.052 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.053 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:32.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.375696+0000", 2020-08-07T15:12:32.053 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:32.053 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.054 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.054 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:32.054 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.375696+0000", 2020-08-07T15:12:32.054 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0922e-05 2020-08-07T15:12:32.055 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.055 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:32.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.375727+0000", 2020-08-07T15:12:32.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2900000000000002e-06 2020-08-07T15:12:32.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:32.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.375732+0000", 2020-08-07T15:12:32.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.403431708 2020-08-07T15:12:32.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:32.066 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.779164+0000", 2020-08-07T15:12:32.066 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5200000000000002e-06 2020-08-07T15:12:32.067 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.067 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.067 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:32.067 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.779168+0000", 2020-08-07T15:12:32.068 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00082257200000000004 2020-08-07T15:12:32.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.068 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.068 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:32.069 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.779990+0000", 2020-08-07T15:12:32.069 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1599999999999998e-06 2020-08-07T15:12:32.069 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.070 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.070 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:32.070 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.779998+0000", 2020-08-07T15:12:32.070 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8645000000000003e-05 2020-08-07T15:12:32.070 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.071 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.071 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:32.071 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.780077+0000", 2020-08-07T15:12:32.071 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.404380817 2020-08-07T15:12:32.072 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:32.072 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:32.072 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:32.072 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.072 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.073 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:18560 6.e 6:7c5f8c5e:::benchmark_data_smithi074_63843_object2319:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e240)", 2020-08-07T15:12:32.073 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.407761+0000", 2020-08-07T15:12:32.073 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.9879136940000004, 2020-08-07T15:12:32.073 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31539320100000001, 2020-08-07T15:12:32.082 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:32.082 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:32.082 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:32.083 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:32.083 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:32.083 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18560 2020-08-07T15:12:32.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.084 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:32.084 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:32.084 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407761+0000", 2020-08-07T15:12:32.085 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:32.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.085 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:32.094 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407761+0000", 2020-08-07T15:12:32.094 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0897000000000003e-05 2020-08-07T15:12:32.094 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.095 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.095 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:32.095 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407822+0000", 2020-08-07T15:12:32.095 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8390000000000001e-06 2020-08-07T15:12:32.096 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.096 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.096 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:32.096 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407829+0000", 2020-08-07T15:12:32.097 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3609e-05 2020-08-07T15:12:32.097 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.097 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.097 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:32.106 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407843+0000", 2020-08-07T15:12:32.106 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3169999999999998e-06 2020-08-07T15:12:32.108 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.108 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.109 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:32.109 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407849+0000", 2020-08-07T15:12:32.109 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002504886 2020-08-07T15:12:32.110 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.110 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.110 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:32.110 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.410354+0000", 2020-08-07T15:12:32.111 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7235e-05 2020-08-07T15:12:32.111 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.111 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.111 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:32.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.410451+0000", 2020-08-07T15:12:32.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016856999999999999 2020-08-07T15:12:32.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:12:32.113 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.410620+0000", 2020-08-07T15:12:32.113 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026816259999999999 2020-08-07T15:12:32.113 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.121 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.121 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:32.121 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.413301+0000", 2020-08-07T15:12:32.122 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0020516309999999999 2020-08-07T15:12:32.122 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.122 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.122 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:32.123 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.415353+0000", 2020-08-07T15:12:32.123 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30769196399999998 2020-08-07T15:12:32.123 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.124 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.124 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:32.124 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.723045+0000", 2020-08-07T15:12:32.124 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1600000000000001e-07 2020-08-07T15:12:32.124 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.125 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.125 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:32.125 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.723046+0000", 2020-08-07T15:12:32.125 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1449000000000001e-05 2020-08-07T15:12:32.138 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.138 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.139 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:32.139 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.723107+0000", 2020-08-07T15:12:32.139 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7462000000000001e-05 2020-08-07T15:12:32.140 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.140 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.140 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:32.140 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.723155+0000", 2020-08-07T15:12:32.141 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31539320100000001 2020-08-07T15:12:32.141 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:32.141 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:32.141 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:32.142 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.142 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.142 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:18608 6.4 6:2dddf5bf:::benchmark_data_smithi074_63843_object2325:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e240)", 2020-08-07T15:12:32.142 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.438031+0000", 2020-08-07T15:12:32.146 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.9576442979999999, 2020-08-07T15:12:32.146 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28378579100000001, 2020-08-07T15:12:32.146 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:32.147 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:32.147 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:32.147 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:32.148 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:32.148 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18608 2020-08-07T15:12:32.148 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.148 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:32.149 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.149 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:32.149 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438031+0000", 2020-08-07T15:12:32.149 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:32.160 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.160 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.161 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:32.161 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438031+0000", 2020-08-07T15:12:32.161 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0028999999999998e-05 2020-08-07T15:12:32.161 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.162 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.162 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:32.162 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438081+0000", 2020-08-07T15:12:32.162 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7289999999999997e-06 2020-08-07T15:12:32.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.163 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.163 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:32.163 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438087+0000", 2020-08-07T15:12:32.163 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0959e-05 2020-08-07T15:12:32.164 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:32.164 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438098+0000", 2020-08-07T15:12:32.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0329999999999999e-06 2020-08-07T15:12:32.165 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.165 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.165 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:32.166 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438103+0000", 2020-08-07T15:12:32.166 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001770911 2020-08-07T15:12:32.166 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.166 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.167 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:32.167 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.439873+0000", 2020-08-07T15:12:32.167 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3503000000000005e-05 2020-08-07T15:12:32.167 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.168 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.168 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:32.168 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.439937+0000", 2020-08-07T15:12:32.168 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000110179 2020-08-07T15:12:32.169 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.181 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,5", 2020-08-07T15:12:32.182 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.440047+0000", 2020-08-07T15:12:32.183 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033910759999999998 2020-08-07T15:12:32.183 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.183 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.184 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:32.184 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443438+0000", 2020-08-07T15:12:32.184 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2877999999999999e-05 2020-08-07T15:12:32.185 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.185 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.185 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:32.186 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443471+0000", 2020-08-07T15:12:32.186 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27823471799999999 2020-08-07T15:12:32.186 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.197 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.197 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:32.198 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721706+0000", 2020-08-07T15:12:32.198 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4300000000000003e-07 2020-08-07T15:12:32.199 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.199 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.199 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:32.200 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721706+0000", 2020-08-07T15:12:32.200 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2877000000000001e-05 2020-08-07T15:12:32.202 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.203 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.203 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:32.203 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721769+0000", 2020-08-07T15:12:32.204 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7355999999999999e-05 2020-08-07T15:12:32.204 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.204 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.205 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:32.205 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721817+0000", 2020-08-07T15:12:32.205 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28378579100000001 2020-08-07T15:12:32.205 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:32.210 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:32.210 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:32.217 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.217 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.217 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:18608 6.4 e240/230 ondisk, result = 0)", 2020-08-07T15:12:32.218 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.443879+0000", 2020-08-07T15:12:32.218 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.9517960600000004, 2020-08-07T15:12:32.219 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.277941838, 2020-08-07T15:12:32.219 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:32.219 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:12:32.220 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:32.220 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.220 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:32.220 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443879+0000", 2020-08-07T15:12:32.221 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:32.221 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.221 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.221 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:32.225 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443879+0000", 2020-08-07T15:12:32.225 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3692000000000002e-05 2020-08-07T15:12:32.225 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.226 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.226 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:32.226 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443913+0000", 2020-08-07T15:12:32.227 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.02e-06 2020-08-07T15:12:32.227 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.227 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.228 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:32.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443919+0000", 2020-08-07T15:12:32.228 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27734076499999999 2020-08-07T15:12:32.229 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.229 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.241 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:32.242 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721260+0000", 2020-08-07T15:12:32.242 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.78e-06 2020-08-07T15:12:32.243 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.244 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.244 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:32.244 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721264+0000", 2020-08-07T15:12:32.245 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00042663999999999999 2020-08-07T15:12:32.245 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.245 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.246 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:32.246 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721691+0000", 2020-08-07T15:12:32.246 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1509999999999998e-06 2020-08-07T15:12:32.247 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.249 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.250 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:32.250 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721698+0000", 2020-08-07T15:12:32.250 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012279000000000001 2020-08-07T15:12:32.251 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.251 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.251 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:32.252 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721821+0000", 2020-08-07T15:12:32.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.277941838 2020-08-07T15:12:32.252 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:32.262 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:32.263 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:32.263 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:32.263 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:32.264 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:32.270 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:12:32.270 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:12:32.271 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:12:32.271 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:12:32.271 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:12:32.272 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:12:32.275 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:12:32.276 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:12:32.276 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:12:32.276 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:12:32.277 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:12:32.279 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:2020-08-07T15:12:32.267142+0000 min lat: 0.000920324 max lat: 0.380184 avg lat: 0.0105092 2020-08-07T15:12:32.280 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-08-07T15:12:32.280 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 20 16 30428 30412 11.8779 12.4062 0.0551037 0.0105092 2020-08-07T15:12:32.280 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:32.264+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:32.281 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:12:32.373 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:32.368+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:32.477 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:32.473+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:32.498 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:32.498 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:12:32.499 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.499 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5481.0:30791 6.0 e245/230 6:07e644ba:::benchmark_data_smithi074_63843_object3848:head v 245'1855, mlcod=245'1847)", 2020-08-07T15:12:32.499 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:32.492590+0000", 2020-08-07T15:12:32.499 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.00257164, 2020-08-07T15:12:32.500 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0025896349999999999, 2020-08-07T15:12:32.500 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:32.500 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:12:32.500 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:32.500 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.501 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:32.501 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:32.492590+0000", 2020-08-07T15:12:32.501 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:32.501 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.501 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.502 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:32.502 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:32.492590+0000", 2020-08-07T15:12:32.502 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9101999999999998e-05 2020-08-07T15:12:32.502 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.502 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.503 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:32.503 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:32.492639+0000", 2020-08-07T15:12:32.503 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5250000000000002e-06 2020-08-07T15:12:32.503 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.503 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.504 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:32.504 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:32.492645+0000", 2020-08-07T15:12:32.514 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8397999999999999e-05 2020-08-07T15:12:32.515 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.515 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.515 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:32.515 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:32.492704+0000", 2020-08-07T15:12:32.515 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5629999999999999e-06 2020-08-07T15:12:32.516 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.516 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.516 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:32.516 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:32.492709+0000", 2020-08-07T15:12:32.516 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000167505 2020-08-07T15:12:32.517 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.517 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.517 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:32.517 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:32.492877+0000", 2020-08-07T15:12:32.517 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1691e-05 2020-08-07T15:12:32.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.518 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:32.518 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:32.492898+0000", 2020-08-07T15:12:32.519 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00030878399999999999 2020-08-07T15:12:32.519 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:32.519 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:32.519 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:32.519 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.520 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.520 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5481.0:30792 6.0 e245/230 6:07e644ba:::benchmark_data_smithi074_63843_object3848:head v 245'1856, mlcod=245'1847)", 2020-08-07T15:12:32.520 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:32.493277+0000", 2020-08-07T15:12:32.520 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.001884021, 2020-08-07T15:12:32.520 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0020581409999999999, 2020-08-07T15:12:32.521 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:32.521 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:12:32.521 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:32.521 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.521 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:32.524 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:32.493277+0000", 2020-08-07T15:12:32.524 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:32.525 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.525 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.525 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:32.525 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:32.493277+0000", 2020-08-07T15:12:32.525 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8674000000000002e-05 2020-08-07T15:12:32.526 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.526 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.526 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:32.526 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:32.493326+0000", 2020-08-07T15:12:32.526 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5579999999999997e-06 2020-08-07T15:12:32.527 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.527 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.527 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:32.527 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:32.493333+0000", 2020-08-07T15:12:32.527 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4558999999999997e-05 2020-08-07T15:12:32.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.528 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:32.528 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:32.493387+0000", 2020-08-07T15:12:32.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2209999999999996e-06 2020-08-07T15:12:32.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.529 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.529 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:32.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:32.493392+0000", 2020-08-07T15:12:32.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00029925399999999999 2020-08-07T15:12:32.529 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.530 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.530 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:32.530 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:32.493692+0000", 2020-08-07T15:12:32.530 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1712000000000002e-05 2020-08-07T15:12:32.530 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.531 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.531 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:32.531 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:32.493713+0000", 2020-08-07T15:12:32.532 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00043597800000000001 2020-08-07T15:12:32.532 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:32.532 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:32.532 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:32.532 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:32.533 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:12:32.533 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 2 2020-08-07T15:12:32.533 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:32.552 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:12:32.610 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:32.605+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:32.679 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:32.677+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:32.750 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:32.751 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:12:32.751 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:12:32.751 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:12:32.751 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:32.761 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:12:32.803 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:32.801+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:32.883 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:32.881+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:32.965 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:32.966 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:12:32.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:12:32.966 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:12:32.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.966 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:145801 3.6 e75/26 3:69e4cb22:::benchmark_data_smithi074_35447_object16192:head v 75'9405, mlcod=75'9401)", 2020-08-07T15:12:32.967 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:12:32.967 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 258.28969145399998, 2020-08-07T15:12:32.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639, 2020-08-07T15:12:32.967 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:32.967 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:32.968 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:32.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:32.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:12:32.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:32.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:32.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:12:32.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6511000000000002e-05 2020-08-07T15:12:32.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.985 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:32.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671803+0000", 2020-08-07T15:12:32.985 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7190000000000004e-06 2020-08-07T15:12:32.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.986 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:32.986 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671809+0000", 2020-08-07T15:12:32.986 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74210928300000001 2020-08-07T15:12:32.987 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.987 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:32.987 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413919+0000", 2020-08-07T15:12:32.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1759999999999999e-05 2020-08-07T15:12:32.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:32.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413930+0000", 2020-08-07T15:12:32.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000108601 2020-08-07T15:12:32.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:32.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414039+0000", 2020-08-07T15:12:32.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9948999999999999e-05 2020-08-07T15:12:32.990 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.990 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.990 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:32.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414089+0000", 2020-08-07T15:12:32.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00079656499999999995 2020-08-07T15:12:32.990 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:32.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414885+0000", 2020-08-07T15:12:32.991 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2250999999999998e-05 2020-08-07T15:12:32.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:32.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414938+0000", 2020-08-07T15:12:32.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639 2020-08-07T15:12:32.992 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:32.993 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:32.993 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:32.993 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.993 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.994 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:104104 4.0 e146/131 4:0ffb764a:::benchmark_data_smithi074_44586_object13012:head v 146'6960, mlcod=146'6953)", 2020-08-07T15:12:32.994 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:12:32.994 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 164.763144678, 2020-08-07T15:12:32.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75443331599999996, 2020-08-07T15:12:32.994 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:32.995 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:32.995 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:32.995 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.995 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:32.995 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:12:32.996 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:32.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.996 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.996 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:32.996 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:12:32.997 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1743000000000002e-05 2020-08-07T15:12:32.997 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.997 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.997 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:32.997 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198355+0000", 2020-08-07T15:12:32.998 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1249999999999998e-06 2020-08-07T15:12:32.998 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.998 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.998 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:32.998 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198361+0000", 2020-08-07T15:12:32.999 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74719179599999996 2020-08-07T15:12:32.999 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:32.999 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:32.999 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:32.999 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945553+0000", 2020-08-07T15:12:32.999 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6800000000000001e-06 2020-08-07T15:12:33.000 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.000 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.000 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:33.000 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945557+0000", 2020-08-07T15:12:33.000 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012101500000000001 2020-08-07T15:12:33.001 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.001 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.001 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.001 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945678+0000", 2020-08-07T15:12:33.002 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6621000000000001e-05 2020-08-07T15:12:33.002 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:33.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945705+0000", 2020-08-07T15:12:33.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069879540000000002 2020-08-07T15:12:33.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:33.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.952693+0000", 2020-08-07T15:12:33.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3381999999999999e-05 2020-08-07T15:12:33.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:33.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.952736+0000", 2020-08-07T15:12:33.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75443331599999996 2020-08-07T15:12:33.007 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.007 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:33.007 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.008 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89641 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:33.008 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.550933+0000", 2020-08-07T15:12:33.008 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 71.410514257000003, 2020-08-07T15:12:33.008 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69051440900000005, 2020-08-07T15:12:33.008 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:33.009 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:33.009 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:33.009 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:33.009 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:33.009 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89641 2020-08-07T15:12:33.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.011 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:33.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.011 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:33.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550933+0000", 2020-08-07T15:12:33.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:33.012 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.012 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:33.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550933+0000", 2020-08-07T15:12:33.013 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6520000000000002e-05 2020-08-07T15:12:33.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:33.014 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550980+0000", 2020-08-07T15:12:33.014 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2410000000000003e-06 2020-08-07T15:12:33.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.015 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.015 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:33.015 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550985+0000", 2020-08-07T15:12:33.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2707e-05 2020-08-07T15:12:33.016 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:33.017 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550998+0000", 2020-08-07T15:12:33.017 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8629999999999996e-06 2020-08-07T15:12:33.018 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.018 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.018 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:33.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551004+0000", 2020-08-07T15:12:33.019 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4987000000000001e-05 2020-08-07T15:12:33.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.020 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.020 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.020 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551059+0000", 2020-08-07T15:12:33.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1718e-05 2020-08-07T15:12:33.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:33.022 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551070+0000", 2020-08-07T15:12:33.022 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67720431000000003 2020-08-07T15:12:33.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.023 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.228275+0000", 2020-08-07T15:12:33.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000731965 2020-08-07T15:12:33.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:33.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229007+0000", 2020-08-07T15:12:33.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00025396199999999999 2020-08-07T15:12:33.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:33.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229261+0000", 2020-08-07T15:12:33.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002511189 2020-08-07T15:12:33.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:33.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231772+0000", 2020-08-07T15:12:33.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3399999999999997e-07 2020-08-07T15:12:33.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.030 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.030 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:33.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231773+0000", 2020-08-07T15:12:33.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0274e-05 2020-08-07T15:12:33.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:33.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231823+0000", 2020-08-07T15:12:33.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0094095440000000006 2020-08-07T15:12:33.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.033 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241233+0000", 2020-08-07T15:12:33.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4011999999999998e-05 2020-08-07T15:12:33.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.034 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.034 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.034 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241277+0000", 2020-08-07T15:12:33.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7199999999999999e-07 2020-08-07T15:12:33.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:33.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241277+0000", 2020-08-07T15:12:33.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017061100000000001 2020-08-07T15:12:33.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:33.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241448+0000", 2020-08-07T15:12:33.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69051440900000005 2020-08-07T15:12:33.036 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.037 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:33.037 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.037 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89642 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:33.038 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.551807+0000", 2020-08-07T15:12:33.038 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 71.409640705000001, 2020-08-07T15:12:33.038 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68991613100000004, 2020-08-07T15:12:33.039 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:33.039 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:33.039 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:33.039 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:33.039 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:33.040 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89642 2020-08-07T15:12:33.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.040 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:33.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:33.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551807+0000", 2020-08-07T15:12:33.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:33.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.041 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:33.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551807+0000", 2020-08-07T15:12:33.042 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7049000000000001e-05 2020-08-07T15:12:33.042 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:33.043 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551834+0000", 2020-08-07T15:12:33.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9249999999999997e-06 2020-08-07T15:12:33.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.044 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:33.044 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551838+0000", 2020-08-07T15:12:33.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9160000000000001e-06 2020-08-07T15:12:33.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:33.045 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551846+0000", 2020-08-07T15:12:33.045 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3510000000000002e-06 2020-08-07T15:12:33.045 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.045 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.046 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:33.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551849+0000", 2020-08-07T15:12:33.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9276000000000001e-05 2020-08-07T15:12:33.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551898+0000", 2020-08-07T15:12:33.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3095000000000001e-05 2020-08-07T15:12:33.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.062 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:33.062 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551911+0000", 2020-08-07T15:12:33.062 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67789159799999998 2020-08-07T15:12:33.062 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.062 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.062 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229803+0000", 2020-08-07T15:12:33.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8069000000000001e-05 2020-08-07T15:12:33.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.075 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:33.076 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229881+0000", 2020-08-07T15:12:33.076 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000152917 2020-08-07T15:12:33.076 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.077 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.077 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:33.077 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230034+0000", 2020-08-07T15:12:33.077 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044936760000000003 2020-08-07T15:12:33.078 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.078 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.078 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:33.079 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234528+0000", 2020-08-07T15:12:33.079 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8999999999999995e-07 2020-08-07T15:12:33.079 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.079 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.079 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:33.080 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234529+0000", 2020-08-07T15:12:33.080 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7741000000000002e-05 2020-08-07T15:12:33.080 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.080 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.080 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:33.081 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234586+0000", 2020-08-07T15:12:33.081 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069143959999999997 2020-08-07T15:12:33.081 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.081 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.081 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.082 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241501+0000", 2020-08-07T15:12:33.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1433000000000001e-05 2020-08-07T15:12:33.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.083 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.083 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.083 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241542+0000", 2020-08-07T15:12:33.084 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7800000000000001e-07 2020-08-07T15:12:33.084 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.085 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:33.085 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241543+0000", 2020-08-07T15:12:33.085 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000180321 2020-08-07T15:12:33.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.086 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.086 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:33.086 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241723+0000", 2020-08-07T15:12:33.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68991613100000004 2020-08-07T15:12:33.086 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.087 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:33.087 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.087 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89643 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:33.088 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.552181+0000", 2020-08-07T15:12:33.088 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 71.409266711000001, 2020-08-07T15:12:33.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68980872400000004, 2020-08-07T15:12:33.088 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:33.088 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:33.089 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:33.089 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:33.089 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:33.089 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89643 2020-08-07T15:12:33.089 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.090 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:33.090 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.091 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:33.091 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552181+0000", 2020-08-07T15:12:33.091 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:33.091 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.092 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.093 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:33.093 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552181+0000", 2020-08-07T15:12:33.093 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0926999999999997e-05 2020-08-07T15:12:33.093 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.094 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.094 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:33.094 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552212+0000", 2020-08-07T15:12:33.095 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0512e-05 2020-08-07T15:12:33.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.095 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.095 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:33.096 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552222+0000", 2020-08-07T15:12:33.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9379999999999992e-06 2020-08-07T15:12:33.096 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.096 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.096 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:33.097 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552230+0000", 2020-08-07T15:12:33.097 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0260000000000001e-06 2020-08-07T15:12:33.097 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.097 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.098 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:33.098 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552234+0000", 2020-08-07T15:12:33.098 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6109000000000002e-05 2020-08-07T15:12:33.098 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.098 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.099 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.099 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552280+0000", 2020-08-07T15:12:33.099 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5318000000000001e-05 2020-08-07T15:12:33.099 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.099 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.100 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:33.100 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552296+0000", 2020-08-07T15:12:33.100 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67811269100000005 2020-08-07T15:12:33.100 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.100 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.101 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.101 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230408+0000", 2020-08-07T15:12:33.101 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000125518 2020-08-07T15:12:33.101 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.101 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.102 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:33.103 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230534+0000", 2020-08-07T15:12:33.103 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024637999999999998 2020-08-07T15:12:33.104 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.105 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.105 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:33.105 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230780+0000", 2020-08-07T15:12:33.106 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0066398869999999997 2020-08-07T15:12:33.106 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.106 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.106 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:33.106 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237420+0000", 2020-08-07T15:12:33.107 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3500000000000005e-07 2020-08-07T15:12:33.107 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.107 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.107 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:33.107 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237421+0000", 2020-08-07T15:12:33.108 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4005000000000003e-05 2020-08-07T15:12:33.108 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.108 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.108 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:33.108 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237465+0000", 2020-08-07T15:12:33.109 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043133529999999998 2020-08-07T15:12:33.109 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.109 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.109 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.109 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241778+0000", 2020-08-07T15:12:33.110 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0592e-05 2020-08-07T15:12:33.110 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.110 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.110 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.110 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241819+0000", 2020-08-07T15:12:33.111 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4999999999999998e-07 2020-08-07T15:12:33.122 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.122 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.122 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:33.123 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241819+0000", 2020-08-07T15:12:33.123 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017018299999999999 2020-08-07T15:12:33.123 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.123 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.123 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:33.124 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241990+0000", 2020-08-07T15:12:33.124 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68980872400000004 2020-08-07T15:12:33.125 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.125 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:33.125 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.125 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.125 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.126 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89644 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:33.134 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.552384+0000", 2020-08-07T15:12:33.134 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 71.409063677000006, 2020-08-07T15:12:33.135 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68997479299999998, 2020-08-07T15:12:33.135 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:33.135 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:33.136 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:33.136 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:33.136 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:33.137 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89644 2020-08-07T15:12:33.137 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.137 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:33.138 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.138 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:33.138 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552384+0000", 2020-08-07T15:12:33.138 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:33.138 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.139 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.139 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:33.139 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552384+0000", 2020-08-07T15:12:33.139 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9238999999999999e-05 2020-08-07T15:12:33.139 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.140 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.142 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:33.142 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552413+0000", 2020-08-07T15:12:33.142 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3340000000000003e-06 2020-08-07T15:12:33.142 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.142 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.143 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:33.143 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552417+0000", 2020-08-07T15:12:33.143 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3119999999999994e-06 2020-08-07T15:12:33.143 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.143 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.144 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:33.144 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552426+0000", 2020-08-07T15:12:33.144 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4139999999999996e-06 2020-08-07T15:12:33.144 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.144 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.145 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:33.145 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552430+0000", 2020-08-07T15:12:33.154 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6613999999999999e-05 2020-08-07T15:12:33.154 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.154 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.154 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.155 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:12:33.155 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7280000000000007e-06 2020-08-07T15:12:33.155 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.155 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.155 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:33.156 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552477+0000", 2020-08-07T15:12:33.156 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67873990399999995 2020-08-07T15:12:33.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.156 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.157 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231216+0000", 2020-08-07T15:12:33.157 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3629999999999998e-05 2020-08-07T15:12:33.157 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.157 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.157 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:33.158 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231290+0000", 2020-08-07T15:12:33.158 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000132108 2020-08-07T15:12:33.158 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.158 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.158 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:33.159 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231422+0000", 2020-08-07T15:12:33.159 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006049652 2020-08-07T15:12:33.159 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.159 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.159 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:33.160 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237472+0000", 2020-08-07T15:12:33.160 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000005e-07 2020-08-07T15:12:33.160 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.160 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.160 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:33.160 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237472+0000", 2020-08-07T15:12:33.161 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2848000000000001e-05 2020-08-07T15:12:33.161 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.161 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.161 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:33.161 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237505+0000", 2020-08-07T15:12:33.162 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045352179999999997 2020-08-07T15:12:33.162 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.162 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.162 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.162 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242040+0000", 2020-08-07T15:12:33.163 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012843699999999999 2020-08-07T15:12:33.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.163 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.163 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.163 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242169+0000", 2020-08-07T15:12:33.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9999999999999998e-07 2020-08-07T15:12:33.164 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:33.164 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242169+0000", 2020-08-07T15:12:33.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000189354 2020-08-07T15:12:33.165 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.165 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.165 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:33.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242359+0000", 2020-08-07T15:12:33.166 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68997479299999998 2020-08-07T15:12:33.166 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.166 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:33.166 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.166 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.167 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.167 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89645 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:33.167 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:12:33.167 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 71.408980894999999, 2020-08-07T15:12:33.167 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69011194600000003, 2020-08-07T15:12:33.168 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:33.168 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:33.168 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:33.168 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:33.168 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:33.169 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89645 2020-08-07T15:12:33.169 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.169 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:33.169 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.169 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:33.170 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:12:33.170 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:33.170 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.170 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.170 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:33.171 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:12:33.171 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6750999999999998e-05 2020-08-07T15:12:33.171 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.171 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.171 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:33.172 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552493+0000", 2020-08-07T15:12:33.172 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0350000000000002e-06 2020-08-07T15:12:33.172 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.172 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.172 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:33.172 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552496+0000", 2020-08-07T15:12:33.173 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3029999999999996e-06 2020-08-07T15:12:33.173 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.173 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.173 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:33.174 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552503+0000", 2020-08-07T15:12:33.174 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7580000000000001e-06 2020-08-07T15:12:33.174 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.174 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.174 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:33.175 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552506+0000", 2020-08-07T15:12:33.175 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4868000000000003e-05 2020-08-07T15:12:33.175 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.175 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.175 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.175 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552570+0000", 2020-08-07T15:12:33.176 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9649999999999997e-06 2020-08-07T15:12:33.176 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.176 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.176 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:33.177 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552579+0000", 2020-08-07T15:12:33.177 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67928357699999997 2020-08-07T15:12:33.177 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.177 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.177 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.178 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231863+0000", 2020-08-07T15:12:33.178 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000115696 2020-08-07T15:12:33.178 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.178 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.178 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:33.178 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231979+0000", 2020-08-07T15:12:33.179 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020456599999999999 2020-08-07T15:12:33.179 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.179 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.179 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:33.179 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232183+0000", 2020-08-07T15:12:33.180 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005327923 2020-08-07T15:12:33.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.180 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.181 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:33.181 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237511+0000", 2020-08-07T15:12:33.181 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8999999999999998e-07 2020-08-07T15:12:33.181 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:33.182 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237511+0000", 2020-08-07T15:12:33.182 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6019000000000002e-05 2020-08-07T15:12:33.182 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.183 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.183 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:33.183 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237547+0000", 2020-08-07T15:12:33.183 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045340980000000003 2020-08-07T15:12:33.183 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.184 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.184 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.184 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242082+0000", 2020-08-07T15:12:33.184 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00032620499999999998 2020-08-07T15:12:33.184 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.184 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.185 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.185 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242408+0000", 2020-08-07T15:12:33.185 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3500000000000002e-07 2020-08-07T15:12:33.185 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.186 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.186 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:33.186 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242408+0000", 2020-08-07T15:12:33.186 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000170457 2020-08-07T15:12:33.187 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.188 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.188 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:33.188 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242579+0000", 2020-08-07T15:12:33.188 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69011194600000003 2020-08-07T15:12:33.188 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.189 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:33.189 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.189 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.189 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.189 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89646 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:33.190 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.557244+0000", 2020-08-07T15:12:33.190 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 71.404203123000002, 2020-08-07T15:12:33.190 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68559352299999998, 2020-08-07T15:12:33.190 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:33.190 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:33.191 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:33.191 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:33.191 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:33.191 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89646 2020-08-07T15:12:33.191 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.192 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:33.192 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.192 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:33.192 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557244+0000", 2020-08-07T15:12:33.192 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:33.193 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.193 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.193 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:33.193 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557244+0000", 2020-08-07T15:12:33.193 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9699000000000001e-05 2020-08-07T15:12:33.194 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.194 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.194 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:33.194 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557274+0000", 2020-08-07T15:12:33.194 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1269999999999996e-06 2020-08-07T15:12:33.194 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.195 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.195 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:33.195 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557278+0000", 2020-08-07T15:12:33.195 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0099999999999995e-06 2020-08-07T15:12:33.195 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.196 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.196 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:33.196 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557286+0000", 2020-08-07T15:12:33.196 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7439999999999998e-06 2020-08-07T15:12:33.196 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.197 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.197 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:33.197 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557291+0000", 2020-08-07T15:12:33.197 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5927999999999999e-05 2020-08-07T15:12:33.197 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.197 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.198 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.198 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557327+0000", 2020-08-07T15:12:33.198 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5069999999999997e-06 2020-08-07T15:12:33.198 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.198 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.199 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:33.199 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557333+0000", 2020-08-07T15:12:33.199 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67527419200000005 2020-08-07T15:12:33.199 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.199 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.200 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.200 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232608+0000", 2020-08-07T15:12:33.200 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8699e-05 2020-08-07T15:12:33.200 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.200 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.201 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:33.201 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232686+0000", 2020-08-07T15:12:33.201 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018185099999999999 2020-08-07T15:12:33.201 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.201 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.201 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:33.202 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232868+0000", 2020-08-07T15:12:33.202 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0075819360000000001 2020-08-07T15:12:33.202 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.202 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.202 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:33.203 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240450+0000", 2020-08-07T15:12:33.203 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2799999999999995e-07 2020-08-07T15:12:33.203 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.203 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.203 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:33.204 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240451+0000", 2020-08-07T15:12:33.204 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3751000000000002e-05 2020-08-07T15:12:33.204 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.204 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.205 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:33.205 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240495+0000", 2020-08-07T15:12:33.205 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001628064 2020-08-07T15:12:33.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.205 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.206 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.206 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242123+0000", 2020-08-07T15:12:33.206 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00050955599999999998 2020-08-07T15:12:33.206 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.206 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.207 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.207 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242632+0000", 2020-08-07T15:12:33.207 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2900000000000004e-07 2020-08-07T15:12:33.207 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.207 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.208 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:33.208 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242633+0000", 2020-08-07T15:12:33.208 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000205202 2020-08-07T15:12:33.208 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.208 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.209 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:33.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242838+0000", 2020-08-07T15:12:33.209 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68559352299999998 2020-08-07T15:12:33.209 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.209 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:33.210 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.210 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.210 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.210 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89647 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:33.210 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.557315+0000", 2020-08-07T15:12:33.211 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 71.404132799999999, 2020-08-07T15:12:33.211 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68581815800000001, 2020-08-07T15:12:33.211 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:33.211 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:33.211 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:33.212 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:33.212 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:33.212 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89647 2020-08-07T15:12:33.212 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.212 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:33.212 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.213 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:33.213 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557315+0000", 2020-08-07T15:12:33.213 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:33.213 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.213 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.214 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:33.214 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557315+0000", 2020-08-07T15:12:33.214 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6639999999999999e-05 2020-08-07T15:12:33.214 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.214 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.215 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:33.215 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557341+0000", 2020-08-07T15:12:33.215 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7989999999999998e-06 2020-08-07T15:12:33.215 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.215 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.215 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:33.216 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557344+0000", 2020-08-07T15:12:33.216 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6960000000000002e-06 2020-08-07T15:12:33.216 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.216 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.216 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:33.217 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557350+0000", 2020-08-07T15:12:33.217 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6089999999999999e-06 2020-08-07T15:12:33.217 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.217 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.217 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:33.218 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557352+0000", 2020-08-07T15:12:33.218 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1837e-05 2020-08-07T15:12:33.218 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.218 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.218 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.219 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557384+0000", 2020-08-07T15:12:33.219 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8099999999999998e-06 2020-08-07T15:12:33.219 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.219 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.219 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:33.220 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557392+0000", 2020-08-07T15:12:33.220 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67581117300000004 2020-08-07T15:12:33.220 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.220 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.220 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.220 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233203+0000", 2020-08-07T15:12:33.221 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112316 2020-08-07T15:12:33.221 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.221 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.221 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:33.221 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233316+0000", 2020-08-07T15:12:33.222 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000221766 2020-08-07T15:12:33.222 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.222 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.222 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:33.222 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233537+0000", 2020-08-07T15:12:33.223 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069695019999999998 2020-08-07T15:12:33.223 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.223 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.223 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:33.223 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240507+0000", 2020-08-07T15:12:33.223 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3999999999999997e-07 2020-08-07T15:12:33.224 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.224 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.224 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:33.224 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240507+0000", 2020-08-07T15:12:33.224 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8844000000000003e-05 2020-08-07T15:12:33.225 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.225 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.225 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:33.225 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240546+0000", 2020-08-07T15:12:33.225 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023473399999999998 2020-08-07T15:12:33.226 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.226 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.226 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.226 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242893+0000", 2020-08-07T15:12:33.226 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5410000000000001e-05 2020-08-07T15:12:33.227 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.227 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.227 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.227 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242939+0000", 2020-08-07T15:12:33.227 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8000000000000001e-07 2020-08-07T15:12:33.228 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.228 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.228 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:33.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242939+0000", 2020-08-07T15:12:33.228 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019369600000000001 2020-08-07T15:12:33.228 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.229 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.229 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:33.229 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243133+0000", 2020-08-07T15:12:33.229 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68581815800000001 2020-08-07T15:12:33.229 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.230 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:33.230 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.230 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.230 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.230 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89648 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:33.231 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.557372+0000", 2020-08-07T15:12:33.231 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 71.404075679000002, 2020-08-07T15:12:33.231 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68606040000000001, 2020-08-07T15:12:33.231 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:33.231 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:33.232 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:33.232 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:33.232 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:33.232 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89648 2020-08-07T15:12:33.232 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.232 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:33.233 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.233 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:33.233 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557372+0000", 2020-08-07T15:12:33.233 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:33.233 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.234 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.234 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:33.234 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557372+0000", 2020-08-07T15:12:33.234 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5534e-05 2020-08-07T15:12:33.234 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.235 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.235 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:33.235 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557397+0000", 2020-08-07T15:12:33.235 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.599e-06 2020-08-07T15:12:33.235 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.235 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.236 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:33.236 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557400+0000", 2020-08-07T15:12:33.236 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.288e-06 2020-08-07T15:12:33.236 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.236 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.237 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:33.237 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557405+0000", 2020-08-07T15:12:33.237 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.142e-06 2020-08-07T15:12:33.237 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.237 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.238 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:33.238 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557407+0000", 2020-08-07T15:12:33.238 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0954999999999998e-05 2020-08-07T15:12:33.238 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.238 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.238 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.239 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557428+0000", 2020-08-07T15:12:33.239 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7529999999999998e-06 2020-08-07T15:12:33.239 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.239 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.254 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:33.254 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557434+0000", 2020-08-07T15:12:33.254 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67655471099999998 2020-08-07T15:12:33.255 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.255 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.255 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.255 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233989+0000", 2020-08-07T15:12:33.255 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8301999999999997e-05 2020-08-07T15:12:33.256 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.256 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.256 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:33.256 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234067+0000", 2020-08-07T15:12:33.256 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000129908 2020-08-07T15:12:33.257 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.257 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.257 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:33.257 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234197+0000", 2020-08-07T15:12:33.257 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063551249999999997 2020-08-07T15:12:33.257 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.258 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.258 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:33.258 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240552+0000", 2020-08-07T15:12:33.258 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9400000000000001e-07 2020-08-07T15:12:33.258 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.259 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.259 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:33.259 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240552+0000", 2020-08-07T15:12:33.259 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2787999999999998e-05 2020-08-07T15:12:33.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.260 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.260 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:33.260 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240585+0000", 2020-08-07T15:12:33.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002605652 2020-08-07T15:12:33.260 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.261 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.261 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.261 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243191+0000", 2020-08-07T15:12:33.261 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8690000000000002e-05 2020-08-07T15:12:33.261 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.262 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.262 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.262 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243250+0000", 2020-08-07T15:12:33.262 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4700000000000001e-07 2020-08-07T15:12:33.263 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.263 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.263 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:33.263 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243250+0000", 2020-08-07T15:12:33.263 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000181912 2020-08-07T15:12:33.264 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.264 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.264 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:33.264 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243432+0000", 2020-08-07T15:12:33.264 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68606040000000001 2020-08-07T15:12:33.265 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.265 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:33.265 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.265 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.265 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.266 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89689 5.7 5:e83ee43b:::benchmark_data_smithi074_53697_object11211:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:33.266 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.585493+0000", 2020-08-07T15:12:33.266 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 71.375954860999997, 2020-08-07T15:12:33.267 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66549445299999999, 2020-08-07T15:12:33.267 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:33.267 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:33.268 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:33.268 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:33.268 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:33.268 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89689 2020-08-07T15:12:33.268 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.269 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:33.269 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.269 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:33.269 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585493+0000", 2020-08-07T15:12:33.269 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:33.270 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.270 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.270 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:33.270 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585493+0000", 2020-08-07T15:12:33.270 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7746000000000002e-05 2020-08-07T15:12:33.270 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.271 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:33.271 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585520+0000", 2020-08-07T15:12:33.271 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1099999999999996e-06 2020-08-07T15:12:33.271 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:33.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585525+0000", 2020-08-07T15:12:33.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5359999999999998e-06 2020-08-07T15:12:33.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.273 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:33.273 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585534+0000", 2020-08-07T15:12:33.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5040000000000002e-06 2020-08-07T15:12:33.280 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.280 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.280 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:33.281 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585540+0000", 2020-08-07T15:12:33.281 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3548999999999998e-05 2020-08-07T15:12:33.281 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.282 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.282 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.282 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585593+0000", 2020-08-07T15:12:33.283 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2014000000000001e-05 2020-08-07T15:12:33.283 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.283 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.284 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:33.284 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585605+0000", 2020-08-07T15:12:33.284 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.64901847700000004 2020-08-07T15:12:33.285 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.285 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.285 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.286 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234624+0000", 2020-08-07T15:12:33.297 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000176329 2020-08-07T15:12:33.298 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.298 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.298 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:33.299 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234800+0000", 2020-08-07T15:12:33.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00025457300000000002 2020-08-07T15:12:33.299 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.300 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.300 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:33.300 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.235055+0000", 2020-08-07T15:12:33.301 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0055364669999999998 2020-08-07T15:12:33.301 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.301 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.303 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:33.303 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240591+0000", 2020-08-07T15:12:33.303 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3700000000000001e-07 2020-08-07T15:12:33.304 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.304 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.304 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:33.305 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240591+0000", 2020-08-07T15:12:33.305 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8479999999999997e-05 2020-08-07T15:12:33.306 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.306 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.306 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:33.307 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240640+0000", 2020-08-07T15:12:33.307 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00285564 2020-08-07T15:12:33.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.308 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243496+0000", 2020-08-07T15:12:33.309 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0073477400000000002 2020-08-07T15:12:33.309 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.309 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.310 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.310 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.250843+0000", 2020-08-07T15:12:33.311 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7300000000000005e-07 2020-08-07T15:12:33.311 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.311 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.312 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:33.312 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.250844+0000", 2020-08-07T15:12:33.312 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014307799999999999 2020-08-07T15:12:33.313 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.313 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.313 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:33.314 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.250987+0000", 2020-08-07T15:12:33.314 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66549445299999999 2020-08-07T15:12:33.315 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.315 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:33.315 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.316 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.316 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.316 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103152 5.7 5:e865d79a:::benchmark_data_smithi074_53697_object12893:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e211)", 2020-08-07T15:12:33.316 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.399967+0000", 2020-08-07T15:12:33.317 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 61.561480971999998, 2020-08-07T15:12:33.317 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81348256399999996, 2020-08-07T15:12:33.317 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:33.317 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:33.317 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:33.318 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:33.318 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:33.318 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103152 2020-08-07T15:12:33.318 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.318 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:33.319 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.319 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:33.319 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.399967+0000", 2020-08-07T15:12:33.319 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:33.319 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.320 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.320 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:33.320 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.399967+0000", 2020-08-07T15:12:33.320 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017528899999999999 2020-08-07T15:12:33.320 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.321 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.321 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:33.321 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400142+0000", 2020-08-07T15:12:33.321 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8239999999999996e-06 2020-08-07T15:12:33.321 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.322 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.322 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:33.322 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400148+0000", 2020-08-07T15:12:33.322 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0251000000000001e-05 2020-08-07T15:12:33.323 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.323 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.323 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:33.323 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400158+0000", 2020-08-07T15:12:33.323 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1579999999999998e-06 2020-08-07T15:12:33.324 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.324 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.324 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:33.324 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400162+0000", 2020-08-07T15:12:33.324 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00078404699999999996 2020-08-07T15:12:33.325 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.325 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.325 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.325 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400946+0000", 2020-08-07T15:12:33.325 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010605800000000001 2020-08-07T15:12:33.326 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.326 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.326 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:33.326 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.401052+0000", 2020-08-07T15:12:33.326 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020987499999999999 2020-08-07T15:12:33.327 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.327 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.327 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:33.327 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.401262+0000", 2020-08-07T15:12:33.327 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023707820000000001 2020-08-07T15:12:33.328 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.328 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.328 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:33.328 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.403633+0000", 2020-08-07T15:12:33.329 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.023e-06 2020-08-07T15:12:33.329 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.329 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.329 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:33.329 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.403634+0000", 2020-08-07T15:12:33.330 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3560999999999995e-05 2020-08-07T15:12:33.330 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.330 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.330 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:33.330 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.403717+0000", 2020-08-07T15:12:33.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011669689999999999 2020-08-07T15:12:33.331 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.331 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.331 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.332 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.404884+0000", 2020-08-07T15:12:33.332 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033332850000000001 2020-08-07T15:12:33.332 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.332 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.332 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.333 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.408218+0000", 2020-08-07T15:12:33.333 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0268000000000001e-05 2020-08-07T15:12:33.333 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.333 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.333 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:33.334 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.408228+0000", 2020-08-07T15:12:33.334 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80508180500000004 2020-08-07T15:12:33.334 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.334 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.334 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.335 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213310+0000", 2020-08-07T15:12:33.335 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000139369 2020-08-07T15:12:33.335 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.335 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:33.336 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213449+0000", 2020-08-07T15:12:33.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81348256399999996 2020-08-07T15:12:33.336 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.336 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:33.336 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.337 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.337 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.337 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103265 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:12:33.337 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478593+0000", 2020-08-07T15:12:33.338 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 61.482854916000001, 2020-08-07T15:12:33.338 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74922190499999997, 2020-08-07T15:12:33.338 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:33.338 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:33.338 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:33.339 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:33.339 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:33.339 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103265 2020-08-07T15:12:33.339 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.340 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:33.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.340 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:33.340 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478593+0000", 2020-08-07T15:12:33.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:33.341 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.342 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:33.342 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478593+0000", 2020-08-07T15:12:33.342 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9968000000000001e-05 2020-08-07T15:12:33.343 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.343 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.344 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:33.344 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478633+0000", 2020-08-07T15:12:33.345 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1420000000000003e-06 2020-08-07T15:12:33.345 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.346 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.346 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:33.347 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478637+0000", 2020-08-07T15:12:33.348 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2919999999999996e-06 2020-08-07T15:12:33.351 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.352 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.353 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:33.353 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478645+0000", 2020-08-07T15:12:33.354 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3699999999999997e-06 2020-08-07T15:12:33.354 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.355 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.355 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:33.356 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478649+0000", 2020-08-07T15:12:33.356 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1440000000000002e-05 2020-08-07T15:12:33.357 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.357 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.358 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.358 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478701+0000", 2020-08-07T15:12:33.359 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0831e-05 2020-08-07T15:12:33.359 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.359 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.360 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:33.360 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478712+0000", 2020-08-07T15:12:33.361 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73476696699999999 2020-08-07T15:12:33.361 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.362 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.362 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.363 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213479+0000", 2020-08-07T15:12:33.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000167026 2020-08-07T15:12:33.363 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:33.365 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213646+0000", 2020-08-07T15:12:33.380 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000179557 2020-08-07T15:12:33.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:33.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213825+0000", 2020-08-07T15:12:33.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041300360000000001 2020-08-07T15:12:33.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:33.414 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217955+0000", 2020-08-07T15:12:33.414 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6840000000000001e-06 2020-08-07T15:12:33.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.415 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:33.415 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217957+0000", 2020-08-07T15:12:33.415 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4758e-05 2020-08-07T15:12:33.416 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.416 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.416 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:33.416 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218012+0000", 2020-08-07T15:12:33.416 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0095519990000000003 2020-08-07T15:12:33.417 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.417 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.417 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.417 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227564+0000", 2020-08-07T15:12:33.417 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9956e-05 2020-08-07T15:12:33.418 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.418 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.418 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.418 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227624+0000", 2020-08-07T15:12:33.418 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3790000000000001e-06 2020-08-07T15:12:33.419 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.419 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.419 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:33.419 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227625+0000", 2020-08-07T15:12:33.419 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0001895 2020-08-07T15:12:33.420 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.420 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.420 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:33.420 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227815+0000", 2020-08-07T15:12:33.420 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74922190499999997 2020-08-07T15:12:33.421 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.421 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:33.421 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.421 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.422 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103266 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:12:33.422 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478693+0000", 2020-08-07T15:12:33.422 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 61.482754575000001, 2020-08-07T15:12:33.422 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74942137399999997, 2020-08-07T15:12:33.422 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:33.422 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:33.423 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:33.423 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:33.424 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:33.424 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103266 2020-08-07T15:12:33.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.425 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:33.425 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:33.426 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478693+0000", 2020-08-07T15:12:33.426 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:33.426 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.426 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.426 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:33.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478693+0000", 2020-08-07T15:12:33.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5658000000000002e-05 2020-08-07T15:12:33.427 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.427 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:33.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478739+0000", 2020-08-07T15:12:33.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.113e-06 2020-08-07T15:12:33.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:33.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478744+0000", 2020-08-07T15:12:33.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9690000000000006e-06 2020-08-07T15:12:33.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.429 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:33.430 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478754+0000", 2020-08-07T15:12:33.430 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1849999999999997e-06 2020-08-07T15:12:33.430 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.430 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.442 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:33.442 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478758+0000", 2020-08-07T15:12:33.442 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7024999999999998e-05 2020-08-07T15:12:33.442 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.443 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.443 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.443 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478805+0000", 2020-08-07T15:12:33.443 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1357e-05 2020-08-07T15:12:33.443 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.444 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:33.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478816+0000", 2020-08-07T15:12:33.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73554277499999998 2020-08-07T15:12:33.444 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.445 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.445 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.445 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.214359+0000", 2020-08-07T15:12:33.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000129145 2020-08-07T15:12:33.445 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.446 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.446 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:33.447 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.214488+0000", 2020-08-07T15:12:33.447 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00027129999999999998 2020-08-07T15:12:33.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.447 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:33.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.214760+0000", 2020-08-07T15:12:33.448 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032631079999999998 2020-08-07T15:12:33.448 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:33.449 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218023+0000", 2020-08-07T15:12:33.449 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1000000000000001e-06 2020-08-07T15:12:33.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.449 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.449 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:33.450 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218024+0000", 2020-08-07T15:12:33.450 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9085000000000006e-05 2020-08-07T15:12:33.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.450 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:33.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218103+0000", 2020-08-07T15:12:33.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.009779088 2020-08-07T15:12:33.451 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.451 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.451 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227882+0000", 2020-08-07T15:12:33.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1989999999999994e-05 2020-08-07T15:12:33.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227944+0000", 2020-08-07T15:12:33.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8299999999999996e-07 2020-08-07T15:12:33.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.454 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:33.454 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227945+0000", 2020-08-07T15:12:33.454 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000169693 2020-08-07T15:12:33.454 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.454 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:33.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228114+0000", 2020-08-07T15:12:33.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74942137399999997 2020-08-07T15:12:33.455 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.455 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:33.455 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.456 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103267 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:12:33.456 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478794+0000", 2020-08-07T15:12:33.456 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 61.482653087000003, 2020-08-07T15:12:33.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74960417300000004, 2020-08-07T15:12:33.457 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:33.457 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:33.457 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:33.457 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:33.458 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:33.458 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103267 2020-08-07T15:12:33.458 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.458 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:33.458 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:33.459 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478794+0000", 2020-08-07T15:12:33.459 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:33.459 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.460 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:33.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478794+0000", 2020-08-07T15:12:33.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6646e-05 2020-08-07T15:12:33.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.460 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.460 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:33.461 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478841+0000", 2020-08-07T15:12:33.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:12:33.461 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.461 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.461 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:33.462 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478846+0000", 2020-08-07T15:12:33.462 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0171e-05 2020-08-07T15:12:33.462 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.462 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.462 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:33.463 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478856+0000", 2020-08-07T15:12:33.463 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5929999999999999e-06 2020-08-07T15:12:33.463 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.463 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.463 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:33.463 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478861+0000", 2020-08-07T15:12:33.464 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2032e-05 2020-08-07T15:12:33.464 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.464 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.464 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.464 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478903+0000", 2020-08-07T15:12:33.465 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0312000000000001e-05 2020-08-07T15:12:33.465 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.465 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.465 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:33.465 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478913+0000", 2020-08-07T15:12:33.466 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73634238600000002 2020-08-07T15:12:33.466 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.466 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.466 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.466 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.215256+0000", 2020-08-07T15:12:33.467 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000124933 2020-08-07T15:12:33.467 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.467 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.467 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:33.467 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.215380+0000", 2020-08-07T15:12:33.467 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024624 2020-08-07T15:12:33.468 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.468 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.468 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:33.468 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.215627+0000", 2020-08-07T15:12:33.468 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0040879540000000004 2020-08-07T15:12:33.469 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.469 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.469 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:33.469 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219715+0000", 2020-08-07T15:12:33.469 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9230000000000001e-06 2020-08-07T15:12:33.470 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.470 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.470 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:33.470 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219717+0000", 2020-08-07T15:12:33.470 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8881999999999995e-05 2020-08-07T15:12:33.470 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.471 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.471 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:33.471 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219785+0000", 2020-08-07T15:12:33.471 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0083925119999999995 2020-08-07T15:12:33.481 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.481 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.481 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.481 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228178+0000", 2020-08-07T15:12:33.482 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2254000000000004e-05 2020-08-07T15:12:33.482 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.482 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.482 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.482 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228240+0000", 2020-08-07T15:12:33.483 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4499999999999996e-07 2020-08-07T15:12:33.483 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.483 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.483 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:33.483 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228241+0000", 2020-08-07T15:12:33.484 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000157656 2020-08-07T15:12:33.484 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.484 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.484 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:33.484 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228399+0000", 2020-08-07T15:12:33.485 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74960417300000004 2020-08-07T15:12:33.485 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.485 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:33.485 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.485 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.486 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.486 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103268 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:12:33.486 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478898+0000", 2020-08-07T15:12:33.486 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 61.482549370000001, 2020-08-07T15:12:33.487 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74981275700000005, 2020-08-07T15:12:33.487 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:33.487 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:33.487 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:33.487 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:33.488 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:33.488 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103268 2020-08-07T15:12:33.500 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.500 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:33.500 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.500 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:33.501 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478898+0000", 2020-08-07T15:12:33.501 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:33.502 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.502 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.503 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:33.503 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478898+0000", 2020-08-07T15:12:33.503 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2558999999999997e-05 2020-08-07T15:12:33.504 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.504 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.508 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:33.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478941+0000", 2020-08-07T15:12:33.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6469999999999997e-06 2020-08-07T15:12:33.509 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.510 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:33.510 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478945+0000", 2020-08-07T15:12:33.511 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3109999999999995e-06 2020-08-07T15:12:33.511 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.512 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:33.512 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478955+0000", 2020-08-07T15:12:33.512 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.799e-06 2020-08-07T15:12:33.513 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.513 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.513 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:33.514 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478958+0000", 2020-08-07T15:12:33.514 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5682000000000002e-05 2020-08-07T15:12:33.514 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.515 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.515 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.515 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:12:33.516 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3260000000000002e-06 2020-08-07T15:12:33.516 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.516 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.517 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:33.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479004+0000", 2020-08-07T15:12:33.530 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73703730499999998 2020-08-07T15:12:33.530 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.530 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.531 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.531 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216041+0000", 2020-08-07T15:12:33.531 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4445999999999998e-05 2020-08-07T15:12:33.531 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.532 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.532 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:33.532 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216115+0000", 2020-08-07T15:12:33.533 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013977 2020-08-07T15:12:33.534 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.534 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.534 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:33.535 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216255+0000", 2020-08-07T15:12:33.535 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0051267359999999998 2020-08-07T15:12:33.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.536 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:33.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221382+0000", 2020-08-07T15:12:33.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0310000000000001e-06 2020-08-07T15:12:33.539 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.539 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.539 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:33.540 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221384+0000", 2020-08-07T15:12:33.540 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8097999999999998e-05 2020-08-07T15:12:33.540 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.541 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:33.541 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221452+0000", 2020-08-07T15:12:33.542 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0070259249999999997 2020-08-07T15:12:33.542 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.543 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.546 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228478+0000", 2020-08-07T15:12:33.546 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7479999999999999e-05 2020-08-07T15:12:33.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.547 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228535+0000", 2020-08-07T15:12:33.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4399999999999999e-07 2020-08-07T15:12:33.548 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.549 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:33.549 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228536+0000", 2020-08-07T15:12:33.549 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017489799999999999 2020-08-07T15:12:33.555 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:33.556 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228711+0000", 2020-08-07T15:12:33.557 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74981275700000005 2020-08-07T15:12:33.557 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.557 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:33.558 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.558 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.558 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.558 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103269 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:12:33.559 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:12:33.559 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 61.482453262, 2020-08-07T15:12:33.559 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74992452799999998, 2020-08-07T15:12:33.560 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:33.560 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:33.560 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:33.561 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:33.561 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:33.561 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103269 2020-08-07T15:12:33.561 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.561 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:33.562 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.562 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:33.562 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:12:33.562 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:33.562 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.563 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.563 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:33.563 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:12:33.563 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5529999999999999e-05 2020-08-07T15:12:33.563 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.564 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:33.564 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479040+0000", 2020-08-07T15:12:33.564 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6550000000000003e-06 2020-08-07T15:12:33.564 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.565 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.565 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:33.565 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479044+0000", 2020-08-07T15:12:33.565 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4285e-05 2020-08-07T15:12:33.565 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.565 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.566 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:33.566 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479059+0000", 2020-08-07T15:12:33.566 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2050000000000001e-06 2020-08-07T15:12:33.566 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.567 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.567 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:33.567 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479064+0000", 2020-08-07T15:12:33.567 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0952999999999999e-05 2020-08-07T15:12:33.569 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.569 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.569 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.569 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479105+0000", 2020-08-07T15:12:33.570 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3559999999999998e-06 2020-08-07T15:12:33.570 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.570 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.570 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:33.570 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479111+0000", 2020-08-07T15:12:33.571 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.737508104 2020-08-07T15:12:33.571 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.571 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.571 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.571 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216619+0000", 2020-08-07T15:12:33.572 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000126367 2020-08-07T15:12:33.572 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.572 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.572 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:33.572 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216746+0000", 2020-08-07T15:12:33.573 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014069899999999999 2020-08-07T15:12:33.573 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.573 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.573 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:33.573 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216886+0000", 2020-08-07T15:12:33.574 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045753160000000003 2020-08-07T15:12:33.574 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.574 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.574 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:33.574 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221462+0000", 2020-08-07T15:12:33.575 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.142e-06 2020-08-07T15:12:33.575 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.575 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:33.575 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221463+0000", 2020-08-07T15:12:33.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7718e-05 2020-08-07T15:12:33.577 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.577 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.577 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:33.578 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221501+0000", 2020-08-07T15:12:33.578 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0072733210000000001 2020-08-07T15:12:33.578 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.578 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.578 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.579 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228774+0000", 2020-08-07T15:12:33.579 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3828e-05 2020-08-07T15:12:33.579 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.580 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.580 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.580 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228828+0000", 2020-08-07T15:12:33.580 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9800000000000003e-07 2020-08-07T15:12:33.580 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.581 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.581 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:33.581 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228828+0000", 2020-08-07T15:12:33.582 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0451000000000005e-05 2020-08-07T15:12:33.582 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.582 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.582 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:33.582 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228919+0000", 2020-08-07T15:12:33.583 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74992452799999998 2020-08-07T15:12:33.583 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.583 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:33.583 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.583 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.583 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.584 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103270 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:12:33.584 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.479168+0000", 2020-08-07T15:12:33.584 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 61.482279224999999, 2020-08-07T15:12:33.584 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75043371699999994, 2020-08-07T15:12:33.584 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:33.585 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:33.585 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:33.585 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:33.585 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:33.586 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103270 2020-08-07T15:12:33.586 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.586 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:33.586 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.587 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:33.587 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479168+0000", 2020-08-07T15:12:33.587 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:33.587 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.588 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:33.588 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479168+0000", 2020-08-07T15:12:33.588 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1557000000000002e-05 2020-08-07T15:12:33.588 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.589 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:33.589 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479210+0000", 2020-08-07T15:12:33.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9840000000000001e-06 2020-08-07T15:12:33.589 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.590 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.590 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:33.590 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479215+0000", 2020-08-07T15:12:33.590 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9539999999999999e-06 2020-08-07T15:12:33.590 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.591 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.591 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:33.591 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479225+0000", 2020-08-07T15:12:33.592 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.942e-06 2020-08-07T15:12:33.592 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.592 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.592 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:33.592 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479231+0000", 2020-08-07T15:12:33.593 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2493000000000002e-05 2020-08-07T15:12:33.593 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.593 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.593 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.593 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479263+0000", 2020-08-07T15:12:33.594 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6640000000000003e-06 2020-08-07T15:12:33.594 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.594 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.594 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:33.595 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479269+0000", 2020-08-07T15:12:33.595 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73802486499999997 2020-08-07T15:12:33.595 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.595 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.595 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.596 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217294+0000", 2020-08-07T15:12:33.596 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8640000000000006e-05 2020-08-07T15:12:33.596 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.596 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.596 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:33.597 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217372+0000", 2020-08-07T15:12:33.597 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000140338 2020-08-07T15:12:33.597 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.597 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.597 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:33.598 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217513+0000", 2020-08-07T15:12:33.598 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039952119999999997 2020-08-07T15:12:33.598 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.598 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.598 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:33.599 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221508+0000", 2020-08-07T15:12:33.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8899999999999999e-07 2020-08-07T15:12:33.599 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.599 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.599 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:33.600 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221509+0000", 2020-08-07T15:12:33.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8108999999999997e-05 2020-08-07T15:12:33.600 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.601 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:33.601 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221547+0000", 2020-08-07T15:12:33.601 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074232029999999997 2020-08-07T15:12:33.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.602 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.602 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.602 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228970+0000", 2020-08-07T15:12:33.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00046710099999999999 2020-08-07T15:12:33.602 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.603 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.603 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229437+0000", 2020-08-07T15:12:33.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5400000000000001e-07 2020-08-07T15:12:33.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:33.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229438+0000", 2020-08-07T15:12:33.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016441200000000001 2020-08-07T15:12:33.606 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.607 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.607 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:33.607 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229602+0000", 2020-08-07T15:12:33.607 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75043371699999994 2020-08-07T15:12:33.608 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.608 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:33.609 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.609 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.609 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.609 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103271 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:12:33.609 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.479356+0000", 2020-08-07T15:12:33.610 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 61.482091257, 2020-08-07T15:12:33.610 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75047653800000003, 2020-08-07T15:12:33.610 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:33.610 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:33.610 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:33.611 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:33.611 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:33.611 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103271 2020-08-07T15:12:33.611 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.612 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:33.612 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.612 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:33.612 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479356+0000", 2020-08-07T15:12:33.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:33.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.613 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:33.613 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479356+0000", 2020-08-07T15:12:33.613 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2416000000000002e-05 2020-08-07T15:12:33.614 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.614 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.614 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:33.614 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479399+0000", 2020-08-07T15:12:33.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0819999999999998e-06 2020-08-07T15:12:33.614 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.615 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.615 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:33.623 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479404+0000", 2020-08-07T15:12:33.623 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2486999999999999e-05 2020-08-07T15:12:33.623 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.623 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.624 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:33.624 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479416+0000", 2020-08-07T15:12:33.624 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3810000000000001e-06 2020-08-07T15:12:33.624 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.624 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.633 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:33.633 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479421+0000", 2020-08-07T15:12:33.633 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2311999999999998e-05 2020-08-07T15:12:33.634 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.634 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.635 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.635 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479453+0000", 2020-08-07T15:12:33.635 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7690000000000002e-06 2020-08-07T15:12:33.635 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.635 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.636 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:33.636 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479459+0000", 2020-08-07T15:12:33.636 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73869566499999995 2020-08-07T15:12:33.636 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.636 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.637 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.637 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218154+0000", 2020-08-07T15:12:33.637 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000124535 2020-08-07T15:12:33.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:33.646 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218279+0000", 2020-08-07T15:12:33.646 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000223441 2020-08-07T15:12:33.647 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.647 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.647 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:33.648 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218502+0000", 2020-08-07T15:12:33.648 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003054761 2020-08-07T15:12:33.649 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.649 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.649 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:33.666 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221557+0000", 2020-08-07T15:12:33.666 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0449999999999999e-06 2020-08-07T15:12:33.666 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.666 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.667 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:33.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221558+0000", 2020-08-07T15:12:33.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3655e-05 2020-08-07T15:12:33.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:33.668 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221622+0000", 2020-08-07T15:12:33.668 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.007661434 2020-08-07T15:12:33.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.670 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229283+0000", 2020-08-07T15:12:33.670 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00036023099999999997 2020-08-07T15:12:33.670 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.670 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.670 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.671 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229644+0000", 2020-08-07T15:12:33.671 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.63e-07 2020-08-07T15:12:33.671 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.671 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.671 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:33.672 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229644+0000", 2020-08-07T15:12:33.672 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018886099999999999 2020-08-07T15:12:33.672 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.672 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.672 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:33.673 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229833+0000", 2020-08-07T15:12:33.673 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75047653800000003 2020-08-07T15:12:33.673 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.673 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:33.673 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.674 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.674 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.674 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103272 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:12:33.675 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.481658+0000", 2020-08-07T15:12:33.675 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 61.479789834000002, 2020-08-07T15:12:33.675 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74920410900000001, 2020-08-07T15:12:33.675 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:33.675 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:33.675 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:33.676 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:33.676 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:33.676 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103272 2020-08-07T15:12:33.676 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.676 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:33.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:33.677 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481658+0000", 2020-08-07T15:12:33.677 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:33.677 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.678 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.678 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:33.678 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481658+0000", 2020-08-07T15:12:33.678 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8787e-05 2020-08-07T15:12:33.678 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.679 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:33.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481697+0000", 2020-08-07T15:12:33.679 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0559999999999998e-06 2020-08-07T15:12:33.688 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.689 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.689 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:33.689 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481701+0000", 2020-08-07T15:12:33.690 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1329999999999999e-06 2020-08-07T15:12:33.690 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.690 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.691 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:33.691 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481709+0000", 2020-08-07T15:12:33.691 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.189e-06 2020-08-07T15:12:33.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.692 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:33.692 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481713+0000", 2020-08-07T15:12:33.693 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6025999999999997e-05 2020-08-07T15:12:33.693 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.693 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.699 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.699 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481749+0000", 2020-08-07T15:12:33.700 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5619999999999998e-06 2020-08-07T15:12:33.702 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.702 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.702 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:33.703 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481756+0000", 2020-08-07T15:12:33.703 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73729155000000002 2020-08-07T15:12:33.703 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.704 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.704 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219048+0000", 2020-08-07T15:12:33.705 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000120387 2020-08-07T15:12:33.705 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.705 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.706 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:33.706 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219168+0000", 2020-08-07T15:12:33.707 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017865599999999999 2020-08-07T15:12:33.707 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.708 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:12:33.708 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219347+0000", 2020-08-07T15:12:33.709 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039049169999999999 2020-08-07T15:12:33.709 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.709 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.710 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:33.710 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.223252+0000", 2020-08-07T15:12:33.710 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.412e-06 2020-08-07T15:12:33.711 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.711 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:33.712 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.223253+0000", 2020-08-07T15:12:33.712 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7521999999999997e-05 2020-08-07T15:12:33.713 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.713 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.713 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:33.714 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.223301+0000", 2020-08-07T15:12:33.714 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00603478 2020-08-07T15:12:33.714 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.715 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.715 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.715 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229336+0000", 2020-08-07T15:12:33.716 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013309579999999999 2020-08-07T15:12:33.716 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.717 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.717 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:33.717 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.230667+0000", 2020-08-07T15:12:33.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7400000000000003e-07 2020-08-07T15:12:33.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.718 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:33.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.230667+0000", 2020-08-07T15:12:33.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019450000000000001 2020-08-07T15:12:33.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:33.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.230862+0000", 2020-08-07T15:12:33.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74920410900000001 2020-08-07T15:12:33.721 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.722 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:33.722 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.723 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.723 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:33.723 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:33.726 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:12:33.727 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 21 16 32053 32037 11.9168 12.6953 0.00852774 0.0104832 2020-08-07T15:12:33.733 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:33.728+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:33.833 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:33.831+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:33.920 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:33.920 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:12:33.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.921 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5481.0:33229 6.2 e246/230 6:496190d3:::benchmark_data_smithi074_63843_object4153:head v 246'2037, mlcod=246'2030)", 2020-08-07T15:12:33.921 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:33.914543+0000", 2020-08-07T15:12:33.921 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0047969060000000001, 2020-08-07T15:12:33.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0048196200000000002, 2020-08-07T15:12:33.922 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:33.922 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:12:33.922 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:33.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:33.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.914543+0000", 2020-08-07T15:12:33.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:33.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:33.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.914543+0000", 2020-08-07T15:12:33.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2840000000000001e-05 2020-08-07T15:12:33.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:33.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.914606+0000", 2020-08-07T15:12:33.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4790000000000001e-06 2020-08-07T15:12:33.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:33.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.914613+0000", 2020-08-07T15:12:33.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1427000000000007e-05 2020-08-07T15:12:33.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:33.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.914674+0000", 2020-08-07T15:12:33.931 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3190000000000002e-06 2020-08-07T15:12:33.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:33.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.914679+0000", 2020-08-07T15:12:33.932 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0226000000000002e-05 2020-08-07T15:12:33.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.932 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.932 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.933 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.914710+0000", 2020-08-07T15:12:33.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5914999999999999e-05 2020-08-07T15:12:33.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.933 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.933 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:33.946 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.914726+0000", 2020-08-07T15:12:33.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000182206 2020-08-07T15:12:33.947 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.948 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:33.948 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.949 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5481.0:33230 6.2 e246/230 6:496190d3:::benchmark_data_smithi074_63843_object4153:head v 246'2038, mlcod=246'2030)", 2020-08-07T15:12:33.949 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:33.917024+0000", 2020-08-07T15:12:33.949 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.002316556, 2020-08-07T15:12:33.955 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023798640000000002, 2020-08-07T15:12:33.955 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:33.955 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:12:33.955 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:33.956 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.956 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:33.956 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.917024+0000", 2020-08-07T15:12:33.956 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:33.956 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.957 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:33.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.917024+0000", 2020-08-07T15:12:33.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6786000000000001e-05 2020-08-07T15:12:33.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:33.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.917080+0000", 2020-08-07T15:12:33.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3685e-05 2020-08-07T15:12:33.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:33.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.917094+0000", 2020-08-07T15:12:33.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6486e-05 2020-08-07T15:12:33.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:33.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.917161+0000", 2020-08-07T15:12:33.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3059999999999997e-06 2020-08-07T15:12:33.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:33.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.917166+0000", 2020-08-07T15:12:33.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1136000000000001e-05 2020-08-07T15:12:33.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.966 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.917207+0000", 2020-08-07T15:12:33.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5548e-05 2020-08-07T15:12:33.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:33.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.917233+0000", 2020-08-07T15:12:33.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020894699999999999 2020-08-07T15:12:33.968 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.968 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:33.968 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:33.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.970 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5481.0:33231 6.2 e246/230 6:496190d3:::benchmark_data_smithi074_63843_object4153:head v 246'2039, mlcod=246'2034)", 2020-08-07T15:12:33.971 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:33.918327+0000", 2020-08-07T15:12:33.971 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0010133150000000001, 2020-08-07T15:12:33.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001115551, 2020-08-07T15:12:33.972 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:33.973 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:12:33.973 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:33.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:33.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.918327+0000", 2020-08-07T15:12:33.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:33.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:33.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.918327+0000", 2020-08-07T15:12:33.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5611000000000005e-05 2020-08-07T15:12:33.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.978 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:33.979 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.918403+0000", 2020-08-07T15:12:33.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9129999999999997e-06 2020-08-07T15:12:33.979 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:33.980 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.918409+0000", 2020-08-07T15:12:33.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6066e-05 2020-08-07T15:12:33.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:33.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.918486+0000", 2020-08-07T15:12:33.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7819999999999996e-06 2020-08-07T15:12:33.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:33.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.918492+0000", 2020-08-07T15:12:33.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0747000000000001e-05 2020-08-07T15:12:33.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.990 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.990 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:33.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.918533+0000", 2020-08-07T15:12:33.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.472e-05 2020-08-07T15:12:33.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:33.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:33.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:33.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.918558+0000", 2020-08-07T15:12:34.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00023083899999999999 2020-08-07T15:12:34.003 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.004 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:34.004 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.004 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.004 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:12:34.004 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 3 2020-08-07T15:12:34.005 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:34.006 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:12:34.016 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:34.007+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:34.114 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:34.112+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:34.208 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:34.209 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:12:34.209 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:12:34.210 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:12:34.210 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:34.215 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:34.214+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:34.219 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:12:34.267 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 22 16 33822 33806 12.0032 13.8203 0.00746983 0.010408 2020-08-07T15:12:34.278 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:12:34.283 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:12:34.315 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:34.314+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:34.419 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:34.417+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:34.434 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:34.434 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:12:34.435 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:12:34.435 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:12:34.435 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.435 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:30968 4.4 e98/81 ondisk, result = 0)", 2020-08-07T15:12:34.436 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:12:34.436 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 228.69886881799999, 2020-08-07T15:12:34.436 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96211418999999998, 2020-08-07T15:12:34.436 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:34.436 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:12:34.438 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:34.438 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.438 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:34.439 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:12:34.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:34.439 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.439 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.439 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:34.440 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:12:34.440 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2387e-05 2020-08-07T15:12:34.440 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.440 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:34.440 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729821+0000", 2020-08-07T15:12:34.441 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9909999999999999e-06 2020-08-07T15:12:34.441 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.442 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.442 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:34.442 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729827+0000", 2020-08-07T15:12:34.442 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96185524600000005 2020-08-07T15:12:34.443 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.443 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.443 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:34.443 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691682+0000", 2020-08-07T15:12:34.443 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8760000000000005e-06 2020-08-07T15:12:34.443 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.444 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:34.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691690+0000", 2020-08-07T15:12:34.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9682000000000001e-05 2020-08-07T15:12:34.444 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.445 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.445 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.445 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691759+0000", 2020-08-07T15:12:34.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5381000000000001e-05 2020-08-07T15:12:34.445 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.447 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:34.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691775+0000", 2020-08-07T15:12:34.448 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000127627 2020-08-07T15:12:34.448 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:34.449 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691902+0000", 2020-08-07T15:12:34.449 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96211418999999998 2020-08-07T15:12:34.449 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.450 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:34.450 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.451 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42472 4.4 4:2ff82af8:::benchmark_data_smithi074_44586_object5308:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:12:34.451 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:12:34.451 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 214.16140604399999, 2020-08-07T15:12:34.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89653816399999997, 2020-08-07T15:12:34.452 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:34.452 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:34.452 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:34.452 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:34.453 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:34.453 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42472 2020-08-07T15:12:34.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.453 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:34.454 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.454 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:34.454 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:12:34.454 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:34.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:34.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:12:34.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9591e-05 2020-08-07T15:12:34.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:34.456 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267301+0000", 2020-08-07T15:12:34.456 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7490000000000004e-06 2020-08-07T15:12:34.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:34.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267306+0000", 2020-08-07T15:12:34.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2850000000000001e-05 2020-08-07T15:12:34.458 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.458 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.458 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:34.458 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267319+0000", 2020-08-07T15:12:34.458 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9100000000000002e-06 2020-08-07T15:12:34.459 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:34.459 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267325+0000", 2020-08-07T15:12:34.459 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6584000000000001e-05 2020-08-07T15:12:34.459 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.460 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.460 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267362+0000", 2020-08-07T15:12:34.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9188000000000001e-05 2020-08-07T15:12:34.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.461 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.461 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:34.461 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267451+0000", 2020-08-07T15:12:34.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000182917 2020-08-07T15:12:34.461 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.463 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.463 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:34.464 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267634+0000", 2020-08-07T15:12:34.464 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002484708 2020-08-07T15:12:34.464 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.464 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.465 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:34.465 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270119+0000", 2020-08-07T15:12:34.465 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0299999999999997e-07 2020-08-07T15:12:34.465 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.465 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.466 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:34.466 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270119+0000", 2020-08-07T15:12:34.466 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8251999999999996e-05 2020-08-07T15:12:34.466 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.466 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.467 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:34.467 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270188+0000", 2020-08-07T15:12:34.467 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24443688 2020-08-07T15:12:34.467 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.467 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.468 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.468 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.514625+0000", 2020-08-07T15:12:34.468 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.64911260599999998 2020-08-07T15:12:34.469 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.469 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.469 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.469 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163737+0000", 2020-08-07T15:12:34.469 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2100000000000001e-06 2020-08-07T15:12:34.470 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.470 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.470 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:34.470 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163738+0000", 2020-08-07T15:12:34.470 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0816000000000003e-05 2020-08-07T15:12:34.470 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.471 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.471 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:34.471 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163789+0000", 2020-08-07T15:12:34.471 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89653816399999997 2020-08-07T15:12:34.471 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.472 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:34.472 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.472 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.472 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.472 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:42472 4.4 e112/81 ondisk, result = 0)", 2020-08-07T15:12:34.473 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.271353+0000", 2020-08-07T15:12:34.474 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 214.15730400699999, 2020-08-07T15:12:34.474 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89244306299999998, 2020-08-07T15:12:34.474 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:34.474 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:12:34.474 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:34.475 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.475 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:34.475 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.271353+0000", 2020-08-07T15:12:34.476 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:34.476 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.476 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.476 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:34.476 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.271353+0000", 2020-08-07T15:12:34.477 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7875999999999999e-05 2020-08-07T15:12:34.477 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.477 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.477 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:34.478 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.271381+0000", 2020-08-07T15:12:34.478 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4819999999999996e-06 2020-08-07T15:12:34.478 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.478 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.478 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:34.479 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.271385+0000", 2020-08-07T15:12:34.479 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89228763300000002 2020-08-07T15:12:34.479 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.479 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.479 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:34.480 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163673+0000", 2020-08-07T15:12:34.480 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9949999999999996e-06 2020-08-07T15:12:34.480 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.480 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:34.481 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163678+0000", 2020-08-07T15:12:34.481 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5485000000000003e-05 2020-08-07T15:12:34.481 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.481 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.481 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.482 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163714+0000", 2020-08-07T15:12:34.482 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4785e-05 2020-08-07T15:12:34.482 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.482 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.482 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:34.483 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163728+0000", 2020-08-07T15:12:34.483 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7806999999999996e-05 2020-08-07T15:12:34.483 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.484 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.484 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:34.484 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163796+0000", 2020-08-07T15:12:34.484 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89244306299999998 2020-08-07T15:12:34.485 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.485 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:34.485 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.485 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.485 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.485 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72688 4.4 4:295d3f6e:::benchmark_data_smithi074_44586_object9085:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e129)", 2020-08-07T15:12:34.486 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:12:34.486 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 189.03182385700001, 2020-08-07T15:12:34.486 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7869782759999999, 2020-08-07T15:12:34.486 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:34.486 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:34.487 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:34.487 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:34.487 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:34.487 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72688 2020-08-07T15:12:34.487 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.488 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:34.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:34.488 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:12:34.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:34.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.489 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.489 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:34.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:12:34.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8721999999999999e-05 2020-08-07T15:12:34.490 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.490 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.490 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:34.491 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396882+0000", 2020-08-07T15:12:34.491 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2639999999999999e-06 2020-08-07T15:12:34.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.491 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.491 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:34.492 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396887+0000", 2020-08-07T15:12:34.492 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0767999999999999e-05 2020-08-07T15:12:34.492 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.492 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.492 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:34.493 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396898+0000", 2020-08-07T15:12:34.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1860000000000002e-06 2020-08-07T15:12:34.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.493 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:34.494 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396902+0000", 2020-08-07T15:12:34.494 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0037638649999999999 2020-08-07T15:12:34.494 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.495 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.495 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.495 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400666+0000", 2020-08-07T15:12:34.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000118077 2020-08-07T15:12:34.495 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.496 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.496 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:34.496 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400784+0000", 2020-08-07T15:12:34.496 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000136728 2020-08-07T15:12:34.496 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.497 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.497 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:12:34.497 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400921+0000", 2020-08-07T15:12:34.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001595443 2020-08-07T15:12:34.497 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.498 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.498 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:34.498 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.402516+0000", 2020-08-07T15:12:34.498 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.014817907 2020-08-07T15:12:34.498 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.498 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.499 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.499 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417334+0000", 2020-08-07T15:12:34.499 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.75e-06 2020-08-07T15:12:34.499 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.499 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.500 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:34.500 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417336+0000", 2020-08-07T15:12:34.500 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8066999999999996e-05 2020-08-07T15:12:34.500 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.500 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.501 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:34.501 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417424+0000", 2020-08-07T15:12:34.501 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002210573 2020-08-07T15:12:34.501 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.501 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.502 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.502 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.419635+0000", 2020-08-07T15:12:34.502 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1379000000000001e-05 2020-08-07T15:12:34.502 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.503 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.503 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.503 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.419646+0000", 2020-08-07T15:12:34.503 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.004643409 2020-08-07T15:12:34.503 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.504 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.504 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.504 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424289+0000", 2020-08-07T15:12:34.505 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7563000000000001e-05 2020-08-07T15:12:34.505 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.505 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.505 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.506 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424307+0000", 2020-08-07T15:12:34.506 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75943629400000001 2020-08-07T15:12:34.506 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.506 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.506 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.506 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183743+0000", 2020-08-07T15:12:34.507 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8281000000000006e-05 2020-08-07T15:12:34.507 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.507 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.507 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:34.507 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183812+0000", 2020-08-07T15:12:34.508 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7869782759999999 2020-08-07T15:12:34.508 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.508 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:34.508 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.508 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.509 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.509 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72985 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:34.509 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:12:34.509 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 188.81583803800001, 2020-08-07T15:12:34.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869466839999999, 2020-08-07T15:12:34.510 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:34.510 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:34.510 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:34.510 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:34.510 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:34.510 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72985 2020-08-07T15:12:34.511 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.511 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:34.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:34.511 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:12:34.512 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:34.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.512 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.512 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:34.512 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:12:34.513 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8003999999999997e-05 2020-08-07T15:12:34.513 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.513 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.513 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:34.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612857+0000", 2020-08-07T15:12:34.514 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.246e-06 2020-08-07T15:12:34.514 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.514 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:34.514 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612863+0000", 2020-08-07T15:12:34.514 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5146e-05 2020-08-07T15:12:34.515 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.515 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.516 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:34.516 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612878+0000", 2020-08-07T15:12:34.516 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1940000000000003e-06 2020-08-07T15:12:34.516 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.516 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.517 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:34.517 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612885+0000", 2020-08-07T15:12:34.517 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0349999999999997e-05 2020-08-07T15:12:34.517 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.517 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.517 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.518 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612935+0000", 2020-08-07T15:12:34.518 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1029999999999996e-06 2020-08-07T15:12:34.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.518 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.519 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612942+0000", 2020-08-07T15:12:34.519 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81140097700000002 2020-08-07T15:12:34.519 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.519 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.519 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.520 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424343+0000", 2020-08-07T15:12:34.520 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1799999999999996e-06 2020-08-07T15:12:34.520 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.520 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.521 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424351+0000", 2020-08-07T15:12:34.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75948665299999996 2020-08-07T15:12:34.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.521 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.521 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.522 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183838+0000", 2020-08-07T15:12:34.522 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000179187 2020-08-07T15:12:34.522 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.522 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.522 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:34.523 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.184017+0000", 2020-08-07T15:12:34.523 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024691600000000002 2020-08-07T15:12:34.523 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.523 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.523 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:34.523 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.184264+0000", 2020-08-07T15:12:34.524 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006362721 2020-08-07T15:12:34.524 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.524 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.524 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:34.524 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190627+0000", 2020-08-07T15:12:34.525 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7400000000000001e-06 2020-08-07T15:12:34.525 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.525 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.526 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:34.526 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190629+0000", 2020-08-07T15:12:34.526 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1305999999999999e-05 2020-08-07T15:12:34.526 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.527 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.527 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:34.527 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190690+0000", 2020-08-07T15:12:34.527 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0088507159999999998 2020-08-07T15:12:34.527 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.528 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.528 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199541+0000", 2020-08-07T15:12:34.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015864699999999999 2020-08-07T15:12:34.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.529 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199699+0000", 2020-08-07T15:12:34.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.23e-07 2020-08-07T15:12:34.529 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.529 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.530 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:34.530 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199700+0000", 2020-08-07T15:12:34.530 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5775e-05 2020-08-07T15:12:34.530 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.530 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.531 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:34.531 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199766+0000", 2020-08-07T15:12:34.531 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869466839999999 2020-08-07T15:12:34.531 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.532 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:34.532 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.532 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.532 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.532 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72986 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:34.533 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:12:34.533 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 188.81572745599999, 2020-08-07T15:12:34.533 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.586946441, 2020-08-07T15:12:34.534 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:34.534 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:34.534 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:34.534 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:34.535 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:34.535 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72986 2020-08-07T15:12:34.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.535 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:34.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:34.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:12:34.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:34.537 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.537 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.537 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:34.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:12:34.538 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9861999999999997e-05 2020-08-07T15:12:34.538 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.538 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.538 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:34.538 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612990+0000", 2020-08-07T15:12:34.539 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4350000000000001e-06 2020-08-07T15:12:34.539 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.539 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.539 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:34.539 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612993+0000", 2020-08-07T15:12:34.539 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9529999999999999e-06 2020-08-07T15:12:34.540 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.540 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.540 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:34.540 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613001+0000", 2020-08-07T15:12:34.540 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4309999999999998e-06 2020-08-07T15:12:34.541 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.541 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:34.541 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613004+0000", 2020-08-07T15:12:34.541 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5284000000000001e-05 2020-08-07T15:12:34.542 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.542 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.542 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.542 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613030+0000", 2020-08-07T15:12:34.542 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2400000000000001e-06 2020-08-07T15:12:34.543 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.543 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.543 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.543 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613034+0000", 2020-08-07T15:12:34.543 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81134136199999995 2020-08-07T15:12:34.543 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.544 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.544 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.544 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424375+0000", 2020-08-07T15:12:34.544 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3520000000000001e-06 2020-08-07T15:12:34.544 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.545 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424383+0000", 2020-08-07T15:12:34.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76064437799999995 2020-08-07T15:12:34.545 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.547 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185027+0000", 2020-08-07T15:12:34.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000119719 2020-08-07T15:12:34.548 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.548 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:34.548 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185147+0000", 2020-08-07T15:12:34.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016829400000000001 2020-08-07T15:12:34.549 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.549 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.549 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:34.549 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185315+0000", 2020-08-07T15:12:34.549 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0053821199999999998 2020-08-07T15:12:34.550 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.550 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.550 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:34.550 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190697+0000", 2020-08-07T15:12:34.550 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6899999999999996e-07 2020-08-07T15:12:34.551 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.551 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.551 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:34.551 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190698+0000", 2020-08-07T15:12:34.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2072000000000001e-05 2020-08-07T15:12:34.552 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.552 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:34.552 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190730+0000", 2020-08-07T15:12:34.552 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0088501440000000008 2020-08-07T15:12:34.553 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.553 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.553 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.553 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199580+0000", 2020-08-07T15:12:34.553 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00023199900000000001 2020-08-07T15:12:34.553 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.554 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.554 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.554 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199812+0000", 2020-08-07T15:12:34.554 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6300000000000005e-07 2020-08-07T15:12:34.554 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.555 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.555 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:34.555 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199813+0000", 2020-08-07T15:12:34.555 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3363999999999998e-05 2020-08-07T15:12:34.556 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:34.556 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199876+0000", 2020-08-07T15:12:34.556 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.586946441 2020-08-07T15:12:34.557 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.557 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:34.558 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.566 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.566 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.567 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72987 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:34.567 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:12:34.567 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 188.815631701, 2020-08-07T15:12:34.567 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869598709999999, 2020-08-07T15:12:34.568 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:34.568 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:34.569 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:34.569 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:34.569 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:34.569 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72987 2020-08-07T15:12:34.569 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.570 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:34.570 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.570 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:34.570 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:12:34.570 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:34.571 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.571 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.571 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:34.571 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:12:34.571 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1851999999999999e-05 2020-08-07T15:12:34.572 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.572 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.572 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:34.572 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613057+0000", 2020-08-07T15:12:34.572 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0429999999999999e-06 2020-08-07T15:12:34.572 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.573 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.573 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:34.573 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613060+0000", 2020-08-07T15:12:34.573 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1060000000000002e-06 2020-08-07T15:12:34.574 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.574 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.574 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:34.574 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613067+0000", 2020-08-07T15:12:34.575 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8040000000000002e-06 2020-08-07T15:12:34.575 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.575 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:34.575 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613070+0000", 2020-08-07T15:12:34.575 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6857000000000001e-05 2020-08-07T15:12:34.576 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.576 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.576 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.576 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613097+0000", 2020-08-07T15:12:34.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8770000000000001e-06 2020-08-07T15:12:34.577 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.577 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.577 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.577 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613102+0000", 2020-08-07T15:12:34.577 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81131066399999996 2020-08-07T15:12:34.578 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.579 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.579 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.579 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424413+0000", 2020-08-07T15:12:34.579 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4599999999999997e-06 2020-08-07T15:12:34.579 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.580 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.580 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.580 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424420+0000", 2020-08-07T15:12:34.580 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76132809700000004 2020-08-07T15:12:34.580 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.581 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.581 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.581 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185748+0000", 2020-08-07T15:12:34.582 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.4264000000000005e-05 2020-08-07T15:12:34.582 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.582 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.582 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:34.582 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185832+0000", 2020-08-07T15:12:34.583 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112803 2020-08-07T15:12:34.583 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.584 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.584 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:34.584 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185945+0000", 2020-08-07T15:12:34.584 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0089954130000000007 2020-08-07T15:12:34.585 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.585 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.585 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:34.585 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194941+0000", 2020-08-07T15:12:34.585 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.091e-06 2020-08-07T15:12:34.586 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.586 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.586 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:34.587 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194943+0000", 2020-08-07T15:12:34.587 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7423999999999996e-05 2020-08-07T15:12:34.587 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.587 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.587 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:34.588 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195020+0000", 2020-08-07T15:12:34.588 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045973300000000002 2020-08-07T15:12:34.588 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.589 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.589 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199618+0000", 2020-08-07T15:12:34.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000302009 2020-08-07T15:12:34.589 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.590 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.590 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.590 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199920+0000", 2020-08-07T15:12:34.590 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7999999999999996e-07 2020-08-07T15:12:34.590 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.591 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.591 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:34.591 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199920+0000", 2020-08-07T15:12:34.591 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5296999999999995e-05 2020-08-07T15:12:34.591 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.592 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.592 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:34.592 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199985+0000", 2020-08-07T15:12:34.592 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869598709999999 2020-08-07T15:12:34.592 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.593 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:34.593 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.593 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.593 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.593 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72988 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:34.601 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:12:34.601 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 188.81556712099999, 2020-08-07T15:12:34.601 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587111143, 2020-08-07T15:12:34.602 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:34.602 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:34.602 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:34.602 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:34.602 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:34.603 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72988 2020-08-07T15:12:34.603 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.603 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:34.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:34.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:12:34.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:34.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.605 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.605 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:34.605 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:12:34.605 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6563e-05 2020-08-07T15:12:34.605 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.606 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.606 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:34.606 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613117+0000", 2020-08-07T15:12:34.606 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7180000000000001e-06 2020-08-07T15:12:34.606 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.607 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.607 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:34.607 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613119+0000", 2020-08-07T15:12:34.607 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1789999999999999e-06 2020-08-07T15:12:34.607 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.608 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.608 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:34.608 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613124+0000", 2020-08-07T15:12:34.608 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3539999999999998e-06 2020-08-07T15:12:34.608 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.608 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.609 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:34.609 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613127+0000", 2020-08-07T15:12:34.609 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3697e-05 2020-08-07T15:12:34.609 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.609 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.610 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.610 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613151+0000", 2020-08-07T15:12:34.610 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7559999999999999e-06 2020-08-07T15:12:34.610 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.610 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.611 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.611 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613155+0000", 2020-08-07T15:12:34.611 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81128859600000003 2020-08-07T15:12:34.611 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.611 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.612 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.612 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424444+0000", 2020-08-07T15:12:34.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9619999999999996e-06 2020-08-07T15:12:34.612 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.612 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.612 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.613 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424451+0000", 2020-08-07T15:12:34.613 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76286712899999998 2020-08-07T15:12:34.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.614 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.614 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187318+0000", 2020-08-07T15:12:34.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011616300000000001 2020-08-07T15:12:34.615 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.615 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.615 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:34.615 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187434+0000", 2020-08-07T15:12:34.616 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019347400000000001 2020-08-07T15:12:34.616 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.616 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.616 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:34.616 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187628+0000", 2020-08-07T15:12:34.617 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074049090000000003 2020-08-07T15:12:34.617 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.617 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.617 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:34.617 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195033+0000", 2020-08-07T15:12:34.618 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1090000000000001e-06 2020-08-07T15:12:34.618 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.618 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.618 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:34.624 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195034+0000", 2020-08-07T15:12:34.625 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8566999999999997e-05 2020-08-07T15:12:34.625 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.627 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:34.628 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195092+0000", 2020-08-07T15:12:34.628 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004570457 2020-08-07T15:12:34.628 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.628 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.628 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.629 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199663+0000", 2020-08-07T15:12:34.629 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00046757899999999998 2020-08-07T15:12:34.629 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.629 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.629 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.630 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200130+0000", 2020-08-07T15:12:34.630 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6700000000000002e-07 2020-08-07T15:12:34.630 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.630 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.630 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:34.631 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200131+0000", 2020-08-07T15:12:34.631 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9963999999999996e-05 2020-08-07T15:12:34.631 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.631 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.631 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:34.631 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200201+0000", 2020-08-07T15:12:34.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587111143 2020-08-07T15:12:34.632 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.632 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:34.632 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.632 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.633 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.633 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72989 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:34.633 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:12:34.633 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 188.815510733, 2020-08-07T15:12:34.633 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5872506399999999, 2020-08-07T15:12:34.634 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:34.634 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:34.634 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:34.634 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:34.635 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:34.635 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72989 2020-08-07T15:12:34.635 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.635 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:34.636 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.636 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:34.636 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:12:34.636 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:34.636 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.636 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.637 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:34.637 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:12:34.637 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6081999999999999e-05 2020-08-07T15:12:34.637 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.637 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.638 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:34.638 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613172+0000", 2020-08-07T15:12:34.638 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6570000000000001e-06 2020-08-07T15:12:34.638 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.639 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.639 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:34.639 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613175+0000", 2020-08-07T15:12:34.639 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7740000000000001e-06 2020-08-07T15:12:34.640 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.640 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.640 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:34.640 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613181+0000", 2020-08-07T15:12:34.640 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2759999999999999e-06 2020-08-07T15:12:34.641 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.641 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.641 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:34.641 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613183+0000", 2020-08-07T15:12:34.641 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3892000000000001e-05 2020-08-07T15:12:34.641 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.642 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.642 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613207+0000", 2020-08-07T15:12:34.642 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9870000000000001e-06 2020-08-07T15:12:34.642 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.643 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.643 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.643 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613211+0000", 2020-08-07T15:12:34.643 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81126230700000002 2020-08-07T15:12:34.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.644 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.644 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.644 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424473+0000", 2020-08-07T15:12:34.644 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1460000000000001e-06 2020-08-07T15:12:34.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.645 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424480+0000", 2020-08-07T15:12:34.645 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76413073099999995 2020-08-07T15:12:34.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.646 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.646 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.646 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188610+0000", 2020-08-07T15:12:34.646 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011278899999999999 2020-08-07T15:12:34.646 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.647 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.647 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:34.647 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188723+0000", 2020-08-07T15:12:34.647 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000213422 2020-08-07T15:12:34.647 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.647 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.648 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:34.648 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188936+0000", 2020-08-07T15:12:34.648 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0061672929999999999 2020-08-07T15:12:34.648 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.648 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.649 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:34.649 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195104+0000", 2020-08-07T15:12:34.649 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8199999999999999e-07 2020-08-07T15:12:34.649 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.649 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.650 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:34.650 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195105+0000", 2020-08-07T15:12:34.650 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2056000000000004e-05 2020-08-07T15:12:34.650 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.650 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.651 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:34.651 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195167+0000", 2020-08-07T15:12:34.651 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0051009769999999996 2020-08-07T15:12:34.651 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.651 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.651 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.652 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200268+0000", 2020-08-07T15:12:34.652 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9428000000000001e-05 2020-08-07T15:12:34.652 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.652 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.652 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.653 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200327+0000", 2020-08-07T15:12:34.653 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9100000000000003e-07 2020-08-07T15:12:34.653 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.653 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.653 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:34.654 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200328+0000", 2020-08-07T15:12:34.654 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9250000000000003e-05 2020-08-07T15:12:34.654 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.654 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.654 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:34.655 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200397+0000", 2020-08-07T15:12:34.655 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5872506399999999 2020-08-07T15:12:34.655 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.655 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:34.655 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.655 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.656 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.656 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72990 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:34.656 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:12:34.656 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 188.81545344099999, 2020-08-07T15:12:34.656 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587335105, 2020-08-07T15:12:34.657 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:34.657 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:34.657 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:34.657 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:34.657 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:34.658 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72990 2020-08-07T15:12:34.658 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.658 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:34.658 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.658 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:34.659 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:12:34.659 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:34.659 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.659 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.659 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:34.660 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:12:34.660 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5276999999999999e-05 2020-08-07T15:12:34.660 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.660 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.660 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:34.660 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613229+0000", 2020-08-07T15:12:34.661 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7219999999999999e-06 2020-08-07T15:12:34.661 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.661 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.661 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:34.661 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613232+0000", 2020-08-07T15:12:34.662 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1170000000000003e-06 2020-08-07T15:12:34.662 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.662 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.662 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:34.662 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613237+0000", 2020-08-07T15:12:34.663 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2309999999999999e-06 2020-08-07T15:12:34.663 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.663 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.663 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:34.663 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613239+0000", 2020-08-07T15:12:34.664 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2469000000000001e-05 2020-08-07T15:12:34.664 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.674 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.675 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.675 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613261+0000", 2020-08-07T15:12:34.675 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9280000000000001e-06 2020-08-07T15:12:34.675 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.676 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.676 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.676 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613266+0000", 2020-08-07T15:12:34.676 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81123414599999999 2020-08-07T15:12:34.676 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.677 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424501+0000", 2020-08-07T15:12:34.677 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5300000000000002e-06 2020-08-07T15:12:34.678 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.678 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.678 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.678 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424507+0000", 2020-08-07T15:12:34.678 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76488660200000003 2020-08-07T15:12:34.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.679 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189394+0000", 2020-08-07T15:12:34.679 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6878000000000003e-05 2020-08-07T15:12:34.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.680 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:34.680 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189491+0000", 2020-08-07T15:12:34.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000204477 2020-08-07T15:12:34.682 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.682 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.683 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:34.683 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189695+0000", 2020-08-07T15:12:34.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0065180940000000003 2020-08-07T15:12:34.683 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.683 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.684 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:34.686 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196213+0000", 2020-08-07T15:12:34.687 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4920000000000001e-06 2020-08-07T15:12:34.687 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.687 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.687 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:34.688 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196215+0000", 2020-08-07T15:12:34.688 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9950000000000006e-05 2020-08-07T15:12:34.688 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.688 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.688 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:34.688 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196285+0000", 2020-08-07T15:12:34.689 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004163674 2020-08-07T15:12:34.689 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.689 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.689 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.690 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200448+0000", 2020-08-07T15:12:34.690 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6919999999999999e-05 2020-08-07T15:12:34.690 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.690 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.690 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.691 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200485+0000", 2020-08-07T15:12:34.691 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1699999999999998e-07 2020-08-07T15:12:34.691 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.691 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.691 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:34.692 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200486+0000", 2020-08-07T15:12:34.692 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3081000000000002e-05 2020-08-07T15:12:34.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.692 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:34.693 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200539+0000", 2020-08-07T15:12:34.693 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587335105 2020-08-07T15:12:34.693 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.693 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:34.693 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.694 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.694 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.694 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72991 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:34.694 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:12:34.694 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 188.81539791599999, 2020-08-07T15:12:34.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5876210340000001, 2020-08-07T15:12:34.695 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:34.695 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:34.695 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:34.695 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:34.696 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:34.696 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72991 2020-08-07T15:12:34.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.696 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:34.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.696 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:34.698 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:12:34.698 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:34.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.699 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.699 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:34.699 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:12:34.699 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5763999999999999e-05 2020-08-07T15:12:34.699 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.700 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:34.701 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613285+0000", 2020-08-07T15:12:34.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6479999999999999e-06 2020-08-07T15:12:34.701 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.701 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:34.702 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613288+0000", 2020-08-07T15:12:34.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1909999999999999e-06 2020-08-07T15:12:34.702 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.702 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.702 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:34.703 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613293+0000", 2020-08-07T15:12:34.703 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2560000000000001e-06 2020-08-07T15:12:34.703 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.703 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:34.704 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613295+0000", 2020-08-07T15:12:34.704 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2561e-05 2020-08-07T15:12:34.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.704 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.705 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613318+0000", 2020-08-07T15:12:34.705 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6349999999999999e-06 2020-08-07T15:12:34.705 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.705 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.709 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.709 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613321+0000", 2020-08-07T15:12:34.709 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81120592400000002 2020-08-07T15:12:34.709 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.710 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.710 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.710 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424527+0000", 2020-08-07T15:12:34.710 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8540000000000002e-06 2020-08-07T15:12:34.710 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.711 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.711 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424533+0000", 2020-08-07T15:12:34.711 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76623392800000001 2020-08-07T15:12:34.711 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.712 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.712 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190767+0000", 2020-08-07T15:12:34.712 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3990000000000001e-05 2020-08-07T15:12:34.712 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.712 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.713 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:34.713 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190841+0000", 2020-08-07T15:12:34.713 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011444599999999999 2020-08-07T15:12:34.713 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.713 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.715 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:34.715 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190955+0000", 2020-08-07T15:12:34.715 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0053397499999999999 2020-08-07T15:12:34.716 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.716 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.716 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:34.716 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196295+0000", 2020-08-07T15:12:34.716 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006e-06 2020-08-07T15:12:34.717 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.717 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.717 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:34.717 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196296+0000", 2020-08-07T15:12:34.717 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8940999999999997e-05 2020-08-07T15:12:34.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.718 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.718 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:34.718 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196355+0000", 2020-08-07T15:12:34.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042276700000000002 2020-08-07T15:12:34.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200583+0000", 2020-08-07T15:12:34.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00026035700000000001 2020-08-07T15:12:34.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.721 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200843+0000", 2020-08-07T15:12:34.722 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6700000000000003e-07 2020-08-07T15:12:34.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.722 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:34.722 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200844+0000", 2020-08-07T15:12:34.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6545999999999999e-05 2020-08-07T15:12:34.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.723 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:34.724 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200880+0000", 2020-08-07T15:12:34.724 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5876210340000001 2020-08-07T15:12:34.724 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.724 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:34.725 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.725 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72992 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:34.725 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:12:34.726 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 188.81431043699999, 2020-08-07T15:12:34.726 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5866241210000001, 2020-08-07T15:12:34.726 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:34.726 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:34.730 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:34.730 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:34.731 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:34.731 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72992 2020-08-07T15:12:34.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.731 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:34.732 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:34.732 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:12:34.732 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:34.732 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.733 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:34.733 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:12:34.733 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7543e-05 2020-08-07T15:12:34.733 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.734 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.734 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:34.734 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614374+0000", 2020-08-07T15:12:34.734 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9740000000000004e-06 2020-08-07T15:12:34.734 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.735 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.735 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:34.736 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614378+0000", 2020-08-07T15:12:34.736 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2660000000000007e-06 2020-08-07T15:12:34.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.736 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.737 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:34.737 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614387+0000", 2020-08-07T15:12:34.737 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2950000000000001e-06 2020-08-07T15:12:34.737 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.737 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.738 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:34.738 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614393+0000", 2020-08-07T15:12:34.738 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7279000000000001e-05 2020-08-07T15:12:34.738 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.738 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.739 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.739 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614420+0000", 2020-08-07T15:12:34.739 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9040000000000002e-06 2020-08-07T15:12:34.739 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.739 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.739 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.740 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614424+0000", 2020-08-07T15:12:34.740 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81012815699999996 2020-08-07T15:12:34.740 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.740 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.740 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.741 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424552+0000", 2020-08-07T15:12:34.741 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2879999999999998e-06 2020-08-07T15:12:34.742 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.742 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.742 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.743 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424558+0000", 2020-08-07T15:12:34.743 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76678665400000001 2020-08-07T15:12:34.743 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.743 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.743 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191345+0000", 2020-08-07T15:12:34.744 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010819600000000001 2020-08-07T15:12:34.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:34.744 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191453+0000", 2020-08-07T15:12:34.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000170829 2020-08-07T15:12:34.745 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.745 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:34.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191624+0000", 2020-08-07T15:12:34.746 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0047386679999999997 2020-08-07T15:12:34.746 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.746 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:34.746 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196363+0000", 2020-08-07T15:12:34.747 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6000000000000003e-07 2020-08-07T15:12:34.747 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.747 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.747 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:34.747 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196363+0000", 2020-08-07T15:12:34.748 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3421000000000002e-05 2020-08-07T15:12:34.748 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.753 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.753 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:34.753 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196417+0000", 2020-08-07T15:12:34.753 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042225539999999999 2020-08-07T15:12:34.754 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.754 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.754 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.754 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200639+0000", 2020-08-07T15:12:34.754 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00028031399999999999 2020-08-07T15:12:34.755 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.755 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200920+0000", 2020-08-07T15:12:34.755 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0299999999999999e-07 2020-08-07T15:12:34.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:34.756 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200920+0000", 2020-08-07T15:12:34.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0516000000000002e-05 2020-08-07T15:12:34.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.757 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:34.757 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200971+0000", 2020-08-07T15:12:34.757 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5866241210000001 2020-08-07T15:12:34.757 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.758 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:34.758 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.758 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.758 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.758 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73017 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:34.759 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:12:34.759 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 188.75880218399999, 2020-08-07T15:12:34.759 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.531181608, 2020-08-07T15:12:34.759 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:34.759 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:34.760 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:34.760 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:34.760 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:34.760 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73017 2020-08-07T15:12:34.760 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.760 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:34.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.761 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:34.761 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:12:34.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:34.761 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.762 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.762 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:34.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:12:34.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3018e-05 2020-08-07T15:12:34.763 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.763 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:34.763 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669898+0000", 2020-08-07T15:12:34.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5340000000000002e-06 2020-08-07T15:12:34.764 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.764 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:34.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669905+0000", 2020-08-07T15:12:34.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3307e-05 2020-08-07T15:12:34.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.765 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:34.765 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669919+0000", 2020-08-07T15:12:34.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7729999999999999e-06 2020-08-07T15:12:34.766 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.766 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:34.766 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669927+0000", 2020-08-07T15:12:34.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0361000000000001e-05 2020-08-07T15:12:34.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.767 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669977+0000", 2020-08-07T15:12:34.768 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0049999999999993e-06 2020-08-07T15:12:34.768 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.768 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.769 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.770 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669986+0000", 2020-08-07T15:12:34.770 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75459235199999997 2020-08-07T15:12:34.770 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.770 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.770 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.771 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424578+0000", 2020-08-07T15:12:34.771 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.925e-06 2020-08-07T15:12:34.771 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.771 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.771 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.772 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424584+0000", 2020-08-07T15:12:34.772 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76741703900000002 2020-08-07T15:12:34.772 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.772 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.772 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.773 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192001+0000", 2020-08-07T15:12:34.773 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000181196 2020-08-07T15:12:34.773 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.773 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.773 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:34.774 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192182+0000", 2020-08-07T15:12:34.774 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00021149299999999999 2020-08-07T15:12:34.774 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.774 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.774 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:34.775 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192394+0000", 2020-08-07T15:12:34.775 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004031242 2020-08-07T15:12:34.775 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.775 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:34.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196425+0000", 2020-08-07T15:12:34.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7800000000000001e-07 2020-08-07T15:12:34.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:34.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196426+0000", 2020-08-07T15:12:34.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5244999999999997e-05 2020-08-07T15:12:34.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:34.779 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196481+0000", 2020-08-07T15:12:34.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041989360000000003 2020-08-07T15:12:34.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.779 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.780 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200680+0000", 2020-08-07T15:12:34.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00032415299999999998 2020-08-07T15:12:34.780 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.780 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.781 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201004+0000", 2020-08-07T15:12:34.781 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9299999999999999e-07 2020-08-07T15:12:34.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.781 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:34.782 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201005+0000", 2020-08-07T15:12:34.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1857999999999999e-05 2020-08-07T15:12:34.782 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.782 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.782 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:34.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201037+0000", 2020-08-07T15:12:34.783 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.531181608 2020-08-07T15:12:34.783 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.783 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:34.783 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.783 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.784 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73018 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:34.784 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:12:34.784 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 188.757834379, 2020-08-07T15:12:34.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.530301616, 2020-08-07T15:12:34.785 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:34.785 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:34.785 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:34.785 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:34.785 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:34.786 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73018 2020-08-07T15:12:34.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.786 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:34.787 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:34.787 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:12:34.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:34.787 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.788 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:34.788 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:12:34.788 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9747000000000002e-05 2020-08-07T15:12:34.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.789 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:34.789 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670852+0000", 2020-08-07T15:12:34.789 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9430000000000002e-06 2020-08-07T15:12:34.789 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.789 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.790 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:34.790 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670856+0000", 2020-08-07T15:12:34.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0649999999999999e-06 2020-08-07T15:12:34.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:34.791 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670864+0000", 2020-08-07T15:12:34.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.924e-06 2020-08-07T15:12:34.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.792 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:34.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670868+0000", 2020-08-07T15:12:34.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6205000000000003e-05 2020-08-07T15:12:34.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.792 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.792 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.793 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670905+0000", 2020-08-07T15:12:34.793 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3289999999999996e-06 2020-08-07T15:12:34.793 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.793 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.793 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.794 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670912+0000", 2020-08-07T15:12:34.794 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75370200200000004 2020-08-07T15:12:34.794 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.794 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.794 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.795 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424614+0000", 2020-08-07T15:12:34.795 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5150000000000003e-06 2020-08-07T15:12:34.795 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.795 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.795 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.796 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424620+0000", 2020-08-07T15:12:34.796 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.768200153 2020-08-07T15:12:34.796 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.796 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.796 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192821+0000", 2020-08-07T15:12:34.797 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000111764 2020-08-07T15:12:34.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:34.800 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192932+0000", 2020-08-07T15:12:34.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017542400000000001 2020-08-07T15:12:34.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:34.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193108+0000", 2020-08-07T15:12:34.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004100851 2020-08-07T15:12:34.801 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.801 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.801 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:34.802 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197209+0000", 2020-08-07T15:12:34.802 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0100000000000001e-06 2020-08-07T15:12:34.802 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.802 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.802 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:34.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197210+0000", 2020-08-07T15:12:34.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9672999999999995e-05 2020-08-07T15:12:34.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:34.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197279+0000", 2020-08-07T15:12:34.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034275109999999998 2020-08-07T15:12:34.806 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200707+0000", 2020-08-07T15:12:34.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00036853100000000001 2020-08-07T15:12:34.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.808 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201075+0000", 2020-08-07T15:12:34.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8599999999999998e-07 2020-08-07T15:12:34.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:34.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201076+0000", 2020-08-07T15:12:34.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8483000000000003e-05 2020-08-07T15:12:34.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:34.821 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201124+0000", 2020-08-07T15:12:34.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.530301616 2020-08-07T15:12:34.821 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.821 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:34.822 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.823 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73019 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:34.824 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:12:34.824 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 188.75718672799999, 2020-08-07T15:12:34.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.529718224, 2020-08-07T15:12:34.824 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:34.824 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:34.825 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:34.825 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:34.825 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:34.825 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73019 2020-08-07T15:12:34.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.826 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:34.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:34.826 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:12:34.826 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:34.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.827 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:34.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:12:34.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3309000000000002e-05 2020-08-07T15:12:34.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:34.829 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671504+0000", 2020-08-07T15:12:34.829 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6619999999999998e-06 2020-08-07T15:12:34.829 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.829 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.829 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:34.830 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671507+0000", 2020-08-07T15:12:34.830 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2279999999999998e-06 2020-08-07T15:12:34.830 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.830 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.830 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:34.831 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671516+0000", 2020-08-07T15:12:34.831 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4860000000000002e-06 2020-08-07T15:12:34.831 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.831 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.831 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:34.831 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671519+0000", 2020-08-07T15:12:34.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0564000000000001e-05 2020-08-07T15:12:34.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.832 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.832 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671550+0000", 2020-08-07T15:12:34.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1029999999999998e-06 2020-08-07T15:12:34.833 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.833 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671556+0000", 2020-08-07T15:12:34.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75308759199999997 2020-08-07T15:12:34.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424643+0000", 2020-08-07T15:12:34.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9179999999999997e-06 2020-08-07T15:12:34.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424650+0000", 2020-08-07T15:12:34.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76890373599999995 2020-08-07T15:12:34.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.836 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.836 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193554+0000", 2020-08-07T15:12:34.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012345600000000001 2020-08-07T15:12:34.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:34.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193677+0000", 2020-08-07T15:12:34.838 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013528300000000001 2020-08-07T15:12:34.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.838 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:34.838 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193813+0000", 2020-08-07T15:12:34.838 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034777710000000002 2020-08-07T15:12:34.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.839 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:34.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197291+0000", 2020-08-07T15:12:34.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8000000000000004e-07 2020-08-07T15:12:34.840 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.840 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.840 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:34.842 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197291+0000", 2020-08-07T15:12:34.842 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9176999999999999e-05 2020-08-07T15:12:34.842 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.842 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.842 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:34.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197351+0000", 2020-08-07T15:12:34.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033931009999999999 2020-08-07T15:12:34.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.843 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.844 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200744+0000", 2020-08-07T15:12:34.844 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00041308000000000002 2020-08-07T15:12:34.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.844 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.845 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201157+0000", 2020-08-07T15:12:34.845 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8999999999999998e-07 2020-08-07T15:12:34.845 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:34.846 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201157+0000", 2020-08-07T15:12:34.846 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1588000000000003e-05 2020-08-07T15:12:34.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.846 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.846 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:34.847 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201189+0000", 2020-08-07T15:12:34.847 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.529718224 2020-08-07T15:12:34.847 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.847 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:34.848 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.848 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.848 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.848 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73020 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:34.848 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:12:34.849 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 188.68884886800001, 2020-08-07T15:12:34.849 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.461468373, 2020-08-07T15:12:34.849 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:34.849 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:34.849 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:34.850 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:34.850 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:34.850 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73020 2020-08-07T15:12:34.850 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.850 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:34.851 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.851 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:34.851 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:12:34.851 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:34.851 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:34.852 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:12:34.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.777e-05 2020-08-07T15:12:34.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.853 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:34.853 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739836+0000", 2020-08-07T15:12:34.853 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0180000000000003e-06 2020-08-07T15:12:34.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.854 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:34.854 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739840+0000", 2020-08-07T15:12:34.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6930000000000002e-06 2020-08-07T15:12:34.854 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.854 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.855 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:34.855 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739850+0000", 2020-08-07T15:12:34.855 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3619999999999999e-06 2020-08-07T15:12:34.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.856 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:34.856 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739854+0000", 2020-08-07T15:12:34.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7432999999999999e-05 2020-08-07T15:12:34.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.858 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739892+0000", 2020-08-07T15:12:34.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1149999999999996e-06 2020-08-07T15:12:34.858 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.858 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.859 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739899+0000", 2020-08-07T15:12:34.859 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68477883299999998 2020-08-07T15:12:34.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.859 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.859 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424677+0000", 2020-08-07T15:12:34.860 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7710000000000001e-06 2020-08-07T15:12:34.860 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.860 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.861 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424684+0000", 2020-08-07T15:12:34.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76949970700000003 2020-08-07T15:12:34.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.861 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.862 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194184+0000", 2020-08-07T15:12:34.862 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3957999999999997e-05 2020-08-07T15:12:34.862 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.862 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:34.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194278+0000", 2020-08-07T15:12:34.863 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017489599999999999 2020-08-07T15:12:34.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:34.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194453+0000", 2020-08-07T15:12:34.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002908908 2020-08-07T15:12:34.864 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:34.865 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197362+0000", 2020-08-07T15:12:34.865 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7299999999999995e-07 2020-08-07T15:12:34.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.865 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:34.866 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197362+0000", 2020-08-07T15:12:34.866 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6848000000000002e-05 2020-08-07T15:12:34.866 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.866 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.866 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:34.867 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197419+0000", 2020-08-07T15:12:34.867 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033597079999999999 2020-08-07T15:12:34.867 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.867 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.867 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.869 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200779+0000", 2020-08-07T15:12:34.869 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00044797899999999999 2020-08-07T15:12:34.869 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.869 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.869 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.870 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201227+0000", 2020-08-07T15:12:34.870 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7100000000000002e-07 2020-08-07T15:12:34.870 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.870 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.870 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:34.871 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201227+0000", 2020-08-07T15:12:34.871 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9230000000000001e-05 2020-08-07T15:12:34.871 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.871 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:34.872 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201277+0000", 2020-08-07T15:12:34.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.461468373 2020-08-07T15:12:34.872 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.872 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:34.872 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.873 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.873 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73021 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:34.873 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:12:34.873 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 188.62581526100001, 2020-08-07T15:12:34.874 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.398505079, 2020-08-07T15:12:34.874 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:34.874 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:34.874 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:34.874 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:34.875 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:34.875 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73021 2020-08-07T15:12:34.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.875 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:34.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:34.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:12:34.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:34.876 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.876 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:34.877 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:12:34.877 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7429000000000001e-05 2020-08-07T15:12:34.877 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.877 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.877 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:34.878 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802869+0000", 2020-08-07T15:12:34.878 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7840000000000001e-06 2020-08-07T15:12:34.880 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.880 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:34.880 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802873+0000", 2020-08-07T15:12:34.881 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0119999999999994e-06 2020-08-07T15:12:34.881 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.881 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.881 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:34.881 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802882+0000", 2020-08-07T15:12:34.882 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.04e-06 2020-08-07T15:12:34.882 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.882 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.882 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:34.883 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802887+0000", 2020-08-07T15:12:34.883 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7076000000000002e-05 2020-08-07T15:12:34.883 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.884 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.884 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802924+0000", 2020-08-07T15:12:34.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9280000000000004e-06 2020-08-07T15:12:34.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.885 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802931+0000", 2020-08-07T15:12:34.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.62178051000000001 2020-08-07T15:12:34.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.886 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.886 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424712+0000", 2020-08-07T15:12:34.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4749999999999996e-06 2020-08-07T15:12:34.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424719+0000", 2020-08-07T15:12:34.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77047786799999995 2020-08-07T15:12:34.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195197+0000", 2020-08-07T15:12:34.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000101909 2020-08-07T15:12:34.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:34.890 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195299+0000", 2020-08-07T15:12:34.890 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000248247 2020-08-07T15:12:34.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.890 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:34.890 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195547+0000", 2020-08-07T15:12:34.891 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033458699999999999 2020-08-07T15:12:34.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.891 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.891 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:34.892 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198893+0000", 2020-08-07T15:12:34.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0419999999999999e-06 2020-08-07T15:12:34.892 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.892 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.892 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:34.893 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198894+0000", 2020-08-07T15:12:34.893 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9354999999999997e-05 2020-08-07T15:12:34.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.893 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:34.894 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198953+0000", 2020-08-07T15:12:34.896 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001861096 2020-08-07T15:12:34.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.897 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.897 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.897 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200815+0000", 2020-08-07T15:12:34.897 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00049697299999999995 2020-08-07T15:12:34.898 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.898 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.898 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.898 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201311+0000", 2020-08-07T15:12:34.899 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7099999999999998e-07 2020-08-07T15:12:34.899 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.899 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.899 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:34.899 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201312+0000", 2020-08-07T15:12:34.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5194000000000002e-05 2020-08-07T15:12:34.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.901 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:34.901 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201347+0000", 2020-08-07T15:12:34.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.398505079 2020-08-07T15:12:34.901 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.901 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:34.902 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.902 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.904 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73022 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:34.905 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:12:34.905 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 188.54464320599999, 2020-08-07T15:12:34.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3238320669999999, 2020-08-07T15:12:34.905 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:34.906 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:34.906 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:34.906 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:34.906 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:34.906 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73022 2020-08-07T15:12:34.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.907 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:34.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:34.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:12:34.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:34.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.908 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:34.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:12:34.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6769999999999999e-05 2020-08-07T15:12:34.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:34.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884041+0000", 2020-08-07T15:12:34.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.703e-06 2020-08-07T15:12:34.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.910 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:34.910 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884044+0000", 2020-08-07T15:12:34.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5949999999999999e-06 2020-08-07T15:12:34.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:34.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884053+0000", 2020-08-07T15:12:34.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000003e-06 2020-08-07T15:12:34.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:34.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884058+0000", 2020-08-07T15:12:34.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7281999999999999e-05 2020-08-07T15:12:34.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884105+0000", 2020-08-07T15:12:34.914 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.044e-05 2020-08-07T15:12:34.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884116+0000", 2020-08-07T15:12:34.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.54062884300000003 2020-08-07T15:12:34.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424745+0000", 2020-08-07T15:12:34.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3149999999999998e-06 2020-08-07T15:12:34.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424754+0000", 2020-08-07T15:12:34.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77175732200000002 2020-08-07T15:12:34.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.918 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.918 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196511+0000", 2020-08-07T15:12:34.918 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010110799999999999 2020-08-07T15:12:34.918 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.919 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:34.919 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196612+0000", 2020-08-07T15:12:34.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017543100000000001 2020-08-07T15:12:34.919 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:34.920 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196788+0000", 2020-08-07T15:12:34.920 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032134569999999999 2020-08-07T15:12:34.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:34.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200001+0000", 2020-08-07T15:12:34.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.071e-06 2020-08-07T15:12:34.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.922 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:34.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200002+0000", 2020-08-07T15:12:34.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2527999999999995e-05 2020-08-07T15:12:34.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:34.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200065+0000", 2020-08-07T15:12:34.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001327433 2020-08-07T15:12:34.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201392+0000", 2020-08-07T15:12:34.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063854030000000004 2020-08-07T15:12:34.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207778+0000", 2020-08-07T15:12:34.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.285e-06 2020-08-07T15:12:34.925 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.926 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:34.926 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207779+0000", 2020-08-07T15:12:34.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7070999999999995e-05 2020-08-07T15:12:34.926 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:34.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207846+0000", 2020-08-07T15:12:34.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3238320669999999 2020-08-07T15:12:34.927 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.927 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:34.927 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.928 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73023 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:12:34.928 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:12:34.928 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 188.43148156500001, 2020-08-07T15:12:34.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2113701729999999, 2020-08-07T15:12:34.929 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:34.930 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:34.930 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:34.930 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:34.930 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:34.930 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73023 2020-08-07T15:12:34.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.931 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:34.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:34.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:12:34.932 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:34.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.932 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.932 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:34.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:12:34.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4105e-05 2020-08-07T15:12:34.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.933 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.933 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:34.933 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997210+0000", 2020-08-07T15:12:34.934 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1089999999999997e-06 2020-08-07T15:12:34.934 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.934 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.934 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:34.934 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997215+0000", 2020-08-07T15:12:34.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7040000000000006e-06 2020-08-07T15:12:34.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:34.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997224+0000", 2020-08-07T15:12:34.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4439999999999999e-06 2020-08-07T15:12:34.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:34.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997230+0000", 2020-08-07T15:12:34.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5977999999999999e-05 2020-08-07T15:12:34.937 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997276+0000", 2020-08-07T15:12:34.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2660000000000007e-06 2020-08-07T15:12:34.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.938 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.938 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997285+0000", 2020-08-07T15:12:34.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.42749189599999998 2020-08-07T15:12:34.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424777+0000", 2020-08-07T15:12:34.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8940000000000004e-06 2020-08-07T15:12:34.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.940 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.940 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:34.940 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424784+0000", 2020-08-07T15:12:34.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77268199699999995 2020-08-07T15:12:34.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197466+0000", 2020-08-07T15:12:34.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011277699999999999 2020-08-07T15:12:34.942 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.942 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:34.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197579+0000", 2020-08-07T15:12:34.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018970599999999999 2020-08-07T15:12:34.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.944 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:12:34.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197768+0000", 2020-08-07T15:12:34.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043168529999999998 2020-08-07T15:12:34.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:34.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202085+0000", 2020-08-07T15:12:34.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.186e-06 2020-08-07T15:12:34.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.946 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:34.946 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202086+0000", 2020-08-07T15:12:34.946 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8975999999999999e-05 2020-08-07T15:12:34.946 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.947 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.947 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:34.947 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202145+0000", 2020-08-07T15:12:34.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00035331499999999999 2020-08-07T15:12:34.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.948 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.948 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202499+0000", 2020-08-07T15:12:34.948 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0059703159999999998 2020-08-07T15:12:34.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.949 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.949 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208469+0000", 2020-08-07T15:12:34.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7400000000000002e-07 2020-08-07T15:12:34.949 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.950 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.950 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:34.950 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208470+0000", 2020-08-07T15:12:34.950 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5877000000000006e-05 2020-08-07T15:12:34.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:34.951 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208546+0000", 2020-08-07T15:12:34.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2113701729999999 2020-08-07T15:12:34.952 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.952 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:34.952 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.952 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.953 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.954 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103336 5.a 5:5e0ebd9f:::benchmark_data_smithi074_53697_object12916:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:12:34.954 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.550524+0000", 2020-08-07T15:12:34.954 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 62.878133059, 2020-08-07T15:12:34.954 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.71324962300000005, 2020-08-07T15:12:34.955 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:34.955 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:34.955 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:34.955 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:12:34.955 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:34.956 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103336 2020-08-07T15:12:34.956 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.956 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:34.956 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.956 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:34.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.550524+0000", 2020-08-07T15:12:34.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:34.957 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.957 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:34.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.550524+0000", 2020-08-07T15:12:34.958 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4780000000000002e-05 2020-08-07T15:12:34.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:34.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.550569+0000", 2020-08-07T15:12:34.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9410000000000004e-06 2020-08-07T15:12:34.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:34.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.550575+0000", 2020-08-07T15:12:34.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0842999999999999e-05 2020-08-07T15:12:34.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:34.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.550586+0000", 2020-08-07T15:12:34.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6670000000000001e-06 2020-08-07T15:12:34.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:34.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.550591+0000", 2020-08-07T15:12:34.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010161409999999999 2020-08-07T15:12:34.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:34.964 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.551607+0000", 2020-08-07T15:12:34.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9899e-05 2020-08-07T15:12:34.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:34.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.551707+0000", 2020-08-07T15:12:34.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000122602 2020-08-07T15:12:34.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,4", 2020-08-07T15:12:34.966 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.551830+0000", 2020-08-07T15:12:34.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011245630000000001 2020-08-07T15:12:34.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:34.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.552954+0000", 2020-08-07T15:12:34.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9400000000000004e-07 2020-08-07T15:12:34.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:34.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.552955+0000", 2020-08-07T15:12:34.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2538000000000003e-05 2020-08-07T15:12:34.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.969 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:34.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.553018+0000", 2020-08-07T15:12:34.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034281989999999998 2020-08-07T15:12:34.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.970 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.556446+0000", 2020-08-07T15:12:34.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.707284417 2020-08-07T15:12:34.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:34.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.263731+0000", 2020-08-07T15:12:34.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3399999999999998e-07 2020-08-07T15:12:34.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:34.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.263731+0000", 2020-08-07T15:12:34.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2305000000000002e-05 2020-08-07T15:12:34.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:34.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:34.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:34.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.263774+0000", 2020-08-07T15:12:34.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.71324962300000005 2020-08-07T15:12:34.975 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.975 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:34.975 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.975 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:34.976 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:34.976 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:34.979 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:12:34.980 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:12:34.980 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:12:34.980 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:12:34.980 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:12:34.980 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:12:34.981 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:12:34.981 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:12:34.981 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:12:34.981 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:12:34.982 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:12:34.982 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:12:34.982 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:12:34.982 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:12:34.982 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:12:34.983 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:12:34.983 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:12:34.983 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:34.976+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:35.078 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:35.076+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:35.182 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:35.177+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:35.206 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:35.206 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:35.221 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:12:35.267 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 23 16 35245 35229 11.9647 11.1172 0.00215853 0.0104387 2020-08-07T15:12:35.279 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:35.278+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:35.355 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2020-08-07T15:12:35.356 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 0 1 2020-08-07T15:12:35.400 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:12:35.405 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:35.404+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:35.416 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:35.417 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:35.436 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:12:35.483 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:35.482+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:35.583 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:35.583+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:35.672 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:35.673 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:35.684 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:35.684+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:35.688 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:12:35.786 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:35.785+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:35.881 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:35.881 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:35.888 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:35.887+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:35.895 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:12:35.999 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:35.998+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:36.100 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:36.097+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:36.126 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:36.126 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:36.145 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:12:36.202 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:36.199+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:36.276 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 24 16 36861 36845 11.9922 12.625 0.0201553 0.0104136 2020-08-07T15:12:36.309 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:36.300+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:36.345 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:36.346 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:36.357 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:12:36.397 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.0 to 1 (10000) 2020-08-07T15:12:36.406 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:36.406+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:36.415 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 1 1 2020-08-07T15:12:36.503 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:36.502+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:36.549 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:36.549 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:36.560 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:12:36.604 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:36.603+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:36.709 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:36.706+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:36.778 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:36.778 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:36.787 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:12:36.834 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:36.833+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:36.911 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:36.910+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:36.959 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:36.959 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:36.972 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:12:36.990 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:12:37.014 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:37.013+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:37.117 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:37.113+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:37.176 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:37.177 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:37.206 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:12:37.215 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:37.215+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:37.242 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:12:37.272 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 25 16 38591 38575 12.0531 13.5156 0.00854664 0.0103662 2020-08-07T15:12:37.283 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:12:37.315 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:37.315+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:37.318 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:12:37.374 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:12:37.389 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:37.390 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:37.395 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:12:37.404 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:12:37.406 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.1 to 1 (10000) 2020-08-07T15:12:37.422 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 2 1 2020-08-07T15:12:37.438 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:12:37.444 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:37.444+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:37.445 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:12:37.500 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:12:37.521 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:37.517+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:37.549 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:12:37.556 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:12:37.556 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:12:37.556 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:12:37.556 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:12:37.557 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:12:37.557 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:12:37.557 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:12:37.557 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:12:37.615 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:37.616 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:12:37.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:12:37.616 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:12:37.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.617 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106232 3.a 3:5599d7e6:::benchmark_data_smithi074_35447_object13278:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:12:37.617 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:12:37.617 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 290.26633335999998, 2020-08-07T15:12:37.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52578228699999996, 2020-08-07T15:12:37.617 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:37.618 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:37.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:37.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:12:37.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:12:37.619 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106232 2020-08-07T15:12:37.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.619 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:37.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:37.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:12:37.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:37.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:37.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:12:37.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4214999999999997e-05 2020-08-07T15:12:37.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:37.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345641+0000", 2020-08-07T15:12:37.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9279999999999993e-06 2020-08-07T15:12:37.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:37.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345649+0000", 2020-08-07T15:12:37.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4391e-05 2020-08-07T15:12:37.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:37.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345664+0000", 2020-08-07T15:12:37.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7170000000000002e-06 2020-08-07T15:12:37.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:37.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345670+0000", 2020-08-07T15:12:37.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.459e-05 2020-08-07T15:12:37.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:37.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345695+0000", 2020-08-07T15:12:37.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6070999999999999e-05 2020-08-07T15:12:37.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:37.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345751+0000", 2020-08-07T15:12:37.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010069300000000001 2020-08-07T15:12:37.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:12:37.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345852+0000", 2020-08-07T15:12:37.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013512299999999999 2020-08-07T15:12:37.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.347203+0000", 2020-08-07T15:12:37.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0844000000000001e-05 2020-08-07T15:12:37.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:37.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.347274+0000", 2020-08-07T15:12:37.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52399471600000003 2020-08-07T15:12:37.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871269+0000", 2020-08-07T15:12:37.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.311e-06 2020-08-07T15:12:37.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:37.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871270+0000", 2020-08-07T15:12:37.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3509000000000004e-05 2020-08-07T15:12:37.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:37.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871343+0000", 2020-08-07T15:12:37.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6072000000000002e-05 2020-08-07T15:12:37.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:37.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871389+0000", 2020-08-07T15:12:37.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52578228699999996 2020-08-07T15:12:37.666 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:37.667 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:37.667 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:37.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.668 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106232 3.a e59/20 ondisk, result = 0)", 2020-08-07T15:12:37.668 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:12:37.668 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 290.26328718500002, 2020-08-07T15:12:37.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52274359299999995, 2020-08-07T15:12:37.668 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:37.669 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:37.669 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:37.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:37.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:12:37.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:37.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:37.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:12:37.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5011000000000001e-05 2020-08-07T15:12:37.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:37.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348668+0000", 2020-08-07T15:12:37.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1959999999999998e-06 2020-08-07T15:12:37.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:37.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348671+0000", 2020-08-07T15:12:37.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52253456499999995 2020-08-07T15:12:37.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:37.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871205+0000", 2020-08-07T15:12:37.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0230000000000003e-06 2020-08-07T15:12:37.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:37.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871209+0000", 2020-08-07T15:12:37.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0708999999999999e-05 2020-08-07T15:12:37.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:37.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871250+0000", 2020-08-07T15:12:37.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0158e-05 2020-08-07T15:12:37.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:37.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871260+0000", 2020-08-07T15:12:37.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013693099999999999 2020-08-07T15:12:37.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:37.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871397+0000", 2020-08-07T15:12:37.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52274359299999995 2020-08-07T15:12:37.705 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:37.705 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:37.706 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:37.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.706 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:30976 4.5 e98/81 ondisk, result = 0)", 2020-08-07T15:12:37.707 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:45.736493+0000", 2020-08-07T15:12:37.707 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 231.87544746, 2020-08-07T15:12:37.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54677504399999999, 2020-08-07T15:12:37.707 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:37.707 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:37.708 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:37.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:37.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736493+0000", 2020-08-07T15:12:37.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:37.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:37.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736493+0000", 2020-08-07T15:12:37.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.012e-05 2020-08-07T15:12:37.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:37.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736523+0000", 2020-08-07T15:12:37.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6669999999999998e-06 2020-08-07T15:12:37.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:37.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736527+0000", 2020-08-07T15:12:37.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54657333200000002 2020-08-07T15:12:37.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:37.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283100+0000", 2020-08-07T15:12:37.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.549e-06 2020-08-07T15:12:37.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:37.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283105+0000", 2020-08-07T15:12:37.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1853000000000006e-05 2020-08-07T15:12:37.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:37.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283167+0000", 2020-08-07T15:12:37.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015e-05 2020-08-07T15:12:37.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:37.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283177+0000", 2020-08-07T15:12:37.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1508000000000005e-05 2020-08-07T15:12:37.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:37.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283268+0000", 2020-08-07T15:12:37.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54677504399999999 2020-08-07T15:12:37.719 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:37.719 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:37.720 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:37.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.720 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:37048 4.5 4:a403bd70:::benchmark_data_smithi074_44586_object4630:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e106)", 2020-08-07T15:12:37.721 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:54.237330+0000", 2020-08-07T15:12:37.721 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 223.374610383, 2020-08-07T15:12:37.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.474059283, 2020-08-07T15:12:37.721 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:37.721 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:37.722 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:37.722 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:37.722 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:37.722 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 37048 2020-08-07T15:12:37.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.723 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:37.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:37.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237330+0000", 2020-08-07T15:12:37.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:37.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:37.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237330+0000", 2020-08-07T15:12:37.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0713999999999998e-05 2020-08-07T15:12:37.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:37.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237361+0000", 2020-08-07T15:12:37.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.756e-06 2020-08-07T15:12:37.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:37.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237364+0000", 2020-08-07T15:12:37.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.694e-06 2020-08-07T15:12:37.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:37.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237369+0000", 2020-08-07T15:12:37.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3709999999999998e-06 2020-08-07T15:12:37.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:37.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237372+0000", 2020-08-07T15:12:37.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015644700000000001 2020-08-07T15:12:37.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:37.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.238936+0000", 2020-08-07T15:12:37.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9940999999999998e-05 2020-08-07T15:12:37.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:37.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.239026+0000", 2020-08-07T15:12:37.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017423800000000001 2020-08-07T15:12:37.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,7", 2020-08-07T15:12:37.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.239200+0000", 2020-08-07T15:12:37.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041267530000000004 2020-08-07T15:12:37.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:37.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.243327+0000", 2020-08-07T15:12:37.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7700000000000004e-07 2020-08-07T15:12:37.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:37.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.243328+0000", 2020-08-07T15:12:37.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1211999999999997e-05 2020-08-07T15:12:37.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:37.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.243379+0000", 2020-08-07T15:12:37.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075155150000000004 2020-08-07T15:12:37.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.250894+0000", 2020-08-07T15:12:37.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.460450313 2020-08-07T15:12:37.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.711345+0000", 2020-08-07T15:12:37.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1990000000000001e-06 2020-08-07T15:12:37.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:37.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.711346+0000", 2020-08-07T15:12:37.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3529999999999998e-05 2020-08-07T15:12:37.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:37.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.711389+0000", 2020-08-07T15:12:37.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.474059283 2020-08-07T15:12:37.742 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:37.742 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:37.743 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:37.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.743 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118153 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:37.743 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.222760+0000", 2020-08-07T15:12:37.744 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.389180260000003, 2020-08-07T15:12:37.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0520728660000001, 2020-08-07T15:12:37.744 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:37.744 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:37.744 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:37.745 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:37.745 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:37.745 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118153 2020-08-07T15:12:37.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.745 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:37.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:37.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222760+0000", 2020-08-07T15:12:37.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:37.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:37.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222760+0000", 2020-08-07T15:12:37.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1894000000000002e-05 2020-08-07T15:12:37.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222812+0000", 2020-08-07T15:12:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4880000000000001e-06 2020-08-07T15:12:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222820+0000", 2020-08-07T15:12:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6744000000000001e-05 2020-08-07T15:12:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222836+0000", 2020-08-07T15:12:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5470000000000002e-06 2020-08-07T15:12:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222846+0000", 2020-08-07T15:12:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9478e-05 2020-08-07T15:12:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222925+0000", 2020-08-07T15:12:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0966e-05 2020-08-07T15:12:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2020-08-07T15:12:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222936+0000", 2020-08-07T15:12:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002974099 2020-08-07T15:12:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.225910+0000", 2020-08-07T15:12:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5197e-05 2020-08-07T15:12:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.225926+0000", 2020-08-07T15:12:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.02944747 2020-08-07T15:12:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.255373+0000", 2020-08-07T15:12:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022597200000000001 2020-08-07T15:12:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.255599+0000", 2020-08-07T15:12:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000194415 2020-08-07T15:12:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.255794+0000", 2020-08-07T15:12:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0076831360000000001 2020-08-07T15:12:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263477+0000", 2020-08-07T15:12:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01099552 2020-08-07T15:12:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274472+0000", 2020-08-07T15:12:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2948999999999998e-05 2020-08-07T15:12:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274515+0000", 2020-08-07T15:12:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.569999999999999e-07 2020-08-07T15:12:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274516+0000", 2020-08-07T15:12:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2750999999999997e-05 2020-08-07T15:12:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274569+0000", 2020-08-07T15:12:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4109000000000001e-05 2020-08-07T15:12:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274613+0000", 2020-08-07T15:12:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0432999999999997e-05 2020-08-07T15:12:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274653+0000", 2020-08-07T15:12:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5900000000000002e-07 2020-08-07T15:12:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274654+0000", 2020-08-07T15:12:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017928200000000001 2020-08-07T15:12:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274833+0000", 2020-08-07T15:12:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0520728660000001 2020-08-07T15:12:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118154 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.222899+0000", 2020-08-07T15:12:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.389041958, 2020-08-07T15:12:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0539106499999999, 2020-08-07T15:12:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118154 2020-08-07T15:12:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222899+0000", 2020-08-07T15:12:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222899+0000", 2020-08-07T15:12:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4440000000000001e-05 2020-08-07T15:12:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222953+0000", 2020-08-07T15:12:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4560000000000003e-06 2020-08-07T15:12:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222960+0000", 2020-08-07T15:12:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.201e-05 2020-08-07T15:12:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222972+0000", 2020-08-07T15:12:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4779999999999996e-06 2020-08-07T15:12:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222979+0000", 2020-08-07T15:12:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2364000000000002e-05 2020-08-07T15:12:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223031+0000", 2020-08-07T15:12:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7779999999999996e-06 2020-08-07T15:12:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223040+0000", 2020-08-07T15:12:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029381720000000002 2020-08-07T15:12:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.225978+0000", 2020-08-07T15:12:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1806e-05 2020-08-07T15:12:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.225990+0000", 2020-08-07T15:12:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0316258110000001 2020-08-07T15:12:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.257616+0000", 2020-08-07T15:12:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000143759 2020-08-07T15:12:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.257760+0000", 2020-08-07T15:12:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000296249 2020-08-07T15:12:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.258056+0000", 2020-08-07T15:12:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054347309999999999 2020-08-07T15:12:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263491+0000", 2020-08-07T15:12:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011408567999999999 2020-08-07T15:12:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274899+0000", 2020-08-07T15:12:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030791400000000001 2020-08-07T15:12:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275207+0000", 2020-08-07T15:12:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6499999999999998e-07 2020-08-07T15:12:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275208+0000", 2020-08-07T15:12:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3047999999999998e-05 2020-08-07T15:12:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275261+0000", 2020-08-07T15:12:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000419706 2020-08-07T15:12:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275681+0000", 2020-08-07T15:12:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091386099999999997 2020-08-07T15:12:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276594+0000", 2020-08-07T15:12:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6000000000000001e-06 2020-08-07T15:12:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276597+0000", 2020-08-07T15:12:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000212134 2020-08-07T15:12:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276809+0000", 2020-08-07T15:12:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0539106499999999 2020-08-07T15:12:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:37.806 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:37.806 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:37.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.806 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118155 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:37.806 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223018+0000", 2020-08-07T15:12:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.388922147999999, 2020-08-07T15:12:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0540362350000001, 2020-08-07T15:12:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118155 2020-08-07T15:12:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223018+0000", 2020-08-07T15:12:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223018+0000", 2020-08-07T15:12:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8291999999999999e-05 2020-08-07T15:12:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223057+0000", 2020-08-07T15:12:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.805e-06 2020-08-07T15:12:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223060+0000", 2020-08-07T15:12:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0050000000000004e-06 2020-08-07T15:12:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223068+0000", 2020-08-07T15:12:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4846e-05 2020-08-07T15:12:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223083+0000", 2020-08-07T15:12:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8695000000000005e-05 2020-08-07T15:12:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223162+0000", 2020-08-07T15:12:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4919999999999996e-06 2020-08-07T15:12:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223169+0000", 2020-08-07T15:12:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028562219999999998 2020-08-07T15:12:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226026+0000", 2020-08-07T15:12:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.027e-05 2020-08-07T15:12:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226036+0000", 2020-08-07T15:12:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0327165199999999 2020-08-07T15:12:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.258752+0000", 2020-08-07T15:12:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012614899999999999 2020-08-07T15:12:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.258879+0000", 2020-08-07T15:12:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000258008 2020-08-07T15:12:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.259137+0000", 2020-08-07T15:12:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0067332820000000002 2020-08-07T15:12:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:37.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.265870+0000", 2020-08-07T15:12:37.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0090796000000000002 2020-08-07T15:12:37.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274950+0000", 2020-08-07T15:12:37.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000632601 2020-08-07T15:12:37.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275582+0000", 2020-08-07T15:12:37.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8999999999999997e-07 2020-08-07T15:12:37.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:37.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275583+0000", 2020-08-07T15:12:37.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6400999999999998e-05 2020-08-07T15:12:37.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:37.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275629+0000", 2020-08-07T15:12:37.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014887 2020-08-07T15:12:37.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275778+0000", 2020-08-07T15:12:37.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010622559999999999 2020-08-07T15:12:37.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276840+0000", 2020-08-07T15:12:37.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2199999999999999e-07 2020-08-07T15:12:37.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:37.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276841+0000", 2020-08-07T15:12:37.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000214009 2020-08-07T15:12:37.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:37.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277055+0000", 2020-08-07T15:12:37.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0540362350000001 2020-08-07T15:12:37.831 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:37.831 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:37.832 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:37.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.832 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118156 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:37.832 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223123+0000", 2020-08-07T15:12:37.833 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.388817281999998, 2020-08-07T15:12:37.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0542400009999999, 2020-08-07T15:12:37.833 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:37.833 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:37.833 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:37.834 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:37.834 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:37.836 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118156 2020-08-07T15:12:37.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.836 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:37.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:37.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223123+0000", 2020-08-07T15:12:37.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:37.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:37.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223123+0000", 2020-08-07T15:12:37.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2212999999999995e-05 2020-08-07T15:12:37.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:37.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223195+0000", 2020-08-07T15:12:37.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8709999999999999e-06 2020-08-07T15:12:37.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:37.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223202+0000", 2020-08-07T15:12:37.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3509999999999999e-05 2020-08-07T15:12:37.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:37.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223216+0000", 2020-08-07T15:12:37.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2729999999999997e-06 2020-08-07T15:12:37.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:37.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223223+0000", 2020-08-07T15:12:37.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6728000000000003e-05 2020-08-07T15:12:37.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:37.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223280+0000", 2020-08-07T15:12:37.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1046999999999999e-05 2020-08-07T15:12:37.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:37.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223291+0000", 2020-08-07T15:12:37.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027721030000000002 2020-08-07T15:12:37.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:37.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226063+0000", 2020-08-07T15:12:37.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1809000000000004e-05 2020-08-07T15:12:37.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:37.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226125+0000", 2020-08-07T15:12:37.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0337493870000001 2020-08-07T15:12:37.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:37.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.259874+0000", 2020-08-07T15:12:37.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000133974 2020-08-07T15:12:37.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:37.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.260008+0000", 2020-08-07T15:12:37.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00064730600000000005 2020-08-07T15:12:37.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:37.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.260655+0000", 2020-08-07T15:12:37.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052233740000000002 2020-08-07T15:12:37.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:37.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.265879+0000", 2020-08-07T15:12:37.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.009433238 2020-08-07T15:12:37.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275312+0000", 2020-08-07T15:12:37.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050955399999999995 2020-08-07T15:12:37.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275822+0000", 2020-08-07T15:12:37.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.58e-07 2020-08-07T15:12:37.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:37.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275822+0000", 2020-08-07T15:12:37.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5469000000000006e-05 2020-08-07T15:12:37.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:37.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275887+0000", 2020-08-07T15:12:37.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000371212 2020-08-07T15:12:37.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276259+0000", 2020-08-07T15:12:37.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00085558499999999996 2020-08-07T15:12:37.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277114+0000", 2020-08-07T15:12:37.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2020-08-07T15:12:37.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:37.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277115+0000", 2020-08-07T15:12:37.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024832000000000001 2020-08-07T15:12:37.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:37.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277363+0000", 2020-08-07T15:12:37.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0542400009999999 2020-08-07T15:12:37.859 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:37.860 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:37.860 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:37.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.860 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118157 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:37.861 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223267+0000", 2020-08-07T15:12:37.861 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.388673046999997, 2020-08-07T15:12:37.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0543752369999999, 2020-08-07T15:12:37.861 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:37.861 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:37.862 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:37.863 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:37.863 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:37.863 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118157 2020-08-07T15:12:37.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.863 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:37.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:37.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223267+0000", 2020-08-07T15:12:37.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:37.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:37.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223267+0000", 2020-08-07T15:12:37.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9616000000000001e-05 2020-08-07T15:12:37.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:37.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223317+0000", 2020-08-07T15:12:37.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1359999999999996e-06 2020-08-07T15:12:37.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:37.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223322+0000", 2020-08-07T15:12:37.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0529e-05 2020-08-07T15:12:37.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:37.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223333+0000", 2020-08-07T15:12:37.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1059999999999999e-06 2020-08-07T15:12:37.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:37.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223338+0000", 2020-08-07T15:12:37.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8609000000000002e-05 2020-08-07T15:12:37.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:37.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223386+0000", 2020-08-07T15:12:37.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3830000000000003e-06 2020-08-07T15:12:37.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:37.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223394+0000", 2020-08-07T15:12:37.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027643580000000002 2020-08-07T15:12:37.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:37.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226158+0000", 2020-08-07T15:12:37.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6809999999999993e-06 2020-08-07T15:12:37.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:37.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226168+0000", 2020-08-07T15:12:37.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.035171356 2020-08-07T15:12:37.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:37.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.261339+0000", 2020-08-07T15:12:37.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000130514 2020-08-07T15:12:37.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:37.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.261470+0000", 2020-08-07T15:12:37.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025685699999999998 2020-08-07T15:12:37.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:37.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.261727+0000", 2020-08-07T15:12:37.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075810410000000002 2020-08-07T15:12:37.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:37.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269308+0000", 2020-08-07T15:12:37.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064252490000000001 2020-08-07T15:12:37.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275733+0000", 2020-08-07T15:12:37.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020602000000000001 2020-08-07T15:12:37.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275939+0000", 2020-08-07T15:12:37.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999999e-07 2020-08-07T15:12:37.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:37.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275939+0000", 2020-08-07T15:12:37.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7071999999999997e-05 2020-08-07T15:12:37.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:37.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276016+0000", 2020-08-07T15:12:37.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029090700000000003 2020-08-07T15:12:37.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276307+0000", 2020-08-07T15:12:37.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001114929 2020-08-07T15:12:37.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277422+0000", 2020-08-07T15:12:37.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1200000000000003e-07 2020-08-07T15:12:37.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:37.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277423+0000", 2020-08-07T15:12:37.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021982600000000001 2020-08-07T15:12:37.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:37.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277643+0000", 2020-08-07T15:12:37.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0543752369999999 2020-08-07T15:12:37.894 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:37.895 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:37.895 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:37.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.896 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118158 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:37.896 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223375+0000", 2020-08-07T15:12:37.896 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.388565311000001, 2020-08-07T15:12:37.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0545579519999999, 2020-08-07T15:12:37.897 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:37.897 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:37.899 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:37.899 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:37.899 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:37.899 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118158 2020-08-07T15:12:37.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.900 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:37.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:37.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223375+0000", 2020-08-07T15:12:37.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:37.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:37.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223375+0000", 2020-08-07T15:12:37.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7657000000000001e-05 2020-08-07T15:12:37.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:37.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223423+0000", 2020-08-07T15:12:37.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0810000000000001e-06 2020-08-07T15:12:37.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:37.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223428+0000", 2020-08-07T15:12:37.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7839999999999998e-06 2020-08-07T15:12:37.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:37.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223438+0000", 2020-08-07T15:12:37.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.634e-06 2020-08-07T15:12:37.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:37.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223442+0000", 2020-08-07T15:12:37.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0154999999999998e-05 2020-08-07T15:12:37.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:37.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223482+0000", 2020-08-07T15:12:37.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9579999999999998e-06 2020-08-07T15:12:37.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:37.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223487+0000", 2020-08-07T15:12:37.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027060999999999999 2020-08-07T15:12:37.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:37.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226194+0000", 2020-08-07T15:12:37.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1311999999999999e-05 2020-08-07T15:12:37.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:37.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226205+0000", 2020-08-07T15:12:37.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0361494609999999 2020-08-07T15:12:37.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:37.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.262354+0000", 2020-08-07T15:12:37.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9950000000000004e-05 2020-08-07T15:12:37.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:37.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.262454+0000", 2020-08-07T15:12:37.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000244093 2020-08-07T15:12:37.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:37.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.262698+0000", 2020-08-07T15:12:37.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0066176150000000003 2020-08-07T15:12:37.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:37.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269316+0000", 2020-08-07T15:12:37.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006768998 2020-08-07T15:12:37.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276085+0000", 2020-08-07T15:12:37.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9524000000000002e-05 2020-08-07T15:12:37.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276135+0000", 2020-08-07T15:12:37.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6100000000000001e-07 2020-08-07T15:12:37.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:37.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276135+0000", 2020-08-07T15:12:37.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1214e-05 2020-08-07T15:12:37.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:37.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276206+0000", 2020-08-07T15:12:37.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014914999999999999 2020-08-07T15:12:37.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276355+0000", 2020-08-07T15:12:37.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013427770000000001 2020-08-07T15:12:37.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277698+0000", 2020-08-07T15:12:37.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8599999999999998e-07 2020-08-07T15:12:37.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:37.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277699+0000", 2020-08-07T15:12:37.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023444199999999999 2020-08-07T15:12:37.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:37.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277933+0000", 2020-08-07T15:12:37.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0545579519999999 2020-08-07T15:12:37.925 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:37.925 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:37.925 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:37.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.935 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118159 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:37.935 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223476+0000", 2020-08-07T15:12:37.935 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.388464145, 2020-08-07T15:12:37.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0547920749999999, 2020-08-07T15:12:37.937 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:37.937 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:37.937 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:37.938 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:37.938 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:37.938 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118159 2020-08-07T15:12:37.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.938 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:37.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:37.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223476+0000", 2020-08-07T15:12:37.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:37.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:37.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223476+0000", 2020-08-07T15:12:37.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6094000000000003e-05 2020-08-07T15:12:37.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:37.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223522+0000", 2020-08-07T15:12:37.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6650000000000002e-06 2020-08-07T15:12:37.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:37.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223527+0000", 2020-08-07T15:12:37.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5629999999999998e-06 2020-08-07T15:12:37.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:37.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223537+0000", 2020-08-07T15:12:37.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6810000000000003e-06 2020-08-07T15:12:37.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:37.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223542+0000", 2020-08-07T15:12:37.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6680000000000001e-05 2020-08-07T15:12:37.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:37.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223579+0000", 2020-08-07T15:12:37.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2020-08-07T15:12:37.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:37.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223586+0000", 2020-08-07T15:12:37.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026469250000000001 2020-08-07T15:12:37.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:37.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226233+0000", 2020-08-07T15:12:37.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3130000000000004e-06 2020-08-07T15:12:37.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:37.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226242+0000", 2020-08-07T15:12:37.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.037297957 2020-08-07T15:12:37.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:37.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263540+0000", 2020-08-07T15:12:37.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4307000000000004e-05 2020-08-07T15:12:37.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:37.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263634+0000", 2020-08-07T15:12:37.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019544300000000001 2020-08-07T15:12:37.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:37.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263830+0000", 2020-08-07T15:12:37.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054939949999999998 2020-08-07T15:12:37.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:37.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269324+0000", 2020-08-07T15:12:37.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070859139999999996 2020-08-07T15:12:37.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276410+0000", 2020-08-07T15:12:37.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0924000000000002e-05 2020-08-07T15:12:37.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276461+0000", 2020-08-07T15:12:37.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4099999999999998e-07 2020-08-07T15:12:37.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:37.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276461+0000", 2020-08-07T15:12:37.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8027000000000004e-05 2020-08-07T15:12:37.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:37.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276539+0000", 2020-08-07T15:12:37.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001458453 2020-08-07T15:12:37.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277998+0000", 2020-08-07T15:12:37.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7283000000000001e-05 2020-08-07T15:12:37.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:37.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278045+0000", 2020-08-07T15:12:37.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9100000000000004e-07 2020-08-07T15:12:37.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:37.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278046+0000", 2020-08-07T15:12:37.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022290400000000001 2020-08-07T15:12:37.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:37.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278268+0000", 2020-08-07T15:12:37.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0547920749999999 2020-08-07T15:12:37.996 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:37.996 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:37.997 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:37.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:37.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:37.997 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118160 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:37.998 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223579+0000", 2020-08-07T15:12:37.998 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.388361164000003, 2020-08-07T15:12:37.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0553689980000001, 2020-08-07T15:12:37.998 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:37.998 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:38.006 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:38.006 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:38.006 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:38.006 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118160 2020-08-07T15:12:38.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.007 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:38.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:38.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223579+0000", 2020-08-07T15:12:38.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:38.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:38.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223579+0000", 2020-08-07T15:12:38.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5401000000000001e-05 2020-08-07T15:12:38.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:38.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223625+0000", 2020-08-07T15:12:38.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9069999999999998e-06 2020-08-07T15:12:38.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:38.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223630+0000", 2020-08-07T15:12:38.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4800000000000007e-06 2020-08-07T15:12:38.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:38.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223639+0000", 2020-08-07T15:12:38.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5410000000000002e-06 2020-08-07T15:12:38.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:38.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223644+0000", 2020-08-07T15:12:38.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1476000000000002e-05 2020-08-07T15:12:38.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223675+0000", 2020-08-07T15:12:38.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0240000000000003e-06 2020-08-07T15:12:38.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:38.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223681+0000", 2020-08-07T15:12:38.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025870820000000001 2020-08-07T15:12:38.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226268+0000", 2020-08-07T15:12:38.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0540000000000001e-05 2020-08-07T15:12:38.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:38.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226279+0000", 2020-08-07T15:12:38.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.038209656 2020-08-07T15:12:38.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.264488+0000", 2020-08-07T15:12:38.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029908499999999999 2020-08-07T15:12:38.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:38.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.264788+0000", 2020-08-07T15:12:38.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000216997 2020-08-07T15:12:38.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:38.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.265005+0000", 2020-08-07T15:12:38.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0060308230000000003 2020-08-07T15:12:38.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:38.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271035+0000", 2020-08-07T15:12:38.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0072889499999999998 2020-08-07T15:12:38.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278324+0000", 2020-08-07T15:12:38.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4982999999999999e-05 2020-08-07T15:12:38.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278369+0000", 2020-08-07T15:12:38.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4000000000000002e-07 2020-08-07T15:12:38.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:38.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278370+0000", 2020-08-07T15:12:38.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7656000000000003e-05 2020-08-07T15:12:38.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:38.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278447+0000", 2020-08-07T15:12:38.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000277416 2020-08-07T15:12:38.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278725+0000", 2020-08-07T15:12:38.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2345000000000001e-05 2020-08-07T15:12:38.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278777+0000", 2020-08-07T15:12:38.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8500000000000002e-07 2020-08-07T15:12:38.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:38.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278778+0000", 2020-08-07T15:12:38.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017071100000000001 2020-08-07T15:12:38.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:38.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278948+0000", 2020-08-07T15:12:38.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0553689980000001 2020-08-07T15:12:38.049 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:38.049 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:38.049 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:38.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.050 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118441 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:38.050 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223679+0000", 2020-08-07T15:12:38.050 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.388261149000002, 2020-08-07T15:12:38.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057502779, 2020-08-07T15:12:38.051 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:38.051 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:38.051 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:38.051 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:38.052 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:38.052 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118441 2020-08-07T15:12:38.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.052 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:38.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:38.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223679+0000", 2020-08-07T15:12:38.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:38.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:38.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223679+0000", 2020-08-07T15:12:38.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5559999999999997e-05 2020-08-07T15:12:38.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:38.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223725+0000", 2020-08-07T15:12:38.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1250000000000001e-06 2020-08-07T15:12:38.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:38.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223730+0000", 2020-08-07T15:12:38.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0046e-05 2020-08-07T15:12:38.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:38.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223740+0000", 2020-08-07T15:12:38.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5979999999999999e-06 2020-08-07T15:12:38.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:38.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223745+0000", 2020-08-07T15:12:38.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7234999999999998e-05 2020-08-07T15:12:38.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223782+0000", 2020-08-07T15:12:38.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7789999999999998e-06 2020-08-07T15:12:38.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:38.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223789+0000", 2020-08-07T15:12:38.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002516169 2020-08-07T15:12:38.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226305+0000", 2020-08-07T15:12:38.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3860000000000007e-06 2020-08-07T15:12:38.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:38.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226313+0000", 2020-08-07T15:12:38.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0396126269999999 2020-08-07T15:12:38.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.265926+0000", 2020-08-07T15:12:38.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00079104700000000002 2020-08-07T15:12:38.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:38.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.266717+0000", 2020-08-07T15:12:38.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024298599999999999 2020-08-07T15:12:38.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:38.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.266960+0000", 2020-08-07T15:12:38.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040855509999999998 2020-08-07T15:12:38.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:38.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271045+0000", 2020-08-07T15:12:38.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0074549170000000001 2020-08-07T15:12:38.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278500+0000", 2020-08-07T15:12:38.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8816999999999997e-05 2020-08-07T15:12:38.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278549+0000", 2020-08-07T15:12:38.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9000000000000002e-07 2020-08-07T15:12:38.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:38.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278550+0000", 2020-08-07T15:12:38.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0120999999999997e-05 2020-08-07T15:12:38.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:38.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278600+0000", 2020-08-07T15:12:38.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00092631699999999996 2020-08-07T15:12:38.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279526+0000", 2020-08-07T15:12:38.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014504520000000001 2020-08-07T15:12:38.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280976+0000", 2020-08-07T15:12:38.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6899999999999997e-07 2020-08-07T15:12:38.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:38.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280977+0000", 2020-08-07T15:12:38.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020508700000000001 2020-08-07T15:12:38.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:38.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281182+0000", 2020-08-07T15:12:38.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057502779 2020-08-07T15:12:38.078 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:38.085 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:38.086 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:38.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.086 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118442 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:38.087 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223780+0000", 2020-08-07T15:12:38.087 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.388160278999997, 2020-08-07T15:12:38.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0575529850000001, 2020-08-07T15:12:38.087 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:38.088 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:38.088 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:38.088 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:38.088 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:38.089 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118442 2020-08-07T15:12:38.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.089 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:38.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:38.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223780+0000", 2020-08-07T15:12:38.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:38.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:38.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223780+0000", 2020-08-07T15:12:38.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0340999999999998e-05 2020-08-07T15:12:38.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:38.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223831+0000", 2020-08-07T15:12:38.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3499999999999996e-06 2020-08-07T15:12:38.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:38.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223836+0000", 2020-08-07T15:12:38.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0414999999999999e-05 2020-08-07T15:12:38.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:38.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223846+0000", 2020-08-07T15:12:38.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9239999999999998e-06 2020-08-07T15:12:38.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:38.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223851+0000", 2020-08-07T15:12:38.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5946000000000003e-05 2020-08-07T15:12:38.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223907+0000", 2020-08-07T15:12:38.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6189999999999996e-06 2020-08-07T15:12:38.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:38.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223914+0000", 2020-08-07T15:12:38.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002424149 2020-08-07T15:12:38.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226338+0000", 2020-08-07T15:12:38.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2830000000000001e-06 2020-08-07T15:12:38.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:38.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226346+0000", 2020-08-07T15:12:38.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0412357219999999 2020-08-07T15:12:38.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.267582+0000", 2020-08-07T15:12:38.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000123924 2020-08-07T15:12:38.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:38.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.267706+0000", 2020-08-07T15:12:38.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000216376 2020-08-07T15:12:38.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:38.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.267922+0000", 2020-08-07T15:12:38.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031340790000000001 2020-08-07T15:12:38.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:38.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271056+0000", 2020-08-07T15:12:38.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.007623276 2020-08-07T15:12:38.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278680+0000", 2020-08-07T15:12:38.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000333853 2020-08-07T15:12:38.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279013+0000", 2020-08-07T15:12:38.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.02e-06 2020-08-07T15:12:38.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:38.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279014+0000", 2020-08-07T15:12:38.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6306999999999999e-05 2020-08-07T15:12:38.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:38.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279101+0000", 2020-08-07T15:12:38.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050739800000000005 2020-08-07T15:12:38.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279608+0000", 2020-08-07T15:12:38.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001617098 2020-08-07T15:12:38.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281225+0000", 2020-08-07T15:12:38.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9700000000000002e-07 2020-08-07T15:12:38.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:38.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281226+0000", 2020-08-07T15:12:38.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107508 2020-08-07T15:12:38.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:38.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281333+0000", 2020-08-07T15:12:38.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0575529850000001 2020-08-07T15:12:38.113 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:38.114 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:38.114 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:38.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.114 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118443 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:38.115 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223889+0000", 2020-08-07T15:12:38.115 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.388051609000001, 2020-08-07T15:12:38.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.05758606, 2020-08-07T15:12:38.115 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:38.115 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:38.116 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:38.116 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:38.116 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:38.116 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118443 2020-08-07T15:12:38.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.117 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:38.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:38.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223889+0000", 2020-08-07T15:12:38.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:38.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:38.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223889+0000", 2020-08-07T15:12:38.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8148e-05 2020-08-07T15:12:38.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:38.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223937+0000", 2020-08-07T15:12:38.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2789999999999998e-06 2020-08-07T15:12:38.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:38.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223942+0000", 2020-08-07T15:12:38.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0472000000000001e-05 2020-08-07T15:12:38.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:38.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223953+0000", 2020-08-07T15:12:38.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6580000000000001e-06 2020-08-07T15:12:38.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:38.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223957+0000", 2020-08-07T15:12:38.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5981000000000001e-05 2020-08-07T15:12:38.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223983+0000", 2020-08-07T15:12:38.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.288e-06 2020-08-07T15:12:38.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:38.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223989+0000", 2020-08-07T15:12:38.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024051889999999999 2020-08-07T15:12:38.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226394+0000", 2020-08-07T15:12:38.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2439000000000001e-05 2020-08-07T15:12:38.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:38.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226406+0000", 2020-08-07T15:12:38.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0421168519999999 2020-08-07T15:12:38.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.268523+0000", 2020-08-07T15:12:38.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0449000000000002e-05 2020-08-07T15:12:38.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:38.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.268604+0000", 2020-08-07T15:12:38.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014104300000000001 2020-08-07T15:12:38.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:38.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.268745+0000", 2020-08-07T15:12:38.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051249829999999996 2020-08-07T15:12:38.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:38.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.273870+0000", 2020-08-07T15:12:38.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052772840000000001 2020-08-07T15:12:38.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279147+0000", 2020-08-07T15:12:38.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6060000000000003e-05 2020-08-07T15:12:38.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279193+0000", 2020-08-07T15:12:38.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9999999999999997e-07 2020-08-07T15:12:38.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:38.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279194+0000", 2020-08-07T15:12:38.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3566000000000003e-05 2020-08-07T15:12:38.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:38.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279237+0000", 2020-08-07T15:12:38.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011546359999999999 2020-08-07T15:12:38.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280392+0000", 2020-08-07T15:12:38.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00097857799999999991 2020-08-07T15:12:38.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281370+0000", 2020-08-07T15:12:38.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999999e-07 2020-08-07T15:12:38.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:38.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281371+0000", 2020-08-07T15:12:38.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104119 2020-08-07T15:12:38.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:38.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281475+0000", 2020-08-07T15:12:38.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.05758606 2020-08-07T15:12:38.149 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:38.150 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:38.150 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:38.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.151 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118444 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:38.151 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223994+0000", 2020-08-07T15:12:38.151 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.387946233999998, 2020-08-07T15:12:38.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057634572, 2020-08-07T15:12:38.152 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:38.152 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:38.152 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:38.152 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:38.153 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:38.153 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118444 2020-08-07T15:12:38.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.153 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:38.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:38.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223994+0000", 2020-08-07T15:12:38.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:38.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:38.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223994+0000", 2020-08-07T15:12:38.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1279999999999997e-05 2020-08-07T15:12:38.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:38.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224046+0000", 2020-08-07T15:12:38.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6930000000000004e-06 2020-08-07T15:12:38.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:38.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224051+0000", 2020-08-07T15:12:38.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.095e-05 2020-08-07T15:12:38.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:38.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224062+0000", 2020-08-07T15:12:38.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9529999999999998e-06 2020-08-07T15:12:38.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:38.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224067+0000", 2020-08-07T15:12:38.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000162091 2020-08-07T15:12:38.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224229+0000", 2020-08-07T15:12:38.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.276e-06 2020-08-07T15:12:38.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:38.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224238+0000", 2020-08-07T15:12:38.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022140409999999999 2020-08-07T15:12:38.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226452+0000", 2020-08-07T15:12:38.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5951999999999999e-05 2020-08-07T15:12:38.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:38.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226467+0000", 2020-08-07T15:12:38.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0428868099999999 2020-08-07T15:12:38.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269354+0000", 2020-08-07T15:12:38.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5264000000000003e-05 2020-08-07T15:12:38.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:38.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269430+0000", 2020-08-07T15:12:38.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020438499999999999 2020-08-07T15:12:38.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:38.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269634+0000", 2020-08-07T15:12:38.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042509959999999999 2020-08-07T15:12:38.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:38.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.273885+0000", 2020-08-07T15:12:38.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054290140000000002 2020-08-07T15:12:38.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279314+0000", 2020-08-07T15:12:38.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00038996700000000002 2020-08-07T15:12:38.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279704+0000", 2020-08-07T15:12:38.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5700000000000002e-07 2020-08-07T15:12:38.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:38.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279705+0000", 2020-08-07T15:12:38.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9377999999999997e-05 2020-08-07T15:12:38.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:38.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279794+0000", 2020-08-07T15:12:38.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00064521299999999995 2020-08-07T15:12:38.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280439+0000", 2020-08-07T15:12:38.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001072185 2020-08-07T15:12:38.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281511+0000", 2020-08-07T15:12:38.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7300000000000002e-07 2020-08-07T15:12:38.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:38.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281512+0000", 2020-08-07T15:12:38.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117094 2020-08-07T15:12:38.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:38.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281629+0000", 2020-08-07T15:12:38.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057634572 2020-08-07T15:12:38.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:38.183 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:38.184 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:38.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.184 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118445 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:38.184 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.224116+0000", 2020-08-07T15:12:38.185 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.387824293000001, 2020-08-07T15:12:38.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057744163, 2020-08-07T15:12:38.186 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:38.186 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:38.187 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:38.187 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:38.187 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:38.187 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118445 2020-08-07T15:12:38.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224116+0000", 2020-08-07T15:12:38.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:38.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224116+0000", 2020-08-07T15:12:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3542999999999999e-05 2020-08-07T15:12:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:38.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224170+0000", 2020-08-07T15:12:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.446e-06 2020-08-07T15:12:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:38.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224175+0000", 2020-08-07T15:12:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1473e-05 2020-08-07T15:12:38.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224187+0000", 2020-08-07T15:12:38.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9449999999999998e-06 2020-08-07T15:12:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224193+0000", 2020-08-07T15:12:38.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4945999999999997e-05 2020-08-07T15:12:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224268+0000", 2020-08-07T15:12:38.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5799999999999992e-06 2020-08-07T15:12:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224276+0000", 2020-08-07T15:12:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022246980000000002 2020-08-07T15:12:38.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226501+0000", 2020-08-07T15:12:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1359999999999997e-06 2020-08-07T15:12:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:38.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226509+0000", 2020-08-07T15:12:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.043599741 2020-08-07T15:12:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.270109+0000", 2020-08-07T15:12:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000129503 2020-08-07T15:12:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.270238+0000", 2020-08-07T15:12:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021162500000000001 2020-08-07T15:12:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:38.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.270450+0000", 2020-08-07T15:12:38.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036313090000000001 2020-08-07T15:12:38.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:38.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274081+0000", 2020-08-07T15:12:38.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00557165 2020-08-07T15:12:38.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279653+0000", 2020-08-07T15:12:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000196658 2020-08-07T15:12:38.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279849+0000", 2020-08-07T15:12:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6700000000000003e-07 2020-08-07T15:12:38.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279850+0000", 2020-08-07T15:12:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6522999999999998e-05 2020-08-07T15:12:38.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279937+0000", 2020-08-07T15:12:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00055050400000000005 2020-08-07T15:12:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280487+0000", 2020-08-07T15:12:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011833760000000001 2020-08-07T15:12:38.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281670+0000", 2020-08-07T15:12:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2020-08-07T15:12:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:38.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281671+0000", 2020-08-07T15:12:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001896 2020-08-07T15:12:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:38.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281860+0000", 2020-08-07T15:12:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057744163 2020-08-07T15:12:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:38.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.217 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118446 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.224256+0000", 2020-08-07T15:12:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.387684508, 2020-08-07T15:12:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057864804, 2020-08-07T15:12:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:38.218 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:38.219 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:38.219 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:38.219 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:38.219 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118446 2020-08-07T15:12:38.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224256+0000", 2020-08-07T15:12:38.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224256+0000", 2020-08-07T15:12:38.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5838000000000003e-05 2020-08-07T15:12:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224312+0000", 2020-08-07T15:12:38.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.879e-06 2020-08-07T15:12:38.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224318+0000", 2020-08-07T15:12:38.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3593e-05 2020-08-07T15:12:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224331+0000", 2020-08-07T15:12:38.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6169999999999998e-06 2020-08-07T15:12:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:38.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224337+0000", 2020-08-07T15:12:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7246000000000001e-05 2020-08-07T15:12:38.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224384+0000", 2020-08-07T15:12:38.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7690000000000008e-06 2020-08-07T15:12:38.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224394+0000", 2020-08-07T15:12:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021394589999999998 2020-08-07T15:12:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226533+0000", 2020-08-07T15:12:38.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3849999999999996e-06 2020-08-07T15:12:38.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:38.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226541+0000", 2020-08-07T15:12:38.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0445569109999999 2020-08-07T15:12:38.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271098+0000", 2020-08-07T15:12:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012137 2020-08-07T15:12:38.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271219+0000", 2020-08-07T15:12:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013978300000000001 2020-08-07T15:12:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271359+0000", 2020-08-07T15:12:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027367699999999999 2020-08-07T15:12:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274096+0000", 2020-08-07T15:12:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058973660000000002 2020-08-07T15:12:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279993+0000", 2020-08-07T15:12:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116291 2020-08-07T15:12:38.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280109+0000", 2020-08-07T15:12:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1099999999999995e-07 2020-08-07T15:12:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:38.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280110+0000", 2020-08-07T15:12:38.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.564e-05 2020-08-07T15:12:38.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:38.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280196+0000", 2020-08-07T15:12:38.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061969 2020-08-07T15:12:38.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280815+0000", 2020-08-07T15:12:38.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001097612 2020-08-07T15:12:38.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281913+0000", 2020-08-07T15:12:38.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4300000000000003e-07 2020-08-07T15:12:38.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:38.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281913+0000", 2020-08-07T15:12:38.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000207431 2020-08-07T15:12:38.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:38.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282121+0000", 2020-08-07T15:12:38.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057864804 2020-08-07T15:12:38.248 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:38.248 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:38.248 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:38.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.250 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118447 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:38.251 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.224379+0000", 2020-08-07T15:12:38.251 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.387561742000003, 2020-08-07T15:12:38.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0579997480000001, 2020-08-07T15:12:38.252 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:38.252 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:38.252 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:38.252 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:38.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:38.253 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118447 2020-08-07T15:12:38.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.254 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:38.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:38.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224379+0000", 2020-08-07T15:12:38.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:38.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224379+0000", 2020-08-07T15:12:38.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9775999999999999e-05 2020-08-07T15:12:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:38.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224429+0000", 2020-08-07T15:12:38.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8500000000000002e-06 2020-08-07T15:12:38.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:38.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224433+0000", 2020-08-07T15:12:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7340000000000008e-06 2020-08-07T15:12:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224441+0000", 2020-08-07T15:12:38.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.031e-06 2020-08-07T15:12:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224445+0000", 2020-08-07T15:12:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4264000000000003e-05 2020-08-07T15:12:38.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224479+0000", 2020-08-07T15:12:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0039999999999999e-06 2020-08-07T15:12:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224487+0000", 2020-08-07T15:12:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002241682 2020-08-07T15:12:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226729+0000", 2020-08-07T15:12:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0087e-05 2020-08-07T15:12:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226739+0000", 2020-08-07T15:12:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.045170006 2020-08-07T15:12:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271909+0000", 2020-08-07T15:12:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012782900000000001 2020-08-07T15:12:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.272037+0000", 2020-08-07T15:12:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024079400000000001 2020-08-07T15:12:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.272278+0000", 2020-08-07T15:12:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023454399999999999 2020-08-07T15:12:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274623+0000", 2020-08-07T15:12:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054268190000000003 2020-08-07T15:12:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280050+0000", 2020-08-07T15:12:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00048391200000000002 2020-08-07T15:12:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280534+0000", 2020-08-07T15:12:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5400000000000001e-07 2020-08-07T15:12:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280535+0000", 2020-08-07T15:12:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8922e-05 2020-08-07T15:12:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280623+0000", 2020-08-07T15:12:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024637999999999998 2020-08-07T15:12:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280870+0000", 2020-08-07T15:12:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001300142 2020-08-07T15:12:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282170+0000", 2020-08-07T15:12:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2099999999999997e-07 2020-08-07T15:12:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:38.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282170+0000", 2020-08-07T15:12:38.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000208001 2020-08-07T15:12:38.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282378+0000", 2020-08-07T15:12:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0579997480000001 2020-08-07T15:12:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118448 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.224474+0000", 2020-08-07T15:12:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.387466756000002, 2020-08-07T15:12:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0581683310000001, 2020-08-07T15:12:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118448 2020-08-07T15:12:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224474+0000", 2020-08-07T15:12:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224474+0000", 2020-08-07T15:12:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3804000000000002e-05 2020-08-07T15:12:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224528+0000", 2020-08-07T15:12:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5140000000000001e-06 2020-08-07T15:12:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224533+0000", 2020-08-07T15:12:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3315999999999999e-05 2020-08-07T15:12:38.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:38.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224546+0000", 2020-08-07T15:12:38.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7240000000000003e-06 2020-08-07T15:12:38.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:38.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224553+0000", 2020-08-07T15:12:38.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002226263 2020-08-07T15:12:38.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226779+0000", 2020-08-07T15:12:38.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1362000000000001e-05 2020-08-07T15:12:38.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226791+0000", 2020-08-07T15:12:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.046092652 2020-08-07T15:12:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.272883+0000", 2020-08-07T15:12:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000115981 2020-08-07T15:12:38.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:38.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.272999+0000", 2020-08-07T15:12:38.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029648899999999999 2020-08-07T15:12:38.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.273296+0000", 2020-08-07T15:12:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001852521 2020-08-07T15:12:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:38.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275148+0000", 2020-08-07T15:12:38.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051948309999999996 2020-08-07T15:12:38.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280343+0000", 2020-08-07T15:12:38.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033301799999999998 2020-08-07T15:12:38.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280676+0000", 2020-08-07T15:12:38.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8400000000000004e-07 2020-08-07T15:12:38.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:38.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280677+0000", 2020-08-07T15:12:38.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9652000000000002e-05 2020-08-07T15:12:38.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:38.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280756+0000", 2020-08-07T15:12:38.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017072699999999999 2020-08-07T15:12:38.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280927+0000", 2020-08-07T15:12:38.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015026530000000001 2020-08-07T15:12:38.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282430+0000", 2020-08-07T15:12:38.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4700000000000001e-07 2020-08-07T15:12:38.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:38.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282430+0000", 2020-08-07T15:12:38.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021169299999999999 2020-08-07T15:12:38.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:38.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282642+0000", 2020-08-07T15:12:38.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0581683310000001 2020-08-07T15:12:38.375 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:38.375 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:38.376 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:38.376 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:38.376 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:38.377 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:38.381 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 26 16 40091 40075 12.0402 11.7188 0.013174 0.0103746 2020-08-07T15:12:38.382 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:12:38.383 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:38.379+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:38.428 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.2 to 1 (10000) 2020-08-07T15:12:38.429 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:12:38.444 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 3 1 2020-08-07T15:12:38.481 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:38.479+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:38.573 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:38.573 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:12:38.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.574 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:40561 6.8 6:1c8963c3:::benchmark_data_smithi074_63843_object5070:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e251)", 2020-08-07T15:12:38.574 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:38.570488+0000", 2020-08-07T15:12:38.574 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.001769362, 2020-08-07T15:12:38.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017892509999999999, 2020-08-07T15:12:38.574 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:38.575 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:12:38.575 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:38.575 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:12:38.575 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:38.575 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 40561 2020-08-07T15:12:38.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.576 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:38.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:38.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.570488+0000", 2020-08-07T15:12:38.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:38.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:38.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.570488+0000", 2020-08-07T15:12:38.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5384999999999999e-05 2020-08-07T15:12:38.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:38.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.570543+0000", 2020-08-07T15:12:38.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2200000000000003e-06 2020-08-07T15:12:38.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:38.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.570550+0000", 2020-08-07T15:12:38.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5062000000000001e-05 2020-08-07T15:12:38.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:38.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.570566+0000", 2020-08-07T15:12:38.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.191e-06 2020-08-07T15:12:38.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:38.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.570574+0000", 2020-08-07T15:12:38.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2135999999999997e-05 2020-08-07T15:12:38.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.570626+0000", 2020-08-07T15:12:38.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016971600000000001 2020-08-07T15:12:38.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:38.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.570796+0000", 2020-08-07T15:12:38.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000215751 2020-08-07T15:12:38.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:12:38.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.571011+0000", 2020-08-07T15:12:38.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00052346100000000002 2020-08-07T15:12:38.585 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:38.585 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:38.585 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:38.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.586 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:40562 6.8 6:1c8963c3:::benchmark_data_smithi074_63843_object5070:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e251)", 2020-08-07T15:12:38.586 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:38.570620+0000", 2020-08-07T15:12:38.586 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.001637447, 2020-08-07T15:12:38.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017083269999999999, 2020-08-07T15:12:38.586 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:38.587 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:12:38.587 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:38.587 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:12:38.587 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:38.587 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 40562 2020-08-07T15:12:38.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.588 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:38.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:38.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.570620+0000", 2020-08-07T15:12:38.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:38.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:38.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.570620+0000", 2020-08-07T15:12:38.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6582999999999998e-05 2020-08-07T15:12:38.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:38.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.570666+0000", 2020-08-07T15:12:38.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.429e-06 2020-08-07T15:12:38.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:38.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.570672+0000", 2020-08-07T15:12:38.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1093999999999999e-05 2020-08-07T15:12:38.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:38.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.570683+0000", 2020-08-07T15:12:38.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7419999999999997e-06 2020-08-07T15:12:38.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:38.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.570688+0000", 2020-08-07T15:12:38.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000723378 2020-08-07T15:12:38.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.571411+0000", 2020-08-07T15:12:38.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4268000000000002e-05 2020-08-07T15:12:38.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:38.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.571475+0000", 2020-08-07T15:12:38.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000119605 2020-08-07T15:12:38.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:12:38.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.571595+0000", 2020-08-07T15:12:38.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000975099 2020-08-07T15:12:38.598 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:38.598 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:38.599 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:38.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.599 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:40488 6.5 6:aab4fda6:::benchmark_data_smithi074_63843_object5060:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e251)", 2020-08-07T15:12:38.599 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:38.522283+0000", 2020-08-07T15:12:38.600 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.049974345000000003, 2020-08-07T15:12:38.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050090548999999998, 2020-08-07T15:12:38.600 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:38.600 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:38.600 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:38.601 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:12:38.601 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:38.601 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 40488 2020-08-07T15:12:38.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.602 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:38.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:38.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.522283+0000", 2020-08-07T15:12:38.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:38.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:38.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.522283+0000", 2020-08-07T15:12:38.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7831000000000003e-05 2020-08-07T15:12:38.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:38.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.522331+0000", 2020-08-07T15:12:38.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1800000000000001e-06 2020-08-07T15:12:38.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:38.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.522337+0000", 2020-08-07T15:12:38.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2495e-05 2020-08-07T15:12:38.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:38.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.522349+0000", 2020-08-07T15:12:38.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2740000000000002e-06 2020-08-07T15:12:38.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:38.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.522357+0000", 2020-08-07T15:12:38.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5920000000000002e-05 2020-08-07T15:12:38.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:38.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.522393+0000", 2020-08-07T15:12:38.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5135000000000006e-05 2020-08-07T15:12:38.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:38.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.522458+0000", 2020-08-07T15:12:38.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107903 2020-08-07T15:12:38.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:12:38.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.522566+0000", 2020-08-07T15:12:38.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071908320000000003 2020-08-07T15:12:38.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:38.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.529756+0000", 2020-08-07T15:12:38.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6799999999999999e-07 2020-08-07T15:12:38.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:38.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.529757+0000", 2020-08-07T15:12:38.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7190999999999997e-05 2020-08-07T15:12:38.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:38.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.529795+0000", 2020-08-07T15:12:38.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00174971 2020-08-07T15:12:38.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:38.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:38.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:38.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:38.531544+0000", 2020-08-07T15:12:38.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0092613390000000004 2020-08-07T15:12:38.617 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:38.617 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:38.617 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:38.617 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:38.618 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:12:38.618 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 3 2020-08-07T15:12:38.618 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:38.621 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:12:38.622 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:38.618+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:38.721 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:38.719+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:38.820 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:12:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:12:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:12:38.821 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:38.823 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:38.822+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:38.833 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:12:38.927 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:38.924+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:39.026 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:39.024+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:39.063 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:39.063 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:12:39.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:12:39.064 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:12:39.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.064 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15576 3.e 3:727724a5:::benchmark_data_smithi074_35447_object1946:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:12:39.064 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:12:39.065 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 352.39310188299999, 2020-08-07T15:12:39.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999, 2020-08-07T15:12:39.065 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:39.065 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:39.065 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:39.066 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:12:39.066 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:12:39.066 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15576 2020-08-07T15:12:39.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.067 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:39.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:39.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:12:39.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:39.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:39.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:12:39.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7156e-05 2020-08-07T15:12:39.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:39.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666682+0000", 2020-08-07T15:12:39.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0209999999999999e-06 2020-08-07T15:12:39.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:39.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666687+0000", 2020-08-07T15:12:39.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0730999999999999e-05 2020-08-07T15:12:39.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:39.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666698+0000", 2020-08-07T15:12:39.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2020-08-07T15:12:39.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666702+0000", 2020-08-07T15:12:39.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023978459999999999 2020-08-07T15:12:39.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:39.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669100+0000", 2020-08-07T15:12:39.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9664000000000001e-05 2020-08-07T15:12:39.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:39.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669190+0000", 2020-08-07T15:12:39.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015957100000000001 2020-08-07T15:12:39.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:12:39.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669350+0000", 2020-08-07T15:12:39.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016470149999999999 2020-08-07T15:12:39.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:39.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:12:39.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1299999999999999e-07 2020-08-07T15:12:39.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:39.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:12:39.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5269000000000002e-05 2020-08-07T15:12:39.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:39.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.671022+0000", 2020-08-07T15:12:39.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031414400000000001 2020-08-07T15:12:39.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:39.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.674163+0000", 2020-08-07T15:12:39.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45641417400000001 2020-08-07T15:12:39.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:39.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:12:39.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0399999999999995e-07 2020-08-07T15:12:39.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:39.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:12:39.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3343999999999997e-05 2020-08-07T15:12:39.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:39.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130622+0000", 2020-08-07T15:12:39.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999 2020-08-07T15:12:39.084 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.085 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:39.085 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.085 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106248 3.f 3:f950fd9d:::benchmark_data_smithi074_35447_object13280:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:12:39.086 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:12:39.086 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 291.699572343, 2020-08-07T15:12:39.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002, 2020-08-07T15:12:39.086 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:39.086 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:39.087 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:39.087 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:12:39.087 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:12:39.087 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106248 2020-08-07T15:12:39.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.088 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:39.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:39.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:12:39.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:39.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:39.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:12:39.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7184e-05 2020-08-07T15:12:39.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:39.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360222+0000", 2020-08-07T15:12:39.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0906e-05 2020-08-07T15:12:39.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:39.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360233+0000", 2020-08-07T15:12:39.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2972999999999999e-05 2020-08-07T15:12:39.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:39.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360246+0000", 2020-08-07T15:12:39.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0114e-05 2020-08-07T15:12:39.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360256+0000", 2020-08-07T15:12:39.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1303999999999997e-05 2020-08-07T15:12:39.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:39.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360287+0000", 2020-08-07T15:12:39.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3798000000000001e-05 2020-08-07T15:12:39.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:39.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360351+0000", 2020-08-07T15:12:39.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111308 2020-08-07T15:12:39.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:12:39.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360462+0000", 2020-08-07T15:12:39.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003480674 2020-08-07T15:12:39.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:39.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:12:39.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0900000000000001e-07 2020-08-07T15:12:39.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:39.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:12:39.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8899999999999994e-05 2020-08-07T15:12:39.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:39.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.364012+0000", 2020-08-07T15:12:39.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35291637199999998 2020-08-07T15:12:39.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:39.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716929+0000", 2020-08-07T15:12:39.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.076244233999999994 2020-08-07T15:12:39.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:39.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793173+0000", 2020-08-07T15:12:39.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6109999999999999e-06 2020-08-07T15:12:39.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:39.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793175+0000", 2020-08-07T15:12:39.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7492999999999999e-05 2020-08-07T15:12:39.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:39.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793242+0000", 2020-08-07T15:12:39.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002 2020-08-07T15:12:39.114 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.114 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:39.114 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.115 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106248 3.f e59/20 ondisk, result = 0)", 2020-08-07T15:12:39.115 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:12:39.115 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 291.69430905799999, 2020-08-07T15:12:39.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001, 2020-08-07T15:12:39.116 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:39.117 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:39.117 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:39.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:39.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:12:39.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:39.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:39.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:12:39.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7518999999999999e-05 2020-08-07T15:12:39.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:39.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365445+0000", 2020-08-07T15:12:39.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-06 2020-08-07T15:12:39.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:39.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365449+0000", 2020-08-07T15:12:39.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42764147600000002 2020-08-07T15:12:39.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:39.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793091+0000", 2020-08-07T15:12:39.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6500000000000002e-06 2020-08-07T15:12:39.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793099+0000", 2020-08-07T15:12:39.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3664e-05 2020-08-07T15:12:39.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:39.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793153+0000", 2020-08-07T15:12:39.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1078e-05 2020-08-07T15:12:39.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:39.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793164+0000", 2020-08-07T15:12:39.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3404000000000003e-05 2020-08-07T15:12:39.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:39.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793248+0000", 2020-08-07T15:12:39.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001 2020-08-07T15:12:39.126 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.126 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:39.126 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.127 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:107952 3.d e61/35)", 2020-08-07T15:12:39.127 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:12:39.128 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 288.73337935000001, 2020-08-07T15:12:39.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001, 2020-08-07T15:12:39.128 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:39.128 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:39.128 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:39.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:39.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:12:39.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:39.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:39.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:12:39.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8826000000000003e-05 2020-08-07T15:12:39.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:39.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326406+0000", 2020-08-07T15:12:39.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9199999999999998e-06 2020-08-07T15:12:39.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:39.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326413+0000", 2020-08-07T15:12:39.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75082853500000002 2020-08-07T15:12:39.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:39.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077242+0000", 2020-08-07T15:12:39.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1479999999999999e-06 2020-08-07T15:12:39.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077246+0000", 2020-08-07T15:12:39.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3331999999999998e-05 2020-08-07T15:12:39.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:39.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077289+0000", 2020-08-07T15:12:39.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8977000000000001e-05 2020-08-07T15:12:39.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:39.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077308+0000", 2020-08-07T15:12:39.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001 2020-08-07T15:12:39.136 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.136 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:39.137 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.139 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31696 4.0 4:0335eed9:::benchmark_data_smithi074_44586_object3961:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e100)", 2020-08-07T15:12:39.139 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:12:39.139 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 230.95547179100001, 2020-08-07T15:12:39.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237, 2020-08-07T15:12:39.140 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:39.140 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:39.140 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:39.140 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:39.140 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:39.141 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31696 2020-08-07T15:12:39.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.141 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:39.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:39.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:12:39.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:39.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:39.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:12:39.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7654e-05 2020-08-07T15:12:39.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:39.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104293+0000", 2020-08-07T15:12:39.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.061e-06 2020-08-07T15:12:39.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:39.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104296+0000", 2020-08-07T15:12:39.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0859999999999998e-06 2020-08-07T15:12:39.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:39.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104302+0000", 2020-08-07T15:12:39.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8030000000000001e-06 2020-08-07T15:12:39.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104305+0000", 2020-08-07T15:12:39.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037634769999999999 2020-08-07T15:12:39.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:39.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108068+0000", 2020-08-07T15:12:39.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8814999999999998e-05 2020-08-07T15:12:39.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:39.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108137+0000", 2020-08-07T15:12:39.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111455 2020-08-07T15:12:39.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:12:39.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108248+0000", 2020-08-07T15:12:39.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028756820000000001 2020-08-07T15:12:39.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:39.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111124+0000", 2020-08-07T15:12:39.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061054499999999999 2020-08-07T15:12:39.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:39.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111735+0000", 2020-08-07T15:12:39.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.423966167 2020-08-07T15:12:39.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:39.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535701+0000", 2020-08-07T15:12:39.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4929999999999999e-06 2020-08-07T15:12:39.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:39.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535702+0000", 2020-08-07T15:12:39.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2571999999999997e-05 2020-08-07T15:12:39.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:39.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535755+0000", 2020-08-07T15:12:39.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1427000000000003e-05 2020-08-07T15:12:39.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:39.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535786+0000", 2020-08-07T15:12:39.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237 2020-08-07T15:12:39.156 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.157 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:39.157 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.157 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:31696 4.0 e100/81 ondisk, result = 0)", 2020-08-07T15:12:39.158 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:12:39.158 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 230.945857233, 2020-08-07T15:12:39.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954, 2020-08-07T15:12:39.158 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:39.158 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:39.159 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:39.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:39.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:12:39.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:39.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:39.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:12:39.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1185000000000001e-05 2020-08-07T15:12:39.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:39.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113901+0000", 2020-08-07T15:12:39.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2930000000000002e-06 2020-08-07T15:12:39.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:39.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113905+0000", 2020-08-07T15:12:39.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42168640400000001 2020-08-07T15:12:39.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:39.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535591+0000", 2020-08-07T15:12:39.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2820000000000002e-06 2020-08-07T15:12:39.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535598+0000", 2020-08-07T15:12:39.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6578999999999999e-05 2020-08-07T15:12:39.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:39.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535684+0000", 2020-08-07T15:12:39.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8449999999999992e-06 2020-08-07T15:12:39.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:39.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535693+0000", 2020-08-07T15:12:39.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7366000000000003e-05 2020-08-07T15:12:39.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:39.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535791+0000", 2020-08-07T15:12:39.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954 2020-08-07T15:12:39.167 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.167 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:39.168 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.168 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:37024 4.0 e106/81)", 2020-08-07T15:12:39.168 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:12:39.169 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 224.83106833900001, 2020-08-07T15:12:39.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999, 2020-08-07T15:12:39.169 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:39.169 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:39.169 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:39.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:39.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:12:39.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:39.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:39.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:12:39.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4926000000000001e-05 2020-08-07T15:12:39.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:39.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228683+0000", 2020-08-07T15:12:39.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1280000000000002e-06 2020-08-07T15:12:39.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:39.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228686+0000", 2020-08-07T15:12:39.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69740012100000004 2020-08-07T15:12:39.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:39.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926086+0000", 2020-08-07T15:12:39.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000004e-06 2020-08-07T15:12:39.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926090+0000", 2020-08-07T15:12:39.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113526 2020-08-07T15:12:39.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:39.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926204+0000", 2020-08-07T15:12:39.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8824000000000001e-05 2020-08-07T15:12:39.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:39.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926222+0000", 2020-08-07T15:12:39.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999 2020-08-07T15:12:39.177 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.177 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:39.178 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.179 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42503 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2615, mlcod=112'2607)", 2020-08-07T15:12:39.179 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:12:39.179 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 218.76349642299999, 2020-08-07T15:12:39.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998, 2020-08-07T15:12:39.180 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:39.180 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:39.180 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:39.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:39.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:12:39.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:39.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:39.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:12:39.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270999999999998e-05 2020-08-07T15:12:39.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:39.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296292+0000", 2020-08-07T15:12:39.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4729999999999997e-06 2020-08-07T15:12:39.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:39.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296298+0000", 2020-08-07T15:12:39.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.448795049 2020-08-07T15:12:39.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:39.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745093+0000", 2020-08-07T15:12:39.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0140000000000001e-06 2020-08-07T15:12:39.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745100+0000", 2020-08-07T15:12:39.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2596999999999998e-05 2020-08-07T15:12:39.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:39.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745153+0000", 2020-08-07T15:12:39.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1662000000000003e-05 2020-08-07T15:12:39.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:39.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745184+0000", 2020-08-07T15:12:39.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093174800000000004 2020-08-07T15:12:39.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:39.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746116+0000", 2020-08-07T15:12:39.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0926999999999994e-05 2020-08-07T15:12:39.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:39.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746187+0000", 2020-08-07T15:12:39.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998 2020-08-07T15:12:39.190 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.190 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:39.190 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.191 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42504 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2616, mlcod=113'2609)", 2020-08-07T15:12:39.191 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:12:39.191 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 218.76015953000001, 2020-08-07T15:12:39.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998, 2020-08-07T15:12:39.192 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:39.192 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:39.192 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:39.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:39.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:12:39.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:39.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:39.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:12:39.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2154000000000002e-05 2020-08-07T15:12:39.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:39.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299629+0000", 2020-08-07T15:12:39.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1929999999999998e-06 2020-08-07T15:12:39.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:39.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299636+0000", 2020-08-07T15:12:39.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44766315400000001 2020-08-07T15:12:39.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:39.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747299+0000", 2020-08-07T15:12:39.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5889999999999999e-06 2020-08-07T15:12:39.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747305+0000", 2020-08-07T15:12:39.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3094999999999999e-05 2020-08-07T15:12:39.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:39.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747358+0000", 2020-08-07T15:12:39.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3283e-05 2020-08-07T15:12:39.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:39.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747392+0000", 2020-08-07T15:12:39.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015319509999999999 2020-08-07T15:12:39.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:39.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748924+0000", 2020-08-07T15:12:39.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5235999999999996e-05 2020-08-07T15:12:39.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:39.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748999+0000", 2020-08-07T15:12:39.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998 2020-08-07T15:12:39.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.202 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:39.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.204 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42520 4.8 e113/81 4:13071de5:::benchmark_data_smithi074_44586_object5314:head v 113'2624, mlcod=113'2618)", 2020-08-07T15:12:39.204 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:12:39.205 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 218.294215321, 2020-08-07T15:12:39.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001, 2020-08-07T15:12:39.205 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:39.205 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:39.205 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:39.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:39.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:12:39.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:39.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:39.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:12:39.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3967999999999995e-05 2020-08-07T15:12:39.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:39.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765585+0000", 2020-08-07T15:12:39.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0949999999999998e-06 2020-08-07T15:12:39.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:39.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765593+0000", 2020-08-07T15:12:39.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52703464200000005 2020-08-07T15:12:39.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:39.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292627+0000", 2020-08-07T15:12:39.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6929999999999996e-06 2020-08-07T15:12:39.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292636+0000", 2020-08-07T15:12:39.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9292000000000001e-05 2020-08-07T15:12:39.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:39.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292695+0000", 2020-08-07T15:12:39.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5395999999999999e-05 2020-08-07T15:12:39.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:39.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292721+0000", 2020-08-07T15:12:39.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00090361399999999996 2020-08-07T15:12:39.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:39.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293624+0000", 2020-08-07T15:12:39.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1761000000000002e-05 2020-08-07T15:12:39.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:39.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293666+0000", 2020-08-07T15:12:39.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001 2020-08-07T15:12:39.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.216 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:39.216 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.216 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:19776 5.4 5:2bb0c0d9:::benchmark_data_smithi074_53697_object2471:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e173)", 2020-08-07T15:12:39.217 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:12:39.217 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 126.66002263599999, 2020-08-07T15:12:39.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.402061844, 2020-08-07T15:12:39.217 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:39.217 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:39.218 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:39.218 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:39.218 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:39.219 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 19776 2020-08-07T15:12:39.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.219 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:39.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:39.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:12:39.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:39.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:39.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:12:39.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-05 2020-08-07T15:12:39.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:39.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399752+0000", 2020-08-07T15:12:39.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8799999999999999e-06 2020-08-07T15:12:39.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:39.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399757+0000", 2020-08-07T15:12:39.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7690000000000008e-06 2020-08-07T15:12:39.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:39.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399767+0000", 2020-08-07T15:12:39.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-06 2020-08-07T15:12:39.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399771+0000", 2020-08-07T15:12:39.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003255109 2020-08-07T15:12:39.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:39.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403026+0000", 2020-08-07T15:12:39.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101044 2020-08-07T15:12:39.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:39.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403127+0000", 2020-08-07T15:12:39.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000193158 2020-08-07T15:12:39.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,5", 2020-08-07T15:12:39.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403320+0000", 2020-08-07T15:12:39.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017185530000000001 2020-08-07T15:12:39.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:39.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405038+0000", 2020-08-07T15:12:39.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6300000000000004e-07 2020-08-07T15:12:39.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:39.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405039+0000", 2020-08-07T15:12:39.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012590600000000001 2020-08-07T15:12:39.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:39.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405165+0000", 2020-08-07T15:12:39.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109757 2020-08-07T15:12:39.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:39.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405275+0000", 2020-08-07T15:12:39.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39642174699999999 2020-08-07T15:12:39.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:39.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801697+0000", 2020-08-07T15:12:39.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2799999999999996e-07 2020-08-07T15:12:39.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:39.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801697+0000", 2020-08-07T15:12:39.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8740999999999997e-05 2020-08-07T15:12:39.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:39.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801776+0000", 2020-08-07T15:12:39.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.402061844 2020-08-07T15:12:39.235 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.236 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:39.236 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.236 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:19776 5.4 e173/164 ondisk, result = 0)", 2020-08-07T15:12:39.237 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:12:39.237 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 126.654274333, 2020-08-07T15:12:39.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39631765200000002, 2020-08-07T15:12:39.237 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:39.237 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:39.238 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:39.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:39.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:12:39.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:39.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:39.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:12:39.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9332e-05 2020-08-07T15:12:39.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:39.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405492+0000", 2020-08-07T15:12:39.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8900000000000004e-06 2020-08-07T15:12:39.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:39.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405498+0000", 2020-08-07T15:12:39.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39566664200000001 2020-08-07T15:12:39.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:39.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801164+0000", 2020-08-07T15:12:39.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7440000000000001e-06 2020-08-07T15:12:39.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801168+0000", 2020-08-07T15:12:39.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050828199999999998 2020-08-07T15:12:39.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:39.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801676+0000", 2020-08-07T15:12:39.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1011e-05 2020-08-07T15:12:39.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:39.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801687+0000", 2020-08-07T15:12:39.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2750999999999993e-05 2020-08-07T15:12:39.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:39.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801780+0000", 2020-08-07T15:12:39.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39631765200000002 2020-08-07T15:12:39.248 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.248 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:39.248 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.249 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:85551 5.9 5:9d69b7cc:::benchmark_data_smithi074_53697_object10693:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e202)", 2020-08-07T15:12:39.249 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.807277+0000", 2020-08-07T15:12:39.249 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.252460266, 2020-08-07T15:12:39.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39614286500000001, 2020-08-07T15:12:39.250 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:39.250 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:39.250 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:39.251 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:39.251 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:39.251 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 85551 2020-08-07T15:12:39.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.251 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:39.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:39.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807277+0000", 2020-08-07T15:12:39.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:39.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:39.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807277+0000", 2020-08-07T15:12:39.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8867999999999999e-05 2020-08-07T15:12:39.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807305+0000", 2020-08-07T15:12:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.775e-06 2020-08-07T15:12:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807310+0000", 2020-08-07T15:12:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2747e-05 2020-08-07T15:12:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:39.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807323+0000", 2020-08-07T15:12:39.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2669999999999997e-06 2020-08-07T15:12:39.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807328+0000", 2020-08-07T15:12:39.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001048513 2020-08-07T15:12:39.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:39.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.808377+0000", 2020-08-07T15:12:39.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010425399999999999 2020-08-07T15:12:39.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:39.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.808481+0000", 2020-08-07T15:12:39.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013761900000000001 2020-08-07T15:12:39.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:12:39.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.808619+0000", 2020-08-07T15:12:39.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042907500000000003 2020-08-07T15:12:39.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:39.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.812909+0000", 2020-08-07T15:12:39.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.077e-06 2020-08-07T15:12:39.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:39.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.812910+0000", 2020-08-07T15:12:39.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8186000000000001e-05 2020-08-07T15:12:39.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:39.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.812979+0000", 2020-08-07T15:12:39.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00081526099999999996 2020-08-07T15:12:39.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:39.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813794+0000", 2020-08-07T15:12:39.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38955804599999999 2020-08-07T15:12:39.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:39.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203352+0000", 2020-08-07T15:12:39.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5239999999999999e-06 2020-08-07T15:12:39.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:39.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203353+0000", 2020-08-07T15:12:39.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5977999999999997e-05 2020-08-07T15:12:39.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:39.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203419+0000", 2020-08-07T15:12:39.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39614286500000001 2020-08-07T15:12:39.267 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.267 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:39.268 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.268 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:85552 5.9 5:9d69b7cc:::benchmark_data_smithi074_53697_object10693:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e202)", 2020-08-07T15:12:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:12:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.252219542000006, 2020-08-07T15:12:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54429374399999997, 2020-08-07T15:12:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:39.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:39.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:39.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:39.270 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 85552 2020-08-07T15:12:39.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.271 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:39.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:39.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:12:39.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:39.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:39.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:12:39.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2691000000000001e-05 2020-08-07T15:12:39.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:39.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807570+0000", 2020-08-07T15:12:39.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0619999999999997e-06 2020-08-07T15:12:39.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:39.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807576+0000", 2020-08-07T15:12:39.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1558e-05 2020-08-07T15:12:39.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:39.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807588+0000", 2020-08-07T15:12:39.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4600000000000002e-06 2020-08-07T15:12:39.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807593+0000", 2020-08-07T15:12:39.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022126279999999999 2020-08-07T15:12:39.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:39.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.809806+0000", 2020-08-07T15:12:39.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104868 2020-08-07T15:12:39.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:39.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.809911+0000", 2020-08-07T15:12:39.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000186019 2020-08-07T15:12:39.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:12:39.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.810097+0000", 2020-08-07T15:12:39.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037966739999999999 2020-08-07T15:12:39.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:39.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813893+0000", 2020-08-07T15:12:39.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1310000000000001e-06 2020-08-07T15:12:39.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:39.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813894+0000", 2020-08-07T15:12:39.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4704999999999999e-05 2020-08-07T15:12:39.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:39.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813949+0000", 2020-08-07T15:12:39.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015487280000000001 2020-08-07T15:12:39.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:39.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.815498+0000", 2020-08-07T15:12:39.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53621442900000005 2020-08-07T15:12:39.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:39.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351712+0000", 2020-08-07T15:12:39.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8199999999999998e-07 2020-08-07T15:12:39.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:39.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351713+0000", 2020-08-07T15:12:39.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7909000000000001e-05 2020-08-07T15:12:39.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:39.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351811+0000", 2020-08-07T15:12:39.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54429374399999997 2020-08-07T15:12:39.287 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.287 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:39.287 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.288 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:85551 5.9 e202/164 ondisk, result = 0)", 2020-08-07T15:12:39.288 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.813683+0000", 2020-08-07T15:12:39.288 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.246054239000003, 2020-08-07T15:12:39.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38973959000000002, 2020-08-07T15:12:39.288 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:39.289 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:39.289 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:39.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:39.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813683+0000", 2020-08-07T15:12:39.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:39.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:39.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813683+0000", 2020-08-07T15:12:39.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9434999999999999e-05 2020-08-07T15:12:39.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:39.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813712+0000", 2020-08-07T15:12:39.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6759999999999998e-06 2020-08-07T15:12:39.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:39.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813717+0000", 2020-08-07T15:12:39.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.389373209 2020-08-07T15:12:39.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:39.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203090+0000", 2020-08-07T15:12:39.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0859999999999998e-06 2020-08-07T15:12:39.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203096+0000", 2020-08-07T15:12:39.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024188899999999999 2020-08-07T15:12:39.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:39.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203338+0000", 2020-08-07T15:12:39.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2490000000000001e-06 2020-08-07T15:12:39.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:39.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203346+0000", 2020-08-07T15:12:39.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6045999999999997e-05 2020-08-07T15:12:39.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:39.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.203422+0000", 2020-08-07T15:12:39.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.38973959000000002 2020-08-07T15:12:39.299 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.300 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:39.300 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.300 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:85552 5.9 e202/164 ondisk, result = 0)", 2020-08-07T15:12:39.301 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:12:39.301 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.245993139999996, 2020-08-07T15:12:39.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53813836800000003, 2020-08-07T15:12:39.301 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:39.301 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:39.309 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:39.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:39.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:12:39.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:39.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:39.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:12:39.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8833e-05 2020-08-07T15:12:39.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:39.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813772+0000", 2020-08-07T15:12:39.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8459999999999999e-06 2020-08-07T15:12:39.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:39.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813777+0000", 2020-08-07T15:12:39.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53732224900000003 2020-08-07T15:12:39.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:39.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351100+0000", 2020-08-07T15:12:39.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1869999999999996e-06 2020-08-07T15:12:39.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351105+0000", 2020-08-07T15:12:39.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00058858100000000002 2020-08-07T15:12:39.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:39.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351693+0000", 2020-08-07T15:12:39.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4229999999999994e-06 2020-08-07T15:12:39.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:39.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351703+0000", 2020-08-07T15:12:39.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000179249 2020-08-07T15:12:39.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:39.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351882+0000", 2020-08-07T15:12:39.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53813836800000003 2020-08-07T15:12:39.321 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.321 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:39.321 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.322 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:85552 5.9 5:9d69b7cc:::benchmark_data_smithi074_53697_object10693:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e202)", 2020-08-07T15:12:39.322 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:12:39.322 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.241245015999993, 2020-08-07T15:12:39.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53326610600000002, 2020-08-07T15:12:39.323 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:39.323 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:12:39.323 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:39.323 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:39.324 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:39.324 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 85552 2020-08-07T15:12:39.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.324 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:39.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:39.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:12:39.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:39.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:39.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:12:39.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5001e-05 2020-08-07T15:12:39.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:39.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818517+0000", 2020-08-07T15:12:39.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5000000000000002e-06 2020-08-07T15:12:39.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:39.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818519+0000", 2020-08-07T15:12:39.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0699999999999997e-06 2020-08-07T15:12:39.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:39.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818524+0000", 2020-08-07T15:12:39.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2910000000000002e-06 2020-08-07T15:12:39.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818527+0000", 2020-08-07T15:12:39.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030912499999999999 2020-08-07T15:12:39.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:39.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818836+0000", 2020-08-07T15:12:39.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2908000000000002e-05 2020-08-07T15:12:39.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for ondisk", 2020-08-07T15:12:39.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818889+0000", 2020-08-07T15:12:39.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53286921099999995 2020-08-07T15:12:39.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:39.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351758+0000", 2020-08-07T15:12:39.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53326610600000002 2020-08-07T15:12:39.333 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.333 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:39.333 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.334 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:89464 5.9 5:94c69ba5:::benchmark_data_smithi074_53697_object11182:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:39.334 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:12:39.334 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.643405061999999, 2020-08-07T15:12:39.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67692916199999997, 2020-08-07T15:12:39.335 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:39.335 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:39.335 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:39.335 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:39.336 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:39.336 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 89464 2020-08-07T15:12:39.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.336 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:39.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:39.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:12:39.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:39.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:39.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:12:39.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0074000000000002e-05 2020-08-07T15:12:39.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:39.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416382+0000", 2020-08-07T15:12:39.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5000000000000001e-06 2020-08-07T15:12:39.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:39.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416386+0000", 2020-08-07T15:12:39.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2447e-05 2020-08-07T15:12:39.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:39.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416399+0000", 2020-08-07T15:12:39.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1279999999999996e-06 2020-08-07T15:12:39.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416405+0000", 2020-08-07T15:12:39.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016641805999999999 2020-08-07T15:12:39.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:39.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433047+0000", 2020-08-07T15:12:39.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107995 2020-08-07T15:12:39.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:39.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433155+0000", 2020-08-07T15:12:39.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018818599999999999 2020-08-07T15:12:39.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:12:39.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433343+0000", 2020-08-07T15:12:39.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039911679999999998 2020-08-07T15:12:39.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:39.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437334+0000", 2020-08-07T15:12:39.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2020-08-07T15:12:39.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:39.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437335+0000", 2020-08-07T15:12:39.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6351000000000002e-05 2020-08-07T15:12:39.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:39.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437401+0000", 2020-08-07T15:12:39.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001267949 2020-08-07T15:12:39.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:39.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438669+0000", 2020-08-07T15:12:39.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65454035600000005 2020-08-07T15:12:39.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:39.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093209+0000", 2020-08-07T15:12:39.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7859999999999999e-06 2020-08-07T15:12:39.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:39.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093211+0000", 2020-08-07T15:12:39.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9665e-05 2020-08-07T15:12:39.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:39.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093261+0000", 2020-08-07T15:12:39.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67692916199999997 2020-08-07T15:12:39.352 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.352 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:39.352 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.353 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:89464 5.9 e207/164 ondisk, result = 0)", 2020-08-07T15:12:39.353 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:12:39.353 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.620775300000005, 2020-08-07T15:12:39.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65430647200000003, 2020-08-07T15:12:39.354 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:39.354 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:39.354 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:39.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:39.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:12:39.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:39.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:39.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:12:39.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6529000000000002e-05 2020-08-07T15:12:39.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:39.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438998+0000", 2020-08-07T15:12:39.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.567e-06 2020-08-07T15:12:39.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:39.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.439005+0000", 2020-08-07T15:12:39.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65408475099999996 2020-08-07T15:12:39.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:39.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093089+0000", 2020-08-07T15:12:39.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6880000000000002e-06 2020-08-07T15:12:39.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093096+0000", 2020-08-07T15:12:39.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4980000000000002e-05 2020-08-07T15:12:39.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:39.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093191+0000", 2020-08-07T15:12:39.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3090000000000001e-06 2020-08-07T15:12:39.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:39.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093200+0000", 2020-08-07T15:12:39.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7648e-05 2020-08-07T15:12:39.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:39.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093268+0000", 2020-08-07T15:12:39.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65430647200000003 2020-08-07T15:12:39.363 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.363 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:39.363 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.364 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:114080 5.4 5:24adcd54:::benchmark_data_smithi074_53697_object14259:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e215)", 2020-08-07T15:12:39.364 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:12:39.364 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 59.460067842999997, 2020-08-07T15:12:39.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.57290862300000001, 2020-08-07T15:12:39.365 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:39.365 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:39.365 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:39.365 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:39.365 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:39.366 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 114080 2020-08-07T15:12:39.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.366 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:39.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:39.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:12:39.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:39.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:39.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:12:39.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9238e-05 2020-08-07T15:12:39.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:39.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599698+0000", 2020-08-07T15:12:39.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6260000000000003e-06 2020-08-07T15:12:39.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:39.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599703+0000", 2020-08-07T15:12:39.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5577999999999999e-05 2020-08-07T15:12:39.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:39.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599718+0000", 2020-08-07T15:12:39.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9869999999999999e-06 2020-08-07T15:12:39.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599726+0000", 2020-08-07T15:12:39.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00077929599999999996 2020-08-07T15:12:39.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:39.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600506+0000", 2020-08-07T15:12:39.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1717000000000003e-05 2020-08-07T15:12:39.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:39.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600597+0000", 2020-08-07T15:12:39.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011828999999999999 2020-08-07T15:12:39.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,5", 2020-08-07T15:12:39.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600716+0000", 2020-08-07T15:12:39.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020861479999999999 2020-08-07T15:12:39.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:39.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602802+0000", 2020-08-07T15:12:39.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4600000000000005e-07 2020-08-07T15:12:39.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:39.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602802+0000", 2020-08-07T15:12:39.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9165e-05 2020-08-07T15:12:39.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:39.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602862+0000", 2020-08-07T15:12:39.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0093485080000000002 2020-08-07T15:12:39.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:39.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.612210+0000", 2020-08-07T15:12:39.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.56030197699999995 2020-08-07T15:12:39.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:39.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172512+0000", 2020-08-07T15:12:39.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8800000000000001e-07 2020-08-07T15:12:39.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:39.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172513+0000", 2020-08-07T15:12:39.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4659000000000006e-05 2020-08-07T15:12:39.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:39.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:39.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:39.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172578+0000", 2020-08-07T15:12:39.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.57290862300000001 2020-08-07T15:12:39.381 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.381 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:39.381 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.382 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:39.382 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:39.382 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:39.383 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 27 16 41836 41820 12.0991 13.6328 0.00234933 0.0103251 2020-08-07T15:12:39.384 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:12:39.385 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:39.383+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:39.435 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.3 to 1 (10000) 2020-08-07T15:12:39.451 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 4 1 2020-08-07T15:12:39.488 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:39.487+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:39.572 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:12:39.582 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:39.583 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:12:39.583 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.583 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:42208 6.e 6:706ecd5b:::benchmark_data_smithi074_63843_object5275:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e252)", 2020-08-07T15:12:39.583 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:39.494630+0000", 2020-08-07T15:12:39.583 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.086161536999999996, 2020-08-07T15:12:39.584 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.086195174999999999, 2020-08-07T15:12:39.584 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:39.584 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:39.584 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:39.584 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:39.585 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:39.585 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42208 2020-08-07T15:12:39.585 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.585 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:39.585 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.586 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:39.586 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.494630+0000", 2020-08-07T15:12:39.586 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:39.587 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.587 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.587 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:39.587 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.494630+0000", 2020-08-07T15:12:39.588 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4549999999999998e-05 2020-08-07T15:12:39.588 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.588 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:39.589 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.494684+0000", 2020-08-07T15:12:39.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0440000000000003e-06 2020-08-07T15:12:39.589 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.590 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:39.590 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.494689+0000", 2020-08-07T15:12:39.590 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1226000000000001e-05 2020-08-07T15:12:39.590 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.590 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.591 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:39.591 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.494701+0000", 2020-08-07T15:12:39.591 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9409999999999998e-06 2020-08-07T15:12:39.591 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.591 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.592 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.592 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.494706+0000", 2020-08-07T15:12:39.592 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0091600039999999994 2020-08-07T15:12:39.592 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.592 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.592 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:39.593 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.503866+0000", 2020-08-07T15:12:39.593 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6057999999999994e-05 2020-08-07T15:12:39.593 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.593 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.593 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:39.594 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.503962+0000", 2020-08-07T15:12:39.594 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000189421 2020-08-07T15:12:39.594 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.594 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.594 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:12:39.595 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.504151+0000", 2020-08-07T15:12:39.595 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0022941760000000002 2020-08-07T15:12:39.595 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.595 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.595 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:39.596 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.506445+0000", 2020-08-07T15:12:39.597 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3699999999999999e-07 2020-08-07T15:12:39.597 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.597 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.598 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:39.598 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.506446+0000", 2020-08-07T15:12:39.598 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3221999999999998e-05 2020-08-07T15:12:39.598 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.598 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.599 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:39.599 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.506519+0000", 2020-08-07T15:12:39.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010420850000000001 2020-08-07T15:12:39.599 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.599 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.600 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:39.600 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.507561+0000", 2020-08-07T15:12:39.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.012931564 2020-08-07T15:12:39.600 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:39.600 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:39.601 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:39.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.601 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:42315 6.4 6:2fcaeaec:::benchmark_data_smithi074_63843_object5289:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e252)", 2020-08-07T15:12:39.602 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:39.564683+0000", 2020-08-07T15:12:39.602 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.016108567000000001, 2020-08-07T15:12:39.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.016261113000000001, 2020-08-07T15:12:39.602 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:39.602 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:39.603 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:39.603 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:39.603 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:39.603 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42315 2020-08-07T15:12:39.603 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.604 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:39.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:39.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.564683+0000", 2020-08-07T15:12:39.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:39.605 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.605 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.605 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:39.605 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.564683+0000", 2020-08-07T15:12:39.605 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.969e-05 2020-08-07T15:12:39.606 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.606 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.606 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:39.606 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.564732+0000", 2020-08-07T15:12:39.606 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3960000000000006e-06 2020-08-07T15:12:39.607 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.607 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.607 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:39.607 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.564741+0000", 2020-08-07T15:12:39.607 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3436e-05 2020-08-07T15:12:39.608 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.608 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.608 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:39.608 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.564754+0000", 2020-08-07T15:12:39.608 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3360000000000002e-06 2020-08-07T15:12:39.608 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.609 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.609 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.609 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.564760+0000", 2020-08-07T15:12:39.609 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0014956189999999999 2020-08-07T15:12:39.609 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.610 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.610 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:39.610 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.566255+0000", 2020-08-07T15:12:39.611 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5847999999999995e-05 2020-08-07T15:12:39.611 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.611 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.611 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:39.611 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.566341+0000", 2020-08-07T15:12:39.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015039599999999999 2020-08-07T15:12:39.612 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.612 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.612 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,5", 2020-08-07T15:12:39.612 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.566491+0000", 2020-08-07T15:12:39.613 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001537954 2020-08-07T15:12:39.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.614 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:39.614 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.568029+0000", 2020-08-07T15:12:39.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7599999999999996e-07 2020-08-07T15:12:39.615 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.616 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.616 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:39.616 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.568030+0000", 2020-08-07T15:12:39.618 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0519000000000001e-05 2020-08-07T15:12:39.618 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.618 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.618 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:39.619 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.568091+0000", 2020-08-07T15:12:39.619 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0080312979999999992 2020-08-07T15:12:39.619 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.619 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.619 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:39.620 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.576122+0000", 2020-08-07T15:12:39.620 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.011439268000000001 2020-08-07T15:12:39.620 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:39.620 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:39.620 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:39.621 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.621 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.621 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:42316 6.4 6:2fcaeaec:::benchmark_data_smithi074_63843_object5289:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e252)", 2020-08-07T15:12:39.621 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:39.568456+0000", 2020-08-07T15:12:39.621 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.012334974, 2020-08-07T15:12:39.622 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.012601858000000001, 2020-08-07T15:12:39.622 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:39.622 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:39.622 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:39.622 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:39.623 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:39.623 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42316 2020-08-07T15:12:39.623 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.623 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:39.623 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.623 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:39.624 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.568456+0000", 2020-08-07T15:12:39.624 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:39.624 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.624 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.624 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:39.625 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.568456+0000", 2020-08-07T15:12:39.625 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2695000000000003e-05 2020-08-07T15:12:39.625 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.625 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.625 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:39.626 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.568509+0000", 2020-08-07T15:12:39.626 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5930000000000002e-06 2020-08-07T15:12:39.626 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.626 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.626 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:39.627 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.568516+0000", 2020-08-07T15:12:39.627 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3529999999999999e-05 2020-08-07T15:12:39.627 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.630 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:39.630 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.568529+0000", 2020-08-07T15:12:39.630 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6409999999999996e-06 2020-08-07T15:12:39.630 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.630 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.631 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.631 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.568536+0000", 2020-08-07T15:12:39.631 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014704900000000001 2020-08-07T15:12:39.631 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.631 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.632 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:39.632 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.568683+0000", 2020-08-07T15:12:39.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1187000000000006e-05 2020-08-07T15:12:39.632 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.632 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.633 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:39.633 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.568774+0000", 2020-08-07T15:12:39.633 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017884199999999999 2020-08-07T15:12:39.633 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.633 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.634 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,5", 2020-08-07T15:12:39.634 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.568953+0000", 2020-08-07T15:12:39.634 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019498390000000001 2020-08-07T15:12:39.634 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.634 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.635 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:39.635 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.570903+0000", 2020-08-07T15:12:39.635 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.187e-06 2020-08-07T15:12:39.635 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.635 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.635 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:39.636 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.570904+0000", 2020-08-07T15:12:39.636 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0846000000000006e-05 2020-08-07T15:12:39.636 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.636 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.636 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:39.637 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.570975+0000", 2020-08-07T15:12:39.637 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0051818330000000003 2020-08-07T15:12:39.637 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.637 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.637 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:39.638 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.576157+0000", 2020-08-07T15:12:39.638 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0077002420000000004 2020-08-07T15:12:39.638 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:39.638 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:39.638 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:39.639 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.639 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.639 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:42317 6.4 6:2fcaeaec:::benchmark_data_smithi074_63843_object5289:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e252)", 2020-08-07T15:12:39.639 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:39.568593+0000", 2020-08-07T15:12:39.639 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.012198426, 2020-08-07T15:12:39.640 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.012578068, 2020-08-07T15:12:39.640 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:39.640 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:39.640 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:39.640 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:39.640 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:39.641 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42317 2020-08-07T15:12:39.641 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.641 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:39.641 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.641 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:39.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.568593+0000", 2020-08-07T15:12:39.642 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:39.642 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.642 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.642 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:39.643 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.568593+0000", 2020-08-07T15:12:39.643 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8114e-05 2020-08-07T15:12:39.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.643 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.643 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:39.644 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.568641+0000", 2020-08-07T15:12:39.644 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9109999999999998e-06 2020-08-07T15:12:39.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.644 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.644 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:39.645 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.568647+0000", 2020-08-07T15:12:39.645 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8604e-05 2020-08-07T15:12:39.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:39.645 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.568666+0000", 2020-08-07T15:12:39.646 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8170000000000004e-06 2020-08-07T15:12:39.647 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.647 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.647 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.647 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.568671+0000", 2020-08-07T15:12:39.647 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00067947500000000002 2020-08-07T15:12:39.648 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.648 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.648 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:39.648 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.569351+0000", 2020-08-07T15:12:39.648 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000103812 2020-08-07T15:12:39.649 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.649 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.649 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:39.649 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.569455+0000", 2020-08-07T15:12:39.649 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017241399999999999 2020-08-07T15:12:39.650 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.650 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.650 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,5", 2020-08-07T15:12:39.650 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.569627+0000", 2020-08-07T15:12:39.650 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001357296 2020-08-07T15:12:39.650 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.651 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.651 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:39.651 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.570984+0000", 2020-08-07T15:12:39.651 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8800000000000003e-07 2020-08-07T15:12:39.651 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.652 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.652 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:39.652 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.570985+0000", 2020-08-07T15:12:39.652 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7428e-05 2020-08-07T15:12:39.652 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.653 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.653 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:39.653 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.571042+0000", 2020-08-07T15:12:39.653 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0051480600000000003 2020-08-07T15:12:39.653 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.654 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.654 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:39.654 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.576190+0000", 2020-08-07T15:12:39.654 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0075974190000000002 2020-08-07T15:12:39.654 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:39.654 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:39.655 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:39.655 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.655 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.655 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:42318 6.4 6:2fcaeaec:::benchmark_data_smithi074_63843_object5289:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e252)", 2020-08-07T15:12:39.655 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:39.571299+0000", 2020-08-07T15:12:39.656 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0094919589999999995, 2020-08-07T15:12:39.656 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0099513510000000006, 2020-08-07T15:12:39.656 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:39.656 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:39.657 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:39.657 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:39.657 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:39.657 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42318 2020-08-07T15:12:39.657 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.658 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:39.658 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.658 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:39.658 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.571299+0000", 2020-08-07T15:12:39.658 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:39.659 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.659 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.659 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:39.659 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.571299+0000", 2020-08-07T15:12:39.659 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0355000000000002e-05 2020-08-07T15:12:39.660 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.660 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.660 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:39.660 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.571350+0000", 2020-08-07T15:12:39.660 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6370000000000002e-06 2020-08-07T15:12:39.660 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.661 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.661 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:39.661 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.571356+0000", 2020-08-07T15:12:39.661 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2686e-05 2020-08-07T15:12:39.661 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.662 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.662 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:39.662 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.571369+0000", 2020-08-07T15:12:39.662 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.896e-06 2020-08-07T15:12:39.662 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.663 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.663 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.663 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.571375+0000", 2020-08-07T15:12:39.663 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000155227 2020-08-07T15:12:39.663 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.664 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:39.664 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.571530+0000", 2020-08-07T15:12:39.664 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9869000000000002e-05 2020-08-07T15:12:39.664 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.665 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:39.665 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.571630+0000", 2020-08-07T15:12:39.665 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000204966 2020-08-07T15:12:39.665 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.665 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.666 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,5", 2020-08-07T15:12:39.666 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.571835+0000", 2020-08-07T15:12:39.666 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023832580000000001 2020-08-07T15:12:39.666 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.666 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.667 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:39.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.574218+0000", 2020-08-07T15:12:39.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8299999999999997e-07 2020-08-07T15:12:39.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:39.668 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.574219+0000", 2020-08-07T15:12:39.668 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3161000000000001e-05 2020-08-07T15:12:39.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:39.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.574282+0000", 2020-08-07T15:12:39.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0029955429999999998 2020-08-07T15:12:39.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:39.670 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.577277+0000", 2020-08-07T15:12:39.670 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0059780809999999997 2020-08-07T15:12:39.670 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:39.670 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:39.670 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:39.671 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.671 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.671 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:42319 6.4 6:2fcaeaec:::benchmark_data_smithi074_63843_object5289:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e252)", 2020-08-07T15:12:39.671 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:39.571415+0000", 2020-08-07T15:12:39.671 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0093759659999999995, 2020-08-07T15:12:39.672 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0099068090000000008, 2020-08-07T15:12:39.672 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:39.672 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:39.672 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:39.672 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:39.672 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:39.673 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42319 2020-08-07T15:12:39.673 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.673 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:39.673 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.673 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:39.674 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.571415+0000", 2020-08-07T15:12:39.674 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:39.674 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.674 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.674 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:39.675 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.571415+0000", 2020-08-07T15:12:39.675 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7172000000000002e-05 2020-08-07T15:12:39.675 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.675 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.675 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:39.676 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.571463+0000", 2020-08-07T15:12:39.676 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1889999999999998e-06 2020-08-07T15:12:39.676 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.676 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.676 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:39.676 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.571468+0000", 2020-08-07T15:12:39.677 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0795000000000001e-05 2020-08-07T15:12:39.677 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:39.677 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.571479+0000", 2020-08-07T15:12:39.678 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5000000000000001e-06 2020-08-07T15:12:39.678 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.678 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.678 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.678 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.571483+0000", 2020-08-07T15:12:39.679 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00075612900000000002 2020-08-07T15:12:39.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.679 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:39.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.572239+0000", 2020-08-07T15:12:39.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5289000000000002e-05 2020-08-07T15:12:39.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.680 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:39.680 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.572334+0000", 2020-08-07T15:12:39.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016937500000000001 2020-08-07T15:12:39.681 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.681 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,5", 2020-08-07T15:12:39.681 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.572504+0000", 2020-08-07T15:12:39.681 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001863183 2020-08-07T15:12:39.682 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.682 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.682 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:39.682 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.574367+0000", 2020-08-07T15:12:39.682 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7899999999999998e-07 2020-08-07T15:12:39.683 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.683 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.683 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:39.683 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.574368+0000", 2020-08-07T15:12:39.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9205e-05 2020-08-07T15:12:39.683 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.684 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.684 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:39.684 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.574427+0000", 2020-08-07T15:12:39.684 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028872329999999999 2020-08-07T15:12:39.684 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.685 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.685 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:39.685 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.577314+0000", 2020-08-07T15:12:39.685 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0058986489999999997 2020-08-07T15:12:39.685 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:39.686 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:39.686 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:39.686 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.686 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.686 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:42320 6.4 6:2fcaeaec:::benchmark_data_smithi074_63843_object5289:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e252)", 2020-08-07T15:12:39.687 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:39.573252+0000", 2020-08-07T15:12:39.687 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0075388419999999996, 2020-08-07T15:12:39.687 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0081803440000000009, 2020-08-07T15:12:39.687 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:39.687 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:39.687 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:39.688 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:39.688 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:39.688 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42320 2020-08-07T15:12:39.688 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.688 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:39.689 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.689 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:39.689 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.573252+0000", 2020-08-07T15:12:39.689 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:39.689 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.690 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.690 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:39.690 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.573252+0000", 2020-08-07T15:12:39.690 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9190999999999997e-05 2020-08-07T15:12:39.691 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.691 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.691 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:39.691 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.573302+0000", 2020-08-07T15:12:39.691 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4540000000000004e-06 2020-08-07T15:12:39.691 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.692 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:39.692 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.573308+0000", 2020-08-07T15:12:39.692 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3101e-05 2020-08-07T15:12:39.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.693 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.693 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:39.693 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.573321+0000", 2020-08-07T15:12:39.693 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.105e-06 2020-08-07T15:12:39.693 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.694 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.694 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:39.694 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.573327+0000", 2020-08-07T15:12:39.694 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6587e-05 2020-08-07T15:12:39.694 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.695 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:39.695 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.573374+0000", 2020-08-07T15:12:39.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8122999999999995e-05 2020-08-07T15:12:39.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.696 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:39.696 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.573472+0000", 2020-08-07T15:12:39.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000329845 2020-08-07T15:12:39.697 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,5", 2020-08-07T15:12:39.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.573802+0000", 2020-08-07T15:12:39.697 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00087066200000000002 2020-08-07T15:12:39.697 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.698 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.698 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:39.698 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.574673+0000", 2020-08-07T15:12:39.698 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7300000000000005e-07 2020-08-07T15:12:39.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.699 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.699 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:39.699 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.574673+0000", 2020-08-07T15:12:39.699 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.049e-05 2020-08-07T15:12:39.699 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.700 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:39.700 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.574714+0000", 2020-08-07T15:12:39.700 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0027020109999999998 2020-08-07T15:12:39.700 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:39.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:39.701 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:39.701 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:39.577416+0000", 2020-08-07T15:12:39.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041634419999999998 2020-08-07T15:12:39.701 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:39.701 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:39.702 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:39.702 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:39.702 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:12:39.702 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 7 2020-08-07T15:12:39.702 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:39.704 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:12:39.705 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:39.702+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:39.785 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:12:39.805 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:39.804+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:39.828 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:12:39.865 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:12:39.908 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:39.907+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:39.924 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:12:39.944 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:39.944 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:39.958 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:12:39.963 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:12:40.002 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:12:40.008 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:40.007+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:40.050 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:12:40.110 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:40.109+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:40.144 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:40.144 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:40.166 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:12:40.211 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:40.210+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:40.268 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 28 16 43284 43268 12.071 11.3125 0.00725154 0.0103486 2020-08-07T15:12:40.313 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:40.311+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:40.373 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:40.373 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:40.388 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:12:40.415 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:40.413+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:40.441 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.4 to 1 (10000) 2020-08-07T15:12:40.444 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:12:40.461 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 5 1 2020-08-07T15:12:40.515 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:40.514+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:40.582 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:40.582 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:40.597 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:12:40.646 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:40.645+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:40.721 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:40.717+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:40.819 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:40.817+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:40.830 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:40.830 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:40.838 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:12:40.895 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:12:40.901 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:12:40.901 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:12:40.902 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:12:40.902 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:12:40.902 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:12:40.902 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:12:40.903 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:12:40.903 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:12:40.920 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:40.918+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:41.021 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:41.019+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:41.093 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:41.094 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:41.100 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:12:41.148 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:41.147+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:41.229 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:41.228+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:41.270 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 29 16 44848 44832 12.076 12.2188 0.00672132 0.0103471 2020-08-07T15:12:41.330 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:41.329+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:41.331 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:41.331 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:41.352 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:12:41.432 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:41.431+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:41.467 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:12:41.479 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.5 to 1 (10000) 2020-08-07T15:12:41.493 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 6 1 2020-08-07T15:12:41.533 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:41.532+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:41.615 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:41.616 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:41.629 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:12:41.634 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:41.633+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:41.736 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:41.734+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:41.832 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:41.833 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:41.840 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:41.837+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:41.848 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:12:41.947 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:41.946+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:42.052 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:42.051+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:42.076 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:42.076 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:42.081 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:12:42.157 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:42.153+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:42.231 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.6 to 1 (10000) 2020-08-07T15:12:42.243 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 7 1 2020-08-07T15:12:42.255 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:42.254+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:42.268 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 30 16 46392 46376 12.0756 12.0625 0.00197558 0.0103248 2020-08-07T15:12:42.301 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:42.301 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:42.329 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:12:42.373 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:42.372+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:42.481 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:42.473+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:42.482 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:42.482 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:42.499 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:12:42.586 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:42.576+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:42.673 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:42.674 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:42.688 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:12:42.689 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:42.687+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:42.794 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:42.789+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:42.894 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:42.889+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:42.910 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:42.910 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:42.921 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:12:42.930 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:12:42.992 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:42.991+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:43.075 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:43.075 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:43.087 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:12:43.092 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:43.091+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:43.169 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:12:43.204 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:43.203+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:43.205 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:12:43.240 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.7 to 1 (10000) 2020-08-07T15:12:43.248 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:12:43.249 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:43.249 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:43.261 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:12:43.262 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2020-08-07T15:12:43.262 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:12:43.273 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 31 16 47912 47896 12.0691 11.875 0.007818 0.0103532 2020-08-07T15:12:43.299 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:12:43.307 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:43.304+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:43.344 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:12:43.394 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:12:43.647 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:12:43.655 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:43.655 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:12:43.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:12:43.656 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:12:43.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.656 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42440 4.9 4:9b3fd4e3:::benchmark_data_smithi074_44586_object5304:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:12:43.656 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:12:43.657 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 223.24163878600001, 2020-08-07T15:12:43.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9384075030000001, 2020-08-07T15:12:43.657 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:43.657 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:43.658 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:43.659 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:43.659 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:43.659 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42440 2020-08-07T15:12:43.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.660 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:43.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:43.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:12:43.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:43.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:43.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:12:43.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6295000000000002e-05 2020-08-07T15:12:43.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:43.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237116+0000", 2020-08-07T15:12:43.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0390000000000001e-06 2020-08-07T15:12:43.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:43.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237122+0000", 2020-08-07T15:12:43.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8340000000000001e-05 2020-08-07T15:12:43.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:43.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237140+0000", 2020-08-07T15:12:43.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.045e-06 2020-08-07T15:12:43.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:43.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237145+0000", 2020-08-07T15:12:43.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044548920000000002 2020-08-07T15:12:43.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:43.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241600+0000", 2020-08-07T15:12:43.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103336 2020-08-07T15:12:43.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:43.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241703+0000", 2020-08-07T15:12:43.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018660399999999999 2020-08-07T15:12:43.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:12:43.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241890+0000", 2020-08-07T15:12:43.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00096796000000000002 2020-08-07T15:12:43.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:43.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.242858+0000", 2020-08-07T15:12:43.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039845000000000002 2020-08-07T15:12:43.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:43.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246843+0000", 2020-08-07T15:12:43.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3499999999999996e-07 2020-08-07T15:12:43.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:43.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246843+0000", 2020-08-07T15:12:43.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7132999999999995e-05 2020-08-07T15:12:43.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:43.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246920+0000", 2020-08-07T15:12:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046993410000000001 2020-08-07T15:12:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.251620+0000", 2020-08-07T15:12:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040332956000000003 2020-08-07T15:12:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.291953+0000", 2020-08-07T15:12:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8390000000000003e-06 2020-08-07T15:12:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.291958+0000", 2020-08-07T15:12:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99961357500000003 2020-08-07T15:12:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291572+0000", 2020-08-07T15:12:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3913999999999999e-05 2020-08-07T15:12:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291586+0000", 2020-08-07T15:12:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88379723099999996 2020-08-07T15:12:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175383+0000", 2020-08-07T15:12:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3868000000000004e-05 2020-08-07T15:12:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175467+0000", 2020-08-07T15:12:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9384075030000001 2020-08-07T15:12:43.718 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:43.725 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:43.725 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:43.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.726 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42489 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:12:43.726 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:12:43.726 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 223.198338724, 2020-08-07T15:12:43.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952608639999999, 2020-08-07T15:12:43.727 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:43.727 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:43.727 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:43.727 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:43.728 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:43.728 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42489 2020-08-07T15:12:43.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.728 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:43.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:43.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:12:43.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:43.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:43.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:12:43.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0379e-05 2020-08-07T15:12:43.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:43.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280410+0000", 2020-08-07T15:12:43.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0489999999999998e-06 2020-08-07T15:12:43.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:43.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280417+0000", 2020-08-07T15:12:43.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8281e-05 2020-08-07T15:12:43.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:43.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280435+0000", 2020-08-07T15:12:43.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7510000000000008e-06 2020-08-07T15:12:43.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:43.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280443+0000", 2020-08-07T15:12:43.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018142899999999999 2020-08-07T15:12:43.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280624+0000", 2020-08-07T15:12:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013473599999999999 2020-08-07T15:12:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280759+0000", 2020-08-07T15:12:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019964200000000001 2020-08-07T15:12:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:12:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280959+0000", 2020-08-07T15:12:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005588038 2020-08-07T15:12:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286547+0000", 2020-08-07T15:12:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.376e-05 2020-08-07T15:12:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286561+0000", 2020-08-07T15:12:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4199999999999995e-07 2020-08-07T15:12:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286561+0000", 2020-08-07T15:12:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9218e-05 2020-08-07T15:12:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286631+0000", 2020-08-07T15:12:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027909199999999999 2020-08-07T15:12:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286910+0000", 2020-08-07T15:12:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050904089999999997 2020-08-07T15:12:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292000+0000", 2020-08-07T15:12:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9579999999999996e-06 2020-08-07T15:12:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292009+0000", 2020-08-07T15:12:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99961402799999999 2020-08-07T15:12:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291623+0000", 2020-08-07T15:12:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2490000000000001e-06 2020-08-07T15:12:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291631+0000", 2020-08-07T15:12:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88387422199999999 2020-08-07T15:12:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175506+0000", 2020-08-07T15:12:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000114881 2020-08-07T15:12:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175620+0000", 2020-08-07T15:12:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952608639999999 2020-08-07T15:12:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42490 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:12:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:12:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 223.19821155599999, 2020-08-07T15:12:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89528059, 2020-08-07T15:12:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:43.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:43.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:43.786 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42490 2020-08-07T15:12:43.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.786 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:43.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:43.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:12:43.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:43.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:43.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:12:43.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2571999999999997e-05 2020-08-07T15:12:43.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:43.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280539+0000", 2020-08-07T15:12:43.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999998e-06 2020-08-07T15:12:43.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:43.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280545+0000", 2020-08-07T15:12:43.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.134e-05 2020-08-07T15:12:43.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:43.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280556+0000", 2020-08-07T15:12:43.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.746e-06 2020-08-07T15:12:43.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:43.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280561+0000", 2020-08-07T15:12:43.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00089258999999999996 2020-08-07T15:12:43.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:43.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281453+0000", 2020-08-07T15:12:43.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5641000000000001e-05 2020-08-07T15:12:43.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:43.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281549+0000", 2020-08-07T15:12:43.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020555200000000001 2020-08-07T15:12:43.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:12:43.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281755+0000", 2020-08-07T15:12:43.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048878230000000003 2020-08-07T15:12:43.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:43.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286642+0000", 2020-08-07T15:12:43.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0954e-05 2020-08-07T15:12:43.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:43.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286693+0000", 2020-08-07T15:12:43.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0699999999999997e-07 2020-08-07T15:12:43.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:43.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286694+0000", 2020-08-07T15:12:43.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3697999999999998e-05 2020-08-07T15:12:43.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:43.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286768+0000", 2020-08-07T15:12:43.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017185900000000001 2020-08-07T15:12:43.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:43.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286940+0000", 2020-08-07T15:12:43.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051207819999999999 2020-08-07T15:12:43.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:43.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292060+0000", 2020-08-07T15:12:43.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8000000000000004e-06 2020-08-07T15:12:43.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:43.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292069+0000", 2020-08-07T15:12:43.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99959520599999996 2020-08-07T15:12:43.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:43.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291664+0000", 2020-08-07T15:12:43.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0160000000000008e-06 2020-08-07T15:12:43.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:43.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291672+0000", 2020-08-07T15:12:43.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88399374500000005 2020-08-07T15:12:43.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:43.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175666+0000", 2020-08-07T15:12:43.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101239 2020-08-07T15:12:43.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:43.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175767+0000", 2020-08-07T15:12:43.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89528059 2020-08-07T15:12:43.828 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:43.828 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:43.828 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:43.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.829 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42491 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:12:43.829 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:12:43.830 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 223.198104621, 2020-08-07T15:12:43.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895291936, 2020-08-07T15:12:43.830 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:43.830 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:43.831 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:43.831 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:43.831 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:43.831 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42491 2020-08-07T15:12:43.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.832 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:43.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:43.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:12:43.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:43.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:43.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:12:43.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0402000000000001e-05 2020-08-07T15:12:43.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:43.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280624+0000", 2020-08-07T15:12:43.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0340000000000001e-06 2020-08-07T15:12:43.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:43.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280627+0000", 2020-08-07T15:12:43.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1886999999999999e-05 2020-08-07T15:12:43.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:43.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280649+0000", 2020-08-07T15:12:43.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5430000000000004e-06 2020-08-07T15:12:43.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:43.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280654+0000", 2020-08-07T15:12:43.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032800960000000001 2020-08-07T15:12:43.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:43.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.283934+0000", 2020-08-07T15:12:43.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011264699999999999 2020-08-07T15:12:43.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:43.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284046+0000", 2020-08-07T15:12:43.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019987299999999999 2020-08-07T15:12:43.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:12:43.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284246+0000", 2020-08-07T15:12:43.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025610960000000001 2020-08-07T15:12:43.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:43.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286807+0000", 2020-08-07T15:12:43.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1409999999999999e-06 2020-08-07T15:12:43.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:43.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286814+0000", 2020-08-07T15:12:43.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0499999999999999e-07 2020-08-07T15:12:43.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:43.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286815+0000", 2020-08-07T15:12:43.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8538000000000003e-05 2020-08-07T15:12:43.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:43.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286853+0000", 2020-08-07T15:12:43.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112914 2020-08-07T15:12:43.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:43.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286966+0000", 2020-08-07T15:12:43.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051403120000000002 2020-08-07T15:12:43.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:43.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292107+0000", 2020-08-07T15:12:43.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8019999999999992e-06 2020-08-07T15:12:43.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:43.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292114+0000", 2020-08-07T15:12:43.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.999590859 2020-08-07T15:12:43.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:43.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291705+0000", 2020-08-07T15:12:43.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.174e-06 2020-08-07T15:12:43.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:43.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291713+0000", 2020-08-07T15:12:43.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88407676000000002 2020-08-07T15:12:43.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:43.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175790+0000", 2020-08-07T15:12:43.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5452999999999996e-05 2020-08-07T15:12:43.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:43.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175886+0000", 2020-08-07T15:12:43.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895291936 2020-08-07T15:12:43.925 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:43.926 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:43.926 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:43.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.927 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42492 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:12:43.927 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:12:43.927 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 223.198011571, 2020-08-07T15:12:43.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895293774, 2020-08-07T15:12:43.928 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:43.928 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:43.928 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:43.928 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:43.928 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:43.929 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42492 2020-08-07T15:12:43.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.929 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:43.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:43.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:12:43.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:43.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:43.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:12:43.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8272000000000001e-05 2020-08-07T15:12:43.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:43.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280735+0000", 2020-08-07T15:12:43.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2159999999999997e-06 2020-08-07T15:12:43.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:43.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280740+0000", 2020-08-07T15:12:43.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0251000000000001e-05 2020-08-07T15:12:43.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:43.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280750+0000", 2020-08-07T15:12:43.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3139999999999997e-06 2020-08-07T15:12:43.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:43.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280755+0000", 2020-08-07T15:12:43.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039343579999999998 2020-08-07T15:12:43.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:43.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284689+0000", 2020-08-07T15:12:43.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111648 2020-08-07T15:12:43.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:43.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284801+0000", 2020-08-07T15:12:43.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019563600000000001 2020-08-07T15:12:43.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:12:43.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284996+0000", 2020-08-07T15:12:43.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018636709999999999 2020-08-07T15:12:43.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:43.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286860+0000", 2020-08-07T15:12:43.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001865802 2020-08-07T15:12:43.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:43.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288726+0000", 2020-08-07T15:12:43.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0399999999999996e-07 2020-08-07T15:12:43.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:43.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288726+0000", 2020-08-07T15:12:43.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6573999999999998e-05 2020-08-07T15:12:43.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:43.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288773+0000", 2020-08-07T15:12:43.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003455283 2020-08-07T15:12:43.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:43.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292228+0000", 2020-08-07T15:12:43.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0099999999999995e-06 2020-08-07T15:12:43.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:43.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292236+0000", 2020-08-07T15:12:43.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99950963999999998 2020-08-07T15:12:43.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:43.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291746+0000", 2020-08-07T15:12:43.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0779999999999996e-06 2020-08-07T15:12:43.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:43.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291754+0000", 2020-08-07T15:12:43.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.884152508 2020-08-07T15:12:43.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:43.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175906+0000", 2020-08-07T15:12:43.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4009000000000002e-05 2020-08-07T15:12:43.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:43.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175980+0000", 2020-08-07T15:12:43.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895293774 2020-08-07T15:12:43.956 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:43.956 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:43.957 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:43.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.957 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42493 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:12:43.957 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:12:43.958 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 223.19790992399999, 2020-08-07T15:12:43.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952963789999999, 2020-08-07T15:12:43.959 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:43.959 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:43.959 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:43.959 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:43.959 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:43.960 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42493 2020-08-07T15:12:43.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.960 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:43.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:43.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:12:43.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:43.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:43.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:12:43.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9740999999999997e-05 2020-08-07T15:12:43.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:43.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280838+0000", 2020-08-07T15:12:43.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7809999999999996e-06 2020-08-07T15:12:43.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:43.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280843+0000", 2020-08-07T15:12:43.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0648999999999999e-05 2020-08-07T15:12:43.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:43.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280854+0000", 2020-08-07T15:12:43.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3669999999999999e-06 2020-08-07T15:12:43.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:43.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280858+0000", 2020-08-07T15:12:43.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004850867 2020-08-07T15:12:43.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:43.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.285709+0000", 2020-08-07T15:12:43.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110037 2020-08-07T15:12:43.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:43.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.285819+0000", 2020-08-07T15:12:43.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020935699999999999 2020-08-07T15:12:43.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:12:43.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286028+0000", 2020-08-07T15:12:43.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020900850000000002 2020-08-07T15:12:43.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:43.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288118+0000", 2020-08-07T15:12:43.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041669649999999999 2020-08-07T15:12:43.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:43.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292285+0000", 2020-08-07T15:12:43.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3820000000000004e-06 2020-08-07T15:12:43.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:43.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292294+0000", 2020-08-07T15:12:43.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949242900000002 2020-08-07T15:12:43.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:43.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291786+0000", 2020-08-07T15:12:43.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000003e-06 2020-08-07T15:12:43.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:43.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:43.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:43.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291794+0000", 2020-08-07T15:12:44.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88420652700000002 2020-08-07T15:12:44.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176000+0000", 2020-08-07T15:12:44.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4634000000000003e-05 2020-08-07T15:12:44.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:44.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176085+0000", 2020-08-07T15:12:44.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952963789999999 2020-08-07T15:12:44.013 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.013 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:44.013 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.014 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42494 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:12:44.014 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:12:44.014 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 223.19780782500001, 2020-08-07T15:12:44.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952882820000001, 2020-08-07T15:12:44.015 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:44.015 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:44.015 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:44.015 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:44.015 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:44.016 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42494 2020-08-07T15:12:44.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.016 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:44.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:44.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:12:44.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:44.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:44.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:12:44.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0117999999999997e-05 2020-08-07T15:12:44.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:44.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280941+0000", 2020-08-07T15:12:44.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1240000000000004e-06 2020-08-07T15:12:44.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:44.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280946+0000", 2020-08-07T15:12:44.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0000000000000001e-05 2020-08-07T15:12:44.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:44.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280956+0000", 2020-08-07T15:12:44.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.279e-06 2020-08-07T15:12:44.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:44.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280960+0000", 2020-08-07T15:12:44.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0062727080000000001 2020-08-07T15:12:44.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287233+0000", 2020-08-07T15:12:44.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018362900000000001 2020-08-07T15:12:44.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:44.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287416+0000", 2020-08-07T15:12:44.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029536100000000001 2020-08-07T15:12:44.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:12:44.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287712+0000", 2020-08-07T15:12:44.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046231739999999999 2020-08-07T15:12:44.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292335+0000", 2020-08-07T15:12:44.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3860000000000001e-06 2020-08-07T15:12:44.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:44.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292342+0000", 2020-08-07T15:12:44.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99948501000000001 2020-08-07T15:12:44.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291827+0000", 2020-08-07T15:12:44.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2272999999999999e-05 2020-08-07T15:12:44.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:44.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291840+0000", 2020-08-07T15:12:44.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88426668100000005 2020-08-07T15:12:44.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176106+0000", 2020-08-07T15:12:44.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2539000000000005e-05 2020-08-07T15:12:44.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:44.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176179+0000", 2020-08-07T15:12:44.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952882820000001 2020-08-07T15:12:44.048 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.049 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:44.049 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.065 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42495 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:12:44.066 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:12:44.066 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 223.197706735, 2020-08-07T15:12:44.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895286867, 2020-08-07T15:12:44.066 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:44.066 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:44.067 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:44.067 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:44.067 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:44.067 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42495 2020-08-07T15:12:44.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.068 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:44.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:44.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:12:44.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:44.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:44.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:12:44.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6690999999999997e-05 2020-08-07T15:12:44.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:44.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281038+0000", 2020-08-07T15:12:44.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7860000000000002e-06 2020-08-07T15:12:44.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:44.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281044+0000", 2020-08-07T15:12:44.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0429e-05 2020-08-07T15:12:44.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:44.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281054+0000", 2020-08-07T15:12:44.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9040000000000002e-06 2020-08-07T15:12:44.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:44.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281058+0000", 2020-08-07T15:12:44.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070941490000000001 2020-08-07T15:12:44.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288153+0000", 2020-08-07T15:12:44.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9006999999999998e-05 2020-08-07T15:12:44.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:44.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288222+0000", 2020-08-07T15:12:44.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012106400000000001 2020-08-07T15:12:44.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:12:44.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288343+0000", 2020-08-07T15:12:44.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004034752 2020-08-07T15:12:44.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292377+0000", 2020-08-07T15:12:44.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.537e-06 2020-08-07T15:12:44.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:44.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292385+0000", 2020-08-07T15:12:44.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949123500000003 2020-08-07T15:12:44.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291876+0000", 2020-08-07T15:12:44.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7729999999999999e-06 2020-08-07T15:12:44.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:44.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291884+0000", 2020-08-07T15:12:44.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88431700899999999 2020-08-07T15:12:44.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176201+0000", 2020-08-07T15:12:44.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7531e-05 2020-08-07T15:12:44.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:44.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176278+0000", 2020-08-07T15:12:44.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895286867 2020-08-07T15:12:44.110 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.110 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:44.110 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.111 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42496 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:12:44.111 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:12:44.112 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 223.19760182600001, 2020-08-07T15:12:44.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89526942, 2020-08-07T15:12:44.113 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:44.113 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:44.113 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:44.113 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:44.113 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:44.114 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42496 2020-08-07T15:12:44.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.114 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:44.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:44.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:12:44.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:44.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:44.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:12:44.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9227999999999997e-05 2020-08-07T15:12:44.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:44.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281146+0000", 2020-08-07T15:12:44.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1120000000000004e-06 2020-08-07T15:12:44.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:44.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281151+0000", 2020-08-07T15:12:44.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2208999999999999e-05 2020-08-07T15:12:44.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:44.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281163+0000", 2020-08-07T15:12:44.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5469999999999998e-06 2020-08-07T15:12:44.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:44.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281168+0000", 2020-08-07T15:12:44.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0076520829999999996 2020-08-07T15:12:44.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288820+0000", 2020-08-07T15:12:44.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5035000000000003e-05 2020-08-07T15:12:44.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:44.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288885+0000", 2020-08-07T15:12:44.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116507 2020-08-07T15:12:44.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:12:44.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.289001+0000", 2020-08-07T15:12:44.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034121960000000002 2020-08-07T15:12:44.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292413+0000", 2020-08-07T15:12:44.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8109999999999997e-06 2020-08-07T15:12:44.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:44.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292420+0000", 2020-08-07T15:12:44.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949504600000005 2020-08-07T15:12:44.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291915+0000", 2020-08-07T15:12:44.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3999999999999992e-06 2020-08-07T15:12:44.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:44.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291924+0000", 2020-08-07T15:12:44.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88437394599999997 2020-08-07T15:12:44.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176298+0000", 2020-08-07T15:12:44.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8300000000000007e-05 2020-08-07T15:12:44.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:44.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176366+0000", 2020-08-07T15:12:44.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89526942 2020-08-07T15:12:44.132 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.132 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:44.132 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.133 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73081 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:12:44.133 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:12:44.133 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 196.25213255200001, 2020-08-07T15:12:44.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0005038420000001, 2020-08-07T15:12:44.134 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:44.134 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:44.135 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:44.135 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:44.135 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:44.135 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73081 2020-08-07T15:12:44.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.135 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:44.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:44.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:12:44.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:44.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:44.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:12:44.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3016000000000003e-05 2020-08-07T15:12:44.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:44.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226609+0000", 2020-08-07T15:12:44.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4479999999999996e-06 2020-08-07T15:12:44.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:44.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226613+0000", 2020-08-07T15:12:44.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0189999999999995e-06 2020-08-07T15:12:44.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:44.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226622+0000", 2020-08-07T15:12:44.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5980000000000001e-06 2020-08-07T15:12:44.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:44.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226626+0000", 2020-08-07T15:12:44.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8088999999999999e-05 2020-08-07T15:12:44.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226684+0000", 2020-08-07T15:12:44.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2162e-05 2020-08-07T15:12:44.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:44.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226696+0000", 2020-08-07T15:12:44.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98675689 2020-08-07T15:12:44.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213453+0000", 2020-08-07T15:12:44.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012916800000000001 2020-08-07T15:12:44.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:44.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213582+0000", 2020-08-07T15:12:44.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015665899999999999 2020-08-07T15:12:44.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:12:44.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213739+0000", 2020-08-07T15:12:44.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049940239999999997 2020-08-07T15:12:44.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:44.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218733+0000", 2020-08-07T15:12:44.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0079774779999999997 2020-08-07T15:12:44.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:44.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226710+0000", 2020-08-07T15:12:44.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.161e-06 2020-08-07T15:12:44.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:44.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226711+0000", 2020-08-07T15:12:44.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0105000000000001e-05 2020-08-07T15:12:44.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:44.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226772+0000", 2020-08-07T15:12:44.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5633999999999997e-05 2020-08-07T15:12:44.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:44.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226817+0000", 2020-08-07T15:12:44.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7379000000000001e-05 2020-08-07T15:12:44.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:44.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226875+0000", 2020-08-07T15:12:44.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8000000000000005e-07 2020-08-07T15:12:44.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:44.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226875+0000", 2020-08-07T15:12:44.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019423199999999999 2020-08-07T15:12:44.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:44.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227070+0000", 2020-08-07T15:12:44.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0005038420000001 2020-08-07T15:12:44.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.198 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:44.198 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.198 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73082 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:12:44.199 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:12:44.199 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 196.25176208799999, 2020-08-07T15:12:44.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001326693, 2020-08-07T15:12:44.199 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:44.199 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:44.200 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:44.200 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:44.200 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:44.200 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73082 2020-08-07T15:12:44.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.201 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:44.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:44.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:12:44.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:44.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:44.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:12:44.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6928000000000003e-05 2020-08-07T15:12:44.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226973+0000", 2020-08-07T15:12:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3630000000000004e-06 2020-08-07T15:12:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226977+0000", 2020-08-07T15:12:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9659999999999996e-06 2020-08-07T15:12:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226985+0000", 2020-08-07T15:12:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5800000000000002e-06 2020-08-07T15:12:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226990+0000", 2020-08-07T15:12:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9101999999999998e-05 2020-08-07T15:12:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227039+0000", 2020-08-07T15:12:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9620000000000005e-06 2020-08-07T15:12:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227049+0000", 2020-08-07T15:12:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98722513999999995 2020-08-07T15:12:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214274+0000", 2020-08-07T15:12:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2700999999999994e-05 2020-08-07T15:12:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214347+0000", 2020-08-07T15:12:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015346899999999999 2020-08-07T15:12:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:12:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214500+0000", 2020-08-07T15:12:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042450309999999998 2020-08-07T15:12:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218745+0000", 2020-08-07T15:12:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008379503 2020-08-07T15:12:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227125+0000", 2020-08-07T15:12:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7899999999999998e-07 2020-08-07T15:12:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227126+0000", 2020-08-07T15:12:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1610000000000002e-05 2020-08-07T15:12:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227177+0000", 2020-08-07T15:12:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489000000000003e-05 2020-08-07T15:12:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227226+0000", 2020-08-07T15:12:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00084338600000000001 2020-08-07T15:12:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228069+0000", 2020-08-07T15:12:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7e-07 2020-08-07T15:12:44.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228069+0000", 2020-08-07T15:12:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019351399999999999 2020-08-07T15:12:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228263+0000", 2020-08-07T15:12:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001326693 2020-08-07T15:12:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73083 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:12:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:12:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 196.25166782900001, 2020-08-07T15:12:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015968209999999, 2020-08-07T15:12:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73083 2020-08-07T15:12:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:12:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:12:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1627000000000001e-05 2020-08-07T15:12:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227062+0000", 2020-08-07T15:12:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.253e-06 2020-08-07T15:12:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227065+0000", 2020-08-07T15:12:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3168e-05 2020-08-07T15:12:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227079+0000", 2020-08-07T15:12:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1549999999999999e-06 2020-08-07T15:12:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227082+0000", 2020-08-07T15:12:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5534999999999997e-05 2020-08-07T15:12:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227127+0000", 2020-08-07T15:12:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6823000000000001e-05 2020-08-07T15:12:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227144+0000", 2020-08-07T15:12:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98771770599999997 2020-08-07T15:12:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214862+0000", 2020-08-07T15:12:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8252999999999999e-05 2020-08-07T15:12:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214930+0000", 2020-08-07T15:12:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013646400000000001 2020-08-07T15:12:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:12:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215066+0000", 2020-08-07T15:12:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036907089999999999 2020-08-07T15:12:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218757+0000", 2020-08-07T15:12:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0085133450000000003 2020-08-07T15:12:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227270+0000", 2020-08-07T15:12:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4899999999999995e-07 2020-08-07T15:12:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227271+0000", 2020-08-07T15:12:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4555999999999997e-05 2020-08-07T15:12:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227326+0000", 2020-08-07T15:12:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024594800000000001 2020-08-07T15:12:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227572+0000", 2020-08-07T15:12:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00086125700000000004 2020-08-07T15:12:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228433+0000", 2020-08-07T15:12:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3799999999999998e-07 2020-08-07T15:12:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228433+0000", 2020-08-07T15:12:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019413500000000001 2020-08-07T15:12:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228627+0000", 2020-08-07T15:12:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015968209999999 2020-08-07T15:12:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73084 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:12:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:12:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 196.25157888999999, 2020-08-07T15:12:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0016919049999999, 2020-08-07T15:12:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73084 2020-08-07T15:12:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:12:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:12:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5565999999999997e-05 2020-08-07T15:12:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227165+0000", 2020-08-07T15:12:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2020-08-07T15:12:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227169+0000", 2020-08-07T15:12:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6850000000000001e-06 2020-08-07T15:12:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227176+0000", 2020-08-07T15:12:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1949999999999999e-06 2020-08-07T15:12:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227179+0000", 2020-08-07T15:12:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0318000000000003e-05 2020-08-07T15:12:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227220+0000", 2020-08-07T15:12:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0349e-05 2020-08-07T15:12:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227230+0000", 2020-08-07T15:12:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98832962400000002 2020-08-07T15:12:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215560+0000", 2020-08-07T15:12:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013913099999999999 2020-08-07T15:12:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215699+0000", 2020-08-07T15:12:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000200006 2020-08-07T15:12:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:12:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215899+0000", 2020-08-07T15:12:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028673650000000002 2020-08-07T15:12:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218766+0000", 2020-08-07T15:12:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086101779999999996 2020-08-07T15:12:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227376+0000", 2020-08-07T15:12:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0999999999999998e-07 2020-08-07T15:12:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227377+0000", 2020-08-07T15:12:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6552999999999997e-05 2020-08-07T15:12:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227454+0000", 2020-08-07T15:12:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000421717 2020-08-07T15:12:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227875+0000", 2020-08-07T15:12:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00081781300000000003 2020-08-07T15:12:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228693+0000", 2020-08-07T15:12:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1699999999999997e-07 2020-08-07T15:12:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228694+0000", 2020-08-07T15:12:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117483 2020-08-07T15:12:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228811+0000", 2020-08-07T15:12:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0016919049999999 2020-08-07T15:12:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73085 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:12:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:12:44.298 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 196.251443075, 2020-08-07T15:12:44.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0018015629999999, 2020-08-07T15:12:44.302 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73085 2020-08-07T15:12:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:12:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:12:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6570999999999997e-05 2020-08-07T15:12:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227322+0000", 2020-08-07T15:12:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5509999999999996e-06 2020-08-07T15:12:44.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227328+0000", 2020-08-07T15:12:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2498e-05 2020-08-07T15:12:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227341+0000", 2020-08-07T15:12:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4389999999999999e-06 2020-08-07T15:12:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:44.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227346+0000", 2020-08-07T15:12:44.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3711999999999998e-05 2020-08-07T15:12:44.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227390+0000", 2020-08-07T15:12:44.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1868999999999999e-05 2020-08-07T15:12:44.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:44.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227402+0000", 2020-08-07T15:12:44.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98896525099999999 2020-08-07T15:12:44.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216367+0000", 2020-08-07T15:12:44.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012573 2020-08-07T15:12:44.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:44.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216493+0000", 2020-08-07T15:12:44.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020525200000000001 2020-08-07T15:12:44.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:12:44.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216698+0000", 2020-08-07T15:12:44.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037596470000000001 2020-08-07T15:12:44.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:44.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.220458+0000", 2020-08-07T15:12:44.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070227190000000002 2020-08-07T15:12:44.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:44.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227480+0000", 2020-08-07T15:12:44.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6600000000000003e-07 2020-08-07T15:12:44.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:44.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227481+0000", 2020-08-07T15:12:44.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9898000000000003e-05 2020-08-07T15:12:44.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:44.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227521+0000", 2020-08-07T15:12:44.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039130000000000002 2020-08-07T15:12:44.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:44.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227912+0000", 2020-08-07T15:12:44.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00095502399999999996 2020-08-07T15:12:44.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:44.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228867+0000", 2020-08-07T15:12:44.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2020-08-07T15:12:44.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:44.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228867+0000", 2020-08-07T15:12:44.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018933599999999999 2020-08-07T15:12:44.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:44.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229057+0000", 2020-08-07T15:12:44.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0018015629999999 2020-08-07T15:12:44.330 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.330 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:44.330 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.331 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73086 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:12:44.331 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:12:44.331 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 196.25131057900001, 2020-08-07T15:12:44.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002364923, 2020-08-07T15:12:44.332 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:44.332 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:44.332 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:44.332 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:44.333 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:44.333 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73086 2020-08-07T15:12:44.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.333 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:44.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:44.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:12:44.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:44.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:44.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:12:44.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000238426 2020-08-07T15:12:44.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:44.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227626+0000", 2020-08-07T15:12:44.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9609999999999999e-06 2020-08-07T15:12:44.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:44.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227633+0000", 2020-08-07T15:12:44.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2727e-05 2020-08-07T15:12:44.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:44.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227646+0000", 2020-08-07T15:12:44.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1739999999999997e-06 2020-08-07T15:12:44.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:44.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227652+0000", 2020-08-07T15:12:44.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5004000000000001e-05 2020-08-07T15:12:44.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227737+0000", 2020-08-07T15:12:44.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2744e-05 2020-08-07T15:12:44.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:44.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227750+0000", 2020-08-07T15:12:44.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98938776500000003 2020-08-07T15:12:44.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217138+0000", 2020-08-07T15:12:44.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012361499999999999 2020-08-07T15:12:44.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:44.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217261+0000", 2020-08-07T15:12:44.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000212504 2020-08-07T15:12:44.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:12:44.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217474+0000", 2020-08-07T15:12:44.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002995036 2020-08-07T15:12:44.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:44.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.220469+0000", 2020-08-07T15:12:44.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071283249999999996 2020-08-07T15:12:44.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:44.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227597+0000", 2020-08-07T15:12:44.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0499999999999999e-07 2020-08-07T15:12:44.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:44.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227597+0000", 2020-08-07T15:12:44.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1872999999999997e-05 2020-08-07T15:12:44.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:44.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227639+0000", 2020-08-07T15:12:44.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032010200000000001 2020-08-07T15:12:44.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:44.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227959+0000", 2020-08-07T15:12:44.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015980899999999999 2020-08-07T15:12:44.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:44.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229557+0000", 2020-08-07T15:12:44.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9299999999999999e-07 2020-08-07T15:12:44.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:44.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229558+0000", 2020-08-07T15:12:44.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019477900000000001 2020-08-07T15:12:44.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:44.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229753+0000", 2020-08-07T15:12:44.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002364923 2020-08-07T15:12:44.356 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.356 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:44.356 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.357 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73087 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:12:44.358 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:12:44.359 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 196.25036942599999, 2020-08-07T15:12:44.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001896506, 2020-08-07T15:12:44.359 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:44.359 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:44.360 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:44.360 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:44.360 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:44.360 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73087 2020-08-07T15:12:44.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.361 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:44.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:44.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:12:44.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:44.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:44.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:12:44.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7354e-05 2020-08-07T15:12:44.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:44.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228386+0000", 2020-08-07T15:12:44.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8000000000000001e-06 2020-08-07T15:12:44.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:44.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228393+0000", 2020-08-07T15:12:44.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3784999999999999e-05 2020-08-07T15:12:44.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:44.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228407+0000", 2020-08-07T15:12:44.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8090000000000004e-06 2020-08-07T15:12:44.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:44.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228414+0000", 2020-08-07T15:12:44.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6907000000000003e-05 2020-08-07T15:12:44.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228461+0000", 2020-08-07T15:12:44.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8929999999999994e-06 2020-08-07T15:12:44.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:44.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228469+0000", 2020-08-07T15:12:44.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98950057700000005 2020-08-07T15:12:44.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217970+0000", 2020-08-07T15:12:44.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000132182 2020-08-07T15:12:44.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:44.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218102+0000", 2020-08-07T15:12:44.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020073399999999999 2020-08-07T15:12:44.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:12:44.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218303+0000", 2020-08-07T15:12:44.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035951120000000001 2020-08-07T15:12:44.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:44.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.221898+0000", 2020-08-07T15:12:44.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057898660000000003 2020-08-07T15:12:44.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:44.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227688+0000", 2020-08-07T15:12:44.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0200000000000002e-07 2020-08-07T15:12:44.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:44.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227688+0000", 2020-08-07T15:12:44.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1477999999999996e-05 2020-08-07T15:12:44.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:44.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227760+0000", 2020-08-07T15:12:44.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016657480000000001 2020-08-07T15:12:44.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:44.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229426+0000", 2020-08-07T15:12:44.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061290400000000005 2020-08-07T15:12:44.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:44.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230039+0000", 2020-08-07T15:12:44.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2e-07 2020-08-07T15:12:44.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:44.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230039+0000", 2020-08-07T15:12:44.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018643500000000001 2020-08-07T15:12:44.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:44.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230225+0000", 2020-08-07T15:12:44.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001896506 2020-08-07T15:12:44.385 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.385 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:44.385 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.386 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73088 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:12:44.387 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:12:44.387 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 196.249978736, 2020-08-07T15:12:44.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001627153, 2020-08-07T15:12:44.387 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:44.388 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:44.388 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:44.388 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:44.389 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:44.389 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73088 2020-08-07T15:12:44.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.390 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:44.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:44.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:12:44.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:44.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:44.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:12:44.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7741000000000003e-05 2020-08-07T15:12:44.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:44.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228767+0000", 2020-08-07T15:12:44.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6409999999999999e-06 2020-08-07T15:12:44.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:44.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228773+0000", 2020-08-07T15:12:44.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0874e-05 2020-08-07T15:12:44.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:44.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228784+0000", 2020-08-07T15:12:44.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1529999999999996e-06 2020-08-07T15:12:44.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:44.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228789+0000", 2020-08-07T15:12:44.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3945e-05 2020-08-07T15:12:44.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228833+0000", 2020-08-07T15:12:44.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4020000000000002e-06 2020-08-07T15:12:44.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:44.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228841+0000", 2020-08-07T15:12:44.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98996913900000005 2020-08-07T15:12:44.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218810+0000", 2020-08-07T15:12:44.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112484 2020-08-07T15:12:44.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:44.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218923+0000", 2020-08-07T15:12:44.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019891 2020-08-07T15:12:44.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:12:44.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.219122+0000", 2020-08-07T15:12:44.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027884099999999998 2020-08-07T15:12:44.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:44.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.221910+0000", 2020-08-07T15:12:44.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058753399999999997 2020-08-07T15:12:44.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:44.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227786+0000", 2020-08-07T15:12:44.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9200000000000002e-07 2020-08-07T15:12:44.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:44.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227786+0000", 2020-08-07T15:12:44.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8572000000000003e-05 2020-08-07T15:12:44.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:44.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227824+0000", 2020-08-07T15:12:44.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001651296 2020-08-07T15:12:44.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:44.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229476+0000", 2020-08-07T15:12:44.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00077620399999999996 2020-08-07T15:12:44.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:44.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230252+0000", 2020-08-07T15:12:44.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1600000000000002e-07 2020-08-07T15:12:44.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:44.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230252+0000", 2020-08-07T15:12:44.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4433999999999997e-05 2020-08-07T15:12:44.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:44.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230347+0000", 2020-08-07T15:12:44.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001627153 2020-08-07T15:12:44.423 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.423 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:44.424 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.426 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:128738 4.9 4.370c6199 (undecoded) ondisk+write+known_if_redirected e155)", 2020-08-07T15:12:44.428 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:03.204387+0000", 2020-08-07T15:12:44.428 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 160.27431108299999, 2020-08-07T15:12:44.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99077720899999999, 2020-08-07T15:12:44.430 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:44.430 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:44.430 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:44.430 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:44.431 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:44.431 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128738 2020-08-07T15:12:44.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.433 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:44.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:44.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204387+0000", 2020-08-07T15:12:44.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:44.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:44.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204387+0000", 2020-08-07T15:12:44.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4561999999999999e-05 2020-08-07T15:12:44.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:44.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204422+0000", 2020-08-07T15:12:44.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7270000000000001e-06 2020-08-07T15:12:44.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:44.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204428+0000", 2020-08-07T15:12:44.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3451e-05 2020-08-07T15:12:44.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:44.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204442+0000", 2020-08-07T15:12:44.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1589999999999998e-06 2020-08-07T15:12:44.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:44.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204448+0000", 2020-08-07T15:12:44.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3755999999999999e-05 2020-08-07T15:12:44.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204492+0000", 2020-08-07T15:12:44.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2450000000000001e-06 2020-08-07T15:12:44.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:44.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204499+0000", 2020-08-07T15:12:44.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99065224600000001 2020-08-07T15:12:44.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195151+0000", 2020-08-07T15:12:44.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3063e-05 2020-08-07T15:12:44.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:44.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195164+0000", 2020-08-07T15:12:44.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99077720899999999 2020-08-07T15:12:44.463 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.463 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:44.465 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.466 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:128742 4.9 4.69b70ab9 (undecoded) ondisk+write+known_if_redirected e155)", 2020-08-07T15:12:44.466 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:03.205224+0000", 2020-08-07T15:12:44.466 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 160.27347444899999, 2020-08-07T15:12:44.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.989964548, 2020-08-07T15:12:44.467 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:44.467 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:44.467 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:44.468 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:44.468 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:44.469 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128742 2020-08-07T15:12:44.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.469 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:44.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:44.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205224+0000", 2020-08-07T15:12:44.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:44.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:44.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205224+0000", 2020-08-07T15:12:44.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1018e-05 2020-08-07T15:12:44.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:44.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205255+0000", 2020-08-07T15:12:44.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5840000000000002e-06 2020-08-07T15:12:44.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:44.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205260+0000", 2020-08-07T15:12:44.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2388999999999999e-05 2020-08-07T15:12:44.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:44.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205273+0000", 2020-08-07T15:12:44.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3799999999999999e-06 2020-08-07T15:12:44.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:44.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205279+0000", 2020-08-07T15:12:44.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6684999999999998e-05 2020-08-07T15:12:44.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205326+0000", 2020-08-07T15:12:44.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6200000000000004e-06 2020-08-07T15:12:44.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:44.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205332+0000", 2020-08-07T15:12:44.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98984944600000002 2020-08-07T15:12:44.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195181+0000", 2020-08-07T15:12:44.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4259999999999997e-06 2020-08-07T15:12:44.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:44.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195188+0000", 2020-08-07T15:12:44.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.989964548 2020-08-07T15:12:44.489 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.489 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:44.489 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.490 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:128743 4.9 4.1824cba9 (undecoded) ondisk+write+known_if_redirected e155)", 2020-08-07T15:12:44.490 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:03.205500+0000", 2020-08-07T15:12:44.490 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 160.273198587, 2020-08-07T15:12:44.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98971761999999996, 2020-08-07T15:12:44.491 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:44.491 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:44.491 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:44.491 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:44.491 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:44.492 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128743 2020-08-07T15:12:44.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.492 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:44.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:44.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205500+0000", 2020-08-07T15:12:44.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:44.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:44.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205500+0000", 2020-08-07T15:12:44.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6441000000000001e-05 2020-08-07T15:12:44.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:44.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205516+0000", 2020-08-07T15:12:44.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4429999999999999e-06 2020-08-07T15:12:44.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:44.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205520+0000", 2020-08-07T15:12:44.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4590000000000003e-06 2020-08-07T15:12:44.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205526+0000", 2020-08-07T15:12:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0429999999999999e-06 2020-08-07T15:12:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205529+0000", 2020-08-07T15:12:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1313000000000003e-05 2020-08-07T15:12:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205560+0000", 2020-08-07T15:12:44.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.934e-06 2020-08-07T15:12:44.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:44.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205567+0000", 2020-08-07T15:12:44.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98964017599999998 2020-08-07T15:12:44.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:44.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195208+0000", 2020-08-07T15:12:44.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8109999999999997e-06 2020-08-07T15:12:44.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:44.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:44.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:44.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195217+0000", 2020-08-07T15:12:44.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98971761999999996 2020-08-07T15:12:44.517 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.517 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:44.517 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.517 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:44.518 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:44.518 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:44.519 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:12:44.519 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:43.646+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:44.522 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:12:44.522 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:12:44.522 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:12:44.523 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:12:44.523 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:12:44.523 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:12:44.523 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:12:44.523 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:12:44.524 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:12:44.524 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:12:44.529 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:12:44.531 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":450,"stamp":"2020-08-07T15:12:43.230053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":369287168,"num_objects":5637,"num_object_clones":0,"num_object_copies":16911,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45079,"num_write_kb":360632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":196,"ondisk_log_size":196,"up":195,"acting":195,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":198,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9477968,"kb_used_data":1089296,"kb_used_omap":2,"kb_used_meta":8388605,"kb_avail":745496752,"statfs":{"total":773094113280,"available":763388674048,"internally_reserved":8589934592,"allocated":1115439104,"data_stored":1105561820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":8589932013},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":65462272,"num_objects":999,"num_object_clones":0,"num_object_copies":2997,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7991,"num_write_kb":63928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.804943"},"pg_stats":[{"pgid":"1.28","version":"16'2","reported_seq":"18","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.433782+0000","last_change":"2020-08-07T15:12:37.495634+0000","last_active":"2020-08-07T15:12:40.433782+0000","last_peered":"2020-08-07T15:12:40.433782+0000","last_clean":"2020-08-07T15:12:40.433782+0000","last_became_active":"2020-08-07T15:12:37.495085+0000","last_became_peered":"2020-08-07T15:12:37.495085+0000","last_unstale":"2020-08-07T15:12:40.433782+0000","last_undegraded":"2020-08-07T15:12:40.433782+0000","last_fullsized":"2020-08-07T15:12:40.433782+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.490655+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.490655+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.27","version":"16'2","reported_seq":"364","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.461533+0000","last_change":"2020-08-07T15:12:37.412480+0000","last_active":"2020-08-07T15:12:41.461533+0000","last_peered":"2020-08-07T15:12:41.461533+0000","last_clean":"2020-08-07T15:12:41.461533+0000","last_became_active":"2020-08-07T15:12:37.411869+0000","last_became_peered":"2020-08-07T15:12:37.411869+0000","last_unstale":"2020-08-07T15:12:41.461533+0000","last_undegraded":"2020-08-07T15:12:41.461533+0000","last_fullsized":"2020-08-07T15:12:41.461533+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:15.163639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:15.163639+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.26","version":"16'2","reported_seq":"23","reported_epoch":"251","state":"active+clean","last_fresh":"2020-08-07T15:12:38.422097+0000","last_change":"2020-08-07T15:12:37.458242+0000","last_active":"2020-08-07T15:12:38.422097+0000","last_peered":"2020-08-07T15:12:38.422097+0000","last_clean":"2020-08-07T15:12:38.422097+0000","last_became_active":"2020-08-07T15:12:37.457426+0000","last_became_peered":"2020-08-07T15:12:37.457426+0000","last_unstale":"2020-08-07T15:12:38.422097+0000","last_undegraded":"2020-08-07T15:12:38.422097+0000","last_fullsized":"2020-08-07T15:12:38.422097+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_clean_scrub_stamp":"2020-08-07T15:12:14.121397+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.25","version":"75'3","reported_seq":"251","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.441396+0000","last_change":"2020-08-07T15:12:37.428239+0000","last_active":"2020-08-07T15:12:40.441396+0000","last_peered":"2020-08-07T15:12:40.441396+0000","last_clean":"2020-08-07T15:12:40.441396+0000","last_became_active":"2020-08-07T15:12:37.427816+0000","last_became_peered":"2020-08-07T15:12:37.427816+0000","last_unstale":"2020-08-07T15:12:40.441396+0000","last_undegraded":"2020-08-07T15:12:40.441396+0000","last_fullsized":"2020-08-07T15:12:40.441396+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:12.545945+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.545945+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.24","version":"16'2","reported_seq":"189","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.439772+0000","last_change":"2020-08-07T15:12:34.415315+0000","last_active":"2020-08-07T15:12:40.439772+0000","last_peered":"2020-08-07T15:12:40.439772+0000","last_clean":"2020-08-07T15:12:40.439772+0000","last_became_active":"2020-08-07T15:12:34.414144+0000","last_became_peered":"2020-08-07T15:12:34.414144+0000","last_unstale":"2020-08-07T15:12:40.439772+0000","last_undegraded":"2020-08-07T15:12:40.439772+0000","last_fullsized":"2020-08-07T15:12:40.439772+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:10.265395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:10.265395+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.23","version":"16'2","reported_seq":"299","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.461359+0000","last_change":"2020-08-07T15:12:37.410705+0000","last_active":"2020-08-07T15:12:41.461359+0000","last_peered":"2020-08-07T15:12:41.461359+0000","last_clean":"2020-08-07T15:12:41.461359+0000","last_became_active":"2020-08-07T15:12:37.410157+0000","last_became_peered":"2020-08-07T15:12:37.410157+0000","last_unstale":"2020-08-07T15:12:41.461359+0000","last_undegraded":"2020-08-07T15:12:41.461359+0000","last_fullsized":"2020-08-07T15:12:41.461359+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:10.175739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:10.175739+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1],"acting":[6,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.22","version":"16'2","reported_seq":"19","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.461584+0000","last_change":"2020-08-07T15:12:37.413542+0000","last_active":"2020-08-07T15:12:41.461584+0000","last_peered":"2020-08-07T15:12:41.461584+0000","last_clean":"2020-08-07T15:12:41.461584+0000","last_became_active":"2020-08-07T15:12:37.413046+0000","last_became_peered":"2020-08-07T15:12:37.413046+0000","last_unstale":"2020-08-07T15:12:41.461584+0000","last_undegraded":"2020-08-07T15:12:41.461584+0000","last_fullsized":"2020-08-07T15:12:41.461584+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:11.658923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:11.658923+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.21","version":"16'2","reported_seq":"25","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.439010+0000","last_change":"2020-08-07T15:12:37.419228+0000","last_active":"2020-08-07T15:12:40.439010+0000","last_peered":"2020-08-07T15:12:40.439010+0000","last_clean":"2020-08-07T15:12:40.439010+0000","last_became_active":"2020-08-07T15:12:37.418612+0000","last_became_peered":"2020-08-07T15:12:37.418612+0000","last_unstale":"2020-08-07T15:12:40.439010+0000","last_undegraded":"2020-08-07T15:12:40.439010+0000","last_fullsized":"2020-08-07T15:12:40.439010+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:09.261101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.261101+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.20","version":"16'2","reported_seq":"25","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.440988+0000","last_change":"2020-08-07T15:12:37.409857+0000","last_active":"2020-08-07T15:12:40.440988+0000","last_peered":"2020-08-07T15:12:40.440988+0000","last_clean":"2020-08-07T15:12:40.440988+0000","last_became_active":"2020-08-07T15:12:37.409249+0000","last_became_peered":"2020-08-07T15:12:37.409249+0000","last_unstale":"2020-08-07T15:12:40.440988+0000","last_undegraded":"2020-08-07T15:12:40.440988+0000","last_fullsized":"2020-08-07T15:12:40.440988+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:40.696862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:40.696862+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1f","version":"16'2","reported_seq":"18","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.444823+0000","last_change":"2020-08-07T15:12:37.411233+0000","last_active":"2020-08-07T15:12:40.444823+0000","last_peered":"2020-08-07T15:12:40.444823+0000","last_clean":"2020-08-07T15:12:40.444823+0000","last_became_active":"2020-08-07T15:12:37.410716+0000","last_became_peered":"2020-08-07T15:12:37.410716+0000","last_unstale":"2020-08-07T15:12:40.444823+0000","last_undegraded":"2020-08-07T15:12:40.444823+0000","last_fullsized":"2020-08-07T15:12:40.444823+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.176383+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1e","version":"16'2","reported_seq":"105","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.439485+0000","last_change":"2020-08-07T15:11:37.457186+0000","last_active":"2020-08-07T15:12:40.439485+0000","last_peered":"2020-08-07T15:12:40.439485+0000","last_clean":"2020-08-07T15:12:40.439485+0000","last_became_active":"2020-08-07T15:10:51.559894+0000","last_became_peered":"2020-08-07T15:10:51.559894+0000","last_unstale":"2020-08-07T15:12:40.439485+0000","last_undegraded":"2020-08-07T15:12:40.439485+0000","last_fullsized":"2020-08-07T15:12:40.439485+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.457111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.457111+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1d","version":"75'3","reported_seq":"116","reported_epoch":"251","state":"active+clean","last_fresh":"2020-08-07T15:12:38.422054+0000","last_change":"2020-08-07T15:12:28.855487+0000","last_active":"2020-08-07T15:12:38.422054+0000","last_peered":"2020-08-07T15:12:38.422054+0000","last_clean":"2020-08-07T15:12:38.422054+0000","last_became_active":"2020-08-07T15:12:07.356691+0000","last_became_peered":"2020-08-07T15:12:07.356691+0000","last_unstale":"2020-08-07T15:12:38.422054+0000","last_undegraded":"2020-08-07T15:12:38.422054+0000","last_fullsized":"2020-08-07T15:12:38.422054+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:28.855432+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:28.855432+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1c","version":"16'2","reported_seq":"193","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.435626+0000","last_change":"2020-08-07T15:12:07.262797+0000","last_active":"2020-08-07T15:12:40.435626+0000","last_peered":"2020-08-07T15:12:40.435626+0000","last_clean":"2020-08-07T15:12:40.435626+0000","last_became_active":"2020-08-07T15:12:07.245731+0000","last_became_peered":"2020-08-07T15:12:07.245731+0000","last_unstale":"2020-08-07T15:12:40.435626+0000","last_undegraded":"2020-08-07T15:12:40.435626+0000","last_fullsized":"2020-08-07T15:12:40.435626+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:39.738160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:39.738160+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1b","version":"16'2","reported_seq":"226","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.433916+0000","last_change":"2020-08-07T15:10:57.797029+0000","last_active":"2020-08-07T15:12:40.433916+0000","last_peered":"2020-08-07T15:12:40.433916+0000","last_clean":"2020-08-07T15:12:40.433916+0000","last_became_active":"2020-08-07T15:10:51.557392+0000","last_became_peered":"2020-08-07T15:10:51.557392+0000","last_unstale":"2020-08-07T15:12:40.433916+0000","last_undegraded":"2020-08-07T15:12:40.433916+0000","last_fullsized":"2020-08-07T15:12:40.433916+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:57.796946+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:57.796946+0000","last_clean_scrub_stamp":"2020-08-07T15:10:57.796946+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1a","version":"16'2","reported_seq":"93","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.834667+0000","last_change":"2020-08-07T15:12:40.834667+0000","last_active":"2020-08-07T15:12:40.834667+0000","last_peered":"2020-08-07T15:12:40.834667+0000","last_clean":"2020-08-07T15:12:40.834667+0000","last_became_active":"2020-08-07T15:12:07.243542+0000","last_became_peered":"2020-08-07T15:12:07.243542+0000","last_unstale":"2020-08-07T15:12:40.834667+0000","last_undegraded":"2020-08-07T15:12:40.834667+0000","last_fullsized":"2020-08-07T15:12:40.834667+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:40.834612+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:12:40.834612+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.19","version":"16'2","reported_seq":"151","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.439121+0000","last_change":"2020-08-07T15:12:15.372999+0000","last_active":"2020-08-07T15:12:40.439121+0000","last_peered":"2020-08-07T15:12:40.439121+0000","last_clean":"2020-08-07T15:12:40.439121+0000","last_became_active":"2020-08-07T15:10:51.559752+0000","last_became_peered":"2020-08-07T15:10:51.559752+0000","last_unstale":"2020-08-07T15:12:40.439121+0000","last_undegraded":"2020-08-07T15:12:40.439121+0000","last_fullsized":"2020-08-07T15:12:40.439121+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:15.372910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:15.372910+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.18","version":"16'2","reported_seq":"251","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.433782+0000","last_change":"2020-08-07T15:11:06.955336+0000","last_active":"2020-08-07T15:12:40.433782+0000","last_peered":"2020-08-07T15:12:40.433782+0000","last_clean":"2020-08-07T15:12:40.433782+0000","last_became_active":"2020-08-07T15:10:51.557665+0000","last_became_peered":"2020-08-07T15:10:51.557665+0000","last_unstale":"2020-08-07T15:12:40.433782+0000","last_undegraded":"2020-08-07T15:12:40.433782+0000","last_fullsized":"2020-08-07T15:12:40.433782+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:06.955262+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:06.955262+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.17","version":"16'2","reported_seq":"79","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.434011+0000","last_change":"2020-08-07T15:11:08.916219+0000","last_active":"2020-08-07T15:12:40.434011+0000","last_peered":"2020-08-07T15:12:40.434011+0000","last_clean":"2020-08-07T15:12:40.434011+0000","last_became_active":"2020-08-07T15:10:51.561769+0000","last_became_peered":"2020-08-07T15:10:51.561769+0000","last_unstale":"2020-08-07T15:12:40.434011+0000","last_undegraded":"2020-08-07T15:12:40.434011+0000","last_fullsized":"2020-08-07T15:12:40.434011+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:08.916169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:08.916169+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.16","version":"16'2","reported_seq":"54","reported_epoch":"251","state":"active+clean","last_fresh":"2020-08-07T15:12:38.422194+0000","last_change":"2020-08-07T15:12:12.883619+0000","last_active":"2020-08-07T15:12:38.422194+0000","last_peered":"2020-08-07T15:12:38.422194+0000","last_clean":"2020-08-07T15:12:38.422194+0000","last_became_active":"2020-08-07T15:12:07.231877+0000","last_became_peered":"2020-08-07T15:12:07.231877+0000","last_unstale":"2020-08-07T15:12:38.422194+0000","last_undegraded":"2020-08-07T15:12:38.422194+0000","last_fullsized":"2020-08-07T15:12:38.422194+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:12.883558+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:12.883558+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.883558+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.15","version":"75'3","reported_seq":"90","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.435699+0000","last_change":"2020-08-07T15:12:07.284341+0000","last_active":"2020-08-07T15:12:40.435699+0000","last_peered":"2020-08-07T15:12:40.435699+0000","last_clean":"2020-08-07T15:12:40.435699+0000","last_became_active":"2020-08-07T15:12:07.276899+0000","last_became_peered":"2020-08-07T15:12:07.276899+0000","last_unstale":"2020-08-07T15:12:40.435699+0000","last_undegraded":"2020-08-07T15:12:40.435699+0000","last_fullsized":"2020-08-07T15:12:40.435699+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:11:20.562710+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:11:20.562710+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.14","version":"16'2","reported_seq":"189","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.441265+0000","last_change":"2020-08-07T15:12:09.224231+0000","last_active":"2020-08-07T15:12:40.441265+0000","last_peered":"2020-08-07T15:12:40.441265+0000","last_clean":"2020-08-07T15:12:40.441265+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:12:40.441265+0000","last_undegraded":"2020-08-07T15:12:40.441265+0000","last_fullsized":"2020-08-07T15:12:40.441265+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:09.224158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.224158+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"202","reported_epoch":"251","state":"active+clean","last_fresh":"2020-08-07T15:12:38.421837+0000","last_change":"2020-08-07T15:12:19.577926+0000","last_active":"2020-08-07T15:12:38.421837+0000","last_peered":"2020-08-07T15:12:38.421837+0000","last_clean":"2020-08-07T15:12:38.421837+0000","last_became_active":"2020-08-07T15:12:06.375861+0000","last_became_peered":"2020-08-07T15:12:06.375861+0000","last_unstale":"2020-08-07T15:12:38.421837+0000","last_undegraded":"2020-08-07T15:12:38.421837+0000","last_fullsized":"2020-08-07T15:12:38.421837+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:07.887356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.887356+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"204","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.434178+0000","last_change":"2020-08-07T15:12:26.104598+0000","last_active":"2020-08-07T15:12:40.434178+0000","last_peered":"2020-08-07T15:12:40.434178+0000","last_clean":"2020-08-07T15:12:40.434178+0000","last_became_active":"2020-08-07T15:12:06.367362+0000","last_became_peered":"2020-08-07T15:12:06.367362+0000","last_unstale":"2020-08-07T15:12:40.434178+0000","last_undegraded":"2020-08-07T15:12:40.434178+0000","last_fullsized":"2020-08-07T15:12:40.434178+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:26.104526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:26.104526+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"302","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.433819+0000","last_change":"2020-08-07T15:12:24.105593+0000","last_active":"2020-08-07T15:12:40.433819+0000","last_peered":"2020-08-07T15:12:40.433819+0000","last_clean":"2020-08-07T15:12:40.433819+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:12:40.433819+0000","last_undegraded":"2020-08-07T15:12:40.433819+0000","last_fullsized":"2020-08-07T15:12:40.433819+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:24.105531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:24.105531+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"296","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.433911+0000","last_change":"2020-08-07T15:12:13.231819+0000","last_active":"2020-08-07T15:12:40.433911+0000","last_peered":"2020-08-07T15:12:40.433911+0000","last_clean":"2020-08-07T15:12:40.433911+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:12:40.433911+0000","last_undegraded":"2020-08-07T15:12:40.433911+0000","last_fullsized":"2020-08-07T15:12:40.433911+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:13.231762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.231762+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3","version":"254'2584","reported_seq":"2616","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.677041+0000","last_change":"2020-08-07T15:12:10.464875+0000","last_active":"2020-08-07T15:12:41.677041+0000","last_peered":"2020-08-07T15:12:41.677041+0000","last_clean":"2020-08-07T15:12:41.677041+0000","last_became_active":"2020-08-07T15:12:10.464346+0000","last_became_peered":"2020-08-07T15:12:10.464346+0000","last_unstale":"2020-08-07T15:12:41.677041+0000","last_undegraded":"2020-08-07T15:12:41.677041+0000","last_fullsized":"2020-08-07T15:12:41.677041+0000","mapping_epoch":230,"log_start":"254'2579","ondisk_log_start":"254'2579","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21168128,"num_objects":323,"num_object_clones":0,"num_object_copies":969,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2584,"num_write_kb":20672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"257","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.461712+0000","last_change":"2020-08-07T15:12:27.410339+0000","last_active":"2020-08-07T15:12:41.461712+0000","last_peered":"2020-08-07T15:12:41.461712+0000","last_clean":"2020-08-07T15:12:41.461712+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:12:41.461712+0000","last_undegraded":"2020-08-07T15:12:41.461712+0000","last_fullsized":"2020-08-07T15:12:41.461712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:27.410262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:27.410262+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"184","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.439861+0000","last_change":"2020-08-07T15:12:39.258928+0000","last_active":"2020-08-07T15:12:40.439861+0000","last_peered":"2020-08-07T15:12:40.439861+0000","last_clean":"2020-08-07T15:12:40.439861+0000","last_became_active":"2020-08-07T15:12:32.259956+0000","last_became_peered":"2020-08-07T15:12:32.259956+0000","last_unstale":"2020-08-07T15:12:40.439861+0000","last_undegraded":"2020-08-07T15:12:40.439861+0000","last_fullsized":"2020-08-07T15:12:40.439861+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:39.258857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:39.258857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"254'2904","reported_seq":"2936","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.737320+0000","last_change":"2020-08-07T15:12:10.465048+0000","last_active":"2020-08-07T15:12:41.737320+0000","last_peered":"2020-08-07T15:12:41.737320+0000","last_clean":"2020-08-07T15:12:41.737320+0000","last_became_active":"2020-08-07T15:12:10.464493+0000","last_became_peered":"2020-08-07T15:12:10.464493+0000","last_unstale":"2020-08-07T15:12:41.737320+0000","last_undegraded":"2020-08-07T15:12:41.737320+0000","last_fullsized":"2020-08-07T15:12:41.737320+0000","mapping_epoch":230,"log_start":"254'2896","ondisk_log_start":"254'2896","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23789568,"num_objects":363,"num_object_clones":0,"num_object_copies":1089,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2904,"num_write_kb":23232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"210","reported_epoch":"251","state":"active+clean","last_fresh":"2020-08-07T15:12:38.422187+0000","last_change":"2020-08-07T15:12:19.561477+0000","last_active":"2020-08-07T15:12:38.422187+0000","last_peered":"2020-08-07T15:12:38.422187+0000","last_clean":"2020-08-07T15:12:38.422187+0000","last_became_active":"2020-08-07T15:12:06.367376+0000","last_became_peered":"2020-08-07T15:12:06.367376+0000","last_unstale":"2020-08-07T15:12:38.422187+0000","last_undegraded":"2020-08-07T15:12:38.422187+0000","last_fullsized":"2020-08-07T15:12:38.422187+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:12.319383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.319383+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"235","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.441511+0000","last_change":"2020-08-07T15:12:32.241624+0000","last_active":"2020-08-07T15:12:40.441511+0000","last_peered":"2020-08-07T15:12:40.441511+0000","last_clean":"2020-08-07T15:12:40.441511+0000","last_became_active":"2020-08-07T15:12:32.241146+0000","last_became_peered":"2020-08-07T15:12:32.241146+0000","last_unstale":"2020-08-07T15:12:40.441511+0000","last_undegraded":"2020-08-07T15:12:40.441511+0000","last_fullsized":"2020-08-07T15:12:40.441511+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:12.545945+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.545945+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"252'2631","reported_seq":"2657","reported_epoch":"252","state":"active+clean","last_fresh":"2020-08-07T15:12:39.859177+0000","last_change":"2020-08-07T15:12:10.464412+0000","last_active":"2020-08-07T15:12:39.859177+0000","last_peered":"2020-08-07T15:12:39.859177+0000","last_clean":"2020-08-07T15:12:39.859177+0000","last_became_active":"2020-08-07T15:12:10.463950+0000","last_became_peered":"2020-08-07T15:12:10.463950+0000","last_unstale":"2020-08-07T15:12:39.859177+0000","last_undegraded":"2020-08-07T15:12:39.859177+0000","last_fullsized":"2020-08-07T15:12:39.859177+0000","mapping_epoch":230,"log_start":"252'2625","ondisk_log_start":"252'2625","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21553152,"num_objects":329,"num_object_clones":0,"num_object_copies":987,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2631,"num_write_kb":21048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"254","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.435855+0000","last_change":"2020-08-07T15:11:55.731391+0000","last_active":"2020-08-07T15:12:40.435855+0000","last_peered":"2020-08-07T15:12:40.435855+0000","last_clean":"2020-08-07T15:12:40.435855+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:12:40.435855+0000","last_undegraded":"2020-08-07T15:12:40.435855+0000","last_fullsized":"2020-08-07T15:12:40.435855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:11:55.731304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:11:55.731304+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"314","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.461587+0000","last_change":"2020-08-07T15:12:32.240238+0000","last_active":"2020-08-07T15:12:41.461587+0000","last_peered":"2020-08-07T15:12:41.461587+0000","last_clean":"2020-08-07T15:12:41.461587+0000","last_became_active":"2020-08-07T15:12:32.239755+0000","last_became_peered":"2020-08-07T15:12:32.239755+0000","last_unstale":"2020-08-07T15:12:41.461587+0000","last_undegraded":"2020-08-07T15:12:41.461587+0000","last_fullsized":"2020-08-07T15:12:41.461587+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_clean_scrub_stamp":"2020-08-07T15:12:14.121397+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"252'2760","reported_seq":"2790","reported_epoch":"252","state":"active+clean","last_fresh":"2020-08-07T15:12:39.838300+0000","last_change":"2020-08-07T15:12:10.398415+0000","last_active":"2020-08-07T15:12:39.838300+0000","last_peered":"2020-08-07T15:12:39.838300+0000","last_clean":"2020-08-07T15:12:39.838300+0000","last_became_active":"2020-08-07T15:12:10.397834+0000","last_became_peered":"2020-08-07T15:12:10.397834+0000","last_unstale":"2020-08-07T15:12:39.838300+0000","last_undegraded":"2020-08-07T15:12:39.838300+0000","last_fullsized":"2020-08-07T15:12:39.838300+0000","mapping_epoch":230,"log_start":"252'2752","ondisk_log_start":"252'2752","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22609920,"num_objects":345,"num_object_clones":0,"num_object_copies":1035,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2760,"num_write_kb":22080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"269","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.435427+0000","last_change":"2020-08-07T15:12:08.622779+0000","last_active":"2020-08-07T15:12:40.435427+0000","last_peered":"2020-08-07T15:12:40.435427+0000","last_clean":"2020-08-07T15:12:40.435427+0000","last_became_active":"2020-08-07T15:12:06.367457+0000","last_became_peered":"2020-08-07T15:12:06.367457+0000","last_unstale":"2020-08-07T15:12:40.435427+0000","last_undegraded":"2020-08-07T15:12:40.435427+0000","last_fullsized":"2020-08-07T15:12:40.435427+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:08.622683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:08.622683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"286","reported_epoch":"255","state":"active+clean","last_fresh":"2020-08-07T15:12:42.371230+0000","last_change":"2020-08-07T15:12:42.371309+0000","last_active":"2020-08-07T15:12:42.371230+0000","last_peered":"2020-08-07T15:12:42.371230+0000","last_clean":"2020-08-07T15:12:42.371230+0000","last_became_active":"2020-08-07T15:12:32.237984+0000","last_became_peered":"2020-08-07T15:12:32.237984+0000","last_unstale":"2020-08-07T15:12:42.371230+0000","last_undegraded":"2020-08-07T15:12:42.371230+0000","last_fullsized":"2020-08-07T15:12:42.371230+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:42.371229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:42.371229+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.7","version":"254'2896","reported_seq":"2928","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.695741+0000","last_change":"2020-08-07T15:12:10.464810+0000","last_active":"2020-08-07T15:12:41.695741+0000","last_peered":"2020-08-07T15:12:41.695741+0000","last_clean":"2020-08-07T15:12:41.695741+0000","last_became_active":"2020-08-07T15:12:10.464050+0000","last_became_peered":"2020-08-07T15:12:10.464050+0000","last_unstale":"2020-08-07T15:12:41.695741+0000","last_undegraded":"2020-08-07T15:12:41.695741+0000","last_fullsized":"2020-08-07T15:12:41.695741+0000","mapping_epoch":230,"log_start":"254'2888","ondisk_log_start":"254'2888","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23724032,"num_objects":362,"num_object_clones":0,"num_object_copies":1086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2896,"num_write_kb":23168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"298","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.435410+0000","last_change":"2020-08-07T15:12:39.812944+0000","last_active":"2020-08-07T15:12:40.435410+0000","last_peered":"2020-08-07T15:12:40.435410+0000","last_clean":"2020-08-07T15:12:40.435410+0000","last_became_active":"2020-08-07T15:12:32.255633+0000","last_became_peered":"2020-08-07T15:12:32.255633+0000","last_unstale":"2020-08-07T15:12:40.435410+0000","last_undegraded":"2020-08-07T15:12:40.435410+0000","last_fullsized":"2020-08-07T15:12:40.435410+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:39.812876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:39.812876+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"254","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.434109+0000","last_change":"2020-08-07T15:11:40.328534+0000","last_active":"2020-08-07T15:12:40.434109+0000","last_peered":"2020-08-07T15:12:40.434109+0000","last_clean":"2020-08-07T15:12:40.434109+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:12:40.434109+0000","last_undegraded":"2020-08-07T15:12:40.434109+0000","last_fullsized":"2020-08-07T15:12:40.434109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:11:40.328482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:11:40.328482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"253'2832","reported_seq":"2864","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.507395+0000","last_change":"2020-08-07T15:12:10.464478+0000","last_active":"2020-08-07T15:12:41.507395+0000","last_peered":"2020-08-07T15:12:41.507395+0000","last_clean":"2020-08-07T15:12:41.507395+0000","last_became_active":"2020-08-07T15:12:10.463997+0000","last_became_peered":"2020-08-07T15:12:10.463997+0000","last_unstale":"2020-08-07T15:12:41.507395+0000","last_undegraded":"2020-08-07T15:12:41.507395+0000","last_fullsized":"2020-08-07T15:12:41.507395+0000","mapping_epoch":230,"log_start":"253'2824","ondisk_log_start":"253'2824","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23199744,"num_objects":354,"num_object_clones":0,"num_object_copies":1062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2832,"num_write_kb":22656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"275","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.441399+0000","last_change":"2020-08-07T15:12:10.585667+0000","last_active":"2020-08-07T15:12:40.441399+0000","last_peered":"2020-08-07T15:12:40.441399+0000","last_clean":"2020-08-07T15:12:40.441399+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:12:40.441399+0000","last_undegraded":"2020-08-07T15:12:40.441399+0000","last_fullsized":"2020-08-07T15:12:40.441399+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:10.585580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:10.585580+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"314","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.435608+0000","last_change":"2020-08-07T15:12:38.850878+0000","last_active":"2020-08-07T15:12:40.435608+0000","last_peered":"2020-08-07T15:12:40.435608+0000","last_clean":"2020-08-07T15:12:40.435608+0000","last_became_active":"2020-08-07T15:12:32.239794+0000","last_became_peered":"2020-08-07T15:12:32.239794+0000","last_unstale":"2020-08-07T15:12:40.435608+0000","last_undegraded":"2020-08-07T15:12:40.435608+0000","last_fullsized":"2020-08-07T15:12:40.435608+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:38.850809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:38.850809+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"254'3000","reported_seq":"3031","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.676648+0000","last_change":"2020-08-07T15:12:10.465113+0000","last_active":"2020-08-07T15:12:41.676648+0000","last_peered":"2020-08-07T15:12:41.676648+0000","last_clean":"2020-08-07T15:12:41.676648+0000","last_became_active":"2020-08-07T15:12:10.464582+0000","last_became_peered":"2020-08-07T15:12:10.464582+0000","last_unstale":"2020-08-07T15:12:41.676648+0000","last_undegraded":"2020-08-07T15:12:41.676648+0000","last_fullsized":"2020-08-07T15:12:41.676648+0000","mapping_epoch":230,"log_start":"254'2989","ondisk_log_start":"254'2989","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24576000,"num_objects":375,"num_object_clones":0,"num_object_copies":1125,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3000,"num_write_kb":24000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"269","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.434223+0000","last_change":"2020-08-07T15:12:07.340958+0000","last_active":"2020-08-07T15:12:40.434223+0000","last_peered":"2020-08-07T15:12:40.434223+0000","last_clean":"2020-08-07T15:12:40.434223+0000","last_became_active":"2020-08-07T15:12:06.366967+0000","last_became_peered":"2020-08-07T15:12:06.366967+0000","last_unstale":"2020-08-07T15:12:40.434223+0000","last_undegraded":"2020-08-07T15:12:40.434223+0000","last_fullsized":"2020-08-07T15:12:40.434223+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:07.340855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.340855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"261","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.433827+0000","last_change":"2020-08-07T15:12:32.247332+0000","last_active":"2020-08-07T15:12:40.433827+0000","last_peered":"2020-08-07T15:12:40.433827+0000","last_clean":"2020-08-07T15:12:40.433827+0000","last_became_active":"2020-08-07T15:12:32.246800+0000","last_became_peered":"2020-08-07T15:12:32.246800+0000","last_unstale":"2020-08-07T15:12:40.433827+0000","last_undegraded":"2020-08-07T15:12:40.433827+0000","last_fullsized":"2020-08-07T15:12:40.433827+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:09.261101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.261101+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"254'2760","reported_seq":"2791","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.746878+0000","last_change":"2020-08-07T15:12:10.466261+0000","last_active":"2020-08-07T15:12:41.746878+0000","last_peered":"2020-08-07T15:12:41.746878+0000","last_clean":"2020-08-07T15:12:41.746878+0000","last_became_active":"2020-08-07T15:12:10.465583+0000","last_became_peered":"2020-08-07T15:12:10.465583+0000","last_unstale":"2020-08-07T15:12:41.746878+0000","last_undegraded":"2020-08-07T15:12:41.746878+0000","last_fullsized":"2020-08-07T15:12:41.746878+0000","mapping_epoch":230,"log_start":"254'2755","ondisk_log_start":"254'2755","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22609920,"num_objects":345,"num_object_clones":0,"num_object_copies":1035,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2760,"num_write_kb":22080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"266","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.435731+0000","last_change":"2020-08-07T15:12:07.583222+0000","last_active":"2020-08-07T15:12:40.435731+0000","last_peered":"2020-08-07T15:12:40.435731+0000","last_clean":"2020-08-07T15:12:40.435731+0000","last_became_active":"2020-08-07T15:12:06.366915+0000","last_became_peered":"2020-08-07T15:12:06.366915+0000","last_unstale":"2020-08-07T15:12:40.435731+0000","last_undegraded":"2020-08-07T15:12:40.435731+0000","last_fullsized":"2020-08-07T15:12:40.435731+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:07.583152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.583152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"348","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.461807+0000","last_change":"2020-08-07T15:12:32.255274+0000","last_active":"2020-08-07T15:12:41.461807+0000","last_peered":"2020-08-07T15:12:41.461807+0000","last_clean":"2020-08-07T15:12:41.461807+0000","last_became_active":"2020-08-07T15:12:32.254842+0000","last_became_peered":"2020-08-07T15:12:32.254842+0000","last_unstale":"2020-08-07T15:12:41.461807+0000","last_undegraded":"2020-08-07T15:12:41.461807+0000","last_fullsized":"2020-08-07T15:12:41.461807+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:15.163639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:15.163639+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"254'2744","reported_seq":"2773","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.746173+0000","last_change":"2020-08-07T15:12:10.467447+0000","last_active":"2020-08-07T15:12:41.746173+0000","last_peered":"2020-08-07T15:12:41.746173+0000","last_clean":"2020-08-07T15:12:41.746173+0000","last_became_active":"2020-08-07T15:12:10.467000+0000","last_became_peered":"2020-08-07T15:12:10.467000+0000","last_unstale":"2020-08-07T15:12:41.746173+0000","last_undegraded":"2020-08-07T15:12:41.746173+0000","last_fullsized":"2020-08-07T15:12:41.746173+0000","mapping_epoch":230,"log_start":"254'2737","ondisk_log_start":"254'2737","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22478848,"num_objects":343,"num_object_clones":0,"num_object_copies":1029,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2744,"num_write_kb":21952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"238","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.434008+0000","last_change":"2020-08-07T15:10:51.644951+0000","last_active":"2020-08-07T15:12:40.434008+0000","last_peered":"2020-08-07T15:12:40.434008+0000","last_clean":"2020-08-07T15:12:40.434008+0000","last_became_active":"2020-08-07T15:10:46.498230+0000","last_became_peered":"2020-08-07T15:10:46.498230+0000","last_unstale":"2020-08-07T15:12:40.434008+0000","last_undegraded":"2020-08-07T15:12:40.434008+0000","last_fullsized":"2020-08-07T15:12:40.434008+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:51.644875+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:10:51.644875+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"254'2880","reported_seq":"2912","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.647808+0000","last_change":"2020-08-07T15:12:10.465075+0000","last_active":"2020-08-07T15:12:41.647808+0000","last_peered":"2020-08-07T15:12:41.647808+0000","last_clean":"2020-08-07T15:12:41.647808+0000","last_became_active":"2020-08-07T15:12:10.464640+0000","last_became_peered":"2020-08-07T15:12:10.464640+0000","last_unstale":"2020-08-07T15:12:41.647808+0000","last_undegraded":"2020-08-07T15:12:41.647808+0000","last_fullsized":"2020-08-07T15:12:41.647808+0000","mapping_epoch":230,"log_start":"254'2873","ondisk_log_start":"254'2873","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23592960,"num_objects":360,"num_object_clones":0,"num_object_copies":1080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2880,"num_write_kb":23040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"82","reported_epoch":"248","state":"active+clean","last_fresh":"2020-08-07T15:12:35.391961+0000","last_change":"2020-08-07T15:12:32.239634+0000","last_active":"2020-08-07T15:12:35.391961+0000","last_peered":"2020-08-07T15:12:35.391961+0000","last_clean":"2020-08-07T15:12:35.391961+0000","last_became_active":"2020-08-07T15:12:32.238859+0000","last_became_peered":"2020-08-07T15:12:32.238859+0000","last_unstale":"2020-08-07T15:12:35.391961+0000","last_undegraded":"2020-08-07T15:12:35.391961+0000","last_fullsized":"2020-08-07T15:12:35.391961+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.490655+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.490655+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"252'2616","reported_seq":"2646","reported_epoch":"252","state":"active+clean","last_fresh":"2020-08-07T15:12:39.637109+0000","last_change":"2020-08-07T15:12:10.465359+0000","last_active":"2020-08-07T15:12:39.637109+0000","last_peered":"2020-08-07T15:12:39.637109+0000","last_clean":"2020-08-07T15:12:39.637109+0000","last_became_active":"2020-08-07T15:12:10.464900+0000","last_became_peered":"2020-08-07T15:12:10.464900+0000","last_unstale":"2020-08-07T15:12:39.637109+0000","last_undegraded":"2020-08-07T15:12:39.637109+0000","last_fullsized":"2020-08-07T15:12:39.637109+0000","mapping_epoch":230,"log_start":"252'2612","ondisk_log_start":"252'2612","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21430272,"num_objects":327,"num_object_clones":0,"num_object_copies":981,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2616,"num_write_kb":20928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"305","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.435384+0000","last_change":"2020-08-07T15:12:23.886345+0000","last_active":"2020-08-07T15:12:40.435384+0000","last_peered":"2020-08-07T15:12:40.435384+0000","last_clean":"2020-08-07T15:12:40.435384+0000","last_became_active":"2020-08-07T15:10:46.499635+0000","last_became_peered":"2020-08-07T15:10:46.499635+0000","last_unstale":"2020-08-07T15:12:40.435384+0000","last_undegraded":"2020-08-07T15:12:40.435384+0000","last_fullsized":"2020-08-07T15:12:40.435384+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:23.886261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:23.886261+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"254'2992","reported_seq":"3024","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.691321+0000","last_change":"2020-08-07T15:12:10.468110+0000","last_active":"2020-08-07T15:12:41.691321+0000","last_peered":"2020-08-07T15:12:41.691321+0000","last_clean":"2020-08-07T15:12:41.691321+0000","last_became_active":"2020-08-07T15:12:10.467697+0000","last_became_peered":"2020-08-07T15:12:10.467697+0000","last_unstale":"2020-08-07T15:12:41.691321+0000","last_undegraded":"2020-08-07T15:12:41.691321+0000","last_fullsized":"2020-08-07T15:12:41.691321+0000","mapping_epoch":230,"log_start":"254'2985","ondisk_log_start":"254'2985","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24510464,"num_objects":374,"num_object_clones":0,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2992,"num_write_kb":23936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"272","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.435789+0000","last_change":"2020-08-07T15:12:13.680022+0000","last_active":"2020-08-07T15:12:40.435789+0000","last_peered":"2020-08-07T15:12:40.435789+0000","last_clean":"2020-08-07T15:12:40.435789+0000","last_became_active":"2020-08-07T15:12:06.380392+0000","last_became_peered":"2020-08-07T15:12:06.380392+0000","last_unstale":"2020-08-07T15:12:40.435789+0000","last_undegraded":"2020-08-07T15:12:40.435789+0000","last_fullsized":"2020-08-07T15:12:40.435789+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:13.679943+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.679943+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"254'2888","reported_seq":"2919","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.665237+0000","last_change":"2020-08-07T15:12:10.398816+0000","last_active":"2020-08-07T15:12:41.665237+0000","last_peered":"2020-08-07T15:12:41.665237+0000","last_clean":"2020-08-07T15:12:41.665237+0000","last_became_active":"2020-08-07T15:12:10.398332+0000","last_became_peered":"2020-08-07T15:12:10.398332+0000","last_unstale":"2020-08-07T15:12:41.665237+0000","last_undegraded":"2020-08-07T15:12:41.665237+0000","last_fullsized":"2020-08-07T15:12:41.665237+0000","mapping_epoch":230,"log_start":"254'2883","ondisk_log_start":"254'2883","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23658496,"num_objects":361,"num_object_clones":0,"num_object_copies":1083,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2888,"num_write_kb":23104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"227","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.439886+0000","last_change":"2020-08-07T15:12:14.241339+0000","last_active":"2020-08-07T15:12:40.439886+0000","last_peered":"2020-08-07T15:12:40.439886+0000","last_clean":"2020-08-07T15:12:40.439886+0000","last_became_active":"2020-08-07T15:10:46.498051+0000","last_became_peered":"2020-08-07T15:10:46.498051+0000","last_unstale":"2020-08-07T15:12:40.439886+0000","last_undegraded":"2020-08-07T15:12:40.439886+0000","last_fullsized":"2020-08-07T15:12:40.439886+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:14.241262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:14.241262+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"254'2936","reported_seq":"2966","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.691717+0000","last_change":"2020-08-07T15:12:10.398015+0000","last_active":"2020-08-07T15:12:41.691717+0000","last_peered":"2020-08-07T15:12:41.691717+0000","last_clean":"2020-08-07T15:12:41.691717+0000","last_became_active":"2020-08-07T15:12:10.397610+0000","last_became_peered":"2020-08-07T15:12:10.397610+0000","last_unstale":"2020-08-07T15:12:41.691717+0000","last_undegraded":"2020-08-07T15:12:41.691717+0000","last_fullsized":"2020-08-07T15:12:41.691717+0000","mapping_epoch":230,"log_start":"254'2928","ondisk_log_start":"254'2928","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24051712,"num_objects":367,"num_object_clones":0,"num_object_copies":1101,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2936,"num_write_kb":23488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"304","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.461703+0000","last_change":"2020-08-07T15:12:41.445456+0000","last_active":"2020-08-07T15:12:41.461703+0000","last_peered":"2020-08-07T15:12:41.461703+0000","last_clean":"2020-08-07T15:12:41.461703+0000","last_became_active":"2020-08-07T15:12:32.241722+0000","last_became_peered":"2020-08-07T15:12:32.241722+0000","last_unstale":"2020-08-07T15:12:41.461703+0000","last_undegraded":"2020-08-07T15:12:41.461703+0000","last_fullsized":"2020-08-07T15:12:41.461703+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:41.445379+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_clean_scrub_stamp":"2020-08-07T15:12:41.445379+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"254'2664","reported_seq":"2696","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.618126+0000","last_change":"2020-08-07T15:12:10.468495+0000","last_active":"2020-08-07T15:12:41.618126+0000","last_peered":"2020-08-07T15:12:41.618126+0000","last_clean":"2020-08-07T15:12:41.618126+0000","last_became_active":"2020-08-07T15:12:10.467915+0000","last_became_peered":"2020-08-07T15:12:10.467915+0000","last_unstale":"2020-08-07T15:12:41.618126+0000","last_undegraded":"2020-08-07T15:12:41.618126+0000","last_fullsized":"2020-08-07T15:12:41.618126+0000","mapping_epoch":230,"log_start":"254'2656","ondisk_log_start":"254'2656","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21823488,"num_objects":333,"num_object_clones":0,"num_object_copies":999,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2664,"num_write_kb":21312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,2],"acting":[7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"191","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.435718+0000","last_change":"2020-08-07T15:12:30.942002+0000","last_active":"2020-08-07T15:12:40.435718+0000","last_peered":"2020-08-07T15:12:40.435718+0000","last_clean":"2020-08-07T15:12:40.435718+0000","last_became_active":"2020-08-07T15:10:46.493498+0000","last_became_peered":"2020-08-07T15:10:46.493498+0000","last_unstale":"2020-08-07T15:12:40.435718+0000","last_undegraded":"2020-08-07T15:12:40.435718+0000","last_fullsized":"2020-08-07T15:12:40.435718+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:30.941943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:30.941943+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"254'2992","reported_seq":"3024","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.758625+0000","last_change":"2020-08-07T15:12:10.465251+0000","last_active":"2020-08-07T15:12:41.758625+0000","last_peered":"2020-08-07T15:12:41.758625+0000","last_clean":"2020-08-07T15:12:41.758625+0000","last_became_active":"2020-08-07T15:12:10.464698+0000","last_became_peered":"2020-08-07T15:12:10.464698+0000","last_unstale":"2020-08-07T15:12:41.758625+0000","last_undegraded":"2020-08-07T15:12:41.758625+0000","last_fullsized":"2020-08-07T15:12:41.758625+0000","mapping_epoch":230,"log_start":"254'2988","ondisk_log_start":"254'2988","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24510464,"num_objects":374,"num_object_clones":0,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2992,"num_write_kb":23936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"222","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.434066+0000","last_change":"2020-08-07T15:12:06.367666+0000","last_active":"2020-08-07T15:12:40.434066+0000","last_peered":"2020-08-07T15:12:40.434066+0000","last_clean":"2020-08-07T15:12:40.434066+0000","last_became_active":"2020-08-07T15:12:06.367081+0000","last_became_peered":"2020-08-07T15:12:06.367081+0000","last_unstale":"2020-08-07T15:12:40.434066+0000","last_undegraded":"2020-08-07T15:12:40.434066+0000","last_fullsized":"2020-08-07T15:12:40.434066+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:37.601099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:37.601099+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":369287168,"num_objects":5635,"num_object_clones":0,"num_object_copies":16905,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45079,"num_write_kb":360632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1117216768,"data_stored":1117216768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":41,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":123,"acting":123,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542223,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1166244,"kb_used_data":117660,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93205596,"statfs":{"total":96636764160,"available":95442530304,"internally_reserved":1073741824,"allocated":120483840,"data_stored":119236527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":9.5359999999999996,"15min":3.774},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":2.294,"5min":856.07100000000003,"15min":856.07100000000003},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.77700000000000002,"5min":1.002,"15min":0.97599999999999998},"min":{"1min":0.52300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.55,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":1.5209999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.335,"15min":1.3080000000000001},"min":{"1min":0.64200000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.9199999999999999,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.141,"15min":1.024},"min":{"1min":0.61799999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":4.0060000000000002,"5min":4.0060000000000002,"15min":4.0060000000000002},"last":0.76800000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.321,"15min":1.1910000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.5630000000000002,"5min":5.6619999999999999,"15min":5.6619999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.081,"5min":1.3280000000000001,"15min":1.1879999999999999},"min":{"1min":0.69299999999999995,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.3769999999999998,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":0.85799999999999998}]},{"osd":3,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":30.957999999999998,"5min":21.91,"15min":8.4179999999999993},"min":{"1min":0.68899999999999995,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":656.99099999999999,"5min":917.38699999999994,"15min":917.38699999999994},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.173,"5min":4.8879999999999999,"15min":2.524},"min":{"1min":0.61599999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":4.0460000000000003,"5min":333.59199999999998,"15min":333.59199999999998},"last":2.383}]},{"osd":4,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":8.8599999999999994,"5min":2.944,"15min":1.8320000000000001},"min":{"1min":0.66000000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":172.012,"5min":172.012,"15min":172.012},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.5509999999999999,"5min":11.529,"15min":4.7169999999999996},"min":{"1min":0.81200000000000006,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":5.4500000000000002,"5min":866.24099999999999,"15min":866.24099999999999},"last":1.5569999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":9.3930000000000007,"15min":28.135000000000002},"min":{"1min":0.68400000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":3.536,"5min":678.76099999999997,"15min":831.88099999999997},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.623,"5min":1.831,"15min":1.607},"min":{"1min":0.89000000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":3.593,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":1.321}]},{"osd":7,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":3.9929999999999999,"15min":2.343},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":4.0810000000000004,"5min":224.14500000000001,"15min":224.14500000000001},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.671,"5min":1.7649999999999999,"15min":1.593},"min":{"1min":0.92000000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":3.6200000000000001,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":0.91900000000000004}]}]},{"osd":7,"up_from":13,"seq":55834574928,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1181924,"kb_used_data":133340,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93189916,"statfs":{"total":96636764160,"available":95426473984,"internally_reserved":1073741824,"allocated":136540160,"data_stored":135307876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":37.372999999999998,"5min":16.672000000000001,"15min":6.4459999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":763.39099999999996,"5min":763.39099999999996,"15min":763.39099999999996},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.157,"5min":9.7789999999999999,"15min":4.0259999999999998},"min":{"1min":0.61699999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.8279999999999998,"5min":529.88699999999994,"15min":529.88699999999994},"last":1.095}]},{"osd":1,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":5.758,"15min":2.5579999999999998},"min":{"1min":0.47699999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.7290000000000001,"5min":547.65300000000002,"15min":547.65300000000002},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.4159999999999999,"5min":4.9189999999999996,"15min":2.5640000000000001},"min":{"1min":0.57399999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":5.4960000000000004,"5min":372.50799999999998,"15min":372.50799999999998},"last":1.119}]},{"osd":2,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":3.0009999999999999,"5min":1.6850000000000001,"15min":1.3580000000000001},"min":{"1min":0.52000000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":33.131999999999998,"5min":33.131999999999998,"15min":33.131999999999998},"last":1.4890000000000001},{"interface":"front","average":{"1min":2.8410000000000002,"5min":1.494,"15min":1.3460000000000001},"min":{"1min":0.61799999999999999,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":32.924999999999997,"5min":32.924999999999997,"15min":32.924999999999997},"last":1.1200000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":9.4659999999999993,"15min":4.1719999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":5.9329999999999998,"5min":698.19899999999996,"15min":698.19899999999996},"last":3.1480000000000001},{"interface":"front","average":{"1min":1.238,"5min":1.2669999999999999,"15min":1.347},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":3.8599999999999999,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":1.155}]},{"osd":4,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":25.448,"5min":6.069,"15min":3.1309999999999998},"min":{"1min":0.76400000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":460.78399999999999,"5min":460.78399999999999,"15min":460.78399999999999},"last":1.544},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.474,"15min":1.6060000000000001},"min":{"1min":0.86899999999999999,"5min":0.621,"15min":0.621},"max":{"1min":2.4079999999999999,"5min":3.694,"15min":5.2990000000000004},"last":1.577}]},{"osd":5,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.3720000000000001,"15min":1.3660000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":5.9989999999999997,"5min":6.2080000000000002,"15min":6.2080000000000002},"last":1.6659999999999999},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.492,"15min":1.6739999999999999},"min":{"1min":0.83599999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":4.407,"5min":4.407,"15min":5.327},"last":1.6160000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":9.1839999999999993,"15min":4.016},"min":{"1min":0.88,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.3290000000000002,"5min":695.92899999999997,"15min":695.92899999999997},"last":1.6890000000000001},{"interface":"front","average":{"1min":1.597,"5min":1.55,"15min":12.734},"min":{"1min":0.89500000000000002,"5min":0.87,"15min":0.87},"max":{"1min":3.302,"5min":3.7309999999999999,"15min":266.26999999999998},"last":1.7070000000000001}]}]},{"osd":1,"up_from":227,"seq":974957576200,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1156284,"kb_used_data":107700,"kb_used_omap":2,"kb_used_meta":1048573,"kb_avail":93215556,"statfs":{"total":96636764160,"available":95452729344,"internally_reserved":1073741824,"allocated":110284800,"data_stored":109063607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":1073739245},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3380000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7229999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.548}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.609}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.978}]}]},{"osd":0,"up_from":13,"seq":55834574928,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1137572,"kb_used_data":88988,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93234268,"statfs":{"total":96636764160,"available":95471890432,"internally_reserved":1073741824,"allocated":91123712,"data_stored":89891428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":24.291,"5min":21.719999999999999,"15min":7.891},"min":{"1min":0.52700000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":432.899,"5min":845.57100000000003,"15min":845.57100000000003},"last":1.9199999999999999},{"interface":"front","average":{"1min":1.752,"5min":1.181,"15min":1.0149999999999999},"min":{"1min":0.62,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":9.3650000000000002,"5min":9.3650000000000002,"15min":9.3650000000000002},"last":1.4099999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.0620000000000001,"15min":0.97199999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":7.3470000000000004,"5min":7.3470000000000004,"15min":7.3470000000000004},"last":1.8939999999999999},{"interface":"front","average":{"1min":6.2839999999999998,"5min":4.665,"15min":23.381},"min":{"1min":0.47399999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":75.622,"5min":226.911,"15min":606.18499999999995},"last":2.593}]},{"osd":3,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":28.079000000000001,"5min":6.6920000000000002,"15min":2.8540000000000001},"min":{"1min":0.60999999999999999,"5min":0.54100000000000004,"15min":0.52300000000000002},"max":{"1min":431.95299999999997,"5min":431.95299999999997,"15min":431.95299999999997},"last":1.109},{"interface":"front","average":{"1min":1.425,"5min":1.46,"15min":1.2649999999999999},"min":{"1min":0.66400000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":5.3780000000000001,"5min":11.19,"15min":11.19},"last":2.2360000000000002}]},{"osd":4,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.302,"15min":1.2110000000000001},"min":{"1min":0.57399999999999995,"5min":0.501,"15min":0.501},"max":{"1min":1.819,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.448,"15min":1.321},"min":{"1min":0.71099999999999997,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.8849999999999998,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":2.1800000000000002}]},{"osd":5,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.6200000000000001,"5min":14.598000000000001,"15min":5.6840000000000002},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":7.3799999999999999,"5min":539.31100000000004,"15min":539.31100000000004},"last":1.7969999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.49,"15min":1.234},"min":{"1min":0.81399999999999995,"5min":0.752,"15min":0.65900000000000003},"max":{"1min":2.9420000000000002,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":1.3740000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:12:34 2020","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.4399999999999999,"15min":6.2220000000000004},"min":{"1min":0.80200000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.8610000000000002,"5min":4.1310000000000002,"15min":124.738},"last":2.21},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.5549999999999999,"15min":12.952999999999999},"min":{"1min":0.73499999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":5.274,"5min":5.274,"15min":294.94},"last":1.831}]},{"osd":7,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.4790000000000001,"15min":1.55},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.79300000000000004},"max":{"1min":1.8620000000000001,"5min":4.7169999999999996,"15min":6.468},"last":1.8560000000000001},{"interface":"front","average":{"1min":66.063000000000002,"5min":23.518000000000001,"15min":8.7629999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":971.12699999999995,"5min":971.12699999999995,"15min":971.12699999999995},"last":1.8759999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1227200,"kb_used_data":178616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93144640,"statfs":{"total":96636764160,"available":95380111360,"internally_reserved":1073741824,"allocated":182902784,"data_stored":181662963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":32.965000000000003,"5min":7.8639999999999999,"15min":3.3660000000000001},"min":{"1min":0.41899999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":501.53800000000001,"5min":501.53800000000001,"15min":501.53800000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":27.341999999999999,"5min":6.6900000000000004,"15min":2.8799999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":501.36000000000001,"5min":501.36000000000001,"15min":501.36000000000001},"last":0.88600000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:10:21 2020","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.6970000000000001,"15min":1.2929999999999999},"min":{"1min":0.52000000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":5.5919999999999996,"5min":18.672000000000001,"15min":18.672000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.264,"5min":1.3320000000000001,"15min":1.3500000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":3.7970000000000002,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":1.637}]},{"osd":3,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.2130000000000001,"15min":1.179},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":3.0760000000000001},{"interface":"front","average":{"1min":23.064,"5min":15.281000000000001,"15min":6.1310000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":412.61500000000001,"5min":501.75599999999997,"15min":501.75599999999997},"last":1.3480000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.5940000000000001,"15min":1.2709999999999999},"min":{"1min":0.67400000000000004,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.712,"5min":19.783000000000001,"15min":19.783000000000001},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.4610000000000001,"15min":1.5820000000000001},"min":{"1min":0.68999999999999995,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8979999999999999,"5min":8.048,"15min":8.048},"last":1.6120000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.3460000000000001,"15min":1.29},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.4009999999999998,"5min":5.2220000000000004,"15min":5.2220000000000004},"last":1.79},{"interface":"front","average":{"1min":1.4990000000000001,"5min":15.715,"15min":22.254999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":8.0649999999999995,"5min":627.53300000000002,"15min":627.53300000000002},"last":0.85899999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:12:37 2020","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":9.9480000000000004,"15min":4.1299999999999999},"min":{"1min":0.875,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.9450000000000001,"5min":821.91899999999998,"15min":821.91899999999998},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.3959999999999999,"5min":9.9580000000000002,"15min":4.2000000000000002},"min":{"1min":0.70199999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.9540000000000002,"5min":751.70000000000005,"15min":751.70000000000005},"last":1.76}]},{"osd":7,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":12.522,"15min":5.335},"min":{"1min":0.85599999999999998,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.3519999999999999,"5min":501.91699999999997,"15min":501.91699999999997},"last":1.54},{"interface":"front","average":{"1min":1.7230000000000001,"5min":2.1080000000000001,"15min":1.7},"min":{"1min":0.88500000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":7.2149999999999999,"5min":35.646999999999998,"15min":35.646999999999998},"last":1.5620000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574928,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1278236,"kb_used_data":229652,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93093604,"statfs":{"total":96636764160,"available":95327850496,"internally_reserved":1073741824,"allocated":235163648,"data_stored":233931364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.226,"15min":1.244},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":4.6580000000000004,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":1.524},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.2050000000000001,"15min":1.4930000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":4.6349999999999998,"5min":4.9119999999999999,"15min":13.067},"last":0.93799999999999994}]},{"osd":1,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":4.4779999999999998,"15min":2.758},"min":{"1min":0.59099999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.847,"5min":270.68700000000001,"15min":270.68700000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.109,"5min":1.8580000000000001,"15min":1.96},"min":{"1min":0.54100000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.0550000000000002,"5min":14.349,"15min":14.349},"last":0.97299999999999998}]},{"osd":2,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.365,"15min":1.22},"min":{"1min":0.65900000000000003,"5min":0.56999999999999995,"15min":0.56899999999999995},"max":{"1min":2.8300000000000001,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":1.6679999999999999},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.9690000000000001,"15min":1.7789999999999999},"min":{"1min":0.64500000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":5.3090000000000002,"5min":51.530000000000001,"15min":51.530000000000001},"last":1.409}]},{"osd":4,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":3.831,"5min":17.792999999999999,"15min":7.0620000000000003},"min":{"1min":0.54500000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":56.688000000000002,"5min":646.26700000000005,"15min":646.26700000000005},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.421,"5min":15.494999999999999,"15min":26.565000000000001},"min":{"1min":0.68100000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":3.2109999999999999,"5min":844.10500000000002,"15min":844.10500000000002},"last":1.5760000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":14.452999999999999,"5min":8.5429999999999993,"15min":3.859},"min":{"1min":0.95499999999999996,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":244.08099999999999,"5min":403.88400000000001,"15min":403.88400000000001},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.78,"5min":1.8979999999999999,"15min":1.8120000000000001},"min":{"1min":0.98499999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":4.5960000000000001,"5min":8.9120000000000008,"15min":8.9120000000000008},"last":1.6419999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.7969999999999999,"5min":1.706,"15min":1.5900000000000001},"min":{"1min":0.74099999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":5.2729999999999997,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":1.95},{"interface":"front","average":{"1min":1.907,"5min":13.411,"15min":5.7949999999999999},"min":{"1min":1.032,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":5.3460000000000001,"5min":688.15999999999997,"15min":688.15999999999997},"last":1.976}]},{"osd":7,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.7450000000000001,"5min":1.5960000000000001,"15min":1.9730000000000001},"min":{"1min":0.85599999999999998,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":4.8099999999999996,"5min":6.8209999999999997,"15min":6.8209999999999997},"last":1.296},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.6559999999999999,"15min":1.8700000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.6099999999999999,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":1.5409999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574927,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1165464,"kb_used_data":116880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93206376,"statfs":{"total":96636764160,"available":95443329024,"internally_reserved":1073741824,"allocated":119685120,"data_stored":118445299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":49.795000000000002,"5min":19.646000000000001,"15min":7.3360000000000003},"min":{"1min":0.46800000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":829.15099999999995,"5min":829.15099999999995,"15min":829.15099999999995},"last":0.628},{"interface":"front","average":{"1min":3.8149999999999999,"5min":1.639,"15min":1.1120000000000001},"min":{"1min":0.45300000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":48.274000000000001,"5min":48.274000000000001,"15min":48.274000000000001},"last":0.81100000000000005}]},{"osd":1,"last update":"Fri Aug 7 15:10:24 2020","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":5.7629999999999999,"15min":2.823},"min":{"1min":0.52900000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.8109999999999999,"5min":320.67000000000002,"15min":320.67000000000002},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.177,"5min":7.5300000000000002,"15min":3.4279999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.7200000000000002,"5min":689.846,"15min":689.846},"last":0.76500000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":4.1159999999999997,"5min":1.901,"15min":1.601},"min":{"1min":0.64300000000000002,"5min":0.56399999999999995,"15min":0.52100000000000002},"max":{"1min":49.878999999999998,"5min":49.878999999999998,"15min":49.878999999999998},"last":1.1060000000000001},{"interface":"front","average":{"1min":4.2270000000000003,"5min":2.0529999999999999,"15min":1.694},"min":{"1min":0.97899999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":49.939999999999998,"5min":49.939999999999998,"15min":49.939999999999998},"last":0.51700000000000002}]},{"osd":3,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":27.59,"5min":17.260999999999999,"15min":6.7590000000000003},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":426.50799999999998,"5min":638.92100000000005,"15min":638.92100000000005},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.411,"5min":1.3799999999999999,"15min":1.294},"min":{"1min":0.75700000000000001,"5min":0.67800000000000005,"15min":0.66700000000000004},"max":{"1min":3.2360000000000002,"5min":6.431,"15min":6.431},"last":0.92000000000000004}]},{"osd":5,"last update":"Fri Aug 7 15:12:29 2020","interfaces":[{"interface":"back","average":{"1min":52.308999999999997,"5min":28.574999999999999,"15min":10.609999999999999},"min":{"1min":0.56100000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":862.97299999999996,"5min":990.71600000000001,"15min":990.71600000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":49.549999999999997,"5min":11.01,"15min":4.7539999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":780.00699999999995,"5min":780.00699999999995,"15min":780.00699999999995},"last":0.66200000000000003}]},{"osd":6,"last update":"Fri Aug 7 15:12:34 2020","interfaces":[{"interface":"back","average":{"1min":3.8239999999999998,"5min":1.9870000000000001,"15min":1.855},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":48.575000000000003,"5min":48.575000000000003,"15min":48.575000000000003},"last":0.878},{"interface":"front","average":{"1min":1.109,"5min":1.333,"15min":13.183},"min":{"1min":0.64600000000000002,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.847,"5min":6.7729999999999997,"15min":389.14600000000002},"last":0.85699999999999998}]},{"osd":7,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":4.1349999999999998,"5min":4.327,"15min":2.3809999999999998},"min":{"1min":0.74199999999999999,"5min":0.62,"15min":0.62},"max":{"1min":48.75,"5min":252.798,"15min":252.798},"last":1.1639999999999999},{"interface":"front","average":{"1min":4.2610000000000001,"5min":14.635,"15min":5.9749999999999996},"min":{"1min":0.84799999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":49.654000000000003,"5min":501.93000000000001,"15min":501.93000000000001},"last":1.077}]}]},{"osd":5,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1165044,"kb_used_data":116460,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93206796,"statfs":{"total":96636764160,"available":95443759104,"internally_reserved":1073741824,"allocated":119255040,"data_stored":118022756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":3.629,"5min":1.506,"15min":1.22},"min":{"1min":0.42199999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":43.292999999999999,"5min":43.292999999999999,"15min":43.292999999999999},"last":1.3160000000000001},{"interface":"front","average":{"1min":3.1520000000000001,"5min":4.4249999999999998,"15min":2.0110000000000001},"min":{"1min":0.44700000000000001,"5min":0.438,"15min":0.438},"max":{"1min":42.933,"5min":329.11399999999998,"15min":329.11399999999998},"last":0.56499999999999995}]},{"osd":1,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":8.5239999999999991,"5min":12.039999999999999,"15min":4.6319999999999997},"min":{"1min":0.56799999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":139.88200000000001,"5min":787.22900000000004,"15min":787.22900000000004},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.448,"5min":2.5649999999999999,"15min":1.593},"min":{"1min":0.59899999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":4.3700000000000001,"5min":116.548,"15min":116.548},"last":1.1339999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":3.5499999999999998,"5min":1.5800000000000001,"15min":6.673},"min":{"1min":0.65700000000000003,"5min":0.499,"15min":0.499},"max":{"1min":43.923000000000002,"5min":43.923000000000002,"15min":168.56899999999999},"last":1.3500000000000001},{"interface":"front","average":{"1min":3.23,"5min":1.6399999999999999,"15min":22.634},"min":{"1min":0.52800000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":43.076999999999998,"5min":43.076999999999998,"15min":676.27099999999996},"last":1.0049999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.728,"15min":1.321},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":4.7640000000000002,"5min":29.129000000000001,"15min":29.129000000000001},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":10.029999999999999,"15min":4.1520000000000001},"min":{"1min":0.73999999999999999,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":5.2009999999999996,"5min":712.98699999999997,"15min":712.98699999999997},"last":1.375}]},{"osd":4,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":12.779999999999999,"15min":38.188000000000002},"min":{"1min":0.77200000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":4.9969999999999999,"5min":971.67999999999995,"15min":971.67999999999995},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.2470000000000001,"15min":1.286},"min":{"1min":0.73599999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.1160000000000001,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":1.032}]},{"osd":6,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":35.393000000000001,"5min":28.459,"15min":43.091999999999999},"min":{"1min":0.72199999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":676.20000000000005,"5min":966.60900000000004,"15min":966.60900000000004},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.528,"5min":6.7789999999999999,"15min":19.603999999999999},"min":{"1min":0.89500000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":5.2220000000000004,"5min":501.79000000000002,"15min":517.26700000000005},"last":1.6930000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.462,"15min":1.3600000000000001},"min":{"1min":0.85399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":4.8600000000000003,"5min":5.1559999999999997,"15min":5.1559999999999997},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.5669999999999999,"5min":12.834,"15min":5.3220000000000001},"min":{"1min":0.70099999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":3.8999999999999999,"5min":515.09100000000001,"15min":515.09100000000001},"last":1.0569999999999999}]}]}]}} 2020-08-07T15:12:44.535 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:12:44.539 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:12:44.540 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 32 16 49590 49574 12.1015 13.1094 0.00489183 0.0103202 2020-08-07T15:12:44.541 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:44.525+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:44.637 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:44.635+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:44.694 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:44.694 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:12:44.695 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2020-08-07T15:12:44.695 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:44.696 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:12:44.773 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:44.770+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:44.842 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:44.840+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:44.868 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:44.869 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:12:44.869 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:12:44.869 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:12:44.869 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:44.881 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:12:44.945 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:44.942+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:45.036 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:12:45.050 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":452,"stamp":"2020-08-07T15:12:44.686317+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":369287168,"num_objects":5637,"num_object_clones":0,"num_object_copies":16911,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45079,"num_write_kb":360632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":196,"ondisk_log_size":196,"up":195,"acting":195,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":198,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9477968,"kb_used_data":1089296,"kb_used_omap":2,"kb_used_meta":8388605,"kb_avail":745496752,"statfs":{"total":773094113280,"available":763388674048,"internally_reserved":8589934592,"allocated":1115439104,"data_stored":1105561820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":8589932013},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":52822016,"num_objects":806,"num_object_clones":0,"num_object_copies":2418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6448,"num_write_kb":51584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000742"},"pg_stats":[{"pgid":"1.28","version":"16'2","reported_seq":"18","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.433782+0000","last_change":"2020-08-07T15:12:37.495634+0000","last_active":"2020-08-07T15:12:40.433782+0000","last_peered":"2020-08-07T15:12:40.433782+0000","last_clean":"2020-08-07T15:12:40.433782+0000","last_became_active":"2020-08-07T15:12:37.495085+0000","last_became_peered":"2020-08-07T15:12:37.495085+0000","last_unstale":"2020-08-07T15:12:40.433782+0000","last_undegraded":"2020-08-07T15:12:40.433782+0000","last_fullsized":"2020-08-07T15:12:40.433782+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.490655+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.490655+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.27","version":"16'2","reported_seq":"364","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.461533+0000","last_change":"2020-08-07T15:12:37.412480+0000","last_active":"2020-08-07T15:12:41.461533+0000","last_peered":"2020-08-07T15:12:41.461533+0000","last_clean":"2020-08-07T15:12:41.461533+0000","last_became_active":"2020-08-07T15:12:37.411869+0000","last_became_peered":"2020-08-07T15:12:37.411869+0000","last_unstale":"2020-08-07T15:12:41.461533+0000","last_undegraded":"2020-08-07T15:12:41.461533+0000","last_fullsized":"2020-08-07T15:12:41.461533+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:15.163639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:15.163639+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.26","version":"16'2","reported_seq":"23","reported_epoch":"251","state":"active+clean","last_fresh":"2020-08-07T15:12:38.422097+0000","last_change":"2020-08-07T15:12:37.458242+0000","last_active":"2020-08-07T15:12:38.422097+0000","last_peered":"2020-08-07T15:12:38.422097+0000","last_clean":"2020-08-07T15:12:38.422097+0000","last_became_active":"2020-08-07T15:12:37.457426+0000","last_became_peered":"2020-08-07T15:12:37.457426+0000","last_unstale":"2020-08-07T15:12:38.422097+0000","last_undegraded":"2020-08-07T15:12:38.422097+0000","last_fullsized":"2020-08-07T15:12:38.422097+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_clean_scrub_stamp":"2020-08-07T15:12:14.121397+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.25","version":"75'3","reported_seq":"251","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.441396+0000","last_change":"2020-08-07T15:12:37.428239+0000","last_active":"2020-08-07T15:12:40.441396+0000","last_peered":"2020-08-07T15:12:40.441396+0000","last_clean":"2020-08-07T15:12:40.441396+0000","last_became_active":"2020-08-07T15:12:37.427816+0000","last_became_peered":"2020-08-07T15:12:37.427816+0000","last_unstale":"2020-08-07T15:12:40.441396+0000","last_undegraded":"2020-08-07T15:12:40.441396+0000","last_fullsized":"2020-08-07T15:12:40.441396+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:12.545945+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.545945+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.24","version":"16'2","reported_seq":"189","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.439772+0000","last_change":"2020-08-07T15:12:34.415315+0000","last_active":"2020-08-07T15:12:40.439772+0000","last_peered":"2020-08-07T15:12:40.439772+0000","last_clean":"2020-08-07T15:12:40.439772+0000","last_became_active":"2020-08-07T15:12:34.414144+0000","last_became_peered":"2020-08-07T15:12:34.414144+0000","last_unstale":"2020-08-07T15:12:40.439772+0000","last_undegraded":"2020-08-07T15:12:40.439772+0000","last_fullsized":"2020-08-07T15:12:40.439772+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:10.265395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:10.265395+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.23","version":"16'2","reported_seq":"299","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.461359+0000","last_change":"2020-08-07T15:12:37.410705+0000","last_active":"2020-08-07T15:12:41.461359+0000","last_peered":"2020-08-07T15:12:41.461359+0000","last_clean":"2020-08-07T15:12:41.461359+0000","last_became_active":"2020-08-07T15:12:37.410157+0000","last_became_peered":"2020-08-07T15:12:37.410157+0000","last_unstale":"2020-08-07T15:12:41.461359+0000","last_undegraded":"2020-08-07T15:12:41.461359+0000","last_fullsized":"2020-08-07T15:12:41.461359+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:10.175739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:10.175739+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1],"acting":[6,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.22","version":"16'2","reported_seq":"19","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.461584+0000","last_change":"2020-08-07T15:12:37.413542+0000","last_active":"2020-08-07T15:12:41.461584+0000","last_peered":"2020-08-07T15:12:41.461584+0000","last_clean":"2020-08-07T15:12:41.461584+0000","last_became_active":"2020-08-07T15:12:37.413046+0000","last_became_peered":"2020-08-07T15:12:37.413046+0000","last_unstale":"2020-08-07T15:12:41.461584+0000","last_undegraded":"2020-08-07T15:12:41.461584+0000","last_fullsized":"2020-08-07T15:12:41.461584+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:11.658923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:11.658923+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.21","version":"16'2","reported_seq":"25","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.439010+0000","last_change":"2020-08-07T15:12:37.419228+0000","last_active":"2020-08-07T15:12:40.439010+0000","last_peered":"2020-08-07T15:12:40.439010+0000","last_clean":"2020-08-07T15:12:40.439010+0000","last_became_active":"2020-08-07T15:12:37.418612+0000","last_became_peered":"2020-08-07T15:12:37.418612+0000","last_unstale":"2020-08-07T15:12:40.439010+0000","last_undegraded":"2020-08-07T15:12:40.439010+0000","last_fullsized":"2020-08-07T15:12:40.439010+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:09.261101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.261101+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.20","version":"16'2","reported_seq":"25","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.440988+0000","last_change":"2020-08-07T15:12:37.409857+0000","last_active":"2020-08-07T15:12:40.440988+0000","last_peered":"2020-08-07T15:12:40.440988+0000","last_clean":"2020-08-07T15:12:40.440988+0000","last_became_active":"2020-08-07T15:12:37.409249+0000","last_became_peered":"2020-08-07T15:12:37.409249+0000","last_unstale":"2020-08-07T15:12:40.440988+0000","last_undegraded":"2020-08-07T15:12:40.440988+0000","last_fullsized":"2020-08-07T15:12:40.440988+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:40.696862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:40.696862+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1f","version":"16'2","reported_seq":"18","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.444823+0000","last_change":"2020-08-07T15:12:37.411233+0000","last_active":"2020-08-07T15:12:40.444823+0000","last_peered":"2020-08-07T15:12:40.444823+0000","last_clean":"2020-08-07T15:12:40.444823+0000","last_became_active":"2020-08-07T15:12:37.410716+0000","last_became_peered":"2020-08-07T15:12:37.410716+0000","last_unstale":"2020-08-07T15:12:40.444823+0000","last_undegraded":"2020-08-07T15:12:40.444823+0000","last_fullsized":"2020-08-07T15:12:40.444823+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.176383+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1e","version":"16'2","reported_seq":"105","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.439485+0000","last_change":"2020-08-07T15:11:37.457186+0000","last_active":"2020-08-07T15:12:40.439485+0000","last_peered":"2020-08-07T15:12:40.439485+0000","last_clean":"2020-08-07T15:12:40.439485+0000","last_became_active":"2020-08-07T15:10:51.559894+0000","last_became_peered":"2020-08-07T15:10:51.559894+0000","last_unstale":"2020-08-07T15:12:40.439485+0000","last_undegraded":"2020-08-07T15:12:40.439485+0000","last_fullsized":"2020-08-07T15:12:40.439485+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.457111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.457111+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1d","version":"75'3","reported_seq":"116","reported_epoch":"251","state":"active+clean","last_fresh":"2020-08-07T15:12:38.422054+0000","last_change":"2020-08-07T15:12:28.855487+0000","last_active":"2020-08-07T15:12:38.422054+0000","last_peered":"2020-08-07T15:12:38.422054+0000","last_clean":"2020-08-07T15:12:38.422054+0000","last_became_active":"2020-08-07T15:12:07.356691+0000","last_became_peered":"2020-08-07T15:12:07.356691+0000","last_unstale":"2020-08-07T15:12:38.422054+0000","last_undegraded":"2020-08-07T15:12:38.422054+0000","last_fullsized":"2020-08-07T15:12:38.422054+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:28.855432+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:28.855432+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1c","version":"16'2","reported_seq":"193","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.435626+0000","last_change":"2020-08-07T15:12:07.262797+0000","last_active":"2020-08-07T15:12:40.435626+0000","last_peered":"2020-08-07T15:12:40.435626+0000","last_clean":"2020-08-07T15:12:40.435626+0000","last_became_active":"2020-08-07T15:12:07.245731+0000","last_became_peered":"2020-08-07T15:12:07.245731+0000","last_unstale":"2020-08-07T15:12:40.435626+0000","last_undegraded":"2020-08-07T15:12:40.435626+0000","last_fullsized":"2020-08-07T15:12:40.435626+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:39.738160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:39.738160+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1b","version":"16'2","reported_seq":"226","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.433916+0000","last_change":"2020-08-07T15:10:57.797029+0000","last_active":"2020-08-07T15:12:40.433916+0000","last_peered":"2020-08-07T15:12:40.433916+0000","last_clean":"2020-08-07T15:12:40.433916+0000","last_became_active":"2020-08-07T15:10:51.557392+0000","last_became_peered":"2020-08-07T15:10:51.557392+0000","last_unstale":"2020-08-07T15:12:40.433916+0000","last_undegraded":"2020-08-07T15:12:40.433916+0000","last_fullsized":"2020-08-07T15:12:40.433916+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:57.796946+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:57.796946+0000","last_clean_scrub_stamp":"2020-08-07T15:10:57.796946+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1a","version":"16'2","reported_seq":"93","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.834667+0000","last_change":"2020-08-07T15:12:40.834667+0000","last_active":"2020-08-07T15:12:40.834667+0000","last_peered":"2020-08-07T15:12:40.834667+0000","last_clean":"2020-08-07T15:12:40.834667+0000","last_became_active":"2020-08-07T15:12:07.243542+0000","last_became_peered":"2020-08-07T15:12:07.243542+0000","last_unstale":"2020-08-07T15:12:40.834667+0000","last_undegraded":"2020-08-07T15:12:40.834667+0000","last_fullsized":"2020-08-07T15:12:40.834667+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:40.834612+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:12:40.834612+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.19","version":"16'2","reported_seq":"151","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.439121+0000","last_change":"2020-08-07T15:12:15.372999+0000","last_active":"2020-08-07T15:12:40.439121+0000","last_peered":"2020-08-07T15:12:40.439121+0000","last_clean":"2020-08-07T15:12:40.439121+0000","last_became_active":"2020-08-07T15:10:51.559752+0000","last_became_peered":"2020-08-07T15:10:51.559752+0000","last_unstale":"2020-08-07T15:12:40.439121+0000","last_undegraded":"2020-08-07T15:12:40.439121+0000","last_fullsized":"2020-08-07T15:12:40.439121+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:15.372910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:15.372910+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.18","version":"16'2","reported_seq":"251","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.433782+0000","last_change":"2020-08-07T15:11:06.955336+0000","last_active":"2020-08-07T15:12:40.433782+0000","last_peered":"2020-08-07T15:12:40.433782+0000","last_clean":"2020-08-07T15:12:40.433782+0000","last_became_active":"2020-08-07T15:10:51.557665+0000","last_became_peered":"2020-08-07T15:10:51.557665+0000","last_unstale":"2020-08-07T15:12:40.433782+0000","last_undegraded":"2020-08-07T15:12:40.433782+0000","last_fullsized":"2020-08-07T15:12:40.433782+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:06.955262+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:06.955262+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.17","version":"16'2","reported_seq":"79","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.434011+0000","last_change":"2020-08-07T15:11:08.916219+0000","last_active":"2020-08-07T15:12:40.434011+0000","last_peered":"2020-08-07T15:12:40.434011+0000","last_clean":"2020-08-07T15:12:40.434011+0000","last_became_active":"2020-08-07T15:10:51.561769+0000","last_became_peered":"2020-08-07T15:10:51.561769+0000","last_unstale":"2020-08-07T15:12:40.434011+0000","last_undegraded":"2020-08-07T15:12:40.434011+0000","last_fullsized":"2020-08-07T15:12:40.434011+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:08.916169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:08.916169+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.16","version":"16'2","reported_seq":"54","reported_epoch":"251","state":"active+clean","last_fresh":"2020-08-07T15:12:38.422194+0000","last_change":"2020-08-07T15:12:12.883619+0000","last_active":"2020-08-07T15:12:38.422194+0000","last_peered":"2020-08-07T15:12:38.422194+0000","last_clean":"2020-08-07T15:12:38.422194+0000","last_became_active":"2020-08-07T15:12:07.231877+0000","last_became_peered":"2020-08-07T15:12:07.231877+0000","last_unstale":"2020-08-07T15:12:38.422194+0000","last_undegraded":"2020-08-07T15:12:38.422194+0000","last_fullsized":"2020-08-07T15:12:38.422194+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:12.883558+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:12.883558+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.883558+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.15","version":"75'3","reported_seq":"90","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.435699+0000","last_change":"2020-08-07T15:12:07.284341+0000","last_active":"2020-08-07T15:12:40.435699+0000","last_peered":"2020-08-07T15:12:40.435699+0000","last_clean":"2020-08-07T15:12:40.435699+0000","last_became_active":"2020-08-07T15:12:07.276899+0000","last_became_peered":"2020-08-07T15:12:07.276899+0000","last_unstale":"2020-08-07T15:12:40.435699+0000","last_undegraded":"2020-08-07T15:12:40.435699+0000","last_fullsized":"2020-08-07T15:12:40.435699+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:11:20.562710+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:11:20.562710+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.14","version":"16'2","reported_seq":"189","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.441265+0000","last_change":"2020-08-07T15:12:09.224231+0000","last_active":"2020-08-07T15:12:40.441265+0000","last_peered":"2020-08-07T15:12:40.441265+0000","last_clean":"2020-08-07T15:12:40.441265+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:12:40.441265+0000","last_undegraded":"2020-08-07T15:12:40.441265+0000","last_fullsized":"2020-08-07T15:12:40.441265+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:09.224158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.224158+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"202","reported_epoch":"251","state":"active+clean","last_fresh":"2020-08-07T15:12:38.421837+0000","last_change":"2020-08-07T15:12:19.577926+0000","last_active":"2020-08-07T15:12:38.421837+0000","last_peered":"2020-08-07T15:12:38.421837+0000","last_clean":"2020-08-07T15:12:38.421837+0000","last_became_active":"2020-08-07T15:12:06.375861+0000","last_became_peered":"2020-08-07T15:12:06.375861+0000","last_unstale":"2020-08-07T15:12:38.421837+0000","last_undegraded":"2020-08-07T15:12:38.421837+0000","last_fullsized":"2020-08-07T15:12:38.421837+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:07.887356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.887356+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"204","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.434178+0000","last_change":"2020-08-07T15:12:26.104598+0000","last_active":"2020-08-07T15:12:40.434178+0000","last_peered":"2020-08-07T15:12:40.434178+0000","last_clean":"2020-08-07T15:12:40.434178+0000","last_became_active":"2020-08-07T15:12:06.367362+0000","last_became_peered":"2020-08-07T15:12:06.367362+0000","last_unstale":"2020-08-07T15:12:40.434178+0000","last_undegraded":"2020-08-07T15:12:40.434178+0000","last_fullsized":"2020-08-07T15:12:40.434178+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:26.104526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:26.104526+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"302","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.433819+0000","last_change":"2020-08-07T15:12:24.105593+0000","last_active":"2020-08-07T15:12:40.433819+0000","last_peered":"2020-08-07T15:12:40.433819+0000","last_clean":"2020-08-07T15:12:40.433819+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:12:40.433819+0000","last_undegraded":"2020-08-07T15:12:40.433819+0000","last_fullsized":"2020-08-07T15:12:40.433819+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:24.105531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:24.105531+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"296","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.433911+0000","last_change":"2020-08-07T15:12:13.231819+0000","last_active":"2020-08-07T15:12:40.433911+0000","last_peered":"2020-08-07T15:12:40.433911+0000","last_clean":"2020-08-07T15:12:40.433911+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:12:40.433911+0000","last_undegraded":"2020-08-07T15:12:40.433911+0000","last_fullsized":"2020-08-07T15:12:40.433911+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:13.231762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.231762+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3","version":"254'2584","reported_seq":"2616","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.677041+0000","last_change":"2020-08-07T15:12:10.464875+0000","last_active":"2020-08-07T15:12:41.677041+0000","last_peered":"2020-08-07T15:12:41.677041+0000","last_clean":"2020-08-07T15:12:41.677041+0000","last_became_active":"2020-08-07T15:12:10.464346+0000","last_became_peered":"2020-08-07T15:12:10.464346+0000","last_unstale":"2020-08-07T15:12:41.677041+0000","last_undegraded":"2020-08-07T15:12:41.677041+0000","last_fullsized":"2020-08-07T15:12:41.677041+0000","mapping_epoch":230,"log_start":"254'2579","ondisk_log_start":"254'2579","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21168128,"num_objects":323,"num_object_clones":0,"num_object_copies":969,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2584,"num_write_kb":20672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"257","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.461712+0000","last_change":"2020-08-07T15:12:27.410339+0000","last_active":"2020-08-07T15:12:41.461712+0000","last_peered":"2020-08-07T15:12:41.461712+0000","last_clean":"2020-08-07T15:12:41.461712+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:12:41.461712+0000","last_undegraded":"2020-08-07T15:12:41.461712+0000","last_fullsized":"2020-08-07T15:12:41.461712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:27.410262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:27.410262+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"184","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.439861+0000","last_change":"2020-08-07T15:12:39.258928+0000","last_active":"2020-08-07T15:12:40.439861+0000","last_peered":"2020-08-07T15:12:40.439861+0000","last_clean":"2020-08-07T15:12:40.439861+0000","last_became_active":"2020-08-07T15:12:32.259956+0000","last_became_peered":"2020-08-07T15:12:32.259956+0000","last_unstale":"2020-08-07T15:12:40.439861+0000","last_undegraded":"2020-08-07T15:12:40.439861+0000","last_fullsized":"2020-08-07T15:12:40.439861+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:39.258857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:39.258857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"254'2904","reported_seq":"2936","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.737320+0000","last_change":"2020-08-07T15:12:10.465048+0000","last_active":"2020-08-07T15:12:41.737320+0000","last_peered":"2020-08-07T15:12:41.737320+0000","last_clean":"2020-08-07T15:12:41.737320+0000","last_became_active":"2020-08-07T15:12:10.464493+0000","last_became_peered":"2020-08-07T15:12:10.464493+0000","last_unstale":"2020-08-07T15:12:41.737320+0000","last_undegraded":"2020-08-07T15:12:41.737320+0000","last_fullsized":"2020-08-07T15:12:41.737320+0000","mapping_epoch":230,"log_start":"254'2896","ondisk_log_start":"254'2896","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23789568,"num_objects":363,"num_object_clones":0,"num_object_copies":1089,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2904,"num_write_kb":23232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"210","reported_epoch":"251","state":"active+clean","last_fresh":"2020-08-07T15:12:38.422187+0000","last_change":"2020-08-07T15:12:19.561477+0000","last_active":"2020-08-07T15:12:38.422187+0000","last_peered":"2020-08-07T15:12:38.422187+0000","last_clean":"2020-08-07T15:12:38.422187+0000","last_became_active":"2020-08-07T15:12:06.367376+0000","last_became_peered":"2020-08-07T15:12:06.367376+0000","last_unstale":"2020-08-07T15:12:38.422187+0000","last_undegraded":"2020-08-07T15:12:38.422187+0000","last_fullsized":"2020-08-07T15:12:38.422187+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:12.319383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.319383+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"235","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.441511+0000","last_change":"2020-08-07T15:12:32.241624+0000","last_active":"2020-08-07T15:12:40.441511+0000","last_peered":"2020-08-07T15:12:40.441511+0000","last_clean":"2020-08-07T15:12:40.441511+0000","last_became_active":"2020-08-07T15:12:32.241146+0000","last_became_peered":"2020-08-07T15:12:32.241146+0000","last_unstale":"2020-08-07T15:12:40.441511+0000","last_undegraded":"2020-08-07T15:12:40.441511+0000","last_fullsized":"2020-08-07T15:12:40.441511+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:12.545945+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.545945+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"252'2631","reported_seq":"2657","reported_epoch":"252","state":"active+clean","last_fresh":"2020-08-07T15:12:39.859177+0000","last_change":"2020-08-07T15:12:10.464412+0000","last_active":"2020-08-07T15:12:39.859177+0000","last_peered":"2020-08-07T15:12:39.859177+0000","last_clean":"2020-08-07T15:12:39.859177+0000","last_became_active":"2020-08-07T15:12:10.463950+0000","last_became_peered":"2020-08-07T15:12:10.463950+0000","last_unstale":"2020-08-07T15:12:39.859177+0000","last_undegraded":"2020-08-07T15:12:39.859177+0000","last_fullsized":"2020-08-07T15:12:39.859177+0000","mapping_epoch":230,"log_start":"252'2625","ondisk_log_start":"252'2625","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21553152,"num_objects":329,"num_object_clones":0,"num_object_copies":987,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2631,"num_write_kb":21048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"254","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.435855+0000","last_change":"2020-08-07T15:11:55.731391+0000","last_active":"2020-08-07T15:12:40.435855+0000","last_peered":"2020-08-07T15:12:40.435855+0000","last_clean":"2020-08-07T15:12:40.435855+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:12:40.435855+0000","last_undegraded":"2020-08-07T15:12:40.435855+0000","last_fullsized":"2020-08-07T15:12:40.435855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:11:55.731304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:11:55.731304+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"314","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.461587+0000","last_change":"2020-08-07T15:12:32.240238+0000","last_active":"2020-08-07T15:12:41.461587+0000","last_peered":"2020-08-07T15:12:41.461587+0000","last_clean":"2020-08-07T15:12:41.461587+0000","last_became_active":"2020-08-07T15:12:32.239755+0000","last_became_peered":"2020-08-07T15:12:32.239755+0000","last_unstale":"2020-08-07T15:12:41.461587+0000","last_undegraded":"2020-08-07T15:12:41.461587+0000","last_fullsized":"2020-08-07T15:12:41.461587+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_clean_scrub_stamp":"2020-08-07T15:12:14.121397+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"252'2760","reported_seq":"2790","reported_epoch":"252","state":"active+clean","last_fresh":"2020-08-07T15:12:39.838300+0000","last_change":"2020-08-07T15:12:10.398415+0000","last_active":"2020-08-07T15:12:39.838300+0000","last_peered":"2020-08-07T15:12:39.838300+0000","last_clean":"2020-08-07T15:12:39.838300+0000","last_became_active":"2020-08-07T15:12:10.397834+0000","last_became_peered":"2020-08-07T15:12:10.397834+0000","last_unstale":"2020-08-07T15:12:39.838300+0000","last_undegraded":"2020-08-07T15:12:39.838300+0000","last_fullsized":"2020-08-07T15:12:39.838300+0000","mapping_epoch":230,"log_start":"252'2752","ondisk_log_start":"252'2752","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22609920,"num_objects":345,"num_object_clones":0,"num_object_copies":1035,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2760,"num_write_kb":22080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"269","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.435427+0000","last_change":"2020-08-07T15:12:08.622779+0000","last_active":"2020-08-07T15:12:40.435427+0000","last_peered":"2020-08-07T15:12:40.435427+0000","last_clean":"2020-08-07T15:12:40.435427+0000","last_became_active":"2020-08-07T15:12:06.367457+0000","last_became_peered":"2020-08-07T15:12:06.367457+0000","last_unstale":"2020-08-07T15:12:40.435427+0000","last_undegraded":"2020-08-07T15:12:40.435427+0000","last_fullsized":"2020-08-07T15:12:40.435427+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:08.622683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:08.622683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"286","reported_epoch":"255","state":"active+clean","last_fresh":"2020-08-07T15:12:42.371230+0000","last_change":"2020-08-07T15:12:42.371309+0000","last_active":"2020-08-07T15:12:42.371230+0000","last_peered":"2020-08-07T15:12:42.371230+0000","last_clean":"2020-08-07T15:12:42.371230+0000","last_became_active":"2020-08-07T15:12:32.237984+0000","last_became_peered":"2020-08-07T15:12:32.237984+0000","last_unstale":"2020-08-07T15:12:42.371230+0000","last_undegraded":"2020-08-07T15:12:42.371230+0000","last_fullsized":"2020-08-07T15:12:42.371230+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:42.371229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:42.371229+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.7","version":"254'2896","reported_seq":"2928","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.695741+0000","last_change":"2020-08-07T15:12:10.464810+0000","last_active":"2020-08-07T15:12:41.695741+0000","last_peered":"2020-08-07T15:12:41.695741+0000","last_clean":"2020-08-07T15:12:41.695741+0000","last_became_active":"2020-08-07T15:12:10.464050+0000","last_became_peered":"2020-08-07T15:12:10.464050+0000","last_unstale":"2020-08-07T15:12:41.695741+0000","last_undegraded":"2020-08-07T15:12:41.695741+0000","last_fullsized":"2020-08-07T15:12:41.695741+0000","mapping_epoch":230,"log_start":"254'2888","ondisk_log_start":"254'2888","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23724032,"num_objects":362,"num_object_clones":0,"num_object_copies":1086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2896,"num_write_kb":23168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"298","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.435410+0000","last_change":"2020-08-07T15:12:39.812944+0000","last_active":"2020-08-07T15:12:40.435410+0000","last_peered":"2020-08-07T15:12:40.435410+0000","last_clean":"2020-08-07T15:12:40.435410+0000","last_became_active":"2020-08-07T15:12:32.255633+0000","last_became_peered":"2020-08-07T15:12:32.255633+0000","last_unstale":"2020-08-07T15:12:40.435410+0000","last_undegraded":"2020-08-07T15:12:40.435410+0000","last_fullsized":"2020-08-07T15:12:40.435410+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:39.812876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:39.812876+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"254","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.434109+0000","last_change":"2020-08-07T15:11:40.328534+0000","last_active":"2020-08-07T15:12:40.434109+0000","last_peered":"2020-08-07T15:12:40.434109+0000","last_clean":"2020-08-07T15:12:40.434109+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:12:40.434109+0000","last_undegraded":"2020-08-07T15:12:40.434109+0000","last_fullsized":"2020-08-07T15:12:40.434109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:11:40.328482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:11:40.328482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"253'2832","reported_seq":"2864","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.507395+0000","last_change":"2020-08-07T15:12:10.464478+0000","last_active":"2020-08-07T15:12:41.507395+0000","last_peered":"2020-08-07T15:12:41.507395+0000","last_clean":"2020-08-07T15:12:41.507395+0000","last_became_active":"2020-08-07T15:12:10.463997+0000","last_became_peered":"2020-08-07T15:12:10.463997+0000","last_unstale":"2020-08-07T15:12:41.507395+0000","last_undegraded":"2020-08-07T15:12:41.507395+0000","last_fullsized":"2020-08-07T15:12:41.507395+0000","mapping_epoch":230,"log_start":"253'2824","ondisk_log_start":"253'2824","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23199744,"num_objects":354,"num_object_clones":0,"num_object_copies":1062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2832,"num_write_kb":22656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"275","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.441399+0000","last_change":"2020-08-07T15:12:10.585667+0000","last_active":"2020-08-07T15:12:40.441399+0000","last_peered":"2020-08-07T15:12:40.441399+0000","last_clean":"2020-08-07T15:12:40.441399+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:12:40.441399+0000","last_undegraded":"2020-08-07T15:12:40.441399+0000","last_fullsized":"2020-08-07T15:12:40.441399+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:10.585580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:10.585580+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"314","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.435608+0000","last_change":"2020-08-07T15:12:38.850878+0000","last_active":"2020-08-07T15:12:40.435608+0000","last_peered":"2020-08-07T15:12:40.435608+0000","last_clean":"2020-08-07T15:12:40.435608+0000","last_became_active":"2020-08-07T15:12:32.239794+0000","last_became_peered":"2020-08-07T15:12:32.239794+0000","last_unstale":"2020-08-07T15:12:40.435608+0000","last_undegraded":"2020-08-07T15:12:40.435608+0000","last_fullsized":"2020-08-07T15:12:40.435608+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:38.850809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:38.850809+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"254'3000","reported_seq":"3031","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.676648+0000","last_change":"2020-08-07T15:12:10.465113+0000","last_active":"2020-08-07T15:12:41.676648+0000","last_peered":"2020-08-07T15:12:41.676648+0000","last_clean":"2020-08-07T15:12:41.676648+0000","last_became_active":"2020-08-07T15:12:10.464582+0000","last_became_peered":"2020-08-07T15:12:10.464582+0000","last_unstale":"2020-08-07T15:12:41.676648+0000","last_undegraded":"2020-08-07T15:12:41.676648+0000","last_fullsized":"2020-08-07T15:12:41.676648+0000","mapping_epoch":230,"log_start":"254'2989","ondisk_log_start":"254'2989","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24576000,"num_objects":375,"num_object_clones":0,"num_object_copies":1125,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3000,"num_write_kb":24000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"269","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.434223+0000","last_change":"2020-08-07T15:12:07.340958+0000","last_active":"2020-08-07T15:12:40.434223+0000","last_peered":"2020-08-07T15:12:40.434223+0000","last_clean":"2020-08-07T15:12:40.434223+0000","last_became_active":"2020-08-07T15:12:06.366967+0000","last_became_peered":"2020-08-07T15:12:06.366967+0000","last_unstale":"2020-08-07T15:12:40.434223+0000","last_undegraded":"2020-08-07T15:12:40.434223+0000","last_fullsized":"2020-08-07T15:12:40.434223+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:07.340855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.340855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"261","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.433827+0000","last_change":"2020-08-07T15:12:32.247332+0000","last_active":"2020-08-07T15:12:40.433827+0000","last_peered":"2020-08-07T15:12:40.433827+0000","last_clean":"2020-08-07T15:12:40.433827+0000","last_became_active":"2020-08-07T15:12:32.246800+0000","last_became_peered":"2020-08-07T15:12:32.246800+0000","last_unstale":"2020-08-07T15:12:40.433827+0000","last_undegraded":"2020-08-07T15:12:40.433827+0000","last_fullsized":"2020-08-07T15:12:40.433827+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:09.261101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.261101+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"254'2760","reported_seq":"2791","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.746878+0000","last_change":"2020-08-07T15:12:10.466261+0000","last_active":"2020-08-07T15:12:41.746878+0000","last_peered":"2020-08-07T15:12:41.746878+0000","last_clean":"2020-08-07T15:12:41.746878+0000","last_became_active":"2020-08-07T15:12:10.465583+0000","last_became_peered":"2020-08-07T15:12:10.465583+0000","last_unstale":"2020-08-07T15:12:41.746878+0000","last_undegraded":"2020-08-07T15:12:41.746878+0000","last_fullsized":"2020-08-07T15:12:41.746878+0000","mapping_epoch":230,"log_start":"254'2755","ondisk_log_start":"254'2755","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22609920,"num_objects":345,"num_object_clones":0,"num_object_copies":1035,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2760,"num_write_kb":22080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"266","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.435731+0000","last_change":"2020-08-07T15:12:07.583222+0000","last_active":"2020-08-07T15:12:40.435731+0000","last_peered":"2020-08-07T15:12:40.435731+0000","last_clean":"2020-08-07T15:12:40.435731+0000","last_became_active":"2020-08-07T15:12:06.366915+0000","last_became_peered":"2020-08-07T15:12:06.366915+0000","last_unstale":"2020-08-07T15:12:40.435731+0000","last_undegraded":"2020-08-07T15:12:40.435731+0000","last_fullsized":"2020-08-07T15:12:40.435731+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:07.583152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.583152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"348","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.461807+0000","last_change":"2020-08-07T15:12:32.255274+0000","last_active":"2020-08-07T15:12:41.461807+0000","last_peered":"2020-08-07T15:12:41.461807+0000","last_clean":"2020-08-07T15:12:41.461807+0000","last_became_active":"2020-08-07T15:12:32.254842+0000","last_became_peered":"2020-08-07T15:12:32.254842+0000","last_unstale":"2020-08-07T15:12:41.461807+0000","last_undegraded":"2020-08-07T15:12:41.461807+0000","last_fullsized":"2020-08-07T15:12:41.461807+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:15.163639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:15.163639+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"254'2744","reported_seq":"2773","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.746173+0000","last_change":"2020-08-07T15:12:10.467447+0000","last_active":"2020-08-07T15:12:41.746173+0000","last_peered":"2020-08-07T15:12:41.746173+0000","last_clean":"2020-08-07T15:12:41.746173+0000","last_became_active":"2020-08-07T15:12:10.467000+0000","last_became_peered":"2020-08-07T15:12:10.467000+0000","last_unstale":"2020-08-07T15:12:41.746173+0000","last_undegraded":"2020-08-07T15:12:41.746173+0000","last_fullsized":"2020-08-07T15:12:41.746173+0000","mapping_epoch":230,"log_start":"254'2737","ondisk_log_start":"254'2737","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22478848,"num_objects":343,"num_object_clones":0,"num_object_copies":1029,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2744,"num_write_kb":21952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"238","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.434008+0000","last_change":"2020-08-07T15:10:51.644951+0000","last_active":"2020-08-07T15:12:40.434008+0000","last_peered":"2020-08-07T15:12:40.434008+0000","last_clean":"2020-08-07T15:12:40.434008+0000","last_became_active":"2020-08-07T15:10:46.498230+0000","last_became_peered":"2020-08-07T15:10:46.498230+0000","last_unstale":"2020-08-07T15:12:40.434008+0000","last_undegraded":"2020-08-07T15:12:40.434008+0000","last_fullsized":"2020-08-07T15:12:40.434008+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:51.644875+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:10:51.644875+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"254'2880","reported_seq":"2912","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.647808+0000","last_change":"2020-08-07T15:12:10.465075+0000","last_active":"2020-08-07T15:12:41.647808+0000","last_peered":"2020-08-07T15:12:41.647808+0000","last_clean":"2020-08-07T15:12:41.647808+0000","last_became_active":"2020-08-07T15:12:10.464640+0000","last_became_peered":"2020-08-07T15:12:10.464640+0000","last_unstale":"2020-08-07T15:12:41.647808+0000","last_undegraded":"2020-08-07T15:12:41.647808+0000","last_fullsized":"2020-08-07T15:12:41.647808+0000","mapping_epoch":230,"log_start":"254'2873","ondisk_log_start":"254'2873","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23592960,"num_objects":360,"num_object_clones":0,"num_object_copies":1080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2880,"num_write_kb":23040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"82","reported_epoch":"248","state":"active+clean","last_fresh":"2020-08-07T15:12:35.391961+0000","last_change":"2020-08-07T15:12:32.239634+0000","last_active":"2020-08-07T15:12:35.391961+0000","last_peered":"2020-08-07T15:12:35.391961+0000","last_clean":"2020-08-07T15:12:35.391961+0000","last_became_active":"2020-08-07T15:12:32.238859+0000","last_became_peered":"2020-08-07T15:12:32.238859+0000","last_unstale":"2020-08-07T15:12:35.391961+0000","last_undegraded":"2020-08-07T15:12:35.391961+0000","last_fullsized":"2020-08-07T15:12:35.391961+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.490655+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.490655+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"252'2616","reported_seq":"2646","reported_epoch":"252","state":"active+clean","last_fresh":"2020-08-07T15:12:39.637109+0000","last_change":"2020-08-07T15:12:10.465359+0000","last_active":"2020-08-07T15:12:39.637109+0000","last_peered":"2020-08-07T15:12:39.637109+0000","last_clean":"2020-08-07T15:12:39.637109+0000","last_became_active":"2020-08-07T15:12:10.464900+0000","last_became_peered":"2020-08-07T15:12:10.464900+0000","last_unstale":"2020-08-07T15:12:39.637109+0000","last_undegraded":"2020-08-07T15:12:39.637109+0000","last_fullsized":"2020-08-07T15:12:39.637109+0000","mapping_epoch":230,"log_start":"252'2612","ondisk_log_start":"252'2612","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21430272,"num_objects":327,"num_object_clones":0,"num_object_copies":981,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2616,"num_write_kb":20928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"305","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.435384+0000","last_change":"2020-08-07T15:12:23.886345+0000","last_active":"2020-08-07T15:12:40.435384+0000","last_peered":"2020-08-07T15:12:40.435384+0000","last_clean":"2020-08-07T15:12:40.435384+0000","last_became_active":"2020-08-07T15:10:46.499635+0000","last_became_peered":"2020-08-07T15:10:46.499635+0000","last_unstale":"2020-08-07T15:12:40.435384+0000","last_undegraded":"2020-08-07T15:12:40.435384+0000","last_fullsized":"2020-08-07T15:12:40.435384+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:23.886261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:23.886261+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"254'2992","reported_seq":"3024","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.691321+0000","last_change":"2020-08-07T15:12:10.468110+0000","last_active":"2020-08-07T15:12:41.691321+0000","last_peered":"2020-08-07T15:12:41.691321+0000","last_clean":"2020-08-07T15:12:41.691321+0000","last_became_active":"2020-08-07T15:12:10.467697+0000","last_became_peered":"2020-08-07T15:12:10.467697+0000","last_unstale":"2020-08-07T15:12:41.691321+0000","last_undegraded":"2020-08-07T15:12:41.691321+0000","last_fullsized":"2020-08-07T15:12:41.691321+0000","mapping_epoch":230,"log_start":"254'2985","ondisk_log_start":"254'2985","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24510464,"num_objects":374,"num_object_clones":0,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2992,"num_write_kb":23936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"272","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.435789+0000","last_change":"2020-08-07T15:12:13.680022+0000","last_active":"2020-08-07T15:12:40.435789+0000","last_peered":"2020-08-07T15:12:40.435789+0000","last_clean":"2020-08-07T15:12:40.435789+0000","last_became_active":"2020-08-07T15:12:06.380392+0000","last_became_peered":"2020-08-07T15:12:06.380392+0000","last_unstale":"2020-08-07T15:12:40.435789+0000","last_undegraded":"2020-08-07T15:12:40.435789+0000","last_fullsized":"2020-08-07T15:12:40.435789+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:13.679943+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.679943+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"254'2888","reported_seq":"2919","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.665237+0000","last_change":"2020-08-07T15:12:10.398816+0000","last_active":"2020-08-07T15:12:41.665237+0000","last_peered":"2020-08-07T15:12:41.665237+0000","last_clean":"2020-08-07T15:12:41.665237+0000","last_became_active":"2020-08-07T15:12:10.398332+0000","last_became_peered":"2020-08-07T15:12:10.398332+0000","last_unstale":"2020-08-07T15:12:41.665237+0000","last_undegraded":"2020-08-07T15:12:41.665237+0000","last_fullsized":"2020-08-07T15:12:41.665237+0000","mapping_epoch":230,"log_start":"254'2883","ondisk_log_start":"254'2883","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":23658496,"num_objects":361,"num_object_clones":0,"num_object_copies":1083,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2888,"num_write_kb":23104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"227","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.439886+0000","last_change":"2020-08-07T15:12:14.241339+0000","last_active":"2020-08-07T15:12:40.439886+0000","last_peered":"2020-08-07T15:12:40.439886+0000","last_clean":"2020-08-07T15:12:40.439886+0000","last_became_active":"2020-08-07T15:10:46.498051+0000","last_became_peered":"2020-08-07T15:10:46.498051+0000","last_unstale":"2020-08-07T15:12:40.439886+0000","last_undegraded":"2020-08-07T15:12:40.439886+0000","last_fullsized":"2020-08-07T15:12:40.439886+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:14.241262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:14.241262+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"254'2936","reported_seq":"2966","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.691717+0000","last_change":"2020-08-07T15:12:10.398015+0000","last_active":"2020-08-07T15:12:41.691717+0000","last_peered":"2020-08-07T15:12:41.691717+0000","last_clean":"2020-08-07T15:12:41.691717+0000","last_became_active":"2020-08-07T15:12:10.397610+0000","last_became_peered":"2020-08-07T15:12:10.397610+0000","last_unstale":"2020-08-07T15:12:41.691717+0000","last_undegraded":"2020-08-07T15:12:41.691717+0000","last_fullsized":"2020-08-07T15:12:41.691717+0000","mapping_epoch":230,"log_start":"254'2928","ondisk_log_start":"254'2928","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24051712,"num_objects":367,"num_object_clones":0,"num_object_copies":1101,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2936,"num_write_kb":23488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"304","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.461703+0000","last_change":"2020-08-07T15:12:41.445456+0000","last_active":"2020-08-07T15:12:41.461703+0000","last_peered":"2020-08-07T15:12:41.461703+0000","last_clean":"2020-08-07T15:12:41.461703+0000","last_became_active":"2020-08-07T15:12:32.241722+0000","last_became_peered":"2020-08-07T15:12:32.241722+0000","last_unstale":"2020-08-07T15:12:41.461703+0000","last_undegraded":"2020-08-07T15:12:41.461703+0000","last_fullsized":"2020-08-07T15:12:41.461703+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:41.445379+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_clean_scrub_stamp":"2020-08-07T15:12:41.445379+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"254'2664","reported_seq":"2696","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.618126+0000","last_change":"2020-08-07T15:12:10.468495+0000","last_active":"2020-08-07T15:12:41.618126+0000","last_peered":"2020-08-07T15:12:41.618126+0000","last_clean":"2020-08-07T15:12:41.618126+0000","last_became_active":"2020-08-07T15:12:10.467915+0000","last_became_peered":"2020-08-07T15:12:10.467915+0000","last_unstale":"2020-08-07T15:12:41.618126+0000","last_undegraded":"2020-08-07T15:12:41.618126+0000","last_fullsized":"2020-08-07T15:12:41.618126+0000","mapping_epoch":230,"log_start":"254'2656","ondisk_log_start":"254'2656","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21823488,"num_objects":333,"num_object_clones":0,"num_object_copies":999,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2664,"num_write_kb":21312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,2],"acting":[7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"191","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.435718+0000","last_change":"2020-08-07T15:12:30.942002+0000","last_active":"2020-08-07T15:12:40.435718+0000","last_peered":"2020-08-07T15:12:40.435718+0000","last_clean":"2020-08-07T15:12:40.435718+0000","last_became_active":"2020-08-07T15:10:46.493498+0000","last_became_peered":"2020-08-07T15:10:46.493498+0000","last_unstale":"2020-08-07T15:12:40.435718+0000","last_undegraded":"2020-08-07T15:12:40.435718+0000","last_fullsized":"2020-08-07T15:12:40.435718+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:30.941943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:30.941943+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"254'2992","reported_seq":"3024","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.758625+0000","last_change":"2020-08-07T15:12:10.465251+0000","last_active":"2020-08-07T15:12:41.758625+0000","last_peered":"2020-08-07T15:12:41.758625+0000","last_clean":"2020-08-07T15:12:41.758625+0000","last_became_active":"2020-08-07T15:12:10.464698+0000","last_became_peered":"2020-08-07T15:12:10.464698+0000","last_unstale":"2020-08-07T15:12:41.758625+0000","last_undegraded":"2020-08-07T15:12:41.758625+0000","last_fullsized":"2020-08-07T15:12:41.758625+0000","mapping_epoch":230,"log_start":"254'2988","ondisk_log_start":"254'2988","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24510464,"num_objects":374,"num_object_clones":0,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2992,"num_write_kb":23936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"222","reported_epoch":"253","state":"active+clean","last_fresh":"2020-08-07T15:12:40.434066+0000","last_change":"2020-08-07T15:12:06.367666+0000","last_active":"2020-08-07T15:12:40.434066+0000","last_peered":"2020-08-07T15:12:40.434066+0000","last_clean":"2020-08-07T15:12:40.434066+0000","last_became_active":"2020-08-07T15:12:06.367081+0000","last_became_peered":"2020-08-07T15:12:06.367081+0000","last_unstale":"2020-08-07T15:12:40.434066+0000","last_undegraded":"2020-08-07T15:12:40.434066+0000","last_fullsized":"2020-08-07T15:12:40.434066+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:37.601099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:10:37.601099+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":369287168,"num_objects":5635,"num_object_clones":0,"num_object_copies":16905,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45079,"num_write_kb":360632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1117216768,"data_stored":1117216768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":41,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":123,"acting":123,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542223,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1166244,"kb_used_data":117660,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93205596,"statfs":{"total":96636764160,"available":95442530304,"internally_reserved":1073741824,"allocated":120483840,"data_stored":119236527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":9.5359999999999996,"15min":3.774},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":2.294,"5min":856.07100000000003,"15min":856.07100000000003},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.77700000000000002,"5min":1.002,"15min":0.97599999999999998},"min":{"1min":0.52300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.55,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":1.5209999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.335,"15min":1.3080000000000001},"min":{"1min":0.64200000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.9199999999999999,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.141,"15min":1.024},"min":{"1min":0.61799999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":4.0060000000000002,"5min":4.0060000000000002,"15min":4.0060000000000002},"last":0.76800000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.321,"15min":1.1910000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.5630000000000002,"5min":5.6619999999999999,"15min":5.6619999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.081,"5min":1.3280000000000001,"15min":1.1879999999999999},"min":{"1min":0.69299999999999995,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.3769999999999998,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":0.85799999999999998}]},{"osd":3,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":30.957999999999998,"5min":21.91,"15min":8.4179999999999993},"min":{"1min":0.68899999999999995,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":656.99099999999999,"5min":917.38699999999994,"15min":917.38699999999994},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.173,"5min":4.8879999999999999,"15min":2.524},"min":{"1min":0.61599999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":4.0460000000000003,"5min":333.59199999999998,"15min":333.59199999999998},"last":2.383}]},{"osd":4,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":8.8599999999999994,"5min":2.944,"15min":1.8320000000000001},"min":{"1min":0.66000000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":172.012,"5min":172.012,"15min":172.012},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.5509999999999999,"5min":11.529,"15min":4.7169999999999996},"min":{"1min":0.81200000000000006,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":5.4500000000000002,"5min":866.24099999999999,"15min":866.24099999999999},"last":1.5569999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":9.3930000000000007,"15min":28.135000000000002},"min":{"1min":0.68400000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":3.536,"5min":678.76099999999997,"15min":831.88099999999997},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.623,"5min":1.831,"15min":1.607},"min":{"1min":0.89000000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":3.593,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":1.321}]},{"osd":7,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":3.9929999999999999,"15min":2.343},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":4.0810000000000004,"5min":224.14500000000001,"15min":224.14500000000001},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.671,"5min":1.7649999999999999,"15min":1.593},"min":{"1min":0.92000000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":3.6200000000000001,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":0.91900000000000004}]}]},{"osd":7,"up_from":13,"seq":55834574928,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1181924,"kb_used_data":133340,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93189916,"statfs":{"total":96636764160,"available":95426473984,"internally_reserved":1073741824,"allocated":136540160,"data_stored":135307876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":37.372999999999998,"5min":16.672000000000001,"15min":6.4459999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":763.39099999999996,"5min":763.39099999999996,"15min":763.39099999999996},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.157,"5min":9.7789999999999999,"15min":4.0259999999999998},"min":{"1min":0.61699999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.8279999999999998,"5min":529.88699999999994,"15min":529.88699999999994},"last":1.095}]},{"osd":1,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":5.758,"15min":2.5579999999999998},"min":{"1min":0.47699999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.7290000000000001,"5min":547.65300000000002,"15min":547.65300000000002},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.4159999999999999,"5min":4.9189999999999996,"15min":2.5640000000000001},"min":{"1min":0.57399999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":5.4960000000000004,"5min":372.50799999999998,"15min":372.50799999999998},"last":1.119}]},{"osd":2,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":3.0009999999999999,"5min":1.6850000000000001,"15min":1.3580000000000001},"min":{"1min":0.52000000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":33.131999999999998,"5min":33.131999999999998,"15min":33.131999999999998},"last":1.4890000000000001},{"interface":"front","average":{"1min":2.8410000000000002,"5min":1.494,"15min":1.3460000000000001},"min":{"1min":0.61799999999999999,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":32.924999999999997,"5min":32.924999999999997,"15min":32.924999999999997},"last":1.1200000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":9.4659999999999993,"15min":4.1719999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":5.9329999999999998,"5min":698.19899999999996,"15min":698.19899999999996},"last":3.1480000000000001},{"interface":"front","average":{"1min":1.238,"5min":1.2669999999999999,"15min":1.347},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":3.8599999999999999,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":1.155}]},{"osd":4,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":25.448,"5min":6.069,"15min":3.1309999999999998},"min":{"1min":0.76400000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":460.78399999999999,"5min":460.78399999999999,"15min":460.78399999999999},"last":1.544},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.474,"15min":1.6060000000000001},"min":{"1min":0.86899999999999999,"5min":0.621,"15min":0.621},"max":{"1min":2.4079999999999999,"5min":3.694,"15min":5.2990000000000004},"last":1.577}]},{"osd":5,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.3720000000000001,"15min":1.3660000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":5.9989999999999997,"5min":6.2080000000000002,"15min":6.2080000000000002},"last":1.6659999999999999},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.492,"15min":1.6739999999999999},"min":{"1min":0.83599999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":4.407,"5min":4.407,"15min":5.327},"last":1.6160000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":9.1839999999999993,"15min":4.016},"min":{"1min":0.88,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.3290000000000002,"5min":695.92899999999997,"15min":695.92899999999997},"last":1.6890000000000001},{"interface":"front","average":{"1min":1.597,"5min":1.55,"15min":12.734},"min":{"1min":0.89500000000000002,"5min":0.87,"15min":0.87},"max":{"1min":3.302,"5min":3.7309999999999999,"15min":266.26999999999998},"last":1.7070000000000001}]}]},{"osd":1,"up_from":227,"seq":974957576200,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1156284,"kb_used_data":107700,"kb_used_omap":2,"kb_used_meta":1048573,"kb_avail":93215556,"statfs":{"total":96636764160,"available":95452729344,"internally_reserved":1073741824,"allocated":110284800,"data_stored":109063607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":1073739245},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3380000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7229999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.548}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.609}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.978}]}]},{"osd":0,"up_from":13,"seq":55834574928,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1137572,"kb_used_data":88988,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93234268,"statfs":{"total":96636764160,"available":95471890432,"internally_reserved":1073741824,"allocated":91123712,"data_stored":89891428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":24.291,"5min":21.719999999999999,"15min":7.891},"min":{"1min":0.52700000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":432.899,"5min":845.57100000000003,"15min":845.57100000000003},"last":1.9199999999999999},{"interface":"front","average":{"1min":1.752,"5min":1.181,"15min":1.0149999999999999},"min":{"1min":0.62,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":9.3650000000000002,"5min":9.3650000000000002,"15min":9.3650000000000002},"last":1.4099999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.0620000000000001,"15min":0.97199999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":7.3470000000000004,"5min":7.3470000000000004,"15min":7.3470000000000004},"last":1.8939999999999999},{"interface":"front","average":{"1min":6.2839999999999998,"5min":4.665,"15min":23.381},"min":{"1min":0.47399999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":75.622,"5min":226.911,"15min":606.18499999999995},"last":2.593}]},{"osd":3,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":28.079000000000001,"5min":6.6920000000000002,"15min":2.8540000000000001},"min":{"1min":0.60999999999999999,"5min":0.54100000000000004,"15min":0.52300000000000002},"max":{"1min":431.95299999999997,"5min":431.95299999999997,"15min":431.95299999999997},"last":1.109},{"interface":"front","average":{"1min":1.425,"5min":1.46,"15min":1.2649999999999999},"min":{"1min":0.66400000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":5.3780000000000001,"5min":11.19,"15min":11.19},"last":2.2360000000000002}]},{"osd":4,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.302,"15min":1.2110000000000001},"min":{"1min":0.57399999999999995,"5min":0.501,"15min":0.501},"max":{"1min":1.819,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.448,"15min":1.321},"min":{"1min":0.71099999999999997,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.8849999999999998,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":2.1800000000000002}]},{"osd":5,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.6200000000000001,"5min":14.598000000000001,"15min":5.6840000000000002},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":7.3799999999999999,"5min":539.31100000000004,"15min":539.31100000000004},"last":1.7969999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.49,"15min":1.234},"min":{"1min":0.81399999999999995,"5min":0.752,"15min":0.65900000000000003},"max":{"1min":2.9420000000000002,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":1.3740000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:12:34 2020","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.4399999999999999,"15min":6.2220000000000004},"min":{"1min":0.80200000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.8610000000000002,"5min":4.1310000000000002,"15min":124.738},"last":2.21},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.5549999999999999,"15min":12.952999999999999},"min":{"1min":0.73499999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":5.274,"5min":5.274,"15min":294.94},"last":1.831}]},{"osd":7,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.4790000000000001,"15min":1.55},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.79300000000000004},"max":{"1min":1.8620000000000001,"5min":4.7169999999999996,"15min":6.468},"last":1.8560000000000001},{"interface":"front","average":{"1min":66.063000000000002,"5min":23.518000000000001,"15min":8.7629999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":971.12699999999995,"5min":971.12699999999995,"15min":971.12699999999995},"last":1.8759999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1227200,"kb_used_data":178616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93144640,"statfs":{"total":96636764160,"available":95380111360,"internally_reserved":1073741824,"allocated":182902784,"data_stored":181662963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":32.965000000000003,"5min":7.8639999999999999,"15min":3.3660000000000001},"min":{"1min":0.41899999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":501.53800000000001,"5min":501.53800000000001,"15min":501.53800000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":27.341999999999999,"5min":6.6900000000000004,"15min":2.8799999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":501.36000000000001,"5min":501.36000000000001,"15min":501.36000000000001},"last":0.88600000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:10:21 2020","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.6970000000000001,"15min":1.2929999999999999},"min":{"1min":0.52000000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":5.5919999999999996,"5min":18.672000000000001,"15min":18.672000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.264,"5min":1.3320000000000001,"15min":1.3500000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":3.7970000000000002,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":1.637}]},{"osd":3,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.2130000000000001,"15min":1.179},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":3.0760000000000001},{"interface":"front","average":{"1min":23.064,"5min":15.281000000000001,"15min":6.1310000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":412.61500000000001,"5min":501.75599999999997,"15min":501.75599999999997},"last":1.3480000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.5940000000000001,"15min":1.2709999999999999},"min":{"1min":0.67400000000000004,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.712,"5min":19.783000000000001,"15min":19.783000000000001},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.4610000000000001,"15min":1.5820000000000001},"min":{"1min":0.68999999999999995,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8979999999999999,"5min":8.048,"15min":8.048},"last":1.6120000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.3460000000000001,"15min":1.29},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.4009999999999998,"5min":5.2220000000000004,"15min":5.2220000000000004},"last":1.79},{"interface":"front","average":{"1min":1.4990000000000001,"5min":15.715,"15min":22.254999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":8.0649999999999995,"5min":627.53300000000002,"15min":627.53300000000002},"last":0.85899999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:12:37 2020","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":9.9480000000000004,"15min":4.1299999999999999},"min":{"1min":0.875,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.9450000000000001,"5min":821.91899999999998,"15min":821.91899999999998},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.3959999999999999,"5min":9.9580000000000002,"15min":4.2000000000000002},"min":{"1min":0.70199999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.9540000000000002,"5min":751.70000000000005,"15min":751.70000000000005},"last":1.76}]},{"osd":7,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":12.522,"15min":5.335},"min":{"1min":0.85599999999999998,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.3519999999999999,"5min":501.91699999999997,"15min":501.91699999999997},"last":1.54},{"interface":"front","average":{"1min":1.7230000000000001,"5min":2.1080000000000001,"15min":1.7},"min":{"1min":0.88500000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":7.2149999999999999,"5min":35.646999999999998,"15min":35.646999999999998},"last":1.5620000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574928,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1278236,"kb_used_data":229652,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93093604,"statfs":{"total":96636764160,"available":95327850496,"internally_reserved":1073741824,"allocated":235163648,"data_stored":233931364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.226,"15min":1.244},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":4.6580000000000004,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":1.524},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.2050000000000001,"15min":1.4930000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":4.6349999999999998,"5min":4.9119999999999999,"15min":13.067},"last":0.93799999999999994}]},{"osd":1,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":4.4779999999999998,"15min":2.758},"min":{"1min":0.59099999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.847,"5min":270.68700000000001,"15min":270.68700000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.109,"5min":1.8580000000000001,"15min":1.96},"min":{"1min":0.54100000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.0550000000000002,"5min":14.349,"15min":14.349},"last":0.97299999999999998}]},{"osd":2,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.365,"15min":1.22},"min":{"1min":0.65900000000000003,"5min":0.56999999999999995,"15min":0.56899999999999995},"max":{"1min":2.8300000000000001,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":1.6679999999999999},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.9690000000000001,"15min":1.7789999999999999},"min":{"1min":0.64500000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":5.3090000000000002,"5min":51.530000000000001,"15min":51.530000000000001},"last":1.409}]},{"osd":4,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":3.831,"5min":17.792999999999999,"15min":7.0620000000000003},"min":{"1min":0.54500000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":56.688000000000002,"5min":646.26700000000005,"15min":646.26700000000005},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.421,"5min":15.494999999999999,"15min":26.565000000000001},"min":{"1min":0.68100000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":3.2109999999999999,"5min":844.10500000000002,"15min":844.10500000000002},"last":1.5760000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":14.452999999999999,"5min":8.5429999999999993,"15min":3.859},"min":{"1min":0.95499999999999996,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":244.08099999999999,"5min":403.88400000000001,"15min":403.88400000000001},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.78,"5min":1.8979999999999999,"15min":1.8120000000000001},"min":{"1min":0.98499999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":4.5960000000000001,"5min":8.9120000000000008,"15min":8.9120000000000008},"last":1.6419999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.7969999999999999,"5min":1.706,"15min":1.5900000000000001},"min":{"1min":0.74099999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":5.2729999999999997,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":1.95},{"interface":"front","average":{"1min":1.907,"5min":13.411,"15min":5.7949999999999999},"min":{"1min":1.032,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":5.3460000000000001,"5min":688.15999999999997,"15min":688.15999999999997},"last":1.976}]},{"osd":7,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.7450000000000001,"5min":1.5960000000000001,"15min":1.9730000000000001},"min":{"1min":0.85599999999999998,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":4.8099999999999996,"5min":6.8209999999999997,"15min":6.8209999999999997},"last":1.296},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.6559999999999999,"15min":1.8700000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.6099999999999999,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":1.5409999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574927,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1165464,"kb_used_data":116880,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93206376,"statfs":{"total":96636764160,"available":95443329024,"internally_reserved":1073741824,"allocated":119685120,"data_stored":118445299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":49.795000000000002,"5min":19.646000000000001,"15min":7.3360000000000003},"min":{"1min":0.46800000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":829.15099999999995,"5min":829.15099999999995,"15min":829.15099999999995},"last":0.628},{"interface":"front","average":{"1min":3.8149999999999999,"5min":1.639,"15min":1.1120000000000001},"min":{"1min":0.45300000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":48.274000000000001,"5min":48.274000000000001,"15min":48.274000000000001},"last":0.81100000000000005}]},{"osd":1,"last update":"Fri Aug 7 15:10:24 2020","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":5.7629999999999999,"15min":2.823},"min":{"1min":0.52900000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.8109999999999999,"5min":320.67000000000002,"15min":320.67000000000002},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.177,"5min":7.5300000000000002,"15min":3.4279999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.7200000000000002,"5min":689.846,"15min":689.846},"last":0.76500000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":4.1159999999999997,"5min":1.901,"15min":1.601},"min":{"1min":0.64300000000000002,"5min":0.56399999999999995,"15min":0.52100000000000002},"max":{"1min":49.878999999999998,"5min":49.878999999999998,"15min":49.878999999999998},"last":1.1060000000000001},{"interface":"front","average":{"1min":4.2270000000000003,"5min":2.0529999999999999,"15min":1.694},"min":{"1min":0.97899999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":49.939999999999998,"5min":49.939999999999998,"15min":49.939999999999998},"last":0.51700000000000002}]},{"osd":3,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":27.59,"5min":17.260999999999999,"15min":6.7590000000000003},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":426.50799999999998,"5min":638.92100000000005,"15min":638.92100000000005},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.411,"5min":1.3799999999999999,"15min":1.294},"min":{"1min":0.75700000000000001,"5min":0.67800000000000005,"15min":0.66700000000000004},"max":{"1min":3.2360000000000002,"5min":6.431,"15min":6.431},"last":0.92000000000000004}]},{"osd":5,"last update":"Fri Aug 7 15:12:29 2020","interfaces":[{"interface":"back","average":{"1min":52.308999999999997,"5min":28.574999999999999,"15min":10.609999999999999},"min":{"1min":0.56100000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":862.97299999999996,"5min":990.71600000000001,"15min":990.71600000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":49.549999999999997,"5min":11.01,"15min":4.7539999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":780.00699999999995,"5min":780.00699999999995,"15min":780.00699999999995},"last":0.66200000000000003}]},{"osd":6,"last update":"Fri Aug 7 15:12:34 2020","interfaces":[{"interface":"back","average":{"1min":3.8239999999999998,"5min":1.9870000000000001,"15min":1.855},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":48.575000000000003,"5min":48.575000000000003,"15min":48.575000000000003},"last":0.878},{"interface":"front","average":{"1min":1.109,"5min":1.333,"15min":13.183},"min":{"1min":0.64600000000000002,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.847,"5min":6.7729999999999997,"15min":389.14600000000002},"last":0.85699999999999998}]},{"osd":7,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":4.1349999999999998,"5min":4.327,"15min":2.3809999999999998},"min":{"1min":0.74199999999999999,"5min":0.62,"15min":0.62},"max":{"1min":48.75,"5min":252.798,"15min":252.798},"last":1.1639999999999999},{"interface":"front","average":{"1min":4.2610000000000001,"5min":14.635,"15min":5.9749999999999996},"min":{"1min":0.84799999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":49.654000000000003,"5min":501.93000000000001,"15min":501.93000000000001},"last":1.077}]}]},{"osd":5,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1165044,"kb_used_data":116460,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93206796,"statfs":{"total":96636764160,"available":95443759104,"internally_reserved":1073741824,"allocated":119255040,"data_stored":118022756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":3.629,"5min":1.506,"15min":1.22},"min":{"1min":0.42199999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":43.292999999999999,"5min":43.292999999999999,"15min":43.292999999999999},"last":1.3160000000000001},{"interface":"front","average":{"1min":3.1520000000000001,"5min":4.4249999999999998,"15min":2.0110000000000001},"min":{"1min":0.44700000000000001,"5min":0.438,"15min":0.438},"max":{"1min":42.933,"5min":329.11399999999998,"15min":329.11399999999998},"last":0.56499999999999995}]},{"osd":1,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":8.5239999999999991,"5min":12.039999999999999,"15min":4.6319999999999997},"min":{"1min":0.56799999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":139.88200000000001,"5min":787.22900000000004,"15min":787.22900000000004},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.448,"5min":2.5649999999999999,"15min":1.593},"min":{"1min":0.59899999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":4.3700000000000001,"5min":116.548,"15min":116.548},"last":1.1339999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":3.5499999999999998,"5min":1.5800000000000001,"15min":6.673},"min":{"1min":0.65700000000000003,"5min":0.499,"15min":0.499},"max":{"1min":43.923000000000002,"5min":43.923000000000002,"15min":168.56899999999999},"last":1.3500000000000001},{"interface":"front","average":{"1min":3.23,"5min":1.6399999999999999,"15min":22.634},"min":{"1min":0.52800000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":43.076999999999998,"5min":43.076999999999998,"15min":676.27099999999996},"last":1.0049999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.728,"15min":1.321},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":4.7640000000000002,"5min":29.129000000000001,"15min":29.129000000000001},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":10.029999999999999,"15min":4.1520000000000001},"min":{"1min":0.73999999999999999,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":5.2009999999999996,"5min":712.98699999999997,"15min":712.98699999999997},"last":1.375}]},{"osd":4,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":12.779999999999999,"15min":38.188000000000002},"min":{"1min":0.77200000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":4.9969999999999999,"5min":971.67999999999995,"15min":971.67999999999995},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.2470000000000001,"15min":1.286},"min":{"1min":0.73599999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.1160000000000001,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":1.032}]},{"osd":6,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":35.393000000000001,"5min":28.459,"15min":43.091999999999999},"min":{"1min":0.72199999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":676.20000000000005,"5min":966.60900000000004,"15min":966.60900000000004},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.528,"5min":6.7789999999999999,"15min":19.603999999999999},"min":{"1min":0.89500000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":5.2220000000000004,"5min":501.79000000000002,"15min":517.26700000000005},"last":1.6930000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.462,"15min":1.3600000000000001},"min":{"1min":0.85399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":4.8600000000000003,"5min":5.1559999999999997,"15min":5.1559999999999997},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.5669999999999999,"5min":12.834,"15min":5.3220000000000001},"min":{"1min":0.70099999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":3.8999999999999999,"5min":515.09100000000001,"15min":515.09100000000001},"last":1.0569999999999999}]}]}]}} 2020-08-07T15:12:45.069 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:12:45.069 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:45.044+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:45.084 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2020-08-07T15:12:45.084 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:12:45.085 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump -f json-pretty 2020-08-07T15:12:45.102 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:45.103 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:12:45.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:12:45.103 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:12:45.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.104 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106232 3.a 3:5599d7e6:::benchmark_data_smithi074_35447_object13278:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:12:45.104 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:12:45.104 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 297.741101415, 2020-08-07T15:12:45.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52578228699999996, 2020-08-07T15:12:45.104 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:45.105 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:45.105 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:45.105 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:12:45.106 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:12:45.106 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106232 2020-08-07T15:12:45.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.106 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:45.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:45.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:12:45.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:45.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:45.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:12:45.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4214999999999997e-05 2020-08-07T15:12:45.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:45.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345641+0000", 2020-08-07T15:12:45.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9279999999999993e-06 2020-08-07T15:12:45.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:45.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345649+0000", 2020-08-07T15:12:45.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4391e-05 2020-08-07T15:12:45.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:45.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345664+0000", 2020-08-07T15:12:45.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7170000000000002e-06 2020-08-07T15:12:45.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:45.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345670+0000", 2020-08-07T15:12:45.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.459e-05 2020-08-07T15:12:45.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345695+0000", 2020-08-07T15:12:45.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6070999999999999e-05 2020-08-07T15:12:45.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:45.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345751+0000", 2020-08-07T15:12:45.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010069300000000001 2020-08-07T15:12:45.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:12:45.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345852+0000", 2020-08-07T15:12:45.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013512299999999999 2020-08-07T15:12:45.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.347203+0000", 2020-08-07T15:12:45.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0844000000000001e-05 2020-08-07T15:12:45.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:45.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.347274+0000", 2020-08-07T15:12:45.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52399471600000003 2020-08-07T15:12:45.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871269+0000", 2020-08-07T15:12:45.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.311e-06 2020-08-07T15:12:45.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:45.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871270+0000", 2020-08-07T15:12:45.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3509000000000004e-05 2020-08-07T15:12:45.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:45.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871343+0000", 2020-08-07T15:12:45.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6072000000000002e-05 2020-08-07T15:12:45.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:45.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871389+0000", 2020-08-07T15:12:45.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52578228699999996 2020-08-07T15:12:45.136 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.136 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:45.136 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.137 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106232 3.a e59/20 ondisk, result = 0)", 2020-08-07T15:12:45.137 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:12:45.137 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 297.73805523999999, 2020-08-07T15:12:45.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52274359299999995, 2020-08-07T15:12:45.138 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:45.138 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:45.138 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:45.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:45.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:12:45.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:45.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:45.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348653+0000", 2020-08-07T15:12:45.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5011000000000001e-05 2020-08-07T15:12:45.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:45.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348668+0000", 2020-08-07T15:12:45.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1959999999999998e-06 2020-08-07T15:12:45.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:45.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.348671+0000", 2020-08-07T15:12:45.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52253456499999995 2020-08-07T15:12:45.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:45.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871205+0000", 2020-08-07T15:12:45.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0230000000000003e-06 2020-08-07T15:12:45.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:45.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871209+0000", 2020-08-07T15:12:45.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0708999999999999e-05 2020-08-07T15:12:45.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871250+0000", 2020-08-07T15:12:45.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0158e-05 2020-08-07T15:12:45.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:45.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871260+0000", 2020-08-07T15:12:45.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013693099999999999 2020-08-07T15:12:45.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:45.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871397+0000", 2020-08-07T15:12:45.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52274359299999995 2020-08-07T15:12:45.149 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.149 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:45.149 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.150 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:30976 4.5 e98/81 ondisk, result = 0)", 2020-08-07T15:12:45.150 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:45.736493+0000", 2020-08-07T15:12:45.150 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 239.350215515, 2020-08-07T15:12:45.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54677504399999999, 2020-08-07T15:12:45.151 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:45.151 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:45.151 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:45.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:45.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736493+0000", 2020-08-07T15:12:45.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:45.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:45.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736493+0000", 2020-08-07T15:12:45.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.012e-05 2020-08-07T15:12:45.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:45.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736523+0000", 2020-08-07T15:12:45.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6669999999999998e-06 2020-08-07T15:12:45.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:45.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736527+0000", 2020-08-07T15:12:45.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54657333200000002 2020-08-07T15:12:45.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:45.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283100+0000", 2020-08-07T15:12:45.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.549e-06 2020-08-07T15:12:45.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:45.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283105+0000", 2020-08-07T15:12:45.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1853000000000006e-05 2020-08-07T15:12:45.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283167+0000", 2020-08-07T15:12:45.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015e-05 2020-08-07T15:12:45.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:45.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283177+0000", 2020-08-07T15:12:45.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1508000000000005e-05 2020-08-07T15:12:45.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:45.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283268+0000", 2020-08-07T15:12:45.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54677504399999999 2020-08-07T15:12:45.164 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.164 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:45.164 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.172 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:37048 4.5 4:a403bd70:::benchmark_data_smithi074_44586_object4630:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e106)", 2020-08-07T15:12:45.172 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:54.237330+0000", 2020-08-07T15:12:45.173 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 230.849378438, 2020-08-07T15:12:45.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.474059283, 2020-08-07T15:12:45.173 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:45.173 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:45.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:45.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:45.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:45.174 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 37048 2020-08-07T15:12:45.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.175 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:45.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:45.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237330+0000", 2020-08-07T15:12:45.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:45.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:45.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237330+0000", 2020-08-07T15:12:45.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0713999999999998e-05 2020-08-07T15:12:45.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:45.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237361+0000", 2020-08-07T15:12:45.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.756e-06 2020-08-07T15:12:45.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:45.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237364+0000", 2020-08-07T15:12:45.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.694e-06 2020-08-07T15:12:45.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:45.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237369+0000", 2020-08-07T15:12:45.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3709999999999998e-06 2020-08-07T15:12:45.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:45.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.237372+0000", 2020-08-07T15:12:45.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015644700000000001 2020-08-07T15:12:45.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.238936+0000", 2020-08-07T15:12:45.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9940999999999998e-05 2020-08-07T15:12:45.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:45.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.239026+0000", 2020-08-07T15:12:45.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017423800000000001 2020-08-07T15:12:45.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,7", 2020-08-07T15:12:45.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.239200+0000", 2020-08-07T15:12:45.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041267530000000004 2020-08-07T15:12:45.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:45.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.243327+0000", 2020-08-07T15:12:45.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7700000000000004e-07 2020-08-07T15:12:45.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:45.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.243328+0000", 2020-08-07T15:12:45.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1211999999999997e-05 2020-08-07T15:12:45.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:45.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.243379+0000", 2020-08-07T15:12:45.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075155150000000004 2020-08-07T15:12:45.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.250894+0000", 2020-08-07T15:12:45.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.460450313 2020-08-07T15:12:45.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.711345+0000", 2020-08-07T15:12:45.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1990000000000001e-06 2020-08-07T15:12:45.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:45.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.711346+0000", 2020-08-07T15:12:45.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3529999999999998e-05 2020-08-07T15:12:45.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:45.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.711389+0000", 2020-08-07T15:12:45.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.474059283 2020-08-07T15:12:45.195 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.195 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:45.195 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.196 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118153 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:45.196 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.222760+0000", 2020-08-07T15:12:45.196 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.863948315000002, 2020-08-07T15:12:45.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0520728660000001, 2020-08-07T15:12:45.197 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:45.197 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:45.200 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:45.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:45.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:45.201 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118153 2020-08-07T15:12:45.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.201 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:45.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:45.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222760+0000", 2020-08-07T15:12:45.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:45.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:45.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222760+0000", 2020-08-07T15:12:45.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1894000000000002e-05 2020-08-07T15:12:45.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:45.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222812+0000", 2020-08-07T15:12:45.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4880000000000001e-06 2020-08-07T15:12:45.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:45.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222820+0000", 2020-08-07T15:12:45.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6744000000000001e-05 2020-08-07T15:12:45.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:45.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222836+0000", 2020-08-07T15:12:45.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5470000000000002e-06 2020-08-07T15:12:45.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:45.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222846+0000", 2020-08-07T15:12:45.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9478e-05 2020-08-07T15:12:45.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222925+0000", 2020-08-07T15:12:45.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0966e-05 2020-08-07T15:12:45.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2020-08-07T15:12:45.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222936+0000", 2020-08-07T15:12:45.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002974099 2020-08-07T15:12:45.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.225910+0000", 2020-08-07T15:12:45.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5197e-05 2020-08-07T15:12:45.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:45.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.225926+0000", 2020-08-07T15:12:45.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.02944747 2020-08-07T15:12:45.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.255373+0000", 2020-08-07T15:12:45.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022597200000000001 2020-08-07T15:12:45.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:45.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.255599+0000", 2020-08-07T15:12:45.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000194415 2020-08-07T15:12:45.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:45.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.255794+0000", 2020-08-07T15:12:45.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0076831360000000001 2020-08-07T15:12:45.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:45.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263477+0000", 2020-08-07T15:12:45.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01099552 2020-08-07T15:12:45.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274472+0000", 2020-08-07T15:12:45.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2948999999999998e-05 2020-08-07T15:12:45.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274515+0000", 2020-08-07T15:12:45.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.569999999999999e-07 2020-08-07T15:12:45.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:45.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274516+0000", 2020-08-07T15:12:45.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2750999999999997e-05 2020-08-07T15:12:45.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:45.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274569+0000", 2020-08-07T15:12:45.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4109000000000001e-05 2020-08-07T15:12:45.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274613+0000", 2020-08-07T15:12:45.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0432999999999997e-05 2020-08-07T15:12:45.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274653+0000", 2020-08-07T15:12:45.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5900000000000002e-07 2020-08-07T15:12:45.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:45.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274654+0000", 2020-08-07T15:12:45.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017928200000000001 2020-08-07T15:12:45.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:45.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274833+0000", 2020-08-07T15:12:45.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0520728660000001 2020-08-07T15:12:45.252 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.252 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:45.253 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.253 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118154 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:45.254 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.222899+0000", 2020-08-07T15:12:45.254 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.863810012999998, 2020-08-07T15:12:45.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0539106499999999, 2020-08-07T15:12:45.256 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:45.257 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:45.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:45.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:45.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:45.257 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118154 2020-08-07T15:12:45.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.258 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:45.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:45.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222899+0000", 2020-08-07T15:12:45.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:45.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:45.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222899+0000", 2020-08-07T15:12:45.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4440000000000001e-05 2020-08-07T15:12:45.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:45.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222953+0000", 2020-08-07T15:12:45.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4560000000000003e-06 2020-08-07T15:12:45.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:45.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222960+0000", 2020-08-07T15:12:45.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.201e-05 2020-08-07T15:12:45.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:45.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222972+0000", 2020-08-07T15:12:45.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4779999999999996e-06 2020-08-07T15:12:45.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:45.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222979+0000", 2020-08-07T15:12:45.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2364000000000002e-05 2020-08-07T15:12:45.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223031+0000", 2020-08-07T15:12:45.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7779999999999996e-06 2020-08-07T15:12:45.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:45.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223040+0000", 2020-08-07T15:12:45.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029381720000000002 2020-08-07T15:12:45.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.225978+0000", 2020-08-07T15:12:45.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1806e-05 2020-08-07T15:12:45.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:45.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.225990+0000", 2020-08-07T15:12:45.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0316258110000001 2020-08-07T15:12:45.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.257616+0000", 2020-08-07T15:12:45.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000143759 2020-08-07T15:12:45.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:45.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.257760+0000", 2020-08-07T15:12:45.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000296249 2020-08-07T15:12:45.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:45.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.258056+0000", 2020-08-07T15:12:45.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054347309999999999 2020-08-07T15:12:45.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:45.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263491+0000", 2020-08-07T15:12:45.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011408567999999999 2020-08-07T15:12:45.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274899+0000", 2020-08-07T15:12:45.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030791400000000001 2020-08-07T15:12:45.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275207+0000", 2020-08-07T15:12:45.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6499999999999998e-07 2020-08-07T15:12:45.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:45.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275208+0000", 2020-08-07T15:12:45.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3047999999999998e-05 2020-08-07T15:12:45.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:45.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275261+0000", 2020-08-07T15:12:45.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000419706 2020-08-07T15:12:45.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275681+0000", 2020-08-07T15:12:45.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091386099999999997 2020-08-07T15:12:45.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276594+0000", 2020-08-07T15:12:45.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6000000000000001e-06 2020-08-07T15:12:45.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:45.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276597+0000", 2020-08-07T15:12:45.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000212134 2020-08-07T15:12:45.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:45.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276809+0000", 2020-08-07T15:12:45.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0539106499999999 2020-08-07T15:12:45.314 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.315 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:45.315 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.315 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118155 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:45.316 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223018+0000", 2020-08-07T15:12:45.316 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.863690202999997, 2020-08-07T15:12:45.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0540362350000001, 2020-08-07T15:12:45.316 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:45.316 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:45.317 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:45.317 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:45.317 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:45.317 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118155 2020-08-07T15:12:45.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.318 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:45.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:45.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223018+0000", 2020-08-07T15:12:45.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:45.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:45.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223018+0000", 2020-08-07T15:12:45.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8291999999999999e-05 2020-08-07T15:12:45.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:45.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223057+0000", 2020-08-07T15:12:45.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.805e-06 2020-08-07T15:12:45.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:45.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223060+0000", 2020-08-07T15:12:45.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0050000000000004e-06 2020-08-07T15:12:45.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:45.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223068+0000", 2020-08-07T15:12:45.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4846e-05 2020-08-07T15:12:45.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:45.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223083+0000", 2020-08-07T15:12:45.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8695000000000005e-05 2020-08-07T15:12:45.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223162+0000", 2020-08-07T15:12:45.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4919999999999996e-06 2020-08-07T15:12:45.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:45.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223169+0000", 2020-08-07T15:12:45.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028562219999999998 2020-08-07T15:12:45.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226026+0000", 2020-08-07T15:12:45.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.027e-05 2020-08-07T15:12:45.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:45.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226036+0000", 2020-08-07T15:12:45.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0327165199999999 2020-08-07T15:12:45.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.258752+0000", 2020-08-07T15:12:45.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012614899999999999 2020-08-07T15:12:45.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:45.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.258879+0000", 2020-08-07T15:12:45.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000258008 2020-08-07T15:12:45.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:45.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.259137+0000", 2020-08-07T15:12:45.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0067332820000000002 2020-08-07T15:12:45.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:45.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.265870+0000", 2020-08-07T15:12:45.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0090796000000000002 2020-08-07T15:12:45.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274950+0000", 2020-08-07T15:12:45.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000632601 2020-08-07T15:12:45.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275582+0000", 2020-08-07T15:12:45.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8999999999999997e-07 2020-08-07T15:12:45.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:45.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275583+0000", 2020-08-07T15:12:45.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6400999999999998e-05 2020-08-07T15:12:45.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:45.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275629+0000", 2020-08-07T15:12:45.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014887 2020-08-07T15:12:45.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275778+0000", 2020-08-07T15:12:45.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010622559999999999 2020-08-07T15:12:45.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276840+0000", 2020-08-07T15:12:45.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2199999999999999e-07 2020-08-07T15:12:45.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:45.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276841+0000", 2020-08-07T15:12:45.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000214009 2020-08-07T15:12:45.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:45.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277055+0000", 2020-08-07T15:12:45.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0540362350000001 2020-08-07T15:12:45.366 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.366 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:45.371 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.372 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118156 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:45.372 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223123+0000", 2020-08-07T15:12:45.372 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.863585337000004, 2020-08-07T15:12:45.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0542400009999999, 2020-08-07T15:12:45.375 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:45.376 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:45.376 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:45.376 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:45.377 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:45.377 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118156 2020-08-07T15:12:45.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.377 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:45.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:45.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223123+0000", 2020-08-07T15:12:45.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:45.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:45.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223123+0000", 2020-08-07T15:12:45.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2212999999999995e-05 2020-08-07T15:12:45.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:45.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223195+0000", 2020-08-07T15:12:45.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8709999999999999e-06 2020-08-07T15:12:45.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:45.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223202+0000", 2020-08-07T15:12:45.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3509999999999999e-05 2020-08-07T15:12:45.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:45.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223216+0000", 2020-08-07T15:12:45.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2729999999999997e-06 2020-08-07T15:12:45.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:45.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223223+0000", 2020-08-07T15:12:45.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6728000000000003e-05 2020-08-07T15:12:45.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223280+0000", 2020-08-07T15:12:45.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1046999999999999e-05 2020-08-07T15:12:45.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:45.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223291+0000", 2020-08-07T15:12:45.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027721030000000002 2020-08-07T15:12:45.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226063+0000", 2020-08-07T15:12:45.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1809000000000004e-05 2020-08-07T15:12:45.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:45.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226125+0000", 2020-08-07T15:12:45.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0337493870000001 2020-08-07T15:12:45.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.259874+0000", 2020-08-07T15:12:45.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000133974 2020-08-07T15:12:45.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:45.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.260008+0000", 2020-08-07T15:12:45.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00064730600000000005 2020-08-07T15:12:45.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:45.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.260655+0000", 2020-08-07T15:12:45.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052233740000000002 2020-08-07T15:12:45.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:45.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.265879+0000", 2020-08-07T15:12:45.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.009433238 2020-08-07T15:12:45.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275312+0000", 2020-08-07T15:12:45.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050955399999999995 2020-08-07T15:12:45.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275822+0000", 2020-08-07T15:12:45.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.58e-07 2020-08-07T15:12:45.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:45.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275822+0000", 2020-08-07T15:12:45.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5469000000000006e-05 2020-08-07T15:12:45.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:45.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275887+0000", 2020-08-07T15:12:45.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000371212 2020-08-07T15:12:45.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276259+0000", 2020-08-07T15:12:45.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00085558499999999996 2020-08-07T15:12:45.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277114+0000", 2020-08-07T15:12:45.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2020-08-07T15:12:45.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:45.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277115+0000", 2020-08-07T15:12:45.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024832000000000001 2020-08-07T15:12:45.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:45.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277363+0000", 2020-08-07T15:12:45.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0542400009999999 2020-08-07T15:12:45.409 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.409 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:45.409 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.410 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118157 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:45.411 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223267+0000", 2020-08-07T15:12:45.411 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.863441102000003, 2020-08-07T15:12:45.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0543752369999999, 2020-08-07T15:12:45.411 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:45.411 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:45.412 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:45.412 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:45.412 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:45.412 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118157 2020-08-07T15:12:45.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.413 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:45.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:45.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223267+0000", 2020-08-07T15:12:45.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:45.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:45.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223267+0000", 2020-08-07T15:12:45.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9616000000000001e-05 2020-08-07T15:12:45.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:45.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223317+0000", 2020-08-07T15:12:45.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1359999999999996e-06 2020-08-07T15:12:45.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:45.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223322+0000", 2020-08-07T15:12:45.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0529e-05 2020-08-07T15:12:45.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:45.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223333+0000", 2020-08-07T15:12:45.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1059999999999999e-06 2020-08-07T15:12:45.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:45.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223338+0000", 2020-08-07T15:12:45.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8609000000000002e-05 2020-08-07T15:12:45.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223386+0000", 2020-08-07T15:12:45.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3830000000000003e-06 2020-08-07T15:12:45.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:45.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223394+0000", 2020-08-07T15:12:45.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027643580000000002 2020-08-07T15:12:45.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226158+0000", 2020-08-07T15:12:45.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6809999999999993e-06 2020-08-07T15:12:45.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:45.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226168+0000", 2020-08-07T15:12:45.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.035171356 2020-08-07T15:12:45.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.261339+0000", 2020-08-07T15:12:45.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000130514 2020-08-07T15:12:45.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:45.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.261470+0000", 2020-08-07T15:12:45.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025685699999999998 2020-08-07T15:12:45.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:45.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.261727+0000", 2020-08-07T15:12:45.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075810410000000002 2020-08-07T15:12:45.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:45.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269308+0000", 2020-08-07T15:12:45.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064252490000000001 2020-08-07T15:12:45.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275733+0000", 2020-08-07T15:12:45.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020602000000000001 2020-08-07T15:12:45.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275939+0000", 2020-08-07T15:12:45.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999999e-07 2020-08-07T15:12:45.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:45.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275939+0000", 2020-08-07T15:12:45.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7071999999999997e-05 2020-08-07T15:12:45.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:45.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276016+0000", 2020-08-07T15:12:45.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029090700000000003 2020-08-07T15:12:45.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276307+0000", 2020-08-07T15:12:45.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001114929 2020-08-07T15:12:45.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277422+0000", 2020-08-07T15:12:45.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1200000000000003e-07 2020-08-07T15:12:45.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:45.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277423+0000", 2020-08-07T15:12:45.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021982600000000001 2020-08-07T15:12:45.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:45.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277643+0000", 2020-08-07T15:12:45.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0543752369999999 2020-08-07T15:12:45.446 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.446 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:45.446 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.447 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118158 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:45.447 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223375+0000", 2020-08-07T15:12:45.448 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.863333365999999, 2020-08-07T15:12:45.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0545579519999999, 2020-08-07T15:12:45.450 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:45.450 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:45.451 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:45.451 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:45.451 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:45.451 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118158 2020-08-07T15:12:45.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.452 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:45.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:45.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223375+0000", 2020-08-07T15:12:45.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:45.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:45.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223375+0000", 2020-08-07T15:12:45.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7657000000000001e-05 2020-08-07T15:12:45.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:45.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223423+0000", 2020-08-07T15:12:45.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0810000000000001e-06 2020-08-07T15:12:45.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:45.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223428+0000", 2020-08-07T15:12:45.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7839999999999998e-06 2020-08-07T15:12:45.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:45.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223438+0000", 2020-08-07T15:12:45.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.634e-06 2020-08-07T15:12:45.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:45.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223442+0000", 2020-08-07T15:12:45.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0154999999999998e-05 2020-08-07T15:12:45.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223482+0000", 2020-08-07T15:12:45.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9579999999999998e-06 2020-08-07T15:12:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223487+0000", 2020-08-07T15:12:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027060999999999999 2020-08-07T15:12:45.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226194+0000", 2020-08-07T15:12:45.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1311999999999999e-05 2020-08-07T15:12:45.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:45.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226205+0000", 2020-08-07T15:12:45.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0361494609999999 2020-08-07T15:12:45.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.262354+0000", 2020-08-07T15:12:45.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9950000000000004e-05 2020-08-07T15:12:45.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:45.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.262454+0000", 2020-08-07T15:12:45.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000244093 2020-08-07T15:12:45.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:45.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.262698+0000", 2020-08-07T15:12:45.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0066176150000000003 2020-08-07T15:12:45.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:45.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269316+0000", 2020-08-07T15:12:45.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006768998 2020-08-07T15:12:45.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276085+0000", 2020-08-07T15:12:45.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9524000000000002e-05 2020-08-07T15:12:45.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276135+0000", 2020-08-07T15:12:45.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6100000000000001e-07 2020-08-07T15:12:45.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:45.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276135+0000", 2020-08-07T15:12:45.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1214e-05 2020-08-07T15:12:45.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:45.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276206+0000", 2020-08-07T15:12:45.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014914999999999999 2020-08-07T15:12:45.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276355+0000", 2020-08-07T15:12:45.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013427770000000001 2020-08-07T15:12:45.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277698+0000", 2020-08-07T15:12:45.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8599999999999998e-07 2020-08-07T15:12:45.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:45.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277699+0000", 2020-08-07T15:12:45.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023444199999999999 2020-08-07T15:12:45.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:45.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277933+0000", 2020-08-07T15:12:45.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0545579519999999 2020-08-07T15:12:45.506 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.506 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:45.507 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.508 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118159 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:45.508 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223476+0000", 2020-08-07T15:12:45.508 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.863232199999999, 2020-08-07T15:12:45.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0547920749999999, 2020-08-07T15:12:45.508 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:45.512 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:45.512 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:45.513 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:45.513 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:45.516 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118159 2020-08-07T15:12:45.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.516 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:45.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:45.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223476+0000", 2020-08-07T15:12:45.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:45.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:45.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223476+0000", 2020-08-07T15:12:45.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6094000000000003e-05 2020-08-07T15:12:45.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:45.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223522+0000", 2020-08-07T15:12:45.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6650000000000002e-06 2020-08-07T15:12:45.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:45.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223527+0000", 2020-08-07T15:12:45.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5629999999999998e-06 2020-08-07T15:12:45.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:45.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223537+0000", 2020-08-07T15:12:45.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6810000000000003e-06 2020-08-07T15:12:45.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:45.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223542+0000", 2020-08-07T15:12:45.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6680000000000001e-05 2020-08-07T15:12:45.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223579+0000", 2020-08-07T15:12:45.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2020-08-07T15:12:45.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:45.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223586+0000", 2020-08-07T15:12:45.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026469250000000001 2020-08-07T15:12:45.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226233+0000", 2020-08-07T15:12:45.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3130000000000004e-06 2020-08-07T15:12:45.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:45.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226242+0000", 2020-08-07T15:12:45.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.037297957 2020-08-07T15:12:45.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263540+0000", 2020-08-07T15:12:45.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4307000000000004e-05 2020-08-07T15:12:45.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:45.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263634+0000", 2020-08-07T15:12:45.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019544300000000001 2020-08-07T15:12:45.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:45.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263830+0000", 2020-08-07T15:12:45.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054939949999999998 2020-08-07T15:12:45.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:45.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269324+0000", 2020-08-07T15:12:45.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070859139999999996 2020-08-07T15:12:45.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276410+0000", 2020-08-07T15:12:45.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0924000000000002e-05 2020-08-07T15:12:45.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276461+0000", 2020-08-07T15:12:45.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4099999999999998e-07 2020-08-07T15:12:45.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:45.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276461+0000", 2020-08-07T15:12:45.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8027000000000004e-05 2020-08-07T15:12:45.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:45.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276539+0000", 2020-08-07T15:12:45.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001458453 2020-08-07T15:12:45.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277998+0000", 2020-08-07T15:12:45.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7283000000000001e-05 2020-08-07T15:12:45.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278045+0000", 2020-08-07T15:12:45.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9100000000000004e-07 2020-08-07T15:12:45.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:45.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278046+0000", 2020-08-07T15:12:45.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022290400000000001 2020-08-07T15:12:45.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:45.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278268+0000", 2020-08-07T15:12:45.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0547920749999999 2020-08-07T15:12:45.549 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.550 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:45.550 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.551 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118160 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:45.552 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223579+0000", 2020-08-07T15:12:45.552 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.863129219000001, 2020-08-07T15:12:45.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0553689980000001, 2020-08-07T15:12:45.552 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:45.552 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:45.553 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:45.553 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:45.553 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:45.553 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118160 2020-08-07T15:12:45.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.554 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:45.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:45.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223579+0000", 2020-08-07T15:12:45.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:45.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:45.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223579+0000", 2020-08-07T15:12:45.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5401000000000001e-05 2020-08-07T15:12:45.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:45.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223625+0000", 2020-08-07T15:12:45.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9069999999999998e-06 2020-08-07T15:12:45.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:45.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223630+0000", 2020-08-07T15:12:45.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4800000000000007e-06 2020-08-07T15:12:45.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:45.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223639+0000", 2020-08-07T15:12:45.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5410000000000002e-06 2020-08-07T15:12:45.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:45.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223644+0000", 2020-08-07T15:12:45.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1476000000000002e-05 2020-08-07T15:12:45.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223675+0000", 2020-08-07T15:12:45.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0240000000000003e-06 2020-08-07T15:12:45.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:45.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223681+0000", 2020-08-07T15:12:45.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025870820000000001 2020-08-07T15:12:45.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226268+0000", 2020-08-07T15:12:45.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0540000000000001e-05 2020-08-07T15:12:45.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:45.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226279+0000", 2020-08-07T15:12:45.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.038209656 2020-08-07T15:12:45.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.264488+0000", 2020-08-07T15:12:45.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029908499999999999 2020-08-07T15:12:45.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:45.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.264788+0000", 2020-08-07T15:12:45.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000216997 2020-08-07T15:12:45.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:45.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.265005+0000", 2020-08-07T15:12:45.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0060308230000000003 2020-08-07T15:12:45.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:45.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271035+0000", 2020-08-07T15:12:45.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0072889499999999998 2020-08-07T15:12:45.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278324+0000", 2020-08-07T15:12:45.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4982999999999999e-05 2020-08-07T15:12:45.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278369+0000", 2020-08-07T15:12:45.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4000000000000002e-07 2020-08-07T15:12:45.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:45.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278370+0000", 2020-08-07T15:12:45.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7656000000000003e-05 2020-08-07T15:12:45.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:45.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278447+0000", 2020-08-07T15:12:45.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000277416 2020-08-07T15:12:45.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278725+0000", 2020-08-07T15:12:45.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2345000000000001e-05 2020-08-07T15:12:45.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278777+0000", 2020-08-07T15:12:45.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8500000000000002e-07 2020-08-07T15:12:45.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:45.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278778+0000", 2020-08-07T15:12:45.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017071100000000001 2020-08-07T15:12:45.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:45.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278948+0000", 2020-08-07T15:12:45.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0553689980000001 2020-08-07T15:12:45.583 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.584 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:45.584 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118441 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223679+0000", 2020-08-07T15:12:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.863029204, 2020-08-07T15:12:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057502779, 2020-08-07T15:12:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:45.597 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:45.597 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:45.597 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:45.597 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:45.597 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118441 2020-08-07T15:12:45.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.598 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:45.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:45.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223679+0000", 2020-08-07T15:12:45.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:45.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:45.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223679+0000", 2020-08-07T15:12:45.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5559999999999997e-05 2020-08-07T15:12:45.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:45.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223725+0000", 2020-08-07T15:12:45.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1250000000000001e-06 2020-08-07T15:12:45.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:45.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223730+0000", 2020-08-07T15:12:45.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0046e-05 2020-08-07T15:12:45.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:45.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223740+0000", 2020-08-07T15:12:45.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5979999999999999e-06 2020-08-07T15:12:45.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:45.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223745+0000", 2020-08-07T15:12:45.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7234999999999998e-05 2020-08-07T15:12:45.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223782+0000", 2020-08-07T15:12:45.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7789999999999998e-06 2020-08-07T15:12:45.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:45.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223789+0000", 2020-08-07T15:12:45.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002516169 2020-08-07T15:12:45.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226305+0000", 2020-08-07T15:12:45.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3860000000000007e-06 2020-08-07T15:12:45.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:45.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226313+0000", 2020-08-07T15:12:45.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0396126269999999 2020-08-07T15:12:45.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.265926+0000", 2020-08-07T15:12:45.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00079104700000000002 2020-08-07T15:12:45.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:45.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.266717+0000", 2020-08-07T15:12:45.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024298599999999999 2020-08-07T15:12:45.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:45.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.266960+0000", 2020-08-07T15:12:45.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040855509999999998 2020-08-07T15:12:45.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:45.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271045+0000", 2020-08-07T15:12:45.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0074549170000000001 2020-08-07T15:12:45.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278500+0000", 2020-08-07T15:12:45.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8816999999999997e-05 2020-08-07T15:12:45.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278549+0000", 2020-08-07T15:12:45.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9000000000000002e-07 2020-08-07T15:12:45.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:45.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278550+0000", 2020-08-07T15:12:45.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0120999999999997e-05 2020-08-07T15:12:45.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:45.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278600+0000", 2020-08-07T15:12:45.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00092631699999999996 2020-08-07T15:12:45.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279526+0000", 2020-08-07T15:12:45.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014504520000000001 2020-08-07T15:12:45.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280976+0000", 2020-08-07T15:12:45.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6899999999999997e-07 2020-08-07T15:12:45.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:45.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280977+0000", 2020-08-07T15:12:45.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020508700000000001 2020-08-07T15:12:45.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:45.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281182+0000", 2020-08-07T15:12:45.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057502779 2020-08-07T15:12:45.642 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.643 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:45.643 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.643 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118442 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:45.644 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223780+0000", 2020-08-07T15:12:45.644 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.862928334000003, 2020-08-07T15:12:45.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0575529850000001, 2020-08-07T15:12:45.644 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:45.644 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:45.645 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:45.645 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:45.645 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:45.645 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118442 2020-08-07T15:12:45.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.654 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:45.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:45.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223780+0000", 2020-08-07T15:12:45.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:45.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:45.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223780+0000", 2020-08-07T15:12:45.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0340999999999998e-05 2020-08-07T15:12:45.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:45.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223831+0000", 2020-08-07T15:12:45.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3499999999999996e-06 2020-08-07T15:12:45.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:45.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223836+0000", 2020-08-07T15:12:45.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0414999999999999e-05 2020-08-07T15:12:45.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:45.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223846+0000", 2020-08-07T15:12:45.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9239999999999998e-06 2020-08-07T15:12:45.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:45.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223851+0000", 2020-08-07T15:12:45.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5946000000000003e-05 2020-08-07T15:12:45.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223907+0000", 2020-08-07T15:12:45.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6189999999999996e-06 2020-08-07T15:12:45.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:45.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223914+0000", 2020-08-07T15:12:45.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002424149 2020-08-07T15:12:45.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226338+0000", 2020-08-07T15:12:45.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2830000000000001e-06 2020-08-07T15:12:45.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:45.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226346+0000", 2020-08-07T15:12:45.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0412357219999999 2020-08-07T15:12:45.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.267582+0000", 2020-08-07T15:12:45.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000123924 2020-08-07T15:12:45.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:45.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.267706+0000", 2020-08-07T15:12:45.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000216376 2020-08-07T15:12:45.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:45.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.267922+0000", 2020-08-07T15:12:45.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031340790000000001 2020-08-07T15:12:45.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:45.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271056+0000", 2020-08-07T15:12:45.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.007623276 2020-08-07T15:12:45.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278680+0000", 2020-08-07T15:12:45.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000333853 2020-08-07T15:12:45.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279013+0000", 2020-08-07T15:12:45.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.02e-06 2020-08-07T15:12:45.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:45.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279014+0000", 2020-08-07T15:12:45.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6306999999999999e-05 2020-08-07T15:12:45.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:45.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279101+0000", 2020-08-07T15:12:45.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050739800000000005 2020-08-07T15:12:45.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279608+0000", 2020-08-07T15:12:45.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001617098 2020-08-07T15:12:45.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281225+0000", 2020-08-07T15:12:45.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9700000000000002e-07 2020-08-07T15:12:45.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:45.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281226+0000", 2020-08-07T15:12:45.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107508 2020-08-07T15:12:45.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:45.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281333+0000", 2020-08-07T15:12:45.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0575529850000001 2020-08-07T15:12:45.705 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.706 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:45.706 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.706 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118443 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:45.707 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223889+0000", 2020-08-07T15:12:45.707 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.862819664, 2020-08-07T15:12:45.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.05758606, 2020-08-07T15:12:45.707 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:45.707 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:45.708 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:45.708 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:45.708 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:45.708 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118443 2020-08-07T15:12:45.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.709 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:45.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:45.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223889+0000", 2020-08-07T15:12:45.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:45.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:45.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223889+0000", 2020-08-07T15:12:45.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8148e-05 2020-08-07T15:12:45.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:45.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223937+0000", 2020-08-07T15:12:45.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2789999999999998e-06 2020-08-07T15:12:45.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:45.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223942+0000", 2020-08-07T15:12:45.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0472000000000001e-05 2020-08-07T15:12:45.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:45.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223953+0000", 2020-08-07T15:12:45.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6580000000000001e-06 2020-08-07T15:12:45.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:45.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223957+0000", 2020-08-07T15:12:45.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5981000000000001e-05 2020-08-07T15:12:45.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223983+0000", 2020-08-07T15:12:45.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.288e-06 2020-08-07T15:12:45.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:45.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223989+0000", 2020-08-07T15:12:45.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024051889999999999 2020-08-07T15:12:45.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226394+0000", 2020-08-07T15:12:45.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2439000000000001e-05 2020-08-07T15:12:45.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:45.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226406+0000", 2020-08-07T15:12:45.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0421168519999999 2020-08-07T15:12:45.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.268523+0000", 2020-08-07T15:12:45.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0449000000000002e-05 2020-08-07T15:12:45.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:45.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.268604+0000", 2020-08-07T15:12:45.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014104300000000001 2020-08-07T15:12:45.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:45.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.268745+0000", 2020-08-07T15:12:45.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051249829999999996 2020-08-07T15:12:45.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:45.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.273870+0000", 2020-08-07T15:12:45.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052772840000000001 2020-08-07T15:12:45.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279147+0000", 2020-08-07T15:12:45.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6060000000000003e-05 2020-08-07T15:12:45.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279193+0000", 2020-08-07T15:12:45.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9999999999999997e-07 2020-08-07T15:12:45.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:45.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279194+0000", 2020-08-07T15:12:45.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3566000000000003e-05 2020-08-07T15:12:45.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:45.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279237+0000", 2020-08-07T15:12:45.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011546359999999999 2020-08-07T15:12:45.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280392+0000", 2020-08-07T15:12:45.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00097857799999999991 2020-08-07T15:12:45.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281370+0000", 2020-08-07T15:12:45.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999999e-07 2020-08-07T15:12:45.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:45.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281371+0000", 2020-08-07T15:12:45.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104119 2020-08-07T15:12:45.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:45.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281475+0000", 2020-08-07T15:12:45.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.05758606 2020-08-07T15:12:45.760 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.760 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:45.761 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.761 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118444 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:45.761 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223994+0000", 2020-08-07T15:12:45.762 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.862714289000003, 2020-08-07T15:12:45.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057634572, 2020-08-07T15:12:45.762 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:45.762 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:45.762 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:45.763 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:45.763 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:45.763 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118444 2020-08-07T15:12:45.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.763 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:45.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:45.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223994+0000", 2020-08-07T15:12:45.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:45.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:45.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223994+0000", 2020-08-07T15:12:45.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1279999999999997e-05 2020-08-07T15:12:45.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:45.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224046+0000", 2020-08-07T15:12:45.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6930000000000004e-06 2020-08-07T15:12:45.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:45.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224051+0000", 2020-08-07T15:12:45.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.095e-05 2020-08-07T15:12:45.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:45.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224062+0000", 2020-08-07T15:12:45.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9529999999999998e-06 2020-08-07T15:12:45.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:45.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224067+0000", 2020-08-07T15:12:45.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000162091 2020-08-07T15:12:45.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224229+0000", 2020-08-07T15:12:45.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.276e-06 2020-08-07T15:12:45.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:45.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224238+0000", 2020-08-07T15:12:45.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022140409999999999 2020-08-07T15:12:45.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226452+0000", 2020-08-07T15:12:45.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5951999999999999e-05 2020-08-07T15:12:45.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:45.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226467+0000", 2020-08-07T15:12:45.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0428868099999999 2020-08-07T15:12:45.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269354+0000", 2020-08-07T15:12:45.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5264000000000003e-05 2020-08-07T15:12:45.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:45.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269430+0000", 2020-08-07T15:12:45.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020438499999999999 2020-08-07T15:12:45.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:45.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269634+0000", 2020-08-07T15:12:45.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042509959999999999 2020-08-07T15:12:45.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:45.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.273885+0000", 2020-08-07T15:12:45.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054290140000000002 2020-08-07T15:12:45.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279314+0000", 2020-08-07T15:12:45.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00038996700000000002 2020-08-07T15:12:45.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279704+0000", 2020-08-07T15:12:45.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5700000000000002e-07 2020-08-07T15:12:45.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:45.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279705+0000", 2020-08-07T15:12:45.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9377999999999997e-05 2020-08-07T15:12:45.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:45.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279794+0000", 2020-08-07T15:12:45.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00064521299999999995 2020-08-07T15:12:45.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280439+0000", 2020-08-07T15:12:45.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001072185 2020-08-07T15:12:45.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281511+0000", 2020-08-07T15:12:45.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7300000000000002e-07 2020-08-07T15:12:45.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:45.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281512+0000", 2020-08-07T15:12:45.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117094 2020-08-07T15:12:45.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:45.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281629+0000", 2020-08-07T15:12:45.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057634572 2020-08-07T15:12:45.785 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.785 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:45.786 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.786 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118445 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:45.786 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.224116+0000", 2020-08-07T15:12:45.787 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.862592348, 2020-08-07T15:12:45.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057744163, 2020-08-07T15:12:45.787 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:45.787 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:45.787 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:45.788 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:45.788 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:45.788 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118445 2020-08-07T15:12:45.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.788 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:45.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:45.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224116+0000", 2020-08-07T15:12:45.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:45.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:45.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224116+0000", 2020-08-07T15:12:45.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3542999999999999e-05 2020-08-07T15:12:45.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:45.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224170+0000", 2020-08-07T15:12:45.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.446e-06 2020-08-07T15:12:45.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:45.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224175+0000", 2020-08-07T15:12:45.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1473e-05 2020-08-07T15:12:45.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:45.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224187+0000", 2020-08-07T15:12:45.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9449999999999998e-06 2020-08-07T15:12:45.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:45.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224193+0000", 2020-08-07T15:12:45.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4945999999999997e-05 2020-08-07T15:12:45.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224268+0000", 2020-08-07T15:12:45.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5799999999999992e-06 2020-08-07T15:12:45.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:45.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224276+0000", 2020-08-07T15:12:45.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022246980000000002 2020-08-07T15:12:45.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226501+0000", 2020-08-07T15:12:45.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1359999999999997e-06 2020-08-07T15:12:45.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:45.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226509+0000", 2020-08-07T15:12:45.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.043599741 2020-08-07T15:12:45.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.270109+0000", 2020-08-07T15:12:45.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000129503 2020-08-07T15:12:45.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:45.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.270238+0000", 2020-08-07T15:12:45.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021162500000000001 2020-08-07T15:12:45.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:45.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.270450+0000", 2020-08-07T15:12:45.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036313090000000001 2020-08-07T15:12:45.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:45.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274081+0000", 2020-08-07T15:12:45.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00557165 2020-08-07T15:12:45.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279653+0000", 2020-08-07T15:12:45.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000196658 2020-08-07T15:12:45.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279849+0000", 2020-08-07T15:12:45.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6700000000000003e-07 2020-08-07T15:12:45.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:45.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279850+0000", 2020-08-07T15:12:45.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6522999999999998e-05 2020-08-07T15:12:45.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:45.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279937+0000", 2020-08-07T15:12:45.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00055050400000000005 2020-08-07T15:12:45.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280487+0000", 2020-08-07T15:12:45.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011833760000000001 2020-08-07T15:12:45.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281670+0000", 2020-08-07T15:12:45.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2020-08-07T15:12:45.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:45.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281671+0000", 2020-08-07T15:12:45.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001896 2020-08-07T15:12:45.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:45.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281860+0000", 2020-08-07T15:12:45.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057744163 2020-08-07T15:12:45.809 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.810 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:45.810 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.810 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118446 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:45.811 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.224256+0000", 2020-08-07T15:12:45.811 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.862452562999998, 2020-08-07T15:12:45.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057864804, 2020-08-07T15:12:45.811 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:45.811 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:45.812 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:45.812 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:45.812 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:45.812 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118446 2020-08-07T15:12:45.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.813 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:45.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:45.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224256+0000", 2020-08-07T15:12:45.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:45.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:45.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224256+0000", 2020-08-07T15:12:45.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5838000000000003e-05 2020-08-07T15:12:45.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:45.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224312+0000", 2020-08-07T15:12:45.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.879e-06 2020-08-07T15:12:45.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:45.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224318+0000", 2020-08-07T15:12:45.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3593e-05 2020-08-07T15:12:45.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:45.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224331+0000", 2020-08-07T15:12:45.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6169999999999998e-06 2020-08-07T15:12:45.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:45.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224337+0000", 2020-08-07T15:12:45.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7246000000000001e-05 2020-08-07T15:12:45.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224384+0000", 2020-08-07T15:12:45.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7690000000000008e-06 2020-08-07T15:12:45.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:45.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224394+0000", 2020-08-07T15:12:45.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021394589999999998 2020-08-07T15:12:45.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226533+0000", 2020-08-07T15:12:45.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3849999999999996e-06 2020-08-07T15:12:45.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:45.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226541+0000", 2020-08-07T15:12:45.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0445569109999999 2020-08-07T15:12:45.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271098+0000", 2020-08-07T15:12:45.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012137 2020-08-07T15:12:45.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:45.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271219+0000", 2020-08-07T15:12:45.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013978300000000001 2020-08-07T15:12:45.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:45.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271359+0000", 2020-08-07T15:12:45.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027367699999999999 2020-08-07T15:12:45.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:45.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274096+0000", 2020-08-07T15:12:45.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058973660000000002 2020-08-07T15:12:45.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279993+0000", 2020-08-07T15:12:45.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116291 2020-08-07T15:12:45.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280109+0000", 2020-08-07T15:12:45.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1099999999999995e-07 2020-08-07T15:12:45.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:45.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280110+0000", 2020-08-07T15:12:45.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.564e-05 2020-08-07T15:12:45.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:45.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280196+0000", 2020-08-07T15:12:45.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061969 2020-08-07T15:12:45.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280815+0000", 2020-08-07T15:12:45.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001097612 2020-08-07T15:12:45.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281913+0000", 2020-08-07T15:12:45.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4300000000000003e-07 2020-08-07T15:12:45.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:45.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281913+0000", 2020-08-07T15:12:45.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000207431 2020-08-07T15:12:45.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:45.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282121+0000", 2020-08-07T15:12:45.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057864804 2020-08-07T15:12:45.854 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.855 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:45.855 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.855 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118447 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:45.856 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.224379+0000", 2020-08-07T15:12:45.856 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.862329797000001, 2020-08-07T15:12:45.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0579997480000001, 2020-08-07T15:12:45.856 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:45.856 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:45.857 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:45.857 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:45.857 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:45.857 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118447 2020-08-07T15:12:45.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.858 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:45.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:45.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224379+0000", 2020-08-07T15:12:45.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:45.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:45.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224379+0000", 2020-08-07T15:12:45.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9775999999999999e-05 2020-08-07T15:12:45.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:45.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224429+0000", 2020-08-07T15:12:45.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8500000000000002e-06 2020-08-07T15:12:45.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:45.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224433+0000", 2020-08-07T15:12:45.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7340000000000008e-06 2020-08-07T15:12:45.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:45.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224441+0000", 2020-08-07T15:12:45.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.031e-06 2020-08-07T15:12:45.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:45.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224445+0000", 2020-08-07T15:12:45.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4264000000000003e-05 2020-08-07T15:12:45.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224479+0000", 2020-08-07T15:12:45.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0039999999999999e-06 2020-08-07T15:12:45.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:12:45.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224487+0000", 2020-08-07T15:12:45.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002241682 2020-08-07T15:12:45.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226729+0000", 2020-08-07T15:12:45.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0087e-05 2020-08-07T15:12:45.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:45.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226739+0000", 2020-08-07T15:12:45.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.045170006 2020-08-07T15:12:45.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271909+0000", 2020-08-07T15:12:45.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012782900000000001 2020-08-07T15:12:45.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:45.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.272037+0000", 2020-08-07T15:12:45.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024079400000000001 2020-08-07T15:12:45.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:45.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.272278+0000", 2020-08-07T15:12:45.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023454399999999999 2020-08-07T15:12:45.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:45.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274623+0000", 2020-08-07T15:12:45.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054268190000000003 2020-08-07T15:12:45.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280050+0000", 2020-08-07T15:12:45.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00048391200000000002 2020-08-07T15:12:45.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280534+0000", 2020-08-07T15:12:45.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5400000000000001e-07 2020-08-07T15:12:45.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:45.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280535+0000", 2020-08-07T15:12:45.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8922e-05 2020-08-07T15:12:45.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:45.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280623+0000", 2020-08-07T15:12:45.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024637999999999998 2020-08-07T15:12:45.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280870+0000", 2020-08-07T15:12:45.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001300142 2020-08-07T15:12:45.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282170+0000", 2020-08-07T15:12:45.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2099999999999997e-07 2020-08-07T15:12:45.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:45.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282170+0000", 2020-08-07T15:12:45.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000208001 2020-08-07T15:12:45.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:45.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282378+0000", 2020-08-07T15:12:45.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0579997480000001 2020-08-07T15:12:45.880 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.880 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:45.881 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.881 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118448 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:12:45.881 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.224474+0000", 2020-08-07T15:12:45.882 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.862234811, 2020-08-07T15:12:45.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0581683310000001, 2020-08-07T15:12:45.882 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:45.882 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:45.882 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:45.883 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:45.883 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:45.884 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118448 2020-08-07T15:12:45.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.884 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:45.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:45.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224474+0000", 2020-08-07T15:12:45.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:45.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:45.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224474+0000", 2020-08-07T15:12:45.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3804000000000002e-05 2020-08-07T15:12:45.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:45.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224528+0000", 2020-08-07T15:12:45.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5140000000000001e-06 2020-08-07T15:12:45.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:45.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224533+0000", 2020-08-07T15:12:45.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3315999999999999e-05 2020-08-07T15:12:45.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:45.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224546+0000", 2020-08-07T15:12:45.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7240000000000003e-06 2020-08-07T15:12:45.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:45.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224553+0000", 2020-08-07T15:12:45.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002226263 2020-08-07T15:12:45.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226779+0000", 2020-08-07T15:12:45.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1362000000000001e-05 2020-08-07T15:12:45.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:45.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226791+0000", 2020-08-07T15:12:45.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.046092652 2020-08-07T15:12:45.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:45.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.272883+0000", 2020-08-07T15:12:45.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000115981 2020-08-07T15:12:45.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:45.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.272999+0000", 2020-08-07T15:12:45.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029648899999999999 2020-08-07T15:12:45.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:12:45.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.273296+0000", 2020-08-07T15:12:45.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001852521 2020-08-07T15:12:45.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:45.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275148+0000", 2020-08-07T15:12:45.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051948309999999996 2020-08-07T15:12:45.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280343+0000", 2020-08-07T15:12:45.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033301799999999998 2020-08-07T15:12:45.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280676+0000", 2020-08-07T15:12:45.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8400000000000004e-07 2020-08-07T15:12:45.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:45.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280677+0000", 2020-08-07T15:12:45.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9652000000000002e-05 2020-08-07T15:12:45.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:45.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280756+0000", 2020-08-07T15:12:45.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017072699999999999 2020-08-07T15:12:45.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280927+0000", 2020-08-07T15:12:45.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015026530000000001 2020-08-07T15:12:45.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:45.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282430+0000", 2020-08-07T15:12:45.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4700000000000001e-07 2020-08-07T15:12:45.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:45.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282430+0000", 2020-08-07T15:12:45.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021169299999999999 2020-08-07T15:12:45.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:45.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:45.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:45.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282642+0000", 2020-08-07T15:12:45.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0581683310000001 2020-08-07T15:12:45.906 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.906 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:45.906 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.907 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:45.907 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:45.907 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:45.908 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:12:45.910 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:12:45.910 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 33 16 50883 50867 12.0409 10.1016 0.0696406 0.0103739 2020-08-07T15:12:45.912 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:45.910+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:46.011 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:46.010+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:46.098 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:46.099 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:12:46.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.100 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:52232 6.8 6:13ee2fbb:::benchmark_data_smithi074_63843_object6528:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e258)", 2020-08-07T15:12:46.100 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:46.018753+0000", 2020-08-07T15:12:46.100 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.07912342, 2020-08-07T15:12:46.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.079147771000000006, 2020-08-07T15:12:46.101 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:46.101 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:46.101 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:46.106 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:12:46.106 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:46.107 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 52232 2020-08-07T15:12:46.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.115 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:46.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:46.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:46.018753+0000", 2020-08-07T15:12:46.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:46.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:46.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:46.018753+0000", 2020-08-07T15:12:46.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5710000000000002e-05 2020-08-07T15:12:46.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:46.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:46.018788+0000", 2020-08-07T15:12:46.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0369999999999999e-06 2020-08-07T15:12:46.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:46.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:46.018791+0000", 2020-08-07T15:12:46.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.674e-06 2020-08-07T15:12:46.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:46.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:46.018798+0000", 2020-08-07T15:12:46.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.666e-06 2020-08-07T15:12:46.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:46.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:46.018801+0000", 2020-08-07T15:12:46.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002508672 2020-08-07T15:12:46.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:46.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:46.021309+0000", 2020-08-07T15:12:46.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0309999999999994e-05 2020-08-07T15:12:46.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:46.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:46.021400+0000", 2020-08-07T15:12:46.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111638 2020-08-07T15:12:46.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,3", 2020-08-07T15:12:46.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:46.021511+0000", 2020-08-07T15:12:46.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0074190690000000004 2020-08-07T15:12:46.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:46.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:46.028930+0000", 2020-08-07T15:12:46.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3799999999999996e-07 2020-08-07T15:12:46.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:46.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:46.028931+0000", 2020-08-07T15:12:46.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7457000000000002e-05 2020-08-07T15:12:46.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:46.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:46.028989+0000", 2020-08-07T15:12:46.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027996899999999998 2020-08-07T15:12:46.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:46.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:46.029269+0000", 2020-08-07T15:12:46.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010516040000000001 2020-08-07T15:12:46.144 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:46.144 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:46.145 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:46.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.147 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:52280 6.3 6:c6e63970:::benchmark_data_smithi074_63843_object6534:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e258)", 2020-08-07T15:12:46.147 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:46.056459+0000", 2020-08-07T15:12:46.148 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.041416644000000002, 2020-08-07T15:12:46.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041511575000000002, 2020-08-07T15:12:46.148 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:46.150 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:46.151 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:46.151 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:12:46.151 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:46.152 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 52280 2020-08-07T15:12:46.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.152 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:46.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:46.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:46.056459+0000", 2020-08-07T15:12:46.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:46.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:46.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:46.056459+0000", 2020-08-07T15:12:46.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5986000000000002e-05 2020-08-07T15:12:46.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:46.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:46.056485+0000", 2020-08-07T15:12:46.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.794e-06 2020-08-07T15:12:46.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:46.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:46.056489+0000", 2020-08-07T15:12:46.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.323e-06 2020-08-07T15:12:46.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:46.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:46.056496+0000", 2020-08-07T15:12:46.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9500000000000003e-06 2020-08-07T15:12:46.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:46.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:46.056500+0000", 2020-08-07T15:12:46.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000638964 2020-08-07T15:12:46.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:46.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:46.057139+0000", 2020-08-07T15:12:46.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6681000000000006e-05 2020-08-07T15:12:46.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:46.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:46.057226+0000", 2020-08-07T15:12:46.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017863699999999999 2020-08-07T15:12:46.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:12:46.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:46.057405+0000", 2020-08-07T15:12:46.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002860877 2020-08-07T15:12:46.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:46.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:46.060266+0000", 2020-08-07T15:12:46.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7999999999999995e-07 2020-08-07T15:12:46.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:46.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:46.060266+0000", 2020-08-07T15:12:46.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2299999999999997e-05 2020-08-07T15:12:46.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:46.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:46.060318+0000", 2020-08-07T15:12:46.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0078580270000000001 2020-08-07T15:12:46.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:46.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:46.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:46.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:46.068176+0000", 2020-08-07T15:12:46.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011717119 2020-08-07T15:12:46.174 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:46.174 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:46.174 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:46.175 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:46.175 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:12:46.175 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 2 2020-08-07T15:12:46.176 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:46.178 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:12:46.179 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:46.176+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:46.276 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 34 16 52607 52591 12.0829 13.4688 0.00892988 0.0103386 2020-08-07T15:12:46.278 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:46.277+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:46.379 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:46.379+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:46.398 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:46.410 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:12:46.410 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:12:46.410 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:12:46.410 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:46.412 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:12:46.412 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:46.412 INFO:teuthology.orchestra.run.smithi074.stdout: "epoch": 258, 2020-08-07T15:12:46.412 INFO:teuthology.orchestra.run.smithi074.stdout: "fsid": "a01d3e82-8d24-4288-9e42-576ab3ff4957", 2020-08-07T15:12:46.413 INFO:teuthology.orchestra.run.smithi074.stdout: "created": "2020-08-07T15:06:07.342912+0000", 2020-08-07T15:12:46.413 INFO:teuthology.orchestra.run.smithi074.stdout: "modified": "2020-08-07T15:12:45.230375+0000", 2020-08-07T15:12:46.413 INFO:teuthology.orchestra.run.smithi074.stdout: "last_up_change": "2020-08-07T15:12:05.338271+0000", 2020-08-07T15:12:46.413 INFO:teuthology.orchestra.run.smithi074.stdout: "last_in_change": "2020-08-07T15:06:17.674292+0000", 2020-08-07T15:12:46.413 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-08-07T15:12:46.414 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_num": 5801984, 2020-08-07T15:12:46.414 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_set": [ 2020-08-07T15:12:46.414 INFO:teuthology.orchestra.run.smithi074.stdout: "noscrub", 2020-08-07T15:12:46.414 INFO:teuthology.orchestra.run.smithi074.stdout: "pglog_hardlimit", 2020-08-07T15:12:46.414 INFO:teuthology.orchestra.run.smithi074.stdout: "purged_snapdirs", 2020-08-07T15:12:46.415 INFO:teuthology.orchestra.run.smithi074.stdout: "recovery_deletes", 2020-08-07T15:12:46.415 INFO:teuthology.orchestra.run.smithi074.stdout: "sortbitwise" 2020-08-07T15:12:46.415 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:12:46.415 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_version": 5, 2020-08-07T15:12:46.415 INFO:teuthology.orchestra.run.smithi074.stdout: "full_ratio": 0.94999998807907104, 2020-08-07T15:12:46.416 INFO:teuthology.orchestra.run.smithi074.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-08-07T15:12:46.416 INFO:teuthology.orchestra.run.smithi074.stdout: "nearfull_ratio": 0.85000002384185791, 2020-08-07T15:12:46.416 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_snapshot": "", 2020-08-07T15:12:46.416 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_max": 6, 2020-08-07T15:12:46.416 INFO:teuthology.orchestra.run.smithi074.stdout: "max_osd": 8, 2020-08-07T15:12:46.417 INFO:teuthology.orchestra.run.smithi074.stdout: "require_min_compat_client": "luminous", 2020-08-07T15:12:46.417 INFO:teuthology.orchestra.run.smithi074.stdout: "min_compat_client": "jewel", 2020-08-07T15:12:46.417 INFO:teuthology.orchestra.run.smithi074.stdout: "require_osd_release": "pacific", 2020-08-07T15:12:46.417 INFO:teuthology.orchestra.run.smithi074.stdout: "pools": [ 2020-08-07T15:12:46.417 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.418 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 1, 2020-08-07T15:12:46.418 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "device_health_metrics", 2020-08-07T15:12:46.418 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:06:10.648876+0000", 2020-08-07T15:12:46.418 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:12:46.418 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:12:46.419 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:12:46.419 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:12:46.420 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:12:46.462 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:12:46.462 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:12:46.463 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:12:46.463 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 41, 2020-08-07T15:12:46.463 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 41, 2020-08-07T15:12:46.463 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 41, 2020-08-07T15:12:46.463 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 41, 2020-08-07T15:12:46.464 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 41, 2020-08-07T15:12:46.464 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:12:46.464 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:12:46.464 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:12:46.464 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:12:46.465 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:12:46.465 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:12:46.465 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:12:46.465 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.465 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "246", 2020-08-07T15:12:46.477 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:12:46.477 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:12:46.478 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "244", 2020-08-07T15:12:46.478 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:12:46.478 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:12:46.478 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:12:46.479 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:12:46.479 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:12:46.479 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:12:46.479 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:12:46.479 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:12:46.479 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:12:46.480 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:12:46.480 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:12:46.480 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:12:46.480 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:12:46.480 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:12:46.481 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:12:46.481 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:12:46.481 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:12:46.481 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:12:46.481 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:12:46.502 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:12:46.502 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:12:46.503 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:12:46.503 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:12:46.503 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.503 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:12:46.503 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:12:46.504 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:12:46.504 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:12:46.504 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:12:46.504 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:12:46.504 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:12:46.505 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:12:46.505 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:12:46.505 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:12:46.505 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:12:46.505 INFO:teuthology.orchestra.run.smithi074.stdout: "options": { 2020-08-07T15:12:46.511 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_min": 1 2020-08-07T15:12:46.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.512 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:12:46.512 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_devicehealth": {} 2020-08-07T15:12:46.512 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.513 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.513 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 2, 2020-08-07T15:12:46.513 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "rbd", 2020-08-07T15:12:46.513 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:06:21.474648+0000", 2020-08-07T15:12:46.513 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:12:46.514 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:12:46.514 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:12:46.514 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:12:46.514 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:12:46.515 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:12:46.515 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:12:46.515 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:12:46.515 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 8, 2020-08-07T15:12:46.515 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 8, 2020-08-07T15:12:46.516 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 8, 2020-08-07T15:12:46.516 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 8, 2020-08-07T15:12:46.516 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 8, 2020-08-07T15:12:46.516 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:12:46.516 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:12:46.516 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:12:46.517 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:12:46.517 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:12:46.517 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:12:46.517 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:12:46.534 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.534 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "19", 2020-08-07T15:12:46.534 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:12:46.534 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:12:46.535 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "0", 2020-08-07T15:12:46.535 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:12:46.535 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:12:46.535 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:12:46.536 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:12:46.536 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:12:46.536 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:12:46.536 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:12:46.536 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:12:46.537 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:12:46.537 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:12:46.537 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:12:46.537 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:12:46.537 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:12:46.538 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:12:46.538 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:12:46.538 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:12:46.538 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:12:46.538 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:12:46.539 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:12:46.539 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:12:46.539 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:12:46.539 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:12:46.539 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:12:46.540 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.540 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:12:46.540 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:12:46.540 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:12:46.541 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:12:46.541 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:12:46.541 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:12:46.541 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:12:46.541 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:12:46.550 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:12:46.550 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:12:46.550 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:12:46.551 INFO:teuthology.orchestra.run.smithi074.stdout: "options": {}, 2020-08-07T15:12:46.551 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:12:46.551 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd": {} 2020-08-07T15:12:46.551 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.551 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.552 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 6, 2020-08-07T15:12:46.552 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "unique_pool_3", 2020-08-07T15:12:46.552 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:12:08.847598+0000", 2020-08-07T15:12:46.552 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:12:46.553 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:12:46.553 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:12:46.553 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:12:46.553 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:12:46.554 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:12:46.554 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:12:46.554 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:12:46.554 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 16, 2020-08-07T15:12:46.554 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 16, 2020-08-07T15:12:46.555 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 16, 2020-08-07T15:12:46.555 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 16, 2020-08-07T15:12:46.555 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 16, 2020-08-07T15:12:46.555 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:12:46.555 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:12:46.556 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:12:46.556 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:12:46.556 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:12:46.556 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:12:46.556 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:12:46.557 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.557 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "232", 2020-08-07T15:12:46.557 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:12:46.557 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:12:46.565 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "0", 2020-08-07T15:12:46.566 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:12:46.566 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:12:46.567 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:12:46.568 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:12:46.568 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:12:46.568 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:12:46.569 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:12:46.569 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:12:46.570 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:12:46.574 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:12:46.574 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:12:46.575 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:12:46.575 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:12:46.575 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:12:46.576 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:12:46.576 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:12:46.576 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:12:46.577 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:12:46.577 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:12:46.577 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:12:46.578 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:12:46.578 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:12:46.578 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:12:46.579 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.579 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:12:46.579 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:12:46.580 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:12:46.580 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:12:46.580 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:12:46.581 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:12:46.581 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:12:46.581 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:12:46.602 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:12:46.602 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:12:46.602 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:12:46.603 INFO:teuthology.orchestra.run.smithi074.stdout: "options": {}, 2020-08-07T15:12:46.603 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:12:46.603 INFO:teuthology.orchestra.run.smithi074.stdout: "rados": {} 2020-08-07T15:12:46.603 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.603 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.604 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:12:46.604 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:12:46.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.604 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 0, 2020-08-07T15:12:46.604 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "fc1ebb23-3ef4-4fc0-b395-382e385885a3", 2020-08-07T15:12:46.605 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:12:46.605 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:12:46.605 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:12:46.605 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0, 2020-08-07T15:12:46.606 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:12:46.606 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:12:46.606 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:12:46.606 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 248, 2020-08-07T15:12:46.607 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:12:46.607 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:12:46.607 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:12:46.607 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.608 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.608 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.608 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6801", 2020-08-07T15:12:46.608 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:12:46.608 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.609 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.609 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.609 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:12:46.609 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.609 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.610 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.610 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6802", 2020-08-07T15:12:46.610 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:12:46.611 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.611 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.611 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.612 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:12:46.612 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.612 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.613 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.613 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6804", 2020-08-07T15:12:46.613 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:12:46.623 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.623 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.623 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.624 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:12:46.624 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.624 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.625 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.625 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6803", 2020-08-07T15:12:46.626 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:12:46.626 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.626 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.627 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.627 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6801/31577", 2020-08-07T15:12:46.627 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6802/31577", 2020-08-07T15:12:46.628 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6804/31577", 2020-08-07T15:12:46.628 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6803/31577", 2020-08-07T15:12:46.628 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:12:46.629 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:12:46.629 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:12:46.629 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.638 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.638 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.639 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 1, 2020-08-07T15:12:46.639 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "1fbe530a-63ce-4dc8-82ab-18793d6ebc0b", 2020-08-07T15:12:46.640 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:12:46.640 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:12:46.640 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:12:46.641 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0.7196044921875, 2020-08-07T15:12:46.641 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 13, 2020-08-07T15:12:46.642 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 190, 2020-08-07T15:12:46.642 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 227, 2020-08-07T15:12:46.643 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 249, 2020-08-07T15:12:46.643 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 191, 2020-08-07T15:12:46.643 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:12:46.644 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:12:46.644 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.675 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.676 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6809", 2020-08-07T15:12:46.677 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 62587 2020-08-07T15:12:46.677 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.678 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.678 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.679 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:12:46.679 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.680 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.680 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6810", 2020-08-07T15:12:46.681 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 62587 2020-08-07T15:12:46.681 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.698 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.698 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:12:46.698 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.699 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.699 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.699 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6812", 2020-08-07T15:12:46.699 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 62587 2020-08-07T15:12:46.699 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.700 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.700 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.700 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:12:46.700 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.701 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.701 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6811", 2020-08-07T15:12:46.701 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 62587 2020-08-07T15:12:46.701 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.701 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.714 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.714 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6809/62587", 2020-08-07T15:12:46.714 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6810/62587", 2020-08-07T15:12:46.715 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6812/62587", 2020-08-07T15:12:46.715 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6811/62587", 2020-08-07T15:12:46.715 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:12:46.715 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:12:46.715 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:12:46.716 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.716 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.716 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.716 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 2, 2020-08-07T15:12:46.716 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "d6fe4338-3e11-4f73-9cbb-e1b046ce286a", 2020-08-07T15:12:46.716 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:12:46.717 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:12:46.717 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:12:46.717 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:12:46.717 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:12:46.748 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:12:46.758 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:12:46.758 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 256, 2020-08-07T15:12:46.759 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:12:46.759 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:12:46.759 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:12:46.760 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.760 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.760 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.761 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6805", 2020-08-07T15:12:46.761 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:12:46.770 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.770 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.771 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.771 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:12:46.771 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.771 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.771 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.772 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6806", 2020-08-07T15:12:46.772 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:12:46.772 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.772 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.772 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.772 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:12:46.773 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.773 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.773 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.773 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6808", 2020-08-07T15:12:46.776 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:12:46.784 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.784 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.785 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.785 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:12:46.785 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.786 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.786 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6807", 2020-08-07T15:12:46.793 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:12:46.794 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.794 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.794 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.795 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6805/31576", 2020-08-07T15:12:46.795 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6806/31576", 2020-08-07T15:12:46.795 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6808/31576", 2020-08-07T15:12:46.796 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6807/31576", 2020-08-07T15:12:46.796 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:12:46.796 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:12:46.796 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:12:46.797 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.797 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.797 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.797 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 3, 2020-08-07T15:12:46.798 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9", 2020-08-07T15:12:46.798 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:12:46.798 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:12:46.798 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:12:46.799 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:12:46.799 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:12:46.799 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:12:46.800 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:12:46.800 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 249, 2020-08-07T15:12:46.800 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:12:46.800 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:12:46.801 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:12:46.801 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.802 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.802 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.802 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6813", 2020-08-07T15:12:46.803 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:12:46.803 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.803 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.804 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:12:46.804 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.805 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.805 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6814", 2020-08-07T15:12:46.805 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:12:46.806 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.806 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.806 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.807 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:12:46.807 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.808 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.808 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6816", 2020-08-07T15:12:46.808 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:12:46.809 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.809 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.810 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:12:46.810 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.810 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.811 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.811 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6815", 2020-08-07T15:12:46.811 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:12:46.812 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.812 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.813 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6813/31579", 2020-08-07T15:12:46.813 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6814/31579", 2020-08-07T15:12:46.814 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6816/31579", 2020-08-07T15:12:46.815 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6815/31579", 2020-08-07T15:12:46.815 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:12:46.815 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:12:46.816 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:12:46.816 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.817 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 4, 2020-08-07T15:12:46.818 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "01aadc13-29c5-4ddd-9bb9-de24b2b20412", 2020-08-07T15:12:46.818 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:12:46.819 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:12:46.819 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:12:46.820 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0, 2020-08-07T15:12:46.820 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:12:46.820 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:12:46.821 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:12:46.821 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 248, 2020-08-07T15:12:46.822 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:12:46.822 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:12:46.822 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:12:46.822 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.823 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.823 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6800", 2020-08-07T15:12:46.824 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:12:46.824 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.824 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.825 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:12:46.825 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.825 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.826 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6801", 2020-08-07T15:12:46.826 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:12:46.826 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.827 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.827 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:12:46.828 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.828 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.828 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6803", 2020-08-07T15:12:46.829 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:12:46.829 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.830 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.831 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.831 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:12:46.831 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.832 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.832 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6802", 2020-08-07T15:12:46.832 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:12:46.833 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.835 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.835 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6800/31545", 2020-08-07T15:12:46.836 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6801/31545", 2020-08-07T15:12:46.836 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6803/31545", 2020-08-07T15:12:46.836 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6802/31545", 2020-08-07T15:12:46.837 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:12:46.837 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:12:46.838 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:12:46.838 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.839 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 5, 2020-08-07T15:12:46.839 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "f5556a82-73b4-47b1-8e29-1d2fe031d8fd", 2020-08-07T15:12:46.840 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:12:46.840 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:12:46.840 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:12:46.841 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:12:46.841 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:12:46.841 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:12:46.842 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:12:46.842 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 257, 2020-08-07T15:12:46.843 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:12:46.843 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:12:46.843 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:12:46.844 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.844 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.845 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6804", 2020-08-07T15:12:46.845 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:12:46.845 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.846 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.846 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:12:46.847 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.847 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.847 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.848 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6805", 2020-08-07T15:12:46.848 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:12:46.848 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.849 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.849 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.849 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:12:46.850 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.851 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.851 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6807", 2020-08-07T15:12:46.852 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:12:46.852 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.853 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.854 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:12:46.854 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.854 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.855 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.855 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6806", 2020-08-07T15:12:46.856 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:12:46.856 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.857 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.857 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6804/31547", 2020-08-07T15:12:46.858 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6805/31547", 2020-08-07T15:12:46.858 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6807/31547", 2020-08-07T15:12:46.859 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6806/31547", 2020-08-07T15:12:46.859 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:12:46.860 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:12:46.860 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:12:46.861 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.862 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 6, 2020-08-07T15:12:46.862 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "9b98b337-e1cc-43cd-9e9b-f21ae3f20899", 2020-08-07T15:12:46.863 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:12:46.863 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:12:46.863 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:12:46.863 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:12:46.864 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:12:46.864 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:12:46.864 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 14, 2020-08-07T15:12:46.865 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 249, 2020-08-07T15:12:46.865 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:12:46.865 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:12:46.866 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:12:46.866 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.866 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.867 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.867 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6808", 2020-08-07T15:12:46.867 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:12:46.868 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.868 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.868 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.869 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:12:46.870 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.870 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.871 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.871 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6809", 2020-08-07T15:12:46.871 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:12:46.872 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.872 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.873 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:12:46.873 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.874 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.874 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6811", 2020-08-07T15:12:46.874 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:12:46.875 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.875 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.876 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:12:46.877 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.877 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.877 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.878 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6810", 2020-08-07T15:12:46.878 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:12:46.878 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.879 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.879 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.879 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6808/31546", 2020-08-07T15:12:46.880 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6809/31546", 2020-08-07T15:12:46.880 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6811/31546", 2020-08-07T15:12:46.880 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6810/31546", 2020-08-07T15:12:46.880 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:12:46.881 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:12:46.881 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:12:46.881 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.881 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.881 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.882 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 7, 2020-08-07T15:12:46.882 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "777662d8-bd34-48e8-83e1-24ce9e61f14b", 2020-08-07T15:12:46.882 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:12:46.882 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:12:46.883 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:12:46.883 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:12:46.883 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:12:46.883 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:12:46.883 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:12:46.883 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 244, 2020-08-07T15:12:46.884 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:12:46.884 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:12:46.884 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:12:46.884 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.884 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.885 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.885 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6812", 2020-08-07T15:12:46.885 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:12:46.885 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.885 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.886 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:12:46.886 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.886 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.886 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.887 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6813", 2020-08-07T15:12:46.887 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:12:46.887 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.887 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.888 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:12:46.888 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.888 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.888 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6815", 2020-08-07T15:12:46.888 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:12:46.889 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.889 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.889 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:12:46.889 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:46.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.890 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:46.890 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6814", 2020-08-07T15:12:46.890 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:12:46.890 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.891 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.891 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6812/31570", 2020-08-07T15:12:46.891 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6813/31570", 2020-08-07T15:12:46.891 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6815/31570", 2020-08-07T15:12:46.892 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6814/31570", 2020-08-07T15:12:46.892 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:12:46.892 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:12:46.892 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:12:46.892 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:46.892 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.893 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:12:46.893 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_xinfo": [ 2020-08-07T15:12:46.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.893 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 0, 2020-08-07T15:12:46.893 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:12:46.894 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:12:46.894 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:12:46.895 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:12:46.895 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:12:46.895 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:12:19.121056+0000", 2020-08-07T15:12:46.895 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:12:46.895 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.895 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.896 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 1, 2020-08-07T15:12:46.896 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:12:46.896 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:12:46.896 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:12:46.896 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:12:46.897 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:12:46.897 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:12:05.850993+0000", 2020-08-07T15:12:46.897 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 191 2020-08-07T15:12:46.897 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.897 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.898 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 2, 2020-08-07T15:12:46.898 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:12:46.898 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:12:46.898 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:12:46.898 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:12:46.899 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:12:46.899 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:12:17.947567+0000", 2020-08-07T15:12:46.899 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:12:46.899 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.899 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.899 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 3, 2020-08-07T15:12:46.900 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:12:46.900 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:12:46.900 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:12:46.900 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:12:46.900 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:12:46.901 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:12:19.857070+0000", 2020-08-07T15:12:46.901 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:12:46.901 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.910 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 4, 2020-08-07T15:12:46.910 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:12:46.911 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:12:46.911 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:12:46.911 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:12:46.911 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:12:46.911 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:12:18.727770+0000", 2020-08-07T15:12:46.912 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:12:46.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.912 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 5, 2020-08-07T15:12:46.912 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:12:46.913 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:12:46.913 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:12:46.913 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:12:46.913 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:12:46.914 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:12:19.039747+0000", 2020-08-07T15:12:46.914 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:12:46.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.914 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 6, 2020-08-07T15:12:46.915 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:12:46.915 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:12:46.915 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:12:46.915 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:12:46.915 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:12:46.916 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:12:17.971384+0000", 2020-08-07T15:12:46.916 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:12:46.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:46.916 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 7, 2020-08-07T15:12:46.917 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:12:46.917 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:12:46.917 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:12:46.917 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:12:46.917 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:12:46.917 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:12:17.214169+0000", 2020-08-07T15:12:46.918 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:12:46.918 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.918 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:12:46.918 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_upmap": [], 2020-08-07T15:12:46.918 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_upmap_items": [], 2020-08-07T15:12:46.919 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_temp": [], 2020-08-07T15:12:46.919 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_temp": [], 2020-08-07T15:12:46.919 INFO:teuthology.orchestra.run.smithi074.stdout: "blacklist": {}, 2020-08-07T15:12:46.919 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profiles": { 2020-08-07T15:12:46.919 INFO:teuthology.orchestra.run.smithi074.stdout: "default": { 2020-08-07T15:12:46.920 INFO:teuthology.orchestra.run.smithi074.stdout: "crush-failure-domain": "osd", 2020-08-07T15:12:46.920 INFO:teuthology.orchestra.run.smithi074.stdout: "k": "2", 2020-08-07T15:12:46.920 INFO:teuthology.orchestra.run.smithi074.stdout: "m": "1", 2020-08-07T15:12:46.920 INFO:teuthology.orchestra.run.smithi074.stdout: "plugin": "jerasure", 2020-08-07T15:12:46.920 INFO:teuthology.orchestra.run.smithi074.stdout: "ruleset-failure-domain": "osd", 2020-08-07T15:12:46.921 INFO:teuthology.orchestra.run.smithi074.stdout: "technique": "reed_sol_van" 2020-08-07T15:12:46.921 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:46.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:46.921 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps_queue": [], 2020-08-07T15:12:46.921 INFO:teuthology.orchestra.run.smithi074.stdout: "new_removed_snaps": [], 2020-08-07T15:12:46.923 INFO:teuthology.orchestra.run.smithi074.stdout: "new_purged_snaps": [], 2020-08-07T15:12:46.923 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_node_flags": {}, 2020-08-07T15:12:46.924 INFO:teuthology.orchestra.run.smithi074.stdout: "device_class_flags": {} 2020-08-07T15:12:46.924 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:46.924 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:12:46.926 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:12:46.927 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:46.925+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:46.928 INFO:tasks.thrashosds.thrasher:j is {'epoch': 258, 'fsid': 'a01d3e82-8d24-4288-9e42-576ab3ff4957', 'created': '2020-08-07T15:06:07.342912+0000', 'modified': '2020-08-07T15:12:45.230375+0000', 'last_up_change': '2020-08-07T15:12:05.338271+0000', 'last_in_change': '2020-08-07T15:06:17.674292+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 6, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-08-07T15:06:10.648876+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 41, 'pg_placement_num': 41, 'pg_placement_num_target': 41, 'pg_num_target': 41, 'pg_num_pending': 41, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '246', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '244', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-08-07T15:06:21.474648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 6, 'pool_name': 'unique_pool_3', 'create_time': '2020-08-07T15:12:08.847598+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '232', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': 'fc1ebb23-3ef4-4fc0-b395-382e385885a3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 248, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6801', 'nonce': 31577}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6802', 'nonce': 31577}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6804', 'nonce': 31577}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6803', 'nonce': 31577}]}, 'public_addr': '172.21.15.74:6801/31577', 'cluster_addr': '172.21.15.74:6802/31577', 'heartbeat_back_addr': '172.21.15.74:6804/31577', 'heartbeat_front_addr': '172.21.15.74:6803/31577', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '1fbe530a-63ce-4dc8-82ab-18793d6ebc0b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.7196044921875, 'last_clean_begin': 13, 'last_clean_end': 190, 'up_from': 227, 'up_thru': 249, 'down_at': 191, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6809', 'nonce': 62587}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6810', 'nonce': 62587}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6812', 'nonce': 62587}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6811', 'nonce': 62587}]}, 'public_addr': '172.21.15.74:6809/62587', 'cluster_addr': '172.21.15.74:6810/62587', 'heartbeat_back_addr': '172.21.15.74:6812/62587', 'heartbeat_front_addr': '172.21.15.74:6811/62587', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd6fe4338-3e11-4f73-9cbb-e1b046ce286a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 256, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6805', 'nonce': 31576}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6806', 'nonce': 31576}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6808', 'nonce': 31576}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6807', 'nonce': 31576}]}, 'public_addr': '172.21.15.74:6805/31576', 'cluster_addr': '172.21.15.74:6806/31576', 'heartbeat_back_addr': '172.21.15.74:6808/31576', 'heartbeat_front_addr': '172.21.15.74:6807/31576', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 249, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6813', 'nonce': 31579}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6814', 'nonce': 31579}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6816', 'nonce': 31579}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6815', 'nonce': 31579}]}, 'public_addr': '172.21.15.74:6813/31579', 'cluster_addr': '172.21.15.74:6814/31579', 'heartbeat_back_addr': '172.21.15.74:6816/31579', 'heartbeat_front_addr': '172.21.15.74:6815/31579', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '01aadc13-29c5-4ddd-9bb9-de24b2b20412', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 248, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6800', 'nonce': 31545}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6801', 'nonce': 31545}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6803', 'nonce': 31545}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6802', 'nonce': 31545}]}, 'public_addr': '172.21.15.96:6800/31545', 'cluster_addr': '172.21.15.96:6801/31545', 'heartbeat_back_addr': '172.21.15.96:6803/31545', 'heartbeat_front_addr': '172.21.15.96:6802/31545', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'f5556a82-73b4-47b1-8e29-1d2fe031d8fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 257, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6804', 'nonce': 31547}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6805', 'nonce': 31547}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6807', 'nonce': 31547}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6806', 'nonce': 31547}]}, 'public_addr': '172.21.15.96:6804/31547', 'cluster_addr': '172.21.15.96:6805/31547', 'heartbeat_back_addr': '172.21.15.96:6807/31547', 'heartbeat_front_addr': '172.21.15.96:6806/31547', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9b98b337-e1cc-43cd-9e9b-f21ae3f20899', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 249, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6808', 'nonce': 31546}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6809', 'nonce': 31546}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6811', 'nonce': 31546}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6810', 'nonce': 31546}]}, 'public_addr': '172.21.15.96:6808/31546', 'cluster_addr': '172.21.15.96:6809/31546', 'heartbeat_back_addr': '172.21.15.96:6811/31546', 'heartbeat_front_addr': '172.21.15.96:6810/31546', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '777662d8-bd34-48e8-83e1-24ce9e61f14b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 244, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6812', 'nonce': 31570}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6813', 'nonce': 31570}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6815', 'nonce': 31570}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6814', 'nonce': 31570}]}, 'public_addr': '172.21.15.96:6812/31570', 'cluster_addr': '172.21.15.96:6813/31570', 'heartbeat_back_addr': '172.21.15.96:6815/31570', 'heartbeat_front_addr': '172.21.15.96:6814/31570', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:12:19.121056+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:12:05.850993+0000', 'dead_epoch': 191}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:12:17.947567+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:12:19.857070+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:12:18.727770+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:12:19.039747+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:12:17.971384+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:12:17.214169+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blacklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}} 2020-08-07T15:12:46.929 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:12:47.028 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:47.027+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:47.117 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:47.118 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:12:47.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:12:47.118 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:12:47.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.118 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15576 3.e 3:727724a5:::benchmark_data_smithi074_35447_object1946:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:12:47.119 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:12:47.119 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 360.44384581600002, 2020-08-07T15:12:47.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999, 2020-08-07T15:12:47.119 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:47.120 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:47.120 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:47.120 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:12:47.120 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:12:47.120 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15576 2020-08-07T15:12:47.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.134 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:47.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:47.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:12:47.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:47.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:47.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:12:47.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7156e-05 2020-08-07T15:12:47.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:47.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666682+0000", 2020-08-07T15:12:47.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0209999999999999e-06 2020-08-07T15:12:47.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:47.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666687+0000", 2020-08-07T15:12:47.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0730999999999999e-05 2020-08-07T15:12:47.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:47.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666698+0000", 2020-08-07T15:12:47.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2020-08-07T15:12:47.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:47.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666702+0000", 2020-08-07T15:12:47.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023978459999999999 2020-08-07T15:12:47.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:47.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669100+0000", 2020-08-07T15:12:47.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9664000000000001e-05 2020-08-07T15:12:47.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:47.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669190+0000", 2020-08-07T15:12:47.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015957100000000001 2020-08-07T15:12:47.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:12:47.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669350+0000", 2020-08-07T15:12:47.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016470149999999999 2020-08-07T15:12:47.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:47.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:12:47.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1299999999999999e-07 2020-08-07T15:12:47.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:47.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:12:47.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5269000000000002e-05 2020-08-07T15:12:47.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:47.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.671022+0000", 2020-08-07T15:12:47.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031414400000000001 2020-08-07T15:12:47.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:47.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.674163+0000", 2020-08-07T15:12:47.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45641417400000001 2020-08-07T15:12:47.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:47.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:12:47.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0399999999999995e-07 2020-08-07T15:12:47.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:47.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:12:47.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3343999999999997e-05 2020-08-07T15:12:47.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:47.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130622+0000", 2020-08-07T15:12:47.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999 2020-08-07T15:12:47.161 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.161 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:47.161 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.166 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106248 3.f 3:f950fd9d:::benchmark_data_smithi074_35447_object13280:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:12:47.167 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:12:47.167 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 299.75031627599998, 2020-08-07T15:12:47.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002, 2020-08-07T15:12:47.167 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:47.167 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:47.168 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:47.168 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:12:47.168 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:12:47.168 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106248 2020-08-07T15:12:47.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.169 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:47.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:47.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:12:47.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:47.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:47.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:12:47.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7184e-05 2020-08-07T15:12:47.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:47.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360222+0000", 2020-08-07T15:12:47.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0906e-05 2020-08-07T15:12:47.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:47.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360233+0000", 2020-08-07T15:12:47.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2972999999999999e-05 2020-08-07T15:12:47.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:47.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360246+0000", 2020-08-07T15:12:47.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0114e-05 2020-08-07T15:12:47.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:47.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360256+0000", 2020-08-07T15:12:47.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1303999999999997e-05 2020-08-07T15:12:47.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:47.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360287+0000", 2020-08-07T15:12:47.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3798000000000001e-05 2020-08-07T15:12:47.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:47.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360351+0000", 2020-08-07T15:12:47.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111308 2020-08-07T15:12:47.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:12:47.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360462+0000", 2020-08-07T15:12:47.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003480674 2020-08-07T15:12:47.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:47.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:12:47.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0900000000000001e-07 2020-08-07T15:12:47.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:47.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:12:47.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8899999999999994e-05 2020-08-07T15:12:47.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:47.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.364012+0000", 2020-08-07T15:12:47.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35291637199999998 2020-08-07T15:12:47.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:47.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716929+0000", 2020-08-07T15:12:47.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.076244233999999994 2020-08-07T15:12:47.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:47.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793173+0000", 2020-08-07T15:12:47.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6109999999999999e-06 2020-08-07T15:12:47.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:47.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793175+0000", 2020-08-07T15:12:47.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7492999999999999e-05 2020-08-07T15:12:47.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:47.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793242+0000", 2020-08-07T15:12:47.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002 2020-08-07T15:12:47.228 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.229 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:47.229 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.229 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106248 3.f e59/20 ondisk, result = 0)", 2020-08-07T15:12:47.249 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:12:47.249 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 299.74505299100002, 2020-08-07T15:12:47.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001, 2020-08-07T15:12:47.250 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:47.250 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:47.250 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:47.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:47.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:12:47.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:47.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:47.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:12:47.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7518999999999999e-05 2020-08-07T15:12:47.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:47.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365445+0000", 2020-08-07T15:12:47.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-06 2020-08-07T15:12:47.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:47.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365449+0000", 2020-08-07T15:12:47.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42764147600000002 2020-08-07T15:12:47.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:47.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793091+0000", 2020-08-07T15:12:47.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6500000000000002e-06 2020-08-07T15:12:47.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:47.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793099+0000", 2020-08-07T15:12:47.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3664e-05 2020-08-07T15:12:47.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:47.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793153+0000", 2020-08-07T15:12:47.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1078e-05 2020-08-07T15:12:47.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:47.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793164+0000", 2020-08-07T15:12:47.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3404000000000003e-05 2020-08-07T15:12:47.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:47.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793248+0000", 2020-08-07T15:12:47.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001 2020-08-07T15:12:47.261 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.262 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:47.262 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.262 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:107952 3.d e61/35)", 2020-08-07T15:12:47.262 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:12:47.263 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 296.78412328299999, 2020-08-07T15:12:47.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001, 2020-08-07T15:12:47.263 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:47.263 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:47.263 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:47.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:47.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:12:47.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:47.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:47.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:12:47.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8826000000000003e-05 2020-08-07T15:12:47.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:47.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326406+0000", 2020-08-07T15:12:47.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9199999999999998e-06 2020-08-07T15:12:47.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:47.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326413+0000", 2020-08-07T15:12:47.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75082853500000002 2020-08-07T15:12:47.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:47.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077242+0000", 2020-08-07T15:12:47.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1479999999999999e-06 2020-08-07T15:12:47.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:47.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077246+0000", 2020-08-07T15:12:47.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3331999999999998e-05 2020-08-07T15:12:47.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:47.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077289+0000", 2020-08-07T15:12:47.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8977000000000001e-05 2020-08-07T15:12:47.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:47.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077308+0000", 2020-08-07T15:12:47.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001 2020-08-07T15:12:47.271 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.272 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:47.272 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.272 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31696 4.0 4:0335eed9:::benchmark_data_smithi074_44586_object3961:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e100)", 2020-08-07T15:12:47.273 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:12:47.273 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 239.00621572399999, 2020-08-07T15:12:47.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237, 2020-08-07T15:12:47.273 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:47.273 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:47.274 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:47.274 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:47.274 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:47.274 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31696 2020-08-07T15:12:47.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.274 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:47.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:47.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:12:47.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:47.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:47.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:12:47.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7654e-05 2020-08-07T15:12:47.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:47.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104293+0000", 2020-08-07T15:12:47.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.061e-06 2020-08-07T15:12:47.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:47.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104296+0000", 2020-08-07T15:12:47.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0859999999999998e-06 2020-08-07T15:12:47.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:47.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104302+0000", 2020-08-07T15:12:47.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8030000000000001e-06 2020-08-07T15:12:47.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:47.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104305+0000", 2020-08-07T15:12:47.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037634769999999999 2020-08-07T15:12:47.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:47.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108068+0000", 2020-08-07T15:12:47.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8814999999999998e-05 2020-08-07T15:12:47.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:47.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108137+0000", 2020-08-07T15:12:47.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111455 2020-08-07T15:12:47.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:12:47.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108248+0000", 2020-08-07T15:12:47.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028756820000000001 2020-08-07T15:12:47.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:47.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111124+0000", 2020-08-07T15:12:47.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061054499999999999 2020-08-07T15:12:47.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:47.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111735+0000", 2020-08-07T15:12:47.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.423966167 2020-08-07T15:12:47.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:47.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535701+0000", 2020-08-07T15:12:47.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4929999999999999e-06 2020-08-07T15:12:47.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:47.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535702+0000", 2020-08-07T15:12:47.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2571999999999997e-05 2020-08-07T15:12:47.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:47.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535755+0000", 2020-08-07T15:12:47.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1427000000000003e-05 2020-08-07T15:12:47.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:47.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535786+0000", 2020-08-07T15:12:47.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237 2020-08-07T15:12:47.302 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.302 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:47.302 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.303 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:31696 4.0 e100/81 ondisk, result = 0)", 2020-08-07T15:12:47.303 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:12:47.303 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 238.996601166, 2020-08-07T15:12:47.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954, 2020-08-07T15:12:47.304 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:47.304 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:47.304 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:47.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:47.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:12:47.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:47.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:47.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:12:47.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1185000000000001e-05 2020-08-07T15:12:47.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:47.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113901+0000", 2020-08-07T15:12:47.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2930000000000002e-06 2020-08-07T15:12:47.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:47.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113905+0000", 2020-08-07T15:12:47.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42168640400000001 2020-08-07T15:12:47.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:47.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535591+0000", 2020-08-07T15:12:47.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2820000000000002e-06 2020-08-07T15:12:47.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:47.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535598+0000", 2020-08-07T15:12:47.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6578999999999999e-05 2020-08-07T15:12:47.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:47.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535684+0000", 2020-08-07T15:12:47.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8449999999999992e-06 2020-08-07T15:12:47.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:47.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535693+0000", 2020-08-07T15:12:47.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7366000000000003e-05 2020-08-07T15:12:47.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:47.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535791+0000", 2020-08-07T15:12:47.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954 2020-08-07T15:12:47.313 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.313 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:47.314 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.314 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:37024 4.0 e106/81)", 2020-08-07T15:12:47.314 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:12:47.315 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 232.88181227199999, 2020-08-07T15:12:47.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999, 2020-08-07T15:12:47.315 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:47.315 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:47.315 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:47.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:47.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:12:47.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:47.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:47.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:12:47.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4926000000000001e-05 2020-08-07T15:12:47.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:47.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228683+0000", 2020-08-07T15:12:47.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1280000000000002e-06 2020-08-07T15:12:47.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:47.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228686+0000", 2020-08-07T15:12:47.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69740012100000004 2020-08-07T15:12:47.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:47.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926086+0000", 2020-08-07T15:12:47.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000004e-06 2020-08-07T15:12:47.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:47.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926090+0000", 2020-08-07T15:12:47.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113526 2020-08-07T15:12:47.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:47.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926204+0000", 2020-08-07T15:12:47.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8824000000000001e-05 2020-08-07T15:12:47.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:47.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926222+0000", 2020-08-07T15:12:47.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999 2020-08-07T15:12:47.336 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.336 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:47.336 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.337 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42503 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2615, mlcod=112'2607)", 2020-08-07T15:12:47.337 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:12:47.337 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 226.814240356, 2020-08-07T15:12:47.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998, 2020-08-07T15:12:47.338 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:47.347 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:47.348 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:47.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:47.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:12:47.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:47.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:47.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:12:47.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270999999999998e-05 2020-08-07T15:12:47.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:47.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296292+0000", 2020-08-07T15:12:47.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4729999999999997e-06 2020-08-07T15:12:47.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:47.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296298+0000", 2020-08-07T15:12:47.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.448795049 2020-08-07T15:12:47.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:47.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745093+0000", 2020-08-07T15:12:47.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0140000000000001e-06 2020-08-07T15:12:47.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:47.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745100+0000", 2020-08-07T15:12:47.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2596999999999998e-05 2020-08-07T15:12:47.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:47.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745153+0000", 2020-08-07T15:12:47.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1662000000000003e-05 2020-08-07T15:12:47.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:47.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745184+0000", 2020-08-07T15:12:47.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093174800000000004 2020-08-07T15:12:47.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:47.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746116+0000", 2020-08-07T15:12:47.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0926999999999994e-05 2020-08-07T15:12:47.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:47.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746187+0000", 2020-08-07T15:12:47.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998 2020-08-07T15:12:47.369 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.370 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:47.370 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.370 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42504 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2616, mlcod=113'2609)", 2020-08-07T15:12:47.371 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:12:47.371 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 226.81090346299999, 2020-08-07T15:12:47.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998, 2020-08-07T15:12:47.371 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:47.371 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:47.372 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:47.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:47.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:12:47.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:47.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:47.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:12:47.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2154000000000002e-05 2020-08-07T15:12:47.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:47.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299629+0000", 2020-08-07T15:12:47.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1929999999999998e-06 2020-08-07T15:12:47.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:47.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299636+0000", 2020-08-07T15:12:47.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44766315400000001 2020-08-07T15:12:47.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:47.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747299+0000", 2020-08-07T15:12:47.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5889999999999999e-06 2020-08-07T15:12:47.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:47.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747305+0000", 2020-08-07T15:12:47.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3094999999999999e-05 2020-08-07T15:12:47.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:47.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747358+0000", 2020-08-07T15:12:47.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3283e-05 2020-08-07T15:12:47.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:47.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747392+0000", 2020-08-07T15:12:47.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015319509999999999 2020-08-07T15:12:47.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:47.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748924+0000", 2020-08-07T15:12:47.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5235999999999996e-05 2020-08-07T15:12:47.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:47.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748999+0000", 2020-08-07T15:12:47.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998 2020-08-07T15:12:47.387 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.388 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:47.388 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.389 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42520 4.8 e113/81 4:13071de5:::benchmark_data_smithi074_44586_object5314:head v 113'2624, mlcod=113'2618)", 2020-08-07T15:12:47.389 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:12:47.389 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 226.344959254, 2020-08-07T15:12:47.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001, 2020-08-07T15:12:47.389 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:47.390 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:47.390 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:47.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:47.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:12:47.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:47.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:47.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:12:47.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3967999999999995e-05 2020-08-07T15:12:47.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:47.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765585+0000", 2020-08-07T15:12:47.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0949999999999998e-06 2020-08-07T15:12:47.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:47.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765593+0000", 2020-08-07T15:12:47.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52703464200000005 2020-08-07T15:12:47.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:47.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292627+0000", 2020-08-07T15:12:47.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6929999999999996e-06 2020-08-07T15:12:47.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:47.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292636+0000", 2020-08-07T15:12:47.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9292000000000001e-05 2020-08-07T15:12:47.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:47.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292695+0000", 2020-08-07T15:12:47.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5395999999999999e-05 2020-08-07T15:12:47.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:47.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292721+0000", 2020-08-07T15:12:47.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00090361399999999996 2020-08-07T15:12:47.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:47.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293624+0000", 2020-08-07T15:12:47.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1761000000000002e-05 2020-08-07T15:12:47.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:47.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293666+0000", 2020-08-07T15:12:47.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001 2020-08-07T15:12:47.399 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.400 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:47.400 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.400 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:19776 5.4 5:2bb0c0d9:::benchmark_data_smithi074_53697_object2471:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e173)", 2020-08-07T15:12:47.401 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:12:47.401 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 134.71076656899999, 2020-08-07T15:12:47.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.402061844, 2020-08-07T15:12:47.401 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:47.401 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:47.402 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:47.402 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:47.402 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:47.402 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 19776 2020-08-07T15:12:47.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.403 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:47.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:47.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:12:47.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:47.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:47.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:12:47.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-05 2020-08-07T15:12:47.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:47.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399752+0000", 2020-08-07T15:12:47.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8799999999999999e-06 2020-08-07T15:12:47.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:47.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399757+0000", 2020-08-07T15:12:47.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7690000000000008e-06 2020-08-07T15:12:47.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:47.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399767+0000", 2020-08-07T15:12:47.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-06 2020-08-07T15:12:47.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:47.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399771+0000", 2020-08-07T15:12:47.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003255109 2020-08-07T15:12:47.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:47.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403026+0000", 2020-08-07T15:12:47.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101044 2020-08-07T15:12:47.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:47.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403127+0000", 2020-08-07T15:12:47.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000193158 2020-08-07T15:12:47.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,5", 2020-08-07T15:12:47.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403320+0000", 2020-08-07T15:12:47.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017185530000000001 2020-08-07T15:12:47.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:47.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405038+0000", 2020-08-07T15:12:47.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6300000000000004e-07 2020-08-07T15:12:47.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:47.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405039+0000", 2020-08-07T15:12:47.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012590600000000001 2020-08-07T15:12:47.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:47.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405165+0000", 2020-08-07T15:12:47.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109757 2020-08-07T15:12:47.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:47.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405275+0000", 2020-08-07T15:12:47.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39642174699999999 2020-08-07T15:12:47.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:47.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801697+0000", 2020-08-07T15:12:47.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2799999999999996e-07 2020-08-07T15:12:47.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:47.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801697+0000", 2020-08-07T15:12:47.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8740999999999997e-05 2020-08-07T15:12:47.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:47.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801776+0000", 2020-08-07T15:12:47.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.402061844 2020-08-07T15:12:47.419 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.419 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:47.427 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.428 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:19776 5.4 e173/164 ondisk, result = 0)", 2020-08-07T15:12:47.429 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:12:47.429 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 134.705018266, 2020-08-07T15:12:47.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39631765200000002, 2020-08-07T15:12:47.429 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:47.429 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:47.430 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:47.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:47.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:12:47.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:47.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:47.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:12:47.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9332e-05 2020-08-07T15:12:47.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:47.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405492+0000", 2020-08-07T15:12:47.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8900000000000004e-06 2020-08-07T15:12:47.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:47.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405498+0000", 2020-08-07T15:12:47.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39566664200000001 2020-08-07T15:12:47.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:47.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801164+0000", 2020-08-07T15:12:47.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7440000000000001e-06 2020-08-07T15:12:47.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:47.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801168+0000", 2020-08-07T15:12:47.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050828199999999998 2020-08-07T15:12:47.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:47.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801676+0000", 2020-08-07T15:12:47.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1011e-05 2020-08-07T15:12:47.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:47.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801687+0000", 2020-08-07T15:12:47.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2750999999999993e-05 2020-08-07T15:12:47.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:47.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801780+0000", 2020-08-07T15:12:47.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39631765200000002 2020-08-07T15:12:47.440 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.440 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:47.440 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.441 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:85552 5.9 5:9d69b7cc:::benchmark_data_smithi074_53697_object10693:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e202)", 2020-08-07T15:12:47.441 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:12:47.441 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 90.302963474999999, 2020-08-07T15:12:47.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54429374399999997, 2020-08-07T15:12:47.442 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:47.442 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:47.442 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:47.442 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:47.442 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:47.443 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 85552 2020-08-07T15:12:47.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.443 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:47.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:47.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:12:47.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:47.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:47.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:12:47.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2691000000000001e-05 2020-08-07T15:12:47.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:47.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807570+0000", 2020-08-07T15:12:47.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0619999999999997e-06 2020-08-07T15:12:47.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:47.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807576+0000", 2020-08-07T15:12:47.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1558e-05 2020-08-07T15:12:47.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:47.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807588+0000", 2020-08-07T15:12:47.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4600000000000002e-06 2020-08-07T15:12:47.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:47.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807593+0000", 2020-08-07T15:12:47.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022126279999999999 2020-08-07T15:12:47.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:47.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.809806+0000", 2020-08-07T15:12:47.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104868 2020-08-07T15:12:47.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:47.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.809911+0000", 2020-08-07T15:12:47.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000186019 2020-08-07T15:12:47.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:12:47.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.810097+0000", 2020-08-07T15:12:47.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037966739999999999 2020-08-07T15:12:47.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:47.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813893+0000", 2020-08-07T15:12:47.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1310000000000001e-06 2020-08-07T15:12:47.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:47.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813894+0000", 2020-08-07T15:12:47.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4704999999999999e-05 2020-08-07T15:12:47.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:47.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813949+0000", 2020-08-07T15:12:47.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015487280000000001 2020-08-07T15:12:47.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:47.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.815498+0000", 2020-08-07T15:12:47.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53621442900000005 2020-08-07T15:12:47.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:47.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351712+0000", 2020-08-07T15:12:47.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8199999999999998e-07 2020-08-07T15:12:47.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:47.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351713+0000", 2020-08-07T15:12:47.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7909000000000001e-05 2020-08-07T15:12:47.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:47.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351811+0000", 2020-08-07T15:12:47.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54429374399999997 2020-08-07T15:12:47.470 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.471 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:47.471 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.471 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:85552 5.9 e202/164 ondisk, result = 0)", 2020-08-07T15:12:47.472 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:12:47.472 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 90.296737073000003, 2020-08-07T15:12:47.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53813836800000003, 2020-08-07T15:12:47.472 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:47.472 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:47.473 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:47.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:47.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:12:47.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:47.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:47.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:12:47.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8833e-05 2020-08-07T15:12:47.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:47.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813772+0000", 2020-08-07T15:12:47.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8459999999999999e-06 2020-08-07T15:12:47.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:47.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813777+0000", 2020-08-07T15:12:47.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53732224900000003 2020-08-07T15:12:47.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:47.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351100+0000", 2020-08-07T15:12:47.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1869999999999996e-06 2020-08-07T15:12:47.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:47.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351105+0000", 2020-08-07T15:12:47.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00058858100000000002 2020-08-07T15:12:47.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:47.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351693+0000", 2020-08-07T15:12:47.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4229999999999994e-06 2020-08-07T15:12:47.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:47.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351703+0000", 2020-08-07T15:12:47.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000179249 2020-08-07T15:12:47.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:47.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351882+0000", 2020-08-07T15:12:47.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53813836800000003 2020-08-07T15:12:47.481 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.482 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:47.482 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.483 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:85552 5.9 5:9d69b7cc:::benchmark_data_smithi074_53697_object10693:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e202)", 2020-08-07T15:12:47.483 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:12:47.483 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 90.291988949, 2020-08-07T15:12:47.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53326610600000002, 2020-08-07T15:12:47.483 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:47.484 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:12:47.484 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:47.484 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:47.484 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:47.484 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 85552 2020-08-07T15:12:47.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.485 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:47.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:47.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:12:47.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:47.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:47.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:12:47.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5001e-05 2020-08-07T15:12:47.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:47.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818517+0000", 2020-08-07T15:12:47.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5000000000000002e-06 2020-08-07T15:12:47.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:47.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818519+0000", 2020-08-07T15:12:47.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0699999999999997e-06 2020-08-07T15:12:47.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:47.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818524+0000", 2020-08-07T15:12:47.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2910000000000002e-06 2020-08-07T15:12:47.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:47.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818527+0000", 2020-08-07T15:12:47.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030912499999999999 2020-08-07T15:12:47.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:47.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818836+0000", 2020-08-07T15:12:47.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2908000000000002e-05 2020-08-07T15:12:47.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for ondisk", 2020-08-07T15:12:47.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818889+0000", 2020-08-07T15:12:47.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53286921099999995 2020-08-07T15:12:47.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:47.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351758+0000", 2020-08-07T15:12:47.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53326610600000002 2020-08-07T15:12:47.495 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.496 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:47.496 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.496 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:89464 5.9 5:94c69ba5:::benchmark_data_smithi074_53697_object11182:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:12:47.497 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:12:47.497 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.694148995000006, 2020-08-07T15:12:47.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67692916199999997, 2020-08-07T15:12:47.497 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:47.497 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:47.498 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:47.498 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:47.498 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:47.498 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 89464 2020-08-07T15:12:47.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.499 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:47.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:47.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:12:47.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:47.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:47.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:12:47.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0074000000000002e-05 2020-08-07T15:12:47.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:47.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416382+0000", 2020-08-07T15:12:47.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5000000000000001e-06 2020-08-07T15:12:47.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:47.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416386+0000", 2020-08-07T15:12:47.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2447e-05 2020-08-07T15:12:47.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:47.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416399+0000", 2020-08-07T15:12:47.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1279999999999996e-06 2020-08-07T15:12:47.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:47.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416405+0000", 2020-08-07T15:12:47.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016641805999999999 2020-08-07T15:12:47.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:47.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433047+0000", 2020-08-07T15:12:47.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107995 2020-08-07T15:12:47.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:47.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433155+0000", 2020-08-07T15:12:47.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018818599999999999 2020-08-07T15:12:47.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:12:47.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433343+0000", 2020-08-07T15:12:47.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039911679999999998 2020-08-07T15:12:47.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:47.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437334+0000", 2020-08-07T15:12:47.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2020-08-07T15:12:47.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:47.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437335+0000", 2020-08-07T15:12:47.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6351000000000002e-05 2020-08-07T15:12:47.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:47.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437401+0000", 2020-08-07T15:12:47.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001267949 2020-08-07T15:12:47.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:47.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438669+0000", 2020-08-07T15:12:47.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65454035600000005 2020-08-07T15:12:47.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:47.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093209+0000", 2020-08-07T15:12:47.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7859999999999999e-06 2020-08-07T15:12:47.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:47.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093211+0000", 2020-08-07T15:12:47.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9665e-05 2020-08-07T15:12:47.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:47.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093261+0000", 2020-08-07T15:12:47.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67692916199999997 2020-08-07T15:12:47.515 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.515 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:47.526 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.527 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:89464 5.9 e207/164 ondisk, result = 0)", 2020-08-07T15:12:47.527 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:12:47.527 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.671519232999998, 2020-08-07T15:12:47.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65430647200000003, 2020-08-07T15:12:47.527 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:47.528 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:47.528 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:47.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:47.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:12:47.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:47.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:47.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:12:47.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6529000000000002e-05 2020-08-07T15:12:47.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:47.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438998+0000", 2020-08-07T15:12:47.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.567e-06 2020-08-07T15:12:47.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:47.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.439005+0000", 2020-08-07T15:12:47.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65408475099999996 2020-08-07T15:12:47.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:47.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093089+0000", 2020-08-07T15:12:47.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6880000000000002e-06 2020-08-07T15:12:47.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:47.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093096+0000", 2020-08-07T15:12:47.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4980000000000002e-05 2020-08-07T15:12:47.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:47.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093191+0000", 2020-08-07T15:12:47.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3090000000000001e-06 2020-08-07T15:12:47.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:47.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093200+0000", 2020-08-07T15:12:47.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7648e-05 2020-08-07T15:12:47.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:47.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093268+0000", 2020-08-07T15:12:47.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65430647200000003 2020-08-07T15:12:47.538 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.538 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:47.539 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.539 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:114080 5.4 5:24adcd54:::benchmark_data_smithi074_53697_object14259:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e215)", 2020-08-07T15:12:47.539 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:12:47.540 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 67.510811775999997, 2020-08-07T15:12:47.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.57290862300000001, 2020-08-07T15:12:47.540 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:47.540 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:47.540 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:47.541 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:12:47.541 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:12:47.541 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 114080 2020-08-07T15:12:47.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.541 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:47.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:47.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:12:47.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:47.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:47.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:12:47.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9238e-05 2020-08-07T15:12:47.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:47.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599698+0000", 2020-08-07T15:12:47.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6260000000000003e-06 2020-08-07T15:12:47.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:47.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599703+0000", 2020-08-07T15:12:47.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5577999999999999e-05 2020-08-07T15:12:47.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:47.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599718+0000", 2020-08-07T15:12:47.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9869999999999999e-06 2020-08-07T15:12:47.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:47.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599726+0000", 2020-08-07T15:12:47.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00077929599999999996 2020-08-07T15:12:47.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:47.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600506+0000", 2020-08-07T15:12:47.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1717000000000003e-05 2020-08-07T15:12:47.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:47.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600597+0000", 2020-08-07T15:12:47.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011828999999999999 2020-08-07T15:12:47.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,5", 2020-08-07T15:12:47.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600716+0000", 2020-08-07T15:12:47.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020861479999999999 2020-08-07T15:12:47.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:47.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602802+0000", 2020-08-07T15:12:47.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4600000000000005e-07 2020-08-07T15:12:47.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:47.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602802+0000", 2020-08-07T15:12:47.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9165e-05 2020-08-07T15:12:47.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:47.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602862+0000", 2020-08-07T15:12:47.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0093485080000000002 2020-08-07T15:12:47.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:47.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.612210+0000", 2020-08-07T15:12:47.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.56030197699999995 2020-08-07T15:12:47.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:47.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172512+0000", 2020-08-07T15:12:47.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8800000000000001e-07 2020-08-07T15:12:47.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:47.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172513+0000", 2020-08-07T15:12:47.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4659000000000006e-05 2020-08-07T15:12:47.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:47.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172578+0000", 2020-08-07T15:12:47.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.57290862300000001 2020-08-07T15:12:47.575 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.575 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:47.575 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.576 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:48616 6.9 6:955383e0:::benchmark_data_smithi074_63843_object6076:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e256)", 2020-08-07T15:12:47.576 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:43.702845+0000", 2020-08-07T15:12:47.576 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.4076354790000001, 2020-08-07T15:12:47.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40912302499999997, 2020-08-07T15:12:47.577 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:47.577 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:47.577 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:47.577 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:12:47.582 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:47.583 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48616 2020-08-07T15:12:47.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.583 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:47.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:47.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702845+0000", 2020-08-07T15:12:47.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:47.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:47.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702845+0000", 2020-08-07T15:12:47.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1931999999999997e-05 2020-08-07T15:12:47.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:47.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702897+0000", 2020-08-07T15:12:47.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0650000000000004e-06 2020-08-07T15:12:47.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:47.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702903+0000", 2020-08-07T15:12:47.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3084999999999999e-05 2020-08-07T15:12:47.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:47.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702916+0000", 2020-08-07T15:12:47.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4179999999999996e-06 2020-08-07T15:12:47.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:47.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702922+0000", 2020-08-07T15:12:47.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045509080000000002 2020-08-07T15:12:47.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:47.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.707473+0000", 2020-08-07T15:12:47.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2619000000000005e-05 2020-08-07T15:12:47.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:47.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.707535+0000", 2020-08-07T15:12:47.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000126919 2020-08-07T15:12:47.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2", 2020-08-07T15:12:47.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.707662+0000", 2020-08-07T15:12:47.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001200135 2020-08-07T15:12:47.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:47.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.708862+0000", 2020-08-07T15:12:47.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0109999999999999e-06 2020-08-07T15:12:47.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:47.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.708863+0000", 2020-08-07T15:12:47.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8837999999999994e-05 2020-08-07T15:12:47.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:47.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.708932+0000", 2020-08-07T15:12:47.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054206710000000002 2020-08-07T15:12:47.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:47.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.714353+0000", 2020-08-07T15:12:47.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39754767099999999 2020-08-07T15:12:47.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:47.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111901+0000", 2020-08-07T15:12:47.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.023e-06 2020-08-07T15:12:47.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:47.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111902+0000", 2020-08-07T15:12:47.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6730000000000007e-05 2020-08-07T15:12:47.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:47.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111968+0000", 2020-08-07T15:12:47.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40912302499999997 2020-08-07T15:12:47.612 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.612 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:47.612 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.613 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5481.0:48616 6.9 e256/230 ondisk, result = 0)", 2020-08-07T15:12:47.613 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:43.714790+0000", 2020-08-07T15:12:47.613 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.3956909139999998, 2020-08-07T15:12:47.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39718457899999998, 2020-08-07T15:12:47.613 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:47.614 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:47.615 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:47.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:47.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.714790+0000", 2020-08-07T15:12:47.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:47.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:47.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.714790+0000", 2020-08-07T15:12:47.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8272000000000001e-05 2020-08-07T15:12:47.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:47.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.714808+0000", 2020-08-07T15:12:47.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1599999999999998e-06 2020-08-07T15:12:47.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:47.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.714811+0000", 2020-08-07T15:12:47.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39639543300000002 2020-08-07T15:12:47.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:47.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111207+0000", 2020-08-07T15:12:47.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7680000000000002e-06 2020-08-07T15:12:47.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:47.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111210+0000", 2020-08-07T15:12:47.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067142299999999999 2020-08-07T15:12:47.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:47.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111882+0000", 2020-08-07T15:12:47.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6849999999999996e-06 2020-08-07T15:12:47.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:47.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111892+0000", 2020-08-07T15:12:47.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2837999999999995e-05 2020-08-07T15:12:47.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:47.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:47.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:47.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111974+0000", 2020-08-07T15:12:47.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39718457899999998 2020-08-07T15:12:47.623 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.623 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:47.624 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.624 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:47.624 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:47.624 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:47.632 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:12:47.632 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:12:47.632 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:12:47.633 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:12:47.633 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:12:47.633 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:12:47.633 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:12:47.633 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:12:47.634 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 35 16 54181 54165 12.089 12.2969 0.00418804 0.0103314 2020-08-07T15:12:47.634 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:12:47.635 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2020-08-07T15:12:47.230774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":433233920,"num_objects":6613,"num_object_clones":0,"num_object_copies":19839,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52885,"num_write_kb":423080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":195,"acting":195,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":196,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9643540,"kb_used_data":1254868,"kb_used_omap":2,"kb_used_meta":8388605,"kb_avail":745331180,"statfs":{"total":773094113280,"available":763219128320,"internally_reserved":8589934592,"allocated":1284984832,"data_stored":1274859107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":8589932013},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,5,5,4,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":63946752,"num_objects":976,"num_object_clones":0,"num_object_copies":2928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7806,"num_write_kb":62448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.544810"},"pg_stats":[{"pgid":"1.28","version":"16'2","reported_seq":"22","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.233241+0000","last_change":"2020-08-07T15:12:37.495634+0000","last_active":"2020-08-07T15:12:44.233241+0000","last_peered":"2020-08-07T15:12:44.233241+0000","last_clean":"2020-08-07T15:12:44.233241+0000","last_became_active":"2020-08-07T15:12:37.495085+0000","last_became_peered":"2020-08-07T15:12:37.495085+0000","last_unstale":"2020-08-07T15:12:44.233241+0000","last_undegraded":"2020-08-07T15:12:44.233241+0000","last_fullsized":"2020-08-07T15:12:44.233241+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.490655+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.490655+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.27","version":"16'2","reported_seq":"364","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.461533+0000","last_change":"2020-08-07T15:12:37.412480+0000","last_active":"2020-08-07T15:12:41.461533+0000","last_peered":"2020-08-07T15:12:41.461533+0000","last_clean":"2020-08-07T15:12:41.461533+0000","last_became_active":"2020-08-07T15:12:37.411869+0000","last_became_peered":"2020-08-07T15:12:37.411869+0000","last_unstale":"2020-08-07T15:12:41.461533+0000","last_undegraded":"2020-08-07T15:12:41.461533+0000","last_fullsized":"2020-08-07T15:12:41.461533+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:15.163639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:15.163639+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.26","version":"16'2","reported_seq":"31","reported_epoch":"256","state":"active+clean","last_fresh":"2020-08-07T15:12:43.239553+0000","last_change":"2020-08-07T15:12:40.765120+0000","last_active":"2020-08-07T15:12:43.239553+0000","last_peered":"2020-08-07T15:12:43.239553+0000","last_clean":"2020-08-07T15:12:43.239553+0000","last_became_active":"2020-08-07T15:12:37.457426+0000","last_became_peered":"2020-08-07T15:12:37.457426+0000","last_unstale":"2020-08-07T15:12:43.239553+0000","last_undegraded":"2020-08-07T15:12:43.239553+0000","last_fullsized":"2020-08-07T15:12:43.239553+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:40.765027+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_clean_scrub_stamp":"2020-08-07T15:12:40.765027+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.25","version":"75'3","reported_seq":"255","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.237179+0000","last_change":"2020-08-07T15:12:37.428239+0000","last_active":"2020-08-07T15:12:44.237179+0000","last_peered":"2020-08-07T15:12:44.237179+0000","last_clean":"2020-08-07T15:12:44.237179+0000","last_became_active":"2020-08-07T15:12:37.427816+0000","last_became_peered":"2020-08-07T15:12:37.427816+0000","last_unstale":"2020-08-07T15:12:44.237179+0000","last_undegraded":"2020-08-07T15:12:44.237179+0000","last_fullsized":"2020-08-07T15:12:44.237179+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:12.545945+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.545945+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.24","version":"16'2","reported_seq":"196","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.239623+0000","last_change":"2020-08-07T15:12:42.295755+0000","last_active":"2020-08-07T15:12:44.239623+0000","last_peered":"2020-08-07T15:12:44.239623+0000","last_clean":"2020-08-07T15:12:44.239623+0000","last_became_active":"2020-08-07T15:12:34.414144+0000","last_became_peered":"2020-08-07T15:12:34.414144+0000","last_unstale":"2020-08-07T15:12:44.239623+0000","last_undegraded":"2020-08-07T15:12:44.239623+0000","last_fullsized":"2020-08-07T15:12:44.239623+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:42.295672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:42.295672+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.23","version":"16'2","reported_seq":"299","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.461359+0000","last_change":"2020-08-07T15:12:37.410705+0000","last_active":"2020-08-07T15:12:41.461359+0000","last_peered":"2020-08-07T15:12:41.461359+0000","last_clean":"2020-08-07T15:12:41.461359+0000","last_became_active":"2020-08-07T15:12:37.410157+0000","last_became_peered":"2020-08-07T15:12:37.410157+0000","last_unstale":"2020-08-07T15:12:41.461359+0000","last_undegraded":"2020-08-07T15:12:41.461359+0000","last_fullsized":"2020-08-07T15:12:41.461359+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:10.175739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:10.175739+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1],"acting":[6,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.22","version":"16'2","reported_seq":"19","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.461584+0000","last_change":"2020-08-07T15:12:37.413542+0000","last_active":"2020-08-07T15:12:41.461584+0000","last_peered":"2020-08-07T15:12:41.461584+0000","last_clean":"2020-08-07T15:12:41.461584+0000","last_became_active":"2020-08-07T15:12:37.413046+0000","last_became_peered":"2020-08-07T15:12:37.413046+0000","last_unstale":"2020-08-07T15:12:41.461584+0000","last_undegraded":"2020-08-07T15:12:41.461584+0000","last_fullsized":"2020-08-07T15:12:41.461584+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:11.658923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:11.658923+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.21","version":"16'2","reported_seq":"29","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.239423+0000","last_change":"2020-08-07T15:12:37.419228+0000","last_active":"2020-08-07T15:12:44.239423+0000","last_peered":"2020-08-07T15:12:44.239423+0000","last_clean":"2020-08-07T15:12:44.239423+0000","last_became_active":"2020-08-07T15:12:37.418612+0000","last_became_peered":"2020-08-07T15:12:37.418612+0000","last_unstale":"2020-08-07T15:12:44.239423+0000","last_undegraded":"2020-08-07T15:12:44.239423+0000","last_fullsized":"2020-08-07T15:12:44.239423+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:09.261101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.261101+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.20","version":"16'2","reported_seq":"29","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.237116+0000","last_change":"2020-08-07T15:12:37.409857+0000","last_active":"2020-08-07T15:12:44.237116+0000","last_peered":"2020-08-07T15:12:44.237116+0000","last_clean":"2020-08-07T15:12:44.237116+0000","last_became_active":"2020-08-07T15:12:37.409249+0000","last_became_peered":"2020-08-07T15:12:37.409249+0000","last_unstale":"2020-08-07T15:12:44.237116+0000","last_undegraded":"2020-08-07T15:12:44.237116+0000","last_fullsized":"2020-08-07T15:12:44.237116+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:40.696862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:40.696862+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1f","version":"16'2","reported_seq":"22","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.236882+0000","last_change":"2020-08-07T15:12:37.411233+0000","last_active":"2020-08-07T15:12:44.236882+0000","last_peered":"2020-08-07T15:12:44.236882+0000","last_clean":"2020-08-07T15:12:44.236882+0000","last_became_active":"2020-08-07T15:12:37.410716+0000","last_became_peered":"2020-08-07T15:12:37.410716+0000","last_unstale":"2020-08-07T15:12:44.236882+0000","last_undegraded":"2020-08-07T15:12:44.236882+0000","last_fullsized":"2020-08-07T15:12:44.236882+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.176383+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1e","version":"16'2","reported_seq":"109","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.239577+0000","last_change":"2020-08-07T15:11:37.457186+0000","last_active":"2020-08-07T15:12:44.239577+0000","last_peered":"2020-08-07T15:12:44.239577+0000","last_clean":"2020-08-07T15:12:44.239577+0000","last_became_active":"2020-08-07T15:10:51.559894+0000","last_became_peered":"2020-08-07T15:10:51.559894+0000","last_unstale":"2020-08-07T15:12:44.239577+0000","last_undegraded":"2020-08-07T15:12:44.239577+0000","last_fullsized":"2020-08-07T15:12:44.239577+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.457111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.457111+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1d","version":"75'3","reported_seq":"121","reported_epoch":"256","state":"active+clean","last_fresh":"2020-08-07T15:12:43.239618+0000","last_change":"2020-08-07T15:12:28.855487+0000","last_active":"2020-08-07T15:12:43.239618+0000","last_peered":"2020-08-07T15:12:43.239618+0000","last_clean":"2020-08-07T15:12:43.239618+0000","last_became_active":"2020-08-07T15:12:07.356691+0000","last_became_peered":"2020-08-07T15:12:07.356691+0000","last_unstale":"2020-08-07T15:12:43.239618+0000","last_undegraded":"2020-08-07T15:12:43.239618+0000","last_fullsized":"2020-08-07T15:12:43.239618+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:28.855432+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:28.855432+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1c","version":"16'2","reported_seq":"197","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.251740+0000","last_change":"2020-08-07T15:12:07.262797+0000","last_active":"2020-08-07T15:12:44.251740+0000","last_peered":"2020-08-07T15:12:44.251740+0000","last_clean":"2020-08-07T15:12:44.251740+0000","last_became_active":"2020-08-07T15:12:07.245731+0000","last_became_peered":"2020-08-07T15:12:07.245731+0000","last_unstale":"2020-08-07T15:12:44.251740+0000","last_undegraded":"2020-08-07T15:12:44.251740+0000","last_fullsized":"2020-08-07T15:12:44.251740+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:39.738160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:39.738160+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1b","version":"16'2","reported_seq":"230","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.233394+0000","last_change":"2020-08-07T15:10:57.797029+0000","last_active":"2020-08-07T15:12:44.233394+0000","last_peered":"2020-08-07T15:12:44.233394+0000","last_clean":"2020-08-07T15:12:44.233394+0000","last_became_active":"2020-08-07T15:10:51.557392+0000","last_became_peered":"2020-08-07T15:10:51.557392+0000","last_unstale":"2020-08-07T15:12:44.233394+0000","last_undegraded":"2020-08-07T15:12:44.233394+0000","last_fullsized":"2020-08-07T15:12:44.233394+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:57.796946+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:57.796946+0000","last_clean_scrub_stamp":"2020-08-07T15:10:57.796946+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1a","version":"16'2","reported_seq":"97","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.251444+0000","last_change":"2020-08-07T15:12:40.834667+0000","last_active":"2020-08-07T15:12:44.251444+0000","last_peered":"2020-08-07T15:12:44.251444+0000","last_clean":"2020-08-07T15:12:44.251444+0000","last_became_active":"2020-08-07T15:12:07.243542+0000","last_became_peered":"2020-08-07T15:12:07.243542+0000","last_unstale":"2020-08-07T15:12:44.251444+0000","last_undegraded":"2020-08-07T15:12:44.251444+0000","last_fullsized":"2020-08-07T15:12:44.251444+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:40.834612+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:12:40.834612+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.19","version":"16'2","reported_seq":"155","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.239448+0000","last_change":"2020-08-07T15:12:15.372999+0000","last_active":"2020-08-07T15:12:44.239448+0000","last_peered":"2020-08-07T15:12:44.239448+0000","last_clean":"2020-08-07T15:12:44.239448+0000","last_became_active":"2020-08-07T15:10:51.559752+0000","last_became_peered":"2020-08-07T15:10:51.559752+0000","last_unstale":"2020-08-07T15:12:44.239448+0000","last_undegraded":"2020-08-07T15:12:44.239448+0000","last_fullsized":"2020-08-07T15:12:44.239448+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:15.372910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:15.372910+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.18","version":"16'2","reported_seq":"255","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.233372+0000","last_change":"2020-08-07T15:11:06.955336+0000","last_active":"2020-08-07T15:12:44.233372+0000","last_peered":"2020-08-07T15:12:44.233372+0000","last_clean":"2020-08-07T15:12:44.233372+0000","last_became_active":"2020-08-07T15:10:51.557665+0000","last_became_peered":"2020-08-07T15:10:51.557665+0000","last_unstale":"2020-08-07T15:12:44.233372+0000","last_undegraded":"2020-08-07T15:12:44.233372+0000","last_fullsized":"2020-08-07T15:12:44.233372+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:06.955262+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:06.955262+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.17","version":"16'2","reported_seq":"83","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.233624+0000","last_change":"2020-08-07T15:11:08.916219+0000","last_active":"2020-08-07T15:12:44.233624+0000","last_peered":"2020-08-07T15:12:44.233624+0000","last_clean":"2020-08-07T15:12:44.233624+0000","last_became_active":"2020-08-07T15:10:51.561769+0000","last_became_peered":"2020-08-07T15:10:51.561769+0000","last_unstale":"2020-08-07T15:12:44.233624+0000","last_undegraded":"2020-08-07T15:12:44.233624+0000","last_fullsized":"2020-08-07T15:12:44.233624+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:08.916169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:08.916169+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.16","version":"16'2","reported_seq":"59","reported_epoch":"256","state":"active+clean","last_fresh":"2020-08-07T15:12:43.239740+0000","last_change":"2020-08-07T15:12:12.883619+0000","last_active":"2020-08-07T15:12:43.239740+0000","last_peered":"2020-08-07T15:12:43.239740+0000","last_clean":"2020-08-07T15:12:43.239740+0000","last_became_active":"2020-08-07T15:12:07.231877+0000","last_became_peered":"2020-08-07T15:12:07.231877+0000","last_unstale":"2020-08-07T15:12:43.239740+0000","last_undegraded":"2020-08-07T15:12:43.239740+0000","last_fullsized":"2020-08-07T15:12:43.239740+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:12.883558+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:12.883558+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.883558+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.15","version":"75'3","reported_seq":"94","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.251556+0000","last_change":"2020-08-07T15:12:07.284341+0000","last_active":"2020-08-07T15:12:44.251556+0000","last_peered":"2020-08-07T15:12:44.251556+0000","last_clean":"2020-08-07T15:12:44.251556+0000","last_became_active":"2020-08-07T15:12:07.276899+0000","last_became_peered":"2020-08-07T15:12:07.276899+0000","last_unstale":"2020-08-07T15:12:44.251556+0000","last_undegraded":"2020-08-07T15:12:44.251556+0000","last_fullsized":"2020-08-07T15:12:44.251556+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:11:20.562710+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:11:20.562710+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.14","version":"16'2","reported_seq":"193","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.237048+0000","last_change":"2020-08-07T15:12:09.224231+0000","last_active":"2020-08-07T15:12:44.237048+0000","last_peered":"2020-08-07T15:12:44.237048+0000","last_clean":"2020-08-07T15:12:44.237048+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:12:44.237048+0000","last_undegraded":"2020-08-07T15:12:44.237048+0000","last_fullsized":"2020-08-07T15:12:44.237048+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:09.224158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.224158+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"207","reported_epoch":"256","state":"active+clean","last_fresh":"2020-08-07T15:12:43.237576+0000","last_change":"2020-08-07T15:12:19.577926+0000","last_active":"2020-08-07T15:12:43.237576+0000","last_peered":"2020-08-07T15:12:43.237576+0000","last_clean":"2020-08-07T15:12:43.237576+0000","last_became_active":"2020-08-07T15:12:06.375861+0000","last_became_peered":"2020-08-07T15:12:06.375861+0000","last_unstale":"2020-08-07T15:12:43.237576+0000","last_undegraded":"2020-08-07T15:12:43.237576+0000","last_fullsized":"2020-08-07T15:12:43.237576+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:07.887356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.887356+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"208","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.233374+0000","last_change":"2020-08-07T15:12:26.104598+0000","last_active":"2020-08-07T15:12:44.233374+0000","last_peered":"2020-08-07T15:12:44.233374+0000","last_clean":"2020-08-07T15:12:44.233374+0000","last_became_active":"2020-08-07T15:12:06.367362+0000","last_became_peered":"2020-08-07T15:12:06.367362+0000","last_unstale":"2020-08-07T15:12:44.233374+0000","last_undegraded":"2020-08-07T15:12:44.233374+0000","last_fullsized":"2020-08-07T15:12:44.233374+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:26.104526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:26.104526+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"306","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.233257+0000","last_change":"2020-08-07T15:12:24.105593+0000","last_active":"2020-08-07T15:12:44.233257+0000","last_peered":"2020-08-07T15:12:44.233257+0000","last_clean":"2020-08-07T15:12:44.233257+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:12:44.233257+0000","last_undegraded":"2020-08-07T15:12:44.233257+0000","last_fullsized":"2020-08-07T15:12:44.233257+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:24.105531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:24.105531+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"300","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.233497+0000","last_change":"2020-08-07T15:12:13.231819+0000","last_active":"2020-08-07T15:12:44.233497+0000","last_peered":"2020-08-07T15:12:44.233497+0000","last_clean":"2020-08-07T15:12:44.233497+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:12:44.233497+0000","last_undegraded":"2020-08-07T15:12:44.233497+0000","last_fullsized":"2020-08-07T15:12:44.233497+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:13.231762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.231762+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3","version":"258'3112","reported_seq":"3148","reported_epoch":"258","state":"active+clean","last_fresh":"2020-08-07T15:12:46.726688+0000","last_change":"2020-08-07T15:12:10.464875+0000","last_active":"2020-08-07T15:12:46.726688+0000","last_peered":"2020-08-07T15:12:46.726688+0000","last_clean":"2020-08-07T15:12:46.726688+0000","last_became_active":"2020-08-07T15:12:10.464346+0000","last_became_peered":"2020-08-07T15:12:10.464346+0000","last_unstale":"2020-08-07T15:12:46.726688+0000","last_undegraded":"2020-08-07T15:12:46.726688+0000","last_fullsized":"2020-08-07T15:12:46.726688+0000","mapping_epoch":230,"log_start":"258'3096","ondisk_log_start":"258'3096","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25493504,"num_objects":389,"num_object_clones":0,"num_object_copies":1167,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3112,"num_write_kb":24896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"257","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.461712+0000","last_change":"2020-08-07T15:12:27.410339+0000","last_active":"2020-08-07T15:12:41.461712+0000","last_peered":"2020-08-07T15:12:41.461712+0000","last_clean":"2020-08-07T15:12:41.461712+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:12:41.461712+0000","last_undegraded":"2020-08-07T15:12:41.461712+0000","last_fullsized":"2020-08-07T15:12:41.461712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:27.410262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:27.410262+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"188","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.239754+0000","last_change":"2020-08-07T15:12:39.258928+0000","last_active":"2020-08-07T15:12:44.239754+0000","last_peered":"2020-08-07T15:12:44.239754+0000","last_clean":"2020-08-07T15:12:44.239754+0000","last_became_active":"2020-08-07T15:12:32.259956+0000","last_became_peered":"2020-08-07T15:12:32.259956+0000","last_unstale":"2020-08-07T15:12:44.239754+0000","last_undegraded":"2020-08-07T15:12:44.239754+0000","last_fullsized":"2020-08-07T15:12:44.239754+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:39.258857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:39.258857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"258'3408","reported_seq":"3443","reported_epoch":"258","state":"active+clean","last_fresh":"2020-08-07T15:12:46.756550+0000","last_change":"2020-08-07T15:12:10.465048+0000","last_active":"2020-08-07T15:12:46.756550+0000","last_peered":"2020-08-07T15:12:46.756550+0000","last_clean":"2020-08-07T15:12:46.756550+0000","last_became_active":"2020-08-07T15:12:10.464493+0000","last_became_peered":"2020-08-07T15:12:10.464493+0000","last_unstale":"2020-08-07T15:12:46.756550+0000","last_undegraded":"2020-08-07T15:12:46.756550+0000","last_fullsized":"2020-08-07T15:12:46.756550+0000","mapping_epoch":230,"log_start":"258'3400","ondisk_log_start":"258'3400","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27918336,"num_objects":426,"num_object_clones":0,"num_object_copies":1278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3408,"num_write_kb":27264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"215","reported_epoch":"256","state":"active+clean","last_fresh":"2020-08-07T15:12:43.239645+0000","last_change":"2020-08-07T15:12:19.561477+0000","last_active":"2020-08-07T15:12:43.239645+0000","last_peered":"2020-08-07T15:12:43.239645+0000","last_clean":"2020-08-07T15:12:43.239645+0000","last_became_active":"2020-08-07T15:12:06.367376+0000","last_became_peered":"2020-08-07T15:12:06.367376+0000","last_unstale":"2020-08-07T15:12:43.239645+0000","last_undegraded":"2020-08-07T15:12:43.239645+0000","last_fullsized":"2020-08-07T15:12:43.239645+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:12.319383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.319383+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"242","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.858389+0000","last_change":"2020-08-07T15:12:44.858389+0000","last_active":"2020-08-07T15:12:44.858389+0000","last_peered":"2020-08-07T15:12:44.858389+0000","last_clean":"2020-08-07T15:12:44.858389+0000","last_became_active":"2020-08-07T15:12:32.241146+0000","last_became_peered":"2020-08-07T15:12:32.241146+0000","last_unstale":"2020-08-07T15:12:44.858389+0000","last_undegraded":"2020-08-07T15:12:44.858389+0000","last_fullsized":"2020-08-07T15:12:44.858389+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:44.858310+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:44.858310+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"257'3168","reported_seq":"3201","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.852666+0000","last_change":"2020-08-07T15:12:10.464412+0000","last_active":"2020-08-07T15:12:44.852666+0000","last_peered":"2020-08-07T15:12:44.852666+0000","last_clean":"2020-08-07T15:12:44.852666+0000","last_became_active":"2020-08-07T15:12:10.463950+0000","last_became_peered":"2020-08-07T15:12:10.463950+0000","last_unstale":"2020-08-07T15:12:44.852666+0000","last_undegraded":"2020-08-07T15:12:44.852666+0000","last_fullsized":"2020-08-07T15:12:44.852666+0000","mapping_epoch":230,"log_start":"257'3161","ondisk_log_start":"257'3161","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25952256,"num_objects":396,"num_object_clones":0,"num_object_copies":1188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3168,"num_write_kb":25344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"258","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.252103+0000","last_change":"2020-08-07T15:11:55.731391+0000","last_active":"2020-08-07T15:12:44.252103+0000","last_peered":"2020-08-07T15:12:44.252103+0000","last_clean":"2020-08-07T15:12:44.252103+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:12:44.252103+0000","last_undegraded":"2020-08-07T15:12:44.252103+0000","last_fullsized":"2020-08-07T15:12:44.252103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:11:55.731304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:11:55.731304+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"314","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.461587+0000","last_change":"2020-08-07T15:12:32.240238+0000","last_active":"2020-08-07T15:12:41.461587+0000","last_peered":"2020-08-07T15:12:41.461587+0000","last_clean":"2020-08-07T15:12:41.461587+0000","last_became_active":"2020-08-07T15:12:32.239755+0000","last_became_peered":"2020-08-07T15:12:32.239755+0000","last_unstale":"2020-08-07T15:12:41.461587+0000","last_undegraded":"2020-08-07T15:12:41.461587+0000","last_fullsized":"2020-08-07T15:12:41.461587+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_clean_scrub_stamp":"2020-08-07T15:12:14.121397+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"257'3304","reported_seq":"3337","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.811076+0000","last_change":"2020-08-07T15:12:10.398415+0000","last_active":"2020-08-07T15:12:44.811076+0000","last_peered":"2020-08-07T15:12:44.811076+0000","last_clean":"2020-08-07T15:12:44.811076+0000","last_became_active":"2020-08-07T15:12:10.397834+0000","last_became_peered":"2020-08-07T15:12:10.397834+0000","last_unstale":"2020-08-07T15:12:44.811076+0000","last_undegraded":"2020-08-07T15:12:44.811076+0000","last_fullsized":"2020-08-07T15:12:44.811076+0000","mapping_epoch":230,"log_start":"257'3296","ondisk_log_start":"257'3296","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27066368,"num_objects":413,"num_object_clones":0,"num_object_copies":1239,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3304,"num_write_kb":26432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"273","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.253398+0000","last_change":"2020-08-07T15:12:08.622779+0000","last_active":"2020-08-07T15:12:44.253398+0000","last_peered":"2020-08-07T15:12:44.253398+0000","last_clean":"2020-08-07T15:12:44.253398+0000","last_became_active":"2020-08-07T15:12:06.367457+0000","last_became_peered":"2020-08-07T15:12:06.367457+0000","last_unstale":"2020-08-07T15:12:44.253398+0000","last_undegraded":"2020-08-07T15:12:44.253398+0000","last_fullsized":"2020-08-07T15:12:44.253398+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:08.622683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:08.622683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"286","reported_epoch":"255","state":"active+clean","last_fresh":"2020-08-07T15:12:42.371230+0000","last_change":"2020-08-07T15:12:42.371309+0000","last_active":"2020-08-07T15:12:42.371230+0000","last_peered":"2020-08-07T15:12:42.371230+0000","last_clean":"2020-08-07T15:12:42.371230+0000","last_became_active":"2020-08-07T15:12:32.237984+0000","last_became_peered":"2020-08-07T15:12:32.237984+0000","last_unstale":"2020-08-07T15:12:42.371230+0000","last_undegraded":"2020-08-07T15:12:42.371230+0000","last_fullsized":"2020-08-07T15:12:42.371230+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:42.371229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:42.371229+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.7","version":"258'3456","reported_seq":"3492","reported_epoch":"258","state":"active+clean","last_fresh":"2020-08-07T15:12:46.659377+0000","last_change":"2020-08-07T15:12:10.464810+0000","last_active":"2020-08-07T15:12:46.659377+0000","last_peered":"2020-08-07T15:12:46.659377+0000","last_clean":"2020-08-07T15:12:46.659377+0000","last_became_active":"2020-08-07T15:12:10.464050+0000","last_became_peered":"2020-08-07T15:12:10.464050+0000","last_unstale":"2020-08-07T15:12:46.659377+0000","last_undegraded":"2020-08-07T15:12:46.659377+0000","last_fullsized":"2020-08-07T15:12:46.659377+0000","mapping_epoch":230,"log_start":"258'3448","ondisk_log_start":"258'3448","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":28311552,"num_objects":432,"num_object_clones":0,"num_object_copies":1296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3456,"num_write_kb":27648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"302","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.253292+0000","last_change":"2020-08-07T15:12:39.812944+0000","last_active":"2020-08-07T15:12:44.253292+0000","last_peered":"2020-08-07T15:12:44.253292+0000","last_clean":"2020-08-07T15:12:44.253292+0000","last_became_active":"2020-08-07T15:12:32.255633+0000","last_became_peered":"2020-08-07T15:12:32.255633+0000","last_unstale":"2020-08-07T15:12:44.253292+0000","last_undegraded":"2020-08-07T15:12:44.253292+0000","last_fullsized":"2020-08-07T15:12:44.253292+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:39.812876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:39.812876+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"258","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.233797+0000","last_change":"2020-08-07T15:11:40.328534+0000","last_active":"2020-08-07T15:12:44.233797+0000","last_peered":"2020-08-07T15:12:44.233797+0000","last_clean":"2020-08-07T15:12:44.233797+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:12:44.233797+0000","last_undegraded":"2020-08-07T15:12:44.233797+0000","last_fullsized":"2020-08-07T15:12:44.233797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:11:40.328482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:11:40.328482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"258'3256","reported_seq":"3292","reported_epoch":"258","state":"active+clean","last_fresh":"2020-08-07T15:12:46.622757+0000","last_change":"2020-08-07T15:12:10.464478+0000","last_active":"2020-08-07T15:12:46.622757+0000","last_peered":"2020-08-07T15:12:46.622757+0000","last_clean":"2020-08-07T15:12:46.622757+0000","last_became_active":"2020-08-07T15:12:10.463997+0000","last_became_peered":"2020-08-07T15:12:10.463997+0000","last_unstale":"2020-08-07T15:12:46.622757+0000","last_undegraded":"2020-08-07T15:12:46.622757+0000","last_fullsized":"2020-08-07T15:12:46.622757+0000","mapping_epoch":230,"log_start":"258'3248","ondisk_log_start":"258'3248","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":26673152,"num_objects":407,"num_object_clones":0,"num_object_copies":1221,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3256,"num_write_kb":26048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"279","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.236887+0000","last_change":"2020-08-07T15:12:10.585667+0000","last_active":"2020-08-07T15:12:44.236887+0000","last_peered":"2020-08-07T15:12:44.236887+0000","last_clean":"2020-08-07T15:12:44.236887+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:12:44.236887+0000","last_undegraded":"2020-08-07T15:12:44.236887+0000","last_fullsized":"2020-08-07T15:12:44.236887+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:10.585580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:10.585580+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"318","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.251557+0000","last_change":"2020-08-07T15:12:38.850878+0000","last_active":"2020-08-07T15:12:44.251557+0000","last_peered":"2020-08-07T15:12:44.251557+0000","last_clean":"2020-08-07T15:12:44.251557+0000","last_became_active":"2020-08-07T15:12:32.239794+0000","last_became_peered":"2020-08-07T15:12:32.239794+0000","last_unstale":"2020-08-07T15:12:44.251557+0000","last_undegraded":"2020-08-07T15:12:44.251557+0000","last_fullsized":"2020-08-07T15:12:44.251557+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:38.850809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:38.850809+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"258'3504","reported_seq":"3540","reported_epoch":"258","state":"active+clean","last_fresh":"2020-08-07T15:12:46.735124+0000","last_change":"2020-08-07T15:12:10.465113+0000","last_active":"2020-08-07T15:12:46.735124+0000","last_peered":"2020-08-07T15:12:46.735124+0000","last_clean":"2020-08-07T15:12:46.735124+0000","last_became_active":"2020-08-07T15:12:10.464582+0000","last_became_peered":"2020-08-07T15:12:10.464582+0000","last_unstale":"2020-08-07T15:12:46.735124+0000","last_undegraded":"2020-08-07T15:12:46.735124+0000","last_fullsized":"2020-08-07T15:12:46.735124+0000","mapping_epoch":230,"log_start":"258'3499","ondisk_log_start":"258'3499","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":28704768,"num_objects":438,"num_object_clones":0,"num_object_copies":1314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3504,"num_write_kb":28032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"273","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.233501+0000","last_change":"2020-08-07T15:12:07.340958+0000","last_active":"2020-08-07T15:12:44.233501+0000","last_peered":"2020-08-07T15:12:44.233501+0000","last_clean":"2020-08-07T15:12:44.233501+0000","last_became_active":"2020-08-07T15:12:06.366967+0000","last_became_peered":"2020-08-07T15:12:06.366967+0000","last_unstale":"2020-08-07T15:12:44.233501+0000","last_undegraded":"2020-08-07T15:12:44.233501+0000","last_fullsized":"2020-08-07T15:12:44.233501+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:07.340855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.340855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"265","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.233261+0000","last_change":"2020-08-07T15:12:32.247332+0000","last_active":"2020-08-07T15:12:44.233261+0000","last_peered":"2020-08-07T15:12:44.233261+0000","last_clean":"2020-08-07T15:12:44.233261+0000","last_became_active":"2020-08-07T15:12:32.246800+0000","last_became_peered":"2020-08-07T15:12:32.246800+0000","last_unstale":"2020-08-07T15:12:44.233261+0000","last_undegraded":"2020-08-07T15:12:44.233261+0000","last_fullsized":"2020-08-07T15:12:44.233261+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:09.261101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.261101+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"258'3216","reported_seq":"3252","reported_epoch":"258","state":"active+clean","last_fresh":"2020-08-07T15:12:46.722371+0000","last_change":"2020-08-07T15:12:10.466261+0000","last_active":"2020-08-07T15:12:46.722371+0000","last_peered":"2020-08-07T15:12:46.722371+0000","last_clean":"2020-08-07T15:12:46.722371+0000","last_became_active":"2020-08-07T15:12:10.465583+0000","last_became_peered":"2020-08-07T15:12:10.465583+0000","last_unstale":"2020-08-07T15:12:46.722371+0000","last_undegraded":"2020-08-07T15:12:46.722371+0000","last_fullsized":"2020-08-07T15:12:46.722371+0000","mapping_epoch":230,"log_start":"258'3209","ondisk_log_start":"258'3209","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":26345472,"num_objects":402,"num_object_clones":0,"num_object_copies":1206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3216,"num_write_kb":25728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"270","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.251902+0000","last_change":"2020-08-07T15:12:07.583222+0000","last_active":"2020-08-07T15:12:44.251902+0000","last_peered":"2020-08-07T15:12:44.251902+0000","last_clean":"2020-08-07T15:12:44.251902+0000","last_became_active":"2020-08-07T15:12:06.366915+0000","last_became_peered":"2020-08-07T15:12:06.366915+0000","last_unstale":"2020-08-07T15:12:44.251902+0000","last_undegraded":"2020-08-07T15:12:44.251902+0000","last_fullsized":"2020-08-07T15:12:44.251902+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:07.583152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.583152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"348","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.461807+0000","last_change":"2020-08-07T15:12:32.255274+0000","last_active":"2020-08-07T15:12:41.461807+0000","last_peered":"2020-08-07T15:12:41.461807+0000","last_clean":"2020-08-07T15:12:41.461807+0000","last_became_active":"2020-08-07T15:12:32.254842+0000","last_became_peered":"2020-08-07T15:12:32.254842+0000","last_unstale":"2020-08-07T15:12:41.461807+0000","last_undegraded":"2020-08-07T15:12:41.461807+0000","last_fullsized":"2020-08-07T15:12:41.461807+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:15.163639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:15.163639+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"258'3176","reported_seq":"3212","reported_epoch":"258","state":"active+clean","last_fresh":"2020-08-07T15:12:46.625485+0000","last_change":"2020-08-07T15:12:10.467447+0000","last_active":"2020-08-07T15:12:46.625485+0000","last_peered":"2020-08-07T15:12:46.625485+0000","last_clean":"2020-08-07T15:12:46.625485+0000","last_became_active":"2020-08-07T15:12:10.467000+0000","last_became_peered":"2020-08-07T15:12:10.467000+0000","last_unstale":"2020-08-07T15:12:46.625485+0000","last_undegraded":"2020-08-07T15:12:46.625485+0000","last_fullsized":"2020-08-07T15:12:46.625485+0000","mapping_epoch":230,"log_start":"258'3167","ondisk_log_start":"258'3167","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":26017792,"num_objects":397,"num_object_clones":0,"num_object_copies":1191,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3176,"num_write_kb":25408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"242","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.233467+0000","last_change":"2020-08-07T15:10:51.644951+0000","last_active":"2020-08-07T15:12:44.233467+0000","last_peered":"2020-08-07T15:12:44.233467+0000","last_clean":"2020-08-07T15:12:44.233467+0000","last_became_active":"2020-08-07T15:10:46.498230+0000","last_became_peered":"2020-08-07T15:10:46.498230+0000","last_unstale":"2020-08-07T15:12:44.233467+0000","last_undegraded":"2020-08-07T15:12:44.233467+0000","last_fullsized":"2020-08-07T15:12:44.233467+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:51.644875+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:10:51.644875+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"258'3320","reported_seq":"3356","reported_epoch":"258","state":"active+clean","last_fresh":"2020-08-07T15:12:46.698776+0000","last_change":"2020-08-07T15:12:10.465075+0000","last_active":"2020-08-07T15:12:46.698776+0000","last_peered":"2020-08-07T15:12:46.698776+0000","last_clean":"2020-08-07T15:12:46.698776+0000","last_became_active":"2020-08-07T15:12:10.464640+0000","last_became_peered":"2020-08-07T15:12:10.464640+0000","last_unstale":"2020-08-07T15:12:46.698776+0000","last_undegraded":"2020-08-07T15:12:46.698776+0000","last_fullsized":"2020-08-07T15:12:46.698776+0000","mapping_epoch":230,"log_start":"258'3312","ondisk_log_start":"258'3312","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27197440,"num_objects":415,"num_object_clones":0,"num_object_copies":1245,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3320,"num_write_kb":26560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"101","reported_epoch":"258","state":"active+clean","last_fresh":"2020-08-07T15:12:45.243933+0000","last_change":"2020-08-07T15:12:45.243933+0000","last_active":"2020-08-07T15:12:45.243933+0000","last_peered":"2020-08-07T15:12:45.243933+0000","last_clean":"2020-08-07T15:12:45.243933+0000","last_became_active":"2020-08-07T15:12:45.243349+0000","last_became_peered":"2020-08-07T15:12:45.243349+0000","last_unstale":"2020-08-07T15:12:45.243933+0000","last_undegraded":"2020-08-07T15:12:45.243933+0000","last_fullsized":"2020-08-07T15:12:45.243933+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.490655+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.490655+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.e","version":"257'3112","reported_seq":"3145","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.823859+0000","last_change":"2020-08-07T15:12:10.465359+0000","last_active":"2020-08-07T15:12:44.823859+0000","last_peered":"2020-08-07T15:12:44.823859+0000","last_clean":"2020-08-07T15:12:44.823859+0000","last_became_active":"2020-08-07T15:12:10.464900+0000","last_became_peered":"2020-08-07T15:12:10.464900+0000","last_unstale":"2020-08-07T15:12:44.823859+0000","last_undegraded":"2020-08-07T15:12:44.823859+0000","last_fullsized":"2020-08-07T15:12:44.823859+0000","mapping_epoch":230,"log_start":"257'3102","ondisk_log_start":"257'3102","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25493504,"num_objects":389,"num_object_clones":0,"num_object_copies":1167,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3112,"num_write_kb":24896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"309","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.251379+0000","last_change":"2020-08-07T15:12:23.886345+0000","last_active":"2020-08-07T15:12:44.251379+0000","last_peered":"2020-08-07T15:12:44.251379+0000","last_clean":"2020-08-07T15:12:44.251379+0000","last_became_active":"2020-08-07T15:10:46.499635+0000","last_became_peered":"2020-08-07T15:10:46.499635+0000","last_unstale":"2020-08-07T15:12:44.251379+0000","last_undegraded":"2020-08-07T15:12:44.251379+0000","last_fullsized":"2020-08-07T15:12:44.251379+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:23.886261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:23.886261+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"258'3453","reported_seq":"3485","reported_epoch":"258","state":"active+clean","last_fresh":"2020-08-07T15:12:46.742673+0000","last_change":"2020-08-07T15:12:10.468110+0000","last_active":"2020-08-07T15:12:46.742673+0000","last_peered":"2020-08-07T15:12:46.742673+0000","last_clean":"2020-08-07T15:12:46.742673+0000","last_became_active":"2020-08-07T15:12:10.467697+0000","last_became_peered":"2020-08-07T15:12:10.467697+0000","last_unstale":"2020-08-07T15:12:46.742673+0000","last_undegraded":"2020-08-07T15:12:46.742673+0000","last_fullsized":"2020-08-07T15:12:46.742673+0000","mapping_epoch":230,"log_start":"258'3448","ondisk_log_start":"258'3448","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":28286976,"num_objects":432,"num_object_clones":0,"num_object_copies":1296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3453,"num_write_kb":27624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"276","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.252105+0000","last_change":"2020-08-07T15:12:13.680022+0000","last_active":"2020-08-07T15:12:44.252105+0000","last_peered":"2020-08-07T15:12:44.252105+0000","last_clean":"2020-08-07T15:12:44.252105+0000","last_became_active":"2020-08-07T15:12:06.380392+0000","last_became_peered":"2020-08-07T15:12:06.380392+0000","last_unstale":"2020-08-07T15:12:44.252105+0000","last_undegraded":"2020-08-07T15:12:44.252105+0000","last_fullsized":"2020-08-07T15:12:44.252105+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:13.679943+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.679943+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"258'3360","reported_seq":"3396","reported_epoch":"258","state":"active+clean","last_fresh":"2020-08-07T15:12:46.748945+0000","last_change":"2020-08-07T15:12:10.398816+0000","last_active":"2020-08-07T15:12:46.748945+0000","last_peered":"2020-08-07T15:12:46.748945+0000","last_clean":"2020-08-07T15:12:46.748945+0000","last_became_active":"2020-08-07T15:12:10.398332+0000","last_became_peered":"2020-08-07T15:12:10.398332+0000","last_unstale":"2020-08-07T15:12:46.748945+0000","last_undegraded":"2020-08-07T15:12:46.748945+0000","last_fullsized":"2020-08-07T15:12:46.748945+0000","mapping_epoch":230,"log_start":"258'3354","ondisk_log_start":"258'3354","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27525120,"num_objects":420,"num_object_clones":0,"num_object_copies":1260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3360,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"231","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.239801+0000","last_change":"2020-08-07T15:12:14.241339+0000","last_active":"2020-08-07T15:12:44.239801+0000","last_peered":"2020-08-07T15:12:44.239801+0000","last_clean":"2020-08-07T15:12:44.239801+0000","last_became_active":"2020-08-07T15:10:46.498051+0000","last_became_peered":"2020-08-07T15:10:46.498051+0000","last_unstale":"2020-08-07T15:12:44.239801+0000","last_undegraded":"2020-08-07T15:12:44.239801+0000","last_fullsized":"2020-08-07T15:12:44.239801+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:14.241262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:14.241262+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"258'3456","reported_seq":"3491","reported_epoch":"258","state":"active+clean","last_fresh":"2020-08-07T15:12:46.735018+0000","last_change":"2020-08-07T15:12:10.398015+0000","last_active":"2020-08-07T15:12:46.735018+0000","last_peered":"2020-08-07T15:12:46.735018+0000","last_clean":"2020-08-07T15:12:46.735018+0000","last_became_active":"2020-08-07T15:12:10.397610+0000","last_became_peered":"2020-08-07T15:12:10.397610+0000","last_unstale":"2020-08-07T15:12:46.735018+0000","last_undegraded":"2020-08-07T15:12:46.735018+0000","last_fullsized":"2020-08-07T15:12:46.735018+0000","mapping_epoch":230,"log_start":"258'3447","ondisk_log_start":"258'3447","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":28311552,"num_objects":432,"num_object_clones":0,"num_object_copies":1296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3456,"num_write_kb":27648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"304","reported_epoch":"254","state":"active+clean","last_fresh":"2020-08-07T15:12:41.461703+0000","last_change":"2020-08-07T15:12:41.445456+0000","last_active":"2020-08-07T15:12:41.461703+0000","last_peered":"2020-08-07T15:12:41.461703+0000","last_clean":"2020-08-07T15:12:41.461703+0000","last_became_active":"2020-08-07T15:12:32.241722+0000","last_became_peered":"2020-08-07T15:12:32.241722+0000","last_unstale":"2020-08-07T15:12:41.461703+0000","last_undegraded":"2020-08-07T15:12:41.461703+0000","last_fullsized":"2020-08-07T15:12:41.461703+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:41.445379+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_clean_scrub_stamp":"2020-08-07T15:12:41.445379+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"258'3048","reported_seq":"3084","reported_epoch":"258","state":"active+clean","last_fresh":"2020-08-07T15:12:46.543492+0000","last_change":"2020-08-07T15:12:10.468495+0000","last_active":"2020-08-07T15:12:46.543492+0000","last_peered":"2020-08-07T15:12:46.543492+0000","last_clean":"2020-08-07T15:12:46.543492+0000","last_became_active":"2020-08-07T15:12:10.467915+0000","last_became_peered":"2020-08-07T15:12:10.467915+0000","last_unstale":"2020-08-07T15:12:46.543492+0000","last_undegraded":"2020-08-07T15:12:46.543492+0000","last_fullsized":"2020-08-07T15:12:46.543492+0000","mapping_epoch":230,"log_start":"258'3043","ondisk_log_start":"258'3043","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24969216,"num_objects":381,"num_object_clones":0,"num_object_copies":1143,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3048,"num_write_kb":24384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,2],"acting":[7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"195","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.252108+0000","last_change":"2020-08-07T15:12:30.942002+0000","last_active":"2020-08-07T15:12:44.252108+0000","last_peered":"2020-08-07T15:12:44.252108+0000","last_clean":"2020-08-07T15:12:44.252108+0000","last_became_active":"2020-08-07T15:10:46.493498+0000","last_became_peered":"2020-08-07T15:10:46.493498+0000","last_unstale":"2020-08-07T15:12:44.252108+0000","last_undegraded":"2020-08-07T15:12:44.252108+0000","last_fullsized":"2020-08-07T15:12:44.252108+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:30.941943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:30.941943+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"258'3536","reported_seq":"3570","reported_epoch":"258","state":"active+clean","last_fresh":"2020-08-07T15:12:46.737564+0000","last_change":"2020-08-07T15:12:10.465251+0000","last_active":"2020-08-07T15:12:46.737564+0000","last_peered":"2020-08-07T15:12:46.737564+0000","last_clean":"2020-08-07T15:12:46.737564+0000","last_became_active":"2020-08-07T15:12:10.464698+0000","last_became_peered":"2020-08-07T15:12:10.464698+0000","last_unstale":"2020-08-07T15:12:46.737564+0000","last_undegraded":"2020-08-07T15:12:46.737564+0000","last_fullsized":"2020-08-07T15:12:46.737564+0000","mapping_epoch":230,"log_start":"258'3527","ondisk_log_start":"258'3527","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":28966912,"num_objects":442,"num_object_clones":0,"num_object_copies":1326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3536,"num_write_kb":28288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"229","reported_epoch":"257","state":"active+clean","last_fresh":"2020-08-07T15:12:44.233825+0000","last_change":"2020-08-07T15:12:44.203274+0000","last_active":"2020-08-07T15:12:44.233825+0000","last_peered":"2020-08-07T15:12:44.233825+0000","last_clean":"2020-08-07T15:12:44.233825+0000","last_became_active":"2020-08-07T15:12:06.367081+0000","last_became_peered":"2020-08-07T15:12:06.367081+0000","last_unstale":"2020-08-07T15:12:44.233825+0000","last_undegraded":"2020-08-07T15:12:44.233825+0000","last_fullsized":"2020-08-07T15:12:44.233825+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:44.203221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:44.203221+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":433233920,"num_objects":6611,"num_object_clones":0,"num_object_copies":19833,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52885,"num_write_kb":423080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1287749632,"data_stored":1287749632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":41,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":123,"acting":123,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542223,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1166244,"kb_used_data":117660,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93205596,"statfs":{"total":96636764160,"available":95442530304,"internally_reserved":1073741824,"allocated":120483840,"data_stored":119236527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":9.5359999999999996,"15min":3.774},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":2.294,"5min":856.07100000000003,"15min":856.07100000000003},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.77700000000000002,"5min":1.002,"15min":0.97599999999999998},"min":{"1min":0.52300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.55,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":1.5209999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.335,"15min":1.3080000000000001},"min":{"1min":0.64200000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.9199999999999999,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.141,"15min":1.024},"min":{"1min":0.61799999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":4.0060000000000002,"5min":4.0060000000000002,"15min":4.0060000000000002},"last":0.76800000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.321,"15min":1.1910000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.5630000000000002,"5min":5.6619999999999999,"15min":5.6619999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.081,"5min":1.3280000000000001,"15min":1.1879999999999999},"min":{"1min":0.69299999999999995,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.3769999999999998,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":0.85799999999999998}]},{"osd":3,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":30.957999999999998,"5min":21.91,"15min":8.4179999999999993},"min":{"1min":0.68899999999999995,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":656.99099999999999,"5min":917.38699999999994,"15min":917.38699999999994},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.173,"5min":4.8879999999999999,"15min":2.524},"min":{"1min":0.61599999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":4.0460000000000003,"5min":333.59199999999998,"15min":333.59199999999998},"last":2.383}]},{"osd":4,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":8.8599999999999994,"5min":2.944,"15min":1.8320000000000001},"min":{"1min":0.66000000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":172.012,"5min":172.012,"15min":172.012},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.5509999999999999,"5min":11.529,"15min":4.7169999999999996},"min":{"1min":0.81200000000000006,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":5.4500000000000002,"5min":866.24099999999999,"15min":866.24099999999999},"last":1.5569999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":9.3930000000000007,"15min":28.135000000000002},"min":{"1min":0.68400000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":3.536,"5min":678.76099999999997,"15min":831.88099999999997},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.623,"5min":1.831,"15min":1.607},"min":{"1min":0.89000000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":3.593,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":1.321}]},{"osd":7,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":3.9929999999999999,"15min":2.343},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":4.0810000000000004,"5min":224.14500000000001,"15min":224.14500000000001},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.671,"5min":1.7649999999999999,"15min":1.593},"min":{"1min":0.92000000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":3.6200000000000001,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":0.91900000000000004}]}]},{"osd":7,"up_from":13,"seq":55834574929,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1203888,"kb_used_data":155304,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93167952,"statfs":{"total":96636764160,"available":95403982848,"internally_reserved":1073741824,"allocated":159031296,"data_stored":157761367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":37.372999999999998,"5min":16.672000000000001,"15min":6.4459999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":763.39099999999996,"5min":763.39099999999996,"15min":763.39099999999996},"last":1.129},{"interface":"front","average":{"1min":1.157,"5min":9.7789999999999999,"15min":4.0259999999999998},"min":{"1min":0.61699999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.8279999999999998,"5min":529.88699999999994,"15min":529.88699999999994},"last":0.85699999999999998}]},{"osd":1,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":5.758,"15min":2.5579999999999998},"min":{"1min":0.47699999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.7290000000000001,"5min":547.65300000000002,"15min":547.65300000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":1.4159999999999999,"5min":4.9189999999999996,"15min":2.5640000000000001},"min":{"1min":0.57399999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":5.4960000000000004,"5min":372.50799999999998,"15min":372.50799999999998},"last":0.71499999999999997}]},{"osd":2,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":3.0009999999999999,"5min":1.6850000000000001,"15min":1.3580000000000001},"min":{"1min":0.52000000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":33.131999999999998,"5min":33.131999999999998,"15min":33.131999999999998},"last":1.8859999999999999},{"interface":"front","average":{"1min":2.8410000000000002,"5min":1.494,"15min":1.3460000000000001},"min":{"1min":0.61799999999999999,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":32.924999999999997,"5min":32.924999999999997,"15min":32.924999999999997},"last":1.901}]},{"osd":3,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":9.4659999999999993,"15min":4.1719999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":5.9329999999999998,"5min":698.19899999999996,"15min":698.19899999999996},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.238,"5min":1.2669999999999999,"15min":1.347},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":3.8599999999999999,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":2.0579999999999998}]},{"osd":4,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":25.448,"5min":6.069,"15min":3.1309999999999998},"min":{"1min":0.76400000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":460.78399999999999,"5min":460.78399999999999,"15min":460.78399999999999},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.474,"15min":1.6060000000000001},"min":{"1min":0.86899999999999999,"5min":0.621,"15min":0.621},"max":{"1min":2.4079999999999999,"5min":3.694,"15min":5.2990000000000004},"last":1.1080000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.3720000000000001,"15min":1.3660000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":5.9989999999999997,"5min":6.2080000000000002,"15min":6.2080000000000002},"last":0.67900000000000005},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.492,"15min":1.6739999999999999},"min":{"1min":0.83599999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":4.407,"5min":4.407,"15min":5.327},"last":0.77400000000000002}]},{"osd":6,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":9.1839999999999993,"15min":4.016},"min":{"1min":0.88,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.3290000000000002,"5min":695.92899999999997,"15min":695.92899999999997},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.597,"5min":1.55,"15min":12.734},"min":{"1min":0.89500000000000002,"5min":0.87,"15min":0.87},"max":{"1min":3.302,"5min":3.7309999999999999,"15min":266.26999999999998},"last":1.149}]}]},{"osd":1,"up_from":227,"seq":974957576201,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1175920,"kb_used_data":127336,"kb_used_omap":2,"kb_used_meta":1048573,"kb_avail":93195920,"statfs":{"total":96636764160,"available":95432622080,"internally_reserved":1073741824,"allocated":130392064,"data_stored":129133226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":1073739245},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7869999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0760000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.4299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5510000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574929,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1150776,"kb_used_data":102192,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93221064,"statfs":{"total":96636764160,"available":95458369536,"internally_reserved":1073741824,"allocated":104644608,"data_stored":103374679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":24.291,"5min":21.719999999999999,"15min":7.891},"min":{"1min":0.52700000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":432.899,"5min":845.57100000000003,"15min":845.57100000000003},"last":0.623},{"interface":"front","average":{"1min":1.752,"5min":1.181,"15min":1.0149999999999999},"min":{"1min":0.62,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":9.3650000000000002,"5min":9.3650000000000002,"15min":9.3650000000000002},"last":0.82699999999999996}]},{"osd":2,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.0620000000000001,"15min":0.97199999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":7.3470000000000004,"5min":7.3470000000000004,"15min":7.3470000000000004},"last":0.65000000000000002},{"interface":"front","average":{"1min":6.2839999999999998,"5min":4.665,"15min":23.381},"min":{"1min":0.47399999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":75.622,"5min":226.911,"15min":606.18499999999995},"last":0.875}]},{"osd":3,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":28.079000000000001,"5min":6.6920000000000002,"15min":2.8540000000000001},"min":{"1min":0.60999999999999999,"5min":0.54100000000000004,"15min":0.52300000000000002},"max":{"1min":431.95299999999997,"5min":431.95299999999997,"15min":431.95299999999997},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.425,"5min":1.46,"15min":1.2649999999999999},"min":{"1min":0.66400000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":5.3780000000000001,"5min":11.19,"15min":11.19},"last":1.135}]},{"osd":4,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.302,"15min":1.2110000000000001},"min":{"1min":0.57399999999999995,"5min":0.501,"15min":0.501},"max":{"1min":1.819,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.304,"5min":1.448,"15min":1.321},"min":{"1min":0.71099999999999997,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.8849999999999998,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":1.0609999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.6200000000000001,"5min":14.598000000000001,"15min":5.6840000000000002},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":7.3799999999999999,"5min":539.31100000000004,"15min":539.31100000000004},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.49,"15min":1.234},"min":{"1min":0.81399999999999995,"5min":0.752,"15min":0.65900000000000003},"max":{"1min":2.9420000000000002,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":0.86299999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:12:34 2020","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.4399999999999999,"15min":6.2220000000000004},"min":{"1min":0.80200000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.8610000000000002,"5min":4.1310000000000002,"15min":124.738},"last":1.028},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.5549999999999999,"15min":12.952999999999999},"min":{"1min":0.73499999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":5.274,"5min":5.274,"15min":294.94},"last":1.181}]},{"osd":7,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.4790000000000001,"15min":1.55},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.79300000000000004},"max":{"1min":1.8620000000000001,"5min":4.7169999999999996,"15min":6.468},"last":1.196},{"interface":"front","average":{"1min":66.063000000000002,"5min":23.518000000000001,"15min":8.7629999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":971.12699999999995,"5min":971.12699999999995,"15min":971.12699999999995},"last":1.21}]}]},{"osd":2,"up_from":13,"seq":55834574929,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1257896,"kb_used_data":209312,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93113944,"statfs":{"total":96636764160,"available":95348678656,"internally_reserved":1073741824,"allocated":214335488,"data_stored":213065559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,5,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":32.965000000000003,"5min":7.8639999999999999,"15min":3.3660000000000001},"min":{"1min":0.41899999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":501.53800000000001,"5min":501.53800000000001,"15min":501.53800000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":27.341999999999999,"5min":6.6900000000000004,"15min":2.8799999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":501.36000000000001,"5min":501.36000000000001,"15min":501.36000000000001},"last":1.175}]},{"osd":1,"last update":"Fri Aug 7 15:10:21 2020","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.6970000000000001,"15min":1.2929999999999999},"min":{"1min":0.52000000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":5.5919999999999996,"5min":18.672000000000001,"15min":18.672000000000001},"last":2.181},{"interface":"front","average":{"1min":1.264,"5min":1.3320000000000001,"15min":1.3500000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":3.7970000000000002,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":0.77800000000000002}]},{"osd":3,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.2130000000000001,"15min":1.179},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":2.1299999999999999},{"interface":"front","average":{"1min":23.064,"5min":15.281000000000001,"15min":6.1310000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":412.61500000000001,"5min":501.75599999999997,"15min":501.75599999999997},"last":0.627}]},{"osd":4,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.5940000000000001,"15min":1.2709999999999999},"min":{"1min":0.67400000000000004,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.712,"5min":19.783000000000001,"15min":19.783000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.4610000000000001,"15min":1.5820000000000001},"min":{"1min":0.68999999999999995,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8979999999999999,"5min":8.048,"15min":8.048},"last":1.3260000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.3460000000000001,"15min":1.29},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.4009999999999998,"5min":5.2220000000000004,"15min":5.2220000000000004},"last":1.347},{"interface":"front","average":{"1min":1.4990000000000001,"5min":15.715,"15min":22.254999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":8.0649999999999995,"5min":627.53300000000002,"15min":627.53300000000002},"last":0.752}]},{"osd":6,"last update":"Fri Aug 7 15:12:37 2020","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":9.9480000000000004,"15min":4.1299999999999999},"min":{"1min":0.875,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.9450000000000001,"5min":821.91899999999998,"15min":821.91899999999998},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.3959999999999999,"5min":9.9580000000000002,"15min":4.2000000000000002},"min":{"1min":0.70199999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.9540000000000002,"5min":751.70000000000005,"15min":751.70000000000005},"last":1.222}]},{"osd":7,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":12.522,"15min":5.335},"min":{"1min":0.85599999999999998,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.3519999999999999,"5min":501.91699999999997,"15min":501.91699999999997},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.7230000000000001,"5min":2.1080000000000001,"15min":1.7},"min":{"1min":0.88500000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":7.2149999999999999,"5min":35.646999999999998,"15min":35.646999999999998},"last":1.385}]}]},{"osd":3,"up_from":13,"seq":55834574929,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1318640,"kb_used_data":270056,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93053200,"statfs":{"total":96636764160,"available":95286476800,"internally_reserved":1073741824,"allocated":276537344,"data_stored":275267415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.226,"15min":1.244},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":4.6580000000000004,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":2.1779999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.2050000000000001,"15min":1.4930000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":4.6349999999999998,"5min":4.9119999999999999,"15min":13.067},"last":2.1400000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":4.4779999999999998,"15min":2.758},"min":{"1min":0.59099999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.847,"5min":270.68700000000001,"15min":270.68700000000001},"last":1.482},{"interface":"front","average":{"1min":1.109,"5min":1.8580000000000001,"15min":1.96},"min":{"1min":0.54100000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.0550000000000002,"5min":14.349,"15min":14.349},"last":2.851}]},{"osd":2,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.365,"15min":1.22},"min":{"1min":0.65900000000000003,"5min":0.56999999999999995,"15min":0.56899999999999995},"max":{"1min":2.8300000000000001,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":2.2349999999999999},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.9690000000000001,"15min":1.7789999999999999},"min":{"1min":0.64500000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":5.3090000000000002,"5min":51.530000000000001,"15min":51.530000000000001},"last":1.7450000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":3.831,"5min":17.792999999999999,"15min":7.0620000000000003},"min":{"1min":0.54500000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":56.688000000000002,"5min":646.26700000000005,"15min":646.26700000000005},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.421,"5min":15.494999999999999,"15min":26.565000000000001},"min":{"1min":0.68100000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":3.2109999999999999,"5min":844.10500000000002,"15min":844.10500000000002},"last":2.2669999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":14.452999999999999,"5min":8.5429999999999993,"15min":3.859},"min":{"1min":0.95499999999999996,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":244.08099999999999,"5min":403.88400000000001,"15min":403.88400000000001},"last":2.4689999999999999},{"interface":"front","average":{"1min":1.78,"5min":1.8979999999999999,"15min":1.8120000000000001},"min":{"1min":0.98499999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":4.5960000000000001,"5min":8.9120000000000008,"15min":8.9120000000000008},"last":2.5150000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.7969999999999999,"5min":1.706,"15min":1.5900000000000001},"min":{"1min":0.74099999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":5.2729999999999997,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":1.8999999999999999},{"interface":"front","average":{"1min":1.907,"5min":13.411,"15min":5.7949999999999999},"min":{"1min":1.032,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":5.3460000000000001,"5min":688.15999999999997,"15min":688.15999999999997},"last":2.4129999999999998}]},{"osd":7,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.7450000000000001,"5min":1.5960000000000001,"15min":1.9730000000000001},"min":{"1min":0.85599999999999998,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":4.8099999999999996,"5min":6.8209999999999997,"15min":6.8209999999999997},"last":2.2010000000000001},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.6559999999999999,"15min":1.8700000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.6099999999999999,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":1.0469999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574928,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1185216,"kb_used_data":136632,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93186624,"statfs":{"total":96636764160,"available":95423102976,"internally_reserved":1073741824,"allocated":139911168,"data_stored":138641239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":49.795000000000002,"5min":19.646000000000001,"15min":7.3360000000000003},"min":{"1min":0.46800000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":829.15099999999995,"5min":829.15099999999995,"15min":829.15099999999995},"last":0.48199999999999998},{"interface":"front","average":{"1min":3.8149999999999999,"5min":1.639,"15min":1.1120000000000001},"min":{"1min":0.45300000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":48.274000000000001,"5min":48.274000000000001,"15min":48.274000000000001},"last":0.60499999999999998}]},{"osd":1,"last update":"Fri Aug 7 15:10:24 2020","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":5.7629999999999999,"15min":2.823},"min":{"1min":0.52900000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.8109999999999999,"5min":320.67000000000002,"15min":320.67000000000002},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.177,"5min":7.5300000000000002,"15min":3.4279999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.7200000000000002,"5min":689.846,"15min":689.846},"last":1.1519999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":4.1159999999999997,"5min":1.901,"15min":1.601},"min":{"1min":0.64300000000000002,"5min":0.56399999999999995,"15min":0.52100000000000002},"max":{"1min":49.878999999999998,"5min":49.878999999999998,"15min":49.878999999999998},"last":0.97399999999999998},{"interface":"front","average":{"1min":4.2270000000000003,"5min":2.0529999999999999,"15min":1.694},"min":{"1min":0.97899999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":49.939999999999998,"5min":49.939999999999998,"15min":49.939999999999998},"last":0.996}]},{"osd":3,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":27.59,"5min":17.260999999999999,"15min":6.7590000000000003},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":426.50799999999998,"5min":638.92100000000005,"15min":638.92100000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.411,"5min":1.3799999999999999,"15min":1.294},"min":{"1min":0.75700000000000001,"5min":0.67800000000000005,"15min":0.66700000000000004},"max":{"1min":3.2360000000000002,"5min":6.431,"15min":6.431},"last":1.4350000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:12:29 2020","interfaces":[{"interface":"back","average":{"1min":52.308999999999997,"5min":28.574999999999999,"15min":10.609999999999999},"min":{"1min":0.56100000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":862.97299999999996,"5min":990.71600000000001,"15min":990.71600000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":49.549999999999997,"5min":11.01,"15min":4.7539999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":780.00699999999995,"5min":780.00699999999995,"15min":780.00699999999995},"last":0.77100000000000002}]},{"osd":6,"last update":"Fri Aug 7 15:12:34 2020","interfaces":[{"interface":"back","average":{"1min":3.8239999999999998,"5min":1.9870000000000001,"15min":1.855},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":48.575000000000003,"5min":48.575000000000003,"15min":48.575000000000003},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.109,"5min":1.333,"15min":13.183},"min":{"1min":0.64600000000000002,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.847,"5min":6.7729999999999997,"15min":389.14600000000002},"last":0.875}]},{"osd":7,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":4.1349999999999998,"5min":4.327,"15min":2.3809999999999998},"min":{"1min":0.74199999999999999,"5min":0.62,"15min":0.62},"max":{"1min":48.75,"5min":252.798,"15min":252.798},"last":1.1319999999999999},{"interface":"front","average":{"1min":4.2610000000000001,"5min":14.635,"15min":5.9749999999999996},"min":{"1min":0.84799999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":49.654000000000003,"5min":501.93000000000001,"15min":501.93000000000001},"last":1.175}]}]},{"osd":5,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1184960,"kb_used_data":136376,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93186880,"statfs":{"total":96636764160,"available":95423365120,"internally_reserved":1073741824,"allocated":139649024,"data_stored":138379095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":3.629,"5min":1.506,"15min":1.22},"min":{"1min":0.42199999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":43.292999999999999,"5min":43.292999999999999,"15min":43.292999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":3.1520000000000001,"5min":4.4249999999999998,"15min":2.0110000000000001},"min":{"1min":0.44700000000000001,"5min":0.438,"15min":0.438},"max":{"1min":42.933,"5min":329.11399999999998,"15min":329.11399999999998},"last":0.84799999999999998}]},{"osd":1,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":8.5239999999999991,"5min":12.039999999999999,"15min":4.6319999999999997},"min":{"1min":0.56799999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":139.88200000000001,"5min":787.22900000000004,"15min":787.22900000000004},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.448,"5min":2.5649999999999999,"15min":1.593},"min":{"1min":0.59899999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":4.3700000000000001,"5min":116.548,"15min":116.548},"last":1.2749999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":3.5499999999999998,"5min":1.5800000000000001,"15min":6.673},"min":{"1min":0.65700000000000003,"5min":0.499,"15min":0.499},"max":{"1min":43.923000000000002,"5min":43.923000000000002,"15min":168.56899999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":3.23,"5min":1.6399999999999999,"15min":22.634},"min":{"1min":0.52800000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":43.076999999999998,"5min":43.076999999999998,"15min":676.27099999999996},"last":1.1080000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.728,"15min":1.321},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":4.7640000000000002,"5min":29.129000000000001,"15min":29.129000000000001},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":10.029999999999999,"15min":4.1520000000000001},"min":{"1min":0.73999999999999999,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":5.2009999999999996,"5min":712.98699999999997,"15min":712.98699999999997},"last":0.72899999999999998}]},{"osd":4,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":12.779999999999999,"15min":38.188000000000002},"min":{"1min":0.77200000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":4.9969999999999999,"5min":971.67999999999995,"15min":971.67999999999995},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.2470000000000001,"15min":1.286},"min":{"1min":0.73599999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.1160000000000001,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":1.171}]},{"osd":6,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":35.393000000000001,"5min":28.459,"15min":43.091999999999999},"min":{"1min":0.72199999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":676.20000000000005,"5min":966.60900000000004,"15min":966.60900000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":1.528,"5min":6.7789999999999999,"15min":19.603999999999999},"min":{"1min":0.89500000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":5.2220000000000004,"5min":501.79000000000002,"15min":517.26700000000005},"last":1.135}]},{"osd":7,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.462,"15min":1.3600000000000001},"min":{"1min":0.85399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":4.8600000000000003,"5min":5.1559999999999997,"15min":5.1559999999999997},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.5669999999999999,"5min":12.834,"15min":5.3220000000000001},"min":{"1min":0.70099999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":3.8999999999999999,"5min":515.09100000000001,"15min":515.09100000000001},"last":1.347}]}]}]}} 2020-08-07T15:12:47.638 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:12:47.644 INFO:tasks.thrashosds.thrasher:Setting 1.16 to [0, 5, 4] 2020-08-07T15:12:47.644 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '1.16', '0', '5', '4'] 2020-08-07T15:12:47.645 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pg-upmap 1.16 0 5 4 2020-08-07T15:12:47.646 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:12:47.647 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:12:47.647 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:12:47.647 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:12:47.648 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:12:47.648 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:12:47.648 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:12:47.648 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:12:47.649 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:47.625+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:47.649 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:12:47.732 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:47.730+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:47.836 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:47.833+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:47.870 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:47.871 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:47.889 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:12:47.912 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:12:47.982 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:47.973+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:48.040 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:48.039+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:48.130 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:48.130 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:48.141 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:48.140+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:48.151 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:12:48.242 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:48.241+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:48.268 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 36 16 55720 55704 12.0871 12.0234 0.00745527 0.010334 2020-08-07T15:12:48.362 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:48.363 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:48.371 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:12:48.374 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:48.342+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:48.447 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:48.445+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:48.497 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:12:48.500 INFO:teuthology.orchestra.run.smithi074.stderr:set 1.16 pg_upmap mapping to [0,5,4] 2020-08-07T15:12:48.549 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:48.546+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:48.582 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:48.582 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:48.595 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:12:48.656 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:48.648+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:48.752 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:48.749+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:48.821 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:48.822 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:48.835 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:12:48.892 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:48.887+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:48.958 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:48.955+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:49.024 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:49.025 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:49.037 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:12:49.080 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:49.077+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:49.160 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:49.158+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:49.263 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:49.262+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:49.264 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:49.264 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:49.270 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 37 16 57197 57181 12.0723 11.5391 0.00658488 0.0103494 2020-08-07T15:12:49.277 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:12:49.376 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:49.370+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:49.480 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:49.471+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:49.495 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:49.496 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:49.513 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:12:49.580 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:49.577+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:49.650 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:12:49.682 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:49.680+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:49.738 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:49.739 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:49.761 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:12:49.822 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:49.817+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:49.888 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:49.884+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:49.903 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:12:49.954 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:12:49.986 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:49.986 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:49.987 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:49.984+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:49.995 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:12:49.997 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:12:50.043 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:12:50.075 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:12:50.095 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:50.093+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:50.125 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:12:50.163 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:12:50.207 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:12:50.207 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:50.205+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:50.222 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:12:50.223 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:12:50.223 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:12:50.223 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:12:50.223 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:12:50.223 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:12:50.224 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:12:50.224 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:12:50.245 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:50.246 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:12:50.246 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:12:50.246 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:12:50.246 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:50.285 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:12:50.296 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 38 16 58722 58706 12.0681 11.9141 0.00404909 0.0103528 2020-08-07T15:12:50.314 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:50.312+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:50.420 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:50.418+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:50.491 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:50.492 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:12:50.492 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:12:50.492 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:12:50.492 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.493 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106240 3.7 3:ebdd8242:::benchmark_data_smithi074_35447_object13279:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:12:50.493 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:12:50.493 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 303.13250333000002, 2020-08-07T15:12:50.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.90079396, 2020-08-07T15:12:50.494 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:50.494 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:50.494 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:50.494 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:12:50.494 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:12:50.510 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106240 2020-08-07T15:12:50.510 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.510 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:50.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.510 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:50.511 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:12:50.511 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:50.526 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.526 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.526 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:50.527 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:12:50.527 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2522000000000003e-05 2020-08-07T15:12:50.527 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.527 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.527 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:50.528 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355118+0000", 2020-08-07T15:12:50.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6159999999999998e-06 2020-08-07T15:12:50.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.528 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:50.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355125+0000", 2020-08-07T15:12:50.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2699e-05 2020-08-07T15:12:50.529 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.529 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.529 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:50.543 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355138+0000", 2020-08-07T15:12:50.544 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2310000000000001e-06 2020-08-07T15:12:50.544 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.544 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.544 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:50.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355144+0000", 2020-08-07T15:12:50.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9855000000000003e-05 2020-08-07T15:12:50.545 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.545 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.546 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355194+0000", 2020-08-07T15:12:50.546 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112713 2020-08-07T15:12:50.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.546 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.571 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:50.571 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355306+0000", 2020-08-07T15:12:50.572 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019794600000000001 2020-08-07T15:12:50.572 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.572 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.572 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:12:50.572 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355504+0000", 2020-08-07T15:12:50.573 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0048346029999999998 2020-08-07T15:12:50.573 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.573 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.573 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:50.573 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.360339+0000", 2020-08-07T15:12:50.574 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00033646400000000002 2020-08-07T15:12:50.574 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.574 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.574 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:50.574 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.360675+0000", 2020-08-07T15:12:50.575 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89508481200000001 2020-08-07T15:12:50.575 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.575 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:50.575 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255760+0000", 2020-08-07T15:12:50.575 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5659999999999999e-06 2020-08-07T15:12:50.576 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.576 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.576 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:50.576 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255762+0000", 2020-08-07T15:12:50.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6595000000000003e-05 2020-08-07T15:12:50.577 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.577 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.577 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:50.577 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255818+0000", 2020-08-07T15:12:50.577 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1338000000000003e-05 2020-08-07T15:12:50.578 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.578 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.579 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:50.579 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255860+0000", 2020-08-07T15:12:50.579 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.90079396 2020-08-07T15:12:50.579 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.587 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:50.587 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.587 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.588 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106240 3.7 e59/20 ondisk, result = 0)", 2020-08-07T15:12:50.588 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:12:50.589 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 303.036107378, 2020-08-07T15:12:50.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80440215100000001, 2020-08-07T15:12:50.589 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:50.589 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:12:50.589 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:50.590 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.590 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:50.590 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:12:50.590 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:50.590 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.591 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.591 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:50.591 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:12:50.591 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0052999999999999e-05 2020-08-07T15:12:50.591 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.592 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.592 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:50.592 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451482+0000", 2020-08-07T15:12:50.592 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3229999999999999e-06 2020-08-07T15:12:50.592 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.593 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.593 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:50.593 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451486+0000", 2020-08-07T15:12:50.593 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80420304799999998 2020-08-07T15:12:50.593 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.595 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.595 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:50.595 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255689+0000", 2020-08-07T15:12:50.595 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7759999999999997e-06 2020-08-07T15:12:50.595 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.596 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.596 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:50.596 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255694+0000", 2020-08-07T15:12:50.596 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5828999999999998e-05 2020-08-07T15:12:50.596 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.597 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.597 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.597 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255740+0000", 2020-08-07T15:12:50.597 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1466e-05 2020-08-07T15:12:50.597 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.597 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.598 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:50.598 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255751+0000", 2020-08-07T15:12:50.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112656 2020-08-07T15:12:50.599 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.599 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.599 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:50.600 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255864+0000", 2020-08-07T15:12:50.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80440215100000001 2020-08-07T15:12:50.600 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.600 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:50.600 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.601 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37057 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:50.601 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:12:50.601 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 236.24399866799999, 2020-08-07T15:12:50.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007248691, 2020-08-07T15:12:50.602 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:50.602 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:50.602 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:50.602 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:50.602 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:50.603 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37057 2020-08-07T15:12:50.603 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.603 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:50.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.603 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:50.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:12:50.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:50.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.605 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.605 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:50.605 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:12:50.605 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3661e-05 2020-08-07T15:12:50.605 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.606 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.606 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:50.606 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243624+0000", 2020-08-07T15:12:50.606 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7519999999999996e-06 2020-08-07T15:12:50.607 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.607 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.607 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:50.607 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243632+0000", 2020-08-07T15:12:50.608 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6392000000000002e-05 2020-08-07T15:12:50.608 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.608 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.608 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:50.608 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243648+0000", 2020-08-07T15:12:50.609 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6659999999999997e-06 2020-08-07T15:12:50.609 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.609 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.609 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:50.609 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243657+0000", 2020-08-07T15:12:50.609 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0070484609999999998 2020-08-07T15:12:50.610 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.610 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.610 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.610 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250705+0000", 2020-08-07T15:12:50.610 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0083e-05 2020-08-07T15:12:50.611 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.611 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.611 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:50.611 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250715+0000", 2020-08-07T15:12:50.611 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000086695 2020-08-07T15:12:50.612 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.612 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.612 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.614 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.250802+0000", 2020-08-07T15:12:50.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6980999999999999e-05 2020-08-07T15:12:50.614 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.614 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.615 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:50.615 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.250819+0000", 2020-08-07T15:12:50.615 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007248691 2020-08-07T15:12:50.615 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.615 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:50.616 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.616 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.616 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.616 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37058 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:50.616 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:12:50.617 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 236.24385693400001, 2020-08-07T15:12:50.617 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007368357, 2020-08-07T15:12:50.617 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:50.617 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:50.617 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:50.618 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:50.618 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:50.618 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37058 2020-08-07T15:12:50.618 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.618 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:50.618 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.619 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:50.619 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:12:50.619 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:50.619 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.619 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.620 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:50.620 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:12:50.638 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8838999999999998e-05 2020-08-07T15:12:50.638 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.638 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.639 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:50.639 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243761+0000", 2020-08-07T15:12:50.639 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2079999999999999e-06 2020-08-07T15:12:50.639 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.639 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.640 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:50.640 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243766+0000", 2020-08-07T15:12:50.640 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0567999999999999e-05 2020-08-07T15:12:50.640 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.640 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.640 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:50.641 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243777+0000", 2020-08-07T15:12:50.641 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7210000000000002e-06 2020-08-07T15:12:50.641 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.641 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.641 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:50.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243782+0000", 2020-08-07T15:12:50.642 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069508749999999996 2020-08-07T15:12:50.642 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.642 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.642 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.643 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250732+0000", 2020-08-07T15:12:50.643 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7899999999999999e-06 2020-08-07T15:12:50.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.643 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.643 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:50.644 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250737+0000", 2020-08-07T15:12:50.644 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003272409999999 2020-08-07T15:12:50.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.644 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.644 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.644 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251064+0000", 2020-08-07T15:12:50.645 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6115000000000001e-05 2020-08-07T15:12:50.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:50.645 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251081+0000", 2020-08-07T15:12:50.660 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007368357 2020-08-07T15:12:50.660 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.660 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:50.661 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.661 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.662 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.662 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37059 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:50.662 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:12:50.662 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 236.243613885, 2020-08-07T15:12:50.662 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0071655319999999, 2020-08-07T15:12:50.663 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:50.663 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:50.663 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:50.663 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:50.663 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:50.664 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37059 2020-08-07T15:12:50.664 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.664 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:50.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.664 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:50.665 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:12:50.665 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:50.665 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.665 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.665 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:50.678 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:12:50.678 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6533000000000002e-05 2020-08-07T15:12:50.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.679 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:50.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244012+0000", 2020-08-07T15:12:50.679 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3860000000000001e-06 2020-08-07T15:12:50.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.680 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:50.680 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244019+0000", 2020-08-07T15:12:50.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6982000000000001e-05 2020-08-07T15:12:50.681 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.681 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:50.681 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244036+0000", 2020-08-07T15:12:50.681 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0310000000000001e-06 2020-08-07T15:12:50.700 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.700 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:50.700 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244043+0000", 2020-08-07T15:12:50.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0067169259999999998 2020-08-07T15:12:50.701 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.701 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.701 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250760+0000", 2020-08-07T15:12:50.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4499999999999998e-06 2020-08-07T15:12:50.702 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.702 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.702 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:50.702 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250768+0000", 2020-08-07T15:12:50.703 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000341691 2020-08-07T15:12:50.703 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.703 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.703 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251109+0000", 2020-08-07T15:12:50.703 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1533e-05 2020-08-07T15:12:50.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.704 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:50.704 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251121+0000", 2020-08-07T15:12:50.704 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0071655319999999 2020-08-07T15:12:50.705 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.705 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:50.705 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.705 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.705 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.734 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37060 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:50.734 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:12:50.734 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 236.24348994799999, 2020-08-07T15:12:50.735 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070762280000001, 2020-08-07T15:12:50.735 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:50.735 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:50.735 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:50.735 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:50.736 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:50.736 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37060 2020-08-07T15:12:50.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.736 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:50.736 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.737 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:50.737 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:12:50.737 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:50.737 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.737 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.749 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:50.749 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:12:50.749 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5739000000000003e-05 2020-08-07T15:12:50.750 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.750 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.750 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:50.750 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244135+0000", 2020-08-07T15:12:50.750 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7740000000000001e-06 2020-08-07T15:12:50.751 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.751 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.761 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:50.761 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244141+0000", 2020-08-07T15:12:50.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3817e-05 2020-08-07T15:12:50.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.762 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.762 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:50.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244155+0000", 2020-08-07T15:12:50.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3829999999999997e-06 2020-08-07T15:12:50.763 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.763 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:50.763 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244161+0000", 2020-08-07T15:12:50.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0066303669999999999 2020-08-07T15:12:50.764 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.764 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250791+0000", 2020-08-07T15:12:50.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.392e-06 2020-08-07T15:12:50.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.766 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:50.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250798+0000", 2020-08-07T15:12:50.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003485860000001 2020-08-07T15:12:50.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251146+0000", 2020-08-07T15:12:50.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1700000000000003e-06 2020-08-07T15:12:50.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:50.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251155+0000", 2020-08-07T15:12:50.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070762280000001 2020-08-07T15:12:50.778 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.778 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:50.779 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.779 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37061 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:50.779 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:12:50.780 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 236.243375321, 2020-08-07T15:12:50.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070006920000001, 2020-08-07T15:12:50.780 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:50.780 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:50.780 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:50.781 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:50.781 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:50.781 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37061 2020-08-07T15:12:50.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.781 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:50.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.782 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:50.782 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:12:50.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:50.782 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.782 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:50.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:12:50.783 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5453e-05 2020-08-07T15:12:50.783 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:50.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244239+0000", 2020-08-07T15:12:50.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3179999999999997e-06 2020-08-07T15:12:50.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:50.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244245+0000", 2020-08-07T15:12:50.785 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2138e-05 2020-08-07T15:12:50.785 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:50.787 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244257+0000", 2020-08-07T15:12:50.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.69e-06 2020-08-07T15:12:50.787 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.787 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.788 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:50.788 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244261+0000", 2020-08-07T15:12:50.788 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0065598940000000001 2020-08-07T15:12:50.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.789 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.789 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250821+0000", 2020-08-07T15:12:50.789 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4799999999999998e-06 2020-08-07T15:12:50.789 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.789 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.790 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:50.790 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250828+0000", 2020-08-07T15:12:50.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003567310000001 2020-08-07T15:12:50.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.791 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251185+0000", 2020-08-07T15:12:50.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9879999999999999e-06 2020-08-07T15:12:50.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.792 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:50.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251195+0000", 2020-08-07T15:12:50.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070006920000001 2020-08-07T15:12:50.792 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.792 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:50.792 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.793 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.793 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.793 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37062 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:50.793 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:12:50.793 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 236.243275103, 2020-08-07T15:12:50.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006933662, 2020-08-07T15:12:50.804 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:50.804 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:50.804 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:50.804 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:50.804 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:50.805 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37062 2020-08-07T15:12:50.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.805 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:50.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:50.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:12:50.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:50.806 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:50.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:12:50.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4041e-05 2020-08-07T15:12:50.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:50.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244348+0000", 2020-08-07T15:12:50.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2079999999999997e-06 2020-08-07T15:12:50.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.808 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:50.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244354+0000", 2020-08-07T15:12:50.809 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1483e-05 2020-08-07T15:12:50.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.809 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:50.809 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244366+0000", 2020-08-07T15:12:50.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7989999999999999e-06 2020-08-07T15:12:50.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.811 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:50.811 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244372+0000", 2020-08-07T15:12:50.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0064783699999999998 2020-08-07T15:12:50.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250850+0000", 2020-08-07T15:12:50.813 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4729999999999997e-06 2020-08-07T15:12:50.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.813 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:50.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250856+0000", 2020-08-07T15:12:50.814 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003630240000001 2020-08-07T15:12:50.814 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.814 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.814 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.814 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251219+0000", 2020-08-07T15:12:50.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2640000000000008e-06 2020-08-07T15:12:50.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:50.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251228+0000", 2020-08-07T15:12:50.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006933662 2020-08-07T15:12:50.816 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.816 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:50.816 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.817 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37063 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:50.817 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:12:50.817 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 236.24316276900001, 2020-08-07T15:12:50.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006858051, 2020-08-07T15:12:50.818 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:50.818 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:50.818 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:50.818 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:50.819 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:50.819 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37063 2020-08-07T15:12:50.819 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.819 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:50.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:50.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:12:50.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:50.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:50.821 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:12:50.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5043000000000001e-05 2020-08-07T15:12:50.821 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:50.822 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244461+0000", 2020-08-07T15:12:50.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4600000000000002e-06 2020-08-07T15:12:50.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.822 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:50.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244467+0000", 2020-08-07T15:12:50.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1435e-05 2020-08-07T15:12:50.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:50.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244478+0000", 2020-08-07T15:12:50.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0190000000000003e-06 2020-08-07T15:12:50.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:50.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244484+0000", 2020-08-07T15:12:50.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006392895 2020-08-07T15:12:50.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.826 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250877+0000", 2020-08-07T15:12:50.826 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4280000000000001e-06 2020-08-07T15:12:50.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.827 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:50.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250884+0000", 2020-08-07T15:12:50.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000370414 2020-08-07T15:12:50.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.827 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.828 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251254+0000", 2020-08-07T15:12:50.828 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0356999999999999e-05 2020-08-07T15:12:50.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:50.829 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251264+0000", 2020-08-07T15:12:50.829 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006858051 2020-08-07T15:12:50.829 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.829 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:50.829 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.830 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.830 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.830 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37064 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:50.830 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:12:50.830 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 236.24305063599999, 2020-08-07T15:12:50.831 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0067798189999999, 2020-08-07T15:12:50.831 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:50.831 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:50.831 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:50.831 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:50.831 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:50.832 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37064 2020-08-07T15:12:50.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.832 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:50.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.832 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:50.833 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:12:50.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:50.833 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:50.865 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:12:50.865 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4007e-05 2020-08-07T15:12:50.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.866 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.866 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:50.866 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244573+0000", 2020-08-07T15:12:50.866 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.831e-06 2020-08-07T15:12:50.866 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.867 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.867 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:50.867 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244577+0000", 2020-08-07T15:12:50.867 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2269000000000001e-05 2020-08-07T15:12:50.867 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.868 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.868 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:50.868 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244590+0000", 2020-08-07T15:12:50.868 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7149999999999997e-06 2020-08-07T15:12:50.868 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.869 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.869 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:50.869 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244594+0000", 2020-08-07T15:12:50.869 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063105670000000004 2020-08-07T15:12:50.869 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.878 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.878 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.878 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250905+0000", 2020-08-07T15:12:50.878 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1970000000000001e-06 2020-08-07T15:12:50.878 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.879 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:50.879 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250911+0000", 2020-08-07T15:12:50.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003783020000001 2020-08-07T15:12:50.879 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.880 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.880 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251289+0000", 2020-08-07T15:12:50.880 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9309999999999997e-06 2020-08-07T15:12:50.880 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.881 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.881 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:50.881 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251298+0000", 2020-08-07T15:12:50.881 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0067798189999999 2020-08-07T15:12:50.881 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.892 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:50.892 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.893 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37065 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:50.893 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:12:50.893 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 236.242940417, 2020-08-07T15:12:50.894 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006708545, 2020-08-07T15:12:50.894 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:50.894 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:50.894 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:50.894 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:50.895 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:50.895 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37065 2020-08-07T15:12:50.895 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.895 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:50.895 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.896 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:50.896 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:12:50.896 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:50.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.897 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:50.897 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:12:50.897 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2750000000000001e-05 2020-08-07T15:12:50.897 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.897 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.898 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:50.898 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244681+0000", 2020-08-07T15:12:50.898 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0950000000000003e-06 2020-08-07T15:12:50.898 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.898 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.899 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:50.899 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244687+0000", 2020-08-07T15:12:50.899 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2485e-05 2020-08-07T15:12:50.899 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.899 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.900 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:50.900 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244699+0000", 2020-08-07T15:12:50.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8800000000000002e-06 2020-08-07T15:12:50.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.901 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:50.901 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244706+0000", 2020-08-07T15:12:50.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0062281569999999998 2020-08-07T15:12:50.901 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.901 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.902 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.902 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250934+0000", 2020-08-07T15:12:50.902 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1719999999999998e-06 2020-08-07T15:12:50.902 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.902 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.903 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:50.903 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250939+0000", 2020-08-07T15:12:50.903 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003861810000001 2020-08-07T15:12:50.903 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.903 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.904 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251325+0000", 2020-08-07T15:12:50.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1824999999999999e-05 2020-08-07T15:12:50.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.904 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:50.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251337+0000", 2020-08-07T15:12:50.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006708545 2020-08-07T15:12:50.905 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.905 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:50.905 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.906 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.906 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37066 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:50.906 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:12:50.906 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 236.23827144699999, 2020-08-07T15:12:50.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002077205, 2020-08-07T15:12:50.907 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:50.907 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:50.907 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:50.907 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:50.907 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:50.908 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37066 2020-08-07T15:12:50.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.908 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:50.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.908 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:50.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:12:50.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:50.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:50.910 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:12:50.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4654000000000002e-05 2020-08-07T15:12:50.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.910 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:50.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249352+0000", 2020-08-07T15:12:50.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6210000000000004e-06 2020-08-07T15:12:50.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:50.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249360+0000", 2020-08-07T15:12:50.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1895999999999998e-05 2020-08-07T15:12:50.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:50.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249382+0000", 2020-08-07T15:12:50.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1479999999999995e-06 2020-08-07T15:12:50.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:50.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249391+0000", 2020-08-07T15:12:50.914 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015731510000000001 2020-08-07T15:12:50.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250964+0000", 2020-08-07T15:12:50.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5400000000000001e-06 2020-08-07T15:12:50.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:50.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250971+0000", 2020-08-07T15:12:50.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000392379 2020-08-07T15:12:50.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251363+0000", 2020-08-07T15:12:50.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1816e-05 2020-08-07T15:12:50.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:50.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251375+0000", 2020-08-07T15:12:50.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002077205 2020-08-07T15:12:50.918 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.918 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:50.918 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.918 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.919 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37067 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:50.919 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:12:50.919 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 236.23813075300001, 2020-08-07T15:12:50.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0019761119999999, 2020-08-07T15:12:50.920 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:50.920 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:50.920 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:50.920 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:50.920 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:50.921 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37067 2020-08-07T15:12:50.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.921 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:50.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:50.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:12:50.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:50.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.922 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:50.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:12:50.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5961999999999997e-05 2020-08-07T15:12:50.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:50.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249494+0000", 2020-08-07T15:12:50.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3210000000000004e-06 2020-08-07T15:12:50.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:50.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249500+0000", 2020-08-07T15:12:50.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1012000000000001e-05 2020-08-07T15:12:50.925 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:50.926 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249511+0000", 2020-08-07T15:12:50.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.809e-06 2020-08-07T15:12:50.926 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:50.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249515+0000", 2020-08-07T15:12:50.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0014777670000000001 2020-08-07T15:12:50.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250993+0000", 2020-08-07T15:12:50.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9499999999999998e-06 2020-08-07T15:12:50.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:50.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250999+0000", 2020-08-07T15:12:50.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0004032519999999 2020-08-07T15:12:50.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251402+0000", 2020-08-07T15:12:50.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2038999999999999e-05 2020-08-07T15:12:50.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:50.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251414+0000", 2020-08-07T15:12:50.931 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0019761119999999 2020-08-07T15:12:50.931 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.932 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:50.932 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.932 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.932 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37068 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:50.933 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:12:50.933 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 236.235886182, 2020-08-07T15:12:50.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99977072, 2020-08-07T15:12:50.933 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:50.933 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:50.934 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:50.934 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:50.934 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:50.935 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37068 2020-08-07T15:12:50.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.935 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:50.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:50.942 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:12:50.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:50.942 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.942 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:50.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:12:50.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1369000000000003e-05 2020-08-07T15:12:50.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:50.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251724+0000", 2020-08-07T15:12:50.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.19e-06 2020-08-07T15:12:50.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.944 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:50.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251731+0000", 2020-08-07T15:12:50.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1368e-05 2020-08-07T15:12:50.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:50.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251742+0000", 2020-08-07T15:12:50.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.682e-06 2020-08-07T15:12:50.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:50.952 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251748+0000", 2020-08-07T15:12:50.952 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5275000000000003e-05 2020-08-07T15:12:50.952 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.952 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.952 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.953 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251793+0000", 2020-08-07T15:12:50.953 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5000000000000005e-06 2020-08-07T15:12:50.953 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.953 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.953 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:50.954 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251802+0000", 2020-08-07T15:12:50.954 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99963953500000002 2020-08-07T15:12:50.954 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.954 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.954 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.954 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251442+0000", 2020-08-07T15:12:50.955 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1800999999999999e-05 2020-08-07T15:12:50.955 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.955 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.955 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:50.955 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251454+0000", 2020-08-07T15:12:50.956 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99977072 2020-08-07T15:12:50.956 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.956 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:50.956 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.956 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.957 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37069 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:50.957 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:12:50.957 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 236.234782384, 2020-08-07T15:12:50.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99870607899999997, 2020-08-07T15:12:50.958 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:50.958 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:50.958 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:50.958 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:50.958 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:50.958 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37069 2020-08-07T15:12:50.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.959 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:50.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:50.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:12:50.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:50.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:50.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:12:50.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3755999999999999e-05 2020-08-07T15:12:50.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.970 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:50.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252831+0000", 2020-08-07T15:12:50.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7919999999999998e-06 2020-08-07T15:12:50.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:50.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252836+0000", 2020-08-07T15:12:50.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1409999999999999e-05 2020-08-07T15:12:50.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:50.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252848+0000", 2020-08-07T15:12:50.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6380000000000001e-06 2020-08-07T15:12:50.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:50.986 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252853+0000", 2020-08-07T15:12:50.986 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3244e-05 2020-08-07T15:12:50.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.987 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.987 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252907+0000", 2020-08-07T15:12:50.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1470000000000007e-06 2020-08-07T15:12:50.987 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:50.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252916+0000", 2020-08-07T15:12:50.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99856497499999997 2020-08-07T15:12:50.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:50.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251481+0000", 2020-08-07T15:12:50.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2116999999999999e-05 2020-08-07T15:12:50.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.990 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.990 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:50.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251493+0000", 2020-08-07T15:12:50.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99870607899999997 2020-08-07T15:12:50.990 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.991 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:50.991 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:50.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:50.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:50.991 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37070 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:12:50.992 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:12:50.992 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 236.234678121, 2020-08-07T15:12:50.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99863946699999995, 2020-08-07T15:12:50.992 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:50.992 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:12:50.993 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:50.993 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:50.993 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:50.993 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37070 2020-08-07T15:12:50.993 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.002 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:51.002 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.002 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:51.002 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:12:51.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:51.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:51.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:12:51.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9984000000000001e-05 2020-08-07T15:12:51.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:51.004 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252941+0000", 2020-08-07T15:12:51.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7010000000000004e-06 2020-08-07T15:12:51.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:51.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252946+0000", 2020-08-07T15:12:51.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1144999999999999e-05 2020-08-07T15:12:51.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:51.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252957+0000", 2020-08-07T15:12:51.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6920000000000001e-06 2020-08-07T15:12:51.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:51.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252962+0000", 2020-08-07T15:12:51.008 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6739000000000002e-05 2020-08-07T15:12:51.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.009 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.009 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:51.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252998+0000", 2020-08-07T15:12:51.010 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.503e-06 2020-08-07T15:12:51.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.010 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:51.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.253006+0000", 2020-08-07T15:12:51.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99851375499999995 2020-08-07T15:12:51.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:51.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251520+0000", 2020-08-07T15:12:51.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0947999999999999e-05 2020-08-07T15:12:51.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.013 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:51.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251530+0000", 2020-08-07T15:12:51.015 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99863946699999995 2020-08-07T15:12:51.016 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:51.016 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:51.022 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:51.023 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.023 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42505 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:12:51.023 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:12:51.024 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 230.18533759100001, 2020-08-07T15:12:51.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0024736009999999, 2020-08-07T15:12:51.024 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:51.024 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:51.024 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:51.025 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:51.025 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:51.025 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42505 2020-08-07T15:12:51.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.026 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:51.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:51.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:12:51.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:51.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:51.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:12:51.030 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011052600000000001 2020-08-07T15:12:51.030 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.045 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.046 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:51.048 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302342+0000", 2020-08-07T15:12:51.048 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.781e-06 2020-08-07T15:12:51.048 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.049 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.049 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:51.049 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302352+0000", 2020-08-07T15:12:51.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7646e-05 2020-08-07T15:12:51.050 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.050 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.050 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:51.050 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302369+0000", 2020-08-07T15:12:51.051 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3629999999999994e-06 2020-08-07T15:12:51.051 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.051 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:51.052 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302378+0000", 2020-08-07T15:12:51.052 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8007000000000003e-05 2020-08-07T15:12:51.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.052 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.053 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:51.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302426+0000", 2020-08-07T15:12:51.053 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5159999999999994e-06 2020-08-07T15:12:51.053 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.076 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.077 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:51.077 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302434+0000", 2020-08-07T15:12:51.077 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99151562699999995 2020-08-07T15:12:51.078 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.078 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.078 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:51.078 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.293950+0000", 2020-08-07T15:12:51.079 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020138099999999999 2020-08-07T15:12:51.079 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.079 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.079 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:51.079 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294151+0000", 2020-08-07T15:12:51.080 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024761 2020-08-07T15:12:51.080 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.080 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.080 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:12:51.081 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294399+0000", 2020-08-07T15:12:51.081 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0027714369999999999 2020-08-07T15:12:51.081 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.082 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:51.084 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297170+0000", 2020-08-07T15:12:51.084 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.234e-06 2020-08-07T15:12:51.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:51.087 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297172+0000", 2020-08-07T15:12:51.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5384999999999998e-05 2020-08-07T15:12:51.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:51.102 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297237+0000", 2020-08-07T15:12:51.102 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0073576980000000002 2020-08-07T15:12:51.103 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.103 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.103 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:51.103 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304595+0000", 2020-08-07T15:12:51.103 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1006000000000001e-05 2020-08-07T15:12:51.104 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.104 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.104 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:51.104 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304626+0000", 2020-08-07T15:12:51.105 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6000000000000003e-07 2020-08-07T15:12:51.105 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.105 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.105 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:51.106 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304626+0000", 2020-08-07T15:12:51.106 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8724e-05 2020-08-07T15:12:51.106 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.106 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.107 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:51.107 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304705+0000", 2020-08-07T15:12:51.107 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0024736009999999 2020-08-07T15:12:51.107 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:51.108 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:51.108 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:51.108 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.122 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.123 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42506 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:12:51.123 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:12:51.123 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 230.18514637800001, 2020-08-07T15:12:51.123 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002465226, 2020-08-07T15:12:51.123 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:51.124 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:51.124 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:51.124 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:51.124 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:51.125 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42506 2020-08-07T15:12:51.125 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.125 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:51.125 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.130 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:51.131 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:12:51.131 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:51.131 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.131 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.132 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:51.132 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:12:51.132 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8780999999999999e-05 2020-08-07T15:12:51.133 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.133 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.133 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:51.133 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302472+0000", 2020-08-07T15:12:51.134 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3740000000000003e-06 2020-08-07T15:12:51.134 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.134 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.135 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:51.135 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302478+0000", 2020-08-07T15:12:51.135 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1155000000000001e-05 2020-08-07T15:12:51.135 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.136 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.136 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:51.136 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302489+0000", 2020-08-07T15:12:51.137 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7559999999999996e-06 2020-08-07T15:12:51.137 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.137 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.137 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:51.138 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302495+0000", 2020-08-07T15:12:51.144 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4128000000000003e-05 2020-08-07T15:12:51.145 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.145 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.145 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:51.145 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302529+0000", 2020-08-07T15:12:51.146 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.516e-06 2020-08-07T15:12:51.146 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.146 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.146 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:51.147 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302535+0000", 2020-08-07T15:12:51.147 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99240899299999996 2020-08-07T15:12:51.147 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.148 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.148 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:51.148 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294944+0000", 2020-08-07T15:12:51.149 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000115806 2020-08-07T15:12:51.149 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.150 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.150 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:51.150 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295060+0000", 2020-08-07T15:12:51.151 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000205002 2020-08-07T15:12:51.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:12:51.171 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295265+0000", 2020-08-07T15:12:51.172 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043414400000000002 2020-08-07T15:12:51.172 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.173 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.173 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:51.173 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299607+0000", 2020-08-07T15:12:51.174 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2229999999999998e-06 2020-08-07T15:12:51.179 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.179 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.179 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:51.180 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299612+0000", 2020-08-07T15:12:51.180 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5797000000000002e-05 2020-08-07T15:12:51.181 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.181 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.181 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:51.195 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299658+0000", 2020-08-07T15:12:51.196 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005095044 2020-08-07T15:12:51.196 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.197 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.197 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:51.198 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304753+0000", 2020-08-07T15:12:51.198 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3121999999999996e-05 2020-08-07T15:12:51.198 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.199 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.199 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:51.199 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304816+0000", 2020-08-07T15:12:51.199 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5700000000000001e-07 2020-08-07T15:12:51.200 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.200 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.200 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:51.201 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304817+0000", 2020-08-07T15:12:51.201 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1231999999999998e-05 2020-08-07T15:12:51.201 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.203 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.203 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:51.204 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304888+0000", 2020-08-07T15:12:51.204 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002465226 2020-08-07T15:12:51.204 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:51.205 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:51.205 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:51.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.206 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.206 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42507 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:12:51.206 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:12:51.207 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 230.18503560799999, 2020-08-07T15:12:51.207 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002567097, 2020-08-07T15:12:51.207 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:51.207 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:51.207 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:51.208 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:51.208 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:51.208 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42507 2020-08-07T15:12:51.208 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.208 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:51.209 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.209 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:51.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:12:51.209 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:51.209 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.210 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.210 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:51.216 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:12:51.216 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7679000000000002e-05 2020-08-07T15:12:51.216 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.217 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.217 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:51.217 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302581+0000", 2020-08-07T15:12:51.217 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2299999999999999e-06 2020-08-07T15:12:51.217 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.218 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.218 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:51.218 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302586+0000", 2020-08-07T15:12:51.218 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0886999999999999e-05 2020-08-07T15:12:51.218 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.219 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.219 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:51.219 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302597+0000", 2020-08-07T15:12:51.219 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4129999999999997e-06 2020-08-07T15:12:51.219 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.219 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.220 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:51.220 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302603+0000", 2020-08-07T15:12:51.220 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3963000000000001e-05 2020-08-07T15:12:51.220 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.220 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.221 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:51.221 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302627+0000", 2020-08-07T15:12:51.221 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.33e-06 2020-08-07T15:12:51.221 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.221 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.244 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:51.244 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302631+0000", 2020-08-07T15:12:51.244 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99319860900000001 2020-08-07T15:12:51.245 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.245 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.245 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:51.245 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295830+0000", 2020-08-07T15:12:51.246 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000116012 2020-08-07T15:12:51.246 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.246 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.246 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:51.246 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295946+0000", 2020-08-07T15:12:51.247 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000230242 2020-08-07T15:12:51.247 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.247 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.247 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:12:51.247 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296176+0000", 2020-08-07T15:12:51.248 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003494455 2020-08-07T15:12:51.248 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.248 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.248 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:51.248 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299670+0000", 2020-08-07T15:12:51.249 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0299999999999998e-07 2020-08-07T15:12:51.249 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.249 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.249 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:51.249 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299671+0000", 2020-08-07T15:12:51.262 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7298000000000001e-05 2020-08-07T15:12:51.262 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.262 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.263 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:51.263 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299708+0000", 2020-08-07T15:12:51.263 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0052628839999999998 2020-08-07T15:12:51.263 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.263 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.264 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:51.264 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304971+0000", 2020-08-07T15:12:51.264 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3754e-05 2020-08-07T15:12:51.264 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.264 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.265 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:51.265 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305035+0000", 2020-08-07T15:12:51.265 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9500000000000001e-07 2020-08-07T15:12:51.265 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.265 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.266 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:51.266 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305036+0000", 2020-08-07T15:12:51.267 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4843000000000002e-05 2020-08-07T15:12:51.267 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.267 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.267 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:51.267 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305101+0000", 2020-08-07T15:12:51.268 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002567097 2020-08-07T15:12:51.268 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:51.268 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:51.268 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:51.268 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.269 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.269 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42508 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:12:51.269 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:12:51.269 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 230.17451058899999, 2020-08-07T15:12:51.269 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99221695600000004, 2020-08-07T15:12:51.274 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:51.274 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:51.274 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:51.274 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:12:51.275 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:51.275 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42508 2020-08-07T15:12:51.275 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.275 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:51.275 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.276 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:51.278 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:12:51.278 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:51.279 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.279 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.279 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:51.279 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:12:51.279 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3692e-05 2020-08-07T15:12:51.280 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.280 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.280 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:51.280 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313112+0000", 2020-08-07T15:12:51.280 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7399999999999998e-06 2020-08-07T15:12:51.281 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.281 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.281 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:51.281 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313119+0000", 2020-08-07T15:12:51.281 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5153999999999999e-05 2020-08-07T15:12:51.297 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.298 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.298 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:51.298 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313134+0000", 2020-08-07T15:12:51.298 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3039999999999999e-06 2020-08-07T15:12:51.298 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.299 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.299 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:51.299 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313141+0000", 2020-08-07T15:12:51.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6516000000000001e-05 2020-08-07T15:12:51.299 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.300 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.300 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:51.300 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313178+0000", 2020-08-07T15:12:51.300 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0949999999999998e-06 2020-08-07T15:12:51.300 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.301 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.301 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:12:51.307 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313185+0000", 2020-08-07T15:12:51.307 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.98342834999999995 2020-08-07T15:12:51.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.308 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:51.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296613+0000", 2020-08-07T15:12:51.308 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2230999999999993e-05 2020-08-07T15:12:51.308 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.309 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.309 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:51.309 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296686+0000", 2020-08-07T15:12:51.309 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013121300000000001 2020-08-07T15:12:51.309 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.310 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.310 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:12:51.310 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296817+0000", 2020-08-07T15:12:51.310 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044228050000000001 2020-08-07T15:12:51.310 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.310 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.311 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:51.311 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301240+0000", 2020-08-07T15:12:51.311 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0500000000000002e-07 2020-08-07T15:12:51.311 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.311 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.312 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:51.312 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301241+0000", 2020-08-07T15:12:51.312 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2434000000000006e-05 2020-08-07T15:12:51.312 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.312 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.313 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:51.313 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301303+0000", 2020-08-07T15:12:51.313 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038622919999999998 2020-08-07T15:12:51.323 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.323 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.323 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:51.324 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305165+0000", 2020-08-07T15:12:51.324 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8012000000000001e-05 2020-08-07T15:12:51.324 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.324 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.324 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:51.325 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305203+0000", 2020-08-07T15:12:51.325 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5299999999999999e-07 2020-08-07T15:12:51.325 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.325 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.325 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:51.326 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305204+0000", 2020-08-07T15:12:51.326 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1760000000000004e-05 2020-08-07T15:12:51.326 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:51.326 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:51.326 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:51.326 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305275+0000", 2020-08-07T15:12:51.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99221695600000004 2020-08-07T15:12:51.340 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:51.340 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:51.340 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:51.340 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:51.340 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:51.341 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:51.341 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:12:51.343 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 39 16 60303 60287 12.0753 12.3516 0.00795387 0.0103454 2020-08-07T15:12:51.344 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:51.341+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:51.373 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:12:51.449 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:51.447+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:51.558 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:51.550+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:51.571 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:51.571 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:12:51.571 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2020-08-07T15:12:51.572 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:51.579 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:12:51.669 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:51.666+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:51.765 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:51.761+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:51.803 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:12:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:12:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:12:51.804 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:51.818 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:12:51.895 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:51.891+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:51.968 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:51.967+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:52.023 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:12:52.023 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:12:52.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:12:52.024 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:12:52.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.025 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "MOSDPGPushReply(3.1 62/60 [PushReplyOp(3:8329f37e:::benchmark_data_smithi074_35447_object11915:head)])", 2020-08-07T15:12:52.029 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:12:52.030 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 301.16080908100002, 2020-08-07T15:12:52.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996, 2020-08-07T15:12:52.030 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:52.030 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:52.030 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:52.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:52.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:12:52.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:52.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:52.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:12:52.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5372000000000001e-05 2020-08-07T15:12:52.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:52.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858416+0000", 2020-08-07T15:12:52.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6899999999999998e-06 2020-08-07T15:12:52.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:52.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858419+0000", 2020-08-07T15:12:52.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63907904299999996 2020-08-07T15:12:52.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:52.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497498+0000", 2020-08-07T15:12:52.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3332999999999999e-05 2020-08-07T15:12:52.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:52.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497512+0000", 2020-08-07T15:12:52.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000450875 2020-08-07T15:12:52.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497962+0000", 2020-08-07T15:12:52.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000267779 2020-08-07T15:12:52.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:52.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.498230+0000", 2020-08-07T15:12:52.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996 2020-08-07T15:12:52.083 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.083 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:52.084 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.084 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:145792 3.1 3:881e0420:::benchmark_data_smithi074_35447_object16183:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:12:52.084 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:12:52.085 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 277.35240320000003, 2020-08-07T15:12:52.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997, 2020-08-07T15:12:52.085 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:52.085 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:52.089 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:52.092 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:12:52.092 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:12:52.092 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 145792 2020-08-07T15:12:52.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.094 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:52.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:52.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:12:52.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:52.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:52.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:12:52.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7489000000000001e-05 2020-08-07T15:12:52.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:52.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666824+0000", 2020-08-07T15:12:52.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0169999999999998e-06 2020-08-07T15:12:52.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:52.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666828+0000", 2020-08-07T15:12:52.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9829999999999996e-06 2020-08-07T15:12:52.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:52.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666836+0000", 2020-08-07T15:12:52.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.481e-06 2020-08-07T15:12:52.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:52.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666840+0000", 2020-08-07T15:12:52.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5299000000000001e-05 2020-08-07T15:12:52.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666885+0000", 2020-08-07T15:12:52.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014456500000000001 2020-08-07T15:12:52.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:52.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667030+0000", 2020-08-07T15:12:52.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017677799999999999 2020-08-07T15:12:52.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,5", 2020-08-07T15:12:52.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667207+0000", 2020-08-07T15:12:52.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004149649 2020-08-07T15:12:52.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:52.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671356+0000", 2020-08-07T15:12:52.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019339649999999999 2020-08-07T15:12:52.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:52.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673290+0000", 2020-08-07T15:12:52.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73346620500000004 2020-08-07T15:12:52.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:52.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406756+0000", 2020-08-07T15:12:52.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4700000000000001e-06 2020-08-07T15:12:52.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:52.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406759+0000", 2020-08-07T15:12:52.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6258999999999998e-05 2020-08-07T15:12:52.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:52.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406815+0000", 2020-08-07T15:12:52.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0126000000000002e-05 2020-08-07T15:12:52.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:52.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406855+0000", 2020-08-07T15:12:52.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997 2020-08-07T15:12:52.132 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.132 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:52.132 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.133 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:145792 3.1 e75/67 ondisk, result = 0)", 2020-08-07T15:12:52.133 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:12:52.133 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 277.34801831200002, 2020-08-07T15:12:52.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003, 2020-08-07T15:12:52.134 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:52.134 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:12:52.134 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:52.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:52.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:12:52.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:52.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:52.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:12:52.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2830999999999998e-05 2020-08-07T15:12:52.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:52.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671224+0000", 2020-08-07T15:12:52.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:12:52.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:52.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671229+0000", 2020-08-07T15:12:52.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.735414864 2020-08-07T15:12:52.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:52.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406644+0000", 2020-08-07T15:12:52.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1250000000000004e-06 2020-08-07T15:12:52.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:52.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406651+0000", 2020-08-07T15:12:52.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6567e-05 2020-08-07T15:12:52.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406737+0000", 2020-08-07T15:12:52.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0054000000000001e-05 2020-08-07T15:12:52.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:52.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406747+0000", 2020-08-07T15:12:52.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011391999999999999 2020-08-07T15:12:52.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:52.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406861+0000", 2020-08-07T15:12:52.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003 2020-08-07T15:12:52.156 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.156 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:52.157 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.157 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30921 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:12:52.157 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:12:52.158 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 245.31326993799999, 2020-08-07T15:12:52.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001, 2020-08-07T15:12:52.158 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:52.158 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:52.159 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:52.159 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:52.160 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:52.160 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30921 2020-08-07T15:12:52.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.160 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:52.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:52.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:12:52.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:52.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:52.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:12:52.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6954999999999998e-05 2020-08-07T15:12:52.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:52.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706016+0000", 2020-08-07T15:12:52.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0229999999999992e-06 2020-08-07T15:12:52.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:52.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706024+0000", 2020-08-07T15:12:52.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.432e-05 2020-08-07T15:12:52.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:52.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706039+0000", 2020-08-07T15:12:52.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8490000000000006e-06 2020-08-07T15:12:52.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:52.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706046+0000", 2020-08-07T15:12:52.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6837999999999999e-05 2020-08-07T15:12:52.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706113+0000", 2020-08-07T15:12:52.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3553000000000001e-05 2020-08-07T15:12:52.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:52.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706127+0000", 2020-08-07T15:12:52.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77277201699999998 2020-08-07T15:12:52.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.478899+0000", 2020-08-07T15:12:52.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093978599999999996 2020-08-07T15:12:52.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:12:52.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479839+0000", 2020-08-07T15:12:52.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010376029 2020-08-07T15:12:52.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490215+0000", 2020-08-07T15:12:52.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000190657 2020-08-07T15:12:52.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:52.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490405+0000", 2020-08-07T15:12:52.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001 2020-08-07T15:12:52.191 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.191 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:52.192 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.192 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30922 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:12:52.194 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:12:52.195 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 245.31310977499999, 2020-08-07T15:12:52.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247, 2020-08-07T15:12:52.195 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:52.195 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:52.195 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:52.196 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:52.196 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:52.196 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30922 2020-08-07T15:12:52.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.196 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:52.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:52.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:12:52.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:52.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:52.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:12:52.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9178999999999997e-05 2020-08-07T15:12:52.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:52.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706159+0000", 2020-08-07T15:12:52.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4999999999999999e-06 2020-08-07T15:12:52.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:52.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706164+0000", 2020-08-07T15:12:52.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1792e-05 2020-08-07T15:12:52.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:52.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706176+0000", 2020-08-07T15:12:52.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.139e-06 2020-08-07T15:12:52.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:52.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706182+0000", 2020-08-07T15:12:52.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039724900000000002 2020-08-07T15:12:52.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706579+0000", 2020-08-07T15:12:52.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1191e-05 2020-08-07T15:12:52.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:52.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706591+0000", 2020-08-07T15:12:52.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329101300000003 2020-08-07T15:12:52.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479882+0000", 2020-08-07T15:12:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9565000000000001e-05 2020-08-07T15:12:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:12:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479971+0000", 2020-08-07T15:12:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010471263 2020-08-07T15:12:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490442+0000", 2020-08-07T15:12:52.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015035600000000001 2020-08-07T15:12:52.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:52.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490593+0000", 2020-08-07T15:12:52.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247 2020-08-07T15:12:52.255 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.255 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:52.255 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.256 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30923 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:12:52.256 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:12:52.256 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 245.31263471400001, 2020-08-07T15:12:52.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998, 2020-08-07T15:12:52.257 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:52.257 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:52.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:52.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:52.258 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:52.258 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30923 2020-08-07T15:12:52.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.259 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:52.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:52.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:12:52.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:52.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:52.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:12:52.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5531000000000001e-05 2020-08-07T15:12:52.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:52.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706630+0000", 2020-08-07T15:12:52.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7689999999999999e-06 2020-08-07T15:12:52.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:52.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706637+0000", 2020-08-07T15:12:52.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3577999999999999e-05 2020-08-07T15:12:52.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:52.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706650+0000", 2020-08-07T15:12:52.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1439999999999997e-06 2020-08-07T15:12:52.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:52.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706658+0000", 2020-08-07T15:12:52.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1663999999999999e-05 2020-08-07T15:12:52.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706709+0000", 2020-08-07T15:12:52.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0149000000000001e-05 2020-08-07T15:12:52.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:52.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706719+0000", 2020-08-07T15:12:52.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329694900000001 2020-08-07T15:12:52.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480016+0000", 2020-08-07T15:12:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7998999999999997e-05 2020-08-07T15:12:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:12:52.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480084+0000", 2020-08-07T15:12:52.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010547333000000001 2020-08-07T15:12:52.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490632+0000", 2020-08-07T15:12:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7439e-05 2020-08-07T15:12:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490729+0000", 2020-08-07T15:12:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998 2020-08-07T15:12:52.299 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.299 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:52.299 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.300 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30924 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:12:52.300 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:12:52.300 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 245.312507442, 2020-08-07T15:12:52.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995, 2020-08-07T15:12:52.300 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30924 2020-08-07T15:12:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:12:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:12:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9698999999999999e-05 2020-08-07T15:12:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706762+0000", 2020-08-07T15:12:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8080000000000001e-06 2020-08-07T15:12:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706767+0000", 2020-08-07T15:12:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1069e-05 2020-08-07T15:12:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706778+0000", 2020-08-07T15:12:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6969999999999998e-06 2020-08-07T15:12:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706784+0000", 2020-08-07T15:12:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8792999999999998e-05 2020-08-07T15:12:52.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706843+0000", 2020-08-07T15:12:52.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5449999999999998e-06 2020-08-07T15:12:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706850+0000", 2020-08-07T15:12:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.773278354 2020-08-07T15:12:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480129+0000", 2020-08-07T15:12:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1577999999999997e-05 2020-08-07T15:12:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:12:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480210+0000", 2020-08-07T15:12:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010541915000000001 2020-08-07T15:12:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490752+0000", 2020-08-07T15:12:52.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103937 2020-08-07T15:12:52.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:52.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490856+0000", 2020-08-07T15:12:52.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995 2020-08-07T15:12:52.323 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.323 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:52.324 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30925 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:12:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:12:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 245.31238367099999, 2020-08-07T15:12:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995, 2020-08-07T15:12:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30925 2020-08-07T15:12:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.328 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:52.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:52.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:12:52.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:52.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:52.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:12:52.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4664999999999999e-05 2020-08-07T15:12:52.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:52.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706880+0000", 2020-08-07T15:12:52.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7479999999999999e-06 2020-08-07T15:12:52.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:52.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706886+0000", 2020-08-07T15:12:52.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1838000000000001e-05 2020-08-07T15:12:52.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:52.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706898+0000", 2020-08-07T15:12:52.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0089999999999998e-06 2020-08-07T15:12:52.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:52.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706903+0000", 2020-08-07T15:12:52.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3559e-05 2020-08-07T15:12:52.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706956+0000", 2020-08-07T15:12:52.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0309000000000001e-05 2020-08-07T15:12:52.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:52.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706967+0000", 2020-08-07T15:12:52.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77328290499999996 2020-08-07T15:12:52.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480250+0000", 2020-08-07T15:12:52.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8680999999999997e-05 2020-08-07T15:12:52.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:12:52.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480298+0000", 2020-08-07T15:12:52.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01059272 2020-08-07T15:12:52.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490891+0000", 2020-08-07T15:12:52.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014580300000000001 2020-08-07T15:12:52.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:52.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491037+0000", 2020-08-07T15:12:52.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995 2020-08-07T15:12:52.356 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.356 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:52.356 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.357 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30926 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:12:52.357 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:12:52.357 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 245.31226680200001, 2020-08-07T15:12:52.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005, 2020-08-07T15:12:52.358 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:52.358 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:52.358 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:52.358 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:52.358 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:52.359 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30926 2020-08-07T15:12:52.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.359 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:52.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:52.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:12:52.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:52.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:52.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:12:52.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5223000000000003e-05 2020-08-07T15:12:52.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:52.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706998+0000", 2020-08-07T15:12:52.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4809999999999997e-06 2020-08-07T15:12:52.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:52.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707003+0000", 2020-08-07T15:12:52.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1663e-05 2020-08-07T15:12:52.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:52.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707015+0000", 2020-08-07T15:12:52.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9729999999999997e-06 2020-08-07T15:12:52.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:52.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707020+0000", 2020-08-07T15:12:52.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2565999999999999e-05 2020-08-07T15:12:52.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707062+0000", 2020-08-07T15:12:52.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8944e-05 2020-08-07T15:12:52.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:52.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707081+0000", 2020-08-07T15:12:52.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77325885000000005 2020-08-07T15:12:52.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480340+0000", 2020-08-07T15:12:52.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0321000000000006e-05 2020-08-07T15:12:52.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:12:52.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480420+0000", 2020-08-07T15:12:52.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010655958 2020-08-07T15:12:52.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491076+0000", 2020-08-07T15:12:52.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9784999999999995e-05 2020-08-07T15:12:52.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:52.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491166+0000", 2020-08-07T15:12:52.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005 2020-08-07T15:12:52.394 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.394 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:52.395 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.395 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30927 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:12:52.395 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:12:52.396 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 245.31214898299999, 2020-08-07T15:12:52.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999, 2020-08-07T15:12:52.396 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:52.396 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:52.396 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:52.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:52.400 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:52.401 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30927 2020-08-07T15:12:52.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.410 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:52.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:52.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:12:52.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:52.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:52.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:12:52.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9861999999999997e-05 2020-08-07T15:12:52.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:52.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707120+0000", 2020-08-07T15:12:52.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8969999999999996e-06 2020-08-07T15:12:52.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:52.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707126+0000", 2020-08-07T15:12:52.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1248e-05 2020-08-07T15:12:52.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:52.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707137+0000", 2020-08-07T15:12:52.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.955e-06 2020-08-07T15:12:52.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:52.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707142+0000", 2020-08-07T15:12:52.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0643999999999998e-05 2020-08-07T15:12:52.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707193+0000", 2020-08-07T15:12:52.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1983e-05 2020-08-07T15:12:52.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:52.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707205+0000", 2020-08-07T15:12:52.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77324914 2020-08-07T15:12:52.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480454+0000", 2020-08-07T15:12:52.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4119999999999998e-05 2020-08-07T15:12:52.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:12:52.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480498+0000", 2020-08-07T15:12:52.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010709707000000001 2020-08-07T15:12:52.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491208+0000", 2020-08-07T15:12:52.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014433800000000001 2020-08-07T15:12:52.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:52.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491352+0000", 2020-08-07T15:12:52.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999 2020-08-07T15:12:52.436 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.437 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:52.437 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.437 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30928 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:12:52.438 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:12:52.438 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 245.31202462600001, 2020-08-07T15:12:52.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998, 2020-08-07T15:12:52.438 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:52.439 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:12:52.439 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:52.439 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:52.439 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:52.439 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30928 2020-08-07T15:12:52.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.440 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:52.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:52.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:12:52.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:52.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:52.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:12:52.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6582000000000001e-05 2020-08-07T15:12:52.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:52.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707241+0000", 2020-08-07T15:12:52.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9190000000000004e-06 2020-08-07T15:12:52.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:52.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707247+0000", 2020-08-07T15:12:52.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000188822 2020-08-07T15:12:52.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:52.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707436+0000", 2020-08-07T15:12:52.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2940000000000002e-06 2020-08-07T15:12:52.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:52.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707442+0000", 2020-08-07T15:12:52.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7800000000000003e-05 2020-08-07T15:12:52.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707490+0000", 2020-08-07T15:12:52.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2020-08-07T15:12:52.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:52.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707501+0000", 2020-08-07T15:12:52.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77303498800000003 2020-08-07T15:12:52.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480536+0000", 2020-08-07T15:12:52.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7723000000000001e-05 2020-08-07T15:12:52.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:12:52.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480614+0000", 2020-08-07T15:12:52.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010812183 2020-08-07T15:12:52.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491426+0000", 2020-08-07T15:12:52.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5502999999999999e-05 2020-08-07T15:12:52.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:52.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491502+0000", 2020-08-07T15:12:52.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998 2020-08-07T15:12:52.460 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.460 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:52.460 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.461 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31009 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:12:52.461 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:12:52.461 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 245.28806832000001, 2020-08-07T15:12:52.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002, 2020-08-07T15:12:52.462 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:52.462 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:52.462 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:52.462 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:52.462 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:52.463 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31009 2020-08-07T15:12:52.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.463 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:52.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:52.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:12:52.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:52.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:52.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:12:52.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2429000000000002e-05 2020-08-07T15:12:52.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:52.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731193+0000", 2020-08-07T15:12:52.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9760000000000003e-06 2020-08-07T15:12:52.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:52.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731198+0000", 2020-08-07T15:12:52.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3597e-05 2020-08-07T15:12:52.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:52.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731212+0000", 2020-08-07T15:12:52.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1709999999999996e-06 2020-08-07T15:12:52.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:52.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731219+0000", 2020-08-07T15:12:52.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5402999999999997e-05 2020-08-07T15:12:52.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731285+0000", 2020-08-07T15:12:52.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3495e-05 2020-08-07T15:12:52.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:52.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731298+0000", 2020-08-07T15:12:52.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74935423599999995 2020-08-07T15:12:52.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480652+0000", 2020-08-07T15:12:52.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000137806 2020-08-07T15:12:52.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:52.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480790+0000", 2020-08-07T15:12:52.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022272399999999999 2020-08-07T15:12:52.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:12:52.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481013+0000", 2020-08-07T15:12:52.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003223381 2020-08-07T15:12:52.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:52.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484236+0000", 2020-08-07T15:12:52.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1370000000000001e-06 2020-08-07T15:12:52.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:52.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484237+0000", 2020-08-07T15:12:52.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6288e-05 2020-08-07T15:12:52.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:52.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484294+0000", 2020-08-07T15:12:52.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.007260402 2020-08-07T15:12:52.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:52.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491554+0000", 2020-08-07T15:12:52.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7358000000000003e-05 2020-08-07T15:12:52.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:52.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491601+0000", 2020-08-07T15:12:52.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1100000000000004e-07 2020-08-07T15:12:52.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:52.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491602+0000", 2020-08-07T15:12:52.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013436500000000001 2020-08-07T15:12:52.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:52.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491737+0000", 2020-08-07T15:12:52.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002 2020-08-07T15:12:52.488 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.488 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:52.488 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.489 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31010 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:12:52.489 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:12:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 245.28794236600001, 2020-08-07T15:12:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001, 2020-08-07T15:12:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31010 2020-08-07T15:12:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:12:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:12:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2398000000000002e-05 2020-08-07T15:12:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731319+0000", 2020-08-07T15:12:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7100000000000004e-06 2020-08-07T15:12:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731325+0000", 2020-08-07T15:12:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4328000000000001e-05 2020-08-07T15:12:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731339+0000", 2020-08-07T15:12:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3310000000000003e-06 2020-08-07T15:12:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731345+0000", 2020-08-07T15:12:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8952999999999998e-05 2020-08-07T15:12:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731384+0000", 2020-08-07T15:12:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0036e-05 2020-08-07T15:12:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731394+0000", 2020-08-07T15:12:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75027681300000004 2020-08-07T15:12:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481670+0000", 2020-08-07T15:12:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016555899999999999 2020-08-07T15:12:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481836+0000", 2020-08-07T15:12:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028760999999999999 2020-08-07T15:12:52.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:12:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482124+0000", 2020-08-07T15:12:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038342680000000001 2020-08-07T15:12:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485958+0000", 2020-08-07T15:12:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2559999999999999e-06 2020-08-07T15:12:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485959+0000", 2020-08-07T15:12:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3758999999999995e-05 2020-08-07T15:12:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486043+0000", 2020-08-07T15:12:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057590419999999998 2020-08-07T15:12:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491802+0000", 2020-08-07T15:12:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5757000000000001e-05 2020-08-07T15:12:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:12:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9299999999999997e-07 2020-08-07T15:12:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:12:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012496999999999999 2020-08-07T15:12:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491993+0000", 2020-08-07T15:12:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001 2020-08-07T15:12:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31011 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:12:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:12:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 245.287826161, 2020-08-07T15:12:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997, 2020-08-07T15:12:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31011 2020-08-07T15:12:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:12:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:52.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:12:52.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9539999999999997e-05 2020-08-07T15:12:52.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:52.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731433+0000", 2020-08-07T15:12:52.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9209999999999997e-06 2020-08-07T15:12:52.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:52.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731439+0000", 2020-08-07T15:12:52.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4246999999999999e-05 2020-08-07T15:12:52.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:52.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731453+0000", 2020-08-07T15:12:52.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8130000000000003e-06 2020-08-07T15:12:52.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:52.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731458+0000", 2020-08-07T15:12:52.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6668000000000001e-05 2020-08-07T15:12:52.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731504+0000", 2020-08-07T15:12:52.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1112e-05 2020-08-07T15:12:52.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:52.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731515+0000", 2020-08-07T15:12:52.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75121332100000004 2020-08-07T15:12:52.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482729+0000", 2020-08-07T15:12:52.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1810000000000004e-05 2020-08-07T15:12:52.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:52.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482811+0000", 2020-08-07T15:12:52.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018001399999999999 2020-08-07T15:12:52.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:12:52.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482991+0000", 2020-08-07T15:12:52.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030701140000000001 2020-08-07T15:12:52.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:52.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486061+0000", 2020-08-07T15:12:52.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4499999999999996e-07 2020-08-07T15:12:52.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:52.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486062+0000", 2020-08-07T15:12:52.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5366999999999998e-05 2020-08-07T15:12:52.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:52.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486147+0000", 2020-08-07T15:12:52.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056872779999999996 2020-08-07T15:12:52.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:52.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491834+0000", 2020-08-07T15:12:52.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024645100000000002 2020-08-07T15:12:52.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:52.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:12:52.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1200000000000003e-07 2020-08-07T15:12:52.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:52.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:12:52.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1451000000000002e-05 2020-08-07T15:12:52.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:52.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492173+0000", 2020-08-07T15:12:52.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997 2020-08-07T15:12:52.625 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.625 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:52.625 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.626 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31012 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:12:52.626 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:12:52.627 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 245.28717204200001, 2020-08-07T15:12:52.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003, 2020-08-07T15:12:52.627 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:52.627 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:52.627 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:52.627 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:52.628 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:52.628 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31012 2020-08-07T15:12:52.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.628 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:52.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:52.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:12:52.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:52.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:52.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:12:52.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6344999999999997e-05 2020-08-07T15:12:52.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:52.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732094+0000", 2020-08-07T15:12:52.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2920000000000001e-06 2020-08-07T15:12:52.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:52.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732100+0000", 2020-08-07T15:12:52.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.189e-05 2020-08-07T15:12:52.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:52.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732112+0000", 2020-08-07T15:12:52.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3720000000000001e-06 2020-08-07T15:12:52.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:52.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732118+0000", 2020-08-07T15:12:52.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2514000000000004e-05 2020-08-07T15:12:52.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732191+0000", 2020-08-07T15:12:52.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4610000000000001e-05 2020-08-07T15:12:52.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:52.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732205+0000", 2020-08-07T15:12:52.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75132974100000005 2020-08-07T15:12:52.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483535+0000", 2020-08-07T15:12:52.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9185999999999996e-05 2020-08-07T15:12:52.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:52.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483624+0000", 2020-08-07T15:12:52.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015683 2020-08-07T15:12:52.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:12:52.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483781+0000", 2020-08-07T15:12:52.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046202889999999996 2020-08-07T15:12:52.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:52.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488401+0000", 2020-08-07T15:12:52.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6700000000000002e-07 2020-08-07T15:12:52.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:52.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488402+0000", 2020-08-07T15:12:52.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1487000000000001e-05 2020-08-07T15:12:52.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:52.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488484+0000", 2020-08-07T15:12:52.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035520830000000002 2020-08-07T15:12:52.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:52.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492036+0000", 2020-08-07T15:12:52.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029266600000000002 2020-08-07T15:12:52.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:52.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492328+0000", 2020-08-07T15:12:52.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5099999999999998e-07 2020-08-07T15:12:52.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:52.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492329+0000", 2020-08-07T15:12:52.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000201299 2020-08-07T15:12:52.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:52.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492531+0000", 2020-08-07T15:12:52.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003 2020-08-07T15:12:52.657 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.657 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:52.658 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.660 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31013 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:12:52.661 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:12:52.661 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 245.286960411, 2020-08-07T15:12:52.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996, 2020-08-07T15:12:52.662 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:52.662 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:52.662 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:52.662 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:52.662 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:52.663 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31013 2020-08-07T15:12:52.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.664 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:52.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:52.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:12:52.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:52.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:52.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:12:52.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8949000000000002e-05 2020-08-07T15:12:52.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:52.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732298+0000", 2020-08-07T15:12:52.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1399999999999997e-06 2020-08-07T15:12:52.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:52.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732304+0000", 2020-08-07T15:12:52.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.154e-05 2020-08-07T15:12:52.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:52.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732315+0000", 2020-08-07T15:12:52.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1319999999999999e-06 2020-08-07T15:12:52.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:52.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732322+0000", 2020-08-07T15:12:52.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7431e-05 2020-08-07T15:12:52.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732369+0000", 2020-08-07T15:12:52.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0621999999999999e-05 2020-08-07T15:12:52.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:52.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732380+0000", 2020-08-07T15:12:52.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75196845999999995 2020-08-07T15:12:52.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484348+0000", 2020-08-07T15:12:52.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000123008 2020-08-07T15:12:52.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:52.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484471+0000", 2020-08-07T15:12:52.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017572700000000001 2020-08-07T15:12:52.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:12:52.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484647+0000", 2020-08-07T15:12:52.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038557650000000001 2020-08-07T15:12:52.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:52.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:12:52.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2020-08-07T15:12:52.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:52.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:12:52.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0714e-05 2020-08-07T15:12:52.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:52.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488584+0000", 2020-08-07T15:12:52.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036298319999999999 2020-08-07T15:12:52.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:52.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492214+0000", 2020-08-07T15:12:52.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039530099999999997 2020-08-07T15:12:52.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:52.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492609+0000", 2020-08-07T15:12:52.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5300000000000002e-07 2020-08-07T15:12:52.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:52.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492610+0000", 2020-08-07T15:12:52.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197321 2020-08-07T15:12:52.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:52.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492807+0000", 2020-08-07T15:12:52.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996 2020-08-07T15:12:52.706 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.707 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:52.707 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.707 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31014 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:12:52.708 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:12:52.708 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 245.28557826700001, 2020-08-07T15:12:52.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998, 2020-08-07T15:12:52.709 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:52.709 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:52.710 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:52.710 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:52.710 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:52.710 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31014 2020-08-07T15:12:52.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.711 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:52.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:52.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:12:52.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:52.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:52.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:12:52.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3743999999999999e-05 2020-08-07T15:12:52.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:52.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733685+0000", 2020-08-07T15:12:52.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1720000000000007e-06 2020-08-07T15:12:52.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:52.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733693+0000", 2020-08-07T15:12:52.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3674e-05 2020-08-07T15:12:52.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:52.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733707+0000", 2020-08-07T15:12:52.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9249999999999998e-06 2020-08-07T15:12:52.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:52.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733714+0000", 2020-08-07T15:12:52.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8338000000000003e-05 2020-08-07T15:12:52.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733772+0000", 2020-08-07T15:12:52.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3019e-05 2020-08-07T15:12:52.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:52.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733785+0000", 2020-08-07T15:12:52.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75134856299999997 2020-08-07T15:12:52.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485133+0000", 2020-08-07T15:12:52.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8739999999999995e-05 2020-08-07T15:12:52.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:52.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485212+0000", 2020-08-07T15:12:52.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151244 2020-08-07T15:12:52.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:12:52.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485363+0000", 2020-08-07T15:12:52.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032929090000000001 2020-08-07T15:12:52.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:52.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488656+0000", 2020-08-07T15:12:52.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.065e-06 2020-08-07T15:12:52.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:52.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488657+0000", 2020-08-07T15:12:52.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1123000000000003e-05 2020-08-07T15:12:52.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:52.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488729+0000", 2020-08-07T15:12:52.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035508330000000002 2020-08-07T15:12:52.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:52.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492279+0000", 2020-08-07T15:12:52.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00059376000000000003 2020-08-07T15:12:52.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:52.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492873+0000", 2020-08-07T15:12:52.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5700000000000002e-07 2020-08-07T15:12:52.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:52.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492874+0000", 2020-08-07T15:12:52.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019264 2020-08-07T15:12:52.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:52.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493066+0000", 2020-08-07T15:12:52.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998 2020-08-07T15:12:52.752 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.753 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:52.753 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.754 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31015 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:12:52.754 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:12:52.754 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 245.28545230099999, 2020-08-07T15:12:52.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002, 2020-08-07T15:12:52.754 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:52.755 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:52.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:52.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:52.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:52.759 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31015 2020-08-07T15:12:52.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.759 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:52.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:52.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:12:52.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:52.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:52.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:12:52.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3707000000000001e-05 2020-08-07T15:12:52.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:52.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733791+0000", 2020-08-07T15:12:52.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8120000000000001e-06 2020-08-07T15:12:52.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:52.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733794+0000", 2020-08-07T15:12:52.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0728e-05 2020-08-07T15:12:52.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:52.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733805+0000", 2020-08-07T15:12:52.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8279999999999999e-06 2020-08-07T15:12:52.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:52.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733811+0000", 2020-08-07T15:12:52.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2955999999999999e-05 2020-08-07T15:12:52.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733864+0000", 2020-08-07T15:12:52.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1389000000000001e-05 2020-08-07T15:12:52.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:52.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733875+0000", 2020-08-07T15:12:52.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75232711100000005 2020-08-07T15:12:52.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486202+0000", 2020-08-07T15:12:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027503400000000001 2020-08-07T15:12:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:52.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486478+0000", 2020-08-07T15:12:52.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033010400000000001 2020-08-07T15:12:52.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:12:52.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486808+0000", 2020-08-07T15:12:52.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019382609999999999 2020-08-07T15:12:52.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:52.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488746+0000", 2020-08-07T15:12:52.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9400000000000004e-07 2020-08-07T15:12:52.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:52.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488747+0000", 2020-08-07T15:12:52.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4324000000000002e-05 2020-08-07T15:12:52.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:52.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488821+0000", 2020-08-07T15:12:52.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043745640000000001 2020-08-07T15:12:52.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:52.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493196+0000", 2020-08-07T15:12:52.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030381000000000001 2020-08-07T15:12:52.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:52.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493499+0000", 2020-08-07T15:12:52.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4e-07 2020-08-07T15:12:52.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:52.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493500+0000", 2020-08-07T15:12:52.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017499 2020-08-07T15:12:52.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:52.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493675+0000", 2020-08-07T15:12:52.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002 2020-08-07T15:12:52.833 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.833 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:52.834 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.840 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31016 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:12:52.841 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:12:52.841 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 245.28485632600001, 2020-08-07T15:12:52.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971, 2020-08-07T15:12:52.841 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:52.842 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:52.842 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:12:52.843 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:12:52.843 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:12:52.844 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31016 2020-08-07T15:12:52.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.844 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:52.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:52.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:12:52.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:52.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:52.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:12:52.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0924000000000002e-05 2020-08-07T15:12:52.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:52.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734404+0000", 2020-08-07T15:12:52.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2020-08-07T15:12:52.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:52.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734410+0000", 2020-08-07T15:12:52.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3459999999999994e-06 2020-08-07T15:12:52.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:52.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734418+0000", 2020-08-07T15:12:52.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9640000000000002e-06 2020-08-07T15:12:52.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:52.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734423+0000", 2020-08-07T15:12:52.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7253999999999999e-05 2020-08-07T15:12:52.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734470+0000", 2020-08-07T15:12:52.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0249999999999997e-06 2020-08-07T15:12:52.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:12:52.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734477+0000", 2020-08-07T15:12:52.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75294592900000001 2020-08-07T15:12:52.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487423+0000", 2020-08-07T15:12:52.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031604300000000001 2020-08-07T15:12:52.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:52.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487739+0000", 2020-08-07T15:12:52.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000171149 2020-08-07T15:12:52.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:12:52.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487910+0000", 2020-08-07T15:12:52.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024207769999999998 2020-08-07T15:12:52.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:12:52.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490331+0000", 2020-08-07T15:12:52.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1650000000000001e-06 2020-08-07T15:12:52.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:52.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490332+0000", 2020-08-07T15:12:52.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9796999999999996e-05 2020-08-07T15:12:52.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:52.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490402+0000", 2020-08-07T15:12:52.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029928849999999998 2020-08-07T15:12:52.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:52.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493395+0000", 2020-08-07T15:12:52.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034903199999999999 2020-08-07T15:12:52.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:52.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493744+0000", 2020-08-07T15:12:52.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8899999999999999e-07 2020-08-07T15:12:52.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:12:52.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493745+0000", 2020-08-07T15:12:52.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197362 2020-08-07T15:12:52.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:52.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493942+0000", 2020-08-07T15:12:52.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971 2020-08-07T15:12:52.882 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.883 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:52.883 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.883 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:89632 5.f e207/164 5:fd260020:::benchmark_data_smithi074_53697_object11203:head v 207'5744, mlcod=207'5734)", 2020-08-07T15:12:52.884 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:21.551762+0000", 2020-08-07T15:12:52.884 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 90.467447469000007, 2020-08-07T15:12:52.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67527211499999995, 2020-08-07T15:12:52.884 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:12:52.884 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:52.885 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:12:52.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:12:52.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551762+0000", 2020-08-07T15:12:52.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:12:52.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:12:52.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551762+0000", 2020-08-07T15:12:52.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8560000000000003e-05 2020-08-07T15:12:52.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:12:52.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551810+0000", 2020-08-07T15:12:52.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9430000000000002e-06 2020-08-07T15:12:52.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:12:52.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551814+0000", 2020-08-07T15:12:52.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67230272000000002 2020-08-07T15:12:52.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:12:52.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.224117+0000", 2020-08-07T15:12:52.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7120000000000003e-06 2020-08-07T15:12:52.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:12:52.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.224124+0000", 2020-08-07T15:12:52.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001719307 2020-08-07T15:12:52.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:12:52.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.225843+0000", 2020-08-07T15:12:52.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3902999999999997e-05 2020-08-07T15:12:52.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:12:52.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.225877+0000", 2020-08-07T15:12:52.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001110279 2020-08-07T15:12:52.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:12:52.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.226987+0000", 2020-08-07T15:12:52.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6690999999999997e-05 2020-08-07T15:12:52.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:12:52.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:12:52.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:12:52.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.227034+0000", 2020-08-07T15:12:52.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67527211499999995 2020-08-07T15:12:52.901 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.901 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:52.901 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.901 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:12:52.902 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:12:52.902 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:12:52.903 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:12:52.905 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:2020-08-07T15:12:52.268666+0000 min lat: 0.000920324 max lat: 0.487071 avg lat: 0.0103609 2020-08-07T15:12:52.905 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-08-07T15:12:52.905 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 40 16 61763 61747 12.0586 11.4062 0.00503648 0.0103609 2020-08-07T15:12:52.905 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:12:52.906 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:12:52.929 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:52.903+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:53.007 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:53.004+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:53.107 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:53.105+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:53.211 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:53.210+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:53.270 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 41 16 63428 63412 12.0817 13.0078 0.00621115 0.0103396 2020-08-07T15:12:53.315 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:53.313+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:53.425 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:53.424+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:53.478 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:12:53.518 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2020-08-07T15:12:53.518 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:12:53.519 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump -f json-pretty 2020-08-07T15:12:53.520 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:12:53.534 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:53.533+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:53.554 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:12:53.596 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:12:53.642 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:53.634+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:53.643 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:12:53.686 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:12:53.732 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:12:53.742 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:53.735+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:53.774 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:12:53.779 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:12:53.780 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:12:53.780 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:12:53.780 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:12:53.781 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:12:53.781 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:12:53.781 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:12:53.781 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:12:53.826 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:53.827 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:53.836 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:12:53.843 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:53.841+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:53.946 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:53.942+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:53.994 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:12:53.995 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:53.995 INFO:teuthology.orchestra.run.smithi074.stdout: "epoch": 264, 2020-08-07T15:12:53.996 INFO:teuthology.orchestra.run.smithi074.stdout: "fsid": "a01d3e82-8d24-4288-9e42-576ab3ff4957", 2020-08-07T15:12:53.996 INFO:teuthology.orchestra.run.smithi074.stdout: "created": "2020-08-07T15:06:07.342912+0000", 2020-08-07T15:12:53.996 INFO:teuthology.orchestra.run.smithi074.stdout: "modified": "2020-08-07T15:12:52.516917+0000", 2020-08-07T15:12:53.996 INFO:teuthology.orchestra.run.smithi074.stdout: "last_up_change": "2020-08-07T15:12:05.338271+0000", 2020-08-07T15:12:53.996 INFO:teuthology.orchestra.run.smithi074.stdout: "last_in_change": "2020-08-07T15:06:17.674292+0000", 2020-08-07T15:12:53.997 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-08-07T15:12:53.997 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_num": 5804032, 2020-08-07T15:12:53.997 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_set": [ 2020-08-07T15:12:53.997 INFO:teuthology.orchestra.run.smithi074.stdout: "nodeep-scrub", 2020-08-07T15:12:53.997 INFO:teuthology.orchestra.run.smithi074.stdout: "pglog_hardlimit", 2020-08-07T15:12:53.998 INFO:teuthology.orchestra.run.smithi074.stdout: "purged_snapdirs", 2020-08-07T15:12:53.998 INFO:teuthology.orchestra.run.smithi074.stdout: "recovery_deletes", 2020-08-07T15:12:53.998 INFO:teuthology.orchestra.run.smithi074.stdout: "sortbitwise" 2020-08-07T15:12:53.998 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:12:53.999 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_version": 5, 2020-08-07T15:12:53.999 INFO:teuthology.orchestra.run.smithi074.stdout: "full_ratio": 0.94999998807907104, 2020-08-07T15:12:53.999 INFO:teuthology.orchestra.run.smithi074.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-08-07T15:12:53.999 INFO:teuthology.orchestra.run.smithi074.stdout: "nearfull_ratio": 0.85000002384185791, 2020-08-07T15:12:53.999 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_snapshot": "", 2020-08-07T15:12:54.000 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_max": 6, 2020-08-07T15:12:54.000 INFO:teuthology.orchestra.run.smithi074.stdout: "max_osd": 8, 2020-08-07T15:12:54.000 INFO:teuthology.orchestra.run.smithi074.stdout: "require_min_compat_client": "luminous", 2020-08-07T15:12:54.000 INFO:teuthology.orchestra.run.smithi074.stdout: "min_compat_client": "luminous", 2020-08-07T15:12:54.000 INFO:teuthology.orchestra.run.smithi074.stdout: "require_osd_release": "pacific", 2020-08-07T15:12:54.001 INFO:teuthology.orchestra.run.smithi074.stdout: "pools": [ 2020-08-07T15:12:54.001 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.001 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 1, 2020-08-07T15:12:54.001 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "device_health_metrics", 2020-08-07T15:12:54.001 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:06:10.648876+0000", 2020-08-07T15:12:54.007 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:12:54.007 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:12:54.008 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:12:54.008 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:12:54.008 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:12:54.008 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:12:54.009 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:12:54.009 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:12:54.009 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 41, 2020-08-07T15:12:54.009 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 41, 2020-08-07T15:12:54.009 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 41, 2020-08-07T15:12:54.010 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 41, 2020-08-07T15:12:54.010 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 41, 2020-08-07T15:12:54.010 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:12:54.010 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:12:54.010 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:12:54.011 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:12:54.011 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:12:54.011 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:12:54.011 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:12:54.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.012 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "246", 2020-08-07T15:12:54.012 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:12:54.012 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:12:54.033 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "244", 2020-08-07T15:12:54.033 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:12:54.033 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:12:54.033 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:12:54.033 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:12:54.034 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:12:54.034 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:12:54.034 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:12:54.034 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:12:54.035 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:12:54.035 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:12:54.035 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:12:54.035 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:12:54.038 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:12:54.039 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:12:54.039 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:12:54.039 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:12:54.039 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:12:54.039 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:12:54.040 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:12:54.040 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:12:54.050 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:12:54.050 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:12:54.054 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:12:54.055 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.055 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:12:54.055 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:12:54.056 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:12:54.056 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:12:54.056 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:12:54.056 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:12:54.056 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:12:54.056 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:12:54.061 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:12:54.061 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:12:54.061 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:12:54.062 INFO:teuthology.orchestra.run.smithi074.stdout: "options": { 2020-08-07T15:12:54.090 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_min": 1 2020-08-07T15:12:54.090 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.090 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:12:54.091 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_devicehealth": {} 2020-08-07T15:12:54.091 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.091 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.091 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.091 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 2, 2020-08-07T15:12:54.092 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "rbd", 2020-08-07T15:12:54.092 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:06:21.474648+0000", 2020-08-07T15:12:54.092 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:12:54.092 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:12:54.092 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:12:54.092 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:12:54.093 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:12:54.093 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:12:54.093 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:12:54.093 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:12:54.094 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 8, 2020-08-07T15:12:54.094 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 8, 2020-08-07T15:12:54.094 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 8, 2020-08-07T15:12:54.094 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 8, 2020-08-07T15:12:54.095 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 8, 2020-08-07T15:12:54.095 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:12:54.095 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:12:54.095 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:12:54.095 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:12:54.096 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:12:54.096 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:12:54.096 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:12:54.096 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.096 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "19", 2020-08-07T15:12:54.096 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:12:54.097 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:12:54.097 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "0", 2020-08-07T15:12:54.097 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:12:54.097 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:12:54.097 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:12:54.098 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:12:54.098 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:12:54.098 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:12:54.098 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:12:54.098 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:12:54.099 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:12:54.099 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:12:54.099 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:12:54.099 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:12:54.099 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:12:54.099 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:12:54.100 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:12:54.100 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:12:54.100 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:12:54.100 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:12:54.100 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:12:54.101 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:12:54.101 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:12:54.101 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:12:54.101 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:12:54.101 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.105 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:12:54.106 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:12:54.106 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:12:54.106 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:12:54.106 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:12:54.107 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:12:54.107 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:12:54.107 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:12:54.107 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:12:54.107 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:12:54.107 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:12:54.108 INFO:teuthology.orchestra.run.smithi074.stdout: "options": {}, 2020-08-07T15:12:54.108 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:12:54.108 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd": {} 2020-08-07T15:12:54.108 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.108 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.109 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.109 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 6, 2020-08-07T15:12:54.109 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "unique_pool_3", 2020-08-07T15:12:54.109 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:12:08.847598+0000", 2020-08-07T15:12:54.109 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:12:54.110 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:12:54.110 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:12:54.110 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:12:54.110 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:12:54.111 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:12:54.111 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:12:54.111 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:12:54.111 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 16, 2020-08-07T15:12:54.111 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 16, 2020-08-07T15:12:54.112 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 16, 2020-08-07T15:12:54.112 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 16, 2020-08-07T15:12:54.112 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 16, 2020-08-07T15:12:54.112 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:12:54.112 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:12:54.113 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:12:54.113 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:12:54.113 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:12:54.113 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:12:54.113 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:12:54.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.114 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "232", 2020-08-07T15:12:54.114 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:12:54.114 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:12:54.114 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "0", 2020-08-07T15:12:54.115 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:12:54.115 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:12:54.115 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:12:54.115 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:12:54.115 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:12:54.115 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:12:54.116 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:12:54.116 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:12:54.116 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:12:54.116 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:12:54.116 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:12:54.117 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:12:54.117 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:12:54.117 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:12:54.117 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:12:54.118 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:12:54.118 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:12:54.118 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:12:54.118 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:12:54.118 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:12:54.119 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:12:54.119 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:12:54.119 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:12:54.119 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.119 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:12:54.119 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:12:54.120 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:12:54.120 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:12:54.120 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:12:54.120 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:12:54.120 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:12:54.121 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:12:54.121 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:12:54.121 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:12:54.121 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:12:54.121 INFO:teuthology.orchestra.run.smithi074.stdout: "options": {}, 2020-08-07T15:12:54.122 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:12:54.122 INFO:teuthology.orchestra.run.smithi074.stdout: "rados": {} 2020-08-07T15:12:54.122 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.122 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.122 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:12:54.122 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:12:54.123 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.123 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 0, 2020-08-07T15:12:54.123 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "fc1ebb23-3ef4-4fc0-b395-382e385885a3", 2020-08-07T15:12:54.123 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:12:54.123 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:12:54.124 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:12:54.124 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0, 2020-08-07T15:12:54.124 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:12:54.125 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:12:54.125 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:12:54.125 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 262, 2020-08-07T15:12:54.125 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:12:54.125 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:12:54.126 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:12:54.126 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.126 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.126 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.126 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6801", 2020-08-07T15:12:54.127 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:12:54.127 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.127 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.127 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.127 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:12:54.128 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.128 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.128 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.128 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6802", 2020-08-07T15:12:54.128 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:12:54.129 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.129 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.129 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.129 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:12:54.129 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.130 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.130 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.130 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6804", 2020-08-07T15:12:54.130 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:12:54.131 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.131 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.131 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.131 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:12:54.131 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.131 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.132 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.132 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6803", 2020-08-07T15:12:54.132 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:12:54.132 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.132 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.133 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.133 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6801/31577", 2020-08-07T15:12:54.133 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6802/31577", 2020-08-07T15:12:54.133 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6804/31577", 2020-08-07T15:12:54.133 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6803/31577", 2020-08-07T15:12:54.134 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:12:54.134 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:12:54.134 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:12:54.134 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.134 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.134 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.135 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 1, 2020-08-07T15:12:54.135 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "1fbe530a-63ce-4dc8-82ab-18793d6ebc0b", 2020-08-07T15:12:54.135 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:12:54.135 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:12:54.135 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:12:54.136 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0.7196044921875, 2020-08-07T15:12:54.136 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 13, 2020-08-07T15:12:54.136 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 190, 2020-08-07T15:12:54.136 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 227, 2020-08-07T15:12:54.136 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 249, 2020-08-07T15:12:54.137 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 191, 2020-08-07T15:12:54.137 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:12:54.137 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:12:54.137 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.137 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.138 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.138 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6809", 2020-08-07T15:12:54.138 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 62587 2020-08-07T15:12:54.138 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.138 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.139 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.139 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:12:54.139 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.139 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.139 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.139 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6810", 2020-08-07T15:12:54.140 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 62587 2020-08-07T15:12:54.140 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.140 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.140 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.140 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:12:54.141 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.141 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.141 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.141 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6812", 2020-08-07T15:12:54.141 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 62587 2020-08-07T15:12:54.142 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.142 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.142 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.142 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:12:54.142 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.143 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.143 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.143 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6811", 2020-08-07T15:12:54.143 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 62587 2020-08-07T15:12:54.143 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.143 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.144 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.144 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6809/62587", 2020-08-07T15:12:54.144 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6810/62587", 2020-08-07T15:12:54.144 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6812/62587", 2020-08-07T15:12:54.144 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6811/62587", 2020-08-07T15:12:54.145 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:12:54.145 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:12:54.145 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:12:54.145 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.145 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.146 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.146 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 2, 2020-08-07T15:12:54.146 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "d6fe4338-3e11-4f73-9cbb-e1b046ce286a", 2020-08-07T15:12:54.146 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:12:54.147 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:12:54.147 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:12:54.147 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:12:54.147 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:12:54.147 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:12:54.148 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:12:54.148 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 256, 2020-08-07T15:12:54.148 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:12:54.148 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:12:54.148 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:12:54.148 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.149 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.149 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.149 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6805", 2020-08-07T15:12:54.149 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:12:54.149 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.150 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.150 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.150 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:12:54.150 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.151 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.151 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6806", 2020-08-07T15:12:54.151 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:12:54.151 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.152 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.152 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:12:54.152 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.152 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.153 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6808", 2020-08-07T15:12:54.153 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:12:54.153 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.153 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.153 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.154 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:12:54.154 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.154 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.154 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.154 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6807", 2020-08-07T15:12:54.155 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:12:54.155 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.155 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.155 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.155 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6805/31576", 2020-08-07T15:12:54.155 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6806/31576", 2020-08-07T15:12:54.156 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6808/31576", 2020-08-07T15:12:54.156 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6807/31576", 2020-08-07T15:12:54.156 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:12:54.156 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:12:54.156 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:12:54.157 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.157 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.157 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.157 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 3, 2020-08-07T15:12:54.157 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9", 2020-08-07T15:12:54.158 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:12:54.158 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:12:54.158 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:12:54.158 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:12:54.158 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:12:54.159 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:12:54.159 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:12:54.159 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 249, 2020-08-07T15:12:54.159 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:12:54.159 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:12:54.159 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:12:54.160 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.160 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.160 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.160 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6813", 2020-08-07T15:12:54.160 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:12:54.161 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.161 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.161 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.161 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:12:54.161 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.162 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.162 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.162 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6814", 2020-08-07T15:12:54.163 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:12:54.163 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.163 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.163 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:12:54.164 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.164 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.164 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6816", 2020-08-07T15:12:54.164 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:12:54.165 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.165 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.165 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.165 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:12:54.166 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.166 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.166 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.166 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6815", 2020-08-07T15:12:54.166 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:12:54.167 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.167 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.167 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.167 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6813/31579", 2020-08-07T15:12:54.167 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6814/31579", 2020-08-07T15:12:54.168 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6816/31579", 2020-08-07T15:12:54.168 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6815/31579", 2020-08-07T15:12:54.168 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:12:54.168 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:12:54.168 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:12:54.168 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.169 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.169 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.169 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 4, 2020-08-07T15:12:54.169 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "01aadc13-29c5-4ddd-9bb9-de24b2b20412", 2020-08-07T15:12:54.169 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:12:54.170 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:12:54.170 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:12:54.170 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0, 2020-08-07T15:12:54.170 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:12:54.170 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:12:54.171 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:12:54.171 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 248, 2020-08-07T15:12:54.171 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:12:54.171 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:12:54.171 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:12:54.171 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.172 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.172 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.172 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6800", 2020-08-07T15:12:54.172 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:12:54.172 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.173 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.173 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.173 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:12:54.173 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.174 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.174 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.174 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6801", 2020-08-07T15:12:54.174 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:12:54.174 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.175 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.175 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.175 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:12:54.175 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.175 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.176 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.176 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6803", 2020-08-07T15:12:54.176 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:12:54.176 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.176 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.177 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.177 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:12:54.177 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.177 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.177 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.178 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6802", 2020-08-07T15:12:54.178 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:12:54.178 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.178 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.178 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.178 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6800/31545", 2020-08-07T15:12:54.179 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6801/31545", 2020-08-07T15:12:54.179 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6803/31545", 2020-08-07T15:12:54.179 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6802/31545", 2020-08-07T15:12:54.179 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:12:54.179 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:12:54.180 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:12:54.180 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.180 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.180 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 5, 2020-08-07T15:12:54.181 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "f5556a82-73b4-47b1-8e29-1d2fe031d8fd", 2020-08-07T15:12:54.181 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:12:54.181 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:12:54.181 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:12:54.181 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:12:54.181 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:12:54.182 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:12:54.182 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:12:54.182 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 263, 2020-08-07T15:12:54.182 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:12:54.182 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:12:54.186 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:12:54.186 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.186 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.187 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.187 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6804", 2020-08-07T15:12:54.187 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:12:54.187 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.187 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.188 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.188 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:12:54.188 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.188 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.188 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.189 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6805", 2020-08-07T15:12:54.189 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:12:54.189 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.189 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.189 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.192 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:12:54.192 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.192 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.193 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.193 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6807", 2020-08-07T15:12:54.196 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:12:54.196 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.196 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.197 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.197 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:12:54.197 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.197 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.198 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.198 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6806", 2020-08-07T15:12:54.198 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:12:54.198 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.198 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.198 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.199 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6804/31547", 2020-08-07T15:12:54.199 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6805/31547", 2020-08-07T15:12:54.199 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6807/31547", 2020-08-07T15:12:54.199 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6806/31547", 2020-08-07T15:12:54.199 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:12:54.200 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:12:54.200 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:12:54.200 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.212 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.212 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.212 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 6, 2020-08-07T15:12:54.212 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "9b98b337-e1cc-43cd-9e9b-f21ae3f20899", 2020-08-07T15:12:54.213 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:12:54.213 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:12:54.213 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:12:54.213 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:12:54.213 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:12:54.214 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:12:54.214 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 14, 2020-08-07T15:12:54.214 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 249, 2020-08-07T15:12:54.214 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:12:54.214 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:12:54.217 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:12:54.217 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.217 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.218 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.218 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6808", 2020-08-07T15:12:54.218 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:12:54.218 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.218 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.219 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.219 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:12:54.219 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.219 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.219 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.220 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6809", 2020-08-07T15:12:54.220 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:12:54.220 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.220 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.220 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.221 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:12:54.221 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.221 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.221 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.221 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6811", 2020-08-07T15:12:54.222 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:12:54.223 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.223 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.223 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.223 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:12:54.223 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.223 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.224 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.224 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6810", 2020-08-07T15:12:54.224 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:12:54.224 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.224 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.225 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.225 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6808/31546", 2020-08-07T15:12:54.225 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6809/31546", 2020-08-07T15:12:54.225 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6811/31546", 2020-08-07T15:12:54.225 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6810/31546", 2020-08-07T15:12:54.233 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:12:54.233 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:12:54.234 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:12:54.234 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.234 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.234 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.234 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 7, 2020-08-07T15:12:54.235 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "777662d8-bd34-48e8-83e1-24ce9e61f14b", 2020-08-07T15:12:54.235 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:12:54.235 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:12:54.235 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:12:54.235 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:12:54.235 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:12:54.236 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:12:54.236 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:12:54.236 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 244, 2020-08-07T15:12:54.238 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:12:54.238 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:12:54.238 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:12:54.239 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.239 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.239 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.239 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6812", 2020-08-07T15:12:54.240 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:12:54.240 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.240 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.240 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.240 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:12:54.241 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.241 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.241 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.241 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6813", 2020-08-07T15:12:54.241 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:12:54.249 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.249 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.249 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.250 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:12:54.250 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.251 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.251 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.251 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6815", 2020-08-07T15:12:54.252 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:12:54.252 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.252 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.252 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.253 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:12:54.253 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:12:54.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.253 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:12:54.260 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6814", 2020-08-07T15:12:54.260 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:12:54.260 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.260 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.261 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.261 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6812/31570", 2020-08-07T15:12:54.261 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6813/31570", 2020-08-07T15:12:54.261 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6815/31570", 2020-08-07T15:12:54.261 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6814/31570", 2020-08-07T15:12:54.262 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:12:54.262 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:12:54.262 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:12:54.262 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.263 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.263 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:12:54.263 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_xinfo": [ 2020-08-07T15:12:54.275 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.275 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 0, 2020-08-07T15:12:54.276 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:12:54.276 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:12:54.276 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:12:54.276 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:12:54.277 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:12:54.277 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:12:19.121056+0000", 2020-08-07T15:12:54.277 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:12:54.277 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.277 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.278 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 1, 2020-08-07T15:12:54.278 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:12:54.278 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:12:54.278 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:12:54.279 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:12:54.279 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:12:54.279 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:12:05.850993+0000", 2020-08-07T15:12:54.279 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 191 2020-08-07T15:12:54.280 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.280 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.280 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 2, 2020-08-07T15:12:54.280 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:12:54.281 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:12:54.281 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:12:54.281 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:12:54.281 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:12:54.285 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:12:17.947567+0000", 2020-08-07T15:12:54.285 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:12:54.286 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.286 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.286 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 3, 2020-08-07T15:12:54.286 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:12:54.286 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:12:54.287 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:12:54.287 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:12:54.287 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:12:54.287 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:12:19.857070+0000", 2020-08-07T15:12:54.287 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:12:54.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.288 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.288 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 4, 2020-08-07T15:12:54.288 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:12:54.288 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:12:54.288 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:12:54.289 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:12:54.289 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:12:54.289 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:12:18.727770+0000", 2020-08-07T15:12:54.289 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:12:54.289 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.298 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.298 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 5, 2020-08-07T15:12:54.299 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:12:54.299 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:12:54.299 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:12:54.299 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:12:54.300 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:12:54.300 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:12:19.039747+0000", 2020-08-07T15:12:54.300 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:12:54.300 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.301 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.301 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 6, 2020-08-07T15:12:54.301 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:12:54.301 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:12:54.302 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:12:54.303 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:12:54.303 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:12:54.303 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:12:17.971384+0000", 2020-08-07T15:12:54.304 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:12:54.304 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.304 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.304 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 7, 2020-08-07T15:12:54.304 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:12:54.305 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:12:54.305 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:12:54.305 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:12:54.308 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:12:54.308 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:12:17.214169+0000", 2020-08-07T15:12:54.309 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:12:54.309 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.309 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:12:54.309 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_upmap": [ 2020-08-07T15:12:54.310 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:54.310 INFO:teuthology.orchestra.run.smithi074.stdout: "pgid": "1.16", 2020-08-07T15:12:54.310 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:12:54.310 INFO:teuthology.orchestra.run.smithi074.stdout: 0, 2020-08-07T15:12:54.310 INFO:teuthology.orchestra.run.smithi074.stdout: 5, 2020-08-07T15:12:54.311 INFO:teuthology.orchestra.run.smithi074.stdout: 4 2020-08-07T15:12:54.311 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:54.311 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.311 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:12:54.312 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_upmap_items": [], 2020-08-07T15:12:54.312 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_temp": [], 2020-08-07T15:12:54.312 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_temp": [], 2020-08-07T15:12:54.313 INFO:teuthology.orchestra.run.smithi074.stdout: "blacklist": {}, 2020-08-07T15:12:54.313 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profiles": { 2020-08-07T15:12:54.313 INFO:teuthology.orchestra.run.smithi074.stdout: "default": { 2020-08-07T15:12:54.313 INFO:teuthology.orchestra.run.smithi074.stdout: "crush-failure-domain": "osd", 2020-08-07T15:12:54.314 INFO:teuthology.orchestra.run.smithi074.stdout: "k": "2", 2020-08-07T15:12:54.314 INFO:teuthology.orchestra.run.smithi074.stdout: "m": "1", 2020-08-07T15:12:54.314 INFO:teuthology.orchestra.run.smithi074.stdout: "plugin": "jerasure", 2020-08-07T15:12:54.314 INFO:teuthology.orchestra.run.smithi074.stdout: "ruleset-failure-domain": "osd", 2020-08-07T15:12:54.314 INFO:teuthology.orchestra.run.smithi074.stdout: "technique": "reed_sol_van" 2020-08-07T15:12:54.315 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:54.315 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:54.315 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps_queue": [], 2020-08-07T15:12:54.315 INFO:teuthology.orchestra.run.smithi074.stdout: "new_removed_snaps": [], 2020-08-07T15:12:54.315 INFO:teuthology.orchestra.run.smithi074.stdout: "new_purged_snaps": [], 2020-08-07T15:12:54.316 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_node_flags": {}, 2020-08-07T15:12:54.317 INFO:teuthology.orchestra.run.smithi074.stdout: "device_class_flags": {} 2020-08-07T15:12:54.318 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:54.320 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:54.320 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:54.321 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:12:54.322 INFO:tasks.thrashosds.thrasher:j is {'epoch': 264, 'fsid': 'a01d3e82-8d24-4288-9e42-576ab3ff4957', 'created': '2020-08-07T15:06:07.342912+0000', 'modified': '2020-08-07T15:12:52.516917+0000', 'last_up_change': '2020-08-07T15:12:05.338271+0000', 'last_in_change': '2020-08-07T15:06:17.674292+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 6, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-08-07T15:06:10.648876+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 41, 'pg_placement_num': 41, 'pg_placement_num_target': 41, 'pg_num_target': 41, 'pg_num_pending': 41, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '246', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '244', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-08-07T15:06:21.474648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 6, 'pool_name': 'unique_pool_3', 'create_time': '2020-08-07T15:12:08.847598+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '232', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': 'fc1ebb23-3ef4-4fc0-b395-382e385885a3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 262, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6801', 'nonce': 31577}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6802', 'nonce': 31577}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6804', 'nonce': 31577}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6803', 'nonce': 31577}]}, 'public_addr': '172.21.15.74:6801/31577', 'cluster_addr': '172.21.15.74:6802/31577', 'heartbeat_back_addr': '172.21.15.74:6804/31577', 'heartbeat_front_addr': '172.21.15.74:6803/31577', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '1fbe530a-63ce-4dc8-82ab-18793d6ebc0b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.7196044921875, 'last_clean_begin': 13, 'last_clean_end': 190, 'up_from': 227, 'up_thru': 249, 'down_at': 191, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6809', 'nonce': 62587}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6810', 'nonce': 62587}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6812', 'nonce': 62587}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6811', 'nonce': 62587}]}, 'public_addr': '172.21.15.74:6809/62587', 'cluster_addr': '172.21.15.74:6810/62587', 'heartbeat_back_addr': '172.21.15.74:6812/62587', 'heartbeat_front_addr': '172.21.15.74:6811/62587', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd6fe4338-3e11-4f73-9cbb-e1b046ce286a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 256, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6805', 'nonce': 31576}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6806', 'nonce': 31576}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6808', 'nonce': 31576}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6807', 'nonce': 31576}]}, 'public_addr': '172.21.15.74:6805/31576', 'cluster_addr': '172.21.15.74:6806/31576', 'heartbeat_back_addr': '172.21.15.74:6808/31576', 'heartbeat_front_addr': '172.21.15.74:6807/31576', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 249, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6813', 'nonce': 31579}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6814', 'nonce': 31579}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6816', 'nonce': 31579}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6815', 'nonce': 31579}]}, 'public_addr': '172.21.15.74:6813/31579', 'cluster_addr': '172.21.15.74:6814/31579', 'heartbeat_back_addr': '172.21.15.74:6816/31579', 'heartbeat_front_addr': '172.21.15.74:6815/31579', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '01aadc13-29c5-4ddd-9bb9-de24b2b20412', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 248, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6800', 'nonce': 31545}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6801', 'nonce': 31545}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6803', 'nonce': 31545}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6802', 'nonce': 31545}]}, 'public_addr': '172.21.15.96:6800/31545', 'cluster_addr': '172.21.15.96:6801/31545', 'heartbeat_back_addr': '172.21.15.96:6803/31545', 'heartbeat_front_addr': '172.21.15.96:6802/31545', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'f5556a82-73b4-47b1-8e29-1d2fe031d8fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 263, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6804', 'nonce': 31547}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6805', 'nonce': 31547}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6807', 'nonce': 31547}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6806', 'nonce': 31547}]}, 'public_addr': '172.21.15.96:6804/31547', 'cluster_addr': '172.21.15.96:6805/31547', 'heartbeat_back_addr': '172.21.15.96:6807/31547', 'heartbeat_front_addr': '172.21.15.96:6806/31547', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9b98b337-e1cc-43cd-9e9b-f21ae3f20899', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 249, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6808', 'nonce': 31546}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6809', 'nonce': 31546}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6811', 'nonce': 31546}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6810', 'nonce': 31546}]}, 'public_addr': '172.21.15.96:6808/31546', 'cluster_addr': '172.21.15.96:6809/31546', 'heartbeat_back_addr': '172.21.15.96:6811/31546', 'heartbeat_front_addr': '172.21.15.96:6810/31546', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '777662d8-bd34-48e8-83e1-24ce9e61f14b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 244, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6812', 'nonce': 31570}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6813', 'nonce': 31570}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6815', 'nonce': 31570}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6814', 'nonce': 31570}]}, 'public_addr': '172.21.15.96:6812/31570', 'cluster_addr': '172.21.15.96:6813/31570', 'heartbeat_back_addr': '172.21.15.96:6815/31570', 'heartbeat_front_addr': '172.21.15.96:6814/31570', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:12:19.121056+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:12:05.850993+0000', 'dead_epoch': 191}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:12:17.947567+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:12:19.857070+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:12:18.727770+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:12:19.039747+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:12:17.971384+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:12:17.214169+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '1.16', 'osds': [0, 5, 4]}], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blacklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}} 2020-08-07T15:12:54.323 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:12:54.323 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 42 16 64982 64966 12.0831 12.1406 0.00516213 0.0103385 2020-08-07T15:12:54.324 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:54.318+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:54.420 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:54.419+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:54.530 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:54.524+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:54.626 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:54.625+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:54.728 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:54.726+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:54.740 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:54.741 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:54.753 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:12:54.832 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:12:54.835 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:12:54.856 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:54.845+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:54.859 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":464,"stamp":"2020-08-07T15:12:54.688445+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":494895104,"num_objects":7554,"num_object_clones":0,"num_object_copies":22662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60412,"num_write_kb":483296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":195,"acting":195,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":196,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9849164,"kb_used_data":1460492,"kb_used_omap":2,"kb_used_meta":8388605,"kb_avail":745125556,"statfs":{"total":773094113280,"available":763008569344,"internally_reserved":8589934592,"allocated":1495543808,"data_stored":1485147669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":8589932013},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":61661184,"num_objects":941,"num_object_clones":0,"num_object_copies":2823,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7527,"num_write_kb":60216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.195468"},"pg_stats":[{"pgid":"1.28","version":"16'2","reported_seq":"27","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.494049+0000","last_change":"2020-08-07T15:12:37.495634+0000","last_active":"2020-08-07T15:12:50.494049+0000","last_peered":"2020-08-07T15:12:50.494049+0000","last_clean":"2020-08-07T15:12:50.494049+0000","last_became_active":"2020-08-07T15:12:37.495085+0000","last_became_peered":"2020-08-07T15:12:37.495085+0000","last_unstale":"2020-08-07T15:12:50.494049+0000","last_undegraded":"2020-08-07T15:12:50.494049+0000","last_fullsized":"2020-08-07T15:12:50.494049+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.490655+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.490655+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.27","version":"16'2","reported_seq":"373","reported_epoch":"263","state":"active+clean","last_fresh":"2020-08-07T15:12:51.530045+0000","last_change":"2020-08-07T15:12:37.412480+0000","last_active":"2020-08-07T15:12:51.530045+0000","last_peered":"2020-08-07T15:12:51.530045+0000","last_clean":"2020-08-07T15:12:51.530045+0000","last_became_active":"2020-08-07T15:12:37.411869+0000","last_became_peered":"2020-08-07T15:12:37.411869+0000","last_unstale":"2020-08-07T15:12:51.530045+0000","last_undegraded":"2020-08-07T15:12:51.530045+0000","last_fullsized":"2020-08-07T15:12:51.530045+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:15.163639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:15.163639+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.26","version":"16'2","reported_seq":"35","reported_epoch":"260","state":"active+clean","last_fresh":"2020-08-07T15:12:48.483955+0000","last_change":"2020-08-07T15:12:40.765120+0000","last_active":"2020-08-07T15:12:48.483955+0000","last_peered":"2020-08-07T15:12:48.483955+0000","last_clean":"2020-08-07T15:12:48.483955+0000","last_became_active":"2020-08-07T15:12:37.457426+0000","last_became_peered":"2020-08-07T15:12:37.457426+0000","last_unstale":"2020-08-07T15:12:48.483955+0000","last_undegraded":"2020-08-07T15:12:48.483955+0000","last_fullsized":"2020-08-07T15:12:48.483955+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:40.765027+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_clean_scrub_stamp":"2020-08-07T15:12:40.765027+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.25","version":"75'3","reported_seq":"260","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.513226+0000","last_change":"2020-08-07T15:12:37.428239+0000","last_active":"2020-08-07T15:12:50.513226+0000","last_peered":"2020-08-07T15:12:50.513226+0000","last_clean":"2020-08-07T15:12:50.513226+0000","last_became_active":"2020-08-07T15:12:37.427816+0000","last_became_peered":"2020-08-07T15:12:37.427816+0000","last_unstale":"2020-08-07T15:12:50.513226+0000","last_undegraded":"2020-08-07T15:12:50.513226+0000","last_fullsized":"2020-08-07T15:12:50.513226+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:12.545945+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.545945+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.24","version":"16'2","reported_seq":"201","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.505477+0000","last_change":"2020-08-07T15:12:42.295755+0000","last_active":"2020-08-07T15:12:50.505477+0000","last_peered":"2020-08-07T15:12:50.505477+0000","last_clean":"2020-08-07T15:12:50.505477+0000","last_became_active":"2020-08-07T15:12:34.414144+0000","last_became_peered":"2020-08-07T15:12:34.414144+0000","last_unstale":"2020-08-07T15:12:50.505477+0000","last_undegraded":"2020-08-07T15:12:50.505477+0000","last_fullsized":"2020-08-07T15:12:50.505477+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:42.295672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:42.295672+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.23","version":"16'2","reported_seq":"308","reported_epoch":"263","state":"active+clean","last_fresh":"2020-08-07T15:12:51.529721+0000","last_change":"2020-08-07T15:12:37.410705+0000","last_active":"2020-08-07T15:12:51.529721+0000","last_peered":"2020-08-07T15:12:51.529721+0000","last_clean":"2020-08-07T15:12:51.529721+0000","last_became_active":"2020-08-07T15:12:37.410157+0000","last_became_peered":"2020-08-07T15:12:37.410157+0000","last_unstale":"2020-08-07T15:12:51.529721+0000","last_undegraded":"2020-08-07T15:12:51.529721+0000","last_fullsized":"2020-08-07T15:12:51.529721+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:10.175739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:10.175739+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1],"acting":[6,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.22","version":"16'2","reported_seq":"28","reported_epoch":"263","state":"active+clean","last_fresh":"2020-08-07T15:12:51.529853+0000","last_change":"2020-08-07T15:12:37.413542+0000","last_active":"2020-08-07T15:12:51.529853+0000","last_peered":"2020-08-07T15:12:51.529853+0000","last_clean":"2020-08-07T15:12:51.529853+0000","last_became_active":"2020-08-07T15:12:37.413046+0000","last_became_peered":"2020-08-07T15:12:37.413046+0000","last_unstale":"2020-08-07T15:12:51.529853+0000","last_undegraded":"2020-08-07T15:12:51.529853+0000","last_fullsized":"2020-08-07T15:12:51.529853+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:11.658923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:11.658923+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.21","version":"16'2","reported_seq":"34","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.504982+0000","last_change":"2020-08-07T15:12:37.419228+0000","last_active":"2020-08-07T15:12:50.504982+0000","last_peered":"2020-08-07T15:12:50.504982+0000","last_clean":"2020-08-07T15:12:50.504982+0000","last_became_active":"2020-08-07T15:12:37.418612+0000","last_became_peered":"2020-08-07T15:12:37.418612+0000","last_unstale":"2020-08-07T15:12:50.504982+0000","last_undegraded":"2020-08-07T15:12:50.504982+0000","last_fullsized":"2020-08-07T15:12:50.504982+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:09.261101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.261101+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.20","version":"16'2","reported_seq":"34","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.513089+0000","last_change":"2020-08-07T15:12:37.409857+0000","last_active":"2020-08-07T15:12:50.513089+0000","last_peered":"2020-08-07T15:12:50.513089+0000","last_clean":"2020-08-07T15:12:50.513089+0000","last_became_active":"2020-08-07T15:12:37.409249+0000","last_became_peered":"2020-08-07T15:12:37.409249+0000","last_unstale":"2020-08-07T15:12:50.513089+0000","last_undegraded":"2020-08-07T15:12:50.513089+0000","last_fullsized":"2020-08-07T15:12:50.513089+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:40.696862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:40.696862+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1f","version":"16'2","reported_seq":"27","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.513694+0000","last_change":"2020-08-07T15:12:37.411233+0000","last_active":"2020-08-07T15:12:50.513694+0000","last_peered":"2020-08-07T15:12:50.513694+0000","last_clean":"2020-08-07T15:12:50.513694+0000","last_became_active":"2020-08-07T15:12:37.410716+0000","last_became_peered":"2020-08-07T15:12:37.410716+0000","last_unstale":"2020-08-07T15:12:50.513694+0000","last_undegraded":"2020-08-07T15:12:50.513694+0000","last_fullsized":"2020-08-07T15:12:50.513694+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.176383+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1e","version":"16'2","reported_seq":"114","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.505477+0000","last_change":"2020-08-07T15:11:37.457186+0000","last_active":"2020-08-07T15:12:50.505477+0000","last_peered":"2020-08-07T15:12:50.505477+0000","last_clean":"2020-08-07T15:12:50.505477+0000","last_became_active":"2020-08-07T15:10:51.559894+0000","last_became_peered":"2020-08-07T15:10:51.559894+0000","last_unstale":"2020-08-07T15:12:50.505477+0000","last_undegraded":"2020-08-07T15:12:50.505477+0000","last_fullsized":"2020-08-07T15:12:50.505477+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.457111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.457111+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1d","version":"75'3","reported_seq":"125","reported_epoch":"260","state":"active+clean","last_fresh":"2020-08-07T15:12:48.483915+0000","last_change":"2020-08-07T15:12:28.855487+0000","last_active":"2020-08-07T15:12:48.483915+0000","last_peered":"2020-08-07T15:12:48.483915+0000","last_clean":"2020-08-07T15:12:48.483915+0000","last_became_active":"2020-08-07T15:12:07.356691+0000","last_became_peered":"2020-08-07T15:12:07.356691+0000","last_unstale":"2020-08-07T15:12:48.483915+0000","last_undegraded":"2020-08-07T15:12:48.483915+0000","last_fullsized":"2020-08-07T15:12:48.483915+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:28.855432+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:28.855432+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1c","version":"16'2","reported_seq":"202","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.499682+0000","last_change":"2020-08-07T15:12:07.262797+0000","last_active":"2020-08-07T15:12:50.499682+0000","last_peered":"2020-08-07T15:12:50.499682+0000","last_clean":"2020-08-07T15:12:50.499682+0000","last_became_active":"2020-08-07T15:12:07.245731+0000","last_became_peered":"2020-08-07T15:12:07.245731+0000","last_unstale":"2020-08-07T15:12:50.499682+0000","last_undegraded":"2020-08-07T15:12:50.499682+0000","last_fullsized":"2020-08-07T15:12:50.499682+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:39.738160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:39.738160+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1b","version":"16'2","reported_seq":"235","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.494195+0000","last_change":"2020-08-07T15:10:57.797029+0000","last_active":"2020-08-07T15:12:50.494195+0000","last_peered":"2020-08-07T15:12:50.494195+0000","last_clean":"2020-08-07T15:12:50.494195+0000","last_became_active":"2020-08-07T15:10:51.557392+0000","last_became_peered":"2020-08-07T15:10:51.557392+0000","last_unstale":"2020-08-07T15:12:50.494195+0000","last_undegraded":"2020-08-07T15:12:50.494195+0000","last_fullsized":"2020-08-07T15:12:50.494195+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:57.796946+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:57.796946+0000","last_clean_scrub_stamp":"2020-08-07T15:10:57.796946+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1a","version":"16'2","reported_seq":"102","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.499676+0000","last_change":"2020-08-07T15:12:40.834667+0000","last_active":"2020-08-07T15:12:50.499676+0000","last_peered":"2020-08-07T15:12:50.499676+0000","last_clean":"2020-08-07T15:12:50.499676+0000","last_became_active":"2020-08-07T15:12:07.243542+0000","last_became_peered":"2020-08-07T15:12:07.243542+0000","last_unstale":"2020-08-07T15:12:50.499676+0000","last_undegraded":"2020-08-07T15:12:50.499676+0000","last_fullsized":"2020-08-07T15:12:50.499676+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:40.834612+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:12:40.834612+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.19","version":"16'2","reported_seq":"160","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.505096+0000","last_change":"2020-08-07T15:12:15.372999+0000","last_active":"2020-08-07T15:12:50.505096+0000","last_peered":"2020-08-07T15:12:50.505096+0000","last_clean":"2020-08-07T15:12:50.505096+0000","last_became_active":"2020-08-07T15:10:51.559752+0000","last_became_peered":"2020-08-07T15:10:51.559752+0000","last_unstale":"2020-08-07T15:12:50.505096+0000","last_undegraded":"2020-08-07T15:12:50.505096+0000","last_fullsized":"2020-08-07T15:12:50.505096+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:15.372910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:15.372910+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.18","version":"16'2","reported_seq":"260","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.494181+0000","last_change":"2020-08-07T15:11:06.955336+0000","last_active":"2020-08-07T15:12:50.494181+0000","last_peered":"2020-08-07T15:12:50.494181+0000","last_clean":"2020-08-07T15:12:50.494181+0000","last_became_active":"2020-08-07T15:10:51.557665+0000","last_became_peered":"2020-08-07T15:10:51.557665+0000","last_unstale":"2020-08-07T15:12:50.494181+0000","last_undegraded":"2020-08-07T15:12:50.494181+0000","last_fullsized":"2020-08-07T15:12:50.494181+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:06.955262+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:06.955262+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.17","version":"16'2","reported_seq":"88","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.494593+0000","last_change":"2020-08-07T15:11:08.916219+0000","last_active":"2020-08-07T15:12:50.494593+0000","last_peered":"2020-08-07T15:12:50.494593+0000","last_clean":"2020-08-07T15:12:50.494593+0000","last_became_active":"2020-08-07T15:10:51.561769+0000","last_became_peered":"2020-08-07T15:10:51.561769+0000","last_unstale":"2020-08-07T15:12:50.494593+0000","last_undegraded":"2020-08-07T15:12:50.494593+0000","last_fullsized":"2020-08-07T15:12:50.494593+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:08.916169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:11:08.916169+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.16","version":"16'2","reported_seq":"28","reported_epoch":"263","state":"peering","last_fresh":"2020-08-07T15:12:51.524218+0000","last_change":"2020-08-07T15:12:51.524218+0000","last_active":"2020-08-07T15:12:49.500046+0000","last_peered":"2020-08-07T15:10:49.542071+0000","last_clean":"2020-08-07T15:06:20.642821+0000","last_became_active":"2020-08-07T15:10:49.541681+0000","last_became_peered":"2020-08-07T15:10:49.541681+0000","last_unstale":"2020-08-07T15:12:51.524218+0000","last_undegraded":"2020-08-07T15:12:51.524218+0000","last_fullsized":"2020-08-07T15:12:51.524218+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:12.883558+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:12.883558+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.883558+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.15","version":"75'3","reported_seq":"99","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.500088+0000","last_change":"2020-08-07T15:12:07.284341+0000","last_active":"2020-08-07T15:12:50.500088+0000","last_peered":"2020-08-07T15:12:50.500088+0000","last_clean":"2020-08-07T15:12:50.500088+0000","last_became_active":"2020-08-07T15:12:07.276899+0000","last_became_peered":"2020-08-07T15:12:07.276899+0000","last_unstale":"2020-08-07T15:12:50.500088+0000","last_undegraded":"2020-08-07T15:12:50.500088+0000","last_fullsized":"2020-08-07T15:12:50.500088+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:11:20.562710+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:11:20.562710+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.14","version":"16'2","reported_seq":"198","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.513202+0000","last_change":"2020-08-07T15:12:09.224231+0000","last_active":"2020-08-07T15:12:50.513202+0000","last_peered":"2020-08-07T15:12:50.513202+0000","last_clean":"2020-08-07T15:12:50.513202+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:12:50.513202+0000","last_undegraded":"2020-08-07T15:12:50.513202+0000","last_fullsized":"2020-08-07T15:12:50.513202+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:09.224158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.224158+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"211","reported_epoch":"260","state":"active+clean","last_fresh":"2020-08-07T15:12:48.483471+0000","last_change":"2020-08-07T15:12:19.577926+0000","last_active":"2020-08-07T15:12:48.483471+0000","last_peered":"2020-08-07T15:12:48.483471+0000","last_clean":"2020-08-07T15:12:48.483471+0000","last_became_active":"2020-08-07T15:12:06.375861+0000","last_became_peered":"2020-08-07T15:12:06.375861+0000","last_unstale":"2020-08-07T15:12:48.483471+0000","last_undegraded":"2020-08-07T15:12:48.483471+0000","last_fullsized":"2020-08-07T15:12:48.483471+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:07.887356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.887356+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"213","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.494165+0000","last_change":"2020-08-07T15:12:26.104598+0000","last_active":"2020-08-07T15:12:50.494165+0000","last_peered":"2020-08-07T15:12:50.494165+0000","last_clean":"2020-08-07T15:12:50.494165+0000","last_became_active":"2020-08-07T15:12:06.367362+0000","last_became_peered":"2020-08-07T15:12:06.367362+0000","last_unstale":"2020-08-07T15:12:50.494165+0000","last_undegraded":"2020-08-07T15:12:50.494165+0000","last_fullsized":"2020-08-07T15:12:50.494165+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:26.104526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:26.104526+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"311","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.494258+0000","last_change":"2020-08-07T15:12:24.105593+0000","last_active":"2020-08-07T15:12:50.494258+0000","last_peered":"2020-08-07T15:12:50.494258+0000","last_clean":"2020-08-07T15:12:50.494258+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:12:50.494258+0000","last_undegraded":"2020-08-07T15:12:50.494258+0000","last_fullsized":"2020-08-07T15:12:50.494258+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:24.105531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:24.105531+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"305","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.494301+0000","last_change":"2020-08-07T15:12:13.231819+0000","last_active":"2020-08-07T15:12:50.494301+0000","last_peered":"2020-08-07T15:12:50.494301+0000","last_clean":"2020-08-07T15:12:50.494301+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:12:50.494301+0000","last_undegraded":"2020-08-07T15:12:50.494301+0000","last_fullsized":"2020-08-07T15:12:50.494301+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:13.231762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.231762+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3","version":"263'3524","reported_seq":"3564","reported_epoch":"263","state":"active+clean","last_fresh":"2020-08-07T15:12:51.726796+0000","last_change":"2020-08-07T15:12:10.464875+0000","last_active":"2020-08-07T15:12:51.726796+0000","last_peered":"2020-08-07T15:12:51.726796+0000","last_clean":"2020-08-07T15:12:51.726796+0000","last_became_active":"2020-08-07T15:12:10.464346+0000","last_became_peered":"2020-08-07T15:12:10.464346+0000","last_unstale":"2020-08-07T15:12:51.726796+0000","last_undegraded":"2020-08-07T15:12:51.726796+0000","last_fullsized":"2020-08-07T15:12:51.726796+0000","mapping_epoch":230,"log_start":"263'3519","ondisk_log_start":"263'3519","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":28868608,"num_objects":441,"num_object_clones":0,"num_object_copies":1323,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3524,"num_write_kb":28192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"266","reported_epoch":"263","state":"active+clean","last_fresh":"2020-08-07T15:12:51.530143+0000","last_change":"2020-08-07T15:12:27.410339+0000","last_active":"2020-08-07T15:12:51.530143+0000","last_peered":"2020-08-07T15:12:51.530143+0000","last_clean":"2020-08-07T15:12:51.530143+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:12:51.530143+0000","last_undegraded":"2020-08-07T15:12:51.530143+0000","last_fullsized":"2020-08-07T15:12:51.530143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:27.410262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:27.410262+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"193","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.506561+0000","last_change":"2020-08-07T15:12:39.258928+0000","last_active":"2020-08-07T15:12:50.506561+0000","last_peered":"2020-08-07T15:12:50.506561+0000","last_clean":"2020-08-07T15:12:50.506561+0000","last_became_active":"2020-08-07T15:12:32.259956+0000","last_became_peered":"2020-08-07T15:12:32.259956+0000","last_unstale":"2020-08-07T15:12:50.506561+0000","last_undegraded":"2020-08-07T15:12:50.506561+0000","last_fullsized":"2020-08-07T15:12:50.506561+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:39.258857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:39.258857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"263'4008","reported_seq":"4049","reported_epoch":"263","state":"active+clean","last_fresh":"2020-08-07T15:12:51.670412+0000","last_change":"2020-08-07T15:12:10.465048+0000","last_active":"2020-08-07T15:12:51.670412+0000","last_peered":"2020-08-07T15:12:51.670412+0000","last_clean":"2020-08-07T15:12:51.670412+0000","last_became_active":"2020-08-07T15:12:10.464493+0000","last_became_peered":"2020-08-07T15:12:10.464493+0000","last_unstale":"2020-08-07T15:12:51.670412+0000","last_undegraded":"2020-08-07T15:12:51.670412+0000","last_fullsized":"2020-08-07T15:12:51.670412+0000","mapping_epoch":230,"log_start":"263'4000","ondisk_log_start":"263'4000","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":32833536,"num_objects":501,"num_object_clones":0,"num_object_copies":1503,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4008,"num_write_kb":32064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"219","reported_epoch":"260","state":"active+clean","last_fresh":"2020-08-07T15:12:48.485381+0000","last_change":"2020-08-07T15:12:19.561477+0000","last_active":"2020-08-07T15:12:48.485381+0000","last_peered":"2020-08-07T15:12:48.485381+0000","last_clean":"2020-08-07T15:12:48.485381+0000","last_became_active":"2020-08-07T15:12:06.367376+0000","last_became_peered":"2020-08-07T15:12:06.367376+0000","last_unstale":"2020-08-07T15:12:48.485381+0000","last_undegraded":"2020-08-07T15:12:48.485381+0000","last_fullsized":"2020-08-07T15:12:48.485381+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:12.319383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.319383+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"247","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.513565+0000","last_change":"2020-08-07T15:12:44.858389+0000","last_active":"2020-08-07T15:12:50.513565+0000","last_peered":"2020-08-07T15:12:50.513565+0000","last_clean":"2020-08-07T15:12:50.513565+0000","last_became_active":"2020-08-07T15:12:32.241146+0000","last_became_peered":"2020-08-07T15:12:32.241146+0000","last_unstale":"2020-08-07T15:12:50.513565+0000","last_undegraded":"2020-08-07T15:12:50.513565+0000","last_fullsized":"2020-08-07T15:12:50.513565+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:44.858310+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:44.858310+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"261'3592","reported_seq":"3631","reported_epoch":"261","state":"active+clean","last_fresh":"2020-08-07T15:12:49.822489+0000","last_change":"2020-08-07T15:12:10.464412+0000","last_active":"2020-08-07T15:12:49.822489+0000","last_peered":"2020-08-07T15:12:49.822489+0000","last_clean":"2020-08-07T15:12:49.822489+0000","last_became_active":"2020-08-07T15:12:10.463950+0000","last_became_peered":"2020-08-07T15:12:10.463950+0000","last_unstale":"2020-08-07T15:12:49.822489+0000","last_undegraded":"2020-08-07T15:12:49.822489+0000","last_fullsized":"2020-08-07T15:12:49.822489+0000","mapping_epoch":230,"log_start":"261'3585","ondisk_log_start":"261'3585","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":29425664,"num_objects":449,"num_object_clones":0,"num_object_copies":1347,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3592,"num_write_kb":28736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"263","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.500194+0000","last_change":"2020-08-07T15:11:55.731391+0000","last_active":"2020-08-07T15:12:50.500194+0000","last_peered":"2020-08-07T15:12:50.500194+0000","last_clean":"2020-08-07T15:12:50.500194+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:12:50.500194+0000","last_undegraded":"2020-08-07T15:12:50.500194+0000","last_fullsized":"2020-08-07T15:12:50.500194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:11:55.731304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:11:55.731304+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"323","reported_epoch":"263","state":"active+clean","last_fresh":"2020-08-07T15:12:51.529956+0000","last_change":"2020-08-07T15:12:32.240238+0000","last_active":"2020-08-07T15:12:51.529956+0000","last_peered":"2020-08-07T15:12:51.529956+0000","last_clean":"2020-08-07T15:12:51.529956+0000","last_became_active":"2020-08-07T15:12:32.239755+0000","last_became_peered":"2020-08-07T15:12:32.239755+0000","last_unstale":"2020-08-07T15:12:51.529956+0000","last_undegraded":"2020-08-07T15:12:51.529956+0000","last_fullsized":"2020-08-07T15:12:51.529956+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_clean_scrub_stamp":"2020-08-07T15:12:14.121397+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"261'3784","reported_seq":"3822","reported_epoch":"261","state":"active+clean","last_fresh":"2020-08-07T15:12:49.832134+0000","last_change":"2020-08-07T15:12:10.398415+0000","last_active":"2020-08-07T15:12:49.832134+0000","last_peered":"2020-08-07T15:12:49.832134+0000","last_clean":"2020-08-07T15:12:49.832134+0000","last_became_active":"2020-08-07T15:12:10.397834+0000","last_became_peered":"2020-08-07T15:12:10.397834+0000","last_unstale":"2020-08-07T15:12:49.832134+0000","last_undegraded":"2020-08-07T15:12:49.832134+0000","last_fullsized":"2020-08-07T15:12:49.832134+0000","mapping_epoch":230,"log_start":"261'3777","ondisk_log_start":"261'3777","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":30998528,"num_objects":473,"num_object_clones":0,"num_object_copies":1419,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3784,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"278","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.499607+0000","last_change":"2020-08-07T15:12:08.622779+0000","last_active":"2020-08-07T15:12:50.499607+0000","last_peered":"2020-08-07T15:12:50.499607+0000","last_clean":"2020-08-07T15:12:50.499607+0000","last_became_active":"2020-08-07T15:12:06.367457+0000","last_became_peered":"2020-08-07T15:12:06.367457+0000","last_unstale":"2020-08-07T15:12:50.499607+0000","last_undegraded":"2020-08-07T15:12:50.499607+0000","last_fullsized":"2020-08-07T15:12:50.499607+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:08.622683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:08.622683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"295","reported_epoch":"263","state":"active+clean","last_fresh":"2020-08-07T15:12:51.529844+0000","last_change":"2020-08-07T15:12:42.371309+0000","last_active":"2020-08-07T15:12:51.529844+0000","last_peered":"2020-08-07T15:12:51.529844+0000","last_clean":"2020-08-07T15:12:51.529844+0000","last_became_active":"2020-08-07T15:12:32.237984+0000","last_became_peered":"2020-08-07T15:12:32.237984+0000","last_unstale":"2020-08-07T15:12:51.529844+0000","last_undegraded":"2020-08-07T15:12:51.529844+0000","last_fullsized":"2020-08-07T15:12:51.529844+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:42.371229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:42.371229+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.7","version":"263'3968","reported_seq":"4009","reported_epoch":"263","state":"active+clean","last_fresh":"2020-08-07T15:12:51.727991+0000","last_change":"2020-08-07T15:12:10.464810+0000","last_active":"2020-08-07T15:12:51.727991+0000","last_peered":"2020-08-07T15:12:51.727991+0000","last_clean":"2020-08-07T15:12:51.727991+0000","last_became_active":"2020-08-07T15:12:10.464050+0000","last_became_peered":"2020-08-07T15:12:10.464050+0000","last_unstale":"2020-08-07T15:12:51.727991+0000","last_undegraded":"2020-08-07T15:12:51.727991+0000","last_fullsized":"2020-08-07T15:12:51.727991+0000","mapping_epoch":230,"log_start":"263'3961","ondisk_log_start":"263'3961","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":32505856,"num_objects":496,"num_object_clones":0,"num_object_copies":1488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3968,"num_write_kb":31744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"307","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.499541+0000","last_change":"2020-08-07T15:12:39.812944+0000","last_active":"2020-08-07T15:12:50.499541+0000","last_peered":"2020-08-07T15:12:50.499541+0000","last_clean":"2020-08-07T15:12:50.499541+0000","last_became_active":"2020-08-07T15:12:32.255633+0000","last_became_peered":"2020-08-07T15:12:32.255633+0000","last_unstale":"2020-08-07T15:12:50.499541+0000","last_undegraded":"2020-08-07T15:12:50.499541+0000","last_fullsized":"2020-08-07T15:12:50.499541+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:39.812876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:39.812876+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"263","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.494376+0000","last_change":"2020-08-07T15:11:40.328534+0000","last_active":"2020-08-07T15:12:50.494376+0000","last_peered":"2020-08-07T15:12:50.494376+0000","last_clean":"2020-08-07T15:12:50.494376+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:12:50.494376+0000","last_undegraded":"2020-08-07T15:12:50.494376+0000","last_fullsized":"2020-08-07T15:12:50.494376+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:11:40.328482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:11:40.328482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"263'3824","reported_seq":"3865","reported_epoch":"263","state":"active+clean","last_fresh":"2020-08-07T15:12:51.698512+0000","last_change":"2020-08-07T15:12:10.464478+0000","last_active":"2020-08-07T15:12:51.698512+0000","last_peered":"2020-08-07T15:12:51.698512+0000","last_clean":"2020-08-07T15:12:51.698512+0000","last_became_active":"2020-08-07T15:12:10.463997+0000","last_became_peered":"2020-08-07T15:12:10.463997+0000","last_unstale":"2020-08-07T15:12:51.698512+0000","last_undegraded":"2020-08-07T15:12:51.698512+0000","last_fullsized":"2020-08-07T15:12:51.698512+0000","mapping_epoch":230,"log_start":"263'3816","ondisk_log_start":"263'3816","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":31326208,"num_objects":478,"num_object_clones":0,"num_object_copies":1434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3824,"num_write_kb":30592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"284","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.513367+0000","last_change":"2020-08-07T15:12:10.585667+0000","last_active":"2020-08-07T15:12:50.513367+0000","last_peered":"2020-08-07T15:12:50.513367+0000","last_clean":"2020-08-07T15:12:50.513367+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:12:50.513367+0000","last_undegraded":"2020-08-07T15:12:50.513367+0000","last_fullsized":"2020-08-07T15:12:50.513367+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:10.585580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:10.585580+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"323","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.499798+0000","last_change":"2020-08-07T15:12:38.850878+0000","last_active":"2020-08-07T15:12:50.499798+0000","last_peered":"2020-08-07T15:12:50.499798+0000","last_clean":"2020-08-07T15:12:50.499798+0000","last_became_active":"2020-08-07T15:12:32.239794+0000","last_became_peered":"2020-08-07T15:12:32.239794+0000","last_unstale":"2020-08-07T15:12:50.499798+0000","last_undegraded":"2020-08-07T15:12:50.499798+0000","last_fullsized":"2020-08-07T15:12:50.499798+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:38.850809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:38.850809+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"263'3904","reported_seq":"3945","reported_epoch":"263","state":"active+clean","last_fresh":"2020-08-07T15:12:51.702058+0000","last_change":"2020-08-07T15:12:10.465113+0000","last_active":"2020-08-07T15:12:51.702058+0000","last_peered":"2020-08-07T15:12:51.702058+0000","last_clean":"2020-08-07T15:12:51.702058+0000","last_became_active":"2020-08-07T15:12:10.464582+0000","last_became_peered":"2020-08-07T15:12:10.464582+0000","last_unstale":"2020-08-07T15:12:51.702058+0000","last_undegraded":"2020-08-07T15:12:51.702058+0000","last_fullsized":"2020-08-07T15:12:51.702058+0000","mapping_epoch":230,"log_start":"263'3897","ondisk_log_start":"263'3897","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":31981568,"num_objects":488,"num_object_clones":0,"num_object_copies":1464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3904,"num_write_kb":31232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"278","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.494302+0000","last_change":"2020-08-07T15:12:07.340958+0000","last_active":"2020-08-07T15:12:50.494302+0000","last_peered":"2020-08-07T15:12:50.494302+0000","last_clean":"2020-08-07T15:12:50.494302+0000","last_became_active":"2020-08-07T15:12:06.366967+0000","last_became_peered":"2020-08-07T15:12:06.366967+0000","last_unstale":"2020-08-07T15:12:50.494302+0000","last_undegraded":"2020-08-07T15:12:50.494302+0000","last_fullsized":"2020-08-07T15:12:50.494302+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:07.340855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.340855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"270","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.494128+0000","last_change":"2020-08-07T15:12:32.247332+0000","last_active":"2020-08-07T15:12:50.494128+0000","last_peered":"2020-08-07T15:12:50.494128+0000","last_clean":"2020-08-07T15:12:50.494128+0000","last_became_active":"2020-08-07T15:12:32.246800+0000","last_became_peered":"2020-08-07T15:12:32.246800+0000","last_unstale":"2020-08-07T15:12:50.494128+0000","last_undegraded":"2020-08-07T15:12:50.494128+0000","last_fullsized":"2020-08-07T15:12:50.494128+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:09.261101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.261101+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"263'3800","reported_seq":"3841","reported_epoch":"263","state":"active+clean","last_fresh":"2020-08-07T15:12:51.717411+0000","last_change":"2020-08-07T15:12:10.466261+0000","last_active":"2020-08-07T15:12:51.717411+0000","last_peered":"2020-08-07T15:12:51.717411+0000","last_clean":"2020-08-07T15:12:51.717411+0000","last_became_active":"2020-08-07T15:12:10.465583+0000","last_became_peered":"2020-08-07T15:12:10.465583+0000","last_unstale":"2020-08-07T15:12:51.717411+0000","last_undegraded":"2020-08-07T15:12:51.717411+0000","last_fullsized":"2020-08-07T15:12:51.717411+0000","mapping_epoch":230,"log_start":"263'3792","ondisk_log_start":"263'3792","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":31129600,"num_objects":475,"num_object_clones":0,"num_object_copies":1425,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3800,"num_write_kb":30400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"275","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.499899+0000","last_change":"2020-08-07T15:12:07.583222+0000","last_active":"2020-08-07T15:12:50.499899+0000","last_peered":"2020-08-07T15:12:50.499899+0000","last_clean":"2020-08-07T15:12:50.499899+0000","last_became_active":"2020-08-07T15:12:06.366915+0000","last_became_peered":"2020-08-07T15:12:06.366915+0000","last_unstale":"2020-08-07T15:12:50.499899+0000","last_undegraded":"2020-08-07T15:12:50.499899+0000","last_fullsized":"2020-08-07T15:12:50.499899+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:07.583152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.583152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"360","reported_epoch":"263","state":"active+clean","last_fresh":"2020-08-07T15:12:51.530178+0000","last_change":"2020-08-07T15:12:43.351056+0000","last_active":"2020-08-07T15:12:51.530178+0000","last_peered":"2020-08-07T15:12:51.530178+0000","last_clean":"2020-08-07T15:12:51.530178+0000","last_became_active":"2020-08-07T15:12:32.254842+0000","last_became_peered":"2020-08-07T15:12:32.254842+0000","last_unstale":"2020-08-07T15:12:51.530178+0000","last_undegraded":"2020-08-07T15:12:51.530178+0000","last_fullsized":"2020-08-07T15:12:51.530178+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:43.350971+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:43.350971+0000","last_clean_scrub_stamp":"2020-08-07T15:12:43.350971+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"263'3480","reported_seq":"3521","reported_epoch":"263","state":"active+clean","last_fresh":"2020-08-07T15:12:51.738056+0000","last_change":"2020-08-07T15:12:10.467447+0000","last_active":"2020-08-07T15:12:51.738056+0000","last_peered":"2020-08-07T15:12:51.738056+0000","last_clean":"2020-08-07T15:12:51.738056+0000","last_became_active":"2020-08-07T15:12:10.467000+0000","last_became_peered":"2020-08-07T15:12:10.467000+0000","last_unstale":"2020-08-07T15:12:51.738056+0000","last_undegraded":"2020-08-07T15:12:51.738056+0000","last_fullsized":"2020-08-07T15:12:51.738056+0000","mapping_epoch":230,"log_start":"263'3475","ondisk_log_start":"263'3475","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":28508160,"num_objects":435,"num_object_clones":0,"num_object_copies":1305,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3480,"num_write_kb":27840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"247","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.494230+0000","last_change":"2020-08-07T15:10:51.644951+0000","last_active":"2020-08-07T15:12:50.494230+0000","last_peered":"2020-08-07T15:12:50.494230+0000","last_clean":"2020-08-07T15:12:50.494230+0000","last_became_active":"2020-08-07T15:10:46.498230+0000","last_became_peered":"2020-08-07T15:10:46.498230+0000","last_unstale":"2020-08-07T15:12:50.494230+0000","last_undegraded":"2020-08-07T15:12:50.494230+0000","last_fullsized":"2020-08-07T15:12:50.494230+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:51.644875+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:10:51.644875+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"263'3816","reported_seq":"3857","reported_epoch":"263","state":"active+clean","last_fresh":"2020-08-07T15:12:51.719030+0000","last_change":"2020-08-07T15:12:10.465075+0000","last_active":"2020-08-07T15:12:51.719030+0000","last_peered":"2020-08-07T15:12:51.719030+0000","last_clean":"2020-08-07T15:12:51.719030+0000","last_became_active":"2020-08-07T15:12:10.464640+0000","last_became_peered":"2020-08-07T15:12:10.464640+0000","last_unstale":"2020-08-07T15:12:51.719030+0000","last_undegraded":"2020-08-07T15:12:51.719030+0000","last_fullsized":"2020-08-07T15:12:51.719030+0000","mapping_epoch":230,"log_start":"263'3808","ondisk_log_start":"263'3808","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":31260672,"num_objects":477,"num_object_clones":0,"num_object_copies":1431,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3816,"num_write_kb":30528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"106","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.494046+0000","last_change":"2020-08-07T15:12:45.243933+0000","last_active":"2020-08-07T15:12:50.494046+0000","last_peered":"2020-08-07T15:12:50.494046+0000","last_clean":"2020-08-07T15:12:50.494046+0000","last_became_active":"2020-08-07T15:12:45.243349+0000","last_became_peered":"2020-08-07T15:12:45.243349+0000","last_unstale":"2020-08-07T15:12:50.494046+0000","last_undegraded":"2020-08-07T15:12:50.494046+0000","last_fullsized":"2020-08-07T15:12:50.494046+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.490655+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.490655+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.e","version":"261'3536","reported_seq":"3575","reported_epoch":"261","state":"active+clean","last_fresh":"2020-08-07T15:12:49.765943+0000","last_change":"2020-08-07T15:12:10.465359+0000","last_active":"2020-08-07T15:12:49.765943+0000","last_peered":"2020-08-07T15:12:49.765943+0000","last_clean":"2020-08-07T15:12:49.765943+0000","last_became_active":"2020-08-07T15:12:10.464900+0000","last_became_peered":"2020-08-07T15:12:10.464900+0000","last_unstale":"2020-08-07T15:12:49.765943+0000","last_undegraded":"2020-08-07T15:12:49.765943+0000","last_fullsized":"2020-08-07T15:12:49.765943+0000","mapping_epoch":230,"log_start":"261'3528","ondisk_log_start":"261'3528","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":28966912,"num_objects":442,"num_object_clones":0,"num_object_copies":1326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3536,"num_write_kb":28288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"314","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.499700+0000","last_change":"2020-08-07T15:12:23.886345+0000","last_active":"2020-08-07T15:12:50.499700+0000","last_peered":"2020-08-07T15:12:50.499700+0000","last_clean":"2020-08-07T15:12:50.499700+0000","last_became_active":"2020-08-07T15:10:46.499635+0000","last_became_peered":"2020-08-07T15:10:46.499635+0000","last_unstale":"2020-08-07T15:12:50.499700+0000","last_undegraded":"2020-08-07T15:12:50.499700+0000","last_fullsized":"2020-08-07T15:12:50.499700+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:23.886261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:23.886261+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"263'3976","reported_seq":"4017","reported_epoch":"263","state":"active+clean","last_fresh":"2020-08-07T15:12:51.742177+0000","last_change":"2020-08-07T15:12:10.468110+0000","last_active":"2020-08-07T15:12:51.742177+0000","last_peered":"2020-08-07T15:12:51.742177+0000","last_clean":"2020-08-07T15:12:51.742177+0000","last_became_active":"2020-08-07T15:12:10.467697+0000","last_became_peered":"2020-08-07T15:12:10.467697+0000","last_unstale":"2020-08-07T15:12:51.742177+0000","last_undegraded":"2020-08-07T15:12:51.742177+0000","last_fullsized":"2020-08-07T15:12:51.742177+0000","mapping_epoch":230,"log_start":"263'3968","ondisk_log_start":"263'3968","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":32571392,"num_objects":497,"num_object_clones":0,"num_object_copies":1491,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3976,"num_write_kb":31808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"281","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.500114+0000","last_change":"2020-08-07T15:12:13.680022+0000","last_active":"2020-08-07T15:12:50.500114+0000","last_peered":"2020-08-07T15:12:50.500114+0000","last_clean":"2020-08-07T15:12:50.500114+0000","last_became_active":"2020-08-07T15:12:06.380392+0000","last_became_peered":"2020-08-07T15:12:06.380392+0000","last_unstale":"2020-08-07T15:12:50.500114+0000","last_undegraded":"2020-08-07T15:12:50.500114+0000","last_fullsized":"2020-08-07T15:12:50.500114+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:13.679943+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.679943+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"263'3872","reported_seq":"3913","reported_epoch":"263","state":"active+clean","last_fresh":"2020-08-07T15:12:51.751184+0000","last_change":"2020-08-07T15:12:10.398816+0000","last_active":"2020-08-07T15:12:51.751184+0000","last_peered":"2020-08-07T15:12:51.751184+0000","last_clean":"2020-08-07T15:12:51.751184+0000","last_became_active":"2020-08-07T15:12:10.398332+0000","last_became_peered":"2020-08-07T15:12:10.398332+0000","last_unstale":"2020-08-07T15:12:51.751184+0000","last_undegraded":"2020-08-07T15:12:51.751184+0000","last_fullsized":"2020-08-07T15:12:51.751184+0000","mapping_epoch":230,"log_start":"263'3867","ondisk_log_start":"263'3867","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":31719424,"num_objects":484,"num_object_clones":0,"num_object_copies":1452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3872,"num_write_kb":30976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"236","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.506686+0000","last_change":"2020-08-07T15:12:14.241339+0000","last_active":"2020-08-07T15:12:50.506686+0000","last_peered":"2020-08-07T15:12:50.506686+0000","last_clean":"2020-08-07T15:12:50.506686+0000","last_became_active":"2020-08-07T15:10:46.498051+0000","last_became_peered":"2020-08-07T15:10:46.498051+0000","last_unstale":"2020-08-07T15:12:50.506686+0000","last_undegraded":"2020-08-07T15:12:50.506686+0000","last_fullsized":"2020-08-07T15:12:50.506686+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:14.241262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:14.241262+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"263'3824","reported_seq":"3865","reported_epoch":"263","state":"active+clean","last_fresh":"2020-08-07T15:12:51.587946+0000","last_change":"2020-08-07T15:12:10.398015+0000","last_active":"2020-08-07T15:12:51.587946+0000","last_peered":"2020-08-07T15:12:51.587946+0000","last_clean":"2020-08-07T15:12:51.587946+0000","last_became_active":"2020-08-07T15:12:10.397610+0000","last_became_peered":"2020-08-07T15:12:10.397610+0000","last_unstale":"2020-08-07T15:12:51.587946+0000","last_undegraded":"2020-08-07T15:12:51.587946+0000","last_fullsized":"2020-08-07T15:12:51.587946+0000","mapping_epoch":230,"log_start":"263'3816","ondisk_log_start":"263'3816","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":31326208,"num_objects":478,"num_object_clones":0,"num_object_copies":1434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3824,"num_write_kb":30592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"313","reported_epoch":"263","state":"active+clean","last_fresh":"2020-08-07T15:12:51.530085+0000","last_change":"2020-08-07T15:12:41.445456+0000","last_active":"2020-08-07T15:12:51.530085+0000","last_peered":"2020-08-07T15:12:51.530085+0000","last_clean":"2020-08-07T15:12:51.530085+0000","last_became_active":"2020-08-07T15:12:32.241722+0000","last_became_peered":"2020-08-07T15:12:32.241722+0000","last_unstale":"2020-08-07T15:12:51.530085+0000","last_undegraded":"2020-08-07T15:12:51.530085+0000","last_fullsized":"2020-08-07T15:12:51.530085+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:41.445379+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_clean_scrub_stamp":"2020-08-07T15:12:41.445379+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"263'3544","reported_seq":"3584","reported_epoch":"263","state":"active+clean","last_fresh":"2020-08-07T15:12:51.694440+0000","last_change":"2020-08-07T15:12:10.468495+0000","last_active":"2020-08-07T15:12:51.694440+0000","last_peered":"2020-08-07T15:12:51.694440+0000","last_clean":"2020-08-07T15:12:51.694440+0000","last_became_active":"2020-08-07T15:12:10.467915+0000","last_became_peered":"2020-08-07T15:12:10.467915+0000","last_unstale":"2020-08-07T15:12:51.694440+0000","last_undegraded":"2020-08-07T15:12:51.694440+0000","last_fullsized":"2020-08-07T15:12:51.694440+0000","mapping_epoch":230,"log_start":"263'3536","ondisk_log_start":"263'3536","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":29032448,"num_objects":443,"num_object_clones":0,"num_object_copies":1329,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3544,"num_write_kb":28352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,2],"acting":[7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"200","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.500184+0000","last_change":"2020-08-07T15:12:30.942002+0000","last_active":"2020-08-07T15:12:50.500184+0000","last_peered":"2020-08-07T15:12:50.500184+0000","last_clean":"2020-08-07T15:12:50.500184+0000","last_became_active":"2020-08-07T15:10:46.493498+0000","last_became_peered":"2020-08-07T15:10:46.493498+0000","last_unstale":"2020-08-07T15:12:50.500184+0000","last_undegraded":"2020-08-07T15:12:50.500184+0000","last_fullsized":"2020-08-07T15:12:50.500184+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:30.941943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:30.941943+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"263'3960","reported_seq":"4000","reported_epoch":"263","state":"active+clean","last_fresh":"2020-08-07T15:12:51.677889+0000","last_change":"2020-08-07T15:12:10.465251+0000","last_active":"2020-08-07T15:12:51.677889+0000","last_peered":"2020-08-07T15:12:51.677889+0000","last_clean":"2020-08-07T15:12:51.677889+0000","last_became_active":"2020-08-07T15:12:10.464698+0000","last_became_peered":"2020-08-07T15:12:10.464698+0000","last_unstale":"2020-08-07T15:12:51.677889+0000","last_undegraded":"2020-08-07T15:12:51.677889+0000","last_fullsized":"2020-08-07T15:12:51.677889+0000","mapping_epoch":230,"log_start":"263'3955","ondisk_log_start":"263'3955","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":32440320,"num_objects":495,"num_object_clones":0,"num_object_copies":1485,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3960,"num_write_kb":31680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"234","reported_epoch":"262","state":"active+clean","last_fresh":"2020-08-07T15:12:50.494309+0000","last_change":"2020-08-07T15:12:44.203274+0000","last_active":"2020-08-07T15:12:50.494309+0000","last_peered":"2020-08-07T15:12:50.494309+0000","last_clean":"2020-08-07T15:12:50.494309+0000","last_became_active":"2020-08-07T15:12:06.367081+0000","last_became_peered":"2020-08-07T15:12:06.367081+0000","last_unstale":"2020-08-07T15:12:50.494309+0000","last_undegraded":"2020-08-07T15:12:50.494309+0000","last_fullsized":"2020-08-07T15:12:50.494309+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:44.203221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:44.203221+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":494895104,"num_objects":7552,"num_object_clones":0,"num_object_copies":22656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60412,"num_write_kb":483296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1495416832,"data_stored":1495416832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":41,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":123,"acting":123,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542225,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1205804,"kb_used_data":157220,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93166036,"statfs":{"total":96636764160,"available":95402020864,"internally_reserved":1073741824,"allocated":160993280,"data_stored":159685818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":9.5359999999999996,"15min":3.774},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":2.294,"5min":856.07100000000003,"15min":856.07100000000003},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":1.002,"15min":0.97599999999999998},"min":{"1min":0.52300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.55,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":1.1830000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.335,"15min":1.3080000000000001},"min":{"1min":0.64200000000000002,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.9199999999999999,"5min":5.7519999999999998,"15min":5.7519999999999998},"last":1.026},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.141,"15min":1.024},"min":{"1min":0.61799999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":4.0060000000000002,"5min":4.0060000000000002,"15min":4.0060000000000002},"last":0.64800000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.321,"15min":1.1910000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.5630000000000002,"5min":5.6619999999999999,"15min":5.6619999999999999},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.081,"5min":1.3280000000000001,"15min":1.1879999999999999},"min":{"1min":0.69299999999999995,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.3769999999999998,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":1.0529999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":30.957999999999998,"5min":21.91,"15min":8.4179999999999993},"min":{"1min":0.68899999999999995,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":656.99099999999999,"5min":917.38699999999994,"15min":917.38699999999994},"last":1.22},{"interface":"front","average":{"1min":1.173,"5min":4.8879999999999999,"15min":2.524},"min":{"1min":0.61599999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":4.0460000000000003,"5min":333.59199999999998,"15min":333.59199999999998},"last":1.0800000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":8.8599999999999994,"5min":2.944,"15min":1.8320000000000001},"min":{"1min":0.66000000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":172.012,"5min":172.012,"15min":172.012},"last":0.996},{"interface":"front","average":{"1min":1.5509999999999999,"5min":11.529,"15min":4.7169999999999996},"min":{"1min":0.81200000000000006,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":5.4500000000000002,"5min":866.24099999999999,"15min":866.24099999999999},"last":1.1240000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":9.3930000000000007,"15min":28.135000000000002},"min":{"1min":0.68400000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":3.536,"5min":678.76099999999997,"15min":831.88099999999997},"last":1.153},{"interface":"front","average":{"1min":1.623,"5min":1.831,"15min":1.607},"min":{"1min":0.89000000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":3.593,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":1.343}]},{"osd":7,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":3.9929999999999999,"15min":2.343},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":4.0810000000000004,"5min":224.14500000000001,"15min":224.14500000000001},"last":1.246},{"interface":"front","average":{"1min":1.671,"5min":1.7649999999999999,"15min":1.593},"min":{"1min":0.92000000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":3.6200000000000001,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":1.4590000000000001}]}]},{"osd":7,"up_from":13,"seq":55834574930,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1226400,"kb_used_data":177816,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93145440,"statfs":{"total":96636764160,"available":95380930560,"internally_reserved":1073741824,"allocated":182083584,"data_stored":180783626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":37.372999999999998,"5min":16.672000000000001,"15min":6.4459999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":763.39099999999996,"5min":763.39099999999996,"15min":763.39099999999996},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.157,"5min":9.7789999999999999,"15min":4.0259999999999998},"min":{"1min":0.61699999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.8279999999999998,"5min":529.88699999999994,"15min":529.88699999999994},"last":0.84799999999999998}]},{"osd":1,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":5.758,"15min":2.5579999999999998},"min":{"1min":0.47699999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.7290000000000001,"5min":547.65300000000002,"15min":547.65300000000002},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.4159999999999999,"5min":4.9189999999999996,"15min":2.5640000000000001},"min":{"1min":0.57399999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":5.4960000000000004,"5min":372.50799999999998,"15min":372.50799999999998},"last":1.393}]},{"osd":2,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":3.0009999999999999,"5min":1.6850000000000001,"15min":1.3580000000000001},"min":{"1min":0.52000000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":33.131999999999998,"5min":33.131999999999998,"15min":33.131999999999998},"last":1.006},{"interface":"front","average":{"1min":2.8410000000000002,"5min":1.494,"15min":1.3460000000000001},"min":{"1min":0.61799999999999999,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":32.924999999999997,"5min":32.924999999999997,"15min":32.924999999999997},"last":0.97299999999999998}]},{"osd":3,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":9.4659999999999993,"15min":4.1719999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":5.9329999999999998,"5min":698.19899999999996,"15min":698.19899999999996},"last":0.871},{"interface":"front","average":{"1min":1.238,"5min":1.2669999999999999,"15min":1.347},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":3.8599999999999999,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":1.641}]},{"osd":4,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":25.448,"5min":6.069,"15min":3.1309999999999998},"min":{"1min":0.76400000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":460.78399999999999,"5min":460.78399999999999,"15min":460.78399999999999},"last":1.365},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.474,"15min":1.6060000000000001},"min":{"1min":0.86899999999999999,"5min":0.621,"15min":0.621},"max":{"1min":2.4079999999999999,"5min":3.694,"15min":5.2990000000000004},"last":1.5209999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.3720000000000001,"15min":1.3660000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":5.9989999999999997,"5min":6.2080000000000002,"15min":6.2080000000000002},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.492,"15min":1.6739999999999999},"min":{"1min":0.83599999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":4.407,"5min":4.407,"15min":5.327},"last":1.4550000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":9.1839999999999993,"15min":4.016},"min":{"1min":0.88,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.3290000000000002,"5min":695.92899999999997,"15min":695.92899999999997},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.597,"5min":1.55,"15min":12.734},"min":{"1min":0.89500000000000002,"5min":0.87,"15min":0.87},"max":{"1min":3.302,"5min":3.7309999999999999,"15min":266.26999999999998},"last":1.552}]}]},{"osd":1,"up_from":227,"seq":974957576202,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1193976,"kb_used_data":145392,"kb_used_omap":2,"kb_used_meta":1048573,"kb_avail":93177864,"statfs":{"total":96636764160,"available":95414132736,"internally_reserved":1073741824,"allocated":148881408,"data_stored":147592479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":1073739245},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.819}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5470000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.49}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2569999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]}]},{"osd":0,"up_from":13,"seq":55834574930,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1166728,"kb_used_data":118144,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93205112,"statfs":{"total":96636764160,"available":95442034688,"internally_reserved":1073741824,"allocated":120979456,"data_stored":119679498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:10:29 2020","interfaces":[{"interface":"back","average":{"1min":24.291,"5min":21.719999999999999,"15min":7.891},"min":{"1min":0.52700000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":432.899,"5min":845.57100000000003,"15min":845.57100000000003},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.752,"5min":1.181,"15min":1.0149999999999999},"min":{"1min":0.62,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":9.3650000000000002,"5min":9.3650000000000002,"15min":9.3650000000000002},"last":0.53600000000000003}]},{"osd":2,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.0620000000000001,"15min":0.97199999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":7.3470000000000004,"5min":7.3470000000000004,"15min":7.3470000000000004},"last":0.75900000000000001},{"interface":"front","average":{"1min":6.2839999999999998,"5min":4.665,"15min":23.381},"min":{"1min":0.47399999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":75.622,"5min":226.911,"15min":606.18499999999995},"last":0.74199999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":28.079000000000001,"5min":6.6920000000000002,"15min":2.8540000000000001},"min":{"1min":0.60999999999999999,"5min":0.54100000000000004,"15min":0.52300000000000002},"max":{"1min":431.95299999999997,"5min":431.95299999999997,"15min":431.95299999999997},"last":0.51000000000000001},{"interface":"front","average":{"1min":1.425,"5min":1.46,"15min":1.2649999999999999},"min":{"1min":0.66400000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":5.3780000000000001,"5min":11.19,"15min":11.19},"last":1.2150000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.302,"15min":1.2110000000000001},"min":{"1min":0.57399999999999995,"5min":0.501,"15min":0.501},"max":{"1min":1.819,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":0.629},{"interface":"front","average":{"1min":1.304,"5min":1.448,"15min":1.321},"min":{"1min":0.71099999999999997,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.8849999999999998,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":1.24}]},{"osd":5,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.6200000000000001,"5min":14.598000000000001,"15min":5.6840000000000002},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":7.3799999999999999,"5min":539.31100000000004,"15min":539.31100000000004},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.347,"5min":1.49,"15min":1.234},"min":{"1min":0.81399999999999995,"5min":0.752,"15min":0.65900000000000003},"max":{"1min":2.9420000000000002,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":0.92000000000000004}]},{"osd":6,"last update":"Fri Aug 7 15:12:34 2020","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.4399999999999999,"15min":6.2220000000000004},"min":{"1min":0.80200000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.8610000000000002,"5min":4.1310000000000002,"15min":124.738},"last":1.264},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.5549999999999999,"15min":12.952999999999999},"min":{"1min":0.73499999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":5.274,"5min":5.274,"15min":294.94},"last":0.79200000000000004}]},{"osd":7,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.4790000000000001,"15min":1.55},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.79300000000000004},"max":{"1min":1.8620000000000001,"5min":4.7169999999999996,"15min":6.468},"last":0.995},{"interface":"front","average":{"1min":66.063000000000002,"5min":23.518000000000001,"15min":8.7629999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":971.12699999999995,"5min":971.12699999999995,"15min":971.12699999999995},"last":1.1080000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574930,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1289104,"kb_used_data":240520,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93082736,"statfs":{"total":96636764160,"available":95316721664,"internally_reserved":1073741824,"allocated":246292480,"data_stored":244992522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":32.965000000000003,"5min":7.8639999999999999,"15min":3.3660000000000001},"min":{"1min":0.41899999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":501.53800000000001,"5min":501.53800000000001,"15min":501.53800000000001},"last":0.754},{"interface":"front","average":{"1min":27.341999999999999,"5min":6.6900000000000004,"15min":2.8799999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":501.36000000000001,"5min":501.36000000000001,"15min":501.36000000000001},"last":0.54800000000000004}]},{"osd":1,"last update":"Fri Aug 7 15:10:21 2020","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.6970000000000001,"15min":1.2929999999999999},"min":{"1min":0.52000000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":5.5919999999999996,"5min":18.672000000000001,"15min":18.672000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":1.264,"5min":1.3320000000000001,"15min":1.3500000000000001},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":3.7970000000000002,"5min":5.1130000000000004,"15min":5.1130000000000004},"last":465.69200000000001}]},{"osd":3,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.2130000000000001,"15min":1.179},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":8.0830000000000002},{"interface":"front","average":{"1min":23.064,"5min":15.281000000000001,"15min":6.1310000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":412.61500000000001,"5min":501.75599999999997,"15min":501.75599999999997},"last":7.7519999999999998}]},{"osd":4,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.5940000000000001,"15min":1.2709999999999999},"min":{"1min":0.67400000000000004,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.712,"5min":19.783000000000001,"15min":19.783000000000001},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.4610000000000001,"15min":1.5820000000000001},"min":{"1min":0.68999999999999995,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8979999999999999,"5min":8.048,"15min":8.048},"last":1.351}]},{"osd":5,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.3460000000000001,"15min":1.29},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.4009999999999998,"5min":5.2220000000000004,"15min":5.2220000000000004},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.4990000000000001,"5min":15.715,"15min":22.254999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":8.0649999999999995,"5min":627.53300000000002,"15min":627.53300000000002},"last":0.95399999999999996}]},{"osd":6,"last update":"Fri Aug 7 15:12:37 2020","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":9.9480000000000004,"15min":4.1299999999999999},"min":{"1min":0.875,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.9450000000000001,"5min":821.91899999999998,"15min":821.91899999999998},"last":1.381},{"interface":"front","average":{"1min":1.3959999999999999,"5min":9.9580000000000002,"15min":4.2000000000000002},"min":{"1min":0.70199999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.9540000000000002,"5min":751.70000000000005,"15min":751.70000000000005},"last":1.0720000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":12.522,"15min":5.335},"min":{"1min":0.85599999999999998,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.3519999999999999,"5min":501.91699999999997,"15min":501.91699999999997},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.7230000000000001,"5min":2.1080000000000001,"15min":1.7},"min":{"1min":0.88500000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":7.2149999999999999,"5min":35.646999999999998,"15min":35.646999999999998},"last":1.131}]}]},{"osd":3,"up_from":13,"seq":55834574930,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1359032,"kb_used_data":310448,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93012808,"statfs":{"total":96636764160,"available":95245115392,"internally_reserved":1073741824,"allocated":317898752,"data_stored":316598794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.226,"15min":1.244},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":4.6580000000000004,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.2050000000000001,"15min":1.4930000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":4.6349999999999998,"5min":4.9119999999999999,"15min":13.067},"last":1.119}]},{"osd":1,"last update":"Fri Aug 7 15:10:25 2020","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":4.4779999999999998,"15min":2.758},"min":{"1min":0.59099999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.847,"5min":270.68700000000001,"15min":270.68700000000001},"last":2.3889999999999998},{"interface":"front","average":{"1min":1.109,"5min":1.8580000000000001,"15min":1.96},"min":{"1min":0.54100000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.0550000000000002,"5min":14.349,"15min":14.349},"last":1.0620000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.365,"15min":1.22},"min":{"1min":0.65900000000000003,"5min":0.56999999999999995,"15min":0.56899999999999995},"max":{"1min":2.8300000000000001,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":1.093},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.9690000000000001,"15min":1.7789999999999999},"min":{"1min":0.64500000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":5.3090000000000002,"5min":51.530000000000001,"15min":51.530000000000001},"last":2.4359999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":3.831,"5min":17.792999999999999,"15min":7.0620000000000003},"min":{"1min":0.54500000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":56.688000000000002,"5min":646.26700000000005,"15min":646.26700000000005},"last":0.66600000000000004},{"interface":"front","average":{"1min":1.421,"5min":15.494999999999999,"15min":26.565000000000001},"min":{"1min":0.68100000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":3.2109999999999999,"5min":844.10500000000002,"15min":844.10500000000002},"last":1.242}]},{"osd":5,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":14.452999999999999,"5min":8.5429999999999993,"15min":3.859},"min":{"1min":0.95499999999999996,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":244.08099999999999,"5min":403.88400000000001,"15min":403.88400000000001},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.78,"5min":1.8979999999999999,"15min":1.8120000000000001},"min":{"1min":0.98499999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":4.5960000000000001,"5min":8.9120000000000008,"15min":8.9120000000000008},"last":1.556}]},{"osd":6,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.7969999999999999,"5min":1.706,"15min":1.5900000000000001},"min":{"1min":0.74099999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":5.2729999999999997,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":2.6880000000000002},{"interface":"front","average":{"1min":1.907,"5min":13.411,"15min":5.7949999999999999},"min":{"1min":1.032,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":5.3460000000000001,"5min":688.15999999999997,"15min":688.15999999999997},"last":2.7440000000000002}]},{"osd":7,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.7450000000000001,"5min":1.5960000000000001,"15min":1.9730000000000001},"min":{"1min":0.85599999999999998,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":4.8099999999999996,"5min":6.8209999999999997,"15min":6.8209999999999997},"last":2.7160000000000002},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.6559999999999999,"15min":1.8700000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.6099999999999999,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":1.325}]}]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1204736,"kb_used_data":156152,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93167104,"statfs":{"total":96636764160,"available":95403114496,"internally_reserved":1073741824,"allocated":159899648,"data_stored":158599690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":49.795000000000002,"5min":19.646000000000001,"15min":7.3360000000000003},"min":{"1min":0.46800000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":829.15099999999995,"5min":829.15099999999995,"15min":829.15099999999995},"last":1.0329999999999999},{"interface":"front","average":{"1min":3.8149999999999999,"5min":1.639,"15min":1.1120000000000001},"min":{"1min":0.45300000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":48.274000000000001,"5min":48.274000000000001,"15min":48.274000000000001},"last":0.99099999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:10:24 2020","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":5.7629999999999999,"15min":2.823},"min":{"1min":0.52900000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.8109999999999999,"5min":320.67000000000002,"15min":320.67000000000002},"last":1.099},{"interface":"front","average":{"1min":1.177,"5min":7.5300000000000002,"15min":3.4279999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.7200000000000002,"5min":689.846,"15min":689.846},"last":1.6160000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":4.1159999999999997,"5min":1.901,"15min":1.601},"min":{"1min":0.64300000000000002,"5min":0.56399999999999995,"15min":0.52100000000000002},"max":{"1min":49.878999999999998,"5min":49.878999999999998,"15min":49.878999999999998},"last":1.0649999999999999},{"interface":"front","average":{"1min":4.2270000000000003,"5min":2.0529999999999999,"15min":1.694},"min":{"1min":0.97899999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":49.939999999999998,"5min":49.939999999999998,"15min":49.939999999999998},"last":1.1279999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":27.59,"5min":17.260999999999999,"15min":6.7590000000000003},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":426.50799999999998,"5min":638.92100000000005,"15min":638.92100000000005},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.411,"5min":1.3799999999999999,"15min":1.294},"min":{"1min":0.75700000000000001,"5min":0.67800000000000005,"15min":0.66700000000000004},"max":{"1min":3.2360000000000002,"5min":6.431,"15min":6.431},"last":1.2070000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:12:29 2020","interfaces":[{"interface":"back","average":{"1min":52.308999999999997,"5min":28.574999999999999,"15min":10.609999999999999},"min":{"1min":0.56100000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":862.97299999999996,"5min":990.71600000000001,"15min":990.71600000000001},"last":1.169},{"interface":"front","average":{"1min":49.549999999999997,"5min":11.01,"15min":4.7539999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":780.00699999999995,"5min":780.00699999999995,"15min":780.00699999999995},"last":1.4390000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:12:34 2020","interfaces":[{"interface":"back","average":{"1min":3.8239999999999998,"5min":1.9870000000000001,"15min":1.855},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":48.575000000000003,"5min":48.575000000000003,"15min":48.575000000000003},"last":1.415},{"interface":"front","average":{"1min":1.109,"5min":1.333,"15min":13.183},"min":{"1min":0.64600000000000002,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.847,"5min":6.7729999999999997,"15min":389.14600000000002},"last":1.4830000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":4.1349999999999998,"5min":4.327,"15min":2.3809999999999998},"min":{"1min":0.74199999999999999,"5min":0.62,"15min":0.62},"max":{"1min":48.75,"5min":252.798,"15min":252.798},"last":1.1319999999999999},{"interface":"front","average":{"1min":4.2610000000000001,"5min":14.635,"15min":5.9749999999999996},"min":{"1min":0.84799999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":49.654000000000003,"5min":501.93000000000001,"15min":501.93000000000001},"last":1.175}]}]},{"osd":5,"up_from":13,"seq":55834574929,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1203384,"kb_used_data":154800,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93168456,"statfs":{"total":96636764160,"available":95404498944,"internally_reserved":1073741824,"allocated":158515200,"data_stored":157215242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":3.629,"5min":1.506,"15min":1.22},"min":{"1min":0.42199999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":43.292999999999999,"5min":43.292999999999999,"15min":43.292999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":3.1520000000000001,"5min":4.4249999999999998,"15min":2.0110000000000001},"min":{"1min":0.44700000000000001,"5min":0.438,"15min":0.438},"max":{"1min":42.933,"5min":329.11399999999998,"15min":329.11399999999998},"last":0.51300000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:10:26 2020","interfaces":[{"interface":"back","average":{"1min":8.5239999999999991,"5min":12.039999999999999,"15min":4.6319999999999997},"min":{"1min":0.56799999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":139.88200000000001,"5min":787.22900000000004,"15min":787.22900000000004},"last":1.246},{"interface":"front","average":{"1min":1.448,"5min":2.5649999999999999,"15min":1.593},"min":{"1min":0.59899999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":4.3700000000000001,"5min":116.548,"15min":116.548},"last":1.2749999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":3.5499999999999998,"5min":1.5800000000000001,"15min":6.673},"min":{"1min":0.65700000000000003,"5min":0.499,"15min":0.499},"max":{"1min":43.923000000000002,"5min":43.923000000000002,"15min":168.56899999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":3.23,"5min":1.6399999999999999,"15min":22.634},"min":{"1min":0.52800000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":43.076999999999998,"5min":43.076999999999998,"15min":676.27099999999996},"last":0.74299999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.728,"15min":1.321},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":4.7640000000000002,"5min":29.129000000000001,"15min":29.129000000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":10.029999999999999,"15min":4.1520000000000001},"min":{"1min":0.73999999999999999,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":5.2009999999999996,"5min":712.98699999999997,"15min":712.98699999999997},"last":1.1719999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":12.779999999999999,"15min":38.188000000000002},"min":{"1min":0.77200000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":4.9969999999999999,"5min":971.67999999999995,"15min":971.67999999999995},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.2470000000000001,"15min":1.286},"min":{"1min":0.73599999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.1160000000000001,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":0.78300000000000003}]},{"osd":6,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":35.393000000000001,"5min":28.459,"15min":43.091999999999999},"min":{"1min":0.72199999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":676.20000000000005,"5min":966.60900000000004,"15min":966.60900000000004},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.528,"5min":6.7789999999999999,"15min":19.603999999999999},"min":{"1min":0.89500000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":5.2220000000000004,"5min":501.79000000000002,"15min":517.26700000000005},"last":1.212}]},{"osd":7,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.462,"15min":1.3600000000000001},"min":{"1min":0.85399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":4.8600000000000003,"5min":5.1559999999999997,"15min":5.1559999999999997},"last":1.625},{"interface":"front","average":{"1min":1.5669999999999999,"5min":12.834,"15min":5.3220000000000001},"min":{"1min":0.70099999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":3.8999999999999999,"5min":515.09100000000001,"15min":515.09100000000001},"last":1.379}]}]}]}} 2020-08-07T15:12:54.871 INFO:tasks.thrashosds.thrasher:Setting 6.a to [6, 5, 7] 2020-08-07T15:12:54.872 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '6.a', '6', '5', '7'] 2020-08-07T15:12:54.872 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pg-upmap 6.a 6 5 7 2020-08-07T15:12:54.910 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:12:54.938 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:54.931+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:54.968 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:54.969 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:54.980 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:12:55.069 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:55.068+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:55.142 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:55.138+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:55.188 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:55.216 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:55.218 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:12:55.255 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:55.242+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:55.271 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 43 16 66618 66602 12.0993 12.7812 0.00698135 0.0103253 2020-08-07T15:12:55.359 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:55.344+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:55.464 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:55.464 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:55.465 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:55.462+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:55.465 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:12:55.555 INFO:teuthology.orchestra.run.smithi074.stderr:set 6.a pg_upmap mapping to [6,5,7] 2020-08-07T15:12:55.556 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:12:55.572 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:55.566+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:55.674 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:55.675 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:55.695 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:12:55.696 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:55.668+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:55.780 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:55.772+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:55.813 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:12:55.854 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:55.855 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:55.860 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:12:55.875 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:55.874+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:55.970 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:12:55.979 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:55.977+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:55.990 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:12:55.991 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:55.991 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:55.996 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:12:56.015 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:12:56.050 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:12:56.084 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:56.081+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:56.094 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:12:56.119 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:12:56.152 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:12:56.153 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:56.153 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:12:56.153 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:12:56.153 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:56.167 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:12:56.169 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:12:56.169 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:12:56.191 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:12:56.191 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:12:56.192 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:12:56.192 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:12:56.193 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:12:56.193 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:12:56.193 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:12:56.201 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:56.195+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:56.269 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 44 16 67302 67286 11.9458 5.34375 0.233566 0.0103345 2020-08-07T15:12:56.296 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:12:56.295+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:12:56.671 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:56.671 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:12:56.672 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:12:56.672 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:12:56.672 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:56.672 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:12:56.670+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:12:56.673 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:12:56.775 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:12:56.773+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:12:56.853 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:56.859 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:12:56.859 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:12:56.860 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:12:56.860 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:56.860 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "MOSDScrubReserve(2.6 GRANT e232)", 2020-08-07T15:12:56.860 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:11.909673+0000", 2020-08-07T15:12:56.860 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 44.935826398000003, 2020-08-07T15:12:56.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.36313456, 2020-08-07T15:12:56.861 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:56.861 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:12:56.861 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:56.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:56.898 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:56.898 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:11.909673+0000", 2020-08-07T15:12:56.898 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:56.898 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:56.898 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:56.908 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:56.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:11.909673+0000", 2020-08-07T15:12:56.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0409000000000001e-05 2020-08-07T15:12:56.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:56.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:56.910 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:56.910 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:11.909693+0000", 2020-08-07T15:12:56.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8580000000000002e-06 2020-08-07T15:12:56.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:56.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:56.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:56.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:11.909697+0000", 2020-08-07T15:12:56.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.36297494299999999 2020-08-07T15:12:56.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:56.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:56.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:56.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272672+0000", 2020-08-07T15:12:56.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4490000000000001e-06 2020-08-07T15:12:56.934 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:56.934 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:56.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:56.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272676+0000", 2020-08-07T15:12:56.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0395999999999994e-05 2020-08-07T15:12:56.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:56.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:56.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:56.936 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272757+0000", 2020-08-07T15:12:56.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6416e-05 2020-08-07T15:12:56.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:56.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:56.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:56.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272773+0000", 2020-08-07T15:12:56.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4088999999999998e-05 2020-08-07T15:12:56.937 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:56.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:56.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:56.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272807+0000", 2020-08-07T15:12:56.953 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.36313456 2020-08-07T15:12:56.954 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:56.954 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:56.954 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:56.954 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:56.955 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:56.955 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:7272 6.e 6:74263cb8:::benchmark_data_smithi074_63843_object908:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e234)", 2020-08-07T15:12:56.960 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:16.977784+0000", 2020-08-07T15:12:56.961 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 39.867714650000003, 2020-08-07T15:12:56.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31581518600000003, 2020-08-07T15:12:56.967 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:56.967 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:56.968 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:56.968 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:56.968 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:56.968 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7272 2020-08-07T15:12:56.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:56.969 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:56.969 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:56.969 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:56.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977784+0000", 2020-08-07T15:12:56.969 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:56.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:56.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:56.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:56.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977784+0000", 2020-08-07T15:12:56.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6940999999999998e-05 2020-08-07T15:12:56.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:56.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:56.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:56.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977811+0000", 2020-08-07T15:12:56.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1870000000000001e-06 2020-08-07T15:12:56.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:56.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:56.978 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:56.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977815+0000", 2020-08-07T15:12:57.001 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1249999999999998e-06 2020-08-07T15:12:57.002 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.002 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.002 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:57.002 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977821+0000", 2020-08-07T15:12:57.002 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8739999999999999e-06 2020-08-07T15:12:57.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:57.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977824+0000", 2020-08-07T15:12:57.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001740655 2020-08-07T15:12:57.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.049 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:57.049 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.979564+0000", 2020-08-07T15:12:57.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6272000000000003e-05 2020-08-07T15:12:57.050 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.067 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.067 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:57.067 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.979650+0000", 2020-08-07T15:12:57.067 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012951600000000001 2020-08-07T15:12:57.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.068 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.068 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:12:57.068 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.979780+0000", 2020-08-07T15:12:57.068 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00085408999999999995 2020-08-07T15:12:57.069 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.074 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.075 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:57.075 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.980634+0000", 2020-08-07T15:12:57.080 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8099999999999999e-07 2020-08-07T15:12:57.080 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.081 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.081 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:57.081 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.980634+0000", 2020-08-07T15:12:57.082 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5664000000000003e-05 2020-08-07T15:12:57.082 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.083 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:57.083 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.980670+0000", 2020-08-07T15:12:57.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.104650663 2020-08-07T15:12:57.084 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.084 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:57.085 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.085321+0000", 2020-08-07T15:12:57.085 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20821616000000001 2020-08-07T15:12:57.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:57.087 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.293537+0000", 2020-08-07T15:12:57.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1900000000000001e-07 2020-08-07T15:12:57.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.089 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.089 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:57.090 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.293538+0000", 2020-08-07T15:12:57.090 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1939000000000004e-05 2020-08-07T15:12:57.091 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.091 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.093 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:57.093 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.293600+0000", 2020-08-07T15:12:57.093 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31581518600000003 2020-08-07T15:12:57.094 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:57.095 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:57.095 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:57.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.096 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.107 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:7984 6.1 6:83bbbe87:::benchmark_data_smithi074_63843_object997:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e235)", 2020-08-07T15:12:57.108 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:17.448389+0000", 2020-08-07T15:12:57.108 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 39.397110458999997, 2020-08-07T15:12:57.108 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26151915100000001, 2020-08-07T15:12:57.109 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:57.109 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:57.109 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:57.110 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:57.110 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:57.150 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7984 2020-08-07T15:12:57.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.151 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:57.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:57.152 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.448389+0000", 2020-08-07T15:12:57.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:57.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.158 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.158 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:57.158 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.448389+0000", 2020-08-07T15:12:57.158 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0078999999999999e-05 2020-08-07T15:12:57.158 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.159 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.159 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:57.159 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.448439+0000", 2020-08-07T15:12:57.159 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7170000000000008e-06 2020-08-07T15:12:57.159 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.160 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.160 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:57.160 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.448446+0000", 2020-08-07T15:12:57.160 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2069000000000001e-05 2020-08-07T15:12:57.160 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.161 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.161 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:57.161 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.448458+0000", 2020-08-07T15:12:57.161 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4489999999999998e-06 2020-08-07T15:12:57.161 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.162 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.162 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:57.162 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.448464+0000", 2020-08-07T15:12:57.162 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00053153599999999999 2020-08-07T15:12:57.162 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.163 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.163 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:57.170 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.448995+0000", 2020-08-07T15:12:57.170 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000102731 2020-08-07T15:12:57.170 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.171 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.171 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:57.171 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.449098+0000", 2020-08-07T15:12:57.171 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013514700000000001 2020-08-07T15:12:57.171 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.172 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.172 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:12:57.172 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.449233+0000", 2020-08-07T15:12:57.172 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000629399 2020-08-07T15:12:57.172 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.173 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.173 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:57.173 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.449863+0000", 2020-08-07T15:12:57.173 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8100000000000003e-07 2020-08-07T15:12:57.173 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.178 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.178 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:57.178 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.449863+0000", 2020-08-07T15:12:57.178 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6907999999999998e-05 2020-08-07T15:12:57.178 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.179 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.179 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:57.179 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.449920+0000", 2020-08-07T15:12:57.179 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024481630000000002 2020-08-07T15:12:57.179 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.180 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.180 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:57.180 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.452368+0000", 2020-08-07T15:12:57.180 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.257502279 2020-08-07T15:12:57.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.181 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.181 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:57.181 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.709871+0000", 2020-08-07T15:12:57.181 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2300000000000001e-07 2020-08-07T15:12:57.181 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.187 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.187 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:57.187 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.709871+0000", 2020-08-07T15:12:57.187 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6470000000000001e-05 2020-08-07T15:12:57.188 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.188 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.188 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:57.188 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.709908+0000", 2020-08-07T15:12:57.188 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26151915100000001 2020-08-07T15:12:57.189 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:57.189 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:57.189 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:57.190 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.190 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.190 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:8359 6.1 6:844c6d67:::benchmark_data_smithi074_63843_object1044:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e235)", 2020-08-07T15:12:57.191 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:17.720675+0000", 2020-08-07T15:12:57.191 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 39.124824492999998, 2020-08-07T15:12:57.191 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.309231859, 2020-08-07T15:12:57.191 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:57.191 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:57.192 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:57.192 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:57.192 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:57.192 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 8359 2020-08-07T15:12:57.192 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.193 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:57.193 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.193 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:57.193 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720675+0000", 2020-08-07T15:12:57.193 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:57.194 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.194 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.194 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:57.194 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720675+0000", 2020-08-07T15:12:57.194 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.884e-05 2020-08-07T15:12:57.194 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.195 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.195 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:57.195 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720723+0000", 2020-08-07T15:12:57.195 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2989999999999996e-06 2020-08-07T15:12:57.195 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.196 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.196 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:57.206 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720729+0000", 2020-08-07T15:12:57.206 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0288999999999999e-05 2020-08-07T15:12:57.206 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.206 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.207 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:57.207 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720739+0000", 2020-08-07T15:12:57.207 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7849999999999999e-06 2020-08-07T15:12:57.207 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.208 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.208 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:57.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720744+0000", 2020-08-07T15:12:57.209 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001862991 2020-08-07T15:12:57.209 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.209 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.231 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:57.231 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.722607+0000", 2020-08-07T15:12:57.232 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000100839 2020-08-07T15:12:57.232 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.232 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.232 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:57.232 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.722708+0000", 2020-08-07T15:12:57.233 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017526600000000001 2020-08-07T15:12:57.233 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.233 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.233 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:12:57.250 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.722883+0000", 2020-08-07T15:12:57.250 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001504619 2020-08-07T15:12:57.250 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.250 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.251 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:57.290 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.724387+0000", 2020-08-07T15:12:57.290 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4499999999999996e-07 2020-08-07T15:12:57.291 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.291 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.291 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:57.291 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.724388+0000", 2020-08-07T15:12:57.291 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8369000000000003e-05 2020-08-07T15:12:57.292 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.292 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.292 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:57.292 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.724447+0000", 2020-08-07T15:12:57.307 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00097551199999999999 2020-08-07T15:12:57.308 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.308 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:57.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.725422+0000", 2020-08-07T15:12:57.308 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30441636900000002 2020-08-07T15:12:57.309 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.309 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.309 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:57.309 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029838+0000", 2020-08-07T15:12:57.309 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7699999999999992e-07 2020-08-07T15:12:57.318 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.318 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.318 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:57.319 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029839+0000", 2020-08-07T15:12:57.319 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6958999999999994e-05 2020-08-07T15:12:57.319 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.319 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.319 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:57.320 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029906+0000", 2020-08-07T15:12:57.320 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.309231859 2020-08-07T15:12:57.320 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:57.320 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:57.320 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:57.320 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.321 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.321 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:8360 6.1 6:844c6d67:::benchmark_data_smithi074_63843_object1044:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e235)", 2020-08-07T15:12:57.321 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:17.721396+0000", 2020-08-07T15:12:57.321 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 39.124102829999998, 2020-08-07T15:12:57.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31055011399999999, 2020-08-07T15:12:57.341 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:57.370 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:57.370 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:57.371 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:57.371 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:57.371 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 8360 2020-08-07T15:12:57.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.371 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:57.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.372 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:57.372 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721396+0000", 2020-08-07T15:12:57.372 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:57.372 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.373 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.373 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:57.373 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721396+0000", 2020-08-07T15:12:57.373 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8222e-05 2020-08-07T15:12:57.373 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:57.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721444+0000", 2020-08-07T15:12:57.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.297e-06 2020-08-07T15:12:57.379 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:57.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721451+0000", 2020-08-07T15:12:57.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.172e-05 2020-08-07T15:12:57.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:57.380 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721462+0000", 2020-08-07T15:12:57.380 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7639999999999996e-06 2020-08-07T15:12:57.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:57.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721467+0000", 2020-08-07T15:12:57.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023051790000000001 2020-08-07T15:12:57.395 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.395 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.395 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:57.395 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.723772+0000", 2020-08-07T15:12:57.396 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1051000000000006e-05 2020-08-07T15:12:57.396 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.396 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.396 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:57.396 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.723863+0000", 2020-08-07T15:12:57.398 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017915699999999999 2020-08-07T15:12:57.398 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.398 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.398 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:12:57.398 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.724043+0000", 2020-08-07T15:12:57.399 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001191646 2020-08-07T15:12:57.399 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.399 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:57.399 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.725234+0000", 2020-08-07T15:12:57.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9100000000000002e-07 2020-08-07T15:12:57.400 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.400 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:57.400 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.725235+0000", 2020-08-07T15:12:57.401 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0677999999999997e-05 2020-08-07T15:12:57.401 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.401 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.401 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:57.401 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.725296+0000", 2020-08-07T15:12:57.402 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001252663 2020-08-07T15:12:57.402 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.402 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.402 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:57.402 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.726548+0000", 2020-08-07T15:12:57.403 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30531503100000001 2020-08-07T15:12:57.403 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.403 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.403 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:57.403 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.031864+0000", 2020-08-07T15:12:57.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.057e-06 2020-08-07T15:12:57.404 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.404 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.404 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:57.404 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.031865+0000", 2020-08-07T15:12:57.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1757999999999999e-05 2020-08-07T15:12:57.405 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.405 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:57.405 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.031946+0000", 2020-08-07T15:12:57.405 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31055011399999999 2020-08-07T15:12:57.419 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:57.419 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:57.420 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:57.420 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.420 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.421 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:8359 6.1 e235/230 ondisk, result = 0)", 2020-08-07T15:12:57.421 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:17.726764+0000", 2020-08-07T15:12:57.421 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 39.118734814, 2020-08-07T15:12:57.421 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30314883199999998, 2020-08-07T15:12:57.421 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:57.422 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:12:57.422 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:57.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.422 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:57.422 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.726764+0000", 2020-08-07T15:12:57.423 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:57.423 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.423 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.423 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:57.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.726764+0000", 2020-08-07T15:12:57.424 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1386000000000001e-05 2020-08-07T15:12:57.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.424 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.424 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:57.424 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.726796+0000", 2020-08-07T15:12:57.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7989999999999999e-06 2020-08-07T15:12:57.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.425 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:57.425 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.726801+0000", 2020-08-07T15:12:57.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30298438700000002 2020-08-07T15:12:57.426 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:57.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029786+0000", 2020-08-07T15:12:57.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5040000000000002e-06 2020-08-07T15:12:57.438 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.438 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.438 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:57.438 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029791+0000", 2020-08-07T15:12:57.438 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0737000000000001e-05 2020-08-07T15:12:57.439 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.439 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.439 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:57.439 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029822+0000", 2020-08-07T15:12:57.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9359999999999999e-06 2020-08-07T15:12:57.440 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.441 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:57.441 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029830+0000", 2020-08-07T15:12:57.441 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3083000000000004e-05 2020-08-07T15:12:57.441 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.441 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.442 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:57.442 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029913+0000", 2020-08-07T15:12:57.442 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30314883199999998 2020-08-07T15:12:57.442 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:57.442 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:57.442 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:57.443 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.443 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.443 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:10224 6.e 6:77f719e6:::benchmark_data_smithi074_63843_object1277:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e236)", 2020-08-07T15:12:57.443 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:19.038704+0000", 2020-08-07T15:12:57.443 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 37.806795512000001, 2020-08-07T15:12:57.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32955690900000001, 2020-08-07T15:12:57.444 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:57.444 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:57.444 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:57.444 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:57.456 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:57.457 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 10224 2020-08-07T15:12:57.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.457 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:57.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:57.461 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038704+0000", 2020-08-07T15:12:57.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:57.461 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.462 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.462 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:57.462 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038704+0000", 2020-08-07T15:12:57.462 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6672000000000001e-05 2020-08-07T15:12:57.462 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.463 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.463 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:57.463 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038730+0000", 2020-08-07T15:12:57.463 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.27e-06 2020-08-07T15:12:57.463 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.464 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.464 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:57.464 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038733+0000", 2020-08-07T15:12:57.464 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4030000000000004e-06 2020-08-07T15:12:57.464 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.465 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.465 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:57.465 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038740+0000", 2020-08-07T15:12:57.465 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.712e-06 2020-08-07T15:12:57.465 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:57.488 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038743+0000", 2020-08-07T15:12:57.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0025767279999999999 2020-08-07T15:12:57.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.489 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.489 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:57.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.041319+0000", 2020-08-07T15:12:57.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4727e-05 2020-08-07T15:12:57.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.521 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.522 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:57.522 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.041374+0000", 2020-08-07T15:12:57.522 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9403999999999999e-05 2020-08-07T15:12:57.522 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.522 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.523 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:12:57.523 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.041473+0000", 2020-08-07T15:12:57.523 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010041290000000001 2020-08-07T15:12:57.523 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.523 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.524 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:57.524 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.042478+0000", 2020-08-07T15:12:57.524 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0100000000000003e-07 2020-08-07T15:12:57.524 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.524 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.525 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:57.525 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.042478+0000", 2020-08-07T15:12:57.525 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7923999999999995e-05 2020-08-07T15:12:57.525 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.525 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:57.535 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.042546+0000", 2020-08-07T15:12:57.535 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0046169310000000003 2020-08-07T15:12:57.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.536 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:57.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.047163+0000", 2020-08-07T15:12:57.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.321023999 2020-08-07T15:12:57.537 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.537 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.537 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:57.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368187+0000", 2020-08-07T15:12:57.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.048e-06 2020-08-07T15:12:57.538 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.538 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.538 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:57.538 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368188+0000", 2020-08-07T15:12:57.539 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2061e-05 2020-08-07T15:12:57.539 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.539 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.539 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:57.539 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368260+0000", 2020-08-07T15:12:57.540 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32955690900000001 2020-08-07T15:12:57.540 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:57.540 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:57.540 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:57.540 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.541 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:10224 6.e e236/230 ondisk, result = 0)", 2020-08-07T15:12:57.541 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:19.047929+0000", 2020-08-07T15:12:57.541 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 37.797569912, 2020-08-07T15:12:57.541 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32041415400000001, 2020-08-07T15:12:57.542 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:57.542 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:12:57.542 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:57.542 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.542 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:57.542 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.047929+0000", 2020-08-07T15:12:57.543 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:57.543 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.543 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.543 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:57.543 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.047929+0000", 2020-08-07T15:12:57.544 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9706999999999999e-05 2020-08-07T15:12:57.544 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.544 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.544 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:57.544 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.047959+0000", 2020-08-07T15:12:57.575 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9100000000000004e-06 2020-08-07T15:12:57.575 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.575 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:57.576 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.047964+0000", 2020-08-07T15:12:57.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31873185399999998 2020-08-07T15:12:57.576 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.576 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.576 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:57.577 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.366696+0000", 2020-08-07T15:12:57.577 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2970000000000003e-06 2020-08-07T15:12:57.577 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.606 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.606 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:57.606 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.366701+0000", 2020-08-07T15:12:57.606 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001472522 2020-08-07T15:12:57.607 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.607 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.607 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:57.607 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368173+0000", 2020-08-07T15:12:57.607 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5949999999999998e-06 2020-08-07T15:12:57.608 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.608 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.608 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:57.608 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368179+0000", 2020-08-07T15:12:57.608 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016426900000000001 2020-08-07T15:12:57.609 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.609 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.609 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:57.609 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368343+0000", 2020-08-07T15:12:57.609 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32041415400000001 2020-08-07T15:12:57.625 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:57.625 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:57.625 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:57.625 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.626 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.626 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:18480 6.1 6:8a2c1569:::benchmark_data_smithi074_63843_object2309:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e240)", 2020-08-07T15:12:57.626 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.367850+0000", 2020-08-07T15:12:57.626 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 32.477649501999998, 2020-08-07T15:12:57.626 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41222172800000001, 2020-08-07T15:12:57.627 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:57.627 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:57.627 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:57.627 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:57.627 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:57.628 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18480 2020-08-07T15:12:57.628 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.628 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:57.628 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.628 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:57.629 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367850+0000", 2020-08-07T15:12:57.629 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:57.629 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.629 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.629 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:57.657 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367850+0000", 2020-08-07T15:12:57.658 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6174999999999999e-05 2020-08-07T15:12:57.658 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.658 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.658 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:57.658 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367876+0000", 2020-08-07T15:12:57.659 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.061e-06 2020-08-07T15:12:57.659 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.659 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.659 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:57.659 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367879+0000", 2020-08-07T15:12:57.660 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7899999999999996e-06 2020-08-07T15:12:57.660 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.661 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.662 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:57.662 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367885+0000", 2020-08-07T15:12:57.662 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6589999999999999e-06 2020-08-07T15:12:57.662 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.662 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.663 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:57.663 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367887+0000", 2020-08-07T15:12:57.663 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001651275 2020-08-07T15:12:57.663 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.663 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.664 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:57.664 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.369538+0000", 2020-08-07T15:12:57.664 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7794000000000006e-05 2020-08-07T15:12:57.664 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.665 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:57.665 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.369636+0000", 2020-08-07T15:12:57.665 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017274 2020-08-07T15:12:57.665 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.665 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.666 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:12:57.666 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.369809+0000", 2020-08-07T15:12:57.666 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010733769999999999 2020-08-07T15:12:57.666 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.666 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.666 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:57.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.370882+0000", 2020-08-07T15:12:57.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0100000000000003e-07 2020-08-07T15:12:57.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.667 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:57.668 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.370883+0000", 2020-08-07T15:12:57.668 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6001000000000001e-05 2020-08-07T15:12:57.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:57.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.370939+0000", 2020-08-07T15:12:57.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074658490000000001 2020-08-07T15:12:57.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:57.682 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.378405+0000", 2020-08-07T15:12:57.682 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40160148899999998 2020-08-07T15:12:57.683 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.683 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.683 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:57.683 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.780007+0000", 2020-08-07T15:12:57.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9700000000000005e-07 2020-08-07T15:12:57.684 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.684 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.690 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:57.690 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.780008+0000", 2020-08-07T15:12:57.690 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3720000000000007e-05 2020-08-07T15:12:57.690 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.691 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.691 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:57.691 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.780071+0000", 2020-08-07T15:12:57.691 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41222172800000001 2020-08-07T15:12:57.691 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:57.692 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:57.692 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:57.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.692 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:18480 6.1 e240/230 ondisk, result = 0)", 2020-08-07T15:12:57.693 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.375696+0000", 2020-08-07T15:12:57.693 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 32.469802899000001, 2020-08-07T15:12:57.693 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.404380817, 2020-08-07T15:12:57.693 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:57.693 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:12:57.718 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:57.718 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.718 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:57.718 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.375696+0000", 2020-08-07T15:12:57.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:57.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:57.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.375696+0000", 2020-08-07T15:12:57.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0922e-05 2020-08-07T15:12:57.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:57.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.375727+0000", 2020-08-07T15:12:57.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2900000000000002e-06 2020-08-07T15:12:57.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.721 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:57.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.375732+0000", 2020-08-07T15:12:57.722 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.403431708 2020-08-07T15:12:57.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.722 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:57.722 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.779164+0000", 2020-08-07T15:12:57.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5200000000000002e-06 2020-08-07T15:12:57.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.723 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:57.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.779168+0000", 2020-08-07T15:12:57.724 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00082257200000000004 2020-08-07T15:12:57.724 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.724 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.724 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:57.724 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.779990+0000", 2020-08-07T15:12:57.725 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1599999999999998e-06 2020-08-07T15:12:57.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.725 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:57.725 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.779998+0000", 2020-08-07T15:12:57.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8645000000000003e-05 2020-08-07T15:12:57.782 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.782 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:57.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.780077+0000", 2020-08-07T15:12:57.783 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.404380817 2020-08-07T15:12:57.783 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:57.783 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:57.784 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:57.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.784 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:18560 6.e 6:7c5f8c5e:::benchmark_data_smithi074_63843_object2319:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e240)", 2020-08-07T15:12:57.784 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.407761+0000", 2020-08-07T15:12:57.785 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 32.437737636000001, 2020-08-07T15:12:57.785 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31539320100000001, 2020-08-07T15:12:57.785 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:57.785 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:57.785 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:57.794 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:57.794 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:57.795 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18560 2020-08-07T15:12:57.795 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.795 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:57.795 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.795 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:57.796 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407761+0000", 2020-08-07T15:12:57.796 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:57.796 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.796 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:57.797 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407761+0000", 2020-08-07T15:12:57.797 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0897000000000003e-05 2020-08-07T15:12:57.797 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.797 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.797 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:57.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407822+0000", 2020-08-07T15:12:57.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8390000000000001e-06 2020-08-07T15:12:57.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:57.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407829+0000", 2020-08-07T15:12:57.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3609e-05 2020-08-07T15:12:57.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:57.826 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407843+0000", 2020-08-07T15:12:57.826 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3169999999999998e-06 2020-08-07T15:12:57.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.827 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:57.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407849+0000", 2020-08-07T15:12:57.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002504886 2020-08-07T15:12:57.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:57.828 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.410354+0000", 2020-08-07T15:12:57.828 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7235e-05 2020-08-07T15:12:57.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.829 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.829 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:57.829 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.410451+0000", 2020-08-07T15:12:57.829 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016856999999999999 2020-08-07T15:12:57.829 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.838 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:12:57.838 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.410620+0000", 2020-08-07T15:12:57.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026816259999999999 2020-08-07T15:12:57.873 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.874 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:57.874 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.413301+0000", 2020-08-07T15:12:57.874 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0020516309999999999 2020-08-07T15:12:57.874 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.874 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:57.875 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.415353+0000", 2020-08-07T15:12:57.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30769196399999998 2020-08-07T15:12:57.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:57.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.723045+0000", 2020-08-07T15:12:57.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1600000000000001e-07 2020-08-07T15:12:57.876 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.876 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:57.877 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.723046+0000", 2020-08-07T15:12:57.877 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1449000000000001e-05 2020-08-07T15:12:57.877 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.877 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.877 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:57.887 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.723107+0000", 2020-08-07T15:12:57.887 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7462000000000001e-05 2020-08-07T15:12:57.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:57.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.723155+0000", 2020-08-07T15:12:57.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31539320100000001 2020-08-07T15:12:57.888 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:57.888 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:57.889 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:57.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.919 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:18608 6.4 6:2dddf5bf:::benchmark_data_smithi074_63843_object2325:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e240)", 2020-08-07T15:12:57.919 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.438031+0000", 2020-08-07T15:12:57.919 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 32.40746824, 2020-08-07T15:12:57.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28378579100000001, 2020-08-07T15:12:57.919 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:57.920 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:57.920 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:57.920 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:57.920 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:57.920 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18608 2020-08-07T15:12:57.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.921 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:57.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:57.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438031+0000", 2020-08-07T15:12:57.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:57.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.922 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:57.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438031+0000", 2020-08-07T15:12:57.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0028999999999998e-05 2020-08-07T15:12:57.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:57.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438081+0000", 2020-08-07T15:12:57.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7289999999999997e-06 2020-08-07T15:12:57.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:57.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438087+0000", 2020-08-07T15:12:57.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0959e-05 2020-08-07T15:12:57.925 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:57.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438098+0000", 2020-08-07T15:12:57.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0329999999999999e-06 2020-08-07T15:12:57.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:57.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438103+0000", 2020-08-07T15:12:57.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001770911 2020-08-07T15:12:57.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:57.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.439873+0000", 2020-08-07T15:12:57.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3503000000000005e-05 2020-08-07T15:12:57.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:57.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.439937+0000", 2020-08-07T15:12:57.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000110179 2020-08-07T15:12:57.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,5", 2020-08-07T15:12:57.966 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.440047+0000", 2020-08-07T15:12:57.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033910759999999998 2020-08-07T15:12:57.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:57.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443438+0000", 2020-08-07T15:12:57.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2877999999999999e-05 2020-08-07T15:12:57.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:57.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443471+0000", 2020-08-07T15:12:57.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27823471799999999 2020-08-07T15:12:57.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:57.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721706+0000", 2020-08-07T15:12:57.969 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4300000000000003e-07 2020-08-07T15:12:57.969 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.969 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.969 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:57.970 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721706+0000", 2020-08-07T15:12:57.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2877000000000001e-05 2020-08-07T15:12:57.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.970 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:57.970 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721769+0000", 2020-08-07T15:12:57.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7355999999999999e-05 2020-08-07T15:12:57.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:57.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:57.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:57.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721817+0000", 2020-08-07T15:12:58.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28378579100000001 2020-08-07T15:12:58.004 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:58.004 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:58.005 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:58.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.005 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:18608 6.4 e240/230 ondisk, result = 0)", 2020-08-07T15:12:58.005 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.443879+0000", 2020-08-07T15:12:58.006 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 32.401620002000001, 2020-08-07T15:12:58.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.277941838, 2020-08-07T15:12:58.006 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:58.006 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:12:58.006 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:58.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:58.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443879+0000", 2020-08-07T15:12:58.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:58.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:58.026 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443879+0000", 2020-08-07T15:12:58.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3692000000000002e-05 2020-08-07T15:12:58.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:58.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443913+0000", 2020-08-07T15:12:58.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.02e-06 2020-08-07T15:12:58.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:58.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443919+0000", 2020-08-07T15:12:58.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27734076499999999 2020-08-07T15:12:58.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:58.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721260+0000", 2020-08-07T15:12:58.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.78e-06 2020-08-07T15:12:58.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:58.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721264+0000", 2020-08-07T15:12:58.047 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00042663999999999999 2020-08-07T15:12:58.047 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.047 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:58.047 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721691+0000", 2020-08-07T15:12:58.048 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1509999999999998e-06 2020-08-07T15:12:58.048 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.048 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.048 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:58.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721698+0000", 2020-08-07T15:12:58.059 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012279000000000001 2020-08-07T15:12:58.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:58.060 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721821+0000", 2020-08-07T15:12:58.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.277941838 2020-08-07T15:12:58.061 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:58.061 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:58.061 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:58.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.062 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:29256 6.1 6:8599f289:::benchmark_data_smithi074_63843_object3656:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e244)", 2020-08-07T15:12:58.062 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:31.539560+0000", 2020-08-07T15:12:58.062 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 25.305939499000001, 2020-08-07T15:12:58.062 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28482166599999997, 2020-08-07T15:12:58.062 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:58.063 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:58.063 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:58.063 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:58.063 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:58.063 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 29256 2020-08-07T15:12:58.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.064 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:58.064 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:58.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.539560+0000", 2020-08-07T15:12:58.064 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:58.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.130 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:58.130 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.539560+0000", 2020-08-07T15:12:58.130 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5965e-05 2020-08-07T15:12:58.131 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.131 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.131 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:58.131 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.539586+0000", 2020-08-07T15:12:58.131 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1719999999999999e-06 2020-08-07T15:12:58.132 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.132 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.132 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:58.132 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.539589+0000", 2020-08-07T15:12:58.132 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9700000000000005e-06 2020-08-07T15:12:58.132 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.133 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.133 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:58.133 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.539598+0000", 2020-08-07T15:12:58.133 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9999999999999998e-06 2020-08-07T15:12:58.133 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.134 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.134 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:58.134 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.539602+0000", 2020-08-07T15:12:58.134 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011500200000000001 2020-08-07T15:12:58.135 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.135 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.135 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:58.135 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.540752+0000", 2020-08-07T15:12:58.135 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000107188 2020-08-07T15:12:58.135 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.136 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.136 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:58.136 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.540859+0000", 2020-08-07T15:12:58.136 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011065600000000001 2020-08-07T15:12:58.136 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.137 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.137 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:12:58.137 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.540970+0000", 2020-08-07T15:12:58.137 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0018553440000000001 2020-08-07T15:12:58.164 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:58.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.542825+0000", 2020-08-07T15:12:58.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.048e-06 2020-08-07T15:12:58.165 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.165 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.165 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:58.166 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.542826+0000", 2020-08-07T15:12:58.166 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2224000000000005e-05 2020-08-07T15:12:58.166 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.166 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.166 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:58.167 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.542898+0000", 2020-08-07T15:12:58.167 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.029129893 2020-08-07T15:12:58.167 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.167 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.167 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:58.168 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.572028+0000", 2020-08-07T15:12:58.168 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.252307642 2020-08-07T15:12:58.168 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.168 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.168 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:58.169 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.824336+0000", 2020-08-07T15:12:58.169 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9899999999999999e-07 2020-08-07T15:12:58.169 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.169 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.169 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:58.171 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.824337+0000", 2020-08-07T15:12:58.171 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4644999999999997e-05 2020-08-07T15:12:58.171 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.171 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.172 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:58.172 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.824381+0000", 2020-08-07T15:12:58.172 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28482166599999997 2020-08-07T15:12:58.172 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:58.172 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:58.173 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:58.173 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.173 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.173 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:32560 6.e 6:77482cb7:::benchmark_data_smithi074_63843_object4069:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e246)", 2020-08-07T15:12:58.173 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:33.538197+0000", 2020-08-07T15:12:58.174 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 23.307302225000001, 2020-08-07T15:12:58.186 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28301590900000001, 2020-08-07T15:12:58.186 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:58.186 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:58.187 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:58.187 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:58.187 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:58.187 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 32560 2020-08-07T15:12:58.187 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.188 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:58.188 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.188 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:58.188 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.538197+0000", 2020-08-07T15:12:58.188 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:58.189 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.189 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.189 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:58.189 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.538197+0000", 2020-08-07T15:12:58.189 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6372000000000003e-05 2020-08-07T15:12:58.190 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.190 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.190 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:58.190 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.538243+0000", 2020-08-07T15:12:58.190 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9100000000000004e-06 2020-08-07T15:12:58.191 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.191 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.191 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:58.191 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.538248+0000", 2020-08-07T15:12:58.191 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0688e-05 2020-08-07T15:12:58.192 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.192 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.192 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:58.192 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.538259+0000", 2020-08-07T15:12:58.192 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0649999999999998e-06 2020-08-07T15:12:58.192 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.193 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.193 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:58.193 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.538264+0000", 2020-08-07T15:12:58.193 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003496645 2020-08-07T15:12:58.193 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.194 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.195 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:58.195 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.541760+0000", 2020-08-07T15:12:58.195 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019299399999999999 2020-08-07T15:12:58.195 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.195 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.196 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:58.196 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.541953+0000", 2020-08-07T15:12:58.199 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018635200000000001 2020-08-07T15:12:58.199 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.199 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.200 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:12:58.200 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.542140+0000", 2020-08-07T15:12:58.200 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001795608 2020-08-07T15:12:58.200 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.206 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.207 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:58.207 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.543935+0000", 2020-08-07T15:12:58.207 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2399999999999998e-07 2020-08-07T15:12:58.207 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.207 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.208 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:58.208 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.543936+0000", 2020-08-07T15:12:58.208 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7708999999999999e-05 2020-08-07T15:12:58.208 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.208 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.209 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:58.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.543994+0000", 2020-08-07T15:12:58.209 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032640360000000001 2020-08-07T15:12:58.210 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.211 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.211 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:58.211 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.547258+0000", 2020-08-07T15:12:58.211 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27388560899999997 2020-08-07T15:12:58.211 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.212 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.212 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:58.212 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.821143+0000", 2020-08-07T15:12:58.212 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0410000000000001e-06 2020-08-07T15:12:58.212 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.213 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.213 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:58.213 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.821144+0000", 2020-08-07T15:12:58.213 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8355999999999994e-05 2020-08-07T15:12:58.213 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.213 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.254 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:58.254 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.821213+0000", 2020-08-07T15:12:58.254 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28301590900000001 2020-08-07T15:12:58.255 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:58.255 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:58.255 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:58.255 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.255 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.256 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:48584 6.e 6:72419a6c:::benchmark_data_smithi074_63843_object6072:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e256)", 2020-08-07T15:12:58.256 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:43.685874+0000", 2020-08-07T15:12:58.256 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 13.159625208, 2020-08-07T15:12:58.256 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.336964296, 2020-08-07T15:12:58.256 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:58.257 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:58.257 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:58.257 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:58.257 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:58.257 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48584 2020-08-07T15:12:58.268 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.269 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:58.269 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.269 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:58.269 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.685874+0000", 2020-08-07T15:12:58.269 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:58.270 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.270 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.270 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:58.270 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.685874+0000", 2020-08-07T15:12:58.270 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9709000000000006e-05 2020-08-07T15:12:58.271 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.271 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:58.271 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.685944+0000", 2020-08-07T15:12:58.271 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7719999999999994e-06 2020-08-07T15:12:58.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:58.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.685951+0000", 2020-08-07T15:12:58.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5424e-05 2020-08-07T15:12:58.273 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.273 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.282 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:58.283 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.685967+0000", 2020-08-07T15:12:58.283 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3549999999999999e-06 2020-08-07T15:12:58.283 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.283 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.284 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:58.284 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.685974+0000", 2020-08-07T15:12:58.284 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2768000000000001e-05 2020-08-07T15:12:58.284 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.284 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.285 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:58.285 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.686017+0000", 2020-08-07T15:12:58.285 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6916999999999994e-05 2020-08-07T15:12:58.285 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.285 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.303 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:58.303 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.686114+0000", 2020-08-07T15:12:58.304 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018684600000000001 2020-08-07T15:12:58.304 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.304 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.304 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:12:58.304 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.686301+0000", 2020-08-07T15:12:58.305 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0050644929999999998 2020-08-07T15:12:58.305 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.305 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.305 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:58.305 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.691365+0000", 2020-08-07T15:12:58.306 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.142e-06 2020-08-07T15:12:58.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.308 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:58.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.691366+0000", 2020-08-07T15:12:58.308 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2626999999999994e-05 2020-08-07T15:12:58.308 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.309 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.309 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:58.309 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.691439+0000", 2020-08-07T15:12:58.309 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0093850860000000008 2020-08-07T15:12:58.309 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.310 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.310 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:58.310 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.700824+0000", 2020-08-07T15:12:58.310 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32193047299999999 2020-08-07T15:12:58.310 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.310 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.311 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:58.311 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022754+0000", 2020-08-07T15:12:58.311 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.147e-06 2020-08-07T15:12:58.311 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.311 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.312 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:58.312 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022756+0000", 2020-08-07T15:12:58.312 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2536999999999999e-05 2020-08-07T15:12:58.312 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.312 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.313 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:58.313 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022838+0000", 2020-08-07T15:12:58.313 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.336964296 2020-08-07T15:12:58.313 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:58.313 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:58.326 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:58.326 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.326 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.326 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:48584 6.e e256/230 ondisk, result = 0)", 2020-08-07T15:12:58.327 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:43.688495+0000", 2020-08-07T15:12:58.327 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 13.157003563, 2020-08-07T15:12:58.327 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33435100299999998, 2020-08-07T15:12:58.327 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:58.327 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:12:58.328 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:58.328 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.328 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:58.328 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.688495+0000", 2020-08-07T15:12:58.328 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:58.329 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.367 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:58.367 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.688495+0000", 2020-08-07T15:12:58.367 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2947999999999997e-05 2020-08-07T15:12:58.367 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.368 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.368 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:58.368 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.688528+0000", 2020-08-07T15:12:58.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4979999999999997e-06 2020-08-07T15:12:58.368 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.369 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:58.369 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.688534+0000", 2020-08-07T15:12:58.369 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33414495300000002 2020-08-07T15:12:58.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:58.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022679+0000", 2020-08-07T15:12:58.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6669999999999999e-06 2020-08-07T15:12:58.379 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:58.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022686+0000", 2020-08-07T15:12:58.380 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1094999999999998e-05 2020-08-07T15:12:58.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:58.380 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022737+0000", 2020-08-07T15:12:58.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2629999999999992e-06 2020-08-07T15:12:58.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:58.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022746+0000", 2020-08-07T15:12:58.395 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000100579 2020-08-07T15:12:58.395 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.395 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.395 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:58.395 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022846+0000", 2020-08-07T15:12:58.396 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33435100299999998 2020-08-07T15:12:58.396 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:58.396 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:58.396 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:58.396 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.397 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.397 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:59216 6.e 6:73f691fb:::benchmark_data_smithi074_63843_object7401:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e262)", 2020-08-07T15:12:58.397 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:50.536526+0000", 2020-08-07T15:12:58.397 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.3089731660000004, 2020-08-07T15:12:58.411 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26535118800000002, 2020-08-07T15:12:58.412 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:58.412 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:58.412 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:58.412 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:58.412 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:58.413 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 59216 2020-08-07T15:12:58.413 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.413 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:58.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.413 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:58.414 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.536526+0000", 2020-08-07T15:12:58.414 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:58.414 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.414 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.414 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:58.415 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.536526+0000", 2020-08-07T15:12:58.415 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9113999999999998e-05 2020-08-07T15:12:58.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.415 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:58.416 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.536575+0000", 2020-08-07T15:12:58.416 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4090000000000002e-06 2020-08-07T15:12:58.416 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.416 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.416 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:58.416 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.536580+0000", 2020-08-07T15:12:58.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0973e-05 2020-08-07T15:12:58.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.426 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.426 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:58.426 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.536591+0000", 2020-08-07T15:12:58.426 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4469999999999999e-06 2020-08-07T15:12:58.426 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.427 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:58.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.536596+0000", 2020-08-07T15:12:58.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028173590000000002 2020-08-07T15:12:58.444 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.445 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:58.445 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.539413+0000", 2020-08-07T15:12:58.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000103162 2020-08-07T15:12:58.445 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.445 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.446 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:58.446 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.539516+0000", 2020-08-07T15:12:58.446 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000109373 2020-08-07T15:12:58.446 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.446 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.454 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:12:58.454 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.539626+0000", 2020-08-07T15:12:58.454 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010865110000000001 2020-08-07T15:12:58.454 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:58.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.540712+0000", 2020-08-07T15:12:58.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0699999999999996e-07 2020-08-07T15:12:58.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.473 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.473 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:58.474 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.540713+0000", 2020-08-07T15:12:58.474 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5903999999999999e-05 2020-08-07T15:12:58.474 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.474 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.474 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:58.475 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.540769+0000", 2020-08-07T15:12:58.475 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.010000762999999999 2020-08-07T15:12:58.475 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.475 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.475 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:58.476 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.550770+0000", 2020-08-07T15:12:58.476 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25103749800000003 2020-08-07T15:12:58.476 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.476 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.476 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:58.477 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.801807+0000", 2020-08-07T15:12:58.477 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1399999999999996e-07 2020-08-07T15:12:58.477 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.477 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.477 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:58.479 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.801808+0000", 2020-08-07T15:12:58.480 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9153999999999996e-05 2020-08-07T15:12:58.480 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.481 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.481 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:58.481 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.801877+0000", 2020-08-07T15:12:58.481 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26535118800000002 2020-08-07T15:12:58.481 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:58.482 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:58.482 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:58.482 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.482 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.482 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:59624 6.e 6:794abcd3:::benchmark_data_smithi074_63843_object7452:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e262)", 2020-08-07T15:12:58.482 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:50.800774+0000", 2020-08-07T15:12:58.483 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 6.0447255049999997, 2020-08-07T15:12:58.483 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26607003600000001, 2020-08-07T15:12:58.483 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:58.483 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:58.483 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:58.484 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:58.484 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:58.484 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 59624 2020-08-07T15:12:58.484 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.484 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:58.485 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.485 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:58.485 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.800774+0000", 2020-08-07T15:12:58.485 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:58.485 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.508 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.508 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:58.508 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.800774+0000", 2020-08-07T15:12:58.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4698e-05 2020-08-07T15:12:58.509 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.509 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.509 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:58.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.800798+0000", 2020-08-07T15:12:58.510 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0299999999999998e-06 2020-08-07T15:12:58.510 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.510 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:58.510 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.800801+0000", 2020-08-07T15:12:58.511 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5209999999999999e-06 2020-08-07T15:12:58.511 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.533 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:58.534 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.800808+0000", 2020-08-07T15:12:58.534 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3340000000000002e-06 2020-08-07T15:12:58.534 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.534 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:58.535 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.800811+0000", 2020-08-07T15:12:58.535 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028861410000000001 2020-08-07T15:12:58.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:58.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.803697+0000", 2020-08-07T15:12:58.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6056000000000004e-05 2020-08-07T15:12:58.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.536 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.537 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:58.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.803783+0000", 2020-08-07T15:12:58.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00028963200000000001 2020-08-07T15:12:58.537 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.537 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.538 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:12:58.538 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.804073+0000", 2020-08-07T15:12:58.538 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033154199999999999 2020-08-07T15:12:58.538 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.539 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.539 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:58.550 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.807388+0000", 2020-08-07T15:12:58.550 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2899999999999999e-07 2020-08-07T15:12:58.551 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.551 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.551 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:58.551 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.807389+0000", 2020-08-07T15:12:58.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4131e-05 2020-08-07T15:12:58.552 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.552 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:58.552 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.807453+0000", 2020-08-07T15:12:58.552 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00090008099999999995 2020-08-07T15:12:58.553 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.553 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.554 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:58.554 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:50.808353+0000", 2020-08-07T15:12:58.554 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.25841253800000002 2020-08-07T15:12:58.554 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.555 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.555 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:58.555 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:51.066766+0000", 2020-08-07T15:12:58.555 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.421e-06 2020-08-07T15:12:58.555 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:58.556 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:51.066767+0000", 2020-08-07T15:12:58.556 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6604e-05 2020-08-07T15:12:58.557 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.558 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.558 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:58.558 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:51.066844+0000", 2020-08-07T15:12:58.558 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26607003600000001 2020-08-07T15:12:58.559 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:58.559 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:58.559 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:58.559 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.559 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.559 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:61896 6.e 6:75870b8d:::benchmark_data_smithi074_63843_object7736:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e263)", 2020-08-07T15:12:58.560 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:52.341370+0000", 2020-08-07T15:12:58.560 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 4.5041294189999999, 2020-08-07T15:12:58.560 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.281931072, 2020-08-07T15:12:58.560 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:12:58.560 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:12:58.561 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:12:58.561 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:12:58.561 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:12:58.561 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 61896 2020-08-07T15:12:58.567 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.568 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:12:58.568 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.568 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:12:58.568 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.341370+0000", 2020-08-07T15:12:58.568 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:12:58.569 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.578 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.578 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:12:58.578 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.341370+0000", 2020-08-07T15:12:58.579 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7135000000000003e-05 2020-08-07T15:12:58.579 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.579 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.579 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:12:58.579 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.341407+0000", 2020-08-07T15:12:58.580 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0359999999999997e-06 2020-08-07T15:12:58.580 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.580 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.580 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:12:58.580 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.341412+0000", 2020-08-07T15:12:58.581 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0572e-05 2020-08-07T15:12:58.581 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.581 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.581 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:12:58.581 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.341422+0000", 2020-08-07T15:12:58.598 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.279e-06 2020-08-07T15:12:58.598 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.599 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.599 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:12:58.599 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.341427+0000", 2020-08-07T15:12:58.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024648579999999999 2020-08-07T15:12:58.600 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.600 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.600 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:12:58.600 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.343891+0000", 2020-08-07T15:12:58.601 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1310000000000005e-05 2020-08-07T15:12:58.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.601 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:12:58.601 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.343983+0000", 2020-08-07T15:12:58.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000175744 2020-08-07T15:12:58.602 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.602 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.602 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:12:58.602 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.344159+0000", 2020-08-07T15:12:58.603 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005415202 2020-08-07T15:12:58.603 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.603 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:12:58.603 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.349574+0000", 2020-08-07T15:12:58.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8599999999999998e-07 2020-08-07T15:12:58.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:12:58.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.349574+0000", 2020-08-07T15:12:58.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9826999999999994e-05 2020-08-07T15:12:58.605 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.605 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.605 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:12:58.605 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.349644+0000", 2020-08-07T15:12:58.605 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.05151385 2020-08-07T15:12:58.650 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.650 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.650 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:58.651 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.401158+0000", 2020-08-07T15:12:58.651 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22207370600000001 2020-08-07T15:12:58.651 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.651 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.651 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:12:58.652 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.623232+0000", 2020-08-07T15:12:58.652 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1999999999999998e-07 2020-08-07T15:12:58.652 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.653 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.653 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:12:58.653 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.623233+0000", 2020-08-07T15:12:58.654 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7947000000000005e-05 2020-08-07T15:12:58.654 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:12:58.654 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:12:58.654 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:12:58.654 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.623301+0000", 2020-08-07T15:12:58.655 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.281931072 2020-08-07T15:12:58.655 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:58.655 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:58.655 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:58.655 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:12:58.655 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:12:58.656 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:58.657 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:12:58.657 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:12:58.659 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 45 16 67304 67288 11.6807 0.015625 0.745903 0.0103543 2020-08-07T15:12:58.659 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 46 16 69166 69150 11.7429 14.5469 0.00236503 0.0106408 2020-08-07T15:12:58.660 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:58.656+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:58.661 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:12:58.758 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:58.757+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:58.863 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:12:58.861+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:12:58.913 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:12:58.913 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:12:58.913 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:12:58.913 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:12:58.939 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:12:58.971 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:58.964+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:58.972 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:12:59.001 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:12:59.033 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:12:59.062 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:59.060+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:12:59.108 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:12:59.166 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:59.179 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:59.181 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:12:59.182 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:12:59.184 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:59.179+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:59.198 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:12:59.239 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:12:59.269 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 47 16 70636 70620 11.7374 11.4844 0.00426563 0.0106441 2020-08-07T15:12:59.279 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:12:59.280 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:12:59.280 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:12:59.280 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:12:59.281 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:12:59.281 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:12:59.281 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:12:59.287 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:12:59.287 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:12:59.305 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:59.302+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:59.403 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:59.404 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:59.415 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:12:59.404+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:12:59.417 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:12:59.516 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:59.511+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:59.591 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:12:59.646 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:59.644+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:59.652 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:59.653 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:59.674 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:12:59.716 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:12:59.715+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:12:59.819 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:12:59.817+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:12:59.898 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:12:59.899 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:12:59.911 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:12:59.954 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:12:59.950+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:00.025 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:00.020+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:00.131 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:00.122+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:00.132 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:00.132 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:00.142 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:13:00.276 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 48 16 72182 72166 11.7445 12.0781 0.0376987 0.0106221 2020-08-07T15:13:00.314 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:00.314 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:00.319 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:00.226+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:00.327 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:13:00.332 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:00.329+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:00.434 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:00.431+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:00.531 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:00.531 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:00.543 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:00.535+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:00.547 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:13:00.573 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2020-08-07T15:13:00.574 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:13:00.574 INFO:tasks.thrashosds.thrasher:Setting osd 7 primary_affinity to 0.000000 2020-08-07T15:13:00.575 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd primary-affinity 7 0 2020-08-07T15:13:00.636 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:00.632+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:00.734 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:00.733+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:00.786 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:00.786 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:00.800 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:13:00.846 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:00.845+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:00.940 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:00.938+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:01.046 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:01.040+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:01.158 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:01.149+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:01.256 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:01.250+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:01.270 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 49 16 73723 73707 11.7505 12.0391 0.0610103 0.0106331 2020-08-07T15:13:01.306 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:13:01.358 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:01.353+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:01.445 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:01.446 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:01.454 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:01.451+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:01.455 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:13:01.532 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:13:01.552 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:01.551+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:01.574 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:13:01.583 INFO:teuthology.orchestra.run.smithi074.stderr:set osd.7 primary-affinity to 0 (802) 2020-08-07T15:13:01.606 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:13:01.610 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:13:01.638 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:13:01.649 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:01.651 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:01.658 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:01.655+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:01.659 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:13:01.662 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:13:01.690 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:13:01.714 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:13:01.747 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:13:01.750 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:13:01.751 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:13:01.751 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:13:01.752 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:13:01.752 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:13:01.753 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:13:01.753 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:13:01.754 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:13:01.764 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:01.762+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:01.823 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:01.824 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:13:01.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:13:01.824 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:13:01.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:01.825 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "MOSDPGPushReply(3.1 62/60 [PushReplyOp(3:8329f37e:::benchmark_data_smithi074_35447_object11915:head)])", 2020-08-07T15:13:01.825 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:13:01.825 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 310.96089356599998, 2020-08-07T15:13:01.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996, 2020-08-07T15:13:01.825 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:01.850 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:01.850 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:01.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:01.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:01.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:13:01.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:01.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:01.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:01.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:01.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:13:01.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5372000000000001e-05 2020-08-07T15:13:01.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:01.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:01.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:01.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858416+0000", 2020-08-07T15:13:01.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6899999999999998e-06 2020-08-07T15:13:01.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:01.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:01.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:01.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858419+0000", 2020-08-07T15:13:01.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63907904299999996 2020-08-07T15:13:01.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:01.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:01.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:01.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497498+0000", 2020-08-07T15:13:01.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3332999999999999e-05 2020-08-07T15:13:01.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:01.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:01.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:01.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497512+0000", 2020-08-07T15:13:01.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000450875 2020-08-07T15:13:01.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:01.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:01.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:01.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497962+0000", 2020-08-07T15:13:01.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000267779 2020-08-07T15:13:01.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:01.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:01.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:01.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.498230+0000", 2020-08-07T15:13:01.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996 2020-08-07T15:13:01.916 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:01.916 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:01.916 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:01.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:01.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:01.917 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:145792 3.1 3:881e0420:::benchmark_data_smithi074_35447_object16183:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:13:01.917 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:13:01.917 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 287.15248768499998, 2020-08-07T15:13:01.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997, 2020-08-07T15:13:01.917 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:01.942 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:01.942 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:01.942 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:13:01.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:13:01.943 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 145792 2020-08-07T15:13:01.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:01.943 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:01.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:01.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:01.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:13:01.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:01.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:01.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:01.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:01.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:13:01.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7489000000000001e-05 2020-08-07T15:13:01.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:01.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:01.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:01.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666824+0000", 2020-08-07T15:13:01.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0169999999999998e-06 2020-08-07T15:13:01.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:01.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:01.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:01.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666828+0000", 2020-08-07T15:13:01.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9829999999999996e-06 2020-08-07T15:13:01.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:01.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:01.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:01.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666836+0000", 2020-08-07T15:13:01.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.481e-06 2020-08-07T15:13:02.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:02.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666840+0000", 2020-08-07T15:13:02.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5299000000000001e-05 2020-08-07T15:13:02.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666885+0000", 2020-08-07T15:13:02.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014456500000000001 2020-08-07T15:13:02.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:02.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667030+0000", 2020-08-07T15:13:02.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017677799999999999 2020-08-07T15:13:02.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,5", 2020-08-07T15:13:02.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667207+0000", 2020-08-07T15:13:02.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004149649 2020-08-07T15:13:02.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:02.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671356+0000", 2020-08-07T15:13:02.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019339649999999999 2020-08-07T15:13:02.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:02.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673290+0000", 2020-08-07T15:13:02.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73346620500000004 2020-08-07T15:13:02.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:02.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406756+0000", 2020-08-07T15:13:02.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4700000000000001e-06 2020-08-07T15:13:02.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:02.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406759+0000", 2020-08-07T15:13:02.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6258999999999998e-05 2020-08-07T15:13:02.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:02.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406815+0000", 2020-08-07T15:13:02.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0126000000000002e-05 2020-08-07T15:13:02.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:02.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406855+0000", 2020-08-07T15:13:02.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997 2020-08-07T15:13:02.127 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:02.127 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:02.127 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:02.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.128 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:145792 3.1 e75/67 ondisk, result = 0)", 2020-08-07T15:13:02.128 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:13:02.129 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 287.14810279699998, 2020-08-07T15:13:02.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003, 2020-08-07T15:13:02.129 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:02.129 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:02.130 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:02.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:02.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:13:02.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:02.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:02.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:13:02.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2830999999999998e-05 2020-08-07T15:13:02.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:02.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671224+0000", 2020-08-07T15:13:02.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:13:02.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:02.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671229+0000", 2020-08-07T15:13:02.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.735414864 2020-08-07T15:13:02.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:02.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406644+0000", 2020-08-07T15:13:02.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1250000000000004e-06 2020-08-07T15:13:02.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:02.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406651+0000", 2020-08-07T15:13:02.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6567e-05 2020-08-07T15:13:02.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406737+0000", 2020-08-07T15:13:02.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0054000000000001e-05 2020-08-07T15:13:02.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:02.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406747+0000", 2020-08-07T15:13:02.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011391999999999999 2020-08-07T15:13:02.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:02.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406861+0000", 2020-08-07T15:13:02.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003 2020-08-07T15:13:02.172 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:02.172 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:02.172 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:02.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.173 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30921 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:13:02.174 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:13:02.178 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 255.113354423, 2020-08-07T15:13:02.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001, 2020-08-07T15:13:02.207 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:02.207 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:02.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:02.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:02.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:02.208 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30921 2020-08-07T15:13:02.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.208 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:02.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:02.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:13:02.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:02.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:02.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:13:02.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6954999999999998e-05 2020-08-07T15:13:02.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:02.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706016+0000", 2020-08-07T15:13:02.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0229999999999992e-06 2020-08-07T15:13:02.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:02.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706024+0000", 2020-08-07T15:13:02.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.432e-05 2020-08-07T15:13:02.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:02.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706039+0000", 2020-08-07T15:13:02.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8490000000000006e-06 2020-08-07T15:13:02.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:02.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706046+0000", 2020-08-07T15:13:02.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6837999999999999e-05 2020-08-07T15:13:02.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706113+0000", 2020-08-07T15:13:02.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3553000000000001e-05 2020-08-07T15:13:02.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:02.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706127+0000", 2020-08-07T15:13:02.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77277201699999998 2020-08-07T15:13:02.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.478899+0000", 2020-08-07T15:13:02.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093978599999999996 2020-08-07T15:13:02.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:13:02.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479839+0000", 2020-08-07T15:13:02.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010376029 2020-08-07T15:13:02.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490215+0000", 2020-08-07T15:13:02.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000190657 2020-08-07T15:13:02.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:02.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490405+0000", 2020-08-07T15:13:02.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001 2020-08-07T15:13:02.348 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:02.349 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:02.349 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:02.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.363 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30922 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:13:02.364 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:13:02.364 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 255.11319426, 2020-08-07T15:13:02.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247, 2020-08-07T15:13:02.364 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:02.364 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:02.365 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:02.365 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:02.365 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:02.365 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30922 2020-08-07T15:13:02.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.381 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:02.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:02.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:13:02.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:02.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:02.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:13:02.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9178999999999997e-05 2020-08-07T15:13:02.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:02.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706159+0000", 2020-08-07T15:13:02.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4999999999999999e-06 2020-08-07T15:13:02.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:02.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706164+0000", 2020-08-07T15:13:02.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1792e-05 2020-08-07T15:13:02.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:02.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706176+0000", 2020-08-07T15:13:02.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.139e-06 2020-08-07T15:13:02.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:02.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706182+0000", 2020-08-07T15:13:02.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039724900000000002 2020-08-07T15:13:02.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706579+0000", 2020-08-07T15:13:02.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1191e-05 2020-08-07T15:13:02.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:02.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706591+0000", 2020-08-07T15:13:02.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329101300000003 2020-08-07T15:13:02.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479882+0000", 2020-08-07T15:13:02.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9565000000000001e-05 2020-08-07T15:13:02.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:13:02.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479971+0000", 2020-08-07T15:13:02.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010471263 2020-08-07T15:13:02.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490442+0000", 2020-08-07T15:13:02.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015035600000000001 2020-08-07T15:13:02.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:02.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490593+0000", 2020-08-07T15:13:02.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247 2020-08-07T15:13:02.451 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:02.452 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:02.452 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:02.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.452 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30923 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:13:02.453 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:13:02.453 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 255.112719199, 2020-08-07T15:13:02.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998, 2020-08-07T15:13:02.454 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:02.454 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:02.454 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:02.455 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:02.455 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:02.455 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30923 2020-08-07T15:13:02.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.459 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:02.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:02.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:13:02.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:02.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:02.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:13:02.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5531000000000001e-05 2020-08-07T15:13:02.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:02.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706630+0000", 2020-08-07T15:13:02.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7689999999999999e-06 2020-08-07T15:13:02.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:02.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706637+0000", 2020-08-07T15:13:02.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3577999999999999e-05 2020-08-07T15:13:02.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:02.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706650+0000", 2020-08-07T15:13:02.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1439999999999997e-06 2020-08-07T15:13:02.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:02.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706658+0000", 2020-08-07T15:13:02.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1663999999999999e-05 2020-08-07T15:13:02.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706709+0000", 2020-08-07T15:13:02.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0149000000000001e-05 2020-08-07T15:13:02.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:02.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706719+0000", 2020-08-07T15:13:02.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329694900000001 2020-08-07T15:13:02.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480016+0000", 2020-08-07T15:13:02.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7998999999999997e-05 2020-08-07T15:13:02.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:13:02.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480084+0000", 2020-08-07T15:13:02.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010547333000000001 2020-08-07T15:13:02.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490632+0000", 2020-08-07T15:13:02.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7439e-05 2020-08-07T15:13:02.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:02.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490729+0000", 2020-08-07T15:13:02.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998 2020-08-07T15:13:02.535 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:02.535 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:02.535 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:02.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.536 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30924 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:13:02.536 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:13:02.536 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 255.11259192700001, 2020-08-07T15:13:02.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995, 2020-08-07T15:13:02.537 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:02.537 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:02.537 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:02.537 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:02.554 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:02.555 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30924 2020-08-07T15:13:02.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.556 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:02.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:02.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:13:02.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:02.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:02.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:13:02.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9698999999999999e-05 2020-08-07T15:13:02.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:02.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706762+0000", 2020-08-07T15:13:02.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8080000000000001e-06 2020-08-07T15:13:02.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:02.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706767+0000", 2020-08-07T15:13:02.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1069e-05 2020-08-07T15:13:02.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:02.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706778+0000", 2020-08-07T15:13:02.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6969999999999998e-06 2020-08-07T15:13:02.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:02.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706784+0000", 2020-08-07T15:13:02.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8792999999999998e-05 2020-08-07T15:13:02.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706843+0000", 2020-08-07T15:13:02.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5449999999999998e-06 2020-08-07T15:13:02.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:02.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706850+0000", 2020-08-07T15:13:02.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.773278354 2020-08-07T15:13:02.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480129+0000", 2020-08-07T15:13:02.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1577999999999997e-05 2020-08-07T15:13:02.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:13:02.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480210+0000", 2020-08-07T15:13:02.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010541915000000001 2020-08-07T15:13:02.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490752+0000", 2020-08-07T15:13:02.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103937 2020-08-07T15:13:02.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:02.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490856+0000", 2020-08-07T15:13:02.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995 2020-08-07T15:13:02.618 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:02.618 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:02.619 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:02.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.619 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30925 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:13:02.619 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:13:02.620 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 255.11246815600001, 2020-08-07T15:13:02.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995, 2020-08-07T15:13:02.620 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:02.620 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:02.620 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:02.620 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:02.621 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:02.621 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30925 2020-08-07T15:13:02.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.630 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:02.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:02.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:13:02.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:02.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:02.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:13:02.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4664999999999999e-05 2020-08-07T15:13:02.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:02.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706880+0000", 2020-08-07T15:13:02.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7479999999999999e-06 2020-08-07T15:13:02.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:02.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706886+0000", 2020-08-07T15:13:02.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1838000000000001e-05 2020-08-07T15:13:02.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:02.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706898+0000", 2020-08-07T15:13:02.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0089999999999998e-06 2020-08-07T15:13:02.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:02.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706903+0000", 2020-08-07T15:13:02.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3559e-05 2020-08-07T15:13:02.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706956+0000", 2020-08-07T15:13:02.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0309000000000001e-05 2020-08-07T15:13:02.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:02.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706967+0000", 2020-08-07T15:13:02.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77328290499999996 2020-08-07T15:13:02.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480250+0000", 2020-08-07T15:13:02.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8680999999999997e-05 2020-08-07T15:13:02.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:13:02.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480298+0000", 2020-08-07T15:13:02.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01059272 2020-08-07T15:13:02.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490891+0000", 2020-08-07T15:13:02.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014580300000000001 2020-08-07T15:13:02.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:02.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491037+0000", 2020-08-07T15:13:02.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995 2020-08-07T15:13:02.676 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:02.676 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:02.676 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:02.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.677 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30926 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:13:02.677 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:13:02.678 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 255.112351287, 2020-08-07T15:13:02.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005, 2020-08-07T15:13:02.678 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:02.678 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:02.679 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:02.679 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:02.679 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:02.679 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30926 2020-08-07T15:13:02.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.680 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:02.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:02.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:13:02.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:02.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:02.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:13:02.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5223000000000003e-05 2020-08-07T15:13:02.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:02.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706998+0000", 2020-08-07T15:13:02.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4809999999999997e-06 2020-08-07T15:13:02.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:02.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707003+0000", 2020-08-07T15:13:02.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1663e-05 2020-08-07T15:13:02.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:02.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707015+0000", 2020-08-07T15:13:02.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9729999999999997e-06 2020-08-07T15:13:02.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:02.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707020+0000", 2020-08-07T15:13:02.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2565999999999999e-05 2020-08-07T15:13:02.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707062+0000", 2020-08-07T15:13:02.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8944e-05 2020-08-07T15:13:02.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:02.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707081+0000", 2020-08-07T15:13:02.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77325885000000005 2020-08-07T15:13:02.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480340+0000", 2020-08-07T15:13:02.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0321000000000006e-05 2020-08-07T15:13:02.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:13:02.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480420+0000", 2020-08-07T15:13:02.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010655958 2020-08-07T15:13:02.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491076+0000", 2020-08-07T15:13:02.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9784999999999995e-05 2020-08-07T15:13:02.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:02.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491166+0000", 2020-08-07T15:13:02.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005 2020-08-07T15:13:02.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:02.734 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:02.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:02.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.735 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30927 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:13:02.735 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:13:02.735 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 255.112233468, 2020-08-07T15:13:02.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999, 2020-08-07T15:13:02.736 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:02.736 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:02.736 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:02.736 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:02.736 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:02.737 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30927 2020-08-07T15:13:02.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.737 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:02.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:02.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:13:02.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:02.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:13:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9861999999999997e-05 2020-08-07T15:13:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707120+0000", 2020-08-07T15:13:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8969999999999996e-06 2020-08-07T15:13:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:02.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707126+0000", 2020-08-07T15:13:02.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1248e-05 2020-08-07T15:13:02.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707137+0000", 2020-08-07T15:13:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.955e-06 2020-08-07T15:13:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707142+0000", 2020-08-07T15:13:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0643999999999998e-05 2020-08-07T15:13:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707193+0000", 2020-08-07T15:13:02.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1983e-05 2020-08-07T15:13:02.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:02.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707205+0000", 2020-08-07T15:13:02.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77324914 2020-08-07T15:13:02.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480454+0000", 2020-08-07T15:13:02.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4119999999999998e-05 2020-08-07T15:13:02.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:13:02.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480498+0000", 2020-08-07T15:13:02.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010709707000000001 2020-08-07T15:13:02.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491208+0000", 2020-08-07T15:13:02.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014433800000000001 2020-08-07T15:13:02.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:02.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491352+0000", 2020-08-07T15:13:02.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999 2020-08-07T15:13:02.774 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.775 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30928 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:13:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:13:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 255.112109111, 2020-08-07T15:13:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998, 2020-08-07T15:13:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:02.776 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30928 2020-08-07T15:13:02.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.787 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:02.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:02.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:13:02.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:02.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:02.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:13:02.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6582000000000001e-05 2020-08-07T15:13:02.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:02.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707241+0000", 2020-08-07T15:13:02.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9190000000000004e-06 2020-08-07T15:13:02.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:02.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707247+0000", 2020-08-07T15:13:02.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000188822 2020-08-07T15:13:02.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707436+0000", 2020-08-07T15:13:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2940000000000002e-06 2020-08-07T15:13:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707442+0000", 2020-08-07T15:13:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7800000000000003e-05 2020-08-07T15:13:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707490+0000", 2020-08-07T15:13:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2020-08-07T15:13:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707501+0000", 2020-08-07T15:13:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77303498800000003 2020-08-07T15:13:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480536+0000", 2020-08-07T15:13:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7723000000000001e-05 2020-08-07T15:13:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:13:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480614+0000", 2020-08-07T15:13:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010812183 2020-08-07T15:13:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491426+0000", 2020-08-07T15:13:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5502999999999999e-05 2020-08-07T15:13:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491502+0000", 2020-08-07T15:13:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998 2020-08-07T15:13:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31009 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:13:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:13:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 255.08815280499999, 2020-08-07T15:13:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002, 2020-08-07T15:13:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:02.833 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31009 2020-08-07T15:13:02.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.833 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:02.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:02.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:13:02.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:02.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:02.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:13:02.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2429000000000002e-05 2020-08-07T15:13:02.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731193+0000", 2020-08-07T15:13:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9760000000000003e-06 2020-08-07T15:13:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731198+0000", 2020-08-07T15:13:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3597e-05 2020-08-07T15:13:02.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:02.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731212+0000", 2020-08-07T15:13:02.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1709999999999996e-06 2020-08-07T15:13:02.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:02.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731219+0000", 2020-08-07T15:13:02.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5402999999999997e-05 2020-08-07T15:13:02.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731285+0000", 2020-08-07T15:13:02.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3495e-05 2020-08-07T15:13:02.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:02.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731298+0000", 2020-08-07T15:13:02.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74935423599999995 2020-08-07T15:13:02.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480652+0000", 2020-08-07T15:13:02.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000137806 2020-08-07T15:13:02.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:02.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480790+0000", 2020-08-07T15:13:02.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022272399999999999 2020-08-07T15:13:02.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:13:02.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481013+0000", 2020-08-07T15:13:02.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003223381 2020-08-07T15:13:02.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:02.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484236+0000", 2020-08-07T15:13:02.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1370000000000001e-06 2020-08-07T15:13:02.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:02.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484237+0000", 2020-08-07T15:13:02.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6288e-05 2020-08-07T15:13:02.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:02.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484294+0000", 2020-08-07T15:13:02.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.007260402 2020-08-07T15:13:02.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:02.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491554+0000", 2020-08-07T15:13:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7358000000000003e-05 2020-08-07T15:13:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491601+0000", 2020-08-07T15:13:02.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1100000000000004e-07 2020-08-07T15:13:02.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491602+0000", 2020-08-07T15:13:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013436500000000001 2020-08-07T15:13:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491737+0000", 2020-08-07T15:13:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002 2020-08-07T15:13:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:02.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.896 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31010 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:13:02.896 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:13:02.896 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 255.088026851, 2020-08-07T15:13:02.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001, 2020-08-07T15:13:02.897 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:02.897 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:02.897 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:02.897 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:02.897 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:02.900 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31010 2020-08-07T15:13:02.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.906 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:02.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:02.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:13:02.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:02.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:02.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:13:02.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2398000000000002e-05 2020-08-07T15:13:02.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:02.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731319+0000", 2020-08-07T15:13:02.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7100000000000004e-06 2020-08-07T15:13:02.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:02.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731325+0000", 2020-08-07T15:13:02.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4328000000000001e-05 2020-08-07T15:13:02.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:02.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731339+0000", 2020-08-07T15:13:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3310000000000003e-06 2020-08-07T15:13:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731345+0000", 2020-08-07T15:13:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8952999999999998e-05 2020-08-07T15:13:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731384+0000", 2020-08-07T15:13:02.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0036e-05 2020-08-07T15:13:02.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:02.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731394+0000", 2020-08-07T15:13:02.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75027681300000004 2020-08-07T15:13:02.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:02.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481670+0000", 2020-08-07T15:13:02.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016555899999999999 2020-08-07T15:13:02.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:02.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481836+0000", 2020-08-07T15:13:02.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028760999999999999 2020-08-07T15:13:02.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:13:02.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482124+0000", 2020-08-07T15:13:02.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038342680000000001 2020-08-07T15:13:02.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:02.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485958+0000", 2020-08-07T15:13:02.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2559999999999999e-06 2020-08-07T15:13:02.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:02.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485959+0000", 2020-08-07T15:13:02.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3758999999999995e-05 2020-08-07T15:13:02.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:02.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:02.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:02.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486043+0000", 2020-08-07T15:13:03.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057590419999999998 2020-08-07T15:13:03.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:03.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491802+0000", 2020-08-07T15:13:03.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5757000000000001e-05 2020-08-07T15:13:03.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:03.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:13:03.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9299999999999997e-07 2020-08-07T15:13:03.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:03.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:13:03.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012496999999999999 2020-08-07T15:13:03.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:03.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491993+0000", 2020-08-07T15:13:03.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001 2020-08-07T15:13:03.021 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:03.021 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:03.021 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:03.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.022 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31011 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:13:03.022 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:13:03.023 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 255.08791064600001, 2020-08-07T15:13:03.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997, 2020-08-07T15:13:03.023 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:03.023 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:03.023 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:03.024 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:03.024 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:03.024 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31011 2020-08-07T15:13:03.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.024 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:03.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:03.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:13:03.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:03.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:03.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:13:03.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9539999999999997e-05 2020-08-07T15:13:03.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:03.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731433+0000", 2020-08-07T15:13:03.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9209999999999997e-06 2020-08-07T15:13:03.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:03.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731439+0000", 2020-08-07T15:13:03.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4246999999999999e-05 2020-08-07T15:13:03.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:03.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731453+0000", 2020-08-07T15:13:03.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8130000000000003e-06 2020-08-07T15:13:03.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:03.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731458+0000", 2020-08-07T15:13:03.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6668000000000001e-05 2020-08-07T15:13:03.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:03.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731504+0000", 2020-08-07T15:13:03.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1112e-05 2020-08-07T15:13:03.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:03.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731515+0000", 2020-08-07T15:13:03.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75121332100000004 2020-08-07T15:13:03.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:03.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482729+0000", 2020-08-07T15:13:03.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1810000000000004e-05 2020-08-07T15:13:03.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:03.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482811+0000", 2020-08-07T15:13:03.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018001399999999999 2020-08-07T15:13:03.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:13:03.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482991+0000", 2020-08-07T15:13:03.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030701140000000001 2020-08-07T15:13:03.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:03.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486061+0000", 2020-08-07T15:13:03.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4499999999999996e-07 2020-08-07T15:13:03.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:03.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486062+0000", 2020-08-07T15:13:03.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5366999999999998e-05 2020-08-07T15:13:03.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:03.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486147+0000", 2020-08-07T15:13:03.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056872779999999996 2020-08-07T15:13:03.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:03.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491834+0000", 2020-08-07T15:13:03.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024645100000000002 2020-08-07T15:13:03.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:03.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:13:03.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1200000000000003e-07 2020-08-07T15:13:03.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:03.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:13:03.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1451000000000002e-05 2020-08-07T15:13:03.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:03.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492173+0000", 2020-08-07T15:13:03.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997 2020-08-07T15:13:03.140 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:03.140 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:03.140 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:03.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.141 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31012 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:13:03.141 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:13:03.141 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 255.08725652699999, 2020-08-07T15:13:03.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003, 2020-08-07T15:13:03.142 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:03.142 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:03.142 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:03.142 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:03.152 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:03.152 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31012 2020-08-07T15:13:03.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.153 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:03.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:03.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:13:03.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:03.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:03.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:13:03.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6344999999999997e-05 2020-08-07T15:13:03.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:03.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732094+0000", 2020-08-07T15:13:03.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2920000000000001e-06 2020-08-07T15:13:03.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:03.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732100+0000", 2020-08-07T15:13:03.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.189e-05 2020-08-07T15:13:03.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:03.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732112+0000", 2020-08-07T15:13:03.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3720000000000001e-06 2020-08-07T15:13:03.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:03.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732118+0000", 2020-08-07T15:13:03.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2514000000000004e-05 2020-08-07T15:13:03.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:03.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732191+0000", 2020-08-07T15:13:03.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4610000000000001e-05 2020-08-07T15:13:03.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:03.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732205+0000", 2020-08-07T15:13:03.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75132974100000005 2020-08-07T15:13:03.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:03.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483535+0000", 2020-08-07T15:13:03.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9185999999999996e-05 2020-08-07T15:13:03.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:03.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483624+0000", 2020-08-07T15:13:03.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015683 2020-08-07T15:13:03.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:13:03.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483781+0000", 2020-08-07T15:13:03.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046202889999999996 2020-08-07T15:13:03.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:03.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488401+0000", 2020-08-07T15:13:03.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6700000000000002e-07 2020-08-07T15:13:03.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:03.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488402+0000", 2020-08-07T15:13:03.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1487000000000001e-05 2020-08-07T15:13:03.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:03.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488484+0000", 2020-08-07T15:13:03.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035520830000000002 2020-08-07T15:13:03.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:03.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492036+0000", 2020-08-07T15:13:03.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029266600000000002 2020-08-07T15:13:03.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:03.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492328+0000", 2020-08-07T15:13:03.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5099999999999998e-07 2020-08-07T15:13:03.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:03.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492329+0000", 2020-08-07T15:13:03.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000201299 2020-08-07T15:13:03.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:03.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492531+0000", 2020-08-07T15:13:03.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003 2020-08-07T15:13:03.200 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:03.201 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:03.201 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:03.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.202 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31013 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:13:03.202 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:13:03.202 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 255.08704489600001, 2020-08-07T15:13:03.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996, 2020-08-07T15:13:03.203 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:03.203 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:03.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:03.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:03.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:03.204 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31013 2020-08-07T15:13:03.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.204 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:03.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:03.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:13:03.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:03.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:03.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:13:03.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8949000000000002e-05 2020-08-07T15:13:03.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:03.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732298+0000", 2020-08-07T15:13:03.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1399999999999997e-06 2020-08-07T15:13:03.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:03.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732304+0000", 2020-08-07T15:13:03.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.154e-05 2020-08-07T15:13:03.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:03.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732315+0000", 2020-08-07T15:13:03.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1319999999999999e-06 2020-08-07T15:13:03.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:03.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732322+0000", 2020-08-07T15:13:03.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7431e-05 2020-08-07T15:13:03.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:03.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732369+0000", 2020-08-07T15:13:03.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0621999999999999e-05 2020-08-07T15:13:03.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:03.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732380+0000", 2020-08-07T15:13:03.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75196845999999995 2020-08-07T15:13:03.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:03.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484348+0000", 2020-08-07T15:13:03.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000123008 2020-08-07T15:13:03.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:03.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484471+0000", 2020-08-07T15:13:03.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017572700000000001 2020-08-07T15:13:03.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:13:03.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484647+0000", 2020-08-07T15:13:03.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038557650000000001 2020-08-07T15:13:03.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:03.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:13:03.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2020-08-07T15:13:03.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:03.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:13:03.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0714e-05 2020-08-07T15:13:03.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:03.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488584+0000", 2020-08-07T15:13:03.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036298319999999999 2020-08-07T15:13:03.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:03.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492214+0000", 2020-08-07T15:13:03.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039530099999999997 2020-08-07T15:13:03.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:03.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492609+0000", 2020-08-07T15:13:03.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5300000000000002e-07 2020-08-07T15:13:03.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:03.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492610+0000", 2020-08-07T15:13:03.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197321 2020-08-07T15:13:03.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:03.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492807+0000", 2020-08-07T15:13:03.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996 2020-08-07T15:13:03.229 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:03.230 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:03.230 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:03.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.230 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31014 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:13:03.230 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:13:03.231 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 255.08566275199999, 2020-08-07T15:13:03.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998, 2020-08-07T15:13:03.231 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:03.231 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:03.231 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:03.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:03.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:03.232 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31014 2020-08-07T15:13:03.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.232 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:03.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:03.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:13:03.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:03.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:03.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:13:03.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3743999999999999e-05 2020-08-07T15:13:03.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:03.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733685+0000", 2020-08-07T15:13:03.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1720000000000007e-06 2020-08-07T15:13:03.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:03.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733693+0000", 2020-08-07T15:13:03.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3674e-05 2020-08-07T15:13:03.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:03.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733707+0000", 2020-08-07T15:13:03.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9249999999999998e-06 2020-08-07T15:13:03.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:03.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733714+0000", 2020-08-07T15:13:03.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8338000000000003e-05 2020-08-07T15:13:03.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:03.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733772+0000", 2020-08-07T15:13:03.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3019e-05 2020-08-07T15:13:03.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:03.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733785+0000", 2020-08-07T15:13:03.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75134856299999997 2020-08-07T15:13:03.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:03.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485133+0000", 2020-08-07T15:13:03.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8739999999999995e-05 2020-08-07T15:13:03.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:03.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485212+0000", 2020-08-07T15:13:03.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151244 2020-08-07T15:13:03.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:13:03.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485363+0000", 2020-08-07T15:13:03.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032929090000000001 2020-08-07T15:13:03.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:03.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488656+0000", 2020-08-07T15:13:03.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.065e-06 2020-08-07T15:13:03.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:03.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488657+0000", 2020-08-07T15:13:03.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1123000000000003e-05 2020-08-07T15:13:03.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:03.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488729+0000", 2020-08-07T15:13:03.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035508330000000002 2020-08-07T15:13:03.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:03.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492279+0000", 2020-08-07T15:13:03.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00059376000000000003 2020-08-07T15:13:03.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:03.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492873+0000", 2020-08-07T15:13:03.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5700000000000002e-07 2020-08-07T15:13:03.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:03.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492874+0000", 2020-08-07T15:13:03.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019264 2020-08-07T15:13:03.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:03.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493066+0000", 2020-08-07T15:13:03.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998 2020-08-07T15:13:03.268 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:03.268 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:03.268 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:03.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.269 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31015 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:13:03.269 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:13:03.269 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 255.08553678600001, 2020-08-07T15:13:03.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002, 2020-08-07T15:13:03.269 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:03.270 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:03.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:03.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:03.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:03.270 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31015 2020-08-07T15:13:03.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.271 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:03.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:03.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:13:03.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:03.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:03.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:13:03.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3707000000000001e-05 2020-08-07T15:13:03.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:03.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733791+0000", 2020-08-07T15:13:03.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8120000000000001e-06 2020-08-07T15:13:03.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:03.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733794+0000", 2020-08-07T15:13:03.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0728e-05 2020-08-07T15:13:03.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:03.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733805+0000", 2020-08-07T15:13:03.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8279999999999999e-06 2020-08-07T15:13:03.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:03.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733811+0000", 2020-08-07T15:13:03.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2955999999999999e-05 2020-08-07T15:13:03.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:03.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733864+0000", 2020-08-07T15:13:03.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1389000000000001e-05 2020-08-07T15:13:03.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:03.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733875+0000", 2020-08-07T15:13:03.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75232711100000005 2020-08-07T15:13:03.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:03.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486202+0000", 2020-08-07T15:13:03.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027503400000000001 2020-08-07T15:13:03.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:03.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486478+0000", 2020-08-07T15:13:03.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033010400000000001 2020-08-07T15:13:03.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:13:03.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486808+0000", 2020-08-07T15:13:03.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019382609999999999 2020-08-07T15:13:03.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:03.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488746+0000", 2020-08-07T15:13:03.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9400000000000004e-07 2020-08-07T15:13:03.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:03.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488747+0000", 2020-08-07T15:13:03.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4324000000000002e-05 2020-08-07T15:13:03.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:03.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488821+0000", 2020-08-07T15:13:03.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043745640000000001 2020-08-07T15:13:03.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:03.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493196+0000", 2020-08-07T15:13:03.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030381000000000001 2020-08-07T15:13:03.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:03.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493499+0000", 2020-08-07T15:13:03.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4e-07 2020-08-07T15:13:03.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:03.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493500+0000", 2020-08-07T15:13:03.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017499 2020-08-07T15:13:03.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:03.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493675+0000", 2020-08-07T15:13:03.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002 2020-08-07T15:13:03.296 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:03.296 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:03.296 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:03.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.297 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31016 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:13:03.298 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:13:03.298 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 255.084940811, 2020-08-07T15:13:03.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971, 2020-08-07T15:13:03.299 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:03.299 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:03.299 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:03.300 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:03.300 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:03.300 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31016 2020-08-07T15:13:03.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.300 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:03.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:03.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:13:03.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:03.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:03.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:13:03.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0924000000000002e-05 2020-08-07T15:13:03.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:03.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734404+0000", 2020-08-07T15:13:03.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2020-08-07T15:13:03.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:03.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734410+0000", 2020-08-07T15:13:03.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3459999999999994e-06 2020-08-07T15:13:03.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:03.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734418+0000", 2020-08-07T15:13:03.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9640000000000002e-06 2020-08-07T15:13:03.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:03.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734423+0000", 2020-08-07T15:13:03.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7253999999999999e-05 2020-08-07T15:13:03.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:03.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734470+0000", 2020-08-07T15:13:03.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0249999999999997e-06 2020-08-07T15:13:03.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:03.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734477+0000", 2020-08-07T15:13:03.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75294592900000001 2020-08-07T15:13:03.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:03.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487423+0000", 2020-08-07T15:13:03.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031604300000000001 2020-08-07T15:13:03.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:03.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487739+0000", 2020-08-07T15:13:03.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000171149 2020-08-07T15:13:03.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:13:03.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487910+0000", 2020-08-07T15:13:03.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024207769999999998 2020-08-07T15:13:03.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:03.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490331+0000", 2020-08-07T15:13:03.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1650000000000001e-06 2020-08-07T15:13:03.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:03.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490332+0000", 2020-08-07T15:13:03.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9796999999999996e-05 2020-08-07T15:13:03.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:03.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490402+0000", 2020-08-07T15:13:03.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029928849999999998 2020-08-07T15:13:03.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:03.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493395+0000", 2020-08-07T15:13:03.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034903199999999999 2020-08-07T15:13:03.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:03.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493744+0000", 2020-08-07T15:13:03.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8899999999999999e-07 2020-08-07T15:13:03.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:03.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493745+0000", 2020-08-07T15:13:03.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197362 2020-08-07T15:13:03.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:03.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493942+0000", 2020-08-07T15:13:03.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971 2020-08-07T15:13:03.322 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:03.323 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:03.323 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:03.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.324 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:89632 5.f e207/164 5:fd260020:::benchmark_data_smithi074_53697_object11203:head v 207'5744, mlcod=207'5734)", 2020-08-07T15:13:03.324 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:21.551762+0000", 2020-08-07T15:13:03.324 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 100.26753195400001, 2020-08-07T15:13:03.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67527211499999995, 2020-08-07T15:13:03.324 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:03.325 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:03.325 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:03.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:03.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551762+0000", 2020-08-07T15:13:03.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:03.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:03.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551762+0000", 2020-08-07T15:13:03.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8560000000000003e-05 2020-08-07T15:13:03.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:03.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551810+0000", 2020-08-07T15:13:03.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9430000000000002e-06 2020-08-07T15:13:03.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:03.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551814+0000", 2020-08-07T15:13:03.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67230272000000002 2020-08-07T15:13:03.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:03.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.224117+0000", 2020-08-07T15:13:03.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7120000000000003e-06 2020-08-07T15:13:03.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:03.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.224124+0000", 2020-08-07T15:13:03.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001719307 2020-08-07T15:13:03.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:03.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.225843+0000", 2020-08-07T15:13:03.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3902999999999997e-05 2020-08-07T15:13:03.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:03.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.225877+0000", 2020-08-07T15:13:03.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001110279 2020-08-07T15:13:03.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:03.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.226987+0000", 2020-08-07T15:13:03.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6690999999999997e-05 2020-08-07T15:13:03.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:03.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.227034+0000", 2020-08-07T15:13:03.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67527211499999995 2020-08-07T15:13:03.351 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:03.351 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:03.356 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:03.356 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:03.356 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:03.356 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:03.359 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 50 16 74205 74189 11.5908 3.76562 0.426565 0.0106406 2020-08-07T15:13:03.359 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 51 16 75396 75380 11.5459 9.30469 0.0201081 0.0108221 2020-08-07T15:13:03.359 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:13:03.360 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:03.356+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:03.360 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:13:03.461 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:03.458+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:03.546 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:03.546 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:13:03.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.547 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:75696 6.d 6:b7664df3:::benchmark_data_smithi074_63843_object9461:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e270)", 2020-08-07T15:13:03.547 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:13:03.451864+0000", 2020-08-07T15:13:03.547 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.093775263999999997, 2020-08-07T15:13:03.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.093796637000000002, 2020-08-07T15:13:03.548 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:03.548 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:03.548 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:03.548 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:13:03.548 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:03.549 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 75696 2020-08-07T15:13:03.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.549 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:03.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:03.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:03.451864+0000", 2020-08-07T15:13:03.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:03.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:03.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:03.451864+0000", 2020-08-07T15:13:03.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9762000000000002e-05 2020-08-07T15:13:03.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:03.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:03.451914+0000", 2020-08-07T15:13:03.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4910000000000002e-06 2020-08-07T15:13:03.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:03.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:03.451920+0000", 2020-08-07T15:13:03.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2707999999999999e-05 2020-08-07T15:13:03.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:03.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:03.451933+0000", 2020-08-07T15:13:03.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1369999999999999e-06 2020-08-07T15:13:03.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:03.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:03.451939+0000", 2020-08-07T15:13:03.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00065118900000000004 2020-08-07T15:13:03.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:03.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:03.452590+0000", 2020-08-07T15:13:03.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2755000000000002e-05 2020-08-07T15:13:03.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:03.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:03.452683+0000", 2020-08-07T15:13:03.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018626600000000001 2020-08-07T15:13:03.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:13:03.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:03.452869+0000", 2020-08-07T15:13:03.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00232737 2020-08-07T15:13:03.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:03.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:03.455197+0000", 2020-08-07T15:13:03.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0399999999999995e-07 2020-08-07T15:13:03.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:03.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:03.455197+0000", 2020-08-07T15:13:03.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2212999999999995e-05 2020-08-07T15:13:03.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:03.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:03.455260+0000", 2020-08-07T15:13:03.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001076542 2020-08-07T15:13:03.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:03.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:03.456336+0000", 2020-08-07T15:13:03.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044721370000000002 2020-08-07T15:13:03.596 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:03.597 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:03.597 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:03.597 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:03.602 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:13:03.602 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 1 2020-08-07T15:13:03.603 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:03.605 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:13:03.611 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:03.603+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:03.705 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:03.703+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:03.765 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:13:03.773 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:03.773 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:13:03.773 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:13:03.774 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:13:03.774 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:03.786 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:13:03.830 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:03.829+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:03.908 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:03.905+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:03.941 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:03.941 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:13:03.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:13:03.942 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:13:03.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.942 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42440 4.9 4:9b3fd4e3:::benchmark_data_smithi074_44586_object5304:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:03.943 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:13:03.943 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 243.699625056, 2020-08-07T15:13:03.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9384075030000001, 2020-08-07T15:13:03.943 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:03.943 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:03.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:03.944 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:03.944 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:03.944 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42440 2020-08-07T15:13:03.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.944 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:03.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:03.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:13:03.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:03.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:03.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:13:03.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6295000000000002e-05 2020-08-07T15:13:03.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:03.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237116+0000", 2020-08-07T15:13:03.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0390000000000001e-06 2020-08-07T15:13:03.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:03.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237122+0000", 2020-08-07T15:13:03.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8340000000000001e-05 2020-08-07T15:13:03.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:03.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237140+0000", 2020-08-07T15:13:03.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.045e-06 2020-08-07T15:13:03.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:03.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237145+0000", 2020-08-07T15:13:03.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044548920000000002 2020-08-07T15:13:03.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:03.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241600+0000", 2020-08-07T15:13:03.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103336 2020-08-07T15:13:03.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:03.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241703+0000", 2020-08-07T15:13:03.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018660399999999999 2020-08-07T15:13:03.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:03.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241890+0000", 2020-08-07T15:13:03.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00096796000000000002 2020-08-07T15:13:03.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:03.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.242858+0000", 2020-08-07T15:13:03.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039845000000000002 2020-08-07T15:13:03.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:03.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246843+0000", 2020-08-07T15:13:03.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3499999999999996e-07 2020-08-07T15:13:03.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:03.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246843+0000", 2020-08-07T15:13:03.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7132999999999995e-05 2020-08-07T15:13:03.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:03.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246920+0000", 2020-08-07T15:13:03.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046993410000000001 2020-08-07T15:13:03.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:03.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.251620+0000", 2020-08-07T15:13:03.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040332956000000003 2020-08-07T15:13:03.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:03.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.291953+0000", 2020-08-07T15:13:03.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8390000000000003e-06 2020-08-07T15:13:03.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:03.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.291958+0000", 2020-08-07T15:13:03.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99961357500000003 2020-08-07T15:13:03.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:03.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291572+0000", 2020-08-07T15:13:03.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3913999999999999e-05 2020-08-07T15:13:03.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:03.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291586+0000", 2020-08-07T15:13:03.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88379723099999996 2020-08-07T15:13:03.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:03.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175383+0000", 2020-08-07T15:13:03.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3868000000000004e-05 2020-08-07T15:13:03.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:03.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175467+0000", 2020-08-07T15:13:03.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9384075030000001 2020-08-07T15:13:03.968 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:03.969 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:03.969 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:03.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.969 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42489 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:03.970 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:13:03.970 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 243.65632499399999, 2020-08-07T15:13:03.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952608639999999, 2020-08-07T15:13:03.970 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:03.971 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:03.971 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:03.971 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:03.971 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:03.971 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42489 2020-08-07T15:13:03.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.972 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:03.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:03.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:13:03.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:03.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:03.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:13:03.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0379e-05 2020-08-07T15:13:03.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:03.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280410+0000", 2020-08-07T15:13:03.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0489999999999998e-06 2020-08-07T15:13:03.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:03.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280417+0000", 2020-08-07T15:13:03.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8281e-05 2020-08-07T15:13:03.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:03.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280435+0000", 2020-08-07T15:13:03.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7510000000000008e-06 2020-08-07T15:13:03.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:03.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280443+0000", 2020-08-07T15:13:03.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018142899999999999 2020-08-07T15:13:03.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:03.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280624+0000", 2020-08-07T15:13:03.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013473599999999999 2020-08-07T15:13:03.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:03.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280759+0000", 2020-08-07T15:13:03.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019964200000000001 2020-08-07T15:13:03.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:03.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280959+0000", 2020-08-07T15:13:03.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005588038 2020-08-07T15:13:03.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:03.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286547+0000", 2020-08-07T15:13:03.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.376e-05 2020-08-07T15:13:03.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:03.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286561+0000", 2020-08-07T15:13:03.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4199999999999995e-07 2020-08-07T15:13:03.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:03.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286561+0000", 2020-08-07T15:13:03.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9218e-05 2020-08-07T15:13:03.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:03.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286631+0000", 2020-08-07T15:13:03.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027909199999999999 2020-08-07T15:13:03.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:03.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286910+0000", 2020-08-07T15:13:03.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050904089999999997 2020-08-07T15:13:03.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:03.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292000+0000", 2020-08-07T15:13:03.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9579999999999996e-06 2020-08-07T15:13:03.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:03.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292009+0000", 2020-08-07T15:13:03.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99961402799999999 2020-08-07T15:13:03.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:03.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291623+0000", 2020-08-07T15:13:03.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2490000000000001e-06 2020-08-07T15:13:03.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:03.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:03.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:03.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291631+0000", 2020-08-07T15:13:03.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88387422199999999 2020-08-07T15:13:04.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175506+0000", 2020-08-07T15:13:04.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000114881 2020-08-07T15:13:04.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:04.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175620+0000", 2020-08-07T15:13:04.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952608639999999 2020-08-07T15:13:04.008 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.008 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:04.008 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.009 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42490 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:04.009 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:13:04.009 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 243.65619782600001, 2020-08-07T15:13:04.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89528059, 2020-08-07T15:13:04.018 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:04.019 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:04.019 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:04.019 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:04.019 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:04.020 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42490 2020-08-07T15:13:04.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.020 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:04.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:04.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:13:04.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:04.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:04.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:13:04.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2571999999999997e-05 2020-08-07T15:13:04.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:04.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280539+0000", 2020-08-07T15:13:04.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999998e-06 2020-08-07T15:13:04.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:04.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280545+0000", 2020-08-07T15:13:04.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.134e-05 2020-08-07T15:13:04.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:04.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280556+0000", 2020-08-07T15:13:04.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.746e-06 2020-08-07T15:13:04.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:04.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280561+0000", 2020-08-07T15:13:04.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00089258999999999996 2020-08-07T15:13:04.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281453+0000", 2020-08-07T15:13:04.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5641000000000001e-05 2020-08-07T15:13:04.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:04.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281549+0000", 2020-08-07T15:13:04.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020555200000000001 2020-08-07T15:13:04.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:04.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281755+0000", 2020-08-07T15:13:04.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048878230000000003 2020-08-07T15:13:04.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:04.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286642+0000", 2020-08-07T15:13:04.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0954e-05 2020-08-07T15:13:04.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286693+0000", 2020-08-07T15:13:04.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0699999999999997e-07 2020-08-07T15:13:04.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:04.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286694+0000", 2020-08-07T15:13:04.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3697999999999998e-05 2020-08-07T15:13:04.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:04.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286768+0000", 2020-08-07T15:13:04.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017185900000000001 2020-08-07T15:13:04.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286940+0000", 2020-08-07T15:13:04.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051207819999999999 2020-08-07T15:13:04.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292060+0000", 2020-08-07T15:13:04.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8000000000000004e-06 2020-08-07T15:13:04.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:04.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292069+0000", 2020-08-07T15:13:04.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99959520599999996 2020-08-07T15:13:04.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291664+0000", 2020-08-07T15:13:04.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0160000000000008e-06 2020-08-07T15:13:04.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:04.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291672+0000", 2020-08-07T15:13:04.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88399374500000005 2020-08-07T15:13:04.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175666+0000", 2020-08-07T15:13:04.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101239 2020-08-07T15:13:04.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:04.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175767+0000", 2020-08-07T15:13:04.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89528059 2020-08-07T15:13:04.072 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.072 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:04.073 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.073 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42491 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:04.073 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:13:04.074 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 243.65609089099999, 2020-08-07T15:13:04.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895291936, 2020-08-07T15:13:04.074 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:04.074 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:04.074 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:04.075 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:04.075 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:04.075 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42491 2020-08-07T15:13:04.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.075 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:04.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:04.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:13:04.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:04.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:04.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:13:04.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0402000000000001e-05 2020-08-07T15:13:04.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:04.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280624+0000", 2020-08-07T15:13:04.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0340000000000001e-06 2020-08-07T15:13:04.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:04.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280627+0000", 2020-08-07T15:13:04.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1886999999999999e-05 2020-08-07T15:13:04.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:04.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280649+0000", 2020-08-07T15:13:04.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5430000000000004e-06 2020-08-07T15:13:04.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:04.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280654+0000", 2020-08-07T15:13:04.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032800960000000001 2020-08-07T15:13:04.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.283934+0000", 2020-08-07T15:13:04.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011264699999999999 2020-08-07T15:13:04.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:04.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284046+0000", 2020-08-07T15:13:04.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019987299999999999 2020-08-07T15:13:04.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:04.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284246+0000", 2020-08-07T15:13:04.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025610960000000001 2020-08-07T15:13:04.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286807+0000", 2020-08-07T15:13:04.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1409999999999999e-06 2020-08-07T15:13:04.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:04.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286814+0000", 2020-08-07T15:13:04.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0499999999999999e-07 2020-08-07T15:13:04.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:04.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286815+0000", 2020-08-07T15:13:04.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8538000000000003e-05 2020-08-07T15:13:04.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:04.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286853+0000", 2020-08-07T15:13:04.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112914 2020-08-07T15:13:04.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286966+0000", 2020-08-07T15:13:04.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051403120000000002 2020-08-07T15:13:04.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292107+0000", 2020-08-07T15:13:04.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8019999999999992e-06 2020-08-07T15:13:04.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:04.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292114+0000", 2020-08-07T15:13:04.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.999590859 2020-08-07T15:13:04.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291705+0000", 2020-08-07T15:13:04.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.174e-06 2020-08-07T15:13:04.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:04.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291713+0000", 2020-08-07T15:13:04.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88407676000000002 2020-08-07T15:13:04.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175790+0000", 2020-08-07T15:13:04.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5452999999999996e-05 2020-08-07T15:13:04.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:04.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175886+0000", 2020-08-07T15:13:04.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895291936 2020-08-07T15:13:04.097 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.097 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:04.098 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.098 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42492 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:04.099 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:13:04.099 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 243.65599784099999, 2020-08-07T15:13:04.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895293774, 2020-08-07T15:13:04.099 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:04.099 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:04.100 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:04.100 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:04.100 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:04.100 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42492 2020-08-07T15:13:04.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.100 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:04.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:04.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:13:04.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:04.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:04.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:13:04.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8272000000000001e-05 2020-08-07T15:13:04.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:04.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280735+0000", 2020-08-07T15:13:04.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2159999999999997e-06 2020-08-07T15:13:04.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:04.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280740+0000", 2020-08-07T15:13:04.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0251000000000001e-05 2020-08-07T15:13:04.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:04.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280750+0000", 2020-08-07T15:13:04.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3139999999999997e-06 2020-08-07T15:13:04.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:04.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280755+0000", 2020-08-07T15:13:04.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039343579999999998 2020-08-07T15:13:04.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284689+0000", 2020-08-07T15:13:04.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111648 2020-08-07T15:13:04.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:04.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284801+0000", 2020-08-07T15:13:04.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019563600000000001 2020-08-07T15:13:04.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:04.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284996+0000", 2020-08-07T15:13:04.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018636709999999999 2020-08-07T15:13:04.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:04.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286860+0000", 2020-08-07T15:13:04.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001865802 2020-08-07T15:13:04.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288726+0000", 2020-08-07T15:13:04.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0399999999999996e-07 2020-08-07T15:13:04.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:04.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288726+0000", 2020-08-07T15:13:04.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6573999999999998e-05 2020-08-07T15:13:04.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:04.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288773+0000", 2020-08-07T15:13:04.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003455283 2020-08-07T15:13:04.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292228+0000", 2020-08-07T15:13:04.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0099999999999995e-06 2020-08-07T15:13:04.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:04.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292236+0000", 2020-08-07T15:13:04.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99950963999999998 2020-08-07T15:13:04.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291746+0000", 2020-08-07T15:13:04.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0779999999999996e-06 2020-08-07T15:13:04.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:04.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291754+0000", 2020-08-07T15:13:04.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.884152508 2020-08-07T15:13:04.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175906+0000", 2020-08-07T15:13:04.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4009000000000002e-05 2020-08-07T15:13:04.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:04.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175980+0000", 2020-08-07T15:13:04.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895293774 2020-08-07T15:13:04.129 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.129 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:04.130 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.130 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42493 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:04.130 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:13:04.131 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 243.65589619400001, 2020-08-07T15:13:04.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952963789999999, 2020-08-07T15:13:04.142 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:04.142 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:04.142 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:04.143 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:04.143 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:04.143 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42493 2020-08-07T15:13:04.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.144 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:04.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:04.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:13:04.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:04.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:04.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:13:04.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9740999999999997e-05 2020-08-07T15:13:04.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:04.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280838+0000", 2020-08-07T15:13:04.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7809999999999996e-06 2020-08-07T15:13:04.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:04.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280843+0000", 2020-08-07T15:13:04.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0648999999999999e-05 2020-08-07T15:13:04.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:04.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280854+0000", 2020-08-07T15:13:04.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3669999999999999e-06 2020-08-07T15:13:04.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:04.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280858+0000", 2020-08-07T15:13:04.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004850867 2020-08-07T15:13:04.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.285709+0000", 2020-08-07T15:13:04.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110037 2020-08-07T15:13:04.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:04.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.285819+0000", 2020-08-07T15:13:04.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020935699999999999 2020-08-07T15:13:04.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:04.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286028+0000", 2020-08-07T15:13:04.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020900850000000002 2020-08-07T15:13:04.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:04.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288118+0000", 2020-08-07T15:13:04.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041669649999999999 2020-08-07T15:13:04.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292285+0000", 2020-08-07T15:13:04.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3820000000000004e-06 2020-08-07T15:13:04.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:04.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292294+0000", 2020-08-07T15:13:04.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949242900000002 2020-08-07T15:13:04.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291786+0000", 2020-08-07T15:13:04.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000003e-06 2020-08-07T15:13:04.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:04.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291794+0000", 2020-08-07T15:13:04.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88420652700000002 2020-08-07T15:13:04.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176000+0000", 2020-08-07T15:13:04.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4634000000000003e-05 2020-08-07T15:13:04.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:04.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176085+0000", 2020-08-07T15:13:04.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952963789999999 2020-08-07T15:13:04.178 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.179 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:04.179 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.179 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42494 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:04.180 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:13:04.180 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 243.655794095, 2020-08-07T15:13:04.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952882820000001, 2020-08-07T15:13:04.180 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:04.180 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:04.180 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:04.181 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:04.181 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:04.181 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42494 2020-08-07T15:13:04.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.182 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:04.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:04.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:13:04.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:04.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:04.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:13:04.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0117999999999997e-05 2020-08-07T15:13:04.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:04.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280941+0000", 2020-08-07T15:13:04.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1240000000000004e-06 2020-08-07T15:13:04.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:04.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280946+0000", 2020-08-07T15:13:04.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0000000000000001e-05 2020-08-07T15:13:04.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:04.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280956+0000", 2020-08-07T15:13:04.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.279e-06 2020-08-07T15:13:04.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:04.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280960+0000", 2020-08-07T15:13:04.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0062727080000000001 2020-08-07T15:13:04.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287233+0000", 2020-08-07T15:13:04.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018362900000000001 2020-08-07T15:13:04.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:04.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287416+0000", 2020-08-07T15:13:04.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029536100000000001 2020-08-07T15:13:04.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:04.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287712+0000", 2020-08-07T15:13:04.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046231739999999999 2020-08-07T15:13:04.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292335+0000", 2020-08-07T15:13:04.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3860000000000001e-06 2020-08-07T15:13:04.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:04.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292342+0000", 2020-08-07T15:13:04.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99948501000000001 2020-08-07T15:13:04.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291827+0000", 2020-08-07T15:13:04.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2272999999999999e-05 2020-08-07T15:13:04.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:04.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291840+0000", 2020-08-07T15:13:04.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88426668100000005 2020-08-07T15:13:04.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176106+0000", 2020-08-07T15:13:04.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2539000000000005e-05 2020-08-07T15:13:04.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:04.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176179+0000", 2020-08-07T15:13:04.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952882820000001 2020-08-07T15:13:04.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.198 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:04.198 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.198 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42495 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:04.198 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:13:04.199 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 243.65569300499999, 2020-08-07T15:13:04.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895286867, 2020-08-07T15:13:04.199 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:04.199 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:04.199 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:04.200 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:04.200 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:04.200 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42495 2020-08-07T15:13:04.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.200 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:04.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:04.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:13:04.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:04.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:04.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:13:04.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6690999999999997e-05 2020-08-07T15:13:04.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:04.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281038+0000", 2020-08-07T15:13:04.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7860000000000002e-06 2020-08-07T15:13:04.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:04.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281044+0000", 2020-08-07T15:13:04.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0429e-05 2020-08-07T15:13:04.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:04.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281054+0000", 2020-08-07T15:13:04.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9040000000000002e-06 2020-08-07T15:13:04.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:04.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281058+0000", 2020-08-07T15:13:04.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070941490000000001 2020-08-07T15:13:04.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288153+0000", 2020-08-07T15:13:04.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9006999999999998e-05 2020-08-07T15:13:04.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:04.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288222+0000", 2020-08-07T15:13:04.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012106400000000001 2020-08-07T15:13:04.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:04.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288343+0000", 2020-08-07T15:13:04.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004034752 2020-08-07T15:13:04.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292377+0000", 2020-08-07T15:13:04.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.537e-06 2020-08-07T15:13:04.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:04.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292385+0000", 2020-08-07T15:13:04.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949123500000003 2020-08-07T15:13:04.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291876+0000", 2020-08-07T15:13:04.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7729999999999999e-06 2020-08-07T15:13:04.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:04.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291884+0000", 2020-08-07T15:13:04.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88431700899999999 2020-08-07T15:13:04.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176201+0000", 2020-08-07T15:13:04.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7531e-05 2020-08-07T15:13:04.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:04.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176278+0000", 2020-08-07T15:13:04.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895286867 2020-08-07T15:13:04.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.216 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:04.216 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.216 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42496 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:04.217 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:13:04.217 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 243.655588096, 2020-08-07T15:13:04.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89526942, 2020-08-07T15:13:04.217 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:04.217 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:04.218 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:04.218 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:04.218 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:04.218 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42496 2020-08-07T15:13:04.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.218 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:04.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:04.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:13:04.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:04.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:04.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:13:04.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9227999999999997e-05 2020-08-07T15:13:04.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:04.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281146+0000", 2020-08-07T15:13:04.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1120000000000004e-06 2020-08-07T15:13:04.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:04.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281151+0000", 2020-08-07T15:13:04.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2208999999999999e-05 2020-08-07T15:13:04.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:04.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281163+0000", 2020-08-07T15:13:04.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5469999999999998e-06 2020-08-07T15:13:04.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:04.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281168+0000", 2020-08-07T15:13:04.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0076520829999999996 2020-08-07T15:13:04.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288820+0000", 2020-08-07T15:13:04.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5035000000000003e-05 2020-08-07T15:13:04.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:04.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288885+0000", 2020-08-07T15:13:04.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116507 2020-08-07T15:13:04.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:04.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.289001+0000", 2020-08-07T15:13:04.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034121960000000002 2020-08-07T15:13:04.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292413+0000", 2020-08-07T15:13:04.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8109999999999997e-06 2020-08-07T15:13:04.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:04.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292420+0000", 2020-08-07T15:13:04.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949504600000005 2020-08-07T15:13:04.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291915+0000", 2020-08-07T15:13:04.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3999999999999992e-06 2020-08-07T15:13:04.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:04.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291924+0000", 2020-08-07T15:13:04.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88437394599999997 2020-08-07T15:13:04.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176298+0000", 2020-08-07T15:13:04.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8300000000000007e-05 2020-08-07T15:13:04.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:04.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176366+0000", 2020-08-07T15:13:04.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89526942 2020-08-07T15:13:04.235 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.235 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:04.236 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.236 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73081 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:13:04.236 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:13:04.237 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 216.710118822, 2020-08-07T15:13:04.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0005038420000001, 2020-08-07T15:13:04.237 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:04.237 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:04.237 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:04.238 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:04.238 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:04.238 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73081 2020-08-07T15:13:04.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.238 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:04.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:04.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:13:04.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:04.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:04.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:13:04.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3016000000000003e-05 2020-08-07T15:13:04.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:04.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226609+0000", 2020-08-07T15:13:04.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4479999999999996e-06 2020-08-07T15:13:04.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:04.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226613+0000", 2020-08-07T15:13:04.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0189999999999995e-06 2020-08-07T15:13:04.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:04.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226622+0000", 2020-08-07T15:13:04.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5980000000000001e-06 2020-08-07T15:13:04.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:04.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226626+0000", 2020-08-07T15:13:04.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8088999999999999e-05 2020-08-07T15:13:04.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226684+0000", 2020-08-07T15:13:04.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2162e-05 2020-08-07T15:13:04.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:04.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226696+0000", 2020-08-07T15:13:04.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98675689 2020-08-07T15:13:04.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213453+0000", 2020-08-07T15:13:04.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012916800000000001 2020-08-07T15:13:04.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:04.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213582+0000", 2020-08-07T15:13:04.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015665899999999999 2020-08-07T15:13:04.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:13:04.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213739+0000", 2020-08-07T15:13:04.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049940239999999997 2020-08-07T15:13:04.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:04.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218733+0000", 2020-08-07T15:13:04.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0079774779999999997 2020-08-07T15:13:04.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226710+0000", 2020-08-07T15:13:04.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.161e-06 2020-08-07T15:13:04.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:04.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226711+0000", 2020-08-07T15:13:04.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0105000000000001e-05 2020-08-07T15:13:04.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:04.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226772+0000", 2020-08-07T15:13:04.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5633999999999997e-05 2020-08-07T15:13:04.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226817+0000", 2020-08-07T15:13:04.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7379000000000001e-05 2020-08-07T15:13:04.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226875+0000", 2020-08-07T15:13:04.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8000000000000005e-07 2020-08-07T15:13:04.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:04.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226875+0000", 2020-08-07T15:13:04.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019423199999999999 2020-08-07T15:13:04.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:04.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227070+0000", 2020-08-07T15:13:04.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0005038420000001 2020-08-07T15:13:04.282 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.283 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:04.283 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.283 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73082 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:13:04.284 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:13:04.284 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 216.70974835800001, 2020-08-07T15:13:04.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001326693, 2020-08-07T15:13:04.284 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:04.284 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:04.285 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:04.285 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:04.285 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:04.285 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73082 2020-08-07T15:13:04.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.286 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:04.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:04.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:13:04.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:04.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:04.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:13:04.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6928000000000003e-05 2020-08-07T15:13:04.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:04.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226973+0000", 2020-08-07T15:13:04.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3630000000000004e-06 2020-08-07T15:13:04.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:04.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226977+0000", 2020-08-07T15:13:04.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9659999999999996e-06 2020-08-07T15:13:04.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:04.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226985+0000", 2020-08-07T15:13:04.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5800000000000002e-06 2020-08-07T15:13:04.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:04.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226990+0000", 2020-08-07T15:13:04.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9101999999999998e-05 2020-08-07T15:13:04.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227039+0000", 2020-08-07T15:13:04.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9620000000000005e-06 2020-08-07T15:13:04.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:04.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227049+0000", 2020-08-07T15:13:04.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98722513999999995 2020-08-07T15:13:04.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214274+0000", 2020-08-07T15:13:04.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2700999999999994e-05 2020-08-07T15:13:04.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:04.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214347+0000", 2020-08-07T15:13:04.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015346899999999999 2020-08-07T15:13:04.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:13:04.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214500+0000", 2020-08-07T15:13:04.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042450309999999998 2020-08-07T15:13:04.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:04.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218745+0000", 2020-08-07T15:13:04.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008379503 2020-08-07T15:13:04.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227125+0000", 2020-08-07T15:13:04.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7899999999999998e-07 2020-08-07T15:13:04.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:04.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227126+0000", 2020-08-07T15:13:04.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1610000000000002e-05 2020-08-07T15:13:04.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:04.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227177+0000", 2020-08-07T15:13:04.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489000000000003e-05 2020-08-07T15:13:04.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227226+0000", 2020-08-07T15:13:04.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00084338600000000001 2020-08-07T15:13:04.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228069+0000", 2020-08-07T15:13:04.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7e-07 2020-08-07T15:13:04.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:04.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228069+0000", 2020-08-07T15:13:04.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019351399999999999 2020-08-07T15:13:04.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:04.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228263+0000", 2020-08-07T15:13:04.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001326693 2020-08-07T15:13:04.306 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.306 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:04.306 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.306 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73083 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:13:04.307 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:13:04.307 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 216.70965409900001, 2020-08-07T15:13:04.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015968209999999, 2020-08-07T15:13:04.307 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:04.307 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:04.308 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:04.308 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:04.308 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:04.308 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73083 2020-08-07T15:13:04.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.309 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:04.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:04.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:13:04.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:04.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:04.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:13:04.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1627000000000001e-05 2020-08-07T15:13:04.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:04.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227062+0000", 2020-08-07T15:13:04.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.253e-06 2020-08-07T15:13:04.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:04.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227065+0000", 2020-08-07T15:13:04.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3168e-05 2020-08-07T15:13:04.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:04.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227079+0000", 2020-08-07T15:13:04.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1549999999999999e-06 2020-08-07T15:13:04.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:04.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227082+0000", 2020-08-07T15:13:04.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5534999999999997e-05 2020-08-07T15:13:04.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227127+0000", 2020-08-07T15:13:04.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6823000000000001e-05 2020-08-07T15:13:04.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:04.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227144+0000", 2020-08-07T15:13:04.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98771770599999997 2020-08-07T15:13:04.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214862+0000", 2020-08-07T15:13:04.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8252999999999999e-05 2020-08-07T15:13:04.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:04.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214930+0000", 2020-08-07T15:13:04.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013646400000000001 2020-08-07T15:13:04.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:13:04.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215066+0000", 2020-08-07T15:13:04.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036907089999999999 2020-08-07T15:13:04.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:04.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218757+0000", 2020-08-07T15:13:04.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0085133450000000003 2020-08-07T15:13:04.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227270+0000", 2020-08-07T15:13:04.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4899999999999995e-07 2020-08-07T15:13:04.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:04.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227271+0000", 2020-08-07T15:13:04.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4555999999999997e-05 2020-08-07T15:13:04.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:04.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227326+0000", 2020-08-07T15:13:04.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024594800000000001 2020-08-07T15:13:04.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227572+0000", 2020-08-07T15:13:04.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00086125700000000004 2020-08-07T15:13:04.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228433+0000", 2020-08-07T15:13:04.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3799999999999998e-07 2020-08-07T15:13:04.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:04.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228433+0000", 2020-08-07T15:13:04.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019413500000000001 2020-08-07T15:13:04.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:04.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228627+0000", 2020-08-07T15:13:04.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015968209999999 2020-08-07T15:13:04.328 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.328 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:04.328 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.329 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73084 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:13:04.329 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:13:04.329 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 216.70956516000001, 2020-08-07T15:13:04.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0016919049999999, 2020-08-07T15:13:04.329 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:04.330 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:04.330 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:04.330 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:04.330 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:04.330 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73084 2020-08-07T15:13:04.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.331 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:04.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:04.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:13:04.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:04.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:04.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:13:04.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5565999999999997e-05 2020-08-07T15:13:04.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:04.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227165+0000", 2020-08-07T15:13:04.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2020-08-07T15:13:04.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:04.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227169+0000", 2020-08-07T15:13:04.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6850000000000001e-06 2020-08-07T15:13:04.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:04.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227176+0000", 2020-08-07T15:13:04.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1949999999999999e-06 2020-08-07T15:13:04.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:04.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227179+0000", 2020-08-07T15:13:04.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0318000000000003e-05 2020-08-07T15:13:04.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227220+0000", 2020-08-07T15:13:04.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0349e-05 2020-08-07T15:13:04.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:04.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227230+0000", 2020-08-07T15:13:04.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98832962400000002 2020-08-07T15:13:04.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215560+0000", 2020-08-07T15:13:04.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013913099999999999 2020-08-07T15:13:04.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:04.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215699+0000", 2020-08-07T15:13:04.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000200006 2020-08-07T15:13:04.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:13:04.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215899+0000", 2020-08-07T15:13:04.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028673650000000002 2020-08-07T15:13:04.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:04.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218766+0000", 2020-08-07T15:13:04.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086101779999999996 2020-08-07T15:13:04.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227376+0000", 2020-08-07T15:13:04.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0999999999999998e-07 2020-08-07T15:13:04.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:04.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227377+0000", 2020-08-07T15:13:04.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6552999999999997e-05 2020-08-07T15:13:04.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:04.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227454+0000", 2020-08-07T15:13:04.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000421717 2020-08-07T15:13:04.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227875+0000", 2020-08-07T15:13:04.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00081781300000000003 2020-08-07T15:13:04.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228693+0000", 2020-08-07T15:13:04.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1699999999999997e-07 2020-08-07T15:13:04.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:04.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228694+0000", 2020-08-07T15:13:04.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117483 2020-08-07T15:13:04.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:04.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228811+0000", 2020-08-07T15:13:04.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0016919049999999 2020-08-07T15:13:04.350 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.351 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:04.351 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.351 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73085 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:13:04.352 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:13:04.352 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 216.70942934499999, 2020-08-07T15:13:04.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0018015629999999, 2020-08-07T15:13:04.352 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:04.352 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:04.352 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:04.353 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:04.353 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:04.353 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73085 2020-08-07T15:13:04.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.353 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:04.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:04.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:13:04.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:04.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:04.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:13:04.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6570999999999997e-05 2020-08-07T15:13:04.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:04.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227322+0000", 2020-08-07T15:13:04.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5509999999999996e-06 2020-08-07T15:13:04.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:04.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227328+0000", 2020-08-07T15:13:04.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2498e-05 2020-08-07T15:13:04.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:04.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227341+0000", 2020-08-07T15:13:04.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4389999999999999e-06 2020-08-07T15:13:04.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:04.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227346+0000", 2020-08-07T15:13:04.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3711999999999998e-05 2020-08-07T15:13:04.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227390+0000", 2020-08-07T15:13:04.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1868999999999999e-05 2020-08-07T15:13:04.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227402+0000", 2020-08-07T15:13:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98896525099999999 2020-08-07T15:13:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216367+0000", 2020-08-07T15:13:04.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012573 2020-08-07T15:13:04.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:04.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216493+0000", 2020-08-07T15:13:04.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020525200000000001 2020-08-07T15:13:04.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:13:04.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216698+0000", 2020-08-07T15:13:04.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037596470000000001 2020-08-07T15:13:04.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.220458+0000", 2020-08-07T15:13:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070227190000000002 2020-08-07T15:13:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227480+0000", 2020-08-07T15:13:04.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6600000000000003e-07 2020-08-07T15:13:04.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:04.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227481+0000", 2020-08-07T15:13:04.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9898000000000003e-05 2020-08-07T15:13:04.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:04.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227521+0000", 2020-08-07T15:13:04.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039130000000000002 2020-08-07T15:13:04.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227912+0000", 2020-08-07T15:13:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00095502399999999996 2020-08-07T15:13:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228867+0000", 2020-08-07T15:13:04.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2020-08-07T15:13:04.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:04.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228867+0000", 2020-08-07T15:13:04.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018933599999999999 2020-08-07T15:13:04.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:04.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229057+0000", 2020-08-07T15:13:04.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0018015629999999 2020-08-07T15:13:04.372 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.372 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:04.372 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.373 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73086 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:13:04.373 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:13:04.373 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 216.709296849, 2020-08-07T15:13:04.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002364923, 2020-08-07T15:13:04.374 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:04.374 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:04.374 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:04.375 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:04.375 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:04.375 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73086 2020-08-07T15:13:04.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.375 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:04.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:04.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:13:04.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:04.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:04.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:13:04.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000238426 2020-08-07T15:13:04.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:04.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227626+0000", 2020-08-07T15:13:04.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9609999999999999e-06 2020-08-07T15:13:04.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:04.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227633+0000", 2020-08-07T15:13:04.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2727e-05 2020-08-07T15:13:04.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:04.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227646+0000", 2020-08-07T15:13:04.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1739999999999997e-06 2020-08-07T15:13:04.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:04.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227652+0000", 2020-08-07T15:13:04.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5004000000000001e-05 2020-08-07T15:13:04.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227737+0000", 2020-08-07T15:13:04.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2744e-05 2020-08-07T15:13:04.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:04.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227750+0000", 2020-08-07T15:13:04.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98938776500000003 2020-08-07T15:13:04.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217138+0000", 2020-08-07T15:13:04.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012361499999999999 2020-08-07T15:13:04.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:04.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217261+0000", 2020-08-07T15:13:04.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000212504 2020-08-07T15:13:04.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:13:04.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217474+0000", 2020-08-07T15:13:04.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002995036 2020-08-07T15:13:04.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:04.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.220469+0000", 2020-08-07T15:13:04.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071283249999999996 2020-08-07T15:13:04.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227597+0000", 2020-08-07T15:13:04.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0499999999999999e-07 2020-08-07T15:13:04.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:04.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227597+0000", 2020-08-07T15:13:04.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1872999999999997e-05 2020-08-07T15:13:04.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:04.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227639+0000", 2020-08-07T15:13:04.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032010200000000001 2020-08-07T15:13:04.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227959+0000", 2020-08-07T15:13:04.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015980899999999999 2020-08-07T15:13:04.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229557+0000", 2020-08-07T15:13:04.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9299999999999999e-07 2020-08-07T15:13:04.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:04.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229558+0000", 2020-08-07T15:13:04.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019477900000000001 2020-08-07T15:13:04.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:04.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229753+0000", 2020-08-07T15:13:04.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002364923 2020-08-07T15:13:04.394 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.394 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:04.395 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.395 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73087 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:13:04.395 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:13:04.395 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 216.70835569600001, 2020-08-07T15:13:04.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001896506, 2020-08-07T15:13:04.397 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:04.397 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:04.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:04.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:04.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:04.398 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73087 2020-08-07T15:13:04.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.398 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:04.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:04.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:13:04.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:04.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:04.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:13:04.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7354e-05 2020-08-07T15:13:04.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:04.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228386+0000", 2020-08-07T15:13:04.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8000000000000001e-06 2020-08-07T15:13:04.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:04.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228393+0000", 2020-08-07T15:13:04.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3784999999999999e-05 2020-08-07T15:13:04.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:04.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228407+0000", 2020-08-07T15:13:04.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8090000000000004e-06 2020-08-07T15:13:04.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:04.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228414+0000", 2020-08-07T15:13:04.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6907000000000003e-05 2020-08-07T15:13:04.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228461+0000", 2020-08-07T15:13:04.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8929999999999994e-06 2020-08-07T15:13:04.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:04.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228469+0000", 2020-08-07T15:13:04.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98950057700000005 2020-08-07T15:13:04.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217970+0000", 2020-08-07T15:13:04.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000132182 2020-08-07T15:13:04.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:04.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218102+0000", 2020-08-07T15:13:04.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020073399999999999 2020-08-07T15:13:04.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:13:04.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218303+0000", 2020-08-07T15:13:04.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035951120000000001 2020-08-07T15:13:04.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:04.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.221898+0000", 2020-08-07T15:13:04.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057898660000000003 2020-08-07T15:13:04.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227688+0000", 2020-08-07T15:13:04.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0200000000000002e-07 2020-08-07T15:13:04.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:04.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227688+0000", 2020-08-07T15:13:04.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1477999999999996e-05 2020-08-07T15:13:04.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:04.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227760+0000", 2020-08-07T15:13:04.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016657480000000001 2020-08-07T15:13:04.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229426+0000", 2020-08-07T15:13:04.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061290400000000005 2020-08-07T15:13:04.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230039+0000", 2020-08-07T15:13:04.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2e-07 2020-08-07T15:13:04.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:04.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230039+0000", 2020-08-07T15:13:04.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018643500000000001 2020-08-07T15:13:04.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:04.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230225+0000", 2020-08-07T15:13:04.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001896506 2020-08-07T15:13:04.419 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.420 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:04.420 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.420 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73088 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:13:04.421 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:13:04.421 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 216.70796500599999, 2020-08-07T15:13:04.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001627153, 2020-08-07T15:13:04.421 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:04.421 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:04.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:04.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:04.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:04.422 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73088 2020-08-07T15:13:04.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.423 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:04.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:04.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:13:04.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:04.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:04.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:13:04.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7741000000000003e-05 2020-08-07T15:13:04.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:04.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228767+0000", 2020-08-07T15:13:04.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6409999999999999e-06 2020-08-07T15:13:04.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:04.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228773+0000", 2020-08-07T15:13:04.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0874e-05 2020-08-07T15:13:04.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:04.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228784+0000", 2020-08-07T15:13:04.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1529999999999996e-06 2020-08-07T15:13:04.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:04.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228789+0000", 2020-08-07T15:13:04.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3945e-05 2020-08-07T15:13:04.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228833+0000", 2020-08-07T15:13:04.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4020000000000002e-06 2020-08-07T15:13:04.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:04.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228841+0000", 2020-08-07T15:13:04.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98996913900000005 2020-08-07T15:13:04.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218810+0000", 2020-08-07T15:13:04.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112484 2020-08-07T15:13:04.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:04.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218923+0000", 2020-08-07T15:13:04.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019891 2020-08-07T15:13:04.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:13:04.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.219122+0000", 2020-08-07T15:13:04.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027884099999999998 2020-08-07T15:13:04.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:04.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.221910+0000", 2020-08-07T15:13:04.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058753399999999997 2020-08-07T15:13:04.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227786+0000", 2020-08-07T15:13:04.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9200000000000002e-07 2020-08-07T15:13:04.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:04.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227786+0000", 2020-08-07T15:13:04.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8572000000000003e-05 2020-08-07T15:13:04.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:04.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227824+0000", 2020-08-07T15:13:04.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001651296 2020-08-07T15:13:04.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229476+0000", 2020-08-07T15:13:04.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00077620399999999996 2020-08-07T15:13:04.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:04.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230252+0000", 2020-08-07T15:13:04.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1600000000000002e-07 2020-08-07T15:13:04.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:04.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230252+0000", 2020-08-07T15:13:04.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4433999999999997e-05 2020-08-07T15:13:04.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:04.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230347+0000", 2020-08-07T15:13:04.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001627153 2020-08-07T15:13:04.442 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.442 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:04.442 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.443 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:128738 4.9 4.370c6199 (undecoded) ondisk+write+known_if_redirected e155)", 2020-08-07T15:13:04.443 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:03.204387+0000", 2020-08-07T15:13:04.443 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 180.73229735300001, 2020-08-07T15:13:04.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99077720899999999, 2020-08-07T15:13:04.443 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:04.444 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:04.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:04.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:04.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:04.444 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128738 2020-08-07T15:13:04.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.445 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:04.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:04.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204387+0000", 2020-08-07T15:13:04.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:04.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:04.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204387+0000", 2020-08-07T15:13:04.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4561999999999999e-05 2020-08-07T15:13:04.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:04.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204422+0000", 2020-08-07T15:13:04.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7270000000000001e-06 2020-08-07T15:13:04.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:04.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204428+0000", 2020-08-07T15:13:04.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3451e-05 2020-08-07T15:13:04.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:04.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204442+0000", 2020-08-07T15:13:04.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1589999999999998e-06 2020-08-07T15:13:04.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:04.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204448+0000", 2020-08-07T15:13:04.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3755999999999999e-05 2020-08-07T15:13:04.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204492+0000", 2020-08-07T15:13:04.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2450000000000001e-06 2020-08-07T15:13:04.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:04.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204499+0000", 2020-08-07T15:13:04.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99065224600000001 2020-08-07T15:13:04.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195151+0000", 2020-08-07T15:13:04.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3063e-05 2020-08-07T15:13:04.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:04.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195164+0000", 2020-08-07T15:13:04.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99077720899999999 2020-08-07T15:13:04.455 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.455 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:04.455 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.456 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:128742 4.9 4.69b70ab9 (undecoded) ondisk+write+known_if_redirected e155)", 2020-08-07T15:13:04.456 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:03.205224+0000", 2020-08-07T15:13:04.456 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 180.73146071900001, 2020-08-07T15:13:04.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.989964548, 2020-08-07T15:13:04.456 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:04.457 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:04.457 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:04.457 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:04.457 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:04.457 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128742 2020-08-07T15:13:04.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.458 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:04.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:04.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205224+0000", 2020-08-07T15:13:04.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:04.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:04.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205224+0000", 2020-08-07T15:13:04.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1018e-05 2020-08-07T15:13:04.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:04.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205255+0000", 2020-08-07T15:13:04.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5840000000000002e-06 2020-08-07T15:13:04.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:04.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205260+0000", 2020-08-07T15:13:04.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2388999999999999e-05 2020-08-07T15:13:04.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:04.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205273+0000", 2020-08-07T15:13:04.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3799999999999999e-06 2020-08-07T15:13:04.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:04.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205279+0000", 2020-08-07T15:13:04.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6684999999999998e-05 2020-08-07T15:13:04.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205326+0000", 2020-08-07T15:13:04.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6200000000000004e-06 2020-08-07T15:13:04.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:04.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205332+0000", 2020-08-07T15:13:04.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98984944600000002 2020-08-07T15:13:04.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195181+0000", 2020-08-07T15:13:04.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4259999999999997e-06 2020-08-07T15:13:04.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:04.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195188+0000", 2020-08-07T15:13:04.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.989964548 2020-08-07T15:13:04.467 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.468 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:04.468 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.468 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:128743 4.9 4.1824cba9 (undecoded) ondisk+write+known_if_redirected e155)", 2020-08-07T15:13:04.469 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:03.205500+0000", 2020-08-07T15:13:04.469 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 180.73118485699999, 2020-08-07T15:13:04.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98971761999999996, 2020-08-07T15:13:04.469 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:04.469 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:04.469 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:04.470 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:04.470 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:04.470 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128743 2020-08-07T15:13:04.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.470 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:04.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:04.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205500+0000", 2020-08-07T15:13:04.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:04.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:04.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205500+0000", 2020-08-07T15:13:04.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6441000000000001e-05 2020-08-07T15:13:04.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:04.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205516+0000", 2020-08-07T15:13:04.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4429999999999999e-06 2020-08-07T15:13:04.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:04.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205520+0000", 2020-08-07T15:13:04.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4590000000000003e-06 2020-08-07T15:13:04.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:04.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205526+0000", 2020-08-07T15:13:04.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0429999999999999e-06 2020-08-07T15:13:04.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:04.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205529+0000", 2020-08-07T15:13:04.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1313000000000003e-05 2020-08-07T15:13:04.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205560+0000", 2020-08-07T15:13:04.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.934e-06 2020-08-07T15:13:04.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:04.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205567+0000", 2020-08-07T15:13:04.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98964017599999998 2020-08-07T15:13:04.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:04.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195208+0000", 2020-08-07T15:13:04.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8109999999999997e-06 2020-08-07T15:13:04.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:04.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:04.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:04.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195217+0000", 2020-08-07T15:13:04.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98971761999999996 2020-08-07T15:13:04.480 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.480 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:04.480 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.481 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:04.481 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:04.481 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:04.483 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:13:04.484 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:13:04.484 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:13:04.484 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:13:04.484 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:13:04.485 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:13:04.485 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:13:04.485 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:13:04.485 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:13:04.485 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:13:04.486 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:13:04.486 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:13:04.486 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:13:04.486 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:13:04.486 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:13:04.487 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:13:04.487 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 52 16 76919 76903 11.5527 11.8984 0.00431143 0.0108146 2020-08-07T15:13:04.487 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:04.481+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:04.488 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:13:04.594 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:04.588+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:04.662 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:04.663 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:04.673 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:13:04.684 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:04.682+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:04.785 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:04.783+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:04.839 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:04.839 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:04.851 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:13:04.887 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:04.885+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:04.994 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:04.986+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:05.017 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:05.018 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:05.030 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:13:05.089 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:05.087+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:05.192 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:05.188+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:05.219 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:05.220 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:05.230 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:13:05.270 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 53 16 78436 78420 11.5582 11.8516 0.00375212 0.010811 2020-08-07T15:13:05.319 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:05.314+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:05.362 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:13:05.391 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:05.390+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:05.432 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:05.432 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:05.444 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:13:05.533 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:05.527+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:05.597 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:05.595+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:05.650 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:05.650 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:05.662 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:13:05.710 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:05.705+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:05.810 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:05.804+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:05.859 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:05.860 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:05.872 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:13:05.915 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:05.913+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:06.008 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:06.007+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:06.105 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:06.105 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:06.109 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:06.107+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:06.114 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:13:06.214 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:06.209+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:06.270 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 54 16 79938 79922 11.5615 11.7344 0.00277588 0.0108048 2020-08-07T15:13:06.312 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:06.310+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:06.331 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:06.332 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:06.345 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:13:06.423 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:06.411+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:06.488 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:13:06.545 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:06.540+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:06.575 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:06.575 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:06.590 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:13:06.613 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2020-08-07T15:13:06.614 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 0 1 2020-08-07T15:13:06.618 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:13:06.638 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:06.622+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:06.736 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:06.729+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:06.750 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:13:06.830 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:06.830 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:13:06.830 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:13:06.830 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:13:06.831 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:06.839 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:06.833+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:06.847 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:13:06.938 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:06.935+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:07.005 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:13:07.041 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:07.036+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:07.062 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:13:07.085 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:07.086 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:13:07.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:13:07.086 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:13:07.086 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.087 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:145801 3.6 e75/26 3:69e4cb22:::benchmark_data_smithi074_35447_object16192:head v 75'9405, mlcod=75'9401)", 2020-08-07T15:13:07.087 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:13:07.087 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 292.40872833499998, 2020-08-07T15:13:07.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639, 2020-08-07T15:13:07.087 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:07.088 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:07.088 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:07.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:07.088 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:13:07.089 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:07.089 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.089 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.089 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:07.089 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:13:07.099 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6511000000000002e-05 2020-08-07T15:13:07.099 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.099 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.099 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:07.100 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671803+0000", 2020-08-07T15:13:07.101 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7190000000000004e-06 2020-08-07T15:13:07.101 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.102 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.103 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:07.103 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671809+0000", 2020-08-07T15:13:07.104 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74210928300000001 2020-08-07T15:13:07.104 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.110 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.110 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:07.110 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413919+0000", 2020-08-07T15:13:07.111 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1759999999999999e-05 2020-08-07T15:13:07.111 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.112 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:07.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413930+0000", 2020-08-07T15:13:07.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000108601 2020-08-07T15:13:07.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.113 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.113 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414039+0000", 2020-08-07T15:13:07.113 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9948999999999999e-05 2020-08-07T15:13:07.113 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.114 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:07.114 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414089+0000", 2020-08-07T15:13:07.114 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00079656499999999995 2020-08-07T15:13:07.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.115 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.137 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:07.137 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414885+0000", 2020-08-07T15:13:07.137 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2250999999999998e-05 2020-08-07T15:13:07.138 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.138 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.138 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:07.138 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414938+0000", 2020-08-07T15:13:07.138 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639 2020-08-07T15:13:07.138 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.139 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:07.139 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.139 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.153 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.153 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:104104 4.0 e146/131 4:0ffb764a:::benchmark_data_smithi074_44586_object13012:head v 146'6960, mlcod=146'6953)", 2020-08-07T15:13:07.153 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:13:07.153 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 198.882181559, 2020-08-07T15:13:07.154 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75443331599999996, 2020-08-07T15:13:07.154 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:07.154 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:07.154 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:07.154 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.155 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:07.155 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:13:07.155 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:07.155 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.155 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.156 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:07.156 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:13:07.156 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1743000000000002e-05 2020-08-07T15:13:07.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.157 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:07.168 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198355+0000", 2020-08-07T15:13:07.172 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1249999999999998e-06 2020-08-07T15:13:07.172 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.172 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.173 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:07.173 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198361+0000", 2020-08-07T15:13:07.179 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74719179599999996 2020-08-07T15:13:07.179 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.179 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.179 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:07.179 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945553+0000", 2020-08-07T15:13:07.180 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6800000000000001e-06 2020-08-07T15:13:07.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.180 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.180 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:07.180 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945557+0000", 2020-08-07T15:13:07.181 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012101500000000001 2020-08-07T15:13:07.181 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.181 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.181 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.181 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945678+0000", 2020-08-07T15:13:07.182 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6621000000000001e-05 2020-08-07T15:13:07.182 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:07.182 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945705+0000", 2020-08-07T15:13:07.183 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069879540000000002 2020-08-07T15:13:07.183 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.183 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.183 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:07.183 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.952693+0000", 2020-08-07T15:13:07.183 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3381999999999999e-05 2020-08-07T15:13:07.184 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.200 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.200 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:07.200 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.952736+0000", 2020-08-07T15:13:07.201 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75443331599999996 2020-08-07T15:13:07.201 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.201 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:07.201 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.201 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.202 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.202 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89641 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:07.202 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.550933+0000", 2020-08-07T15:13:07.202 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 105.529551138, 2020-08-07T15:13:07.202 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69051440900000005, 2020-08-07T15:13:07.203 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:07.206 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:07.206 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:07.206 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:07.209 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:07.209 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89641 2020-08-07T15:13:07.215 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.215 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:07.216 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.216 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:07.216 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550933+0000", 2020-08-07T15:13:07.216 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:07.216 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.217 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.217 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:07.217 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550933+0000", 2020-08-07T15:13:07.217 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6520000000000002e-05 2020-08-07T15:13:07.217 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.218 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.218 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:07.218 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550980+0000", 2020-08-07T15:13:07.218 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2410000000000003e-06 2020-08-07T15:13:07.218 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.219 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.219 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:07.219 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550985+0000", 2020-08-07T15:13:07.219 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2707e-05 2020-08-07T15:13:07.219 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.219 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.220 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:07.220 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550998+0000", 2020-08-07T15:13:07.240 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8629999999999996e-06 2020-08-07T15:13:07.240 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.240 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.241 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:07.241 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551004+0000", 2020-08-07T15:13:07.241 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4987000000000001e-05 2020-08-07T15:13:07.241 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.241 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.242 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.242 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551059+0000", 2020-08-07T15:13:07.242 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1718e-05 2020-08-07T15:13:07.243 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.243 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.243 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:07.243 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551070+0000", 2020-08-07T15:13:07.244 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67720431000000003 2020-08-07T15:13:07.244 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.244 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.244 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.244 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.228275+0000", 2020-08-07T15:13:07.245 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000731965 2020-08-07T15:13:07.245 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.245 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.245 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:07.286 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229007+0000", 2020-08-07T15:13:07.286 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00025396199999999999 2020-08-07T15:13:07.286 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.286 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.287 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:07.287 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229261+0000", 2020-08-07T15:13:07.287 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002511189 2020-08-07T15:13:07.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.288 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.288 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:07.288 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231772+0000", 2020-08-07T15:13:07.289 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3399999999999997e-07 2020-08-07T15:13:07.289 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.289 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.290 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:07.290 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231773+0000", 2020-08-07T15:13:07.290 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0274e-05 2020-08-07T15:13:07.290 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.291 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.291 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:07.291 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231823+0000", 2020-08-07T15:13:07.291 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0094095440000000006 2020-08-07T15:13:07.291 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.292 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.292 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.292 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241233+0000", 2020-08-07T15:13:07.292 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4011999999999998e-05 2020-08-07T15:13:07.292 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.292 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.293 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.293 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241277+0000", 2020-08-07T15:13:07.293 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7199999999999999e-07 2020-08-07T15:13:07.293 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.293 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.294 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:07.294 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241277+0000", 2020-08-07T15:13:07.294 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017061100000000001 2020-08-07T15:13:07.294 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.294 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.295 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:07.295 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241448+0000", 2020-08-07T15:13:07.295 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69051440900000005 2020-08-07T15:13:07.295 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.295 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:07.296 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.296 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.296 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.296 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89642 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:07.296 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.551807+0000", 2020-08-07T15:13:07.297 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 105.528677586, 2020-08-07T15:13:07.297 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68991613100000004, 2020-08-07T15:13:07.297 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:07.297 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:07.297 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:07.298 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:07.298 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:07.298 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89642 2020-08-07T15:13:07.298 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.298 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:07.299 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.299 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:07.299 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551807+0000", 2020-08-07T15:13:07.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:07.299 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.299 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.300 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:07.300 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551807+0000", 2020-08-07T15:13:07.300 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7049000000000001e-05 2020-08-07T15:13:07.300 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.300 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.301 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:07.301 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551834+0000", 2020-08-07T15:13:07.301 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9249999999999997e-06 2020-08-07T15:13:07.301 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.301 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.302 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:07.302 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551838+0000", 2020-08-07T15:13:07.302 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9160000000000001e-06 2020-08-07T15:13:07.302 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.302 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.303 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:07.303 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551846+0000", 2020-08-07T15:13:07.303 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3510000000000002e-06 2020-08-07T15:13:07.303 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.303 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.304 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:07.304 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551849+0000", 2020-08-07T15:13:07.304 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9276000000000001e-05 2020-08-07T15:13:07.304 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.304 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.305 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.305 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551898+0000", 2020-08-07T15:13:07.305 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3095000000000001e-05 2020-08-07T15:13:07.305 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.305 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.306 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:07.306 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551911+0000", 2020-08-07T15:13:07.306 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67789159799999998 2020-08-07T15:13:07.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.338 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.338 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.339 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229803+0000", 2020-08-07T15:13:07.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8069000000000001e-05 2020-08-07T15:13:07.339 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.339 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.339 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:07.340 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229881+0000", 2020-08-07T15:13:07.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000152917 2020-08-07T15:13:07.340 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.340 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:07.341 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230034+0000", 2020-08-07T15:13:07.341 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044936760000000003 2020-08-07T15:13:07.341 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.341 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:07.342 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234528+0000", 2020-08-07T15:13:07.342 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8999999999999995e-07 2020-08-07T15:13:07.349 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.350 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.350 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:07.350 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234529+0000", 2020-08-07T15:13:07.351 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7741000000000002e-05 2020-08-07T15:13:07.351 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.351 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.351 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:07.352 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234586+0000", 2020-08-07T15:13:07.352 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069143959999999997 2020-08-07T15:13:07.352 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.352 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.353 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.353 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241501+0000", 2020-08-07T15:13:07.362 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1433000000000001e-05 2020-08-07T15:13:07.362 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.363 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.363 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.363 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241542+0000", 2020-08-07T15:13:07.364 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7800000000000001e-07 2020-08-07T15:13:07.364 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:07.365 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241543+0000", 2020-08-07T15:13:07.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000180321 2020-08-07T15:13:07.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:07.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241723+0000", 2020-08-07T15:13:07.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68991613100000004 2020-08-07T15:13:07.380 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.380 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:07.380 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.401 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89643 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:07.402 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.552181+0000", 2020-08-07T15:13:07.414 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 105.528303592, 2020-08-07T15:13:07.414 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68980872400000004, 2020-08-07T15:13:07.414 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:07.424 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:07.425 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:07.425 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:07.425 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:07.425 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89643 2020-08-07T15:13:07.426 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.426 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:07.426 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.426 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:07.426 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552181+0000", 2020-08-07T15:13:07.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:07.427 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.427 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:07.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552181+0000", 2020-08-07T15:13:07.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0926999999999997e-05 2020-08-07T15:13:07.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:07.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552212+0000", 2020-08-07T15:13:07.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0512e-05 2020-08-07T15:13:07.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.429 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:07.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552222+0000", 2020-08-07T15:13:07.434 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9379999999999992e-06 2020-08-07T15:13:07.440 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.440 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:07.440 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552230+0000", 2020-08-07T15:13:07.441 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0260000000000001e-06 2020-08-07T15:13:07.441 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.441 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.441 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:07.442 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552234+0000", 2020-08-07T15:13:07.442 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6109000000000002e-05 2020-08-07T15:13:07.442 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.442 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.442 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.443 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552280+0000", 2020-08-07T15:13:07.443 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5318000000000001e-05 2020-08-07T15:13:07.443 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.443 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.443 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:07.443 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552296+0000", 2020-08-07T15:13:07.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67811269100000005 2020-08-07T15:13:07.444 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.445 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.445 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.445 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230408+0000", 2020-08-07T15:13:07.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000125518 2020-08-07T15:13:07.445 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.446 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.446 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:07.446 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230534+0000", 2020-08-07T15:13:07.446 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024637999999999998 2020-08-07T15:13:07.446 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.447 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:07.447 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230780+0000", 2020-08-07T15:13:07.447 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0066398869999999997 2020-08-07T15:13:07.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:07.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237420+0000", 2020-08-07T15:13:07.448 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3500000000000005e-07 2020-08-07T15:13:07.448 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.449 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.449 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:07.449 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237421+0000", 2020-08-07T15:13:07.449 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4005000000000003e-05 2020-08-07T15:13:07.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.450 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:07.450 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237465+0000", 2020-08-07T15:13:07.450 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043133529999999998 2020-08-07T15:13:07.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.451 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241778+0000", 2020-08-07T15:13:07.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0592e-05 2020-08-07T15:13:07.451 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.451 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241819+0000", 2020-08-07T15:13:07.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4999999999999998e-07 2020-08-07T15:13:07.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.453 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:07.453 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241819+0000", 2020-08-07T15:13:07.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017018299999999999 2020-08-07T15:13:07.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.454 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:07.454 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241990+0000", 2020-08-07T15:13:07.454 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68980872400000004 2020-08-07T15:13:07.454 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.454 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:07.455 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.456 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89644 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:07.456 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.552384+0000", 2020-08-07T15:13:07.456 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 105.52810055800001, 2020-08-07T15:13:07.456 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68997479299999998, 2020-08-07T15:13:07.457 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:07.457 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:07.457 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:07.457 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:07.458 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:07.458 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89644 2020-08-07T15:13:07.458 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.458 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:07.458 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:07.459 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552384+0000", 2020-08-07T15:13:07.459 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:07.459 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.460 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.460 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:07.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552384+0000", 2020-08-07T15:13:07.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9238999999999999e-05 2020-08-07T15:13:07.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.461 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.461 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:07.461 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552413+0000", 2020-08-07T15:13:07.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3340000000000003e-06 2020-08-07T15:13:07.461 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.462 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.462 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:07.462 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552417+0000", 2020-08-07T15:13:07.462 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3119999999999994e-06 2020-08-07T15:13:07.462 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.462 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.463 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:07.463 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552426+0000", 2020-08-07T15:13:07.463 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4139999999999996e-06 2020-08-07T15:13:07.463 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.463 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.464 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:07.464 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552430+0000", 2020-08-07T15:13:07.464 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6613999999999999e-05 2020-08-07T15:13:07.464 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.464 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.465 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.465 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:13:07.465 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7280000000000007e-06 2020-08-07T15:13:07.466 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.466 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.466 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:07.467 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552477+0000", 2020-08-07T15:13:07.467 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67873990399999995 2020-08-07T15:13:07.467 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.467 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.467 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.468 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231216+0000", 2020-08-07T15:13:07.468 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3629999999999998e-05 2020-08-07T15:13:07.468 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.468 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.468 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:07.469 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231290+0000", 2020-08-07T15:13:07.469 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000132108 2020-08-07T15:13:07.469 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.469 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.469 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:07.469 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231422+0000", 2020-08-07T15:13:07.470 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006049652 2020-08-07T15:13:07.470 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.470 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.470 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:07.470 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237472+0000", 2020-08-07T15:13:07.471 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000005e-07 2020-08-07T15:13:07.471 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.471 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.471 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:07.471 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237472+0000", 2020-08-07T15:13:07.472 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2848000000000001e-05 2020-08-07T15:13:07.472 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.472 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.472 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:07.472 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237505+0000", 2020-08-07T15:13:07.472 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045352179999999997 2020-08-07T15:13:07.473 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.473 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.473 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.473 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242040+0000", 2020-08-07T15:13:07.473 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012843699999999999 2020-08-07T15:13:07.474 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.474 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.474 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.474 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242169+0000", 2020-08-07T15:13:07.474 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9999999999999998e-07 2020-08-07T15:13:07.475 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.475 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.475 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:07.475 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242169+0000", 2020-08-07T15:13:07.475 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000189354 2020-08-07T15:13:07.476 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.476 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.476 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:07.477 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242359+0000", 2020-08-07T15:13:07.477 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68997479299999998 2020-08-07T15:13:07.477 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.477 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:07.477 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.478 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.478 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.478 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89645 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:07.478 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:13:07.478 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 105.528017776, 2020-08-07T15:13:07.479 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69011194600000003, 2020-08-07T15:13:07.479 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:07.479 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:07.479 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:07.479 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:07.480 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:07.480 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89645 2020-08-07T15:13:07.480 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.480 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:07.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.481 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:07.481 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:13:07.481 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:07.481 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.481 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.482 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:07.482 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:13:07.482 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6750999999999998e-05 2020-08-07T15:13:07.482 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.482 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.483 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:07.483 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552493+0000", 2020-08-07T15:13:07.483 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0350000000000002e-06 2020-08-07T15:13:07.483 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.483 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.483 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:07.484 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552496+0000", 2020-08-07T15:13:07.484 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3029999999999996e-06 2020-08-07T15:13:07.484 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.484 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.484 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:07.485 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552503+0000", 2020-08-07T15:13:07.485 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7580000000000001e-06 2020-08-07T15:13:07.485 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.485 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.485 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:07.486 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552506+0000", 2020-08-07T15:13:07.487 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4868000000000003e-05 2020-08-07T15:13:07.487 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.487 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.488 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552570+0000", 2020-08-07T15:13:07.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9649999999999997e-06 2020-08-07T15:13:07.488 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:07.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552579+0000", 2020-08-07T15:13:07.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67928357699999997 2020-08-07T15:13:07.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.489 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.489 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.490 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231863+0000", 2020-08-07T15:13:07.490 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000115696 2020-08-07T15:13:07.490 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.490 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:07.493 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231979+0000", 2020-08-07T15:13:07.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020456599999999999 2020-08-07T15:13:07.494 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.494 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:07.494 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232183+0000", 2020-08-07T15:13:07.502 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005327923 2020-08-07T15:13:07.503 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.503 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.503 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:07.503 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237511+0000", 2020-08-07T15:13:07.503 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8999999999999998e-07 2020-08-07T15:13:07.504 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.504 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.504 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:07.504 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237511+0000", 2020-08-07T15:13:07.504 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6019000000000002e-05 2020-08-07T15:13:07.505 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.505 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.505 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:07.505 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237547+0000", 2020-08-07T15:13:07.505 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045340980000000003 2020-08-07T15:13:07.505 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.506 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.506 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.506 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242082+0000", 2020-08-07T15:13:07.506 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00032620499999999998 2020-08-07T15:13:07.506 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.507 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.507 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.507 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242408+0000", 2020-08-07T15:13:07.507 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3500000000000002e-07 2020-08-07T15:13:07.507 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.508 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.508 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:07.508 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242408+0000", 2020-08-07T15:13:07.508 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000170457 2020-08-07T15:13:07.508 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.509 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.509 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:07.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242579+0000", 2020-08-07T15:13:07.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69011194600000003 2020-08-07T15:13:07.509 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.510 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:07.510 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.510 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.510 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89646 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:07.510 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.557244+0000", 2020-08-07T15:13:07.511 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 105.523240004, 2020-08-07T15:13:07.511 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68559352299999998, 2020-08-07T15:13:07.511 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:07.511 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:07.511 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:07.512 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:07.512 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:07.512 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89646 2020-08-07T15:13:07.513 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.513 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:07.513 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.513 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:07.514 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557244+0000", 2020-08-07T15:13:07.514 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:07.514 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.514 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:07.515 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557244+0000", 2020-08-07T15:13:07.515 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9699000000000001e-05 2020-08-07T15:13:07.515 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.515 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.515 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:07.516 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557274+0000", 2020-08-07T15:13:07.516 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1269999999999996e-06 2020-08-07T15:13:07.516 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.516 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.516 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:07.517 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557278+0000", 2020-08-07T15:13:07.517 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0099999999999995e-06 2020-08-07T15:13:07.517 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.517 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.517 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:07.518 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557286+0000", 2020-08-07T15:13:07.518 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7439999999999998e-06 2020-08-07T15:13:07.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.518 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:07.519 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557291+0000", 2020-08-07T15:13:07.519 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5927999999999999e-05 2020-08-07T15:13:07.519 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.519 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.519 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.520 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557327+0000", 2020-08-07T15:13:07.520 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5069999999999997e-06 2020-08-07T15:13:07.520 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.520 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:07.520 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557333+0000", 2020-08-07T15:13:07.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67527419200000005 2020-08-07T15:13:07.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.521 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.521 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.521 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232608+0000", 2020-08-07T15:13:07.522 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8699e-05 2020-08-07T15:13:07.522 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.522 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.522 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:07.522 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232686+0000", 2020-08-07T15:13:07.523 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018185099999999999 2020-08-07T15:13:07.523 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.524 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.524 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:07.524 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232868+0000", 2020-08-07T15:13:07.524 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0075819360000000001 2020-08-07T15:13:07.524 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.525 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.525 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:07.525 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240450+0000", 2020-08-07T15:13:07.525 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2799999999999995e-07 2020-08-07T15:13:07.525 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.526 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.526 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:07.526 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240451+0000", 2020-08-07T15:13:07.531 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3751000000000002e-05 2020-08-07T15:13:07.532 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.532 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.532 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:07.532 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240495+0000", 2020-08-07T15:13:07.533 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001628064 2020-08-07T15:13:07.533 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.533 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.533 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.534 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242123+0000", 2020-08-07T15:13:07.534 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00050955599999999998 2020-08-07T15:13:07.534 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.535 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242632+0000", 2020-08-07T15:13:07.535 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2900000000000004e-07 2020-08-07T15:13:07.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:07.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242633+0000", 2020-08-07T15:13:07.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000205202 2020-08-07T15:13:07.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.537 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.537 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:07.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242838+0000", 2020-08-07T15:13:07.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68559352299999998 2020-08-07T15:13:07.537 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.538 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:07.538 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.538 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.539 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.539 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89647 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:07.539 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.557315+0000", 2020-08-07T15:13:07.539 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 105.523169681, 2020-08-07T15:13:07.539 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68581815800000001, 2020-08-07T15:13:07.540 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:07.540 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:07.540 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:07.540 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:07.540 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:07.541 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89647 2020-08-07T15:13:07.541 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.541 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:07.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.541 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:07.542 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557315+0000", 2020-08-07T15:13:07.542 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:07.542 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.542 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.542 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:07.542 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557315+0000", 2020-08-07T15:13:07.543 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6639999999999999e-05 2020-08-07T15:13:07.543 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.543 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.543 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:07.543 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557341+0000", 2020-08-07T15:13:07.544 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7989999999999998e-06 2020-08-07T15:13:07.544 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.545 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:07.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557344+0000", 2020-08-07T15:13:07.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6960000000000002e-06 2020-08-07T15:13:07.545 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.546 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.546 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:07.546 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557350+0000", 2020-08-07T15:13:07.546 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6089999999999999e-06 2020-08-07T15:13:07.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.547 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:07.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557352+0000", 2020-08-07T15:13:07.547 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1837e-05 2020-08-07T15:13:07.547 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.548 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.548 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557384+0000", 2020-08-07T15:13:07.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8099999999999998e-06 2020-08-07T15:13:07.548 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.549 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:07.549 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557392+0000", 2020-08-07T15:13:07.549 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67581117300000004 2020-08-07T15:13:07.549 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.549 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.550 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.550 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233203+0000", 2020-08-07T15:13:07.550 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112316 2020-08-07T15:13:07.550 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.550 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.551 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:07.551 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233316+0000", 2020-08-07T15:13:07.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000221766 2020-08-07T15:13:07.551 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.551 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.552 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:07.552 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233537+0000", 2020-08-07T15:13:07.552 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069695019999999998 2020-08-07T15:13:07.552 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.553 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.553 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:07.553 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240507+0000", 2020-08-07T15:13:07.554 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3999999999999997e-07 2020-08-07T15:13:07.554 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.554 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:07.557 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240507+0000", 2020-08-07T15:13:07.557 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8844000000000003e-05 2020-08-07T15:13:07.557 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.558 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.558 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:07.558 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240546+0000", 2020-08-07T15:13:07.559 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023473399999999998 2020-08-07T15:13:07.559 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.560 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.560 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.560 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242893+0000", 2020-08-07T15:13:07.561 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5410000000000001e-05 2020-08-07T15:13:07.561 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.561 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.561 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.561 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242939+0000", 2020-08-07T15:13:07.562 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8000000000000001e-07 2020-08-07T15:13:07.562 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.562 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.564 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:07.571 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242939+0000", 2020-08-07T15:13:07.571 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019369600000000001 2020-08-07T15:13:07.572 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.572 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.572 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:07.572 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243133+0000", 2020-08-07T15:13:07.572 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68581815800000001 2020-08-07T15:13:07.573 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.573 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:07.573 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.573 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.573 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.574 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89648 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:07.574 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.557372+0000", 2020-08-07T15:13:07.574 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 105.52311256, 2020-08-07T15:13:07.574 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68606040000000001, 2020-08-07T15:13:07.575 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:07.575 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:07.575 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:07.575 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:07.575 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:07.576 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89648 2020-08-07T15:13:07.576 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.576 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:07.576 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.576 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:07.576 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557372+0000", 2020-08-07T15:13:07.577 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:07.577 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.577 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.577 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:07.577 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557372+0000", 2020-08-07T15:13:07.579 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5534e-05 2020-08-07T15:13:07.580 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.581 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.582 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:07.582 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557397+0000", 2020-08-07T15:13:07.582 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.599e-06 2020-08-07T15:13:07.582 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.582 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.583 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:07.583 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557400+0000", 2020-08-07T15:13:07.583 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.288e-06 2020-08-07T15:13:07.583 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.583 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.584 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:07.584 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557405+0000", 2020-08-07T15:13:07.584 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.142e-06 2020-08-07T15:13:07.584 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.584 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.585 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:07.585 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557407+0000", 2020-08-07T15:13:07.585 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0954999999999998e-05 2020-08-07T15:13:07.585 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.585 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.586 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.586 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557428+0000", 2020-08-07T15:13:07.587 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7529999999999998e-06 2020-08-07T15:13:07.587 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.587 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.587 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:07.587 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557434+0000", 2020-08-07T15:13:07.588 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67655471099999998 2020-08-07T15:13:07.588 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.588 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.588 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233989+0000", 2020-08-07T15:13:07.588 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8301999999999997e-05 2020-08-07T15:13:07.589 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.589 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:07.589 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234067+0000", 2020-08-07T15:13:07.590 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000129908 2020-08-07T15:13:07.592 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.592 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.592 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:07.593 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234197+0000", 2020-08-07T15:13:07.593 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063551249999999997 2020-08-07T15:13:07.593 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.593 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.593 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:07.594 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240552+0000", 2020-08-07T15:13:07.594 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9400000000000001e-07 2020-08-07T15:13:07.594 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.594 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.594 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:07.595 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240552+0000", 2020-08-07T15:13:07.595 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2787999999999998e-05 2020-08-07T15:13:07.595 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.595 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.595 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:07.596 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240585+0000", 2020-08-07T15:13:07.596 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002605652 2020-08-07T15:13:07.596 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.596 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.596 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.597 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243191+0000", 2020-08-07T15:13:07.597 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8690000000000002e-05 2020-08-07T15:13:07.597 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.597 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.597 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.598 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243250+0000", 2020-08-07T15:13:07.598 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4700000000000001e-07 2020-08-07T15:13:07.598 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.598 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.598 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:07.599 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243250+0000", 2020-08-07T15:13:07.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000181912 2020-08-07T15:13:07.599 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.599 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.599 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:07.599 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243432+0000", 2020-08-07T15:13:07.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68606040000000001 2020-08-07T15:13:07.600 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.601 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:07.601 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.602 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89689 5.7 5:e83ee43b:::benchmark_data_smithi074_53697_object11211:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:07.603 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.585493+0000", 2020-08-07T15:13:07.603 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 105.494991742, 2020-08-07T15:13:07.603 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66549445299999999, 2020-08-07T15:13:07.603 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:07.604 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:07.604 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:07.604 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:07.604 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:07.604 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89689 2020-08-07T15:13:07.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.605 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:07.605 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.605 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:07.605 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585493+0000", 2020-08-07T15:13:07.605 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:07.606 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.606 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.606 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:07.606 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585493+0000", 2020-08-07T15:13:07.606 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7746000000000002e-05 2020-08-07T15:13:07.607 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.607 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.607 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:07.607 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585520+0000", 2020-08-07T15:13:07.607 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1099999999999996e-06 2020-08-07T15:13:07.608 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.608 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.608 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:07.608 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585525+0000", 2020-08-07T15:13:07.608 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5359999999999998e-06 2020-08-07T15:13:07.609 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.609 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.609 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:07.609 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585534+0000", 2020-08-07T15:13:07.609 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5040000000000002e-06 2020-08-07T15:13:07.610 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.610 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.610 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:07.610 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585540+0000", 2020-08-07T15:13:07.610 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3548999999999998e-05 2020-08-07T15:13:07.611 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.611 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.611 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.612 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585593+0000", 2020-08-07T15:13:07.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2014000000000001e-05 2020-08-07T15:13:07.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.613 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:07.614 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585605+0000", 2020-08-07T15:13:07.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.64901847700000004 2020-08-07T15:13:07.614 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.614 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.614 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.615 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234624+0000", 2020-08-07T15:13:07.615 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000176329 2020-08-07T15:13:07.615 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.615 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.615 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:07.616 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234800+0000", 2020-08-07T15:13:07.616 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00025457300000000002 2020-08-07T15:13:07.616 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.616 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.616 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:07.617 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.235055+0000", 2020-08-07T15:13:07.617 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0055364669999999998 2020-08-07T15:13:07.617 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.617 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.617 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:07.618 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240591+0000", 2020-08-07T15:13:07.618 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3700000000000001e-07 2020-08-07T15:13:07.618 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.618 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.618 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:07.619 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240591+0000", 2020-08-07T15:13:07.619 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8479999999999997e-05 2020-08-07T15:13:07.619 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.619 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.619 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:07.620 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240640+0000", 2020-08-07T15:13:07.620 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00285564 2020-08-07T15:13:07.620 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.620 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.620 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.621 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243496+0000", 2020-08-07T15:13:07.621 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0073477400000000002 2020-08-07T15:13:07.621 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.622 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.622 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.622 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.250843+0000", 2020-08-07T15:13:07.624 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7300000000000005e-07 2020-08-07T15:13:07.624 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.624 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.624 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:07.624 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.250844+0000", 2020-08-07T15:13:07.625 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014307799999999999 2020-08-07T15:13:07.625 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.625 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.625 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:07.625 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.250987+0000", 2020-08-07T15:13:07.626 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66549445299999999 2020-08-07T15:13:07.626 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.626 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:07.626 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.626 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.627 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103152 5.7 5:e865d79a:::benchmark_data_smithi074_53697_object12893:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e211)", 2020-08-07T15:13:07.627 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.399967+0000", 2020-08-07T15:13:07.627 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 95.680517852999998, 2020-08-07T15:13:07.627 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81348256399999996, 2020-08-07T15:13:07.627 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:07.628 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:13:07.628 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:07.628 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:07.628 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:07.628 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103152 2020-08-07T15:13:07.629 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.629 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:07.629 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.629 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:07.629 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.399967+0000", 2020-08-07T15:13:07.630 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:07.630 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.630 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.630 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:07.630 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.399967+0000", 2020-08-07T15:13:07.631 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017528899999999999 2020-08-07T15:13:07.631 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.631 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.631 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:07.631 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400142+0000", 2020-08-07T15:13:07.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8239999999999996e-06 2020-08-07T15:13:07.632 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.633 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.634 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:07.634 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400148+0000", 2020-08-07T15:13:07.635 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0251000000000001e-05 2020-08-07T15:13:07.635 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.635 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.635 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:07.635 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400158+0000", 2020-08-07T15:13:07.636 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1579999999999998e-06 2020-08-07T15:13:07.636 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.636 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.636 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:07.636 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400162+0000", 2020-08-07T15:13:07.636 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00078404699999999996 2020-08-07T15:13:07.637 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.637 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.637 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.637 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400946+0000", 2020-08-07T15:13:07.637 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010605800000000001 2020-08-07T15:13:07.638 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.638 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.638 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:07.638 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.401052+0000", 2020-08-07T15:13:07.638 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020987499999999999 2020-08-07T15:13:07.639 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.639 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.639 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:07.639 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.401262+0000", 2020-08-07T15:13:07.639 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023707820000000001 2020-08-07T15:13:07.640 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.640 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.640 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:07.640 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.403633+0000", 2020-08-07T15:13:07.641 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.023e-06 2020-08-07T15:13:07.641 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.642 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.642 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:07.643 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.403634+0000", 2020-08-07T15:13:07.643 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3560999999999995e-05 2020-08-07T15:13:07.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.643 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.643 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:07.645 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.403717+0000", 2020-08-07T15:13:07.645 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011669689999999999 2020-08-07T15:13:07.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.646 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.404884+0000", 2020-08-07T15:13:07.646 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033332850000000001 2020-08-07T15:13:07.646 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.646 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.646 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.647 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.408218+0000", 2020-08-07T15:13:07.647 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0268000000000001e-05 2020-08-07T15:13:07.647 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.647 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.647 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:07.648 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.408228+0000", 2020-08-07T15:13:07.648 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80508180500000004 2020-08-07T15:13:07.648 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.648 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.648 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.649 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213310+0000", 2020-08-07T15:13:07.649 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000139369 2020-08-07T15:13:07.649 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.649 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.649 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:07.650 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213449+0000", 2020-08-07T15:13:07.650 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81348256399999996 2020-08-07T15:13:07.651 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.651 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:07.651 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.651 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.651 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.652 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103265 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:13:07.652 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478593+0000", 2020-08-07T15:13:07.652 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 95.601891796999993, 2020-08-07T15:13:07.652 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74922190499999997, 2020-08-07T15:13:07.652 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:07.652 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:07.654 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:07.654 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:07.655 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:07.655 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103265 2020-08-07T15:13:07.656 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.656 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:07.656 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.656 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:07.656 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478593+0000", 2020-08-07T15:13:07.657 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:07.657 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.657 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.657 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:07.657 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478593+0000", 2020-08-07T15:13:07.658 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9968000000000001e-05 2020-08-07T15:13:07.658 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.658 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.658 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:07.658 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478633+0000", 2020-08-07T15:13:07.659 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1420000000000003e-06 2020-08-07T15:13:07.659 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.659 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.659 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:07.659 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478637+0000", 2020-08-07T15:13:07.660 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2919999999999996e-06 2020-08-07T15:13:07.660 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.660 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.660 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:07.660 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478645+0000", 2020-08-07T15:13:07.660 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3699999999999997e-06 2020-08-07T15:13:07.661 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.661 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.661 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:07.661 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478649+0000", 2020-08-07T15:13:07.661 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1440000000000002e-05 2020-08-07T15:13:07.662 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.662 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.662 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.662 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478701+0000", 2020-08-07T15:13:07.662 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0831e-05 2020-08-07T15:13:07.663 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.663 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.663 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:07.664 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478712+0000", 2020-08-07T15:13:07.664 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73476696699999999 2020-08-07T15:13:07.664 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.664 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.666 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213479+0000", 2020-08-07T15:13:07.666 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000167026 2020-08-07T15:13:07.666 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.666 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.667 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:07.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213646+0000", 2020-08-07T15:13:07.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000179557 2020-08-07T15:13:07.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.667 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:07.668 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213825+0000", 2020-08-07T15:13:07.681 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041300360000000001 2020-08-07T15:13:07.681 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.681 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:07.691 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217955+0000", 2020-08-07T15:13:07.692 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6840000000000001e-06 2020-08-07T15:13:07.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.693 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:07.693 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217957+0000", 2020-08-07T15:13:07.693 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4758e-05 2020-08-07T15:13:07.694 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.694 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.694 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:07.695 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218012+0000", 2020-08-07T15:13:07.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0095519990000000003 2020-08-07T15:13:07.695 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.710 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.710 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227564+0000", 2020-08-07T15:13:07.711 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9956e-05 2020-08-07T15:13:07.711 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.712 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.712 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227624+0000", 2020-08-07T15:13:07.712 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3790000000000001e-06 2020-08-07T15:13:07.712 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.713 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.713 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:07.724 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227625+0000", 2020-08-07T15:13:07.725 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0001895 2020-08-07T15:13:07.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.725 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:07.726 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227815+0000", 2020-08-07T15:13:07.726 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74922190499999997 2020-08-07T15:13:07.726 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.727 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:07.727 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.727 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.727 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.728 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103266 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:13:07.731 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478693+0000", 2020-08-07T15:13:07.731 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 95.601791456000001, 2020-08-07T15:13:07.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74942137399999997, 2020-08-07T15:13:07.731 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:07.731 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:07.732 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:07.732 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:07.732 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:07.732 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103266 2020-08-07T15:13:07.733 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.733 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:07.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.734 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:07.734 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478693+0000", 2020-08-07T15:13:07.734 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:07.734 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.734 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.735 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:07.735 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478693+0000", 2020-08-07T15:13:07.735 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5658000000000002e-05 2020-08-07T15:13:07.735 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.735 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.736 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:07.736 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478739+0000", 2020-08-07T15:13:07.736 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.113e-06 2020-08-07T15:13:07.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.736 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.736 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:07.737 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478744+0000", 2020-08-07T15:13:07.737 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9690000000000006e-06 2020-08-07T15:13:07.737 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.737 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.737 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:07.738 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478754+0000", 2020-08-07T15:13:07.738 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1849999999999997e-06 2020-08-07T15:13:07.738 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.738 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.738 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:07.739 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478758+0000", 2020-08-07T15:13:07.740 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7024999999999998e-05 2020-08-07T15:13:07.740 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.740 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.741 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.742 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478805+0000", 2020-08-07T15:13:07.742 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1357e-05 2020-08-07T15:13:07.752 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.753 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.753 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:07.753 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478816+0000", 2020-08-07T15:13:07.753 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73554277499999998 2020-08-07T15:13:07.754 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.754 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.754 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.754 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.214359+0000", 2020-08-07T15:13:07.754 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000129145 2020-08-07T15:13:07.755 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.755 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:07.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.214488+0000", 2020-08-07T15:13:07.755 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00027129999999999998 2020-08-07T15:13:07.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:07.756 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.214760+0000", 2020-08-07T15:13:07.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032631079999999998 2020-08-07T15:13:07.758 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.758 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.759 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:07.759 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218023+0000", 2020-08-07T15:13:07.759 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1000000000000001e-06 2020-08-07T15:13:07.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.760 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:07.760 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218024+0000", 2020-08-07T15:13:07.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9085000000000006e-05 2020-08-07T15:13:07.761 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.761 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:07.761 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218103+0000", 2020-08-07T15:13:07.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.009779088 2020-08-07T15:13:07.763 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.763 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.764 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227882+0000", 2020-08-07T15:13:07.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1989999999999994e-05 2020-08-07T15:13:07.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227944+0000", 2020-08-07T15:13:07.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8299999999999996e-07 2020-08-07T15:13:07.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:07.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227945+0000", 2020-08-07T15:13:07.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000169693 2020-08-07T15:13:07.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:07.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228114+0000", 2020-08-07T15:13:07.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74942137399999997 2020-08-07T15:13:07.779 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.780 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:07.780 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.780 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.780 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103267 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:13:07.781 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478794+0000", 2020-08-07T15:13:07.781 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 95.601689968000002, 2020-08-07T15:13:07.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74960417300000004, 2020-08-07T15:13:07.782 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:07.782 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:07.783 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:07.787 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:07.788 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:07.788 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103267 2020-08-07T15:13:07.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.789 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:07.789 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.789 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:07.789 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478794+0000", 2020-08-07T15:13:07.789 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:07.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.790 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:07.790 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478794+0000", 2020-08-07T15:13:07.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6646e-05 2020-08-07T15:13:07.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:07.791 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478841+0000", 2020-08-07T15:13:07.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:13:07.793 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.793 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.793 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:07.793 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478846+0000", 2020-08-07T15:13:07.793 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0171e-05 2020-08-07T15:13:07.794 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.794 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.794 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:07.794 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478856+0000", 2020-08-07T15:13:07.794 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5929999999999999e-06 2020-08-07T15:13:07.795 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.795 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.795 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:07.795 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478861+0000", 2020-08-07T15:13:07.826 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2032e-05 2020-08-07T15:13:07.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.828 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478903+0000", 2020-08-07T15:13:07.829 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0312000000000001e-05 2020-08-07T15:13:07.829 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.836 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:07.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478913+0000", 2020-08-07T15:13:07.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73634238600000002 2020-08-07T15:13:07.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.838 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.215256+0000", 2020-08-07T15:13:07.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000124933 2020-08-07T15:13:07.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.840 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:07.840 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.215380+0000", 2020-08-07T15:13:07.840 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024624 2020-08-07T15:13:07.841 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:07.841 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.215627+0000", 2020-08-07T15:13:07.842 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0040879540000000004 2020-08-07T15:13:07.842 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.843 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:07.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219715+0000", 2020-08-07T15:13:07.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9230000000000001e-06 2020-08-07T15:13:07.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:07.845 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219717+0000", 2020-08-07T15:13:07.846 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8881999999999995e-05 2020-08-07T15:13:07.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.846 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.847 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:07.847 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219785+0000", 2020-08-07T15:13:07.847 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0083925119999999995 2020-08-07T15:13:07.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.858 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.858 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228178+0000", 2020-08-07T15:13:07.859 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2254000000000004e-05 2020-08-07T15:13:07.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.859 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228240+0000", 2020-08-07T15:13:07.860 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4499999999999996e-07 2020-08-07T15:13:07.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.861 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:07.862 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228241+0000", 2020-08-07T15:13:07.862 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000157656 2020-08-07T15:13:07.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:07.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228399+0000", 2020-08-07T15:13:07.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74960417300000004 2020-08-07T15:13:07.864 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.865 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:07.865 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.865 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103268 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:13:07.866 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478898+0000", 2020-08-07T15:13:07.866 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 95.601586251000001, 2020-08-07T15:13:07.867 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74981275700000005, 2020-08-07T15:13:07.867 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:07.867 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:07.867 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:07.867 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:07.868 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:07.868 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103268 2020-08-07T15:13:07.868 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.868 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:07.868 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.869 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:07.869 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478898+0000", 2020-08-07T15:13:07.869 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:07.869 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.869 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:07.885 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478898+0000", 2020-08-07T15:13:07.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2558999999999997e-05 2020-08-07T15:13:07.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.886 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.886 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:07.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478941+0000", 2020-08-07T15:13:07.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6469999999999997e-06 2020-08-07T15:13:07.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.890 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:07.895 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478945+0000", 2020-08-07T15:13:07.895 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3109999999999995e-06 2020-08-07T15:13:07.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.896 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:07.896 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478955+0000", 2020-08-07T15:13:07.896 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.799e-06 2020-08-07T15:13:07.897 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.897 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.897 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:07.897 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478958+0000", 2020-08-07T15:13:07.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5682000000000002e-05 2020-08-07T15:13:07.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.904 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.904 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:13:07.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3260000000000002e-06 2020-08-07T15:13:07.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:07.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479004+0000", 2020-08-07T15:13:07.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73703730499999998 2020-08-07T15:13:07.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.906 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.906 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.906 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216041+0000", 2020-08-07T15:13:07.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4445999999999998e-05 2020-08-07T15:13:07.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:07.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216115+0000", 2020-08-07T15:13:07.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013977 2020-08-07T15:13:07.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.910 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:07.910 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216255+0000", 2020-08-07T15:13:07.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0051267359999999998 2020-08-07T15:13:07.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:07.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221382+0000", 2020-08-07T15:13:07.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0310000000000001e-06 2020-08-07T15:13:07.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:07.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221384+0000", 2020-08-07T15:13:07.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8097999999999998e-05 2020-08-07T15:13:07.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:07.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221452+0000", 2020-08-07T15:13:07.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0070259249999999997 2020-08-07T15:13:07.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228478+0000", 2020-08-07T15:13:07.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7479999999999999e-05 2020-08-07T15:13:07.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228535+0000", 2020-08-07T15:13:07.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4399999999999999e-07 2020-08-07T15:13:07.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:07.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228536+0000", 2020-08-07T15:13:07.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017489799999999999 2020-08-07T15:13:07.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:07.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228711+0000", 2020-08-07T15:13:07.918 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74981275700000005 2020-08-07T15:13:07.918 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.918 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:07.918 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:07.918 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.920 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103269 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:13:07.921 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:13:07.921 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 95.601490143000007, 2020-08-07T15:13:07.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74992452799999998, 2020-08-07T15:13:07.921 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:07.922 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:07.922 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:07.922 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:07.922 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:07.922 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103269 2020-08-07T15:13:07.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.923 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:07.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:07.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:13:07.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:07.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:07.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:13:07.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5529999999999999e-05 2020-08-07T15:13:07.925 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:07.926 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479040+0000", 2020-08-07T15:13:07.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6550000000000003e-06 2020-08-07T15:13:07.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:07.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479044+0000", 2020-08-07T15:13:07.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4285e-05 2020-08-07T15:13:07.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:07.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479059+0000", 2020-08-07T15:13:07.931 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2050000000000001e-06 2020-08-07T15:13:07.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.932 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.932 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:07.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479064+0000", 2020-08-07T15:13:07.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0952999999999999e-05 2020-08-07T15:13:07.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.933 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.934 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.942 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479105+0000", 2020-08-07T15:13:07.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3559999999999998e-06 2020-08-07T15:13:07.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:07.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479111+0000", 2020-08-07T15:13:07.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.737508104 2020-08-07T15:13:07.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:07.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216619+0000", 2020-08-07T15:13:07.946 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000126367 2020-08-07T15:13:07.946 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.947 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:07.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216746+0000", 2020-08-07T15:13:07.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014069899999999999 2020-08-07T15:13:07.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:07.964 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216886+0000", 2020-08-07T15:13:07.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045753160000000003 2020-08-07T15:13:07.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:07.966 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221462+0000", 2020-08-07T15:13:07.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.142e-06 2020-08-07T15:13:07.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:07.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221463+0000", 2020-08-07T15:13:07.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7718e-05 2020-08-07T15:13:07.990 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.995 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.995 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:07.996 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221501+0000", 2020-08-07T15:13:07.996 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0072733210000000001 2020-08-07T15:13:07.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.996 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.996 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.997 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228774+0000", 2020-08-07T15:13:07.997 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3828e-05 2020-08-07T15:13:07.997 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.997 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.997 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:07.998 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228828+0000", 2020-08-07T15:13:07.998 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9800000000000003e-07 2020-08-07T15:13:07.998 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:07.998 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:07.998 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:07.999 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228828+0000", 2020-08-07T15:13:08.014 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0451000000000005e-05 2020-08-07T15:13:08.014 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.014 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:08.014 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228919+0000", 2020-08-07T15:13:08.015 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74992452799999998 2020-08-07T15:13:08.015 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:08.015 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:08.015 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:08.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.018 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103270 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:13:08.018 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.479168+0000", 2020-08-07T15:13:08.019 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 95.601316105999999, 2020-08-07T15:13:08.019 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75043371699999994, 2020-08-07T15:13:08.019 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:08.019 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:08.019 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:08.020 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:08.020 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:08.020 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103270 2020-08-07T15:13:08.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.020 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:08.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:08.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479168+0000", 2020-08-07T15:13:08.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:08.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:08.022 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479168+0000", 2020-08-07T15:13:08.022 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1557000000000002e-05 2020-08-07T15:13:08.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.023 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:08.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479210+0000", 2020-08-07T15:13:08.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9840000000000001e-06 2020-08-07T15:13:08.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:08.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479215+0000", 2020-08-07T15:13:08.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9539999999999999e-06 2020-08-07T15:13:08.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:08.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479225+0000", 2020-08-07T15:13:08.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.942e-06 2020-08-07T15:13:08.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:08.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479231+0000", 2020-08-07T15:13:08.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2493000000000002e-05 2020-08-07T15:13:08.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:08.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479263+0000", 2020-08-07T15:13:08.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6640000000000003e-06 2020-08-07T15:13:08.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:08.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479269+0000", 2020-08-07T15:13:08.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73802486499999997 2020-08-07T15:13:08.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.030 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:08.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217294+0000", 2020-08-07T15:13:08.030 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8640000000000006e-05 2020-08-07T15:13:08.030 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.030 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:08.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217372+0000", 2020-08-07T15:13:08.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000140338 2020-08-07T15:13:08.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:08.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217513+0000", 2020-08-07T15:13:08.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039952119999999997 2020-08-07T15:13:08.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:08.033 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221508+0000", 2020-08-07T15:13:08.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8899999999999999e-07 2020-08-07T15:13:08.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.034 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.034 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:08.034 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221509+0000", 2020-08-07T15:13:08.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8108999999999997e-05 2020-08-07T15:13:08.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:08.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221547+0000", 2020-08-07T15:13:08.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074232029999999997 2020-08-07T15:13:08.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:08.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228970+0000", 2020-08-07T15:13:08.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00046710099999999999 2020-08-07T15:13:08.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.039 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:08.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229437+0000", 2020-08-07T15:13:08.040 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5400000000000001e-07 2020-08-07T15:13:08.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:08.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229438+0000", 2020-08-07T15:13:08.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016441200000000001 2020-08-07T15:13:08.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.041 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:08.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229602+0000", 2020-08-07T15:13:08.042 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75043371699999994 2020-08-07T15:13:08.042 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:08.042 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:08.042 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:08.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.043 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103271 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:13:08.043 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.479356+0000", 2020-08-07T15:13:08.043 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 95.601128138000007, 2020-08-07T15:13:08.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75047653800000003, 2020-08-07T15:13:08.045 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:08.045 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:08.045 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:08.045 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:08.046 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:08.046 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103271 2020-08-07T15:13:08.046 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.046 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:08.046 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.047 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:08.047 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479356+0000", 2020-08-07T15:13:08.047 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:08.047 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.048 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.048 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:08.049 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479356+0000", 2020-08-07T15:13:08.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2416000000000002e-05 2020-08-07T15:13:08.049 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.049 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.049 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:08.050 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479399+0000", 2020-08-07T15:13:08.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0819999999999998e-06 2020-08-07T15:13:08.050 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.050 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.050 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:08.050 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479404+0000", 2020-08-07T15:13:08.051 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2486999999999999e-05 2020-08-07T15:13:08.051 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.051 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:08.051 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479416+0000", 2020-08-07T15:13:08.052 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3810000000000001e-06 2020-08-07T15:13:08.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.052 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.052 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:08.052 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479421+0000", 2020-08-07T15:13:08.053 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2311999999999998e-05 2020-08-07T15:13:08.053 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.053 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:08.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479453+0000", 2020-08-07T15:13:08.054 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7690000000000002e-06 2020-08-07T15:13:08.054 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.055 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:08.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479459+0000", 2020-08-07T15:13:08.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73869566499999995 2020-08-07T15:13:08.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:08.057 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218154+0000", 2020-08-07T15:13:08.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000124535 2020-08-07T15:13:08.058 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.059 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.059 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:08.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218279+0000", 2020-08-07T15:13:08.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000223441 2020-08-07T15:13:08.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.061 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:08.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218502+0000", 2020-08-07T15:13:08.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003054761 2020-08-07T15:13:08.070 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.070 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.070 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:08.071 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221557+0000", 2020-08-07T15:13:08.071 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0449999999999999e-06 2020-08-07T15:13:08.071 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.072 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.072 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:08.072 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221558+0000", 2020-08-07T15:13:08.073 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3655e-05 2020-08-07T15:13:08.073 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.086 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:08.087 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221622+0000", 2020-08-07T15:13:08.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.007661434 2020-08-07T15:13:08.091 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.091 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.091 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:08.091 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229283+0000", 2020-08-07T15:13:08.109 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00036023099999999997 2020-08-07T15:13:08.110 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.110 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.110 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:08.110 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229644+0000", 2020-08-07T15:13:08.111 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.63e-07 2020-08-07T15:13:08.111 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.111 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.111 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:08.111 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229644+0000", 2020-08-07T15:13:08.111 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018886099999999999 2020-08-07T15:13:08.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.112 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:08.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229833+0000", 2020-08-07T15:13:08.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75047653800000003 2020-08-07T15:13:08.113 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:08.113 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:08.113 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:08.113 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.113 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.114 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103272 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:13:08.114 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.481658+0000", 2020-08-07T15:13:08.114 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 95.598826715000001, 2020-08-07T15:13:08.114 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74920410900000001, 2020-08-07T15:13:08.114 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:08.115 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:08.115 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:08.115 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:08.115 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:08.115 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103272 2020-08-07T15:13:08.116 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.116 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:08.116 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.116 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:08.116 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481658+0000", 2020-08-07T15:13:08.117 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:08.117 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.117 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.118 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:08.118 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481658+0000", 2020-08-07T15:13:08.118 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8787e-05 2020-08-07T15:13:08.118 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.119 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.119 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:08.119 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481697+0000", 2020-08-07T15:13:08.121 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0559999999999998e-06 2020-08-07T15:13:08.121 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.121 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.121 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:08.122 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481701+0000", 2020-08-07T15:13:08.122 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1329999999999999e-06 2020-08-07T15:13:08.122 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.122 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.122 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:08.123 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481709+0000", 2020-08-07T15:13:08.123 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.189e-06 2020-08-07T15:13:08.123 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.123 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.123 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:08.124 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481713+0000", 2020-08-07T15:13:08.124 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6025999999999997e-05 2020-08-07T15:13:08.124 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.124 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.124 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:08.125 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481749+0000", 2020-08-07T15:13:08.125 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5619999999999998e-06 2020-08-07T15:13:08.125 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.126 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.126 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:08.126 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481756+0000", 2020-08-07T15:13:08.126 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73729155000000002 2020-08-07T15:13:08.126 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.127 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.127 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:08.127 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219048+0000", 2020-08-07T15:13:08.127 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000120387 2020-08-07T15:13:08.127 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.128 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.128 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:08.129 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219168+0000", 2020-08-07T15:13:08.129 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017865599999999999 2020-08-07T15:13:08.129 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.129 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.129 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:08.130 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219347+0000", 2020-08-07T15:13:08.130 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039049169999999999 2020-08-07T15:13:08.130 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.130 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.130 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:08.131 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.223252+0000", 2020-08-07T15:13:08.131 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.412e-06 2020-08-07T15:13:08.131 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.131 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.131 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:08.132 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.223253+0000", 2020-08-07T15:13:08.132 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7521999999999997e-05 2020-08-07T15:13:08.132 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.132 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.132 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:08.132 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.223301+0000", 2020-08-07T15:13:08.133 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00603478 2020-08-07T15:13:08.133 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.133 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.133 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:08.133 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229336+0000", 2020-08-07T15:13:08.134 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013309579999999999 2020-08-07T15:13:08.134 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.134 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.134 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:08.134 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.230667+0000", 2020-08-07T15:13:08.135 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7400000000000003e-07 2020-08-07T15:13:08.135 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.135 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.135 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:08.135 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.230667+0000", 2020-08-07T15:13:08.135 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019450000000000001 2020-08-07T15:13:08.136 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.136 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.136 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:08.136 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.230862+0000", 2020-08-07T15:13:08.136 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74920410900000001 2020-08-07T15:13:08.137 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:08.137 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:08.137 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:08.137 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:08.137 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:08.138 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:08.141 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:13:08.142 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:13:08.142 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:13:08.143 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:13:08.143 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:13:08.143 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:13:08.143 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 55 16 81412 81396 11.5606 11.5156 0.00598009 0.0108081 2020-08-07T15:13:08.144 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:13:08.144 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:13:08.144 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:13:08.144 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:13:08.144 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:13:08.145 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:13:08.145 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:13:08.145 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:13:08.145 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.0 to 1 (10000) 2020-08-07T15:13:08.146 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 1 1 2020-08-07T15:13:08.146 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:08.138+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:08.246 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:08.241+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:08.272 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 56 16 83071 83055 11.5856 12.9609 0.00743886 0.0107854 2020-08-07T15:13:08.354 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:08.352+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:08.455 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:08.453+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:08.556 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:08.555+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:08.641 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:13:08.661 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:08.657+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:08.703 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:08.711 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:13:08.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.711 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:83560 6.9 6:97386f15:::benchmark_data_smithi074_63843_object10444:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e272)", 2020-08-07T15:13:08.711 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:08.555100+0000", 2020-08-07T15:13:08.712 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.14636353499999999, 2020-08-07T15:13:08.712 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.14639941100000001, 2020-08-07T15:13:08.712 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:08.713 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:08.713 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:08.713 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:08.716 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:08.716 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 83560 2020-08-07T15:13:08.716 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.717 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:08.717 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.718 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:08.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.555100+0000", 2020-08-07T15:13:08.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:08.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:08.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.555100+0000", 2020-08-07T15:13:08.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3485000000000001e-05 2020-08-07T15:13:08.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.721 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:08.722 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.555144+0000", 2020-08-07T15:13:08.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4549999999999997e-06 2020-08-07T15:13:08.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.742 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.742 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:08.742 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.555148+0000", 2020-08-07T15:13:08.742 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4390000000000006e-06 2020-08-07T15:13:08.743 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.743 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:08.743 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.555158+0000", 2020-08-07T15:13:08.744 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4770000000000002e-06 2020-08-07T15:13:08.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.745 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:08.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.555163+0000", 2020-08-07T15:13:08.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001420985 2020-08-07T15:13:08.746 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.746 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:08.747 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.556584+0000", 2020-08-07T15:13:08.759 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6114999999999997e-05 2020-08-07T15:13:08.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.760 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:08.767 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.556680+0000", 2020-08-07T15:13:08.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000162585 2020-08-07T15:13:08.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.768 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.768 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:13:08.768 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.556843+0000", 2020-08-07T15:13:08.769 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034076919999999999 2020-08-07T15:13:08.769 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.769 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.770 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:08.771 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.560251+0000", 2020-08-07T15:13:08.771 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7300000000000002e-07 2020-08-07T15:13:08.771 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.772 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.772 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:08.780 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.560251+0000", 2020-08-07T15:13:08.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3405000000000003e-05 2020-08-07T15:13:08.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.781 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:08.781 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.560284+0000", 2020-08-07T15:13:08.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003132644 2020-08-07T15:13:08.782 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.782 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.782 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:08.782 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.563417+0000", 2020-08-07T15:13:08.783 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0083166549999999992 2020-08-07T15:13:08.783 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:08.783 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:08.784 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:08.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.784 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5481.0:83778 6.2 e272/230 6:42de6f18:::benchmark_data_smithi074_63843_object10472:head v 272'5602, mlcod=272'5599)", 2020-08-07T15:13:08.785 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:08.694556+0000", 2020-08-07T15:13:08.790 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0069078170000000001, 2020-08-07T15:13:08.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0070643040000000004, 2020-08-07T15:13:08.791 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:08.791 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:08.791 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:08.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:08.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.694556+0000", 2020-08-07T15:13:08.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:08.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.792 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.810 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:08.811 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.694556+0000", 2020-08-07T15:13:08.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4234999999999997e-05 2020-08-07T15:13:08.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:08.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.694620+0000", 2020-08-07T15:13:08.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0199999999999997e-06 2020-08-07T15:13:08.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.813 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:08.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.694627+0000", 2020-08-07T15:13:08.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5000999999999996e-05 2020-08-07T15:13:08.821 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.822 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:08.822 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.694692+0000", 2020-08-07T15:13:08.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6799999999999993e-06 2020-08-07T15:13:08.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.822 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:08.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.694700+0000", 2020-08-07T15:13:08.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7471e-05 2020-08-07T15:13:08.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:08.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.694748+0000", 2020-08-07T15:13:08.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4440999999999999e-05 2020-08-07T15:13:08.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:08.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.694772+0000", 2020-08-07T15:13:08.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00021584800000000001 2020-08-07T15:13:08.825 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:08.834 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:08.834 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:08.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.835 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5481.0:83779 6.2 e272/230 6:42de6f18:::benchmark_data_smithi074_63843_object10472:head v 272'5603, mlcod=272'5599)", 2020-08-07T15:13:08.835 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:08.695435+0000", 2020-08-07T15:13:08.835 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.006028622, 2020-08-07T15:13:08.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006246616, 2020-08-07T15:13:08.835 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:08.836 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:08.843 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:08.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.844 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:08.844 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.695435+0000", 2020-08-07T15:13:08.844 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:08.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.844 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:08.845 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.695435+0000", 2020-08-07T15:13:08.845 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0986999999999998e-05 2020-08-07T15:13:08.845 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:08.846 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.695486+0000", 2020-08-07T15:13:08.846 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7360000000000004e-06 2020-08-07T15:13:08.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.846 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.866 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:08.866 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.695493+0000", 2020-08-07T15:13:08.866 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3109000000000003e-05 2020-08-07T15:13:08.867 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.867 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.867 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:08.867 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.695546+0000", 2020-08-07T15:13:08.867 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8980000000000001e-06 2020-08-07T15:13:08.868 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.868 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.869 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:08.869 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.695552+0000", 2020-08-07T15:13:08.869 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.214e-05 2020-08-07T15:13:08.869 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.869 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.870 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:08.870 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.695594+0000", 2020-08-07T15:13:08.870 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2149000000000001e-05 2020-08-07T15:13:08.870 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.870 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.870 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:08.871 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.695616+0000", 2020-08-07T15:13:08.871 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018101900000000001 2020-08-07T15:13:08.871 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:08.871 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:08.871 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:08.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.872 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.872 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5481.0:83780 6.2 e272/230 6:42de6f18:::benchmark_data_smithi074_63843_object10472:head v 272'5604, mlcod=272'5599)", 2020-08-07T15:13:08.872 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:08.696112+0000", 2020-08-07T15:13:08.872 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0053524419999999998, 2020-08-07T15:13:08.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0056289729999999998, 2020-08-07T15:13:08.873 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:08.873 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:08.873 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:08.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.886 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:08.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.696112+0000", 2020-08-07T15:13:08.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:08.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.886 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:08.887 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.696112+0000", 2020-08-07T15:13:08.887 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1329000000000003e-05 2020-08-07T15:13:08.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:08.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.696163+0000", 2020-08-07T15:13:08.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8990000000000004e-06 2020-08-07T15:13:08.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.891 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:08.891 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.696170+0000", 2020-08-07T15:13:08.891 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9512000000000002e-05 2020-08-07T15:13:08.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.892 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.892 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:08.892 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.696229+0000", 2020-08-07T15:13:08.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1190000000000002e-06 2020-08-07T15:13:08.892 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.893 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:08.893 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.696235+0000", 2020-08-07T15:13:08.893 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1786999999999998e-05 2020-08-07T15:13:08.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.894 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.894 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:08.894 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.696277+0000", 2020-08-07T15:13:08.894 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5528999999999999e-05 2020-08-07T15:13:08.894 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.894 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.895 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:08.895 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.696303+0000", 2020-08-07T15:13:08.895 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000191175 2020-08-07T15:13:08.895 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:08.895 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:08.896 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:08.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.896 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5481.0:83781 6.2 e272/230 6:42de6f18:::benchmark_data_smithi074_63843_object10472:head v 272'5605, mlcod=272'5599)", 2020-08-07T15:13:08.896 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:08.696805+0000", 2020-08-07T15:13:08.897 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0046592559999999996, 2020-08-07T15:13:08.897 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0050064280000000003, 2020-08-07T15:13:08.897 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:08.897 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:08.897 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:08.898 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.898 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:08.899 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.696805+0000", 2020-08-07T15:13:08.899 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:08.899 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.900 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:08.900 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.696805+0000", 2020-08-07T15:13:08.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1521999999999999e-05 2020-08-07T15:13:08.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.901 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.901 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:08.901 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.696856+0000", 2020-08-07T15:13:08.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8000000000000001e-06 2020-08-07T15:13:08.901 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.902 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.902 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:08.902 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.696863+0000", 2020-08-07T15:13:08.902 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3643999999999997e-05 2020-08-07T15:13:08.902 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.903 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:08.903 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.696917+0000", 2020-08-07T15:13:08.903 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6370000000000004e-06 2020-08-07T15:13:08.903 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.904 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:08.904 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.696922+0000", 2020-08-07T15:13:08.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4456999999999998e-05 2020-08-07T15:13:08.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:08.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.696967+0000", 2020-08-07T15:13:08.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2357999999999998e-05 2020-08-07T15:13:08.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.906 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.906 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:08.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.696989+0000", 2020-08-07T15:13:08.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000184418 2020-08-07T15:13:08.907 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:08.907 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:08.907 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:08.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.908 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5481.0:83782 6.2 e272/230 6:42de6f18:::benchmark_data_smithi074_63843_object10472:head v 272'5606, mlcod=272'5599)", 2020-08-07T15:13:08.908 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:08.697502+0000", 2020-08-07T15:13:08.908 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0039623469999999997, 2020-08-07T15:13:08.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043739069999999998, 2020-08-07T15:13:08.909 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:08.909 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:08.909 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:08.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.910 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:08.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.697502+0000", 2020-08-07T15:13:08.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:08.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:08.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.697502+0000", 2020-08-07T15:13:08.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1047000000000001e-05 2020-08-07T15:13:08.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:08.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.697553+0000", 2020-08-07T15:13:08.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7920000000000004e-06 2020-08-07T15:13:08.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:08.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.697563+0000", 2020-08-07T15:13:08.914 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8260000000000001e-05 2020-08-07T15:13:08.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:08.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.697621+0000", 2020-08-07T15:13:08.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6570000000000002e-06 2020-08-07T15:13:08.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:08.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.697626+0000", 2020-08-07T15:13:08.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3798000000000003e-05 2020-08-07T15:13:08.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:08.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.697670+0000", 2020-08-07T15:13:08.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7637e-05 2020-08-07T15:13:08.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:08.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.697698+0000", 2020-08-07T15:13:08.918 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019619099999999999 2020-08-07T15:13:08.918 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:08.918 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:08.918 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:08.918 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.919 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5481.0:83783 6.2 e272/230 6:42de6f18:::benchmark_data_smithi074_63843_object10472:head v 272'5607, mlcod=272'5599)", 2020-08-07T15:13:08.919 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:08.698060+0000", 2020-08-07T15:13:08.919 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0034039719999999999, 2020-08-07T15:13:08.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038803309999999999, 2020-08-07T15:13:08.920 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:08.920 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:08.920 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:08.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.922 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:08.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.698060+0000", 2020-08-07T15:13:08.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:08.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:08.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.698060+0000", 2020-08-07T15:13:08.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7803000000000003e-05 2020-08-07T15:13:08.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:08.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.698108+0000", 2020-08-07T15:13:08.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2809999999999997e-06 2020-08-07T15:13:08.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.933 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.933 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:08.934 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.698114+0000", 2020-08-07T15:13:08.934 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9257e-05 2020-08-07T15:13:08.934 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.934 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.934 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:08.934 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.698163+0000", 2020-08-07T15:13:08.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2360000000000004e-06 2020-08-07T15:13:08.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:08.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.698169+0000", 2020-08-07T15:13:08.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6048999999999999e-05 2020-08-07T15:13:08.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:08.936 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.698215+0000", 2020-08-07T15:13:08.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4054999999999999e-05 2020-08-07T15:13:08.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:08.964 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.698239+0000", 2020-08-07T15:13:08.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017868100000000001 2020-08-07T15:13:08.965 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:08.965 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:08.965 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:08.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.966 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5481.0:83784 6.2 e272/230 6:42de6f18:::benchmark_data_smithi074_63843_object10472:head v 272'5608, mlcod=272'5599)", 2020-08-07T15:13:08.967 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:08.698867+0000", 2020-08-07T15:13:08.967 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.002596615, 2020-08-07T15:13:08.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0031431219999999999, 2020-08-07T15:13:08.968 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:08.968 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:08.968 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:08.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:08.969 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:08.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.698867+0000", 2020-08-07T15:13:08.969 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:08.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:08.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.001 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:09.002 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.698867+0000", 2020-08-07T15:13:09.002 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7138000000000002e-05 2020-08-07T15:13:09.002 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.002 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.002 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:09.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.698915+0000", 2020-08-07T15:13:09.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.63e-06 2020-08-07T15:13:09.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:09.004 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.698921+0000", 2020-08-07T15:13:09.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2777e-05 2020-08-07T15:13:09.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:09.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.698974+0000", 2020-08-07T15:13:09.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.614e-06 2020-08-07T15:13:09.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:09.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.698980+0000", 2020-08-07T15:13:09.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2697999999999997e-05 2020-08-07T15:13:09.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.699022+0000", 2020-08-07T15:13:09.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9907999999999999e-05 2020-08-07T15:13:09.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:09.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:08.699052+0000", 2020-08-07T15:13:09.008 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018476500000000001 2020-08-07T15:13:09.009 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:09.009 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:09.009 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:09.013 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:09.013 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:13:09.013 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 8 2020-08-07T15:13:09.013 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:09.015 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:13:09.017 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:09.013+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:09.123 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:09.114+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:09.220 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:09.215+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:09.229 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:09.230 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:13:09.230 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:13:09.230 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:13:09.231 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:09.239 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:13:09.272 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 57 16 84691 84675 11.6044 12.6562 0.00684115 0.0107667 2020-08-07T15:13:09.323 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:09.321+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:09.435 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:09.432+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:09.476 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:09.477 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:13:09.477 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:13:09.477 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:13:09.478 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.478 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:30968 4.4 e98/81 ondisk, result = 0)", 2020-08-07T15:13:09.478 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:13:09.486 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 263.73930892200002, 2020-08-07T15:13:09.486 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96211418999999998, 2020-08-07T15:13:09.486 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:09.486 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:09.486 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:09.487 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.487 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:09.487 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:13:09.487 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:09.487 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.494 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:09.495 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:13:09.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2387e-05 2020-08-07T15:13:09.496 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.496 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.496 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:09.496 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729821+0000", 2020-08-07T15:13:09.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9909999999999999e-06 2020-08-07T15:13:09.497 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.499 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.499 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:09.501 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729827+0000", 2020-08-07T15:13:09.501 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96185524600000005 2020-08-07T15:13:09.501 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.502 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.502 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:09.502 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691682+0000", 2020-08-07T15:13:09.502 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8760000000000005e-06 2020-08-07T15:13:09.503 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.503 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.503 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:09.503 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691690+0000", 2020-08-07T15:13:09.503 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9682000000000001e-05 2020-08-07T15:13:09.504 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.504 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.504 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.504 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691759+0000", 2020-08-07T15:13:09.504 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5381000000000001e-05 2020-08-07T15:13:09.505 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.505 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.505 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:09.505 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691775+0000", 2020-08-07T15:13:09.505 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000127627 2020-08-07T15:13:09.506 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.506 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.506 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:09.506 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691902+0000", 2020-08-07T15:13:09.507 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96211418999999998 2020-08-07T15:13:09.507 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:09.507 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:09.507 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:09.507 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.508 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.508 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42472 4.4 4:2ff82af8:::benchmark_data_smithi074_44586_object5308:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:09.508 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:13:09.509 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 249.20184614799999, 2020-08-07T15:13:09.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89653816399999997, 2020-08-07T15:13:09.509 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:09.510 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:09.510 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:09.510 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:09.511 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:09.511 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42472 2020-08-07T15:13:09.511 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.512 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:09.512 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.512 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:09.512 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:13:09.513 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:09.513 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.514 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:09.514 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:13:09.514 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9591e-05 2020-08-07T15:13:09.515 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.515 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.515 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:09.516 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267301+0000", 2020-08-07T15:13:09.516 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7490000000000004e-06 2020-08-07T15:13:09.516 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.516 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.517 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:09.517 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267306+0000", 2020-08-07T15:13:09.517 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2850000000000001e-05 2020-08-07T15:13:09.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.520 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:09.520 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267319+0000", 2020-08-07T15:13:09.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9100000000000002e-06 2020-08-07T15:13:09.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.521 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.522 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:09.522 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267325+0000", 2020-08-07T15:13:09.523 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6584000000000001e-05 2020-08-07T15:13:09.523 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.523 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.524 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.524 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267362+0000", 2020-08-07T15:13:09.524 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9188000000000001e-05 2020-08-07T15:13:09.525 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.525 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.525 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:09.525 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267451+0000", 2020-08-07T15:13:09.526 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000182917 2020-08-07T15:13:09.526 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.526 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.527 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:09.527 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267634+0000", 2020-08-07T15:13:09.527 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002484708 2020-08-07T15:13:09.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.529 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.529 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:09.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270119+0000", 2020-08-07T15:13:09.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0299999999999997e-07 2020-08-07T15:13:09.530 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.530 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.530 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:09.530 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270119+0000", 2020-08-07T15:13:09.531 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8251999999999996e-05 2020-08-07T15:13:09.531 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.531 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.531 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:09.532 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270188+0000", 2020-08-07T15:13:09.550 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24443688 2020-08-07T15:13:09.550 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.551 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.551 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:09.551 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.514625+0000", 2020-08-07T15:13:09.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.64911260599999998 2020-08-07T15:13:09.552 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.552 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:09.553 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163737+0000", 2020-08-07T15:13:09.553 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2100000000000001e-06 2020-08-07T15:13:09.554 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.554 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.554 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:09.555 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163738+0000", 2020-08-07T15:13:09.555 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0816000000000003e-05 2020-08-07T15:13:09.555 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:09.556 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163789+0000", 2020-08-07T15:13:09.557 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89653816399999997 2020-08-07T15:13:09.557 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:09.557 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:09.558 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:09.558 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.558 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.559 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72688 4.4 4:295d3f6e:::benchmark_data_smithi074_44586_object9085:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e129)", 2020-08-07T15:13:09.559 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:13:09.559 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 224.072263961, 2020-08-07T15:13:09.560 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7869782759999999, 2020-08-07T15:13:09.560 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:09.561 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:13:09.561 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:09.561 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:09.563 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:09.563 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72688 2020-08-07T15:13:09.564 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.564 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:09.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.564 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:09.564 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:13:09.565 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:09.565 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.565 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.565 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:09.565 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:13:09.566 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8721999999999999e-05 2020-08-07T15:13:09.566 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.566 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.566 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:09.566 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396882+0000", 2020-08-07T15:13:09.581 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2639999999999999e-06 2020-08-07T15:13:09.581 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.582 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.582 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:09.583 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396887+0000", 2020-08-07T15:13:09.583 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0767999999999999e-05 2020-08-07T15:13:09.583 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.584 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.584 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:09.585 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396898+0000", 2020-08-07T15:13:09.585 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1860000000000002e-06 2020-08-07T15:13:09.585 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.586 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.586 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:09.586 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396902+0000", 2020-08-07T15:13:09.587 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0037638649999999999 2020-08-07T15:13:09.587 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.587 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.588 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.588 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400666+0000", 2020-08-07T15:13:09.597 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000118077 2020-08-07T15:13:09.598 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.607 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.608 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:09.608 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400784+0000", 2020-08-07T15:13:09.608 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000136728 2020-08-07T15:13:09.610 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.610 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.610 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:13:09.610 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400921+0000", 2020-08-07T15:13:09.611 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001595443 2020-08-07T15:13:09.611 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.611 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.611 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:09.611 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.402516+0000", 2020-08-07T15:13:09.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.014817907 2020-08-07T15:13:09.612 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.612 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.612 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:09.612 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417334+0000", 2020-08-07T15:13:09.613 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.75e-06 2020-08-07T15:13:09.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.613 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:09.613 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417336+0000", 2020-08-07T15:13:09.616 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8066999999999996e-05 2020-08-07T15:13:09.627 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.627 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:09.628 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417424+0000", 2020-08-07T15:13:09.628 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002210573 2020-08-07T15:13:09.628 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.628 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.628 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.629 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.419635+0000", 2020-08-07T15:13:09.629 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1379000000000001e-05 2020-08-07T15:13:09.629 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.629 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.629 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:09.629 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.419646+0000", 2020-08-07T15:13:09.630 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.004643409 2020-08-07T15:13:09.630 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.630 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.630 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.630 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424289+0000", 2020-08-07T15:13:09.631 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7563000000000001e-05 2020-08-07T15:13:09.631 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.631 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.631 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:09.631 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424307+0000", 2020-08-07T15:13:09.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75943629400000001 2020-08-07T15:13:09.632 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.632 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.632 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.632 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183743+0000", 2020-08-07T15:13:09.633 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8281000000000006e-05 2020-08-07T15:13:09.633 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.633 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.633 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:09.633 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183812+0000", 2020-08-07T15:13:09.653 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7869782759999999 2020-08-07T15:13:09.654 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:09.654 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:09.655 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:09.655 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.655 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.656 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72985 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:09.656 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:13:09.656 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 223.85627814200001, 2020-08-07T15:13:09.657 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869466839999999, 2020-08-07T15:13:09.657 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:09.657 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:09.659 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:09.659 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:09.659 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:09.660 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72985 2020-08-07T15:13:09.660 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.660 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:09.660 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.660 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:09.661 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:13:09.661 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:09.661 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.662 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.662 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:09.662 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:13:09.663 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8003999999999997e-05 2020-08-07T15:13:09.663 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.663 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.663 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:09.663 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612857+0000", 2020-08-07T15:13:09.664 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.246e-06 2020-08-07T15:13:09.664 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.664 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:09.664 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612863+0000", 2020-08-07T15:13:09.664 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5146e-05 2020-08-07T15:13:09.665 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.665 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.665 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:09.665 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612878+0000", 2020-08-07T15:13:09.666 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1940000000000003e-06 2020-08-07T15:13:09.666 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.666 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:09.668 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612885+0000", 2020-08-07T15:13:09.668 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0349999999999997e-05 2020-08-07T15:13:09.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612935+0000", 2020-08-07T15:13:09.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1029999999999996e-06 2020-08-07T15:13:09.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.670 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:09.670 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612942+0000", 2020-08-07T15:13:09.670 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81140097700000002 2020-08-07T15:13:09.670 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.670 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.671 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.671 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424343+0000", 2020-08-07T15:13:09.671 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1799999999999996e-06 2020-08-07T15:13:09.671 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.671 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.671 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:09.674 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424351+0000", 2020-08-07T15:13:09.674 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75948665299999996 2020-08-07T15:13:09.674 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.674 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.674 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.675 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183838+0000", 2020-08-07T15:13:09.675 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000179187 2020-08-07T15:13:09.675 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.675 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.675 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:09.676 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.184017+0000", 2020-08-07T15:13:09.676 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024691600000000002 2020-08-07T15:13:09.676 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.676 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.676 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:09.677 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.184264+0000", 2020-08-07T15:13:09.677 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006362721 2020-08-07T15:13:09.677 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:09.678 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190627+0000", 2020-08-07T15:13:09.678 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7400000000000001e-06 2020-08-07T15:13:09.678 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.678 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.678 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:09.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190629+0000", 2020-08-07T15:13:09.679 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1305999999999999e-05 2020-08-07T15:13:09.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.679 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:09.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190690+0000", 2020-08-07T15:13:09.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0088507159999999998 2020-08-07T15:13:09.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.680 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:09.680 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199541+0000", 2020-08-07T15:13:09.681 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015864699999999999 2020-08-07T15:13:09.681 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.681 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:09.682 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199699+0000", 2020-08-07T15:13:09.682 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.23e-07 2020-08-07T15:13:09.682 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.682 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.682 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:09.683 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199700+0000", 2020-08-07T15:13:09.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5775e-05 2020-08-07T15:13:09.683 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.683 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.685 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:09.685 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199766+0000", 2020-08-07T15:13:09.686 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869466839999999 2020-08-07T15:13:09.686 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:09.686 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:09.686 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:09.686 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.687 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.687 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72986 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:09.687 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:13:09.687 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 223.85616755999999, 2020-08-07T15:13:09.687 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.586946441, 2020-08-07T15:13:09.688 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:09.688 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:09.688 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:09.688 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:09.688 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:09.689 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72986 2020-08-07T15:13:09.689 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.689 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:09.689 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.689 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:09.690 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:13:09.690 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:09.690 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.690 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.690 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:09.691 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:13:09.691 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9861999999999997e-05 2020-08-07T15:13:09.691 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.691 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.691 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:09.692 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612990+0000", 2020-08-07T15:13:09.692 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4350000000000001e-06 2020-08-07T15:13:09.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.692 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:09.693 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612993+0000", 2020-08-07T15:13:09.693 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9529999999999999e-06 2020-08-07T15:13:09.693 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.693 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.693 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:09.694 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613001+0000", 2020-08-07T15:13:09.694 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4309999999999998e-06 2020-08-07T15:13:09.694 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.694 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.695 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:09.695 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613004+0000", 2020-08-07T15:13:09.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5284000000000001e-05 2020-08-07T15:13:09.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.696 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.696 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613030+0000", 2020-08-07T15:13:09.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2400000000000001e-06 2020-08-07T15:13:09.697 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:09.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613034+0000", 2020-08-07T15:13:09.697 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81134136199999995 2020-08-07T15:13:09.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.698 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.698 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.698 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424375+0000", 2020-08-07T15:13:09.698 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3520000000000001e-06 2020-08-07T15:13:09.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.699 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.699 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:09.699 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424383+0000", 2020-08-07T15:13:09.700 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76064437799999995 2020-08-07T15:13:09.700 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.700 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.701 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185027+0000", 2020-08-07T15:13:09.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000119719 2020-08-07T15:13:09.701 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.701 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:09.702 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185147+0000", 2020-08-07T15:13:09.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016829400000000001 2020-08-07T15:13:09.702 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.702 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.702 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:09.703 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185315+0000", 2020-08-07T15:13:09.703 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0053821199999999998 2020-08-07T15:13:09.703 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.703 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:09.703 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190697+0000", 2020-08-07T15:13:09.704 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6899999999999996e-07 2020-08-07T15:13:09.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.705 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:09.705 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190698+0000", 2020-08-07T15:13:09.705 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2072000000000001e-05 2020-08-07T15:13:09.706 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.706 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.706 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:09.706 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190730+0000", 2020-08-07T15:13:09.707 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0088501440000000008 2020-08-07T15:13:09.707 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.707 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:09.707 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199580+0000", 2020-08-07T15:13:09.707 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00023199900000000001 2020-08-07T15:13:09.708 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.708 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:09.708 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199812+0000", 2020-08-07T15:13:09.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6300000000000005e-07 2020-08-07T15:13:09.709 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.709 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.709 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:09.709 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199813+0000", 2020-08-07T15:13:09.709 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3363999999999998e-05 2020-08-07T15:13:09.710 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.710 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.710 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:09.710 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199876+0000", 2020-08-07T15:13:09.710 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.586946441 2020-08-07T15:13:09.711 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:09.711 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:09.711 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:09.711 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.712 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.712 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72987 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:09.712 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:13:09.712 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 223.856071805, 2020-08-07T15:13:09.712 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869598709999999, 2020-08-07T15:13:09.713 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:09.713 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:09.713 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:09.713 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:09.713 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:09.714 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72987 2020-08-07T15:13:09.714 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.714 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:09.714 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.714 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:09.715 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:13:09.715 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:09.715 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.715 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.715 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:09.716 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:13:09.716 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1851999999999999e-05 2020-08-07T15:13:09.717 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.717 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.717 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:09.717 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613057+0000", 2020-08-07T15:13:09.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0429999999999999e-06 2020-08-07T15:13:09.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.718 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.718 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:09.718 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613060+0000", 2020-08-07T15:13:09.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1060000000000002e-06 2020-08-07T15:13:09.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:09.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613067+0000", 2020-08-07T15:13:09.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8040000000000002e-06 2020-08-07T15:13:09.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:09.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613070+0000", 2020-08-07T15:13:09.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6857000000000001e-05 2020-08-07T15:13:09.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.722 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.722 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613097+0000", 2020-08-07T15:13:09.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8770000000000001e-06 2020-08-07T15:13:09.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.723 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:09.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613102+0000", 2020-08-07T15:13:09.724 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81131066399999996 2020-08-07T15:13:09.724 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.724 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.724 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.725 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424413+0000", 2020-08-07T15:13:09.725 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4599999999999997e-06 2020-08-07T15:13:09.726 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.726 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.727 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:09.727 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424420+0000", 2020-08-07T15:13:09.727 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76132809700000004 2020-08-07T15:13:09.727 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.727 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.728 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185748+0000", 2020-08-07T15:13:09.728 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.4264000000000005e-05 2020-08-07T15:13:09.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.729 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:09.729 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185832+0000", 2020-08-07T15:13:09.729 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112803 2020-08-07T15:13:09.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.737 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.737 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:09.737 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185945+0000", 2020-08-07T15:13:09.737 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0089954130000000007 2020-08-07T15:13:09.737 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.738 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.738 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:09.738 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194941+0000", 2020-08-07T15:13:09.738 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.091e-06 2020-08-07T15:13:09.739 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.739 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.739 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:09.739 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194943+0000", 2020-08-07T15:13:09.739 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7423999999999996e-05 2020-08-07T15:13:09.740 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.740 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.740 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:09.741 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195020+0000", 2020-08-07T15:13:09.741 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045973300000000002 2020-08-07T15:13:09.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.741 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.742 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:09.742 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199618+0000", 2020-08-07T15:13:09.742 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000302009 2020-08-07T15:13:09.743 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.743 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:09.744 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199920+0000", 2020-08-07T15:13:09.744 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7999999999999996e-07 2020-08-07T15:13:09.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.745 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:09.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199920+0000", 2020-08-07T15:13:09.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5296999999999995e-05 2020-08-07T15:13:09.746 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.746 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:09.747 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199985+0000", 2020-08-07T15:13:09.747 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869598709999999 2020-08-07T15:13:09.747 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:09.748 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:09.748 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:09.748 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.754 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.755 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72988 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:09.755 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:13:09.756 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 223.85600722500001, 2020-08-07T15:13:09.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587111143, 2020-08-07T15:13:09.756 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:09.756 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:09.757 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:09.757 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:09.757 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:09.758 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72988 2020-08-07T15:13:09.758 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.759 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:09.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.759 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:09.760 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:13:09.760 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:09.760 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.761 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:09.761 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:13:09.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6563e-05 2020-08-07T15:13:09.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.762 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:09.763 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613117+0000", 2020-08-07T15:13:09.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7180000000000001e-06 2020-08-07T15:13:09.764 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.765 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:09.765 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613119+0000", 2020-08-07T15:13:09.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1789999999999999e-06 2020-08-07T15:13:09.766 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:09.767 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613124+0000", 2020-08-07T15:13:09.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3539999999999998e-06 2020-08-07T15:13:09.768 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.768 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.768 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:09.769 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613127+0000", 2020-08-07T15:13:09.769 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3697e-05 2020-08-07T15:13:09.769 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.769 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.789 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.789 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613151+0000", 2020-08-07T15:13:09.789 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7559999999999999e-06 2020-08-07T15:13:09.789 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.790 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:09.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613155+0000", 2020-08-07T15:13:09.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81128859600000003 2020-08-07T15:13:09.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424444+0000", 2020-08-07T15:13:09.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9619999999999996e-06 2020-08-07T15:13:09.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.813 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:09.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424451+0000", 2020-08-07T15:13:09.814 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76286712899999998 2020-08-07T15:13:09.814 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.814 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.814 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.814 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187318+0000", 2020-08-07T15:13:09.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011616300000000001 2020-08-07T15:13:09.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.815 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:09.815 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187434+0000", 2020-08-07T15:13:09.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019347400000000001 2020-08-07T15:13:09.816 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:09.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187628+0000", 2020-08-07T15:13:09.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074049090000000003 2020-08-07T15:13:09.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.817 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:09.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195033+0000", 2020-08-07T15:13:09.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1090000000000001e-06 2020-08-07T15:13:09.830 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.830 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.830 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:09.830 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195034+0000", 2020-08-07T15:13:09.831 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8566999999999997e-05 2020-08-07T15:13:09.831 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.831 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.831 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:09.831 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195092+0000", 2020-08-07T15:13:09.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004570457 2020-08-07T15:13:09.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:09.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199663+0000", 2020-08-07T15:13:09.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00046757899999999998 2020-08-07T15:13:09.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:09.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200130+0000", 2020-08-07T15:13:09.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6700000000000002e-07 2020-08-07T15:13:09.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:09.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200131+0000", 2020-08-07T15:13:09.850 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9963999999999996e-05 2020-08-07T15:13:09.850 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.851 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:09.851 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200201+0000", 2020-08-07T15:13:09.851 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587111143 2020-08-07T15:13:09.851 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:09.851 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:09.852 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:09.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.852 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72989 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:09.853 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:13:09.853 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 223.85595083699999, 2020-08-07T15:13:09.853 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5872506399999999, 2020-08-07T15:13:09.853 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:09.854 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:09.854 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:09.855 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:09.855 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:09.855 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72989 2020-08-07T15:13:09.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.856 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:09.856 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.856 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:09.856 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:13:09.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:09.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:09.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:13:09.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6081999999999999e-05 2020-08-07T15:13:09.858 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.858 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:09.858 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613172+0000", 2020-08-07T15:13:09.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6570000000000001e-06 2020-08-07T15:13:09.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.859 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.859 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:09.859 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613175+0000", 2020-08-07T15:13:09.859 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7740000000000001e-06 2020-08-07T15:13:09.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.860 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:09.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613181+0000", 2020-08-07T15:13:09.860 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2759999999999999e-06 2020-08-07T15:13:09.860 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.861 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:09.861 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613183+0000", 2020-08-07T15:13:09.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3892000000000001e-05 2020-08-07T15:13:09.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.862 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.862 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.862 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613207+0000", 2020-08-07T15:13:09.862 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9870000000000001e-06 2020-08-07T15:13:09.862 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:09.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613211+0000", 2020-08-07T15:13:09.863 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81126230700000002 2020-08-07T15:13:09.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424473+0000", 2020-08-07T15:13:09.871 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1460000000000001e-06 2020-08-07T15:13:09.871 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.871 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:09.872 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424480+0000", 2020-08-07T15:13:09.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76413073099999995 2020-08-07T15:13:09.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.872 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.872 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.873 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188610+0000", 2020-08-07T15:13:09.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011278899999999999 2020-08-07T15:13:09.873 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.873 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:09.874 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188723+0000", 2020-08-07T15:13:09.874 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000213422 2020-08-07T15:13:09.874 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.874 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.874 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:09.875 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188936+0000", 2020-08-07T15:13:09.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0061672929999999999 2020-08-07T15:13:09.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:09.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195104+0000", 2020-08-07T15:13:09.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8199999999999999e-07 2020-08-07T15:13:09.876 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.876 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:09.877 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195105+0000", 2020-08-07T15:13:09.877 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2056000000000004e-05 2020-08-07T15:13:09.877 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.877 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.877 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:09.878 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195167+0000", 2020-08-07T15:13:09.878 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0051009769999999996 2020-08-07T15:13:09.878 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.878 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.879 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:09.879 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200268+0000", 2020-08-07T15:13:09.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9428000000000001e-05 2020-08-07T15:13:09.880 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:09.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200327+0000", 2020-08-07T15:13:09.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9100000000000003e-07 2020-08-07T15:13:09.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:09.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200328+0000", 2020-08-07T15:13:09.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9250000000000003e-05 2020-08-07T15:13:09.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.890 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:09.891 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200397+0000", 2020-08-07T15:13:09.891 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5872506399999999 2020-08-07T15:13:09.891 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:09.891 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:09.891 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:09.892 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.892 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.892 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72990 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:09.893 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:13:09.893 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 223.85589354499999, 2020-08-07T15:13:09.894 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587335105, 2020-08-07T15:13:09.894 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:09.894 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:09.895 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:09.895 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:09.895 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:09.896 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72990 2020-08-07T15:13:09.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.896 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:09.897 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.897 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:09.897 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:13:09.898 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:09.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:09.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:13:09.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5276999999999999e-05 2020-08-07T15:13:09.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:09.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613229+0000", 2020-08-07T15:13:09.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7219999999999999e-06 2020-08-07T15:13:09.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.933 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.933 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:09.934 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613232+0000", 2020-08-07T15:13:09.934 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1170000000000003e-06 2020-08-07T15:13:09.934 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.934 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:09.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613237+0000", 2020-08-07T15:13:09.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2309999999999999e-06 2020-08-07T15:13:09.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:09.936 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613239+0000", 2020-08-07T15:13:09.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2469000000000001e-05 2020-08-07T15:13:09.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613261+0000", 2020-08-07T15:13:09.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9280000000000001e-06 2020-08-07T15:13:09.937 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.938 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:09.938 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613266+0000", 2020-08-07T15:13:09.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81123414599999999 2020-08-07T15:13:09.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424501+0000", 2020-08-07T15:13:09.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5300000000000002e-06 2020-08-07T15:13:09.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.940 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.940 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:09.940 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424507+0000", 2020-08-07T15:13:09.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76488660200000003 2020-08-07T15:13:09.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189394+0000", 2020-08-07T15:13:09.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6878000000000003e-05 2020-08-07T15:13:09.942 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.942 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:09.942 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189491+0000", 2020-08-07T15:13:09.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000204477 2020-08-07T15:13:09.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:09.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189695+0000", 2020-08-07T15:13:09.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0065180940000000003 2020-08-07T15:13:09.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.944 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:09.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196213+0000", 2020-08-07T15:13:09.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4920000000000001e-06 2020-08-07T15:13:09.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:09.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196215+0000", 2020-08-07T15:13:09.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9950000000000006e-05 2020-08-07T15:13:09.946 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.946 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.946 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:09.946 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196285+0000", 2020-08-07T15:13:09.946 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004163674 2020-08-07T15:13:09.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.947 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.947 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:09.947 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200448+0000", 2020-08-07T15:13:09.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6919999999999999e-05 2020-08-07T15:13:09.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.948 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:09.948 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200485+0000", 2020-08-07T15:13:09.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1699999999999998e-07 2020-08-07T15:13:09.949 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.949 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:09.949 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200486+0000", 2020-08-07T15:13:09.950 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3081000000000002e-05 2020-08-07T15:13:09.950 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.950 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:09.951 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200539+0000", 2020-08-07T15:13:09.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587335105 2020-08-07T15:13:09.951 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:09.951 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:09.952 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:09.952 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.952 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.952 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72991 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:09.952 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:13:09.953 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 223.85583801999999, 2020-08-07T15:13:09.953 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5876210340000001, 2020-08-07T15:13:09.953 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:09.953 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:09.953 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:09.954 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:09.954 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:09.954 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72991 2020-08-07T15:13:09.954 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.955 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:09.955 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.955 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:09.955 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:13:09.955 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:09.956 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.956 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.956 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:09.956 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:13:09.956 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5763999999999999e-05 2020-08-07T15:13:09.957 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.957 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:09.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613285+0000", 2020-08-07T15:13:09.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6479999999999999e-06 2020-08-07T15:13:09.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:09.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613288+0000", 2020-08-07T15:13:09.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1909999999999999e-06 2020-08-07T15:13:09.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:09.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613293+0000", 2020-08-07T15:13:09.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2560000000000001e-06 2020-08-07T15:13:09.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:09.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613295+0000", 2020-08-07T15:13:09.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2561e-05 2020-08-07T15:13:09.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.962 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613318+0000", 2020-08-07T15:13:09.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6349999999999999e-06 2020-08-07T15:13:09.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:09.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613321+0000", 2020-08-07T15:13:09.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81120592400000002 2020-08-07T15:13:09.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424527+0000", 2020-08-07T15:13:09.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8540000000000002e-06 2020-08-07T15:13:09.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:09.966 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424533+0000", 2020-08-07T15:13:09.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76623392800000001 2020-08-07T15:13:09.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:09.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190767+0000", 2020-08-07T15:13:09.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3990000000000001e-05 2020-08-07T15:13:09.982 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.983 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.983 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:09.983 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190841+0000", 2020-08-07T15:13:09.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011444599999999999 2020-08-07T15:13:09.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.985 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:09.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190955+0000", 2020-08-07T15:13:09.985 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0053397499999999999 2020-08-07T15:13:09.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.986 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:09.986 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196295+0000", 2020-08-07T15:13:09.986 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006e-06 2020-08-07T15:13:09.987 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.987 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:09.987 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196296+0000", 2020-08-07T15:13:09.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8940999999999997e-05 2020-08-07T15:13:09.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:09.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196355+0000", 2020-08-07T15:13:09.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042276700000000002 2020-08-07T15:13:09.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:09.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200583+0000", 2020-08-07T15:13:09.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00026035700000000001 2020-08-07T15:13:09.994 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.995 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.995 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:09.996 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200843+0000", 2020-08-07T15:13:09.996 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6700000000000003e-07 2020-08-07T15:13:09.997 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:09.997 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:09.998 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:09.998 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200844+0000", 2020-08-07T15:13:09.998 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6545999999999999e-05 2020-08-07T15:13:09.999 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.000 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.000 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:10.001 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200880+0000", 2020-08-07T15:13:10.001 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5876210340000001 2020-08-07T15:13:10.002 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.002 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:10.003 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.003 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72992 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:10.003 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:13:10.004 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 223.85475054099999, 2020-08-07T15:13:10.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5866241210000001, 2020-08-07T15:13:10.004 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:10.004 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:10.004 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:10.005 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:10.005 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:10.005 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72992 2020-08-07T15:13:10.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.006 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:10.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:10.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:13:10.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:10.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:10.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:13:10.008 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7543e-05 2020-08-07T15:13:10.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:10.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614374+0000", 2020-08-07T15:13:10.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9740000000000004e-06 2020-08-07T15:13:10.009 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.009 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.009 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:10.010 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614378+0000", 2020-08-07T15:13:10.010 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2660000000000007e-06 2020-08-07T15:13:10.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.023 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:10.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614387+0000", 2020-08-07T15:13:10.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2950000000000001e-06 2020-08-07T15:13:10.023 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:10.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614393+0000", 2020-08-07T15:13:10.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7279000000000001e-05 2020-08-07T15:13:10.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614420+0000", 2020-08-07T15:13:10.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9040000000000002e-06 2020-08-07T15:13:10.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:10.026 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614424+0000", 2020-08-07T15:13:10.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81012815699999996 2020-08-07T15:13:10.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424552+0000", 2020-08-07T15:13:10.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2879999999999998e-06 2020-08-07T15:13:10.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.030 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.030 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:10.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424558+0000", 2020-08-07T15:13:10.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76678665400000001 2020-08-07T15:13:10.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191345+0000", 2020-08-07T15:13:10.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010819600000000001 2020-08-07T15:13:10.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:10.034 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191453+0000", 2020-08-07T15:13:10.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000170829 2020-08-07T15:13:10.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:10.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191624+0000", 2020-08-07T15:13:10.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0047386679999999997 2020-08-07T15:13:10.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.038 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:10.038 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196363+0000", 2020-08-07T15:13:10.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6000000000000003e-07 2020-08-07T15:13:10.039 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.039 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.039 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:10.040 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196363+0000", 2020-08-07T15:13:10.040 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3421000000000002e-05 2020-08-07T15:13:10.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:10.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196417+0000", 2020-08-07T15:13:10.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042225539999999999 2020-08-07T15:13:10.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.044 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:10.044 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200639+0000", 2020-08-07T15:13:10.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00028031399999999999 2020-08-07T15:13:10.045 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.045 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:10.045 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200920+0000", 2020-08-07T15:13:10.046 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0299999999999999e-07 2020-08-07T15:13:10.046 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.046 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.046 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:10.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200920+0000", 2020-08-07T15:13:10.047 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0516000000000002e-05 2020-08-07T15:13:10.047 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.048 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:10.048 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200971+0000", 2020-08-07T15:13:10.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5866241210000001 2020-08-07T15:13:10.049 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.049 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:10.049 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.050 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.050 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.050 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73017 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:10.050 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:13:10.051 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 223.79924228799999, 2020-08-07T15:13:10.051 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.531181608, 2020-08-07T15:13:10.051 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:10.051 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:10.052 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:10.052 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:10.052 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:10.053 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73017 2020-08-07T15:13:10.053 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.053 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:10.054 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:10.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:13:10.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:10.055 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:10.065 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:13:10.066 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3018e-05 2020-08-07T15:13:10.066 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.066 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.066 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:10.067 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669898+0000", 2020-08-07T15:13:10.067 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5340000000000002e-06 2020-08-07T15:13:10.067 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.067 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.067 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:10.068 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669905+0000", 2020-08-07T15:13:10.068 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3307e-05 2020-08-07T15:13:10.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.068 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.068 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:10.069 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669919+0000", 2020-08-07T15:13:10.069 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7729999999999999e-06 2020-08-07T15:13:10.074 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.074 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.075 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:10.075 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669927+0000", 2020-08-07T15:13:10.075 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0361000000000001e-05 2020-08-07T15:13:10.075 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.075 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.076 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.076 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669977+0000", 2020-08-07T15:13:10.076 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0049999999999993e-06 2020-08-07T15:13:10.076 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.076 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.077 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:10.077 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669986+0000", 2020-08-07T15:13:10.077 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75459235199999997 2020-08-07T15:13:10.077 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.077 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.077 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.078 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424578+0000", 2020-08-07T15:13:10.078 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.925e-06 2020-08-07T15:13:10.078 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.078 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.078 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:10.079 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424584+0000", 2020-08-07T15:13:10.079 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76741703900000002 2020-08-07T15:13:10.079 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.079 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.079 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.080 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192001+0000", 2020-08-07T15:13:10.080 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000181196 2020-08-07T15:13:10.080 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.080 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.080 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:10.081 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192182+0000", 2020-08-07T15:13:10.081 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00021149299999999999 2020-08-07T15:13:10.081 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.081 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.081 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:10.082 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192394+0000", 2020-08-07T15:13:10.082 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004031242 2020-08-07T15:13:10.082 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.082 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:10.083 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196425+0000", 2020-08-07T15:13:10.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7800000000000001e-07 2020-08-07T15:13:10.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.084 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:10.085 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196426+0000", 2020-08-07T15:13:10.085 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5244999999999997e-05 2020-08-07T15:13:10.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.085 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:10.086 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196481+0000", 2020-08-07T15:13:10.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041989360000000003 2020-08-07T15:13:10.086 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.086 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.086 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:10.087 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200680+0000", 2020-08-07T15:13:10.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00032415299999999998 2020-08-07T15:13:10.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:10.088 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201004+0000", 2020-08-07T15:13:10.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9299999999999999e-07 2020-08-07T15:13:10.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:10.088 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201005+0000", 2020-08-07T15:13:10.089 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1857999999999999e-05 2020-08-07T15:13:10.089 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.089 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.089 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:10.090 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201037+0000", 2020-08-07T15:13:10.090 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.531181608 2020-08-07T15:13:10.091 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.091 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:10.091 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.092 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.092 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.092 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73018 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:10.092 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:13:10.092 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 223.798274483, 2020-08-07T15:13:10.093 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.530301616, 2020-08-07T15:13:10.093 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:10.093 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:10.093 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:10.093 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:10.094 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:10.095 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73018 2020-08-07T15:13:10.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.095 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:10.095 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.095 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:10.096 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:13:10.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:10.096 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.097 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.097 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:10.097 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:13:10.097 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9747000000000002e-05 2020-08-07T15:13:10.097 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.098 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.098 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:10.098 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670852+0000", 2020-08-07T15:13:10.098 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9430000000000002e-06 2020-08-07T15:13:10.098 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.099 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.099 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:10.099 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670856+0000", 2020-08-07T15:13:10.099 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0649999999999999e-06 2020-08-07T15:13:10.099 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.099 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.100 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:10.100 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670864+0000", 2020-08-07T15:13:10.100 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.924e-06 2020-08-07T15:13:10.100 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.100 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.101 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:10.101 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670868+0000", 2020-08-07T15:13:10.101 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6205000000000003e-05 2020-08-07T15:13:10.101 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.101 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.102 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.102 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670905+0000", 2020-08-07T15:13:10.102 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3289999999999996e-06 2020-08-07T15:13:10.102 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.102 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.103 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:10.103 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670912+0000", 2020-08-07T15:13:10.103 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75370200200000004 2020-08-07T15:13:10.104 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.104 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.104 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.105 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424614+0000", 2020-08-07T15:13:10.105 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5150000000000003e-06 2020-08-07T15:13:10.105 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.106 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.106 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:10.106 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424620+0000", 2020-08-07T15:13:10.106 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.768200153 2020-08-07T15:13:10.106 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.107 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.107 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.107 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192821+0000", 2020-08-07T15:13:10.107 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000111764 2020-08-07T15:13:10.107 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.108 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.108 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:10.108 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192932+0000", 2020-08-07T15:13:10.108 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017542400000000001 2020-08-07T15:13:10.108 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.108 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.109 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:10.110 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193108+0000", 2020-08-07T15:13:10.110 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004100851 2020-08-07T15:13:10.110 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.110 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.111 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:10.111 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197209+0000", 2020-08-07T15:13:10.111 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0100000000000001e-06 2020-08-07T15:13:10.111 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.111 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.112 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:10.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197210+0000", 2020-08-07T15:13:10.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9672999999999995e-05 2020-08-07T15:13:10.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:10.113 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197279+0000", 2020-08-07T15:13:10.113 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034275109999999998 2020-08-07T15:13:10.113 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.113 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.114 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:10.115 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200707+0000", 2020-08-07T15:13:10.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00036853100000000001 2020-08-07T15:13:10.116 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.116 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.116 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:10.116 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201075+0000", 2020-08-07T15:13:10.117 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8599999999999998e-07 2020-08-07T15:13:10.117 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.117 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:10.117 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201076+0000", 2020-08-07T15:13:10.118 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8483000000000003e-05 2020-08-07T15:13:10.118 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.118 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.118 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:10.118 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201124+0000", 2020-08-07T15:13:10.119 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.530301616 2020-08-07T15:13:10.119 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.119 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:10.120 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.120 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.120 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.120 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73019 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:10.121 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:13:10.121 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 223.79762683199999, 2020-08-07T15:13:10.121 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.529718224, 2020-08-07T15:13:10.121 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:10.121 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:10.122 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:10.122 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:10.122 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:10.122 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73019 2020-08-07T15:13:10.122 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.123 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:10.123 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.123 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:10.123 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:13:10.123 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:10.124 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.124 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.126 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:10.138 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:13:10.138 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3309000000000002e-05 2020-08-07T15:13:10.139 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.139 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.139 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:10.139 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671504+0000", 2020-08-07T15:13:10.139 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6619999999999998e-06 2020-08-07T15:13:10.140 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.140 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.140 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:10.140 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671507+0000", 2020-08-07T15:13:10.140 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2279999999999998e-06 2020-08-07T15:13:10.141 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.141 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.141 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:10.141 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671516+0000", 2020-08-07T15:13:10.141 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4860000000000002e-06 2020-08-07T15:13:10.142 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.143 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.143 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:10.143 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671519+0000", 2020-08-07T15:13:10.143 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0564000000000001e-05 2020-08-07T15:13:10.143 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.144 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.144 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.144 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671550+0000", 2020-08-07T15:13:10.144 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1029999999999998e-06 2020-08-07T15:13:10.144 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.144 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.145 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:10.145 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671556+0000", 2020-08-07T15:13:10.145 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75308759199999997 2020-08-07T15:13:10.145 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.145 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.146 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.146 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424643+0000", 2020-08-07T15:13:10.146 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9179999999999997e-06 2020-08-07T15:13:10.146 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.146 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.147 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:10.147 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424650+0000", 2020-08-07T15:13:10.147 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76890373599999995 2020-08-07T15:13:10.147 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.147 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.148 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.148 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193554+0000", 2020-08-07T15:13:10.148 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012345600000000001 2020-08-07T15:13:10.148 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.148 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.148 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:10.149 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193677+0000", 2020-08-07T15:13:10.149 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013528300000000001 2020-08-07T15:13:10.149 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.149 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.149 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:10.150 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193813+0000", 2020-08-07T15:13:10.150 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034777710000000002 2020-08-07T15:13:10.150 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.150 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.150 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:10.151 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197291+0000", 2020-08-07T15:13:10.151 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8000000000000004e-07 2020-08-07T15:13:10.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.151 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:10.152 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197291+0000", 2020-08-07T15:13:10.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9176999999999999e-05 2020-08-07T15:13:10.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:10.153 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197351+0000", 2020-08-07T15:13:10.161 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033931009999999999 2020-08-07T15:13:10.161 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.162 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.162 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:10.162 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200744+0000", 2020-08-07T15:13:10.162 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00041308000000000002 2020-08-07T15:13:10.162 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.163 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.163 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:10.163 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201157+0000", 2020-08-07T15:13:10.163 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8999999999999998e-07 2020-08-07T15:13:10.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:10.164 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201157+0000", 2020-08-07T15:13:10.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1588000000000003e-05 2020-08-07T15:13:10.164 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.165 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.165 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:10.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201189+0000", 2020-08-07T15:13:10.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.529718224 2020-08-07T15:13:10.165 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.166 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:10.166 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.166 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.166 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.167 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73020 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:10.167 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:13:10.167 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 223.72928897200001, 2020-08-07T15:13:10.167 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.461468373, 2020-08-07T15:13:10.167 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:10.168 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:10.168 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:10.168 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:10.168 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:10.168 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73020 2020-08-07T15:13:10.169 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.169 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:10.169 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.169 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:10.169 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:13:10.170 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:10.170 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.170 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.170 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:10.170 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:13:10.172 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.777e-05 2020-08-07T15:13:10.173 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.173 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.173 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:10.173 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739836+0000", 2020-08-07T15:13:10.173 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0180000000000003e-06 2020-08-07T15:13:10.174 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.174 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.174 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:10.174 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739840+0000", 2020-08-07T15:13:10.174 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6930000000000002e-06 2020-08-07T15:13:10.175 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.175 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.175 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:10.175 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739850+0000", 2020-08-07T15:13:10.175 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3619999999999999e-06 2020-08-07T15:13:10.176 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.176 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.176 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:10.177 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739854+0000", 2020-08-07T15:13:10.177 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7432999999999999e-05 2020-08-07T15:13:10.177 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.178 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.178 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.187 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739892+0000", 2020-08-07T15:13:10.188 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1149999999999996e-06 2020-08-07T15:13:10.188 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.188 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.189 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:10.189 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739899+0000", 2020-08-07T15:13:10.189 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68477883299999998 2020-08-07T15:13:10.190 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.190 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.190 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.191 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424677+0000", 2020-08-07T15:13:10.191 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7710000000000001e-06 2020-08-07T15:13:10.191 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.191 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.192 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:10.192 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424684+0000", 2020-08-07T15:13:10.192 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76949970700000003 2020-08-07T15:13:10.193 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.193 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.193 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.194 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194184+0000", 2020-08-07T15:13:10.194 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3957999999999997e-05 2020-08-07T15:13:10.194 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.194 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.195 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:10.195 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194278+0000", 2020-08-07T15:13:10.195 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017489599999999999 2020-08-07T15:13:10.204 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.204 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.205 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:10.205 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194453+0000", 2020-08-07T15:13:10.205 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002908908 2020-08-07T15:13:10.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.206 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.206 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:10.206 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197362+0000", 2020-08-07T15:13:10.206 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7299999999999995e-07 2020-08-07T15:13:10.207 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.207 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.207 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:10.207 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197362+0000", 2020-08-07T15:13:10.207 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6848000000000002e-05 2020-08-07T15:13:10.208 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.208 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.208 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:10.208 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197419+0000", 2020-08-07T15:13:10.209 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033597079999999999 2020-08-07T15:13:10.209 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.209 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.209 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:10.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200779+0000", 2020-08-07T15:13:10.210 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00044797899999999999 2020-08-07T15:13:10.210 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.210 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.210 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:10.210 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201227+0000", 2020-08-07T15:13:10.211 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7100000000000002e-07 2020-08-07T15:13:10.211 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.211 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.211 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:10.211 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201227+0000", 2020-08-07T15:13:10.212 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9230000000000001e-05 2020-08-07T15:13:10.222 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.222 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.222 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:10.222 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201277+0000", 2020-08-07T15:13:10.223 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.461468373 2020-08-07T15:13:10.223 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.223 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:10.223 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.223 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.224 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.224 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73021 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:10.224 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:13:10.224 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 223.66625536500001, 2020-08-07T15:13:10.225 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.398505079, 2020-08-07T15:13:10.225 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:10.226 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:10.226 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:10.226 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:10.226 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:10.227 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73021 2020-08-07T15:13:10.227 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.227 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:10.227 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.227 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:10.227 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:13:10.228 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:10.228 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.228 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.228 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:10.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:13:10.229 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7429000000000001e-05 2020-08-07T15:13:10.229 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.229 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.229 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:10.230 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802869+0000", 2020-08-07T15:13:10.230 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7840000000000001e-06 2020-08-07T15:13:10.230 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.230 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.230 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:10.231 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802873+0000", 2020-08-07T15:13:10.231 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0119999999999994e-06 2020-08-07T15:13:10.231 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.231 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.231 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:10.232 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802882+0000", 2020-08-07T15:13:10.232 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.04e-06 2020-08-07T15:13:10.232 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.232 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.232 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:10.233 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802887+0000", 2020-08-07T15:13:10.233 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7076000000000002e-05 2020-08-07T15:13:10.233 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.233 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.233 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.234 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802924+0000", 2020-08-07T15:13:10.234 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9280000000000004e-06 2020-08-07T15:13:10.234 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.234 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.234 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:10.235 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802931+0000", 2020-08-07T15:13:10.235 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.62178051000000001 2020-08-07T15:13:10.235 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.235 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.235 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.236 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424712+0000", 2020-08-07T15:13:10.236 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4749999999999996e-06 2020-08-07T15:13:10.236 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.237 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.237 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:10.238 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424719+0000", 2020-08-07T15:13:10.238 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77047786799999995 2020-08-07T15:13:10.239 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.239 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.240 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.240 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195197+0000", 2020-08-07T15:13:10.241 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000101909 2020-08-07T15:13:10.241 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.241 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.242 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:10.242 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195299+0000", 2020-08-07T15:13:10.242 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000248247 2020-08-07T15:13:10.244 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.244 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.244 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:10.244 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195547+0000", 2020-08-07T15:13:10.245 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033458699999999999 2020-08-07T15:13:10.245 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.246 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.246 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:10.246 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198893+0000", 2020-08-07T15:13:10.246 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0419999999999999e-06 2020-08-07T15:13:10.247 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.247 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.248 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:10.248 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198894+0000", 2020-08-07T15:13:10.248 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9354999999999997e-05 2020-08-07T15:13:10.249 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.249 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.249 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:10.250 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198953+0000", 2020-08-07T15:13:10.250 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001861096 2020-08-07T15:13:10.250 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.251 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.251 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:10.251 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200815+0000", 2020-08-07T15:13:10.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00049697299999999995 2020-08-07T15:13:10.252 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.252 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.253 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:10.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201311+0000", 2020-08-07T15:13:10.253 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7099999999999998e-07 2020-08-07T15:13:10.254 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.262 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.262 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:10.262 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201312+0000", 2020-08-07T15:13:10.263 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5194000000000002e-05 2020-08-07T15:13:10.263 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.263 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.263 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:10.263 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201347+0000", 2020-08-07T15:13:10.263 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.398505079 2020-08-07T15:13:10.264 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.264 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:10.264 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.264 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.264 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.265 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73022 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:10.265 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:13:10.265 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 223.58508330999999, 2020-08-07T15:13:10.265 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3238320669999999, 2020-08-07T15:13:10.266 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:10.266 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:10.266 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:10.266 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:10.266 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:10.267 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73022 2020-08-07T15:13:10.267 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.267 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:10.267 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.268 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:10.268 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:13:10.268 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:10.268 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.268 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.269 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:10.269 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:13:10.269 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6769999999999999e-05 2020-08-07T15:13:10.269 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.269 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.270 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:10.270 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884041+0000", 2020-08-07T15:13:10.270 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.703e-06 2020-08-07T15:13:10.270 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.270 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:10.271 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884044+0000", 2020-08-07T15:13:10.271 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5949999999999999e-06 2020-08-07T15:13:10.271 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.271 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:10.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884053+0000", 2020-08-07T15:13:10.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000003e-06 2020-08-07T15:13:10.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:10.273 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884058+0000", 2020-08-07T15:13:10.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7281999999999999e-05 2020-08-07T15:13:10.273 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.273 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.274 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884105+0000", 2020-08-07T15:13:10.274 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.044e-05 2020-08-07T15:13:10.274 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.274 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.275 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:10.275 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884116+0000", 2020-08-07T15:13:10.275 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.54062884300000003 2020-08-07T15:13:10.275 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.275 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.276 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.276 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424745+0000", 2020-08-07T15:13:10.276 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3149999999999998e-06 2020-08-07T15:13:10.276 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.276 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.276 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:10.277 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424754+0000", 2020-08-07T15:13:10.277 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77175732200000002 2020-08-07T15:13:10.277 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.277 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.277 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.278 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196511+0000", 2020-08-07T15:13:10.278 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010110799999999999 2020-08-07T15:13:10.278 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.278 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.278 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:10.279 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196612+0000", 2020-08-07T15:13:10.279 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017543100000000001 2020-08-07T15:13:10.279 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.279 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.279 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:10.280 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196788+0000", 2020-08-07T15:13:10.280 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032134569999999999 2020-08-07T15:13:10.280 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.280 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.281 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:10.281 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200001+0000", 2020-08-07T15:13:10.281 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.071e-06 2020-08-07T15:13:10.281 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.282 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.282 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:10.282 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200002+0000", 2020-08-07T15:13:10.282 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2527999999999995e-05 2020-08-07T15:13:10.283 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.283 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.283 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:10.284 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200065+0000", 2020-08-07T15:13:10.284 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001327433 2020-08-07T15:13:10.284 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.285 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.285 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:10.285 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201392+0000", 2020-08-07T15:13:10.286 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063854030000000004 2020-08-07T15:13:10.286 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.286 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.286 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:10.287 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207778+0000", 2020-08-07T15:13:10.287 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.285e-06 2020-08-07T15:13:10.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.288 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:10.288 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207779+0000", 2020-08-07T15:13:10.288 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7070999999999995e-05 2020-08-07T15:13:10.288 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.289 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.289 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:10.289 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207846+0000", 2020-08-07T15:13:10.290 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3238320669999999 2020-08-07T15:13:10.290 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.291 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:10.291 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.291 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.291 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.292 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73023 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:10.292 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:13:10.292 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 223.47192166900001, 2020-08-07T15:13:10.293 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2113701729999999, 2020-08-07T15:13:10.293 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:10.294 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:10.294 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:10.294 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:10.295 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:10.295 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73023 2020-08-07T15:13:10.295 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.296 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:10.296 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.296 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:10.297 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:13:10.297 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:10.297 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.298 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.298 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:10.298 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:13:10.298 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4105e-05 2020-08-07T15:13:10.298 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.299 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.299 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:10.299 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997210+0000", 2020-08-07T15:13:10.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1089999999999997e-06 2020-08-07T15:13:10.299 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.300 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.300 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:10.300 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997215+0000", 2020-08-07T15:13:10.300 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7040000000000006e-06 2020-08-07T15:13:10.300 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.300 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.301 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:10.301 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997224+0000", 2020-08-07T15:13:10.301 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4439999999999999e-06 2020-08-07T15:13:10.301 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.301 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.302 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:10.302 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997230+0000", 2020-08-07T15:13:10.302 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5977999999999999e-05 2020-08-07T15:13:10.302 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.302 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.303 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.303 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997276+0000", 2020-08-07T15:13:10.303 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2660000000000007e-06 2020-08-07T15:13:10.303 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.303 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.304 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:10.304 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997285+0000", 2020-08-07T15:13:10.304 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.42749189599999998 2020-08-07T15:13:10.304 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.304 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.304 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.305 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424777+0000", 2020-08-07T15:13:10.305 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8940000000000004e-06 2020-08-07T15:13:10.305 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.305 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.305 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:10.306 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424784+0000", 2020-08-07T15:13:10.306 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77268199699999995 2020-08-07T15:13:10.306 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.306 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.306 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.307 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197466+0000", 2020-08-07T15:13:10.307 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011277699999999999 2020-08-07T15:13:10.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.307 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.307 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:10.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197579+0000", 2020-08-07T15:13:10.308 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018970599999999999 2020-08-07T15:13:10.308 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.308 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:10.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197768+0000", 2020-08-07T15:13:10.309 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043168529999999998 2020-08-07T15:13:10.309 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.309 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.309 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:10.309 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202085+0000", 2020-08-07T15:13:10.310 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.186e-06 2020-08-07T15:13:10.310 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.310 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.310 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:10.310 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202086+0000", 2020-08-07T15:13:10.311 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8975999999999999e-05 2020-08-07T15:13:10.311 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.311 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.311 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:10.311 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202145+0000", 2020-08-07T15:13:10.312 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00035331499999999999 2020-08-07T15:13:10.312 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.312 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.312 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:10.312 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202499+0000", 2020-08-07T15:13:10.312 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0059703159999999998 2020-08-07T15:13:10.313 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.313 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.313 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:10.313 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208469+0000", 2020-08-07T15:13:10.313 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7400000000000002e-07 2020-08-07T15:13:10.314 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.314 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.314 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:10.314 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208470+0000", 2020-08-07T15:13:10.314 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5877000000000006e-05 2020-08-07T15:13:10.315 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.315 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.315 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:10.315 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208546+0000", 2020-08-07T15:13:10.315 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2113701729999999 2020-08-07T15:13:10.315 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.316 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:10.316 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.316 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.316 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.316 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5481.0:74192 6.e e269/230 6:7ab1a263:::benchmark_data_smithi074_63843_object9273:head v 269'4368, mlcod=268'4362)", 2020-08-07T15:13:10.317 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:01.594532+0000", 2020-08-07T15:13:10.317 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 7.8745649719999999, 2020-08-07T15:13:10.317 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.95244114199999996, 2020-08-07T15:13:10.317 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:10.317 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:10.318 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:10.318 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.318 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:10.318 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.594532+0000", 2020-08-07T15:13:10.318 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:10.319 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.319 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.319 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:10.319 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.594532+0000", 2020-08-07T15:13:10.319 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1347999999999997e-05 2020-08-07T15:13:10.319 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.320 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.320 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:10.320 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.594584+0000", 2020-08-07T15:13:10.321 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8689999999999998e-06 2020-08-07T15:13:10.321 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.321 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.322 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:10.322 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.594590+0000", 2020-08-07T15:13:10.322 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.95108761900000005 2020-08-07T15:13:10.323 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.323 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.323 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:10.324 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.545678+0000", 2020-08-07T15:13:10.324 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.271e-06 2020-08-07T15:13:10.324 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.324 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.324 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:10.325 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.545683+0000", 2020-08-07T15:13:10.325 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000113051 2020-08-07T15:13:10.325 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.326 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.326 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.326 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.545796+0000", 2020-08-07T15:13:10.326 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6154000000000001e-05 2020-08-07T15:13:10.326 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.327 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.327 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:10.327 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.545823+0000", 2020-08-07T15:13:10.327 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010668959999999999 2020-08-07T15:13:10.327 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.328 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.328 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:10.328 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.546889+0000", 2020-08-07T15:13:10.328 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3934e-05 2020-08-07T15:13:10.328 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.328 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.329 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:10.329 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.546973+0000", 2020-08-07T15:13:10.329 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.95244114199999996 2020-08-07T15:13:10.329 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.329 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:10.330 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.330 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.330 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.330 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:74203 6.9 6:9b24f442:::benchmark_data_smithi074_63843_object9275:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e269)", 2020-08-07T15:13:10.330 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:01.691147+0000", 2020-08-07T15:13:10.331 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 7.7779498680000003, 2020-08-07T15:13:10.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.91286480199999998, 2020-08-07T15:13:10.331 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:10.331 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:10.331 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:10.332 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:10.332 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:10.332 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 74203 2020-08-07T15:13:10.332 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.332 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:10.332 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.333 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:10.333 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.691147+0000", 2020-08-07T15:13:10.333 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:10.333 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.333 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.334 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:10.334 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.691147+0000", 2020-08-07T15:13:10.334 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0080999999999999e-05 2020-08-07T15:13:10.334 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.334 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.335 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:10.335 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.691177+0000", 2020-08-07T15:13:10.335 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7670000000000003e-06 2020-08-07T15:13:10.335 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.336 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:10.336 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.691182+0000", 2020-08-07T15:13:10.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0392e-05 2020-08-07T15:13:10.336 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.336 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.336 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:10.337 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.691193+0000", 2020-08-07T15:13:10.337 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5570000000000003e-06 2020-08-07T15:13:10.337 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.337 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.337 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:10.338 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.691198+0000", 2020-08-07T15:13:10.338 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0670000000000001e-05 2020-08-07T15:13:10.344 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.344 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.345 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.345 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.691249+0000", 2020-08-07T15:13:10.345 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0593e-05 2020-08-07T15:13:10.345 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.345 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.346 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:10.346 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.691259+0000", 2020-08-07T15:13:10.346 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.90310043900000003 2020-08-07T15:13:10.346 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.346 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.349 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.349 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.594360+0000", 2020-08-07T15:13:10.349 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000107182 2020-08-07T15:13:10.349 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.349 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.350 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:10.350 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.594467+0000", 2020-08-07T15:13:10.350 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00022166199999999999 2020-08-07T15:13:10.350 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.350 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.351 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:13:10.351 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.594689+0000", 2020-08-07T15:13:10.351 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042962399999999998 2020-08-07T15:13:10.351 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.351 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.352 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:10.352 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.598985+0000", 2020-08-07T15:13:10.352 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0759999999999999e-06 2020-08-07T15:13:10.352 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.352 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.353 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:10.353 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.598986+0000", 2020-08-07T15:13:10.353 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0080000000000002e-05 2020-08-07T15:13:10.353 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.353 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.355 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:10.355 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.599036+0000", 2020-08-07T15:13:10.355 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00053325400000000004 2020-08-07T15:13:10.355 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.355 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.356 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:10.356 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.599569+0000", 2020-08-07T15:13:10.356 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043431800000000003 2020-08-07T15:13:10.356 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.356 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.356 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:10.357 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.603913+0000", 2020-08-07T15:13:10.357 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5600000000000005e-07 2020-08-07T15:13:10.357 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.357 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.358 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:10.358 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.603913+0000", 2020-08-07T15:13:10.358 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8973000000000003e-05 2020-08-07T15:13:10.358 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.358 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.358 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:10.359 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.604012+0000", 2020-08-07T15:13:10.359 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.91286480199999998 2020-08-07T15:13:10.359 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.359 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:10.359 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.360 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.360 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:10.360 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:10.361 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:13:10.364 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:13:10.365 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.1 to 1 (10000) 2020-08-07T15:13:10.365 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:13:10.366 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 2 1 2020-08-07T15:13:10.366 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 58 15 86329 86314 11.625 12.8047 0.00811472 0.0107485 2020-08-07T15:13:10.367 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:10.360+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:10.462 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:10.461+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:10.564 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:10.561+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:10.646 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:10.646 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:13:10.646 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.647 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:86895 6.2 6:4937bcb2:::benchmark_data_smithi074_63843_object10861:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e273)", 2020-08-07T15:13:10.647 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:10.633810+0000", 2020-08-07T15:13:10.647 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0097116100000000007, 2020-08-07T15:13:10.647 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0097315449999999994, 2020-08-07T15:13:10.649 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:10.649 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:10.649 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:10.650 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:10.650 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:10.650 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 86895 2020-08-07T15:13:10.650 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.650 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:10.651 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.651 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:10.651 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.633810+0000", 2020-08-07T15:13:10.651 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:10.651 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.652 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.654 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:10.654 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.633810+0000", 2020-08-07T15:13:10.654 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8835000000000003e-05 2020-08-07T15:13:10.654 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.655 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.655 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:10.655 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.633859+0000", 2020-08-07T15:13:10.655 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7799999999999997e-06 2020-08-07T15:13:10.655 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.671 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.678 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:10.678 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.633865+0000", 2020-08-07T15:13:10.678 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.291e-05 2020-08-07T15:13:10.678 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.678 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.679 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:10.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.633877+0000", 2020-08-07T15:13:10.679 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4199999999999998e-06 2020-08-07T15:13:10.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.680 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:10.680 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.633883+0000", 2020-08-07T15:13:10.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0030749660000000002 2020-08-07T15:13:10.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.681 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.636958+0000", 2020-08-07T15:13:10.681 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000105005 2020-08-07T15:13:10.683 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.683 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.683 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:10.683 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.637063+0000", 2020-08-07T15:13:10.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019677400000000001 2020-08-07T15:13:10.684 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.684 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.684 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:10.684 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.637260+0000", 2020-08-07T15:13:10.684 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002905609 2020-08-07T15:13:10.685 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.686 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.688 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:10.688 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640165+0000", 2020-08-07T15:13:10.688 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4700000000000001e-07 2020-08-07T15:13:10.688 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.688 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.689 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:10.689 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640166+0000", 2020-08-07T15:13:10.689 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6582000000000001e-05 2020-08-07T15:13:10.689 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.690 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.690 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:10.690 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640223+0000", 2020-08-07T15:13:10.691 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026060300000000001 2020-08-07T15:13:10.691 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.691 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.691 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:10.691 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.642829+0000", 2020-08-07T15:13:10.692 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0090186580000000006 2020-08-07T15:13:10.692 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.692 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:10.692 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.693 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:86896 6.2 6:4937bcb2:::benchmark_data_smithi074_63843_object10861:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e273)", 2020-08-07T15:13:10.693 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:10.633921+0000", 2020-08-07T15:13:10.693 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0096009609999999999, 2020-08-07T15:13:10.693 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0096883899999999998, 2020-08-07T15:13:10.695 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:10.695 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:10.695 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:10.695 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:10.695 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:10.696 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 86896 2020-08-07T15:13:10.701 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.701 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:10.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.701 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:10.702 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.633921+0000", 2020-08-07T15:13:10.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:10.702 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.702 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.702 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:10.703 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.633921+0000", 2020-08-07T15:13:10.703 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0818e-05 2020-08-07T15:13:10.703 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.703 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:10.704 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.633971+0000", 2020-08-07T15:13:10.704 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4350000000000002e-06 2020-08-07T15:13:10.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.704 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:10.705 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.633978+0000", 2020-08-07T15:13:10.705 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0964999999999999e-05 2020-08-07T15:13:10.705 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.705 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.705 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:10.706 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.633989+0000", 2020-08-07T15:13:10.706 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9710000000000003e-06 2020-08-07T15:13:10.707 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.707 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:10.707 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.633994+0000", 2020-08-07T15:13:10.707 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042299979999999996 2020-08-07T15:13:10.708 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.708 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.708 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.638224+0000", 2020-08-07T15:13:10.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000102217 2020-08-07T15:13:10.709 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.709 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.709 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:10.709 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.638326+0000", 2020-08-07T15:13:10.709 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000162471 2020-08-07T15:13:10.710 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.710 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.710 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:10.710 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.638488+0000", 2020-08-07T15:13:10.710 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0022634729999999998 2020-08-07T15:13:10.710 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.711 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:10.711 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640752+0000", 2020-08-07T15:13:10.711 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1999999999999998e-07 2020-08-07T15:13:10.711 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.712 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.712 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:10.712 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640753+0000", 2020-08-07T15:13:10.712 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2519000000000001e-05 2020-08-07T15:13:10.712 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.713 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.713 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:10.714 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640825+0000", 2020-08-07T15:13:10.714 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069047869999999999 2020-08-07T15:13:10.714 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.714 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:10.715 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.715 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.715 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.715 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:86848 6.0 6:0e6ea492:::benchmark_data_smithi074_63843_object10855:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e273)", 2020-08-07T15:13:10.716 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:10.606603+0000", 2020-08-07T15:13:10.716 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.036918769999999997, 2020-08-07T15:13:10.716 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.037080720999999997, 2020-08-07T15:13:10.716 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:10.716 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:10.717 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:10.717 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:10.717 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:10.717 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 86848 2020-08-07T15:13:10.717 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.718 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:10.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:10.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.606603+0000", 2020-08-07T15:13:10.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:10.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:10.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.606603+0000", 2020-08-07T15:13:10.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0163000000000001e-05 2020-08-07T15:13:10.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.721 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:10.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.606653+0000", 2020-08-07T15:13:10.722 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0108e-05 2020-08-07T15:13:10.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.722 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:10.722 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.606663+0000", 2020-08-07T15:13:10.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2537999999999999e-05 2020-08-07T15:13:10.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.723 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:10.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.606676+0000", 2020-08-07T15:13:10.724 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0719999999999996e-06 2020-08-07T15:13:10.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.726 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.726 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:10.726 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.606682+0000", 2020-08-07T15:13:10.727 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00029502400000000001 2020-08-07T15:13:10.727 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.727 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.727 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.727 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.606977+0000", 2020-08-07T15:13:10.728 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000102279 2020-08-07T15:13:10.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:10.728 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.607079+0000", 2020-08-07T15:13:10.728 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017829400000000001 2020-08-07T15:13:10.729 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.729 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.729 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,2", 2020-08-07T15:13:10.729 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.607257+0000", 2020-08-07T15:13:10.729 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0040968990000000002 2020-08-07T15:13:10.730 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.730 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.730 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:10.730 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.611354+0000", 2020-08-07T15:13:10.730 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4679999999999999e-06 2020-08-07T15:13:10.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.731 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:10.731 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.611357+0000", 2020-08-07T15:13:10.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.562e-05 2020-08-07T15:13:10.732 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.732 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:10.732 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.611402+0000", 2020-08-07T15:13:10.732 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0088247900000000008 2020-08-07T15:13:10.733 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.742 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.742 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:10.742 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.620227+0000", 2020-08-07T15:13:10.742 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.013624255 2020-08-07T15:13:10.743 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.743 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:10.743 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.743 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.744 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:86905 6.c 6:3e66e64e:::benchmark_data_smithi074_63843_object10863:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e273)", 2020-08-07T15:13:10.744 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:10.637956+0000", 2020-08-07T15:13:10.744 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0055659890000000004, 2020-08-07T15:13:10.744 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0058740900000000002, 2020-08-07T15:13:10.744 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:10.745 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:10.745 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:10.745 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:10.745 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:10.745 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 86905 2020-08-07T15:13:10.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.759 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:10.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.759 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:10.760 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.637956+0000", 2020-08-07T15:13:10.760 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:10.760 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.760 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.760 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:10.761 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.637956+0000", 2020-08-07T15:13:10.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0843000000000001e-05 2020-08-07T15:13:10.761 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.782 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:10.782 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.638006+0000", 2020-08-07T15:13:10.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8589999999999999e-06 2020-08-07T15:13:10.782 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.782 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:10.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.638013+0000", 2020-08-07T15:13:10.783 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4243999999999999e-05 2020-08-07T15:13:10.783 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:10.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.638028+0000", 2020-08-07T15:13:10.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9170000000000003e-06 2020-08-07T15:13:10.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:10.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.638033+0000", 2020-08-07T15:13:10.785 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000109669 2020-08-07T15:13:10.785 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.638143+0000", 2020-08-07T15:13:10.799 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013961799999999999 2020-08-07T15:13:10.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:10.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.638283+0000", 2020-08-07T15:13:10.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013395799999999999 2020-08-07T15:13:10.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.808 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:13:10.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.638417+0000", 2020-08-07T15:13:10.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0031369119999999999 2020-08-07T15:13:10.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.809 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:10.809 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.641554+0000", 2020-08-07T15:13:10.809 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0529999999999999e-06 2020-08-07T15:13:10.810 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.810 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.810 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:10.810 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.641555+0000", 2020-08-07T15:13:10.810 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0825999999999997e-05 2020-08-07T15:13:10.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.811 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:10.811 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.641615+0000", 2020-08-07T15:13:10.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000672677 2020-08-07T15:13:10.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:10.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.642288+0000", 2020-08-07T15:13:10.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043325760000000003 2020-08-07T15:13:10.812 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.813 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:10.813 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.813 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:86906 6.c 6:3e66e64e:::benchmark_data_smithi074_63843_object10863:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e273)", 2020-08-07T15:13:10.825 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:10.640081+0000", 2020-08-07T15:13:10.825 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0034404520000000001, 2020-08-07T15:13:10.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038690489999999998, 2020-08-07T15:13:10.826 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:10.826 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:13:10.826 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:10.826 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:10.826 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:10.827 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 86906 2020-08-07T15:13:10.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.827 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:10.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:10.828 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640081+0000", 2020-08-07T15:13:10.829 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:10.829 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:10.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640081+0000", 2020-08-07T15:13:10.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2247000000000003e-05 2020-08-07T15:13:10.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:10.836 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640133+0000", 2020-08-07T15:13:10.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.725e-06 2020-08-07T15:13:10.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.836 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:10.836 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640140+0000", 2020-08-07T15:13:10.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3033e-05 2020-08-07T15:13:10.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:10.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640153+0000", 2020-08-07T15:13:10.838 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2930000000000003e-06 2020-08-07T15:13:10.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.838 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:10.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640160+0000", 2020-08-07T15:13:10.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4855999999999998e-05 2020-08-07T15:13:10.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.839 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640215+0000", 2020-08-07T15:13:10.840 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000118559 2020-08-07T15:13:10.840 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:10.841 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640334+0000", 2020-08-07T15:13:10.841 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020808099999999999 2020-08-07T15:13:10.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.843 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:13:10.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640542+0000", 2020-08-07T15:13:10.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00046079399999999999 2020-08-07T15:13:10.845 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.845 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:10.845 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.846 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.846 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:86907 6.c 6:3e66e64e:::benchmark_data_smithi074_63843_object10863:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e273)", 2020-08-07T15:13:10.846 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:10.640537+0000", 2020-08-07T15:13:10.846 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0029848019999999999, 2020-08-07T15:13:10.847 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003472934, 2020-08-07T15:13:10.847 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:10.847 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:13:10.847 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:10.847 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:10.848 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:10.848 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 86907 2020-08-07T15:13:10.848 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.848 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:10.848 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.849 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:10.849 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640537+0000", 2020-08-07T15:13:10.849 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:10.849 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.850 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:10.850 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640537+0000", 2020-08-07T15:13:10.850 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9755000000000001e-05 2020-08-07T15:13:10.851 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.851 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.851 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:10.852 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640587+0000", 2020-08-07T15:13:10.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4509999999999997e-06 2020-08-07T15:13:10.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.854 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:10.854 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640593+0000", 2020-08-07T15:13:10.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2412000000000001e-05 2020-08-07T15:13:10.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.855 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:10.855 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640605+0000", 2020-08-07T15:13:10.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5180000000000004e-06 2020-08-07T15:13:10.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.856 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.856 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:10.856 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640611+0000", 2020-08-07T15:13:10.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00028103399999999999 2020-08-07T15:13:10.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.857 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.857 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640892+0000", 2020-08-07T15:13:10.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9705999999999997e-05 2020-08-07T15:13:10.858 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.858 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:10.858 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640992+0000", 2020-08-07T15:13:10.877 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019395799999999999 2020-08-07T15:13:10.878 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.878 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.878 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:13:10.878 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.641186+0000", 2020-08-07T15:13:10.878 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00064883400000000004 2020-08-07T15:13:10.879 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.879 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:10.879 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.879 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.880 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:86908 6.c 6:3e66e64e:::benchmark_data_smithi074_63843_object10863:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e273)", 2020-08-07T15:13:10.880 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:10.640648+0000", 2020-08-07T15:13:10.880 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.002873277, 2020-08-07T15:13:10.880 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034365620000000002, 2020-08-07T15:13:10.880 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:10.881 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:13:10.881 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:10.881 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:10.881 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:10.881 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 86908 2020-08-07T15:13:10.882 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.882 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:10.883 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.883 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:10.883 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640648+0000", 2020-08-07T15:13:10.883 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:10.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:10.887 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640648+0000", 2020-08-07T15:13:10.887 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1749000000000002e-05 2020-08-07T15:13:10.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:10.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640700+0000", 2020-08-07T15:13:10.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4659999999999998e-06 2020-08-07T15:13:10.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.891 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:10.891 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640706+0000", 2020-08-07T15:13:10.891 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1194e-05 2020-08-07T15:13:10.892 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.892 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.892 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:10.893 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640717+0000", 2020-08-07T15:13:10.894 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.702e-06 2020-08-07T15:13:10.895 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.895 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.896 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:10.896 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.640721+0000", 2020-08-07T15:13:10.896 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00092631600000000005 2020-08-07T15:13:10.897 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.897 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.897 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.898 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.641648+0000", 2020-08-07T15:13:10.898 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000108632 2020-08-07T15:13:10.898 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.899 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.899 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:10.899 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.641756+0000", 2020-08-07T15:13:10.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000171836 2020-08-07T15:13:10.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.901 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:13:10.901 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.641928+0000", 2020-08-07T15:13:10.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012798950000000001 2020-08-07T15:13:10.902 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.902 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:10.902 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.903 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.903 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:86909 6.c 6:3e66e64e:::benchmark_data_smithi074_63843_object10863:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e273)", 2020-08-07T15:13:10.904 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:10.641044+0000", 2020-08-07T15:13:10.905 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0024778030000000002, 2020-08-07T15:13:10.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0030859329999999999, 2020-08-07T15:13:10.906 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:10.906 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:13:10.907 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:10.907 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:10.907 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:10.908 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 86909 2020-08-07T15:13:10.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.912 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:10.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:10.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.641044+0000", 2020-08-07T15:13:10.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:10.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:10.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.641044+0000", 2020-08-07T15:13:10.914 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2623000000000001e-05 2020-08-07T15:13:10.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:10.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.641096+0000", 2020-08-07T15:13:10.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4919999999999998e-06 2020-08-07T15:13:10.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:10.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.641103+0000", 2020-08-07T15:13:10.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2478e-05 2020-08-07T15:13:10.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:10.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.641115+0000", 2020-08-07T15:13:10.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6919999999999999e-06 2020-08-07T15:13:10.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.918 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:10.918 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.641121+0000", 2020-08-07T15:13:10.918 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012342360000000001 2020-08-07T15:13:10.919 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.919 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.919 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.642355+0000", 2020-08-07T15:13:10.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5829999999999996e-05 2020-08-07T15:13:10.919 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:10.920 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.642451+0000", 2020-08-07T15:13:10.920 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000167505 2020-08-07T15:13:10.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:13:10.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.642619+0000", 2020-08-07T15:13:10.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015748559999999999 2020-08-07T15:13:10.921 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.922 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:10.922 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.922 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.922 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:86910 6.c 6:3e66e64e:::benchmark_data_smithi074_63843_object10863:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e273)", 2020-08-07T15:13:10.923 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:10.642438+0000", 2020-08-07T15:13:10.923 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0010831440000000001, 2020-08-07T15:13:10.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017416720000000001, 2020-08-07T15:13:10.923 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:10.923 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:13:10.924 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:10.924 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:10.924 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:10.924 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 86910 2020-08-07T15:13:10.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.925 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:10.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:10.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.642438+0000", 2020-08-07T15:13:10.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:10.926 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.926 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:10.926 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.642438+0000", 2020-08-07T15:13:10.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1020999999999998e-05 2020-08-07T15:13:10.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:10.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.642489+0000", 2020-08-07T15:13:10.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9199999999999998e-06 2020-08-07T15:13:10.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:10.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.642496+0000", 2020-08-07T15:13:10.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3677e-05 2020-08-07T15:13:10.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:10.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.642510+0000", 2020-08-07T15:13:10.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4130000000000003e-06 2020-08-07T15:13:10.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:10.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.642516+0000", 2020-08-07T15:13:10.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00048449000000000002 2020-08-07T15:13:10.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:10.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.643001+0000", 2020-08-07T15:13:10.931 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9235999999999997e-05 2020-08-07T15:13:10.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.932 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.932 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:10.933 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.643090+0000", 2020-08-07T15:13:10.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020582400000000001 2020-08-07T15:13:10.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.933 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.933 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:13:10.934 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.643296+0000", 2020-08-07T15:13:10.934 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00085758099999999995 2020-08-07T15:13:10.934 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.935 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:10.935 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.935 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:86906 6.c e273/230)", 2020-08-07T15:13:10.936 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:10.643207+0000", 2020-08-07T15:13:10.936 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.00031482299999999999, 2020-08-07T15:13:10.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010187760000000001, 2020-08-07T15:13:10.936 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:10.936 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:13:10.937 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:10.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:10.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.643207+0000", 2020-08-07T15:13:10.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:10.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.938 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:10.938 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.643207+0000", 2020-08-07T15:13:10.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4613e-05 2020-08-07T15:13:10.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:10.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.643231+0000", 2020-08-07T15:13:10.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.087e-06 2020-08-07T15:13:10.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.940 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.940 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:10.940 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.643235+0000", 2020-08-07T15:13:10.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6167999999999994e-05 2020-08-07T15:13:10.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:10.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.643302+0000", 2020-08-07T15:13:10.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.794e-06 2020-08-07T15:13:10.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.942 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:10.942 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.643305+0000", 2020-08-07T15:13:10.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8661999999999998e-05 2020-08-07T15:13:10.942 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.943 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:10.943 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.943 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:86907 6.c e273/230)", 2020-08-07T15:13:10.944 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:10.643266+0000", 2020-08-07T15:13:10.944 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.00025546199999999997, 2020-08-07T15:13:10.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00099167999999999999, 2020-08-07T15:13:10.944 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:10.944 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:13:10.944 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:10.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:10.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.643266+0000", 2020-08-07T15:13:10.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:10.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.946 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.946 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:10.946 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.643266+0000", 2020-08-07T15:13:10.946 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.8581e-05 2020-08-07T15:13:10.946 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.947 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.947 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:10.947 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.643285+0000", 2020-08-07T15:13:10.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6869999999999999e-06 2020-08-07T15:13:10.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.948 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:10.948 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.643287+0000", 2020-08-07T15:13:10.948 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0304999999999999e-05 2020-08-07T15:13:10.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.949 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:10.949 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.643318+0000", 2020-08-07T15:13:10.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1700000000000001e-06 2020-08-07T15:13:10.949 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:10.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:10.950 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:10.950 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:10.643321+0000", 2020-08-07T15:13:10.950 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4743000000000001e-05 2020-08-07T15:13:10.950 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.950 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:10.951 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.951 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:10.951 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:13:10.951 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 11 2020-08-07T15:13:10.951 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:10.955 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:13:10.955 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:13:10.955 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:13:10.956 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:13:10.956 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:13:10.956 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:13:10.957 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:13:10.957 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:13:10.957 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:10.952+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:10.983 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:13:10.984 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:13:10.984 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:13:10.985 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:13:10.985 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:13:10.985 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:13:10.986 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:13:10.986 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:13:10.987 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:13:11.055 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:11.053+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:11.161 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:11.154+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:11.172 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:11.172 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:11.184 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:13:11.262 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:11.256+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:11.277 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 59 16 87869 87853 11.6317 12.0234 0.00711643 0.0107418 2020-08-07T15:13:11.362 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:11.359+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:11.430 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:11.430 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:11.445 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:13:11.489 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:11.485+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:11.576 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:11.564+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:11.665 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.2 to 1 (10000) 2020-08-07T15:13:11.670 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:11.668+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:11.680 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 3 1 2020-08-07T15:13:11.706 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:11.706 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:11.717 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:13:11.785 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:11.777+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:11.882 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:11.879+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:11.908 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:11.908 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:11.919 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:13:11.995 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:11.993+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:12.077 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:12.075+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:12.085 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:12.086 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:12.096 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:13:12.187 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:12.183+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:12.251 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.3 to 1 (10000) 2020-08-07T15:13:12.267 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 4 1 2020-08-07T15:13:12.274 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:2020-08-07T15:13:12.270972+0000 min lat: 0.000909485 max lat: 1.58913 avg lat: 0.0107384 2020-08-07T15:13:12.274 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-08-07T15:13:12.274 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 60 16 89372 89356 11.6336 11.7422 0.0155178 0.0107384 2020-08-07T15:13:12.282 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:12.280+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:12.286 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:12.287 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:12.300 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:13:12.368 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:13:12.391 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:12.387+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:12.488 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:12.485+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:12.498 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:12.498 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:12.505 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:13:12.605 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:12.589+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:12.703 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:12.689+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:12.705 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:12.705 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:12.715 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:13:12.793 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:12.791+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:12.874 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:12.874 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:12.888 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:13:12.894 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:12.891+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:12.991 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:13:13.013 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:13.011+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:13.086 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:13.087 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:13.098 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:13:13.115 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:13.113+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:13.220 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:13.217+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:13.220 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:13:13.258 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:13.258 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:13.259 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:13:13.269 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:13:13.272 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 61 16 90795 90779 11.6251 11.1172 0.0038162 0.0107472 2020-08-07T15:13:13.305 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:13:13.338 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:13.327+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:13.343 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:13:13.392 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:13:13.434 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:13.434 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:13.435 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:13.433+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:13.442 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:13:13.450 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:13:13.479 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:13:13.508 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:13:13.508 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:13:13.509 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:13:13.509 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:13:13.509 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:13:13.509 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:13:13.509 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:13:13.518 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:13:13.518 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:13:13.565 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:13.560+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:13.606 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:13:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:13:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:13:13.608 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:13.615 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:13:13.643 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:13.642+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:13.670 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.4 to 1 (10000) 2020-08-07T15:13:13.670 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:13:13.694 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 5 1 2020-08-07T15:13:13.744 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:13.743+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:13.766 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:13.766 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:13:13.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:13:13.767 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:13:13.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.767 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15576 3.e 3:727724a5:::benchmark_data_smithi074_35447_object1946:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:13:13.767 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:13:13.768 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 387.09611560500002, 2020-08-07T15:13:13.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999, 2020-08-07T15:13:13.768 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:13.768 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:13.769 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:13.769 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:13:13.769 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:13:13.769 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15576 2020-08-07T15:13:13.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.772 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:13.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:13.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:13:13.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:13.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:13.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:13:13.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7156e-05 2020-08-07T15:13:13.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:13.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666682+0000", 2020-08-07T15:13:13.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0209999999999999e-06 2020-08-07T15:13:13.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:13.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666687+0000", 2020-08-07T15:13:13.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0730999999999999e-05 2020-08-07T15:13:13.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:13.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666698+0000", 2020-08-07T15:13:13.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2020-08-07T15:13:13.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:13.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666702+0000", 2020-08-07T15:13:13.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023978459999999999 2020-08-07T15:13:13.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:13.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669100+0000", 2020-08-07T15:13:13.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9664000000000001e-05 2020-08-07T15:13:13.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:13.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669190+0000", 2020-08-07T15:13:13.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015957100000000001 2020-08-07T15:13:13.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:13:13.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669350+0000", 2020-08-07T15:13:13.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016470149999999999 2020-08-07T15:13:13.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:13.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:13:13.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1299999999999999e-07 2020-08-07T15:13:13.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:13.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:13:13.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5269000000000002e-05 2020-08-07T15:13:13.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:13.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.671022+0000", 2020-08-07T15:13:13.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031414400000000001 2020-08-07T15:13:13.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:13.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.674163+0000", 2020-08-07T15:13:13.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45641417400000001 2020-08-07T15:13:13.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:13.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:13:13.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0399999999999995e-07 2020-08-07T15:13:13.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:13.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:13:13.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3343999999999997e-05 2020-08-07T15:13:13.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:13.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130622+0000", 2020-08-07T15:13:13.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999 2020-08-07T15:13:13.827 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:13.828 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:13.828 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:13.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.828 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106248 3.f 3:f950fd9d:::benchmark_data_smithi074_35447_object13280:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:13:13.829 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:13:13.829 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 326.40258606499998, 2020-08-07T15:13:13.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002, 2020-08-07T15:13:13.829 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:13.829 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:13.839 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:13.839 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:13:13.839 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:13:13.840 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106248 2020-08-07T15:13:13.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.840 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:13.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:13.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:13:13.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:13.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:13.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:13:13.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7184e-05 2020-08-07T15:13:13.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:13.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360222+0000", 2020-08-07T15:13:13.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0906e-05 2020-08-07T15:13:13.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:13.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360233+0000", 2020-08-07T15:13:13.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2972999999999999e-05 2020-08-07T15:13:13.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360246+0000", 2020-08-07T15:13:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0114e-05 2020-08-07T15:13:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360256+0000", 2020-08-07T15:13:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1303999999999997e-05 2020-08-07T15:13:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360287+0000", 2020-08-07T15:13:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3798000000000001e-05 2020-08-07T15:13:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360351+0000", 2020-08-07T15:13:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111308 2020-08-07T15:13:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:13:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360462+0000", 2020-08-07T15:13:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003480674 2020-08-07T15:13:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:13:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0900000000000001e-07 2020-08-07T15:13:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:13:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8899999999999994e-05 2020-08-07T15:13:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.364012+0000", 2020-08-07T15:13:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35291637199999998 2020-08-07T15:13:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716929+0000", 2020-08-07T15:13:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.076244233999999994 2020-08-07T15:13:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793173+0000", 2020-08-07T15:13:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6109999999999999e-06 2020-08-07T15:13:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793175+0000", 2020-08-07T15:13:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7492999999999999e-05 2020-08-07T15:13:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793242+0000", 2020-08-07T15:13:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002 2020-08-07T15:13:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106248 3.f e59/20 ondisk, result = 0)", 2020-08-07T15:13:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:13:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 326.39732278000002, 2020-08-07T15:13:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001, 2020-08-07T15:13:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:13:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:13:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7518999999999999e-05 2020-08-07T15:13:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365445+0000", 2020-08-07T15:13:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-06 2020-08-07T15:13:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365449+0000", 2020-08-07T15:13:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42764147600000002 2020-08-07T15:13:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793091+0000", 2020-08-07T15:13:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6500000000000002e-06 2020-08-07T15:13:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:13.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793099+0000", 2020-08-07T15:13:13.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3664e-05 2020-08-07T15:13:13.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:13.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793153+0000", 2020-08-07T15:13:13.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1078e-05 2020-08-07T15:13:13.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:13.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793164+0000", 2020-08-07T15:13:13.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3404000000000003e-05 2020-08-07T15:13:13.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:13.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793248+0000", 2020-08-07T15:13:13.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001 2020-08-07T15:13:13.990 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:13.990 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:13.991 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:13.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:13.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.991 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:107952 3.d e61/35)", 2020-08-07T15:13:13.991 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:13:13.992 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 323.43639307199999, 2020-08-07T15:13:13.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001, 2020-08-07T15:13:13.992 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:13.992 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:13.992 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:13.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:13.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:13.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:13:13.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:14.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:14.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:13:14.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8826000000000003e-05 2020-08-07T15:13:14.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:14.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326406+0000", 2020-08-07T15:13:14.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9199999999999998e-06 2020-08-07T15:13:14.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:14.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326413+0000", 2020-08-07T15:13:14.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75082853500000002 2020-08-07T15:13:14.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:14.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077242+0000", 2020-08-07T15:13:14.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1479999999999999e-06 2020-08-07T15:13:14.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:14.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077246+0000", 2020-08-07T15:13:14.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3331999999999998e-05 2020-08-07T15:13:14.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:14.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077289+0000", 2020-08-07T15:13:14.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8977000000000001e-05 2020-08-07T15:13:14.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:14.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077308+0000", 2020-08-07T15:13:14.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001 2020-08-07T15:13:14.022 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.022 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:14.022 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.023 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31696 4.0 4:0335eed9:::benchmark_data_smithi074_44586_object3961:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e100)", 2020-08-07T15:13:14.023 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:13:14.024 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 265.65848551300002, 2020-08-07T15:13:14.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237, 2020-08-07T15:13:14.024 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:14.024 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:14.024 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:14.025 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:14.025 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:14.025 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31696 2020-08-07T15:13:14.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.025 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:14.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:14.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:13:14.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:14.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:14.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:13:14.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7654e-05 2020-08-07T15:13:14.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:14.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104293+0000", 2020-08-07T15:13:14.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.061e-06 2020-08-07T15:13:14.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:14.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104296+0000", 2020-08-07T15:13:14.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0859999999999998e-06 2020-08-07T15:13:14.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:14.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104302+0000", 2020-08-07T15:13:14.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8030000000000001e-06 2020-08-07T15:13:14.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:14.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104305+0000", 2020-08-07T15:13:14.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037634769999999999 2020-08-07T15:13:14.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:14.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108068+0000", 2020-08-07T15:13:14.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8814999999999998e-05 2020-08-07T15:13:14.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:14.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108137+0000", 2020-08-07T15:13:14.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111455 2020-08-07T15:13:14.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:13:14.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108248+0000", 2020-08-07T15:13:14.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028756820000000001 2020-08-07T15:13:14.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:14.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111124+0000", 2020-08-07T15:13:14.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061054499999999999 2020-08-07T15:13:14.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:14.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111735+0000", 2020-08-07T15:13:14.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.423966167 2020-08-07T15:13:14.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:14.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535701+0000", 2020-08-07T15:13:14.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4929999999999999e-06 2020-08-07T15:13:14.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:14.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535702+0000", 2020-08-07T15:13:14.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2571999999999997e-05 2020-08-07T15:13:14.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:14.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535755+0000", 2020-08-07T15:13:14.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1427000000000003e-05 2020-08-07T15:13:14.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:14.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535786+0000", 2020-08-07T15:13:14.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237 2020-08-07T15:13:14.052 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.052 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:14.053 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.053 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:31696 4.0 e100/81 ondisk, result = 0)", 2020-08-07T15:13:14.054 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:13:14.054 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 265.64887095500001, 2020-08-07T15:13:14.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954, 2020-08-07T15:13:14.054 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:14.054 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:14.055 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:14.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:14.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:13:14.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:14.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:14.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:13:14.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1185000000000001e-05 2020-08-07T15:13:14.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:14.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113901+0000", 2020-08-07T15:13:14.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2930000000000002e-06 2020-08-07T15:13:14.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:14.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113905+0000", 2020-08-07T15:13:14.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42168640400000001 2020-08-07T15:13:14.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:14.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535591+0000", 2020-08-07T15:13:14.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2820000000000002e-06 2020-08-07T15:13:14.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:14.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535598+0000", 2020-08-07T15:13:14.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6578999999999999e-05 2020-08-07T15:13:14.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:14.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535684+0000", 2020-08-07T15:13:14.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8449999999999992e-06 2020-08-07T15:13:14.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:14.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535693+0000", 2020-08-07T15:13:14.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7366000000000003e-05 2020-08-07T15:13:14.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:14.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535791+0000", 2020-08-07T15:13:14.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954 2020-08-07T15:13:14.064 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.064 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:14.064 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.065 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:37024 4.0 e106/81)", 2020-08-07T15:13:14.065 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:13:14.065 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 259.53408206099999, 2020-08-07T15:13:14.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999, 2020-08-07T15:13:14.066 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:14.066 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:14.066 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:14.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:14.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:13:14.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:14.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:14.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:13:14.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4926000000000001e-05 2020-08-07T15:13:14.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:14.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228683+0000", 2020-08-07T15:13:14.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1280000000000002e-06 2020-08-07T15:13:14.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:14.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228686+0000", 2020-08-07T15:13:14.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69740012100000004 2020-08-07T15:13:14.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:14.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926086+0000", 2020-08-07T15:13:14.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000004e-06 2020-08-07T15:13:14.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:14.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926090+0000", 2020-08-07T15:13:14.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113526 2020-08-07T15:13:14.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:14.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926204+0000", 2020-08-07T15:13:14.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8824000000000001e-05 2020-08-07T15:13:14.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:14.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926222+0000", 2020-08-07T15:13:14.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999 2020-08-07T15:13:14.083 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.083 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:14.083 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.084 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42503 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2615, mlcod=112'2607)", 2020-08-07T15:13:14.084 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:13:14.084 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 253.466510145, 2020-08-07T15:13:14.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998, 2020-08-07T15:13:14.085 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:14.087 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:14.087 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:14.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:14.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:13:14.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:14.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:14.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:13:14.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270999999999998e-05 2020-08-07T15:13:14.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:14.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296292+0000", 2020-08-07T15:13:14.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4729999999999997e-06 2020-08-07T15:13:14.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:14.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296298+0000", 2020-08-07T15:13:14.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.448795049 2020-08-07T15:13:14.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:14.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745093+0000", 2020-08-07T15:13:14.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0140000000000001e-06 2020-08-07T15:13:14.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:14.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745100+0000", 2020-08-07T15:13:14.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2596999999999998e-05 2020-08-07T15:13:14.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:14.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745153+0000", 2020-08-07T15:13:14.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1662000000000003e-05 2020-08-07T15:13:14.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:14.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745184+0000", 2020-08-07T15:13:14.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093174800000000004 2020-08-07T15:13:14.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:14.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746116+0000", 2020-08-07T15:13:14.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0926999999999994e-05 2020-08-07T15:13:14.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:14.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746187+0000", 2020-08-07T15:13:14.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998 2020-08-07T15:13:14.098 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.098 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:14.099 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.099 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42504 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2616, mlcod=113'2609)", 2020-08-07T15:13:14.099 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:13:14.100 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 253.46317325199999, 2020-08-07T15:13:14.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998, 2020-08-07T15:13:14.100 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:14.102 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:14.102 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:14.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:14.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:13:14.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:14.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:14.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:13:14.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2154000000000002e-05 2020-08-07T15:13:14.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:14.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299629+0000", 2020-08-07T15:13:14.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1929999999999998e-06 2020-08-07T15:13:14.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:14.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299636+0000", 2020-08-07T15:13:14.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44766315400000001 2020-08-07T15:13:14.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:14.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747299+0000", 2020-08-07T15:13:14.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5889999999999999e-06 2020-08-07T15:13:14.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:14.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747305+0000", 2020-08-07T15:13:14.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3094999999999999e-05 2020-08-07T15:13:14.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:14.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747358+0000", 2020-08-07T15:13:14.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3283e-05 2020-08-07T15:13:14.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:14.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747392+0000", 2020-08-07T15:13:14.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015319509999999999 2020-08-07T15:13:14.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:14.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748924+0000", 2020-08-07T15:13:14.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5235999999999996e-05 2020-08-07T15:13:14.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:14.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748999+0000", 2020-08-07T15:13:14.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998 2020-08-07T15:13:14.114 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.114 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:14.114 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.115 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42520 4.8 e113/81 4:13071de5:::benchmark_data_smithi074_44586_object5314:head v 113'2624, mlcod=113'2618)", 2020-08-07T15:13:14.115 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:13:14.115 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 252.997229043, 2020-08-07T15:13:14.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001, 2020-08-07T15:13:14.116 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:14.116 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:14.116 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:14.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:14.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:13:14.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:14.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:14.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:13:14.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3967999999999995e-05 2020-08-07T15:13:14.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:14.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765585+0000", 2020-08-07T15:13:14.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0949999999999998e-06 2020-08-07T15:13:14.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:14.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765593+0000", 2020-08-07T15:13:14.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52703464200000005 2020-08-07T15:13:14.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:14.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292627+0000", 2020-08-07T15:13:14.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6929999999999996e-06 2020-08-07T15:13:14.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:14.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292636+0000", 2020-08-07T15:13:14.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9292000000000001e-05 2020-08-07T15:13:14.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:14.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292695+0000", 2020-08-07T15:13:14.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5395999999999999e-05 2020-08-07T15:13:14.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:14.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292721+0000", 2020-08-07T15:13:14.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00090361399999999996 2020-08-07T15:13:14.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:14.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293624+0000", 2020-08-07T15:13:14.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1761000000000002e-05 2020-08-07T15:13:14.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:14.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293666+0000", 2020-08-07T15:13:14.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001 2020-08-07T15:13:14.129 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.129 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.130 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:19776 5.4 5:2bb0c0d9:::benchmark_data_smithi074_53697_object2471:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e173)", 2020-08-07T15:13:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:13:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 161.36303635799999, 2020-08-07T15:13:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.402061844, 2020-08-07T15:13:14.131 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:14.132 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 19776 2020-08-07T15:13:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:14.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:13:14.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:14.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:14.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:13:14.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-05 2020-08-07T15:13:14.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:14.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399752+0000", 2020-08-07T15:13:14.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8799999999999999e-06 2020-08-07T15:13:14.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:14.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399757+0000", 2020-08-07T15:13:14.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7690000000000008e-06 2020-08-07T15:13:14.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:14.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399767+0000", 2020-08-07T15:13:14.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-06 2020-08-07T15:13:14.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:14.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399771+0000", 2020-08-07T15:13:14.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003255109 2020-08-07T15:13:14.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:14.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403026+0000", 2020-08-07T15:13:14.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101044 2020-08-07T15:13:14.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:14.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403127+0000", 2020-08-07T15:13:14.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000193158 2020-08-07T15:13:14.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,5", 2020-08-07T15:13:14.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403320+0000", 2020-08-07T15:13:14.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017185530000000001 2020-08-07T15:13:14.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:14.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405038+0000", 2020-08-07T15:13:14.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6300000000000004e-07 2020-08-07T15:13:14.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:14.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405039+0000", 2020-08-07T15:13:14.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012590600000000001 2020-08-07T15:13:14.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:14.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405165+0000", 2020-08-07T15:13:14.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109757 2020-08-07T15:13:14.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:14.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405275+0000", 2020-08-07T15:13:14.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39642174699999999 2020-08-07T15:13:14.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:14.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801697+0000", 2020-08-07T15:13:14.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2799999999999996e-07 2020-08-07T15:13:14.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:14.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801697+0000", 2020-08-07T15:13:14.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8740999999999997e-05 2020-08-07T15:13:14.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:14.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801776+0000", 2020-08-07T15:13:14.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.402061844 2020-08-07T15:13:14.154 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.154 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:14.154 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:19776 5.4 e173/164 ondisk, result = 0)", 2020-08-07T15:13:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:13:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 161.357288055, 2020-08-07T15:13:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39631765200000002, 2020-08-07T15:13:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:13:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:13:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9332e-05 2020-08-07T15:13:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405492+0000", 2020-08-07T15:13:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8900000000000004e-06 2020-08-07T15:13:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405498+0000", 2020-08-07T15:13:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39566664200000001 2020-08-07T15:13:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801164+0000", 2020-08-07T15:13:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7440000000000001e-06 2020-08-07T15:13:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801168+0000", 2020-08-07T15:13:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050828199999999998 2020-08-07T15:13:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801676+0000", 2020-08-07T15:13:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1011e-05 2020-08-07T15:13:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801687+0000", 2020-08-07T15:13:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2750999999999993e-05 2020-08-07T15:13:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801780+0000", 2020-08-07T15:13:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39631765200000002 2020-08-07T15:13:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:85552 5.9 5:9d69b7cc:::benchmark_data_smithi074_53697_object10693:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e202)", 2020-08-07T15:13:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:13:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 116.955233264, 2020-08-07T15:13:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54429374399999997, 2020-08-07T15:13:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 85552 2020-08-07T15:13:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:13:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:13:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2691000000000001e-05 2020-08-07T15:13:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807570+0000", 2020-08-07T15:13:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0619999999999997e-06 2020-08-07T15:13:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807576+0000", 2020-08-07T15:13:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1558e-05 2020-08-07T15:13:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807588+0000", 2020-08-07T15:13:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4600000000000002e-06 2020-08-07T15:13:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807593+0000", 2020-08-07T15:13:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022126279999999999 2020-08-07T15:13:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.809806+0000", 2020-08-07T15:13:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104868 2020-08-07T15:13:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.809911+0000", 2020-08-07T15:13:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000186019 2020-08-07T15:13:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:13:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.810097+0000", 2020-08-07T15:13:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037966739999999999 2020-08-07T15:13:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813893+0000", 2020-08-07T15:13:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1310000000000001e-06 2020-08-07T15:13:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813894+0000", 2020-08-07T15:13:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4704999999999999e-05 2020-08-07T15:13:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813949+0000", 2020-08-07T15:13:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015487280000000001 2020-08-07T15:13:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.815498+0000", 2020-08-07T15:13:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53621442900000005 2020-08-07T15:13:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351712+0000", 2020-08-07T15:13:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8199999999999998e-07 2020-08-07T15:13:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351713+0000", 2020-08-07T15:13:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7909000000000001e-05 2020-08-07T15:13:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351811+0000", 2020-08-07T15:13:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54429374399999997 2020-08-07T15:13:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:85552 5.9 e202/164 ondisk, result = 0)", 2020-08-07T15:13:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:13:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 116.949006862, 2020-08-07T15:13:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53813836800000003, 2020-08-07T15:13:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:13:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:13:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8833e-05 2020-08-07T15:13:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813772+0000", 2020-08-07T15:13:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8459999999999999e-06 2020-08-07T15:13:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813777+0000", 2020-08-07T15:13:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53732224900000003 2020-08-07T15:13:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351100+0000", 2020-08-07T15:13:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1869999999999996e-06 2020-08-07T15:13:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351105+0000", 2020-08-07T15:13:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00058858100000000002 2020-08-07T15:13:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351693+0000", 2020-08-07T15:13:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4229999999999994e-06 2020-08-07T15:13:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351703+0000", 2020-08-07T15:13:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000179249 2020-08-07T15:13:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351882+0000", 2020-08-07T15:13:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53813836800000003 2020-08-07T15:13:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:85552 5.9 5:9d69b7cc:::benchmark_data_smithi074_53697_object10693:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e202)", 2020-08-07T15:13:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:13:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 116.944258738, 2020-08-07T15:13:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53326610600000002, 2020-08-07T15:13:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:13:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 85552 2020-08-07T15:13:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:13:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:13:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5001e-05 2020-08-07T15:13:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818517+0000", 2020-08-07T15:13:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5000000000000002e-06 2020-08-07T15:13:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818519+0000", 2020-08-07T15:13:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0699999999999997e-06 2020-08-07T15:13:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818524+0000", 2020-08-07T15:13:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2910000000000002e-06 2020-08-07T15:13:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818527+0000", 2020-08-07T15:13:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030912499999999999 2020-08-07T15:13:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818836+0000", 2020-08-07T15:13:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2908000000000002e-05 2020-08-07T15:13:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for ondisk", 2020-08-07T15:13:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818889+0000", 2020-08-07T15:13:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53286921099999995 2020-08-07T15:13:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351758+0000", 2020-08-07T15:13:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53326610600000002 2020-08-07T15:13:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:89464 5.9 5:94c69ba5:::benchmark_data_smithi074_53697_object11182:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:13:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 112.34641878399999, 2020-08-07T15:13:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67692916199999997, 2020-08-07T15:13:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 89464 2020-08-07T15:13:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:13:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:13:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0074000000000002e-05 2020-08-07T15:13:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416382+0000", 2020-08-07T15:13:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5000000000000001e-06 2020-08-07T15:13:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416386+0000", 2020-08-07T15:13:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2447e-05 2020-08-07T15:13:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416399+0000", 2020-08-07T15:13:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1279999999999996e-06 2020-08-07T15:13:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416405+0000", 2020-08-07T15:13:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016641805999999999 2020-08-07T15:13:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433047+0000", 2020-08-07T15:13:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107995 2020-08-07T15:13:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433155+0000", 2020-08-07T15:13:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018818599999999999 2020-08-07T15:13:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:13:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433343+0000", 2020-08-07T15:13:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039911679999999998 2020-08-07T15:13:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437334+0000", 2020-08-07T15:13:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2020-08-07T15:13:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437335+0000", 2020-08-07T15:13:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6351000000000002e-05 2020-08-07T15:13:14.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:14.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437401+0000", 2020-08-07T15:13:14.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001267949 2020-08-07T15:13:14.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:14.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438669+0000", 2020-08-07T15:13:14.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65454035600000005 2020-08-07T15:13:14.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:14.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093209+0000", 2020-08-07T15:13:14.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7859999999999999e-06 2020-08-07T15:13:14.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:14.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093211+0000", 2020-08-07T15:13:14.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9665e-05 2020-08-07T15:13:14.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:14.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093261+0000", 2020-08-07T15:13:14.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67692916199999997 2020-08-07T15:13:14.279 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.279 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:14.279 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.280 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:89464 5.9 e207/164 ondisk, result = 0)", 2020-08-07T15:13:14.280 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:13:14.281 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 112.323789022, 2020-08-07T15:13:14.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65430647200000003, 2020-08-07T15:13:14.281 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:14.281 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:14.281 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:14.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:14.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:13:14.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:14.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:14.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:13:14.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6529000000000002e-05 2020-08-07T15:13:14.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:14.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438998+0000", 2020-08-07T15:13:14.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.567e-06 2020-08-07T15:13:14.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:14.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.439005+0000", 2020-08-07T15:13:14.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65408475099999996 2020-08-07T15:13:14.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:14.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093089+0000", 2020-08-07T15:13:14.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6880000000000002e-06 2020-08-07T15:13:14.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:14.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093096+0000", 2020-08-07T15:13:14.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4980000000000002e-05 2020-08-07T15:13:14.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:14.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093191+0000", 2020-08-07T15:13:14.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3090000000000001e-06 2020-08-07T15:13:14.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:14.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093200+0000", 2020-08-07T15:13:14.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7648e-05 2020-08-07T15:13:14.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:14.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093268+0000", 2020-08-07T15:13:14.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65430647200000003 2020-08-07T15:13:14.313 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.313 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:14.313 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.316 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:114080 5.4 5:24adcd54:::benchmark_data_smithi074_53697_object14259:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e215)", 2020-08-07T15:13:14.317 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:13:14.317 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 94.163081564999999, 2020-08-07T15:13:14.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.57290862300000001, 2020-08-07T15:13:14.317 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:14.318 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:14.318 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:14.318 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:14.318 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:14.318 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 114080 2020-08-07T15:13:14.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.319 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:14.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:14.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:13:14.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:14.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:14.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:13:14.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9238e-05 2020-08-07T15:13:14.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:14.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599698+0000", 2020-08-07T15:13:14.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6260000000000003e-06 2020-08-07T15:13:14.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:14.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599703+0000", 2020-08-07T15:13:14.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5577999999999999e-05 2020-08-07T15:13:14.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:14.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599718+0000", 2020-08-07T15:13:14.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9869999999999999e-06 2020-08-07T15:13:14.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:14.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599726+0000", 2020-08-07T15:13:14.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00077929599999999996 2020-08-07T15:13:14.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:14.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600506+0000", 2020-08-07T15:13:14.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1717000000000003e-05 2020-08-07T15:13:14.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:14.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600597+0000", 2020-08-07T15:13:14.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011828999999999999 2020-08-07T15:13:14.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,5", 2020-08-07T15:13:14.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600716+0000", 2020-08-07T15:13:14.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020861479999999999 2020-08-07T15:13:14.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:14.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602802+0000", 2020-08-07T15:13:14.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4600000000000005e-07 2020-08-07T15:13:14.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:14.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602802+0000", 2020-08-07T15:13:14.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9165e-05 2020-08-07T15:13:14.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:14.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602862+0000", 2020-08-07T15:13:14.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0093485080000000002 2020-08-07T15:13:14.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:14.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.612210+0000", 2020-08-07T15:13:14.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.56030197699999995 2020-08-07T15:13:14.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:14.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172512+0000", 2020-08-07T15:13:14.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8800000000000001e-07 2020-08-07T15:13:14.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:14.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172513+0000", 2020-08-07T15:13:14.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4659000000000006e-05 2020-08-07T15:13:14.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:14.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172578+0000", 2020-08-07T15:13:14.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.57290862300000001 2020-08-07T15:13:14.339 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.340 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:14.340 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.340 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:48616 6.9 6:955383e0:::benchmark_data_smithi074_63843_object6076:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e256)", 2020-08-07T15:13:14.341 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:43.702845+0000", 2020-08-07T15:13:14.341 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 30.059905268000001, 2020-08-07T15:13:14.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40912302499999997, 2020-08-07T15:13:14.341 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:14.342 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:14.342 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:14.342 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:13:14.342 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:14.342 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48616 2020-08-07T15:13:14.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.343 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:14.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:14.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702845+0000", 2020-08-07T15:13:14.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:14.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:14.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702845+0000", 2020-08-07T15:13:14.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1931999999999997e-05 2020-08-07T15:13:14.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:14.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702897+0000", 2020-08-07T15:13:14.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0650000000000004e-06 2020-08-07T15:13:14.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:14.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702903+0000", 2020-08-07T15:13:14.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3084999999999999e-05 2020-08-07T15:13:14.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:14.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702916+0000", 2020-08-07T15:13:14.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4179999999999996e-06 2020-08-07T15:13:14.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:14.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702922+0000", 2020-08-07T15:13:14.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045509080000000002 2020-08-07T15:13:14.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:14.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.707473+0000", 2020-08-07T15:13:14.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2619000000000005e-05 2020-08-07T15:13:14.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:14.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.707535+0000", 2020-08-07T15:13:14.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000126919 2020-08-07T15:13:14.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2", 2020-08-07T15:13:14.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.707662+0000", 2020-08-07T15:13:14.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001200135 2020-08-07T15:13:14.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:14.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.708862+0000", 2020-08-07T15:13:14.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0109999999999999e-06 2020-08-07T15:13:14.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:14.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.708863+0000", 2020-08-07T15:13:14.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8837999999999994e-05 2020-08-07T15:13:14.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:14.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.708932+0000", 2020-08-07T15:13:14.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054206710000000002 2020-08-07T15:13:14.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:14.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.714353+0000", 2020-08-07T15:13:14.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39754767099999999 2020-08-07T15:13:14.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:14.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111901+0000", 2020-08-07T15:13:14.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.023e-06 2020-08-07T15:13:14.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:14.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111902+0000", 2020-08-07T15:13:14.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6730000000000007e-05 2020-08-07T15:13:14.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:14.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111968+0000", 2020-08-07T15:13:14.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40912302499999997 2020-08-07T15:13:14.360 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.361 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:14.361 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.361 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5481.0:48616 6.9 e256/230 ondisk, result = 0)", 2020-08-07T15:13:14.362 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:43.714790+0000", 2020-08-07T15:13:14.362 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 30.047960703000001, 2020-08-07T15:13:14.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39718457899999998, 2020-08-07T15:13:14.362 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:14.362 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:14.363 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:14.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:14.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.714790+0000", 2020-08-07T15:13:14.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:14.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:14.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.714790+0000", 2020-08-07T15:13:14.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8272000000000001e-05 2020-08-07T15:13:14.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:14.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.714808+0000", 2020-08-07T15:13:14.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1599999999999998e-06 2020-08-07T15:13:14.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:14.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.714811+0000", 2020-08-07T15:13:14.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39639543300000002 2020-08-07T15:13:14.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:14.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111207+0000", 2020-08-07T15:13:14.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7680000000000002e-06 2020-08-07T15:13:14.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:14.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111210+0000", 2020-08-07T15:13:14.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067142299999999999 2020-08-07T15:13:14.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:14.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111882+0000", 2020-08-07T15:13:14.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6849999999999996e-06 2020-08-07T15:13:14.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:14.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111892+0000", 2020-08-07T15:13:14.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2837999999999995e-05 2020-08-07T15:13:14.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:14.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:14.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:14.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111974+0000", 2020-08-07T15:13:14.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39718457899999998 2020-08-07T15:13:14.372 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.372 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:14.372 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.372 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:14.373 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:14.373 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:14.375 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 62 16 92086 92070 11.6003 10.0859 0.00830641 0.0107699 2020-08-07T15:13:14.375 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:13:14.385 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:14.381+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:14.489 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:14.481+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:14.578 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:14.576+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:14.589 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:14.590 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:13:14.590 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:13:14.590 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:14.603 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:13:14.679 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.5 to 1 (10000) 2020-08-07T15:13:14.679 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:14.678+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:14.703 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 6 1 2020-08-07T15:13:14.789 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:14.788+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:14.854 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:14.855 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:13:14.855 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:13:14.855 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:13:14.855 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:14.857 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:13:14.901 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:14.898+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:14.985 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:14.982+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:15.089 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:15.084+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:15.138 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:15.139 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:13:15.139 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:13:15.139 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:13:15.139 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.140 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "MOSDScrubReserve(2.6 GRANT e232)", 2020-08-07T15:13:15.140 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:11.909673+0000", 2020-08-07T15:13:15.140 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 63.204402795999997, 2020-08-07T15:13:15.140 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.36313456, 2020-08-07T15:13:15.140 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:15.141 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:15.141 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:15.141 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.141 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:15.156 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:11.909673+0000", 2020-08-07T15:13:15.156 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:15.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.157 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:15.157 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:11.909673+0000", 2020-08-07T15:13:15.157 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0409000000000001e-05 2020-08-07T15:13:15.157 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.157 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.158 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:15.158 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:11.909693+0000", 2020-08-07T15:13:15.158 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8580000000000002e-06 2020-08-07T15:13:15.158 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.158 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.159 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:15.159 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:11.909697+0000", 2020-08-07T15:13:15.159 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.36297494299999999 2020-08-07T15:13:15.159 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.160 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.160 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:15.160 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272672+0000", 2020-08-07T15:13:15.160 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4490000000000001e-06 2020-08-07T15:13:15.160 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.161 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.161 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:15.161 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272676+0000", 2020-08-07T15:13:15.161 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0395999999999994e-05 2020-08-07T15:13:15.181 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:15.182 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272757+0000", 2020-08-07T15:13:15.183 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6416e-05 2020-08-07T15:13:15.183 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.183 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.183 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:15.183 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272773+0000", 2020-08-07T15:13:15.184 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4088999999999998e-05 2020-08-07T15:13:15.184 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.184 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.184 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:15.184 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272807+0000", 2020-08-07T15:13:15.185 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.36313456 2020-08-07T15:13:15.185 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.185 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:15.185 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.186 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.186 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.186 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:7272 6.e 6:74263cb8:::benchmark_data_smithi074_63843_object908:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e234)", 2020-08-07T15:13:15.187 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:16.977784+0000", 2020-08-07T15:13:15.187 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 58.136291047999997, 2020-08-07T15:13:15.188 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31581518600000003, 2020-08-07T15:13:15.188 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:15.188 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:15.188 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:15.188 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:15.189 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:15.189 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7272 2020-08-07T15:13:15.189 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.189 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:15.189 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.190 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:15.190 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977784+0000", 2020-08-07T15:13:15.190 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:15.190 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.190 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.191 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:15.191 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977784+0000", 2020-08-07T15:13:15.191 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6940999999999998e-05 2020-08-07T15:13:15.193 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.193 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.194 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:15.194 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977811+0000", 2020-08-07T15:13:15.194 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1870000000000001e-06 2020-08-07T15:13:15.194 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.195 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.195 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:15.195 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977815+0000", 2020-08-07T15:13:15.195 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1249999999999998e-06 2020-08-07T15:13:15.195 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.196 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.196 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:15.196 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977821+0000", 2020-08-07T15:13:15.196 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8739999999999999e-06 2020-08-07T15:13:15.196 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.197 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.197 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:15.198 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977824+0000", 2020-08-07T15:13:15.198 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001740655 2020-08-07T15:13:15.198 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.198 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.199 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:15.199 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.979564+0000", 2020-08-07T15:13:15.199 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6272000000000003e-05 2020-08-07T15:13:15.199 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.199 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.200 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:15.200 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.979650+0000", 2020-08-07T15:13:15.200 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012951600000000001 2020-08-07T15:13:15.200 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.201 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.201 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:15.201 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.979780+0000", 2020-08-07T15:13:15.201 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00085408999999999995 2020-08-07T15:13:15.201 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.202 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.202 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:15.202 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.980634+0000", 2020-08-07T15:13:15.202 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8099999999999999e-07 2020-08-07T15:13:15.202 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.203 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.203 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:15.203 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.980634+0000", 2020-08-07T15:13:15.203 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5664000000000003e-05 2020-08-07T15:13:15.203 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.204 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.204 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:15.204 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.980670+0000", 2020-08-07T15:13:15.204 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.104650663 2020-08-07T15:13:15.204 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.205 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.205 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.205 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.085321+0000", 2020-08-07T15:13:15.205 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20821616000000001 2020-08-07T15:13:15.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.206 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.206 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.207 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.293537+0000", 2020-08-07T15:13:15.207 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1900000000000001e-07 2020-08-07T15:13:15.207 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.207 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.207 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:15.208 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.293538+0000", 2020-08-07T15:13:15.208 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1939000000000004e-05 2020-08-07T15:13:15.209 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.209 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.209 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:15.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.293600+0000", 2020-08-07T15:13:15.209 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31581518600000003 2020-08-07T15:13:15.210 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.210 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:15.210 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.210 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.210 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.211 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:8359 6.1 6:844c6d67:::benchmark_data_smithi074_63843_object1044:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e235)", 2020-08-07T15:13:15.211 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:17.720675+0000", 2020-08-07T15:13:15.211 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.393400890999999, 2020-08-07T15:13:15.211 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.309231859, 2020-08-07T15:13:15.211 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:15.212 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:15.212 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:15.212 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:15.212 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:15.212 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 8359 2020-08-07T15:13:15.213 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.213 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:15.213 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.213 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:15.213 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720675+0000", 2020-08-07T15:13:15.214 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:15.214 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.214 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.214 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:15.214 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720675+0000", 2020-08-07T15:13:15.215 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.884e-05 2020-08-07T15:13:15.215 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.215 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.215 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:15.215 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720723+0000", 2020-08-07T15:13:15.216 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2989999999999996e-06 2020-08-07T15:13:15.216 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.216 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.220 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:15.220 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720729+0000", 2020-08-07T15:13:15.221 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0288999999999999e-05 2020-08-07T15:13:15.221 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.221 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.221 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:15.222 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720739+0000", 2020-08-07T15:13:15.222 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7849999999999999e-06 2020-08-07T15:13:15.222 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.222 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.222 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:15.223 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720744+0000", 2020-08-07T15:13:15.223 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001862991 2020-08-07T15:13:15.224 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.224 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.225 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:15.225 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.722607+0000", 2020-08-07T15:13:15.225 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000100839 2020-08-07T15:13:15.225 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.225 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.226 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:15.227 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.722708+0000", 2020-08-07T15:13:15.227 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017526600000000001 2020-08-07T15:13:15.227 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.227 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.228 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:13:15.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.722883+0000", 2020-08-07T15:13:15.228 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001504619 2020-08-07T15:13:15.228 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.228 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.229 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:15.229 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.724387+0000", 2020-08-07T15:13:15.230 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4499999999999996e-07 2020-08-07T15:13:15.230 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.232 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.232 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:15.233 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.724388+0000", 2020-08-07T15:13:15.233 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8369000000000003e-05 2020-08-07T15:13:15.233 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.233 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.234 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:15.234 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.724447+0000", 2020-08-07T15:13:15.234 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00097551199999999999 2020-08-07T15:13:15.235 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.235 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.235 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.235 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.725422+0000", 2020-08-07T15:13:15.235 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30441636900000002 2020-08-07T15:13:15.236 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.236 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.236 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.236 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029838+0000", 2020-08-07T15:13:15.236 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7699999999999992e-07 2020-08-07T15:13:15.237 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.237 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.237 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:15.237 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029839+0000", 2020-08-07T15:13:15.237 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6958999999999994e-05 2020-08-07T15:13:15.238 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.238 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.238 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:15.238 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029906+0000", 2020-08-07T15:13:15.239 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.309231859 2020-08-07T15:13:15.239 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.239 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:15.239 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.239 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.240 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.240 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:8360 6.1 6:844c6d67:::benchmark_data_smithi074_63843_object1044:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e235)", 2020-08-07T15:13:15.240 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:17.721396+0000", 2020-08-07T15:13:15.240 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.392679227999999, 2020-08-07T15:13:15.240 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31055011399999999, 2020-08-07T15:13:15.241 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:15.241 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:15.241 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:15.241 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:15.241 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:15.242 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 8360 2020-08-07T15:13:15.242 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.242 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:15.242 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.243 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:15.243 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721396+0000", 2020-08-07T15:13:15.243 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:15.243 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.243 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.244 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:15.244 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721396+0000", 2020-08-07T15:13:15.244 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8222e-05 2020-08-07T15:13:15.244 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.244 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.245 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:15.245 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721444+0000", 2020-08-07T15:13:15.245 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.297e-06 2020-08-07T15:13:15.245 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.245 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.247 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:15.248 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721451+0000", 2020-08-07T15:13:15.248 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.172e-05 2020-08-07T15:13:15.248 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.248 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.249 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:15.249 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721462+0000", 2020-08-07T15:13:15.249 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7639999999999996e-06 2020-08-07T15:13:15.249 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.249 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.250 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:15.250 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721467+0000", 2020-08-07T15:13:15.250 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023051790000000001 2020-08-07T15:13:15.250 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.250 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.251 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:15.251 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.723772+0000", 2020-08-07T15:13:15.251 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1051000000000006e-05 2020-08-07T15:13:15.251 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.251 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.252 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:15.252 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.723863+0000", 2020-08-07T15:13:15.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017915699999999999 2020-08-07T15:13:15.252 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.252 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.253 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:13:15.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.724043+0000", 2020-08-07T15:13:15.253 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001191646 2020-08-07T15:13:15.253 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.254 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:15.254 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.725234+0000", 2020-08-07T15:13:15.254 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9100000000000002e-07 2020-08-07T15:13:15.254 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.255 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.255 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:15.255 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.725235+0000", 2020-08-07T15:13:15.255 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0677999999999997e-05 2020-08-07T15:13:15.255 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.256 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.256 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:15.256 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.725296+0000", 2020-08-07T15:13:15.256 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001252663 2020-08-07T15:13:15.256 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.257 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.257 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.257 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.726548+0000", 2020-08-07T15:13:15.257 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30531503100000001 2020-08-07T15:13:15.257 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.258 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.258 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.258 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.031864+0000", 2020-08-07T15:13:15.258 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.057e-06 2020-08-07T15:13:15.258 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.259 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.259 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:15.259 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.031865+0000", 2020-08-07T15:13:15.259 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1757999999999999e-05 2020-08-07T15:13:15.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.260 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.260 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:15.260 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.031946+0000", 2020-08-07T15:13:15.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31055011399999999 2020-08-07T15:13:15.260 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.261 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:15.261 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.261 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.261 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.262 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:8359 6.1 e235/230 ondisk, result = 0)", 2020-08-07T15:13:15.262 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:17.726764+0000", 2020-08-07T15:13:15.263 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 57.387311212, 2020-08-07T15:13:15.263 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30314883199999998, 2020-08-07T15:13:15.263 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:15.263 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:15.263 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:15.264 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.264 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:15.264 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.726764+0000", 2020-08-07T15:13:15.264 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:15.265 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.265 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.265 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:15.265 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.726764+0000", 2020-08-07T15:13:15.265 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1386000000000001e-05 2020-08-07T15:13:15.266 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.266 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.266 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:15.280 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.726796+0000", 2020-08-07T15:13:15.280 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7989999999999999e-06 2020-08-07T15:13:15.280 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.281 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.281 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:15.281 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.726801+0000", 2020-08-07T15:13:15.281 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30298438700000002 2020-08-07T15:13:15.282 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.282 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.282 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:15.283 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029786+0000", 2020-08-07T15:13:15.283 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5040000000000002e-06 2020-08-07T15:13:15.283 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.283 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.283 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:15.284 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029791+0000", 2020-08-07T15:13:15.284 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0737000000000001e-05 2020-08-07T15:13:15.284 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.284 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.284 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:15.285 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029822+0000", 2020-08-07T15:13:15.285 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9359999999999999e-06 2020-08-07T15:13:15.286 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.286 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.286 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:15.286 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029830+0000", 2020-08-07T15:13:15.287 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3083000000000004e-05 2020-08-07T15:13:15.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.287 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:15.287 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029913+0000", 2020-08-07T15:13:15.288 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30314883199999998 2020-08-07T15:13:15.288 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.288 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:15.288 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.288 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.288 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.289 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:10224 6.e 6:77f719e6:::benchmark_data_smithi074_63843_object1277:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e236)", 2020-08-07T15:13:15.289 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:19.038704+0000", 2020-08-07T15:13:15.319 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 56.075371910000001, 2020-08-07T15:13:15.320 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32955690900000001, 2020-08-07T15:13:15.320 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:15.320 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:15.320 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:15.320 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:15.327 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:15.327 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 10224 2020-08-07T15:13:15.328 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.328 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:15.329 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.329 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:15.329 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038704+0000", 2020-08-07T15:13:15.329 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:15.329 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.330 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.330 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:15.330 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038704+0000", 2020-08-07T15:13:15.330 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6672000000000001e-05 2020-08-07T15:13:15.331 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.331 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.331 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:15.331 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038730+0000", 2020-08-07T15:13:15.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.27e-06 2020-08-07T15:13:15.339 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.339 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.339 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:15.340 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038733+0000", 2020-08-07T15:13:15.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4030000000000004e-06 2020-08-07T15:13:15.340 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.340 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:15.341 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038740+0000", 2020-08-07T15:13:15.341 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.712e-06 2020-08-07T15:13:15.341 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.342 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:15.342 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038743+0000", 2020-08-07T15:13:15.342 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0025767279999999999 2020-08-07T15:13:15.342 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.342 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.343 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:15.343 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.041319+0000", 2020-08-07T15:13:15.343 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4727e-05 2020-08-07T15:13:15.343 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.343 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.344 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:15.344 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.041374+0000", 2020-08-07T15:13:15.344 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9403999999999999e-05 2020-08-07T15:13:15.344 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.344 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.345 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:15.345 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.041473+0000", 2020-08-07T15:13:15.345 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010041290000000001 2020-08-07T15:13:15.345 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.345 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.361 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:15.361 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.042478+0000", 2020-08-07T15:13:15.361 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0100000000000003e-07 2020-08-07T15:13:15.362 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.362 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.362 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:15.362 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.042478+0000", 2020-08-07T15:13:15.362 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7923999999999995e-05 2020-08-07T15:13:15.363 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.363 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.363 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:15.363 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.042546+0000", 2020-08-07T15:13:15.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0046169310000000003 2020-08-07T15:13:15.364 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.364 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.047163+0000", 2020-08-07T15:13:15.364 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.321023999 2020-08-07T15:13:15.365 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.365 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.365 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.365 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368187+0000", 2020-08-07T15:13:15.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.048e-06 2020-08-07T15:13:15.366 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.367 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:15.367 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368188+0000", 2020-08-07T15:13:15.367 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2061e-05 2020-08-07T15:13:15.367 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.367 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.368 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:15.368 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368260+0000", 2020-08-07T15:13:15.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32955690900000001 2020-08-07T15:13:15.368 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.368 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:15.368 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.369 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:10224 6.e e236/230 ondisk, result = 0)", 2020-08-07T15:13:15.369 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:19.047929+0000", 2020-08-07T15:13:15.371 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 56.066146310000001, 2020-08-07T15:13:15.371 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32041415400000001, 2020-08-07T15:13:15.372 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:15.372 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:15.372 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:15.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.372 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:15.373 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.047929+0000", 2020-08-07T15:13:15.373 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:15.373 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.373 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.374 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:15.374 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.047929+0000", 2020-08-07T15:13:15.374 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9706999999999999e-05 2020-08-07T15:13:15.374 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.374 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.375 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:15.375 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.047959+0000", 2020-08-07T15:13:15.375 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9100000000000004e-06 2020-08-07T15:13:15.375 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.376 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:15.376 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.047964+0000", 2020-08-07T15:13:15.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31873185399999998 2020-08-07T15:13:15.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:15.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.366696+0000", 2020-08-07T15:13:15.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2970000000000003e-06 2020-08-07T15:13:15.385 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.385 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.385 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:15.385 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.366701+0000", 2020-08-07T15:13:15.385 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001472522 2020-08-07T15:13:15.386 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.386 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.386 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:15.386 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368173+0000", 2020-08-07T15:13:15.386 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5949999999999998e-06 2020-08-07T15:13:15.387 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.387 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.387 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:15.387 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368179+0000", 2020-08-07T15:13:15.387 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016426900000000001 2020-08-07T15:13:15.388 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.388 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.388 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:15.388 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368343+0000", 2020-08-07T15:13:15.388 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32041415400000001 2020-08-07T15:13:15.389 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.389 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:15.389 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.389 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.389 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.402 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:18480 6.1 6:8a2c1569:::benchmark_data_smithi074_63843_object2309:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e240)", 2020-08-07T15:13:15.402 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.367850+0000", 2020-08-07T15:13:15.403 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 50.746225899999999, 2020-08-07T15:13:15.403 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41222172800000001, 2020-08-07T15:13:15.403 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:15.404 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:15.404 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:15.404 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:15.404 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:15.404 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18480 2020-08-07T15:13:15.405 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.405 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:15.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.405 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:15.405 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367850+0000", 2020-08-07T15:13:15.406 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:15.406 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.406 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.406 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:15.406 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367850+0000", 2020-08-07T15:13:15.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6174999999999999e-05 2020-08-07T15:13:15.407 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.407 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.407 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:15.407 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367876+0000", 2020-08-07T15:13:15.408 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.061e-06 2020-08-07T15:13:15.408 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.408 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.408 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:15.408 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367879+0000", 2020-08-07T15:13:15.409 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7899999999999996e-06 2020-08-07T15:13:15.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.409 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.409 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:15.409 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367885+0000", 2020-08-07T15:13:15.418 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6589999999999999e-06 2020-08-07T15:13:15.419 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.419 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.419 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:15.420 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367887+0000", 2020-08-07T15:13:15.420 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001651275 2020-08-07T15:13:15.420 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.420 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.421 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:15.421 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.369538+0000", 2020-08-07T15:13:15.421 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7794000000000006e-05 2020-08-07T15:13:15.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.421 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.422 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:15.422 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.369636+0000", 2020-08-07T15:13:15.422 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017274 2020-08-07T15:13:15.422 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.423 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.423 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:13:15.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.369809+0000", 2020-08-07T15:13:15.423 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010733769999999999 2020-08-07T15:13:15.430 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.430 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.430 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:15.430 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.370882+0000", 2020-08-07T15:13:15.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0100000000000003e-07 2020-08-07T15:13:15.431 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.431 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:15.432 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.370883+0000", 2020-08-07T15:13:15.432 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6001000000000001e-05 2020-08-07T15:13:15.432 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.432 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.433 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:15.433 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.370939+0000", 2020-08-07T15:13:15.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074658490000000001 2020-08-07T15:13:15.433 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.433 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.434 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.434 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.378405+0000", 2020-08-07T15:13:15.435 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40160148899999998 2020-08-07T15:13:15.435 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.435 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.435 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.435 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.780007+0000", 2020-08-07T15:13:15.436 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9700000000000005e-07 2020-08-07T15:13:15.436 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.442 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.442 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:15.443 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.780008+0000", 2020-08-07T15:13:15.443 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3720000000000007e-05 2020-08-07T15:13:15.443 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.443 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.443 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:15.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.780071+0000", 2020-08-07T15:13:15.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41222172800000001 2020-08-07T15:13:15.444 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.444 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:15.444 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.455 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:18480 6.1 e240/230 ondisk, result = 0)", 2020-08-07T15:13:15.465 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.375696+0000", 2020-08-07T15:13:15.466 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 50.738379297000002, 2020-08-07T15:13:15.467 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.404380817, 2020-08-07T15:13:15.467 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:15.467 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:15.467 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:15.467 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.468 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:15.468 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.375696+0000", 2020-08-07T15:13:15.468 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:15.468 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.468 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.469 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:15.469 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.375696+0000", 2020-08-07T15:13:15.469 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0922e-05 2020-08-07T15:13:15.469 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.469 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.470 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:15.470 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.375727+0000", 2020-08-07T15:13:15.470 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2900000000000002e-06 2020-08-07T15:13:15.470 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.470 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.471 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:15.471 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.375732+0000", 2020-08-07T15:13:15.471 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.403431708 2020-08-07T15:13:15.471 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.471 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.472 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:15.472 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.779164+0000", 2020-08-07T15:13:15.472 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5200000000000002e-06 2020-08-07T15:13:15.472 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.472 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.473 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:15.473 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.779168+0000", 2020-08-07T15:13:15.473 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00082257200000000004 2020-08-07T15:13:15.473 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.473 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:15.488 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.779990+0000", 2020-08-07T15:13:15.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1599999999999998e-06 2020-08-07T15:13:15.488 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.489 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.489 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:15.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.779998+0000", 2020-08-07T15:13:15.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8645000000000003e-05 2020-08-07T15:13:15.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.490 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.490 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:15.490 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.780077+0000", 2020-08-07T15:13:15.490 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.404380817 2020-08-07T15:13:15.490 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.491 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:15.491 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.491 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.491 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:18560 6.e 6:7c5f8c5e:::benchmark_data_smithi074_63843_object2319:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e240)", 2020-08-07T15:13:15.492 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.407761+0000", 2020-08-07T15:13:15.492 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 50.706314034000002, 2020-08-07T15:13:15.492 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31539320100000001, 2020-08-07T15:13:15.495 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:15.495 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:15.495 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:15.496 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:15.496 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:15.496 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18560 2020-08-07T15:13:15.496 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.496 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:15.496 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.497 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:15.497 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407761+0000", 2020-08-07T15:13:15.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:15.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.512 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.513 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:15.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407761+0000", 2020-08-07T15:13:15.513 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0897000000000003e-05 2020-08-07T15:13:15.513 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.513 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.517 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:15.517 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407822+0000", 2020-08-07T15:13:15.517 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8390000000000001e-06 2020-08-07T15:13:15.517 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.518 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:15.518 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407829+0000", 2020-08-07T15:13:15.518 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3609e-05 2020-08-07T15:13:15.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.519 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.519 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:15.519 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407843+0000", 2020-08-07T15:13:15.519 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3169999999999998e-06 2020-08-07T15:13:15.520 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.520 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:15.520 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407849+0000", 2020-08-07T15:13:15.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002504886 2020-08-07T15:13:15.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.521 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.521 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:15.521 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.410354+0000", 2020-08-07T15:13:15.522 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7235e-05 2020-08-07T15:13:15.523 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.523 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.524 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:15.524 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.410451+0000", 2020-08-07T15:13:15.524 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016856999999999999 2020-08-07T15:13:15.524 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.524 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.525 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:15.525 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.410620+0000", 2020-08-07T15:13:15.525 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026816259999999999 2020-08-07T15:13:15.525 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.526 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.526 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.526 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.413301+0000", 2020-08-07T15:13:15.526 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0020516309999999999 2020-08-07T15:13:15.526 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.527 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.527 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:15.527 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.415353+0000", 2020-08-07T15:13:15.527 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30769196399999998 2020-08-07T15:13:15.527 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.527 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.528 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.528 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.723045+0000", 2020-08-07T15:13:15.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1600000000000001e-07 2020-08-07T15:13:15.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.529 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:15.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.723046+0000", 2020-08-07T15:13:15.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1449000000000001e-05 2020-08-07T15:13:15.529 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.529 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.530 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:15.530 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.723107+0000", 2020-08-07T15:13:15.530 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7462000000000001e-05 2020-08-07T15:13:15.530 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.530 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.531 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:15.531 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.723155+0000", 2020-08-07T15:13:15.531 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31539320100000001 2020-08-07T15:13:15.531 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.531 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:15.532 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.532 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.532 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.532 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:18608 6.4 6:2dddf5bf:::benchmark_data_smithi074_63843_object2325:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e240)", 2020-08-07T15:13:15.533 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.438031+0000", 2020-08-07T15:13:15.533 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 50.676044638, 2020-08-07T15:13:15.534 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28378579100000001, 2020-08-07T15:13:15.534 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:15.534 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:15.534 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:15.534 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:15.535 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:15.535 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18608 2020-08-07T15:13:15.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.535 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:15.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:15.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438031+0000", 2020-08-07T15:13:15.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:15.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.536 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.537 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:15.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438031+0000", 2020-08-07T15:13:15.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0028999999999998e-05 2020-08-07T15:13:15.537 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.537 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.538 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:15.538 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438081+0000", 2020-08-07T15:13:15.539 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7289999999999997e-06 2020-08-07T15:13:15.539 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.539 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.539 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:15.539 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438087+0000", 2020-08-07T15:13:15.540 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0959e-05 2020-08-07T15:13:15.540 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.542 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:15.542 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438098+0000", 2020-08-07T15:13:15.542 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0329999999999999e-06 2020-08-07T15:13:15.542 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.542 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.543 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:15.544 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438103+0000", 2020-08-07T15:13:15.544 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001770911 2020-08-07T15:13:15.544 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.544 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:15.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.439873+0000", 2020-08-07T15:13:15.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3503000000000005e-05 2020-08-07T15:13:15.545 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.545 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:15.546 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.439937+0000", 2020-08-07T15:13:15.546 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000110179 2020-08-07T15:13:15.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.546 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.546 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,5", 2020-08-07T15:13:15.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.440047+0000", 2020-08-07T15:13:15.547 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033910759999999998 2020-08-07T15:13:15.547 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.547 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.548 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443438+0000", 2020-08-07T15:13:15.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2877999999999999e-05 2020-08-07T15:13:15.548 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.551 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.551 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:15.552 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443471+0000", 2020-08-07T15:13:15.552 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27823471799999999 2020-08-07T15:13:15.552 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.553 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.553 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.553 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721706+0000", 2020-08-07T15:13:15.553 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4300000000000003e-07 2020-08-07T15:13:15.554 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.554 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.555 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:15.555 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721706+0000", 2020-08-07T15:13:15.555 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2877000000000001e-05 2020-08-07T15:13:15.555 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.555 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:15.556 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721769+0000", 2020-08-07T15:13:15.556 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7355999999999999e-05 2020-08-07T15:13:15.556 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.557 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:15.557 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721817+0000", 2020-08-07T15:13:15.557 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28378579100000001 2020-08-07T15:13:15.557 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.557 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:15.558 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.558 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.558 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.558 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:18608 6.4 e240/230 ondisk, result = 0)", 2020-08-07T15:13:15.558 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.443879+0000", 2020-08-07T15:13:15.559 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 50.670196400000002, 2020-08-07T15:13:15.559 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.277941838, 2020-08-07T15:13:15.559 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:15.559 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:15.559 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:15.559 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.560 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:15.560 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443879+0000", 2020-08-07T15:13:15.560 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:15.560 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.560 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.579 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:15.579 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443879+0000", 2020-08-07T15:13:15.579 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3692000000000002e-05 2020-08-07T15:13:15.579 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.579 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.580 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:15.580 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443913+0000", 2020-08-07T15:13:15.580 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.02e-06 2020-08-07T15:13:15.580 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.580 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.581 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:15.581 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443919+0000", 2020-08-07T15:13:15.581 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27734076499999999 2020-08-07T15:13:15.587 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.587 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.588 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:15.588 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721260+0000", 2020-08-07T15:13:15.588 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.78e-06 2020-08-07T15:13:15.588 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.589 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:15.589 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721264+0000", 2020-08-07T15:13:15.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00042663999999999999 2020-08-07T15:13:15.589 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.590 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.590 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:15.590 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721691+0000", 2020-08-07T15:13:15.590 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1509999999999998e-06 2020-08-07T15:13:15.595 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.595 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.595 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:15.595 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721698+0000", 2020-08-07T15:13:15.596 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012279000000000001 2020-08-07T15:13:15.596 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.596 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.596 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:15.596 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721821+0000", 2020-08-07T15:13:15.597 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.277941838 2020-08-07T15:13:15.597 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.597 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:15.597 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.598 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.598 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.598 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:29256 6.1 6:8599f289:::benchmark_data_smithi074_63843_object3656:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e244)", 2020-08-07T15:13:15.598 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:31.539560+0000", 2020-08-07T15:13:15.598 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 43.574515896999998, 2020-08-07T15:13:15.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28482166599999997, 2020-08-07T15:13:15.599 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:15.599 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:15.600 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:15.600 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:15.601 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:15.601 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 29256 2020-08-07T15:13:15.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.601 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:15.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.602 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:15.602 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.539560+0000", 2020-08-07T15:13:15.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:15.602 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.602 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.603 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:15.603 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.539560+0000", 2020-08-07T15:13:15.603 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5965e-05 2020-08-07T15:13:15.603 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.606 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.606 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:15.606 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.539586+0000", 2020-08-07T15:13:15.606 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1719999999999999e-06 2020-08-07T15:13:15.606 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.607 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.607 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:15.607 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.539589+0000", 2020-08-07T15:13:15.607 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9700000000000005e-06 2020-08-07T15:13:15.607 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.608 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.608 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:15.608 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.539598+0000", 2020-08-07T15:13:15.608 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9999999999999998e-06 2020-08-07T15:13:15.608 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.609 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.609 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:15.609 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.539602+0000", 2020-08-07T15:13:15.609 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011500200000000001 2020-08-07T15:13:15.609 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.611 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.611 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:15.612 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.540752+0000", 2020-08-07T15:13:15.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000107188 2020-08-07T15:13:15.612 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.612 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.612 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:15.613 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.540859+0000", 2020-08-07T15:13:15.613 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011065600000000001 2020-08-07T15:13:15.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.613 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:13:15.614 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.540970+0000", 2020-08-07T15:13:15.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0018553440000000001 2020-08-07T15:13:15.614 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.614 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.616 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:15.616 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.542825+0000", 2020-08-07T15:13:15.616 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.048e-06 2020-08-07T15:13:15.616 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.617 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.617 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:15.617 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.542826+0000", 2020-08-07T15:13:15.617 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2224000000000005e-05 2020-08-07T15:13:15.617 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.618 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.618 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:15.618 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.542898+0000", 2020-08-07T15:13:15.618 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.029129893 2020-08-07T15:13:15.618 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.619 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.619 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.619 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.572028+0000", 2020-08-07T15:13:15.619 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.252307642 2020-08-07T15:13:15.619 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.620 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.620 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.620 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.824336+0000", 2020-08-07T15:13:15.620 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9899999999999999e-07 2020-08-07T15:13:15.620 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.621 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.621 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:15.621 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.824337+0000", 2020-08-07T15:13:15.621 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4644999999999997e-05 2020-08-07T15:13:15.621 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.622 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.622 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:15.622 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.824381+0000", 2020-08-07T15:13:15.622 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28482166599999997 2020-08-07T15:13:15.622 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.622 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:15.623 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.623 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.623 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.623 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:32560 6.e 6:77482cb7:::benchmark_data_smithi074_63843_object4069:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e246)", 2020-08-07T15:13:15.623 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:33.538197+0000", 2020-08-07T15:13:15.624 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 41.575878623000001, 2020-08-07T15:13:15.624 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28301590900000001, 2020-08-07T15:13:15.624 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:15.624 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:15.625 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:15.625 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:15.625 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:15.625 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 32560 2020-08-07T15:13:15.625 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.626 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:15.626 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.626 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:15.626 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.538197+0000", 2020-08-07T15:13:15.626 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:15.627 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.628 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:15.629 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.538197+0000", 2020-08-07T15:13:15.629 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6372000000000003e-05 2020-08-07T15:13:15.629 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.630 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.630 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:15.631 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.538243+0000", 2020-08-07T15:13:15.631 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9100000000000004e-06 2020-08-07T15:13:15.631 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.631 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.631 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:15.632 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.538248+0000", 2020-08-07T15:13:15.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0688e-05 2020-08-07T15:13:15.632 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.632 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.632 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:15.633 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.538259+0000", 2020-08-07T15:13:15.633 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0649999999999998e-06 2020-08-07T15:13:15.633 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.633 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.633 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:15.634 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.538264+0000", 2020-08-07T15:13:15.634 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003496645 2020-08-07T15:13:15.634 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.634 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.634 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:15.635 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.541760+0000", 2020-08-07T15:13:15.635 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019299399999999999 2020-08-07T15:13:15.635 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.635 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.636 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:15.636 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.541953+0000", 2020-08-07T15:13:15.636 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018635200000000001 2020-08-07T15:13:15.636 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.636 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.637 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:15.637 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.542140+0000", 2020-08-07T15:13:15.637 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001795608 2020-08-07T15:13:15.637 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.637 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.638 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:15.638 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.543935+0000", 2020-08-07T15:13:15.638 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2399999999999998e-07 2020-08-07T15:13:15.638 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.638 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.639 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:15.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.543936+0000", 2020-08-07T15:13:15.642 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7708999999999999e-05 2020-08-07T15:13:15.642 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.642 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.643 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:15.643 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.543994+0000", 2020-08-07T15:13:15.643 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032640360000000001 2020-08-07T15:13:15.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.643 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.644 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.644 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.547258+0000", 2020-08-07T15:13:15.644 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27388560899999997 2020-08-07T15:13:15.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.646 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.821143+0000", 2020-08-07T15:13:15.646 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0410000000000001e-06 2020-08-07T15:13:15.647 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.647 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.647 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:15.647 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.821144+0000", 2020-08-07T15:13:15.647 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8355999999999994e-05 2020-08-07T15:13:15.648 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.648 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.648 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:15.648 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.821213+0000", 2020-08-07T15:13:15.648 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28301590900000001 2020-08-07T15:13:15.649 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.649 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:15.649 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.650 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.650 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.650 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:48584 6.e 6:72419a6c:::benchmark_data_smithi074_63843_object6072:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e256)", 2020-08-07T15:13:15.650 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:43.685874+0000", 2020-08-07T15:13:15.651 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 31.428201605999998, 2020-08-07T15:13:15.651 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.336964296, 2020-08-07T15:13:15.651 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:15.651 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:15.651 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:15.652 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:15.652 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:15.652 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48584 2020-08-07T15:13:15.653 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.653 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:15.653 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.654 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:15.654 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.685874+0000", 2020-08-07T15:13:15.654 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:15.654 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.654 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.655 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:15.655 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.685874+0000", 2020-08-07T15:13:15.655 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9709000000000006e-05 2020-08-07T15:13:15.655 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.655 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.656 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:15.656 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.685944+0000", 2020-08-07T15:13:15.656 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7719999999999994e-06 2020-08-07T15:13:15.656 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.656 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.657 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:15.657 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.685951+0000", 2020-08-07T15:13:15.657 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5424e-05 2020-08-07T15:13:15.657 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.657 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.658 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:15.658 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.685967+0000", 2020-08-07T15:13:15.658 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3549999999999999e-06 2020-08-07T15:13:15.658 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.658 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.659 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:15.659 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.685974+0000", 2020-08-07T15:13:15.659 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2768000000000001e-05 2020-08-07T15:13:15.659 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.659 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.660 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:15.660 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.686017+0000", 2020-08-07T15:13:15.660 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6916999999999994e-05 2020-08-07T15:13:15.660 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.660 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.661 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:15.661 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.686114+0000", 2020-08-07T15:13:15.662 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018684600000000001 2020-08-07T15:13:15.662 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.662 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.662 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:15.663 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.686301+0000", 2020-08-07T15:13:15.663 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0050644929999999998 2020-08-07T15:13:15.665 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.665 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.665 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:15.665 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.691365+0000", 2020-08-07T15:13:15.666 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.142e-06 2020-08-07T15:13:15.666 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.666 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.666 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:15.666 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.691366+0000", 2020-08-07T15:13:15.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2626999999999994e-05 2020-08-07T15:13:15.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.667 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:15.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.691439+0000", 2020-08-07T15:13:15.668 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0093850860000000008 2020-08-07T15:13:15.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.700824+0000", 2020-08-07T15:13:15.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32193047299999999 2020-08-07T15:13:15.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.670 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022754+0000", 2020-08-07T15:13:15.670 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.147e-06 2020-08-07T15:13:15.670 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.671 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.671 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:15.671 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022756+0000", 2020-08-07T15:13:15.671 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2536999999999999e-05 2020-08-07T15:13:15.671 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.672 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.673 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:15.673 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022838+0000", 2020-08-07T15:13:15.673 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.336964296 2020-08-07T15:13:15.673 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.673 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:15.674 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.674 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.674 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.674 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:48584 6.e e256/230 ondisk, result = 0)", 2020-08-07T15:13:15.682 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:43.688495+0000", 2020-08-07T15:13:15.682 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 31.425579961, 2020-08-07T15:13:15.682 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33435100299999998, 2020-08-07T15:13:15.682 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:15.683 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:15.683 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:15.683 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.683 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:15.684 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.688495+0000", 2020-08-07T15:13:15.685 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:15.685 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.685 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.686 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:15.686 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.688495+0000", 2020-08-07T15:13:15.686 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2947999999999997e-05 2020-08-07T15:13:15.687 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.687 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.687 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:15.687 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.688528+0000", 2020-08-07T15:13:15.687 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4979999999999997e-06 2020-08-07T15:13:15.688 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.688 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.688 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:15.688 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.688534+0000", 2020-08-07T15:13:15.688 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33414495300000002 2020-08-07T15:13:15.689 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.689 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.689 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:15.689 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022679+0000", 2020-08-07T15:13:15.689 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6669999999999999e-06 2020-08-07T15:13:15.690 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.690 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.690 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:15.690 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022686+0000", 2020-08-07T15:13:15.690 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1094999999999998e-05 2020-08-07T15:13:15.690 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.691 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.691 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:15.691 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022737+0000", 2020-08-07T15:13:15.691 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2629999999999992e-06 2020-08-07T15:13:15.691 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.692 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:15.692 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022746+0000", 2020-08-07T15:13:15.692 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000100579 2020-08-07T15:13:15.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.693 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.693 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:15.693 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022846+0000", 2020-08-07T15:13:15.693 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33435100299999998 2020-08-07T15:13:15.693 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.698 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:15.698 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.698 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.710 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:61896 6.e 6:75870b8d:::benchmark_data_smithi074_63843_object7736:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e263)", 2020-08-07T15:13:15.711 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:52.341370+0000", 2020-08-07T15:13:15.711 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 22.772705816999999, 2020-08-07T15:13:15.711 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.281931072, 2020-08-07T15:13:15.711 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:15.712 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:15.712 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:15.712 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:15.712 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:15.712 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 61896 2020-08-07T15:13:15.713 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.713 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:15.713 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.713 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:15.713 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.341370+0000", 2020-08-07T15:13:15.714 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:15.714 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.714 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.714 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:15.714 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.341370+0000", 2020-08-07T15:13:15.715 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7135000000000003e-05 2020-08-07T15:13:15.715 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.715 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.715 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:15.715 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.341407+0000", 2020-08-07T15:13:15.715 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0359999999999997e-06 2020-08-07T15:13:15.716 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.716 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.716 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:15.716 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.341412+0000", 2020-08-07T15:13:15.716 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0572e-05 2020-08-07T15:13:15.717 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.717 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.717 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:15.717 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.341422+0000", 2020-08-07T15:13:15.717 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.279e-06 2020-08-07T15:13:15.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.718 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.718 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:15.718 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.341427+0000", 2020-08-07T15:13:15.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024648579999999999 2020-08-07T15:13:15.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:15.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.343891+0000", 2020-08-07T15:13:15.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1310000000000005e-05 2020-08-07T15:13:15.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:15.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.343983+0000", 2020-08-07T15:13:15.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000175744 2020-08-07T15:13:15.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.721 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:15.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.344159+0000", 2020-08-07T15:13:15.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005415202 2020-08-07T15:13:15.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.722 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:15.722 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.349574+0000", 2020-08-07T15:13:15.722 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8599999999999998e-07 2020-08-07T15:13:15.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.723 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:15.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.349574+0000", 2020-08-07T15:13:15.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9826999999999994e-05 2020-08-07T15:13:15.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.724 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.724 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:15.724 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.349644+0000", 2020-08-07T15:13:15.724 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.05151385 2020-08-07T15:13:15.724 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.731 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.731 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.401158+0000", 2020-08-07T15:13:15.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22207370600000001 2020-08-07T15:13:15.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.735 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.736 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.623232+0000", 2020-08-07T15:13:15.736 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1999999999999998e-07 2020-08-07T15:13:15.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.736 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.737 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:15.737 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.623233+0000", 2020-08-07T15:13:15.737 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7947000000000005e-05 2020-08-07T15:13:15.737 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.737 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.738 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:15.738 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.623301+0000", 2020-08-07T15:13:15.738 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.281931072 2020-08-07T15:13:15.738 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.738 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:15.738 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.739 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.739 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.739 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:74192 6.e 6:7ab1a263:::benchmark_data_smithi074_63843_object9273:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e269)", 2020-08-07T15:13:15.739 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:01.584372+0000", 2020-08-07T15:13:15.739 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 13.529703357000001, 2020-08-07T15:13:15.740 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96302370800000003, 2020-08-07T15:13:15.740 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:15.740 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:15.740 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:15.740 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:15.741 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:15.741 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 74192 2020-08-07T15:13:15.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.741 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:15.741 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.743 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:15.743 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.584372+0000", 2020-08-07T15:13:15.743 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:15.743 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:15.744 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.584372+0000", 2020-08-07T15:13:15.744 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2189e-05 2020-08-07T15:13:15.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.745 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:15.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.584404+0000", 2020-08-07T15:13:15.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9859999999999999e-06 2020-08-07T15:13:15.746 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.746 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:15.746 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.584407+0000", 2020-08-07T15:13:15.746 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9560000000000002e-06 2020-08-07T15:13:15.747 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.747 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.747 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:15.747 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.584413+0000", 2020-08-07T15:13:15.748 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6790000000000001e-06 2020-08-07T15:13:15.748 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.748 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.748 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:15.748 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.584416+0000", 2020-08-07T15:13:15.749 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7514000000000001e-05 2020-08-07T15:13:15.749 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.749 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.749 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:15.750 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.584443+0000", 2020-08-07T15:13:15.750 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8560000000000001e-06 2020-08-07T15:13:15.750 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.750 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.751 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:15.751 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.584450+0000", 2020-08-07T15:13:15.751 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0091272760000000001 2020-08-07T15:13:15.751 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.751 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.753 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:15.753 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.593578+0000", 2020-08-07T15:13:15.753 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00055358000000000002 2020-08-07T15:13:15.753 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.754 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:15.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.594131+0000", 2020-08-07T15:13:15.755 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000182704 2020-08-07T15:13:15.755 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:15.756 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.594314+0000", 2020-08-07T15:13:15.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0025760470000000001 2020-08-07T15:13:15.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.757 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.757 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.596890+0000", 2020-08-07T15:13:15.757 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00028334900000000001 2020-08-07T15:13:15.757 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.758 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.758 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:15.758 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.597173+0000", 2020-08-07T15:13:15.758 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.95006429999999997 2020-08-07T15:13:15.758 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.759 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.759 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.547238+0000", 2020-08-07T15:13:15.759 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0430000000000002e-06 2020-08-07T15:13:15.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.760 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.760 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:15.760 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.547240+0000", 2020-08-07T15:13:15.760 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6477000000000006e-05 2020-08-07T15:13:15.760 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.761 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:15.761 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.547326+0000", 2020-08-07T15:13:15.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9752000000000006e-05 2020-08-07T15:13:15.761 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.762 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.762 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:15.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.547396+0000", 2020-08-07T15:13:15.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96302370800000003 2020-08-07T15:13:15.762 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.763 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:15.763 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.763 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.763 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.763 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:91031 6.e 6:703cf75c:::benchmark_data_smithi074_63843_object11378:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e275)", 2020-08-07T15:13:15.765 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:13.429600+0000", 2020-08-07T15:13:15.765 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 1.6844750559999999, 2020-08-07T15:13:15.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27365956600000002, 2020-08-07T15:13:15.765 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:15.766 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:15.766 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:15.766 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:15.766 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:15.766 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 91031 2020-08-07T15:13:15.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.767 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:15.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:15.767 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429600+0000", 2020-08-07T15:13:15.768 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:15.768 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.768 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.768 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:15.768 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429600+0000", 2020-08-07T15:13:15.769 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4419000000000002e-05 2020-08-07T15:13:15.769 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.769 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.769 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:15.770 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429655+0000", 2020-08-07T15:13:15.770 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3889999999999994e-06 2020-08-07T15:13:15.770 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.770 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.771 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:15.771 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429664+0000", 2020-08-07T15:13:15.771 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2275e-05 2020-08-07T15:13:15.771 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.771 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.772 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:15.772 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429676+0000", 2020-08-07T15:13:15.772 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1419999999999998e-06 2020-08-07T15:13:15.772 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.772 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.773 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:15.773 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429683+0000", 2020-08-07T15:13:15.773 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00078575699999999999 2020-08-07T15:13:15.773 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.774 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.774 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:15.775 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.430468+0000", 2020-08-07T15:13:15.775 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0164000000000002e-05 2020-08-07T15:13:15.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:15.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.430529+0000", 2020-08-07T15:13:15.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000156971 2020-08-07T15:13:15.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:15.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.430685+0000", 2020-08-07T15:13:15.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042982089999999999 2020-08-07T15:13:15.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:15.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.434984+0000", 2020-08-07T15:13:15.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1900000000000003e-07 2020-08-07T15:13:15.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.779 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:15.779 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.434984+0000", 2020-08-07T15:13:15.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5408000000000002e-05 2020-08-07T15:13:15.780 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.780 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:15.780 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.435040+0000", 2020-08-07T15:13:15.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045225700000000001 2020-08-07T15:13:15.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.781 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.781 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.439562+0000", 2020-08-07T15:13:15.781 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26362828900000002 2020-08-07T15:13:15.782 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.782 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.782 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.782 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.703190+0000", 2020-08-07T15:13:15.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.046e-06 2020-08-07T15:13:15.783 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:15.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.703192+0000", 2020-08-07T15:13:15.783 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8408e-05 2020-08-07T15:13:15.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:15.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.703260+0000", 2020-08-07T15:13:15.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27365956600000002 2020-08-07T15:13:15.784 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.786 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:15.786 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.787 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.787 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:91032 6.e 6:703cf75c:::benchmark_data_smithi074_63843_object11378:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e275)", 2020-08-07T15:13:15.787 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:13.429721+0000", 2020-08-07T15:13:15.787 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 1.6843549090000001, 2020-08-07T15:13:15.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29695800300000003, 2020-08-07T15:13:15.788 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:15.788 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:15.788 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:15.788 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:15.788 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:15.789 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 91032 2020-08-07T15:13:15.789 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.789 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:15.789 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.789 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:15.789 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429721+0000", 2020-08-07T15:13:15.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:15.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.790 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:15.790 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429721+0000", 2020-08-07T15:13:15.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3047000000000002e-05 2020-08-07T15:13:15.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:15.791 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429774+0000", 2020-08-07T15:13:15.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.339e-06 2020-08-07T15:13:15.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.792 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.792 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:15.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429779+0000", 2020-08-07T15:13:15.793 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0742e-05 2020-08-07T15:13:15.793 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.793 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.793 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:15.793 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429790+0000", 2020-08-07T15:13:15.794 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1860000000000002e-06 2020-08-07T15:13:15.794 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.794 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.794 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:15.794 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429794+0000", 2020-08-07T15:13:15.795 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001193794 2020-08-07T15:13:15.795 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.797 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:15.797 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.430988+0000", 2020-08-07T15:13:15.797 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7213000000000006e-05 2020-08-07T15:13:15.797 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.798 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.798 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:15.798 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.431085+0000", 2020-08-07T15:13:15.798 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000129875 2020-08-07T15:13:15.798 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:15.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.431215+0000", 2020-08-07T15:13:15.799 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038327349999999999 2020-08-07T15:13:15.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:15.800 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.435047+0000", 2020-08-07T15:13:15.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3499999999999996e-07 2020-08-07T15:13:15.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.801 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:15.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.435048+0000", 2020-08-07T15:13:15.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9911999999999999e-05 2020-08-07T15:13:15.801 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.801 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.802 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:15.802 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.435098+0000", 2020-08-07T15:13:15.802 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0046481059999999999 2020-08-07T15:13:15.802 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.802 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.439746+0000", 2020-08-07T15:13:15.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.286860114 2020-08-07T15:13:15.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:15.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.726606+0000", 2020-08-07T15:13:15.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0800000000000003e-07 2020-08-07T15:13:15.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:15.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.726607+0000", 2020-08-07T15:13:15.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1396999999999994e-05 2020-08-07T15:13:15.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:15.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:15.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:15.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.726679+0000", 2020-08-07T15:13:15.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29695800300000003 2020-08-07T15:13:15.807 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.807 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:15.808 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.808 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:15.808 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:15.808 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:15.809 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:13:15.811 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:13:15.811 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:13:15.813 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 63 16 93697 93681 11.6159 12.5859 0.00316773 0.0107566 2020-08-07T15:13:15.813 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.6 to 1 (10000) 2020-08-07T15:13:15.814 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 7 1 2020-08-07T15:13:15.814 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:15.810+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:15.912 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:15.911+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:16.013 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:16.012+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:16.119 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:16.118+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:16.220 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:16.219+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:16.272 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 64 16 95300 95284 11.63 12.5234 0.0115384 0.0107443 2020-08-07T15:13:16.324 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:16.321+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:16.426 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:16.423+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:16.537 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:16.538 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:13:16.538 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:13:16.539 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:16.539 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:16.530+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:16.552 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:13:16.586 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:13:16.638 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:16.635+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:16.640 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:13:16.720 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:13:16.737 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:16.734+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:16.778 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:13:16.839 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:13:16.840 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:16.838+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:16.874 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:16.875 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:16.883 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:13:16.886 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:13:16.944 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:13:16.951 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:16.950+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:16.970 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:13:16.972 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:13:16.973 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:13:16.973 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:13:16.973 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:13:16.973 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:13:16.973 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:13:16.974 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:13:16.974 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:13:17.066 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:17.054+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:17.117 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:17.117 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:17.131 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:13:17.164 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:17.156+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:17.247 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.7 to 1 (10000) 2020-08-07T15:13:17.248 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:13:17.260 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:17.258+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:17.262 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2020-08-07T15:13:17.263 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:13:17.271 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 65 16 96740 96724 11.6241 11.25 0.00346427 0.0107491 2020-08-07T15:13:17.364 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:17.360+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:17.368 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:17.368 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:17.385 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:13:17.475 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:17.474+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:17.573 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:17.565+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:17.634 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:17.635 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:17.647 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:13:17.668 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:17.666+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:17.758 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:13:17.762 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:13:17.769 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:17.767+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:17.775 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":490,"stamp":"2020-08-07T15:13:17.236489+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":772759552,"num_objects":11794,"num_object_clones":0,"num_object_copies":35382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":722,"num_objects_unfound":0,"num_objects_dirty":11794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94331,"num_write_kb":754648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":187,"ondisk_log_size":187,"up":195,"acting":195,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":196,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":10671696,"kb_used_data":2283024,"kb_used_omap":2,"kb_used_meta":8388605,"kb_avail":744303024,"statfs":{"total":773094113280,"available":762166296576,"internally_reserved":8589934592,"allocated":2337816576,"data_stored":2326502828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":8589932013},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,1,1,5,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":60973056,"num_objects":931,"num_object_clones":0,"num_object_copies":2793,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":55,"num_objects_unfound":0,"num_objects_dirty":931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7443,"num_write_kb":59544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.544143"},"pg_stats":[{"pgid":"1.28","version":"16'2","reported_seq":"42","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670295+0000","last_change":"2020-08-07T15:12:37.495634+0000","last_active":"2020-08-07T15:13:14.670295+0000","last_peered":"2020-08-07T15:13:14.670295+0000","last_clean":"2020-08-07T15:13:14.670295+0000","last_became_active":"2020-08-07T15:12:37.495085+0000","last_became_peered":"2020-08-07T15:12:37.495085+0000","last_unstale":"2020-08-07T15:13:14.670295+0000","last_undegraded":"2020-08-07T15:13:14.670295+0000","last_fullsized":"2020-08-07T15:13:14.670295+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.490655+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.490655+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.27","version":"16'2","reported_seq":"387","reported_epoch":"274","state":"active+clean","last_fresh":"2020-08-07T15:13:11.670546+0000","last_change":"2020-08-07T15:12:57.250017+0000","last_active":"2020-08-07T15:13:11.670546+0000","last_peered":"2020-08-07T15:13:11.670546+0000","last_clean":"2020-08-07T15:13:11.670546+0000","last_became_active":"2020-08-07T15:12:37.411869+0000","last_became_peered":"2020-08-07T15:12:37.411869+0000","last_unstale":"2020-08-07T15:13:11.670546+0000","last_undegraded":"2020-08-07T15:13:11.670546+0000","last_fullsized":"2020-08-07T15:13:11.670546+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:57.249927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:57.249927+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.26","version":"16'2","reported_seq":"51","reported_epoch":"276","state":"active+clean","last_fresh":"2020-08-07T15:13:13.685550+0000","last_change":"2020-08-07T15:12:40.765120+0000","last_active":"2020-08-07T15:13:13.685550+0000","last_peered":"2020-08-07T15:13:13.685550+0000","last_clean":"2020-08-07T15:13:13.685550+0000","last_became_active":"2020-08-07T15:12:37.457426+0000","last_became_peered":"2020-08-07T15:12:37.457426+0000","last_unstale":"2020-08-07T15:13:13.685550+0000","last_undegraded":"2020-08-07T15:13:13.685550+0000","last_fullsized":"2020-08-07T15:13:13.685550+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:40.765027+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_clean_scrub_stamp":"2020-08-07T15:12:40.765027+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.25","version":"75'3","reported_seq":"278","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.682783+0000","last_change":"2020-08-07T15:13:07.465488+0000","last_active":"2020-08-07T15:13:14.682783+0000","last_peered":"2020-08-07T15:13:14.682783+0000","last_clean":"2020-08-07T15:13:14.682783+0000","last_became_active":"2020-08-07T15:12:37.427816+0000","last_became_peered":"2020-08-07T15:12:37.427816+0000","last_unstale":"2020-08-07T15:13:14.682783+0000","last_undegraded":"2020-08-07T15:13:14.682783+0000","last_fullsized":"2020-08-07T15:13:14.682783+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:13:07.465410+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:13:07.465410+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.24","version":"16'2","reported_seq":"216","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.674070+0000","last_change":"2020-08-07T15:12:42.295755+0000","last_active":"2020-08-07T15:13:14.674070+0000","last_peered":"2020-08-07T15:13:14.674070+0000","last_clean":"2020-08-07T15:13:14.674070+0000","last_became_active":"2020-08-07T15:12:34.414144+0000","last_became_peered":"2020-08-07T15:12:34.414144+0000","last_unstale":"2020-08-07T15:13:14.674070+0000","last_undegraded":"2020-08-07T15:13:14.674070+0000","last_fullsized":"2020-08-07T15:13:14.674070+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:42.295672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:42.295672+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.23","version":"16'2","reported_seq":"322","reported_epoch":"274","state":"active+clean","last_fresh":"2020-08-07T15:13:11.670049+0000","last_change":"2020-08-07T15:12:55.290134+0000","last_active":"2020-08-07T15:13:11.670049+0000","last_peered":"2020-08-07T15:13:11.670049+0000","last_clean":"2020-08-07T15:13:11.670049+0000","last_became_active":"2020-08-07T15:12:37.410157+0000","last_became_peered":"2020-08-07T15:12:37.410157+0000","last_unstale":"2020-08-07T15:13:11.670049+0000","last_undegraded":"2020-08-07T15:13:11.670049+0000","last_fullsized":"2020-08-07T15:13:11.670049+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:55.290028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:55.290028+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1],"acting":[6,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.22","version":"16'2","reported_seq":"42","reported_epoch":"274","state":"active+clean","last_fresh":"2020-08-07T15:13:11.670308+0000","last_change":"2020-08-07T15:12:58.227750+0000","last_active":"2020-08-07T15:13:11.670308+0000","last_peered":"2020-08-07T15:13:11.670308+0000","last_clean":"2020-08-07T15:13:11.670308+0000","last_became_active":"2020-08-07T15:12:37.413046+0000","last_became_peered":"2020-08-07T15:12:37.413046+0000","last_unstale":"2020-08-07T15:13:11.670308+0000","last_undegraded":"2020-08-07T15:13:11.670308+0000","last_fullsized":"2020-08-07T15:13:11.670308+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:58.227699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:58.227699+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.21","version":"16'2","reported_seq":"52","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.673701+0000","last_change":"2020-08-07T15:12:58.387474+0000","last_active":"2020-08-07T15:13:14.673701+0000","last_peered":"2020-08-07T15:13:14.673701+0000","last_clean":"2020-08-07T15:13:14.673701+0000","last_became_active":"2020-08-07T15:12:37.418612+0000","last_became_peered":"2020-08-07T15:12:37.418612+0000","last_unstale":"2020-08-07T15:13:14.673701+0000","last_undegraded":"2020-08-07T15:13:14.673701+0000","last_fullsized":"2020-08-07T15:13:14.673701+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:58.387424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:58.387424+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.20","version":"16'2","reported_seq":"52","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.682629+0000","last_change":"2020-08-07T15:12:54.433326+0000","last_active":"2020-08-07T15:13:14.682629+0000","last_peered":"2020-08-07T15:13:14.682629+0000","last_clean":"2020-08-07T15:13:14.682629+0000","last_became_active":"2020-08-07T15:12:37.409249+0000","last_became_peered":"2020-08-07T15:12:37.409249+0000","last_unstale":"2020-08-07T15:13:14.682629+0000","last_undegraded":"2020-08-07T15:13:14.682629+0000","last_fullsized":"2020-08-07T15:13:14.682629+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:54.433245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:54.433245+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1f","version":"16'2","reported_seq":"45","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.682914+0000","last_change":"2020-08-07T15:13:09.471546+0000","last_active":"2020-08-07T15:13:14.682914+0000","last_peered":"2020-08-07T15:13:14.682914+0000","last_clean":"2020-08-07T15:13:14.682914+0000","last_became_active":"2020-08-07T15:12:37.410716+0000","last_became_peered":"2020-08-07T15:12:37.410716+0000","last_unstale":"2020-08-07T15:13:14.682914+0000","last_undegraded":"2020-08-07T15:13:14.682914+0000","last_fullsized":"2020-08-07T15:13:14.682914+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:09.471471+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_clean_scrub_stamp":"2020-08-07T15:13:09.471471+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1e","version":"16'2","reported_seq":"132","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.674282+0000","last_change":"2020-08-07T15:13:09.396723+0000","last_active":"2020-08-07T15:13:14.674282+0000","last_peered":"2020-08-07T15:13:14.674282+0000","last_clean":"2020-08-07T15:13:14.674282+0000","last_became_active":"2020-08-07T15:10:51.559894+0000","last_became_peered":"2020-08-07T15:10:51.559894+0000","last_unstale":"2020-08-07T15:13:14.674282+0000","last_undegraded":"2020-08-07T15:13:14.674282+0000","last_fullsized":"2020-08-07T15:13:14.674282+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:09.396633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:09.396633+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1d","version":"75'3","reported_seq":"141","reported_epoch":"276","state":"active+clean","last_fresh":"2020-08-07T15:13:13.685428+0000","last_change":"2020-08-07T15:12:28.855487+0000","last_active":"2020-08-07T15:13:13.685428+0000","last_peered":"2020-08-07T15:13:13.685428+0000","last_clean":"2020-08-07T15:13:13.685428+0000","last_became_active":"2020-08-07T15:12:07.356691+0000","last_became_peered":"2020-08-07T15:12:07.356691+0000","last_unstale":"2020-08-07T15:13:13.685428+0000","last_undegraded":"2020-08-07T15:13:13.685428+0000","last_fullsized":"2020-08-07T15:13:13.685428+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:28.855432+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:28.855432+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1c","version":"16'2","reported_seq":"130","reported_epoch":"274","state":"active+clean","last_fresh":"2020-08-07T15:13:11.670473+0000","last_change":"2020-08-07T15:13:07.471270+0000","last_active":"2020-08-07T15:13:11.670473+0000","last_peered":"2020-08-07T15:13:11.670473+0000","last_clean":"2020-08-07T15:13:11.670473+0000","last_became_active":"2020-08-07T15:13:02.591669+0000","last_became_peered":"2020-08-07T15:13:02.591669+0000","last_unstale":"2020-08-07T15:13:11.670473+0000","last_undegraded":"2020-08-07T15:13:11.670473+0000","last_fullsized":"2020-08-07T15:13:11.670473+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:07.471192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:07.471192+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1b","version":"16'2","reported_seq":"250","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670647+0000","last_change":"2020-08-07T15:10:57.797029+0000","last_active":"2020-08-07T15:13:14.670647+0000","last_peered":"2020-08-07T15:13:14.670647+0000","last_clean":"2020-08-07T15:13:14.670647+0000","last_became_active":"2020-08-07T15:10:51.557392+0000","last_became_peered":"2020-08-07T15:10:51.557392+0000","last_unstale":"2020-08-07T15:13:14.670647+0000","last_undegraded":"2020-08-07T15:13:14.670647+0000","last_fullsized":"2020-08-07T15:13:14.670647+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:57.796946+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:57.796946+0000","last_clean_scrub_stamp":"2020-08-07T15:10:57.796946+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1a","version":"16'2","reported_seq":"23","reported_epoch":"276","state":"active+clean","last_fresh":"2020-08-07T15:13:13.685115+0000","last_change":"2020-08-07T15:13:06.881943+0000","last_active":"2020-08-07T15:13:13.685115+0000","last_peered":"2020-08-07T15:13:13.685115+0000","last_clean":"2020-08-07T15:13:13.685115+0000","last_became_active":"2020-08-07T15:13:02.590409+0000","last_became_peered":"2020-08-07T15:13:02.590409+0000","last_unstale":"2020-08-07T15:13:13.685115+0000","last_undegraded":"2020-08-07T15:13:13.685115+0000","last_fullsized":"2020-08-07T15:13:13.685115+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:06.881863+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:13:06.881863+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.19","version":"16'2","reported_seq":"175","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.673825+0000","last_change":"2020-08-07T15:12:15.372999+0000","last_active":"2020-08-07T15:13:14.673825+0000","last_peered":"2020-08-07T15:13:14.673825+0000","last_clean":"2020-08-07T15:13:14.673825+0000","last_became_active":"2020-08-07T15:10:51.559752+0000","last_became_peered":"2020-08-07T15:10:51.559752+0000","last_unstale":"2020-08-07T15:13:14.673825+0000","last_undegraded":"2020-08-07T15:13:14.673825+0000","last_fullsized":"2020-08-07T15:13:14.673825+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:15.372910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:15.372910+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.18","version":"16'2","reported_seq":"275","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670410+0000","last_change":"2020-08-07T15:11:06.955336+0000","last_active":"2020-08-07T15:13:14.670410+0000","last_peered":"2020-08-07T15:13:14.670410+0000","last_clean":"2020-08-07T15:13:14.670410+0000","last_became_active":"2020-08-07T15:10:51.557665+0000","last_became_peered":"2020-08-07T15:10:51.557665+0000","last_unstale":"2020-08-07T15:13:14.670410+0000","last_undegraded":"2020-08-07T15:13:14.670410+0000","last_fullsized":"2020-08-07T15:13:14.670410+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:06.955262+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:06.955262+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.17","version":"16'2","reported_seq":"106","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.671445+0000","last_change":"2020-08-07T15:12:57.172008+0000","last_active":"2020-08-07T15:13:14.671445+0000","last_peered":"2020-08-07T15:13:14.671445+0000","last_clean":"2020-08-07T15:13:14.671445+0000","last_became_active":"2020-08-07T15:10:51.561769+0000","last_became_peered":"2020-08-07T15:10:51.561769+0000","last_unstale":"2020-08-07T15:13:14.671445+0000","last_undegraded":"2020-08-07T15:13:14.671445+0000","last_fullsized":"2020-08-07T15:13:14.671445+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:57.171893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:57.171893+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.16","version":"16'2","reported_seq":"51","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.671213+0000","last_change":"2020-08-07T15:12:52.576339+0000","last_active":"2020-08-07T15:13:14.671213+0000","last_peered":"2020-08-07T15:13:14.671213+0000","last_clean":"2020-08-07T15:13:14.671213+0000","last_became_active":"2020-08-07T15:12:52.575577+0000","last_became_peered":"2020-08-07T15:12:52.575577+0000","last_unstale":"2020-08-07T15:13:14.671213+0000","last_undegraded":"2020-08-07T15:13:14.671213+0000","last_fullsized":"2020-08-07T15:13:14.671213+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:12.883558+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:12.883558+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.883558+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.15","version":"75'3","reported_seq":"34","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.671173+0000","last_change":"2020-08-07T15:13:02.590465+0000","last_active":"2020-08-07T15:13:14.671173+0000","last_peered":"2020-08-07T15:13:14.671173+0000","last_clean":"2020-08-07T15:13:14.671173+0000","last_became_active":"2020-08-07T15:13:02.589879+0000","last_became_peered":"2020-08-07T15:13:02.589879+0000","last_unstale":"2020-08-07T15:13:14.671173+0000","last_undegraded":"2020-08-07T15:13:14.671173+0000","last_fullsized":"2020-08-07T15:13:14.671173+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:53.824617+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:53.824617+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.14","version":"16'2","reported_seq":"213","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.683186+0000","last_change":"2020-08-07T15:12:09.224231+0000","last_active":"2020-08-07T15:13:14.683186+0000","last_peered":"2020-08-07T15:13:14.683186+0000","last_clean":"2020-08-07T15:13:14.683186+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:13:14.683186+0000","last_undegraded":"2020-08-07T15:13:14.683186+0000","last_fullsized":"2020-08-07T15:13:14.683186+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:09.224158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.224158+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"227","reported_epoch":"276","state":"active+clean","last_fresh":"2020-08-07T15:13:13.685167+0000","last_change":"2020-08-07T15:12:19.577926+0000","last_active":"2020-08-07T15:13:13.685167+0000","last_peered":"2020-08-07T15:13:13.685167+0000","last_clean":"2020-08-07T15:13:13.685167+0000","last_became_active":"2020-08-07T15:12:06.375861+0000","last_became_peered":"2020-08-07T15:12:06.375861+0000","last_unstale":"2020-08-07T15:13:13.685167+0000","last_undegraded":"2020-08-07T15:13:13.685167+0000","last_fullsized":"2020-08-07T15:13:13.685167+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:07.887356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.887356+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"228","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670644+0000","last_change":"2020-08-07T15:12:26.104598+0000","last_active":"2020-08-07T15:13:14.670644+0000","last_peered":"2020-08-07T15:13:14.670644+0000","last_clean":"2020-08-07T15:13:14.670644+0000","last_became_active":"2020-08-07T15:12:06.367362+0000","last_became_peered":"2020-08-07T15:12:06.367362+0000","last_unstale":"2020-08-07T15:13:14.670644+0000","last_undegraded":"2020-08-07T15:13:14.670644+0000","last_fullsized":"2020-08-07T15:13:14.670644+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:26.104526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:26.104526+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"326","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670284+0000","last_change":"2020-08-07T15:12:24.105593+0000","last_active":"2020-08-07T15:13:14.670284+0000","last_peered":"2020-08-07T15:13:14.670284+0000","last_clean":"2020-08-07T15:13:14.670284+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:13:14.670284+0000","last_undegraded":"2020-08-07T15:13:14.670284+0000","last_fullsized":"2020-08-07T15:13:14.670284+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:24.105531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:24.105531+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"320","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670523+0000","last_change":"2020-08-07T15:12:13.231819+0000","last_active":"2020-08-07T15:13:14.670523+0000","last_peered":"2020-08-07T15:13:14.670523+0000","last_clean":"2020-08-07T15:13:14.670523+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:13:14.670523+0000","last_undegraded":"2020-08-07T15:13:14.670523+0000","last_fullsized":"2020-08-07T15:13:14.670523+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:13.231762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.231762+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3","version":"275'5336","reported_seq":"5390","reported_epoch":"275","state":"active+clean","last_fresh":"2020-08-07T15:13:12.748236+0000","last_change":"2020-08-07T15:13:02.600977+0000","last_active":"2020-08-07T15:13:12.748236+0000","last_peered":"2020-08-07T15:13:12.748236+0000","last_clean":"2020-08-07T15:13:12.748236+0000","last_became_active":"2020-08-07T15:13:02.590681+0000","last_became_peered":"2020-08-07T15:13:02.590681+0000","last_unstale":"2020-08-07T15:13:12.748236+0000","last_undegraded":"2020-08-07T15:13:12.748236+0000","last_fullsized":"2020-08-07T15:13:12.748236+0000","mapping_epoch":269,"log_start":"275'5328","ondisk_log_start":"275'5328","created":230,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":43712512,"num_objects":667,"num_object_clones":0,"num_object_copies":2001,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5336,"num_write_kb":42688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"277","reported_epoch":"274","state":"active+clean","last_fresh":"2020-08-07T15:13:11.670753+0000","last_change":"2020-08-07T15:12:27.410339+0000","last_active":"2020-08-07T15:13:11.670753+0000","last_peered":"2020-08-07T15:13:11.670753+0000","last_clean":"2020-08-07T15:13:11.670753+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:13:11.670753+0000","last_undegraded":"2020-08-07T15:13:11.670753+0000","last_fullsized":"2020-08-07T15:13:11.670753+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:27.410262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:27.410262+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"208","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.674235+0000","last_change":"2020-08-07T15:12:39.258928+0000","last_active":"2020-08-07T15:13:14.674235+0000","last_peered":"2020-08-07T15:13:14.674235+0000","last_clean":"2020-08-07T15:13:14.674235+0000","last_became_active":"2020-08-07T15:12:32.259956+0000","last_became_peered":"2020-08-07T15:12:32.259956+0000","last_unstale":"2020-08-07T15:13:14.674235+0000","last_undegraded":"2020-08-07T15:13:14.674235+0000","last_fullsized":"2020-08-07T15:13:14.674235+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:39.258857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:39.258857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"278'6480","reported_seq":"6536","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.728874+0000","last_change":"2020-08-07T15:12:10.465048+0000","last_active":"2020-08-07T15:13:16.728874+0000","last_peered":"2020-08-07T15:13:16.728874+0000","last_clean":"2020-08-07T15:13:16.728874+0000","last_became_active":"2020-08-07T15:12:10.464493+0000","last_became_peered":"2020-08-07T15:12:10.464493+0000","last_unstale":"2020-08-07T15:13:16.728874+0000","last_undegraded":"2020-08-07T15:13:16.728874+0000","last_fullsized":"2020-08-07T15:13:16.728874+0000","mapping_epoch":230,"log_start":"278'6477","ondisk_log_start":"278'6477","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53084160,"num_objects":810,"num_object_clones":0,"num_object_copies":2430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6480,"num_write_kb":51840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"235","reported_epoch":"276","state":"active+clean","last_fresh":"2020-08-07T15:13:13.685575+0000","last_change":"2020-08-07T15:12:19.561477+0000","last_active":"2020-08-07T15:13:13.685575+0000","last_peered":"2020-08-07T15:13:13.685575+0000","last_clean":"2020-08-07T15:13:13.685575+0000","last_became_active":"2020-08-07T15:12:06.367376+0000","last_became_peered":"2020-08-07T15:12:06.367376+0000","last_unstale":"2020-08-07T15:13:13.685575+0000","last_undegraded":"2020-08-07T15:13:13.685575+0000","last_fullsized":"2020-08-07T15:13:13.685575+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:12.319383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.319383+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"262","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.682902+0000","last_change":"2020-08-07T15:12:44.858389+0000","last_active":"2020-08-07T15:13:14.682902+0000","last_peered":"2020-08-07T15:13:14.682902+0000","last_clean":"2020-08-07T15:13:14.682902+0000","last_became_active":"2020-08-07T15:12:32.241146+0000","last_became_peered":"2020-08-07T15:12:32.241146+0000","last_unstale":"2020-08-07T15:13:14.682902+0000","last_undegraded":"2020-08-07T15:13:14.682902+0000","last_fullsized":"2020-08-07T15:13:14.682902+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:44.858310+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:44.858310+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"276'5896","reported_seq":"5951","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.676395+0000","last_change":"2020-08-07T15:12:10.464412+0000","last_active":"2020-08-07T15:13:14.676395+0000","last_peered":"2020-08-07T15:13:14.676395+0000","last_clean":"2020-08-07T15:13:14.676395+0000","last_became_active":"2020-08-07T15:12:10.463950+0000","last_became_peered":"2020-08-07T15:12:10.463950+0000","last_unstale":"2020-08-07T15:13:14.676395+0000","last_undegraded":"2020-08-07T15:13:14.676395+0000","last_fullsized":"2020-08-07T15:13:14.676395+0000","mapping_epoch":230,"log_start":"276'5890","ondisk_log_start":"276'5890","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48300032,"num_objects":737,"num_object_clones":0,"num_object_copies":2211,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5896,"num_write_kb":47168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"281","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.673568+0000","last_change":"2020-08-07T15:13:11.273543+0000","last_active":"2020-08-07T15:13:14.673568+0000","last_peered":"2020-08-07T15:13:14.673568+0000","last_clean":"2020-08-07T15:13:14.673568+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:13:14.673568+0000","last_undegraded":"2020-08-07T15:13:14.673568+0000","last_fullsized":"2020-08-07T15:13:14.673568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:11.273469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:13:11.273469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"337","reported_epoch":"274","state":"active+clean","last_fresh":"2020-08-07T15:13:11.670419+0000","last_change":"2020-08-07T15:12:53.210176+0000","last_active":"2020-08-07T15:13:11.670419+0000","last_peered":"2020-08-07T15:13:11.670419+0000","last_clean":"2020-08-07T15:13:11.670419+0000","last_became_active":"2020-08-07T15:12:32.239755+0000","last_became_peered":"2020-08-07T15:12:32.239755+0000","last_unstale":"2020-08-07T15:13:11.670419+0000","last_undegraded":"2020-08-07T15:13:11.670419+0000","last_fullsized":"2020-08-07T15:13:11.670419+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:53.210093+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_clean_scrub_stamp":"2020-08-07T15:12:53.210093+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"277'5832","reported_seq":"5884","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.858570+0000","last_change":"2020-08-07T15:12:10.398415+0000","last_active":"2020-08-07T15:13:14.858570+0000","last_peered":"2020-08-07T15:13:14.858570+0000","last_clean":"2020-08-07T15:13:14.858570+0000","last_became_active":"2020-08-07T15:12:10.397834+0000","last_became_peered":"2020-08-07T15:12:10.397834+0000","last_unstale":"2020-08-07T15:13:14.858570+0000","last_undegraded":"2020-08-07T15:13:14.858570+0000","last_fullsized":"2020-08-07T15:13:14.858570+0000","mapping_epoch":230,"log_start":"277'5825","ondisk_log_start":"277'5825","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47775744,"num_objects":729,"num_object_clones":0,"num_object_copies":2187,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5832,"num_write_kb":46656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"250","reported_epoch":"276","state":"active+clean","last_fresh":"2020-08-07T15:13:13.684883+0000","last_change":"2020-08-07T15:13:02.591821+0000","last_active":"2020-08-07T15:13:13.684883+0000","last_peered":"2020-08-07T15:13:13.684883+0000","last_clean":"2020-08-07T15:13:13.684883+0000","last_became_active":"2020-08-07T15:13:02.591402+0000","last_became_peered":"2020-08-07T15:13:02.591402+0000","last_unstale":"2020-08-07T15:13:13.684883+0000","last_undegraded":"2020-08-07T15:13:13.684883+0000","last_fullsized":"2020-08-07T15:13:13.684883+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:08.622683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:08.622683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"306","reported_epoch":"274","state":"active+clean","last_fresh":"2020-08-07T15:13:11.675972+0000","last_change":"2020-08-07T15:12:42.371309+0000","last_active":"2020-08-07T15:13:11.675972+0000","last_peered":"2020-08-07T15:13:11.675972+0000","last_clean":"2020-08-07T15:13:11.675972+0000","last_became_active":"2020-08-07T15:12:32.237984+0000","last_became_peered":"2020-08-07T15:12:32.237984+0000","last_unstale":"2020-08-07T15:13:11.675972+0000","last_undegraded":"2020-08-07T15:13:11.675972+0000","last_fullsized":"2020-08-07T15:13:11.675972+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:42.371229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:42.371229+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.7","version":"278'6080","reported_seq":"6136","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.717430+0000","last_change":"2020-08-07T15:12:10.464810+0000","last_active":"2020-08-07T15:13:16.717430+0000","last_peered":"2020-08-07T15:13:16.717430+0000","last_clean":"2020-08-07T15:13:16.717430+0000","last_became_active":"2020-08-07T15:12:10.464050+0000","last_became_peered":"2020-08-07T15:12:10.464050+0000","last_unstale":"2020-08-07T15:13:16.717430+0000","last_undegraded":"2020-08-07T15:13:16.717430+0000","last_fullsized":"2020-08-07T15:13:16.717430+0000","mapping_epoch":230,"log_start":"278'6075","ondisk_log_start":"278'6075","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49807360,"num_objects":760,"num_object_clones":0,"num_object_copies":2280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6080,"num_write_kb":48640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"29","reported_epoch":"274","state":"active+clean","last_fresh":"2020-08-07T15:13:11.669808+0000","last_change":"2020-08-07T15:13:02.591537+0000","last_active":"2020-08-07T15:13:11.669808+0000","last_peered":"2020-08-07T15:13:11.669808+0000","last_clean":"2020-08-07T15:13:11.669808+0000","last_became_active":"2020-08-07T15:13:02.590656+0000","last_became_peered":"2020-08-07T15:13:02.590656+0000","last_unstale":"2020-08-07T15:13:11.669808+0000","last_undegraded":"2020-08-07T15:13:11.669808+0000","last_fullsized":"2020-08-07T15:13:11.669808+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:39.812876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:39.812876+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"278","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670917+0000","last_change":"2020-08-07T15:11:40.328534+0000","last_active":"2020-08-07T15:13:14.670917+0000","last_peered":"2020-08-07T15:13:14.670917+0000","last_clean":"2020-08-07T15:13:14.670917+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:13:14.670917+0000","last_undegraded":"2020-08-07T15:13:14.670917+0000","last_fullsized":"2020-08-07T15:13:14.670917+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:11:40.328482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:11:40.328482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"275'5784","reported_seq":"5839","reported_epoch":"275","state":"active+clean","last_fresh":"2020-08-07T15:13:12.610600+0000","last_change":"2020-08-07T15:13:02.591551+0000","last_active":"2020-08-07T15:13:12.610600+0000","last_peered":"2020-08-07T15:13:12.610600+0000","last_clean":"2020-08-07T15:13:12.610600+0000","last_became_active":"2020-08-07T15:13:02.591000+0000","last_became_peered":"2020-08-07T15:13:02.591000+0000","last_unstale":"2020-08-07T15:13:12.610600+0000","last_undegraded":"2020-08-07T15:13:12.610600+0000","last_fullsized":"2020-08-07T15:13:12.610600+0000","mapping_epoch":269,"log_start":"275'5776","ondisk_log_start":"275'5776","created":230,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47382528,"num_objects":723,"num_object_clones":0,"num_object_copies":2169,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5784,"num_write_kb":46272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"299","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.683614+0000","last_change":"2020-08-07T15:12:10.585667+0000","last_active":"2020-08-07T15:13:14.683614+0000","last_peered":"2020-08-07T15:13:14.683614+0000","last_clean":"2020-08-07T15:13:14.683614+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:13:14.683614+0000","last_undegraded":"2020-08-07T15:13:14.683614+0000","last_fullsized":"2020-08-07T15:13:14.683614+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:10.585580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:10.585580+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"338","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.673444+0000","last_change":"2020-08-07T15:12:38.850878+0000","last_active":"2020-08-07T15:13:14.673444+0000","last_peered":"2020-08-07T15:13:14.673444+0000","last_clean":"2020-08-07T15:13:14.673444+0000","last_became_active":"2020-08-07T15:12:32.239794+0000","last_became_peered":"2020-08-07T15:12:32.239794+0000","last_unstale":"2020-08-07T15:13:14.673444+0000","last_undegraded":"2020-08-07T15:13:14.673444+0000","last_fullsized":"2020-08-07T15:13:14.673444+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:38.850809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:38.850809+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"278'6136","reported_seq":"6192","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.746670+0000","last_change":"2020-08-07T15:12:10.465113+0000","last_active":"2020-08-07T15:13:16.746670+0000","last_peered":"2020-08-07T15:13:16.746670+0000","last_clean":"2020-08-07T15:13:16.746670+0000","last_became_active":"2020-08-07T15:12:10.464582+0000","last_became_peered":"2020-08-07T15:12:10.464582+0000","last_unstale":"2020-08-07T15:13:16.746670+0000","last_undegraded":"2020-08-07T15:13:16.746670+0000","last_fullsized":"2020-08-07T15:13:16.746670+0000","mapping_epoch":230,"log_start":"278'6131","ondisk_log_start":"278'6131","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50266112,"num_objects":767,"num_object_clones":0,"num_object_copies":2301,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6136,"num_write_kb":49088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"293","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670935+0000","last_change":"2020-08-07T15:12:07.340958+0000","last_active":"2020-08-07T15:13:14.670935+0000","last_peered":"2020-08-07T15:13:14.670935+0000","last_clean":"2020-08-07T15:13:14.670935+0000","last_became_active":"2020-08-07T15:12:06.366967+0000","last_became_peered":"2020-08-07T15:12:06.366967+0000","last_unstale":"2020-08-07T15:13:14.670935+0000","last_undegraded":"2020-08-07T15:13:14.670935+0000","last_fullsized":"2020-08-07T15:13:14.670935+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:07.340855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.340855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"285","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670406+0000","last_change":"2020-08-07T15:12:32.247332+0000","last_active":"2020-08-07T15:13:14.670406+0000","last_peered":"2020-08-07T15:13:14.670406+0000","last_clean":"2020-08-07T15:13:14.670406+0000","last_became_active":"2020-08-07T15:12:32.246800+0000","last_became_peered":"2020-08-07T15:12:32.246800+0000","last_unstale":"2020-08-07T15:13:14.670406+0000","last_undegraded":"2020-08-07T15:13:14.670406+0000","last_fullsized":"2020-08-07T15:13:14.670406+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:09.261101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.261101+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"278'5976","reported_seq":"6032","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.677019+0000","last_change":"2020-08-07T15:12:10.466261+0000","last_active":"2020-08-07T15:13:16.677019+0000","last_peered":"2020-08-07T15:13:16.677019+0000","last_clean":"2020-08-07T15:13:16.677019+0000","last_became_active":"2020-08-07T15:12:10.465583+0000","last_became_peered":"2020-08-07T15:12:10.465583+0000","last_unstale":"2020-08-07T15:13:16.677019+0000","last_undegraded":"2020-08-07T15:13:16.677019+0000","last_fullsized":"2020-08-07T15:13:16.677019+0000","mapping_epoch":230,"log_start":"278'5967","ondisk_log_start":"278'5967","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48955392,"num_objects":747,"num_object_clones":0,"num_object_copies":2241,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5976,"num_write_kb":47808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"247","reported_epoch":"276","state":"active+clean","last_fresh":"2020-08-07T15:13:13.685321+0000","last_change":"2020-08-07T15:13:02.591378+0000","last_active":"2020-08-07T15:13:13.685321+0000","last_peered":"2020-08-07T15:13:13.685321+0000","last_clean":"2020-08-07T15:13:13.685321+0000","last_became_active":"2020-08-07T15:13:02.590901+0000","last_became_peered":"2020-08-07T15:13:02.590901+0000","last_unstale":"2020-08-07T15:13:13.685321+0000","last_undegraded":"2020-08-07T15:13:13.685321+0000","last_fullsized":"2020-08-07T15:13:13.685321+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:07.583152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.583152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"371","reported_epoch":"274","state":"active+clean","last_fresh":"2020-08-07T15:13:11.670808+0000","last_change":"2020-08-07T15:12:43.351056+0000","last_active":"2020-08-07T15:13:11.670808+0000","last_peered":"2020-08-07T15:13:11.670808+0000","last_clean":"2020-08-07T15:13:11.670808+0000","last_became_active":"2020-08-07T15:12:32.254842+0000","last_became_peered":"2020-08-07T15:12:32.254842+0000","last_unstale":"2020-08-07T15:13:11.670808+0000","last_undegraded":"2020-08-07T15:13:11.670808+0000","last_fullsized":"2020-08-07T15:13:11.670808+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:43.350971+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:43.350971+0000","last_clean_scrub_stamp":"2020-08-07T15:12:43.350971+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"278'5704","reported_seq":"5758","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.676316+0000","last_change":"2020-08-07T15:12:10.467447+0000","last_active":"2020-08-07T15:13:16.676316+0000","last_peered":"2020-08-07T15:13:16.676316+0000","last_clean":"2020-08-07T15:13:16.676316+0000","last_became_active":"2020-08-07T15:12:10.467000+0000","last_became_peered":"2020-08-07T15:12:10.467000+0000","last_unstale":"2020-08-07T15:13:16.676316+0000","last_undegraded":"2020-08-07T15:13:16.676316+0000","last_fullsized":"2020-08-07T15:13:16.676316+0000","mapping_epoch":230,"log_start":"278'5696","ondisk_log_start":"278'5696","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":46727168,"num_objects":713,"num_object_clones":0,"num_object_copies":2139,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5704,"num_write_kb":45632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"265","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670752+0000","last_change":"2020-08-07T15:13:11.097271+0000","last_active":"2020-08-07T15:13:14.670752+0000","last_peered":"2020-08-07T15:13:14.670752+0000","last_clean":"2020-08-07T15:13:14.670752+0000","last_became_active":"2020-08-07T15:10:46.498230+0000","last_became_peered":"2020-08-07T15:10:46.498230+0000","last_unstale":"2020-08-07T15:13:14.670752+0000","last_undegraded":"2020-08-07T15:13:14.670752+0000","last_fullsized":"2020-08-07T15:13:14.670752+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:11.097214+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:13:11.097214+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"278'5976","reported_seq":"6032","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.640275+0000","last_change":"2020-08-07T15:12:10.465075+0000","last_active":"2020-08-07T15:13:16.640275+0000","last_peered":"2020-08-07T15:13:16.640275+0000","last_clean":"2020-08-07T15:13:16.640275+0000","last_became_active":"2020-08-07T15:12:10.464640+0000","last_became_peered":"2020-08-07T15:12:10.464640+0000","last_unstale":"2020-08-07T15:13:16.640275+0000","last_undegraded":"2020-08-07T15:13:16.640275+0000","last_fullsized":"2020-08-07T15:13:16.640275+0000","mapping_epoch":230,"log_start":"278'5969","ondisk_log_start":"278'5969","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48955392,"num_objects":747,"num_object_clones":0,"num_object_copies":2241,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5976,"num_write_kb":47808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"121","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670174+0000","last_change":"2020-08-07T15:12:45.243933+0000","last_active":"2020-08-07T15:13:14.670174+0000","last_peered":"2020-08-07T15:13:14.670174+0000","last_clean":"2020-08-07T15:13:14.670174+0000","last_became_active":"2020-08-07T15:12:45.243349+0000","last_became_peered":"2020-08-07T15:12:45.243349+0000","last_unstale":"2020-08-07T15:13:14.670174+0000","last_undegraded":"2020-08-07T15:13:14.670174+0000","last_fullsized":"2020-08-07T15:13:14.670174+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.490655+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.490655+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.e","version":"277'5536","reported_seq":"5593","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.834720+0000","last_change":"2020-08-07T15:13:12.351009+0000","last_active":"2020-08-07T15:13:14.834720+0000","last_peered":"2020-08-07T15:13:14.834720+0000","last_clean":"2020-08-07T15:13:14.834720+0000","last_became_active":"2020-08-07T15:12:10.464900+0000","last_became_peered":"2020-08-07T15:12:10.464900+0000","last_unstale":"2020-08-07T15:13:14.834720+0000","last_undegraded":"2020-08-07T15:13:14.834720+0000","last_fullsized":"2020-08-07T15:13:14.834720+0000","mapping_epoch":230,"log_start":"277'5532","ondisk_log_start":"277'5532","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"275'5310","last_scrub_stamp":"2020-08-07T15:13:12.350955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:13:12.350955+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":45350912,"num_objects":692,"num_object_clones":0,"num_object_copies":2076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5536,"num_write_kb":44288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"83","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.674040+0000","last_change":"2020-08-07T15:13:10.415039+0000","last_active":"2020-08-07T15:13:14.674040+0000","last_peered":"2020-08-07T15:13:14.674040+0000","last_clean":"2020-08-07T15:13:14.674040+0000","last_became_active":"2020-08-07T15:13:02.590018+0000","last_became_peered":"2020-08-07T15:13:02.590018+0000","last_unstale":"2020-08-07T15:13:14.674040+0000","last_undegraded":"2020-08-07T15:13:14.674040+0000","last_fullsized":"2020-08-07T15:13:14.674040+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:10.414954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:10.414954+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"275'5776","reported_seq":"5833","reported_epoch":"275","state":"active+remapped+backfill_toofull","last_fresh":"2020-08-07T15:13:12.571215+0000","last_change":"2020-08-07T15:12:57.279725+0000","last_active":"2020-08-07T15:13:12.571215+0000","last_peered":"2020-08-07T15:13:12.571215+0000","last_clean":"2020-08-07T15:12:55.240424+0000","last_became_active":"2020-08-07T15:12:57.254078+0000","last_became_peered":"2020-08-07T15:12:57.254078+0000","last_unstale":"2020-08-07T15:13:12.571215+0000","last_undegraded":"2020-08-07T15:13:12.571215+0000","last_fullsized":"2020-08-07T15:13:12.571215+0000","mapping_epoch":266,"log_start":"275'5768","ondisk_log_start":"275'5768","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47316992,"num_objects":722,"num_object_clones":0,"num_object_copies":2166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":722,"num_objects_unfound":0,"num_objects_dirty":722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5776,"num_write_kb":46208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,0],"avail_no_missing":["6","0","5"],"object_location_counts":[{"shards":"0,5,6","objects":722}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"296","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.673462+0000","last_change":"2020-08-07T15:12:13.680022+0000","last_active":"2020-08-07T15:13:14.673462+0000","last_peered":"2020-08-07T15:13:14.673462+0000","last_clean":"2020-08-07T15:13:14.673462+0000","last_became_active":"2020-08-07T15:12:06.380392+0000","last_became_peered":"2020-08-07T15:12:06.380392+0000","last_unstale":"2020-08-07T15:13:14.673462+0000","last_undegraded":"2020-08-07T15:13:14.673462+0000","last_fullsized":"2020-08-07T15:13:14.673462+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:13.679943+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.679943+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"278'5877","reported_seq":"5931","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.761337+0000","last_change":"2020-08-07T15:12:10.398816+0000","last_active":"2020-08-07T15:13:16.761337+0000","last_peered":"2020-08-07T15:13:16.761337+0000","last_clean":"2020-08-07T15:13:16.761337+0000","last_became_active":"2020-08-07T15:12:10.398332+0000","last_became_peered":"2020-08-07T15:12:10.398332+0000","last_unstale":"2020-08-07T15:13:16.761337+0000","last_undegraded":"2020-08-07T15:13:16.761337+0000","last_fullsized":"2020-08-07T15:13:16.761337+0000","mapping_epoch":230,"log_start":"278'5872","ondisk_log_start":"278'5872","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48144384,"num_objects":735,"num_object_clones":0,"num_object_copies":2205,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5877,"num_write_kb":47016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"251","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.674265+0000","last_change":"2020-08-07T15:12:14.241339+0000","last_active":"2020-08-07T15:13:14.674265+0000","last_peered":"2020-08-07T15:13:14.674265+0000","last_clean":"2020-08-07T15:13:14.674265+0000","last_became_active":"2020-08-07T15:10:46.498051+0000","last_became_peered":"2020-08-07T15:10:46.498051+0000","last_unstale":"2020-08-07T15:13:14.674265+0000","last_undegraded":"2020-08-07T15:13:14.674265+0000","last_fullsized":"2020-08-07T15:13:14.674265+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:14.241262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:14.241262+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"278'5968","reported_seq":"6025","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.729214+0000","last_change":"2020-08-07T15:13:02.590816+0000","last_active":"2020-08-07T15:13:16.729214+0000","last_peered":"2020-08-07T15:13:16.729214+0000","last_clean":"2020-08-07T15:13:16.729214+0000","last_became_active":"2020-08-07T15:13:02.590300+0000","last_became_peered":"2020-08-07T15:13:02.590300+0000","last_unstale":"2020-08-07T15:13:16.729214+0000","last_undegraded":"2020-08-07T15:13:16.729214+0000","last_fullsized":"2020-08-07T15:13:16.729214+0000","mapping_epoch":269,"log_start":"278'5960","ondisk_log_start":"278'5960","created":230,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48889856,"num_objects":746,"num_object_clones":0,"num_object_copies":2238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5968,"num_write_kb":47744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3],"acting":[2,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"324","reported_epoch":"274","state":"active+clean","last_fresh":"2020-08-07T15:13:11.670694+0000","last_change":"2020-08-07T15:12:41.445456+0000","last_active":"2020-08-07T15:13:11.670694+0000","last_peered":"2020-08-07T15:13:11.670694+0000","last_clean":"2020-08-07T15:13:11.670694+0000","last_became_active":"2020-08-07T15:12:32.241722+0000","last_became_peered":"2020-08-07T15:12:32.241722+0000","last_unstale":"2020-08-07T15:13:11.670694+0000","last_undegraded":"2020-08-07T15:13:11.670694+0000","last_fullsized":"2020-08-07T15:13:11.670694+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:41.445379+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_clean_scrub_stamp":"2020-08-07T15:12:41.445379+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"278'5774","reported_seq":"5827","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.726224+0000","last_change":"2020-08-07T15:13:02.596246+0000","last_active":"2020-08-07T15:13:16.726224+0000","last_peered":"2020-08-07T15:13:16.726224+0000","last_clean":"2020-08-07T15:13:16.726224+0000","last_became_active":"2020-08-07T15:13:02.591329+0000","last_became_peered":"2020-08-07T15:13:02.591329+0000","last_unstale":"2020-08-07T15:13:16.726224+0000","last_undegraded":"2020-08-07T15:13:16.726224+0000","last_fullsized":"2020-08-07T15:13:16.726224+0000","mapping_epoch":269,"log_start":"278'5768","ondisk_log_start":"278'5768","created":230,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47300608,"num_objects":722,"num_object_clones":0,"num_object_copies":2166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5774,"num_write_kb":46192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"40","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.674376+0000","last_change":"2020-08-07T15:13:11.459633+0000","last_active":"2020-08-07T15:13:14.674376+0000","last_peered":"2020-08-07T15:13:14.674376+0000","last_clean":"2020-08-07T15:13:14.674376+0000","last_became_active":"2020-08-07T15:13:02.591018+0000","last_became_peered":"2020-08-07T15:13:02.591018+0000","last_unstale":"2020-08-07T15:13:14.674376+0000","last_undegraded":"2020-08-07T15:13:14.674376+0000","last_fullsized":"2020-08-07T15:13:14.674376+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:11.459555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:11.459555+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.c","version":"278'6200","reported_seq":"6256","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.701385+0000","last_change":"2020-08-07T15:12:10.465251+0000","last_active":"2020-08-07T15:13:16.701385+0000","last_peered":"2020-08-07T15:13:16.701385+0000","last_clean":"2020-08-07T15:13:16.701385+0000","last_became_active":"2020-08-07T15:12:10.464698+0000","last_became_peered":"2020-08-07T15:12:10.464698+0000","last_unstale":"2020-08-07T15:13:16.701385+0000","last_undegraded":"2020-08-07T15:13:16.701385+0000","last_fullsized":"2020-08-07T15:13:16.701385+0000","mapping_epoch":230,"log_start":"278'6197","ondisk_log_start":"278'6197","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50790400,"num_objects":775,"num_object_clones":0,"num_object_copies":2325,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6200,"num_write_kb":49600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"249","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670814+0000","last_change":"2020-08-07T15:12:44.203274+0000","last_active":"2020-08-07T15:13:14.670814+0000","last_peered":"2020-08-07T15:13:14.670814+0000","last_clean":"2020-08-07T15:13:14.670814+0000","last_became_active":"2020-08-07T15:12:06.367081+0000","last_became_peered":"2020-08-07T15:12:06.367081+0000","last_unstale":"2020-08-07T15:13:14.670814+0000","last_undegraded":"2020-08-07T15:13:14.670814+0000","last_fullsized":"2020-08-07T15:13:14.670814+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:44.203221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:44.203221+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":772759552,"num_objects":11792,"num_object_clones":0,"num_object_copies":35376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":722,"num_objects_unfound":0,"num_objects_dirty":11792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94331,"num_write_kb":754648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2337095680,"data_stored":2337095680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":41,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":123,"acting":123,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542229,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1277068,"kb_used_data":228484,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93094772,"statfs":{"total":96636764160,"available":95329046528,"internally_reserved":1073741824,"allocated":233967616,"data_stored":232570671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":9.5359999999999996,"15min":3.774},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":2.294,"5min":856.07100000000003,"15min":856.07100000000003},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":1.002,"15min":0.97599999999999998},"min":{"1min":0.52300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.55,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":0.90100000000000002}]},{"osd":1,"last update":"Fri Aug 7 15:13:10 2020","interfaces":[{"interface":"back","average":{"1min":10.148999999999999,"5min":10.148999999999999,"15min":10.148999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":205.73699999999999,"5min":205.73699999999999,"15min":205.73699999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":3.827,"5min":3.827,"15min":3.827},"last":1.4550000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.321,"15min":1.1910000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.5630000000000002,"5min":5.6619999999999999,"15min":5.6619999999999999},"last":297.721},{"interface":"front","average":{"1min":1.081,"5min":1.3280000000000001,"15min":1.1879999999999999},"min":{"1min":0.69299999999999995,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.3769999999999998,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":1.7809999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":30.957999999999998,"5min":21.91,"15min":8.4179999999999993},"min":{"1min":0.68899999999999995,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":656.99099999999999,"5min":917.38699999999994,"15min":917.38699999999994},"last":1.194},{"interface":"front","average":{"1min":1.173,"5min":4.8879999999999999,"15min":2.524},"min":{"1min":0.61599999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":4.0460000000000003,"5min":333.59199999999998,"15min":333.59199999999998},"last":1.3049999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":8.8599999999999994,"5min":2.944,"15min":1.8320000000000001},"min":{"1min":0.66000000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":172.012,"5min":172.012,"15min":172.012},"last":1.9810000000000001},{"interface":"front","average":{"1min":1.5509999999999999,"5min":11.529,"15min":4.7169999999999996},"min":{"1min":0.81200000000000006,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":5.4500000000000002,"5min":866.24099999999999,"15min":866.24099999999999},"last":1.165}]},{"osd":5,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":9.3930000000000007,"15min":28.135000000000002},"min":{"1min":0.68400000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":3.536,"5min":678.76099999999997,"15min":831.88099999999997},"last":1.49},{"interface":"front","average":{"1min":1.623,"5min":1.831,"15min":1.607},"min":{"1min":0.89000000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":3.593,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":1.609}]},{"osd":7,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":3.9929999999999999,"15min":2.343},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":4.0810000000000004,"5min":224.14500000000001,"15min":224.14500000000001},"last":2.1840000000000002},{"interface":"front","average":{"1min":1.671,"5min":1.7649999999999999,"15min":1.593},"min":{"1min":0.92000000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":3.6200000000000001,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":2.2149999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574935,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1331576,"kb_used_data":282992,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93040264,"statfs":{"total":96636764160,"available":95273230336,"internally_reserved":1073741824,"allocated":289783808,"data_stored":288364486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":37.372999999999998,"5min":16.672000000000001,"15min":6.4459999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":763.39099999999996,"5min":763.39099999999996,"15min":763.39099999999996},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.157,"5min":9.7789999999999999,"15min":4.0259999999999998},"min":{"1min":0.61699999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.8279999999999998,"5min":529.88699999999994,"15min":529.88699999999994},"last":1.1299999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:13:05 2020","interfaces":[{"interface":"back","average":{"1min":23.431000000000001,"5min":23.431000000000001,"15min":23.431000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":472.59699999999998,"5min":472.59699999999998,"15min":472.59699999999998},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.8650000000000002,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.3109999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":3.0009999999999999,"5min":1.6850000000000001,"15min":1.3580000000000001},"min":{"1min":0.52000000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":33.131999999999998,"5min":33.131999999999998,"15min":33.131999999999998},"last":0.84899999999999998},{"interface":"front","average":{"1min":2.8410000000000002,"5min":1.494,"15min":1.3460000000000001},"min":{"1min":0.61799999999999999,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":32.924999999999997,"5min":32.924999999999997,"15min":32.924999999999997},"last":1.169}]},{"osd":3,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":9.4659999999999993,"15min":4.1719999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":5.9329999999999998,"5min":698.19899999999996,"15min":698.19899999999996},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.238,"5min":1.2669999999999999,"15min":1.347},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":3.8599999999999999,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":1.903}]},{"osd":4,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":25.448,"5min":6.069,"15min":3.1309999999999998},"min":{"1min":0.76400000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":460.78399999999999,"5min":460.78399999999999,"15min":460.78399999999999},"last":1.347},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.474,"15min":1.6060000000000001},"min":{"1min":0.86899999999999999,"5min":0.621,"15min":0.621},"max":{"1min":2.4079999999999999,"5min":3.694,"15min":5.2990000000000004},"last":668.06799999999998}]},{"osd":5,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.3720000000000001,"15min":1.3660000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":5.9989999999999997,"5min":6.2080000000000002,"15min":6.2080000000000002},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.492,"15min":1.6739999999999999},"min":{"1min":0.83599999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":4.407,"5min":4.407,"15min":5.327},"last":1.3779999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":9.1839999999999993,"15min":4.016},"min":{"1min":0.88,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.3290000000000002,"5min":695.92899999999997,"15min":695.92899999999997},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.597,"5min":1.55,"15min":12.734},"min":{"1min":0.89500000000000002,"5min":0.87,"15min":0.87},"max":{"1min":3.302,"5min":3.7309999999999999,"15min":266.26999999999998},"last":1.4299999999999999}]}]},{"osd":1,"up_from":227,"seq":974957576207,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1275548,"kb_used_data":226964,"kb_used_omap":2,"kb_used_meta":1048573,"kb_avail":93096292,"statfs":{"total":96636764160,"available":95330603008,"internally_reserved":1073741824,"allocated":232411136,"data_stored":231010265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":1073739245},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":27.861000000000001,"5min":27.861000000000001,"15min":27.861000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":540.92499999999995,"5min":540.92499999999995,"15min":540.92499999999995},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":3.7069999999999999,"5min":3.7069999999999999,"15min":3.7069999999999999},"last":1.1910000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":2.5779999999999998},{"interface":"front","average":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":3.758,"5min":3.758,"15min":3.758},"last":3.1760000000000002}]},{"osd":3,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":13.909000000000001,"5min":13.909000000000001,"15min":13.909000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":250.995,"5min":250.995,"15min":250.995},"last":2.5510000000000002},{"interface":"front","average":{"1min":2.04,"5min":2.04,"15min":2.04},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":6.6580000000000004,"5min":6.6580000000000004,"15min":6.6580000000000004},"last":2.9990000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":2.6230000000000002},{"interface":"front","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.746}]},{"osd":5,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":2.0150000000000001},{"interface":"front","average":{"1min":1.498,"5min":1.498,"15min":1.498},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.6720000000000002,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":2.665}]},{"osd":6,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.1019999999999999},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":9.4299999999999997,"5min":9.4299999999999997,"15min":9.4299999999999997},"last":2.9630000000000001},{"interface":"front","average":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":3.4199999999999999,"5min":3.4199999999999999,"15min":3.4199999999999999},"last":2.5299999999999998}]},{"osd":7,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":40.520000000000003,"5min":40.520000000000003,"15min":40.520000000000003},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":778.745,"5min":778.745,"15min":778.745},"last":3.0529999999999999},{"interface":"front","average":{"1min":24.100000000000001,"5min":24.100000000000001,"15min":24.100000000000001},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":442.62799999999999,"5min":442.62799999999999,"15min":442.62799999999999},"last":3.0299999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574935,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1236688,"kb_used_data":188104,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93135152,"statfs":{"total":96636764160,"available":95370395648,"internally_reserved":1073741824,"allocated":192618496,"data_stored":191199174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:13:07 2020","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.468,"5min":2.468,"15min":2.468},"last":3.5249999999999999},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.5550000000000002,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":2.9460000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.0620000000000001,"15min":0.97199999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":7.3470000000000004,"5min":7.3470000000000004,"15min":7.3470000000000004},"last":3.6099999999999999},{"interface":"front","average":{"1min":6.2839999999999998,"5min":4.665,"15min":23.381},"min":{"1min":0.47399999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":75.622,"5min":226.911,"15min":606.18499999999995},"last":2.2709999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":28.079000000000001,"5min":6.6920000000000002,"15min":2.8540000000000001},"min":{"1min":0.60999999999999999,"5min":0.54100000000000004,"15min":0.52300000000000002},"max":{"1min":431.95299999999997,"5min":431.95299999999997,"15min":431.95299999999997},"last":4.1619999999999999},{"interface":"front","average":{"1min":1.425,"5min":1.46,"15min":1.2649999999999999},"min":{"1min":0.66400000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":5.3780000000000001,"5min":11.19,"15min":11.19},"last":4.3449999999999998}]},{"osd":4,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.302,"15min":1.2110000000000001},"min":{"1min":0.57399999999999995,"5min":0.501,"15min":0.501},"max":{"1min":1.819,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.304,"5min":1.448,"15min":1.321},"min":{"1min":0.71099999999999997,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.8849999999999998,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":3.5579999999999998}]},{"osd":5,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.6200000000000001,"5min":14.598000000000001,"15min":5.6840000000000002},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":7.3799999999999999,"5min":539.31100000000004,"15min":539.31100000000004},"last":2.6259999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.49,"15min":1.234},"min":{"1min":0.81399999999999995,"5min":0.752,"15min":0.65900000000000003},"max":{"1min":2.9420000000000002,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":0.78700000000000003}]},{"osd":6,"last update":"Fri Aug 7 15:12:34 2020","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.4399999999999999,"15min":6.2220000000000004},"min":{"1min":0.80200000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.8610000000000002,"5min":4.1310000000000002,"15min":124.738},"last":3.585},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.5549999999999999,"15min":12.952999999999999},"min":{"1min":0.73499999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":5.274,"5min":5.274,"15min":294.94},"last":2.6629999999999998}]},{"osd":7,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.4790000000000001,"15min":1.55},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.79300000000000004},"max":{"1min":1.8620000000000001,"5min":4.7169999999999996,"15min":6.468},"last":2.7149999999999999},{"interface":"front","average":{"1min":66.063000000000002,"5min":23.518000000000001,"15min":8.7629999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":971.12699999999995,"5min":971.12699999999995,"15min":971.12699999999995},"last":2.6890000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574935,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1432128,"kb_used_data":383544,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92939712,"statfs":{"total":96636764160,"available":95170265088,"internally_reserved":1073741824,"allocated":392749056,"data_stored":391329734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":32.965000000000003,"5min":7.8639999999999999,"15min":3.3660000000000001},"min":{"1min":0.41899999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":501.53800000000001,"5min":501.53800000000001,"15min":501.53800000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":27.341999999999999,"5min":6.6900000000000004,"15min":2.8799999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":501.36000000000001,"5min":501.36000000000001,"15min":501.36000000000001},"last":0.60199999999999998}]},{"osd":1,"last update":"Fri Aug 7 15:13:07 2020","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":5.7869999999999999,"5min":5.7869999999999999,"15min":5.7869999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":24.699999999999999,"5min":24.699999999999999,"15min":24.699999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":465.69200000000001,"5min":465.69200000000001,"15min":465.69200000000001},"last":0.88400000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.2130000000000001,"15min":1.179},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":23.064,"5min":15.281000000000001,"15min":6.1310000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":412.61500000000001,"5min":501.75599999999997,"15min":501.75599999999997},"last":0.748}]},{"osd":4,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.5940000000000001,"15min":1.2709999999999999},"min":{"1min":0.67400000000000004,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.712,"5min":19.783000000000001,"15min":19.783000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.4610000000000001,"15min":1.5820000000000001},"min":{"1min":0.68999999999999995,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8979999999999999,"5min":8.048,"15min":8.048},"last":0.95799999999999996}]},{"osd":5,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.3460000000000001,"15min":1.29},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.4009999999999998,"5min":5.2220000000000004,"15min":5.2220000000000004},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.4990000000000001,"5min":15.715,"15min":22.254999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":8.0649999999999995,"5min":627.53300000000002,"15min":627.53300000000002},"last":0.93500000000000005}]},{"osd":6,"last update":"Fri Aug 7 15:12:37 2020","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":9.9480000000000004,"15min":4.1299999999999999},"min":{"1min":0.875,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.9450000000000001,"5min":821.91899999999998,"15min":821.91899999999998},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.3959999999999999,"5min":9.9580000000000002,"15min":4.2000000000000002},"min":{"1min":0.70199999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.9540000000000002,"5min":751.70000000000005,"15min":751.70000000000005},"last":0.97999999999999998}]},{"osd":7,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":12.522,"15min":5.335},"min":{"1min":0.85599999999999998,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.3519999999999999,"5min":501.91699999999997,"15min":501.91699999999997},"last":1.113},{"interface":"front","average":{"1min":1.7230000000000001,"5min":2.1080000000000001,"15min":1.7},"min":{"1min":0.88500000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":7.2149999999999999,"5min":35.646999999999998,"15min":35.646999999999998},"last":1.143}]}]},{"osd":3,"up_from":13,"seq":55834574935,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1535456,"kb_used_data":486872,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92836384,"statfs":{"total":96636764160,"available":95064457216,"internally_reserved":1073741824,"allocated":498556928,"data_stored":497137606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,5,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.226,"15min":1.244},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":4.6580000000000004,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.2050000000000001,"15min":1.4930000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":4.6349999999999998,"5min":4.9119999999999999,"15min":13.067},"last":1.157}]},{"osd":1,"last update":"Fri Aug 7 15:13:08 2020","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.639,"15min":1.639},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":4.6890000000000001,"5min":4.6890000000000001,"15min":4.6890000000000001},"last":1.234},{"interface":"front","average":{"1min":1.841,"5min":1.841,"15min":1.841},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":5.0330000000000004,"5min":5.0330000000000004,"15min":5.0330000000000004},"last":3.9460000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.365,"15min":1.22},"min":{"1min":0.65900000000000003,"5min":0.56999999999999995,"15min":0.56899999999999995},"max":{"1min":2.8300000000000001,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":4.181},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.9690000000000001,"15min":1.7789999999999999},"min":{"1min":0.64500000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":5.3090000000000002,"5min":51.530000000000001,"15min":51.530000000000001},"last":1.8580000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":3.831,"5min":17.792999999999999,"15min":7.0620000000000003},"min":{"1min":0.54500000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":56.688000000000002,"5min":646.26700000000005,"15min":646.26700000000005},"last":5.2720000000000002},{"interface":"front","average":{"1min":1.421,"5min":15.494999999999999,"15min":26.565000000000001},"min":{"1min":0.68100000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":3.2109999999999999,"5min":844.10500000000002,"15min":844.10500000000002},"last":3.988}]},{"osd":5,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":14.452999999999999,"5min":8.5429999999999993,"15min":3.859},"min":{"1min":0.95499999999999996,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":244.08099999999999,"5min":403.88400000000001,"15min":403.88400000000001},"last":4.1429999999999998},{"interface":"front","average":{"1min":1.78,"5min":1.8979999999999999,"15min":1.8120000000000001},"min":{"1min":0.98499999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":4.5960000000000001,"5min":8.9120000000000008,"15min":8.9120000000000008},"last":4.2409999999999997}]},{"osd":6,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.7969999999999999,"5min":1.706,"15min":1.5900000000000001},"min":{"1min":0.74099999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":5.2729999999999997,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":1.609},{"interface":"front","average":{"1min":1.907,"5min":13.411,"15min":5.7949999999999999},"min":{"1min":1.032,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":5.3460000000000001,"5min":688.15999999999997,"15min":688.15999999999997},"last":1.635}]},{"osd":7,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.7450000000000001,"5min":1.5960000000000001,"15min":1.9730000000000001},"min":{"1min":0.85599999999999998,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":4.8099999999999996,"5min":6.8209999999999997,"15min":6.8209999999999997},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.6559999999999999,"15min":1.8700000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.6099999999999999,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":5.3109999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574935,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1293584,"kb_used_data":245000,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93078256,"statfs":{"total":96636764160,"available":95312134144,"internally_reserved":1073741824,"allocated":250880000,"data_stored":249460678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":49.795000000000002,"5min":19.646000000000001,"15min":7.3360000000000003},"min":{"1min":0.46800000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":829.15099999999995,"5min":829.15099999999995,"15min":829.15099999999995},"last":0.58799999999999997},{"interface":"front","average":{"1min":3.8149999999999999,"5min":1.639,"15min":1.1120000000000001},"min":{"1min":0.45300000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":48.274000000000001,"5min":48.274000000000001,"15min":48.274000000000001},"last":0.64600000000000002}]},{"osd":1,"last update":"Fri Aug 7 15:13:08 2020","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.80800000000000005}]},{"osd":2,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":4.1159999999999997,"5min":1.901,"15min":1.601},"min":{"1min":0.64300000000000002,"5min":0.56399999999999995,"15min":0.52100000000000002},"max":{"1min":49.878999999999998,"5min":49.878999999999998,"15min":49.878999999999998},"last":0.78700000000000003},{"interface":"front","average":{"1min":4.2270000000000003,"5min":2.0529999999999999,"15min":1.694},"min":{"1min":0.97899999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":49.939999999999998,"5min":49.939999999999998,"15min":49.939999999999998},"last":0.84999999999999998}]},{"osd":3,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":27.59,"5min":17.260999999999999,"15min":6.7590000000000003},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":426.50799999999998,"5min":638.92100000000005,"15min":638.92100000000005},"last":1.595},{"interface":"front","average":{"1min":1.411,"5min":1.3799999999999999,"15min":1.294},"min":{"1min":0.75700000000000001,"5min":0.67800000000000005,"15min":0.66700000000000004},"max":{"1min":3.2360000000000002,"5min":6.431,"15min":6.431},"last":1.153}]},{"osd":5,"last update":"Fri Aug 7 15:12:29 2020","interfaces":[{"interface":"back","average":{"1min":52.308999999999997,"5min":28.574999999999999,"15min":10.609999999999999},"min":{"1min":0.56100000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":862.97299999999996,"5min":990.71600000000001,"15min":990.71600000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":49.549999999999997,"5min":11.01,"15min":4.7539999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":780.00699999999995,"5min":780.00699999999995,"15min":780.00699999999995},"last":0.95599999999999996}]},{"osd":6,"last update":"Fri Aug 7 15:12:34 2020","interfaces":[{"interface":"back","average":{"1min":3.8239999999999998,"5min":1.9870000000000001,"15min":1.855},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":48.575000000000003,"5min":48.575000000000003,"15min":48.575000000000003},"last":1.179},{"interface":"front","average":{"1min":1.109,"5min":1.333,"15min":13.183},"min":{"1min":0.64600000000000002,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.847,"5min":6.7729999999999997,"15min":389.14600000000002},"last":0.92900000000000005}]},{"osd":7,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":4.1349999999999998,"5min":4.327,"15min":2.3809999999999998},"min":{"1min":0.74199999999999999,"5min":0.62,"15min":0.62},"max":{"1min":48.75,"5min":252.798,"15min":252.798},"last":1.728},{"interface":"front","average":{"1min":4.2610000000000001,"5min":14.635,"15min":5.9749999999999996},"min":{"1min":0.84799999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":49.654000000000003,"5min":501.93000000000001,"15min":501.93000000000001},"last":1.486}]}]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1289648,"kb_used_data":241064,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93082192,"statfs":{"total":96636764160,"available":95316164608,"internally_reserved":1073741824,"allocated":246849536,"data_stored":245430214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":3.629,"5min":1.506,"15min":1.22},"min":{"1min":0.42199999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":43.292999999999999,"5min":43.292999999999999,"15min":43.292999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":3.1520000000000001,"5min":4.4249999999999998,"15min":2.0110000000000001},"min":{"1min":0.44700000000000001,"5min":0.438,"15min":0.438},"max":{"1min":42.933,"5min":329.11399999999998,"15min":329.11399999999998},"last":2.5049999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:13:11 2020","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":4.4039999999999999,"5min":4.4039999999999999,"15min":4.4039999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":4.0810000000000004,"5min":4.0810000000000004,"15min":4.0810000000000004},"last":0.95899999999999996}]},{"osd":2,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":3.5499999999999998,"5min":1.5800000000000001,"15min":6.673},"min":{"1min":0.65700000000000003,"5min":0.499,"15min":0.499},"max":{"1min":43.923000000000002,"5min":43.923000000000002,"15min":168.56899999999999},"last":1.1180000000000001},{"interface":"front","average":{"1min":3.23,"5min":1.6399999999999999,"15min":22.634},"min":{"1min":0.52800000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":43.076999999999998,"5min":43.076999999999998,"15min":676.27099999999996},"last":2.5430000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.728,"15min":1.321},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":4.7640000000000002,"5min":29.129000000000001,"15min":29.129000000000001},"last":1.377},{"interface":"front","average":{"1min":1.3779999999999999,"5min":10.029999999999999,"15min":4.1520000000000001},"min":{"1min":0.73999999999999999,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":5.2009999999999996,"5min":712.98699999999997,"15min":712.98699999999997},"last":1.1539999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":12.779999999999999,"15min":38.188000000000002},"min":{"1min":0.77200000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":4.9969999999999999,"5min":971.67999999999995,"15min":971.67999999999995},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.2470000000000001,"15min":1.286},"min":{"1min":0.73599999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.1160000000000001,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":2.581}]},{"osd":6,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":35.393000000000001,"5min":28.459,"15min":43.091999999999999},"min":{"1min":0.72199999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":676.20000000000005,"5min":966.60900000000004,"15min":966.60900000000004},"last":1.579},{"interface":"front","average":{"1min":1.528,"5min":6.7789999999999999,"15min":19.603999999999999},"min":{"1min":0.89500000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":5.2220000000000004,"5min":501.79000000000002,"15min":517.26700000000005},"last":1.9219999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.462,"15min":1.3600000000000001},"min":{"1min":0.85399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":4.8600000000000003,"5min":5.1559999999999997,"15min":5.1559999999999997},"last":208.03399999999999},{"interface":"front","average":{"1min":1.5669999999999999,"5min":12.834,"15min":5.3220000000000001},"min":{"1min":0.70099999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":3.8999999999999999,"5min":515.09100000000001,"15min":515.09100000000001},"last":2.7069999999999999}]}]}]}} 2020-08-07T15:13:17.782 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:13:17.869 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:17.868+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:17.893 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:17.893 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:17.908 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:13:17.997 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:17.970+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:18.079 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:18.071+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:18.094 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:18.095 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:18.111 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:13:18.198 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:18.197+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:18.272 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 66 16 98060 98044 11.6042 10.3125 0.0241185 0.0107683 2020-08-07T15:13:18.281 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:18.273+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:18.318 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:13:18.326 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:13:18.334 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2020-08-07T15:13:18.247139+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":784760832,"num_objects":11978,"num_object_clones":0,"num_object_copies":35934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":784,"num_objects_unfound":0,"num_objects_dirty":11978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95796,"num_write_kb":766368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":195,"acting":195,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":196,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":10690904,"kb_used_data":2302232,"kb_used_omap":2,"kb_used_meta":8388605,"kb_avail":744283816,"statfs":{"total":773094113280,"available":762146627584,"internally_reserved":8589934592,"allocated":2357485568,"data_stored":2346141984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":8589932013},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,4,2,5,2,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":61833216,"num_objects":945,"num_object_clones":0,"num_object_copies":2835,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":62,"num_objects_unfound":0,"num_objects_dirty":945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7548,"num_write_kb":60384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.587839"},"pg_stats":[{"pgid":"1.28","version":"16'2","reported_seq":"42","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670295+0000","last_change":"2020-08-07T15:12:37.495634+0000","last_active":"2020-08-07T15:13:14.670295+0000","last_peered":"2020-08-07T15:13:14.670295+0000","last_clean":"2020-08-07T15:13:14.670295+0000","last_became_active":"2020-08-07T15:12:37.495085+0000","last_became_peered":"2020-08-07T15:12:37.495085+0000","last_unstale":"2020-08-07T15:13:14.670295+0000","last_undegraded":"2020-08-07T15:13:14.670295+0000","last_fullsized":"2020-08-07T15:13:14.670295+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.490655+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.490655+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.27","version":"16'2","reported_seq":"391","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:15.686151+0000","last_change":"2020-08-07T15:12:57.250017+0000","last_active":"2020-08-07T15:13:15.686151+0000","last_peered":"2020-08-07T15:13:15.686151+0000","last_clean":"2020-08-07T15:13:15.686151+0000","last_became_active":"2020-08-07T15:12:37.411869+0000","last_became_peered":"2020-08-07T15:12:37.411869+0000","last_unstale":"2020-08-07T15:13:15.686151+0000","last_undegraded":"2020-08-07T15:13:15.686151+0000","last_fullsized":"2020-08-07T15:13:15.686151+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:57.249927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:57.249927+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.26","version":"16'2","reported_seq":"51","reported_epoch":"276","state":"active+clean","last_fresh":"2020-08-07T15:13:13.685550+0000","last_change":"2020-08-07T15:12:40.765120+0000","last_active":"2020-08-07T15:13:13.685550+0000","last_peered":"2020-08-07T15:13:13.685550+0000","last_clean":"2020-08-07T15:13:13.685550+0000","last_became_active":"2020-08-07T15:12:37.457426+0000","last_became_peered":"2020-08-07T15:12:37.457426+0000","last_unstale":"2020-08-07T15:13:13.685550+0000","last_undegraded":"2020-08-07T15:13:13.685550+0000","last_fullsized":"2020-08-07T15:13:13.685550+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:40.765027+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_clean_scrub_stamp":"2020-08-07T15:12:40.765027+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.25","version":"75'3","reported_seq":"278","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.682783+0000","last_change":"2020-08-07T15:13:07.465488+0000","last_active":"2020-08-07T15:13:14.682783+0000","last_peered":"2020-08-07T15:13:14.682783+0000","last_clean":"2020-08-07T15:13:14.682783+0000","last_became_active":"2020-08-07T15:12:37.427816+0000","last_became_peered":"2020-08-07T15:12:37.427816+0000","last_unstale":"2020-08-07T15:13:14.682783+0000","last_undegraded":"2020-08-07T15:13:14.682783+0000","last_fullsized":"2020-08-07T15:13:14.682783+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:13:07.465410+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:13:07.465410+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.24","version":"16'2","reported_seq":"216","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.674070+0000","last_change":"2020-08-07T15:12:42.295755+0000","last_active":"2020-08-07T15:13:14.674070+0000","last_peered":"2020-08-07T15:13:14.674070+0000","last_clean":"2020-08-07T15:13:14.674070+0000","last_became_active":"2020-08-07T15:12:34.414144+0000","last_became_peered":"2020-08-07T15:12:34.414144+0000","last_unstale":"2020-08-07T15:13:14.674070+0000","last_undegraded":"2020-08-07T15:13:14.674070+0000","last_fullsized":"2020-08-07T15:13:14.674070+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:42.295672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:42.295672+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.23","version":"16'2","reported_seq":"326","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:15.685911+0000","last_change":"2020-08-07T15:12:55.290134+0000","last_active":"2020-08-07T15:13:15.685911+0000","last_peered":"2020-08-07T15:13:15.685911+0000","last_clean":"2020-08-07T15:13:15.685911+0000","last_became_active":"2020-08-07T15:12:37.410157+0000","last_became_peered":"2020-08-07T15:12:37.410157+0000","last_unstale":"2020-08-07T15:13:15.685911+0000","last_undegraded":"2020-08-07T15:13:15.685911+0000","last_fullsized":"2020-08-07T15:13:15.685911+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:55.290028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:55.290028+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1],"acting":[6,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.22","version":"16'2","reported_seq":"46","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:15.686042+0000","last_change":"2020-08-07T15:12:58.227750+0000","last_active":"2020-08-07T15:13:15.686042+0000","last_peered":"2020-08-07T15:13:15.686042+0000","last_clean":"2020-08-07T15:13:15.686042+0000","last_became_active":"2020-08-07T15:12:37.413046+0000","last_became_peered":"2020-08-07T15:12:37.413046+0000","last_unstale":"2020-08-07T15:13:15.686042+0000","last_undegraded":"2020-08-07T15:13:15.686042+0000","last_fullsized":"2020-08-07T15:13:15.686042+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:58.227699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:58.227699+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.21","version":"16'2","reported_seq":"52","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.673701+0000","last_change":"2020-08-07T15:12:58.387474+0000","last_active":"2020-08-07T15:13:14.673701+0000","last_peered":"2020-08-07T15:13:14.673701+0000","last_clean":"2020-08-07T15:13:14.673701+0000","last_became_active":"2020-08-07T15:12:37.418612+0000","last_became_peered":"2020-08-07T15:12:37.418612+0000","last_unstale":"2020-08-07T15:13:14.673701+0000","last_undegraded":"2020-08-07T15:13:14.673701+0000","last_fullsized":"2020-08-07T15:13:14.673701+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:58.387424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:58.387424+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.20","version":"16'2","reported_seq":"52","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.682629+0000","last_change":"2020-08-07T15:12:54.433326+0000","last_active":"2020-08-07T15:13:14.682629+0000","last_peered":"2020-08-07T15:13:14.682629+0000","last_clean":"2020-08-07T15:13:14.682629+0000","last_became_active":"2020-08-07T15:12:37.409249+0000","last_became_peered":"2020-08-07T15:12:37.409249+0000","last_unstale":"2020-08-07T15:13:14.682629+0000","last_undegraded":"2020-08-07T15:13:14.682629+0000","last_fullsized":"2020-08-07T15:13:14.682629+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:54.433245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:54.433245+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1f","version":"16'2","reported_seq":"45","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.682914+0000","last_change":"2020-08-07T15:13:09.471546+0000","last_active":"2020-08-07T15:13:14.682914+0000","last_peered":"2020-08-07T15:13:14.682914+0000","last_clean":"2020-08-07T15:13:14.682914+0000","last_became_active":"2020-08-07T15:12:37.410716+0000","last_became_peered":"2020-08-07T15:12:37.410716+0000","last_unstale":"2020-08-07T15:13:14.682914+0000","last_undegraded":"2020-08-07T15:13:14.682914+0000","last_fullsized":"2020-08-07T15:13:14.682914+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:09.471471+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_clean_scrub_stamp":"2020-08-07T15:13:09.471471+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1e","version":"16'2","reported_seq":"132","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.674282+0000","last_change":"2020-08-07T15:13:09.396723+0000","last_active":"2020-08-07T15:13:14.674282+0000","last_peered":"2020-08-07T15:13:14.674282+0000","last_clean":"2020-08-07T15:13:14.674282+0000","last_became_active":"2020-08-07T15:10:51.559894+0000","last_became_peered":"2020-08-07T15:10:51.559894+0000","last_unstale":"2020-08-07T15:13:14.674282+0000","last_undegraded":"2020-08-07T15:13:14.674282+0000","last_fullsized":"2020-08-07T15:13:14.674282+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:09.396633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:09.396633+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1d","version":"75'3","reported_seq":"141","reported_epoch":"276","state":"active+clean","last_fresh":"2020-08-07T15:13:13.685428+0000","last_change":"2020-08-07T15:12:28.855487+0000","last_active":"2020-08-07T15:13:13.685428+0000","last_peered":"2020-08-07T15:13:13.685428+0000","last_clean":"2020-08-07T15:13:13.685428+0000","last_became_active":"2020-08-07T15:12:07.356691+0000","last_became_peered":"2020-08-07T15:12:07.356691+0000","last_unstale":"2020-08-07T15:13:13.685428+0000","last_undegraded":"2020-08-07T15:13:13.685428+0000","last_fullsized":"2020-08-07T15:13:13.685428+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:28.855432+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:28.855432+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1c","version":"16'2","reported_seq":"134","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:15.686010+0000","last_change":"2020-08-07T15:13:07.471270+0000","last_active":"2020-08-07T15:13:15.686010+0000","last_peered":"2020-08-07T15:13:15.686010+0000","last_clean":"2020-08-07T15:13:15.686010+0000","last_became_active":"2020-08-07T15:13:02.591669+0000","last_became_peered":"2020-08-07T15:13:02.591669+0000","last_unstale":"2020-08-07T15:13:15.686010+0000","last_undegraded":"2020-08-07T15:13:15.686010+0000","last_fullsized":"2020-08-07T15:13:15.686010+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:07.471192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:07.471192+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1b","version":"16'2","reported_seq":"250","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670647+0000","last_change":"2020-08-07T15:10:57.797029+0000","last_active":"2020-08-07T15:13:14.670647+0000","last_peered":"2020-08-07T15:13:14.670647+0000","last_clean":"2020-08-07T15:13:14.670647+0000","last_became_active":"2020-08-07T15:10:51.557392+0000","last_became_peered":"2020-08-07T15:10:51.557392+0000","last_unstale":"2020-08-07T15:13:14.670647+0000","last_undegraded":"2020-08-07T15:13:14.670647+0000","last_fullsized":"2020-08-07T15:13:14.670647+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:57.796946+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:57.796946+0000","last_clean_scrub_stamp":"2020-08-07T15:10:57.796946+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1a","version":"16'2","reported_seq":"23","reported_epoch":"276","state":"active+clean","last_fresh":"2020-08-07T15:13:13.685115+0000","last_change":"2020-08-07T15:13:06.881943+0000","last_active":"2020-08-07T15:13:13.685115+0000","last_peered":"2020-08-07T15:13:13.685115+0000","last_clean":"2020-08-07T15:13:13.685115+0000","last_became_active":"2020-08-07T15:13:02.590409+0000","last_became_peered":"2020-08-07T15:13:02.590409+0000","last_unstale":"2020-08-07T15:13:13.685115+0000","last_undegraded":"2020-08-07T15:13:13.685115+0000","last_fullsized":"2020-08-07T15:13:13.685115+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:06.881863+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:13:06.881863+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.19","version":"16'2","reported_seq":"175","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.673825+0000","last_change":"2020-08-07T15:12:15.372999+0000","last_active":"2020-08-07T15:13:14.673825+0000","last_peered":"2020-08-07T15:13:14.673825+0000","last_clean":"2020-08-07T15:13:14.673825+0000","last_became_active":"2020-08-07T15:10:51.559752+0000","last_became_peered":"2020-08-07T15:10:51.559752+0000","last_unstale":"2020-08-07T15:13:14.673825+0000","last_undegraded":"2020-08-07T15:13:14.673825+0000","last_fullsized":"2020-08-07T15:13:14.673825+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:15.372910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:15.372910+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.18","version":"16'2","reported_seq":"275","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670410+0000","last_change":"2020-08-07T15:11:06.955336+0000","last_active":"2020-08-07T15:13:14.670410+0000","last_peered":"2020-08-07T15:13:14.670410+0000","last_clean":"2020-08-07T15:13:14.670410+0000","last_became_active":"2020-08-07T15:10:51.557665+0000","last_became_peered":"2020-08-07T15:10:51.557665+0000","last_unstale":"2020-08-07T15:13:14.670410+0000","last_undegraded":"2020-08-07T15:13:14.670410+0000","last_fullsized":"2020-08-07T15:13:14.670410+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:06.955262+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:06.955262+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.17","version":"16'2","reported_seq":"106","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.671445+0000","last_change":"2020-08-07T15:12:57.172008+0000","last_active":"2020-08-07T15:13:14.671445+0000","last_peered":"2020-08-07T15:13:14.671445+0000","last_clean":"2020-08-07T15:13:14.671445+0000","last_became_active":"2020-08-07T15:10:51.561769+0000","last_became_peered":"2020-08-07T15:10:51.561769+0000","last_unstale":"2020-08-07T15:13:14.671445+0000","last_undegraded":"2020-08-07T15:13:14.671445+0000","last_fullsized":"2020-08-07T15:13:14.671445+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:57.171893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:57.171893+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.16","version":"16'2","reported_seq":"51","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.671213+0000","last_change":"2020-08-07T15:12:52.576339+0000","last_active":"2020-08-07T15:13:14.671213+0000","last_peered":"2020-08-07T15:13:14.671213+0000","last_clean":"2020-08-07T15:13:14.671213+0000","last_became_active":"2020-08-07T15:12:52.575577+0000","last_became_peered":"2020-08-07T15:12:52.575577+0000","last_unstale":"2020-08-07T15:13:14.671213+0000","last_undegraded":"2020-08-07T15:13:14.671213+0000","last_fullsized":"2020-08-07T15:13:14.671213+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:12.883558+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:12.883558+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.883558+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.15","version":"75'3","reported_seq":"34","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.671173+0000","last_change":"2020-08-07T15:13:02.590465+0000","last_active":"2020-08-07T15:13:14.671173+0000","last_peered":"2020-08-07T15:13:14.671173+0000","last_clean":"2020-08-07T15:13:14.671173+0000","last_became_active":"2020-08-07T15:13:02.589879+0000","last_became_peered":"2020-08-07T15:13:02.589879+0000","last_unstale":"2020-08-07T15:13:14.671173+0000","last_undegraded":"2020-08-07T15:13:14.671173+0000","last_fullsized":"2020-08-07T15:13:14.671173+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:53.824617+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:53.824617+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.14","version":"16'2","reported_seq":"213","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.683186+0000","last_change":"2020-08-07T15:12:09.224231+0000","last_active":"2020-08-07T15:13:14.683186+0000","last_peered":"2020-08-07T15:13:14.683186+0000","last_clean":"2020-08-07T15:13:14.683186+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:13:14.683186+0000","last_undegraded":"2020-08-07T15:13:14.683186+0000","last_fullsized":"2020-08-07T15:13:14.683186+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:09.224158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.224158+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"227","reported_epoch":"276","state":"active+clean","last_fresh":"2020-08-07T15:13:13.685167+0000","last_change":"2020-08-07T15:12:19.577926+0000","last_active":"2020-08-07T15:13:13.685167+0000","last_peered":"2020-08-07T15:13:13.685167+0000","last_clean":"2020-08-07T15:13:13.685167+0000","last_became_active":"2020-08-07T15:12:06.375861+0000","last_became_peered":"2020-08-07T15:12:06.375861+0000","last_unstale":"2020-08-07T15:13:13.685167+0000","last_undegraded":"2020-08-07T15:13:13.685167+0000","last_fullsized":"2020-08-07T15:13:13.685167+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:07.887356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.887356+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"228","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670644+0000","last_change":"2020-08-07T15:12:26.104598+0000","last_active":"2020-08-07T15:13:14.670644+0000","last_peered":"2020-08-07T15:13:14.670644+0000","last_clean":"2020-08-07T15:13:14.670644+0000","last_became_active":"2020-08-07T15:12:06.367362+0000","last_became_peered":"2020-08-07T15:12:06.367362+0000","last_unstale":"2020-08-07T15:13:14.670644+0000","last_undegraded":"2020-08-07T15:13:14.670644+0000","last_fullsized":"2020-08-07T15:13:14.670644+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:26.104526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:26.104526+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"326","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670284+0000","last_change":"2020-08-07T15:12:24.105593+0000","last_active":"2020-08-07T15:13:14.670284+0000","last_peered":"2020-08-07T15:13:14.670284+0000","last_clean":"2020-08-07T15:13:14.670284+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:13:14.670284+0000","last_undegraded":"2020-08-07T15:13:14.670284+0000","last_fullsized":"2020-08-07T15:13:14.670284+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:24.105531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:24.105531+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"320","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670523+0000","last_change":"2020-08-07T15:12:13.231819+0000","last_active":"2020-08-07T15:13:14.670523+0000","last_peered":"2020-08-07T15:13:14.670523+0000","last_clean":"2020-08-07T15:13:14.670523+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:13:14.670523+0000","last_undegraded":"2020-08-07T15:13:14.670523+0000","last_fullsized":"2020-08-07T15:13:14.670523+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:13.231762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.231762+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3","version":"279'5897","reported_seq":"5952","reported_epoch":"279","state":"active+clean","last_fresh":"2020-08-07T15:13:17.743576+0000","last_change":"2020-08-07T15:13:02.600977+0000","last_active":"2020-08-07T15:13:17.743576+0000","last_peered":"2020-08-07T15:13:17.743576+0000","last_clean":"2020-08-07T15:13:17.743576+0000","last_became_active":"2020-08-07T15:13:02.590681+0000","last_became_peered":"2020-08-07T15:13:02.590681+0000","last_unstale":"2020-08-07T15:13:17.743576+0000","last_undegraded":"2020-08-07T15:13:17.743576+0000","last_fullsized":"2020-08-07T15:13:17.743576+0000","mapping_epoch":269,"log_start":"279'5892","ondisk_log_start":"279'5892","created":230,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48308224,"num_objects":738,"num_object_clones":0,"num_object_copies":2214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5897,"num_write_kb":47176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"281","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:15.686324+0000","last_change":"2020-08-07T15:12:27.410339+0000","last_active":"2020-08-07T15:13:15.686324+0000","last_peered":"2020-08-07T15:13:15.686324+0000","last_clean":"2020-08-07T15:13:15.686324+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:13:15.686324+0000","last_undegraded":"2020-08-07T15:13:15.686324+0000","last_fullsized":"2020-08-07T15:13:15.686324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:27.410262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:27.410262+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"208","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.674235+0000","last_change":"2020-08-07T15:12:39.258928+0000","last_active":"2020-08-07T15:13:14.674235+0000","last_peered":"2020-08-07T15:13:14.674235+0000","last_clean":"2020-08-07T15:13:14.674235+0000","last_became_active":"2020-08-07T15:12:32.259956+0000","last_became_peered":"2020-08-07T15:12:32.259956+0000","last_unstale":"2020-08-07T15:13:14.674235+0000","last_undegraded":"2020-08-07T15:13:14.674235+0000","last_fullsized":"2020-08-07T15:13:14.674235+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:39.258857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:39.258857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"278'6480","reported_seq":"6536","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.728874+0000","last_change":"2020-08-07T15:12:10.465048+0000","last_active":"2020-08-07T15:13:16.728874+0000","last_peered":"2020-08-07T15:13:16.728874+0000","last_clean":"2020-08-07T15:13:16.728874+0000","last_became_active":"2020-08-07T15:12:10.464493+0000","last_became_peered":"2020-08-07T15:12:10.464493+0000","last_unstale":"2020-08-07T15:13:16.728874+0000","last_undegraded":"2020-08-07T15:13:16.728874+0000","last_fullsized":"2020-08-07T15:13:16.728874+0000","mapping_epoch":230,"log_start":"278'6477","ondisk_log_start":"278'6477","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53084160,"num_objects":810,"num_object_clones":0,"num_object_copies":2430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6480,"num_write_kb":51840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"235","reported_epoch":"276","state":"active+clean","last_fresh":"2020-08-07T15:13:13.685575+0000","last_change":"2020-08-07T15:12:19.561477+0000","last_active":"2020-08-07T15:13:13.685575+0000","last_peered":"2020-08-07T15:13:13.685575+0000","last_clean":"2020-08-07T15:13:13.685575+0000","last_became_active":"2020-08-07T15:12:06.367376+0000","last_became_peered":"2020-08-07T15:12:06.367376+0000","last_unstale":"2020-08-07T15:13:13.685575+0000","last_undegraded":"2020-08-07T15:13:13.685575+0000","last_fullsized":"2020-08-07T15:13:13.685575+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:12.319383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.319383+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"262","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.682902+0000","last_change":"2020-08-07T15:12:44.858389+0000","last_active":"2020-08-07T15:13:14.682902+0000","last_peered":"2020-08-07T15:13:14.682902+0000","last_clean":"2020-08-07T15:13:14.682902+0000","last_became_active":"2020-08-07T15:12:32.241146+0000","last_became_peered":"2020-08-07T15:12:32.241146+0000","last_unstale":"2020-08-07T15:13:14.682902+0000","last_undegraded":"2020-08-07T15:13:14.682902+0000","last_fullsized":"2020-08-07T15:13:14.682902+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:44.858310+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:44.858310+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"276'5896","reported_seq":"5951","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.676395+0000","last_change":"2020-08-07T15:12:10.464412+0000","last_active":"2020-08-07T15:13:14.676395+0000","last_peered":"2020-08-07T15:13:14.676395+0000","last_clean":"2020-08-07T15:13:14.676395+0000","last_became_active":"2020-08-07T15:12:10.463950+0000","last_became_peered":"2020-08-07T15:12:10.463950+0000","last_unstale":"2020-08-07T15:13:14.676395+0000","last_undegraded":"2020-08-07T15:13:14.676395+0000","last_fullsized":"2020-08-07T15:13:14.676395+0000","mapping_epoch":230,"log_start":"276'5890","ondisk_log_start":"276'5890","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48300032,"num_objects":737,"num_object_clones":0,"num_object_copies":2211,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5896,"num_write_kb":47168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"281","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.673568+0000","last_change":"2020-08-07T15:13:11.273543+0000","last_active":"2020-08-07T15:13:14.673568+0000","last_peered":"2020-08-07T15:13:14.673568+0000","last_clean":"2020-08-07T15:13:14.673568+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:13:14.673568+0000","last_undegraded":"2020-08-07T15:13:14.673568+0000","last_fullsized":"2020-08-07T15:13:14.673568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:11.273469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:13:11.273469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"341","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:15.686255+0000","last_change":"2020-08-07T15:12:53.210176+0000","last_active":"2020-08-07T15:13:15.686255+0000","last_peered":"2020-08-07T15:13:15.686255+0000","last_clean":"2020-08-07T15:13:15.686255+0000","last_became_active":"2020-08-07T15:12:32.239755+0000","last_became_peered":"2020-08-07T15:12:32.239755+0000","last_unstale":"2020-08-07T15:13:15.686255+0000","last_undegraded":"2020-08-07T15:13:15.686255+0000","last_fullsized":"2020-08-07T15:13:15.686255+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:53.210093+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_clean_scrub_stamp":"2020-08-07T15:12:53.210093+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"277'5832","reported_seq":"5884","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.858570+0000","last_change":"2020-08-07T15:12:10.398415+0000","last_active":"2020-08-07T15:13:14.858570+0000","last_peered":"2020-08-07T15:13:14.858570+0000","last_clean":"2020-08-07T15:13:14.858570+0000","last_became_active":"2020-08-07T15:12:10.397834+0000","last_became_peered":"2020-08-07T15:12:10.397834+0000","last_unstale":"2020-08-07T15:13:14.858570+0000","last_undegraded":"2020-08-07T15:13:14.858570+0000","last_fullsized":"2020-08-07T15:13:14.858570+0000","mapping_epoch":230,"log_start":"277'5825","ondisk_log_start":"277'5825","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47775744,"num_objects":729,"num_object_clones":0,"num_object_copies":2187,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5832,"num_write_kb":46656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"250","reported_epoch":"276","state":"active+clean","last_fresh":"2020-08-07T15:13:13.684883+0000","last_change":"2020-08-07T15:13:02.591821+0000","last_active":"2020-08-07T15:13:13.684883+0000","last_peered":"2020-08-07T15:13:13.684883+0000","last_clean":"2020-08-07T15:13:13.684883+0000","last_became_active":"2020-08-07T15:13:02.591402+0000","last_became_peered":"2020-08-07T15:13:02.591402+0000","last_unstale":"2020-08-07T15:13:13.684883+0000","last_undegraded":"2020-08-07T15:13:13.684883+0000","last_fullsized":"2020-08-07T15:13:13.684883+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:08.622683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:08.622683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"310","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:15.686144+0000","last_change":"2020-08-07T15:12:42.371309+0000","last_active":"2020-08-07T15:13:15.686144+0000","last_peered":"2020-08-07T15:13:15.686144+0000","last_clean":"2020-08-07T15:13:15.686144+0000","last_became_active":"2020-08-07T15:12:32.237984+0000","last_became_peered":"2020-08-07T15:12:32.237984+0000","last_unstale":"2020-08-07T15:13:15.686144+0000","last_undegraded":"2020-08-07T15:13:15.686144+0000","last_fullsized":"2020-08-07T15:13:15.686144+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:42.371229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:42.371229+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.7","version":"278'6080","reported_seq":"6136","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.717430+0000","last_change":"2020-08-07T15:12:10.464810+0000","last_active":"2020-08-07T15:13:16.717430+0000","last_peered":"2020-08-07T15:13:16.717430+0000","last_clean":"2020-08-07T15:13:16.717430+0000","last_became_active":"2020-08-07T15:12:10.464050+0000","last_became_peered":"2020-08-07T15:12:10.464050+0000","last_unstale":"2020-08-07T15:13:16.717430+0000","last_undegraded":"2020-08-07T15:13:16.717430+0000","last_fullsized":"2020-08-07T15:13:16.717430+0000","mapping_epoch":230,"log_start":"278'6075","ondisk_log_start":"278'6075","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49807360,"num_objects":760,"num_object_clones":0,"num_object_copies":2280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6080,"num_write_kb":48640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"36","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:15.685876+0000","last_change":"2020-08-07T15:13:13.399410+0000","last_active":"2020-08-07T15:13:15.685876+0000","last_peered":"2020-08-07T15:13:15.685876+0000","last_clean":"2020-08-07T15:13:15.685876+0000","last_became_active":"2020-08-07T15:13:02.590656+0000","last_became_peered":"2020-08-07T15:13:02.590656+0000","last_unstale":"2020-08-07T15:13:15.685876+0000","last_undegraded":"2020-08-07T15:13:15.685876+0000","last_fullsized":"2020-08-07T15:13:15.685876+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:13.399327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:13.399327+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"278","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670917+0000","last_change":"2020-08-07T15:11:40.328534+0000","last_active":"2020-08-07T15:13:14.670917+0000","last_peered":"2020-08-07T15:13:14.670917+0000","last_clean":"2020-08-07T15:13:14.670917+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:13:14.670917+0000","last_undegraded":"2020-08-07T15:13:14.670917+0000","last_fullsized":"2020-08-07T15:13:14.670917+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:11:40.328482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:11:40.328482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"279'6192","reported_seq":"6251","reported_epoch":"279","state":"active+clean","last_fresh":"2020-08-07T15:13:17.651270+0000","last_change":"2020-08-07T15:13:02.591551+0000","last_active":"2020-08-07T15:13:17.651270+0000","last_peered":"2020-08-07T15:13:17.651270+0000","last_clean":"2020-08-07T15:13:17.651270+0000","last_became_active":"2020-08-07T15:13:02.591000+0000","last_became_peered":"2020-08-07T15:13:02.591000+0000","last_unstale":"2020-08-07T15:13:17.651270+0000","last_undegraded":"2020-08-07T15:13:17.651270+0000","last_fullsized":"2020-08-07T15:13:17.651270+0000","mapping_epoch":269,"log_start":"279'6185","ondisk_log_start":"279'6185","created":230,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50724864,"num_objects":774,"num_object_clones":0,"num_object_copies":2322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6192,"num_write_kb":49536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"299","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.683614+0000","last_change":"2020-08-07T15:12:10.585667+0000","last_active":"2020-08-07T15:13:14.683614+0000","last_peered":"2020-08-07T15:13:14.683614+0000","last_clean":"2020-08-07T15:13:14.683614+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:13:14.683614+0000","last_undegraded":"2020-08-07T15:13:14.683614+0000","last_fullsized":"2020-08-07T15:13:14.683614+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:10.585580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:10.585580+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"338","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.673444+0000","last_change":"2020-08-07T15:12:38.850878+0000","last_active":"2020-08-07T15:13:14.673444+0000","last_peered":"2020-08-07T15:13:14.673444+0000","last_clean":"2020-08-07T15:13:14.673444+0000","last_became_active":"2020-08-07T15:12:32.239794+0000","last_became_peered":"2020-08-07T15:12:32.239794+0000","last_unstale":"2020-08-07T15:13:14.673444+0000","last_undegraded":"2020-08-07T15:13:14.673444+0000","last_fullsized":"2020-08-07T15:13:14.673444+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:38.850809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:38.850809+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"278'6136","reported_seq":"6192","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.746670+0000","last_change":"2020-08-07T15:12:10.465113+0000","last_active":"2020-08-07T15:13:16.746670+0000","last_peered":"2020-08-07T15:13:16.746670+0000","last_clean":"2020-08-07T15:13:16.746670+0000","last_became_active":"2020-08-07T15:12:10.464582+0000","last_became_peered":"2020-08-07T15:12:10.464582+0000","last_unstale":"2020-08-07T15:13:16.746670+0000","last_undegraded":"2020-08-07T15:13:16.746670+0000","last_fullsized":"2020-08-07T15:13:16.746670+0000","mapping_epoch":230,"log_start":"278'6131","ondisk_log_start":"278'6131","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50266112,"num_objects":767,"num_object_clones":0,"num_object_copies":2301,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6136,"num_write_kb":49088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"293","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670935+0000","last_change":"2020-08-07T15:12:07.340958+0000","last_active":"2020-08-07T15:13:14.670935+0000","last_peered":"2020-08-07T15:13:14.670935+0000","last_clean":"2020-08-07T15:13:14.670935+0000","last_became_active":"2020-08-07T15:12:06.366967+0000","last_became_peered":"2020-08-07T15:12:06.366967+0000","last_unstale":"2020-08-07T15:13:14.670935+0000","last_undegraded":"2020-08-07T15:13:14.670935+0000","last_fullsized":"2020-08-07T15:13:14.670935+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:07.340855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.340855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"285","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670406+0000","last_change":"2020-08-07T15:12:32.247332+0000","last_active":"2020-08-07T15:13:14.670406+0000","last_peered":"2020-08-07T15:13:14.670406+0000","last_clean":"2020-08-07T15:13:14.670406+0000","last_became_active":"2020-08-07T15:12:32.246800+0000","last_became_peered":"2020-08-07T15:12:32.246800+0000","last_unstale":"2020-08-07T15:13:14.670406+0000","last_undegraded":"2020-08-07T15:13:14.670406+0000","last_fullsized":"2020-08-07T15:13:14.670406+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:09.261101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.261101+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"278'5976","reported_seq":"6032","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.677019+0000","last_change":"2020-08-07T15:12:10.466261+0000","last_active":"2020-08-07T15:13:16.677019+0000","last_peered":"2020-08-07T15:13:16.677019+0000","last_clean":"2020-08-07T15:13:16.677019+0000","last_became_active":"2020-08-07T15:12:10.465583+0000","last_became_peered":"2020-08-07T15:12:10.465583+0000","last_unstale":"2020-08-07T15:13:16.677019+0000","last_undegraded":"2020-08-07T15:13:16.677019+0000","last_fullsized":"2020-08-07T15:13:16.677019+0000","mapping_epoch":230,"log_start":"278'5967","ondisk_log_start":"278'5967","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48955392,"num_objects":747,"num_object_clones":0,"num_object_copies":2241,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5976,"num_write_kb":47808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"247","reported_epoch":"276","state":"active+clean","last_fresh":"2020-08-07T15:13:13.685321+0000","last_change":"2020-08-07T15:13:02.591378+0000","last_active":"2020-08-07T15:13:13.685321+0000","last_peered":"2020-08-07T15:13:13.685321+0000","last_clean":"2020-08-07T15:13:13.685321+0000","last_became_active":"2020-08-07T15:13:02.590901+0000","last_became_peered":"2020-08-07T15:13:02.590901+0000","last_unstale":"2020-08-07T15:13:13.685321+0000","last_undegraded":"2020-08-07T15:13:13.685321+0000","last_fullsized":"2020-08-07T15:13:13.685321+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:07.583152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.583152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"375","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:15.686363+0000","last_change":"2020-08-07T15:12:43.351056+0000","last_active":"2020-08-07T15:13:15.686363+0000","last_peered":"2020-08-07T15:13:15.686363+0000","last_clean":"2020-08-07T15:13:15.686363+0000","last_became_active":"2020-08-07T15:12:32.254842+0000","last_became_peered":"2020-08-07T15:12:32.254842+0000","last_unstale":"2020-08-07T15:13:15.686363+0000","last_undegraded":"2020-08-07T15:13:15.686363+0000","last_fullsized":"2020-08-07T15:13:15.686363+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:43.350971+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:43.350971+0000","last_clean_scrub_stamp":"2020-08-07T15:12:43.350971+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"278'5704","reported_seq":"5758","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.676316+0000","last_change":"2020-08-07T15:12:10.467447+0000","last_active":"2020-08-07T15:13:16.676316+0000","last_peered":"2020-08-07T15:13:16.676316+0000","last_clean":"2020-08-07T15:13:16.676316+0000","last_became_active":"2020-08-07T15:12:10.467000+0000","last_became_peered":"2020-08-07T15:12:10.467000+0000","last_unstale":"2020-08-07T15:13:16.676316+0000","last_undegraded":"2020-08-07T15:13:16.676316+0000","last_fullsized":"2020-08-07T15:13:16.676316+0000","mapping_epoch":230,"log_start":"278'5696","ondisk_log_start":"278'5696","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":46727168,"num_objects":713,"num_object_clones":0,"num_object_copies":2139,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5704,"num_write_kb":45632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"265","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670752+0000","last_change":"2020-08-07T15:13:11.097271+0000","last_active":"2020-08-07T15:13:14.670752+0000","last_peered":"2020-08-07T15:13:14.670752+0000","last_clean":"2020-08-07T15:13:14.670752+0000","last_became_active":"2020-08-07T15:10:46.498230+0000","last_became_peered":"2020-08-07T15:10:46.498230+0000","last_unstale":"2020-08-07T15:13:14.670752+0000","last_undegraded":"2020-08-07T15:13:14.670752+0000","last_fullsized":"2020-08-07T15:13:14.670752+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:11.097214+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:13:11.097214+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"278'5976","reported_seq":"6032","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.640275+0000","last_change":"2020-08-07T15:12:10.465075+0000","last_active":"2020-08-07T15:13:16.640275+0000","last_peered":"2020-08-07T15:13:16.640275+0000","last_clean":"2020-08-07T15:13:16.640275+0000","last_became_active":"2020-08-07T15:12:10.464640+0000","last_became_peered":"2020-08-07T15:12:10.464640+0000","last_unstale":"2020-08-07T15:13:16.640275+0000","last_undegraded":"2020-08-07T15:13:16.640275+0000","last_fullsized":"2020-08-07T15:13:16.640275+0000","mapping_epoch":230,"log_start":"278'5969","ondisk_log_start":"278'5969","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48955392,"num_objects":747,"num_object_clones":0,"num_object_copies":2241,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5976,"num_write_kb":47808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"121","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670174+0000","last_change":"2020-08-07T15:12:45.243933+0000","last_active":"2020-08-07T15:13:14.670174+0000","last_peered":"2020-08-07T15:13:14.670174+0000","last_clean":"2020-08-07T15:13:14.670174+0000","last_became_active":"2020-08-07T15:12:45.243349+0000","last_became_peered":"2020-08-07T15:12:45.243349+0000","last_unstale":"2020-08-07T15:13:14.670174+0000","last_undegraded":"2020-08-07T15:13:14.670174+0000","last_fullsized":"2020-08-07T15:13:14.670174+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.490655+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.490655+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.e","version":"277'5536","reported_seq":"5593","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.834720+0000","last_change":"2020-08-07T15:13:12.351009+0000","last_active":"2020-08-07T15:13:14.834720+0000","last_peered":"2020-08-07T15:13:14.834720+0000","last_clean":"2020-08-07T15:13:14.834720+0000","last_became_active":"2020-08-07T15:12:10.464900+0000","last_became_peered":"2020-08-07T15:12:10.464900+0000","last_unstale":"2020-08-07T15:13:14.834720+0000","last_undegraded":"2020-08-07T15:13:14.834720+0000","last_fullsized":"2020-08-07T15:13:14.834720+0000","mapping_epoch":230,"log_start":"277'5532","ondisk_log_start":"277'5532","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"275'5310","last_scrub_stamp":"2020-08-07T15:13:12.350955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:13:12.350955+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":45350912,"num_objects":692,"num_object_clones":0,"num_object_copies":2076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5536,"num_write_kb":44288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"83","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.674040+0000","last_change":"2020-08-07T15:13:10.415039+0000","last_active":"2020-08-07T15:13:14.674040+0000","last_peered":"2020-08-07T15:13:14.674040+0000","last_clean":"2020-08-07T15:13:14.674040+0000","last_became_active":"2020-08-07T15:13:02.590018+0000","last_became_peered":"2020-08-07T15:13:02.590018+0000","last_unstale":"2020-08-07T15:13:14.674040+0000","last_undegraded":"2020-08-07T15:13:14.674040+0000","last_fullsized":"2020-08-07T15:13:14.674040+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:10.414954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:10.414954+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"279'6272","reported_seq":"6332","reported_epoch":"279","state":"active+remapped+backfill_toofull","last_fresh":"2020-08-07T15:13:17.720161+0000","last_change":"2020-08-07T15:12:57.279725+0000","last_active":"2020-08-07T15:13:17.720161+0000","last_peered":"2020-08-07T15:13:17.720161+0000","last_clean":"2020-08-07T15:12:55.240424+0000","last_became_active":"2020-08-07T15:12:57.254078+0000","last_became_peered":"2020-08-07T15:12:57.254078+0000","last_unstale":"2020-08-07T15:13:17.720161+0000","last_undegraded":"2020-08-07T15:13:17.720161+0000","last_fullsized":"2020-08-07T15:13:17.720161+0000","mapping_epoch":266,"log_start":"279'6264","ondisk_log_start":"279'6264","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":51380224,"num_objects":784,"num_object_clones":0,"num_object_copies":2352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":784,"num_objects_unfound":0,"num_objects_dirty":784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6272,"num_write_kb":50176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,0],"avail_no_missing":["6","0","5"],"object_location_counts":[{"shards":"0,5,6","objects":784}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"296","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.673462+0000","last_change":"2020-08-07T15:12:13.680022+0000","last_active":"2020-08-07T15:13:14.673462+0000","last_peered":"2020-08-07T15:13:14.673462+0000","last_clean":"2020-08-07T15:13:14.673462+0000","last_became_active":"2020-08-07T15:12:06.380392+0000","last_became_peered":"2020-08-07T15:12:06.380392+0000","last_unstale":"2020-08-07T15:13:14.673462+0000","last_undegraded":"2020-08-07T15:13:14.673462+0000","last_fullsized":"2020-08-07T15:13:14.673462+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:13.679943+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.679943+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"278'5877","reported_seq":"5931","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.761337+0000","last_change":"2020-08-07T15:12:10.398816+0000","last_active":"2020-08-07T15:13:16.761337+0000","last_peered":"2020-08-07T15:13:16.761337+0000","last_clean":"2020-08-07T15:13:16.761337+0000","last_became_active":"2020-08-07T15:12:10.398332+0000","last_became_peered":"2020-08-07T15:12:10.398332+0000","last_unstale":"2020-08-07T15:13:16.761337+0000","last_undegraded":"2020-08-07T15:13:16.761337+0000","last_fullsized":"2020-08-07T15:13:16.761337+0000","mapping_epoch":230,"log_start":"278'5872","ondisk_log_start":"278'5872","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48144384,"num_objects":735,"num_object_clones":0,"num_object_copies":2205,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5877,"num_write_kb":47016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"251","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.674265+0000","last_change":"2020-08-07T15:12:14.241339+0000","last_active":"2020-08-07T15:13:14.674265+0000","last_peered":"2020-08-07T15:13:14.674265+0000","last_clean":"2020-08-07T15:13:14.674265+0000","last_became_active":"2020-08-07T15:10:46.498051+0000","last_became_peered":"2020-08-07T15:10:46.498051+0000","last_unstale":"2020-08-07T15:13:14.674265+0000","last_undegraded":"2020-08-07T15:13:14.674265+0000","last_fullsized":"2020-08-07T15:13:14.674265+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:14.241262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:14.241262+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"278'5968","reported_seq":"6025","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.729214+0000","last_change":"2020-08-07T15:13:02.590816+0000","last_active":"2020-08-07T15:13:16.729214+0000","last_peered":"2020-08-07T15:13:16.729214+0000","last_clean":"2020-08-07T15:13:16.729214+0000","last_became_active":"2020-08-07T15:13:02.590300+0000","last_became_peered":"2020-08-07T15:13:02.590300+0000","last_unstale":"2020-08-07T15:13:16.729214+0000","last_undegraded":"2020-08-07T15:13:16.729214+0000","last_fullsized":"2020-08-07T15:13:16.729214+0000","mapping_epoch":269,"log_start":"278'5960","ondisk_log_start":"278'5960","created":230,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48889856,"num_objects":746,"num_object_clones":0,"num_object_copies":2238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5968,"num_write_kb":47744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3],"acting":[2,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"328","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:15.686300+0000","last_change":"2020-08-07T15:12:41.445456+0000","last_active":"2020-08-07T15:13:15.686300+0000","last_peered":"2020-08-07T15:13:15.686300+0000","last_clean":"2020-08-07T15:13:15.686300+0000","last_became_active":"2020-08-07T15:12:32.241722+0000","last_became_peered":"2020-08-07T15:12:32.241722+0000","last_unstale":"2020-08-07T15:13:15.686300+0000","last_undegraded":"2020-08-07T15:13:15.686300+0000","last_fullsized":"2020-08-07T15:13:15.686300+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:41.445379+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_clean_scrub_stamp":"2020-08-07T15:12:41.445379+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"278'5774","reported_seq":"5827","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.726224+0000","last_change":"2020-08-07T15:13:02.596246+0000","last_active":"2020-08-07T15:13:16.726224+0000","last_peered":"2020-08-07T15:13:16.726224+0000","last_clean":"2020-08-07T15:13:16.726224+0000","last_became_active":"2020-08-07T15:13:02.591329+0000","last_became_peered":"2020-08-07T15:13:02.591329+0000","last_unstale":"2020-08-07T15:13:16.726224+0000","last_undegraded":"2020-08-07T15:13:16.726224+0000","last_fullsized":"2020-08-07T15:13:16.726224+0000","mapping_epoch":269,"log_start":"278'5768","ondisk_log_start":"278'5768","created":230,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47300608,"num_objects":722,"num_object_clones":0,"num_object_copies":2166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5774,"num_write_kb":46192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"40","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.674376+0000","last_change":"2020-08-07T15:13:11.459633+0000","last_active":"2020-08-07T15:13:14.674376+0000","last_peered":"2020-08-07T15:13:14.674376+0000","last_clean":"2020-08-07T15:13:14.674376+0000","last_became_active":"2020-08-07T15:13:02.591018+0000","last_became_peered":"2020-08-07T15:13:02.591018+0000","last_unstale":"2020-08-07T15:13:14.674376+0000","last_undegraded":"2020-08-07T15:13:14.674376+0000","last_fullsized":"2020-08-07T15:13:14.674376+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:11.459555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:11.459555+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.c","version":"278'6200","reported_seq":"6256","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.701385+0000","last_change":"2020-08-07T15:12:10.465251+0000","last_active":"2020-08-07T15:13:16.701385+0000","last_peered":"2020-08-07T15:13:16.701385+0000","last_clean":"2020-08-07T15:13:16.701385+0000","last_became_active":"2020-08-07T15:12:10.464698+0000","last_became_peered":"2020-08-07T15:12:10.464698+0000","last_unstale":"2020-08-07T15:13:16.701385+0000","last_undegraded":"2020-08-07T15:13:16.701385+0000","last_fullsized":"2020-08-07T15:13:16.701385+0000","mapping_epoch":230,"log_start":"278'6197","ondisk_log_start":"278'6197","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50790400,"num_objects":775,"num_object_clones":0,"num_object_copies":2325,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6200,"num_write_kb":49600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"249","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670814+0000","last_change":"2020-08-07T15:12:44.203274+0000","last_active":"2020-08-07T15:13:14.670814+0000","last_peered":"2020-08-07T15:13:14.670814+0000","last_clean":"2020-08-07T15:13:14.670814+0000","last_became_active":"2020-08-07T15:12:06.367081+0000","last_became_peered":"2020-08-07T15:12:06.367081+0000","last_unstale":"2020-08-07T15:13:14.670814+0000","last_undegraded":"2020-08-07T15:13:14.670814+0000","last_fullsized":"2020-08-07T15:13:14.670814+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:44.203221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:44.203221+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":784760832,"num_objects":11976,"num_object_clones":0,"num_object_copies":35928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":784,"num_objects_unfound":0,"num_objects_dirty":11976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95796,"num_write_kb":766368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2356854784,"data_stored":2356854784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":41,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":123,"acting":123,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542230,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1296276,"kb_used_data":247692,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93075564,"statfs":{"total":96636764160,"available":95309377536,"internally_reserved":1073741824,"allocated":253636608,"data_stored":252209827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,1,0,2,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":9.5359999999999996,"15min":3.774},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":2.294,"5min":856.07100000000003,"15min":856.07100000000003},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":1.002,"15min":0.97599999999999998},"min":{"1min":0.52300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.55,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":0.84099999999999997}]},{"osd":1,"last update":"Fri Aug 7 15:13:10 2020","interfaces":[{"interface":"back","average":{"1min":10.148999999999999,"5min":10.148999999999999,"15min":10.148999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":205.73699999999999,"5min":205.73699999999999,"15min":205.73699999999999},"last":2.2309999999999999},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":3.827,"5min":3.827,"15min":3.827},"last":1.141}]},{"osd":2,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.321,"15min":1.1910000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.5630000000000002,"5min":5.6619999999999999,"15min":5.6619999999999999},"last":1.675},{"interface":"front","average":{"1min":1.081,"5min":1.3280000000000001,"15min":1.1879999999999999},"min":{"1min":0.69299999999999995,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.3769999999999998,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":1.1719999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":30.957999999999998,"5min":21.91,"15min":8.4179999999999993},"min":{"1min":0.68899999999999995,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":656.99099999999999,"5min":917.38699999999994,"15min":917.38699999999994},"last":1.946},{"interface":"front","average":{"1min":1.173,"5min":4.8879999999999999,"15min":2.524},"min":{"1min":0.61599999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":4.0460000000000003,"5min":333.59199999999998,"15min":333.59199999999998},"last":1.5940000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":8.8599999999999994,"5min":2.944,"15min":1.8320000000000001},"min":{"1min":0.66000000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":172.012,"5min":172.012,"15min":172.012},"last":1.196},{"interface":"front","average":{"1min":1.5509999999999999,"5min":11.529,"15min":4.7169999999999996},"min":{"1min":0.81200000000000006,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":5.4500000000000002,"5min":866.24099999999999,"15min":866.24099999999999},"last":0.86799999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":9.3930000000000007,"15min":28.135000000000002},"min":{"1min":0.68400000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":3.536,"5min":678.76099999999997,"15min":831.88099999999997},"last":2.0299999999999998},{"interface":"front","average":{"1min":1.623,"5min":1.831,"15min":1.607},"min":{"1min":0.89000000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":3.593,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":2.056}]},{"osd":7,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":3.9929999999999999,"15min":2.343},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":4.0810000000000004,"5min":224.14500000000001,"15min":224.14500000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.671,"5min":1.7649999999999999,"15min":1.593},"min":{"1min":0.92000000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":3.6200000000000001,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":1.99}]}]},{"osd":7,"up_from":13,"seq":55834574935,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1331576,"kb_used_data":282992,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93040264,"statfs":{"total":96636764160,"available":95273230336,"internally_reserved":1073741824,"allocated":289783808,"data_stored":288364486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":37.372999999999998,"5min":16.672000000000001,"15min":6.4459999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":763.39099999999996,"5min":763.39099999999996,"15min":763.39099999999996},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.157,"5min":9.7789999999999999,"15min":4.0259999999999998},"min":{"1min":0.61699999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.8279999999999998,"5min":529.88699999999994,"15min":529.88699999999994},"last":1.1299999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:13:05 2020","interfaces":[{"interface":"back","average":{"1min":23.431000000000001,"5min":23.431000000000001,"15min":23.431000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":472.59699999999998,"5min":472.59699999999998,"15min":472.59699999999998},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.8650000000000002,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.3109999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":3.0009999999999999,"5min":1.6850000000000001,"15min":1.3580000000000001},"min":{"1min":0.52000000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":33.131999999999998,"5min":33.131999999999998,"15min":33.131999999999998},"last":0.84899999999999998},{"interface":"front","average":{"1min":2.8410000000000002,"5min":1.494,"15min":1.3460000000000001},"min":{"1min":0.61799999999999999,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":32.924999999999997,"5min":32.924999999999997,"15min":32.924999999999997},"last":1.169}]},{"osd":3,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":9.4659999999999993,"15min":4.1719999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":5.9329999999999998,"5min":698.19899999999996,"15min":698.19899999999996},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.238,"5min":1.2669999999999999,"15min":1.347},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":3.8599999999999999,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":1.903}]},{"osd":4,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":25.448,"5min":6.069,"15min":3.1309999999999998},"min":{"1min":0.76400000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":460.78399999999999,"5min":460.78399999999999,"15min":460.78399999999999},"last":1.347},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.474,"15min":1.6060000000000001},"min":{"1min":0.86899999999999999,"5min":0.621,"15min":0.621},"max":{"1min":2.4079999999999999,"5min":3.694,"15min":5.2990000000000004},"last":668.06799999999998}]},{"osd":5,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.3720000000000001,"15min":1.3660000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":5.9989999999999997,"5min":6.2080000000000002,"15min":6.2080000000000002},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.492,"15min":1.6739999999999999},"min":{"1min":0.83599999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":4.407,"5min":4.407,"15min":5.327},"last":1.3779999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":9.1839999999999993,"15min":4.016},"min":{"1min":0.88,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.3290000000000002,"5min":695.92899999999997,"15min":695.92899999999997},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.597,"5min":1.55,"15min":12.734},"min":{"1min":0.89500000000000002,"5min":0.87,"15min":0.87},"max":{"1min":3.302,"5min":3.7309999999999999,"15min":266.26999999999998},"last":1.4299999999999999}]}]},{"osd":1,"up_from":227,"seq":974957576207,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1275548,"kb_used_data":226964,"kb_used_omap":2,"kb_used_meta":1048573,"kb_avail":93096292,"statfs":{"total":96636764160,"available":95330603008,"internally_reserved":1073741824,"allocated":232411136,"data_stored":231010265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":1073739245},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":27.861000000000001,"5min":27.861000000000001,"15min":27.861000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":540.92499999999995,"5min":540.92499999999995,"15min":540.92499999999995},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":3.7069999999999999,"5min":3.7069999999999999,"15min":3.7069999999999999},"last":1.1910000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":2.5779999999999998},{"interface":"front","average":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":3.758,"5min":3.758,"15min":3.758},"last":3.1760000000000002}]},{"osd":3,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":13.909000000000001,"5min":13.909000000000001,"15min":13.909000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":250.995,"5min":250.995,"15min":250.995},"last":2.5510000000000002},{"interface":"front","average":{"1min":2.04,"5min":2.04,"15min":2.04},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":6.6580000000000004,"5min":6.6580000000000004,"15min":6.6580000000000004},"last":2.9990000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":2.6230000000000002},{"interface":"front","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.746}]},{"osd":5,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":2.0150000000000001},{"interface":"front","average":{"1min":1.498,"5min":1.498,"15min":1.498},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.6720000000000002,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":2.665}]},{"osd":6,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.1019999999999999},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":9.4299999999999997,"5min":9.4299999999999997,"15min":9.4299999999999997},"last":2.9630000000000001},{"interface":"front","average":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":3.4199999999999999,"5min":3.4199999999999999,"15min":3.4199999999999999},"last":2.5299999999999998}]},{"osd":7,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":40.520000000000003,"5min":40.520000000000003,"15min":40.520000000000003},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":778.745,"5min":778.745,"15min":778.745},"last":3.0529999999999999},{"interface":"front","average":{"1min":24.100000000000001,"5min":24.100000000000001,"15min":24.100000000000001},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":442.62799999999999,"5min":442.62799999999999,"15min":442.62799999999999},"last":3.0299999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574935,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1236688,"kb_used_data":188104,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93135152,"statfs":{"total":96636764160,"available":95370395648,"internally_reserved":1073741824,"allocated":192618496,"data_stored":191199174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:13:07 2020","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.468,"5min":2.468,"15min":2.468},"last":3.5249999999999999},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.5550000000000002,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":2.9460000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.0620000000000001,"15min":0.97199999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":7.3470000000000004,"5min":7.3470000000000004,"15min":7.3470000000000004},"last":3.6099999999999999},{"interface":"front","average":{"1min":6.2839999999999998,"5min":4.665,"15min":23.381},"min":{"1min":0.47399999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":75.622,"5min":226.911,"15min":606.18499999999995},"last":2.2709999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":28.079000000000001,"5min":6.6920000000000002,"15min":2.8540000000000001},"min":{"1min":0.60999999999999999,"5min":0.54100000000000004,"15min":0.52300000000000002},"max":{"1min":431.95299999999997,"5min":431.95299999999997,"15min":431.95299999999997},"last":4.1619999999999999},{"interface":"front","average":{"1min":1.425,"5min":1.46,"15min":1.2649999999999999},"min":{"1min":0.66400000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":5.3780000000000001,"5min":11.19,"15min":11.19},"last":4.3449999999999998}]},{"osd":4,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.302,"15min":1.2110000000000001},"min":{"1min":0.57399999999999995,"5min":0.501,"15min":0.501},"max":{"1min":1.819,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.304,"5min":1.448,"15min":1.321},"min":{"1min":0.71099999999999997,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.8849999999999998,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":3.5579999999999998}]},{"osd":5,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.6200000000000001,"5min":14.598000000000001,"15min":5.6840000000000002},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":7.3799999999999999,"5min":539.31100000000004,"15min":539.31100000000004},"last":2.6259999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.49,"15min":1.234},"min":{"1min":0.81399999999999995,"5min":0.752,"15min":0.65900000000000003},"max":{"1min":2.9420000000000002,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":0.78700000000000003}]},{"osd":6,"last update":"Fri Aug 7 15:12:34 2020","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.4399999999999999,"15min":6.2220000000000004},"min":{"1min":0.80200000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.8610000000000002,"5min":4.1310000000000002,"15min":124.738},"last":3.585},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.5549999999999999,"15min":12.952999999999999},"min":{"1min":0.73499999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":5.274,"5min":5.274,"15min":294.94},"last":2.6629999999999998}]},{"osd":7,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.4790000000000001,"15min":1.55},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.79300000000000004},"max":{"1min":1.8620000000000001,"5min":4.7169999999999996,"15min":6.468},"last":2.7149999999999999},{"interface":"front","average":{"1min":66.063000000000002,"5min":23.518000000000001,"15min":8.7629999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":971.12699999999995,"5min":971.12699999999995,"15min":971.12699999999995},"last":2.6890000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574935,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1432128,"kb_used_data":383544,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92939712,"statfs":{"total":96636764160,"available":95170265088,"internally_reserved":1073741824,"allocated":392749056,"data_stored":391329734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":32.965000000000003,"5min":7.8639999999999999,"15min":3.3660000000000001},"min":{"1min":0.41899999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":501.53800000000001,"5min":501.53800000000001,"15min":501.53800000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":27.341999999999999,"5min":6.6900000000000004,"15min":2.8799999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":501.36000000000001,"5min":501.36000000000001,"15min":501.36000000000001},"last":0.60199999999999998}]},{"osd":1,"last update":"Fri Aug 7 15:13:07 2020","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":5.7869999999999999,"5min":5.7869999999999999,"15min":5.7869999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":24.699999999999999,"5min":24.699999999999999,"15min":24.699999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":465.69200000000001,"5min":465.69200000000001,"15min":465.69200000000001},"last":0.88400000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.2130000000000001,"15min":1.179},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":23.064,"5min":15.281000000000001,"15min":6.1310000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":412.61500000000001,"5min":501.75599999999997,"15min":501.75599999999997},"last":0.748}]},{"osd":4,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.5940000000000001,"15min":1.2709999999999999},"min":{"1min":0.67400000000000004,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.712,"5min":19.783000000000001,"15min":19.783000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.4610000000000001,"15min":1.5820000000000001},"min":{"1min":0.68999999999999995,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8979999999999999,"5min":8.048,"15min":8.048},"last":0.95799999999999996}]},{"osd":5,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.3460000000000001,"15min":1.29},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.4009999999999998,"5min":5.2220000000000004,"15min":5.2220000000000004},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.4990000000000001,"5min":15.715,"15min":22.254999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":8.0649999999999995,"5min":627.53300000000002,"15min":627.53300000000002},"last":0.93500000000000005}]},{"osd":6,"last update":"Fri Aug 7 15:12:37 2020","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":9.9480000000000004,"15min":4.1299999999999999},"min":{"1min":0.875,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.9450000000000001,"5min":821.91899999999998,"15min":821.91899999999998},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.3959999999999999,"5min":9.9580000000000002,"15min":4.2000000000000002},"min":{"1min":0.70199999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.9540000000000002,"5min":751.70000000000005,"15min":751.70000000000005},"last":0.97999999999999998}]},{"osd":7,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":12.522,"15min":5.335},"min":{"1min":0.85599999999999998,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.3519999999999999,"5min":501.91699999999997,"15min":501.91699999999997},"last":1.113},{"interface":"front","average":{"1min":1.7230000000000001,"5min":2.1080000000000001,"15min":1.7},"min":{"1min":0.88500000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":7.2149999999999999,"5min":35.646999999999998,"15min":35.646999999999998},"last":1.143}]}]},{"osd":3,"up_from":13,"seq":55834574935,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1535456,"kb_used_data":486872,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92836384,"statfs":{"total":96636764160,"available":95064457216,"internally_reserved":1073741824,"allocated":498556928,"data_stored":497137606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,5,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.226,"15min":1.244},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":4.6580000000000004,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.2050000000000001,"15min":1.4930000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":4.6349999999999998,"5min":4.9119999999999999,"15min":13.067},"last":1.157}]},{"osd":1,"last update":"Fri Aug 7 15:13:08 2020","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.639,"15min":1.639},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":4.6890000000000001,"5min":4.6890000000000001,"15min":4.6890000000000001},"last":1.234},{"interface":"front","average":{"1min":1.841,"5min":1.841,"15min":1.841},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":5.0330000000000004,"5min":5.0330000000000004,"15min":5.0330000000000004},"last":3.9460000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.365,"15min":1.22},"min":{"1min":0.65900000000000003,"5min":0.56999999999999995,"15min":0.56899999999999995},"max":{"1min":2.8300000000000001,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":4.181},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.9690000000000001,"15min":1.7789999999999999},"min":{"1min":0.64500000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":5.3090000000000002,"5min":51.530000000000001,"15min":51.530000000000001},"last":1.8580000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":3.831,"5min":17.792999999999999,"15min":7.0620000000000003},"min":{"1min":0.54500000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":56.688000000000002,"5min":646.26700000000005,"15min":646.26700000000005},"last":5.2720000000000002},{"interface":"front","average":{"1min":1.421,"5min":15.494999999999999,"15min":26.565000000000001},"min":{"1min":0.68100000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":3.2109999999999999,"5min":844.10500000000002,"15min":844.10500000000002},"last":3.988}]},{"osd":5,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":14.452999999999999,"5min":8.5429999999999993,"15min":3.859},"min":{"1min":0.95499999999999996,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":244.08099999999999,"5min":403.88400000000001,"15min":403.88400000000001},"last":4.1429999999999998},{"interface":"front","average":{"1min":1.78,"5min":1.8979999999999999,"15min":1.8120000000000001},"min":{"1min":0.98499999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":4.5960000000000001,"5min":8.9120000000000008,"15min":8.9120000000000008},"last":4.2409999999999997}]},{"osd":6,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.7969999999999999,"5min":1.706,"15min":1.5900000000000001},"min":{"1min":0.74099999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":5.2729999999999997,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":1.609},{"interface":"front","average":{"1min":1.907,"5min":13.411,"15min":5.7949999999999999},"min":{"1min":1.032,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":5.3460000000000001,"5min":688.15999999999997,"15min":688.15999999999997},"last":1.635}]},{"osd":7,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.7450000000000001,"5min":1.5960000000000001,"15min":1.9730000000000001},"min":{"1min":0.85599999999999998,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":4.8099999999999996,"5min":6.8209999999999997,"15min":6.8209999999999997},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.6559999999999999,"15min":1.8700000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.6099999999999999,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":5.3109999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574935,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1293584,"kb_used_data":245000,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93078256,"statfs":{"total":96636764160,"available":95312134144,"internally_reserved":1073741824,"allocated":250880000,"data_stored":249460678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":49.795000000000002,"5min":19.646000000000001,"15min":7.3360000000000003},"min":{"1min":0.46800000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":829.15099999999995,"5min":829.15099999999995,"15min":829.15099999999995},"last":0.58799999999999997},{"interface":"front","average":{"1min":3.8149999999999999,"5min":1.639,"15min":1.1120000000000001},"min":{"1min":0.45300000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":48.274000000000001,"5min":48.274000000000001,"15min":48.274000000000001},"last":0.64600000000000002}]},{"osd":1,"last update":"Fri Aug 7 15:13:08 2020","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.80800000000000005}]},{"osd":2,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":4.1159999999999997,"5min":1.901,"15min":1.601},"min":{"1min":0.64300000000000002,"5min":0.56399999999999995,"15min":0.52100000000000002},"max":{"1min":49.878999999999998,"5min":49.878999999999998,"15min":49.878999999999998},"last":0.78700000000000003},{"interface":"front","average":{"1min":4.2270000000000003,"5min":2.0529999999999999,"15min":1.694},"min":{"1min":0.97899999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":49.939999999999998,"5min":49.939999999999998,"15min":49.939999999999998},"last":0.84999999999999998}]},{"osd":3,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":27.59,"5min":17.260999999999999,"15min":6.7590000000000003},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":426.50799999999998,"5min":638.92100000000005,"15min":638.92100000000005},"last":1.595},{"interface":"front","average":{"1min":1.411,"5min":1.3799999999999999,"15min":1.294},"min":{"1min":0.75700000000000001,"5min":0.67800000000000005,"15min":0.66700000000000004},"max":{"1min":3.2360000000000002,"5min":6.431,"15min":6.431},"last":1.153}]},{"osd":5,"last update":"Fri Aug 7 15:12:29 2020","interfaces":[{"interface":"back","average":{"1min":52.308999999999997,"5min":28.574999999999999,"15min":10.609999999999999},"min":{"1min":0.56100000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":862.97299999999996,"5min":990.71600000000001,"15min":990.71600000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":49.549999999999997,"5min":11.01,"15min":4.7539999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":780.00699999999995,"5min":780.00699999999995,"15min":780.00699999999995},"last":0.95599999999999996}]},{"osd":6,"last update":"Fri Aug 7 15:12:34 2020","interfaces":[{"interface":"back","average":{"1min":3.8239999999999998,"5min":1.9870000000000001,"15min":1.855},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":48.575000000000003,"5min":48.575000000000003,"15min":48.575000000000003},"last":1.179},{"interface":"front","average":{"1min":1.109,"5min":1.333,"15min":13.183},"min":{"1min":0.64600000000000002,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.847,"5min":6.7729999999999997,"15min":389.14600000000002},"last":0.92900000000000005}]},{"osd":7,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":4.1349999999999998,"5min":4.327,"15min":2.3809999999999998},"min":{"1min":0.74199999999999999,"5min":0.62,"15min":0.62},"max":{"1min":48.75,"5min":252.798,"15min":252.798},"last":1.728},{"interface":"front","average":{"1min":4.2610000000000001,"5min":14.635,"15min":5.9749999999999996},"min":{"1min":0.84799999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":49.654000000000003,"5min":501.93000000000001,"15min":501.93000000000001},"last":1.486}]}]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1289648,"kb_used_data":241064,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93082192,"statfs":{"total":96636764160,"available":95316164608,"internally_reserved":1073741824,"allocated":246849536,"data_stored":245430214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":3.629,"5min":1.506,"15min":1.22},"min":{"1min":0.42199999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":43.292999999999999,"5min":43.292999999999999,"15min":43.292999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":3.1520000000000001,"5min":4.4249999999999998,"15min":2.0110000000000001},"min":{"1min":0.44700000000000001,"5min":0.438,"15min":0.438},"max":{"1min":42.933,"5min":329.11399999999998,"15min":329.11399999999998},"last":2.5049999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:13:11 2020","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":4.4039999999999999,"5min":4.4039999999999999,"15min":4.4039999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":4.0810000000000004,"5min":4.0810000000000004,"15min":4.0810000000000004},"last":0.95899999999999996}]},{"osd":2,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":3.5499999999999998,"5min":1.5800000000000001,"15min":6.673},"min":{"1min":0.65700000000000003,"5min":0.499,"15min":0.499},"max":{"1min":43.923000000000002,"5min":43.923000000000002,"15min":168.56899999999999},"last":1.1180000000000001},{"interface":"front","average":{"1min":3.23,"5min":1.6399999999999999,"15min":22.634},"min":{"1min":0.52800000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":43.076999999999998,"5min":43.076999999999998,"15min":676.27099999999996},"last":2.5430000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.728,"15min":1.321},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":4.7640000000000002,"5min":29.129000000000001,"15min":29.129000000000001},"last":1.377},{"interface":"front","average":{"1min":1.3779999999999999,"5min":10.029999999999999,"15min":4.1520000000000001},"min":{"1min":0.73999999999999999,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":5.2009999999999996,"5min":712.98699999999997,"15min":712.98699999999997},"last":1.1539999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":12.779999999999999,"15min":38.188000000000002},"min":{"1min":0.77200000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":4.9969999999999999,"5min":971.67999999999995,"15min":971.67999999999995},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.2470000000000001,"15min":1.286},"min":{"1min":0.73599999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.1160000000000001,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":2.581}]},{"osd":6,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":35.393000000000001,"5min":28.459,"15min":43.091999999999999},"min":{"1min":0.72199999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":676.20000000000005,"5min":966.60900000000004,"15min":966.60900000000004},"last":1.579},{"interface":"front","average":{"1min":1.528,"5min":6.7789999999999999,"15min":19.603999999999999},"min":{"1min":0.89500000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":5.2220000000000004,"5min":501.79000000000002,"15min":517.26700000000005},"last":1.9219999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.462,"15min":1.3600000000000001},"min":{"1min":0.85399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":4.8600000000000003,"5min":5.1559999999999997,"15min":5.1559999999999997},"last":208.03399999999999},{"interface":"front","average":{"1min":1.5669999999999999,"5min":12.834,"15min":5.3220000000000001},"min":{"1min":0.70099999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":3.8999999999999999,"5min":515.09100000000001,"15min":515.09100000000001},"last":2.7069999999999999}]}]}]}} 2020-08-07T15:13:18.342 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2020-08-07T15:13:18.343 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:13:18.344 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump -f json-pretty 2020-08-07T15:13:18.349 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:18.349 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:18.373 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:13:18.387 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:18.384+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:18.487 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:18.486+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:18.596 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:18.592+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:18.630 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:18.631 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:18.644 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:13:18.691 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:18.691+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:18.792 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:18.792+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:18.809 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:13:18.810 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:18.810 INFO:teuthology.orchestra.run.smithi074.stdout: "epoch": 280, 2020-08-07T15:13:18.810 INFO:teuthology.orchestra.run.smithi074.stdout: "fsid": "a01d3e82-8d24-4288-9e42-576ab3ff4957", 2020-08-07T15:13:18.810 INFO:teuthology.orchestra.run.smithi074.stdout: "created": "2020-08-07T15:06:07.342912+0000", 2020-08-07T15:13:18.811 INFO:teuthology.orchestra.run.smithi074.stdout: "modified": "2020-08-07T15:13:18.241648+0000", 2020-08-07T15:13:18.811 INFO:teuthology.orchestra.run.smithi074.stdout: "last_up_change": "2020-08-07T15:12:05.338271+0000", 2020-08-07T15:13:18.811 INFO:teuthology.orchestra.run.smithi074.stdout: "last_in_change": "2020-08-07T15:06:17.674292+0000", 2020-08-07T15:13:18.811 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-08-07T15:13:18.811 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_num": 5806080, 2020-08-07T15:13:18.812 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_set": [ 2020-08-07T15:13:18.812 INFO:teuthology.orchestra.run.smithi074.stdout: "nodeep-scrub", 2020-08-07T15:13:18.812 INFO:teuthology.orchestra.run.smithi074.stdout: "noscrub", 2020-08-07T15:13:18.812 INFO:teuthology.orchestra.run.smithi074.stdout: "pglog_hardlimit", 2020-08-07T15:13:18.812 INFO:teuthology.orchestra.run.smithi074.stdout: "purged_snapdirs", 2020-08-07T15:13:18.813 INFO:teuthology.orchestra.run.smithi074.stdout: "recovery_deletes", 2020-08-07T15:13:18.813 INFO:teuthology.orchestra.run.smithi074.stdout: "sortbitwise" 2020-08-07T15:13:18.813 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:13:18.813 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_version": 5, 2020-08-07T15:13:18.813 INFO:teuthology.orchestra.run.smithi074.stdout: "full_ratio": 0.94999998807907104, 2020-08-07T15:13:18.814 INFO:teuthology.orchestra.run.smithi074.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-08-07T15:13:18.814 INFO:teuthology.orchestra.run.smithi074.stdout: "nearfull_ratio": 0.85000002384185791, 2020-08-07T15:13:18.814 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_snapshot": "", 2020-08-07T15:13:18.814 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_max": 6, 2020-08-07T15:13:18.814 INFO:teuthology.orchestra.run.smithi074.stdout: "max_osd": 8, 2020-08-07T15:13:18.815 INFO:teuthology.orchestra.run.smithi074.stdout: "require_min_compat_client": "luminous", 2020-08-07T15:13:18.815 INFO:teuthology.orchestra.run.smithi074.stdout: "min_compat_client": "luminous", 2020-08-07T15:13:18.815 INFO:teuthology.orchestra.run.smithi074.stdout: "require_osd_release": "pacific", 2020-08-07T15:13:18.815 INFO:teuthology.orchestra.run.smithi074.stdout: "pools": [ 2020-08-07T15:13:18.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.816 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 1, 2020-08-07T15:13:18.816 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "device_health_metrics", 2020-08-07T15:13:18.816 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:06:10.648876+0000", 2020-08-07T15:13:18.816 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:13:18.816 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:13:18.817 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:13:18.817 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:13:18.817 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:13:18.817 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:13:18.819 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:13:18.819 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:13:18.820 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 41, 2020-08-07T15:13:18.820 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 41, 2020-08-07T15:13:18.820 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 41, 2020-08-07T15:13:18.820 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 41, 2020-08-07T15:13:18.820 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 41, 2020-08-07T15:13:18.821 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:13:18.821 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:13:18.821 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:13:18.821 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:13:18.822 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:13:18.822 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:13:18.822 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:13:18.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.822 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "246", 2020-08-07T15:13:18.823 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:13:18.823 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:13:18.823 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "244", 2020-08-07T15:13:18.823 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:13:18.824 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:13:18.824 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:13:18.825 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:13:18.825 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:13:18.825 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:13:18.826 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:13:18.826 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:13:18.826 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:13:18.826 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:13:18.826 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:13:18.827 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:13:18.827 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:13:18.827 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:13:18.827 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:13:18.827 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:13:18.828 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:13:18.828 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:13:18.828 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:13:18.828 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:13:18.828 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:13:18.829 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:13:18.829 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:13:18.829 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.829 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:13:18.829 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:13:18.830 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:13:18.830 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:13:18.830 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:13:18.830 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:13:18.830 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:13:18.831 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:13:18.831 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:13:18.831 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:13:18.831 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:13:18.831 INFO:teuthology.orchestra.run.smithi074.stdout: "options": { 2020-08-07T15:13:18.832 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_min": 1 2020-08-07T15:13:18.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.832 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:13:18.832 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_devicehealth": {} 2020-08-07T15:13:18.832 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.833 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.833 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 2, 2020-08-07T15:13:18.833 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "rbd", 2020-08-07T15:13:18.833 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:06:21.474648+0000", 2020-08-07T15:13:18.834 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:13:18.834 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:13:18.834 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:13:18.834 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:13:18.835 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:13:18.835 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:13:18.835 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:13:18.835 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:13:18.835 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 8, 2020-08-07T15:13:18.836 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 8, 2020-08-07T15:13:18.836 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 8, 2020-08-07T15:13:18.836 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 8, 2020-08-07T15:13:18.836 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 8, 2020-08-07T15:13:18.836 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:13:18.846 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:13:18.846 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:13:18.846 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:13:18.847 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:13:18.847 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:13:18.847 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:13:18.847 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.847 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "19", 2020-08-07T15:13:18.848 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:13:18.848 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:13:18.848 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "0", 2020-08-07T15:13:18.848 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:13:18.848 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:13:18.849 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:13:18.849 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:13:18.849 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:13:18.849 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:13:18.849 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:13:18.850 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:13:18.850 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:13:18.850 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:13:18.850 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:13:18.850 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:13:18.851 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:13:18.851 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:13:18.851 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:13:18.851 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:13:18.851 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:13:18.852 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:13:18.852 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:13:18.852 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:13:18.852 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:13:18.852 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:13:18.853 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:13:18.854 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.854 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:13:18.854 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:13:18.854 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:13:18.855 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:13:18.855 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:13:18.855 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:13:18.855 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:13:18.855 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:13:18.856 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:13:18.856 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:13:18.856 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:13:18.856 INFO:teuthology.orchestra.run.smithi074.stdout: "options": {}, 2020-08-07T15:13:18.856 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:13:18.857 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd": {} 2020-08-07T15:13:18.857 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.857 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 6, 2020-08-07T15:13:18.858 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "unique_pool_3", 2020-08-07T15:13:18.858 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:12:08.847598+0000", 2020-08-07T15:13:18.858 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:13:18.858 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:13:18.858 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:13:18.859 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:13:18.859 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:13:18.859 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:13:18.859 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:13:18.859 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:13:18.859 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 16, 2020-08-07T15:13:18.860 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 16, 2020-08-07T15:13:18.860 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 16, 2020-08-07T15:13:18.860 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 16, 2020-08-07T15:13:18.860 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 16, 2020-08-07T15:13:18.860 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:13:18.861 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:13:18.861 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:13:18.861 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:13:18.861 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:13:18.861 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:13:18.862 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:13:18.862 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.862 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "232", 2020-08-07T15:13:18.862 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:13:18.862 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:13:18.863 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "0", 2020-08-07T15:13:18.863 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:13:18.863 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:13:18.863 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:13:18.863 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:13:18.863 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:13:18.864 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:13:18.864 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:13:18.864 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:13:18.864 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:13:18.864 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:13:18.865 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:13:18.865 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:13:18.865 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:13:18.865 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:13:18.865 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:13:18.866 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:13:18.866 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:13:18.866 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:13:18.866 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:13:18.866 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:13:18.867 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:13:18.867 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:13:18.867 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:13:18.867 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.867 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:13:18.868 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:13:18.868 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:13:18.868 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:13:18.868 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:13:18.868 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:13:18.868 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:13:18.869 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:13:18.869 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:13:18.869 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:13:18.869 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:13:18.869 INFO:teuthology.orchestra.run.smithi074.stdout: "options": {}, 2020-08-07T15:13:18.870 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:13:18.870 INFO:teuthology.orchestra.run.smithi074.stdout: "rados": {} 2020-08-07T15:13:18.870 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.870 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.870 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:13:18.871 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:13:18.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.871 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 0, 2020-08-07T15:13:18.871 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "fc1ebb23-3ef4-4fc0-b395-382e385885a3", 2020-08-07T15:13:18.871 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:13:18.872 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:13:18.872 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:13:18.872 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0, 2020-08-07T15:13:18.872 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:13:18.872 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:13:18.872 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:13:18.873 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 262, 2020-08-07T15:13:18.873 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:13:18.873 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:13:18.873 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:13:18.873 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.874 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.874 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.874 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6801", 2020-08-07T15:13:18.874 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:13:18.874 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.875 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.875 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:13:18.875 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.876 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.876 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6802", 2020-08-07T15:13:18.876 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:13:18.876 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.876 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.876 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.877 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:13:18.877 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.877 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.877 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.878 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6804", 2020-08-07T15:13:18.878 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:13:18.878 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.878 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.878 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.879 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:13:18.879 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.879 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.879 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6803", 2020-08-07T15:13:18.880 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:13:18.880 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.880 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.880 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.881 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6801/31577", 2020-08-07T15:13:18.881 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6802/31577", 2020-08-07T15:13:18.881 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6804/31577", 2020-08-07T15:13:18.881 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6803/31577", 2020-08-07T15:13:18.881 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:13:18.882 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:13:18.882 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:13:18.882 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.882 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.882 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.883 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 1, 2020-08-07T15:13:18.883 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "1fbe530a-63ce-4dc8-82ab-18793d6ebc0b", 2020-08-07T15:13:18.883 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:13:18.883 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:13:18.883 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:13:18.884 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0.7196044921875, 2020-08-07T15:13:18.884 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 13, 2020-08-07T15:13:18.884 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 190, 2020-08-07T15:13:18.884 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 227, 2020-08-07T15:13:18.884 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 269, 2020-08-07T15:13:18.884 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 191, 2020-08-07T15:13:18.885 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:13:18.885 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:13:18.885 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.885 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.886 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6809", 2020-08-07T15:13:18.886 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 62587 2020-08-07T15:13:18.886 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.886 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.887 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:13:18.887 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.887 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.887 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6810", 2020-08-07T15:13:18.888 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 62587 2020-08-07T15:13:18.888 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.888 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.888 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:13:18.888 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.890 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.890 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6812", 2020-08-07T15:13:18.891 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 62587 2020-08-07T15:13:18.891 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.891 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.891 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:13:18.892 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.892 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.892 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.892 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6811", 2020-08-07T15:13:18.892 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 62587 2020-08-07T15:13:18.892 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.893 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.893 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6809/62587", 2020-08-07T15:13:18.893 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6810/62587", 2020-08-07T15:13:18.893 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6812/62587", 2020-08-07T15:13:18.894 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6811/62587", 2020-08-07T15:13:18.894 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:13:18.894 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:13:18.894 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:13:18.894 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.895 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.895 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.895 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 2, 2020-08-07T15:13:18.895 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "d6fe4338-3e11-4f73-9cbb-e1b046ce286a", 2020-08-07T15:13:18.895 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:13:18.896 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:13:18.896 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:13:18.896 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:13:18.896 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:13:18.896 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:13:18.897 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:13:18.897 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 269, 2020-08-07T15:13:18.897 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:13:18.897 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:13:18.897 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:13:18.897 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.898 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.898 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.898 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6805", 2020-08-07T15:13:18.898 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:13:18.898 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.899 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.900 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:13:18.901 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.901 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.901 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.901 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6806", 2020-08-07T15:13:18.901 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:13:18.902 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.902 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.902 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.902 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:13:18.903 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.903 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.903 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6808", 2020-08-07T15:13:18.903 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:13:18.904 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.904 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.904 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:13:18.904 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.905 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.905 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6807", 2020-08-07T15:13:18.905 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:13:18.905 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.906 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.906 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6805/31576", 2020-08-07T15:13:18.906 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6806/31576", 2020-08-07T15:13:18.906 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6808/31576", 2020-08-07T15:13:18.906 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6807/31576", 2020-08-07T15:13:18.907 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:13:18.907 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:13:18.907 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:13:18.907 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.908 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 3, 2020-08-07T15:13:18.908 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9", 2020-08-07T15:13:18.908 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:13:18.908 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:13:18.909 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:13:18.909 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:13:18.909 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:13:18.909 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:13:18.909 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:13:18.910 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 269, 2020-08-07T15:13:18.910 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:13:18.915 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:13:18.915 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:13:18.915 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.916 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.916 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6813", 2020-08-07T15:13:18.916 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:13:18.917 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.917 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.917 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:13:18.917 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.918 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.918 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6814", 2020-08-07T15:13:18.918 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:13:18.918 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.919 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.919 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.919 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:13:18.919 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.920 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.920 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6816", 2020-08-07T15:13:18.920 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:13:18.920 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.920 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.921 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:13:18.921 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.921 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.922 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6815", 2020-08-07T15:13:18.922 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:13:18.922 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.922 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.922 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6813/31579", 2020-08-07T15:13:18.923 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6814/31579", 2020-08-07T15:13:18.923 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6816/31579", 2020-08-07T15:13:18.923 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6815/31579", 2020-08-07T15:13:18.923 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:13:18.923 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:13:18.924 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:13:18.924 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.924 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 4, 2020-08-07T15:13:18.925 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "01aadc13-29c5-4ddd-9bb9-de24b2b20412", 2020-08-07T15:13:18.925 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:13:18.925 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:13:18.925 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:13:18.925 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0, 2020-08-07T15:13:18.926 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:13:18.926 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:13:18.926 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:13:18.926 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 248, 2020-08-07T15:13:18.926 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:13:18.927 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:13:18.929 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:13:18.929 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.929 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.930 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6800", 2020-08-07T15:13:18.930 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:13:18.930 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.930 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.931 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:13:18.931 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.931 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.931 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6801", 2020-08-07T15:13:18.932 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:13:18.932 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.932 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.932 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:13:18.933 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.933 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.933 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.933 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6803", 2020-08-07T15:13:18.933 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:13:18.934 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.934 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.934 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.934 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:13:18.934 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.934 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.935 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.935 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6802", 2020-08-07T15:13:18.935 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:13:18.935 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.935 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.936 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6800/31545", 2020-08-07T15:13:18.936 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6801/31545", 2020-08-07T15:13:18.936 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6803/31545", 2020-08-07T15:13:18.937 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6802/31545", 2020-08-07T15:13:18.937 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:13:18.937 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:13:18.937 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:13:18.937 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.938 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 5, 2020-08-07T15:13:18.938 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "f5556a82-73b4-47b1-8e29-1d2fe031d8fd", 2020-08-07T15:13:18.938 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:13:18.939 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:13:18.939 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:13:18.939 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:13:18.939 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:13:18.939 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:13:18.940 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:13:18.940 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 269, 2020-08-07T15:13:18.940 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:13:18.940 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:13:18.940 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:13:18.941 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.941 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.941 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6804", 2020-08-07T15:13:18.941 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:13:18.941 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.942 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.942 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.942 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:13:18.942 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.943 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.946 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6805", 2020-08-07T15:13:18.946 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:13:18.947 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.947 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.948 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:13:18.948 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.948 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.948 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6807", 2020-08-07T15:13:18.949 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:13:18.949 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.949 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.949 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.949 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:13:18.950 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.950 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.950 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.950 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6806", 2020-08-07T15:13:18.950 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:13:18.951 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.951 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.951 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6804/31547", 2020-08-07T15:13:18.951 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6805/31547", 2020-08-07T15:13:18.952 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6807/31547", 2020-08-07T15:13:18.952 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6806/31547", 2020-08-07T15:13:18.952 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:13:18.952 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:13:18.952 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:13:18.952 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.953 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.953 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.953 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 6, 2020-08-07T15:13:18.953 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "9b98b337-e1cc-43cd-9e9b-f21ae3f20899", 2020-08-07T15:13:18.953 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:13:18.954 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:13:18.954 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:13:18.954 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:13:18.954 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:13:18.954 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:13:18.955 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 14, 2020-08-07T15:13:18.955 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 269, 2020-08-07T15:13:18.955 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:13:18.955 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:13:18.955 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:13:18.956 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.956 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.956 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.957 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6808", 2020-08-07T15:13:18.957 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:13:18.957 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.957 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.958 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:13:18.958 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.958 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.959 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6809", 2020-08-07T15:13:18.959 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:13:18.959 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.959 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.960 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:13:18.960 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.960 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.960 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6811", 2020-08-07T15:13:18.960 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:13:18.961 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.961 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.961 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:13:18.961 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.962 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.962 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6810", 2020-08-07T15:13:18.962 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:13:18.962 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.963 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.963 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6808/31546", 2020-08-07T15:13:18.963 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6809/31546", 2020-08-07T15:13:18.963 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6811/31546", 2020-08-07T15:13:18.964 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6810/31546", 2020-08-07T15:13:18.964 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:13:18.964 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:13:18.964 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:13:18.964 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.965 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 7, 2020-08-07T15:13:18.965 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "777662d8-bd34-48e8-83e1-24ce9e61f14b", 2020-08-07T15:13:18.965 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:13:18.966 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:13:18.966 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:13:18.966 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0, 2020-08-07T15:13:18.967 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:13:18.967 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:13:18.967 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:13:18.967 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 244, 2020-08-07T15:13:18.967 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:13:18.968 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:13:18.968 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:13:18.968 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.968 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.969 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6812", 2020-08-07T15:13:18.969 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:13:18.969 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.969 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.969 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.969 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:13:18.970 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.970 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.970 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6813", 2020-08-07T15:13:18.970 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:13:18.971 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.971 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.971 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:13:18.971 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.972 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.972 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6815", 2020-08-07T15:13:18.972 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:13:18.972 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.972 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.973 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:13:18.973 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:18.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.973 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:18.974 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6814", 2020-08-07T15:13:18.974 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:13:18.983 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.983 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.983 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.984 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6812/31570", 2020-08-07T15:13:18.984 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6813/31570", 2020-08-07T15:13:18.987 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6815/31570", 2020-08-07T15:13:18.987 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6814/31570", 2020-08-07T15:13:18.987 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:13:18.987 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:13:18.988 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:13:18.988 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:18.989 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:18.989 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:13:18.989 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_xinfo": [ 2020-08-07T15:13:18.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.989 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 0, 2020-08-07T15:13:18.989 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:13:18.990 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:13:18.990 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:13:18.990 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:13:18.990 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:13:18.990 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:12:19.121056+0000", 2020-08-07T15:13:18.991 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:13:18.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.991 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 1, 2020-08-07T15:13:18.991 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:13:18.992 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:13:18.992 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:13:18.992 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:13:18.992 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:13:18.992 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:13:06.000449+0000", 2020-08-07T15:13:18.993 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 191 2020-08-07T15:13:18.993 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.993 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.993 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 2, 2020-08-07T15:13:18.993 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:13:18.993 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:13:18.994 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:13:18.994 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:13:18.994 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:13:18.994 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:13:18.172058+0000", 2020-08-07T15:13:18.994 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:13:18.995 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.995 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.995 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 3, 2020-08-07T15:13:18.995 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:13:18.995 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:13:18.996 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:13:18.996 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:13:18.996 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:13:18.996 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:12:19.857070+0000", 2020-08-07T15:13:18.996 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:13:18.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.997 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.997 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 4, 2020-08-07T15:13:18.997 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:13:18.997 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:13:18.997 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:13:18.998 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:13:18.998 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:13:18.998 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:12:18.727770+0000", 2020-08-07T15:13:18.998 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:13:18.998 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:18.999 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:18.999 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 5, 2020-08-07T15:13:18.999 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:13:18.999 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:13:18.999 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:13:19.000 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:13:19.000 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:13:19.000 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:12:19.039747+0000", 2020-08-07T15:13:19.000 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:13:19.000 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:19.000 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:19.001 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 6, 2020-08-07T15:13:19.001 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:13:19.001 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:13:19.001 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:13:19.001 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:13:19.002 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:13:19.002 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:12:17.971384+0000", 2020-08-07T15:13:19.002 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:13:19.002 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:19.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:19.003 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 7, 2020-08-07T15:13:19.003 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:13:19.003 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:13:19.003 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:13:19.004 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:13:19.004 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:13:19.004 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:13:17.455633+0000", 2020-08-07T15:13:19.004 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:13:19.005 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:19.005 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:13:19.005 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_upmap": [ 2020-08-07T15:13:19.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:19.005 INFO:teuthology.orchestra.run.smithi074.stdout: "pgid": "1.16", 2020-08-07T15:13:19.006 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:13:19.006 INFO:teuthology.orchestra.run.smithi074.stdout: 0, 2020-08-07T15:13:19.006 INFO:teuthology.orchestra.run.smithi074.stdout: 5, 2020-08-07T15:13:19.006 INFO:teuthology.orchestra.run.smithi074.stdout: 4 2020-08-07T15:13:19.006 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:19.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:19.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:19.007 INFO:teuthology.orchestra.run.smithi074.stdout: "pgid": "6.a", 2020-08-07T15:13:19.007 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:13:19.007 INFO:teuthology.orchestra.run.smithi074.stdout: 6, 2020-08-07T15:13:19.007 INFO:teuthology.orchestra.run.smithi074.stdout: 5, 2020-08-07T15:13:19.008 INFO:teuthology.orchestra.run.smithi074.stdout: 7 2020-08-07T15:13:19.008 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:19.008 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:19.008 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:13:19.008 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_upmap_items": [], 2020-08-07T15:13:19.009 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_temp": [ 2020-08-07T15:13:19.009 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:19.009 INFO:teuthology.orchestra.run.smithi074.stdout: "pgid": "6.a", 2020-08-07T15:13:19.009 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:13:19.009 INFO:teuthology.orchestra.run.smithi074.stdout: 6, 2020-08-07T15:13:19.010 INFO:teuthology.orchestra.run.smithi074.stdout: 5, 2020-08-07T15:13:19.010 INFO:teuthology.orchestra.run.smithi074.stdout: 0 2020-08-07T15:13:19.010 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:19.010 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:19.010 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:13:19.011 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_temp": [], 2020-08-07T15:13:19.011 INFO:teuthology.orchestra.run.smithi074.stdout: "blacklist": {}, 2020-08-07T15:13:19.011 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profiles": { 2020-08-07T15:13:19.011 INFO:teuthology.orchestra.run.smithi074.stdout: "default": { 2020-08-07T15:13:19.012 INFO:teuthology.orchestra.run.smithi074.stdout: "crush-failure-domain": "osd", 2020-08-07T15:13:19.012 INFO:teuthology.orchestra.run.smithi074.stdout: "k": "2", 2020-08-07T15:13:19.012 INFO:teuthology.orchestra.run.smithi074.stdout: "m": "1", 2020-08-07T15:13:19.012 INFO:teuthology.orchestra.run.smithi074.stdout: "plugin": "jerasure", 2020-08-07T15:13:19.012 INFO:teuthology.orchestra.run.smithi074.stdout: "ruleset-failure-domain": "osd", 2020-08-07T15:13:19.013 INFO:teuthology.orchestra.run.smithi074.stdout: "technique": "reed_sol_van" 2020-08-07T15:13:19.013 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:19.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:19.013 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps_queue": [], 2020-08-07T15:13:19.013 INFO:teuthology.orchestra.run.smithi074.stdout: "new_removed_snaps": [], 2020-08-07T15:13:19.014 INFO:teuthology.orchestra.run.smithi074.stdout: "new_purged_snaps": [], 2020-08-07T15:13:19.014 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_node_flags": {}, 2020-08-07T15:13:19.014 INFO:teuthology.orchestra.run.smithi074.stdout: "device_class_flags": {} 2020-08-07T15:13:19.014 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:19.015 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:13:19.017 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:19.018 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:19.019 INFO:tasks.thrashosds.thrasher:j is {'epoch': 280, 'fsid': 'a01d3e82-8d24-4288-9e42-576ab3ff4957', 'created': '2020-08-07T15:06:07.342912+0000', 'modified': '2020-08-07T15:13:18.241648+0000', 'last_up_change': '2020-08-07T15:12:05.338271+0000', 'last_in_change': '2020-08-07T15:06:17.674292+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 6, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-08-07T15:06:10.648876+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 41, 'pg_placement_num': 41, 'pg_placement_num_target': 41, 'pg_num_target': 41, 'pg_num_pending': 41, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '246', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '244', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-08-07T15:06:21.474648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 6, 'pool_name': 'unique_pool_3', 'create_time': '2020-08-07T15:12:08.847598+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '232', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': 'fc1ebb23-3ef4-4fc0-b395-382e385885a3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 262, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6801', 'nonce': 31577}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6802', 'nonce': 31577}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6804', 'nonce': 31577}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6803', 'nonce': 31577}]}, 'public_addr': '172.21.15.74:6801/31577', 'cluster_addr': '172.21.15.74:6802/31577', 'heartbeat_back_addr': '172.21.15.74:6804/31577', 'heartbeat_front_addr': '172.21.15.74:6803/31577', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '1fbe530a-63ce-4dc8-82ab-18793d6ebc0b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.7196044921875, 'last_clean_begin': 13, 'last_clean_end': 190, 'up_from': 227, 'up_thru': 269, 'down_at': 191, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6809', 'nonce': 62587}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6810', 'nonce': 62587}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6812', 'nonce': 62587}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6811', 'nonce': 62587}]}, 'public_addr': '172.21.15.74:6809/62587', 'cluster_addr': '172.21.15.74:6810/62587', 'heartbeat_back_addr': '172.21.15.74:6812/62587', 'heartbeat_front_addr': '172.21.15.74:6811/62587', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd6fe4338-3e11-4f73-9cbb-e1b046ce286a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 269, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6805', 'nonce': 31576}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6806', 'nonce': 31576}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6808', 'nonce': 31576}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6807', 'nonce': 31576}]}, 'public_addr': '172.21.15.74:6805/31576', 'cluster_addr': '172.21.15.74:6806/31576', 'heartbeat_back_addr': '172.21.15.74:6808/31576', 'heartbeat_front_addr': '172.21.15.74:6807/31576', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 269, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6813', 'nonce': 31579}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6814', 'nonce': 31579}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6816', 'nonce': 31579}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6815', 'nonce': 31579}]}, 'public_addr': '172.21.15.74:6813/31579', 'cluster_addr': '172.21.15.74:6814/31579', 'heartbeat_back_addr': '172.21.15.74:6816/31579', 'heartbeat_front_addr': '172.21.15.74:6815/31579', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '01aadc13-29c5-4ddd-9bb9-de24b2b20412', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 248, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6800', 'nonce': 31545}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6801', 'nonce': 31545}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6803', 'nonce': 31545}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6802', 'nonce': 31545}]}, 'public_addr': '172.21.15.96:6800/31545', 'cluster_addr': '172.21.15.96:6801/31545', 'heartbeat_back_addr': '172.21.15.96:6803/31545', 'heartbeat_front_addr': '172.21.15.96:6802/31545', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'f5556a82-73b4-47b1-8e29-1d2fe031d8fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 269, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6804', 'nonce': 31547}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6805', 'nonce': 31547}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6807', 'nonce': 31547}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6806', 'nonce': 31547}]}, 'public_addr': '172.21.15.96:6804/31547', 'cluster_addr': '172.21.15.96:6805/31547', 'heartbeat_back_addr': '172.21.15.96:6807/31547', 'heartbeat_front_addr': '172.21.15.96:6806/31547', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9b98b337-e1cc-43cd-9e9b-f21ae3f20899', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 269, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6808', 'nonce': 31546}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6809', 'nonce': 31546}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6811', 'nonce': 31546}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6810', 'nonce': 31546}]}, 'public_addr': '172.21.15.96:6808/31546', 'cluster_addr': '172.21.15.96:6809/31546', 'heartbeat_back_addr': '172.21.15.96:6811/31546', 'heartbeat_front_addr': '172.21.15.96:6810/31546', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '777662d8-bd34-48e8-83e1-24ce9e61f14b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 244, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6812', 'nonce': 31570}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6813', 'nonce': 31570}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6815', 'nonce': 31570}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6814', 'nonce': 31570}]}, 'public_addr': '172.21.15.96:6812/31570', 'cluster_addr': '172.21.15.96:6813/31570', 'heartbeat_back_addr': '172.21.15.96:6815/31570', 'heartbeat_front_addr': '172.21.15.96:6814/31570', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:12:19.121056+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:13:06.000449+0000', 'dead_epoch': 191}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:13:18.172058+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:12:19.857070+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:12:18.727770+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:12:19.039747+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:12:17.971384+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:13:17.455633+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '1.16', 'osds': [0, 5, 4]}, {'pgid': '6.a', 'osds': [6, 5, 7]}], 'pg_upmap_items': [], 'pg_temp': [{'pgid': '6.a', 'osds': [6, 5, 0]}], 'primary_temp': [], 'blacklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}} 2020-08-07T15:13:19.019 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:13:19.020 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:19.015+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:19.020 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:13:19.116 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:19.116+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:19.230 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:19.217+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:19.231 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:19.231 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:19.232 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:13:19.265 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:13:19.271 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 67 16 99288 99272 11.5742 9.59375 0.0168208 0.010795 2020-08-07T15:13:19.318 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:19.317+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:19.324 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:13:19.377 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:13:19.420 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:19.418+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:19.420 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:13:19.429 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:19.429 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:19.439 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:13:19.470 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:13:19.520 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:19.519+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:19.521 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:13:19.570 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:13:19.579 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:13:19.583 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:13:19.599 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:19.600 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:13:19.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.600 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5481.0:99737 6.4 e281/230 6:225ee863:::benchmark_data_smithi074_63843_object12467:head v 281'6241, mlcod=281'6237)", 2020-08-07T15:13:19.600 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:13:19.592118+0000", 2020-08-07T15:13:19.600 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0022427910000000001, 2020-08-07T15:13:19.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022791370000000001, 2020-08-07T15:13:19.601 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:19.601 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:19.601 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:19.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:19.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:19.592118+0000", 2020-08-07T15:13:19.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:19.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:19.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:19.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:19.592118+0000", 2020-08-07T15:13:19.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8501e-05 2020-08-07T15:13:19.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:19.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:19.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:19.592147+0000", 2020-08-07T15:13:19.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5559999999999999e-06 2020-08-07T15:13:19.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:19.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:19.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:19.592150+0000", 2020-08-07T15:13:19.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017652849999999999 2020-08-07T15:13:19.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:19.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:19.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:19.593916+0000", 2020-08-07T15:13:19.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7649999999999999e-06 2020-08-07T15:13:19.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:19.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:19.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:19.593922+0000", 2020-08-07T15:13:19.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4851999999999998e-05 2020-08-07T15:13:19.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:19.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:19.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:19.593956+0000", 2020-08-07T15:13:19.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1789000000000001e-05 2020-08-07T15:13:19.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:19.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:19.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:19.593978+0000", 2020-08-07T15:13:19.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001859748 2020-08-07T15:13:19.611 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:19.611 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:19.611 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:19.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:19.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.612 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5481.0:99738 6.4 e281/230 6:225ee863:::benchmark_data_smithi074_63843_object12467:head v 281'6242, mlcod=281'6237)", 2020-08-07T15:13:19.612 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:13:19.593816+0000", 2020-08-07T15:13:19.612 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.000544751, 2020-08-07T15:13:19.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067256599999999998, 2020-08-07T15:13:19.613 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:19.613 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:19.613 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:19.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:19.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:19.593816+0000", 2020-08-07T15:13:19.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:19.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:19.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:19.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:19.593816+0000", 2020-08-07T15:13:19.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8554000000000001e-05 2020-08-07T15:13:19.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:19.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:19.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:19.593845+0000", 2020-08-07T15:13:19.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3780000000000003e-06 2020-08-07T15:13:19.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:19.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:19.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:19.593849+0000", 2020-08-07T15:13:19.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5665000000000001e-05 2020-08-07T15:13:19.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:19.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:19.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:19.593935+0000", 2020-08-07T15:13:19.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3129999999999999e-06 2020-08-07T15:13:19.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:19.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:19.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:19.593938+0000", 2020-08-07T15:13:19.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000334152 2020-08-07T15:13:19.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:19.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:19.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:19.594273+0000", 2020-08-07T15:13:19.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5302999999999999e-05 2020-08-07T15:13:19.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:19.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:19.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:19.594288+0000", 2020-08-07T15:13:19.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00047136500000000001 2020-08-07T15:13:19.626 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:19.627 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:19.627 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:19.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:19.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.627 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5481.0:99730 6.c e281/230 6:3d1700c2:::benchmark_data_smithi074_63843_object12466:head v 281'6466, mlcod=281'6454)", 2020-08-07T15:13:19.628 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:13:19.590109+0000", 2020-08-07T15:13:19.628 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0042522560000000003, 2020-08-07T15:13:19.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044432389999999999, 2020-08-07T15:13:19.628 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:19.628 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:19.629 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:19.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:19.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:19.590109+0000", 2020-08-07T15:13:19.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:19.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:19.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:19.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:19.590109+0000", 2020-08-07T15:13:19.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2896000000000003e-05 2020-08-07T15:13:19.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:19.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:19.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:19.590162+0000", 2020-08-07T15:13:19.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6440000000000003e-06 2020-08-07T15:13:19.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:19.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:19.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:19.590169+0000", 2020-08-07T15:13:19.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039249940000000002 2020-08-07T15:13:19.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:19.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:19.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:19.594094+0000", 2020-08-07T15:13:19.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0969999999999999e-06 2020-08-07T15:13:19.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:19.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:19.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:19.594098+0000", 2020-08-07T15:13:19.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000163936 2020-08-07T15:13:19.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:19.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:19.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:19.594262+0000", 2020-08-07T15:13:19.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3006999999999999e-05 2020-08-07T15:13:19.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:19.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:19.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:19.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:19.594275+0000", 2020-08-07T15:13:19.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004165574 2020-08-07T15:13:19.646 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:19.647 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:19.647 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:19.647 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:19.647 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:13:19.647 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 3 2020-08-07T15:13:19.647 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:19.664 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2020-08-07T15:13:19.249668+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":784760832,"num_objects":11978,"num_object_clones":0,"num_object_copies":35934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":784,"num_objects_unfound":0,"num_objects_dirty":11978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95796,"num_write_kb":766368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":195,"acting":195,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":196,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":10690904,"kb_used_data":2302232,"kb_used_omap":2,"kb_used_meta":8388605,"kb_avail":744283816,"statfs":{"total":773094113280,"available":762146627584,"internally_reserved":8589934592,"allocated":2357485568,"data_stored":2346141984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":8589932013},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,4,2,5,2,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":61833216,"num_objects":945,"num_object_clones":0,"num_object_copies":2835,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":62,"num_objects_unfound":0,"num_objects_dirty":945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7548,"num_write_kb":60384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.556957"},"pg_stats":[{"pgid":"1.28","version":"16'2","reported_seq":"42","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670295+0000","last_change":"2020-08-07T15:12:37.495634+0000","last_active":"2020-08-07T15:13:14.670295+0000","last_peered":"2020-08-07T15:13:14.670295+0000","last_clean":"2020-08-07T15:13:14.670295+0000","last_became_active":"2020-08-07T15:12:37.495085+0000","last_became_peered":"2020-08-07T15:12:37.495085+0000","last_unstale":"2020-08-07T15:13:14.670295+0000","last_undegraded":"2020-08-07T15:13:14.670295+0000","last_fullsized":"2020-08-07T15:13:14.670295+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.490655+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.490655+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.27","version":"16'2","reported_seq":"391","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:15.686151+0000","last_change":"2020-08-07T15:12:57.250017+0000","last_active":"2020-08-07T15:13:15.686151+0000","last_peered":"2020-08-07T15:13:15.686151+0000","last_clean":"2020-08-07T15:13:15.686151+0000","last_became_active":"2020-08-07T15:12:37.411869+0000","last_became_peered":"2020-08-07T15:12:37.411869+0000","last_unstale":"2020-08-07T15:13:15.686151+0000","last_undegraded":"2020-08-07T15:13:15.686151+0000","last_fullsized":"2020-08-07T15:13:15.686151+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:57.249927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:57.249927+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.26","version":"16'2","reported_seq":"51","reported_epoch":"276","state":"active+clean","last_fresh":"2020-08-07T15:13:13.685550+0000","last_change":"2020-08-07T15:12:40.765120+0000","last_active":"2020-08-07T15:13:13.685550+0000","last_peered":"2020-08-07T15:13:13.685550+0000","last_clean":"2020-08-07T15:13:13.685550+0000","last_became_active":"2020-08-07T15:12:37.457426+0000","last_became_peered":"2020-08-07T15:12:37.457426+0000","last_unstale":"2020-08-07T15:13:13.685550+0000","last_undegraded":"2020-08-07T15:13:13.685550+0000","last_fullsized":"2020-08-07T15:13:13.685550+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:40.765027+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_clean_scrub_stamp":"2020-08-07T15:12:40.765027+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.25","version":"75'3","reported_seq":"278","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.682783+0000","last_change":"2020-08-07T15:13:07.465488+0000","last_active":"2020-08-07T15:13:14.682783+0000","last_peered":"2020-08-07T15:13:14.682783+0000","last_clean":"2020-08-07T15:13:14.682783+0000","last_became_active":"2020-08-07T15:12:37.427816+0000","last_became_peered":"2020-08-07T15:12:37.427816+0000","last_unstale":"2020-08-07T15:13:14.682783+0000","last_undegraded":"2020-08-07T15:13:14.682783+0000","last_fullsized":"2020-08-07T15:13:14.682783+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:13:07.465410+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:13:07.465410+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.24","version":"16'2","reported_seq":"216","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.674070+0000","last_change":"2020-08-07T15:12:42.295755+0000","last_active":"2020-08-07T15:13:14.674070+0000","last_peered":"2020-08-07T15:13:14.674070+0000","last_clean":"2020-08-07T15:13:14.674070+0000","last_became_active":"2020-08-07T15:12:34.414144+0000","last_became_peered":"2020-08-07T15:12:34.414144+0000","last_unstale":"2020-08-07T15:13:14.674070+0000","last_undegraded":"2020-08-07T15:13:14.674070+0000","last_fullsized":"2020-08-07T15:13:14.674070+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:42.295672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:42.295672+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.23","version":"16'2","reported_seq":"326","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:15.685911+0000","last_change":"2020-08-07T15:12:55.290134+0000","last_active":"2020-08-07T15:13:15.685911+0000","last_peered":"2020-08-07T15:13:15.685911+0000","last_clean":"2020-08-07T15:13:15.685911+0000","last_became_active":"2020-08-07T15:12:37.410157+0000","last_became_peered":"2020-08-07T15:12:37.410157+0000","last_unstale":"2020-08-07T15:13:15.685911+0000","last_undegraded":"2020-08-07T15:13:15.685911+0000","last_fullsized":"2020-08-07T15:13:15.685911+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:55.290028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:55.290028+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1],"acting":[6,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.22","version":"16'2","reported_seq":"46","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:15.686042+0000","last_change":"2020-08-07T15:12:58.227750+0000","last_active":"2020-08-07T15:13:15.686042+0000","last_peered":"2020-08-07T15:13:15.686042+0000","last_clean":"2020-08-07T15:13:15.686042+0000","last_became_active":"2020-08-07T15:12:37.413046+0000","last_became_peered":"2020-08-07T15:12:37.413046+0000","last_unstale":"2020-08-07T15:13:15.686042+0000","last_undegraded":"2020-08-07T15:13:15.686042+0000","last_fullsized":"2020-08-07T15:13:15.686042+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:58.227699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:58.227699+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.21","version":"16'2","reported_seq":"52","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.673701+0000","last_change":"2020-08-07T15:12:58.387474+0000","last_active":"2020-08-07T15:13:14.673701+0000","last_peered":"2020-08-07T15:13:14.673701+0000","last_clean":"2020-08-07T15:13:14.673701+0000","last_became_active":"2020-08-07T15:12:37.418612+0000","last_became_peered":"2020-08-07T15:12:37.418612+0000","last_unstale":"2020-08-07T15:13:14.673701+0000","last_undegraded":"2020-08-07T15:13:14.673701+0000","last_fullsized":"2020-08-07T15:13:14.673701+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:58.387424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:58.387424+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.20","version":"16'2","reported_seq":"52","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.682629+0000","last_change":"2020-08-07T15:12:54.433326+0000","last_active":"2020-08-07T15:13:14.682629+0000","last_peered":"2020-08-07T15:13:14.682629+0000","last_clean":"2020-08-07T15:13:14.682629+0000","last_became_active":"2020-08-07T15:12:37.409249+0000","last_became_peered":"2020-08-07T15:12:37.409249+0000","last_unstale":"2020-08-07T15:13:14.682629+0000","last_undegraded":"2020-08-07T15:13:14.682629+0000","last_fullsized":"2020-08-07T15:13:14.682629+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:54.433245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:54.433245+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1f","version":"16'2","reported_seq":"45","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.682914+0000","last_change":"2020-08-07T15:13:09.471546+0000","last_active":"2020-08-07T15:13:14.682914+0000","last_peered":"2020-08-07T15:13:14.682914+0000","last_clean":"2020-08-07T15:13:14.682914+0000","last_became_active":"2020-08-07T15:12:37.410716+0000","last_became_peered":"2020-08-07T15:12:37.410716+0000","last_unstale":"2020-08-07T15:13:14.682914+0000","last_undegraded":"2020-08-07T15:13:14.682914+0000","last_fullsized":"2020-08-07T15:13:14.682914+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:09.471471+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_clean_scrub_stamp":"2020-08-07T15:13:09.471471+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1e","version":"16'2","reported_seq":"132","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.674282+0000","last_change":"2020-08-07T15:13:09.396723+0000","last_active":"2020-08-07T15:13:14.674282+0000","last_peered":"2020-08-07T15:13:14.674282+0000","last_clean":"2020-08-07T15:13:14.674282+0000","last_became_active":"2020-08-07T15:10:51.559894+0000","last_became_peered":"2020-08-07T15:10:51.559894+0000","last_unstale":"2020-08-07T15:13:14.674282+0000","last_undegraded":"2020-08-07T15:13:14.674282+0000","last_fullsized":"2020-08-07T15:13:14.674282+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:09.396633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:09.396633+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1d","version":"75'3","reported_seq":"141","reported_epoch":"276","state":"active+clean","last_fresh":"2020-08-07T15:13:13.685428+0000","last_change":"2020-08-07T15:12:28.855487+0000","last_active":"2020-08-07T15:13:13.685428+0000","last_peered":"2020-08-07T15:13:13.685428+0000","last_clean":"2020-08-07T15:13:13.685428+0000","last_became_active":"2020-08-07T15:12:07.356691+0000","last_became_peered":"2020-08-07T15:12:07.356691+0000","last_unstale":"2020-08-07T15:13:13.685428+0000","last_undegraded":"2020-08-07T15:13:13.685428+0000","last_fullsized":"2020-08-07T15:13:13.685428+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:28.855432+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:28.855432+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1c","version":"16'2","reported_seq":"134","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:15.686010+0000","last_change":"2020-08-07T15:13:07.471270+0000","last_active":"2020-08-07T15:13:15.686010+0000","last_peered":"2020-08-07T15:13:15.686010+0000","last_clean":"2020-08-07T15:13:15.686010+0000","last_became_active":"2020-08-07T15:13:02.591669+0000","last_became_peered":"2020-08-07T15:13:02.591669+0000","last_unstale":"2020-08-07T15:13:15.686010+0000","last_undegraded":"2020-08-07T15:13:15.686010+0000","last_fullsized":"2020-08-07T15:13:15.686010+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:07.471192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:07.471192+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1b","version":"16'2","reported_seq":"250","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670647+0000","last_change":"2020-08-07T15:10:57.797029+0000","last_active":"2020-08-07T15:13:14.670647+0000","last_peered":"2020-08-07T15:13:14.670647+0000","last_clean":"2020-08-07T15:13:14.670647+0000","last_became_active":"2020-08-07T15:10:51.557392+0000","last_became_peered":"2020-08-07T15:10:51.557392+0000","last_unstale":"2020-08-07T15:13:14.670647+0000","last_undegraded":"2020-08-07T15:13:14.670647+0000","last_fullsized":"2020-08-07T15:13:14.670647+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:10:57.796946+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:57.796946+0000","last_clean_scrub_stamp":"2020-08-07T15:10:57.796946+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1a","version":"16'2","reported_seq":"23","reported_epoch":"276","state":"active+clean","last_fresh":"2020-08-07T15:13:13.685115+0000","last_change":"2020-08-07T15:13:06.881943+0000","last_active":"2020-08-07T15:13:13.685115+0000","last_peered":"2020-08-07T15:13:13.685115+0000","last_clean":"2020-08-07T15:13:13.685115+0000","last_became_active":"2020-08-07T15:13:02.590409+0000","last_became_peered":"2020-08-07T15:13:02.590409+0000","last_unstale":"2020-08-07T15:13:13.685115+0000","last_undegraded":"2020-08-07T15:13:13.685115+0000","last_fullsized":"2020-08-07T15:13:13.685115+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:06.881863+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:13:06.881863+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.19","version":"16'2","reported_seq":"175","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.673825+0000","last_change":"2020-08-07T15:12:15.372999+0000","last_active":"2020-08-07T15:13:14.673825+0000","last_peered":"2020-08-07T15:13:14.673825+0000","last_clean":"2020-08-07T15:13:14.673825+0000","last_became_active":"2020-08-07T15:10:51.559752+0000","last_became_peered":"2020-08-07T15:10:51.559752+0000","last_unstale":"2020-08-07T15:13:14.673825+0000","last_undegraded":"2020-08-07T15:13:14.673825+0000","last_fullsized":"2020-08-07T15:13:14.673825+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:15.372910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:15.372910+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.18","version":"16'2","reported_seq":"275","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670410+0000","last_change":"2020-08-07T15:11:06.955336+0000","last_active":"2020-08-07T15:13:14.670410+0000","last_peered":"2020-08-07T15:13:14.670410+0000","last_clean":"2020-08-07T15:13:14.670410+0000","last_became_active":"2020-08-07T15:10:51.557665+0000","last_became_peered":"2020-08-07T15:10:51.557665+0000","last_unstale":"2020-08-07T15:13:14.670410+0000","last_undegraded":"2020-08-07T15:13:14.670410+0000","last_fullsized":"2020-08-07T15:13:14.670410+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:06.955262+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:06.955262+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.17","version":"16'2","reported_seq":"106","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.671445+0000","last_change":"2020-08-07T15:12:57.172008+0000","last_active":"2020-08-07T15:13:14.671445+0000","last_peered":"2020-08-07T15:13:14.671445+0000","last_clean":"2020-08-07T15:13:14.671445+0000","last_became_active":"2020-08-07T15:10:51.561769+0000","last_became_peered":"2020-08-07T15:10:51.561769+0000","last_unstale":"2020-08-07T15:13:14.671445+0000","last_undegraded":"2020-08-07T15:13:14.671445+0000","last_fullsized":"2020-08-07T15:13:14.671445+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:57.171893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:57.171893+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.16","version":"16'2","reported_seq":"51","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.671213+0000","last_change":"2020-08-07T15:12:52.576339+0000","last_active":"2020-08-07T15:13:14.671213+0000","last_peered":"2020-08-07T15:13:14.671213+0000","last_clean":"2020-08-07T15:13:14.671213+0000","last_became_active":"2020-08-07T15:12:52.575577+0000","last_became_peered":"2020-08-07T15:12:52.575577+0000","last_unstale":"2020-08-07T15:13:14.671213+0000","last_undegraded":"2020-08-07T15:13:14.671213+0000","last_fullsized":"2020-08-07T15:13:14.671213+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:12.883558+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:12.883558+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.883558+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.15","version":"75'3","reported_seq":"34","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.671173+0000","last_change":"2020-08-07T15:13:02.590465+0000","last_active":"2020-08-07T15:13:14.671173+0000","last_peered":"2020-08-07T15:13:14.671173+0000","last_clean":"2020-08-07T15:13:14.671173+0000","last_became_active":"2020-08-07T15:13:02.589879+0000","last_became_peered":"2020-08-07T15:13:02.589879+0000","last_unstale":"2020-08-07T15:13:14.671173+0000","last_undegraded":"2020-08-07T15:13:14.671173+0000","last_fullsized":"2020-08-07T15:13:14.671173+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:53.824617+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:53.824617+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.14","version":"16'2","reported_seq":"213","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.683186+0000","last_change":"2020-08-07T15:12:09.224231+0000","last_active":"2020-08-07T15:13:14.683186+0000","last_peered":"2020-08-07T15:13:14.683186+0000","last_clean":"2020-08-07T15:13:14.683186+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:13:14.683186+0000","last_undegraded":"2020-08-07T15:13:14.683186+0000","last_fullsized":"2020-08-07T15:13:14.683186+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:09.224158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.224158+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"227","reported_epoch":"276","state":"active+clean","last_fresh":"2020-08-07T15:13:13.685167+0000","last_change":"2020-08-07T15:12:19.577926+0000","last_active":"2020-08-07T15:13:13.685167+0000","last_peered":"2020-08-07T15:13:13.685167+0000","last_clean":"2020-08-07T15:13:13.685167+0000","last_became_active":"2020-08-07T15:12:06.375861+0000","last_became_peered":"2020-08-07T15:12:06.375861+0000","last_unstale":"2020-08-07T15:13:13.685167+0000","last_undegraded":"2020-08-07T15:13:13.685167+0000","last_fullsized":"2020-08-07T15:13:13.685167+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:07.887356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.887356+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"228","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670644+0000","last_change":"2020-08-07T15:12:26.104598+0000","last_active":"2020-08-07T15:13:14.670644+0000","last_peered":"2020-08-07T15:13:14.670644+0000","last_clean":"2020-08-07T15:13:14.670644+0000","last_became_active":"2020-08-07T15:12:06.367362+0000","last_became_peered":"2020-08-07T15:12:06.367362+0000","last_unstale":"2020-08-07T15:13:14.670644+0000","last_undegraded":"2020-08-07T15:13:14.670644+0000","last_fullsized":"2020-08-07T15:13:14.670644+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:26.104526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:26.104526+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"326","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670284+0000","last_change":"2020-08-07T15:12:24.105593+0000","last_active":"2020-08-07T15:13:14.670284+0000","last_peered":"2020-08-07T15:13:14.670284+0000","last_clean":"2020-08-07T15:13:14.670284+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:13:14.670284+0000","last_undegraded":"2020-08-07T15:13:14.670284+0000","last_fullsized":"2020-08-07T15:13:14.670284+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:24.105531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:24.105531+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"320","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670523+0000","last_change":"2020-08-07T15:12:13.231819+0000","last_active":"2020-08-07T15:13:14.670523+0000","last_peered":"2020-08-07T15:13:14.670523+0000","last_clean":"2020-08-07T15:13:14.670523+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:13:14.670523+0000","last_undegraded":"2020-08-07T15:13:14.670523+0000","last_fullsized":"2020-08-07T15:13:14.670523+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:13.231762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.231762+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3","version":"279'5897","reported_seq":"5952","reported_epoch":"279","state":"active+clean","last_fresh":"2020-08-07T15:13:17.743576+0000","last_change":"2020-08-07T15:13:02.600977+0000","last_active":"2020-08-07T15:13:17.743576+0000","last_peered":"2020-08-07T15:13:17.743576+0000","last_clean":"2020-08-07T15:13:17.743576+0000","last_became_active":"2020-08-07T15:13:02.590681+0000","last_became_peered":"2020-08-07T15:13:02.590681+0000","last_unstale":"2020-08-07T15:13:17.743576+0000","last_undegraded":"2020-08-07T15:13:17.743576+0000","last_fullsized":"2020-08-07T15:13:17.743576+0000","mapping_epoch":269,"log_start":"279'5892","ondisk_log_start":"279'5892","created":230,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48308224,"num_objects":738,"num_object_clones":0,"num_object_copies":2214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5897,"num_write_kb":47176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"281","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:15.686324+0000","last_change":"2020-08-07T15:12:27.410339+0000","last_active":"2020-08-07T15:13:15.686324+0000","last_peered":"2020-08-07T15:13:15.686324+0000","last_clean":"2020-08-07T15:13:15.686324+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:13:15.686324+0000","last_undegraded":"2020-08-07T15:13:15.686324+0000","last_fullsized":"2020-08-07T15:13:15.686324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:27.410262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:27.410262+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"208","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.674235+0000","last_change":"2020-08-07T15:12:39.258928+0000","last_active":"2020-08-07T15:13:14.674235+0000","last_peered":"2020-08-07T15:13:14.674235+0000","last_clean":"2020-08-07T15:13:14.674235+0000","last_became_active":"2020-08-07T15:12:32.259956+0000","last_became_peered":"2020-08-07T15:12:32.259956+0000","last_unstale":"2020-08-07T15:13:14.674235+0000","last_undegraded":"2020-08-07T15:13:14.674235+0000","last_fullsized":"2020-08-07T15:13:14.674235+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:39.258857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:39.258857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"278'6480","reported_seq":"6536","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.728874+0000","last_change":"2020-08-07T15:12:10.465048+0000","last_active":"2020-08-07T15:13:16.728874+0000","last_peered":"2020-08-07T15:13:16.728874+0000","last_clean":"2020-08-07T15:13:16.728874+0000","last_became_active":"2020-08-07T15:12:10.464493+0000","last_became_peered":"2020-08-07T15:12:10.464493+0000","last_unstale":"2020-08-07T15:13:16.728874+0000","last_undegraded":"2020-08-07T15:13:16.728874+0000","last_fullsized":"2020-08-07T15:13:16.728874+0000","mapping_epoch":230,"log_start":"278'6477","ondisk_log_start":"278'6477","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53084160,"num_objects":810,"num_object_clones":0,"num_object_copies":2430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6480,"num_write_kb":51840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"235","reported_epoch":"276","state":"active+clean","last_fresh":"2020-08-07T15:13:13.685575+0000","last_change":"2020-08-07T15:12:19.561477+0000","last_active":"2020-08-07T15:13:13.685575+0000","last_peered":"2020-08-07T15:13:13.685575+0000","last_clean":"2020-08-07T15:13:13.685575+0000","last_became_active":"2020-08-07T15:12:06.367376+0000","last_became_peered":"2020-08-07T15:12:06.367376+0000","last_unstale":"2020-08-07T15:13:13.685575+0000","last_undegraded":"2020-08-07T15:13:13.685575+0000","last_fullsized":"2020-08-07T15:13:13.685575+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:12.319383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.319383+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"262","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.682902+0000","last_change":"2020-08-07T15:12:44.858389+0000","last_active":"2020-08-07T15:13:14.682902+0000","last_peered":"2020-08-07T15:13:14.682902+0000","last_clean":"2020-08-07T15:13:14.682902+0000","last_became_active":"2020-08-07T15:12:32.241146+0000","last_became_peered":"2020-08-07T15:12:32.241146+0000","last_unstale":"2020-08-07T15:13:14.682902+0000","last_undegraded":"2020-08-07T15:13:14.682902+0000","last_fullsized":"2020-08-07T15:13:14.682902+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:44.858310+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:44.858310+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"276'5896","reported_seq":"5951","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.676395+0000","last_change":"2020-08-07T15:12:10.464412+0000","last_active":"2020-08-07T15:13:14.676395+0000","last_peered":"2020-08-07T15:13:14.676395+0000","last_clean":"2020-08-07T15:13:14.676395+0000","last_became_active":"2020-08-07T15:12:10.463950+0000","last_became_peered":"2020-08-07T15:12:10.463950+0000","last_unstale":"2020-08-07T15:13:14.676395+0000","last_undegraded":"2020-08-07T15:13:14.676395+0000","last_fullsized":"2020-08-07T15:13:14.676395+0000","mapping_epoch":230,"log_start":"276'5890","ondisk_log_start":"276'5890","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48300032,"num_objects":737,"num_object_clones":0,"num_object_copies":2211,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5896,"num_write_kb":47168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"281","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.673568+0000","last_change":"2020-08-07T15:13:11.273543+0000","last_active":"2020-08-07T15:13:14.673568+0000","last_peered":"2020-08-07T15:13:14.673568+0000","last_clean":"2020-08-07T15:13:14.673568+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:13:14.673568+0000","last_undegraded":"2020-08-07T15:13:14.673568+0000","last_fullsized":"2020-08-07T15:13:14.673568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:11.273469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:13:11.273469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"341","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:15.686255+0000","last_change":"2020-08-07T15:12:53.210176+0000","last_active":"2020-08-07T15:13:15.686255+0000","last_peered":"2020-08-07T15:13:15.686255+0000","last_clean":"2020-08-07T15:13:15.686255+0000","last_became_active":"2020-08-07T15:12:32.239755+0000","last_became_peered":"2020-08-07T15:12:32.239755+0000","last_unstale":"2020-08-07T15:13:15.686255+0000","last_undegraded":"2020-08-07T15:13:15.686255+0000","last_fullsized":"2020-08-07T15:13:15.686255+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:53.210093+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_clean_scrub_stamp":"2020-08-07T15:12:53.210093+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"277'5832","reported_seq":"5884","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.858570+0000","last_change":"2020-08-07T15:12:10.398415+0000","last_active":"2020-08-07T15:13:14.858570+0000","last_peered":"2020-08-07T15:13:14.858570+0000","last_clean":"2020-08-07T15:13:14.858570+0000","last_became_active":"2020-08-07T15:12:10.397834+0000","last_became_peered":"2020-08-07T15:12:10.397834+0000","last_unstale":"2020-08-07T15:13:14.858570+0000","last_undegraded":"2020-08-07T15:13:14.858570+0000","last_fullsized":"2020-08-07T15:13:14.858570+0000","mapping_epoch":230,"log_start":"277'5825","ondisk_log_start":"277'5825","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47775744,"num_objects":729,"num_object_clones":0,"num_object_copies":2187,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5832,"num_write_kb":46656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"250","reported_epoch":"276","state":"active+clean","last_fresh":"2020-08-07T15:13:13.684883+0000","last_change":"2020-08-07T15:13:02.591821+0000","last_active":"2020-08-07T15:13:13.684883+0000","last_peered":"2020-08-07T15:13:13.684883+0000","last_clean":"2020-08-07T15:13:13.684883+0000","last_became_active":"2020-08-07T15:13:02.591402+0000","last_became_peered":"2020-08-07T15:13:02.591402+0000","last_unstale":"2020-08-07T15:13:13.684883+0000","last_undegraded":"2020-08-07T15:13:13.684883+0000","last_fullsized":"2020-08-07T15:13:13.684883+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:08.622683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:08.622683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"310","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:15.686144+0000","last_change":"2020-08-07T15:12:42.371309+0000","last_active":"2020-08-07T15:13:15.686144+0000","last_peered":"2020-08-07T15:13:15.686144+0000","last_clean":"2020-08-07T15:13:15.686144+0000","last_became_active":"2020-08-07T15:12:32.237984+0000","last_became_peered":"2020-08-07T15:12:32.237984+0000","last_unstale":"2020-08-07T15:13:15.686144+0000","last_undegraded":"2020-08-07T15:13:15.686144+0000","last_fullsized":"2020-08-07T15:13:15.686144+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:42.371229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:42.371229+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.7","version":"278'6080","reported_seq":"6136","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.717430+0000","last_change":"2020-08-07T15:12:10.464810+0000","last_active":"2020-08-07T15:13:16.717430+0000","last_peered":"2020-08-07T15:13:16.717430+0000","last_clean":"2020-08-07T15:13:16.717430+0000","last_became_active":"2020-08-07T15:12:10.464050+0000","last_became_peered":"2020-08-07T15:12:10.464050+0000","last_unstale":"2020-08-07T15:13:16.717430+0000","last_undegraded":"2020-08-07T15:13:16.717430+0000","last_fullsized":"2020-08-07T15:13:16.717430+0000","mapping_epoch":230,"log_start":"278'6075","ondisk_log_start":"278'6075","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49807360,"num_objects":760,"num_object_clones":0,"num_object_copies":2280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6080,"num_write_kb":48640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"36","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:15.685876+0000","last_change":"2020-08-07T15:13:13.399410+0000","last_active":"2020-08-07T15:13:15.685876+0000","last_peered":"2020-08-07T15:13:15.685876+0000","last_clean":"2020-08-07T15:13:15.685876+0000","last_became_active":"2020-08-07T15:13:02.590656+0000","last_became_peered":"2020-08-07T15:13:02.590656+0000","last_unstale":"2020-08-07T15:13:15.685876+0000","last_undegraded":"2020-08-07T15:13:15.685876+0000","last_fullsized":"2020-08-07T15:13:15.685876+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:13.399327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:13.399327+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"278","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670917+0000","last_change":"2020-08-07T15:11:40.328534+0000","last_active":"2020-08-07T15:13:14.670917+0000","last_peered":"2020-08-07T15:13:14.670917+0000","last_clean":"2020-08-07T15:13:14.670917+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:13:14.670917+0000","last_undegraded":"2020-08-07T15:13:14.670917+0000","last_fullsized":"2020-08-07T15:13:14.670917+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:11:40.328482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:11:40.328482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"279'6192","reported_seq":"6251","reported_epoch":"279","state":"active+clean","last_fresh":"2020-08-07T15:13:17.651270+0000","last_change":"2020-08-07T15:13:02.591551+0000","last_active":"2020-08-07T15:13:17.651270+0000","last_peered":"2020-08-07T15:13:17.651270+0000","last_clean":"2020-08-07T15:13:17.651270+0000","last_became_active":"2020-08-07T15:13:02.591000+0000","last_became_peered":"2020-08-07T15:13:02.591000+0000","last_unstale":"2020-08-07T15:13:17.651270+0000","last_undegraded":"2020-08-07T15:13:17.651270+0000","last_fullsized":"2020-08-07T15:13:17.651270+0000","mapping_epoch":269,"log_start":"279'6185","ondisk_log_start":"279'6185","created":230,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50724864,"num_objects":774,"num_object_clones":0,"num_object_copies":2322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6192,"num_write_kb":49536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"299","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.683614+0000","last_change":"2020-08-07T15:12:10.585667+0000","last_active":"2020-08-07T15:13:14.683614+0000","last_peered":"2020-08-07T15:13:14.683614+0000","last_clean":"2020-08-07T15:13:14.683614+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:13:14.683614+0000","last_undegraded":"2020-08-07T15:13:14.683614+0000","last_fullsized":"2020-08-07T15:13:14.683614+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:10.585580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:10.585580+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"338","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.673444+0000","last_change":"2020-08-07T15:12:38.850878+0000","last_active":"2020-08-07T15:13:14.673444+0000","last_peered":"2020-08-07T15:13:14.673444+0000","last_clean":"2020-08-07T15:13:14.673444+0000","last_became_active":"2020-08-07T15:12:32.239794+0000","last_became_peered":"2020-08-07T15:12:32.239794+0000","last_unstale":"2020-08-07T15:13:14.673444+0000","last_undegraded":"2020-08-07T15:13:14.673444+0000","last_fullsized":"2020-08-07T15:13:14.673444+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:38.850809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:38.850809+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"278'6136","reported_seq":"6192","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.746670+0000","last_change":"2020-08-07T15:12:10.465113+0000","last_active":"2020-08-07T15:13:16.746670+0000","last_peered":"2020-08-07T15:13:16.746670+0000","last_clean":"2020-08-07T15:13:16.746670+0000","last_became_active":"2020-08-07T15:12:10.464582+0000","last_became_peered":"2020-08-07T15:12:10.464582+0000","last_unstale":"2020-08-07T15:13:16.746670+0000","last_undegraded":"2020-08-07T15:13:16.746670+0000","last_fullsized":"2020-08-07T15:13:16.746670+0000","mapping_epoch":230,"log_start":"278'6131","ondisk_log_start":"278'6131","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50266112,"num_objects":767,"num_object_clones":0,"num_object_copies":2301,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6136,"num_write_kb":49088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"293","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670935+0000","last_change":"2020-08-07T15:12:07.340958+0000","last_active":"2020-08-07T15:13:14.670935+0000","last_peered":"2020-08-07T15:13:14.670935+0000","last_clean":"2020-08-07T15:13:14.670935+0000","last_became_active":"2020-08-07T15:12:06.366967+0000","last_became_peered":"2020-08-07T15:12:06.366967+0000","last_unstale":"2020-08-07T15:13:14.670935+0000","last_undegraded":"2020-08-07T15:13:14.670935+0000","last_fullsized":"2020-08-07T15:13:14.670935+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:07.340855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.340855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"285","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670406+0000","last_change":"2020-08-07T15:12:32.247332+0000","last_active":"2020-08-07T15:13:14.670406+0000","last_peered":"2020-08-07T15:13:14.670406+0000","last_clean":"2020-08-07T15:13:14.670406+0000","last_became_active":"2020-08-07T15:12:32.246800+0000","last_became_peered":"2020-08-07T15:12:32.246800+0000","last_unstale":"2020-08-07T15:13:14.670406+0000","last_undegraded":"2020-08-07T15:13:14.670406+0000","last_fullsized":"2020-08-07T15:13:14.670406+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:09.261101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.261101+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"278'5976","reported_seq":"6032","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.677019+0000","last_change":"2020-08-07T15:12:10.466261+0000","last_active":"2020-08-07T15:13:16.677019+0000","last_peered":"2020-08-07T15:13:16.677019+0000","last_clean":"2020-08-07T15:13:16.677019+0000","last_became_active":"2020-08-07T15:12:10.465583+0000","last_became_peered":"2020-08-07T15:12:10.465583+0000","last_unstale":"2020-08-07T15:13:16.677019+0000","last_undegraded":"2020-08-07T15:13:16.677019+0000","last_fullsized":"2020-08-07T15:13:16.677019+0000","mapping_epoch":230,"log_start":"278'5967","ondisk_log_start":"278'5967","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48955392,"num_objects":747,"num_object_clones":0,"num_object_copies":2241,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5976,"num_write_kb":47808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"247","reported_epoch":"276","state":"active+clean","last_fresh":"2020-08-07T15:13:13.685321+0000","last_change":"2020-08-07T15:13:02.591378+0000","last_active":"2020-08-07T15:13:13.685321+0000","last_peered":"2020-08-07T15:13:13.685321+0000","last_clean":"2020-08-07T15:13:13.685321+0000","last_became_active":"2020-08-07T15:13:02.590901+0000","last_became_peered":"2020-08-07T15:13:02.590901+0000","last_unstale":"2020-08-07T15:13:13.685321+0000","last_undegraded":"2020-08-07T15:13:13.685321+0000","last_fullsized":"2020-08-07T15:13:13.685321+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:07.583152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.583152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"375","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:15.686363+0000","last_change":"2020-08-07T15:12:43.351056+0000","last_active":"2020-08-07T15:13:15.686363+0000","last_peered":"2020-08-07T15:13:15.686363+0000","last_clean":"2020-08-07T15:13:15.686363+0000","last_became_active":"2020-08-07T15:12:32.254842+0000","last_became_peered":"2020-08-07T15:12:32.254842+0000","last_unstale":"2020-08-07T15:13:15.686363+0000","last_undegraded":"2020-08-07T15:13:15.686363+0000","last_fullsized":"2020-08-07T15:13:15.686363+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:43.350971+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:43.350971+0000","last_clean_scrub_stamp":"2020-08-07T15:12:43.350971+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"278'5704","reported_seq":"5758","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.676316+0000","last_change":"2020-08-07T15:12:10.467447+0000","last_active":"2020-08-07T15:13:16.676316+0000","last_peered":"2020-08-07T15:13:16.676316+0000","last_clean":"2020-08-07T15:13:16.676316+0000","last_became_active":"2020-08-07T15:12:10.467000+0000","last_became_peered":"2020-08-07T15:12:10.467000+0000","last_unstale":"2020-08-07T15:13:16.676316+0000","last_undegraded":"2020-08-07T15:13:16.676316+0000","last_fullsized":"2020-08-07T15:13:16.676316+0000","mapping_epoch":230,"log_start":"278'5696","ondisk_log_start":"278'5696","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":46727168,"num_objects":713,"num_object_clones":0,"num_object_copies":2139,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5704,"num_write_kb":45632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"265","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670752+0000","last_change":"2020-08-07T15:13:11.097271+0000","last_active":"2020-08-07T15:13:14.670752+0000","last_peered":"2020-08-07T15:13:14.670752+0000","last_clean":"2020-08-07T15:13:14.670752+0000","last_became_active":"2020-08-07T15:10:46.498230+0000","last_became_peered":"2020-08-07T15:10:46.498230+0000","last_unstale":"2020-08-07T15:13:14.670752+0000","last_undegraded":"2020-08-07T15:13:14.670752+0000","last_fullsized":"2020-08-07T15:13:14.670752+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:11.097214+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:13:11.097214+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"278'5976","reported_seq":"6032","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.640275+0000","last_change":"2020-08-07T15:12:10.465075+0000","last_active":"2020-08-07T15:13:16.640275+0000","last_peered":"2020-08-07T15:13:16.640275+0000","last_clean":"2020-08-07T15:13:16.640275+0000","last_became_active":"2020-08-07T15:12:10.464640+0000","last_became_peered":"2020-08-07T15:12:10.464640+0000","last_unstale":"2020-08-07T15:13:16.640275+0000","last_undegraded":"2020-08-07T15:13:16.640275+0000","last_fullsized":"2020-08-07T15:13:16.640275+0000","mapping_epoch":230,"log_start":"278'5969","ondisk_log_start":"278'5969","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48955392,"num_objects":747,"num_object_clones":0,"num_object_copies":2241,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5976,"num_write_kb":47808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"121","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670174+0000","last_change":"2020-08-07T15:12:45.243933+0000","last_active":"2020-08-07T15:13:14.670174+0000","last_peered":"2020-08-07T15:13:14.670174+0000","last_clean":"2020-08-07T15:13:14.670174+0000","last_became_active":"2020-08-07T15:12:45.243349+0000","last_became_peered":"2020-08-07T15:12:45.243349+0000","last_unstale":"2020-08-07T15:13:14.670174+0000","last_undegraded":"2020-08-07T15:13:14.670174+0000","last_fullsized":"2020-08-07T15:13:14.670174+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.490655+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.490655+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.e","version":"277'5536","reported_seq":"5593","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.834720+0000","last_change":"2020-08-07T15:13:12.351009+0000","last_active":"2020-08-07T15:13:14.834720+0000","last_peered":"2020-08-07T15:13:14.834720+0000","last_clean":"2020-08-07T15:13:14.834720+0000","last_became_active":"2020-08-07T15:12:10.464900+0000","last_became_peered":"2020-08-07T15:12:10.464900+0000","last_unstale":"2020-08-07T15:13:14.834720+0000","last_undegraded":"2020-08-07T15:13:14.834720+0000","last_fullsized":"2020-08-07T15:13:14.834720+0000","mapping_epoch":230,"log_start":"277'5532","ondisk_log_start":"277'5532","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"275'5310","last_scrub_stamp":"2020-08-07T15:13:12.350955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:13:12.350955+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":45350912,"num_objects":692,"num_object_clones":0,"num_object_copies":2076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5536,"num_write_kb":44288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"83","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.674040+0000","last_change":"2020-08-07T15:13:10.415039+0000","last_active":"2020-08-07T15:13:14.674040+0000","last_peered":"2020-08-07T15:13:14.674040+0000","last_clean":"2020-08-07T15:13:14.674040+0000","last_became_active":"2020-08-07T15:13:02.590018+0000","last_became_peered":"2020-08-07T15:13:02.590018+0000","last_unstale":"2020-08-07T15:13:14.674040+0000","last_undegraded":"2020-08-07T15:13:14.674040+0000","last_fullsized":"2020-08-07T15:13:14.674040+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:10.414954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:10.414954+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"279'6272","reported_seq":"6332","reported_epoch":"279","state":"active+remapped+backfill_toofull","last_fresh":"2020-08-07T15:13:17.720161+0000","last_change":"2020-08-07T15:12:57.279725+0000","last_active":"2020-08-07T15:13:17.720161+0000","last_peered":"2020-08-07T15:13:17.720161+0000","last_clean":"2020-08-07T15:12:55.240424+0000","last_became_active":"2020-08-07T15:12:57.254078+0000","last_became_peered":"2020-08-07T15:12:57.254078+0000","last_unstale":"2020-08-07T15:13:17.720161+0000","last_undegraded":"2020-08-07T15:13:17.720161+0000","last_fullsized":"2020-08-07T15:13:17.720161+0000","mapping_epoch":266,"log_start":"279'6264","ondisk_log_start":"279'6264","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":51380224,"num_objects":784,"num_object_clones":0,"num_object_copies":2352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":784,"num_objects_unfound":0,"num_objects_dirty":784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6272,"num_write_kb":50176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,0],"avail_no_missing":["6","0","5"],"object_location_counts":[{"shards":"0,5,6","objects":784}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"296","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.673462+0000","last_change":"2020-08-07T15:12:13.680022+0000","last_active":"2020-08-07T15:13:14.673462+0000","last_peered":"2020-08-07T15:13:14.673462+0000","last_clean":"2020-08-07T15:13:14.673462+0000","last_became_active":"2020-08-07T15:12:06.380392+0000","last_became_peered":"2020-08-07T15:12:06.380392+0000","last_unstale":"2020-08-07T15:13:14.673462+0000","last_undegraded":"2020-08-07T15:13:14.673462+0000","last_fullsized":"2020-08-07T15:13:14.673462+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:13.679943+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.679943+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"278'5877","reported_seq":"5931","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.761337+0000","last_change":"2020-08-07T15:12:10.398816+0000","last_active":"2020-08-07T15:13:16.761337+0000","last_peered":"2020-08-07T15:13:16.761337+0000","last_clean":"2020-08-07T15:13:16.761337+0000","last_became_active":"2020-08-07T15:12:10.398332+0000","last_became_peered":"2020-08-07T15:12:10.398332+0000","last_unstale":"2020-08-07T15:13:16.761337+0000","last_undegraded":"2020-08-07T15:13:16.761337+0000","last_fullsized":"2020-08-07T15:13:16.761337+0000","mapping_epoch":230,"log_start":"278'5872","ondisk_log_start":"278'5872","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48144384,"num_objects":735,"num_object_clones":0,"num_object_copies":2205,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5877,"num_write_kb":47016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"251","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.674265+0000","last_change":"2020-08-07T15:12:14.241339+0000","last_active":"2020-08-07T15:13:14.674265+0000","last_peered":"2020-08-07T15:13:14.674265+0000","last_clean":"2020-08-07T15:13:14.674265+0000","last_became_active":"2020-08-07T15:10:46.498051+0000","last_became_peered":"2020-08-07T15:10:46.498051+0000","last_unstale":"2020-08-07T15:13:14.674265+0000","last_undegraded":"2020-08-07T15:13:14.674265+0000","last_fullsized":"2020-08-07T15:13:14.674265+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:14.241262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:14.241262+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"278'5968","reported_seq":"6025","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.729214+0000","last_change":"2020-08-07T15:13:02.590816+0000","last_active":"2020-08-07T15:13:16.729214+0000","last_peered":"2020-08-07T15:13:16.729214+0000","last_clean":"2020-08-07T15:13:16.729214+0000","last_became_active":"2020-08-07T15:13:02.590300+0000","last_became_peered":"2020-08-07T15:13:02.590300+0000","last_unstale":"2020-08-07T15:13:16.729214+0000","last_undegraded":"2020-08-07T15:13:16.729214+0000","last_fullsized":"2020-08-07T15:13:16.729214+0000","mapping_epoch":269,"log_start":"278'5960","ondisk_log_start":"278'5960","created":230,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":48889856,"num_objects":746,"num_object_clones":0,"num_object_copies":2238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5968,"num_write_kb":47744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3],"acting":[2,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"328","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:15.686300+0000","last_change":"2020-08-07T15:12:41.445456+0000","last_active":"2020-08-07T15:13:15.686300+0000","last_peered":"2020-08-07T15:13:15.686300+0000","last_clean":"2020-08-07T15:13:15.686300+0000","last_became_active":"2020-08-07T15:12:32.241722+0000","last_became_peered":"2020-08-07T15:12:32.241722+0000","last_unstale":"2020-08-07T15:13:15.686300+0000","last_undegraded":"2020-08-07T15:13:15.686300+0000","last_fullsized":"2020-08-07T15:13:15.686300+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:41.445379+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_clean_scrub_stamp":"2020-08-07T15:12:41.445379+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"278'5774","reported_seq":"5827","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.726224+0000","last_change":"2020-08-07T15:13:02.596246+0000","last_active":"2020-08-07T15:13:16.726224+0000","last_peered":"2020-08-07T15:13:16.726224+0000","last_clean":"2020-08-07T15:13:16.726224+0000","last_became_active":"2020-08-07T15:13:02.591329+0000","last_became_peered":"2020-08-07T15:13:02.591329+0000","last_unstale":"2020-08-07T15:13:16.726224+0000","last_undegraded":"2020-08-07T15:13:16.726224+0000","last_fullsized":"2020-08-07T15:13:16.726224+0000","mapping_epoch":269,"log_start":"278'5768","ondisk_log_start":"278'5768","created":230,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":47300608,"num_objects":722,"num_object_clones":0,"num_object_copies":2166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5774,"num_write_kb":46192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"40","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.674376+0000","last_change":"2020-08-07T15:13:11.459633+0000","last_active":"2020-08-07T15:13:14.674376+0000","last_peered":"2020-08-07T15:13:14.674376+0000","last_clean":"2020-08-07T15:13:14.674376+0000","last_became_active":"2020-08-07T15:13:02.591018+0000","last_became_peered":"2020-08-07T15:13:02.591018+0000","last_unstale":"2020-08-07T15:13:14.674376+0000","last_undegraded":"2020-08-07T15:13:14.674376+0000","last_fullsized":"2020-08-07T15:13:14.674376+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:11.459555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:11.459555+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.c","version":"278'6200","reported_seq":"6256","reported_epoch":"278","state":"active+clean","last_fresh":"2020-08-07T15:13:16.701385+0000","last_change":"2020-08-07T15:12:10.465251+0000","last_active":"2020-08-07T15:13:16.701385+0000","last_peered":"2020-08-07T15:13:16.701385+0000","last_clean":"2020-08-07T15:13:16.701385+0000","last_became_active":"2020-08-07T15:12:10.464698+0000","last_became_peered":"2020-08-07T15:12:10.464698+0000","last_unstale":"2020-08-07T15:13:16.701385+0000","last_undegraded":"2020-08-07T15:13:16.701385+0000","last_fullsized":"2020-08-07T15:13:16.701385+0000","mapping_epoch":230,"log_start":"278'6197","ondisk_log_start":"278'6197","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50790400,"num_objects":775,"num_object_clones":0,"num_object_copies":2325,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6200,"num_write_kb":49600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"249","reported_epoch":"277","state":"active+clean","last_fresh":"2020-08-07T15:13:14.670814+0000","last_change":"2020-08-07T15:12:44.203274+0000","last_active":"2020-08-07T15:13:14.670814+0000","last_peered":"2020-08-07T15:13:14.670814+0000","last_clean":"2020-08-07T15:13:14.670814+0000","last_became_active":"2020-08-07T15:12:06.367081+0000","last_became_peered":"2020-08-07T15:12:06.367081+0000","last_unstale":"2020-08-07T15:13:14.670814+0000","last_undegraded":"2020-08-07T15:13:14.670814+0000","last_fullsized":"2020-08-07T15:13:14.670814+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:44.203221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:44.203221+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":784760832,"num_objects":11976,"num_object_clones":0,"num_object_copies":35928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":784,"num_objects_unfound":0,"num_objects_dirty":11976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95796,"num_write_kb":766368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2356854784,"data_stored":2356854784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":41,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":123,"acting":123,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542230,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1296276,"kb_used_data":247692,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93075564,"statfs":{"total":96636764160,"available":95309377536,"internally_reserved":1073741824,"allocated":253636608,"data_stored":252209827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,1,0,2,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":9.5359999999999996,"15min":3.774},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":2.294,"5min":856.07100000000003,"15min":856.07100000000003},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":1.002,"15min":0.97599999999999998},"min":{"1min":0.52300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.55,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":0.84099999999999997}]},{"osd":1,"last update":"Fri Aug 7 15:13:10 2020","interfaces":[{"interface":"back","average":{"1min":10.148999999999999,"5min":10.148999999999999,"15min":10.148999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":205.73699999999999,"5min":205.73699999999999,"15min":205.73699999999999},"last":2.2309999999999999},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":3.827,"5min":3.827,"15min":3.827},"last":1.141}]},{"osd":2,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.321,"15min":1.1910000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.5630000000000002,"5min":5.6619999999999999,"15min":5.6619999999999999},"last":1.675},{"interface":"front","average":{"1min":1.081,"5min":1.3280000000000001,"15min":1.1879999999999999},"min":{"1min":0.69299999999999995,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.3769999999999998,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":1.1719999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":30.957999999999998,"5min":21.91,"15min":8.4179999999999993},"min":{"1min":0.68899999999999995,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":656.99099999999999,"5min":917.38699999999994,"15min":917.38699999999994},"last":1.946},{"interface":"front","average":{"1min":1.173,"5min":4.8879999999999999,"15min":2.524},"min":{"1min":0.61599999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":4.0460000000000003,"5min":333.59199999999998,"15min":333.59199999999998},"last":1.5940000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":8.8599999999999994,"5min":2.944,"15min":1.8320000000000001},"min":{"1min":0.66000000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":172.012,"5min":172.012,"15min":172.012},"last":1.196},{"interface":"front","average":{"1min":1.5509999999999999,"5min":11.529,"15min":4.7169999999999996},"min":{"1min":0.81200000000000006,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":5.4500000000000002,"5min":866.24099999999999,"15min":866.24099999999999},"last":0.86799999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":9.3930000000000007,"15min":28.135000000000002},"min":{"1min":0.68400000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":3.536,"5min":678.76099999999997,"15min":831.88099999999997},"last":2.0299999999999998},{"interface":"front","average":{"1min":1.623,"5min":1.831,"15min":1.607},"min":{"1min":0.89000000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":3.593,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":2.056}]},{"osd":7,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":3.9929999999999999,"15min":2.343},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":4.0810000000000004,"5min":224.14500000000001,"15min":224.14500000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.671,"5min":1.7649999999999999,"15min":1.593},"min":{"1min":0.92000000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":3.6200000000000001,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":1.99}]}]},{"osd":7,"up_from":13,"seq":55834574935,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1331576,"kb_used_data":282992,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93040264,"statfs":{"total":96636764160,"available":95273230336,"internally_reserved":1073741824,"allocated":289783808,"data_stored":288364486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":37.372999999999998,"5min":16.672000000000001,"15min":6.4459999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":763.39099999999996,"5min":763.39099999999996,"15min":763.39099999999996},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.157,"5min":9.7789999999999999,"15min":4.0259999999999998},"min":{"1min":0.61699999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.8279999999999998,"5min":529.88699999999994,"15min":529.88699999999994},"last":1.1299999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:13:05 2020","interfaces":[{"interface":"back","average":{"1min":23.431000000000001,"5min":23.431000000000001,"15min":23.431000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":472.59699999999998,"5min":472.59699999999998,"15min":472.59699999999998},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.8650000000000002,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.3109999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":3.0009999999999999,"5min":1.6850000000000001,"15min":1.3580000000000001},"min":{"1min":0.52000000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":33.131999999999998,"5min":33.131999999999998,"15min":33.131999999999998},"last":0.84899999999999998},{"interface":"front","average":{"1min":2.8410000000000002,"5min":1.494,"15min":1.3460000000000001},"min":{"1min":0.61799999999999999,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":32.924999999999997,"5min":32.924999999999997,"15min":32.924999999999997},"last":1.169}]},{"osd":3,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":9.4659999999999993,"15min":4.1719999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":5.9329999999999998,"5min":698.19899999999996,"15min":698.19899999999996},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.238,"5min":1.2669999999999999,"15min":1.347},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":3.8599999999999999,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":1.903}]},{"osd":4,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":25.448,"5min":6.069,"15min":3.1309999999999998},"min":{"1min":0.76400000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":460.78399999999999,"5min":460.78399999999999,"15min":460.78399999999999},"last":1.347},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.474,"15min":1.6060000000000001},"min":{"1min":0.86899999999999999,"5min":0.621,"15min":0.621},"max":{"1min":2.4079999999999999,"5min":3.694,"15min":5.2990000000000004},"last":668.06799999999998}]},{"osd":5,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.3720000000000001,"15min":1.3660000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":5.9989999999999997,"5min":6.2080000000000002,"15min":6.2080000000000002},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.492,"15min":1.6739999999999999},"min":{"1min":0.83599999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":4.407,"5min":4.407,"15min":5.327},"last":1.3779999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":9.1839999999999993,"15min":4.016},"min":{"1min":0.88,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.3290000000000002,"5min":695.92899999999997,"15min":695.92899999999997},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.597,"5min":1.55,"15min":12.734},"min":{"1min":0.89500000000000002,"5min":0.87,"15min":0.87},"max":{"1min":3.302,"5min":3.7309999999999999,"15min":266.26999999999998},"last":1.4299999999999999}]}]},{"osd":1,"up_from":227,"seq":974957576207,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1275548,"kb_used_data":226964,"kb_used_omap":2,"kb_used_meta":1048573,"kb_avail":93096292,"statfs":{"total":96636764160,"available":95330603008,"internally_reserved":1073741824,"allocated":232411136,"data_stored":231010265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":1073739245},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":27.861000000000001,"5min":27.861000000000001,"15min":27.861000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":540.92499999999995,"5min":540.92499999999995,"15min":540.92499999999995},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":3.7069999999999999,"5min":3.7069999999999999,"15min":3.7069999999999999},"last":1.1910000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":2.5779999999999998},{"interface":"front","average":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":3.758,"5min":3.758,"15min":3.758},"last":3.1760000000000002}]},{"osd":3,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":13.909000000000001,"5min":13.909000000000001,"15min":13.909000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":250.995,"5min":250.995,"15min":250.995},"last":2.5510000000000002},{"interface":"front","average":{"1min":2.04,"5min":2.04,"15min":2.04},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":6.6580000000000004,"5min":6.6580000000000004,"15min":6.6580000000000004},"last":2.9990000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":2.6230000000000002},{"interface":"front","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":1.746}]},{"osd":5,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":2.0150000000000001},{"interface":"front","average":{"1min":1.498,"5min":1.498,"15min":1.498},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.6720000000000002,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":2.665}]},{"osd":6,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.1019999999999999},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":9.4299999999999997,"5min":9.4299999999999997,"15min":9.4299999999999997},"last":2.9630000000000001},{"interface":"front","average":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":3.4199999999999999,"5min":3.4199999999999999,"15min":3.4199999999999999},"last":2.5299999999999998}]},{"osd":7,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":40.520000000000003,"5min":40.520000000000003,"15min":40.520000000000003},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":778.745,"5min":778.745,"15min":778.745},"last":3.0529999999999999},{"interface":"front","average":{"1min":24.100000000000001,"5min":24.100000000000001,"15min":24.100000000000001},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":442.62799999999999,"5min":442.62799999999999,"15min":442.62799999999999},"last":3.0299999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574935,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1236688,"kb_used_data":188104,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93135152,"statfs":{"total":96636764160,"available":95370395648,"internally_reserved":1073741824,"allocated":192618496,"data_stored":191199174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:13:07 2020","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.468,"5min":2.468,"15min":2.468},"last":3.5249999999999999},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.5550000000000002,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":2.9460000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.0620000000000001,"15min":0.97199999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":7.3470000000000004,"5min":7.3470000000000004,"15min":7.3470000000000004},"last":3.6099999999999999},{"interface":"front","average":{"1min":6.2839999999999998,"5min":4.665,"15min":23.381},"min":{"1min":0.47399999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":75.622,"5min":226.911,"15min":606.18499999999995},"last":2.2709999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":28.079000000000001,"5min":6.6920000000000002,"15min":2.8540000000000001},"min":{"1min":0.60999999999999999,"5min":0.54100000000000004,"15min":0.52300000000000002},"max":{"1min":431.95299999999997,"5min":431.95299999999997,"15min":431.95299999999997},"last":4.1619999999999999},{"interface":"front","average":{"1min":1.425,"5min":1.46,"15min":1.2649999999999999},"min":{"1min":0.66400000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":5.3780000000000001,"5min":11.19,"15min":11.19},"last":4.3449999999999998}]},{"osd":4,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.302,"15min":1.2110000000000001},"min":{"1min":0.57399999999999995,"5min":0.501,"15min":0.501},"max":{"1min":1.819,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.304,"5min":1.448,"15min":1.321},"min":{"1min":0.71099999999999997,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.8849999999999998,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":3.5579999999999998}]},{"osd":5,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.6200000000000001,"5min":14.598000000000001,"15min":5.6840000000000002},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":7.3799999999999999,"5min":539.31100000000004,"15min":539.31100000000004},"last":2.6259999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.49,"15min":1.234},"min":{"1min":0.81399999999999995,"5min":0.752,"15min":0.65900000000000003},"max":{"1min":2.9420000000000002,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":0.78700000000000003}]},{"osd":6,"last update":"Fri Aug 7 15:12:34 2020","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.4399999999999999,"15min":6.2220000000000004},"min":{"1min":0.80200000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.8610000000000002,"5min":4.1310000000000002,"15min":124.738},"last":3.585},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.5549999999999999,"15min":12.952999999999999},"min":{"1min":0.73499999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":5.274,"5min":5.274,"15min":294.94},"last":2.6629999999999998}]},{"osd":7,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.4790000000000001,"15min":1.55},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.79300000000000004},"max":{"1min":1.8620000000000001,"5min":4.7169999999999996,"15min":6.468},"last":2.7149999999999999},{"interface":"front","average":{"1min":66.063000000000002,"5min":23.518000000000001,"15min":8.7629999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":971.12699999999995,"5min":971.12699999999995,"15min":971.12699999999995},"last":2.6890000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574935,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1432128,"kb_used_data":383544,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92939712,"statfs":{"total":96636764160,"available":95170265088,"internally_reserved":1073741824,"allocated":392749056,"data_stored":391329734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":32.965000000000003,"5min":7.8639999999999999,"15min":3.3660000000000001},"min":{"1min":0.41899999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":501.53800000000001,"5min":501.53800000000001,"15min":501.53800000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":27.341999999999999,"5min":6.6900000000000004,"15min":2.8799999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":501.36000000000001,"5min":501.36000000000001,"15min":501.36000000000001},"last":0.60199999999999998}]},{"osd":1,"last update":"Fri Aug 7 15:13:07 2020","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":5.7869999999999999,"5min":5.7869999999999999,"15min":5.7869999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":24.699999999999999,"5min":24.699999999999999,"15min":24.699999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":465.69200000000001,"5min":465.69200000000001,"15min":465.69200000000001},"last":0.88400000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.2130000000000001,"15min":1.179},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":23.064,"5min":15.281000000000001,"15min":6.1310000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":412.61500000000001,"5min":501.75599999999997,"15min":501.75599999999997},"last":0.748}]},{"osd":4,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.5940000000000001,"15min":1.2709999999999999},"min":{"1min":0.67400000000000004,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.712,"5min":19.783000000000001,"15min":19.783000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.4610000000000001,"15min":1.5820000000000001},"min":{"1min":0.68999999999999995,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8979999999999999,"5min":8.048,"15min":8.048},"last":0.95799999999999996}]},{"osd":5,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.3460000000000001,"15min":1.29},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.4009999999999998,"5min":5.2220000000000004,"15min":5.2220000000000004},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.4990000000000001,"5min":15.715,"15min":22.254999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":8.0649999999999995,"5min":627.53300000000002,"15min":627.53300000000002},"last":0.93500000000000005}]},{"osd":6,"last update":"Fri Aug 7 15:12:37 2020","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":9.9480000000000004,"15min":4.1299999999999999},"min":{"1min":0.875,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.9450000000000001,"5min":821.91899999999998,"15min":821.91899999999998},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.3959999999999999,"5min":9.9580000000000002,"15min":4.2000000000000002},"min":{"1min":0.70199999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.9540000000000002,"5min":751.70000000000005,"15min":751.70000000000005},"last":0.97999999999999998}]},{"osd":7,"last update":"Fri Aug 7 15:12:25 2020","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":12.522,"15min":5.335},"min":{"1min":0.85599999999999998,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.3519999999999999,"5min":501.91699999999997,"15min":501.91699999999997},"last":1.113},{"interface":"front","average":{"1min":1.7230000000000001,"5min":2.1080000000000001,"15min":1.7},"min":{"1min":0.88500000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":7.2149999999999999,"5min":35.646999999999998,"15min":35.646999999999998},"last":1.143}]}]},{"osd":3,"up_from":13,"seq":55834574935,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1535456,"kb_used_data":486872,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92836384,"statfs":{"total":96636764160,"available":95064457216,"internally_reserved":1073741824,"allocated":498556928,"data_stored":497137606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,5,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.226,"15min":1.244},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":4.6580000000000004,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.2050000000000001,"15min":1.4930000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":4.6349999999999998,"5min":4.9119999999999999,"15min":13.067},"last":1.157}]},{"osd":1,"last update":"Fri Aug 7 15:13:08 2020","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.639,"15min":1.639},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":4.6890000000000001,"5min":4.6890000000000001,"15min":4.6890000000000001},"last":1.234},{"interface":"front","average":{"1min":1.841,"5min":1.841,"15min":1.841},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":5.0330000000000004,"5min":5.0330000000000004,"15min":5.0330000000000004},"last":3.9460000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.365,"15min":1.22},"min":{"1min":0.65900000000000003,"5min":0.56999999999999995,"15min":0.56899999999999995},"max":{"1min":2.8300000000000001,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":4.181},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.9690000000000001,"15min":1.7789999999999999},"min":{"1min":0.64500000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":5.3090000000000002,"5min":51.530000000000001,"15min":51.530000000000001},"last":1.8580000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":3.831,"5min":17.792999999999999,"15min":7.0620000000000003},"min":{"1min":0.54500000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":56.688000000000002,"5min":646.26700000000005,"15min":646.26700000000005},"last":5.2720000000000002},{"interface":"front","average":{"1min":1.421,"5min":15.494999999999999,"15min":26.565000000000001},"min":{"1min":0.68100000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":3.2109999999999999,"5min":844.10500000000002,"15min":844.10500000000002},"last":3.988}]},{"osd":5,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":14.452999999999999,"5min":8.5429999999999993,"15min":3.859},"min":{"1min":0.95499999999999996,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":244.08099999999999,"5min":403.88400000000001,"15min":403.88400000000001},"last":4.1429999999999998},{"interface":"front","average":{"1min":1.78,"5min":1.8979999999999999,"15min":1.8120000000000001},"min":{"1min":0.98499999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":4.5960000000000001,"5min":8.9120000000000008,"15min":8.9120000000000008},"last":4.2409999999999997}]},{"osd":6,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.7969999999999999,"5min":1.706,"15min":1.5900000000000001},"min":{"1min":0.74099999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":5.2729999999999997,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":1.609},{"interface":"front","average":{"1min":1.907,"5min":13.411,"15min":5.7949999999999999},"min":{"1min":1.032,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":5.3460000000000001,"5min":688.15999999999997,"15min":688.15999999999997},"last":1.635}]},{"osd":7,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.7450000000000001,"5min":1.5960000000000001,"15min":1.9730000000000001},"min":{"1min":0.85599999999999998,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":4.8099999999999996,"5min":6.8209999999999997,"15min":6.8209999999999997},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.6559999999999999,"15min":1.8700000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.6099999999999999,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":5.3109999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574935,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1293584,"kb_used_data":245000,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93078256,"statfs":{"total":96636764160,"available":95312134144,"internally_reserved":1073741824,"allocated":250880000,"data_stored":249460678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":49.795000000000002,"5min":19.646000000000001,"15min":7.3360000000000003},"min":{"1min":0.46800000000000003,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":829.15099999999995,"5min":829.15099999999995,"15min":829.15099999999995},"last":0.58799999999999997},{"interface":"front","average":{"1min":3.8149999999999999,"5min":1.639,"15min":1.1120000000000001},"min":{"1min":0.45300000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":48.274000000000001,"5min":48.274000000000001,"15min":48.274000000000001},"last":0.64600000000000002}]},{"osd":1,"last update":"Fri Aug 7 15:13:08 2020","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.80800000000000005}]},{"osd":2,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":4.1159999999999997,"5min":1.901,"15min":1.601},"min":{"1min":0.64300000000000002,"5min":0.56399999999999995,"15min":0.52100000000000002},"max":{"1min":49.878999999999998,"5min":49.878999999999998,"15min":49.878999999999998},"last":0.78700000000000003},{"interface":"front","average":{"1min":4.2270000000000003,"5min":2.0529999999999999,"15min":1.694},"min":{"1min":0.97899999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":49.939999999999998,"5min":49.939999999999998,"15min":49.939999999999998},"last":0.84999999999999998}]},{"osd":3,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":27.59,"5min":17.260999999999999,"15min":6.7590000000000003},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":426.50799999999998,"5min":638.92100000000005,"15min":638.92100000000005},"last":1.595},{"interface":"front","average":{"1min":1.411,"5min":1.3799999999999999,"15min":1.294},"min":{"1min":0.75700000000000001,"5min":0.67800000000000005,"15min":0.66700000000000004},"max":{"1min":3.2360000000000002,"5min":6.431,"15min":6.431},"last":1.153}]},{"osd":5,"last update":"Fri Aug 7 15:12:29 2020","interfaces":[{"interface":"back","average":{"1min":52.308999999999997,"5min":28.574999999999999,"15min":10.609999999999999},"min":{"1min":0.56100000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":862.97299999999996,"5min":990.71600000000001,"15min":990.71600000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":49.549999999999997,"5min":11.01,"15min":4.7539999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":780.00699999999995,"5min":780.00699999999995,"15min":780.00699999999995},"last":0.95599999999999996}]},{"osd":6,"last update":"Fri Aug 7 15:12:34 2020","interfaces":[{"interface":"back","average":{"1min":3.8239999999999998,"5min":1.9870000000000001,"15min":1.855},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":48.575000000000003,"5min":48.575000000000003,"15min":48.575000000000003},"last":1.179},{"interface":"front","average":{"1min":1.109,"5min":1.333,"15min":13.183},"min":{"1min":0.64600000000000002,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.847,"5min":6.7729999999999997,"15min":389.14600000000002},"last":0.92900000000000005}]},{"osd":7,"last update":"Fri Aug 7 15:12:28 2020","interfaces":[{"interface":"back","average":{"1min":4.1349999999999998,"5min":4.327,"15min":2.3809999999999998},"min":{"1min":0.74199999999999999,"5min":0.62,"15min":0.62},"max":{"1min":48.75,"5min":252.798,"15min":252.798},"last":1.728},{"interface":"front","average":{"1min":4.2610000000000001,"5min":14.635,"15min":5.9749999999999996},"min":{"1min":0.84799999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":49.654000000000003,"5min":501.93000000000001,"15min":501.93000000000001},"last":1.486}]}]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1289648,"kb_used_data":241064,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93082192,"statfs":{"total":96636764160,"available":95316164608,"internally_reserved":1073741824,"allocated":246849536,"data_stored":245430214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":3.629,"5min":1.506,"15min":1.22},"min":{"1min":0.42199999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":43.292999999999999,"5min":43.292999999999999,"15min":43.292999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":3.1520000000000001,"5min":4.4249999999999998,"15min":2.0110000000000001},"min":{"1min":0.44700000000000001,"5min":0.438,"15min":0.438},"max":{"1min":42.933,"5min":329.11399999999998,"15min":329.11399999999998},"last":2.5049999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:13:11 2020","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":4.4039999999999999,"5min":4.4039999999999999,"15min":4.4039999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":4.0810000000000004,"5min":4.0810000000000004,"15min":4.0810000000000004},"last":0.95899999999999996}]},{"osd":2,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":3.5499999999999998,"5min":1.5800000000000001,"15min":6.673},"min":{"1min":0.65700000000000003,"5min":0.499,"15min":0.499},"max":{"1min":43.923000000000002,"5min":43.923000000000002,"15min":168.56899999999999},"last":1.1180000000000001},{"interface":"front","average":{"1min":3.23,"5min":1.6399999999999999,"15min":22.634},"min":{"1min":0.52800000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":43.076999999999998,"5min":43.076999999999998,"15min":676.27099999999996},"last":2.5430000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.728,"15min":1.321},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":4.7640000000000002,"5min":29.129000000000001,"15min":29.129000000000001},"last":1.377},{"interface":"front","average":{"1min":1.3779999999999999,"5min":10.029999999999999,"15min":4.1520000000000001},"min":{"1min":0.73999999999999999,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":5.2009999999999996,"5min":712.98699999999997,"15min":712.98699999999997},"last":1.1539999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":12.779999999999999,"15min":38.188000000000002},"min":{"1min":0.77200000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":4.9969999999999999,"5min":971.67999999999995,"15min":971.67999999999995},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.2470000000000001,"15min":1.286},"min":{"1min":0.73599999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":3.1160000000000001,"5min":5.4900000000000002,"15min":5.4900000000000002},"last":2.581}]},{"osd":6,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":35.393000000000001,"5min":28.459,"15min":43.091999999999999},"min":{"1min":0.72199999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":676.20000000000005,"5min":966.60900000000004,"15min":966.60900000000004},"last":1.579},{"interface":"front","average":{"1min":1.528,"5min":6.7789999999999999,"15min":19.603999999999999},"min":{"1min":0.89500000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":5.2220000000000004,"5min":501.79000000000002,"15min":517.26700000000005},"last":1.9219999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:12:27 2020","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.462,"15min":1.3600000000000001},"min":{"1min":0.85399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":4.8600000000000003,"5min":5.1559999999999997,"15min":5.1559999999999997},"last":208.03399999999999},{"interface":"front","average":{"1min":1.5669999999999999,"5min":12.834,"15min":5.3220000000000001},"min":{"1min":0.70099999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":3.8999999999999999,"5min":515.09100000000001,"15min":515.09100000000001},"last":2.7069999999999999}]}]}]}} 2020-08-07T15:13:19.666 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:19.648+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:19.679 INFO:tasks.thrashosds.thrasher:Setting 1.1d to [1, 5, 0] 2020-08-07T15:13:19.679 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '1.1d', '1', '5', '0'] 2020-08-07T15:13:19.680 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pg-upmap 1.1d 1 5 0 2020-08-07T15:13:19.681 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:13:19.753 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:19.750+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:19.826 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:13:19.848 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:19.849 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:13:19.849 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:13:19.849 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:13:19.849 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:19.852 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:19.852+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:19.860 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:13:19.874 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:13:19.880 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:13:19.880 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:13:19.881 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:13:19.881 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:13:19.881 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:13:19.881 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:13:19.881 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:13:19.882 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:13:19.954 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:19.953+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:20.060 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:20.054+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:20.093 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:20.094 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:13:20.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:13:20.094 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:13:20.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.095 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42440 4.9 4:9b3fd4e3:::benchmark_data_smithi074_44586_object5304:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:20.095 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:13:20.095 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 259.84853943399997, 2020-08-07T15:13:20.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9384075030000001, 2020-08-07T15:13:20.095 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:20.096 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:20.096 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:20.096 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:20.096 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:20.096 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42440 2020-08-07T15:13:20.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.097 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:20.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:20.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:13:20.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:20.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:20.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:13:20.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6295000000000002e-05 2020-08-07T15:13:20.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:20.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237116+0000", 2020-08-07T15:13:20.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0390000000000001e-06 2020-08-07T15:13:20.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:20.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237122+0000", 2020-08-07T15:13:20.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8340000000000001e-05 2020-08-07T15:13:20.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:20.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237140+0000", 2020-08-07T15:13:20.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.045e-06 2020-08-07T15:13:20.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:20.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237145+0000", 2020-08-07T15:13:20.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044548920000000002 2020-08-07T15:13:20.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241600+0000", 2020-08-07T15:13:20.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103336 2020-08-07T15:13:20.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:20.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241703+0000", 2020-08-07T15:13:20.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018660399999999999 2020-08-07T15:13:20.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:20.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241890+0000", 2020-08-07T15:13:20.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00096796000000000002 2020-08-07T15:13:20.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:20.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.242858+0000", 2020-08-07T15:13:20.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039845000000000002 2020-08-07T15:13:20.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246843+0000", 2020-08-07T15:13:20.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3499999999999996e-07 2020-08-07T15:13:20.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:20.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246843+0000", 2020-08-07T15:13:20.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7132999999999995e-05 2020-08-07T15:13:20.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:20.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246920+0000", 2020-08-07T15:13:20.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046993410000000001 2020-08-07T15:13:20.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.251620+0000", 2020-08-07T15:13:20.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040332956000000003 2020-08-07T15:13:20.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.291953+0000", 2020-08-07T15:13:20.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8390000000000003e-06 2020-08-07T15:13:20.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.291958+0000", 2020-08-07T15:13:20.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99961357500000003 2020-08-07T15:13:20.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291572+0000", 2020-08-07T15:13:20.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3913999999999999e-05 2020-08-07T15:13:20.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291586+0000", 2020-08-07T15:13:20.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88379723099999996 2020-08-07T15:13:20.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175383+0000", 2020-08-07T15:13:20.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3868000000000004e-05 2020-08-07T15:13:20.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:20.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175467+0000", 2020-08-07T15:13:20.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9384075030000001 2020-08-07T15:13:20.128 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.129 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:20.129 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.130 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42489 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:20.132 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:13:20.133 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 259.80523937200002, 2020-08-07T15:13:20.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952608639999999, 2020-08-07T15:13:20.133 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:20.133 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:20.133 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:20.134 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:20.134 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:20.134 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42489 2020-08-07T15:13:20.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.134 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:20.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:20.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:13:20.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:20.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:20.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:13:20.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0379e-05 2020-08-07T15:13:20.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:20.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280410+0000", 2020-08-07T15:13:20.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0489999999999998e-06 2020-08-07T15:13:20.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:20.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280417+0000", 2020-08-07T15:13:20.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8281e-05 2020-08-07T15:13:20.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:20.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280435+0000", 2020-08-07T15:13:20.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7510000000000008e-06 2020-08-07T15:13:20.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:20.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280443+0000", 2020-08-07T15:13:20.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018142899999999999 2020-08-07T15:13:20.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280624+0000", 2020-08-07T15:13:20.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013473599999999999 2020-08-07T15:13:20.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:20.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280759+0000", 2020-08-07T15:13:20.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019964200000000001 2020-08-07T15:13:20.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:20.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280959+0000", 2020-08-07T15:13:20.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005588038 2020-08-07T15:13:20.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286547+0000", 2020-08-07T15:13:20.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.376e-05 2020-08-07T15:13:20.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:20.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286561+0000", 2020-08-07T15:13:20.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4199999999999995e-07 2020-08-07T15:13:20.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:20.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286561+0000", 2020-08-07T15:13:20.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9218e-05 2020-08-07T15:13:20.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:20.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286631+0000", 2020-08-07T15:13:20.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027909199999999999 2020-08-07T15:13:20.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286910+0000", 2020-08-07T15:13:20.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050904089999999997 2020-08-07T15:13:20.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292000+0000", 2020-08-07T15:13:20.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9579999999999996e-06 2020-08-07T15:13:20.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292009+0000", 2020-08-07T15:13:20.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99961402799999999 2020-08-07T15:13:20.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291623+0000", 2020-08-07T15:13:20.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2490000000000001e-06 2020-08-07T15:13:20.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291631+0000", 2020-08-07T15:13:20.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88387422199999999 2020-08-07T15:13:20.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175506+0000", 2020-08-07T15:13:20.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000114881 2020-08-07T15:13:20.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:20.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175620+0000", 2020-08-07T15:13:20.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952608639999999 2020-08-07T15:13:20.167 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.167 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:20.167 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.168 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42490 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:20.168 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:13:20.168 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 259.80511220400001, 2020-08-07T15:13:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89528059, 2020-08-07T15:13:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:20.170 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:20.170 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42490 2020-08-07T15:13:20.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.170 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:20.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:20.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:13:20.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:20.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:20.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:13:20.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2571999999999997e-05 2020-08-07T15:13:20.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:20.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280539+0000", 2020-08-07T15:13:20.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999998e-06 2020-08-07T15:13:20.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:20.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280545+0000", 2020-08-07T15:13:20.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.134e-05 2020-08-07T15:13:20.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:20.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280556+0000", 2020-08-07T15:13:20.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.746e-06 2020-08-07T15:13:20.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:20.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280561+0000", 2020-08-07T15:13:20.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00089258999999999996 2020-08-07T15:13:20.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281453+0000", 2020-08-07T15:13:20.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5641000000000001e-05 2020-08-07T15:13:20.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:20.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281549+0000", 2020-08-07T15:13:20.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020555200000000001 2020-08-07T15:13:20.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:20.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281755+0000", 2020-08-07T15:13:20.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048878230000000003 2020-08-07T15:13:20.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:20.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286642+0000", 2020-08-07T15:13:20.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0954e-05 2020-08-07T15:13:20.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286693+0000", 2020-08-07T15:13:20.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0699999999999997e-07 2020-08-07T15:13:20.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:20.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286694+0000", 2020-08-07T15:13:20.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3697999999999998e-05 2020-08-07T15:13:20.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:20.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286768+0000", 2020-08-07T15:13:20.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017185900000000001 2020-08-07T15:13:20.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286940+0000", 2020-08-07T15:13:20.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051207819999999999 2020-08-07T15:13:20.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292060+0000", 2020-08-07T15:13:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8000000000000004e-06 2020-08-07T15:13:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292069+0000", 2020-08-07T15:13:20.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99959520599999996 2020-08-07T15:13:20.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291664+0000", 2020-08-07T15:13:20.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0160000000000008e-06 2020-08-07T15:13:20.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291672+0000", 2020-08-07T15:13:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88399374500000005 2020-08-07T15:13:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175666+0000", 2020-08-07T15:13:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101239 2020-08-07T15:13:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175767+0000", 2020-08-07T15:13:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89528059 2020-08-07T15:13:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42491 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:13:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 259.80500526899999, 2020-08-07T15:13:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895291936, 2020-08-07T15:13:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42491 2020-08-07T15:13:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:13:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:13:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0402000000000001e-05 2020-08-07T15:13:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280624+0000", 2020-08-07T15:13:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0340000000000001e-06 2020-08-07T15:13:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280627+0000", 2020-08-07T15:13:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1886999999999999e-05 2020-08-07T15:13:20.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280649+0000", 2020-08-07T15:13:20.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5430000000000004e-06 2020-08-07T15:13:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280654+0000", 2020-08-07T15:13:20.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032800960000000001 2020-08-07T15:13:20.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.283934+0000", 2020-08-07T15:13:20.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011264699999999999 2020-08-07T15:13:20.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:20.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284046+0000", 2020-08-07T15:13:20.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019987299999999999 2020-08-07T15:13:20.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:20.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284246+0000", 2020-08-07T15:13:20.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025610960000000001 2020-08-07T15:13:20.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286807+0000", 2020-08-07T15:13:20.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1409999999999999e-06 2020-08-07T15:13:20.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:20.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286814+0000", 2020-08-07T15:13:20.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0499999999999999e-07 2020-08-07T15:13:20.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:20.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286815+0000", 2020-08-07T15:13:20.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8538000000000003e-05 2020-08-07T15:13:20.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:20.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286853+0000", 2020-08-07T15:13:20.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112914 2020-08-07T15:13:20.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286966+0000", 2020-08-07T15:13:20.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051403120000000002 2020-08-07T15:13:20.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292107+0000", 2020-08-07T15:13:20.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8019999999999992e-06 2020-08-07T15:13:20.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292114+0000", 2020-08-07T15:13:20.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.999590859 2020-08-07T15:13:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291705+0000", 2020-08-07T15:13:20.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.174e-06 2020-08-07T15:13:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291713+0000", 2020-08-07T15:13:20.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88407676000000002 2020-08-07T15:13:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175790+0000", 2020-08-07T15:13:20.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5452999999999996e-05 2020-08-07T15:13:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175886+0000", 2020-08-07T15:13:20.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895291936 2020-08-07T15:13:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.232 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.233 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42492 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:13:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 259.80491221900002, 2020-08-07T15:13:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895293774, 2020-08-07T15:13:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:20.234 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42492 2020-08-07T15:13:20.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:13:20.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:20.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:13:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8272000000000001e-05 2020-08-07T15:13:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:20.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280735+0000", 2020-08-07T15:13:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2159999999999997e-06 2020-08-07T15:13:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:20.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280740+0000", 2020-08-07T15:13:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0251000000000001e-05 2020-08-07T15:13:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280750+0000", 2020-08-07T15:13:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3139999999999997e-06 2020-08-07T15:13:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280755+0000", 2020-08-07T15:13:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039343579999999998 2020-08-07T15:13:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284689+0000", 2020-08-07T15:13:20.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111648 2020-08-07T15:13:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284801+0000", 2020-08-07T15:13:20.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019563600000000001 2020-08-07T15:13:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:20.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284996+0000", 2020-08-07T15:13:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018636709999999999 2020-08-07T15:13:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:20.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286860+0000", 2020-08-07T15:13:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001865802 2020-08-07T15:13:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288726+0000", 2020-08-07T15:13:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0399999999999996e-07 2020-08-07T15:13:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:20.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288726+0000", 2020-08-07T15:13:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6573999999999998e-05 2020-08-07T15:13:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288773+0000", 2020-08-07T15:13:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003455283 2020-08-07T15:13:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292228+0000", 2020-08-07T15:13:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0099999999999995e-06 2020-08-07T15:13:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292236+0000", 2020-08-07T15:13:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99950963999999998 2020-08-07T15:13:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291746+0000", 2020-08-07T15:13:20.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0779999999999996e-06 2020-08-07T15:13:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291754+0000", 2020-08-07T15:13:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.884152508 2020-08-07T15:13:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175906+0000", 2020-08-07T15:13:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4009000000000002e-05 2020-08-07T15:13:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175980+0000", 2020-08-07T15:13:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895293774 2020-08-07T15:13:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:20.257 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42493 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:13:20.258 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 259.80481057200001, 2020-08-07T15:13:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952963789999999, 2020-08-07T15:13:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:20.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42493 2020-08-07T15:13:20.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:13:20.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:20.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:13:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9740999999999997e-05 2020-08-07T15:13:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280838+0000", 2020-08-07T15:13:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7809999999999996e-06 2020-08-07T15:13:20.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280843+0000", 2020-08-07T15:13:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0648999999999999e-05 2020-08-07T15:13:20.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280854+0000", 2020-08-07T15:13:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3669999999999999e-06 2020-08-07T15:13:20.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280858+0000", 2020-08-07T15:13:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004850867 2020-08-07T15:13:20.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.285709+0000", 2020-08-07T15:13:20.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110037 2020-08-07T15:13:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.285819+0000", 2020-08-07T15:13:20.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020935699999999999 2020-08-07T15:13:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286028+0000", 2020-08-07T15:13:20.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020900850000000002 2020-08-07T15:13:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288118+0000", 2020-08-07T15:13:20.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041669649999999999 2020-08-07T15:13:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292285+0000", 2020-08-07T15:13:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3820000000000004e-06 2020-08-07T15:13:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292294+0000", 2020-08-07T15:13:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949242900000002 2020-08-07T15:13:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291786+0000", 2020-08-07T15:13:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000003e-06 2020-08-07T15:13:20.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291794+0000", 2020-08-07T15:13:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88420652700000002 2020-08-07T15:13:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176000+0000", 2020-08-07T15:13:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4634000000000003e-05 2020-08-07T15:13:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:20.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176085+0000", 2020-08-07T15:13:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952963789999999 2020-08-07T15:13:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.280 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42494 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:20.280 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:13:20.280 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 259.80470847300001, 2020-08-07T15:13:20.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952882820000001, 2020-08-07T15:13:20.281 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:20.281 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:20.281 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:20.281 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:20.282 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:20.282 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42494 2020-08-07T15:13:20.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.282 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:20.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:20.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:13:20.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:20.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:20.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:13:20.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0117999999999997e-05 2020-08-07T15:13:20.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:20.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280941+0000", 2020-08-07T15:13:20.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1240000000000004e-06 2020-08-07T15:13:20.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280946+0000", 2020-08-07T15:13:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0000000000000001e-05 2020-08-07T15:13:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280956+0000", 2020-08-07T15:13:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.279e-06 2020-08-07T15:13:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:20.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280960+0000", 2020-08-07T15:13:20.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0062727080000000001 2020-08-07T15:13:20.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287233+0000", 2020-08-07T15:13:20.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018362900000000001 2020-08-07T15:13:20.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:20.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287416+0000", 2020-08-07T15:13:20.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029536100000000001 2020-08-07T15:13:20.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:20.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287712+0000", 2020-08-07T15:13:20.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046231739999999999 2020-08-07T15:13:20.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292335+0000", 2020-08-07T15:13:20.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3860000000000001e-06 2020-08-07T15:13:20.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292342+0000", 2020-08-07T15:13:20.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99948501000000001 2020-08-07T15:13:20.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291827+0000", 2020-08-07T15:13:20.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2272999999999999e-05 2020-08-07T15:13:20.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291840+0000", 2020-08-07T15:13:20.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88426668100000005 2020-08-07T15:13:20.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176106+0000", 2020-08-07T15:13:20.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2539000000000005e-05 2020-08-07T15:13:20.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:20.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176179+0000", 2020-08-07T15:13:20.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952882820000001 2020-08-07T15:13:20.300 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.300 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:20.301 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.301 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42495 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:20.301 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:13:20.302 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 259.80460738300002, 2020-08-07T15:13:20.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895286867, 2020-08-07T15:13:20.302 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:20.302 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:20.303 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:20.303 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:20.303 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:20.303 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42495 2020-08-07T15:13:20.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.304 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:20.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:20.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:13:20.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:20.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:20.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:13:20.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6690999999999997e-05 2020-08-07T15:13:20.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:20.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281038+0000", 2020-08-07T15:13:20.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7860000000000002e-06 2020-08-07T15:13:20.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:20.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281044+0000", 2020-08-07T15:13:20.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0429e-05 2020-08-07T15:13:20.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:20.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281054+0000", 2020-08-07T15:13:20.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9040000000000002e-06 2020-08-07T15:13:20.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:20.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281058+0000", 2020-08-07T15:13:20.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070941490000000001 2020-08-07T15:13:20.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288153+0000", 2020-08-07T15:13:20.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9006999999999998e-05 2020-08-07T15:13:20.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:20.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288222+0000", 2020-08-07T15:13:20.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012106400000000001 2020-08-07T15:13:20.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:20.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288343+0000", 2020-08-07T15:13:20.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004034752 2020-08-07T15:13:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292377+0000", 2020-08-07T15:13:20.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.537e-06 2020-08-07T15:13:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292385+0000", 2020-08-07T15:13:20.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949123500000003 2020-08-07T15:13:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291876+0000", 2020-08-07T15:13:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7729999999999999e-06 2020-08-07T15:13:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291884+0000", 2020-08-07T15:13:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88431700899999999 2020-08-07T15:13:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176201+0000", 2020-08-07T15:13:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7531e-05 2020-08-07T15:13:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:20.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176278+0000", 2020-08-07T15:13:20.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895286867 2020-08-07T15:13:20.322 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.322 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:20.322 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.323 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42496 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:20.323 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:13:20.323 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 259.804502474, 2020-08-07T15:13:20.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89526942, 2020-08-07T15:13:20.323 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:20.324 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:20.324 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:20.324 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:20.324 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:20.324 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42496 2020-08-07T15:13:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:20.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:13:20.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:20.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:20.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:13:20.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9227999999999997e-05 2020-08-07T15:13:20.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:20.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281146+0000", 2020-08-07T15:13:20.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1120000000000004e-06 2020-08-07T15:13:20.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:20.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281151+0000", 2020-08-07T15:13:20.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2208999999999999e-05 2020-08-07T15:13:20.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:20.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281163+0000", 2020-08-07T15:13:20.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5469999999999998e-06 2020-08-07T15:13:20.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281168+0000", 2020-08-07T15:13:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0076520829999999996 2020-08-07T15:13:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288820+0000", 2020-08-07T15:13:20.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5035000000000003e-05 2020-08-07T15:13:20.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:20.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288885+0000", 2020-08-07T15:13:20.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116507 2020-08-07T15:13:20.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:20.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.289001+0000", 2020-08-07T15:13:20.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034121960000000002 2020-08-07T15:13:20.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292413+0000", 2020-08-07T15:13:20.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8109999999999997e-06 2020-08-07T15:13:20.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292420+0000", 2020-08-07T15:13:20.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949504600000005 2020-08-07T15:13:20.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291915+0000", 2020-08-07T15:13:20.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3999999999999992e-06 2020-08-07T15:13:20.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291924+0000", 2020-08-07T15:13:20.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88437394599999997 2020-08-07T15:13:20.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176298+0000", 2020-08-07T15:13:20.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8300000000000007e-05 2020-08-07T15:13:20.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:20.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176366+0000", 2020-08-07T15:13:20.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89526942 2020-08-07T15:13:20.342 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.342 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:20.342 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.343 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73081 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:13:20.343 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:13:20.343 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 232.8590332, 2020-08-07T15:13:20.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0005038420000001, 2020-08-07T15:13:20.343 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:20.344 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:20.344 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:20.344 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:20.344 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:20.345 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73081 2020-08-07T15:13:20.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.345 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:20.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:20.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:13:20.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:20.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:20.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:13:20.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3016000000000003e-05 2020-08-07T15:13:20.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:20.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226609+0000", 2020-08-07T15:13:20.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4479999999999996e-06 2020-08-07T15:13:20.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:20.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226613+0000", 2020-08-07T15:13:20.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0189999999999995e-06 2020-08-07T15:13:20.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:20.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226622+0000", 2020-08-07T15:13:20.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5980000000000001e-06 2020-08-07T15:13:20.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:20.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226626+0000", 2020-08-07T15:13:20.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8088999999999999e-05 2020-08-07T15:13:20.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226684+0000", 2020-08-07T15:13:20.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2162e-05 2020-08-07T15:13:20.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226696+0000", 2020-08-07T15:13:20.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98675689 2020-08-07T15:13:20.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213453+0000", 2020-08-07T15:13:20.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012916800000000001 2020-08-07T15:13:20.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:20.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213582+0000", 2020-08-07T15:13:20.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015665899999999999 2020-08-07T15:13:20.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:13:20.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213739+0000", 2020-08-07T15:13:20.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049940239999999997 2020-08-07T15:13:20.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:20.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218733+0000", 2020-08-07T15:13:20.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0079774779999999997 2020-08-07T15:13:20.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226710+0000", 2020-08-07T15:13:20.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.161e-06 2020-08-07T15:13:20.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:20.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226711+0000", 2020-08-07T15:13:20.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0105000000000001e-05 2020-08-07T15:13:20.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:20.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226772+0000", 2020-08-07T15:13:20.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5633999999999997e-05 2020-08-07T15:13:20.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226817+0000", 2020-08-07T15:13:20.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7379000000000001e-05 2020-08-07T15:13:20.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226875+0000", 2020-08-07T15:13:20.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8000000000000005e-07 2020-08-07T15:13:20.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:20.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226875+0000", 2020-08-07T15:13:20.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019423199999999999 2020-08-07T15:13:20.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:20.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227070+0000", 2020-08-07T15:13:20.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0005038420000001 2020-08-07T15:13:20.366 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.366 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:20.367 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.367 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73082 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:13:20.367 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:13:20.368 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 232.85866273600001, 2020-08-07T15:13:20.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001326693, 2020-08-07T15:13:20.368 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:20.368 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:20.368 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:20.369 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:20.369 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:20.369 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73082 2020-08-07T15:13:20.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.370 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:20.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:20.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:13:20.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:20.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:20.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:13:20.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6928000000000003e-05 2020-08-07T15:13:20.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:20.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226973+0000", 2020-08-07T15:13:20.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3630000000000004e-06 2020-08-07T15:13:20.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:20.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226977+0000", 2020-08-07T15:13:20.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9659999999999996e-06 2020-08-07T15:13:20.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:20.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226985+0000", 2020-08-07T15:13:20.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5800000000000002e-06 2020-08-07T15:13:20.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226990+0000", 2020-08-07T15:13:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9101999999999998e-05 2020-08-07T15:13:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227039+0000", 2020-08-07T15:13:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9620000000000005e-06 2020-08-07T15:13:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227049+0000", 2020-08-07T15:13:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98722513999999995 2020-08-07T15:13:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214274+0000", 2020-08-07T15:13:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2700999999999994e-05 2020-08-07T15:13:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214347+0000", 2020-08-07T15:13:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015346899999999999 2020-08-07T15:13:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:13:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214500+0000", 2020-08-07T15:13:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042450309999999998 2020-08-07T15:13:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218745+0000", 2020-08-07T15:13:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008379503 2020-08-07T15:13:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227125+0000", 2020-08-07T15:13:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7899999999999998e-07 2020-08-07T15:13:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227126+0000", 2020-08-07T15:13:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1610000000000002e-05 2020-08-07T15:13:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227177+0000", 2020-08-07T15:13:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489000000000003e-05 2020-08-07T15:13:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227226+0000", 2020-08-07T15:13:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00084338600000000001 2020-08-07T15:13:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228069+0000", 2020-08-07T15:13:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7e-07 2020-08-07T15:13:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228069+0000", 2020-08-07T15:13:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019351399999999999 2020-08-07T15:13:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228263+0000", 2020-08-07T15:13:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001326693 2020-08-07T15:13:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73083 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:13:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:13:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 232.85856847700001, 2020-08-07T15:13:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015968209999999, 2020-08-07T15:13:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73083 2020-08-07T15:13:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:13:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:13:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1627000000000001e-05 2020-08-07T15:13:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227062+0000", 2020-08-07T15:13:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.253e-06 2020-08-07T15:13:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227065+0000", 2020-08-07T15:13:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3168e-05 2020-08-07T15:13:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227079+0000", 2020-08-07T15:13:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1549999999999999e-06 2020-08-07T15:13:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227082+0000", 2020-08-07T15:13:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5534999999999997e-05 2020-08-07T15:13:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227127+0000", 2020-08-07T15:13:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6823000000000001e-05 2020-08-07T15:13:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227144+0000", 2020-08-07T15:13:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98771770599999997 2020-08-07T15:13:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214862+0000", 2020-08-07T15:13:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8252999999999999e-05 2020-08-07T15:13:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214930+0000", 2020-08-07T15:13:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013646400000000001 2020-08-07T15:13:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:13:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215066+0000", 2020-08-07T15:13:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036907089999999999 2020-08-07T15:13:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218757+0000", 2020-08-07T15:13:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0085133450000000003 2020-08-07T15:13:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227270+0000", 2020-08-07T15:13:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4899999999999995e-07 2020-08-07T15:13:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227271+0000", 2020-08-07T15:13:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4555999999999997e-05 2020-08-07T15:13:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227326+0000", 2020-08-07T15:13:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024594800000000001 2020-08-07T15:13:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227572+0000", 2020-08-07T15:13:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00086125700000000004 2020-08-07T15:13:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228433+0000", 2020-08-07T15:13:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3799999999999998e-07 2020-08-07T15:13:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228433+0000", 2020-08-07T15:13:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019413500000000001 2020-08-07T15:13:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228627+0000", 2020-08-07T15:13:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015968209999999 2020-08-07T15:13:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73084 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:13:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:13:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 232.85847953800001, 2020-08-07T15:13:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0016919049999999, 2020-08-07T15:13:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73084 2020-08-07T15:13:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:13:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:13:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5565999999999997e-05 2020-08-07T15:13:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227165+0000", 2020-08-07T15:13:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2020-08-07T15:13:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227169+0000", 2020-08-07T15:13:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6850000000000001e-06 2020-08-07T15:13:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227176+0000", 2020-08-07T15:13:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1949999999999999e-06 2020-08-07T15:13:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227179+0000", 2020-08-07T15:13:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0318000000000003e-05 2020-08-07T15:13:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227220+0000", 2020-08-07T15:13:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0349e-05 2020-08-07T15:13:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227230+0000", 2020-08-07T15:13:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98832962400000002 2020-08-07T15:13:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215560+0000", 2020-08-07T15:13:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013913099999999999 2020-08-07T15:13:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215699+0000", 2020-08-07T15:13:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000200006 2020-08-07T15:13:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:13:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215899+0000", 2020-08-07T15:13:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028673650000000002 2020-08-07T15:13:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218766+0000", 2020-08-07T15:13:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086101779999999996 2020-08-07T15:13:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227376+0000", 2020-08-07T15:13:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0999999999999998e-07 2020-08-07T15:13:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227377+0000", 2020-08-07T15:13:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6552999999999997e-05 2020-08-07T15:13:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227454+0000", 2020-08-07T15:13:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000421717 2020-08-07T15:13:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227875+0000", 2020-08-07T15:13:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00081781300000000003 2020-08-07T15:13:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228693+0000", 2020-08-07T15:13:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1699999999999997e-07 2020-08-07T15:13:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228694+0000", 2020-08-07T15:13:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117483 2020-08-07T15:13:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228811+0000", 2020-08-07T15:13:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0016919049999999 2020-08-07T15:13:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73085 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:13:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:13:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 232.85834372299999, 2020-08-07T15:13:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0018015629999999, 2020-08-07T15:13:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73085 2020-08-07T15:13:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:13:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:13:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6570999999999997e-05 2020-08-07T15:13:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227322+0000", 2020-08-07T15:13:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5509999999999996e-06 2020-08-07T15:13:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227328+0000", 2020-08-07T15:13:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2498e-05 2020-08-07T15:13:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227341+0000", 2020-08-07T15:13:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4389999999999999e-06 2020-08-07T15:13:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227346+0000", 2020-08-07T15:13:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3711999999999998e-05 2020-08-07T15:13:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227390+0000", 2020-08-07T15:13:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1868999999999999e-05 2020-08-07T15:13:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227402+0000", 2020-08-07T15:13:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98896525099999999 2020-08-07T15:13:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216367+0000", 2020-08-07T15:13:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012573 2020-08-07T15:13:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216493+0000", 2020-08-07T15:13:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020525200000000001 2020-08-07T15:13:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:13:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216698+0000", 2020-08-07T15:13:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037596470000000001 2020-08-07T15:13:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.220458+0000", 2020-08-07T15:13:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070227190000000002 2020-08-07T15:13:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227480+0000", 2020-08-07T15:13:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6600000000000003e-07 2020-08-07T15:13:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227481+0000", 2020-08-07T15:13:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9898000000000003e-05 2020-08-07T15:13:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227521+0000", 2020-08-07T15:13:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039130000000000002 2020-08-07T15:13:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227912+0000", 2020-08-07T15:13:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00095502399999999996 2020-08-07T15:13:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228867+0000", 2020-08-07T15:13:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2020-08-07T15:13:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228867+0000", 2020-08-07T15:13:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018933599999999999 2020-08-07T15:13:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229057+0000", 2020-08-07T15:13:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0018015629999999 2020-08-07T15:13:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73086 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:13:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:13:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 232.858211227, 2020-08-07T15:13:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002364923, 2020-08-07T15:13:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73086 2020-08-07T15:13:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:13:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:13:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000238426 2020-08-07T15:13:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227626+0000", 2020-08-07T15:13:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9609999999999999e-06 2020-08-07T15:13:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227633+0000", 2020-08-07T15:13:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2727e-05 2020-08-07T15:13:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227646+0000", 2020-08-07T15:13:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1739999999999997e-06 2020-08-07T15:13:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227652+0000", 2020-08-07T15:13:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5004000000000001e-05 2020-08-07T15:13:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227737+0000", 2020-08-07T15:13:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2744e-05 2020-08-07T15:13:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227750+0000", 2020-08-07T15:13:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98938776500000003 2020-08-07T15:13:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217138+0000", 2020-08-07T15:13:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012361499999999999 2020-08-07T15:13:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:20.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217261+0000", 2020-08-07T15:13:20.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000212504 2020-08-07T15:13:20.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:13:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217474+0000", 2020-08-07T15:13:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002995036 2020-08-07T15:13:20.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:20.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.220469+0000", 2020-08-07T15:13:20.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071283249999999996 2020-08-07T15:13:20.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227597+0000", 2020-08-07T15:13:20.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0499999999999999e-07 2020-08-07T15:13:20.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:20.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227597+0000", 2020-08-07T15:13:20.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1872999999999997e-05 2020-08-07T15:13:20.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:20.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227639+0000", 2020-08-07T15:13:20.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032010200000000001 2020-08-07T15:13:20.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227959+0000", 2020-08-07T15:13:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015980899999999999 2020-08-07T15:13:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229557+0000", 2020-08-07T15:13:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9299999999999999e-07 2020-08-07T15:13:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229558+0000", 2020-08-07T15:13:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019477900000000001 2020-08-07T15:13:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229753+0000", 2020-08-07T15:13:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002364923 2020-08-07T15:13:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:20.488 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73087 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:13:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:13:20.489 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 232.85727007400001, 2020-08-07T15:13:20.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001896506, 2020-08-07T15:13:20.490 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:20.490 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:20.490 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:20.490 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:20.490 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73087 2020-08-07T15:13:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:20.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:13:20.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:20.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:20.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:13:20.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7354e-05 2020-08-07T15:13:20.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:20.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228386+0000", 2020-08-07T15:13:20.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8000000000000001e-06 2020-08-07T15:13:20.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:20.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228393+0000", 2020-08-07T15:13:20.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3784999999999999e-05 2020-08-07T15:13:20.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:20.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228407+0000", 2020-08-07T15:13:20.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8090000000000004e-06 2020-08-07T15:13:20.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:20.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228414+0000", 2020-08-07T15:13:20.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6907000000000003e-05 2020-08-07T15:13:20.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228461+0000", 2020-08-07T15:13:20.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8929999999999994e-06 2020-08-07T15:13:20.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228469+0000", 2020-08-07T15:13:20.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98950057700000005 2020-08-07T15:13:20.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217970+0000", 2020-08-07T15:13:20.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000132182 2020-08-07T15:13:20.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:20.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218102+0000", 2020-08-07T15:13:20.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020073399999999999 2020-08-07T15:13:20.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:13:20.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218303+0000", 2020-08-07T15:13:20.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035951120000000001 2020-08-07T15:13:20.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:20.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.221898+0000", 2020-08-07T15:13:20.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057898660000000003 2020-08-07T15:13:20.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227688+0000", 2020-08-07T15:13:20.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0200000000000002e-07 2020-08-07T15:13:20.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:20.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227688+0000", 2020-08-07T15:13:20.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1477999999999996e-05 2020-08-07T15:13:20.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:20.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227760+0000", 2020-08-07T15:13:20.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016657480000000001 2020-08-07T15:13:20.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229426+0000", 2020-08-07T15:13:20.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061290400000000005 2020-08-07T15:13:20.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230039+0000", 2020-08-07T15:13:20.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2e-07 2020-08-07T15:13:20.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:20.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230039+0000", 2020-08-07T15:13:20.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018643500000000001 2020-08-07T15:13:20.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:20.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230225+0000", 2020-08-07T15:13:20.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001896506 2020-08-07T15:13:20.510 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.511 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:20.511 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.511 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73088 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:13:20.511 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:13:20.512 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 232.856879384, 2020-08-07T15:13:20.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001627153, 2020-08-07T15:13:20.512 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:20.512 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:20.512 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:20.513 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:20.513 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:20.513 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73088 2020-08-07T15:13:20.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.513 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:20.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:20.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:13:20.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:20.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:20.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:13:20.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7741000000000003e-05 2020-08-07T15:13:20.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:20.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228767+0000", 2020-08-07T15:13:20.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6409999999999999e-06 2020-08-07T15:13:20.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:20.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228773+0000", 2020-08-07T15:13:20.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0874e-05 2020-08-07T15:13:20.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:20.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228784+0000", 2020-08-07T15:13:20.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1529999999999996e-06 2020-08-07T15:13:20.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:20.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228789+0000", 2020-08-07T15:13:20.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3945e-05 2020-08-07T15:13:20.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228833+0000", 2020-08-07T15:13:20.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4020000000000002e-06 2020-08-07T15:13:20.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228841+0000", 2020-08-07T15:13:20.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98996913900000005 2020-08-07T15:13:20.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218810+0000", 2020-08-07T15:13:20.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112484 2020-08-07T15:13:20.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:20.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218923+0000", 2020-08-07T15:13:20.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019891 2020-08-07T15:13:20.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:13:20.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.219122+0000", 2020-08-07T15:13:20.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027884099999999998 2020-08-07T15:13:20.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:20.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.221910+0000", 2020-08-07T15:13:20.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058753399999999997 2020-08-07T15:13:20.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227786+0000", 2020-08-07T15:13:20.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9200000000000002e-07 2020-08-07T15:13:20.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:20.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227786+0000", 2020-08-07T15:13:20.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8572000000000003e-05 2020-08-07T15:13:20.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:20.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227824+0000", 2020-08-07T15:13:20.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001651296 2020-08-07T15:13:20.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229476+0000", 2020-08-07T15:13:20.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00077620399999999996 2020-08-07T15:13:20.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230252+0000", 2020-08-07T15:13:20.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1600000000000002e-07 2020-08-07T15:13:20.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:20.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230252+0000", 2020-08-07T15:13:20.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4433999999999997e-05 2020-08-07T15:13:20.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:20.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230347+0000", 2020-08-07T15:13:20.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001627153 2020-08-07T15:13:20.532 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.532 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:20.532 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.533 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:128738 4.9 4.370c6199 (undecoded) ondisk+write+known_if_redirected e155)", 2020-08-07T15:13:20.533 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:03.204387+0000", 2020-08-07T15:13:20.533 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 196.88121173100001, 2020-08-07T15:13:20.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99077720899999999, 2020-08-07T15:13:20.533 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:20.534 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:20.534 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:20.534 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:20.534 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:20.534 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128738 2020-08-07T15:13:20.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.535 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:20.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:20.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204387+0000", 2020-08-07T15:13:20.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:20.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:20.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204387+0000", 2020-08-07T15:13:20.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4561999999999999e-05 2020-08-07T15:13:20.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:20.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204422+0000", 2020-08-07T15:13:20.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7270000000000001e-06 2020-08-07T15:13:20.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:20.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204428+0000", 2020-08-07T15:13:20.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3451e-05 2020-08-07T15:13:20.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:20.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204442+0000", 2020-08-07T15:13:20.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1589999999999998e-06 2020-08-07T15:13:20.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:20.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204448+0000", 2020-08-07T15:13:20.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3755999999999999e-05 2020-08-07T15:13:20.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204492+0000", 2020-08-07T15:13:20.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2450000000000001e-06 2020-08-07T15:13:20.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204499+0000", 2020-08-07T15:13:20.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99065224600000001 2020-08-07T15:13:20.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195151+0000", 2020-08-07T15:13:20.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3063e-05 2020-08-07T15:13:20.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:20.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195164+0000", 2020-08-07T15:13:20.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99077720899999999 2020-08-07T15:13:20.545 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.545 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:20.545 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.546 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:128742 4.9 4.69b70ab9 (undecoded) ondisk+write+known_if_redirected e155)", 2020-08-07T15:13:20.546 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:03.205224+0000", 2020-08-07T15:13:20.546 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 196.88037509700001, 2020-08-07T15:13:20.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.989964548, 2020-08-07T15:13:20.547 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:20.547 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:20.548 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:20.548 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:20.548 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:20.548 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128742 2020-08-07T15:13:20.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.549 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:20.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:20.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205224+0000", 2020-08-07T15:13:20.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:20.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:20.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205224+0000", 2020-08-07T15:13:20.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1018e-05 2020-08-07T15:13:20.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:20.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205255+0000", 2020-08-07T15:13:20.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5840000000000002e-06 2020-08-07T15:13:20.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:20.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205260+0000", 2020-08-07T15:13:20.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2388999999999999e-05 2020-08-07T15:13:20.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:20.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205273+0000", 2020-08-07T15:13:20.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3799999999999999e-06 2020-08-07T15:13:20.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:20.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205279+0000", 2020-08-07T15:13:20.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6684999999999998e-05 2020-08-07T15:13:20.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205326+0000", 2020-08-07T15:13:20.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6200000000000004e-06 2020-08-07T15:13:20.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205332+0000", 2020-08-07T15:13:20.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98984944600000002 2020-08-07T15:13:20.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195181+0000", 2020-08-07T15:13:20.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4259999999999997e-06 2020-08-07T15:13:20.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:20.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195188+0000", 2020-08-07T15:13:20.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.989964548 2020-08-07T15:13:20.563 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.564 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:20.564 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.570 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:128743 4.9 4.1824cba9 (undecoded) ondisk+write+known_if_redirected e155)", 2020-08-07T15:13:20.570 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:03.205500+0000", 2020-08-07T15:13:20.571 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 196.88009923499999, 2020-08-07T15:13:20.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98971761999999996, 2020-08-07T15:13:20.571 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:20.571 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:20.572 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:20.572 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:20.572 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:20.572 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128743 2020-08-07T15:13:20.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.573 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:20.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:20.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205500+0000", 2020-08-07T15:13:20.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:20.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:20.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205500+0000", 2020-08-07T15:13:20.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6441000000000001e-05 2020-08-07T15:13:20.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:20.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205516+0000", 2020-08-07T15:13:20.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4429999999999999e-06 2020-08-07T15:13:20.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:20.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205520+0000", 2020-08-07T15:13:20.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4590000000000003e-06 2020-08-07T15:13:20.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205526+0000", 2020-08-07T15:13:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0429999999999999e-06 2020-08-07T15:13:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205529+0000", 2020-08-07T15:13:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1313000000000003e-05 2020-08-07T15:13:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205560+0000", 2020-08-07T15:13:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.934e-06 2020-08-07T15:13:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205567+0000", 2020-08-07T15:13:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98964017599999998 2020-08-07T15:13:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195208+0000", 2020-08-07T15:13:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8109999999999997e-06 2020-08-07T15:13:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195217+0000", 2020-08-07T15:13:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98971761999999996 2020-08-07T15:13:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:20.597 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:20.600 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 68 16 100903 100887 11.5895 12.6172 0.00416706 0.0107809 2020-08-07T15:13:20.600 INFO:teuthology.orchestra.run.smithi074.stderr:set 1.1d pg_upmap mapping to [1,5,0] 2020-08-07T15:13:20.600 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:13:20.604 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:13:20.604 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:20.598+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:20.700 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:20.699+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:20.801 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:20.800+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:20.824 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:13:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:101718 6.3 6:c1d1237e:::benchmark_data_smithi074_63843_object12714:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e282)", 2020-08-07T15:13:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:13:20.800181+0000", 2020-08-07T15:13:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.016086794000000001, 2020-08-07T15:13:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016115038000000002, 2020-08-07T15:13:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:13:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 101718 2020-08-07T15:13:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.800181+0000", 2020-08-07T15:13:20.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:20.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:20.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.800181+0000", 2020-08-07T15:13:20.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1928000000000002e-05 2020-08-07T15:13:20.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:20.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.800233+0000", 2020-08-07T15:13:20.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3730000000000001e-06 2020-08-07T15:13:20.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:20.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.800238+0000", 2020-08-07T15:13:20.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0772e-05 2020-08-07T15:13:20.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:20.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.800249+0000", 2020-08-07T15:13:20.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9359999999999998e-06 2020-08-07T15:13:20.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:20.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.800254+0000", 2020-08-07T15:13:20.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017499569999999999 2020-08-07T15:13:20.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.802004+0000", 2020-08-07T15:13:20.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104793 2020-08-07T15:13:20.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:20.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.802109+0000", 2020-08-07T15:13:20.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016766099999999999 2020-08-07T15:13:20.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:13:20.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.802276+0000", 2020-08-07T15:13:20.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001669872 2020-08-07T15:13:20.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:20.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.803946+0000", 2020-08-07T15:13:20.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2600000000000001e-07 2020-08-07T15:13:20.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:20.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.803947+0000", 2020-08-07T15:13:20.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010998200000000001 2020-08-07T15:13:20.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:20.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.804057+0000", 2020-08-07T15:13:20.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039055019999999999 2020-08-07T15:13:20.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.807962+0000", 2020-08-07T15:13:20.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0077816020000000003 2020-08-07T15:13:20.864 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.864 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:20.864 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.865 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:101719 6.3 6:c1d1237e:::benchmark_data_smithi074_63843_object12714:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e282)", 2020-08-07T15:13:20.865 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:13:20.800838+0000", 2020-08-07T15:13:20.865 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.015430078, 2020-08-07T15:13:20.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015541770999999999, 2020-08-07T15:13:20.866 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:20.866 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:20.866 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:20.866 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:13:20.866 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:20.867 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 101719 2020-08-07T15:13:20.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.867 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:20.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:20.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.800838+0000", 2020-08-07T15:13:20.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:20.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:20.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.800838+0000", 2020-08-07T15:13:20.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6752000000000001e-05 2020-08-07T15:13:20.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:20.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.800904+0000", 2020-08-07T15:13:20.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5060000000000003e-06 2020-08-07T15:13:20.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:20.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.800910+0000", 2020-08-07T15:13:20.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1719000000000001e-05 2020-08-07T15:13:20.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:20.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.800921+0000", 2020-08-07T15:13:20.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5520000000000004e-06 2020-08-07T15:13:20.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:20.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.800927+0000", 2020-08-07T15:13:20.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023050729999999999 2020-08-07T15:13:20.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.803232+0000", 2020-08-07T15:13:20.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3545000000000001e-05 2020-08-07T15:13:20.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:20.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.803306+0000", 2020-08-07T15:13:20.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018096099999999999 2020-08-07T15:13:20.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:13:20.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.803487+0000", 2020-08-07T15:13:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026139589999999999 2020-08-07T15:13:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:20.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.806101+0000", 2020-08-07T15:13:20.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9900000000000009e-07 2020-08-07T15:13:20.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:20.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.806102+0000", 2020-08-07T15:13:20.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013447599999999999 2020-08-07T15:13:20.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:20.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.806236+0000", 2020-08-07T15:13:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075203199999999996 2020-08-07T15:13:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.813756+0000", 2020-08-07T15:13:20.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012918862 2020-08-07T15:13:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.882 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:101720 6.3 6:c1d1237e:::benchmark_data_smithi074_63843_object12714:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e282)", 2020-08-07T15:13:20.882 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:13:20.801011+0000", 2020-08-07T15:13:20.882 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.015256189999999999, 2020-08-07T15:13:20.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015428433, 2020-08-07T15:13:20.882 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:20.883 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:20.883 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:20.883 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:13:20.883 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:20.883 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 101720 2020-08-07T15:13:20.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.884 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:20.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:20.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.801011+0000", 2020-08-07T15:13:20.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:20.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:20.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.801011+0000", 2020-08-07T15:13:20.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7609999999999999e-05 2020-08-07T15:13:20.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:20.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.801069+0000", 2020-08-07T15:13:20.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0659999999999993e-06 2020-08-07T15:13:20.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:20.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.801078+0000", 2020-08-07T15:13:20.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0434999999999999e-05 2020-08-07T15:13:20.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.801089+0000", 2020-08-07T15:13:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3380000000000001e-06 2020-08-07T15:13:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:20.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.801095+0000", 2020-08-07T15:13:20.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031321140000000001 2020-08-07T15:13:20.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.804227+0000", 2020-08-07T15:13:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010592800000000001 2020-08-07T15:13:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:20.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.804333+0000", 2020-08-07T15:13:20.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019890199999999999 2020-08-07T15:13:20.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:13:20.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.804532+0000", 2020-08-07T15:13:20.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021240909999999998 2020-08-07T15:13:20.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:20.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.806656+0000", 2020-08-07T15:13:20.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3900000000000003e-07 2020-08-07T15:13:20.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:20.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.806657+0000", 2020-08-07T15:13:20.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1713999999999997e-05 2020-08-07T15:13:20.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:20.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.806739+0000", 2020-08-07T15:13:20.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070533360000000003 2020-08-07T15:13:20.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.813792+0000", 2020-08-07T15:13:20.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012780473000000001 2020-08-07T15:13:20.898 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.898 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:20.898 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.899 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:101592 6.a 6:53f46f7b:::benchmark_data_smithi074_63843_object12698:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e282)", 2020-08-07T15:13:20.899 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:13:20.723371+0000", 2020-08-07T15:13:20.899 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.092896700999999998, 2020-08-07T15:13:20.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.093131034000000001, 2020-08-07T15:13:20.900 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:20.900 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:20.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:20.901 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:13:20.902 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:20.902 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 101592 2020-08-07T15:13:20.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.902 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:20.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:20.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.723371+0000", 2020-08-07T15:13:20.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:20.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:20.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.723371+0000", 2020-08-07T15:13:20.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4253999999999999e-05 2020-08-07T15:13:20.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:20.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.723425+0000", 2020-08-07T15:13:20.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1689999999999997e-06 2020-08-07T15:13:20.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:20.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.723431+0000", 2020-08-07T15:13:20.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5675000000000002e-05 2020-08-07T15:13:20.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:20.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.723447+0000", 2020-08-07T15:13:20.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4740000000000005e-06 2020-08-07T15:13:20.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:20.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.723455+0000", 2020-08-07T15:13:20.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043900709999999997 2020-08-07T15:13:20.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:20.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.727846+0000", 2020-08-07T15:13:20.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107357 2020-08-07T15:13:20.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:20.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.727953+0000", 2020-08-07T15:13:20.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023732000000000001 2020-08-07T15:13:20.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,5,7", 2020-08-07T15:13:20.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.728190+0000", 2020-08-07T15:13:20.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002313725 2020-08-07T15:13:20.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:20.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.730504+0000", 2020-08-07T15:13:20.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00096687699999999999 2020-08-07T15:13:20.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.731471+0000", 2020-08-07T15:13:20.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5499999999999998e-07 2020-08-07T15:13:20.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:20.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.731471+0000", 2020-08-07T15:13:20.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1765000000000001e-05 2020-08-07T15:13:20.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:20.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.731553+0000", 2020-08-07T15:13:20.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042572111000000003 2020-08-07T15:13:20.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:20.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:20.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:20.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:20.774125+0000", 2020-08-07T15:13:20.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050754452999999998 2020-08-07T15:13:20.921 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.921 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:20.921 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.921 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:20.921 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:13:20.922 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 4 2020-08-07T15:13:20.922 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:20.923 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:13:20.924 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:20.922+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:21.028 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:21.027+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:21.082 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:13:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:13:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:13:21.084 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:21.093 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:13:21.130 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:21.129+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:21.242 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:21.234+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:21.277 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 69 16 102452 102436 11.5969 12.1016 0.00904506 0.0107714 2020-08-07T15:13:21.304 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:13:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:13:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:13:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106232 3.a 3:5599d7e6:::benchmark_data_smithi074_35447_object13278:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:13:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:13:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 333.95335288899997, 2020-08-07T15:13:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52578228699999996, 2020-08-07T15:13:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:13:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:13:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106232 2020-08-07T15:13:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:13:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345607+0000", 2020-08-07T15:13:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4214999999999997e-05 2020-08-07T15:13:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345641+0000", 2020-08-07T15:13:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9279999999999993e-06 2020-08-07T15:13:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345649+0000", 2020-08-07T15:13:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4391e-05 2020-08-07T15:13:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345664+0000", 2020-08-07T15:13:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7170000000000002e-06 2020-08-07T15:13:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345670+0000", 2020-08-07T15:13:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.459e-05 2020-08-07T15:13:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345695+0000", 2020-08-07T15:13:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6070999999999999e-05 2020-08-07T15:13:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345751+0000", 2020-08-07T15:13:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010069300000000001 2020-08-07T15:13:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:13:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.345852+0000", 2020-08-07T15:13:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013512299999999999 2020-08-07T15:13:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.347203+0000", 2020-08-07T15:13:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0844000000000001e-05 2020-08-07T15:13:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.347274+0000", 2020-08-07T15:13:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52399471600000003 2020-08-07T15:13:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871269+0000", 2020-08-07T15:13:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.311e-06 2020-08-07T15:13:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871270+0000", 2020-08-07T15:13:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3509000000000004e-05 2020-08-07T15:13:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871343+0000", 2020-08-07T15:13:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6072000000000002e-05 2020-08-07T15:13:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.871389+0000", 2020-08-07T15:13:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52578228699999996 2020-08-07T15:13:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:30976 4.5 e98/81 ondisk, result = 0)", 2020-08-07T15:13:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:45.736493+0000", 2020-08-07T15:13:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 275.56246698899997, 2020-08-07T15:13:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54677504399999999, 2020-08-07T15:13:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736493+0000", 2020-08-07T15:13:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736493+0000", 2020-08-07T15:13:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.012e-05 2020-08-07T15:13:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736523+0000", 2020-08-07T15:13:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6669999999999998e-06 2020-08-07T15:13:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:45.736527+0000", 2020-08-07T15:13:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54657333200000002 2020-08-07T15:13:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283100+0000", 2020-08-07T15:13:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.549e-06 2020-08-07T15:13:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283105+0000", 2020-08-07T15:13:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1853000000000006e-05 2020-08-07T15:13:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283167+0000", 2020-08-07T15:13:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015e-05 2020-08-07T15:13:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283177+0000", 2020-08-07T15:13:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1508000000000005e-05 2020-08-07T15:13:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.283268+0000", 2020-08-07T15:13:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54677504399999999 2020-08-07T15:13:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118153 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.222760+0000", 2020-08-07T15:13:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 98.076199789, 2020-08-07T15:13:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0520728660000001, 2020-08-07T15:13:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118153 2020-08-07T15:13:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222760+0000", 2020-08-07T15:13:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222760+0000", 2020-08-07T15:13:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1894000000000002e-05 2020-08-07T15:13:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222812+0000", 2020-08-07T15:13:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4880000000000001e-06 2020-08-07T15:13:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222820+0000", 2020-08-07T15:13:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6744000000000001e-05 2020-08-07T15:13:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222836+0000", 2020-08-07T15:13:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5470000000000002e-06 2020-08-07T15:13:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222846+0000", 2020-08-07T15:13:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9478e-05 2020-08-07T15:13:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222925+0000", 2020-08-07T15:13:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0966e-05 2020-08-07T15:13:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2020-08-07T15:13:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222936+0000", 2020-08-07T15:13:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002974099 2020-08-07T15:13:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.225910+0000", 2020-08-07T15:13:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5197e-05 2020-08-07T15:13:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.225926+0000", 2020-08-07T15:13:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.02944747 2020-08-07T15:13:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.255373+0000", 2020-08-07T15:13:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022597200000000001 2020-08-07T15:13:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.255599+0000", 2020-08-07T15:13:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000194415 2020-08-07T15:13:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.255794+0000", 2020-08-07T15:13:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0076831360000000001 2020-08-07T15:13:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263477+0000", 2020-08-07T15:13:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01099552 2020-08-07T15:13:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274472+0000", 2020-08-07T15:13:21.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2948999999999998e-05 2020-08-07T15:13:21.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274515+0000", 2020-08-07T15:13:21.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.569999999999999e-07 2020-08-07T15:13:21.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:21.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274516+0000", 2020-08-07T15:13:21.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2750999999999997e-05 2020-08-07T15:13:21.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:21.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274569+0000", 2020-08-07T15:13:21.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4109000000000001e-05 2020-08-07T15:13:21.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274613+0000", 2020-08-07T15:13:21.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0432999999999997e-05 2020-08-07T15:13:21.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274653+0000", 2020-08-07T15:13:21.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5900000000000002e-07 2020-08-07T15:13:21.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:21.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274654+0000", 2020-08-07T15:13:21.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017928200000000001 2020-08-07T15:13:21.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:21.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274833+0000", 2020-08-07T15:13:21.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0520728660000001 2020-08-07T15:13:21.412 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.412 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:21.413 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.414 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118154 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:21.415 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.222899+0000", 2020-08-07T15:13:21.415 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 98.076061487000004, 2020-08-07T15:13:21.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0539106499999999, 2020-08-07T15:13:21.416 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:21.416 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:21.416 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:21.417 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:21.417 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:21.417 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118154 2020-08-07T15:13:21.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.418 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:21.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:21.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222899+0000", 2020-08-07T15:13:21.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:21.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:21.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222899+0000", 2020-08-07T15:13:21.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4440000000000001e-05 2020-08-07T15:13:21.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:21.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222953+0000", 2020-08-07T15:13:21.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4560000000000003e-06 2020-08-07T15:13:21.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:21.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222960+0000", 2020-08-07T15:13:21.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.201e-05 2020-08-07T15:13:21.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:21.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222972+0000", 2020-08-07T15:13:21.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4779999999999996e-06 2020-08-07T15:13:21.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:21.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222979+0000", 2020-08-07T15:13:21.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2364000000000002e-05 2020-08-07T15:13:21.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223031+0000", 2020-08-07T15:13:21.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7779999999999996e-06 2020-08-07T15:13:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:21.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223040+0000", 2020-08-07T15:13:21.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029381720000000002 2020-08-07T15:13:21.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.225978+0000", 2020-08-07T15:13:21.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1806e-05 2020-08-07T15:13:21.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:21.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.225990+0000", 2020-08-07T15:13:21.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0316258110000001 2020-08-07T15:13:21.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.257616+0000", 2020-08-07T15:13:21.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000143759 2020-08-07T15:13:21.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:21.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.257760+0000", 2020-08-07T15:13:21.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000296249 2020-08-07T15:13:21.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:21.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.258056+0000", 2020-08-07T15:13:21.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054347309999999999 2020-08-07T15:13:21.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:21.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263491+0000", 2020-08-07T15:13:21.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011408567999999999 2020-08-07T15:13:21.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274899+0000", 2020-08-07T15:13:21.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030791400000000001 2020-08-07T15:13:21.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275207+0000", 2020-08-07T15:13:21.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6499999999999998e-07 2020-08-07T15:13:21.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:21.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275208+0000", 2020-08-07T15:13:21.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3047999999999998e-05 2020-08-07T15:13:21.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:21.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275261+0000", 2020-08-07T15:13:21.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000419706 2020-08-07T15:13:21.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275681+0000", 2020-08-07T15:13:21.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091386099999999997 2020-08-07T15:13:21.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276594+0000", 2020-08-07T15:13:21.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6000000000000001e-06 2020-08-07T15:13:21.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276597+0000", 2020-08-07T15:13:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000212134 2020-08-07T15:13:21.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:21.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276809+0000", 2020-08-07T15:13:21.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0539106499999999 2020-08-07T15:13:21.469 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.469 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:21.469 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.470 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118155 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:21.470 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223018+0000", 2020-08-07T15:13:21.470 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 98.075941677000003, 2020-08-07T15:13:21.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0540362350000001, 2020-08-07T15:13:21.471 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:21.471 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:21.471 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:21.475 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:21.476 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:21.476 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118155 2020-08-07T15:13:21.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.480 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:21.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:21.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223018+0000", 2020-08-07T15:13:21.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:21.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:21.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223018+0000", 2020-08-07T15:13:21.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8291999999999999e-05 2020-08-07T15:13:21.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223057+0000", 2020-08-07T15:13:21.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.805e-06 2020-08-07T15:13:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:21.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223060+0000", 2020-08-07T15:13:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0050000000000004e-06 2020-08-07T15:13:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:21.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223068+0000", 2020-08-07T15:13:21.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4846e-05 2020-08-07T15:13:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223083+0000", 2020-08-07T15:13:21.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8695000000000005e-05 2020-08-07T15:13:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223162+0000", 2020-08-07T15:13:21.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4919999999999996e-06 2020-08-07T15:13:21.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:21.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223169+0000", 2020-08-07T15:13:21.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028562219999999998 2020-08-07T15:13:21.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226026+0000", 2020-08-07T15:13:21.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.027e-05 2020-08-07T15:13:21.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:21.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226036+0000", 2020-08-07T15:13:21.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0327165199999999 2020-08-07T15:13:21.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.258752+0000", 2020-08-07T15:13:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012614899999999999 2020-08-07T15:13:21.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:21.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.258879+0000", 2020-08-07T15:13:21.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000258008 2020-08-07T15:13:21.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:21.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.259137+0000", 2020-08-07T15:13:21.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0067332820000000002 2020-08-07T15:13:21.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:21.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.265870+0000", 2020-08-07T15:13:21.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0090796000000000002 2020-08-07T15:13:21.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274950+0000", 2020-08-07T15:13:21.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000632601 2020-08-07T15:13:21.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275582+0000", 2020-08-07T15:13:21.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8999999999999997e-07 2020-08-07T15:13:21.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:21.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275583+0000", 2020-08-07T15:13:21.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6400999999999998e-05 2020-08-07T15:13:21.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:21.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275629+0000", 2020-08-07T15:13:21.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014887 2020-08-07T15:13:21.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275778+0000", 2020-08-07T15:13:21.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010622559999999999 2020-08-07T15:13:21.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276840+0000", 2020-08-07T15:13:21.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2199999999999999e-07 2020-08-07T15:13:21.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:21.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276841+0000", 2020-08-07T15:13:21.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000214009 2020-08-07T15:13:21.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:21.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277055+0000", 2020-08-07T15:13:21.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0540362350000001 2020-08-07T15:13:21.515 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.515 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:21.516 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.517 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118156 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:21.517 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223123+0000", 2020-08-07T15:13:21.517 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 98.075836811000002, 2020-08-07T15:13:21.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0542400009999999, 2020-08-07T15:13:21.523 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:21.523 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:21.523 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:21.524 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:21.524 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:21.524 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118156 2020-08-07T15:13:21.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.524 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:21.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:21.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223123+0000", 2020-08-07T15:13:21.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:21.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:21.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223123+0000", 2020-08-07T15:13:21.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2212999999999995e-05 2020-08-07T15:13:21.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:21.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223195+0000", 2020-08-07T15:13:21.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8709999999999999e-06 2020-08-07T15:13:21.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:21.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223202+0000", 2020-08-07T15:13:21.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3509999999999999e-05 2020-08-07T15:13:21.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:21.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223216+0000", 2020-08-07T15:13:21.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2729999999999997e-06 2020-08-07T15:13:21.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223223+0000", 2020-08-07T15:13:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6728000000000003e-05 2020-08-07T15:13:21.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223280+0000", 2020-08-07T15:13:21.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1046999999999999e-05 2020-08-07T15:13:21.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:21.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223291+0000", 2020-08-07T15:13:21.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027721030000000002 2020-08-07T15:13:21.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226063+0000", 2020-08-07T15:13:21.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1809000000000004e-05 2020-08-07T15:13:21.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:21.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226125+0000", 2020-08-07T15:13:21.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0337493870000001 2020-08-07T15:13:21.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.259874+0000", 2020-08-07T15:13:21.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000133974 2020-08-07T15:13:21.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:21.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.260008+0000", 2020-08-07T15:13:21.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00064730600000000005 2020-08-07T15:13:21.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:21.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.260655+0000", 2020-08-07T15:13:21.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052233740000000002 2020-08-07T15:13:21.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:21.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.265879+0000", 2020-08-07T15:13:21.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.009433238 2020-08-07T15:13:21.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275312+0000", 2020-08-07T15:13:21.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050955399999999995 2020-08-07T15:13:21.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275822+0000", 2020-08-07T15:13:21.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.58e-07 2020-08-07T15:13:21.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:21.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275822+0000", 2020-08-07T15:13:21.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5469000000000006e-05 2020-08-07T15:13:21.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:21.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275887+0000", 2020-08-07T15:13:21.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000371212 2020-08-07T15:13:21.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276259+0000", 2020-08-07T15:13:21.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00085558499999999996 2020-08-07T15:13:21.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277114+0000", 2020-08-07T15:13:21.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2020-08-07T15:13:21.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:21.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277115+0000", 2020-08-07T15:13:21.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024832000000000001 2020-08-07T15:13:21.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:21.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277363+0000", 2020-08-07T15:13:21.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0542400009999999 2020-08-07T15:13:21.550 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.550 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:21.550 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.551 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118157 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:21.551 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223267+0000", 2020-08-07T15:13:21.551 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 98.075692575999994, 2020-08-07T15:13:21.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0543752369999999, 2020-08-07T15:13:21.551 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:21.555 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:21.555 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:21.556 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:21.556 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:21.556 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118157 2020-08-07T15:13:21.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.556 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:21.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:21.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223267+0000", 2020-08-07T15:13:21.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:21.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:21.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223267+0000", 2020-08-07T15:13:21.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9616000000000001e-05 2020-08-07T15:13:21.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:21.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223317+0000", 2020-08-07T15:13:21.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1359999999999996e-06 2020-08-07T15:13:21.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:21.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223322+0000", 2020-08-07T15:13:21.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0529e-05 2020-08-07T15:13:21.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:21.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223333+0000", 2020-08-07T15:13:21.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1059999999999999e-06 2020-08-07T15:13:21.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:21.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223338+0000", 2020-08-07T15:13:21.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8609000000000002e-05 2020-08-07T15:13:21.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223386+0000", 2020-08-07T15:13:21.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3830000000000003e-06 2020-08-07T15:13:21.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:21.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223394+0000", 2020-08-07T15:13:21.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027643580000000002 2020-08-07T15:13:21.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226158+0000", 2020-08-07T15:13:21.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6809999999999993e-06 2020-08-07T15:13:21.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:21.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226168+0000", 2020-08-07T15:13:21.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.035171356 2020-08-07T15:13:21.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.261339+0000", 2020-08-07T15:13:21.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000130514 2020-08-07T15:13:21.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:21.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.261470+0000", 2020-08-07T15:13:21.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025685699999999998 2020-08-07T15:13:21.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:21.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.261727+0000", 2020-08-07T15:13:21.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075810410000000002 2020-08-07T15:13:21.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:21.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269308+0000", 2020-08-07T15:13:21.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064252490000000001 2020-08-07T15:13:21.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275733+0000", 2020-08-07T15:13:21.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020602000000000001 2020-08-07T15:13:21.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275939+0000", 2020-08-07T15:13:21.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999999e-07 2020-08-07T15:13:21.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:21.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275939+0000", 2020-08-07T15:13:21.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7071999999999997e-05 2020-08-07T15:13:21.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:21.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276016+0000", 2020-08-07T15:13:21.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029090700000000003 2020-08-07T15:13:21.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276307+0000", 2020-08-07T15:13:21.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001114929 2020-08-07T15:13:21.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277422+0000", 2020-08-07T15:13:21.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1200000000000003e-07 2020-08-07T15:13:21.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:21.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277423+0000", 2020-08-07T15:13:21.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021982600000000001 2020-08-07T15:13:21.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:21.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277643+0000", 2020-08-07T15:13:21.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0543752369999999 2020-08-07T15:13:21.585 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.595 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:21.596 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.597 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118158 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:21.597 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223375+0000", 2020-08-07T15:13:21.598 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 98.075584840000005, 2020-08-07T15:13:21.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0545579519999999, 2020-08-07T15:13:21.598 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:21.599 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:21.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:21.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:21.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:21.600 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118158 2020-08-07T15:13:21.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.600 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:21.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:21.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223375+0000", 2020-08-07T15:13:21.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:21.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:21.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223375+0000", 2020-08-07T15:13:21.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7657000000000001e-05 2020-08-07T15:13:21.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:21.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223423+0000", 2020-08-07T15:13:21.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0810000000000001e-06 2020-08-07T15:13:21.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:21.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223428+0000", 2020-08-07T15:13:21.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7839999999999998e-06 2020-08-07T15:13:21.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:21.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223438+0000", 2020-08-07T15:13:21.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.634e-06 2020-08-07T15:13:21.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:21.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223442+0000", 2020-08-07T15:13:21.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0154999999999998e-05 2020-08-07T15:13:21.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223482+0000", 2020-08-07T15:13:21.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9579999999999998e-06 2020-08-07T15:13:21.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:21.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223487+0000", 2020-08-07T15:13:21.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027060999999999999 2020-08-07T15:13:21.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226194+0000", 2020-08-07T15:13:21.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1311999999999999e-05 2020-08-07T15:13:21.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:21.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226205+0000", 2020-08-07T15:13:21.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0361494609999999 2020-08-07T15:13:21.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.262354+0000", 2020-08-07T15:13:21.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9950000000000004e-05 2020-08-07T15:13:21.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:21.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.262454+0000", 2020-08-07T15:13:21.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000244093 2020-08-07T15:13:21.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:21.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.262698+0000", 2020-08-07T15:13:21.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0066176150000000003 2020-08-07T15:13:21.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:21.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269316+0000", 2020-08-07T15:13:21.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006768998 2020-08-07T15:13:21.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276085+0000", 2020-08-07T15:13:21.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9524000000000002e-05 2020-08-07T15:13:21.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276135+0000", 2020-08-07T15:13:21.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6100000000000001e-07 2020-08-07T15:13:21.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:21.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276135+0000", 2020-08-07T15:13:21.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1214e-05 2020-08-07T15:13:21.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:21.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276206+0000", 2020-08-07T15:13:21.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014914999999999999 2020-08-07T15:13:21.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276355+0000", 2020-08-07T15:13:21.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013427770000000001 2020-08-07T15:13:21.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277698+0000", 2020-08-07T15:13:21.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8599999999999998e-07 2020-08-07T15:13:21.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:21.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277699+0000", 2020-08-07T15:13:21.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023444199999999999 2020-08-07T15:13:21.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:21.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277933+0000", 2020-08-07T15:13:21.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0545579519999999 2020-08-07T15:13:21.655 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.655 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:21.655 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118159 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223476+0000", 2020-08-07T15:13:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 98.075483673999997, 2020-08-07T15:13:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0547920749999999, 2020-08-07T15:13:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:21.658 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:21.658 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118159 2020-08-07T15:13:21.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.658 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:21.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223476+0000", 2020-08-07T15:13:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:21.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223476+0000", 2020-08-07T15:13:21.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6094000000000003e-05 2020-08-07T15:13:21.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:21.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223522+0000", 2020-08-07T15:13:21.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6650000000000002e-06 2020-08-07T15:13:21.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:21.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223527+0000", 2020-08-07T15:13:21.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5629999999999998e-06 2020-08-07T15:13:21.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:21.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223537+0000", 2020-08-07T15:13:21.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6810000000000003e-06 2020-08-07T15:13:21.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:21.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223542+0000", 2020-08-07T15:13:21.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6680000000000001e-05 2020-08-07T15:13:21.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223579+0000", 2020-08-07T15:13:21.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2020-08-07T15:13:21.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223586+0000", 2020-08-07T15:13:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026469250000000001 2020-08-07T15:13:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226233+0000", 2020-08-07T15:13:21.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3130000000000004e-06 2020-08-07T15:13:21.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:21.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226242+0000", 2020-08-07T15:13:21.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.037297957 2020-08-07T15:13:21.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263540+0000", 2020-08-07T15:13:21.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4307000000000004e-05 2020-08-07T15:13:21.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:21.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263634+0000", 2020-08-07T15:13:21.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019544300000000001 2020-08-07T15:13:21.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:21.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263830+0000", 2020-08-07T15:13:21.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054939949999999998 2020-08-07T15:13:21.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:21.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269324+0000", 2020-08-07T15:13:21.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070859139999999996 2020-08-07T15:13:21.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276410+0000", 2020-08-07T15:13:21.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0924000000000002e-05 2020-08-07T15:13:21.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276461+0000", 2020-08-07T15:13:21.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4099999999999998e-07 2020-08-07T15:13:21.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276461+0000", 2020-08-07T15:13:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8027000000000004e-05 2020-08-07T15:13:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276539+0000", 2020-08-07T15:13:21.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001458453 2020-08-07T15:13:21.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277998+0000", 2020-08-07T15:13:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7283000000000001e-05 2020-08-07T15:13:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278045+0000", 2020-08-07T15:13:21.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9100000000000004e-07 2020-08-07T15:13:21.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:21.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278046+0000", 2020-08-07T15:13:21.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022290400000000001 2020-08-07T15:13:21.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:21.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278268+0000", 2020-08-07T15:13:21.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0547920749999999 2020-08-07T15:13:21.704 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.704 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:21.704 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.705 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118160 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:21.705 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223579+0000", 2020-08-07T15:13:21.705 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 98.075380693, 2020-08-07T15:13:21.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0553689980000001, 2020-08-07T15:13:21.706 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:21.706 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:21.706 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:21.706 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:21.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:21.707 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118160 2020-08-07T15:13:21.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.707 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:21.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:21.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223579+0000", 2020-08-07T15:13:21.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:21.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:21.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223579+0000", 2020-08-07T15:13:21.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5401000000000001e-05 2020-08-07T15:13:21.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:21.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223625+0000", 2020-08-07T15:13:21.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9069999999999998e-06 2020-08-07T15:13:21.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:21.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223630+0000", 2020-08-07T15:13:21.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4800000000000007e-06 2020-08-07T15:13:21.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:21.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223639+0000", 2020-08-07T15:13:21.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5410000000000002e-06 2020-08-07T15:13:21.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:21.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223644+0000", 2020-08-07T15:13:21.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1476000000000002e-05 2020-08-07T15:13:21.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223675+0000", 2020-08-07T15:13:21.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0240000000000003e-06 2020-08-07T15:13:21.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:21.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223681+0000", 2020-08-07T15:13:21.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025870820000000001 2020-08-07T15:13:21.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226268+0000", 2020-08-07T15:13:21.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0540000000000001e-05 2020-08-07T15:13:21.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:21.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226279+0000", 2020-08-07T15:13:21.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.038209656 2020-08-07T15:13:21.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.264488+0000", 2020-08-07T15:13:21.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029908499999999999 2020-08-07T15:13:21.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:21.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.264788+0000", 2020-08-07T15:13:21.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000216997 2020-08-07T15:13:21.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:21.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.265005+0000", 2020-08-07T15:13:21.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0060308230000000003 2020-08-07T15:13:21.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:21.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271035+0000", 2020-08-07T15:13:21.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0072889499999999998 2020-08-07T15:13:21.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278324+0000", 2020-08-07T15:13:21.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4982999999999999e-05 2020-08-07T15:13:21.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278369+0000", 2020-08-07T15:13:21.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4000000000000002e-07 2020-08-07T15:13:21.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:21.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278370+0000", 2020-08-07T15:13:21.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7656000000000003e-05 2020-08-07T15:13:21.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:21.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278447+0000", 2020-08-07T15:13:21.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000277416 2020-08-07T15:13:21.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278725+0000", 2020-08-07T15:13:21.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2345000000000001e-05 2020-08-07T15:13:21.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278777+0000", 2020-08-07T15:13:21.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8500000000000002e-07 2020-08-07T15:13:21.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:21.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278778+0000", 2020-08-07T15:13:21.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017071100000000001 2020-08-07T15:13:21.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:21.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278948+0000", 2020-08-07T15:13:21.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0553689980000001 2020-08-07T15:13:21.740 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.740 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:21.740 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.741 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118441 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:21.741 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223679+0000", 2020-08-07T15:13:21.741 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 98.075280677999999, 2020-08-07T15:13:21.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057502779, 2020-08-07T15:13:21.742 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:21.742 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:21.742 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:21.742 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:21.743 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:21.743 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118441 2020-08-07T15:13:21.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.743 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:21.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:21.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223679+0000", 2020-08-07T15:13:21.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:21.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:21.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223679+0000", 2020-08-07T15:13:21.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5559999999999997e-05 2020-08-07T15:13:21.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:21.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223725+0000", 2020-08-07T15:13:21.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1250000000000001e-06 2020-08-07T15:13:21.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:21.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223730+0000", 2020-08-07T15:13:21.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0046e-05 2020-08-07T15:13:21.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:21.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223740+0000", 2020-08-07T15:13:21.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5979999999999999e-06 2020-08-07T15:13:21.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:21.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223745+0000", 2020-08-07T15:13:21.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7234999999999998e-05 2020-08-07T15:13:21.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223782+0000", 2020-08-07T15:13:21.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7789999999999998e-06 2020-08-07T15:13:21.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:21.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223789+0000", 2020-08-07T15:13:21.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002516169 2020-08-07T15:13:21.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226305+0000", 2020-08-07T15:13:21.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3860000000000007e-06 2020-08-07T15:13:21.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:21.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226313+0000", 2020-08-07T15:13:21.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0396126269999999 2020-08-07T15:13:21.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.265926+0000", 2020-08-07T15:13:21.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00079104700000000002 2020-08-07T15:13:21.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:21.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.266717+0000", 2020-08-07T15:13:21.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024298599999999999 2020-08-07T15:13:21.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:21.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.266960+0000", 2020-08-07T15:13:21.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040855509999999998 2020-08-07T15:13:21.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:21.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271045+0000", 2020-08-07T15:13:21.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0074549170000000001 2020-08-07T15:13:21.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278500+0000", 2020-08-07T15:13:21.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8816999999999997e-05 2020-08-07T15:13:21.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278549+0000", 2020-08-07T15:13:21.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9000000000000002e-07 2020-08-07T15:13:21.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:21.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278550+0000", 2020-08-07T15:13:21.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0120999999999997e-05 2020-08-07T15:13:21.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:21.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278600+0000", 2020-08-07T15:13:21.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00092631699999999996 2020-08-07T15:13:21.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279526+0000", 2020-08-07T15:13:21.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014504520000000001 2020-08-07T15:13:21.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280976+0000", 2020-08-07T15:13:21.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6899999999999997e-07 2020-08-07T15:13:21.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:21.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280977+0000", 2020-08-07T15:13:21.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020508700000000001 2020-08-07T15:13:21.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:21.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281182+0000", 2020-08-07T15:13:21.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057502779 2020-08-07T15:13:21.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.769 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:21.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.770 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118442 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:21.770 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223780+0000", 2020-08-07T15:13:21.770 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 98.075179808000001, 2020-08-07T15:13:21.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0575529850000001, 2020-08-07T15:13:21.771 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:21.771 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:21.771 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:21.771 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:21.772 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:21.772 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118442 2020-08-07T15:13:21.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.772 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:21.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:21.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223780+0000", 2020-08-07T15:13:21.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:21.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:21.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223780+0000", 2020-08-07T15:13:21.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0340999999999998e-05 2020-08-07T15:13:21.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:21.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223831+0000", 2020-08-07T15:13:21.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3499999999999996e-06 2020-08-07T15:13:21.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:21.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223836+0000", 2020-08-07T15:13:21.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0414999999999999e-05 2020-08-07T15:13:21.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:21.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223846+0000", 2020-08-07T15:13:21.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9239999999999998e-06 2020-08-07T15:13:21.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:21.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223851+0000", 2020-08-07T15:13:21.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5946000000000003e-05 2020-08-07T15:13:21.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223907+0000", 2020-08-07T15:13:21.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6189999999999996e-06 2020-08-07T15:13:21.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:21.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223914+0000", 2020-08-07T15:13:21.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002424149 2020-08-07T15:13:21.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226338+0000", 2020-08-07T15:13:21.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2830000000000001e-06 2020-08-07T15:13:21.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:21.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226346+0000", 2020-08-07T15:13:21.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0412357219999999 2020-08-07T15:13:21.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.267582+0000", 2020-08-07T15:13:21.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000123924 2020-08-07T15:13:21.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:21.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.267706+0000", 2020-08-07T15:13:21.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000216376 2020-08-07T15:13:21.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:21.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.267922+0000", 2020-08-07T15:13:21.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031340790000000001 2020-08-07T15:13:21.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:21.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271056+0000", 2020-08-07T15:13:21.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.007623276 2020-08-07T15:13:21.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278680+0000", 2020-08-07T15:13:21.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000333853 2020-08-07T15:13:21.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279013+0000", 2020-08-07T15:13:21.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.02e-06 2020-08-07T15:13:21.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:21.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279014+0000", 2020-08-07T15:13:21.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6306999999999999e-05 2020-08-07T15:13:21.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:21.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279101+0000", 2020-08-07T15:13:21.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050739800000000005 2020-08-07T15:13:21.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279608+0000", 2020-08-07T15:13:21.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001617098 2020-08-07T15:13:21.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281225+0000", 2020-08-07T15:13:21.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9700000000000002e-07 2020-08-07T15:13:21.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:21.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281226+0000", 2020-08-07T15:13:21.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107508 2020-08-07T15:13:21.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:21.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281333+0000", 2020-08-07T15:13:21.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0575529850000001 2020-08-07T15:13:21.809 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.809 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:21.810 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.810 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118443 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:21.810 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223889+0000", 2020-08-07T15:13:21.811 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 98.075071137999998, 2020-08-07T15:13:21.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.05758606, 2020-08-07T15:13:21.811 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:21.811 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:21.811 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:21.812 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:21.812 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:21.812 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118443 2020-08-07T15:13:21.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.812 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:21.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:21.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223889+0000", 2020-08-07T15:13:21.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:21.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:21.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223889+0000", 2020-08-07T15:13:21.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8148e-05 2020-08-07T15:13:21.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:21.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223937+0000", 2020-08-07T15:13:21.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2789999999999998e-06 2020-08-07T15:13:21.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:21.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223942+0000", 2020-08-07T15:13:21.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0472000000000001e-05 2020-08-07T15:13:21.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:21.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223953+0000", 2020-08-07T15:13:21.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6580000000000001e-06 2020-08-07T15:13:21.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:21.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223957+0000", 2020-08-07T15:13:21.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5981000000000001e-05 2020-08-07T15:13:21.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223983+0000", 2020-08-07T15:13:21.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.288e-06 2020-08-07T15:13:21.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:21.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223989+0000", 2020-08-07T15:13:21.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024051889999999999 2020-08-07T15:13:21.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226394+0000", 2020-08-07T15:13:21.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2439000000000001e-05 2020-08-07T15:13:21.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:21.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226406+0000", 2020-08-07T15:13:21.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0421168519999999 2020-08-07T15:13:21.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.268523+0000", 2020-08-07T15:13:21.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0449000000000002e-05 2020-08-07T15:13:21.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:21.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.268604+0000", 2020-08-07T15:13:21.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014104300000000001 2020-08-07T15:13:21.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:21.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.268745+0000", 2020-08-07T15:13:21.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051249829999999996 2020-08-07T15:13:21.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:21.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.273870+0000", 2020-08-07T15:13:21.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052772840000000001 2020-08-07T15:13:21.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279147+0000", 2020-08-07T15:13:21.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6060000000000003e-05 2020-08-07T15:13:21.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279193+0000", 2020-08-07T15:13:21.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9999999999999997e-07 2020-08-07T15:13:21.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:21.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279194+0000", 2020-08-07T15:13:21.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3566000000000003e-05 2020-08-07T15:13:21.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:21.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279237+0000", 2020-08-07T15:13:21.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011546359999999999 2020-08-07T15:13:21.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280392+0000", 2020-08-07T15:13:21.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00097857799999999991 2020-08-07T15:13:21.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281370+0000", 2020-08-07T15:13:21.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999999e-07 2020-08-07T15:13:21.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:21.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281371+0000", 2020-08-07T15:13:21.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104119 2020-08-07T15:13:21.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:21.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281475+0000", 2020-08-07T15:13:21.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.05758606 2020-08-07T15:13:21.849 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.851 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:21.851 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.852 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118444 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:21.852 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223994+0000", 2020-08-07T15:13:21.852 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 98.074965762999994, 2020-08-07T15:13:21.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057634572, 2020-08-07T15:13:21.853 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:21.853 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:21.853 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:21.853 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:21.853 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:21.854 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118444 2020-08-07T15:13:21.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.854 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:21.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:21.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223994+0000", 2020-08-07T15:13:21.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:21.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:21.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223994+0000", 2020-08-07T15:13:21.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1279999999999997e-05 2020-08-07T15:13:21.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:21.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224046+0000", 2020-08-07T15:13:21.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6930000000000004e-06 2020-08-07T15:13:21.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:21.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224051+0000", 2020-08-07T15:13:21.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.095e-05 2020-08-07T15:13:21.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:21.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224062+0000", 2020-08-07T15:13:21.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9529999999999998e-06 2020-08-07T15:13:21.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:21.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224067+0000", 2020-08-07T15:13:21.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000162091 2020-08-07T15:13:21.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224229+0000", 2020-08-07T15:13:21.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.276e-06 2020-08-07T15:13:21.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:21.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224238+0000", 2020-08-07T15:13:21.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022140409999999999 2020-08-07T15:13:21.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226452+0000", 2020-08-07T15:13:21.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5951999999999999e-05 2020-08-07T15:13:21.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:21.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226467+0000", 2020-08-07T15:13:21.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0428868099999999 2020-08-07T15:13:21.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269354+0000", 2020-08-07T15:13:21.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5264000000000003e-05 2020-08-07T15:13:21.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:21.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269430+0000", 2020-08-07T15:13:21.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020438499999999999 2020-08-07T15:13:21.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:21.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269634+0000", 2020-08-07T15:13:21.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042509959999999999 2020-08-07T15:13:21.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:21.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.273885+0000", 2020-08-07T15:13:21.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054290140000000002 2020-08-07T15:13:21.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279314+0000", 2020-08-07T15:13:21.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00038996700000000002 2020-08-07T15:13:21.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279704+0000", 2020-08-07T15:13:21.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5700000000000002e-07 2020-08-07T15:13:21.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279705+0000", 2020-08-07T15:13:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9377999999999997e-05 2020-08-07T15:13:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279794+0000", 2020-08-07T15:13:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00064521299999999995 2020-08-07T15:13:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280439+0000", 2020-08-07T15:13:21.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001072185 2020-08-07T15:13:21.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281511+0000", 2020-08-07T15:13:21.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7300000000000002e-07 2020-08-07T15:13:21.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:21.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281512+0000", 2020-08-07T15:13:21.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117094 2020-08-07T15:13:21.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281629+0000", 2020-08-07T15:13:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057634572 2020-08-07T15:13:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.913 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118445 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:21.913 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.224116+0000", 2020-08-07T15:13:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 98.074843822000005, 2020-08-07T15:13:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057744163, 2020-08-07T15:13:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118445 2020-08-07T15:13:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224116+0000", 2020-08-07T15:13:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224116+0000", 2020-08-07T15:13:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3542999999999999e-05 2020-08-07T15:13:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224170+0000", 2020-08-07T15:13:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.446e-06 2020-08-07T15:13:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224175+0000", 2020-08-07T15:13:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1473e-05 2020-08-07T15:13:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224187+0000", 2020-08-07T15:13:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9449999999999998e-06 2020-08-07T15:13:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224193+0000", 2020-08-07T15:13:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4945999999999997e-05 2020-08-07T15:13:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224268+0000", 2020-08-07T15:13:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5799999999999992e-06 2020-08-07T15:13:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224276+0000", 2020-08-07T15:13:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022246980000000002 2020-08-07T15:13:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226501+0000", 2020-08-07T15:13:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1359999999999997e-06 2020-08-07T15:13:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226509+0000", 2020-08-07T15:13:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.043599741 2020-08-07T15:13:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.270109+0000", 2020-08-07T15:13:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000129503 2020-08-07T15:13:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.270238+0000", 2020-08-07T15:13:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021162500000000001 2020-08-07T15:13:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.270450+0000", 2020-08-07T15:13:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036313090000000001 2020-08-07T15:13:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274081+0000", 2020-08-07T15:13:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00557165 2020-08-07T15:13:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279653+0000", 2020-08-07T15:13:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000196658 2020-08-07T15:13:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279849+0000", 2020-08-07T15:13:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6700000000000003e-07 2020-08-07T15:13:21.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:21.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279850+0000", 2020-08-07T15:13:21.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6522999999999998e-05 2020-08-07T15:13:21.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279937+0000", 2020-08-07T15:13:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00055050400000000005 2020-08-07T15:13:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280487+0000", 2020-08-07T15:13:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011833760000000001 2020-08-07T15:13:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281670+0000", 2020-08-07T15:13:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2020-08-07T15:13:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281671+0000", 2020-08-07T15:13:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001896 2020-08-07T15:13:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281860+0000", 2020-08-07T15:13:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057744163 2020-08-07T15:13:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118446 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.224256+0000", 2020-08-07T15:13:21.970 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 98.074704037000004, 2020-08-07T15:13:21.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057864804, 2020-08-07T15:13:21.971 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:21.971 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:21.971 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:21.971 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118446 2020-08-07T15:13:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224256+0000", 2020-08-07T15:13:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:21.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224256+0000", 2020-08-07T15:13:21.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5838000000000003e-05 2020-08-07T15:13:21.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:21.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224312+0000", 2020-08-07T15:13:21.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.879e-06 2020-08-07T15:13:21.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:21.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224318+0000", 2020-08-07T15:13:21.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3593e-05 2020-08-07T15:13:21.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:21.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224331+0000", 2020-08-07T15:13:21.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6169999999999998e-06 2020-08-07T15:13:21.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:21.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224337+0000", 2020-08-07T15:13:21.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7246000000000001e-05 2020-08-07T15:13:21.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224384+0000", 2020-08-07T15:13:21.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7690000000000008e-06 2020-08-07T15:13:21.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:21.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224394+0000", 2020-08-07T15:13:21.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021394589999999998 2020-08-07T15:13:21.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226533+0000", 2020-08-07T15:13:21.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3849999999999996e-06 2020-08-07T15:13:21.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:21.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226541+0000", 2020-08-07T15:13:21.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0445569109999999 2020-08-07T15:13:21.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:21.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271098+0000", 2020-08-07T15:13:21.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012137 2020-08-07T15:13:21.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:21.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271219+0000", 2020-08-07T15:13:21.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013978300000000001 2020-08-07T15:13:21.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:21.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271359+0000", 2020-08-07T15:13:21.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027367699999999999 2020-08-07T15:13:21.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:21.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274096+0000", 2020-08-07T15:13:21.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058973660000000002 2020-08-07T15:13:21.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279993+0000", 2020-08-07T15:13:21.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116291 2020-08-07T15:13:21.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:21.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280109+0000", 2020-08-07T15:13:21.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1099999999999995e-07 2020-08-07T15:13:21.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:21.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280110+0000", 2020-08-07T15:13:21.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.564e-05 2020-08-07T15:13:21.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:21.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:21.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:21.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280196+0000", 2020-08-07T15:13:22.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061969 2020-08-07T15:13:22.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:22.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280815+0000", 2020-08-07T15:13:22.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001097612 2020-08-07T15:13:22.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:22.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281913+0000", 2020-08-07T15:13:22.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4300000000000003e-07 2020-08-07T15:13:22.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:22.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281913+0000", 2020-08-07T15:13:22.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000207431 2020-08-07T15:13:22.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:22.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282121+0000", 2020-08-07T15:13:22.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057864804 2020-08-07T15:13:22.004 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:22.005 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:22.005 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:22.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.006 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118447 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:22.006 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.224379+0000", 2020-08-07T15:13:22.006 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 98.074581271, 2020-08-07T15:13:22.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0579997480000001, 2020-08-07T15:13:22.006 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:22.007 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:22.014 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:22.014 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:22.014 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:22.014 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118447 2020-08-07T15:13:22.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.015 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:22.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:22.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224379+0000", 2020-08-07T15:13:22.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:22.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:22.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224379+0000", 2020-08-07T15:13:22.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9775999999999999e-05 2020-08-07T15:13:22.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:22.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224429+0000", 2020-08-07T15:13:22.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8500000000000002e-06 2020-08-07T15:13:22.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:22.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224433+0000", 2020-08-07T15:13:22.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7340000000000008e-06 2020-08-07T15:13:22.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:22.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224441+0000", 2020-08-07T15:13:22.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.031e-06 2020-08-07T15:13:22.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:22.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224445+0000", 2020-08-07T15:13:22.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4264000000000003e-05 2020-08-07T15:13:22.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:22.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224479+0000", 2020-08-07T15:13:22.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0039999999999999e-06 2020-08-07T15:13:22.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:22.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224487+0000", 2020-08-07T15:13:22.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002241682 2020-08-07T15:13:22.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:22.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226729+0000", 2020-08-07T15:13:22.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0087e-05 2020-08-07T15:13:22.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:22.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226739+0000", 2020-08-07T15:13:22.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.045170006 2020-08-07T15:13:22.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:22.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271909+0000", 2020-08-07T15:13:22.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012782900000000001 2020-08-07T15:13:22.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:22.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.272037+0000", 2020-08-07T15:13:22.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024079400000000001 2020-08-07T15:13:22.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:22.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.272278+0000", 2020-08-07T15:13:22.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023454399999999999 2020-08-07T15:13:22.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:22.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274623+0000", 2020-08-07T15:13:22.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054268190000000003 2020-08-07T15:13:22.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:22.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280050+0000", 2020-08-07T15:13:22.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00048391200000000002 2020-08-07T15:13:22.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:22.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280534+0000", 2020-08-07T15:13:22.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5400000000000001e-07 2020-08-07T15:13:22.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:22.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280535+0000", 2020-08-07T15:13:22.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8922e-05 2020-08-07T15:13:22.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:22.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280623+0000", 2020-08-07T15:13:22.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024637999999999998 2020-08-07T15:13:22.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:22.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280870+0000", 2020-08-07T15:13:22.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001300142 2020-08-07T15:13:22.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:22.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282170+0000", 2020-08-07T15:13:22.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2099999999999997e-07 2020-08-07T15:13:22.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:22.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282170+0000", 2020-08-07T15:13:22.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000208001 2020-08-07T15:13:22.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:22.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282378+0000", 2020-08-07T15:13:22.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0579997480000001 2020-08-07T15:13:22.056 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:22.056 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:22.056 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:22.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.057 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118448 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:22.058 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.224474+0000", 2020-08-07T15:13:22.071 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 98.074486285000006, 2020-08-07T15:13:22.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0581683310000001, 2020-08-07T15:13:22.072 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:22.072 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:22.072 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:22.073 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:22.073 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:22.082 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118448 2020-08-07T15:13:22.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.084 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:22.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:22.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224474+0000", 2020-08-07T15:13:22.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:22.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:22.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224474+0000", 2020-08-07T15:13:22.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3804000000000002e-05 2020-08-07T15:13:22.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:22.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224528+0000", 2020-08-07T15:13:22.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5140000000000001e-06 2020-08-07T15:13:22.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:22.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224533+0000", 2020-08-07T15:13:22.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3315999999999999e-05 2020-08-07T15:13:22.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:22.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224546+0000", 2020-08-07T15:13:22.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7240000000000003e-06 2020-08-07T15:13:22.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:22.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224553+0000", 2020-08-07T15:13:22.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002226263 2020-08-07T15:13:22.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:22.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226779+0000", 2020-08-07T15:13:22.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1362000000000001e-05 2020-08-07T15:13:22.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:22.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226791+0000", 2020-08-07T15:13:22.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.046092652 2020-08-07T15:13:22.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:22.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.272883+0000", 2020-08-07T15:13:22.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000115981 2020-08-07T15:13:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:22.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.272999+0000", 2020-08-07T15:13:22.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029648899999999999 2020-08-07T15:13:22.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:22.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.273296+0000", 2020-08-07T15:13:22.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001852521 2020-08-07T15:13:22.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:22.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275148+0000", 2020-08-07T15:13:22.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051948309999999996 2020-08-07T15:13:22.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:22.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280343+0000", 2020-08-07T15:13:22.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033301799999999998 2020-08-07T15:13:22.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:22.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280676+0000", 2020-08-07T15:13:22.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8400000000000004e-07 2020-08-07T15:13:22.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:22.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280677+0000", 2020-08-07T15:13:22.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9652000000000002e-05 2020-08-07T15:13:22.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:22.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280756+0000", 2020-08-07T15:13:22.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017072699999999999 2020-08-07T15:13:22.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:22.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280927+0000", 2020-08-07T15:13:22.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015026530000000001 2020-08-07T15:13:22.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:22.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282430+0000", 2020-08-07T15:13:22.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4700000000000001e-07 2020-08-07T15:13:22.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:22.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282430+0000", 2020-08-07T15:13:22.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021169299999999999 2020-08-07T15:13:22.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:22.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282642+0000", 2020-08-07T15:13:22.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0581683310000001 2020-08-07T15:13:22.187 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:22.187 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:22.187 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:22.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.188 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:67303 6.a 6:5c7a55ef:::benchmark_data_smithi074_63843_object8412:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e265)", 2020-08-07T15:13:22.189 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:56.271945+0000", 2020-08-07T15:13:22.189 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.027014545, 2020-08-07T15:13:22.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.007577197, 2020-08-07T15:13:22.192 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:22.192 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:22.193 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:22.193 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:13:22.193 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:22.194 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 67303 2020-08-07T15:13:22.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.194 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:22.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:22.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.271945+0000", 2020-08-07T15:13:22.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:22.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:22.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.271945+0000", 2020-08-07T15:13:22.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9969000000000002e-05 2020-08-07T15:13:22.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:22.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.271995+0000", 2020-08-07T15:13:22.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3690000000000001e-06 2020-08-07T15:13:22.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:22.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.272003+0000", 2020-08-07T15:13:22.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2183000000000001e-05 2020-08-07T15:13:22.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:22.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.272025+0000", 2020-08-07T15:13:22.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7299999999999999e-06 2020-08-07T15:13:22.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:22.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.272032+0000", 2020-08-07T15:13:22.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5831e-05 2020-08-07T15:13:22.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:22.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.272108+0000", 2020-08-07T15:13:22.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2233e-05 2020-08-07T15:13:22.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:22.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.272120+0000", 2020-08-07T15:13:22.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27969333699999999 2020-08-07T15:13:22.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:22.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.551813+0000", 2020-08-07T15:13:22.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8549999999999997e-06 2020-08-07T15:13:22.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:22.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.551823+0000", 2020-08-07T15:13:22.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.71905319899999998 2020-08-07T15:13:22.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:22.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.270876+0000", 2020-08-07T15:13:22.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116937 2020-08-07T15:13:22.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:22.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.270993+0000", 2020-08-07T15:13:22.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020587399999999999 2020-08-07T15:13:22.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,5,7", 2020-08-07T15:13:22.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.271199+0000", 2020-08-07T15:13:22.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023173429999999999 2020-08-07T15:13:22.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:22.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.273516+0000", 2020-08-07T15:13:22.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053300810000000004 2020-08-07T15:13:22.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:22.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.278846+0000", 2020-08-07T15:13:22.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1699999999999998e-07 2020-08-07T15:13:22.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:22.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.278847+0000", 2020-08-07T15:13:22.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7559000000000003e-05 2020-08-07T15:13:22.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:22.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.278894+0000", 2020-08-07T15:13:22.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000119676 2020-08-07T15:13:22.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:22.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.279014+0000", 2020-08-07T15:13:22.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039420500000000001 2020-08-07T15:13:22.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:22.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.279408+0000", 2020-08-07T15:13:22.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0200000000000003e-07 2020-08-07T15:13:22.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:22.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.279409+0000", 2020-08-07T15:13:22.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113897 2020-08-07T15:13:22.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:22.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.279523+0000", 2020-08-07T15:13:22.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.007577197 2020-08-07T15:13:22.267 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:22.267 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:22.268 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:22.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.269 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:67304 6.a 6:5c7a55ef:::benchmark_data_smithi074_63843_object8412:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e265)", 2020-08-07T15:13:22.269 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:56.410012+0000", 2020-08-07T15:13:22.270 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.888947794, 2020-08-07T15:13:22.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.86966374199999996, 2020-08-07T15:13:22.270 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:22.271 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:22.271 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:22.271 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:13:22.272 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:22.272 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 67304 2020-08-07T15:13:22.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.273 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:22.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:22.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.410012+0000", 2020-08-07T15:13:22.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:22.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:22.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.410012+0000", 2020-08-07T15:13:22.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1640000000000002e-05 2020-08-07T15:13:22.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:22.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.410044+0000", 2020-08-07T15:13:22.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8099999999999997e-06 2020-08-07T15:13:22.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:22.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.410049+0000", 2020-08-07T15:13:22.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1989999999999995e-06 2020-08-07T15:13:22.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:22.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.410058+0000", 2020-08-07T15:13:22.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6249999999999998e-06 2020-08-07T15:13:22.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:22.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.410066+0000", 2020-08-07T15:13:22.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2998999999999998e-05 2020-08-07T15:13:22.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:22.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.410129+0000", 2020-08-07T15:13:22.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6420000000000002e-06 2020-08-07T15:13:22.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:22.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.410138+0000", 2020-08-07T15:13:22.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14171407799999999 2020-08-07T15:13:22.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:22.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.551852+0000", 2020-08-07T15:13:22.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0237e-05 2020-08-07T15:13:22.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:22.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.551862+0000", 2020-08-07T15:13:22.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.71982975800000004 2020-08-07T15:13:22.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:22.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.271692+0000", 2020-08-07T15:13:22.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0882000000000002e-05 2020-08-07T15:13:22.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:22.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.271783+0000", 2020-08-07T15:13:22.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000212067 2020-08-07T15:13:22.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,5,7", 2020-08-07T15:13:22.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.271995+0000", 2020-08-07T15:13:22.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021518420000000002 2020-08-07T15:13:22.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:22.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.274147+0000", 2020-08-07T15:13:22.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0047817959999999996 2020-08-07T15:13:22.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:22.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.278929+0000", 2020-08-07T15:13:22.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2099999999999997e-07 2020-08-07T15:13:22.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:22.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.278929+0000", 2020-08-07T15:13:22.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4972000000000002e-05 2020-08-07T15:13:22.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:22.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.278974+0000", 2020-08-07T15:13:22.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5584e-05 2020-08-07T15:13:22.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:22.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.279050+0000", 2020-08-07T15:13:22.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051383899999999998 2020-08-07T15:13:22.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:22.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.279564+0000", 2020-08-07T15:13:22.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9900000000000001e-07 2020-08-07T15:13:22.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:22.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.279564+0000", 2020-08-07T15:13:22.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111852 2020-08-07T15:13:22.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:22.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:22.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:22.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.279676+0000", 2020-08-07T15:13:22.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.86966374199999996 2020-08-07T15:13:22.362 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:22.362 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:22.362 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:22.363 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:22.364 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:22.365 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:22.377 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:13:22.379 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 70 16 104145 104129 11.6201 13.2266 0.00723427 0.0107529 2020-08-07T15:13:22.379 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:22.365+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:22.380 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:13:22.474 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:22.470+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:22.572 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:22.571+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:22.590 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:22.590 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:13:22.591 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2020-08-07T15:13:22.591 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:22.603 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:13:22.607 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:13:22.690 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:22.688+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:22.776 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:22.773+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:22.802 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:22.803 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:13:22.803 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:13:22.803 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:13:22.803 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:22.813 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:13:22.889 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:22.879+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:22.981 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:22.980+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:23.052 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:23.052 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:13:23.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:13:23.053 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:13:23.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.053 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15576 3.e 3:727724a5:::benchmark_data_smithi074_35447_object1946:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:13:23.053 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:13:23.054 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 396.38158071100003, 2020-08-07T15:13:23.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999, 2020-08-07T15:13:23.054 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:23.054 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:23.054 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:23.055 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:13:23.055 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:13:23.055 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15576 2020-08-07T15:13:23.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.055 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:23.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:23.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:13:23.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:23.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:23.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:13:23.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7156e-05 2020-08-07T15:13:23.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:23.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666682+0000", 2020-08-07T15:13:23.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0209999999999999e-06 2020-08-07T15:13:23.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:23.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666687+0000", 2020-08-07T15:13:23.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0730999999999999e-05 2020-08-07T15:13:23.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:23.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666698+0000", 2020-08-07T15:13:23.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2020-08-07T15:13:23.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:23.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666702+0000", 2020-08-07T15:13:23.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023978459999999999 2020-08-07T15:13:23.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:23.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669100+0000", 2020-08-07T15:13:23.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9664000000000001e-05 2020-08-07T15:13:23.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:23.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669190+0000", 2020-08-07T15:13:23.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015957100000000001 2020-08-07T15:13:23.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:13:23.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669350+0000", 2020-08-07T15:13:23.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016470149999999999 2020-08-07T15:13:23.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:23.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:13:23.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1299999999999999e-07 2020-08-07T15:13:23.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:23.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:13:23.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5269000000000002e-05 2020-08-07T15:13:23.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:23.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.671022+0000", 2020-08-07T15:13:23.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031414400000000001 2020-08-07T15:13:23.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:23.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.674163+0000", 2020-08-07T15:13:23.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45641417400000001 2020-08-07T15:13:23.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:23.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:13:23.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0399999999999995e-07 2020-08-07T15:13:23.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:23.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:13:23.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3343999999999997e-05 2020-08-07T15:13:23.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:23.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130622+0000", 2020-08-07T15:13:23.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999 2020-08-07T15:13:23.101 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.101 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:23.101 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.121 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106248 3.f 3:f950fd9d:::benchmark_data_smithi074_35447_object13280:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:13:23.122 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:13:23.122 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 335.68805117099998, 2020-08-07T15:13:23.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002, 2020-08-07T15:13:23.123 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:23.123 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:23.123 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:23.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:13:23.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:13:23.124 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106248 2020-08-07T15:13:23.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.125 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:23.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:13:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:13:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7184e-05 2020-08-07T15:13:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360222+0000", 2020-08-07T15:13:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0906e-05 2020-08-07T15:13:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360233+0000", 2020-08-07T15:13:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2972999999999999e-05 2020-08-07T15:13:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360246+0000", 2020-08-07T15:13:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0114e-05 2020-08-07T15:13:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360256+0000", 2020-08-07T15:13:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1303999999999997e-05 2020-08-07T15:13:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360287+0000", 2020-08-07T15:13:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3798000000000001e-05 2020-08-07T15:13:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360351+0000", 2020-08-07T15:13:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111308 2020-08-07T15:13:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:13:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360462+0000", 2020-08-07T15:13:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003480674 2020-08-07T15:13:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:13:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0900000000000001e-07 2020-08-07T15:13:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:13:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8899999999999994e-05 2020-08-07T15:13:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.364012+0000", 2020-08-07T15:13:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35291637199999998 2020-08-07T15:13:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716929+0000", 2020-08-07T15:13:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.076244233999999994 2020-08-07T15:13:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793173+0000", 2020-08-07T15:13:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6109999999999999e-06 2020-08-07T15:13:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793175+0000", 2020-08-07T15:13:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7492999999999999e-05 2020-08-07T15:13:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793242+0000", 2020-08-07T15:13:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002 2020-08-07T15:13:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106248 3.f e59/20 ondisk, result = 0)", 2020-08-07T15:13:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:13:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 335.68278788600003, 2020-08-07T15:13:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001, 2020-08-07T15:13:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:13:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:13:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7518999999999999e-05 2020-08-07T15:13:23.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:23.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365445+0000", 2020-08-07T15:13:23.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-06 2020-08-07T15:13:23.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:23.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365449+0000", 2020-08-07T15:13:23.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42764147600000002 2020-08-07T15:13:23.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:23.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793091+0000", 2020-08-07T15:13:23.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6500000000000002e-06 2020-08-07T15:13:23.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:23.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793099+0000", 2020-08-07T15:13:23.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3664e-05 2020-08-07T15:13:23.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:23.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793153+0000", 2020-08-07T15:13:23.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1078e-05 2020-08-07T15:13:23.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:23.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793164+0000", 2020-08-07T15:13:23.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3404000000000003e-05 2020-08-07T15:13:23.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:23.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793248+0000", 2020-08-07T15:13:23.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001 2020-08-07T15:13:23.196 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.197 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:23.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.197 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:107952 3.d e61/35)", 2020-08-07T15:13:23.198 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:13:23.198 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 332.72185817799999, 2020-08-07T15:13:23.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001, 2020-08-07T15:13:23.198 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:23.198 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:23.199 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:23.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:23.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:13:23.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:23.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:23.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:13:23.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8826000000000003e-05 2020-08-07T15:13:23.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:23.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326406+0000", 2020-08-07T15:13:23.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9199999999999998e-06 2020-08-07T15:13:23.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:23.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326413+0000", 2020-08-07T15:13:23.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75082853500000002 2020-08-07T15:13:23.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:23.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077242+0000", 2020-08-07T15:13:23.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1479999999999999e-06 2020-08-07T15:13:23.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:23.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077246+0000", 2020-08-07T15:13:23.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3331999999999998e-05 2020-08-07T15:13:23.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:23.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077289+0000", 2020-08-07T15:13:23.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8977000000000001e-05 2020-08-07T15:13:23.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:23.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077308+0000", 2020-08-07T15:13:23.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001 2020-08-07T15:13:23.206 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.206 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:23.207 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.207 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31696 4.0 4:0335eed9:::benchmark_data_smithi074_44586_object3961:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e100)", 2020-08-07T15:13:23.207 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:13:23.208 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 274.94395061900002, 2020-08-07T15:13:23.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237, 2020-08-07T15:13:23.208 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:23.208 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:23.208 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:23.209 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:23.209 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:23.209 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31696 2020-08-07T15:13:23.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.209 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:23.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:23.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:13:23.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:23.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:23.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:13:23.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7654e-05 2020-08-07T15:13:23.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:23.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104293+0000", 2020-08-07T15:13:23.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.061e-06 2020-08-07T15:13:23.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:23.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104296+0000", 2020-08-07T15:13:23.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0859999999999998e-06 2020-08-07T15:13:23.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:23.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104302+0000", 2020-08-07T15:13:23.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8030000000000001e-06 2020-08-07T15:13:23.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:23.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104305+0000", 2020-08-07T15:13:23.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037634769999999999 2020-08-07T15:13:23.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:23.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108068+0000", 2020-08-07T15:13:23.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8814999999999998e-05 2020-08-07T15:13:23.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:23.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108137+0000", 2020-08-07T15:13:23.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111455 2020-08-07T15:13:23.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:13:23.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108248+0000", 2020-08-07T15:13:23.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028756820000000001 2020-08-07T15:13:23.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:23.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111124+0000", 2020-08-07T15:13:23.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061054499999999999 2020-08-07T15:13:23.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:23.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111735+0000", 2020-08-07T15:13:23.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.423966167 2020-08-07T15:13:23.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:23.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535701+0000", 2020-08-07T15:13:23.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4929999999999999e-06 2020-08-07T15:13:23.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:23.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535702+0000", 2020-08-07T15:13:23.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2571999999999997e-05 2020-08-07T15:13:23.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:23.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535755+0000", 2020-08-07T15:13:23.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1427000000000003e-05 2020-08-07T15:13:23.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:23.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535786+0000", 2020-08-07T15:13:23.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237 2020-08-07T15:13:23.246 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.247 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:23.247 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.247 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:31696 4.0 e100/81 ondisk, result = 0)", 2020-08-07T15:13:23.247 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:13:23.248 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 274.93433606100001, 2020-08-07T15:13:23.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954, 2020-08-07T15:13:23.248 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:23.248 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:23.248 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:23.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:23.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:13:23.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:23.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:23.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:13:23.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1185000000000001e-05 2020-08-07T15:13:23.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:23.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113901+0000", 2020-08-07T15:13:23.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2930000000000002e-06 2020-08-07T15:13:23.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:23.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113905+0000", 2020-08-07T15:13:23.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42168640400000001 2020-08-07T15:13:23.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:23.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535591+0000", 2020-08-07T15:13:23.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2820000000000002e-06 2020-08-07T15:13:23.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:23.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535598+0000", 2020-08-07T15:13:23.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6578999999999999e-05 2020-08-07T15:13:23.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:23.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535684+0000", 2020-08-07T15:13:23.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8449999999999992e-06 2020-08-07T15:13:23.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:23.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535693+0000", 2020-08-07T15:13:23.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7366000000000003e-05 2020-08-07T15:13:23.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:23.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535791+0000", 2020-08-07T15:13:23.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954 2020-08-07T15:13:23.277 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.277 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:23.277 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.278 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:37024 4.0 e106/81)", 2020-08-07T15:13:23.278 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:13:23.279 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 268.819547167, 2020-08-07T15:13:23.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999, 2020-08-07T15:13:23.279 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:23.280 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:23.280 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:23.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:23.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:13:23.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:23.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:23.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:13:23.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4926000000000001e-05 2020-08-07T15:13:23.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:23.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228683+0000", 2020-08-07T15:13:23.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1280000000000002e-06 2020-08-07T15:13:23.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:23.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228686+0000", 2020-08-07T15:13:23.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69740012100000004 2020-08-07T15:13:23.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:23.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926086+0000", 2020-08-07T15:13:23.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000004e-06 2020-08-07T15:13:23.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:23.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926090+0000", 2020-08-07T15:13:23.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113526 2020-08-07T15:13:23.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:23.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926204+0000", 2020-08-07T15:13:23.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8824000000000001e-05 2020-08-07T15:13:23.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:23.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926222+0000", 2020-08-07T15:13:23.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999 2020-08-07T15:13:23.293 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.294 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:23.294 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.295 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42503 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2615, mlcod=112'2607)", 2020-08-07T15:13:23.295 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:13:23.296 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 262.75197525099998, 2020-08-07T15:13:23.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998, 2020-08-07T15:13:23.296 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:23.297 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:23.297 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:23.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:23.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:13:23.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:23.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:23.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:13:23.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270999999999998e-05 2020-08-07T15:13:23.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:23.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296292+0000", 2020-08-07T15:13:23.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4729999999999997e-06 2020-08-07T15:13:23.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:23.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296298+0000", 2020-08-07T15:13:23.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.448795049 2020-08-07T15:13:23.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:23.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745093+0000", 2020-08-07T15:13:23.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0140000000000001e-06 2020-08-07T15:13:23.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:23.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745100+0000", 2020-08-07T15:13:23.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2596999999999998e-05 2020-08-07T15:13:23.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:23.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745153+0000", 2020-08-07T15:13:23.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1662000000000003e-05 2020-08-07T15:13:23.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:23.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745184+0000", 2020-08-07T15:13:23.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093174800000000004 2020-08-07T15:13:23.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:23.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746116+0000", 2020-08-07T15:13:23.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0926999999999994e-05 2020-08-07T15:13:23.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:23.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746187+0000", 2020-08-07T15:13:23.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998 2020-08-07T15:13:23.313 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.314 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:23.314 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.315 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42504 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2616, mlcod=113'2609)", 2020-08-07T15:13:23.315 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:13:23.315 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 262.74863835799999, 2020-08-07T15:13:23.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998, 2020-08-07T15:13:23.316 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:23.316 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:23.317 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:23.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:23.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:13:23.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:23.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:23.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:13:23.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2154000000000002e-05 2020-08-07T15:13:23.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:23.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299629+0000", 2020-08-07T15:13:23.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1929999999999998e-06 2020-08-07T15:13:23.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:23.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299636+0000", 2020-08-07T15:13:23.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44766315400000001 2020-08-07T15:13:23.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:23.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747299+0000", 2020-08-07T15:13:23.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5889999999999999e-06 2020-08-07T15:13:23.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:23.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747305+0000", 2020-08-07T15:13:23.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3094999999999999e-05 2020-08-07T15:13:23.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:23.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747358+0000", 2020-08-07T15:13:23.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3283e-05 2020-08-07T15:13:23.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:23.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747392+0000", 2020-08-07T15:13:23.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015319509999999999 2020-08-07T15:13:23.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:23.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748924+0000", 2020-08-07T15:13:23.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5235999999999996e-05 2020-08-07T15:13:23.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:23.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748999+0000", 2020-08-07T15:13:23.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998 2020-08-07T15:13:23.669 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.669 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:23.670 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.671 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42520 4.8 e113/81 4:13071de5:::benchmark_data_smithi074_44586_object5314:head v 113'2624, mlcod=113'2618)", 2020-08-07T15:13:23.671 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:13:23.671 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 262.28269414900001, 2020-08-07T15:13:23.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001, 2020-08-07T15:13:23.672 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:23.672 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:23.672 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:23.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:23.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:13:23.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:23.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:23.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:13:23.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3967999999999995e-05 2020-08-07T15:13:23.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:23.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765585+0000", 2020-08-07T15:13:23.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0949999999999998e-06 2020-08-07T15:13:23.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:23.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765593+0000", 2020-08-07T15:13:23.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52703464200000005 2020-08-07T15:13:23.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:23.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292627+0000", 2020-08-07T15:13:23.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6929999999999996e-06 2020-08-07T15:13:23.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:23.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292636+0000", 2020-08-07T15:13:23.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9292000000000001e-05 2020-08-07T15:13:23.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:23.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292695+0000", 2020-08-07T15:13:23.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5395999999999999e-05 2020-08-07T15:13:23.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:23.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292721+0000", 2020-08-07T15:13:23.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00090361399999999996 2020-08-07T15:13:23.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:23.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293624+0000", 2020-08-07T15:13:23.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1761000000000002e-05 2020-08-07T15:13:23.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:23.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293666+0000", 2020-08-07T15:13:23.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001 2020-08-07T15:13:23.699 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.699 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:23.699 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.700 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:19776 5.4 5:2bb0c0d9:::benchmark_data_smithi074_53697_object2471:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e173)", 2020-08-07T15:13:23.700 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:13:23.700 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 170.64850146399999, 2020-08-07T15:13:23.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.402061844, 2020-08-07T15:13:23.701 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:23.701 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:23.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:23.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:23.708 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:23.708 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 19776 2020-08-07T15:13:23.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.708 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:23.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:23.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:13:23.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:23.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:23.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:13:23.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-05 2020-08-07T15:13:23.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:23.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399752+0000", 2020-08-07T15:13:23.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8799999999999999e-06 2020-08-07T15:13:23.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:23.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399757+0000", 2020-08-07T15:13:23.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7690000000000008e-06 2020-08-07T15:13:23.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:23.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399767+0000", 2020-08-07T15:13:23.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-06 2020-08-07T15:13:23.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:23.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399771+0000", 2020-08-07T15:13:23.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003255109 2020-08-07T15:13:23.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:23.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403026+0000", 2020-08-07T15:13:23.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101044 2020-08-07T15:13:23.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:23.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403127+0000", 2020-08-07T15:13:23.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000193158 2020-08-07T15:13:23.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,5", 2020-08-07T15:13:23.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403320+0000", 2020-08-07T15:13:23.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017185530000000001 2020-08-07T15:13:23.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:23.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405038+0000", 2020-08-07T15:13:23.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6300000000000004e-07 2020-08-07T15:13:23.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:23.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405039+0000", 2020-08-07T15:13:23.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012590600000000001 2020-08-07T15:13:23.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:23.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405165+0000", 2020-08-07T15:13:23.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109757 2020-08-07T15:13:23.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:23.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405275+0000", 2020-08-07T15:13:23.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39642174699999999 2020-08-07T15:13:23.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:23.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801697+0000", 2020-08-07T15:13:23.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2799999999999996e-07 2020-08-07T15:13:23.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:23.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801697+0000", 2020-08-07T15:13:23.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8740999999999997e-05 2020-08-07T15:13:23.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:23.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801776+0000", 2020-08-07T15:13:23.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.402061844 2020-08-07T15:13:23.724 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.724 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:23.724 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.725 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:19776 5.4 e173/164 ondisk, result = 0)", 2020-08-07T15:13:23.725 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:13:23.725 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 170.642753161, 2020-08-07T15:13:23.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39631765200000002, 2020-08-07T15:13:23.726 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:23.726 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:23.726 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:23.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:23.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:13:23.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:23.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:23.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405462+0000", 2020-08-07T15:13:23.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9332e-05 2020-08-07T15:13:23.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:23.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405492+0000", 2020-08-07T15:13:23.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8900000000000004e-06 2020-08-07T15:13:23.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:23.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405498+0000", 2020-08-07T15:13:23.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39566664200000001 2020-08-07T15:13:23.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:23.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801164+0000", 2020-08-07T15:13:23.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7440000000000001e-06 2020-08-07T15:13:23.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:23.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801168+0000", 2020-08-07T15:13:23.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050828199999999998 2020-08-07T15:13:23.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:23.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801676+0000", 2020-08-07T15:13:23.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1011e-05 2020-08-07T15:13:23.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:23.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801687+0000", 2020-08-07T15:13:23.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2750999999999993e-05 2020-08-07T15:13:23.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:23.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801780+0000", 2020-08-07T15:13:23.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39631765200000002 2020-08-07T15:13:23.739 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.739 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:23.739 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.740 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:85552 5.9 5:9d69b7cc:::benchmark_data_smithi074_53697_object10693:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e202)", 2020-08-07T15:13:23.740 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:13:23.740 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 126.24069837, 2020-08-07T15:13:23.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54429374399999997, 2020-08-07T15:13:23.741 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:23.741 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:23.741 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:23.741 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:23.741 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:23.742 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 85552 2020-08-07T15:13:23.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.742 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:23.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:23.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:13:23.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:23.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:23.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:13:23.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2691000000000001e-05 2020-08-07T15:13:23.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:23.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807570+0000", 2020-08-07T15:13:23.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0619999999999997e-06 2020-08-07T15:13:23.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:23.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807576+0000", 2020-08-07T15:13:23.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1558e-05 2020-08-07T15:13:23.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:23.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807588+0000", 2020-08-07T15:13:23.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4600000000000002e-06 2020-08-07T15:13:23.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:23.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807593+0000", 2020-08-07T15:13:23.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022126279999999999 2020-08-07T15:13:23.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:23.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.809806+0000", 2020-08-07T15:13:23.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104868 2020-08-07T15:13:23.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:23.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.809911+0000", 2020-08-07T15:13:23.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000186019 2020-08-07T15:13:23.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:13:23.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.810097+0000", 2020-08-07T15:13:23.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037966739999999999 2020-08-07T15:13:23.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:23.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813893+0000", 2020-08-07T15:13:23.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1310000000000001e-06 2020-08-07T15:13:23.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:23.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813894+0000", 2020-08-07T15:13:23.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4704999999999999e-05 2020-08-07T15:13:23.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:23.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813949+0000", 2020-08-07T15:13:23.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015487280000000001 2020-08-07T15:13:23.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:23.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.815498+0000", 2020-08-07T15:13:23.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53621442900000005 2020-08-07T15:13:23.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:23.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351712+0000", 2020-08-07T15:13:23.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8199999999999998e-07 2020-08-07T15:13:23.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:23.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351713+0000", 2020-08-07T15:13:23.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7909000000000001e-05 2020-08-07T15:13:23.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:23.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351811+0000", 2020-08-07T15:13:23.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54429374399999997 2020-08-07T15:13:23.782 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.782 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:23.782 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.787 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:85552 5.9 e202/164 ondisk, result = 0)", 2020-08-07T15:13:23.787 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:13:23.787 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 126.23447196799999, 2020-08-07T15:13:23.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53813836800000003, 2020-08-07T15:13:23.787 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:23.788 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:23.788 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:23.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:23.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:13:23.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:23.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:23.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:13:23.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8833e-05 2020-08-07T15:13:23.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813772+0000", 2020-08-07T15:13:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8459999999999999e-06 2020-08-07T15:13:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:23.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813777+0000", 2020-08-07T15:13:23.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53732224900000003 2020-08-07T15:13:23.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:23.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351100+0000", 2020-08-07T15:13:23.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1869999999999996e-06 2020-08-07T15:13:23.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:23.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351105+0000", 2020-08-07T15:13:23.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00058858100000000002 2020-08-07T15:13:23.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:23.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351693+0000", 2020-08-07T15:13:23.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4229999999999994e-06 2020-08-07T15:13:23.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:23.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351703+0000", 2020-08-07T15:13:23.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000179249 2020-08-07T15:13:23.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:23.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351882+0000", 2020-08-07T15:13:23.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53813836800000003 2020-08-07T15:13:23.822 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.822 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:23.822 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.823 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:85552 5.9 5:9d69b7cc:::benchmark_data_smithi074_53697_object10693:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e202)", 2020-08-07T15:13:23.823 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:13:23.823 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 126.22972384400001, 2020-08-07T15:13:23.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53326610600000002, 2020-08-07T15:13:23.836 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:23.836 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:13:23.837 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:23.837 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:23.837 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:23.837 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 85552 2020-08-07T15:13:23.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.838 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:23.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:23.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:13:23.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:23.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:23.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:13:23.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5001e-05 2020-08-07T15:13:23.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:23.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818517+0000", 2020-08-07T15:13:23.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5000000000000002e-06 2020-08-07T15:13:23.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:23.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818519+0000", 2020-08-07T15:13:23.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0699999999999997e-06 2020-08-07T15:13:23.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:23.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818524+0000", 2020-08-07T15:13:23.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2910000000000002e-06 2020-08-07T15:13:23.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:23.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818527+0000", 2020-08-07T15:13:23.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030912499999999999 2020-08-07T15:13:23.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:23.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818836+0000", 2020-08-07T15:13:23.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2908000000000002e-05 2020-08-07T15:13:23.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for ondisk", 2020-08-07T15:13:23.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818889+0000", 2020-08-07T15:13:23.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53286921099999995 2020-08-07T15:13:23.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:23.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351758+0000", 2020-08-07T15:13:23.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53326610600000002 2020-08-07T15:13:23.905 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.905 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:23.905 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:89464 5.9 5:94c69ba5:::benchmark_data_smithi074_53697_object11182:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:13:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 121.63188389, 2020-08-07T15:13:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67692916199999997, 2020-08-07T15:13:23.923 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:23.923 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:23.924 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:23.924 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 89464 2020-08-07T15:13:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:23.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:13:23.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:23.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:23.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:13:23.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0074000000000002e-05 2020-08-07T15:13:23.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:23.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416382+0000", 2020-08-07T15:13:23.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5000000000000001e-06 2020-08-07T15:13:23.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416386+0000", 2020-08-07T15:13:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2447e-05 2020-08-07T15:13:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:23.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416399+0000", 2020-08-07T15:13:23.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1279999999999996e-06 2020-08-07T15:13:23.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:23.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416405+0000", 2020-08-07T15:13:23.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016641805999999999 2020-08-07T15:13:23.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:23.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433047+0000", 2020-08-07T15:13:23.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107995 2020-08-07T15:13:23.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:23.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433155+0000", 2020-08-07T15:13:23.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018818599999999999 2020-08-07T15:13:23.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:13:23.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433343+0000", 2020-08-07T15:13:23.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039911679999999998 2020-08-07T15:13:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437334+0000", 2020-08-07T15:13:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2020-08-07T15:13:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437335+0000", 2020-08-07T15:13:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6351000000000002e-05 2020-08-07T15:13:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437401+0000", 2020-08-07T15:13:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001267949 2020-08-07T15:13:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438669+0000", 2020-08-07T15:13:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65454035600000005 2020-08-07T15:13:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093209+0000", 2020-08-07T15:13:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7859999999999999e-06 2020-08-07T15:13:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093211+0000", 2020-08-07T15:13:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9665e-05 2020-08-07T15:13:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:23.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093261+0000", 2020-08-07T15:13:23.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67692916199999997 2020-08-07T15:13:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:89464 5.9 e207/164 ondisk, result = 0)", 2020-08-07T15:13:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:13:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 121.609254128, 2020-08-07T15:13:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65430647200000003, 2020-08-07T15:13:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:13:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:23.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:23.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:13:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6529000000000002e-05 2020-08-07T15:13:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438998+0000", 2020-08-07T15:13:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.567e-06 2020-08-07T15:13:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.439005+0000", 2020-08-07T15:13:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65408475099999996 2020-08-07T15:13:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093089+0000", 2020-08-07T15:13:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6880000000000002e-06 2020-08-07T15:13:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093096+0000", 2020-08-07T15:13:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4980000000000002e-05 2020-08-07T15:13:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093191+0000", 2020-08-07T15:13:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3090000000000001e-06 2020-08-07T15:13:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093200+0000", 2020-08-07T15:13:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7648e-05 2020-08-07T15:13:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093268+0000", 2020-08-07T15:13:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65430647200000003 2020-08-07T15:13:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:114080 5.4 5:24adcd54:::benchmark_data_smithi074_53697_object14259:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e215)", 2020-08-07T15:13:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:13:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 103.448546671, 2020-08-07T15:13:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.57290862300000001, 2020-08-07T15:13:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 114080 2020-08-07T15:13:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:13:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:13:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9238e-05 2020-08-07T15:13:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599698+0000", 2020-08-07T15:13:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6260000000000003e-06 2020-08-07T15:13:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599703+0000", 2020-08-07T15:13:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5577999999999999e-05 2020-08-07T15:13:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599718+0000", 2020-08-07T15:13:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9869999999999999e-06 2020-08-07T15:13:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599726+0000", 2020-08-07T15:13:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00077929599999999996 2020-08-07T15:13:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600506+0000", 2020-08-07T15:13:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1717000000000003e-05 2020-08-07T15:13:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:24.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600597+0000", 2020-08-07T15:13:24.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011828999999999999 2020-08-07T15:13:24.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,5", 2020-08-07T15:13:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600716+0000", 2020-08-07T15:13:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020861479999999999 2020-08-07T15:13:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602802+0000", 2020-08-07T15:13:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4600000000000005e-07 2020-08-07T15:13:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602802+0000", 2020-08-07T15:13:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9165e-05 2020-08-07T15:13:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:24.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602862+0000", 2020-08-07T15:13:24.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0093485080000000002 2020-08-07T15:13:24.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:24.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.612210+0000", 2020-08-07T15:13:24.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.56030197699999995 2020-08-07T15:13:24.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:24.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172512+0000", 2020-08-07T15:13:24.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8800000000000001e-07 2020-08-07T15:13:24.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:24.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172513+0000", 2020-08-07T15:13:24.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4659000000000006e-05 2020-08-07T15:13:24.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:24.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172578+0000", 2020-08-07T15:13:24.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.57290862300000001 2020-08-07T15:13:24.024 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:24.025 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:24.025 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:24.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.025 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:48616 6.9 6:955383e0:::benchmark_data_smithi074_63843_object6076:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e256)", 2020-08-07T15:13:24.026 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:43.702845+0000", 2020-08-07T15:13:24.026 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.345370373999998, 2020-08-07T15:13:24.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40912302499999997, 2020-08-07T15:13:24.026 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:24.027 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:24.027 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:24.027 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:13:24.027 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:24.027 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48616 2020-08-07T15:13:24.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.028 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:24.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:24.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702845+0000", 2020-08-07T15:13:24.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:24.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:24.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702845+0000", 2020-08-07T15:13:24.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1931999999999997e-05 2020-08-07T15:13:24.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:24.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702897+0000", 2020-08-07T15:13:24.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0650000000000004e-06 2020-08-07T15:13:24.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:24.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702903+0000", 2020-08-07T15:13:24.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3084999999999999e-05 2020-08-07T15:13:24.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:24.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702916+0000", 2020-08-07T15:13:24.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4179999999999996e-06 2020-08-07T15:13:24.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:24.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702922+0000", 2020-08-07T15:13:24.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045509080000000002 2020-08-07T15:13:24.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:24.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.707473+0000", 2020-08-07T15:13:24.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2619000000000005e-05 2020-08-07T15:13:24.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:24.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.707535+0000", 2020-08-07T15:13:24.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000126919 2020-08-07T15:13:24.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2", 2020-08-07T15:13:24.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.707662+0000", 2020-08-07T15:13:24.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001200135 2020-08-07T15:13:24.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:24.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.708862+0000", 2020-08-07T15:13:24.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0109999999999999e-06 2020-08-07T15:13:24.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:24.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.708863+0000", 2020-08-07T15:13:24.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8837999999999994e-05 2020-08-07T15:13:24.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:24.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.708932+0000", 2020-08-07T15:13:24.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054206710000000002 2020-08-07T15:13:24.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:24.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.714353+0000", 2020-08-07T15:13:24.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39754767099999999 2020-08-07T15:13:24.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:24.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111901+0000", 2020-08-07T15:13:24.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.023e-06 2020-08-07T15:13:24.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:24.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111902+0000", 2020-08-07T15:13:24.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6730000000000007e-05 2020-08-07T15:13:24.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:24.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111968+0000", 2020-08-07T15:13:24.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40912302499999997 2020-08-07T15:13:24.045 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:24.045 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:24.046 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:24.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.046 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5481.0:48616 6.9 e256/230 ondisk, result = 0)", 2020-08-07T15:13:24.047 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:43.714790+0000", 2020-08-07T15:13:24.047 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.333425808999998, 2020-08-07T15:13:24.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39718457899999998, 2020-08-07T15:13:24.047 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:24.048 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:24.048 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:24.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:24.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.714790+0000", 2020-08-07T15:13:24.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:24.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:24.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.714790+0000", 2020-08-07T15:13:24.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8272000000000001e-05 2020-08-07T15:13:24.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:24.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.714808+0000", 2020-08-07T15:13:24.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1599999999999998e-06 2020-08-07T15:13:24.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:24.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.714811+0000", 2020-08-07T15:13:24.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39639543300000002 2020-08-07T15:13:24.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:24.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111207+0000", 2020-08-07T15:13:24.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7680000000000002e-06 2020-08-07T15:13:24.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:24.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111210+0000", 2020-08-07T15:13:24.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067142299999999999 2020-08-07T15:13:24.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:24.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111882+0000", 2020-08-07T15:13:24.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6849999999999996e-06 2020-08-07T15:13:24.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:24.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111892+0000", 2020-08-07T15:13:24.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2837999999999995e-05 2020-08-07T15:13:24.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:24.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:24.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:24.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111974+0000", 2020-08-07T15:13:24.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39718457899999998 2020-08-07T15:13:24.057 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:24.057 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:24.058 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:24.058 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:24.059 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:24.059 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:24.064 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:13:24.064 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:13:24.065 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:13:24.065 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:13:24.065 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:13:24.066 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:13:24.066 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:13:24.067 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:13:24.067 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 71 16 105744 105728 11.6324 12.4922 0.0076183 0.0107421 2020-08-07T15:13:24.067 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:13:24.068 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:13:24.068 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:13:24.069 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:13:24.069 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:13:24.069 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:13:24.070 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:13:24.070 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:13:24.071 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:13:24.072 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:24.059+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:24.161 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:24.160+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:24.247 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:24.248 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:24.265 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:13:24.266 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:24.263+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:24.278 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 72 16 107244 107228 11.6336 11.7188 0.00448086 0.0107389 2020-08-07T15:13:24.316 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:13:24.382 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:24.375+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:24.460 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:24.460 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:24.468 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:24.467+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:24.471 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:13:24.569 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:24.568+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:24.670 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:24.669+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:24.676 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:24.677 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:24.687 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:13:24.772 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:24.770+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:24.874 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:24.871+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:24.914 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:24.931 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:24.933 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:13:24.975 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:24.974+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:25.082 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:25.073+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:25.161 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:25.161 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:25.176 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:13:25.186 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:25.177+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:25.273 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 73 16 108628 108612 11.6223 10.8125 0.0057438 0.0107511 2020-08-07T15:13:25.281 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:25.279+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:25.375 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:25.376 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:25.386 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:13:25.393 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:25.389+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:25.539 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:25.528+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:25.609 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2020-08-07T15:13:25.609 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:13:25.610 INFO:tasks.thrashosds.thrasher:inject_pause on 6 2020-08-07T15:13:25.610 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2020-08-07T15:13:25.610 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2020-08-07T15:13:25.611 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2020-08-07T15:13:25.612 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:25.594+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:25.659 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:25.660 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:25.673 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:13:25.726 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:25.721+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:25.788 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:25.788 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "" 2020-08-07T15:13:25.788 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:25.805 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:25.802+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:25.905 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:25.904+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:25.918 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:25.918 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:25.933 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:13:26.034 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:26.031+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:26.074 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:13:26.124 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:26.122+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:26.168 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:26.169 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:26.186 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:13:26.214 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:26.208+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:26.278 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 74 16 110147 110131 11.6256 11.8672 0.105735 0.0107429 2020-08-07T15:13:26.292 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:13:26.316 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:26.313+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:26.336 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:13:26.338 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:13:26.374 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:13:26.399 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:26.399 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:26.422 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:13:26.423 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:13:26.423 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:26.420+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:26.468 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:13:26.514 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:13:26.522 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:26.521+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:26.580 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:13:26.623 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:26.621+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:26.636 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:13:26.642 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:13:26.643 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:13:26.643 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:13:26.643 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:13:26.644 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:13:26.644 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:13:26.644 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:13:26.647 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:13:26.731 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:26.725+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:26.844 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:26.845 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:13:26.847 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:13:26.848 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:13:26.849 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:26.849 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:26.844+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:26.863 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:13:26.954 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:26.952+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:27.057 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:27.053+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:27.170 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:27.155+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:27.212 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:27.212 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:13:27.212 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:13:27.212 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:13:27.213 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.213 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106240 3.7 3:ebdd8242:::benchmark_data_smithi074_35447_object13279:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:13:27.213 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:13:27.213 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 339.84879391300001, 2020-08-07T15:13:27.225 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.90079396, 2020-08-07T15:13:27.225 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:27.225 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:27.225 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:27.226 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:13:27.226 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:13:27.226 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106240 2020-08-07T15:13:27.226 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.226 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:27.227 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.227 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:27.227 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:13:27.227 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:27.227 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.228 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.228 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:27.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:13:27.228 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2522000000000003e-05 2020-08-07T15:13:27.228 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.229 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.251 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:27.252 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355118+0000", 2020-08-07T15:13:27.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6159999999999998e-06 2020-08-07T15:13:27.252 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.252 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.253 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:27.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355125+0000", 2020-08-07T15:13:27.253 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2699e-05 2020-08-07T15:13:27.253 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.254 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:27.254 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355138+0000", 2020-08-07T15:13:27.254 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2310000000000001e-06 2020-08-07T15:13:27.254 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.254 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.255 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:27.255 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355144+0000", 2020-08-07T15:13:27.255 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9855000000000003e-05 2020-08-07T15:13:27.302 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.302 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.303 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:27.303 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355194+0000", 2020-08-07T15:13:27.303 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112713 2020-08-07T15:13:27.303 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.303 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.304 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:27.304 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355306+0000", 2020-08-07T15:13:27.304 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019794600000000001 2020-08-07T15:13:27.304 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.304 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.305 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:13:27.305 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355504+0000", 2020-08-07T15:13:27.305 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0048346029999999998 2020-08-07T15:13:27.305 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.305 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.322 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:27.322 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.360339+0000", 2020-08-07T15:13:27.322 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00033646400000000002 2020-08-07T15:13:27.323 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.323 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.323 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:27.323 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.360675+0000", 2020-08-07T15:13:27.323 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89508481200000001 2020-08-07T15:13:27.324 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.324 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.324 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:27.332 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255760+0000", 2020-08-07T15:13:27.332 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5659999999999999e-06 2020-08-07T15:13:27.332 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.333 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.333 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:27.333 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255762+0000", 2020-08-07T15:13:27.333 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6595000000000003e-05 2020-08-07T15:13:27.333 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.334 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.334 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:27.335 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255818+0000", 2020-08-07T15:13:27.335 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1338000000000003e-05 2020-08-07T15:13:27.335 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.335 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:27.336 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255860+0000", 2020-08-07T15:13:27.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.90079396 2020-08-07T15:13:27.336 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:27.336 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:27.336 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:27.337 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.337 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.337 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4329.0:106240 3.7 e59/20 ondisk, result = 0)", 2020-08-07T15:13:27.337 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:13:27.337 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 339.75239796099999, 2020-08-07T15:13:27.342 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80440215100000001, 2020-08-07T15:13:27.343 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:27.343 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:27.343 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:27.343 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.343 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:27.344 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:13:27.344 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:27.344 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.344 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.344 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:27.345 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451462+0000", 2020-08-07T15:13:27.345 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0052999999999999e-05 2020-08-07T15:13:27.345 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.345 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.346 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:27.346 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451482+0000", 2020-08-07T15:13:27.347 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3229999999999999e-06 2020-08-07T15:13:27.347 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.372 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:27.373 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.451486+0000", 2020-08-07T15:13:27.373 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80420304799999998 2020-08-07T15:13:27.374 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.374 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.375 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:27.375 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255689+0000", 2020-08-07T15:13:27.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7759999999999997e-06 2020-08-07T15:13:27.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:27.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255694+0000", 2020-08-07T15:13:27.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5828999999999998e-05 2020-08-07T15:13:27.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.382 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.383 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:27.383 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255740+0000", 2020-08-07T15:13:27.383 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1466e-05 2020-08-07T15:13:27.384 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.385 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.388 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:27.388 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255751+0000", 2020-08-07T15:13:27.388 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112656 2020-08-07T15:13:27.389 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.389 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.390 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:27.390 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255864+0000", 2020-08-07T15:13:27.391 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80440215100000001 2020-08-07T15:13:27.391 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:27.392 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:27.393 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:27.393 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.395 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.395 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37057 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:13:27.396 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:13:27.396 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 272.96028925100001, 2020-08-07T15:13:27.396 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007248691, 2020-08-07T15:13:27.397 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:27.397 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:13:27.398 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:27.398 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:27.399 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:27.399 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37057 2020-08-07T15:13:27.399 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.400 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:27.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.402 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:27.403 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:13:27.414 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:27.420 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.421 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:27.425 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:13:27.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3661e-05 2020-08-07T15:13:27.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.426 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.426 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:27.426 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243624+0000", 2020-08-07T15:13:27.426 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7519999999999996e-06 2020-08-07T15:13:27.426 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.427 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:27.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243632+0000", 2020-08-07T15:13:27.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6392000000000002e-05 2020-08-07T15:13:27.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:27.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243648+0000", 2020-08-07T15:13:27.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6659999999999997e-06 2020-08-07T15:13:27.432 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.434 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.434 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:27.434 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243657+0000", 2020-08-07T15:13:27.434 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0070484609999999998 2020-08-07T15:13:27.436 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.437 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.437 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:27.437 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250705+0000", 2020-08-07T15:13:27.437 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0083e-05 2020-08-07T15:13:27.437 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.438 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.438 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:27.438 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250715+0000", 2020-08-07T15:13:27.438 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000086695 2020-08-07T15:13:27.439 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.439 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.439 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:27.439 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.250802+0000", 2020-08-07T15:13:27.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6980999999999999e-05 2020-08-07T15:13:27.440 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.440 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:27.441 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.250819+0000", 2020-08-07T15:13:27.442 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007248691 2020-08-07T15:13:27.442 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:27.442 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:27.442 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:27.443 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.443 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.443 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37058 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:13:27.443 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:13:27.443 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 272.960147517, 2020-08-07T15:13:27.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007368357, 2020-08-07T15:13:27.447 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:27.448 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:13:27.449 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:27.449 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:27.449 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:27.449 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37058 2020-08-07T15:13:27.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.450 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:27.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.450 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:27.450 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:13:27.450 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:27.451 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.451 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.451 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:27.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:13:27.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8838999999999998e-05 2020-08-07T15:13:27.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:27.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243761+0000", 2020-08-07T15:13:27.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2079999999999999e-06 2020-08-07T15:13:27.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.453 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:27.453 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243766+0000", 2020-08-07T15:13:27.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0567999999999999e-05 2020-08-07T15:13:27.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:27.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243777+0000", 2020-08-07T15:13:27.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7210000000000002e-06 2020-08-07T15:13:27.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:27.456 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243782+0000", 2020-08-07T15:13:27.456 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069508749999999996 2020-08-07T15:13:27.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:27.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250732+0000", 2020-08-07T15:13:27.458 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7899999999999999e-06 2020-08-07T15:13:27.458 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:27.459 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250737+0000", 2020-08-07T15:13:27.459 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003272409999999 2020-08-07T15:13:27.459 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.460 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.460 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:27.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251064+0000", 2020-08-07T15:13:27.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6115000000000001e-05 2020-08-07T15:13:27.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.461 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.461 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:27.461 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251081+0000", 2020-08-07T15:13:27.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007368357 2020-08-07T15:13:27.461 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:27.462 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:27.462 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:27.462 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.462 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.462 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37059 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:13:27.463 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:13:27.463 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 272.95990446799999, 2020-08-07T15:13:27.463 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0071655319999999, 2020-08-07T15:13:27.463 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:27.464 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:13:27.464 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:27.464 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:27.465 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:27.465 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37059 2020-08-07T15:13:27.466 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.466 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:27.466 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.466 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:27.466 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:13:27.467 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:27.467 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.467 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.467 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:27.467 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:13:27.468 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6533000000000002e-05 2020-08-07T15:13:27.468 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.468 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.468 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:27.468 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244012+0000", 2020-08-07T15:13:27.469 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3860000000000001e-06 2020-08-07T15:13:27.469 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.469 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.469 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:27.469 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244019+0000", 2020-08-07T15:13:27.471 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6982000000000001e-05 2020-08-07T15:13:27.472 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.472 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.472 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:27.472 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244036+0000", 2020-08-07T15:13:27.473 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0310000000000001e-06 2020-08-07T15:13:27.473 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.473 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.473 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:27.473 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244043+0000", 2020-08-07T15:13:27.474 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0067169259999999998 2020-08-07T15:13:27.474 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.474 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.474 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:27.474 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250760+0000", 2020-08-07T15:13:27.475 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4499999999999998e-06 2020-08-07T15:13:27.475 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.475 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.475 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:27.475 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250768+0000", 2020-08-07T15:13:27.476 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000341691 2020-08-07T15:13:27.477 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.477 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.477 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:27.477 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251109+0000", 2020-08-07T15:13:27.478 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1533e-05 2020-08-07T15:13:27.478 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.478 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.478 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:27.478 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251121+0000", 2020-08-07T15:13:27.479 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0071655319999999 2020-08-07T15:13:27.479 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:27.479 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:27.479 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:27.479 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.490 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37060 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:13:27.491 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:13:27.491 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 272.95978053099998, 2020-08-07T15:13:27.491 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070762280000001, 2020-08-07T15:13:27.491 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:27.492 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:13:27.492 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:27.492 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:27.492 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:27.492 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37060 2020-08-07T15:13:27.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.493 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:27.493 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:27.502 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:13:27.503 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:27.503 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.503 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.503 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:27.504 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:13:27.504 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5739000000000003e-05 2020-08-07T15:13:27.505 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.505 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.505 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:27.505 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244135+0000", 2020-08-07T15:13:27.506 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7740000000000001e-06 2020-08-07T15:13:27.506 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.506 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.506 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:27.508 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244141+0000", 2020-08-07T15:13:27.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3817e-05 2020-08-07T15:13:27.510 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.510 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:27.510 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244155+0000", 2020-08-07T15:13:27.510 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3829999999999997e-06 2020-08-07T15:13:27.511 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.512 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:27.512 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244161+0000", 2020-08-07T15:13:27.512 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0066303669999999999 2020-08-07T15:13:27.513 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.513 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.513 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:27.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250791+0000", 2020-08-07T15:13:27.514 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.392e-06 2020-08-07T15:13:27.514 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.514 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:27.514 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250798+0000", 2020-08-07T15:13:27.515 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003485860000001 2020-08-07T15:13:27.515 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.515 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.515 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:27.515 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251146+0000", 2020-08-07T15:13:27.515 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1700000000000003e-06 2020-08-07T15:13:27.516 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.516 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.516 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:27.516 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251155+0000", 2020-08-07T15:13:27.517 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070762280000001 2020-08-07T15:13:27.517 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:27.517 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:27.517 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:27.532 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.532 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.533 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37061 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:13:27.533 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:13:27.533 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 272.95966590400002, 2020-08-07T15:13:27.533 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070006920000001, 2020-08-07T15:13:27.534 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:27.534 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:13:27.534 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:27.534 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:27.534 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:27.535 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37061 2020-08-07T15:13:27.542 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.543 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:27.543 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.543 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:27.543 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:13:27.543 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:27.544 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.544 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.544 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:27.544 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:13:27.544 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5453e-05 2020-08-07T15:13:27.545 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.545 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:27.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244239+0000", 2020-08-07T15:13:27.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3179999999999997e-06 2020-08-07T15:13:27.550 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.551 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.551 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:27.551 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244245+0000", 2020-08-07T15:13:27.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2138e-05 2020-08-07T15:13:27.551 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.552 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:27.552 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244257+0000", 2020-08-07T15:13:27.552 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.69e-06 2020-08-07T15:13:27.626 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.626 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.626 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:27.627 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244261+0000", 2020-08-07T15:13:27.627 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0065598940000000001 2020-08-07T15:13:27.627 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.627 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:27.628 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250821+0000", 2020-08-07T15:13:27.628 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4799999999999998e-06 2020-08-07T15:13:27.628 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.640 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.640 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:27.641 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250828+0000", 2020-08-07T15:13:27.641 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003567310000001 2020-08-07T15:13:27.641 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.641 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.641 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:27.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251185+0000", 2020-08-07T15:13:27.642 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9879999999999999e-06 2020-08-07T15:13:27.642 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.642 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.642 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:27.643 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251195+0000", 2020-08-07T15:13:27.646 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070006920000001 2020-08-07T15:13:27.646 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:27.647 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:27.656 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:27.657 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.657 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.657 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37062 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:13:27.659 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:13:27.659 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 272.95956568600002, 2020-08-07T15:13:27.673 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006933662, 2020-08-07T15:13:27.674 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:27.674 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:13:27.674 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:27.674 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:27.674 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:27.675 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37062 2020-08-07T15:13:27.675 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.676 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:27.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:27.677 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:13:27.677 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:27.677 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.678 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.678 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:27.678 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:13:27.678 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4041e-05 2020-08-07T15:13:27.678 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:27.682 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244348+0000", 2020-08-07T15:13:27.682 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2079999999999997e-06 2020-08-07T15:13:27.682 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.683 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.683 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:27.683 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244354+0000", 2020-08-07T15:13:27.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1483e-05 2020-08-07T15:13:27.683 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.684 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.684 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:27.684 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244366+0000", 2020-08-07T15:13:27.684 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7989999999999999e-06 2020-08-07T15:13:27.684 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.685 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.685 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:27.685 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244372+0000", 2020-08-07T15:13:27.685 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0064783699999999998 2020-08-07T15:13:27.685 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.686 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.686 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:27.686 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250850+0000", 2020-08-07T15:13:27.686 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4729999999999997e-06 2020-08-07T15:13:27.686 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.687 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.687 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:27.687 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250856+0000", 2020-08-07T15:13:27.687 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003630240000001 2020-08-07T15:13:27.687 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.688 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.688 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:27.688 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251219+0000", 2020-08-07T15:13:27.688 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2640000000000008e-06 2020-08-07T15:13:27.688 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.689 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.689 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:27.689 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251228+0000", 2020-08-07T15:13:27.689 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006933662 2020-08-07T15:13:27.689 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:27.695 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:27.709 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:27.711 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.712 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.712 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37063 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:13:27.712 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:13:27.712 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 272.95945335200003, 2020-08-07T15:13:27.713 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006858051, 2020-08-07T15:13:27.713 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:27.713 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:13:27.713 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:27.714 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:27.716 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:27.716 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37063 2020-08-07T15:13:27.716 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.716 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:27.716 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.717 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:27.717 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:13:27.717 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:27.717 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.717 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.718 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:27.718 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:13:27.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5043000000000001e-05 2020-08-07T15:13:27.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:27.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244461+0000", 2020-08-07T15:13:27.724 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4600000000000002e-06 2020-08-07T15:13:27.724 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.724 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.724 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:27.725 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244467+0000", 2020-08-07T15:13:27.725 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1435e-05 2020-08-07T15:13:27.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.725 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:27.726 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244478+0000", 2020-08-07T15:13:27.726 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0190000000000003e-06 2020-08-07T15:13:27.726 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.726 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.726 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:27.727 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244484+0000", 2020-08-07T15:13:27.727 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006392895 2020-08-07T15:13:27.727 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.727 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.727 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:27.728 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250877+0000", 2020-08-07T15:13:27.728 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4280000000000001e-06 2020-08-07T15:13:27.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:27.729 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250884+0000", 2020-08-07T15:13:27.729 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000370414 2020-08-07T15:13:27.729 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.729 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.731 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:27.731 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251254+0000", 2020-08-07T15:13:27.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0356999999999999e-05 2020-08-07T15:13:27.732 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.732 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:27.748 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251264+0000", 2020-08-07T15:13:27.748 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006858051 2020-08-07T15:13:27.748 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:27.749 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:27.749 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:27.749 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.749 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.749 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37064 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:13:27.750 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:13:27.750 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 272.95934121900001, 2020-08-07T15:13:27.750 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0067798189999999, 2020-08-07T15:13:27.750 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:27.751 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:13:27.751 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:27.751 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:27.751 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:27.751 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37064 2020-08-07T15:13:27.751 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.752 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:27.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.752 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:27.752 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:13:27.752 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:27.753 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.753 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.753 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:27.753 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:13:27.754 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4007e-05 2020-08-07T15:13:27.754 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.754 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:27.760 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244573+0000", 2020-08-07T15:13:27.760 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.831e-06 2020-08-07T15:13:27.761 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.761 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:27.761 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244577+0000", 2020-08-07T15:13:27.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2269000000000001e-05 2020-08-07T15:13:27.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.762 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.762 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:27.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244590+0000", 2020-08-07T15:13:27.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7149999999999997e-06 2020-08-07T15:13:27.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:27.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244594+0000", 2020-08-07T15:13:27.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063105670000000004 2020-08-07T15:13:27.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:27.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250905+0000", 2020-08-07T15:13:27.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1970000000000001e-06 2020-08-07T15:13:27.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:27.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250911+0000", 2020-08-07T15:13:27.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003783020000001 2020-08-07T15:13:27.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:27.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251289+0000", 2020-08-07T15:13:27.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9309999999999997e-06 2020-08-07T15:13:27.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.808 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:27.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251298+0000", 2020-08-07T15:13:27.809 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0067798189999999 2020-08-07T15:13:27.809 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:27.809 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:27.812 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:27.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.813 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37065 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:13:27.815 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:13:27.816 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 272.95923099999999, 2020-08-07T15:13:27.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006708545, 2020-08-07T15:13:27.816 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:27.816 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:13:27.817 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:27.817 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:27.817 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:27.817 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37065 2020-08-07T15:13:27.818 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.818 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:27.818 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.818 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:27.818 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:13:27.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:27.819 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:27.819 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:13:27.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2750000000000001e-05 2020-08-07T15:13:27.821 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:27.821 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244681+0000", 2020-08-07T15:13:27.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0950000000000003e-06 2020-08-07T15:13:27.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.822 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.822 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:27.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244687+0000", 2020-08-07T15:13:27.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2485e-05 2020-08-07T15:13:27.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:27.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244699+0000", 2020-08-07T15:13:27.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8800000000000002e-06 2020-08-07T15:13:27.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:27.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244706+0000", 2020-08-07T15:13:27.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0062281569999999998 2020-08-07T15:13:27.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:27.826 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250934+0000", 2020-08-07T15:13:27.826 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1719999999999998e-06 2020-08-07T15:13:27.851 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.853 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:27.853 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250939+0000", 2020-08-07T15:13:27.853 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003861810000001 2020-08-07T15:13:27.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.854 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.854 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:27.854 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251325+0000", 2020-08-07T15:13:27.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1824999999999999e-05 2020-08-07T15:13:27.854 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.855 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:27.855 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251337+0000", 2020-08-07T15:13:27.855 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006708545 2020-08-07T15:13:27.855 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:27.856 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:27.856 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:27.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.856 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.856 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37066 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:13:27.857 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:13:27.857 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 272.95456202999998, 2020-08-07T15:13:27.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002077205, 2020-08-07T15:13:27.858 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:27.858 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:13:27.858 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:27.858 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:27.859 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:27.859 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37066 2020-08-07T15:13:27.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.859 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:27.859 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:27.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:13:27.860 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:27.860 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.861 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:27.861 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:13:27.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4654000000000002e-05 2020-08-07T15:13:27.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:27.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249352+0000", 2020-08-07T15:13:27.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6210000000000004e-06 2020-08-07T15:13:27.873 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.874 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.874 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:27.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249360+0000", 2020-08-07T15:13:27.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1895999999999998e-05 2020-08-07T15:13:27.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:27.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249382+0000", 2020-08-07T15:13:27.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1479999999999995e-06 2020-08-07T15:13:27.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:27.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249391+0000", 2020-08-07T15:13:27.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015731510000000001 2020-08-07T15:13:27.925 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.926 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:27.926 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250964+0000", 2020-08-07T15:13:27.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5400000000000001e-06 2020-08-07T15:13:27.926 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:27.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250971+0000", 2020-08-07T15:13:27.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000392379 2020-08-07T15:13:27.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:27.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251363+0000", 2020-08-07T15:13:27.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1816e-05 2020-08-07T15:13:27.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:27.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251375+0000", 2020-08-07T15:13:27.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002077205 2020-08-07T15:13:27.929 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:27.953 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:27.953 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:27.954 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.959 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37067 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:13:27.959 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:13:27.959 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 272.954421336, 2020-08-07T15:13:27.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0019761119999999, 2020-08-07T15:13:27.960 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:27.960 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:13:27.960 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:27.960 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:27.960 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:27.960 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37067 2020-08-07T15:13:27.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.964 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:27.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:27.970 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:13:27.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:27.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:27.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:13:27.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5961999999999997e-05 2020-08-07T15:13:27.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:27.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249494+0000", 2020-08-07T15:13:27.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3210000000000004e-06 2020-08-07T15:13:27.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:27.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249500+0000", 2020-08-07T15:13:27.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1012000000000001e-05 2020-08-07T15:13:27.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:27.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249511+0000", 2020-08-07T15:13:27.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.809e-06 2020-08-07T15:13:27.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:27.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249515+0000", 2020-08-07T15:13:27.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0014777670000000001 2020-08-07T15:13:27.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.990 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.990 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:27.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250993+0000", 2020-08-07T15:13:27.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9499999999999998e-06 2020-08-07T15:13:27.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:27.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250999+0000", 2020-08-07T15:13:27.991 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0004032519999999 2020-08-07T15:13:27.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:27.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251402+0000", 2020-08-07T15:13:27.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2038999999999999e-05 2020-08-07T15:13:27.993 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.993 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.993 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:27.993 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251414+0000", 2020-08-07T15:13:27.995 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0019761119999999 2020-08-07T15:13:27.995 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:27.995 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:27.996 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:27.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.996 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.996 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37068 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:13:27.996 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:13:27.997 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 272.95217676499999, 2020-08-07T15:13:27.997 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99977072, 2020-08-07T15:13:27.997 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:27.997 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:13:27.998 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:27.998 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:27.998 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:27.998 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37068 2020-08-07T15:13:27.998 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:27.999 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:27.999 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:27.999 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:28.002 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:13:28.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:28.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.011 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:28.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:13:28.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1369000000000003e-05 2020-08-07T15:13:28.012 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.012 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:28.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251724+0000", 2020-08-07T15:13:28.014 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.19e-06 2020-08-07T15:13:28.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.015 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.015 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:28.015 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251731+0000", 2020-08-07T15:13:28.015 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1368e-05 2020-08-07T15:13:28.016 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.016 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:28.016 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251742+0000", 2020-08-07T15:13:28.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.682e-06 2020-08-07T15:13:28.017 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:28.017 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251748+0000", 2020-08-07T15:13:28.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5275000000000003e-05 2020-08-07T15:13:28.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:28.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251793+0000", 2020-08-07T15:13:28.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5000000000000005e-06 2020-08-07T15:13:28.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:28.033 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251802+0000", 2020-08-07T15:13:28.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99963953500000002 2020-08-07T15:13:28.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.034 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.034 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:28.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251442+0000", 2020-08-07T15:13:28.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1800999999999999e-05 2020-08-07T15:13:28.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:28.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251454+0000", 2020-08-07T15:13:28.046 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99977072 2020-08-07T15:13:28.047 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:28.047 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:28.047 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:28.047 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.048 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37069 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:13:28.048 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:13:28.048 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 272.95107296700002, 2020-08-07T15:13:28.048 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99870607899999997, 2020-08-07T15:13:28.048 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:28.049 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:13:28.049 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:28.049 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:28.049 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:28.049 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37069 2020-08-07T15:13:28.070 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.070 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:28.070 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.070 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:28.071 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:13:28.076 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:28.076 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.076 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.076 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:28.076 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:13:28.077 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3755999999999999e-05 2020-08-07T15:13:28.077 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.077 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.077 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:28.077 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252831+0000", 2020-08-07T15:13:28.078 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7919999999999998e-06 2020-08-07T15:13:28.078 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.078 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.078 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:28.078 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252836+0000", 2020-08-07T15:13:28.079 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1409999999999999e-05 2020-08-07T15:13:28.079 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.079 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.086 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:28.086 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252848+0000", 2020-08-07T15:13:28.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6380000000000001e-06 2020-08-07T15:13:28.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:28.087 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252853+0000", 2020-08-07T15:13:28.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3244e-05 2020-08-07T15:13:28.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.089 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:28.090 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252907+0000", 2020-08-07T15:13:28.090 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1470000000000007e-06 2020-08-07T15:13:28.090 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.090 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.091 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:28.091 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252916+0000", 2020-08-07T15:13:28.091 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99856497499999997 2020-08-07T15:13:28.091 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.091 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.092 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:28.092 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251481+0000", 2020-08-07T15:13:28.092 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2116999999999999e-05 2020-08-07T15:13:28.092 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.092 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.093 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:28.093 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251493+0000", 2020-08-07T15:13:28.093 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99870607899999997 2020-08-07T15:13:28.093 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:28.093 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:28.095 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:28.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.096 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.097 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37070 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:13:28.097 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:13:28.097 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 272.95096870399999, 2020-08-07T15:13:28.098 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99863946699999995, 2020-08-07T15:13:28.098 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:28.098 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:13:28.098 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:28.099 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:28.099 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:28.099 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37070 2020-08-07T15:13:28.100 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.100 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:28.100 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.101 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:28.101 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:13:28.101 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:28.101 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.102 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.102 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:28.102 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:13:28.103 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9984000000000001e-05 2020-08-07T15:13:28.103 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.103 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.119 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:28.119 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252941+0000", 2020-08-07T15:13:28.122 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7010000000000004e-06 2020-08-07T15:13:28.122 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.122 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.122 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:28.123 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252946+0000", 2020-08-07T15:13:28.123 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1144999999999999e-05 2020-08-07T15:13:28.123 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.123 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.124 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:28.124 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252957+0000", 2020-08-07T15:13:28.124 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6920000000000001e-06 2020-08-07T15:13:28.124 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.125 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.125 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:28.125 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252962+0000", 2020-08-07T15:13:28.125 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6739000000000002e-05 2020-08-07T15:13:28.125 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.137 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.137 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:28.137 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252998+0000", 2020-08-07T15:13:28.137 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.503e-06 2020-08-07T15:13:28.137 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.138 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.139 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:28.139 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.253006+0000", 2020-08-07T15:13:28.139 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99851375499999995 2020-08-07T15:13:28.139 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.151 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:28.151 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251520+0000", 2020-08-07T15:13:28.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0947999999999999e-05 2020-08-07T15:13:28.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.153 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.153 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:28.153 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251530+0000", 2020-08-07T15:13:28.153 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99863946699999995 2020-08-07T15:13:28.153 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:28.154 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:28.154 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:28.154 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.154 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.154 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42505 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:13:28.155 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:13:28.155 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 266.901628174, 2020-08-07T15:13:28.155 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0024736009999999, 2020-08-07T15:13:28.155 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:28.155 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:28.156 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:28.156 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:28.156 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:28.156 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42505 2020-08-07T15:13:28.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.174 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:28.175 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.175 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:28.175 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:13:28.176 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:28.176 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.176 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.177 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:28.177 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:13:28.179 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011052600000000001 2020-08-07T15:13:28.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.180 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.180 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:28.181 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302342+0000", 2020-08-07T15:13:28.181 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.781e-06 2020-08-07T15:13:28.181 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.182 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:28.182 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302352+0000", 2020-08-07T15:13:28.183 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7646e-05 2020-08-07T15:13:28.183 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.183 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.183 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:28.184 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302369+0000", 2020-08-07T15:13:28.184 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3629999999999994e-06 2020-08-07T15:13:28.184 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.184 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.185 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:28.185 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302378+0000", 2020-08-07T15:13:28.185 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8007000000000003e-05 2020-08-07T15:13:28.199 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.200 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.200 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:28.200 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302426+0000", 2020-08-07T15:13:28.201 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5159999999999994e-06 2020-08-07T15:13:28.201 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.201 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.209 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:28.210 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302434+0000", 2020-08-07T15:13:28.210 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99151562699999995 2020-08-07T15:13:28.210 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.211 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.211 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:28.211 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.293950+0000", 2020-08-07T15:13:28.212 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020138099999999999 2020-08-07T15:13:28.212 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.212 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.213 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:28.213 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294151+0000", 2020-08-07T15:13:28.213 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024761 2020-08-07T15:13:28.213 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.216 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.216 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:13:28.224 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294399+0000", 2020-08-07T15:13:28.224 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0027714369999999999 2020-08-07T15:13:28.225 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.225 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.225 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:28.226 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297170+0000", 2020-08-07T15:13:28.226 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.234e-06 2020-08-07T15:13:28.226 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.227 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.227 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:28.234 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297172+0000", 2020-08-07T15:13:28.235 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5384999999999998e-05 2020-08-07T15:13:28.235 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.236 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.236 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:28.236 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297237+0000", 2020-08-07T15:13:28.236 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0073576980000000002 2020-08-07T15:13:28.237 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.237 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.253 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:28.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304595+0000", 2020-08-07T15:13:28.253 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1006000000000001e-05 2020-08-07T15:13:28.254 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.261 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.262 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:28.262 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304626+0000", 2020-08-07T15:13:28.262 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6000000000000003e-07 2020-08-07T15:13:28.263 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.263 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.263 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:28.264 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304626+0000", 2020-08-07T15:13:28.264 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8724e-05 2020-08-07T15:13:28.264 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:28.273 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304705+0000", 2020-08-07T15:13:28.274 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0024736009999999 2020-08-07T15:13:28.274 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:28.275 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:28.275 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:28.275 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.283 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.283 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42506 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:13:28.284 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:13:28.284 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 266.901436961, 2020-08-07T15:13:28.284 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002465226, 2020-08-07T15:13:28.285 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:28.285 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:28.285 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:28.302 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:28.302 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:28.302 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42506 2020-08-07T15:13:28.303 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.303 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:28.303 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.303 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:28.303 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:13:28.304 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:28.304 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.304 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.304 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:28.304 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:13:28.305 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8780999999999999e-05 2020-08-07T15:13:28.305 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.305 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.305 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:28.305 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302472+0000", 2020-08-07T15:13:28.306 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3740000000000003e-06 2020-08-07T15:13:28.306 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.306 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.306 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:28.307 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302478+0000", 2020-08-07T15:13:28.307 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1155000000000001e-05 2020-08-07T15:13:28.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.307 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:28.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302489+0000", 2020-08-07T15:13:28.309 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7559999999999996e-06 2020-08-07T15:13:28.309 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.309 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.310 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:28.310 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302495+0000", 2020-08-07T15:13:28.311 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4128000000000003e-05 2020-08-07T15:13:28.311 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.311 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.319 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:28.319 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302529+0000", 2020-08-07T15:13:28.320 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.516e-06 2020-08-07T15:13:28.320 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.320 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.320 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:28.320 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302535+0000", 2020-08-07T15:13:28.321 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99240899299999996 2020-08-07T15:13:28.321 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.321 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.321 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:28.321 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294944+0000", 2020-08-07T15:13:28.322 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000115806 2020-08-07T15:13:28.322 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.322 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.322 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:28.322 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295060+0000", 2020-08-07T15:13:28.323 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000205002 2020-08-07T15:13:28.323 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.323 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.323 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:13:28.323 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295265+0000", 2020-08-07T15:13:28.324 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043414400000000002 2020-08-07T15:13:28.324 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.324 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.324 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:28.324 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299607+0000", 2020-08-07T15:13:28.325 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2229999999999998e-06 2020-08-07T15:13:28.325 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.325 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.326 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:28.327 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299612+0000", 2020-08-07T15:13:28.327 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5797000000000002e-05 2020-08-07T15:13:28.327 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.327 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.328 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:28.328 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299658+0000", 2020-08-07T15:13:28.328 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005095044 2020-08-07T15:13:28.328 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.328 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.329 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:28.329 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304753+0000", 2020-08-07T15:13:28.329 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3121999999999996e-05 2020-08-07T15:13:28.329 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.330 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.330 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:28.330 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304816+0000", 2020-08-07T15:13:28.330 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5700000000000001e-07 2020-08-07T15:13:28.331 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.331 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.331 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:28.331 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304817+0000", 2020-08-07T15:13:28.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1231999999999998e-05 2020-08-07T15:13:28.332 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.332 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.332 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:28.333 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304888+0000", 2020-08-07T15:13:28.333 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002465226 2020-08-07T15:13:28.333 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:28.333 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:28.334 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:28.334 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.334 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.334 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42507 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:13:28.335 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:13:28.335 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 266.90132619100001, 2020-08-07T15:13:28.335 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002567097, 2020-08-07T15:13:28.335 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:28.336 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:28.336 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:28.336 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:28.337 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:28.337 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42507 2020-08-07T15:13:28.337 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.338 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:28.338 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.339 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:28.339 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:13:28.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:28.340 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.340 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:28.340 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:13:28.341 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7679000000000002e-05 2020-08-07T15:13:28.341 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.341 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:28.342 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302581+0000", 2020-08-07T15:13:28.342 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2299999999999999e-06 2020-08-07T15:13:28.342 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.342 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.343 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:28.343 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302586+0000", 2020-08-07T15:13:28.344 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0886999999999999e-05 2020-08-07T15:13:28.344 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.344 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.344 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:28.345 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302597+0000", 2020-08-07T15:13:28.345 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4129999999999997e-06 2020-08-07T15:13:28.346 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.346 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.346 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:28.347 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302603+0000", 2020-08-07T15:13:28.347 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3963000000000001e-05 2020-08-07T15:13:28.347 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.347 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.348 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:28.348 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302627+0000", 2020-08-07T15:13:28.348 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.33e-06 2020-08-07T15:13:28.348 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.350 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.350 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:28.350 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302631+0000", 2020-08-07T15:13:28.351 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99319860900000001 2020-08-07T15:13:28.351 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.351 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.352 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:28.352 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295830+0000", 2020-08-07T15:13:28.356 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000116012 2020-08-07T15:13:28.356 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.356 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.357 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:28.357 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295946+0000", 2020-08-07T15:13:28.357 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000230242 2020-08-07T15:13:28.357 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.358 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.358 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:13:28.358 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296176+0000", 2020-08-07T15:13:28.358 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003494455 2020-08-07T15:13:28.359 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.359 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.359 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:28.360 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299670+0000", 2020-08-07T15:13:28.360 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0299999999999998e-07 2020-08-07T15:13:28.360 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.360 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.361 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:28.368 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299671+0000", 2020-08-07T15:13:28.369 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7298000000000001e-05 2020-08-07T15:13:28.372 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.373 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:28.373 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299708+0000", 2020-08-07T15:13:28.373 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0052628839999999998 2020-08-07T15:13:28.373 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:28.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304971+0000", 2020-08-07T15:13:28.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3754e-05 2020-08-07T15:13:28.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:28.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305035+0000", 2020-08-07T15:13:28.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9500000000000001e-07 2020-08-07T15:13:28.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:28.380 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305036+0000", 2020-08-07T15:13:28.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4843000000000002e-05 2020-08-07T15:13:28.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:28.382 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305101+0000", 2020-08-07T15:13:28.384 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002567097 2020-08-07T15:13:28.384 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:28.384 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:28.384 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:28.385 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.385 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.385 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42508 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:13:28.385 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:13:28.386 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 266.89080117200001, 2020-08-07T15:13:28.386 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99221695600000004, 2020-08-07T15:13:28.386 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:28.386 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:28.387 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:28.387 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:28.387 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:28.387 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42508 2020-08-07T15:13:28.388 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.388 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:28.388 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.388 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:28.389 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:13:28.389 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:28.390 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.390 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.391 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:28.391 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:13:28.391 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3692e-05 2020-08-07T15:13:28.392 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.392 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.392 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:28.393 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313112+0000", 2020-08-07T15:13:28.393 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7399999999999998e-06 2020-08-07T15:13:28.393 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.394 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.394 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:28.395 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313119+0000", 2020-08-07T15:13:28.395 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5153999999999999e-05 2020-08-07T15:13:28.395 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.396 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.398 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:28.398 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313134+0000", 2020-08-07T15:13:28.398 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3039999999999999e-06 2020-08-07T15:13:28.405 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.406 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:28.406 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313141+0000", 2020-08-07T15:13:28.406 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6516000000000001e-05 2020-08-07T15:13:28.407 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.407 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.408 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:28.408 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313178+0000", 2020-08-07T15:13:28.408 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0949999999999998e-06 2020-08-07T15:13:28.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.409 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.409 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:28.410 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313185+0000", 2020-08-07T15:13:28.410 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.98342834999999995 2020-08-07T15:13:28.411 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.411 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.411 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:28.412 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296613+0000", 2020-08-07T15:13:28.412 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2230999999999993e-05 2020-08-07T15:13:28.412 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.414 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.414 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:28.415 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296686+0000", 2020-08-07T15:13:28.415 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013121300000000001 2020-08-07T15:13:28.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.416 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.416 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:13:28.416 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296817+0000", 2020-08-07T15:13:28.417 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044228050000000001 2020-08-07T15:13:28.417 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.417 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.418 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:28.418 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301240+0000", 2020-08-07T15:13:28.420 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0500000000000002e-07 2020-08-07T15:13:28.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.421 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.421 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:28.421 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301241+0000", 2020-08-07T15:13:28.422 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2434000000000006e-05 2020-08-07T15:13:28.422 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.423 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.423 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:28.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301303+0000", 2020-08-07T15:13:28.424 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038622919999999998 2020-08-07T15:13:28.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.424 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:28.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305165+0000", 2020-08-07T15:13:28.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8012000000000001e-05 2020-08-07T15:13:28.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:28.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305203+0000", 2020-08-07T15:13:28.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5299999999999999e-07 2020-08-07T15:13:28.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.430 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.430 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:28.430 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305204+0000", 2020-08-07T15:13:28.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1760000000000004e-05 2020-08-07T15:13:28.431 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:28.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:28.432 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:28.432 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305275+0000", 2020-08-07T15:13:28.432 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99221695600000004 2020-08-07T15:13:28.433 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:28.434 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:28.434 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:28.434 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:28.435 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:28.435 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:28.508 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:13:28.509 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 75 15 111696 111681 11.632 12.1094 0.00413422 0.0107422 2020-08-07T15:13:28.510 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 76 16 113129 113113 11.6261 11.1875 0.00419287 0.0107433 2020-08-07T15:13:28.510 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:13:28.511 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:28.506+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:28.609 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:28.607+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:28.666 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:13:28.678 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:28.678 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:13:28.678 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2020-08-07T15:13:28.679 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:28.689 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:13:28.714 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:28.713+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:28.826 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:28.816+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:28.879 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:28.880 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:13:28.880 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:13:28.880 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:13:28.880 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:28.895 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:13:28.922 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:28.918+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:29.026 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:29.021+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:29.042 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:13:29.072 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:13:29.090 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:29.090 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:13:29.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:13:29.091 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:13:29.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.091 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "MOSDPGPushReply(3.1 62/60 [PushReplyOp(3:8329f37e:::benchmark_data_smithi074_35447_object11915:head)])", 2020-08-07T15:13:29.091 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:13:29.092 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 338.223552805, 2020-08-07T15:13:29.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996, 2020-08-07T15:13:29.092 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:29.092 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:29.092 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:29.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:29.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:13:29.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:29.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:29.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:13:29.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5372000000000001e-05 2020-08-07T15:13:29.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:29.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858416+0000", 2020-08-07T15:13:29.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6899999999999998e-06 2020-08-07T15:13:29.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:29.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858419+0000", 2020-08-07T15:13:29.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63907904299999996 2020-08-07T15:13:29.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:29.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497498+0000", 2020-08-07T15:13:29.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3332999999999999e-05 2020-08-07T15:13:29.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:29.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497512+0000", 2020-08-07T15:13:29.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000450875 2020-08-07T15:13:29.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497962+0000", 2020-08-07T15:13:29.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000267779 2020-08-07T15:13:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.498230+0000", 2020-08-07T15:13:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996 2020-08-07T15:13:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:145792 3.1 3:881e0420:::benchmark_data_smithi074_35447_object16183:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:13:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:13:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 314.415146924, 2020-08-07T15:13:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997, 2020-08-07T15:13:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:13:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:13:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 145792 2020-08-07T15:13:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:13:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:13:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7489000000000001e-05 2020-08-07T15:13:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666824+0000", 2020-08-07T15:13:29.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0169999999999998e-06 2020-08-07T15:13:29.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:29.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666828+0000", 2020-08-07T15:13:29.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9829999999999996e-06 2020-08-07T15:13:29.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:29.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666836+0000", 2020-08-07T15:13:29.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.481e-06 2020-08-07T15:13:29.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:29.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666840+0000", 2020-08-07T15:13:29.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5299000000000001e-05 2020-08-07T15:13:29.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666885+0000", 2020-08-07T15:13:29.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014456500000000001 2020-08-07T15:13:29.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:29.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667030+0000", 2020-08-07T15:13:29.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017677799999999999 2020-08-07T15:13:29.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,5", 2020-08-07T15:13:29.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667207+0000", 2020-08-07T15:13:29.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004149649 2020-08-07T15:13:29.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:29.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671356+0000", 2020-08-07T15:13:29.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019339649999999999 2020-08-07T15:13:29.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:29.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673290+0000", 2020-08-07T15:13:29.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73346620500000004 2020-08-07T15:13:29.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:29.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406756+0000", 2020-08-07T15:13:29.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4700000000000001e-06 2020-08-07T15:13:29.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:29.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406759+0000", 2020-08-07T15:13:29.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6258999999999998e-05 2020-08-07T15:13:29.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:29.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406815+0000", 2020-08-07T15:13:29.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0126000000000002e-05 2020-08-07T15:13:29.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:29.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406855+0000", 2020-08-07T15:13:29.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997 2020-08-07T15:13:29.167 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:29.169 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.170 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:145792 3.1 e75/67 ondisk, result = 0)", 2020-08-07T15:13:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:13:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 314.41076203599999, 2020-08-07T15:13:29.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003, 2020-08-07T15:13:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:29.172 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:13:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:29.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:29.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:13:29.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2830999999999998e-05 2020-08-07T15:13:29.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:29.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671224+0000", 2020-08-07T15:13:29.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:13:29.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:29.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671229+0000", 2020-08-07T15:13:29.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.735414864 2020-08-07T15:13:29.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:29.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406644+0000", 2020-08-07T15:13:29.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1250000000000004e-06 2020-08-07T15:13:29.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:29.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406651+0000", 2020-08-07T15:13:29.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6567e-05 2020-08-07T15:13:29.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406737+0000", 2020-08-07T15:13:29.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0054000000000001e-05 2020-08-07T15:13:29.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:29.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406747+0000", 2020-08-07T15:13:29.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011391999999999999 2020-08-07T15:13:29.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:29.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406861+0000", 2020-08-07T15:13:29.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003 2020-08-07T15:13:29.187 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.187 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:29.187 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.188 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30921 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:13:29.190 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:13:29.190 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 282.37601366199999, 2020-08-07T15:13:29.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001, 2020-08-07T15:13:29.191 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:29.191 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:29.191 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:29.192 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:29.194 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:29.195 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30921 2020-08-07T15:13:29.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.196 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:29.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:29.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:13:29.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:29.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:29.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:13:29.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6954999999999998e-05 2020-08-07T15:13:29.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:29.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706016+0000", 2020-08-07T15:13:29.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0229999999999992e-06 2020-08-07T15:13:29.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:29.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706024+0000", 2020-08-07T15:13:29.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.432e-05 2020-08-07T15:13:29.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:29.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706039+0000", 2020-08-07T15:13:29.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8490000000000006e-06 2020-08-07T15:13:29.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:29.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706046+0000", 2020-08-07T15:13:29.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6837999999999999e-05 2020-08-07T15:13:29.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706113+0000", 2020-08-07T15:13:29.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3553000000000001e-05 2020-08-07T15:13:29.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:29.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706127+0000", 2020-08-07T15:13:29.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77277201699999998 2020-08-07T15:13:29.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.478899+0000", 2020-08-07T15:13:29.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093978599999999996 2020-08-07T15:13:29.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:13:29.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479839+0000", 2020-08-07T15:13:29.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010376029 2020-08-07T15:13:29.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490215+0000", 2020-08-07T15:13:29.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000190657 2020-08-07T15:13:29.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:29.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490405+0000", 2020-08-07T15:13:29.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001 2020-08-07T15:13:29.230 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.230 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:29.230 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.231 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30922 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:13:29.232 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:13:29.232 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 282.37585349900002, 2020-08-07T15:13:29.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247, 2020-08-07T15:13:29.233 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:29.233 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:29.233 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:29.234 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:29.234 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:29.234 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30922 2020-08-07T15:13:29.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:13:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:13:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9178999999999997e-05 2020-08-07T15:13:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706159+0000", 2020-08-07T15:13:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4999999999999999e-06 2020-08-07T15:13:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706164+0000", 2020-08-07T15:13:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1792e-05 2020-08-07T15:13:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706176+0000", 2020-08-07T15:13:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.139e-06 2020-08-07T15:13:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706182+0000", 2020-08-07T15:13:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039724900000000002 2020-08-07T15:13:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706579+0000", 2020-08-07T15:13:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1191e-05 2020-08-07T15:13:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706591+0000", 2020-08-07T15:13:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329101300000003 2020-08-07T15:13:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479882+0000", 2020-08-07T15:13:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9565000000000001e-05 2020-08-07T15:13:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:13:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479971+0000", 2020-08-07T15:13:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010471263 2020-08-07T15:13:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490442+0000", 2020-08-07T15:13:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015035600000000001 2020-08-07T15:13:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490593+0000", 2020-08-07T15:13:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247 2020-08-07T15:13:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30923 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:13:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:13:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 282.37537843799998, 2020-08-07T15:13:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998, 2020-08-07T15:13:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30923 2020-08-07T15:13:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:13:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:13:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5531000000000001e-05 2020-08-07T15:13:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706630+0000", 2020-08-07T15:13:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7689999999999999e-06 2020-08-07T15:13:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706637+0000", 2020-08-07T15:13:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3577999999999999e-05 2020-08-07T15:13:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706650+0000", 2020-08-07T15:13:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1439999999999997e-06 2020-08-07T15:13:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706658+0000", 2020-08-07T15:13:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1663999999999999e-05 2020-08-07T15:13:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706709+0000", 2020-08-07T15:13:29.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0149000000000001e-05 2020-08-07T15:13:29.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:29.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706719+0000", 2020-08-07T15:13:29.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329694900000001 2020-08-07T15:13:29.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480016+0000", 2020-08-07T15:13:29.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7998999999999997e-05 2020-08-07T15:13:29.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:13:29.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480084+0000", 2020-08-07T15:13:29.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010547333000000001 2020-08-07T15:13:29.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490632+0000", 2020-08-07T15:13:29.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7439e-05 2020-08-07T15:13:29.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:29.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490729+0000", 2020-08-07T15:13:29.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998 2020-08-07T15:13:29.269 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.269 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:29.279 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.280 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30924 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:13:29.280 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:13:29.280 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 282.375251166, 2020-08-07T15:13:29.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995, 2020-08-07T15:13:29.281 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:29.281 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:29.281 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:29.281 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:29.281 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:29.282 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30924 2020-08-07T15:13:29.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.282 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:29.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:29.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:13:29.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:29.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:29.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:13:29.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9698999999999999e-05 2020-08-07T15:13:29.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:29.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706762+0000", 2020-08-07T15:13:29.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8080000000000001e-06 2020-08-07T15:13:29.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:29.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706767+0000", 2020-08-07T15:13:29.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1069e-05 2020-08-07T15:13:29.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:29.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706778+0000", 2020-08-07T15:13:29.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6969999999999998e-06 2020-08-07T15:13:29.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:29.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706784+0000", 2020-08-07T15:13:29.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8792999999999998e-05 2020-08-07T15:13:29.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706843+0000", 2020-08-07T15:13:29.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5449999999999998e-06 2020-08-07T15:13:29.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:29.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706850+0000", 2020-08-07T15:13:29.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.773278354 2020-08-07T15:13:29.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480129+0000", 2020-08-07T15:13:29.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1577999999999997e-05 2020-08-07T15:13:29.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:13:29.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480210+0000", 2020-08-07T15:13:29.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010541915000000001 2020-08-07T15:13:29.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490752+0000", 2020-08-07T15:13:29.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103937 2020-08-07T15:13:29.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:29.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490856+0000", 2020-08-07T15:13:29.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995 2020-08-07T15:13:29.317 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.317 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:29.317 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.318 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30925 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:13:29.318 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:13:29.319 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 282.37512739499999, 2020-08-07T15:13:29.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995, 2020-08-07T15:13:29.319 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:29.320 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:29.320 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:29.322 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:29.322 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:29.322 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30925 2020-08-07T15:13:29.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.323 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:29.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:29.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:13:29.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:29.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:29.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:13:29.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4664999999999999e-05 2020-08-07T15:13:29.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:29.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706880+0000", 2020-08-07T15:13:29.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7479999999999999e-06 2020-08-07T15:13:29.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:29.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706886+0000", 2020-08-07T15:13:29.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1838000000000001e-05 2020-08-07T15:13:29.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:29.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706898+0000", 2020-08-07T15:13:29.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0089999999999998e-06 2020-08-07T15:13:29.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:29.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706903+0000", 2020-08-07T15:13:29.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3559e-05 2020-08-07T15:13:29.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706956+0000", 2020-08-07T15:13:29.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0309000000000001e-05 2020-08-07T15:13:29.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:29.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706967+0000", 2020-08-07T15:13:29.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77328290499999996 2020-08-07T15:13:29.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480250+0000", 2020-08-07T15:13:29.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8680999999999997e-05 2020-08-07T15:13:29.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:13:29.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480298+0000", 2020-08-07T15:13:29.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01059272 2020-08-07T15:13:29.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490891+0000", 2020-08-07T15:13:29.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014580300000000001 2020-08-07T15:13:29.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:29.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491037+0000", 2020-08-07T15:13:29.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995 2020-08-07T15:13:29.394 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.394 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:29.395 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.396 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30926 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:13:29.396 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:13:29.398 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 282.37501052599998, 2020-08-07T15:13:29.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005, 2020-08-07T15:13:29.398 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:29.399 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:29.399 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:29.399 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:29.399 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:29.399 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30926 2020-08-07T15:13:29.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.400 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:29.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:29.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:13:29.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:29.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:29.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:13:29.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5223000000000003e-05 2020-08-07T15:13:29.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:29.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706998+0000", 2020-08-07T15:13:29.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4809999999999997e-06 2020-08-07T15:13:29.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:29.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707003+0000", 2020-08-07T15:13:29.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1663e-05 2020-08-07T15:13:29.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:29.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707015+0000", 2020-08-07T15:13:29.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9729999999999997e-06 2020-08-07T15:13:29.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:29.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707020+0000", 2020-08-07T15:13:29.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2565999999999999e-05 2020-08-07T15:13:29.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707062+0000", 2020-08-07T15:13:29.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8944e-05 2020-08-07T15:13:29.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:29.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707081+0000", 2020-08-07T15:13:29.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77325885000000005 2020-08-07T15:13:29.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480340+0000", 2020-08-07T15:13:29.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0321000000000006e-05 2020-08-07T15:13:29.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:13:29.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480420+0000", 2020-08-07T15:13:29.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010655958 2020-08-07T15:13:29.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491076+0000", 2020-08-07T15:13:29.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9784999999999995e-05 2020-08-07T15:13:29.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:29.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491166+0000", 2020-08-07T15:13:29.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005 2020-08-07T15:13:29.413 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.414 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:29.414 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.414 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30927 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:13:29.415 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:13:29.415 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 282.37489270700001, 2020-08-07T15:13:29.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999, 2020-08-07T15:13:29.415 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:29.415 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:29.416 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:29.416 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:29.416 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:29.417 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30927 2020-08-07T15:13:29.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.417 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:29.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:29.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:13:29.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:29.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:29.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:13:29.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9861999999999997e-05 2020-08-07T15:13:29.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:29.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707120+0000", 2020-08-07T15:13:29.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8969999999999996e-06 2020-08-07T15:13:29.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:29.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707126+0000", 2020-08-07T15:13:29.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1248e-05 2020-08-07T15:13:29.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:29.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707137+0000", 2020-08-07T15:13:29.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.955e-06 2020-08-07T15:13:29.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:29.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707142+0000", 2020-08-07T15:13:29.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0643999999999998e-05 2020-08-07T15:13:29.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707193+0000", 2020-08-07T15:13:29.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1983e-05 2020-08-07T15:13:29.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:29.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707205+0000", 2020-08-07T15:13:29.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77324914 2020-08-07T15:13:29.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480454+0000", 2020-08-07T15:13:29.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4119999999999998e-05 2020-08-07T15:13:29.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:13:29.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480498+0000", 2020-08-07T15:13:29.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010709707000000001 2020-08-07T15:13:29.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491208+0000", 2020-08-07T15:13:29.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014433800000000001 2020-08-07T15:13:29.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:29.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491352+0000", 2020-08-07T15:13:29.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999 2020-08-07T15:13:29.442 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.442 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:29.443 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.443 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30928 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:13:29.444 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:13:29.444 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 282.37476835000001, 2020-08-07T15:13:29.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998, 2020-08-07T15:13:29.444 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:29.444 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:29.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:29.445 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:29.445 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:29.445 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30928 2020-08-07T15:13:29.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.445 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:29.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:29.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:13:29.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:29.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:29.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:13:29.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6582000000000001e-05 2020-08-07T15:13:29.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:29.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707241+0000", 2020-08-07T15:13:29.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9190000000000004e-06 2020-08-07T15:13:29.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:29.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707247+0000", 2020-08-07T15:13:29.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000188822 2020-08-07T15:13:29.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:29.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707436+0000", 2020-08-07T15:13:29.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2940000000000002e-06 2020-08-07T15:13:29.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:29.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707442+0000", 2020-08-07T15:13:29.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7800000000000003e-05 2020-08-07T15:13:29.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707490+0000", 2020-08-07T15:13:29.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2020-08-07T15:13:29.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:29.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707501+0000", 2020-08-07T15:13:29.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77303498800000003 2020-08-07T15:13:29.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480536+0000", 2020-08-07T15:13:29.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7723000000000001e-05 2020-08-07T15:13:29.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:13:29.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480614+0000", 2020-08-07T15:13:29.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010812183 2020-08-07T15:13:29.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491426+0000", 2020-08-07T15:13:29.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5502999999999999e-05 2020-08-07T15:13:29.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:29.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491502+0000", 2020-08-07T15:13:29.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998 2020-08-07T15:13:29.460 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.460 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:29.460 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.461 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31009 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:13:29.461 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:13:29.462 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 282.35081204400001, 2020-08-07T15:13:29.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002, 2020-08-07T15:13:29.462 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:29.462 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:29.462 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:29.463 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:29.463 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:29.463 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31009 2020-08-07T15:13:29.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.463 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:29.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:29.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:13:29.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:29.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:29.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:13:29.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2429000000000002e-05 2020-08-07T15:13:29.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:29.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731193+0000", 2020-08-07T15:13:29.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9760000000000003e-06 2020-08-07T15:13:29.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:29.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731198+0000", 2020-08-07T15:13:29.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3597e-05 2020-08-07T15:13:29.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:29.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731212+0000", 2020-08-07T15:13:29.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1709999999999996e-06 2020-08-07T15:13:29.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:29.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731219+0000", 2020-08-07T15:13:29.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5402999999999997e-05 2020-08-07T15:13:29.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731285+0000", 2020-08-07T15:13:29.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3495e-05 2020-08-07T15:13:29.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:29.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731298+0000", 2020-08-07T15:13:29.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74935423599999995 2020-08-07T15:13:29.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480652+0000", 2020-08-07T15:13:29.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000137806 2020-08-07T15:13:29.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:29.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480790+0000", 2020-08-07T15:13:29.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022272399999999999 2020-08-07T15:13:29.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:13:29.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481013+0000", 2020-08-07T15:13:29.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003223381 2020-08-07T15:13:29.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:29.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484236+0000", 2020-08-07T15:13:29.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1370000000000001e-06 2020-08-07T15:13:29.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:29.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484237+0000", 2020-08-07T15:13:29.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6288e-05 2020-08-07T15:13:29.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:29.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484294+0000", 2020-08-07T15:13:29.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.007260402 2020-08-07T15:13:29.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:29.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491554+0000", 2020-08-07T15:13:29.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7358000000000003e-05 2020-08-07T15:13:29.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:29.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491601+0000", 2020-08-07T15:13:29.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1100000000000004e-07 2020-08-07T15:13:29.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:29.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491602+0000", 2020-08-07T15:13:29.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013436500000000001 2020-08-07T15:13:29.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:29.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491737+0000", 2020-08-07T15:13:29.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002 2020-08-07T15:13:29.486 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.486 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:29.486 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.487 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31010 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:13:29.487 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:13:29.487 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 282.35068609000001, 2020-08-07T15:13:29.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001, 2020-08-07T15:13:29.488 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:29.488 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:29.488 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:29.488 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:29.489 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:29.489 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31010 2020-08-07T15:13:29.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.489 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:29.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:29.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:13:29.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:29.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:29.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:13:29.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2398000000000002e-05 2020-08-07T15:13:29.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:29.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731319+0000", 2020-08-07T15:13:29.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7100000000000004e-06 2020-08-07T15:13:29.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:29.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731325+0000", 2020-08-07T15:13:29.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4328000000000001e-05 2020-08-07T15:13:29.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:29.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731339+0000", 2020-08-07T15:13:29.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3310000000000003e-06 2020-08-07T15:13:29.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:29.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731345+0000", 2020-08-07T15:13:29.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8952999999999998e-05 2020-08-07T15:13:29.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731384+0000", 2020-08-07T15:13:29.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0036e-05 2020-08-07T15:13:29.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:29.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731394+0000", 2020-08-07T15:13:29.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75027681300000004 2020-08-07T15:13:29.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481670+0000", 2020-08-07T15:13:29.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016555899999999999 2020-08-07T15:13:29.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:29.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481836+0000", 2020-08-07T15:13:29.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028760999999999999 2020-08-07T15:13:29.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:13:29.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482124+0000", 2020-08-07T15:13:29.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038342680000000001 2020-08-07T15:13:29.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:29.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485958+0000", 2020-08-07T15:13:29.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2559999999999999e-06 2020-08-07T15:13:29.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:29.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485959+0000", 2020-08-07T15:13:29.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3758999999999995e-05 2020-08-07T15:13:29.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:29.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486043+0000", 2020-08-07T15:13:29.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057590419999999998 2020-08-07T15:13:29.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:29.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491802+0000", 2020-08-07T15:13:29.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5757000000000001e-05 2020-08-07T15:13:29.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:29.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:13:29.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9299999999999997e-07 2020-08-07T15:13:29.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:29.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:13:29.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012496999999999999 2020-08-07T15:13:29.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:29.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491993+0000", 2020-08-07T15:13:29.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001 2020-08-07T15:13:29.513 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.513 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:29.514 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.514 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31011 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:13:29.515 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:13:29.515 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 282.35056988500003, 2020-08-07T15:13:29.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997, 2020-08-07T15:13:29.515 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:29.516 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:29.516 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:29.516 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:29.516 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:29.517 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31011 2020-08-07T15:13:29.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.517 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:29.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:29.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:13:29.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:29.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:29.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:13:29.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9539999999999997e-05 2020-08-07T15:13:29.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:29.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731433+0000", 2020-08-07T15:13:29.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9209999999999997e-06 2020-08-07T15:13:29.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:29.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731439+0000", 2020-08-07T15:13:29.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4246999999999999e-05 2020-08-07T15:13:29.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:29.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731453+0000", 2020-08-07T15:13:29.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8130000000000003e-06 2020-08-07T15:13:29.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:29.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731458+0000", 2020-08-07T15:13:29.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6668000000000001e-05 2020-08-07T15:13:29.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731504+0000", 2020-08-07T15:13:29.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1112e-05 2020-08-07T15:13:29.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:29.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731515+0000", 2020-08-07T15:13:29.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75121332100000004 2020-08-07T15:13:29.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482729+0000", 2020-08-07T15:13:29.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1810000000000004e-05 2020-08-07T15:13:29.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:29.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482811+0000", 2020-08-07T15:13:29.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018001399999999999 2020-08-07T15:13:29.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:13:29.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482991+0000", 2020-08-07T15:13:29.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030701140000000001 2020-08-07T15:13:29.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:29.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486061+0000", 2020-08-07T15:13:29.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4499999999999996e-07 2020-08-07T15:13:29.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:29.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486062+0000", 2020-08-07T15:13:29.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5366999999999998e-05 2020-08-07T15:13:29.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:29.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486147+0000", 2020-08-07T15:13:29.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056872779999999996 2020-08-07T15:13:29.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:29.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491834+0000", 2020-08-07T15:13:29.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024645100000000002 2020-08-07T15:13:29.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:29.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:13:29.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1200000000000003e-07 2020-08-07T15:13:29.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:29.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:13:29.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1451000000000002e-05 2020-08-07T15:13:29.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:29.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492173+0000", 2020-08-07T15:13:29.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997 2020-08-07T15:13:29.539 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.539 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:29.539 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.540 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31012 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:13:29.540 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:13:29.540 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 282.34991576599998, 2020-08-07T15:13:29.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003, 2020-08-07T15:13:29.541 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:29.541 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:29.541 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:29.541 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:29.542 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:29.542 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31012 2020-08-07T15:13:29.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.543 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:29.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:29.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:13:29.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:29.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:29.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:13:29.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6344999999999997e-05 2020-08-07T15:13:29.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:29.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732094+0000", 2020-08-07T15:13:29.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2920000000000001e-06 2020-08-07T15:13:29.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:29.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732100+0000", 2020-08-07T15:13:29.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.189e-05 2020-08-07T15:13:29.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:29.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732112+0000", 2020-08-07T15:13:29.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3720000000000001e-06 2020-08-07T15:13:29.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:29.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732118+0000", 2020-08-07T15:13:29.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2514000000000004e-05 2020-08-07T15:13:29.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732191+0000", 2020-08-07T15:13:29.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4610000000000001e-05 2020-08-07T15:13:29.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:29.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732205+0000", 2020-08-07T15:13:29.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75132974100000005 2020-08-07T15:13:29.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483535+0000", 2020-08-07T15:13:29.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9185999999999996e-05 2020-08-07T15:13:29.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:29.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483624+0000", 2020-08-07T15:13:29.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015683 2020-08-07T15:13:29.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:13:29.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483781+0000", 2020-08-07T15:13:29.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046202889999999996 2020-08-07T15:13:29.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:29.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488401+0000", 2020-08-07T15:13:29.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6700000000000002e-07 2020-08-07T15:13:29.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:29.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488402+0000", 2020-08-07T15:13:29.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1487000000000001e-05 2020-08-07T15:13:29.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:29.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488484+0000", 2020-08-07T15:13:29.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035520830000000002 2020-08-07T15:13:29.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:29.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492036+0000", 2020-08-07T15:13:29.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029266600000000002 2020-08-07T15:13:29.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:29.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492328+0000", 2020-08-07T15:13:29.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5099999999999998e-07 2020-08-07T15:13:29.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:29.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492329+0000", 2020-08-07T15:13:29.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000201299 2020-08-07T15:13:29.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:29.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492531+0000", 2020-08-07T15:13:29.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003 2020-08-07T15:13:29.562 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.562 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:29.562 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.563 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31013 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:13:29.563 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:13:29.563 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 282.34970413500002, 2020-08-07T15:13:29.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996, 2020-08-07T15:13:29.563 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:29.564 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:29.564 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:29.564 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:29.564 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:29.564 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31013 2020-08-07T15:13:29.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.566 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:29.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:29.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:13:29.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:29.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:29.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:13:29.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8949000000000002e-05 2020-08-07T15:13:29.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:29.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732298+0000", 2020-08-07T15:13:29.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1399999999999997e-06 2020-08-07T15:13:29.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:29.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732304+0000", 2020-08-07T15:13:29.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.154e-05 2020-08-07T15:13:29.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:29.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732315+0000", 2020-08-07T15:13:29.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1319999999999999e-06 2020-08-07T15:13:29.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:29.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732322+0000", 2020-08-07T15:13:29.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7431e-05 2020-08-07T15:13:29.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732369+0000", 2020-08-07T15:13:29.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0621999999999999e-05 2020-08-07T15:13:29.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:29.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732380+0000", 2020-08-07T15:13:29.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75196845999999995 2020-08-07T15:13:29.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484348+0000", 2020-08-07T15:13:29.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000123008 2020-08-07T15:13:29.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:29.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484471+0000", 2020-08-07T15:13:29.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017572700000000001 2020-08-07T15:13:29.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:13:29.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484647+0000", 2020-08-07T15:13:29.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038557650000000001 2020-08-07T15:13:29.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:29.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:13:29.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2020-08-07T15:13:29.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:29.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:13:29.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0714e-05 2020-08-07T15:13:29.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:29.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488584+0000", 2020-08-07T15:13:29.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036298319999999999 2020-08-07T15:13:29.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:29.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492214+0000", 2020-08-07T15:13:29.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039530099999999997 2020-08-07T15:13:29.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:29.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492609+0000", 2020-08-07T15:13:29.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5300000000000002e-07 2020-08-07T15:13:29.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:29.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492610+0000", 2020-08-07T15:13:29.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197321 2020-08-07T15:13:29.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:29.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492807+0000", 2020-08-07T15:13:29.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996 2020-08-07T15:13:29.585 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.585 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:29.585 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.586 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31014 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:13:29.586 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:13:29.587 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 282.34832199099998, 2020-08-07T15:13:29.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998, 2020-08-07T15:13:29.587 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:29.587 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:29.587 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:29.588 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:29.588 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:29.588 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31014 2020-08-07T15:13:29.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.589 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:29.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:29.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:13:29.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:29.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:29.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:13:29.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3743999999999999e-05 2020-08-07T15:13:29.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:29.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733685+0000", 2020-08-07T15:13:29.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1720000000000007e-06 2020-08-07T15:13:29.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:29.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733693+0000", 2020-08-07T15:13:29.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3674e-05 2020-08-07T15:13:29.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:29.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733707+0000", 2020-08-07T15:13:29.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9249999999999998e-06 2020-08-07T15:13:29.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:29.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733714+0000", 2020-08-07T15:13:29.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8338000000000003e-05 2020-08-07T15:13:29.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733772+0000", 2020-08-07T15:13:29.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3019e-05 2020-08-07T15:13:29.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:29.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733785+0000", 2020-08-07T15:13:29.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75134856299999997 2020-08-07T15:13:29.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485133+0000", 2020-08-07T15:13:29.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8739999999999995e-05 2020-08-07T15:13:29.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:29.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485212+0000", 2020-08-07T15:13:29.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151244 2020-08-07T15:13:29.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:13:29.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485363+0000", 2020-08-07T15:13:29.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032929090000000001 2020-08-07T15:13:29.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:29.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488656+0000", 2020-08-07T15:13:29.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.065e-06 2020-08-07T15:13:29.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:29.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488657+0000", 2020-08-07T15:13:29.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1123000000000003e-05 2020-08-07T15:13:29.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:29.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488729+0000", 2020-08-07T15:13:29.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035508330000000002 2020-08-07T15:13:29.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:29.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492279+0000", 2020-08-07T15:13:29.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00059376000000000003 2020-08-07T15:13:29.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:29.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492873+0000", 2020-08-07T15:13:29.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5700000000000002e-07 2020-08-07T15:13:29.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:29.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492874+0000", 2020-08-07T15:13:29.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019264 2020-08-07T15:13:29.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:29.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493066+0000", 2020-08-07T15:13:29.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998 2020-08-07T15:13:29.617 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.617 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:29.617 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.618 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31015 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:13:29.618 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:13:29.618 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 282.34819602499999, 2020-08-07T15:13:29.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002, 2020-08-07T15:13:29.619 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:29.619 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:29.619 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:29.619 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:29.619 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:29.620 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31015 2020-08-07T15:13:29.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.620 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:29.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:29.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:13:29.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:29.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:29.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:13:29.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3707000000000001e-05 2020-08-07T15:13:29.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:29.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733791+0000", 2020-08-07T15:13:29.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8120000000000001e-06 2020-08-07T15:13:29.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:29.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733794+0000", 2020-08-07T15:13:29.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0728e-05 2020-08-07T15:13:29.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:29.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733805+0000", 2020-08-07T15:13:29.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8279999999999999e-06 2020-08-07T15:13:29.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:29.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733811+0000", 2020-08-07T15:13:29.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2955999999999999e-05 2020-08-07T15:13:29.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733864+0000", 2020-08-07T15:13:29.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1389000000000001e-05 2020-08-07T15:13:29.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:29.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733875+0000", 2020-08-07T15:13:29.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75232711100000005 2020-08-07T15:13:29.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486202+0000", 2020-08-07T15:13:29.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027503400000000001 2020-08-07T15:13:29.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:29.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486478+0000", 2020-08-07T15:13:29.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033010400000000001 2020-08-07T15:13:29.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:13:29.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486808+0000", 2020-08-07T15:13:29.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019382609999999999 2020-08-07T15:13:29.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488746+0000", 2020-08-07T15:13:29.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9400000000000004e-07 2020-08-07T15:13:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488747+0000", 2020-08-07T15:13:29.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4324000000000002e-05 2020-08-07T15:13:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488821+0000", 2020-08-07T15:13:29.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043745640000000001 2020-08-07T15:13:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493196+0000", 2020-08-07T15:13:29.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030381000000000001 2020-08-07T15:13:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493499+0000", 2020-08-07T15:13:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4e-07 2020-08-07T15:13:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493500+0000", 2020-08-07T15:13:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017499 2020-08-07T15:13:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493675+0000", 2020-08-07T15:13:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002 2020-08-07T15:13:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:29.652 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31016 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:13:29.655 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:13:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 282.34760004999998, 2020-08-07T15:13:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971, 2020-08-07T15:13:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:29.656 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:29.664 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:29.664 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:29.664 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31016 2020-08-07T15:13:29.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:13:29.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:29.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:13:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0924000000000002e-05 2020-08-07T15:13:29.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:29.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734404+0000", 2020-08-07T15:13:29.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2020-08-07T15:13:29.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734410+0000", 2020-08-07T15:13:29.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3459999999999994e-06 2020-08-07T15:13:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:29.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734418+0000", 2020-08-07T15:13:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9640000000000002e-06 2020-08-07T15:13:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:29.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734423+0000", 2020-08-07T15:13:29.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7253999999999999e-05 2020-08-07T15:13:29.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734470+0000", 2020-08-07T15:13:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0249999999999997e-06 2020-08-07T15:13:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734477+0000", 2020-08-07T15:13:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75294592900000001 2020-08-07T15:13:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487423+0000", 2020-08-07T15:13:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031604300000000001 2020-08-07T15:13:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487739+0000", 2020-08-07T15:13:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000171149 2020-08-07T15:13:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:13:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487910+0000", 2020-08-07T15:13:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024207769999999998 2020-08-07T15:13:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490331+0000", 2020-08-07T15:13:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1650000000000001e-06 2020-08-07T15:13:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490332+0000", 2020-08-07T15:13:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9796999999999996e-05 2020-08-07T15:13:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490402+0000", 2020-08-07T15:13:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029928849999999998 2020-08-07T15:13:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493395+0000", 2020-08-07T15:13:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034903199999999999 2020-08-07T15:13:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493744+0000", 2020-08-07T15:13:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8899999999999999e-07 2020-08-07T15:13:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493745+0000", 2020-08-07T15:13:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197362 2020-08-07T15:13:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493942+0000", 2020-08-07T15:13:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971 2020-08-07T15:13:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:89632 5.f e207/164 5:fd260020:::benchmark_data_smithi074_53697_object11203:head v 207'5744, mlcod=207'5734)", 2020-08-07T15:13:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:21.551762+0000", 2020-08-07T15:13:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 127.53019119299999, 2020-08-07T15:13:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67527211499999995, 2020-08-07T15:13:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551762+0000", 2020-08-07T15:13:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551762+0000", 2020-08-07T15:13:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8560000000000003e-05 2020-08-07T15:13:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551810+0000", 2020-08-07T15:13:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9430000000000002e-06 2020-08-07T15:13:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551814+0000", 2020-08-07T15:13:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67230272000000002 2020-08-07T15:13:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.224117+0000", 2020-08-07T15:13:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7120000000000003e-06 2020-08-07T15:13:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.224124+0000", 2020-08-07T15:13:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001719307 2020-08-07T15:13:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.225843+0000", 2020-08-07T15:13:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3902999999999997e-05 2020-08-07T15:13:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.225877+0000", 2020-08-07T15:13:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001110279 2020-08-07T15:13:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.226987+0000", 2020-08-07T15:13:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6690999999999997e-05 2020-08-07T15:13:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.227034+0000", 2020-08-07T15:13:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67527211499999995 2020-08-07T15:13:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:29.724 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:29.736 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:13:29.736 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:13:29.737 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:13:29.737 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:13:29.738 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:13:29.739 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:13:29.740 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:13:29.740 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:13:29.741 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:13:29.741 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:13:29.741 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:13:29.742 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:13:29.742 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:13:29.742 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:13:29.743 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 77 16 114449 114433 11.609 10.3125 0.0201896 0.0107602 2020-08-07T15:13:29.743 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:29.729+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:29.744 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:13:29.860 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:29.859+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:29.950 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:29.951 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:29.968 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:13:29.969 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:29.965+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:30.074 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:30.067+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:30.170 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:30.169+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:30.174 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:30.175 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:30.184 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:13:30.271 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:30.270+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:30.274 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 78 16 116077 116061 11.6232 12.7188 0.00703367 0.0107469 2020-08-07T15:13:30.382 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:30.374+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:30.486 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:30.477+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:30.487 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:30.487 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:30.499 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:13:30.514 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:13:30.594 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:30.587+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:30.688 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:30.687+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:30.714 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:30.715 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:30.728 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:13:30.804 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2020-08-07T15:13:30.804 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:13:30.805 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump -f json-pretty 2020-08-07T15:13:30.812 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:30.811+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:30.890 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:30.889+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:30.945 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:30.945 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:30.959 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:13:30.994 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:30.991+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:31.097 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:31.094+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:31.187 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:31.187 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:31.188 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:13:31.208 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:31.205+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:31.282 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 79 16 117772 117756 11.6437 13.2422 0.0089863 0.0107317 2020-08-07T15:13:31.322 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:31.313+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:31.325 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:13:31.325 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:31.325 INFO:teuthology.orchestra.run.smithi074.stdout: "epoch": 287, 2020-08-07T15:13:31.325 INFO:teuthology.orchestra.run.smithi074.stdout: "fsid": "a01d3e82-8d24-4288-9e42-576ab3ff4957", 2020-08-07T15:13:31.326 INFO:teuthology.orchestra.run.smithi074.stdout: "created": "2020-08-07T15:06:07.342912+0000", 2020-08-07T15:13:31.326 INFO:teuthology.orchestra.run.smithi074.stdout: "modified": "2020-08-07T15:13:27.316516+0000", 2020-08-07T15:13:31.326 INFO:teuthology.orchestra.run.smithi074.stdout: "last_up_change": "2020-08-07T15:12:05.338271+0000", 2020-08-07T15:13:31.326 INFO:teuthology.orchestra.run.smithi074.stdout: "last_in_change": "2020-08-07T15:06:17.674292+0000", 2020-08-07T15:13:31.330 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-08-07T15:13:31.331 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_num": 5801984, 2020-08-07T15:13:31.331 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_set": [ 2020-08-07T15:13:31.331 INFO:teuthology.orchestra.run.smithi074.stdout: "noscrub", 2020-08-07T15:13:31.331 INFO:teuthology.orchestra.run.smithi074.stdout: "pglog_hardlimit", 2020-08-07T15:13:31.333 INFO:teuthology.orchestra.run.smithi074.stdout: "purged_snapdirs", 2020-08-07T15:13:31.333 INFO:teuthology.orchestra.run.smithi074.stdout: "recovery_deletes", 2020-08-07T15:13:31.333 INFO:teuthology.orchestra.run.smithi074.stdout: "sortbitwise" 2020-08-07T15:13:31.333 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:13:31.334 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_version": 5, 2020-08-07T15:13:31.343 INFO:teuthology.orchestra.run.smithi074.stdout: "full_ratio": 0.94999998807907104, 2020-08-07T15:13:31.343 INFO:teuthology.orchestra.run.smithi074.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-08-07T15:13:31.343 INFO:teuthology.orchestra.run.smithi074.stdout: "nearfull_ratio": 0.85000002384185791, 2020-08-07T15:13:31.343 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_snapshot": "", 2020-08-07T15:13:31.344 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_max": 6, 2020-08-07T15:13:31.349 INFO:teuthology.orchestra.run.smithi074.stdout: "max_osd": 8, 2020-08-07T15:13:31.349 INFO:teuthology.orchestra.run.smithi074.stdout: "require_min_compat_client": "luminous", 2020-08-07T15:13:31.349 INFO:teuthology.orchestra.run.smithi074.stdout: "min_compat_client": "luminous", 2020-08-07T15:13:31.349 INFO:teuthology.orchestra.run.smithi074.stdout: "require_osd_release": "pacific", 2020-08-07T15:13:31.350 INFO:teuthology.orchestra.run.smithi074.stdout: "pools": [ 2020-08-07T15:13:31.350 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.350 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 1, 2020-08-07T15:13:31.350 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "device_health_metrics", 2020-08-07T15:13:31.372 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:06:10.648876+0000", 2020-08-07T15:13:31.373 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:13:31.373 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:13:31.373 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:13:31.379 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:13:31.379 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:13:31.380 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:13:31.380 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:13:31.380 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:13:31.380 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 41, 2020-08-07T15:13:31.380 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 41, 2020-08-07T15:13:31.381 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 41, 2020-08-07T15:13:31.381 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 41, 2020-08-07T15:13:31.381 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 41, 2020-08-07T15:13:31.381 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:13:31.381 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:13:31.382 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:13:31.382 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:13:31.382 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:13:31.382 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:13:31.382 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:13:31.383 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.383 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "246", 2020-08-07T15:13:31.383 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:13:31.383 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:13:31.383 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "244", 2020-08-07T15:13:31.383 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:13:31.384 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:13:31.384 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:13:31.384 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:13:31.384 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:13:31.384 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:13:31.385 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:13:31.385 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:13:31.385 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:13:31.385 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:13:31.385 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:13:31.418 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:13:31.420 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:13:31.420 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:13:31.420 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:13:31.420 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:13:31.420 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:13:31.421 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:13:31.421 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:13:31.421 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:13:31.421 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:13:31.421 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:13:31.422 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:13:31.422 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.422 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:13:31.422 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:13:31.422 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:13:31.423 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:13:31.423 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:13:31.423 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:13:31.423 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:13:31.439 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:13:31.440 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:13:31.440 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:13:31.440 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:13:31.441 INFO:teuthology.orchestra.run.smithi074.stdout: "options": { 2020-08-07T15:13:31.441 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_min": 1 2020-08-07T15:13:31.441 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.441 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:13:31.452 INFO:teuthology.orchestra.run.smithi074.stdout: "mgr_devicehealth": {} 2020-08-07T15:13:31.453 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:31.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.454 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.454 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 2, 2020-08-07T15:13:31.454 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "rbd", 2020-08-07T15:13:31.454 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:06:21.474648+0000", 2020-08-07T15:13:31.455 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:13:31.455 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:13:31.455 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:13:31.455 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:13:31.455 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:13:31.455 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:13:31.456 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:13:31.456 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:13:31.456 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 8, 2020-08-07T15:13:31.456 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 8, 2020-08-07T15:13:31.456 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 8, 2020-08-07T15:13:31.457 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 8, 2020-08-07T15:13:31.457 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 8, 2020-08-07T15:13:31.457 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:13:31.457 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:13:31.457 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:13:31.460 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:13:31.460 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:13:31.460 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:13:31.460 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:13:31.466 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.466 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "19", 2020-08-07T15:13:31.466 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:13:31.466 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:13:31.466 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "0", 2020-08-07T15:13:31.467 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:13:31.467 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:13:31.467 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:13:31.467 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:13:31.467 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:13:31.468 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:13:31.468 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:13:31.468 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:13:31.468 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:13:31.468 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:13:31.469 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:13:31.469 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:13:31.469 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:13:31.469 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:13:31.469 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:13:31.470 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:13:31.470 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:13:31.470 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:13:31.470 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:13:31.470 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:13:31.471 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:13:31.471 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:13:31.471 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:13:31.471 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.472 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:13:31.472 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:13:31.478 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:13:31.479 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:13:31.479 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:13:31.479 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:13:31.479 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:13:31.480 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:13:31.480 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:13:31.480 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:13:31.480 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:13:31.480 INFO:teuthology.orchestra.run.smithi074.stdout: "options": {}, 2020-08-07T15:13:31.481 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:13:31.481 INFO:teuthology.orchestra.run.smithi074.stdout: "rbd": {} 2020-08-07T15:13:31.481 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:31.481 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.481 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.482 INFO:teuthology.orchestra.run.smithi074.stdout: "pool": 6, 2020-08-07T15:13:31.482 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_name": "unique_pool_3", 2020-08-07T15:13:31.515 INFO:teuthology.orchestra.run.smithi074.stdout: "create_time": "2020-08-07T15:12:08.847598+0000", 2020-08-07T15:13:31.515 INFO:teuthology.orchestra.run.smithi074.stdout: "flags": 1, 2020-08-07T15:13:31.516 INFO:teuthology.orchestra.run.smithi074.stdout: "flags_names": "hashpspool", 2020-08-07T15:13:31.516 INFO:teuthology.orchestra.run.smithi074.stdout: "type": 1, 2020-08-07T15:13:31.516 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 3, 2020-08-07T15:13:31.516 INFO:teuthology.orchestra.run.smithi074.stdout: "min_size": 2, 2020-08-07T15:13:31.516 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_rule": 0, 2020-08-07T15:13:31.517 INFO:teuthology.orchestra.run.smithi074.stdout: "object_hash": 2, 2020-08-07T15:13:31.517 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_autoscale_mode": "off", 2020-08-07T15:13:31.517 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num": 16, 2020-08-07T15:13:31.517 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num": 16, 2020-08-07T15:13:31.517 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_placement_num_target": 16, 2020-08-07T15:13:31.536 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_target": 16, 2020-08-07T15:13:31.536 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_num_pending": 16, 2020-08-07T15:13:31.536 INFO:teuthology.orchestra.run.smithi074.stdout: "last_pg_merge_meta": { 2020-08-07T15:13:31.537 INFO:teuthology.orchestra.run.smithi074.stdout: "source_pgid": "0.0", 2020-08-07T15:13:31.537 INFO:teuthology.orchestra.run.smithi074.stdout: "ready_epoch": 0, 2020-08-07T15:13:31.537 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_started": 0, 2020-08-07T15:13:31.538 INFO:teuthology.orchestra.run.smithi074.stdout: "last_epoch_clean": 0, 2020-08-07T15:13:31.538 INFO:teuthology.orchestra.run.smithi074.stdout: "source_version": "0'0", 2020-08-07T15:13:31.538 INFO:teuthology.orchestra.run.smithi074.stdout: "target_version": "0'0" 2020-08-07T15:13:31.552 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.553 INFO:teuthology.orchestra.run.smithi074.stdout: "last_change": "232", 2020-08-07T15:13:31.553 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend": "0", 2020-08-07T15:13:31.553 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_prenautilus": "0", 2020-08-07T15:13:31.554 INFO:teuthology.orchestra.run.smithi074.stdout: "last_force_op_resend_preluminous": "0", 2020-08-07T15:13:31.554 INFO:teuthology.orchestra.run.smithi074.stdout: "auid": 0, 2020-08-07T15:13:31.555 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_mode": "selfmanaged", 2020-08-07T15:13:31.555 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_seq": 0, 2020-08-07T15:13:31.555 INFO:teuthology.orchestra.run.smithi074.stdout: "snap_epoch": 0, 2020-08-07T15:13:31.555 INFO:teuthology.orchestra.run.smithi074.stdout: "pool_snaps": [], 2020-08-07T15:13:31.555 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps": "[]", 2020-08-07T15:13:31.556 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_bytes": 0, 2020-08-07T15:13:31.556 INFO:teuthology.orchestra.run.smithi074.stdout: "quota_max_objects": 0, 2020-08-07T15:13:31.556 INFO:teuthology.orchestra.run.smithi074.stdout: "tiers": [], 2020-08-07T15:13:31.556 INFO:teuthology.orchestra.run.smithi074.stdout: "tier_of": -1, 2020-08-07T15:13:31.556 INFO:teuthology.orchestra.run.smithi074.stdout: "read_tier": -1, 2020-08-07T15:13:31.557 INFO:teuthology.orchestra.run.smithi074.stdout: "write_tier": -1, 2020-08-07T15:13:31.557 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_mode": "none", 2020-08-07T15:13:31.557 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_bytes": 0, 2020-08-07T15:13:31.557 INFO:teuthology.orchestra.run.smithi074.stdout: "target_max_objects": 0, 2020-08-07T15:13:31.557 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-08-07T15:13:31.566 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-08-07T15:13:31.566 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_target_full_ratio_micro": 800000, 2020-08-07T15:13:31.566 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_flush_age": 0, 2020-08-07T15:13:31.566 INFO:teuthology.orchestra.run.smithi074.stdout: "cache_min_evict_age": 0, 2020-08-07T15:13:31.567 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profile": "", 2020-08-07T15:13:31.567 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_params": { 2020-08-07T15:13:31.567 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "none" 2020-08-07T15:13:31.567 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.567 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_period": 0, 2020-08-07T15:13:31.568 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_count": 0, 2020-08-07T15:13:31.568 INFO:teuthology.orchestra.run.smithi074.stdout: "use_gmt_hitset": true, 2020-08-07T15:13:31.568 INFO:teuthology.orchestra.run.smithi074.stdout: "min_read_recency_for_promote": 0, 2020-08-07T15:13:31.568 INFO:teuthology.orchestra.run.smithi074.stdout: "min_write_recency_for_promote": 0, 2020-08-07T15:13:31.568 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_grade_decay_rate": 0, 2020-08-07T15:13:31.569 INFO:teuthology.orchestra.run.smithi074.stdout: "hit_set_search_last_n": 0, 2020-08-07T15:13:31.569 INFO:teuthology.orchestra.run.smithi074.stdout: "grade_table": [], 2020-08-07T15:13:31.569 INFO:teuthology.orchestra.run.smithi074.stdout: "stripe_width": 0, 2020-08-07T15:13:31.569 INFO:teuthology.orchestra.run.smithi074.stdout: "expected_num_objects": 0, 2020-08-07T15:13:31.569 INFO:teuthology.orchestra.run.smithi074.stdout: "fast_read": false, 2020-08-07T15:13:31.572 INFO:teuthology.orchestra.run.smithi074.stdout: "options": {}, 2020-08-07T15:13:31.572 INFO:teuthology.orchestra.run.smithi074.stdout: "application_metadata": { 2020-08-07T15:13:31.573 INFO:teuthology.orchestra.run.smithi074.stdout: "rados": {} 2020-08-07T15:13:31.573 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:31.573 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:31.573 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:13:31.574 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:13:31.578 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.578 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 0, 2020-08-07T15:13:31.579 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "fc1ebb23-3ef4-4fc0-b395-382e385885a3", 2020-08-07T15:13:31.579 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:13:31.579 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:13:31.579 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:13:31.579 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0, 2020-08-07T15:13:31.580 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:13:31.580 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:13:31.580 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:13:31.580 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 262, 2020-08-07T15:13:31.580 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:13:31.581 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:13:31.581 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:13:31.581 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:31.581 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.581 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:31.590 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6801", 2020-08-07T15:13:31.590 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:13:31.590 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:31.590 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:31.590 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.591 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:13:31.591 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:31.591 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.591 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:31.591 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6802", 2020-08-07T15:13:31.592 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:13:31.592 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:31.592 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:31.592 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.592 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:13:31.593 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:31.593 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.594 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:31.594 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6804", 2020-08-07T15:13:31.594 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:13:31.594 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:31.594 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:31.595 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.595 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:13:31.595 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:31.595 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.595 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:31.596 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6803", 2020-08-07T15:13:31.596 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31577 2020-08-07T15:13:31.596 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:31.596 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:31.596 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.597 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6801/31577", 2020-08-07T15:13:31.597 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6802/31577", 2020-08-07T15:13:31.597 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6804/31577", 2020-08-07T15:13:31.597 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6803/31577", 2020-08-07T15:13:31.597 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:13:31.622 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:13:31.626 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:13:31.626 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:31.626 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.627 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 1, 2020-08-07T15:13:31.627 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "1fbe530a-63ce-4dc8-82ab-18793d6ebc0b", 2020-08-07T15:13:31.627 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:13:31.627 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:13:31.627 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:13:31.628 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0.7196044921875, 2020-08-07T15:13:31.646 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 13, 2020-08-07T15:13:31.647 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 190, 2020-08-07T15:13:31.647 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 227, 2020-08-07T15:13:31.647 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 285, 2020-08-07T15:13:31.649 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 191, 2020-08-07T15:13:31.663 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:13:31.663 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:13:31.663 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:31.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.664 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:31.664 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6809", 2020-08-07T15:13:31.664 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 62587 2020-08-07T15:13:31.664 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:31.665 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:31.665 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.665 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:13:31.665 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:31.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.696 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:31.697 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6810", 2020-08-07T15:13:31.697 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 62587 2020-08-07T15:13:31.697 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:31.697 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:31.697 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.698 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:13:31.698 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:31.698 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.698 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:31.698 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6812", 2020-08-07T15:13:31.699 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 62587 2020-08-07T15:13:31.699 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:31.699 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:31.699 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.699 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:13:31.700 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:31.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.700 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:31.700 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6811", 2020-08-07T15:13:31.729 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 62587 2020-08-07T15:13:31.729 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:31.729 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:31.729 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.734 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6809/62587", 2020-08-07T15:13:31.734 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6810/62587", 2020-08-07T15:13:31.735 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6812/62587", 2020-08-07T15:13:31.735 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6811/62587", 2020-08-07T15:13:31.736 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:13:31.736 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:13:31.736 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:13:31.736 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:31.737 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.737 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.737 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 2, 2020-08-07T15:13:31.737 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "d6fe4338-3e11-4f73-9cbb-e1b046ce286a", 2020-08-07T15:13:31.737 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:13:31.737 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:13:31.738 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:13:31.738 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:13:31.738 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:13:31.738 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:13:31.738 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:13:31.739 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 269, 2020-08-07T15:13:31.739 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:13:31.739 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:13:31.739 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:13:31.739 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:31.740 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.740 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:31.763 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6805", 2020-08-07T15:13:31.763 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:13:31.763 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:31.764 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:31.764 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.764 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:13:31.765 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:31.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.768 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:31.768 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6806", 2020-08-07T15:13:31.768 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:13:31.769 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:31.769 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:31.769 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.769 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:13:31.769 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:31.770 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.774 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:31.774 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6808", 2020-08-07T15:13:31.774 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:13:31.774 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:31.775 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:31.775 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.775 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:13:31.775 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:31.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.776 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:31.776 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6807", 2020-08-07T15:13:31.776 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31576 2020-08-07T15:13:31.776 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:31.777 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:31.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.777 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6805/31576", 2020-08-07T15:13:31.777 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6806/31576", 2020-08-07T15:13:31.777 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6808/31576", 2020-08-07T15:13:31.783 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6807/31576", 2020-08-07T15:13:31.787 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:13:31.787 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:13:31.787 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:13:31.787 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:31.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.788 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 3, 2020-08-07T15:13:31.788 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9", 2020-08-07T15:13:31.788 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:13:31.789 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:13:31.793 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:13:31.793 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:13:31.793 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:13:31.794 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:13:31.794 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:13:31.795 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 269, 2020-08-07T15:13:31.795 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:13:31.795 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:13:31.795 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:13:31.795 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:31.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.796 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:31.796 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6813", 2020-08-07T15:13:31.797 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:13:31.797 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:31.797 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:31.797 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.797 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:13:31.805 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:31.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.805 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:31.806 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6814", 2020-08-07T15:13:31.806 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:13:31.806 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:31.806 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:31.806 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.835 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:13:31.835 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:31.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.836 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:31.837 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6816", 2020-08-07T15:13:31.837 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:13:31.837 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:31.837 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:31.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.838 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:13:31.838 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:31.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.838 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:31.839 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.74:6815", 2020-08-07T15:13:31.839 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31579 2020-08-07T15:13:31.839 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:31.839 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:31.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.840 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.74:6813/31579", 2020-08-07T15:13:31.840 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.74:6814/31579", 2020-08-07T15:13:31.840 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.74:6816/31579", 2020-08-07T15:13:31.840 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.74:6815/31579", 2020-08-07T15:13:31.840 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:13:31.841 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:13:31.841 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:13:31.841 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:31.841 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.874 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 4, 2020-08-07T15:13:31.874 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "01aadc13-29c5-4ddd-9bb9-de24b2b20412", 2020-08-07T15:13:31.875 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:13:31.875 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:13:31.875 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:13:31.875 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0, 2020-08-07T15:13:31.876 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:13:31.876 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:13:31.876 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:13:31.876 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 248, 2020-08-07T15:13:31.876 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:13:31.877 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:13:31.877 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:13:31.878 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:31.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.888 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:31.894 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6800", 2020-08-07T15:13:31.894 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:13:31.894 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:31.896 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:31.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.896 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:13:31.920 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:31.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.921 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:31.921 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6801", 2020-08-07T15:13:31.922 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:13:31.922 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:31.922 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:31.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.922 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:13:31.923 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:31.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.923 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:31.923 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6803", 2020-08-07T15:13:31.923 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:13:31.924 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:31.924 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:31.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.924 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:13:31.924 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:31.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:31.925 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:31.925 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6802", 2020-08-07T15:13:31.925 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31545 2020-08-07T15:13:31.950 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:31.951 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:31.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:31.951 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6800/31545", 2020-08-07T15:13:31.958 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6801/31545", 2020-08-07T15:13:31.959 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6803/31545", 2020-08-07T15:13:32.006 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6802/31545", 2020-08-07T15:13:32.008 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:13:32.014 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:13:32.015 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:13:32.015 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:32.016 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:32.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.017 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 5, 2020-08-07T15:13:32.017 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "f5556a82-73b4-47b1-8e29-1d2fe031d8fd", 2020-08-07T15:13:32.018 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:13:32.018 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:13:32.018 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:13:32.019 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:13:32.019 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:13:32.019 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:13:32.020 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:13:32.020 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 269, 2020-08-07T15:13:32.020 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:13:32.021 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:13:32.033 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:13:32.033 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:32.034 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.034 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:32.034 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6804", 2020-08-07T15:13:32.035 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:13:32.035 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:32.035 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:32.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:32.036 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:13:32.036 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:32.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.037 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:32.077 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6805", 2020-08-07T15:13:32.077 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:13:32.078 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:32.078 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:32.078 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:32.079 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:13:32.079 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:32.079 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.080 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:32.080 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6807", 2020-08-07T15:13:32.080 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:13:32.081 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:32.081 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:32.081 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:32.113 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:13:32.114 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:32.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.114 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:32.115 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6806", 2020-08-07T15:13:32.130 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31547 2020-08-07T15:13:32.130 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:32.130 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:32.130 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:32.131 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6804/31547", 2020-08-07T15:13:32.131 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6805/31547", 2020-08-07T15:13:32.131 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6807/31547", 2020-08-07T15:13:32.131 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6806/31547", 2020-08-07T15:13:32.131 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:13:32.132 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:13:32.132 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:13:32.132 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:32.132 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:32.132 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.133 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 6, 2020-08-07T15:13:32.133 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "9b98b337-e1cc-43cd-9e9b-f21ae3f20899", 2020-08-07T15:13:32.133 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:13:32.133 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:13:32.133 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:13:32.142 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 1, 2020-08-07T15:13:32.143 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:13:32.143 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:13:32.143 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 14, 2020-08-07T15:13:32.144 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 269, 2020-08-07T15:13:32.144 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:13:32.145 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:13:32.155 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:13:32.155 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:32.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.156 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:32.156 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6808", 2020-08-07T15:13:32.157 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:13:32.157 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:32.158 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:32.158 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:32.159 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:13:32.159 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:32.159 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.160 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:32.160 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6809", 2020-08-07T15:13:32.169 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:13:32.169 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:32.174 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:32.174 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:32.175 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:13:32.175 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:32.176 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.176 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:32.185 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6811", 2020-08-07T15:13:32.185 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:13:32.186 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:32.186 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:32.187 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:32.187 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:13:32.187 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:32.198 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.198 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:32.203 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6810", 2020-08-07T15:13:32.204 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31546 2020-08-07T15:13:32.204 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:32.204 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:32.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:32.205 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6808/31546", 2020-08-07T15:13:32.205 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6809/31546", 2020-08-07T15:13:32.207 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6811/31546", 2020-08-07T15:13:32.207 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6810/31546", 2020-08-07T15:13:32.208 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:13:32.208 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:13:32.208 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:13:32.209 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:32.214 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:32.214 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.221 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 7, 2020-08-07T15:13:32.221 INFO:teuthology.orchestra.run.smithi074.stdout: "uuid": "777662d8-bd34-48e8-83e1-24ce9e61f14b", 2020-08-07T15:13:32.221 INFO:teuthology.orchestra.run.smithi074.stdout: "up": 1, 2020-08-07T15:13:32.222 INFO:teuthology.orchestra.run.smithi074.stdout: "in": 1, 2020-08-07T15:13:32.222 INFO:teuthology.orchestra.run.smithi074.stdout: "weight": 1, 2020-08-07T15:13:32.222 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_affinity": 0, 2020-08-07T15:13:32.223 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_begin": 0, 2020-08-07T15:13:32.223 INFO:teuthology.orchestra.run.smithi074.stdout: "last_clean_end": 0, 2020-08-07T15:13:32.223 INFO:teuthology.orchestra.run.smithi074.stdout: "up_from": 13, 2020-08-07T15:13:32.223 INFO:teuthology.orchestra.run.smithi074.stdout: "up_thru": 244, 2020-08-07T15:13:32.223 INFO:teuthology.orchestra.run.smithi074.stdout: "down_at": 0, 2020-08-07T15:13:32.224 INFO:teuthology.orchestra.run.smithi074.stdout: "lost_at": 0, 2020-08-07T15:13:32.224 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addrs": { 2020-08-07T15:13:32.224 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:32.224 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.224 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:32.225 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6812", 2020-08-07T15:13:32.225 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:13:32.225 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:32.225 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:32.225 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:32.230 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addrs": { 2020-08-07T15:13:32.230 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:32.230 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.231 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:32.231 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6813", 2020-08-07T15:13:32.231 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:13:32.231 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:32.231 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:32.232 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:32.232 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addrs": { 2020-08-07T15:13:32.232 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:32.238 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.239 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:32.239 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6815", 2020-08-07T15:13:32.239 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:13:32.239 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:32.239 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:32.240 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:32.240 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addrs": { 2020-08-07T15:13:32.240 INFO:teuthology.orchestra.run.smithi074.stdout: "addrvec": [ 2020-08-07T15:13:32.240 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.240 INFO:teuthology.orchestra.run.smithi074.stdout: "type": "v2", 2020-08-07T15:13:32.241 INFO:teuthology.orchestra.run.smithi074.stdout: "addr": "172.21.15.96:6814", 2020-08-07T15:13:32.241 INFO:teuthology.orchestra.run.smithi074.stdout: "nonce": 31570 2020-08-07T15:13:32.241 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:32.242 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:32.242 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:32.242 INFO:teuthology.orchestra.run.smithi074.stdout: "public_addr": "172.21.15.96:6812/31570", 2020-08-07T15:13:32.251 INFO:teuthology.orchestra.run.smithi074.stdout: "cluster_addr": "172.21.15.96:6813/31570", 2020-08-07T15:13:32.251 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_back_addr": "172.21.15.96:6815/31570", 2020-08-07T15:13:32.252 INFO:teuthology.orchestra.run.smithi074.stdout: "heartbeat_front_addr": "172.21.15.96:6814/31570", 2020-08-07T15:13:32.252 INFO:teuthology.orchestra.run.smithi074.stdout: "state": [ 2020-08-07T15:13:32.252 INFO:teuthology.orchestra.run.smithi074.stdout: "exists", 2020-08-07T15:13:32.253 INFO:teuthology.orchestra.run.smithi074.stdout: "up" 2020-08-07T15:13:32.262 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:32.262 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:32.263 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:13:32.263 INFO:teuthology.orchestra.run.smithi074.stdout: "osd_xinfo": [ 2020-08-07T15:13:32.263 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.264 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 0, 2020-08-07T15:13:32.264 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:13:32.264 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:13:32.265 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:13:32.265 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:13:32.265 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:13:32.274 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:13:19.219042+0000", 2020-08-07T15:13:32.274 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:13:32.275 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:32.275 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.275 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 1, 2020-08-07T15:13:32.276 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:13:32.276 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:13:32.276 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:13:32.277 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:13:32.277 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:13:32.277 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:13:06.000449+0000", 2020-08-07T15:13:32.306 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 191 2020-08-07T15:13:32.306 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:32.307 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.307 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 2, 2020-08-07T15:13:32.307 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:13:32.308 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:13:32.308 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:13:32.308 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:13:32.309 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:13:32.309 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:13:18.172058+0000", 2020-08-07T15:13:32.309 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:13:32.314 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:32.314 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.314 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 3, 2020-08-07T15:13:32.315 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:13:32.315 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:13:32.315 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:13:32.316 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:13:32.316 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:13:32.316 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:13:20.003208+0000", 2020-08-07T15:13:32.317 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:13:32.317 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:32.317 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.334 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 4, 2020-08-07T15:13:32.334 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:13:32.335 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:13:32.335 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:13:32.335 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:13:32.336 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:13:32.336 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:13:18.887538+0000", 2020-08-07T15:13:32.336 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:13:32.337 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:32.337 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.337 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 5, 2020-08-07T15:13:32.362 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:13:32.363 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:13:32.363 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:13:32.363 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:13:32.364 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:13:32.364 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:13:19.118356+0000", 2020-08-07T15:13:32.364 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:13:32.365 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:32.365 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.365 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 6, 2020-08-07T15:13:32.377 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:13:32.377 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:13:32.377 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:13:32.378 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:13:32.378 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:13:32.378 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:13:18.841797+0000", 2020-08-07T15:13:32.378 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:13:32.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:32.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.379 INFO:teuthology.orchestra.run.smithi074.stdout: "osd": 7, 2020-08-07T15:13:32.379 INFO:teuthology.orchestra.run.smithi074.stdout: "down_stamp": "0.000000", 2020-08-07T15:13:32.379 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_probability": 0, 2020-08-07T15:13:32.379 INFO:teuthology.orchestra.run.smithi074.stdout: "laggy_interval": 0, 2020-08-07T15:13:32.380 INFO:teuthology.orchestra.run.smithi074.stdout: "features": 4540138292837744639, 2020-08-07T15:13:32.380 INFO:teuthology.orchestra.run.smithi074.stdout: "old_weight": 0, 2020-08-07T15:13:32.380 INFO:teuthology.orchestra.run.smithi074.stdout: "last_purged_snaps_scrub": "2020-08-07T15:13:17.455633+0000", 2020-08-07T15:13:32.380 INFO:teuthology.orchestra.run.smithi074.stdout: "dead_epoch": 0 2020-08-07T15:13:32.380 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:32.381 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:13:32.381 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_upmap": [ 2020-08-07T15:13:32.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.381 INFO:teuthology.orchestra.run.smithi074.stdout: "pgid": "1.16", 2020-08-07T15:13:32.381 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:13:32.398 INFO:teuthology.orchestra.run.smithi074.stdout: 0, 2020-08-07T15:13:32.399 INFO:teuthology.orchestra.run.smithi074.stdout: 5, 2020-08-07T15:13:32.406 INFO:teuthology.orchestra.run.smithi074.stdout: 4 2020-08-07T15:13:32.406 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:32.407 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:32.407 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.407 INFO:teuthology.orchestra.run.smithi074.stdout: "pgid": "1.1d", 2020-08-07T15:13:32.408 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:13:32.408 INFO:teuthology.orchestra.run.smithi074.stdout: 1, 2020-08-07T15:13:32.408 INFO:teuthology.orchestra.run.smithi074.stdout: 5, 2020-08-07T15:13:32.409 INFO:teuthology.orchestra.run.smithi074.stdout: 0 2020-08-07T15:13:32.409 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:32.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:32.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.423 INFO:teuthology.orchestra.run.smithi074.stdout: "pgid": "6.a", 2020-08-07T15:13:32.423 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:13:32.423 INFO:teuthology.orchestra.run.smithi074.stdout: 6, 2020-08-07T15:13:32.424 INFO:teuthology.orchestra.run.smithi074.stdout: 5, 2020-08-07T15:13:32.424 INFO:teuthology.orchestra.run.smithi074.stdout: 7 2020-08-07T15:13:32.424 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:32.424 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:32.425 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:13:32.433 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_upmap_items": [], 2020-08-07T15:13:32.433 INFO:teuthology.orchestra.run.smithi074.stdout: "pg_temp": [ 2020-08-07T15:13:32.433 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:32.434 INFO:teuthology.orchestra.run.smithi074.stdout: "pgid": "6.a", 2020-08-07T15:13:32.434 INFO:teuthology.orchestra.run.smithi074.stdout: "osds": [ 2020-08-07T15:13:32.434 INFO:teuthology.orchestra.run.smithi074.stdout: 6, 2020-08-07T15:13:32.435 INFO:teuthology.orchestra.run.smithi074.stdout: 5, 2020-08-07T15:13:32.435 INFO:teuthology.orchestra.run.smithi074.stdout: 0 2020-08-07T15:13:32.436 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:32.436 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:32.436 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:13:32.436 INFO:teuthology.orchestra.run.smithi074.stdout: "primary_temp": [], 2020-08-07T15:13:32.436 INFO:teuthology.orchestra.run.smithi074.stdout: "blacklist": {}, 2020-08-07T15:13:32.437 INFO:teuthology.orchestra.run.smithi074.stdout: "erasure_code_profiles": { 2020-08-07T15:13:32.437 INFO:teuthology.orchestra.run.smithi074.stdout: "default": { 2020-08-07T15:13:32.437 INFO:teuthology.orchestra.run.smithi074.stdout: "crush-failure-domain": "osd", 2020-08-07T15:13:32.437 INFO:teuthology.orchestra.run.smithi074.stdout: "k": "2", 2020-08-07T15:13:32.462 INFO:teuthology.orchestra.run.smithi074.stdout: "m": "1", 2020-08-07T15:13:32.462 INFO:teuthology.orchestra.run.smithi074.stdout: "plugin": "jerasure", 2020-08-07T15:13:32.462 INFO:teuthology.orchestra.run.smithi074.stdout: "ruleset-failure-domain": "osd", 2020-08-07T15:13:32.462 INFO:teuthology.orchestra.run.smithi074.stdout: "technique": "reed_sol_van" 2020-08-07T15:13:32.463 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:32.463 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:32.463 INFO:teuthology.orchestra.run.smithi074.stdout: "removed_snaps_queue": [], 2020-08-07T15:13:32.463 INFO:teuthology.orchestra.run.smithi074.stdout: "new_removed_snaps": [], 2020-08-07T15:13:32.463 INFO:teuthology.orchestra.run.smithi074.stdout: "new_purged_snaps": [], 2020-08-07T15:13:32.464 INFO:teuthology.orchestra.run.smithi074.stdout: "crush_node_flags": {}, 2020-08-07T15:13:32.464 INFO:teuthology.orchestra.run.smithi074.stdout: "device_class_flags": {} 2020-08-07T15:13:32.464 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:32.465 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:13:32.484 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:32.484 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:32.485 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:13:32.485 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:32.464+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:32.501 INFO:tasks.thrashosds.thrasher:j is {'epoch': 287, 'fsid': 'a01d3e82-8d24-4288-9e42-576ab3ff4957', 'created': '2020-08-07T15:06:07.342912+0000', 'modified': '2020-08-07T15:13:27.316516+0000', 'last_up_change': '2020-08-07T15:12:05.338271+0000', 'last_in_change': '2020-08-07T15:06:17.674292+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 6, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-08-07T15:06:10.648876+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 41, 'pg_placement_num': 41, 'pg_placement_num_target': 41, 'pg_num_target': 41, 'pg_num_pending': 41, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '246', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '244', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-08-07T15:06:21.474648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 6, 'pool_name': 'unique_pool_3', 'create_time': '2020-08-07T15:12:08.847598+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '232', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': 'fc1ebb23-3ef4-4fc0-b395-382e385885a3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 262, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6801', 'nonce': 31577}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6802', 'nonce': 31577}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6804', 'nonce': 31577}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6803', 'nonce': 31577}]}, 'public_addr': '172.21.15.74:6801/31577', 'cluster_addr': '172.21.15.74:6802/31577', 'heartbeat_back_addr': '172.21.15.74:6804/31577', 'heartbeat_front_addr': '172.21.15.74:6803/31577', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '1fbe530a-63ce-4dc8-82ab-18793d6ebc0b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.7196044921875, 'last_clean_begin': 13, 'last_clean_end': 190, 'up_from': 227, 'up_thru': 285, 'down_at': 191, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6809', 'nonce': 62587}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6810', 'nonce': 62587}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6812', 'nonce': 62587}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6811', 'nonce': 62587}]}, 'public_addr': '172.21.15.74:6809/62587', 'cluster_addr': '172.21.15.74:6810/62587', 'heartbeat_back_addr': '172.21.15.74:6812/62587', 'heartbeat_front_addr': '172.21.15.74:6811/62587', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd6fe4338-3e11-4f73-9cbb-e1b046ce286a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 269, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6805', 'nonce': 31576}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6806', 'nonce': 31576}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6808', 'nonce': 31576}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6807', 'nonce': 31576}]}, 'public_addr': '172.21.15.74:6805/31576', 'cluster_addr': '172.21.15.74:6806/31576', 'heartbeat_back_addr': '172.21.15.74:6808/31576', 'heartbeat_front_addr': '172.21.15.74:6807/31576', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'bef9ce0e-6750-4ca0-b58f-e5dfd25e00f9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 269, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6813', 'nonce': 31579}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6814', 'nonce': 31579}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6816', 'nonce': 31579}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.74:6815', 'nonce': 31579}]}, 'public_addr': '172.21.15.74:6813/31579', 'cluster_addr': '172.21.15.74:6814/31579', 'heartbeat_back_addr': '172.21.15.74:6816/31579', 'heartbeat_front_addr': '172.21.15.74:6815/31579', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '01aadc13-29c5-4ddd-9bb9-de24b2b20412', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 248, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6800', 'nonce': 31545}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6801', 'nonce': 31545}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6803', 'nonce': 31545}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6802', 'nonce': 31545}]}, 'public_addr': '172.21.15.96:6800/31545', 'cluster_addr': '172.21.15.96:6801/31545', 'heartbeat_back_addr': '172.21.15.96:6803/31545', 'heartbeat_front_addr': '172.21.15.96:6802/31545', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'f5556a82-73b4-47b1-8e29-1d2fe031d8fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 269, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6804', 'nonce': 31547}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6805', 'nonce': 31547}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6807', 'nonce': 31547}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6806', 'nonce': 31547}]}, 'public_addr': '172.21.15.96:6804/31547', 'cluster_addr': '172.21.15.96:6805/31547', 'heartbeat_back_addr': '172.21.15.96:6807/31547', 'heartbeat_front_addr': '172.21.15.96:6806/31547', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9b98b337-e1cc-43cd-9e9b-f21ae3f20899', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 269, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6808', 'nonce': 31546}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6809', 'nonce': 31546}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6811', 'nonce': 31546}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6810', 'nonce': 31546}]}, 'public_addr': '172.21.15.96:6808/31546', 'cluster_addr': '172.21.15.96:6809/31546', 'heartbeat_back_addr': '172.21.15.96:6811/31546', 'heartbeat_front_addr': '172.21.15.96:6810/31546', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '777662d8-bd34-48e8-83e1-24ce9e61f14b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 244, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6812', 'nonce': 31570}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6813', 'nonce': 31570}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6815', 'nonce': 31570}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6814', 'nonce': 31570}]}, 'public_addr': '172.21.15.96:6812/31570', 'cluster_addr': '172.21.15.96:6813/31570', 'heartbeat_back_addr': '172.21.15.96:6815/31570', 'heartbeat_front_addr': '172.21.15.96:6814/31570', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:13:19.219042+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:13:06.000449+0000', 'dead_epoch': 191}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:13:18.172058+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:13:20.003208+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:13:18.887538+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:13:19.118356+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:13:18.841797+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138292837744639, 'old_weight': 0, 'last_purged_snaps_scrub': '2020-08-07T15:13:17.455633+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '1.16', 'osds': [0, 5, 4]}, {'pgid': '1.1d', 'osds': [1, 5, 0]}, {'pgid': '6.a', 'osds': [6, 5, 7]}], 'pg_upmap_items': [], 'pg_temp': [{'pgid': '6.a', 'osds': [6, 5, 0]}], 'primary_temp': [], 'blacklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}} 2020-08-07T15:13:32.502 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:13:32.507 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:13:32.508 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:2020-08-07T15:13:32.274175+0000 min lat: 0.000909485 max lat: 1.58913 avg lat: 0.0107181 2020-08-07T15:13:32.508 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-08-07T15:13:32.508 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 80 16 119400 119384 11.6571 12.7188 0.0066873 0.0107181 2020-08-07T15:13:32.575 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:32.569+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:32.674 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:32.671+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:32.689 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:32.690 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:32.751 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:13:32.786 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:32.785+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:32.879 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:13:32.901 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:32.890+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:32.902 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:13:32.934 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:32.936 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:32.950 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:13:32.951 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:13:33.000 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:13:33.001 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:32.998+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:33.033 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:13:33.067 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:13:33.100 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:13:33.136 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:13:33.136 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:13:33.137 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":507,"stamp":"2020-08-07T15:13:32.696087+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":956792832,"num_objects":14602,"num_object_clones":0,"num_object_copies":43806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":858,"num_objects_unfound":0,"num_objects_dirty":14602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116796,"num_write_kb":934368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":3670016,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":198,"ondisk_log_size":198,"up":195,"acting":195,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":196,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":11234024,"kb_used_data":2844236,"kb_used_omap":2,"kb_used_meta":8388605,"kb_avail":743740696,"statfs":{"total":773094113280,"available":761590472704,"internally_reserved":8589934592,"allocated":2912497664,"data_stored":2908169756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":8589932013},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,5,10,1,1,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":113672192,"num_objects":1734,"num_object_clones":0,"num_object_copies":5202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":11,"num_objects_unfound":0,"num_objects_dirty":1734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13876,"num_write_kb":111008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":3670016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001563"},"pg_stats":[{"pgid":"1.28","version":"16'2","reported_seq":"52","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.334121+0000","last_change":"2020-08-07T15:12:37.495634+0000","last_active":"2020-08-07T15:13:27.334121+0000","last_peered":"2020-08-07T15:13:27.334121+0000","last_clean":"2020-08-07T15:13:27.334121+0000","last_became_active":"2020-08-07T15:12:37.495085+0000","last_became_peered":"2020-08-07T15:12:37.495085+0000","last_unstale":"2020-08-07T15:13:27.334121+0000","last_undegraded":"2020-08-07T15:13:27.334121+0000","last_fullsized":"2020-08-07T15:13:27.334121+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.490655+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.490655+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.27","version":"16'2","reported_seq":"399","reported_epoch":"286","state":"active+clean","last_fresh":"2020-08-07T15:13:24.328142+0000","last_change":"2020-08-07T15:12:57.250017+0000","last_active":"2020-08-07T15:13:24.328142+0000","last_peered":"2020-08-07T15:13:24.328142+0000","last_clean":"2020-08-07T15:13:24.328142+0000","last_became_active":"2020-08-07T15:12:37.411869+0000","last_became_peered":"2020-08-07T15:12:37.411869+0000","last_unstale":"2020-08-07T15:13:24.328142+0000","last_undegraded":"2020-08-07T15:13:24.328142+0000","last_fullsized":"2020-08-07T15:13:24.328142+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:57.249927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:57.249927+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.26","version":"16'2","reported_seq":"61","reported_epoch":"286","state":"active+clean","last_fresh":"2020-08-07T15:13:24.319056+0000","last_change":"2020-08-07T15:12:40.765120+0000","last_active":"2020-08-07T15:13:24.319056+0000","last_peered":"2020-08-07T15:13:24.319056+0000","last_clean":"2020-08-07T15:13:24.319056+0000","last_became_active":"2020-08-07T15:12:37.457426+0000","last_became_peered":"2020-08-07T15:12:37.457426+0000","last_unstale":"2020-08-07T15:13:24.319056+0000","last_undegraded":"2020-08-07T15:13:24.319056+0000","last_fullsized":"2020-08-07T15:13:24.319056+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:40.765027+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_clean_scrub_stamp":"2020-08-07T15:12:40.765027+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.25","version":"75'3","reported_seq":"288","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.342294+0000","last_change":"2020-08-07T15:13:07.465488+0000","last_active":"2020-08-07T15:13:27.342294+0000","last_peered":"2020-08-07T15:13:27.342294+0000","last_clean":"2020-08-07T15:13:27.342294+0000","last_became_active":"2020-08-07T15:12:37.427816+0000","last_became_peered":"2020-08-07T15:12:37.427816+0000","last_unstale":"2020-08-07T15:13:27.342294+0000","last_undegraded":"2020-08-07T15:13:27.342294+0000","last_fullsized":"2020-08-07T15:13:27.342294+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:13:07.465410+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:13:07.465410+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.24","version":"16'2","reported_seq":"226","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.337242+0000","last_change":"2020-08-07T15:12:42.295755+0000","last_active":"2020-08-07T15:13:27.337242+0000","last_peered":"2020-08-07T15:13:27.337242+0000","last_clean":"2020-08-07T15:13:27.337242+0000","last_became_active":"2020-08-07T15:12:34.414144+0000","last_became_peered":"2020-08-07T15:12:34.414144+0000","last_unstale":"2020-08-07T15:13:27.337242+0000","last_undegraded":"2020-08-07T15:13:27.337242+0000","last_fullsized":"2020-08-07T15:13:27.337242+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:42.295672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:42.295672+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.23","version":"16'2","reported_seq":"334","reported_epoch":"286","state":"active+clean","last_fresh":"2020-08-07T15:13:24.327948+0000","last_change":"2020-08-07T15:12:55.290134+0000","last_active":"2020-08-07T15:13:24.327948+0000","last_peered":"2020-08-07T15:13:24.327948+0000","last_clean":"2020-08-07T15:13:24.327948+0000","last_became_active":"2020-08-07T15:12:37.410157+0000","last_became_peered":"2020-08-07T15:12:37.410157+0000","last_unstale":"2020-08-07T15:13:24.327948+0000","last_undegraded":"2020-08-07T15:13:24.327948+0000","last_fullsized":"2020-08-07T15:13:24.327948+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:55.290028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:55.290028+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1],"acting":[6,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.22","version":"16'2","reported_seq":"54","reported_epoch":"286","state":"active+clean","last_fresh":"2020-08-07T15:13:24.331975+0000","last_change":"2020-08-07T15:12:58.227750+0000","last_active":"2020-08-07T15:13:24.331975+0000","last_peered":"2020-08-07T15:13:24.331975+0000","last_clean":"2020-08-07T15:13:24.331975+0000","last_became_active":"2020-08-07T15:12:37.413046+0000","last_became_peered":"2020-08-07T15:12:37.413046+0000","last_unstale":"2020-08-07T15:13:24.331975+0000","last_undegraded":"2020-08-07T15:13:24.331975+0000","last_fullsized":"2020-08-07T15:13:24.331975+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:58.227699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:58.227699+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.21","version":"16'2","reported_seq":"62","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.336907+0000","last_change":"2020-08-07T15:12:58.387474+0000","last_active":"2020-08-07T15:13:27.336907+0000","last_peered":"2020-08-07T15:13:27.336907+0000","last_clean":"2020-08-07T15:13:27.336907+0000","last_became_active":"2020-08-07T15:12:37.418612+0000","last_became_peered":"2020-08-07T15:12:37.418612+0000","last_unstale":"2020-08-07T15:13:27.336907+0000","last_undegraded":"2020-08-07T15:13:27.336907+0000","last_fullsized":"2020-08-07T15:13:27.336907+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:58.387424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:58.387424+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.20","version":"16'2","reported_seq":"62","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.341545+0000","last_change":"2020-08-07T15:12:54.433326+0000","last_active":"2020-08-07T15:13:27.341545+0000","last_peered":"2020-08-07T15:13:27.341545+0000","last_clean":"2020-08-07T15:13:27.341545+0000","last_became_active":"2020-08-07T15:12:37.409249+0000","last_became_peered":"2020-08-07T15:12:37.409249+0000","last_unstale":"2020-08-07T15:13:27.341545+0000","last_undegraded":"2020-08-07T15:13:27.341545+0000","last_fullsized":"2020-08-07T15:13:27.341545+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:54.433245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:54.433245+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1f","version":"16'2","reported_seq":"55","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.342585+0000","last_change":"2020-08-07T15:13:09.471546+0000","last_active":"2020-08-07T15:13:27.342585+0000","last_peered":"2020-08-07T15:13:27.342585+0000","last_clean":"2020-08-07T15:13:27.342585+0000","last_became_active":"2020-08-07T15:12:37.410716+0000","last_became_peered":"2020-08-07T15:12:37.410716+0000","last_unstale":"2020-08-07T15:13:27.342585+0000","last_undegraded":"2020-08-07T15:13:27.342585+0000","last_fullsized":"2020-08-07T15:13:27.342585+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:09.471471+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_clean_scrub_stamp":"2020-08-07T15:13:09.471471+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1e","version":"16'2","reported_seq":"142","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.337343+0000","last_change":"2020-08-07T15:13:09.396723+0000","last_active":"2020-08-07T15:13:27.337343+0000","last_peered":"2020-08-07T15:13:27.337343+0000","last_clean":"2020-08-07T15:13:27.337343+0000","last_became_active":"2020-08-07T15:10:51.559894+0000","last_became_peered":"2020-08-07T15:10:51.559894+0000","last_unstale":"2020-08-07T15:13:27.337343+0000","last_undegraded":"2020-08-07T15:13:27.337343+0000","last_fullsized":"2020-08-07T15:13:27.337343+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:09.396633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:09.396633+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1d","version":"75'3","reported_seq":"168","reported_epoch":"286","state":"active+clean","last_fresh":"2020-08-07T15:13:24.345776+0000","last_change":"2020-08-07T15:13:24.343240+0000","last_active":"2020-08-07T15:13:24.345776+0000","last_peered":"2020-08-07T15:13:24.345776+0000","last_clean":"2020-08-07T15:13:24.345776+0000","last_became_active":"2020-08-07T15:13:24.342549+0000","last_became_peered":"2020-08-07T15:13:24.342549+0000","last_unstale":"2020-08-07T15:13:24.345776+0000","last_undegraded":"2020-08-07T15:13:24.345776+0000","last_fullsized":"2020-08-07T15:13:24.345776+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:28.855432+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:28.855432+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1c","version":"16'2","reported_seq":"142","reported_epoch":"286","state":"active+clean","last_fresh":"2020-08-07T15:13:24.329270+0000","last_change":"2020-08-07T15:13:07.471270+0000","last_active":"2020-08-07T15:13:24.329270+0000","last_peered":"2020-08-07T15:13:24.329270+0000","last_clean":"2020-08-07T15:13:24.329270+0000","last_became_active":"2020-08-07T15:13:02.591669+0000","last_became_peered":"2020-08-07T15:13:02.591669+0000","last_unstale":"2020-08-07T15:13:24.329270+0000","last_undegraded":"2020-08-07T15:13:24.329270+0000","last_fullsized":"2020-08-07T15:13:24.329270+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:07.471192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:07.471192+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1b","version":"16'2","reported_seq":"263","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.334858+0000","last_change":"2020-08-07T15:13:22.022126+0000","last_active":"2020-08-07T15:13:27.334858+0000","last_peered":"2020-08-07T15:13:27.334858+0000","last_clean":"2020-08-07T15:13:27.334858+0000","last_became_active":"2020-08-07T15:10:51.557392+0000","last_became_peered":"2020-08-07T15:10:51.557392+0000","last_unstale":"2020-08-07T15:13:27.334858+0000","last_undegraded":"2020-08-07T15:13:27.334858+0000","last_fullsized":"2020-08-07T15:13:27.334858+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:22.022047+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:57.796946+0000","last_clean_scrub_stamp":"2020-08-07T15:13:22.022047+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1a","version":"16'2","reported_seq":"33","reported_epoch":"286","state":"active+clean","last_fresh":"2020-08-07T15:13:24.315738+0000","last_change":"2020-08-07T15:13:06.881943+0000","last_active":"2020-08-07T15:13:24.315738+0000","last_peered":"2020-08-07T15:13:24.315738+0000","last_clean":"2020-08-07T15:13:24.315738+0000","last_became_active":"2020-08-07T15:13:02.590409+0000","last_became_peered":"2020-08-07T15:13:02.590409+0000","last_unstale":"2020-08-07T15:13:24.315738+0000","last_undegraded":"2020-08-07T15:13:24.315738+0000","last_fullsized":"2020-08-07T15:13:24.315738+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:06.881863+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:13:06.881863+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.19","version":"16'2","reported_seq":"185","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.336996+0000","last_change":"2020-08-07T15:12:15.372999+0000","last_active":"2020-08-07T15:13:27.336996+0000","last_peered":"2020-08-07T15:13:27.336996+0000","last_clean":"2020-08-07T15:13:27.336996+0000","last_became_active":"2020-08-07T15:10:51.559752+0000","last_became_peered":"2020-08-07T15:10:51.559752+0000","last_unstale":"2020-08-07T15:13:27.336996+0000","last_undegraded":"2020-08-07T15:13:27.336996+0000","last_fullsized":"2020-08-07T15:13:27.336996+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:15.372910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:15.372910+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.18","version":"16'2","reported_seq":"285","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.334227+0000","last_change":"2020-08-07T15:11:06.955336+0000","last_active":"2020-08-07T15:13:27.334227+0000","last_peered":"2020-08-07T15:13:27.334227+0000","last_clean":"2020-08-07T15:13:27.334227+0000","last_became_active":"2020-08-07T15:10:51.557665+0000","last_became_peered":"2020-08-07T15:10:51.557665+0000","last_unstale":"2020-08-07T15:13:27.334227+0000","last_undegraded":"2020-08-07T15:13:27.334227+0000","last_fullsized":"2020-08-07T15:13:27.334227+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:06.955262+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:06.955262+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.17","version":"16'2","reported_seq":"116","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.335098+0000","last_change":"2020-08-07T15:12:57.172008+0000","last_active":"2020-08-07T15:13:27.335098+0000","last_peered":"2020-08-07T15:13:27.335098+0000","last_clean":"2020-08-07T15:13:27.335098+0000","last_became_active":"2020-08-07T15:10:51.561769+0000","last_became_peered":"2020-08-07T15:10:51.561769+0000","last_unstale":"2020-08-07T15:13:27.335098+0000","last_undegraded":"2020-08-07T15:13:27.335098+0000","last_fullsized":"2020-08-07T15:13:27.335098+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:57.171893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:57.171893+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.16","version":"16'2","reported_seq":"61","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.334985+0000","last_change":"2020-08-07T15:12:52.576339+0000","last_active":"2020-08-07T15:13:27.334985+0000","last_peered":"2020-08-07T15:13:27.334985+0000","last_clean":"2020-08-07T15:13:27.334985+0000","last_became_active":"2020-08-07T15:12:52.575577+0000","last_became_peered":"2020-08-07T15:12:52.575577+0000","last_unstale":"2020-08-07T15:13:27.334985+0000","last_undegraded":"2020-08-07T15:13:27.334985+0000","last_fullsized":"2020-08-07T15:13:27.334985+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:12.883558+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:12.883558+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.883558+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.15","version":"75'3","reported_seq":"44","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.335166+0000","last_change":"2020-08-07T15:13:02.590465+0000","last_active":"2020-08-07T15:13:27.335166+0000","last_peered":"2020-08-07T15:13:27.335166+0000","last_clean":"2020-08-07T15:13:27.335166+0000","last_became_active":"2020-08-07T15:13:02.589879+0000","last_became_peered":"2020-08-07T15:13:02.589879+0000","last_unstale":"2020-08-07T15:13:27.335166+0000","last_undegraded":"2020-08-07T15:13:27.335166+0000","last_fullsized":"2020-08-07T15:13:27.335166+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:53.824617+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:53.824617+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.14","version":"16'2","reported_seq":"226","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.341851+0000","last_change":"2020-08-07T15:13:26.549332+0000","last_active":"2020-08-07T15:13:27.341851+0000","last_peered":"2020-08-07T15:13:27.341851+0000","last_clean":"2020-08-07T15:13:27.341851+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:13:27.341851+0000","last_undegraded":"2020-08-07T15:13:27.341851+0000","last_fullsized":"2020-08-07T15:13:27.341851+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:26.549247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:26.549247+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"237","reported_epoch":"286","state":"active+clean","last_fresh":"2020-08-07T15:13:24.315979+0000","last_change":"2020-08-07T15:12:19.577926+0000","last_active":"2020-08-07T15:13:24.315979+0000","last_peered":"2020-08-07T15:13:24.315979+0000","last_clean":"2020-08-07T15:13:24.315979+0000","last_became_active":"2020-08-07T15:12:06.375861+0000","last_became_peered":"2020-08-07T15:12:06.375861+0000","last_unstale":"2020-08-07T15:13:24.315979+0000","last_undegraded":"2020-08-07T15:13:24.315979+0000","last_fullsized":"2020-08-07T15:13:24.315979+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:07.887356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.887356+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"238","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.334681+0000","last_change":"2020-08-07T15:12:26.104598+0000","last_active":"2020-08-07T15:13:27.334681+0000","last_peered":"2020-08-07T15:13:27.334681+0000","last_clean":"2020-08-07T15:13:27.334681+0000","last_became_active":"2020-08-07T15:12:06.367362+0000","last_became_peered":"2020-08-07T15:12:06.367362+0000","last_unstale":"2020-08-07T15:13:27.334681+0000","last_undegraded":"2020-08-07T15:13:27.334681+0000","last_fullsized":"2020-08-07T15:13:27.334681+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:26.104526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:26.104526+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"336","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.334506+0000","last_change":"2020-08-07T15:12:24.105593+0000","last_active":"2020-08-07T15:13:27.334506+0000","last_peered":"2020-08-07T15:13:27.334506+0000","last_clean":"2020-08-07T15:13:27.334506+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:13:27.334506+0000","last_undegraded":"2020-08-07T15:13:27.334506+0000","last_fullsized":"2020-08-07T15:13:27.334506+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:24.105531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:24.105531+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"330","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.334330+0000","last_change":"2020-08-07T15:12:13.231819+0000","last_active":"2020-08-07T15:13:27.334330+0000","last_peered":"2020-08-07T15:13:27.334330+0000","last_clean":"2020-08-07T15:13:27.334330+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:13:27.334330+0000","last_undegraded":"2020-08-07T15:13:27.334330+0000","last_fullsized":"2020-08-07T15:13:27.334330+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:13.231762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.231762+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3","version":"287'6616","reported_seq":"6683","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.711158+0000","last_change":"2020-08-07T15:13:02.600977+0000","last_active":"2020-08-07T15:13:27.711158+0000","last_peered":"2020-08-07T15:13:27.711158+0000","last_clean":"2020-08-07T15:13:27.711158+0000","last_became_active":"2020-08-07T15:13:02.590681+0000","last_became_peered":"2020-08-07T15:13:02.590681+0000","last_unstale":"2020-08-07T15:13:27.711158+0000","last_undegraded":"2020-08-07T15:13:27.711158+0000","last_fullsized":"2020-08-07T15:13:27.711158+0000","mapping_epoch":269,"log_start":"287'6609","ondisk_log_start":"287'6609","created":230,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":54198272,"num_objects":827,"num_object_clones":0,"num_object_copies":2481,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6616,"num_write_kb":52928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"289","reported_epoch":"286","state":"active+clean","last_fresh":"2020-08-07T15:13:24.328400+0000","last_change":"2020-08-07T15:12:27.410339+0000","last_active":"2020-08-07T15:13:24.328400+0000","last_peered":"2020-08-07T15:13:24.328400+0000","last_clean":"2020-08-07T15:13:24.328400+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:13:24.328400+0000","last_undegraded":"2020-08-07T15:13:24.328400+0000","last_fullsized":"2020-08-07T15:13:24.328400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:27.410262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:27.410262+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"218","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.337322+0000","last_change":"2020-08-07T15:12:39.258928+0000","last_active":"2020-08-07T15:13:27.337322+0000","last_peered":"2020-08-07T15:13:27.337322+0000","last_clean":"2020-08-07T15:13:27.337322+0000","last_became_active":"2020-08-07T15:12:32.259956+0000","last_became_peered":"2020-08-07T15:12:32.259956+0000","last_unstale":"2020-08-07T15:13:27.337322+0000","last_undegraded":"2020-08-07T15:13:27.337322+0000","last_fullsized":"2020-08-07T15:13:27.337322+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:39.258857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:39.258857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"288'7816","reported_seq":"7884","reported_epoch":"288","state":"active+clean","last_fresh":"2020-08-07T15:13:31.766376+0000","last_change":"2020-08-07T15:13:20.918113+0000","last_active":"2020-08-07T15:13:31.766376+0000","last_peered":"2020-08-07T15:13:31.766376+0000","last_clean":"2020-08-07T15:13:31.766376+0000","last_became_active":"2020-08-07T15:12:10.464493+0000","last_became_peered":"2020-08-07T15:12:10.464493+0000","last_unstale":"2020-08-07T15:13:31.766376+0000","last_undegraded":"2020-08-07T15:13:31.766376+0000","last_fullsized":"2020-08-07T15:13:31.766376+0000","mapping_epoch":230,"log_start":"288'7809","ondisk_log_start":"288'7809","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"282'6856","last_scrub_stamp":"2020-08-07T15:13:20.918036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:13:20.918036+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":64028672,"num_objects":977,"num_object_clones":0,"num_object_copies":2931,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7816,"num_write_kb":62528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"245","reported_epoch":"286","state":"active+clean","last_fresh":"2020-08-07T15:13:24.319162+0000","last_change":"2020-08-07T15:12:19.561477+0000","last_active":"2020-08-07T15:13:24.319162+0000","last_peered":"2020-08-07T15:13:24.319162+0000","last_clean":"2020-08-07T15:13:24.319162+0000","last_became_active":"2020-08-07T15:12:06.367376+0000","last_became_peered":"2020-08-07T15:12:06.367376+0000","last_unstale":"2020-08-07T15:13:24.319162+0000","last_undegraded":"2020-08-07T15:13:24.319162+0000","last_fullsized":"2020-08-07T15:13:24.319162+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:12.319383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.319383+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"272","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.342507+0000","last_change":"2020-08-07T15:12:44.858389+0000","last_active":"2020-08-07T15:13:27.342507+0000","last_peered":"2020-08-07T15:13:27.342507+0000","last_clean":"2020-08-07T15:13:27.342507+0000","last_became_active":"2020-08-07T15:12:32.241146+0000","last_became_peered":"2020-08-07T15:12:32.241146+0000","last_unstale":"2020-08-07T15:13:27.342507+0000","last_undegraded":"2020-08-07T15:13:27.342507+0000","last_fullsized":"2020-08-07T15:13:27.342507+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:44.858310+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:44.858310+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"287'7344","reported_seq":"7408","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:29.853885+0000","last_change":"2020-08-07T15:12:10.464412+0000","last_active":"2020-08-07T15:13:29.853885+0000","last_peered":"2020-08-07T15:13:29.853885+0000","last_clean":"2020-08-07T15:13:29.853885+0000","last_became_active":"2020-08-07T15:12:10.463950+0000","last_became_peered":"2020-08-07T15:12:10.463950+0000","last_unstale":"2020-08-07T15:13:29.853885+0000","last_undegraded":"2020-08-07T15:13:29.853885+0000","last_fullsized":"2020-08-07T15:13:29.853885+0000","mapping_epoch":230,"log_start":"287'7336","ondisk_log_start":"287'7336","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60162048,"num_objects":918,"num_object_clones":0,"num_object_copies":2754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7344,"num_write_kb":58752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"291","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.345421+0000","last_change":"2020-08-07T15:13:11.273543+0000","last_active":"2020-08-07T15:13:27.345421+0000","last_peered":"2020-08-07T15:13:27.345421+0000","last_clean":"2020-08-07T15:13:27.345421+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:13:27.345421+0000","last_undegraded":"2020-08-07T15:13:27.345421+0000","last_fullsized":"2020-08-07T15:13:27.345421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:11.273469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:13:11.273469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"349","reported_epoch":"286","state":"active+clean","last_fresh":"2020-08-07T15:13:24.328111+0000","last_change":"2020-08-07T15:12:53.210176+0000","last_active":"2020-08-07T15:13:24.328111+0000","last_peered":"2020-08-07T15:13:24.328111+0000","last_clean":"2020-08-07T15:13:24.328111+0000","last_became_active":"2020-08-07T15:12:32.239755+0000","last_became_peered":"2020-08-07T15:12:32.239755+0000","last_unstale":"2020-08-07T15:13:24.328111+0000","last_undegraded":"2020-08-07T15:13:24.328111+0000","last_fullsized":"2020-08-07T15:13:24.328111+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:53.210093+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_clean_scrub_stamp":"2020-08-07T15:12:53.210093+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"287'7336","reported_seq":"7401","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:29.828306+0000","last_change":"2020-08-07T15:12:10.398415+0000","last_active":"2020-08-07T15:13:29.828306+0000","last_peered":"2020-08-07T15:13:29.828306+0000","last_clean":"2020-08-07T15:13:29.828306+0000","last_became_active":"2020-08-07T15:12:10.397834+0000","last_became_peered":"2020-08-07T15:12:10.397834+0000","last_unstale":"2020-08-07T15:13:29.828306+0000","last_undegraded":"2020-08-07T15:13:29.828306+0000","last_fullsized":"2020-08-07T15:13:29.828306+0000","mapping_epoch":230,"log_start":"287'7329","ondisk_log_start":"287'7329","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60096512,"num_objects":917,"num_object_clones":0,"num_object_copies":2751,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7336,"num_write_kb":58688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"260","reported_epoch":"286","state":"active+clean","last_fresh":"2020-08-07T15:13:24.315552+0000","last_change":"2020-08-07T15:13:02.591821+0000","last_active":"2020-08-07T15:13:24.315552+0000","last_peered":"2020-08-07T15:13:24.315552+0000","last_clean":"2020-08-07T15:13:24.315552+0000","last_became_active":"2020-08-07T15:13:02.591402+0000","last_became_peered":"2020-08-07T15:13:02.591402+0000","last_unstale":"2020-08-07T15:13:24.315552+0000","last_undegraded":"2020-08-07T15:13:24.315552+0000","last_fullsized":"2020-08-07T15:13:24.315552+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:08.622683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:08.622683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"318","reported_epoch":"286","state":"active+clean","last_fresh":"2020-08-07T15:13:24.328078+0000","last_change":"2020-08-07T15:12:42.371309+0000","last_active":"2020-08-07T15:13:24.328078+0000","last_peered":"2020-08-07T15:13:24.328078+0000","last_clean":"2020-08-07T15:13:24.328078+0000","last_became_active":"2020-08-07T15:12:32.237984+0000","last_became_peered":"2020-08-07T15:12:32.237984+0000","last_unstale":"2020-08-07T15:13:24.328078+0000","last_undegraded":"2020-08-07T15:13:24.328078+0000","last_fullsized":"2020-08-07T15:13:24.328078+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:42.371229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:42.371229+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.7","version":"288'7528","reported_seq":"7594","reported_epoch":"288","state":"active+clean","last_fresh":"2020-08-07T15:13:31.625777+0000","last_change":"2020-08-07T15:12:10.464810+0000","last_active":"2020-08-07T15:13:31.625777+0000","last_peered":"2020-08-07T15:13:31.625777+0000","last_clean":"2020-08-07T15:13:31.625777+0000","last_became_active":"2020-08-07T15:12:10.464050+0000","last_became_peered":"2020-08-07T15:12:10.464050+0000","last_unstale":"2020-08-07T15:13:31.625777+0000","last_undegraded":"2020-08-07T15:13:31.625777+0000","last_fullsized":"2020-08-07T15:13:31.625777+0000","mapping_epoch":230,"log_start":"288'7519","ondisk_log_start":"288'7519","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61669376,"num_objects":941,"num_object_clones":0,"num_object_copies":2823,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7528,"num_write_kb":60224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"44","reported_epoch":"286","state":"active+clean","last_fresh":"2020-08-07T15:13:24.327767+0000","last_change":"2020-08-07T15:13:13.399410+0000","last_active":"2020-08-07T15:13:24.327767+0000","last_peered":"2020-08-07T15:13:24.327767+0000","last_clean":"2020-08-07T15:13:24.327767+0000","last_became_active":"2020-08-07T15:13:02.590656+0000","last_became_peered":"2020-08-07T15:13:02.590656+0000","last_unstale":"2020-08-07T15:13:24.327767+0000","last_undegraded":"2020-08-07T15:13:24.327767+0000","last_fullsized":"2020-08-07T15:13:24.327767+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:13.399327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:13.399327+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"288","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.334968+0000","last_change":"2020-08-07T15:11:40.328534+0000","last_active":"2020-08-07T15:13:27.334968+0000","last_peered":"2020-08-07T15:13:27.334968+0000","last_clean":"2020-08-07T15:13:27.334968+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:13:27.334968+0000","last_undegraded":"2020-08-07T15:13:27.334968+0000","last_fullsized":"2020-08-07T15:13:27.334968+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:11:40.328482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:11:40.328482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"287'7104","reported_seq":"7171","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.649971+0000","last_change":"2020-08-07T15:13:02.591551+0000","last_active":"2020-08-07T15:13:27.649971+0000","last_peered":"2020-08-07T15:13:27.649971+0000","last_clean":"2020-08-07T15:13:27.649971+0000","last_became_active":"2020-08-07T15:13:02.591000+0000","last_became_peered":"2020-08-07T15:13:02.591000+0000","last_unstale":"2020-08-07T15:13:27.649971+0000","last_undegraded":"2020-08-07T15:13:27.649971+0000","last_fullsized":"2020-08-07T15:13:27.649971+0000","mapping_epoch":269,"log_start":"287'7096","ondisk_log_start":"287'7096","created":230,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":58195968,"num_objects":888,"num_object_clones":0,"num_object_copies":2664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7104,"num_write_kb":56832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"309","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.342444+0000","last_change":"2020-08-07T15:12:10.585667+0000","last_active":"2020-08-07T15:13:27.342444+0000","last_peered":"2020-08-07T15:13:27.342444+0000","last_clean":"2020-08-07T15:13:27.342444+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:13:27.342444+0000","last_undegraded":"2020-08-07T15:13:27.342444+0000","last_fullsized":"2020-08-07T15:13:27.342444+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:10.585580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:10.585580+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"348","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.345291+0000","last_change":"2020-08-07T15:12:38.850878+0000","last_active":"2020-08-07T15:13:27.345291+0000","last_peered":"2020-08-07T15:13:27.345291+0000","last_clean":"2020-08-07T15:13:27.345291+0000","last_became_active":"2020-08-07T15:12:32.239794+0000","last_became_peered":"2020-08-07T15:12:32.239794+0000","last_unstale":"2020-08-07T15:13:27.345291+0000","last_undegraded":"2020-08-07T15:13:27.345291+0000","last_fullsized":"2020-08-07T15:13:27.345291+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:38.850809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:38.850809+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"288'7536","reported_seq":"7604","reported_epoch":"288","state":"active+clean","last_fresh":"2020-08-07T15:13:31.768589+0000","last_change":"2020-08-07T15:13:21.793789+0000","last_active":"2020-08-07T15:13:31.768589+0000","last_peered":"2020-08-07T15:13:31.768589+0000","last_clean":"2020-08-07T15:13:31.768589+0000","last_became_active":"2020-08-07T15:12:10.464582+0000","last_became_peered":"2020-08-07T15:12:10.464582+0000","last_unstale":"2020-08-07T15:13:31.768589+0000","last_undegraded":"2020-08-07T15:13:31.768589+0000","last_fullsized":"2020-08-07T15:13:31.768589+0000","mapping_epoch":230,"log_start":"288'7528","ondisk_log_start":"288'7528","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"283'6536","last_scrub_stamp":"2020-08-07T15:13:21.793703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:13:21.793703+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61734912,"num_objects":942,"num_object_clones":0,"num_object_copies":2826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7536,"num_write_kb":60288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"303","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.334879+0000","last_change":"2020-08-07T15:12:07.340958+0000","last_active":"2020-08-07T15:13:27.334879+0000","last_peered":"2020-08-07T15:13:27.334879+0000","last_clean":"2020-08-07T15:13:27.334879+0000","last_became_active":"2020-08-07T15:12:06.366967+0000","last_became_peered":"2020-08-07T15:12:06.366967+0000","last_unstale":"2020-08-07T15:13:27.334879+0000","last_undegraded":"2020-08-07T15:13:27.334879+0000","last_fullsized":"2020-08-07T15:13:27.334879+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:07.340855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.340855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"298","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.334614+0000","last_change":"2020-08-07T15:13:24.998985+0000","last_active":"2020-08-07T15:13:27.334614+0000","last_peered":"2020-08-07T15:13:27.334614+0000","last_clean":"2020-08-07T15:13:27.334614+0000","last_became_active":"2020-08-07T15:12:32.246800+0000","last_became_peered":"2020-08-07T15:12:32.246800+0000","last_unstale":"2020-08-07T15:13:27.334614+0000","last_undegraded":"2020-08-07T15:13:27.334614+0000","last_fullsized":"2020-08-07T15:13:27.334614+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:24.998919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:24.998919+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"288'7424","reported_seq":"7489","reported_epoch":"288","state":"active+clean","last_fresh":"2020-08-07T15:13:31.731300+0000","last_change":"2020-08-07T15:12:10.466261+0000","last_active":"2020-08-07T15:13:31.731300+0000","last_peered":"2020-08-07T15:13:31.731300+0000","last_clean":"2020-08-07T15:13:31.731300+0000","last_became_active":"2020-08-07T15:12:10.465583+0000","last_became_peered":"2020-08-07T15:12:10.465583+0000","last_unstale":"2020-08-07T15:13:31.731300+0000","last_undegraded":"2020-08-07T15:13:31.731300+0000","last_fullsized":"2020-08-07T15:13:31.731300+0000","mapping_epoch":230,"log_start":"288'7416","ondisk_log_start":"288'7416","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60817408,"num_objects":928,"num_object_clones":0,"num_object_copies":2784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7424,"num_write_kb":59392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"257","reported_epoch":"286","state":"active+clean","last_fresh":"2020-08-07T15:13:24.316212+0000","last_change":"2020-08-07T15:13:02.591378+0000","last_active":"2020-08-07T15:13:24.316212+0000","last_peered":"2020-08-07T15:13:24.316212+0000","last_clean":"2020-08-07T15:13:24.316212+0000","last_became_active":"2020-08-07T15:13:02.590901+0000","last_became_peered":"2020-08-07T15:13:02.590901+0000","last_unstale":"2020-08-07T15:13:24.316212+0000","last_undegraded":"2020-08-07T15:13:24.316212+0000","last_fullsized":"2020-08-07T15:13:24.316212+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:07.583152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.583152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"383","reported_epoch":"286","state":"active+clean","last_fresh":"2020-08-07T15:13:24.328520+0000","last_change":"2020-08-07T15:12:43.351056+0000","last_active":"2020-08-07T15:13:24.328520+0000","last_peered":"2020-08-07T15:13:24.328520+0000","last_clean":"2020-08-07T15:13:24.328520+0000","last_became_active":"2020-08-07T15:12:32.254842+0000","last_became_peered":"2020-08-07T15:12:32.254842+0000","last_unstale":"2020-08-07T15:13:24.328520+0000","last_undegraded":"2020-08-07T15:13:24.328520+0000","last_fullsized":"2020-08-07T15:13:24.328520+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:43.350971+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:43.350971+0000","last_clean_scrub_stamp":"2020-08-07T15:12:43.350971+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"288'6952","reported_seq":"7018","reported_epoch":"288","state":"active+clean","last_fresh":"2020-08-07T15:13:31.689861+0000","last_change":"2020-08-07T15:12:10.467447+0000","last_active":"2020-08-07T15:13:31.689861+0000","last_peered":"2020-08-07T15:13:31.689861+0000","last_clean":"2020-08-07T15:13:31.689861+0000","last_became_active":"2020-08-07T15:12:10.467000+0000","last_became_peered":"2020-08-07T15:12:10.467000+0000","last_unstale":"2020-08-07T15:13:31.689861+0000","last_undegraded":"2020-08-07T15:13:31.689861+0000","last_fullsized":"2020-08-07T15:13:31.689861+0000","mapping_epoch":230,"log_start":"288'6947","ondisk_log_start":"288'6947","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56950784,"num_objects":869,"num_object_clones":0,"num_object_copies":2607,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6952,"num_write_kb":55616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"275","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.334806+0000","last_change":"2020-08-07T15:13:11.097271+0000","last_active":"2020-08-07T15:13:27.334806+0000","last_peered":"2020-08-07T15:13:27.334806+0000","last_clean":"2020-08-07T15:13:27.334806+0000","last_became_active":"2020-08-07T15:10:46.498230+0000","last_became_peered":"2020-08-07T15:10:46.498230+0000","last_unstale":"2020-08-07T15:13:27.334806+0000","last_undegraded":"2020-08-07T15:13:27.334806+0000","last_fullsized":"2020-08-07T15:13:27.334806+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:11.097214+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:13:11.097214+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"288'7504","reported_seq":"7571","reported_epoch":"288","state":"active+clean","last_fresh":"2020-08-07T15:13:31.661900+0000","last_change":"2020-08-07T15:13:25.974837+0000","last_active":"2020-08-07T15:13:31.661900+0000","last_peered":"2020-08-07T15:13:31.661900+0000","last_clean":"2020-08-07T15:13:31.661900+0000","last_became_active":"2020-08-07T15:12:10.464640+0000","last_became_peered":"2020-08-07T15:12:10.464640+0000","last_unstale":"2020-08-07T15:13:31.661900+0000","last_undegraded":"2020-08-07T15:13:31.661900+0000","last_fullsized":"2020-08-07T15:13:31.661900+0000","mapping_epoch":230,"log_start":"288'7495","ondisk_log_start":"288'7495","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"286'6920","last_scrub_stamp":"2020-08-07T15:13:25.974765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:13:25.974765+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":61472768,"num_objects":938,"num_object_clones":0,"num_object_copies":2814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7504,"num_write_kb":60032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"131","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.333997+0000","last_change":"2020-08-07T15:12:45.243933+0000","last_active":"2020-08-07T15:13:27.333997+0000","last_peered":"2020-08-07T15:13:27.333997+0000","last_clean":"2020-08-07T15:13:27.333997+0000","last_became_active":"2020-08-07T15:12:45.243349+0000","last_became_peered":"2020-08-07T15:12:45.243349+0000","last_unstale":"2020-08-07T15:13:27.333997+0000","last_undegraded":"2020-08-07T15:13:27.333997+0000","last_fullsized":"2020-08-07T15:13:27.333997+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.490655+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.490655+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.e","version":"287'6840","reported_seq":"6904","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:29.866365+0000","last_change":"2020-08-07T15:13:12.351009+0000","last_active":"2020-08-07T15:13:29.866365+0000","last_peered":"2020-08-07T15:13:29.866365+0000","last_clean":"2020-08-07T15:13:29.866365+0000","last_became_active":"2020-08-07T15:12:10.464900+0000","last_became_peered":"2020-08-07T15:12:10.464900+0000","last_unstale":"2020-08-07T15:13:29.866365+0000","last_undegraded":"2020-08-07T15:13:29.866365+0000","last_fullsized":"2020-08-07T15:13:29.866365+0000","mapping_epoch":230,"log_start":"287'6834","ondisk_log_start":"287'6834","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"275'5310","last_scrub_stamp":"2020-08-07T15:13:12.350955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:13:12.350955+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":56033280,"num_objects":855,"num_object_clones":0,"num_object_copies":2565,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6840,"num_write_kb":54720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"93","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.337135+0000","last_change":"2020-08-07T15:13:10.415039+0000","last_active":"2020-08-07T15:13:27.337135+0000","last_peered":"2020-08-07T15:13:27.337135+0000","last_clean":"2020-08-07T15:13:27.337135+0000","last_became_active":"2020-08-07T15:13:02.590018+0000","last_became_peered":"2020-08-07T15:13:02.590018+0000","last_unstale":"2020-08-07T15:13:27.337135+0000","last_undegraded":"2020-08-07T15:13:27.337135+0000","last_fullsized":"2020-08-07T15:13:27.337135+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:10.414954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:10.414954+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"287'7328","reported_seq":"7510","reported_epoch":"287","state":"active+remapped+backfilling","last_fresh":"2020-08-07T15:13:27.727850+0000","last_change":"2020-08-07T15:13:27.281680+0000","last_active":"2020-08-07T15:13:27.727850+0000","last_peered":"2020-08-07T15:13:27.727850+0000","last_clean":"2020-08-07T15:12:55.240424+0000","last_became_active":"2020-08-07T15:12:57.254078+0000","last_became_peered":"2020-08-07T15:12:57.254078+0000","last_unstale":"2020-08-07T15:13:27.727850+0000","last_undegraded":"2020-08-07T15:13:27.727850+0000","last_fullsized":"2020-08-07T15:13:27.727850+0000","mapping_epoch":266,"log_start":"287'7324","ondisk_log_start":"287'7324","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60030976,"num_objects":916,"num_object_clones":0,"num_object_copies":2748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":858,"num_objects_unfound":0,"num_objects_dirty":916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7328,"num_write_kb":58624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":3670016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,0],"avail_no_missing":["6","0","5"],"object_location_counts":[{"shards":"0,5,6","objects":916}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"306","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.345359+0000","last_change":"2020-08-07T15:12:13.680022+0000","last_active":"2020-08-07T15:13:27.345359+0000","last_peered":"2020-08-07T15:13:27.345359+0000","last_clean":"2020-08-07T15:13:27.345359+0000","last_became_active":"2020-08-07T15:12:06.380392+0000","last_became_peered":"2020-08-07T15:12:06.380392+0000","last_unstale":"2020-08-07T15:13:27.345359+0000","last_undegraded":"2020-08-07T15:13:27.345359+0000","last_fullsized":"2020-08-07T15:13:27.345359+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:13.679943+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.679943+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"288'7144","reported_seq":"7212","reported_epoch":"288","state":"active+clean","last_fresh":"2020-08-07T15:13:31.722474+0000","last_change":"2020-08-07T15:13:24.340252+0000","last_active":"2020-08-07T15:13:31.722474+0000","last_peered":"2020-08-07T15:13:31.722474+0000","last_clean":"2020-08-07T15:13:31.722474+0000","last_became_active":"2020-08-07T15:12:10.398332+0000","last_became_peered":"2020-08-07T15:12:10.398332+0000","last_unstale":"2020-08-07T15:13:31.722474+0000","last_undegraded":"2020-08-07T15:13:31.722474+0000","last_fullsized":"2020-08-07T15:13:31.722474+0000","mapping_epoch":230,"log_start":"288'7138","ondisk_log_start":"288'7138","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'6512","last_scrub_stamp":"2020-08-07T15:13:24.340162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:13:24.340162+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":58523648,"num_objects":893,"num_object_clones":0,"num_object_copies":2679,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7144,"num_write_kb":57152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"261","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.337359+0000","last_change":"2020-08-07T15:12:14.241339+0000","last_active":"2020-08-07T15:13:27.337359+0000","last_peered":"2020-08-07T15:13:27.337359+0000","last_clean":"2020-08-07T15:13:27.337359+0000","last_became_active":"2020-08-07T15:10:46.498051+0000","last_became_peered":"2020-08-07T15:10:46.498051+0000","last_unstale":"2020-08-07T15:13:27.337359+0000","last_undegraded":"2020-08-07T15:13:27.337359+0000","last_fullsized":"2020-08-07T15:13:27.337359+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:14.241262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:14.241262+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"288'7424","reported_seq":"7490","reported_epoch":"288","state":"active+clean","last_fresh":"2020-08-07T15:13:31.756068+0000","last_change":"2020-08-07T15:13:02.590816+0000","last_active":"2020-08-07T15:13:31.756068+0000","last_peered":"2020-08-07T15:13:31.756068+0000","last_clean":"2020-08-07T15:13:31.756068+0000","last_became_active":"2020-08-07T15:13:02.590300+0000","last_became_peered":"2020-08-07T15:13:02.590300+0000","last_unstale":"2020-08-07T15:13:31.756068+0000","last_undegraded":"2020-08-07T15:13:31.756068+0000","last_fullsized":"2020-08-07T15:13:31.756068+0000","mapping_epoch":269,"log_start":"288'7416","ondisk_log_start":"288'7416","created":230,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60817408,"num_objects":928,"num_object_clones":0,"num_object_copies":2784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7424,"num_write_kb":59392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3],"acting":[2,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"336","reported_epoch":"286","state":"active+clean","last_fresh":"2020-08-07T15:13:24.328314+0000","last_change":"2020-08-07T15:12:41.445456+0000","last_active":"2020-08-07T15:13:24.328314+0000","last_peered":"2020-08-07T15:13:24.328314+0000","last_clean":"2020-08-07T15:13:24.328314+0000","last_became_active":"2020-08-07T15:12:32.241722+0000","last_became_peered":"2020-08-07T15:12:32.241722+0000","last_unstale":"2020-08-07T15:13:24.328314+0000","last_undegraded":"2020-08-07T15:13:24.328314+0000","last_fullsized":"2020-08-07T15:13:24.328314+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:41.445379+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_clean_scrub_stamp":"2020-08-07T15:12:41.445379+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"288'7236","reported_seq":"7303","reported_epoch":"288","state":"active+clean","last_fresh":"2020-08-07T15:13:31.748530+0000","last_change":"2020-08-07T15:13:23.229211+0000","last_active":"2020-08-07T15:13:31.748530+0000","last_peered":"2020-08-07T15:13:31.748530+0000","last_clean":"2020-08-07T15:13:31.748530+0000","last_became_active":"2020-08-07T15:13:02.591329+0000","last_became_peered":"2020-08-07T15:13:02.591329+0000","last_unstale":"2020-08-07T15:13:31.748530+0000","last_undegraded":"2020-08-07T15:13:31.748530+0000","last_fullsized":"2020-08-07T15:13:31.748530+0000","mapping_epoch":269,"log_start":"288'7231","ondisk_log_start":"288'7231","created":230,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"284'6440","last_scrub_stamp":"2020-08-07T15:13:23.229131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:13:23.229131+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":59277312,"num_objects":905,"num_object_clones":0,"num_object_copies":2715,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7236,"num_write_kb":57888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"50","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.337527+0000","last_change":"2020-08-07T15:13:11.459633+0000","last_active":"2020-08-07T15:13:27.337527+0000","last_peered":"2020-08-07T15:13:27.337527+0000","last_clean":"2020-08-07T15:13:27.337527+0000","last_became_active":"2020-08-07T15:13:02.591018+0000","last_became_peered":"2020-08-07T15:13:02.591018+0000","last_unstale":"2020-08-07T15:13:27.337527+0000","last_undegraded":"2020-08-07T15:13:27.337527+0000","last_fullsized":"2020-08-07T15:13:27.337527+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:11.459555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:11.459555+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.c","version":"288'7664","reported_seq":"7731","reported_epoch":"288","state":"active+clean","last_fresh":"2020-08-07T15:13:31.706226+0000","last_change":"2020-08-07T15:13:23.210964+0000","last_active":"2020-08-07T15:13:31.706226+0000","last_peered":"2020-08-07T15:13:31.706226+0000","last_clean":"2020-08-07T15:13:31.706226+0000","last_became_active":"2020-08-07T15:12:10.464698+0000","last_became_peered":"2020-08-07T15:12:10.464698+0000","last_unstale":"2020-08-07T15:13:31.706226+0000","last_undegraded":"2020-08-07T15:13:31.706226+0000","last_fullsized":"2020-08-07T15:13:31.706226+0000","mapping_epoch":230,"log_start":"288'7658","ondisk_log_start":"288'7658","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"284'6848","last_scrub_stamp":"2020-08-07T15:13:23.210886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:13:23.210886+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":62783488,"num_objects":958,"num_object_clones":0,"num_object_copies":2874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7664,"num_write_kb":61312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"259","reported_epoch":"287","state":"active+clean","last_fresh":"2020-08-07T15:13:27.334903+0000","last_change":"2020-08-07T15:12:44.203274+0000","last_active":"2020-08-07T15:13:27.334903+0000","last_peered":"2020-08-07T15:13:27.334903+0000","last_clean":"2020-08-07T15:13:27.334903+0000","last_became_active":"2020-08-07T15:12:06.367081+0000","last_became_peered":"2020-08-07T15:12:06.367081+0000","last_unstale":"2020-08-07T15:13:27.334903+0000","last_undegraded":"2020-08-07T15:13:27.334903+0000","last_fullsized":"2020-08-07T15:13:27.334903+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:44.203221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:44.203221+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":956792832,"num_objects":14600,"num_object_clones":0,"num_object_copies":43800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":858,"num_objects_unfound":0,"num_objects_dirty":14600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116796,"num_write_kb":934368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":3670016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2921218048,"data_stored":2921218048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":41,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":123,"acting":123,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542232,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1332708,"kb_used_data":283764,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93039132,"statfs":{"total":96636764160,"available":95272071168,"internally_reserved":1073741824,"allocated":290574336,"data_stored":289978181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,7,1,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":9.5359999999999996,"15min":3.774},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":2.294,"5min":856.07100000000003,"15min":856.07100000000003},"last":753.82299999999998},{"interface":"front","average":{"1min":0.77700000000000002,"5min":1.002,"15min":0.97599999999999998},"min":{"1min":0.52300000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.55,"5min":3.4159999999999999,"15min":3.4159999999999999},"last":1.0960000000000001}]},{"osd":1,"last update":"Fri Aug 7 15:13:10 2020","interfaces":[{"interface":"back","average":{"1min":10.148999999999999,"5min":10.148999999999999,"15min":10.148999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":205.73699999999999,"5min":205.73699999999999,"15min":205.73699999999999},"last":5.6219999999999999},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":3.827,"5min":3.827,"15min":3.827},"last":0.67200000000000004}]},{"osd":2,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.321,"15min":1.1910000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.5630000000000002,"5min":5.6619999999999999,"15min":5.6619999999999999},"last":1.7230000000000001},{"interface":"front","average":{"1min":1.081,"5min":1.3280000000000001,"15min":1.1879999999999999},"min":{"1min":0.69299999999999995,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.3769999999999998,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":1.1599999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":30.957999999999998,"5min":21.91,"15min":8.4179999999999993},"min":{"1min":0.68899999999999995,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":656.99099999999999,"5min":917.38699999999994,"15min":917.38699999999994},"last":1.482},{"interface":"front","average":{"1min":1.173,"5min":4.8879999999999999,"15min":2.524},"min":{"1min":0.61599999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":4.0460000000000003,"5min":333.59199999999998,"15min":333.59199999999998},"last":10.589}]},{"osd":4,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":8.8599999999999994,"5min":2.944,"15min":1.8320000000000001},"min":{"1min":0.66000000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":172.012,"5min":172.012,"15min":172.012},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.5509999999999999,"5min":11.529,"15min":4.7169999999999996},"min":{"1min":0.81200000000000006,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":5.4500000000000002,"5min":866.24099999999999,"15min":866.24099999999999},"last":1.512}]},{"osd":5,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":9.3930000000000007,"15min":28.135000000000002},"min":{"1min":0.68400000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":3.536,"5min":678.76099999999997,"15min":831.88099999999997},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.623,"5min":1.831,"15min":1.607},"min":{"1min":0.89000000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":3.593,"5min":7.5350000000000001,"15min":7.5350000000000001},"last":1.6799999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:12:30 2020","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":3.9929999999999999,"15min":2.343},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":4.0810000000000004,"5min":224.14500000000001,"15min":224.14500000000001},"last":1.843},{"interface":"front","average":{"1min":1.671,"5min":1.7649999999999999,"15min":1.593},"min":{"1min":0.92000000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":3.6200000000000001,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":1.2689999999999999}]}]},{"osd":7,"up_from":13,"seq":55834574938,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1425972,"kb_used_data":377388,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92945868,"statfs":{"total":96636764160,"available":95176568832,"internally_reserved":1073741824,"allocated":386445312,"data_stored":385849157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":16.684999999999999,"15min":6.4219999999999997},"min":{"1min":0.61099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.95,"5min":763.39099999999996,"15min":763.39099999999996},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.026,"5min":9.7690000000000001,"15min":4.0179999999999998},"min":{"1min":0.57399999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.9970000000000001,"5min":529.88699999999994,"15min":529.88699999999994},"last":0.70499999999999996}]},{"osd":1,"last update":"Fri Aug 7 15:13:05 2020","interfaces":[{"interface":"back","average":{"1min":23.431000000000001,"5min":23.431000000000001,"15min":23.431000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":472.59699999999998,"5min":472.59699999999998,"15min":472.59699999999998},"last":1.254},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.8650000000000002,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.319}]},{"osd":2,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":1.677,"5min":1.776,"15min":1.3899999999999999},"min":{"1min":0.53400000000000003,"5min":0.502,"15min":0.49299999999999999},"max":{"1min":5.806,"5min":33.131999999999998,"15min":33.131999999999998},"last":1.423},{"interface":"front","average":{"1min":1.667,"5min":1.6080000000000001,"15min":1.3720000000000001},"min":{"1min":0.70099999999999996,"5min":0.58599999999999997,"15min":0.58399999999999996},"max":{"1min":5.3170000000000002,"5min":32.924999999999997,"15min":32.924999999999997},"last":1.339}]},{"osd":3,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":9.4909999999999997,"15min":4.1680000000000001},"min":{"1min":0.68200000000000005,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":5.7030000000000003,"5min":698.19899999999996,"15min":698.19899999999996},"last":5.7030000000000003},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.3,"15min":1.3500000000000001},"min":{"1min":0.69699999999999995,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.2909999999999999,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":1.294}]},{"osd":4,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":6.0369999999999999,"15min":3.0960000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.821,"5min":460.78399999999999,"15min":460.78399999999999},"last":1.282},{"interface":"front","average":{"1min":38.359000000000002,"5min":8.8149999999999995,"15min":4.0510000000000002},"min":{"1min":0.80900000000000005,"5min":0.621,"15min":0.621},"max":{"1min":668.06799999999998,"5min":668.06799999999998,"15min":668.06799999999998},"last":1.526}]},{"osd":5,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.367,"15min":1.3520000000000001},"min":{"1min":0.67900000000000005,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.0489999999999999,"5min":6.2080000000000002,"15min":6.2080000000000002},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.47,"15min":1.6419999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.6160000000000001,"5min":4.407,"15min":5.327},"last":1.544}]},{"osd":6,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":3.5600000000000001,"5min":9.5960000000000001,"15min":4.1580000000000004},"min":{"1min":0.872,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":40.079999999999998,"5min":695.92899999999997,"15min":695.92899999999997},"last":1.5089999999999999},{"interface":"front","average":{"1min":3.6760000000000002,"5min":1.9910000000000001,"15min":11.757},"min":{"1min":0.91900000000000004,"5min":0.88700000000000001,"15min":0.87},"max":{"1min":41.655999999999999,"5min":41.655999999999999,"15min":266.26999999999998},"last":1.5600000000000001}]}]},{"osd":1,"up_from":227,"seq":974957576210,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1331624,"kb_used_data":283040,"kb_used_omap":2,"kb_used_meta":1048573,"kb_avail":93040216,"statfs":{"total":96636764160,"available":95273181184,"internally_reserved":1073741824,"allocated":289832960,"data_stored":289240338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":1073739245},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":27.861000000000001,"5min":27.861000000000001,"15min":27.861000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":540.92499999999995,"5min":540.92499999999995,"15min":540.92499999999995},"last":0.48299999999999998},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":3.7069999999999999,"5min":3.7069999999999999,"15min":3.7069999999999999},"last":3.2789999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":4.093},{"interface":"front","average":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":3.758,"5min":3.758,"15min":3.758},"last":3.3679999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":13.909000000000001,"5min":13.909000000000001,"15min":13.909000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":250.995,"5min":250.995,"15min":250.995},"last":2.528},{"interface":"front","average":{"1min":2.04,"5min":2.04,"15min":2.04},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":6.6580000000000004,"5min":6.6580000000000004,"15min":6.6580000000000004},"last":3.3889999999999998}]},{"osd":4,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":4.1239999999999997},{"interface":"front","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":2.4750000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":2.5030000000000001},{"interface":"front","average":{"1min":1.498,"5min":1.498,"15min":1.498},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.6720000000000002,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":4.149}]},{"osd":6,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.1019999999999999},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":9.4299999999999997,"5min":9.4299999999999997,"15min":9.4299999999999997},"last":3.3159999999999998},{"interface":"front","average":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":3.4199999999999999,"5min":3.4199999999999999,"15min":3.4199999999999999},"last":4.2539999999999996}]},{"osd":7,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":40.520000000000003,"5min":40.520000000000003,"15min":40.520000000000003},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":778.745,"5min":778.745,"15min":778.745},"last":3.3420000000000001},{"interface":"front","average":{"1min":24.100000000000001,"5min":24.100000000000001,"15min":24.100000000000001},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":442.62799999999999,"5min":442.62799999999999,"15min":442.62799999999999},"last":3.4209999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574938,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1281132,"kb_used_data":232548,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93090708,"statfs":{"total":96636764160,"available":95324884992,"internally_reserved":1073741824,"allocated":238129152,"data_stored":237532997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:13:07 2020","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.468,"5min":2.468,"15min":2.468},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.5550000000000002,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.5389999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.0620000000000001,"15min":0.97199999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":7.3470000000000004,"5min":7.3470000000000004,"15min":7.3470000000000004},"last":0.78100000000000003},{"interface":"front","average":{"1min":6.2839999999999998,"5min":4.665,"15min":23.381},"min":{"1min":0.47399999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":75.622,"5min":226.911,"15min":606.18499999999995},"last":1.512}]},{"osd":3,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":28.079000000000001,"5min":6.6920000000000002,"15min":2.8540000000000001},"min":{"1min":0.60999999999999999,"5min":0.54100000000000004,"15min":0.52300000000000002},"max":{"1min":431.95299999999997,"5min":431.95299999999997,"15min":431.95299999999997},"last":0.871},{"interface":"front","average":{"1min":1.425,"5min":1.46,"15min":1.2649999999999999},"min":{"1min":0.66400000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":5.3780000000000001,"5min":11.19,"15min":11.19},"last":1.109}]},{"osd":4,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.302,"15min":1.2110000000000001},"min":{"1min":0.57399999999999995,"5min":0.501,"15min":0.501},"max":{"1min":1.819,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.304,"5min":1.448,"15min":1.321},"min":{"1min":0.71099999999999997,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.8849999999999998,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":1.141}]},{"osd":5,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.6200000000000001,"5min":14.598000000000001,"15min":5.6840000000000002},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":7.3799999999999999,"5min":539.31100000000004,"15min":539.31100000000004},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.49,"15min":1.234},"min":{"1min":0.81399999999999995,"5min":0.752,"15min":0.65900000000000003},"max":{"1min":2.9420000000000002,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":0.92400000000000004}]},{"osd":6,"last update":"Fri Aug 7 15:12:34 2020","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.4399999999999999,"15min":6.2220000000000004},"min":{"1min":0.80200000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.8610000000000002,"5min":4.1310000000000002,"15min":124.738},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.5549999999999999,"15min":12.952999999999999},"min":{"1min":0.73499999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":5.274,"5min":5.274,"15min":294.94},"last":1.079}]},{"osd":7,"last update":"Fri Aug 7 15:12:32 2020","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.4790000000000001,"15min":1.55},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.79300000000000004},"max":{"1min":1.8620000000000001,"5min":4.7169999999999996,"15min":6.468},"last":1.4450000000000001},{"interface":"front","average":{"1min":66.063000000000002,"5min":23.518000000000001,"15min":8.7629999999999999},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":971.12699999999995,"5min":971.12699999999995,"15min":971.12699999999995},"last":1.472}]}]},{"osd":2,"up_from":13,"seq":55834574938,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1520544,"kb_used_data":471708,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92851296,"statfs":{"total":96636764160,"available":95079727104,"internally_reserved":1073741824,"allocated":483028992,"data_stored":482578770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:28 2020","interfaces":[{"interface":"back","average":{"1min":18.065999999999999,"5min":11.218,"15min":4.4960000000000004},"min":{"1min":0.441,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":377.32900000000001,"5min":501.53800000000001,"15min":501.53800000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.0760000000000001,"5min":6.649,"15min":2.8860000000000001},"min":{"1min":0.46800000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":3.0139999999999998,"5min":501.36000000000001,"15min":501.36000000000001},"last":0.83499999999999996}]},{"osd":1,"last update":"Fri Aug 7 15:13:07 2020","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":5.7869999999999999,"5min":5.7869999999999999,"15min":5.7869999999999999},"last":2.2559999999999998},{"interface":"front","average":{"1min":24.699999999999999,"5min":24.699999999999999,"15min":24.699999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":465.69200000000001,"5min":465.69200000000001,"15min":465.69200000000001},"last":5.5369999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:13:28 2020","interfaces":[{"interface":"back","average":{"1min":3.847,"5min":1.696,"15min":1.3580000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":29.111000000000001,"5min":29.111000000000001,"15min":29.111000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":3.3849999999999998,"5min":15.673999999999999,"15min":6.2530000000000001},"min":{"1min":0.627,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":29.788,"5min":501.75599999999997,"15min":501.75599999999997},"last":2.4940000000000002}]},{"osd":4,"last update":"Fri Aug 7 15:13:29 2020","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.397,"15min":1.2869999999999999},"min":{"1min":0.73499999999999999,"5min":0.623,"15min":0.58099999999999996},"max":{"1min":2.5489999999999999,"5min":7.9569999999999999,"15min":19.783000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":44.813000000000002,"5min":10.178000000000001,"15min":4.46},"min":{"1min":0.84299999999999997,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":954.97900000000004,"5min":954.97900000000004,"15min":954.97900000000004},"last":1.4319999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:13:28 2020","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.3420000000000001,"15min":1.294},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.992,"5min":5.2220000000000004,"15min":5.2220000000000004},"last":1.5900000000000001},{"interface":"front","average":{"1min":45.744999999999997,"5min":24.559000000000001,"15min":23.603000000000002},"min":{"1min":0.57099999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":971.51400000000001,"5min":971.51400000000001,"15min":971.51400000000001},"last":6.2489999999999997}]},{"osd":6,"last update":"Fri Aug 7 15:12:37 2020","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":9.9480000000000004,"15min":4.1299999999999999},"min":{"1min":0.875,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.9450000000000001,"5min":821.91899999999998,"15min":821.91899999999998},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.3959999999999999,"5min":9.9580000000000002,"15min":4.2000000000000002},"min":{"1min":0.70199999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.9540000000000002,"5min":751.70000000000005,"15min":751.70000000000005},"last":1.804}]},{"osd":7,"last update":"Fri Aug 7 15:13:28 2020","interfaces":[{"interface":"back","average":{"1min":43.765999999999998,"5min":20.567,"15min":8.1370000000000005},"min":{"1min":0.89200000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":930.55499999999995,"5min":930.55499999999995,"15min":930.55499999999995},"last":1.8759999999999999},{"interface":"front","average":{"1min":1.8560000000000001,"5min":1.79,"15min":1.724},"min":{"1min":0.98999999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":5.6950000000000003,"5min":8.0779999999999994,"15min":35.646999999999998},"last":2.5819999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574938,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1648312,"kb_used_data":599476,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92723528,"statfs":{"total":96636764160,"available":94948892672,"internally_reserved":1073741824,"allocated":613863424,"data_stored":613413202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,3,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.226,"15min":1.244},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":4.6580000000000004,"5min":8.6020000000000003,"15min":8.6020000000000003},"last":1.827},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.2050000000000001,"15min":1.4930000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":4.6349999999999998,"5min":4.9119999999999999,"15min":13.067},"last":1.851}]},{"osd":1,"last update":"Fri Aug 7 15:13:08 2020","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.639,"15min":1.639},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":4.6890000000000001,"5min":4.6890000000000001,"15min":4.6890000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.841,"5min":1.841,"15min":1.841},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":5.0330000000000004,"5min":5.0330000000000004,"15min":5.0330000000000004},"last":1.7909999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.365,"15min":1.22},"min":{"1min":0.65900000000000003,"5min":0.56999999999999995,"15min":0.56899999999999995},"max":{"1min":2.8300000000000001,"5min":4.9580000000000002,"15min":4.9580000000000002},"last":1.879},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.9690000000000001,"15min":1.7789999999999999},"min":{"1min":0.64500000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":5.3090000000000002,"5min":51.530000000000001,"15min":51.530000000000001},"last":1.605}]},{"osd":4,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":3.831,"5min":17.792999999999999,"15min":7.0620000000000003},"min":{"1min":0.54500000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":56.688000000000002,"5min":646.26700000000005,"15min":646.26700000000005},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.421,"5min":15.494999999999999,"15min":26.565000000000001},"min":{"1min":0.68100000000000005,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":3.2109999999999999,"5min":844.10500000000002,"15min":844.10500000000002},"last":2.1509999999999998}]},{"osd":5,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":14.452999999999999,"5min":8.5429999999999993,"15min":3.859},"min":{"1min":0.95499999999999996,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":244.08099999999999,"5min":403.88400000000001,"15min":403.88400000000001},"last":2.2320000000000002},{"interface":"front","average":{"1min":1.78,"5min":1.8979999999999999,"15min":1.8120000000000001},"min":{"1min":0.98499999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":4.5960000000000001,"5min":8.9120000000000008,"15min":8.9120000000000008},"last":2.2570000000000001}]},{"osd":6,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.7969999999999999,"5min":1.706,"15min":1.5900000000000001},"min":{"1min":0.74099999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":5.2729999999999997,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.907,"5min":13.411,"15min":5.7949999999999999},"min":{"1min":1.032,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":5.3460000000000001,"5min":688.15999999999997,"15min":688.15999999999997},"last":1.5600000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:12:33 2020","interfaces":[{"interface":"back","average":{"1min":1.7450000000000001,"5min":1.5960000000000001,"15min":1.9730000000000001},"min":{"1min":0.85599999999999998,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":4.8099999999999996,"5min":6.8209999999999997,"15min":6.8209999999999997},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.6559999999999999,"15min":1.8700000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.6099999999999999,"5min":8.6790000000000003,"15min":8.6790000000000003},"last":1.1759999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574938,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1348736,"kb_used_data":299900,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93023104,"statfs":{"total":96636764160,"available":95255658496,"internally_reserved":1073741824,"allocated":307097600,"data_stored":306647378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:30 2020","interfaces":[{"interface":"back","average":{"1min":3.0019999999999998,"5min":20.065999999999999,"15min":7.4569999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":37.307000000000002,"5min":829.15099999999995,"15min":829.15099999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":3.242,"5min":2.0179999999999998,"15min":1.272},"min":{"1min":0.499,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":42.095999999999997,"5min":48.274000000000001,"15min":48.274000000000001},"last":1.0589999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:13:08 2020","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.1200000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:13:30 2020","interfaces":[{"interface":"back","average":{"1min":3.718,"5min":2.3460000000000001,"15min":1.752},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.52100000000000002},"max":{"1min":37.365000000000002,"5min":49.878999999999998,"15min":49.878999999999998},"last":5.5499999999999998},{"interface":"front","average":{"1min":12.497999999999999,"5min":4.1970000000000001,"15min":2.4260000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":146.34399999999999,"5min":146.34399999999999,"15min":146.34399999999999},"last":10.507999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:13:30 2020","interfaces":[{"interface":"back","average":{"1min":3.4969999999999999,"5min":15.436999999999999,"15min":6.891},"min":{"1min":0.63100000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":37.582000000000001,"5min":638.92100000000005,"15min":638.92100000000005},"last":1.339},{"interface":"front","average":{"1min":3.4500000000000002,"5min":1.754,"15min":1.4399999999999999},"min":{"1min":0.72299999999999998,"5min":0.68500000000000005,"15min":0.66700000000000004},"max":{"1min":37.956000000000003,"5min":37.956000000000003,"15min":37.956000000000003},"last":2.077}]},{"osd":5,"last update":"Fri Aug 7 15:13:30 2020","interfaces":[{"interface":"back","average":{"1min":3.6819999999999999,"5min":19.954000000000001,"15min":10.747},"min":{"1min":0.69999999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":43.692,"5min":862.97299999999996,"15min":990.71600000000001},"last":1.3660000000000001},{"interface":"front","average":{"1min":40.039000000000001,"5min":18.669,"15min":7.3150000000000004},"min":{"1min":0.58899999999999997,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":625.35000000000002,"5min":780.00699999999995,"15min":780.00699999999995},"last":1.1619999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:12:34 2020","interfaces":[{"interface":"back","average":{"1min":3.8239999999999998,"5min":1.9870000000000001,"15min":1.855},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":48.575000000000003,"5min":48.575000000000003,"15min":48.575000000000003},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.109,"5min":1.333,"15min":13.183},"min":{"1min":0.64600000000000002,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.847,"5min":6.7729999999999997,"15min":389.14600000000002},"last":0.84299999999999997}]},{"osd":7,"last update":"Fri Aug 7 15:13:30 2020","interfaces":[{"interface":"back","average":{"1min":3.8620000000000001,"5min":2.4239999999999999,"15min":2.544},"min":{"1min":1.1319999999999999,"5min":0.62,"15min":0.62},"max":{"1min":38.979999999999997,"5min":48.75,"15min":252.798},"last":1.698},{"interface":"front","average":{"1min":3.7989999999999999,"5min":6.46,"15min":6.1189999999999998},"min":{"1min":0.91300000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":38.899999999999999,"5min":326.327,"15min":501.93000000000001},"last":1.1779999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574937,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1344996,"kb_used_data":296412,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93026844,"statfs":{"total":96636764160,"available":95259488256,"internally_reserved":1073741824,"allocated":303525888,"data_stored":302929733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:27 2020","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":1.4770000000000001,"15min":1.204},"min":{"1min":0.45700000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.7070000000000001,"5min":43.292999999999999,"15min":43.292999999999999},"last":0.5},{"interface":"front","average":{"1min":1.03,"5min":4.3860000000000001,"15min":2.0259999999999998},"min":{"1min":0.47399999999999998,"5min":0.438,"15min":0.438},"max":{"1min":2.6059999999999999,"5min":329.11399999999998,"15min":329.11399999999998},"last":0.748}]},{"osd":1,"last update":"Fri Aug 7 15:13:11 2020","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":4.4039999999999999,"5min":4.4039999999999999,"15min":4.4039999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":4.0810000000000004,"5min":4.0810000000000004,"15min":4.0810000000000004},"last":1.016}]},{"osd":2,"last update":"Fri Aug 7 15:13:27 2020","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.6100000000000001,"15min":6.1319999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.4129999999999998,"5min":43.923000000000002,"15min":168.56899999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.615,"15min":20.503},"min":{"1min":0.70499999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.5750000000000002,"5min":43.076999999999998,"15min":676.27099999999996},"last":0.78900000000000003}]},{"osd":3,"last update":"Fri Aug 7 15:13:27 2020","interfaces":[{"interface":"back","average":{"1min":4.1559999999999997,"5min":2.2570000000000001,"15min":1.5229999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":47.371000000000002,"5min":47.371000000000002,"15min":47.371000000000002},"last":0.53300000000000003},{"interface":"front","average":{"1min":4.0990000000000002,"5min":2.2349999999999999,"15min":4.3440000000000003},"min":{"1min":0.72899999999999998,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":48.338999999999999,"5min":48.338999999999999,"15min":712.98699999999997},"last":1.0469999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:13:27 2020","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":12.773,"15min":34.875999999999998},"min":{"1min":0.68400000000000005,"5min":0.54800000000000004,"15min":0.53400000000000003},"max":{"1min":1.9139999999999999,"5min":971.67999999999995,"15min":971.67999999999995},"last":1.0880000000000001},{"interface":"front","average":{"1min":16.669,"5min":4.3479999999999999,"15min":2.3100000000000001},"min":{"1min":0.78300000000000003,"5min":0.61899999999999999,"15min":0.57099999999999995},"max":{"1min":261.08499999999998,"5min":261.08499999999998,"15min":261.08499999999998},"last":1.2709999999999999}]},{"osd":6,"last update":"Fri Aug 7 15:13:27 2020","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":28.48,"15min":39.819000000000003},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.4940000000000002,"5min":966.60900000000004,"15min":966.60900000000004},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":6.8049999999999997,"15min":17.957999999999998},"min":{"1min":0.71199999999999997,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.9450000000000001,"5min":501.79000000000002,"15min":517.26700000000005},"last":1.1259999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:13:27 2020","interfaces":[{"interface":"back","average":{"1min":13.619999999999999,"5min":3.8769999999999998,"15min":2.1800000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":208.03399999999999,"5min":208.03399999999999,"15min":208.03399999999999},"last":1.103},{"interface":"front","average":{"1min":1.4470000000000001,"5min":12.840999999999999,"15min":5.3140000000000001},"min":{"1min":0.89400000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.7069999999999999,"5min":515.09100000000001,"15min":515.09100000000001},"last":0.99199999999999999}]}]}]}} 2020-08-07T15:13:33.148 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:33.134+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:33.176 INFO:tasks.thrashosds.thrasher:Setting 1.23 to [7, 6, 1, 3, 5, 2] 2020-08-07T15:13:33.179 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '1.23', '7', '6', '1', '3', '5', '2'] 2020-08-07T15:13:33.181 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pg-upmap-items 1.23 7 6 1 3 5 2 2020-08-07T15:13:33.183 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:13:33.183 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:13:33.183 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:13:33.184 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:13:33.204 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:13:33.204 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:13:33.205 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:13:33.205 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:13:33.205 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:13:33.211 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:33.211 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:13:33.211 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.212 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:120752 6.e 6:79a2b884:::benchmark_data_smithi074_63843_object15093:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e288)", 2020-08-07T15:13:33.212 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:33.170649+0000", 2020-08-07T15:13:33.212 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.027519355999999998, 2020-08-07T15:13:33.212 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.027552950999999999, 2020-08-07T15:13:33.213 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:33.213 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:13:33.213 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:33.213 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:33.215 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:33.215 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 120752 2020-08-07T15:13:33.215 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.215 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:33.216 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.216 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:33.216 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:33.170649+0000", 2020-08-07T15:13:33.217 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:33.217 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.217 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.217 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:33.226 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:33.170649+0000", 2020-08-07T15:13:33.227 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4052999999999999e-05 2020-08-07T15:13:33.227 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.227 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.227 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:33.228 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:33.170703+0000", 2020-08-07T15:13:33.228 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7859999999999999e-06 2020-08-07T15:13:33.228 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.228 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.229 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:33.229 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:33.170710+0000", 2020-08-07T15:13:33.229 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3039e-05 2020-08-07T15:13:33.231 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.232 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.232 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:33.232 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:33.170723+0000", 2020-08-07T15:13:33.232 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3219999999999998e-06 2020-08-07T15:13:33.232 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.233 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.233 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:33.233 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:33.170729+0000", 2020-08-07T15:13:33.233 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00051151400000000002 2020-08-07T15:13:33.234 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.234 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.234 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:33.234 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:33.171241+0000", 2020-08-07T15:13:33.234 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2070000000000004e-05 2020-08-07T15:13:33.235 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.235 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.235 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:33.235 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:33.171333+0000", 2020-08-07T15:13:33.235 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016290400000000001 2020-08-07T15:13:33.236 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.236 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.236 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:33.236 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:33.171496+0000", 2020-08-07T15:13:33.237 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002406306 2020-08-07T15:13:33.237 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.237 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.237 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:33.238 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:33.173902+0000", 2020-08-07T15:13:33.238 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4042999999999999e-05 2020-08-07T15:13:33.238 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.238 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.238 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:33.239 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:33.173916+0000", 2020-08-07T15:13:33.239 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003267037 2020-08-07T15:13:33.239 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:33.239 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:33.240 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:33.240 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:33.241 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:13:33.242 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 1 2020-08-07T15:13:33.242 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:33.244 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:13:33.245 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:33.242+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:33.274 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 81 16 120898 120882 11.6577 11.7031 0.00283829 0.0107185 2020-08-07T15:13:33.346 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:33.345+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:33.447 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:33.447 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:13:33.447 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:13:33.448 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:13:33.448 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:33.448 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:33.446+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:33.461 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:13:33.559 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:33.547+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:33.650 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:33.649+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:33.707 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:33.707 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:13:33.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:13:33.708 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:13:33.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.708 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "MOSDScrubReserve(2.6 GRANT e232)", 2020-08-07T15:13:33.709 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:11.909673+0000", 2020-08-07T15:13:33.709 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 81.792174541999998, 2020-08-07T15:13:33.709 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.36313456, 2020-08-07T15:13:33.709 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:33.709 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:33.717 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:33.717 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.722 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:33.722 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:11.909673+0000", 2020-08-07T15:13:33.722 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:33.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.723 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:33.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:11.909673+0000", 2020-08-07T15:13:33.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0409000000000001e-05 2020-08-07T15:13:33.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.724 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.724 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:33.724 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:11.909693+0000", 2020-08-07T15:13:33.737 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8580000000000002e-06 2020-08-07T15:13:33.738 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.738 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.738 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:33.738 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:11.909697+0000", 2020-08-07T15:13:33.738 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.36297494299999999 2020-08-07T15:13:33.739 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.739 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.739 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:33.739 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272672+0000", 2020-08-07T15:13:33.739 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4490000000000001e-06 2020-08-07T15:13:33.740 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.740 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.740 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:33.740 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272676+0000", 2020-08-07T15:13:33.740 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0395999999999994e-05 2020-08-07T15:13:33.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.750 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.762 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:33.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272757+0000", 2020-08-07T15:13:33.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6416e-05 2020-08-07T15:13:33.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.766 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:33.766 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272773+0000", 2020-08-07T15:13:33.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4088999999999998e-05 2020-08-07T15:13:33.766 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:33.767 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272807+0000", 2020-08-07T15:13:33.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.36313456 2020-08-07T15:13:33.767 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:33.767 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:33.778 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:33.782 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.782 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.782 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:7272 6.e 6:74263cb8:::benchmark_data_smithi074_63843_object908:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e234)", 2020-08-07T15:13:33.783 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:16.977784+0000", 2020-08-07T15:13:33.783 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 76.724062794000005, 2020-08-07T15:13:33.783 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31581518600000003, 2020-08-07T15:13:33.783 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:33.783 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:33.784 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:33.784 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:33.784 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:33.784 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7272 2020-08-07T15:13:33.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.785 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:33.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:33.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977784+0000", 2020-08-07T15:13:33.785 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:33.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.786 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:33.786 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977784+0000", 2020-08-07T15:13:33.786 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6940999999999998e-05 2020-08-07T15:13:33.787 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.787 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:33.787 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977811+0000", 2020-08-07T15:13:33.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1870000000000001e-06 2020-08-07T15:13:33.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.788 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:33.788 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977815+0000", 2020-08-07T15:13:33.788 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1249999999999998e-06 2020-08-07T15:13:33.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.789 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.789 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:33.789 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977821+0000", 2020-08-07T15:13:33.842 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8739999999999999e-06 2020-08-07T15:13:33.842 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.842 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.843 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:33.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977824+0000", 2020-08-07T15:13:33.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001740655 2020-08-07T15:13:33.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.844 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:33.844 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.979564+0000", 2020-08-07T15:13:33.844 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6272000000000003e-05 2020-08-07T15:13:33.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:33.845 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.979650+0000", 2020-08-07T15:13:33.845 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012951600000000001 2020-08-07T15:13:33.845 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.866 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.866 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:33.866 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.979780+0000", 2020-08-07T15:13:33.867 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00085408999999999995 2020-08-07T15:13:33.867 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.867 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.867 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:33.868 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.980634+0000", 2020-08-07T15:13:33.868 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8099999999999999e-07 2020-08-07T15:13:33.868 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.868 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.868 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:33.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.980634+0000", 2020-08-07T15:13:33.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5664000000000003e-05 2020-08-07T15:13:33.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:33.887 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.980670+0000", 2020-08-07T15:13:33.887 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.104650663 2020-08-07T15:13:33.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:33.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.085321+0000", 2020-08-07T15:13:33.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20821616000000001 2020-08-07T15:13:33.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:33.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.293537+0000", 2020-08-07T15:13:33.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1900000000000001e-07 2020-08-07T15:13:33.901 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.901 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.901 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:33.902 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.293538+0000", 2020-08-07T15:13:33.902 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1939000000000004e-05 2020-08-07T15:13:33.902 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.902 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.902 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:33.903 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.293600+0000", 2020-08-07T15:13:33.903 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31581518600000003 2020-08-07T15:13:33.903 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:33.903 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:33.904 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:33.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.904 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:8359 6.1 6:844c6d67:::benchmark_data_smithi074_63843_object1044:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e235)", 2020-08-07T15:13:33.904 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:17.720675+0000", 2020-08-07T15:13:33.905 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 75.981172637, 2020-08-07T15:13:33.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.309231859, 2020-08-07T15:13:33.905 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:33.905 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:33.905 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:33.957 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:33.957 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:33.958 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 8359 2020-08-07T15:13:33.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.958 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:33.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:33.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720675+0000", 2020-08-07T15:13:33.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:33.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:33.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720675+0000", 2020-08-07T15:13:33.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.884e-05 2020-08-07T15:13:33.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:33.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720723+0000", 2020-08-07T15:13:33.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2989999999999996e-06 2020-08-07T15:13:33.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.969 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:33.970 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720729+0000", 2020-08-07T15:13:33.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0288999999999999e-05 2020-08-07T15:13:33.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.970 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:33.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720739+0000", 2020-08-07T15:13:33.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7849999999999999e-06 2020-08-07T15:13:33.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:33.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720744+0000", 2020-08-07T15:13:33.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001862991 2020-08-07T15:13:33.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:33.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:33.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.722607+0000", 2020-08-07T15:13:33.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000100839 2020-08-07T15:13:33.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:33.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:34.014 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.722708+0000", 2020-08-07T15:13:34.014 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017526600000000001 2020-08-07T15:13:34.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.015 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.015 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:13:34.015 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.722883+0000", 2020-08-07T15:13:34.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001504619 2020-08-07T15:13:34.016 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.016 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:34.016 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.724387+0000", 2020-08-07T15:13:34.017 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4499999999999996e-07 2020-08-07T15:13:34.017 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:34.017 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.724388+0000", 2020-08-07T15:13:34.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8369000000000003e-05 2020-08-07T15:13:34.018 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.018 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.018 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:34.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.724447+0000", 2020-08-07T15:13:34.019 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00097551199999999999 2020-08-07T15:13:34.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.019 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:34.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.725422+0000", 2020-08-07T15:13:34.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30441636900000002 2020-08-07T15:13:34.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:34.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029838+0000", 2020-08-07T15:13:34.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7699999999999992e-07 2020-08-07T15:13:34.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:34.034 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029839+0000", 2020-08-07T15:13:34.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6958999999999994e-05 2020-08-07T15:13:34.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:34.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029906+0000", 2020-08-07T15:13:34.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.309231859 2020-08-07T15:13:34.036 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:34.036 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:34.036 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:34.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.037 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:8360 6.1 6:844c6d67:::benchmark_data_smithi074_63843_object1044:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e235)", 2020-08-07T15:13:34.037 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:17.721396+0000", 2020-08-07T15:13:34.062 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 75.980450973999993, 2020-08-07T15:13:34.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31055011399999999, 2020-08-07T15:13:34.063 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:34.063 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:34.063 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:34.064 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:34.064 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:34.064 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 8360 2020-08-07T15:13:34.064 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.064 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:34.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.065 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:34.065 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721396+0000", 2020-08-07T15:13:34.065 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:34.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.090 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:34.090 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721396+0000", 2020-08-07T15:13:34.091 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8222e-05 2020-08-07T15:13:34.091 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.091 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.092 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:34.092 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721444+0000", 2020-08-07T15:13:34.092 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.297e-06 2020-08-07T15:13:34.093 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.093 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.114 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:34.114 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721451+0000", 2020-08-07T15:13:34.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.172e-05 2020-08-07T15:13:34.116 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.116 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.116 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:34.117 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721462+0000", 2020-08-07T15:13:34.117 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7639999999999996e-06 2020-08-07T15:13:34.118 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.118 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.118 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:34.119 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721467+0000", 2020-08-07T15:13:34.119 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023051790000000001 2020-08-07T15:13:34.119 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.120 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.120 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:34.120 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.723772+0000", 2020-08-07T15:13:34.120 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1051000000000006e-05 2020-08-07T15:13:34.121 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.121 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.121 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:34.142 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.723863+0000", 2020-08-07T15:13:34.143 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017915699999999999 2020-08-07T15:13:34.147 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.147 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.148 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:13:34.148 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.724043+0000", 2020-08-07T15:13:34.148 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001191646 2020-08-07T15:13:34.148 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.149 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.149 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:34.149 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.725234+0000", 2020-08-07T15:13:34.149 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9100000000000002e-07 2020-08-07T15:13:34.149 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.150 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.150 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:34.150 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.725235+0000", 2020-08-07T15:13:34.150 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0677999999999997e-05 2020-08-07T15:13:34.150 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.151 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:34.151 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.725296+0000", 2020-08-07T15:13:34.151 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001252663 2020-08-07T15:13:34.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:34.152 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.726548+0000", 2020-08-07T15:13:34.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30531503100000001 2020-08-07T15:13:34.153 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.153 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.153 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:34.153 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.031864+0000", 2020-08-07T15:13:34.153 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.057e-06 2020-08-07T15:13:34.193 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.194 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.194 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:34.194 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.031865+0000", 2020-08-07T15:13:34.194 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1757999999999999e-05 2020-08-07T15:13:34.195 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.195 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.195 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:34.195 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.031946+0000", 2020-08-07T15:13:34.195 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31055011399999999 2020-08-07T15:13:34.196 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:34.196 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:34.196 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:34.196 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.196 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.197 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:8359 6.1 e235/230 ondisk, result = 0)", 2020-08-07T15:13:34.197 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:17.726764+0000", 2020-08-07T15:13:34.197 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 75.975082958000002, 2020-08-07T15:13:34.197 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30314883199999998, 2020-08-07T15:13:34.198 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:34.198 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:34.198 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:34.222 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.222 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:34.222 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.726764+0000", 2020-08-07T15:13:34.223 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:34.223 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.223 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.223 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:34.238 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.726764+0000", 2020-08-07T15:13:34.238 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1386000000000001e-05 2020-08-07T15:13:34.238 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.238 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.239 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:34.239 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.726796+0000", 2020-08-07T15:13:34.239 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7989999999999999e-06 2020-08-07T15:13:34.239 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.239 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.240 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:34.240 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.726801+0000", 2020-08-07T15:13:34.240 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30298438700000002 2020-08-07T15:13:34.240 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.240 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.241 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:34.241 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029786+0000", 2020-08-07T15:13:34.241 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5040000000000002e-06 2020-08-07T15:13:34.251 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.251 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.251 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:34.252 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029791+0000", 2020-08-07T15:13:34.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0737000000000001e-05 2020-08-07T15:13:34.252 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.252 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.252 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:34.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029822+0000", 2020-08-07T15:13:34.253 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9359999999999999e-06 2020-08-07T15:13:34.253 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.253 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:34.254 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029830+0000", 2020-08-07T15:13:34.254 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3083000000000004e-05 2020-08-07T15:13:34.254 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.254 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.254 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:34.255 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029913+0000", 2020-08-07T15:13:34.255 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30314883199999998 2020-08-07T15:13:34.255 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:34.255 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:34.256 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:34.256 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.256 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.256 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:10224 6.e 6:77f719e6:::benchmark_data_smithi074_63843_object1277:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e236)", 2020-08-07T15:13:34.256 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:19.038704+0000", 2020-08-07T15:13:34.257 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 74.663143656000003, 2020-08-07T15:13:34.257 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32955690900000001, 2020-08-07T15:13:34.257 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:34.257 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:34.257 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:34.287 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:34.288 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:34.288 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 10224 2020-08-07T15:13:34.289 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.289 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:34.289 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.290 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:34.290 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038704+0000", 2020-08-07T15:13:34.290 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:34.291 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.291 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.291 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:34.292 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038704+0000", 2020-08-07T15:13:34.330 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6672000000000001e-05 2020-08-07T15:13:34.330 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.330 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.330 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:34.331 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038730+0000", 2020-08-07T15:13:34.331 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.27e-06 2020-08-07T15:13:34.331 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.331 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.331 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:34.332 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038733+0000", 2020-08-07T15:13:34.332 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4030000000000004e-06 2020-08-07T15:13:34.332 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.332 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.332 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:34.333 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038740+0000", 2020-08-07T15:13:34.333 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.712e-06 2020-08-07T15:13:34.333 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.333 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.333 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:34.354 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038743+0000", 2020-08-07T15:13:34.354 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0025767279999999999 2020-08-07T15:13:34.354 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.355 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.355 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:34.355 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.041319+0000", 2020-08-07T15:13:34.355 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4727e-05 2020-08-07T15:13:34.355 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.355 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.356 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:34.356 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.041374+0000", 2020-08-07T15:13:34.356 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9403999999999999e-05 2020-08-07T15:13:34.356 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.356 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.357 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:34.357 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.041473+0000", 2020-08-07T15:13:34.357 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010041290000000001 2020-08-07T15:13:34.357 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:34.380 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.042478+0000", 2020-08-07T15:13:34.380 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0100000000000003e-07 2020-08-07T15:13:34.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:34.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.042478+0000", 2020-08-07T15:13:34.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7923999999999995e-05 2020-08-07T15:13:34.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.382 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.382 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:34.382 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.042546+0000", 2020-08-07T15:13:34.398 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0046169310000000003 2020-08-07T15:13:34.398 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.398 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.398 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:34.399 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.047163+0000", 2020-08-07T15:13:34.399 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.321023999 2020-08-07T15:13:34.399 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.399 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:34.400 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368187+0000", 2020-08-07T15:13:34.418 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.048e-06 2020-08-07T15:13:34.418 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.418 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.419 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:34.419 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368188+0000", 2020-08-07T15:13:34.419 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2061e-05 2020-08-07T15:13:34.419 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.419 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.420 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:34.420 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368260+0000", 2020-08-07T15:13:34.420 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32955690900000001 2020-08-07T15:13:34.420 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:34.420 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:34.421 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:34.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.432 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.432 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:10224 6.e e236/230 ondisk, result = 0)", 2020-08-07T15:13:34.433 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:19.047929+0000", 2020-08-07T15:13:34.433 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 74.653918055999995, 2020-08-07T15:13:34.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32041415400000001, 2020-08-07T15:13:34.433 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:34.433 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:34.434 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:34.434 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.434 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:34.434 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.047929+0000", 2020-08-07T15:13:34.434 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:34.435 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.435 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.435 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:34.435 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.047929+0000", 2020-08-07T15:13:34.435 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9706999999999999e-05 2020-08-07T15:13:34.436 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.436 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.436 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:34.436 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.047959+0000", 2020-08-07T15:13:34.436 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9100000000000004e-06 2020-08-07T15:13:34.437 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.437 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.437 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:34.437 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.047964+0000", 2020-08-07T15:13:34.437 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31873185399999998 2020-08-07T15:13:34.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:34.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.366696+0000", 2020-08-07T15:13:34.467 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2970000000000003e-06 2020-08-07T15:13:34.468 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.468 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.468 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:34.469 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.366701+0000", 2020-08-07T15:13:34.469 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001472522 2020-08-07T15:13:34.469 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.469 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.470 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:34.470 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368173+0000", 2020-08-07T15:13:34.470 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5949999999999998e-06 2020-08-07T15:13:34.471 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.471 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.471 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:34.471 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368179+0000", 2020-08-07T15:13:34.471 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016426900000000001 2020-08-07T15:13:34.472 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.472 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.472 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:34.472 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368343+0000", 2020-08-07T15:13:34.472 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32041415400000001 2020-08-07T15:13:34.473 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:34.473 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:34.473 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:34.473 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.494 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:18480 6.1 6:8a2c1569:::benchmark_data_smithi074_63843_object2309:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e240)", 2020-08-07T15:13:34.494 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.367850+0000", 2020-08-07T15:13:34.495 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 69.333997646, 2020-08-07T15:13:34.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41222172800000001, 2020-08-07T15:13:34.495 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:34.496 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:34.496 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:34.496 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:34.496 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:34.497 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18480 2020-08-07T15:13:34.497 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.497 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:34.497 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.497 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:34.497 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367850+0000", 2020-08-07T15:13:34.498 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:34.498 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.498 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.498 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:34.499 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367850+0000", 2020-08-07T15:13:34.499 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6174999999999999e-05 2020-08-07T15:13:34.499 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.499 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.499 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:34.499 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367876+0000", 2020-08-07T15:13:34.500 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.061e-06 2020-08-07T15:13:34.500 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.500 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.500 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:34.500 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367879+0000", 2020-08-07T15:13:34.501 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7899999999999996e-06 2020-08-07T15:13:34.501 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.501 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.501 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:34.501 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367885+0000", 2020-08-07T15:13:34.516 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6589999999999999e-06 2020-08-07T15:13:34.517 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.517 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.517 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:34.517 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367887+0000", 2020-08-07T15:13:34.517 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001651275 2020-08-07T15:13:34.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.518 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:34.518 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.369538+0000", 2020-08-07T15:13:34.518 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7794000000000006e-05 2020-08-07T15:13:34.519 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.519 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.519 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:34.519 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.369636+0000", 2020-08-07T15:13:34.519 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017274 2020-08-07T15:13:34.519 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.520 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:13:34.520 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.369809+0000", 2020-08-07T15:13:34.520 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010733769999999999 2020-08-07T15:13:34.520 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.521 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.521 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:34.521 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.370882+0000", 2020-08-07T15:13:34.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0100000000000003e-07 2020-08-07T15:13:34.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.534 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.534 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:34.534 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.370883+0000", 2020-08-07T15:13:34.535 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6001000000000001e-05 2020-08-07T15:13:34.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:34.535 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.370939+0000", 2020-08-07T15:13:34.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074658490000000001 2020-08-07T15:13:34.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.536 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:34.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.378405+0000", 2020-08-07T15:13:34.539 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40160148899999998 2020-08-07T15:13:34.539 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.539 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.539 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:34.540 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.780007+0000", 2020-08-07T15:13:34.540 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9700000000000005e-07 2020-08-07T15:13:34.540 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.540 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.540 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:34.541 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.780008+0000", 2020-08-07T15:13:34.541 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3720000000000007e-05 2020-08-07T15:13:34.541 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.541 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:34.551 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.780071+0000", 2020-08-07T15:13:34.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41222172800000001 2020-08-07T15:13:34.551 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:34.551 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:34.552 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:34.552 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.553 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:18480 6.1 e240/230 ondisk, result = 0)", 2020-08-07T15:13:34.553 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.375696+0000", 2020-08-07T15:13:34.553 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 69.326151042999996, 2020-08-07T15:13:34.553 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.404380817, 2020-08-07T15:13:34.561 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:34.562 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:34.562 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:34.562 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.562 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:34.562 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.375696+0000", 2020-08-07T15:13:34.563 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:34.563 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.563 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.563 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:34.563 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.375696+0000", 2020-08-07T15:13:34.564 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0922e-05 2020-08-07T15:13:34.564 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.564 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:34.564 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.375727+0000", 2020-08-07T15:13:34.565 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2900000000000002e-06 2020-08-07T15:13:34.565 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.565 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.565 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:34.565 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.375732+0000", 2020-08-07T15:13:34.584 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.403431708 2020-08-07T15:13:34.584 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.585 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.585 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:34.585 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.779164+0000", 2020-08-07T15:13:34.585 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5200000000000002e-06 2020-08-07T15:13:34.585 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.586 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.586 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:34.586 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.779168+0000", 2020-08-07T15:13:34.586 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00082257200000000004 2020-08-07T15:13:34.586 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.587 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.587 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:34.587 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.779990+0000", 2020-08-07T15:13:34.587 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1599999999999998e-06 2020-08-07T15:13:34.587 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.588 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:34.588 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.779998+0000", 2020-08-07T15:13:34.588 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8645000000000003e-05 2020-08-07T15:13:34.588 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.589 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:34.589 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.780077+0000", 2020-08-07T15:13:34.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.404380817 2020-08-07T15:13:34.589 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:34.613 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:34.623 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:34.624 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.625 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.625 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:18560 6.e 6:7c5f8c5e:::benchmark_data_smithi074_63843_object2319:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e240)", 2020-08-07T15:13:34.625 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.407761+0000", 2020-08-07T15:13:34.625 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 69.294085780000003, 2020-08-07T15:13:34.625 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31539320100000001, 2020-08-07T15:13:34.626 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:34.626 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:34.626 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:34.626 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:34.626 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:34.627 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18560 2020-08-07T15:13:34.627 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.627 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:34.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.627 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:34.628 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407761+0000", 2020-08-07T15:13:34.628 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:34.628 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.628 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.628 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:34.628 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407761+0000", 2020-08-07T15:13:34.629 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0897000000000003e-05 2020-08-07T15:13:34.629 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.629 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.629 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:34.654 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407822+0000", 2020-08-07T15:13:34.655 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8390000000000001e-06 2020-08-07T15:13:34.656 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.656 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.657 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:34.657 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407829+0000", 2020-08-07T15:13:34.657 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3609e-05 2020-08-07T15:13:34.658 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.658 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.658 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:34.659 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407843+0000", 2020-08-07T15:13:34.659 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3169999999999998e-06 2020-08-07T15:13:34.659 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.660 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.660 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:34.660 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407849+0000", 2020-08-07T15:13:34.661 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002504886 2020-08-07T15:13:34.661 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.661 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:34.677 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.410354+0000", 2020-08-07T15:13:34.678 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7235e-05 2020-08-07T15:13:34.678 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.678 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.678 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:34.678 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.410451+0000", 2020-08-07T15:13:34.679 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016856999999999999 2020-08-07T15:13:34.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.679 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:34.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.410620+0000", 2020-08-07T15:13:34.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026816259999999999 2020-08-07T15:13:34.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.680 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:34.680 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.413301+0000", 2020-08-07T15:13:34.681 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0020516309999999999 2020-08-07T15:13:34.681 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.681 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:34.681 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.415353+0000", 2020-08-07T15:13:34.686 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30769196399999998 2020-08-07T15:13:34.686 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.687 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.687 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:34.694 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.723045+0000", 2020-08-07T15:13:34.694 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1600000000000001e-07 2020-08-07T15:13:34.694 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.695 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:34.695 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.723046+0000", 2020-08-07T15:13:34.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1449000000000001e-05 2020-08-07T15:13:34.695 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.696 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:34.696 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.723107+0000", 2020-08-07T15:13:34.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7462000000000001e-05 2020-08-07T15:13:34.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:34.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.723155+0000", 2020-08-07T15:13:34.697 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31539320100000001 2020-08-07T15:13:34.703 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:34.703 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:34.704 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:34.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.704 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:18608 6.4 6:2dddf5bf:::benchmark_data_smithi074_63843_object2325:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e240)", 2020-08-07T15:13:34.704 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.438031+0000", 2020-08-07T15:13:34.705 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 69.263816383999995, 2020-08-07T15:13:34.705 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28378579100000001, 2020-08-07T15:13:34.705 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:34.705 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:34.706 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:34.706 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:34.706 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:34.706 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18608 2020-08-07T15:13:34.706 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.706 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:34.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.707 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:34.707 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438031+0000", 2020-08-07T15:13:34.707 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:34.707 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:34.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438031+0000", 2020-08-07T15:13:34.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0028999999999998e-05 2020-08-07T15:13:34.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.721 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:34.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438081+0000", 2020-08-07T15:13:34.722 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7289999999999997e-06 2020-08-07T15:13:34.729 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.730 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.730 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:34.731 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438087+0000", 2020-08-07T15:13:34.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0959e-05 2020-08-07T15:13:34.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.731 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:34.732 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438098+0000", 2020-08-07T15:13:34.732 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0329999999999999e-06 2020-08-07T15:13:34.732 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.732 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:34.733 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438103+0000", 2020-08-07T15:13:34.733 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001770911 2020-08-07T15:13:34.733 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.733 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:34.741 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.439873+0000", 2020-08-07T15:13:34.741 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3503000000000005e-05 2020-08-07T15:13:34.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.741 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.742 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:34.742 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.439937+0000", 2020-08-07T15:13:34.742 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000110179 2020-08-07T15:13:34.742 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.742 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.743 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,5", 2020-08-07T15:13:34.743 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.440047+0000", 2020-08-07T15:13:34.743 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033910759999999998 2020-08-07T15:13:34.743 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.743 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:34.744 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443438+0000", 2020-08-07T15:13:34.744 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2877999999999999e-05 2020-08-07T15:13:34.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:34.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443471+0000", 2020-08-07T15:13:34.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27823471799999999 2020-08-07T15:13:34.745 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.745 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:34.760 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721706+0000", 2020-08-07T15:13:34.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4300000000000003e-07 2020-08-07T15:13:34.761 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.761 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:34.761 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721706+0000", 2020-08-07T15:13:34.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2877000000000001e-05 2020-08-07T15:13:34.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.762 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.762 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:34.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721769+0000", 2020-08-07T15:13:34.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7355999999999999e-05 2020-08-07T15:13:34.763 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.763 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:34.763 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721817+0000", 2020-08-07T15:13:34.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28378579100000001 2020-08-07T15:13:34.764 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:34.764 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:34.764 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:34.764 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.765 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:18608 6.4 e240/230 ondisk, result = 0)", 2020-08-07T15:13:34.765 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.443879+0000", 2020-08-07T15:13:34.765 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 69.257968145999996, 2020-08-07T15:13:34.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.277941838, 2020-08-07T15:13:34.798 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:34.798 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:34.798 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:34.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:34.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443879+0000", 2020-08-07T15:13:34.799 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:34.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:34.800 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443879+0000", 2020-08-07T15:13:34.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3692000000000002e-05 2020-08-07T15:13:34.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.801 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.801 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:34.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443913+0000", 2020-08-07T15:13:34.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.02e-06 2020-08-07T15:13:34.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.813 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:34.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443919+0000", 2020-08-07T15:13:34.813 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27734076499999999 2020-08-07T15:13:34.814 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.814 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.814 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:34.814 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721260+0000", 2020-08-07T15:13:34.814 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.78e-06 2020-08-07T15:13:34.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.815 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:34.815 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721264+0000", 2020-08-07T15:13:34.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00042663999999999999 2020-08-07T15:13:34.816 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:34.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721691+0000", 2020-08-07T15:13:34.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1509999999999998e-06 2020-08-07T15:13:34.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.817 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:34.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721698+0000", 2020-08-07T15:13:34.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012279000000000001 2020-08-07T15:13:34.848 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.849 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.849 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:34.849 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721821+0000", 2020-08-07T15:13:34.850 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.277941838 2020-08-07T15:13:34.850 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:34.850 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:34.851 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:34.851 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.851 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.851 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:29256 6.1 6:8599f289:::benchmark_data_smithi074_63843_object3656:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e244)", 2020-08-07T15:13:34.851 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:31.539560+0000", 2020-08-07T15:13:34.852 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 62.162287642999999, 2020-08-07T15:13:34.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28482166599999997, 2020-08-07T15:13:34.862 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:34.862 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:34.863 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:34.863 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:34.863 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:34.863 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 29256 2020-08-07T15:13:34.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.864 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:34.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.865 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:34.865 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.539560+0000", 2020-08-07T15:13:34.865 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:34.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.866 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:34.866 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.539560+0000", 2020-08-07T15:13:34.866 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5965e-05 2020-08-07T15:13:34.866 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.866 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.867 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:34.867 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.539586+0000", 2020-08-07T15:13:34.867 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1719999999999999e-06 2020-08-07T15:13:34.867 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.867 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.868 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:34.868 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.539589+0000", 2020-08-07T15:13:34.868 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9700000000000005e-06 2020-08-07T15:13:34.868 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.868 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.868 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:34.869 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.539598+0000", 2020-08-07T15:13:34.869 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9999999999999998e-06 2020-08-07T15:13:34.869 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.869 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:34.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.539602+0000", 2020-08-07T15:13:34.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011500200000000001 2020-08-07T15:13:34.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:34.887 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.540752+0000", 2020-08-07T15:13:34.887 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000107188 2020-08-07T15:13:34.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:34.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.540859+0000", 2020-08-07T15:13:34.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011065600000000001 2020-08-07T15:13:34.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:13:34.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.540970+0000", 2020-08-07T15:13:34.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0018553440000000001 2020-08-07T15:13:34.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.890 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:34.890 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.542825+0000", 2020-08-07T15:13:34.890 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.048e-06 2020-08-07T15:13:34.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.891 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:34.891 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.542826+0000", 2020-08-07T15:13:34.891 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2224000000000005e-05 2020-08-07T15:13:34.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.891 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.892 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:34.892 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.542898+0000", 2020-08-07T15:13:34.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.029129893 2020-08-07T15:13:34.892 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.892 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.893 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:34.893 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.572028+0000", 2020-08-07T15:13:34.893 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.252307642 2020-08-07T15:13:34.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:34.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.824336+0000", 2020-08-07T15:13:34.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9899999999999999e-07 2020-08-07T15:13:34.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.908 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:34.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.824337+0000", 2020-08-07T15:13:34.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4644999999999997e-05 2020-08-07T15:13:34.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:34.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.824381+0000", 2020-08-07T15:13:34.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28482166599999997 2020-08-07T15:13:34.909 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:34.917 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:34.917 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:34.918 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.918 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:32560 6.e 6:77482cb7:::benchmark_data_smithi074_63843_object4069:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e246)", 2020-08-07T15:13:34.918 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:33.538197+0000", 2020-08-07T15:13:34.918 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 60.163650369000003, 2020-08-07T15:13:34.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28301590900000001, 2020-08-07T15:13:34.919 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:34.919 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:34.919 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:34.919 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:34.920 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:34.920 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 32560 2020-08-07T15:13:34.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.920 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:34.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:34.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.538197+0000", 2020-08-07T15:13:34.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:34.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:34.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.538197+0000", 2020-08-07T15:13:34.934 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6372000000000003e-05 2020-08-07T15:13:34.934 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.934 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.934 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:34.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.538243+0000", 2020-08-07T15:13:34.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9100000000000004e-06 2020-08-07T15:13:34.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:34.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.538248+0000", 2020-08-07T15:13:34.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0688e-05 2020-08-07T15:13:34.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:34.936 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.538259+0000", 2020-08-07T15:13:34.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0649999999999998e-06 2020-08-07T15:13:34.937 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:34.949 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.538264+0000", 2020-08-07T15:13:34.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003496645 2020-08-07T15:13:34.950 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.950 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.950 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:34.950 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.541760+0000", 2020-08-07T15:13:34.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019299399999999999 2020-08-07T15:13:34.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:34.951 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.541953+0000", 2020-08-07T15:13:34.952 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018635200000000001 2020-08-07T15:13:34.952 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.952 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.952 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:34.952 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.542140+0000", 2020-08-07T15:13:34.953 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001795608 2020-08-07T15:13:34.953 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.953 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.953 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:34.953 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.543935+0000", 2020-08-07T15:13:34.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2399999999999998e-07 2020-08-07T15:13:34.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:34.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.543936+0000", 2020-08-07T15:13:34.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7708999999999999e-05 2020-08-07T15:13:34.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.978 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:34.978 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.543994+0000", 2020-08-07T15:13:34.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032640360000000001 2020-08-07T15:13:34.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.979 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.979 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:34.979 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.547258+0000", 2020-08-07T15:13:34.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27388560899999997 2020-08-07T15:13:34.979 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:34.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.821143+0000", 2020-08-07T15:13:34.991 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0410000000000001e-06 2020-08-07T15:13:34.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:34.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.821144+0000", 2020-08-07T15:13:34.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8355999999999994e-05 2020-08-07T15:13:34.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:34.993 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:34.993 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:34.993 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.821213+0000", 2020-08-07T15:13:34.993 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28301590900000001 2020-08-07T15:13:34.993 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:35.001 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:35.002 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:35.002 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.002 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.002 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:48584 6.e 6:72419a6c:::benchmark_data_smithi074_63843_object6072:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e256)", 2020-08-07T15:13:35.002 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:43.685874+0000", 2020-08-07T15:13:35.003 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 50.015973352000003, 2020-08-07T15:13:35.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.336964296, 2020-08-07T15:13:35.003 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:35.003 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:35.003 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:35.004 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:35.004 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:35.004 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48584 2020-08-07T15:13:35.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.010 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:35.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.010 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:35.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.685874+0000", 2020-08-07T15:13:35.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:35.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.023 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:35.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.685874+0000", 2020-08-07T15:13:35.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9709000000000006e-05 2020-08-07T15:13:35.023 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:35.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.685944+0000", 2020-08-07T15:13:35.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7719999999999994e-06 2020-08-07T15:13:35.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:35.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.685951+0000", 2020-08-07T15:13:35.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5424e-05 2020-08-07T15:13:35.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:35.033 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.685967+0000", 2020-08-07T15:13:35.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3549999999999999e-06 2020-08-07T15:13:35.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.034 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.034 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:35.034 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.685974+0000", 2020-08-07T15:13:35.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2768000000000001e-05 2020-08-07T15:13:35.035 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:35.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.686017+0000", 2020-08-07T15:13:35.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6916999999999994e-05 2020-08-07T15:13:35.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:35.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.686114+0000", 2020-08-07T15:13:35.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018684600000000001 2020-08-07T15:13:35.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.037 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.037 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:35.037 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.686301+0000", 2020-08-07T15:13:35.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0050644929999999998 2020-08-07T15:13:35.050 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.050 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.050 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:35.051 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.691365+0000", 2020-08-07T15:13:35.051 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.142e-06 2020-08-07T15:13:35.051 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.051 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.052 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:35.052 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.691366+0000", 2020-08-07T15:13:35.052 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2626999999999994e-05 2020-08-07T15:13:35.052 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.052 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.052 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:35.053 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.691439+0000", 2020-08-07T15:13:35.053 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0093850860000000008 2020-08-07T15:13:35.062 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.062 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.062 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:35.063 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.700824+0000", 2020-08-07T15:13:35.063 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32193047299999999 2020-08-07T15:13:35.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.063 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:35.065 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022754+0000", 2020-08-07T15:13:35.065 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.147e-06 2020-08-07T15:13:35.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.065 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:35.066 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022756+0000", 2020-08-07T15:13:35.066 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2536999999999999e-05 2020-08-07T15:13:35.066 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.066 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.066 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:35.067 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022838+0000", 2020-08-07T15:13:35.067 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.336964296 2020-08-07T15:13:35.067 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:35.067 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:35.067 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:35.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.068 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.068 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:48584 6.e e256/230 ondisk, result = 0)", 2020-08-07T15:13:35.068 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:43.688495+0000", 2020-08-07T15:13:35.068 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 50.013351706999998, 2020-08-07T15:13:35.069 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33435100299999998, 2020-08-07T15:13:35.069 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:35.069 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:35.069 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:35.069 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:35.088 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.688495+0000", 2020-08-07T15:13:35.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:35.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:35.089 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.688495+0000", 2020-08-07T15:13:35.089 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2947999999999997e-05 2020-08-07T15:13:35.089 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.089 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.089 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:35.090 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.688528+0000", 2020-08-07T15:13:35.090 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4979999999999997e-06 2020-08-07T15:13:35.090 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.090 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.090 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:35.091 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.688534+0000", 2020-08-07T15:13:35.091 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33414495300000002 2020-08-07T15:13:35.091 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.091 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.091 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:35.092 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022679+0000", 2020-08-07T15:13:35.092 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6669999999999999e-06 2020-08-07T15:13:35.092 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.092 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.092 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:35.093 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022686+0000", 2020-08-07T15:13:35.093 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1094999999999998e-05 2020-08-07T15:13:35.093 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.093 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.093 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:35.106 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022737+0000", 2020-08-07T15:13:35.106 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2629999999999992e-06 2020-08-07T15:13:35.107 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.107 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.107 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:35.107 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022746+0000", 2020-08-07T15:13:35.108 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000100579 2020-08-07T15:13:35.108 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.108 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.109 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:35.109 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022846+0000", 2020-08-07T15:13:35.109 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33435100299999998 2020-08-07T15:13:35.109 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:35.109 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:35.110 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:35.110 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.110 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.110 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:61896 6.e 6:75870b8d:::benchmark_data_smithi074_63843_object7736:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e263)", 2020-08-07T15:13:35.110 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:52.341370+0000", 2020-08-07T15:13:35.111 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 41.360477563000003, 2020-08-07T15:13:35.111 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.281931072, 2020-08-07T15:13:35.111 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:35.111 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:35.111 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:35.112 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:35.112 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:35.112 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 61896 2020-08-07T15:13:35.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.112 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:35.113 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:35.113 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.341370+0000", 2020-08-07T15:13:35.113 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:35.113 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.119 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.119 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:35.128 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.341370+0000", 2020-08-07T15:13:35.128 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7135000000000003e-05 2020-08-07T15:13:35.128 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.128 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.133 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:35.134 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.341407+0000", 2020-08-07T15:13:35.134 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0359999999999997e-06 2020-08-07T15:13:35.134 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.135 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.135 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:35.135 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.341412+0000", 2020-08-07T15:13:35.135 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0572e-05 2020-08-07T15:13:35.135 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.136 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.136 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:35.136 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.341422+0000", 2020-08-07T15:13:35.136 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.279e-06 2020-08-07T15:13:35.136 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.136 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.137 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:35.137 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.341427+0000", 2020-08-07T15:13:35.137 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024648579999999999 2020-08-07T15:13:35.137 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.137 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.138 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:35.138 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.343891+0000", 2020-08-07T15:13:35.138 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1310000000000005e-05 2020-08-07T15:13:35.138 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.138 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.139 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:35.139 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.343983+0000", 2020-08-07T15:13:35.139 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000175744 2020-08-07T15:13:35.146 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.146 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.147 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:35.147 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.344159+0000", 2020-08-07T15:13:35.147 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005415202 2020-08-07T15:13:35.147 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.147 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.148 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:35.148 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.349574+0000", 2020-08-07T15:13:35.148 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8599999999999998e-07 2020-08-07T15:13:35.148 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.148 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.149 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:35.149 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.349574+0000", 2020-08-07T15:13:35.149 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9826999999999994e-05 2020-08-07T15:13:35.149 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.149 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.155 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:35.155 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.349644+0000", 2020-08-07T15:13:35.155 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.05151385 2020-08-07T15:13:35.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.156 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:35.156 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.401158+0000", 2020-08-07T15:13:35.156 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22207370600000001 2020-08-07T15:13:35.157 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.157 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.157 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:35.157 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.623232+0000", 2020-08-07T15:13:35.157 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1999999999999998e-07 2020-08-07T15:13:35.158 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.158 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.158 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:35.213 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.623233+0000", 2020-08-07T15:13:35.213 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7947000000000005e-05 2020-08-07T15:13:35.213 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.213 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.214 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:35.214 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.623301+0000", 2020-08-07T15:13:35.214 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.281931072 2020-08-07T15:13:35.214 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:35.214 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:35.215 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:35.215 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.215 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.215 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:74192 6.e 6:7ab1a263:::benchmark_data_smithi074_63843_object9273:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e269)", 2020-08-07T15:13:35.215 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:01.584372+0000", 2020-08-07T15:13:35.216 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 32.117475102999997, 2020-08-07T15:13:35.216 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96302370800000003, 2020-08-07T15:13:35.216 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:35.216 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:35.216 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:35.217 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:35.217 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:35.217 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 74192 2020-08-07T15:13:35.217 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.217 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:35.233 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.234 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:35.234 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.584372+0000", 2020-08-07T15:13:35.234 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:35.234 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.234 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.235 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:35.235 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.584372+0000", 2020-08-07T15:13:35.235 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2189e-05 2020-08-07T15:13:35.235 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.235 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.236 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:35.236 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.584404+0000", 2020-08-07T15:13:35.236 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9859999999999999e-06 2020-08-07T15:13:35.236 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.236 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.237 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:35.237 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.584407+0000", 2020-08-07T15:13:35.237 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9560000000000002e-06 2020-08-07T15:13:35.237 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.237 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.250 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:35.251 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.584413+0000", 2020-08-07T15:13:35.251 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6790000000000001e-06 2020-08-07T15:13:35.251 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.254 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.255 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:35.255 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.584416+0000", 2020-08-07T15:13:35.255 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7514000000000001e-05 2020-08-07T15:13:35.255 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.255 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.256 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:35.256 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.584443+0000", 2020-08-07T15:13:35.256 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8560000000000001e-06 2020-08-07T15:13:35.256 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.256 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.257 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:35.257 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.584450+0000", 2020-08-07T15:13:35.257 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0091272760000000001 2020-08-07T15:13:35.257 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.257 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.270 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:35.270 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.593578+0000", 2020-08-07T15:13:35.270 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00055358000000000002 2020-08-07T15:13:35.271 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.271 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:35.271 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.594131+0000", 2020-08-07T15:13:35.271 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000182704 2020-08-07T15:13:35.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:35.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.594314+0000", 2020-08-07T15:13:35.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0025760470000000001 2020-08-07T15:13:35.273 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.273 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:35.273 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.596890+0000", 2020-08-07T15:13:35.273 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00028334900000000001 2020-08-07T15:13:35.298 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.298 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.299 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:35.306 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.597173+0000", 2020-08-07T15:13:35.306 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.95006429999999997 2020-08-07T15:13:35.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.321 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.322 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:35.322 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.547238+0000", 2020-08-07T15:13:35.322 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0430000000000002e-06 2020-08-07T15:13:35.323 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.323 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.323 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:35.324 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.547240+0000", 2020-08-07T15:13:35.324 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6477000000000006e-05 2020-08-07T15:13:35.324 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.325 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.325 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:35.325 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.547326+0000", 2020-08-07T15:13:35.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9752000000000006e-05 2020-08-07T15:13:35.339 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.340 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:35.340 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.547396+0000", 2020-08-07T15:13:35.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96302370800000003 2020-08-07T15:13:35.340 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:35.341 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:35.341 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:35.341 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.341 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:91031 6.e 6:703cf75c:::benchmark_data_smithi074_63843_object11378:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e275)", 2020-08-07T15:13:35.352 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:13.429600+0000", 2020-08-07T15:13:35.353 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 20.272246802000002, 2020-08-07T15:13:35.353 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27365956600000002, 2020-08-07T15:13:35.353 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:35.354 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:35.354 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:35.354 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:35.354 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:35.354 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 91031 2020-08-07T15:13:35.355 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.355 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:35.355 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.355 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:35.355 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429600+0000", 2020-08-07T15:13:35.356 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:35.356 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.356 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.356 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:35.356 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429600+0000", 2020-08-07T15:13:35.357 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4419000000000002e-05 2020-08-07T15:13:35.357 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.357 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.357 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:35.357 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429655+0000", 2020-08-07T15:13:35.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3889999999999994e-06 2020-08-07T15:13:35.365 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.376 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:35.376 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429664+0000", 2020-08-07T15:13:35.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2275e-05 2020-08-07T15:13:35.377 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:35.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429676+0000", 2020-08-07T15:13:35.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1419999999999998e-06 2020-08-07T15:13:35.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.393 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.394 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:35.394 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429683+0000", 2020-08-07T15:13:35.394 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00078575699999999999 2020-08-07T15:13:35.394 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.394 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.395 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:35.395 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.430468+0000", 2020-08-07T15:13:35.395 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0164000000000002e-05 2020-08-07T15:13:35.395 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.395 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.396 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:35.396 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.430529+0000", 2020-08-07T15:13:35.396 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000156971 2020-08-07T15:13:35.396 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.406 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.406 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:35.406 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.430685+0000", 2020-08-07T15:13:35.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042982089999999999 2020-08-07T15:13:35.407 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.407 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.407 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:35.415 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.434984+0000", 2020-08-07T15:13:35.415 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1900000000000003e-07 2020-08-07T15:13:35.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.416 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:35.416 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.434984+0000", 2020-08-07T15:13:35.416 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5408000000000002e-05 2020-08-07T15:13:35.416 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.417 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.417 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:35.417 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.435040+0000", 2020-08-07T15:13:35.417 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045225700000000001 2020-08-07T15:13:35.417 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.418 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:35.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.439562+0000", 2020-08-07T15:13:35.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.26362828900000002 2020-08-07T15:13:35.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:35.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.703190+0000", 2020-08-07T15:13:35.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.046e-06 2020-08-07T15:13:35.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.430 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.430 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:35.430 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.703192+0000", 2020-08-07T15:13:35.430 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8408e-05 2020-08-07T15:13:35.430 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.431 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:35.431 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.703260+0000", 2020-08-07T15:13:35.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27365956600000002 2020-08-07T15:13:35.431 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:35.431 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:35.432 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:35.432 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.432 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.432 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:91032 6.e 6:703cf75c:::benchmark_data_smithi074_63843_object11378:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e275)", 2020-08-07T15:13:35.432 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:13.429721+0000", 2020-08-07T15:13:35.433 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 20.272126655000001, 2020-08-07T15:13:35.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29695800300000003, 2020-08-07T15:13:35.433 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:35.433 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:35.433 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:35.441 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:35.441 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:35.442 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 91032 2020-08-07T15:13:35.442 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.442 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:35.442 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.443 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:35.443 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429721+0000", 2020-08-07T15:13:35.443 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:35.443 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.443 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.444 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:35.445 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429721+0000", 2020-08-07T15:13:35.445 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3047000000000002e-05 2020-08-07T15:13:35.445 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.445 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.445 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:35.446 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429774+0000", 2020-08-07T15:13:35.446 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.339e-06 2020-08-07T15:13:35.446 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.446 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.446 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:35.447 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429779+0000", 2020-08-07T15:13:35.447 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0742e-05 2020-08-07T15:13:35.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:35.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429790+0000", 2020-08-07T15:13:35.449 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1860000000000002e-06 2020-08-07T15:13:35.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.451 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:35.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429794+0000", 2020-08-07T15:13:35.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001193794 2020-08-07T15:13:35.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.453 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:35.453 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.430988+0000", 2020-08-07T15:13:35.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7213000000000006e-05 2020-08-07T15:13:35.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.454 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:35.454 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.431085+0000", 2020-08-07T15:13:35.454 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000129875 2020-08-07T15:13:35.454 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.454 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:35.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.431215+0000", 2020-08-07T15:13:35.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038327349999999999 2020-08-07T15:13:35.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:35.456 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.435047+0000", 2020-08-07T15:13:35.456 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3499999999999996e-07 2020-08-07T15:13:35.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:35.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.435048+0000", 2020-08-07T15:13:35.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9911999999999999e-05 2020-08-07T15:13:35.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:35.458 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.435098+0000", 2020-08-07T15:13:35.458 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0046481059999999999 2020-08-07T15:13:35.458 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.458 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.458 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:35.459 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.439746+0000", 2020-08-07T15:13:35.459 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.286860114 2020-08-07T15:13:35.459 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:35.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.726606+0000", 2020-08-07T15:13:35.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0800000000000003e-07 2020-08-07T15:13:35.469 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.469 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.470 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:35.470 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.726607+0000", 2020-08-07T15:13:35.470 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1396999999999994e-05 2020-08-07T15:13:35.471 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.471 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.471 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:35.471 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.726679+0000", 2020-08-07T15:13:35.471 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29695800300000003 2020-08-07T15:13:35.472 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:35.472 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:35.472 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:35.472 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:35.472 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:35.473 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:35.473 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:13:35.474 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:13:35.476 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 82 16 122483 122467 11.6665 12.3828 0.050206 0.0107108 2020-08-07T15:13:35.477 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 83 16 124015 123999 11.6701 11.9688 0.00936217 0.0107065 2020-08-07T15:13:35.477 INFO:teuthology.orchestra.run.smithi074.stderr:set 1.23 pg_upmap_items mapping to [7->6,1->3,5->2] 2020-08-07T15:13:35.478 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:35.473+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:35.491 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:13:35.586 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:35.584+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:35.688 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:35.687+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:35.729 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:35.730 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:13:35.730 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.730 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5481.0:124620 6.9 e290/269 6:9dcbe73e:::benchmark_data_smithi074_63843_object15577:head v 290'7724, mlcod=290'7718)", 2020-08-07T15:13:35.730 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:35.712558+0000", 2020-08-07T15:13:35.731 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.004867405, 2020-08-07T15:13:35.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0048933350000000004, 2020-08-07T15:13:35.731 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:35.731 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:35.731 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:35.732 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:35.732 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.712558+0000", 2020-08-07T15:13:35.732 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:35.732 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.733 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:35.733 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.712558+0000", 2020-08-07T15:13:35.733 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9481000000000003e-05 2020-08-07T15:13:35.743 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.743 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:35.743 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.712607+0000", 2020-08-07T15:13:35.744 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7389999999999996e-06 2020-08-07T15:13:35.754 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.755 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:35.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.712613+0000", 2020-08-07T15:13:35.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001221106 2020-08-07T15:13:35.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.757 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:35.757 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.713834+0000", 2020-08-07T15:13:35.757 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.118e-06 2020-08-07T15:13:35.757 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.758 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.758 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:35.758 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.713839+0000", 2020-08-07T15:13:35.758 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000345383 2020-08-07T15:13:35.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.759 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:35.759 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.714184+0000", 2020-08-07T15:13:35.759 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2932999999999999e-05 2020-08-07T15:13:35.760 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.760 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.760 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:35.760 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.714207+0000", 2020-08-07T15:13:35.760 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0016497599999999999 2020-08-07T15:13:35.761 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:35.761 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:35.761 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:35.761 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.815 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5481.0:124621 6.9 e290/269 6:9dcbe73e:::benchmark_data_smithi074_63843_object15577:head v 290'7725, mlcod=290'7718)", 2020-08-07T15:13:35.816 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:35.713621+0000", 2020-08-07T15:13:35.816 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.003803814, 2020-08-07T15:13:35.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039041530000000001, 2020-08-07T15:13:35.816 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:35.816 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:35.817 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:35.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.817 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:35.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.713621+0000", 2020-08-07T15:13:35.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:35.818 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.818 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.818 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:35.818 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.713621+0000", 2020-08-07T15:13:35.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5806000000000001e-05 2020-08-07T15:13:35.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.854 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.854 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:35.854 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.713677+0000", 2020-08-07T15:13:35.855 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4989999999999999e-06 2020-08-07T15:13:35.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.856 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:35.856 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.713683+0000", 2020-08-07T15:13:35.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017071999999999999 2020-08-07T15:13:35.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.878 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:35.879 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.713854+0000", 2020-08-07T15:13:35.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8399999999999997e-06 2020-08-07T15:13:35.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.900 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:35.901 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.713858+0000", 2020-08-07T15:13:35.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000617769 2020-08-07T15:13:35.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:35.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.714476+0000", 2020-08-07T15:13:35.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1237999999999999e-05 2020-08-07T15:13:35.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:35.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.714497+0000", 2020-08-07T15:13:35.914 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00087587199999999998 2020-08-07T15:13:35.915 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:35.915 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:35.916 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:35.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.917 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5481.0:124622 6.9 e290/269 6:9dcbe73e:::benchmark_data_smithi074_63843_object15577:head v 290'7726, mlcod=290'7718)", 2020-08-07T15:13:35.917 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:35.713719+0000", 2020-08-07T15:13:35.917 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0037060790000000001, 2020-08-07T15:13:35.918 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038742440000000002, 2020-08-07T15:13:35.918 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:35.918 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:35.930 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:35.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:35.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.713719+0000", 2020-08-07T15:13:35.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:35.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:35.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.713719+0000", 2020-08-07T15:13:35.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9129999999999999e-05 2020-08-07T15:13:35.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.948 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:35.948 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.713768+0000", 2020-08-07T15:13:35.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7309999999999998e-06 2020-08-07T15:13:35.949 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.949 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:35.950 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.713774+0000", 2020-08-07T15:13:35.950 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00064204900000000001 2020-08-07T15:13:35.950 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:35.951 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.714416+0000", 2020-08-07T15:13:35.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.429e-06 2020-08-07T15:13:35.952 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.952 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.952 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:35.952 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.714421+0000", 2020-08-07T15:13:35.952 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000339723 2020-08-07T15:13:35.953 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.953 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.953 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:35.953 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.714761+0000", 2020-08-07T15:13:35.953 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1622000000000001e-05 2020-08-07T15:13:35.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:35.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.714783+0000", 2020-08-07T15:13:35.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010636840000000001 2020-08-07T15:13:35.974 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:35.974 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:35.975 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:35.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.975 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5481.0:124623 6.9 e290/269 6:9dcbe73e:::benchmark_data_smithi074_63843_object15577:head v 290'7727, mlcod=290'7719)", 2020-08-07T15:13:35.976 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:35.714329+0000", 2020-08-07T15:13:35.976 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0030956360000000001, 2020-08-07T15:13:35.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033380179999999999, 2020-08-07T15:13:35.976 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:35.977 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:35.977 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:35.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:35.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.714329+0000", 2020-08-07T15:13:35.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:35.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.978 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:35.978 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.714329+0000", 2020-08-07T15:13:35.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2757000000000003e-05 2020-08-07T15:13:35.979 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.979 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.979 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:35.979 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.714382+0000", 2020-08-07T15:13:35.980 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4110000000000001e-06 2020-08-07T15:13:35.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:35.980 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.714389+0000", 2020-08-07T15:13:35.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00081231999999999995 2020-08-07T15:13:35.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.981 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:35.981 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.715201+0000", 2020-08-07T15:13:35.982 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9020000000000004e-06 2020-08-07T15:13:35.982 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.982 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:35.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.715207+0000", 2020-08-07T15:13:35.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9096000000000003e-05 2020-08-07T15:13:35.983 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.983 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.983 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:35.983 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.715286+0000", 2020-08-07T15:13:35.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9839000000000001e-05 2020-08-07T15:13:35.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.984 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:35.984 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.715306+0000", 2020-08-07T15:13:35.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000976325 2020-08-07T15:13:35.985 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:35.985 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:35.985 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:35.985 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.985 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.986 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5481.0:124618 6.9 e290/269 6:9dcbe73e:::benchmark_data_smithi074_63843_object15577:head v 290'7722, mlcod=290'7718)", 2020-08-07T15:13:35.986 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:35.709514+0000", 2020-08-07T15:13:35.986 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0079112499999999999, 2020-08-07T15:13:35.986 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0082262419999999999, 2020-08-07T15:13:35.986 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:35.987 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:35.987 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:35.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.996 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:35.996 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.709514+0000", 2020-08-07T15:13:35.996 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:35.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.996 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.997 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:35.997 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.709514+0000", 2020-08-07T15:13:35.997 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5528000000000001e-05 2020-08-07T15:13:35.997 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.997 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.998 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:35.998 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.709569+0000", 2020-08-07T15:13:35.998 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2130000000000002e-06 2020-08-07T15:13:35.998 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.998 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:35.999 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:35.999 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.709576+0000", 2020-08-07T15:13:35.999 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0030618279999999999 2020-08-07T15:13:35.999 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:35.999 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:36.000 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:36.000 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.712638+0000", 2020-08-07T15:13:36.000 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8549999999999998e-06 2020-08-07T15:13:36.000 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:36.000 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:36.001 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:36.001 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.712644+0000", 2020-08-07T15:13:36.001 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015028 2020-08-07T15:13:36.001 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:36.001 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:36.002 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:36.002 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.712794+0000", 2020-08-07T15:13:36.002 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.4226000000000001e-05 2020-08-07T15:13:36.002 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:36.002 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:36.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:36.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.712819+0000", 2020-08-07T15:13:36.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033049300000000002 2020-08-07T15:13:36.003 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:36.004 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:36.004 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:36.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:36.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:36.004 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5481.0:124619 6.9 e290/269 6:9dcbe73e:::benchmark_data_smithi074_63843_object15577:head v 290'7723, mlcod=290'7718)", 2020-08-07T15:13:36.005 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:35.712464+0000", 2020-08-07T15:13:36.005 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0049610119999999999, 2020-08-07T15:13:36.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005343413, 2020-08-07T15:13:36.005 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:36.005 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:36.006 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:36.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:36.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:36.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.712464+0000", 2020-08-07T15:13:36.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:36.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:36.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:36.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:36.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.712464+0000", 2020-08-07T15:13:36.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1857000000000002e-05 2020-08-07T15:13:36.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:36.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:36.010 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:36.010 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.712516+0000", 2020-08-07T15:13:36.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3950000000000003e-06 2020-08-07T15:13:36.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:36.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:36.011 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:36.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.712523+0000", 2020-08-07T15:13:36.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000141395 2020-08-07T15:13:36.012 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:36.012 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:36.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:36.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.712665+0000", 2020-08-07T15:13:36.013 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.552e-06 2020-08-07T15:13:36.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:36.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:36.013 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:36.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.712668+0000", 2020-08-07T15:13:36.014 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00046474999999999999 2020-08-07T15:13:36.014 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:36.014 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:36.015 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:36.015 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.713133+0000", 2020-08-07T15:13:36.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2861e-05 2020-08-07T15:13:36.016 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:36.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:36.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:36.017 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:35.713156+0000", 2020-08-07T15:13:36.017 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00069180999999999995 2020-08-07T15:13:36.018 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:36.018 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:36.018 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:36.019 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:36.019 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:13:36.019 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 6 2020-08-07T15:13:36.020 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:36.022 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:13:36.023 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:13:36.023 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:13:36.024 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:13:36.024 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:13:36.024 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:13:36.024 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:13:36.024 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:36.020+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:36.032 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:13:36.081 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:13:36.089 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:13:36.090 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:13:36.090 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:13:36.090 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:13:36.091 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:13:36.091 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:13:36.094 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:13:36.094 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:13:36.123 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:36.121+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:36.232 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:36.230+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:36.307 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 84 16 125511 125495 11.6703 11.6875 0.0107325 0.0107066 2020-08-07T15:13:36.307 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:36.307 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:36.323 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:13:36.346 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:36.338+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:36.414 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:13:36.450 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:36.442+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:36.522 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:36.522 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:36.533 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:13:36.577 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:36.573+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:36.654 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:36.649+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:36.694 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:36.694 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:36.712 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:13:36.752 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:36.750+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:36.856 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:36.855+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:36.890 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:36.891 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:36.905 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:13:36.957 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:36.956+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:37.065 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:37.058+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:37.142 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:37.142 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:37.163 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:37.162+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:37.164 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:13:37.265 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:37.262+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:37.275 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 85 16 126732 126716 11.6453 9.53906 0.00506833 0.0107308 2020-08-07T15:13:37.366 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:37.364+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:37.396 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:37.397 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:37.411 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:13:37.498 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:37.487+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:37.570 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:37.565+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:37.599 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:37.599 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:37.615 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:13:37.674 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:37.669+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:37.786 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:37.774+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:37.837 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:37.837 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:37.848 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:13:37.881 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:37.880+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:37.990 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:37.983+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:38.089 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:38.090 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:38.091 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:38.086+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:38.101 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:13:38.103 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:13:38.193 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:38.190+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:38.290 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 86 16 128129 128113 11.6358 10.9141 0.00345833 0.0107389 2020-08-07T15:13:38.295 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:38.296 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:38.296 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:38.295+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:38.312 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:13:38.312 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:13:38.330 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:13:38.350 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:13:38.386 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:13:38.402 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:38.398+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:38.426 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:13:38.427 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:13:38.474 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:13:38.483 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:38.483 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:13:38.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:13:38.484 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:13:38.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.485 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "MOSDPGPushReply(3.1 62/60 [PushReplyOp(3:8329f37e:::benchmark_data_smithi074_35447_object11915:head)])", 2020-08-07T15:13:38.485 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:13:38.485 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 347.604825533, 2020-08-07T15:13:38.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996, 2020-08-07T15:13:38.494 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:38.494 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:38.495 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:38.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:38.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:13:38.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:38.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:38.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:13:38.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5372000000000001e-05 2020-08-07T15:13:38.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:38.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858416+0000", 2020-08-07T15:13:38.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6899999999999998e-06 2020-08-07T15:13:38.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:38.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858419+0000", 2020-08-07T15:13:38.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63907904299999996 2020-08-07T15:13:38.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:38.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497498+0000", 2020-08-07T15:13:38.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3332999999999999e-05 2020-08-07T15:13:38.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:38.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497512+0000", 2020-08-07T15:13:38.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000450875 2020-08-07T15:13:38.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497962+0000", 2020-08-07T15:13:38.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000267779 2020-08-07T15:13:38.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:38.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.498230+0000", 2020-08-07T15:13:38.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996 2020-08-07T15:13:38.517 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:38.517 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:38.518 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:38.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.519 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:145792 3.1 3:881e0420:::benchmark_data_smithi074_35447_object16183:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:13:38.519 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:13:38.519 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 323.796419652, 2020-08-07T15:13:38.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997, 2020-08-07T15:13:38.519 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:38.520 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:38.520 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:38.520 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:13:38.520 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:13:38.520 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 145792 2020-08-07T15:13:38.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.521 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:38.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:38.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:13:38.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:38.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:38.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:13:38.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7489000000000001e-05 2020-08-07T15:13:38.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:38.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666824+0000", 2020-08-07T15:13:38.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0169999999999998e-06 2020-08-07T15:13:38.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:38.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666828+0000", 2020-08-07T15:13:38.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9829999999999996e-06 2020-08-07T15:13:38.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:38.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666836+0000", 2020-08-07T15:13:38.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.481e-06 2020-08-07T15:13:38.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:38.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666840+0000", 2020-08-07T15:13:38.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5299000000000001e-05 2020-08-07T15:13:38.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666885+0000", 2020-08-07T15:13:38.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014456500000000001 2020-08-07T15:13:38.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:38.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667030+0000", 2020-08-07T15:13:38.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017677799999999999 2020-08-07T15:13:38.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,5", 2020-08-07T15:13:38.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667207+0000", 2020-08-07T15:13:38.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004149649 2020-08-07T15:13:38.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:38.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671356+0000", 2020-08-07T15:13:38.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019339649999999999 2020-08-07T15:13:38.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:38.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673290+0000", 2020-08-07T15:13:38.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73346620500000004 2020-08-07T15:13:38.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:38.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406756+0000", 2020-08-07T15:13:38.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4700000000000001e-06 2020-08-07T15:13:38.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:38.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406759+0000", 2020-08-07T15:13:38.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6258999999999998e-05 2020-08-07T15:13:38.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:38.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406815+0000", 2020-08-07T15:13:38.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0126000000000002e-05 2020-08-07T15:13:38.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:38.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406855+0000", 2020-08-07T15:13:38.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997 2020-08-07T15:13:38.570 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:38.570 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:38.570 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:38.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.571 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:145792 3.1 e75/67 ondisk, result = 0)", 2020-08-07T15:13:38.571 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:13:38.571 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 323.79203476399999, 2020-08-07T15:13:38.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003, 2020-08-07T15:13:38.572 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:38.572 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:38.572 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:38.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:38.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:13:38.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:38.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:38.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:13:38.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2830999999999998e-05 2020-08-07T15:13:38.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:38.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671224+0000", 2020-08-07T15:13:38.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:13:38.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:38.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671229+0000", 2020-08-07T15:13:38.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.735414864 2020-08-07T15:13:38.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:38.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406644+0000", 2020-08-07T15:13:38.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1250000000000004e-06 2020-08-07T15:13:38.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:38.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406651+0000", 2020-08-07T15:13:38.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6567e-05 2020-08-07T15:13:38.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406737+0000", 2020-08-07T15:13:38.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0054000000000001e-05 2020-08-07T15:13:38.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:38.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406747+0000", 2020-08-07T15:13:38.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011391999999999999 2020-08-07T15:13:38.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:38.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406861+0000", 2020-08-07T15:13:38.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003 2020-08-07T15:13:38.594 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:38.594 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:38.594 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:38.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.595 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30921 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:13:38.595 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:13:38.595 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 291.75728638999999, 2020-08-07T15:13:38.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001, 2020-08-07T15:13:38.602 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:38.602 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:38.602 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:38.603 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:38.603 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:38.603 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30921 2020-08-07T15:13:38.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.603 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:38.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:38.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:13:38.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:38.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:38.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:13:38.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6954999999999998e-05 2020-08-07T15:13:38.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:38.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706016+0000", 2020-08-07T15:13:38.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0229999999999992e-06 2020-08-07T15:13:38.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:38.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706024+0000", 2020-08-07T15:13:38.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.432e-05 2020-08-07T15:13:38.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:38.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706039+0000", 2020-08-07T15:13:38.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8490000000000006e-06 2020-08-07T15:13:38.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:38.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706046+0000", 2020-08-07T15:13:38.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6837999999999999e-05 2020-08-07T15:13:38.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706113+0000", 2020-08-07T15:13:38.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3553000000000001e-05 2020-08-07T15:13:38.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:38.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706127+0000", 2020-08-07T15:13:38.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77277201699999998 2020-08-07T15:13:38.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.478899+0000", 2020-08-07T15:13:38.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093978599999999996 2020-08-07T15:13:38.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:13:38.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479839+0000", 2020-08-07T15:13:38.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010376029 2020-08-07T15:13:38.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490215+0000", 2020-08-07T15:13:38.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000190657 2020-08-07T15:13:38.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:38.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490405+0000", 2020-08-07T15:13:38.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001 2020-08-07T15:13:38.633 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:38.633 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:38.633 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:38.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.638 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30922 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:13:38.638 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:13:38.639 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 291.75712622700001, 2020-08-07T15:13:38.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247, 2020-08-07T15:13:38.639 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:38.639 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:38.639 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:38.640 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:38.640 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:38.640 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30922 2020-08-07T15:13:38.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.640 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:38.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:38.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:13:38.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:38.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:38.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:13:38.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9178999999999997e-05 2020-08-07T15:13:38.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:38.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706159+0000", 2020-08-07T15:13:38.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4999999999999999e-06 2020-08-07T15:13:38.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:38.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706164+0000", 2020-08-07T15:13:38.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1792e-05 2020-08-07T15:13:38.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:38.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706176+0000", 2020-08-07T15:13:38.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.139e-06 2020-08-07T15:13:38.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:38.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706182+0000", 2020-08-07T15:13:38.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039724900000000002 2020-08-07T15:13:38.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706579+0000", 2020-08-07T15:13:38.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1191e-05 2020-08-07T15:13:38.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:38.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706591+0000", 2020-08-07T15:13:38.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329101300000003 2020-08-07T15:13:38.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479882+0000", 2020-08-07T15:13:38.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9565000000000001e-05 2020-08-07T15:13:38.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:13:38.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479971+0000", 2020-08-07T15:13:38.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010471263 2020-08-07T15:13:38.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490442+0000", 2020-08-07T15:13:38.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015035600000000001 2020-08-07T15:13:38.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:38.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490593+0000", 2020-08-07T15:13:38.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247 2020-08-07T15:13:38.682 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:38.683 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:38.683 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:38.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.683 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30923 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:13:38.684 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:13:38.684 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 291.75665116599998, 2020-08-07T15:13:38.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998, 2020-08-07T15:13:38.684 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:38.684 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:38.685 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:38.686 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:38.686 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:38.688 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30923 2020-08-07T15:13:38.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.688 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:38.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:38.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:13:38.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:38.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:38.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:13:38.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5531000000000001e-05 2020-08-07T15:13:38.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:38.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706630+0000", 2020-08-07T15:13:38.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7689999999999999e-06 2020-08-07T15:13:38.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:38.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706637+0000", 2020-08-07T15:13:38.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3577999999999999e-05 2020-08-07T15:13:38.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:38.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706650+0000", 2020-08-07T15:13:38.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1439999999999997e-06 2020-08-07T15:13:38.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:38.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706658+0000", 2020-08-07T15:13:38.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1663999999999999e-05 2020-08-07T15:13:38.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706709+0000", 2020-08-07T15:13:38.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0149000000000001e-05 2020-08-07T15:13:38.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:38.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706719+0000", 2020-08-07T15:13:38.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329694900000001 2020-08-07T15:13:38.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480016+0000", 2020-08-07T15:13:38.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7998999999999997e-05 2020-08-07T15:13:38.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:13:38.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480084+0000", 2020-08-07T15:13:38.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010547333000000001 2020-08-07T15:13:38.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490632+0000", 2020-08-07T15:13:38.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7439e-05 2020-08-07T15:13:38.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:38.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490729+0000", 2020-08-07T15:13:38.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998 2020-08-07T15:13:38.712 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:38.713 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:38.713 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:38.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.713 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30924 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:13:38.720 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:13:38.720 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 291.756523894, 2020-08-07T15:13:38.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995, 2020-08-07T15:13:38.722 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:38.724 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:38.724 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:38.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:38.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:38.725 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30924 2020-08-07T15:13:38.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.725 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:38.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:38.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:13:38.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:38.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:38.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:13:38.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9698999999999999e-05 2020-08-07T15:13:38.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:38.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706762+0000", 2020-08-07T15:13:38.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8080000000000001e-06 2020-08-07T15:13:38.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:38.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706767+0000", 2020-08-07T15:13:38.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1069e-05 2020-08-07T15:13:38.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:38.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706778+0000", 2020-08-07T15:13:38.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6969999999999998e-06 2020-08-07T15:13:38.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:38.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706784+0000", 2020-08-07T15:13:38.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8792999999999998e-05 2020-08-07T15:13:38.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706843+0000", 2020-08-07T15:13:38.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5449999999999998e-06 2020-08-07T15:13:38.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:38.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706850+0000", 2020-08-07T15:13:38.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.773278354 2020-08-07T15:13:38.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480129+0000", 2020-08-07T15:13:38.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1577999999999997e-05 2020-08-07T15:13:38.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:13:38.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480210+0000", 2020-08-07T15:13:38.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010541915000000001 2020-08-07T15:13:38.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490752+0000", 2020-08-07T15:13:38.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103937 2020-08-07T15:13:38.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:38.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490856+0000", 2020-08-07T15:13:38.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995 2020-08-07T15:13:38.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:38.769 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:38.770 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:38.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.770 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30925 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:13:38.770 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:13:38.771 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 291.75640012299999, 2020-08-07T15:13:38.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995, 2020-08-07T15:13:38.771 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:38.771 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:38.771 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:38.772 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:38.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:38.786 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30925 2020-08-07T15:13:38.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.787 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:38.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:38.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:13:38.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:38.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:38.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:13:38.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4664999999999999e-05 2020-08-07T15:13:38.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:38.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706880+0000", 2020-08-07T15:13:38.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7479999999999999e-06 2020-08-07T15:13:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706886+0000", 2020-08-07T15:13:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1838000000000001e-05 2020-08-07T15:13:38.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:38.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706898+0000", 2020-08-07T15:13:38.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0089999999999998e-06 2020-08-07T15:13:38.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:38.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706903+0000", 2020-08-07T15:13:38.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3559e-05 2020-08-07T15:13:38.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706956+0000", 2020-08-07T15:13:38.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0309000000000001e-05 2020-08-07T15:13:38.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706967+0000", 2020-08-07T15:13:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77328290499999996 2020-08-07T15:13:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480250+0000", 2020-08-07T15:13:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8680999999999997e-05 2020-08-07T15:13:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:13:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480298+0000", 2020-08-07T15:13:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01059272 2020-08-07T15:13:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490891+0000", 2020-08-07T15:13:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014580300000000001 2020-08-07T15:13:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491037+0000", 2020-08-07T15:13:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995 2020-08-07T15:13:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30926 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:13:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:13:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 291.75628325399998, 2020-08-07T15:13:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005, 2020-08-07T15:13:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30926 2020-08-07T15:13:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:38.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:13:38.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:38.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:38.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:13:38.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5223000000000003e-05 2020-08-07T15:13:38.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:38.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706998+0000", 2020-08-07T15:13:38.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4809999999999997e-06 2020-08-07T15:13:38.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707003+0000", 2020-08-07T15:13:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1663e-05 2020-08-07T15:13:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707015+0000", 2020-08-07T15:13:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9729999999999997e-06 2020-08-07T15:13:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707020+0000", 2020-08-07T15:13:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2565999999999999e-05 2020-08-07T15:13:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707062+0000", 2020-08-07T15:13:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8944e-05 2020-08-07T15:13:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707081+0000", 2020-08-07T15:13:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77325885000000005 2020-08-07T15:13:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480340+0000", 2020-08-07T15:13:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0321000000000006e-05 2020-08-07T15:13:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:13:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480420+0000", 2020-08-07T15:13:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010655958 2020-08-07T15:13:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491076+0000", 2020-08-07T15:13:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9784999999999995e-05 2020-08-07T15:13:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491166+0000", 2020-08-07T15:13:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005 2020-08-07T15:13:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30927 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:13:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:13:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 291.75616543500001, 2020-08-07T15:13:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999, 2020-08-07T15:13:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30927 2020-08-07T15:13:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:13:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:13:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9861999999999997e-05 2020-08-07T15:13:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707120+0000", 2020-08-07T15:13:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8969999999999996e-06 2020-08-07T15:13:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707126+0000", 2020-08-07T15:13:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1248e-05 2020-08-07T15:13:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707137+0000", 2020-08-07T15:13:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.955e-06 2020-08-07T15:13:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707142+0000", 2020-08-07T15:13:38.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0643999999999998e-05 2020-08-07T15:13:38.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707193+0000", 2020-08-07T15:13:38.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1983e-05 2020-08-07T15:13:38.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:38.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707205+0000", 2020-08-07T15:13:38.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77324914 2020-08-07T15:13:38.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480454+0000", 2020-08-07T15:13:38.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4119999999999998e-05 2020-08-07T15:13:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:13:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480498+0000", 2020-08-07T15:13:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010709707000000001 2020-08-07T15:13:38.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491208+0000", 2020-08-07T15:13:38.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014433800000000001 2020-08-07T15:13:38.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:38.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491352+0000", 2020-08-07T15:13:38.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999 2020-08-07T15:13:38.902 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:38.902 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:38.902 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:38.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.907 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30928 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:13:38.907 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:13:38.907 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 291.75604107800001, 2020-08-07T15:13:38.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998, 2020-08-07T15:13:38.908 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:38.908 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:38.908 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:38.908 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:38.908 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:38.909 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30928 2020-08-07T15:13:38.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.909 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:38.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:38.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:13:38.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:38.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:38.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:13:38.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6582000000000001e-05 2020-08-07T15:13:38.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.934 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:13:38.935 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:13:38.935 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:13:38.935 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:13:38.935 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:13:38.936 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:13:38.936 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:13:38.936 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:13:38.936 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:13:38.936 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:13:38.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:38.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707241+0000", 2020-08-07T15:13:38.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9190000000000004e-06 2020-08-07T15:13:38.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:38.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707247+0000", 2020-08-07T15:13:38.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000188822 2020-08-07T15:13:38.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:38.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707436+0000", 2020-08-07T15:13:38.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2940000000000002e-06 2020-08-07T15:13:38.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:38.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707442+0000", 2020-08-07T15:13:38.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7800000000000003e-05 2020-08-07T15:13:38.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707490+0000", 2020-08-07T15:13:38.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2020-08-07T15:13:38.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:38.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707501+0000", 2020-08-07T15:13:38.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77303498800000003 2020-08-07T15:13:38.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480536+0000", 2020-08-07T15:13:38.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7723000000000001e-05 2020-08-07T15:13:38.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:13:38.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480614+0000", 2020-08-07T15:13:38.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010812183 2020-08-07T15:13:38.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491426+0000", 2020-08-07T15:13:38.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5502999999999999e-05 2020-08-07T15:13:38.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491502+0000", 2020-08-07T15:13:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998 2020-08-07T15:13:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31009 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:13:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:13:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 291.73208477200001, 2020-08-07T15:13:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002, 2020-08-07T15:13:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31009 2020-08-07T15:13:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:13:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:38.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:13:38.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2429000000000002e-05 2020-08-07T15:13:38.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:38.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731193+0000", 2020-08-07T15:13:38.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9760000000000003e-06 2020-08-07T15:13:38.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:38.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731198+0000", 2020-08-07T15:13:38.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3597e-05 2020-08-07T15:13:38.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:38.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731212+0000", 2020-08-07T15:13:38.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1709999999999996e-06 2020-08-07T15:13:38.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:38.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731219+0000", 2020-08-07T15:13:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5402999999999997e-05 2020-08-07T15:13:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731285+0000", 2020-08-07T15:13:38.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3495e-05 2020-08-07T15:13:38.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:38.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731298+0000", 2020-08-07T15:13:38.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74935423599999995 2020-08-07T15:13:38.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:38.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480652+0000", 2020-08-07T15:13:38.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000137806 2020-08-07T15:13:38.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:38.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480790+0000", 2020-08-07T15:13:38.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022272399999999999 2020-08-07T15:13:38.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:13:38.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481013+0000", 2020-08-07T15:13:38.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003223381 2020-08-07T15:13:38.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:38.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484236+0000", 2020-08-07T15:13:38.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1370000000000001e-06 2020-08-07T15:13:38.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:38.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484237+0000", 2020-08-07T15:13:38.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6288e-05 2020-08-07T15:13:38.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:38.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484294+0000", 2020-08-07T15:13:38.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.007260402 2020-08-07T15:13:38.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:38.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491554+0000", 2020-08-07T15:13:38.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7358000000000003e-05 2020-08-07T15:13:38.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:38.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491601+0000", 2020-08-07T15:13:38.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1100000000000004e-07 2020-08-07T15:13:38.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:38.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491602+0000", 2020-08-07T15:13:38.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013436500000000001 2020-08-07T15:13:38.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:38.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:38.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:38.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491737+0000", 2020-08-07T15:13:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002 2020-08-07T15:13:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.000 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31010 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:13:39.000 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:13:39.000 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 291.73195881800001, 2020-08-07T15:13:39.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001, 2020-08-07T15:13:39.001 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:39.001 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:39.001 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:39.001 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:39.006 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:39.006 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31010 2020-08-07T15:13:39.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.006 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:39.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:39.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:13:39.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:39.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:39.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:13:39.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2398000000000002e-05 2020-08-07T15:13:39.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:39.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731319+0000", 2020-08-07T15:13:39.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7100000000000004e-06 2020-08-07T15:13:39.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:39.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731325+0000", 2020-08-07T15:13:39.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4328000000000001e-05 2020-08-07T15:13:39.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:39.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731339+0000", 2020-08-07T15:13:39.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3310000000000003e-06 2020-08-07T15:13:39.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:39.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731345+0000", 2020-08-07T15:13:39.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8952999999999998e-05 2020-08-07T15:13:39.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:39.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731384+0000", 2020-08-07T15:13:39.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0036e-05 2020-08-07T15:13:39.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:39.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731394+0000", 2020-08-07T15:13:39.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75027681300000004 2020-08-07T15:13:39.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:39.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481670+0000", 2020-08-07T15:13:39.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016555899999999999 2020-08-07T15:13:39.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:39.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481836+0000", 2020-08-07T15:13:39.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028760999999999999 2020-08-07T15:13:39.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:13:39.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482124+0000", 2020-08-07T15:13:39.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038342680000000001 2020-08-07T15:13:39.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:39.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485958+0000", 2020-08-07T15:13:39.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2559999999999999e-06 2020-08-07T15:13:39.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:39.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485959+0000", 2020-08-07T15:13:39.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3758999999999995e-05 2020-08-07T15:13:39.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:39.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486043+0000", 2020-08-07T15:13:39.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057590419999999998 2020-08-07T15:13:39.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:39.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491802+0000", 2020-08-07T15:13:39.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5757000000000001e-05 2020-08-07T15:13:39.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:39.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:13:39.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9299999999999997e-07 2020-08-07T15:13:39.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:39.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:13:39.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012496999999999999 2020-08-07T15:13:39.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:39.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491993+0000", 2020-08-07T15:13:39.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001 2020-08-07T15:13:39.098 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:39.098 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:39.098 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:39.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.099 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31011 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:13:39.108 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:13:39.108 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 291.73184261300003, 2020-08-07T15:13:39.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997, 2020-08-07T15:13:39.109 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:39.109 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:39.109 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:39.110 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:39.110 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:39.110 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31011 2020-08-07T15:13:39.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.111 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:39.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:39.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:13:39.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:39.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:39.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:13:39.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9539999999999997e-05 2020-08-07T15:13:39.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:39.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731433+0000", 2020-08-07T15:13:39.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9209999999999997e-06 2020-08-07T15:13:39.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:39.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731439+0000", 2020-08-07T15:13:39.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4246999999999999e-05 2020-08-07T15:13:39.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:39.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731453+0000", 2020-08-07T15:13:39.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8130000000000003e-06 2020-08-07T15:13:39.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:39.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731458+0000", 2020-08-07T15:13:39.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6668000000000001e-05 2020-08-07T15:13:39.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:39.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731504+0000", 2020-08-07T15:13:39.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1112e-05 2020-08-07T15:13:39.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:39.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731515+0000", 2020-08-07T15:13:39.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75121332100000004 2020-08-07T15:13:39.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:39.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482729+0000", 2020-08-07T15:13:39.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1810000000000004e-05 2020-08-07T15:13:39.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:39.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482811+0000", 2020-08-07T15:13:39.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018001399999999999 2020-08-07T15:13:39.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:13:39.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482991+0000", 2020-08-07T15:13:39.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030701140000000001 2020-08-07T15:13:39.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:39.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486061+0000", 2020-08-07T15:13:39.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4499999999999996e-07 2020-08-07T15:13:39.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:39.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486062+0000", 2020-08-07T15:13:39.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5366999999999998e-05 2020-08-07T15:13:39.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:39.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486147+0000", 2020-08-07T15:13:39.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056872779999999996 2020-08-07T15:13:39.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:39.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491834+0000", 2020-08-07T15:13:39.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024645100000000002 2020-08-07T15:13:39.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:39.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:13:39.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1200000000000003e-07 2020-08-07T15:13:39.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:39.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:13:39.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1451000000000002e-05 2020-08-07T15:13:39.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:39.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492173+0000", 2020-08-07T15:13:39.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997 2020-08-07T15:13:39.196 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:39.196 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:39.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:39.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.199 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31012 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:13:39.199 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:13:39.199 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 291.73118849399998, 2020-08-07T15:13:39.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003, 2020-08-07T15:13:39.200 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:39.200 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:39.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:39.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:39.230 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:39.231 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31012 2020-08-07T15:13:39.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.234 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:39.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:39.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:13:39.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:39.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:39.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:13:39.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6344999999999997e-05 2020-08-07T15:13:39.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732094+0000", 2020-08-07T15:13:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2920000000000001e-06 2020-08-07T15:13:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732100+0000", 2020-08-07T15:13:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.189e-05 2020-08-07T15:13:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:39.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732112+0000", 2020-08-07T15:13:39.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3720000000000001e-06 2020-08-07T15:13:39.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:39.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732118+0000", 2020-08-07T15:13:39.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2514000000000004e-05 2020-08-07T15:13:39.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:39.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732191+0000", 2020-08-07T15:13:39.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4610000000000001e-05 2020-08-07T15:13:39.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:39.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732205+0000", 2020-08-07T15:13:39.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75132974100000005 2020-08-07T15:13:39.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483535+0000", 2020-08-07T15:13:39.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9185999999999996e-05 2020-08-07T15:13:39.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:39.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483624+0000", 2020-08-07T15:13:39.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015683 2020-08-07T15:13:39.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:13:39.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483781+0000", 2020-08-07T15:13:39.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046202889999999996 2020-08-07T15:13:39.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:39.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488401+0000", 2020-08-07T15:13:39.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6700000000000002e-07 2020-08-07T15:13:39.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:39.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488402+0000", 2020-08-07T15:13:39.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1487000000000001e-05 2020-08-07T15:13:39.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:39.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488484+0000", 2020-08-07T15:13:39.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035520830000000002 2020-08-07T15:13:39.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:39.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492036+0000", 2020-08-07T15:13:39.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029266600000000002 2020-08-07T15:13:39.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:39.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492328+0000", 2020-08-07T15:13:39.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5099999999999998e-07 2020-08-07T15:13:39.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:39.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492329+0000", 2020-08-07T15:13:39.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000201299 2020-08-07T15:13:39.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:39.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492531+0000", 2020-08-07T15:13:39.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003 2020-08-07T15:13:39.297 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:39.297 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:39.297 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:39.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.298 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31013 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:13:39.298 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:13:39.298 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 291.73097686300002, 2020-08-07T15:13:39.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996, 2020-08-07T15:13:39.298 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:39.299 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:39.299 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:39.299 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:39.299 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:39.299 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31013 2020-08-07T15:13:39.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.316 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:39.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:39.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:13:39.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:39.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:39.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:13:39.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8949000000000002e-05 2020-08-07T15:13:39.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:39.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732298+0000", 2020-08-07T15:13:39.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1399999999999997e-06 2020-08-07T15:13:39.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:39.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732304+0000", 2020-08-07T15:13:39.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.154e-05 2020-08-07T15:13:39.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:39.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732315+0000", 2020-08-07T15:13:39.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1319999999999999e-06 2020-08-07T15:13:39.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:39.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732322+0000", 2020-08-07T15:13:39.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7431e-05 2020-08-07T15:13:39.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:39.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732369+0000", 2020-08-07T15:13:39.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0621999999999999e-05 2020-08-07T15:13:39.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:39.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732380+0000", 2020-08-07T15:13:39.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75196845999999995 2020-08-07T15:13:39.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:39.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484348+0000", 2020-08-07T15:13:39.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000123008 2020-08-07T15:13:39.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:39.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484471+0000", 2020-08-07T15:13:39.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017572700000000001 2020-08-07T15:13:39.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:13:39.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484647+0000", 2020-08-07T15:13:39.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038557650000000001 2020-08-07T15:13:39.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:39.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:13:39.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2020-08-07T15:13:39.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:39.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:13:39.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0714e-05 2020-08-07T15:13:39.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:39.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488584+0000", 2020-08-07T15:13:39.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036298319999999999 2020-08-07T15:13:39.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:39.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492214+0000", 2020-08-07T15:13:39.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039530099999999997 2020-08-07T15:13:39.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:39.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492609+0000", 2020-08-07T15:13:39.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5300000000000002e-07 2020-08-07T15:13:39.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:39.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492610+0000", 2020-08-07T15:13:39.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197321 2020-08-07T15:13:39.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:39.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492807+0000", 2020-08-07T15:13:39.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996 2020-08-07T15:13:39.371 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:39.371 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:39.371 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:39.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.372 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31014 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:13:39.372 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:13:39.372 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 291.72959471899998, 2020-08-07T15:13:39.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998, 2020-08-07T15:13:39.373 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:39.373 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:39.373 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:39.373 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:39.373 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:39.374 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31014 2020-08-07T15:13:39.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.374 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:39.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:39.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:13:39.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:39.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:39.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:13:39.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3743999999999999e-05 2020-08-07T15:13:39.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:39.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733685+0000", 2020-08-07T15:13:39.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1720000000000007e-06 2020-08-07T15:13:39.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:39.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733693+0000", 2020-08-07T15:13:39.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3674e-05 2020-08-07T15:13:39.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:39.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733707+0000", 2020-08-07T15:13:39.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9249999999999998e-06 2020-08-07T15:13:39.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:39.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733714+0000", 2020-08-07T15:13:39.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8338000000000003e-05 2020-08-07T15:13:39.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:39.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733772+0000", 2020-08-07T15:13:39.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3019e-05 2020-08-07T15:13:39.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:39.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733785+0000", 2020-08-07T15:13:39.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75134856299999997 2020-08-07T15:13:39.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:39.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485133+0000", 2020-08-07T15:13:39.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8739999999999995e-05 2020-08-07T15:13:39.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:39.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485212+0000", 2020-08-07T15:13:39.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151244 2020-08-07T15:13:39.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:13:39.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485363+0000", 2020-08-07T15:13:39.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032929090000000001 2020-08-07T15:13:39.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:39.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488656+0000", 2020-08-07T15:13:39.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.065e-06 2020-08-07T15:13:39.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:39.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488657+0000", 2020-08-07T15:13:39.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1123000000000003e-05 2020-08-07T15:13:39.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:39.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488729+0000", 2020-08-07T15:13:39.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035508330000000002 2020-08-07T15:13:39.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:39.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492279+0000", 2020-08-07T15:13:39.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00059376000000000003 2020-08-07T15:13:39.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:39.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492873+0000", 2020-08-07T15:13:39.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5700000000000002e-07 2020-08-07T15:13:39.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:39.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492874+0000", 2020-08-07T15:13:39.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019264 2020-08-07T15:13:39.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:39.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493066+0000", 2020-08-07T15:13:39.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998 2020-08-07T15:13:39.443 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:39.444 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:39.444 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:39.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.453 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31015 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:13:39.453 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:13:39.453 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 291.72946875299999, 2020-08-07T15:13:39.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002, 2020-08-07T15:13:39.466 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:39.466 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:39.467 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:39.467 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:39.467 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:39.468 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31015 2020-08-07T15:13:39.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.468 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:39.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:39.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:13:39.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:39.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:39.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:13:39.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3707000000000001e-05 2020-08-07T15:13:39.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:39.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733791+0000", 2020-08-07T15:13:39.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8120000000000001e-06 2020-08-07T15:13:39.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:39.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733794+0000", 2020-08-07T15:13:39.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0728e-05 2020-08-07T15:13:39.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:39.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733805+0000", 2020-08-07T15:13:39.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8279999999999999e-06 2020-08-07T15:13:39.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:39.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733811+0000", 2020-08-07T15:13:39.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2955999999999999e-05 2020-08-07T15:13:39.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:39.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733864+0000", 2020-08-07T15:13:39.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1389000000000001e-05 2020-08-07T15:13:39.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:39.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733875+0000", 2020-08-07T15:13:39.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75232711100000005 2020-08-07T15:13:39.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:39.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486202+0000", 2020-08-07T15:13:39.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027503400000000001 2020-08-07T15:13:39.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:39.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486478+0000", 2020-08-07T15:13:39.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033010400000000001 2020-08-07T15:13:39.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:13:39.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486808+0000", 2020-08-07T15:13:39.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019382609999999999 2020-08-07T15:13:39.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:39.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488746+0000", 2020-08-07T15:13:39.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9400000000000004e-07 2020-08-07T15:13:39.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:39.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488747+0000", 2020-08-07T15:13:39.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4324000000000002e-05 2020-08-07T15:13:39.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:39.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488821+0000", 2020-08-07T15:13:39.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043745640000000001 2020-08-07T15:13:39.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:39.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493196+0000", 2020-08-07T15:13:39.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030381000000000001 2020-08-07T15:13:39.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:39.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493499+0000", 2020-08-07T15:13:39.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4e-07 2020-08-07T15:13:39.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:39.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493500+0000", 2020-08-07T15:13:39.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017499 2020-08-07T15:13:39.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:39.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493675+0000", 2020-08-07T15:13:39.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002 2020-08-07T15:13:39.538 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:39.538 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:39.538 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:39.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.539 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31016 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:13:39.539 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:13:39.539 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 291.72887277799998, 2020-08-07T15:13:39.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971, 2020-08-07T15:13:39.540 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:39.540 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:39.540 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:39.540 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:39.540 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:39.541 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31016 2020-08-07T15:13:39.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.541 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:39.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:39.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:13:39.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:39.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:39.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:13:39.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0924000000000002e-05 2020-08-07T15:13:39.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:39.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734404+0000", 2020-08-07T15:13:39.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2020-08-07T15:13:39.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:39.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734410+0000", 2020-08-07T15:13:39.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3459999999999994e-06 2020-08-07T15:13:39.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:39.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734418+0000", 2020-08-07T15:13:39.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9640000000000002e-06 2020-08-07T15:13:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734423+0000", 2020-08-07T15:13:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7253999999999999e-05 2020-08-07T15:13:39.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:39.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734470+0000", 2020-08-07T15:13:39.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0249999999999997e-06 2020-08-07T15:13:39.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:39.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734477+0000", 2020-08-07T15:13:39.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75294592900000001 2020-08-07T15:13:39.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:39.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487423+0000", 2020-08-07T15:13:39.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031604300000000001 2020-08-07T15:13:39.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:39.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487739+0000", 2020-08-07T15:13:39.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000171149 2020-08-07T15:13:39.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:13:39.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487910+0000", 2020-08-07T15:13:39.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024207769999999998 2020-08-07T15:13:39.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:39.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490331+0000", 2020-08-07T15:13:39.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1650000000000001e-06 2020-08-07T15:13:39.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:39.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490332+0000", 2020-08-07T15:13:39.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9796999999999996e-05 2020-08-07T15:13:39.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:39.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490402+0000", 2020-08-07T15:13:39.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029928849999999998 2020-08-07T15:13:39.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:39.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493395+0000", 2020-08-07T15:13:39.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034903199999999999 2020-08-07T15:13:39.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:39.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493744+0000", 2020-08-07T15:13:39.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8899999999999999e-07 2020-08-07T15:13:39.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:39.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493745+0000", 2020-08-07T15:13:39.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197362 2020-08-07T15:13:39.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:39.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493942+0000", 2020-08-07T15:13:39.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971 2020-08-07T15:13:39.638 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:39.638 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:39.638 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:39.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.639 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:89632 5.f e207/164 5:fd260020:::benchmark_data_smithi074_53697_object11203:head v 207'5744, mlcod=207'5734)", 2020-08-07T15:13:39.639 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:21.551762+0000", 2020-08-07T15:13:39.639 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 136.91146392100001, 2020-08-07T15:13:39.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67527211499999995, 2020-08-07T15:13:39.640 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:39.640 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:39.640 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:39.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:39.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551762+0000", 2020-08-07T15:13:39.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:39.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:39.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551762+0000", 2020-08-07T15:13:39.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8560000000000003e-05 2020-08-07T15:13:39.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:39.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551810+0000", 2020-08-07T15:13:39.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9430000000000002e-06 2020-08-07T15:13:39.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:39.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551814+0000", 2020-08-07T15:13:39.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67230272000000002 2020-08-07T15:13:39.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:39.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.224117+0000", 2020-08-07T15:13:39.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7120000000000003e-06 2020-08-07T15:13:39.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:39.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.224124+0000", 2020-08-07T15:13:39.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001719307 2020-08-07T15:13:39.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:39.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.225843+0000", 2020-08-07T15:13:39.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3902999999999997e-05 2020-08-07T15:13:39.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:39.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.225877+0000", 2020-08-07T15:13:39.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001110279 2020-08-07T15:13:39.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:39.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.226987+0000", 2020-08-07T15:13:39.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6690999999999997e-05 2020-08-07T15:13:39.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:39.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:39.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:39.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.227034+0000", 2020-08-07T15:13:39.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67527211499999995 2020-08-07T15:13:39.748 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:39.748 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:39.748 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:39.748 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:39.748 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:39.749 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:39.749 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:13:39.750 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:38.911+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:39.751 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:13:39.752 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 87 16 128353 128337 11.5222 1.75 0.719698 0.0107631 2020-08-07T15:13:39.752 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:39.750+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:39.863 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:39.853+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:39.924 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:39.924 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:13:39.925 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2020-08-07T15:13:39.925 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:39.943 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:13:39.994 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:39.984+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:40.063 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:40.061+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:40.154 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:40.155 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:13:40.155 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:13:40.155 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:13:40.156 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:40.175 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:13:40.176 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:40.174+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:40.286 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:40.277+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:40.287 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 88 16 130029 130013 11.54 13.0938 0.0147328 0.0108273 2020-08-07T15:13:40.329 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:40.330 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:13:40.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:13:40.330 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:13:40.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.331 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42440 4.9 4:9b3fd4e3:::benchmark_data_smithi074_44586_object5304:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:40.331 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:13:40.331 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 280.08594892799999, 2020-08-07T15:13:40.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9384075030000001, 2020-08-07T15:13:40.331 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:40.332 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:40.332 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:40.332 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:40.332 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:40.332 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42440 2020-08-07T15:13:40.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.333 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:40.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:40.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:13:40.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:40.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:40.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:13:40.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6295000000000002e-05 2020-08-07T15:13:40.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:40.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237116+0000", 2020-08-07T15:13:40.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0390000000000001e-06 2020-08-07T15:13:40.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:40.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237122+0000", 2020-08-07T15:13:40.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8340000000000001e-05 2020-08-07T15:13:40.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:40.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237140+0000", 2020-08-07T15:13:40.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.045e-06 2020-08-07T15:13:40.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:40.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237145+0000", 2020-08-07T15:13:40.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044548920000000002 2020-08-07T15:13:40.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241600+0000", 2020-08-07T15:13:40.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103336 2020-08-07T15:13:40.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:40.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241703+0000", 2020-08-07T15:13:40.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018660399999999999 2020-08-07T15:13:40.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:40.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241890+0000", 2020-08-07T15:13:40.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00096796000000000002 2020-08-07T15:13:40.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:40.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.242858+0000", 2020-08-07T15:13:40.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039845000000000002 2020-08-07T15:13:40.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:40.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246843+0000", 2020-08-07T15:13:40.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3499999999999996e-07 2020-08-07T15:13:40.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:40.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246843+0000", 2020-08-07T15:13:40.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7132999999999995e-05 2020-08-07T15:13:40.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:40.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246920+0000", 2020-08-07T15:13:40.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046993410000000001 2020-08-07T15:13:40.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:40.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.251620+0000", 2020-08-07T15:13:40.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040332956000000003 2020-08-07T15:13:40.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.291953+0000", 2020-08-07T15:13:40.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8390000000000003e-06 2020-08-07T15:13:40.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:40.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.291958+0000", 2020-08-07T15:13:40.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99961357500000003 2020-08-07T15:13:40.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291572+0000", 2020-08-07T15:13:40.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3913999999999999e-05 2020-08-07T15:13:40.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:40.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291586+0000", 2020-08-07T15:13:40.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88379723099999996 2020-08-07T15:13:40.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175383+0000", 2020-08-07T15:13:40.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3868000000000004e-05 2020-08-07T15:13:40.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:40.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175467+0000", 2020-08-07T15:13:40.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9384075030000001 2020-08-07T15:13:40.496 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:40.496 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:40.496 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:40.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.497 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42489 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:40.497 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:13:40.497 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 280.04264886599998, 2020-08-07T15:13:40.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952608639999999, 2020-08-07T15:13:40.498 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:40.498 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:40.498 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:40.498 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:40.499 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:40.499 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42489 2020-08-07T15:13:40.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.501 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:40.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:40.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:13:40.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:40.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:40.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:13:40.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0379e-05 2020-08-07T15:13:40.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:40.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280410+0000", 2020-08-07T15:13:40.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0489999999999998e-06 2020-08-07T15:13:40.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:40.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280417+0000", 2020-08-07T15:13:40.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8281e-05 2020-08-07T15:13:40.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:40.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280435+0000", 2020-08-07T15:13:40.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7510000000000008e-06 2020-08-07T15:13:40.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:40.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280443+0000", 2020-08-07T15:13:40.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018142899999999999 2020-08-07T15:13:40.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280624+0000", 2020-08-07T15:13:40.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013473599999999999 2020-08-07T15:13:40.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:40.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280759+0000", 2020-08-07T15:13:40.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019964200000000001 2020-08-07T15:13:40.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:40.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280959+0000", 2020-08-07T15:13:40.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005588038 2020-08-07T15:13:40.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:40.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286547+0000", 2020-08-07T15:13:40.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.376e-05 2020-08-07T15:13:40.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:40.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286561+0000", 2020-08-07T15:13:40.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4199999999999995e-07 2020-08-07T15:13:40.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:40.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286561+0000", 2020-08-07T15:13:40.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9218e-05 2020-08-07T15:13:40.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:40.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286631+0000", 2020-08-07T15:13:40.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027909199999999999 2020-08-07T15:13:40.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:40.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286910+0000", 2020-08-07T15:13:40.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050904089999999997 2020-08-07T15:13:40.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292000+0000", 2020-08-07T15:13:40.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9579999999999996e-06 2020-08-07T15:13:40.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:40.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292009+0000", 2020-08-07T15:13:40.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99961402799999999 2020-08-07T15:13:40.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291623+0000", 2020-08-07T15:13:40.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2490000000000001e-06 2020-08-07T15:13:40.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:40.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291631+0000", 2020-08-07T15:13:40.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88387422199999999 2020-08-07T15:13:40.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175506+0000", 2020-08-07T15:13:40.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000114881 2020-08-07T15:13:40.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:40.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175620+0000", 2020-08-07T15:13:40.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952608639999999 2020-08-07T15:13:40.559 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:40.562 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:40.562 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:40.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.562 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42490 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:40.572 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:13:40.572 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 280.04252169799997, 2020-08-07T15:13:40.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89528059, 2020-08-07T15:13:40.572 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:40.572 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:40.573 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:40.573 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:40.573 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:40.573 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42490 2020-08-07T15:13:40.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.574 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:40.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:40.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:13:40.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:40.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:40.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:13:40.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2571999999999997e-05 2020-08-07T15:13:40.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:40.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280539+0000", 2020-08-07T15:13:40.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999998e-06 2020-08-07T15:13:40.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:40.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280545+0000", 2020-08-07T15:13:40.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.134e-05 2020-08-07T15:13:40.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:40.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280556+0000", 2020-08-07T15:13:40.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.746e-06 2020-08-07T15:13:40.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:40.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280561+0000", 2020-08-07T15:13:40.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00089258999999999996 2020-08-07T15:13:40.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281453+0000", 2020-08-07T15:13:40.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5641000000000001e-05 2020-08-07T15:13:40.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:40.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281549+0000", 2020-08-07T15:13:40.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020555200000000001 2020-08-07T15:13:40.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:40.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281755+0000", 2020-08-07T15:13:40.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048878230000000003 2020-08-07T15:13:40.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:40.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286642+0000", 2020-08-07T15:13:40.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0954e-05 2020-08-07T15:13:40.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:40.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286693+0000", 2020-08-07T15:13:40.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0699999999999997e-07 2020-08-07T15:13:40.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:40.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286694+0000", 2020-08-07T15:13:40.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3697999999999998e-05 2020-08-07T15:13:40.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:40.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286768+0000", 2020-08-07T15:13:40.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017185900000000001 2020-08-07T15:13:40.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:40.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286940+0000", 2020-08-07T15:13:40.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051207819999999999 2020-08-07T15:13:40.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292060+0000", 2020-08-07T15:13:40.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8000000000000004e-06 2020-08-07T15:13:40.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:40.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292069+0000", 2020-08-07T15:13:40.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99959520599999996 2020-08-07T15:13:40.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291664+0000", 2020-08-07T15:13:40.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0160000000000008e-06 2020-08-07T15:13:40.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:40.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291672+0000", 2020-08-07T15:13:40.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88399374500000005 2020-08-07T15:13:40.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175666+0000", 2020-08-07T15:13:40.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101239 2020-08-07T15:13:40.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:40.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175767+0000", 2020-08-07T15:13:40.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89528059 2020-08-07T15:13:40.640 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:40.640 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:40.641 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:40.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.641 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42491 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:40.641 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:13:40.642 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 280.04241476300001, 2020-08-07T15:13:40.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895291936, 2020-08-07T15:13:40.642 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:40.642 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:40.642 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:40.643 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:40.643 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:40.643 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42491 2020-08-07T15:13:40.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.654 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:40.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:40.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:13:40.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:40.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:40.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:13:40.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0402000000000001e-05 2020-08-07T15:13:40.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:40.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280624+0000", 2020-08-07T15:13:40.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0340000000000001e-06 2020-08-07T15:13:40.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:40.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280627+0000", 2020-08-07T15:13:40.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1886999999999999e-05 2020-08-07T15:13:40.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:40.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280649+0000", 2020-08-07T15:13:40.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5430000000000004e-06 2020-08-07T15:13:40.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:40.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280654+0000", 2020-08-07T15:13:40.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032800960000000001 2020-08-07T15:13:40.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.283934+0000", 2020-08-07T15:13:40.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011264699999999999 2020-08-07T15:13:40.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:40.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284046+0000", 2020-08-07T15:13:40.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019987299999999999 2020-08-07T15:13:40.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:40.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284246+0000", 2020-08-07T15:13:40.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025610960000000001 2020-08-07T15:13:40.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:40.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286807+0000", 2020-08-07T15:13:40.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1409999999999999e-06 2020-08-07T15:13:40.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:40.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286814+0000", 2020-08-07T15:13:40.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0499999999999999e-07 2020-08-07T15:13:40.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:40.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286815+0000", 2020-08-07T15:13:40.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8538000000000003e-05 2020-08-07T15:13:40.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:40.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286853+0000", 2020-08-07T15:13:40.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112914 2020-08-07T15:13:40.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:40.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286966+0000", 2020-08-07T15:13:40.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051403120000000002 2020-08-07T15:13:40.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292107+0000", 2020-08-07T15:13:40.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8019999999999992e-06 2020-08-07T15:13:40.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:40.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292114+0000", 2020-08-07T15:13:40.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.999590859 2020-08-07T15:13:40.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291705+0000", 2020-08-07T15:13:40.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.174e-06 2020-08-07T15:13:40.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:40.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291713+0000", 2020-08-07T15:13:40.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88407676000000002 2020-08-07T15:13:40.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175790+0000", 2020-08-07T15:13:40.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5452999999999996e-05 2020-08-07T15:13:40.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:40.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175886+0000", 2020-08-07T15:13:40.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895291936 2020-08-07T15:13:40.721 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:40.721 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:40.722 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:40.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.723 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42492 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:40.723 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:13:40.723 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 280.04232171299998, 2020-08-07T15:13:40.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895293774, 2020-08-07T15:13:40.724 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:40.724 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:40.724 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:40.724 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:40.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:40.725 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42492 2020-08-07T15:13:40.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.725 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:40.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:40.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:13:40.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:40.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:40.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:13:40.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8272000000000001e-05 2020-08-07T15:13:40.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:40.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280735+0000", 2020-08-07T15:13:40.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2159999999999997e-06 2020-08-07T15:13:40.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:40.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280740+0000", 2020-08-07T15:13:40.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0251000000000001e-05 2020-08-07T15:13:40.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:40.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280750+0000", 2020-08-07T15:13:40.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3139999999999997e-06 2020-08-07T15:13:40.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:40.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280755+0000", 2020-08-07T15:13:40.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039343579999999998 2020-08-07T15:13:40.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284689+0000", 2020-08-07T15:13:40.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111648 2020-08-07T15:13:40.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:40.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284801+0000", 2020-08-07T15:13:40.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019563600000000001 2020-08-07T15:13:40.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:40.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284996+0000", 2020-08-07T15:13:40.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018636709999999999 2020-08-07T15:13:40.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:40.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286860+0000", 2020-08-07T15:13:40.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001865802 2020-08-07T15:13:40.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:40.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288726+0000", 2020-08-07T15:13:40.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0399999999999996e-07 2020-08-07T15:13:40.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:40.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288726+0000", 2020-08-07T15:13:40.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6573999999999998e-05 2020-08-07T15:13:40.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:40.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288773+0000", 2020-08-07T15:13:40.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003455283 2020-08-07T15:13:40.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292228+0000", 2020-08-07T15:13:40.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0099999999999995e-06 2020-08-07T15:13:40.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:40.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292236+0000", 2020-08-07T15:13:40.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99950963999999998 2020-08-07T15:13:40.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291746+0000", 2020-08-07T15:13:40.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0779999999999996e-06 2020-08-07T15:13:40.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:40.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291754+0000", 2020-08-07T15:13:40.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.884152508 2020-08-07T15:13:40.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175906+0000", 2020-08-07T15:13:40.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4009000000000002e-05 2020-08-07T15:13:40.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:40.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175980+0000", 2020-08-07T15:13:40.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895293774 2020-08-07T15:13:40.795 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:40.795 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:40.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:40.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.805 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42493 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:40.806 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:13:40.806 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 280.04222006600003, 2020-08-07T15:13:40.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952963789999999, 2020-08-07T15:13:40.806 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:40.806 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:40.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:40.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:40.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:40.807 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42493 2020-08-07T15:13:40.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.808 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:40.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:40.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:13:40.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:40.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:40.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:13:40.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9740999999999997e-05 2020-08-07T15:13:40.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:40.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280838+0000", 2020-08-07T15:13:40.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7809999999999996e-06 2020-08-07T15:13:40.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:40.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280843+0000", 2020-08-07T15:13:40.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0648999999999999e-05 2020-08-07T15:13:40.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:40.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280854+0000", 2020-08-07T15:13:40.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3669999999999999e-06 2020-08-07T15:13:40.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:40.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280858+0000", 2020-08-07T15:13:40.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004850867 2020-08-07T15:13:40.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.285709+0000", 2020-08-07T15:13:40.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110037 2020-08-07T15:13:40.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:40.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.285819+0000", 2020-08-07T15:13:40.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020935699999999999 2020-08-07T15:13:40.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:40.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286028+0000", 2020-08-07T15:13:40.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020900850000000002 2020-08-07T15:13:40.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:40.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288118+0000", 2020-08-07T15:13:40.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041669649999999999 2020-08-07T15:13:40.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292285+0000", 2020-08-07T15:13:40.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3820000000000004e-06 2020-08-07T15:13:40.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:40.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292294+0000", 2020-08-07T15:13:40.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949242900000002 2020-08-07T15:13:40.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291786+0000", 2020-08-07T15:13:40.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000003e-06 2020-08-07T15:13:40.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:40.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291794+0000", 2020-08-07T15:13:40.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88420652700000002 2020-08-07T15:13:40.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176000+0000", 2020-08-07T15:13:40.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4634000000000003e-05 2020-08-07T15:13:40.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:40.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176085+0000", 2020-08-07T15:13:40.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952963789999999 2020-08-07T15:13:40.879 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:40.880 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:40.880 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:40.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.880 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42494 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:40.881 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:13:40.881 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 280.04211796700002, 2020-08-07T15:13:40.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952882820000001, 2020-08-07T15:13:40.881 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:40.890 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:40.890 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:40.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:40.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:40.891 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42494 2020-08-07T15:13:40.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.891 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:40.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:40.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:13:40.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:40.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:40.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:13:40.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0117999999999997e-05 2020-08-07T15:13:40.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:40.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280941+0000", 2020-08-07T15:13:40.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1240000000000004e-06 2020-08-07T15:13:40.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:40.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280946+0000", 2020-08-07T15:13:40.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0000000000000001e-05 2020-08-07T15:13:40.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:40.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280956+0000", 2020-08-07T15:13:40.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.279e-06 2020-08-07T15:13:40.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:40.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280960+0000", 2020-08-07T15:13:40.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0062727080000000001 2020-08-07T15:13:40.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287233+0000", 2020-08-07T15:13:40.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018362900000000001 2020-08-07T15:13:40.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:40.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287416+0000", 2020-08-07T15:13:40.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029536100000000001 2020-08-07T15:13:40.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:40.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287712+0000", 2020-08-07T15:13:40.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046231739999999999 2020-08-07T15:13:40.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292335+0000", 2020-08-07T15:13:40.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3860000000000001e-06 2020-08-07T15:13:40.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:40.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292342+0000", 2020-08-07T15:13:40.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99948501000000001 2020-08-07T15:13:40.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291827+0000", 2020-08-07T15:13:40.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2272999999999999e-05 2020-08-07T15:13:40.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:40.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291840+0000", 2020-08-07T15:13:40.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88426668100000005 2020-08-07T15:13:40.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176106+0000", 2020-08-07T15:13:40.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2539000000000005e-05 2020-08-07T15:13:40.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:40.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176179+0000", 2020-08-07T15:13:40.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952882820000001 2020-08-07T15:13:40.922 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:40.922 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:40.923 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:40.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.927 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42495 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:40.927 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:13:40.927 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 280.04201687699998, 2020-08-07T15:13:40.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895286867, 2020-08-07T15:13:40.928 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:40.928 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:40.928 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:40.928 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:40.928 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:40.929 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42495 2020-08-07T15:13:40.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.929 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:40.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:40.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:13:40.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:40.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:40.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:13:40.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6690999999999997e-05 2020-08-07T15:13:40.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:40.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281038+0000", 2020-08-07T15:13:40.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7860000000000002e-06 2020-08-07T15:13:40.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:40.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281044+0000", 2020-08-07T15:13:40.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0429e-05 2020-08-07T15:13:40.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:40.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281054+0000", 2020-08-07T15:13:40.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9040000000000002e-06 2020-08-07T15:13:40.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:40.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281058+0000", 2020-08-07T15:13:40.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070941490000000001 2020-08-07T15:13:40.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288153+0000", 2020-08-07T15:13:40.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9006999999999998e-05 2020-08-07T15:13:40.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:40.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288222+0000", 2020-08-07T15:13:40.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012106400000000001 2020-08-07T15:13:40.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:40.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288343+0000", 2020-08-07T15:13:40.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004034752 2020-08-07T15:13:40.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292377+0000", 2020-08-07T15:13:40.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.537e-06 2020-08-07T15:13:40.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:40.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292385+0000", 2020-08-07T15:13:40.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949123500000003 2020-08-07T15:13:40.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291876+0000", 2020-08-07T15:13:40.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7729999999999999e-06 2020-08-07T15:13:40.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:40.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291884+0000", 2020-08-07T15:13:40.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88431700899999999 2020-08-07T15:13:40.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:40.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176201+0000", 2020-08-07T15:13:40.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7531e-05 2020-08-07T15:13:40.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:40.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:40.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:40.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176278+0000", 2020-08-07T15:13:40.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895286867 2020-08-07T15:13:40.997 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:40.997 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:41.010 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:41.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.011 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42496 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:41.012 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:13:41.012 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 280.04191196800002, 2020-08-07T15:13:41.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89526942, 2020-08-07T15:13:41.012 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:41.012 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:41.013 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:41.013 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:41.013 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:41.013 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42496 2020-08-07T15:13:41.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.014 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:41.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:41.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:13:41.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:41.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:41.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:13:41.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9227999999999997e-05 2020-08-07T15:13:41.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:41.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281146+0000", 2020-08-07T15:13:41.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1120000000000004e-06 2020-08-07T15:13:41.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:41.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281151+0000", 2020-08-07T15:13:41.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2208999999999999e-05 2020-08-07T15:13:41.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:41.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281163+0000", 2020-08-07T15:13:41.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5469999999999998e-06 2020-08-07T15:13:41.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:41.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281168+0000", 2020-08-07T15:13:41.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0076520829999999996 2020-08-07T15:13:41.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:41.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288820+0000", 2020-08-07T15:13:41.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5035000000000003e-05 2020-08-07T15:13:41.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:41.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288885+0000", 2020-08-07T15:13:41.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116507 2020-08-07T15:13:41.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:13:41.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.289001+0000", 2020-08-07T15:13:41.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034121960000000002 2020-08-07T15:13:41.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:41.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292413+0000", 2020-08-07T15:13:41.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8109999999999997e-06 2020-08-07T15:13:41.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:41.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292420+0000", 2020-08-07T15:13:41.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949504600000005 2020-08-07T15:13:41.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:41.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291915+0000", 2020-08-07T15:13:41.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3999999999999992e-06 2020-08-07T15:13:41.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:41.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291924+0000", 2020-08-07T15:13:41.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88437394599999997 2020-08-07T15:13:41.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:41.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176298+0000", 2020-08-07T15:13:41.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8300000000000007e-05 2020-08-07T15:13:41.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:41.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176366+0000", 2020-08-07T15:13:41.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89526942 2020-08-07T15:13:41.053 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:41.053 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:41.053 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:41.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.054 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73081 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:13:41.054 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:13:41.054 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 253.09644269399999, 2020-08-07T15:13:41.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0005038420000001, 2020-08-07T15:13:41.055 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:41.055 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:41.055 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:41.055 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:41.056 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:41.056 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73081 2020-08-07T15:13:41.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.058 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:41.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:41.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:13:41.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:41.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:41.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:13:41.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3016000000000003e-05 2020-08-07T15:13:41.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:41.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226609+0000", 2020-08-07T15:13:41.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4479999999999996e-06 2020-08-07T15:13:41.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:41.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226613+0000", 2020-08-07T15:13:41.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0189999999999995e-06 2020-08-07T15:13:41.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:41.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226622+0000", 2020-08-07T15:13:41.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5980000000000001e-06 2020-08-07T15:13:41.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:41.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226626+0000", 2020-08-07T15:13:41.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8088999999999999e-05 2020-08-07T15:13:41.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:41.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226684+0000", 2020-08-07T15:13:41.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2162e-05 2020-08-07T15:13:41.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:41.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226696+0000", 2020-08-07T15:13:41.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98675689 2020-08-07T15:13:41.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:41.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213453+0000", 2020-08-07T15:13:41.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012916800000000001 2020-08-07T15:13:41.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:41.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213582+0000", 2020-08-07T15:13:41.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015665899999999999 2020-08-07T15:13:41.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:13:41.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213739+0000", 2020-08-07T15:13:41.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049940239999999997 2020-08-07T15:13:41.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:41.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218733+0000", 2020-08-07T15:13:41.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0079774779999999997 2020-08-07T15:13:41.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:41.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226710+0000", 2020-08-07T15:13:41.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.161e-06 2020-08-07T15:13:41.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:41.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226711+0000", 2020-08-07T15:13:41.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0105000000000001e-05 2020-08-07T15:13:41.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:41.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226772+0000", 2020-08-07T15:13:41.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5633999999999997e-05 2020-08-07T15:13:41.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:41.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226817+0000", 2020-08-07T15:13:41.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7379000000000001e-05 2020-08-07T15:13:41.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:41.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226875+0000", 2020-08-07T15:13:41.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8000000000000005e-07 2020-08-07T15:13:41.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:41.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226875+0000", 2020-08-07T15:13:41.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019423199999999999 2020-08-07T15:13:41.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:41.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227070+0000", 2020-08-07T15:13:41.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0005038420000001 2020-08-07T15:13:41.103 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:41.103 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:41.104 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:41.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.104 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73082 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:13:41.104 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:13:41.105 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 253.09607223, 2020-08-07T15:13:41.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001326693, 2020-08-07T15:13:41.105 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:41.105 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:41.105 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:41.112 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:41.113 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:41.113 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73082 2020-08-07T15:13:41.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.113 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:41.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:41.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:13:41.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:41.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:41.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:13:41.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6928000000000003e-05 2020-08-07T15:13:41.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:41.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226973+0000", 2020-08-07T15:13:41.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3630000000000004e-06 2020-08-07T15:13:41.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:41.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226977+0000", 2020-08-07T15:13:41.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9659999999999996e-06 2020-08-07T15:13:41.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:41.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226985+0000", 2020-08-07T15:13:41.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5800000000000002e-06 2020-08-07T15:13:41.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:41.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226990+0000", 2020-08-07T15:13:41.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9101999999999998e-05 2020-08-07T15:13:41.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:41.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227039+0000", 2020-08-07T15:13:41.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9620000000000005e-06 2020-08-07T15:13:41.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:41.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227049+0000", 2020-08-07T15:13:41.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98722513999999995 2020-08-07T15:13:41.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:41.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214274+0000", 2020-08-07T15:13:41.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2700999999999994e-05 2020-08-07T15:13:41.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:41.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214347+0000", 2020-08-07T15:13:41.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015346899999999999 2020-08-07T15:13:41.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:13:41.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214500+0000", 2020-08-07T15:13:41.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042450309999999998 2020-08-07T15:13:41.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:41.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218745+0000", 2020-08-07T15:13:41.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008379503 2020-08-07T15:13:41.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:41.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227125+0000", 2020-08-07T15:13:41.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7899999999999998e-07 2020-08-07T15:13:41.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:41.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227126+0000", 2020-08-07T15:13:41.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1610000000000002e-05 2020-08-07T15:13:41.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:41.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227177+0000", 2020-08-07T15:13:41.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489000000000003e-05 2020-08-07T15:13:41.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:41.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227226+0000", 2020-08-07T15:13:41.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00084338600000000001 2020-08-07T15:13:41.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:41.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228069+0000", 2020-08-07T15:13:41.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7e-07 2020-08-07T15:13:41.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:41.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228069+0000", 2020-08-07T15:13:41.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019351399999999999 2020-08-07T15:13:41.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:41.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228263+0000", 2020-08-07T15:13:41.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001326693 2020-08-07T15:13:41.210 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:41.211 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:41.228 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:41.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.245 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73083 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:13:41.245 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:13:41.246 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 253.095977971, 2020-08-07T15:13:41.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015968209999999, 2020-08-07T15:13:41.246 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:41.246 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:41.247 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:41.247 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:41.274 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:41.275 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73083 2020-08-07T15:13:41.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.275 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:41.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:41.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:13:41.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:41.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:41.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:13:41.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1627000000000001e-05 2020-08-07T15:13:41.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:41.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227062+0000", 2020-08-07T15:13:41.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.253e-06 2020-08-07T15:13:41.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:41.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227065+0000", 2020-08-07T15:13:41.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3168e-05 2020-08-07T15:13:41.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:41.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227079+0000", 2020-08-07T15:13:41.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1549999999999999e-06 2020-08-07T15:13:41.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:41.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227082+0000", 2020-08-07T15:13:41.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5534999999999997e-05 2020-08-07T15:13:41.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:41.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227127+0000", 2020-08-07T15:13:41.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6823000000000001e-05 2020-08-07T15:13:41.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:41.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227144+0000", 2020-08-07T15:13:41.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98771770599999997 2020-08-07T15:13:41.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:41.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214862+0000", 2020-08-07T15:13:41.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8252999999999999e-05 2020-08-07T15:13:41.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:41.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214930+0000", 2020-08-07T15:13:41.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013646400000000001 2020-08-07T15:13:41.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:13:41.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215066+0000", 2020-08-07T15:13:41.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036907089999999999 2020-08-07T15:13:41.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:41.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218757+0000", 2020-08-07T15:13:41.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0085133450000000003 2020-08-07T15:13:41.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:41.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227270+0000", 2020-08-07T15:13:41.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4899999999999995e-07 2020-08-07T15:13:41.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:41.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227271+0000", 2020-08-07T15:13:41.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4555999999999997e-05 2020-08-07T15:13:41.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:41.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227326+0000", 2020-08-07T15:13:41.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024594800000000001 2020-08-07T15:13:41.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:41.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227572+0000", 2020-08-07T15:13:41.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00086125700000000004 2020-08-07T15:13:41.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:41.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228433+0000", 2020-08-07T15:13:41.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3799999999999998e-07 2020-08-07T15:13:41.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:41.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228433+0000", 2020-08-07T15:13:41.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019413500000000001 2020-08-07T15:13:41.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:41.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228627+0000", 2020-08-07T15:13:41.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015968209999999 2020-08-07T15:13:41.435 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:41.436 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:41.436 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:41.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.437 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73084 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:13:41.438 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:13:41.438 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 253.095889032, 2020-08-07T15:13:41.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0016919049999999, 2020-08-07T15:13:41.443 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:41.443 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:41.443 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:41.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:41.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:41.444 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73084 2020-08-07T15:13:41.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.447 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:41.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:41.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:13:41.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:41.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:41.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:13:41.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5565999999999997e-05 2020-08-07T15:13:41.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:41.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227165+0000", 2020-08-07T15:13:41.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2020-08-07T15:13:41.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:41.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227169+0000", 2020-08-07T15:13:41.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6850000000000001e-06 2020-08-07T15:13:41.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:41.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227176+0000", 2020-08-07T15:13:41.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1949999999999999e-06 2020-08-07T15:13:41.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:41.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227179+0000", 2020-08-07T15:13:41.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0318000000000003e-05 2020-08-07T15:13:41.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:41.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227220+0000", 2020-08-07T15:13:41.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0349e-05 2020-08-07T15:13:41.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:41.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227230+0000", 2020-08-07T15:13:41.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98832962400000002 2020-08-07T15:13:41.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:41.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215560+0000", 2020-08-07T15:13:41.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013913099999999999 2020-08-07T15:13:41.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:41.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215699+0000", 2020-08-07T15:13:41.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000200006 2020-08-07T15:13:41.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:13:41.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215899+0000", 2020-08-07T15:13:41.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028673650000000002 2020-08-07T15:13:41.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:41.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218766+0000", 2020-08-07T15:13:41.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086101779999999996 2020-08-07T15:13:41.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:41.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227376+0000", 2020-08-07T15:13:41.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0999999999999998e-07 2020-08-07T15:13:41.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:41.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227377+0000", 2020-08-07T15:13:41.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6552999999999997e-05 2020-08-07T15:13:41.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:41.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227454+0000", 2020-08-07T15:13:41.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000421717 2020-08-07T15:13:41.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:41.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227875+0000", 2020-08-07T15:13:41.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00081781300000000003 2020-08-07T15:13:41.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:41.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228693+0000", 2020-08-07T15:13:41.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1699999999999997e-07 2020-08-07T15:13:41.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:41.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228694+0000", 2020-08-07T15:13:41.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117483 2020-08-07T15:13:41.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:41.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228811+0000", 2020-08-07T15:13:41.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0016919049999999 2020-08-07T15:13:41.469 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:41.469 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:41.469 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:41.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.470 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73085 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:13:41.470 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:13:41.470 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 253.09575321700001, 2020-08-07T15:13:41.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0018015629999999, 2020-08-07T15:13:41.471 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:41.471 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:41.471 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:41.471 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:41.472 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:41.472 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73085 2020-08-07T15:13:41.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.472 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:41.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:41.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:13:41.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:41.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:41.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:13:41.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6570999999999997e-05 2020-08-07T15:13:41.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:41.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227322+0000", 2020-08-07T15:13:41.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5509999999999996e-06 2020-08-07T15:13:41.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:41.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227328+0000", 2020-08-07T15:13:41.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2498e-05 2020-08-07T15:13:41.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:41.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227341+0000", 2020-08-07T15:13:41.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4389999999999999e-06 2020-08-07T15:13:41.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:41.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227346+0000", 2020-08-07T15:13:41.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3711999999999998e-05 2020-08-07T15:13:41.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:41.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227390+0000", 2020-08-07T15:13:41.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1868999999999999e-05 2020-08-07T15:13:41.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:41.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227402+0000", 2020-08-07T15:13:41.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98896525099999999 2020-08-07T15:13:41.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:41.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216367+0000", 2020-08-07T15:13:41.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012573 2020-08-07T15:13:41.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:41.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216493+0000", 2020-08-07T15:13:41.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020525200000000001 2020-08-07T15:13:41.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:13:41.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216698+0000", 2020-08-07T15:13:41.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037596470000000001 2020-08-07T15:13:41.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:41.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.220458+0000", 2020-08-07T15:13:41.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070227190000000002 2020-08-07T15:13:41.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:41.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227480+0000", 2020-08-07T15:13:41.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6600000000000003e-07 2020-08-07T15:13:41.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:41.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227481+0000", 2020-08-07T15:13:41.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9898000000000003e-05 2020-08-07T15:13:41.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:41.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227521+0000", 2020-08-07T15:13:41.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039130000000000002 2020-08-07T15:13:41.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:41.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227912+0000", 2020-08-07T15:13:41.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00095502399999999996 2020-08-07T15:13:41.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:41.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228867+0000", 2020-08-07T15:13:41.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2020-08-07T15:13:41.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:41.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228867+0000", 2020-08-07T15:13:41.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018933599999999999 2020-08-07T15:13:41.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:41.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229057+0000", 2020-08-07T15:13:41.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0018015629999999 2020-08-07T15:13:41.598 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:41.598 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:41.599 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:41.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.599 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73086 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:13:41.600 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:13:41.600 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 253.09562072099999, 2020-08-07T15:13:41.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002364923, 2020-08-07T15:13:41.600 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:41.600 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:41.601 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:41.601 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:41.601 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:41.601 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73086 2020-08-07T15:13:41.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.602 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:41.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:41.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:13:41.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:41.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:41.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:13:41.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000238426 2020-08-07T15:13:41.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:41.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227626+0000", 2020-08-07T15:13:41.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9609999999999999e-06 2020-08-07T15:13:41.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:41.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227633+0000", 2020-08-07T15:13:41.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2727e-05 2020-08-07T15:13:41.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:41.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227646+0000", 2020-08-07T15:13:41.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1739999999999997e-06 2020-08-07T15:13:41.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:41.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227652+0000", 2020-08-07T15:13:41.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5004000000000001e-05 2020-08-07T15:13:41.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:41.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227737+0000", 2020-08-07T15:13:41.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2744e-05 2020-08-07T15:13:41.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:41.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227750+0000", 2020-08-07T15:13:41.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98938776500000003 2020-08-07T15:13:41.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:41.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217138+0000", 2020-08-07T15:13:41.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012361499999999999 2020-08-07T15:13:41.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:41.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217261+0000", 2020-08-07T15:13:41.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000212504 2020-08-07T15:13:41.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:13:41.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217474+0000", 2020-08-07T15:13:41.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002995036 2020-08-07T15:13:41.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:41.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.220469+0000", 2020-08-07T15:13:41.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071283249999999996 2020-08-07T15:13:41.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:41.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227597+0000", 2020-08-07T15:13:41.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0499999999999999e-07 2020-08-07T15:13:41.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:41.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227597+0000", 2020-08-07T15:13:41.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1872999999999997e-05 2020-08-07T15:13:41.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:41.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227639+0000", 2020-08-07T15:13:41.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032010200000000001 2020-08-07T15:13:41.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:41.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227959+0000", 2020-08-07T15:13:41.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015980899999999999 2020-08-07T15:13:41.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:41.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229557+0000", 2020-08-07T15:13:41.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9299999999999999e-07 2020-08-07T15:13:41.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:41.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229558+0000", 2020-08-07T15:13:41.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019477900000000001 2020-08-07T15:13:41.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:41.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229753+0000", 2020-08-07T15:13:41.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002364923 2020-08-07T15:13:41.648 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:41.648 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:41.648 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:41.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.649 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73087 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:13:41.649 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:13:41.649 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 253.094679568, 2020-08-07T15:13:41.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001896506, 2020-08-07T15:13:41.650 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:41.650 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:41.650 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:41.650 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:41.650 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:41.651 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73087 2020-08-07T15:13:41.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.651 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:41.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:41.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:13:41.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:41.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:41.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:13:41.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7354e-05 2020-08-07T15:13:41.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:41.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228386+0000", 2020-08-07T15:13:41.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8000000000000001e-06 2020-08-07T15:13:41.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:41.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228393+0000", 2020-08-07T15:13:41.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3784999999999999e-05 2020-08-07T15:13:41.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:41.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228407+0000", 2020-08-07T15:13:41.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8090000000000004e-06 2020-08-07T15:13:41.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:41.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228414+0000", 2020-08-07T15:13:41.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6907000000000003e-05 2020-08-07T15:13:41.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:41.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228461+0000", 2020-08-07T15:13:41.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8929999999999994e-06 2020-08-07T15:13:41.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:41.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228469+0000", 2020-08-07T15:13:41.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98950057700000005 2020-08-07T15:13:41.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:41.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217970+0000", 2020-08-07T15:13:41.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000132182 2020-08-07T15:13:41.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:41.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218102+0000", 2020-08-07T15:13:41.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020073399999999999 2020-08-07T15:13:41.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:13:41.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218303+0000", 2020-08-07T15:13:41.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035951120000000001 2020-08-07T15:13:41.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:41.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.221898+0000", 2020-08-07T15:13:41.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057898660000000003 2020-08-07T15:13:41.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:41.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227688+0000", 2020-08-07T15:13:41.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0200000000000002e-07 2020-08-07T15:13:41.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:41.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227688+0000", 2020-08-07T15:13:41.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1477999999999996e-05 2020-08-07T15:13:41.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:41.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227760+0000", 2020-08-07T15:13:41.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016657480000000001 2020-08-07T15:13:41.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:41.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229426+0000", 2020-08-07T15:13:41.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061290400000000005 2020-08-07T15:13:41.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:41.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230039+0000", 2020-08-07T15:13:41.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2e-07 2020-08-07T15:13:41.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:41.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230039+0000", 2020-08-07T15:13:41.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018643500000000001 2020-08-07T15:13:41.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:41.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230225+0000", 2020-08-07T15:13:41.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001896506 2020-08-07T15:13:41.674 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:41.675 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:41.675 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:41.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.675 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73088 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:13:41.676 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:13:41.676 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 253.09428887799999, 2020-08-07T15:13:41.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001627153, 2020-08-07T15:13:41.676 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:41.676 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:41.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:41.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:41.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:41.677 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73088 2020-08-07T15:13:41.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.678 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:41.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:41.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:13:41.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:41.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:41.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:13:41.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7741000000000003e-05 2020-08-07T15:13:41.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:41.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228767+0000", 2020-08-07T15:13:41.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6409999999999999e-06 2020-08-07T15:13:41.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:41.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228773+0000", 2020-08-07T15:13:41.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0874e-05 2020-08-07T15:13:41.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:41.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228784+0000", 2020-08-07T15:13:41.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1529999999999996e-06 2020-08-07T15:13:41.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:41.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228789+0000", 2020-08-07T15:13:41.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3945e-05 2020-08-07T15:13:41.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:41.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228833+0000", 2020-08-07T15:13:41.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4020000000000002e-06 2020-08-07T15:13:41.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:41.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228841+0000", 2020-08-07T15:13:41.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98996913900000005 2020-08-07T15:13:41.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:41.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218810+0000", 2020-08-07T15:13:41.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112484 2020-08-07T15:13:41.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:41.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218923+0000", 2020-08-07T15:13:41.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019891 2020-08-07T15:13:41.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:13:41.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.219122+0000", 2020-08-07T15:13:41.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027884099999999998 2020-08-07T15:13:41.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:41.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.221910+0000", 2020-08-07T15:13:41.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058753399999999997 2020-08-07T15:13:41.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:41.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227786+0000", 2020-08-07T15:13:41.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9200000000000002e-07 2020-08-07T15:13:41.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:41.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227786+0000", 2020-08-07T15:13:41.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8572000000000003e-05 2020-08-07T15:13:41.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:41.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227824+0000", 2020-08-07T15:13:41.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001651296 2020-08-07T15:13:41.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:41.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229476+0000", 2020-08-07T15:13:41.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00077620399999999996 2020-08-07T15:13:41.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:41.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230252+0000", 2020-08-07T15:13:41.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1600000000000002e-07 2020-08-07T15:13:41.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:41.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230252+0000", 2020-08-07T15:13:41.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4433999999999997e-05 2020-08-07T15:13:41.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:41.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230347+0000", 2020-08-07T15:13:41.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001627153 2020-08-07T15:13:41.699 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:41.699 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:41.699 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:41.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.700 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:128738 4.9 4.370c6199 (undecoded) ondisk+write+known_if_redirected e155)", 2020-08-07T15:13:41.700 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:03.204387+0000", 2020-08-07T15:13:41.700 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 217.118621225, 2020-08-07T15:13:41.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99077720899999999, 2020-08-07T15:13:41.701 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:41.701 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:41.701 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:41.701 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:41.702 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:41.702 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128738 2020-08-07T15:13:41.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.702 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:41.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:41.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204387+0000", 2020-08-07T15:13:41.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:41.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:41.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204387+0000", 2020-08-07T15:13:41.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4561999999999999e-05 2020-08-07T15:13:41.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:41.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204422+0000", 2020-08-07T15:13:41.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7270000000000001e-06 2020-08-07T15:13:41.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:41.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204428+0000", 2020-08-07T15:13:41.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3451e-05 2020-08-07T15:13:41.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:41.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204442+0000", 2020-08-07T15:13:41.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1589999999999998e-06 2020-08-07T15:13:41.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:41.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204448+0000", 2020-08-07T15:13:41.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3755999999999999e-05 2020-08-07T15:13:41.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:41.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204492+0000", 2020-08-07T15:13:41.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2450000000000001e-06 2020-08-07T15:13:41.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:41.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204499+0000", 2020-08-07T15:13:41.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99065224600000001 2020-08-07T15:13:41.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:41.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195151+0000", 2020-08-07T15:13:41.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3063e-05 2020-08-07T15:13:41.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:41.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195164+0000", 2020-08-07T15:13:41.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99077720899999999 2020-08-07T15:13:41.719 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:41.719 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:41.719 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:41.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.720 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:128742 4.9 4.69b70ab9 (undecoded) ondisk+write+known_if_redirected e155)", 2020-08-07T15:13:41.720 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:03.205224+0000", 2020-08-07T15:13:41.720 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 217.117784591, 2020-08-07T15:13:41.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.989964548, 2020-08-07T15:13:41.720 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:41.721 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:41.722 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:41.722 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:41.722 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:41.722 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128742 2020-08-07T15:13:41.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.723 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:41.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:41.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205224+0000", 2020-08-07T15:13:41.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:41.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:41.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205224+0000", 2020-08-07T15:13:41.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1018e-05 2020-08-07T15:13:41.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:41.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205255+0000", 2020-08-07T15:13:41.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5840000000000002e-06 2020-08-07T15:13:41.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:41.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205260+0000", 2020-08-07T15:13:41.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2388999999999999e-05 2020-08-07T15:13:41.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:41.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205273+0000", 2020-08-07T15:13:41.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3799999999999999e-06 2020-08-07T15:13:41.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:41.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205279+0000", 2020-08-07T15:13:41.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6684999999999998e-05 2020-08-07T15:13:41.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:41.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205326+0000", 2020-08-07T15:13:41.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6200000000000004e-06 2020-08-07T15:13:41.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:41.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205332+0000", 2020-08-07T15:13:41.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98984944600000002 2020-08-07T15:13:41.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:41.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195181+0000", 2020-08-07T15:13:41.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4259999999999997e-06 2020-08-07T15:13:41.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:41.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195188+0000", 2020-08-07T15:13:41.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.989964548 2020-08-07T15:13:41.777 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:41.778 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:41.778 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:41.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.778 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:128743 4.9 4.1824cba9 (undecoded) ondisk+write+known_if_redirected e155)", 2020-08-07T15:13:41.779 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:03.205500+0000", 2020-08-07T15:13:41.779 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 217.11750872900001, 2020-08-07T15:13:41.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98971761999999996, 2020-08-07T15:13:41.779 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:41.779 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:41.780 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:41.780 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:41.780 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:41.780 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128743 2020-08-07T15:13:41.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.781 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:41.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:41.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205500+0000", 2020-08-07T15:13:41.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:41.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:41.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205500+0000", 2020-08-07T15:13:41.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6441000000000001e-05 2020-08-07T15:13:41.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:41.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205516+0000", 2020-08-07T15:13:41.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4429999999999999e-06 2020-08-07T15:13:41.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:41.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205520+0000", 2020-08-07T15:13:41.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4590000000000003e-06 2020-08-07T15:13:41.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:41.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205526+0000", 2020-08-07T15:13:41.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0429999999999999e-06 2020-08-07T15:13:41.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:41.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205529+0000", 2020-08-07T15:13:41.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1313000000000003e-05 2020-08-07T15:13:41.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:41.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205560+0000", 2020-08-07T15:13:41.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.934e-06 2020-08-07T15:13:41.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:41.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205567+0000", 2020-08-07T15:13:41.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98964017599999998 2020-08-07T15:13:41.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:41.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195208+0000", 2020-08-07T15:13:41.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8109999999999997e-06 2020-08-07T15:13:41.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:41.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195217+0000", 2020-08-07T15:13:41.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98971761999999996 2020-08-07T15:13:41.794 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:41.795 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:41.795 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:41.795 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:41.795 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:41.795 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:41.796 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2020-08-07T15:13:41.796 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:13:41.797 INFO:tasks.thrashosds.thrasher:inject_pause on 1 2020-08-07T15:13:41.797 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2020-08-07T15:13:41.797 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2020-08-07T15:13:41.798 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2020-08-07T15:13:41.800 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:13:41.800 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:13:41.802 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 89 16 131790 131774 11.5649 13.7578 0.00863678 0.0108041 2020-08-07T15:13:41.803 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:13:41.803 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:41.800+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:41.904 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:41.901+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:41.992 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:41.993 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:13:41.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.993 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:132815 6.5 6:a0ff1a0e:::benchmark_data_smithi074_63843_object16601:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e294)", 2020-08-07T15:13:41.994 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:13:41.978133+0000", 2020-08-07T15:13:41.994 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0047001480000000003, 2020-08-07T15:13:41.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0047302730000000001, 2020-08-07T15:13:41.994 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:41.994 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:13:41.995 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:41.995 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:13:41.995 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:41.996 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 132815 2020-08-07T15:13:41.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.996 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:41.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:41.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.978133+0000", 2020-08-07T15:13:41.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:41.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:41.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:41.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:42.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.978133+0000", 2020-08-07T15:13:42.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0947e-05 2020-08-07T15:13:42.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:42.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.978164+0000", 2020-08-07T15:13:42.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6569999999999999e-06 2020-08-07T15:13:42.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:42.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.978168+0000", 2020-08-07T15:13:42.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1809999999999998e-06 2020-08-07T15:13:42.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:42.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.978174+0000", 2020-08-07T15:13:42.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3060000000000002e-06 2020-08-07T15:13:42.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:42.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.978177+0000", 2020-08-07T15:13:42.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00074641400000000004 2020-08-07T15:13:42.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:42.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.978923+0000", 2020-08-07T15:13:42.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2717000000000003e-05 2020-08-07T15:13:42.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:42.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.978986+0000", 2020-08-07T15:13:42.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010959300000000001 2020-08-07T15:13:42.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,7", 2020-08-07T15:13:42.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.979096+0000", 2020-08-07T15:13:42.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000962815 2020-08-07T15:13:42.045 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:42.046 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:42.046 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:42.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.047 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:132816 6.5 6:a0ff1a0e:::benchmark_data_smithi074_63843_object16601:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e294)", 2020-08-07T15:13:42.047 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:13:41.978203+0000", 2020-08-07T15:13:42.048 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0046300719999999998, 2020-08-07T15:13:42.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0047546450000000001, 2020-08-07T15:13:42.048 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:42.048 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:13:42.049 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:42.049 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:13:42.050 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:42.050 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 132816 2020-08-07T15:13:42.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.050 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:42.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:42.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.978203+0000", 2020-08-07T15:13:42.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:42.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:42.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.978203+0000", 2020-08-07T15:13:42.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.633e-05 2020-08-07T15:13:42.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:42.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.978229+0000", 2020-08-07T15:13:42.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.112e-06 2020-08-07T15:13:42.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:42.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.978233+0000", 2020-08-07T15:13:42.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6690000000000003e-06 2020-08-07T15:13:42.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:42.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.978238+0000", 2020-08-07T15:13:42.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5720000000000001e-06 2020-08-07T15:13:42.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:42.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.978241+0000", 2020-08-07T15:13:42.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001514046 2020-08-07T15:13:42.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:42.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.979755+0000", 2020-08-07T15:13:42.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3087000000000001e-05 2020-08-07T15:13:42.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:42.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.979818+0000", 2020-08-07T15:13:42.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013118799999999999 2020-08-07T15:13:42.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,7", 2020-08-07T15:13:42.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.979949+0000", 2020-08-07T15:13:42.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001746004 2020-08-07T15:13:42.094 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:42.094 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:42.095 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:42.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.098 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5481.0:132805 6.1 e294/230 6:8e60b1c5:::benchmark_data_smithi074_63843_object16600:head v 294'8277, mlcod=294'8274)", 2020-08-07T15:13:42.098 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:13:41.979423+0000", 2020-08-07T15:13:42.099 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0034104880000000001, 2020-08-07T15:13:42.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036279300000000001, 2020-08-07T15:13:42.099 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:42.100 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:42.100 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:42.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:42.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.979423+0000", 2020-08-07T15:13:42.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:42.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:42.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.979423+0000", 2020-08-07T15:13:42.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9170999999999999e-05 2020-08-07T15:13:42.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:42.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.979452+0000", 2020-08-07T15:13:42.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9149999999999998e-06 2020-08-07T15:13:42.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:42.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.979456+0000", 2020-08-07T15:13:42.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0482999999999999e-05 2020-08-07T15:13:42.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:42.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.979486+0000", 2020-08-07T15:13:42.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7799999999999998e-06 2020-08-07T15:13:42.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:42.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.979490+0000", 2020-08-07T15:13:42.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9040999999999999e-05 2020-08-07T15:13:42.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:42.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.979519+0000", 2020-08-07T15:13:42.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8929000000000001e-05 2020-08-07T15:13:42.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:42.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.979538+0000", 2020-08-07T15:13:42.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000115319 2020-08-07T15:13:42.158 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:42.158 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:42.158 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:42.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.159 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:132752 6.3 6:cb6c12aa:::benchmark_data_smithi074_63843_object16593:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e294)", 2020-08-07T15:13:42.159 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:13:41.936763+0000", 2020-08-07T15:13:42.159 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.046070180000000002, 2020-08-07T15:13:42.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046370750000000002, 2020-08-07T15:13:42.159 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:42.160 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:13:42.160 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:42.160 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:13:42.160 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:42.161 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 132752 2020-08-07T15:13:42.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.161 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:42.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:42.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.936763+0000", 2020-08-07T15:13:42.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:42.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:42.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.936763+0000", 2020-08-07T15:13:42.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5446e-05 2020-08-07T15:13:42.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:42.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.936788+0000", 2020-08-07T15:13:42.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6740000000000001e-06 2020-08-07T15:13:42.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:42.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.936791+0000", 2020-08-07T15:13:42.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1080000000000001e-06 2020-08-07T15:13:42.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:42.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.936796+0000", 2020-08-07T15:13:42.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.711e-06 2020-08-07T15:13:42.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:42.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.936799+0000", 2020-08-07T15:13:42.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042148929999999999 2020-08-07T15:13:42.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:42.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.941014+0000", 2020-08-07T15:13:42.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8035999999999994e-05 2020-08-07T15:13:42.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:42.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.941112+0000", 2020-08-07T15:13:42.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016571700000000001 2020-08-07T15:13:42.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,7", 2020-08-07T15:13:42.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.941278+0000", 2020-08-07T15:13:42.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016725690000000001 2020-08-07T15:13:42.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:42.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.942950+0000", 2020-08-07T15:13:42.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015061429999999999 2020-08-07T15:13:42.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:42.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.944456+0000", 2020-08-07T15:13:42.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.007693297 2020-08-07T15:13:42.246 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:42.246 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:42.246 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:42.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.247 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5481.0:132806 6.1 e294/230 6:8e60b1c5:::benchmark_data_smithi074_63843_object16600:head v 294'8278, mlcod=294'8275)", 2020-08-07T15:13:42.247 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:13:41.980517+0000", 2020-08-07T15:13:42.247 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.002315779, 2020-08-07T15:13:42.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002717049, 2020-08-07T15:13:42.263 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:42.263 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:42.264 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:42.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:42.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.980517+0000", 2020-08-07T15:13:42.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:42.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:42.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.980517+0000", 2020-08-07T15:13:42.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0104e-05 2020-08-07T15:13:42.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:42.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.980567+0000", 2020-08-07T15:13:42.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1519999999999997e-06 2020-08-07T15:13:42.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:42.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.980574+0000", 2020-08-07T15:13:42.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1659000000000001e-05 2020-08-07T15:13:42.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:42.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.980625+0000", 2020-08-07T15:13:42.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.383e-06 2020-08-07T15:13:42.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:42.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.980631+0000", 2020-08-07T15:13:42.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0525999999999998e-05 2020-08-07T15:13:42.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:42.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.980671+0000", 2020-08-07T15:13:42.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1246999999999998e-05 2020-08-07T15:13:42.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:42.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.980692+0000", 2020-08-07T15:13:42.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000175071 2020-08-07T15:13:42.298 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:42.298 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:42.306 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:42.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.308 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5481.0:132807 6.1 e294/230 6:8e60b1c5:::benchmark_data_smithi074_63843_object16600:head v 294'8279, mlcod=294'8275)", 2020-08-07T15:13:42.308 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:13:41.981384+0000", 2020-08-07T15:13:42.308 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0014486939999999999, 2020-08-07T15:13:42.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019295339999999999, 2020-08-07T15:13:42.309 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:42.309 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:42.316 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:42.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:42.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.981384+0000", 2020-08-07T15:13:42.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:42.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:42.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.981384+0000", 2020-08-07T15:13:42.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6991999999999999e-05 2020-08-07T15:13:42.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:42.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.981441+0000", 2020-08-07T15:13:42.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7270000000000001e-06 2020-08-07T15:13:42.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:42.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.981448+0000", 2020-08-07T15:13:42.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018123999999999999 2020-08-07T15:13:42.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:42.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.981629+0000", 2020-08-07T15:13:42.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1749999999999999e-06 2020-08-07T15:13:42.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:42.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.981637+0000", 2020-08-07T15:13:42.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6480999999999999e-05 2020-08-07T15:13:42.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:42.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.981673+0000", 2020-08-07T15:13:42.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2583999999999999e-05 2020-08-07T15:13:42.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:42.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:42.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:42.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:41.981696+0000", 2020-08-07T15:13:42.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031119900000000001 2020-08-07T15:13:42.351 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:42.351 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:42.352 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:42.352 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:42.352 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:13:42.353 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 6 2020-08-07T15:13:42.353 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:42.380 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:42.381 INFO:teuthology.orchestra.run.smithi074.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2020-08-07T15:13:42.381 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:42.381 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:13:42.382 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:13:42.382 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:13:42.382 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:13:42.382 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:13:42.382 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:13:42.383 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:13:42.383 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:13:42.383 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 90 4 133249 133245 11.5641 11.4922 0.00396841 0.0108032 2020-08-07T15:13:42.383 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:13:42.384 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:13:42.384 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:13:42.385 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:13:42.385 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:13:42.385 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:13:42.385 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:13:42.386 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:13:42.386 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:13:42.387 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:42.355+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:42.471 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:42.469+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:42.543 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:42.544 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:42.564 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:13:42.565 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:42.561+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:42.670 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:42.669+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:42.722 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:42.723 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:42.747 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:13:42.774 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:42.772+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:42.874 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:42.875 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:42.876 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:42.874+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:42.876 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:13:42.980 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:42.977+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:43.008 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:43.009 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:43.013 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:13:43.110 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:43.109+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:43.179 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:43.179 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:43.209 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:43.203+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:43.210 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:13:43.249 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:13:43.284 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Total time run: 90.3627 2020-08-07T15:13:43.285 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Total writes made: 133249 2020-08-07T15:13:43.285 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Write size: 8192 2020-08-07T15:13:43.285 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Object size: 65536 2020-08-07T15:13:43.285 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Bandwidth (MB/sec): 11.5203 2020-08-07T15:13:43.286 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Stddev Bandwidth: 2.19128 2020-08-07T15:13:43.286 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Max bandwidth (MB/sec): 14.5469 2020-08-07T15:13:43.286 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Min bandwidth (MB/sec): 0.015625 2020-08-07T15:13:43.286 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Average IOPS: 1474 2020-08-07T15:13:43.286 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Stddev IOPS: 280.484 2020-08-07T15:13:43.287 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Max IOPS: 1862 2020-08-07T15:13:43.287 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Min IOPS: 2 2020-08-07T15:13:43.287 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Average Latency(s): 0.0108148 2020-08-07T15:13:43.287 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Stddev Latency(s): 0.0256699 2020-08-07T15:13:43.287 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Max latency(s): 1.58913 2020-08-07T15:13:43.288 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Min latency(s): 0.000909485 2020-08-07T15:13:43.288 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Cleaning up (deleting benchmark objects) 2020-08-07T15:13:43.307 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:43.306+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:43.357 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:43.359 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:43.379 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:13:43.420 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:43.419+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:43.522 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:43.518+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:43.636 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:43.622+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:43.637 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:43.637 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:43.659 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:13:43.762 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:43.758+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:43.832 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:43.826+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:43.941 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:43.927+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:43.995 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:43.995 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:43.996 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:13:44.042 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:44.034+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:44.136 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:44.134+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:44.242 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:44.238+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:44.273 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:44.274 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:44.288 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:13:44.376 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:44.372+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:44.390 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:13:44.446 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:44.445+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:44.551 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:44.546+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:44.577 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:44.586 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:44.596 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:13:44.658 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:44.652+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:44.760 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:44.753+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:44.782 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:13:44.837 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:13:44.860 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:44.857+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:44.892 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:44.893 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:13:44.893 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:13:44.893 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:13:44.894 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:44.894 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:145801 3.6 e75/26 3:69e4cb22:::benchmark_data_smithi074_35447_object16192:head v 75'9405, mlcod=75'9401)", 2020-08-07T15:13:44.894 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:13:44.895 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 330.20854526400001, 2020-08-07T15:13:44.895 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639, 2020-08-07T15:13:44.895 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:44.895 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:44.896 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:44.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:44.896 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:44.897 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:13:44.897 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:44.897 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:44.898 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:44.910 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:44.910 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:13:44.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6511000000000002e-05 2020-08-07T15:13:44.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:44.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:44.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:44.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671803+0000", 2020-08-07T15:13:44.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7190000000000004e-06 2020-08-07T15:13:44.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:44.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:44.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:44.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671809+0000", 2020-08-07T15:13:44.914 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74210928300000001 2020-08-07T15:13:44.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:44.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:44.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:44.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413919+0000", 2020-08-07T15:13:44.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1759999999999999e-05 2020-08-07T15:13:44.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:44.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:44.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:44.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413930+0000", 2020-08-07T15:13:44.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000108601 2020-08-07T15:13:44.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:44.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:44.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:44.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414039+0000", 2020-08-07T15:13:44.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9948999999999999e-05 2020-08-07T15:13:44.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:44.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:44.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:44.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414089+0000", 2020-08-07T15:13:44.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00079656499999999995 2020-08-07T15:13:44.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:44.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:44.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:44.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414885+0000", 2020-08-07T15:13:44.946 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2250999999999998e-05 2020-08-07T15:13:44.946 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:44.946 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:44.946 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:44.947 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414938+0000", 2020-08-07T15:13:44.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639 2020-08-07T15:13:44.947 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:44.947 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:44.948 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:44.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:44.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:44.948 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:104104 4.0 e146/131 4:0ffb764a:::benchmark_data_smithi074_44586_object13012:head v 146'6960, mlcod=146'6953)", 2020-08-07T15:13:44.948 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:13:44.949 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 236.681998488, 2020-08-07T15:13:44.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75443331599999996, 2020-08-07T15:13:44.949 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:44.949 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:44.949 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:44.950 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:44.950 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:44.950 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:13:44.950 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:44.950 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:44.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:44.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:44.951 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:13:44.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1743000000000002e-05 2020-08-07T15:13:44.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:44.952 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:44.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:44.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198355+0000", 2020-08-07T15:13:44.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1249999999999998e-06 2020-08-07T15:13:44.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:44.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:44.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:44.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198361+0000", 2020-08-07T15:13:44.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74719179599999996 2020-08-07T15:13:44.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:44.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:44.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:44.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945553+0000", 2020-08-07T15:13:44.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6800000000000001e-06 2020-08-07T15:13:44.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:44.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:44.993 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:44.993 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945557+0000", 2020-08-07T15:13:44.993 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012101500000000001 2020-08-07T15:13:44.994 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:44.994 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:44.994 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:44.995 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945678+0000", 2020-08-07T15:13:44.995 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6621000000000001e-05 2020-08-07T15:13:44.995 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:44.996 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:44.996 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:44.996 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945705+0000", 2020-08-07T15:13:44.997 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069879540000000002 2020-08-07T15:13:44.997 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:44.997 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:44.998 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:44.998 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.952693+0000", 2020-08-07T15:13:44.998 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3381999999999999e-05 2020-08-07T15:13:44.999 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:44.999 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:44.999 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:45.000 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.952736+0000", 2020-08-07T15:13:45.000 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75443331599999996 2020-08-07T15:13:45.000 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.001 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:45.001 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.001 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.002 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.022 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89641 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:45.022 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.550933+0000", 2020-08-07T15:13:45.023 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 143.32936806699999, 2020-08-07T15:13:45.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69051440900000005, 2020-08-07T15:13:45.023 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:45.023 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:45.024 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:45.024 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:45.024 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:45.024 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89641 2020-08-07T15:13:45.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.025 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:45.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:45.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550933+0000", 2020-08-07T15:13:45.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:45.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:45.026 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550933+0000", 2020-08-07T15:13:45.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6520000000000002e-05 2020-08-07T15:13:45.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:45.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550980+0000", 2020-08-07T15:13:45.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2410000000000003e-06 2020-08-07T15:13:45.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:45.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550985+0000", 2020-08-07T15:13:45.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2707e-05 2020-08-07T15:13:45.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:45.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550998+0000", 2020-08-07T15:13:45.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8629999999999996e-06 2020-08-07T15:13:45.030 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.030 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.030 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:45.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551004+0000", 2020-08-07T15:13:45.030 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4987000000000001e-05 2020-08-07T15:13:45.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551059+0000", 2020-08-07T15:13:45.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1718e-05 2020-08-07T15:13:45.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:45.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551070+0000", 2020-08-07T15:13:45.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67720431000000003 2020-08-07T15:13:45.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.033 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.228275+0000", 2020-08-07T15:13:45.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000731965 2020-08-07T15:13:45.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.042 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:45.042 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229007+0000", 2020-08-07T15:13:45.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00025396199999999999 2020-08-07T15:13:45.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.043 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:45.043 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229261+0000", 2020-08-07T15:13:45.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002511189 2020-08-07T15:13:45.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.045 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:45.045 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231772+0000", 2020-08-07T15:13:45.045 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3399999999999997e-07 2020-08-07T15:13:45.045 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.046 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.046 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:45.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231773+0000", 2020-08-07T15:13:45.046 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0274e-05 2020-08-07T15:13:45.046 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.047 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:45.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231823+0000", 2020-08-07T15:13:45.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0094095440000000006 2020-08-07T15:13:45.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241233+0000", 2020-08-07T15:13:45.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4011999999999998e-05 2020-08-07T15:13:45.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.057 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241277+0000", 2020-08-07T15:13:45.071 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7199999999999999e-07 2020-08-07T15:13:45.071 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.071 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.072 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:45.072 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241277+0000", 2020-08-07T15:13:45.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017061100000000001 2020-08-07T15:13:45.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:45.089 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241448+0000", 2020-08-07T15:13:45.089 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69051440900000005 2020-08-07T15:13:45.090 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.090 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:45.090 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.090 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.091 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.091 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89642 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:45.091 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.551807+0000", 2020-08-07T15:13:45.092 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 143.32849451499999, 2020-08-07T15:13:45.092 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68991613100000004, 2020-08-07T15:13:45.092 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:45.092 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:45.093 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:45.093 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:45.093 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:45.139 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89642 2020-08-07T15:13:45.139 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.140 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:45.140 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.147 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:45.147 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551807+0000", 2020-08-07T15:13:45.147 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:45.150 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.150 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.151 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:45.152 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551807+0000", 2020-08-07T15:13:45.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7049000000000001e-05 2020-08-07T15:13:45.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.153 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.153 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:45.153 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551834+0000", 2020-08-07T15:13:45.161 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9249999999999997e-06 2020-08-07T15:13:45.161 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.161 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.162 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:45.162 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551838+0000", 2020-08-07T15:13:45.162 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9160000000000001e-06 2020-08-07T15:13:45.162 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.163 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.163 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:45.163 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551846+0000", 2020-08-07T15:13:45.163 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3510000000000002e-06 2020-08-07T15:13:45.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:45.164 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551849+0000", 2020-08-07T15:13:45.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9276000000000001e-05 2020-08-07T15:13:45.164 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.165 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.165 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551898+0000", 2020-08-07T15:13:45.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3095000000000001e-05 2020-08-07T15:13:45.165 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.166 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.166 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:45.184 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551911+0000", 2020-08-07T15:13:45.184 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67789159799999998 2020-08-07T15:13:45.211 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.211 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.212 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.222 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229803+0000", 2020-08-07T15:13:45.222 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8069000000000001e-05 2020-08-07T15:13:45.222 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.223 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.223 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:45.223 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229881+0000", 2020-08-07T15:13:45.223 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000152917 2020-08-07T15:13:45.223 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.224 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.224 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:45.224 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230034+0000", 2020-08-07T15:13:45.224 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044936760000000003 2020-08-07T15:13:45.224 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.225 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.225 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:45.225 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234528+0000", 2020-08-07T15:13:45.225 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8999999999999995e-07 2020-08-07T15:13:45.250 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.250 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.250 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:45.251 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234529+0000", 2020-08-07T15:13:45.251 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7741000000000002e-05 2020-08-07T15:13:45.251 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.251 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.252 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:45.252 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234586+0000", 2020-08-07T15:13:45.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069143959999999997 2020-08-07T15:13:45.252 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.253 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.253 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241501+0000", 2020-08-07T15:13:45.261 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1433000000000001e-05 2020-08-07T15:13:45.261 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.261 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.262 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.262 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241542+0000", 2020-08-07T15:13:45.262 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7800000000000001e-07 2020-08-07T15:13:45.262 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.262 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.263 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:45.263 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241543+0000", 2020-08-07T15:13:45.263 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000180321 2020-08-07T15:13:45.263 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.263 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.264 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:45.264 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241723+0000", 2020-08-07T15:13:45.264 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68991613100000004 2020-08-07T15:13:45.264 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.264 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:45.265 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.265 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.265 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.265 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89643 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:45.265 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.552181+0000", 2020-08-07T15:13:45.266 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 143.32812052099999, 2020-08-07T15:13:45.266 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68980872400000004, 2020-08-07T15:13:45.266 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:45.266 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:45.266 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:45.267 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:45.267 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:45.267 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89643 2020-08-07T15:13:45.267 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.268 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:45.268 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.268 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:45.268 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552181+0000", 2020-08-07T15:13:45.268 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:45.290 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.290 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.290 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:45.290 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552181+0000", 2020-08-07T15:13:45.291 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0926999999999997e-05 2020-08-07T15:13:45.291 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.291 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.291 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:45.291 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552212+0000", 2020-08-07T15:13:45.292 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0512e-05 2020-08-07T15:13:45.292 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.292 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.292 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:45.292 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552222+0000", 2020-08-07T15:13:45.293 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9379999999999992e-06 2020-08-07T15:13:45.293 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.293 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.293 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:45.293 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552230+0000", 2020-08-07T15:13:45.302 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0260000000000001e-06 2020-08-07T15:13:45.302 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.302 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.303 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:45.303 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552234+0000", 2020-08-07T15:13:45.303 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6109000000000002e-05 2020-08-07T15:13:45.303 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.303 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.304 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.304 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552280+0000", 2020-08-07T15:13:45.304 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5318000000000001e-05 2020-08-07T15:13:45.304 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.304 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.305 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:45.305 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552296+0000", 2020-08-07T15:13:45.305 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67811269100000005 2020-08-07T15:13:45.306 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.319 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.319 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.319 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230408+0000", 2020-08-07T15:13:45.319 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000125518 2020-08-07T15:13:45.320 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.320 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.320 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:45.320 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230534+0000", 2020-08-07T15:13:45.320 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024637999999999998 2020-08-07T15:13:45.321 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.321 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.321 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:45.321 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230780+0000", 2020-08-07T15:13:45.321 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0066398869999999997 2020-08-07T15:13:45.322 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.322 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.322 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:45.322 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237420+0000", 2020-08-07T15:13:45.322 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3500000000000005e-07 2020-08-07T15:13:45.323 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.323 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.323 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:45.323 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237421+0000", 2020-08-07T15:13:45.323 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4005000000000003e-05 2020-08-07T15:13:45.334 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.334 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.334 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:45.335 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237465+0000", 2020-08-07T15:13:45.335 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043133529999999998 2020-08-07T15:13:45.335 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.335 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.336 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241778+0000", 2020-08-07T15:13:45.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0592e-05 2020-08-07T15:13:45.336 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.336 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.336 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.336 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241819+0000", 2020-08-07T15:13:45.337 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4999999999999998e-07 2020-08-07T15:13:45.337 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.337 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.337 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:45.337 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241819+0000", 2020-08-07T15:13:45.338 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017018299999999999 2020-08-07T15:13:45.338 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.339 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.339 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:45.339 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241990+0000", 2020-08-07T15:13:45.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68980872400000004 2020-08-07T15:13:45.339 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.340 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:45.340 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.340 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.340 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89644 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:45.341 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.552384+0000", 2020-08-07T15:13:45.341 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 143.32791748700001, 2020-08-07T15:13:45.341 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68997479299999998, 2020-08-07T15:13:45.341 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:45.341 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:45.342 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:45.342 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:45.342 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:45.342 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89644 2020-08-07T15:13:45.342 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.343 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:45.343 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.343 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:45.343 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552384+0000", 2020-08-07T15:13:45.343 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:45.343 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.344 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.350 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:45.351 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552384+0000", 2020-08-07T15:13:45.351 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9238999999999999e-05 2020-08-07T15:13:45.351 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.351 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.352 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:45.352 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552413+0000", 2020-08-07T15:13:45.352 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3340000000000003e-06 2020-08-07T15:13:45.352 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.353 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.353 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:45.353 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552417+0000", 2020-08-07T15:13:45.353 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3119999999999994e-06 2020-08-07T15:13:45.353 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.354 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.354 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:45.354 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552426+0000", 2020-08-07T15:13:45.354 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4139999999999996e-06 2020-08-07T15:13:45.354 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.355 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.355 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:45.355 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552430+0000", 2020-08-07T15:13:45.355 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6613999999999999e-05 2020-08-07T15:13:45.355 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.355 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.356 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.356 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:13:45.356 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7280000000000007e-06 2020-08-07T15:13:45.356 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.356 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.357 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:45.357 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552477+0000", 2020-08-07T15:13:45.357 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67873990399999995 2020-08-07T15:13:45.357 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.357 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.358 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.358 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231216+0000", 2020-08-07T15:13:45.358 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3629999999999998e-05 2020-08-07T15:13:45.358 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.358 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.359 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:45.359 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231290+0000", 2020-08-07T15:13:45.359 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000132108 2020-08-07T15:13:45.359 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.359 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.360 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:45.360 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231422+0000", 2020-08-07T15:13:45.360 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006049652 2020-08-07T15:13:45.360 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.360 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.360 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:45.361 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237472+0000", 2020-08-07T15:13:45.361 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000005e-07 2020-08-07T15:13:45.361 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.361 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.362 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:45.362 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237472+0000", 2020-08-07T15:13:45.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2848000000000001e-05 2020-08-07T15:13:45.363 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.363 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.363 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:45.363 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237505+0000", 2020-08-07T15:13:45.364 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045352179999999997 2020-08-07T15:13:45.364 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.364 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242040+0000", 2020-08-07T15:13:45.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012843699999999999 2020-08-07T15:13:45.365 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.365 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.365 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.365 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242169+0000", 2020-08-07T15:13:45.366 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9999999999999998e-07 2020-08-07T15:13:45.366 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.366 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:45.366 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242169+0000", 2020-08-07T15:13:45.367 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000189354 2020-08-07T15:13:45.367 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.367 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.367 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:45.367 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242359+0000", 2020-08-07T15:13:45.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68997479299999998 2020-08-07T15:13:45.368 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.368 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:45.368 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.368 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.368 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.369 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89645 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:45.369 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:13:45.379 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 143.32783470499999, 2020-08-07T15:13:45.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69011194600000003, 2020-08-07T15:13:45.379 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:45.380 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:45.380 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:45.380 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:45.380 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:45.381 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89645 2020-08-07T15:13:45.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.381 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:45.390 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.391 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:45.391 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:13:45.391 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:45.402 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.402 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.402 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:45.402 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:13:45.403 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6750999999999998e-05 2020-08-07T15:13:45.403 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.403 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.403 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:45.403 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552493+0000", 2020-08-07T15:13:45.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0350000000000002e-06 2020-08-07T15:13:45.404 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.404 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.404 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:45.405 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552496+0000", 2020-08-07T15:13:45.405 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3029999999999996e-06 2020-08-07T15:13:45.405 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.405 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:45.414 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552503+0000", 2020-08-07T15:13:45.414 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7580000000000001e-06 2020-08-07T15:13:45.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.416 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:45.416 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552506+0000", 2020-08-07T15:13:45.416 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4868000000000003e-05 2020-08-07T15:13:45.416 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.417 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.417 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.417 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552570+0000", 2020-08-07T15:13:45.417 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9649999999999997e-06 2020-08-07T15:13:45.417 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.418 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.418 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:45.418 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552579+0000", 2020-08-07T15:13:45.418 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67928357699999997 2020-08-07T15:13:45.419 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.419 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.420 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.420 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231863+0000", 2020-08-07T15:13:45.420 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000115696 2020-08-07T15:13:45.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.421 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.421 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:45.424 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231979+0000", 2020-08-07T15:13:45.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020456599999999999 2020-08-07T15:13:45.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.425 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:45.425 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232183+0000", 2020-08-07T15:13:45.426 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005327923 2020-08-07T15:13:45.426 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.426 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.426 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:45.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237511+0000", 2020-08-07T15:13:45.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8999999999999998e-07 2020-08-07T15:13:45.427 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.427 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:45.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237511+0000", 2020-08-07T15:13:45.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6019000000000002e-05 2020-08-07T15:13:45.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:45.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237547+0000", 2020-08-07T15:13:45.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045340980000000003 2020-08-07T15:13:45.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.429 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.430 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242082+0000", 2020-08-07T15:13:45.430 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00032620499999999998 2020-08-07T15:13:45.430 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.430 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.431 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.431 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242408+0000", 2020-08-07T15:13:45.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3500000000000002e-07 2020-08-07T15:13:45.431 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.432 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:45.432 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242408+0000", 2020-08-07T15:13:45.432 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000170457 2020-08-07T15:13:45.432 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.432 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.433 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:45.433 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242579+0000", 2020-08-07T15:13:45.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69011194600000003 2020-08-07T15:13:45.433 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.434 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:45.435 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.435 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.435 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.435 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89646 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:45.435 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.557244+0000", 2020-08-07T15:13:45.436 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 143.323056933, 2020-08-07T15:13:45.436 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68559352299999998, 2020-08-07T15:13:45.436 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:45.436 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:45.436 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:45.437 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:45.437 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:45.437 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89646 2020-08-07T15:13:45.437 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.437 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:45.438 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.438 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:45.438 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557244+0000", 2020-08-07T15:13:45.438 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:45.438 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.439 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.439 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:45.440 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557244+0000", 2020-08-07T15:13:45.440 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9699000000000001e-05 2020-08-07T15:13:45.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.451 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.451 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:45.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557274+0000", 2020-08-07T15:13:45.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1269999999999996e-06 2020-08-07T15:13:45.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:45.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557278+0000", 2020-08-07T15:13:45.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0099999999999995e-06 2020-08-07T15:13:45.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.453 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:45.453 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557286+0000", 2020-08-07T15:13:45.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7439999999999998e-06 2020-08-07T15:13:45.454 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.454 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.454 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:45.454 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557291+0000", 2020-08-07T15:13:45.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5927999999999999e-05 2020-08-07T15:13:45.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557327+0000", 2020-08-07T15:13:45.456 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5069999999999997e-06 2020-08-07T15:13:45.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:45.456 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557333+0000", 2020-08-07T15:13:45.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67527419200000005 2020-08-07T15:13:45.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232608+0000", 2020-08-07T15:13:45.458 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8699e-05 2020-08-07T15:13:45.458 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.458 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:45.459 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232686+0000", 2020-08-07T15:13:45.459 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018185099999999999 2020-08-07T15:13:45.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.460 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.460 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:45.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232868+0000", 2020-08-07T15:13:45.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0075819360000000001 2020-08-07T15:13:45.461 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.461 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.461 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:45.461 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240450+0000", 2020-08-07T15:13:45.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2799999999999995e-07 2020-08-07T15:13:45.462 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.462 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.462 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:45.462 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240451+0000", 2020-08-07T15:13:45.462 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3751000000000002e-05 2020-08-07T15:13:45.463 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.463 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.463 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:45.464 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240495+0000", 2020-08-07T15:13:45.464 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001628064 2020-08-07T15:13:45.464 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.465 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.465 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.465 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242123+0000", 2020-08-07T15:13:45.465 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00050955599999999998 2020-08-07T15:13:45.465 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.466 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.466 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.466 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242632+0000", 2020-08-07T15:13:45.466 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2900000000000004e-07 2020-08-07T15:13:45.466 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.467 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.467 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:45.467 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242633+0000", 2020-08-07T15:13:45.467 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000205202 2020-08-07T15:13:45.467 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.468 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.483 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:45.483 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242838+0000", 2020-08-07T15:13:45.484 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68559352299999998 2020-08-07T15:13:45.484 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.484 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:45.484 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.484 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.485 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.485 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89647 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:45.485 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.557315+0000", 2020-08-07T15:13:45.487 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 143.32298660999999, 2020-08-07T15:13:45.487 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68581815800000001, 2020-08-07T15:13:45.488 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:45.489 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:45.489 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:45.489 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:45.490 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:45.490 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89647 2020-08-07T15:13:45.490 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.490 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:45.490 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.491 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:45.491 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557315+0000", 2020-08-07T15:13:45.491 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:45.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.491 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.492 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:45.492 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557315+0000", 2020-08-07T15:13:45.492 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6639999999999999e-05 2020-08-07T15:13:45.492 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.493 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:45.494 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557341+0000", 2020-08-07T15:13:45.494 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7989999999999998e-06 2020-08-07T15:13:45.494 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.494 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:45.495 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557344+0000", 2020-08-07T15:13:45.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6960000000000002e-06 2020-08-07T15:13:45.495 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.495 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.495 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:45.496 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557350+0000", 2020-08-07T15:13:45.496 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6089999999999999e-06 2020-08-07T15:13:45.496 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.496 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.497 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:45.497 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557352+0000", 2020-08-07T15:13:45.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1837e-05 2020-08-07T15:13:45.497 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.497 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.498 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.498 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557384+0000", 2020-08-07T15:13:45.498 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8099999999999998e-06 2020-08-07T15:13:45.498 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.498 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.499 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:45.499 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557392+0000", 2020-08-07T15:13:45.499 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67581117300000004 2020-08-07T15:13:45.499 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.499 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.500 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.500 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233203+0000", 2020-08-07T15:13:45.500 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112316 2020-08-07T15:13:45.500 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.501 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.501 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:45.501 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233316+0000", 2020-08-07T15:13:45.501 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000221766 2020-08-07T15:13:45.501 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.502 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.502 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:45.502 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233537+0000", 2020-08-07T15:13:45.502 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069695019999999998 2020-08-07T15:13:45.503 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.503 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.503 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:45.503 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240507+0000", 2020-08-07T15:13:45.504 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3999999999999997e-07 2020-08-07T15:13:45.504 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.504 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.504 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:45.504 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240507+0000", 2020-08-07T15:13:45.505 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8844000000000003e-05 2020-08-07T15:13:45.505 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.505 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.505 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:45.506 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240546+0000", 2020-08-07T15:13:45.506 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023473399999999998 2020-08-07T15:13:45.507 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.507 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.507 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.507 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242893+0000", 2020-08-07T15:13:45.508 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5410000000000001e-05 2020-08-07T15:13:45.508 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.511 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242939+0000", 2020-08-07T15:13:45.512 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8000000000000001e-07 2020-08-07T15:13:45.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.513 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.513 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:45.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242939+0000", 2020-08-07T15:13:45.513 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019369600000000001 2020-08-07T15:13:45.513 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.514 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:45.514 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243133+0000", 2020-08-07T15:13:45.514 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68581815800000001 2020-08-07T15:13:45.515 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.515 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:45.515 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.515 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.515 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.516 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89648 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:45.516 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.557372+0000", 2020-08-07T15:13:45.516 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 143.32292948899999, 2020-08-07T15:13:45.516 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68606040000000001, 2020-08-07T15:13:45.516 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:45.517 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:45.517 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:45.517 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:45.517 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:45.517 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89648 2020-08-07T15:13:45.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.518 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:45.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.518 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:45.519 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557372+0000", 2020-08-07T15:13:45.519 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:45.519 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.520 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:45.520 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557372+0000", 2020-08-07T15:13:45.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5534e-05 2020-08-07T15:13:45.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.522 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.523 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:45.523 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557397+0000", 2020-08-07T15:13:45.523 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.599e-06 2020-08-07T15:13:45.523 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.523 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.524 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:45.524 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557400+0000", 2020-08-07T15:13:45.524 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.288e-06 2020-08-07T15:13:45.524 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.525 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.525 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:45.525 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557405+0000", 2020-08-07T15:13:45.525 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.142e-06 2020-08-07T15:13:45.525 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.526 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.526 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:45.526 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557407+0000", 2020-08-07T15:13:45.526 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0954999999999998e-05 2020-08-07T15:13:45.526 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.527 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.527 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.527 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557428+0000", 2020-08-07T15:13:45.527 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7529999999999998e-06 2020-08-07T15:13:45.527 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.528 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:45.528 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557434+0000", 2020-08-07T15:13:45.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67655471099999998 2020-08-07T15:13:45.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.529 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.529 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233989+0000", 2020-08-07T15:13:45.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8301999999999997e-05 2020-08-07T15:13:45.531 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.531 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.531 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:45.531 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234067+0000", 2020-08-07T15:13:45.531 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000129908 2020-08-07T15:13:45.532 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.532 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.532 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:45.533 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234197+0000", 2020-08-07T15:13:45.533 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063551249999999997 2020-08-07T15:13:45.533 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.533 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.534 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:45.534 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240552+0000", 2020-08-07T15:13:45.534 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9400000000000001e-07 2020-08-07T15:13:45.550 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.550 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.550 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:45.551 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240552+0000", 2020-08-07T15:13:45.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2787999999999998e-05 2020-08-07T15:13:45.551 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.551 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.552 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:45.552 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240585+0000", 2020-08-07T15:13:45.552 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002605652 2020-08-07T15:13:45.552 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.553 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.553 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243191+0000", 2020-08-07T15:13:45.553 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8690000000000002e-05 2020-08-07T15:13:45.553 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.553 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.554 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.554 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243250+0000", 2020-08-07T15:13:45.554 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4700000000000001e-07 2020-08-07T15:13:45.554 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.555 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.555 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:45.555 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243250+0000", 2020-08-07T15:13:45.555 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000181912 2020-08-07T15:13:45.555 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:45.556 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243432+0000", 2020-08-07T15:13:45.556 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68606040000000001 2020-08-07T15:13:45.557 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.558 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:45.558 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.558 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.558 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.559 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89689 5.7 5:e83ee43b:::benchmark_data_smithi074_53697_object11211:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:45.559 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.585493+0000", 2020-08-07T15:13:45.559 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 143.294808671, 2020-08-07T15:13:45.559 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66549445299999999, 2020-08-07T15:13:45.560 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:45.560 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:45.560 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:45.561 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:45.561 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:45.562 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89689 2020-08-07T15:13:45.562 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.562 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:45.563 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.563 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:45.563 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585493+0000", 2020-08-07T15:13:45.564 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:45.564 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.565 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:45.565 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585493+0000", 2020-08-07T15:13:45.566 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7746000000000002e-05 2020-08-07T15:13:45.566 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.566 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.567 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:45.567 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585520+0000", 2020-08-07T15:13:45.567 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1099999999999996e-06 2020-08-07T15:13:45.568 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.568 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.568 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:45.569 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585525+0000", 2020-08-07T15:13:45.569 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5359999999999998e-06 2020-08-07T15:13:45.570 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.575 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:45.575 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585534+0000", 2020-08-07T15:13:45.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5040000000000002e-06 2020-08-07T15:13:45.576 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.576 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.576 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:45.577 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585540+0000", 2020-08-07T15:13:45.577 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3548999999999998e-05 2020-08-07T15:13:45.577 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.577 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.577 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.578 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585593+0000", 2020-08-07T15:13:45.578 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2014000000000001e-05 2020-08-07T15:13:45.578 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.579 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.579 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:45.579 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585605+0000", 2020-08-07T15:13:45.579 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.64901847700000004 2020-08-07T15:13:45.579 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.580 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.580 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.580 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234624+0000", 2020-08-07T15:13:45.580 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000176329 2020-08-07T15:13:45.580 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.581 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.581 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:45.581 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234800+0000", 2020-08-07T15:13:45.582 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00025457300000000002 2020-08-07T15:13:45.582 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.582 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.582 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:45.583 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.235055+0000", 2020-08-07T15:13:45.583 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0055364669999999998 2020-08-07T15:13:45.583 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.583 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.584 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:45.584 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240591+0000", 2020-08-07T15:13:45.584 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3700000000000001e-07 2020-08-07T15:13:45.584 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.584 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.585 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:45.585 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240591+0000", 2020-08-07T15:13:45.585 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8479999999999997e-05 2020-08-07T15:13:45.585 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.585 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.586 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:45.586 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240640+0000", 2020-08-07T15:13:45.586 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00285564 2020-08-07T15:13:45.586 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.587 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.587 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.587 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243496+0000", 2020-08-07T15:13:45.587 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0073477400000000002 2020-08-07T15:13:45.587 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.588 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.588 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.250843+0000", 2020-08-07T15:13:45.588 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7300000000000005e-07 2020-08-07T15:13:45.588 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.589 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:45.589 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.250844+0000", 2020-08-07T15:13:45.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014307799999999999 2020-08-07T15:13:45.590 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.597 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.597 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:45.597 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.250987+0000", 2020-08-07T15:13:45.597 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66549445299999999 2020-08-07T15:13:45.598 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.598 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:45.598 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.598 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.599 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.599 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103152 5.7 5:e865d79a:::benchmark_data_smithi074_53697_object12893:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e211)", 2020-08-07T15:13:45.599 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.399967+0000", 2020-08-07T15:13:45.599 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 133.480334782, 2020-08-07T15:13:45.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81348256399999996, 2020-08-07T15:13:45.600 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:45.600 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:13:45.611 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:45.611 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:45.611 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:45.612 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103152 2020-08-07T15:13:45.612 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.612 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:45.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.613 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:45.613 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.399967+0000", 2020-08-07T15:13:45.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:45.614 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.614 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.615 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:45.615 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.399967+0000", 2020-08-07T15:13:45.616 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017528899999999999 2020-08-07T15:13:45.616 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.616 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.617 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:45.617 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400142+0000", 2020-08-07T15:13:45.617 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8239999999999996e-06 2020-08-07T15:13:45.618 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.619 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.619 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:45.619 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400148+0000", 2020-08-07T15:13:45.619 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0251000000000001e-05 2020-08-07T15:13:45.619 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.620 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.620 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:45.620 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400158+0000", 2020-08-07T15:13:45.620 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1579999999999998e-06 2020-08-07T15:13:45.620 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.621 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.621 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:45.621 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400162+0000", 2020-08-07T15:13:45.621 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00078404699999999996 2020-08-07T15:13:45.621 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.622 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.622 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.622 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400946+0000", 2020-08-07T15:13:45.622 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010605800000000001 2020-08-07T15:13:45.623 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.623 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.623 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:45.623 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.401052+0000", 2020-08-07T15:13:45.623 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020987499999999999 2020-08-07T15:13:45.624 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.624 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.624 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:45.624 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.401262+0000", 2020-08-07T15:13:45.624 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023707820000000001 2020-08-07T15:13:45.625 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.625 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.625 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:45.625 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.403633+0000", 2020-08-07T15:13:45.625 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.023e-06 2020-08-07T15:13:45.626 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.626 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.626 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:45.626 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.403634+0000", 2020-08-07T15:13:45.626 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3560999999999995e-05 2020-08-07T15:13:45.627 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.627 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:45.627 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.403717+0000", 2020-08-07T15:13:45.627 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011669689999999999 2020-08-07T15:13:45.627 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.628 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.628 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.628 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.404884+0000", 2020-08-07T15:13:45.629 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033332850000000001 2020-08-07T15:13:45.629 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.629 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.629 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.630 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.408218+0000", 2020-08-07T15:13:45.630 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0268000000000001e-05 2020-08-07T15:13:45.630 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.630 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.630 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:45.631 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.408228+0000", 2020-08-07T15:13:45.631 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80508180500000004 2020-08-07T15:13:45.631 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.632 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.632 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.632 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213310+0000", 2020-08-07T15:13:45.633 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000139369 2020-08-07T15:13:45.633 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.633 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.633 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:45.633 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213449+0000", 2020-08-07T15:13:45.634 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81348256399999996 2020-08-07T15:13:45.634 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.634 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:45.634 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.634 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.635 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.635 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103265 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:13:45.635 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478593+0000", 2020-08-07T15:13:45.635 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 133.40170872600001, 2020-08-07T15:13:45.635 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74922190499999997, 2020-08-07T15:13:45.636 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:45.636 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:45.636 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:45.636 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:45.636 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:45.637 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103265 2020-08-07T15:13:45.637 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.637 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:45.637 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.637 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:45.638 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478593+0000", 2020-08-07T15:13:45.638 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:45.638 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.638 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.638 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:45.639 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478593+0000", 2020-08-07T15:13:45.639 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9968000000000001e-05 2020-08-07T15:13:45.639 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.640 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.640 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:45.640 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478633+0000", 2020-08-07T15:13:45.640 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1420000000000003e-06 2020-08-07T15:13:45.640 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.641 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.641 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:45.641 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478637+0000", 2020-08-07T15:13:45.641 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2919999999999996e-06 2020-08-07T15:13:45.642 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.642 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.642 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:45.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478645+0000", 2020-08-07T15:13:45.642 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3699999999999997e-06 2020-08-07T15:13:45.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.643 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.643 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:45.643 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478649+0000", 2020-08-07T15:13:45.644 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1440000000000002e-05 2020-08-07T15:13:45.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.644 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.644 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.644 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478701+0000", 2020-08-07T15:13:45.645 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0831e-05 2020-08-07T15:13:45.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:45.645 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478712+0000", 2020-08-07T15:13:45.646 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73476696699999999 2020-08-07T15:13:45.646 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.646 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.646 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.646 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213479+0000", 2020-08-07T15:13:45.646 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000167026 2020-08-07T15:13:45.647 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.647 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.647 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:45.647 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213646+0000", 2020-08-07T15:13:45.647 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000179557 2020-08-07T15:13:45.648 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.648 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.648 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:45.648 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213825+0000", 2020-08-07T15:13:45.648 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041300360000000001 2020-08-07T15:13:45.649 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.649 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.649 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:45.649 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217955+0000", 2020-08-07T15:13:45.650 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6840000000000001e-06 2020-08-07T15:13:45.650 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.651 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.651 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:45.651 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217957+0000", 2020-08-07T15:13:45.651 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4758e-05 2020-08-07T15:13:45.651 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.652 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.652 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:45.652 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218012+0000", 2020-08-07T15:13:45.652 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0095519990000000003 2020-08-07T15:13:45.652 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.653 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.653 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.653 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227564+0000", 2020-08-07T15:13:45.653 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9956e-05 2020-08-07T15:13:45.653 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.654 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.654 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.654 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227624+0000", 2020-08-07T15:13:45.654 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3790000000000001e-06 2020-08-07T15:13:45.655 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.655 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.655 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:45.655 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227625+0000", 2020-08-07T15:13:45.655 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0001895 2020-08-07T15:13:45.656 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.656 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.656 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:45.656 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227815+0000", 2020-08-07T15:13:45.656 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74922190499999997 2020-08-07T15:13:45.656 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.657 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:45.657 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.657 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.657 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.657 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103266 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:13:45.658 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478693+0000", 2020-08-07T15:13:45.658 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 133.401608385, 2020-08-07T15:13:45.658 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74942137399999997, 2020-08-07T15:13:45.658 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:45.659 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:45.659 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:45.659 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:45.659 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:45.659 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103266 2020-08-07T15:13:45.660 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.660 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:45.661 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.661 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:45.661 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478693+0000", 2020-08-07T15:13:45.661 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:45.661 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.662 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.662 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:45.662 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478693+0000", 2020-08-07T15:13:45.662 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5658000000000002e-05 2020-08-07T15:13:45.662 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.663 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.663 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:45.663 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478739+0000", 2020-08-07T15:13:45.663 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.113e-06 2020-08-07T15:13:45.663 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.664 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:45.664 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478744+0000", 2020-08-07T15:13:45.664 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9690000000000006e-06 2020-08-07T15:13:45.664 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.665 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.665 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:45.665 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478754+0000", 2020-08-07T15:13:45.665 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1849999999999997e-06 2020-08-07T15:13:45.666 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.667 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:45.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478758+0000", 2020-08-07T15:13:45.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7024999999999998e-05 2020-08-07T15:13:45.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.668 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478805+0000", 2020-08-07T15:13:45.668 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1357e-05 2020-08-07T15:13:45.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:45.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478816+0000", 2020-08-07T15:13:45.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73554277499999998 2020-08-07T15:13:45.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.670 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.670 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.214359+0000", 2020-08-07T15:13:45.670 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000129145 2020-08-07T15:13:45.670 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.670 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.673 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:45.673 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.214488+0000", 2020-08-07T15:13:45.673 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00027129999999999998 2020-08-07T15:13:45.673 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.674 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.674 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:45.674 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.214760+0000", 2020-08-07T15:13:45.674 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032631079999999998 2020-08-07T15:13:45.674 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.675 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.675 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:45.675 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218023+0000", 2020-08-07T15:13:45.675 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1000000000000001e-06 2020-08-07T15:13:45.675 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.676 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.676 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:45.676 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218024+0000", 2020-08-07T15:13:45.676 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9085000000000006e-05 2020-08-07T15:13:45.676 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.676 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:45.677 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218103+0000", 2020-08-07T15:13:45.677 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.009779088 2020-08-07T15:13:45.677 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.678 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.678 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227882+0000", 2020-08-07T15:13:45.678 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1989999999999994e-05 2020-08-07T15:13:45.678 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.678 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.679 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227944+0000", 2020-08-07T15:13:45.679 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8299999999999996e-07 2020-08-07T15:13:45.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.680 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:45.680 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227945+0000", 2020-08-07T15:13:45.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000169693 2020-08-07T15:13:45.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:45.681 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228114+0000", 2020-08-07T15:13:45.681 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74942137399999997 2020-08-07T15:13:45.682 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.682 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:45.682 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.682 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.682 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.683 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103267 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:13:45.683 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478794+0000", 2020-08-07T15:13:45.683 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 133.40150689699999, 2020-08-07T15:13:45.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74960417300000004, 2020-08-07T15:13:45.683 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:45.684 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:45.684 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:45.684 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:45.684 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:45.684 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103267 2020-08-07T15:13:45.685 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.685 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:45.685 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.685 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:45.685 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478794+0000", 2020-08-07T15:13:45.686 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:45.686 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.686 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.687 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:45.687 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478794+0000", 2020-08-07T15:13:45.687 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6646e-05 2020-08-07T15:13:45.687 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.687 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.687 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:45.688 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478841+0000", 2020-08-07T15:13:45.688 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:13:45.688 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.688 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.688 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:45.689 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478846+0000", 2020-08-07T15:13:45.689 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0171e-05 2020-08-07T15:13:45.689 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.689 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.690 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:45.690 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478856+0000", 2020-08-07T15:13:45.690 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5929999999999999e-06 2020-08-07T15:13:45.691 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.691 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.691 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:45.691 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478861+0000", 2020-08-07T15:13:45.691 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2032e-05 2020-08-07T15:13:45.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.693 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.693 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478903+0000", 2020-08-07T15:13:45.693 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0312000000000001e-05 2020-08-07T15:13:45.695 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.696 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:45.696 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478913+0000", 2020-08-07T15:13:45.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73634238600000002 2020-08-07T15:13:45.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.215256+0000", 2020-08-07T15:13:45.697 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000124933 2020-08-07T15:13:45.697 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:45.698 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.215380+0000", 2020-08-07T15:13:45.698 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024624 2020-08-07T15:13:45.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.698 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.698 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:45.699 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.215627+0000", 2020-08-07T15:13:45.699 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0040879540000000004 2020-08-07T15:13:45.699 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.699 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.699 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:45.700 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219715+0000", 2020-08-07T15:13:45.700 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9230000000000001e-06 2020-08-07T15:13:45.700 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.700 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:45.701 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219717+0000", 2020-08-07T15:13:45.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8881999999999995e-05 2020-08-07T15:13:45.701 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.701 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:45.702 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219785+0000", 2020-08-07T15:13:45.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0083925119999999995 2020-08-07T15:13:45.702 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.703 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.703 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228178+0000", 2020-08-07T15:13:45.703 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2254000000000004e-05 2020-08-07T15:13:45.703 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.704 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.704 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228240+0000", 2020-08-07T15:13:45.704 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4499999999999996e-07 2020-08-07T15:13:45.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.705 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.705 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:45.705 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228241+0000", 2020-08-07T15:13:45.705 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000157656 2020-08-07T15:13:45.705 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.706 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.706 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:45.706 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228399+0000", 2020-08-07T15:13:45.706 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74960417300000004 2020-08-07T15:13:45.706 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.707 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:45.707 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.707 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.707 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103268 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:13:45.708 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478898+0000", 2020-08-07T15:13:45.708 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 133.40140317999999, 2020-08-07T15:13:45.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74981275700000005, 2020-08-07T15:13:45.708 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:45.708 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:45.709 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:45.709 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:45.709 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:45.709 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103268 2020-08-07T15:13:45.709 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.710 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:45.710 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.710 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:45.710 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478898+0000", 2020-08-07T15:13:45.710 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:45.710 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.711 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:45.711 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478898+0000", 2020-08-07T15:13:45.711 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2558999999999997e-05 2020-08-07T15:13:45.711 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.712 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.712 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:45.712 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478941+0000", 2020-08-07T15:13:45.712 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6469999999999997e-06 2020-08-07T15:13:45.712 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.713 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.713 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:45.714 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478945+0000", 2020-08-07T15:13:45.714 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3109999999999995e-06 2020-08-07T15:13:45.714 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.714 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.714 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:45.715 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478955+0000", 2020-08-07T15:13:45.715 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.799e-06 2020-08-07T15:13:45.715 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.715 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.715 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:45.716 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478958+0000", 2020-08-07T15:13:45.716 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5682000000000002e-05 2020-08-07T15:13:45.716 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.716 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.717 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.717 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:13:45.717 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3260000000000002e-06 2020-08-07T15:13:45.717 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.717 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.718 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:45.718 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479004+0000", 2020-08-07T15:13:45.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73703730499999998 2020-08-07T15:13:45.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.718 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216041+0000", 2020-08-07T15:13:45.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4445999999999998e-05 2020-08-07T15:13:45.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:45.720 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216115+0000", 2020-08-07T15:13:45.720 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013977 2020-08-07T15:13:45.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.720 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:45.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216255+0000", 2020-08-07T15:13:45.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0051267359999999998 2020-08-07T15:13:45.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.721 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:45.722 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221382+0000", 2020-08-07T15:13:45.722 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0310000000000001e-06 2020-08-07T15:13:45.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.722 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:45.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221384+0000", 2020-08-07T15:13:45.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8097999999999998e-05 2020-08-07T15:13:45.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.724 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.724 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:45.724 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221452+0000", 2020-08-07T15:13:45.724 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0070259249999999997 2020-08-07T15:13:45.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.725 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.725 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228478+0000", 2020-08-07T15:13:45.725 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7479999999999999e-05 2020-08-07T15:13:45.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.726 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.726 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.726 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228535+0000", 2020-08-07T15:13:45.726 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4399999999999999e-07 2020-08-07T15:13:45.726 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.727 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.727 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:45.727 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228536+0000", 2020-08-07T15:13:45.727 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017489799999999999 2020-08-07T15:13:45.727 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:45.728 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228711+0000", 2020-08-07T15:13:45.728 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74981275700000005 2020-08-07T15:13:45.728 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.729 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:45.729 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.729 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.729 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.729 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103269 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:13:45.730 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:13:45.730 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 133.40130707200001, 2020-08-07T15:13:45.730 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74992452799999998, 2020-08-07T15:13:45.730 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:45.730 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:45.730 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:45.731 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:45.731 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:45.731 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103269 2020-08-07T15:13:45.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.731 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:45.732 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:45.732 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:13:45.732 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:45.732 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.733 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:45.733 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:13:45.734 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5529999999999999e-05 2020-08-07T15:13:45.735 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.735 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.735 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:45.735 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479040+0000", 2020-08-07T15:13:45.735 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6550000000000003e-06 2020-08-07T15:13:45.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.736 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.736 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:45.736 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479044+0000", 2020-08-07T15:13:45.736 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4285e-05 2020-08-07T15:13:45.737 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.737 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.737 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:45.737 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479059+0000", 2020-08-07T15:13:45.737 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2050000000000001e-06 2020-08-07T15:13:45.738 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.738 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.738 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:45.738 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479064+0000", 2020-08-07T15:13:45.738 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0952999999999999e-05 2020-08-07T15:13:45.739 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.739 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.739 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.739 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479105+0000", 2020-08-07T15:13:45.739 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3559999999999998e-06 2020-08-07T15:13:45.739 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.740 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.740 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:45.740 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479111+0000", 2020-08-07T15:13:45.740 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.737508104 2020-08-07T15:13:45.740 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.741 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.741 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.741 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216619+0000", 2020-08-07T15:13:45.741 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000126367 2020-08-07T15:13:45.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.742 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.742 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:45.742 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216746+0000", 2020-08-07T15:13:45.742 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014069899999999999 2020-08-07T15:13:45.742 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.743 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:45.743 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216886+0000", 2020-08-07T15:13:45.743 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045753160000000003 2020-08-07T15:13:45.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:45.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221462+0000", 2020-08-07T15:13:45.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.142e-06 2020-08-07T15:13:45.746 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.746 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:45.746 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221463+0000", 2020-08-07T15:13:45.746 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7718e-05 2020-08-07T15:13:45.747 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.747 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.747 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:45.747 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221501+0000", 2020-08-07T15:13:45.747 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0072733210000000001 2020-08-07T15:13:45.748 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.748 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.748 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.748 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228774+0000", 2020-08-07T15:13:45.748 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3828e-05 2020-08-07T15:13:45.749 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.749 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.749 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.749 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228828+0000", 2020-08-07T15:13:45.749 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9800000000000003e-07 2020-08-07T15:13:45.750 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.750 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.750 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:45.750 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228828+0000", 2020-08-07T15:13:45.750 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0451000000000005e-05 2020-08-07T15:13:45.750 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.751 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.751 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:45.751 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228919+0000", 2020-08-07T15:13:45.751 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74992452799999998 2020-08-07T15:13:45.751 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.752 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:45.752 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.752 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.752 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103270 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:13:45.753 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.479168+0000", 2020-08-07T15:13:45.753 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 133.40113303499999, 2020-08-07T15:13:45.753 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75043371699999994, 2020-08-07T15:13:45.753 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:45.753 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:45.754 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:45.754 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:45.754 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:45.755 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103270 2020-08-07T15:13:45.755 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.755 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:45.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:45.756 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479168+0000", 2020-08-07T15:13:45.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:45.757 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.757 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:45.757 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479168+0000", 2020-08-07T15:13:45.757 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1557000000000002e-05 2020-08-07T15:13:45.758 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.758 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.758 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:45.758 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479210+0000", 2020-08-07T15:13:45.758 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9840000000000001e-06 2020-08-07T15:13:45.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.759 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:45.759 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479215+0000", 2020-08-07T15:13:45.760 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9539999999999999e-06 2020-08-07T15:13:45.760 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.760 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.760 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:45.760 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479225+0000", 2020-08-07T15:13:45.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.942e-06 2020-08-07T15:13:45.761 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.761 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:45.761 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479231+0000", 2020-08-07T15:13:45.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2493000000000002e-05 2020-08-07T15:13:45.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.762 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.762 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479263+0000", 2020-08-07T15:13:45.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6640000000000003e-06 2020-08-07T15:13:45.763 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.763 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:45.763 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479269+0000", 2020-08-07T15:13:45.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73802486499999997 2020-08-07T15:13:45.764 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.764 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217294+0000", 2020-08-07T15:13:45.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8640000000000006e-05 2020-08-07T15:13:45.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.765 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:45.765 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217372+0000", 2020-08-07T15:13:45.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000140338 2020-08-07T15:13:45.766 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:45.767 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217513+0000", 2020-08-07T15:13:45.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039952119999999997 2020-08-07T15:13:45.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.768 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:45.768 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221508+0000", 2020-08-07T15:13:45.768 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8899999999999999e-07 2020-08-07T15:13:45.768 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.769 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.769 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:45.769 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221509+0000", 2020-08-07T15:13:45.770 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8108999999999997e-05 2020-08-07T15:13:45.770 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.770 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.771 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:45.771 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221547+0000", 2020-08-07T15:13:45.771 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074232029999999997 2020-08-07T15:13:45.772 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.772 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.773 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.773 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228970+0000", 2020-08-07T15:13:45.773 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00046710099999999999 2020-08-07T15:13:45.774 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.774 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.774 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.775 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229437+0000", 2020-08-07T15:13:45.775 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5400000000000001e-07 2020-08-07T15:13:45.775 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:45.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229438+0000", 2020-08-07T15:13:45.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016441200000000001 2020-08-07T15:13:45.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:45.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229602+0000", 2020-08-07T15:13:45.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75043371699999994 2020-08-07T15:13:45.779 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.779 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:45.780 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.780 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.780 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103271 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:13:45.781 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.479356+0000", 2020-08-07T15:13:45.781 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 133.40094506700001, 2020-08-07T15:13:45.781 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75047653800000003, 2020-08-07T15:13:45.781 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:45.781 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:45.782 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:45.782 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:45.783 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:45.783 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103271 2020-08-07T15:13:45.783 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.783 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:45.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:45.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479356+0000", 2020-08-07T15:13:45.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:45.785 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:45.786 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479356+0000", 2020-08-07T15:13:45.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2416000000000002e-05 2020-08-07T15:13:45.787 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.787 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:45.788 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479399+0000", 2020-08-07T15:13:45.788 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0819999999999998e-06 2020-08-07T15:13:45.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.788 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:45.789 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479404+0000", 2020-08-07T15:13:45.789 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2486999999999999e-05 2020-08-07T15:13:45.789 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.789 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.789 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:45.790 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479416+0000", 2020-08-07T15:13:45.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3810000000000001e-06 2020-08-07T15:13:45.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.790 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:45.791 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479421+0000", 2020-08-07T15:13:45.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2311999999999998e-05 2020-08-07T15:13:45.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479453+0000", 2020-08-07T15:13:45.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7690000000000002e-06 2020-08-07T15:13:45.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.792 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.792 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:45.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479459+0000", 2020-08-07T15:13:45.793 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73869566499999995 2020-08-07T15:13:45.793 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.793 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.793 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.794 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218154+0000", 2020-08-07T15:13:45.794 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000124535 2020-08-07T15:13:45.801 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.801 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.801 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:45.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218279+0000", 2020-08-07T15:13:45.802 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000223441 2020-08-07T15:13:45.802 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.802 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.802 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:45.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218502+0000", 2020-08-07T15:13:45.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003054761 2020-08-07T15:13:45.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:45.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221557+0000", 2020-08-07T15:13:45.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0449999999999999e-06 2020-08-07T15:13:45.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:45.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221558+0000", 2020-08-07T15:13:45.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3655e-05 2020-08-07T15:13:45.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:45.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221622+0000", 2020-08-07T15:13:45.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.007661434 2020-08-07T15:13:45.806 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229283+0000", 2020-08-07T15:13:45.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00036023099999999997 2020-08-07T15:13:45.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229644+0000", 2020-08-07T15:13:45.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.63e-07 2020-08-07T15:13:45.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.808 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:45.809 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229644+0000", 2020-08-07T15:13:45.809 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018886099999999999 2020-08-07T15:13:45.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.809 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:45.810 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229833+0000", 2020-08-07T15:13:45.810 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75047653800000003 2020-08-07T15:13:45.810 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.810 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:45.810 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.811 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103272 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:13:45.811 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.481658+0000", 2020-08-07T15:13:45.811 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 133.398643644, 2020-08-07T15:13:45.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74920410900000001, 2020-08-07T15:13:45.812 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:45.812 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:45.812 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:45.812 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:45.813 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:45.813 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103272 2020-08-07T15:13:45.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.813 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:45.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.814 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:45.814 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481658+0000", 2020-08-07T15:13:45.814 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:45.814 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.814 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.815 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:45.815 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481658+0000", 2020-08-07T15:13:45.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8787e-05 2020-08-07T15:13:45.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:45.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481697+0000", 2020-08-07T15:13:45.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0559999999999998e-06 2020-08-07T15:13:45.816 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.817 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:45.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481701+0000", 2020-08-07T15:13:45.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1329999999999999e-06 2020-08-07T15:13:45.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.818 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:45.818 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481709+0000", 2020-08-07T15:13:45.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.189e-06 2020-08-07T15:13:45.818 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.818 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:45.819 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481713+0000", 2020-08-07T15:13:45.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6025999999999997e-05 2020-08-07T15:13:45.819 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481749+0000", 2020-08-07T15:13:45.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5619999999999998e-06 2020-08-07T15:13:45.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:45.821 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481756+0000", 2020-08-07T15:13:45.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73729155000000002 2020-08-07T15:13:45.821 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:45.822 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219048+0000", 2020-08-07T15:13:45.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000120387 2020-08-07T15:13:45.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.822 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.822 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:45.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219168+0000", 2020-08-07T15:13:45.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017865599999999999 2020-08-07T15:13:45.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:45.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219347+0000", 2020-08-07T15:13:45.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039049169999999999 2020-08-07T15:13:45.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:45.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.223252+0000", 2020-08-07T15:13:45.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.412e-06 2020-08-07T15:13:45.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:45.826 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.223253+0000", 2020-08-07T15:13:45.826 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7521999999999997e-05 2020-08-07T15:13:45.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:45.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.223301+0000", 2020-08-07T15:13:45.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00603478 2020-08-07T15:13:45.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.827 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.828 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229336+0000", 2020-08-07T15:13:45.828 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013309579999999999 2020-08-07T15:13:45.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:45.829 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.230667+0000", 2020-08-07T15:13:45.829 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7400000000000003e-07 2020-08-07T15:13:45.829 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.829 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.829 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:45.830 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.230667+0000", 2020-08-07T15:13:45.830 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019450000000000001 2020-08-07T15:13:45.830 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:45.830 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:45.830 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:45.831 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.230862+0000", 2020-08-07T15:13:45.831 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74920410900000001 2020-08-07T15:13:45.831 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.831 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:45.831 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.832 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:45.832 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:45.832 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:45.833 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:13:45.836 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:13:45.836 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:13:45.837 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:13:45.837 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:13:45.837 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:13:45.837 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:13:45.838 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:13:45.838 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:13:45.838 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:13:45.838 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:13:45.839 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:13:45.839 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:13:45.839 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:13:45.839 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:13:45.840 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:45.832+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:45.841 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:13:45.934 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:45.933+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:46.040 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:46.034+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:46.113 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:46.114 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:13:46.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.114 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:139322 6.9 6:9a4132b7:::benchmark_data_smithi074_63843_object6070:head [delete] snapc 0=[] ondisk+write+known_if_redirected e295)", 2020-08-07T15:13:46.114 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:46.064878+0000", 2020-08-07T15:13:46.114 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.047320200999999999, 2020-08-07T15:13:46.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.047349079000000002, 2020-08-07T15:13:46.115 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:46.115 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:46.115 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:46.115 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:46.116 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:46.116 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 139322 2020-08-07T15:13:46.116 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.116 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:46.116 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:46.117 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.064878+0000", 2020-08-07T15:13:46.117 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:46.117 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.117 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.118 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:46.118 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.064878+0000", 2020-08-07T15:13:46.119 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9821000000000001e-05 2020-08-07T15:13:46.119 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.119 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.119 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:46.120 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.064908+0000", 2020-08-07T15:13:46.120 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0530000000000003e-06 2020-08-07T15:13:46.120 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.120 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.120 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:46.121 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.064914+0000", 2020-08-07T15:13:46.121 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0315e-05 2020-08-07T15:13:46.121 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.121 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.121 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:46.122 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.064925+0000", 2020-08-07T15:13:46.122 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6009999999999997e-06 2020-08-07T15:13:46.122 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.122 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.122 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:46.123 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.064929+0000", 2020-08-07T15:13:46.123 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00072401599999999996 2020-08-07T15:13:46.123 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.123 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.123 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:46.124 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.065653+0000", 2020-08-07T15:13:46.124 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015248400000000001 2020-08-07T15:13:46.124 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.124 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.124 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:46.130 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.065806+0000", 2020-08-07T15:13:46.130 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013947300000000001 2020-08-07T15:13:46.131 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.131 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.131 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:13:46.131 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.065945+0000", 2020-08-07T15:13:46.131 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0089107140000000001 2020-08-07T15:13:46.150 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.151 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:46.151 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.074856+0000", 2020-08-07T15:13:46.151 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9600000000000008e-07 2020-08-07T15:13:46.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:46.152 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.074857+0000", 2020-08-07T15:13:46.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6607999999999997e-05 2020-08-07T15:13:46.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.153 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.153 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:46.153 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.074913+0000", 2020-08-07T15:13:46.153 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00059411000000000002 2020-08-07T15:13:46.153 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.154 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.154 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:46.154 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.075508+0000", 2020-08-07T15:13:46.154 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.010629191 2020-08-07T15:13:46.154 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:46.155 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:46.155 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:46.155 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.155 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.155 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:139393 6.8 6:1e680c74:::benchmark_data_smithi074_63843_object6141:head [delete] snapc 0=[] ondisk+write+known_if_redirected e295)", 2020-08-07T15:13:46.156 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:46.098806+0000", 2020-08-07T15:13:46.156 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.01339279, 2020-08-07T15:13:46.156 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.013542190000000001, 2020-08-07T15:13:46.156 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:46.156 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:46.156 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:46.161 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:46.162 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:46.162 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 139393 2020-08-07T15:13:46.162 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.162 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:46.162 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.163 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:46.163 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.098806+0000", 2020-08-07T15:13:46.163 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:46.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.163 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:46.164 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.098806+0000", 2020-08-07T15:13:46.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3862999999999997e-05 2020-08-07T15:13:46.164 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.165 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:46.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.098840+0000", 2020-08-07T15:13:46.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3779999999999998e-06 2020-08-07T15:13:46.166 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.166 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.166 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:46.167 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.098846+0000", 2020-08-07T15:13:46.167 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3148e-05 2020-08-07T15:13:46.167 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.168 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.168 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:46.168 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.098859+0000", 2020-08-07T15:13:46.169 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4909999999999994e-06 2020-08-07T15:13:46.169 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.169 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.169 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:46.183 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.098869+0000", 2020-08-07T15:13:46.183 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5459999999999999e-05 2020-08-07T15:13:46.184 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.184 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.184 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:46.185 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.098944+0000", 2020-08-07T15:13:46.185 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015305900000000001 2020-08-07T15:13:46.186 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.186 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.187 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:46.204 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.099097+0000", 2020-08-07T15:13:46.204 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000173053 2020-08-07T15:13:46.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.205 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.206 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,7", 2020-08-07T15:13:46.206 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.099270+0000", 2020-08-07T15:13:46.206 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017555909999999999 2020-08-07T15:13:46.207 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.207 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.207 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:46.208 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.101026+0000", 2020-08-07T15:13:46.208 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.186e-06 2020-08-07T15:13:46.208 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.209 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.209 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:46.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.101027+0000", 2020-08-07T15:13:46.210 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9462999999999995e-05 2020-08-07T15:13:46.210 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.210 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.211 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:46.211 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.101096+0000", 2020-08-07T15:13:46.212 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004070812 2020-08-07T15:13:46.212 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.213 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.213 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:46.222 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.105167+0000", 2020-08-07T15:13:46.223 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063615039999999996 2020-08-07T15:13:46.225 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:46.225 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:46.226 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:46.226 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.226 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.227 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:139328 6.9 6:955383e0:::benchmark_data_smithi074_63843_object6076:head [delete] snapc 0=[] ondisk+write+known_if_redirected e295)", 2020-08-07T15:13:46.227 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:46.068505+0000", 2020-08-07T15:13:46.227 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.043694036999999998, 2020-08-07T15:13:46.227 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.043969034999999997, 2020-08-07T15:13:46.228 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:46.228 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:46.228 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:46.228 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:46.228 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:46.229 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 139328 2020-08-07T15:13:46.233 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.233 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:46.233 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.233 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:46.233 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.068505+0000", 2020-08-07T15:13:46.234 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:46.234 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.234 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.234 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:46.234 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.068505+0000", 2020-08-07T15:13:46.235 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3969e-05 2020-08-07T15:13:46.247 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.247 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.248 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:46.248 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.068538+0000", 2020-08-07T15:13:46.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4799999999999998e-06 2020-08-07T15:13:46.253 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.253 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.253 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:46.254 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.068545+0000", 2020-08-07T15:13:46.254 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2367e-05 2020-08-07T15:13:46.254 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.254 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.255 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:46.255 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.068557+0000", 2020-08-07T15:13:46.255 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9309999999999996e-06 2020-08-07T15:13:46.255 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.255 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.255 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:46.256 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.068563+0000", 2020-08-07T15:13:46.256 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2778999999999997e-05 2020-08-07T15:13:46.257 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.257 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.257 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:46.257 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.068616+0000", 2020-08-07T15:13:46.257 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015554699999999999 2020-08-07T15:13:46.258 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.258 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.258 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:46.258 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.068772+0000", 2020-08-07T15:13:46.258 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000155091 2020-08-07T15:13:46.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.259 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.259 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:13:46.259 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.068927+0000", 2020-08-07T15:13:46.259 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0060391660000000003 2020-08-07T15:13:46.260 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.260 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.260 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:46.260 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.074966+0000", 2020-08-07T15:13:46.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.16e-07 2020-08-07T15:13:46.261 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.261 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.261 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:46.261 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.074967+0000", 2020-08-07T15:13:46.261 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2803999999999998e-05 2020-08-07T15:13:46.262 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.262 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.262 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:46.262 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.075020+0000", 2020-08-07T15:13:46.262 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.015829739999999998 2020-08-07T15:13:46.262 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.263 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.263 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:46.263 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.090849+0000", 2020-08-07T15:13:46.278 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.02234479 2020-08-07T15:13:46.278 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:46.278 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:46.278 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:46.279 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.279 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.279 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:139400 6.8 6:1694241c:::benchmark_data_smithi074_63843_object6148:head [delete] snapc 0=[] ondisk+write+known_if_redirected e295)", 2020-08-07T15:13:46.279 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:46.103726+0000", 2020-08-07T15:13:46.279 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0084728089999999995, 2020-08-07T15:13:46.279 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0088609150000000005, 2020-08-07T15:13:46.280 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:46.280 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:13:46.283 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:46.283 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:46.284 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:46.284 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 139400 2020-08-07T15:13:46.285 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.285 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:46.285 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.286 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:46.286 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.103726+0000", 2020-08-07T15:13:46.286 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:46.287 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.287 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.287 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:46.288 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.103726+0000", 2020-08-07T15:13:46.289 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3340999999999998e-05 2020-08-07T15:13:46.289 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.289 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.290 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:46.290 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.103759+0000", 2020-08-07T15:13:46.290 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4300000000000003e-06 2020-08-07T15:13:46.291 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.291 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.292 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:46.292 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.103766+0000", 2020-08-07T15:13:46.292 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3351e-05 2020-08-07T15:13:46.299 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.299 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.299 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:46.299 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.103779+0000", 2020-08-07T15:13:46.299 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0299999999999999e-06 2020-08-07T15:13:46.300 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.300 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.300 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:46.300 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.103785+0000", 2020-08-07T15:13:46.300 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4164000000000003e-05 2020-08-07T15:13:46.301 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.301 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.301 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:46.301 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.103859+0000", 2020-08-07T15:13:46.302 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018275100000000001 2020-08-07T15:13:46.302 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.302 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.302 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:46.302 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.104042+0000", 2020-08-07T15:13:46.302 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016123299999999999 2020-08-07T15:13:46.303 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.303 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.303 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,7", 2020-08-07T15:13:46.303 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.104203+0000", 2020-08-07T15:13:46.303 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019901770000000001 2020-08-07T15:13:46.304 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.304 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.304 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:46.304 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.106193+0000", 2020-08-07T15:13:46.304 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.7978000000000004e-05 2020-08-07T15:13:46.305 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.321 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.322 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:46.322 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:46.106281+0000", 2020-08-07T15:13:46.322 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0025554549999999999 2020-08-07T15:13:46.323 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:46.323 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:46.324 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:46.324 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:46.324 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:13:46.324 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 4 2020-08-07T15:13:46.325 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:46.327 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:13:46.328 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:46.325+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:46.428 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:46.426+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:46.497 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:13:46.532 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:46.531+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:46.611 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:46.612 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:13:46.612 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:13:46.612 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:13:46.612 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:46.624 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:13:46.640 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:46.634+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:46.739 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:46.737+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:46.844 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:46.844 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:13:46.845 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:13:46.845 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:13:46.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.845 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:30968 4.4 e98/81 ondisk, result = 0)", 2020-08-07T15:13:46.845 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:13:46.846 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 301.10649345299998, 2020-08-07T15:13:46.846 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96211418999999998, 2020-08-07T15:13:46.846 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:46.846 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:46.846 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:46.847 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.847 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:46.847 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:13:46.847 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:46.847 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.848 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.848 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:46.849 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:13:46.849 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2387e-05 2020-08-07T15:13:46.849 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.850 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:46.850 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729821+0000", 2020-08-07T15:13:46.850 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9909999999999999e-06 2020-08-07T15:13:46.850 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.851 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.851 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:46.851 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729827+0000", 2020-08-07T15:13:46.851 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96185524600000005 2020-08-07T15:13:46.851 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:46.852 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691682+0000", 2020-08-07T15:13:46.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8760000000000005e-06 2020-08-07T15:13:46.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.853 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:46.853 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691690+0000", 2020-08-07T15:13:46.853 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9682000000000001e-05 2020-08-07T15:13:46.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.854 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.854 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:46.854 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691759+0000", 2020-08-07T15:13:46.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5381000000000001e-05 2020-08-07T15:13:46.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.872 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.873 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:46.873 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691775+0000", 2020-08-07T15:13:46.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000127627 2020-08-07T15:13:46.873 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.874 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:46.874 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691902+0000", 2020-08-07T15:13:46.874 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96211418999999998 2020-08-07T15:13:46.874 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:46.874 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:46.875 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:46.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.875 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42472 4.4 4:2ff82af8:::benchmark_data_smithi074_44586_object5308:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:13:46.875 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:13:46.876 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 286.56903067899998, 2020-08-07T15:13:46.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89653816399999997, 2020-08-07T15:13:46.876 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:46.876 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:46.876 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:46.877 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:46.877 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:46.877 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42472 2020-08-07T15:13:46.877 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.877 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:46.894 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.894 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:46.894 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:13:46.895 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:46.895 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.895 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.896 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:46.896 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:13:46.896 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9591e-05 2020-08-07T15:13:46.897 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.898 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.898 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:46.898 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267301+0000", 2020-08-07T15:13:46.899 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7490000000000004e-06 2020-08-07T15:13:46.899 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.899 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.899 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:46.900 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267306+0000", 2020-08-07T15:13:46.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2850000000000001e-05 2020-08-07T15:13:46.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.901 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.901 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:46.901 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267319+0000", 2020-08-07T15:13:46.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9100000000000002e-06 2020-08-07T15:13:46.902 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.902 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.902 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:46.902 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267325+0000", 2020-08-07T15:13:46.903 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6584000000000001e-05 2020-08-07T15:13:46.903 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.903 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:46.904 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267362+0000", 2020-08-07T15:13:46.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9188000000000001e-05 2020-08-07T15:13:46.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.904 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:46.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267451+0000", 2020-08-07T15:13:46.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000182917 2020-08-07T15:13:46.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:46.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267634+0000", 2020-08-07T15:13:46.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002484708 2020-08-07T15:13:46.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:46.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270119+0000", 2020-08-07T15:13:46.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0299999999999997e-07 2020-08-07T15:13:46.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.918 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:46.918 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270119+0000", 2020-08-07T15:13:46.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8251999999999996e-05 2020-08-07T15:13:46.919 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.919 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:46.919 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270188+0000", 2020-08-07T15:13:46.920 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24443688 2020-08-07T15:13:46.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:46.920 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.514625+0000", 2020-08-07T15:13:46.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.64911260599999998 2020-08-07T15:13:46.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:46.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163737+0000", 2020-08-07T15:13:46.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2100000000000001e-06 2020-08-07T15:13:46.922 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:46.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163738+0000", 2020-08-07T15:13:46.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0816000000000003e-05 2020-08-07T15:13:46.937 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:46.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163789+0000", 2020-08-07T15:13:46.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89653816399999997 2020-08-07T15:13:46.939 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:46.940 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:46.940 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:46.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.941 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72688 4.4 4:295d3f6e:::benchmark_data_smithi074_44586_object9085:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e129)", 2020-08-07T15:13:46.942 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:13:46.942 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 261.439448492, 2020-08-07T15:13:46.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7869782759999999, 2020-08-07T15:13:46.943 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:46.943 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:13:46.943 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:46.944 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:46.944 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:46.944 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72688 2020-08-07T15:13:46.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.945 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:46.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.946 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:46.947 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:13:46.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:46.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:46.951 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:13:46.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8721999999999999e-05 2020-08-07T15:13:46.952 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.952 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.953 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:46.953 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396882+0000", 2020-08-07T15:13:46.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2639999999999999e-06 2020-08-07T15:13:46.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:46.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396887+0000", 2020-08-07T15:13:46.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0767999999999999e-05 2020-08-07T15:13:46.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:46.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396898+0000", 2020-08-07T15:13:46.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1860000000000002e-06 2020-08-07T15:13:46.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:46.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396902+0000", 2020-08-07T15:13:46.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0037638649999999999 2020-08-07T15:13:46.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:46.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400666+0000", 2020-08-07T15:13:46.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000118077 2020-08-07T15:13:46.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.969 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:46.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400784+0000", 2020-08-07T15:13:46.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000136728 2020-08-07T15:13:46.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.975 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:13:46.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400921+0000", 2020-08-07T15:13:46.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001595443 2020-08-07T15:13:46.976 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.976 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.976 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:46.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.402516+0000", 2020-08-07T15:13:46.977 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.014817907 2020-08-07T15:13:46.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:46.977 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417334+0000", 2020-08-07T15:13:46.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.75e-06 2020-08-07T15:13:46.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.978 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:46.979 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417336+0000", 2020-08-07T15:13:46.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8066999999999996e-05 2020-08-07T15:13:46.979 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.979 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:46.980 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417424+0000", 2020-08-07T15:13:46.980 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002210573 2020-08-07T15:13:46.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.987 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:46.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.419635+0000", 2020-08-07T15:13:46.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1379000000000001e-05 2020-08-07T15:13:46.995 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:46.998 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:46.998 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:46.998 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.419646+0000", 2020-08-07T15:13:46.999 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.004643409 2020-08-07T15:13:46.999 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.000 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.000 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.000 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424289+0000", 2020-08-07T15:13:47.001 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7563000000000001e-05 2020-08-07T15:13:47.001 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.002 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.002 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.002 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424307+0000", 2020-08-07T15:13:47.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75943629400000001 2020-08-07T15:13:47.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183743+0000", 2020-08-07T15:13:47.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8281000000000006e-05 2020-08-07T15:13:47.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:47.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183812+0000", 2020-08-07T15:13:47.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7869782759999999 2020-08-07T15:13:47.007 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.007 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:47.007 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.008 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72985 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:47.008 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:13:47.008 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 261.22346267299997, 2020-08-07T15:13:47.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869466839999999, 2020-08-07T15:13:47.009 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:47.009 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:47.009 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:47.009 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:47.010 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:47.010 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72985 2020-08-07T15:13:47.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.010 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:47.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.011 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:47.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:13:47.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:47.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:47.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:13:47.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8003999999999997e-05 2020-08-07T15:13:47.012 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.012 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:47.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612857+0000", 2020-08-07T15:13:47.013 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.246e-06 2020-08-07T15:13:47.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:47.015 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612863+0000", 2020-08-07T15:13:47.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5146e-05 2020-08-07T15:13:47.016 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.016 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:47.017 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612878+0000", 2020-08-07T15:13:47.017 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1940000000000003e-06 2020-08-07T15:13:47.017 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:47.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612885+0000", 2020-08-07T15:13:47.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0349999999999997e-05 2020-08-07T15:13:47.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.036 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612935+0000", 2020-08-07T15:13:47.036 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1029999999999996e-06 2020-08-07T15:13:47.038 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.038 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612942+0000", 2020-08-07T15:13:47.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81140097700000002 2020-08-07T15:13:47.039 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.039 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.039 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.040 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424343+0000", 2020-08-07T15:13:47.040 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1799999999999996e-06 2020-08-07T15:13:47.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.041 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424351+0000", 2020-08-07T15:13:47.042 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75948665299999996 2020-08-07T15:13:47.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.043 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.044 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.044 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183838+0000", 2020-08-07T15:13:47.044 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000179187 2020-08-07T15:13:47.044 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.046 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:47.047 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.184017+0000", 2020-08-07T15:13:47.049 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024691600000000002 2020-08-07T15:13:47.049 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.049 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.050 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:47.050 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.184264+0000", 2020-08-07T15:13:47.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006362721 2020-08-07T15:13:47.051 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.054 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:47.054 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190627+0000", 2020-08-07T15:13:47.054 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7400000000000001e-06 2020-08-07T15:13:47.055 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.055 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:47.057 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190629+0000", 2020-08-07T15:13:47.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1305999999999999e-05 2020-08-07T15:13:47.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.058 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.059 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:47.060 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190690+0000", 2020-08-07T15:13:47.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0088507159999999998 2020-08-07T15:13:47.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199541+0000", 2020-08-07T15:13:47.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015864699999999999 2020-08-07T15:13:47.061 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.065 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.065 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199699+0000", 2020-08-07T15:13:47.066 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.23e-07 2020-08-07T15:13:47.066 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.066 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.067 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:47.067 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199700+0000", 2020-08-07T15:13:47.067 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5775e-05 2020-08-07T15:13:47.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.068 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.071 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:47.071 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199766+0000", 2020-08-07T15:13:47.071 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869466839999999 2020-08-07T15:13:47.071 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.072 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:47.072 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.072 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.072 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.072 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72986 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:47.073 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:13:47.073 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 261.22335209099998, 2020-08-07T15:13:47.073 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.586946441, 2020-08-07T15:13:47.073 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:47.073 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:47.074 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:47.074 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:47.075 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:47.075 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72986 2020-08-07T15:13:47.083 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.083 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:47.083 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:47.084 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:13:47.084 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:47.084 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.084 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.084 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:47.085 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:13:47.085 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9861999999999997e-05 2020-08-07T15:13:47.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.086 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:47.086 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612990+0000", 2020-08-07T15:13:47.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4350000000000001e-06 2020-08-07T15:13:47.086 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.086 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.086 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:47.087 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612993+0000", 2020-08-07T15:13:47.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9529999999999999e-06 2020-08-07T15:13:47.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.112 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:47.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613001+0000", 2020-08-07T15:13:47.113 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4309999999999998e-06 2020-08-07T15:13:47.113 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.113 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:47.114 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613004+0000", 2020-08-07T15:13:47.116 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5284000000000001e-05 2020-08-07T15:13:47.116 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.117 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.117 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613030+0000", 2020-08-07T15:13:47.117 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2400000000000001e-06 2020-08-07T15:13:47.117 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.118 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.118 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.118 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613034+0000", 2020-08-07T15:13:47.118 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81134136199999995 2020-08-07T15:13:47.118 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.119 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.119 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.119 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424375+0000", 2020-08-07T15:13:47.119 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3520000000000001e-06 2020-08-07T15:13:47.119 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.119 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.120 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.120 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424383+0000", 2020-08-07T15:13:47.120 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76064437799999995 2020-08-07T15:13:47.120 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.120 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.121 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.121 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185027+0000", 2020-08-07T15:13:47.121 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000119719 2020-08-07T15:13:47.124 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.125 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.126 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:47.127 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185147+0000", 2020-08-07T15:13:47.127 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016829400000000001 2020-08-07T15:13:47.127 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.127 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.128 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:47.128 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185315+0000", 2020-08-07T15:13:47.128 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0053821199999999998 2020-08-07T15:13:47.129 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.129 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.129 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:47.130 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190697+0000", 2020-08-07T15:13:47.130 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6899999999999996e-07 2020-08-07T15:13:47.130 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.131 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.131 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:47.131 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190698+0000", 2020-08-07T15:13:47.132 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2072000000000001e-05 2020-08-07T15:13:47.132 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.132 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.132 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:47.133 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190730+0000", 2020-08-07T15:13:47.134 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0088501440000000008 2020-08-07T15:13:47.134 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.134 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.135 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.137 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199580+0000", 2020-08-07T15:13:47.138 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00023199900000000001 2020-08-07T15:13:47.138 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.138 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.139 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.139 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199812+0000", 2020-08-07T15:13:47.139 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6300000000000005e-07 2020-08-07T15:13:47.140 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.140 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.140 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:47.141 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199813+0000", 2020-08-07T15:13:47.141 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3363999999999998e-05 2020-08-07T15:13:47.141 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.141 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.142 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:47.142 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199876+0000", 2020-08-07T15:13:47.142 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.586946441 2020-08-07T15:13:47.143 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.143 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:47.143 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.144 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.144 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.144 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72987 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:47.144 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:13:47.145 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 261.22325633600002, 2020-08-07T15:13:47.145 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869598709999999, 2020-08-07T15:13:47.145 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:47.145 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:47.146 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:47.146 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:47.148 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:47.148 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72987 2020-08-07T15:13:47.149 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.149 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:47.149 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.150 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:47.150 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:13:47.151 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:47.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.151 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:47.156 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:13:47.158 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1851999999999999e-05 2020-08-07T15:13:47.158 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.159 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.159 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:47.159 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613057+0000", 2020-08-07T15:13:47.160 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0429999999999999e-06 2020-08-07T15:13:47.160 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.160 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.161 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:47.161 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613060+0000", 2020-08-07T15:13:47.161 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1060000000000002e-06 2020-08-07T15:13:47.162 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.162 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.163 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:47.163 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613067+0000", 2020-08-07T15:13:47.163 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8040000000000002e-06 2020-08-07T15:13:47.164 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:47.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613070+0000", 2020-08-07T15:13:47.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6857000000000001e-05 2020-08-07T15:13:47.166 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.166 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.166 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.167 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613097+0000", 2020-08-07T15:13:47.167 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8770000000000001e-06 2020-08-07T15:13:47.167 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.168 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.168 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.169 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613102+0000", 2020-08-07T15:13:47.169 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81131066399999996 2020-08-07T15:13:47.169 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.169 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.169 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.170 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424413+0000", 2020-08-07T15:13:47.170 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4599999999999997e-06 2020-08-07T15:13:47.170 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.170 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.170 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.171 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424420+0000", 2020-08-07T15:13:47.171 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76132809700000004 2020-08-07T15:13:47.171 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.171 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.173 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.173 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185748+0000", 2020-08-07T15:13:47.173 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.4264000000000005e-05 2020-08-07T15:13:47.175 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.176 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.176 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:47.177 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185832+0000", 2020-08-07T15:13:47.177 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112803 2020-08-07T15:13:47.177 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.178 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.178 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:47.178 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185945+0000", 2020-08-07T15:13:47.179 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0089954130000000007 2020-08-07T15:13:47.179 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.179 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.180 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:47.180 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194941+0000", 2020-08-07T15:13:47.180 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.091e-06 2020-08-07T15:13:47.180 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.180 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.181 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:47.181 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194943+0000", 2020-08-07T15:13:47.181 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7423999999999996e-05 2020-08-07T15:13:47.181 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.181 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.182 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:47.182 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195020+0000", 2020-08-07T15:13:47.186 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045973300000000002 2020-08-07T15:13:47.187 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.187 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.187 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.187 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199618+0000", 2020-08-07T15:13:47.187 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000302009 2020-08-07T15:13:47.188 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.188 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.189 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.189 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199920+0000", 2020-08-07T15:13:47.191 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7999999999999996e-07 2020-08-07T15:13:47.191 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.191 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.191 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:47.192 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199920+0000", 2020-08-07T15:13:47.192 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5296999999999995e-05 2020-08-07T15:13:47.192 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.192 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.192 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:47.193 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199985+0000", 2020-08-07T15:13:47.193 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869598709999999 2020-08-07T15:13:47.193 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.193 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:47.193 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.193 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.194 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.194 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72988 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:47.194 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:13:47.194 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 261.22319175600001, 2020-08-07T15:13:47.194 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587111143, 2020-08-07T15:13:47.195 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:47.195 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:47.195 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:47.195 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:47.196 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:47.197 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72988 2020-08-07T15:13:47.197 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.197 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:47.197 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.197 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:47.198 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:13:47.198 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:47.198 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.199 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.199 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:47.199 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:13:47.199 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6563e-05 2020-08-07T15:13:47.200 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.200 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.200 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:47.201 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613117+0000", 2020-08-07T15:13:47.201 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7180000000000001e-06 2020-08-07T15:13:47.201 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.201 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.201 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:47.202 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613119+0000", 2020-08-07T15:13:47.202 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1789999999999999e-06 2020-08-07T15:13:47.202 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.202 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.202 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:47.203 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613124+0000", 2020-08-07T15:13:47.203 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3539999999999998e-06 2020-08-07T15:13:47.203 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.204 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.204 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:47.204 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613127+0000", 2020-08-07T15:13:47.206 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3697e-05 2020-08-07T15:13:47.207 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.207 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.208 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.208 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613151+0000", 2020-08-07T15:13:47.208 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7559999999999999e-06 2020-08-07T15:13:47.212 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.213 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.213 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.213 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613155+0000", 2020-08-07T15:13:47.213 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81128859600000003 2020-08-07T15:13:47.222 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.232 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.233 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.233 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424444+0000", 2020-08-07T15:13:47.239 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9619999999999996e-06 2020-08-07T15:13:47.240 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.240 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.240 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.241 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424451+0000", 2020-08-07T15:13:47.241 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76286712899999998 2020-08-07T15:13:47.241 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.241 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.247 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.247 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187318+0000", 2020-08-07T15:13:47.247 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011616300000000001 2020-08-07T15:13:47.248 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.248 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.248 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:47.249 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187434+0000", 2020-08-07T15:13:47.249 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019347400000000001 2020-08-07T15:13:47.249 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.249 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.249 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:47.250 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187628+0000", 2020-08-07T15:13:47.250 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074049090000000003 2020-08-07T15:13:47.250 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.250 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.250 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:47.251 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195033+0000", 2020-08-07T15:13:47.251 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1090000000000001e-06 2020-08-07T15:13:47.251 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.251 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.251 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:47.252 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195034+0000", 2020-08-07T15:13:47.252 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8566999999999997e-05 2020-08-07T15:13:47.253 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.254 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.254 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:47.254 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195092+0000", 2020-08-07T15:13:47.254 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004570457 2020-08-07T15:13:47.255 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.255 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.255 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.255 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199663+0000", 2020-08-07T15:13:47.255 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00046757899999999998 2020-08-07T15:13:47.255 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.256 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.256 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.256 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200130+0000", 2020-08-07T15:13:47.256 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6700000000000002e-07 2020-08-07T15:13:47.256 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.257 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.257 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:47.257 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200131+0000", 2020-08-07T15:13:47.257 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9963999999999996e-05 2020-08-07T15:13:47.257 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.258 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.258 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:47.258 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200201+0000", 2020-08-07T15:13:47.258 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587111143 2020-08-07T15:13:47.258 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.259 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:47.259 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.259 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.259 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.260 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72989 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:47.260 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:13:47.260 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 261.22313536799999, 2020-08-07T15:13:47.260 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5872506399999999, 2020-08-07T15:13:47.260 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:47.261 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:47.262 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:47.262 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:47.264 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:47.264 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72989 2020-08-07T15:13:47.264 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.264 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:47.265 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.265 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:47.265 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:13:47.265 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:47.266 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.266 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.267 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:47.267 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:13:47.267 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6081999999999999e-05 2020-08-07T15:13:47.267 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.268 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.268 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:47.268 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613172+0000", 2020-08-07T15:13:47.268 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6570000000000001e-06 2020-08-07T15:13:47.268 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.269 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.269 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:47.269 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613175+0000", 2020-08-07T15:13:47.269 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7740000000000001e-06 2020-08-07T15:13:47.269 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.270 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.270 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:47.270 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613181+0000", 2020-08-07T15:13:47.270 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2759999999999999e-06 2020-08-07T15:13:47.270 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.271 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.271 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:47.271 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613183+0000", 2020-08-07T15:13:47.271 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3892000000000001e-05 2020-08-07T15:13:47.271 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.272 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.272 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613207+0000", 2020-08-07T15:13:47.272 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9870000000000001e-06 2020-08-07T15:13:47.272 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.272 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.273 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.273 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613211+0000", 2020-08-07T15:13:47.279 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81126230700000002 2020-08-07T15:13:47.279 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.280 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.280 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.280 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424473+0000", 2020-08-07T15:13:47.280 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1460000000000001e-06 2020-08-07T15:13:47.280 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.281 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.281 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.282 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424480+0000", 2020-08-07T15:13:47.283 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76413073099999995 2020-08-07T15:13:47.283 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.305 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.306 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.306 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188610+0000", 2020-08-07T15:13:47.306 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011278899999999999 2020-08-07T15:13:47.306 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.307 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.307 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:47.307 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188723+0000", 2020-08-07T15:13:47.307 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000213422 2020-08-07T15:13:47.307 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.308 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.308 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:47.308 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188936+0000", 2020-08-07T15:13:47.308 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0061672929999999999 2020-08-07T15:13:47.308 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.309 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.309 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:47.309 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195104+0000", 2020-08-07T15:13:47.309 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8199999999999999e-07 2020-08-07T15:13:47.309 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.310 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.310 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:47.310 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195105+0000", 2020-08-07T15:13:47.310 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2056000000000004e-05 2020-08-07T15:13:47.310 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.311 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.311 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:47.311 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195167+0000", 2020-08-07T15:13:47.311 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0051009769999999996 2020-08-07T15:13:47.311 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.312 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.312 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.312 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200268+0000", 2020-08-07T15:13:47.312 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9428000000000001e-05 2020-08-07T15:13:47.312 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.313 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.313 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.313 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200327+0000", 2020-08-07T15:13:47.314 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9100000000000003e-07 2020-08-07T15:13:47.314 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.314 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.314 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:47.314 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200328+0000", 2020-08-07T15:13:47.315 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9250000000000003e-05 2020-08-07T15:13:47.315 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.315 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.315 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:47.315 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200397+0000", 2020-08-07T15:13:47.316 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5872506399999999 2020-08-07T15:13:47.316 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.316 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:47.316 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.316 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.317 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.317 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72990 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:47.317 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:13:47.317 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 261.22307807599998, 2020-08-07T15:13:47.318 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587335105, 2020-08-07T15:13:47.318 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:47.318 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:47.318 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:47.318 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:47.319 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:47.319 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72990 2020-08-07T15:13:47.319 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.319 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:47.319 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.320 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:47.320 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:13:47.321 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:47.321 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.321 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.322 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:47.322 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:13:47.322 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5276999999999999e-05 2020-08-07T15:13:47.322 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.323 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.323 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:47.323 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613229+0000", 2020-08-07T15:13:47.323 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7219999999999999e-06 2020-08-07T15:13:47.323 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.324 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.324 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:47.324 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613232+0000", 2020-08-07T15:13:47.324 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1170000000000003e-06 2020-08-07T15:13:47.324 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.324 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.325 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:47.325 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613237+0000", 2020-08-07T15:13:47.325 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2309999999999999e-06 2020-08-07T15:13:47.325 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.325 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.326 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:47.326 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613239+0000", 2020-08-07T15:13:47.326 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2469000000000001e-05 2020-08-07T15:13:47.326 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.327 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.327 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.327 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613261+0000", 2020-08-07T15:13:47.327 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9280000000000001e-06 2020-08-07T15:13:47.327 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.328 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.328 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.328 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613266+0000", 2020-08-07T15:13:47.328 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81123414599999999 2020-08-07T15:13:47.329 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.329 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.329 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.329 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424501+0000", 2020-08-07T15:13:47.332 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5300000000000002e-06 2020-08-07T15:13:47.332 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.332 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.332 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.332 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424507+0000", 2020-08-07T15:13:47.333 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76488660200000003 2020-08-07T15:13:47.333 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.333 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.333 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.333 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189394+0000", 2020-08-07T15:13:47.334 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6878000000000003e-05 2020-08-07T15:13:47.334 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.334 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.334 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:47.334 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189491+0000", 2020-08-07T15:13:47.335 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000204477 2020-08-07T15:13:47.335 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.335 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.335 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:47.335 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189695+0000", 2020-08-07T15:13:47.336 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0065180940000000003 2020-08-07T15:13:47.336 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.336 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.336 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:47.337 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196213+0000", 2020-08-07T15:13:47.337 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4920000000000001e-06 2020-08-07T15:13:47.337 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.337 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.338 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:47.338 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196215+0000", 2020-08-07T15:13:47.338 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9950000000000006e-05 2020-08-07T15:13:47.338 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.339 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.339 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:47.339 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196285+0000", 2020-08-07T15:13:47.339 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004163674 2020-08-07T15:13:47.340 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.340 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.340 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.340 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200448+0000", 2020-08-07T15:13:47.340 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6919999999999999e-05 2020-08-07T15:13:47.341 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.341 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.341 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.341 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200485+0000", 2020-08-07T15:13:47.341 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1699999999999998e-07 2020-08-07T15:13:47.341 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.342 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.342 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:47.342 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200486+0000", 2020-08-07T15:13:47.342 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3081000000000002e-05 2020-08-07T15:13:47.343 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.343 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.343 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:47.343 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200539+0000", 2020-08-07T15:13:47.343 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587335105 2020-08-07T15:13:47.344 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.344 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:47.344 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.344 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.344 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.344 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72991 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:47.345 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:13:47.345 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 261.22302255099999, 2020-08-07T15:13:47.345 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5876210340000001, 2020-08-07T15:13:47.345 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:47.346 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:47.346 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:47.347 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:47.347 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:47.347 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72991 2020-08-07T15:13:47.347 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.347 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:47.348 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.348 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:47.348 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:13:47.348 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:47.348 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.349 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.349 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:47.349 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:13:47.349 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5763999999999999e-05 2020-08-07T15:13:47.349 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.350 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.350 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:47.350 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613285+0000", 2020-08-07T15:13:47.350 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6479999999999999e-06 2020-08-07T15:13:47.350 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.350 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.351 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:47.351 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613288+0000", 2020-08-07T15:13:47.351 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1909999999999999e-06 2020-08-07T15:13:47.351 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.351 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.352 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:47.352 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613293+0000", 2020-08-07T15:13:47.352 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2560000000000001e-06 2020-08-07T15:13:47.352 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.352 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.353 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:47.353 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613295+0000", 2020-08-07T15:13:47.353 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2561e-05 2020-08-07T15:13:47.353 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.353 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.354 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.354 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613318+0000", 2020-08-07T15:13:47.354 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6349999999999999e-06 2020-08-07T15:13:47.354 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.355 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.355 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.355 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613321+0000", 2020-08-07T15:13:47.356 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81120592400000002 2020-08-07T15:13:47.356 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.356 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.356 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.356 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424527+0000", 2020-08-07T15:13:47.357 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8540000000000002e-06 2020-08-07T15:13:47.357 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.357 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.357 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.357 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424533+0000", 2020-08-07T15:13:47.358 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76623392800000001 2020-08-07T15:13:47.358 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.358 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.358 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.358 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190767+0000", 2020-08-07T15:13:47.359 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3990000000000001e-05 2020-08-07T15:13:47.359 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.359 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.359 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:47.359 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190841+0000", 2020-08-07T15:13:47.360 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011444599999999999 2020-08-07T15:13:47.360 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.360 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.360 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:47.360 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190955+0000", 2020-08-07T15:13:47.361 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0053397499999999999 2020-08-07T15:13:47.361 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.361 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.361 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:47.361 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196295+0000", 2020-08-07T15:13:47.362 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006e-06 2020-08-07T15:13:47.362 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.362 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.362 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:47.362 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196296+0000", 2020-08-07T15:13:47.363 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8940999999999997e-05 2020-08-07T15:13:47.363 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.363 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.364 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:47.364 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196355+0000", 2020-08-07T15:13:47.364 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042276700000000002 2020-08-07T15:13:47.364 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.364 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.365 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.365 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200583+0000", 2020-08-07T15:13:47.365 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00026035700000000001 2020-08-07T15:13:47.365 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.366 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.366 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.366 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200843+0000", 2020-08-07T15:13:47.366 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6700000000000003e-07 2020-08-07T15:13:47.366 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.367 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.367 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:47.367 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200844+0000", 2020-08-07T15:13:47.367 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6545999999999999e-05 2020-08-07T15:13:47.367 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.368 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.368 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:47.368 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200880+0000", 2020-08-07T15:13:47.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5876210340000001 2020-08-07T15:13:47.368 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.369 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:47.369 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.369 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.369 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72992 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:47.370 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:13:47.370 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 261.22193507200001, 2020-08-07T15:13:47.370 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5866241210000001, 2020-08-07T15:13:47.370 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:47.370 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:47.371 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:47.371 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:47.371 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:47.371 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72992 2020-08-07T15:13:47.372 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.372 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:47.372 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.372 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:47.372 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:13:47.373 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:47.373 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.373 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.373 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:47.373 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:13:47.374 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7543e-05 2020-08-07T15:13:47.374 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.374 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.374 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:47.374 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614374+0000", 2020-08-07T15:13:47.374 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9740000000000004e-06 2020-08-07T15:13:47.375 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.375 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:47.375 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614378+0000", 2020-08-07T15:13:47.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2660000000000007e-06 2020-08-07T15:13:47.376 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.376 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.376 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:47.376 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614387+0000", 2020-08-07T15:13:47.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2950000000000001e-06 2020-08-07T15:13:47.377 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:47.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614393+0000", 2020-08-07T15:13:47.377 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7279000000000001e-05 2020-08-07T15:13:47.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.378 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614420+0000", 2020-08-07T15:13:47.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9040000000000002e-06 2020-08-07T15:13:47.379 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.379 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.379 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.380 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614424+0000", 2020-08-07T15:13:47.380 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81012815699999996 2020-08-07T15:13:47.380 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424552+0000", 2020-08-07T15:13:47.381 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2879999999999998e-06 2020-08-07T15:13:47.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.382 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.382 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424558+0000", 2020-08-07T15:13:47.382 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76678665400000001 2020-08-07T15:13:47.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.383 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.383 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.383 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191345+0000", 2020-08-07T15:13:47.383 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010819600000000001 2020-08-07T15:13:47.383 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.383 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.384 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:47.384 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191453+0000", 2020-08-07T15:13:47.384 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000170829 2020-08-07T15:13:47.384 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.385 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.385 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:47.385 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191624+0000", 2020-08-07T15:13:47.385 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0047386679999999997 2020-08-07T15:13:47.386 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.386 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.386 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:47.386 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196363+0000", 2020-08-07T15:13:47.386 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6000000000000003e-07 2020-08-07T15:13:47.387 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.387 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.387 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:47.387 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196363+0000", 2020-08-07T15:13:47.388 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3421000000000002e-05 2020-08-07T15:13:47.388 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.388 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.389 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:47.389 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196417+0000", 2020-08-07T15:13:47.389 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042225539999999999 2020-08-07T15:13:47.389 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.390 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.390 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.390 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200639+0000", 2020-08-07T15:13:47.390 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00028031399999999999 2020-08-07T15:13:47.391 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.391 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.391 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.391 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200920+0000", 2020-08-07T15:13:47.392 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0299999999999999e-07 2020-08-07T15:13:47.392 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.392 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.392 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:47.392 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200920+0000", 2020-08-07T15:13:47.393 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0516000000000002e-05 2020-08-07T15:13:47.393 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.393 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.393 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:47.394 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200971+0000", 2020-08-07T15:13:47.394 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5866241210000001 2020-08-07T15:13:47.394 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.394 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:47.395 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.395 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.395 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.395 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73017 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:47.396 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:13:47.396 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 261.16642681899998, 2020-08-07T15:13:47.396 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.531181608, 2020-08-07T15:13:47.396 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:47.397 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:47.397 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:47.397 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:47.397 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:47.398 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73017 2020-08-07T15:13:47.399 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.399 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:47.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.400 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:47.400 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:13:47.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:47.400 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.400 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.401 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:47.401 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:13:47.401 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3018e-05 2020-08-07T15:13:47.401 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.403 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.403 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:47.403 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669898+0000", 2020-08-07T15:13:47.403 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5340000000000002e-06 2020-08-07T15:13:47.404 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.404 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.404 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:47.405 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669905+0000", 2020-08-07T15:13:47.405 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3307e-05 2020-08-07T15:13:47.405 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.406 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:47.406 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669919+0000", 2020-08-07T15:13:47.406 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7729999999999999e-06 2020-08-07T15:13:47.406 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.407 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.407 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:47.407 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669927+0000", 2020-08-07T15:13:47.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0361000000000001e-05 2020-08-07T15:13:47.408 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.408 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.408 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.409 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669977+0000", 2020-08-07T15:13:47.409 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0049999999999993e-06 2020-08-07T15:13:47.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.410 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.410 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.410 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669986+0000", 2020-08-07T15:13:47.410 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75459235199999997 2020-08-07T15:13:47.411 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.411 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.411 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.411 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424578+0000", 2020-08-07T15:13:47.412 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.925e-06 2020-08-07T15:13:47.412 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.412 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.413 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.413 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424584+0000", 2020-08-07T15:13:47.413 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76741703900000002 2020-08-07T15:13:47.413 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.414 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.414 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.414 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192001+0000", 2020-08-07T15:13:47.414 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000181196 2020-08-07T15:13:47.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.415 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:47.415 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192182+0000", 2020-08-07T15:13:47.416 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00021149299999999999 2020-08-07T15:13:47.416 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.416 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.416 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:47.417 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192394+0000", 2020-08-07T15:13:47.417 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004031242 2020-08-07T15:13:47.417 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.417 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.419 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:47.419 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196425+0000", 2020-08-07T15:13:47.419 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7800000000000001e-07 2020-08-07T15:13:47.419 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.420 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.420 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:47.420 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196426+0000", 2020-08-07T15:13:47.420 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5244999999999997e-05 2020-08-07T15:13:47.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.421 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.421 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:47.422 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196481+0000", 2020-08-07T15:13:47.422 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041989360000000003 2020-08-07T15:13:47.422 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.423 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.423 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200680+0000", 2020-08-07T15:13:47.423 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00032415299999999998 2020-08-07T15:13:47.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.424 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.424 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.424 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201004+0000", 2020-08-07T15:13:47.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9299999999999999e-07 2020-08-07T15:13:47.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.425 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:47.426 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201005+0000", 2020-08-07T15:13:47.426 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1857999999999999e-05 2020-08-07T15:13:47.426 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.427 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:47.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201037+0000", 2020-08-07T15:13:47.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.531181608 2020-08-07T15:13:47.428 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.428 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:47.428 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.434 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.434 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.435 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73018 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:47.435 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:13:47.435 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 261.16545901400002, 2020-08-07T15:13:47.436 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.530301616, 2020-08-07T15:13:47.436 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:47.436 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:47.437 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:47.437 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:47.437 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:47.437 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73018 2020-08-07T15:13:47.438 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.438 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:47.438 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.439 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:47.439 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:13:47.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:47.439 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.440 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:47.440 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:13:47.440 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9747000000000002e-05 2020-08-07T15:13:47.440 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.441 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.441 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:47.441 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670852+0000", 2020-08-07T15:13:47.443 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9430000000000002e-06 2020-08-07T15:13:47.443 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.443 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.444 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:47.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670856+0000", 2020-08-07T15:13:47.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0649999999999999e-06 2020-08-07T15:13:47.444 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.444 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.445 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:47.446 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670864+0000", 2020-08-07T15:13:47.446 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.924e-06 2020-08-07T15:13:47.446 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.447 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.447 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:47.447 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670868+0000", 2020-08-07T15:13:47.447 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6205000000000003e-05 2020-08-07T15:13:47.448 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670905+0000", 2020-08-07T15:13:47.449 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3289999999999996e-06 2020-08-07T15:13:47.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.449 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.449 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.450 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670912+0000", 2020-08-07T15:13:47.450 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75370200200000004 2020-08-07T15:13:47.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.451 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424614+0000", 2020-08-07T15:13:47.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5150000000000003e-06 2020-08-07T15:13:47.451 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424620+0000", 2020-08-07T15:13:47.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.768200153 2020-08-07T15:13:47.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.454 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.454 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192821+0000", 2020-08-07T15:13:47.454 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000111764 2020-08-07T15:13:47.454 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:47.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192932+0000", 2020-08-07T15:13:47.456 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017542400000000001 2020-08-07T15:13:47.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:47.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193108+0000", 2020-08-07T15:13:47.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004100851 2020-08-07T15:13:47.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.458 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:47.458 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197209+0000", 2020-08-07T15:13:47.458 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0100000000000001e-06 2020-08-07T15:13:47.458 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:47.459 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197210+0000", 2020-08-07T15:13:47.459 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9672999999999995e-05 2020-08-07T15:13:47.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.460 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.460 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:47.461 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197279+0000", 2020-08-07T15:13:47.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034275109999999998 2020-08-07T15:13:47.461 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.462 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.462 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.462 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200707+0000", 2020-08-07T15:13:47.463 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00036853100000000001 2020-08-07T15:13:47.463 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.463 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.463 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.464 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201075+0000", 2020-08-07T15:13:47.464 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8599999999999998e-07 2020-08-07T15:13:47.464 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.465 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.465 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:47.465 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201076+0000", 2020-08-07T15:13:47.465 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8483000000000003e-05 2020-08-07T15:13:47.466 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.466 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.466 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:47.467 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201124+0000", 2020-08-07T15:13:47.467 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.530301616 2020-08-07T15:13:47.467 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.467 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:47.468 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.468 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.468 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.469 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73019 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:47.469 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:13:47.469 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 261.16481136300001, 2020-08-07T15:13:47.469 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.529718224, 2020-08-07T15:13:47.470 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:47.470 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:47.470 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:47.471 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:47.471 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:47.471 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73019 2020-08-07T15:13:47.472 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.472 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:47.472 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.473 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:47.473 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:13:47.473 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:47.473 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.474 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.474 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:47.475 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:13:47.475 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3309000000000002e-05 2020-08-07T15:13:47.475 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.476 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.476 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:47.476 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671504+0000", 2020-08-07T15:13:47.476 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6619999999999998e-06 2020-08-07T15:13:47.477 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.477 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.477 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:47.477 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671507+0000", 2020-08-07T15:13:47.478 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2279999999999998e-06 2020-08-07T15:13:47.478 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.478 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.478 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:47.479 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671516+0000", 2020-08-07T15:13:47.479 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4860000000000002e-06 2020-08-07T15:13:47.479 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.480 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.480 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:47.480 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671519+0000", 2020-08-07T15:13:47.481 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0564000000000001e-05 2020-08-07T15:13:47.481 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.481 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.481 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.482 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671550+0000", 2020-08-07T15:13:47.482 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1029999999999998e-06 2020-08-07T15:13:47.482 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.483 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.483 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.483 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671556+0000", 2020-08-07T15:13:47.483 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75308759199999997 2020-08-07T15:13:47.484 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.484 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.484 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.484 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424643+0000", 2020-08-07T15:13:47.485 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9179999999999997e-06 2020-08-07T15:13:47.485 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.485 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.485 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.486 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424650+0000", 2020-08-07T15:13:47.486 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76890373599999995 2020-08-07T15:13:47.486 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.487 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.487 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.487 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193554+0000", 2020-08-07T15:13:47.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012345600000000001 2020-08-07T15:13:47.488 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:47.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193677+0000", 2020-08-07T15:13:47.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013528300000000001 2020-08-07T15:13:47.490 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.490 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.490 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:47.490 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193813+0000", 2020-08-07T15:13:47.491 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034777710000000002 2020-08-07T15:13:47.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.491 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.491 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:47.492 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197291+0000", 2020-08-07T15:13:47.492 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8000000000000004e-07 2020-08-07T15:13:47.492 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.492 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:47.493 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197291+0000", 2020-08-07T15:13:47.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9176999999999999e-05 2020-08-07T15:13:47.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.494 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:47.494 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197351+0000", 2020-08-07T15:13:47.494 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033931009999999999 2020-08-07T15:13:47.495 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.495 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.495 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.496 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200744+0000", 2020-08-07T15:13:47.496 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00041308000000000002 2020-08-07T15:13:47.496 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.496 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.497 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.497 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201157+0000", 2020-08-07T15:13:47.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8999999999999998e-07 2020-08-07T15:13:47.497 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.498 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.498 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:47.499 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201157+0000", 2020-08-07T15:13:47.499 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1588000000000003e-05 2020-08-07T15:13:47.499 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.499 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.500 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:47.500 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201189+0000", 2020-08-07T15:13:47.500 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.529718224 2020-08-07T15:13:47.500 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.500 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:47.501 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.501 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.501 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.503 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73020 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:47.503 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:13:47.504 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 261.09647350300003, 2020-08-07T15:13:47.504 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.461468373, 2020-08-07T15:13:47.504 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:47.504 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:47.505 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:47.505 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:47.505 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:47.506 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73020 2020-08-07T15:13:47.506 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.506 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:47.507 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.507 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:47.507 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:13:47.508 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:47.508 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.508 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.508 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:47.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:13:47.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.777e-05 2020-08-07T15:13:47.509 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.509 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.510 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:47.510 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739836+0000", 2020-08-07T15:13:47.510 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0180000000000003e-06 2020-08-07T15:13:47.511 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.511 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:47.512 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739840+0000", 2020-08-07T15:13:47.512 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6930000000000002e-06 2020-08-07T15:13:47.513 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.513 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.513 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:47.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739850+0000", 2020-08-07T15:13:47.514 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3619999999999999e-06 2020-08-07T15:13:47.514 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.514 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.515 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:47.515 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739854+0000", 2020-08-07T15:13:47.515 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7432999999999999e-05 2020-08-07T15:13:47.515 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.516 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.516 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.516 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739892+0000", 2020-08-07T15:13:47.517 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1149999999999996e-06 2020-08-07T15:13:47.517 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.517 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.517 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.518 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739899+0000", 2020-08-07T15:13:47.518 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68477883299999998 2020-08-07T15:13:47.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.519 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.526 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424677+0000", 2020-08-07T15:13:47.526 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7710000000000001e-06 2020-08-07T15:13:47.527 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.527 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.527 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.528 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424684+0000", 2020-08-07T15:13:47.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76949970700000003 2020-08-07T15:13:47.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.529 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194184+0000", 2020-08-07T15:13:47.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3957999999999997e-05 2020-08-07T15:13:47.530 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.530 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.530 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:47.530 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194278+0000", 2020-08-07T15:13:47.531 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017489599999999999 2020-08-07T15:13:47.531 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.531 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.531 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:47.532 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194453+0000", 2020-08-07T15:13:47.532 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002908908 2020-08-07T15:13:47.532 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.532 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.533 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:47.533 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197362+0000", 2020-08-07T15:13:47.533 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7299999999999995e-07 2020-08-07T15:13:47.534 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.534 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.534 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:47.534 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197362+0000", 2020-08-07T15:13:47.535 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6848000000000002e-05 2020-08-07T15:13:47.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:47.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197419+0000", 2020-08-07T15:13:47.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033597079999999999 2020-08-07T15:13:47.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.537 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.537 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200779+0000", 2020-08-07T15:13:47.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00044797899999999999 2020-08-07T15:13:47.538 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.538 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.538 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.539 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201227+0000", 2020-08-07T15:13:47.539 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7100000000000002e-07 2020-08-07T15:13:47.539 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.539 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.540 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:47.540 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201227+0000", 2020-08-07T15:13:47.540 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9230000000000001e-05 2020-08-07T15:13:47.540 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.541 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:47.541 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201277+0000", 2020-08-07T15:13:47.542 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.461468373 2020-08-07T15:13:47.542 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.542 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:47.543 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.543 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.543 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.543 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73021 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:47.544 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:13:47.545 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 261.033439896, 2020-08-07T15:13:47.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.398505079, 2020-08-07T15:13:47.545 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:47.546 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:47.546 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:47.546 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:47.546 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:47.547 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73021 2020-08-07T15:13:47.547 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.547 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:47.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.548 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:47.548 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:13:47.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:47.548 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.549 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.549 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:47.549 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:13:47.550 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7429000000000001e-05 2020-08-07T15:13:47.550 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.550 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.550 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:47.551 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802869+0000", 2020-08-07T15:13:47.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7840000000000001e-06 2020-08-07T15:13:47.551 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.552 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:47.552 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802873+0000", 2020-08-07T15:13:47.552 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0119999999999994e-06 2020-08-07T15:13:47.553 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.553 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.553 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:47.554 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802882+0000", 2020-08-07T15:13:47.554 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.04e-06 2020-08-07T15:13:47.554 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.554 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.555 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:47.555 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802887+0000", 2020-08-07T15:13:47.555 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7076000000000002e-05 2020-08-07T15:13:47.555 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.556 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802924+0000", 2020-08-07T15:13:47.556 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9280000000000004e-06 2020-08-07T15:13:47.557 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.557 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.557 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.557 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802931+0000", 2020-08-07T15:13:47.558 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.62178051000000001 2020-08-07T15:13:47.558 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.558 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.558 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.559 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424712+0000", 2020-08-07T15:13:47.559 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4749999999999996e-06 2020-08-07T15:13:47.559 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.559 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.560 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.560 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424719+0000", 2020-08-07T15:13:47.560 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77047786799999995 2020-08-07T15:13:47.560 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.561 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.561 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.561 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195197+0000", 2020-08-07T15:13:47.562 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000101909 2020-08-07T15:13:47.562 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.562 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.562 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:47.563 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195299+0000", 2020-08-07T15:13:47.563 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000248247 2020-08-07T15:13:47.563 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.564 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:47.564 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195547+0000", 2020-08-07T15:13:47.564 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033458699999999999 2020-08-07T15:13:47.565 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.565 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.566 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:47.566 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198893+0000", 2020-08-07T15:13:47.567 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0419999999999999e-06 2020-08-07T15:13:47.567 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.567 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.568 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:47.568 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198894+0000", 2020-08-07T15:13:47.568 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9354999999999997e-05 2020-08-07T15:13:47.569 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.569 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.569 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:47.569 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198953+0000", 2020-08-07T15:13:47.570 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001861096 2020-08-07T15:13:47.570 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.570 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.571 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.571 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200815+0000", 2020-08-07T15:13:47.571 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00049697299999999995 2020-08-07T15:13:47.571 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.572 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.572 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.572 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201311+0000", 2020-08-07T15:13:47.573 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7099999999999998e-07 2020-08-07T15:13:47.573 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.573 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.573 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:47.574 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201312+0000", 2020-08-07T15:13:47.574 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5194000000000002e-05 2020-08-07T15:13:47.574 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.575 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:47.575 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201347+0000", 2020-08-07T15:13:47.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.398505079 2020-08-07T15:13:47.576 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.576 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:47.576 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.577 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.577 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.577 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73022 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:47.578 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:13:47.578 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 260.95226784099998, 2020-08-07T15:13:47.578 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3238320669999999, 2020-08-07T15:13:47.579 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:47.579 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:47.579 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:47.579 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:47.580 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:47.580 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73022 2020-08-07T15:13:47.580 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.580 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:47.581 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.581 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:47.581 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:13:47.581 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:47.582 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.582 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.582 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:47.583 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:13:47.583 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6769999999999999e-05 2020-08-07T15:13:47.583 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.583 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.584 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:47.584 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884041+0000", 2020-08-07T15:13:47.584 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.703e-06 2020-08-07T15:13:47.584 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.585 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.585 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:47.585 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884044+0000", 2020-08-07T15:13:47.586 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5949999999999999e-06 2020-08-07T15:13:47.586 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.586 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.587 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:47.587 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884053+0000", 2020-08-07T15:13:47.587 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000003e-06 2020-08-07T15:13:47.587 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.588 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:47.588 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884058+0000", 2020-08-07T15:13:47.588 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7281999999999999e-05 2020-08-07T15:13:47.589 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.589 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.589 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884105+0000", 2020-08-07T15:13:47.590 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.044e-05 2020-08-07T15:13:47.590 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.590 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.590 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.591 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884116+0000", 2020-08-07T15:13:47.591 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.54062884300000003 2020-08-07T15:13:47.591 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.591 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.592 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.592 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424745+0000", 2020-08-07T15:13:47.592 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3149999999999998e-06 2020-08-07T15:13:47.592 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.593 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.593 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.593 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424754+0000", 2020-08-07T15:13:47.594 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77175732200000002 2020-08-07T15:13:47.594 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.594 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.595 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.595 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196511+0000", 2020-08-07T15:13:47.595 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010110799999999999 2020-08-07T15:13:47.595 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.596 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.596 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:47.596 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196612+0000", 2020-08-07T15:13:47.597 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017543100000000001 2020-08-07T15:13:47.597 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.597 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.597 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:47.598 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196788+0000", 2020-08-07T15:13:47.598 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032134569999999999 2020-08-07T15:13:47.598 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.598 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.599 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:47.599 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200001+0000", 2020-08-07T15:13:47.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.071e-06 2020-08-07T15:13:47.599 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.600 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.600 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:47.600 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200002+0000", 2020-08-07T15:13:47.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2527999999999995e-05 2020-08-07T15:13:47.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.601 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:47.601 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200065+0000", 2020-08-07T15:13:47.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001327433 2020-08-07T15:13:47.602 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.602 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.603 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.603 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201392+0000", 2020-08-07T15:13:47.603 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063854030000000004 2020-08-07T15:13:47.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207778+0000", 2020-08-07T15:13:47.605 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.285e-06 2020-08-07T15:13:47.605 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.605 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.609 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:47.609 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207779+0000", 2020-08-07T15:13:47.609 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7070999999999995e-05 2020-08-07T15:13:47.610 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.610 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.610 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:47.611 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207846+0000", 2020-08-07T15:13:47.611 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3238320669999999 2020-08-07T15:13:47.611 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.611 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:47.612 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.612 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.612 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.613 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73023 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:13:47.613 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:13:47.613 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 260.8391062, 2020-08-07T15:13:47.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2113701729999999, 2020-08-07T15:13:47.614 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:47.614 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:47.615 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:47.615 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:13:47.615 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:47.615 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73023 2020-08-07T15:13:47.616 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.616 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:47.616 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.617 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:47.617 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:13:47.617 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:47.617 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.618 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.618 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:47.619 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:13:47.619 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4105e-05 2020-08-07T15:13:47.619 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.619 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.620 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:47.620 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997210+0000", 2020-08-07T15:13:47.620 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1089999999999997e-06 2020-08-07T15:13:47.621 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.621 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.621 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:47.621 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997215+0000", 2020-08-07T15:13:47.622 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7040000000000006e-06 2020-08-07T15:13:47.622 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.622 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.623 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:47.623 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997224+0000", 2020-08-07T15:13:47.623 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4439999999999999e-06 2020-08-07T15:13:47.623 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.624 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.624 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:47.624 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997230+0000", 2020-08-07T15:13:47.624 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5977999999999999e-05 2020-08-07T15:13:47.625 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.625 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.625 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.626 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997276+0000", 2020-08-07T15:13:47.626 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2660000000000007e-06 2020-08-07T15:13:47.626 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.627 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.627 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997285+0000", 2020-08-07T15:13:47.627 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.42749189599999998 2020-08-07T15:13:47.628 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.628 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.628 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.628 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424777+0000", 2020-08-07T15:13:47.629 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8940000000000004e-06 2020-08-07T15:13:47.629 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.629 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.630 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.643 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424784+0000", 2020-08-07T15:13:47.644 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77268199699999995 2020-08-07T15:13:47.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.644 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.644 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.645 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197466+0000", 2020-08-07T15:13:47.645 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011277699999999999 2020-08-07T15:13:47.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:47.646 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197579+0000", 2020-08-07T15:13:47.646 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018970599999999999 2020-08-07T15:13:47.646 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.646 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.647 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:13:47.647 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197768+0000", 2020-08-07T15:13:47.647 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043168529999999998 2020-08-07T15:13:47.647 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.647 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.648 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:47.648 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202085+0000", 2020-08-07T15:13:47.648 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.186e-06 2020-08-07T15:13:47.648 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.648 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.649 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:47.649 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202086+0000", 2020-08-07T15:13:47.649 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8975999999999999e-05 2020-08-07T15:13:47.650 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.650 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.651 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:47.651 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202145+0000", 2020-08-07T15:13:47.651 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00035331499999999999 2020-08-07T15:13:47.651 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.651 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.652 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.652 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202499+0000", 2020-08-07T15:13:47.652 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0059703159999999998 2020-08-07T15:13:47.652 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.652 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.653 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.653 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208469+0000", 2020-08-07T15:13:47.653 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7400000000000002e-07 2020-08-07T15:13:47.653 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.653 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.654 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:47.654 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208470+0000", 2020-08-07T15:13:47.654 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5877000000000006e-05 2020-08-07T15:13:47.654 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.654 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.655 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:47.655 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208546+0000", 2020-08-07T15:13:47.655 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2113701729999999 2020-08-07T15:13:47.655 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.655 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:47.656 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.656 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.656 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.656 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5481.0:74192 6.e e269/230 6:7ab1a263:::benchmark_data_smithi074_63843_object9273:head v 269'4368, mlcod=268'4362)", 2020-08-07T15:13:47.656 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:01.594532+0000", 2020-08-07T15:13:47.656 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 45.241749503000001, 2020-08-07T15:13:47.657 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.95244114199999996, 2020-08-07T15:13:47.657 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:47.657 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:47.658 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:47.659 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.659 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:47.659 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.594532+0000", 2020-08-07T15:13:47.659 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:47.663 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.663 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.663 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:47.663 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.594532+0000", 2020-08-07T15:13:47.664 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1347999999999997e-05 2020-08-07T15:13:47.664 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.664 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:47.664 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.594584+0000", 2020-08-07T15:13:47.665 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8689999999999998e-06 2020-08-07T15:13:47.665 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.665 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.665 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:47.665 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.594590+0000", 2020-08-07T15:13:47.666 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.95108761900000005 2020-08-07T15:13:47.666 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.666 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.666 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:47.666 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.545678+0000", 2020-08-07T15:13:47.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.271e-06 2020-08-07T15:13:47.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.667 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:47.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.545683+0000", 2020-08-07T15:13:47.668 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000113051 2020-08-07T15:13:47.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.668 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.545796+0000", 2020-08-07T15:13:47.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6154000000000001e-05 2020-08-07T15:13:47.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:47.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.545823+0000", 2020-08-07T15:13:47.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010668959999999999 2020-08-07T15:13:47.670 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.670 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.670 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:47.670 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.546889+0000", 2020-08-07T15:13:47.670 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3934e-05 2020-08-07T15:13:47.671 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.671 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.672 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:47.672 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.546973+0000", 2020-08-07T15:13:47.672 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.95244114199999996 2020-08-07T15:13:47.672 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.672 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:47.676 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.677 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.677 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:74203 6.9 6:9b24f442:::benchmark_data_smithi074_63843_object9275:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e269)", 2020-08-07T15:13:47.678 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:01.691147+0000", 2020-08-07T15:13:47.678 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 45.145134399, 2020-08-07T15:13:47.678 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.91286480199999998, 2020-08-07T15:13:47.678 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:47.678 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:47.679 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:47.679 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:47.679 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:47.679 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 74203 2020-08-07T15:13:47.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.680 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:47.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:47.681 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.691147+0000", 2020-08-07T15:13:47.681 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:47.682 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.682 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.682 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:47.682 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.691147+0000", 2020-08-07T15:13:47.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0080999999999999e-05 2020-08-07T15:13:47.683 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.683 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.683 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:47.683 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.691177+0000", 2020-08-07T15:13:47.684 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7670000000000003e-06 2020-08-07T15:13:47.684 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.684 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.684 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:47.684 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.691182+0000", 2020-08-07T15:13:47.685 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0392e-05 2020-08-07T15:13:47.685 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.685 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.685 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:47.685 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.691193+0000", 2020-08-07T15:13:47.686 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5570000000000003e-06 2020-08-07T15:13:47.686 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.686 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.687 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:47.687 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.691198+0000", 2020-08-07T15:13:47.687 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0670000000000001e-05 2020-08-07T15:13:47.687 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.688 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.688 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.688 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.691249+0000", 2020-08-07T15:13:47.688 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0593e-05 2020-08-07T15:13:47.688 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.689 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.689 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:47.689 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.691259+0000", 2020-08-07T15:13:47.689 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.90310043900000003 2020-08-07T15:13:47.689 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.690 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.690 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:47.690 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.594360+0000", 2020-08-07T15:13:47.690 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000107182 2020-08-07T15:13:47.690 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.691 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.692 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:47.692 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.594467+0000", 2020-08-07T15:13:47.692 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00022166199999999999 2020-08-07T15:13:47.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.693 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:13:47.693 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.594689+0000", 2020-08-07T15:13:47.693 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042962399999999998 2020-08-07T15:13:47.693 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.693 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.694 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:47.694 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.598985+0000", 2020-08-07T15:13:47.694 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0759999999999999e-06 2020-08-07T15:13:47.694 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.694 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.695 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:47.695 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.598986+0000", 2020-08-07T15:13:47.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0080000000000002e-05 2020-08-07T15:13:47.695 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.696 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:47.696 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.599036+0000", 2020-08-07T15:13:47.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00053325400000000004 2020-08-07T15:13:47.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.599569+0000", 2020-08-07T15:13:47.698 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043431800000000003 2020-08-07T15:13:47.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.698 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.698 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:47.698 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.603913+0000", 2020-08-07T15:13:47.699 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5600000000000005e-07 2020-08-07T15:13:47.699 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.699 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.699 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:47.699 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.603913+0000", 2020-08-07T15:13:47.700 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8973000000000003e-05 2020-08-07T15:13:47.700 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:47.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:47.700 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:47.700 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.604012+0000", 2020-08-07T15:13:47.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.91286480199999998 2020-08-07T15:13:47.702 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.702 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:47.703 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.703 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:47.703 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:47.703 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:47.703 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:46.841+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:47.704 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2020-08-07T15:13:47.705 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:13:47.705 INFO:tasks.thrashosds.thrasher:inject_pause on 6 2020-08-07T15:13:47.705 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2020-08-07T15:13:47.705 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2020-08-07T15:13:47.706 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2020-08-07T15:13:47.708 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:47.703+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:47.709 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:13:47.831 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:47.804+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:47.848 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:13:47.878 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:47.878 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "" 2020-08-07T15:13:47.878 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:47.909 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:47.906+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:48.010 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:48.007+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:48.022 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:48.022 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:13:48.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.023 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:143407 6.c 6:39f59b26:::benchmark_data_smithi074_63843_object10155:head [delete] snapc 0=[] ondisk+write+known_if_redirected e297)", 2020-08-07T15:13:48.023 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:48.005678+0000", 2020-08-07T15:13:48.023 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.011915426, 2020-08-07T15:13:48.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.011940498000000001, 2020-08-07T15:13:48.024 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:48.024 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:48.025 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:48.025 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:48.025 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:48.025 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 143407 2020-08-07T15:13:48.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.026 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:48.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:48.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.005678+0000", 2020-08-07T15:13:48.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:48.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:48.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.005678+0000", 2020-08-07T15:13:48.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4627000000000003e-05 2020-08-07T15:13:48.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.030 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.030 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:48.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.005743+0000", 2020-08-07T15:13:48.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1120000000000001e-06 2020-08-07T15:13:48.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:48.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.005749+0000", 2020-08-07T15:13:48.033 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011206199999999999 2020-08-07T15:13:48.033 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.033 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.033 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:48.034 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.006870+0000", 2020-08-07T15:13:48.034 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3189999999999997e-06 2020-08-07T15:13:48.034 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.035 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.035 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:48.035 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.006877+0000", 2020-08-07T15:13:48.035 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8891000000000003e-05 2020-08-07T15:13:48.036 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.036 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.036 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:48.037 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.006926+0000", 2020-08-07T15:13:48.037 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000160366 2020-08-07T15:13:48.037 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.038 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:48.038 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.007086+0000", 2020-08-07T15:13:48.038 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00033018199999999999 2020-08-07T15:13:48.039 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.039 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.039 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:13:48.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.007416+0000", 2020-08-07T15:13:48.040 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001232304 2020-08-07T15:13:48.040 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.041 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:48.041 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.008649+0000", 2020-08-07T15:13:48.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1299999999999998e-07 2020-08-07T15:13:48.051 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.053 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:48.054 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.008650+0000", 2020-08-07T15:13:48.054 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8214999999999997e-05 2020-08-07T15:13:48.054 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.055 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:48.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.008718+0000", 2020-08-07T15:13:48.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0030395489999999999 2020-08-07T15:13:48.056 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:48.056 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:48.057 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:48.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.058 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:143420 6.b 6:dc55dbfc:::benchmark_data_smithi074_63843_object10168:head [delete] snapc 0=[] ondisk+write+known_if_redirected e297)", 2020-08-07T15:13:48.058 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:48.007671+0000", 2020-08-07T15:13:48.058 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0099221980000000001, 2020-08-07T15:13:48.058 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.010072446000000001, 2020-08-07T15:13:48.059 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:48.059 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:48.059 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:48.059 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:48.060 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:48.060 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 143420 2020-08-07T15:13:48.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.060 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:48.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.061 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:48.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.007671+0000", 2020-08-07T15:13:48.061 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:48.063 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.064 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.064 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:48.064 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.007671+0000", 2020-08-07T15:13:48.065 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00071352000000000004 2020-08-07T15:13:48.065 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.065 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.065 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:48.066 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.008385+0000", 2020-08-07T15:13:48.066 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7710000000000001e-06 2020-08-07T15:13:48.066 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.067 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.067 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:48.067 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.008392+0000", 2020-08-07T15:13:48.067 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3096e-05 2020-08-07T15:13:48.068 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.068 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.068 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:48.068 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.008415+0000", 2020-08-07T15:13:48.069 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4479999999999999e-06 2020-08-07T15:13:48.069 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.069 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.070 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:48.070 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.008421+0000", 2020-08-07T15:13:48.070 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0890000000000002e-05 2020-08-07T15:13:48.071 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.071 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.072 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:48.072 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.008472+0000", 2020-08-07T15:13:48.072 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000159097 2020-08-07T15:13:48.072 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.073 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.073 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:48.073 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.008631+0000", 2020-08-07T15:13:48.073 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000139355 2020-08-07T15:13:48.074 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.075 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.075 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,7", 2020-08-07T15:13:48.075 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.008771+0000", 2020-08-07T15:13:48.076 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045540770000000001 2020-08-07T15:13:48.076 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.076 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.076 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:48.076 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.013325+0000", 2020-08-07T15:13:48.077 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.176e-06 2020-08-07T15:13:48.077 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.077 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.077 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:48.078 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.013326+0000", 2020-08-07T15:13:48.078 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4447000000000001e-05 2020-08-07T15:13:48.078 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.079 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.079 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:48.079 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.013390+0000", 2020-08-07T15:13:48.080 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023470040000000002 2020-08-07T15:13:48.080 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.080 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.081 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:48.081 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.015737+0000", 2020-08-07T15:13:48.081 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0080658810000000004 2020-08-07T15:13:48.081 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:48.082 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:48.082 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:48.082 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.083 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:143399 6.c 6:3a48120a:::benchmark_data_smithi074_63843_object10147:head [delete] snapc 0=[] ondisk+write+known_if_redirected e297)", 2020-08-07T15:13:48.083 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:47.997487+0000", 2020-08-07T15:13:48.083 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.020107074999999999, 2020-08-07T15:13:48.084 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.020401109000000001, 2020-08-07T15:13:48.084 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:48.085 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:48.085 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:48.086 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:48.086 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:48.086 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 143399 2020-08-07T15:13:48.086 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.086 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:48.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:48.087 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.997487+0000", 2020-08-07T15:13:48.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:48.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:48.088 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.997487+0000", 2020-08-07T15:13:48.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0787000000000002e-05 2020-08-07T15:13:48.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.089 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.089 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:48.089 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.997517+0000", 2020-08-07T15:13:48.089 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8140000000000003e-06 2020-08-07T15:13:48.089 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.090 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.090 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:48.090 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.997524+0000", 2020-08-07T15:13:48.090 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1576000000000001e-05 2020-08-07T15:13:48.090 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.091 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.091 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:48.091 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.997536+0000", 2020-08-07T15:13:48.091 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4820000000000002e-06 2020-08-07T15:13:48.091 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.091 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.092 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:48.092 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.997541+0000", 2020-08-07T15:13:48.092 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0022596959999999998 2020-08-07T15:13:48.092 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.092 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.093 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:48.093 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.999801+0000", 2020-08-07T15:13:48.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016754399999999999 2020-08-07T15:13:48.096 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.096 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.096 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:48.097 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.999968+0000", 2020-08-07T15:13:48.097 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016019600000000001 2020-08-07T15:13:48.097 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.097 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.097 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:13:48.098 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.000129+0000", 2020-08-07T15:13:48.098 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026796860000000001 2020-08-07T15:13:48.098 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.098 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.098 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:48.099 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.002808+0000", 2020-08-07T15:13:48.099 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.4900000000000004e-07 2020-08-07T15:13:48.099 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.099 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.099 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:48.100 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.002809+0000", 2020-08-07T15:13:48.100 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4030999999999997e-05 2020-08-07T15:13:48.100 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.100 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.100 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:48.101 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.002873+0000", 2020-08-07T15:13:48.101 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0060986640000000002 2020-08-07T15:13:48.101 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.101 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.101 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:48.102 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.008972+0000", 2020-08-07T15:13:48.102 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0088735159999999997 2020-08-07T15:13:48.102 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.102 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.102 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:48.103 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.017846+0000", 2020-08-07T15:13:48.103 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2669999999999999e-06 2020-08-07T15:13:48.103 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.103 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.103 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:48.103 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.017847+0000", 2020-08-07T15:13:48.105 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.020360208000000001 2020-08-07T15:13:48.106 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:48.106 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:48.106 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:48.106 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.107 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.107 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:143367 6.6 6:66f2c516:::benchmark_data_smithi074_63843_object10115:head [delete] snapc 0=[] ondisk+write+known_if_redirected e297)", 2020-08-07T15:13:48.108 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:47.979729+0000", 2020-08-07T15:13:48.108 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.037864653999999998, 2020-08-07T15:13:48.108 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.038299847999999997, 2020-08-07T15:13:48.108 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:48.108 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:48.109 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:48.109 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:48.109 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:48.109 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 143367 2020-08-07T15:13:48.109 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.110 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:48.110 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.110 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:48.110 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.979729+0000", 2020-08-07T15:13:48.111 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:48.111 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.111 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.111 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:48.111 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.979729+0000", 2020-08-07T15:13:48.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2923999999999999e-05 2020-08-07T15:13:48.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.112 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:48.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.979762+0000", 2020-08-07T15:13:48.113 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8480000000000003e-06 2020-08-07T15:13:48.113 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.113 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.113 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:48.113 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.979769+0000", 2020-08-07T15:13:48.114 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4717e-05 2020-08-07T15:13:48.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.114 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:48.114 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.979784+0000", 2020-08-07T15:13:48.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4429999999999997e-06 2020-08-07T15:13:48.115 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.116 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.116 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:48.116 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.979791+0000", 2020-08-07T15:13:48.116 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6245000000000001e-05 2020-08-07T15:13:48.117 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.117 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:48.118 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.979847+0000", 2020-08-07T15:13:48.118 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018302499999999999 2020-08-07T15:13:48.118 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.118 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.119 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:48.119 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.980030+0000", 2020-08-07T15:13:48.119 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000195711 2020-08-07T15:13:48.119 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.119 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.120 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,7", 2020-08-07T15:13:48.120 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.980226+0000", 2020-08-07T15:13:48.120 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001871824 2020-08-07T15:13:48.120 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.120 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.121 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:48.121 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.982098+0000", 2020-08-07T15:13:48.121 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.353e-06 2020-08-07T15:13:48.121 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.122 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.122 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:48.122 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.982099+0000", 2020-08-07T15:13:48.122 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8849000000000004e-05 2020-08-07T15:13:48.122 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.123 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.123 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:48.123 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.982168+0000", 2020-08-07T15:13:48.123 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024389390000000002 2020-08-07T15:13:48.123 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:48.124 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:48.124 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:48.124 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.124 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.124 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:143427 6.c 6:3fe02041:::benchmark_data_smithi074_63843_object10175:head [delete] snapc 0=[] ondisk+write+known_if_redirected e297)", 2020-08-07T15:13:48.125 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:48.012971+0000", 2020-08-07T15:13:48.125 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0046222640000000001, 2020-08-07T15:13:48.125 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0051615810000000002, 2020-08-07T15:13:48.126 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:48.126 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:48.126 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:48.126 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:48.127 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:48.127 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 143427 2020-08-07T15:13:48.127 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.127 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:48.128 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.128 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:48.128 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.012971+0000", 2020-08-07T15:13:48.129 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:48.129 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.129 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.129 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:48.130 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.012971+0000", 2020-08-07T15:13:48.130 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6207000000000001e-05 2020-08-07T15:13:48.130 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.130 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.130 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:48.131 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.013008+0000", 2020-08-07T15:13:48.131 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6699999999999997e-06 2020-08-07T15:13:48.131 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.131 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.131 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:48.132 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.013014+0000", 2020-08-07T15:13:48.132 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3882e-05 2020-08-07T15:13:48.132 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.132 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.132 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:48.133 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.013028+0000", 2020-08-07T15:13:48.133 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5930000000000006e-06 2020-08-07T15:13:48.133 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.133 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.133 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:48.134 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.013037+0000", 2020-08-07T15:13:48.134 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6056e-05 2020-08-07T15:13:48.134 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.134 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.135 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:48.135 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.013093+0000", 2020-08-07T15:13:48.135 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016228799999999999 2020-08-07T15:13:48.135 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.135 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.136 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:48.136 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.013255+0000", 2020-08-07T15:13:48.136 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000191953 2020-08-07T15:13:48.137 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.137 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.137 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:13:48.137 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.013447+0000", 2020-08-07T15:13:48.138 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0025061300000000001 2020-08-07T15:13:48.138 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.138 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.139 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:48.139 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.015953+0000", 2020-08-07T15:13:48.139 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0160000000000001e-06 2020-08-07T15:13:48.139 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.140 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.140 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:48.140 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.015954+0000", 2020-08-07T15:13:48.140 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000111399 2020-08-07T15:13:48.140 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.141 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.141 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:48.141 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.016066+0000", 2020-08-07T15:13:48.141 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0030941940000000002 2020-08-07T15:13:48.142 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:48.142 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:48.142 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:48.142 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.142 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.143 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:143433 6.6 6:6b3c7ae9:::benchmark_data_smithi074_63843_object10181:head [delete] snapc 0=[] ondisk+write+known_if_redirected e297)", 2020-08-07T15:13:48.143 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:48.015141+0000", 2020-08-07T15:13:48.143 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.002453092, 2020-08-07T15:13:48.143 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0031336630000000001, 2020-08-07T15:13:48.143 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:48.144 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:48.144 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:48.144 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:48.144 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:48.144 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 143433 2020-08-07T15:13:48.145 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.145 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:48.145 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.145 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:48.145 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.015141+0000", 2020-08-07T15:13:48.146 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:48.146 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.146 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.147 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:48.147 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.015141+0000", 2020-08-07T15:13:48.147 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0942999999999998e-05 2020-08-07T15:13:48.148 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.148 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.148 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:48.148 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.015172+0000", 2020-08-07T15:13:48.148 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4919999999999998e-06 2020-08-07T15:13:48.149 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.149 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.150 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:48.150 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.015178+0000", 2020-08-07T15:13:48.150 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2415000000000001e-05 2020-08-07T15:13:48.150 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.150 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.151 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:48.151 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.015190+0000", 2020-08-07T15:13:48.151 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8420000000000001e-06 2020-08-07T15:13:48.151 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:48.152 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.015196+0000", 2020-08-07T15:13:48.152 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7027999999999998e-05 2020-08-07T15:13:48.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.153 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.153 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:48.153 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.015243+0000", 2020-08-07T15:13:48.153 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016484 2020-08-07T15:13:48.153 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.154 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.154 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:48.154 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.015408+0000", 2020-08-07T15:13:48.154 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000173501 2020-08-07T15:13:48.154 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.155 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.155 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,7", 2020-08-07T15:13:48.155 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.015582+0000", 2020-08-07T15:13:48.155 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019833450000000001 2020-08-07T15:13:48.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.156 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:48.156 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.017565+0000", 2020-08-07T15:13:48.157 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.035e-06 2020-08-07T15:13:48.157 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.158 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.158 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:48.158 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.017566+0000", 2020-08-07T15:13:48.158 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2923000000000001e-05 2020-08-07T15:13:48.158 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.159 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.159 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:48.159 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.017619+0000", 2020-08-07T15:13:48.160 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024783639999999998 2020-08-07T15:13:48.160 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:48.160 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:48.161 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:48.161 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.161 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.161 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:143403 6.0 6:00b402d2:::benchmark_data_smithi074_63843_object10151:head [delete] snapc 0=[] ondisk+write+known_if_redirected e297)", 2020-08-07T15:13:48.161 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:47.998271+0000", 2020-08-07T15:13:48.162 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.019322269999999999, 2020-08-07T15:13:48.162 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.020121842000000001, 2020-08-07T15:13:48.162 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:48.162 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:48.162 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:48.163 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:48.163 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:48.163 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 143403 2020-08-07T15:13:48.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.163 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:48.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:48.164 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.998271+0000", 2020-08-07T15:13:48.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:48.164 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.165 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.165 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:48.165 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.998271+0000", 2020-08-07T15:13:48.165 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1375999999999999e-05 2020-08-07T15:13:48.166 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.166 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.166 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:48.166 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.998303+0000", 2020-08-07T15:13:48.166 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0510000000000002e-06 2020-08-07T15:13:48.167 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.167 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.167 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:48.167 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.998309+0000", 2020-08-07T15:13:48.168 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.208e-06 2020-08-07T15:13:48.168 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.168 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.169 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:48.169 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.998317+0000", 2020-08-07T15:13:48.169 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6380000000000002e-06 2020-08-07T15:13:48.169 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.169 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.170 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:48.170 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.998321+0000", 2020-08-07T15:13:48.171 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00028547799999999998 2020-08-07T15:13:48.171 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.171 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.171 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:48.171 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.998606+0000", 2020-08-07T15:13:48.172 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015973800000000001 2020-08-07T15:13:48.172 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.172 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.172 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:48.172 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.998766+0000", 2020-08-07T15:13:48.173 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00052247399999999999 2020-08-07T15:13:48.173 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.173 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.173 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,2", 2020-08-07T15:13:48.174 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:47.999288+0000", 2020-08-07T15:13:48.174 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00084502899999999996 2020-08-07T15:13:48.174 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.174 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.175 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:48.175 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.000133+0000", 2020-08-07T15:13:48.175 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9599999999999998e-07 2020-08-07T15:13:48.175 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.176 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.176 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:48.176 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.000134+0000", 2020-08-07T15:13:48.177 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9242000000000001e-05 2020-08-07T15:13:48.177 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.177 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.178 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:48.178 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.000173+0000", 2020-08-07T15:13:48.178 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019020300000000001 2020-08-07T15:13:48.179 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:48.179 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:48.179 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:48.179 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.180 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.180 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:143436 6.2 6:45e453b6:::benchmark_data_smithi074_63843_object10184:head [delete] snapc 0=[] ondisk+write+known_if_redirected e297)", 2020-08-07T15:13:48.180 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:48.016603+0000", 2020-08-07T15:13:48.180 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.00099076000000000003, 2020-08-07T15:13:48.181 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0019216999999999999, 2020-08-07T15:13:48.181 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:48.182 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:48.182 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:48.182 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:48.184 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:48.184 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 143436 2020-08-07T15:13:48.185 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.185 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:48.185 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.185 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:48.186 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.016603+0000", 2020-08-07T15:13:48.186 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:48.186 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.186 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.186 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:48.187 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.016603+0000", 2020-08-07T15:13:48.187 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2598000000000002e-05 2020-08-07T15:13:48.187 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.187 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.187 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:48.188 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.016636+0000", 2020-08-07T15:13:48.188 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6829999999999996e-06 2020-08-07T15:13:48.188 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.188 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.188 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:48.188 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.016641+0000", 2020-08-07T15:13:48.189 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2379e-05 2020-08-07T15:13:48.189 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.189 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.189 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:48.189 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.016654+0000", 2020-08-07T15:13:48.190 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7187999999999999e-05 2020-08-07T15:13:48.190 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.190 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.190 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:48.191 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.016671+0000", 2020-08-07T15:13:48.191 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1715000000000002e-05 2020-08-07T15:13:48.191 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.191 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.191 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:48.192 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.016722+0000", 2020-08-07T15:13:48.192 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000175713 2020-08-07T15:13:48.192 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.192 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.192 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:48.193 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.016898+0000", 2020-08-07T15:13:48.193 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016003 2020-08-07T15:13:48.193 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.193 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.193 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:48.194 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.017058+0000", 2020-08-07T15:13:48.194 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001220608 2020-08-07T15:13:48.194 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.194 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.194 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:48.195 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.018279+0000", 2020-08-07T15:13:48.195 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5199999999999996e-07 2020-08-07T15:13:48.195 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.196 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.196 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:48.196 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.018280+0000", 2020-08-07T15:13:48.196 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5522000000000001e-05 2020-08-07T15:13:48.196 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.197 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.197 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:48.197 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.018335+0000", 2020-08-07T15:13:48.197 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001732188 2020-08-07T15:13:48.197 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:48.198 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:48.198 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:48.198 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.198 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.198 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5481.0:143423 6.7 e297/230 6:e0b019d2:::benchmark_data_smithi074_63843_object10171:head v 297'9163, mlcod=297'9158)", 2020-08-07T15:13:48.199 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:48.011076+0000", 2020-08-07T15:13:48.199 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0065172429999999998, 2020-08-07T15:13:48.199 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0075530500000000004, 2020-08-07T15:13:48.199 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:48.199 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:48.200 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:48.200 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.200 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:48.200 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.011076+0000", 2020-08-07T15:13:48.200 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:48.201 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.201 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.201 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:48.201 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.011076+0000", 2020-08-07T15:13:48.201 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8099000000000001e-05 2020-08-07T15:13:48.202 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.202 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.202 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:48.203 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.011125+0000", 2020-08-07T15:13:48.203 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8689999999999998e-06 2020-08-07T15:13:48.203 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.204 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.204 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:48.204 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.011131+0000", 2020-08-07T15:13:48.205 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0066370350000000003 2020-08-07T15:13:48.205 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.206 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.206 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:48.206 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.017768+0000", 2020-08-07T15:13:48.206 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2379999999999997e-06 2020-08-07T15:13:48.206 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.207 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.207 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:48.207 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.017774+0000", 2020-08-07T15:13:48.207 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8804999999999997e-05 2020-08-07T15:13:48.207 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.208 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.208 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:48.208 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.017832+0000", 2020-08-07T15:13:48.208 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3781000000000002e-05 2020-08-07T15:13:48.208 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.208 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.209 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:48.209 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.017856+0000", 2020-08-07T15:13:48.209 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0067798270000000004 2020-08-07T15:13:48.209 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:48.211 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:48.211 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:48.211 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.212 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.212 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:143407 6.c e297/230)", 2020-08-07T15:13:48.212 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:48.010452+0000", 2020-08-07T15:13:48.212 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0071411909999999999, 2020-08-07T15:13:48.212 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0082482000000000007, 2020-08-07T15:13:48.213 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:48.213 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:13:48.213 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:48.213 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.213 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:48.214 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.010452+0000", 2020-08-07T15:13:48.214 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:48.214 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.214 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.214 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:48.215 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.010452+0000", 2020-08-07T15:13:48.215 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8739999999999998e-05 2020-08-07T15:13:48.215 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.215 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.215 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:48.215 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.010491+0000", 2020-08-07T15:13:48.216 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5399999999999998e-06 2020-08-07T15:13:48.217 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.217 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.217 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:48.217 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.010499+0000", 2020-08-07T15:13:48.218 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074303119999999997 2020-08-07T15:13:48.218 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.219 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.219 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:48.219 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.017929+0000", 2020-08-07T15:13:48.220 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6149999999999999e-06 2020-08-07T15:13:48.220 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.220 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.220 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:48.220 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.017934+0000", 2020-08-07T15:13:48.221 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00076296999999999997 2020-08-07T15:13:48.221 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:48.221 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:48.221 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:48.221 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:48.018697+0000", 2020-08-07T15:13:48.222 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0082441770000000001 2020-08-07T15:13:48.222 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:48.222 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:48.222 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:48.222 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:48.223 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:13:48.223 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 10 2020-08-07T15:13:48.223 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:48.225 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:13:48.225 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:48.223+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:48.225 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:13:48.235 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:13:48.289 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:13:48.329 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:48.324+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:48.343 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:13:48.394 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:13:48.429 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:48.426+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:48.439 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:13:48.454 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:48.454 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:48.469 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:13:48.487 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:13:48.514 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:13:48.529 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:48.526+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:48.534 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:13:48.542 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:13:48.718 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:13:48.718 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:13:48.718 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:13:48.718 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:13:48.719 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:13:48.719 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:13:48.719 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:13:48.725 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:48.720+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:48.750 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:48.750 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:48.770 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:13:48.842 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:48.838+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:48.931 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:48.929+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:48.938 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:48.939 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:48.947 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:13:49.043 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:49.041+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:49.130 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:49.130 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:49.140 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:49.138+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:49.151 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:13:49.306 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:49.245+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:49.347 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:49.344+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:49.360 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:49.361 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:49.374 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:13:49.448 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:49.446+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:49.520 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:13:49.573 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:49.574 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:49.578 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:49.575+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:49.588 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:13:49.675 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:49.672+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:49.762 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:49.759+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:49.813 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:49.814 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:49.826 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:13:49.866 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:49.861+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:49.970 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:49.964+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:50.051 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:50.052 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:50.063 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:13:50.068 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:50.066+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:50.169 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:50.168+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:50.271 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:50.269+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:50.378 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:50.378 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:50.379 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:50.373+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:50.393 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:13:50.481 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:50.479+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:50.579 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:50.577+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:50.621 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:50.621 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:50.635 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:13:50.679 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:50.677+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:50.726 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:13:50.798 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:50.794+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:50.874 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:50.875 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:50.887 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:13:50.888 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:50.886+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:50.992 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:50.988+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:51.085 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:51.085 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:51.090 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:51.088+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:51.100 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:13:51.200 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:51.191+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:51.233 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:13:51.274 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:13:51.278 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:51.278 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:51.287 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:13:51.298 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:13:51.326 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:13:51.352 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:13:51.353 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:51.341+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:51.374 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:13:51.398 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:13:51.399 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:51.397+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:51.423 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:13:51.424 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:13:51.424 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:13:51.424 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:13:51.424 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:13:51.424 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:13:51.425 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:13:51.425 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:13:51.425 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:13:51.474 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:13:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:13:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:13:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15576 3.e 3:727724a5:::benchmark_data_smithi074_35447_object1946:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:13:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:13:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 424.80554082200001, 2020-08-07T15:13:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999, 2020-08-07T15:13:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:13:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:13:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15576 2020-08-07T15:13:51.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.503 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:51.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:51.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:13:51.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:51.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:51.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:13:51.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7156e-05 2020-08-07T15:13:51.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:51.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666682+0000", 2020-08-07T15:13:51.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0209999999999999e-06 2020-08-07T15:13:51.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:51.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666687+0000", 2020-08-07T15:13:51.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0730999999999999e-05 2020-08-07T15:13:51.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:51.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666698+0000", 2020-08-07T15:13:51.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2020-08-07T15:13:51.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:51.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666702+0000", 2020-08-07T15:13:51.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023978459999999999 2020-08-07T15:13:51.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:51.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669100+0000", 2020-08-07T15:13:51.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9664000000000001e-05 2020-08-07T15:13:51.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:51.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669190+0000", 2020-08-07T15:13:51.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015957100000000001 2020-08-07T15:13:51.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:13:51.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669350+0000", 2020-08-07T15:13:51.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016470149999999999 2020-08-07T15:13:51.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:51.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:13:51.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1299999999999999e-07 2020-08-07T15:13:51.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:51.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:13:51.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5269000000000002e-05 2020-08-07T15:13:51.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:51.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.671022+0000", 2020-08-07T15:13:51.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031414400000000001 2020-08-07T15:13:51.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:51.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.674163+0000", 2020-08-07T15:13:51.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45641417400000001 2020-08-07T15:13:51.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:51.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:13:51.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0399999999999995e-07 2020-08-07T15:13:51.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:51.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:13:51.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3343999999999997e-05 2020-08-07T15:13:51.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:51.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130622+0000", 2020-08-07T15:13:51.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999 2020-08-07T15:13:51.520 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.520 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:51.520 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.521 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106248 3.f 3:f950fd9d:::benchmark_data_smithi074_35447_object13280:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:13:51.521 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:13:51.521 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 364.11201128200003, 2020-08-07T15:13:51.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002, 2020-08-07T15:13:51.522 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:51.522 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:51.522 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:51.522 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:13:51.522 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:13:51.523 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106248 2020-08-07T15:13:51.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.523 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:51.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:51.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:13:51.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:51.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:51.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:13:51.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7184e-05 2020-08-07T15:13:51.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:51.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360222+0000", 2020-08-07T15:13:51.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0906e-05 2020-08-07T15:13:51.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:51.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360233+0000", 2020-08-07T15:13:51.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2972999999999999e-05 2020-08-07T15:13:51.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360246+0000", 2020-08-07T15:13:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0114e-05 2020-08-07T15:13:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:51.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360256+0000", 2020-08-07T15:13:51.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1303999999999997e-05 2020-08-07T15:13:51.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:51.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360287+0000", 2020-08-07T15:13:51.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3798000000000001e-05 2020-08-07T15:13:51.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:51.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360351+0000", 2020-08-07T15:13:51.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111308 2020-08-07T15:13:51.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:13:51.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360462+0000", 2020-08-07T15:13:51.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003480674 2020-08-07T15:13:51.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:51.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:13:51.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0900000000000001e-07 2020-08-07T15:13:51.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:51.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:13:51.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8899999999999994e-05 2020-08-07T15:13:51.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:51.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.364012+0000", 2020-08-07T15:13:51.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35291637199999998 2020-08-07T15:13:51.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:51.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716929+0000", 2020-08-07T15:13:51.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.076244233999999994 2020-08-07T15:13:51.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:51.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793173+0000", 2020-08-07T15:13:51.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6109999999999999e-06 2020-08-07T15:13:51.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:51.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793175+0000", 2020-08-07T15:13:51.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7492999999999999e-05 2020-08-07T15:13:51.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:51.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793242+0000", 2020-08-07T15:13:51.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002 2020-08-07T15:13:51.550 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.550 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:51.551 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.551 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106248 3.f e59/20 ondisk, result = 0)", 2020-08-07T15:13:51.551 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:13:51.552 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 364.10674799700001, 2020-08-07T15:13:51.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001, 2020-08-07T15:13:51.552 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:51.552 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:51.552 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:51.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:51.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:13:51.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:51.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:51.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:13:51.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7518999999999999e-05 2020-08-07T15:13:51.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:51.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365445+0000", 2020-08-07T15:13:51.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-06 2020-08-07T15:13:51.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:51.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365449+0000", 2020-08-07T15:13:51.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42764147600000002 2020-08-07T15:13:51.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:51.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793091+0000", 2020-08-07T15:13:51.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6500000000000002e-06 2020-08-07T15:13:51.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:51.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793099+0000", 2020-08-07T15:13:51.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3664e-05 2020-08-07T15:13:51.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:51.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793153+0000", 2020-08-07T15:13:51.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1078e-05 2020-08-07T15:13:51.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:51.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793164+0000", 2020-08-07T15:13:51.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3404000000000003e-05 2020-08-07T15:13:51.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:51.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793248+0000", 2020-08-07T15:13:51.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001 2020-08-07T15:13:51.566 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.566 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:51.566 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.567 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:107952 3.d e61/35)", 2020-08-07T15:13:51.567 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:13:51.567 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 361.14581828899998, 2020-08-07T15:13:51.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001, 2020-08-07T15:13:51.568 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:51.568 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:51.568 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:51.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:51.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:13:51.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:51.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:51.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:13:51.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8826000000000003e-05 2020-08-07T15:13:51.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:51.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326406+0000", 2020-08-07T15:13:51.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9199999999999998e-06 2020-08-07T15:13:51.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:51.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326413+0000", 2020-08-07T15:13:51.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75082853500000002 2020-08-07T15:13:51.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:51.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077242+0000", 2020-08-07T15:13:51.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1479999999999999e-06 2020-08-07T15:13:51.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:51.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077246+0000", 2020-08-07T15:13:51.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3331999999999998e-05 2020-08-07T15:13:51.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:51.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077289+0000", 2020-08-07T15:13:51.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8977000000000001e-05 2020-08-07T15:13:51.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:51.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077308+0000", 2020-08-07T15:13:51.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001 2020-08-07T15:13:51.576 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.577 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:51.577 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.578 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31696 4.0 4:0335eed9:::benchmark_data_smithi074_44586_object3961:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e100)", 2020-08-07T15:13:51.578 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:13:51.578 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 303.36791073000001, 2020-08-07T15:13:51.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237, 2020-08-07T15:13:51.578 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:51.579 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:51.579 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:51.579 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:51.579 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:51.579 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31696 2020-08-07T15:13:51.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.580 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:51.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:51.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:13:51.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:51.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:51.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:13:51.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7654e-05 2020-08-07T15:13:51.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:51.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104293+0000", 2020-08-07T15:13:51.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.061e-06 2020-08-07T15:13:51.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:51.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104296+0000", 2020-08-07T15:13:51.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0859999999999998e-06 2020-08-07T15:13:51.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:51.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104302+0000", 2020-08-07T15:13:51.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8030000000000001e-06 2020-08-07T15:13:51.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:51.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104305+0000", 2020-08-07T15:13:51.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037634769999999999 2020-08-07T15:13:51.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:51.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108068+0000", 2020-08-07T15:13:51.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8814999999999998e-05 2020-08-07T15:13:51.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:51.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108137+0000", 2020-08-07T15:13:51.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111455 2020-08-07T15:13:51.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:13:51.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108248+0000", 2020-08-07T15:13:51.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028756820000000001 2020-08-07T15:13:51.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:51.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111124+0000", 2020-08-07T15:13:51.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061054499999999999 2020-08-07T15:13:51.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:51.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111735+0000", 2020-08-07T15:13:51.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.423966167 2020-08-07T15:13:51.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:51.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535701+0000", 2020-08-07T15:13:51.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4929999999999999e-06 2020-08-07T15:13:51.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535702+0000", 2020-08-07T15:13:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2571999999999997e-05 2020-08-07T15:13:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535755+0000", 2020-08-07T15:13:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1427000000000003e-05 2020-08-07T15:13:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535786+0000", 2020-08-07T15:13:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237 2020-08-07T15:13:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:31696 4.0 e100/81 ondisk, result = 0)", 2020-08-07T15:13:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:13:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 303.358296172, 2020-08-07T15:13:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954, 2020-08-07T15:13:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:13:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:13:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1185000000000001e-05 2020-08-07T15:13:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113901+0000", 2020-08-07T15:13:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2930000000000002e-06 2020-08-07T15:13:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113905+0000", 2020-08-07T15:13:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42168640400000001 2020-08-07T15:13:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535591+0000", 2020-08-07T15:13:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2820000000000002e-06 2020-08-07T15:13:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535598+0000", 2020-08-07T15:13:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6578999999999999e-05 2020-08-07T15:13:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535684+0000", 2020-08-07T15:13:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8449999999999992e-06 2020-08-07T15:13:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535693+0000", 2020-08-07T15:13:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7366000000000003e-05 2020-08-07T15:13:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535791+0000", 2020-08-07T15:13:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954 2020-08-07T15:13:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:37024 4.0 e106/81)", 2020-08-07T15:13:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:13:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 297.24350727799998, 2020-08-07T15:13:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999, 2020-08-07T15:13:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:13:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:13:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4926000000000001e-05 2020-08-07T15:13:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:51.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228683+0000", 2020-08-07T15:13:51.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1280000000000002e-06 2020-08-07T15:13:51.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228686+0000", 2020-08-07T15:13:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69740012100000004 2020-08-07T15:13:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926086+0000", 2020-08-07T15:13:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000004e-06 2020-08-07T15:13:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926090+0000", 2020-08-07T15:13:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113526 2020-08-07T15:13:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926204+0000", 2020-08-07T15:13:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8824000000000001e-05 2020-08-07T15:13:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926222+0000", 2020-08-07T15:13:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999 2020-08-07T15:13:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42503 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2615, mlcod=112'2607)", 2020-08-07T15:13:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:13:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 291.17593536200002, 2020-08-07T15:13:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998, 2020-08-07T15:13:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:13:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:13:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270999999999998e-05 2020-08-07T15:13:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296292+0000", 2020-08-07T15:13:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4729999999999997e-06 2020-08-07T15:13:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296298+0000", 2020-08-07T15:13:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.448795049 2020-08-07T15:13:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745093+0000", 2020-08-07T15:13:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0140000000000001e-06 2020-08-07T15:13:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745100+0000", 2020-08-07T15:13:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2596999999999998e-05 2020-08-07T15:13:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745153+0000", 2020-08-07T15:13:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1662000000000003e-05 2020-08-07T15:13:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745184+0000", 2020-08-07T15:13:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093174800000000004 2020-08-07T15:13:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746116+0000", 2020-08-07T15:13:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0926999999999994e-05 2020-08-07T15:13:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746187+0000", 2020-08-07T15:13:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998 2020-08-07T15:13:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42504 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2616, mlcod=113'2609)", 2020-08-07T15:13:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:13:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 291.17259846899998, 2020-08-07T15:13:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998, 2020-08-07T15:13:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:13:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:13:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2154000000000002e-05 2020-08-07T15:13:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299629+0000", 2020-08-07T15:13:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1929999999999998e-06 2020-08-07T15:13:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299636+0000", 2020-08-07T15:13:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44766315400000001 2020-08-07T15:13:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747299+0000", 2020-08-07T15:13:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5889999999999999e-06 2020-08-07T15:13:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747305+0000", 2020-08-07T15:13:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3094999999999999e-05 2020-08-07T15:13:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747358+0000", 2020-08-07T15:13:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3283e-05 2020-08-07T15:13:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747392+0000", 2020-08-07T15:13:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015319509999999999 2020-08-07T15:13:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748924+0000", 2020-08-07T15:13:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5235999999999996e-05 2020-08-07T15:13:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748999+0000", 2020-08-07T15:13:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998 2020-08-07T15:13:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42520 4.8 e113/81 4:13071de5:::benchmark_data_smithi074_44586_object5314:head v 113'2624, mlcod=113'2618)", 2020-08-07T15:13:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:13:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 290.70665425999999, 2020-08-07T15:13:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001, 2020-08-07T15:13:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:13:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:13:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3967999999999995e-05 2020-08-07T15:13:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765585+0000", 2020-08-07T15:13:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0949999999999998e-06 2020-08-07T15:13:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765593+0000", 2020-08-07T15:13:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52703464200000005 2020-08-07T15:13:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292627+0000", 2020-08-07T15:13:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6929999999999996e-06 2020-08-07T15:13:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292636+0000", 2020-08-07T15:13:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9292000000000001e-05 2020-08-07T15:13:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292695+0000", 2020-08-07T15:13:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5395999999999999e-05 2020-08-07T15:13:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292721+0000", 2020-08-07T15:13:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00090361399999999996 2020-08-07T15:13:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293624+0000", 2020-08-07T15:13:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1761000000000002e-05 2020-08-07T15:13:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293666+0000", 2020-08-07T15:13:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001 2020-08-07T15:13:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:19776 5.4 5:2bb0c0d9:::benchmark_data_smithi074_53697_object2471:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e173)", 2020-08-07T15:13:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:13:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 199.07246157500001, 2020-08-07T15:13:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.402061844, 2020-08-07T15:13:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 19776 2020-08-07T15:13:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:13:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399714+0000", 2020-08-07T15:13:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-05 2020-08-07T15:13:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399752+0000", 2020-08-07T15:13:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8799999999999999e-06 2020-08-07T15:13:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399757+0000", 2020-08-07T15:13:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7690000000000008e-06 2020-08-07T15:13:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399767+0000", 2020-08-07T15:13:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.799e-06 2020-08-07T15:13:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.399771+0000", 2020-08-07T15:13:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003255109 2020-08-07T15:13:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403026+0000", 2020-08-07T15:13:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101044 2020-08-07T15:13:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403127+0000", 2020-08-07T15:13:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000193158 2020-08-07T15:13:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,5", 2020-08-07T15:13:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.403320+0000", 2020-08-07T15:13:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017185530000000001 2020-08-07T15:13:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405038+0000", 2020-08-07T15:13:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6300000000000004e-07 2020-08-07T15:13:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405039+0000", 2020-08-07T15:13:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012590600000000001 2020-08-07T15:13:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405165+0000", 2020-08-07T15:13:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109757 2020-08-07T15:13:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.405275+0000", 2020-08-07T15:13:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39642174699999999 2020-08-07T15:13:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801697+0000", 2020-08-07T15:13:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2799999999999996e-07 2020-08-07T15:13:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801697+0000", 2020-08-07T15:13:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8740999999999997e-05 2020-08-07T15:13:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:32.801776+0000", 2020-08-07T15:13:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.402061844 2020-08-07T15:13:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:85552 5.9 5:9d69b7cc:::benchmark_data_smithi074_53697_object10693:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e202)", 2020-08-07T15:13:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:13:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 154.664658481, 2020-08-07T15:13:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54429374399999997, 2020-08-07T15:13:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 85552 2020-08-07T15:13:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:13:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:13:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2691000000000001e-05 2020-08-07T15:13:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807570+0000", 2020-08-07T15:13:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0619999999999997e-06 2020-08-07T15:13:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807576+0000", 2020-08-07T15:13:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1558e-05 2020-08-07T15:13:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807588+0000", 2020-08-07T15:13:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4600000000000002e-06 2020-08-07T15:13:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807593+0000", 2020-08-07T15:13:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022126279999999999 2020-08-07T15:13:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.809806+0000", 2020-08-07T15:13:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104868 2020-08-07T15:13:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.809911+0000", 2020-08-07T15:13:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000186019 2020-08-07T15:13:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:13:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.810097+0000", 2020-08-07T15:13:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037966739999999999 2020-08-07T15:13:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813893+0000", 2020-08-07T15:13:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1310000000000001e-06 2020-08-07T15:13:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813894+0000", 2020-08-07T15:13:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4704999999999999e-05 2020-08-07T15:13:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813949+0000", 2020-08-07T15:13:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015487280000000001 2020-08-07T15:13:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.815498+0000", 2020-08-07T15:13:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53621442900000005 2020-08-07T15:13:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351712+0000", 2020-08-07T15:13:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8199999999999998e-07 2020-08-07T15:13:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351713+0000", 2020-08-07T15:13:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7909000000000001e-05 2020-08-07T15:13:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351811+0000", 2020-08-07T15:13:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54429374399999997 2020-08-07T15:13:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:85552 5.9 e202/164 ondisk, result = 0)", 2020-08-07T15:13:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:13:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 154.65843207899999, 2020-08-07T15:13:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53813836800000003, 2020-08-07T15:13:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:13:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:13:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8833e-05 2020-08-07T15:13:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813772+0000", 2020-08-07T15:13:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8459999999999999e-06 2020-08-07T15:13:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813777+0000", 2020-08-07T15:13:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53732224900000003 2020-08-07T15:13:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351100+0000", 2020-08-07T15:13:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1869999999999996e-06 2020-08-07T15:13:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351105+0000", 2020-08-07T15:13:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00058858100000000002 2020-08-07T15:13:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351693+0000", 2020-08-07T15:13:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4229999999999994e-06 2020-08-07T15:13:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351703+0000", 2020-08-07T15:13:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000179249 2020-08-07T15:13:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351882+0000", 2020-08-07T15:13:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53813836800000003 2020-08-07T15:13:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:85552 5.9 5:9d69b7cc:::benchmark_data_smithi074_53697_object10693:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e202)", 2020-08-07T15:13:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:13:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 154.65368395499999, 2020-08-07T15:13:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53326610600000002, 2020-08-07T15:13:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:13:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 85552 2020-08-07T15:13:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:13:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:13:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5001e-05 2020-08-07T15:13:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818517+0000", 2020-08-07T15:13:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5000000000000002e-06 2020-08-07T15:13:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818519+0000", 2020-08-07T15:13:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0699999999999997e-06 2020-08-07T15:13:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818524+0000", 2020-08-07T15:13:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2910000000000002e-06 2020-08-07T15:13:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818527+0000", 2020-08-07T15:13:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030912499999999999 2020-08-07T15:13:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818836+0000", 2020-08-07T15:13:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2908000000000002e-05 2020-08-07T15:13:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for ondisk", 2020-08-07T15:13:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818889+0000", 2020-08-07T15:13:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53286921099999995 2020-08-07T15:13:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351758+0000", 2020-08-07T15:13:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53326610600000002 2020-08-07T15:13:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:89464 5.9 5:94c69ba5:::benchmark_data_smithi074_53697_object11182:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:13:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 150.055844001, 2020-08-07T15:13:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67692916199999997, 2020-08-07T15:13:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 89464 2020-08-07T15:13:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:13:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:13:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0074000000000002e-05 2020-08-07T15:13:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416382+0000", 2020-08-07T15:13:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5000000000000001e-06 2020-08-07T15:13:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416386+0000", 2020-08-07T15:13:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2447e-05 2020-08-07T15:13:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416399+0000", 2020-08-07T15:13:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1279999999999996e-06 2020-08-07T15:13:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416405+0000", 2020-08-07T15:13:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016641805999999999 2020-08-07T15:13:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433047+0000", 2020-08-07T15:13:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107995 2020-08-07T15:13:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433155+0000", 2020-08-07T15:13:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018818599999999999 2020-08-07T15:13:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:13:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433343+0000", 2020-08-07T15:13:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039911679999999998 2020-08-07T15:13:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437334+0000", 2020-08-07T15:13:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2020-08-07T15:13:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437335+0000", 2020-08-07T15:13:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6351000000000002e-05 2020-08-07T15:13:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437401+0000", 2020-08-07T15:13:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001267949 2020-08-07T15:13:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438669+0000", 2020-08-07T15:13:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65454035600000005 2020-08-07T15:13:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093209+0000", 2020-08-07T15:13:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7859999999999999e-06 2020-08-07T15:13:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093211+0000", 2020-08-07T15:13:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9665e-05 2020-08-07T15:13:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093261+0000", 2020-08-07T15:13:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67692916199999997 2020-08-07T15:13:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:89464 5.9 e207/164 ondisk, result = 0)", 2020-08-07T15:13:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:13:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 150.03321423899999, 2020-08-07T15:13:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65430647200000003, 2020-08-07T15:13:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:13:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:13:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6529000000000002e-05 2020-08-07T15:13:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438998+0000", 2020-08-07T15:13:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.567e-06 2020-08-07T15:13:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.439005+0000", 2020-08-07T15:13:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65408475099999996 2020-08-07T15:13:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:51.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093089+0000", 2020-08-07T15:13:51.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6880000000000002e-06 2020-08-07T15:13:51.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:51.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093096+0000", 2020-08-07T15:13:51.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4980000000000002e-05 2020-08-07T15:13:51.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:51.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093191+0000", 2020-08-07T15:13:51.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3090000000000001e-06 2020-08-07T15:13:51.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:51.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093200+0000", 2020-08-07T15:13:51.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7648e-05 2020-08-07T15:13:51.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:51.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093268+0000", 2020-08-07T15:13:51.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65430647200000003 2020-08-07T15:13:51.782 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.782 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:51.782 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.783 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:114080 5.4 5:24adcd54:::benchmark_data_smithi074_53697_object14259:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e215)", 2020-08-07T15:13:51.783 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:13:51.783 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 131.87250678199999, 2020-08-07T15:13:51.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.57290862300000001, 2020-08-07T15:13:51.783 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:51.784 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:51.784 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:51.784 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:51.784 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:51.785 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 114080 2020-08-07T15:13:51.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.785 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:51.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:51.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:13:51.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:51.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:51.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:13:51.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9238e-05 2020-08-07T15:13:51.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:51.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599698+0000", 2020-08-07T15:13:51.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6260000000000003e-06 2020-08-07T15:13:51.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:51.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599703+0000", 2020-08-07T15:13:51.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5577999999999999e-05 2020-08-07T15:13:51.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:51.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599718+0000", 2020-08-07T15:13:51.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9869999999999999e-06 2020-08-07T15:13:51.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:51.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599726+0000", 2020-08-07T15:13:51.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00077929599999999996 2020-08-07T15:13:51.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:51.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600506+0000", 2020-08-07T15:13:51.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1717000000000003e-05 2020-08-07T15:13:51.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600597+0000", 2020-08-07T15:13:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011828999999999999 2020-08-07T15:13:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,5", 2020-08-07T15:13:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600716+0000", 2020-08-07T15:13:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020861479999999999 2020-08-07T15:13:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:51.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602802+0000", 2020-08-07T15:13:51.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4600000000000005e-07 2020-08-07T15:13:51.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:51.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602802+0000", 2020-08-07T15:13:51.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9165e-05 2020-08-07T15:13:51.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602862+0000", 2020-08-07T15:13:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0093485080000000002 2020-08-07T15:13:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.612210+0000", 2020-08-07T15:13:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.56030197699999995 2020-08-07T15:13:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172512+0000", 2020-08-07T15:13:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8800000000000001e-07 2020-08-07T15:13:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172513+0000", 2020-08-07T15:13:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4659000000000006e-05 2020-08-07T15:13:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172578+0000", 2020-08-07T15:13:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.57290862300000001 2020-08-07T15:13:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:48616 6.9 6:955383e0:::benchmark_data_smithi074_63843_object6076:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e256)", 2020-08-07T15:13:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:43.702845+0000", 2020-08-07T15:13:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 67.769330484999998, 2020-08-07T15:13:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40912302499999997, 2020-08-07T15:13:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:13:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48616 2020-08-07T15:13:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:51.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702845+0000", 2020-08-07T15:13:51.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:51.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:51.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702845+0000", 2020-08-07T15:13:51.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1931999999999997e-05 2020-08-07T15:13:51.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:51.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702897+0000", 2020-08-07T15:13:51.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0650000000000004e-06 2020-08-07T15:13:51.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:51.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702903+0000", 2020-08-07T15:13:51.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3084999999999999e-05 2020-08-07T15:13:51.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:51.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702916+0000", 2020-08-07T15:13:51.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4179999999999996e-06 2020-08-07T15:13:51.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:51.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702922+0000", 2020-08-07T15:13:51.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045509080000000002 2020-08-07T15:13:51.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:51.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.707473+0000", 2020-08-07T15:13:51.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2619000000000005e-05 2020-08-07T15:13:51.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:51.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.707535+0000", 2020-08-07T15:13:51.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000126919 2020-08-07T15:13:51.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2", 2020-08-07T15:13:51.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.707662+0000", 2020-08-07T15:13:51.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001200135 2020-08-07T15:13:51.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:51.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.708862+0000", 2020-08-07T15:13:51.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0109999999999999e-06 2020-08-07T15:13:51.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:51.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.708863+0000", 2020-08-07T15:13:51.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8837999999999994e-05 2020-08-07T15:13:51.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:51.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.708932+0000", 2020-08-07T15:13:51.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054206710000000002 2020-08-07T15:13:51.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:51.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.714353+0000", 2020-08-07T15:13:51.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39754767099999999 2020-08-07T15:13:51.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:51.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111901+0000", 2020-08-07T15:13:51.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.023e-06 2020-08-07T15:13:51.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:51.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111902+0000", 2020-08-07T15:13:51.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6730000000000007e-05 2020-08-07T15:13:51.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:51.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111968+0000", 2020-08-07T15:13:51.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40912302499999997 2020-08-07T15:13:51.830 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.830 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:51.830 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.831 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5481.0:48616 6.9 e256/230 ondisk, result = 0)", 2020-08-07T15:13:51.831 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:43.714790+0000", 2020-08-07T15:13:51.831 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 67.757385920000004, 2020-08-07T15:13:51.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39718457899999998, 2020-08-07T15:13:51.831 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:51.832 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:51.832 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:51.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:51.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.714790+0000", 2020-08-07T15:13:51.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:51.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:51.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.714790+0000", 2020-08-07T15:13:51.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8272000000000001e-05 2020-08-07T15:13:51.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:51.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.714808+0000", 2020-08-07T15:13:51.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1599999999999998e-06 2020-08-07T15:13:51.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:51.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.714811+0000", 2020-08-07T15:13:51.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39639543300000002 2020-08-07T15:13:51.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:51.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111207+0000", 2020-08-07T15:13:51.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7680000000000002e-06 2020-08-07T15:13:51.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:51.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111210+0000", 2020-08-07T15:13:51.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067142299999999999 2020-08-07T15:13:51.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:51.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111882+0000", 2020-08-07T15:13:51.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6849999999999996e-06 2020-08-07T15:13:51.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:51.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111892+0000", 2020-08-07T15:13:51.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2837999999999995e-05 2020-08-07T15:13:51.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:51.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111974+0000", 2020-08-07T15:13:51.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39718457899999998 2020-08-07T15:13:51.841 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.841 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:51.841 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.842 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5481.0:128184 6.8 e293/269 6:13c658be:::benchmark_data_smithi074_63843_object16022:head v 293'8152, mlcod=293'8147)", 2020-08-07T15:13:51.842 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:13:38.394036+0000", 2020-08-07T15:13:51.842 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.078139619, 2020-08-07T15:13:51.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.58484205899999997, 2020-08-07T15:13:51.842 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:51.843 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:51.843 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:51.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:51.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.394036+0000", 2020-08-07T15:13:51.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:51.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:51.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.394036+0000", 2020-08-07T15:13:51.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3461999999999996e-05 2020-08-07T15:13:51.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:51.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.394100+0000", 2020-08-07T15:13:51.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9980000000000003e-06 2020-08-07T15:13:51.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:51.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.394108+0000", 2020-08-07T15:13:51.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.58342110700000005 2020-08-07T15:13:51.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:51.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.977529+0000", 2020-08-07T15:13:51.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2359e-05 2020-08-07T15:13:51.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:51.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.977541+0000", 2020-08-07T15:13:51.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4761999999999998e-05 2020-08-07T15:13:51.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:51.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.977636+0000", 2020-08-07T15:13:51.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4950999999999999e-05 2020-08-07T15:13:51.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:51.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.977661+0000", 2020-08-07T15:13:51.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001160204 2020-08-07T15:13:51.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:51.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.978821+0000", 2020-08-07T15:13:51.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7216000000000002e-05 2020-08-07T15:13:51.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:51.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:51.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:51.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.978878+0000", 2020-08-07T15:13:51.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.58484205899999997 2020-08-07T15:13:51.853 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.856 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:51.857 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.857 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:51.858 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:51.858 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:51.859 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:13:51.860 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:13:51.861 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:51.858+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:51.929 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Removed 16657 objects 2020-08-07T15:13:51.929 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Clean up completed and total clean up time :8.64297 2020-08-07T15:13:51.983 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:51.975+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:51.998 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:51.999 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:13:51.999 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:13:51.999 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:52.015 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:13:52.065 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:52.063+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:52.168 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:52.164+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:52.182 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:52.182 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:13:52.183 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:13:52.183 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:13:52.183 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:52.192 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:13:52.251 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:13:52.306 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:52.304+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:52.365 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:52.365 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:13:52.366 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:13:52.366 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:13:52.367 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.367 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "MOSDScrubReserve(2.6 GRANT e232)", 2020-08-07T15:13:52.367 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:11.909673+0000", 2020-08-07T15:13:52.368 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 100.451618716, 2020-08-07T15:13:52.368 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.36313456, 2020-08-07T15:13:52.368 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:52.369 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:52.369 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:52.369 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.370 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:52.370 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:11.909673+0000", 2020-08-07T15:13:52.370 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:52.371 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.371 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.371 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:52.372 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:11.909673+0000", 2020-08-07T15:13:52.372 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0409000000000001e-05 2020-08-07T15:13:52.372 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.373 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.374 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:52.374 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:11.909693+0000", 2020-08-07T15:13:52.375 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8580000000000002e-06 2020-08-07T15:13:52.375 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.375 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.376 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:52.376 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:11.909697+0000", 2020-08-07T15:13:52.376 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.36297494299999999 2020-08-07T15:13:52.377 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.377 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.377 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:52.377 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272672+0000", 2020-08-07T15:13:52.378 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4490000000000001e-06 2020-08-07T15:13:52.378 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.378 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.378 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:52.379 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272676+0000", 2020-08-07T15:13:52.379 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0395999999999994e-05 2020-08-07T15:13:52.379 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.380 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.380 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:52.380 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272757+0000", 2020-08-07T15:13:52.380 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6416e-05 2020-08-07T15:13:52.381 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.381 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.381 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:52.381 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272773+0000", 2020-08-07T15:13:52.382 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4088999999999998e-05 2020-08-07T15:13:52.382 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.382 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.383 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:52.383 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:12.272807+0000", 2020-08-07T15:13:52.395 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.36313456 2020-08-07T15:13:52.395 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.396 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:52.396 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.397 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.397 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.397 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:7272 6.e 6:74263cb8:::benchmark_data_smithi074_63843_object908:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e234)", 2020-08-07T15:13:52.398 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:16.977784+0000", 2020-08-07T15:13:52.398 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 95.383506968000006, 2020-08-07T15:13:52.398 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31581518600000003, 2020-08-07T15:13:52.399 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:52.399 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:52.399 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:52.400 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:52.400 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:52.401 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 7272 2020-08-07T15:13:52.401 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.401 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:52.402 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.402 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:52.402 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977784+0000", 2020-08-07T15:13:52.403 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:52.403 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.403 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.404 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:52.404 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977784+0000", 2020-08-07T15:13:52.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6940999999999998e-05 2020-08-07T15:13:52.405 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.416 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.416 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:52.417 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977811+0000", 2020-08-07T15:13:52.417 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1870000000000001e-06 2020-08-07T15:13:52.417 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.417 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.418 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:52.418 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977815+0000", 2020-08-07T15:13:52.418 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1249999999999998e-06 2020-08-07T15:13:52.418 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.418 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.419 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:52.420 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977821+0000", 2020-08-07T15:13:52.421 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8739999999999999e-06 2020-08-07T15:13:52.421 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.421 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.422 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:52.422 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.977824+0000", 2020-08-07T15:13:52.422 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001740655 2020-08-07T15:13:52.422 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.423 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:52.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.979564+0000", 2020-08-07T15:13:52.423 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6272000000000003e-05 2020-08-07T15:13:52.423 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.423 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.424 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:52.424 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.979650+0000", 2020-08-07T15:13:52.424 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012951600000000001 2020-08-07T15:13:52.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.424 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.424 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:52.425 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.979780+0000", 2020-08-07T15:13:52.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00085408999999999995 2020-08-07T15:13:52.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.425 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.425 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:52.426 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.980634+0000", 2020-08-07T15:13:52.426 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8099999999999999e-07 2020-08-07T15:13:52.438 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.438 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.438 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:52.439 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.980634+0000", 2020-08-07T15:13:52.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5664000000000003e-05 2020-08-07T15:13:52.439 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.440 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:52.440 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:16.980670+0000", 2020-08-07T15:13:52.441 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.104650663 2020-08-07T15:13:52.441 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.441 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.442 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:52.442 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.085321+0000", 2020-08-07T15:13:52.443 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.20821616000000001 2020-08-07T15:13:52.443 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.443 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.444 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:52.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.293537+0000", 2020-08-07T15:13:52.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1900000000000001e-07 2020-08-07T15:13:52.451 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.451 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.451 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:52.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.293538+0000", 2020-08-07T15:13:52.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1939000000000004e-05 2020-08-07T15:13:52.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:52.453 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.293600+0000", 2020-08-07T15:13:52.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31581518600000003 2020-08-07T15:13:52.453 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.453 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:52.453 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.454 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.454 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.454 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:8359 6.1 6:844c6d67:::benchmark_data_smithi074_63843_object1044:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e235)", 2020-08-07T15:13:52.454 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:17.720675+0000", 2020-08-07T15:13:52.454 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 94.640616811000001, 2020-08-07T15:13:52.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.309231859, 2020-08-07T15:13:52.455 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:52.455 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:52.455 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:52.455 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:52.455 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:52.456 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 8359 2020-08-07T15:13:52.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.456 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:52.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:52.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720675+0000", 2020-08-07T15:13:52.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:52.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:52.458 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720675+0000", 2020-08-07T15:13:52.458 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.884e-05 2020-08-07T15:13:52.458 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.458 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.458 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:52.459 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720723+0000", 2020-08-07T15:13:52.459 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2989999999999996e-06 2020-08-07T15:13:52.459 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:52.459 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720729+0000", 2020-08-07T15:13:52.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0288999999999999e-05 2020-08-07T15:13:52.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.460 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.460 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:52.472 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720739+0000", 2020-08-07T15:13:52.472 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7849999999999999e-06 2020-08-07T15:13:52.473 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.473 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.473 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:52.478 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.720744+0000", 2020-08-07T15:13:52.478 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001862991 2020-08-07T15:13:52.479 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.479 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.479 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:52.479 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.722607+0000", 2020-08-07T15:13:52.480 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000100839 2020-08-07T15:13:52.481 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.481 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.481 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:52.502 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.722708+0000", 2020-08-07T15:13:52.502 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017526600000000001 2020-08-07T15:13:52.503 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.511 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:13:52.511 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.722883+0000", 2020-08-07T15:13:52.511 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001504619 2020-08-07T15:13:52.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.512 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.513 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:52.513 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.724387+0000", 2020-08-07T15:13:52.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4499999999999996e-07 2020-08-07T15:13:52.530 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.530 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.530 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:52.530 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.724388+0000", 2020-08-07T15:13:52.531 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8369000000000003e-05 2020-08-07T15:13:52.531 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.531 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.531 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:52.531 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.724447+0000", 2020-08-07T15:13:52.532 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00097551199999999999 2020-08-07T15:13:52.532 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.532 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.532 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:52.532 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.725422+0000", 2020-08-07T15:13:52.533 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30441636900000002 2020-08-07T15:13:52.533 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.533 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.533 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:52.533 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029838+0000", 2020-08-07T15:13:52.553 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7699999999999992e-07 2020-08-07T15:13:52.553 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.572 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.572 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:52.573 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029839+0000", 2020-08-07T15:13:52.573 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6958999999999994e-05 2020-08-07T15:13:52.573 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.573 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.573 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:52.574 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029906+0000", 2020-08-07T15:13:52.574 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.309231859 2020-08-07T15:13:52.574 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.574 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:52.574 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.575 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.575 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:8360 6.1 6:844c6d67:::benchmark_data_smithi074_63843_object1044:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e235)", 2020-08-07T15:13:52.575 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:17.721396+0000", 2020-08-07T15:13:52.575 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 94.639895147999994, 2020-08-07T15:13:52.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31055011399999999, 2020-08-07T15:13:52.576 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:52.576 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:52.576 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:52.576 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:52.577 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:52.577 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 8360 2020-08-07T15:13:52.577 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.577 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:52.577 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.611 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:52.611 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721396+0000", 2020-08-07T15:13:52.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:52.612 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.612 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.612 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:52.612 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721396+0000", 2020-08-07T15:13:52.613 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8222e-05 2020-08-07T15:13:52.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.613 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:52.630 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721444+0000", 2020-08-07T15:13:52.631 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.297e-06 2020-08-07T15:13:52.631 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.631 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.631 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:52.632 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721451+0000", 2020-08-07T15:13:52.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.172e-05 2020-08-07T15:13:52.632 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.632 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.632 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:52.633 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721462+0000", 2020-08-07T15:13:52.633 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7639999999999996e-06 2020-08-07T15:13:52.633 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.633 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.633 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:52.634 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.721467+0000", 2020-08-07T15:13:52.634 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023051790000000001 2020-08-07T15:13:52.634 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.634 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.634 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:52.634 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.723772+0000", 2020-08-07T15:13:52.635 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1051000000000006e-05 2020-08-07T15:13:52.635 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.635 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.635 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:52.635 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.723863+0000", 2020-08-07T15:13:52.636 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017915699999999999 2020-08-07T15:13:52.636 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.636 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.636 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:13:52.636 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.724043+0000", 2020-08-07T15:13:52.637 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001191646 2020-08-07T15:13:52.637 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.637 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.637 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:52.637 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.725234+0000", 2020-08-07T15:13:52.638 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9100000000000002e-07 2020-08-07T15:13:52.638 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.638 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.638 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:52.639 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.725235+0000", 2020-08-07T15:13:52.640 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0677999999999997e-05 2020-08-07T15:13:52.640 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.641 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.641 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:52.641 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.725296+0000", 2020-08-07T15:13:52.641 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001252663 2020-08-07T15:13:52.641 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.642 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.642 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:52.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.726548+0000", 2020-08-07T15:13:52.642 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30531503100000001 2020-08-07T15:13:52.642 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.643 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.643 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:52.643 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.031864+0000", 2020-08-07T15:13:52.643 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.057e-06 2020-08-07T15:13:52.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.644 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.644 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:52.644 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.031865+0000", 2020-08-07T15:13:52.644 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1757999999999999e-05 2020-08-07T15:13:52.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:52.645 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.031946+0000", 2020-08-07T15:13:52.645 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31055011399999999 2020-08-07T15:13:52.645 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.646 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:52.646 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.646 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.646 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.646 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:8359 6.1 e235/230 ondisk, result = 0)", 2020-08-07T15:13:52.647 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:17.726764+0000", 2020-08-07T15:13:52.647 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 94.634527132000002, 2020-08-07T15:13:52.647 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30314883199999998, 2020-08-07T15:13:52.647 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:52.647 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:52.648 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:52.648 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.648 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:52.648 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.726764+0000", 2020-08-07T15:13:52.648 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:52.649 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.649 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.649 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:52.663 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.726764+0000", 2020-08-07T15:13:52.664 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1386000000000001e-05 2020-08-07T15:13:52.664 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.665 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:52.665 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.726796+0000", 2020-08-07T15:13:52.665 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7989999999999999e-06 2020-08-07T15:13:52.665 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.665 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.666 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:52.666 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:17.726801+0000", 2020-08-07T15:13:52.666 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30298438700000002 2020-08-07T15:13:52.666 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.666 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.667 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:52.667 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029786+0000", 2020-08-07T15:13:52.667 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5040000000000002e-06 2020-08-07T15:13:52.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.668 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:52.668 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029791+0000", 2020-08-07T15:13:52.668 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0737000000000001e-05 2020-08-07T15:13:52.668 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.668 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.669 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:52.669 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029822+0000", 2020-08-07T15:13:52.669 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9359999999999999e-06 2020-08-07T15:13:52.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.669 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.672 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:52.672 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029830+0000", 2020-08-07T15:13:52.672 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3083000000000004e-05 2020-08-07T15:13:52.673 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.673 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.673 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:52.674 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:18.029913+0000", 2020-08-07T15:13:52.674 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30314883199999998 2020-08-07T15:13:52.674 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.674 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:52.675 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.675 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.675 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.675 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:10224 6.e 6:77f719e6:::benchmark_data_smithi074_63843_object1277:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e236)", 2020-08-07T15:13:52.676 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:19.038704+0000", 2020-08-07T15:13:52.676 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 93.322587830000003, 2020-08-07T15:13:52.676 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32955690900000001, 2020-08-07T15:13:52.677 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:52.677 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:52.677 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:52.677 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:52.678 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:52.678 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 10224 2020-08-07T15:13:52.678 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.678 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:52.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.679 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:52.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038704+0000", 2020-08-07T15:13:52.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:52.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.680 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:52.681 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038704+0000", 2020-08-07T15:13:52.681 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6672000000000001e-05 2020-08-07T15:13:52.681 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.682 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.682 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:52.682 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038730+0000", 2020-08-07T15:13:52.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.27e-06 2020-08-07T15:13:52.683 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.684 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.687 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:52.687 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038733+0000", 2020-08-07T15:13:52.688 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4030000000000004e-06 2020-08-07T15:13:52.688 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.688 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.689 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:52.689 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038740+0000", 2020-08-07T15:13:52.690 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.712e-06 2020-08-07T15:13:52.690 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.691 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.691 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:52.691 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.038743+0000", 2020-08-07T15:13:52.692 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0025767279999999999 2020-08-07T15:13:52.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.698 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:52.699 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.041319+0000", 2020-08-07T15:13:52.699 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4727e-05 2020-08-07T15:13:52.699 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.700 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:52.700 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.041374+0000", 2020-08-07T15:13:52.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9403999999999999e-05 2020-08-07T15:13:52.701 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.701 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:52.702 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.041473+0000", 2020-08-07T15:13:52.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010041290000000001 2020-08-07T15:13:52.702 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.702 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.703 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:52.703 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.042478+0000", 2020-08-07T15:13:52.703 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0100000000000003e-07 2020-08-07T15:13:52.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.704 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:52.704 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.042478+0000", 2020-08-07T15:13:52.704 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7923999999999995e-05 2020-08-07T15:13:52.705 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.705 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.705 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:52.706 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.042546+0000", 2020-08-07T15:13:52.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0046169310000000003 2020-08-07T15:13:52.708 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.709 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.709 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:52.709 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.047163+0000", 2020-08-07T15:13:52.709 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.321023999 2020-08-07T15:13:52.710 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.710 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.710 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:52.710 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368187+0000", 2020-08-07T15:13:52.710 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.048e-06 2020-08-07T15:13:52.711 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.711 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:52.712 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368188+0000", 2020-08-07T15:13:52.712 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2061e-05 2020-08-07T15:13:52.712 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.712 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.713 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:52.713 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368260+0000", 2020-08-07T15:13:52.713 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32955690900000001 2020-08-07T15:13:52.714 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.724 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:52.729 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.729 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.729 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.729 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:10224 6.e e236/230 ondisk, result = 0)", 2020-08-07T15:13:52.730 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:19.047929+0000", 2020-08-07T15:13:52.730 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 93.313362229999996, 2020-08-07T15:13:52.730 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32041415400000001, 2020-08-07T15:13:52.730 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:52.730 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:52.731 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:52.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.731 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:52.732 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.047929+0000", 2020-08-07T15:13:52.733 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:52.733 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.733 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:52.733 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.047929+0000", 2020-08-07T15:13:52.734 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9706999999999999e-05 2020-08-07T15:13:52.734 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.734 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.734 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:52.734 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.047959+0000", 2020-08-07T15:13:52.735 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9100000000000004e-06 2020-08-07T15:13:52.735 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.735 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.735 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:52.735 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.047964+0000", 2020-08-07T15:13:52.736 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31873185399999998 2020-08-07T15:13:52.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.736 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.736 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:52.736 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.366696+0000", 2020-08-07T15:13:52.737 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2970000000000003e-06 2020-08-07T15:13:52.737 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.737 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.737 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:52.737 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.366701+0000", 2020-08-07T15:13:52.738 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001472522 2020-08-07T15:13:52.738 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.738 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.738 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:52.738 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368173+0000", 2020-08-07T15:13:52.739 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5949999999999998e-06 2020-08-07T15:13:52.739 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.739 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.739 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:52.739 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368179+0000", 2020-08-07T15:13:52.740 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016426900000000001 2020-08-07T15:13:52.740 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.740 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.740 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:52.740 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:19.368343+0000", 2020-08-07T15:13:52.741 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32041415400000001 2020-08-07T15:13:52.741 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.741 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:52.741 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.742 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.742 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:18480 6.1 6:8a2c1569:::benchmark_data_smithi074_63843_object2309:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e240)", 2020-08-07T15:13:52.742 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.367850+0000", 2020-08-07T15:13:52.742 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 87.993441820000001, 2020-08-07T15:13:52.742 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41222172800000001, 2020-08-07T15:13:52.743 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:52.743 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:52.743 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:52.743 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:52.743 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:52.744 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18480 2020-08-07T15:13:52.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.744 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:52.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:52.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367850+0000", 2020-08-07T15:13:52.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:52.745 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.745 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:52.746 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367850+0000", 2020-08-07T15:13:52.746 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6174999999999999e-05 2020-08-07T15:13:52.746 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.746 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:52.747 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367876+0000", 2020-08-07T15:13:52.747 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.061e-06 2020-08-07T15:13:52.747 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.747 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.747 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:52.748 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367879+0000", 2020-08-07T15:13:52.748 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7899999999999996e-06 2020-08-07T15:13:52.748 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.748 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.748 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:52.749 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367885+0000", 2020-08-07T15:13:52.749 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6589999999999999e-06 2020-08-07T15:13:52.749 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.749 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.749 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:52.750 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.367887+0000", 2020-08-07T15:13:52.750 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001651275 2020-08-07T15:13:52.750 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.750 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.750 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:52.750 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.369538+0000", 2020-08-07T15:13:52.751 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7794000000000006e-05 2020-08-07T15:13:52.751 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.751 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.751 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:52.751 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.369636+0000", 2020-08-07T15:13:52.752 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017274 2020-08-07T15:13:52.752 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.752 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:13:52.752 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.369809+0000", 2020-08-07T15:13:52.753 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010733769999999999 2020-08-07T15:13:52.753 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.753 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.753 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:52.753 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.370882+0000", 2020-08-07T15:13:52.754 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0100000000000003e-07 2020-08-07T15:13:52.754 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.754 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:52.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.370883+0000", 2020-08-07T15:13:52.755 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6001000000000001e-05 2020-08-07T15:13:52.755 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.755 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:52.756 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.370939+0000", 2020-08-07T15:13:52.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074658490000000001 2020-08-07T15:13:52.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.757 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:52.757 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.378405+0000", 2020-08-07T15:13:52.757 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.40160148899999998 2020-08-07T15:13:52.757 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.758 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.758 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:52.758 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.780007+0000", 2020-08-07T15:13:52.758 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9700000000000005e-07 2020-08-07T15:13:52.758 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.759 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:52.759 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.780008+0000", 2020-08-07T15:13:52.759 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3720000000000007e-05 2020-08-07T15:13:52.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.760 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.760 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:52.760 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.780071+0000", 2020-08-07T15:13:52.760 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.41222172800000001 2020-08-07T15:13:52.760 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.761 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:52.761 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.761 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.761 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:18480 6.1 e240/230 ondisk, result = 0)", 2020-08-07T15:13:52.762 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.375696+0000", 2020-08-07T15:13:52.762 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 87.985595216999997, 2020-08-07T15:13:52.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.404380817, 2020-08-07T15:13:52.763 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:52.763 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:52.763 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:52.763 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:52.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.375696+0000", 2020-08-07T15:13:52.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:52.764 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.764 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:52.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.375696+0000", 2020-08-07T15:13:52.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0922e-05 2020-08-07T15:13:52.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.765 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:52.765 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.375727+0000", 2020-08-07T15:13:52.774 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2900000000000002e-06 2020-08-07T15:13:52.775 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.775 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:52.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.375732+0000", 2020-08-07T15:13:52.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.403431708 2020-08-07T15:13:52.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:52.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.779164+0000", 2020-08-07T15:13:52.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5200000000000002e-06 2020-08-07T15:13:52.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:52.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.779168+0000", 2020-08-07T15:13:52.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00082257200000000004 2020-08-07T15:13:52.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.792 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.793 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:52.793 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.779990+0000", 2020-08-07T15:13:52.793 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1599999999999998e-06 2020-08-07T15:13:52.793 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.793 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.794 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:52.794 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.779998+0000", 2020-08-07T15:13:52.794 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8645000000000003e-05 2020-08-07T15:13:52.794 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.794 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.795 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:52.795 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.780077+0000", 2020-08-07T15:13:52.795 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.404380817 2020-08-07T15:13:52.795 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.795 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:52.796 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.796 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.801 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:18560 6.e 6:7c5f8c5e:::benchmark_data_smithi074_63843_object2319:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e240)", 2020-08-07T15:13:52.801 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.407761+0000", 2020-08-07T15:13:52.801 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 87.953529954000004, 2020-08-07T15:13:52.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31539320100000001, 2020-08-07T15:13:52.802 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:52.802 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:52.802 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:52.802 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:52.802 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:52.803 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18560 2020-08-07T15:13:52.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.803 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:52.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:52.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407761+0000", 2020-08-07T15:13:52.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:52.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:52.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407761+0000", 2020-08-07T15:13:52.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0897000000000003e-05 2020-08-07T15:13:52.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:52.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407822+0000", 2020-08-07T15:13:52.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8390000000000001e-06 2020-08-07T15:13:52.806 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:52.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407829+0000", 2020-08-07T15:13:52.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3609e-05 2020-08-07T15:13:52.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:52.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407843+0000", 2020-08-07T15:13:52.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3169999999999998e-06 2020-08-07T15:13:52.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.808 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:52.809 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.407849+0000", 2020-08-07T15:13:52.809 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002504886 2020-08-07T15:13:52.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.809 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:52.810 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.410354+0000", 2020-08-07T15:13:52.810 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7235e-05 2020-08-07T15:13:52.810 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.810 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.810 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:52.811 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.410451+0000", 2020-08-07T15:13:52.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016856999999999999 2020-08-07T15:13:52.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.811 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:52.811 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.410620+0000", 2020-08-07T15:13:52.813 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0026816259999999999 2020-08-07T15:13:52.814 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.814 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.814 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:52.814 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.413301+0000", 2020-08-07T15:13:52.814 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0020516309999999999 2020-08-07T15:13:52.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.815 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:52.815 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.415353+0000", 2020-08-07T15:13:52.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.30769196399999998 2020-08-07T15:13:52.816 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:52.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.723045+0000", 2020-08-07T15:13:52.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1600000000000001e-07 2020-08-07T15:13:52.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.817 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:52.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.723046+0000", 2020-08-07T15:13:52.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1449000000000001e-05 2020-08-07T15:13:52.818 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.818 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.818 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:52.818 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.723107+0000", 2020-08-07T15:13:52.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7462000000000001e-05 2020-08-07T15:13:52.819 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:52.819 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.723155+0000", 2020-08-07T15:13:52.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.31539320100000001 2020-08-07T15:13:52.820 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.820 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:52.820 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.821 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:18608 6.4 6:2dddf5bf:::benchmark_data_smithi074_63843_object2325:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e240)", 2020-08-07T15:13:52.821 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.438031+0000", 2020-08-07T15:13:52.821 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 87.923260557999996, 2020-08-07T15:13:52.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28378579100000001, 2020-08-07T15:13:52.822 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:52.822 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:52.822 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:52.822 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:52.822 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:52.823 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 18608 2020-08-07T15:13:52.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.823 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:52.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:52.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438031+0000", 2020-08-07T15:13:52.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:52.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:52.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438031+0000", 2020-08-07T15:13:52.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0028999999999998e-05 2020-08-07T15:13:52.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:52.826 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438081+0000", 2020-08-07T15:13:52.826 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7289999999999997e-06 2020-08-07T15:13:52.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.827 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:52.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438087+0000", 2020-08-07T15:13:52.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0959e-05 2020-08-07T15:13:52.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:52.828 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438098+0000", 2020-08-07T15:13:52.828 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0329999999999999e-06 2020-08-07T15:13:52.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.829 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:52.829 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.438103+0000", 2020-08-07T15:13:52.829 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001770911 2020-08-07T15:13:52.829 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.829 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.830 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:52.830 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.439873+0000", 2020-08-07T15:13:52.830 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3503000000000005e-05 2020-08-07T15:13:52.830 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.830 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.831 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:52.831 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.439937+0000", 2020-08-07T15:13:52.831 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000110179 2020-08-07T15:13:52.831 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.831 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.831 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 3,5", 2020-08-07T15:13:52.832 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.440047+0000", 2020-08-07T15:13:52.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033910759999999998 2020-08-07T15:13:52.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.832 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:52.833 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443438+0000", 2020-08-07T15:13:52.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2877999999999999e-05 2020-08-07T15:13:52.833 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:52.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443471+0000", 2020-08-07T15:13:52.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27823471799999999 2020-08-07T15:13:52.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:52.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721706+0000", 2020-08-07T15:13:52.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4300000000000003e-07 2020-08-07T15:13:52.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:52.836 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721706+0000", 2020-08-07T15:13:52.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2877000000000001e-05 2020-08-07T15:13:52.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.836 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:52.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721769+0000", 2020-08-07T15:13:52.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7355999999999999e-05 2020-08-07T15:13:52.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.838 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:52.838 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721817+0000", 2020-08-07T15:13:52.838 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28378579100000001 2020-08-07T15:13:52.838 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.838 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:52.839 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.840 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:18608 6.4 e240/230 ondisk, result = 0)", 2020-08-07T15:13:52.840 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:24.443879+0000", 2020-08-07T15:13:52.840 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 87.917412319999997, 2020-08-07T15:13:52.840 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.277941838, 2020-08-07T15:13:52.840 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:52.841 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:52.841 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:52.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:52.841 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443879+0000", 2020-08-07T15:13:52.842 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:52.842 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.842 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.842 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:52.842 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443879+0000", 2020-08-07T15:13:52.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3692000000000002e-05 2020-08-07T15:13:52.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.843 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:52.844 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443913+0000", 2020-08-07T15:13:52.844 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.02e-06 2020-08-07T15:13:52.845 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:52.846 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.443919+0000", 2020-08-07T15:13:52.846 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27734076499999999 2020-08-07T15:13:52.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.846 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.846 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:52.847 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721260+0000", 2020-08-07T15:13:52.847 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.78e-06 2020-08-07T15:13:52.847 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.847 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.847 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:52.848 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721264+0000", 2020-08-07T15:13:52.848 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00042663999999999999 2020-08-07T15:13:52.848 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.848 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.848 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:52.849 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721691+0000", 2020-08-07T15:13:52.849 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1509999999999998e-06 2020-08-07T15:13:52.849 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.849 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.849 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:52.850 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721698+0000", 2020-08-07T15:13:52.850 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012279000000000001 2020-08-07T15:13:52.850 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.850 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:52.851 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:24.721821+0000", 2020-08-07T15:13:52.851 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.277941838 2020-08-07T15:13:52.851 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.851 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:52.851 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.852 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:29256 6.1 6:8599f289:::benchmark_data_smithi074_63843_object3656:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e244)", 2020-08-07T15:13:52.852 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:31.539560+0000", 2020-08-07T15:13:52.852 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 80.821731817, 2020-08-07T15:13:52.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28482166599999997, 2020-08-07T15:13:52.857 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:52.857 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:52.857 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:52.857 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:52.858 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:52.858 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 29256 2020-08-07T15:13:52.858 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.858 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:52.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.859 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:52.859 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.539560+0000", 2020-08-07T15:13:52.860 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:52.860 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.860 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:52.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.539560+0000", 2020-08-07T15:13:52.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5965e-05 2020-08-07T15:13:52.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.861 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:52.861 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.539586+0000", 2020-08-07T15:13:52.862 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1719999999999999e-06 2020-08-07T15:13:52.862 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.862 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.862 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:52.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.539589+0000", 2020-08-07T15:13:52.863 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9700000000000005e-06 2020-08-07T15:13:52.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:52.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.539598+0000", 2020-08-07T15:13:52.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9999999999999998e-06 2020-08-07T15:13:52.864 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:52.865 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.539602+0000", 2020-08-07T15:13:52.865 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011500200000000001 2020-08-07T15:13:52.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.867 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:52.868 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.540752+0000", 2020-08-07T15:13:52.868 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000107188 2020-08-07T15:13:52.868 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.868 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.869 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:52.872 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.540859+0000", 2020-08-07T15:13:52.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011065600000000001 2020-08-07T15:13:52.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.872 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.872 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,6", 2020-08-07T15:13:52.873 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.540970+0000", 2020-08-07T15:13:52.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0018553440000000001 2020-08-07T15:13:52.873 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.874 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:52.874 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.542825+0000", 2020-08-07T15:13:52.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.048e-06 2020-08-07T15:13:52.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:52.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.542826+0000", 2020-08-07T15:13:52.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2224000000000005e-05 2020-08-07T15:13:52.876 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.876 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:52.877 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.542898+0000", 2020-08-07T15:13:52.877 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.029129893 2020-08-07T15:13:52.877 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.877 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.878 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:52.879 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.572028+0000", 2020-08-07T15:13:52.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.252307642 2020-08-07T15:13:52.879 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.879 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:52.880 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.824336+0000", 2020-08-07T15:13:52.880 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9899999999999999e-07 2020-08-07T15:13:52.880 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.880 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:52.881 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.824337+0000", 2020-08-07T15:13:52.881 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4644999999999997e-05 2020-08-07T15:13:52.881 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.881 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.881 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:52.882 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:31.824381+0000", 2020-08-07T15:13:52.882 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28482166599999997 2020-08-07T15:13:52.883 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.883 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:52.883 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.883 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.883 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.884 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:32560 6.e 6:77482cb7:::benchmark_data_smithi074_63843_object4069:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e246)", 2020-08-07T15:13:52.884 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:33.538197+0000", 2020-08-07T15:13:52.884 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 78.823094542999996, 2020-08-07T15:13:52.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28301590900000001, 2020-08-07T15:13:52.884 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:52.885 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:52.886 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:52.886 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:52.886 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:52.887 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 32560 2020-08-07T15:13:52.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.887 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:52.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:52.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.538197+0000", 2020-08-07T15:13:52.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:52.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:52.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.538197+0000", 2020-08-07T15:13:52.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6372000000000003e-05 2020-08-07T15:13:52.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:52.890 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.538243+0000", 2020-08-07T15:13:52.890 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9100000000000004e-06 2020-08-07T15:13:52.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.890 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:52.891 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.538248+0000", 2020-08-07T15:13:52.891 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0688e-05 2020-08-07T15:13:52.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.891 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.891 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:52.892 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.538259+0000", 2020-08-07T15:13:52.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0649999999999998e-06 2020-08-07T15:13:52.892 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.892 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.892 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:52.893 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.538264+0000", 2020-08-07T15:13:52.893 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003496645 2020-08-07T15:13:52.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.893 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:52.894 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.541760+0000", 2020-08-07T15:13:52.894 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019299399999999999 2020-08-07T15:13:52.894 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.894 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.894 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:52.895 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.541953+0000", 2020-08-07T15:13:52.895 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018635200000000001 2020-08-07T15:13:52.895 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.895 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.895 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:52.896 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.542140+0000", 2020-08-07T15:13:52.896 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001795608 2020-08-07T15:13:52.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.896 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:52.897 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.543935+0000", 2020-08-07T15:13:52.897 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2399999999999998e-07 2020-08-07T15:13:52.897 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.897 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.897 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:52.898 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.543936+0000", 2020-08-07T15:13:52.898 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7708999999999999e-05 2020-08-07T15:13:52.898 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.898 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.898 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:52.898 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.543994+0000", 2020-08-07T15:13:52.899 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032640360000000001 2020-08-07T15:13:52.899 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.899 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.899 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:52.899 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.547258+0000", 2020-08-07T15:13:52.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.27388560899999997 2020-08-07T15:13:52.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.901 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:52.901 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.821143+0000", 2020-08-07T15:13:52.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0410000000000001e-06 2020-08-07T15:13:52.901 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.902 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.902 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:52.902 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.821144+0000", 2020-08-07T15:13:52.902 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8355999999999994e-05 2020-08-07T15:13:52.903 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.903 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:52.903 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:33.821213+0000", 2020-08-07T15:13:52.903 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.28301590900000001 2020-08-07T15:13:52.904 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.904 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:52.904 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.905 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:48584 6.e 6:72419a6c:::benchmark_data_smithi074_63843_object6072:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e256)", 2020-08-07T15:13:52.905 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:43.685874+0000", 2020-08-07T15:13:52.905 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 68.675417526000004, 2020-08-07T15:13:52.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.336964296, 2020-08-07T15:13:52.905 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:52.906 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:52.906 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:52.906 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:52.906 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:52.906 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 48584 2020-08-07T15:13:52.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.907 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:52.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:52.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.685874+0000", 2020-08-07T15:13:52.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:52.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.908 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:52.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.685874+0000", 2020-08-07T15:13:52.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9709000000000006e-05 2020-08-07T15:13:52.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:52.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.685944+0000", 2020-08-07T15:13:52.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7719999999999994e-06 2020-08-07T15:13:52.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.910 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:52.910 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.685951+0000", 2020-08-07T15:13:52.910 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5424e-05 2020-08-07T15:13:52.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:52.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.685967+0000", 2020-08-07T15:13:52.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3549999999999999e-06 2020-08-07T15:13:52.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:52.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.685974+0000", 2020-08-07T15:13:52.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2768000000000001e-05 2020-08-07T15:13:52.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:52.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.686017+0000", 2020-08-07T15:13:52.913 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6916999999999994e-05 2020-08-07T15:13:52.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:52.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.686114+0000", 2020-08-07T15:13:52.914 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018684600000000001 2020-08-07T15:13:52.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:52.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.686301+0000", 2020-08-07T15:13:52.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0050644929999999998 2020-08-07T15:13:52.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:52.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.691365+0000", 2020-08-07T15:13:52.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.142e-06 2020-08-07T15:13:52.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:52.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.691366+0000", 2020-08-07T15:13:52.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2626999999999994e-05 2020-08-07T15:13:52.918 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.918 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:52.918 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.691439+0000", 2020-08-07T15:13:52.918 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0093850860000000008 2020-08-07T15:13:52.919 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.919 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:52.919 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.700824+0000", 2020-08-07T15:13:52.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.32193047299999999 2020-08-07T15:13:52.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:52.920 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022754+0000", 2020-08-07T15:13:52.920 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.147e-06 2020-08-07T15:13:52.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:52.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022756+0000", 2020-08-07T15:13:52.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2536999999999999e-05 2020-08-07T15:13:52.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:52.923 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022838+0000", 2020-08-07T15:13:52.923 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.336964296 2020-08-07T15:13:52.924 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.924 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:52.924 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.925 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:48584 6.e e256/230 ondisk, result = 0)", 2020-08-07T15:13:52.925 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:43.688495+0000", 2020-08-07T15:13:52.925 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 68.672795880999999, 2020-08-07T15:13:52.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33435100299999998, 2020-08-07T15:13:52.926 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:52.926 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:13:52.926 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:52.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:52.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.688495+0000", 2020-08-07T15:13:52.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:52.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:52.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.688495+0000", 2020-08-07T15:13:52.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2947999999999997e-05 2020-08-07T15:13:52.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:52.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.688528+0000", 2020-08-07T15:13:52.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4979999999999997e-06 2020-08-07T15:13:52.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:52.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:43.688534+0000", 2020-08-07T15:13:52.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33414495300000002 2020-08-07T15:13:52.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:52.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022679+0000", 2020-08-07T15:13:52.931 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6669999999999999e-06 2020-08-07T15:13:52.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:52.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022686+0000", 2020-08-07T15:13:52.932 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1094999999999998e-05 2020-08-07T15:13:52.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.932 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.932 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:52.933 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022737+0000", 2020-08-07T15:13:52.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2629999999999992e-06 2020-08-07T15:13:52.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.933 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.933 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:52.934 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022746+0000", 2020-08-07T15:13:52.934 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000100579 2020-08-07T15:13:52.934 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.934 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.934 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:52.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:44.022846+0000", 2020-08-07T15:13:52.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.33435100299999998 2020-08-07T15:13:52.935 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.935 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:52.935 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.936 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:61896 6.e 6:75870b8d:::benchmark_data_smithi074_63843_object7736:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e263)", 2020-08-07T15:13:52.936 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:12:52.341370+0000", 2020-08-07T15:13:52.936 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 60.019921736999997, 2020-08-07T15:13:52.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.281931072, 2020-08-07T15:13:52.937 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:52.937 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:52.937 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:52.937 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:52.937 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:52.938 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 61896 2020-08-07T15:13:52.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.938 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:52.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:52.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.341370+0000", 2020-08-07T15:13:52.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:52.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.939 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:52.940 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.341370+0000", 2020-08-07T15:13:52.940 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7135000000000003e-05 2020-08-07T15:13:52.940 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.940 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.940 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:52.941 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.341407+0000", 2020-08-07T15:13:52.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0359999999999997e-06 2020-08-07T15:13:52.941 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.941 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.941 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:52.942 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.341412+0000", 2020-08-07T15:13:52.942 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0572e-05 2020-08-07T15:13:52.942 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.942 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.942 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:52.943 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.341422+0000", 2020-08-07T15:13:52.943 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.279e-06 2020-08-07T15:13:52.943 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.943 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.943 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:52.944 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.341427+0000", 2020-08-07T15:13:52.944 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0024648579999999999 2020-08-07T15:13:52.944 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.944 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.944 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:52.945 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.343891+0000", 2020-08-07T15:13:52.945 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1310000000000005e-05 2020-08-07T15:13:52.945 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.945 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.945 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:52.946 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.343983+0000", 2020-08-07T15:13:52.946 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000175744 2020-08-07T15:13:52.946 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.946 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.947 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:52.947 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.344159+0000", 2020-08-07T15:13:52.947 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005415202 2020-08-07T15:13:52.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.947 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.948 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:52.948 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.349574+0000", 2020-08-07T15:13:52.948 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8599999999999998e-07 2020-08-07T15:13:52.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.949 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:52.949 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.349574+0000", 2020-08-07T15:13:52.949 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9826999999999994e-05 2020-08-07T15:13:52.949 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.950 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:52.950 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.349644+0000", 2020-08-07T15:13:52.950 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.05151385 2020-08-07T15:13:52.950 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.950 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:52.951 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.401158+0000", 2020-08-07T15:13:52.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.22207370600000001 2020-08-07T15:13:52.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.952 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:52.952 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.623232+0000", 2020-08-07T15:13:52.952 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1999999999999998e-07 2020-08-07T15:13:52.952 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.952 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.953 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:52.953 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.623233+0000", 2020-08-07T15:13:52.953 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7947000000000005e-05 2020-08-07T15:13:52.953 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.953 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.954 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:52.954 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:12:52.623301+0000", 2020-08-07T15:13:52.954 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.281931072 2020-08-07T15:13:52.954 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.954 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:52.955 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.955 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.955 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.955 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:74192 6.e 6:7ab1a263:::benchmark_data_smithi074_63843_object9273:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e269)", 2020-08-07T15:13:52.956 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:01.584372+0000", 2020-08-07T15:13:52.956 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 50.776919276999998, 2020-08-07T15:13:52.956 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96302370800000003, 2020-08-07T15:13:52.956 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:52.956 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:52.957 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:52.957 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:52.957 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:52.957 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 74192 2020-08-07T15:13:52.957 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.958 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:52.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:52.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.584372+0000", 2020-08-07T15:13:52.958 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:52.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:52.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.584372+0000", 2020-08-07T15:13:52.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2189e-05 2020-08-07T15:13:52.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.960 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:52.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.584404+0000", 2020-08-07T15:13:52.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9859999999999999e-06 2020-08-07T15:13:52.961 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:52.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.584407+0000", 2020-08-07T15:13:52.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9560000000000002e-06 2020-08-07T15:13:52.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.962 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:52.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.584413+0000", 2020-08-07T15:13:52.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6790000000000001e-06 2020-08-07T15:13:52.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:52.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.584416+0000", 2020-08-07T15:13:52.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7514000000000001e-05 2020-08-07T15:13:52.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:52.964 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.584443+0000", 2020-08-07T15:13:52.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8560000000000001e-06 2020-08-07T15:13:52.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:52.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.584450+0000", 2020-08-07T15:13:52.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0091272760000000001 2020-08-07T15:13:52.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:52.966 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.593578+0000", 2020-08-07T15:13:52.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00055358000000000002 2020-08-07T15:13:52.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:52.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.594131+0000", 2020-08-07T15:13:52.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000182704 2020-08-07T15:13:52.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:52.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.594314+0000", 2020-08-07T15:13:52.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0025760470000000001 2020-08-07T15:13:52.969 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.969 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.969 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:52.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.596890+0000", 2020-08-07T15:13:52.969 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00028334900000000001 2020-08-07T15:13:52.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.970 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:52.970 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.597173+0000", 2020-08-07T15:13:52.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.95006429999999997 2020-08-07T15:13:52.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:52.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.547238+0000", 2020-08-07T15:13:52.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0430000000000002e-06 2020-08-07T15:13:52.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:52.978 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.547240+0000", 2020-08-07T15:13:52.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6477000000000006e-05 2020-08-07T15:13:52.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.978 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.978 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:52.979 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.547326+0000", 2020-08-07T15:13:52.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9752000000000006e-05 2020-08-07T15:13:52.979 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.979 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:52.980 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.547396+0000", 2020-08-07T15:13:52.980 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96302370800000003 2020-08-07T15:13:52.980 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.980 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:52.981 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:52.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.981 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:91032 6.e 6:703cf75c:::benchmark_data_smithi074_63843_object11378:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e275)", 2020-08-07T15:13:52.981 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:13.429721+0000", 2020-08-07T15:13:52.982 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 38.931570829000002, 2020-08-07T15:13:52.982 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29695800300000003, 2020-08-07T15:13:52.982 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:52.982 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:52.983 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:52.983 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:13:52.984 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:52.984 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 91032 2020-08-07T15:13:52.985 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.985 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:52.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.986 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:52.986 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429721+0000", 2020-08-07T15:13:52.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:52.987 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:52.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429721+0000", 2020-08-07T15:13:52.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3047000000000002e-05 2020-08-07T15:13:52.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.990 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:52.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429774+0000", 2020-08-07T15:13:52.991 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.339e-06 2020-08-07T15:13:52.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:52.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429779+0000", 2020-08-07T15:13:52.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0742e-05 2020-08-07T15:13:52.993 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.993 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.993 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:52.994 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429790+0000", 2020-08-07T15:13:52.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1860000000000002e-06 2020-08-07T15:13:52.994 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.995 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.995 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:52.995 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.429794+0000", 2020-08-07T15:13:52.995 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001193794 2020-08-07T15:13:52.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.996 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.996 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:52.996 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.430988+0000", 2020-08-07T15:13:52.997 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7213000000000006e-05 2020-08-07T15:13:52.997 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.997 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.998 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:52.998 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.431085+0000", 2020-08-07T15:13:52.998 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000129875 2020-08-07T15:13:52.998 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:52.999 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:52.999 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 2,4", 2020-08-07T15:13:52.999 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.431215+0000", 2020-08-07T15:13:52.999 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038327349999999999 2020-08-07T15:13:53.000 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.000 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.000 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:53.001 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.435047+0000", 2020-08-07T15:13:53.002 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3499999999999996e-07 2020-08-07T15:13:53.002 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:53.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.435048+0000", 2020-08-07T15:13:53.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9911999999999999e-05 2020-08-07T15:13:53.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:53.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.435098+0000", 2020-08-07T15:13:53.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0046481059999999999 2020-08-07T15:13:53.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.439746+0000", 2020-08-07T15:13:53.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.286860114 2020-08-07T15:13:53.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.726606+0000", 2020-08-07T15:13:53.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0800000000000003e-07 2020-08-07T15:13:53.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.010 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:53.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.726607+0000", 2020-08-07T15:13:53.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1396999999999994e-05 2020-08-07T15:13:53.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.012 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:53.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:13.726679+0000", 2020-08-07T15:13:53.013 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.29695800300000003 2020-08-07T15:13:53.013 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.013 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:53.014 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.014 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.014 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.015 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5481.0:149907 6.d e298/230 6:b8bf7ad4:::benchmark_data_smithi074_63843_object16655:head v 298'9028, mlcod=298'9024)", 2020-08-07T15:13:53.015 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:51.266973+0000", 2020-08-07T15:13:53.015 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 1.0943184210000001, 2020-08-07T15:13:53.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.372849241, 2020-08-07T15:13:53.016 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:53.016 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:53.017 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:53.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:53.017 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:51.266973+0000", 2020-08-07T15:13:53.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:53.018 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.018 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.018 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:53.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:51.266973+0000", 2020-08-07T15:13:53.019 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4483000000000001e-05 2020-08-07T15:13:53.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.019 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.020 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:53.020 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:51.267017+0000", 2020-08-07T15:13:53.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6189999999999996e-06 2020-08-07T15:13:53.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.020 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:53.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:51.267024+0000", 2020-08-07T15:13:53.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.37198234000000002 2020-08-07T15:13:53.021 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.021 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:53.022 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:51.639006+0000", 2020-08-07T15:13:53.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6197000000000001e-05 2020-08-07T15:13:53.023 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:53.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:51.639023+0000", 2020-08-07T15:13:53.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000138014 2020-08-07T15:13:53.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:51.639161+0000", 2020-08-07T15:13:53.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7029e-05 2020-08-07T15:13:53.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:53.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:51.639208+0000", 2020-08-07T15:13:53.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00056590199999999996 2020-08-07T15:13:53.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:53.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:51.639774+0000", 2020-08-07T15:13:53.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8656999999999998e-05 2020-08-07T15:13:53.030 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.030 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.030 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:53.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:51.639822+0000", 2020-08-07T15:13:53.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.372849241 2020-08-07T15:13:53.031 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.032 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:53.032 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.032 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.033 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:53.033 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:53.034 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:13:53.035 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2020-08-07T15:13:53.036 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:13:53.036 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:13:53.038 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:52.819+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:53.051 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:53.049+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:53.141 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:53.140+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:53.202 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:53.203 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:13:53.203 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 0 2020-08-07T15:13:53.203 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:53.212 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:13:53.246 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:53.242+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:53.340 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:53.341 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:13:53.341 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:13:53.341 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:13:53.341 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:53.344 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:53.343+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:53.364 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:13:53.377 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:13:53.379 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:13:53.391 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":528,"stamp":"2020-08-07T15:13:52.700200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":121643032,"num_objects":1860,"num_object_clones":0,"num_object_copies":5580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":148050,"num_write_kb":1065993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":962,"num_bytes_recovered":62922752,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":195,"acting":195,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":196,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8720812,"kb_used_data":332012,"kb_used_omap":2,"kb_used_meta":8388605,"kb_avail":746253908,"statfs":{"total":773094113280,"available":764164001792,"internally_reserved":8589934592,"allocated":339980288,"data_stored":335788836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":8589932013},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,4,2,5,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-923795432,"num_objects":-14095,"num_object_clones":0,"num_object_copies":-42285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-14095,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14673,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001212"},"pg_stats":[{"pgid":"1.28","version":"16'2","reported_seq":"62","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.252317+0000","last_change":"2020-08-07T15:12:37.495634+0000","last_active":"2020-08-07T15:13:47.252317+0000","last_peered":"2020-08-07T15:13:47.252317+0000","last_clean":"2020-08-07T15:13:47.252317+0000","last_became_active":"2020-08-07T15:12:37.495085+0000","last_became_peered":"2020-08-07T15:12:37.495085+0000","last_unstale":"2020-08-07T15:13:47.252317+0000","last_undegraded":"2020-08-07T15:13:47.252317+0000","last_fullsized":"2020-08-07T15:13:47.252317+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.490655+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.490655+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.27","version":"16'2","reported_seq":"409","reported_epoch":"296","state":"active+clean","last_fresh":"2020-08-07T15:13:46.494545+0000","last_change":"2020-08-07T15:12:57.250017+0000","last_active":"2020-08-07T15:13:46.494545+0000","last_peered":"2020-08-07T15:13:46.494545+0000","last_clean":"2020-08-07T15:13:46.494545+0000","last_became_active":"2020-08-07T15:12:37.411869+0000","last_became_peered":"2020-08-07T15:12:37.411869+0000","last_unstale":"2020-08-07T15:13:46.494545+0000","last_undegraded":"2020-08-07T15:13:46.494545+0000","last_fullsized":"2020-08-07T15:13:46.494545+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:57.249927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:57.249927+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.26","version":"16'2","reported_seq":"72","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.253634+0000","last_change":"2020-08-07T15:12:40.765120+0000","last_active":"2020-08-07T15:13:47.253634+0000","last_peered":"2020-08-07T15:13:47.253634+0000","last_clean":"2020-08-07T15:13:47.253634+0000","last_became_active":"2020-08-07T15:12:37.457426+0000","last_became_peered":"2020-08-07T15:12:37.457426+0000","last_unstale":"2020-08-07T15:13:47.253634+0000","last_undegraded":"2020-08-07T15:13:47.253634+0000","last_fullsized":"2020-08-07T15:13:47.253634+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:40.765027+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_clean_scrub_stamp":"2020-08-07T15:12:40.765027+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.25","version":"75'3","reported_seq":"298","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.252927+0000","last_change":"2020-08-07T15:13:07.465488+0000","last_active":"2020-08-07T15:13:47.252927+0000","last_peered":"2020-08-07T15:13:47.252927+0000","last_clean":"2020-08-07T15:13:47.252927+0000","last_became_active":"2020-08-07T15:12:37.427816+0000","last_became_peered":"2020-08-07T15:12:37.427816+0000","last_unstale":"2020-08-07T15:13:47.252927+0000","last_undegraded":"2020-08-07T15:13:47.252927+0000","last_fullsized":"2020-08-07T15:13:47.252927+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:13:07.465410+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:13:07.465410+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.24","version":"16'2","reported_seq":"236","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.258081+0000","last_change":"2020-08-07T15:12:42.295755+0000","last_active":"2020-08-07T15:13:47.258081+0000","last_peered":"2020-08-07T15:13:47.258081+0000","last_clean":"2020-08-07T15:13:47.258081+0000","last_became_active":"2020-08-07T15:12:34.414144+0000","last_became_peered":"2020-08-07T15:12:34.414144+0000","last_unstale":"2020-08-07T15:13:47.258081+0000","last_undegraded":"2020-08-07T15:13:47.258081+0000","last_fullsized":"2020-08-07T15:13:47.258081+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:42.295672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:42.295672+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.23","version":"16'2","reported_seq":"297","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.257766+0000","last_change":"2020-08-07T15:13:38.275094+0000","last_active":"2020-08-07T15:13:47.257766+0000","last_peered":"2020-08-07T15:13:47.257766+0000","last_clean":"2020-08-07T15:13:47.257766+0000","last_became_active":"2020-08-07T15:13:38.274095+0000","last_became_peered":"2020-08-07T15:13:38.274095+0000","last_unstale":"2020-08-07T15:13:47.257766+0000","last_undegraded":"2020-08-07T15:13:47.257766+0000","last_fullsized":"2020-08-07T15:13:47.257766+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":293,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:55.290028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:55.290028+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.22","version":"16'2","reported_seq":"64","reported_epoch":"296","state":"active+clean","last_fresh":"2020-08-07T15:13:46.494764+0000","last_change":"2020-08-07T15:12:58.227750+0000","last_active":"2020-08-07T15:13:46.494764+0000","last_peered":"2020-08-07T15:13:46.494764+0000","last_clean":"2020-08-07T15:13:46.494764+0000","last_became_active":"2020-08-07T15:12:37.413046+0000","last_became_peered":"2020-08-07T15:12:37.413046+0000","last_unstale":"2020-08-07T15:13:46.494764+0000","last_undegraded":"2020-08-07T15:13:46.494764+0000","last_fullsized":"2020-08-07T15:13:46.494764+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:58.227699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:58.227699+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.21","version":"16'2","reported_seq":"72","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.257234+0000","last_change":"2020-08-07T15:12:58.387474+0000","last_active":"2020-08-07T15:13:47.257234+0000","last_peered":"2020-08-07T15:13:47.257234+0000","last_clean":"2020-08-07T15:13:47.257234+0000","last_became_active":"2020-08-07T15:12:37.418612+0000","last_became_peered":"2020-08-07T15:12:37.418612+0000","last_unstale":"2020-08-07T15:13:47.257234+0000","last_undegraded":"2020-08-07T15:13:47.257234+0000","last_fullsized":"2020-08-07T15:13:47.257234+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:58.387424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:58.387424+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.20","version":"16'2","reported_seq":"72","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.252711+0000","last_change":"2020-08-07T15:12:54.433326+0000","last_active":"2020-08-07T15:13:47.252711+0000","last_peered":"2020-08-07T15:13:47.252711+0000","last_clean":"2020-08-07T15:13:47.252711+0000","last_became_active":"2020-08-07T15:12:37.409249+0000","last_became_peered":"2020-08-07T15:12:37.409249+0000","last_unstale":"2020-08-07T15:13:47.252711+0000","last_undegraded":"2020-08-07T15:13:47.252711+0000","last_fullsized":"2020-08-07T15:13:47.252711+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:54.433245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:54.433245+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1f","version":"16'2","reported_seq":"65","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.253155+0000","last_change":"2020-08-07T15:13:09.471546+0000","last_active":"2020-08-07T15:13:47.253155+0000","last_peered":"2020-08-07T15:13:47.253155+0000","last_clean":"2020-08-07T15:13:47.253155+0000","last_became_active":"2020-08-07T15:12:37.410716+0000","last_became_peered":"2020-08-07T15:12:37.410716+0000","last_unstale":"2020-08-07T15:13:47.253155+0000","last_undegraded":"2020-08-07T15:13:47.253155+0000","last_fullsized":"2020-08-07T15:13:47.253155+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:09.471471+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_clean_scrub_stamp":"2020-08-07T15:13:09.471471+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1e","version":"16'2","reported_seq":"152","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.258136+0000","last_change":"2020-08-07T15:13:09.396723+0000","last_active":"2020-08-07T15:13:47.258136+0000","last_peered":"2020-08-07T15:13:47.258136+0000","last_clean":"2020-08-07T15:13:47.258136+0000","last_became_active":"2020-08-07T15:10:51.559894+0000","last_became_peered":"2020-08-07T15:10:51.559894+0000","last_unstale":"2020-08-07T15:13:47.258136+0000","last_undegraded":"2020-08-07T15:13:47.258136+0000","last_fullsized":"2020-08-07T15:13:47.258136+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:09.396633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:09.396633+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1d","version":"75'3","reported_seq":"179","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.253919+0000","last_change":"2020-08-07T15:13:24.343240+0000","last_active":"2020-08-07T15:13:47.253919+0000","last_peered":"2020-08-07T15:13:47.253919+0000","last_clean":"2020-08-07T15:13:47.253919+0000","last_became_active":"2020-08-07T15:13:24.342549+0000","last_became_peered":"2020-08-07T15:13:24.342549+0000","last_unstale":"2020-08-07T15:13:47.253919+0000","last_undegraded":"2020-08-07T15:13:47.253919+0000","last_fullsized":"2020-08-07T15:13:47.253919+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:28.855432+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:28.855432+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1c","version":"16'2","reported_seq":"152","reported_epoch":"296","state":"active+clean","last_fresh":"2020-08-07T15:13:46.494273+0000","last_change":"2020-08-07T15:13:07.471270+0000","last_active":"2020-08-07T15:13:46.494273+0000","last_peered":"2020-08-07T15:13:46.494273+0000","last_clean":"2020-08-07T15:13:46.494273+0000","last_became_active":"2020-08-07T15:13:02.591669+0000","last_became_peered":"2020-08-07T15:13:02.591669+0000","last_unstale":"2020-08-07T15:13:46.494273+0000","last_undegraded":"2020-08-07T15:13:46.494273+0000","last_fullsized":"2020-08-07T15:13:46.494273+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:07.471192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:07.471192+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1b","version":"16'2","reported_seq":"273","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.252741+0000","last_change":"2020-08-07T15:13:22.022126+0000","last_active":"2020-08-07T15:13:47.252741+0000","last_peered":"2020-08-07T15:13:47.252741+0000","last_clean":"2020-08-07T15:13:47.252741+0000","last_became_active":"2020-08-07T15:10:51.557392+0000","last_became_peered":"2020-08-07T15:10:51.557392+0000","last_unstale":"2020-08-07T15:13:47.252741+0000","last_undegraded":"2020-08-07T15:13:47.252741+0000","last_fullsized":"2020-08-07T15:13:47.252741+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:22.022047+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:57.796946+0000","last_clean_scrub_stamp":"2020-08-07T15:13:22.022047+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1a","version":"16'2","reported_seq":"44","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.252674+0000","last_change":"2020-08-07T15:13:06.881943+0000","last_active":"2020-08-07T15:13:47.252674+0000","last_peered":"2020-08-07T15:13:47.252674+0000","last_clean":"2020-08-07T15:13:47.252674+0000","last_became_active":"2020-08-07T15:13:02.590409+0000","last_became_peered":"2020-08-07T15:13:02.590409+0000","last_unstale":"2020-08-07T15:13:47.252674+0000","last_undegraded":"2020-08-07T15:13:47.252674+0000","last_fullsized":"2020-08-07T15:13:47.252674+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:06.881863+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:13:06.881863+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.19","version":"16'2","reported_seq":"195","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.257380+0000","last_change":"2020-08-07T15:12:15.372999+0000","last_active":"2020-08-07T15:13:47.257380+0000","last_peered":"2020-08-07T15:13:47.257380+0000","last_clean":"2020-08-07T15:13:47.257380+0000","last_became_active":"2020-08-07T15:10:51.559752+0000","last_became_peered":"2020-08-07T15:10:51.559752+0000","last_unstale":"2020-08-07T15:13:47.257380+0000","last_undegraded":"2020-08-07T15:13:47.257380+0000","last_fullsized":"2020-08-07T15:13:47.257380+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:15.372910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:15.372910+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.18","version":"16'2","reported_seq":"295","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.252437+0000","last_change":"2020-08-07T15:11:06.955336+0000","last_active":"2020-08-07T15:13:47.252437+0000","last_peered":"2020-08-07T15:13:47.252437+0000","last_clean":"2020-08-07T15:13:47.252437+0000","last_became_active":"2020-08-07T15:10:51.557665+0000","last_became_peered":"2020-08-07T15:10:51.557665+0000","last_unstale":"2020-08-07T15:13:47.252437+0000","last_undegraded":"2020-08-07T15:13:47.252437+0000","last_fullsized":"2020-08-07T15:13:47.252437+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:06.955262+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:06.955262+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.17","version":"16'2","reported_seq":"126","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.253423+0000","last_change":"2020-08-07T15:12:57.172008+0000","last_active":"2020-08-07T15:13:47.253423+0000","last_peered":"2020-08-07T15:13:47.253423+0000","last_clean":"2020-08-07T15:13:47.253423+0000","last_became_active":"2020-08-07T15:10:51.561769+0000","last_became_peered":"2020-08-07T15:10:51.561769+0000","last_unstale":"2020-08-07T15:13:47.253423+0000","last_undegraded":"2020-08-07T15:13:47.253423+0000","last_fullsized":"2020-08-07T15:13:47.253423+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:57.171893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:57.171893+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.16","version":"16'2","reported_seq":"71","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.253151+0000","last_change":"2020-08-07T15:12:52.576339+0000","last_active":"2020-08-07T15:13:47.253151+0000","last_peered":"2020-08-07T15:13:47.253151+0000","last_clean":"2020-08-07T15:13:47.253151+0000","last_became_active":"2020-08-07T15:12:52.575577+0000","last_became_peered":"2020-08-07T15:12:52.575577+0000","last_unstale":"2020-08-07T15:13:47.253151+0000","last_undegraded":"2020-08-07T15:13:47.253151+0000","last_fullsized":"2020-08-07T15:13:47.253151+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:12.883558+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:12.883558+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.883558+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.15","version":"75'3","reported_seq":"54","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.252986+0000","last_change":"2020-08-07T15:13:02.590465+0000","last_active":"2020-08-07T15:13:47.252986+0000","last_peered":"2020-08-07T15:13:47.252986+0000","last_clean":"2020-08-07T15:13:47.252986+0000","last_became_active":"2020-08-07T15:13:02.589879+0000","last_became_peered":"2020-08-07T15:13:02.589879+0000","last_unstale":"2020-08-07T15:13:47.252986+0000","last_undegraded":"2020-08-07T15:13:47.252986+0000","last_fullsized":"2020-08-07T15:13:47.252986+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:53.824617+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:53.824617+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.14","version":"16'2","reported_seq":"236","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.252798+0000","last_change":"2020-08-07T15:13:26.549332+0000","last_active":"2020-08-07T15:13:47.252798+0000","last_peered":"2020-08-07T15:13:47.252798+0000","last_clean":"2020-08-07T15:13:47.252798+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:13:47.252798+0000","last_undegraded":"2020-08-07T15:13:47.252798+0000","last_fullsized":"2020-08-07T15:13:47.252798+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:26.549247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:26.549247+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"248","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.252621+0000","last_change":"2020-08-07T15:12:19.577926+0000","last_active":"2020-08-07T15:13:47.252621+0000","last_peered":"2020-08-07T15:13:47.252621+0000","last_clean":"2020-08-07T15:13:47.252621+0000","last_became_active":"2020-08-07T15:12:06.375861+0000","last_became_peered":"2020-08-07T15:12:06.375861+0000","last_unstale":"2020-08-07T15:13:47.252621+0000","last_undegraded":"2020-08-07T15:13:47.252621+0000","last_fullsized":"2020-08-07T15:13:47.252621+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:07.887356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.887356+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"248","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.252610+0000","last_change":"2020-08-07T15:12:26.104598+0000","last_active":"2020-08-07T15:13:47.252610+0000","last_peered":"2020-08-07T15:13:47.252610+0000","last_clean":"2020-08-07T15:13:47.252610+0000","last_became_active":"2020-08-07T15:12:06.367362+0000","last_became_peered":"2020-08-07T15:12:06.367362+0000","last_unstale":"2020-08-07T15:13:47.252610+0000","last_undegraded":"2020-08-07T15:13:47.252610+0000","last_fullsized":"2020-08-07T15:13:47.252610+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:26.104526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:26.104526+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"346","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.252292+0000","last_change":"2020-08-07T15:12:24.105593+0000","last_active":"2020-08-07T15:13:47.252292+0000","last_peered":"2020-08-07T15:13:47.252292+0000","last_clean":"2020-08-07T15:13:47.252292+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:13:47.252292+0000","last_undegraded":"2020-08-07T15:13:47.252292+0000","last_fullsized":"2020-08-07T15:13:47.252292+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:24.105531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:24.105531+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"340","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.252542+0000","last_change":"2020-08-07T15:12:13.231819+0000","last_active":"2020-08-07T15:13:47.252542+0000","last_peered":"2020-08-07T15:13:47.252542+0000","last_clean":"2020-08-07T15:13:47.252542+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:13:47.252542+0000","last_undegraded":"2020-08-07T15:13:47.252542+0000","last_fullsized":"2020-08-07T15:13:47.252542+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:13.231762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.231762+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3","version":"297'8515","reported_seq":"8595","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.731990+0000","last_change":"2020-08-07T15:13:39.430983+0000","last_active":"2020-08-07T15:13:47.731990+0000","last_peered":"2020-08-07T15:13:47.731990+0000","last_clean":"2020-08-07T15:13:47.731990+0000","last_became_active":"2020-08-07T15:13:02.590681+0000","last_became_peered":"2020-08-07T15:13:02.590681+0000","last_unstale":"2020-08-07T15:13:47.731990+0000","last_undegraded":"2020-08-07T15:13:47.731990+0000","last_fullsized":"2020-08-07T15:13:47.731990+0000","mapping_epoch":269,"log_start":"297'8512","ondisk_log_start":"297'8512","created":230,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"294'7704","last_scrub_stamp":"2020-08-07T15:13:39.430917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:13:39.430917+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27140096,"num_objects":415,"num_object_clones":0,"num_object_copies":1245,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8515,"num_write_kb":63496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"299","reported_epoch":"296","state":"active+clean","last_fresh":"2020-08-07T15:13:46.494871+0000","last_change":"2020-08-07T15:12:27.410339+0000","last_active":"2020-08-07T15:13:46.494871+0000","last_peered":"2020-08-07T15:13:46.494871+0000","last_clean":"2020-08-07T15:13:46.494871+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:13:46.494871+0000","last_undegraded":"2020-08-07T15:13:46.494871+0000","last_fullsized":"2020-08-07T15:13:46.494871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:27.410262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:27.410262+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"228","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.257930+0000","last_change":"2020-08-07T15:12:39.258928+0000","last_active":"2020-08-07T15:13:47.257930+0000","last_peered":"2020-08-07T15:13:47.257930+0000","last_clean":"2020-08-07T15:13:47.257930+0000","last_became_active":"2020-08-07T15:12:32.259956+0000","last_became_peered":"2020-08-07T15:12:32.259956+0000","last_unstale":"2020-08-07T15:13:47.257930+0000","last_undegraded":"2020-08-07T15:13:47.257930+0000","last_fullsized":"2020-08-07T15:13:47.257930+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:39.258857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:39.258857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"298'9764","reported_seq":"9843","reported_epoch":"298","state":"active+clean","last_fresh":"2020-08-07T15:13:51.259312+0000","last_change":"2020-08-07T15:13:20.918113+0000","last_active":"2020-08-07T15:13:51.259312+0000","last_peered":"2020-08-07T15:13:51.259312+0000","last_clean":"2020-08-07T15:13:51.259312+0000","last_became_active":"2020-08-07T15:12:10.464493+0000","last_became_peered":"2020-08-07T15:12:10.464493+0000","last_unstale":"2020-08-07T15:13:51.259312+0000","last_undegraded":"2020-08-07T15:13:51.259312+0000","last_fullsized":"2020-08-07T15:13:51.259312+0000","mapping_epoch":230,"log_start":"298'9761","ondisk_log_start":"298'9761","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"282'6856","last_scrub_stamp":"2020-08-07T15:13:20.918036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:13:20.918036+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9764,"num_write_kb":69440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"256","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.253735+0000","last_change":"2020-08-07T15:12:19.561477+0000","last_active":"2020-08-07T15:13:47.253735+0000","last_peered":"2020-08-07T15:13:47.253735+0000","last_clean":"2020-08-07T15:13:47.253735+0000","last_became_active":"2020-08-07T15:12:06.367376+0000","last_became_peered":"2020-08-07T15:12:06.367376+0000","last_unstale":"2020-08-07T15:13:47.253735+0000","last_undegraded":"2020-08-07T15:13:47.253735+0000","last_fullsized":"2020-08-07T15:13:47.253735+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:12.319383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.319383+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"282","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.252950+0000","last_change":"2020-08-07T15:12:44.858389+0000","last_active":"2020-08-07T15:13:47.252950+0000","last_peered":"2020-08-07T15:13:47.252950+0000","last_clean":"2020-08-07T15:13:47.252950+0000","last_became_active":"2020-08-07T15:12:32.241146+0000","last_became_peered":"2020-08-07T15:12:32.241146+0000","last_unstale":"2020-08-07T15:13:47.252950+0000","last_undegraded":"2020-08-07T15:13:47.252950+0000","last_fullsized":"2020-08-07T15:13:47.252950+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:44.858310+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:44.858310+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"298'9182","reported_seq":"9257","reported_epoch":"298","state":"active+clean+scrubbing","last_fresh":"2020-08-07T15:13:49.858649+0000","last_change":"2020-08-07T15:13:49.815653+0000","last_active":"2020-08-07T15:13:49.858649+0000","last_peered":"2020-08-07T15:13:49.858649+0000","last_clean":"2020-08-07T15:13:49.858649+0000","last_became_active":"2020-08-07T15:12:10.463950+0000","last_became_peered":"2020-08-07T15:12:10.463950+0000","last_unstale":"2020-08-07T15:13:49.858649+0000","last_undegraded":"2020-08-07T15:13:49.858649+0000","last_fullsized":"2020-08-07T15:13:49.858649+0000","mapping_epoch":230,"log_start":"298'9179","ondisk_log_start":"298'9179","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11665408,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9182,"num_write_kb":66560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"301","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.267045+0000","last_change":"2020-08-07T15:13:11.273543+0000","last_active":"2020-08-07T15:13:47.267045+0000","last_peered":"2020-08-07T15:13:47.267045+0000","last_clean":"2020-08-07T15:13:47.267045+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:13:47.267045+0000","last_undegraded":"2020-08-07T15:13:47.267045+0000","last_fullsized":"2020-08-07T15:13:47.267045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:11.273469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:13:11.273469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"359","reported_epoch":"296","state":"active+clean","last_fresh":"2020-08-07T15:13:46.494615+0000","last_change":"2020-08-07T15:12:53.210176+0000","last_active":"2020-08-07T15:13:46.494615+0000","last_peered":"2020-08-07T15:13:46.494615+0000","last_clean":"2020-08-07T15:13:46.494615+0000","last_became_active":"2020-08-07T15:12:32.239755+0000","last_became_peered":"2020-08-07T15:12:32.239755+0000","last_unstale":"2020-08-07T15:13:46.494615+0000","last_undegraded":"2020-08-07T15:13:46.494615+0000","last_fullsized":"2020-08-07T15:13:46.494615+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:53.210093+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_clean_scrub_stamp":"2020-08-07T15:12:53.210093+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"298'9250","reported_seq":"9326","reported_epoch":"298","state":"active+clean","last_fresh":"2020-08-07T15:13:49.852642+0000","last_change":"2020-08-07T15:12:10.398415+0000","last_active":"2020-08-07T15:13:49.852642+0000","last_peered":"2020-08-07T15:13:49.852642+0000","last_clean":"2020-08-07T15:13:49.852642+0000","last_became_active":"2020-08-07T15:12:10.397834+0000","last_became_peered":"2020-08-07T15:12:10.397834+0000","last_unstale":"2020-08-07T15:13:49.852642+0000","last_undegraded":"2020-08-07T15:13:49.852642+0000","last_fullsized":"2020-08-07T15:13:49.852642+0000","mapping_epoch":230,"log_start":"298'9247","ondisk_log_start":"298'9247","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11927552,"num_objects":182,"num_object_clones":0,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9250,"num_write_kb":67072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"271","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.252503+0000","last_change":"2020-08-07T15:13:02.591821+0000","last_active":"2020-08-07T15:13:47.252503+0000","last_peered":"2020-08-07T15:13:47.252503+0000","last_clean":"2020-08-07T15:13:47.252503+0000","last_became_active":"2020-08-07T15:13:02.591402+0000","last_became_peered":"2020-08-07T15:13:02.591402+0000","last_unstale":"2020-08-07T15:13:47.252503+0000","last_undegraded":"2020-08-07T15:13:47.252503+0000","last_fullsized":"2020-08-07T15:13:47.252503+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:08.622683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:08.622683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"328","reported_epoch":"296","state":"active+clean","last_fresh":"2020-08-07T15:13:46.494534+0000","last_change":"2020-08-07T15:12:42.371309+0000","last_active":"2020-08-07T15:13:46.494534+0000","last_peered":"2020-08-07T15:13:46.494534+0000","last_clean":"2020-08-07T15:13:46.494534+0000","last_became_active":"2020-08-07T15:12:32.237984+0000","last_became_peered":"2020-08-07T15:12:32.237984+0000","last_unstale":"2020-08-07T15:13:46.494534+0000","last_undegraded":"2020-08-07T15:13:46.494534+0000","last_fullsized":"2020-08-07T15:13:46.494534+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:42.371229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:42.371229+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.7","version":"298'9576","reported_seq":"9651","reported_epoch":"298","state":"active+clean","last_fresh":"2020-08-07T15:13:51.252211+0000","last_change":"2020-08-07T15:12:10.464810+0000","last_active":"2020-08-07T15:13:51.252211+0000","last_peered":"2020-08-07T15:13:51.252211+0000","last_clean":"2020-08-07T15:13:51.252211+0000","last_became_active":"2020-08-07T15:12:10.464050+0000","last_became_peered":"2020-08-07T15:12:10.464050+0000","last_unstale":"2020-08-07T15:13:51.252211+0000","last_undegraded":"2020-08-07T15:13:51.252211+0000","last_fullsized":"2020-08-07T15:13:51.252211+0000","mapping_epoch":230,"log_start":"298'9573","ondisk_log_start":"298'9573","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9576,"num_write_kb":68096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"54","reported_epoch":"296","state":"active+clean","last_fresh":"2020-08-07T15:13:46.493369+0000","last_change":"2020-08-07T15:13:13.399410+0000","last_active":"2020-08-07T15:13:46.493369+0000","last_peered":"2020-08-07T15:13:46.493369+0000","last_clean":"2020-08-07T15:13:46.493369+0000","last_became_active":"2020-08-07T15:13:02.590656+0000","last_became_peered":"2020-08-07T15:13:02.590656+0000","last_unstale":"2020-08-07T15:13:46.493369+0000","last_undegraded":"2020-08-07T15:13:46.493369+0000","last_fullsized":"2020-08-07T15:13:46.493369+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:13.399327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:13.399327+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"298","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.252999+0000","last_change":"2020-08-07T15:11:40.328534+0000","last_active":"2020-08-07T15:13:47.252999+0000","last_peered":"2020-08-07T15:13:47.252999+0000","last_clean":"2020-08-07T15:13:47.252999+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:13:47.252999+0000","last_undegraded":"2020-08-07T15:13:47.252999+0000","last_fullsized":"2020-08-07T15:13:47.252999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:11:40.328482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:11:40.328482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"297'9047","reported_seq":"9124","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.752038+0000","last_change":"2020-08-07T15:13:02.591551+0000","last_active":"2020-08-07T15:13:47.752038+0000","last_peered":"2020-08-07T15:13:47.752038+0000","last_clean":"2020-08-07T15:13:47.752038+0000","last_became_active":"2020-08-07T15:13:02.591000+0000","last_became_peered":"2020-08-07T15:13:02.591000+0000","last_unstale":"2020-08-07T15:13:47.752038+0000","last_undegraded":"2020-08-07T15:13:47.752038+0000","last_fullsized":"2020-08-07T15:13:47.752038+0000","mapping_epoch":269,"log_start":"297'9044","ondisk_log_start":"297'9044","created":230,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":28180480,"num_objects":430,"num_object_clones":0,"num_object_copies":1290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9047,"num_write_kb":67392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"319","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.253401+0000","last_change":"2020-08-07T15:12:10.585667+0000","last_active":"2020-08-07T15:13:47.253401+0000","last_peered":"2020-08-07T15:13:47.253401+0000","last_clean":"2020-08-07T15:13:47.253401+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:13:47.253401+0000","last_undegraded":"2020-08-07T15:13:47.253401+0000","last_fullsized":"2020-08-07T15:13:47.253401+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:10.585580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:10.585580+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"358","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.266759+0000","last_change":"2020-08-07T15:12:38.850878+0000","last_active":"2020-08-07T15:13:47.266759+0000","last_peered":"2020-08-07T15:13:47.266759+0000","last_clean":"2020-08-07T15:13:47.266759+0000","last_became_active":"2020-08-07T15:12:32.239794+0000","last_became_peered":"2020-08-07T15:12:32.239794+0000","last_unstale":"2020-08-07T15:13:47.266759+0000","last_undegraded":"2020-08-07T15:13:47.266759+0000","last_fullsized":"2020-08-07T15:13:47.266759+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:38.850809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:38.850809+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"298'9423","reported_seq":"9501","reported_epoch":"298","state":"active+clean","last_fresh":"2020-08-07T15:13:51.260445+0000","last_change":"2020-08-07T15:13:21.793789+0000","last_active":"2020-08-07T15:13:51.260445+0000","last_peered":"2020-08-07T15:13:51.260445+0000","last_clean":"2020-08-07T15:13:51.260445+0000","last_became_active":"2020-08-07T15:12:10.464582+0000","last_became_peered":"2020-08-07T15:12:10.464582+0000","last_unstale":"2020-08-07T15:13:51.260445+0000","last_undegraded":"2020-08-07T15:13:51.260445+0000","last_fullsized":"2020-08-07T15:13:51.260445+0000","mapping_epoch":230,"log_start":"298'9420","ondisk_log_start":"298'9420","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"283'6536","last_scrub_stamp":"2020-08-07T15:13:21.793703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:13:21.793703+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9423,"num_write_kb":67008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"313","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.252765+0000","last_change":"2020-08-07T15:12:07.340958+0000","last_active":"2020-08-07T15:13:47.252765+0000","last_peered":"2020-08-07T15:13:47.252765+0000","last_clean":"2020-08-07T15:13:47.252765+0000","last_became_active":"2020-08-07T15:12:06.366967+0000","last_became_peered":"2020-08-07T15:12:06.366967+0000","last_unstale":"2020-08-07T15:13:47.252765+0000","last_undegraded":"2020-08-07T15:13:47.252765+0000","last_fullsized":"2020-08-07T15:13:47.252765+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:07.340855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.340855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"308","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.252392+0000","last_change":"2020-08-07T15:13:24.998985+0000","last_active":"2020-08-07T15:13:47.252392+0000","last_peered":"2020-08-07T15:13:47.252392+0000","last_clean":"2020-08-07T15:13:47.252392+0000","last_became_active":"2020-08-07T15:12:32.246800+0000","last_became_peered":"2020-08-07T15:12:32.246800+0000","last_unstale":"2020-08-07T15:13:47.252392+0000","last_undegraded":"2020-08-07T15:13:47.252392+0000","last_fullsized":"2020-08-07T15:13:47.252392+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:24.998919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:24.998919+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.0","version":"298'9324","reported_seq":"9400","reported_epoch":"298","state":"active+clean","last_fresh":"2020-08-07T15:13:51.327684+0000","last_change":"2020-08-07T15:12:10.466261+0000","last_active":"2020-08-07T15:13:51.327684+0000","last_peered":"2020-08-07T15:13:51.327684+0000","last_clean":"2020-08-07T15:13:51.327684+0000","last_became_active":"2020-08-07T15:12:10.465583+0000","last_became_peered":"2020-08-07T15:12:10.465583+0000","last_unstale":"2020-08-07T15:13:51.327684+0000","last_undegraded":"2020-08-07T15:13:51.327684+0000","last_fullsized":"2020-08-07T15:13:51.327684+0000","mapping_epoch":230,"log_start":"298'9321","ondisk_log_start":"298'9321","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9324,"num_write_kb":66304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"271","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.253779+0000","last_change":"2020-08-07T15:13:41.820648+0000","last_active":"2020-08-07T15:13:47.253779+0000","last_peered":"2020-08-07T15:13:47.253779+0000","last_clean":"2020-08-07T15:13:47.253779+0000","last_became_active":"2020-08-07T15:13:02.590901+0000","last_became_peered":"2020-08-07T15:13:02.590901+0000","last_unstale":"2020-08-07T15:13:47.253779+0000","last_undegraded":"2020-08-07T15:13:47.253779+0000","last_fullsized":"2020-08-07T15:13:47.253779+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:41.820573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:13:41.820573+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"393","reported_epoch":"296","state":"active+clean","last_fresh":"2020-08-07T15:13:46.494994+0000","last_change":"2020-08-07T15:12:43.351056+0000","last_active":"2020-08-07T15:13:46.494994+0000","last_peered":"2020-08-07T15:13:46.494994+0000","last_clean":"2020-08-07T15:13:46.494994+0000","last_became_active":"2020-08-07T15:12:32.254842+0000","last_became_peered":"2020-08-07T15:12:32.254842+0000","last_unstale":"2020-08-07T15:13:46.494994+0000","last_undegraded":"2020-08-07T15:13:46.494994+0000","last_fullsized":"2020-08-07T15:13:46.494994+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:43.350971+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:43.350971+0000","last_clean_scrub_stamp":"2020-08-07T15:12:43.350971+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"298'9028","reported_seq":"9105","reported_epoch":"298","state":"active+clean","last_fresh":"2020-08-07T15:13:51.460230+0000","last_change":"2020-08-07T15:12:10.467447+0000","last_active":"2020-08-07T15:13:51.460230+0000","last_peered":"2020-08-07T15:13:51.460230+0000","last_clean":"2020-08-07T15:13:51.460230+0000","last_became_active":"2020-08-07T15:12:10.467000+0000","last_became_peered":"2020-08-07T15:12:10.467000+0000","last_unstale":"2020-08-07T15:13:51.460230+0000","last_undegraded":"2020-08-07T15:13:51.460230+0000","last_fullsized":"2020-08-07T15:13:51.460230+0000","mapping_epoch":230,"log_start":"298'9025","ondisk_log_start":"298'9025","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9028,"num_write_kb":64193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"285","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.252645+0000","last_change":"2020-08-07T15:13:11.097271+0000","last_active":"2020-08-07T15:13:47.252645+0000","last_peered":"2020-08-07T15:13:47.252645+0000","last_clean":"2020-08-07T15:13:47.252645+0000","last_became_active":"2020-08-07T15:10:46.498230+0000","last_became_peered":"2020-08-07T15:10:46.498230+0000","last_unstale":"2020-08-07T15:13:47.252645+0000","last_undegraded":"2020-08-07T15:13:47.252645+0000","last_fullsized":"2020-08-07T15:13:47.252645+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:11.097214+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:13:11.097214+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.f","version":"298'9297","reported_seq":"9376","reported_epoch":"298","state":"active+clean","last_fresh":"2020-08-07T15:13:51.571089+0000","last_change":"2020-08-07T15:13:25.974837+0000","last_active":"2020-08-07T15:13:51.571089+0000","last_peered":"2020-08-07T15:13:51.571089+0000","last_clean":"2020-08-07T15:13:51.571089+0000","last_became_active":"2020-08-07T15:12:10.464640+0000","last_became_peered":"2020-08-07T15:12:10.464640+0000","last_unstale":"2020-08-07T15:13:51.571089+0000","last_undegraded":"2020-08-07T15:13:51.571089+0000","last_fullsized":"2020-08-07T15:13:51.571089+0000","mapping_epoch":230,"log_start":"298'9293","ondisk_log_start":"298'9293","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"286'6920","last_scrub_stamp":"2020-08-07T15:13:25.974765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:13:25.974765+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9297,"num_write_kb":66112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"141","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.252493+0000","last_change":"2020-08-07T15:12:45.243933+0000","last_active":"2020-08-07T15:13:47.252493+0000","last_peered":"2020-08-07T15:13:47.252493+0000","last_clean":"2020-08-07T15:13:47.252493+0000","last_became_active":"2020-08-07T15:12:45.243349+0000","last_became_peered":"2020-08-07T15:12:45.243349+0000","last_unstale":"2020-08-07T15:13:47.252493+0000","last_undegraded":"2020-08-07T15:13:47.252493+0000","last_fullsized":"2020-08-07T15:13:47.252493+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.490655+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.490655+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.e","version":"298'8817","reported_seq":"8895","reported_epoch":"298","state":"active+clean","last_fresh":"2020-08-07T15:13:49.858357+0000","last_change":"2020-08-07T15:13:12.351009+0000","last_active":"2020-08-07T15:13:49.858357+0000","last_peered":"2020-08-07T15:13:49.858357+0000","last_clean":"2020-08-07T15:13:49.858357+0000","last_became_active":"2020-08-07T15:12:10.464900+0000","last_became_peered":"2020-08-07T15:12:10.464900+0000","last_unstale":"2020-08-07T15:13:49.858357+0000","last_undegraded":"2020-08-07T15:13:49.858357+0000","last_fullsized":"2020-08-07T15:13:49.858357+0000","mapping_epoch":230,"log_start":"298'8814","ondisk_log_start":"298'8814","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"275'5310","last_scrub_stamp":"2020-08-07T15:13:12.350955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:13:12.350955+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13172736,"num_objects":201,"num_object_clones":0,"num_object_copies":603,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8817,"num_write_kb":64128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"103","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.257638+0000","last_change":"2020-08-07T15:13:10.415039+0000","last_active":"2020-08-07T15:13:47.257638+0000","last_peered":"2020-08-07T15:13:47.257638+0000","last_clean":"2020-08-07T15:13:47.257638+0000","last_became_active":"2020-08-07T15:13:02.590018+0000","last_became_peered":"2020-08-07T15:13:02.590018+0000","last_unstale":"2020-08-07T15:13:47.257638+0000","last_undegraded":"2020-08-07T15:13:47.257638+0000","last_fullsized":"2020-08-07T15:13:47.257638+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:10.414954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:10.414954+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"297'9207","reported_seq":"11225","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.738481+0000","last_change":"2020-08-07T15:13:39.303407+0000","last_active":"2020-08-07T15:13:47.738481+0000","last_peered":"2020-08-07T15:13:47.738481+0000","last_clean":"2020-08-07T15:13:47.738481+0000","last_became_active":"2020-08-07T15:13:39.273597+0000","last_became_peered":"2020-08-07T15:13:39.273597+0000","last_unstale":"2020-08-07T15:13:47.738481+0000","last_undegraded":"2020-08-07T15:13:47.738481+0000","last_fullsized":"2020-08-07T15:13:47.738481+0000","mapping_epoch":293,"log_start":"297'9204","ondisk_log_start":"297'9204","created":230,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":29491200,"num_objects":450,"num_object_clones":0,"num_object_copies":1350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9207,"num_write_kb":68672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":961,"num_bytes_recovered":62922752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"319","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.267024+0000","last_change":"2020-08-07T15:13:38.089823+0000","last_active":"2020-08-07T15:13:47.267024+0000","last_peered":"2020-08-07T15:13:47.267024+0000","last_clean":"2020-08-07T15:13:47.267024+0000","last_became_active":"2020-08-07T15:12:06.380392+0000","last_became_peered":"2020-08-07T15:12:06.380392+0000","last_unstale":"2020-08-07T15:13:47.267024+0000","last_undegraded":"2020-08-07T15:13:47.267024+0000","last_fullsized":"2020-08-07T15:13:47.267024+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:13:38.089747+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:13:38.089747+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"298'9117","reported_seq":"9196","reported_epoch":"298","state":"active+clean","last_fresh":"2020-08-07T15:13:51.248681+0000","last_change":"2020-08-07T15:13:24.340252+0000","last_active":"2020-08-07T15:13:51.248681+0000","last_peered":"2020-08-07T15:13:51.248681+0000","last_clean":"2020-08-07T15:13:51.248681+0000","last_became_active":"2020-08-07T15:12:10.398332+0000","last_became_peered":"2020-08-07T15:12:10.398332+0000","last_unstale":"2020-08-07T15:13:51.248681+0000","last_undegraded":"2020-08-07T15:13:51.248681+0000","last_fullsized":"2020-08-07T15:13:51.248681+0000","mapping_epoch":230,"log_start":"298'9114","ondisk_log_start":"298'9114","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'6512","last_scrub_stamp":"2020-08-07T15:13:24.340162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:13:24.340162+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9117,"num_write_kb":64832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"271","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.258053+0000","last_change":"2020-08-07T15:12:14.241339+0000","last_active":"2020-08-07T15:13:47.258053+0000","last_peered":"2020-08-07T15:13:47.258053+0000","last_clean":"2020-08-07T15:13:47.258053+0000","last_became_active":"2020-08-07T15:10:46.498051+0000","last_became_peered":"2020-08-07T15:10:46.498051+0000","last_unstale":"2020-08-07T15:13:47.258053+0000","last_undegraded":"2020-08-07T15:13:47.258053+0000","last_fullsized":"2020-08-07T15:13:47.258053+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:14.241262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:14.241262+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"298'9558","reported_seq":"9635","reported_epoch":"298","state":"active+clean","last_fresh":"2020-08-07T15:13:51.266607+0000","last_change":"2020-08-07T15:13:02.590816+0000","last_active":"2020-08-07T15:13:51.266607+0000","last_peered":"2020-08-07T15:13:51.266607+0000","last_clean":"2020-08-07T15:13:51.266607+0000","last_became_active":"2020-08-07T15:13:02.590300+0000","last_became_peered":"2020-08-07T15:13:02.590300+0000","last_unstale":"2020-08-07T15:13:51.266607+0000","last_undegraded":"2020-08-07T15:13:51.266607+0000","last_fullsized":"2020-08-07T15:13:51.266607+0000","mapping_epoch":269,"log_start":"298'9554","ondisk_log_start":"298'9554","created":230,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:12:09.376713+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9558,"num_write_kb":67968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3],"acting":[2,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"346","reported_epoch":"296","state":"active+clean","last_fresh":"2020-08-07T15:13:46.494753+0000","last_change":"2020-08-07T15:12:41.445456+0000","last_active":"2020-08-07T15:13:46.494753+0000","last_peered":"2020-08-07T15:13:46.494753+0000","last_clean":"2020-08-07T15:13:46.494753+0000","last_became_active":"2020-08-07T15:12:32.241722+0000","last_became_peered":"2020-08-07T15:12:32.241722+0000","last_unstale":"2020-08-07T15:13:46.494753+0000","last_undegraded":"2020-08-07T15:13:46.494753+0000","last_fullsized":"2020-08-07T15:13:46.494753+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:41.445379+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_clean_scrub_stamp":"2020-08-07T15:12:41.445379+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"298'9360","reported_seq":"9440","reported_epoch":"298","state":"active+clean","last_fresh":"2020-08-07T15:13:51.268322+0000","last_change":"2020-08-07T15:13:23.229211+0000","last_active":"2020-08-07T15:13:51.268322+0000","last_peered":"2020-08-07T15:13:51.268322+0000","last_clean":"2020-08-07T15:13:51.268322+0000","last_became_active":"2020-08-07T15:13:02.591329+0000","last_became_peered":"2020-08-07T15:13:02.591329+0000","last_unstale":"2020-08-07T15:13:51.268322+0000","last_undegraded":"2020-08-07T15:13:51.268322+0000","last_fullsized":"2020-08-07T15:13:51.268322+0000","mapping_epoch":269,"log_start":"298'9357","ondisk_log_start":"298'9357","created":230,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"284'6440","last_scrub_stamp":"2020-08-07T15:13:23.229131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:13:23.229131+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9360,"num_write_kb":66560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"60","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.258255+0000","last_change":"2020-08-07T15:13:11.459633+0000","last_active":"2020-08-07T15:13:47.258255+0000","last_peered":"2020-08-07T15:13:47.258255+0000","last_clean":"2020-08-07T15:13:47.258255+0000","last_became_active":"2020-08-07T15:13:02.591018+0000","last_became_peered":"2020-08-07T15:13:02.591018+0000","last_unstale":"2020-08-07T15:13:47.258255+0000","last_undegraded":"2020-08-07T15:13:47.258255+0000","last_fullsized":"2020-08-07T15:13:47.258255+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:11.459555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:11.459555+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.c","version":"298'9585","reported_seq":"9664","reported_epoch":"298","state":"active+clean","last_fresh":"2020-08-07T15:13:51.244956+0000","last_change":"2020-08-07T15:13:23.210964+0000","last_active":"2020-08-07T15:13:51.244956+0000","last_peered":"2020-08-07T15:13:51.244956+0000","last_clean":"2020-08-07T15:13:51.244956+0000","last_became_active":"2020-08-07T15:12:10.464698+0000","last_became_peered":"2020-08-07T15:12:10.464698+0000","last_unstale":"2020-08-07T15:13:51.244956+0000","last_undegraded":"2020-08-07T15:13:51.244956+0000","last_fullsized":"2020-08-07T15:13:51.244956+0000","mapping_epoch":230,"log_start":"298'9582","ondisk_log_start":"298'9582","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"284'6848","last_scrub_stamp":"2020-08-07T15:13:23.210886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:12:09.376713+0000","last_clean_scrub_stamp":"2020-08-07T15:13:23.210886+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9585,"num_write_kb":68160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"269","reported_epoch":"297","state":"active+clean","last_fresh":"2020-08-07T15:13:47.252844+0000","last_change":"2020-08-07T15:12:44.203274+0000","last_active":"2020-08-07T15:13:47.252844+0000","last_peered":"2020-08-07T15:13:47.252844+0000","last_clean":"2020-08-07T15:13:47.252844+0000","last_became_active":"2020-08-07T15:12:06.367081+0000","last_became_peered":"2020-08-07T15:12:06.367081+0000","last_unstale":"2020-08-07T15:13:47.252844+0000","last_undegraded":"2020-08-07T15:13:47.252844+0000","last_fullsized":"2020-08-07T15:13:47.252844+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:44.203221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:44.203221+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":121643032,"num_objects":1858,"num_object_clones":0,"num_object_copies":5574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1858,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":148050,"num_write_kb":1065993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":961,"num_bytes_recovered":62922752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":246960128,"data_stored":246947912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":41,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":123,"acting":123,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542236,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1216700,"kb_used_data":168052,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93155140,"statfs":{"total":96636764160,"available":95390863360,"internally_reserved":1073741824,"allocated":172085248,"data_stored":171575860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,1,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":36.792000000000002,"5min":16.657,"15min":6.1669999999999998},"min":{"1min":0.498,"5min":0.371,"15min":0.371},"max":{"1min":753.82299999999998,"5min":856.07100000000003,"15min":856.07100000000003},"last":1.8400000000000001},{"interface":"front","average":{"1min":5.7770000000000001,"5min":1.9390000000000001,"15min":1.2969999999999999},"min":{"1min":0.47099999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":103.30200000000001,"5min":103.30200000000001,"15min":103.30200000000001},"last":0.95499999999999996}]},{"osd":1,"last update":"Fri Aug 7 15:13:10 2020","interfaces":[{"interface":"back","average":{"1min":10.148999999999999,"5min":10.148999999999999,"15min":10.148999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":205.73699999999999,"5min":205.73699999999999,"15min":205.73699999999999},"last":2.6120000000000001},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":3.827,"5min":3.827,"15min":3.827},"last":1.8899999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":15.460000000000001,"5min":4.1349999999999998,"15min":2.1469999999999998},"min":{"1min":0.63900000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":297.721,"5min":297.721,"15min":297.721},"last":2.6440000000000001},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.248,"15min":1.1859999999999999},"min":{"1min":0.66500000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.7809999999999999,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":2.2480000000000002}]},{"osd":3,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":3.081,"5min":22.274000000000001,"15min":8.5120000000000005},"min":{"1min":0.60699999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":32.908000000000001,"5min":917.38699999999994,"15min":917.38699999999994},"last":1.7989999999999999},{"interface":"front","average":{"1min":3.1760000000000002,"5min":5.2690000000000001,"15min":2.6469999999999998},"min":{"1min":0.69499999999999995,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":32.881999999999998,"5min":333.59199999999998,"15min":333.59199999999998},"last":1.2689999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":42.588999999999999,"5min":11.105,"15min":4.5860000000000003},"min":{"1min":0.71399999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":873.673,"5min":873.673,"15min":873.673},"last":2.2160000000000002},{"interface":"front","average":{"1min":1.0660000000000001,"5min":11.461,"15min":4.7009999999999996},"min":{"1min":0.71299999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.571,"5min":866.24099999999999,"15min":866.24099999999999},"last":1.2390000000000001}]},{"osd":5,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":9.1910000000000007,"15min":25.727},"min":{"1min":0.66200000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":3.234,"5min":678.76099999999997,"15min":831.88099999999997},"last":2.2709999999999999},{"interface":"front","average":{"1min":1.538,"5min":1.647,"15min":1.6100000000000001},"min":{"1min":0.85299999999999998,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":3.4340000000000002,"5min":6.6790000000000003,"15min":7.5350000000000001},"last":2.5870000000000002}]},{"osd":7,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":3.9100000000000001,"15min":2.323},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.1840000000000002,"5min":224.14500000000001,"15min":224.14500000000001},"last":1.774},{"interface":"front","average":{"1min":1.512,"5min":1.6830000000000001,"15min":1.593},"min":{"1min":0.91900000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":3.4049999999999998,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":2.8740000000000001}]}]},{"osd":7,"up_from":13,"seq":55834574942,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1051092,"kb_used_data":2508,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93320748,"statfs":{"total":96636764160,"available":95560445952,"internally_reserved":1073741824,"allocated":2568192,"data_stored":2043928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":16.684999999999999,"15min":6.4219999999999997},"min":{"1min":0.61099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.95,"5min":763.39099999999996,"15min":763.39099999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":1.026,"5min":9.7690000000000001,"15min":4.0179999999999998},"min":{"1min":0.57399999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.9970000000000001,"5min":529.88699999999994,"15min":529.88699999999994},"last":0.65700000000000003}]},{"osd":1,"last update":"Fri Aug 7 15:13:05 2020","interfaces":[{"interface":"back","average":{"1min":23.431000000000001,"5min":23.431000000000001,"15min":23.431000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":472.59699999999998,"5min":472.59699999999998,"15min":472.59699999999998},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.8650000000000002,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.2709999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":1.677,"5min":1.776,"15min":1.3899999999999999},"min":{"1min":0.53400000000000003,"5min":0.502,"15min":0.49299999999999999},"max":{"1min":5.806,"5min":33.131999999999998,"15min":33.131999999999998},"last":0.61099999999999999},{"interface":"front","average":{"1min":1.667,"5min":1.6080000000000001,"15min":1.3720000000000001},"min":{"1min":0.70099999999999996,"5min":0.58599999999999997,"15min":0.58399999999999996},"max":{"1min":5.3170000000000002,"5min":32.924999999999997,"15min":32.924999999999997},"last":0.85899999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":9.4909999999999997,"15min":4.1680000000000001},"min":{"1min":0.68200000000000005,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":5.7030000000000003,"5min":698.19899999999996,"15min":698.19899999999996},"last":1.7430000000000001},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.3,"15min":1.3500000000000001},"min":{"1min":0.69699999999999995,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.2909999999999999,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":1.498}]},{"osd":4,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":6.0369999999999999,"15min":3.0960000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.821,"5min":460.78399999999999,"15min":460.78399999999999},"last":1.7809999999999999},{"interface":"front","average":{"1min":38.359000000000002,"5min":8.8149999999999995,"15min":4.0510000000000002},"min":{"1min":0.80900000000000005,"5min":0.621,"15min":0.621},"max":{"1min":668.06799999999998,"5min":668.06799999999998,"15min":668.06799999999998},"last":1.796}]},{"osd":5,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.367,"15min":1.3520000000000001},"min":{"1min":0.67900000000000005,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.0489999999999999,"5min":6.2080000000000002,"15min":6.2080000000000002},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.47,"15min":1.6419999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.6160000000000001,"5min":4.407,"15min":5.327},"last":1.242}]},{"osd":6,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":3.5600000000000001,"5min":9.5960000000000001,"15min":4.1580000000000004},"min":{"1min":0.872,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":40.079999999999998,"5min":695.92899999999997,"15min":695.92899999999997},"last":1.2150000000000001},{"interface":"front","average":{"1min":3.6760000000000002,"5min":1.9910000000000001,"15min":11.757},"min":{"1min":0.91900000000000004,"5min":0.88700000000000001,"15min":0.87},"max":{"1min":41.655999999999999,"5min":41.655999999999999,"15min":266.26999999999998},"last":1.298}]}]},{"osd":1,"up_from":227,"seq":974957576214,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1112336,"kb_used_data":63752,"kb_used_omap":2,"kb_used_meta":1048573,"kb_avail":93259504,"statfs":{"total":96636764160,"available":95497732096,"internally_reserved":1073741824,"allocated":65282048,"data_stored":64753712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":1073739245},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":27.861000000000001,"5min":27.861000000000001,"15min":27.861000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":540.92499999999995,"5min":540.92499999999995,"15min":540.92499999999995},"last":0.59399999999999997},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":3.7069999999999999,"5min":3.7069999999999999,"15min":3.7069999999999999},"last":0.745}]},{"osd":2,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":3.758,"5min":3.758,"15min":3.758},"last":0.80900000000000005}]},{"osd":3,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":13.909000000000001,"5min":13.909000000000001,"15min":13.909000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":250.995,"5min":250.995,"15min":250.995},"last":0.79200000000000004},{"interface":"front","average":{"1min":2.04,"5min":2.04,"15min":2.04},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":6.6580000000000004,"5min":6.6580000000000004,"15min":6.6580000000000004},"last":0.77400000000000002}]},{"osd":4,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":0.82599999999999996}]},{"osd":5,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.498,"5min":1.498,"15min":1.498},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.6720000000000002,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":0.97199999999999998}]},{"osd":6,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.1019999999999999},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":9.4299999999999997,"5min":9.4299999999999997,"15min":9.4299999999999997},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":3.4199999999999999,"5min":3.4199999999999999,"15min":3.4199999999999999},"last":1.0980000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":40.520000000000003,"5min":40.520000000000003,"15min":40.520000000000003},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":778.745,"5min":778.745,"15min":778.745},"last":1.0229999999999999},{"interface":"front","average":{"1min":24.100000000000001,"5min":24.100000000000001,"15min":24.100000000000001},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":442.62799999999999,"5min":442.62799999999999,"15min":442.62799999999999},"last":1.0369999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1103568,"kb_used_data":54984,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93268272,"statfs":{"total":96636764160,"available":95506710528,"internally_reserved":1073741824,"allocated":56303616,"data_stored":55775280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:13:07 2020","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.468,"5min":2.468,"15min":2.468},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.5550000000000002,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.89800000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:13:36 2020","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.0489999999999999,"15min":0.99399999999999999},"min":{"1min":0.47199999999999998,"5min":0.441,"15min":0.441},"max":{"1min":3.6099999999999999,"5min":7.3470000000000004,"15min":7.3470000000000004},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.742,"5min":4.8120000000000003,"15min":21.315000000000001},"min":{"1min":0.64800000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.4909999999999997,"5min":226.911,"15min":606.18499999999995},"last":1.8720000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:13:36 2020","interfaces":[{"interface":"back","average":{"1min":1.615,"5min":6.7089999999999996,"15min":2.8999999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":4.1619999999999999,"5min":431.95299999999997,"15min":431.95299999999997},"last":1.399},{"interface":"front","average":{"1min":1.6359999999999999,"5min":1.4019999999999999,"15min":1.296},"min":{"1min":0.73099999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":4.3449999999999998,"5min":11.19,"15min":11.19},"last":1.4319999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:13:36 2020","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.25,"15min":1.218},"min":{"1min":0.629,"5min":0.501,"15min":0.501},"max":{"1min":2.7610000000000001,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.407,"15min":1.3360000000000001},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":3.5600000000000001,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":1.073}]},{"osd":5,"last update":"Fri Aug 7 15:13:36 2020","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":6.9930000000000003,"15min":5.702},"min":{"1min":0.76800000000000002,"5min":0.62,"15min":0.62},"max":{"1min":2.7400000000000002,"5min":539.31100000000004,"15min":539.31100000000004},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3979999999999999,"15min":1.248},"min":{"1min":0.78700000000000003,"5min":0.752,"15min":0.65900000000000003},"max":{"1min":2.714,"5min":4.0140000000000002,"15min":6.0010000000000003},"last":1.095}]},{"osd":6,"last update":"Fri Aug 7 15:13:36 2020","interfaces":[{"interface":"back","average":{"1min":1.5900000000000001,"5min":1.403,"15min":5.7539999999999996},"min":{"1min":0.90500000000000003,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":3.585,"5min":3.585,"15min":124.738},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5700000000000001,"15min":11.813000000000001},"min":{"1min":0.79200000000000004,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.867,"5min":5.274,"15min":294.94},"last":1.466}]},{"osd":7,"last update":"Fri Aug 7 15:13:36 2020","interfaces":[{"interface":"back","average":{"1min":1.6970000000000001,"5min":1.5349999999999999,"15min":1.5569999999999999},"min":{"1min":0.995,"5min":0.84199999999999997,"15min":0.79300000000000004},"max":{"1min":2.9529999999999998,"5min":4.7169999999999996,"15min":6.468},"last":1.637},{"interface":"front","average":{"1min":1.74,"5min":23.527999999999999,"15min":8.7870000000000008},"min":{"1min":1.097,"5min":0.752,"15min":0.752},"max":{"1min":2.9220000000000002,"5min":971.12699999999995,"15min":971.12699999999995},"last":1.5449999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574942,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1085524,"kb_used_data":36876,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93286316,"statfs":{"total":96636764160,"available":95525187584,"internally_reserved":1073741824,"allocated":37761024,"data_stored":37236760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:28 2020","interfaces":[{"interface":"back","average":{"1min":18.065999999999999,"5min":11.218,"15min":4.4960000000000004},"min":{"1min":0.441,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":377.32900000000001,"5min":501.53800000000001,"15min":501.53800000000001},"last":0.877},{"interface":"front","average":{"1min":1.0760000000000001,"5min":6.649,"15min":2.8860000000000001},"min":{"1min":0.46800000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":3.0139999999999998,"5min":501.36000000000001,"15min":501.36000000000001},"last":0.96699999999999997}]},{"osd":1,"last update":"Fri Aug 7 15:13:07 2020","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":5.7869999999999999,"5min":5.7869999999999999,"15min":5.7869999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":24.699999999999999,"5min":24.699999999999999,"15min":24.699999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":465.69200000000001,"5min":465.69200000000001,"15min":465.69200000000001},"last":0.92500000000000004}]},{"osd":3,"last update":"Fri Aug 7 15:13:28 2020","interfaces":[{"interface":"back","average":{"1min":3.847,"5min":1.696,"15min":1.3580000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":29.111000000000001,"5min":29.111000000000001,"15min":29.111000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":3.3849999999999998,"5min":15.673999999999999,"15min":6.2530000000000001},"min":{"1min":0.627,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":29.788,"5min":501.75599999999997,"15min":501.75599999999997},"last":0.90200000000000002}]},{"osd":4,"last update":"Fri Aug 7 15:13:29 2020","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.397,"15min":1.2869999999999999},"min":{"1min":0.73499999999999999,"5min":0.623,"15min":0.58099999999999996},"max":{"1min":2.5489999999999999,"5min":7.9569999999999999,"15min":19.783000000000001},"last":1.149},{"interface":"front","average":{"1min":44.813000000000002,"5min":10.178000000000001,"15min":4.46},"min":{"1min":0.84299999999999997,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":954.97900000000004,"5min":954.97900000000004,"15min":954.97900000000004},"last":1.165}]},{"osd":5,"last update":"Fri Aug 7 15:13:28 2020","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.3420000000000001,"15min":1.294},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.992,"5min":5.2220000000000004,"15min":5.2220000000000004},"last":1.075},{"interface":"front","average":{"1min":45.744999999999997,"5min":24.559000000000001,"15min":23.603000000000002},"min":{"1min":0.57099999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":971.51400000000001,"5min":971.51400000000001,"15min":971.51400000000001},"last":1.127}]},{"osd":6,"last update":"Fri Aug 7 15:13:39 2020","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":9.766,"15min":4.1550000000000002},"min":{"1min":0.872,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.6480000000000001,"5min":821.91899999999998,"15min":821.91899999999998},"last":1.3580000000000001},{"interface":"front","average":{"1min":25.716999999999999,"5min":6.2949999999999999,"15min":5.8259999999999996},"min":{"1min":0.83799999999999997,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":477.69600000000003,"5min":477.69600000000003,"15min":751.70000000000005},"last":1.321}]},{"osd":7,"last update":"Fri Aug 7 15:13:28 2020","interfaces":[{"interface":"back","average":{"1min":43.765999999999998,"5min":20.567,"15min":8.1370000000000005},"min":{"1min":0.89200000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":930.55499999999995,"5min":930.55499999999995,"15min":930.55499999999995},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.8560000000000001,"5min":1.79,"15min":1.724},"min":{"1min":0.98999999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":5.6950000000000003,"5min":8.0779999999999994,"15min":35.646999999999998},"last":1.3420000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574942,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049420,"kb_used_data":836,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322420,"statfs":{"total":96636764160,"available":95562158080,"internally_reserved":1073741824,"allocated":856064,"data_stored":331800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:38 2020","interfaces":[{"interface":"back","average":{"1min":53.417000000000002,"5min":11.500999999999999,"15min":4.7210000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":929.68899999999996,"5min":929.68899999999996,"15min":929.68899999999996},"last":0.52300000000000002},{"interface":"front","average":{"1min":42.985999999999997,"5min":9.5129999999999999,"15min":4.2489999999999997},"min":{"1min":0.496,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":744.76800000000003,"5min":744.76800000000003,"15min":744.76800000000003},"last":0.55600000000000005}]},{"osd":1,"last update":"Fri Aug 7 15:13:08 2020","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.639,"15min":1.639},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":4.6890000000000001,"5min":4.6890000000000001,"15min":4.6890000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.841,"5min":1.841,"15min":1.841},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":5.0330000000000004,"5min":5.0330000000000004,"15min":5.0330000000000004},"last":0.92900000000000005}]},{"osd":2,"last update":"Fri Aug 7 15:13:38 2020","interfaces":[{"interface":"back","average":{"1min":42.640000000000001,"5min":9.5429999999999993,"15min":3.9860000000000002},"min":{"1min":0.78800000000000003,"5min":0.56999999999999995,"15min":0.56899999999999995},"max":{"1min":810.50999999999999,"5min":810.50999999999999,"15min":810.50999999999999},"last":0.874},{"interface":"front","average":{"1min":1.536,"5min":1.3720000000000001,"15min":1.77},"min":{"1min":0.66100000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.9079999999999999,"5min":6.7889999999999997,"15min":51.530000000000001},"last":0.71299999999999997}]},{"osd":4,"last update":"Fri Aug 7 15:13:38 2020","interfaces":[{"interface":"back","average":{"1min":13.329000000000001,"5min":10.928000000000001,"15min":7.8369999999999997},"min":{"1min":0.66600000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":234.19999999999999,"5min":646.26700000000005,"15min":646.26700000000005},"last":0.94299999999999995},{"interface":"front","average":{"1min":28.314,"5min":15.204000000000001,"15min":26.312000000000001},"min":{"1min":0.82099999999999995,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":519.82600000000002,"5min":844.10500000000002,"15min":844.10500000000002},"last":0.91300000000000003}]},{"osd":5,"last update":"Fri Aug 7 15:13:38 2020","interfaces":[{"interface":"back","average":{"1min":2.5899999999999999,"5min":8.6590000000000007,"15min":3.931},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":5.5170000000000003,"5min":403.88400000000001,"15min":403.88400000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":2.6150000000000002,"5min":1.976,"15min":1.8680000000000001},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":5.5430000000000001,"5min":8.9120000000000008,"15min":8.9120000000000008},"last":0.97899999999999998}]},{"osd":6,"last update":"Fri Aug 7 15:13:38 2020","interfaces":[{"interface":"back","average":{"1min":1.7070000000000001,"5min":1.6499999999999999,"15min":1.601},"min":{"1min":0.76500000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":3.508,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":1.3},{"interface":"front","average":{"1min":1.9139999999999999,"5min":13.398999999999999,"15min":5.79},"min":{"1min":0.97799999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":3.524,"5min":688.15999999999997,"15min":688.15999999999997},"last":1.46}]},{"osd":7,"last update":"Fri Aug 7 15:13:38 2020","interfaces":[{"interface":"back","average":{"1min":1.7250000000000001,"5min":1.5389999999999999,"15min":1.944},"min":{"1min":0.96999999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":3.6120000000000001,"5min":6.8209999999999997,"15min":6.8209999999999997},"last":0.89900000000000002},{"interface":"front","average":{"1min":2.3050000000000002,"5min":1.593,"15min":1.891},"min":{"1min":1.0469999999999999,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":7.0090000000000003,"5min":7.0090000000000003,"15min":8.6790000000000003},"last":1.0620000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574942,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1049420,"kb_used_data":836,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93322420,"statfs":{"total":96636764160,"available":95562158080,"internally_reserved":1073741824,"allocated":856064,"data_stored":331800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:30 2020","interfaces":[{"interface":"back","average":{"1min":3.0019999999999998,"5min":20.065999999999999,"15min":7.4569999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":37.307000000000002,"5min":829.15099999999995,"15min":829.15099999999995},"last":1.0940000000000001},{"interface":"front","average":{"1min":3.242,"5min":2.0179999999999998,"15min":1.272},"min":{"1min":0.499,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":42.095999999999997,"5min":48.274000000000001,"15min":48.274000000000001},"last":0.60399999999999998}]},{"osd":1,"last update":"Fri Aug 7 15:13:08 2020","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.749},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.02}]},{"osd":2,"last update":"Fri Aug 7 15:13:30 2020","interfaces":[{"interface":"back","average":{"1min":3.718,"5min":2.3460000000000001,"15min":1.752},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.52100000000000002},"max":{"1min":37.365000000000002,"5min":49.878999999999998,"15min":49.878999999999998},"last":1.3149999999999999},{"interface":"front","average":{"1min":12.497999999999999,"5min":4.1970000000000001,"15min":2.4260000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":146.34399999999999,"5min":146.34399999999999,"15min":146.34399999999999},"last":1.3400000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:13:30 2020","interfaces":[{"interface":"back","average":{"1min":3.4969999999999999,"5min":15.436999999999999,"15min":6.891},"min":{"1min":0.63100000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":37.582000000000001,"5min":638.92100000000005,"15min":638.92100000000005},"last":1.778},{"interface":"front","average":{"1min":3.4500000000000002,"5min":1.754,"15min":1.4399999999999999},"min":{"1min":0.72299999999999998,"5min":0.68500000000000005,"15min":0.66700000000000004},"max":{"1min":37.956000000000003,"5min":37.956000000000003,"15min":37.956000000000003},"last":0.85399999999999998}]},{"osd":5,"last update":"Fri Aug 7 15:13:30 2020","interfaces":[{"interface":"back","average":{"1min":3.6819999999999999,"5min":19.954000000000001,"15min":10.747},"min":{"1min":0.69999999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":43.692,"5min":862.97299999999996,"15min":990.71600000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":40.039000000000001,"5min":18.669,"15min":7.3150000000000004},"min":{"1min":0.58899999999999997,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":625.35000000000002,"5min":780.00699999999995,"15min":780.00699999999995},"last":0.80100000000000005}]},{"osd":6,"last update":"Fri Aug 7 15:13:38 2020","interfaces":[{"interface":"back","average":{"1min":3.3999999999999999,"5min":2.3260000000000001,"15min":1.9630000000000001},"min":{"1min":0.70799999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":41.585999999999999,"5min":48.575000000000003,"15min":48.575000000000003},"last":0.82399999999999995},{"interface":"front","average":{"1min":3.1400000000000001,"5min":1.657,"15min":12.119},"min":{"1min":0.69399999999999995,"5min":0.64600000000000002,"15min":0.59599999999999997},"max":{"1min":37.991,"5min":37.991,"15min":389.14600000000002},"last":0.88800000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:13:30 2020","interfaces":[{"interface":"back","average":{"1min":3.8620000000000001,"5min":2.4239999999999999,"15min":2.544},"min":{"1min":1.1319999999999999,"5min":0.62,"15min":0.62},"max":{"1min":38.979999999999997,"5min":48.75,"15min":252.798},"last":1.2529999999999999},{"interface":"front","average":{"1min":3.7989999999999999,"5min":6.46,"15min":6.1189999999999998},"min":{"1min":0.91300000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":38.899999999999999,"5min":326.327,"15min":501.93000000000001},"last":1.139}]}]},{"osd":5,"up_from":13,"seq":55834574941,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1052752,"kb_used_data":4168,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93319088,"statfs":{"total":96636764160,"available":95558746112,"internally_reserved":1073741824,"allocated":4268032,"data_stored":3739696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,3,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:27 2020","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":1.4770000000000001,"15min":1.204},"min":{"1min":0.45700000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.7070000000000001,"5min":43.292999999999999,"15min":43.292999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":1.03,"5min":4.3860000000000001,"15min":2.0259999999999998},"min":{"1min":0.47399999999999998,"5min":0.438,"15min":0.438},"max":{"1min":2.6059999999999999,"5min":329.11399999999998,"15min":329.11399999999998},"last":1.294}]},{"osd":1,"last update":"Fri Aug 7 15:13:11 2020","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":4.4039999999999999,"5min":4.4039999999999999,"15min":4.4039999999999999},"last":2.6019999999999999},{"interface":"front","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":4.0810000000000004,"5min":4.0810000000000004,"15min":4.0810000000000004},"last":2.6299999999999999}]},{"osd":2,"last update":"Fri Aug 7 15:13:27 2020","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.6100000000000001,"15min":6.1319999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.4129999999999998,"5min":43.923000000000002,"15min":168.56899999999999},"last":340.91899999999998},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.615,"15min":20.503},"min":{"1min":0.70499999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.5750000000000002,"5min":43.076999999999998,"15min":676.27099999999996},"last":10.103999999999999}]},{"osd":3,"last update":"Fri Aug 7 15:13:27 2020","interfaces":[{"interface":"back","average":{"1min":4.1559999999999997,"5min":2.2570000000000001,"15min":1.5229999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":47.371000000000002,"5min":47.371000000000002,"15min":47.371000000000002},"last":2.8029999999999999},{"interface":"front","average":{"1min":4.0990000000000002,"5min":2.2349999999999999,"15min":4.3440000000000003},"min":{"1min":0.72899999999999998,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":48.338999999999999,"5min":48.338999999999999,"15min":712.98699999999997},"last":3.0539999999999998}]},{"osd":4,"last update":"Fri Aug 7 15:13:27 2020","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":12.773,"15min":34.875999999999998},"min":{"1min":0.68400000000000005,"5min":0.54800000000000004,"15min":0.53400000000000003},"max":{"1min":1.9139999999999999,"5min":971.67999999999995,"15min":971.67999999999995},"last":2.0430000000000001},{"interface":"front","average":{"1min":16.669,"5min":4.3479999999999999,"15min":2.3100000000000001},"min":{"1min":0.78300000000000003,"5min":0.61899999999999999,"15min":0.57099999999999995},"max":{"1min":261.08499999999998,"5min":261.08499999999998,"15min":261.08499999999998},"last":1.466}]},{"osd":6,"last update":"Fri Aug 7 15:13:27 2020","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":28.48,"15min":39.819000000000003},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.4940000000000002,"5min":966.60900000000004,"15min":966.60900000000004},"last":2.6909999999999998},{"interface":"front","average":{"1min":1.3200000000000001,"5min":6.8049999999999997,"15min":17.957999999999998},"min":{"1min":0.71199999999999997,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.9450000000000001,"5min":501.79000000000002,"15min":517.26700000000005},"last":2.0750000000000002}]},{"osd":7,"last update":"Fri Aug 7 15:13:27 2020","interfaces":[{"interface":"back","average":{"1min":13.619999999999999,"5min":3.8769999999999998,"15min":2.1800000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":208.03399999999999,"5min":208.03399999999999,"15min":208.03399999999999},"last":2.6629999999999998},{"interface":"front","average":{"1min":1.4470000000000001,"5min":12.840999999999999,"15min":5.3140000000000001},"min":{"1min":0.89400000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.7069999999999999,"5min":515.09100000000001,"15min":515.09100000000001},"last":1.1859999999999999}]}]}]}} 2020-08-07T15:13:53.433 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:13:53.486 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:53.484+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:53.503 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:53.504 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:13:53.504 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:13:53.504 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:13:53.506 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.506 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4329.0:145801 3.6 e75/26 3:69e4cb22:::benchmark_data_smithi074_35447_object16192:head v 75'9405, mlcod=75'9401)", 2020-08-07T15:13:53.506 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:13:53.506 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 338.82605632399998, 2020-08-07T15:13:53.507 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639, 2020-08-07T15:13:53.508 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:53.508 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:53.508 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:53.508 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.509 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:53.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:13:53.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:53.512 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.512 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.515 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:53.515 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671756+0000", 2020-08-07T15:13:53.515 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6511000000000002e-05 2020-08-07T15:13:53.516 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.516 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.516 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:53.516 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671803+0000", 2020-08-07T15:13:53.518 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7190000000000004e-06 2020-08-07T15:13:53.518 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.518 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.519 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:53.519 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:14.671809+0000", 2020-08-07T15:13:53.519 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74210928300000001 2020-08-07T15:13:53.519 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.519 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:53.520 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413919+0000", 2020-08-07T15:13:53.520 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1759999999999999e-05 2020-08-07T15:13:53.520 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.520 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.521 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:53.521 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.413930+0000", 2020-08-07T15:13:53.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000108601 2020-08-07T15:13:53.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.521 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.526 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.526 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414039+0000", 2020-08-07T15:13:53.526 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9948999999999999e-05 2020-08-07T15:13:53.526 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.527 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.527 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:53.527 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414089+0000", 2020-08-07T15:13:53.527 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00079656499999999995 2020-08-07T15:13:53.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.528 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:53.528 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414885+0000", 2020-08-07T15:13:53.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2250999999999998e-05 2020-08-07T15:13:53.529 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.529 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.529 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:53.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:15.414938+0000", 2020-08-07T15:13:53.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.743181639 2020-08-07T15:13:53.530 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.530 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:53.530 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.530 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.530 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.531 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.4726.0:104104 4.0 e146/131 4:0ffb764a:::benchmark_data_smithi074_44586_object13012:head v 146'6960, mlcod=146'6953)", 2020-08-07T15:13:53.531 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:13:53.531 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 245.299509548, 2020-08-07T15:13:53.531 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75443331599999996, 2020-08-07T15:13:53.531 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:53.531 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:53.532 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:53.532 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.532 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:53.532 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:13:53.532 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:53.533 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.533 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.533 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:53.533 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198303+0000", 2020-08-07T15:13:53.533 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1743000000000002e-05 2020-08-07T15:13:53.534 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.534 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.534 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:53.534 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198355+0000", 2020-08-07T15:13:53.534 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1249999999999998e-06 2020-08-07T15:13:53.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:53.535 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.198361+0000", 2020-08-07T15:13:53.535 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74719179599999996 2020-08-07T15:13:53.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.536 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:53.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945553+0000", 2020-08-07T15:13:53.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6800000000000001e-06 2020-08-07T15:13:53.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.537 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.537 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:53.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945557+0000", 2020-08-07T15:13:53.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012101500000000001 2020-08-07T15:13:53.537 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.538 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.538 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.538 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945678+0000", 2020-08-07T15:13:53.538 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6621000000000001e-05 2020-08-07T15:13:53.538 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.539 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.539 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:53.539 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.945705+0000", 2020-08-07T15:13:53.539 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069879540000000002 2020-08-07T15:13:53.539 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.540 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.540 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:53.540 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.952693+0000", 2020-08-07T15:13:53.540 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3381999999999999e-05 2020-08-07T15:13:53.540 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.540 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.541 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:53.541 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:48.952736+0000", 2020-08-07T15:13:53.541 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75443331599999996 2020-08-07T15:13:53.541 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.541 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:53.542 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.542 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.542 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.542 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89641 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:53.542 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.550933+0000", 2020-08-07T15:13:53.543 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 151.94687912699999, 2020-08-07T15:13:53.543 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69051440900000005, 2020-08-07T15:13:53.543 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:53.543 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:53.543 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:53.544 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:53.544 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:53.544 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89641 2020-08-07T15:13:53.544 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.544 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:53.544 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:53.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550933+0000", 2020-08-07T15:13:53.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:53.547 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.547 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:53.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550933+0000", 2020-08-07T15:13:53.547 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6520000000000002e-05 2020-08-07T15:13:53.548 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.548 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:53.548 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550980+0000", 2020-08-07T15:13:53.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2410000000000003e-06 2020-08-07T15:13:53.548 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.549 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.549 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:53.549 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550985+0000", 2020-08-07T15:13:53.549 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2707e-05 2020-08-07T15:13:53.549 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.550 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.550 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:53.550 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.550998+0000", 2020-08-07T15:13:53.550 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8629999999999996e-06 2020-08-07T15:13:53.550 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.551 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.551 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:53.551 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551004+0000", 2020-08-07T15:13:53.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4987000000000001e-05 2020-08-07T15:13:53.551 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.552 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.552 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.552 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551059+0000", 2020-08-07T15:13:53.552 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1718e-05 2020-08-07T15:13:53.553 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.553 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.553 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:53.553 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551070+0000", 2020-08-07T15:13:53.553 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67720431000000003 2020-08-07T15:13:53.553 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.554 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.554 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.554 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.228275+0000", 2020-08-07T15:13:53.555 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000731965 2020-08-07T15:13:53.555 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.555 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.555 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:53.555 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229007+0000", 2020-08-07T15:13:53.556 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00025396199999999999 2020-08-07T15:13:53.556 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:53.556 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229261+0000", 2020-08-07T15:13:53.557 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002511189 2020-08-07T15:13:53.557 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.557 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.557 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:53.557 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231772+0000", 2020-08-07T15:13:53.558 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3399999999999997e-07 2020-08-07T15:13:53.558 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.558 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.558 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:53.558 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231773+0000", 2020-08-07T15:13:53.559 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0274e-05 2020-08-07T15:13:53.559 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.559 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.559 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:53.559 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231823+0000", 2020-08-07T15:13:53.559 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0094095440000000006 2020-08-07T15:13:53.560 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.560 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.560 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.560 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241233+0000", 2020-08-07T15:13:53.560 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4011999999999998e-05 2020-08-07T15:13:53.561 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.561 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.561 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.561 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241277+0000", 2020-08-07T15:13:53.561 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7199999999999999e-07 2020-08-07T15:13:53.562 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.562 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.562 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:53.562 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241277+0000", 2020-08-07T15:13:53.562 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017061100000000001 2020-08-07T15:13:53.563 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.563 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.563 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:53.563 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241448+0000", 2020-08-07T15:13:53.564 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69051440900000005 2020-08-07T15:13:53.564 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.564 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:53.564 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.564 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.565 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.565 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89642 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:53.565 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.551807+0000", 2020-08-07T15:13:53.565 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 151.94600557499999, 2020-08-07T15:13:53.565 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68991613100000004, 2020-08-07T15:13:53.566 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:53.566 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:53.566 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:53.566 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:53.566 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:53.566 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89642 2020-08-07T15:13:53.567 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.567 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:53.567 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.567 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:53.567 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551807+0000", 2020-08-07T15:13:53.568 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:53.568 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.568 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.568 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:53.568 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551807+0000", 2020-08-07T15:13:53.569 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7049000000000001e-05 2020-08-07T15:13:53.569 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.569 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.569 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:53.569 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551834+0000", 2020-08-07T15:13:53.570 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9249999999999997e-06 2020-08-07T15:13:53.570 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.570 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.570 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:53.570 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551838+0000", 2020-08-07T15:13:53.571 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9160000000000001e-06 2020-08-07T15:13:53.571 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.571 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.571 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:53.571 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551846+0000", 2020-08-07T15:13:53.571 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3510000000000002e-06 2020-08-07T15:13:53.572 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.572 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.572 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:53.572 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551849+0000", 2020-08-07T15:13:53.572 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9276000000000001e-05 2020-08-07T15:13:53.573 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.573 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.573 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.573 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551898+0000", 2020-08-07T15:13:53.573 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3095000000000001e-05 2020-08-07T15:13:53.574 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.574 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.574 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:53.574 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.551911+0000", 2020-08-07T15:13:53.574 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67789159799999998 2020-08-07T15:13:53.575 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.575 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.575 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.575 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229803+0000", 2020-08-07T15:13:53.575 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8069000000000001e-05 2020-08-07T15:13:53.575 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.576 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.576 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:53.576 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.229881+0000", 2020-08-07T15:13:53.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000152917 2020-08-07T15:13:53.576 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.577 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.577 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:53.577 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230034+0000", 2020-08-07T15:13:53.577 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044936760000000003 2020-08-07T15:13:53.577 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.578 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.578 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:53.578 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234528+0000", 2020-08-07T15:13:53.578 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8999999999999995e-07 2020-08-07T15:13:53.578 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.579 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.579 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:53.579 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234529+0000", 2020-08-07T15:13:53.579 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7741000000000002e-05 2020-08-07T15:13:53.579 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.579 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.580 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:53.580 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234586+0000", 2020-08-07T15:13:53.580 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069143959999999997 2020-08-07T15:13:53.580 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.580 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.581 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.581 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241501+0000", 2020-08-07T15:13:53.581 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1433000000000001e-05 2020-08-07T15:13:53.581 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.581 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.582 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.582 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241542+0000", 2020-08-07T15:13:53.582 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7800000000000001e-07 2020-08-07T15:13:53.582 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.582 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.582 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:53.583 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241543+0000", 2020-08-07T15:13:53.583 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000180321 2020-08-07T15:13:53.583 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.583 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.583 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:53.584 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241723+0000", 2020-08-07T15:13:53.584 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68991613100000004 2020-08-07T15:13:53.584 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.584 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:53.584 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.585 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.585 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.585 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89643 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:53.585 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.552181+0000", 2020-08-07T15:13:53.585 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 151.94563158099999, 2020-08-07T15:13:53.586 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68980872400000004, 2020-08-07T15:13:53.593 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:53.594 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:53.594 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:53.594 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:53.594 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:53.594 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89643 2020-08-07T15:13:53.595 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.595 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:53.595 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.595 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:53.595 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552181+0000", 2020-08-07T15:13:53.596 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:53.596 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.597 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.597 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:53.597 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552181+0000", 2020-08-07T15:13:53.597 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0926999999999997e-05 2020-08-07T15:13:53.598 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.598 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.598 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:53.598 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552212+0000", 2020-08-07T15:13:53.598 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0512e-05 2020-08-07T15:13:53.598 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.599 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.599 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:53.599 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552222+0000", 2020-08-07T15:13:53.599 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9379999999999992e-06 2020-08-07T15:13:53.599 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.600 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.600 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:53.600 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552230+0000", 2020-08-07T15:13:53.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0260000000000001e-06 2020-08-07T15:13:53.600 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.601 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:53.601 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552234+0000", 2020-08-07T15:13:53.601 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6109000000000002e-05 2020-08-07T15:13:53.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.602 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.602 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.602 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552280+0000", 2020-08-07T15:13:53.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5318000000000001e-05 2020-08-07T15:13:53.602 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.603 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:53.603 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552296+0000", 2020-08-07T15:13:53.603 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67811269100000005 2020-08-07T15:13:53.603 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230408+0000", 2020-08-07T15:13:53.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000125518 2020-08-07T15:13:53.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.605 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:53.605 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230534+0000", 2020-08-07T15:13:53.605 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024637999999999998 2020-08-07T15:13:53.605 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.605 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.606 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:53.606 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.230780+0000", 2020-08-07T15:13:53.606 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0066398869999999997 2020-08-07T15:13:53.606 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.606 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.607 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:53.607 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237420+0000", 2020-08-07T15:13:53.607 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3500000000000005e-07 2020-08-07T15:13:53.607 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.607 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.608 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:53.608 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237421+0000", 2020-08-07T15:13:53.608 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4005000000000003e-05 2020-08-07T15:13:53.608 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.608 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.609 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:53.609 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237465+0000", 2020-08-07T15:13:53.609 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043133529999999998 2020-08-07T15:13:53.609 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.609 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.610 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.610 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241778+0000", 2020-08-07T15:13:53.610 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0592e-05 2020-08-07T15:13:53.610 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.610 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.611 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.611 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241819+0000", 2020-08-07T15:13:53.611 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4999999999999998e-07 2020-08-07T15:13:53.611 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.612 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.612 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:53.612 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241819+0000", 2020-08-07T15:13:53.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017018299999999999 2020-08-07T15:13:53.612 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.613 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:53.613 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.241990+0000", 2020-08-07T15:13:53.613 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68980872400000004 2020-08-07T15:13:53.613 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.613 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:53.614 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.614 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.614 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.614 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89644 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:53.614 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.552384+0000", 2020-08-07T15:13:53.615 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 151.94542854700001, 2020-08-07T15:13:53.615 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68997479299999998, 2020-08-07T15:13:53.615 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:53.615 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:53.615 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:53.616 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:53.616 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:53.616 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89644 2020-08-07T15:13:53.616 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.616 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:53.617 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.617 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:53.617 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552384+0000", 2020-08-07T15:13:53.617 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:53.617 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.618 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.618 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:53.618 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552384+0000", 2020-08-07T15:13:53.618 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9238999999999999e-05 2020-08-07T15:13:53.618 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.619 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.619 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:53.619 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552413+0000", 2020-08-07T15:13:53.619 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3340000000000003e-06 2020-08-07T15:13:53.619 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.619 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.620 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:53.620 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552417+0000", 2020-08-07T15:13:53.620 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3119999999999994e-06 2020-08-07T15:13:53.620 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.620 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.621 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:53.621 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552426+0000", 2020-08-07T15:13:53.621 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4139999999999996e-06 2020-08-07T15:13:53.621 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.621 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.622 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:53.622 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552430+0000", 2020-08-07T15:13:53.622 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6613999999999999e-05 2020-08-07T15:13:53.622 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.622 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.623 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.623 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:13:53.623 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7280000000000007e-06 2020-08-07T15:13:53.623 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.623 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.623 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:53.624 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552477+0000", 2020-08-07T15:13:53.624 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67873990399999995 2020-08-07T15:13:53.624 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.624 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.624 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.625 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231216+0000", 2020-08-07T15:13:53.625 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3629999999999998e-05 2020-08-07T15:13:53.625 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.625 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.625 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:53.626 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231290+0000", 2020-08-07T15:13:53.626 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000132108 2020-08-07T15:13:53.626 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.626 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.626 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:53.627 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231422+0000", 2020-08-07T15:13:53.627 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006049652 2020-08-07T15:13:53.627 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.627 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:53.627 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237472+0000", 2020-08-07T15:13:53.628 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000005e-07 2020-08-07T15:13:53.628 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.628 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.628 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:53.628 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237472+0000", 2020-08-07T15:13:53.629 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2848000000000001e-05 2020-08-07T15:13:53.629 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.629 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.629 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:53.629 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237505+0000", 2020-08-07T15:13:53.630 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045352179999999997 2020-08-07T15:13:53.630 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.630 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.630 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.630 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242040+0000", 2020-08-07T15:13:53.630 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012843699999999999 2020-08-07T15:13:53.631 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.631 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.631 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.631 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242169+0000", 2020-08-07T15:13:53.631 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9999999999999998e-07 2020-08-07T15:13:53.632 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.632 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.632 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:53.632 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242169+0000", 2020-08-07T15:13:53.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000189354 2020-08-07T15:13:53.633 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.633 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.633 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:53.633 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242359+0000", 2020-08-07T15:13:53.633 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68997479299999998 2020-08-07T15:13:53.634 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.634 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:53.634 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.634 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.634 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.634 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89645 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:53.635 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:13:53.635 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 151.94534576500001, 2020-08-07T15:13:53.635 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69011194600000003, 2020-08-07T15:13:53.635 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:53.635 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:53.636 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:53.636 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:53.636 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:53.636 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89645 2020-08-07T15:13:53.636 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.637 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:53.637 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.637 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:53.637 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:13:53.637 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:53.638 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.638 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.638 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:53.638 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552467+0000", 2020-08-07T15:13:53.638 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6750999999999998e-05 2020-08-07T15:13:53.639 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.639 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.639 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:53.639 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552493+0000", 2020-08-07T15:13:53.639 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0350000000000002e-06 2020-08-07T15:13:53.639 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.640 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.640 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:53.640 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552496+0000", 2020-08-07T15:13:53.640 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3029999999999996e-06 2020-08-07T15:13:53.640 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.641 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.641 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:53.641 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552503+0000", 2020-08-07T15:13:53.641 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7580000000000001e-06 2020-08-07T15:13:53.641 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.642 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.642 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:53.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552506+0000", 2020-08-07T15:13:53.642 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4868000000000003e-05 2020-08-07T15:13:53.642 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.643 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.643 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.643 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552570+0000", 2020-08-07T15:13:53.643 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9649999999999997e-06 2020-08-07T15:13:53.643 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.643 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.644 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:53.644 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.552579+0000", 2020-08-07T15:13:53.644 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67928357699999997 2020-08-07T15:13:53.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.644 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.645 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.645 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231863+0000", 2020-08-07T15:13:53.645 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000115696 2020-08-07T15:13:53.645 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.645 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.646 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:53.646 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.231979+0000", 2020-08-07T15:13:53.646 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020456599999999999 2020-08-07T15:13:53.646 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.646 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.647 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:53.647 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232183+0000", 2020-08-07T15:13:53.647 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005327923 2020-08-07T15:13:53.647 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.647 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.647 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:53.648 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237511+0000", 2020-08-07T15:13:53.648 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8999999999999998e-07 2020-08-07T15:13:53.648 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.648 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.648 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:53.649 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237511+0000", 2020-08-07T15:13:53.649 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6019000000000002e-05 2020-08-07T15:13:53.649 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.649 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.649 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:53.650 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.237547+0000", 2020-08-07T15:13:53.650 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045340980000000003 2020-08-07T15:13:53.650 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.650 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.650 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.651 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242082+0000", 2020-08-07T15:13:53.651 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00032620499999999998 2020-08-07T15:13:53.651 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.651 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.651 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.651 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242408+0000", 2020-08-07T15:13:53.652 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3500000000000002e-07 2020-08-07T15:13:53.652 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.652 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.652 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:53.671 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242408+0000", 2020-08-07T15:13:53.672 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000170457 2020-08-07T15:13:53.672 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.672 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.673 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:53.673 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242579+0000", 2020-08-07T15:13:53.673 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.69011194600000003 2020-08-07T15:13:53.673 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.673 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:53.674 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.674 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.674 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.674 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89646 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:53.674 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.557244+0000", 2020-08-07T15:13:53.675 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 151.940567993, 2020-08-07T15:13:53.675 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68559352299999998, 2020-08-07T15:13:53.675 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:53.675 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:53.675 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:53.677 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:53.678 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:53.678 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89646 2020-08-07T15:13:53.678 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.678 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:53.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.679 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:53.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557244+0000", 2020-08-07T15:13:53.679 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:53.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.680 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:53.680 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557244+0000", 2020-08-07T15:13:53.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9699000000000001e-05 2020-08-07T15:13:53.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:53.681 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557274+0000", 2020-08-07T15:13:53.681 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1269999999999996e-06 2020-08-07T15:13:53.681 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.681 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.688 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:53.688 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557278+0000", 2020-08-07T15:13:53.688 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0099999999999995e-06 2020-08-07T15:13:53.689 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.689 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.689 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:53.690 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557286+0000", 2020-08-07T15:13:53.690 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7439999999999998e-06 2020-08-07T15:13:53.690 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.691 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.691 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:53.691 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557291+0000", 2020-08-07T15:13:53.691 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5927999999999999e-05 2020-08-07T15:13:53.691 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.692 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.692 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557327+0000", 2020-08-07T15:13:53.692 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5069999999999997e-06 2020-08-07T15:13:53.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.693 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.693 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:53.693 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557333+0000", 2020-08-07T15:13:53.693 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67527419200000005 2020-08-07T15:13:53.693 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.694 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.694 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.694 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232608+0000", 2020-08-07T15:13:53.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8699e-05 2020-08-07T15:13:53.695 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.695 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:53.696 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232686+0000", 2020-08-07T15:13:53.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018185099999999999 2020-08-07T15:13:53.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.696 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:53.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.232868+0000", 2020-08-07T15:13:53.697 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0075819360000000001 2020-08-07T15:13:53.697 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:53.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240450+0000", 2020-08-07T15:13:53.698 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2799999999999995e-07 2020-08-07T15:13:53.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.698 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.698 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:53.698 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240451+0000", 2020-08-07T15:13:53.699 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3751000000000002e-05 2020-08-07T15:13:53.699 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.699 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.699 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:53.699 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240495+0000", 2020-08-07T15:13:53.700 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001628064 2020-08-07T15:13:53.700 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.700 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.700 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242123+0000", 2020-08-07T15:13:53.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00050955599999999998 2020-08-07T15:13:53.701 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.701 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.701 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242632+0000", 2020-08-07T15:13:53.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2900000000000004e-07 2020-08-07T15:13:53.702 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.702 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.702 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:53.702 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242633+0000", 2020-08-07T15:13:53.703 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000205202 2020-08-07T15:13:53.703 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.703 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:53.703 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242838+0000", 2020-08-07T15:13:53.704 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68559352299999998 2020-08-07T15:13:53.704 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.704 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:53.704 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.705 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.705 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89647 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:53.705 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.557315+0000", 2020-08-07T15:13:53.705 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 151.94049767000001, 2020-08-07T15:13:53.706 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68581815800000001, 2020-08-07T15:13:53.706 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:53.706 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:53.706 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:53.706 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:53.707 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:53.707 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89647 2020-08-07T15:13:53.707 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.707 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:53.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.708 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:53.708 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557315+0000", 2020-08-07T15:13:53.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:53.708 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.708 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:53.709 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557315+0000", 2020-08-07T15:13:53.709 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6639999999999999e-05 2020-08-07T15:13:53.709 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.709 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.710 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:53.710 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557341+0000", 2020-08-07T15:13:53.710 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7989999999999998e-06 2020-08-07T15:13:53.710 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.711 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.711 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:53.711 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557344+0000", 2020-08-07T15:13:53.712 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6960000000000002e-06 2020-08-07T15:13:53.712 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.713 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.713 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:53.713 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557350+0000", 2020-08-07T15:13:53.713 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6089999999999999e-06 2020-08-07T15:13:53.714 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.714 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.714 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:53.714 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557352+0000", 2020-08-07T15:13:53.715 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1837e-05 2020-08-07T15:13:53.715 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.715 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.715 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.715 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557384+0000", 2020-08-07T15:13:53.716 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8099999999999998e-06 2020-08-07T15:13:53.716 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.716 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.716 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:53.716 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557392+0000", 2020-08-07T15:13:53.717 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67581117300000004 2020-08-07T15:13:53.717 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.717 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.717 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.717 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233203+0000", 2020-08-07T15:13:53.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112316 2020-08-07T15:13:53.720 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:53.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233316+0000", 2020-08-07T15:13:53.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000221766 2020-08-07T15:13:53.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.723 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:53.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233537+0000", 2020-08-07T15:13:53.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069695019999999998 2020-08-07T15:13:53.724 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.725 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:53.725 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240507+0000", 2020-08-07T15:13:53.726 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3999999999999997e-07 2020-08-07T15:13:53.726 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.727 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:53.728 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240507+0000", 2020-08-07T15:13:53.729 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8844000000000003e-05 2020-08-07T15:13:53.729 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.729 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.730 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:53.730 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240546+0000", 2020-08-07T15:13:53.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023473399999999998 2020-08-07T15:13:53.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.732 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242893+0000", 2020-08-07T15:13:53.732 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5410000000000001e-05 2020-08-07T15:13:53.733 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.733 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.734 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242939+0000", 2020-08-07T15:13:53.734 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8000000000000001e-07 2020-08-07T15:13:53.734 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.735 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.735 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:53.735 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.242939+0000", 2020-08-07T15:13:53.736 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019369600000000001 2020-08-07T15:13:53.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.737 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.737 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:53.738 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243133+0000", 2020-08-07T15:13:53.739 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68581815800000001 2020-08-07T15:13:53.739 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.739 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:53.740 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.740 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.740 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.741 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89648 5.7 5:e9d7653d:::benchmark_data_smithi074_53697_object11205:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:53.741 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.557372+0000", 2020-08-07T15:13:53.741 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 151.94044054899999, 2020-08-07T15:13:53.742 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68606040000000001, 2020-08-07T15:13:53.742 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:53.743 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:53.743 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:53.743 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:53.744 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:53.744 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89648 2020-08-07T15:13:53.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.745 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:53.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.745 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:53.746 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557372+0000", 2020-08-07T15:13:53.746 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:53.746 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.747 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.747 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:53.747 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557372+0000", 2020-08-07T15:13:53.748 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5534e-05 2020-08-07T15:13:53.748 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.749 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.750 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:53.750 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557397+0000", 2020-08-07T15:13:53.750 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.599e-06 2020-08-07T15:13:53.751 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.751 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.751 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:53.752 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557400+0000", 2020-08-07T15:13:53.752 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.288e-06 2020-08-07T15:13:53.752 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.753 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.753 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:53.753 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557405+0000", 2020-08-07T15:13:53.754 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.142e-06 2020-08-07T15:13:53.754 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.754 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:53.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557407+0000", 2020-08-07T15:13:53.755 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0954999999999998e-05 2020-08-07T15:13:53.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.757 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557428+0000", 2020-08-07T15:13:53.757 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7529999999999998e-06 2020-08-07T15:13:53.757 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.758 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.758 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:53.758 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.557434+0000", 2020-08-07T15:13:53.759 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.67655471099999998 2020-08-07T15:13:53.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.760 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.760 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.760 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.233989+0000", 2020-08-07T15:13:53.760 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8301999999999997e-05 2020-08-07T15:13:53.760 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.761 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:53.761 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234067+0000", 2020-08-07T15:13:53.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000129908 2020-08-07T15:13:53.761 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.762 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.762 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:53.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234197+0000", 2020-08-07T15:13:53.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063551249999999997 2020-08-07T15:13:53.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.763 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:53.763 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240552+0000", 2020-08-07T15:13:53.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9400000000000001e-07 2020-08-07T15:13:53.763 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.764 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:53.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240552+0000", 2020-08-07T15:13:53.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2787999999999998e-05 2020-08-07T15:13:53.764 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.765 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:53.765 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240585+0000", 2020-08-07T15:13:53.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002605652 2020-08-07T15:13:53.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.766 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.766 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243191+0000", 2020-08-07T15:13:53.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8690000000000002e-05 2020-08-07T15:13:53.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.768 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.768 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243250+0000", 2020-08-07T15:13:53.768 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4700000000000001e-07 2020-08-07T15:13:53.769 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.769 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.770 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:53.770 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243250+0000", 2020-08-07T15:13:53.771 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000181912 2020-08-07T15:13:53.771 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.771 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.772 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:53.772 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243432+0000", 2020-08-07T15:13:53.772 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68606040000000001 2020-08-07T15:13:53.773 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.773 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:53.774 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.774 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.774 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.775 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:89689 5.7 5:e83ee43b:::benchmark_data_smithi074_53697_object11211:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:53.775 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:21.585493+0000", 2020-08-07T15:13:53.775 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 151.912319731, 2020-08-07T15:13:53.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66549445299999999, 2020-08-07T15:13:53.776 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:53.776 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:53.777 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:53.777 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:53.777 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:53.777 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 89689 2020-08-07T15:13:53.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.778 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:53.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:53.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585493+0000", 2020-08-07T15:13:53.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:53.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.779 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:53.779 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585493+0000", 2020-08-07T15:13:53.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7746000000000002e-05 2020-08-07T15:13:53.780 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.781 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:53.781 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585520+0000", 2020-08-07T15:13:53.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1099999999999996e-06 2020-08-07T15:13:53.782 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.782 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.782 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:53.782 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585525+0000", 2020-08-07T15:13:53.783 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5359999999999998e-06 2020-08-07T15:13:53.783 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:53.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585534+0000", 2020-08-07T15:13:53.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5040000000000002e-06 2020-08-07T15:13:53.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:53.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585540+0000", 2020-08-07T15:13:53.785 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3548999999999998e-05 2020-08-07T15:13:53.785 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585593+0000", 2020-08-07T15:13:53.786 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2014000000000001e-05 2020-08-07T15:13:53.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.786 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:53.786 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:21.585605+0000", 2020-08-07T15:13:53.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.64901847700000004 2020-08-07T15:13:53.787 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.787 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.787 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234624+0000", 2020-08-07T15:13:53.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000176329 2020-08-07T15:13:53.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.788 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:53.788 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.234800+0000", 2020-08-07T15:13:53.788 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00025457300000000002 2020-08-07T15:13:53.789 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.789 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.789 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:53.789 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.235055+0000", 2020-08-07T15:13:53.789 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0055364669999999998 2020-08-07T15:13:53.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:53.800 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240591+0000", 2020-08-07T15:13:53.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3700000000000001e-07 2020-08-07T15:13:53.801 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.801 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.801 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:53.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240591+0000", 2020-08-07T15:13:53.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8479999999999997e-05 2020-08-07T15:13:53.802 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.802 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.802 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:53.802 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.240640+0000", 2020-08-07T15:13:53.802 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00285564 2020-08-07T15:13:53.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.243496+0000", 2020-08-07T15:13:53.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0073477400000000002 2020-08-07T15:13:53.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.250843+0000", 2020-08-07T15:13:53.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7300000000000005e-07 2020-08-07T15:13:53.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:53.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.250844+0000", 2020-08-07T15:13:53.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014307799999999999 2020-08-07T15:13:53.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:53.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:22.250987+0000", 2020-08-07T15:13:53.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.66549445299999999 2020-08-07T15:13:53.807 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.807 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:53.807 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.808 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103152 5.7 5:e865d79a:::benchmark_data_smithi074_53697_object12893:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e211)", 2020-08-07T15:13:53.808 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.399967+0000", 2020-08-07T15:13:53.808 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 142.097845842, 2020-08-07T15:13:53.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81348256399999996, 2020-08-07T15:13:53.808 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:53.809 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:13:53.809 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:53.810 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:53.810 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:53.810 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103152 2020-08-07T15:13:53.810 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.811 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:53.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.811 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:53.811 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.399967+0000", 2020-08-07T15:13:53.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:53.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:53.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.399967+0000", 2020-08-07T15:13:53.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017528899999999999 2020-08-07T15:13:53.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.813 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:53.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400142+0000", 2020-08-07T15:13:53.813 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8239999999999996e-06 2020-08-07T15:13:53.814 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.814 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.814 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:53.814 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400148+0000", 2020-08-07T15:13:53.814 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0251000000000001e-05 2020-08-07T15:13:53.814 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.815 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:53.815 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400158+0000", 2020-08-07T15:13:53.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1579999999999998e-06 2020-08-07T15:13:53.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:53.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400162+0000", 2020-08-07T15:13:53.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00078404699999999996 2020-08-07T15:13:53.816 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.817 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.400946+0000", 2020-08-07T15:13:53.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010605800000000001 2020-08-07T15:13:53.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.818 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.818 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:53.818 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.401052+0000", 2020-08-07T15:13:53.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020987499999999999 2020-08-07T15:13:53.818 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:53.819 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.401262+0000", 2020-08-07T15:13:53.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0023707820000000001 2020-08-07T15:13:53.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:53.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.403633+0000", 2020-08-07T15:13:53.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.023e-06 2020-08-07T15:13:53.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:53.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.403634+0000", 2020-08-07T15:13:53.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3560999999999995e-05 2020-08-07T15:13:53.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:53.826 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.403717+0000", 2020-08-07T15:13:53.826 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0011669689999999999 2020-08-07T15:13:53.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.404884+0000", 2020-08-07T15:13:53.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033332850000000001 2020-08-07T15:13:53.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.827 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.828 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.408218+0000", 2020-08-07T15:13:53.828 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0268000000000001e-05 2020-08-07T15:13:53.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.829 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:53.829 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.408228+0000", 2020-08-07T15:13:53.829 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.80508180500000004 2020-08-07T15:13:53.829 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.830 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.830 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.831 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213310+0000", 2020-08-07T15:13:53.831 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000139369 2020-08-07T15:13:53.831 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.831 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.832 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:53.832 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213449+0000", 2020-08-07T15:13:53.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81348256399999996 2020-08-07T15:13:53.834 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.834 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:53.834 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.835 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103265 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:13:53.835 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478593+0000", 2020-08-07T15:13:53.835 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 142.01921978600001, 2020-08-07T15:13:53.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74922190499999997, 2020-08-07T15:13:53.836 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:53.836 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:53.836 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:53.836 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:53.837 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:53.837 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103265 2020-08-07T15:13:53.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.837 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:53.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.838 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:53.838 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478593+0000", 2020-08-07T15:13:53.838 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:53.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.839 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:53.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478593+0000", 2020-08-07T15:13:53.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9968000000000001e-05 2020-08-07T15:13:53.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.840 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:53.840 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478633+0000", 2020-08-07T15:13:53.840 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1420000000000003e-06 2020-08-07T15:13:53.840 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.840 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:53.841 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478637+0000", 2020-08-07T15:13:53.841 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2919999999999996e-06 2020-08-07T15:13:53.841 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.842 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:53.842 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478645+0000", 2020-08-07T15:13:53.842 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3699999999999997e-06 2020-08-07T15:13:53.842 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.842 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.842 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:53.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478649+0000", 2020-08-07T15:13:53.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1440000000000002e-05 2020-08-07T15:13:53.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.844 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.844 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478701+0000", 2020-08-07T15:13:53.844 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0831e-05 2020-08-07T15:13:53.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:53.845 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478712+0000", 2020-08-07T15:13:53.845 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73476696699999999 2020-08-07T15:13:53.845 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.846 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.846 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.846 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213479+0000", 2020-08-07T15:13:53.846 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000167026 2020-08-07T15:13:53.847 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.847 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.847 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:53.847 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213646+0000", 2020-08-07T15:13:53.848 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000179557 2020-08-07T15:13:53.848 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.848 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.848 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:53.848 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.213825+0000", 2020-08-07T15:13:53.849 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041300360000000001 2020-08-07T15:13:53.849 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.849 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.849 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:53.849 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217955+0000", 2020-08-07T15:13:53.849 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6840000000000001e-06 2020-08-07T15:13:53.850 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.850 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:53.850 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217957+0000", 2020-08-07T15:13:53.850 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4758e-05 2020-08-07T15:13:53.851 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.851 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.851 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:53.851 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218012+0000", 2020-08-07T15:13:53.851 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0095519990000000003 2020-08-07T15:13:53.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.852 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227564+0000", 2020-08-07T15:13:53.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9956e-05 2020-08-07T15:13:53.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.853 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.853 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227624+0000", 2020-08-07T15:13:53.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3790000000000001e-06 2020-08-07T15:13:53.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.855 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:53.855 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227625+0000", 2020-08-07T15:13:53.855 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0001895 2020-08-07T15:13:53.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.856 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.856 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:53.856 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227815+0000", 2020-08-07T15:13:53.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74922190499999997 2020-08-07T15:13:53.856 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.857 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:53.857 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.857 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103266 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:13:53.858 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478693+0000", 2020-08-07T15:13:53.858 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 142.019119445, 2020-08-07T15:13:53.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74942137399999997, 2020-08-07T15:13:53.858 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:53.858 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:53.859 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:53.859 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:53.859 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:53.859 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103266 2020-08-07T15:13:53.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.860 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:53.860 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:53.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478693+0000", 2020-08-07T15:13:53.860 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:53.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.861 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:53.862 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478693+0000", 2020-08-07T15:13:53.862 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5658000000000002e-05 2020-08-07T15:13:53.862 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:53.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478739+0000", 2020-08-07T15:13:53.863 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.113e-06 2020-08-07T15:13:53.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.864 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:53.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478744+0000", 2020-08-07T15:13:53.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9690000000000006e-06 2020-08-07T15:13:53.864 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.865 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:53.865 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478754+0000", 2020-08-07T15:13:53.865 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1849999999999997e-06 2020-08-07T15:13:53.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.866 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.866 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:53.866 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478758+0000", 2020-08-07T15:13:53.866 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7024999999999998e-05 2020-08-07T15:13:53.866 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.866 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.867 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.867 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478805+0000", 2020-08-07T15:13:53.867 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1357e-05 2020-08-07T15:13:53.867 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.867 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.868 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:53.868 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478816+0000", 2020-08-07T15:13:53.868 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73554277499999998 2020-08-07T15:13:53.868 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.868 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.869 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.869 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.214359+0000", 2020-08-07T15:13:53.869 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000129145 2020-08-07T15:13:53.869 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.869 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.870 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:53.870 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.214488+0000", 2020-08-07T15:13:53.870 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00027129999999999998 2020-08-07T15:13:53.870 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.870 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.871 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:53.871 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.214760+0000", 2020-08-07T15:13:53.871 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032631079999999998 2020-08-07T15:13:53.871 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.871 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:53.872 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218023+0000", 2020-08-07T15:13:53.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1000000000000001e-06 2020-08-07T15:13:53.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.872 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.872 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:53.873 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218024+0000", 2020-08-07T15:13:53.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9085000000000006e-05 2020-08-07T15:13:53.873 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.873 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:53.874 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218103+0000", 2020-08-07T15:13:53.874 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.009779088 2020-08-07T15:13:53.874 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.874 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.874 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.875 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227882+0000", 2020-08-07T15:13:53.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1989999999999994e-05 2020-08-07T15:13:53.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.875 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227944+0000", 2020-08-07T15:13:53.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8299999999999996e-07 2020-08-07T15:13:53.876 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.876 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:53.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.227945+0000", 2020-08-07T15:13:53.877 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000169693 2020-08-07T15:13:53.877 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.877 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.877 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:53.877 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228114+0000", 2020-08-07T15:13:53.878 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74942137399999997 2020-08-07T15:13:53.878 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.878 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:53.878 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.878 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.879 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103267 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:13:53.879 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478794+0000", 2020-08-07T15:13:53.879 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 142.01901795699999, 2020-08-07T15:13:53.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74960417300000004, 2020-08-07T15:13:53.880 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:53.880 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:53.880 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:53.880 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:53.880 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:53.880 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103267 2020-08-07T15:13:53.881 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.881 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:53.881 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.881 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:53.881 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478794+0000", 2020-08-07T15:13:53.882 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:53.882 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.882 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.882 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:53.882 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478794+0000", 2020-08-07T15:13:53.883 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6646e-05 2020-08-07T15:13:53.883 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.883 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.883 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:53.883 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478841+0000", 2020-08-07T15:13:53.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:13:53.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.884 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.884 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:53.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478846+0000", 2020-08-07T15:13:53.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0171e-05 2020-08-07T15:13:53.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:53.885 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478856+0000", 2020-08-07T15:13:53.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5929999999999999e-06 2020-08-07T15:13:53.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.886 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.886 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:53.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478861+0000", 2020-08-07T15:13:53.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2032e-05 2020-08-07T15:13:53.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.887 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478903+0000", 2020-08-07T15:13:53.887 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0312000000000001e-05 2020-08-07T15:13:53.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:53.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478913+0000", 2020-08-07T15:13:53.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73634238600000002 2020-08-07T15:13:53.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.215256+0000", 2020-08-07T15:13:53.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000124933 2020-08-07T15:13:53.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.890 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:53.890 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.215380+0000", 2020-08-07T15:13:53.890 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024624 2020-08-07T15:13:53.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.891 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.891 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:53.891 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.215627+0000", 2020-08-07T15:13:53.891 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0040879540000000004 2020-08-07T15:13:53.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.892 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.892 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:53.892 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219715+0000", 2020-08-07T15:13:53.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9230000000000001e-06 2020-08-07T15:13:53.892 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.892 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.893 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:53.893 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219717+0000", 2020-08-07T15:13:53.893 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8881999999999995e-05 2020-08-07T15:13:53.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.894 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:53.894 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219785+0000", 2020-08-07T15:13:53.894 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0083925119999999995 2020-08-07T15:13:53.894 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.894 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.895 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.895 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228178+0000", 2020-08-07T15:13:53.895 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2254000000000004e-05 2020-08-07T15:13:53.895 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.895 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.896 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.896 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228240+0000", 2020-08-07T15:13:53.896 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4499999999999996e-07 2020-08-07T15:13:53.896 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.897 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:53.897 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228241+0000", 2020-08-07T15:13:53.897 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000157656 2020-08-07T15:13:53.897 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.897 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.898 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:53.898 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228399+0000", 2020-08-07T15:13:53.898 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74960417300000004 2020-08-07T15:13:53.898 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.899 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:53.899 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.899 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.899 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.900 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103268 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:13:53.900 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478898+0000", 2020-08-07T15:13:53.900 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 142.01891423999999, 2020-08-07T15:13:53.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74981275700000005, 2020-08-07T15:13:53.901 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:53.901 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:53.901 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:53.901 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:53.901 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:53.902 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103268 2020-08-07T15:13:53.902 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.902 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:53.902 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.903 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:53.903 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478898+0000", 2020-08-07T15:13:53.903 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:53.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.904 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:53.904 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478898+0000", 2020-08-07T15:13:53.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2558999999999997e-05 2020-08-07T15:13:53.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:53.906 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478941+0000", 2020-08-07T15:13:53.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6469999999999997e-06 2020-08-07T15:13:53.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:53.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478945+0000", 2020-08-07T15:13:53.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3109999999999995e-06 2020-08-07T15:13:53.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.908 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.908 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:53.908 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478955+0000", 2020-08-07T15:13:53.908 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.799e-06 2020-08-07T15:13:53.909 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.909 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:53.909 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478958+0000", 2020-08-07T15:13:53.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5682000000000002e-05 2020-08-07T15:13:53.910 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.910 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.910 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.910 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:13:53.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3260000000000002e-06 2020-08-07T15:13:53.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:53.911 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479004+0000", 2020-08-07T15:13:53.911 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73703730499999998 2020-08-07T15:13:53.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.912 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.912 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216041+0000", 2020-08-07T15:13:53.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4445999999999998e-05 2020-08-07T15:13:53.913 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:53.913 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216115+0000", 2020-08-07T15:13:53.914 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013977 2020-08-07T15:13:53.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:53.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216255+0000", 2020-08-07T15:13:53.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0051267359999999998 2020-08-07T15:13:53.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:53.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221382+0000", 2020-08-07T15:13:53.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.0310000000000001e-06 2020-08-07T15:13:53.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.923 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:53.924 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221384+0000", 2020-08-07T15:13:53.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8097999999999998e-05 2020-08-07T15:13:53.924 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.924 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.924 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:53.925 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221452+0000", 2020-08-07T15:13:53.925 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0070259249999999997 2020-08-07T15:13:53.925 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.925 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228478+0000", 2020-08-07T15:13:53.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7479999999999999e-05 2020-08-07T15:13:53.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228535+0000", 2020-08-07T15:13:53.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4399999999999999e-07 2020-08-07T15:13:53.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:53.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228536+0000", 2020-08-07T15:13:53.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017489799999999999 2020-08-07T15:13:53.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:53.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228711+0000", 2020-08-07T15:13:53.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74981275700000005 2020-08-07T15:13:53.930 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.931 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:53.931 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.931 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103269 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:13:53.932 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:13:53.932 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 142.01881813200001, 2020-08-07T15:13:53.932 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74992452799999998, 2020-08-07T15:13:53.932 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:53.932 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:53.933 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:53.933 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:53.933 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:53.934 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103269 2020-08-07T15:13:53.934 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.934 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:53.934 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:53.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:13:53.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:53.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:53.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.478994+0000", 2020-08-07T15:13:53.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5529999999999999e-05 2020-08-07T15:13:53.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.938 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.939 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:53.939 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479040+0000", 2020-08-07T15:13:53.939 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6550000000000003e-06 2020-08-07T15:13:53.939 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.940 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.940 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:53.940 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479044+0000", 2020-08-07T15:13:53.941 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4285e-05 2020-08-07T15:13:53.947 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.947 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.947 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:53.948 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479059+0000", 2020-08-07T15:13:53.948 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2050000000000001e-06 2020-08-07T15:13:53.948 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.949 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:53.949 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479064+0000", 2020-08-07T15:13:53.950 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0952999999999999e-05 2020-08-07T15:13:53.950 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.950 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.951 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479105+0000", 2020-08-07T15:13:53.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3559999999999998e-06 2020-08-07T15:13:53.952 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.952 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.952 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:53.953 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479111+0000", 2020-08-07T15:13:53.953 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.737508104 2020-08-07T15:13:53.953 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.954 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.954 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.954 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216619+0000", 2020-08-07T15:13:53.955 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000126367 2020-08-07T15:13:53.955 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.955 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.956 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:53.956 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216746+0000", 2020-08-07T15:13:53.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00014069899999999999 2020-08-07T15:13:53.957 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:53.958 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.216886+0000", 2020-08-07T15:13:53.958 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045753160000000003 2020-08-07T15:13:53.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:53.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221462+0000", 2020-08-07T15:13:53.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.142e-06 2020-08-07T15:13:53.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.961 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:53.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221463+0000", 2020-08-07T15:13:53.962 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7718e-05 2020-08-07T15:13:53.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:53.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221501+0000", 2020-08-07T15:13:53.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0072733210000000001 2020-08-07T15:13:53.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228774+0000", 2020-08-07T15:13:53.965 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3828e-05 2020-08-07T15:13:53.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:53.966 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228828+0000", 2020-08-07T15:13:53.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9800000000000003e-07 2020-08-07T15:13:53.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:53.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228828+0000", 2020-08-07T15:13:53.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0451000000000005e-05 2020-08-07T15:13:53.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:53.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228919+0000", 2020-08-07T15:13:53.969 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74992452799999998 2020-08-07T15:13:53.969 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.969 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:53.970 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:53.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.971 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103270 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:13:53.972 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.479168+0000", 2020-08-07T15:13:53.973 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 142.01864409500001, 2020-08-07T15:13:53.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75043371699999994, 2020-08-07T15:13:53.974 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:53.975 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:53.976 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:53.976 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:53.976 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:53.977 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103270 2020-08-07T15:13:53.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.977 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:53.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.977 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:53.978 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479168+0000", 2020-08-07T15:13:53.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:53.978 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.979 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.979 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:53.979 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479168+0000", 2020-08-07T15:13:53.979 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1557000000000002e-05 2020-08-07T15:13:53.979 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:53.980 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479210+0000", 2020-08-07T15:13:53.980 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9840000000000001e-06 2020-08-07T15:13:53.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.981 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:53.981 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479215+0000", 2020-08-07T15:13:53.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9539999999999999e-06 2020-08-07T15:13:53.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.982 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:53.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479225+0000", 2020-08-07T15:13:53.982 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.942e-06 2020-08-07T15:13:53.982 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.983 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.983 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:53.983 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479231+0000", 2020-08-07T15:13:53.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2493000000000002e-05 2020-08-07T15:13:53.983 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.984 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.984 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479263+0000", 2020-08-07T15:13:53.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6640000000000003e-06 2020-08-07T15:13:53.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.985 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:53.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479269+0000", 2020-08-07T15:13:53.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73802486499999997 2020-08-07T15:13:53.989 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.989 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:53.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217294+0000", 2020-08-07T15:13:53.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8640000000000006e-05 2020-08-07T15:13:53.990 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:53.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217372+0000", 2020-08-07T15:13:53.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000140338 2020-08-07T15:13:53.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.993 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:53.993 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.217513+0000", 2020-08-07T15:13:53.993 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039952119999999997 2020-08-07T15:13:53.994 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:53.994 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:53.994 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:53.995 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221508+0000", 2020-08-07T15:13:54.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8899999999999999e-07 2020-08-07T15:13:54.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:54.004 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221509+0000", 2020-08-07T15:13:54.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8108999999999997e-05 2020-08-07T15:13:54.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:54.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221547+0000", 2020-08-07T15:13:54.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074232029999999997 2020-08-07T15:13:54.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.018 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:54.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.228970+0000", 2020-08-07T15:13:54.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00046710099999999999 2020-08-07T15:13:54.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.020 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:54.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229437+0000", 2020-08-07T15:13:54.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5400000000000001e-07 2020-08-07T15:13:54.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.022 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:54.023 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229438+0000", 2020-08-07T15:13:54.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016441200000000001 2020-08-07T15:13:54.023 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.023 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:54.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229602+0000", 2020-08-07T15:13:54.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75043371699999994 2020-08-07T15:13:54.025 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:54.025 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:54.025 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:54.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.026 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103271 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:13:54.027 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.479356+0000", 2020-08-07T15:13:54.027 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 142.01845612700001, 2020-08-07T15:13:54.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75047653800000003, 2020-08-07T15:13:54.028 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:54.028 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:54.029 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:54.029 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:54.030 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:54.030 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103271 2020-08-07T15:13:54.030 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.031 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:54.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:54.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479356+0000", 2020-08-07T15:13:54.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:54.032 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.038 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.038 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:54.039 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479356+0000", 2020-08-07T15:13:54.039 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2416000000000002e-05 2020-08-07T15:13:54.039 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.040 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.040 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:54.040 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479399+0000", 2020-08-07T15:13:54.041 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0819999999999998e-06 2020-08-07T15:13:54.041 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.041 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.042 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:54.043 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479404+0000", 2020-08-07T15:13:54.043 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2486999999999999e-05 2020-08-07T15:13:54.043 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.044 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.044 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:54.044 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479416+0000", 2020-08-07T15:13:54.045 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3810000000000001e-06 2020-08-07T15:13:54.045 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.045 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.046 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:54.046 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479421+0000", 2020-08-07T15:13:54.047 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2311999999999998e-05 2020-08-07T15:13:54.047 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.047 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.048 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:54.048 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479453+0000", 2020-08-07T15:13:54.048 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7690000000000002e-06 2020-08-07T15:13:54.048 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.049 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.049 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:54.049 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.479459+0000", 2020-08-07T15:13:54.050 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73869566499999995 2020-08-07T15:13:54.053 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.053 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.054 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:54.054 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218154+0000", 2020-08-07T15:13:54.054 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000124535 2020-08-07T15:13:54.054 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.055 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.055 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:54.055 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218279+0000", 2020-08-07T15:13:54.055 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000223441 2020-08-07T15:13:54.055 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.056 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.056 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:54.056 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.218502+0000", 2020-08-07T15:13:54.056 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003054761 2020-08-07T15:13:54.056 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.057 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.057 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:54.057 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221557+0000", 2020-08-07T15:13:54.057 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0449999999999999e-06 2020-08-07T15:13:54.057 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.058 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.058 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:54.058 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221558+0000", 2020-08-07T15:13:54.058 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3655e-05 2020-08-07T15:13:54.058 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.059 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.059 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:54.059 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.221622+0000", 2020-08-07T15:13:54.059 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.007661434 2020-08-07T15:13:54.059 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.060 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.060 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:54.060 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229283+0000", 2020-08-07T15:13:54.060 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00036023099999999997 2020-08-07T15:13:54.060 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.061 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.061 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:54.061 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229644+0000", 2020-08-07T15:13:54.067 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.63e-07 2020-08-07T15:13:54.067 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.067 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.068 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:54.068 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229644+0000", 2020-08-07T15:13:54.068 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018886099999999999 2020-08-07T15:13:54.069 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.069 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.069 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:54.069 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229833+0000", 2020-08-07T15:13:54.069 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75047653800000003 2020-08-07T15:13:54.070 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:54.070 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:54.070 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:54.070 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.070 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.071 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5099.0:103272 5.7 5:e2667273:::benchmark_data_smithi074_53697_object12908:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e212)", 2020-08-07T15:13:54.071 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:11:31.481658+0000", 2020-08-07T15:13:54.071 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 142.016154704, 2020-08-07T15:13:54.071 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74920410900000001, 2020-08-07T15:13:54.071 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:54.072 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:54.072 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:54.072 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5099", 2020-08-07T15:13:54.072 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:54.074 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 103272 2020-08-07T15:13:54.074 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.074 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:54.074 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.075 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:54.075 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481658+0000", 2020-08-07T15:13:54.075 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:54.075 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.075 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.076 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:54.076 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481658+0000", 2020-08-07T15:13:54.076 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8787e-05 2020-08-07T15:13:54.076 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.076 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.077 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:54.077 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481697+0000", 2020-08-07T15:13:54.077 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0559999999999998e-06 2020-08-07T15:13:54.077 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.078 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.078 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:54.078 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481701+0000", 2020-08-07T15:13:54.078 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1329999999999999e-06 2020-08-07T15:13:54.078 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.079 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.079 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:54.079 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481709+0000", 2020-08-07T15:13:54.079 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.189e-06 2020-08-07T15:13:54.079 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.079 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.080 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:54.080 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481713+0000", 2020-08-07T15:13:54.080 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6025999999999997e-05 2020-08-07T15:13:54.080 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.080 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.081 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:54.081 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481749+0000", 2020-08-07T15:13:54.081 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5619999999999998e-06 2020-08-07T15:13:54.081 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.081 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.082 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:13:54.082 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:31.481756+0000", 2020-08-07T15:13:54.082 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.73729155000000002 2020-08-07T15:13:54.082 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.082 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.083 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:54.083 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219048+0000", 2020-08-07T15:13:54.083 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000120387 2020-08-07T15:13:54.084 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.085 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.085 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:54.085 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219168+0000", 2020-08-07T15:13:54.085 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017865599999999999 2020-08-07T15:13:54.085 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.086 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.086 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 4,5", 2020-08-07T15:13:54.086 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.219347+0000", 2020-08-07T15:13:54.086 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0039049169999999999 2020-08-07T15:13:54.086 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.087 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:54.087 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.223252+0000", 2020-08-07T15:13:54.087 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.412e-06 2020-08-07T15:13:54.087 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.088 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:54.088 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.223253+0000", 2020-08-07T15:13:54.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7521999999999997e-05 2020-08-07T15:13:54.088 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.088 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.089 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:54.089 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.223301+0000", 2020-08-07T15:13:54.089 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00603478 2020-08-07T15:13:54.089 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.089 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.090 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:54.090 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.229336+0000", 2020-08-07T15:13:54.090 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0013309579999999999 2020-08-07T15:13:54.090 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.090 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.091 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:54.091 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.230667+0000", 2020-08-07T15:13:54.091 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7400000000000003e-07 2020-08-07T15:13:54.091 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.091 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.091 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:13:54.092 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.230667+0000", 2020-08-07T15:13:54.092 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019450000000000001 2020-08-07T15:13:54.092 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:54.092 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:54.092 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:13:54.093 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:11:32.230862+0000", 2020-08-07T15:13:54.093 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.74920410900000001 2020-08-07T15:13:54.093 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:54.094 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:54.094 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:54.094 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:54.094 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:54.094 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:54.099 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:13:54.099 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:13:54.100 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:13:54.100 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:13:54.100 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:13:54.100 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:13:54.101 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:13:54.101 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:13:54.101 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:13:54.101 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:13:54.102 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:13:54.102 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:13:54.102 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:13:54.103 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:13:54.103 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:13:54.103 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:13:54.103 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:13:54.104 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:54.094+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:54.198 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:54.196+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:54.265 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:13:54.271 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:54.272 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:54.283 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:13:54.314 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:54.303+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:54.415 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:54.410+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:54.446 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:54.446 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:54.455 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:13:54.544 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:54.542+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:54.618 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:54.618 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:54.627 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:54.625+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:54.627 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:13:54.722 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_3 2020-08-07T15:13:54.723 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool rm unique_pool_3 unique_pool_3 --yes-i-really-really-mean-it 2020-08-07T15:13:54.743 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:54.742+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:54.790 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:54.791 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:54.803 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:13:54.846 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:54.845+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:54.954 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:54.947+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:54.968 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:54.969 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:54.979 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:13:55.055 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:55.053+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:55.106 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:55.107 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:55.114 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:13:55.158 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:55.154+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:55.253 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:55.253 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:55.254 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:13:55.271 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:55.269+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:55.377 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:55.376+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:55.390 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:55.410 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:55.411 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:13:55.498 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:55.495+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:55.532 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:13:55.557 INFO:teuthology.orchestra.run.smithi074.stderr:pool 'unique_pool_3' removed 2020-08-07T15:13:55.586 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:55.582+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:55.613 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2020-08-07T15:13:55.615 INFO:tasks.radosbench:Beginning radosbench... 2020-08-07T15:13:55.616 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_4 2020-08-07T15:13:55.616 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool create unique_pool_4 16 2020-08-07T15:13:55.617 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:55.626 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:55.629 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:13:55.697 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:55.688+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:55.801 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:55.801 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:55.802 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:55.796+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:55.818 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:13:55.902 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:55.899+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:55.954 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:55.955 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:55.964 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:13:56.005 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:56.003+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:56.105 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:13:56.123 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:56.103+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:56.124 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:56.124 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:56.135 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:13:56.220 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:56.209+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:56.265 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:13:56.293 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:56.293 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:56.294 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:13:56.304 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:13:56.313 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:56.311+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:56.314 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:13:56.338 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:13:56.362 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:13:56.392 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:13:56.411 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:13:56.417 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:56.416+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:56.430 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:56.430 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:13:56.431 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:13:56.431 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:13:56.431 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:56.436 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:13:56.436 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:13:56.437 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:13:56.437 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:13:56.437 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:13:56.437 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:13:56.437 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:13:56.438 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:13:56.438 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:13:56.439 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:13:56.528 INFO:teuthology.orchestra.run.smithi074.stderr:pool 'unique_pool_4' created 2020-08-07T15:13:56.529 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:56.526+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:56.543 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd pool application enable unique_pool_4 rados --yes-i-really-mean-it || true 2020-08-07T15:13:56.608 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:56.608 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:13:56.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:13:56.609 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:13:56.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.614 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118153 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:56.614 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.222760+0000", 2020-08-07T15:13:56.615 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 133.37945335000001, 2020-08-07T15:13:56.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0520728660000001, 2020-08-07T15:13:56.615 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:56.615 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:56.616 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:56.616 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:56.616 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:56.616 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118153 2020-08-07T15:13:56.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.636 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:56.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:56.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222760+0000", 2020-08-07T15:13:56.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:56.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:56.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222760+0000", 2020-08-07T15:13:56.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1894000000000002e-05 2020-08-07T15:13:56.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:56.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222812+0000", 2020-08-07T15:13:56.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4880000000000001e-06 2020-08-07T15:13:56.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:56.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222820+0000", 2020-08-07T15:13:56.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6744000000000001e-05 2020-08-07T15:13:56.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:56.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222836+0000", 2020-08-07T15:13:56.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5470000000000002e-06 2020-08-07T15:13:56.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:56.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222846+0000", 2020-08-07T15:13:56.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9478e-05 2020-08-07T15:13:56.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:56.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222925+0000", 2020-08-07T15:13:56.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0966e-05 2020-08-07T15:13:56.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2020-08-07T15:13:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222936+0000", 2020-08-07T15:13:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002974099 2020-08-07T15:13:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:56.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.225910+0000", 2020-08-07T15:13:56.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5197e-05 2020-08-07T15:13:56.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:56.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.225926+0000", 2020-08-07T15:13:56.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.02944747 2020-08-07T15:13:56.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:56.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.255373+0000", 2020-08-07T15:13:56.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022597200000000001 2020-08-07T15:13:56.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:56.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.255599+0000", 2020-08-07T15:13:56.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000194415 2020-08-07T15:13:56.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:56.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.255794+0000", 2020-08-07T15:13:56.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0076831360000000001 2020-08-07T15:13:56.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:56.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263477+0000", 2020-08-07T15:13:56.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01099552 2020-08-07T15:13:56.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274472+0000", 2020-08-07T15:13:56.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2948999999999998e-05 2020-08-07T15:13:56.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274515+0000", 2020-08-07T15:13:56.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.569999999999999e-07 2020-08-07T15:13:56.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:56.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274516+0000", 2020-08-07T15:13:56.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2750999999999997e-05 2020-08-07T15:13:56.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:56.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274569+0000", 2020-08-07T15:13:56.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4109000000000001e-05 2020-08-07T15:13:56.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274613+0000", 2020-08-07T15:13:56.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0432999999999997e-05 2020-08-07T15:13:56.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274653+0000", 2020-08-07T15:13:56.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5900000000000002e-07 2020-08-07T15:13:56.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:56.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274654+0000", 2020-08-07T15:13:56.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017928200000000001 2020-08-07T15:13:56.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:56.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274833+0000", 2020-08-07T15:13:56.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0520728660000001 2020-08-07T15:13:56.713 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:56.713 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:56.713 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:56.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.715 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118154 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:56.715 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.222899+0000", 2020-08-07T15:13:56.715 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 133.379315048, 2020-08-07T15:13:56.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0539106499999999, 2020-08-07T15:13:56.715 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:56.716 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:56.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:56.726 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118154 2020-08-07T15:13:56.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.726 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:56.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:56.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222899+0000", 2020-08-07T15:13:56.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:56.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:56.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222899+0000", 2020-08-07T15:13:56.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4440000000000001e-05 2020-08-07T15:13:56.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:56.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222953+0000", 2020-08-07T15:13:56.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4560000000000003e-06 2020-08-07T15:13:56.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:56.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222960+0000", 2020-08-07T15:13:56.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.201e-05 2020-08-07T15:13:56.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:56.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222972+0000", 2020-08-07T15:13:56.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4779999999999996e-06 2020-08-07T15:13:56.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:56.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222979+0000", 2020-08-07T15:13:56.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2364000000000002e-05 2020-08-07T15:13:56.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:56.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223031+0000", 2020-08-07T15:13:56.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7779999999999996e-06 2020-08-07T15:13:56.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:56.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223040+0000", 2020-08-07T15:13:56.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029381720000000002 2020-08-07T15:13:56.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:56.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.225978+0000", 2020-08-07T15:13:56.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1806e-05 2020-08-07T15:13:56.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:56.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.225990+0000", 2020-08-07T15:13:56.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0316258110000001 2020-08-07T15:13:56.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:56.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.257616+0000", 2020-08-07T15:13:56.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000143759 2020-08-07T15:13:56.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:56.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.257760+0000", 2020-08-07T15:13:56.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000296249 2020-08-07T15:13:56.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:56.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.258056+0000", 2020-08-07T15:13:56.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054347309999999999 2020-08-07T15:13:56.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:56.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263491+0000", 2020-08-07T15:13:56.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011408567999999999 2020-08-07T15:13:56.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274899+0000", 2020-08-07T15:13:56.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030791400000000001 2020-08-07T15:13:56.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275207+0000", 2020-08-07T15:13:56.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6499999999999998e-07 2020-08-07T15:13:56.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:56.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275208+0000", 2020-08-07T15:13:56.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3047999999999998e-05 2020-08-07T15:13:56.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:56.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275261+0000", 2020-08-07T15:13:56.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000419706 2020-08-07T15:13:56.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275681+0000", 2020-08-07T15:13:56.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091386099999999997 2020-08-07T15:13:56.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276594+0000", 2020-08-07T15:13:56.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6000000000000001e-06 2020-08-07T15:13:56.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:56.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276597+0000", 2020-08-07T15:13:56.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000212134 2020-08-07T15:13:56.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:56.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276809+0000", 2020-08-07T15:13:56.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0539106499999999 2020-08-07T15:13:56.753 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:56.755 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:56.756 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:56.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.757 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118155 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:56.757 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223018+0000", 2020-08-07T15:13:56.758 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 133.37919523799999, 2020-08-07T15:13:56.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0540362350000001, 2020-08-07T15:13:56.758 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:56.759 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:56.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:56.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:56.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:56.760 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118155 2020-08-07T15:13:56.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.761 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:56.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:56.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223018+0000", 2020-08-07T15:13:56.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:56.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:56.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223018+0000", 2020-08-07T15:13:56.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8291999999999999e-05 2020-08-07T15:13:56.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:56.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223057+0000", 2020-08-07T15:13:56.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.805e-06 2020-08-07T15:13:56.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:56.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223060+0000", 2020-08-07T15:13:56.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0050000000000004e-06 2020-08-07T15:13:56.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:56.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223068+0000", 2020-08-07T15:13:56.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4846e-05 2020-08-07T15:13:56.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:56.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223083+0000", 2020-08-07T15:13:56.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8695000000000005e-05 2020-08-07T15:13:56.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223162+0000", 2020-08-07T15:13:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4919999999999996e-06 2020-08-07T15:13:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223169+0000", 2020-08-07T15:13:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028562219999999998 2020-08-07T15:13:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226026+0000", 2020-08-07T15:13:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.027e-05 2020-08-07T15:13:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226036+0000", 2020-08-07T15:13:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0327165199999999 2020-08-07T15:13:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.258752+0000", 2020-08-07T15:13:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012614899999999999 2020-08-07T15:13:56.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:56.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.258879+0000", 2020-08-07T15:13:56.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000258008 2020-08-07T15:13:56.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:56.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.259137+0000", 2020-08-07T15:13:56.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0067332820000000002 2020-08-07T15:13:56.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:56.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.265870+0000", 2020-08-07T15:13:56.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0090796000000000002 2020-08-07T15:13:56.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274950+0000", 2020-08-07T15:13:56.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000632601 2020-08-07T15:13:56.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275582+0000", 2020-08-07T15:13:56.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8999999999999997e-07 2020-08-07T15:13:56.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:56.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275583+0000", 2020-08-07T15:13:56.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6400999999999998e-05 2020-08-07T15:13:56.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:56.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275629+0000", 2020-08-07T15:13:56.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014887 2020-08-07T15:13:56.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275778+0000", 2020-08-07T15:13:56.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010622559999999999 2020-08-07T15:13:56.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276840+0000", 2020-08-07T15:13:56.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2199999999999999e-07 2020-08-07T15:13:56.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:56.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276841+0000", 2020-08-07T15:13:56.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000214009 2020-08-07T15:13:56.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:56.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277055+0000", 2020-08-07T15:13:56.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0540362350000001 2020-08-07T15:13:56.822 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:56.822 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:56.822 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:56.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.823 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118156 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:56.823 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223123+0000", 2020-08-07T15:13:56.825 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 133.37909037200001, 2020-08-07T15:13:56.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0542400009999999, 2020-08-07T15:13:56.825 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:56.826 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:56.826 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:56.826 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:56.826 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:56.826 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118156 2020-08-07T15:13:56.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.827 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:56.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:56.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223123+0000", 2020-08-07T15:13:56.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:56.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:56.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223123+0000", 2020-08-07T15:13:56.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2212999999999995e-05 2020-08-07T15:13:56.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:56.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223195+0000", 2020-08-07T15:13:56.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8709999999999999e-06 2020-08-07T15:13:56.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:56.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223202+0000", 2020-08-07T15:13:56.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3509999999999999e-05 2020-08-07T15:13:56.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:56.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223216+0000", 2020-08-07T15:13:56.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2729999999999997e-06 2020-08-07T15:13:56.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:56.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223223+0000", 2020-08-07T15:13:56.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6728000000000003e-05 2020-08-07T15:13:56.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:56.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223280+0000", 2020-08-07T15:13:56.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1046999999999999e-05 2020-08-07T15:13:56.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:56.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223291+0000", 2020-08-07T15:13:56.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027721030000000002 2020-08-07T15:13:56.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:56.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226063+0000", 2020-08-07T15:13:56.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1809000000000004e-05 2020-08-07T15:13:56.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:56.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226125+0000", 2020-08-07T15:13:56.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0337493870000001 2020-08-07T15:13:56.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:56.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.259874+0000", 2020-08-07T15:13:56.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000133974 2020-08-07T15:13:56.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:56.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.260008+0000", 2020-08-07T15:13:56.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00064730600000000005 2020-08-07T15:13:56.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:56.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.260655+0000", 2020-08-07T15:13:56.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052233740000000002 2020-08-07T15:13:56.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:56.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.265879+0000", 2020-08-07T15:13:56.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.009433238 2020-08-07T15:13:56.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275312+0000", 2020-08-07T15:13:56.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050955399999999995 2020-08-07T15:13:56.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275822+0000", 2020-08-07T15:13:56.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.58e-07 2020-08-07T15:13:56.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:56.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275822+0000", 2020-08-07T15:13:56.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5469000000000006e-05 2020-08-07T15:13:56.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:56.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275887+0000", 2020-08-07T15:13:56.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000371212 2020-08-07T15:13:56.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276259+0000", 2020-08-07T15:13:56.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00085558499999999996 2020-08-07T15:13:56.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277114+0000", 2020-08-07T15:13:56.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2020-08-07T15:13:56.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:56.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277115+0000", 2020-08-07T15:13:56.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024832000000000001 2020-08-07T15:13:56.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:56.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277363+0000", 2020-08-07T15:13:56.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0542400009999999 2020-08-07T15:13:56.884 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:56.884 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:56.885 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:56.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.885 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118157 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:56.885 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223267+0000", 2020-08-07T15:13:56.886 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 133.37894613700001, 2020-08-07T15:13:56.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0543752369999999, 2020-08-07T15:13:56.886 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:56.886 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:56.887 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:56.887 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:56.887 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:56.887 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118157 2020-08-07T15:13:56.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.888 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:56.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:56.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223267+0000", 2020-08-07T15:13:56.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:56.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:56.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223267+0000", 2020-08-07T15:13:56.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9616000000000001e-05 2020-08-07T15:13:56.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:56.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223317+0000", 2020-08-07T15:13:56.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1359999999999996e-06 2020-08-07T15:13:56.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:56.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223322+0000", 2020-08-07T15:13:56.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0529e-05 2020-08-07T15:13:56.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:56.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223333+0000", 2020-08-07T15:13:56.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1059999999999999e-06 2020-08-07T15:13:56.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:56.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223338+0000", 2020-08-07T15:13:56.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8609000000000002e-05 2020-08-07T15:13:56.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:56.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223386+0000", 2020-08-07T15:13:56.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3830000000000003e-06 2020-08-07T15:13:56.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:56.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223394+0000", 2020-08-07T15:13:56.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027643580000000002 2020-08-07T15:13:56.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:56.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226158+0000", 2020-08-07T15:13:56.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6809999999999993e-06 2020-08-07T15:13:56.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:56.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226168+0000", 2020-08-07T15:13:56.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.035171356 2020-08-07T15:13:56.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.261339+0000", 2020-08-07T15:13:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000130514 2020-08-07T15:13:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.261470+0000", 2020-08-07T15:13:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025685699999999998 2020-08-07T15:13:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.261727+0000", 2020-08-07T15:13:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075810410000000002 2020-08-07T15:13:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269308+0000", 2020-08-07T15:13:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064252490000000001 2020-08-07T15:13:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275733+0000", 2020-08-07T15:13:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020602000000000001 2020-08-07T15:13:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275939+0000", 2020-08-07T15:13:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999999e-07 2020-08-07T15:13:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275939+0000", 2020-08-07T15:13:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7071999999999997e-05 2020-08-07T15:13:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276016+0000", 2020-08-07T15:13:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029090700000000003 2020-08-07T15:13:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276307+0000", 2020-08-07T15:13:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001114929 2020-08-07T15:13:56.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277422+0000", 2020-08-07T15:13:56.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1200000000000003e-07 2020-08-07T15:13:56.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:56.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277423+0000", 2020-08-07T15:13:56.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021982600000000001 2020-08-07T15:13:56.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:56.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277643+0000", 2020-08-07T15:13:56.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0543752369999999 2020-08-07T15:13:56.914 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:56.914 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:56.914 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:56.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.915 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118158 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:56.915 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223375+0000", 2020-08-07T15:13:56.915 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 133.378838401, 2020-08-07T15:13:56.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0545579519999999, 2020-08-07T15:13:56.915 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:56.916 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:56.916 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:56.916 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:56.916 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:56.916 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118158 2020-08-07T15:13:56.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.917 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:56.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:56.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223375+0000", 2020-08-07T15:13:56.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:56.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:56.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223375+0000", 2020-08-07T15:13:56.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7657000000000001e-05 2020-08-07T15:13:56.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:56.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223423+0000", 2020-08-07T15:13:56.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0810000000000001e-06 2020-08-07T15:13:56.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:56.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223428+0000", 2020-08-07T15:13:56.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7839999999999998e-06 2020-08-07T15:13:56.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:56.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223438+0000", 2020-08-07T15:13:56.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.634e-06 2020-08-07T15:13:56.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:56.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223442+0000", 2020-08-07T15:13:56.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0154999999999998e-05 2020-08-07T15:13:56.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:56.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223482+0000", 2020-08-07T15:13:56.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9579999999999998e-06 2020-08-07T15:13:56.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:56.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223487+0000", 2020-08-07T15:13:56.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027060999999999999 2020-08-07T15:13:56.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:56.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226194+0000", 2020-08-07T15:13:56.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1311999999999999e-05 2020-08-07T15:13:56.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:56.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226205+0000", 2020-08-07T15:13:56.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0361494609999999 2020-08-07T15:13:56.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:56.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.262354+0000", 2020-08-07T15:13:56.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9950000000000004e-05 2020-08-07T15:13:56.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:56.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.262454+0000", 2020-08-07T15:13:56.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000244093 2020-08-07T15:13:56.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:56.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.262698+0000", 2020-08-07T15:13:56.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0066176150000000003 2020-08-07T15:13:56.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:56.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269316+0000", 2020-08-07T15:13:56.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006768998 2020-08-07T15:13:56.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276085+0000", 2020-08-07T15:13:56.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9524000000000002e-05 2020-08-07T15:13:56.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276135+0000", 2020-08-07T15:13:56.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6100000000000001e-07 2020-08-07T15:13:56.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:56.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276135+0000", 2020-08-07T15:13:56.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1214e-05 2020-08-07T15:13:56.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:56.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276206+0000", 2020-08-07T15:13:56.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014914999999999999 2020-08-07T15:13:56.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276355+0000", 2020-08-07T15:13:56.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013427770000000001 2020-08-07T15:13:56.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277698+0000", 2020-08-07T15:13:56.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8599999999999998e-07 2020-08-07T15:13:56.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:56.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277699+0000", 2020-08-07T15:13:56.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023444199999999999 2020-08-07T15:13:56.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:56.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277933+0000", 2020-08-07T15:13:56.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0545579519999999 2020-08-07T15:13:56.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:56.942 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:56.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:56.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.943 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118159 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:56.943 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223476+0000", 2020-08-07T15:13:56.943 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 133.37873723499999, 2020-08-07T15:13:56.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0547920749999999, 2020-08-07T15:13:56.944 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:56.944 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:56.944 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:56.944 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:56.945 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:56.945 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118159 2020-08-07T15:13:56.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.945 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:56.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:56.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223476+0000", 2020-08-07T15:13:56.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:56.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:56.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223476+0000", 2020-08-07T15:13:56.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6094000000000003e-05 2020-08-07T15:13:56.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:56.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223522+0000", 2020-08-07T15:13:56.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6650000000000002e-06 2020-08-07T15:13:56.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:56.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223527+0000", 2020-08-07T15:13:56.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5629999999999998e-06 2020-08-07T15:13:56.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:56.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223537+0000", 2020-08-07T15:13:56.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6810000000000003e-06 2020-08-07T15:13:56.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:56.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223542+0000", 2020-08-07T15:13:56.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6680000000000001e-05 2020-08-07T15:13:56.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:56.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223579+0000", 2020-08-07T15:13:56.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2020-08-07T15:13:56.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:56.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223586+0000", 2020-08-07T15:13:56.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026469250000000001 2020-08-07T15:13:56.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:56.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226233+0000", 2020-08-07T15:13:56.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3130000000000004e-06 2020-08-07T15:13:56.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:56.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226242+0000", 2020-08-07T15:13:56.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.037297957 2020-08-07T15:13:56.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:56.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263540+0000", 2020-08-07T15:13:56.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4307000000000004e-05 2020-08-07T15:13:56.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:56.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263634+0000", 2020-08-07T15:13:56.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019544300000000001 2020-08-07T15:13:56.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:56.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263830+0000", 2020-08-07T15:13:56.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054939949999999998 2020-08-07T15:13:56.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:56.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269324+0000", 2020-08-07T15:13:56.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070859139999999996 2020-08-07T15:13:56.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276410+0000", 2020-08-07T15:13:56.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0924000000000002e-05 2020-08-07T15:13:56.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276461+0000", 2020-08-07T15:13:56.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4099999999999998e-07 2020-08-07T15:13:56.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:56.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276461+0000", 2020-08-07T15:13:56.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8027000000000004e-05 2020-08-07T15:13:56.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:56.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276539+0000", 2020-08-07T15:13:56.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001458453 2020-08-07T15:13:56.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277998+0000", 2020-08-07T15:13:56.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7283000000000001e-05 2020-08-07T15:13:56.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278045+0000", 2020-08-07T15:13:56.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9100000000000004e-07 2020-08-07T15:13:56.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:56.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278046+0000", 2020-08-07T15:13:56.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022290400000000001 2020-08-07T15:13:56.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:56.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278268+0000", 2020-08-07T15:13:56.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0547920749999999 2020-08-07T15:13:56.971 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:56.971 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:56.971 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:56.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.972 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118160 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:56.972 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223579+0000", 2020-08-07T15:13:56.972 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 133.37863425399999, 2020-08-07T15:13:56.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0553689980000001, 2020-08-07T15:13:56.973 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:56.973 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:56.973 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:56.973 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:56.974 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:56.974 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118160 2020-08-07T15:13:56.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.974 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:56.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:56.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223579+0000", 2020-08-07T15:13:56.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:56.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:56.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223579+0000", 2020-08-07T15:13:56.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5401000000000001e-05 2020-08-07T15:13:56.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:56.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223625+0000", 2020-08-07T15:13:56.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9069999999999998e-06 2020-08-07T15:13:56.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:56.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223630+0000", 2020-08-07T15:13:56.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4800000000000007e-06 2020-08-07T15:13:56.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:56.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223639+0000", 2020-08-07T15:13:56.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5410000000000002e-06 2020-08-07T15:13:56.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:56.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223644+0000", 2020-08-07T15:13:56.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1476000000000002e-05 2020-08-07T15:13:56.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:56.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223675+0000", 2020-08-07T15:13:56.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0240000000000003e-06 2020-08-07T15:13:56.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:56.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223681+0000", 2020-08-07T15:13:56.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025870820000000001 2020-08-07T15:13:56.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:56.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226268+0000", 2020-08-07T15:13:56.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0540000000000001e-05 2020-08-07T15:13:56.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:56.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226279+0000", 2020-08-07T15:13:56.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.038209656 2020-08-07T15:13:56.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:56.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.264488+0000", 2020-08-07T15:13:56.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029908499999999999 2020-08-07T15:13:56.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:56.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.264788+0000", 2020-08-07T15:13:56.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000216997 2020-08-07T15:13:56.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:56.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.265005+0000", 2020-08-07T15:13:56.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0060308230000000003 2020-08-07T15:13:56.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:56.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271035+0000", 2020-08-07T15:13:56.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0072889499999999998 2020-08-07T15:13:56.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278324+0000", 2020-08-07T15:13:56.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4982999999999999e-05 2020-08-07T15:13:56.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:56.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:56.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:56.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278369+0000", 2020-08-07T15:13:56.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4000000000000002e-07 2020-08-07T15:13:57.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:57.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278370+0000", 2020-08-07T15:13:57.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7656000000000003e-05 2020-08-07T15:13:57.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:57.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278447+0000", 2020-08-07T15:13:57.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000277416 2020-08-07T15:13:57.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278725+0000", 2020-08-07T15:13:57.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2345000000000001e-05 2020-08-07T15:13:57.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278777+0000", 2020-08-07T15:13:57.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8500000000000002e-07 2020-08-07T15:13:57.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:57.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278778+0000", 2020-08-07T15:13:57.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017071100000000001 2020-08-07T15:13:57.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:57.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278948+0000", 2020-08-07T15:13:57.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0553689980000001 2020-08-07T15:13:57.012 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.012 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:57.012 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.014 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118441 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:57.014 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223679+0000", 2020-08-07T15:13:57.014 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 133.378534239, 2020-08-07T15:13:57.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057502779, 2020-08-07T15:13:57.014 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:57.015 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:57.015 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:57.015 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:57.015 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:57.015 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118441 2020-08-07T15:13:57.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.016 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:57.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:57.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223679+0000", 2020-08-07T15:13:57.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:57.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:57.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223679+0000", 2020-08-07T15:13:57.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5559999999999997e-05 2020-08-07T15:13:57.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:57.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223725+0000", 2020-08-07T15:13:57.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1250000000000001e-06 2020-08-07T15:13:57.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:57.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223730+0000", 2020-08-07T15:13:57.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0046e-05 2020-08-07T15:13:57.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:57.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223740+0000", 2020-08-07T15:13:57.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5979999999999999e-06 2020-08-07T15:13:57.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:57.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223745+0000", 2020-08-07T15:13:57.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7234999999999998e-05 2020-08-07T15:13:57.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223782+0000", 2020-08-07T15:13:57.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7789999999999998e-06 2020-08-07T15:13:57.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:57.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223789+0000", 2020-08-07T15:13:57.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002516169 2020-08-07T15:13:57.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226305+0000", 2020-08-07T15:13:57.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3860000000000007e-06 2020-08-07T15:13:57.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:57.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226313+0000", 2020-08-07T15:13:57.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0396126269999999 2020-08-07T15:13:57.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.265926+0000", 2020-08-07T15:13:57.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00079104700000000002 2020-08-07T15:13:57.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:57.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.266717+0000", 2020-08-07T15:13:57.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024298599999999999 2020-08-07T15:13:57.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:57.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.266960+0000", 2020-08-07T15:13:57.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040855509999999998 2020-08-07T15:13:57.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:57.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271045+0000", 2020-08-07T15:13:57.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0074549170000000001 2020-08-07T15:13:57.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278500+0000", 2020-08-07T15:13:57.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8816999999999997e-05 2020-08-07T15:13:57.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278549+0000", 2020-08-07T15:13:57.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9000000000000002e-07 2020-08-07T15:13:57.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:57.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278550+0000", 2020-08-07T15:13:57.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0120999999999997e-05 2020-08-07T15:13:57.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:57.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278600+0000", 2020-08-07T15:13:57.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00092631699999999996 2020-08-07T15:13:57.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279526+0000", 2020-08-07T15:13:57.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014504520000000001 2020-08-07T15:13:57.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280976+0000", 2020-08-07T15:13:57.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6899999999999997e-07 2020-08-07T15:13:57.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:57.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280977+0000", 2020-08-07T15:13:57.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020508700000000001 2020-08-07T15:13:57.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:57.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281182+0000", 2020-08-07T15:13:57.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057502779 2020-08-07T15:13:57.040 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.040 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:57.040 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.041 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118442 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:57.041 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223780+0000", 2020-08-07T15:13:57.041 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 133.37843336899999, 2020-08-07T15:13:57.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0575529850000001, 2020-08-07T15:13:57.042 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:57.042 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:57.042 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:57.043 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:57.043 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:57.043 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118442 2020-08-07T15:13:57.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.043 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:57.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:57.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223780+0000", 2020-08-07T15:13:57.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:57.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:57.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223780+0000", 2020-08-07T15:13:57.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0340999999999998e-05 2020-08-07T15:13:57.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:57.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223831+0000", 2020-08-07T15:13:57.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3499999999999996e-06 2020-08-07T15:13:57.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:57.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223836+0000", 2020-08-07T15:13:57.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0414999999999999e-05 2020-08-07T15:13:57.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:57.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223846+0000", 2020-08-07T15:13:57.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9239999999999998e-06 2020-08-07T15:13:57.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:57.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223851+0000", 2020-08-07T15:13:57.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5946000000000003e-05 2020-08-07T15:13:57.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223907+0000", 2020-08-07T15:13:57.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6189999999999996e-06 2020-08-07T15:13:57.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:57.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223914+0000", 2020-08-07T15:13:57.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002424149 2020-08-07T15:13:57.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226338+0000", 2020-08-07T15:13:57.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2830000000000001e-06 2020-08-07T15:13:57.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:57.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226346+0000", 2020-08-07T15:13:57.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0412357219999999 2020-08-07T15:13:57.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.267582+0000", 2020-08-07T15:13:57.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000123924 2020-08-07T15:13:57.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:57.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.267706+0000", 2020-08-07T15:13:57.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000216376 2020-08-07T15:13:57.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:57.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.267922+0000", 2020-08-07T15:13:57.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031340790000000001 2020-08-07T15:13:57.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:57.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271056+0000", 2020-08-07T15:13:57.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.007623276 2020-08-07T15:13:57.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278680+0000", 2020-08-07T15:13:57.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000333853 2020-08-07T15:13:57.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279013+0000", 2020-08-07T15:13:57.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.02e-06 2020-08-07T15:13:57.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:57.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279014+0000", 2020-08-07T15:13:57.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6306999999999999e-05 2020-08-07T15:13:57.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:57.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279101+0000", 2020-08-07T15:13:57.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050739800000000005 2020-08-07T15:13:57.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279608+0000", 2020-08-07T15:13:57.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001617098 2020-08-07T15:13:57.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281225+0000", 2020-08-07T15:13:57.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9700000000000002e-07 2020-08-07T15:13:57.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:57.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281226+0000", 2020-08-07T15:13:57.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107508 2020-08-07T15:13:57.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:57.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281333+0000", 2020-08-07T15:13:57.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0575529850000001 2020-08-07T15:13:57.077 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.077 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:57.078 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.080 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118443 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:57.080 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223889+0000", 2020-08-07T15:13:57.080 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 133.37832469899999, 2020-08-07T15:13:57.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.05758606, 2020-08-07T15:13:57.081 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:57.081 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:57.081 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:57.081 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:57.081 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:57.082 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118443 2020-08-07T15:13:57.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.082 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:57.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:57.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223889+0000", 2020-08-07T15:13:57.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:57.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:57.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223889+0000", 2020-08-07T15:13:57.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8148e-05 2020-08-07T15:13:57.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:57.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223937+0000", 2020-08-07T15:13:57.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2789999999999998e-06 2020-08-07T15:13:57.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:57.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223942+0000", 2020-08-07T15:13:57.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0472000000000001e-05 2020-08-07T15:13:57.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:57.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223953+0000", 2020-08-07T15:13:57.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6580000000000001e-06 2020-08-07T15:13:57.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:57.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223957+0000", 2020-08-07T15:13:57.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5981000000000001e-05 2020-08-07T15:13:57.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223983+0000", 2020-08-07T15:13:57.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.288e-06 2020-08-07T15:13:57.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:57.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223989+0000", 2020-08-07T15:13:57.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024051889999999999 2020-08-07T15:13:57.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226394+0000", 2020-08-07T15:13:57.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2439000000000001e-05 2020-08-07T15:13:57.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:57.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226406+0000", 2020-08-07T15:13:57.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0421168519999999 2020-08-07T15:13:57.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.268523+0000", 2020-08-07T15:13:57.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0449000000000002e-05 2020-08-07T15:13:57.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:57.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.268604+0000", 2020-08-07T15:13:57.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014104300000000001 2020-08-07T15:13:57.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:57.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.268745+0000", 2020-08-07T15:13:57.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051249829999999996 2020-08-07T15:13:57.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:57.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.273870+0000", 2020-08-07T15:13:57.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052772840000000001 2020-08-07T15:13:57.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279147+0000", 2020-08-07T15:13:57.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6060000000000003e-05 2020-08-07T15:13:57.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279193+0000", 2020-08-07T15:13:57.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9999999999999997e-07 2020-08-07T15:13:57.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:57.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279194+0000", 2020-08-07T15:13:57.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3566000000000003e-05 2020-08-07T15:13:57.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:57.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279237+0000", 2020-08-07T15:13:57.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011546359999999999 2020-08-07T15:13:57.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280392+0000", 2020-08-07T15:13:57.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00097857799999999991 2020-08-07T15:13:57.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281370+0000", 2020-08-07T15:13:57.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999999e-07 2020-08-07T15:13:57.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:57.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281371+0000", 2020-08-07T15:13:57.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104119 2020-08-07T15:13:57.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:57.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281475+0000", 2020-08-07T15:13:57.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.05758606 2020-08-07T15:13:57.145 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.145 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:57.145 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.146 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118444 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:57.146 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223994+0000", 2020-08-07T15:13:57.146 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 133.37821932400001, 2020-08-07T15:13:57.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057634572, 2020-08-07T15:13:57.147 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:57.149 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:57.149 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:57.149 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:57.149 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:57.150 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118444 2020-08-07T15:13:57.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.150 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:57.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:57.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223994+0000", 2020-08-07T15:13:57.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:57.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:57.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223994+0000", 2020-08-07T15:13:57.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1279999999999997e-05 2020-08-07T15:13:57.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:57.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224046+0000", 2020-08-07T15:13:57.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6930000000000004e-06 2020-08-07T15:13:57.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:57.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224051+0000", 2020-08-07T15:13:57.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.095e-05 2020-08-07T15:13:57.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:57.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224062+0000", 2020-08-07T15:13:57.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9529999999999998e-06 2020-08-07T15:13:57.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:57.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224067+0000", 2020-08-07T15:13:57.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000162091 2020-08-07T15:13:57.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224229+0000", 2020-08-07T15:13:57.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.276e-06 2020-08-07T15:13:57.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:57.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224238+0000", 2020-08-07T15:13:57.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022140409999999999 2020-08-07T15:13:57.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226452+0000", 2020-08-07T15:13:57.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5951999999999999e-05 2020-08-07T15:13:57.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:57.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226467+0000", 2020-08-07T15:13:57.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0428868099999999 2020-08-07T15:13:57.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269354+0000", 2020-08-07T15:13:57.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5264000000000003e-05 2020-08-07T15:13:57.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:57.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269430+0000", 2020-08-07T15:13:57.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020438499999999999 2020-08-07T15:13:57.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:57.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269634+0000", 2020-08-07T15:13:57.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042509959999999999 2020-08-07T15:13:57.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:57.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.273885+0000", 2020-08-07T15:13:57.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054290140000000002 2020-08-07T15:13:57.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279314+0000", 2020-08-07T15:13:57.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00038996700000000002 2020-08-07T15:13:57.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279704+0000", 2020-08-07T15:13:57.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5700000000000002e-07 2020-08-07T15:13:57.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:57.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279705+0000", 2020-08-07T15:13:57.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9377999999999997e-05 2020-08-07T15:13:57.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:57.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279794+0000", 2020-08-07T15:13:57.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00064521299999999995 2020-08-07T15:13:57.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280439+0000", 2020-08-07T15:13:57.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001072185 2020-08-07T15:13:57.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281511+0000", 2020-08-07T15:13:57.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7300000000000002e-07 2020-08-07T15:13:57.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:57.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281512+0000", 2020-08-07T15:13:57.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117094 2020-08-07T15:13:57.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:57.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281629+0000", 2020-08-07T15:13:57.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057634572 2020-08-07T15:13:57.174 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.174 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:57.175 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.175 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118445 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:57.175 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.224116+0000", 2020-08-07T15:13:57.176 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 133.37809738300001, 2020-08-07T15:13:57.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057744163, 2020-08-07T15:13:57.176 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:57.176 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:57.176 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:57.177 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:57.177 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:57.177 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118445 2020-08-07T15:13:57.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.177 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:57.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:57.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224116+0000", 2020-08-07T15:13:57.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:57.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:57.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224116+0000", 2020-08-07T15:13:57.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3542999999999999e-05 2020-08-07T15:13:57.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:57.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224170+0000", 2020-08-07T15:13:57.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.446e-06 2020-08-07T15:13:57.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:57.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224175+0000", 2020-08-07T15:13:57.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1473e-05 2020-08-07T15:13:57.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:57.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224187+0000", 2020-08-07T15:13:57.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9449999999999998e-06 2020-08-07T15:13:57.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:57.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224193+0000", 2020-08-07T15:13:57.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4945999999999997e-05 2020-08-07T15:13:57.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224268+0000", 2020-08-07T15:13:57.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5799999999999992e-06 2020-08-07T15:13:57.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:57.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224276+0000", 2020-08-07T15:13:57.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022246980000000002 2020-08-07T15:13:57.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226501+0000", 2020-08-07T15:13:57.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1359999999999997e-06 2020-08-07T15:13:57.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:57.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226509+0000", 2020-08-07T15:13:57.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.043599741 2020-08-07T15:13:57.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.270109+0000", 2020-08-07T15:13:57.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000129503 2020-08-07T15:13:57.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:57.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.270238+0000", 2020-08-07T15:13:57.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021162500000000001 2020-08-07T15:13:57.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:57.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.270450+0000", 2020-08-07T15:13:57.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036313090000000001 2020-08-07T15:13:57.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:57.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274081+0000", 2020-08-07T15:13:57.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00557165 2020-08-07T15:13:57.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279653+0000", 2020-08-07T15:13:57.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000196658 2020-08-07T15:13:57.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279849+0000", 2020-08-07T15:13:57.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6700000000000003e-07 2020-08-07T15:13:57.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:57.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279850+0000", 2020-08-07T15:13:57.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6522999999999998e-05 2020-08-07T15:13:57.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:57.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279937+0000", 2020-08-07T15:13:57.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00055050400000000005 2020-08-07T15:13:57.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280487+0000", 2020-08-07T15:13:57.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011833760000000001 2020-08-07T15:13:57.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281670+0000", 2020-08-07T15:13:57.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2020-08-07T15:13:57.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:57.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281671+0000", 2020-08-07T15:13:57.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001896 2020-08-07T15:13:57.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:57.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281860+0000", 2020-08-07T15:13:57.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057744163 2020-08-07T15:13:57.205 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.206 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:57.206 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.207 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118446 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:57.207 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.224256+0000", 2020-08-07T15:13:57.207 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 133.37795759799999, 2020-08-07T15:13:57.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057864804, 2020-08-07T15:13:57.208 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:57.208 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:57.208 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:57.208 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:57.208 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:57.208 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118446 2020-08-07T15:13:57.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.209 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:57.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:57.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224256+0000", 2020-08-07T15:13:57.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:57.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:57.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224256+0000", 2020-08-07T15:13:57.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5838000000000003e-05 2020-08-07T15:13:57.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:57.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224312+0000", 2020-08-07T15:13:57.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.879e-06 2020-08-07T15:13:57.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:57.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224318+0000", 2020-08-07T15:13:57.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3593e-05 2020-08-07T15:13:57.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:57.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224331+0000", 2020-08-07T15:13:57.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6169999999999998e-06 2020-08-07T15:13:57.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:57.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224337+0000", 2020-08-07T15:13:57.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7246000000000001e-05 2020-08-07T15:13:57.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224384+0000", 2020-08-07T15:13:57.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7690000000000008e-06 2020-08-07T15:13:57.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:57.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224394+0000", 2020-08-07T15:13:57.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021394589999999998 2020-08-07T15:13:57.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226533+0000", 2020-08-07T15:13:57.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3849999999999996e-06 2020-08-07T15:13:57.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:57.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226541+0000", 2020-08-07T15:13:57.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0445569109999999 2020-08-07T15:13:57.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271098+0000", 2020-08-07T15:13:57.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012137 2020-08-07T15:13:57.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:57.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271219+0000", 2020-08-07T15:13:57.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013978300000000001 2020-08-07T15:13:57.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:57.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271359+0000", 2020-08-07T15:13:57.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027367699999999999 2020-08-07T15:13:57.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:57.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274096+0000", 2020-08-07T15:13:57.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058973660000000002 2020-08-07T15:13:57.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279993+0000", 2020-08-07T15:13:57.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116291 2020-08-07T15:13:57.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280109+0000", 2020-08-07T15:13:57.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1099999999999995e-07 2020-08-07T15:13:57.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:57.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280110+0000", 2020-08-07T15:13:57.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.564e-05 2020-08-07T15:13:57.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:57.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280196+0000", 2020-08-07T15:13:57.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061969 2020-08-07T15:13:57.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280815+0000", 2020-08-07T15:13:57.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001097612 2020-08-07T15:13:57.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281913+0000", 2020-08-07T15:13:57.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4300000000000003e-07 2020-08-07T15:13:57.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:57.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281913+0000", 2020-08-07T15:13:57.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000207431 2020-08-07T15:13:57.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:57.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282121+0000", 2020-08-07T15:13:57.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057864804 2020-08-07T15:13:57.236 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.236 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:57.236 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.237 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118447 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:57.237 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.224379+0000", 2020-08-07T15:13:57.237 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 133.37783483199999, 2020-08-07T15:13:57.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0579997480000001, 2020-08-07T15:13:57.238 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:57.238 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:57.238 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:57.238 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:57.238 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:57.239 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118447 2020-08-07T15:13:57.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.239 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:57.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:57.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224379+0000", 2020-08-07T15:13:57.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:57.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:57.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224379+0000", 2020-08-07T15:13:57.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9775999999999999e-05 2020-08-07T15:13:57.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:57.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224429+0000", 2020-08-07T15:13:57.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8500000000000002e-06 2020-08-07T15:13:57.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:57.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224433+0000", 2020-08-07T15:13:57.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7340000000000008e-06 2020-08-07T15:13:57.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:57.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224441+0000", 2020-08-07T15:13:57.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.031e-06 2020-08-07T15:13:57.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:57.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224445+0000", 2020-08-07T15:13:57.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4264000000000003e-05 2020-08-07T15:13:57.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224479+0000", 2020-08-07T15:13:57.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0039999999999999e-06 2020-08-07T15:13:57.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:13:57.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224487+0000", 2020-08-07T15:13:57.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002241682 2020-08-07T15:13:57.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226729+0000", 2020-08-07T15:13:57.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0087e-05 2020-08-07T15:13:57.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:57.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226739+0000", 2020-08-07T15:13:57.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.045170006 2020-08-07T15:13:57.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271909+0000", 2020-08-07T15:13:57.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012782900000000001 2020-08-07T15:13:57.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:57.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.272037+0000", 2020-08-07T15:13:57.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024079400000000001 2020-08-07T15:13:57.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.272278+0000", 2020-08-07T15:13:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023454399999999999 2020-08-07T15:13:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274623+0000", 2020-08-07T15:13:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054268190000000003 2020-08-07T15:13:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280050+0000", 2020-08-07T15:13:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00048391200000000002 2020-08-07T15:13:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280534+0000", 2020-08-07T15:13:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5400000000000001e-07 2020-08-07T15:13:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280535+0000", 2020-08-07T15:13:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8922e-05 2020-08-07T15:13:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280623+0000", 2020-08-07T15:13:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024637999999999998 2020-08-07T15:13:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280870+0000", 2020-08-07T15:13:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001300142 2020-08-07T15:13:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282170+0000", 2020-08-07T15:13:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2099999999999997e-07 2020-08-07T15:13:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:57.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282170+0000", 2020-08-07T15:13:57.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000208001 2020-08-07T15:13:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282378+0000", 2020-08-07T15:13:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0579997480000001 2020-08-07T15:13:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118448 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:13:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.224474+0000", 2020-08-07T15:13:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 133.377739846, 2020-08-07T15:13:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0581683310000001, 2020-08-07T15:13:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:57.272 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118448 2020-08-07T15:13:57.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.272 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:57.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:57.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224474+0000", 2020-08-07T15:13:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224474+0000", 2020-08-07T15:13:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3804000000000002e-05 2020-08-07T15:13:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224528+0000", 2020-08-07T15:13:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5140000000000001e-06 2020-08-07T15:13:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224533+0000", 2020-08-07T15:13:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3315999999999999e-05 2020-08-07T15:13:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:57.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224546+0000", 2020-08-07T15:13:57.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7240000000000003e-06 2020-08-07T15:13:57.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224553+0000", 2020-08-07T15:13:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002226263 2020-08-07T15:13:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226779+0000", 2020-08-07T15:13:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1362000000000001e-05 2020-08-07T15:13:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226791+0000", 2020-08-07T15:13:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.046092652 2020-08-07T15:13:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.272883+0000", 2020-08-07T15:13:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000115981 2020-08-07T15:13:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.272999+0000", 2020-08-07T15:13:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029648899999999999 2020-08-07T15:13:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:13:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.273296+0000", 2020-08-07T15:13:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001852521 2020-08-07T15:13:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275148+0000", 2020-08-07T15:13:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051948309999999996 2020-08-07T15:13:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280343+0000", 2020-08-07T15:13:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033301799999999998 2020-08-07T15:13:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280676+0000", 2020-08-07T15:13:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8400000000000004e-07 2020-08-07T15:13:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280677+0000", 2020-08-07T15:13:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9652000000000002e-05 2020-08-07T15:13:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280756+0000", 2020-08-07T15:13:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017072699999999999 2020-08-07T15:13:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280927+0000", 2020-08-07T15:13:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015026530000000001 2020-08-07T15:13:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282430+0000", 2020-08-07T15:13:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4700000000000001e-07 2020-08-07T15:13:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282430+0000", 2020-08-07T15:13:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021169299999999999 2020-08-07T15:13:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:57.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282642+0000", 2020-08-07T15:13:57.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0581683310000001 2020-08-07T15:13:57.316 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.316 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:57.317 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.317 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:67303 6.a 6:5c7a55ef:::benchmark_data_smithi074_63843_object8412:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e265)", 2020-08-07T15:13:57.318 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:56.271945+0000", 2020-08-07T15:13:57.318 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.330268105999998, 2020-08-07T15:13:57.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.007577197, 2020-08-07T15:13:57.318 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:57.321 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:57.321 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:57.322 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:13:57.322 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:57.322 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 67303 2020-08-07T15:13:57.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.322 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:57.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:57.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.271945+0000", 2020-08-07T15:13:57.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:57.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:57.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.271945+0000", 2020-08-07T15:13:57.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9969000000000002e-05 2020-08-07T15:13:57.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:57.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.271995+0000", 2020-08-07T15:13:57.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3690000000000001e-06 2020-08-07T15:13:57.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:57.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.272003+0000", 2020-08-07T15:13:57.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2183000000000001e-05 2020-08-07T15:13:57.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:57.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.272025+0000", 2020-08-07T15:13:57.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7299999999999999e-06 2020-08-07T15:13:57.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:57.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.272032+0000", 2020-08-07T15:13:57.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5831e-05 2020-08-07T15:13:57.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.272108+0000", 2020-08-07T15:13:57.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2233e-05 2020-08-07T15:13:57.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:57.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.272120+0000", 2020-08-07T15:13:57.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27969333699999999 2020-08-07T15:13:57.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.551813+0000", 2020-08-07T15:13:57.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8549999999999997e-06 2020-08-07T15:13:57.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:57.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.551823+0000", 2020-08-07T15:13:57.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.71905319899999998 2020-08-07T15:13:57.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.270876+0000", 2020-08-07T15:13:57.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116937 2020-08-07T15:13:57.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:57.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.270993+0000", 2020-08-07T15:13:57.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020587399999999999 2020-08-07T15:13:57.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,5,7", 2020-08-07T15:13:57.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.271199+0000", 2020-08-07T15:13:57.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023173429999999999 2020-08-07T15:13:57.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:57.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.273516+0000", 2020-08-07T15:13:57.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053300810000000004 2020-08-07T15:13:57.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.278846+0000", 2020-08-07T15:13:57.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1699999999999998e-07 2020-08-07T15:13:57.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:57.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.278847+0000", 2020-08-07T15:13:57.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7559000000000003e-05 2020-08-07T15:13:57.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:57.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.278894+0000", 2020-08-07T15:13:57.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000119676 2020-08-07T15:13:57.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.279014+0000", 2020-08-07T15:13:57.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039420500000000001 2020-08-07T15:13:57.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.279408+0000", 2020-08-07T15:13:57.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0200000000000003e-07 2020-08-07T15:13:57.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:57.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.279409+0000", 2020-08-07T15:13:57.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113897 2020-08-07T15:13:57.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:57.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.279523+0000", 2020-08-07T15:13:57.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.007577197 2020-08-07T15:13:57.395 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.395 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:57.396 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.396 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:67304 6.a 6:5c7a55ef:::benchmark_data_smithi074_63843_object8412:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e265)", 2020-08-07T15:13:57.396 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:56.410012+0000", 2020-08-07T15:13:57.397 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.192201355000002, 2020-08-07T15:13:57.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.86966374199999996, 2020-08-07T15:13:57.397 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:57.397 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:57.404 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:57.404 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:13:57.404 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:57.404 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 67304 2020-08-07T15:13:57.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.405 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:57.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:57.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.410012+0000", 2020-08-07T15:13:57.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:57.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:57.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.410012+0000", 2020-08-07T15:13:57.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1640000000000002e-05 2020-08-07T15:13:57.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:57.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.410044+0000", 2020-08-07T15:13:57.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8099999999999997e-06 2020-08-07T15:13:57.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:57.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.410049+0000", 2020-08-07T15:13:57.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1989999999999995e-06 2020-08-07T15:13:57.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:57.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.410058+0000", 2020-08-07T15:13:57.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6249999999999998e-06 2020-08-07T15:13:57.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:57.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.410066+0000", 2020-08-07T15:13:57.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2998999999999998e-05 2020-08-07T15:13:57.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.410129+0000", 2020-08-07T15:13:57.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6420000000000002e-06 2020-08-07T15:13:57.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:57.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.410138+0000", 2020-08-07T15:13:57.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14171407799999999 2020-08-07T15:13:57.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.551852+0000", 2020-08-07T15:13:57.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0237e-05 2020-08-07T15:13:57.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:57.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.551862+0000", 2020-08-07T15:13:57.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.71982975800000004 2020-08-07T15:13:57.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.271692+0000", 2020-08-07T15:13:57.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0882000000000002e-05 2020-08-07T15:13:57.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:57.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.271783+0000", 2020-08-07T15:13:57.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000212067 2020-08-07T15:13:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,5,7", 2020-08-07T15:13:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.271995+0000", 2020-08-07T15:13:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021518420000000002 2020-08-07T15:13:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.274147+0000", 2020-08-07T15:13:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0047817959999999996 2020-08-07T15:13:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.278929+0000", 2020-08-07T15:13:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2099999999999997e-07 2020-08-07T15:13:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.278929+0000", 2020-08-07T15:13:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4972000000000002e-05 2020-08-07T15:13:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.278974+0000", 2020-08-07T15:13:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5584e-05 2020-08-07T15:13:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.279050+0000", 2020-08-07T15:13:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051383899999999998 2020-08-07T15:13:57.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.279564+0000", 2020-08-07T15:13:57.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9900000000000001e-07 2020-08-07T15:13:57.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:57.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.279564+0000", 2020-08-07T15:13:57.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111852 2020-08-07T15:13:57.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:57.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.279676+0000", 2020-08-07T15:13:57.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.86966374199999996 2020-08-07T15:13:57.460 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.460 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:57.460 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.461 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:128347 6.a 6:56780bc2:::benchmark_data_smithi074_63843_object16043:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e293)", 2020-08-07T15:13:57.461 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:13:38.566993+0000", 2020-08-07T15:13:57.462 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.035220281000001, 2020-08-07T15:13:57.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73194675099999995, 2020-08-07T15:13:57.462 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:57.462 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:57.462 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:57.463 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:13:57.463 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:57.463 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128347 2020-08-07T15:13:57.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.463 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:57.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:57.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.566993+0000", 2020-08-07T15:13:57.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:57.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:57.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.566993+0000", 2020-08-07T15:13:57.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3297000000000001e-05 2020-08-07T15:13:57.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:57.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.567047+0000", 2020-08-07T15:13:57.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5899999999999996e-06 2020-08-07T15:13:57.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:57.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.567053+0000", 2020-08-07T15:13:57.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3277e-05 2020-08-07T15:13:57.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:57.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.567066+0000", 2020-08-07T15:13:57.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2949999999999999e-06 2020-08-07T15:13:57.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:57.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.567076+0000", 2020-08-07T15:13:57.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1841999999999997e-05 2020-08-07T15:13:57.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.567118+0000", 2020-08-07T15:13:57.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4059999999999999e-06 2020-08-07T15:13:57.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:13:57.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.567125+0000", 2020-08-07T15:13:57.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.71691992999999998 2020-08-07T15:13:57.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:39.284045+0000", 2020-08-07T15:13:57.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109627 2020-08-07T15:13:57.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:57.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:39.284155+0000", 2020-08-07T15:13:57.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000203873 2020-08-07T15:13:57.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,7", 2020-08-07T15:13:57.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:39.284358+0000", 2020-08-07T15:13:57.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050021429999999997 2020-08-07T15:13:57.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:57.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:39.289361+0000", 2020-08-07T15:13:57.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3099999999999997e-07 2020-08-07T15:13:57.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:57.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:39.289361+0000", 2020-08-07T15:13:57.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4251e-05 2020-08-07T15:13:57.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:57.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:39.289405+0000", 2020-08-07T15:13:57.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005939827 2020-08-07T15:13:57.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:39.295345+0000", 2020-08-07T15:13:57.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034308920000000001 2020-08-07T15:13:57.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:57.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:39.298776+0000", 2020-08-07T15:13:57.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5499999999999997e-07 2020-08-07T15:13:57.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:57.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:39.298777+0000", 2020-08-07T15:13:57.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000163115 2020-08-07T15:13:57.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:57.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:39.298940+0000", 2020-08-07T15:13:57.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73194675099999995 2020-08-07T15:13:57.491 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.491 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:57.492 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5481.0:149908 6.3 e298/269 ondisk, result = 0)", 2020-08-07T15:13:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:13:51.271126+0000", 2020-08-07T15:13:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.3310877699999999, 2020-08-07T15:13:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.89109337700000002, 2020-08-07T15:13:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:51.271126+0000", 2020-08-07T15:13:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:51.271126+0000", 2020-08-07T15:13:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2252999999999998e-05 2020-08-07T15:13:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:51.271158+0000", 2020-08-07T15:13:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3130000000000001e-06 2020-08-07T15:13:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:51.271165+0000", 2020-08-07T15:13:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.89092263199999999 2020-08-07T15:13:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:52.162088+0000", 2020-08-07T15:13:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0300000000000004e-06 2020-08-07T15:13:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:52.162092+0000", 2020-08-07T15:13:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5454999999999997e-05 2020-08-07T15:13:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:52.162147+0000", 2020-08-07T15:13:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5919999999999997e-06 2020-08-07T15:13:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:52.162154+0000", 2020-08-07T15:13:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5102000000000001e-05 2020-08-07T15:13:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:57.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:57.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:57.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:52.162219+0000", 2020-08-07T15:13:57.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.89109337700000002 2020-08-07T15:13:57.521 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.521 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:57.521 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.521 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:57.522 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:57.522 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:57.524 INFO:teuthology.orchestra.run.smithi074.stderr:enabled application 'rados' on pool 'unique_pool_4' 2020-08-07T15:13:57.525 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:57.522+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:57.526 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:13:57.560 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set nodeep-scrub 2020-08-07T15:13:57.660 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:57.625+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:57.699 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:57.700 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:13:57.700 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2020-08-07T15:13:57.700 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:57.707 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:13:57.750 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:57.747+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:57.829 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:57.827+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:57.874 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:57.874 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:13:57.887 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:13:57.887 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:13:57.887 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:57.889 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:13:57.931 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:57.930+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:58.033 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:58.030+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:58.052 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:13:58.052 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:13:58.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:13:58.053 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:13:58.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.054 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:15576 3.e 3:727724a5:::benchmark_data_smithi074_35447_object1946:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e27)", 2020-08-07T15:13:58.055 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:13:58.055 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 431.37897671299999, 2020-08-07T15:13:58.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999, 2020-08-07T15:13:58.056 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:58.056 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:58.056 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:58.056 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:13:58.057 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:13:58.057 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15576 2020-08-07T15:13:58.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.057 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:58.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:58.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:13:58.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:58.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:58.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666635+0000", 2020-08-07T15:13:58.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7156e-05 2020-08-07T15:13:58.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:58.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666682+0000", 2020-08-07T15:13:58.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0209999999999999e-06 2020-08-07T15:13:58.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:58.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666687+0000", 2020-08-07T15:13:58.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0730999999999999e-05 2020-08-07T15:13:58.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:58.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666698+0000", 2020-08-07T15:13:58.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2020-08-07T15:13:58.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.666702+0000", 2020-08-07T15:13:58.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023978459999999999 2020-08-07T15:13:58.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:58.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669100+0000", 2020-08-07T15:13:58.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9664000000000001e-05 2020-08-07T15:13:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669190+0000", 2020-08-07T15:13:58.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015957100000000001 2020-08-07T15:13:58.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,4", 2020-08-07T15:13:58.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.669350+0000", 2020-08-07T15:13:58.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016470149999999999 2020-08-07T15:13:58.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:58.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:13:58.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1299999999999999e-07 2020-08-07T15:13:58.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:58.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.670997+0000", 2020-08-07T15:13:58.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5269000000000002e-05 2020-08-07T15:13:58.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:58.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.671022+0000", 2020-08-07T15:13:58.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031414400000000001 2020-08-07T15:13:58.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:58.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:46.674163+0000", 2020-08-07T15:13:58.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.45641417400000001 2020-08-07T15:13:58.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:58.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:13:58.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0399999999999995e-07 2020-08-07T15:13:58.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:58.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130578+0000", 2020-08-07T15:13:58.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3343999999999997e-05 2020-08-07T15:13:58.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:58.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:06:47.130622+0000", 2020-08-07T15:13:58.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.46398676599999999 2020-08-07T15:13:58.112 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.112 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:58.113 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.113 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:106248 3.f 3:f950fd9d:::benchmark_data_smithi074_35447_object13280:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:13:58.113 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:13:58.114 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 370.685447173, 2020-08-07T15:13:58.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002, 2020-08-07T15:13:58.114 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:58.114 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:58.114 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:58.115 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:13:58.115 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:13:58.115 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 106248 2020-08-07T15:13:58.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.116 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:58.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:58.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:13:58.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:58.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:58.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360164+0000", 2020-08-07T15:13:58.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7184e-05 2020-08-07T15:13:58.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:58.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360222+0000", 2020-08-07T15:13:58.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0906e-05 2020-08-07T15:13:58.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:58.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360233+0000", 2020-08-07T15:13:58.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2972999999999999e-05 2020-08-07T15:13:58.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:58.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360246+0000", 2020-08-07T15:13:58.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0114e-05 2020-08-07T15:13:58.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360256+0000", 2020-08-07T15:13:58.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1303999999999997e-05 2020-08-07T15:13:58.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:58.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360287+0000", 2020-08-07T15:13:58.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3798000000000001e-05 2020-08-07T15:13:58.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:58.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360351+0000", 2020-08-07T15:13:58.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111308 2020-08-07T15:13:58.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,4", 2020-08-07T15:13:58.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.360462+0000", 2020-08-07T15:13:58.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003480674 2020-08-07T15:13:58.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:58.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:13:58.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0900000000000001e-07 2020-08-07T15:13:58.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:58.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.363943+0000", 2020-08-07T15:13:58.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8899999999999994e-05 2020-08-07T15:13:58.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:58.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.364012+0000", 2020-08-07T15:13:58.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.35291637199999998 2020-08-07T15:13:58.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:58.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.716929+0000", 2020-08-07T15:13:58.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.076244233999999994 2020-08-07T15:13:58.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:58.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793173+0000", 2020-08-07T15:13:58.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6109999999999999e-06 2020-08-07T15:13:58.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:58.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793175+0000", 2020-08-07T15:13:58.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7492999999999999e-05 2020-08-07T15:13:58.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:58.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793242+0000", 2020-08-07T15:13:58.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.43307758000000002 2020-08-07T15:13:58.136 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.136 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:58.137 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.138 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:106248 3.f e59/20 ondisk, result = 0)", 2020-08-07T15:13:58.138 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:13:58.138 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 370.68018388799999, 2020-08-07T15:13:58.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001, 2020-08-07T15:13:58.138 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:58.138 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:58.139 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:58.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:58.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:13:58.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:58.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:58.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365428+0000", 2020-08-07T15:13:58.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7518999999999999e-05 2020-08-07T15:13:58.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:58.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365445+0000", 2020-08-07T15:13:58.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-06 2020-08-07T15:13:58.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.365449+0000", 2020-08-07T15:13:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42764147600000002 2020-08-07T15:13:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793091+0000", 2020-08-07T15:13:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6500000000000002e-06 2020-08-07T15:13:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793099+0000", 2020-08-07T15:13:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3664e-05 2020-08-07T15:13:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793153+0000", 2020-08-07T15:13:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1078e-05 2020-08-07T15:13:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793164+0000", 2020-08-07T15:13:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3404000000000003e-05 2020-08-07T15:13:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:47.793248+0000", 2020-08-07T15:13:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42781983000000001 2020-08-07T15:13:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4329.0:107952 3.d e61/35)", 2020-08-07T15:13:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:13:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 367.71925418000001, 2020-08-07T15:13:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001, 2020-08-07T15:13:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:13:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326357+0000", 2020-08-07T15:13:58.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8826000000000003e-05 2020-08-07T15:13:58.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:58.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326406+0000", 2020-08-07T15:13:58.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9199999999999998e-06 2020-08-07T15:13:58.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:58.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.326413+0000", 2020-08-07T15:13:58.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75082853500000002 2020-08-07T15:13:58.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:58.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077242+0000", 2020-08-07T15:13:58.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1479999999999999e-06 2020-08-07T15:13:58.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077246+0000", 2020-08-07T15:13:58.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3331999999999998e-05 2020-08-07T15:13:58.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:58.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077289+0000", 2020-08-07T15:13:58.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8977000000000001e-05 2020-08-07T15:13:58.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:58.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.077308+0000", 2020-08-07T15:13:58.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75095073800000001 2020-08-07T15:13:58.158 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.159 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:58.159 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.159 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31696 4.0 4:0335eed9:::benchmark_data_smithi074_44586_object3961:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e100)", 2020-08-07T15:13:58.159 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:13:58.160 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 309.94134662099998, 2020-08-07T15:13:58.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237, 2020-08-07T15:13:58.160 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:58.160 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:58.160 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:58.161 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:13:58.161 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:13:58.161 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31696 2020-08-07T15:13:58.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.161 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:58.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:58.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:13:58.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:58.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:58.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104265+0000", 2020-08-07T15:13:58.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7654e-05 2020-08-07T15:13:58.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104293+0000", 2020-08-07T15:13:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.061e-06 2020-08-07T15:13:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:58.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104296+0000", 2020-08-07T15:13:58.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0859999999999998e-06 2020-08-07T15:13:58.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:58.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104302+0000", 2020-08-07T15:13:58.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8030000000000001e-06 2020-08-07T15:13:58.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.104305+0000", 2020-08-07T15:13:58.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037634769999999999 2020-08-07T15:13:58.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:58.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108068+0000", 2020-08-07T15:13:58.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8814999999999998e-05 2020-08-07T15:13:58.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:58.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108137+0000", 2020-08-07T15:13:58.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111455 2020-08-07T15:13:58.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,1", 2020-08-07T15:13:58.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.108248+0000", 2020-08-07T15:13:58.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028756820000000001 2020-08-07T15:13:58.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:58.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111124+0000", 2020-08-07T15:13:58.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061054499999999999 2020-08-07T15:13:58.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:58.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.111735+0000", 2020-08-07T15:13:58.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.423966167 2020-08-07T15:13:58.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:58.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535701+0000", 2020-08-07T15:13:58.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4929999999999999e-06 2020-08-07T15:13:58.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:58.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535702+0000", 2020-08-07T15:13:58.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2571999999999997e-05 2020-08-07T15:13:58.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:58.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535755+0000", 2020-08-07T15:13:58.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1427000000000003e-05 2020-08-07T15:13:58.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:58.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535786+0000", 2020-08-07T15:13:58.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.431521237 2020-08-07T15:13:58.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.184 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:58.184 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.184 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:31696 4.0 e100/81 ondisk, result = 0)", 2020-08-07T15:13:58.185 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:13:58.185 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 309.93173206300003, 2020-08-07T15:13:58.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954, 2020-08-07T15:13:58.185 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:58.185 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:58.186 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:58.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:58.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:13:58.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:58.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:58.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113880+0000", 2020-08-07T15:13:58.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1185000000000001e-05 2020-08-07T15:13:58.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:58.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113901+0000", 2020-08-07T15:13:58.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2930000000000002e-06 2020-08-07T15:13:58.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:58.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.113905+0000", 2020-08-07T15:13:58.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.42168640400000001 2020-08-07T15:13:58.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:58.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535591+0000", 2020-08-07T15:13:58.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2820000000000002e-06 2020-08-07T15:13:58.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535598+0000", 2020-08-07T15:13:58.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6578999999999999e-05 2020-08-07T15:13:58.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:58.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535684+0000", 2020-08-07T15:13:58.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8449999999999992e-06 2020-08-07T15:13:58.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:58.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535693+0000", 2020-08-07T15:13:58.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7366000000000003e-05 2020-08-07T15:13:58.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:58.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:48.535791+0000", 2020-08-07T15:13:58.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.421910954 2020-08-07T15:13:58.209 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.209 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:58.209 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.210 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4726.0:37024 4.0 e106/81)", 2020-08-07T15:13:58.210 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:13:58.211 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 303.81694316900001, 2020-08-07T15:13:58.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999, 2020-08-07T15:13:58.211 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:58.211 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:13:58.211 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:58.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:58.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:13:58.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:58.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:58.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228668+0000", 2020-08-07T15:13:58.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4926000000000001e-05 2020-08-07T15:13:58.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:58.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228683+0000", 2020-08-07T15:13:58.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1280000000000002e-06 2020-08-07T15:13:58.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:58.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.228686+0000", 2020-08-07T15:13:58.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69740012100000004 2020-08-07T15:13:58.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:58.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926086+0000", 2020-08-07T15:13:58.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3970000000000004e-06 2020-08-07T15:13:58.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926090+0000", 2020-08-07T15:13:58.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113526 2020-08-07T15:13:58.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:58.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926204+0000", 2020-08-07T15:13:58.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8824000000000001e-05 2020-08-07T15:13:58.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:58.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:54.926222+0000", 2020-08-07T15:13:58.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.69755392199999999 2020-08-07T15:13:58.221 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.221 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:58.221 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.222 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42503 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2615, mlcod=112'2607)", 2020-08-07T15:13:58.222 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:13:58.222 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 297.74937125299999, 2020-08-07T15:13:58.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998, 2020-08-07T15:13:58.223 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:58.223 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:58.223 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:58.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:58.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:13:58.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:58.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:58.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296240+0000", 2020-08-07T15:13:58.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270999999999998e-05 2020-08-07T15:13:58.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:58.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296292+0000", 2020-08-07T15:13:58.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4729999999999997e-06 2020-08-07T15:13:58.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:58.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.296298+0000", 2020-08-07T15:13:58.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.448795049 2020-08-07T15:13:58.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:58.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745093+0000", 2020-08-07T15:13:58.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0140000000000001e-06 2020-08-07T15:13:58.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745100+0000", 2020-08-07T15:13:58.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2596999999999998e-05 2020-08-07T15:13:58.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:58.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745153+0000", 2020-08-07T15:13:58.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1662000000000003e-05 2020-08-07T15:13:58.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:58.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.745184+0000", 2020-08-07T15:13:58.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093174800000000004 2020-08-07T15:13:58.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:58.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746116+0000", 2020-08-07T15:13:58.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0926999999999994e-05 2020-08-07T15:13:58.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:58.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.746187+0000", 2020-08-07T15:13:58.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44994674099999998 2020-08-07T15:13:58.238 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.239 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:58.239 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.239 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42504 4.8 e113/81 4:12c902cf:::benchmark_data_smithi074_44586_object5312:head v 113'2616, mlcod=113'2609)", 2020-08-07T15:13:58.240 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:13:58.240 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 297.74603436000001, 2020-08-07T15:13:58.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998, 2020-08-07T15:13:58.240 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:58.240 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:58.241 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:58.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:58.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:13:58.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:58.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:58.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299577+0000", 2020-08-07T15:13:58.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2154000000000002e-05 2020-08-07T15:13:58.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:58.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299629+0000", 2020-08-07T15:13:58.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1929999999999998e-06 2020-08-07T15:13:58.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:58.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.299636+0000", 2020-08-07T15:13:58.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44766315400000001 2020-08-07T15:13:58.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:58.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747299+0000", 2020-08-07T15:13:58.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5889999999999999e-06 2020-08-07T15:13:58.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747305+0000", 2020-08-07T15:13:58.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3094999999999999e-05 2020-08-07T15:13:58.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:58.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747358+0000", 2020-08-07T15:13:58.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3283e-05 2020-08-07T15:13:58.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:58.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.747392+0000", 2020-08-07T15:13:58.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015319509999999999 2020-08-07T15:13:58.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:58.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748924+0000", 2020-08-07T15:13:58.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5235999999999996e-05 2020-08-07T15:13:58.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:58.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.748999+0000", 2020-08-07T15:13:58.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.44942165499999998 2020-08-07T15:13:58.258 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.258 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:58.258 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.259 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4726.0:42520 4.8 e113/81 4:13071de5:::benchmark_data_smithi074_44586_object5314:head v 113'2624, mlcod=113'2618)", 2020-08-07T15:13:58.259 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:13:58.259 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 297.28009015100002, 2020-08-07T15:13:58.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001, 2020-08-07T15:13:58.260 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:58.260 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:58.260 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:58.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:58.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:13:58.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:58.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:58.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765521+0000", 2020-08-07T15:13:58.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3967999999999995e-05 2020-08-07T15:13:58.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:58.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765585+0000", 2020-08-07T15:13:58.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0949999999999998e-06 2020-08-07T15:13:58.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:58.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.765593+0000", 2020-08-07T15:13:58.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52703464200000005 2020-08-07T15:13:58.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:58.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292627+0000", 2020-08-07T15:13:58.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6929999999999996e-06 2020-08-07T15:13:58.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292636+0000", 2020-08-07T15:13:58.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9292000000000001e-05 2020-08-07T15:13:58.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:58.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292695+0000", 2020-08-07T15:13:58.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5395999999999999e-05 2020-08-07T15:13:58.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:58.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.292721+0000", 2020-08-07T15:13:58.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00090361399999999996 2020-08-07T15:13:58.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:58.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293624+0000", 2020-08-07T15:13:58.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1761000000000002e-05 2020-08-07T15:13:58.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:58.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.293666+0000", 2020-08-07T15:13:58.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.52814446100000001 2020-08-07T15:13:58.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.270 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:58.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.271 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:85552 5.9 5:9d69b7cc:::benchmark_data_smithi074_53697_object10693:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e202)", 2020-08-07T15:13:58.271 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:13:58.271 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 161.23809437200001, 2020-08-07T15:13:58.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54429374399999997, 2020-08-07T15:13:58.271 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:58.272 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:58.272 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:58.272 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:58.272 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:58.272 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 85552 2020-08-07T15:13:58.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.273 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:58.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:58.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:13:58.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:58.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:58.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807517+0000", 2020-08-07T15:13:58.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2691000000000001e-05 2020-08-07T15:13:58.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:58.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807570+0000", 2020-08-07T15:13:58.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0619999999999997e-06 2020-08-07T15:13:58.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:58.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807576+0000", 2020-08-07T15:13:58.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1558e-05 2020-08-07T15:13:58.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:58.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807588+0000", 2020-08-07T15:13:58.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4600000000000002e-06 2020-08-07T15:13:58.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.807593+0000", 2020-08-07T15:13:58.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022126279999999999 2020-08-07T15:13:58.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:58.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.809806+0000", 2020-08-07T15:13:58.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104868 2020-08-07T15:13:58.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:58.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.809911+0000", 2020-08-07T15:13:58.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000186019 2020-08-07T15:13:58.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:13:58.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.810097+0000", 2020-08-07T15:13:58.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037966739999999999 2020-08-07T15:13:58.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:58.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813893+0000", 2020-08-07T15:13:58.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1310000000000001e-06 2020-08-07T15:13:58.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:58.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813894+0000", 2020-08-07T15:13:58.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4704999999999999e-05 2020-08-07T15:13:58.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:58.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813949+0000", 2020-08-07T15:13:58.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015487280000000001 2020-08-07T15:13:58.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:58.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.815498+0000", 2020-08-07T15:13:58.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53621442900000005 2020-08-07T15:13:58.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:58.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351712+0000", 2020-08-07T15:13:58.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8199999999999998e-07 2020-08-07T15:13:58.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:58.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351713+0000", 2020-08-07T15:13:58.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7909000000000001e-05 2020-08-07T15:13:58.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:58.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351811+0000", 2020-08-07T15:13:58.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.54429374399999997 2020-08-07T15:13:58.316 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.316 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:58.316 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.317 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:85552 5.9 e202/164 ondisk, result = 0)", 2020-08-07T15:13:58.318 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:13:58.318 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 161.23186797, 2020-08-07T15:13:58.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53813836800000003, 2020-08-07T15:13:58.319 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:58.319 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:58.319 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:58.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:58.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:13:58.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:58.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:58.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813744+0000", 2020-08-07T15:13:58.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8833e-05 2020-08-07T15:13:58.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:58.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813772+0000", 2020-08-07T15:13:58.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8459999999999999e-06 2020-08-07T15:13:58.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:58.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.813777+0000", 2020-08-07T15:13:58.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53732224900000003 2020-08-07T15:13:58.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:58.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351100+0000", 2020-08-07T15:13:58.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1869999999999996e-06 2020-08-07T15:13:58.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351105+0000", 2020-08-07T15:13:58.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00058858100000000002 2020-08-07T15:13:58.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:58.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351693+0000", 2020-08-07T15:13:58.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4229999999999994e-06 2020-08-07T15:13:58.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:58.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351703+0000", 2020-08-07T15:13:58.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000179249 2020-08-07T15:13:58.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:58.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351882+0000", 2020-08-07T15:13:58.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53813836800000003 2020-08-07T15:13:58.346 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.346 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:58.346 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.347 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:85552 5.9 5:9d69b7cc:::benchmark_data_smithi074_53697_object10693:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e202)", 2020-08-07T15:13:58.347 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:13:58.348 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 161.22711984599999, 2020-08-07T15:13:58.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53326610600000002, 2020-08-07T15:13:58.348 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:58.348 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "delayed", 2020-08-07T15:13:58.349 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:58.349 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:58.349 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:58.349 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 85552 2020-08-07T15:13:58.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.351 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:58.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:58.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:13:58.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:58.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:58.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818492+0000", 2020-08-07T15:13:58.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5001e-05 2020-08-07T15:13:58.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:58.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818517+0000", 2020-08-07T15:13:58.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5000000000000002e-06 2020-08-07T15:13:58.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:58.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818519+0000", 2020-08-07T15:13:58.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0699999999999997e-06 2020-08-07T15:13:58.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:58.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818524+0000", 2020-08-07T15:13:58.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2910000000000002e-06 2020-08-07T15:13:58.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818527+0000", 2020-08-07T15:13:58.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030912499999999999 2020-08-07T15:13:58.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:58.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818836+0000", 2020-08-07T15:13:58.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2908000000000002e-05 2020-08-07T15:13:58.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for ondisk", 2020-08-07T15:13:58.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:16.818889+0000", 2020-08-07T15:13:58.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53286921099999995 2020-08-07T15:13:58.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:58.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:17.351758+0000", 2020-08-07T15:13:58.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.53326610600000002 2020-08-07T15:13:58.364 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.365 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:58.365 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.366 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:89464 5.9 5:94c69ba5:::benchmark_data_smithi074_53697_object11182:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e207)", 2020-08-07T15:13:58.366 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:13:58.366 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 156.629279892, 2020-08-07T15:13:58.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67692916199999997, 2020-08-07T15:13:58.367 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:58.367 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:58.367 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:58.367 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:58.368 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:58.368 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 89464 2020-08-07T15:13:58.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.369 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:58.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:58.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:13:58.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:58.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:58.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416332+0000", 2020-08-07T15:13:58.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0074000000000002e-05 2020-08-07T15:13:58.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:58.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416382+0000", 2020-08-07T15:13:58.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5000000000000001e-06 2020-08-07T15:13:58.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:58.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416386+0000", 2020-08-07T15:13:58.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2447e-05 2020-08-07T15:13:58.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:58.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416399+0000", 2020-08-07T15:13:58.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1279999999999996e-06 2020-08-07T15:13:58.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.416405+0000", 2020-08-07T15:13:58.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016641805999999999 2020-08-07T15:13:58.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:58.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433047+0000", 2020-08-07T15:13:58.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107995 2020-08-07T15:13:58.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:58.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433155+0000", 2020-08-07T15:13:58.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018818599999999999 2020-08-07T15:13:58.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4,6", 2020-08-07T15:13:58.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.433343+0000", 2020-08-07T15:13:58.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039911679999999998 2020-08-07T15:13:58.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:58.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437334+0000", 2020-08-07T15:13:58.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2020-08-07T15:13:58.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:58.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437335+0000", 2020-08-07T15:13:58.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6351000000000002e-05 2020-08-07T15:13:58.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:58.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.437401+0000", 2020-08-07T15:13:58.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001267949 2020-08-07T15:13:58.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:58.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438669+0000", 2020-08-07T15:13:58.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65454035600000005 2020-08-07T15:13:58.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:58.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093209+0000", 2020-08-07T15:13:58.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7859999999999999e-06 2020-08-07T15:13:58.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:58.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093211+0000", 2020-08-07T15:13:58.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9665e-05 2020-08-07T15:13:58.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:58.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093261+0000", 2020-08-07T15:13:58.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67692916199999997 2020-08-07T15:13:58.387 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.387 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:58.387 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.388 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5099.0:89464 5.9 e207/164 ondisk, result = 0)", 2020-08-07T15:13:58.388 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:13:58.388 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 156.60665012999999, 2020-08-07T15:13:58.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65430647200000003, 2020-08-07T15:13:58.389 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:58.389 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:13:58.389 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:58.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:58.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:13:58.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:58.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:58.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438961+0000", 2020-08-07T15:13:58.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6529000000000002e-05 2020-08-07T15:13:58.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:58.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.438998+0000", 2020-08-07T15:13:58.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.567e-06 2020-08-07T15:13:58.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:58.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.439005+0000", 2020-08-07T15:13:58.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65408475099999996 2020-08-07T15:13:58.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:58.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093089+0000", 2020-08-07T15:13:58.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6880000000000002e-06 2020-08-07T15:13:58.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093096+0000", 2020-08-07T15:13:58.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4980000000000002e-05 2020-08-07T15:13:58.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:58.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093191+0000", 2020-08-07T15:13:58.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3090000000000001e-06 2020-08-07T15:13:58.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:58.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093200+0000", 2020-08-07T15:13:58.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7648e-05 2020-08-07T15:13:58.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:58.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.093268+0000", 2020-08-07T15:13:58.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.65430647200000003 2020-08-07T15:13:58.399 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.399 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:58.399 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.400 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:114080 5.4 5:24adcd54:::benchmark_data_smithi074_53697_object14259:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e215)", 2020-08-07T15:13:58.402 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:13:58.402 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 138.44594267299999, 2020-08-07T15:13:58.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.57290862300000001, 2020-08-07T15:13:58.403 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:58.403 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:58.403 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:58.403 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:13:58.404 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:13:58.404 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 114080 2020-08-07T15:13:58.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.404 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:58.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:58.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:13:58.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:58.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:58.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599669+0000", 2020-08-07T15:13:58.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9238e-05 2020-08-07T15:13:58.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:58.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599698+0000", 2020-08-07T15:13:58.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6260000000000003e-06 2020-08-07T15:13:58.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:58.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599703+0000", 2020-08-07T15:13:58.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5577999999999999e-05 2020-08-07T15:13:58.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:58.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599718+0000", 2020-08-07T15:13:58.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9869999999999999e-06 2020-08-07T15:13:58.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.599726+0000", 2020-08-07T15:13:58.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00077929599999999996 2020-08-07T15:13:58.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:58.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600506+0000", 2020-08-07T15:13:58.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1717000000000003e-05 2020-08-07T15:13:58.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:58.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600597+0000", 2020-08-07T15:13:58.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011828999999999999 2020-08-07T15:13:58.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2,5", 2020-08-07T15:13:58.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.600716+0000", 2020-08-07T15:13:58.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020861479999999999 2020-08-07T15:13:58.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:58.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602802+0000", 2020-08-07T15:13:58.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4600000000000005e-07 2020-08-07T15:13:58.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:58.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602802+0000", 2020-08-07T15:13:58.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9165e-05 2020-08-07T15:13:58.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:58.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.602862+0000", 2020-08-07T15:13:58.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0093485080000000002 2020-08-07T15:13:58.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:58.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:39.612210+0000", 2020-08-07T15:13:58.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.56030197699999995 2020-08-07T15:13:58.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:58.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172512+0000", 2020-08-07T15:13:58.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8800000000000001e-07 2020-08-07T15:13:58.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:58.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172513+0000", 2020-08-07T15:13:58.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4659000000000006e-05 2020-08-07T15:13:58.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:58.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:40.172578+0000", 2020-08-07T15:13:58.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.57290862300000001 2020-08-07T15:13:58.430 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.430 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:58.431 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.431 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:48616 6.9 6:955383e0:::benchmark_data_smithi074_63843_object6076:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e256)", 2020-08-07T15:13:58.431 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:43.702845+0000", 2020-08-07T15:13:58.432 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.342766376, 2020-08-07T15:13:58.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40912302499999997, 2020-08-07T15:13:58.432 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:58.432 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:58.432 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:13:58.433 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:13:58.433 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:13:58.433 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 48616 2020-08-07T15:13:58.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.434 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:58.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:58.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702845+0000", 2020-08-07T15:13:58.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:58.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:58.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702845+0000", 2020-08-07T15:13:58.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1931999999999997e-05 2020-08-07T15:13:58.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:58.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702897+0000", 2020-08-07T15:13:58.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0650000000000004e-06 2020-08-07T15:13:58.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:58.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702903+0000", 2020-08-07T15:13:58.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3084999999999999e-05 2020-08-07T15:13:58.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:58.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702916+0000", 2020-08-07T15:13:58.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4179999999999996e-06 2020-08-07T15:13:58.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.702922+0000", 2020-08-07T15:13:58.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045509080000000002 2020-08-07T15:13:58.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:58.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.707473+0000", 2020-08-07T15:13:58.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2619000000000005e-05 2020-08-07T15:13:58.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:58.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.707535+0000", 2020-08-07T15:13:58.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000126919 2020-08-07T15:13:58.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2", 2020-08-07T15:13:58.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.707662+0000", 2020-08-07T15:13:58.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001200135 2020-08-07T15:13:58.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:13:58.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.708862+0000", 2020-08-07T15:13:58.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0109999999999999e-06 2020-08-07T15:13:58.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:58.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.708863+0000", 2020-08-07T15:13:58.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8837999999999994e-05 2020-08-07T15:13:58.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:58.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.708932+0000", 2020-08-07T15:13:58.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054206710000000002 2020-08-07T15:13:58.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:58.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:43.714353+0000", 2020-08-07T15:13:58.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.39754767099999999 2020-08-07T15:13:58.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:13:58.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111901+0000", 2020-08-07T15:13:58.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.023e-06 2020-08-07T15:13:58.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:13:58.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111902+0000", 2020-08-07T15:13:58.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6730000000000007e-05 2020-08-07T15:13:58.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:58.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:44.111968+0000", 2020-08-07T15:13:58.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.40912302499999997 2020-08-07T15:13:58.457 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.457 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:58.457 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.458 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5481.0:128184 6.8 e293/269 6:13c658be:::benchmark_data_smithi074_63843_object16022:head v 293'8152, mlcod=293'8147)", 2020-08-07T15:13:58.458 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:13:38.394036+0000", 2020-08-07T15:13:58.458 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.651575510000001, 2020-08-07T15:13:58.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.58484205899999997, 2020-08-07T15:13:58.459 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:58.459 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:58.459 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:58.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:58.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.394036+0000", 2020-08-07T15:13:58.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:58.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:58.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.394036+0000", 2020-08-07T15:13:58.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3461999999999996e-05 2020-08-07T15:13:58.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:58.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.394100+0000", 2020-08-07T15:13:58.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9980000000000003e-06 2020-08-07T15:13:58.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:58.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.394108+0000", 2020-08-07T15:13:58.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.58342110700000005 2020-08-07T15:13:58.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:58.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.977529+0000", 2020-08-07T15:13:58.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2359e-05 2020-08-07T15:13:58.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.977541+0000", 2020-08-07T15:13:58.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4761999999999998e-05 2020-08-07T15:13:58.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:58.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.977636+0000", 2020-08-07T15:13:58.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4950999999999999e-05 2020-08-07T15:13:58.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:58.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.977661+0000", 2020-08-07T15:13:58.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001160204 2020-08-07T15:13:58.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:58.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.978821+0000", 2020-08-07T15:13:58.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7216000000000002e-05 2020-08-07T15:13:58.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:58.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.978878+0000", 2020-08-07T15:13:58.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.58484205899999997 2020-08-07T15:13:58.469 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.469 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:58.469 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.471 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5481.0:149899 6.6 e298/230 6:60c8cef7:::benchmark_data_smithi074_63843_object16647:head v 298'9423, mlcod=298'9420)", 2020-08-07T15:13:58.472 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:13:51.262585+0000", 2020-08-07T15:13:58.472 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.783027047, 2020-08-07T15:13:58.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.808915045, 2020-08-07T15:13:58.472 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:58.472 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:58.473 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:58.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:58.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:51.262585+0000", 2020-08-07T15:13:58.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:58.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:58.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:51.262585+0000", 2020-08-07T15:13:58.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9212000000000003e-05 2020-08-07T15:13:58.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:58.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:51.262624+0000", 2020-08-07T15:13:58.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0129999999999999e-06 2020-08-07T15:13:58.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:58.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:51.262630+0000", 2020-08-07T15:13:58.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.80795992599999999 2020-08-07T15:13:58.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:58.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:52.070590+0000", 2020-08-07T15:13:58.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1303e-05 2020-08-07T15:13:58.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:52.070601+0000", 2020-08-07T15:13:58.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013806300000000001 2020-08-07T15:13:58.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:58.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:52.070739+0000", 2020-08-07T15:13:58.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0618999999999998e-05 2020-08-07T15:13:58.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:58.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:52.070790+0000", 2020-08-07T15:13:58.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00066292 2020-08-07T15:13:58.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:58.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:52.071453+0000", 2020-08-07T15:13:58.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6989e-05 2020-08-07T15:13:58.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:58.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:52.071500+0000", 2020-08-07T15:13:58.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.808915045 2020-08-07T15:13:58.482 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.483 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:58.483 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.483 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5481.0:149905 6.9 e298/269 6:93d2e5a7:::benchmark_data_smithi074_63843_object16653:head v 298'9360, mlcod=298'9356)", 2020-08-07T15:13:58.484 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:13:51.267685+0000", 2020-08-07T15:13:58.484 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.77792657, 2020-08-07T15:13:58.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63932675400000005, 2020-08-07T15:13:58.484 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:13:58.484 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:58.484 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:13:58.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:13:58.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:51.267685+0000", 2020-08-07T15:13:58.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:13:58.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:13:58.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:51.267685+0000", 2020-08-07T15:13:58.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2468999999999997e-05 2020-08-07T15:13:58.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:13:58.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:51.267718+0000", 2020-08-07T15:13:58.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1989999999999999e-06 2020-08-07T15:13:58.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:13:58.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:51.267722+0000", 2020-08-07T15:13:58.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63836942699999999 2020-08-07T15:13:58.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:13:58.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:51.906091+0000", 2020-08-07T15:13:58.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8409999999999999e-06 2020-08-07T15:13:58.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:51.906096+0000", 2020-08-07T15:13:58.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4526999999999998e-05 2020-08-07T15:13:58.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:13:58.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:51.906191+0000", 2020-08-07T15:13:58.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3039e-05 2020-08-07T15:13:58.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:13:58.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:51.906224+0000", 2020-08-07T15:13:58.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00072055999999999997 2020-08-07T15:13:58.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:13:58.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:51.906944+0000", 2020-08-07T15:13:58.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7692000000000002e-05 2020-08-07T15:13:58.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:13:58.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:13:58.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:13:58.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:51.907012+0000", 2020-08-07T15:13:58.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63932675400000005 2020-08-07T15:13:58.504 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.504 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:58.505 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.505 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:13:58.505 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:13:58.505 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:13:58.506 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2020-08-07T15:13:58.506 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 0 1 2020-08-07T15:13:58.507 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:13:58.510 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is set 2020-08-07T15:13:58.511 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:13:58.512 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:13:58.505+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:13:58.527 INFO:teuthology.orchestra.run.smithi074:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 8192 --object-size 65536 -p unique_pool_4 bench 90 write' 2020-08-07T15:13:58.559 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2020-08-07T15:13:58.560 INFO:teuthology.orchestra.run:waiting for 3000 2020-08-07T15:13:58.608 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:13:58.607+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:13:58.622 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:hints = 1 2020-08-07T15:13:58.623 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Maintaining 16 concurrent writes of 8192 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2020-08-07T15:13:58.623 INFO:tasks.radosbench.radosbench.0.smithi074.stdout:Object prefix: benchmark_data_smithi074_71701 2020-08-07T15:13:58.623 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2020-08-07T15:13:58.623 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 0 0 0 0 0 0 - 0 2020-08-07T15:13:58.710 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:58.707+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:58.738 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:13:58.739 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:13:58.739 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.739 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5792.0:121 7.b 7:d40f3f75:::benchmark_data_smithi074_71701_object15:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e303)", 2020-08-07T15:13:58.739 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:58.729853+0000", 2020-08-07T15:13:58.740 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0064677399999999996, 2020-08-07T15:13:58.740 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0064965750000000001, 2020-08-07T15:13:58.740 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:58.740 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:58.740 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:58.741 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5792", 2020-08-07T15:13:58.741 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/3840862776", 2020-08-07T15:13:58.741 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 121 2020-08-07T15:13:58.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.741 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:58.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.743 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:58.744 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.729853+0000", 2020-08-07T15:13:58.746 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:58.746 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.746 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:58.753 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.729853+0000", 2020-08-07T15:13:58.754 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.973e-05 2020-08-07T15:13:58.754 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.754 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.754 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:58.754 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.729903+0000", 2020-08-07T15:13:58.755 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.109e-06 2020-08-07T15:13:58.755 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.755 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:58.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.729910+0000", 2020-08-07T15:13:58.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5087e-05 2020-08-07T15:13:58.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:58.756 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.729925+0000", 2020-08-07T15:13:58.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0564000000000001e-05 2020-08-07T15:13:58.757 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.757 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.757 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.729936+0000", 2020-08-07T15:13:58.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0779999999999997e-05 2020-08-07T15:13:58.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.762 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.762 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:58.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.730006+0000", 2020-08-07T15:13:58.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00022483900000000001 2020-08-07T15:13:58.764 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.764 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:58.765 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.730231+0000", 2020-08-07T15:13:58.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000218359 2020-08-07T15:13:58.766 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.766 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:13:58.766 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.730450+0000", 2020-08-07T15:13:58.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010882859999999999 2020-08-07T15:13:58.766 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:58.767 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.731538+0000", 2020-08-07T15:13:58.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3090000000000001e-06 2020-08-07T15:13:58.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.768 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.768 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:58.768 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.731539+0000", 2020-08-07T15:13:58.768 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7478999999999995e-05 2020-08-07T15:13:58.768 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.769 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.769 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:58.769 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.731617+0000", 2020-08-07T15:13:58.769 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0017635420000000001 2020-08-07T15:13:58.769 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:58.770 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:58.770 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:58.770 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.770 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.770 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5792.0:122 7.b 7:d40f3f75:::benchmark_data_smithi074_71701_object15:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e303)", 2020-08-07T15:13:58.771 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:58.731829+0000", 2020-08-07T15:13:58.771 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0044913840000000002, 2020-08-07T15:13:58.771 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0046334499999999999, 2020-08-07T15:13:58.771 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:58.771 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:58.772 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:58.772 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5792", 2020-08-07T15:13:58.772 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/3840862776", 2020-08-07T15:13:58.772 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 122 2020-08-07T15:13:58.772 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.772 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:58.773 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.773 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:58.773 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.731829+0000", 2020-08-07T15:13:58.773 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:58.773 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.774 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.774 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:58.775 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.731829+0000", 2020-08-07T15:13:58.775 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7671999999999999e-05 2020-08-07T15:13:58.775 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.775 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:58.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.731857+0000", 2020-08-07T15:13:58.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5210000000000004e-06 2020-08-07T15:13:58.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:58.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.731862+0000", 2020-08-07T15:13:58.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1559999999999995e-06 2020-08-07T15:13:58.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:58.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.731870+0000", 2020-08-07T15:13:58.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7729999999999999e-06 2020-08-07T15:13:58.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.731875+0000", 2020-08-07T15:13:58.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3096000000000002e-05 2020-08-07T15:13:58.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.779 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:58.779 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.731918+0000", 2020-08-07T15:13:58.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9438000000000006e-05 2020-08-07T15:13:58.780 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.780 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:58.780 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.732017+0000", 2020-08-07T15:13:58.781 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019416799999999999 2020-08-07T15:13:58.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.781 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:13:58.781 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.732211+0000", 2020-08-07T15:13:58.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00079753799999999998 2020-08-07T15:13:58.782 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.782 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.782 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:58.782 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.733009+0000", 2020-08-07T15:13:58.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3900000000000003e-07 2020-08-07T15:13:58.783 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:58.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.733010+0000", 2020-08-07T15:13:58.783 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9153999999999996e-05 2020-08-07T15:13:58.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:58.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.733079+0000", 2020-08-07T15:13:58.785 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001249455 2020-08-07T15:13:58.785 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:58.785 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:58.786 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:58.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.786 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5792.0:123 7.b 7:d40f3f75:::benchmark_data_smithi074_71701_object15:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e303)", 2020-08-07T15:13:58.786 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:58.733476+0000", 2020-08-07T15:13:58.787 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0028452949999999999, 2020-08-07T15:13:58.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0030647040000000001, 2020-08-07T15:13:58.787 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:58.787 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:13:58.787 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:58.788 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5792", 2020-08-07T15:13:58.788 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/3840862776", 2020-08-07T15:13:58.788 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 123 2020-08-07T15:13:58.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.788 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:58.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.789 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:58.789 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.733476+0000", 2020-08-07T15:13:58.789 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:58.789 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.789 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.790 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:58.790 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.733476+0000", 2020-08-07T15:13:58.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6783000000000003e-05 2020-08-07T15:13:58.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:58.791 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.733522+0000", 2020-08-07T15:13:58.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3450000000000002e-06 2020-08-07T15:13:58.791 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.791 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:58.792 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.733529+0000", 2020-08-07T15:13:58.792 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4583000000000001e-05 2020-08-07T15:13:58.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.792 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.793 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:58.793 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.733543+0000", 2020-08-07T15:13:58.793 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1389999999999995e-06 2020-08-07T15:13:58.793 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.793 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.793 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.794 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.733551+0000", 2020-08-07T15:13:58.794 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0708000000000003e-05 2020-08-07T15:13:58.794 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.794 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.794 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:58.795 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.733602+0000", 2020-08-07T15:13:58.795 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000116944 2020-08-07T15:13:58.796 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.796 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:58.796 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.733719+0000", 2020-08-07T15:13:58.797 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00021708200000000001 2020-08-07T15:13:58.798 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.798 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.798 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:13:58.798 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.733936+0000", 2020-08-07T15:13:58.799 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00087213099999999995 2020-08-07T15:13:58.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:13:58.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.734808+0000", 2020-08-07T15:13:58.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.049e-06 2020-08-07T15:13:58.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:13:58.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.734809+0000", 2020-08-07T15:13:58.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7075000000000004e-05 2020-08-07T15:13:58.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:13:58.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.734876+0000", 2020-08-07T15:13:58.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0014008390000000001 2020-08-07T15:13:58.806 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:58.806 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:58.806 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:58.806 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.807 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5792.0:124 7.b 7:d40f3f75:::benchmark_data_smithi074_71701_object15:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e303)", 2020-08-07T15:13:58.807 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:58.734823+0000", 2020-08-07T15:13:58.807 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0014980289999999999, 2020-08-07T15:13:58.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0018092379999999999, 2020-08-07T15:13:58.808 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:58.809 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:13:58.809 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:58.809 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5792", 2020-08-07T15:13:58.809 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/3840862776", 2020-08-07T15:13:58.809 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 124 2020-08-07T15:13:58.810 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.810 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:58.810 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.810 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:58.810 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.734823+0000", 2020-08-07T15:13:58.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:58.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.811 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:58.811 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.734823+0000", 2020-08-07T15:13:58.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3894999999999999e-05 2020-08-07T15:13:58.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:58.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.734867+0000", 2020-08-07T15:13:58.813 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.13e-06 2020-08-07T15:13:58.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.813 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:58.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.734872+0000", 2020-08-07T15:13:58.814 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6990000000000003e-06 2020-08-07T15:13:58.814 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.814 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.814 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:58.815 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.734880+0000", 2020-08-07T15:13:58.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2910000000000001e-06 2020-08-07T15:13:58.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.734884+0000", 2020-08-07T15:13:58.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4618e-05 2020-08-07T15:13:58.816 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.817 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:58.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.734918+0000", 2020-08-07T15:13:58.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010097100000000001 2020-08-07T15:13:58.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:58.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.735019+0000", 2020-08-07T15:13:58.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000192377 2020-08-07T15:13:58.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:13:58.821 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.735212+0000", 2020-08-07T15:13:58.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00038898099999999999 2020-08-07T15:13:58.821 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:58.822 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:58.822 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:58.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.822 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.822 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5792.0:125 7.b 7:d40f3f75:::benchmark_data_smithi074_71701_object15:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e303)", 2020-08-07T15:13:58.823 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:58.734906+0000", 2020-08-07T15:13:58.823 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0014143719999999999, 2020-08-07T15:13:58.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001816472, 2020-08-07T15:13:58.823 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:58.823 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "waiting for sub ops", 2020-08-07T15:13:58.824 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:58.824 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5792", 2020-08-07T15:13:58.824 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/3840862776", 2020-08-07T15:13:58.824 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 125 2020-08-07T15:13:58.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.825 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:58.825 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.825 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:58.825 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.734906+0000", 2020-08-07T15:13:58.826 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:58.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:58.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.734906+0000", 2020-08-07T15:13:58.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5449999999999999e-05 2020-08-07T15:13:58.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.827 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:58.828 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.734932+0000", 2020-08-07T15:13:58.828 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9170000000000002e-06 2020-08-07T15:13:58.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:58.829 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.734935+0000", 2020-08-07T15:13:58.829 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6110000000000001e-06 2020-08-07T15:13:58.829 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.829 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.830 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:58.830 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.734940+0000", 2020-08-07T15:13:58.831 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5969999999999999e-06 2020-08-07T15:13:58.831 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.831 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.831 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.831 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.734943+0000", 2020-08-07T15:13:58.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00068419200000000002 2020-08-07T15:13:58.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.832 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:13:58.833 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.735627+0000", 2020-08-07T15:13:58.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00057318500000000004 2020-08-07T15:13:58.833 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:13:58.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.736200+0000", 2020-08-07T15:13:58.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017571199999999999 2020-08-07T15:13:58.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 5,6", 2020-08-07T15:13:58.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.736376+0000", 2020-08-07T15:13:58.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0014696640000000001 2020-08-07T15:13:58.835 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:58.835 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:58.835 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:58.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.836 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5792.0:126 7.b 7.aefcf02b (undecoded) ondisk+write+known_if_redirected e303)", 2020-08-07T15:13:58.836 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:58.734960+0000", 2020-08-07T15:13:58.836 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0013607459999999999, 2020-08-07T15:13:58.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0018427509999999999, 2020-08-07T15:13:58.837 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:58.837 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:13:58.837 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:58.837 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5792", 2020-08-07T15:13:58.838 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/3840862776", 2020-08-07T15:13:58.838 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 126 2020-08-07T15:13:58.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.838 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:58.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.839 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:58.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.734960+0000", 2020-08-07T15:13:58.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:58.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.840 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:58.840 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.734960+0000", 2020-08-07T15:13:58.841 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5737000000000001e-05 2020-08-07T15:13:58.841 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:58.842 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.734986+0000", 2020-08-07T15:13:58.842 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5899999999999999e-06 2020-08-07T15:13:58.842 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.842 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.842 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:58.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.734989+0000", 2020-08-07T15:13:58.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5430000000000001e-06 2020-08-07T15:13:58.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.843 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:58.844 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.734995+0000", 2020-08-07T15:13:58.844 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3070000000000001e-06 2020-08-07T15:13:58.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.844 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.845 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.734997+0000", 2020-08-07T15:13:58.845 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7177e-05 2020-08-07T15:13:58.845 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:58.845 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:58.845 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:58.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.846 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.846 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5792.0:127 7.b 7.aefcf02b (undecoded) ondisk+write+known_if_redirected e303)", 2020-08-07T15:13:58.847 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:58.735017+0000", 2020-08-07T15:13:58.847 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0013042360000000001, 2020-08-07T15:13:58.847 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0018571810000000001, 2020-08-07T15:13:58.848 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:58.848 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:13:58.848 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:58.848 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5792", 2020-08-07T15:13:58.848 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/3840862776", 2020-08-07T15:13:58.849 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 127 2020-08-07T15:13:58.849 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.849 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:58.849 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.849 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:58.850 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.735017+0000", 2020-08-07T15:13:58.850 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:58.850 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.850 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:58.851 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.735017+0000", 2020-08-07T15:13:58.852 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5225999999999997e-05 2020-08-07T15:13:58.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:58.852 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.735062+0000", 2020-08-07T15:13:58.853 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1259999999999997e-06 2020-08-07T15:13:58.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.853 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:58.854 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.735067+0000", 2020-08-07T15:13:58.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0473e-05 2020-08-07T15:13:58.854 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.854 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.854 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:58.855 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.735077+0000", 2020-08-07T15:13:58.855 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.4959999999999998e-06 2020-08-07T15:13:58.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.855 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.856 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.735082+0000", 2020-08-07T15:13:58.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5320999999999993e-05 2020-08-07T15:13:58.856 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:58.856 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:58.856 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:58.857 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.857 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.857 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5792.0:128 7.b 7.aefcf02b (undecoded) ondisk+write+known_if_redirected e303)", 2020-08-07T15:13:58.857 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:58.735114+0000", 2020-08-07T15:13:58.858 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.0012063460000000001, 2020-08-07T15:13:58.858 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001820648, 2020-08-07T15:13:58.858 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:58.858 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:13:58.858 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:13:58.859 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5792", 2020-08-07T15:13:58.859 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/3840862776", 2020-08-07T15:13:58.859 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 128 2020-08-07T15:13:58.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.859 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:58.860 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.860 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:58.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.735114+0000", 2020-08-07T15:13:58.860 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:58.860 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.861 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:58.861 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.735114+0000", 2020-08-07T15:13:58.861 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7052999999999998e-05 2020-08-07T15:13:58.862 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.862 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:58.864 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.735162+0000", 2020-08-07T15:13:58.864 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3850000000000001e-06 2020-08-07T15:13:58.865 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.865 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.865 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:58.865 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.735167+0000", 2020-08-07T15:13:58.865 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1056000000000001e-05 2020-08-07T15:13:58.866 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.866 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.866 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:58.866 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.735178+0000", 2020-08-07T15:13:58.866 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7319999999999998e-06 2020-08-07T15:13:58.867 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.867 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.867 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.867 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.735183+0000", 2020-08-07T15:13:58.867 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8225999999999994e-05 2020-08-07T15:13:58.868 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:58.868 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:58.868 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:58.868 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.868 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.869 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5792.0:122 7.b e303/301)", 2020-08-07T15:13:58.869 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:58.736061+0000", 2020-08-07T15:13:58.869 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.00025989900000000001, 2020-08-07T15:13:58.869 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00094090000000000005, 2020-08-07T15:13:58.869 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:58.870 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:13:58.870 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:58.870 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.870 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:58.870 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.736061+0000", 2020-08-07T15:13:58.871 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:58.871 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.871 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.871 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:58.871 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.736061+0000", 2020-08-07T15:13:58.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2700999999999997e-05 2020-08-07T15:13:58.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.873 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:58.873 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.736094+0000", 2020-08-07T15:13:58.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6130000000000003e-06 2020-08-07T15:13:58.874 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.874 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.874 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:58.874 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.736099+0000", 2020-08-07T15:13:58.874 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00088507199999999999 2020-08-07T15:13:58.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:58.875 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.736984+0000", 2020-08-07T15:13:58.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0650000000000004e-06 2020-08-07T15:13:58.876 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.876 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.736990+0000", 2020-08-07T15:13:58.877 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00092945099999999998 2020-08-07T15:13:58.877 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:58.877 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:58.877 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:58.877 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.878 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.878 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5792.0:123 7.b e303/301)", 2020-08-07T15:13:58.878 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:58.736130+0000", 2020-08-07T15:13:58.878 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.00019060999999999999, 2020-08-07T15:13:58.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00092602299999999999, 2020-08-07T15:13:58.879 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:13:58.879 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "queued for pg", 2020-08-07T15:13:58.879 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:13:58.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.880 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:13:58.880 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.736130+0000", 2020-08-07T15:13:58.880 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:13:58.880 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.881 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:13:58.881 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.736130+0000", 2020-08-07T15:13:58.881 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00082479600000000004 2020-08-07T15:13:58.881 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.881 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.882 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:13:58.882 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.736955+0000", 2020-08-07T15:13:58.882 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6650000000000002e-06 2020-08-07T15:13:58.882 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.882 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.883 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:13:58.884 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.736960+0000", 2020-08-07T15:13:58.884 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9014999999999997e-05 2020-08-07T15:13:58.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.884 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.884 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:13:58.885 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.737009+0000", 2020-08-07T15:13:58.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7890000000000001e-06 2020-08-07T15:13:58.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:13:58.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:13:58.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:13:58.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:58.737012+0000", 2020-08-07T15:13:58.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00088226500000000002 2020-08-07T15:13:58.886 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:58.886 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:13:58.888 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:58.888 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:13:58.888 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:13:58.888 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 10 2020-08-07T15:13:58.889 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:13:58.891 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:13:58.891 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:13:58.892 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:13:58.892 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:13:58.892 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:58.888+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:58.931 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:13:58.975 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:13:58.999 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:13:59.030 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:13:59.031 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:58.992+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:13:59.071 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:13:59.076 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:59.077 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:59.077 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:13:59.081 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:13:59.081 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:13:59.081 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:13:59.081 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:13:59.082 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:13:59.082 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:13:59.082 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:13:59.110 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:13:59.122 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:13:59.108+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:13:59.213 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:59.210+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:59.319 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:59.316+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:59.346 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:59.347 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:59.360 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:13:59.454 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:13:59.450+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:13:59.523 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:59.520+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:59.539 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.0 to 1 (10000) 2020-08-07T15:13:59.564 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 1 1 2020-08-07T15:13:59.586 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:59.587 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:59.608 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:13:59.622 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 1 16 1394 1378 10.765 10.7656 0.00892651 0.0113707 2020-08-07T15:13:59.624 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:13:59.621+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:13:59.727 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:13:59.722+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:13:59.828 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:13:59.824+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:13:59.844 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:13:59.845 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:13:59.859 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:13:59.963 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:13:59.954+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:14:00.048 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:14:00.043+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:14:00.127 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:14:00.128 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:14:00.136 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:14:00.146 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:14:00.145+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:14:00.248 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:14:00.247+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:14:00.352 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:14:00.350+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:14:00.360 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:14:00.362 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:14:00.375 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:14:00.455 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:14:00.453+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:14:00.514 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset noscrub 2020-08-07T15:14:00.548 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.1 to 1 (10000) 2020-08-07T15:14:00.559 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:14:00.557+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:14:00.567 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 2 1 2020-08-07T15:14:00.578 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:14:00.579 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:14:00.587 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:14:00.622 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 2 16 2750 2734 10.679 10.5938 0.00928179 0.0115881 2020-08-07T15:14:00.666 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:14:00.661+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:14:00.770 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:14:00.765+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:14:00.834 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:14:00.837 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:14:00.849 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:14:00.868 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:14:00.866+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:14:00.969 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:14:00.967+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:14:01.074 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:14:01.070+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:14:01.078 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:14:01.079 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:14:01.097 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:14:01.111 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:14:01.206 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:14:01.191+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:14:01.318 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:14:01.280+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:14:01.333 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:14:01.333 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:14:01.353 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:14:01.398 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:14:01.400 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:14:01.398+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:14:01.438 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:14:01.481 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:14:01.480+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:14:01.564 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is unset 2020-08-07T15:14:01.565 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.2 to 1 (10000) 2020-08-07T15:14:01.582 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 3 1 2020-08-07T15:14:01.591 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:14:01.580+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:14:01.592 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:14:01.592 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:14:01.595 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:14:01.621 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 3 15 4048 4033 10.5018 10.1484 0.00634659 0.0118477 2020-08-07T15:14:01.682 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:14:01.681+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:14:01.689 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:14:01.740 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:14:01.741 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:14:01.742 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:14:01.754 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:14:01.787 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:14:01.786+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:14:01.789 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:14:01.844 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:14:01.887 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:14:01.887+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:14:01.891 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:14:01.937 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:14:01.941 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:14:01.942 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:14:01.942 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:14:01.942 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:14:01.943 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:14:01.943 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:14:01.943 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:14:01.949 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:14:01.952 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:14:01.952 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:14:01.953 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:14:01.953 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:14:01.953 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:14:01.971 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:14:01.999 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:14:01.998+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:14:02.091 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:14:02.089+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:14:02.151 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:14:02.152 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:14:02.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:14:02.152 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:14:02.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.152 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "MOSDPGPushReply(3.1 62/60 [PushReplyOp(3:8329f37e:::benchmark_data_smithi074_35447_object11915:head)])", 2020-08-07T15:14:02.153 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:14:02.153 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 371.28922078699998, 2020-08-07T15:14:02.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996, 2020-08-07T15:14:02.153 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:02.154 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:14:02.154 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:02.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:02.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:14:02.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:02.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:02.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858400+0000", 2020-08-07T15:14:02.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5372000000000001e-05 2020-08-07T15:14:02.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:02.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858416+0000", 2020-08-07T15:14:02.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6899999999999998e-06 2020-08-07T15:14:02.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:02.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:50.858419+0000", 2020-08-07T15:14:02.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63907904299999996 2020-08-07T15:14:02.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:02.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497498+0000", 2020-08-07T15:14:02.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3332999999999999e-05 2020-08-07T15:14:02.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:02.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497512+0000", 2020-08-07T15:14:02.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000450875 2020-08-07T15:14:02.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.497962+0000", 2020-08-07T15:14:02.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000267779 2020-08-07T15:14:02.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:02.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:07:51.498230+0000", 2020-08-07T15:14:02.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.63983009199999996 2020-08-07T15:14:02.173 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.173 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:02.173 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.174 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4329.0:145792 3.1 3:881e0420:::benchmark_data_smithi074_35447_object16183:head [delete] snapc 0=[] ondisk+write+known_if_redirected e75)", 2020-08-07T15:14:02.174 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:14:02.174 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 347.48081490599998, 2020-08-07T15:14:02.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997, 2020-08-07T15:14:02.175 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:02.175 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:02.175 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:02.175 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4329", 2020-08-07T15:14:02.175 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:14:02.176 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 145792 2020-08-07T15:14:02.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.176 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:02.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:02.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:14:02.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:02.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:02.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666806+0000", 2020-08-07T15:14:02.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7489000000000001e-05 2020-08-07T15:14:02.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:02.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666824+0000", 2020-08-07T15:14:02.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0169999999999998e-06 2020-08-07T15:14:02.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:02.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666828+0000", 2020-08-07T15:14:02.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9829999999999996e-06 2020-08-07T15:14:02.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:02.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666836+0000", 2020-08-07T15:14:02.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.481e-06 2020-08-07T15:14:02.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:02.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666840+0000", 2020-08-07T15:14:02.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5299000000000001e-05 2020-08-07T15:14:02.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.666885+0000", 2020-08-07T15:14:02.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014456500000000001 2020-08-07T15:14:02.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:02.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667030+0000", 2020-08-07T15:14:02.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017677799999999999 2020-08-07T15:14:02.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,5", 2020-08-07T15:14:02.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.667207+0000", 2020-08-07T15:14:02.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004149649 2020-08-07T15:14:02.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:02.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671356+0000", 2020-08-07T15:14:02.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019339649999999999 2020-08-07T15:14:02.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:02.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.673290+0000", 2020-08-07T15:14:02.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73346620500000004 2020-08-07T15:14:02.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:02.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406756+0000", 2020-08-07T15:14:02.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4700000000000001e-06 2020-08-07T15:14:02.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:02.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406759+0000", 2020-08-07T15:14:02.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6258999999999998e-05 2020-08-07T15:14:02.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:02.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406815+0000", 2020-08-07T15:14:02.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0126000000000002e-05 2020-08-07T15:14:02.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:02.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406855+0000", 2020-08-07T15:14:02.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74004928599999997 2020-08-07T15:14:02.192 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.192 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:02.192 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.193 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4329.0:145792 3.1 e75/67 ondisk, result = 0)", 2020-08-07T15:14:02.193 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:14:02.194 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 347.47643001799997, 2020-08-07T15:14:02.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003, 2020-08-07T15:14:02.194 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:02.194 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:14:02.194 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:02.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:02.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:14:02.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:02.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:02.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671191+0000", 2020-08-07T15:14:02.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2830999999999998e-05 2020-08-07T15:14:02.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:02.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671224+0000", 2020-08-07T15:14:02.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9339999999999997e-06 2020-08-07T15:14:02.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:02.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:14.671229+0000", 2020-08-07T15:14:02.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.735414864 2020-08-07T15:14:02.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:02.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406644+0000", 2020-08-07T15:14:02.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1250000000000004e-06 2020-08-07T15:14:02.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:02.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406651+0000", 2020-08-07T15:14:02.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6567e-05 2020-08-07T15:14:02.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406737+0000", 2020-08-07T15:14:02.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0054000000000001e-05 2020-08-07T15:14:02.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:02.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406747+0000", 2020-08-07T15:14:02.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011391999999999999 2020-08-07T15:14:02.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:02.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:15.406861+0000", 2020-08-07T15:14:02.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73567029500000003 2020-08-07T15:14:02.204 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.204 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:02.204 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.205 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30921 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:14:02.205 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:14:02.205 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 315.44168164400003, 2020-08-07T15:14:02.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001, 2020-08-07T15:14:02.206 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:02.206 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:14:02.206 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:02.206 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:02.206 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:02.207 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30921 2020-08-07T15:14:02.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.207 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:02.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:02.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:14:02.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:02.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:02.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.705939+0000", 2020-08-07T15:14:02.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6954999999999998e-05 2020-08-07T15:14:02.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:02.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706016+0000", 2020-08-07T15:14:02.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0229999999999992e-06 2020-08-07T15:14:02.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:02.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706024+0000", 2020-08-07T15:14:02.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.432e-05 2020-08-07T15:14:02.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:02.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706039+0000", 2020-08-07T15:14:02.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8490000000000006e-06 2020-08-07T15:14:02.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:02.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706046+0000", 2020-08-07T15:14:02.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6837999999999999e-05 2020-08-07T15:14:02.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706113+0000", 2020-08-07T15:14:02.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3553000000000001e-05 2020-08-07T15:14:02.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:02.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706127+0000", 2020-08-07T15:14:02.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77277201699999998 2020-08-07T15:14:02.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.478899+0000", 2020-08-07T15:14:02.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093978599999999996 2020-08-07T15:14:02.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:14:02.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479839+0000", 2020-08-07T15:14:02.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010376029 2020-08-07T15:14:02.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490215+0000", 2020-08-07T15:14:02.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000190657 2020-08-07T15:14:02.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:02.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490405+0000", 2020-08-07T15:14:02.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78446602700000001 2020-08-07T15:14:02.222 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.222 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:02.222 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.223 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30922 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:14:02.223 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:14:02.223 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 315.441521481, 2020-08-07T15:14:02.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247, 2020-08-07T15:14:02.223 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:02.224 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:14:02.224 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:02.224 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:02.224 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:02.224 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30922 2020-08-07T15:14:02.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.225 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:02.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:02.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:14:02.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:02.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:02.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706099+0000", 2020-08-07T15:14:02.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9178999999999997e-05 2020-08-07T15:14:02.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:02.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706159+0000", 2020-08-07T15:14:02.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4999999999999999e-06 2020-08-07T15:14:02.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:02.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706164+0000", 2020-08-07T15:14:02.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1792e-05 2020-08-07T15:14:02.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:02.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706176+0000", 2020-08-07T15:14:02.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.139e-06 2020-08-07T15:14:02.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:02.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706182+0000", 2020-08-07T15:14:02.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039724900000000002 2020-08-07T15:14:02.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706579+0000", 2020-08-07T15:14:02.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1191e-05 2020-08-07T15:14:02.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:02.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706591+0000", 2020-08-07T15:14:02.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329101300000003 2020-08-07T15:14:02.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479882+0000", 2020-08-07T15:14:02.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9565000000000001e-05 2020-08-07T15:14:02.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:14:02.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.479971+0000", 2020-08-07T15:14:02.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010471263 2020-08-07T15:14:02.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490442+0000", 2020-08-07T15:14:02.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015035600000000001 2020-08-07T15:14:02.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:02.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490593+0000", 2020-08-07T15:14:02.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.784493247 2020-08-07T15:14:02.238 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.238 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:02.238 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.239 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30923 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:14:02.239 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:14:02.239 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 315.44104642000002, 2020-08-07T15:14:02.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998, 2020-08-07T15:14:02.240 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:02.240 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:14:02.240 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:02.240 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:02.240 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:02.241 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30923 2020-08-07T15:14:02.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.241 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:02.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:02.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:14:02.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:02.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:02.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706575+0000", 2020-08-07T15:14:02.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5531000000000001e-05 2020-08-07T15:14:02.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:02.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706630+0000", 2020-08-07T15:14:02.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7689999999999999e-06 2020-08-07T15:14:02.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:02.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706637+0000", 2020-08-07T15:14:02.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3577999999999999e-05 2020-08-07T15:14:02.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:02.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706650+0000", 2020-08-07T15:14:02.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1439999999999997e-06 2020-08-07T15:14:02.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:02.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706658+0000", 2020-08-07T15:14:02.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1663999999999999e-05 2020-08-07T15:14:02.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706709+0000", 2020-08-07T15:14:02.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0149000000000001e-05 2020-08-07T15:14:02.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:02.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706719+0000", 2020-08-07T15:14:02.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77329694900000001 2020-08-07T15:14:02.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480016+0000", 2020-08-07T15:14:02.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7998999999999997e-05 2020-08-07T15:14:02.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:14:02.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480084+0000", 2020-08-07T15:14:02.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010547333000000001 2020-08-07T15:14:02.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490632+0000", 2020-08-07T15:14:02.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7439e-05 2020-08-07T15:14:02.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:02.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490729+0000", 2020-08-07T15:14:02.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415455499999998 2020-08-07T15:14:02.257 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.257 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:02.257 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.258 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30924 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:14:02.258 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:14:02.258 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 315.44091914799998, 2020-08-07T15:14:02.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995, 2020-08-07T15:14:02.259 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:02.259 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:14:02.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:02.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:02.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:02.260 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30924 2020-08-07T15:14:02.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.260 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:02.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:02.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:14:02.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:02.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:02.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706702+0000", 2020-08-07T15:14:02.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9698999999999999e-05 2020-08-07T15:14:02.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:02.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706762+0000", 2020-08-07T15:14:02.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8080000000000001e-06 2020-08-07T15:14:02.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:02.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706767+0000", 2020-08-07T15:14:02.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1069e-05 2020-08-07T15:14:02.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:02.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706778+0000", 2020-08-07T15:14:02.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6969999999999998e-06 2020-08-07T15:14:02.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:02.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706784+0000", 2020-08-07T15:14:02.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8792999999999998e-05 2020-08-07T15:14:02.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706843+0000", 2020-08-07T15:14:02.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5449999999999998e-06 2020-08-07T15:14:02.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:02.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706850+0000", 2020-08-07T15:14:02.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.773278354 2020-08-07T15:14:02.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480129+0000", 2020-08-07T15:14:02.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1577999999999997e-05 2020-08-07T15:14:02.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:14:02.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480210+0000", 2020-08-07T15:14:02.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010541915000000001 2020-08-07T15:14:02.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490752+0000", 2020-08-07T15:14:02.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103937 2020-08-07T15:14:02.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:02.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490856+0000", 2020-08-07T15:14:02.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78415439499999995 2020-08-07T15:14:02.278 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.278 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:02.278 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.279 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30925 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:14:02.279 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:14:02.279 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 315.44079537699997, 2020-08-07T15:14:02.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995, 2020-08-07T15:14:02.280 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:02.280 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:14:02.280 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:02.280 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:02.281 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:02.281 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30925 2020-08-07T15:14:02.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.281 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:02.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:02.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:14:02.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:02.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:02.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706826+0000", 2020-08-07T15:14:02.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4664999999999999e-05 2020-08-07T15:14:02.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:02.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706880+0000", 2020-08-07T15:14:02.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7479999999999999e-06 2020-08-07T15:14:02.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:02.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706886+0000", 2020-08-07T15:14:02.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1838000000000001e-05 2020-08-07T15:14:02.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:02.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706898+0000", 2020-08-07T15:14:02.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0089999999999998e-06 2020-08-07T15:14:02.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:02.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706903+0000", 2020-08-07T15:14:02.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3559e-05 2020-08-07T15:14:02.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706956+0000", 2020-08-07T15:14:02.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0309000000000001e-05 2020-08-07T15:14:02.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:02.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706967+0000", 2020-08-07T15:14:02.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77328290499999996 2020-08-07T15:14:02.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480250+0000", 2020-08-07T15:14:02.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8680999999999997e-05 2020-08-07T15:14:02.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:14:02.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480298+0000", 2020-08-07T15:14:02.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01059272 2020-08-07T15:14:02.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490891+0000", 2020-08-07T15:14:02.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014580300000000001 2020-08-07T15:14:02.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:02.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491037+0000", 2020-08-07T15:14:02.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78421123699999995 2020-08-07T15:14:02.297 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.298 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:02.298 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.298 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30926 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:14:02.299 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:14:02.299 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 315.44067850800002, 2020-08-07T15:14:02.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005, 2020-08-07T15:14:02.299 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:02.299 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:14:02.300 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:02.300 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:02.300 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:02.300 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30926 2020-08-07T15:14:02.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.301 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:02.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:02.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:14:02.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:02.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:02.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706942+0000", 2020-08-07T15:14:02.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5223000000000003e-05 2020-08-07T15:14:02.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:02.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.706998+0000", 2020-08-07T15:14:02.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4809999999999997e-06 2020-08-07T15:14:02.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:02.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707003+0000", 2020-08-07T15:14:02.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1663e-05 2020-08-07T15:14:02.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:02.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707015+0000", 2020-08-07T15:14:02.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9729999999999997e-06 2020-08-07T15:14:02.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:02.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707020+0000", 2020-08-07T15:14:02.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2565999999999999e-05 2020-08-07T15:14:02.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707062+0000", 2020-08-07T15:14:02.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8944e-05 2020-08-07T15:14:02.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:02.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707081+0000", 2020-08-07T15:14:02.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77325885000000005 2020-08-07T15:14:02.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480340+0000", 2020-08-07T15:14:02.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0321000000000006e-05 2020-08-07T15:14:02.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:14:02.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480420+0000", 2020-08-07T15:14:02.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010655958 2020-08-07T15:14:02.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491076+0000", 2020-08-07T15:14:02.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9784999999999995e-05 2020-08-07T15:14:02.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:02.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491166+0000", 2020-08-07T15:14:02.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78422376400000005 2020-08-07T15:14:02.312 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.312 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:02.313 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.313 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30927 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:14:02.313 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:14:02.314 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 315.44056068899999, 2020-08-07T15:14:02.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999, 2020-08-07T15:14:02.314 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:02.314 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:14:02.314 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:02.315 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:02.315 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:02.315 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30927 2020-08-07T15:14:02.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.315 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:02.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:02.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:14:02.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:02.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:02.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707060+0000", 2020-08-07T15:14:02.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9861999999999997e-05 2020-08-07T15:14:02.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:02.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707120+0000", 2020-08-07T15:14:02.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8969999999999996e-06 2020-08-07T15:14:02.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:02.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707126+0000", 2020-08-07T15:14:02.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1248e-05 2020-08-07T15:14:02.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:02.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707137+0000", 2020-08-07T15:14:02.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.955e-06 2020-08-07T15:14:02.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:02.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707142+0000", 2020-08-07T15:14:02.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0643999999999998e-05 2020-08-07T15:14:02.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707193+0000", 2020-08-07T15:14:02.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1983e-05 2020-08-07T15:14:02.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:02.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707205+0000", 2020-08-07T15:14:02.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77324914 2020-08-07T15:14:02.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480454+0000", 2020-08-07T15:14:02.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4119999999999998e-05 2020-08-07T15:14:02.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:14:02.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480498+0000", 2020-08-07T15:14:02.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010709707000000001 2020-08-07T15:14:02.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491208+0000", 2020-08-07T15:14:02.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014433800000000001 2020-08-07T15:14:02.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:02.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491352+0000", 2020-08-07T15:14:02.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78429189399999999 2020-08-07T15:14:02.327 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.327 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:02.327 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.328 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:30928 4.c 4:32f80c20:::benchmark_data_smithi074_44586_object3865:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e99)", 2020-08-07T15:14:02.328 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:14:02.328 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 315.44043633199999, 2020-08-07T15:14:02.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998, 2020-08-07T15:14:02.328 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:02.329 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:14:02.329 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:02.329 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:02.329 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:02.333 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 30928 2020-08-07T15:14:02.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.334 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:02.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:02.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:14:02.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:02.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:02.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707185+0000", 2020-08-07T15:14:02.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6582000000000001e-05 2020-08-07T15:14:02.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:02.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707241+0000", 2020-08-07T15:14:02.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9190000000000004e-06 2020-08-07T15:14:02.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:02.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707247+0000", 2020-08-07T15:14:02.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000188822 2020-08-07T15:14:02.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:02.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707436+0000", 2020-08-07T15:14:02.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2940000000000002e-06 2020-08-07T15:14:02.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:02.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707442+0000", 2020-08-07T15:14:02.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7800000000000003e-05 2020-08-07T15:14:02.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707490+0000", 2020-08-07T15:14:02.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2020-08-07T15:14:02.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:02.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.707501+0000", 2020-08-07T15:14:02.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.77303498800000003 2020-08-07T15:14:02.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480536+0000", 2020-08-07T15:14:02.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7723000000000001e-05 2020-08-07T15:14:02.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for degraded object", 2020-08-07T15:14:02.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480614+0000", 2020-08-07T15:14:02.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010812183 2020-08-07T15:14:02.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491426+0000", 2020-08-07T15:14:02.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5502999999999999e-05 2020-08-07T15:14:02.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:02.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491502+0000", 2020-08-07T15:14:02.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.78431711999999998 2020-08-07T15:14:02.351 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.351 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:02.351 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.352 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31009 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:14:02.353 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:14:02.353 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 315.41648002599999, 2020-08-07T15:14:02.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002, 2020-08-07T15:14:02.354 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:02.354 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:02.355 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:02.355 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:02.355 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:02.355 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31009 2020-08-07T15:14:02.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.356 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:02.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:02.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:14:02.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:02.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:02.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731141+0000", 2020-08-07T15:14:02.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2429000000000002e-05 2020-08-07T15:14:02.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:02.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731193+0000", 2020-08-07T15:14:02.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9760000000000003e-06 2020-08-07T15:14:02.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:02.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731198+0000", 2020-08-07T15:14:02.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3597e-05 2020-08-07T15:14:02.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:02.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731212+0000", 2020-08-07T15:14:02.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1709999999999996e-06 2020-08-07T15:14:02.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:02.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731219+0000", 2020-08-07T15:14:02.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5402999999999997e-05 2020-08-07T15:14:02.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731285+0000", 2020-08-07T15:14:02.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3495e-05 2020-08-07T15:14:02.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:02.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731298+0000", 2020-08-07T15:14:02.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.74935423599999995 2020-08-07T15:14:02.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480652+0000", 2020-08-07T15:14:02.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000137806 2020-08-07T15:14:02.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:02.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.480790+0000", 2020-08-07T15:14:02.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022272399999999999 2020-08-07T15:14:02.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:14:02.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481013+0000", 2020-08-07T15:14:02.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003223381 2020-08-07T15:14:02.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:02.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484236+0000", 2020-08-07T15:14:02.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1370000000000001e-06 2020-08-07T15:14:02.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:02.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484237+0000", 2020-08-07T15:14:02.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6288e-05 2020-08-07T15:14:02.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:02.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484294+0000", 2020-08-07T15:14:02.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.007260402 2020-08-07T15:14:02.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:02.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491554+0000", 2020-08-07T15:14:02.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7358000000000003e-05 2020-08-07T15:14:02.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:02.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491601+0000", 2020-08-07T15:14:02.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1100000000000004e-07 2020-08-07T15:14:02.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:02.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491602+0000", 2020-08-07T15:14:02.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013436500000000001 2020-08-07T15:14:02.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:02.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491737+0000", 2020-08-07T15:14:02.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76059567900000002 2020-08-07T15:14:02.387 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.387 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:02.387 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.388 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31010 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:14:02.388 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:14:02.388 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 315.41635407199999, 2020-08-07T15:14:02.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001, 2020-08-07T15:14:02.388 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:02.389 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:02.389 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:02.389 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:02.389 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:02.390 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31010 2020-08-07T15:14:02.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.390 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:02.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:02.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:14:02.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:02.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:02.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731267+0000", 2020-08-07T15:14:02.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2398000000000002e-05 2020-08-07T15:14:02.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:02.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731319+0000", 2020-08-07T15:14:02.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7100000000000004e-06 2020-08-07T15:14:02.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:02.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731325+0000", 2020-08-07T15:14:02.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4328000000000001e-05 2020-08-07T15:14:02.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:02.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731339+0000", 2020-08-07T15:14:02.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3310000000000003e-06 2020-08-07T15:14:02.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:02.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731345+0000", 2020-08-07T15:14:02.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8952999999999998e-05 2020-08-07T15:14:02.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731384+0000", 2020-08-07T15:14:02.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0036e-05 2020-08-07T15:14:02.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:02.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731394+0000", 2020-08-07T15:14:02.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75027681300000004 2020-08-07T15:14:02.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481670+0000", 2020-08-07T15:14:02.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016555899999999999 2020-08-07T15:14:02.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:02.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.481836+0000", 2020-08-07T15:14:02.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028760999999999999 2020-08-07T15:14:02.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:14:02.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482124+0000", 2020-08-07T15:14:02.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038342680000000001 2020-08-07T15:14:02.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:02.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485958+0000", 2020-08-07T15:14:02.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2559999999999999e-06 2020-08-07T15:14:02.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:02.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485959+0000", 2020-08-07T15:14:02.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3758999999999995e-05 2020-08-07T15:14:02.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:02.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486043+0000", 2020-08-07T15:14:02.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057590419999999998 2020-08-07T15:14:02.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:02.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491802+0000", 2020-08-07T15:14:02.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5757000000000001e-05 2020-08-07T15:14:02.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:02.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:14:02.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9299999999999997e-07 2020-08-07T15:14:02.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:02.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491868+0000", 2020-08-07T15:14:02.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012496999999999999 2020-08-07T15:14:02.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:02.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491993+0000", 2020-08-07T15:14:02.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76072648300000001 2020-08-07T15:14:02.413 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.414 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:02.414 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.414 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31011 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:14:02.414 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:14:02.415 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 315.41623786700001, 2020-08-07T15:14:02.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997, 2020-08-07T15:14:02.415 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:02.415 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:02.415 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:02.416 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:02.416 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:02.416 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31011 2020-08-07T15:14:02.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.416 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:02.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:02.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:14:02.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:02.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:02.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731383+0000", 2020-08-07T15:14:02.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9539999999999997e-05 2020-08-07T15:14:02.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:02.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731433+0000", 2020-08-07T15:14:02.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9209999999999997e-06 2020-08-07T15:14:02.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:02.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731439+0000", 2020-08-07T15:14:02.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4246999999999999e-05 2020-08-07T15:14:02.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:02.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731453+0000", 2020-08-07T15:14:02.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8130000000000003e-06 2020-08-07T15:14:02.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:02.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731458+0000", 2020-08-07T15:14:02.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6668000000000001e-05 2020-08-07T15:14:02.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731504+0000", 2020-08-07T15:14:02.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1112e-05 2020-08-07T15:14:02.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:02.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.731515+0000", 2020-08-07T15:14:02.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75121332100000004 2020-08-07T15:14:02.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482729+0000", 2020-08-07T15:14:02.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1810000000000004e-05 2020-08-07T15:14:02.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:02.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482811+0000", 2020-08-07T15:14:02.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018001399999999999 2020-08-07T15:14:02.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:14:02.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.482991+0000", 2020-08-07T15:14:02.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030701140000000001 2020-08-07T15:14:02.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:02.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486061+0000", 2020-08-07T15:14:02.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4499999999999996e-07 2020-08-07T15:14:02.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:02.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486062+0000", 2020-08-07T15:14:02.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5366999999999998e-05 2020-08-07T15:14:02.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:02.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486147+0000", 2020-08-07T15:14:02.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056872779999999996 2020-08-07T15:14:02.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:02.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.491834+0000", 2020-08-07T15:14:02.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024645100000000002 2020-08-07T15:14:02.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:02.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:14:02.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1200000000000003e-07 2020-08-07T15:14:02.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:02.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492081+0000", 2020-08-07T15:14:02.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1451000000000002e-05 2020-08-07T15:14:02.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:02.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492173+0000", 2020-08-07T15:14:02.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76078956399999997 2020-08-07T15:14:02.441 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.441 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:02.442 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.442 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31012 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:14:02.443 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:14:02.443 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 315.41558374800002, 2020-08-07T15:14:02.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003, 2020-08-07T15:14:02.443 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:02.443 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:02.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:02.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:02.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:02.444 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31012 2020-08-07T15:14:02.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.444 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:02.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:02.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:14:02.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:02.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:02.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732037+0000", 2020-08-07T15:14:02.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6344999999999997e-05 2020-08-07T15:14:02.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:02.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732094+0000", 2020-08-07T15:14:02.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2920000000000001e-06 2020-08-07T15:14:02.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:02.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732100+0000", 2020-08-07T15:14:02.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.189e-05 2020-08-07T15:14:02.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:02.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732112+0000", 2020-08-07T15:14:02.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3720000000000001e-06 2020-08-07T15:14:02.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:02.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732118+0000", 2020-08-07T15:14:02.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2514000000000004e-05 2020-08-07T15:14:02.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732191+0000", 2020-08-07T15:14:02.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4610000000000001e-05 2020-08-07T15:14:02.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:02.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732205+0000", 2020-08-07T15:14:02.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75132974100000005 2020-08-07T15:14:02.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483535+0000", 2020-08-07T15:14:02.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9185999999999996e-05 2020-08-07T15:14:02.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:02.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483624+0000", 2020-08-07T15:14:02.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015683 2020-08-07T15:14:02.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:14:02.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.483781+0000", 2020-08-07T15:14:02.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046202889999999996 2020-08-07T15:14:02.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:02.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488401+0000", 2020-08-07T15:14:02.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6700000000000002e-07 2020-08-07T15:14:02.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:02.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488402+0000", 2020-08-07T15:14:02.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1487000000000001e-05 2020-08-07T15:14:02.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:02.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488484+0000", 2020-08-07T15:14:02.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035520830000000002 2020-08-07T15:14:02.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:02.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492036+0000", 2020-08-07T15:14:02.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029266600000000002 2020-08-07T15:14:02.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:02.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492328+0000", 2020-08-07T15:14:02.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5099999999999998e-07 2020-08-07T15:14:02.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:02.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492329+0000", 2020-08-07T15:14:02.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000201299 2020-08-07T15:14:02.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:02.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492531+0000", 2020-08-07T15:14:02.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76049342200000003 2020-08-07T15:14:02.467 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.467 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:02.467 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.468 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31013 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:14:02.468 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:14:02.468 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 315.415372117, 2020-08-07T15:14:02.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996, 2020-08-07T15:14:02.468 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:02.469 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:02.469 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:02.469 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:02.469 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:02.469 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31013 2020-08-07T15:14:02.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.470 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:02.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:02.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:14:02.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:02.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:02.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732249+0000", 2020-08-07T15:14:02.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8949000000000002e-05 2020-08-07T15:14:02.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:02.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732298+0000", 2020-08-07T15:14:02.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1399999999999997e-06 2020-08-07T15:14:02.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:02.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732304+0000", 2020-08-07T15:14:02.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.154e-05 2020-08-07T15:14:02.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:02.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732315+0000", 2020-08-07T15:14:02.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1319999999999999e-06 2020-08-07T15:14:02.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:02.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732322+0000", 2020-08-07T15:14:02.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7431e-05 2020-08-07T15:14:02.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732369+0000", 2020-08-07T15:14:02.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0621999999999999e-05 2020-08-07T15:14:02.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:02.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.732380+0000", 2020-08-07T15:14:02.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75196845999999995 2020-08-07T15:14:02.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484348+0000", 2020-08-07T15:14:02.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000123008 2020-08-07T15:14:02.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:02.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484471+0000", 2020-08-07T15:14:02.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017572700000000001 2020-08-07T15:14:02.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:14:02.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.484647+0000", 2020-08-07T15:14:02.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038557650000000001 2020-08-07T15:14:02.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:02.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:14:02.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2020-08-07T15:14:02.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:02.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488503+0000", 2020-08-07T15:14:02.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0714e-05 2020-08-07T15:14:02.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:02.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488584+0000", 2020-08-07T15:14:02.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036298319999999999 2020-08-07T15:14:02.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:02.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492214+0000", 2020-08-07T15:14:02.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039530099999999997 2020-08-07T15:14:02.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:02.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492609+0000", 2020-08-07T15:14:02.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5300000000000002e-07 2020-08-07T15:14:02.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:02.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492610+0000", 2020-08-07T15:14:02.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197321 2020-08-07T15:14:02.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:02.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492807+0000", 2020-08-07T15:14:02.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.76055856499999996 2020-08-07T15:14:02.489 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.489 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:02.489 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.490 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31014 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:14:02.490 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:14:02.490 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 315.41398997300001, 2020-08-07T15:14:02.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998, 2020-08-07T15:14:02.491 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:02.491 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:02.491 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:02.494 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:02.495 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:02.495 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31014 2020-08-07T15:14:02.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.496 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:02.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:02.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:14:02.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:02.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:02.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733631+0000", 2020-08-07T15:14:02.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3743999999999999e-05 2020-08-07T15:14:02.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:02.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733685+0000", 2020-08-07T15:14:02.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1720000000000007e-06 2020-08-07T15:14:02.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:02.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733693+0000", 2020-08-07T15:14:02.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3674e-05 2020-08-07T15:14:02.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:02.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733707+0000", 2020-08-07T15:14:02.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9249999999999998e-06 2020-08-07T15:14:02.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:02.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733714+0000", 2020-08-07T15:14:02.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8338000000000003e-05 2020-08-07T15:14:02.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733772+0000", 2020-08-07T15:14:02.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3019e-05 2020-08-07T15:14:02.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:02.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733785+0000", 2020-08-07T15:14:02.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75134856299999997 2020-08-07T15:14:02.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485133+0000", 2020-08-07T15:14:02.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8739999999999995e-05 2020-08-07T15:14:02.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:02.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485212+0000", 2020-08-07T15:14:02.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151244 2020-08-07T15:14:02.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:14:02.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.485363+0000", 2020-08-07T15:14:02.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032929090000000001 2020-08-07T15:14:02.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:02.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488656+0000", 2020-08-07T15:14:02.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.065e-06 2020-08-07T15:14:02.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:02.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488657+0000", 2020-08-07T15:14:02.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1123000000000003e-05 2020-08-07T15:14:02.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:02.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488729+0000", 2020-08-07T15:14:02.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035508330000000002 2020-08-07T15:14:02.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:02.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492279+0000", 2020-08-07T15:14:02.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00059376000000000003 2020-08-07T15:14:02.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:02.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492873+0000", 2020-08-07T15:14:02.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5700000000000002e-07 2020-08-07T15:14:02.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:02.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.492874+0000", 2020-08-07T15:14:02.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019264 2020-08-07T15:14:02.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:02.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493066+0000", 2020-08-07T15:14:02.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75943550599999998 2020-08-07T15:14:02.514 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.514 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:02.514 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.515 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31015 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:14:02.515 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:14:02.515 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 315.41386400699997, 2020-08-07T15:14:02.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002, 2020-08-07T15:14:02.516 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:02.516 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:02.516 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:02.516 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:02.516 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:02.517 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31015 2020-08-07T15:14:02.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.517 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:02.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:02.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:14:02.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:02.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:02.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733757+0000", 2020-08-07T15:14:02.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3707000000000001e-05 2020-08-07T15:14:02.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:02.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733791+0000", 2020-08-07T15:14:02.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8120000000000001e-06 2020-08-07T15:14:02.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:02.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733794+0000", 2020-08-07T15:14:02.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0728e-05 2020-08-07T15:14:02.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:02.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733805+0000", 2020-08-07T15:14:02.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8279999999999999e-06 2020-08-07T15:14:02.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:02.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733811+0000", 2020-08-07T15:14:02.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2955999999999999e-05 2020-08-07T15:14:02.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733864+0000", 2020-08-07T15:14:02.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1389000000000001e-05 2020-08-07T15:14:02.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:02.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.733875+0000", 2020-08-07T15:14:02.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75232711100000005 2020-08-07T15:14:02.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486202+0000", 2020-08-07T15:14:02.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027503400000000001 2020-08-07T15:14:02.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:02.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486478+0000", 2020-08-07T15:14:02.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033010400000000001 2020-08-07T15:14:02.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:14:02.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.486808+0000", 2020-08-07T15:14:02.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019382609999999999 2020-08-07T15:14:02.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:02.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488746+0000", 2020-08-07T15:14:02.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9400000000000004e-07 2020-08-07T15:14:02.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:02.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488747+0000", 2020-08-07T15:14:02.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4324000000000002e-05 2020-08-07T15:14:02.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:02.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.488821+0000", 2020-08-07T15:14:02.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043745640000000001 2020-08-07T15:14:02.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:02.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493196+0000", 2020-08-07T15:14:02.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030381000000000001 2020-08-07T15:14:02.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:02.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493499+0000", 2020-08-07T15:14:02.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4e-07 2020-08-07T15:14:02.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:02.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493500+0000", 2020-08-07T15:14:02.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017499 2020-08-07T15:14:02.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:02.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493675+0000", 2020-08-07T15:14:02.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75991835200000002 2020-08-07T15:14:02.540 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.540 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:02.540 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.541 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:31016 4.c 4:3687d5e2:::benchmark_data_smithi074_44586_object3876:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e99)", 2020-08-07T15:14:02.541 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:14:02.541 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 315.41326803200002, 2020-08-07T15:14:02.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971, 2020-08-07T15:14:02.542 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:02.542 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:02.542 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:02.542 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:02.543 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:02.543 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 31016 2020-08-07T15:14:02.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.543 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:02.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:02.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:14:02.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:02.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:02.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734353+0000", 2020-08-07T15:14:02.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0924000000000002e-05 2020-08-07T15:14:02.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:02.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734404+0000", 2020-08-07T15:14:02.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2020-08-07T15:14:02.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:02.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734410+0000", 2020-08-07T15:14:02.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3459999999999994e-06 2020-08-07T15:14:02.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:02.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734418+0000", 2020-08-07T15:14:02.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9640000000000002e-06 2020-08-07T15:14:02.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:02.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734423+0000", 2020-08-07T15:14:02.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7253999999999999e-05 2020-08-07T15:14:02.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734470+0000", 2020-08-07T15:14:02.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0249999999999997e-06 2020-08-07T15:14:02.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:02.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:46.734477+0000", 2020-08-07T15:14:02.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.75294592900000001 2020-08-07T15:14:02.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487423+0000", 2020-08-07T15:14:02.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031604300000000001 2020-08-07T15:14:02.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:02.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487739+0000", 2020-08-07T15:14:02.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000171149 2020-08-07T15:14:02.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3,6", 2020-08-07T15:14:02.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.487910+0000", 2020-08-07T15:14:02.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024207769999999998 2020-08-07T15:14:02.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:02.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490331+0000", 2020-08-07T15:14:02.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1650000000000001e-06 2020-08-07T15:14:02.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:02.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490332+0000", 2020-08-07T15:14:02.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9796999999999996e-05 2020-08-07T15:14:02.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:02.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.490402+0000", 2020-08-07T15:14:02.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029928849999999998 2020-08-07T15:14:02.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:02.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493395+0000", 2020-08-07T15:14:02.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034903199999999999 2020-08-07T15:14:02.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:02.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493744+0000", 2020-08-07T15:14:02.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8899999999999999e-07 2020-08-07T15:14:02.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:02.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493745+0000", 2020-08-07T15:14:02.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000197362 2020-08-07T15:14:02.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:02.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:08:47.493942+0000", 2020-08-07T15:14:02.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.759588971 2020-08-07T15:14:02.565 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.565 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:02.565 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.566 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5099.0:89632 5.f e207/164 5:fd260020:::benchmark_data_smithi074_53697_object11203:head v 207'5744, mlcod=207'5734)", 2020-08-07T15:14:02.566 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:21.551762+0000", 2020-08-07T15:14:02.566 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 160.59585917499999, 2020-08-07T15:14:02.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67527211499999995, 2020-08-07T15:14:02.567 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:02.567 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:02.567 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:02.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:02.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551762+0000", 2020-08-07T15:14:02.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:02.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:02.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551762+0000", 2020-08-07T15:14:02.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8560000000000003e-05 2020-08-07T15:14:02.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:02.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551810+0000", 2020-08-07T15:14:02.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9430000000000002e-06 2020-08-07T15:14:02.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:02.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:21.551814+0000", 2020-08-07T15:14:02.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67230272000000002 2020-08-07T15:14:02.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:02.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.224117+0000", 2020-08-07T15:14:02.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7120000000000003e-06 2020-08-07T15:14:02.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:02.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.224124+0000", 2020-08-07T15:14:02.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001719307 2020-08-07T15:14:02.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:02.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.225843+0000", 2020-08-07T15:14:02.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3902999999999997e-05 2020-08-07T15:14:02.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:02.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.225877+0000", 2020-08-07T15:14:02.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001110279 2020-08-07T15:14:02.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:02.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.226987+0000", 2020-08-07T15:14:02.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6690999999999997e-05 2020-08-07T15:14:02.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:02.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:02.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:02.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:22.227034+0000", 2020-08-07T15:14:02.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.67527211499999995 2020-08-07T15:14:02.585 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.585 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:02.585 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.585 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:02.586 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:02.586 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:14:02.591 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.3 to 1 (10000) 2020-08-07T15:14:02.592 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:14:02.592 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:14:02.590+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:14:02.604 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 4 1 2020-08-07T15:14:02.624 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 4 16 5712 5696 11.1242 12.9922 0.0112319 0.0111903 2020-08-07T15:14:02.696 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:14:02.695+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:14:02.772 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:14:02.773 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:14:02.773 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2020-08-07T15:14:02.773 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:14:02.792 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2020-08-07T15:14:02.795 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:14:02.795+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:14:02.902 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:14:02.894+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:14:02.965 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:14:02.965 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:14:02.966 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:14:02.966 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:14:02.967 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:14:02.977 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2020-08-07T15:14:02.997 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:14:02.996+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:14:03.100 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:14:03.097+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:14:03.156 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:14:03.157 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:14:03.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:14:03.158 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:14:03.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.159 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42440 4.9 4:9b3fd4e3:::benchmark_data_smithi074_44586_object5304:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:14:03.160 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:14:03.160 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 302.915738191, 2020-08-07T15:14:03.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9384075030000001, 2020-08-07T15:14:03.161 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:03.161 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:14:03.161 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:03.161 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:03.162 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:03.162 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42440 2020-08-07T15:14:03.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.162 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:03.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:03.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:14:03.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:03.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:03.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237060+0000", 2020-08-07T15:14:03.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6295000000000002e-05 2020-08-07T15:14:03.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:03.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237116+0000", 2020-08-07T15:14:03.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0390000000000001e-06 2020-08-07T15:14:03.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:03.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237122+0000", 2020-08-07T15:14:03.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8340000000000001e-05 2020-08-07T15:14:03.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:03.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237140+0000", 2020-08-07T15:14:03.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.045e-06 2020-08-07T15:14:03.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:03.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.237145+0000", 2020-08-07T15:14:03.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044548920000000002 2020-08-07T15:14:03.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241600+0000", 2020-08-07T15:14:03.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103336 2020-08-07T15:14:03.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:03.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241703+0000", 2020-08-07T15:14:03.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018660399999999999 2020-08-07T15:14:03.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:14:03.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.241890+0000", 2020-08-07T15:14:03.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00096796000000000002 2020-08-07T15:14:03.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:03.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.242858+0000", 2020-08-07T15:14:03.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039845000000000002 2020-08-07T15:14:03.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246843+0000", 2020-08-07T15:14:03.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3499999999999996e-07 2020-08-07T15:14:03.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:03.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246843+0000", 2020-08-07T15:14:03.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7132999999999995e-05 2020-08-07T15:14:03.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:03.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.246920+0000", 2020-08-07T15:14:03.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046993410000000001 2020-08-07T15:14:03.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.251620+0000", 2020-08-07T15:14:03.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040332956000000003 2020-08-07T15:14:03.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.291953+0000", 2020-08-07T15:14:03.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8390000000000003e-06 2020-08-07T15:14:03.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.291958+0000", 2020-08-07T15:14:03.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99961357500000003 2020-08-07T15:14:03.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291572+0000", 2020-08-07T15:14:03.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3913999999999999e-05 2020-08-07T15:14:03.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291586+0000", 2020-08-07T15:14:03.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88379723099999996 2020-08-07T15:14:03.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175383+0000", 2020-08-07T15:14:03.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3868000000000004e-05 2020-08-07T15:14:03.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:03.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175467+0000", 2020-08-07T15:14:03.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9384075030000001 2020-08-07T15:14:03.242 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.243 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:03.243 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.243 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42489 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:14:03.244 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:14:03.244 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 302.87243812899999, 2020-08-07T15:14:03.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952608639999999, 2020-08-07T15:14:03.244 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:03.244 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:14:03.245 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:03.245 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:03.245 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:03.245 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42489 2020-08-07T15:14:03.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.246 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:03.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:03.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:14:03.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:03.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:03.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280360+0000", 2020-08-07T15:14:03.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0379e-05 2020-08-07T15:14:03.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:03.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280410+0000", 2020-08-07T15:14:03.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0489999999999998e-06 2020-08-07T15:14:03.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:03.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280417+0000", 2020-08-07T15:14:03.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8281e-05 2020-08-07T15:14:03.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:03.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280435+0000", 2020-08-07T15:14:03.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7510000000000008e-06 2020-08-07T15:14:03.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:03.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280443+0000", 2020-08-07T15:14:03.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018142899999999999 2020-08-07T15:14:03.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280624+0000", 2020-08-07T15:14:03.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013473599999999999 2020-08-07T15:14:03.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:03.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280759+0000", 2020-08-07T15:14:03.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019964200000000001 2020-08-07T15:14:03.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:14:03.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280959+0000", 2020-08-07T15:14:03.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005588038 2020-08-07T15:14:03.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286547+0000", 2020-08-07T15:14:03.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.376e-05 2020-08-07T15:14:03.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:03.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286561+0000", 2020-08-07T15:14:03.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4199999999999995e-07 2020-08-07T15:14:03.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:03.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286561+0000", 2020-08-07T15:14:03.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9218e-05 2020-08-07T15:14:03.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:03.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286631+0000", 2020-08-07T15:14:03.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027909199999999999 2020-08-07T15:14:03.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286910+0000", 2020-08-07T15:14:03.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050904089999999997 2020-08-07T15:14:03.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292000+0000", 2020-08-07T15:14:03.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9579999999999996e-06 2020-08-07T15:14:03.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292009+0000", 2020-08-07T15:14:03.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99961402799999999 2020-08-07T15:14:03.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291623+0000", 2020-08-07T15:14:03.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2490000000000001e-06 2020-08-07T15:14:03.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291631+0000", 2020-08-07T15:14:03.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88387422199999999 2020-08-07T15:14:03.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175506+0000", 2020-08-07T15:14:03.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000114881 2020-08-07T15:14:03.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:03.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175620+0000", 2020-08-07T15:14:03.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952608639999999 2020-08-07T15:14:03.271 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.271 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:03.271 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.272 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42490 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:14:03.272 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:14:03.272 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 302.87231096099998, 2020-08-07T15:14:03.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89528059, 2020-08-07T15:14:03.272 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:03.273 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:14:03.273 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:03.273 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:03.273 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:03.275 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42490 2020-08-07T15:14:03.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.277 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:03.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:03.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:14:03.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:03.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:03.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280487+0000", 2020-08-07T15:14:03.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2571999999999997e-05 2020-08-07T15:14:03.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:03.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280539+0000", 2020-08-07T15:14:03.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999998e-06 2020-08-07T15:14:03.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:03.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280545+0000", 2020-08-07T15:14:03.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.134e-05 2020-08-07T15:14:03.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:03.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280556+0000", 2020-08-07T15:14:03.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.746e-06 2020-08-07T15:14:03.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:03.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280561+0000", 2020-08-07T15:14:03.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00089258999999999996 2020-08-07T15:14:03.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281453+0000", 2020-08-07T15:14:03.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5641000000000001e-05 2020-08-07T15:14:03.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:03.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281549+0000", 2020-08-07T15:14:03.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020555200000000001 2020-08-07T15:14:03.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:14:03.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281755+0000", 2020-08-07T15:14:03.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048878230000000003 2020-08-07T15:14:03.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:03.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286642+0000", 2020-08-07T15:14:03.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0954e-05 2020-08-07T15:14:03.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286693+0000", 2020-08-07T15:14:03.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0699999999999997e-07 2020-08-07T15:14:03.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:03.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286694+0000", 2020-08-07T15:14:03.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3697999999999998e-05 2020-08-07T15:14:03.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:03.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286768+0000", 2020-08-07T15:14:03.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017185900000000001 2020-08-07T15:14:03.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286940+0000", 2020-08-07T15:14:03.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051207819999999999 2020-08-07T15:14:03.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292060+0000", 2020-08-07T15:14:03.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8000000000000004e-06 2020-08-07T15:14:03.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292069+0000", 2020-08-07T15:14:03.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99959520599999996 2020-08-07T15:14:03.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291664+0000", 2020-08-07T15:14:03.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0160000000000008e-06 2020-08-07T15:14:03.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291672+0000", 2020-08-07T15:14:03.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88399374500000005 2020-08-07T15:14:03.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175666+0000", 2020-08-07T15:14:03.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101239 2020-08-07T15:14:03.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:03.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175767+0000", 2020-08-07T15:14:03.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89528059 2020-08-07T15:14:03.303 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.303 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:03.303 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.304 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42491 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:14:03.304 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:14:03.304 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 302.87220402600002, 2020-08-07T15:14:03.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895291936, 2020-08-07T15:14:03.305 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:03.305 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:14:03.305 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:03.305 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:03.306 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:03.306 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42491 2020-08-07T15:14:03.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.306 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:03.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:03.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:14:03.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:03.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:03.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280594+0000", 2020-08-07T15:14:03.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0402000000000001e-05 2020-08-07T15:14:03.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:03.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280624+0000", 2020-08-07T15:14:03.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0340000000000001e-06 2020-08-07T15:14:03.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:03.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280627+0000", 2020-08-07T15:14:03.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1886999999999999e-05 2020-08-07T15:14:03.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:03.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280649+0000", 2020-08-07T15:14:03.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5430000000000004e-06 2020-08-07T15:14:03.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:03.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280654+0000", 2020-08-07T15:14:03.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032800960000000001 2020-08-07T15:14:03.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.283934+0000", 2020-08-07T15:14:03.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011264699999999999 2020-08-07T15:14:03.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:03.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284046+0000", 2020-08-07T15:14:03.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019987299999999999 2020-08-07T15:14:03.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:14:03.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284246+0000", 2020-08-07T15:14:03.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025610960000000001 2020-08-07T15:14:03.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286807+0000", 2020-08-07T15:14:03.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1409999999999999e-06 2020-08-07T15:14:03.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:03.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286814+0000", 2020-08-07T15:14:03.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0499999999999999e-07 2020-08-07T15:14:03.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:03.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286815+0000", 2020-08-07T15:14:03.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8538000000000003e-05 2020-08-07T15:14:03.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:03.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286853+0000", 2020-08-07T15:14:03.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112914 2020-08-07T15:14:03.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286966+0000", 2020-08-07T15:14:03.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051403120000000002 2020-08-07T15:14:03.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292107+0000", 2020-08-07T15:14:03.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8019999999999992e-06 2020-08-07T15:14:03.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292114+0000", 2020-08-07T15:14:03.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.999590859 2020-08-07T15:14:03.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291705+0000", 2020-08-07T15:14:03.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.174e-06 2020-08-07T15:14:03.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291713+0000", 2020-08-07T15:14:03.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88407676000000002 2020-08-07T15:14:03.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175790+0000", 2020-08-07T15:14:03.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5452999999999996e-05 2020-08-07T15:14:03.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:03.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175886+0000", 2020-08-07T15:14:03.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895291936 2020-08-07T15:14:03.335 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.335 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:03.336 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.336 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42492 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:14:03.336 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:14:03.337 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 302.87211097599999, 2020-08-07T15:14:03.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895293774, 2020-08-07T15:14:03.337 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:03.337 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:14:03.337 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:03.338 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:03.338 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:03.338 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42492 2020-08-07T15:14:03.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.339 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:03.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:03.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:14:03.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:03.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:03.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280687+0000", 2020-08-07T15:14:03.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8272000000000001e-05 2020-08-07T15:14:03.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:03.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280735+0000", 2020-08-07T15:14:03.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2159999999999997e-06 2020-08-07T15:14:03.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:03.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280740+0000", 2020-08-07T15:14:03.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0251000000000001e-05 2020-08-07T15:14:03.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:03.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280750+0000", 2020-08-07T15:14:03.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3139999999999997e-06 2020-08-07T15:14:03.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:03.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280755+0000", 2020-08-07T15:14:03.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039343579999999998 2020-08-07T15:14:03.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284689+0000", 2020-08-07T15:14:03.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111648 2020-08-07T15:14:03.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:03.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284801+0000", 2020-08-07T15:14:03.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019563600000000001 2020-08-07T15:14:03.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:14:03.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.284996+0000", 2020-08-07T15:14:03.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018636709999999999 2020-08-07T15:14:03.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:03.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286860+0000", 2020-08-07T15:14:03.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001865802 2020-08-07T15:14:03.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288726+0000", 2020-08-07T15:14:03.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0399999999999996e-07 2020-08-07T15:14:03.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:03.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288726+0000", 2020-08-07T15:14:03.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6573999999999998e-05 2020-08-07T15:14:03.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:03.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288773+0000", 2020-08-07T15:14:03.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003455283 2020-08-07T15:14:03.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292228+0000", 2020-08-07T15:14:03.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0099999999999995e-06 2020-08-07T15:14:03.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292236+0000", 2020-08-07T15:14:03.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99950963999999998 2020-08-07T15:14:03.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291746+0000", 2020-08-07T15:14:03.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0779999999999996e-06 2020-08-07T15:14:03.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291754+0000", 2020-08-07T15:14:03.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.884152508 2020-08-07T15:14:03.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175906+0000", 2020-08-07T15:14:03.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4009000000000002e-05 2020-08-07T15:14:03.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:03.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.175980+0000", 2020-08-07T15:14:03.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895293774 2020-08-07T15:14:03.377 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.377 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:03.377 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.378 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42493 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:14:03.378 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:14:03.378 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 302.87200932899998, 2020-08-07T15:14:03.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952963789999999, 2020-08-07T15:14:03.378 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:03.379 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:14:03.379 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:03.379 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:03.379 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:03.379 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42493 2020-08-07T15:14:03.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.380 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:03.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:03.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:14:03.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:03.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:03.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280788+0000", 2020-08-07T15:14:03.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9740999999999997e-05 2020-08-07T15:14:03.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:03.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280838+0000", 2020-08-07T15:14:03.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7809999999999996e-06 2020-08-07T15:14:03.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:03.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280843+0000", 2020-08-07T15:14:03.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0648999999999999e-05 2020-08-07T15:14:03.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:03.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280854+0000", 2020-08-07T15:14:03.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3669999999999999e-06 2020-08-07T15:14:03.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:03.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280858+0000", 2020-08-07T15:14:03.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004850867 2020-08-07T15:14:03.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.285709+0000", 2020-08-07T15:14:03.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110037 2020-08-07T15:14:03.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:03.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.285819+0000", 2020-08-07T15:14:03.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020935699999999999 2020-08-07T15:14:03.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:14:03.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.286028+0000", 2020-08-07T15:14:03.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020900850000000002 2020-08-07T15:14:03.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:03.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288118+0000", 2020-08-07T15:14:03.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041669649999999999 2020-08-07T15:14:03.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292285+0000", 2020-08-07T15:14:03.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3820000000000004e-06 2020-08-07T15:14:03.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292294+0000", 2020-08-07T15:14:03.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949242900000002 2020-08-07T15:14:03.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291786+0000", 2020-08-07T15:14:03.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000003e-06 2020-08-07T15:14:03.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291794+0000", 2020-08-07T15:14:03.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88420652700000002 2020-08-07T15:14:03.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176000+0000", 2020-08-07T15:14:03.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4634000000000003e-05 2020-08-07T15:14:03.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:03.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176085+0000", 2020-08-07T15:14:03.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952963789999999 2020-08-07T15:14:03.398 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.398 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:03.398 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.399 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42494 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:14:03.399 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:14:03.399 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 302.87190722999998, 2020-08-07T15:14:03.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952882820000001, 2020-08-07T15:14:03.400 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:03.400 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:14:03.400 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:03.400 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:03.400 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:03.401 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42494 2020-08-07T15:14:03.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.401 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:03.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:03.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:14:03.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:03.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:03.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280890+0000", 2020-08-07T15:14:03.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0117999999999997e-05 2020-08-07T15:14:03.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:03.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280941+0000", 2020-08-07T15:14:03.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1240000000000004e-06 2020-08-07T15:14:03.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:03.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280946+0000", 2020-08-07T15:14:03.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0000000000000001e-05 2020-08-07T15:14:03.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:03.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280956+0000", 2020-08-07T15:14:03.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.279e-06 2020-08-07T15:14:03.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:03.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280960+0000", 2020-08-07T15:14:03.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0062727080000000001 2020-08-07T15:14:03.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287233+0000", 2020-08-07T15:14:03.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018362900000000001 2020-08-07T15:14:03.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:03.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287416+0000", 2020-08-07T15:14:03.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029536100000000001 2020-08-07T15:14:03.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:14:03.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.287712+0000", 2020-08-07T15:14:03.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0046231739999999999 2020-08-07T15:14:03.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292335+0000", 2020-08-07T15:14:03.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3860000000000001e-06 2020-08-07T15:14:03.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292342+0000", 2020-08-07T15:14:03.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99948501000000001 2020-08-07T15:14:03.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291827+0000", 2020-08-07T15:14:03.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2272999999999999e-05 2020-08-07T15:14:03.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291840+0000", 2020-08-07T15:14:03.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88426668100000005 2020-08-07T15:14:03.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176106+0000", 2020-08-07T15:14:03.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2539000000000005e-05 2020-08-07T15:14:03.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:03.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176179+0000", 2020-08-07T15:14:03.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8952882820000001 2020-08-07T15:14:03.416 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.416 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:03.416 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.417 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42495 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:14:03.418 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:14:03.418 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 302.87180613999999, 2020-08-07T15:14:03.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895286867, 2020-08-07T15:14:03.418 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:03.418 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:14:03.418 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:03.419 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:03.419 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:03.419 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42495 2020-08-07T15:14:03.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.419 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:03.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:03.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:14:03.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:03.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:03.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.280992+0000", 2020-08-07T15:14:03.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6690999999999997e-05 2020-08-07T15:14:03.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:03.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281038+0000", 2020-08-07T15:14:03.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7860000000000002e-06 2020-08-07T15:14:03.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:03.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281044+0000", 2020-08-07T15:14:03.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0429e-05 2020-08-07T15:14:03.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:03.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281054+0000", 2020-08-07T15:14:03.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9040000000000002e-06 2020-08-07T15:14:03.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:03.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281058+0000", 2020-08-07T15:14:03.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070941490000000001 2020-08-07T15:14:03.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288153+0000", 2020-08-07T15:14:03.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9006999999999998e-05 2020-08-07T15:14:03.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:03.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288222+0000", 2020-08-07T15:14:03.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012106400000000001 2020-08-07T15:14:03.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:14:03.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288343+0000", 2020-08-07T15:14:03.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004034752 2020-08-07T15:14:03.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292377+0000", 2020-08-07T15:14:03.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.537e-06 2020-08-07T15:14:03.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292385+0000", 2020-08-07T15:14:03.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949123500000003 2020-08-07T15:14:03.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291876+0000", 2020-08-07T15:14:03.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7729999999999999e-06 2020-08-07T15:14:03.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291884+0000", 2020-08-07T15:14:03.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88431700899999999 2020-08-07T15:14:03.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176201+0000", 2020-08-07T15:14:03.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7531e-05 2020-08-07T15:14:03.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:03.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176278+0000", 2020-08-07T15:14:03.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.895286867 2020-08-07T15:14:03.439 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.439 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:03.439 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.440 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:42496 4.9 4:9bc1fdeb:::benchmark_data_smithi074_44586_object5311:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:14:03.440 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:14:03.440 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 302.87170123099997, 2020-08-07T15:14:03.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89526942, 2020-08-07T15:14:03.440 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:03.441 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:14:03.441 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:03.441 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:03.441 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:03.441 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 42496 2020-08-07T15:14:03.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.442 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:03.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:03.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:14:03.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:03.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:03.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281096+0000", 2020-08-07T15:14:03.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9227999999999997e-05 2020-08-07T15:14:03.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:03.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281146+0000", 2020-08-07T15:14:03.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1120000000000004e-06 2020-08-07T15:14:03.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:03.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281151+0000", 2020-08-07T15:14:03.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2208999999999999e-05 2020-08-07T15:14:03.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:03.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281163+0000", 2020-08-07T15:14:03.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5469999999999998e-06 2020-08-07T15:14:03.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:03.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.281168+0000", 2020-08-07T15:14:03.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0076520829999999996 2020-08-07T15:14:03.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288820+0000", 2020-08-07T15:14:03.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5035000000000003e-05 2020-08-07T15:14:03.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:03.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.288885+0000", 2020-08-07T15:14:03.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116507 2020-08-07T15:14:03.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,7", 2020-08-07T15:14:03.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.289001+0000", 2020-08-07T15:14:03.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034121960000000002 2020-08-07T15:14:03.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292413+0000", 2020-08-07T15:14:03.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8109999999999997e-06 2020-08-07T15:14:03.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:00.292420+0000", 2020-08-07T15:14:03.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99949504600000005 2020-08-07T15:14:03.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291915+0000", 2020-08-07T15:14:03.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3999999999999992e-06 2020-08-07T15:14:03.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:01.291924+0000", 2020-08-07T15:14:03.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.88437394599999997 2020-08-07T15:14:03.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176298+0000", 2020-08-07T15:14:03.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8300000000000007e-05 2020-08-07T15:14:03.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:03.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:02.176366+0000", 2020-08-07T15:14:03.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.89526942 2020-08-07T15:14:03.459 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.459 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:03.459 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.460 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73081 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:14:03.460 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:14:03.460 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 275.92623195700003, 2020-08-07T15:14:03.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0005038420000001, 2020-08-07T15:14:03.460 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:03.461 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:03.461 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:03.461 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:03.461 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:03.461 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73081 2020-08-07T15:14:03.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.462 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:03.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:03.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:14:03.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:03.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:03.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226566+0000", 2020-08-07T15:14:03.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3016000000000003e-05 2020-08-07T15:14:03.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:03.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226609+0000", 2020-08-07T15:14:03.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4479999999999996e-06 2020-08-07T15:14:03.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:03.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226613+0000", 2020-08-07T15:14:03.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0189999999999995e-06 2020-08-07T15:14:03.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:03.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226622+0000", 2020-08-07T15:14:03.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5980000000000001e-06 2020-08-07T15:14:03.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:03.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226626+0000", 2020-08-07T15:14:03.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8088999999999999e-05 2020-08-07T15:14:03.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226684+0000", 2020-08-07T15:14:03.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2162e-05 2020-08-07T15:14:03.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226696+0000", 2020-08-07T15:14:03.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98675689 2020-08-07T15:14:03.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213453+0000", 2020-08-07T15:14:03.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012916800000000001 2020-08-07T15:14:03.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:03.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213582+0000", 2020-08-07T15:14:03.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015665899999999999 2020-08-07T15:14:03.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:14:03.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.213739+0000", 2020-08-07T15:14:03.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049940239999999997 2020-08-07T15:14:03.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:03.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218733+0000", 2020-08-07T15:14:03.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0079774779999999997 2020-08-07T15:14:03.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226710+0000", 2020-08-07T15:14:03.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.161e-06 2020-08-07T15:14:03.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:03.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226711+0000", 2020-08-07T15:14:03.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0105000000000001e-05 2020-08-07T15:14:03.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:03.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226772+0000", 2020-08-07T15:14:03.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5633999999999997e-05 2020-08-07T15:14:03.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226817+0000", 2020-08-07T15:14:03.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7379000000000001e-05 2020-08-07T15:14:03.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226875+0000", 2020-08-07T15:14:03.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8000000000000005e-07 2020-08-07T15:14:03.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:03.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.226875+0000", 2020-08-07T15:14:03.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019423199999999999 2020-08-07T15:14:03.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:03.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227070+0000", 2020-08-07T15:14:03.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0005038420000001 2020-08-07T15:14:03.481 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.481 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:03.481 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.482 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73082 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:14:03.482 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:14:03.482 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 275.92586149300001, 2020-08-07T15:14:03.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001326693, 2020-08-07T15:14:03.483 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:03.483 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:03.483 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:03.483 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:03.484 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:03.484 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73082 2020-08-07T15:14:03.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.484 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:03.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:03.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:14:03.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:03.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:03.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226936+0000", 2020-08-07T15:14:03.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6928000000000003e-05 2020-08-07T15:14:03.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:03.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226973+0000", 2020-08-07T15:14:03.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3630000000000004e-06 2020-08-07T15:14:03.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:03.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226977+0000", 2020-08-07T15:14:03.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9659999999999996e-06 2020-08-07T15:14:03.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:03.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226985+0000", 2020-08-07T15:14:03.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5800000000000002e-06 2020-08-07T15:14:03.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:03.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.226990+0000", 2020-08-07T15:14:03.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9101999999999998e-05 2020-08-07T15:14:03.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227039+0000", 2020-08-07T15:14:03.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9620000000000005e-06 2020-08-07T15:14:03.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227049+0000", 2020-08-07T15:14:03.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98722513999999995 2020-08-07T15:14:03.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214274+0000", 2020-08-07T15:14:03.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2700999999999994e-05 2020-08-07T15:14:03.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:03.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214347+0000", 2020-08-07T15:14:03.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015346899999999999 2020-08-07T15:14:03.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:14:03.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214500+0000", 2020-08-07T15:14:03.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042450309999999998 2020-08-07T15:14:03.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:03.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218745+0000", 2020-08-07T15:14:03.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.008379503 2020-08-07T15:14:03.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227125+0000", 2020-08-07T15:14:03.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7899999999999998e-07 2020-08-07T15:14:03.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:03.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227126+0000", 2020-08-07T15:14:03.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1610000000000002e-05 2020-08-07T15:14:03.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:03.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227177+0000", 2020-08-07T15:14:03.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489000000000003e-05 2020-08-07T15:14:03.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227226+0000", 2020-08-07T15:14:03.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00084338600000000001 2020-08-07T15:14:03.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228069+0000", 2020-08-07T15:14:03.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7e-07 2020-08-07T15:14:03.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:03.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228069+0000", 2020-08-07T15:14:03.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019351399999999999 2020-08-07T15:14:03.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:03.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228263+0000", 2020-08-07T15:14:03.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001326693 2020-08-07T15:14:03.519 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.519 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:03.520 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.520 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73083 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:14:03.520 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:14:03.520 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 275.92576723399998, 2020-08-07T15:14:03.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015968209999999, 2020-08-07T15:14:03.521 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:03.521 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:03.521 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:03.521 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:03.522 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:03.522 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73083 2020-08-07T15:14:03.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.522 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:03.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:03.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:14:03.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:03.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:03.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227030+0000", 2020-08-07T15:14:03.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1627000000000001e-05 2020-08-07T15:14:03.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:03.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227062+0000", 2020-08-07T15:14:03.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.253e-06 2020-08-07T15:14:03.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:03.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227065+0000", 2020-08-07T15:14:03.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3168e-05 2020-08-07T15:14:03.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:03.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227079+0000", 2020-08-07T15:14:03.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1549999999999999e-06 2020-08-07T15:14:03.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:03.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227082+0000", 2020-08-07T15:14:03.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5534999999999997e-05 2020-08-07T15:14:03.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227127+0000", 2020-08-07T15:14:03.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6823000000000001e-05 2020-08-07T15:14:03.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227144+0000", 2020-08-07T15:14:03.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98771770599999997 2020-08-07T15:14:03.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214862+0000", 2020-08-07T15:14:03.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8252999999999999e-05 2020-08-07T15:14:03.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:03.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.214930+0000", 2020-08-07T15:14:03.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013646400000000001 2020-08-07T15:14:03.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:14:03.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215066+0000", 2020-08-07T15:14:03.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036907089999999999 2020-08-07T15:14:03.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:03.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218757+0000", 2020-08-07T15:14:03.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0085133450000000003 2020-08-07T15:14:03.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227270+0000", 2020-08-07T15:14:03.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4899999999999995e-07 2020-08-07T15:14:03.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:03.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227271+0000", 2020-08-07T15:14:03.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4555999999999997e-05 2020-08-07T15:14:03.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:03.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227326+0000", 2020-08-07T15:14:03.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024594800000000001 2020-08-07T15:14:03.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227572+0000", 2020-08-07T15:14:03.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00086125700000000004 2020-08-07T15:14:03.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228433+0000", 2020-08-07T15:14:03.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3799999999999998e-07 2020-08-07T15:14:03.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:03.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228433+0000", 2020-08-07T15:14:03.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019413500000000001 2020-08-07T15:14:03.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:03.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228627+0000", 2020-08-07T15:14:03.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0015968209999999 2020-08-07T15:14:03.542 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.543 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:03.543 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.543 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73084 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:14:03.544 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:14:03.544 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 275.92567829500001, 2020-08-07T15:14:03.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0016919049999999, 2020-08-07T15:14:03.544 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:03.544 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:03.545 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:03.545 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:03.545 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:03.545 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73084 2020-08-07T15:14:03.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.546 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:03.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:03.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:14:03.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:03.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:03.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227119+0000", 2020-08-07T15:14:03.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5565999999999997e-05 2020-08-07T15:14:03.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:03.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227165+0000", 2020-08-07T15:14:03.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2020-08-07T15:14:03.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:03.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227169+0000", 2020-08-07T15:14:03.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6850000000000001e-06 2020-08-07T15:14:03.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:03.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227176+0000", 2020-08-07T15:14:03.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1949999999999999e-06 2020-08-07T15:14:03.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:03.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227179+0000", 2020-08-07T15:14:03.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0318000000000003e-05 2020-08-07T15:14:03.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227220+0000", 2020-08-07T15:14:03.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0349e-05 2020-08-07T15:14:03.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227230+0000", 2020-08-07T15:14:03.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98832962400000002 2020-08-07T15:14:03.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215560+0000", 2020-08-07T15:14:03.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013913099999999999 2020-08-07T15:14:03.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:03.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215699+0000", 2020-08-07T15:14:03.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000200006 2020-08-07T15:14:03.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:14:03.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.215899+0000", 2020-08-07T15:14:03.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028673650000000002 2020-08-07T15:14:03.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:03.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218766+0000", 2020-08-07T15:14:03.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086101779999999996 2020-08-07T15:14:03.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227376+0000", 2020-08-07T15:14:03.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0999999999999998e-07 2020-08-07T15:14:03.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:03.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227377+0000", 2020-08-07T15:14:03.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6552999999999997e-05 2020-08-07T15:14:03.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:03.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227454+0000", 2020-08-07T15:14:03.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000421717 2020-08-07T15:14:03.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227875+0000", 2020-08-07T15:14:03.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00081781300000000003 2020-08-07T15:14:03.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228693+0000", 2020-08-07T15:14:03.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1699999999999997e-07 2020-08-07T15:14:03.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:03.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228694+0000", 2020-08-07T15:14:03.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117483 2020-08-07T15:14:03.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:03.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228811+0000", 2020-08-07T15:14:03.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0016919049999999 2020-08-07T15:14:03.566 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.567 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:03.567 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.568 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73085 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:14:03.568 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:14:03.568 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 275.92554247999999, 2020-08-07T15:14:03.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0018015629999999, 2020-08-07T15:14:03.568 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:03.569 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:03.569 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:03.569 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:03.569 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:03.569 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73085 2020-08-07T15:14:03.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.570 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:03.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:03.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:14:03.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:03.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:03.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227255+0000", 2020-08-07T15:14:03.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6570999999999997e-05 2020-08-07T15:14:03.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:03.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227322+0000", 2020-08-07T15:14:03.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5509999999999996e-06 2020-08-07T15:14:03.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:03.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227328+0000", 2020-08-07T15:14:03.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2498e-05 2020-08-07T15:14:03.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:03.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227341+0000", 2020-08-07T15:14:03.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4389999999999999e-06 2020-08-07T15:14:03.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:03.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227346+0000", 2020-08-07T15:14:03.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3711999999999998e-05 2020-08-07T15:14:03.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227390+0000", 2020-08-07T15:14:03.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1868999999999999e-05 2020-08-07T15:14:03.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227402+0000", 2020-08-07T15:14:03.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98896525099999999 2020-08-07T15:14:03.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216367+0000", 2020-08-07T15:14:03.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012573 2020-08-07T15:14:03.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:03.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216493+0000", 2020-08-07T15:14:03.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020525200000000001 2020-08-07T15:14:03.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:14:03.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.216698+0000", 2020-08-07T15:14:03.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037596470000000001 2020-08-07T15:14:03.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:03.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.220458+0000", 2020-08-07T15:14:03.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070227190000000002 2020-08-07T15:14:03.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227480+0000", 2020-08-07T15:14:03.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6600000000000003e-07 2020-08-07T15:14:03.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:03.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227481+0000", 2020-08-07T15:14:03.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9898000000000003e-05 2020-08-07T15:14:03.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:03.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227521+0000", 2020-08-07T15:14:03.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039130000000000002 2020-08-07T15:14:03.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227912+0000", 2020-08-07T15:14:03.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00095502399999999996 2020-08-07T15:14:03.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228867+0000", 2020-08-07T15:14:03.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2020-08-07T15:14:03.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:03.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.228867+0000", 2020-08-07T15:14:03.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018933599999999999 2020-08-07T15:14:03.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:03.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229057+0000", 2020-08-07T15:14:03.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0018015629999999 2020-08-07T15:14:03.589 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.590 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:03.599 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.600 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73086 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:14:03.600 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:14:03.600 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 275.925409984, 2020-08-07T15:14:03.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002364923, 2020-08-07T15:14:03.602 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:03.603 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:03.603 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:03.603 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:03.603 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:03.603 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73086 2020-08-07T15:14:03.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.604 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:03.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:03.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:14:03.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:03.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:03.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227388+0000", 2020-08-07T15:14:03.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000238426 2020-08-07T15:14:03.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:03.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227626+0000", 2020-08-07T15:14:03.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9609999999999999e-06 2020-08-07T15:14:03.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:03.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227633+0000", 2020-08-07T15:14:03.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2727e-05 2020-08-07T15:14:03.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:03.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227646+0000", 2020-08-07T15:14:03.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1739999999999997e-06 2020-08-07T15:14:03.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:03.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227652+0000", 2020-08-07T15:14:03.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5004000000000001e-05 2020-08-07T15:14:03.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227737+0000", 2020-08-07T15:14:03.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2744e-05 2020-08-07T15:14:03.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.227750+0000", 2020-08-07T15:14:03.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98938776500000003 2020-08-07T15:14:03.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217138+0000", 2020-08-07T15:14:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012361499999999999 2020-08-07T15:14:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217261+0000", 2020-08-07T15:14:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000212504 2020-08-07T15:14:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:14:03.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217474+0000", 2020-08-07T15:14:03.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002995036 2020-08-07T15:14:03.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:03.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.220469+0000", 2020-08-07T15:14:03.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071283249999999996 2020-08-07T15:14:03.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227597+0000", 2020-08-07T15:14:03.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0499999999999999e-07 2020-08-07T15:14:03.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:03.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227597+0000", 2020-08-07T15:14:03.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1872999999999997e-05 2020-08-07T15:14:03.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:03.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227639+0000", 2020-08-07T15:14:03.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032010200000000001 2020-08-07T15:14:03.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227959+0000", 2020-08-07T15:14:03.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015980899999999999 2020-08-07T15:14:03.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229557+0000", 2020-08-07T15:14:03.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9299999999999999e-07 2020-08-07T15:14:03.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:03.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229558+0000", 2020-08-07T15:14:03.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019477900000000001 2020-08-07T15:14:03.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:03.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229753+0000", 2020-08-07T15:14:03.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002364923 2020-08-07T15:14:03.623 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.623 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:03.623 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.624 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73087 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:14:03.624 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:14:03.625 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 275.92446883100001, 2020-08-07T15:14:03.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001896506, 2020-08-07T15:14:03.625 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:03.625 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:03.625 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:03.626 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:03.626 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:03.626 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73087 2020-08-07T15:14:03.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.626 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:03.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:03.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:14:03.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:03.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:03.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228329+0000", 2020-08-07T15:14:03.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7354e-05 2020-08-07T15:14:03.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:03.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228386+0000", 2020-08-07T15:14:03.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8000000000000001e-06 2020-08-07T15:14:03.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:03.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228393+0000", 2020-08-07T15:14:03.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3784999999999999e-05 2020-08-07T15:14:03.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:03.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228407+0000", 2020-08-07T15:14:03.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8090000000000004e-06 2020-08-07T15:14:03.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:03.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228414+0000", 2020-08-07T15:14:03.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6907000000000003e-05 2020-08-07T15:14:03.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228461+0000", 2020-08-07T15:14:03.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8929999999999994e-06 2020-08-07T15:14:03.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228469+0000", 2020-08-07T15:14:03.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98950057700000005 2020-08-07T15:14:03.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.217970+0000", 2020-08-07T15:14:03.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000132182 2020-08-07T15:14:03.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:03.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218102+0000", 2020-08-07T15:14:03.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020073399999999999 2020-08-07T15:14:03.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:14:03.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218303+0000", 2020-08-07T15:14:03.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035951120000000001 2020-08-07T15:14:03.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:03.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.221898+0000", 2020-08-07T15:14:03.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057898660000000003 2020-08-07T15:14:03.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227688+0000", 2020-08-07T15:14:03.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0200000000000002e-07 2020-08-07T15:14:03.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:03.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227688+0000", 2020-08-07T15:14:03.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1477999999999996e-05 2020-08-07T15:14:03.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:03.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227760+0000", 2020-08-07T15:14:03.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016657480000000001 2020-08-07T15:14:03.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229426+0000", 2020-08-07T15:14:03.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061290400000000005 2020-08-07T15:14:03.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230039+0000", 2020-08-07T15:14:03.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2e-07 2020-08-07T15:14:03.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:03.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230039+0000", 2020-08-07T15:14:03.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018643500000000001 2020-08-07T15:14:03.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:03.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230225+0000", 2020-08-07T15:14:03.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001896506 2020-08-07T15:14:03.649 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.649 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:03.649 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.650 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:73088 4.9 4:91558757:::benchmark_data_smithi074_44586_object9135:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e132)", 2020-08-07T15:14:03.650 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:14:03.654 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 275.924078141, 2020-08-07T15:14:03.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001627153, 2020-08-07T15:14:03.655 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:03.655 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:03.655 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:03.655 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:03.655 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:03.656 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73088 2020-08-07T15:14:03.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.656 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:03.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:03.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:14:03.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:03.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:03.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228720+0000", 2020-08-07T15:14:03.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7741000000000003e-05 2020-08-07T15:14:03.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:03.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228767+0000", 2020-08-07T15:14:03.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6409999999999999e-06 2020-08-07T15:14:03.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:03.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228773+0000", 2020-08-07T15:14:03.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0874e-05 2020-08-07T15:14:03.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:03.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228784+0000", 2020-08-07T15:14:03.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1529999999999996e-06 2020-08-07T15:14:03.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:03.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228789+0000", 2020-08-07T15:14:03.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3945e-05 2020-08-07T15:14:03.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228833+0000", 2020-08-07T15:14:03.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4020000000000002e-06 2020-08-07T15:14:03.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:27.228841+0000", 2020-08-07T15:14:03.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98996913900000005 2020-08-07T15:14:03.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218810+0000", 2020-08-07T15:14:03.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112484 2020-08-07T15:14:03.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:03.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.218923+0000", 2020-08-07T15:14:03.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019891 2020-08-07T15:14:03.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:14:03.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.219122+0000", 2020-08-07T15:14:03.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027884099999999998 2020-08-07T15:14:03.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:03.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.221910+0000", 2020-08-07T15:14:03.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058753399999999997 2020-08-07T15:14:03.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227786+0000", 2020-08-07T15:14:03.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9200000000000002e-07 2020-08-07T15:14:03.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:03.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227786+0000", 2020-08-07T15:14:03.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8572000000000003e-05 2020-08-07T15:14:03.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:03.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.227824+0000", 2020-08-07T15:14:03.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001651296 2020-08-07T15:14:03.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.229476+0000", 2020-08-07T15:14:03.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00077620399999999996 2020-08-07T15:14:03.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:03.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230252+0000", 2020-08-07T15:14:03.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1600000000000002e-07 2020-08-07T15:14:03.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:03.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230252+0000", 2020-08-07T15:14:03.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4433999999999997e-05 2020-08-07T15:14:03.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:03.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:09:28.230347+0000", 2020-08-07T15:14:03.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.001627153 2020-08-07T15:14:03.676 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.676 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:03.676 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.677 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:128738 4.9 4.370c6199 (undecoded) ondisk+write+known_if_redirected e155)", 2020-08-07T15:14:03.677 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:03.204387+0000", 2020-08-07T15:14:03.677 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 239.94841048800001, 2020-08-07T15:14:03.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99077720899999999, 2020-08-07T15:14:03.680 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:03.680 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:14:03.680 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:03.681 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:03.681 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:03.681 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128738 2020-08-07T15:14:03.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.682 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:03.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:03.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204387+0000", 2020-08-07T15:14:03.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:03.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:03.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204387+0000", 2020-08-07T15:14:03.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4561999999999999e-05 2020-08-07T15:14:03.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:03.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204422+0000", 2020-08-07T15:14:03.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7270000000000001e-06 2020-08-07T15:14:03.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:03.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204428+0000", 2020-08-07T15:14:03.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3451e-05 2020-08-07T15:14:03.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:03.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204442+0000", 2020-08-07T15:14:03.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1589999999999998e-06 2020-08-07T15:14:03.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:03.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204448+0000", 2020-08-07T15:14:03.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3755999999999999e-05 2020-08-07T15:14:03.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204492+0000", 2020-08-07T15:14:03.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2450000000000001e-06 2020-08-07T15:14:03.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.204499+0000", 2020-08-07T15:14:03.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99065224600000001 2020-08-07T15:14:03.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195151+0000", 2020-08-07T15:14:03.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3063e-05 2020-08-07T15:14:03.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:03.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195164+0000", 2020-08-07T15:14:03.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99077720899999999 2020-08-07T15:14:03.692 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.692 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:03.692 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.693 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:128742 4.9 4.69b70ab9 (undecoded) ondisk+write+known_if_redirected e155)", 2020-08-07T15:14:03.693 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:03.205224+0000", 2020-08-07T15:14:03.693 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 239.94757385400001, 2020-08-07T15:14:03.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.989964548, 2020-08-07T15:14:03.694 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:03.694 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:14:03.694 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:03.694 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:03.695 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:03.695 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128742 2020-08-07T15:14:03.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.695 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:03.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:03.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205224+0000", 2020-08-07T15:14:03.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:03.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:03.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205224+0000", 2020-08-07T15:14:03.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1018e-05 2020-08-07T15:14:03.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:03.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205255+0000", 2020-08-07T15:14:03.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5840000000000002e-06 2020-08-07T15:14:03.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:03.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205260+0000", 2020-08-07T15:14:03.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2388999999999999e-05 2020-08-07T15:14:03.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:03.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205273+0000", 2020-08-07T15:14:03.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3799999999999999e-06 2020-08-07T15:14:03.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:03.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205279+0000", 2020-08-07T15:14:03.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6684999999999998e-05 2020-08-07T15:14:03.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205326+0000", 2020-08-07T15:14:03.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6200000000000004e-06 2020-08-07T15:14:03.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205332+0000", 2020-08-07T15:14:03.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98984944600000002 2020-08-07T15:14:03.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195181+0000", 2020-08-07T15:14:03.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4259999999999997e-06 2020-08-07T15:14:03.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:03.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195188+0000", 2020-08-07T15:14:03.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.989964548 2020-08-07T15:14:03.706 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.706 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:03.706 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.707 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4726.0:128743 4.9 4.1824cba9 (undecoded) ondisk+write+known_if_redirected e155)", 2020-08-07T15:14:03.707 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:10:03.205500+0000", 2020-08-07T15:14:03.707 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 239.94729799199999, 2020-08-07T15:14:03.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98971761999999996, 2020-08-07T15:14:03.708 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:03.708 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2020-08-07T15:14:03.708 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:03.708 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4726", 2020-08-07T15:14:03.709 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:03.709 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128743 2020-08-07T15:14:03.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.709 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:03.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:03.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205500+0000", 2020-08-07T15:14:03.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:03.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:03.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205500+0000", 2020-08-07T15:14:03.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6441000000000001e-05 2020-08-07T15:14:03.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:03.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205516+0000", 2020-08-07T15:14:03.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4429999999999999e-06 2020-08-07T15:14:03.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:03.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205520+0000", 2020-08-07T15:14:03.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4590000000000003e-06 2020-08-07T15:14:03.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:03.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205526+0000", 2020-08-07T15:14:03.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0429999999999999e-06 2020-08-07T15:14:03.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:03.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205529+0000", 2020-08-07T15:14:03.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1313000000000003e-05 2020-08-07T15:14:03.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205560+0000", 2020-08-07T15:14:03.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.934e-06 2020-08-07T15:14:03.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:03.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:03.205567+0000", 2020-08-07T15:14:03.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98964017599999998 2020-08-07T15:14:03.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195208+0000", 2020-08-07T15:14:03.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8109999999999997e-06 2020-08-07T15:14:03.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:03.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:10:04.195217+0000", 2020-08-07T15:14:03.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.98971761999999996 2020-08-07T15:14:03.719 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.719 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:03.719 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.720 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.720 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:03.720 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:14:03.721 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd unset nodeep-scrub 2020-08-07T15:14:03.723 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.4 to 1 (10000) 2020-08-07T15:14:03.723 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 5 1 2020-08-07T15:14:03.724 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 5 16 7217 7201 11.2507 11.7578 0.00988047 0.0110302 2020-08-07T15:14:03.725 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2020-08-07T15:14:03.725 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:14:03.721+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:14:03.823 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:14:03.822+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:14:03.912 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:14:03.913 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:14:03.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.913 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.5792.0:7673 7.b e308/301 7:d455366d:::benchmark_data_smithi074_71701_object959:head v 308'521, mlcod=308'519)", 2020-08-07T15:14:03.913 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:14:03.911249+0000", 2020-08-07T15:14:03.913 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00052993099999999998, 2020-08-07T15:14:03.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00054724899999999998, 2020-08-07T15:14:03.914 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:03.915 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:14:03.915 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:03.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:03.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:14:03.911249+0000", 2020-08-07T15:14:03.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:03.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:03.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:14:03.911249+0000", 2020-08-07T15:14:03.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4802999999999997e-05 2020-08-07T15:14:03.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:03.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:14:03.911304+0000", 2020-08-07T15:14:03.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6969999999999996e-06 2020-08-07T15:14:03.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:03.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:14:03.911311+0000", 2020-08-07T15:14:03.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4647999999999997e-05 2020-08-07T15:14:03.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:03.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:14:03.911355+0000", 2020-08-07T15:14:03.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.843e-06 2020-08-07T15:14:03.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:03.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:14:03.911360+0000", 2020-08-07T15:14:03.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2994999999999998e-05 2020-08-07T15:14:03.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:03.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:14:03.911393+0000", 2020-08-07T15:14:03.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7541e-05 2020-08-07T15:14:03.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:03.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:03.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:03.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:14:03.911411+0000", 2020-08-07T15:14:03.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016152699999999999 2020-08-07T15:14:03.924 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.924 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:03.924 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.924 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:03.925 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2020-08-07T15:14:03.925 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 1 2020-08-07T15:14:03.925 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:14:03.927 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:14:03.926+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:14:03.928 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2020-08-07T15:14:03.959 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:14:04.038 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:14:04.027+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:14:04.121 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:14:04.122 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:14:04.122 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:14:04.122 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:14:04.122 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:14:04.130 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2020-08-07T15:14:04.131 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:14:04.130+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:14:04.256 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:14:04.262 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:14:04.256+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:14:04.282 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:14:04.330 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:14:04.331 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2020-08-07T15:14:04.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2020-08-07T15:14:04.332 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2020-08-07T15:14:04.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.332 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118153 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:14:04.333 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.222760+0000", 2020-08-07T15:14:04.333 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 141.100053774, 2020-08-07T15:14:04.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0520728660000001, 2020-08-07T15:14:04.346 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:04.346 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:04.347 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:04.347 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:14:04.347 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:14:04.347 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118153 2020-08-07T15:14:04.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.348 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:04.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:04.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222760+0000", 2020-08-07T15:14:04.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:04.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:04.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222760+0000", 2020-08-07T15:14:04.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1894000000000002e-05 2020-08-07T15:14:04.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222812+0000", 2020-08-07T15:14:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4880000000000001e-06 2020-08-07T15:14:04.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:04.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222820+0000", 2020-08-07T15:14:04.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6744000000000001e-05 2020-08-07T15:14:04.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:04.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222836+0000", 2020-08-07T15:14:04.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5470000000000002e-06 2020-08-07T15:14:04.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:04.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222846+0000", 2020-08-07T15:14:04.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9478e-05 2020-08-07T15:14:04.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222925+0000", 2020-08-07T15:14:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0966e-05 2020-08-07T15:14:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2020-08-07T15:14:04.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222936+0000", 2020-08-07T15:14:04.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002974099 2020-08-07T15:14:04.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.225910+0000", 2020-08-07T15:14:04.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5197e-05 2020-08-07T15:14:04.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:04.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.225926+0000", 2020-08-07T15:14:04.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.02944747 2020-08-07T15:14:04.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.255373+0000", 2020-08-07T15:14:04.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022597200000000001 2020-08-07T15:14:04.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:04.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.255599+0000", 2020-08-07T15:14:04.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000194415 2020-08-07T15:14:04.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:14:04.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.255794+0000", 2020-08-07T15:14:04.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0076831360000000001 2020-08-07T15:14:04.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:04.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263477+0000", 2020-08-07T15:14:04.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01099552 2020-08-07T15:14:04.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274472+0000", 2020-08-07T15:14:04.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2948999999999998e-05 2020-08-07T15:14:04.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274515+0000", 2020-08-07T15:14:04.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.569999999999999e-07 2020-08-07T15:14:04.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:04.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274516+0000", 2020-08-07T15:14:04.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2750999999999997e-05 2020-08-07T15:14:04.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:04.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274569+0000", 2020-08-07T15:14:04.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4109000000000001e-05 2020-08-07T15:14:04.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274613+0000", 2020-08-07T15:14:04.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0432999999999997e-05 2020-08-07T15:14:04.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274653+0000", 2020-08-07T15:14:04.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5900000000000002e-07 2020-08-07T15:14:04.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:04.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274654+0000", 2020-08-07T15:14:04.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017928200000000001 2020-08-07T15:14:04.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:04.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274833+0000", 2020-08-07T15:14:04.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0520728660000001 2020-08-07T15:14:04.470 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:04.470 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:04.470 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:04.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.471 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118154 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:14:04.471 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.222899+0000", 2020-08-07T15:14:04.471 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 141.09991547199999, 2020-08-07T15:14:04.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0539106499999999, 2020-08-07T15:14:04.472 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:04.472 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:04.472 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:04.472 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:14:04.472 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:14:04.473 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118154 2020-08-07T15:14:04.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.473 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:04.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:04.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222899+0000", 2020-08-07T15:14:04.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:04.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:04.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222899+0000", 2020-08-07T15:14:04.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4440000000000001e-05 2020-08-07T15:14:04.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:04.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222953+0000", 2020-08-07T15:14:04.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4560000000000003e-06 2020-08-07T15:14:04.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:04.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222960+0000", 2020-08-07T15:14:04.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.201e-05 2020-08-07T15:14:04.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:04.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222972+0000", 2020-08-07T15:14:04.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4779999999999996e-06 2020-08-07T15:14:04.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:04.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.222979+0000", 2020-08-07T15:14:04.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2364000000000002e-05 2020-08-07T15:14:04.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223031+0000", 2020-08-07T15:14:04.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7779999999999996e-06 2020-08-07T15:14:04.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:14:04.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223040+0000", 2020-08-07T15:14:04.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029381720000000002 2020-08-07T15:14:04.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.225978+0000", 2020-08-07T15:14:04.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1806e-05 2020-08-07T15:14:04.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:04.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.225990+0000", 2020-08-07T15:14:04.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0316258110000001 2020-08-07T15:14:04.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.257616+0000", 2020-08-07T15:14:04.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000143759 2020-08-07T15:14:04.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:04.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.257760+0000", 2020-08-07T15:14:04.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000296249 2020-08-07T15:14:04.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:14:04.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.258056+0000", 2020-08-07T15:14:04.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054347309999999999 2020-08-07T15:14:04.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:04.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263491+0000", 2020-08-07T15:14:04.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011408567999999999 2020-08-07T15:14:04.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274899+0000", 2020-08-07T15:14:04.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030791400000000001 2020-08-07T15:14:04.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275207+0000", 2020-08-07T15:14:04.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6499999999999998e-07 2020-08-07T15:14:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275208+0000", 2020-08-07T15:14:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3047999999999998e-05 2020-08-07T15:14:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275261+0000", 2020-08-07T15:14:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000419706 2020-08-07T15:14:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275681+0000", 2020-08-07T15:14:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091386099999999997 2020-08-07T15:14:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276594+0000", 2020-08-07T15:14:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6000000000000001e-06 2020-08-07T15:14:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276597+0000", 2020-08-07T15:14:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000212134 2020-08-07T15:14:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276809+0000", 2020-08-07T15:14:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0539106499999999 2020-08-07T15:14:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118155 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:14:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223018+0000", 2020-08-07T15:14:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 141.09979566199999, 2020-08-07T15:14:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0540362350000001, 2020-08-07T15:14:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:14:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:14:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118155 2020-08-07T15:14:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223018+0000", 2020-08-07T15:14:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223018+0000", 2020-08-07T15:14:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8291999999999999e-05 2020-08-07T15:14:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223057+0000", 2020-08-07T15:14:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.805e-06 2020-08-07T15:14:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223060+0000", 2020-08-07T15:14:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0050000000000004e-06 2020-08-07T15:14:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223068+0000", 2020-08-07T15:14:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4846e-05 2020-08-07T15:14:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:04.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223083+0000", 2020-08-07T15:14:04.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8695000000000005e-05 2020-08-07T15:14:04.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223162+0000", 2020-08-07T15:14:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4919999999999996e-06 2020-08-07T15:14:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:14:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223169+0000", 2020-08-07T15:14:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028562219999999998 2020-08-07T15:14:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226026+0000", 2020-08-07T15:14:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.027e-05 2020-08-07T15:14:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226036+0000", 2020-08-07T15:14:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0327165199999999 2020-08-07T15:14:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.258752+0000", 2020-08-07T15:14:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012614899999999999 2020-08-07T15:14:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.258879+0000", 2020-08-07T15:14:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000258008 2020-08-07T15:14:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:14:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.259137+0000", 2020-08-07T15:14:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0067332820000000002 2020-08-07T15:14:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.265870+0000", 2020-08-07T15:14:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0090796000000000002 2020-08-07T15:14:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274950+0000", 2020-08-07T15:14:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000632601 2020-08-07T15:14:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275582+0000", 2020-08-07T15:14:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8999999999999997e-07 2020-08-07T15:14:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275583+0000", 2020-08-07T15:14:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6400999999999998e-05 2020-08-07T15:14:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275629+0000", 2020-08-07T15:14:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014887 2020-08-07T15:14:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275778+0000", 2020-08-07T15:14:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010622559999999999 2020-08-07T15:14:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276840+0000", 2020-08-07T15:14:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2199999999999999e-07 2020-08-07T15:14:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276841+0000", 2020-08-07T15:14:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000214009 2020-08-07T15:14:04.546 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:14:04.546 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:14:04.546 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:14:04.546 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:14:04.547 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:14:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277055+0000", 2020-08-07T15:14:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0540362350000001 2020-08-07T15:14:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118156 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:14:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223123+0000", 2020-08-07T15:14:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 141.099690796, 2020-08-07T15:14:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0542400009999999, 2020-08-07T15:14:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:14:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:14:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118156 2020-08-07T15:14:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223123+0000", 2020-08-07T15:14:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223123+0000", 2020-08-07T15:14:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2212999999999995e-05 2020-08-07T15:14:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223195+0000", 2020-08-07T15:14:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8709999999999999e-06 2020-08-07T15:14:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223202+0000", 2020-08-07T15:14:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3509999999999999e-05 2020-08-07T15:14:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223216+0000", 2020-08-07T15:14:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2729999999999997e-06 2020-08-07T15:14:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223223+0000", 2020-08-07T15:14:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6728000000000003e-05 2020-08-07T15:14:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223280+0000", 2020-08-07T15:14:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1046999999999999e-05 2020-08-07T15:14:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:14:04.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223291+0000", 2020-08-07T15:14:04.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027721030000000002 2020-08-07T15:14:04.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226063+0000", 2020-08-07T15:14:04.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1809000000000004e-05 2020-08-07T15:14:04.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:04.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226125+0000", 2020-08-07T15:14:04.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0337493870000001 2020-08-07T15:14:04.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.259874+0000", 2020-08-07T15:14:04.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000133974 2020-08-07T15:14:04.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:04.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.260008+0000", 2020-08-07T15:14:04.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00064730600000000005 2020-08-07T15:14:04.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:14:04.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.260655+0000", 2020-08-07T15:14:04.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052233740000000002 2020-08-07T15:14:04.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:04.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.265879+0000", 2020-08-07T15:14:04.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.009433238 2020-08-07T15:14:04.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275312+0000", 2020-08-07T15:14:04.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050955399999999995 2020-08-07T15:14:04.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275822+0000", 2020-08-07T15:14:04.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.58e-07 2020-08-07T15:14:04.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:04.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275822+0000", 2020-08-07T15:14:04.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5469000000000006e-05 2020-08-07T15:14:04.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:04.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275887+0000", 2020-08-07T15:14:04.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000371212 2020-08-07T15:14:04.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276259+0000", 2020-08-07T15:14:04.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00085558499999999996 2020-08-07T15:14:04.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277114+0000", 2020-08-07T15:14:04.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2020-08-07T15:14:04.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:04.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277115+0000", 2020-08-07T15:14:04.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024832000000000001 2020-08-07T15:14:04.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:04.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277363+0000", 2020-08-07T15:14:04.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0542400009999999 2020-08-07T15:14:04.575 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:04.575 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:04.575 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:04.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.576 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118157 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:14:04.576 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223267+0000", 2020-08-07T15:14:04.576 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 141.09954656100001, 2020-08-07T15:14:04.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0543752369999999, 2020-08-07T15:14:04.577 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:04.577 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:04.577 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:04.577 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:14:04.577 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:14:04.578 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118157 2020-08-07T15:14:04.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.578 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:04.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:04.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223267+0000", 2020-08-07T15:14:04.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:04.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:04.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223267+0000", 2020-08-07T15:14:04.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9616000000000001e-05 2020-08-07T15:14:04.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:04.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223317+0000", 2020-08-07T15:14:04.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1359999999999996e-06 2020-08-07T15:14:04.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:04.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223322+0000", 2020-08-07T15:14:04.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0529e-05 2020-08-07T15:14:04.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:04.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223333+0000", 2020-08-07T15:14:04.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1059999999999999e-06 2020-08-07T15:14:04.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:04.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223338+0000", 2020-08-07T15:14:04.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8609000000000002e-05 2020-08-07T15:14:04.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223386+0000", 2020-08-07T15:14:04.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3830000000000003e-06 2020-08-07T15:14:04.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:14:04.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223394+0000", 2020-08-07T15:14:04.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027643580000000002 2020-08-07T15:14:04.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226158+0000", 2020-08-07T15:14:04.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6809999999999993e-06 2020-08-07T15:14:04.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:04.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226168+0000", 2020-08-07T15:14:04.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.035171356 2020-08-07T15:14:04.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.261339+0000", 2020-08-07T15:14:04.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000130514 2020-08-07T15:14:04.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:04.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.261470+0000", 2020-08-07T15:14:04.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025685699999999998 2020-08-07T15:14:04.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:14:04.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.261727+0000", 2020-08-07T15:14:04.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075810410000000002 2020-08-07T15:14:04.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:04.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269308+0000", 2020-08-07T15:14:04.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064252490000000001 2020-08-07T15:14:04.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275733+0000", 2020-08-07T15:14:04.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020602000000000001 2020-08-07T15:14:04.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275939+0000", 2020-08-07T15:14:04.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999999e-07 2020-08-07T15:14:04.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:04.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275939+0000", 2020-08-07T15:14:04.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7071999999999997e-05 2020-08-07T15:14:04.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:04.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276016+0000", 2020-08-07T15:14:04.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029090700000000003 2020-08-07T15:14:04.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276307+0000", 2020-08-07T15:14:04.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001114929 2020-08-07T15:14:04.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277422+0000", 2020-08-07T15:14:04.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1200000000000003e-07 2020-08-07T15:14:04.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:04.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277423+0000", 2020-08-07T15:14:04.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021982600000000001 2020-08-07T15:14:04.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:04.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277643+0000", 2020-08-07T15:14:04.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0543752369999999 2020-08-07T15:14:04.643 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:04.644 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:04.644 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:04.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.645 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118158 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:14:04.664 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223375+0000", 2020-08-07T15:14:04.665 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 141.09943882499999, 2020-08-07T15:14:04.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0545579519999999, 2020-08-07T15:14:04.665 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:04.666 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:04.666 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:04.666 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:14:04.666 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:14:04.667 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118158 2020-08-07T15:14:04.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.667 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:04.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:04.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223375+0000", 2020-08-07T15:14:04.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:04.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:04.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223375+0000", 2020-08-07T15:14:04.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7657000000000001e-05 2020-08-07T15:14:04.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:04.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223423+0000", 2020-08-07T15:14:04.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0810000000000001e-06 2020-08-07T15:14:04.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:04.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223428+0000", 2020-08-07T15:14:04.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7839999999999998e-06 2020-08-07T15:14:04.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:04.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223438+0000", 2020-08-07T15:14:04.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.634e-06 2020-08-07T15:14:04.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:04.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223442+0000", 2020-08-07T15:14:04.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0154999999999998e-05 2020-08-07T15:14:04.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223482+0000", 2020-08-07T15:14:04.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9579999999999998e-06 2020-08-07T15:14:04.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:14:04.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223487+0000", 2020-08-07T15:14:04.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027060999999999999 2020-08-07T15:14:04.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226194+0000", 2020-08-07T15:14:04.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1311999999999999e-05 2020-08-07T15:14:04.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:04.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226205+0000", 2020-08-07T15:14:04.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0361494609999999 2020-08-07T15:14:04.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.262354+0000", 2020-08-07T15:14:04.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9950000000000004e-05 2020-08-07T15:14:04.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:04.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.262454+0000", 2020-08-07T15:14:04.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000244093 2020-08-07T15:14:04.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:14:04.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.262698+0000", 2020-08-07T15:14:04.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0066176150000000003 2020-08-07T15:14:04.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:04.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269316+0000", 2020-08-07T15:14:04.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006768998 2020-08-07T15:14:04.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276085+0000", 2020-08-07T15:14:04.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9524000000000002e-05 2020-08-07T15:14:04.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276135+0000", 2020-08-07T15:14:04.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6100000000000001e-07 2020-08-07T15:14:04.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:04.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276135+0000", 2020-08-07T15:14:04.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1214e-05 2020-08-07T15:14:04.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:04.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276206+0000", 2020-08-07T15:14:04.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014914999999999999 2020-08-07T15:14:04.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276355+0000", 2020-08-07T15:14:04.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013427770000000001 2020-08-07T15:14:04.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277698+0000", 2020-08-07T15:14:04.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8599999999999998e-07 2020-08-07T15:14:04.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:04.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277699+0000", 2020-08-07T15:14:04.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023444199999999999 2020-08-07T15:14:04.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:04.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277933+0000", 2020-08-07T15:14:04.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0545579519999999 2020-08-07T15:14:04.777 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:04.777 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:04.777 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:04.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.778 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118159 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:14:04.778 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223476+0000", 2020-08-07T15:14:04.779 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 141.09933765900001, 2020-08-07T15:14:04.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0547920749999999, 2020-08-07T15:14:04.779 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:04.780 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:04.780 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:04.780 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:14:04.780 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:14:04.781 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118159 2020-08-07T15:14:04.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.781 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:04.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:04.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223476+0000", 2020-08-07T15:14:04.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:04.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:04.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223476+0000", 2020-08-07T15:14:04.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6094000000000003e-05 2020-08-07T15:14:04.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:04.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223522+0000", 2020-08-07T15:14:04.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6650000000000002e-06 2020-08-07T15:14:04.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:04.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223527+0000", 2020-08-07T15:14:04.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5629999999999998e-06 2020-08-07T15:14:04.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:04.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223537+0000", 2020-08-07T15:14:04.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6810000000000003e-06 2020-08-07T15:14:04.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:04.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223542+0000", 2020-08-07T15:14:04.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6680000000000001e-05 2020-08-07T15:14:04.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223579+0000", 2020-08-07T15:14:04.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2020-08-07T15:14:04.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:14:04.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223586+0000", 2020-08-07T15:14:04.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026469250000000001 2020-08-07T15:14:04.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226233+0000", 2020-08-07T15:14:04.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3130000000000004e-06 2020-08-07T15:14:04.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:04.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226242+0000", 2020-08-07T15:14:04.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.037297957 2020-08-07T15:14:04.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263540+0000", 2020-08-07T15:14:04.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4307000000000004e-05 2020-08-07T15:14:04.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:04.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263634+0000", 2020-08-07T15:14:04.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019544300000000001 2020-08-07T15:14:04.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:14:04.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.263830+0000", 2020-08-07T15:14:04.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054939949999999998 2020-08-07T15:14:04.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:04.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269324+0000", 2020-08-07T15:14:04.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070859139999999996 2020-08-07T15:14:04.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276410+0000", 2020-08-07T15:14:04.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0924000000000002e-05 2020-08-07T15:14:04.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276461+0000", 2020-08-07T15:14:04.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4099999999999998e-07 2020-08-07T15:14:04.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:04.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276461+0000", 2020-08-07T15:14:04.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8027000000000004e-05 2020-08-07T15:14:04.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:04.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.276539+0000", 2020-08-07T15:14:04.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001458453 2020-08-07T15:14:04.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.277998+0000", 2020-08-07T15:14:04.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7283000000000001e-05 2020-08-07T15:14:04.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278045+0000", 2020-08-07T15:14:04.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9100000000000004e-07 2020-08-07T15:14:04.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:04.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278046+0000", 2020-08-07T15:14:04.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022290400000000001 2020-08-07T15:14:04.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:04.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278268+0000", 2020-08-07T15:14:04.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0547920749999999 2020-08-07T15:14:04.817 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:04.817 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:04.818 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:04.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.819 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118160 5.2 5:4b847943:::benchmark_data_smithi074_53697_object14769:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:14:04.819 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223579+0000", 2020-08-07T15:14:04.819 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 141.09923467799999, 2020-08-07T15:14:04.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0553689980000001, 2020-08-07T15:14:04.820 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:04.820 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:04.820 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:04.821 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:14:04.821 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:14:04.821 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118160 2020-08-07T15:14:04.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.822 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:04.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:04.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223579+0000", 2020-08-07T15:14:04.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:04.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:04.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223579+0000", 2020-08-07T15:14:04.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5401000000000001e-05 2020-08-07T15:14:04.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:04.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223625+0000", 2020-08-07T15:14:04.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9069999999999998e-06 2020-08-07T15:14:04.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:04.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223630+0000", 2020-08-07T15:14:04.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4800000000000007e-06 2020-08-07T15:14:04.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:04.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223639+0000", 2020-08-07T15:14:04.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5410000000000002e-06 2020-08-07T15:14:04.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:04.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223644+0000", 2020-08-07T15:14:04.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1476000000000002e-05 2020-08-07T15:14:04.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223675+0000", 2020-08-07T15:14:04.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0240000000000003e-06 2020-08-07T15:14:04.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:14:04.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223681+0000", 2020-08-07T15:14:04.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025870820000000001 2020-08-07T15:14:04.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226268+0000", 2020-08-07T15:14:04.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0540000000000001e-05 2020-08-07T15:14:04.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:04.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226279+0000", 2020-08-07T15:14:04.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.038209656 2020-08-07T15:14:04.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.264488+0000", 2020-08-07T15:14:04.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029908499999999999 2020-08-07T15:14:04.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:04.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.264788+0000", 2020-08-07T15:14:04.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000216997 2020-08-07T15:14:04.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:14:04.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.265005+0000", 2020-08-07T15:14:04.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0060308230000000003 2020-08-07T15:14:04.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:04.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271035+0000", 2020-08-07T15:14:04.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0072889499999999998 2020-08-07T15:14:04.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278324+0000", 2020-08-07T15:14:04.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4982999999999999e-05 2020-08-07T15:14:04.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278369+0000", 2020-08-07T15:14:04.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4000000000000002e-07 2020-08-07T15:14:04.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:04.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278370+0000", 2020-08-07T15:14:04.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7656000000000003e-05 2020-08-07T15:14:04.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:04.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278447+0000", 2020-08-07T15:14:04.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000277416 2020-08-07T15:14:04.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278725+0000", 2020-08-07T15:14:04.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2345000000000001e-05 2020-08-07T15:14:04.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278777+0000", 2020-08-07T15:14:04.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8500000000000002e-07 2020-08-07T15:14:04.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:04.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278778+0000", 2020-08-07T15:14:04.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017071100000000001 2020-08-07T15:14:04.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:04.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278948+0000", 2020-08-07T15:14:04.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0553689980000001 2020-08-07T15:14:04.863 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:04.863 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:04.863 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:04.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.864 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118441 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:14:04.864 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223679+0000", 2020-08-07T15:14:04.864 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 141.099134663, 2020-08-07T15:14:04.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057502779, 2020-08-07T15:14:04.865 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:04.865 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:04.865 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:04.865 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:14:04.866 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:14:04.866 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118441 2020-08-07T15:14:04.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.866 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:04.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:04.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223679+0000", 2020-08-07T15:14:04.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:04.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:04.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223679+0000", 2020-08-07T15:14:04.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5559999999999997e-05 2020-08-07T15:14:04.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:04.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223725+0000", 2020-08-07T15:14:04.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1250000000000001e-06 2020-08-07T15:14:04.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:04.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223730+0000", 2020-08-07T15:14:04.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0046e-05 2020-08-07T15:14:04.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:04.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223740+0000", 2020-08-07T15:14:04.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5979999999999999e-06 2020-08-07T15:14:04.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:04.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223745+0000", 2020-08-07T15:14:04.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7234999999999998e-05 2020-08-07T15:14:04.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223782+0000", 2020-08-07T15:14:04.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7789999999999998e-06 2020-08-07T15:14:04.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:14:04.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223789+0000", 2020-08-07T15:14:04.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002516169 2020-08-07T15:14:04.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226305+0000", 2020-08-07T15:14:04.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3860000000000007e-06 2020-08-07T15:14:04.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:04.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226313+0000", 2020-08-07T15:14:04.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0396126269999999 2020-08-07T15:14:04.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.265926+0000", 2020-08-07T15:14:04.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00079104700000000002 2020-08-07T15:14:04.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:04.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.266717+0000", 2020-08-07T15:14:04.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024298599999999999 2020-08-07T15:14:04.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:14:04.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.266960+0000", 2020-08-07T15:14:04.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040855509999999998 2020-08-07T15:14:04.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:04.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271045+0000", 2020-08-07T15:14:04.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0074549170000000001 2020-08-07T15:14:04.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278500+0000", 2020-08-07T15:14:04.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8816999999999997e-05 2020-08-07T15:14:04.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278549+0000", 2020-08-07T15:14:04.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9000000000000002e-07 2020-08-07T15:14:04.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:04.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278550+0000", 2020-08-07T15:14:04.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0120999999999997e-05 2020-08-07T15:14:04.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:04.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278600+0000", 2020-08-07T15:14:04.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00092631699999999996 2020-08-07T15:14:04.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279526+0000", 2020-08-07T15:14:04.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014504520000000001 2020-08-07T15:14:04.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280976+0000", 2020-08-07T15:14:04.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6899999999999997e-07 2020-08-07T15:14:04.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:04.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280977+0000", 2020-08-07T15:14:04.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020508700000000001 2020-08-07T15:14:04.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:04.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281182+0000", 2020-08-07T15:14:04.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057502779 2020-08-07T15:14:04.910 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:04.910 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:04.910 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:04.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.911 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118442 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:14:04.912 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223780+0000", 2020-08-07T15:14:04.912 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 141.09903379299999, 2020-08-07T15:14:04.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0575529850000001, 2020-08-07T15:14:04.913 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:04.913 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:04.913 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:04.914 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:14:04.914 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:14:04.914 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118442 2020-08-07T15:14:04.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.915 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:04.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:04.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223780+0000", 2020-08-07T15:14:04.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:04.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:04.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223780+0000", 2020-08-07T15:14:04.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0340999999999998e-05 2020-08-07T15:14:04.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:04.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223831+0000", 2020-08-07T15:14:04.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3499999999999996e-06 2020-08-07T15:14:04.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:04.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223836+0000", 2020-08-07T15:14:04.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0414999999999999e-05 2020-08-07T15:14:04.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:04.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223846+0000", 2020-08-07T15:14:04.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9239999999999998e-06 2020-08-07T15:14:04.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:04.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223851+0000", 2020-08-07T15:14:04.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5946000000000003e-05 2020-08-07T15:14:04.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223907+0000", 2020-08-07T15:14:04.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6189999999999996e-06 2020-08-07T15:14:04.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:14:04.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223914+0000", 2020-08-07T15:14:04.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002424149 2020-08-07T15:14:04.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226338+0000", 2020-08-07T15:14:04.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2830000000000001e-06 2020-08-07T15:14:04.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:04.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226346+0000", 2020-08-07T15:14:04.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0412357219999999 2020-08-07T15:14:04.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.267582+0000", 2020-08-07T15:14:04.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000123924 2020-08-07T15:14:04.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:04.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.267706+0000", 2020-08-07T15:14:04.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000216376 2020-08-07T15:14:04.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:14:04.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.267922+0000", 2020-08-07T15:14:04.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031340790000000001 2020-08-07T15:14:04.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:04.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271056+0000", 2020-08-07T15:14:04.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.007623276 2020-08-07T15:14:04.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.278680+0000", 2020-08-07T15:14:04.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000333853 2020-08-07T15:14:04.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279013+0000", 2020-08-07T15:14:04.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.02e-06 2020-08-07T15:14:04.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:04.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279014+0000", 2020-08-07T15:14:04.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6306999999999999e-05 2020-08-07T15:14:04.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:04.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279101+0000", 2020-08-07T15:14:04.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050739800000000005 2020-08-07T15:14:04.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279608+0000", 2020-08-07T15:14:04.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001617098 2020-08-07T15:14:04.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281225+0000", 2020-08-07T15:14:04.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9700000000000002e-07 2020-08-07T15:14:04.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:04.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281226+0000", 2020-08-07T15:14:04.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107508 2020-08-07T15:14:04.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:04.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281333+0000", 2020-08-07T15:14:04.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0575529850000001 2020-08-07T15:14:04.968 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:04.968 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:04.968 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:04.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.969 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118443 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:14:04.969 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223889+0000", 2020-08-07T15:14:04.970 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 141.09892512299999, 2020-08-07T15:14:04.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.05758606, 2020-08-07T15:14:04.971 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:04.971 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:04.971 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:04.971 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:14:04.971 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:14:04.972 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118443 2020-08-07T15:14:04.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.972 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:04.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:04.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223889+0000", 2020-08-07T15:14:04.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:04.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:04.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223889+0000", 2020-08-07T15:14:04.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8148e-05 2020-08-07T15:14:04.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:04.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223937+0000", 2020-08-07T15:14:04.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2789999999999998e-06 2020-08-07T15:14:04.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:04.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223942+0000", 2020-08-07T15:14:04.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0472000000000001e-05 2020-08-07T15:14:04.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:04.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223953+0000", 2020-08-07T15:14:04.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6580000000000001e-06 2020-08-07T15:14:04.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:04.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223957+0000", 2020-08-07T15:14:04.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5981000000000001e-05 2020-08-07T15:14:04.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223983+0000", 2020-08-07T15:14:04.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.288e-06 2020-08-07T15:14:04.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:14:04.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223989+0000", 2020-08-07T15:14:04.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024051889999999999 2020-08-07T15:14:04.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226394+0000", 2020-08-07T15:14:04.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2439000000000001e-05 2020-08-07T15:14:04.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:04.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226406+0000", 2020-08-07T15:14:04.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0421168519999999 2020-08-07T15:14:04.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:04.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.268523+0000", 2020-08-07T15:14:04.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0449000000000002e-05 2020-08-07T15:14:04.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:04.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.268604+0000", 2020-08-07T15:14:04.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014104300000000001 2020-08-07T15:14:04.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:14:04.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.268745+0000", 2020-08-07T15:14:04.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051249829999999996 2020-08-07T15:14:04.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:04.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.273870+0000", 2020-08-07T15:14:04.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052772840000000001 2020-08-07T15:14:04.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279147+0000", 2020-08-07T15:14:04.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6060000000000003e-05 2020-08-07T15:14:04.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:04.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279193+0000", 2020-08-07T15:14:04.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9999999999999997e-07 2020-08-07T15:14:04.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:04.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279194+0000", 2020-08-07T15:14:04.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3566000000000003e-05 2020-08-07T15:14:04.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:04.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:04.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:05.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279237+0000", 2020-08-07T15:14:05.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011546359999999999 2020-08-07T15:14:05.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280392+0000", 2020-08-07T15:14:05.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00097857799999999991 2020-08-07T15:14:05.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281370+0000", 2020-08-07T15:14:05.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999999e-07 2020-08-07T15:14:05.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:05.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281371+0000", 2020-08-07T15:14:05.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104119 2020-08-07T15:14:05.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:05.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281475+0000", 2020-08-07T15:14:05.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.05758606 2020-08-07T15:14:05.008 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:05.008 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:05.009 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:05.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.011 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118444 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:14:05.011 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.223994+0000", 2020-08-07T15:14:05.011 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 141.09881974800001, 2020-08-07T15:14:05.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057634572, 2020-08-07T15:14:05.012 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:05.012 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:05.012 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:05.012 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:14:05.013 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:14:05.014 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118444 2020-08-07T15:14:05.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.014 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:05.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:05.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223994+0000", 2020-08-07T15:14:05.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:05.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:05.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.223994+0000", 2020-08-07T15:14:05.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1279999999999997e-05 2020-08-07T15:14:05.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:05.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224046+0000", 2020-08-07T15:14:05.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6930000000000004e-06 2020-08-07T15:14:05.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:05.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224051+0000", 2020-08-07T15:14:05.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.095e-05 2020-08-07T15:14:05.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:05.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224062+0000", 2020-08-07T15:14:05.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9529999999999998e-06 2020-08-07T15:14:05.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:05.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224067+0000", 2020-08-07T15:14:05.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000162091 2020-08-07T15:14:05.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:05.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224229+0000", 2020-08-07T15:14:05.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.276e-06 2020-08-07T15:14:05.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:14:05.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224238+0000", 2020-08-07T15:14:05.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022140409999999999 2020-08-07T15:14:05.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:05.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226452+0000", 2020-08-07T15:14:05.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5951999999999999e-05 2020-08-07T15:14:05.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:05.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226467+0000", 2020-08-07T15:14:05.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0428868099999999 2020-08-07T15:14:05.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:05.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269354+0000", 2020-08-07T15:14:05.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5264000000000003e-05 2020-08-07T15:14:05.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:05.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269430+0000", 2020-08-07T15:14:05.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020438499999999999 2020-08-07T15:14:05.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:14:05.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.269634+0000", 2020-08-07T15:14:05.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042509959999999999 2020-08-07T15:14:05.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:05.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.273885+0000", 2020-08-07T15:14:05.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054290140000000002 2020-08-07T15:14:05.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279314+0000", 2020-08-07T15:14:05.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00038996700000000002 2020-08-07T15:14:05.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279704+0000", 2020-08-07T15:14:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5700000000000002e-07 2020-08-07T15:14:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:05.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279705+0000", 2020-08-07T15:14:05.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9377999999999997e-05 2020-08-07T15:14:05.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:05.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279794+0000", 2020-08-07T15:14:05.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00064521299999999995 2020-08-07T15:14:05.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280439+0000", 2020-08-07T15:14:05.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001072185 2020-08-07T15:14:05.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281511+0000", 2020-08-07T15:14:05.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7300000000000002e-07 2020-08-07T15:14:05.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:05.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281512+0000", 2020-08-07T15:14:05.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117094 2020-08-07T15:14:05.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:05.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281629+0000", 2020-08-07T15:14:05.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057634572 2020-08-07T15:14:05.054 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:05.054 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:05.054 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:05.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.055 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118445 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:14:05.055 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.224116+0000", 2020-08-07T15:14:05.055 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 141.09869780700001, 2020-08-07T15:14:05.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057744163, 2020-08-07T15:14:05.055 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:05.056 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:05.056 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:05.056 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:14:05.056 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:14:05.056 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118445 2020-08-07T15:14:05.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.057 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:05.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:05.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224116+0000", 2020-08-07T15:14:05.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:05.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:05.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224116+0000", 2020-08-07T15:14:05.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3542999999999999e-05 2020-08-07T15:14:05.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:05.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224170+0000", 2020-08-07T15:14:05.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.446e-06 2020-08-07T15:14:05.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:05.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224175+0000", 2020-08-07T15:14:05.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1473e-05 2020-08-07T15:14:05.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:05.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224187+0000", 2020-08-07T15:14:05.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9449999999999998e-06 2020-08-07T15:14:05.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:05.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224193+0000", 2020-08-07T15:14:05.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4945999999999997e-05 2020-08-07T15:14:05.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:05.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224268+0000", 2020-08-07T15:14:05.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5799999999999992e-06 2020-08-07T15:14:05.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:14:05.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224276+0000", 2020-08-07T15:14:05.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022246980000000002 2020-08-07T15:14:05.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226501+0000", 2020-08-07T15:14:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1359999999999997e-06 2020-08-07T15:14:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:05.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226509+0000", 2020-08-07T15:14:05.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.043599741 2020-08-07T15:14:05.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.270109+0000", 2020-08-07T15:14:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000129503 2020-08-07T15:14:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.270238+0000", 2020-08-07T15:14:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021162500000000001 2020-08-07T15:14:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:14:05.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.270450+0000", 2020-08-07T15:14:05.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036313090000000001 2020-08-07T15:14:05.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:05.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274081+0000", 2020-08-07T15:14:05.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00557165 2020-08-07T15:14:05.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279653+0000", 2020-08-07T15:14:05.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000196658 2020-08-07T15:14:05.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279849+0000", 2020-08-07T15:14:05.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6700000000000003e-07 2020-08-07T15:14:05.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:05.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279850+0000", 2020-08-07T15:14:05.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6522999999999998e-05 2020-08-07T15:14:05.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:05.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279937+0000", 2020-08-07T15:14:05.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00055050400000000005 2020-08-07T15:14:05.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280487+0000", 2020-08-07T15:14:05.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011833760000000001 2020-08-07T15:14:05.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281670+0000", 2020-08-07T15:14:05.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2020-08-07T15:14:05.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:05.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281671+0000", 2020-08-07T15:14:05.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001896 2020-08-07T15:14:05.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:05.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281860+0000", 2020-08-07T15:14:05.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057744163 2020-08-07T15:14:05.085 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:05.085 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:05.085 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:05.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.086 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118446 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:14:05.086 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.224256+0000", 2020-08-07T15:14:05.087 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 141.09855802199999, 2020-08-07T15:14:05.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057864804, 2020-08-07T15:14:05.088 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:05.089 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:05.089 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:05.089 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:14:05.089 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:14:05.089 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118446 2020-08-07T15:14:05.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.090 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:05.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:05.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224256+0000", 2020-08-07T15:14:05.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:05.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:05.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224256+0000", 2020-08-07T15:14:05.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5838000000000003e-05 2020-08-07T15:14:05.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:05.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224312+0000", 2020-08-07T15:14:05.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.879e-06 2020-08-07T15:14:05.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:05.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224318+0000", 2020-08-07T15:14:05.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3593e-05 2020-08-07T15:14:05.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:05.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224331+0000", 2020-08-07T15:14:05.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6169999999999998e-06 2020-08-07T15:14:05.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:05.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224337+0000", 2020-08-07T15:14:05.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7246000000000001e-05 2020-08-07T15:14:05.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:05.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224384+0000", 2020-08-07T15:14:05.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7690000000000008e-06 2020-08-07T15:14:05.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:14:05.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224394+0000", 2020-08-07T15:14:05.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021394589999999998 2020-08-07T15:14:05.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:05.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226533+0000", 2020-08-07T15:14:05.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3849999999999996e-06 2020-08-07T15:14:05.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:05.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226541+0000", 2020-08-07T15:14:05.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0445569109999999 2020-08-07T15:14:05.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:05.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271098+0000", 2020-08-07T15:14:05.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012137 2020-08-07T15:14:05.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:05.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271219+0000", 2020-08-07T15:14:05.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013978300000000001 2020-08-07T15:14:05.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:14:05.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271359+0000", 2020-08-07T15:14:05.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027367699999999999 2020-08-07T15:14:05.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:05.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274096+0000", 2020-08-07T15:14:05.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058973660000000002 2020-08-07T15:14:05.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.279993+0000", 2020-08-07T15:14:05.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116291 2020-08-07T15:14:05.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280109+0000", 2020-08-07T15:14:05.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1099999999999995e-07 2020-08-07T15:14:05.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:05.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280110+0000", 2020-08-07T15:14:05.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.564e-05 2020-08-07T15:14:05.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:05.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280196+0000", 2020-08-07T15:14:05.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00061969 2020-08-07T15:14:05.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280815+0000", 2020-08-07T15:14:05.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001097612 2020-08-07T15:14:05.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281913+0000", 2020-08-07T15:14:05.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4300000000000003e-07 2020-08-07T15:14:05.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:05.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.281913+0000", 2020-08-07T15:14:05.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000207431 2020-08-07T15:14:05.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:05.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282121+0000", 2020-08-07T15:14:05.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.057864804 2020-08-07T15:14:05.113 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:05.114 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:05.114 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:05.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.115 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118447 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:14:05.115 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.224379+0000", 2020-08-07T15:14:05.115 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 141.09843525599999, 2020-08-07T15:14:05.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0579997480000001, 2020-08-07T15:14:05.115 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:05.116 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:05.116 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:05.116 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:14:05.116 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:14:05.116 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118447 2020-08-07T15:14:05.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.117 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:05.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:05.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224379+0000", 2020-08-07T15:14:05.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:05.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:05.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224379+0000", 2020-08-07T15:14:05.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9775999999999999e-05 2020-08-07T15:14:05.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:05.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224429+0000", 2020-08-07T15:14:05.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8500000000000002e-06 2020-08-07T15:14:05.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:05.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224433+0000", 2020-08-07T15:14:05.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7340000000000008e-06 2020-08-07T15:14:05.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:05.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224441+0000", 2020-08-07T15:14:05.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.031e-06 2020-08-07T15:14:05.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:05.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224445+0000", 2020-08-07T15:14:05.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4264000000000003e-05 2020-08-07T15:14:05.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:05.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224479+0000", 2020-08-07T15:14:05.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0039999999999999e-06 2020-08-07T15:14:05.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting_for_map not empty", 2020-08-07T15:14:05.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224487+0000", 2020-08-07T15:14:05.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002241682 2020-08-07T15:14:05.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:05.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226729+0000", 2020-08-07T15:14:05.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0087e-05 2020-08-07T15:14:05.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:05.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226739+0000", 2020-08-07T15:14:05.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.045170006 2020-08-07T15:14:05.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:05.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.271909+0000", 2020-08-07T15:14:05.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012782900000000001 2020-08-07T15:14:05.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:05.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.272037+0000", 2020-08-07T15:14:05.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024079400000000001 2020-08-07T15:14:05.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:14:05.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.272278+0000", 2020-08-07T15:14:05.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023454399999999999 2020-08-07T15:14:05.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:05.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.274623+0000", 2020-08-07T15:14:05.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054268190000000003 2020-08-07T15:14:05.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280050+0000", 2020-08-07T15:14:05.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00048391200000000002 2020-08-07T15:14:05.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280534+0000", 2020-08-07T15:14:05.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5400000000000001e-07 2020-08-07T15:14:05.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:05.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280535+0000", 2020-08-07T15:14:05.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8922e-05 2020-08-07T15:14:05.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:05.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280623+0000", 2020-08-07T15:14:05.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024637999999999998 2020-08-07T15:14:05.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280870+0000", 2020-08-07T15:14:05.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001300142 2020-08-07T15:14:05.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282170+0000", 2020-08-07T15:14:05.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2099999999999997e-07 2020-08-07T15:14:05.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:05.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282170+0000", 2020-08-07T15:14:05.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000208001 2020-08-07T15:14:05.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:05.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282378+0000", 2020-08-07T15:14:05.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0579997480000001 2020-08-07T15:14:05.141 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:05.141 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:05.141 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:05.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.142 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5099.0:118448 5.2 5:46a3771b:::benchmark_data_smithi074_53697_object14805:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e218)", 2020-08-07T15:14:05.143 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:11:43.224474+0000", 2020-08-07T15:14:05.143 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 141.09834026999999, 2020-08-07T15:14:05.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0581683310000001, 2020-08-07T15:14:05.143 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:05.143 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:05.144 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:05.144 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5099", 2020-08-07T15:14:05.144 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/810617293", 2020-08-07T15:14:05.144 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 118448 2020-08-07T15:14:05.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.145 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:05.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:05.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224474+0000", 2020-08-07T15:14:05.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:05.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:05.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224474+0000", 2020-08-07T15:14:05.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3804000000000002e-05 2020-08-07T15:14:05.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:05.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224528+0000", 2020-08-07T15:14:05.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5140000000000001e-06 2020-08-07T15:14:05.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:05.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224533+0000", 2020-08-07T15:14:05.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3315999999999999e-05 2020-08-07T15:14:05.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:05.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224546+0000", 2020-08-07T15:14:05.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7240000000000003e-06 2020-08-07T15:14:05.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:05.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.224553+0000", 2020-08-07T15:14:05.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002226263 2020-08-07T15:14:05.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:05.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226779+0000", 2020-08-07T15:14:05.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1362000000000001e-05 2020-08-07T15:14:05.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:05.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:43.226791+0000", 2020-08-07T15:14:05.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.046092652 2020-08-07T15:14:05.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:05.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.272883+0000", 2020-08-07T15:14:05.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000115981 2020-08-07T15:14:05.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:05.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.272999+0000", 2020-08-07T15:14:05.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029648899999999999 2020-08-07T15:14:05.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,2,4,5", 2020-08-07T15:14:05.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.273296+0000", 2020-08-07T15:14:05.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001852521 2020-08-07T15:14:05.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:05.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.275148+0000", 2020-08-07T15:14:05.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051948309999999996 2020-08-07T15:14:05.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280343+0000", 2020-08-07T15:14:05.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033301799999999998 2020-08-07T15:14:05.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280676+0000", 2020-08-07T15:14:05.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8400000000000004e-07 2020-08-07T15:14:05.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:05.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280677+0000", 2020-08-07T15:14:05.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9652000000000002e-05 2020-08-07T15:14:05.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:05.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280756+0000", 2020-08-07T15:14:05.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017072699999999999 2020-08-07T15:14:05.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.280927+0000", 2020-08-07T15:14:05.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015026530000000001 2020-08-07T15:14:05.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282430+0000", 2020-08-07T15:14:05.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4700000000000001e-07 2020-08-07T15:14:05.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:05.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282430+0000", 2020-08-07T15:14:05.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021169299999999999 2020-08-07T15:14:05.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:05.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:11:44.282642+0000", 2020-08-07T15:14:05.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0581683310000001 2020-08-07T15:14:05.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:05.197 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:05.199 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:05.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.202 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:67303 6.a 6:5c7a55ef:::benchmark_data_smithi074_63843_object8412:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e265)", 2020-08-07T15:14:05.202 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:56.271945+0000", 2020-08-07T15:14:05.202 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.050868530000002, 2020-08-07T15:14:05.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.007577197, 2020-08-07T15:14:05.203 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:05.203 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:05.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:05.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:14:05.204 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:14:05.204 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 67303 2020-08-07T15:14:05.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.204 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:05.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:05.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.271945+0000", 2020-08-07T15:14:05.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:05.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:05.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.271945+0000", 2020-08-07T15:14:05.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9969000000000002e-05 2020-08-07T15:14:05.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:05.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.271995+0000", 2020-08-07T15:14:05.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3690000000000001e-06 2020-08-07T15:14:05.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:05.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.272003+0000", 2020-08-07T15:14:05.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2183000000000001e-05 2020-08-07T15:14:05.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:05.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.272025+0000", 2020-08-07T15:14:05.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7299999999999999e-06 2020-08-07T15:14:05.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:05.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.272032+0000", 2020-08-07T15:14:05.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5831e-05 2020-08-07T15:14:05.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:05.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.272108+0000", 2020-08-07T15:14:05.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2233e-05 2020-08-07T15:14:05.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:05.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.272120+0000", 2020-08-07T15:14:05.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.27969333699999999 2020-08-07T15:14:05.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:05.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.551813+0000", 2020-08-07T15:14:05.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8549999999999997e-06 2020-08-07T15:14:05.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:05.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.551823+0000", 2020-08-07T15:14:05.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.71905319899999998 2020-08-07T15:14:05.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:05.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.270876+0000", 2020-08-07T15:14:05.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116937 2020-08-07T15:14:05.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:05.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.270993+0000", 2020-08-07T15:14:05.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020587399999999999 2020-08-07T15:14:05.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,5,7", 2020-08-07T15:14:05.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.271199+0000", 2020-08-07T15:14:05.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023173429999999999 2020-08-07T15:14:05.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:05.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.273516+0000", 2020-08-07T15:14:05.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053300810000000004 2020-08-07T15:14:05.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.278846+0000", 2020-08-07T15:14:05.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1699999999999998e-07 2020-08-07T15:14:05.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:05.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.278847+0000", 2020-08-07T15:14:05.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7559000000000003e-05 2020-08-07T15:14:05.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:05.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.278894+0000", 2020-08-07T15:14:05.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000119676 2020-08-07T15:14:05.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.279014+0000", 2020-08-07T15:14:05.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039420500000000001 2020-08-07T15:14:05.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.279408+0000", 2020-08-07T15:14:05.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0200000000000003e-07 2020-08-07T15:14:05.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:05.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.279409+0000", 2020-08-07T15:14:05.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113897 2020-08-07T15:14:05.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:05.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.279523+0000", 2020-08-07T15:14:05.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.007577197 2020-08-07T15:14:05.230 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:05.231 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:05.231 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:05.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.231 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:67304 6.a 6:5c7a55ef:::benchmark_data_smithi074_63843_object8412:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e265)", 2020-08-07T15:14:05.232 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:12:56.410012+0000", 2020-08-07T15:14:05.232 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 67.912801779000006, 2020-08-07T15:14:05.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.86966374199999996, 2020-08-07T15:14:05.232 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:05.232 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:05.233 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:05.233 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:14:05.233 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:14:05.233 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 67304 2020-08-07T15:14:05.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.234 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:05.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:05.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.410012+0000", 2020-08-07T15:14:05.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:05.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:05.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.410012+0000", 2020-08-07T15:14:05.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1640000000000002e-05 2020-08-07T15:14:05.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:05.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.410044+0000", 2020-08-07T15:14:05.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8099999999999997e-06 2020-08-07T15:14:05.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:05.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.410049+0000", 2020-08-07T15:14:05.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1989999999999995e-06 2020-08-07T15:14:05.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:05.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.410058+0000", 2020-08-07T15:14:05.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6249999999999998e-06 2020-08-07T15:14:05.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:05.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.410066+0000", 2020-08-07T15:14:05.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2998999999999998e-05 2020-08-07T15:14:05.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:05.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.410129+0000", 2020-08-07T15:14:05.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6420000000000002e-06 2020-08-07T15:14:05.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:05.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.410138+0000", 2020-08-07T15:14:05.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.14171407799999999 2020-08-07T15:14:05.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:05.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.551852+0000", 2020-08-07T15:14:05.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0237e-05 2020-08-07T15:14:05.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:05.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:56.551862+0000", 2020-08-07T15:14:05.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.71982975800000004 2020-08-07T15:14:05.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:05.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.271692+0000", 2020-08-07T15:14:05.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0882000000000002e-05 2020-08-07T15:14:05.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:05.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.271783+0000", 2020-08-07T15:14:05.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000212067 2020-08-07T15:14:05.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0,5,7", 2020-08-07T15:14:05.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.271995+0000", 2020-08-07T15:14:05.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021518420000000002 2020-08-07T15:14:05.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:05.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.274147+0000", 2020-08-07T15:14:05.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0047817959999999996 2020-08-07T15:14:05.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.278929+0000", 2020-08-07T15:14:05.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2099999999999997e-07 2020-08-07T15:14:05.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:05.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.278929+0000", 2020-08-07T15:14:05.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4972000000000002e-05 2020-08-07T15:14:05.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:05.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.278974+0000", 2020-08-07T15:14:05.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5584e-05 2020-08-07T15:14:05.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.279050+0000", 2020-08-07T15:14:05.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051383899999999998 2020-08-07T15:14:05.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.279564+0000", 2020-08-07T15:14:05.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9900000000000001e-07 2020-08-07T15:14:05.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:05.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.279564+0000", 2020-08-07T15:14:05.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111852 2020-08-07T15:14:05.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:05.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:12:57.279676+0000", 2020-08-07T15:14:05.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.86966374199999996 2020-08-07T15:14:05.258 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:05.258 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:05.258 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:05.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.259 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5481.0:128347 6.a 6:56780bc2:::benchmark_data_smithi074_63843_object16043:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e293)", 2020-08-07T15:14:05.259 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:13:38.566993+0000", 2020-08-07T15:14:05.259 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.755820705000001, 2020-08-07T15:14:05.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73194675099999995, 2020-08-07T15:14:05.259 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:05.260 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:05.260 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2020-08-07T15:14:05.260 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5481", 2020-08-07T15:14:05.260 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:14:05.260 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 128347 2020-08-07T15:14:05.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.261 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:05.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:05.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.566993+0000", 2020-08-07T15:14:05.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:05.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:05.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.566993+0000", 2020-08-07T15:14:05.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3297000000000001e-05 2020-08-07T15:14:05.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:05.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.567047+0000", 2020-08-07T15:14:05.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5899999999999996e-06 2020-08-07T15:14:05.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:05.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.567053+0000", 2020-08-07T15:14:05.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3277e-05 2020-08-07T15:14:05.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:05.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.567066+0000", 2020-08-07T15:14:05.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2949999999999999e-06 2020-08-07T15:14:05.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:05.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.567076+0000", 2020-08-07T15:14:05.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1841999999999997e-05 2020-08-07T15:14:05.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:05.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.567118+0000", 2020-08-07T15:14:05.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4059999999999999e-06 2020-08-07T15:14:05.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2020-08-07T15:14:05.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:38.567125+0000", 2020-08-07T15:14:05.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.71691992999999998 2020-08-07T15:14:05.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:05.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:39.284045+0000", 2020-08-07T15:14:05.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109627 2020-08-07T15:14:05.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:05.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:39.284155+0000", 2020-08-07T15:14:05.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000203873 2020-08-07T15:14:05.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5,7", 2020-08-07T15:14:05.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:39.284358+0000", 2020-08-07T15:14:05.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050021429999999997 2020-08-07T15:14:05.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2020-08-07T15:14:05.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:39.289361+0000", 2020-08-07T15:14:05.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3099999999999997e-07 2020-08-07T15:14:05.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:05.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:39.289361+0000", 2020-08-07T15:14:05.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4251e-05 2020-08-07T15:14:05.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2020-08-07T15:14:05.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:39.289405+0000", 2020-08-07T15:14:05.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005939827 2020-08-07T15:14:05.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:39.295345+0000", 2020-08-07T15:14:05.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034308920000000001 2020-08-07T15:14:05.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:05.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:39.298776+0000", 2020-08-07T15:14:05.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5499999999999997e-07 2020-08-07T15:14:05.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "repop_all_committed", 2020-08-07T15:14:05.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:39.298777+0000", 2020-08-07T15:14:05.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000163115 2020-08-07T15:14:05.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:05.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:39.298940+0000", 2020-08-07T15:14:05.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.73194675099999995 2020-08-07T15:14:05.302 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:05.302 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:05.302 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:05.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.303 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.5481.0:149908 6.3 e298/269 ondisk, result = 0)", 2020-08-07T15:14:05.304 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2020-08-07T15:13:51.271126+0000", 2020-08-07T15:14:05.304 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.051688194, 2020-08-07T15:14:05.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.89109337700000002, 2020-08-07T15:14:05.306 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2020-08-07T15:14:05.306 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2020-08-07T15:14:05.307 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2020-08-07T15:14:05.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2020-08-07T15:14:05.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:51.271126+0000", 2020-08-07T15:14:05.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2020-08-07T15:14:05.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2020-08-07T15:14:05.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:51.271126+0000", 2020-08-07T15:14:05.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2252999999999998e-05 2020-08-07T15:14:05.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2020-08-07T15:14:05.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:51.271158+0000", 2020-08-07T15:14:05.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3130000000000001e-06 2020-08-07T15:14:05.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2020-08-07T15:14:05.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:51.271165+0000", 2020-08-07T15:14:05.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.89092263199999999 2020-08-07T15:14:05.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2020-08-07T15:14:05.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:52.162088+0000", 2020-08-07T15:14:05.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0300000000000004e-06 2020-08-07T15:14:05.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2020-08-07T15:14:05.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:52.162092+0000", 2020-08-07T15:14:05.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5454999999999997e-05 2020-08-07T15:14:05.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2020-08-07T15:14:05.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:52.162147+0000", 2020-08-07T15:14:05.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5919999999999997e-06 2020-08-07T15:14:05.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2020-08-07T15:14:05.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:52.162154+0000", 2020-08-07T15:14:05.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5102000000000001e-05 2020-08-07T15:14:05.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2020-08-07T15:14:05.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2020-08-07T15:14:05.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2020-08-07T15:14:05.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2020-08-07T15:13:52.162219+0000", 2020-08-07T15:14:05.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.89109337700000002 2020-08-07T15:14:05.339 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:05.339 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:05.340 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:05.341 INFO:teuthology.orchestra.run.smithi096.stdout: } 2020-08-07T15:14:05.350 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2020-08-07T15:14:05.350 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:14:05.351 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2020-08-07T15:14:05.355 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:14:05.356 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:14:05.365 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:14:05.366 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:14:05.366 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:14:05.366 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:14:05.367 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:14:05.367 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:14:05.368 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:14:05.368 INFO:teuthology.orchestra.run.smithi074.stderr:nodeep-scrub is unset 2020-08-07T15:14:05.369 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.5 to 1 (10000) 2020-08-07T15:14:05.374 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 6 16 8740 8724 11.3585 11.8984 0.00358367 0.0109728 2020-08-07T15:14:05.375 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:14:04.549+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:14:05.376 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:14:05.352+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:14:05.377 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 6 1 2020-08-07T15:14:05.471 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:14:05.466+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:14:05.538 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:14:05.539 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:14:05.552 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2020-08-07T15:14:05.593 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:14:05.591+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:14:05.626 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 7 16 10280 10264 11.4545 12.0312 0.00363318 0.0108931 2020-08-07T15:14:05.673 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:14:05.668+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:14:05.746 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:14:05.747 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:14:05.767 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2020-08-07T15:14:05.776 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:14:05.774+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:14:05.881 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:14:05.879+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:14:05.966 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:14:05.967 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:14:05.986 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2020-08-07T15:14:05.994 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:14:05.988+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:14:06.095 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:14:06.093+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:14:06.183 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:14:06.183 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:14:06.196 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:14:06.193+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:14:06.198 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2020-08-07T15:14:06.298 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:14:06.297+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:14:06.399 INFO:tasks.ceph.osd.4.smithi096.stderr:2020-08-07T15:14:06.398+0000 7fcd854da700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31533) UID: 0 2020-08-07T15:14:06.424 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:14:06.424 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:14:06.438 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2020-08-07T15:14:06.501 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:14:06.500+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:14:06.608 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:14:06.606+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:14:06.616 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.6 to 1 (10000) 2020-08-07T15:14:06.620 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:14:06.621 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:14:06.621 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 8 16 11825 11809 11.5313 12.0703 0.00870945 0.0108091 2020-08-07T15:14:06.635 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2020-08-07T15:14:06.637 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd reweight 7 1 2020-08-07T15:14:06.723 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:14:06.722+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:14:06.808 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:14:06.807+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:14:06.886 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:14:06.886 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:14:06.897 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2020-08-07T15:14:06.936 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:14:06.935+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:14:07.018 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:14:07.009+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:14:07.120 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:14:07.118+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:14:07.131 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:14:07.131 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:14:07.143 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2020-08-07T15:14:07.222 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:14:07.220+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:14:07.324 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:14:07.323+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:14:07.360 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:14:07.360 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:14:07.370 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2020-08-07T15:14:07.372 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2020-08-07T15:14:07.376 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd set noscrub 2020-08-07T15:14:07.425 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:14:07.424+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:14:07.526 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:14:07.525+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:14:07.607 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:14:07.609 INFO:teuthology.orchestra.run.smithi074.stdout:ERROR: (22) Invalid argument 2020-08-07T15:14:07.609 INFO:teuthology.orchestra.run.smithi074.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:14:07.623 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2020-08-07T15:14:07.624 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 9 16 13109 13093 11.3645 10.0312 0.00827701 0.0109332 2020-08-07T15:14:07.631 INFO:teuthology.orchestra.run.smithi074.stderr:reweighted osd.7 to 1 (10000) 2020-08-07T15:14:07.632 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:14:07.629+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:14:07.638 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:14:07.641 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2020-08-07T15:14:07.642 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:14:07.674 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:14:07.715 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:14:07.732 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:14:07.731+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:14:07.747 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:14:07.781 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:14:07.805 INFO:teuthology.orchestra.run.smithi074.stderr:2020-08-07T15:14:07.804+0000 7f8e85e89700 0 --2- 172.21.15.74:0/3289212126 >> [v2:172.21.15.74:3301/0,v1:172.21.15.74:6790/0] conn(0x7f8e8005b360 0x7f8e80066020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2020-08-07T15:14:07.823 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:14:07.832 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:14:07.831+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:14:07.874 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:14:07.875 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'true' 2020-08-07T15:14:07.875 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'true' 2020-08-07T15:14:07.875 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'true' 2020-08-07T15:14:07.876 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'true' 2020-08-07T15:14:07.876 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'true' 2020-08-07T15:14:07.876 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'true' 2020-08-07T15:14:07.876 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'true' 2020-08-07T15:14:07.876 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'true' 2020-08-07T15:14:07.934 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:14:07.931+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:14:08.044 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:14:08.037+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:14:08.082 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:14:08.082 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:14:08.083 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:14:08.083 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:14:08.083 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:14:08.095 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2020-08-07T15:14:08.149 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:14:08.144+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:14:08.243 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:14:08.241+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:14:08.344 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:14:08.343+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:14:08.387 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:14:08.388 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:14:08.389 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:14:08.389 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:14:08.389 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:14:08.389 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:14:08.390 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.390 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.4726.0:30968 4.4 e98/81 ondisk, result = 0)", 2020-08-07T15:14:08.390 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:14:08.390 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 322.64677495400002, 2020-08-07T15:14:08.390 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96211418999999998, 2020-08-07T15:14:08.391 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:08.391 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:14:08.391 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:08.391 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.391 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:08.392 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:14:08.392 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:08.392 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.392 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.392 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:08.393 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729788+0000", 2020-08-07T15:14:08.393 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2387e-05 2020-08-07T15:14:08.393 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.393 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.393 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:08.394 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729821+0000", 2020-08-07T15:14:08.394 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9909999999999999e-06 2020-08-07T15:14:08.394 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.394 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.394 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:08.395 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:45.729827+0000", 2020-08-07T15:14:08.395 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96185524600000005 2020-08-07T15:14:08.395 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.395 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.395 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:08.396 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691682+0000", 2020-08-07T15:14:08.396 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8760000000000005e-06 2020-08-07T15:14:08.396 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.396 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.396 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:08.397 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691690+0000", 2020-08-07T15:14:08.397 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9682000000000001e-05 2020-08-07T15:14:08.397 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.397 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.397 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.398 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691759+0000", 2020-08-07T15:14:08.398 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5381000000000001e-05 2020-08-07T15:14:08.398 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.398 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.398 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:08.398 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691775+0000", 2020-08-07T15:14:08.399 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000127627 2020-08-07T15:14:08.399 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.399 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.399 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:08.399 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:46.691902+0000", 2020-08-07T15:14:08.400 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.96211418999999998 2020-08-07T15:14:08.400 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.400 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:08.400 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.400 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.401 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.401 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42472 4.4 4:2ff82af8:::benchmark_data_smithi074_44586_object5308:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e112)", 2020-08-07T15:14:08.401 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:14:08.402 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 308.10931218000002, 2020-08-07T15:14:08.402 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89653816399999997, 2020-08-07T15:14:08.402 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:08.403 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:08.403 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:08.403 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:08.403 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:08.403 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42472 2020-08-07T15:14:08.404 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.404 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:08.404 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.404 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:08.404 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:14:08.404 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:08.405 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.405 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.405 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:08.405 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267251+0000", 2020-08-07T15:14:08.406 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9591e-05 2020-08-07T15:14:08.406 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.406 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.406 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:08.406 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267301+0000", 2020-08-07T15:14:08.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7490000000000004e-06 2020-08-07T15:14:08.407 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.407 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.407 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:08.407 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267306+0000", 2020-08-07T15:14:08.407 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2850000000000001e-05 2020-08-07T15:14:08.408 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.408 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.408 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:08.408 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267319+0000", 2020-08-07T15:14:08.408 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9100000000000002e-06 2020-08-07T15:14:08.409 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.409 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.409 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:08.409 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267325+0000", 2020-08-07T15:14:08.409 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6584000000000001e-05 2020-08-07T15:14:08.410 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.410 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.410 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.410 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267362+0000", 2020-08-07T15:14:08.410 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9188000000000001e-05 2020-08-07T15:14:08.411 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.411 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.411 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:08.411 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267451+0000", 2020-08-07T15:14:08.411 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000182917 2020-08-07T15:14:08.411 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.412 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.412 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:14:08.412 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.267634+0000", 2020-08-07T15:14:08.412 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002484708 2020-08-07T15:14:08.412 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.413 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.413 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:14:08.413 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270119+0000", 2020-08-07T15:14:08.413 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0299999999999997e-07 2020-08-07T15:14:08.413 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.414 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.414 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:08.414 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270119+0000", 2020-08-07T15:14:08.414 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8251999999999996e-05 2020-08-07T15:14:08.414 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.415 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:14:08.415 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.270188+0000", 2020-08-07T15:14:08.415 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.24443688 2020-08-07T15:14:08.415 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.415 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.416 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.416 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.514625+0000", 2020-08-07T15:14:08.416 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.64911260599999998 2020-08-07T15:14:08.416 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.416 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.417 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.417 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163737+0000", 2020-08-07T15:14:08.417 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2100000000000001e-06 2020-08-07T15:14:08.417 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.417 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.418 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:14:08.418 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163738+0000", 2020-08-07T15:14:08.418 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0816000000000003e-05 2020-08-07T15:14:08.418 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.418 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.418 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:08.419 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.163789+0000", 2020-08-07T15:14:08.419 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89653816399999997 2020-08-07T15:14:08.419 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.419 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:08.419 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.420 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.420 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.420 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72688 4.4 4:295d3f6e:::benchmark_data_smithi074_44586_object9085:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e129)", 2020-08-07T15:14:08.420 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:14:08.420 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 282.97972999299998, 2020-08-07T15:14:08.421 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7869782759999999, 2020-08-07T15:14:08.421 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:08.421 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:14:08.421 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:08.421 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:08.422 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:08.422 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72688 2020-08-07T15:14:08.422 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.422 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:08.422 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.423 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:08.423 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:14:08.423 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:08.423 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.423 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.423 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:08.424 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396833+0000", 2020-08-07T15:14:08.424 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8721999999999999e-05 2020-08-07T15:14:08.424 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.424 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.424 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:08.425 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396882+0000", 2020-08-07T15:14:08.425 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2639999999999999e-06 2020-08-07T15:14:08.425 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.426 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.427 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:08.427 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396887+0000", 2020-08-07T15:14:08.427 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0767999999999999e-05 2020-08-07T15:14:08.427 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.427 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.428 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:08.428 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396898+0000", 2020-08-07T15:14:08.428 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1860000000000002e-06 2020-08-07T15:14:08.428 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.428 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.429 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:08.429 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.396902+0000", 2020-08-07T15:14:08.429 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0037638649999999999 2020-08-07T15:14:08.429 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.429 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.430 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.430 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400666+0000", 2020-08-07T15:14:08.430 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000118077 2020-08-07T15:14:08.430 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.431 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:08.431 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400784+0000", 2020-08-07T15:14:08.431 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000136728 2020-08-07T15:14:08.431 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.431 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.432 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3,6", 2020-08-07T15:14:08.432 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.400921+0000", 2020-08-07T15:14:08.432 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001595443 2020-08-07T15:14:08.432 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.432 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.433 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:14:08.433 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.402516+0000", 2020-08-07T15:14:08.433 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.014817907 2020-08-07T15:14:08.433 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.433 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.434 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.434 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417334+0000", 2020-08-07T15:14:08.434 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.75e-06 2020-08-07T15:14:08.434 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.434 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.435 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:08.435 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417336+0000", 2020-08-07T15:14:08.435 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8066999999999996e-05 2020-08-07T15:14:08.435 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.435 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.436 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:14:08.436 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.417424+0000", 2020-08-07T15:14:08.437 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002210573 2020-08-07T15:14:08.437 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.438 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.438 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.438 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.419635+0000", 2020-08-07T15:14:08.438 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1379000000000001e-05 2020-08-07T15:14:08.439 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.439 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.439 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.439 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.419646+0000", 2020-08-07T15:14:08.439 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.004643409 2020-08-07T15:14:08.440 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.440 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.440 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.440 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424289+0000", 2020-08-07T15:14:08.440 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7563000000000001e-05 2020-08-07T15:14:08.441 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.441 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.441 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.441 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424307+0000", 2020-08-07T15:14:08.441 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75943629400000001 2020-08-07T15:14:08.442 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.442 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.443 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.443 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183743+0000", 2020-08-07T15:14:08.443 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8281000000000006e-05 2020-08-07T15:14:08.443 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.443 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.444 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:08.444 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183812+0000", 2020-08-07T15:14:08.444 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7869782759999999 2020-08-07T15:14:08.444 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.444 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:08.445 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.445 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.445 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.445 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72985 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:14:08.445 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:14:08.446 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 282.76374417400001, 2020-08-07T15:14:08.446 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869466839999999, 2020-08-07T15:14:08.446 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:08.446 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:08.446 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:08.447 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:08.447 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:08.447 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72985 2020-08-07T15:14:08.447 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.447 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:08.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.448 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:08.448 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:14:08.448 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:08.448 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.448 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.449 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:08.449 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612819+0000", 2020-08-07T15:14:08.449 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8003999999999997e-05 2020-08-07T15:14:08.449 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.449 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.450 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:08.450 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612857+0000", 2020-08-07T15:14:08.450 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.246e-06 2020-08-07T15:14:08.450 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.450 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.451 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:08.451 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612863+0000", 2020-08-07T15:14:08.451 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5146e-05 2020-08-07T15:14:08.451 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.451 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.452 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:08.452 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612878+0000", 2020-08-07T15:14:08.452 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1940000000000003e-06 2020-08-07T15:14:08.452 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.452 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.453 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:08.453 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612885+0000", 2020-08-07T15:14:08.453 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0349999999999997e-05 2020-08-07T15:14:08.453 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.453 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.454 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.454 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612935+0000", 2020-08-07T15:14:08.454 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1029999999999996e-06 2020-08-07T15:14:08.454 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.454 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.454 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.455 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612942+0000", 2020-08-07T15:14:08.455 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81140097700000002 2020-08-07T15:14:08.455 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.455 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.455 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.456 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424343+0000", 2020-08-07T15:14:08.456 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.1799999999999996e-06 2020-08-07T15:14:08.456 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.456 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.456 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.457 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424351+0000", 2020-08-07T15:14:08.457 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75948665299999996 2020-08-07T15:14:08.457 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.457 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.457 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.458 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.183838+0000", 2020-08-07T15:14:08.458 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000179187 2020-08-07T15:14:08.458 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.459 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.459 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:08.459 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.184017+0000", 2020-08-07T15:14:08.459 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024691600000000002 2020-08-07T15:14:08.459 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.460 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.460 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:14:08.460 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.184264+0000", 2020-08-07T15:14:08.460 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006362721 2020-08-07T15:14:08.460 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.461 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.461 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:14:08.461 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190627+0000", 2020-08-07T15:14:08.461 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7400000000000001e-06 2020-08-07T15:14:08.461 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.462 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.462 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:08.462 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190629+0000", 2020-08-07T15:14:08.462 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1305999999999999e-05 2020-08-07T15:14:08.462 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.463 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.463 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:14:08.463 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190690+0000", 2020-08-07T15:14:08.463 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0088507159999999998 2020-08-07T15:14:08.463 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.463 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.464 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.464 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199541+0000", 2020-08-07T15:14:08.464 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00015864699999999999 2020-08-07T15:14:08.464 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.464 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.465 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.465 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199699+0000", 2020-08-07T15:14:08.465 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.23e-07 2020-08-07T15:14:08.465 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.465 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.466 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:14:08.466 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199700+0000", 2020-08-07T15:14:08.466 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5775e-05 2020-08-07T15:14:08.466 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.466 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.467 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:08.467 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199766+0000", 2020-08-07T15:14:08.467 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869466839999999 2020-08-07T15:14:08.467 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.467 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:08.468 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.468 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.468 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.468 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72986 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:14:08.468 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:14:08.468 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 282.76363359200002, 2020-08-07T15:14:08.469 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.586946441, 2020-08-07T15:14:08.469 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:08.469 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:08.469 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:08.470 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:08.470 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:08.470 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72986 2020-08-07T15:14:08.470 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.471 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:08.471 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.471 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:08.471 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:14:08.471 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:08.472 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.472 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.472 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:08.472 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612930+0000", 2020-08-07T15:14:08.472 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9861999999999997e-05 2020-08-07T15:14:08.472 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.473 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.473 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:08.473 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612990+0000", 2020-08-07T15:14:08.473 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4350000000000001e-06 2020-08-07T15:14:08.473 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.474 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.474 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:08.474 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.612993+0000", 2020-08-07T15:14:08.474 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9529999999999999e-06 2020-08-07T15:14:08.475 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.475 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.475 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:08.475 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613001+0000", 2020-08-07T15:14:08.475 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4309999999999998e-06 2020-08-07T15:14:08.476 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.476 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.476 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:08.476 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613004+0000", 2020-08-07T15:14:08.476 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5284000000000001e-05 2020-08-07T15:14:08.477 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.483 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.484 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.484 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613030+0000", 2020-08-07T15:14:08.484 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.2400000000000001e-06 2020-08-07T15:14:08.484 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.484 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.485 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.485 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613034+0000", 2020-08-07T15:14:08.485 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81134136199999995 2020-08-07T15:14:08.485 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.486 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.487 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.487 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424375+0000", 2020-08-07T15:14:08.487 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3520000000000001e-06 2020-08-07T15:14:08.487 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.487 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.488 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.488 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424383+0000", 2020-08-07T15:14:08.488 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76064437799999995 2020-08-07T15:14:08.488 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.488 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.489 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.489 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185027+0000", 2020-08-07T15:14:08.489 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000119719 2020-08-07T15:14:08.489 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.489 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.490 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:08.490 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185147+0000", 2020-08-07T15:14:08.490 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016829400000000001 2020-08-07T15:14:08.490 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.490 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.491 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:14:08.491 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185315+0000", 2020-08-07T15:14:08.491 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0053821199999999998 2020-08-07T15:14:08.491 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.491 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.492 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:14:08.492 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190697+0000", 2020-08-07T15:14:08.492 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6899999999999996e-07 2020-08-07T15:14:08.492 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.492 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.493 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:08.493 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190698+0000", 2020-08-07T15:14:08.493 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.2072000000000001e-05 2020-08-07T15:14:08.493 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.493 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.494 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:14:08.494 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190730+0000", 2020-08-07T15:14:08.494 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0088501440000000008 2020-08-07T15:14:08.494 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.494 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.495 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.495 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199580+0000", 2020-08-07T15:14:08.495 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00023199900000000001 2020-08-07T15:14:08.495 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.495 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.495 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.496 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199812+0000", 2020-08-07T15:14:08.496 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6300000000000005e-07 2020-08-07T15:14:08.496 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.496 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.496 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:14:08.497 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199813+0000", 2020-08-07T15:14:08.497 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3363999999999998e-05 2020-08-07T15:14:08.497 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.497 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.497 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:08.498 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199876+0000", 2020-08-07T15:14:08.498 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.586946441 2020-08-07T15:14:08.498 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.498 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:08.498 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.499 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.499 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.499 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72987 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:14:08.499 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:14:08.499 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 282.763537837, 2020-08-07T15:14:08.500 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869598709999999, 2020-08-07T15:14:08.500 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:08.500 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:08.500 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:08.500 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:08.501 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:08.501 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72987 2020-08-07T15:14:08.501 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.501 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:08.501 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.502 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:08.502 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:14:08.502 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:08.502 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.502 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.503 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:08.503 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613025+0000", 2020-08-07T15:14:08.503 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1851999999999999e-05 2020-08-07T15:14:08.503 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.503 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.503 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:08.504 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613057+0000", 2020-08-07T15:14:08.504 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0429999999999999e-06 2020-08-07T15:14:08.504 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.504 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.504 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:08.505 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613060+0000", 2020-08-07T15:14:08.505 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1060000000000002e-06 2020-08-07T15:14:08.505 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.505 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.505 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:08.506 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613067+0000", 2020-08-07T15:14:08.506 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8040000000000002e-06 2020-08-07T15:14:08.506 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.506 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.506 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:08.507 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613070+0000", 2020-08-07T15:14:08.507 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6857000000000001e-05 2020-08-07T15:14:08.507 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.507 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.507 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.507 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613097+0000", 2020-08-07T15:14:08.508 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8770000000000001e-06 2020-08-07T15:14:08.508 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.508 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.508 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.508 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613102+0000", 2020-08-07T15:14:08.509 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81131066399999996 2020-08-07T15:14:08.509 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.509 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.509 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.509 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424413+0000", 2020-08-07T15:14:08.510 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4599999999999997e-06 2020-08-07T15:14:08.510 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.510 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.519 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.519 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424420+0000", 2020-08-07T15:14:08.520 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76132809700000004 2020-08-07T15:14:08.520 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.520 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.520 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.520 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185748+0000", 2020-08-07T15:14:08.521 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.4264000000000005e-05 2020-08-07T15:14:08.521 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.521 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.521 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:08.522 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185832+0000", 2020-08-07T15:14:08.522 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112803 2020-08-07T15:14:08.522 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.522 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.522 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:14:08.523 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.185945+0000", 2020-08-07T15:14:08.523 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0089954130000000007 2020-08-07T15:14:08.523 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.523 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.523 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:14:08.523 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194941+0000", 2020-08-07T15:14:08.524 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.091e-06 2020-08-07T15:14:08.524 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.524 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.524 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:08.524 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194943+0000", 2020-08-07T15:14:08.525 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7423999999999996e-05 2020-08-07T15:14:08.525 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.525 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.525 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:14:08.525 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195020+0000", 2020-08-07T15:14:08.526 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0045973300000000002 2020-08-07T15:14:08.526 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.526 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.526 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.526 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199618+0000", 2020-08-07T15:14:08.527 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000302009 2020-08-07T15:14:08.527 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.527 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.527 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.527 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199920+0000", 2020-08-07T15:14:08.528 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7999999999999996e-07 2020-08-07T15:14:08.528 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.528 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.528 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:14:08.528 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199920+0000", 2020-08-07T15:14:08.529 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5296999999999995e-05 2020-08-07T15:14:08.529 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.529 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.529 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:08.529 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199985+0000", 2020-08-07T15:14:08.530 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5869598709999999 2020-08-07T15:14:08.530 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.530 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:08.530 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.530 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.531 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.531 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72988 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:14:08.531 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:14:08.531 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 282.76347325699999, 2020-08-07T15:14:08.531 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587111143, 2020-08-07T15:14:08.532 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:08.532 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:08.532 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:08.532 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:08.532 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:08.532 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72988 2020-08-07T15:14:08.533 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.533 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:08.533 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.533 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:08.533 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:14:08.534 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:08.534 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.534 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.534 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:08.534 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613090+0000", 2020-08-07T15:14:08.535 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6563e-05 2020-08-07T15:14:08.535 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.535 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.535 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:08.535 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613117+0000", 2020-08-07T15:14:08.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7180000000000001e-06 2020-08-07T15:14:08.536 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.536 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.536 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:08.536 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613119+0000", 2020-08-07T15:14:08.536 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1789999999999999e-06 2020-08-07T15:14:08.537 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.537 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.537 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:08.537 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613124+0000", 2020-08-07T15:14:08.537 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3539999999999998e-06 2020-08-07T15:14:08.538 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.538 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.538 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:08.538 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613127+0000", 2020-08-07T15:14:08.538 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3697e-05 2020-08-07T15:14:08.539 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.539 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.539 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.539 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613151+0000", 2020-08-07T15:14:08.539 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7559999999999999e-06 2020-08-07T15:14:08.540 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.540 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.540 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.540 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613155+0000", 2020-08-07T15:14:08.540 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81128859600000003 2020-08-07T15:14:08.541 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.541 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.541 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.541 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424444+0000", 2020-08-07T15:14:08.541 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9619999999999996e-06 2020-08-07T15:14:08.541 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.542 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.542 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.542 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424451+0000", 2020-08-07T15:14:08.542 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76286712899999998 2020-08-07T15:14:08.542 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.543 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.543 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.543 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187318+0000", 2020-08-07T15:14:08.543 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011616300000000001 2020-08-07T15:14:08.543 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.544 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.544 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:08.544 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187434+0000", 2020-08-07T15:14:08.544 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019347400000000001 2020-08-07T15:14:08.544 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.545 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.545 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:14:08.545 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.187628+0000", 2020-08-07T15:14:08.545 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0074049090000000003 2020-08-07T15:14:08.545 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.546 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.546 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:14:08.546 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195033+0000", 2020-08-07T15:14:08.546 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1090000000000001e-06 2020-08-07T15:14:08.546 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.547 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.547 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:08.547 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195034+0000", 2020-08-07T15:14:08.547 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8566999999999997e-05 2020-08-07T15:14:08.547 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.548 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:14:08.548 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195092+0000", 2020-08-07T15:14:08.548 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004570457 2020-08-07T15:14:08.548 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.548 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.549 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.549 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.199663+0000", 2020-08-07T15:14:08.549 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00046757899999999998 2020-08-07T15:14:08.549 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.549 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.550 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.550 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200130+0000", 2020-08-07T15:14:08.550 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6700000000000002e-07 2020-08-07T15:14:08.550 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.550 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.551 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:14:08.551 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200131+0000", 2020-08-07T15:14:08.551 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9963999999999996e-05 2020-08-07T15:14:08.551 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.551 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.552 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:08.552 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200201+0000", 2020-08-07T15:14:08.552 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587111143 2020-08-07T15:14:08.552 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.552 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:08.553 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.553 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.553 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.553 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72989 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:14:08.553 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:14:08.554 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 282.76341686900003, 2020-08-07T15:14:08.554 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5872506399999999, 2020-08-07T15:14:08.554 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:08.554 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:08.554 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:08.555 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:08.555 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:08.555 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72989 2020-08-07T15:14:08.555 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.555 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:08.555 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.556 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:08.556 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:14:08.556 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:08.556 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.556 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.557 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:08.557 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613146+0000", 2020-08-07T15:14:08.557 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6081999999999999e-05 2020-08-07T15:14:08.557 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.557 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.558 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:08.558 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613172+0000", 2020-08-07T15:14:08.558 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6570000000000001e-06 2020-08-07T15:14:08.558 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.558 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.559 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:08.559 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613175+0000", 2020-08-07T15:14:08.559 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7740000000000001e-06 2020-08-07T15:14:08.560 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.560 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.560 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:08.560 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613181+0000", 2020-08-07T15:14:08.560 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2759999999999999e-06 2020-08-07T15:14:08.560 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.561 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.561 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:08.561 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613183+0000", 2020-08-07T15:14:08.561 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3892000000000001e-05 2020-08-07T15:14:08.561 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.562 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.562 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.562 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613207+0000", 2020-08-07T15:14:08.562 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9870000000000001e-06 2020-08-07T15:14:08.562 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.563 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.563 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.563 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613211+0000", 2020-08-07T15:14:08.563 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81126230700000002 2020-08-07T15:14:08.563 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.564 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.564 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.564 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424473+0000", 2020-08-07T15:14:08.564 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1460000000000001e-06 2020-08-07T15:14:08.564 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.565 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.565 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.565 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424480+0000", 2020-08-07T15:14:08.565 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76413073099999995 2020-08-07T15:14:08.565 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.565 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.566 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.566 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188610+0000", 2020-08-07T15:14:08.566 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011278899999999999 2020-08-07T15:14:08.566 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.566 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.567 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:08.567 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188723+0000", 2020-08-07T15:14:08.567 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000213422 2020-08-07T15:14:08.567 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.567 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.568 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:14:08.568 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.188936+0000", 2020-08-07T15:14:08.568 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0061672929999999999 2020-08-07T15:14:08.568 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.568 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.569 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:14:08.569 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195104+0000", 2020-08-07T15:14:08.569 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8199999999999999e-07 2020-08-07T15:14:08.569 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.569 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.569 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:08.570 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195105+0000", 2020-08-07T15:14:08.570 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2056000000000004e-05 2020-08-07T15:14:08.570 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.570 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.570 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:14:08.571 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195167+0000", 2020-08-07T15:14:08.571 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0051009769999999996 2020-08-07T15:14:08.571 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.571 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.571 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.572 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200268+0000", 2020-08-07T15:14:08.572 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9428000000000001e-05 2020-08-07T15:14:08.572 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.572 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.572 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.573 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200327+0000", 2020-08-07T15:14:08.573 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9100000000000003e-07 2020-08-07T15:14:08.573 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.573 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.573 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:14:08.573 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200328+0000", 2020-08-07T15:14:08.574 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9250000000000003e-05 2020-08-07T15:14:08.574 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.574 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.574 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:08.574 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200397+0000", 2020-08-07T15:14:08.575 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5872506399999999 2020-08-07T15:14:08.575 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.575 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:08.575 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.575 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.576 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.576 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72990 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:14:08.576 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:14:08.576 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 282.76335957700002, 2020-08-07T15:14:08.576 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587335105, 2020-08-07T15:14:08.577 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:08.577 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:08.577 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:08.577 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:08.577 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:08.578 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72990 2020-08-07T15:14:08.578 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.578 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:08.578 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.578 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:08.579 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:14:08.579 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:08.579 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.579 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.579 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:08.579 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613204+0000", 2020-08-07T15:14:08.580 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5276999999999999e-05 2020-08-07T15:14:08.580 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.580 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.580 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:08.580 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613229+0000", 2020-08-07T15:14:08.581 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7219999999999999e-06 2020-08-07T15:14:08.581 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.581 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.581 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:08.581 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613232+0000", 2020-08-07T15:14:08.582 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1170000000000003e-06 2020-08-07T15:14:08.582 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.582 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.582 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:08.582 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613237+0000", 2020-08-07T15:14:08.583 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2309999999999999e-06 2020-08-07T15:14:08.583 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.583 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.583 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:08.583 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613239+0000", 2020-08-07T15:14:08.584 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2469000000000001e-05 2020-08-07T15:14:08.584 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.584 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.584 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.584 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613261+0000", 2020-08-07T15:14:08.584 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9280000000000001e-06 2020-08-07T15:14:08.585 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.585 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.585 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.585 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613266+0000", 2020-08-07T15:14:08.585 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81123414599999999 2020-08-07T15:14:08.586 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.586 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.586 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.586 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424501+0000", 2020-08-07T15:14:08.586 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5300000000000002e-06 2020-08-07T15:14:08.587 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.587 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.587 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.588 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424507+0000", 2020-08-07T15:14:08.588 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76488660200000003 2020-08-07T15:14:08.588 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.588 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.588 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.588 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189394+0000", 2020-08-07T15:14:08.589 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6878000000000003e-05 2020-08-07T15:14:08.589 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.589 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.589 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:08.590 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189491+0000", 2020-08-07T15:14:08.590 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000204477 2020-08-07T15:14:08.590 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.590 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.590 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:14:08.590 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.189695+0000", 2020-08-07T15:14:08.591 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0065180940000000003 2020-08-07T15:14:08.591 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.591 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.591 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:14:08.592 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196213+0000", 2020-08-07T15:14:08.592 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.4920000000000001e-06 2020-08-07T15:14:08.592 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.592 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.592 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:08.593 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196215+0000", 2020-08-07T15:14:08.593 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9950000000000006e-05 2020-08-07T15:14:08.593 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.593 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.593 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:14:08.594 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196285+0000", 2020-08-07T15:14:08.594 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004163674 2020-08-07T15:14:08.594 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.594 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.594 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.595 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200448+0000", 2020-08-07T15:14:08.595 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6919999999999999e-05 2020-08-07T15:14:08.595 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.595 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.595 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.596 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200485+0000", 2020-08-07T15:14:08.596 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1699999999999998e-07 2020-08-07T15:14:08.596 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.596 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.596 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:14:08.597 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200486+0000", 2020-08-07T15:14:08.597 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3081000000000002e-05 2020-08-07T15:14:08.597 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.597 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.597 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:08.598 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200539+0000", 2020-08-07T15:14:08.598 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.587335105 2020-08-07T15:14:08.598 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.598 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:08.598 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.599 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.599 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.599 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72991 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:14:08.599 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:14:08.599 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 282.76330405200002, 2020-08-07T15:14:08.600 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5876210340000001, 2020-08-07T15:14:08.600 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:08.600 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:08.600 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:08.600 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:08.600 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:08.601 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72991 2020-08-07T15:14:08.601 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.601 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:08.601 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.601 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:08.602 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:14:08.602 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:08.602 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.602 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.602 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:08.603 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613259+0000", 2020-08-07T15:14:08.603 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5763999999999999e-05 2020-08-07T15:14:08.603 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.603 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.603 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:08.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613285+0000", 2020-08-07T15:14:08.604 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6479999999999999e-06 2020-08-07T15:14:08.604 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.604 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.604 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:08.604 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613288+0000", 2020-08-07T15:14:08.605 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1909999999999999e-06 2020-08-07T15:14:08.605 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.605 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.605 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:08.605 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613293+0000", 2020-08-07T15:14:08.606 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2560000000000001e-06 2020-08-07T15:14:08.606 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.606 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.606 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:08.606 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613295+0000", 2020-08-07T15:14:08.607 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.2561e-05 2020-08-07T15:14:08.607 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.607 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.607 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.607 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613318+0000", 2020-08-07T15:14:08.608 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6349999999999999e-06 2020-08-07T15:14:08.608 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.608 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.608 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.608 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.613321+0000", 2020-08-07T15:14:08.608 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81120592400000002 2020-08-07T15:14:08.609 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.609 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.609 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.609 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424527+0000", 2020-08-07T15:14:08.609 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8540000000000002e-06 2020-08-07T15:14:08.610 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.610 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.610 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.610 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424533+0000", 2020-08-07T15:14:08.610 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76623392800000001 2020-08-07T15:14:08.611 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.611 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.611 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.611 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190767+0000", 2020-08-07T15:14:08.611 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3990000000000001e-05 2020-08-07T15:14:08.612 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.612 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.612 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:08.612 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190841+0000", 2020-08-07T15:14:08.612 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011444599999999999 2020-08-07T15:14:08.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.613 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.613 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:14:08.613 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.190955+0000", 2020-08-07T15:14:08.613 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0053397499999999999 2020-08-07T15:14:08.613 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.614 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.614 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:14:08.614 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196295+0000", 2020-08-07T15:14:08.614 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006e-06 2020-08-07T15:14:08.614 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.615 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.615 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:08.615 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196296+0000", 2020-08-07T15:14:08.615 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8940999999999997e-05 2020-08-07T15:14:08.615 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.616 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.616 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:14:08.616 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196355+0000", 2020-08-07T15:14:08.616 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042276700000000002 2020-08-07T15:14:08.616 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.616 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.617 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.617 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200583+0000", 2020-08-07T15:14:08.617 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00026035700000000001 2020-08-07T15:14:08.617 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.618 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.618 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.618 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200843+0000", 2020-08-07T15:14:08.618 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6700000000000003e-07 2020-08-07T15:14:08.618 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.619 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.619 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:14:08.619 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200844+0000", 2020-08-07T15:14:08.619 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6545999999999999e-05 2020-08-07T15:14:08.619 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.620 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.620 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:08.620 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200880+0000", 2020-08-07T15:14:08.620 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5876210340000001 2020-08-07T15:14:08.621 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.621 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:08.621 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.621 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.621 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.622 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:72992 4.4 4:2521623a:::benchmark_data_smithi074_44586_object9123:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:14:08.622 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:14:08.622 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 282.76221657299999, 2020-08-07T15:14:08.622 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5866241210000001, 2020-08-07T15:14:08.622 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:08.623 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:08.623 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:08.623 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:08.623 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:08.623 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 72992 2020-08-07T15:14:08.624 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.624 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:08.624 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.624 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:08.624 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:14:08.625 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:08.625 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.625 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.625 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:08.625 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614347+0000", 2020-08-07T15:14:08.626 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7543e-05 2020-08-07T15:14:08.626 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.626 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.626 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:08.626 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614374+0000", 2020-08-07T15:14:08.627 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9740000000000004e-06 2020-08-07T15:14:08.627 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.627 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.627 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:08.627 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614378+0000", 2020-08-07T15:14:08.628 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2660000000000007e-06 2020-08-07T15:14:08.628 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.628 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.628 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:08.628 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614387+0000", 2020-08-07T15:14:08.629 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2950000000000001e-06 2020-08-07T15:14:08.629 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.629 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.629 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:08.629 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614393+0000", 2020-08-07T15:14:08.630 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7279000000000001e-05 2020-08-07T15:14:08.630 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.630 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.630 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.630 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614420+0000", 2020-08-07T15:14:08.631 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9040000000000002e-06 2020-08-07T15:14:08.631 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.631 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.631 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.631 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.614424+0000", 2020-08-07T15:14:08.632 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.81012815699999996 2020-08-07T15:14:08.632 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.632 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.632 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.632 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424552+0000", 2020-08-07T15:14:08.633 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2879999999999998e-06 2020-08-07T15:14:08.633 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.633 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.633 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.633 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424558+0000", 2020-08-07T15:14:08.634 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76678665400000001 2020-08-07T15:14:08.634 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.634 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.634 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.634 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191345+0000", 2020-08-07T15:14:08.635 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010819600000000001 2020-08-07T15:14:08.635 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.635 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.635 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:08.635 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191453+0000", 2020-08-07T15:14:08.636 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000170829 2020-08-07T15:14:08.636 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.636 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.636 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:14:08.636 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.191624+0000", 2020-08-07T15:14:08.637 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0047386679999999997 2020-08-07T15:14:08.637 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.637 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.637 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:14:08.637 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196363+0000", 2020-08-07T15:14:08.638 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6000000000000003e-07 2020-08-07T15:14:08.638 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.638 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.638 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:08.638 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196363+0000", 2020-08-07T15:14:08.638 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3421000000000002e-05 2020-08-07T15:14:08.639 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.639 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.639 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:14:08.639 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196417+0000", 2020-08-07T15:14:08.639 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042225539999999999 2020-08-07T15:14:08.640 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.640 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.640 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.640 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200639+0000", 2020-08-07T15:14:08.640 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00028031399999999999 2020-08-07T15:14:08.641 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.641 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.641 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.641 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200920+0000", 2020-08-07T15:14:08.641 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0299999999999999e-07 2020-08-07T15:14:08.642 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.642 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.642 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:14:08.642 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200920+0000", 2020-08-07T15:14:08.642 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0516000000000002e-05 2020-08-07T15:14:08.642 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.643 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.643 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:08.643 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200971+0000", 2020-08-07T15:14:08.643 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5866241210000001 2020-08-07T15:14:08.644 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.644 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:08.644 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.644 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.644 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.645 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73017 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:14:08.645 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:14:08.645 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 282.70670832000002, 2020-08-07T15:14:08.645 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.531181608, 2020-08-07T15:14:08.645 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:08.646 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:08.646 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:08.646 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:08.646 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:08.646 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73017 2020-08-07T15:14:08.646 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.647 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:08.647 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.647 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:08.647 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:14:08.647 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:08.648 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.648 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.648 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:08.648 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669855+0000", 2020-08-07T15:14:08.648 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3018e-05 2020-08-07T15:14:08.649 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.649 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.649 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:08.649 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669898+0000", 2020-08-07T15:14:08.649 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5340000000000002e-06 2020-08-07T15:14:08.650 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.650 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.650 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:08.650 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669905+0000", 2020-08-07T15:14:08.650 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3307e-05 2020-08-07T15:14:08.650 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.651 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.651 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:08.651 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669919+0000", 2020-08-07T15:14:08.651 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7729999999999999e-06 2020-08-07T15:14:08.651 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.652 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.652 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:08.652 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669927+0000", 2020-08-07T15:14:08.652 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0361000000000001e-05 2020-08-07T15:14:08.652 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.653 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.653 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.653 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669977+0000", 2020-08-07T15:14:08.653 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0049999999999993e-06 2020-08-07T15:14:08.653 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.654 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.654 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.654 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.669986+0000", 2020-08-07T15:14:08.654 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75459235199999997 2020-08-07T15:14:08.654 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.655 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.655 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.655 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424578+0000", 2020-08-07T15:14:08.655 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.925e-06 2020-08-07T15:14:08.655 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.655 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.656 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.656 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424584+0000", 2020-08-07T15:14:08.656 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76741703900000002 2020-08-07T15:14:08.656 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.656 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.657 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.657 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192001+0000", 2020-08-07T15:14:08.657 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000181196 2020-08-07T15:14:08.657 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.657 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.658 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:08.658 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192182+0000", 2020-08-07T15:14:08.658 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00021149299999999999 2020-08-07T15:14:08.658 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.658 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.659 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:14:08.659 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192394+0000", 2020-08-07T15:14:08.659 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004031242 2020-08-07T15:14:08.659 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.659 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.660 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:14:08.660 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196425+0000", 2020-08-07T15:14:08.660 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7800000000000001e-07 2020-08-07T15:14:08.660 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.661 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.661 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:08.661 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196426+0000", 2020-08-07T15:14:08.661 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5244999999999997e-05 2020-08-07T15:14:08.661 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.662 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.662 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:14:08.662 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196481+0000", 2020-08-07T15:14:08.662 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0041989360000000003 2020-08-07T15:14:08.662 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.663 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.663 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.663 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200680+0000", 2020-08-07T15:14:08.663 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00032415299999999998 2020-08-07T15:14:08.663 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.664 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.664 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201004+0000", 2020-08-07T15:14:08.664 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9299999999999999e-07 2020-08-07T15:14:08.664 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.664 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.665 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:14:08.665 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201005+0000", 2020-08-07T15:14:08.665 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1857999999999999e-05 2020-08-07T15:14:08.665 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.665 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.666 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:08.666 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201037+0000", 2020-08-07T15:14:08.666 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.531181608 2020-08-07T15:14:08.666 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.666 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:08.667 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.667 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.667 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.667 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73018 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:14:08.667 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:14:08.668 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 282.705740515, 2020-08-07T15:14:08.668 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.530301616, 2020-08-07T15:14:08.668 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:08.668 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:08.668 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:08.669 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:08.669 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:08.669 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73018 2020-08-07T15:14:08.669 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.669 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:08.670 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.670 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:08.670 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:14:08.670 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:08.670 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.671 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.671 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:08.671 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670823+0000", 2020-08-07T15:14:08.671 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.9747000000000002e-05 2020-08-07T15:14:08.671 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.672 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.672 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:08.672 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670852+0000", 2020-08-07T15:14:08.672 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.9430000000000002e-06 2020-08-07T15:14:08.672 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.672 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.673 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:08.673 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670856+0000", 2020-08-07T15:14:08.673 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.0649999999999999e-06 2020-08-07T15:14:08.673 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.673 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.674 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:08.674 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670864+0000", 2020-08-07T15:14:08.674 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.924e-06 2020-08-07T15:14:08.674 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.674 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.675 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:08.675 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670868+0000", 2020-08-07T15:14:08.675 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6205000000000003e-05 2020-08-07T15:14:08.675 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.675 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.675 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.676 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670905+0000", 2020-08-07T15:14:08.676 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3289999999999996e-06 2020-08-07T15:14:08.676 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.676 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.676 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.677 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.670912+0000", 2020-08-07T15:14:08.677 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75370200200000004 2020-08-07T15:14:08.677 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.677 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.678 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424614+0000", 2020-08-07T15:14:08.678 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5150000000000003e-06 2020-08-07T15:14:08.678 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.678 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.678 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424620+0000", 2020-08-07T15:14:08.679 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.768200153 2020-08-07T15:14:08.679 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.679 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192821+0000", 2020-08-07T15:14:08.680 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000111764 2020-08-07T15:14:08.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.680 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:08.680 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.192932+0000", 2020-08-07T15:14:08.681 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017542400000000001 2020-08-07T15:14:08.681 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.681 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:14:08.681 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193108+0000", 2020-08-07T15:14:08.682 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.004100851 2020-08-07T15:14:08.682 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.682 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.682 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:14:08.682 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197209+0000", 2020-08-07T15:14:08.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0100000000000001e-06 2020-08-07T15:14:08.683 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.683 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.683 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:08.683 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197210+0000", 2020-08-07T15:14:08.684 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9672999999999995e-05 2020-08-07T15:14:08.684 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.684 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.684 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:14:08.684 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197279+0000", 2020-08-07T15:14:08.684 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034275109999999998 2020-08-07T15:14:08.685 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.685 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.685 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.685 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200707+0000", 2020-08-07T15:14:08.685 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00036853100000000001 2020-08-07T15:14:08.686 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.686 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.686 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.686 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201075+0000", 2020-08-07T15:14:08.686 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8599999999999998e-07 2020-08-07T15:14:08.687 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.687 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.687 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:14:08.687 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201076+0000", 2020-08-07T15:14:08.687 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8483000000000003e-05 2020-08-07T15:14:08.688 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.688 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.688 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:08.688 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201124+0000", 2020-08-07T15:14:08.689 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.530301616 2020-08-07T15:14:08.689 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.689 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:08.689 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.689 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.690 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.690 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73019 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:14:08.690 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:14:08.690 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 282.70509286399999, 2020-08-07T15:14:08.690 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.529718224, 2020-08-07T15:14:08.691 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:08.691 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:08.691 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:08.691 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:08.691 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:08.692 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73019 2020-08-07T15:14:08.692 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.692 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:08.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.692 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:08.692 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:14:08.693 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:08.693 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.693 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.693 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:08.693 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671470+0000", 2020-08-07T15:14:08.694 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.3309000000000002e-05 2020-08-07T15:14:08.694 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.694 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.694 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:08.694 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671504+0000", 2020-08-07T15:14:08.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6619999999999998e-06 2020-08-07T15:14:08.695 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.695 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.695 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:08.695 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671507+0000", 2020-08-07T15:14:08.696 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2279999999999998e-06 2020-08-07T15:14:08.696 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.696 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.696 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:08.696 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671516+0000", 2020-08-07T15:14:08.697 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4860000000000002e-06 2020-08-07T15:14:08.697 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.697 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:08.697 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671519+0000", 2020-08-07T15:14:08.698 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0564000000000001e-05 2020-08-07T15:14:08.698 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.698 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.698 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.698 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671550+0000", 2020-08-07T15:14:08.698 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1029999999999998e-06 2020-08-07T15:14:08.699 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.699 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.699 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.699 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.671556+0000", 2020-08-07T15:14:08.699 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.75308759199999997 2020-08-07T15:14:08.700 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.700 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.700 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424643+0000", 2020-08-07T15:14:08.700 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9179999999999997e-06 2020-08-07T15:14:08.701 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.701 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.701 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.701 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424650+0000", 2020-08-07T15:14:08.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76890373599999995 2020-08-07T15:14:08.702 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.702 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.702 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.702 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193554+0000", 2020-08-07T15:14:08.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00012345600000000001 2020-08-07T15:14:08.703 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.703 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:08.703 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193677+0000", 2020-08-07T15:14:08.703 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013528300000000001 2020-08-07T15:14:08.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.704 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:14:08.704 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.193813+0000", 2020-08-07T15:14:08.704 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0034777710000000002 2020-08-07T15:14:08.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.705 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.705 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:14:08.705 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197291+0000", 2020-08-07T15:14:08.705 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.8000000000000004e-07 2020-08-07T15:14:08.705 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.706 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.706 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:08.706 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197291+0000", 2020-08-07T15:14:08.706 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9176999999999999e-05 2020-08-07T15:14:08.706 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.707 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:14:08.707 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197351+0000", 2020-08-07T15:14:08.707 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033931009999999999 2020-08-07T15:14:08.707 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.708 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.708 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200744+0000", 2020-08-07T15:14:08.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00041308000000000002 2020-08-07T15:14:08.708 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.708 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.709 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.709 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201157+0000", 2020-08-07T15:14:08.709 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.8999999999999998e-07 2020-08-07T15:14:08.709 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.709 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.710 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:14:08.710 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201157+0000", 2020-08-07T15:14:08.710 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1588000000000003e-05 2020-08-07T15:14:08.710 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.710 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.711 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:08.711 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201189+0000", 2020-08-07T15:14:08.711 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.529718224 2020-08-07T15:14:08.711 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.711 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:08.712 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.712 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.712 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.712 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73020 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:14:08.712 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:14:08.713 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 282.63675500400001, 2020-08-07T15:14:08.713 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.461468373, 2020-08-07T15:14:08.713 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:08.713 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:08.713 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:08.714 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:08.714 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:08.714 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73020 2020-08-07T15:14:08.714 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.724 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:08.724 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.724 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:08.725 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:14:08.725 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:08.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.725 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:08.726 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739808+0000", 2020-08-07T15:14:08.727 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.777e-05 2020-08-07T15:14:08.727 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.728 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:08.728 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739836+0000", 2020-08-07T15:14:08.728 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.0180000000000003e-06 2020-08-07T15:14:08.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.729 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.729 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:08.729 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739840+0000", 2020-08-07T15:14:08.729 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.6930000000000002e-06 2020-08-07T15:14:08.729 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.730 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.730 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:08.730 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739850+0000", 2020-08-07T15:14:08.730 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3619999999999999e-06 2020-08-07T15:14:08.730 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.731 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:08.731 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739854+0000", 2020-08-07T15:14:08.731 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7432999999999999e-05 2020-08-07T15:14:08.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.732 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.732 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739892+0000", 2020-08-07T15:14:08.732 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1149999999999996e-06 2020-08-07T15:14:08.732 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.733 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.733 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.739899+0000", 2020-08-07T15:14:08.733 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.68477883299999998 2020-08-07T15:14:08.733 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.734 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.734 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.734 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424677+0000", 2020-08-07T15:14:08.734 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7710000000000001e-06 2020-08-07T15:14:08.734 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.734 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.735 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.735 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424684+0000", 2020-08-07T15:14:08.735 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.76949970700000003 2020-08-07T15:14:08.735 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.735 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.736 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.736 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194184+0000", 2020-08-07T15:14:08.736 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3957999999999997e-05 2020-08-07T15:14:08.736 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.736 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.737 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:08.737 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194278+0000", 2020-08-07T15:14:08.737 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017489599999999999 2020-08-07T15:14:08.737 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.737 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.738 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:14:08.738 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.194453+0000", 2020-08-07T15:14:08.738 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.002908908 2020-08-07T15:14:08.738 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.739 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.739 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:14:08.739 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197362+0000", 2020-08-07T15:14:08.739 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7299999999999995e-07 2020-08-07T15:14:08.739 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.740 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.740 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:08.740 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197362+0000", 2020-08-07T15:14:08.740 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6848000000000002e-05 2020-08-07T15:14:08.740 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.741 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.741 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:14:08.741 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197419+0000", 2020-08-07T15:14:08.741 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033597079999999999 2020-08-07T15:14:08.741 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.742 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.742 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.742 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200779+0000", 2020-08-07T15:14:08.742 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00044797899999999999 2020-08-07T15:14:08.742 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.743 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.743 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201227+0000", 2020-08-07T15:14:08.743 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7100000000000002e-07 2020-08-07T15:14:08.743 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.743 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:14:08.744 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201227+0000", 2020-08-07T15:14:08.744 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9230000000000001e-05 2020-08-07T15:14:08.744 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.745 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:08.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201277+0000", 2020-08-07T15:14:08.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.461468373 2020-08-07T15:14:08.745 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.746 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:08.746 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.746 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.746 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73021 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 32768~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:14:08.746 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:14:08.747 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 282.57372139699999, 2020-08-07T15:14:08.747 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.398505079, 2020-08-07T15:14:08.747 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:08.747 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:08.747 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:08.748 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:08.748 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:08.748 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73021 2020-08-07T15:14:08.748 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.748 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:08.749 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.749 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:08.749 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:14:08.749 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:08.749 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.750 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.750 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:08.750 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802842+0000", 2020-08-07T15:14:08.750 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7429000000000001e-05 2020-08-07T15:14:08.750 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.751 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.751 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:08.751 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802869+0000", 2020-08-07T15:14:08.751 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7840000000000001e-06 2020-08-07T15:14:08.751 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.752 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:08.752 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802873+0000", 2020-08-07T15:14:08.752 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0119999999999994e-06 2020-08-07T15:14:08.752 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.753 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.753 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:08.753 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802882+0000", 2020-08-07T15:14:08.753 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.04e-06 2020-08-07T15:14:08.753 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.753 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.754 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:08.754 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802887+0000", 2020-08-07T15:14:08.754 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7076000000000002e-05 2020-08-07T15:14:08.754 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.754 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.755 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.755 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802924+0000", 2020-08-07T15:14:08.755 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.9280000000000004e-06 2020-08-07T15:14:08.755 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.755 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.756 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.756 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.802931+0000", 2020-08-07T15:14:08.756 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.62178051000000001 2020-08-07T15:14:08.756 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.756 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.757 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.757 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424712+0000", 2020-08-07T15:14:08.757 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4749999999999996e-06 2020-08-07T15:14:08.757 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.757 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.758 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.758 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424719+0000", 2020-08-07T15:14:08.758 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77047786799999995 2020-08-07T15:14:08.758 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.758 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.758 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.759 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195197+0000", 2020-08-07T15:14:08.759 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000101909 2020-08-07T15:14:08.759 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.759 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.759 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:08.760 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195299+0000", 2020-08-07T15:14:08.760 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000248247 2020-08-07T15:14:08.760 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.761 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:14:08.761 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.195547+0000", 2020-08-07T15:14:08.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0033458699999999999 2020-08-07T15:14:08.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.762 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.762 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:14:08.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198893+0000", 2020-08-07T15:14:08.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0419999999999999e-06 2020-08-07T15:14:08.763 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.763 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:08.763 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198894+0000", 2020-08-07T15:14:08.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9354999999999997e-05 2020-08-07T15:14:08.764 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.764 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:14:08.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.198953+0000", 2020-08-07T15:14:08.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001861096 2020-08-07T15:14:08.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.765 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.765 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200815+0000", 2020-08-07T15:14:08.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00049697299999999995 2020-08-07T15:14:08.766 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.766 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.766 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201311+0000", 2020-08-07T15:14:08.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.7099999999999998e-07 2020-08-07T15:14:08.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:14:08.767 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201312+0000", 2020-08-07T15:14:08.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.5194000000000002e-05 2020-08-07T15:14:08.768 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.768 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.768 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:08.768 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201347+0000", 2020-08-07T15:14:08.768 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.398505079 2020-08-07T15:14:08.769 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.769 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:08.769 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.769 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.769 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.770 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73022 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 40960~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:14:08.770 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:14:08.770 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 282.49254934200002, 2020-08-07T15:14:08.770 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3238320669999999, 2020-08-07T15:14:08.770 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:08.771 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:08.771 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:08.771 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:08.771 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:08.771 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73022 2020-08-07T15:14:08.772 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.772 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:08.772 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.772 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:08.772 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:14:08.773 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:08.773 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.773 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.773 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:08.774 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884014+0000", 2020-08-07T15:14:08.774 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6769999999999999e-05 2020-08-07T15:14:08.774 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.774 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.774 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:08.775 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884041+0000", 2020-08-07T15:14:08.775 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.703e-06 2020-08-07T15:14:08.775 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.775 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:08.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884044+0000", 2020-08-07T15:14:08.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5949999999999999e-06 2020-08-07T15:14:08.776 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:08.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884053+0000", 2020-08-07T15:14:08.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0100000000000003e-06 2020-08-07T15:14:08.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:08.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884058+0000", 2020-08-07T15:14:08.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7281999999999999e-05 2020-08-07T15:14:08.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884105+0000", 2020-08-07T15:14:08.779 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.044e-05 2020-08-07T15:14:08.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.779 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.779 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.884116+0000", 2020-08-07T15:14:08.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.54062884300000003 2020-08-07T15:14:08.780 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.780 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.780 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.780 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424745+0000", 2020-08-07T15:14:08.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3149999999999998e-06 2020-08-07T15:14:08.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.781 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.781 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424754+0000", 2020-08-07T15:14:08.781 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77175732200000002 2020-08-07T15:14:08.782 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.782 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.782 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.782 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196511+0000", 2020-08-07T15:14:08.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00010110799999999999 2020-08-07T15:14:08.783 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:08.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196612+0000", 2020-08-07T15:14:08.783 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00017543100000000001 2020-08-07T15:14:08.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:14:08.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.196788+0000", 2020-08-07T15:14:08.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0032134569999999999 2020-08-07T15:14:08.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:14:08.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200001+0000", 2020-08-07T15:14:08.785 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.071e-06 2020-08-07T15:14:08.785 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.786 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.786 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:08.786 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200002+0000", 2020-08-07T15:14:08.786 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2527999999999995e-05 2020-08-07T15:14:08.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.787 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.787 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:14:08.787 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.200065+0000", 2020-08-07T15:14:08.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001327433 2020-08-07T15:14:08.787 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.787 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.788 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.788 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.201392+0000", 2020-08-07T15:14:08.788 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063854030000000004 2020-08-07T15:14:08.788 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.788 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.789 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.789 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207778+0000", 2020-08-07T15:14:08.789 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.285e-06 2020-08-07T15:14:08.789 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.789 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.790 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:14:08.790 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207779+0000", 2020-08-07T15:14:08.790 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7070999999999995e-05 2020-08-07T15:14:08.790 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.790 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.791 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:08.791 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.207846+0000", 2020-08-07T15:14:08.791 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3238320669999999 2020-08-07T15:14:08.791 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.791 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:08.792 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.792 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.792 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.792 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:73023 4.4 4:273a9a06:::benchmark_data_smithi074_44586_object9127:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192] snapc 0=[] ondisk+write+known_if_redirected e130)", 2020-08-07T15:14:08.792 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:14:08.792 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 282.37938770099998, 2020-08-07T15:14:08.793 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2113701729999999, 2020-08-07T15:14:08.793 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:08.793 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:08.793 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:08.793 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:08.794 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:08.794 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 73023 2020-08-07T15:14:08.794 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.794 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:08.794 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.795 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:08.795 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:14:08.795 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:08.795 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.795 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.796 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:08.796 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997176+0000", 2020-08-07T15:14:08.796 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4105e-05 2020-08-07T15:14:08.796 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.796 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:08.797 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997210+0000", 2020-08-07T15:14:08.797 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1089999999999997e-06 2020-08-07T15:14:08.797 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.797 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.797 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:08.798 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997215+0000", 2020-08-07T15:14:08.798 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.7040000000000006e-06 2020-08-07T15:14:08.798 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.798 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.798 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:08.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997224+0000", 2020-08-07T15:14:08.799 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4439999999999999e-06 2020-08-07T15:14:08.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.799 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:08.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997230+0000", 2020-08-07T15:14:08.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5977999999999999e-05 2020-08-07T15:14:08.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.800 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997276+0000", 2020-08-07T15:14:08.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.2660000000000007e-06 2020-08-07T15:14:08.801 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.801 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.801 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:25.997285+0000", 2020-08-07T15:14:08.802 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.42749189599999998 2020-08-07T15:14:08.802 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.802 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.802 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.802 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424777+0000", 2020-08-07T15:14:08.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8940000000000004e-06 2020-08-07T15:14:08.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:26.424784+0000", 2020-08-07T15:14:08.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.77268199699999995 2020-08-07T15:14:08.804 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.804 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197466+0000", 2020-08-07T15:14:08.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011277699999999999 2020-08-07T15:14:08.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.805 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:08.805 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197579+0000", 2020-08-07T15:14:08.805 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00018970599999999999 2020-08-07T15:14:08.806 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.806 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.806 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 1,3", 2020-08-07T15:14:08.806 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.197768+0000", 2020-08-07T15:14:08.806 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043168529999999998 2020-08-07T15:14:08.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.807 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.807 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:14:08.807 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202085+0000", 2020-08-07T15:14:08.807 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.186e-06 2020-08-07T15:14:08.807 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.808 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.808 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:08.808 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202086+0000", 2020-08-07T15:14:08.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.8975999999999999e-05 2020-08-07T15:14:08.808 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.809 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.809 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:14:08.809 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202145+0000", 2020-08-07T15:14:08.809 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00035331499999999999 2020-08-07T15:14:08.809 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.810 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.810 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.810 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.202499+0000", 2020-08-07T15:14:08.810 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0059703159999999998 2020-08-07T15:14:08.810 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.811 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.811 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208469+0000", 2020-08-07T15:14:08.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7400000000000002e-07 2020-08-07T15:14:08.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:14:08.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208470+0000", 2020-08-07T15:14:08.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5877000000000006e-05 2020-08-07T15:14:08.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.813 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:08.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:27.208546+0000", 2020-08-07T15:14:08.813 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2113701729999999 2020-08-07T15:14:08.813 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.813 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:08.814 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.814 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.814 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.814 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5481.0:74192 6.e e269/230 6:7ab1a263:::benchmark_data_smithi074_63843_object9273:head v 269'4368, mlcod=268'4362)", 2020-08-07T15:14:08.814 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:01.594532+0000", 2020-08-07T15:14:08.815 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 66.782031004000004, 2020-08-07T15:14:08.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.95244114199999996, 2020-08-07T15:14:08.815 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:08.815 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:08.815 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:08.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:08.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.594532+0000", 2020-08-07T15:14:08.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:08.816 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.817 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:08.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.594532+0000", 2020-08-07T15:14:08.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1347999999999997e-05 2020-08-07T15:14:08.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.818 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:08.818 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.594584+0000", 2020-08-07T15:14:08.818 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8689999999999998e-06 2020-08-07T15:14:08.818 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.818 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.818 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:08.819 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.594590+0000", 2020-08-07T15:14:08.819 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.95108761900000005 2020-08-07T15:14:08.819 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.819 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.819 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:08.820 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.545678+0000", 2020-08-07T15:14:08.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.271e-06 2020-08-07T15:14:08.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:08.821 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.545683+0000", 2020-08-07T15:14:08.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000113051 2020-08-07T15:14:08.821 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.822 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.545796+0000", 2020-08-07T15:14:08.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6154000000000001e-05 2020-08-07T15:14:08.822 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.822 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.822 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:08.822 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.545823+0000", 2020-08-07T15:14:08.823 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0010668959999999999 2020-08-07T15:14:08.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.823 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:14:08.823 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.546889+0000", 2020-08-07T15:14:08.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3934e-05 2020-08-07T15:14:08.824 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.824 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.824 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:08.824 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.546973+0000", 2020-08-07T15:14:08.825 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.95244114199999996 2020-08-07T15:14:08.825 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.825 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:08.825 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.826 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5481.0:74203 6.9 6:9b24f442:::benchmark_data_smithi074_63843_object9275:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e269)", 2020-08-07T15:14:08.826 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:01.691147+0000", 2020-08-07T15:14:08.826 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 66.685415899999995, 2020-08-07T15:14:08.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.91286480199999998, 2020-08-07T15:14:08.827 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:08.827 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:08.827 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:08.828 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5481", 2020-08-07T15:14:08.828 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2463807600", 2020-08-07T15:14:08.828 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 74203 2020-08-07T15:14:08.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.828 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:08.829 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.829 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:08.829 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.691147+0000", 2020-08-07T15:14:08.829 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:08.829 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.830 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.830 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:08.830 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.691147+0000", 2020-08-07T15:14:08.830 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.0080999999999999e-05 2020-08-07T15:14:08.830 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.831 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.831 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:08.831 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.691177+0000", 2020-08-07T15:14:08.831 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7670000000000003e-06 2020-08-07T15:14:08.831 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.831 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.832 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:08.832 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.691182+0000", 2020-08-07T15:14:08.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0392e-05 2020-08-07T15:14:08.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:08.833 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.691193+0000", 2020-08-07T15:14:08.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5570000000000003e-06 2020-08-07T15:14:08.833 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:08.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.691198+0000", 2020-08-07T15:14:08.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0670000000000001e-05 2020-08-07T15:14:08.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.691249+0000", 2020-08-07T15:14:08.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0593e-05 2020-08-07T15:14:08.835 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.835 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:08.836 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:01.691259+0000", 2020-08-07T15:14:08.836 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.90310043900000003 2020-08-07T15:14:08.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.836 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:08.837 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.594360+0000", 2020-08-07T15:14:08.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000107182 2020-08-07T15:14:08.837 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.837 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.837 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:08.838 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.594467+0000", 2020-08-07T15:14:08.838 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00022166199999999999 2020-08-07T15:14:08.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.838 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.838 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:14:08.838 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.594689+0000", 2020-08-07T15:14:08.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0042962399999999998 2020-08-07T15:14:08.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.839 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:14:08.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.598985+0000", 2020-08-07T15:14:08.840 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0759999999999999e-06 2020-08-07T15:14:08.840 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.840 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.840 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:08.840 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.598986+0000", 2020-08-07T15:14:08.841 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0080000000000002e-05 2020-08-07T15:14:08.841 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:14:08.841 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.599036+0000", 2020-08-07T15:14:08.841 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00053325400000000004 2020-08-07T15:14:08.842 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.842 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.842 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.842 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.599569+0000", 2020-08-07T15:14:08.842 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043431800000000003 2020-08-07T15:14:08.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.843 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:08.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.603913+0000", 2020-08-07T15:14:08.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5600000000000005e-07 2020-08-07T15:14:08.844 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.844 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:14:08.844 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.603913+0000", 2020-08-07T15:14:08.844 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.8973000000000003e-05 2020-08-07T15:14:08.845 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:08.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:08.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:08.845 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:02.604012+0000", 2020-08-07T15:14:08.845 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.91286480199999998 2020-08-07T15:14:08.846 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.846 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:08.846 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.846 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:08.846 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:08.847 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:14:08.849 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":547,"stamp":"2020-08-07T15:14:07.624780+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":85278720,"num_objects":1304,"num_object_clones":0,"num_object_copies":3912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10410,"num_write_kb":83280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":195,"acting":195,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":196,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8653392,"kb_used_data":264720,"kb_used_omap":2,"kb_used_meta":8388605,"kb_avail":746321328,"statfs":{"total":773094113280,"available":764233039872,"internally_reserved":8589934592,"allocated":271073280,"data_stored":266228092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":8589932013},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":59850752,"num_objects":914,"num_object_clones":0,"num_object_copies":2742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7306,"num_write_kb":58448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.922897"},"pg_stats":[{"pgid":"1.28","version":"16'2","reported_seq":"74","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.608159+0000","last_change":"2020-08-07T15:12:37.495634+0000","last_active":"2020-08-07T15:14:04.608159+0000","last_peered":"2020-08-07T15:14:04.608159+0000","last_clean":"2020-08-07T15:14:04.608159+0000","last_became_active":"2020-08-07T15:12:37.495085+0000","last_became_peered":"2020-08-07T15:12:37.495085+0000","last_unstale":"2020-08-07T15:14:04.608159+0000","last_undegraded":"2020-08-07T15:14:04.608159+0000","last_fullsized":"2020-08-07T15:14:04.608159+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.490655+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.490655+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.27","version":"16'2","reported_seq":"419","reported_epoch":"306","state":"active+clean","last_fresh":"2020-08-07T15:14:01.563729+0000","last_change":"2020-08-07T15:12:57.250017+0000","last_active":"2020-08-07T15:14:01.563729+0000","last_peered":"2020-08-07T15:14:01.563729+0000","last_clean":"2020-08-07T15:14:01.563729+0000","last_became_active":"2020-08-07T15:12:37.411869+0000","last_became_peered":"2020-08-07T15:12:37.411869+0000","last_unstale":"2020-08-07T15:14:01.563729+0000","last_undegraded":"2020-08-07T15:14:01.563729+0000","last_fullsized":"2020-08-07T15:14:01.563729+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:57.249927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:57.249927+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.26","version":"16'2","reported_seq":"83","reported_epoch":"308","state":"active+clean","last_fresh":"2020-08-07T15:14:03.621305+0000","last_change":"2020-08-07T15:12:40.765120+0000","last_active":"2020-08-07T15:14:03.621305+0000","last_peered":"2020-08-07T15:14:03.621305+0000","last_clean":"2020-08-07T15:14:03.621305+0000","last_became_active":"2020-08-07T15:12:37.457426+0000","last_became_peered":"2020-08-07T15:12:37.457426+0000","last_unstale":"2020-08-07T15:14:03.621305+0000","last_undegraded":"2020-08-07T15:14:03.621305+0000","last_fullsized":"2020-08-07T15:14:03.621305+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:40.765027+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_clean_scrub_stamp":"2020-08-07T15:12:40.765027+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.25","version":"75'3","reported_seq":"310","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.618965+0000","last_change":"2020-08-07T15:13:07.465488+0000","last_active":"2020-08-07T15:14:04.618965+0000","last_peered":"2020-08-07T15:14:04.618965+0000","last_clean":"2020-08-07T15:14:04.618965+0000","last_became_active":"2020-08-07T15:12:37.427816+0000","last_became_peered":"2020-08-07T15:12:37.427816+0000","last_unstale":"2020-08-07T15:14:04.618965+0000","last_undegraded":"2020-08-07T15:14:04.618965+0000","last_fullsized":"2020-08-07T15:14:04.618965+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:13:07.465410+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:13:07.465410+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.24","version":"16'2","reported_seq":"248","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.614428+0000","last_change":"2020-08-07T15:12:42.295755+0000","last_active":"2020-08-07T15:14:04.614428+0000","last_peered":"2020-08-07T15:14:04.614428+0000","last_clean":"2020-08-07T15:14:04.614428+0000","last_became_active":"2020-08-07T15:12:34.414144+0000","last_became_peered":"2020-08-07T15:12:34.414144+0000","last_unstale":"2020-08-07T15:14:04.614428+0000","last_undegraded":"2020-08-07T15:14:04.614428+0000","last_fullsized":"2020-08-07T15:14:04.614428+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:42.295672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:42.295672+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.23","version":"16'2","reported_seq":"312","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.684082+0000","last_change":"2020-08-07T15:14:04.684082+0000","last_active":"2020-08-07T15:14:04.684082+0000","last_peered":"2020-08-07T15:14:04.684082+0000","last_clean":"2020-08-07T15:14:04.684082+0000","last_became_active":"2020-08-07T15:13:38.274095+0000","last_became_peered":"2020-08-07T15:13:38.274095+0000","last_unstale":"2020-08-07T15:14:04.684082+0000","last_undegraded":"2020-08-07T15:14:04.684082+0000","last_fullsized":"2020-08-07T15:14:04.684082+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":293,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:14:04.684015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:14:04.684015+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.22","version":"16'2","reported_seq":"74","reported_epoch":"306","state":"active+clean","last_fresh":"2020-08-07T15:14:01.563783+0000","last_change":"2020-08-07T15:12:58.227750+0000","last_active":"2020-08-07T15:14:01.563783+0000","last_peered":"2020-08-07T15:14:01.563783+0000","last_clean":"2020-08-07T15:14:01.563783+0000","last_became_active":"2020-08-07T15:12:37.413046+0000","last_became_peered":"2020-08-07T15:12:37.413046+0000","last_unstale":"2020-08-07T15:14:01.563783+0000","last_undegraded":"2020-08-07T15:14:01.563783+0000","last_fullsized":"2020-08-07T15:14:01.563783+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:58.227699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:58.227699+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.21","version":"16'2","reported_seq":"84","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.613815+0000","last_change":"2020-08-07T15:12:58.387474+0000","last_active":"2020-08-07T15:14:04.613815+0000","last_peered":"2020-08-07T15:14:04.613815+0000","last_clean":"2020-08-07T15:14:04.613815+0000","last_became_active":"2020-08-07T15:12:37.418612+0000","last_became_peered":"2020-08-07T15:12:37.418612+0000","last_unstale":"2020-08-07T15:14:04.613815+0000","last_undegraded":"2020-08-07T15:14:04.613815+0000","last_fullsized":"2020-08-07T15:14:04.613815+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:58.387424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:58.387424+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.20","version":"16'2","reported_seq":"84","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.618717+0000","last_change":"2020-08-07T15:12:54.433326+0000","last_active":"2020-08-07T15:14:04.618717+0000","last_peered":"2020-08-07T15:14:04.618717+0000","last_clean":"2020-08-07T15:14:04.618717+0000","last_became_active":"2020-08-07T15:12:37.409249+0000","last_became_peered":"2020-08-07T15:12:37.409249+0000","last_unstale":"2020-08-07T15:14:04.618717+0000","last_undegraded":"2020-08-07T15:14:04.618717+0000","last_fullsized":"2020-08-07T15:14:04.618717+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:54.433245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:54.433245+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1f","version":"16'2","reported_seq":"77","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.618583+0000","last_change":"2020-08-07T15:13:09.471546+0000","last_active":"2020-08-07T15:14:04.618583+0000","last_peered":"2020-08-07T15:14:04.618583+0000","last_clean":"2020-08-07T15:14:04.618583+0000","last_became_active":"2020-08-07T15:12:37.410716+0000","last_became_peered":"2020-08-07T15:12:37.410716+0000","last_unstale":"2020-08-07T15:14:04.618583+0000","last_undegraded":"2020-08-07T15:14:04.618583+0000","last_fullsized":"2020-08-07T15:14:04.618583+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:09.471471+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_clean_scrub_stamp":"2020-08-07T15:13:09.471471+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1e","version":"16'2","reported_seq":"164","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.614277+0000","last_change":"2020-08-07T15:13:09.396723+0000","last_active":"2020-08-07T15:14:04.614277+0000","last_peered":"2020-08-07T15:14:04.614277+0000","last_clean":"2020-08-07T15:14:04.614277+0000","last_became_active":"2020-08-07T15:10:51.559894+0000","last_became_peered":"2020-08-07T15:10:51.559894+0000","last_unstale":"2020-08-07T15:14:04.614277+0000","last_undegraded":"2020-08-07T15:14:04.614277+0000","last_fullsized":"2020-08-07T15:14:04.614277+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:09.396633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:09.396633+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1d","version":"75'3","reported_seq":"190","reported_epoch":"308","state":"active+clean","last_fresh":"2020-08-07T15:14:03.621726+0000","last_change":"2020-08-07T15:13:24.343240+0000","last_active":"2020-08-07T15:14:03.621726+0000","last_peered":"2020-08-07T15:14:03.621726+0000","last_clean":"2020-08-07T15:14:03.621726+0000","last_became_active":"2020-08-07T15:13:24.342549+0000","last_became_peered":"2020-08-07T15:13:24.342549+0000","last_unstale":"2020-08-07T15:14:03.621726+0000","last_undegraded":"2020-08-07T15:14:03.621726+0000","last_fullsized":"2020-08-07T15:14:03.621726+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:28.855432+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:28.855432+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1c","version":"16'2","reported_seq":"162","reported_epoch":"306","state":"active+clean","last_fresh":"2020-08-07T15:14:01.563644+0000","last_change":"2020-08-07T15:13:07.471270+0000","last_active":"2020-08-07T15:14:01.563644+0000","last_peered":"2020-08-07T15:14:01.563644+0000","last_clean":"2020-08-07T15:14:01.563644+0000","last_became_active":"2020-08-07T15:13:02.591669+0000","last_became_peered":"2020-08-07T15:13:02.591669+0000","last_unstale":"2020-08-07T15:14:01.563644+0000","last_undegraded":"2020-08-07T15:14:01.563644+0000","last_fullsized":"2020-08-07T15:14:01.563644+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:07.471192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:07.471192+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1b","version":"16'2","reported_seq":"285","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.608252+0000","last_change":"2020-08-07T15:13:22.022126+0000","last_active":"2020-08-07T15:14:04.608252+0000","last_peered":"2020-08-07T15:14:04.608252+0000","last_clean":"2020-08-07T15:14:04.608252+0000","last_became_active":"2020-08-07T15:10:51.557392+0000","last_became_peered":"2020-08-07T15:10:51.557392+0000","last_unstale":"2020-08-07T15:14:04.608252+0000","last_undegraded":"2020-08-07T15:14:04.608252+0000","last_fullsized":"2020-08-07T15:14:04.608252+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:22.022047+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:57.796946+0000","last_clean_scrub_stamp":"2020-08-07T15:13:22.022047+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1a","version":"16'2","reported_seq":"55","reported_epoch":"308","state":"active+clean","last_fresh":"2020-08-07T15:14:03.621641+0000","last_change":"2020-08-07T15:13:06.881943+0000","last_active":"2020-08-07T15:14:03.621641+0000","last_peered":"2020-08-07T15:14:03.621641+0000","last_clean":"2020-08-07T15:14:03.621641+0000","last_became_active":"2020-08-07T15:13:02.590409+0000","last_became_peered":"2020-08-07T15:13:02.590409+0000","last_unstale":"2020-08-07T15:14:03.621641+0000","last_undegraded":"2020-08-07T15:14:03.621641+0000","last_fullsized":"2020-08-07T15:14:03.621641+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:06.881863+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:13:06.881863+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.19","version":"16'2","reported_seq":"207","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.613929+0000","last_change":"2020-08-07T15:12:15.372999+0000","last_active":"2020-08-07T15:14:04.613929+0000","last_peered":"2020-08-07T15:14:04.613929+0000","last_clean":"2020-08-07T15:14:04.613929+0000","last_became_active":"2020-08-07T15:10:51.559752+0000","last_became_peered":"2020-08-07T15:10:51.559752+0000","last_unstale":"2020-08-07T15:14:04.613929+0000","last_undegraded":"2020-08-07T15:14:04.613929+0000","last_fullsized":"2020-08-07T15:14:04.613929+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:15.372910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:15.372910+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.18","version":"16'2","reported_seq":"310","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.610434+0000","last_change":"2020-08-07T15:13:54.173407+0000","last_active":"2020-08-07T15:14:04.610434+0000","last_peered":"2020-08-07T15:14:04.610434+0000","last_clean":"2020-08-07T15:14:04.610434+0000","last_became_active":"2020-08-07T15:10:51.557665+0000","last_became_peered":"2020-08-07T15:10:51.557665+0000","last_unstale":"2020-08-07T15:14:04.610434+0000","last_undegraded":"2020-08-07T15:14:04.610434+0000","last_fullsized":"2020-08-07T15:14:04.610434+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:54.173329+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:13:54.173329+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.17","version":"16'2","reported_seq":"138","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.608628+0000","last_change":"2020-08-07T15:12:57.172008+0000","last_active":"2020-08-07T15:14:04.608628+0000","last_peered":"2020-08-07T15:14:04.608628+0000","last_clean":"2020-08-07T15:14:04.608628+0000","last_became_active":"2020-08-07T15:10:51.561769+0000","last_became_peered":"2020-08-07T15:10:51.561769+0000","last_unstale":"2020-08-07T15:14:04.608628+0000","last_undegraded":"2020-08-07T15:14:04.608628+0000","last_fullsized":"2020-08-07T15:14:04.608628+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:57.171893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:57.171893+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.16","version":"16'2","reported_seq":"83","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.608571+0000","last_change":"2020-08-07T15:12:52.576339+0000","last_active":"2020-08-07T15:14:04.608571+0000","last_peered":"2020-08-07T15:14:04.608571+0000","last_clean":"2020-08-07T15:14:04.608571+0000","last_became_active":"2020-08-07T15:12:52.575577+0000","last_became_peered":"2020-08-07T15:12:52.575577+0000","last_unstale":"2020-08-07T15:14:04.608571+0000","last_undegraded":"2020-08-07T15:14:04.608571+0000","last_fullsized":"2020-08-07T15:14:04.608571+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:12.883558+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:12.883558+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.883558+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.15","version":"75'3","reported_seq":"66","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.608494+0000","last_change":"2020-08-07T15:13:02.590465+0000","last_active":"2020-08-07T15:14:04.608494+0000","last_peered":"2020-08-07T15:14:04.608494+0000","last_clean":"2020-08-07T15:14:04.608494+0000","last_became_active":"2020-08-07T15:13:02.589879+0000","last_became_peered":"2020-08-07T15:13:02.589879+0000","last_unstale":"2020-08-07T15:14:04.608494+0000","last_undegraded":"2020-08-07T15:14:04.608494+0000","last_fullsized":"2020-08-07T15:14:04.608494+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:53.824617+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:53.824617+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.14","version":"16'2","reported_seq":"248","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.618475+0000","last_change":"2020-08-07T15:13:26.549332+0000","last_active":"2020-08-07T15:14:04.618475+0000","last_peered":"2020-08-07T15:14:04.618475+0000","last_clean":"2020-08-07T15:14:04.618475+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:14:04.618475+0000","last_undegraded":"2020-08-07T15:14:04.618475+0000","last_fullsized":"2020-08-07T15:14:04.618475+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:26.549247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:26.549247+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"259","reported_epoch":"308","state":"active+clean","last_fresh":"2020-08-07T15:14:03.621358+0000","last_change":"2020-08-07T15:12:19.577926+0000","last_active":"2020-08-07T15:14:03.621358+0000","last_peered":"2020-08-07T15:14:03.621358+0000","last_clean":"2020-08-07T15:14:03.621358+0000","last_became_active":"2020-08-07T15:12:06.375861+0000","last_became_peered":"2020-08-07T15:12:06.375861+0000","last_unstale":"2020-08-07T15:14:03.621358+0000","last_undegraded":"2020-08-07T15:14:03.621358+0000","last_fullsized":"2020-08-07T15:14:03.621358+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:07.887356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.887356+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"260","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.608172+0000","last_change":"2020-08-07T15:12:26.104598+0000","last_active":"2020-08-07T15:14:04.608172+0000","last_peered":"2020-08-07T15:14:04.608172+0000","last_clean":"2020-08-07T15:14:04.608172+0000","last_became_active":"2020-08-07T15:12:06.367362+0000","last_became_peered":"2020-08-07T15:12:06.367362+0000","last_unstale":"2020-08-07T15:14:04.608172+0000","last_undegraded":"2020-08-07T15:14:04.608172+0000","last_fullsized":"2020-08-07T15:14:04.608172+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:26.104526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:26.104526+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"358","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.608089+0000","last_change":"2020-08-07T15:12:24.105593+0000","last_active":"2020-08-07T15:14:04.608089+0000","last_peered":"2020-08-07T15:14:04.608089+0000","last_clean":"2020-08-07T15:14:04.608089+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:14:04.608089+0000","last_undegraded":"2020-08-07T15:14:04.608089+0000","last_fullsized":"2020-08-07T15:14:04.608089+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:24.105531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:24.105531+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"352","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.610549+0000","last_change":"2020-08-07T15:12:13.231819+0000","last_active":"2020-08-07T15:14:04.610549+0000","last_peered":"2020-08-07T15:14:04.610549+0000","last_clean":"2020-08-07T15:14:04.610549+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:14:04.610549+0000","last_undegraded":"2020-08-07T15:14:04.610549+0000","last_fullsized":"2020-08-07T15:14:04.610549+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:13.231762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.231762+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"310'746","reported_seq":"754","reported_epoch":"310","state":"active+clean","last_fresh":"2020-08-07T15:14:06.761588+0000","last_change":"2020-08-07T15:13:57.260328+0000","last_active":"2020-08-07T15:14:06.761588+0000","last_peered":"2020-08-07T15:14:06.761588+0000","last_clean":"2020-08-07T15:14:06.761588+0000","last_became_active":"2020-08-07T15:13:57.259915+0000","last_became_peered":"2020-08-07T15:13:57.259915+0000","last_unstale":"2020-08-07T15:14:06.761588+0000","last_undegraded":"2020-08-07T15:14:06.761588+0000","last_fullsized":"2020-08-07T15:14:06.761588+0000","mapping_epoch":301,"log_start":"310'736","ondisk_log_start":"310'736","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6111232,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":746,"num_write_kb":5968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"309","reported_epoch":"306","state":"active+clean","last_fresh":"2020-08-07T15:14:01.564029+0000","last_change":"2020-08-07T15:12:27.410339+0000","last_active":"2020-08-07T15:14:01.564029+0000","last_peered":"2020-08-07T15:14:01.564029+0000","last_clean":"2020-08-07T15:14:01.564029+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:14:01.564029+0000","last_undegraded":"2020-08-07T15:14:01.564029+0000","last_fullsized":"2020-08-07T15:14:01.564029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:27.410262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:27.410262+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"240","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.614536+0000","last_change":"2020-08-07T15:12:39.258928+0000","last_active":"2020-08-07T15:14:04.614536+0000","last_peered":"2020-08-07T15:14:04.614536+0000","last_clean":"2020-08-07T15:14:04.614536+0000","last_became_active":"2020-08-07T15:12:32.259956+0000","last_became_peered":"2020-08-07T15:12:32.259956+0000","last_unstale":"2020-08-07T15:14:04.614536+0000","last_undegraded":"2020-08-07T15:14:04.614536+0000","last_fullsized":"2020-08-07T15:14:04.614536+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:39.258857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:39.258857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"309'640","reported_seq":"657","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:06.598870+0000","last_change":"2020-08-07T15:13:57.260186+0000","last_active":"2020-08-07T15:14:06.598870+0000","last_peered":"2020-08-07T15:14:06.598870+0000","last_clean":"2020-08-07T15:14:06.598870+0000","last_became_active":"2020-08-07T15:13:57.259750+0000","last_became_peered":"2020-08-07T15:13:57.259750+0000","last_unstale":"2020-08-07T15:14:06.598870+0000","last_undegraded":"2020-08-07T15:14:06.598870+0000","last_fullsized":"2020-08-07T15:14:06.598870+0000","mapping_epoch":301,"log_start":"309'632","ondisk_log_start":"309'632","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5242880,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":640,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"267","reported_epoch":"308","state":"active+clean","last_fresh":"2020-08-07T15:14:03.621404+0000","last_change":"2020-08-07T15:12:19.561477+0000","last_active":"2020-08-07T15:14:03.621404+0000","last_peered":"2020-08-07T15:14:03.621404+0000","last_clean":"2020-08-07T15:14:03.621404+0000","last_became_active":"2020-08-07T15:12:06.367376+0000","last_became_peered":"2020-08-07T15:12:06.367376+0000","last_unstale":"2020-08-07T15:14:03.621404+0000","last_undegraded":"2020-08-07T15:14:03.621404+0000","last_fullsized":"2020-08-07T15:14:03.621404+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:12.319383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.319383+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"297","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.619065+0000","last_change":"2020-08-07T15:14:01.819859+0000","last_active":"2020-08-07T15:14:04.619065+0000","last_peered":"2020-08-07T15:14:04.619065+0000","last_clean":"2020-08-07T15:14:04.619065+0000","last_became_active":"2020-08-07T15:12:32.241146+0000","last_became_peered":"2020-08-07T15:12:32.241146+0000","last_unstale":"2020-08-07T15:14:04.619065+0000","last_undegraded":"2020-08-07T15:14:04.619065+0000","last_fullsized":"2020-08-07T15:14:04.619065+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:14:01.819763+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:14:01.819763+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"309'664","reported_seq":"681","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:06.508955+0000","last_change":"2020-08-07T15:13:57.257749+0000","last_active":"2020-08-07T15:14:06.508955+0000","last_peered":"2020-08-07T15:14:06.508955+0000","last_clean":"2020-08-07T15:14:06.508955+0000","last_became_active":"2020-08-07T15:13:57.257285+0000","last_became_peered":"2020-08-07T15:13:57.257285+0000","last_unstale":"2020-08-07T15:14:06.508955+0000","last_undegraded":"2020-08-07T15:14:06.508955+0000","last_fullsized":"2020-08-07T15:14:06.508955+0000","mapping_epoch":301,"log_start":"309'657","ondisk_log_start":"309'657","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":664,"num_write_kb":5312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"313","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.606468+0000","last_change":"2020-08-07T15:13:11.273543+0000","last_active":"2020-08-07T15:14:04.606468+0000","last_peered":"2020-08-07T15:14:04.606468+0000","last_clean":"2020-08-07T15:14:04.606468+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:14:04.606468+0000","last_undegraded":"2020-08-07T15:14:04.606468+0000","last_fullsized":"2020-08-07T15:14:04.606468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:11.273469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:13:11.273469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"369","reported_epoch":"306","state":"active+clean","last_fresh":"2020-08-07T15:14:01.563834+0000","last_change":"2020-08-07T15:12:53.210176+0000","last_active":"2020-08-07T15:14:01.563834+0000","last_peered":"2020-08-07T15:14:01.563834+0000","last_clean":"2020-08-07T15:14:01.563834+0000","last_became_active":"2020-08-07T15:12:32.239755+0000","last_became_peered":"2020-08-07T15:12:32.239755+0000","last_unstale":"2020-08-07T15:14:01.563834+0000","last_undegraded":"2020-08-07T15:14:01.563834+0000","last_fullsized":"2020-08-07T15:14:01.563834+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:53.210093+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_clean_scrub_stamp":"2020-08-07T15:12:53.210093+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.5","version":"310'704","reported_seq":"721","reported_epoch":"310","state":"active+clean","last_fresh":"2020-08-07T15:14:06.758255+0000","last_change":"2020-08-07T15:13:57.263162+0000","last_active":"2020-08-07T15:14:06.758255+0000","last_peered":"2020-08-07T15:14:06.758255+0000","last_clean":"2020-08-07T15:14:06.758255+0000","last_became_active":"2020-08-07T15:13:57.262532+0000","last_became_peered":"2020-08-07T15:13:57.262532+0000","last_unstale":"2020-08-07T15:14:06.758255+0000","last_undegraded":"2020-08-07T15:14:06.758255+0000","last_fullsized":"2020-08-07T15:14:06.758255+0000","mapping_epoch":301,"log_start":"310'696","ondisk_log_start":"310'696","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"282","reported_epoch":"308","state":"active+clean","last_fresh":"2020-08-07T15:14:03.621905+0000","last_change":"2020-08-07T15:13:02.591821+0000","last_active":"2020-08-07T15:14:03.621905+0000","last_peered":"2020-08-07T15:14:03.621905+0000","last_clean":"2020-08-07T15:14:03.621905+0000","last_became_active":"2020-08-07T15:13:02.591402+0000","last_became_peered":"2020-08-07T15:13:02.591402+0000","last_unstale":"2020-08-07T15:14:03.621905+0000","last_undegraded":"2020-08-07T15:14:03.621905+0000","last_fullsized":"2020-08-07T15:14:03.621905+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:08.622683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:08.622683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"338","reported_epoch":"306","state":"active+clean","last_fresh":"2020-08-07T15:14:01.563702+0000","last_change":"2020-08-07T15:12:42.371309+0000","last_active":"2020-08-07T15:14:01.563702+0000","last_peered":"2020-08-07T15:14:01.563702+0000","last_clean":"2020-08-07T15:14:01.563702+0000","last_became_active":"2020-08-07T15:12:32.237984+0000","last_became_peered":"2020-08-07T15:12:32.237984+0000","last_unstale":"2020-08-07T15:14:01.563702+0000","last_undegraded":"2020-08-07T15:14:01.563702+0000","last_fullsized":"2020-08-07T15:14:01.563702+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:42.371229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:42.371229+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"310'656","reported_seq":"673","reported_epoch":"310","state":"active+clean","last_fresh":"2020-08-07T15:14:06.666864+0000","last_change":"2020-08-07T15:13:57.262285+0000","last_active":"2020-08-07T15:14:06.666864+0000","last_peered":"2020-08-07T15:14:06.666864+0000","last_clean":"2020-08-07T15:14:06.666864+0000","last_became_active":"2020-08-07T15:13:57.261819+0000","last_became_peered":"2020-08-07T15:13:57.261819+0000","last_unstale":"2020-08-07T15:14:06.666864+0000","last_undegraded":"2020-08-07T15:14:06.666864+0000","last_fullsized":"2020-08-07T15:14:06.666864+0000","mapping_epoch":301,"log_start":"309'648","ondisk_log_start":"309'648","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"64","reported_epoch":"306","state":"active+clean","last_fresh":"2020-08-07T15:14:01.563576+0000","last_change":"2020-08-07T15:13:13.399410+0000","last_active":"2020-08-07T15:14:01.563576+0000","last_peered":"2020-08-07T15:14:01.563576+0000","last_clean":"2020-08-07T15:14:01.563576+0000","last_became_active":"2020-08-07T15:13:02.590656+0000","last_became_peered":"2020-08-07T15:13:02.590656+0000","last_unstale":"2020-08-07T15:14:01.563576+0000","last_undegraded":"2020-08-07T15:14:01.563576+0000","last_fullsized":"2020-08-07T15:14:01.563576+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:13.399327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:13.399327+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"310","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.608427+0000","last_change":"2020-08-07T15:11:40.328534+0000","last_active":"2020-08-07T15:14:04.608427+0000","last_peered":"2020-08-07T15:14:04.608427+0000","last_clean":"2020-08-07T15:14:04.608427+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:14:04.608427+0000","last_undegraded":"2020-08-07T15:14:04.608427+0000","last_fullsized":"2020-08-07T15:14:04.608427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:11:40.328482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:11:40.328482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"309'656","reported_seq":"670","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.793990+0000","last_change":"2020-08-07T15:13:57.263391+0000","last_active":"2020-08-07T15:14:04.793990+0000","last_peered":"2020-08-07T15:14:04.793990+0000","last_clean":"2020-08-07T15:14:04.793990+0000","last_became_active":"2020-08-07T15:13:57.262826+0000","last_became_peered":"2020-08-07T15:13:57.262826+0000","last_unstale":"2020-08-07T15:14:04.793990+0000","last_undegraded":"2020-08-07T15:14:04.793990+0000","last_fullsized":"2020-08-07T15:14:04.793990+0000","mapping_epoch":301,"log_start":"309'647","ondisk_log_start":"309'647","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"334","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.618619+0000","last_change":"2020-08-07T15:13:53.799387+0000","last_active":"2020-08-07T15:14:04.618619+0000","last_peered":"2020-08-07T15:14:04.618619+0000","last_clean":"2020-08-07T15:14:04.618619+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:14:04.618619+0000","last_undegraded":"2020-08-07T15:14:04.618619+0000","last_fullsized":"2020-08-07T15:14:04.618619+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:53.799305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:13:53.799305+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"370","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.605827+0000","last_change":"2020-08-07T15:12:38.850878+0000","last_active":"2020-08-07T15:14:04.605827+0000","last_peered":"2020-08-07T15:14:04.605827+0000","last_clean":"2020-08-07T15:14:04.605827+0000","last_became_active":"2020-08-07T15:12:32.239794+0000","last_became_peered":"2020-08-07T15:12:32.239794+0000","last_unstale":"2020-08-07T15:14:04.605827+0000","last_undegraded":"2020-08-07T15:14:04.605827+0000","last_fullsized":"2020-08-07T15:14:04.605827+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:38.850809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:38.850809+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"307'408","reported_seq":"422","reported_epoch":"307","state":"active+clean","last_fresh":"2020-08-07T15:14:02.666237+0000","last_change":"2020-08-07T15:13:57.266510+0000","last_active":"2020-08-07T15:14:02.666237+0000","last_peered":"2020-08-07T15:14:02.666237+0000","last_clean":"2020-08-07T15:14:02.666237+0000","last_became_active":"2020-08-07T15:13:57.265882+0000","last_became_peered":"2020-08-07T15:13:57.265882+0000","last_unstale":"2020-08-07T15:14:02.666237+0000","last_undegraded":"2020-08-07T15:14:02.666237+0000","last_fullsized":"2020-08-07T15:14:02.666237+0000","mapping_epoch":301,"log_start":"306'398","ondisk_log_start":"306'398","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3342336,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":408,"num_write_kb":3264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"325","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.608355+0000","last_change":"2020-08-07T15:12:07.340958+0000","last_active":"2020-08-07T15:14:04.608355+0000","last_peered":"2020-08-07T15:14:04.608355+0000","last_clean":"2020-08-07T15:14:04.608355+0000","last_became_active":"2020-08-07T15:12:06.366967+0000","last_became_peered":"2020-08-07T15:12:06.366967+0000","last_unstale":"2020-08-07T15:14:04.608355+0000","last_undegraded":"2020-08-07T15:14:04.608355+0000","last_fullsized":"2020-08-07T15:14:04.608355+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:07.340855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.340855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"320","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.608147+0000","last_change":"2020-08-07T15:13:24.998985+0000","last_active":"2020-08-07T15:14:04.608147+0000","last_peered":"2020-08-07T15:14:04.608147+0000","last_clean":"2020-08-07T15:14:04.608147+0000","last_became_active":"2020-08-07T15:12:32.246800+0000","last_became_peered":"2020-08-07T15:12:32.246800+0000","last_unstale":"2020-08-07T15:14:04.608147+0000","last_undegraded":"2020-08-07T15:14:04.608147+0000","last_fullsized":"2020-08-07T15:14:04.608147+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:24.998919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:24.998919+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"309'776","reported_seq":"793","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:06.575727+0000","last_change":"2020-08-07T15:13:57.266426+0000","last_active":"2020-08-07T15:14:06.575727+0000","last_peered":"2020-08-07T15:14:06.575727+0000","last_clean":"2020-08-07T15:14:06.575727+0000","last_became_active":"2020-08-07T15:13:57.265966+0000","last_became_peered":"2020-08-07T15:13:57.265966+0000","last_unstale":"2020-08-07T15:14:06.575727+0000","last_undegraded":"2020-08-07T15:14:06.575727+0000","last_fullsized":"2020-08-07T15:14:06.575727+0000","mapping_epoch":301,"log_start":"309'767","ondisk_log_start":"309'767","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6356992,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":776,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"282","reported_epoch":"308","state":"active+clean","last_fresh":"2020-08-07T15:14:03.621324+0000","last_change":"2020-08-07T15:13:41.820648+0000","last_active":"2020-08-07T15:14:03.621324+0000","last_peered":"2020-08-07T15:14:03.621324+0000","last_clean":"2020-08-07T15:14:03.621324+0000","last_became_active":"2020-08-07T15:13:02.590901+0000","last_became_peered":"2020-08-07T15:13:02.590901+0000","last_unstale":"2020-08-07T15:14:03.621324+0000","last_undegraded":"2020-08-07T15:14:03.621324+0000","last_fullsized":"2020-08-07T15:14:03.621324+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:41.820573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:13:41.820573+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"403","reported_epoch":"306","state":"active+clean","last_fresh":"2020-08-07T15:14:01.564092+0000","last_change":"2020-08-07T15:12:43.351056+0000","last_active":"2020-08-07T15:14:01.564092+0000","last_peered":"2020-08-07T15:14:01.564092+0000","last_clean":"2020-08-07T15:14:01.564092+0000","last_became_active":"2020-08-07T15:12:32.254842+0000","last_became_peered":"2020-08-07T15:12:32.254842+0000","last_unstale":"2020-08-07T15:14:01.564092+0000","last_undegraded":"2020-08-07T15:14:01.564092+0000","last_fullsized":"2020-08-07T15:14:01.564092+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:43.350971+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:43.350971+0000","last_clean_scrub_stamp":"2020-08-07T15:12:43.350971+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.c","version":"307'416","reported_seq":"427","reported_epoch":"307","state":"active+clean","last_fresh":"2020-08-07T15:14:02.674233+0000","last_change":"2020-08-07T15:13:57.266895+0000","last_active":"2020-08-07T15:14:02.674233+0000","last_peered":"2020-08-07T15:14:02.674233+0000","last_clean":"2020-08-07T15:14:02.674233+0000","last_became_active":"2020-08-07T15:13:57.265943+0000","last_became_peered":"2020-08-07T15:13:57.265943+0000","last_unstale":"2020-08-07T15:14:02.674233+0000","last_undegraded":"2020-08-07T15:14:02.674233+0000","last_fullsized":"2020-08-07T15:14:02.674233+0000","mapping_epoch":301,"log_start":"307'409","ondisk_log_start":"307'409","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3407872,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":3328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"297","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.608235+0000","last_change":"2020-08-07T15:13:11.097271+0000","last_active":"2020-08-07T15:14:04.608235+0000","last_peered":"2020-08-07T15:14:04.608235+0000","last_clean":"2020-08-07T15:14:04.608235+0000","last_became_active":"2020-08-07T15:10:46.498230+0000","last_became_peered":"2020-08-07T15:10:46.498230+0000","last_unstale":"2020-08-07T15:14:04.608235+0000","last_undegraded":"2020-08-07T15:14:04.608235+0000","last_fullsized":"2020-08-07T15:14:04.608235+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:11.097214+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:13:11.097214+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.e","version":"310'792","reported_seq":"807","reported_epoch":"310","state":"active+clean","last_fresh":"2020-08-07T15:14:06.756250+0000","last_change":"2020-08-07T15:13:57.264444+0000","last_active":"2020-08-07T15:14:06.756250+0000","last_peered":"2020-08-07T15:14:06.756250+0000","last_clean":"2020-08-07T15:14:06.756250+0000","last_became_active":"2020-08-07T15:13:57.263649+0000","last_became_peered":"2020-08-07T15:13:57.263649+0000","last_unstale":"2020-08-07T15:14:06.756250+0000","last_undegraded":"2020-08-07T15:14:06.756250+0000","last_fullsized":"2020-08-07T15:14:06.756250+0000","mapping_epoch":301,"log_start":"310'783","ondisk_log_start":"310'783","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6488064,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":6336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"153","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.608034+0000","last_change":"2020-08-07T15:12:45.243933+0000","last_active":"2020-08-07T15:14:04.608034+0000","last_peered":"2020-08-07T15:14:04.608034+0000","last_clean":"2020-08-07T15:14:04.608034+0000","last_became_active":"2020-08-07T15:12:45.243349+0000","last_became_peered":"2020-08-07T15:12:45.243349+0000","last_unstale":"2020-08-07T15:14:04.608034+0000","last_undegraded":"2020-08-07T15:14:04.608034+0000","last_fullsized":"2020-08-07T15:14:04.608034+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.490655+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.490655+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"310'792","reported_seq":"808","reported_epoch":"310","state":"active+clean","last_fresh":"2020-08-07T15:14:06.719244+0000","last_change":"2020-08-07T15:13:57.266272+0000","last_active":"2020-08-07T15:14:06.719244+0000","last_peered":"2020-08-07T15:14:06.719244+0000","last_clean":"2020-08-07T15:14:06.719244+0000","last_became_active":"2020-08-07T15:13:57.265797+0000","last_became_peered":"2020-08-07T15:13:57.265797+0000","last_unstale":"2020-08-07T15:14:06.719244+0000","last_undegraded":"2020-08-07T15:14:06.719244+0000","last_fullsized":"2020-08-07T15:14:06.719244+0000","mapping_epoch":301,"log_start":"310'784","ondisk_log_start":"310'784","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6488064,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":6336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"115","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.615582+0000","last_change":"2020-08-07T15:13:10.415039+0000","last_active":"2020-08-07T15:14:04.615582+0000","last_peered":"2020-08-07T15:14:04.615582+0000","last_clean":"2020-08-07T15:14:04.615582+0000","last_became_active":"2020-08-07T15:13:02.590018+0000","last_became_peered":"2020-08-07T15:13:02.590018+0000","last_unstale":"2020-08-07T15:14:04.615582+0000","last_undegraded":"2020-08-07T15:14:04.615582+0000","last_fullsized":"2020-08-07T15:14:04.615582+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:10.414954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:10.414954+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"309'688","reported_seq":"703","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.865970+0000","last_change":"2020-08-07T15:13:57.257793+0000","last_active":"2020-08-07T15:14:04.865970+0000","last_peered":"2020-08-07T15:14:04.865970+0000","last_clean":"2020-08-07T15:14:04.865970+0000","last_became_active":"2020-08-07T15:13:57.257376+0000","last_became_peered":"2020-08-07T15:13:57.257376+0000","last_unstale":"2020-08-07T15:14:04.865970+0000","last_undegraded":"2020-08-07T15:14:04.865970+0000","last_fullsized":"2020-08-07T15:14:04.865970+0000","mapping_epoch":301,"log_start":"309'680","ondisk_log_start":"309'680","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5636096,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":688,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"331","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.606419+0000","last_change":"2020-08-07T15:13:38.089823+0000","last_active":"2020-08-07T15:14:04.606419+0000","last_peered":"2020-08-07T15:14:04.606419+0000","last_clean":"2020-08-07T15:14:04.606419+0000","last_became_active":"2020-08-07T15:12:06.380392+0000","last_became_peered":"2020-08-07T15:12:06.380392+0000","last_unstale":"2020-08-07T15:14:04.606419+0000","last_undegraded":"2020-08-07T15:14:04.606419+0000","last_fullsized":"2020-08-07T15:14:04.606419+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:13:38.089747+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:13:38.089747+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"310'648","reported_seq":"664","reported_epoch":"310","state":"active+clean","last_fresh":"2020-08-07T15:14:06.757256+0000","last_change":"2020-08-07T15:13:57.263679+0000","last_active":"2020-08-07T15:14:06.757256+0000","last_peered":"2020-08-07T15:14:06.757256+0000","last_clean":"2020-08-07T15:14:06.757256+0000","last_became_active":"2020-08-07T15:13:57.263194+0000","last_became_peered":"2020-08-07T15:13:57.263194+0000","last_unstale":"2020-08-07T15:14:06.757256+0000","last_undegraded":"2020-08-07T15:14:06.757256+0000","last_fullsized":"2020-08-07T15:14:06.757256+0000","mapping_epoch":301,"log_start":"310'643","ondisk_log_start":"310'643","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5308416,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":648,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"286","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.614642+0000","last_change":"2020-08-07T15:14:01.614941+0000","last_active":"2020-08-07T15:14:04.614642+0000","last_peered":"2020-08-07T15:14:04.614642+0000","last_clean":"2020-08-07T15:14:04.614642+0000","last_became_active":"2020-08-07T15:10:46.498051+0000","last_became_peered":"2020-08-07T15:10:46.498051+0000","last_unstale":"2020-08-07T15:14:04.614642+0000","last_undegraded":"2020-08-07T15:14:04.614642+0000","last_fullsized":"2020-08-07T15:14:04.614642+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:14:01.614870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:14:01.614870+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"310'656","reported_seq":"672","reported_epoch":"310","state":"active+clean","last_fresh":"2020-08-07T15:14:06.694189+0000","last_change":"2020-08-07T15:13:57.263784+0000","last_active":"2020-08-07T15:14:06.694189+0000","last_peered":"2020-08-07T15:14:06.694189+0000","last_clean":"2020-08-07T15:14:06.694189+0000","last_became_active":"2020-08-07T15:13:57.263369+0000","last_became_peered":"2020-08-07T15:13:57.263369+0000","last_unstale":"2020-08-07T15:14:06.694189+0000","last_undegraded":"2020-08-07T15:14:06.694189+0000","last_fullsized":"2020-08-07T15:14:06.694189+0000","mapping_epoch":301,"log_start":"310'651","ondisk_log_start":"310'651","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"359","reported_epoch":"306","state":"active+clean","last_fresh":"2020-08-07T15:14:01.564059+0000","last_change":"2020-08-07T15:13:52.144004+0000","last_active":"2020-08-07T15:14:01.564059+0000","last_peered":"2020-08-07T15:14:01.564059+0000","last_clean":"2020-08-07T15:14:01.564059+0000","last_became_active":"2020-08-07T15:12:32.241722+0000","last_became_peered":"2020-08-07T15:12:32.241722+0000","last_unstale":"2020-08-07T15:14:01.564059+0000","last_undegraded":"2020-08-07T15:14:01.564059+0000","last_fullsized":"2020-08-07T15:14:01.564059+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:52.143910+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_clean_scrub_stamp":"2020-08-07T15:13:52.143910+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.8","version":"310'784","reported_seq":"801","reported_epoch":"310","state":"active+clean","last_fresh":"2020-08-07T15:14:06.726938+0000","last_change":"2020-08-07T15:13:57.264595+0000","last_active":"2020-08-07T15:14:06.726938+0000","last_peered":"2020-08-07T15:14:06.726938+0000","last_clean":"2020-08-07T15:14:06.726938+0000","last_became_active":"2020-08-07T15:13:57.264126+0000","last_became_peered":"2020-08-07T15:13:57.264126+0000","last_unstale":"2020-08-07T15:14:06.726938+0000","last_undegraded":"2020-08-07T15:14:06.726938+0000","last_fullsized":"2020-08-07T15:14:06.726938+0000","mapping_epoch":301,"log_start":"310'776","ondisk_log_start":"310'776","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"72","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.615121+0000","last_change":"2020-08-07T15:13:11.459633+0000","last_active":"2020-08-07T15:14:04.615121+0000","last_peered":"2020-08-07T15:14:04.615121+0000","last_clean":"2020-08-07T15:14:04.615121+0000","last_became_active":"2020-08-07T15:13:02.591018+0000","last_became_peered":"2020-08-07T15:13:02.591018+0000","last_unstale":"2020-08-07T15:14:04.615121+0000","last_undegraded":"2020-08-07T15:14:04.615121+0000","last_fullsized":"2020-08-07T15:14:04.615121+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:11.459555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:11.459555+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"307'384","reported_seq":"398","reported_epoch":"307","state":"active+clean","last_fresh":"2020-08-07T15:14:02.708550+0000","last_change":"2020-08-07T15:13:57.260649+0000","last_active":"2020-08-07T15:14:02.708550+0000","last_peered":"2020-08-07T15:14:02.708550+0000","last_clean":"2020-08-07T15:14:02.708550+0000","last_became_active":"2020-08-07T15:13:57.260154+0000","last_became_peered":"2020-08-07T15:13:57.260154+0000","last_unstale":"2020-08-07T15:14:02.708550+0000","last_undegraded":"2020-08-07T15:14:02.708550+0000","last_fullsized":"2020-08-07T15:14:02.708550+0000","mapping_epoch":301,"log_start":"307'376","ondisk_log_start":"307'376","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3145728,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":384,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"281","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.608370+0000","last_change":"2020-08-07T15:12:44.203274+0000","last_active":"2020-08-07T15:14:04.608370+0000","last_peered":"2020-08-07T15:14:04.608370+0000","last_clean":"2020-08-07T15:14:04.608370+0000","last_became_active":"2020-08-07T15:12:06.367081+0000","last_became_peered":"2020-08-07T15:12:06.367081+0000","last_unstale":"2020-08-07T15:14:04.608370+0000","last_undegraded":"2020-08-07T15:14:04.608370+0000","last_fullsized":"2020-08-07T15:14:04.608370+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:44.203221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:44.203221+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":85278720,"num_objects":1302,"num_object_clones":0,"num_object_copies":3906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10410,"num_write_kb":83280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270016512,"data_stored":270016512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":41,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":123,"acting":123,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542239,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1062000,"kb_used_data":13416,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93309840,"statfs":{"total":96636764160,"available":95549276160,"internally_reserved":1073741824,"allocated":13737984,"data_stored":13150993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":36.792000000000002,"5min":16.657,"15min":6.1669999999999998},"min":{"1min":0.498,"5min":0.371,"15min":0.371},"max":{"1min":753.82299999999998,"5min":856.07100000000003,"15min":856.07100000000003},"last":0.40100000000000002},{"interface":"front","average":{"1min":5.7770000000000001,"5min":1.9390000000000001,"15min":1.2969999999999999},"min":{"1min":0.47099999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":103.30200000000001,"5min":103.30200000000001,"15min":103.30200000000001},"last":1.044}]},{"osd":1,"last update":"Fri Aug 7 15:13:10 2020","interfaces":[{"interface":"back","average":{"1min":10.148999999999999,"5min":10.148999999999999,"15min":10.148999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":205.73699999999999,"5min":205.73699999999999,"15min":205.73699999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":3.827,"5min":3.827,"15min":3.827},"last":0.67500000000000004}]},{"osd":2,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":15.460000000000001,"5min":4.1349999999999998,"15min":2.1469999999999998},"min":{"1min":0.63900000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":297.721,"5min":297.721,"15min":297.721},"last":4.1980000000000004},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.248,"15min":1.1859999999999999},"min":{"1min":0.66500000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.7809999999999999,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":4.2249999999999996}]},{"osd":3,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":3.081,"5min":22.274000000000001,"15min":8.5120000000000005},"min":{"1min":0.60699999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":32.908000000000001,"5min":917.38699999999994,"15min":917.38699999999994},"last":1.105},{"interface":"front","average":{"1min":3.1760000000000002,"5min":5.2690000000000001,"15min":2.6469999999999998},"min":{"1min":0.69499999999999995,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":32.881999999999998,"5min":333.59199999999998,"15min":333.59199999999998},"last":1.081}]},{"osd":4,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":42.588999999999999,"5min":11.105,"15min":4.5860000000000003},"min":{"1min":0.71399999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":873.673,"5min":873.673,"15min":873.673},"last":0.878},{"interface":"front","average":{"1min":1.0660000000000001,"5min":11.461,"15min":4.7009999999999996},"min":{"1min":0.71299999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.571,"5min":866.24099999999999,"15min":866.24099999999999},"last":0.65300000000000002}]},{"osd":5,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":9.1910000000000007,"15min":25.727},"min":{"1min":0.66200000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":3.234,"5min":678.76099999999997,"15min":831.88099999999997},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.538,"5min":1.647,"15min":1.6100000000000001},"min":{"1min":0.85299999999999998,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":3.4340000000000002,"5min":6.6790000000000003,"15min":7.5350000000000001},"last":0.92500000000000004}]},{"osd":7,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":3.9100000000000001,"15min":2.323},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.1840000000000002,"5min":224.14500000000001,"15min":224.14500000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.512,"5min":1.6830000000000001,"15min":1.593},"min":{"1min":0.91900000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":3.4049999999999998,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":0.94299999999999995}]}]},{"osd":7,"up_from":13,"seq":55834574945,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1078228,"kb_used_data":29644,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93293612,"statfs":{"total":96636764160,"available":95532658688,"internally_reserved":1073741824,"allocated":30355456,"data_stored":29746076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":16.684999999999999,"15min":6.4219999999999997},"min":{"1min":0.61099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.95,"5min":763.39099999999996,"15min":763.39099999999996},"last":0.625},{"interface":"front","average":{"1min":1.026,"5min":9.7690000000000001,"15min":4.0179999999999998},"min":{"1min":0.57399999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.9970000000000001,"5min":529.88699999999994,"15min":529.88699999999994},"last":1.736}]},{"osd":1,"last update":"Fri Aug 7 15:14:05 2020","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":18.940000000000001,"15min":21.934000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.7090000000000001,"5min":472.59699999999998,"15min":472.59699999999998},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.54,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.198}]},{"osd":2,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":1.677,"5min":1.776,"15min":1.3899999999999999},"min":{"1min":0.53400000000000003,"5min":0.502,"15min":0.49299999999999999},"max":{"1min":5.806,"5min":33.131999999999998,"15min":33.131999999999998},"last":2.415},{"interface":"front","average":{"1min":1.667,"5min":1.6080000000000001,"15min":1.3720000000000001},"min":{"1min":0.70099999999999996,"5min":0.58599999999999997,"15min":0.58399999999999996},"max":{"1min":5.3170000000000002,"5min":32.924999999999997,"15min":32.924999999999997},"last":2.25}]},{"osd":3,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":9.4909999999999997,"15min":4.1680000000000001},"min":{"1min":0.68200000000000005,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":5.7030000000000003,"5min":698.19899999999996,"15min":698.19899999999996},"last":2.3039999999999998},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.3,"15min":1.3500000000000001},"min":{"1min":0.69699999999999995,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.2909999999999999,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":2.222}]},{"osd":4,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":6.0369999999999999,"15min":3.0960000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.821,"5min":460.78399999999999,"15min":460.78399999999999},"last":1.6519999999999999},{"interface":"front","average":{"1min":38.359000000000002,"5min":8.8149999999999995,"15min":4.0510000000000002},"min":{"1min":0.80900000000000005,"5min":0.621,"15min":0.621},"max":{"1min":668.06799999999998,"5min":668.06799999999998,"15min":668.06799999999998},"last":1.6719999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.367,"15min":1.3520000000000001},"min":{"1min":0.67900000000000005,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.0489999999999999,"5min":6.2080000000000002,"15min":6.2080000000000002},"last":2.2709999999999999},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.47,"15min":1.6419999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.6160000000000001,"5min":4.407,"15min":5.327},"last":0.95199999999999996}]},{"osd":6,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":3.5600000000000001,"5min":9.5960000000000001,"15min":4.1580000000000004},"min":{"1min":0.872,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":40.079999999999998,"5min":695.92899999999997,"15min":695.92899999999997},"last":0.98399999999999999},{"interface":"front","average":{"1min":3.6760000000000002,"5min":1.9910000000000001,"15min":11.757},"min":{"1min":0.91900000000000004,"5min":0.88700000000000001,"15min":0.87},"max":{"1min":41.655999999999999,"5min":41.655999999999999,"15min":266.26999999999998},"last":1.2310000000000001}]}]},{"osd":1,"up_from":227,"seq":974957576217,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1072904,"kb_used_data":24320,"kb_used_omap":2,"kb_used_meta":1048573,"kb_avail":93298936,"statfs":{"total":96636764160,"available":95538110464,"internally_reserved":1073741824,"allocated":24903680,"data_stored":24301763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":1073739245},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":27.861000000000001,"5min":27.861000000000001,"15min":27.861000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":540.92499999999995,"5min":540.92499999999995,"15min":540.92499999999995},"last":1.867},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":3.7069999999999999,"5min":3.7069999999999999,"15min":3.7069999999999999},"last":0.52400000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":3.758,"5min":3.758,"15min":3.758},"last":1.026}]},{"osd":3,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":13.909000000000001,"5min":13.909000000000001,"15min":13.909000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":250.995,"5min":250.995,"15min":250.995},"last":0.879},{"interface":"front","average":{"1min":2.04,"5min":2.04,"15min":2.04},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":6.6580000000000004,"5min":6.6580000000000004,"15min":6.6580000000000004},"last":2.52}]},{"osd":4,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":0.60199999999999998},{"interface":"front","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":0.73599999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.498,"5min":1.498,"15min":1.498},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.6720000000000002,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.292}]},{"osd":6,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.1019999999999999},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":9.4299999999999997,"5min":9.4299999999999997,"15min":9.4299999999999997},"last":259.053},{"interface":"front","average":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":3.4199999999999999,"5min":3.4199999999999999,"15min":3.4199999999999999},"last":0.91100000000000003}]},{"osd":7,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":40.520000000000003,"5min":40.520000000000003,"15min":40.520000000000003},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":778.745,"5min":778.745,"15min":778.745},"last":1.1539999999999999},{"interface":"front","average":{"1min":24.100000000000001,"5min":24.100000000000001,"15min":24.100000000000001},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":442.62799999999999,"5min":442.62799999999999,"15min":442.62799999999999},"last":1.8029999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574945,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1108740,"kb_used_data":60156,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93263100,"statfs":{"total":96636764160,"available":95501414400,"internally_reserved":1073741824,"allocated":61599744,"data_stored":60990364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:13:07 2020","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.468,"5min":2.468,"15min":2.468},"last":1.129},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.5550000000000002,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.171}]},{"osd":2,"last update":"Fri Aug 7 15:13:36 2020","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.0489999999999999,"15min":0.99399999999999999},"min":{"1min":0.47199999999999998,"5min":0.441,"15min":0.441},"max":{"1min":3.6099999999999999,"5min":7.3470000000000004,"15min":7.3470000000000004},"last":1.087},{"interface":"front","average":{"1min":1.742,"5min":4.8120000000000003,"15min":21.315000000000001},"min":{"1min":0.64800000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.4909999999999997,"5min":226.911,"15min":606.18499999999995},"last":0.76500000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:13:36 2020","interfaces":[{"interface":"back","average":{"1min":1.615,"5min":6.7089999999999996,"15min":2.8999999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":4.1619999999999999,"5min":431.95299999999997,"15min":431.95299999999997},"last":5.3970000000000002},{"interface":"front","average":{"1min":1.6359999999999999,"5min":1.4019999999999999,"15min":1.296},"min":{"1min":0.73099999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":4.3449999999999998,"5min":11.19,"15min":11.19},"last":0.80500000000000005}]},{"osd":4,"last update":"Fri Aug 7 15:13:36 2020","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.25,"15min":1.218},"min":{"1min":0.629,"5min":0.501,"15min":0.501},"max":{"1min":2.7610000000000001,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":5.4379999999999997},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.407,"15min":1.3360000000000001},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":3.5600000000000001,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":0.85699999999999998}]},{"osd":5,"last update":"Fri Aug 7 15:13:36 2020","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":6.9930000000000003,"15min":5.702},"min":{"1min":0.76800000000000002,"5min":0.62,"15min":0.62},"max":{"1min":2.7400000000000002,"5min":539.31100000000004,"15min":539.31100000000004},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3979999999999999,"15min":1.248},"min":{"1min":0.78700000000000003,"5min":0.752,"15min":0.65900000000000003},"max":{"1min":2.714,"5min":4.0140000000000002,"15min":6.0010000000000003},"last":5.4690000000000003}]},{"osd":6,"last update":"Fri Aug 7 15:13:36 2020","interfaces":[{"interface":"back","average":{"1min":1.5900000000000001,"5min":1.403,"15min":5.7539999999999996},"min":{"1min":0.90500000000000003,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":3.585,"5min":3.585,"15min":124.738},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5700000000000001,"15min":11.813000000000001},"min":{"1min":0.79200000000000004,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.867,"5min":5.274,"15min":294.94},"last":1.2050000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:13:36 2020","interfaces":[{"interface":"back","average":{"1min":1.6970000000000001,"5min":1.5349999999999999,"15min":1.5569999999999999},"min":{"1min":0.995,"5min":0.84199999999999997,"15min":0.79300000000000004},"max":{"1min":2.9529999999999998,"5min":4.7169999999999996,"15min":6.468},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.74,"5min":23.527999999999999,"15min":8.7870000000000008},"min":{"1min":1.097,"5min":0.752,"15min":0.752},"max":{"1min":2.9220000000000002,"5min":971.12699999999995,"15min":971.12699999999995},"last":1.2629999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574945,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1079596,"kb_used_data":31012,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93292244,"statfs":{"total":96636764160,"available":95531257856,"internally_reserved":1073741824,"allocated":31756288,"data_stored":31146908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:28 2020","interfaces":[{"interface":"back","average":{"1min":18.065999999999999,"5min":11.218,"15min":4.4960000000000004},"min":{"1min":0.441,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":377.32900000000001,"5min":501.53800000000001,"15min":501.53800000000001},"last":0.443},{"interface":"front","average":{"1min":1.0760000000000001,"5min":6.649,"15min":2.8860000000000001},"min":{"1min":0.46800000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":3.0139999999999998,"5min":501.36000000000001,"15min":501.36000000000001},"last":0.51800000000000002}]},{"osd":1,"last update":"Fri Aug 7 15:13:07 2020","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":5.7869999999999999,"5min":5.7869999999999999,"15min":5.7869999999999999},"last":1.4399999999999999},{"interface":"front","average":{"1min":24.699999999999999,"5min":24.699999999999999,"15min":24.699999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":465.69200000000001,"5min":465.69200000000001,"15min":465.69200000000001},"last":1.117}]},{"osd":3,"last update":"Fri Aug 7 15:13:28 2020","interfaces":[{"interface":"back","average":{"1min":3.847,"5min":1.696,"15min":1.3580000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":29.111000000000001,"5min":29.111000000000001,"15min":29.111000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":3.3849999999999998,"5min":15.673999999999999,"15min":6.2530000000000001},"min":{"1min":0.627,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":29.788,"5min":501.75599999999997,"15min":501.75599999999997},"last":0.77700000000000002}]},{"osd":4,"last update":"Fri Aug 7 15:13:29 2020","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.397,"15min":1.2869999999999999},"min":{"1min":0.73499999999999999,"5min":0.623,"15min":0.58099999999999996},"max":{"1min":2.5489999999999999,"5min":7.9569999999999999,"15min":19.783000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":44.813000000000002,"5min":10.178000000000001,"15min":4.46},"min":{"1min":0.84299999999999997,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":954.97900000000004,"5min":954.97900000000004,"15min":954.97900000000004},"last":0.83199999999999996}]},{"osd":5,"last update":"Fri Aug 7 15:13:28 2020","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.3420000000000001,"15min":1.294},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.992,"5min":5.2220000000000004,"15min":5.2220000000000004},"last":1.294},{"interface":"front","average":{"1min":45.744999999999997,"5min":24.559000000000001,"15min":23.603000000000002},"min":{"1min":0.57099999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":971.51400000000001,"5min":971.51400000000001,"15min":971.51400000000001},"last":0.80800000000000005}]},{"osd":6,"last update":"Fri Aug 7 15:13:39 2020","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":9.766,"15min":4.1550000000000002},"min":{"1min":0.872,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.6480000000000001,"5min":821.91899999999998,"15min":821.91899999999998},"last":1.145},{"interface":"front","average":{"1min":25.716999999999999,"5min":6.2949999999999999,"15min":5.8259999999999996},"min":{"1min":0.83799999999999997,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":477.69600000000003,"5min":477.69600000000003,"15min":751.70000000000005},"last":1.323}]},{"osd":7,"last update":"Fri Aug 7 15:13:28 2020","interfaces":[{"interface":"back","average":{"1min":43.765999999999998,"5min":20.567,"15min":8.1370000000000005},"min":{"1min":0.89200000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":930.55499999999995,"5min":930.55499999999995,"15min":930.55499999999995},"last":1.169},{"interface":"front","average":{"1min":1.8560000000000001,"5min":1.79,"15min":1.724},"min":{"1min":0.98999999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":5.6950000000000003,"5min":8.0779999999999994,"15min":35.646999999999998},"last":1.266}]}]},{"osd":3,"up_from":13,"seq":55834574945,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1101884,"kb_used_data":53300,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93269956,"statfs":{"total":96636764160,"available":95508434944,"internally_reserved":1073741824,"allocated":54579200,"data_stored":53969820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:38 2020","interfaces":[{"interface":"back","average":{"1min":53.417000000000002,"5min":11.500999999999999,"15min":4.7210000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":929.68899999999996,"5min":929.68899999999996,"15min":929.68899999999996},"last":2.3149999999999999},{"interface":"front","average":{"1min":42.985999999999997,"5min":9.5129999999999999,"15min":4.2489999999999997},"min":{"1min":0.496,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":744.76800000000003,"5min":744.76800000000003,"15min":744.76800000000003},"last":2.3399999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:13:08 2020","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.639,"15min":1.639},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":4.6890000000000001,"5min":4.6890000000000001,"15min":4.6890000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.841,"5min":1.841,"15min":1.841},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":5.0330000000000004,"5min":5.0330000000000004,"15min":5.0330000000000004},"last":1.1060000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:13:38 2020","interfaces":[{"interface":"back","average":{"1min":42.640000000000001,"5min":9.5429999999999993,"15min":3.9860000000000002},"min":{"1min":0.78800000000000003,"5min":0.56999999999999995,"15min":0.56899999999999995},"max":{"1min":810.50999999999999,"5min":810.50999999999999,"15min":810.50999999999999},"last":1.147},{"interface":"front","average":{"1min":1.536,"5min":1.3720000000000001,"15min":1.77},"min":{"1min":0.66100000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.9079999999999999,"5min":6.7889999999999997,"15min":51.530000000000001},"last":1.2070000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:13:38 2020","interfaces":[{"interface":"back","average":{"1min":13.329000000000001,"5min":10.928000000000001,"15min":7.8369999999999997},"min":{"1min":0.66600000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":234.19999999999999,"5min":646.26700000000005,"15min":646.26700000000005},"last":3.0819999999999999},{"interface":"front","average":{"1min":28.314,"5min":15.204000000000001,"15min":26.312000000000001},"min":{"1min":0.82099999999999995,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":519.82600000000002,"5min":844.10500000000002,"15min":844.10500000000002},"last":2.2559999999999998}]},{"osd":5,"last update":"Fri Aug 7 15:13:38 2020","interfaces":[{"interface":"back","average":{"1min":2.5899999999999999,"5min":8.6590000000000007,"15min":3.931},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":5.5170000000000003,"5min":403.88400000000001,"15min":403.88400000000001},"last":2.2890000000000001},{"interface":"front","average":{"1min":2.6150000000000002,"5min":1.976,"15min":1.8680000000000001},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":5.5430000000000001,"5min":8.9120000000000008,"15min":8.9120000000000008},"last":2.7240000000000002}]},{"osd":6,"last update":"Fri Aug 7 15:13:38 2020","interfaces":[{"interface":"back","average":{"1min":1.7070000000000001,"5min":1.6499999999999999,"15min":1.601},"min":{"1min":0.76500000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":3.508,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.9139999999999999,"5min":13.398999999999999,"15min":5.79},"min":{"1min":0.97799999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":3.524,"5min":688.15999999999997,"15min":688.15999999999997},"last":1.5249999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:13:38 2020","interfaces":[{"interface":"back","average":{"1min":1.7250000000000001,"5min":1.5389999999999999,"15min":1.944},"min":{"1min":0.96999999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":3.6120000000000001,"5min":6.8209999999999997,"15min":6.8209999999999997},"last":1.427},{"interface":"front","average":{"1min":2.3050000000000002,"5min":1.593,"15min":1.891},"min":{"1min":1.0469999999999999,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":7.0090000000000003,"5min":7.0090000000000003,"15min":8.6790000000000003},"last":3.25}]}]},{"osd":4,"up_from":13,"seq":55834574945,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1078348,"kb_used_data":29764,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93293492,"statfs":{"total":96636764160,"available":95532535808,"internally_reserved":1073741824,"allocated":30478336,"data_stored":29868956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:30 2020","interfaces":[{"interface":"back","average":{"1min":3.0019999999999998,"5min":20.065999999999999,"15min":7.4569999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":37.307000000000002,"5min":829.15099999999995,"15min":829.15099999999995},"last":0.85599999999999998},{"interface":"front","average":{"1min":3.242,"5min":2.0179999999999998,"15min":1.272},"min":{"1min":0.499,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":42.095999999999997,"5min":48.274000000000001,"15min":48.274000000000001},"last":1.5489999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:13:08 2020","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.075},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.95399999999999996}]},{"osd":2,"last update":"Fri Aug 7 15:13:30 2020","interfaces":[{"interface":"back","average":{"1min":3.718,"5min":2.3460000000000001,"15min":1.752},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.52100000000000002},"max":{"1min":37.365000000000002,"5min":49.878999999999998,"15min":49.878999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":12.497999999999999,"5min":4.1970000000000001,"15min":2.4260000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":146.34399999999999,"5min":146.34399999999999,"15min":146.34399999999999},"last":0.93400000000000005}]},{"osd":3,"last update":"Fri Aug 7 15:13:30 2020","interfaces":[{"interface":"back","average":{"1min":3.4969999999999999,"5min":15.436999999999999,"15min":6.891},"min":{"1min":0.63100000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":37.582000000000001,"5min":638.92100000000005,"15min":638.92100000000005},"last":0.97699999999999998},{"interface":"front","average":{"1min":3.4500000000000002,"5min":1.754,"15min":1.4399999999999999},"min":{"1min":0.72299999999999998,"5min":0.68500000000000005,"15min":0.66700000000000004},"max":{"1min":37.956000000000003,"5min":37.956000000000003,"15min":37.956000000000003},"last":0.84499999999999997}]},{"osd":5,"last update":"Fri Aug 7 15:13:30 2020","interfaces":[{"interface":"back","average":{"1min":3.6819999999999999,"5min":19.954000000000001,"15min":10.747},"min":{"1min":0.69999999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":43.692,"5min":862.97299999999996,"15min":990.71600000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":40.039000000000001,"5min":18.669,"15min":7.3150000000000004},"min":{"1min":0.58899999999999997,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":625.35000000000002,"5min":780.00699999999995,"15min":780.00699999999995},"last":0.70299999999999996}]},{"osd":6,"last update":"Fri Aug 7 15:13:38 2020","interfaces":[{"interface":"back","average":{"1min":3.3999999999999999,"5min":2.3260000000000001,"15min":1.9630000000000001},"min":{"1min":0.70799999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":41.585999999999999,"5min":48.575000000000003,"15min":48.575000000000003},"last":0.82699999999999996},{"interface":"front","average":{"1min":3.1400000000000001,"5min":1.657,"15min":12.119},"min":{"1min":0.69399999999999995,"5min":0.64600000000000002,"15min":0.59599999999999997},"max":{"1min":37.991,"5min":37.991,"15min":389.14600000000002},"last":0.871}]},{"osd":7,"last update":"Fri Aug 7 15:13:30 2020","interfaces":[{"interface":"back","average":{"1min":3.8620000000000001,"5min":2.4239999999999999,"15min":2.544},"min":{"1min":1.1319999999999999,"5min":0.62,"15min":0.62},"max":{"1min":38.979999999999997,"5min":48.75,"15min":252.798},"last":1.1679999999999999},{"interface":"front","average":{"1min":3.7989999999999999,"5min":6.46,"15min":6.1189999999999998},"min":{"1min":0.91300000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":38.899999999999999,"5min":326.327,"15min":501.93000000000001},"last":1.151}]}]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1071692,"kb_used_data":23108,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93300148,"statfs":{"total":96636764160,"available":95539351552,"internally_reserved":1073741824,"allocated":23662592,"data_stored":23053212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:27 2020","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":1.4770000000000001,"15min":1.204},"min":{"1min":0.45700000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.7070000000000001,"5min":43.292999999999999,"15min":43.292999999999999},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.03,"5min":4.3860000000000001,"15min":2.0259999999999998},"min":{"1min":0.47399999999999998,"5min":0.438,"15min":0.438},"max":{"1min":2.6059999999999999,"5min":329.11399999999998,"15min":329.11399999999998},"last":0.55800000000000005}]},{"osd":1,"last update":"Fri Aug 7 15:13:11 2020","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":4.4039999999999999,"5min":4.4039999999999999,"15min":4.4039999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":4.0810000000000004,"5min":4.0810000000000004,"15min":4.0810000000000004},"last":0.85399999999999998}]},{"osd":2,"last update":"Fri Aug 7 15:13:27 2020","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.6100000000000001,"15min":6.1319999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.4129999999999998,"5min":43.923000000000002,"15min":168.56899999999999},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.615,"15min":20.503},"min":{"1min":0.70499999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.5750000000000002,"5min":43.076999999999998,"15min":676.27099999999996},"last":0.59299999999999997}]},{"osd":3,"last update":"Fri Aug 7 15:13:27 2020","interfaces":[{"interface":"back","average":{"1min":4.1559999999999997,"5min":2.2570000000000001,"15min":1.5229999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":47.371000000000002,"5min":47.371000000000002,"15min":47.371000000000002},"last":1.373},{"interface":"front","average":{"1min":4.0990000000000002,"5min":2.2349999999999999,"15min":4.3440000000000003},"min":{"1min":0.72899999999999998,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":48.338999999999999,"5min":48.338999999999999,"15min":712.98699999999997},"last":1.1439999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:13:27 2020","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":12.773,"15min":34.875999999999998},"min":{"1min":0.68400000000000005,"5min":0.54800000000000004,"15min":0.53400000000000003},"max":{"1min":1.9139999999999999,"5min":971.67999999999995,"15min":971.67999999999995},"last":1.111},{"interface":"front","average":{"1min":16.669,"5min":4.3479999999999999,"15min":2.3100000000000001},"min":{"1min":0.78300000000000003,"5min":0.61899999999999999,"15min":0.57099999999999995},"max":{"1min":261.08499999999998,"5min":261.08499999999998,"15min":261.08499999999998},"last":0.754}]},{"osd":6,"last update":"Fri Aug 7 15:13:27 2020","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":28.48,"15min":39.819000000000003},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.4940000000000002,"5min":966.60900000000004,"15min":966.60900000000004},"last":0.871},{"interface":"front","average":{"1min":1.3200000000000001,"5min":6.8049999999999997,"15min":17.957999999999998},"min":{"1min":0.71199999999999997,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.9450000000000001,"5min":501.79000000000002,"15min":517.26700000000005},"last":1.1279999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:13:27 2020","interfaces":[{"interface":"back","average":{"1min":13.619999999999999,"5min":3.8769999999999998,"15min":2.1800000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":208.03399999999999,"5min":208.03399999999999,"15min":208.03399999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.4470000000000001,"5min":12.840999999999999,"15min":5.3140000000000001},"min":{"1min":0.89400000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.7069999999999999,"5min":515.09100000000001,"15min":515.09100000000001},"last":0.73599999999999999}]}]}]}} 2020-08-07T15:14:08.855 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json 2020-08-07T15:14:08.856 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 10 16 14482 14466 11.3001 10.7266 0.0111755 0.0110429 2020-08-07T15:14:08.856 INFO:teuthology.orchestra.run.smithi074.stderr:noscrub is set 2020-08-07T15:14:08.857 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:14:08.847+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:14:08.857 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2020-08-07T15:14:08.951 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:14:08.948+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:14:09.051 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:14:09.049+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:14:09.086 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:14:09.086 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:14:09.087 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.087 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5792.0:15120 7.7 e312/301 7:e87ffc95:::benchmark_data_smithi074_71701_object1889:head v 312'1040, mlcod=312'1031)", 2020-08-07T15:14:09.087 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:14:09.062637+0000", 2020-08-07T15:14:09.088 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.022202469999999998, 2020-08-07T15:14:09.088 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.022229359000000001, 2020-08-07T15:14:09.088 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:09.088 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:14:09.089 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:09.089 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.089 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:09.091 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.062637+0000", 2020-08-07T15:14:09.091 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:09.091 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.092 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.092 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:09.092 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.062637+0000", 2020-08-07T15:14:09.092 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0201999999999997e-05 2020-08-07T15:14:09.093 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.093 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.093 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:09.093 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.062698+0000", 2020-08-07T15:14:09.094 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1280000000000002e-06 2020-08-07T15:14:09.094 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.094 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.094 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:09.095 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.062705+0000", 2020-08-07T15:14:09.095 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.021091456000000001 2020-08-07T15:14:09.095 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.095 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.095 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:09.096 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.083796+0000", 2020-08-07T15:14:09.096 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5560000000000002e-06 2020-08-07T15:14:09.096 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.096 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.096 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:09.096 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.083804+0000", 2020-08-07T15:14:09.097 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00041221299999999998 2020-08-07T15:14:09.097 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.097 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.097 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.097 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.084216+0000", 2020-08-07T15:14:09.098 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.5812e-05 2020-08-07T15:14:09.098 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.101 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.101 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:09.101 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.084242+0000", 2020-08-07T15:14:09.101 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.021604366999999999 2020-08-07T15:14:09.102 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.102 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:09.102 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.102 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.102 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.103 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop(client.5792.0:15145 7.c e312/301 7:391dd3d3:::benchmark_data_smithi074_71701_object1893:head v 312'1121, mlcod=312'1119)", 2020-08-07T15:14:09.103 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:14:09.083702+0000", 2020-08-07T15:14:09.103 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.001138338, 2020-08-07T15:14:09.103 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0012293339999999999, 2020-08-07T15:14:09.103 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:09.104 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:14:09.104 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:09.104 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.104 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:09.104 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.083702+0000", 2020-08-07T15:14:09.105 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:09.105 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.105 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.105 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:09.105 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.083702+0000", 2020-08-07T15:14:09.106 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5399000000000003e-05 2020-08-07T15:14:09.106 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.106 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.106 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:09.107 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.083757+0000", 2020-08-07T15:14:09.107 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2660000000000001e-06 2020-08-07T15:14:09.107 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.107 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.107 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:09.107 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.083765+0000", 2020-08-07T15:14:09.108 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00090814599999999997 2020-08-07T15:14:09.108 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.108 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.108 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:09.108 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.084673+0000", 2020-08-07T15:14:09.109 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5899999999999998e-06 2020-08-07T15:14:09.109 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.110 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.110 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:09.110 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.084679+0000", 2020-08-07T15:14:09.110 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4777999999999995e-05 2020-08-07T15:14:09.110 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.111 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.111 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.112 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.084754+0000", 2020-08-07T15:14:09.112 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.6089999999999999e-05 2020-08-07T15:14:09.112 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.112 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.112 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:09.113 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.084780+0000", 2020-08-07T15:14:09.113 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001078269 2020-08-07T15:14:09.113 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.113 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:09.113 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.114 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.114 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.114 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5792.0:15103 7.f 7:f47e5dde:::benchmark_data_smithi074_71701_object1887:head [set-alloc-hint object_size 65536 write_size 8192,write 49152~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e312)", 2020-08-07T15:14:09.114 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:14:09.048871+0000", 2020-08-07T15:14:09.114 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.035968541, 2020-08-07T15:14:09.115 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.036138918999999999, 2020-08-07T15:14:09.115 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:09.115 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:09.115 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:09.115 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5792", 2020-08-07T15:14:09.115 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/3840862776", 2020-08-07T15:14:09.116 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 15103 2020-08-07T15:14:09.116 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.116 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:09.116 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.116 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:09.117 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.048871+0000", 2020-08-07T15:14:09.117 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:09.117 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.117 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.117 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:09.118 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.048871+0000", 2020-08-07T15:14:09.118 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8974000000000002e-05 2020-08-07T15:14:09.118 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.118 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.118 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:09.119 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.048920+0000", 2020-08-07T15:14:09.119 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0510000000000002e-06 2020-08-07T15:14:09.119 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.120 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.120 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:09.120 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.048926+0000", 2020-08-07T15:14:09.120 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3169e-05 2020-08-07T15:14:09.121 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.121 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.121 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:09.122 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.048940+0000", 2020-08-07T15:14:09.132 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1879999999999999e-06 2020-08-07T15:14:09.134 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.135 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.136 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:09.137 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.048946+0000", 2020-08-07T15:14:09.137 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6230999999999998e-05 2020-08-07T15:14:09.140 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.142 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.142 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.142 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.048992+0000", 2020-08-07T15:14:09.143 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.3249000000000001e-05 2020-08-07T15:14:09.143 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.143 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.143 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:09.143 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.049085+0000", 2020-08-07T15:14:09.144 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00016525399999999999 2020-08-07T15:14:09.144 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.144 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.144 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:14:09.144 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.049250+0000", 2020-08-07T15:14:09.145 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0028751639999999999 2020-08-07T15:14:09.145 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.145 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.145 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:14:09.145 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.052126+0000", 2020-08-07T15:14:09.146 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9999999999999997e-07 2020-08-07T15:14:09.146 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.146 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.146 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:09.146 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.052126+0000", 2020-08-07T15:14:09.147 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4566000000000005e-05 2020-08-07T15:14:09.147 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.147 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.147 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:14:09.147 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.052191+0000", 2020-08-07T15:14:09.148 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0088572930000000005 2020-08-07T15:14:09.148 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.148 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.148 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:09.148 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.061048+0000", 2020-08-07T15:14:09.149 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.012176739000000001 2020-08-07T15:14:09.149 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.149 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:09.149 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.149 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.150 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.150 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.5792.0:15104 7.f 7:f47e5dde:::benchmark_data_smithi074_71701_object1887:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected e312)", 2020-08-07T15:14:09.150 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:14:09.049623+0000", 2020-08-07T15:14:09.150 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 0.035217226999999997, 2020-08-07T15:14:09.150 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.035493202000000001, 2020-08-07T15:14:09.151 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:09.151 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:09.151 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:09.151 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.5792", 2020-08-07T15:14:09.151 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/3840862776", 2020-08-07T15:14:09.152 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 15104 2020-08-07T15:14:09.152 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.152 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:09.152 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.152 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:09.153 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.049623+0000", 2020-08-07T15:14:09.153 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:09.153 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.153 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.153 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:09.154 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.049623+0000", 2020-08-07T15:14:09.154 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0550000000000001e-05 2020-08-07T15:14:09.154 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.154 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.154 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:09.155 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.049683+0000", 2020-08-07T15:14:09.155 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.5859999999999999e-06 2020-08-07T15:14:09.155 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.155 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.155 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:09.156 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.049691+0000", 2020-08-07T15:14:09.156 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6370000000000001e-05 2020-08-07T15:14:09.156 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.156 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.156 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:09.157 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.049707+0000", 2020-08-07T15:14:09.157 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2479999999999996e-06 2020-08-07T15:14:09.157 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.157 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.157 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:09.158 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.049715+0000", 2020-08-07T15:14:09.158 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00097779799999999999 2020-08-07T15:14:09.158 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.158 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.158 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.158 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.050693+0000", 2020-08-07T15:14:09.159 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011749299999999999 2020-08-07T15:14:09.159 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.159 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.159 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:09.159 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.050811+0000", 2020-08-07T15:14:09.160 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000191645 2020-08-07T15:14:09.160 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.160 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.160 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:14:09.160 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.051002+0000", 2020-08-07T15:14:09.161 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.001877419 2020-08-07T15:14:09.161 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.161 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.161 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:14:09.161 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.052880+0000", 2020-08-07T15:14:09.162 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9999999999999996e-07 2020-08-07T15:14:09.162 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.162 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.162 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:09.162 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.052881+0000", 2020-08-07T15:14:09.163 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3449999999999996e-05 2020-08-07T15:14:09.163 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.163 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.163 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:14:09.163 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.052954+0000", 2020-08-07T15:14:09.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0081324469999999992 2020-08-07T15:14:09.164 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.164 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.164 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:09.164 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:14:09.061086+0000", 2020-08-07T15:14:09.164 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.011463806 2020-08-07T15:14:09.166 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.166 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:09.166 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.166 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.167 INFO:teuthology.orchestra.run.smithi074.stdout: ], 2020-08-07T15:14:09.167 INFO:teuthology.orchestra.run.smithi074.stdout: "num_ops": 4 2020-08-07T15:14:09.167 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:14:09.170 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2020-08-07T15:14:09.171 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:14:09.168+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:14:09.271 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:14:09.269+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:14:09.372 INFO:teuthology.orchestra.run.smithi074.stdout: 2020-08-07T15:14:09.376 INFO:tasks.ceph.osd.0.smithi074.stderr:2020-08-07T15:14:09.375+0000 7ff48a32b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31562) UID: 0 2020-08-07T15:14:09.379 INFO:teuthology.orchestra.run.smithi074.stderr:dumped all 2020-08-07T15:14:09.394 INFO:teuthology.orchestra.run.smithi074.stdout:{"pg_ready":true,"pg_map":{"version":549,"stamp":"2020-08-07T15:14:08.702860+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":96878592,"num_objects":1481,"num_object_clones":0,"num_object_copies":4443,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11826,"num_write_kb":94608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":195,"acting":195,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":196,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8674176,"kb_used_data":285504,"kb_used_omap":2,"kb_used_meta":8388605,"kb_avail":746300544,"statfs":{"total":773094113280,"available":764211757056,"internally_reserved":8589934592,"allocated":292356096,"data_stored":287481106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":8589932013},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":61554688,"num_objects":940,"num_object_clones":0,"num_object_copies":2820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7514,"num_write_kb":60112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.100255"},"pg_stats":[{"pgid":"1.28","version":"16'2","reported_seq":"74","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.608159+0000","last_change":"2020-08-07T15:12:37.495634+0000","last_active":"2020-08-07T15:14:04.608159+0000","last_peered":"2020-08-07T15:14:04.608159+0000","last_clean":"2020-08-07T15:14:04.608159+0000","last_became_active":"2020-08-07T15:12:37.495085+0000","last_became_peered":"2020-08-07T15:12:37.495085+0000","last_unstale":"2020-08-07T15:14:04.608159+0000","last_undegraded":"2020-08-07T15:14:04.608159+0000","last_fullsized":"2020-08-07T15:14:04.608159+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.490655+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.490655+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.27","version":"16'2","reported_seq":"423","reported_epoch":"310","state":"active+clean","last_fresh":"2020-08-07T15:14:06.635382+0000","last_change":"2020-08-07T15:12:57.250017+0000","last_active":"2020-08-07T15:14:06.635382+0000","last_peered":"2020-08-07T15:14:06.635382+0000","last_clean":"2020-08-07T15:14:06.635382+0000","last_became_active":"2020-08-07T15:12:37.411869+0000","last_became_peered":"2020-08-07T15:12:37.411869+0000","last_unstale":"2020-08-07T15:14:06.635382+0000","last_undegraded":"2020-08-07T15:14:06.635382+0000","last_fullsized":"2020-08-07T15:14:06.635382+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:57.249927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:57.249927+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.26","version":"16'2","reported_seq":"83","reported_epoch":"308","state":"active+clean","last_fresh":"2020-08-07T15:14:03.621305+0000","last_change":"2020-08-07T15:12:40.765120+0000","last_active":"2020-08-07T15:14:03.621305+0000","last_peered":"2020-08-07T15:14:03.621305+0000","last_clean":"2020-08-07T15:14:03.621305+0000","last_became_active":"2020-08-07T15:12:37.457426+0000","last_became_peered":"2020-08-07T15:12:37.457426+0000","last_unstale":"2020-08-07T15:14:03.621305+0000","last_undegraded":"2020-08-07T15:14:03.621305+0000","last_fullsized":"2020-08-07T15:14:03.621305+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:40.765027+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_clean_scrub_stamp":"2020-08-07T15:12:40.765027+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.25","version":"75'3","reported_seq":"310","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.618965+0000","last_change":"2020-08-07T15:13:07.465488+0000","last_active":"2020-08-07T15:14:04.618965+0000","last_peered":"2020-08-07T15:14:04.618965+0000","last_clean":"2020-08-07T15:14:04.618965+0000","last_became_active":"2020-08-07T15:12:37.427816+0000","last_became_peered":"2020-08-07T15:12:37.427816+0000","last_unstale":"2020-08-07T15:14:04.618965+0000","last_undegraded":"2020-08-07T15:14:04.618965+0000","last_fullsized":"2020-08-07T15:14:04.618965+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:13:07.465410+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:13:07.465410+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.24","version":"16'2","reported_seq":"248","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.614428+0000","last_change":"2020-08-07T15:12:42.295755+0000","last_active":"2020-08-07T15:14:04.614428+0000","last_peered":"2020-08-07T15:14:04.614428+0000","last_clean":"2020-08-07T15:14:04.614428+0000","last_became_active":"2020-08-07T15:12:34.414144+0000","last_became_peered":"2020-08-07T15:12:34.414144+0000","last_unstale":"2020-08-07T15:14:04.614428+0000","last_undegraded":"2020-08-07T15:14:04.614428+0000","last_fullsized":"2020-08-07T15:14:04.614428+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:42.295672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:42.295672+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.23","version":"16'2","reported_seq":"312","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.684082+0000","last_change":"2020-08-07T15:14:04.684082+0000","last_active":"2020-08-07T15:14:04.684082+0000","last_peered":"2020-08-07T15:14:04.684082+0000","last_clean":"2020-08-07T15:14:04.684082+0000","last_became_active":"2020-08-07T15:13:38.274095+0000","last_became_peered":"2020-08-07T15:13:38.274095+0000","last_unstale":"2020-08-07T15:14:04.684082+0000","last_undegraded":"2020-08-07T15:14:04.684082+0000","last_fullsized":"2020-08-07T15:14:04.684082+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":293,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:14:04.684015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:14:04.684015+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.22","version":"16'2","reported_seq":"78","reported_epoch":"310","state":"active+clean","last_fresh":"2020-08-07T15:14:06.635481+0000","last_change":"2020-08-07T15:12:58.227750+0000","last_active":"2020-08-07T15:14:06.635481+0000","last_peered":"2020-08-07T15:14:06.635481+0000","last_clean":"2020-08-07T15:14:06.635481+0000","last_became_active":"2020-08-07T15:12:37.413046+0000","last_became_peered":"2020-08-07T15:12:37.413046+0000","last_unstale":"2020-08-07T15:14:06.635481+0000","last_undegraded":"2020-08-07T15:14:06.635481+0000","last_fullsized":"2020-08-07T15:14:06.635481+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:58.227699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:58.227699+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.21","version":"16'2","reported_seq":"84","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.613815+0000","last_change":"2020-08-07T15:12:58.387474+0000","last_active":"2020-08-07T15:14:04.613815+0000","last_peered":"2020-08-07T15:14:04.613815+0000","last_clean":"2020-08-07T15:14:04.613815+0000","last_became_active":"2020-08-07T15:12:37.418612+0000","last_became_peered":"2020-08-07T15:12:37.418612+0000","last_unstale":"2020-08-07T15:14:04.613815+0000","last_undegraded":"2020-08-07T15:14:04.613815+0000","last_fullsized":"2020-08-07T15:14:04.613815+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:58.387424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:58.387424+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.20","version":"16'2","reported_seq":"84","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.618717+0000","last_change":"2020-08-07T15:12:54.433326+0000","last_active":"2020-08-07T15:14:04.618717+0000","last_peered":"2020-08-07T15:14:04.618717+0000","last_clean":"2020-08-07T15:14:04.618717+0000","last_became_active":"2020-08-07T15:12:37.409249+0000","last_became_peered":"2020-08-07T15:12:37.409249+0000","last_unstale":"2020-08-07T15:14:04.618717+0000","last_undegraded":"2020-08-07T15:14:04.618717+0000","last_fullsized":"2020-08-07T15:14:04.618717+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":6,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:54.433245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:54.433245+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1f","version":"16'2","reported_seq":"77","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.618583+0000","last_change":"2020-08-07T15:13:09.471546+0000","last_active":"2020-08-07T15:14:04.618583+0000","last_peered":"2020-08-07T15:14:04.618583+0000","last_clean":"2020-08-07T15:14:04.618583+0000","last_became_active":"2020-08-07T15:12:37.410716+0000","last_became_peered":"2020-08-07T15:12:37.410716+0000","last_unstale":"2020-08-07T15:14:04.618583+0000","last_undegraded":"2020-08-07T15:14:04.618583+0000","last_fullsized":"2020-08-07T15:14:04.618583+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":244,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:09.471471+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_clean_scrub_stamp":"2020-08-07T15:13:09.471471+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1e","version":"16'2","reported_seq":"164","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.614277+0000","last_change":"2020-08-07T15:13:09.396723+0000","last_active":"2020-08-07T15:14:04.614277+0000","last_peered":"2020-08-07T15:14:04.614277+0000","last_clean":"2020-08-07T15:14:04.614277+0000","last_became_active":"2020-08-07T15:10:51.559894+0000","last_became_peered":"2020-08-07T15:10:51.559894+0000","last_unstale":"2020-08-07T15:14:04.614277+0000","last_undegraded":"2020-08-07T15:14:04.614277+0000","last_fullsized":"2020-08-07T15:14:04.614277+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:09.396633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:09.396633+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1d","version":"75'3","reported_seq":"190","reported_epoch":"308","state":"active+clean","last_fresh":"2020-08-07T15:14:03.621726+0000","last_change":"2020-08-07T15:13:24.343240+0000","last_active":"2020-08-07T15:14:03.621726+0000","last_peered":"2020-08-07T15:14:03.621726+0000","last_clean":"2020-08-07T15:14:03.621726+0000","last_became_active":"2020-08-07T15:13:24.342549+0000","last_became_peered":"2020-08-07T15:13:24.342549+0000","last_unstale":"2020-08-07T15:14:03.621726+0000","last_undegraded":"2020-08-07T15:14:03.621726+0000","last_fullsized":"2020-08-07T15:14:03.621726+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:28.855432+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:28.855432+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1c","version":"16'2","reported_seq":"166","reported_epoch":"310","state":"active+clean","last_fresh":"2020-08-07T15:14:06.635409+0000","last_change":"2020-08-07T15:13:07.471270+0000","last_active":"2020-08-07T15:14:06.635409+0000","last_peered":"2020-08-07T15:14:06.635409+0000","last_clean":"2020-08-07T15:14:06.635409+0000","last_became_active":"2020-08-07T15:13:02.591669+0000","last_became_peered":"2020-08-07T15:13:02.591669+0000","last_unstale":"2020-08-07T15:14:06.635409+0000","last_undegraded":"2020-08-07T15:14:06.635409+0000","last_fullsized":"2020-08-07T15:14:06.635409+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:07.471192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:07.471192+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1b","version":"16'2","reported_seq":"285","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.608252+0000","last_change":"2020-08-07T15:13:22.022126+0000","last_active":"2020-08-07T15:14:04.608252+0000","last_peered":"2020-08-07T15:14:04.608252+0000","last_clean":"2020-08-07T15:14:04.608252+0000","last_became_active":"2020-08-07T15:10:51.557392+0000","last_became_peered":"2020-08-07T15:10:51.557392+0000","last_unstale":"2020-08-07T15:14:04.608252+0000","last_undegraded":"2020-08-07T15:14:04.608252+0000","last_fullsized":"2020-08-07T15:14:04.608252+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:22.022047+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:57.796946+0000","last_clean_scrub_stamp":"2020-08-07T15:13:22.022047+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1a","version":"16'2","reported_seq":"55","reported_epoch":"308","state":"active+clean","last_fresh":"2020-08-07T15:14:03.621641+0000","last_change":"2020-08-07T15:13:06.881943+0000","last_active":"2020-08-07T15:14:03.621641+0000","last_peered":"2020-08-07T15:14:03.621641+0000","last_clean":"2020-08-07T15:14:03.621641+0000","last_became_active":"2020-08-07T15:13:02.590409+0000","last_became_peered":"2020-08-07T15:13:02.590409+0000","last_unstale":"2020-08-07T15:14:03.621641+0000","last_undegraded":"2020-08-07T15:14:03.621641+0000","last_fullsized":"2020-08-07T15:14:03.621641+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:06.881863+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:13:06.881863+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.19","version":"16'2","reported_seq":"207","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.613929+0000","last_change":"2020-08-07T15:12:15.372999+0000","last_active":"2020-08-07T15:14:04.613929+0000","last_peered":"2020-08-07T15:14:04.613929+0000","last_clean":"2020-08-07T15:14:04.613929+0000","last_became_active":"2020-08-07T15:10:51.559752+0000","last_became_peered":"2020-08-07T15:10:51.559752+0000","last_unstale":"2020-08-07T15:14:04.613929+0000","last_undegraded":"2020-08-07T15:14:04.613929+0000","last_fullsized":"2020-08-07T15:14:04.613929+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:15.372910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:15.372910+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.18","version":"16'2","reported_seq":"310","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.610434+0000","last_change":"2020-08-07T15:13:54.173407+0000","last_active":"2020-08-07T15:14:04.610434+0000","last_peered":"2020-08-07T15:14:04.610434+0000","last_clean":"2020-08-07T15:14:04.610434+0000","last_became_active":"2020-08-07T15:10:51.557665+0000","last_became_peered":"2020-08-07T15:10:51.557665+0000","last_unstale":"2020-08-07T15:14:04.610434+0000","last_undegraded":"2020-08-07T15:14:04.610434+0000","last_fullsized":"2020-08-07T15:14:04.610434+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:54.173329+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:13:54.173329+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.17","version":"16'2","reported_seq":"138","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.608628+0000","last_change":"2020-08-07T15:12:57.172008+0000","last_active":"2020-08-07T15:14:04.608628+0000","last_peered":"2020-08-07T15:14:04.608628+0000","last_clean":"2020-08-07T15:14:04.608628+0000","last_became_active":"2020-08-07T15:10:51.561769+0000","last_became_peered":"2020-08-07T15:10:51.561769+0000","last_unstale":"2020-08-07T15:14:04.608628+0000","last_undegraded":"2020-08-07T15:14:04.608628+0000","last_fullsized":"2020-08-07T15:14:04.608628+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:57.171893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:57.171893+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.16","version":"16'2","reported_seq":"83","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.608571+0000","last_change":"2020-08-07T15:12:52.576339+0000","last_active":"2020-08-07T15:14:04.608571+0000","last_peered":"2020-08-07T15:14:04.608571+0000","last_clean":"2020-08-07T15:14:04.608571+0000","last_became_active":"2020-08-07T15:12:52.575577+0000","last_became_peered":"2020-08-07T15:12:52.575577+0000","last_unstale":"2020-08-07T15:14:04.608571+0000","last_undegraded":"2020-08-07T15:14:04.608571+0000","last_fullsized":"2020-08-07T15:14:04.608571+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:12.883558+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:12.883558+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.883558+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.15","version":"75'3","reported_seq":"66","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.608494+0000","last_change":"2020-08-07T15:13:02.590465+0000","last_active":"2020-08-07T15:14:04.608494+0000","last_peered":"2020-08-07T15:14:04.608494+0000","last_clean":"2020-08-07T15:14:04.608494+0000","last_became_active":"2020-08-07T15:13:02.589879+0000","last_became_peered":"2020-08-07T15:13:02.589879+0000","last_unstale":"2020-08-07T15:14:04.608494+0000","last_undegraded":"2020-08-07T15:14:04.608494+0000","last_fullsized":"2020-08-07T15:14:04.608494+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:12:53.824617+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:12:53.824617+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.14","version":"16'2","reported_seq":"248","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.618475+0000","last_change":"2020-08-07T15:13:26.549332+0000","last_active":"2020-08-07T15:14:04.618475+0000","last_peered":"2020-08-07T15:14:04.618475+0000","last_clean":"2020-08-07T15:14:04.618475+0000","last_became_active":"2020-08-07T15:08:41.748807+0000","last_became_peered":"2020-08-07T15:08:41.748807+0000","last_unstale":"2020-08-07T15:14:04.618475+0000","last_undegraded":"2020-08-07T15:14:04.618475+0000","last_fullsized":"2020-08-07T15:14:04.618475+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:26.549247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:26.549247+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"16'2","reported_seq":"259","reported_epoch":"308","state":"active+clean","last_fresh":"2020-08-07T15:14:03.621358+0000","last_change":"2020-08-07T15:12:19.577926+0000","last_active":"2020-08-07T15:14:03.621358+0000","last_peered":"2020-08-07T15:14:03.621358+0000","last_clean":"2020-08-07T15:14:03.621358+0000","last_became_active":"2020-08-07T15:12:06.375861+0000","last_became_peered":"2020-08-07T15:12:06.375861+0000","last_unstale":"2020-08-07T15:14:03.621358+0000","last_undegraded":"2020-08-07T15:14:03.621358+0000","last_fullsized":"2020-08-07T15:14:03.621358+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:07.887356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.887356+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.12","version":"16'2","reported_seq":"260","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.608172+0000","last_change":"2020-08-07T15:12:26.104598+0000","last_active":"2020-08-07T15:14:04.608172+0000","last_peered":"2020-08-07T15:14:04.608172+0000","last_clean":"2020-08-07T15:14:04.608172+0000","last_became_active":"2020-08-07T15:12:06.367362+0000","last_became_peered":"2020-08-07T15:12:06.367362+0000","last_unstale":"2020-08-07T15:14:04.608172+0000","last_undegraded":"2020-08-07T15:14:04.608172+0000","last_fullsized":"2020-08-07T15:14:04.608172+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:26.104526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:26.104526+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.11","version":"16'2","reported_seq":"358","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.608089+0000","last_change":"2020-08-07T15:12:24.105593+0000","last_active":"2020-08-07T15:14:04.608089+0000","last_peered":"2020-08-07T15:14:04.608089+0000","last_clean":"2020-08-07T15:14:04.608089+0000","last_became_active":"2020-08-07T15:09:30.210141+0000","last_became_peered":"2020-08-07T15:09:30.210141+0000","last_unstale":"2020-08-07T15:14:04.608089+0000","last_undegraded":"2020-08-07T15:14:04.608089+0000","last_fullsized":"2020-08-07T15:14:04.608089+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:24.105531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:24.105531+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"16'2","reported_seq":"352","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.610549+0000","last_change":"2020-08-07T15:12:13.231819+0000","last_active":"2020-08-07T15:14:04.610549+0000","last_peered":"2020-08-07T15:14:04.610549+0000","last_clean":"2020-08-07T15:14:04.610549+0000","last_became_active":"2020-08-07T15:08:41.684900+0000","last_became_peered":"2020-08-07T15:08:41.684900+0000","last_unstale":"2020-08-07T15:14:04.610549+0000","last_undegraded":"2020-08-07T15:14:04.610549+0000","last_fullsized":"2020-08-07T15:14:04.610549+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":5,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:13.231762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:13.231762+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"310'746","reported_seq":"754","reported_epoch":"310","state":"active+clean","last_fresh":"2020-08-07T15:14:06.761588+0000","last_change":"2020-08-07T15:13:57.260328+0000","last_active":"2020-08-07T15:14:06.761588+0000","last_peered":"2020-08-07T15:14:06.761588+0000","last_clean":"2020-08-07T15:14:06.761588+0000","last_became_active":"2020-08-07T15:13:57.259915+0000","last_became_peered":"2020-08-07T15:13:57.259915+0000","last_unstale":"2020-08-07T15:14:06.761588+0000","last_undegraded":"2020-08-07T15:14:06.761588+0000","last_fullsized":"2020-08-07T15:14:06.761588+0000","mapping_epoch":301,"log_start":"310'736","ondisk_log_start":"310'736","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6111232,"num_objects":94,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":746,"num_write_kb":5968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"316","reported_epoch":"310","state":"active+clean","last_fresh":"2020-08-07T15:14:06.636835+0000","last_change":"2020-08-07T15:14:03.154832+0000","last_active":"2020-08-07T15:14:06.636835+0000","last_peered":"2020-08-07T15:14:06.636835+0000","last_clean":"2020-08-07T15:14:06.636835+0000","last_became_active":"2020-08-07T15:06:23.073238+0000","last_became_peered":"2020-08-07T15:06:23.073238+0000","last_unstale":"2020-08-07T15:14:06.636835+0000","last_undegraded":"2020-08-07T15:14:06.636835+0000","last_fullsized":"2020-08-07T15:14:06.636835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:14:03.154751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:14:03.154751+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"16'2","reported_seq":"240","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.614536+0000","last_change":"2020-08-07T15:12:39.258928+0000","last_active":"2020-08-07T15:14:04.614536+0000","last_peered":"2020-08-07T15:14:04.614536+0000","last_clean":"2020-08-07T15:14:04.614536+0000","last_became_active":"2020-08-07T15:12:32.259956+0000","last_became_peered":"2020-08-07T15:12:32.259956+0000","last_unstale":"2020-08-07T15:14:04.614536+0000","last_undegraded":"2020-08-07T15:14:04.614536+0000","last_fullsized":"2020-08-07T15:14:04.614536+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:39.258857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:39.258857+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"309'640","reported_seq":"657","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:06.598870+0000","last_change":"2020-08-07T15:13:57.260186+0000","last_active":"2020-08-07T15:14:06.598870+0000","last_peered":"2020-08-07T15:14:06.598870+0000","last_clean":"2020-08-07T15:14:06.598870+0000","last_became_active":"2020-08-07T15:13:57.259750+0000","last_became_peered":"2020-08-07T15:13:57.259750+0000","last_unstale":"2020-08-07T15:14:06.598870+0000","last_undegraded":"2020-08-07T15:14:06.598870+0000","last_fullsized":"2020-08-07T15:14:06.598870+0000","mapping_epoch":301,"log_start":"309'632","ondisk_log_start":"309'632","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5242880,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":640,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"267","reported_epoch":"308","state":"active+clean","last_fresh":"2020-08-07T15:14:03.621404+0000","last_change":"2020-08-07T15:12:19.561477+0000","last_active":"2020-08-07T15:14:03.621404+0000","last_peered":"2020-08-07T15:14:03.621404+0000","last_clean":"2020-08-07T15:14:03.621404+0000","last_became_active":"2020-08-07T15:12:06.367376+0000","last_became_peered":"2020-08-07T15:12:06.367376+0000","last_unstale":"2020-08-07T15:14:03.621404+0000","last_undegraded":"2020-08-07T15:14:03.621404+0000","last_fullsized":"2020-08-07T15:14:03.621404+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:12.319383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:12.319383+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"75'3","reported_seq":"297","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.619065+0000","last_change":"2020-08-07T15:14:01.819859+0000","last_active":"2020-08-07T15:14:04.619065+0000","last_peered":"2020-08-07T15:14:04.619065+0000","last_clean":"2020-08-07T15:14:04.619065+0000","last_became_active":"2020-08-07T15:12:32.241146+0000","last_became_peered":"2020-08-07T15:12:32.241146+0000","last_unstale":"2020-08-07T15:14:04.619065+0000","last_undegraded":"2020-08-07T15:14:04.619065+0000","last_fullsized":"2020-08-07T15:14:04.619065+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:14:01.819763+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:14:01.819763+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"309'664","reported_seq":"681","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:06.508955+0000","last_change":"2020-08-07T15:13:57.257749+0000","last_active":"2020-08-07T15:14:06.508955+0000","last_peered":"2020-08-07T15:14:06.508955+0000","last_clean":"2020-08-07T15:14:06.508955+0000","last_became_active":"2020-08-07T15:13:57.257285+0000","last_became_peered":"2020-08-07T15:13:57.257285+0000","last_unstale":"2020-08-07T15:14:06.508955+0000","last_undegraded":"2020-08-07T15:14:06.508955+0000","last_fullsized":"2020-08-07T15:14:06.508955+0000","mapping_epoch":301,"log_start":"309'657","ondisk_log_start":"309'657","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5439488,"num_objects":83,"num_object_clones":0,"num_object_copies":249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":664,"num_write_kb":5312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"313","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.606468+0000","last_change":"2020-08-07T15:13:11.273543+0000","last_active":"2020-08-07T15:14:04.606468+0000","last_peered":"2020-08-07T15:14:04.606468+0000","last_clean":"2020-08-07T15:14:04.606468+0000","last_became_active":"2020-08-07T15:06:23.225364+0000","last_became_peered":"2020-08-07T15:06:23.225364+0000","last_unstale":"2020-08-07T15:14:04.606468+0000","last_undegraded":"2020-08-07T15:14:04.606468+0000","last_fullsized":"2020-08-07T15:14:04.606468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:11.273469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:13:11.273469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"16'2","reported_seq":"373","reported_epoch":"310","state":"active+clean","last_fresh":"2020-08-07T15:14:06.635552+0000","last_change":"2020-08-07T15:12:53.210176+0000","last_active":"2020-08-07T15:14:06.635552+0000","last_peered":"2020-08-07T15:14:06.635552+0000","last_clean":"2020-08-07T15:14:06.635552+0000","last_became_active":"2020-08-07T15:12:32.239755+0000","last_became_peered":"2020-08-07T15:12:32.239755+0000","last_unstale":"2020-08-07T15:14:06.635552+0000","last_undegraded":"2020-08-07T15:14:06.635552+0000","last_fullsized":"2020-08-07T15:14:06.635552+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:53.210093+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:14.121397+0000","last_clean_scrub_stamp":"2020-08-07T15:12:53.210093+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.5","version":"310'704","reported_seq":"721","reported_epoch":"310","state":"active+clean","last_fresh":"2020-08-07T15:14:06.758255+0000","last_change":"2020-08-07T15:13:57.263162+0000","last_active":"2020-08-07T15:14:06.758255+0000","last_peered":"2020-08-07T15:14:06.758255+0000","last_clean":"2020-08-07T15:14:06.758255+0000","last_became_active":"2020-08-07T15:13:57.262532+0000","last_became_peered":"2020-08-07T15:13:57.262532+0000","last_unstale":"2020-08-07T15:14:06.758255+0000","last_undegraded":"2020-08-07T15:14:06.758255+0000","last_fullsized":"2020-08-07T15:14:06.758255+0000","mapping_epoch":301,"log_start":"310'696","ondisk_log_start":"310'696","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"282","reported_epoch":"308","state":"active+clean","last_fresh":"2020-08-07T15:14:03.621905+0000","last_change":"2020-08-07T15:13:02.591821+0000","last_active":"2020-08-07T15:14:03.621905+0000","last_peered":"2020-08-07T15:14:03.621905+0000","last_clean":"2020-08-07T15:14:03.621905+0000","last_became_active":"2020-08-07T15:13:02.591402+0000","last_became_peered":"2020-08-07T15:13:02.591402+0000","last_unstale":"2020-08-07T15:14:03.621905+0000","last_undegraded":"2020-08-07T15:14:03.621905+0000","last_fullsized":"2020-08-07T15:14:03.621905+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:08.622683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:12:08.622683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"16'2","reported_seq":"342","reported_epoch":"310","state":"active+clean","last_fresh":"2020-08-07T15:14:06.635359+0000","last_change":"2020-08-07T15:12:42.371309+0000","last_active":"2020-08-07T15:14:06.635359+0000","last_peered":"2020-08-07T15:14:06.635359+0000","last_clean":"2020-08-07T15:14:06.635359+0000","last_became_active":"2020-08-07T15:12:32.237984+0000","last_became_peered":"2020-08-07T15:12:32.237984+0000","last_unstale":"2020-08-07T15:14:06.635359+0000","last_undegraded":"2020-08-07T15:14:06.635359+0000","last_fullsized":"2020-08-07T15:14:06.635359+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:42.371229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:42.371229+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"310'656","reported_seq":"673","reported_epoch":"310","state":"active+clean","last_fresh":"2020-08-07T15:14:06.666864+0000","last_change":"2020-08-07T15:13:57.262285+0000","last_active":"2020-08-07T15:14:06.666864+0000","last_peered":"2020-08-07T15:14:06.666864+0000","last_clean":"2020-08-07T15:14:06.666864+0000","last_became_active":"2020-08-07T15:13:57.261819+0000","last_became_peered":"2020-08-07T15:13:57.261819+0000","last_unstale":"2020-08-07T15:14:06.666864+0000","last_undegraded":"2020-08-07T15:14:06.666864+0000","last_fullsized":"2020-08-07T15:14:06.666864+0000","mapping_epoch":301,"log_start":"309'648","ondisk_log_start":"309'648","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"68","reported_epoch":"310","state":"active+clean","last_fresh":"2020-08-07T15:14:06.635551+0000","last_change":"2020-08-07T15:13:13.399410+0000","last_active":"2020-08-07T15:14:06.635551+0000","last_peered":"2020-08-07T15:14:06.635551+0000","last_clean":"2020-08-07T15:14:06.635551+0000","last_became_active":"2020-08-07T15:13:02.590656+0000","last_became_peered":"2020-08-07T15:13:02.590656+0000","last_unstale":"2020-08-07T15:14:06.635551+0000","last_undegraded":"2020-08-07T15:14:06.635551+0000","last_fullsized":"2020-08-07T15:14:06.635551+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":3,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:13.399327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:13.399327+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"310","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.608427+0000","last_change":"2020-08-07T15:11:40.328534+0000","last_active":"2020-08-07T15:14:04.608427+0000","last_peered":"2020-08-07T15:14:04.608427+0000","last_clean":"2020-08-07T15:14:04.608427+0000","last_became_active":"2020-08-07T15:06:23.075388+0000","last_became_peered":"2020-08-07T15:06:23.075388+0000","last_unstale":"2020-08-07T15:14:04.608427+0000","last_undegraded":"2020-08-07T15:14:04.608427+0000","last_fullsized":"2020-08-07T15:14:04.608427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:11:40.328482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:11:40.328482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"309'656","reported_seq":"670","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.793990+0000","last_change":"2020-08-07T15:13:57.263391+0000","last_active":"2020-08-07T15:14:04.793990+0000","last_peered":"2020-08-07T15:14:04.793990+0000","last_clean":"2020-08-07T15:14:04.793990+0000","last_became_active":"2020-08-07T15:13:57.262826+0000","last_became_peered":"2020-08-07T15:13:57.262826+0000","last_unstale":"2020-08-07T15:14:04.793990+0000","last_undegraded":"2020-08-07T15:14:04.793990+0000","last_fullsized":"2020-08-07T15:14:04.793990+0000","mapping_epoch":301,"log_start":"309'647","ondisk_log_start":"309'647","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"334","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.618619+0000","last_change":"2020-08-07T15:13:53.799387+0000","last_active":"2020-08-07T15:14:04.618619+0000","last_peered":"2020-08-07T15:14:04.618619+0000","last_clean":"2020-08-07T15:14:04.618619+0000","last_became_active":"2020-08-07T15:09:27.181148+0000","last_became_peered":"2020-08-07T15:09:27.181148+0000","last_unstale":"2020-08-07T15:14:04.618619+0000","last_undegraded":"2020-08-07T15:14:04.618619+0000","last_fullsized":"2020-08-07T15:14:04.618619+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:53.799305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:13:53.799305+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":"370","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.605827+0000","last_change":"2020-08-07T15:12:38.850878+0000","last_active":"2020-08-07T15:14:04.605827+0000","last_peered":"2020-08-07T15:14:04.605827+0000","last_clean":"2020-08-07T15:14:04.605827+0000","last_became_active":"2020-08-07T15:12:32.239794+0000","last_became_peered":"2020-08-07T15:12:32.239794+0000","last_unstale":"2020-08-07T15:14:04.605827+0000","last_undegraded":"2020-08-07T15:14:04.605827+0000","last_fullsized":"2020-08-07T15:14:04.605827+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:38.850809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:38.850809+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"311'904","reported_seq":"921","reported_epoch":"311","state":"active+clean","last_fresh":"2020-08-07T15:14:07.702042+0000","last_change":"2020-08-07T15:13:57.266510+0000","last_active":"2020-08-07T15:14:07.702042+0000","last_peered":"2020-08-07T15:14:07.702042+0000","last_clean":"2020-08-07T15:14:07.702042+0000","last_became_active":"2020-08-07T15:13:57.265882+0000","last_became_peered":"2020-08-07T15:13:57.265882+0000","last_unstale":"2020-08-07T15:14:07.702042+0000","last_undegraded":"2020-08-07T15:14:07.702042+0000","last_fullsized":"2020-08-07T15:14:07.702042+0000","mapping_epoch":301,"log_start":"311'900","ondisk_log_start":"311'900","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7405568,"num_objects":113,"num_object_clones":0,"num_object_copies":339,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":904,"num_write_kb":7232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"325","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.608355+0000","last_change":"2020-08-07T15:12:07.340958+0000","last_active":"2020-08-07T15:14:04.608355+0000","last_peered":"2020-08-07T15:14:04.608355+0000","last_clean":"2020-08-07T15:14:04.608355+0000","last_became_active":"2020-08-07T15:12:06.366967+0000","last_became_peered":"2020-08-07T15:12:06.366967+0000","last_unstale":"2020-08-07T15:14:04.608355+0000","last_undegraded":"2020-08-07T15:14:04.608355+0000","last_fullsized":"2020-08-07T15:14:04.608355+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:12:07.340855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:08:30.339785+0000","last_clean_scrub_stamp":"2020-08-07T15:12:07.340855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"16'2","reported_seq":"320","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.608147+0000","last_change":"2020-08-07T15:13:24.998985+0000","last_active":"2020-08-07T15:14:04.608147+0000","last_peered":"2020-08-07T15:14:04.608147+0000","last_clean":"2020-08-07T15:14:04.608147+0000","last_became_active":"2020-08-07T15:12:32.246800+0000","last_became_peered":"2020-08-07T15:12:32.246800+0000","last_unstale":"2020-08-07T15:14:04.608147+0000","last_undegraded":"2020-08-07T15:14:04.608147+0000","last_fullsized":"2020-08-07T15:14:04.608147+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:24.998919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:24.998919+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"309'776","reported_seq":"793","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:06.575727+0000","last_change":"2020-08-07T15:13:57.266426+0000","last_active":"2020-08-07T15:14:06.575727+0000","last_peered":"2020-08-07T15:14:06.575727+0000","last_clean":"2020-08-07T15:14:06.575727+0000","last_became_active":"2020-08-07T15:13:57.265966+0000","last_became_peered":"2020-08-07T15:13:57.265966+0000","last_unstale":"2020-08-07T15:14:06.575727+0000","last_undegraded":"2020-08-07T15:14:06.575727+0000","last_fullsized":"2020-08-07T15:14:06.575727+0000","mapping_epoch":301,"log_start":"309'767","ondisk_log_start":"309'767","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6356992,"num_objects":97,"num_object_clones":0,"num_object_copies":291,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":776,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"282","reported_epoch":"308","state":"active+clean","last_fresh":"2020-08-07T15:14:03.621324+0000","last_change":"2020-08-07T15:13:41.820648+0000","last_active":"2020-08-07T15:14:03.621324+0000","last_peered":"2020-08-07T15:14:03.621324+0000","last_clean":"2020-08-07T15:14:03.621324+0000","last_became_active":"2020-08-07T15:13:02.590901+0000","last_became_peered":"2020-08-07T15:13:02.590901+0000","last_unstale":"2020-08-07T15:14:03.621324+0000","last_undegraded":"2020-08-07T15:14:03.621324+0000","last_fullsized":"2020-08-07T15:14:03.621324+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:41.820573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:22.062496+0000","last_clean_scrub_stamp":"2020-08-07T15:13:41.820573+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"16'2","reported_seq":"410","reported_epoch":"310","state":"active+clean","last_fresh":"2020-08-07T15:14:06.636950+0000","last_change":"2020-08-07T15:14:05.128167+0000","last_active":"2020-08-07T15:14:06.636950+0000","last_peered":"2020-08-07T15:14:06.636950+0000","last_clean":"2020-08-07T15:14:06.636950+0000","last_became_active":"2020-08-07T15:12:32.254842+0000","last_became_peered":"2020-08-07T15:12:32.254842+0000","last_unstale":"2020-08-07T15:14:06.636950+0000","last_undegraded":"2020-08-07T15:14:06.636950+0000","last_fullsized":"2020-08-07T15:14:06.636950+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":3,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:14:05.128109+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:43.350971+0000","last_clean_scrub_stamp":"2020-08-07T15:14:05.128109+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.c","version":"310'1016","reported_seq":"1033","reported_epoch":"311","state":"active+clean","last_fresh":"2020-08-07T15:14:07.648054+0000","last_change":"2020-08-07T15:13:57.266895+0000","last_active":"2020-08-07T15:14:07.648054+0000","last_peered":"2020-08-07T15:14:07.648054+0000","last_clean":"2020-08-07T15:14:07.648054+0000","last_became_active":"2020-08-07T15:13:57.265943+0000","last_became_peered":"2020-08-07T15:13:57.265943+0000","last_unstale":"2020-08-07T15:14:07.648054+0000","last_undegraded":"2020-08-07T15:14:07.648054+0000","last_fullsized":"2020-08-07T15:14:07.648054+0000","mapping_epoch":301,"log_start":"310'1012","ondisk_log_start":"310'1012","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8323072,"num_objects":127,"num_object_clones":0,"num_object_copies":381,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1016,"num_write_kb":8128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"16'2","reported_seq":"297","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.608235+0000","last_change":"2020-08-07T15:13:11.097271+0000","last_active":"2020-08-07T15:14:04.608235+0000","last_peered":"2020-08-07T15:14:04.608235+0000","last_clean":"2020-08-07T15:14:04.608235+0000","last_became_active":"2020-08-07T15:10:46.498230+0000","last_became_peered":"2020-08-07T15:10:46.498230+0000","last_unstale":"2020-08-07T15:14:04.608235+0000","last_undegraded":"2020-08-07T15:14:04.608235+0000","last_fullsized":"2020-08-07T15:14:04.608235+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:11.097214+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:10:10.715051+0000","last_clean_scrub_stamp":"2020-08-07T15:13:11.097214+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.e","version":"310'792","reported_seq":"807","reported_epoch":"310","state":"active+clean","last_fresh":"2020-08-07T15:14:06.756250+0000","last_change":"2020-08-07T15:13:57.264444+0000","last_active":"2020-08-07T15:14:06.756250+0000","last_peered":"2020-08-07T15:14:06.756250+0000","last_clean":"2020-08-07T15:14:06.756250+0000","last_became_active":"2020-08-07T15:13:57.263649+0000","last_became_peered":"2020-08-07T15:13:57.263649+0000","last_unstale":"2020-08-07T15:14:06.756250+0000","last_undegraded":"2020-08-07T15:14:06.756250+0000","last_fullsized":"2020-08-07T15:14:06.756250+0000","mapping_epoch":301,"log_start":"310'783","ondisk_log_start":"310'783","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6488064,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":6336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"16'2","reported_seq":"153","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.608034+0000","last_change":"2020-08-07T15:12:45.243933+0000","last_active":"2020-08-07T15:14:04.608034+0000","last_peered":"2020-08-07T15:14:04.608034+0000","last_clean":"2020-08-07T15:14:04.608034+0000","last_became_active":"2020-08-07T15:12:45.243349+0000","last_became_peered":"2020-08-07T15:12:45.243349+0000","last_unstale":"2020-08-07T15:14:04.608034+0000","last_undegraded":"2020-08-07T15:14:04.608034+0000","last_fullsized":"2020-08-07T15:14:04.608034+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:11:37.490655+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:09:09.691753+0000","last_clean_scrub_stamp":"2020-08-07T15:11:37.490655+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"310'792","reported_seq":"808","reported_epoch":"310","state":"active+clean","last_fresh":"2020-08-07T15:14:06.719244+0000","last_change":"2020-08-07T15:13:57.266272+0000","last_active":"2020-08-07T15:14:06.719244+0000","last_peered":"2020-08-07T15:14:06.719244+0000","last_clean":"2020-08-07T15:14:06.719244+0000","last_became_active":"2020-08-07T15:13:57.265797+0000","last_became_peered":"2020-08-07T15:13:57.265797+0000","last_unstale":"2020-08-07T15:14:06.719244+0000","last_undegraded":"2020-08-07T15:14:06.719244+0000","last_fullsized":"2020-08-07T15:14:06.719244+0000","mapping_epoch":301,"log_start":"310'784","ondisk_log_start":"310'784","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6488064,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":6336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"16'2","reported_seq":"115","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.615582+0000","last_change":"2020-08-07T15:13:10.415039+0000","last_active":"2020-08-07T15:14:04.615582+0000","last_peered":"2020-08-07T15:14:04.615582+0000","last_clean":"2020-08-07T15:14:04.615582+0000","last_became_active":"2020-08-07T15:13:02.590018+0000","last_became_peered":"2020-08-07T15:13:02.590018+0000","last_unstale":"2020-08-07T15:14:04.615582+0000","last_undegraded":"2020-08-07T15:14:04.615582+0000","last_fullsized":"2020-08-07T15:14:04.615582+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:10.414954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:10.414954+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"309'688","reported_seq":"703","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.865970+0000","last_change":"2020-08-07T15:13:57.257793+0000","last_active":"2020-08-07T15:14:04.865970+0000","last_peered":"2020-08-07T15:14:04.865970+0000","last_clean":"2020-08-07T15:14:04.865970+0000","last_became_active":"2020-08-07T15:13:57.257376+0000","last_became_peered":"2020-08-07T15:13:57.257376+0000","last_unstale":"2020-08-07T15:14:04.865970+0000","last_undegraded":"2020-08-07T15:14:04.865970+0000","last_fullsized":"2020-08-07T15:14:04.865970+0000","mapping_epoch":301,"log_start":"309'680","ondisk_log_start":"309'680","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5636096,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":688,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.d","version":"75'3","reported_seq":"331","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.606419+0000","last_change":"2020-08-07T15:13:38.089823+0000","last_active":"2020-08-07T15:14:04.606419+0000","last_peered":"2020-08-07T15:14:04.606419+0000","last_clean":"2020-08-07T15:14:04.606419+0000","last_became_active":"2020-08-07T15:12:06.380392+0000","last_became_peered":"2020-08-07T15:12:06.380392+0000","last_unstale":"2020-08-07T15:14:04.606419+0000","last_undegraded":"2020-08-07T15:14:04.606419+0000","last_fullsized":"2020-08-07T15:14:04.606419+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'3","last_scrub_stamp":"2020-08-07T15:13:38.089747+0000","last_deep_scrub":"75'3","last_deep_scrub_stamp":"2020-08-07T15:08:15.585597+0000","last_clean_scrub_stamp":"2020-08-07T15:13:38.089747+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"310'648","reported_seq":"664","reported_epoch":"310","state":"active+clean","last_fresh":"2020-08-07T15:14:06.757256+0000","last_change":"2020-08-07T15:13:57.263679+0000","last_active":"2020-08-07T15:14:06.757256+0000","last_peered":"2020-08-07T15:14:06.757256+0000","last_clean":"2020-08-07T15:14:06.757256+0000","last_became_active":"2020-08-07T15:13:57.263194+0000","last_became_peered":"2020-08-07T15:13:57.263194+0000","last_unstale":"2020-08-07T15:14:06.757256+0000","last_undegraded":"2020-08-07T15:14:06.757256+0000","last_fullsized":"2020-08-07T15:14:06.757256+0000","mapping_epoch":301,"log_start":"310'643","ondisk_log_start":"310'643","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5308416,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":648,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.c","version":"16'2","reported_seq":"286","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.614642+0000","last_change":"2020-08-07T15:14:01.614941+0000","last_active":"2020-08-07T15:14:04.614642+0000","last_peered":"2020-08-07T15:14:04.614642+0000","last_clean":"2020-08-07T15:14:04.614642+0000","last_became_active":"2020-08-07T15:10:46.498051+0000","last_became_peered":"2020-08-07T15:10:46.498051+0000","last_unstale":"2020-08-07T15:14:04.614642+0000","last_undegraded":"2020-08-07T15:14:04.614642+0000","last_fullsized":"2020-08-07T15:14:04.614642+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:14:01.614870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:14:01.614870+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"310'656","reported_seq":"672","reported_epoch":"310","state":"active+clean","last_fresh":"2020-08-07T15:14:06.694189+0000","last_change":"2020-08-07T15:13:57.263784+0000","last_active":"2020-08-07T15:14:06.694189+0000","last_peered":"2020-08-07T15:14:06.694189+0000","last_clean":"2020-08-07T15:14:06.694189+0000","last_became_active":"2020-08-07T15:13:57.263369+0000","last_became_peered":"2020-08-07T15:13:57.263369+0000","last_unstale":"2020-08-07T15:14:06.694189+0000","last_undegraded":"2020-08-07T15:14:06.694189+0000","last_fullsized":"2020-08-07T15:14:06.694189+0000","mapping_epoch":301,"log_start":"310'651","ondisk_log_start":"310'651","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5373952,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":656,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"16'2","reported_seq":"363","reported_epoch":"310","state":"active+clean","last_fresh":"2020-08-07T15:14:06.635695+0000","last_change":"2020-08-07T15:13:52.144004+0000","last_active":"2020-08-07T15:14:06.635695+0000","last_peered":"2020-08-07T15:14:06.635695+0000","last_clean":"2020-08-07T15:14:06.635695+0000","last_became_active":"2020-08-07T15:12:32.241722+0000","last_became_peered":"2020-08-07T15:12:32.241722+0000","last_unstale":"2020-08-07T15:14:06.635695+0000","last_undegraded":"2020-08-07T15:14:06.635695+0000","last_fullsized":"2020-08-07T15:14:06.635695+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:52.143910+0000","last_deep_scrub":"16'2","last_deep_scrub_stamp":"2020-08-07T15:12:13.176383+0000","last_clean_scrub_stamp":"2020-08-07T15:13:52.143910+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.8","version":"310'784","reported_seq":"801","reported_epoch":"310","state":"active+clean","last_fresh":"2020-08-07T15:14:06.726938+0000","last_change":"2020-08-07T15:13:57.264595+0000","last_active":"2020-08-07T15:14:06.726938+0000","last_peered":"2020-08-07T15:14:06.726938+0000","last_clean":"2020-08-07T15:14:06.726938+0000","last_became_active":"2020-08-07T15:13:57.264126+0000","last_became_peered":"2020-08-07T15:13:57.264126+0000","last_unstale":"2020-08-07T15:14:06.726938+0000","last_undegraded":"2020-08-07T15:14:06.726938+0000","last_fullsized":"2020-08-07T15:14:06.726938+0000","mapping_epoch":301,"log_start":"310'776","ondisk_log_start":"310'776","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":6422528,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":784,"num_write_kb":6272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.e","version":"16'2","reported_seq":"72","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.615121+0000","last_change":"2020-08-07T15:13:11.459633+0000","last_active":"2020-08-07T15:14:04.615121+0000","last_peered":"2020-08-07T15:14:04.615121+0000","last_clean":"2020-08-07T15:14:04.615121+0000","last_became_active":"2020-08-07T15:13:02.591018+0000","last_became_peered":"2020-08-07T15:13:02.591018+0000","last_unstale":"2020-08-07T15:14:04.615121+0000","last_undegraded":"2020-08-07T15:14:04.615121+0000","last_fullsized":"2020-08-07T15:14:04.615121+0000","mapping_epoch":269,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":270,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:13:11.459555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:13:11.459555+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"311'704","reported_seq":"722","reported_epoch":"311","state":"active+clean","last_fresh":"2020-08-07T15:14:07.690857+0000","last_change":"2020-08-07T15:13:57.260649+0000","last_active":"2020-08-07T15:14:07.690857+0000","last_peered":"2020-08-07T15:14:07.690857+0000","last_clean":"2020-08-07T15:14:07.690857+0000","last_became_active":"2020-08-07T15:13:57.260154+0000","last_became_peered":"2020-08-07T15:13:57.260154+0000","last_unstale":"2020-08-07T15:14:07.690857+0000","last_undegraded":"2020-08-07T15:14:07.690857+0000","last_fullsized":"2020-08-07T15:14:07.690857+0000","mapping_epoch":301,"log_start":"311'699","ondisk_log_start":"311'699","created":301,"last_epoch_clean":302,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:13:56.516830+0000","last_clean_scrub_stamp":"2020-08-07T15:13:56.516830+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5767168,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":5632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"16'2","reported_seq":"281","reported_epoch":"309","state":"active+clean","last_fresh":"2020-08-07T15:14:04.608370+0000","last_change":"2020-08-07T15:12:44.203274+0000","last_active":"2020-08-07T15:14:04.608370+0000","last_peered":"2020-08-07T15:14:04.608370+0000","last_clean":"2020-08-07T15:14:04.608370+0000","last_became_active":"2020-08-07T15:12:06.367081+0000","last_became_peered":"2020-08-07T15:12:06.367081+0000","last_unstale":"2020-08-07T15:14:04.608370+0000","last_undegraded":"2020-08-07T15:14:04.608370+0000","last_fullsized":"2020-08-07T15:14:04.608370+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":4,"last_scrub":"16'2","last_scrub_stamp":"2020-08-07T15:12:44.203221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-08-07T15:06:10.878833+0000","last_clean_scrub_stamp":"2020-08-07T15:12:44.203221+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":96878592,"num_objects":1479,"num_object_clones":0,"num_object_copies":4437,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11826,"num_write_kb":94608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290021376,"data_stored":290021376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":48,"acting":48,"num_store_stats":8},{"poolid":1,"num_pg":41,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":1,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":123,"acting":123,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8}],"osd_stats":[{"osd":6,"up_from":14,"seq":60129542240,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1082784,"kb_used_data":34200,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93289056,"statfs":{"total":96636764160,"available":95527993344,"internally_reserved":1073741824,"allocated":35020800,"data_stored":34404007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":36.792000000000002,"5min":16.657,"15min":6.1669999999999998},"min":{"1min":0.498,"5min":0.371,"15min":0.371},"max":{"1min":753.82299999999998,"5min":856.07100000000003,"15min":856.07100000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":5.7770000000000001,"5min":1.9390000000000001,"15min":1.2969999999999999},"min":{"1min":0.47099999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":103.30200000000001,"5min":103.30200000000001,"15min":103.30200000000001},"last":0.67500000000000004}]},{"osd":1,"last update":"Fri Aug 7 15:13:10 2020","interfaces":[{"interface":"back","average":{"1min":10.148999999999999,"5min":10.148999999999999,"15min":10.148999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":205.73699999999999,"5min":205.73699999999999,"15min":205.73699999999999},"last":1.165},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":3.827,"5min":3.827,"15min":3.827},"last":0.88700000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":15.460000000000001,"5min":4.1349999999999998,"15min":2.1469999999999998},"min":{"1min":0.63900000000000001,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":297.721,"5min":297.721,"15min":297.721},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.248,"15min":1.1859999999999999},"min":{"1min":0.66500000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.7809999999999999,"5min":5.1319999999999997,"15min":5.1319999999999997},"last":0.63500000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":3.081,"5min":22.274000000000001,"15min":8.5120000000000005},"min":{"1min":0.60699999999999998,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":32.908000000000001,"5min":917.38699999999994,"15min":917.38699999999994},"last":1.008},{"interface":"front","average":{"1min":3.1760000000000002,"5min":5.2690000000000001,"15min":2.6469999999999998},"min":{"1min":0.69499999999999995,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":32.881999999999998,"5min":333.59199999999998,"15min":333.59199999999998},"last":0.71599999999999997}]},{"osd":4,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":42.588999999999999,"5min":11.105,"15min":4.5860000000000003},"min":{"1min":0.71399999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":873.673,"5min":873.673,"15min":873.673},"last":0.61699999999999999},{"interface":"front","average":{"1min":1.0660000000000001,"5min":11.461,"15min":4.7009999999999996},"min":{"1min":0.71299999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.571,"5min":866.24099999999999,"15min":866.24099999999999},"last":0.74199999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":9.1910000000000007,"15min":25.727},"min":{"1min":0.66200000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":3.234,"5min":678.76099999999997,"15min":831.88099999999997},"last":2.0569999999999999},{"interface":"front","average":{"1min":1.538,"5min":1.647,"15min":1.6100000000000001},"min":{"1min":0.85299999999999998,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":3.4340000000000002,"5min":6.6790000000000003,"15min":7.5350000000000001},"last":2.0910000000000002}]},{"osd":7,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":3.9100000000000001,"15min":2.323},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":2.1840000000000002,"5min":224.14500000000001,"15min":224.14500000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.512,"5min":1.6830000000000001,"15min":1.593},"min":{"1min":0.91900000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":3.4049999999999998,"5min":7.4359999999999999,"15min":7.4359999999999999},"last":1.1950000000000001}]}]},{"osd":7,"up_from":13,"seq":55834574945,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1078228,"kb_used_data":29644,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93293612,"statfs":{"total":96636764160,"available":95532658688,"internally_reserved":1073741824,"allocated":30355456,"data_stored":29746076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":16.684999999999999,"15min":6.4219999999999997},"min":{"1min":0.61099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.95,"5min":763.39099999999996,"15min":763.39099999999996},"last":0.625},{"interface":"front","average":{"1min":1.026,"5min":9.7690000000000001,"15min":4.0179999999999998},"min":{"1min":0.57399999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.9970000000000001,"5min":529.88699999999994,"15min":529.88699999999994},"last":1.736}]},{"osd":1,"last update":"Fri Aug 7 15:14:05 2020","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":18.940000000000001,"15min":21.934000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.7090000000000001,"5min":472.59699999999998,"15min":472.59699999999998},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.54,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.198}]},{"osd":2,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":1.677,"5min":1.776,"15min":1.3899999999999999},"min":{"1min":0.53400000000000003,"5min":0.502,"15min":0.49299999999999999},"max":{"1min":5.806,"5min":33.131999999999998,"15min":33.131999999999998},"last":2.415},{"interface":"front","average":{"1min":1.667,"5min":1.6080000000000001,"15min":1.3720000000000001},"min":{"1min":0.70099999999999996,"5min":0.58599999999999997,"15min":0.58399999999999996},"max":{"1min":5.3170000000000002,"5min":32.924999999999997,"15min":32.924999999999997},"last":2.25}]},{"osd":3,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":9.4909999999999997,"15min":4.1680000000000001},"min":{"1min":0.68200000000000005,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":5.7030000000000003,"5min":698.19899999999996,"15min":698.19899999999996},"last":2.3039999999999998},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.3,"15min":1.3500000000000001},"min":{"1min":0.69699999999999995,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.2909999999999999,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":2.222}]},{"osd":4,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":6.0369999999999999,"15min":3.0960000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.821,"5min":460.78399999999999,"15min":460.78399999999999},"last":1.6519999999999999},{"interface":"front","average":{"1min":38.359000000000002,"5min":8.8149999999999995,"15min":4.0510000000000002},"min":{"1min":0.80900000000000005,"5min":0.621,"15min":0.621},"max":{"1min":668.06799999999998,"5min":668.06799999999998,"15min":668.06799999999998},"last":1.6719999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.367,"15min":1.3520000000000001},"min":{"1min":0.67900000000000005,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.0489999999999999,"5min":6.2080000000000002,"15min":6.2080000000000002},"last":2.2709999999999999},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.47,"15min":1.6419999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.6160000000000001,"5min":4.407,"15min":5.327},"last":0.95199999999999996}]},{"osd":6,"last update":"Fri Aug 7 15:13:31 2020","interfaces":[{"interface":"back","average":{"1min":3.5600000000000001,"5min":9.5960000000000001,"15min":4.1580000000000004},"min":{"1min":0.872,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":40.079999999999998,"5min":695.92899999999997,"15min":695.92899999999997},"last":0.98399999999999999},{"interface":"front","average":{"1min":3.6760000000000002,"5min":1.9910000000000001,"15min":11.757},"min":{"1min":0.91900000000000004,"5min":0.88700000000000001,"15min":0.87},"max":{"1min":41.655999999999999,"5min":41.655999999999999,"15min":266.26999999999998},"last":1.2310000000000001}]}]},{"osd":1,"up_from":227,"seq":974957576217,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1072904,"kb_used_data":24320,"kb_used_omap":2,"kb_used_meta":1048573,"kb_avail":93298936,"statfs":{"total":96636764160,"available":95538110464,"internally_reserved":1073741824,"allocated":24903680,"data_stored":24301763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2579,"internal_metadata":1073739245},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":27.861000000000001,"5min":27.861000000000001,"15min":27.861000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":540.92499999999995,"5min":540.92499999999995,"15min":540.92499999999995},"last":1.867},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":3.7069999999999999,"5min":3.7069999999999999,"15min":3.7069999999999999},"last":0.52400000000000002}]},{"osd":2,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":3.1339999999999999,"5min":3.1339999999999999,"15min":3.1339999999999999},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":3.758,"5min":3.758,"15min":3.758},"last":1.026}]},{"osd":3,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":13.909000000000001,"5min":13.909000000000001,"15min":13.909000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":250.995,"5min":250.995,"15min":250.995},"last":0.879},{"interface":"front","average":{"1min":2.04,"5min":2.04,"15min":2.04},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":6.6580000000000004,"5min":6.6580000000000004,"15min":6.6580000000000004},"last":2.52}]},{"osd":4,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":0.60199999999999998},{"interface":"front","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":0.73599999999999999}]},{"osd":5,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.498,"5min":1.498,"15min":1.498},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.6720000000000002,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.292}]},{"osd":6,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.1019999999999999},"min":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":9.4299999999999997,"5min":9.4299999999999997,"15min":9.4299999999999997},"last":259.053},{"interface":"front","average":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":3.4199999999999999,"5min":3.4199999999999999,"15min":3.4199999999999999},"last":0.91100000000000003}]},{"osd":7,"last update":"Fri Aug 7 15:13:06 2020","interfaces":[{"interface":"back","average":{"1min":40.520000000000003,"5min":40.520000000000003,"15min":40.520000000000003},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":778.745,"5min":778.745,"15min":778.745},"last":1.1539999999999999},{"interface":"front","average":{"1min":24.100000000000001,"5min":24.100000000000001,"15min":24.100000000000001},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":442.62799999999999,"5min":442.62799999999999,"15min":442.62799999999999},"last":1.8029999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574945,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1108740,"kb_used_data":60156,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93263100,"statfs":{"total":96636764160,"available":95501414400,"internally_reserved":1073741824,"allocated":61599744,"data_stored":60990364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 7 15:13:07 2020","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":2.468,"5min":2.468,"15min":2.468},"last":1.129},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":2.5550000000000002,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.171}]},{"osd":2,"last update":"Fri Aug 7 15:13:36 2020","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.0489999999999999,"15min":0.99399999999999999},"min":{"1min":0.47199999999999998,"5min":0.441,"15min":0.441},"max":{"1min":3.6099999999999999,"5min":7.3470000000000004,"15min":7.3470000000000004},"last":1.087},{"interface":"front","average":{"1min":1.742,"5min":4.8120000000000003,"15min":21.315000000000001},"min":{"1min":0.64800000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.4909999999999997,"5min":226.911,"15min":606.18499999999995},"last":0.76500000000000001}]},{"osd":3,"last update":"Fri Aug 7 15:13:36 2020","interfaces":[{"interface":"back","average":{"1min":1.615,"5min":6.7089999999999996,"15min":2.8999999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":4.1619999999999999,"5min":431.95299999999997,"15min":431.95299999999997},"last":5.3970000000000002},{"interface":"front","average":{"1min":1.6359999999999999,"5min":1.4019999999999999,"15min":1.296},"min":{"1min":0.73099999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":4.3449999999999998,"5min":11.19,"15min":11.19},"last":0.80500000000000005}]},{"osd":4,"last update":"Fri Aug 7 15:13:36 2020","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.25,"15min":1.218},"min":{"1min":0.629,"5min":0.501,"15min":0.501},"max":{"1min":2.7610000000000001,"5min":8.3100000000000005,"15min":8.3100000000000005},"last":5.4379999999999997},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.407,"15min":1.3360000000000001},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":3.5600000000000001,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":0.85699999999999998}]},{"osd":5,"last update":"Fri Aug 7 15:13:36 2020","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":6.9930000000000003,"15min":5.702},"min":{"1min":0.76800000000000002,"5min":0.62,"15min":0.62},"max":{"1min":2.7400000000000002,"5min":539.31100000000004,"15min":539.31100000000004},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3979999999999999,"15min":1.248},"min":{"1min":0.78700000000000003,"5min":0.752,"15min":0.65900000000000003},"max":{"1min":2.714,"5min":4.0140000000000002,"15min":6.0010000000000003},"last":5.4690000000000003}]},{"osd":6,"last update":"Fri Aug 7 15:13:36 2020","interfaces":[{"interface":"back","average":{"1min":1.5900000000000001,"5min":1.403,"15min":5.7539999999999996},"min":{"1min":0.90500000000000003,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":3.585,"5min":3.585,"15min":124.738},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5700000000000001,"15min":11.813000000000001},"min":{"1min":0.79200000000000004,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.867,"5min":5.274,"15min":294.94},"last":1.2050000000000001}]},{"osd":7,"last update":"Fri Aug 7 15:13:36 2020","interfaces":[{"interface":"back","average":{"1min":1.6970000000000001,"5min":1.5349999999999999,"15min":1.5569999999999999},"min":{"1min":0.995,"5min":0.84199999999999997,"15min":0.79300000000000004},"max":{"1min":2.9529999999999998,"5min":4.7169999999999996,"15min":6.468},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.74,"5min":23.527999999999999,"15min":8.7870000000000008},"min":{"1min":1.097,"5min":0.752,"15min":0.752},"max":{"1min":2.9220000000000002,"5min":971.12699999999995,"15min":971.12699999999995},"last":1.2629999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574945,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1079596,"kb_used_data":31012,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93292244,"statfs":{"total":96636764160,"available":95531257856,"internally_reserved":1073741824,"allocated":31756288,"data_stored":31146908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:28 2020","interfaces":[{"interface":"back","average":{"1min":18.065999999999999,"5min":11.218,"15min":4.4960000000000004},"min":{"1min":0.441,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":377.32900000000001,"5min":501.53800000000001,"15min":501.53800000000001},"last":0.443},{"interface":"front","average":{"1min":1.0760000000000001,"5min":6.649,"15min":2.8860000000000001},"min":{"1min":0.46800000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":3.0139999999999998,"5min":501.36000000000001,"15min":501.36000000000001},"last":0.51800000000000002}]},{"osd":1,"last update":"Fri Aug 7 15:13:07 2020","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":5.7869999999999999,"5min":5.7869999999999999,"15min":5.7869999999999999},"last":1.4399999999999999},{"interface":"front","average":{"1min":24.699999999999999,"5min":24.699999999999999,"15min":24.699999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":465.69200000000001,"5min":465.69200000000001,"15min":465.69200000000001},"last":1.117}]},{"osd":3,"last update":"Fri Aug 7 15:13:28 2020","interfaces":[{"interface":"back","average":{"1min":3.847,"5min":1.696,"15min":1.3580000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":29.111000000000001,"5min":29.111000000000001,"15min":29.111000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":3.3849999999999998,"5min":15.673999999999999,"15min":6.2530000000000001},"min":{"1min":0.627,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":29.788,"5min":501.75599999999997,"15min":501.75599999999997},"last":0.77700000000000002}]},{"osd":4,"last update":"Fri Aug 7 15:13:29 2020","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.397,"15min":1.2869999999999999},"min":{"1min":0.73499999999999999,"5min":0.623,"15min":0.58099999999999996},"max":{"1min":2.5489999999999999,"5min":7.9569999999999999,"15min":19.783000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":44.813000000000002,"5min":10.178000000000001,"15min":4.46},"min":{"1min":0.84299999999999997,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":954.97900000000004,"5min":954.97900000000004,"15min":954.97900000000004},"last":0.83199999999999996}]},{"osd":5,"last update":"Fri Aug 7 15:13:28 2020","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.3420000000000001,"15min":1.294},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.992,"5min":5.2220000000000004,"15min":5.2220000000000004},"last":1.294},{"interface":"front","average":{"1min":45.744999999999997,"5min":24.559000000000001,"15min":23.603000000000002},"min":{"1min":0.57099999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":971.51400000000001,"5min":971.51400000000001,"15min":971.51400000000001},"last":0.80800000000000005}]},{"osd":6,"last update":"Fri Aug 7 15:13:39 2020","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":9.766,"15min":4.1550000000000002},"min":{"1min":0.872,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.6480000000000001,"5min":821.91899999999998,"15min":821.91899999999998},"last":1.145},{"interface":"front","average":{"1min":25.716999999999999,"5min":6.2949999999999999,"15min":5.8259999999999996},"min":{"1min":0.83799999999999997,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":477.69600000000003,"5min":477.69600000000003,"15min":751.70000000000005},"last":1.323}]},{"osd":7,"last update":"Fri Aug 7 15:13:28 2020","interfaces":[{"interface":"back","average":{"1min":43.765999999999998,"5min":20.567,"15min":8.1370000000000005},"min":{"1min":0.89200000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":930.55499999999995,"5min":930.55499999999995,"15min":930.55499999999995},"last":1.169},{"interface":"front","average":{"1min":1.8560000000000001,"5min":1.79,"15min":1.724},"min":{"1min":0.98999999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":5.6950000000000003,"5min":8.0779999999999994,"15min":35.646999999999998},"last":1.266}]}]},{"osd":3,"up_from":13,"seq":55834574945,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1101884,"kb_used_data":53300,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93269956,"statfs":{"total":96636764160,"available":95508434944,"internally_reserved":1073741824,"allocated":54579200,"data_stored":53969820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:38 2020","interfaces":[{"interface":"back","average":{"1min":53.417000000000002,"5min":11.500999999999999,"15min":4.7210000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":929.68899999999996,"5min":929.68899999999996,"15min":929.68899999999996},"last":2.3149999999999999},{"interface":"front","average":{"1min":42.985999999999997,"5min":9.5129999999999999,"15min":4.2489999999999997},"min":{"1min":0.496,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":744.76800000000003,"5min":744.76800000000003,"15min":744.76800000000003},"last":2.3399999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:13:08 2020","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.639,"15min":1.639},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":4.6890000000000001,"5min":4.6890000000000001,"15min":4.6890000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.841,"5min":1.841,"15min":1.841},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":5.0330000000000004,"5min":5.0330000000000004,"15min":5.0330000000000004},"last":1.1060000000000001}]},{"osd":2,"last update":"Fri Aug 7 15:13:38 2020","interfaces":[{"interface":"back","average":{"1min":42.640000000000001,"5min":9.5429999999999993,"15min":3.9860000000000002},"min":{"1min":0.78800000000000003,"5min":0.56999999999999995,"15min":0.56899999999999995},"max":{"1min":810.50999999999999,"5min":810.50999999999999,"15min":810.50999999999999},"last":1.147},{"interface":"front","average":{"1min":1.536,"5min":1.3720000000000001,"15min":1.77},"min":{"1min":0.66100000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.9079999999999999,"5min":6.7889999999999997,"15min":51.530000000000001},"last":1.2070000000000001}]},{"osd":4,"last update":"Fri Aug 7 15:13:38 2020","interfaces":[{"interface":"back","average":{"1min":13.329000000000001,"5min":10.928000000000001,"15min":7.8369999999999997},"min":{"1min":0.66600000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":234.19999999999999,"5min":646.26700000000005,"15min":646.26700000000005},"last":3.0819999999999999},{"interface":"front","average":{"1min":28.314,"5min":15.204000000000001,"15min":26.312000000000001},"min":{"1min":0.82099999999999995,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":519.82600000000002,"5min":844.10500000000002,"15min":844.10500000000002},"last":2.2559999999999998}]},{"osd":5,"last update":"Fri Aug 7 15:13:38 2020","interfaces":[{"interface":"back","average":{"1min":2.5899999999999999,"5min":8.6590000000000007,"15min":3.931},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":5.5170000000000003,"5min":403.88400000000001,"15min":403.88400000000001},"last":2.2890000000000001},{"interface":"front","average":{"1min":2.6150000000000002,"5min":1.976,"15min":1.8680000000000001},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":5.5430000000000001,"5min":8.9120000000000008,"15min":8.9120000000000008},"last":2.7240000000000002}]},{"osd":6,"last update":"Fri Aug 7 15:13:38 2020","interfaces":[{"interface":"back","average":{"1min":1.7070000000000001,"5min":1.6499999999999999,"15min":1.601},"min":{"1min":0.76500000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":3.508,"5min":5.2729999999999997,"15min":5.2729999999999997},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.9139999999999999,"5min":13.398999999999999,"15min":5.79},"min":{"1min":0.97799999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":3.524,"5min":688.15999999999997,"15min":688.15999999999997},"last":1.5249999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:13:38 2020","interfaces":[{"interface":"back","average":{"1min":1.7250000000000001,"5min":1.5389999999999999,"15min":1.944},"min":{"1min":0.96999999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":3.6120000000000001,"5min":6.8209999999999997,"15min":6.8209999999999997},"last":1.427},{"interface":"front","average":{"1min":2.3050000000000002,"5min":1.593,"15min":1.891},"min":{"1min":1.0469999999999999,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":7.0090000000000003,"5min":7.0090000000000003,"15min":8.6790000000000003},"last":3.25}]}]},{"osd":4,"up_from":13,"seq":55834574945,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1078348,"kb_used_data":29764,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93293492,"statfs":{"total":96636764160,"available":95532535808,"internally_reserved":1073741824,"allocated":30478336,"data_stored":29868956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:30 2020","interfaces":[{"interface":"back","average":{"1min":3.0019999999999998,"5min":20.065999999999999,"15min":7.4569999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":37.307000000000002,"5min":829.15099999999995,"15min":829.15099999999995},"last":0.85599999999999998},{"interface":"front","average":{"1min":3.242,"5min":2.0179999999999998,"15min":1.272},"min":{"1min":0.499,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":42.095999999999997,"5min":48.274000000000001,"15min":48.274000000000001},"last":1.5489999999999999}]},{"osd":1,"last update":"Fri Aug 7 15:13:08 2020","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.075},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.95399999999999996}]},{"osd":2,"last update":"Fri Aug 7 15:13:30 2020","interfaces":[{"interface":"back","average":{"1min":3.718,"5min":2.3460000000000001,"15min":1.752},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.52100000000000002},"max":{"1min":37.365000000000002,"5min":49.878999999999998,"15min":49.878999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":12.497999999999999,"5min":4.1970000000000001,"15min":2.4260000000000002},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":146.34399999999999,"5min":146.34399999999999,"15min":146.34399999999999},"last":0.93400000000000005}]},{"osd":3,"last update":"Fri Aug 7 15:13:30 2020","interfaces":[{"interface":"back","average":{"1min":3.4969999999999999,"5min":15.436999999999999,"15min":6.891},"min":{"1min":0.63100000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":37.582000000000001,"5min":638.92100000000005,"15min":638.92100000000005},"last":0.97699999999999998},{"interface":"front","average":{"1min":3.4500000000000002,"5min":1.754,"15min":1.4399999999999999},"min":{"1min":0.72299999999999998,"5min":0.68500000000000005,"15min":0.66700000000000004},"max":{"1min":37.956000000000003,"5min":37.956000000000003,"15min":37.956000000000003},"last":0.84499999999999997}]},{"osd":5,"last update":"Fri Aug 7 15:13:30 2020","interfaces":[{"interface":"back","average":{"1min":3.6819999999999999,"5min":19.954000000000001,"15min":10.747},"min":{"1min":0.69999999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":43.692,"5min":862.97299999999996,"15min":990.71600000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":40.039000000000001,"5min":18.669,"15min":7.3150000000000004},"min":{"1min":0.58899999999999997,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":625.35000000000002,"5min":780.00699999999995,"15min":780.00699999999995},"last":0.70299999999999996}]},{"osd":6,"last update":"Fri Aug 7 15:13:38 2020","interfaces":[{"interface":"back","average":{"1min":3.3999999999999999,"5min":2.3260000000000001,"15min":1.9630000000000001},"min":{"1min":0.70799999999999996,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":41.585999999999999,"5min":48.575000000000003,"15min":48.575000000000003},"last":0.82699999999999996},{"interface":"front","average":{"1min":3.1400000000000001,"5min":1.657,"15min":12.119},"min":{"1min":0.69399999999999995,"5min":0.64600000000000002,"15min":0.59599999999999997},"max":{"1min":37.991,"5min":37.991,"15min":389.14600000000002},"last":0.871}]},{"osd":7,"last update":"Fri Aug 7 15:13:30 2020","interfaces":[{"interface":"back","average":{"1min":3.8620000000000001,"5min":2.4239999999999999,"15min":2.544},"min":{"1min":1.1319999999999999,"5min":0.62,"15min":0.62},"max":{"1min":38.979999999999997,"5min":48.75,"15min":252.798},"last":1.1679999999999999},{"interface":"front","average":{"1min":3.7989999999999999,"5min":6.46,"15min":6.1189999999999998},"min":{"1min":0.91300000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":38.899999999999999,"5min":326.327,"15min":501.93000000000001},"last":1.151}]}]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1071692,"kb_used_data":23108,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93300148,"statfs":{"total":96636764160,"available":95539351552,"internally_reserved":1073741824,"allocated":23662592,"data_stored":23053212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 7 15:13:27 2020","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":1.4770000000000001,"15min":1.204},"min":{"1min":0.45700000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.7070000000000001,"5min":43.292999999999999,"15min":43.292999999999999},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.03,"5min":4.3860000000000001,"15min":2.0259999999999998},"min":{"1min":0.47399999999999998,"5min":0.438,"15min":0.438},"max":{"1min":2.6059999999999999,"5min":329.11399999999998,"15min":329.11399999999998},"last":0.55800000000000005}]},{"osd":1,"last update":"Fri Aug 7 15:13:11 2020","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":4.4039999999999999,"5min":4.4039999999999999,"15min":4.4039999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":4.0810000000000004,"5min":4.0810000000000004,"15min":4.0810000000000004},"last":0.85399999999999998}]},{"osd":2,"last update":"Fri Aug 7 15:13:27 2020","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.6100000000000001,"15min":6.1319999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.4129999999999998,"5min":43.923000000000002,"15min":168.56899999999999},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.615,"15min":20.503},"min":{"1min":0.70499999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.5750000000000002,"5min":43.076999999999998,"15min":676.27099999999996},"last":0.59299999999999997}]},{"osd":3,"last update":"Fri Aug 7 15:13:27 2020","interfaces":[{"interface":"back","average":{"1min":4.1559999999999997,"5min":2.2570000000000001,"15min":1.5229999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":47.371000000000002,"5min":47.371000000000002,"15min":47.371000000000002},"last":1.373},{"interface":"front","average":{"1min":4.0990000000000002,"5min":2.2349999999999999,"15min":4.3440000000000003},"min":{"1min":0.72899999999999998,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":48.338999999999999,"5min":48.338999999999999,"15min":712.98699999999997},"last":1.1439999999999999}]},{"osd":4,"last update":"Fri Aug 7 15:13:27 2020","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":12.773,"15min":34.875999999999998},"min":{"1min":0.68400000000000005,"5min":0.54800000000000004,"15min":0.53400000000000003},"max":{"1min":1.9139999999999999,"5min":971.67999999999995,"15min":971.67999999999995},"last":1.111},{"interface":"front","average":{"1min":16.669,"5min":4.3479999999999999,"15min":2.3100000000000001},"min":{"1min":0.78300000000000003,"5min":0.61899999999999999,"15min":0.57099999999999995},"max":{"1min":261.08499999999998,"5min":261.08499999999998,"15min":261.08499999999998},"last":0.754}]},{"osd":6,"last update":"Fri Aug 7 15:13:27 2020","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":28.48,"15min":39.819000000000003},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.4940000000000002,"5min":966.60900000000004,"15min":966.60900000000004},"last":0.871},{"interface":"front","average":{"1min":1.3200000000000001,"5min":6.8049999999999997,"15min":17.957999999999998},"min":{"1min":0.71199999999999997,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.9450000000000001,"5min":501.79000000000002,"15min":517.26700000000005},"last":1.1279999999999999}]},{"osd":7,"last update":"Fri Aug 7 15:13:27 2020","interfaces":[{"interface":"back","average":{"1min":13.619999999999999,"5min":3.8769999999999998,"15min":2.1800000000000002},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":208.03399999999999,"5min":208.03399999999999,"15min":208.03399999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.4470000000000001,"5min":12.840999999999999,"15min":5.3140000000000001},"min":{"1min":0.89400000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.7069999999999999,"5min":515.09100000000001,"15min":515.09100000000001},"last":0.73599999999999999}]}]}]}} 2020-08-07T15:14:09.400 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2020-08-07T15:14:09.401 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2020-08-07T15:14:09.402 INFO:tasks.thrashosds.thrasher:Removing osd 7, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2020-08-07T15:14:09.402 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd out 7 2020-08-07T15:14:09.411 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:14:09.412 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [], 2020-08-07T15:14:09.412 INFO:teuthology.orchestra.run.smithi074.stdout: "complaint_time": 30, 2020-08-07T15:14:09.412 INFO:teuthology.orchestra.run.smithi074.stdout: "num_blocked_ops": 0 2020-08-07T15:14:09.413 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:14:09.426 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2020-08-07T15:14:09.479 INFO:tasks.ceph.osd.7.smithi096.stderr:2020-08-07T15:14:09.478+0000 7f4e87560700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 31543) UID: 0 2020-08-07T15:14:09.581 INFO:tasks.ceph.osd.2.smithi074.stderr:2020-08-07T15:14:09.580+0000 7f69355db700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31571) UID: 0 2020-08-07T15:14:09.622 INFO:tasks.radosbench.radosbench.0.smithi074.stdout: 11 16 15933 15917 11.3031 11.3359 0.00639955 0.0110436 2020-08-07T15:14:09.669 INFO:teuthology.orchestra.run.smithi074.stdout:{ 2020-08-07T15:14:09.669 INFO:teuthology.orchestra.run.smithi074.stdout: "size": 20, 2020-08-07T15:14:09.670 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 600, 2020-08-07T15:14:09.670 INFO:teuthology.orchestra.run.smithi074.stdout: "ops": [ 2020-08-07T15:14:09.670 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.670 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4329.0:106240 3.7 3:ebdd8242:::benchmark_data_smithi074_35447_object13279:head [set-alloc-hint object_size 65536 write_size 8192,write 57344~8192] snapc 0=[] ondisk+write+known_if_redirected e59)", 2020-08-07T15:14:09.671 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:14:09.671 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 382.31014637200002, 2020-08-07T15:14:09.671 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.90079396, 2020-08-07T15:14:09.671 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:09.671 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:09.672 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:09.672 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4329", 2020-08-07T15:14:09.672 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/2100701048", 2020-08-07T15:14:09.672 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 106240 2020-08-07T15:14:09.672 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.673 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:09.673 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.673 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:09.673 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:14:09.673 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:09.674 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.674 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.674 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:09.675 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355066+0000", 2020-08-07T15:14:09.675 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2522000000000003e-05 2020-08-07T15:14:09.675 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.676 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.676 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:09.676 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355118+0000", 2020-08-07T15:14:09.677 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6159999999999998e-06 2020-08-07T15:14:09.677 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.677 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.678 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:09.678 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355125+0000", 2020-08-07T15:14:09.678 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2699e-05 2020-08-07T15:14:09.678 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.679 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.679 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:09.679 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355138+0000", 2020-08-07T15:14:09.679 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2310000000000001e-06 2020-08-07T15:14:09.680 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.680 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.680 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:09.680 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355144+0000", 2020-08-07T15:14:09.681 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9855000000000003e-05 2020-08-07T15:14:09.681 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.681 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.681 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.681 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355194+0000", 2020-08-07T15:14:09.682 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000112713 2020-08-07T15:14:09.682 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.682 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.682 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:09.683 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355306+0000", 2020-08-07T15:14:09.683 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00019794600000000001 2020-08-07T15:14:09.683 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.684 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.684 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:14:09.684 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.355504+0000", 2020-08-07T15:14:09.684 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0048346029999999998 2020-08-07T15:14:09.685 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.685 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.685 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:09.686 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.360339+0000", 2020-08-07T15:14:09.686 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00033646400000000002 2020-08-07T15:14:09.686 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.686 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.687 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:14:09.687 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:47.360675+0000", 2020-08-07T15:14:09.687 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.89508481200000001 2020-08-07T15:14:09.687 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.688 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.688 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:09.688 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255760+0000", 2020-08-07T15:14:09.688 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5659999999999999e-06 2020-08-07T15:14:09.689 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.689 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.689 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:14:09.689 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255762+0000", 2020-08-07T15:14:09.690 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6595000000000003e-05 2020-08-07T15:14:09.690 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.690 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.691 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:14:09.691 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255818+0000", 2020-08-07T15:14:09.691 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1338000000000003e-05 2020-08-07T15:14:09.691 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.692 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.692 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:09.692 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:07:48.255860+0000", 2020-08-07T15:14:09.692 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.90079396 2020-08-07T15:14:09.693 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.693 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:09.693 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.694 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.694 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.694 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37057 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:14:09.695 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:14:09.695 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 315.42164171000002, 2020-08-07T15:14:09.695 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007248691, 2020-08-07T15:14:09.695 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:09.696 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:14:09.696 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:09.696 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:09.696 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:09.697 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37057 2020-08-07T15:14:09.697 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.697 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:09.697 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.698 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:09.698 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:14:09.698 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:09.699 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.699 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.699 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:09.700 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243570+0000", 2020-08-07T15:14:09.700 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3661e-05 2020-08-07T15:14:09.700 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.700 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.700 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:09.701 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243624+0000", 2020-08-07T15:14:09.701 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.7519999999999996e-06 2020-08-07T15:14:09.701 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.702 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.702 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:09.702 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243632+0000", 2020-08-07T15:14:09.702 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6392000000000002e-05 2020-08-07T15:14:09.703 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.703 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.703 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:09.704 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243648+0000", 2020-08-07T15:14:09.704 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.6659999999999997e-06 2020-08-07T15:14:09.704 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.704 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.705 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:09.705 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243657+0000", 2020-08-07T15:14:09.705 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0070484609999999998 2020-08-07T15:14:09.705 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.706 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.706 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.706 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250705+0000", 2020-08-07T15:14:09.706 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0083e-05 2020-08-07T15:14:09.707 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.707 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.707 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:09.707 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250715+0000", 2020-08-07T15:14:09.708 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000086695 2020-08-07T15:14:09.708 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.709 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.709 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.710 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.250802+0000", 2020-08-07T15:14:09.710 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6980999999999999e-05 2020-08-07T15:14:09.710 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.710 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.710 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:09.711 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.250819+0000", 2020-08-07T15:14:09.711 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007248691 2020-08-07T15:14:09.711 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.711 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:09.711 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.712 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.712 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.712 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37058 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:14:09.712 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:14:09.712 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 315.42149997600001, 2020-08-07T15:14:09.713 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007368357, 2020-08-07T15:14:09.713 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:09.713 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:14:09.713 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:09.714 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:09.714 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:09.715 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37058 2020-08-07T15:14:09.715 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.715 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:09.715 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.717 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:09.717 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:14:09.717 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:09.717 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.717 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.718 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:09.718 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243712+0000", 2020-08-07T15:14:09.718 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8838999999999998e-05 2020-08-07T15:14:09.718 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.719 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.719 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:09.719 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243761+0000", 2020-08-07T15:14:09.719 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2079999999999999e-06 2020-08-07T15:14:09.719 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.720 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.721 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:09.721 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243766+0000", 2020-08-07T15:14:09.721 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0567999999999999e-05 2020-08-07T15:14:09.721 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.721 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.722 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:09.722 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243777+0000", 2020-08-07T15:14:09.722 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7210000000000002e-06 2020-08-07T15:14:09.722 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.722 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.723 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:09.723 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243782+0000", 2020-08-07T15:14:09.723 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0069508749999999996 2020-08-07T15:14:09.723 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.723 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.724 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.724 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250732+0000", 2020-08-07T15:14:09.724 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7899999999999999e-06 2020-08-07T15:14:09.724 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.724 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.725 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:09.725 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250737+0000", 2020-08-07T15:14:09.725 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003272409999999 2020-08-07T15:14:09.725 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.725 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.726 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.726 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251064+0000", 2020-08-07T15:14:09.726 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6115000000000001e-05 2020-08-07T15:14:09.726 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.726 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.726 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:09.727 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251081+0000", 2020-08-07T15:14:09.727 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.007368357 2020-08-07T15:14:09.727 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.727 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:09.727 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.728 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.728 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.728 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37059 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:14:09.728 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:14:09.728 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 315.421256927, 2020-08-07T15:14:09.729 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0071655319999999, 2020-08-07T15:14:09.729 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:09.729 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:14:09.729 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:09.729 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:09.730 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:09.730 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37059 2020-08-07T15:14:09.731 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.731 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:09.731 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.731 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:09.731 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:14:09.732 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:09.732 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.732 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.732 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:09.732 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.243955+0000", 2020-08-07T15:14:09.733 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6533000000000002e-05 2020-08-07T15:14:09.733 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.733 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.733 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:09.733 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244012+0000", 2020-08-07T15:14:09.734 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3860000000000001e-06 2020-08-07T15:14:09.734 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.744 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.744 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:09.745 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244019+0000", 2020-08-07T15:14:09.745 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.6982000000000001e-05 2020-08-07T15:14:09.745 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.745 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.746 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:09.746 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244036+0000", 2020-08-07T15:14:09.746 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0310000000000001e-06 2020-08-07T15:14:09.746 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.746 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.747 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:09.747 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244043+0000", 2020-08-07T15:14:09.747 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0067169259999999998 2020-08-07T15:14:09.747 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.747 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.748 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.748 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250760+0000", 2020-08-07T15:14:09.748 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.4499999999999998e-06 2020-08-07T15:14:09.748 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.748 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.748 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:09.749 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250768+0000", 2020-08-07T15:14:09.749 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000341691 2020-08-07T15:14:09.749 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.749 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.749 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.750 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251109+0000", 2020-08-07T15:14:09.750 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1533e-05 2020-08-07T15:14:09.750 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.751 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.751 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:09.751 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251121+0000", 2020-08-07T15:14:09.751 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0071655319999999 2020-08-07T15:14:09.751 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.752 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:09.752 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.752 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.752 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.752 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37060 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:14:09.753 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:14:09.753 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 315.42113298999999, 2020-08-07T15:14:09.753 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070762280000001, 2020-08-07T15:14:09.753 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:09.753 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:14:09.753 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:09.754 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:09.759 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:09.759 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37060 2020-08-07T15:14:09.760 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.760 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:09.760 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.760 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:09.760 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:14:09.761 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:09.761 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.761 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.762 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:09.762 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244079+0000", 2020-08-07T15:14:09.762 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5739000000000003e-05 2020-08-07T15:14:09.762 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.762 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:09.763 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244135+0000", 2020-08-07T15:14:09.763 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7740000000000001e-06 2020-08-07T15:14:09.763 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.763 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.763 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:09.764 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244141+0000", 2020-08-07T15:14:09.764 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.3817e-05 2020-08-07T15:14:09.764 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.764 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.764 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:09.765 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244155+0000", 2020-08-07T15:14:09.765 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3829999999999997e-06 2020-08-07T15:14:09.765 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.765 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.765 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:09.766 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244161+0000", 2020-08-07T15:14:09.766 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0066303669999999999 2020-08-07T15:14:09.766 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.766 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.766 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.767 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250791+0000", 2020-08-07T15:14:09.767 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.392e-06 2020-08-07T15:14:09.767 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.767 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.767 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:09.767 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250798+0000", 2020-08-07T15:14:09.768 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003485860000001 2020-08-07T15:14:09.768 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.768 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.768 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.768 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251146+0000", 2020-08-07T15:14:09.769 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1700000000000003e-06 2020-08-07T15:14:09.769 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.770 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.770 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:09.770 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251155+0000", 2020-08-07T15:14:09.770 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070762280000001 2020-08-07T15:14:09.771 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.771 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:09.771 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.771 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.771 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.772 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37061 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:14:09.772 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:14:09.772 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 315.42101836299997, 2020-08-07T15:14:09.773 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070006920000001, 2020-08-07T15:14:09.773 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:09.773 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:14:09.773 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:09.774 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:09.774 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:09.774 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37061 2020-08-07T15:14:09.774 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.774 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:09.775 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.775 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:09.775 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:14:09.775 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:09.775 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.776 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.776 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:09.776 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244194+0000", 2020-08-07T15:14:09.776 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5453e-05 2020-08-07T15:14:09.777 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.777 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.777 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:09.777 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244239+0000", 2020-08-07T15:14:09.777 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3179999999999997e-06 2020-08-07T15:14:09.778 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.778 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.778 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:09.778 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244245+0000", 2020-08-07T15:14:09.778 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2138e-05 2020-08-07T15:14:09.779 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.779 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.779 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:09.779 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244257+0000", 2020-08-07T15:14:09.780 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.69e-06 2020-08-07T15:14:09.780 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.781 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.781 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:09.781 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244261+0000", 2020-08-07T15:14:09.781 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0065598940000000001 2020-08-07T15:14:09.781 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.782 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.782 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.782 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250821+0000", 2020-08-07T15:14:09.782 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4799999999999998e-06 2020-08-07T15:14:09.783 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.783 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.783 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:09.783 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250828+0000", 2020-08-07T15:14:09.783 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003567310000001 2020-08-07T15:14:09.784 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.784 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.784 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.784 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251185+0000", 2020-08-07T15:14:09.784 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.9879999999999999e-06 2020-08-07T15:14:09.785 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.785 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.785 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:09.785 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251195+0000", 2020-08-07T15:14:09.785 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0070006920000001 2020-08-07T15:14:09.786 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.786 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:09.786 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.786 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.787 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.787 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37062 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:14:09.787 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:14:09.787 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 315.42091814499997, 2020-08-07T15:14:09.787 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006933662, 2020-08-07T15:14:09.788 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:09.788 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:14:09.789 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:09.790 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:09.790 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:09.793 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37062 2020-08-07T15:14:09.794 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.794 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:09.794 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.794 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:09.794 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:14:09.795 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:09.795 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.795 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.795 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:09.795 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244294+0000", 2020-08-07T15:14:09.796 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4041e-05 2020-08-07T15:14:09.796 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.796 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.796 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:09.796 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244348+0000", 2020-08-07T15:14:09.797 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2079999999999997e-06 2020-08-07T15:14:09.797 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.797 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.797 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:09.797 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244354+0000", 2020-08-07T15:14:09.798 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1483e-05 2020-08-07T15:14:09.798 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.798 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.798 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:09.799 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244366+0000", 2020-08-07T15:14:09.799 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7989999999999999e-06 2020-08-07T15:14:09.799 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.799 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.800 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:09.800 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244372+0000", 2020-08-07T15:14:09.800 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0064783699999999998 2020-08-07T15:14:09.800 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.800 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.801 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.801 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250850+0000", 2020-08-07T15:14:09.801 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4729999999999997e-06 2020-08-07T15:14:09.801 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.801 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.802 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:09.802 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250856+0000", 2020-08-07T15:14:09.802 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003630240000001 2020-08-07T15:14:09.802 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.802 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.803 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.803 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251219+0000", 2020-08-07T15:14:09.803 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.2640000000000008e-06 2020-08-07T15:14:09.803 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.803 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.804 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:09.804 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251228+0000", 2020-08-07T15:14:09.804 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006933662 2020-08-07T15:14:09.804 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.804 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:09.805 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.805 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.805 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.805 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37063 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:14:09.805 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:14:09.808 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 315.42080581099998, 2020-08-07T15:14:09.808 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006858051, 2020-08-07T15:14:09.809 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:09.809 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:14:09.809 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:09.809 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:09.809 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:09.810 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37063 2020-08-07T15:14:09.810 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.810 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:09.810 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.810 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:09.811 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:14:09.811 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:09.811 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.811 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.811 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:09.811 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244406+0000", 2020-08-07T15:14:09.812 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5043000000000001e-05 2020-08-07T15:14:09.812 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.812 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.812 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:09.812 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244461+0000", 2020-08-07T15:14:09.813 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4600000000000002e-06 2020-08-07T15:14:09.813 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.813 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.813 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:09.813 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244467+0000", 2020-08-07T15:14:09.814 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1435e-05 2020-08-07T15:14:09.814 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.814 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.814 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:09.814 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244478+0000", 2020-08-07T15:14:09.815 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.0190000000000003e-06 2020-08-07T15:14:09.815 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.815 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.815 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:09.815 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244484+0000", 2020-08-07T15:14:09.816 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.006392895 2020-08-07T15:14:09.816 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.816 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.816 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.816 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250877+0000", 2020-08-07T15:14:09.817 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4280000000000001e-06 2020-08-07T15:14:09.817 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.817 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.817 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:09.817 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250884+0000", 2020-08-07T15:14:09.820 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000370414 2020-08-07T15:14:09.820 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.820 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.820 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.821 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251254+0000", 2020-08-07T15:14:09.821 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0356999999999999e-05 2020-08-07T15:14:09.821 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.821 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.821 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:09.822 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251264+0000", 2020-08-07T15:14:09.822 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006858051 2020-08-07T15:14:09.822 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.822 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:09.822 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.823 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.823 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.823 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37064 4.0 4.34789080 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:14:09.823 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:14:09.824 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 315.42069367800002, 2020-08-07T15:14:09.824 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0067798189999999, 2020-08-07T15:14:09.824 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:09.824 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:14:09.824 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:09.825 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:09.825 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:09.825 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37064 2020-08-07T15:14:09.825 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.825 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:09.826 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.826 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:09.826 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:14:09.826 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:09.826 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.827 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.827 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:09.827 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244518+0000", 2020-08-07T15:14:09.827 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4007e-05 2020-08-07T15:14:09.827 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.828 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:09.828 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244573+0000", 2020-08-07T15:14:09.828 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.831e-06 2020-08-07T15:14:09.828 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.828 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.829 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:09.829 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244577+0000", 2020-08-07T15:14:09.829 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2269000000000001e-05 2020-08-07T15:14:09.829 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.829 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.830 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:09.830 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244590+0000", 2020-08-07T15:14:09.830 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7149999999999997e-06 2020-08-07T15:14:09.830 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.830 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.831 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:09.831 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244594+0000", 2020-08-07T15:14:09.831 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0063105670000000004 2020-08-07T15:14:09.831 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.831 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.832 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.832 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250905+0000", 2020-08-07T15:14:09.832 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.1970000000000001e-06 2020-08-07T15:14:09.832 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.832 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.833 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:09.833 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250911+0000", 2020-08-07T15:14:09.833 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003783020000001 2020-08-07T15:14:09.833 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.833 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.834 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.834 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251289+0000", 2020-08-07T15:14:09.834 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.9309999999999997e-06 2020-08-07T15:14:09.834 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.834 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.835 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:09.835 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251298+0000", 2020-08-07T15:14:09.835 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0067798189999999 2020-08-07T15:14:09.835 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.835 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:09.836 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.836 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.836 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.836 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37065 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:14:09.836 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:14:09.837 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 315.420583459, 2020-08-07T15:14:09.837 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006708545, 2020-08-07T15:14:09.837 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:09.837 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:14:09.837 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:09.838 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:09.838 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:09.838 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37065 2020-08-07T15:14:09.838 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.838 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:09.839 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.839 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:09.839 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:14:09.839 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:09.839 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.840 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.840 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:09.840 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244629+0000", 2020-08-07T15:14:09.840 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2750000000000001e-05 2020-08-07T15:14:09.840 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.841 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.841 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:09.841 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244681+0000", 2020-08-07T15:14:09.841 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.0950000000000003e-06 2020-08-07T15:14:09.841 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.842 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.842 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:09.842 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244687+0000", 2020-08-07T15:14:09.842 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2485e-05 2020-08-07T15:14:09.842 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.843 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.843 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:09.843 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244699+0000", 2020-08-07T15:14:09.843 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.8800000000000002e-06 2020-08-07T15:14:09.843 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.844 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.844 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:09.844 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.244706+0000", 2020-08-07T15:14:09.845 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0062281569999999998 2020-08-07T15:14:09.845 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.845 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.845 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.845 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250934+0000", 2020-08-07T15:14:09.846 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1719999999999998e-06 2020-08-07T15:14:09.846 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.846 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.846 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:09.846 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250939+0000", 2020-08-07T15:14:09.847 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0003861810000001 2020-08-07T15:14:09.847 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.847 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.847 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.847 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251325+0000", 2020-08-07T15:14:09.848 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1824999999999999e-05 2020-08-07T15:14:09.848 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.848 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.848 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:09.848 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251337+0000", 2020-08-07T15:14:09.849 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.006708545 2020-08-07T15:14:09.849 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.849 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:09.849 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.849 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.850 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.850 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37066 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:14:09.850 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:14:09.850 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 315.41591448899999, 2020-08-07T15:14:09.850 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002077205, 2020-08-07T15:14:09.851 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:09.851 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:14:09.851 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:09.851 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:09.851 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:09.852 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37066 2020-08-07T15:14:09.852 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.852 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:09.852 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.852 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:09.853 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:14:09.853 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:09.853 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.853 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.853 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:09.854 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249298+0000", 2020-08-07T15:14:09.854 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4654000000000002e-05 2020-08-07T15:14:09.854 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.854 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.854 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:09.855 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249352+0000", 2020-08-07T15:14:09.855 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.6210000000000004e-06 2020-08-07T15:14:09.855 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.855 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.855 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:09.855 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249360+0000", 2020-08-07T15:14:09.856 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.1895999999999998e-05 2020-08-07T15:14:09.856 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.856 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.856 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:09.856 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249382+0000", 2020-08-07T15:14:09.857 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1479999999999995e-06 2020-08-07T15:14:09.858 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.858 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.858 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:09.858 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249391+0000", 2020-08-07T15:14:09.859 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0015731510000000001 2020-08-07T15:14:09.859 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.859 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.859 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.860 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250964+0000", 2020-08-07T15:14:09.860 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5400000000000001e-06 2020-08-07T15:14:09.861 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.861 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.861 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:09.861 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250971+0000", 2020-08-07T15:14:09.862 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.000392379 2020-08-07T15:14:09.862 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.862 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.862 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.862 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251363+0000", 2020-08-07T15:14:09.862 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1816e-05 2020-08-07T15:14:09.863 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.863 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.863 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:09.863 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251375+0000", 2020-08-07T15:14:09.863 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002077205 2020-08-07T15:14:09.864 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.864 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:09.864 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.864 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.864 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.865 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37067 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:14:09.865 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:14:09.865 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 315.41577379500001, 2020-08-07T15:14:09.865 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0019761119999999, 2020-08-07T15:14:09.865 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:09.866 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:14:09.866 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:09.866 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:09.866 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:09.866 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37067 2020-08-07T15:14:09.867 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.868 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:09.868 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.871 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:09.871 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:14:09.871 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:09.871 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.872 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.872 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:09.872 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249438+0000", 2020-08-07T15:14:09.872 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.5961999999999997e-05 2020-08-07T15:14:09.872 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.873 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.873 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:09.873 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249494+0000", 2020-08-07T15:14:09.873 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3210000000000004e-06 2020-08-07T15:14:09.874 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.874 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.874 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:09.874 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249500+0000", 2020-08-07T15:14:09.875 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1012000000000001e-05 2020-08-07T15:14:09.875 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.875 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.875 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:09.875 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249511+0000", 2020-08-07T15:14:09.876 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.809e-06 2020-08-07T15:14:09.876 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.876 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.876 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:09.876 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.249515+0000", 2020-08-07T15:14:09.877 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0014777670000000001 2020-08-07T15:14:09.877 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.877 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.877 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.877 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250993+0000", 2020-08-07T15:14:09.878 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.9499999999999998e-06 2020-08-07T15:14:09.878 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.878 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.878 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:09.878 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.250999+0000", 2020-08-07T15:14:09.879 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0004032519999999 2020-08-07T15:14:09.879 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.879 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.879 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.879 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251402+0000", 2020-08-07T15:14:09.880 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2038999999999999e-05 2020-08-07T15:14:09.880 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.880 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.880 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:09.880 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251414+0000", 2020-08-07T15:14:09.881 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0019761119999999 2020-08-07T15:14:09.881 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.881 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:09.881 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.881 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.882 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.882 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37068 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:14:09.882 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:14:09.882 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 315.413529224, 2020-08-07T15:14:09.882 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99977072, 2020-08-07T15:14:09.883 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:09.883 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:14:09.883 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:09.883 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:09.883 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:09.884 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37068 2020-08-07T15:14:09.884 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.884 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:09.884 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.884 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:09.885 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:14:09.885 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:09.885 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.885 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.885 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:09.886 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251683+0000", 2020-08-07T15:14:09.886 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.1369000000000003e-05 2020-08-07T15:14:09.886 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.886 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.886 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:09.887 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251724+0000", 2020-08-07T15:14:09.887 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.19e-06 2020-08-07T15:14:09.887 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.887 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.887 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:09.888 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251731+0000", 2020-08-07T15:14:09.888 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1368e-05 2020-08-07T15:14:09.888 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.888 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.888 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:09.889 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251742+0000", 2020-08-07T15:14:09.889 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.682e-06 2020-08-07T15:14:09.889 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.889 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.889 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:09.890 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251748+0000", 2020-08-07T15:14:09.890 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5275000000000003e-05 2020-08-07T15:14:09.890 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.890 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.890 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.891 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251793+0000", 2020-08-07T15:14:09.891 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.5000000000000005e-06 2020-08-07T15:14:09.891 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.891 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.891 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:09.892 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.251802+0000", 2020-08-07T15:14:09.892 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99963953500000002 2020-08-07T15:14:09.892 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.892 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.892 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.892 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251442+0000", 2020-08-07T15:14:09.893 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1800999999999999e-05 2020-08-07T15:14:09.893 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.893 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.893 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:09.894 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251454+0000", 2020-08-07T15:14:09.894 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99977072 2020-08-07T15:14:09.895 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.895 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:09.895 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.895 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.896 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.896 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37069 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:14:09.896 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:14:09.896 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 315.41242542600003, 2020-08-07T15:14:09.896 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99870607899999997, 2020-08-07T15:14:09.897 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:09.897 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:14:09.897 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:09.897 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:09.897 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:09.898 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37069 2020-08-07T15:14:09.898 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.898 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:09.898 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.898 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:09.899 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:14:09.899 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:09.899 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.899 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.899 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:09.900 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252787+0000", 2020-08-07T15:14:09.900 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.3755999999999999e-05 2020-08-07T15:14:09.900 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.900 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.900 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:09.900 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252831+0000", 2020-08-07T15:14:09.901 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7919999999999998e-06 2020-08-07T15:14:09.901 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.901 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.901 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:09.901 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252836+0000", 2020-08-07T15:14:09.902 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1409999999999999e-05 2020-08-07T15:14:09.902 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.902 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.902 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:09.902 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252848+0000", 2020-08-07T15:14:09.903 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.6380000000000001e-06 2020-08-07T15:14:09.903 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.903 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.903 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:09.903 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252853+0000", 2020-08-07T15:14:09.904 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3244e-05 2020-08-07T15:14:09.904 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.904 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.904 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.904 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252907+0000", 2020-08-07T15:14:09.905 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.1470000000000007e-06 2020-08-07T15:14:09.905 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.905 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.905 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:09.905 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252916+0000", 2020-08-07T15:14:09.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99856497499999997 2020-08-07T15:14:09.906 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.906 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.906 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.906 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251481+0000", 2020-08-07T15:14:09.906 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.2116999999999999e-05 2020-08-07T15:14:09.907 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.907 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.907 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:09.907 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251493+0000", 2020-08-07T15:14:09.907 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99870607899999997 2020-08-07T15:14:09.908 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.908 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:09.908 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.908 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.909 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.909 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:37070 4.0 4.fd380360 (undecoded) ondisk+write+known_if_redirected e107)", 2020-08-07T15:14:09.909 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:14:09.909 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 315.412321163, 2020-08-07T15:14:09.909 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99863946699999995, 2020-08-07T15:14:09.910 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:09.910 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "reached pg", 2020-08-07T15:14:09.910 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:09.910 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:09.910 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:09.911 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 37070 2020-08-07T15:14:09.911 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.911 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:09.911 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.911 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:09.912 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:14:09.912 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:09.912 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.913 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.913 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:09.914 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252891+0000", 2020-08-07T15:14:09.914 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.9984000000000001e-05 2020-08-07T15:14:09.914 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.914 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.914 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:09.915 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252941+0000", 2020-08-07T15:14:09.915 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7010000000000004e-06 2020-08-07T15:14:09.915 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.915 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.915 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:09.916 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252946+0000", 2020-08-07T15:14:09.916 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1144999999999999e-05 2020-08-07T15:14:09.916 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.916 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.916 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:09.917 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252957+0000", 2020-08-07T15:14:09.917 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6920000000000001e-06 2020-08-07T15:14:09.917 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.917 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.917 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:09.918 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252962+0000", 2020-08-07T15:14:09.918 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6739000000000002e-05 2020-08-07T15:14:09.918 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.918 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.918 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.919 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.252998+0000", 2020-08-07T15:14:09.919 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.503e-06 2020-08-07T15:14:09.919 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.919 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.919 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:09.920 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:54.253006+0000", 2020-08-07T15:14:09.920 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99851375499999995 2020-08-07T15:14:09.920 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.920 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.920 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.921 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251520+0000", 2020-08-07T15:14:09.921 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0947999999999999e-05 2020-08-07T15:14:09.921 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.921 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.921 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:09.922 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:08:55.251530+0000", 2020-08-07T15:14:09.922 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99863946699999995 2020-08-07T15:14:09.922 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.922 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:09.922 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.923 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.923 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.923 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42505 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:14:09.923 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:14:09.923 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 309.36298063300001, 2020-08-07T15:14:09.924 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0024736009999999, 2020-08-07T15:14:09.924 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:09.924 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:09.924 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:09.924 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:09.925 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:09.925 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42505 2020-08-07T15:14:09.925 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.925 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:09.925 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.926 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:09.926 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:14:09.926 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:09.926 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.926 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.927 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:09.927 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302232+0000", 2020-08-07T15:14:09.927 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00011052600000000001 2020-08-07T15:14:09.927 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.927 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.928 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:09.928 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302342+0000", 2020-08-07T15:14:09.928 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.781e-06 2020-08-07T15:14:09.928 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.928 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.929 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:09.929 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302352+0000", 2020-08-07T15:14:09.929 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.7646e-05 2020-08-07T15:14:09.929 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.929 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.930 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:09.930 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302369+0000", 2020-08-07T15:14:09.930 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.3629999999999994e-06 2020-08-07T15:14:09.930 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.930 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.931 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:09.931 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302378+0000", 2020-08-07T15:14:09.931 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8007000000000003e-05 2020-08-07T15:14:09.931 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.931 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.932 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.932 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302426+0000", 2020-08-07T15:14:09.932 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5159999999999994e-06 2020-08-07T15:14:09.932 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.932 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.933 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:09.933 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302434+0000", 2020-08-07T15:14:09.933 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99151562699999995 2020-08-07T15:14:09.933 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.933 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.934 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.934 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.293950+0000", 2020-08-07T15:14:09.934 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00020138099999999999 2020-08-07T15:14:09.934 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.934 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:09.935 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294151+0000", 2020-08-07T15:14:09.935 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00024761 2020-08-07T15:14:09.935 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.935 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.935 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:14:09.936 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294399+0000", 2020-08-07T15:14:09.936 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0027714369999999999 2020-08-07T15:14:09.936 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.936 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.936 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:14:09.937 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297170+0000", 2020-08-07T15:14:09.937 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.234e-06 2020-08-07T15:14:09.937 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.937 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.937 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:09.938 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297172+0000", 2020-08-07T15:14:09.938 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.5384999999999998e-05 2020-08-07T15:14:09.938 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.948 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.948 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:14:09.948 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.297237+0000", 2020-08-07T15:14:09.948 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0073576980000000002 2020-08-07T15:14:09.949 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.949 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.949 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:09.950 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304595+0000", 2020-08-07T15:14:09.950 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.1006000000000001e-05 2020-08-07T15:14:09.950 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.951 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.951 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:09.951 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304626+0000", 2020-08-07T15:14:09.951 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6000000000000003e-07 2020-08-07T15:14:09.951 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.952 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.952 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:14:09.952 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304626+0000", 2020-08-07T15:14:09.952 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.8724e-05 2020-08-07T15:14:09.952 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.953 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.953 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:09.953 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304705+0000", 2020-08-07T15:14:09.953 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0024736009999999 2020-08-07T15:14:09.953 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.954 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:09.954 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.954 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.954 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.954 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42506 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:14:09.955 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:14:09.955 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 309.36278942000001, 2020-08-07T15:14:09.955 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002465226, 2020-08-07T15:14:09.955 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:09.955 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:09.956 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:09.956 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:09.956 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:09.956 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42506 2020-08-07T15:14:09.956 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.957 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:09.957 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.957 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:09.957 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:14:09.957 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:09.958 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.958 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.958 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:09.959 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302423+0000", 2020-08-07T15:14:09.959 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.8780999999999999e-05 2020-08-07T15:14:09.959 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.959 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.959 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:09.960 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302472+0000", 2020-08-07T15:14:09.960 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3740000000000003e-06 2020-08-07T15:14:09.960 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.960 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.961 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:09.961 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302478+0000", 2020-08-07T15:14:09.961 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1155000000000001e-05 2020-08-07T15:14:09.962 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.962 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.962 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:09.962 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302489+0000", 2020-08-07T15:14:09.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.7559999999999996e-06 2020-08-07T15:14:09.963 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.963 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.963 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:09.963 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302495+0000", 2020-08-07T15:14:09.963 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.4128000000000003e-05 2020-08-07T15:14:09.964 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.964 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.964 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.964 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302529+0000", 2020-08-07T15:14:09.964 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.516e-06 2020-08-07T15:14:09.965 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.965 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.965 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:09.965 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302535+0000", 2020-08-07T15:14:09.966 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99240899299999996 2020-08-07T15:14:09.966 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.966 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.966 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.967 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.294944+0000", 2020-08-07T15:14:09.967 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000115806 2020-08-07T15:14:09.967 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.967 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.967 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:09.968 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295060+0000", 2020-08-07T15:14:09.968 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000205002 2020-08-07T15:14:09.968 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.968 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.968 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:14:09.969 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295265+0000", 2020-08-07T15:14:09.969 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0043414400000000002 2020-08-07T15:14:09.969 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.969 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.969 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:14:09.970 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299607+0000", 2020-08-07T15:14:09.970 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2229999999999998e-06 2020-08-07T15:14:09.970 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.970 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:09.971 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299612+0000", 2020-08-07T15:14:09.971 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5797000000000002e-05 2020-08-07T15:14:09.971 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.971 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.971 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:14:09.972 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299658+0000", 2020-08-07T15:14:09.972 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.005095044 2020-08-07T15:14:09.972 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.972 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.972 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:09.973 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304753+0000", 2020-08-07T15:14:09.973 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3121999999999996e-05 2020-08-07T15:14:09.973 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.973 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.973 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:09.974 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304816+0000", 2020-08-07T15:14:09.974 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 8.5700000000000001e-07 2020-08-07T15:14:09.974 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.974 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.974 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:14:09.975 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304817+0000", 2020-08-07T15:14:09.975 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1231999999999998e-05 2020-08-07T15:14:09.975 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.975 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.975 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:09.976 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304888+0000", 2020-08-07T15:14:09.976 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002465226 2020-08-07T15:14:09.976 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.976 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:09.977 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.977 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.977 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.977 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42507 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 16384~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:14:09.977 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:14:09.978 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 309.36267865000002, 2020-08-07T15:14:09.978 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002567097, 2020-08-07T15:14:09.979 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:09.979 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:09.979 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:09.979 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:09.979 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:09.980 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42507 2020-08-07T15:14:09.980 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.980 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:09.980 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.980 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:09.981 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:14:09.981 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:09.981 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.981 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.981 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:09.982 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302534+0000", 2020-08-07T15:14:09.982 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.7679000000000002e-05 2020-08-07T15:14:09.982 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.982 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.982 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:09.983 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302581+0000", 2020-08-07T15:14:09.983 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.2299999999999999e-06 2020-08-07T15:14:09.983 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.983 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.983 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:09.984 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302586+0000", 2020-08-07T15:14:09.984 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.0886999999999999e-05 2020-08-07T15:14:09.984 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.984 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.984 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:09.985 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302597+0000", 2020-08-07T15:14:09.985 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.4129999999999997e-06 2020-08-07T15:14:09.985 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.985 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.985 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:09.986 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302603+0000", 2020-08-07T15:14:09.986 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 2.3963000000000001e-05 2020-08-07T15:14:09.986 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.986 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.987 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.987 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302627+0000", 2020-08-07T15:14:09.987 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.33e-06 2020-08-07T15:14:09.987 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.987 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.988 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:09.988 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.302631+0000", 2020-08-07T15:14:09.988 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99319860900000001 2020-08-07T15:14:09.988 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.988 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.989 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:09.989 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295830+0000", 2020-08-07T15:14:09.989 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000116012 2020-08-07T15:14:09.990 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.990 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.990 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:09.990 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.295946+0000", 2020-08-07T15:14:09.990 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.000230242 2020-08-07T15:14:09.991 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.991 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.991 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:14:09.991 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296176+0000", 2020-08-07T15:14:09.992 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.003494455 2020-08-07T15:14:09.992 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.992 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.992 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:14:09.992 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299670+0000", 2020-08-07T15:14:09.993 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0299999999999998e-07 2020-08-07T15:14:09.993 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.993 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.993 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:09.994 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299671+0000", 2020-08-07T15:14:09.994 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.7298000000000001e-05 2020-08-07T15:14:09.994 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.994 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.994 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:14:09.995 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.299708+0000", 2020-08-07T15:14:09.995 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0052628839999999998 2020-08-07T15:14:09.995 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.995 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.995 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:09.996 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.304971+0000", 2020-08-07T15:14:09.996 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.3754e-05 2020-08-07T15:14:09.996 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.996 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.996 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:09.997 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305035+0000", 2020-08-07T15:14:09.997 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.9500000000000001e-07 2020-08-07T15:14:09.997 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.997 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.997 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:14:09.998 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305036+0000", 2020-08-07T15:14:09.998 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.4843000000000002e-05 2020-08-07T15:14:09.998 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:09.998 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:09.998 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:09.999 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305101+0000", 2020-08-07T15:14:09.999 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.002567097 2020-08-07T15:14:09.999 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:09.999 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:09.999 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:10.000 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.000 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.000 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_op(client.4726.0:42508 4.0 4:0eeb9d5a:::benchmark_data_smithi074_44586_object5313:head [set-alloc-hint object_size 65536 write_size 8192,write 24576~8192] snapc 0=[] ondisk+write+known_if_redirected e113)", 2020-08-07T15:14:10.000 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:14:10.000 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 309.35215363100002, 2020-08-07T15:14:10.001 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99221695600000004, 2020-08-07T15:14:10.001 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:10.001 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "commit sent; apply or cleanup", 2020-08-07T15:14:10.001 INFO:teuthology.orchestra.run.smithi074.stdout: "client_info": { 2020-08-07T15:14:10.001 INFO:teuthology.orchestra.run.smithi074.stdout: "client": "client.4726", 2020-08-07T15:14:10.002 INFO:teuthology.orchestra.run.smithi074.stdout: "client_addr": "172.21.15.74:0/1373919694", 2020-08-07T15:14:10.002 INFO:teuthology.orchestra.run.smithi074.stdout: "tid": 42508 2020-08-07T15:14:10.002 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.002 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:10.002 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.003 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:10.003 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:14:10.003 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:10.003 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.003 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.004 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:10.004 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313059+0000", 2020-08-07T15:14:10.004 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.3692e-05 2020-08-07T15:14:10.004 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.004 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.005 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:10.005 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313112+0000", 2020-08-07T15:14:10.005 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.7399999999999998e-06 2020-08-07T15:14:10.005 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.005 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.006 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:10.006 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313119+0000", 2020-08-07T15:14:10.006 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.5153999999999999e-05 2020-08-07T15:14:10.006 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.006 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.007 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:10.007 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313134+0000", 2020-08-07T15:14:10.007 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.3039999999999999e-06 2020-08-07T15:14:10.007 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.007 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.008 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:10.008 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313141+0000", 2020-08-07T15:14:10.008 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.6516000000000001e-05 2020-08-07T15:14:10.008 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.008 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.009 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:10.009 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313178+0000", 2020-08-07T15:14:10.009 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.0949999999999998e-06 2020-08-07T15:14:10.010 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.010 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.010 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for peered", 2020-08-07T15:14:10.011 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:00.313185+0000", 2020-08-07T15:14:10.011 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.98342834999999995 2020-08-07T15:14:10.011 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.011 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.012 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:10.012 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296613+0000", 2020-08-07T15:14:10.012 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.2230999999999993e-05 2020-08-07T15:14:10.012 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.012 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.013 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:10.013 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296686+0000", 2020-08-07T15:14:10.013 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.00013121300000000001 2020-08-07T15:14:10.013 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.013 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.014 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "waiting for subops from 0,7", 2020-08-07T15:14:10.014 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.296817+0000", 2020-08-07T15:14:10.014 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0044228050000000001 2020-08-07T15:14:10.014 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.014 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.015 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "op_commit", 2020-08-07T15:14:10.015 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301240+0000", 2020-08-07T15:14:10.015 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.0500000000000002e-07 2020-08-07T15:14:10.015 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.015 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.016 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_quorum_committed", 2020-08-07T15:14:10.016 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301241+0000", 2020-08-07T15:14:10.016 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.2434000000000006e-05 2020-08-07T15:14:10.016 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.016 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.017 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "commit_sent", 2020-08-07T15:14:10.017 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.301303+0000", 2020-08-07T15:14:10.017 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.0038622919999999998 2020-08-07T15:14:10.017 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.017 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.018 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:10.018 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305165+0000", 2020-08-07T15:14:10.018 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 3.8012000000000001e-05 2020-08-07T15:14:10.018 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.018 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.019 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "sub_op_commit_rec", 2020-08-07T15:14:10.019 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305203+0000", 2020-08-07T15:14:10.019 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.5299999999999999e-07 2020-08-07T15:14:10.019 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.019 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.020 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "repop_all_committed", 2020-08-07T15:14:10.020 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305204+0000", 2020-08-07T15:14:10.020 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 7.1760000000000004e-05 2020-08-07T15:14:10.020 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.020 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.021 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:10.021 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:09:01.305275+0000", 2020-08-07T15:14:10.021 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.99221695600000004 2020-08-07T15:14:10.021 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:10.021 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:10.022 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:10.022 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.022 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.022 INFO:teuthology.orchestra.run.smithi074.stdout: "description": "osd_repop_reply(client.5481.0:149904 6.2 e298/230 ondisk, result = 0)", 2020-08-07T15:14:10.022 INFO:teuthology.orchestra.run.smithi074.stdout: "initiated_at": "2020-08-07T15:13:51.327550+0000", 2020-08-07T15:14:10.023 INFO:teuthology.orchestra.run.smithi074.stdout: "age": 18.337662460000001, 2020-08-07T15:14:10.023 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.82728243300000004, 2020-08-07T15:14:10.023 INFO:teuthology.orchestra.run.smithi074.stdout: "type_data": { 2020-08-07T15:14:10.023 INFO:teuthology.orchestra.run.smithi074.stdout: "flag_point": "started", 2020-08-07T15:14:10.023 INFO:teuthology.orchestra.run.smithi074.stdout: "events": [ 2020-08-07T15:14:10.024 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.024 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "initiated", 2020-08-07T15:14:10.024 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:51.327550+0000", 2020-08-07T15:14:10.024 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0 2020-08-07T15:14:10.024 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.025 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.025 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "throttled", 2020-08-07T15:14:10.025 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:51.327550+0000", 2020-08-07T15:14:10.025 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.9096000000000001e-05 2020-08-07T15:14:10.025 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.026 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.026 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "header_read", 2020-08-07T15:14:10.026 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:51.327569+0000", 2020-08-07T15:14:10.026 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 4.6099999999999999e-06 2020-08-07T15:14:10.026 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.027 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "all_read", 2020-08-07T15:14:10.027 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:51.327573+0000", 2020-08-07T15:14:10.027 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.82711891500000001 2020-08-07T15:14:10.027 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.027 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.028 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "dispatched", 2020-08-07T15:14:10.028 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:52.154692+0000", 2020-08-07T15:14:10.028 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 9.91e-06 2020-08-07T15:14:10.028 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.028 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.029 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "queued_for_pg", 2020-08-07T15:14:10.029 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:52.154702+0000", 2020-08-07T15:14:10.029 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 5.1777000000000002e-05 2020-08-07T15:14:10.029 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.029 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.030 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "reached_pg", 2020-08-07T15:14:10.030 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:52.154754+0000", 2020-08-07T15:14:10.030 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 1.1233000000000001e-05 2020-08-07T15:14:10.030 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.031 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.031 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "started", 2020-08-07T15:14:10.031 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:52.154765+0000", 2020-08-07T15:14:10.031 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 6.6891999999999996e-05 2020-08-07T15:14:10.031 INFO:teuthology.orchestra.run.smithi074.stdout: }, 2020-08-07T15:14:10.032 INFO:teuthology.orchestra.run.smithi074.stdout: { 2020-08-07T15:14:10.032 INFO:teuthology.orchestra.run.smithi074.stdout: "event": "done", 2020-08-07T15:14:10.032 INFO:teuthology.orchestra.run.smithi074.stdout: "time": "2020-08-07T15:13:52.154832+0000", 2020-08-07T15:14:10.032 INFO:teuthology.orchestra.run.smithi074.stdout: "duration": 0.82728243300000004 2020-08-07T15:14:10.032 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:10.032 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:10.033 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:10.033 INFO:teuthology.orchestra.run.smithi074.stdout: } 2020-08-07T15:14:10.033 INFO:teuthology.orchestra.run.smithi074.stdout: ] 2020-08-07T15:14:10.033 INFO:teuthology.orchestra.run.smithi074.stdout:} 2020-08-07T15:14:10.034 INFO:teuthology.orchestra.run.smithi074:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2020-08-07T15:14:10.036 INFO:tasks.ceph.mon.a.smithi074.stderr:2020-08-07T15:14:09.911+0000 7f84a3ea5700 -1 mon.a@0(leader).osd e312 definitely_dead 0 2020-08-07T15:14:10.036 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:14:10.034+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:14:10.037 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2020-08-07T15:14:10.136 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:14:10.135+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:14:10.202 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:14:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:14:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2020-08-07T15:14:10.203 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:14:10.208 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2020-08-07T15:14:10.255 INFO:tasks.ceph.osd.5.smithi096.stderr:2020-08-07T15:14:10.254+0000 7fd43fda9700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31537) UID: 0 2020-08-07T15:14:10.291 INFO:teuthology.orchestra.run.smithi074.stdout:osd.0: {} 2020-08-07T15:14:10.332 INFO:teuthology.orchestra.run.smithi074.stdout:osd.1: {} 2020-08-07T15:14:10.339 INFO:tasks.ceph.osd.6.smithi096.stderr:2020-08-07T15:14:10.337+0000 7fa8b928b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 31541) UID: 0 2020-08-07T15:14:10.362 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2020-08-07T15:14:10.362 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2020-08-07T15:14:10.363 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2020-08-07T15:14:10.363 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2020-08-07T15:14:10.363 INFO:teuthology.orchestra.run.smithi096.stdout:} 2020-08-07T15:14:10.376 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2020-08-07T15:14:10.377 INFO:teuthology.orchestra.run.smithi074.stdout:osd.2: {} 2020-08-07T15:14:10.406 INFO:teuthology.orchestra.run.smithi074.stdout:osd.3: {} 2020-08-07T15:14:10.443 INFO:teuthology.orchestra.run.smithi074.stdout:osd.4: {} 2020-08-07T15:14:10.444 INFO:tasks.ceph.osd.1.smithi074.stderr:2020-08-07T15:14:10.442+0000 7f936153a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 62573) UID: 0 2020-08-07T15:14:10.479 INFO:teuthology.orchestra.run.smithi074.stdout:osd.5: {} 2020-08-07T15:14:10.521 INFO:teuthology.orchestra.run.smithi074.stdout:osd.6: {} 2020-08-07T15:14:10.543 INFO:tasks.ceph.osd.3.smithi074.stderr:2020-08-07T15:14:10.542+0000 7f3221b48700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31574) UID: 0 2020-08-07T15:14:10.564 INFO:teuthology.orchestra.run.smithi074.stdout:osd.7: {} 2020-08-07T15:14:10.565 INFO:teuthology.orchestra.run.smithi074.stderr:osd.0: osd_enable_op_tracker = 'false' 2020-08-07T15:14:10.565 INFO:teuthology.orchestra.run.smithi074.stderr:osd.1: osd_enable_op_tracker = 'false' 2020-08-07T15:14:10.565 INFO:teuthology.orchestra.run.smithi074.stderr:osd.2: osd_enable_op_tracker = 'false' 2020-08-07T15:14:10.565 INFO:teuthology.orchestra.run.smithi074.stderr:osd.3: osd_enable_op_tracker = 'false' 2020-08-07T15:14:10.566 INFO:teuthology.orchestra.run.smithi074.stderr:osd.4: osd_enable_op_tracker = 'false' 2020-08-07T15:14:10.566 INFO:teuthology.orchestra.run.smithi074.stderr:osd.5: osd_enable_op_tracker = 'false' 2020-08-07T15:14:10.566 INFO:teuthology.orchestra.run.smithi074.stderr:osd.6: osd_enable_op_tracker = 'false' 2020-08-07T15:14:10.566 INFO:teuthology.orchestra.run.smithi074.stderr:osd.7: osd_enable_op_tracker = 'false' 2020-08-07T15:14:10.567 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2020-08-07T15:14:10.568 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2020-08-07T15:14:10.571 INFO:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2020-08-07T15:14:10.616 DEBUG:teuthology.exit:Got signal 15; running 2 handlers... 2020-08-07T15:14:10.617 DEBUG:teuthology.task.console_log:Killing console logger for smithi074 2020-08-07T15:14:10.618 DEBUG:teuthology.task.console_log:Killing console logger for smithi096 2020-08-07T15:14:10.619 DEBUG:teuthology.task.console_log:Killing console logger for smithi074 2020-08-07T15:14:10.619 DEBUG:teuthology.task.console_log:Killing console logger for smithi096 2020-08-07T15:14:10.619 DEBUG:teuthology.exit:Finished running handlers